2024-05-20T18:15:31.791 INFO:root:teuthology version: 0.0.1.dev287+gb9e3da8 2024-05-20T18:15:31.791 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_pdonnell@teuthology --name pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi --verbose --archive /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715720 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} -- /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715720/orig.config.yaml 2024-05-20T18:15:31.840 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-20T18:15:31.934 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715720 branch: wip-pdonnell-testing-20240517.203521-squid-debug description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} email: pdonnell@redhat.com first_in_suite: false job_id: '7715720' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240517.203521-squid-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: d2b507c724ef6c8295ee0a35d185cdab167fd61d ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d2b507c724ef6c8295ee0a35d185cdab167fd61d kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240517.203521-squid-debug sha1: d2b507c724ef6c8295ee0a35d185cdab167fd61d owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 9749 sha1: d2b507c724ef6c8295ee0a35d185cdab167fd61d sleep_before_teardown: 0 subset: 272/512 suite: fs suite_branch: wip-pdonnell-testing-20240517.203521-squid-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d2b507c724ef6c8295ee0a35d185cdab167fd61d/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d2b507c724ef6c8295ee0a35d185cdab167fd61d syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi005.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEYJ/KpXyEA26PWVADXwMMimqimeT3Bq3hShJRCeTQQLVqa0N50+xKZu2wXtWzREszDQ/+HQLgmkXm5slBmr2CI= smithi063.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN6azvHe1I0QflL4RLsotOKQOQoplKgmtCiKtucEhHWOG7H/yHO0FSKgXqkCUwB5KOb6GmG9dcgK6+HuPDks4PY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a timestamp: 2024-05-20_15:44:59 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-05-20T18:15:31.934 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d2b507c724ef6c8295ee0a35d185cdab167fd61d/qa; will attempt to use it 2024-05-20T18:15:31.935 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d2b507c724ef6c8295ee0a35d185cdab167fd61d/qa/tasks 2024-05-20T18:15:31.936 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-20T18:15:31.937 INFO:teuthology.task.internal:Checking packages... 2024-05-20T18:15:31.959 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd2b507c724ef6c8295ee0a35d185cdab167fd61d' 2024-05-20T18:15:31.959 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-20T18:15:31.959 INFO:teuthology.packaging:ref: None 2024-05-20T18:15:31.959 INFO:teuthology.packaging:tag: None 2024-05-20T18:15:31.959 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240517.203521-squid-debug 2024-05-20T18:15:31.959 INFO:teuthology.packaging:sha1: d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-20T18:15:31.960 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240517.203521-squid-debug 2024-05-20T18:15:32.098 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2744.gd2b507c7 2024-05-20T18:15:32.100 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-20T18:15:32.108 INFO:teuthology.task.internal:no buildpackages task found 2024-05-20T18:15:32.108 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-20T18:15:32.163 INFO:teuthology.task.internal:Saving configuration 2024-05-20T18:15:32.182 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-20T18:15:32.191 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-20T18:15:32.216 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi005.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715720', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-20 18:10:23.103556', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEYJ/KpXyEA26PWVADXwMMimqimeT3Bq3hShJRCeTQQLVqa0N50+xKZu2wXtWzREszDQ/+HQLgmkXm5slBmr2CI='} 2024-05-20T18:15:32.238 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi063.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715720', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-20 18:10:23.102354', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN6azvHe1I0QflL4RLsotOKQOQoplKgmtCiKtucEhHWOG7H/yHO0FSKgXqkCUwB5KOb6GmG9dcgK6+HuPDks4PY='} 2024-05-20T18:15:32.238 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-20T18:15:32.246 INFO:teuthology.task.internal:roles: ubuntu@smithi005.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-05-20T18:15:32.246 INFO:teuthology.task.internal:roles: ubuntu@smithi063.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-05-20T18:15:32.246 INFO:teuthology.run_tasks:Running task console_log... 2024-05-20T18:15:32.315 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f93cb01fac0>, signals=[15]) 2024-05-20T18:15:32.315 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-20T18:15:32.322 INFO:teuthology.task.internal:Opening connections... 2024-05-20T18:15:32.322 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:15:32.324 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T18:15:32.398 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi063.front.sepia.ceph.com 2024-05-20T18:15:32.399 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T18:15:32.473 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-20T18:15:32.480 DEBUG:teuthology.orchestra.run.smithi005:> uname -m 2024-05-20T18:15:32.498 INFO:teuthology.orchestra.run.smithi005.stdout:x86_64 2024-05-20T18:15:32.499 DEBUG:teuthology.orchestra.run.smithi005:> cat /etc/os-release 2024-05-20T18:15:32.557 INFO:teuthology.orchestra.run.smithi005.stdout:NAME="CentOS Stream" 2024-05-20T18:15:32.557 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION="9" 2024-05-20T18:15:32.557 INFO:teuthology.orchestra.run.smithi005.stdout:ID="centos" 2024-05-20T18:15:32.557 INFO:teuthology.orchestra.run.smithi005.stdout:ID_LIKE="rhel fedora" 2024-05-20T18:15:32.557 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_ID="9" 2024-05-20T18:15:32.557 INFO:teuthology.orchestra.run.smithi005.stdout:PLATFORM_ID="platform:el9" 2024-05-20T18:15:32.557 INFO:teuthology.orchestra.run.smithi005.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-20T18:15:32.557 INFO:teuthology.orchestra.run.smithi005.stdout:ANSI_COLOR="0;31" 2024-05-20T18:15:32.557 INFO:teuthology.orchestra.run.smithi005.stdout:LOGO="fedora-logo-icon" 2024-05-20T18:15:32.557 INFO:teuthology.orchestra.run.smithi005.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-20T18:15:32.558 INFO:teuthology.orchestra.run.smithi005.stdout:HOME_URL="https://centos.org/" 2024-05-20T18:15:32.558 INFO:teuthology.orchestra.run.smithi005.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-20T18:15:32.558 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-20T18:15:32.558 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-20T18:15:32.558 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server 2024-05-20T18:15:32.578 DEBUG:teuthology.orchestra.run.smithi063:> uname -m 2024-05-20T18:15:32.596 INFO:teuthology.orchestra.run.smithi063.stdout:x86_64 2024-05-20T18:15:32.596 DEBUG:teuthology.orchestra.run.smithi063:> cat /etc/os-release 2024-05-20T18:15:32.655 INFO:teuthology.orchestra.run.smithi063.stdout:NAME="CentOS Stream" 2024-05-20T18:15:32.655 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION="9" 2024-05-20T18:15:32.655 INFO:teuthology.orchestra.run.smithi063.stdout:ID="centos" 2024-05-20T18:15:32.655 INFO:teuthology.orchestra.run.smithi063.stdout:ID_LIKE="rhel fedora" 2024-05-20T18:15:32.655 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION_ID="9" 2024-05-20T18:15:32.655 INFO:teuthology.orchestra.run.smithi063.stdout:PLATFORM_ID="platform:el9" 2024-05-20T18:15:32.655 INFO:teuthology.orchestra.run.smithi063.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-20T18:15:32.655 INFO:teuthology.orchestra.run.smithi063.stdout:ANSI_COLOR="0;31" 2024-05-20T18:15:32.655 INFO:teuthology.orchestra.run.smithi063.stdout:LOGO="fedora-logo-icon" 2024-05-20T18:15:32.656 INFO:teuthology.orchestra.run.smithi063.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-20T18:15:32.656 INFO:teuthology.orchestra.run.smithi063.stdout:HOME_URL="https://centos.org/" 2024-05-20T18:15:32.656 INFO:teuthology.orchestra.run.smithi063.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-20T18:15:32.656 INFO:teuthology.orchestra.run.smithi063.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-20T18:15:32.656 INFO:teuthology.orchestra.run.smithi063.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-20T18:15:32.656 INFO:teuthology.lock.ops:Updating smithi063.front.sepia.ceph.com on lock server 2024-05-20T18:15:32.678 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-20T18:15:32.688 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-20T18:15:32.726 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-20T18:15:32.726 DEBUG:teuthology.orchestra.run.smithi005:> test '!' -e /home/ubuntu/cephtest 2024-05-20T18:15:32.729 DEBUG:teuthology.orchestra.run.smithi063:> test '!' -e /home/ubuntu/cephtest 2024-05-20T18:15:32.747 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-20T18:15:32.754 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-20T18:15:32.754 DEBUG:teuthology.orchestra.run.smithi005:> test -z $(ls -A /var/lib/ceph) 2024-05-20T18:15:32.786 DEBUG:teuthology.orchestra.run.smithi063:> test -z $(ls -A /var/lib/ceph) 2024-05-20T18:15:32.815 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-20T18:15:32.866 INFO:teuthology.run_tasks:Running task kernel... 2024-05-20T18:15:32.882 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-20T18:15:32.882 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-05-20T18:15:32.882 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-05-20T18:15:32.883 DEBUG:teuthology.orchestra.run.smithi005:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-20T18:15:32.883 DEBUG:teuthology.orchestra.run.smithi063:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-20T18:15:32.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T18:15:32.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T18:15:32.901 DEBUG:teuthology.orchestra.run.smithi063:> uname -r 2024-05-20T18:15:32.902 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2024-05-20T18:15:32.959 INFO:teuthology.orchestra.run.smithi063.stdout:5.14.0-437.el9.x86_64 2024-05-20T18:15:32.960 INFO:teuthology.task.kernel:Running kernel on smithi063: 5.14.0-437.el9.x86_64 2024-05-20T18:15:32.960 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum install -y kernel 2024-05-20T18:15:32.961 INFO:teuthology.orchestra.run.smithi005.stdout:5.14.0-437.el9.x86_64 2024-05-20T18:15:32.961 INFO:teuthology.task.kernel:Running kernel on smithi005: 5.14.0-437.el9.x86_64 2024-05-20T18:15:32.961 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum install -y kernel 2024-05-20T18:15:35.169 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.1 MB 00:00 2024-05-20T18:15:36.895 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - AppStream 31 MB/s | 19 MB 00:00 2024-05-20T18:15:38.091 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - BaseOS 2.3 MB/s | 8.1 MB 00:03 2024-05-20T18:15:40.057 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-05-20T18:15:41.321 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - CRB 9.1 MB/s | 6.2 MB 00:00 2024-05-20T18:15:43.055 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-05-20T18:15:46.139 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 7.1 MB/s | 21 MB 00:02 2024-05-20T18:15:46.269 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.2 MB 00:02 2024-05-20T18:15:48.043 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - Extras packages 29 kB/s | 16 kB 00:00 2024-05-20T18:15:49.746 INFO:teuthology.orchestra.run.smithi063.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-05-20T18:15:51.497 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-05-20T18:15:53.487 INFO:teuthology.orchestra.run.smithi005.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-20T18:15:53.487 INFO:teuthology.orchestra.run.smithi005.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-20T18:15:53.590 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-20T18:15:53.596 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-20T18:15:53.597 INFO:teuthology.orchestra.run.smithi005.stdout: Package Architecture Version Repository Size 2024-05-20T18:15:53.597 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-20T18:15:53.597 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-05-20T18:15:53.597 INFO:teuthology.orchestra.run.smithi005.stdout: kernel x86_64 5.14.0-447.el9 baseos 576 k 2024-05-20T18:15:53.597 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2024-05-20T18:15:53.597 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-core x86_64 5.14.0-447.el9 baseos 16 M 2024-05-20T18:15:53.597 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules x86_64 5.14.0-447.el9 baseos 34 M 2024-05-20T18:15:53.597 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules-core x86_64 5.14.0-447.el9 baseos 28 M 2024-05-20T18:15:53.597 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:15:53.597 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-05-20T18:15:53.597 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-20T18:15:53.597 INFO:teuthology.orchestra.run.smithi005.stdout:Install 4 Packages 2024-05-20T18:15:53.597 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:15:53.598 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 78 M 2024-05-20T18:15:53.598 INFO:teuthology.orchestra.run.smithi005.stdout:Installed size: 123 M 2024-05-20T18:15:53.598 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-05-20T18:15:54.099 INFO:teuthology.orchestra.run.smithi005.stdout:(1/4): kernel-5.14.0-447.el9.x86_64.rpm 3.0 MB/s | 576 kB 00:00 2024-05-20T18:15:54.424 INFO:teuthology.orchestra.run.smithi005.stdout:(2/4): kernel-core-5.14.0-447.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-05-20T18:15:54.816 INFO:teuthology.orchestra.run.smithi005.stdout:(3/4): kernel-modules-5.14.0-447.el9.x86_64.rpm 38 MB/s | 34 MB 00:00 2024-05-20T18:15:55.083 INFO:teuthology.orchestra.run.smithi005.stdout:(4/4): kernel-modules-core-5.14.0-447.el9.x86_6 29 MB/s | 28 MB 00:00 2024-05-20T18:15:55.083 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-05-20T18:15:55.083 INFO:teuthology.orchestra.run.smithi005.stdout:Total 53 MB/s | 78 MB 00:01 2024-05-20T18:15:55.110 INFO:teuthology.orchestra.run.smithi063.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-05-20T18:15:55.793 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-05-20T18:15:56.178 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-05-20T18:15:56.179 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-05-20T18:15:57.058 INFO:teuthology.orchestra.run.smithi063.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-20T18:15:57.059 INFO:teuthology.orchestra.run.smithi063.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-20T18:15:57.130 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-05-20T18:15:57.131 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-05-20T18:15:57.153 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-20T18:15:57.168 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-20T18:15:57.169 INFO:teuthology.orchestra.run.smithi063.stdout: Package Architecture Version Repository Size 2024-05-20T18:15:57.169 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-20T18:15:57.169 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-05-20T18:15:57.169 INFO:teuthology.orchestra.run.smithi063.stdout: kernel x86_64 5.14.0-447.el9 baseos 576 k 2024-05-20T18:15:57.169 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-05-20T18:15:57.169 INFO:teuthology.orchestra.run.smithi063.stdout: kernel-core x86_64 5.14.0-447.el9 baseos 16 M 2024-05-20T18:15:57.169 INFO:teuthology.orchestra.run.smithi063.stdout: kernel-modules x86_64 5.14.0-447.el9 baseos 34 M 2024-05-20T18:15:57.169 INFO:teuthology.orchestra.run.smithi063.stdout: kernel-modules-core x86_64 5.14.0-447.el9 baseos 28 M 2024-05-20T18:15:57.169 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:15:57.169 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-05-20T18:15:57.169 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-20T18:15:57.169 INFO:teuthology.orchestra.run.smithi063.stdout:Install 4 Packages 2024-05-20T18:15:57.170 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:15:57.170 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 78 M 2024-05-20T18:15:57.170 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 123 M 2024-05-20T18:15:57.170 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-05-20T18:15:57.240 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] kernel-5.14.0-447.el9.x86_64.rpm: Status code: 404 for http://mirror.pilotfiber.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-447.el9.x86_64.rpm (IP: 216.158.133.254) 2024-05-20T18:15:57.242 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] kernel-core-5.14.0-447.el9.x86_64.rpm: Status code: 404 for http://mirror.pilotfiber.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-447.el9.x86_64.rpm (IP: 216.158.133.254) 2024-05-20T18:15:57.242 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] kernel-modules-5.14.0-447.el9.x86_64.rpm: Status code: 404 for http://mirror.pilotfiber.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-447.el9.x86_64.rpm (IP: 216.158.133.254) 2024-05-20T18:15:57.356 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] kernel-core-5.14.0-447.el9.x86_64.rpm: Status code: 404 for https://mirror.pilotfiber.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-447.el9.x86_64.rpm (IP: 216.158.133.254) 2024-05-20T18:15:57.358 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] kernel-5.14.0-447.el9.x86_64.rpm: Status code: 404 for https://mirror.pilotfiber.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-447.el9.x86_64.rpm (IP: 216.158.133.254) 2024-05-20T18:15:57.376 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] kernel-modules-5.14.0-447.el9.x86_64.rpm: Status code: 404 for https://mirror.pilotfiber.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-447.el9.x86_64.rpm (IP: 216.158.133.254) 2024-05-20T18:15:57.528 INFO:teuthology.orchestra.run.smithi063.stdout:(1/4): kernel-5.14.0-447.el9.x86_64.rpm 1.7 MB/s | 576 kB 00:00 2024-05-20T18:15:57.566 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] kernel-modules-core-5.14.0-447.el9.x86_64.rpm: Status code: 404 for https://mirror.pilotfiber.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-447.el9.x86_64.rpm (IP: 216.158.133.254) 2024-05-20T18:15:57.595 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] kernel-modules-core-5.14.0-447.el9.x86_64.rpm: Status code: 404 for http://mirror.pilotfiber.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-447.el9.x86_64.rpm (IP: 216.158.133.254) 2024-05-20T18:15:57.829 INFO:teuthology.orchestra.run.smithi063.stdout:(2/4): kernel-core-5.14.0-447.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-05-20T18:15:58.404 INFO:teuthology.orchestra.run.smithi063.stdout:(3/4): kernel-modules-5.14.0-447.el9.x86_64.rpm 28 MB/s | 34 MB 00:01 2024-05-20T18:15:58.662 INFO:teuthology.orchestra.run.smithi063.stdout:(4/4): kernel-modules-core-5.14.0-447.el9.x86_6 25 MB/s | 28 MB 00:01 2024-05-20T18:15:58.665 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-05-20T18:15:58.666 INFO:teuthology.orchestra.run.smithi063.stdout:Total 52 MB/s | 78 MB 00:01 2024-05-20T18:15:59.379 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-05-20T18:15:59.644 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-05-20T18:15:59.781 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-05-20T18:15:59.781 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-05-20T18:16:00.292 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-modules-core-5.14.0-447.el9.x86_64 1/4 2024-05-20T18:16:00.375 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-20T18:16:00.741 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-05-20T18:16:00.742 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-05-20T18:16:03.172 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-20T18:16:03.265 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-05-20T18:16:03.310 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-20T18:16:03.914 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : kernel-modules-core-5.14.0-447.el9.x86_64 1/4 2024-05-20T18:16:04.004 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-20T18:16:06.793 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-20T18:16:06.911 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-20T18:16:08.385 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-20T18:16:08.429 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-5.14.0-447.el9.x86_64 4/4 2024-05-20T18:16:11.983 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-20T18:16:12.023 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : kernel-5.14.0-447.el9.x86_64 4/4 2024-05-20T18:16:13.524 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-20T18:16:17.117 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-20T18:16:44.724 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 4/4 2024-05-20T18:16:44.764 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 4/4 2024-05-20T18:16:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-5.14.0-447.el9.x86_64 4/4 2024-05-20T18:16:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-5.14.0-447.el9.x86_64 1/4 2024-05-20T18:16:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-20T18:16:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-20T18:16:45.920 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-20T18:16:45.946 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:16:45.946 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-05-20T18:16:45.946 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-5.14.0-447.el9.x86_64 2024-05-20T18:16:45.946 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-core-5.14.0-447.el9.x86_64 2024-05-20T18:16:45.946 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules-5.14.0-447.el9.x86_64 2024-05-20T18:16:45.946 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules-core-5.14.0-447.el9.x86_64 2024-05-20T18:16:45.946 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:16:45.946 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-20T18:16:46.265 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q kernel | sort -rV | head -n 1 2024-05-20T18:16:46.347 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-20T18:16:46.347 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-20T18:16:46.347 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-447.el9.x86_64 2024-05-20T18:16:46.348 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-20T18:16:46.348 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-20T18:16:46.348 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-20T18:16:46.348 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi005.front.sepia.ceph.com, path=None, version=distro) 2024-05-20T18:16:46.348 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q kernel | sort -rV | head -n 1 2024-05-20T18:16:46.399 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-20T18:16:46.400 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-20T18:16:46.400 DEBUG:teuthology.orchestra.run.smithi005:> sudo rpm -qi grub2-tools 2024-05-20T18:16:46.431 INFO:teuthology.orchestra.run.smithi005.stdout:Name : grub2-tools 2024-05-20T18:16:46.431 INFO:teuthology.orchestra.run.smithi005.stdout:Epoch : 1 2024-05-20T18:16:46.431 INFO:teuthology.orchestra.run.smithi005.stdout:Version : 2.06 2024-05-20T18:16:46.431 INFO:teuthology.orchestra.run.smithi005.stdout:Release : 68.el9 2024-05-20T18:16:46.431 INFO:teuthology.orchestra.run.smithi005.stdout:Architecture: x86_64 2024-05-20T18:16:46.431 INFO:teuthology.orchestra.run.smithi005.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-20T18:16:46.431 INFO:teuthology.orchestra.run.smithi005.stdout:Group : Unspecified 2024-05-20T18:16:46.431 INFO:teuthology.orchestra.run.smithi005.stdout:Size : 8264254 2024-05-20T18:16:46.431 INFO:teuthology.orchestra.run.smithi005.stdout:License : GPLv3+ 2024-05-20T18:16:46.431 INFO:teuthology.orchestra.run.smithi005.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-20T18:16:46.431 INFO:teuthology.orchestra.run.smithi005.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-20T18:16:46.431 INFO:teuthology.orchestra.run.smithi005.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-20T18:16:46.432 INFO:teuthology.orchestra.run.smithi005.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-20T18:16:46.432 INFO:teuthology.orchestra.run.smithi005.stdout:Packager : builder@centos.org 2024-05-20T18:16:46.432 INFO:teuthology.orchestra.run.smithi005.stdout:Vendor : CentOS 2024-05-20T18:16:46.432 INFO:teuthology.orchestra.run.smithi005.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-20T18:16:46.432 INFO:teuthology.orchestra.run.smithi005.stdout:Summary : Support tools for GRUB. 2024-05-20T18:16:46.432 INFO:teuthology.orchestra.run.smithi005.stdout:Description : 2024-05-20T18:16:46.432 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:16:46.432 INFO:teuthology.orchestra.run.smithi005.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-20T18:16:46.432 INFO:teuthology.orchestra.run.smithi005.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-20T18:16:46.432 INFO:teuthology.orchestra.run.smithi005.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-20T18:16:46.432 INFO:teuthology.orchestra.run.smithi005.stdout:hardware devices. 2024-05-20T18:16:46.432 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:16:46.432 INFO:teuthology.orchestra.run.smithi005.stdout:This subpackage provides tools for support of all platforms. 2024-05-20T18:16:46.433 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-20T18:16:46.433 INFO:teuthology.task.kernel:Updating grub on smithi005 to boot 5.14.0-447.el9.x86_64 2024-05-20T18:16:46.433 DEBUG:teuthology.orchestra.run.smithi005:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-20T18:16:47.052 INFO:teuthology.orchestra.run.smithi005.stderr:Generating grub configuration file ... 2024-05-20T18:16:48.035 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 4/4 2024-05-20T18:16:48.075 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 4/4 2024-05-20T18:16:48.367 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: kernel-5.14.0-447.el9.x86_64 4/4 2024-05-20T18:16:48.368 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : kernel-5.14.0-447.el9.x86_64 1/4 2024-05-20T18:16:48.368 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-20T18:16:48.368 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-20T18:16:48.500 INFO:teuthology.orchestra.run.smithi005.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-20T18:16:48.540 INFO:teuthology.orchestra.run.smithi005.stderr:done 2024-05-20T18:16:48.542 DEBUG:teuthology.orchestra.run.smithi005:> mktemp 2024-05-20T18:16:48.557 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/tmp.jvJ5pzZt8w 2024-05-20T18:16:48.557 DEBUG:teuthology.orchestra.run.smithi005:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.jvJ5pzZt8w 2024-05-20T18:16:48.625 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0666 /tmp/tmp.jvJ5pzZt8w 2024-05-20T18:16:48.761 DEBUG:teuthology.orchestra.remote:smithi005:/tmp/tmp.jvJ5pzZt8w is 6KB 2024-05-20T18:16:48.812 DEBUG:teuthology.orchestra.run.smithi005:> rm -fr /tmp/tmp.jvJ5pzZt8w 2024-05-20T18:16:48.828 DEBUG:teuthology.orchestra.run.smithi005:> sudo /bin/ls /boot/loader/entries || true 2024-05-20T18:16:48.890 INFO:teuthology.orchestra.run.smithi005.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-20T18:16:48.890 INFO:teuthology.orchestra.run.smithi005.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-20T18:16:48.890 INFO:teuthology.orchestra.run.smithi005.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-20T18:16:48.890 INFO:teuthology.orchestra.run.smithi005.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64.conf 2024-05-20T18:16:48.891 DEBUG:teuthology.orchestra.run.smithi005:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64 2024-05-20T18:16:49.032 DEBUG:teuthology.orchestra.run.smithi005:> sudo shutdown -r now 2024-05-20T18:16:49.228 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-20T18:16:49.229 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:16:49.229 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-05-20T18:16:49.229 INFO:teuthology.orchestra.run.smithi063.stdout: kernel-5.14.0-447.el9.x86_64 2024-05-20T18:16:49.229 INFO:teuthology.orchestra.run.smithi063.stdout: kernel-core-5.14.0-447.el9.x86_64 2024-05-20T18:16:49.229 INFO:teuthology.orchestra.run.smithi063.stdout: kernel-modules-5.14.0-447.el9.x86_64 2024-05-20T18:16:49.229 INFO:teuthology.orchestra.run.smithi063.stdout: kernel-modules-core-5.14.0-447.el9.x86_64 2024-05-20T18:16:49.229 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:16:49.229 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-20T18:16:49.565 DEBUG:teuthology.orchestra.run.smithi063:> rpm -q kernel | sort -rV | head -n 1 2024-05-20T18:16:49.647 INFO:teuthology.orchestra.run.smithi063.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-20T18:16:49.648 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-20T18:16:49.648 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-447.el9.x86_64 2024-05-20T18:16:49.648 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-20T18:16:49.648 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-20T18:16:49.648 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-05-20T18:16:49.648 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi063.front.sepia.ceph.com, path=None, version=distro) 2024-05-20T18:16:49.648 DEBUG:teuthology.orchestra.run.smithi063:> rpm -q kernel | sort -rV | head -n 1 2024-05-20T18:16:49.700 INFO:teuthology.orchestra.run.smithi063.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-20T18:16:49.700 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-20T18:16:49.700 DEBUG:teuthology.orchestra.run.smithi063:> sudo rpm -qi grub2-tools 2024-05-20T18:16:49.743 INFO:teuthology.orchestra.run.smithi063.stdout:Name : grub2-tools 2024-05-20T18:16:49.743 INFO:teuthology.orchestra.run.smithi063.stdout:Epoch : 1 2024-05-20T18:16:49.743 INFO:teuthology.orchestra.run.smithi063.stdout:Version : 2.06 2024-05-20T18:16:49.743 INFO:teuthology.orchestra.run.smithi063.stdout:Release : 68.el9 2024-05-20T18:16:49.743 INFO:teuthology.orchestra.run.smithi063.stdout:Architecture: x86_64 2024-05-20T18:16:49.743 INFO:teuthology.orchestra.run.smithi063.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-20T18:16:49.744 INFO:teuthology.orchestra.run.smithi063.stdout:Group : Unspecified 2024-05-20T18:16:49.744 INFO:teuthology.orchestra.run.smithi063.stdout:Size : 8264254 2024-05-20T18:16:49.744 INFO:teuthology.orchestra.run.smithi063.stdout:License : GPLv3+ 2024-05-20T18:16:49.744 INFO:teuthology.orchestra.run.smithi063.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-20T18:16:49.744 INFO:teuthology.orchestra.run.smithi063.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-20T18:16:49.744 INFO:teuthology.orchestra.run.smithi063.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-20T18:16:49.744 INFO:teuthology.orchestra.run.smithi063.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-20T18:16:49.744 INFO:teuthology.orchestra.run.smithi063.stdout:Packager : builder@centos.org 2024-05-20T18:16:49.745 INFO:teuthology.orchestra.run.smithi063.stdout:Vendor : CentOS 2024-05-20T18:16:49.745 INFO:teuthology.orchestra.run.smithi063.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-20T18:16:49.745 INFO:teuthology.orchestra.run.smithi063.stdout:Summary : Support tools for GRUB. 2024-05-20T18:16:49.745 INFO:teuthology.orchestra.run.smithi063.stdout:Description : 2024-05-20T18:16:49.745 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:16:49.745 INFO:teuthology.orchestra.run.smithi063.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-20T18:16:49.745 INFO:teuthology.orchestra.run.smithi063.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-20T18:16:49.745 INFO:teuthology.orchestra.run.smithi063.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-20T18:16:49.746 INFO:teuthology.orchestra.run.smithi063.stdout:hardware devices. 2024-05-20T18:16:49.746 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:16:49.746 INFO:teuthology.orchestra.run.smithi063.stdout:This subpackage provides tools for support of all platforms. 2024-05-20T18:16:49.747 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-20T18:16:49.747 INFO:teuthology.task.kernel:Updating grub on smithi063 to boot 5.14.0-447.el9.x86_64 2024-05-20T18:16:49.747 DEBUG:teuthology.orchestra.run.smithi063:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-20T18:16:50.335 INFO:teuthology.orchestra.run.smithi063.stderr:Generating grub configuration file ... 2024-05-20T18:16:51.726 INFO:teuthology.orchestra.run.smithi063.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-20T18:16:51.758 INFO:teuthology.orchestra.run.smithi063.stderr:done 2024-05-20T18:16:51.761 DEBUG:teuthology.orchestra.run.smithi063:> mktemp 2024-05-20T18:16:51.780 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/tmp.nu9EzFNQix 2024-05-20T18:16:51.781 DEBUG:teuthology.orchestra.run.smithi063:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.nu9EzFNQix 2024-05-20T18:16:51.846 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 0666 /tmp/tmp.nu9EzFNQix 2024-05-20T18:16:51.983 DEBUG:teuthology.orchestra.remote:smithi063:/tmp/tmp.nu9EzFNQix is 6KB 2024-05-20T18:16:52.031 DEBUG:teuthology.orchestra.run.smithi063:> rm -fr /tmp/tmp.nu9EzFNQix 2024-05-20T18:16:52.046 DEBUG:teuthology.orchestra.run.smithi063:> sudo /bin/ls /boot/loader/entries || true 2024-05-20T18:16:52.109 INFO:teuthology.orchestra.run.smithi063.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-20T18:16:52.109 INFO:teuthology.orchestra.run.smithi063.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-20T18:16:52.109 INFO:teuthology.orchestra.run.smithi063.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-20T18:16:52.109 INFO:teuthology.orchestra.run.smithi063.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64.conf 2024-05-20T18:16:52.110 DEBUG:teuthology.orchestra.run.smithi063:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64 2024-05-20T18:16:52.258 DEBUG:teuthology.orchestra.run.smithi063:> sudo shutdown -r now 2024-05-20T18:17:19.058 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-20T18:17:19.059 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-05-20T18:17:19.060 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T18:17:22.262 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-05-20T18:17:22.262 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-20T18:17:22.263 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T18:17:37.567 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.5 2024-05-20T18:17:40.638 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.63 2024-05-20T18:17:46.570 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-05-20T18:17:46.572 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T18:17:49.642 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-20T18:17:49.643 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T18:17:52.706 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.63 2024-05-20T18:18:04.719 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-20T18:18:04.719 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T18:18:04.962 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.5 2024-05-20T18:18:16.974 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-05-20T18:18:16.974 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T18:18:18.131 DEBUG:teuthology.orchestra.run.smithi005:> true 2024-05-20T18:18:18.545 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-05-20T18:18:18.545 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-447.el9.x86_64"... 2024-05-20T18:18:18.545 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2024-05-20T18:18:18.562 INFO:teuthology.orchestra.run.smithi005.stdout:5.14.0-447.el9.x86_64 2024-05-20T18:18:18.563 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-447.el9.x86_64 vs 5.14.0-447.el9.x86_64 2024-05-20T18:18:18.563 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-20T18:18:18.563 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-20T18:18:19.565 DEBUG:teuthology.parallel:result is None 2024-05-20T18:19:04.722 DEBUG:teuthology.orchestra.remote:timed out 2024-05-20T18:19:19.723 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-20T18:19:19.725 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T18:19:20.061 DEBUG:teuthology.orchestra.run.smithi063:> true 2024-05-20T18:19:20.496 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-20T18:19:20.496 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-447.el9.x86_64"... 2024-05-20T18:19:20.497 DEBUG:teuthology.orchestra.run.smithi063:> uname -r 2024-05-20T18:19:20.513 INFO:teuthology.orchestra.run.smithi063.stdout:5.14.0-447.el9.x86_64 2024-05-20T18:19:20.514 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-447.el9.x86_64 vs 5.14.0-447.el9.x86_64 2024-05-20T18:19:20.514 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-20T18:19:20.514 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-20T18:19:21.515 DEBUG:teuthology.parallel:result is None 2024-05-20T18:19:21.516 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-20T18:19:21.524 INFO:teuthology.task.internal:Creating test directory... 2024-05-20T18:19:21.524 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-20T18:19:21.528 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-20T18:19:21.547 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-20T18:19:21.553 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-20T18:19:21.580 INFO:teuthology.task.internal:Creating archive directory... 2024-05-20T18:19:21.581 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-20T18:19:21.589 DEBUG:teuthology.orchestra.run.smithi063:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-20T18:19:21.624 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-20T18:19:21.630 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-20T18:19:21.630 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-20T18:19:21.656 DEBUG:teuthology.orchestra.run.smithi063:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-20T18:19:21.738 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-20T18:19:21.763 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-20T18:19:21.775 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-20T18:19:21.811 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-20T18:19:21.813 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-20T18:19:21.822 INFO:teuthology.task.internal:Configuring sudo... 2024-05-20T18:19:21.822 DEBUG:teuthology.orchestra.run.smithi005:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-20T18:19:21.825 DEBUG:teuthology.orchestra.run.smithi063:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-20T18:19:21.856 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-20T18:19:21.865 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-20T18:19:21.865 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-20T18:19:21.895 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-20T18:19:21.915 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-20T18:19:21.996 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-20T18:19:22.045 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-20T18:19:22.133 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-20T18:19:22.173 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:19:22.173 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-20T18:19:22.240 DEBUG:teuthology.orchestra.run.smithi063:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-20T18:19:22.286 DEBUG:teuthology.orchestra.run.smithi063:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-20T18:19:22.350 DEBUG:teuthology.orchestra.run.smithi063:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-20T18:19:22.402 DEBUG:teuthology.orchestra.run.smithi063:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-20T18:19:22.439 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:19:22.440 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-20T18:19:22.507 DEBUG:teuthology.orchestra.run.smithi005:> sudo service rsyslog restart 2024-05-20T18:19:22.509 DEBUG:teuthology.orchestra.run.smithi063:> sudo service rsyslog restart 2024-05-20T18:19:22.563 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-20T18:19:22.584 INFO:teuthology.orchestra.run.smithi063.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-20T18:19:22.869 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-20T18:19:22.880 INFO:teuthology.task.internal:Starting timer... 2024-05-20T18:19:22.880 INFO:teuthology.run_tasks:Running task pcp... 2024-05-20T18:19:22.902 INFO:teuthology.run_tasks:Running task selinux... 2024-05-20T18:19:22.933 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-20T18:19:22.934 DEBUG:teuthology.orchestra.run.smithi005:> sudo service auditd rotate 2024-05-20T18:19:23.000 INFO:teuthology.orchestra.run.smithi005.stdout:Rotating logs: 2024-05-20T18:19:23.002 DEBUG:teuthology.orchestra.run.smithi063:> sudo service auditd rotate 2024-05-20T18:19:23.065 INFO:teuthology.orchestra.run.smithi063.stdout:Rotating logs: 2024-05-20T18:19:23.068 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-20T18:19:23.068 DEBUG:teuthology.orchestra.run.smithi005:> /usr/sbin/getenforce 2024-05-20T18:19:23.093 INFO:teuthology.orchestra.run.smithi005.stdout:Permissive 2024-05-20T18:19:23.094 DEBUG:teuthology.orchestra.run.smithi063:> /usr/sbin/getenforce 2024-05-20T18:19:23.114 INFO:teuthology.orchestra.run.smithi063.stdout:Permissive 2024-05-20T18:19:23.114 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi005.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi063.front.sepia.ceph.com': 'permissive'} 2024-05-20T18:19:23.114 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-20T18:19:23.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T18:19:23.162 DEBUG:teuthology.orchestra.run.smithi063:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-20T18:19:23.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T18:19:23.190 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-20T18:19:23.190 DEBUG:teuthology.orchestra.run.smithi005:> sudo /usr/sbin/setenforce permissive 2024-05-20T18:19:23.230 DEBUG:teuthology.orchestra.run.smithi063:> sudo /usr/sbin/setenforce permissive 2024-05-20T18:19:23.259 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-20T18:19:23.269 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-20T18:19:23.283 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-20T18:19:23.284 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi005.front.sepia.ceph.com,smithi063.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-20T18:24:01.371 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi005.front.sepia.ceph.com'), Remote(name='ubuntu@smithi063.front.sepia.ceph.com')] 2024-05-20T18:24:01.373 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-05-20T18:24:01.374 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T18:24:01.451 DEBUG:teuthology.orchestra.run.smithi005:> true 2024-05-20T18:24:01.535 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-05-20T18:24:01.535 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-20T18:24:01.536 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-20T18:24:01.613 DEBUG:teuthology.orchestra.run.smithi063:> true 2024-05-20T18:24:01.699 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-20T18:24:01.699 INFO:teuthology.run_tasks:Running task clock... 2024-05-20T18:24:01.716 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-20T18:24:01.716 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-20T18:24:01.717 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-20T18:24:01.719 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-20T18:24:01.720 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-20T18:24:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-20T18:24:01.766 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-20T18:24:01.773 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-20T18:24:01.788 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-20T18:24:01.792 INFO:teuthology.orchestra.run.smithi005.stderr:sudo: ntpd: command not found 2024-05-20T18:24:01.804 INFO:teuthology.orchestra.run.smithi005.stdout:506 Cannot talk to daemon 2024-05-20T18:24:01.814 INFO:teuthology.orchestra.run.smithi063.stderr:sudo: ntpd: command not found 2024-05-20T18:24:01.819 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-20T18:24:01.826 INFO:teuthology.orchestra.run.smithi063.stdout:506 Cannot talk to daemon 2024-05-20T18:24:01.834 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-20T18:24:01.841 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-20T18:24:01.857 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-20T18:24:01.892 INFO:teuthology.orchestra.run.smithi005.stderr:bash: line 1: ntpq: command not found 2024-05-20T18:24:01.895 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-20T18:24:01.896 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2024-05-20T18:24:01.896 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T18:24:01.896 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T18:24:01.896 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T18:24:01.896 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T18:24:01.915 INFO:teuthology.orchestra.run.smithi063.stderr:bash: line 1: ntpq: command not found 2024-05-20T18:24:01.918 INFO:teuthology.orchestra.run.smithi063.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-20T18:24:01.918 INFO:teuthology.orchestra.run.smithi063.stdout:=============================================================================== 2024-05-20T18:24:01.918 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T18:24:01.918 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T18:24:01.918 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T18:24:01.919 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T18:24:01.919 INFO:teuthology.run_tasks:Running task install... 2024-05-20T18:24:01.930 DEBUG:teuthology.task.install:project ceph 2024-05-20T18:24:01.930 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d'}} 2024-05-20T18:24:01.930 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d'} 2024-05-20T18:24:01.931 INFO:teuthology.task.install:Using flavor: default 2024-05-20T18:24:01.939 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-20T18:24:01.939 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-20T18:24:01.939 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d', 'tag': None, 'wait_for_package': False} 2024-05-20T18:24:01.940 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-20T18:24:01.942 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d', 'tag': None, 'wait_for_package': False} 2024-05-20T18:24:01.943 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-20T18:24:02.095 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/ 2024-05-20T18:24:02.096 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2744.gd2b507c7 2024-05-20T18:24:02.165 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/ 2024-05-20T18:24:02.165 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2744.gd2b507c7 2024-05-20T18:24:02.500 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-20T18:24:02.500 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:24:02.500 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-20T18:24:02.503 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240517.203521-squid-debug/d2b507c724ef6c8295ee0a35d185cdab167fd61d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-20T18:24:02.504 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:24:02.504 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-20T18:24:02.532 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-20T18:24:02.532 DEBUG:teuthology.orchestra.run.smithi005:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2b507c724ef6c8295ee0a35d185cdab167fd61d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-20T18:24:02.533 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-20T18:24:02.534 DEBUG:teuthology.orchestra.run.smithi063:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2b507c724ef6c8295ee0a35d185cdab167fd61d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-20T18:24:02.601 DEBUG:teuthology.orchestra.run.smithi063:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-20T18:24:02.602 DEBUG:teuthology.orchestra.run.smithi005:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-20T18:24:02.680 DEBUG:teuthology.orchestra.run.smithi063:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-20T18:24:02.682 DEBUG:teuthology.orchestra.run.smithi005:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-20T18:24:02.747 INFO:teuthology.orchestra.run.smithi063.stdout:check_obsoletes = 1 2024-05-20T18:24:02.749 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum clean all 2024-05-20T18:24:02.752 INFO:teuthology.orchestra.run.smithi005.stdout:check_obsoletes = 1 2024-05-20T18:24:02.753 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum clean all 2024-05-20T18:24:02.991 INFO:teuthology.orchestra.run.smithi005.stdout:45 files removed 2024-05-20T18:24:03.002 INFO:teuthology.orchestra.run.smithi063.stdout:45 files removed 2024-05-20T18:24:03.018 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-20T18:24:03.029 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-20T18:24:04.103 INFO:teuthology.orchestra.run.smithi005.stdout:ceph packages for x86_64 105 kB/s | 83 kB 00:00 2024-05-20T18:24:04.267 INFO:teuthology.orchestra.run.smithi063.stdout:ceph packages for x86_64 88 kB/s | 83 kB 00:00 2024-05-20T18:24:04.876 INFO:teuthology.orchestra.run.smithi005.stdout:ceph noarch packages 16 kB/s | 11 kB 00:00 2024-05-20T18:24:05.135 INFO:teuthology.orchestra.run.smithi063.stdout:ceph noarch packages 14 kB/s | 11 kB 00:00 2024-05-20T18:24:05.727 INFO:teuthology.orchestra.run.smithi005.stdout:ceph source packages 2.3 kB/s | 1.9 kB 00:00 2024-05-20T18:24:06.019 INFO:teuthology.orchestra.run.smithi063.stdout:ceph source packages 2.3 kB/s | 1.9 kB 00:00 2024-05-20T18:24:08.065 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - BaseOS 3.5 MB/s | 8.1 MB 00:02 2024-05-20T18:24:08.181 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - BaseOS 3.8 MB/s | 8.1 MB 00:02 2024-05-20T18:24:11.590 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - AppStream 7.9 MB/s | 19 MB 00:02 2024-05-20T18:24:12.131 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - AppStream 6.7 MB/s | 19 MB 00:02 2024-05-20T18:24:17.412 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 6.2 MB 00:02 2024-05-20T18:24:18.270 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.2 MB 00:02 2024-05-20T18:24:19.748 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 16 kB 00:00 2024-05-20T18:24:20.090 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 16 kB 00:01 2024-05-20T18:24:20.975 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 28 MB/s | 21 MB 00:00 2024-05-20T18:24:24.486 INFO:teuthology.orchestra.run.smithi063.stdout:Extra Packages for Enterprise Linux 4.6 MB/s | 21 MB 00:04 2024-05-20T18:24:26.250 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-20T18:24:28.511 INFO:teuthology.orchestra.run.smithi005.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-20T18:24:28.511 INFO:teuthology.orchestra.run.smithi005.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-20T18:24:28.523 INFO:teuthology.orchestra.run.smithi005.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-20T18:24:28.523 INFO:teuthology.orchestra.run.smithi005.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-20T18:24:28.524 INFO:teuthology.orchestra.run.smithi005.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-20T18:24:28.525 INFO:teuthology.orchestra.run.smithi005.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-20T18:24:28.527 INFO:teuthology.orchestra.run.smithi005.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-20T18:24:28.529 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-20T18:24:28.530 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-20T18:24:28.596 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-20T18:24:28.618 INFO:teuthology.orchestra.run.smithi005.stdout:======================================================================================= 2024-05-20T18:24:28.618 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repository Size 2024-05-20T18:24:28.618 INFO:teuthology.orchestra.run.smithi005.stdout:======================================================================================= 2024-05-20T18:24:28.618 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-05-20T18:24:28.618 INFO:teuthology.orchestra.run.smithi005.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-20T18:24:28.618 INFO:teuthology.orchestra.run.smithi005.stdout: ceph x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 6.6 k 2024-05-20T18:24:28.619 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 5.4 M 2024-05-20T18:24:28.619 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 863 k 2024-05-20T18:24:28.619 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 147 k 2024-05-20T18:24:28.619 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 1.5 M 2024-05-20T18:24:28.619 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 142 k 2024-05-20T18:24:28.619 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 3.5 M 2024-05-20T18:24:28.619 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 7.4 M 2024-05-20T18:24:28.619 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 49 k 2024-05-20T18:24:28.619 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 9.8 M 2024-05-20T18:24:28.619 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 46 M 2024-05-20T18:24:28.619 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 264 k 2024-05-20T18:24:28.619 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 763 k 2024-05-20T18:24:28.619 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 221 k 2024-05-20T18:24:28.620 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-shell x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 27 k 2024-05-20T18:24:28.620 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 18 k 2024-05-20T18:24:28.620 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-20T18:24:28.620 INFO:teuthology.orchestra.run.smithi005.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-20T18:24:28.620 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-20T18:24:28.620 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 32 k 2024-05-20T18:24:28.620 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 756 k 2024-05-20T18:24:28.620 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 127 k 2024-05-20T18:24:28.620 INFO:teuthology.orchestra.run.smithi005.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-20T18:24:28.620 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-20T18:24:28.620 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-20T18:24:28.620 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-20T18:24:28.620 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-20T18:24:28.621 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-20T18:24:28.621 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 161 k 2024-05-20T18:24:28.621 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 320 k 2024-05-20T18:24:28.621 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 299 k 2024-05-20T18:24:28.621 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 99 k 2024-05-20T18:24:28.621 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 86 k 2024-05-20T18:24:28.621 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.0 M 2024-05-20T18:24:28.621 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 169 k 2024-05-20T18:24:28.621 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-20T18:24:28.621 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrading: 2024-05-20T18:24:28.621 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-20T18:24:28.621 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-20T18:24:28.622 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-20T18:24:28.622 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-20T18:24:28.622 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-20T18:24:28.622 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-20T18:24:28.622 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-20T18:24:28.622 INFO:teuthology.orchestra.run.smithi005.stdout: librados2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.3 M 2024-05-20T18:24:28.622 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.1 M 2024-05-20T18:24:28.622 INFO:teuthology.orchestra.run.smithi005.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-20T18:24:28.622 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-20T18:24:28.622 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2024-05-20T18:24:28.622 INFO:teuthology.orchestra.run.smithi005.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-20T18:24:28.622 INFO:teuthology.orchestra.run.smithi005.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-20T18:24:28.623 INFO:teuthology.orchestra.run.smithi005.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-20T18:24:28.623 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 19 M 2024-05-20T18:24:28.623 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 32 k 2024-05-20T18:24:28.623 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 2.3 M 2024-05-20T18:24:28.623 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 249 k 2024-05-20T18:24:28.623 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 4.9 M 2024-05-20T18:24:28.623 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 18 M 2024-05-20T18:24:28.623 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 16 k 2024-05-20T18:24:28.623 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 25 k 2024-05-20T18:24:28.623 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-20T18:24:28.623 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-20T18:24:28.623 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-20T18:24:28.624 INFO:teuthology.orchestra.run.smithi005.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-20T18:24:28.624 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-20T18:24:28.624 INFO:teuthology.orchestra.run.smithi005.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-20T18:24:28.624 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-20T18:24:28.624 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-20T18:24:28.624 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 158 k 2024-05-20T18:24:28.624 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-20T18:24:28.624 INFO:teuthology.orchestra.run.smithi005.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-20T18:24:28.624 INFO:teuthology.orchestra.run.smithi005.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-20T18:24:28.624 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-20T18:24:28.624 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-20T18:24:28.624 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 497 k 2024-05-20T18:24:28.625 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-20T18:24:28.625 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 5.0 M 2024-05-20T18:24:28.625 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-20T18:24:28.625 INFO:teuthology.orchestra.run.smithi005.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-20T18:24:28.625 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-20T18:24:28.625 INFO:teuthology.orchestra.run.smithi005.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-20T18:24:28.625 INFO:teuthology.orchestra.run.smithi005.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-20T18:24:28.625 INFO:teuthology.orchestra.run.smithi005.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-20T18:24:28.625 INFO:teuthology.orchestra.run.smithi005.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-20T18:24:28.625 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-20T18:24:28.625 INFO:teuthology.orchestra.run.smithi005.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-20T18:24:28.625 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-20T18:24:28.625 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-20T18:24:28.626 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-20T18:24:28.626 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-20T18:24:28.626 INFO:teuthology.orchestra.run.smithi005.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-20T18:24:28.626 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-20T18:24:28.626 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-20T18:24:28.626 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-20T18:24:28.626 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 45 k 2024-05-20T18:24:28.626 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 135 k 2024-05-20T18:24:28.626 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-20T18:24:28.626 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-20T18:24:28.626 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-20T18:24:28.626 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-20T18:24:28.626 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-20T18:24:28.626 INFO:teuthology.orchestra.run.smithi005.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-20T18:24:28.627 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-20T18:24:28.627 INFO:teuthology.orchestra.run.smithi005.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-20T18:24:28.627 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-20T18:24:28.627 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-20T18:24:28.627 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-20T18:24:28.627 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-20T18:24:28.627 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-20T18:24:28.627 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-20T18:24:28.627 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-20T18:24:28.627 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-20T18:24:28.627 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-20T18:24:28.627 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-20T18:24:28.627 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-20T18:24:28.628 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-20T18:24:28.628 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-20T18:24:28.628 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-20T18:24:28.628 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-20T18:24:28.628 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-20T18:24:28.628 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-20T18:24:28.628 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-20T18:24:28.628 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-20T18:24:28.628 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-20T18:24:28.628 INFO:teuthology.orchestra.run.smithi005.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-20T18:24:28.628 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-20T18:24:28.628 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-20T18:24:28.628 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-20T18:24:28.628 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-20T18:24:28.629 INFO:teuthology.orchestra.run.smithi005.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-20T18:24:28.629 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-20T18:24:28.629 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-20T18:24:28.629 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-20T18:24:28.629 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-20T18:24:28.629 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-20T18:24:28.629 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-20T18:24:28.629 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-20T18:24:28.629 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-20T18:24:28.629 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-20T18:24:28.629 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-20T18:24:28.629 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-20T18:24:28.629 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-20T18:24:28.629 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-20T18:24:28.630 INFO:teuthology.orchestra.run.smithi005.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-20T18:24:28.630 INFO:teuthology.orchestra.run.smithi005.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-20T18:24:28.630 INFO:teuthology.orchestra.run.smithi005.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-20T18:24:28.630 INFO:teuthology.orchestra.run.smithi005.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-20T18:24:28.630 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-20T18:24:28.630 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-20T18:24:28.630 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-20T18:24:28.630 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-20T18:24:28.630 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-20T18:24:28.630 INFO:teuthology.orchestra.run.smithi005.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-20T18:24:28.630 INFO:teuthology.orchestra.run.smithi005.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-20T18:24:28.630 INFO:teuthology.orchestra.run.smithi005.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-20T18:24:28.630 INFO:teuthology.orchestra.run.smithi005.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-20T18:24:28.631 INFO:teuthology.orchestra.run.smithi005.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-20T18:24:28.631 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-20T18:24:28.631 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-20T18:24:28.631 INFO:teuthology.orchestra.run.smithi005.stdout:Installing weak dependencies: 2024-05-20T18:24:28.631 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-20T18:24:28.631 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:24:28.631 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-05-20T18:24:28.631 INFO:teuthology.orchestra.run.smithi005.stdout:======================================================================================= 2024-05-20T18:24:28.631 INFO:teuthology.orchestra.run.smithi005.stdout:Install 148 Packages 2024-05-20T18:24:28.631 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrade 11 Packages 2024-05-20T18:24:28.631 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:24:28.632 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 222 M 2024-05-20T18:24:28.632 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-05-20T18:24:29.793 INFO:teuthology.orchestra.run.smithi005.stdout:(1/159): ceph-19.0.0-2744.gd2b507c7.el9.x86_64. 12 kB/s | 6.6 kB 00:00 2024-05-20T18:24:29.824 INFO:teuthology.orchestra.run.smithi063.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-05-20T18:24:30.451 INFO:teuthology.orchestra.run.smithi005.stdout:(2/159): ceph-fuse-19.0.0-2744.gd2b507c7.el9.x8 1.3 MB/s | 863 kB 00:00 2024-05-20T18:24:30.568 INFO:teuthology.orchestra.run.smithi005.stdout:(3/159): ceph-immutable-object-cache-19.0.0-274 1.2 MB/s | 147 kB 00:00 2024-05-20T18:24:31.368 INFO:teuthology.orchestra.run.smithi005.stdout:(4/159): ceph-mds-19.0.0-2744.gd2b507c7.el9.x86 2.8 MB/s | 2.3 MB 00:00 2024-05-20T18:24:31.938 INFO:teuthology.orchestra.run.smithi005.stdout:(5/159): ceph-mgr-19.0.0-2744.gd2b507c7.el9.x86 2.6 MB/s | 1.5 MB 00:00 2024-05-20T18:24:32.011 INFO:teuthology.orchestra.run.smithi005.stdout:(6/159): ceph-base-19.0.0-2744.gd2b507c7.el9.x8 2.0 MB/s | 5.4 MB 00:02 2024-05-20T18:24:32.155 INFO:teuthology.orchestra.run.smithi063.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-20T18:24:32.156 INFO:teuthology.orchestra.run.smithi063.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-20T18:24:32.167 INFO:teuthology.orchestra.run.smithi063.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-20T18:24:32.168 INFO:teuthology.orchestra.run.smithi063.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-20T18:24:32.169 INFO:teuthology.orchestra.run.smithi063.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-20T18:24:32.170 INFO:teuthology.orchestra.run.smithi063.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-20T18:24:32.171 INFO:teuthology.orchestra.run.smithi063.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-20T18:24:32.174 INFO:teuthology.orchestra.run.smithi063.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-20T18:24:32.175 INFO:teuthology.orchestra.run.smithi063.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-20T18:24:32.242 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-20T18:24:32.267 INFO:teuthology.orchestra.run.smithi063.stdout:======================================================================================= 2024-05-20T18:24:32.267 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-05-20T18:24:32.267 INFO:teuthology.orchestra.run.smithi063.stdout:======================================================================================= 2024-05-20T18:24:32.267 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-05-20T18:24:32.267 INFO:teuthology.orchestra.run.smithi063.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-20T18:24:32.267 INFO:teuthology.orchestra.run.smithi063.stdout: ceph x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 6.6 k 2024-05-20T18:24:32.267 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-base x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 5.4 M 2024-05-20T18:24:32.268 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-fuse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 863 k 2024-05-20T18:24:32.268 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 147 k 2024-05-20T18:24:32.268 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 1.5 M 2024-05-20T18:24:32.268 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 142 k 2024-05-20T18:24:32.268 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 3.5 M 2024-05-20T18:24:32.268 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 7.4 M 2024-05-20T18:24:32.268 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-rook noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 49 k 2024-05-20T18:24:32.268 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-radosgw x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 9.8 M 2024-05-20T18:24:32.268 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-test x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 46 M 2024-05-20T18:24:32.268 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-volume noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 264 k 2024-05-20T18:24:32.268 INFO:teuthology.orchestra.run.smithi063.stdout: cephadm noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 763 k 2024-05-20T18:24:32.268 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-mirror x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 221 k 2024-05-20T18:24:32.268 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-shell x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 27 k 2024-05-20T18:24:32.269 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-top noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 18 k 2024-05-20T18:24:32.269 INFO:teuthology.orchestra.run.smithi063.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-20T18:24:32.269 INFO:teuthology.orchestra.run.smithi063.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-20T18:24:32.269 INFO:teuthology.orchestra.run.smithi063.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-20T18:24:32.269 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs-devel x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 32 k 2024-05-20T18:24:32.269 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 756 k 2024-05-20T18:24:32.269 INFO:teuthology.orchestra.run.smithi063.stdout: librados-devel x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 127 k 2024-05-20T18:24:32.269 INFO:teuthology.orchestra.run.smithi063.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-20T18:24:32.269 INFO:teuthology.orchestra.run.smithi063.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-20T18:24:32.269 INFO:teuthology.orchestra.run.smithi063.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-20T18:24:32.269 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-20T18:24:32.269 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-20T18:24:32.270 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-20T18:24:32.270 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cephfs x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 161 k 2024-05-20T18:24:32.270 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rados x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 320 k 2024-05-20T18:24:32.270 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rbd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 299 k 2024-05-20T18:24:32.270 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rgw x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 99 k 2024-05-20T18:24:32.270 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-fuse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 86 k 2024-05-20T18:24:32.270 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-mirror x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.0 M 2024-05-20T18:24:32.270 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-nbd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 169 k 2024-05-20T18:24:32.270 INFO:teuthology.orchestra.run.smithi063.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-20T18:24:32.270 INFO:teuthology.orchestra.run.smithi063.stdout:Upgrading: 2024-05-20T18:24:32.270 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-20T18:24:32.270 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-20T18:24:32.270 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-20T18:24:32.271 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-20T18:24:32.271 INFO:teuthology.orchestra.run.smithi063.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-20T18:24:32.271 INFO:teuthology.orchestra.run.smithi063.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-20T18:24:32.271 INFO:teuthology.orchestra.run.smithi063.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-20T18:24:32.271 INFO:teuthology.orchestra.run.smithi063.stdout: librados2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.3 M 2024-05-20T18:24:32.271 INFO:teuthology.orchestra.run.smithi063.stdout: librbd1 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 3.1 M 2024-05-20T18:24:32.271 INFO:teuthology.orchestra.run.smithi063.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-20T18:24:32.271 INFO:teuthology.orchestra.run.smithi063.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-20T18:24:32.271 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-05-20T18:24:32.271 INFO:teuthology.orchestra.run.smithi063.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-20T18:24:32.271 INFO:teuthology.orchestra.run.smithi063.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-20T18:24:32.272 INFO:teuthology.orchestra.run.smithi063.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-20T18:24:32.272 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-common x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 19 M 2024-05-20T18:24:32.272 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 32 k 2024-05-20T18:24:32.272 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mds x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 2.3 M 2024-05-20T18:24:32.272 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 249 k 2024-05-20T18:24:32.272 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mon x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 4.9 M 2024-05-20T18:24:32.272 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-osd x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 18 M 2024-05-20T18:24:32.272 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2744.gd2b507c7.el9 ceph-noarch 16 k 2024-05-20T18:24:32.272 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-selinux x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 25 k 2024-05-20T18:24:32.272 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-20T18:24:32.272 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-20T18:24:32.273 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-20T18:24:32.273 INFO:teuthology.orchestra.run.smithi063.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-20T18:24:32.273 INFO:teuthology.orchestra.run.smithi063.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-20T18:24:32.273 INFO:teuthology.orchestra.run.smithi063.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-20T18:24:32.273 INFO:teuthology.orchestra.run.smithi063.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-20T18:24:32.273 INFO:teuthology.orchestra.run.smithi063.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-20T18:24:32.273 INFO:teuthology.orchestra.run.smithi063.stdout: libcephsqlite x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 158 k 2024-05-20T18:24:32.273 INFO:teuthology.orchestra.run.smithi063.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-20T18:24:32.273 INFO:teuthology.orchestra.run.smithi063.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-20T18:24:32.273 INFO:teuthology.orchestra.run.smithi063.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-20T18:24:32.273 INFO:teuthology.orchestra.run.smithi063.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-20T18:24:32.273 INFO:teuthology.orchestra.run.smithi063.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-20T18:24:32.273 INFO:teuthology.orchestra.run.smithi063.stdout: libradosstriper1 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 497 k 2024-05-20T18:24:32.274 INFO:teuthology.orchestra.run.smithi063.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-20T18:24:32.274 INFO:teuthology.orchestra.run.smithi063.stdout: librgw2 x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 5.0 M 2024-05-20T18:24:32.274 INFO:teuthology.orchestra.run.smithi063.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-20T18:24:32.274 INFO:teuthology.orchestra.run.smithi063.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-20T18:24:32.274 INFO:teuthology.orchestra.run.smithi063.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-20T18:24:32.274 INFO:teuthology.orchestra.run.smithi063.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-20T18:24:32.274 INFO:teuthology.orchestra.run.smithi063.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-20T18:24:32.274 INFO:teuthology.orchestra.run.smithi063.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-20T18:24:32.274 INFO:teuthology.orchestra.run.smithi063.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-20T18:24:32.274 INFO:teuthology.orchestra.run.smithi063.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-20T18:24:32.274 INFO:teuthology.orchestra.run.smithi063.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-20T18:24:32.274 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-20T18:24:32.274 INFO:teuthology.orchestra.run.smithi063.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-20T18:24:32.275 INFO:teuthology.orchestra.run.smithi063.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-20T18:24:32.275 INFO:teuthology.orchestra.run.smithi063.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-20T18:24:32.275 INFO:teuthology.orchestra.run.smithi063.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-20T18:24:32.275 INFO:teuthology.orchestra.run.smithi063.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-20T18:24:32.275 INFO:teuthology.orchestra.run.smithi063.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-20T18:24:32.275 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-20T18:24:32.275 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-argparse x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 45 k 2024-05-20T18:24:32.275 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-common x86_64 2:19.0.0-2744.gd2b507c7.el9 ceph 135 k 2024-05-20T18:24:32.275 INFO:teuthology.orchestra.run.smithi063.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-20T18:24:32.275 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-20T18:24:32.275 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-20T18:24:32.275 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-20T18:24:32.275 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-20T18:24:32.276 INFO:teuthology.orchestra.run.smithi063.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-20T18:24:32.276 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-20T18:24:32.276 INFO:teuthology.orchestra.run.smithi063.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-20T18:24:32.276 INFO:teuthology.orchestra.run.smithi063.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-20T18:24:32.276 INFO:teuthology.orchestra.run.smithi063.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-20T18:24:32.276 INFO:teuthology.orchestra.run.smithi063.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-20T18:24:32.276 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-20T18:24:32.276 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-20T18:24:32.276 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-20T18:24:32.276 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-20T18:24:32.276 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-20T18:24:32.276 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-20T18:24:32.276 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-20T18:24:32.277 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-20T18:24:32.277 INFO:teuthology.orchestra.run.smithi063.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-20T18:24:32.277 INFO:teuthology.orchestra.run.smithi063.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-20T18:24:32.277 INFO:teuthology.orchestra.run.smithi063.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-20T18:24:32.277 INFO:teuthology.orchestra.run.smithi063.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-20T18:24:32.277 INFO:teuthology.orchestra.run.smithi063.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-20T18:24:32.277 INFO:teuthology.orchestra.run.smithi063.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-20T18:24:32.277 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-20T18:24:32.277 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-20T18:24:32.277 INFO:teuthology.orchestra.run.smithi063.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-20T18:24:32.277 INFO:teuthology.orchestra.run.smithi063.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-20T18:24:32.277 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-20T18:24:32.277 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-20T18:24:32.277 INFO:teuthology.orchestra.run.smithi063.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-20T18:24:32.278 INFO:teuthology.orchestra.run.smithi063.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-20T18:24:32.278 INFO:teuthology.orchestra.run.smithi063.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-20T18:24:32.278 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-20T18:24:32.278 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-20T18:24:32.278 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-20T18:24:32.278 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-20T18:24:32.278 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-20T18:24:32.278 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-20T18:24:32.278 INFO:teuthology.orchestra.run.smithi063.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-20T18:24:32.278 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-20T18:24:32.278 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-20T18:24:32.278 INFO:teuthology.orchestra.run.smithi063.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-20T18:24:32.278 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-20T18:24:32.278 INFO:teuthology.orchestra.run.smithi063.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-20T18:24:32.279 INFO:teuthology.orchestra.run.smithi063.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-20T18:24:32.279 INFO:teuthology.orchestra.run.smithi063.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-20T18:24:32.279 INFO:teuthology.orchestra.run.smithi063.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-20T18:24:32.279 INFO:teuthology.orchestra.run.smithi063.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-20T18:24:32.279 INFO:teuthology.orchestra.run.smithi063.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-20T18:24:32.279 INFO:teuthology.orchestra.run.smithi063.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-20T18:24:32.279 INFO:teuthology.orchestra.run.smithi063.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-20T18:24:32.279 INFO:teuthology.orchestra.run.smithi063.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-20T18:24:32.279 INFO:teuthology.orchestra.run.smithi063.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-20T18:24:32.279 INFO:teuthology.orchestra.run.smithi063.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-20T18:24:32.279 INFO:teuthology.orchestra.run.smithi063.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-20T18:24:32.279 INFO:teuthology.orchestra.run.smithi063.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-20T18:24:32.279 INFO:teuthology.orchestra.run.smithi063.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-20T18:24:32.280 INFO:teuthology.orchestra.run.smithi063.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-20T18:24:32.280 INFO:teuthology.orchestra.run.smithi063.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-20T18:24:32.280 INFO:teuthology.orchestra.run.smithi063.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-20T18:24:32.280 INFO:teuthology.orchestra.run.smithi063.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-20T18:24:32.280 INFO:teuthology.orchestra.run.smithi063.stdout:Installing weak dependencies: 2024-05-20T18:24:32.280 INFO:teuthology.orchestra.run.smithi063.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-20T18:24:32.280 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:24:32.280 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-05-20T18:24:32.280 INFO:teuthology.orchestra.run.smithi063.stdout:======================================================================================= 2024-05-20T18:24:32.280 INFO:teuthology.orchestra.run.smithi063.stdout:Install 148 Packages 2024-05-20T18:24:32.280 INFO:teuthology.orchestra.run.smithi063.stdout:Upgrade 11 Packages 2024-05-20T18:24:32.280 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:24:32.281 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 222 M 2024-05-20T18:24:32.281 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-05-20T18:24:33.201 INFO:teuthology.orchestra.run.smithi063.stdout:(1/159): ceph-19.0.0-2744.gd2b507c7.el9.x86_64. 22 kB/s | 6.6 kB 00:00 2024-05-20T18:24:33.859 INFO:teuthology.orchestra.run.smithi063.stdout:(2/159): ceph-fuse-19.0.0-2744.gd2b507c7.el9.x8 1.3 MB/s | 863 kB 00:00 2024-05-20T18:24:33.886 INFO:teuthology.orchestra.run.smithi005.stdout:(7/159): ceph-common-19.0.0-2744.gd2b507c7.el9. 4.2 MB/s | 19 MB 00:04 2024-05-20T18:24:34.010 INFO:teuthology.orchestra.run.smithi063.stdout:(3/159): ceph-immutable-object-cache-19.0.0-274 981 kB/s | 147 kB 00:00 2024-05-20T18:24:34.686 INFO:teuthology.orchestra.run.smithi005.stdout:(8/159): ceph-mon-19.0.0-2744.gd2b507c7.el9.x86 1.8 MB/s | 4.9 MB 00:02 2024-05-20T18:24:34.795 INFO:teuthology.orchestra.run.smithi005.stdout:(9/159): ceph-selinux-19.0.0-2744.gd2b507c7.el9 233 kB/s | 25 kB 00:00 2024-05-20T18:24:35.143 INFO:teuthology.orchestra.run.smithi063.stdout:(4/159): ceph-mds-19.0.0-2744.gd2b507c7.el9.x86 2.0 MB/s | 2.3 MB 00:01 2024-05-20T18:24:35.628 INFO:teuthology.orchestra.run.smithi005.stdout:(10/159): ceph-radosgw-19.0.0-2744.gd2b507c7.el 5.7 MB/s | 9.8 MB 00:01 2024-05-20T18:24:35.735 INFO:teuthology.orchestra.run.smithi063.stdout:(5/159): ceph-mgr-19.0.0-2744.gd2b507c7.el9.x86 2.5 MB/s | 1.5 MB 00:00 2024-05-20T18:24:35.753 INFO:teuthology.orchestra.run.smithi005.stdout:(11/159): cephfs-mirror-19.0.0-2744.gd2b507c7.e 1.7 MB/s | 221 kB 00:00 2024-05-20T18:24:35.862 INFO:teuthology.orchestra.run.smithi005.stdout:(12/159): cephfs-shell-19.0.0-2744.gd2b507c7.el 249 kB/s | 27 kB 00:00 2024-05-20T18:24:35.979 INFO:teuthology.orchestra.run.smithi005.stdout:(13/159): libcephfs-devel-19.0.0-2744.gd2b507c7 271 kB/s | 32 kB 00:00 2024-05-20T18:24:36.213 INFO:teuthology.orchestra.run.smithi005.stdout:(14/159): libcephfs2-19.0.0-2744.gd2b507c7.el9. 3.2 MB/s | 756 kB 00:00 2024-05-20T18:24:36.304 INFO:teuthology.orchestra.run.smithi005.stdout:(15/159): libcephsqlite-19.0.0-2744.gd2b507c7.e 1.7 MB/s | 158 kB 00:00 2024-05-20T18:24:36.480 INFO:teuthology.orchestra.run.smithi005.stdout:(16/159): ceph-osd-19.0.0-2744.gd2b507c7.el9.x8 4.0 MB/s | 18 MB 00:04 2024-05-20T18:24:36.507 INFO:teuthology.orchestra.run.smithi005.stdout:(17/159): librados-devel-19.0.0-2744.gd2b507c7. 633 kB/s | 127 kB 00:00 2024-05-20T18:24:36.730 INFO:teuthology.orchestra.run.smithi005.stdout:(18/159): libradosstriper1-19.0.0-2744.gd2b507c 2.0 MB/s | 497 kB 00:00 2024-05-20T18:24:36.814 INFO:teuthology.orchestra.run.smithi005.stdout:(19/159): python3-ceph-argparse-19.0.0-2744.gd2 493 kB/s | 45 kB 00:00 2024-05-20T18:24:36.936 INFO:teuthology.orchestra.run.smithi005.stdout:(20/159): python3-ceph-common-19.0.0-2744.gd2b5 1.2 MB/s | 135 kB 00:00 2024-05-20T18:24:37.023 INFO:teuthology.orchestra.run.smithi005.stdout:(21/159): python3-cephfs-19.0.0-2744.gd2b507c7. 1.6 MB/s | 161 kB 00:00 2024-05-20T18:24:37.160 INFO:teuthology.orchestra.run.smithi063.stdout:(6/159): ceph-mon-19.0.0-2744.gd2b507c7.el9.x86 3.4 MB/s | 4.9 MB 00:01 2024-05-20T18:24:37.190 INFO:teuthology.orchestra.run.smithi005.stdout:(22/159): python3-rados-19.0.0-2744.gd2b507c7.e 1.9 MB/s | 320 kB 00:00 2024-05-20T18:24:37.390 INFO:teuthology.orchestra.run.smithi005.stdout:(23/159): python3-rbd-19.0.0-2744.gd2b507c7.el9 1.5 MB/s | 299 kB 00:00 2024-05-20T18:24:37.524 INFO:teuthology.orchestra.run.smithi005.stdout:(24/159): librgw2-19.0.0-2744.gd2b507c7.el9.x86 4.9 MB/s | 5.0 MB 00:01 2024-05-20T18:24:37.559 INFO:teuthology.orchestra.run.smithi005.stdout:(25/159): python3-rgw-19.0.0-2744.gd2b507c7.el9 627 kB/s | 99 kB 00:00 2024-05-20T18:24:37.610 INFO:teuthology.orchestra.run.smithi005.stdout:(26/159): rbd-fuse-19.0.0-2744.gd2b507c7.el9.x8 1.0 MB/s | 86 kB 00:00 2024-05-20T18:24:37.708 INFO:teuthology.orchestra.run.smithi005.stdout:(27/159): rbd-nbd-19.0.0-2744.gd2b507c7.el9.x86 1.7 MB/s | 169 kB 00:00 2024-05-20T18:24:37.736 INFO:teuthology.orchestra.run.smithi063.stdout:(7/159): ceph-base-19.0.0-2744.gd2b507c7.el9.x8 1.1 MB/s | 5.4 MB 00:04 2024-05-20T18:24:37.801 INFO:teuthology.orchestra.run.smithi005.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2744.g 346 kB/s | 32 kB 00:00 2024-05-20T18:24:37.901 INFO:teuthology.orchestra.run.smithi005.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2744.gd2b507c 1.4 MB/s | 142 kB 00:00 2024-05-20T18:24:38.867 INFO:teuthology.orchestra.run.smithi005.stdout:(30/159): ceph-mgr-dashboard-19.0.0-2744.gd2b50 3.6 MB/s | 3.5 MB 00:00 2024-05-20T18:24:39.092 INFO:teuthology.orchestra.run.smithi005.stdout:(31/159): rbd-mirror-19.0.0-2744.gd2b507c7.el9. 2.0 MB/s | 3.0 MB 00:01 2024-05-20T18:24:39.293 INFO:teuthology.orchestra.run.smithi005.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2744.gd2 1.2 MB/s | 249 kB 00:00 2024-05-20T18:24:39.418 INFO:teuthology.orchestra.run.smithi005.stdout:(33/159): ceph-mgr-rook-19.0.0-2744.gd2b507c7.e 394 kB/s | 49 kB 00:00 2024-05-20T18:24:39.544 INFO:teuthology.orchestra.run.smithi005.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2744.gd 132 kB/s | 16 kB 00:00 2024-05-20T18:24:39.769 INFO:teuthology.orchestra.run.smithi005.stdout:(35/159): ceph-volume-19.0.0-2744.gd2b507c7.el9 1.1 MB/s | 264 kB 00:00 2024-05-20T18:24:40.252 INFO:teuthology.orchestra.run.smithi005.stdout:(36/159): cephadm-19.0.0-2744.gd2b507c7.el9.noa 1.5 MB/s | 763 kB 00:00 2024-05-20T18:24:40.369 INFO:teuthology.orchestra.run.smithi005.stdout:(37/159): cephfs-top-19.0.0-2744.gd2b507c7.el9. 156 kB/s | 18 kB 00:00 2024-05-20T18:24:40.478 INFO:teuthology.orchestra.run.smithi005.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 743 kB/s | 80 kB 00:00 2024-05-20T18:24:40.578 INFO:teuthology.orchestra.run.smithi005.stdout:(39/159): libgfortran-11.4.1-3.el9.x86_64.rpm 7.8 MB/s | 803 kB 00:00 2024-05-20T18:24:40.620 INFO:teuthology.orchestra.run.smithi005.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-05-20T18:24:40.662 INFO:teuthology.orchestra.run.smithi005.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.9 MB/s | 253 kB 00:00 2024-05-20T18:24:40.729 INFO:teuthology.orchestra.run.smithi005.stdout:(42/159): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-05-20T18:24:40.771 INFO:teuthology.orchestra.run.smithi005.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-05-20T18:24:40.813 INFO:teuthology.orchestra.run.smithi005.stdout:(44/159): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-05-20T18:24:40.855 INFO:teuthology.orchestra.run.smithi005.stdout:(45/159): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-05-20T18:24:40.897 INFO:teuthology.orchestra.run.smithi005.stdout:(46/159): python3-urllib3-1.26.5-5.el9.noarch.r 5.0 MB/s | 215 kB 00:00 2024-05-20T18:24:41.056 INFO:teuthology.orchestra.run.smithi005.stdout:(47/159): ceph-mgr-diskprediction-local-19.0.0- 3.4 MB/s | 7.4 MB 00:02 2024-05-20T18:24:41.089 INFO:teuthology.orchestra.run.smithi005.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 689 kB 00:00 2024-05-20T18:24:41.156 INFO:teuthology.orchestra.run.smithi005.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-05-20T18:24:41.198 INFO:teuthology.orchestra.run.smithi005.stdout:(50/159): bison-3.7.4-5.el9.x86_64.rpm 6.5 MB/s | 943 kB 00:00 2024-05-20T18:24:41.198 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm (IP: 72.4.120.219) 2024-05-20T18:24:41.257 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Status code: 404 for https://mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm (IP: 72.4.120.219) 2024-05-20T18:24:41.298 INFO:teuthology.orchestra.run.smithi005.stdout:(51/159): flex-2.6.4-9.el9.x86_64.rpm 3.1 MB/s | 313 kB 00:00 2024-05-20T18:24:41.349 INFO:teuthology.orchestra.run.smithi005.stdout:(52/159): flexiblas-3.0.4-8.el9.x86_64.rpm 656 kB/s | 33 kB 00:00 2024-05-20T18:24:41.440 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T18:24:41.507 INFO:teuthology.orchestra.run.smithi005.stdout:(53/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-05-20T18:24:41.558 INFO:teuthology.orchestra.run.smithi005.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-05-20T18:24:41.624 INFO:teuthology.orchestra.run.smithi005.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-05-20T18:24:41.700 INFO:teuthology.orchestra.run.smithi005.stdout:(56/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-05-20T18:24:41.758 INFO:teuthology.orchestra.run.smithi005.stdout:(57/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 797 kB/s | 45 kB 00:00 2024-05-20T18:24:41.786 INFO:teuthology.orchestra.run.smithi063.stdout:(8/159): ceph-osd-19.0.0-2744.gd2b507c7.el9.x86 3.8 MB/s | 18 MB 00:04 2024-05-20T18:24:41.850 INFO:teuthology.orchestra.run.smithi005.stdout:(58/159): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 662 kB 00:00 2024-05-20T18:24:41.877 INFO:teuthology.orchestra.run.smithi063.stdout:(9/159): ceph-selinux-19.0.0-2744.gd2b507c7.el9 275 kB/s | 25 kB 00:00 2024-05-20T18:24:41.887 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T18:24:41.903 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T18:24:41.942 INFO:teuthology.orchestra.run.smithi005.stdout:(59/159): libtool-2.4.6-45.el9.x86_64.rpm 6.2 MB/s | 582 kB 00:00 2024-05-20T18:24:41.992 INFO:teuthology.orchestra.run.smithi005.stdout:(60/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 511 kB/s | 25 kB 00:00 2024-05-20T18:24:42.018 INFO:teuthology.orchestra.run.smithi005.stdout:(61/159): elfutils-libelf-devel-0.191-4.el9.x86 25 kB/s | 22 kB 00:00 2024-05-20T18:24:42.068 INFO:teuthology.orchestra.run.smithi005.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 628 kB/s | 47 kB 00:00 2024-05-20T18:24:42.118 INFO:teuthology.orchestra.run.smithi005.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.9 MB/s | 292 kB 00:00 2024-05-20T18:24:42.160 INFO:teuthology.orchestra.run.smithi005.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-05-20T18:24:42.186 INFO:teuthology.orchestra.run.smithi005.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 526 kB/s | 35 kB 00:00 2024-05-20T18:24:42.386 INFO:teuthology.orchestra.run.smithi005.stdout:(66/159): openblas-openmp-0.3.21-2.el9.x86_64.r 21 MB/s | 4.7 MB 00:00 2024-05-20T18:24:42.512 INFO:teuthology.orchestra.run.smithi005.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-05-20T18:24:42.612 INFO:teuthology.orchestra.run.smithi005.stdout:(68/159): postgresql-contrib-13.14-1.el9.x86_64 8.9 MB/s | 911 kB 00:00 2024-05-20T18:24:42.679 INFO:teuthology.orchestra.run.smithi005.stdout:(69/159): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-05-20T18:24:42.921 INFO:teuthology.orchestra.run.smithi005.stdout:(70/159): postgresql-server-13.14-1.el9.x86_64. 24 MB/s | 5.8 MB 00:00 2024-05-20T18:24:43.021 INFO:teuthology.orchestra.run.smithi005.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-05-20T18:24:43.088 INFO:teuthology.orchestra.run.smithi005.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-05-20T18:24:43.322 INFO:teuthology.orchestra.run.smithi005.stdout:(73/159): python3-babel-2.9.1-2.el9.noarch.rpm 26 MB/s | 6.0 MB 00:00 2024-05-20T18:24:43.339 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] python3-colorama-0.4.6-3.el9.noarch.rpm: Status code: 404 for http://mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-colorama-0.4.6-3.el9.noarch.rpm (IP: 72.4.120.219) 2024-05-20T18:24:43.394 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] python3-colorama-0.4.6-3.el9.noarch.rpm: Status code: 404 for https://mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-colorama-0.4.6-3.el9.noarch.rpm (IP: 72.4.120.219) 2024-05-20T18:24:43.484 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] python3-colorama-0.4.6-3.el9.noarch.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-colorama-0.4.6-3.el9.noarch.rpm (IP: 140.211.166.134) 2024-05-20T18:24:43.565 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] python3-colorama-0.4.6-3.el9.noarch.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-colorama-0.4.6-3.el9.noarch.rpm (IP: 140.211.166.134) 2024-05-20T18:24:43.582 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] python3-colorama-0.4.6-3.el9.noarch.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-colorama-0.4.6-3.el9.noarch.rpm (IP: 199.223.246.113) 2024-05-20T18:24:43.680 INFO:teuthology.orchestra.run.smithi005.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 171 kB/s | 61 kB 00:00 2024-05-20T18:24:43.797 INFO:teuthology.orchestra.run.smithi005.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.0 MB/s | 244 kB 00:00 2024-05-20T18:24:43.884 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] python3-jinja2-2.11.3-6.el9.noarch.rpm: Status code: 404 for https://mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-jinja2-2.11.3-6.el9.noarch.rpm (IP: 72.4.120.219) 2024-05-20T18:24:43.901 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] python3-jinja2-2.11.3-6.el9.noarch.rpm: Status code: 404 for http://mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-jinja2-2.11.3-6.el9.noarch.rpm (IP: 72.4.120.219) 2024-05-20T18:24:44.008 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] python3-jinja2-2.11.3-6.el9.noarch.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-jinja2-2.11.3-6.el9.noarch.rpm (IP: 140.211.166.134) 2024-05-20T18:24:44.096 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] python3-jinja2-2.11.3-6.el9.noarch.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-jinja2-2.11.3-6.el9.noarch.rpm (IP: 140.211.166.134) 2024-05-20T18:24:44.112 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] python3-jinja2-2.11.3-6.el9.noarch.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-jinja2-2.11.3-6.el9.noarch.rpm (IP: 199.223.246.113) 2024-05-20T18:24:44.214 INFO:teuthology.orchestra.run.smithi005.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 596 kB/s | 248 kB 00:00 2024-05-20T18:24:44.264 INFO:teuthology.orchestra.run.smithi005.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-05-20T18:24:44.315 INFO:teuthology.orchestra.run.smithi005.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 384 kB/s | 19 kB 00:00 2024-05-20T18:24:44.390 INFO:teuthology.orchestra.run.smithi005.stdout:(79/159): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-05-20T18:24:44.474 INFO:teuthology.orchestra.run.smithi005.stdout:(80/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 1.9 MB/s | 4.4 MB 00:02 2024-05-20T18:24:44.499 INFO:teuthology.orchestra.run.smithi005.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 320 kB/s | 35 kB 00:00 2024-05-20T18:24:44.574 INFO:teuthology.orchestra.run.smithi005.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-05-20T18:24:44.649 INFO:teuthology.orchestra.run.smithi005.stdout:(83/159): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-05-20T18:24:44.750 INFO:teuthology.orchestra.run.smithi005.stdout:(84/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 18 MB/s | 5.1 MB 00:00 2024-05-20T18:24:44.775 INFO:teuthology.orchestra.run.smithi005.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 616 kB/s | 77 kB 00:00 2024-05-20T18:24:44.825 INFO:teuthology.orchestra.run.smithi005.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 563 kB/s | 42 kB 00:00 2024-05-20T18:24:44.875 INFO:teuthology.orchestra.run.smithi005.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-05-20T18:24:44.918 INFO:teuthology.orchestra.run.smithi005.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-05-20T18:24:44.976 INFO:teuthology.orchestra.run.smithi005.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-05-20T18:24:45.002 INFO:teuthology.orchestra.run.smithi005.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 609 kB/s | 51 kB 00:00 2024-05-20T18:24:45.052 INFO:teuthology.orchestra.run.smithi005.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 712 kB/s | 54 kB 00:00 2024-05-20T18:24:45.110 INFO:teuthology.orchestra.run.smithi005.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 716 kB/s | 42 kB 00:00 2024-05-20T18:24:45.169 INFO:teuthology.orchestra.run.smithi005.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 758 kB/s | 44 kB 00:00 2024-05-20T18:24:45.244 INFO:teuthology.orchestra.run.smithi005.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-05-20T18:24:45.286 INFO:teuthology.orchestra.run.smithi005.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 861 kB/s | 36 kB 00:00 2024-05-20T18:24:45.345 INFO:teuthology.orchestra.run.smithi005.stdout:(96/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 815 kB/s | 47 kB 00:00 2024-05-20T18:24:45.428 INFO:teuthology.orchestra.run.smithi005.stdout:(97/159): socat-1.7.4.1-5.el9.x86_64.rpm 3.6 MB/s | 305 kB 00:00 2024-05-20T18:24:45.596 INFO:teuthology.orchestra.run.smithi005.stdout:(98/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-05-20T18:24:45.637 INFO:teuthology.orchestra.run.smithi005.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 277 kB/s | 58 kB 00:00 2024-05-20T18:24:45.662 INFO:teuthology.orchestra.run.smithi005.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.4 MB/s | 94 kB 00:00 2024-05-20T18:24:45.705 INFO:teuthology.orchestra.run.smithi005.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 950 kB/s | 64 kB 00:00 2024-05-20T18:24:45.721 INFO:teuthology.orchestra.run.smithi005.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 764 kB/s | 45 kB 00:00 2024-05-20T18:24:45.755 INFO:teuthology.orchestra.run.smithi005.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 444 kB/s | 22 kB 00:00 2024-05-20T18:24:45.889 INFO:teuthology.orchestra.run.smithi005.stdout:(104/159): protobuf-compiler-3.14.0-13.el9.x86_ 5.0 MB/s | 863 kB 00:00 2024-05-20T18:24:46.189 INFO:teuthology.orchestra.run.smithi005.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-05-20T18:24:46.248 INFO:teuthology.orchestra.run.smithi005.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 332 kB/s | 19 kB 00:00 2024-05-20T18:24:46.281 INFO:teuthology.orchestra.run.smithi005.stdout:(107/159): gperftools-libs-2.9.1-2.el9.x86_64.r 788 kB/s | 309 kB 00:00 2024-05-20T18:24:46.340 INFO:teuthology.orchestra.run.smithi005.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 444 kB/s | 26 kB 00:00 2024-05-20T18:24:46.398 INFO:teuthology.orchestra.run.smithi005.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 836 kB/s | 49 kB 00:00 2024-05-20T18:24:46.449 INFO:teuthology.orchestra.run.smithi005.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-05-20T18:24:46.524 INFO:teuthology.orchestra.run.smithi005.stdout:(111/159): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-05-20T18:24:46.674 INFO:teuthology.orchestra.run.smithi005.stdout:(112/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.5 MB/s | 839 kB 00:00 2024-05-20T18:24:46.783 INFO:teuthology.orchestra.run.smithi005.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 5.0 MB/s | 548 kB 00:00 2024-05-20T18:24:46.825 INFO:teuthology.orchestra.run.smithi005.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-05-20T18:24:46.867 INFO:teuthology.orchestra.run.smithi005.stdout:(115/159): python3-cachetools-4.2.4-1.el9.noarc 774 kB/s | 32 kB 00:00 2024-05-20T18:24:46.894 INFO:teuthology.orchestra.run.smithi063.stdout:(10/159): ceph-radosgw-19.0.0-2744.gd2b507c7.el 1.1 MB/s | 9.8 MB 00:09 2024-05-20T18:24:46.909 INFO:teuthology.orchestra.run.smithi005.stdout:(116/159): python3-certifi-2021.10.8-2.el9.noar 361 kB/s | 15 kB 00:00 2024-05-20T18:24:46.976 INFO:teuthology.orchestra.run.smithi005.stdout:(117/159): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-05-20T18:24:47.078 INFO:teuthology.orchestra.run.smithi005.stdout:(118/159): libarrow-9.0.0-11.el9.x86_64.rpm 5.3 MB/s | 4.4 MB 00:00 2024-05-20T18:24:47.102 INFO:teuthology.orchestra.run.smithi005.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-05-20T18:24:47.144 INFO:teuthology.orchestra.run.smithi005.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-05-20T18:24:47.153 INFO:teuthology.orchestra.run.smithi063.stdout:(11/159): cephfs-mirror-19.0.0-2744.gd2b507c7.e 855 kB/s | 221 kB 00:00 2024-05-20T18:24:47.186 INFO:teuthology.orchestra.run.smithi005.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-05-20T18:24:47.227 INFO:teuthology.orchestra.run.smithi005.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 3.4 MB/s | 144 kB 00:00 2024-05-20T18:24:47.253 INFO:teuthology.orchestra.run.smithi063.stdout:(12/159): cephfs-shell-19.0.0-2744.gd2b507c7.el 270 kB/s | 27 kB 00:00 2024-05-20T18:24:47.269 INFO:teuthology.orchestra.run.smithi005.stdout:(123/159): python3-jaraco-8.2.1-3.el9.noarch.rp 256 kB/s | 11 kB 00:00 2024-05-20T18:24:47.312 INFO:teuthology.orchestra.run.smithi005.stdout:(124/159): python3-jaraco-classes-3.2.1-5.el9.n 425 kB/s | 18 kB 00:00 2024-05-20T18:24:47.353 INFO:teuthology.orchestra.run.smithi005.stdout:(125/159): python3-jaraco-collections-3.0.0-8.e 560 kB/s | 23 kB 00:00 2024-05-20T18:24:47.361 INFO:teuthology.orchestra.run.smithi063.stdout:(13/159): libcephfs-devel-19.0.0-2744.gd2b507c7 293 kB/s | 32 kB 00:00 2024-05-20T18:24:47.395 INFO:teuthology.orchestra.run.smithi005.stdout:(126/159): python3-jaraco-functools-3.5.0-2.el9 468 kB/s | 19 kB 00:00 2024-05-20T18:24:47.437 INFO:teuthology.orchestra.run.smithi005.stdout:(127/159): python3-jaraco-text-3.2.0-6.el9.noar 472 kB/s | 20 kB 00:00 2024-05-20T18:24:47.479 INFO:teuthology.orchestra.run.smithi005.stdout:(128/159): python3-grpcio-1.46.7-10.el9.x86_64. 6.1 MB/s | 2.0 MB 00:00 2024-05-20T18:24:47.555 INFO:teuthology.orchestra.run.smithi005.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 617 kB/s | 46 kB 00:00 2024-05-20T18:24:47.597 INFO:teuthology.orchestra.run.smithi005.stdout:(130/159): python3-more-itertools-8.12.0-2.el9. 1.9 MB/s | 79 kB 00:00 2024-05-20T18:24:47.638 INFO:teuthology.orchestra.run.smithi005.stdout:(131/159): python3-natsort-7.1.1-5.el9.noarch.r 1.4 MB/s | 58 kB 00:00 2024-05-20T18:24:47.672 INFO:teuthology.orchestra.run.smithi005.stdout:(132/159): python3-kubernetes-26.1.0-2.el9.noar 4.4 MB/s | 1.0 MB 00:00 2024-05-20T18:24:47.714 INFO:teuthology.orchestra.run.smithi005.stdout:(133/159): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-05-20T18:24:47.731 INFO:teuthology.orchestra.run.smithi005.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 281 kB/s | 16 kB 00:00 2024-05-20T18:24:47.840 INFO:teuthology.orchestra.run.smithi005.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 830 kB/s | 90 kB 00:00 2024-05-20T18:24:47.873 INFO:teuthology.orchestra.run.smithi005.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 199 kB/s | 28 kB 00:00 2024-05-20T18:24:47.923 INFO:teuthology.orchestra.run.smithi005.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 368 kB/s | 31 kB 00:00 2024-05-20T18:24:47.965 INFO:teuthology.orchestra.run.smithi005.stdout:(138/159): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-05-20T18:24:47.991 INFO:teuthology.orchestra.run.smithi005.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-05-20T18:24:48.103 INFO:teuthology.orchestra.run.smithi063.stdout:(14/159): libcephfs2-19.0.0-2744.gd2b507c7.el9. 1.0 MB/s | 756 kB 00:00 2024-05-20T18:24:48.116 INFO:teuthology.orchestra.run.smithi005.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 238 kB/s | 36 kB 00:00 2024-05-20T18:24:48.158 INFO:teuthology.orchestra.run.smithi005.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 305 kB/s | 51 kB 00:00 2024-05-20T18:24:48.201 INFO:teuthology.orchestra.run.smithi005.stdout:(142/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-05-20T18:24:48.258 INFO:teuthology.orchestra.run.smithi005.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 895 kB/s | 90 kB 00:00 2024-05-20T18:24:48.287 INFO:teuthology.orchestra.run.smithi063.stdout:(15/159): libcephsqlite-19.0.0-2744.gd2b507c7.e 863 kB/s | 158 kB 00:00 2024-05-20T18:24:48.487 INFO:teuthology.orchestra.run.smithi063.stdout:(16/159): librados-devel-19.0.0-2744.gd2b507c7. 635 kB/s | 127 kB 00:00 2024-05-20T18:24:48.892 INFO:teuthology.orchestra.run.smithi005.stdout:(144/159): python3-zc-lockfile-2.0-10.el9.noarc 32 kB/s | 20 kB 00:00 2024-05-20T18:24:48.946 INFO:teuthology.orchestra.run.smithi063.stdout:(17/159): libradosstriper1-19.0.0-2744.gd2b507c 1.1 MB/s | 497 kB 00:00 2024-05-20T18:24:49.060 INFO:teuthology.orchestra.run.smithi005.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 496 kB/s | 427 kB 00:00 2024-05-20T18:24:49.160 INFO:teuthology.orchestra.run.smithi005.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 716 kB/s | 192 kB 00:00 2024-05-20T18:24:49.660 INFO:teuthology.orchestra.run.smithi005.stdout:(147/159): thrift-0.15.0-2.el9.x86_64.rpm 2.6 MB/s | 1.6 MB 00:00 2024-05-20T18:24:51.210 INFO:teuthology.orchestra.run.smithi005.stdout:(148/159): librados2-19.0.0-2744.gd2b507c7.el9. 1.6 MB/s | 3.3 MB 00:02 2024-05-20T18:24:51.394 INFO:teuthology.orchestra.run.smithi005.stdout:(149/159): NetworkManager-1.47.90-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-05-20T18:24:51.452 INFO:teuthology.orchestra.run.smithi005.stdout:(150/159): NetworkManager-libnm-1.47.90-1.el9.x 31 MB/s | 1.8 MB 00:00 2024-05-20T18:24:51.494 INFO:teuthology.orchestra.run.smithi005.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 1.0 MB/s | 41 kB 00:00 2024-05-20T18:24:51.544 INFO:teuthology.orchestra.run.smithi005.stdout:(152/159): librbd1-19.0.0-2744.gd2b507c7.el9.x8 1.6 MB/s | 3.1 MB 00:01 2024-05-20T18:24:51.570 INFO:teuthology.orchestra.run.smithi005.stdout:(153/159): NetworkManager-tui-1.47.90-1.el9.x86 3.2 MB/s | 248 kB 00:00 2024-05-20T18:24:51.570 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T18:24:51.586 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] elfutils-libelf-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libelf-0.191-4.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T18:24:51.588 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm (IP: 72.4.120.219) 2024-05-20T18:24:51.627 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] elfutils-libelf-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libelf-0.191-4.el9.x86_64.rpm (IP: 72.4.120.219) 2024-05-20T18:24:51.667 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm: Status code: 404 for https://mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm (IP: 72.4.120.219) 2024-05-20T18:24:51.685 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] elfutils-libelf-0.191-4.el9.x86_64.rpm: Status code: 404 for https://mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libelf-0.191-4.el9.x86_64.rpm (IP: 72.4.120.219) 2024-05-20T18:24:51.869 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T18:24:51.885 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] elfutils-libelf-0.191-4.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libelf-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T18:24:52.240 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T18:24:52.285 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] elfutils-libelf-0.191-4.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libelf-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T18:24:52.382 INFO:teuthology.orchestra.run.smithi005.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 44 kB/s | 37 kB 00:00 2024-05-20T18:24:52.396 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] elfutils-libs-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libs-0.191-4.el9.x86_64.rpm (IP: 72.4.120.219) 2024-05-20T18:24:52.462 INFO:teuthology.orchestra.run.smithi005.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 232 kB/s | 207 kB 00:00 2024-05-20T18:24:52.472 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] elfutils-libs-0.191-4.el9.x86_64.rpm: Status code: 404 for https://mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libs-0.191-4.el9.x86_64.rpm (IP: 72.4.120.219) 2024-05-20T18:24:52.516 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] elfutils-libs-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libs-0.191-4.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T18:24:52.612 INFO:teuthology.orchestra.run.smithi005.stdout:(156/159): openssl-3.2.1-1.el9.x86_64.rpm 9.1 MB/s | 1.3 MB 00:00 2024-05-20T18:24:52.612 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] elfutils-libs-0.191-4.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libs-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T18:24:52.693 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] elfutils-libs-0.191-4.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libs-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T18:24:52.787 INFO:teuthology.orchestra.run.smithi005.stdout:(157/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-05-20T18:24:52.813 INFO:teuthology.orchestra.run.smithi005.stdout:(158/159): elfutils-libs-0.191-4.el9.x86_64.rpm 598 kB/s | 260 kB 00:00 2024-05-20T18:24:54.529 INFO:teuthology.orchestra.run.smithi063.stdout:(18/159): ceph-test-19.0.0-2744.gd2b507c7.el9.x 3.7 MB/s | 46 MB 00:12 2024-05-20T18:24:54.596 INFO:teuthology.orchestra.run.smithi063.stdout:(19/159): librgw2-19.0.0-2744.gd2b507c7.el9.x86 913 kB/s | 5.0 MB 00:05 2024-05-20T18:24:54.621 INFO:teuthology.orchestra.run.smithi063.stdout:(20/159): python3-ceph-argparse-19.0.0-2744.gd2 492 kB/s | 45 kB 00:00 2024-05-20T18:24:54.705 INFO:teuthology.orchestra.run.smithi063.stdout:(21/159): python3-cephfs-19.0.0-2744.gd2b507c7. 1.9 MB/s | 161 kB 00:00 2024-05-20T18:24:54.955 INFO:teuthology.orchestra.run.smithi063.stdout:(22/159): ceph-common-19.0.0-2744.gd2b507c7.el9 897 kB/s | 19 MB 00:22 2024-05-20T18:24:54.980 INFO:teuthology.orchestra.run.smithi063.stdout:(23/159): python3-ceph-common-19.0.0-2744.gd2b5 352 kB/s | 135 kB 00:00 2024-05-20T18:24:55.006 INFO:teuthology.orchestra.run.smithi063.stdout:(24/159): python3-rados-19.0.0-2744.gd2b507c7.e 1.0 MB/s | 320 kB 00:00 2024-05-20T18:24:55.056 INFO:teuthology.orchestra.run.smithi063.stdout:(25/159): python3-rgw-19.0.0-2744.gd2b507c7.el9 1.9 MB/s | 99 kB 00:00 2024-05-20T18:24:55.081 INFO:teuthology.orchestra.run.smithi063.stdout:(26/159): python3-rbd-19.0.0-2744.gd2b507c7.el9 2.3 MB/s | 299 kB 00:00 2024-05-20T18:24:55.106 INFO:teuthology.orchestra.run.smithi063.stdout:(27/159): rbd-fuse-19.0.0-2744.gd2b507c7.el9.x8 855 kB/s | 86 kB 00:00 2024-05-20T18:24:55.165 INFO:teuthology.orchestra.run.smithi063.stdout:(28/159): rbd-nbd-19.0.0-2744.gd2b507c7.el9.x86 2.0 MB/s | 169 kB 00:00 2024-05-20T18:24:55.182 INFO:teuthology.orchestra.run.smithi063.stdout:(29/159): ceph-grafana-dashboards-19.0.0-2744.g 422 kB/s | 32 kB 00:00 2024-05-20T18:24:55.249 INFO:teuthology.orchestra.run.smithi063.stdout:(30/159): ceph-mgr-cephadm-19.0.0-2744.gd2b507c 1.7 MB/s | 142 kB 00:00 2024-05-20T18:24:55.491 INFO:teuthology.orchestra.run.smithi063.stdout:(31/159): rbd-mirror-19.0.0-2744.gd2b507c7.el9. 7.0 MB/s | 3.0 MB 00:00 2024-05-20T18:24:55.558 INFO:teuthology.orchestra.run.smithi063.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2744.gd2 3.6 MB/s | 249 kB 00:00 2024-05-20T18:24:55.616 INFO:teuthology.orchestra.run.smithi063.stdout:(33/159): ceph-mgr-rook-19.0.0-2744.gd2b507c7.e 849 kB/s | 49 kB 00:00 2024-05-20T18:24:55.675 INFO:teuthology.orchestra.run.smithi063.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2744.gd 282 kB/s | 16 kB 00:00 2024-05-20T18:24:55.750 INFO:teuthology.orchestra.run.smithi063.stdout:(35/159): ceph-volume-19.0.0-2744.gd2b507c7.el9 3.4 MB/s | 264 kB 00:00 2024-05-20T18:24:55.875 INFO:teuthology.orchestra.run.smithi063.stdout:(36/159): cephadm-19.0.0-2744.gd2b507c7.el9.noa 6.0 MB/s | 763 kB 00:00 2024-05-20T18:24:55.934 INFO:teuthology.orchestra.run.smithi063.stdout:(37/159): cephfs-top-19.0.0-2744.gd2b507c7.el9. 314 kB/s | 18 kB 00:00 2024-05-20T18:24:56.001 INFO:teuthology.orchestra.run.smithi063.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 1.2 MB/s | 80 kB 00:00 2024-05-20T18:24:56.093 INFO:teuthology.orchestra.run.smithi063.stdout:(39/159): libgfortran-11.4.1-3.el9.x86_64.rpm 8.6 MB/s | 803 kB 00:00 2024-05-20T18:24:56.168 INFO:teuthology.orchestra.run.smithi063.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 195 kB 00:00 2024-05-20T18:24:56.271 INFO:teuthology.orchestra.run.smithi063.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.4 MB/s | 253 kB 00:00 2024-05-20T18:24:56.377 INFO:teuthology.orchestra.run.smithi063.stdout:(42/159): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-05-20T18:24:56.444 INFO:teuthology.orchestra.run.smithi063.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-05-20T18:24:56.511 INFO:teuthology.orchestra.run.smithi063.stdout:(44/159): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-05-20T18:24:56.578 INFO:teuthology.orchestra.run.smithi063.stdout:(45/159): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-05-20T18:24:56.653 INFO:teuthology.orchestra.run.smithi063.stdout:(46/159): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-05-20T18:24:56.754 INFO:teuthology.orchestra.run.smithi063.stdout:(47/159): automake-1.16.2-8.el9.noarch.rpm 6.7 MB/s | 689 kB 00:00 2024-05-20T18:24:56.854 INFO:teuthology.orchestra.run.smithi063.stdout:(48/159): bison-3.7.4-5.el9.x86_64.rpm 9.2 MB/s | 943 kB 00:00 2024-05-20T18:24:56.921 INFO:teuthology.orchestra.run.smithi063.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-05-20T18:24:56.938 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm (IP: 72.4.120.219) 2024-05-20T18:24:57.063 INFO:teuthology.orchestra.run.smithi063.stdout:(50/159): ceph-mgr-diskprediction-local-19.0.0- 4.1 MB/s | 7.4 MB 00:01 2024-05-20T18:24:57.100 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Status code: 404 for https://mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm (IP: 72.4.120.219) 2024-05-20T18:24:57.135 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T18:24:57.180 INFO:teuthology.orchestra.run.smithi063.stdout:(51/159): flex-2.6.4-9.el9.x86_64.rpm 2.7 MB/s | 313 kB 00:00 2024-05-20T18:24:57.272 INFO:teuthology.orchestra.run.smithi063.stdout:(52/159): flexiblas-3.0.4-8.el9.x86_64.rpm 358 kB/s | 33 kB 00:00 2024-05-20T18:24:57.307 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T18:24:57.480 INFO:teuthology.orchestra.run.smithi063.stdout:(53/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-05-20T18:24:57.572 INFO:teuthology.orchestra.run.smithi063.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 199 kB/s | 18 kB 00:00 2024-05-20T18:24:57.647 INFO:teuthology.orchestra.run.smithi063.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 149 kB/s | 11 kB 00:00 2024-05-20T18:24:57.706 INFO:teuthology.orchestra.run.smithi063.stdout:(56/159): ceph-mgr-dashboard-19.0.0-2744.gd2b50 1.4 MB/s | 3.5 MB 00:02 2024-05-20T18:24:57.732 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T18:24:57.815 INFO:teuthology.orchestra.run.smithi063.stdout:(57/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 548 kB/s | 45 kB 00:00 2024-05-20T18:24:57.832 INFO:teuthology.orchestra.run.smithi063.stdout:(58/159): elfutils-libelf-devel-0.191-4.el9.x86 24 kB/s | 22 kB 00:00 2024-05-20T18:24:57.882 INFO:teuthology.orchestra.run.smithi063.stdout:(59/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 685 kB/s | 160 kB 00:00 2024-05-20T18:24:57.974 INFO:teuthology.orchestra.run.smithi063.stdout:(60/159): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-05-20T18:24:58.024 INFO:teuthology.orchestra.run.smithi063.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 222 kB/s | 25 kB 00:00 2024-05-20T18:24:58.057 INFO:teuthology.orchestra.run.smithi063.stdout:(62/159): libtool-2.4.6-45.el9.x86_64.rpm 2.8 MB/s | 582 kB 00:00 2024-05-20T18:24:58.100 INFO:teuthology.orchestra.run.smithi063.stdout:(63/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 381 kB/s | 47 kB 00:00 2024-05-20T18:24:58.192 INFO:teuthology.orchestra.run.smithi063.stdout:(64/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 292 kB 00:00 2024-05-20T18:24:58.242 INFO:teuthology.orchestra.run.smithi063.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 304 kB/s | 35 kB 00:00 2024-05-20T18:24:58.267 INFO:teuthology.orchestra.run.smithi063.stdout:(66/159): lua-5.4.4-4.el9.x86_64.rpm 1.0 MB/s | 188 kB 00:00 2024-05-20T18:24:58.484 INFO:teuthology.orchestra.run.smithi063.stdout:(67/159): openblas-openmp-0.3.21-2.el9.x86_64.r 16 MB/s | 4.7 MB 00:00 2024-05-20T18:24:58.526 INFO:teuthology.orchestra.run.smithi063.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-05-20T18:24:58.651 INFO:teuthology.orchestra.run.smithi063.stdout:(69/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-05-20T18:24:58.735 INFO:teuthology.orchestra.run.smithi063.stdout:(70/159): postgresql-contrib-13.14-1.el9.x86_64 3.5 MB/s | 911 kB 00:00 2024-05-20T18:24:58.760 INFO:teuthology.orchestra.run.smithi063.stdout:(71/159): postgresql-private-libs-13.14-1.el9.x 639 kB/s | 133 kB 00:00 2024-05-20T18:24:58.894 INFO:teuthology.orchestra.run.smithi063.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 811 kB/s | 87 kB 00:00 2024-05-20T18:24:59.011 INFO:teuthology.orchestra.run.smithi063.stdout:(73/159): postgresql-server-13.14-1.el9.x86_64. 16 MB/s | 5.8 MB 00:00 2024-05-20T18:24:59.053 INFO:teuthology.orchestra.run.smithi063.stdout:(74/159): protobuf-3.14.0-13.el9.x86_64.rpm 3.2 MB/s | 1.0 MB 00:00 2024-05-20T18:24:59.125 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] python3-colorama-0.4.6-3.el9.noarch.rpm: Status code: 404 for https://mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-colorama-0.4.6-3.el9.noarch.rpm (IP: 72.4.120.219) 2024-05-20T18:24:59.143 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] python3-colorama-0.4.6-3.el9.noarch.rpm: Status code: 404 for http://mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-colorama-0.4.6-3.el9.noarch.rpm (IP: 72.4.120.219) 2024-05-20T18:24:59.160 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] python3-colorama-0.4.6-3.el9.noarch.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-colorama-0.4.6-3.el9.noarch.rpm (IP: 199.223.246.113) 2024-05-20T18:24:59.195 INFO:teuthology.orchestra.run.smithi063.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.1 MB/s | 244 kB 00:00 2024-05-20T18:24:59.240 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] python3-colorama-0.4.6-3.el9.noarch.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-colorama-0.4.6-3.el9.noarch.rpm (IP: 140.211.166.134) 2024-05-20T18:24:59.412 INFO:teuthology.orchestra.run.smithi063.stdout:(76/159): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-05-20T18:24:59.414 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] python3-jinja2-2.11.3-6.el9.noarch.rpm: Status code: 404 for https://mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-jinja2-2.11.3-6.el9.noarch.rpm (IP: 72.4.120.219) 2024-05-20T18:24:59.417 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] python3-colorama-0.4.6-3.el9.noarch.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-colorama-0.4.6-3.el9.noarch.rpm (IP: 140.211.166.134) 2024-05-20T18:24:59.449 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] python3-jinja2-2.11.3-6.el9.noarch.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-jinja2-2.11.3-6.el9.noarch.rpm (IP: 199.223.246.113) 2024-05-20T18:24:59.512 INFO:teuthology.orchestra.run.smithi063.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 260 kB/s | 26 kB 00:00 2024-05-20T18:24:59.512 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] python3-jinja2-2.11.3-6.el9.noarch.rpm: Status code: 404 for http://mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-jinja2-2.11.3-6.el9.noarch.rpm (IP: 72.4.120.219) 2024-05-20T18:24:59.537 INFO:teuthology.orchestra.run.smithi063.stdout:(78/159): python3-colorama-0.4.6-3.el9.noarch.r 117 kB/s | 61 kB 00:00 2024-05-20T18:24:59.627 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] python3-jinja2-2.11.3-6.el9.noarch.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-jinja2-2.11.3-6.el9.noarch.rpm (IP: 140.211.166.134) 2024-05-20T18:24:59.663 INFO:teuthology.orchestra.run.smithi063.stdout:(79/159): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-05-20T18:24:59.879 INFO:teuthology.orchestra.run.smithi063.stdout:(80/159): python3-markupsafe-1.1.1-12.el9.x86_6 161 kB/s | 35 kB 00:00 2024-05-20T18:24:59.880 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] python3-jinja2-2.11.3-6.el9.noarch.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-jinja2-2.11.3-6.el9.noarch.rpm (IP: 140.211.166.134) 2024-05-20T18:24:59.905 INFO:teuthology.orchestra.run.smithi063.stdout:(81/159): python3-jsonpointer-2.0-4.el9.noarch. 49 kB/s | 19 kB 00:00 2024-05-20T18:24:59.988 INFO:teuthology.orchestra.run.smithi063.stdout:(82/159): python3-jinja2-2.11.3-6.el9.noarch.rp 313 kB/s | 248 kB 00:00 2024-05-20T18:25:00.063 INFO:teuthology.orchestra.run.smithi063.stdout:(83/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.8 MB/s | 250 kB 00:00 2024-05-20T18:25:00.130 INFO:teuthology.orchestra.run.smithi063.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 1.5 MB/s | 222 kB 00:00 2024-05-20T18:25:00.197 INFO:teuthology.orchestra.run.smithi063.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 579 kB/s | 77 kB 00:00 2024-05-20T18:25:00.290 INFO:teuthology.orchestra.run.smithi063.stdout:(86/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-05-20T18:25:00.339 INFO:teuthology.orchestra.run.smithi063.stdout:(87/159): python3-prettytable-0.7.2-27.el9.noar 203 kB/s | 42 kB 00:00 2024-05-20T18:25:00.431 INFO:teuthology.orchestra.run.smithi063.stdout:(88/159): python3-protobuf-3.14.0-13.el9.noarch 1.1 MB/s | 269 kB 00:00 2024-05-20T18:25:00.490 INFO:teuthology.orchestra.run.smithi063.stdout:(89/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 910 kB/s | 159 kB 00:00 2024-05-20T18:25:00.548 INFO:teuthology.orchestra.run.smithi063.stdout:(90/159): python3-pyasn1-modules-0.4.8-6.el9.no 1.5 MB/s | 279 kB 00:00 2024-05-20T18:25:00.574 INFO:teuthology.orchestra.run.smithi063.stdout:(91/159): python3-pytz-2021.1-5.el9.noarch.rpm 361 kB/s | 51 kB 00:00 2024-05-20T18:25:00.625 INFO:teuthology.orchestra.run.smithi063.stdout:(92/159): python3-requests-oauthlib-1.3.0-12.el 403 kB/s | 54 kB 00:00 2024-05-20T18:25:00.733 INFO:teuthology.orchestra.run.smithi063.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 312 kB/s | 42 kB 00:00 2024-05-20T18:25:00.758 INFO:teuthology.orchestra.run.smithi063.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 415 kB/s | 44 kB 00:00 2024-05-20T18:25:00.875 INFO:teuthology.orchestra.run.smithi063.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 402 kB/s | 36 kB 00:00 2024-05-20T18:25:00.900 INFO:teuthology.orchestra.run.smithi063.stdout:(96/159): qatlib-24.02.0-1.el9.x86_64.rpm 1.3 MB/s | 221 kB 00:00 2024-05-20T18:25:01.017 INFO:teuthology.orchestra.run.smithi063.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 334 kB/s | 47 kB 00:00 2024-05-20T18:25:01.084 INFO:teuthology.orchestra.run.smithi063.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 1.9 MB/s | 305 kB 00:00 2024-05-20T18:25:01.143 INFO:teuthology.orchestra.run.smithi063.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 464 kB/s | 58 kB 00:00 2024-05-20T18:25:01.218 INFO:teuthology.orchestra.run.smithi063.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 708 kB/s | 94 kB 00:00 2024-05-20T18:25:01.385 INFO:teuthology.orchestra.run.smithi063.stdout:(101/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-05-20T18:25:01.427 INFO:teuthology.orchestra.run.smithi063.stdout:(102/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 224 kB/s | 64 kB 00:00 2024-05-20T18:25:01.461 INFO:teuthology.orchestra.run.smithi063.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 461 kB/s | 22 kB 00:00 2024-05-20T18:25:01.478 INFO:teuthology.orchestra.run.smithi063.stdout:(104/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 173 kB/s | 45 kB 00:00 2024-05-20T18:25:01.545 INFO:teuthology.orchestra.run.smithi063.stdout:(105/159): protobuf-compiler-3.14.0-13.el9.x86_ 7.3 MB/s | 863 kB 00:00 2024-05-20T18:25:01.662 INFO:teuthology.orchestra.run.smithi063.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 167 kB/s | 19 kB 00:00 2024-05-20T18:25:01.720 INFO:teuthology.orchestra.run.smithi063.stdout:(107/159): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-05-20T18:25:01.754 INFO:teuthology.orchestra.run.smithi063.stdout:(108/159): abseil-cpp-20211102.0-3.el9.x86_64.r 1.8 MB/s | 548 kB 00:00 2024-05-20T18:25:01.771 INFO:teuthology.orchestra.run.smithi063.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 519 kB/s | 26 kB 00:00 2024-05-20T18:25:01.813 INFO:teuthology.orchestra.run.smithi063.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 830 kB/s | 49 kB 00:00 2024-05-20T18:25:01.838 INFO:teuthology.orchestra.run.smithi063.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-05-20T18:25:01.888 INFO:teuthology.orchestra.run.smithi063.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-05-20T18:25:02.097 INFO:teuthology.orchestra.run.smithi063.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.2 MB/s | 839 kB 00:00 2024-05-20T18:25:02.130 INFO:teuthology.orchestra.run.smithi063.stdout:(114/159): python3-asyncssh-2.13.2-5.el9.noarch 2.2 MB/s | 548 kB 00:00 2024-05-20T18:25:02.147 INFO:teuthology.orchestra.run.smithi063.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 865 kB/s | 43 kB 00:00 2024-05-20T18:25:02.214 INFO:teuthology.orchestra.run.smithi063.stdout:(116/159): libarrow-9.0.0-11.el9.x86_64.rpm 8.0 MB/s | 4.4 MB 00:00 2024-05-20T18:25:02.231 INFO:teuthology.orchestra.run.smithi063.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 179 kB/s | 15 kB 00:00 2024-05-20T18:25:02.248 INFO:teuthology.orchestra.run.smithi063.stdout:(118/159): python3-cachetools-4.2.4-1.el9.noarc 274 kB/s | 32 kB 00:00 2024-05-20T18:25:02.282 INFO:teuthology.orchestra.run.smithi063.stdout:(119/159): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-05-20T18:25:02.307 INFO:teuthology.orchestra.run.smithi063.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-05-20T18:25:02.332 INFO:teuthology.orchestra.run.smithi063.stdout:(121/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-05-20T18:25:02.358 INFO:teuthology.orchestra.run.smithi063.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-05-20T18:25:02.399 INFO:teuthology.orchestra.run.smithi063.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-05-20T18:25:02.425 INFO:teuthology.orchestra.run.smithi063.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 159 kB/s | 11 kB 00:00 2024-05-20T18:25:02.467 INFO:teuthology.orchestra.run.smithi063.stdout:(125/159): python3-grpcio-1.46.7-10.el9.x86_64. 13 MB/s | 2.0 MB 00:00 2024-05-20T18:25:02.492 INFO:teuthology.orchestra.run.smithi063.stdout:(126/159): python3-jaraco-classes-3.2.1-5.el9.n 191 kB/s | 18 kB 00:00 2024-05-20T18:25:02.517 INFO:teuthology.orchestra.run.smithi063.stdout:(127/159): python3-jaraco-collections-3.0.0-8.e 251 kB/s | 23 kB 00:00 2024-05-20T18:25:02.543 INFO:teuthology.orchestra.run.smithi063.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-05-20T18:25:02.568 INFO:teuthology.orchestra.run.smithi063.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-05-20T18:25:02.619 INFO:teuthology.orchestra.run.smithi063.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 612 kB/s | 46 kB 00:00 2024-05-20T18:25:02.652 INFO:teuthology.orchestra.run.smithi063.stdout:(131/159): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-05-20T18:25:02.677 INFO:teuthology.orchestra.run.smithi063.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 986 kB/s | 58 kB 00:00 2024-05-20T18:25:02.702 INFO:teuthology.orchestra.run.smithi063.stdout:(133/159): python3-more-itertools-8.12.0-2.el9. 586 kB/s | 79 kB 00:00 2024-05-20T18:25:02.728 INFO:teuthology.orchestra.run.smithi063.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-05-20T18:25:02.753 INFO:teuthology.orchestra.run.smithi063.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-05-20T18:25:02.778 INFO:teuthology.orchestra.run.smithi063.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-05-20T18:25:02.829 INFO:teuthology.orchestra.run.smithi063.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 280 kB/s | 28 kB 00:00 2024-05-20T18:25:02.854 INFO:teuthology.orchestra.run.smithi063.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 305 kB/s | 31 kB 00:00 2024-05-20T18:25:02.887 INFO:teuthology.orchestra.run.smithi063.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-05-20T18:25:02.913 INFO:teuthology.orchestra.run.smithi063.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 705 kB/s | 59 kB 00:00 2024-05-20T18:25:02.938 INFO:teuthology.orchestra.run.smithi063.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 426 kB/s | 36 kB 00:00 2024-05-20T18:25:02.963 INFO:teuthology.orchestra.run.smithi063.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 674 kB/s | 51 kB 00:00 2024-05-20T18:25:02.989 INFO:teuthology.orchestra.run.smithi063.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-20T18:25:03.014 INFO:teuthology.orchestra.run.smithi063.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-05-20T18:25:03.048 INFO:teuthology.orchestra.run.smithi063.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-05-20T18:25:03.072 INFO:teuthology.orchestra.run.smithi063.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 239 kB/s | 20 kB 00:00 2024-05-20T18:25:03.098 INFO:teuthology.orchestra.run.smithi063.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-05-20T18:25:03.165 INFO:teuthology.orchestra.run.smithi063.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-05-20T18:25:03.307 INFO:teuthology.orchestra.run.smithi063.stdout:(149/159): NetworkManager-1.47.90-1.el9.x86_64. 16 MB/s | 2.3 MB 00:00 2024-05-20T18:25:03.432 INFO:teuthology.orchestra.run.smithi063.stdout:(150/159): NetworkManager-libnm-1.47.90-1.el9.x 14 MB/s | 1.8 MB 00:00 2024-05-20T18:25:03.490 INFO:teuthology.orchestra.run.smithi063.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 713 kB/s | 41 kB 00:00 2024-05-20T18:25:03.691 INFO:teuthology.orchestra.run.smithi063.stdout:(152/159): NetworkManager-tui-1.47.90-1.el9.x86 1.2 MB/s | 248 kB 00:00 2024-05-20T18:25:03.708 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm (IP: 72.4.120.219) 2024-05-20T18:25:03.764 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm: Status code: 404 for https://mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm (IP: 72.4.120.219) 2024-05-20T18:25:03.843 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T18:25:03.924 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T18:25:03.941 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-debuginfod-client-0.191-4.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T18:25:04.058 INFO:teuthology.orchestra.run.smithi063.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 100 kB/s | 37 kB 00:00 2024-05-20T18:25:04.113 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] elfutils-libelf-0.191-4.el9.x86_64.rpm: Status code: 404 for https://mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libelf-0.191-4.el9.x86_64.rpm (IP: 72.4.120.219) 2024-05-20T18:25:04.130 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] elfutils-libelf-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libelf-0.191-4.el9.x86_64.rpm (IP: 72.4.120.219) 2024-05-20T18:25:04.225 INFO:teuthology.orchestra.run.smithi063.stdout:(154/159): librados2-19.0.0-2744.gd2b507c7.el9. 2.9 MB/s | 3.3 MB 00:01 2024-05-20T18:25:04.225 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] elfutils-libelf-0.191-4.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libelf-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T18:25:04.279 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] elfutils-libs-0.191-4.el9.x86_64.rpm: Status code: 404 for https://mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libs-0.191-4.el9.x86_64.rpm (IP: 72.4.120.219) 2024-05-20T18:25:04.305 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] elfutils-libelf-0.191-4.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libelf-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T18:25:04.325 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] elfutils-libelf-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libelf-0.191-4.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T18:25:04.346 INFO:teuthology.orchestra.run.smithi005.stdout:(159/159): ceph-test-19.0.0-2744.gd2b507c7.el9. 1.6 MB/s | 46 MB 00:29 2024-05-20T18:25:04.352 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-05-20T18:25:04.352 INFO:teuthology.orchestra.run.smithi005.stdout:Total 6.2 MB/s | 222 MB 00:35 2024-05-20T18:25:04.358 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] elfutils-libs-0.191-4.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libs-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T18:25:04.392 INFO:teuthology.orchestra.run.smithi063.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 620 kB/s | 207 kB 00:00 2024-05-20T18:25:04.392 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] elfutils-libs-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libs-0.191-4.el9.x86_64.rpm (IP: 72.4.120.219) 2024-05-20T18:25:04.492 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] elfutils-libs-0.191-4.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libs-0.191-4.el9.x86_64.rpm (IP: 140.211.166.134) 2024-05-20T18:25:04.515 INFO:teuthology.orchestra.run.smithi063.stdout:[MIRROR] elfutils-libs-0.191-4.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/elfutils-libs-0.191-4.el9.x86_64.rpm (IP: 199.223.246.113) 2024-05-20T18:25:04.567 INFO:teuthology.orchestra.run.smithi063.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 759 kB/s | 260 kB 00:00 2024-05-20T18:25:05.142 INFO:teuthology.orchestra.run.smithi063.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 1.8 MB/s | 1.3 MB 00:00 2024-05-20T18:25:05.343 INFO:teuthology.orchestra.run.smithi063.stdout:(158/159): librbd1-19.0.0-2744.gd2b507c7.el9.x8 1.4 MB/s | 3.1 MB 00:02 2024-05-20T18:25:05.629 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-05-20T18:25:05.642 INFO:teuthology.orchestra.run.smithi063.stdout:(159/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.2 MB/s | 2.4 MB 00:01 2024-05-20T18:25:05.648 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-05-20T18:25:05.648 INFO:teuthology.orchestra.run.smithi063.stdout:Total 6.6 MB/s | 222 MB 00:33 2024-05-20T18:25:05.808 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-05-20T18:25:05.808 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-05-20T18:25:06.887 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-05-20T18:25:07.065 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-05-20T18:25:07.065 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-05-20T18:25:08.103 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-05-20T18:25:08.103 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-05-20T18:25:09.351 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-05-20T18:25:09.351 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-05-20T18:25:14.543 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-05-20T18:25:14.625 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-20T18:25:14.672 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-20T18:25:14.733 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-20T18:25:14.768 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-20T18:25:14.793 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-20T18:25:14.848 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-20T18:25:15.285 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-20T18:25:15.327 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/170 2024-05-20T18:25:15.455 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/170 2024-05-20T18:25:15.486 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/170 2024-05-20T18:25:15.578 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/170 2024-05-20T18:25:15.615 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 10/170 2024-05-20T18:25:15.672 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-20T18:25:15.777 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-05-20T18:25:15.857 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-20T18:25:15.880 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-20T18:25:15.903 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-20T18:25:15.931 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-20T18:25:15.958 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/170 2024-05-20T18:25:15.959 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-20T18:25:15.999 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-20T18:25:16.024 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-20T18:25:16.065 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/170 2024-05-20T18:25:16.080 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-20T18:25:16.092 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 15/170 2024-05-20T18:25:16.172 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 15/170 2024-05-20T18:25:16.185 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-20T18:25:16.208 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-20T18:25:16.405 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-20T18:25:16.427 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-20T18:25:16.570 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-20T18:25:16.591 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/170 2024-05-20T18:25:16.691 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-20T18:25:16.727 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/170 2024-05-20T18:25:16.750 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/170 2024-05-20T18:25:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-20T18:25:16.826 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/170 2024-05-20T18:25:16.853 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-20T18:25:16.860 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 10/170 2024-05-20T18:25:16.913 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-20T18:25:16.935 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-20T18:25:17.125 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-20T18:25:17.140 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-20T18:25:17.253 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-20T18:25:17.278 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-20T18:25:17.306 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/170 2024-05-20T18:25:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-20T18:25:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-05-20T18:25:17.413 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/170 2024-05-20T18:25:17.427 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-05-20T18:25:17.452 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 15/170 2024-05-20T18:25:17.616 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-05-20T18:25:17.616 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-05-20T18:25:17.616 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-05-20T18:25:17.616 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 15/170 2024-05-20T18:25:17.616 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-20T18:25:17.616 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-20T18:25:17.633 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-05-20T18:25:17.646 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-05-20T18:25:17.717 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-05-20T18:25:17.753 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-20T18:25:17.770 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-05-20T18:25:17.799 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-05-20T18:25:17.843 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-05-20T18:25:17.921 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-05-20T18:25:17.925 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-20T18:25:17.955 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9. 34/170 2024-05-20T18:25:18.001 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el 35/170 2024-05-20T18:25:18.025 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_6 36/170 2024-05-20T18:25:18.046 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-20T18:25:18.106 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-20T18:25:18.111 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-05-20T18:25:18.149 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-05-20T18:25:18.211 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-20T18:25:18.252 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-05-20T18:25:18.283 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-20T18:25:18.397 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-05-20T18:25:18.453 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-05-20T18:25:18.484 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-20T18:25:18.591 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-20T18:25:18.687 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-20T18:25:18.730 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-05-20T18:25:18.882 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-05-20T18:25:18.957 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-05-20T18:25:18.998 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-05-20T18:25:19.008 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-05-20T18:25:19.042 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-05-20T18:25:19.074 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-05-20T18:25:19.101 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-05-20T18:25:19.108 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-05-20T18:25:19.143 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-05-20T18:25:19.165 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-05-20T18:25:19.193 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-05-20T18:25:19.216 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-05-20T18:25:19.237 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-05-20T18:25:19.246 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-05-20T18:25:19.283 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_6 47/170 2024-05-20T18:25:19.289 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-05-20T18:25:19.325 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-05-20T18:25:19.367 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-05-20T18:25:19.400 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9. 34/170 2024-05-20T18:25:19.444 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el 35/170 2024-05-20T18:25:19.472 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_6 36/170 2024-05-20T18:25:19.480 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-05-20T18:25:19.544 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-05-20T18:25:19.546 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-05-20T18:25:19.587 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-05-20T18:25:19.600 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-05-20T18:25:19.665 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-05-20T18:25:19.673 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-20T18:25:19.711 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-20T18:25:19.747 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-20T18:25:19.795 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-20T18:25:19.830 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-20T18:25:19.874 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-20T18:25:19.916 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-20T18:25:19.927 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-05-20T18:25:19.982 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-05-20T18:25:20.060 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-20T18:25:20.101 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-20T18:25:20.143 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-20T18:25:20.240 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-20T18:25:20.284 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-20T18:25:20.325 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-20T18:25:20.357 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-20T18:25:20.382 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-20T18:25:20.484 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-20T18:25:20.538 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-05-20T18:25:20.615 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-20T18:25:20.629 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-05-20T18:25:20.648 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-20T18:25:20.663 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-05-20T18:25:20.676 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-20T18:25:20.707 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-05-20T18:25:20.714 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-20T18:25:20.758 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-05-20T18:25:20.758 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-20T18:25:20.795 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_6 47/170 2024-05-20T18:25:20.798 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-20T18:25:20.827 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-20T18:25:20.838 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-05-20T18:25:20.862 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-20T18:25:20.895 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-20T18:25:20.937 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-20T18:25:20.972 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-20T18:25:20.999 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-05-20T18:25:21.048 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-05-20T18:25:21.062 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-20T18:25:21.104 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-05-20T18:25:21.111 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-20T18:25:21.142 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-20T18:25:21.179 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-20T18:25:21.190 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-20T18:25:21.215 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-20T18:25:21.252 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-20T18:25:21.291 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-20T18:25:21.299 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-20T18:25:21.336 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-20T18:25:21.344 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-20T18:25:21.380 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-20T18:25:21.422 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-20T18:25:21.566 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-20T18:25:21.607 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-20T18:25:21.640 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-20T18:25:21.739 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-20T18:25:21.773 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-20T18:25:21.815 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-20T18:25:21.847 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-20T18:25:21.871 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-20T18:25:21.963 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-20T18:25:22.096 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-20T18:25:22.108 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-20T18:25:22.138 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-20T18:25:22.166 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-20T18:25:22.167 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-20T18:25:22.201 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-20T18:25:22.210 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-20T18:25:22.249 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-20T18:25:22.280 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-20T18:25:22.304 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-20T18:25:22.309 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-20T18:25:22.348 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-20T18:25:22.352 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-20T18:25:22.386 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-20T18:25:22.393 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-20T18:25:22.428 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-20T18:25:22.461 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-20T18:25:22.485 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-20T18:25:22.520 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-20T18:25:22.552 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-20T18:25:22.601 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-20T18:25:22.632 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-20T18:25:22.681 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-20T18:25:22.704 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-20T18:25:22.734 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-20T18:25:22.790 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-20T18:25:22.815 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-20T18:25:22.842 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-20T18:25:23.505 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-20T18:25:23.608 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-20T18:25:23.627 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-20T18:25:23.655 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-20T18:25:23.689 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-20T18:25:23.792 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-20T18:25:23.845 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-20T18:25:23.890 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-20T18:25:23.980 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-20T18:25:24.017 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-20T18:25:24.192 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-20T18:25:24.222 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-20T18:25:24.304 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-20T18:25:24.962 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-20T18:25:25.004 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-20T18:25:25.046 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-20T18:25:25.075 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-20T18:25:25.108 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-20T18:25:25.125 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-20T18:25:25.444 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-20T18:25:25.487 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 103/170 2024-05-20T18:25:25.561 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 103/170 2024-05-20T18:25:25.592 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 104/170 2024-05-20T18:25:25.632 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-20T18:25:26.131 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-20T18:25:26.156 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 107/170 2024-05-20T18:25:26.270 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 107/170 2024-05-20T18:25:26.300 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 108/170 2024-05-20T18:25:26.470 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-20T18:25:26.552 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-20T18:25:26.581 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-20T18:25:26.614 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-20T18:25:26.951 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-20T18:25:26.984 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 103/170 2024-05-20T18:25:27.076 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 103/170 2024-05-20T18:25:27.106 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 104/170 2024-05-20T18:25:27.143 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-20T18:25:27.645 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-20T18:25:27.679 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 107/170 2024-05-20T18:25:27.751 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 107/170 2024-05-20T18:25:27.781 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 108/170 2024-05-20T18:25:28.630 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/170 2024-05-20T18:25:28.663 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/170 2024-05-20T18:25:29.252 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/170 2024-05-20T18:25:29.291 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 110/170 2024-05-20T18:25:29.383 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 110/170 2024-05-20T18:25:29.383 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-20T18:25:29.384 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-20T18:25:29.384 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:25:29.417 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 111/170 2024-05-20T18:25:30.076 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/170 2024-05-20T18:25:30.111 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/170 2024-05-20T18:25:30.699 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 109/170 2024-05-20T18:25:30.739 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 110/170 2024-05-20T18:25:30.843 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 110/170 2024-05-20T18:25:30.843 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-20T18:25:30.843 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-20T18:25:30.843 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:25:30.874 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 111/170 2024-05-20T18:25:41.815 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 111/170 2024-05-20T18:25:41.816 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /sys 2024-05-20T18:25:41.816 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /proc 2024-05-20T18:25:41.816 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /mnt 2024-05-20T18:25:41.816 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /var/tmp 2024-05-20T18:25:41.816 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /home 2024-05-20T18:25:41.816 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /root 2024-05-20T18:25:41.816 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /tmp 2024-05-20T18:25:41.816 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:25:41.982 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 112/170 2024-05-20T18:25:42.017 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 112/170 2024-05-20T18:25:42.017 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T18:25:42.017 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-20T18:25:42.017 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:25:43.447 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 111/170 2024-05-20T18:25:43.448 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /sys 2024-05-20T18:25:43.448 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /proc 2024-05-20T18:25:43.448 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /mnt 2024-05-20T18:25:43.448 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /var/tmp 2024-05-20T18:25:43.448 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /home 2024-05-20T18:25:43.448 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /root 2024-05-20T18:25:43.448 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /tmp 2024-05-20T18:25:43.448 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:25:43.616 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 112/170 2024-05-20T18:25:43.633 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 112/170 2024-05-20T18:25:43.633 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T18:25:43.633 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-20T18:25:43.633 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:25:43.669 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 113/170 2024-05-20T18:25:43.698 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 113/170 2024-05-20T18:25:43.698 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T18:25:43.698 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-20T18:25:43.698 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-20T18:25:43.698 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-20T18:25:43.698 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:25:43.936 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 114/170 2024-05-20T18:25:43.961 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 114/170 2024-05-20T18:25:43.961 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T18:25:43.961 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-20T18:25:43.961 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-20T18:25:43.961 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-20T18:25:43.961 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:25:44.454 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 115/170 2024-05-20T18:25:44.479 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 115/170 2024-05-20T18:25:44.479 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T18:25:44.479 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-20T18:25:44.479 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-20T18:25:44.479 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-20T18:25:44.479 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:25:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-20T18:25:44.643 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-20T18:25:44.707 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-20T18:25:44.911 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-20T18:25:44.963 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-20T18:25:45.045 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-20T18:25:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-20T18:25:45.284 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-20T18:25:45.294 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 113/170 2024-05-20T18:25:45.322 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 113/170 2024-05-20T18:25:45.322 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T18:25:45.322 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-20T18:25:45.322 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-20T18:25:45.323 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-20T18:25:45.323 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:25:45.339 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-20T18:25:45.388 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-20T18:25:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-20T18:25:45.561 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 114/170 2024-05-20T18:25:45.586 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 114/170 2024-05-20T18:25:45.586 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T18:25:45.586 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-20T18:25:45.586 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-20T18:25:45.587 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-20T18:25:45.587 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:25:45.895 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 127/170 2024-05-20T18:25:45.921 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 127/170 2024-05-20T18:25:45.953 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.e 128/170 2024-05-20T18:25:46.017 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7. 129/170 2024-05-20T18:25:46.053 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 130/170 2024-05-20T18:25:46.078 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 115/170 2024-05-20T18:25:46.103 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 115/170 2024-05-20T18:25:46.104 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T18:25:46.104 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-20T18:25:46.104 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-20T18:25:46.104 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-20T18:25:46.104 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:25:46.218 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-20T18:25:46.276 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-20T18:25:46.340 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-20T18:25:46.535 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-20T18:25:46.588 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-20T18:25:46.669 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-20T18:25:46.758 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-20T18:25:46.910 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-20T18:25:46.972 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-20T18:25:47.008 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-20T18:25:47.043 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-20T18:25:47.145 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 130/170 2024-05-20T18:25:47.179 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 131/170 2024-05-20T18:25:47.462 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 127/170 2024-05-20T18:25:47.488 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 127/170 2024-05-20T18:25:47.537 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.e 128/170 2024-05-20T18:25:47.601 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7. 129/170 2024-05-20T18:25:47.628 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 130/170 2024-05-20T18:25:48.197 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 131/170 2024-05-20T18:25:48.221 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 132/170 2024-05-20T18:25:48.329 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 132/170 2024-05-20T18:25:48.530 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el 133/170 2024-05-20T18:25:48.557 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 134/170 2024-05-20T18:25:48.582 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 134/170 2024-05-20T18:25:48.583 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T18:25:48.583 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-20T18:25:48.583 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-20T18:25:48.583 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-20T18:25:48.583 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:25:48.636 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 135/170 2024-05-20T18:25:48.810 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 130/170 2024-05-20T18:25:48.810 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 131/170 2024-05-20T18:25:48.810 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 135/170 2024-05-20T18:25:48.810 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 136/170 2024-05-20T18:25:48.810 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 137/170 2024-05-20T18:25:49.774 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-20T18:25:49.775 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 131/170 2024-05-20T18:25:49.805 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 132/170 2024-05-20T18:25:49.813 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 139/170 2024-05-20T18:25:49.838 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 139/170 2024-05-20T18:25:49.838 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T18:25:49.838 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-20T18:25:49.838 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-20T18:25:49.839 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-20T18:25:49.839 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:25:49.889 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 140/170 2024-05-20T18:25:49.914 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 140/170 2024-05-20T18:25:49.914 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T18:25:49.915 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-20T18:25:49.915 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:25:49.915 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 132/170 2024-05-20T18:25:49.964 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 141/170 2024-05-20T18:25:49.989 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 141/170 2024-05-20T18:25:49.989 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T18:25:49.989 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-20T18:25:49.989 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-20T18:25:49.989 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-20T18:25:49.990 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:25:50.085 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el 133/170 2024-05-20T18:25:50.135 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 134/170 2024-05-20T18:25:50.160 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 134/170 2024-05-20T18:25:50.160 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T18:25:50.160 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-20T18:25:50.160 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-20T18:25:50.160 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-20T18:25:50.161 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:25:50.211 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 135/170 2024-05-20T18:25:50.222 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 135/170 2024-05-20T18:25:50.315 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 142/170 2024-05-20T18:25:50.326 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 136/170 2024-05-20T18:25:50.340 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 142/170 2024-05-20T18:25:50.340 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T18:25:50.340 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-20T18:25:50.340 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-20T18:25:50.341 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-20T18:25:50.341 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:25:50.378 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 137/170 2024-05-20T18:25:51.347 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-20T18:25:51.377 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 139/170 2024-05-20T18:25:51.401 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 139/170 2024-05-20T18:25:51.401 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T18:25:51.401 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-20T18:25:51.402 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-20T18:25:51.402 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-20T18:25:51.402 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:25:51.453 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 140/170 2024-05-20T18:25:51.476 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 140/170 2024-05-20T18:25:51.477 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T18:25:51.477 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-20T18:25:51.477 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:25:51.528 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 141/170 2024-05-20T18:25:51.555 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 141/170 2024-05-20T18:25:51.555 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T18:25:51.555 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-20T18:25:51.555 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-20T18:25:51.555 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-20T18:25:51.555 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:25:51.879 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 142/170 2024-05-20T18:25:51.903 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 142/170 2024-05-20T18:25:51.903 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-20T18:25:51.903 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-20T18:25:51.904 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-20T18:25:51.904 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-20T18:25:51.904 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:25:55.178 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 143/170 2024-05-20T18:25:55.220 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 144/170 2024-05-20T18:25:55.261 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 145/170 2024-05-20T18:25:55.292 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-20T18:25:55.319 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-20T18:25:55.517 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 148/170 2024-05-20T18:25:55.565 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-20T18:25:55.598 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_ 150/170 2024-05-20T18:25:55.630 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-20T18:25:55.660 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-20T18:25:55.706 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-20T18:25:56.310 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-20T18:25:56.354 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-20T18:25:56.375 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-20T18:25:56.716 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 143/170 2024-05-20T18:25:56.758 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 144/170 2024-05-20T18:25:56.807 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 145/170 2024-05-20T18:25:56.847 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-20T18:25:56.874 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-20T18:25:57.082 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 148/170 2024-05-20T18:25:57.121 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-20T18:25:57.159 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_ 150/170 2024-05-20T18:25:57.192 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-20T18:25:57.222 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-20T18:25:57.268 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-20T18:25:57.294 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 155/170 2024-05-20T18:25:57.376 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-20T18:25:57.418 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-20T18:25:57.520 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-20T18:25:57.554 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-20T18:25:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-20T18:25:57.638 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-20T18:25:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-20T18:25:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-20T18:25:57.749 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-20T18:25:57.782 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-20T18:25:57.782 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-20T18:25:57.875 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-20T18:25:57.898 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-20T18:25:57.900 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-20T18:25:57.906 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-20T18:25:57.906 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-20T18:25:57.940 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-20T18:25:57.961 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-20T18:25:58.102 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-20T18:25:58.102 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-20T18:25:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-20T18:25:58.202 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-20T18:25:58.272 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-20T18:25:58.865 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 155/170 2024-05-20T18:25:58.953 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-20T18:25:58.996 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-20T18:25:59.089 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-20T18:25:59.124 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-20T18:25:59.148 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-20T18:25:59.173 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-20T18:25:59.208 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-20T18:25:59.208 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-20T18:25:59.309 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-20T18:25:59.342 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-20T18:25:59.342 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-20T18:25:59.426 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-20T18:25:59.451 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-20T18:25:59.457 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-20T18:25:59.457 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-20T18:25:59.645 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-20T18:25:59.646 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-20T18:25:59.730 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-20T18:25:59.762 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-20T18:25:59.830 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-20T18:26:04.108 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-20T18:26:04.109 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 1/170 2024-05-20T18:26:04.109 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 2/170 2024-05-20T18:26:04.109 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 3/170 2024-05-20T18:26:04.109 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 4/170 2024-05-20T18:26:04.109 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 5/170 2024-05-20T18:26:04.109 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 6/170 2024-05-20T18:26:04.109 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 7/170 2024-05-20T18:26:04.109 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/170 2024-05-20T18:26:04.109 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/170 2024-05-20T18:26:04.109 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 10/170 2024-05-20T18:26:04.109 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 11/170 2024-05-20T18:26:04.110 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 12/170 2024-05-20T18:26:04.110 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 13/170 2024-05-20T18:26:04.110 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/170 2024-05-20T18:26:04.110 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_ 15/170 2024-05-20T18:26:04.110 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 16/170 2024-05-20T18:26:04.110 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 17/170 2024-05-20T18:26:04.110 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_6 18/170 2024-05-20T18:26:04.110 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 19/170 2024-05-20T18:26:04.110 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 20/170 2024-05-20T18:26:04.110 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el 21/170 2024-05-20T18:26:04.110 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9. 22/170 2024-05-20T18:26:04.110 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_6 23/170 2024-05-20T18:26:04.110 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 24/170 2024-05-20T18:26:04.111 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 25/170 2024-05-20T18:26:04.111 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 26/170 2024-05-20T18:26:04.111 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 27/170 2024-05-20T18:26:04.123 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 28/170 2024-05-20T18:26:04.123 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 29/170 2024-05-20T18:26:04.123 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7. 30/170 2024-05-20T18:26:04.123 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 31/170 2024-05-20T18:26:04.124 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 32/170 2024-05-20T18:26:04.124 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 33/170 2024-05-20T18:26:04.124 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el 34/170 2024-05-20T18:26:04.124 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 35/170 2024-05-20T18:26:04.124 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.e 36/170 2024-05-20T18:26:04.124 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 37/170 2024-05-20T18:26:04.124 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 38/170 2024-05-20T18:26:04.124 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 39/170 2024-05-20T18:26:04.124 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-20T18:26:04.124 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-20T18:26:04.124 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-20T18:26:04.124 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-20T18:26:04.124 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-20T18:26:04.124 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-20T18:26:04.125 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-20T18:26:04.125 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-20T18:26:04.125 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-20T18:26:04.125 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-20T18:26:04.125 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-20T18:26:04.125 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-20T18:26:04.125 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-20T18:26:04.125 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-20T18:26:04.125 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-20T18:26:04.125 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-20T18:26:04.125 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-20T18:26:04.125 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-20T18:26:04.125 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-20T18:26:04.126 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-20T18:26:04.126 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-20T18:26:04.126 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-20T18:26:04.126 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-20T18:26:04.126 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-20T18:26:04.126 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-20T18:26:04.126 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-20T18:26:04.126 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-20T18:26:04.126 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-20T18:26:04.126 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-20T18:26:04.126 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-20T18:26:04.126 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-20T18:26:04.126 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-20T18:26:04.127 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-20T18:26:04.127 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-20T18:26:04.127 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-20T18:26:04.127 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-20T18:26:04.127 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-20T18:26:04.127 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-20T18:26:04.127 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-20T18:26:04.127 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-20T18:26:04.127 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-20T18:26:04.127 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-20T18:26:04.128 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-20T18:26:04.128 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-20T18:26:04.128 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-20T18:26:04.128 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-20T18:26:04.128 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-20T18:26:04.128 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-20T18:26:04.128 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-20T18:26:04.128 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-20T18:26:04.128 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-20T18:26:04.128 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-20T18:26:04.128 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-20T18:26:04.129 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-20T18:26:04.129 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-20T18:26:04.129 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-20T18:26:04.129 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-20T18:26:04.129 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-20T18:26:04.129 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-20T18:26:04.129 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-20T18:26:04.130 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-20T18:26:04.130 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-20T18:26:04.130 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-20T18:26:04.130 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-20T18:26:04.130 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-20T18:26:04.130 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-20T18:26:04.130 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-20T18:26:04.130 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-20T18:26:04.130 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-20T18:26:04.130 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-20T18:26:04.131 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-20T18:26:04.131 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-20T18:26:04.131 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-20T18:26:04.131 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-20T18:26:04.131 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-20T18:26:04.131 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-20T18:26:04.131 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-20T18:26:04.131 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-20T18:26:04.131 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-20T18:26:04.131 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-20T18:26:04.131 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-20T18:26:04.131 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-20T18:26:04.131 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-20T18:26:04.131 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-20T18:26:04.131 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-20T18:26:04.132 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-20T18:26:04.132 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-20T18:26:04.132 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-20T18:26:04.132 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-20T18:26:04.132 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-20T18:26:04.132 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-20T18:26:04.132 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-20T18:26:04.132 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-20T18:26:04.132 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-20T18:26:04.132 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-20T18:26:04.132 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-20T18:26:04.132 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-20T18:26:04.132 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-20T18:26:04.133 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-20T18:26:04.133 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-20T18:26:04.133 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-20T18:26:04.133 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-20T18:26:04.133 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-20T18:26:04.133 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-20T18:26:04.133 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-20T18:26:04.133 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-20T18:26:04.133 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-20T18:26:04.133 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-20T18:26:04.133 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-20T18:26:04.133 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 149/170 2024-05-20T18:26:04.134 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-20T18:26:04.134 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 151/170 2024-05-20T18:26:04.134 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-20T18:26:04.134 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-20T18:26:04.134 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-20T18:26:04.134 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-20T18:26:04.134 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-20T18:26:04.134 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-20T18:26:04.134 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-20T18:26:04.134 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-20T18:26:04.134 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-20T18:26:04.135 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-20T18:26:04.135 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-20T18:26:04.135 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-20T18:26:04.135 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-20T18:26:04.135 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-20T18:26:04.135 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-20T18:26:04.135 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-20T18:26:04.135 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-20T18:26:04.135 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-20T18:26:05.378 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-20T18:26:05.378 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 1/170 2024-05-20T18:26:05.378 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 2/170 2024-05-20T18:26:05.379 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 3/170 2024-05-20T18:26:05.379 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 4/170 2024-05-20T18:26:05.379 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2744.gd2b50 5/170 2024-05-20T18:26:05.379 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 6/170 2024-05-20T18:26:05.379 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 7/170 2024-05-20T18:26:05.379 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 8/170 2024-05-20T18:26:05.379 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 9/170 2024-05-20T18:26:05.379 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 10/170 2024-05-20T18:26:05.379 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 11/170 2024-05-20T18:26:05.379 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 12/170 2024-05-20T18:26:05.379 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 13/170 2024-05-20T18:26:05.379 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 14/170 2024-05-20T18:26:05.379 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_ 15/170 2024-05-20T18:26:05.380 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 16/170 2024-05-20T18:26:05.380 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 17/170 2024-05-20T18:26:05.380 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_6 18/170 2024-05-20T18:26:05.380 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86 19/170 2024-05-20T18:26:05.380 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 20/170 2024-05-20T18:26:05.380 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el 21/170 2024-05-20T18:26:05.380 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9. 22/170 2024-05-20T18:26:05.380 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_6 23/170 2024-05-20T18:26:05.380 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 24/170 2024-05-20T18:26:05.380 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 25/170 2024-05-20T18:26:05.380 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 26/170 2024-05-20T18:26:05.380 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 27/170 2024-05-20T18:26:05.393 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 28/170 2024-05-20T18:26:05.393 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 29/170 2024-05-20T18:26:05.393 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7. 30/170 2024-05-20T18:26:05.393 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noa 31/170 2024-05-20T18:26:05.393 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.n 32/170 2024-05-20T18:26:05.393 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b 33/170 2024-05-20T18:26:05.393 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el 34/170 2024-05-20T18:26:05.393 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 35/170 2024-05-20T18:26:05.393 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.e 36/170 2024-05-20T18:26:05.393 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 37/170 2024-05-20T18:26:05.393 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 38/170 2024-05-20T18:26:05.393 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 39/170 2024-05-20T18:26:05.394 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-20T18:26:05.394 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-20T18:26:05.394 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-20T18:26:05.394 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-20T18:26:05.394 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-20T18:26:05.394 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-20T18:26:05.394 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-20T18:26:05.394 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-20T18:26:05.394 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-20T18:26:05.394 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-20T18:26:05.394 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-20T18:26:05.394 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-20T18:26:05.394 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-20T18:26:05.395 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-20T18:26:05.395 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-20T18:26:05.395 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-20T18:26:05.395 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-20T18:26:05.395 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-20T18:26:05.395 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-20T18:26:05.395 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-20T18:26:05.395 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-20T18:26:05.395 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-20T18:26:05.395 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-20T18:26:05.395 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-20T18:26:05.395 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-20T18:26:05.395 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-20T18:26:05.396 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-20T18:26:05.396 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-20T18:26:05.396 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-20T18:26:05.396 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-20T18:26:05.396 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-20T18:26:05.396 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-20T18:26:05.396 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-20T18:26:05.396 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-20T18:26:05.396 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-20T18:26:05.396 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-20T18:26:05.396 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-20T18:26:05.396 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-20T18:26:05.396 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-20T18:26:05.397 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-20T18:26:05.397 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-20T18:26:05.397 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-20T18:26:05.397 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-20T18:26:05.397 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-20T18:26:05.397 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-20T18:26:05.397 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-20T18:26:05.397 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-20T18:26:05.397 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-20T18:26:05.397 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-20T18:26:05.397 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-20T18:26:05.397 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-20T18:26:05.397 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-20T18:26:05.398 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-20T18:26:05.398 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-20T18:26:05.398 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-20T18:26:05.398 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-20T18:26:05.398 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-20T18:26:05.398 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-20T18:26:05.398 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-20T18:26:05.398 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-20T18:26:05.398 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-20T18:26:05.398 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-20T18:26:05.398 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-20T18:26:05.398 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-20T18:26:05.398 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-20T18:26:05.399 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-20T18:26:05.399 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-20T18:26:05.399 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-20T18:26:05.399 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-20T18:26:05.399 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-20T18:26:05.399 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-20T18:26:05.399 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-20T18:26:05.399 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-20T18:26:05.399 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-20T18:26:05.399 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-20T18:26:05.399 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-20T18:26:05.399 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-20T18:26:05.399 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-20T18:26:05.400 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-20T18:26:05.400 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-20T18:26:05.400 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-20T18:26:05.400 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-20T18:26:05.400 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-20T18:26:05.400 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-20T18:26:05.400 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-20T18:26:05.400 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-20T18:26:05.400 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-20T18:26:05.400 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-20T18:26:05.400 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-20T18:26:05.400 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-20T18:26:05.400 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-20T18:26:05.401 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-20T18:26:05.401 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-20T18:26:05.401 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-20T18:26:05.401 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-20T18:26:05.401 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-20T18:26:05.401 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-20T18:26:05.401 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-20T18:26:05.401 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-20T18:26:05.401 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-20T18:26:05.401 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-20T18:26:05.401 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-20T18:26:05.401 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-20T18:26:05.401 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-20T18:26:05.402 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-20T18:26:05.402 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-20T18:26:05.402 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-20T18:26:05.402 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-20T18:26:05.402 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-20T18:26:05.402 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 149/170 2024-05-20T18:26:05.402 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-20T18:26:05.402 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 151/170 2024-05-20T18:26:05.402 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-20T18:26:05.402 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-20T18:26:05.403 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-20T18:26:05.403 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-20T18:26:05.403 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-20T18:26:05.403 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-20T18:26:05.403 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-20T18:26:05.403 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-20T18:26:05.403 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-20T18:26:05.403 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-20T18:26:05.403 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-20T18:26:05.403 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-20T18:26:05.403 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-20T18:26:05.403 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-20T18:26:05.403 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-20T18:26:05.403 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-20T18:26:05.404 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-20T18:26:05.404 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-20T18:26:06.934 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-20T18:26:06.934 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:06.934 INFO:teuthology.orchestra.run.smithi005.stdout:Upgraded: 2024-05-20T18:26:06.934 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-20T18:26:06.934 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-20T18:26:06.934 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-20T18:26:06.934 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-20T18:26:06.934 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-20T18:26:06.934 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-20T18:26:06.934 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-20T18:26:06.934 INFO:teuthology.orchestra.run.smithi005.stdout: librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.935 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.935 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-20T18:26:06.935 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-20T18:26:06.935 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-05-20T18:26:06.935 INFO:teuthology.orchestra.run.smithi005.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-20T18:26:06.935 INFO:teuthology.orchestra.run.smithi005.stdout: automake-1.16.2-8.el9.noarch 2024-05-20T18:26:06.935 INFO:teuthology.orchestra.run.smithi005.stdout: bison-3.7.4-5.el9.x86_64 2024-05-20T18:26:06.935 INFO:teuthology.orchestra.run.smithi005.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-20T18:26:06.935 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.935 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.935 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.935 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.935 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T18:26:06.935 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.936 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.936 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.936 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T18:26:06.936 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T18:26:06.936 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T18:26:06.936 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T18:26:06.936 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T18:26:06.936 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.936 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.936 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T18:26:06.936 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.936 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.936 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.937 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T18:26:06.937 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T18:26:06.937 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.937 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.937 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T18:26:06.937 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-20T18:26:06.937 INFO:teuthology.orchestra.run.smithi005.stdout: flex-2.6.4-9.el9.x86_64 2024-05-20T18:26:06.937 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-20T18:26:06.937 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-20T18:26:06.937 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-20T18:26:06.937 INFO:teuthology.orchestra.run.smithi005.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-20T18:26:06.937 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-20T18:26:06.937 INFO:teuthology.orchestra.run.smithi005.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-20T18:26:06.938 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-20T18:26:06.938 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-20T18:26:06.938 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-20T18:26:06.938 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.938 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.938 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.938 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-20T18:26:06.938 INFO:teuthology.orchestra.run.smithi005.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-20T18:26:06.938 INFO:teuthology.orchestra.run.smithi005.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-20T18:26:06.938 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-20T18:26:06.938 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-20T18:26:06.938 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.938 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.939 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-20T18:26:06.939 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.939 INFO:teuthology.orchestra.run.smithi005.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-20T18:26:06.939 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-20T18:26:06.939 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-20T18:26:06.939 INFO:teuthology.orchestra.run.smithi005.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-20T18:26:06.939 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-20T18:26:06.939 INFO:teuthology.orchestra.run.smithi005.stdout: lua-5.4.4-4.el9.x86_64 2024-05-20T18:26:06.939 INFO:teuthology.orchestra.run.smithi005.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-20T18:26:06.939 INFO:teuthology.orchestra.run.smithi005.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-20T18:26:06.939 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-20T18:26:06.939 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-20T18:26:06.939 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-20T18:26:06.940 INFO:teuthology.orchestra.run.smithi005.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-20T18:26:06.940 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-13.14-1.el9.x86_64 2024-05-20T18:26:06.940 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-20T18:26:06.940 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-20T18:26:06.940 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-20T18:26:06.940 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-20T18:26:06.940 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-20T18:26:06.940 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-20T18:26:06.940 INFO:teuthology.orchestra.run.smithi005.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-20T18:26:06.940 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-20T18:26:06.940 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-20T18:26:06.940 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-20T18:26:06.940 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.940 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.941 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.941 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-20T18:26:06.941 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-20T18:26:06.941 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-20T18:26:06.941 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-20T18:26:06.941 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-20T18:26:06.941 INFO:teuthology.orchestra.run.smithi005.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-20T18:26:06.941 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-20T18:26:06.941 INFO:teuthology.orchestra.run.smithi005.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-20T18:26:06.941 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-20T18:26:06.941 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-20T18:26:06.941 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-20T18:26:06.941 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-20T18:26:06.942 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-20T18:26:06.942 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-20T18:26:06.942 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-20T18:26:06.942 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-20T18:26:06.942 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-20T18:26:06.942 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-20T18:26:06.942 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-20T18:26:06.942 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-20T18:26:06.942 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-20T18:26:06.942 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-20T18:26:06.942 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-20T18:26:06.942 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-20T18:26:06.942 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-20T18:26:06.942 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-20T18:26:06.943 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-20T18:26:06.943 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-20T18:26:06.943 INFO:teuthology.orchestra.run.smithi005.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-20T18:26:06.943 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-20T18:26:06.943 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ply-3.11-14.el9.noarch 2024-05-20T18:26:06.943 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-20T18:26:06.943 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-20T18:26:06.943 INFO:teuthology.orchestra.run.smithi005.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-20T18:26:06.943 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-20T18:26:06.943 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-20T18:26:06.943 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-20T18:26:06.943 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-20T18:26:06.943 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-20T18:26:06.943 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-20T18:26:06.944 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.944 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.944 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-20T18:26:06.944 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-20T18:26:06.944 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-20T18:26:06.944 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.944 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-20T18:26:06.944 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-20T18:26:06.944 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-20T18:26:06.944 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-20T18:26:06.944 INFO:teuthology.orchestra.run.smithi005.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-20T18:26:06.944 INFO:teuthology.orchestra.run.smithi005.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-20T18:26:06.944 INFO:teuthology.orchestra.run.smithi005.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-20T18:26:06.945 INFO:teuthology.orchestra.run.smithi005.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-20T18:26:06.945 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-20T18:26:06.945 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-20T18:26:06.945 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-20T18:26:06.945 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-20T18:26:06.945 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-20T18:26:06.945 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-20T18:26:06.945 INFO:teuthology.orchestra.run.smithi005.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-20T18:26:06.945 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.945 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.945 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:06.945 INFO:teuthology.orchestra.run.smithi005.stdout: re2-1:20211101-3.el9.x86_64 2024-05-20T18:26:06.945 INFO:teuthology.orchestra.run.smithi005.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-20T18:26:06.945 INFO:teuthology.orchestra.run.smithi005.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-20T18:26:06.946 INFO:teuthology.orchestra.run.smithi005.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-20T18:26:06.946 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-20T18:26:06.946 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-20T18:26:06.946 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-20T18:26:06.946 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:06.947 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-20T18:26:07.387 DEBUG:teuthology.parallel:result is None 2024-05-20T18:26:08.186 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-20T18:26:08.187 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:08.187 INFO:teuthology.orchestra.run.smithi063.stdout:Upgraded: 2024-05-20T18:26:08.187 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-20T18:26:08.187 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-20T18:26:08.187 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-20T18:26:08.187 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-20T18:26:08.187 INFO:teuthology.orchestra.run.smithi063.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-20T18:26:08.187 INFO:teuthology.orchestra.run.smithi063.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-20T18:26:08.187 INFO:teuthology.orchestra.run.smithi063.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-20T18:26:08.187 INFO:teuthology.orchestra.run.smithi063.stdout: librados2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.187 INFO:teuthology.orchestra.run.smithi063.stdout: librbd1-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.188 INFO:teuthology.orchestra.run.smithi063.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-20T18:26:08.188 INFO:teuthology.orchestra.run.smithi063.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-20T18:26:08.188 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-05-20T18:26:08.188 INFO:teuthology.orchestra.run.smithi063.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-20T18:26:08.188 INFO:teuthology.orchestra.run.smithi063.stdout: automake-1.16.2-8.el9.noarch 2024-05-20T18:26:08.188 INFO:teuthology.orchestra.run.smithi063.stdout: bison-3.7.4-5.el9.x86_64 2024-05-20T18:26:08.188 INFO:teuthology.orchestra.run.smithi063.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-20T18:26:08.188 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.188 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-base-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.188 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.188 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.188 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-grafana-dashboards-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T18:26:08.188 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-immutable-object-cache-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.189 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mds-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.189 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.189 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T18:26:08.189 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-dashboard-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T18:26:08.189 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T18:26:08.189 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-modules-core-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T18:26:08.189 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-rook-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T18:26:08.189 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mon-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.189 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-osd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.189 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-prometheus-alerts-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T18:26:08.189 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-radosgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.189 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-selinux-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.189 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-test-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.190 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-volume-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T18:26:08.190 INFO:teuthology.orchestra.run.smithi063.stdout: cephadm-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T18:26:08.190 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.190 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-shell-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.190 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-top-2:19.0.0-2744.gd2b507c7.el9.noarch 2024-05-20T18:26:08.190 INFO:teuthology.orchestra.run.smithi063.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-20T18:26:08.190 INFO:teuthology.orchestra.run.smithi063.stdout: flex-2.6.4-9.el9.x86_64 2024-05-20T18:26:08.190 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-20T18:26:08.190 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-20T18:26:08.190 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-20T18:26:08.190 INFO:teuthology.orchestra.run.smithi063.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-20T18:26:08.190 INFO:teuthology.orchestra.run.smithi063.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-20T18:26:08.191 INFO:teuthology.orchestra.run.smithi063.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-20T18:26:08.191 INFO:teuthology.orchestra.run.smithi063.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-20T18:26:08.191 INFO:teuthology.orchestra.run.smithi063.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-20T18:26:08.191 INFO:teuthology.orchestra.run.smithi063.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-20T18:26:08.191 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs-devel-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.191 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.191 INFO:teuthology.orchestra.run.smithi063.stdout: libcephsqlite-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.191 INFO:teuthology.orchestra.run.smithi063.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-20T18:26:08.191 INFO:teuthology.orchestra.run.smithi063.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-20T18:26:08.191 INFO:teuthology.orchestra.run.smithi063.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-20T18:26:08.191 INFO:teuthology.orchestra.run.smithi063.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-20T18:26:08.191 INFO:teuthology.orchestra.run.smithi063.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-20T18:26:08.191 INFO:teuthology.orchestra.run.smithi063.stdout: librados-devel-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.191 INFO:teuthology.orchestra.run.smithi063.stdout: libradosstriper1-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.192 INFO:teuthology.orchestra.run.smithi063.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-20T18:26:08.192 INFO:teuthology.orchestra.run.smithi063.stdout: librgw2-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.192 INFO:teuthology.orchestra.run.smithi063.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-20T18:26:08.192 INFO:teuthology.orchestra.run.smithi063.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-20T18:26:08.192 INFO:teuthology.orchestra.run.smithi063.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-20T18:26:08.192 INFO:teuthology.orchestra.run.smithi063.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-20T18:26:08.192 INFO:teuthology.orchestra.run.smithi063.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-20T18:26:08.192 INFO:teuthology.orchestra.run.smithi063.stdout: lua-5.4.4-4.el9.x86_64 2024-05-20T18:26:08.192 INFO:teuthology.orchestra.run.smithi063.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-20T18:26:08.192 INFO:teuthology.orchestra.run.smithi063.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-20T18:26:08.192 INFO:teuthology.orchestra.run.smithi063.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-20T18:26:08.192 INFO:teuthology.orchestra.run.smithi063.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-20T18:26:08.193 INFO:teuthology.orchestra.run.smithi063.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-20T18:26:08.193 INFO:teuthology.orchestra.run.smithi063.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-20T18:26:08.193 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-13.14-1.el9.x86_64 2024-05-20T18:26:08.193 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-20T18:26:08.193 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-20T18:26:08.193 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-20T18:26:08.193 INFO:teuthology.orchestra.run.smithi063.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-20T18:26:08.193 INFO:teuthology.orchestra.run.smithi063.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-20T18:26:08.193 INFO:teuthology.orchestra.run.smithi063.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-20T18:26:08.193 INFO:teuthology.orchestra.run.smithi063.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-20T18:26:08.193 INFO:teuthology.orchestra.run.smithi063.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-20T18:26:08.193 INFO:teuthology.orchestra.run.smithi063.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-20T18:26:08.193 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-20T18:26:08.193 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-argparse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.194 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-common-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.194 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cephfs-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.194 INFO:teuthology.orchestra.run.smithi063.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-20T18:26:08.194 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-20T18:26:08.194 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-20T18:26:08.194 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-20T18:26:08.194 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-20T18:26:08.194 INFO:teuthology.orchestra.run.smithi063.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-20T18:26:08.194 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-20T18:26:08.194 INFO:teuthology.orchestra.run.smithi063.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-20T18:26:08.194 INFO:teuthology.orchestra.run.smithi063.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-20T18:26:08.194 INFO:teuthology.orchestra.run.smithi063.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-20T18:26:08.194 INFO:teuthology.orchestra.run.smithi063.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-20T18:26:08.195 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-20T18:26:08.195 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-20T18:26:08.195 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-20T18:26:08.195 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-20T18:26:08.195 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-20T18:26:08.195 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-20T18:26:08.195 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-20T18:26:08.195 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-20T18:26:08.195 INFO:teuthology.orchestra.run.smithi063.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-20T18:26:08.195 INFO:teuthology.orchestra.run.smithi063.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-20T18:26:08.195 INFO:teuthology.orchestra.run.smithi063.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-20T18:26:08.195 INFO:teuthology.orchestra.run.smithi063.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-20T18:26:08.195 INFO:teuthology.orchestra.run.smithi063.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-20T18:26:08.195 INFO:teuthology.orchestra.run.smithi063.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-20T18:26:08.196 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-20T18:26:08.196 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-20T18:26:08.196 INFO:teuthology.orchestra.run.smithi063.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-20T18:26:08.196 INFO:teuthology.orchestra.run.smithi063.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-20T18:26:08.196 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-20T18:26:08.196 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ply-3.11-14.el9.noarch 2024-05-20T18:26:08.196 INFO:teuthology.orchestra.run.smithi063.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-20T18:26:08.196 INFO:teuthology.orchestra.run.smithi063.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-20T18:26:08.196 INFO:teuthology.orchestra.run.smithi063.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-20T18:26:08.196 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-20T18:26:08.196 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-20T18:26:08.196 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-20T18:26:08.196 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-20T18:26:08.196 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-20T18:26:08.196 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-20T18:26:08.197 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rados-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.197 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.197 INFO:teuthology.orchestra.run.smithi063.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-20T18:26:08.197 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-20T18:26:08.197 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-20T18:26:08.197 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rgw-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.197 INFO:teuthology.orchestra.run.smithi063.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-20T18:26:08.197 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-20T18:26:08.197 INFO:teuthology.orchestra.run.smithi063.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-20T18:26:08.197 INFO:teuthology.orchestra.run.smithi063.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-20T18:26:08.197 INFO:teuthology.orchestra.run.smithi063.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-20T18:26:08.197 INFO:teuthology.orchestra.run.smithi063.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-20T18:26:08.197 INFO:teuthology.orchestra.run.smithi063.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-20T18:26:08.197 INFO:teuthology.orchestra.run.smithi063.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-20T18:26:08.197 INFO:teuthology.orchestra.run.smithi063.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-20T18:26:08.198 INFO:teuthology.orchestra.run.smithi063.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-20T18:26:08.198 INFO:teuthology.orchestra.run.smithi063.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-20T18:26:08.198 INFO:teuthology.orchestra.run.smithi063.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-20T18:26:08.198 INFO:teuthology.orchestra.run.smithi063.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-20T18:26:08.198 INFO:teuthology.orchestra.run.smithi063.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-20T18:26:08.198 INFO:teuthology.orchestra.run.smithi063.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-20T18:26:08.198 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-fuse-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.198 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-mirror-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.198 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-nbd-2:19.0.0-2744.gd2b507c7.el9.x86_64 2024-05-20T18:26:08.198 INFO:teuthology.orchestra.run.smithi063.stdout: re2-1:20211101-3.el9.x86_64 2024-05-20T18:26:08.198 INFO:teuthology.orchestra.run.smithi063.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-20T18:26:08.198 INFO:teuthology.orchestra.run.smithi063.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-20T18:26:08.198 INFO:teuthology.orchestra.run.smithi063.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-20T18:26:08.199 INFO:teuthology.orchestra.run.smithi063.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-20T18:26:08.199 INFO:teuthology.orchestra.run.smithi063.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-20T18:26:08.199 INFO:teuthology.orchestra.run.smithi063.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-20T18:26:08.199 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:08.199 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-20T18:26:08.630 DEBUG:teuthology.parallel:result is None 2024-05-20T18:26:08.630 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-20T18:26:08.851 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-20T18:26:08.877 INFO:teuthology.orchestra.run.smithi005.stdout:19.0.0-2744.gd2b507c7.el9 2024-05-20T18:26:08.878 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2744.gd2b507c7.el9 2024-05-20T18:26:08.878 INFO:teuthology.task.install:The correct ceph version 19.0.0-2744.gd2b507c7 is installed. 2024-05-20T18:26:08.880 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2b507c724ef6c8295ee0a35d185cdab167fd61d 2024-05-20T18:26:09.103 DEBUG:teuthology.orchestra.run.smithi063:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-20T18:26:09.128 INFO:teuthology.orchestra.run.smithi063.stdout:19.0.0-2744.gd2b507c7.el9 2024-05-20T18:26:09.128 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2744.gd2b507c7.el9 2024-05-20T18:26:09.128 INFO:teuthology.task.install:The correct ceph version 19.0.0-2744.gd2b507c7 is installed. 2024-05-20T18:26:09.130 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-20T18:26:09.130 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:26:09.130 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-20T18:26:09.164 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:26:09.165 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-20T18:26:09.198 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-20T18:26:09.198 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:26:09.198 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/daemon-helper 2024-05-20T18:26:09.236 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-20T18:26:09.307 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:26:09.307 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/daemon-helper 2024-05-20T18:26:09.335 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-20T18:26:09.403 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-20T18:26:09.403 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:26:09.403 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-20T18:26:09.436 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-20T18:26:09.508 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:26:09.509 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-20T18:26:09.535 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-20T18:26:09.601 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-20T18:26:09.601 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:26:09.601 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/stdin-killer 2024-05-20T18:26:09.632 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-20T18:26:09.702 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:26:09.702 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/stdin-killer 2024-05-20T18:26:09.728 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-20T18:26:09.795 INFO:teuthology.run_tasks:Running task ceph... 2024-05-20T18:26:09.888 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-20T18:26:09.888 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 777 /var/log/ceph 2024-05-20T18:26:09.891 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 777 /var/log/ceph 2024-05-20T18:26:09.918 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-20T18:26:09.919 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-20T18:26:09.961 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-20T18:26:09.988 INFO:tasks.ceph:Creating extra log directories... 2024-05-20T18:26:09.988 DEBUG:teuthology.orchestra.run.smithi005:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-20T18:26:10.032 DEBUG:teuthology.orchestra.run.smithi063:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-20T18:26:10.066 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-20T18:26:10.066 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-20T18:26:10.067 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-20T18:26:10.067 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:26:10.067 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-20T18:26:10.118 DEBUG:teuthology.orchestra.run.smithi005:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-20T18:26:10.239 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-20T18:26:10.268 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:26:10.268 DEBUG:teuthology.orchestra.run.smithi063:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-20T18:26:10.286 DEBUG:teuthology.orchestra.run.smithi063:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-20T18:26:10.399 DEBUG:teuthology.orchestra.run.smithi063:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-20T18:26:10.433 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-20T18:26:10.433 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-20T18:26:10.433 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715720', 'branch': 'wip-pdonnell-testing-20240517.203521-squid-debug', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/forward-scrub}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7715720', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240517.203521-squid-debug'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240517.203521-squid-debug', 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 9749, 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d', 'sleep_before_teardown': 0, 'subset': '272/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240517.203521-squid-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_d2b507c724ef6c8295ee0a35d185cdab167fd61d/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi005.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEYJ/KpXyEA26PWVADXwMMimqimeT3Bq3hShJRCeTQQLVqa0N50+xKZu2wXtWzREszDQ/+HQLgmkXm5slBmr2CI=', 'smithi063.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN6azvHe1I0QflL4RLsotOKQOQoplKgmtCiKtucEhHWOG7H/yHO0FSKgXqkCUwB5KOb6GmG9dcgK6+HuPDks4PY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': 'd2b507c724ef6c8295ee0a35d185cdab167fd61d', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a', 'timestamp': '2024-05-20_15:44:59', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841'} 2024-05-20T18:26:10.434 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-20T18:26:10.437 DEBUG:teuthology.orchestra.run.smithi063:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-20T18:26:10.483 DEBUG:teuthology.orchestra.run.smithi005:> sudo install -d -m0777 -- /var/run/ceph 2024-05-20T18:26:10.496 DEBUG:teuthology.orchestra.run.smithi063:> sudo install -d -m0777 -- /var/run/ceph 2024-05-20T18:26:10.550 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:26:10.550 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/scratch_devs of=/dev/stdout 2024-05-20T18:26:10.582 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-20T18:26:10.583 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_1 2024-05-20T18:26:10.641 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-20T18:26:10.642 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T18:26:10.642 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-05-20T18:26:10.642 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T18:26:10.642 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T18:26:10.642 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-05-20 18:25:58.646598430 +0000 2024-05-20T18:26:10.643 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-05-20 18:23:08.943208444 +0000 2024-05-20T18:26:10.643 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-05-20 18:23:08.943208444 +0000 2024-05-20T18:26:10.643 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-05-20 18:23:08.943208444 +0000 2024-05-20T18:26:10.643 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-20T18:26:10.708 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-05-20T18:26:10.708 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-05-20T18:26:10.708 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000112461 s, 4.6 MB/s 2024-05-20T18:26:10.710 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-20T18:26:10.769 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_2 2024-05-20T18:26:10.827 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-20T18:26:10.827 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T18:26:10.827 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-20T18:26:10.827 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T18:26:10.828 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T18:26:10.828 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-05-20 18:25:58.646598430 +0000 2024-05-20T18:26:10.828 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-05-20 18:23:09.271202017 +0000 2024-05-20T18:26:10.828 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-05-20 18:23:09.271202017 +0000 2024-05-20T18:26:10.828 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-05-20 18:23:09.271202017 +0000 2024-05-20T18:26:10.828 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-20T18:26:10.895 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-05-20T18:26:10.895 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-05-20T18:26:10.895 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.00011603 s, 4.4 MB/s 2024-05-20T18:26:10.896 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-20T18:26:10.956 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_3 2024-05-20T18:26:11.014 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-20T18:26:11.014 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T18:26:11.015 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-20T18:26:11.015 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T18:26:11.015 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T18:26:11.015 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-05-20 18:25:58.646598430 +0000 2024-05-20T18:26:11.015 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-05-20 18:23:09.591195746 +0000 2024-05-20T18:26:11.015 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-05-20 18:23:09.591195746 +0000 2024-05-20T18:26:11.015 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-05-20 18:23:09.591195746 +0000 2024-05-20T18:26:11.015 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-20T18:26:11.082 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-05-20T18:26:11.082 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-05-20T18:26:11.082 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000119919 s, 4.3 MB/s 2024-05-20T18:26:11.084 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-20T18:26:11.144 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_4 2024-05-20T18:26:11.204 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-20T18:26:11.204 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T18:26:11.204 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-20T18:26:11.204 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T18:26:11.204 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T18:26:11.204 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-05-20 18:25:58.647598410 +0000 2024-05-20T18:26:11.204 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-05-20 18:23:09.913189436 +0000 2024-05-20T18:26:11.204 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-05-20 18:23:09.913189436 +0000 2024-05-20T18:26:11.205 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-05-20 18:23:09.913189436 +0000 2024-05-20T18:26:11.205 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-20T18:26:11.274 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-05-20T18:26:11.274 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-05-20T18:26:11.274 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000105331 s, 4.9 MB/s 2024-05-20T18:26:11.275 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-20T18:26:11.335 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-20T18:26:11.336 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:26:11.336 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/scratch_devs of=/dev/stdout 2024-05-20T18:26:11.355 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-20T18:26:11.355 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_1 2024-05-20T18:26:11.413 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-20T18:26:11.413 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T18:26:11.413 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-20T18:26:11.414 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T18:26:11.414 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T18:26:11.414 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-05-20 18:26:00.219645157 +0000 2024-05-20T18:26:11.414 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-05-20 18:23:16.502871427 +0000 2024-05-20T18:26:11.414 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-05-20 18:23:16.502871427 +0000 2024-05-20T18:26:11.414 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-05-20 18:23:16.502871427 +0000 2024-05-20T18:26:11.414 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-20T18:26:11.480 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-05-20T18:26:11.480 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-05-20T18:26:11.480 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000275994 s, 1.9 MB/s 2024-05-20T18:26:11.482 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-20T18:26:11.541 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_2 2024-05-20T18:26:11.599 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-20T18:26:11.599 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T18:26:11.599 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-20T18:26:11.599 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T18:26:11.599 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T18:26:11.599 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-05-20 18:26:00.220645148 +0000 2024-05-20T18:26:11.599 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-05-20 18:23:16.839868307 +0000 2024-05-20T18:26:11.599 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-05-20 18:23:16.839868307 +0000 2024-05-20T18:26:11.599 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-05-20 18:23:16.839868307 +0000 2024-05-20T18:26:11.599 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-20T18:26:11.664 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-05-20T18:26:11.664 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-05-20T18:26:11.664 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000347545 s, 1.5 MB/s 2024-05-20T18:26:11.666 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-20T18:26:11.725 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_3 2024-05-20T18:26:11.782 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-20T18:26:11.782 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T18:26:11.782 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-20T18:26:11.782 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T18:26:11.782 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T18:26:11.782 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-05-20 18:26:00.220645148 +0000 2024-05-20T18:26:11.783 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-05-20 18:23:17.190865059 +0000 2024-05-20T18:26:11.783 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-05-20 18:23:17.190865059 +0000 2024-05-20T18:26:11.783 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-05-20 18:23:17.190865059 +0000 2024-05-20T18:26:11.783 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-20T18:26:11.847 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-05-20T18:26:11.847 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-05-20T18:26:11.847 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000179255 s, 2.9 MB/s 2024-05-20T18:26:11.849 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-20T18:26:11.905 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_4 2024-05-20T18:26:11.961 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-20T18:26:11.962 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-20T18:26:11.962 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-20T18:26:11.962 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-20T18:26:11.962 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-05-20T18:26:11.962 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-05-20 18:26:00.220645148 +0000 2024-05-20T18:26:11.962 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-05-20 18:23:17.532861893 +0000 2024-05-20T18:26:11.962 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-05-20 18:23:17.532861893 +0000 2024-05-20T18:26:11.962 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-05-20 18:23:17.532861893 +0000 2024-05-20T18:26:11.962 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-20T18:26:12.027 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-05-20T18:26:12.028 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-05-20T18:26:12.028 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000294472 s, 1.7 MB/s 2024-05-20T18:26:12.029 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-20T18:26:12.087 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-20T18:26:12.087 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi005.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi063.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-20T18:26:12.087 INFO:tasks.ceph:Generating config... 2024-05-20T18:26:12.088 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-20T18:26:12.088 INFO:tasks.ceph:[client] debug client = 20 2024-05-20T18:26:12.088 INFO:tasks.ceph:[client] debug ms = 1 2024-05-20T18:26:12.089 INFO:tasks.ceph:[client] pid file = 2024-05-20T18:26:12.089 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-20T18:26:12.089 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-20T18:26:12.089 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-05-20T18:26:12.089 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-20T18:26:12.089 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-20T18:26:12.089 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-20T18:26:12.089 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-20T18:26:12.089 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-20T18:26:12.089 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-20T18:26:12.089 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-20T18:26:12.089 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-20T18:26:12.089 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-20T18:26:12.089 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-20T18:26:12.090 INFO:tasks.ceph:[mgr] debug client = 20 2024-05-20T18:26:12.090 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-20T18:26:12.090 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-20T18:26:12.090 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-20T18:26:12.090 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-20T18:26:12.090 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-20T18:26:12.090 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-05-20T18:26:12.090 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-20T18:26:12.090 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-20T18:26:12.090 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-20T18:26:12.090 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-05-20T18:26:12.090 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-05-20T18:26:12.090 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-20T18:26:12.091 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-20T18:26:12.091 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-05-20T18:26:12.091 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-20T18:26:12.091 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-20T18:26:12.091 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-20T18:26:12.091 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-20T18:26:12.091 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-20T18:26:12.091 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-20T18:26:12.091 INFO:tasks.ceph:Setting up mon.a... 2024-05-20T18:26:12.091 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-20T18:26:12.150 INFO:teuthology.orchestra.run.smithi005.stdout:creating /etc/ceph/ceph.keyring 2024-05-20T18:26:12.153 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-20T18:26:12.206 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-20T18:26:12.232 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.5'), ('mon.b', '172.21.15.63'), ('mon.c', '[v2:172.21.15.63:3301,v1:172.21.15.63:6790]')] 2024-05-20T18:26:12.233 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.5,172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-20T18:26:12.234 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:26:12.234 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-20T18:26:12.289 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.5 --add b 172.21.15.63 --addv c '[v2:172.21.15.63:3301,v1:172.21.15.63:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-20T18:26:12.386 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-20T18:26:12.387 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool: generated fsid 399ab250-fd3c-47fa-a1c8-25e8149c9cac 2024-05-20T18:26:12.387 INFO:teuthology.orchestra.run.smithi005.stdout:setting min_mon_release = quincy 2024-05-20T18:26:12.387 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 0 2024-05-20T18:26:12.387 INFO:teuthology.orchestra.run.smithi005.stdout:fsid 399ab250-fd3c-47fa-a1c8-25e8149c9cac 2024-05-20T18:26:12.387 INFO:teuthology.orchestra.run.smithi005.stdout:last_changed 2024-05-20T18:26:12.389218+0000 2024-05-20T18:26:12.387 INFO:teuthology.orchestra.run.smithi005.stdout:created 2024-05-20T18:26:12.389218+0000 2024-05-20T18:26:12.387 INFO:teuthology.orchestra.run.smithi005.stdout:min_mon_release 17 (quincy) 2024-05-20T18:26:12.387 INFO:teuthology.orchestra.run.smithi005.stdout:election_strategy: 1 2024-05-20T18:26:12.387 INFO:teuthology.orchestra.run.smithi005.stdout:0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-05-20T18:26:12.387 INFO:teuthology.orchestra.run.smithi005.stdout:1: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.b 2024-05-20T18:26:12.387 INFO:teuthology.orchestra.run.smithi005.stdout:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-20T18:26:12.387 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-20T18:26:12.389 DEBUG:teuthology.orchestra.run.smithi005:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-20T18:26:12.404 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 399ab250-fd3c-47fa-a1c8-25e8149c9cac... 2024-05-20T18:26:12.406 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-20T18:26:12.447 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-20T18:26:12.489 INFO:teuthology.orchestra.run.smithi005.stdout:[global] 2024-05-20T18:26:12.489 INFO:teuthology.orchestra.run.smithi005.stdout: chdir = "" 2024-05-20T18:26:12.489 INFO:teuthology.orchestra.run.smithi005.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-20T18:26:12.490 INFO:teuthology.orchestra.run.smithi005.stdout: auth supported = cephx 2024-05-20T18:26:12.490 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:12.490 INFO:teuthology.orchestra.run.smithi005.stdout: filestore xattr use omap = true 2024-05-20T18:26:12.490 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:12.490 INFO:teuthology.orchestra.run.smithi005.stdout: mon clock drift allowed = 1.000 2024-05-20T18:26:12.490 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:12.490 INFO:teuthology.orchestra.run.smithi005.stdout: osd crush chooseleaf type = 0 2024-05-20T18:26:12.490 INFO:teuthology.orchestra.run.smithi005.stdout: auth debug = true 2024-05-20T18:26:12.490 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:12.490 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on old message = true 2024-05-20T18:26:12.490 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on bug = true 2024-05-20T18:26:12.491 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:12.491 INFO:teuthology.orchestra.run.smithi005.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-20T18:26:12.491 INFO:teuthology.orchestra.run.smithi005.stdout: mon pg warn max object skew = 0 2024-05-20T18:26:12.491 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:12.491 INFO:teuthology.orchestra.run.smithi005.stdout: # disable pg_autoscaler by default for new pools 2024-05-20T18:26:12.491 INFO:teuthology.orchestra.run.smithi005.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-20T18:26:12.491 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:12.491 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default size = 2 2024-05-20T18:26:12.491 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:12.491 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow primary affinity = true 2024-05-20T18:26:12.491 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow pg remap = true 2024-05-20T18:26:12.492 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on legacy crush tunables = false 2024-05-20T18:26:12.492 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on crush straw calc version zero = false 2024-05-20T18:26:12.492 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on no sortbitwise = false 2024-05-20T18:26:12.492 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on osd down out interval zero = false 2024-05-20T18:26:12.492 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on too few osds = false 2024-05-20T18:26:12.492 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-20T18:26:12.492 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-20T18:26:12.492 INFO:teuthology.orchestra.run.smithi005.stdout: mon_allow_pool_size_one = true 2024-05-20T18:26:12.492 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:12.492 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-20T18:26:12.492 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:12.492 INFO:teuthology.orchestra.run.smithi005.stdout: osd default data pool replay window = 5 2024-05-20T18:26:12.492 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:12.493 INFO:teuthology.orchestra.run.smithi005.stdout: mon allow pool delete = true 2024-05-20T18:26:12.493 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:12.493 INFO:teuthology.orchestra.run.smithi005.stdout: mon cluster log file level = debug 2024-05-20T18:26:12.493 INFO:teuthology.orchestra.run.smithi005.stdout: debug asserts on shutdown = true 2024-05-20T18:26:12.493 INFO:teuthology.orchestra.run.smithi005.stdout: mon health detail to clog = false 2024-05-20T18:26:12.493 INFO:teuthology.orchestra.run.smithi005.stdout: mon host = "172.21.15.5,172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790]" 2024-05-20T18:26:12.493 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on skipped message = False 2024-05-20T18:26:12.493 INFO:teuthology.orchestra.run.smithi005.stdout: fsid = 399ab250-fd3c-47fa-a1c8-25e8149c9cac 2024-05-20T18:26:12.493 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:12.493 INFO:teuthology.orchestra.run.smithi005.stdout:[osd] 2024-05-20T18:26:12.493 INFO:teuthology.orchestra.run.smithi005.stdout: osd journal size = 100 2024-05-20T18:26:12.493 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:12.493 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub load threshold = 5.0 2024-05-20T18:26:12.493 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub max interval = 600 2024-05-20T18:26:12.494 INFO:teuthology.orchestra.run.smithi005.stdout: osd mclock profile = high_recovery_ops 2024-05-20T18:26:12.494 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:12.494 INFO:teuthology.orchestra.run.smithi005.stdout: osd recover clone overlap = true 2024-05-20T18:26:12.494 INFO:teuthology.orchestra.run.smithi005.stdout: osd recovery max chunk = 1048576 2024-05-20T18:26:12.494 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:12.494 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug shutdown = true 2024-05-20T18:26:12.494 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug op order = true 2024-05-20T18:26:12.494 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug verify stray on activate = true 2024-05-20T18:26:12.494 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug trim objects = true 2024-05-20T18:26:12.494 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:12.494 INFO:teuthology.orchestra.run.smithi005.stdout: osd open classes on start = true 2024-05-20T18:26:12.494 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug pg log writeout = true 2024-05-20T18:26:12.494 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:12.495 INFO:teuthology.orchestra.run.smithi005.stdout: osd deep scrub update digest min age = 30 2024-05-20T18:26:12.495 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:12.495 INFO:teuthology.orchestra.run.smithi005.stdout: osd map max advance = 10 2024-05-20T18:26:12.495 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:12.495 INFO:teuthology.orchestra.run.smithi005.stdout: journal zero on create = true 2024-05-20T18:26:12.495 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:12.495 INFO:teuthology.orchestra.run.smithi005.stdout: filestore ondisk finisher threads = 3 2024-05-20T18:26:12.495 INFO:teuthology.orchestra.run.smithi005.stdout: filestore apply finisher threads = 3 2024-05-20T18:26:12.495 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:12.495 INFO:teuthology.orchestra.run.smithi005.stdout: bdev debug aio = true 2024-05-20T18:26:12.495 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug misdirected ops = true 2024-05-20T18:26:12.495 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore block size = 96636764160 2024-05-20T18:26:12.495 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore fsck on mount = True 2024-05-20T18:26:12.495 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluefs = 20 2024-05-20T18:26:12.495 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluestore = 20 2024-05-20T18:26:12.496 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-20T18:26:12.496 INFO:teuthology.orchestra.run.smithi005.stdout: debug osd = 20 2024-05-20T18:26:12.496 INFO:teuthology.orchestra.run.smithi005.stdout: debug rocksdb = 10 2024-05-20T18:26:12.496 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd backfillfull_ratio = 0.85 2024-05-20T18:26:12.496 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd full ratio = 0.9 2024-05-20T18:26:12.496 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd nearfull ratio = 0.8 2024-05-20T18:26:12.496 INFO:teuthology.orchestra.run.smithi005.stdout: osd failsafe full ratio = 0.95 2024-05-20T18:26:12.496 INFO:teuthology.orchestra.run.smithi005.stdout: osd objectstore = bluestore 2024-05-20T18:26:12.496 INFO:teuthology.orchestra.run.smithi005.stdout: osd op complaint time = 180 2024-05-20T18:26:12.496 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:12.496 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr] 2024-05-20T18:26:12.496 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-20T18:26:12.496 INFO:teuthology.orchestra.run.smithi005.stdout: debug mgr = 20 2024-05-20T18:26:12.496 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2024-05-20T18:26:12.497 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2024-05-20T18:26:12.497 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2024-05-20T18:26:12.497 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2024-05-20T18:26:12.497 INFO:teuthology.orchestra.run.smithi005.stdout: mgr/telemetry/nag = false 2024-05-20T18:26:12.497 INFO:teuthology.orchestra.run.smithi005.stdout: debug client = 20 2024-05-20T18:26:12.497 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:12.497 INFO:teuthology.orchestra.run.smithi005.stdout:[mon] 2024-05-20T18:26:12.497 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-20T18:26:12.497 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2024-05-20T18:26:12.497 INFO:teuthology.orchestra.run.smithi005.stdout: debug paxos = 20 2024-05-20T18:26:12.497 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2024-05-20T18:26:12.497 INFO:teuthology.orchestra.run.smithi005.stdout: mon data avail warn = 5 2024-05-20T18:26:12.497 INFO:teuthology.orchestra.run.smithi005.stdout: mon mgr mkfs grace = 240 2024-05-20T18:26:12.497 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2024-05-20T18:26:12.497 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd reporter subtree level = osd 2024-05-20T18:26:12.498 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd prime pg temp = true 2024-05-20T18:26:12.498 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2024-05-20T18:26:12.498 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:12.498 INFO:teuthology.orchestra.run.smithi005.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-20T18:26:12.498 INFO:teuthology.orchestra.run.smithi005.stdout: auth mon ticket ttl = 660 # 11m 2024-05-20T18:26:12.498 INFO:teuthology.orchestra.run.smithi005.stdout: auth service ticket ttl = 240 # 4m 2024-05-20T18:26:12.498 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:12.498 INFO:teuthology.orchestra.run.smithi005.stdout: # don't complain about insecure global_id in the test suite 2024-05-20T18:26:12.498 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-20T18:26:12.498 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-20T18:26:12.498 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:12.498 INFO:teuthology.orchestra.run.smithi005.stdout: # 1m isn't quite enough 2024-05-20T18:26:12.498 INFO:teuthology.orchestra.run.smithi005.stdout: mon_down_mkfs_grace = 2m 2024-05-20T18:26:12.499 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:12.499 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_filestore_osds = false 2024-05-20T18:26:12.499 INFO:teuthology.orchestra.run.smithi005.stdout: mon down mkfs grace = 300 2024-05-20T18:26:12.499 INFO:teuthology.orchestra.run.smithi005.stdout: mon op complaint time = 120 2024-05-20T18:26:12.499 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:12.499 INFO:teuthology.orchestra.run.smithi005.stdout:[client] 2024-05-20T18:26:12.499 INFO:teuthology.orchestra.run.smithi005.stdout: rgw cache enabled = true 2024-05-20T18:26:12.499 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable ops log = true 2024-05-20T18:26:12.499 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable usage log = true 2024-05-20T18:26:12.499 INFO:teuthology.orchestra.run.smithi005.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-20T18:26:12.499 INFO:teuthology.orchestra.run.smithi005.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-20T18:26:12.499 INFO:teuthology.orchestra.run.smithi005.stdout: client mount timeout = 600 2024-05-20T18:26:12.499 INFO:teuthology.orchestra.run.smithi005.stdout: debug client = 20 2024-05-20T18:26:12.499 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-20T18:26:12.500 INFO:teuthology.orchestra.run.smithi005.stdout: pid file = "" 2024-05-20T18:26:12.500 INFO:teuthology.orchestra.run.smithi005.stdout: rados mon op timeout = 900 2024-05-20T18:26:12.500 INFO:teuthology.orchestra.run.smithi005.stdout: rados osd op timeout = 900 2024-05-20T18:26:12.500 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.a] 2024-05-20T18:26:12.500 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.b] 2024-05-20T18:26:12.500 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.c] 2024-05-20T18:26:12.500 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a] 2024-05-20T18:26:12.500 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.c] 2024-05-20T18:26:12.500 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b] 2024-05-20T18:26:12.500 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.d] 2024-05-20T18:26:12.500 INFO:teuthology.orchestra.run.smithi005.stdout:[mds] 2024-05-20T18:26:12.500 INFO:teuthology.orchestra.run.smithi005.stdout: debug mds = 20 2024-05-20T18:26:12.500 INFO:teuthology.orchestra.run.smithi005.stdout: debug mds balancer = 20 2024-05-20T18:26:12.500 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-20T18:26:12.501 INFO:teuthology.orchestra.run.smithi005.stdout: mds debug frag = True 2024-05-20T18:26:12.501 INFO:teuthology.orchestra.run.smithi005.stdout: mds debug scatterstat = True 2024-05-20T18:26:12.501 INFO:teuthology.orchestra.run.smithi005.stdout: mds op complaint time = 180 2024-05-20T18:26:12.501 INFO:teuthology.orchestra.run.smithi005.stdout: mds verify scatter = True 2024-05-20T18:26:12.501 INFO:teuthology.orchestra.run.smithi005.stdout: osd op complaint time = 180 2024-05-20T18:26:12.501 INFO:teuthology.orchestra.run.smithi005.stdout: rados mon op timeout = 900 2024-05-20T18:26:12.501 INFO:teuthology.orchestra.run.smithi005.stdout: rados osd op timeout = 900 2024-05-20T18:26:12.501 INFO:teuthology.orchestra.run.smithi063.stdout:[global] 2024-05-20T18:26:12.501 INFO:teuthology.orchestra.run.smithi063.stdout: chdir = "" 2024-05-20T18:26:12.502 INFO:teuthology.orchestra.run.smithi063.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-20T18:26:12.502 INFO:teuthology.orchestra.run.smithi063.stdout: auth supported = cephx 2024-05-20T18:26:12.502 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:12.502 INFO:teuthology.orchestra.run.smithi063.stdout: filestore xattr use omap = true 2024-05-20T18:26:12.502 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:12.502 INFO:teuthology.orchestra.run.smithi063.stdout: mon clock drift allowed = 1.000 2024-05-20T18:26:12.502 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:12.502 INFO:teuthology.orchestra.run.smithi063.stdout: osd crush chooseleaf type = 0 2024-05-20T18:26:12.502 INFO:teuthology.orchestra.run.smithi063.stdout: auth debug = true 2024-05-20T18:26:12.502 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:12.503 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on old message = true 2024-05-20T18:26:12.503 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on bug = true 2024-05-20T18:26:12.503 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:12.503 INFO:teuthology.orchestra.run.smithi063.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-20T18:26:12.503 INFO:teuthology.orchestra.run.smithi063.stdout: mon pg warn max object skew = 0 2024-05-20T18:26:12.503 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:12.503 INFO:teuthology.orchestra.run.smithi063.stdout: # disable pg_autoscaler by default for new pools 2024-05-20T18:26:12.503 INFO:teuthology.orchestra.run.smithi063.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-20T18:26:12.503 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:12.503 INFO:teuthology.orchestra.run.smithi063.stdout: osd pool default size = 2 2024-05-20T18:26:12.503 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:12.503 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd allow primary affinity = true 2024-05-20T18:26:12.503 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd allow pg remap = true 2024-05-20T18:26:12.503 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on legacy crush tunables = false 2024-05-20T18:26:12.504 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on crush straw calc version zero = false 2024-05-20T18:26:12.504 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on no sortbitwise = false 2024-05-20T18:26:12.504 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on osd down out interval zero = false 2024-05-20T18:26:12.504 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on too few osds = false 2024-05-20T18:26:12.504 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-20T18:26:12.504 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-20T18:26:12.504 INFO:teuthology.orchestra.run.smithi063.stdout: mon_allow_pool_size_one = true 2024-05-20T18:26:12.504 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:12.504 INFO:teuthology.orchestra.run.smithi063.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-20T18:26:12.504 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:12.504 INFO:teuthology.orchestra.run.smithi063.stdout: osd default data pool replay window = 5 2024-05-20T18:26:12.504 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:12.504 INFO:teuthology.orchestra.run.smithi063.stdout: mon allow pool delete = true 2024-05-20T18:26:12.505 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:12.505 INFO:teuthology.orchestra.run.smithi063.stdout: mon cluster log file level = debug 2024-05-20T18:26:12.505 INFO:teuthology.orchestra.run.smithi063.stdout: debug asserts on shutdown = true 2024-05-20T18:26:12.505 INFO:teuthology.orchestra.run.smithi063.stdout: mon health detail to clog = false 2024-05-20T18:26:12.505 INFO:teuthology.orchestra.run.smithi063.stdout: mon host = "172.21.15.5,172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790]" 2024-05-20T18:26:12.505 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on skipped message = False 2024-05-20T18:26:12.505 INFO:teuthology.orchestra.run.smithi063.stdout: fsid = 399ab250-fd3c-47fa-a1c8-25e8149c9cac 2024-05-20T18:26:12.505 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:12.505 INFO:teuthology.orchestra.run.smithi063.stdout:[osd] 2024-05-20T18:26:12.505 INFO:teuthology.orchestra.run.smithi063.stdout: osd journal size = 100 2024-05-20T18:26:12.505 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:12.505 INFO:teuthology.orchestra.run.smithi063.stdout: osd scrub load threshold = 5.0 2024-05-20T18:26:12.505 INFO:teuthology.orchestra.run.smithi063.stdout: osd scrub max interval = 600 2024-05-20T18:26:12.505 INFO:teuthology.orchestra.run.smithi063.stdout: osd mclock profile = high_recovery_ops 2024-05-20T18:26:12.505 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:12.506 INFO:teuthology.orchestra.run.smithi063.stdout: osd recover clone overlap = true 2024-05-20T18:26:12.506 INFO:teuthology.orchestra.run.smithi063.stdout: osd recovery max chunk = 1048576 2024-05-20T18:26:12.506 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:12.506 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug shutdown = true 2024-05-20T18:26:12.506 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug op order = true 2024-05-20T18:26:12.506 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug verify stray on activate = true 2024-05-20T18:26:12.506 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug trim objects = true 2024-05-20T18:26:12.506 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:12.506 INFO:teuthology.orchestra.run.smithi063.stdout: osd open classes on start = true 2024-05-20T18:26:12.507 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug pg log writeout = true 2024-05-20T18:26:12.507 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:12.507 INFO:teuthology.orchestra.run.smithi063.stdout: osd deep scrub update digest min age = 30 2024-05-20T18:26:12.507 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:12.507 INFO:teuthology.orchestra.run.smithi063.stdout: osd map max advance = 10 2024-05-20T18:26:12.507 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:12.507 INFO:teuthology.orchestra.run.smithi063.stdout: journal zero on create = true 2024-05-20T18:26:12.507 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:12.507 INFO:teuthology.orchestra.run.smithi063.stdout: filestore ondisk finisher threads = 3 2024-05-20T18:26:12.507 INFO:teuthology.orchestra.run.smithi063.stdout: filestore apply finisher threads = 3 2024-05-20T18:26:12.507 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:12.507 INFO:teuthology.orchestra.run.smithi063.stdout: bdev debug aio = true 2024-05-20T18:26:12.507 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug misdirected ops = true 2024-05-20T18:26:12.508 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore block size = 96636764160 2024-05-20T18:26:12.508 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore fsck on mount = True 2024-05-20T18:26:12.508 INFO:teuthology.orchestra.run.smithi063.stdout: debug bluefs = 20 2024-05-20T18:26:12.508 INFO:teuthology.orchestra.run.smithi063.stdout: debug bluestore = 20 2024-05-20T18:26:12.508 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-20T18:26:12.508 INFO:teuthology.orchestra.run.smithi063.stdout: debug osd = 20 2024-05-20T18:26:12.508 INFO:teuthology.orchestra.run.smithi063.stdout: debug rocksdb = 10 2024-05-20T18:26:12.508 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd backfillfull_ratio = 0.85 2024-05-20T18:26:12.508 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd full ratio = 0.9 2024-05-20T18:26:12.508 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd nearfull ratio = 0.8 2024-05-20T18:26:12.508 INFO:teuthology.orchestra.run.smithi063.stdout: osd failsafe full ratio = 0.95 2024-05-20T18:26:12.508 INFO:teuthology.orchestra.run.smithi063.stdout: osd objectstore = bluestore 2024-05-20T18:26:12.508 INFO:teuthology.orchestra.run.smithi063.stdout: osd op complaint time = 180 2024-05-20T18:26:12.509 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:12.509 INFO:teuthology.orchestra.run.smithi063.stdout:[mgr] 2024-05-20T18:26:12.509 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-20T18:26:12.509 INFO:teuthology.orchestra.run.smithi063.stdout: debug mgr = 20 2024-05-20T18:26:12.509 INFO:teuthology.orchestra.run.smithi063.stdout: debug mon = 20 2024-05-20T18:26:12.509 INFO:teuthology.orchestra.run.smithi063.stdout: debug auth = 20 2024-05-20T18:26:12.509 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min pgs per osd = 4 2024-05-20T18:26:12.509 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min bytes per osd = 10 2024-05-20T18:26:12.509 INFO:teuthology.orchestra.run.smithi063.stdout: mgr/telemetry/nag = false 2024-05-20T18:26:12.509 INFO:teuthology.orchestra.run.smithi063.stdout: debug client = 20 2024-05-20T18:26:12.509 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:12.509 INFO:teuthology.orchestra.run.smithi063.stdout:[mon] 2024-05-20T18:26:12.509 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-20T18:26:12.509 INFO:teuthology.orchestra.run.smithi063.stdout: debug mon = 20 2024-05-20T18:26:12.510 INFO:teuthology.orchestra.run.smithi063.stdout: debug paxos = 20 2024-05-20T18:26:12.510 INFO:teuthology.orchestra.run.smithi063.stdout: debug auth = 20 2024-05-20T18:26:12.510 INFO:teuthology.orchestra.run.smithi063.stdout: mon data avail warn = 5 2024-05-20T18:26:12.510 INFO:teuthology.orchestra.run.smithi063.stdout: mon mgr mkfs grace = 240 2024-05-20T18:26:12.510 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min pgs per osd = 4 2024-05-20T18:26:12.510 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd reporter subtree level = osd 2024-05-20T18:26:12.510 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd prime pg temp = true 2024-05-20T18:26:12.510 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min bytes per osd = 10 2024-05-20T18:26:12.510 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:12.510 INFO:teuthology.orchestra.run.smithi063.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-20T18:26:12.511 INFO:teuthology.orchestra.run.smithi063.stdout: auth mon ticket ttl = 660 # 11m 2024-05-20T18:26:12.511 INFO:teuthology.orchestra.run.smithi063.stdout: auth service ticket ttl = 240 # 4m 2024-05-20T18:26:12.511 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:12.511 INFO:teuthology.orchestra.run.smithi063.stdout: # don't complain about insecure global_id in the test suite 2024-05-20T18:26:12.511 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-20T18:26:12.511 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-20T18:26:12.511 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:12.511 INFO:teuthology.orchestra.run.smithi063.stdout: # 1m isn't quite enough 2024-05-20T18:26:12.511 INFO:teuthology.orchestra.run.smithi063.stdout: mon_down_mkfs_grace = 2m 2024-05-20T18:26:12.511 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:12.511 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_filestore_osds = false 2024-05-20T18:26:12.511 INFO:teuthology.orchestra.run.smithi063.stdout: mon down mkfs grace = 300 2024-05-20T18:26:12.512 INFO:teuthology.orchestra.run.smithi063.stdout: mon op complaint time = 120 2024-05-20T18:26:12.512 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:26:12.512 INFO:teuthology.orchestra.run.smithi063.stdout:[client] 2024-05-20T18:26:12.512 INFO:teuthology.orchestra.run.smithi063.stdout: rgw cache enabled = true 2024-05-20T18:26:12.512 INFO:teuthology.orchestra.run.smithi063.stdout: rgw enable ops log = true 2024-05-20T18:26:12.512 INFO:teuthology.orchestra.run.smithi063.stdout: rgw enable usage log = true 2024-05-20T18:26:12.512 INFO:teuthology.orchestra.run.smithi063.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-20T18:26:12.512 INFO:teuthology.orchestra.run.smithi063.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-20T18:26:12.512 INFO:teuthology.orchestra.run.smithi063.stdout: client mount timeout = 600 2024-05-20T18:26:12.512 INFO:teuthology.orchestra.run.smithi063.stdout: debug client = 20 2024-05-20T18:26:12.512 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-20T18:26:12.512 INFO:teuthology.orchestra.run.smithi063.stdout: pid file = "" 2024-05-20T18:26:12.513 INFO:teuthology.orchestra.run.smithi063.stdout: rados mon op timeout = 900 2024-05-20T18:26:12.513 INFO:teuthology.orchestra.run.smithi063.stdout: rados osd op timeout = 900 2024-05-20T18:26:12.513 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.a] 2024-05-20T18:26:12.513 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.b] 2024-05-20T18:26:12.513 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.c] 2024-05-20T18:26:12.513 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.a] 2024-05-20T18:26:12.513 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.c] 2024-05-20T18:26:12.513 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.b] 2024-05-20T18:26:12.513 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.d] 2024-05-20T18:26:12.513 INFO:teuthology.orchestra.run.smithi063.stdout:[mds] 2024-05-20T18:26:12.513 INFO:teuthology.orchestra.run.smithi063.stdout: debug mds = 20 2024-05-20T18:26:12.513 INFO:teuthology.orchestra.run.smithi063.stdout: debug mds balancer = 20 2024-05-20T18:26:12.513 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-20T18:26:12.513 INFO:teuthology.orchestra.run.smithi063.stdout: mds debug frag = True 2024-05-20T18:26:12.514 INFO:teuthology.orchestra.run.smithi063.stdout: mds debug scatterstat = True 2024-05-20T18:26:12.514 INFO:teuthology.orchestra.run.smithi063.stdout: mds op complaint time = 180 2024-05-20T18:26:12.514 INFO:teuthology.orchestra.run.smithi063.stdout: mds verify scatter = True 2024-05-20T18:26:12.514 INFO:teuthology.orchestra.run.smithi063.stdout: osd op complaint time = 180 2024-05-20T18:26:12.514 INFO:teuthology.orchestra.run.smithi063.stdout: rados mon op timeout = 900 2024-05-20T18:26:12.514 INFO:teuthology.orchestra.run.smithi063.stdout: rados osd op timeout = 900 2024-05-20T18:26:12.515 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-20T18:26:12.516 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-20T18:26:12.621 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-20T18:26:12.622 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:26:12.622 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-20T18:26:12.637 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:26:12.638 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-20T18:26:12.693 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:26:12.693 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:26:12.693 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-20T18:26:12.693 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-20T18:26:12.769 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:26:12.769 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-20T18:26:12.825 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi063.front.sepia.ceph.com 2024-05-20T18:26:12.826 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:26:12.826 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-20T18:26:12.826 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-20T18:26:12.866 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:26:12.866 DEBUG:teuthology.orchestra.run.smithi063:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-20T18:26:12.924 INFO:tasks.ceph:Setting up mon nodes... 2024-05-20T18:26:12.924 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-20T18:26:12.924 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-20T18:26:12.986 INFO:teuthology.orchestra.run.smithi005.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-20T18:26:12.990 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-20T18:26:13.076 INFO:teuthology.orchestra.run.smithi063.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-20T18:26:13.079 INFO:tasks.ceph:Setting up mds nodes... 2024-05-20T18:26:13.080 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-20T18:26:13.142 INFO:teuthology.orchestra.run.smithi005.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-20T18:26:13.145 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-20T18:26:13.170 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-20T18:26:13.272 INFO:teuthology.orchestra.run.smithi005.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-20T18:26:13.275 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-20T18:26:13.301 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-20T18:26:13.365 INFO:teuthology.orchestra.run.smithi063.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-20T18:26:13.369 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-20T18:26:13.394 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-20T18:26:13.496 INFO:teuthology.orchestra.run.smithi063.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-20T18:26:13.499 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-20T18:26:13.525 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-20T18:26:13.525 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-20T18:26:13.577 INFO:teuthology.orchestra.run.smithi005.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-20T18:26:13.592 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-05-20T18:26:13.642 INFO:teuthology.orchestra.run.smithi005.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-05-20T18:26:13.656 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-20T18:26:13.708 INFO:teuthology.orchestra.run.smithi063.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-20T18:26:13.722 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-05-20T18:26:13.773 INFO:teuthology.orchestra.run.smithi063.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-05-20T18:26:13.786 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-20T18:26:13.787 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi005.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi063.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-20T18:26:13.787 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-20T18:26:13.813 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-20T18:26:13.813 INFO:tasks.ceph:role: osd.0 2024-05-20T18:26:13.813 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:26:13.813 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-20T18:26:13.879 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-20T18:26:13.879 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-20T18:26:13.879 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-20T18:26:13.879 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-20T18:26:13.879 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-20T18:26:13.879 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2024-05-20T18:26:13.879 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-20T18:26:13.879 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-20T18:26:13.880 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-20T18:26:13.880 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-20T18:26:13.912 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2024-05-20T18:26:13.915 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2024-05-20T18:26:13.915 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-20T18:26:13.985 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-20T18:26:14.056 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-20T18:26:14.122 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-20T18:26:14.122 INFO:tasks.ceph:role: osd.1 2024-05-20T18:26:14.122 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:26:14.122 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-20T18:26:14.187 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-20T18:26:14.188 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-20T18:26:14.188 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-20T18:26:14.188 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-20T18:26:14.188 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-20T18:26:14.188 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2024-05-20T18:26:14.188 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-20T18:26:14.188 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-20T18:26:14.188 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-20T18:26:14.188 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-20T18:26:14.220 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2024-05-20T18:26:14.223 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2024-05-20T18:26:14.223 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-20T18:26:14.293 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-20T18:26:14.367 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-20T18:26:14.433 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-20T18:26:14.434 INFO:tasks.ceph:role: osd.2 2024-05-20T18:26:14.434 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:26:14.434 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-20T18:26:14.499 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-20T18:26:14.499 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-20T18:26:14.499 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-20T18:26:14.499 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-20T18:26:14.499 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-20T18:26:14.499 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2024-05-20T18:26:14.499 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-20T18:26:14.499 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-20T18:26:14.499 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-20T18:26:14.500 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-20T18:26:14.532 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2024-05-20T18:26:14.534 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2024-05-20T18:26:14.534 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-20T18:26:14.614 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-20T18:26:14.684 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-20T18:26:14.750 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-20T18:26:14.751 INFO:tasks.ceph:role: osd.3 2024-05-20T18:26:14.751 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:26:14.751 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-20T18:26:14.816 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-20T18:26:14.816 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-20T18:26:14.816 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-20T18:26:14.816 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-20T18:26:14.816 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-20T18:26:14.816 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2024-05-20T18:26:14.816 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-20T18:26:14.817 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-20T18:26:14.817 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-20T18:26:14.817 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-20T18:26:14.849 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2024-05-20T18:26:14.850 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2024-05-20T18:26:14.850 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-20T18:26:14.920 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-20T18:26:14.990 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-20T18:26:15.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:15.094+0000 7f14f1663640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-20T18:26:15.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:15.094+0000 7f14f1663640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-20T18:26:15.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:15.094+0000 7f14f1663640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-20T18:26:15.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:15.094+0000 7f14f1663640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-20T18:26:15.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:15.094+0000 7f14f1663640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-20T18:26:15.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:15.095+0000 7f14f1663640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-20T18:26:17.941 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-20T18:26:17.971 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-20T18:26:18.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:18.077+0000 7f40465a1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-20T18:26:18.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:18.077+0000 7f40465a1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-20T18:26:18.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:18.077+0000 7f40465a1640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-20T18:26:18.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:18.077+0000 7f40465a1640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-20T18:26:18.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:18.077+0000 7f40465a1640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-20T18:26:18.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:18.077+0000 7f40465a1640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-20T18:26:20.924 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-20T18:26:20.955 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-20T18:26:21.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:21.061+0000 7feea640f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-20T18:26:21.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:21.061+0000 7feea640f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-20T18:26:21.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:21.061+0000 7feea640f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-20T18:26:21.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:21.062+0000 7feea640f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-20T18:26:21.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:21.062+0000 7feea640f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-20T18:26:21.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:21.062+0000 7feea640f640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-20T18:26:23.917 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-20T18:26:23.949 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-20T18:26:24.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:24.055+0000 7fea29a0d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-20T18:26:24.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:24.055+0000 7fea29a0d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-20T18:26:24.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:24.055+0000 7fea29a0d640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-20T18:26:24.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:24.055+0000 7fea29a0d640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-20T18:26:24.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:24.055+0000 7fea29a0d640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-20T18:26:24.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:24.055+0000 7fea29a0d640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-20T18:26:26.903 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-20T18:26:26.934 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-20T18:26:26.966 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-20T18:26:26.966 INFO:tasks.ceph:role: osd.4 2024-05-20T18:26:26.966 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi063.front.sepia.ceph.com 2024-05-20T18:26:26.966 DEBUG:teuthology.orchestra.run.smithi063:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-20T18:26:27.044 INFO:teuthology.orchestra.run.smithi063.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-20T18:26:27.044 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-20T18:26:27.044 INFO:teuthology.orchestra.run.smithi063.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-20T18:26:27.044 INFO:teuthology.orchestra.run.smithi063.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-20T18:26:27.044 INFO:teuthology.orchestra.run.smithi063.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-20T18:26:27.044 INFO:teuthology.orchestra.run.smithi063.stdout: = sunit=0 swidth=0 blks 2024-05-20T18:26:27.044 INFO:teuthology.orchestra.run.smithi063.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-20T18:26:27.044 INFO:teuthology.orchestra.run.smithi063.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-20T18:26:27.044 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-20T18:26:27.044 INFO:teuthology.orchestra.run.smithi063.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-20T18:26:27.548 INFO:teuthology.orchestra.run.smithi063.stdout:Discarding blocks...Done. 2024-05-20T18:26:27.550 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi063.front.sepia.ceph.com -o noatime 2024-05-20T18:26:27.550 DEBUG:teuthology.orchestra.run.smithi063:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-20T18:26:27.671 DEBUG:teuthology.orchestra.run.smithi063:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-20T18:26:27.709 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-20T18:26:27.777 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-20T18:26:27.777 INFO:tasks.ceph:role: osd.5 2024-05-20T18:26:27.778 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi063.front.sepia.ceph.com 2024-05-20T18:26:27.778 DEBUG:teuthology.orchestra.run.smithi063:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-20T18:26:27.853 INFO:teuthology.orchestra.run.smithi063.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-20T18:26:27.853 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-20T18:26:27.853 INFO:teuthology.orchestra.run.smithi063.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-20T18:26:27.853 INFO:teuthology.orchestra.run.smithi063.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-20T18:26:27.853 INFO:teuthology.orchestra.run.smithi063.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-20T18:26:27.854 INFO:teuthology.orchestra.run.smithi063.stdout: = sunit=0 swidth=0 blks 2024-05-20T18:26:27.854 INFO:teuthology.orchestra.run.smithi063.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-20T18:26:27.854 INFO:teuthology.orchestra.run.smithi063.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-20T18:26:27.854 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-20T18:26:27.854 INFO:teuthology.orchestra.run.smithi063.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-20T18:26:28.309 INFO:teuthology.orchestra.run.smithi063.stdout:Discarding blocks...Done. 2024-05-20T18:26:28.311 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi063.front.sepia.ceph.com -o noatime 2024-05-20T18:26:28.311 DEBUG:teuthology.orchestra.run.smithi063:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-20T18:26:28.406 DEBUG:teuthology.orchestra.run.smithi063:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-20T18:26:28.436 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-20T18:26:28.502 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-20T18:26:28.502 INFO:tasks.ceph:role: osd.6 2024-05-20T18:26:28.502 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi063.front.sepia.ceph.com 2024-05-20T18:26:28.502 DEBUG:teuthology.orchestra.run.smithi063:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-20T18:26:28.576 INFO:teuthology.orchestra.run.smithi063.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-20T18:26:28.576 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-20T18:26:28.577 INFO:teuthology.orchestra.run.smithi063.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-20T18:26:28.577 INFO:teuthology.orchestra.run.smithi063.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-20T18:26:28.577 INFO:teuthology.orchestra.run.smithi063.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-20T18:26:28.577 INFO:teuthology.orchestra.run.smithi063.stdout: = sunit=0 swidth=0 blks 2024-05-20T18:26:28.577 INFO:teuthology.orchestra.run.smithi063.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-20T18:26:28.577 INFO:teuthology.orchestra.run.smithi063.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-20T18:26:28.577 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-20T18:26:28.577 INFO:teuthology.orchestra.run.smithi063.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-20T18:26:29.054 INFO:teuthology.orchestra.run.smithi063.stdout:Discarding blocks...Done. 2024-05-20T18:26:29.056 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi063.front.sepia.ceph.com -o noatime 2024-05-20T18:26:29.056 DEBUG:teuthology.orchestra.run.smithi063:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-20T18:26:29.188 DEBUG:teuthology.orchestra.run.smithi063:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-20T18:26:29.218 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-20T18:26:29.284 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-20T18:26:29.285 INFO:tasks.ceph:role: osd.7 2024-05-20T18:26:29.285 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi063.front.sepia.ceph.com 2024-05-20T18:26:29.285 DEBUG:teuthology.orchestra.run.smithi063:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-20T18:26:29.357 INFO:teuthology.orchestra.run.smithi063.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-20T18:26:29.357 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-20T18:26:29.358 INFO:teuthology.orchestra.run.smithi063.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-20T18:26:29.358 INFO:teuthology.orchestra.run.smithi063.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-20T18:26:29.358 INFO:teuthology.orchestra.run.smithi063.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-20T18:26:29.358 INFO:teuthology.orchestra.run.smithi063.stdout: = sunit=0 swidth=0 blks 2024-05-20T18:26:29.358 INFO:teuthology.orchestra.run.smithi063.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-20T18:26:29.358 INFO:teuthology.orchestra.run.smithi063.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-20T18:26:29.358 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-20T18:26:29.358 INFO:teuthology.orchestra.run.smithi063.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-20T18:26:29.864 INFO:teuthology.orchestra.run.smithi063.stdout:Discarding blocks...Done. 2024-05-20T18:26:29.866 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi063.front.sepia.ceph.com -o noatime 2024-05-20T18:26:29.866 DEBUG:teuthology.orchestra.run.smithi063:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-20T18:26:30.007 DEBUG:teuthology.orchestra.run.smithi063:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-20T18:26:30.040 DEBUG:teuthology.orchestra.run.smithi063:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-20T18:26:30.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:30.152+0000 7f4ed622c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-20T18:26:30.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:30.153+0000 7f4ed622c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-20T18:26:30.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:30.153+0000 7f4ed622c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-20T18:26:30.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:30.153+0000 7f4ed622c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-20T18:26:30.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:30.153+0000 7f4ed622c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-20T18:26:30.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:30.154+0000 7f4ed622c640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-20T18:26:33.005 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-20T18:26:33.036 DEBUG:teuthology.orchestra.run.smithi063:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-20T18:26:33.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:33.144+0000 7efc35443640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-20T18:26:33.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:33.144+0000 7efc35443640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-20T18:26:33.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:33.144+0000 7efc35443640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-20T18:26:33.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:33.144+0000 7efc35443640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-20T18:26:33.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:33.144+0000 7efc35443640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-20T18:26:33.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:33.145+0000 7efc35443640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-20T18:26:35.995 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-20T18:26:36.029 DEBUG:teuthology.orchestra.run.smithi063:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-20T18:26:36.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:36.137+0000 7f6fc4e18640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-20T18:26:36.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:36.138+0000 7f6fc4e18640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-20T18:26:36.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:36.138+0000 7f6fc4e18640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-20T18:26:36.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:36.138+0000 7f6fc4e18640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-20T18:26:36.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:36.138+0000 7f6fc4e18640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-20T18:26:36.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:36.138+0000 7f6fc4e18640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-20T18:26:38.991 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-20T18:26:39.022 DEBUG:teuthology.orchestra.run.smithi063:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-20T18:26:39.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:39.126+0000 7f9d65168640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-20T18:26:39.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:39.127+0000 7f9d65168640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-20T18:26:39.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:39.127+0000 7f9d65168640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-20T18:26:39.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:39.127+0000 7f9d65168640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-20T18:26:39.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:39.127+0000 7f9d65168640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-20T18:26:39.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:39.127+0000 7f9d65168640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-20T18:26:40.436 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:26:40.438 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:26:41.981 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-20T18:26:42.009 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-20T18:26:42.009 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:26:42.009 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-20T18:26:42.041 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:26:42.041 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-20T18:26:42.110 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:26:42.110 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-20T18:26:42.179 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:26:42.179 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-20T18:26:42.248 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:26:42.248 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-20T18:26:42.315 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:26:42.315 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-20T18:26:42.382 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:26:42.382 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-20T18:26:42.449 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:26:42.449 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-20T18:26:42.483 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:26:42.483 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-20T18:26:42.552 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:26:42.552 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-20T18:26:42.621 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:26:42.621 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-20T18:26:42.690 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:26:42.690 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-20T18:26:42.758 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:26:42.758 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-20T18:26:42.825 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:26:42.825 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-20T18:26:42.891 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:26:42.892 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-20T18:26:42.909 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:26:42.909 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-05-20T18:26:42.966 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:26:42.966 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-20T18:26:42.983 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:26:42.983 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-05-20T18:26:43.039 INFO:tasks.ceph:Adding keys to all mons... 2024-05-20T18:26:43.039 DEBUG:teuthology.orchestra.run.smithi005:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-20T18:26:43.041 DEBUG:teuthology.orchestra.run.smithi063:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-20T18:26:43.082 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr.y] 2024-05-20T18:26:43.082 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDElUtm6Xv3OhAA0doOWTC73jy5rOYm7Vf68Q== 2024-05-20T18:26:43.082 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a] 2024-05-20T18:26:43.082 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDFlUtm9lSeCBAA/PpiEqojk6ueJo60tU4/iA== 2024-05-20T18:26:43.083 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.c] 2024-05-20T18:26:43.083 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDFlUtmpEBlEBAAdgxDXj9x38VSOizUgdFpYA== 2024-05-20T18:26:43.083 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.0] 2024-05-20T18:26:43.083 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDHlUtmWFGzBRAA81qN37dlHf6QDZ1FgwbJ9w== 2024-05-20T18:26:43.083 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.1] 2024-05-20T18:26:43.083 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDKlUtmBMKsBBAAD4tTTvetME6oxwugTubSKQ== 2024-05-20T18:26:43.083 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.2] 2024-05-20T18:26:43.083 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDNlUtm4Rm/AxAAIGWTUxb+LII09Rd+h5cJ8Q== 2024-05-20T18:26:43.083 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.3] 2024-05-20T18:26:43.083 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDQlUtmWVxZAxAAB/w6k4WJ58ulEr9zatCRfw== 2024-05-20T18:26:43.083 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr.x] 2024-05-20T18:26:43.083 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDFlUtmPgq0BBAAAljNeQDsSau0wH2Lh2KdBw== 2024-05-20T18:26:43.083 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b] 2024-05-20T18:26:43.083 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDFlUtmdov0FRAACkWAR/0zaCDjgnkHnoHwgw== 2024-05-20T18:26:43.084 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.d] 2024-05-20T18:26:43.084 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDFlUtm0Gy9HRAAAkPqcBLwxTwuWmCi7aad1A== 2024-05-20T18:26:43.084 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.4] 2024-05-20T18:26:43.084 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDWlUtmIyokCRAAolOiYVCYaNTQZb1PjtR6fg== 2024-05-20T18:26:43.084 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.5] 2024-05-20T18:26:43.084 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDZlUtmSf6mCBAAw73nwq/zzt9kIXZymZh93A== 2024-05-20T18:26:43.084 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.6] 2024-05-20T18:26:43.084 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDclUtmwwhGCBAA94OBEtj9h7cmCAddYMonaA== 2024-05-20T18:26:43.084 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.7] 2024-05-20T18:26:43.084 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDflUtm9wefBxAAvCZTBzRu05koFhc+pH5I8w== 2024-05-20T18:26:43.084 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2024-05-20T18:26:43.084 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDFlUtmevaaIhAATIwyXzr0vGT38ZmcMqR2gA== 2024-05-20T18:26:43.084 INFO:teuthology.orchestra.run.smithi005.stdout:[client.2] 2024-05-20T18:26:43.085 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDFlUtmMJF1JhAAeM+HseqvgvKgG8EnIsZuhg== 2024-05-20T18:26:43.085 INFO:teuthology.orchestra.run.smithi005.stdout:[client.1] 2024-05-20T18:26:43.085 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDFlUtmK7NfKhAADaXl0uVP1Q1gUOlPsstnzg== 2024-05-20T18:26:43.085 INFO:teuthology.orchestra.run.smithi005.stdout:[client.3] 2024-05-20T18:26:43.085 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDFlUtmodA7LhAAUI2v0npB6HHs2n1g3KdTXA== 2024-05-20T18:26:43.103 INFO:teuthology.orchestra.run.smithi063.stdout:[mgr.y] 2024-05-20T18:26:43.103 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDElUtm6Xv3OhAA0doOWTC73jy5rOYm7Vf68Q== 2024-05-20T18:26:43.103 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.a] 2024-05-20T18:26:43.103 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDFlUtm9lSeCBAA/PpiEqojk6ueJo60tU4/iA== 2024-05-20T18:26:43.103 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.c] 2024-05-20T18:26:43.103 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDFlUtmpEBlEBAAdgxDXj9x38VSOizUgdFpYA== 2024-05-20T18:26:43.104 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.0] 2024-05-20T18:26:43.104 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDHlUtmWFGzBRAA81qN37dlHf6QDZ1FgwbJ9w== 2024-05-20T18:26:43.104 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.1] 2024-05-20T18:26:43.104 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDKlUtmBMKsBBAAD4tTTvetME6oxwugTubSKQ== 2024-05-20T18:26:43.104 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.2] 2024-05-20T18:26:43.104 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDNlUtm4Rm/AxAAIGWTUxb+LII09Rd+h5cJ8Q== 2024-05-20T18:26:43.104 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.3] 2024-05-20T18:26:43.104 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDQlUtmWVxZAxAAB/w6k4WJ58ulEr9zatCRfw== 2024-05-20T18:26:43.104 INFO:teuthology.orchestra.run.smithi063.stdout:[mgr.x] 2024-05-20T18:26:43.104 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDFlUtmPgq0BBAAAljNeQDsSau0wH2Lh2KdBw== 2024-05-20T18:26:43.104 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.b] 2024-05-20T18:26:43.104 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDFlUtmdov0FRAACkWAR/0zaCDjgnkHnoHwgw== 2024-05-20T18:26:43.104 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.d] 2024-05-20T18:26:43.105 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDFlUtm0Gy9HRAAAkPqcBLwxTwuWmCi7aad1A== 2024-05-20T18:26:43.105 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.4] 2024-05-20T18:26:43.105 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDWlUtmIyokCRAAolOiYVCYaNTQZb1PjtR6fg== 2024-05-20T18:26:43.105 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.5] 2024-05-20T18:26:43.105 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDZlUtmSf6mCBAAw73nwq/zzt9kIXZymZh93A== 2024-05-20T18:26:43.105 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.6] 2024-05-20T18:26:43.105 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDclUtmwwhGCBAA94OBEtj9h7cmCAddYMonaA== 2024-05-20T18:26:43.105 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.7] 2024-05-20T18:26:43.105 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDflUtm9wefBxAAvCZTBzRu05koFhc+pH5I8w== 2024-05-20T18:26:43.105 INFO:teuthology.orchestra.run.smithi063.stdout:[client.0] 2024-05-20T18:26:43.105 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDFlUtmevaaIhAATIwyXzr0vGT38ZmcMqR2gA== 2024-05-20T18:26:43.105 INFO:teuthology.orchestra.run.smithi063.stdout:[client.2] 2024-05-20T18:26:43.105 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDFlUtmMJF1JhAAeM+HseqvgvKgG8EnIsZuhg== 2024-05-20T18:26:43.106 INFO:teuthology.orchestra.run.smithi063.stdout:[client.1] 2024-05-20T18:26:43.106 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDFlUtmK7NfKhAADaXl0uVP1Q1gUOlPsstnzg== 2024-05-20T18:26:43.106 INFO:teuthology.orchestra.run.smithi063.stdout:[client.3] 2024-05-20T18:26:43.106 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDFlUtmodA7LhAAUI2v0npB6HHs2n1g3KdTXA== 2024-05-20T18:26:43.107 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-20T18:26:43.128 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-20T18:26:43.209 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-20T18:26:43.211 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-20T18:26:43.276 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-20T18:26:43.279 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-20T18:26:43.345 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T18:26:43.347 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T18:26:43.409 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T18:26:43.411 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T18:26:43.473 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T18:26:43.475 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T18:26:43.537 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T18:26:43.540 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T18:26:43.601 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-20T18:26:43.604 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-20T18:26:43.666 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-20T18:26:43.670 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-20T18:26:43.736 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-20T18:26:43.738 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-20T18:26:43.803 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T18:26:43.805 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T18:26:43.868 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T18:26:43.870 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T18:26:43.932 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T18:26:43.935 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T18:26:43.997 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T18:26:43.999 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-20T18:26:44.061 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-20T18:26:44.064 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-20T18:26:44.131 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-20T18:26:44.133 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-20T18:26:44.198 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-20T18:26:44.201 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-20T18:26:44.266 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-20T18:26:44.268 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-20T18:26:44.334 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-20T18:26:44.334 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-20T18:26:44.360 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-20T18:26:44.466 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-20T18:26:44.492 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-20T18:26:44.518 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-20T18:26:44.628 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-20T18:26:44.654 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-20T18:26:44.719 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-20T18:26:44.827 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-20T18:26:44.853 DEBUG:teuthology.orchestra.run.smithi005:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-20T18:26:44.855 DEBUG:teuthology.orchestra.run.smithi063:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-20T18:26:44.909 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-20T18:26:44.910 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-20T18:26:44.910 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-20T18:26:44.912 INFO:tasks.ceph.mon.a:Started 2024-05-20T18:26:44.913 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-20T18:26:44.913 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-20T18:26:44.952 INFO:tasks.ceph.mon.b:Started 2024-05-20T18:26:44.952 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-20T18:26:44.952 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-20T18:26:44.954 INFO:tasks.ceph.mon.c:Started 2024-05-20T18:26:44.954 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-20T18:26:44.955 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-20T18:26:44.955 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-20T18:26:44.957 INFO:tasks.ceph.mgr.y:Started 2024-05-20T18:26:44.957 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-20T18:26:44.957 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-20T18:26:44.959 INFO:tasks.ceph.mgr.x:Started 2024-05-20T18:26:44.959 INFO:tasks.ceph:Setting crush tunables to default 2024-05-20T18:26:44.959 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --cluster ceph osd crush tunables default 2024-05-20T18:26:45.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.078+0000 7f0b7b5c2640 1 Processor -- start 2024-05-20T18:26:45.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.078+0000 7f0b7b5c2640 1 -- start start 2024-05-20T18:26:45.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.079+0000 7f0b7b5c2640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0b740b2bf0 0x7f0b74162f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:45.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.079+0000 7f0b7b5c2640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0b740b0440 0x7f0b74163450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:45.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.079+0000 7f0b7b5c2640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b740ae7b0 con 0x7f0b74163990 2024-05-20T18:26:45.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.079+0000 7f0b7b5c2640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f0b74163da0 con 0x7f0b740b2bf0 2024-05-20T18:26:45.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.079+0000 7f0b7b5c2640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0b740b0b70 con 0x7f0b740b0440 2024-05-20T18:26:45.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.079+0000 7f0b7935e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0b740b2bf0 0x7f0b74162f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:45.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.079+0000 7f0b79b5f640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0b74163990 0x7f0b740d09f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46008/0 (socket says 172.21.15.5:46008) 2024-05-20T18:26:45.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.079+0000 7f0b79b5f640 1 -- 172.21.15.5:0/1704250465 learned_addr learned my addr 172.21.15.5:0/1704250465 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:26:45.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.080+0000 7f0b7935e640 1 -- 172.21.15.5:0/1704250465 >> v2:172.21.15.5:3300/0 conn(0x7f0b740b2bf0 msgr2=0x7f0b74162f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-20T18:26:45.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.080+0000 7f0b7935e640 1 -- 172.21.15.5:0/1704250465 >> v2:172.21.15.5:3300/0 conn(0x7f0b740b2bf0 msgr2=0x7f0b74162f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T18:26:45.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.080+0000 7f0b7935e640 1 --2- 172.21.15.5:0/1704250465 >> v2:172.21.15.5:3300/0 conn(0x7f0b740b2bf0 0x7f0b74162f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T18:26:45.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.080+0000 7f0b7935e640 1 --2- 172.21.15.5:0/1704250465 >> v2:172.21.15.5:3300/0 conn(0x7f0b740b2bf0 0x7f0b74162f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T18:26:45.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.080+0000 7f0b78b5d640 1 --2- 172.21.15.5:0/1704250465 >> v2:172.21.15.63:3300/0 conn(0x7f0b740b0440 0x7f0b74163450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:45.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.080+0000 7f0b78b5d640 1 -- 172.21.15.5:0/1704250465 >> v2:172.21.15.63:3300/0 conn(0x7f0b740b0440 msgr2=0x7f0b74163450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-20T18:26:45.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.080+0000 7f0b78b5d640 1 -- 172.21.15.5:0/1704250465 >> v2:172.21.15.63:3300/0 conn(0x7f0b740b0440 msgr2=0x7f0b74163450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T18:26:45.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.080+0000 7f0b78b5d640 1 --2- 172.21.15.5:0/1704250465 >> v2:172.21.15.63:3300/0 conn(0x7f0b740b0440 0x7f0b74163450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T18:26:45.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.080+0000 7f0b78b5d640 1 --2- 172.21.15.5:0/1704250465 >> v2:172.21.15.63:3300/0 conn(0x7f0b740b0440 0x7f0b74163450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T18:26:45.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.281+0000 7f0b7935e640 1 --2- 172.21.15.5:0/1704250465 >> v2:172.21.15.5:3300/0 conn(0x7f0b740b2bf0 0x7f0b74162f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:45.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.281+0000 7f0b78b5d640 1 --2- 172.21.15.5:0/1704250465 >> v2:172.21.15.63:3300/0 conn(0x7f0b740b0440 0x7f0b74163450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:45.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.281+0000 7f0b7935e640 1 -- 172.21.15.5:0/1704250465 >> v2:172.21.15.5:3300/0 conn(0x7f0b740b2bf0 msgr2=0x7f0b74162f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-20T18:26:45.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.281+0000 7f0b7935e640 1 -- 172.21.15.5:0/1704250465 >> v2:172.21.15.5:3300/0 conn(0x7f0b740b2bf0 msgr2=0x7f0b74162f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T18:26:45.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.281+0000 7f0b7935e640 1 --2- 172.21.15.5:0/1704250465 >> v2:172.21.15.5:3300/0 conn(0x7f0b740b2bf0 0x7f0b74162f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T18:26:45.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.281+0000 7f0b7935e640 1 --2- 172.21.15.5:0/1704250465 >> v2:172.21.15.5:3300/0 conn(0x7f0b740b2bf0 0x7f0b74162f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-20T18:26:45.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.281+0000 7f0b78b5d640 1 -- 172.21.15.5:0/1704250465 >> v2:172.21.15.63:3300/0 conn(0x7f0b740b0440 msgr2=0x7f0b74163450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-20T18:26:45.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.281+0000 7f0b78b5d640 1 -- 172.21.15.5:0/1704250465 >> v2:172.21.15.63:3300/0 conn(0x7f0b740b0440 msgr2=0x7f0b74163450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T18:26:45.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.281+0000 7f0b78b5d640 1 --2- 172.21.15.5:0/1704250465 >> v2:172.21.15.63:3300/0 conn(0x7f0b740b0440 0x7f0b74163450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T18:26:45.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.281+0000 7f0b78b5d640 1 --2- 172.21.15.5:0/1704250465 >> v2:172.21.15.63:3300/0 conn(0x7f0b740b0440 0x7f0b74163450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-20T18:26:45.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.682+0000 7f0b7935e640 1 --2- 172.21.15.5:0/1704250465 >> v2:172.21.15.5:3300/0 conn(0x7f0b740b2bf0 0x7f0b74162f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:45.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.682+0000 7f0b7935e640 1 -- 172.21.15.5:0/1704250465 >> v2:172.21.15.5:3300/0 conn(0x7f0b740b2bf0 msgr2=0x7f0b74162f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-20T18:26:45.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.682+0000 7f0b7935e640 1 -- 172.21.15.5:0/1704250465 >> v2:172.21.15.5:3300/0 conn(0x7f0b740b2bf0 msgr2=0x7f0b74162f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T18:26:45.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.682+0000 7f0b7935e640 1 --2- 172.21.15.5:0/1704250465 >> v2:172.21.15.5:3300/0 conn(0x7f0b740b2bf0 0x7f0b74162f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T18:26:45.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.682+0000 7f0b7935e640 1 --2- 172.21.15.5:0/1704250465 >> v2:172.21.15.5:3300/0 conn(0x7f0b740b2bf0 0x7f0b74162f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-20T18:26:45.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.682+0000 7f0b78b5d640 1 --2- 172.21.15.5:0/1704250465 >> v2:172.21.15.63:3300/0 conn(0x7f0b740b0440 0x7f0b74163450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:45.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.682+0000 7f0b78b5d640 1 -- 172.21.15.5:0/1704250465 >> v2:172.21.15.63:3300/0 conn(0x7f0b740b0440 msgr2=0x7f0b74163450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-20T18:26:45.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.682+0000 7f0b78b5d640 1 -- 172.21.15.5:0/1704250465 >> v2:172.21.15.63:3300/0 conn(0x7f0b740b0440 msgr2=0x7f0b74163450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T18:26:45.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.682+0000 7f0b78b5d640 1 --2- 172.21.15.5:0/1704250465 >> v2:172.21.15.63:3300/0 conn(0x7f0b740b0440 0x7f0b74163450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T18:26:45.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:45.682+0000 7f0b78b5d640 1 --2- 172.21.15.5:0/1704250465 >> v2:172.21.15.63:3300/0 conn(0x7f0b740b0440 0x7f0b74163450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-20T18:26:46.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:46.483+0000 7f0b7935e640 1 --2- 172.21.15.5:0/1704250465 >> v2:172.21.15.5:3300/0 conn(0x7f0b740b2bf0 0x7f0b74162f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:46.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:46.483+0000 7f0b7935e640 1 -- 172.21.15.5:0/1704250465 >> v2:172.21.15.5:3300/0 conn(0x7f0b740b2bf0 msgr2=0x7f0b74162f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-20T18:26:46.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:46.483+0000 7f0b7935e640 1 -- 172.21.15.5:0/1704250465 >> v2:172.21.15.5:3300/0 conn(0x7f0b740b2bf0 msgr2=0x7f0b74162f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T18:26:46.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:46.483+0000 7f0b7935e640 1 --2- 172.21.15.5:0/1704250465 >> v2:172.21.15.5:3300/0 conn(0x7f0b740b2bf0 0x7f0b74162f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T18:26:46.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:46.483+0000 7f0b7935e640 1 --2- 172.21.15.5:0/1704250465 >> v2:172.21.15.5:3300/0 conn(0x7f0b740b2bf0 0x7f0b74162f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-20T18:26:46.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:46.483+0000 7f0b78b5d640 1 --2- 172.21.15.5:0/1704250465 >> v2:172.21.15.63:3300/0 conn(0x7f0b740b0440 0x7f0b74163450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:46.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:46.483+0000 7f0b78b5d640 1 -- 172.21.15.5:0/1704250465 >> v2:172.21.15.63:3300/0 conn(0x7f0b740b0440 msgr2=0x7f0b74163450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-20T18:26:46.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:46.483+0000 7f0b78b5d640 1 -- 172.21.15.5:0/1704250465 >> v2:172.21.15.63:3300/0 conn(0x7f0b740b0440 msgr2=0x7f0b74163450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T18:26:46.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:46.483+0000 7f0b78b5d640 1 --2- 172.21.15.5:0/1704250465 >> v2:172.21.15.63:3300/0 conn(0x7f0b740b0440 0x7f0b74163450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T18:26:46.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:46.483+0000 7f0b78b5d640 1 --2- 172.21.15.5:0/1704250465 >> v2:172.21.15.63:3300/0 conn(0x7f0b740b0440 0x7f0b74163450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-20T18:26:48.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:48.080+0000 7f0b6affd640 1 -- 172.21.15.5:0/1704250465 >> v2:172.21.15.63:3300/0 conn(0x7f0b740b0440 msgr2=0x7f0b74163450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T18:26:48.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:48.080+0000 7f0b6affd640 1 --2- 172.21.15.5:0/1704250465 >> v2:172.21.15.63:3300/0 conn(0x7f0b740b0440 0x7f0b74163450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:48.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:48.080+0000 7f0b6affd640 1 -- 172.21.15.5:0/1704250465 >> v2:172.21.15.5:3300/0 conn(0x7f0b740b2bf0 msgr2=0x7f0b74162f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T18:26:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:48.080+0000 7f0b6affd640 1 --2- 172.21.15.5:0/1704250465 >> v2:172.21.15.5:3300/0 conn(0x7f0b740b2bf0 0x7f0b74162f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:48.080+0000 7f0b6affd640 1 -- 172.21.15.5:0/1704250465 >> v1:172.21.15.5:6789/0 conn(0x7f0b74163990 legacy=0x7f0b740d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:48.080+0000 7f0b6affd640 1 --2- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b58003670 0x7f0b58005b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:48.080+0000 7f0b6affd640 1 -- 172.21.15.5:0/1704250465 --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b740b3170 con 0x7f0b580061a0 2024-05-20T18:26:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:48.080+0000 7f0b6affd640 1 -- 172.21.15.5:0/1704250465 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0b74163da0 con 0x7f0b58003670 2024-05-20T18:26:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:48.080+0000 7f0b7935e640 1 --2- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b58003670 0x7f0b58005b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:48.080+0000 7f0b7935e640 1 -- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b58003670 msgr2=0x7f0b58005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-20T18:26:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:48.080+0000 7f0b7935e640 1 -- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b58003670 msgr2=0x7f0b58005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T18:26:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:48.080+0000 7f0b7935e640 1 --2- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b58003670 0x7f0b58005b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T18:26:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:48.080+0000 7f0b7935e640 1 --2- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b58003670 0x7f0b58005b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-20T18:26:48.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:48.281+0000 7f0b7935e640 1 --2- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b58003670 0x7f0b58005b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:48.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:48.281+0000 7f0b7935e640 1 -- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b58003670 msgr2=0x7f0b58005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-20T18:26:48.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:48.281+0000 7f0b7935e640 1 -- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b58003670 msgr2=0x7f0b58005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T18:26:48.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:48.281+0000 7f0b7935e640 1 --2- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b58003670 0x7f0b58005b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T18:26:48.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:48.281+0000 7f0b7935e640 1 --2- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b58003670 0x7f0b58005b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-20T18:26:48.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:48.682+0000 7f0b7935e640 1 --2- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b58003670 0x7f0b58005b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:48.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:48.682+0000 7f0b7935e640 1 -- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b58003670 msgr2=0x7f0b58005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-20T18:26:48.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:48.682+0000 7f0b7935e640 1 -- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b58003670 msgr2=0x7f0b58005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T18:26:48.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:48.682+0000 7f0b7935e640 1 --2- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b58003670 0x7f0b58005b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T18:26:48.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:48.682+0000 7f0b7935e640 1 --2- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b58003670 0x7f0b58005b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-20T18:26:49.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:49.483+0000 7f0b7935e640 1 --2- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b58003670 0x7f0b58005b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:49.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:49.483+0000 7f0b7935e640 1 -- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b58003670 msgr2=0x7f0b58005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-20T18:26:49.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:49.483+0000 7f0b7935e640 1 -- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b58003670 msgr2=0x7f0b58005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-20T18:26:49.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:49.483+0000 7f0b7935e640 1 --2- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b58003670 0x7f0b58005b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-20T18:26:49.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:49.483+0000 7f0b7935e640 1 --2- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b58003670 0x7f0b58005b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-20T18:26:50.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.249+0000 7f0b6bfff640 1 -- 172.21.15.5:0/1704250465 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2982330367 0 0) 0x7f0b740b3170 con 0x7f0b580061a0 2024-05-20T18:26:50.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.249+0000 7f0b6bfff640 1 -- 172.21.15.5:0/1704250465 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b60003520 con 0x7f0b580061a0 2024-05-20T18:26:50.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.255+0000 7f0b6bfff640 1 -- 172.21.15.5:0/1704250465 <== mon.1 v1:172.21.15.63:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 953761084 0 0) 0x7f0b60003520 con 0x7f0b580061a0 2024-05-20T18:26:50.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.255+0000 7f0b6bfff640 1 -- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b58003670 msgr2=0x7f0b58005b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T18:26:50.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.255+0000 7f0b6bfff640 1 --2- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b58003670 0x7f0b58005b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:50.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.255+0000 7f0b6bfff640 1 -- 172.21.15.5:0/1704250465 --> v1:172.21.15.63:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b5800b020 con 0x7f0b580061a0 2024-05-20T18:26:50.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.255+0000 7f0b6bfff640 1 -- 172.21.15.5:0/1704250465 <== mon.1 v1:172.21.15.63:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f0b6400d580 con 0x7f0b580061a0 2024-05-20T18:26:50.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.255+0000 7f0b6bfff640 1 -- 172.21.15.5:0/1704250465 >> v1:172.21.15.63:6789/0 conn(0x7f0b580061a0 legacy=0x7f0b58008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:50.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.255+0000 7f0b79b5f640 1 -- 172.21.15.5:0/1704250465 reap_dead start 2024-05-20T18:26:50.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.255+0000 7f0b6bfff640 1 --2- 172.21.15.5:0/1704250465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b600039b0 0x7f0b60003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:50.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.255+0000 7f0b6bfff640 1 --2- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b60004800 0x7f0b60004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:50.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.255+0000 7f0b6bfff640 1 --2- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b60005330 0x7f0b6000b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:50.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.255+0000 7f0b6bfff640 1 -- 172.21.15.5:0/1704250465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0b74163da0 con 0x7f0b600039b0 2024-05-20T18:26:50.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.255+0000 7f0b6bfff640 1 -- 172.21.15.5:0/1704250465 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6000c2c0 con 0x7f0b60005330 2024-05-20T18:26:50.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.255+0000 7f0b6bfff640 1 -- 172.21.15.5:0/1704250465 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0b6000c4c0 con 0x7f0b60004800 2024-05-20T18:26:50.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.255+0000 7f0b7935e640 1 --2- 172.21.15.5:0/1704250465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b600039b0 0x7f0b60003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.256+0000 7f0b78b5d640 1 --2- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b60004800 0x7f0b60004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.256+0000 7f0b79b5f640 1 --2- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b60005330 0x7f0b6000b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.256+0000 7f0b7935e640 1 -- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b60004800 msgr2=0x7f0b60004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:26:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.256+0000 7f0b7935e640 1 --2- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b60004800 0x7f0b60004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.256+0000 7f0b7935e640 1 -- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b60005330 msgr2=0x7f0b6000b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:26:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.256+0000 7f0b7935e640 1 --2- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b60005330 0x7f0b6000b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.256+0000 7f0b7935e640 1 -- 172.21.15.5:0/1704250465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b60003520 con 0x7f0b600039b0 2024-05-20T18:26:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.256+0000 7f0b79b5f640 1 --2- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b60005330 0x7f0b6000b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:26:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.257+0000 7f0b7935e640 1 --2- 172.21.15.5:0/1704250465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b600039b0 0x7f0b60003e00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0b70019cd0 tx=0x7f0b60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad67245de793431 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:26:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.257+0000 7f0b6bfff640 1 -- 172.21.15.5:0/1704250465 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7003a070 con 0x7f0b600039b0 2024-05-20T18:26:50.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.258+0000 7f0b6bfff640 1 -- 172.21.15.5:0/1704250465 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b70035410 con 0x7f0b600039b0 2024-05-20T18:26:50.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.258+0000 7f0b6bfff640 1 -- 172.21.15.5:0/1704250465 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b70019950 con 0x7f0b600039b0 2024-05-20T18:26:50.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.259+0000 7f0b7b5c2640 1 -- 172.21.15.5:0/1704250465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b600039b0 msgr2=0x7f0b60003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:50.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.259+0000 7f0b7b5c2640 1 --2- 172.21.15.5:0/1704250465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b600039b0 0x7f0b60003e00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0b70019cd0 tx=0x7f0b60005830 comp rx=0 tx=0).stop 2024-05-20T18:26:50.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.259+0000 7f0b7b5c2640 1 -- 172.21.15.5:0/1704250465 shutdown_connections 2024-05-20T18:26:50.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.259+0000 7f0b7b5c2640 1 --2- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b60005330 0x7f0b6000b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:50.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.259+0000 7f0b7b5c2640 1 --2- 172.21.15.5:0/1704250465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b60004800 0x7f0b60004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:50.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.259+0000 7f0b7b5c2640 1 --2- 172.21.15.5:0/1704250465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b600039b0 0x7f0b60003e00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:50.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.259+0000 7f0b7b5c2640 1 -- 172.21.15.5:0/1704250465 >> 172.21.15.5:0/1704250465 conn(0x7f0b740b6620 msgr2=0x7f0b740b6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:26:50.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.259+0000 7f0b7b5c2640 1 -- 172.21.15.5:0/1704250465 shutdown_connections 2024-05-20T18:26:50.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.259+0000 7f0b7b5c2640 1 -- 172.21.15.5:0/1704250465 wait complete. 2024-05-20T18:26:50.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.261+0000 7f0b7b5c2640 1 Processor -- start 2024-05-20T18:26:50.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.261+0000 7f0b7b5c2640 1 -- start start 2024-05-20T18:26:50.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.263+0000 7f0b7b5c2640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b600039b0 0x7f0b74153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:50.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.263+0000 7f0b7b5c2640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b60004800 0x7f0b740cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:50.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.263+0000 7f0b7b5c2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b60005330 0x7f0b740b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:50.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.263+0000 7f0b7b5c2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0b740b10a0 con 0x7f0b60005330 2024-05-20T18:26:50.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.263+0000 7f0b7b5c2640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0b740b1410 con 0x7f0b60004800 2024-05-20T18:26:50.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.263+0000 7f0b7b5c2640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0b7406df70 con 0x7f0b600039b0 2024-05-20T18:26:50.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.263+0000 7f0b7935e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b600039b0 0x7f0b74153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:50.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.263+0000 7f0b79b5f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b60005330 0x7f0b740b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:50.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.263+0000 7f0b78b5d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b60004800 0x7f0b740cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:50.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.263+0000 7f0b79b5f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b60005330 0x7f0b740b1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39634/0 (socket says 172.21.15.5:39634) 2024-05-20T18:26:50.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.263+0000 7f0b7935e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b600039b0 0x7f0b74153300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:58314/0 (socket says 172.21.15.5:58314) 2024-05-20T18:26:50.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.263+0000 7f0b79b5f640 1 -- 172.21.15.5:0/3045395835 learned_addr learned my addr 172.21.15.5:0/3045395835 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:26:50.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.263+0000 7f0b79b5f640 1 -- 172.21.15.5:0/3045395835 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b600039b0 msgr2=0x7f0b74153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:50.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.263+0000 7f0b79b5f640 1 --2- 172.21.15.5:0/3045395835 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b600039b0 0x7f0b74153300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:50.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.263+0000 7f0b79b5f640 1 -- 172.21.15.5:0/3045395835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b60004800 msgr2=0x7f0b740cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:50.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.263+0000 7f0b79b5f640 1 --2- 172.21.15.5:0/3045395835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b60004800 0x7f0b740cda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:50.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.263+0000 7f0b79b5f640 1 -- 172.21.15.5:0/3045395835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b740ce3a0 con 0x7f0b60005330 2024-05-20T18:26:50.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.264+0000 7f0b78b5d640 1 --2- 172.21.15.5:0/3045395835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b60004800 0x7f0b740cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:26:50.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.264+0000 7f0b79b5f640 1 --2- 172.21.15.5:0/3045395835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b60005330 0x7f0b740b1c60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0b74072750 tx=0x7f0b6c007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:26:50.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.264+0000 7f0b69ffb640 1 -- 172.21.15.5:0/3045395835 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c00e8c0 con 0x7f0b60005330 2024-05-20T18:26:50.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.264+0000 7f0b69ffb640 1 -- 172.21.15.5:0/3045395835 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b6c018070 con 0x7f0b60005330 2024-05-20T18:26:50.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.264+0000 7f0b7b5c2640 1 -- 172.21.15.5:0/3045395835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b740c6a10 con 0x7f0b60005330 2024-05-20T18:26:50.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.264+0000 7f0b69ffb640 1 -- 172.21.15.5:0/3045395835 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c00ec80 con 0x7f0b60005330 2024-05-20T18:26:50.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.264+0000 7f0b7b5c2640 1 -- 172.21.15.5:0/3045395835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b740cec40 con 0x7f0b60005330 2024-05-20T18:26:50.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.264+0000 7f0b69ffb640 1 -- 172.21.15.5:0/3045395835 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0b6c01d460 con 0x7f0b60005330 2024-05-20T18:26:50.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.265+0000 7f0b69ffb640 1 -- 172.21.15.5:0/3045395835 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0b6c0136f0 con 0x7f0b60005330 2024-05-20T18:26:50.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.266+0000 7f0b7b5c2640 1 -- 172.21.15.5:0/3045395835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b740c6be0 con 0x7f0b60005330 2024-05-20T18:26:50.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.268+0000 7f0b69ffb640 1 -- 172.21.15.5:0/3045395835 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f0b740ce3a0 con 0x7f0b60005330 2024-05-20T18:26:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.337+0000 7f0b7b5c2640 1 -- 172.21.15.5:0/3045395835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f0b74075b00 con 0x7f0b60005330 2024-05-20T18:26:50.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.339+0000 7f0b69ffb640 1 -- 172.21.15.5:0/3045395835 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f0b6c00ee20 con 0x7f0b60005330 2024-05-20T18:26:50.338 INFO:teuthology.orchestra.run.smithi005.stderr:adjusted tunables profile to default 2024-05-20T18:26:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.340+0000 7f0b7b5c2640 1 -- 172.21.15.5:0/3045395835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b60005330 msgr2=0x7f0b740b1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.340+0000 7f0b7b5c2640 1 --2- 172.21.15.5:0/3045395835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b60005330 0x7f0b740b1c60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0b74072750 tx=0x7f0b6c007b00 comp rx=0 tx=0).stop 2024-05-20T18:26:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.340+0000 7f0b7b5c2640 1 -- 172.21.15.5:0/3045395835 shutdown_connections 2024-05-20T18:26:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.340+0000 7f0b7b5c2640 1 --2- 172.21.15.5:0/3045395835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b60005330 0x7f0b740b1c60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.340+0000 7f0b7b5c2640 1 --2- 172.21.15.5:0/3045395835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b60004800 0x7f0b740cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.340+0000 7f0b7b5c2640 1 --2- 172.21.15.5:0/3045395835 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b600039b0 0x7f0b74153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.340+0000 7f0b7b5c2640 1 -- 172.21.15.5:0/3045395835 >> 172.21.15.5:0/3045395835 conn(0x7f0b740b6620 msgr2=0x7f0b7406d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:26:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.340+0000 7f0b7b5c2640 1 -- 172.21.15.5:0/3045395835 shutdown_connections 2024-05-20T18:26:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:50.340+0000 7f0b7b5c2640 1 -- 172.21.15.5:0/3045395835 wait complete. 2024-05-20T18:26:50.352 INFO:tasks.ceph:check_enable_crimson: False 2024-05-20T18:26:50.352 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-20T18:26:50.352 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:26:50.352 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-20T18:26:50.369 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-20T18:26:50.371+0000 7fd1b5923040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-20T18:26:50.378 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:26:50.379 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-20T18:26:50.444 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:26:50.444 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-20T18:26:50.510 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:26:50.511 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-20T18:26:50.541 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-20T18:26:50.542+0000 7fd1b5923040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-20T18:26:50.566 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:26:50.566 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-20T18:26:50.598 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-20T18:26:50.599+0000 7fd1b5923040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-20T18:26:50.598 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:26:50.598 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-20T18:26:50.669 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:26:50.669 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-20T18:26:50.737 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:26:50.738 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-20T18:26:50.806 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd new 3385e4a6-b897-4645-86af-19fd42f79fa6 0 2024-05-20T18:26:50.848 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-20T18:26:50.850+0000 7fd1b5923040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-20T18:26:50.896 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-20T18:26:50.897+0000 7fd1b5923040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-20T18:26:50.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.966+0000 7f9c39302640 1 Processor -- start 2024-05-20T18:26:50.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.967+0000 7f9c39302640 1 -- start start 2024-05-20T18:26:50.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.968+0000 7f9c39302640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c34163a10 0x7f9c340d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:50.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.968+0000 7f9c39302640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9c340d14b0 0x7f9c340d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:50.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.968+0000 7f9c39302640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c340d0370 con 0x7f9c340b4010 2024-05-20T18:26:50.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.968+0000 7f9c39302640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f9c340d3ca0 con 0x7f9c340d14b0 2024-05-20T18:26:50.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.968+0000 7f9c39302640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9c340d0650 con 0x7f9c34163a10 2024-05-20T18:26:50.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.968+0000 7f9c337fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9c340d14b0 0x7f9c340d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:50.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.968+0000 7f9c32ffd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c34163a10 0x7f9c340d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:50.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.968+0000 7f9c337fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9c340d14b0 0x7f9c340d18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37464/0 (socket says 172.21.15.63:37464) 2024-05-20T18:26:50.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.968+0000 7f9c32ffd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c34163a10 0x7f9c340d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.63:52336/0 (socket says 172.21.15.63:52336) 2024-05-20T18:26:50.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.968+0000 7f9c337fe640 1 -- 172.21.15.63:0/3606963206 learned_addr learned my addr 172.21.15.63:0/3606963206 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-20T18:26:50.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.968+0000 7f9c337fe640 1 -- 172.21.15.63:0/3606963206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c34163a10 msgr2=0x7f9c340d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:26:50.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.968+0000 7f9c337fe640 1 --2- 172.21.15.63:0/3606963206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c34163a10 0x7f9c340d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:50.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.968+0000 7f9c337fe640 1 -- 172.21.15.63:0/3606963206 >> v1:172.21.15.63:6789/0 conn(0x7f9c340b4010 legacy=0x7f9c340cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:50.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.968+0000 7f9c337fe640 1 -- 172.21.15.63:0/3606963206 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c340d3650 con 0x7f9c340d14b0 2024-05-20T18:26:50.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.968+0000 7f9c32ffd640 1 --2- 172.21.15.63:0/3606963206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c34163a10 0x7f9c340d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:26:50.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.969+0000 7f9c337fe640 1 --2- 172.21.15.63:0/3606963206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c340d14b0 0x7f9c340d18c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009e50 tx=0x7f9c2402a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2feb00ab9b46a886 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:26:50.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.969+0000 7f9c31ffb640 1 -- 172.21.15.63:0/3606963206 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c24036070 con 0x7f9c340d14b0 2024-05-20T18:26:50.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.969+0000 7f9c31ffb640 1 -- 172.21.15.63:0/3606963206 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c24031600 con 0x7f9c340d14b0 2024-05-20T18:26:50.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.969+0000 7f9c31ffb640 1 -- 172.21.15.63:0/3606963206 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c240319d0 con 0x7f9c340d14b0 2024-05-20T18:26:50.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.970+0000 7f9c39302640 1 -- 172.21.15.63:0/3606963206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c340d14b0 msgr2=0x7f9c340d18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:50.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.970+0000 7f9c39302640 1 --2- 172.21.15.63:0/3606963206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c340d14b0 0x7f9c340d18c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009e50 tx=0x7f9c2402a730 comp rx=0 tx=0).stop 2024-05-20T18:26:50.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.970+0000 7f9c39302640 1 -- 172.21.15.63:0/3606963206 shutdown_connections 2024-05-20T18:26:50.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.970+0000 7f9c39302640 1 --2- 172.21.15.63:0/3606963206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c340d14b0 0x7f9c340d18c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:50.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.970+0000 7f9c39302640 1 --2- 172.21.15.63:0/3606963206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c34163a10 0x7f9c340d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:50.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.970+0000 7f9c39302640 1 -- 172.21.15.63:0/3606963206 >> 172.21.15.63:0/3606963206 conn(0x7f9c3406d240 msgr2=0x7f9c3406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:26:50.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.970+0000 7f9c39302640 1 -- 172.21.15.63:0/3606963206 shutdown_connections 2024-05-20T18:26:50.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.970+0000 7f9c39302640 1 -- 172.21.15.63:0/3606963206 wait complete. 2024-05-20T18:26:50.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.972+0000 7f9c39302640 1 Processor -- start 2024-05-20T18:26:50.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.972+0000 7f9c39302640 1 -- start start 2024-05-20T18:26:50.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.973+0000 7f9c39302640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c340b4010 0x7f9c341f6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:50.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.973+0000 7f9c39302640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c340d14b0 0x7f9c340b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:50.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.973+0000 7f9c39302640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c34163a10 0x7f9c340ad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:50.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.974+0000 7f9c39302640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9c340b8210 con 0x7f9c34163a10 2024-05-20T18:26:50.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.974+0000 7f9c39302640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9c340d1e00 con 0x7f9c340b4010 2024-05-20T18:26:50.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.974+0000 7f9c39302640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9c340b8390 con 0x7f9c340d14b0 2024-05-20T18:26:50.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.974+0000 7f9c32ffd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c340b4010 0x7f9c341f6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:50.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.974+0000 7f9c327fc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c340d14b0 0x7f9c340b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:50.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.974+0000 7f9c32ffd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c340b4010 0x7f9c341f6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37466/0 (socket says 172.21.15.63:37466) 2024-05-20T18:26:50.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.974+0000 7f9c32ffd640 1 -- 172.21.15.63:0/222482456 learned_addr learned my addr 172.21.15.63:0/222482456 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-20T18:26:50.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.974+0000 7f9c337fe640 1 --2- 172.21.15.63:0/222482456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c34163a10 0x7f9c340ad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:50.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.974+0000 7f9c32ffd640 1 -- 172.21.15.63:0/222482456 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c340d14b0 msgr2=0x7f9c340b2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:50.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.974+0000 7f9c32ffd640 1 --2- 172.21.15.63:0/222482456 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c340d14b0 0x7f9c340b2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:50.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.974+0000 7f9c32ffd640 1 -- 172.21.15.63:0/222482456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c34163a10 msgr2=0x7f9c340ad4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:50.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.974+0000 7f9c32ffd640 1 --2- 172.21.15.63:0/222482456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c34163a10 0x7f9c340ad4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:50.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.974+0000 7f9c32ffd640 1 -- 172.21.15.63:0/222482456 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c341f6890 con 0x7f9c340b4010 2024-05-20T18:26:50.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.974+0000 7f9c327fc640 1 --2- 172.21.15.63:0/222482456 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c340d14b0 0x7f9c340b2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:26:50.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.974+0000 7f9c337fe640 1 --2- 172.21.15.63:0/222482456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c34163a10 0x7f9c340ad4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:26:50.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.974+0000 7f9c32ffd640 1 --2- 172.21.15.63:0/222482456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c340b4010 0x7f9c341f6350 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c00e940 tx=0x7f9c2c00ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:26:50.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.974+0000 7f9c2b7fe640 1 -- 172.21.15.63:0/222482456 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c017070 con 0x7f9c340b4010 2024-05-20T18:26:50.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.974+0000 7f9c2b7fe640 1 -- 172.21.15.63:0/222482456 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c2c00ca00 con 0x7f9c340b4010 2024-05-20T18:26:50.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.974+0000 7f9c39302640 1 -- 172.21.15.63:0/222482456 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c340ab860 con 0x7f9c340b4010 2024-05-20T18:26:50.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.974+0000 7f9c2b7fe640 1 -- 172.21.15.63:0/222482456 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c00cdd0 con 0x7f9c340b4010 2024-05-20T18:26:50.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.974+0000 7f9c39302640 1 -- 172.21.15.63:0/222482456 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c340abe20 con 0x7f9c340b4010 2024-05-20T18:26:50.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.975+0000 7f9c2b7fe640 1 -- 172.21.15.63:0/222482456 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9c2c01a070 con 0x7f9c340b4010 2024-05-20T18:26:50.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.975+0000 7f9c2b7fe640 1 -- 172.21.15.63:0/222482456 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f9c2c013470 con 0x7f9c340b4010 2024-05-20T18:26:50.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.976+0000 7f9c39302640 1 -- 172.21.15.63:0/222482456 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c34075b00 con 0x7f9c340b4010 2024-05-20T18:26:50.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:50.978+0000 7f9c2b7fe640 1 -- 172.21.15.63:0/222482456 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f9c341f6890 con 0x7f9c340b4010 2024-05-20T18:26:51.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.052+0000 7f9c39302640 1 -- 172.21.15.63:0/222482456 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3385e4a6-b897-4645-86af-19fd42f79fa6", "id": 0} v 0) -- 0x7f9c340d3050 con 0x7f9c340b4010 2024-05-20T18:26:51.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.056+0000 7f9c2b7fe640 1 -- 172.21.15.63:0/222482456 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3385e4a6-b897-4645-86af-19fd42f79fa6", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f9c2c005990 con 0x7f9c340b4010 2024-05-20T18:26:51.055 INFO:teuthology.orchestra.run.smithi063.stdout:0 2024-05-20T18:26:51.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.057+0000 7f9c39302640 1 -- 172.21.15.63:0/222482456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c340b4010 msgr2=0x7f9c341f6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:51.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.057+0000 7f9c39302640 1 --2- 172.21.15.63:0/222482456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c340b4010 0x7f9c341f6350 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c00e940 tx=0x7f9c2c00ee00 comp rx=0 tx=0).stop 2024-05-20T18:26:51.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.057+0000 7f9c39302640 1 -- 172.21.15.63:0/222482456 shutdown_connections 2024-05-20T18:26:51.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.057+0000 7f9c39302640 1 --2- 172.21.15.63:0/222482456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c34163a10 0x7f9c340ad4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.057+0000 7f9c39302640 1 --2- 172.21.15.63:0/222482456 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c340d14b0 0x7f9c340b2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.057+0000 7f9c39302640 1 --2- 172.21.15.63:0/222482456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c340b4010 0x7f9c341f6350 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.057+0000 7f9c39302640 1 -- 172.21.15.63:0/222482456 >> 172.21.15.63:0/222482456 conn(0x7f9c3406d240 msgr2=0x7f9c3415d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:26:51.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.057+0000 7f9c39302640 1 -- 172.21.15.63:0/222482456 shutdown_connections 2024-05-20T18:26:51.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.057+0000 7f9c39302640 1 -- 172.21.15.63:0/222482456 wait complete. 2024-05-20T18:26:51.067 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd new 6a74555c-8e28-4b5a-9897-463604730539 1 2024-05-20T18:26:51.168 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-20T18:26:51.169+0000 7f3923cda040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-20T18:26:51.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.183+0000 7f9f25daa640 1 Processor -- start 2024-05-20T18:26:51.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.183+0000 7f9f25daa640 1 -- start start 2024-05-20T18:26:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.184+0000 7f9f25daa640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f200b0440 0x7f9f20163620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.184+0000 7f9f25daa640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9f200b18a0 0x7f9f200b1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.184+0000 7f9f25daa640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f200ae7b0 con 0x7f9f200b2bf0 2024-05-20T18:26:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.184+0000 7f9f25daa640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f9f200b0b70 con 0x7f9f200b18a0 2024-05-20T18:26:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.184+0000 7f9f25daa640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9f200b1410 con 0x7f9f200b0440 2024-05-20T18:26:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.184+0000 7f9f1effd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f200b0440 0x7f9f20163620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.184+0000 7f9f1effd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f200b0440 0x7f9f20163620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.63:52370/0 (socket says 172.21.15.63:52370) 2024-05-20T18:26:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.184+0000 7f9f1effd640 1 -- 172.21.15.63:0/2879737604 learned_addr learned my addr 172.21.15.63:0/2879737604 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-20T18:26:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.184+0000 7f9f1ffff640 1 --2- 172.21.15.63:0/2879737604 >> v2:172.21.15.5:3300/0 conn(0x7f9f200b18a0 0x7f9f200b1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.185+0000 7f9f1effd640 1 -- 172.21.15.63:0/2879737604 >> v2:172.21.15.5:3300/0 conn(0x7f9f200b18a0 msgr2=0x7f9f200b1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:26:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.185+0000 7f9f1effd640 1 --2- 172.21.15.63:0/2879737604 >> v2:172.21.15.5:3300/0 conn(0x7f9f200b18a0 0x7f9f200b1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.185+0000 7f9f1effd640 1 -- 172.21.15.63:0/2879737604 >> v1:172.21.15.5:6789/0 conn(0x7f9f200b2bf0 legacy=0x7f9f20162f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:51.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.185+0000 7f9f1effd640 1 -- 172.21.15.63:0/2879737604 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f200b3170 con 0x7f9f200b0440 2024-05-20T18:26:51.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.185+0000 7f9f1effd640 1 --2- 172.21.15.63:0/2879737604 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f200b0440 0x7f9f20163620 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9f14009870 tx=0x7f9f1402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67129e00d23d9f9d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:26:51.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.185+0000 7f9f1e7fc640 1 -- 172.21.15.63:0/2879737604 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f14037070 con 0x7f9f200b0440 2024-05-20T18:26:51.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.186+0000 7f9f1e7fc640 1 -- 172.21.15.63:0/2879737604 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f14032440 con 0x7f9f200b0440 2024-05-20T18:26:51.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.186+0000 7f9f1e7fc640 1 -- 172.21.15.63:0/2879737604 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f14032810 con 0x7f9f200b0440 2024-05-20T18:26:51.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.186+0000 7f9f25daa640 1 -- 172.21.15.63:0/2879737604 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f200b0440 msgr2=0x7f9f20163620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:51.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.186+0000 7f9f25daa640 1 --2- 172.21.15.63:0/2879737604 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f200b0440 0x7f9f20163620 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9f14009870 tx=0x7f9f1402a270 comp rx=0 tx=0).stop 2024-05-20T18:26:51.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.186+0000 7f9f25daa640 1 -- 172.21.15.63:0/2879737604 shutdown_connections 2024-05-20T18:26:51.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.186+0000 7f9f25daa640 1 --2- 172.21.15.63:0/2879737604 >> v2:172.21.15.5:3300/0 conn(0x7f9f200b18a0 0x7f9f200b1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.186+0000 7f9f25daa640 1 --2- 172.21.15.63:0/2879737604 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f200b0440 0x7f9f20163620 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.186+0000 7f9f25daa640 1 -- 172.21.15.63:0/2879737604 >> 172.21.15.63:0/2879737604 conn(0x7f9f200b6620 msgr2=0x7f9f200b6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:26:51.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.186+0000 7f9f25daa640 1 -- 172.21.15.63:0/2879737604 shutdown_connections 2024-05-20T18:26:51.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.186+0000 7f9f25daa640 1 -- 172.21.15.63:0/2879737604 wait complete. 2024-05-20T18:26:51.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.188+0000 7f9f25daa640 1 Processor -- start 2024-05-20T18:26:51.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.188+0000 7f9f25daa640 1 -- start start 2024-05-20T18:26:51.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.190+0000 7f9f25daa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f200b0440 0x7f9f20154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:51.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.190+0000 7f9f25daa640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f200b18a0 0x7f9f2006c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:51.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.190+0000 7f9f25daa640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f200b2bf0 0x7f9f2006cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:51.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.190+0000 7f9f25daa640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2006df70 con 0x7f9f200b0440 2024-05-20T18:26:51.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.190+0000 7f9f25daa640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9f200b10a0 con 0x7f9f200b18a0 2024-05-20T18:26:51.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.190+0000 7f9f25daa640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9f20163b60 con 0x7f9f200b2bf0 2024-05-20T18:26:51.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.190+0000 7f9f1ffff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f200b2bf0 0x7f9f2006cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:51.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.190+0000 7f9f1effd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f200b18a0 0x7f9f2006c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:51.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.190+0000 7f9f1ffff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f200b2bf0 0x7f9f2006cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.63:52382/0 (socket says 172.21.15.63:52382) 2024-05-20T18:26:51.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.190+0000 7f9f1effd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f200b18a0 0x7f9f2006c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37492/0 (socket says 172.21.15.63:37492) 2024-05-20T18:26:51.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.190+0000 7f9f1ffff640 1 -- 172.21.15.63:0/38413869 learned_addr learned my addr 172.21.15.63:0/38413869 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-20T18:26:51.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.190+0000 7f9f1f7fe640 1 --2- 172.21.15.63:0/38413869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f200b0440 0x7f9f20154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:51.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.190+0000 7f9f1f7fe640 1 -- 172.21.15.63:0/38413869 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f200b2bf0 msgr2=0x7f9f2006cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:51.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.190+0000 7f9f1f7fe640 1 --2- 172.21.15.63:0/38413869 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f200b2bf0 0x7f9f2006cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.190+0000 7f9f1f7fe640 1 -- 172.21.15.63:0/38413869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f200b18a0 msgr2=0x7f9f2006c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:51.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.190+0000 7f9f1f7fe640 1 --2- 172.21.15.63:0/38413869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f200b18a0 0x7f9f2006c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.190+0000 7f9f1f7fe640 1 -- 172.21.15.63:0/38413869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f201f5790 con 0x7f9f200b0440 2024-05-20T18:26:51.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.190+0000 7f9f1ffff640 1 --2- 172.21.15.63:0/38413869 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f200b2bf0 0x7f9f2006cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:26:51.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.190+0000 7f9f1effd640 1 --2- 172.21.15.63:0/38413869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f200b18a0 0x7f9f2006c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:26:51.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.191+0000 7f9f1f7fe640 1 --2- 172.21.15.63:0/38413869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f200b0440 0x7f9f20154010 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000c3c0 tx=0x7f9f1000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:26:51.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.191+0000 7f9f07fff640 1 -- 172.21.15.63:0/38413869 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f100052d0 con 0x7f9f200b0440 2024-05-20T18:26:51.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.191+0000 7f9f07fff640 1 -- 172.21.15.63:0/38413869 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f10005490 con 0x7f9f200b0440 2024-05-20T18:26:51.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.191+0000 7f9f25daa640 1 -- 172.21.15.63:0/38413869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f201529c0 con 0x7f9f200b0440 2024-05-20T18:26:51.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.191+0000 7f9f07fff640 1 -- 172.21.15.63:0/38413869 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10007260 con 0x7f9f200b0440 2024-05-20T18:26:51.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.191+0000 7f9f25daa640 1 -- 172.21.15.63:0/38413869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f20152f80 con 0x7f9f200b0440 2024-05-20T18:26:51.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.191+0000 7f9f07fff640 1 -- 172.21.15.63:0/38413869 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9f10007400 con 0x7f9f200b0440 2024-05-20T18:26:51.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.191+0000 7f9f07fff640 1 -- 172.21.15.63:0/38413869 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f9f10003b20 con 0x7f9f200b0440 2024-05-20T18:26:51.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.192+0000 7f9f25daa640 1 -- 172.21.15.63:0/38413869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f20075b00 con 0x7f9f200b0440 2024-05-20T18:26:51.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.196+0000 7f9f07fff640 1 -- 172.21.15.63:0/38413869 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f9f10003d70 con 0x7f9f200b0440 2024-05-20T18:26:51.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.272+0000 7f9f25daa640 1 -- 172.21.15.63:0/38413869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6a74555c-8e28-4b5a-9897-463604730539", "id": 1} v 0) -- 0x7f9f200cdb00 con 0x7f9f200b0440 2024-05-20T18:26:51.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.274+0000 7f9f07fff640 1 -- 172.21.15.63:0/38413869 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6a74555c-8e28-4b5a-9897-463604730539", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f9f100067d0 con 0x7f9f200b0440 2024-05-20T18:26:51.273 INFO:teuthology.orchestra.run.smithi063.stdout:1 2024-05-20T18:26:51.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.275+0000 7f9f25daa640 1 -- 172.21.15.63:0/38413869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f200b0440 msgr2=0x7f9f20154010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:51.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.275+0000 7f9f25daa640 1 --2- 172.21.15.63:0/38413869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f200b0440 0x7f9f20154010 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000c3c0 tx=0x7f9f1000c880 comp rx=0 tx=0).stop 2024-05-20T18:26:51.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.275+0000 7f9f25daa640 1 -- 172.21.15.63:0/38413869 shutdown_connections 2024-05-20T18:26:51.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.275+0000 7f9f25daa640 1 --2- 172.21.15.63:0/38413869 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f200b2bf0 0x7f9f2006cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.275+0000 7f9f25daa640 1 --2- 172.21.15.63:0/38413869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f200b18a0 0x7f9f2006c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.275+0000 7f9f25daa640 1 --2- 172.21.15.63:0/38413869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f200b0440 0x7f9f20154010 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.275+0000 7f9f25daa640 1 -- 172.21.15.63:0/38413869 >> 172.21.15.63:0/38413869 conn(0x7f9f200b6620 msgr2=0x7f9f200b34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:26:51.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.275+0000 7f9f25daa640 1 -- 172.21.15.63:0/38413869 shutdown_connections 2024-05-20T18:26:51.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.275+0000 7f9f25daa640 1 -- 172.21.15.63:0/38413869 wait complete. 2024-05-20T18:26:51.285 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd new 2c2a1831-7d9a-4a1c-b102-6181c2f85bda 2 2024-05-20T18:26:51.332 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-20T18:26:51.333+0000 7f3923cda040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-20T18:26:51.391 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-20T18:26:51.392+0000 7f3923cda040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-20T18:26:51.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.401+0000 7f155b53e640 1 Processor -- start 2024-05-20T18:26:51.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.401+0000 7f155b53e640 1 -- start start 2024-05-20T18:26:51.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.402+0000 7f155b53e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f155415d000 0x7f15540b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:51.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.402+0000 7f155b53e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f155406c630 0x7f155406ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:51.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.402+0000 7f155b53e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f155406cf80 con 0x7f1554160510 2024-05-20T18:26:51.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.402+0000 7f155b53e640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f15540cc680 con 0x7f155406c630 2024-05-20T18:26:51.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.402+0000 7f155b53e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f155415d410 con 0x7f155415d000 2024-05-20T18:26:51.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.402+0000 7f1559adb640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f155406c630 0x7f155406ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:51.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.402+0000 7f1558ad9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f155415d000 0x7f15540b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:51.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.402+0000 7f1559adb640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f155406c630 0x7f155406ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37506/0 (socket says 172.21.15.63:37506) 2024-05-20T18:26:51.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.402+0000 7f1558ad9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f155415d000 0x7f15540b2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.63:52392/0 (socket says 172.21.15.63:52392) 2024-05-20T18:26:51.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.402+0000 7f1559adb640 1 -- 172.21.15.63:0/2354940828 learned_addr learned my addr 172.21.15.63:0/2354940828 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-20T18:26:51.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.403+0000 7f1558ad9640 1 -- 172.21.15.63:0/2354940828 >> v2:172.21.15.63:3300/0 conn(0x7f155406c630 msgr2=0x7f155406ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:26:51.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.403+0000 7f1558ad9640 1 --2- 172.21.15.63:0/2354940828 >> v2:172.21.15.63:3300/0 conn(0x7f155406c630 0x7f155406ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.403+0000 7f1558ad9640 1 -- 172.21.15.63:0/2354940828 >> v1:172.21.15.63:6789/0 conn(0x7f1554160510 legacy=0x7f15540cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:51.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.403+0000 7f1558ad9640 1 -- 172.21.15.63:0/2354940828 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f155409ae70 con 0x7f155415d000 2024-05-20T18:26:51.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.403+0000 7f1559adb640 1 --2- 172.21.15.63:0/2354940828 >> v2:172.21.15.63:3300/0 conn(0x7f155406c630 0x7f155406ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:26:51.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.403+0000 7f1558ad9640 1 --2- 172.21.15.63:0/2354940828 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f155415d000 0x7f15540b2d40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f155409b040 tx=0x7f154402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f86aa5b6e1a45cd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:26:51.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.404+0000 7f154bfff640 1 -- 172.21.15.63:0/2354940828 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1544037070 con 0x7f155415d000 2024-05-20T18:26:51.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.404+0000 7f154bfff640 1 -- 172.21.15.63:0/2354940828 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1544032440 con 0x7f155415d000 2024-05-20T18:26:51.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.404+0000 7f154bfff640 1 -- 172.21.15.63:0/2354940828 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1544032810 con 0x7f155415d000 2024-05-20T18:26:51.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.405+0000 7f155b53e640 1 -- 172.21.15.63:0/2354940828 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f155415d000 msgr2=0x7f15540b2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:51.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.405+0000 7f155b53e640 1 --2- 172.21.15.63:0/2354940828 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f155415d000 0x7f15540b2d40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f155409b040 tx=0x7f154402a180 comp rx=0 tx=0).stop 2024-05-20T18:26:51.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.405+0000 7f155b53e640 1 -- 172.21.15.63:0/2354940828 shutdown_connections 2024-05-20T18:26:51.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.405+0000 7f155b53e640 1 --2- 172.21.15.63:0/2354940828 >> v2:172.21.15.63:3300/0 conn(0x7f155406c630 0x7f155406ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.405+0000 7f155b53e640 1 --2- 172.21.15.63:0/2354940828 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f155415d000 0x7f15540b2d40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.405+0000 7f155b53e640 1 -- 172.21.15.63:0/2354940828 >> 172.21.15.63:0/2354940828 conn(0x7f15540b6620 msgr2=0x7f15540b6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:26:51.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.405+0000 7f155b53e640 1 -- 172.21.15.63:0/2354940828 shutdown_connections 2024-05-20T18:26:51.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.405+0000 7f155b53e640 1 -- 172.21.15.63:0/2354940828 wait complete. 2024-05-20T18:26:51.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.407+0000 7f155b53e640 1 Processor -- start 2024-05-20T18:26:51.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.407+0000 7f155b53e640 1 -- start start 2024-05-20T18:26:51.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.408+0000 7f155b53e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f155406c630 0x7f15540d1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:51.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.408+0000 7f155b53e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f155415d000 0x7f15540ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:51.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.408+0000 7f155b53e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15541f5560 0x7f15541f8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:51.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.408+0000 7f155b53e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f15541609d0 con 0x7f155415d000 2024-05-20T18:26:51.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.408+0000 7f155b53e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f15540ce240 con 0x7f15541f5560 2024-05-20T18:26:51.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.408+0000 7f155b53e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f155409b3f0 con 0x7f155406c630 2024-05-20T18:26:51.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.408+0000 7f15592da640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f155406c630 0x7f15540d1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:51.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.408+0000 7f1559adb640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15541f5560 0x7f15541f8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:51.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.408+0000 7f15592da640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f155406c630 0x7f15540d1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.63:52408/0 (socket says 172.21.15.63:52408) 2024-05-20T18:26:51.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.408+0000 7f15592da640 1 -- 172.21.15.63:0/3869638993 learned_addr learned my addr 172.21.15.63:0/3869638993 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-20T18:26:51.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.409+0000 7f1558ad9640 1 --2- 172.21.15.63:0/3869638993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f155415d000 0x7f15540ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:51.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.409+0000 7f1559adb640 1 -- 172.21.15.63:0/3869638993 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f155406c630 msgr2=0x7f15540d1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:51.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.409+0000 7f1559adb640 1 --2- 172.21.15.63:0/3869638993 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f155406c630 0x7f15540d1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.409+0000 7f1559adb640 1 -- 172.21.15.63:0/3869638993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f155415d000 msgr2=0x7f15540ae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:51.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.409+0000 7f1559adb640 1 --2- 172.21.15.63:0/3869638993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f155415d000 0x7f15540ae7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.409+0000 7f1559adb640 1 -- 172.21.15.63:0/3869638993 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15540cbab0 con 0x7f15541f5560 2024-05-20T18:26:51.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.409+0000 7f1559adb640 1 --2- 172.21.15.63:0/3869638993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15541f5560 0x7f15541f8f00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f154c00ef90 tx=0x7f154c00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:26:51.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.409+0000 7f1549ffb640 1 -- 172.21.15.63:0/3869638993 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c00cac0 con 0x7f15541f5560 2024-05-20T18:26:51.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.409+0000 7f1549ffb640 1 -- 172.21.15.63:0/3869638993 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f154c00cc80 con 0x7f15541f5560 2024-05-20T18:26:51.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.409+0000 7f1549ffb640 1 -- 172.21.15.63:0/3869638993 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c003bd0 con 0x7f15541f5560 2024-05-20T18:26:51.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.409+0000 7f155b53e640 1 -- 172.21.15.63:0/3869638993 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15540d24d0 con 0x7f15541f5560 2024-05-20T18:26:51.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.409+0000 7f1549ffb640 1 -- 172.21.15.63:0/3869638993 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f154c01b020 con 0x7f15541f5560 2024-05-20T18:26:51.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.409+0000 7f155b53e640 1 -- 172.21.15.63:0/3869638993 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15540d11d0 con 0x7f15541f5560 2024-05-20T18:26:51.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.410+0000 7f1549ffb640 1 -- 172.21.15.63:0/3869638993 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f154c003d70 con 0x7f15541f5560 2024-05-20T18:26:51.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.410+0000 7f155b53e640 1 -- 172.21.15.63:0/3869638993 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1554075b00 con 0x7f15541f5560 2024-05-20T18:26:51.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.413+0000 7f1549ffb640 1 -- 172.21.15.63:0/3869638993 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f15540cbab0 con 0x7f15541f5560 2024-05-20T18:26:51.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.486+0000 7f155b53e640 1 -- 172.21.15.63:0/3869638993 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2c2a1831-7d9a-4a1c-b102-6181c2f85bda", "id": 2} v 0) -- 0x7f155409a6a0 con 0x7f15541f5560 2024-05-20T18:26:51.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.490+0000 7f1549ffb640 1 -- 172.21.15.63:0/3869638993 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2c2a1831-7d9a-4a1c-b102-6181c2f85bda", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f154c010040 con 0x7f15541f5560 2024-05-20T18:26:51.490 INFO:teuthology.orchestra.run.smithi063.stdout:2 2024-05-20T18:26:51.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.491+0000 7f155b53e640 1 -- 172.21.15.63:0/3869638993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15541f5560 msgr2=0x7f15541f8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:51.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.491+0000 7f155b53e640 1 --2- 172.21.15.63:0/3869638993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15541f5560 0x7f15541f8f00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f154c00ef90 tx=0x7f154c00c490 comp rx=0 tx=0).stop 2024-05-20T18:26:51.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.491+0000 7f155b53e640 1 -- 172.21.15.63:0/3869638993 shutdown_connections 2024-05-20T18:26:51.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.491+0000 7f155b53e640 1 --2- 172.21.15.63:0/3869638993 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15541f5560 0x7f15541f8f00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.491+0000 7f155b53e640 1 --2- 172.21.15.63:0/3869638993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f155415d000 0x7f15540ae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.491+0000 7f155b53e640 1 --2- 172.21.15.63:0/3869638993 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f155406c630 0x7f15540d1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.491+0000 7f155b53e640 1 -- 172.21.15.63:0/3869638993 >> 172.21.15.63:0/3869638993 conn(0x7f15540b6620 msgr2=0x7f15540cc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:26:51.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.491+0000 7f155b53e640 1 -- 172.21.15.63:0/3869638993 shutdown_connections 2024-05-20T18:26:51.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.491+0000 7f155b53e640 1 -- 172.21.15.63:0/3869638993 wait complete. 2024-05-20T18:26:51.501 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd new ef43f394-7be0-4d4f-ab20-96ef28024237 3 2024-05-20T18:26:51.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.616+0000 7fe3405b5640 1 Processor -- start 2024-05-20T18:26:51.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.616+0000 7fe3405b5640 1 -- start start 2024-05-20T18:26:51.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.618+0000 7fe3405b5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe338159ca0 0x7fe3380af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:51.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.618+0000 7fe3405b5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe3380b23f0 0x7fe33815c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:51.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.618+0000 7fe3405b5640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe33809a740 con 0x7fe3380b1650 2024-05-20T18:26:51.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.618+0000 7fe3405b5640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe33809a4f0 con 0x7fe338159ca0 2024-05-20T18:26:51.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.618+0000 7fe3405b5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe33809b350 con 0x7fe3380b23f0 2024-05-20T18:26:51.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.618+0000 7fe33eb52640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe3380b23f0 0x7fe33815c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:51.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.618+0000 7fe33db50640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fe3380b1650 0x7fe3380b1ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.63:59398/0 (socket says 172.21.15.63:59398) 2024-05-20T18:26:51.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.618+0000 7fe33db50640 1 -- 172.21.15.63:0/2058754602 learned_addr learned my addr 172.21.15.63:0/2058754602 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-05-20T18:26:51.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.618+0000 7fe33e351640 1 --2- 172.21.15.63:0/2058754602 >> v2:172.21.15.5:3300/0 conn(0x7fe338159ca0 0x7fe3380af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:51.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.618+0000 7fe33d34f640 1 -- 172.21.15.63:0/2058754602 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4174323592 0 0) 0x7fe33809a740 con 0x7fe3380b1650 2024-05-20T18:26:51.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.618+0000 7fe33eb52640 1 -- 172.21.15.63:0/2058754602 >> v2:172.21.15.5:3300/0 conn(0x7fe338159ca0 msgr2=0x7fe3380af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:26:51.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.618+0000 7fe33eb52640 1 --2- 172.21.15.63:0/2058754602 >> v2:172.21.15.5:3300/0 conn(0x7fe338159ca0 0x7fe3380af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.618+0000 7fe33eb52640 1 -- 172.21.15.63:0/2058754602 >> v1:172.21.15.63:6789/0 conn(0x7fe3380b1650 legacy=0x7fe3380b1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:51.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.618+0000 7fe33eb52640 1 -- 172.21.15.63:0/2058754602 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe33809aa20 con 0x7fe3380b23f0 2024-05-20T18:26:51.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.619+0000 7fe33eb52640 1 --2- 172.21.15.63:0/2058754602 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe3380b23f0 0x7fe33815c760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe33000bfa0 tx=0x7fe33002c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39032612976536a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:26:51.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.619+0000 7fe33d34f640 1 -- 172.21.15.63:0/2058754602 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330038070 con 0x7fe3380b23f0 2024-05-20T18:26:51.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.619+0000 7fe33d34f640 1 -- 172.21.15.63:0/2058754602 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe330009540 con 0x7fe3380b23f0 2024-05-20T18:26:51.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.619+0000 7fe33d34f640 1 -- 172.21.15.63:0/2058754602 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330009910 con 0x7fe3380b23f0 2024-05-20T18:26:51.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.620+0000 7fe3405b5640 1 -- 172.21.15.63:0/2058754602 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe3380b23f0 msgr2=0x7fe33815c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:51.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.620+0000 7fe3405b5640 1 --2- 172.21.15.63:0/2058754602 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe3380b23f0 0x7fe33815c760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe33000bfa0 tx=0x7fe33002c7e0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.620+0000 7fe3405b5640 1 -- 172.21.15.63:0/2058754602 shutdown_connections 2024-05-20T18:26:51.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.620+0000 7fe3405b5640 1 --2- 172.21.15.63:0/2058754602 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe3380b23f0 0x7fe33815c760 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.620+0000 7fe3405b5640 1 --2- 172.21.15.63:0/2058754602 >> v2:172.21.15.5:3300/0 conn(0x7fe338159ca0 0x7fe3380af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.620+0000 7fe3405b5640 1 -- 172.21.15.63:0/2058754602 >> 172.21.15.63:0/2058754602 conn(0x7fe3380b6620 msgr2=0x7fe3380b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:26:51.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.620+0000 7fe3405b5640 1 -- 172.21.15.63:0/2058754602 shutdown_connections 2024-05-20T18:26:51.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.620+0000 7fe3405b5640 1 -- 172.21.15.63:0/2058754602 wait complete. 2024-05-20T18:26:51.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.622+0000 7fe3405b5640 1 Processor -- start 2024-05-20T18:26:51.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.622+0000 7fe3405b5640 1 -- start start 2024-05-20T18:26:51.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.624+0000 7fe3405b5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe3380b1650 0x7fe338154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:51.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.624+0000 7fe3405b5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3380b23f0 0x7fe338070820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:51.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.624+0000 7fe3405b5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe338159ca0 0x7fe338070d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:51.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.624+0000 7fe3405b5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe33809afe0 con 0x7fe338159ca0 2024-05-20T18:26:51.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.624+0000 7fe3405b5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe3380af770 con 0x7fe3380b23f0 2024-05-20T18:26:51.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.624+0000 7fe3405b5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe33815cca0 con 0x7fe3380b1650 2024-05-20T18:26:51.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.624+0000 7fe33e351640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe3380b1650 0x7fe338154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:51.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.624+0000 7fe33e351640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe3380b1650 0x7fe338154790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.63:52432/0 (socket says 172.21.15.63:52432) 2024-05-20T18:26:51.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.624+0000 7fe33e351640 1 -- 172.21.15.63:0/2213277323 learned_addr learned my addr 172.21.15.63:0/2213277323 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-20T18:26:51.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.624+0000 7fe33db50640 1 --2- 172.21.15.63:0/2213277323 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3380b23f0 0x7fe338070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:51.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.624+0000 7fe33eb52640 1 --2- 172.21.15.63:0/2213277323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe338159ca0 0x7fe338070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:51.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.624+0000 7fe33db50640 1 -- 172.21.15.63:0/2213277323 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe3380b1650 msgr2=0x7fe338154790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:51.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.624+0000 7fe33db50640 1 --2- 172.21.15.63:0/2213277323 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe3380b1650 0x7fe338154790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.624+0000 7fe33db50640 1 -- 172.21.15.63:0/2213277323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe338159ca0 msgr2=0x7fe338070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:51.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.624+0000 7fe33db50640 1 --2- 172.21.15.63:0/2213277323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe338159ca0 0x7fe338070d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.624+0000 7fe33db50640 1 -- 172.21.15.63:0/2213277323 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe338156a90 con 0x7fe3380b23f0 2024-05-20T18:26:51.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.624+0000 7fe33e351640 1 --2- 172.21.15.63:0/2213277323 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe3380b1650 0x7fe338154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:26:51.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.624+0000 7fe33db50640 1 --2- 172.21.15.63:0/2213277323 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3380b23f0 0x7fe338070820 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe33400c440 tx=0x7fe33400c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:26:51.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.624+0000 7fe32effd640 1 -- 172.21.15.63:0/2213277323 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3340074e0 con 0x7fe3380b23f0 2024-05-20T18:26:51.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.625+0000 7fe32effd640 1 -- 172.21.15.63:0/2213277323 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3340047e0 con 0x7fe3380b23f0 2024-05-20T18:26:51.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.625+0000 7fe32effd640 1 -- 172.21.15.63:0/2213277323 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334004bd0 con 0x7fe3380b23f0 2024-05-20T18:26:51.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.625+0000 7fe3405b5640 1 -- 172.21.15.63:0/2213277323 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3381587a0 con 0x7fe3380b23f0 2024-05-20T18:26:51.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.625+0000 7fe3405b5640 1 -- 172.21.15.63:0/2213277323 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3381581e0 con 0x7fe3380b23f0 2024-05-20T18:26:51.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.625+0000 7fe32effd640 1 -- 172.21.15.63:0/2213277323 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe334011070 con 0x7fe3380b23f0 2024-05-20T18:26:51.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.625+0000 7fe32effd640 1 -- 172.21.15.63:0/2213277323 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fe334013850 con 0x7fe3380b23f0 2024-05-20T18:26:51.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.626+0000 7fe3405b5640 1 -- 172.21.15.63:0/2213277323 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe338157c50 con 0x7fe3380b23f0 2024-05-20T18:26:51.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.628+0000 7fe32effd640 1 -- 172.21.15.63:0/2213277323 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe334004dc0 con 0x7fe3380b23f0 2024-05-20T18:26:51.651 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-20T18:26:51.652+0000 7fd1b5923040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-20T18:26:51.667 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-20T18:26:51.668+0000 7f3923cda040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-20T18:26:51.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.702+0000 7fe3405b5640 1 -- 172.21.15.63:0/2213277323 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ef43f394-7be0-4d4f-ab20-96ef28024237", "id": 3} v 0) -- 0x7fe338075b00 con 0x7fe3380b23f0 2024-05-20T18:26:51.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.706+0000 7fe32effd640 1 -- 172.21.15.63:0/2213277323 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ef43f394-7be0-4d4f-ab20-96ef28024237", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fe334013c70 con 0x7fe3380b23f0 2024-05-20T18:26:51.705 INFO:teuthology.orchestra.run.smithi063.stdout:3 2024-05-20T18:26:51.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.706+0000 7fe3405b5640 1 -- 172.21.15.63:0/2213277323 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3380b23f0 msgr2=0x7fe338070820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:51.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.706+0000 7fe3405b5640 1 --2- 172.21.15.63:0/2213277323 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3380b23f0 0x7fe338070820 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe33400c440 tx=0x7fe33400c900 comp rx=0 tx=0).stop 2024-05-20T18:26:51.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.707+0000 7fe3405b5640 1 -- 172.21.15.63:0/2213277323 shutdown_connections 2024-05-20T18:26:51.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.707+0000 7fe3405b5640 1 --2- 172.21.15.63:0/2213277323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe338159ca0 0x7fe338070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.707+0000 7fe3405b5640 1 --2- 172.21.15.63:0/2213277323 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3380b23f0 0x7fe338070820 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.707+0000 7fe3405b5640 1 --2- 172.21.15.63:0/2213277323 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe3380b1650 0x7fe338154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.707+0000 7fe3405b5640 1 -- 172.21.15.63:0/2213277323 >> 172.21.15.63:0/2213277323 conn(0x7fe3380b6620 msgr2=0x7fe33806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:26:51.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.707+0000 7fe3405b5640 1 -- 172.21.15.63:0/2213277323 shutdown_connections 2024-05-20T18:26:51.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.707+0000 7fe3405b5640 1 -- 172.21.15.63:0/2213277323 wait complete. 2024-05-20T18:26:51.716 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-20T18:26:51.717+0000 7f3923cda040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-20T18:26:51.717 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd new 07a9fcec-77b3-45ca-b075-8fb0b32754d6 4 2024-05-20T18:26:51.742 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-20T18:26:51.744+0000 7fd1b5923040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-20T18:26:51.787 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-20T18:26:51.788+0000 7fd1b5923040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-20T18:26:51.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.833+0000 7f3236796640 1 Processor -- start 2024-05-20T18:26:51.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.833+0000 7f3236796640 1 -- start start 2024-05-20T18:26:51.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.834+0000 7f3236796640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f323006db00 0x7f323015f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:51.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.834+0000 7f3236796640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f32300af0e0 0x7f32300af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:51.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.834+0000 7f3236796640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f323009a4f0 0x7f323009a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:51.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.834+0000 7f3236796640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f32300b1af0 con 0x7f32300af0e0 2024-05-20T18:26:51.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.834+0000 7f3236796640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f32300afa30 con 0x7f323006db00 2024-05-20T18:26:51.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.834+0000 7f3236796640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3230160080 con 0x7f323009a4f0 2024-05-20T18:26:51.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.835+0000 7f322ffff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f323006db00 0x7f323015f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:51.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.835+0000 7f3234d33640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f323009a4f0 0x7f323009a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:51.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.835+0000 7f322ffff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f323006db00 0x7f323015f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37548/0 (socket says 172.21.15.63:37548) 2024-05-20T18:26:51.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.835+0000 7f322ffff640 1 -- 172.21.15.63:0/3979854850 learned_addr learned my addr 172.21.15.63:0/3979854850 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-20T18:26:51.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.835+0000 7f322f7fe640 1 --2- 172.21.15.63:0/3979854850 >> v2:172.21.15.5:3300/0 conn(0x7f32300af0e0 0x7f32300af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:51.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.835+0000 7f322ffff640 1 -- 172.21.15.63:0/3979854850 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f323009a4f0 msgr2=0x7f323009a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:26:51.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.835+0000 7f322ffff640 1 --2- 172.21.15.63:0/3979854850 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f323009a4f0 0x7f323009a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.835+0000 7f322ffff640 1 -- 172.21.15.63:0/3979854850 >> v2:172.21.15.5:3300/0 conn(0x7f32300af0e0 msgr2=0x7f32300af4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:26:51.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.835+0000 7f322ffff640 1 --2- 172.21.15.63:0/3979854850 >> v2:172.21.15.5:3300/0 conn(0x7f32300af0e0 0x7f32300af4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.835+0000 7f322ffff640 1 -- 172.21.15.63:0/3979854850 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3230160360 con 0x7f323006db00 2024-05-20T18:26:51.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.835+0000 7f3234d33640 1 --2- 172.21.15.63:0/3979854850 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f323009a4f0 0x7f323009a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:26:51.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.836+0000 7f322ffff640 1 --2- 172.21.15.63:0/3979854850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f323006db00 0x7f323015f710 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3230074f80 tx=0x7f322002a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b526d8243dd7ef0f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:26:51.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.836+0000 7f322effd640 1 -- 172.21.15.63:0/3979854850 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3220037070 con 0x7f323006db00 2024-05-20T18:26:51.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.836+0000 7f322effd640 1 -- 172.21.15.63:0/3979854850 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3220032440 con 0x7f323006db00 2024-05-20T18:26:51.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.836+0000 7f322effd640 1 -- 172.21.15.63:0/3979854850 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3220032810 con 0x7f323006db00 2024-05-20T18:26:51.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.837+0000 7f3236796640 1 -- 172.21.15.63:0/3979854850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f323006db00 msgr2=0x7f323015f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:51.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.837+0000 7f3236796640 1 --2- 172.21.15.63:0/3979854850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f323006db00 0x7f323015f710 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3230074f80 tx=0x7f322002a1e0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.837+0000 7f3236796640 1 -- 172.21.15.63:0/3979854850 shutdown_connections 2024-05-20T18:26:51.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.837+0000 7f3236796640 1 --2- 172.21.15.63:0/3979854850 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f323009a4f0 0x7f323009a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.837+0000 7f3236796640 1 --2- 172.21.15.63:0/3979854850 >> v2:172.21.15.5:3300/0 conn(0x7f32300af0e0 0x7f32300af4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.837+0000 7f3236796640 1 --2- 172.21.15.63:0/3979854850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f323006db00 0x7f323015f710 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.837+0000 7f3236796640 1 -- 172.21.15.63:0/3979854850 >> 172.21.15.63:0/3979854850 conn(0x7f32300b6620 msgr2=0x7f32300b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:26:51.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.837+0000 7f3236796640 1 -- 172.21.15.63:0/3979854850 shutdown_connections 2024-05-20T18:26:51.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.837+0000 7f3236796640 1 -- 172.21.15.63:0/3979854850 wait complete. 2024-05-20T18:26:51.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.839+0000 7f3236796640 1 Processor -- start 2024-05-20T18:26:51.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.839+0000 7f3236796640 1 -- start start 2024-05-20T18:26:51.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.840+0000 7f3236796640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f323006db00 0x7f3230151990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:51.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.840+0000 7f3236796640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f323009a4f0 0x7f323009e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:51.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.840+0000 7f3236796640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32300af0e0 0x7f3230157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:51.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.840+0000 7f3236796640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f323015fc50 con 0x7f323009a4f0 2024-05-20T18:26:51.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.840+0000 7f3236796640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f32300b1780 con 0x7f32300af0e0 2024-05-20T18:26:51.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.840+0000 7f3236796640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f323015fdd0 con 0x7f323006db00 2024-05-20T18:26:51.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.840+0000 7f322ffff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f323006db00 0x7f3230151990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:51.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.840+0000 7f322ffff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f323006db00 0x7f3230151990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.63:52454/0 (socket says 172.21.15.63:52454) 2024-05-20T18:26:51.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.840+0000 7f322ffff640 1 -- 172.21.15.63:0/1355608330 learned_addr learned my addr 172.21.15.63:0/1355608330 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-20T18:26:51.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.840+0000 7f3234d33640 1 --2- 172.21.15.63:0/1355608330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32300af0e0 0x7f3230157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:51.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.840+0000 7f322f7fe640 1 --2- 172.21.15.63:0/1355608330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f323009a4f0 0x7f323009e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:51.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.841+0000 7f322ffff640 1 -- 172.21.15.63:0/1355608330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32300af0e0 msgr2=0x7f3230157d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:51.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.841+0000 7f322ffff640 1 --2- 172.21.15.63:0/1355608330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32300af0e0 0x7f3230157d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.841+0000 7f322ffff640 1 -- 172.21.15.63:0/1355608330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f323009a4f0 msgr2=0x7f323009e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:51.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.841+0000 7f322ffff640 1 --2- 172.21.15.63:0/1355608330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f323009a4f0 0x7f323009e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.841+0000 7f322ffff640 1 -- 172.21.15.63:0/1355608330 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f323009f280 con 0x7f323006db00 2024-05-20T18:26:51.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.841+0000 7f322ffff640 1 --2- 172.21.15.63:0/1355608330 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f323006db00 0x7f3230151990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f322002a220 tx=0x7f32200046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:26:51.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.841+0000 7f322cff9640 1 -- 172.21.15.63:0/1355608330 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3220041020 con 0x7f323006db00 2024-05-20T18:26:51.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.841+0000 7f322cff9640 1 -- 172.21.15.63:0/1355608330 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f322003d640 con 0x7f323006db00 2024-05-20T18:26:51.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.841+0000 7f322cff9640 1 -- 172.21.15.63:0/1355608330 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3220037040 con 0x7f323006db00 2024-05-20T18:26:51.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.841+0000 7f3236796640 1 -- 172.21.15.63:0/1355608330 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f323009f560 con 0x7f323006db00 2024-05-20T18:26:51.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.841+0000 7f3236796640 1 -- 172.21.15.63:0/1355608330 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f32301536c0 con 0x7f323006db00 2024-05-20T18:26:51.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.841+0000 7f322cff9640 1 -- 172.21.15.63:0/1355608330 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3220044020 con 0x7f323006db00 2024-05-20T18:26:51.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.841+0000 7f322cff9640 1 -- 172.21.15.63:0/1355608330 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f3220046b30 con 0x7f323006db00 2024-05-20T18:26:51.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.842+0000 7f3236796640 1 -- 172.21.15.63:0/1355608330 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3230075b00 con 0x7f323006db00 2024-05-20T18:26:51.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.844+0000 7f322cff9640 1 -- 172.21.15.63:0/1355608330 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f322004b030 con 0x7f323006db00 2024-05-20T18:26:51.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.919+0000 7f3236796640 1 -- 172.21.15.63:0/1355608330 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "osd new", "uuid": "07a9fcec-77b3-45ca-b075-8fb0b32754d6", "id": 4} v 0) -- 0x7f323006e530 con 0x7f323006db00 2024-05-20T18:26:51.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.922+0000 7f322cff9640 1 -- 172.21.15.63:0/1355608330 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "07a9fcec-77b3-45ca-b075-8fb0b32754d6", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f3220046e10 con 0x7f323006db00 2024-05-20T18:26:51.921 INFO:teuthology.orchestra.run.smithi063.stdout:4 2024-05-20T18:26:51.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.923+0000 7f3236796640 1 -- 172.21.15.63:0/1355608330 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f323006db00 msgr2=0x7f3230151990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:51.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.923+0000 7f3236796640 1 --2- 172.21.15.63:0/1355608330 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f323006db00 0x7f3230151990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f322002a220 tx=0x7f32200046c0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.923+0000 7f3236796640 1 -- 172.21.15.63:0/1355608330 shutdown_connections 2024-05-20T18:26:51.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.923+0000 7f3236796640 1 --2- 172.21.15.63:0/1355608330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32300af0e0 0x7f3230157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.923+0000 7f3236796640 1 --2- 172.21.15.63:0/1355608330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f323009a4f0 0x7f323009e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.923+0000 7f3236796640 1 --2- 172.21.15.63:0/1355608330 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f323006db00 0x7f3230151990 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:51.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.923+0000 7f3236796640 1 -- 172.21.15.63:0/1355608330 >> 172.21.15.63:0/1355608330 conn(0x7f32300b6620 msgr2=0x7f32300b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:26:51.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.923+0000 7f3236796640 1 -- 172.21.15.63:0/1355608330 shutdown_connections 2024-05-20T18:26:51.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:51.923+0000 7f3236796640 1 -- 172.21.15.63:0/1355608330 wait complete. 2024-05-20T18:26:51.933 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd new 4ae74bb5-50be-4128-aceb-02146503d093 5 2024-05-20T18:26:51.961 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-20T18:26:51.962+0000 7fd1b5923040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-20T18:26:52.019 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-20T18:26:52.020+0000 7fd1b5923040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-20T18:26:52.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.048+0000 7fe465d77640 1 Processor -- start 2024-05-20T18:26:52.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.048+0000 7fe465d77640 1 -- start start 2024-05-20T18:26:52.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.049+0000 7fe465d77640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe46015f900 0x7fe460148b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:52.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.049+0000 7fe465d77640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe4600cfcd0 0x7fe4600d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:52.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.049+0000 7fe465d77640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4600b4960 con 0x7fe4600b5720 2024-05-20T18:26:52.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.049+0000 7fe465d77640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe46015fd10 con 0x7fe46015f900 2024-05-20T18:26:52.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.049+0000 7fe465d77640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe4600b4710 con 0x7fe4600cfcd0 2024-05-20T18:26:52.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.050+0000 7fe45effd640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe4600cfcd0 0x7fe4600d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:52.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.050+0000 7fe45effd640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe4600cfcd0 0x7fe4600d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37560/0 (socket says 172.21.15.63:37560) 2024-05-20T18:26:52.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.050+0000 7fe45effd640 1 -- 172.21.15.63:0/3660432441 learned_addr learned my addr 172.21.15.63:0/3660432441 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-20T18:26:52.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.050+0000 7fe45f7fe640 1 --2- 172.21.15.63:0/3660432441 >> v2:172.21.15.5:3300/0 conn(0x7fe46015f900 0x7fe460148b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:52.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.050+0000 7fe45effd640 1 -- 172.21.15.63:0/3660432441 >> v2:172.21.15.5:3300/0 conn(0x7fe46015f900 msgr2=0x7fe460148b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:26:52.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.050+0000 7fe45effd640 1 --2- 172.21.15.63:0/3660432441 >> v2:172.21.15.5:3300/0 conn(0x7fe46015f900 0x7fe460148b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.050+0000 7fe45effd640 1 -- 172.21.15.63:0/3660432441 >> v1:172.21.15.63:6789/0 conn(0x7fe4600b5720 legacy=0x7fe4600b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:52.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.050+0000 7fe45effd640 1 -- 172.21.15.63:0/3660432441 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4600b4c40 con 0x7fe4600cfcd0 2024-05-20T18:26:52.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.051+0000 7fe45effd640 1 --2- 172.21.15.63:0/3660432441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4600cfcd0 0x7fe4600d00e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe4600b44a0 tx=0x7fe45402a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b69ee9d89b9c2a98 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:26:52.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.051+0000 7fe45e7fc640 1 -- 172.21.15.63:0/3660432441 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe454037070 con 0x7fe4600cfcd0 2024-05-20T18:26:52.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.051+0000 7fe45e7fc640 1 -- 172.21.15.63:0/3660432441 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe454032440 con 0x7fe4600cfcd0 2024-05-20T18:26:52.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.051+0000 7fe45e7fc640 1 -- 172.21.15.63:0/3660432441 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe454032810 con 0x7fe4600cfcd0 2024-05-20T18:26:52.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.052+0000 7fe465d77640 1 -- 172.21.15.63:0/3660432441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4600cfcd0 msgr2=0x7fe4600d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:52.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.052+0000 7fe465d77640 1 --2- 172.21.15.63:0/3660432441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4600cfcd0 0x7fe4600d00e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe4600b44a0 tx=0x7fe45402a1e0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.052+0000 7fe465d77640 1 -- 172.21.15.63:0/3660432441 shutdown_connections 2024-05-20T18:26:52.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.052+0000 7fe465d77640 1 --2- 172.21.15.63:0/3660432441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4600cfcd0 0x7fe4600d00e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.052+0000 7fe465d77640 1 --2- 172.21.15.63:0/3660432441 >> v2:172.21.15.5:3300/0 conn(0x7fe46015f900 0x7fe460148b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.052+0000 7fe465d77640 1 -- 172.21.15.63:0/3660432441 >> 172.21.15.63:0/3660432441 conn(0x7fe46006ee50 msgr2=0x7fe46006f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:26:52.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.052+0000 7fe465d77640 1 -- 172.21.15.63:0/3660432441 shutdown_connections 2024-05-20T18:26:52.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.052+0000 7fe465d77640 1 -- 172.21.15.63:0/3660432441 wait complete. 2024-05-20T18:26:52.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.053+0000 7fe465d77640 1 Processor -- start 2024-05-20T18:26:52.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.054+0000 7fe465d77640 1 -- start start 2024-05-20T18:26:52.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.055+0000 7fe465d77640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4600b5720 0x7fe4600ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:52.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.055+0000 7fe465d77640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4600cfcd0 0x7fe460141b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:52.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.055+0000 7fe465d77640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe46015f900 0x7fe460131050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:52.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.055+0000 7fe465d77640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe4600d0620 con 0x7fe4600b5720 2024-05-20T18:26:52.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.055+0000 7fe465d77640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe4600b5290 con 0x7fe46015f900 2024-05-20T18:26:52.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.055+0000 7fe465d77640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe4600d0ae0 con 0x7fe4600cfcd0 2024-05-20T18:26:52.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.055+0000 7fe45ffff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe46015f900 0x7fe460131050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:52.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.055+0000 7fe45effd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4600cfcd0 0x7fe460141b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:52.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.055+0000 7fe45ffff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe46015f900 0x7fe460131050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37562/0 (socket says 172.21.15.63:37562) 2024-05-20T18:26:52.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.055+0000 7fe45ffff640 1 -- 172.21.15.63:0/4101939394 learned_addr learned my addr 172.21.15.63:0/4101939394 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-20T18:26:52.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.055+0000 7fe45f7fe640 1 --2- 172.21.15.63:0/4101939394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4600b5720 0x7fe4600ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:52.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.055+0000 7fe45ffff640 1 -- 172.21.15.63:0/4101939394 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4600cfcd0 msgr2=0x7fe460141b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:52.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.055+0000 7fe45ffff640 1 --2- 172.21.15.63:0/4101939394 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4600cfcd0 0x7fe460141b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.055+0000 7fe45ffff640 1 -- 172.21.15.63:0/4101939394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4600b5720 msgr2=0x7fe4600ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:52.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.055+0000 7fe45ffff640 1 --2- 172.21.15.63:0/4101939394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4600b5720 0x7fe4600ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.055+0000 7fe45ffff640 1 -- 172.21.15.63:0/4101939394 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe46012ffb0 con 0x7fe46015f900 2024-05-20T18:26:52.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.056+0000 7fe45f7fe640 1 --2- 172.21.15.63:0/4101939394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4600b5720 0x7fe4600ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:26:52.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.056+0000 7fe45effd640 1 --2- 172.21.15.63:0/4101939394 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4600cfcd0 0x7fe460141b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:26:52.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.056+0000 7fe45ffff640 1 --2- 172.21.15.63:0/4101939394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe46015f900 0x7fe460131050 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe45800c6b0 tx=0x7fe45800cb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:26:52.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.056+0000 7fe447fff640 1 -- 172.21.15.63:0/4101939394 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe458003b70 con 0x7fe46015f900 2024-05-20T18:26:52.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.056+0000 7fe447fff640 1 -- 172.21.15.63:0/4101939394 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe458003d30 con 0x7fe46015f900 2024-05-20T18:26:52.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.056+0000 7fe465d77640 1 -- 172.21.15.63:0/4101939394 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe460130290 con 0x7fe46015f900 2024-05-20T18:26:52.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.056+0000 7fe447fff640 1 -- 172.21.15.63:0/4101939394 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe458005310 con 0x7fe46015f900 2024-05-20T18:26:52.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.056+0000 7fe465d77640 1 -- 172.21.15.63:0/4101939394 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe460130850 con 0x7fe46015f900 2024-05-20T18:26:52.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.056+0000 7fe447fff640 1 -- 172.21.15.63:0/4101939394 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe4580054b0 con 0x7fe46015f900 2024-05-20T18:26:52.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.056+0000 7fe447fff640 1 -- 172.21.15.63:0/4101939394 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fe458012a90 con 0x7fe46015f900 2024-05-20T18:26:52.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.057+0000 7fe465d77640 1 -- 172.21.15.63:0/4101939394 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe460075b00 con 0x7fe46015f900 2024-05-20T18:26:52.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.059+0000 7fe447fff640 1 -- 172.21.15.63:0/4101939394 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe458012d30 con 0x7fe46015f900 2024-05-20T18:26:52.074 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-20T18:26:52.075+0000 7fd1b5923040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-20T18:26:52.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.133+0000 7fe465d77640 1 -- 172.21.15.63:0/4101939394 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4ae74bb5-50be-4128-aceb-02146503d093", "id": 5} v 0) -- 0x7fe46012fcd0 con 0x7fe46015f900 2024-05-20T18:26:52.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.136+0000 7fe447fff640 1 -- 172.21.15.63:0/4101939394 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4ae74bb5-50be-4128-aceb-02146503d093", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fe458012510 con 0x7fe46015f900 2024-05-20T18:26:52.136 INFO:teuthology.orchestra.run.smithi063.stdout:5 2024-05-20T18:26:52.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.137+0000 7fe465d77640 1 -- 172.21.15.63:0/4101939394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe46015f900 msgr2=0x7fe460131050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:52.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.137+0000 7fe465d77640 1 --2- 172.21.15.63:0/4101939394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe46015f900 0x7fe460131050 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe45800c6b0 tx=0x7fe45800cb70 comp rx=0 tx=0).stop 2024-05-20T18:26:52.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.137+0000 7fe465d77640 1 -- 172.21.15.63:0/4101939394 shutdown_connections 2024-05-20T18:26:52.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.137+0000 7fe465d77640 1 --2- 172.21.15.63:0/4101939394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe46015f900 0x7fe460131050 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.137+0000 7fe465d77640 1 --2- 172.21.15.63:0/4101939394 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4600cfcd0 0x7fe460141b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.137+0000 7fe465d77640 1 --2- 172.21.15.63:0/4101939394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4600b5720 0x7fe4600ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.137+0000 7fe465d77640 1 -- 172.21.15.63:0/4101939394 >> 172.21.15.63:0/4101939394 conn(0x7fe46006ee50 msgr2=0x7fe4600b2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:26:52.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.138+0000 7fe465d77640 1 -- 172.21.15.63:0/4101939394 shutdown_connections 2024-05-20T18:26:52.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.138+0000 7fe465d77640 1 -- 172.21.15.63:0/4101939394 wait complete. 2024-05-20T18:26:52.148 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd new 11f04fa5-adbe-4016-903e-1652248e1395 6 2024-05-20T18:26:52.192 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-20T18:26:52.193+0000 7fd1b5923040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-20T18:26:52.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.262+0000 7fabfdbce640 1 Processor -- start 2024-05-20T18:26:52.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.262+0000 7fabfdbce640 1 -- start start 2024-05-20T18:26:52.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.264+0000 7fabfdbce640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fabf806dce0 0x7fabf80d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:52.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.264+0000 7fabfdbce640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabf815dcd0 con 0x7fabf80d1d30 2024-05-20T18:26:52.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.264+0000 7fabfdbce640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabf815dfb0 con 0x7fabf81435f0 2024-05-20T18:26:52.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.264+0000 7fabfdbce640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fabf815e290 con 0x7fabf806dce0 2024-05-20T18:26:52.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.264+0000 7fabf6ffd640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fabf806dce0 0x7fabf80d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:52.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.264+0000 7fabf6ffd640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fabf806dce0 0x7fabf80d17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37570/0 (socket says 172.21.15.63:37570) 2024-05-20T18:26:52.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.264+0000 7fabf6ffd640 1 -- 172.21.15.63:0/2433209216 learned_addr learned my addr 172.21.15.63:0/2433209216 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-20T18:26:52.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.264+0000 7fabf6ffd640 1 -- 172.21.15.63:0/2433209216 >> v1:172.21.15.63:6789/0 conn(0x7fabf81435f0 legacy=0x7fabf815d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:52.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.264+0000 7fabf6ffd640 1 -- 172.21.15.63:0/2433209216 >> v1:172.21.15.5:6789/0 conn(0x7fabf80d1d30 legacy=0x7fabf80b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:52.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.264+0000 7fabf6ffd640 1 -- 172.21.15.63:0/2433209216 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf80b1390 con 0x7fabf806dce0 2024-05-20T18:26:52.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.265+0000 7fabf6ffd640 1 --2- 172.21.15.63:0/2433209216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabf806dce0 0x7fabf80d17f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fabec009870 tx=0x7fabec02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbad636efa045f61 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:26:52.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.265+0000 7fabf67fc640 1 -- 172.21.15.63:0/2433209216 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabec037070 con 0x7fabf806dce0 2024-05-20T18:26:52.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.265+0000 7fabf67fc640 1 -- 172.21.15.63:0/2433209216 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabec032440 con 0x7fabf806dce0 2024-05-20T18:26:52.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.265+0000 7fabf67fc640 1 -- 172.21.15.63:0/2433209216 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabec032810 con 0x7fabf806dce0 2024-05-20T18:26:52.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.266+0000 7fabfdbce640 1 -- 172.21.15.63:0/2433209216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabf806dce0 msgr2=0x7fabf80d17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:52.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.266+0000 7fabfdbce640 1 --2- 172.21.15.63:0/2433209216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabf806dce0 0x7fabf80d17f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fabec009870 tx=0x7fabec02a270 comp rx=0 tx=0).stop 2024-05-20T18:26:52.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.266+0000 7fabfdbce640 1 -- 172.21.15.63:0/2433209216 shutdown_connections 2024-05-20T18:26:52.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.266+0000 7fabfdbce640 1 --2- 172.21.15.63:0/2433209216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabf806dce0 0x7fabf80d17f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.266+0000 7fabfdbce640 1 -- 172.21.15.63:0/2433209216 >> 172.21.15.63:0/2433209216 conn(0x7fabf80b6620 msgr2=0x7fabf80b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:26:52.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.266+0000 7fabfdbce640 1 -- 172.21.15.63:0/2433209216 shutdown_connections 2024-05-20T18:26:52.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.266+0000 7fabfdbce640 1 -- 172.21.15.63:0/2433209216 wait complete. 2024-05-20T18:26:52.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.268+0000 7fabfdbce640 1 Processor -- start 2024-05-20T18:26:52.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.268+0000 7fabfdbce640 1 -- start start 2024-05-20T18:26:52.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.269+0000 7fabfdbce640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fabf806dce0 0x7fabf813f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:52.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.269+0000 7fabfdbce640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabf80d1d30 0x7fabf8157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:52.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.269+0000 7fabfdbce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf813fee0 0x7fabf8158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:52.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.269+0000 7fabfdbce640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fabf80b2140 con 0x7fabf813fee0 2024-05-20T18:26:52.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.269+0000 7fabfdbce640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fabf80b1fc0 con 0x7fabf80d1d30 2024-05-20T18:26:52.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.269+0000 7fabfdbce640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fabf80b22c0 con 0x7fabf806dce0 2024-05-20T18:26:52.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.269+0000 7fabf6ffd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabf80d1d30 0x7fabf8157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:52.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.269+0000 7fabf77fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fabf806dce0 0x7fabf813f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:52.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.269+0000 7fabf6ffd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabf80d1d30 0x7fabf8157d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37580/0 (socket says 172.21.15.63:37580) 2024-05-20T18:26:52.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.269+0000 7fabf6ffd640 1 -- 172.21.15.63:0/718047556 learned_addr learned my addr 172.21.15.63:0/718047556 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-20T18:26:52.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.270+0000 7fabf7fff640 1 --2- 172.21.15.63:0/718047556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf813fee0 0x7fabf8158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:52.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.270+0000 7fabf6ffd640 1 -- 172.21.15.63:0/718047556 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fabf806dce0 msgr2=0x7fabf813f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:52.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.270+0000 7fabf6ffd640 1 --2- 172.21.15.63:0/718047556 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fabf806dce0 0x7fabf813f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.270+0000 7fabf6ffd640 1 -- 172.21.15.63:0/718047556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf813fee0 msgr2=0x7fabf8158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:52.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.270+0000 7fabf6ffd640 1 --2- 172.21.15.63:0/718047556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf813fee0 0x7fabf8158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.270+0000 7fabf6ffd640 1 -- 172.21.15.63:0/718047556 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf815b8c0 con 0x7fabf80d1d30 2024-05-20T18:26:52.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.270+0000 7fabf77fe640 1 --2- 172.21.15.63:0/718047556 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fabf806dce0 0x7fabf813f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:26:52.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.270+0000 7fabf6ffd640 1 --2- 172.21.15.63:0/718047556 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabf80d1d30 0x7fabf8157d30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fabec000c00 tx=0x7fabec0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:26:52.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.270+0000 7fabdffff640 1 -- 172.21.15.63:0/718047556 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabec037040 con 0x7fabf80d1d30 2024-05-20T18:26:52.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.270+0000 7fabdffff640 1 -- 172.21.15.63:0/718047556 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabec042070 con 0x7fabf80d1d30 2024-05-20T18:26:52.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.270+0000 7fabdffff640 1 -- 172.21.15.63:0/718047556 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabec030bd0 con 0x7fabf80d1d30 2024-05-20T18:26:52.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.270+0000 7fabfdbce640 1 -- 172.21.15.63:0/718047556 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf815bba0 con 0x7fabf80d1d30 2024-05-20T18:26:52.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.270+0000 7fabfdbce640 1 -- 172.21.15.63:0/718047556 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf8155e20 con 0x7fabf80d1d30 2024-05-20T18:26:52.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.271+0000 7fabdffff640 1 -- 172.21.15.63:0/718047556 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fabec047460 con 0x7fabf80d1d30 2024-05-20T18:26:52.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.271+0000 7fabdffff640 1 -- 172.21.15.63:0/718047556 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fabec03da30 con 0x7fabf80d1d30 2024-05-20T18:26:52.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.271+0000 7fabfdbce640 1 -- 172.21.15.63:0/718047556 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabf8075b00 con 0x7fabf80d1d30 2024-05-20T18:26:52.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.273+0000 7fabdffff640 1 -- 172.21.15.63:0/718047556 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fabec03dc30 con 0x7fabf80d1d30 2024-05-20T18:26:52.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.347+0000 7fabfdbce640 1 -- 172.21.15.63:0/718047556 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd new", "uuid": "11f04fa5-adbe-4016-903e-1652248e1395", "id": 6} v 0) -- 0x7fabf80b3c90 con 0x7fabf80d1d30 2024-05-20T18:26:52.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.356+0000 7fabdffff640 1 -- 172.21.15.63:0/718047556 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "11f04fa5-adbe-4016-903e-1652248e1395", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fabec03d440 con 0x7fabf80d1d30 2024-05-20T18:26:52.355 INFO:teuthology.orchestra.run.smithi063.stdout:6 2024-05-20T18:26:52.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.357+0000 7fabfdbce640 1 -- 172.21.15.63:0/718047556 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabf80d1d30 msgr2=0x7fabf8157d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:52.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.357+0000 7fabfdbce640 1 --2- 172.21.15.63:0/718047556 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabf80d1d30 0x7fabf8157d30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fabec000c00 tx=0x7fabec0046c0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.357+0000 7fabfdbce640 1 -- 172.21.15.63:0/718047556 shutdown_connections 2024-05-20T18:26:52.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.357+0000 7fabfdbce640 1 --2- 172.21.15.63:0/718047556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf813fee0 0x7fabf8158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.357+0000 7fabfdbce640 1 --2- 172.21.15.63:0/718047556 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fabf80d1d30 0x7fabf8157d30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.357+0000 7fabfdbce640 1 --2- 172.21.15.63:0/718047556 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fabf806dce0 0x7fabf813f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.357+0000 7fabfdbce640 1 -- 172.21.15.63:0/718047556 >> 172.21.15.63:0/718047556 conn(0x7fabf80b6620 msgr2=0x7fabf8142260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:26:52.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.357+0000 7fabfdbce640 1 -- 172.21.15.63:0/718047556 shutdown_connections 2024-05-20T18:26:52.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.357+0000 7fabfdbce640 1 -- 172.21.15.63:0/718047556 wait complete. 2024-05-20T18:26:52.367 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd new 5ba4629f-35b0-4383-9264-e88757af5aaa 7 2024-05-20T18:26:52.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.482+0000 7fdd6c17e640 1 Processor -- start 2024-05-20T18:26:52.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.482+0000 7fdd6c17e640 1 -- start start 2024-05-20T18:26:52.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.484+0000 7fdd6c17e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdd64074b50 0x7fdd6415f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:52.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.484+0000 7fdd6c17e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fdd640aea30 0x7fdd6406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:52.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.484+0000 7fdd6c17e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd6406b520 con 0x7fdd6406c5c0 2024-05-20T18:26:52.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.484+0000 7fdd6c17e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fdd6406b800 con 0x7fdd64074b50 2024-05-20T18:26:52.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.484+0000 7fdd6c17e640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fdd6406bae0 con 0x7fdd640aea30 2024-05-20T18:26:52.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.484+0000 7fdd6a71b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fdd640aea30 0x7fdd6406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:52.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.484+0000 7fdd6a71b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fdd640aea30 0x7fdd6406c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37586/0 (socket says 172.21.15.63:37586) 2024-05-20T18:26:52.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.484+0000 7fdd6a71b640 1 -- 172.21.15.63:0/2018601554 learned_addr learned my addr 172.21.15.63:0/2018601554 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-20T18:26:52.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.484+0000 7fdd69f1a640 1 --2- 172.21.15.63:0/2018601554 >> v2:172.21.15.5:3300/0 conn(0x7fdd64074b50 0x7fdd6415f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:52.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.484+0000 7fdd6a71b640 1 -- 172.21.15.63:0/2018601554 >> v2:172.21.15.5:3300/0 conn(0x7fdd64074b50 msgr2=0x7fdd6415f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:26:52.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.484+0000 7fdd6a71b640 1 --2- 172.21.15.63:0/2018601554 >> v2:172.21.15.5:3300/0 conn(0x7fdd64074b50 0x7fdd6415f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.484+0000 7fdd6a71b640 1 -- 172.21.15.63:0/2018601554 >> v1:172.21.15.63:6789/0 conn(0x7fdd6406c5c0 legacy=0x7fdd640ae320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:52.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.484+0000 7fdd6a71b640 1 -- 172.21.15.63:0/2018601554 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd6406bdc0 con 0x7fdd640aea30 2024-05-20T18:26:52.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.484+0000 7fdd68f18640 1 -- 172.21.15.63:0/2018601554 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1540560117 0 0) 0x7fdd6406b520 con 0x7fdd6406c5c0 2024-05-20T18:26:52.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.485+0000 7fdd6a71b640 1 --2- 172.21.15.63:0/2018601554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd640aea30 0x7fdd6406c9d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdd64071430 tx=0x7fdd5c02a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1813c608940df24e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:26:52.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.485+0000 7fdd68f18640 1 -- 172.21.15.63:0/2018601554 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c036070 con 0x7fdd640aea30 2024-05-20T18:26:52.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.485+0000 7fdd68f18640 1 -- 172.21.15.63:0/2018601554 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd5c031600 con 0x7fdd640aea30 2024-05-20T18:26:52.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.485+0000 7fdd68f18640 1 -- 172.21.15.63:0/2018601554 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c0319d0 con 0x7fdd640aea30 2024-05-20T18:26:52.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.486+0000 7fdd6c17e640 1 -- 172.21.15.63:0/2018601554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd640aea30 msgr2=0x7fdd6406c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:52.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.486+0000 7fdd6c17e640 1 --2- 172.21.15.63:0/2018601554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd640aea30 0x7fdd6406c9d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdd64071430 tx=0x7fdd5c02a730 comp rx=0 tx=0).stop 2024-05-20T18:26:52.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.486+0000 7fdd6c17e640 1 -- 172.21.15.63:0/2018601554 shutdown_connections 2024-05-20T18:26:52.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.486+0000 7fdd6c17e640 1 --2- 172.21.15.63:0/2018601554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd640aea30 0x7fdd6406c9d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.486+0000 7fdd6c17e640 1 --2- 172.21.15.63:0/2018601554 >> v2:172.21.15.5:3300/0 conn(0x7fdd64074b50 0x7fdd6415f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.486+0000 7fdd6c17e640 1 -- 172.21.15.63:0/2018601554 >> 172.21.15.63:0/2018601554 conn(0x7fdd64072f30 msgr2=0x7fdd640b6080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:26:52.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.486+0000 7fdd6c17e640 1 -- 172.21.15.63:0/2018601554 shutdown_connections 2024-05-20T18:26:52.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.486+0000 7fdd6c17e640 1 -- 172.21.15.63:0/2018601554 wait complete. 2024-05-20T18:26:52.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.488+0000 7fdd6c17e640 1 Processor -- start 2024-05-20T18:26:52.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.488+0000 7fdd6c17e640 1 -- start start 2024-05-20T18:26:52.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.489+0000 7fdd6c17e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd6406c5c0 0x7fdd641527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:52.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.489+0000 7fdd6c17e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd64074b50 0x7fdd640cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:52.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.489+0000 7fdd6c17e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd640aea30 0x7fdd640cc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:52.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.489+0000 7fdd6c17e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdd64163080 con 0x7fdd6406c5c0 2024-05-20T18:26:52.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.489+0000 7fdd6c17e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdd64162f00 con 0x7fdd64074b50 2024-05-20T18:26:52.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.489+0000 7fdd6c17e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdd64163200 con 0x7fdd640aea30 2024-05-20T18:26:52.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.489+0000 7fdd69719640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd64074b50 0x7fdd640cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:52.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.489+0000 7fdd6a71b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd640aea30 0x7fdd640cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:52.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.489+0000 7fdd69719640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd64074b50 0x7fdd640cc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37594/0 (socket says 172.21.15.63:37594) 2024-05-20T18:26:52.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.489+0000 7fdd69719640 1 -- 172.21.15.63:0/2276188434 learned_addr learned my addr 172.21.15.63:0/2276188434 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-20T18:26:52.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.490+0000 7fdd69f1a640 1 --2- 172.21.15.63:0/2276188434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd6406c5c0 0x7fdd641527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:52.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.490+0000 7fdd6a71b640 1 -- 172.21.15.63:0/2276188434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd64074b50 msgr2=0x7fdd640cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:52.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.490+0000 7fdd6a71b640 1 --2- 172.21.15.63:0/2276188434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd64074b50 0x7fdd640cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.490+0000 7fdd6a71b640 1 -- 172.21.15.63:0/2276188434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd6406c5c0 msgr2=0x7fdd641527b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:52.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.490+0000 7fdd6a71b640 1 --2- 172.21.15.63:0/2276188434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd6406c5c0 0x7fdd641527b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.490+0000 7fdd6a71b640 1 -- 172.21.15.63:0/2276188434 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd64153170 con 0x7fdd640aea30 2024-05-20T18:26:52.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.490+0000 7fdd69719640 1 --2- 172.21.15.63:0/2276188434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd64074b50 0x7fdd640cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:26:52.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.490+0000 7fdd6a71b640 1 --2- 172.21.15.63:0/2276188434 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd640aea30 0x7fdd640cc540 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c030000 tx=0x7fdd5c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:26:52.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.490+0000 7fdd5a7fc640 1 -- 172.21.15.63:0/2276188434 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c03f020 con 0x7fdd640aea30 2024-05-20T18:26:52.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.490+0000 7fdd5a7fc640 1 -- 172.21.15.63:0/2276188434 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd5c037040 con 0x7fdd640aea30 2024-05-20T18:26:52.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.490+0000 7fdd6c17e640 1 -- 172.21.15.63:0/2276188434 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd64160d10 con 0x7fdd640aea30 2024-05-20T18:26:52.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.490+0000 7fdd5a7fc640 1 -- 172.21.15.63:0/2276188434 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c036070 con 0x7fdd640aea30 2024-05-20T18:26:52.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.490+0000 7fdd6c17e640 1 -- 172.21.15.63:0/2276188434 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd641612d0 con 0x7fdd640aea30 2024-05-20T18:26:52.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.491+0000 7fdd5a7fc640 1 -- 172.21.15.63:0/2276188434 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdd5c042020 con 0x7fdd640aea30 2024-05-20T18:26:52.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.491+0000 7fdd5a7fc640 1 -- 172.21.15.63:0/2276188434 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fdd5c044ba0 con 0x7fdd640aea30 2024-05-20T18:26:52.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.491+0000 7fdd6c17e640 1 -- 172.21.15.63:0/2276188434 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd64075b00 con 0x7fdd640aea30 2024-05-20T18:26:52.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.494+0000 7fdd5a7fc640 1 -- 172.21.15.63:0/2276188434 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fdd5c049030 con 0x7fdd640aea30 2024-05-20T18:26:52.547 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-20T18:26:52.548+0000 7f3923cda040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-20T18:26:52.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.568+0000 7fdd6c17e640 1 -- 172.21.15.63:0/2276188434 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5ba4629f-35b0-4383-9264-e88757af5aaa", "id": 7} v 0) -- 0x7fdd64073910 con 0x7fdd640aea30 2024-05-20T18:26:52.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.576+0000 7fdd5a7fc640 1 -- 172.21.15.63:0/2276188434 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5ba4629f-35b0-4383-9264-e88757af5aaa", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fdd5c009510 con 0x7fdd640aea30 2024-05-20T18:26:52.576 INFO:teuthology.orchestra.run.smithi063.stdout:7 2024-05-20T18:26:52.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.577+0000 7fdd6c17e640 1 -- 172.21.15.63:0/2276188434 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd640aea30 msgr2=0x7fdd640cc540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:52.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.577+0000 7fdd6c17e640 1 --2- 172.21.15.63:0/2276188434 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd640aea30 0x7fdd640cc540 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c030000 tx=0x7fdd5c004a40 comp rx=0 tx=0).stop 2024-05-20T18:26:52.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.577+0000 7fdd6c17e640 1 -- 172.21.15.63:0/2276188434 shutdown_connections 2024-05-20T18:26:52.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.577+0000 7fdd6c17e640 1 --2- 172.21.15.63:0/2276188434 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd640aea30 0x7fdd640cc540 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.577+0000 7fdd6c17e640 1 --2- 172.21.15.63:0/2276188434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd64074b50 0x7fdd640cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.577+0000 7fdd6c17e640 1 --2- 172.21.15.63:0/2276188434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd6406c5c0 0x7fdd641527b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.577+0000 7fdd6c17e640 1 -- 172.21.15.63:0/2276188434 >> 172.21.15.63:0/2276188434 conn(0x7fdd64072f30 msgr2=0x7fdd6406f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:26:52.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.578+0000 7fdd6c17e640 1 -- 172.21.15.63:0/2276188434 shutdown_connections 2024-05-20T18:26:52.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:26:52.578+0000 7fdd6c17e640 1 -- 172.21.15.63:0/2276188434 wait complete. 2024-05-20T18:26:52.587 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-20T18:26:52.588 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-20T18:26:52.590 INFO:tasks.ceph.osd.0:Started 2024-05-20T18:26:52.590 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-20T18:26:52.590 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-20T18:26:52.592 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-20T18:26:52.593+0000 7fd1b5923040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-20T18:26:52.593 INFO:tasks.ceph.osd.1:Started 2024-05-20T18:26:52.593 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-20T18:26:52.593 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-20T18:26:52.595 INFO:tasks.ceph.osd.2:Started 2024-05-20T18:26:52.595 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-20T18:26:52.595 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-20T18:26:52.598 INFO:tasks.ceph.osd.3:Started 2024-05-20T18:26:52.598 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-20T18:26:52.598 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-20T18:26:52.600 INFO:tasks.ceph.osd.4:Started 2024-05-20T18:26:52.600 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-20T18:26:52.600 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-20T18:26:52.604 INFO:tasks.ceph.osd.5:Started 2024-05-20T18:26:52.604 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-20T18:26:52.604 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-20T18:26:52.607 INFO:tasks.ceph.osd.6:Started 2024-05-20T18:26:52.607 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-20T18:26:52.607 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-20T18:26:52.609 INFO:tasks.ceph.osd.7:Started 2024-05-20T18:26:52.609 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:26:52.652 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-20T18:26:52.653+0000 7f3923cda040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-20T18:26:52.722 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-20T18:26:52.723+0000 7f3923cda040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-20T18:26:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.766+0000 7f58198f9640 1 Processor -- start 2024-05-20T18:26:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.766+0000 7f58198f9640 1 -- start start 2024-05-20T18:26:52.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.768+0000 7f58198f9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f581415d530 0x7f581415aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.768+0000 7f58198f9640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5814070850 0x7f5814070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.768+0000 7f58198f9640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58140cd580 con 0x7f58140b1700 2024-05-20T18:26:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.768+0000 7f58198f9640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f58140ce190 con 0x7f581415d530 2024-05-20T18:26:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.768+0000 7f58198f9640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f58140cd860 con 0x7f5814070850 2024-05-20T18:26:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.768+0000 7f5812ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f581415d530 0x7f581415aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.768+0000 7f58127fc640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f58140b1700 0x7f58140b1b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46056/0 (socket says 172.21.15.5:46056) 2024-05-20T18:26:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.768+0000 7f58127fc640 1 -- 172.21.15.5:0/219909318 learned_addr learned my addr 172.21.15.5:0/219909318 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:26:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.768+0000 7f58137fe640 1 --2- 172.21.15.5:0/219909318 >> v2:172.21.15.63:3300/0 conn(0x7f5814070850 0x7f5814070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.768+0000 7f5811ffb640 1 -- 172.21.15.5:0/219909318 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 914052848 0 0) 0x7f58140cd580 con 0x7f58140b1700 2024-05-20T18:26:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.768+0000 7f5811ffb640 1 -- 172.21.15.5:0/219909318 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5800003520 con 0x7f58140b1700 2024-05-20T18:26:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.768+0000 7f5812ffd640 1 -- 172.21.15.5:0/219909318 >> v2:172.21.15.63:3300/0 conn(0x7f5814070850 msgr2=0x7f5814070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:26:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.768+0000 7f5812ffd640 1 --2- 172.21.15.5:0/219909318 >> v2:172.21.15.63:3300/0 conn(0x7f5814070850 0x7f5814070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.768+0000 7f5812ffd640 1 -- 172.21.15.5:0/219909318 >> v1:172.21.15.5:6789/0 conn(0x7f58140b1700 legacy=0x7f58140b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:52.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.768+0000 7f5812ffd640 1 -- 172.21.15.5:0/219909318 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58140cdb40 con 0x7f581415d530 2024-05-20T18:26:52.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.769+0000 7f5812ffd640 1 --2- 172.21.15.5:0/219909318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f581415d530 0x7f581415aaa0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f58140b8400 tx=0x7f580402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a46eb6c82f94187 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:26:52.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.769+0000 7f5811ffb640 1 -- 172.21.15.5:0/219909318 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804037070 con 0x7f581415d530 2024-05-20T18:26:52.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.769+0000 7f5811ffb640 1 -- 172.21.15.5:0/219909318 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5804032440 con 0x7f581415d530 2024-05-20T18:26:52.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.769+0000 7f5811ffb640 1 -- 172.21.15.5:0/219909318 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804032810 con 0x7f581415d530 2024-05-20T18:26:52.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.770+0000 7f58198f9640 1 -- 172.21.15.5:0/219909318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f581415d530 msgr2=0x7f581415aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:52.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.770+0000 7f58198f9640 1 --2- 172.21.15.5:0/219909318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f581415d530 0x7f581415aaa0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f58140b8400 tx=0x7f580402a1a0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.770+0000 7f58198f9640 1 -- 172.21.15.5:0/219909318 shutdown_connections 2024-05-20T18:26:52.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.770+0000 7f58198f9640 1 --2- 172.21.15.5:0/219909318 >> v2:172.21.15.63:3300/0 conn(0x7f5814070850 0x7f5814070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.770+0000 7f58198f9640 1 --2- 172.21.15.5:0/219909318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f581415d530 0x7f581415aaa0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.770+0000 7f58198f9640 1 -- 172.21.15.5:0/219909318 >> 172.21.15.5:0/219909318 conn(0x7f58140d29d0 msgr2=0x7f58140d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:26:52.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.770+0000 7f58198f9640 1 -- 172.21.15.5:0/219909318 shutdown_connections 2024-05-20T18:26:52.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.770+0000 7f58198f9640 1 -- 172.21.15.5:0/219909318 wait complete. 2024-05-20T18:26:52.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.772+0000 7f58198f9640 1 Processor -- start 2024-05-20T18:26:52.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.772+0000 7f58198f9640 1 -- start start 2024-05-20T18:26:52.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.773+0000 7f58198f9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5814070850 0x7f5814155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:52.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.773+0000 7f58198f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58140b1700 0x7f58140ca290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:52.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.773+0000 7f58198f9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f581415d530 0x7f58140ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:52.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.773+0000 7f58198f9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f58140711a0 con 0x7f58140b1700 2024-05-20T18:26:52.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.773+0000 7f58198f9640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f58140cd230 con 0x7f5814070850 2024-05-20T18:26:52.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.773+0000 7f58198f9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5814071320 con 0x7f581415d530 2024-05-20T18:26:52.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.774+0000 7f58127fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58140b1700 0x7f58140ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:52.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.774+0000 7f5812ffd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5814070850 0x7f5814155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:52.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.774+0000 7f5812ffd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5814070850 0x7f5814155120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:40638/0 (socket says 172.21.15.5:40638) 2024-05-20T18:26:52.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.774+0000 7f5812ffd640 1 -- 172.21.15.5:0/1079337122 learned_addr learned my addr 172.21.15.5:0/1079337122 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:26:52.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.774+0000 7f58127fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58140b1700 0x7f58140ca290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39672/0 (socket says 172.21.15.5:39672) 2024-05-20T18:26:52.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.774+0000 7f58137fe640 1 --2- 172.21.15.5:0/1079337122 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f581415d530 0x7f58140ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:52.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.774+0000 7f58127fc640 1 -- 172.21.15.5:0/1079337122 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f581415d530 msgr2=0x7f58140ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:52.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.774+0000 7f58127fc640 1 --2- 172.21.15.5:0/1079337122 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f581415d530 0x7f58140ca7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.774+0000 7f58127fc640 1 -- 172.21.15.5:0/1079337122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5814070850 msgr2=0x7f5814155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:52.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.774+0000 7f58127fc640 1 --2- 172.21.15.5:0/1079337122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5814070850 0x7f5814155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.774+0000 7f58127fc640 1 -- 172.21.15.5:0/1079337122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58140b0e60 con 0x7f58140b1700 2024-05-20T18:26:52.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.774+0000 7f58127fc640 1 --2- 172.21.15.5:0/1079337122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58140b1700 0x7f58140ca290 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00c3c0 tx=0x7f57fc001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:26:52.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.774+0000 7f580bfff640 1 -- 172.21.15.5:0/1079337122 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc017070 con 0x7f58140b1700 2024-05-20T18:26:52.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.774+0000 7f580bfff640 1 -- 172.21.15.5:0/1079337122 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57fc005ce0 con 0x7f58140b1700 2024-05-20T18:26:52.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.774+0000 7f58198f9640 1 -- 172.21.15.5:0/1079337122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58140b7990 con 0x7f58140b1700 2024-05-20T18:26:52.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.774+0000 7f580bfff640 1 -- 172.21.15.5:0/1079337122 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc007310 con 0x7f58140b1700 2024-05-20T18:26:52.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.775+0000 7f58198f9640 1 -- 172.21.15.5:0/1079337122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58140b7f50 con 0x7f58140b1700 2024-05-20T18:26:52.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.775+0000 7f580bfff640 1 -- 172.21.15.5:0/1079337122 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f57fc01a070 con 0x7f58140b1700 2024-05-20T18:26:52.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.775+0000 7f580bfff640 1 -- 172.21.15.5:0/1079337122 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f57fc0074b0 con 0x7f58140b1700 2024-05-20T18:26:52.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.776+0000 7f58198f9640 1 -- 172.21.15.5:0/1079337122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5814075b00 con 0x7f58140b1700 2024-05-20T18:26:52.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.778+0000 7f580bfff640 1 -- 172.21.15.5:0/1079337122 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f57fc021070 con 0x7f58140b1700 2024-05-20T18:26:52.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.840+0000 7f58198f9640 1 -- 172.21.15.5:0/1079337122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f581415df20 con 0x7f58140b1700 2024-05-20T18:26:52.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.841+0000 7f580bfff640 1 -- 172.21.15.5:0/1079337122 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f58140b0e60 con 0x7f58140b1700 2024-05-20T18:26:52.840 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:26:52.840 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":10,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:26:52.570603+0000","last_up_change":"0.000000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:26:52.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.842+0000 7f58198f9640 1 -- 172.21.15.5:0/1079337122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58140b1700 msgr2=0x7f58140ca290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:52.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.842+0000 7f58198f9640 1 --2- 172.21.15.5:0/1079337122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58140b1700 0x7f58140ca290 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00c3c0 tx=0x7f57fc001e70 comp rx=0 tx=0).stop 2024-05-20T18:26:52.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.842+0000 7f58198f9640 1 -- 172.21.15.5:0/1079337122 shutdown_connections 2024-05-20T18:26:52.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.842+0000 7f58198f9640 1 --2- 172.21.15.5:0/1079337122 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f581415d530 0x7f58140ca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.842+0000 7f58198f9640 1 --2- 172.21.15.5:0/1079337122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58140b1700 0x7f58140ca290 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.842+0000 7f58198f9640 1 --2- 172.21.15.5:0/1079337122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5814070850 0x7f5814155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:52.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.842+0000 7f58198f9640 1 -- 172.21.15.5:0/1079337122 >> 172.21.15.5:0/1079337122 conn(0x7f58140d29d0 msgr2=0x7f581415e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:26:52.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.842+0000 7f58198f9640 1 -- 172.21.15.5:0/1079337122 shutdown_connections 2024-05-20T18:26:52.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:26:52.842+0000 7f58198f9640 1 -- 172.21.15.5:0/1079337122 wait complete. 2024-05-20T18:26:52.852 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-20T18:26:52.852 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-20T18:26:52.898 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-20T18:26:52.899+0000 7f3923cda040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-20T18:26:52.913 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-20T18:26:52.914+0000 7fd1b5923040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-20T18:26:52.947 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-20T18:26:52.948+0000 7f3923cda040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-20T18:26:52.968 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-20T18:26:52.969+0000 7fd1b5923040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-20T18:26:53.002 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-20T18:26:53.004+0000 7f3923cda040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-20T18:26:53.090 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-20T18:26:53.091+0000 7fd1b5923040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-20T18:26:53.120 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-20T18:26:53.122+0000 7f3923cda040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-20T18:26:53.217 INFO:tasks.ceph.osd.4.smithi063.stderr:2024-05-20T18:26:53.218+0000 7fe4afe11640 -1 Falling back to public interface 2024-05-20T18:26:53.221 INFO:tasks.ceph.osd.5.smithi063.stderr:2024-05-20T18:26:53.222+0000 7f1fc5620640 -1 Falling back to public interface 2024-05-20T18:26:53.221 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-20T18:26:53.223+0000 7f08e01bc640 -1 Falling back to public interface 2024-05-20T18:26:53.225 INFO:tasks.ceph.osd.6.smithi063.stderr:2024-05-20T18:26:53.226+0000 7faa8eedd640 -1 Falling back to public interface 2024-05-20T18:26:53.227 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-20T18:26:53.229+0000 7ff019414640 -1 Falling back to public interface 2024-05-20T18:26:53.229 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-20T18:26:53.230+0000 7f49014b5640 -1 Falling back to public interface 2024-05-20T18:26:53.261 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-20T18:26:53.262+0000 7fbac460f640 -1 Falling back to public interface 2024-05-20T18:26:53.264 INFO:tasks.ceph.osd.7.smithi063.stderr:2024-05-20T18:26:53.265+0000 7f71bb150640 -1 Falling back to public interface 2024-05-20T18:26:53.401 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-20T18:26:53.403+0000 7fd1b5923040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-20T18:26:53.451 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-20T18:26:53.452+0000 7fd1b5923040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-20T18:26:53.529 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-20T18:26:53.530+0000 7f3923cda040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-20T18:26:53.616 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-20T18:26:53.618+0000 7fd1b5923040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-20T18:26:53.658 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-20T18:26:53.763 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-20T18:26:53.764+0000 7fd1b5923040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-20T18:26:53.764 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.765+0000 7f4f75177640 1 Processor -- start 2024-05-20T18:26:53.764 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.765+0000 7f4f75177640 1 -- start start 2024-05-20T18:26:53.767 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.766+0000 7f4f75177640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f4f700b40c0 0x7f4f700b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:53.767 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.766+0000 7f4f75177640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f700cea70 0x7f4f700d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:53.767 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.766+0000 7f4f75177640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f700cd9d0 con 0x7f4f7008be00 2024-05-20T18:26:53.767 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.766+0000 7f4f75177640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f4f700cd7a0 con 0x7f4f700b40c0 2024-05-20T18:26:53.767 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.766+0000 7f4f75177640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4f700ce5e0 con 0x7f4f700cea70 2024-05-20T18:26:53.767 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.767+0000 7f4f6f59e640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f4f7008be00 0x7f4f7008c210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46060/0 (socket says 172.21.15.5:46060) 2024-05-20T18:26:53.767 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.767+0000 7f4f6f59e640 1 -- 172.21.15.5:0/2466562359 learned_addr learned my addr 172.21.15.5:0/2466562359 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:26:53.767 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.767+0000 7f4f6e59c640 1 --2- 172.21.15.5:0/2466562359 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f700cea70 0x7f4f700d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:53.768 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.767+0000 7f4f6ed9d640 1 --2- 172.21.15.5:0/2466562359 >> v2:172.21.15.63:3300/0 conn(0x7f4f700b40c0 0x7f4f700b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:53.768 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.767+0000 7f4f6dd9b640 1 -- 172.21.15.5:0/2466562359 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3654974487 0 0) 0x7f4f700cd9d0 con 0x7f4f7008be00 2024-05-20T18:26:53.768 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.767+0000 7f4f6dd9b640 1 -- 172.21.15.5:0/2466562359 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f58003520 con 0x7f4f7008be00 2024-05-20T18:26:53.768 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.767+0000 7f4f6e59c640 1 -- 172.21.15.5:0/2466562359 >> v2:172.21.15.63:3300/0 conn(0x7f4f700b40c0 msgr2=0x7f4f700b80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:26:53.768 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.767+0000 7f4f6e59c640 1 --2- 172.21.15.5:0/2466562359 >> v2:172.21.15.63:3300/0 conn(0x7f4f700b40c0 0x7f4f700b80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:53.768 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.767+0000 7f4f6e59c640 1 -- 172.21.15.5:0/2466562359 >> v1:172.21.15.5:6789/0 conn(0x7f4f7008be00 legacy=0x7f4f7008c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:53.768 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.767+0000 7f4f6e59c640 1 -- 172.21.15.5:0/2466562359 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f700cdcb0 con 0x7f4f700cea70 2024-05-20T18:26:53.768 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.767+0000 7f4f6ed9d640 1 --2- 172.21.15.5:0/2466562359 >> v2:172.21.15.63:3300/0 conn(0x7f4f700b40c0 0x7f4f700b80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:26:53.768 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.768+0000 7f4f6e59c640 1 --2- 172.21.15.5:0/2466562359 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f700cea70 0x7f4f700d0ae0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4f640097a0 tx=0x7f4f6402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6638867879622e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:26:53.768 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.768+0000 7f4f6dd9b640 1 -- 172.21.15.5:0/2466562359 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f64037070 con 0x7f4f700cea70 2024-05-20T18:26:53.768 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.768+0000 7f4f6dd9b640 1 -- 172.21.15.5:0/2466562359 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f64032440 con 0x7f4f700cea70 2024-05-20T18:26:53.768 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.768+0000 7f4f6dd9b640 1 -- 172.21.15.5:0/2466562359 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f64032810 con 0x7f4f700cea70 2024-05-20T18:26:53.769 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.769+0000 7f4f75177640 1 -- 172.21.15.5:0/2466562359 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f700cea70 msgr2=0x7f4f700d0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:53.769 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.769+0000 7f4f75177640 1 --2- 172.21.15.5:0/2466562359 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f700cea70 0x7f4f700d0ae0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4f640097a0 tx=0x7f4f6402a180 comp rx=0 tx=0).stop 2024-05-20T18:26:53.769 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.769+0000 7f4f75177640 1 -- 172.21.15.5:0/2466562359 shutdown_connections 2024-05-20T18:26:53.769 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.769+0000 7f4f75177640 1 --2- 172.21.15.5:0/2466562359 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f700cea70 0x7f4f700d0ae0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:53.769 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.769+0000 7f4f75177640 1 --2- 172.21.15.5:0/2466562359 >> v2:172.21.15.63:3300/0 conn(0x7f4f700b40c0 0x7f4f700b80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:53.769 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.769+0000 7f4f75177640 1 -- 172.21.15.5:0/2466562359 >> 172.21.15.5:0/2466562359 conn(0x7f4f700b4540 msgr2=0x7f4f700b4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:26:53.770 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.769+0000 7f4f75177640 1 -- 172.21.15.5:0/2466562359 shutdown_connections 2024-05-20T18:26:53.770 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.769+0000 7f4f75177640 1 -- 172.21.15.5:0/2466562359 wait complete. 2024-05-20T18:26:53.770 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.771+0000 7f4f75177640 1 Processor -- start 2024-05-20T18:26:53.770 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.771+0000 7f4f75177640 1 -- start start 2024-05-20T18:26:53.772 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.772+0000 7f4f75177640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f7008be00 0x7f4f700cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:53.773 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.772+0000 7f4f75177640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f700b40c0 0x7f4f7014ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:53.773 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.772+0000 7f4f75177640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f700cea70 0x7f4f7014d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:26:53.773 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.772+0000 7f4f75177640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f700b58a0 con 0x7f4f700b40c0 2024-05-20T18:26:53.773 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.772+0000 7f4f75177640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4f700ce270 con 0x7f4f700cea70 2024-05-20T18:26:53.773 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.772+0000 7f4f75177640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4f700b5a20 con 0x7f4f7008be00 2024-05-20T18:26:53.773 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.772+0000 7f4f6e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f700b40c0 0x7f4f7014ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:53.773 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.773+0000 7f4f6e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f700b40c0 0x7f4f7014ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39676/0 (socket says 172.21.15.5:39676) 2024-05-20T18:26:53.773 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.773+0000 7f4f6f59e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f700cea70 0x7f4f7014d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:53.773 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.773+0000 7f4f6ed9d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f7008be00 0x7f4f700cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:26:53.773 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.773+0000 7f4f6e59c640 1 -- 172.21.15.5:0/1400709928 learned_addr learned my addr 172.21.15.5:0/1400709928 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:26:53.773 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.773+0000 7f4f6e59c640 1 -- 172.21.15.5:0/1400709928 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f7008be00 msgr2=0x7f4f700cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:53.773 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.773+0000 7f4f6e59c640 1 --2- 172.21.15.5:0/1400709928 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f7008be00 0x7f4f700cc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:53.774 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.773+0000 7f4f6e59c640 1 -- 172.21.15.5:0/1400709928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f700cea70 msgr2=0x7f4f7014d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:53.774 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.773+0000 7f4f6e59c640 1 --2- 172.21.15.5:0/1400709928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f700cea70 0x7f4f7014d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:53.774 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.773+0000 7f4f6e59c640 1 -- 172.21.15.5:0/1400709928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f700a01d0 con 0x7f4f700b40c0 2024-05-20T18:26:53.774 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.773+0000 7f4f6e59c640 1 --2- 172.21.15.5:0/1400709928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f700b40c0 0x7f4f7014ccf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4f64035040 tx=0x7f4f640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:26:53.774 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.773+0000 7f4f577fe640 1 -- 172.21.15.5:0/1400709928 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f64041030 con 0x7f4f700b40c0 2024-05-20T18:26:53.774 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.773+0000 7f4f577fe640 1 -- 172.21.15.5:0/1400709928 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f6403d4e0 con 0x7f4f700b40c0 2024-05-20T18:26:53.774 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.773+0000 7f4f577fe640 1 -- 172.21.15.5:0/1400709928 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f64037040 con 0x7f4f700b40c0 2024-05-20T18:26:53.774 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.773+0000 7f4f75177640 1 -- 172.21.15.5:0/1400709928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f7014ac60 con 0x7f4f700b40c0 2024-05-20T18:26:53.774 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.773+0000 7f4f75177640 1 -- 172.21.15.5:0/1400709928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f7014b220 con 0x7f4f700b40c0 2024-05-20T18:26:53.774 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.774+0000 7f4f577fe640 1 -- 172.21.15.5:0/1400709928 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4f64044070 con 0x7f4f700b40c0 2024-05-20T18:26:53.774 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.774+0000 7f4f577fe640 1 -- 172.21.15.5:0/1400709928 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f4f6403b3f0 con 0x7f4f700b40c0 2024-05-20T18:26:53.775 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.774+0000 7f4f75177640 1 -- 172.21.15.5:0/1400709928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f7009ff40 con 0x7f4f700b40c0 2024-05-20T18:26:53.775 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.776+0000 7f4f577fe640 1 -- 172.21.15.5:0/1400709928 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4f6404b070 con 0x7f4f700b40c0 2024-05-20T18:26:53.801 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-20T18:26:53.802+0000 7f3923cda040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-20T18:26:53.817 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-20T18:26:53.818+0000 7fd1b5923040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-20T18:26:53.837 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.839+0000 7f4f75177640 1 -- 172.21.15.5:0/1400709928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f70075980 con 0x7f4f700b40c0 2024-05-20T18:26:53.838 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.839+0000 7f4f577fe640 1 -- 172.21.15.5:0/1400709928 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f4f6403d8f0 con 0x7f4f700b40c0 2024-05-20T18:26:53.838 INFO:teuthology.misc.health.smithi005.stdout: 2024-05-20T18:26:53.838 INFO:teuthology.misc.health.smithi005.stdout:{"epoch":10,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:26:52.570603+0000","last_up_change":"0.000000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:26:53.839 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.840+0000 7f4f75177640 1 -- 172.21.15.5:0/1400709928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f700b40c0 msgr2=0x7f4f7014ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:26:53.839 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.840+0000 7f4f75177640 1 --2- 172.21.15.5:0/1400709928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f700b40c0 0x7f4f7014ccf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4f64035040 tx=0x7f4f640046c0 comp rx=0 tx=0).stop 2024-05-20T18:26:53.839 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.840+0000 7f4f75177640 1 -- 172.21.15.5:0/1400709928 shutdown_connections 2024-05-20T18:26:53.840 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.840+0000 7f4f75177640 1 --2- 172.21.15.5:0/1400709928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f700cea70 0x7f4f7014d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:53.840 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.840+0000 7f4f75177640 1 --2- 172.21.15.5:0/1400709928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f700b40c0 0x7f4f7014ccf0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:53.840 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.840+0000 7f4f75177640 1 --2- 172.21.15.5:0/1400709928 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f7008be00 0x7f4f700cc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:26:53.840 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.840+0000 7f4f75177640 1 -- 172.21.15.5:0/1400709928 >> 172.21.15.5:0/1400709928 conn(0x7f4f700b4540 msgr2=0x7f4f700b5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:26:53.840 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.840+0000 7f4f75177640 1 -- 172.21.15.5:0/1400709928 shutdown_connections 2024-05-20T18:26:53.840 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:26:53.840+0000 7f4f75177640 1 -- 172.21.15.5:0/1400709928 wait complete. 2024-05-20T18:26:53.849 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-20T18:26:53.856 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-20T18:26:53.857+0000 7f3923cda040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-20T18:26:53.892 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-20T18:26:53.893+0000 7fd1b5923040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-20T18:26:53.939 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-20T18:26:53.940+0000 7fd1b5923040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-20T18:26:53.978 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-20T18:26:53.979+0000 7f3923cda040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-20T18:26:54.285 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-20T18:26:54.286+0000 7f3923cda040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-20T18:26:54.335 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-20T18:26:54.336+0000 7f3923cda040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-20T18:26:54.441 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-20T18:26:54.442+0000 7fd1b5923040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-20T18:26:54.499 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-20T18:26:54.500+0000 7f3923cda040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-20T18:26:54.562 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-20T18:26:54.563+0000 7fd1b5923040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-20T18:26:54.646 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-20T18:26:54.647+0000 7f3923cda040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-20T18:26:54.698 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-20T18:26:54.699+0000 7f3923cda040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-20T18:26:54.771 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-20T18:26:54.772+0000 7f3923cda040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-20T18:26:54.819 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-20T18:26:54.820+0000 7f3923cda040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-20T18:26:55.320 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-20T18:26:55.321+0000 7f3923cda040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-20T18:26:55.442 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-20T18:26:55.443+0000 7f3923cda040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-20T18:26:55.583 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-20T18:26:55.580+0000 7fd156373640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-20T18:26:55.583 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-20T18:26:55.580+0000 7fd156373640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-20T18:26:56.682 INFO:tasks.ceph.osd.4.smithi063.stderr:2024-05-20T18:26:56.683+0000 7fe4afe11640 -1 osd.4 0 log_to_monitors true 2024-05-20T18:26:56.699 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-20T18:26:56.700+0000 7f08e01bc640 -1 osd.2 0 log_to_monitors true 2024-05-20T18:26:56.730 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-20T18:26:56.731+0000 7f49014b5640 -1 osd.1 0 log_to_monitors true 2024-05-20T18:26:56.737 INFO:tasks.ceph.osd.5.smithi063.stderr:2024-05-20T18:26:56.738+0000 7f1fc5620640 -1 osd.5 0 log_to_monitors true 2024-05-20T18:26:56.746 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-20T18:26:56.747+0000 7fbac460f640 -1 osd.3 0 log_to_monitors true 2024-05-20T18:26:56.753 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-20T18:26:56.754+0000 7ff019414640 -1 osd.0 0 log_to_monitors true 2024-05-20T18:26:56.757 INFO:tasks.ceph.osd.6.smithi063.stderr:2024-05-20T18:26:56.759+0000 7faa8eedd640 -1 osd.6 0 log_to_monitors true 2024-05-20T18:26:56.789 INFO:tasks.ceph.osd.7.smithi063.stderr:2024-05-20T18:26:56.790+0000 7f71bb150640 -1 osd.7 0 log_to_monitors true 2024-05-20T18:26:59.743 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-20T18:26:59.744+0000 7f08dc152640 -1 osd.2 0 waiting for initial osdmap 2024-05-20T18:26:59.764 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-20T18:26:59.765+0000 7f08d3f4d640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T18:26:59.766 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-20T18:26:59.767+0000 7fbabfe9d640 -1 osd.3 0 waiting for initial osdmap 2024-05-20T18:26:59.780 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-20T18:26:59.782+0000 7fbab7c98640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T18:26:59.790 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-20T18:26:59.792+0000 7ff014e59640 -1 osd.0 0 waiting for initial osdmap 2024-05-20T18:26:59.804 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-20T18:26:59.806+0000 7ff00cc54640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T18:26:59.822 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-20T18:26:59.823+0000 7f48fd44b640 -1 osd.1 0 waiting for initial osdmap 2024-05-20T18:26:59.840 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-20T18:26:59.841+0000 7f48f5246640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T18:27:00.441 INFO:tasks.ceph.osd.6.smithi063.stderr:2024-05-20T18:27:00.441+0000 7faa8ae73640 -1 osd.6 0 waiting for initial osdmap 2024-05-20T18:27:00.465 INFO:tasks.ceph.osd.6.smithi063.stderr:2024-05-20T18:27:00.467+0000 7faa82c6e640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T18:27:00.472 INFO:tasks.ceph.osd.5.smithi063.stderr:2024-05-20T18:27:00.474+0000 7f1fc0f17640 -1 osd.5 0 waiting for initial osdmap 2024-05-20T18:27:00.488 INFO:tasks.ceph.osd.5.smithi063.stderr:2024-05-20T18:27:00.490+0000 7f1fb8d12640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T18:27:00.503 INFO:tasks.ceph.osd.7.smithi063.stderr:2024-05-20T18:27:00.504+0000 7f71b70e6640 -1 osd.7 0 waiting for initial osdmap 2024-05-20T18:27:00.516 INFO:tasks.ceph.osd.4.smithi063.stderr:2024-05-20T18:27:00.518+0000 7fe4ab798640 -1 osd.4 0 waiting for initial osdmap 2024-05-20T18:27:00.523 INFO:tasks.ceph.osd.7.smithi063.stderr:2024-05-20T18:27:00.524+0000 7f71aeee1640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T18:27:00.531 INFO:tasks.ceph.osd.4.smithi063.stderr:2024-05-20T18:27:00.534+0000 7fe4a3593640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-20T18:27:00.656 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-20T18:27:00.782 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.783+0000 7f2499698640 1 Processor -- start 2024-05-20T18:27:00.782 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.783+0000 7f2499698640 1 -- start start 2024-05-20T18:27:00.783 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.784+0000 7f2499698640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f249406c630 0x7f2494163860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:00.784 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.784+0000 7f2499698640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f24940cffc0 0x7f24940d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:00.784 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.784+0000 7f2499698640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24940d3550 con 0x7f2494162d00 2024-05-20T18:27:00.784 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.784+0000 7f2499698640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f24940d4440 con 0x7f24940cffc0 2024-05-20T18:27:00.784 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.784+0000 7f2499698640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f24940d3830 con 0x7f249406c630 2024-05-20T18:27:00.784 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.784+0000 7f24937fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f24940cffc0 0x7f24940d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:00.784 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.784+0000 7f24937fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f24940cffc0 0x7f24940d03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45794/0 (socket says 172.21.15.5:45794) 2024-05-20T18:27:00.784 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.784+0000 7f24937fe640 1 -- 172.21.15.5:0/159964865 learned_addr learned my addr 172.21.15.5:0/159964865 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:00.784 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.784+0000 7f248bfff640 1 --2- 172.21.15.5:0/159964865 >> v2:172.21.15.63:3300/0 conn(0x7f249406c630 0x7f2494163860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:00.784 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.785+0000 7f24937fe640 1 -- 172.21.15.5:0/159964865 >> v2:172.21.15.63:3300/0 conn(0x7f249406c630 msgr2=0x7f2494163860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:00.784 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.785+0000 7f24937fe640 1 --2- 172.21.15.5:0/159964865 >> v2:172.21.15.63:3300/0 conn(0x7f249406c630 0x7f2494163860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:00.784 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.785+0000 7f24937fe640 1 -- 172.21.15.5:0/159964865 >> v1:172.21.15.63:6789/0 conn(0x7f2494162d00 legacy=0x7f24940cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:00.784 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.785+0000 7f24937fe640 1 -- 172.21.15.5:0/159964865 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24940d3b10 con 0x7f24940cffc0 2024-05-20T18:27:00.786 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.785+0000 7f24937fe640 1 --2- 172.21.15.5:0/159964865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24940cffc0 0x7f24940d03d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f248c009e50 tx=0x7f248c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f2b5f80d47ccf5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:00.786 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.785+0000 7f24927fc640 1 -- 172.21.15.5:0/159964865 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248c036070 con 0x7f24940cffc0 2024-05-20T18:27:00.786 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.786+0000 7f24927fc640 1 -- 172.21.15.5:0/159964865 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f248c031600 con 0x7f24940cffc0 2024-05-20T18:27:00.786 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.786+0000 7f24927fc640 1 -- 172.21.15.5:0/159964865 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248c0319d0 con 0x7f24940cffc0 2024-05-20T18:27:00.786 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.786+0000 7f2499698640 1 -- 172.21.15.5:0/159964865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24940cffc0 msgr2=0x7f24940d03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:00.786 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.786+0000 7f2499698640 1 --2- 172.21.15.5:0/159964865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24940cffc0 0x7f24940d03d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f248c009e50 tx=0x7f248c02a710 comp rx=0 tx=0).stop 2024-05-20T18:27:00.786 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.786+0000 7f2499698640 1 -- 172.21.15.5:0/159964865 shutdown_connections 2024-05-20T18:27:00.786 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.786+0000 7f2499698640 1 --2- 172.21.15.5:0/159964865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24940cffc0 0x7f24940d03d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:00.786 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.786+0000 7f2499698640 1 --2- 172.21.15.5:0/159964865 >> v2:172.21.15.63:3300/0 conn(0x7f249406c630 0x7f2494163860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:00.786 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.786+0000 7f2499698640 1 -- 172.21.15.5:0/159964865 >> 172.21.15.5:0/159964865 conn(0x7f24940d1680 msgr2=0x7f24940d1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:00.786 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.786+0000 7f2499698640 1 -- 172.21.15.5:0/159964865 shutdown_connections 2024-05-20T18:27:00.786 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.786+0000 7f2499698640 1 -- 172.21.15.5:0/159964865 wait complete. 2024-05-20T18:27:00.787 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.788+0000 7f2499698640 1 Processor -- start 2024-05-20T18:27:00.787 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.788+0000 7f2499698640 1 -- start start 2024-05-20T18:27:00.789 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.789+0000 7f2499698640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f249406c630 0x7f2494156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:00.790 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.789+0000 7f2499698640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24940cffc0 0x7f24940c7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:00.790 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.789+0000 7f2499698640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2494162d00 0x7f24941546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:00.790 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.789+0000 7f2499698640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f249415e850 con 0x7f24940cffc0 2024-05-20T18:27:00.790 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.789+0000 7f2499698640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f24940d40d0 con 0x7f2494162d00 2024-05-20T18:27:00.790 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.789+0000 7f2499698640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f249415e9d0 con 0x7f249406c630 2024-05-20T18:27:00.790 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.790+0000 7f248bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24940cffc0 0x7f24940c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:00.790 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.790+0000 7f248bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24940cffc0 0x7f24940c7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45804/0 (socket says 172.21.15.5:45804) 2024-05-20T18:27:00.790 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.790+0000 7f248bfff640 1 -- 172.21.15.5:0/1674469202 learned_addr learned my addr 172.21.15.5:0/1674469202 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:00.790 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.790+0000 7f24937fe640 1 --2- 172.21.15.5:0/1674469202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2494162d00 0x7f24941546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:00.790 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.790+0000 7f2492ffd640 1 --2- 172.21.15.5:0/1674469202 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f249406c630 0x7f2494156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:00.790 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.790+0000 7f248bfff640 1 -- 172.21.15.5:0/1674469202 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f249406c630 msgr2=0x7f2494156fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:00.791 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.790+0000 7f248bfff640 1 --2- 172.21.15.5:0/1674469202 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f249406c630 0x7f2494156fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:00.791 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.790+0000 7f248bfff640 1 -- 172.21.15.5:0/1674469202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2494162d00 msgr2=0x7f24941546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:00.791 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.790+0000 7f248bfff640 1 --2- 172.21.15.5:0/1674469202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2494162d00 0x7f24941546f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:00.791 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.790+0000 7f248bfff640 1 -- 172.21.15.5:0/1674469202 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2494155290 con 0x7f24940cffc0 2024-05-20T18:27:00.791 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.790+0000 7f248bfff640 1 --2- 172.21.15.5:0/1674469202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24940cffc0 0x7f24940c7610 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f247c00d5a0 tx=0x7f247c00da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:00.791 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.790+0000 7f248b7fe640 1 -- 172.21.15.5:0/1674469202 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c019070 con 0x7f24940cffc0 2024-05-20T18:27:00.791 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.790+0000 7f248b7fe640 1 -- 172.21.15.5:0/1674469202 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f247c004240 con 0x7f24940cffc0 2024-05-20T18:27:00.791 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.790+0000 7f2499698640 1 -- 172.21.15.5:0/1674469202 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2494155570 con 0x7f24940cffc0 2024-05-20T18:27:00.791 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.790+0000 7f248b7fe640 1 -- 172.21.15.5:0/1674469202 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c003610 con 0x7f24940cffc0 2024-05-20T18:27:00.791 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.790+0000 7f2499698640 1 -- 172.21.15.5:0/1674469202 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2494155b30 con 0x7f24940cffc0 2024-05-20T18:27:00.792 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.791+0000 7f2499698640 1 -- 172.21.15.5:0/1674469202 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2494155000 con 0x7f24940cffc0 2024-05-20T18:27:00.792 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.792+0000 7f248b7fe640 1 -- 172.21.15.5:0/1674469202 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f247c01c070 con 0x7f24940cffc0 2024-05-20T18:27:00.792 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.793+0000 7f248b7fe640 1 --2- 172.21.15.5:0/1674469202 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f247406bda0 0x7f247406e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:00.792 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.793+0000 7f2492ffd640 1 --2- 172.21.15.5:0/1674469202 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f247406bda0 0x7f247406e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:00.792 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.793+0000 7f248b7fe640 1 -- 172.21.15.5:0/1674469202 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7f247c08a1f0 con 0x7f24940cffc0 2024-05-20T18:27:00.792 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.793+0000 7f2492ffd640 1 --2- 172.21.15.5:0/1674469202 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f247406bda0 0x7f247406e2b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2484006fd0 tx=0x7f2484009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:00.797 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.798+0000 7f248b7fe640 1 -- 172.21.15.5:0/1674469202 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f247c058a70 con 0x7f24940cffc0 2024-05-20T18:27:00.987 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.989+0000 7f2499698640 1 -- 172.21.15.5:0/1674469202 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2494075980 con 0x7f24940cffc0 2024-05-20T18:27:00.988 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.989+0000 7f248b7fe640 1 -- 172.21.15.5:0/1674469202 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10789 (secure 0 0 0) 0x7f247c05cd70 con 0x7f24940cffc0 2024-05-20T18:27:00.988 INFO:teuthology.misc.health.smithi005.stdout: 2024-05-20T18:27:00.989 INFO:teuthology.misc.health.smithi005.stdout:{"epoch":13,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:27:00.747766+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:27:00.991 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.991+0000 7f2499698640 1 -- 172.21.15.5:0/1674469202 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f247406bda0 msgr2=0x7f247406e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:00.991 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.991+0000 7f2499698640 1 --2- 172.21.15.5:0/1674469202 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f247406bda0 0x7f247406e2b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2484006fd0 tx=0x7f2484009060 comp rx=0 tx=0).stop 2024-05-20T18:27:00.991 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.991+0000 7f2499698640 1 -- 172.21.15.5:0/1674469202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24940cffc0 msgr2=0x7f24940c7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:00.991 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.991+0000 7f2499698640 1 --2- 172.21.15.5:0/1674469202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24940cffc0 0x7f24940c7610 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f247c00d5a0 tx=0x7f247c00da60 comp rx=0 tx=0).stop 2024-05-20T18:27:00.991 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.992+0000 7f2499698640 1 -- 172.21.15.5:0/1674469202 shutdown_connections 2024-05-20T18:27:00.991 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.992+0000 7f2499698640 1 --2- 172.21.15.5:0/1674469202 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f247406bda0 0x7f247406e2b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:00.991 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.992+0000 7f2499698640 1 --2- 172.21.15.5:0/1674469202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2494162d00 0x7f24941546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:00.991 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.992+0000 7f2499698640 1 --2- 172.21.15.5:0/1674469202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24940cffc0 0x7f24940c7610 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:00.992 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.992+0000 7f2499698640 1 --2- 172.21.15.5:0/1674469202 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f249406c630 0x7f2494156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:00.992 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.992+0000 7f2499698640 1 -- 172.21.15.5:0/1674469202 >> 172.21.15.5:0/1674469202 conn(0x7f24940d1680 msgr2=0x7f249406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:00.992 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.992+0000 7f2499698640 1 -- 172.21.15.5:0/1674469202 shutdown_connections 2024-05-20T18:27:00.992 INFO:teuthology.misc.health.smithi005.stderr:2024-05-20T18:27:00.992+0000 7f2499698640 1 -- 172.21.15.5:0/1674469202 wait complete. 2024-05-20T18:27:01.001 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-20T18:27:01.001 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-20T18:27:01.001 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-20T18:27:01.001 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-20T18:27:01.004 INFO:tasks.ceph.mds.a:Started 2024-05-20T18:27:01.004 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-20T18:27:01.004 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-20T18:27:01.006 INFO:tasks.ceph.mds.c:Started 2024-05-20T18:27:01.006 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-20T18:27:01.006 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-20T18:27:01.009 INFO:tasks.ceph.mds.b:Started 2024-05-20T18:27:01.009 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-20T18:27:01.009 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-20T18:27:01.011 INFO:tasks.ceph.mds.d:Started 2024-05-20T18:27:01.011 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-20T18:27:01.012 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:27:01.094 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-05-20T18:27:01.094 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-05-20T18:27:01.105 INFO:tasks.ceph.mds.d.smithi063.stdout:starting mds.d at 2024-05-20T18:27:01.108 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-20T18:27:01.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.134+0000 7f7bdf014640 1 Processor -- start 2024-05-20T18:27:01.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.134+0000 7f7bdf014640 1 -- start start 2024-05-20T18:27:01.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.136+0000 7f7bdf014640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7bd815d530 0x7f7bd815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:01.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.136+0000 7f7bdf014640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7bd80b1700 0x7f7bd80b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:01.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.136+0000 7f7bdf014640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bd80cd580 con 0x7f7bd8070850 2024-05-20T18:27:01.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.136+0000 7f7bdf014640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f7bd80ce190 con 0x7f7bd815d530 2024-05-20T18:27:01.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.136+0000 7f7bdf014640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7bd80cd860 con 0x7f7bd80b1700 2024-05-20T18:27:01.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.136+0000 7f7bdcdb0640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7bd815d530 0x7f7bd815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:01.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.136+0000 7f7bdd5b1640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7bd8070850 0x7f7bd8070c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38492/0 (socket says 172.21.15.5:38492) 2024-05-20T18:27:01.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.136+0000 7f7bdd5b1640 1 -- 172.21.15.5:0/4128071431 learned_addr learned my addr 172.21.15.5:0/4128071431 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:01.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.136+0000 7f7bd7fff640 1 --2- 172.21.15.5:0/4128071431 >> v2:172.21.15.63:3300/0 conn(0x7f7bd80b1700 0x7f7bd80b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:01.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.136+0000 7f7bdcdb0640 1 -- 172.21.15.5:0/4128071431 >> v2:172.21.15.63:3300/0 conn(0x7f7bd80b1700 msgr2=0x7f7bd80b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:01.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.136+0000 7f7bdcdb0640 1 --2- 172.21.15.5:0/4128071431 >> v2:172.21.15.63:3300/0 conn(0x7f7bd80b1700 0x7f7bd80b1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:01.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.136+0000 7f7bdcdb0640 1 -- 172.21.15.5:0/4128071431 >> v1:172.21.15.5:6789/0 conn(0x7f7bd8070850 legacy=0x7f7bd8070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:01.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.136+0000 7f7bd77fe640 1 -- 172.21.15.5:0/4128071431 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1638147834 0 0) 0x7f7bd80cd580 con 0x7f7bd8070850 2024-05-20T18:27:01.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.136+0000 7f7bdcdb0640 1 -- 172.21.15.5:0/4128071431 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd80cdb40 con 0x7f7bd815d530 2024-05-20T18:27:01.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.136+0000 7f7bd7fff640 1 --2- 172.21.15.5:0/4128071431 >> v2:172.21.15.63:3300/0 conn(0x7f7bd80b1700 0x7f7bd80b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:01.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.137+0000 7f7bdcdb0640 1 --2- 172.21.15.5:0/4128071431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bd815d530 0x7f7bd815aaa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7bd80d3ad0 tx=0x7f7bcc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6450481f941ea6bf server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:01.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.137+0000 7f7bd77fe640 1 -- 172.21.15.5:0/4128071431 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc037070 con 0x7f7bd815d530 2024-05-20T18:27:01.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.137+0000 7f7bd77fe640 1 -- 172.21.15.5:0/4128071431 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bcc032440 con 0x7f7bd815d530 2024-05-20T18:27:01.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.137+0000 7f7bd77fe640 1 -- 172.21.15.5:0/4128071431 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc032810 con 0x7f7bd815d530 2024-05-20T18:27:01.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.138+0000 7f7bdf014640 1 -- 172.21.15.5:0/4128071431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bd815d530 msgr2=0x7f7bd815aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:01.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.138+0000 7f7bdf014640 1 --2- 172.21.15.5:0/4128071431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bd815d530 0x7f7bd815aaa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7bd80d3ad0 tx=0x7f7bcc02a1a0 comp rx=0 tx=0).stop 2024-05-20T18:27:01.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.138+0000 7f7bdf014640 1 -- 172.21.15.5:0/4128071431 shutdown_connections 2024-05-20T18:27:01.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.138+0000 7f7bdf014640 1 --2- 172.21.15.5:0/4128071431 >> v2:172.21.15.63:3300/0 conn(0x7f7bd80b1700 0x7f7bd80b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:01.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.138+0000 7f7bdf014640 1 --2- 172.21.15.5:0/4128071431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bd815d530 0x7f7bd815aaa0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:01.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.138+0000 7f7bdf014640 1 -- 172.21.15.5:0/4128071431 >> 172.21.15.5:0/4128071431 conn(0x7f7bd80d29d0 msgr2=0x7f7bd80d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:01.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.138+0000 7f7bdf014640 1 -- 172.21.15.5:0/4128071431 shutdown_connections 2024-05-20T18:27:01.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.138+0000 7f7bdf014640 1 -- 172.21.15.5:0/4128071431 wait complete. 2024-05-20T18:27:01.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.140+0000 7f7bdf014640 1 Processor -- start 2024-05-20T18:27:01.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.140+0000 7f7bdf014640 1 -- start start 2024-05-20T18:27:01.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.141+0000 7f7bdf014640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7bd8070850 0x7f7bd80c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:01.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.141+0000 7f7bdf014640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bd80b1700 0x7f7bd814ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:01.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.141+0000 7f7bdf014640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7bd815d530 0x7f7bd81579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:01.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.141+0000 7f7bdf014640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd80b2050 con 0x7f7bd80b1700 2024-05-20T18:27:01.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.141+0000 7f7bdf014640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd80cd230 con 0x7f7bd815d530 2024-05-20T18:27:01.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.141+0000 7f7bdf014640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7bd8071370 con 0x7f7bd8070850 2024-05-20T18:27:01.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.141+0000 7f7bd7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bd80b1700 0x7f7bd814ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:01.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.141+0000 7f7bd7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bd80b1700 0x7f7bd814ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45834/0 (socket says 172.21.15.5:45834) 2024-05-20T18:27:01.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.141+0000 7f7bd7fff640 1 -- 172.21.15.5:0/3609338328 learned_addr learned my addr 172.21.15.5:0/3609338328 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:01.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.141+0000 7f7bdd5b1640 1 --2- 172.21.15.5:0/3609338328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7bd815d530 0x7f7bd81579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:01.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.142+0000 7f7bdcdb0640 1 --2- 172.21.15.5:0/3609338328 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7bd8070850 0x7f7bd80c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:01.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.142+0000 7f7bd7fff640 1 -- 172.21.15.5:0/3609338328 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7bd8070850 msgr2=0x7f7bd80c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:01.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.142+0000 7f7bd7fff640 1 --2- 172.21.15.5:0/3609338328 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7bd8070850 0x7f7bd80c6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:01.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.142+0000 7f7bd7fff640 1 -- 172.21.15.5:0/3609338328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7bd815d530 msgr2=0x7f7bd81579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:01.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.142+0000 7f7bd7fff640 1 --2- 172.21.15.5:0/3609338328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7bd815d530 0x7f7bd81579b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:01.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.142+0000 7f7bd7fff640 1 -- 172.21.15.5:0/3609338328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd8156bf0 con 0x7f7bd80b1700 2024-05-20T18:27:01.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.142+0000 7f7bd7fff640 1 --2- 172.21.15.5:0/3609338328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bd80b1700 0x7f7bd814ce10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400d7d0 tx=0x7f7bc400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:01.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.142+0000 7f7bd57fa640 1 -- 172.21.15.5:0/3609338328 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4004030 con 0x7f7bd80b1700 2024-05-20T18:27:01.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.142+0000 7f7bd57fa640 1 -- 172.21.15.5:0/3609338328 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bc40041d0 con 0x7f7bd80b1700 2024-05-20T18:27:01.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.142+0000 7f7bdf014640 1 -- 172.21.15.5:0/3609338328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bd8156ed0 con 0x7f7bd80b1700 2024-05-20T18:27:01.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.142+0000 7f7bd57fa640 1 -- 172.21.15.5:0/3609338328 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4009640 con 0x7f7bd80b1700 2024-05-20T18:27:01.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.142+0000 7f7bdf014640 1 -- 172.21.15.5:0/3609338328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bd8157490 con 0x7f7bd80b1700 2024-05-20T18:27:01.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.143+0000 7f7bdf014640 1 -- 172.21.15.5:0/3609338328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bd8075b00 con 0x7f7bd80b1700 2024-05-20T18:27:01.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.144+0000 7f7bd57fa640 1 -- 172.21.15.5:0/3609338328 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f7bc40097e0 con 0x7f7bd80b1700 2024-05-20T18:27:01.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.144+0000 7f7bd57fa640 1 --2- 172.21.15.5:0/3609338328 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7bbc06bda0 0x7f7bbc06e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:01.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.144+0000 7f7bdcdb0640 1 --2- 172.21.15.5:0/3609338328 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7bbc06bda0 0x7f7bbc06e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:01.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.144+0000 7f7bd57fa640 1 -- 172.21.15.5:0/3609338328 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7f7bc408ab20 con 0x7f7bd80b1700 2024-05-20T18:27:01.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.145+0000 7f7bdcdb0640 1 --2- 172.21.15.5:0/3609338328 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7bbc06bda0 0x7f7bbc06e2b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc034000 tx=0x7f7bcc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:01.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.149+0000 7f7bd57fa640 1 -- 172.21.15.5:0/3609338328 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7bc4059a80 con 0x7f7bd80b1700 2024-05-20T18:27:01.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.336+0000 7f7bdf014640 1 -- 172.21.15.5:0/3609338328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7bd8067df0 con 0x7f7bd80b1700 2024-05-20T18:27:01.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.337+0000 7f7bd57fa640 1 -- 172.21.15.5:0/3609338328 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10789 (secure 0 0 0) 0x7f7bc4059380 con 0x7f7bd80b1700 2024-05-20T18:27:01.336 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:01.336 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":13,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:27:00.747766+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:27:01.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.339+0000 7f7bdf014640 1 -- 172.21.15.5:0/3609338328 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7bbc06bda0 msgr2=0x7f7bbc06e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:01.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.339+0000 7f7bdf014640 1 --2- 172.21.15.5:0/3609338328 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7bbc06bda0 0x7f7bbc06e2b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc034000 tx=0x7f7bcc02e000 comp rx=0 tx=0).stop 2024-05-20T18:27:01.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.339+0000 7f7bdf014640 1 -- 172.21.15.5:0/3609338328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bd80b1700 msgr2=0x7f7bd814ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:01.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.339+0000 7f7bdf014640 1 --2- 172.21.15.5:0/3609338328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bd80b1700 0x7f7bd814ce10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400d7d0 tx=0x7f7bc400dc90 comp rx=0 tx=0).stop 2024-05-20T18:27:01.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.339+0000 7f7bdf014640 1 -- 172.21.15.5:0/3609338328 shutdown_connections 2024-05-20T18:27:01.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.339+0000 7f7bdf014640 1 --2- 172.21.15.5:0/3609338328 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7bbc06bda0 0x7f7bbc06e2b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:01.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.339+0000 7f7bdf014640 1 --2- 172.21.15.5:0/3609338328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7bd815d530 0x7f7bd81579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:01.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.339+0000 7f7bdf014640 1 --2- 172.21.15.5:0/3609338328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bd80b1700 0x7f7bd814ce10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:01.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.339+0000 7f7bdf014640 1 --2- 172.21.15.5:0/3609338328 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7bd8070850 0x7f7bd80c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:01.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.339+0000 7f7bdf014640 1 -- 172.21.15.5:0/3609338328 >> 172.21.15.5:0/3609338328 conn(0x7f7bd80d29d0 msgr2=0x7f7bd8158f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:01.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.339+0000 7f7bdf014640 1 -- 172.21.15.5:0/3609338328 shutdown_connections 2024-05-20T18:27:01.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.339+0000 7f7bdf014640 1 -- 172.21.15.5:0/3609338328 wait complete. 2024-05-20T18:27:01.349 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-05-20T18:27:01.350 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:27:01.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.465+0000 7fe17bbd5640 1 Processor -- start 2024-05-20T18:27:01.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.465+0000 7fe17bbd5640 1 -- start start 2024-05-20T18:27:01.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.467+0000 7fe17bbd5640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe174163a10 0x7fe1740d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:01.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.467+0000 7fe17bbd5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1740d14b0 0x7fe1740d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:01.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.467+0000 7fe17bbd5640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1740d0370 con 0x7fe1740b4010 2024-05-20T18:27:01.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.467+0000 7fe17bbd5640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe1740d3ca0 con 0x7fe174163a10 2024-05-20T18:27:01.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.467+0000 7fe17bbd5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe1740d0650 con 0x7fe1740d14b0 2024-05-20T18:27:01.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.467+0000 7fe17a172640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1740d14b0 0x7fe1740d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:01.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.467+0000 7fe179971640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe174163a10 0x7fe1740d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:01.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.467+0000 7fe17a172640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1740d14b0 0x7fe1740d18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:53886/0 (socket says 172.21.15.5:53886) 2024-05-20T18:27:01.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.467+0000 7fe17a172640 1 -- 172.21.15.5:0/1471656990 learned_addr learned my addr 172.21.15.5:0/1471656990 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:01.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.467+0000 7fe179170640 1 --1- 172.21.15.5:0/1471656990 >> v1:172.21.15.63:6789/0 conn(0x7fe1740b4010 0x7fe1740cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:44422/0 (socket says 172.21.15.5:44422) 2024-05-20T18:27:01.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.467+0000 7fe179971640 1 -- 172.21.15.5:0/1471656990 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1740d14b0 msgr2=0x7fe1740d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:01.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.467+0000 7fe179971640 1 --2- 172.21.15.5:0/1471656990 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1740d14b0 0x7fe1740d18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:01.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.467+0000 7fe179971640 1 -- 172.21.15.5:0/1471656990 >> v1:172.21.15.63:6789/0 conn(0x7fe1740b4010 legacy=0x7fe1740cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:01.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.467+0000 7fe179971640 1 -- 172.21.15.5:0/1471656990 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1740d3650 con 0x7fe174163a10 2024-05-20T18:27:01.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.467+0000 7fe17a172640 1 --2- 172.21.15.5:0/1471656990 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1740d14b0 0x7fe1740d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:27:01.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.468+0000 7fe179971640 1 --2- 172.21.15.5:0/1471656990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe174163a10 0x7fe1740d4130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe1700097a0 tx=0x7fe17002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d6b43cb64b421bf server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:01.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.468+0000 7fe17896f640 1 -- 172.21.15.5:0/1471656990 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170037070 con 0x7fe174163a10 2024-05-20T18:27:01.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.468+0000 7fe17896f640 1 -- 172.21.15.5:0/1471656990 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe170032440 con 0x7fe174163a10 2024-05-20T18:27:01.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.468+0000 7fe17896f640 1 -- 172.21.15.5:0/1471656990 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170032810 con 0x7fe174163a10 2024-05-20T18:27:01.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.469+0000 7fe17bbd5640 1 -- 172.21.15.5:0/1471656990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe174163a10 msgr2=0x7fe1740d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:01.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.469+0000 7fe17bbd5640 1 --2- 172.21.15.5:0/1471656990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe174163a10 0x7fe1740d4130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe1700097a0 tx=0x7fe17002a180 comp rx=0 tx=0).stop 2024-05-20T18:27:01.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.469+0000 7fe17bbd5640 1 -- 172.21.15.5:0/1471656990 shutdown_connections 2024-05-20T18:27:01.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.469+0000 7fe17bbd5640 1 --2- 172.21.15.5:0/1471656990 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1740d14b0 0x7fe1740d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:01.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.469+0000 7fe17bbd5640 1 --2- 172.21.15.5:0/1471656990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe174163a10 0x7fe1740d4130 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:01.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.469+0000 7fe17bbd5640 1 -- 172.21.15.5:0/1471656990 >> 172.21.15.5:0/1471656990 conn(0x7fe17406d240 msgr2=0x7fe17406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:01.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.469+0000 7fe17bbd5640 1 -- 172.21.15.5:0/1471656990 shutdown_connections 2024-05-20T18:27:01.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.469+0000 7fe17bbd5640 1 -- 172.21.15.5:0/1471656990 wait complete. 2024-05-20T18:27:01.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.471+0000 7fe17bbd5640 1 Processor -- start 2024-05-20T18:27:01.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.471+0000 7fe17bbd5640 1 -- start start 2024-05-20T18:27:01.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.472+0000 7fe17bbd5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1740b4010 0x7fe1741617e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:01.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.472+0000 7fe17bbd5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1740d14b0 0x7fe174155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:01.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.472+0000 7fe17bbd5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe174163a10 0x7fe174156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:01.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.472+0000 7fe17bbd5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1740b8210 con 0x7fe1740b4010 2024-05-20T18:27:01.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.472+0000 7fe17bbd5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe1740d1e00 con 0x7fe174163a10 2024-05-20T18:27:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.472+0000 7fe17bbd5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe1740b8390 con 0x7fe1740d14b0 2024-05-20T18:27:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.472+0000 7fe179971640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1740b4010 0x7fe1741617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.472+0000 7fe179971640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1740b4010 0x7fe1741617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45850/0 (socket says 172.21.15.5:45850) 2024-05-20T18:27:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.472+0000 7fe179971640 1 -- 172.21.15.5:0/3293100840 learned_addr learned my addr 172.21.15.5:0/3293100840 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.473+0000 7fe17a172640 1 --2- 172.21.15.5:0/3293100840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe174163a10 0x7fe174156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.473+0000 7fe179170640 1 --2- 172.21.15.5:0/3293100840 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1740d14b0 0x7fe174155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.473+0000 7fe179971640 1 -- 172.21.15.5:0/3293100840 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1740d14b0 msgr2=0x7fe174155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.473+0000 7fe179971640 1 --2- 172.21.15.5:0/3293100840 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1740d14b0 0x7fe174155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.473+0000 7fe179971640 1 -- 172.21.15.5:0/3293100840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe174163a10 msgr2=0x7fe174156330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.473+0000 7fe179971640 1 --2- 172.21.15.5:0/3293100840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe174163a10 0x7fe174156330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.473+0000 7fe179971640 1 -- 172.21.15.5:0/3293100840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe174156870 con 0x7fe1740b4010 2024-05-20T18:27:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.473+0000 7fe179971640 1 --2- 172.21.15.5:0/3293100840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1740b4010 0x7fe1741617e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe1700098c0 tx=0x7fe1700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.473+0000 7fe16a7fc640 1 -- 172.21.15.5:0/3293100840 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170037040 con 0x7fe1740b4010 2024-05-20T18:27:01.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.473+0000 7fe16a7fc640 1 -- 172.21.15.5:0/3293100840 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe170030500 con 0x7fe1740b4010 2024-05-20T18:27:01.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.473+0000 7fe17bbd5640 1 -- 172.21.15.5:0/3293100840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe174155030 con 0x7fe1740b4010 2024-05-20T18:27:01.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.473+0000 7fe16a7fc640 1 -- 172.21.15.5:0/3293100840 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1700308d0 con 0x7fe1740b4010 2024-05-20T18:27:01.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.473+0000 7fe17bbd5640 1 -- 172.21.15.5:0/3293100840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1741555f0 con 0x7fe1740b4010 2024-05-20T18:27:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.474+0000 7fe17bbd5640 1 -- 172.21.15.5:0/3293100840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe174075b00 con 0x7fe1740b4010 2024-05-20T18:27:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.475+0000 7fe16a7fc640 1 -- 172.21.15.5:0/3293100840 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7fe170044020 con 0x7fe1740b4010 2024-05-20T18:27:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.475+0000 7fe16a7fc640 1 --2- 172.21.15.5:0/3293100840 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe15806bdd0 0x7fe15806e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.475+0000 7fe16a7fc640 1 -- 172.21.15.5:0/3293100840 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7fe1700aa920 con 0x7fe1740b4010 2024-05-20T18:27:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.475+0000 7fe179170640 1 --2- 172.21.15.5:0/3293100840 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe15806bdd0 0x7fe15806e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.476+0000 7fe179170640 1 --2- 172.21.15.5:0/3293100840 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe15806bdd0 0x7fe15806e2e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe1740d0090 tx=0x7fe164007ce0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:01.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.480+0000 7fe16a7fc640 1 -- 172.21.15.5:0/3293100840 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe170079880 con 0x7fe1740b4010 2024-05-20T18:27:01.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.709+0000 7fe17bbd5640 1 -- 172.21.15.5:0/3293100840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1740d3050 con 0x7fe1740b4010 2024-05-20T18:27:01.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.709+0000 7fe16a7fc640 1 -- 172.21.15.5:0/3293100840 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1389 (secure 0 0 0) 0x7fe170079180 con 0x7fe1740b4010 2024-05-20T18:27:01.708 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:01.709 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2982954341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2982954341},{"type":"v1","addr":"172.21.15.5:6836","nonce":2982954341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-20T18:27:01.709 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 2 2024-05-20T18:27:01.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.711+0000 7fe17bbd5640 1 -- 172.21.15.5:0/3293100840 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe15806bdd0 msgr2=0x7fe15806e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:01.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.711+0000 7fe17bbd5640 1 --2- 172.21.15.5:0/3293100840 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe15806bdd0 0x7fe15806e2e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe1740d0090 tx=0x7fe164007ce0 comp rx=0 tx=0).stop 2024-05-20T18:27:01.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.711+0000 7fe17bbd5640 1 -- 172.21.15.5:0/3293100840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1740b4010 msgr2=0x7fe1741617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:01.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.711+0000 7fe17bbd5640 1 --2- 172.21.15.5:0/3293100840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1740b4010 0x7fe1741617e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe1700098c0 tx=0x7fe1700046c0 comp rx=0 tx=0).stop 2024-05-20T18:27:01.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.711+0000 7fe17bbd5640 1 -- 172.21.15.5:0/3293100840 shutdown_connections 2024-05-20T18:27:01.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.711+0000 7fe17bbd5640 1 --2- 172.21.15.5:0/3293100840 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe15806bdd0 0x7fe15806e2e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:01.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.711+0000 7fe17bbd5640 1 --2- 172.21.15.5:0/3293100840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe174163a10 0x7fe174156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:01.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.711+0000 7fe17bbd5640 1 --2- 172.21.15.5:0/3293100840 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1740d14b0 0x7fe174155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:01.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.711+0000 7fe17bbd5640 1 --2- 172.21.15.5:0/3293100840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1740b4010 0x7fe1741617e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:01.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.711+0000 7fe17bbd5640 1 -- 172.21.15.5:0/3293100840 >> 172.21.15.5:0/3293100840 conn(0x7fe17406d240 msgr2=0x7fe17415b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:01.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.711+0000 7fe17bbd5640 1 -- 172.21.15.5:0/3293100840 shutdown_connections 2024-05-20T18:27:01.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:01.711+0000 7fe17bbd5640 1 -- 172.21.15.5:0/3293100840 wait complete. 2024-05-20T18:27:03.722 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:27:03.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.860+0000 7f259959d640 1 Processor -- start 2024-05-20T18:27:03.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.860+0000 7f259959d640 1 -- start start 2024-05-20T18:27:03.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.862+0000 7f259959d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f259415ce20 0x7f25941607e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:03.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.862+0000 7f259959d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f259415f620 0x7f259415fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:03.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.862+0000 7f259959d640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2594160270 con 0x7f25940d12a0 2024-05-20T18:27:03.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.862+0000 7f259959d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f25940cfb20 con 0x7f259415ce20 2024-05-20T18:27:03.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.862+0000 7f259959d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2594160550 con 0x7f259415f620 2024-05-20T18:27:03.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.862+0000 7f2593fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f259415ce20 0x7f25941607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:03.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.862+0000 7f25937fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f259415f620 0x7f259415fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:03.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.862+0000 7f2593fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f259415ce20 0x7f25941607e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45876/0 (socket says 172.21.15.5:45876) 2024-05-20T18:27:03.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.862+0000 7f2593fff640 1 -- 172.21.15.5:0/1577192296 learned_addr learned my addr 172.21.15.5:0/1577192296 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:03.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.862+0000 7f2593fff640 1 -- 172.21.15.5:0/1577192296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f259415f620 msgr2=0x7f259415fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:03.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.862+0000 7f2593fff640 1 --2- 172.21.15.5:0/1577192296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f259415f620 0x7f259415fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:03.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.862+0000 7f2593fff640 1 -- 172.21.15.5:0/1577192296 >> v1:172.21.15.63:6789/0 conn(0x7f25940d12a0 legacy=0x7f25940d16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:03.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.862+0000 7f2592ffd640 1 -- 172.21.15.5:0/1577192296 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3261804890 0 0) 0x7f2594160270 con 0x7f25940d12a0 2024-05-20T18:27:03.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.862+0000 7f2593fff640 1 -- 172.21.15.5:0/1577192296 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25940d2d60 con 0x7f259415ce20 2024-05-20T18:27:03.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.863+0000 7f2593fff640 1 --2- 172.21.15.5:0/1577192296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f259415ce20 0x7f25941607e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2584009870 tx=0x7f258402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b8b282b2bd6160 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:03.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.863+0000 7f2592ffd640 1 -- 172.21.15.5:0/1577192296 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2584037070 con 0x7f259415ce20 2024-05-20T18:27:03.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.863+0000 7f2592ffd640 1 -- 172.21.15.5:0/1577192296 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2584032440 con 0x7f259415ce20 2024-05-20T18:27:03.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.863+0000 7f2592ffd640 1 -- 172.21.15.5:0/1577192296 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2584032810 con 0x7f259415ce20 2024-05-20T18:27:03.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.864+0000 7f259959d640 1 -- 172.21.15.5:0/1577192296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f259415ce20 msgr2=0x7f25941607e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:03.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.864+0000 7f259959d640 1 --2- 172.21.15.5:0/1577192296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f259415ce20 0x7f25941607e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2584009870 tx=0x7f258402a270 comp rx=0 tx=0).stop 2024-05-20T18:27:03.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.864+0000 7f259959d640 1 -- 172.21.15.5:0/1577192296 shutdown_connections 2024-05-20T18:27:03.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.864+0000 7f259959d640 1 --2- 172.21.15.5:0/1577192296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f259415f620 0x7f259415fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:03.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.864+0000 7f259959d640 1 --2- 172.21.15.5:0/1577192296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f259415ce20 0x7f25941607e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:03.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.864+0000 7f259959d640 1 -- 172.21.15.5:0/1577192296 >> 172.21.15.5:0/1577192296 conn(0x7f259406dbb0 msgr2=0x7f259406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:03.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.864+0000 7f259959d640 1 -- 172.21.15.5:0/1577192296 shutdown_connections 2024-05-20T18:27:03.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.864+0000 7f259959d640 1 -- 172.21.15.5:0/1577192296 wait complete. 2024-05-20T18:27:03.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.866+0000 7f259959d640 1 Processor -- start 2024-05-20T18:27:03.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.866+0000 7f259959d640 1 -- start start 2024-05-20T18:27:03.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.868+0000 7f259959d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f25940d12a0 0x7f25940cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:03.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.868+0000 7f259959d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f259415f620 0x7f25941f85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:03.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.868+0000 7f259959d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2594158140 0x7f25941f8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:03.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.868+0000 7f259959d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f25940d3840 con 0x7f2594158140 2024-05-20T18:27:03.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.868+0000 7f259959d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f259415ff70 con 0x7f259415f620 2024-05-20T18:27:03.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.868+0000 7f259959d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f25940d39c0 con 0x7f25940d12a0 2024-05-20T18:27:03.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.868+0000 7f2593fff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f25940d12a0 0x7f25940cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:03.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.868+0000 7f2598d9c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2594158140 0x7f25941f8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:03.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.868+0000 7f25937fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f259415f620 0x7f25941f85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:03.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.868+0000 7f2593fff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f25940d12a0 0x7f25940cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:53940/0 (socket says 172.21.15.5:53940) 2024-05-20T18:27:03.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.868+0000 7f2598d9c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2594158140 0x7f25941f8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45878/0 (socket says 172.21.15.5:45878) 2024-05-20T18:27:03.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.868+0000 7f2593fff640 1 -- 172.21.15.5:0/1012309998 learned_addr learned my addr 172.21.15.5:0/1012309998 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:03.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.868+0000 7f25937fe640 1 --2- 172.21.15.5:0/1012309998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f259415f620 0x7f25941f85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:49672/0 (socket says 172.21.15.5:49672) 2024-05-20T18:27:03.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.868+0000 7f2598d9c640 1 -- 172.21.15.5:0/1012309998 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f25940d12a0 msgr2=0x7f25940cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:03.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.868+0000 7f2598d9c640 1 --2- 172.21.15.5:0/1012309998 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f25940d12a0 0x7f25940cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:03.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.868+0000 7f2598d9c640 1 -- 172.21.15.5:0/1012309998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f259415f620 msgr2=0x7f25941f85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:03.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.868+0000 7f2598d9c640 1 --2- 172.21.15.5:0/1012309998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f259415f620 0x7f25941f85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:03.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.868+0000 7f2598d9c640 1 -- 172.21.15.5:0/1012309998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25940c8710 con 0x7f2594158140 2024-05-20T18:27:03.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.868+0000 7f2593fff640 1 --2- 172.21.15.5:0/1012309998 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f25940d12a0 0x7f25940cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:27:03.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.868+0000 7f2598d9c640 1 --2- 172.21.15.5:0/1012309998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2594158140 0x7f25941f8ae0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f258c00c750 tx=0x7f258c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:03.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.868+0000 7f2590ff9640 1 -- 172.21.15.5:0/1012309998 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c003b70 con 0x7f2594158140 2024-05-20T18:27:03.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.869+0000 7f2590ff9640 1 -- 172.21.15.5:0/1012309998 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f258c003d30 con 0x7f2594158140 2024-05-20T18:27:03.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.869+0000 7f259959d640 1 -- 172.21.15.5:0/1012309998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25940caae0 con 0x7f2594158140 2024-05-20T18:27:03.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.869+0000 7f2590ff9640 1 -- 172.21.15.5:0/1012309998 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c0052b0 con 0x7f2594158140 2024-05-20T18:27:03.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.869+0000 7f259959d640 1 -- 172.21.15.5:0/1012309998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25940b6550 con 0x7f2594158140 2024-05-20T18:27:03.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.870+0000 7f259959d640 1 -- 172.21.15.5:0/1012309998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25940b67b0 con 0x7f2594158140 2024-05-20T18:27:03.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.870+0000 7f2590ff9640 1 -- 172.21.15.5:0/1012309998 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f258c005450 con 0x7f2594158140 2024-05-20T18:27:03.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.871+0000 7f2590ff9640 1 --2- 172.21.15.5:0/1012309998 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f258006bda0 0x7f258006e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:03.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.871+0000 7f2593fff640 1 --2- 172.21.15.5:0/1012309998 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f258006bda0 0x7f258006e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:03.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.871+0000 7f2590ff9640 1 -- 172.21.15.5:0/1012309998 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f258c08a780 con 0x7f2594158140 2024-05-20T18:27:03.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.871+0000 7f2593fff640 1 --2- 172.21.15.5:0/1012309998 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f258006bda0 0x7f258006e2b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2584000c00 tx=0x7f258402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:03.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:03.876+0000 7f2590ff9640 1 -- 172.21.15.5:0/1012309998 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f258c058e80 con 0x7f2594158140 2024-05-20T18:27:04.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.107+0000 7f259959d640 1 -- 172.21.15.5:0/1012309998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f25940c84a0 con 0x7f2594158140 2024-05-20T18:27:04.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.107+0000 7f2590ff9640 1 -- 172.21.15.5:0/1012309998 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3855 (secure 0 0 0) 0x7f258c05d180 con 0x7f2594158140 2024-05-20T18:27:04.106 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:04.106 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2982954341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2982954341},{"type":"v1","addr":"172.21.15.5:6836","nonce":2982954341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4032850001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":4032850001},{"type":"v1","addr":"172.21.15.5:6837","nonce":4032850001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6833/424580378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":424580378},{"type":"v1","addr":"172.21.15.63:6833","nonce":424580378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3249423677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3249423677},{"type":"v1","addr":"172.21.15.63:6835","nonce":3249423677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-20T18:27:04.107 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 3 2024-05-20T18:27:04.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.109+0000 7f259959d640 1 -- 172.21.15.5:0/1012309998 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f258006bda0 msgr2=0x7f258006e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:04.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.109+0000 7f259959d640 1 --2- 172.21.15.5:0/1012309998 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f258006bda0 0x7f258006e2b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2584000c00 tx=0x7f258402e000 comp rx=0 tx=0).stop 2024-05-20T18:27:04.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.109+0000 7f259959d640 1 -- 172.21.15.5:0/1012309998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2594158140 msgr2=0x7f25941f8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:04.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.109+0000 7f259959d640 1 --2- 172.21.15.5:0/1012309998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2594158140 0x7f25941f8ae0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f258c00c750 tx=0x7f258c00cc10 comp rx=0 tx=0).stop 2024-05-20T18:27:04.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.109+0000 7f259959d640 1 -- 172.21.15.5:0/1012309998 shutdown_connections 2024-05-20T18:27:04.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.109+0000 7f259959d640 1 --2- 172.21.15.5:0/1012309998 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f258006bda0 0x7f258006e2b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:04.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.109+0000 7f259959d640 1 --2- 172.21.15.5:0/1012309998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2594158140 0x7f25941f8ae0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:04.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.109+0000 7f259959d640 1 --2- 172.21.15.5:0/1012309998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f259415f620 0x7f25941f85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:04.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.109+0000 7f259959d640 1 --2- 172.21.15.5:0/1012309998 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f25940d12a0 0x7f25940cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:04.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.109+0000 7f259959d640 1 -- 172.21.15.5:0/1012309998 >> 172.21.15.5:0/1012309998 conn(0x7f259406dbb0 msgr2=0x7f25940d3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:04.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.109+0000 7f259959d640 1 -- 172.21.15.5:0/1012309998 shutdown_connections 2024-05-20T18:27:04.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.109+0000 7f259959d640 1 -- 172.21.15.5:0/1012309998 wait complete. 2024-05-20T18:27:04.120 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:27:04.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.238+0000 7ff354dce640 1 Processor -- start 2024-05-20T18:27:04.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.238+0000 7ff354dce640 1 -- start start 2024-05-20T18:27:04.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.240+0000 7ff354dce640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff3500b23f0 0x7ff3500af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:04.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.240+0000 7ff354dce640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff3500b1650 0x7ff35015c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:04.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.240+0000 7ff354dce640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff35009a740 con 0x7ff350159ca0 2024-05-20T18:27:04.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.240+0000 7ff354dce640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7ff35009a4f0 con 0x7ff3500b1650 2024-05-20T18:27:04.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.240+0000 7ff354dce640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff35009b350 con 0x7ff3500b23f0 2024-05-20T18:27:04.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.241+0000 7ff34ed9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff3500b1650 0x7ff35015c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:04.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.241+0000 7ff34e59c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff350159ca0 0x7ff3500b1ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38508/0 (socket says 172.21.15.5:38508) 2024-05-20T18:27:04.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.241+0000 7ff34e59c640 1 -- 172.21.15.5:0/699672657 learned_addr learned my addr 172.21.15.5:0/699672657 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:04.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.241+0000 7ff34dd9b640 1 --2- 172.21.15.5:0/699672657 >> v2:172.21.15.63:3300/0 conn(0x7ff3500b23f0 0x7ff3500af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:04.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.241+0000 7ff34d59a640 1 -- 172.21.15.5:0/699672657 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2299365608 0 0) 0x7ff35009a740 con 0x7ff350159ca0 2024-05-20T18:27:04.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.241+0000 7ff34d59a640 1 -- 172.21.15.5:0/699672657 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff338003520 con 0x7ff350159ca0 2024-05-20T18:27:04.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.241+0000 7ff34ed9d640 1 -- 172.21.15.5:0/699672657 >> v2:172.21.15.63:3300/0 conn(0x7ff3500b23f0 msgr2=0x7ff3500af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:04.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.241+0000 7ff34ed9d640 1 --2- 172.21.15.5:0/699672657 >> v2:172.21.15.63:3300/0 conn(0x7ff3500b23f0 0x7ff3500af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:04.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.241+0000 7ff34ed9d640 1 -- 172.21.15.5:0/699672657 >> v1:172.21.15.5:6789/0 conn(0x7ff350159ca0 legacy=0x7ff3500b1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:04.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.241+0000 7ff34ed9d640 1 -- 172.21.15.5:0/699672657 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff35009aa20 con 0x7ff3500b1650 2024-05-20T18:27:04.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.242+0000 7ff34ed9d640 1 --2- 172.21.15.5:0/699672657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3500b1650 0x7ff35015c760 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff35006e7f0 tx=0x7ff34802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3505675c9774efec server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:04.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.242+0000 7ff34d59a640 1 -- 172.21.15.5:0/699672657 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff348007740 con 0x7ff3500b1650 2024-05-20T18:27:04.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.242+0000 7ff34d59a640 1 -- 172.21.15.5:0/699672657 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3480078e0 con 0x7ff3500b1650 2024-05-20T18:27:04.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.242+0000 7ff34d59a640 1 -- 172.21.15.5:0/699672657 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff348007cb0 con 0x7ff3500b1650 2024-05-20T18:27:04.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.243+0000 7ff354dce640 1 -- 172.21.15.5:0/699672657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3500b1650 msgr2=0x7ff35015c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:04.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.243+0000 7ff354dce640 1 --2- 172.21.15.5:0/699672657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3500b1650 0x7ff35015c760 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff35006e7f0 tx=0x7ff34802a5f0 comp rx=0 tx=0).stop 2024-05-20T18:27:04.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.243+0000 7ff354dce640 1 -- 172.21.15.5:0/699672657 shutdown_connections 2024-05-20T18:27:04.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.243+0000 7ff354dce640 1 --2- 172.21.15.5:0/699672657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3500b1650 0x7ff35015c760 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:04.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.243+0000 7ff354dce640 1 --2- 172.21.15.5:0/699672657 >> v2:172.21.15.63:3300/0 conn(0x7ff3500b23f0 0x7ff3500af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:04.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.243+0000 7ff354dce640 1 -- 172.21.15.5:0/699672657 >> 172.21.15.5:0/699672657 conn(0x7ff3500b6620 msgr2=0x7ff3500b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:04.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.243+0000 7ff354dce640 1 -- 172.21.15.5:0/699672657 shutdown_connections 2024-05-20T18:27:04.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.243+0000 7ff354dce640 1 -- 172.21.15.5:0/699672657 wait complete. 2024-05-20T18:27:04.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.245+0000 7ff354dce640 1 Processor -- start 2024-05-20T18:27:04.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.245+0000 7ff354dce640 1 -- start start 2024-05-20T18:27:04.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.246+0000 7ff354dce640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3500b1650 0x7ff35009b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:04.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.246+0000 7ff354dce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3500b23f0 0x7ff35014b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:04.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.246+0000 7ff354dce640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff350159ca0 0x7ff35015b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:04.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.246+0000 7ff354dce640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff35009afe0 con 0x7ff3500b23f0 2024-05-20T18:27:04.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.246+0000 7ff354dce640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff3500af770 con 0x7ff350159ca0 2024-05-20T18:27:04.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.246+0000 7ff354dce640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff35006c630 con 0x7ff3500b1650 2024-05-20T18:27:04.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.247+0000 7ff34dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3500b23f0 0x7ff35014b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.247+0000 7ff34dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3500b23f0 0x7ff35014b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45902/0 (socket says 172.21.15.5:45902) 2024-05-20T18:27:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.247+0000 7ff34dd9b640 1 -- 172.21.15.5:0/4106695959 learned_addr learned my addr 172.21.15.5:0/4106695959 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.247+0000 7ff34ed9d640 1 --2- 172.21.15.5:0/4106695959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff350159ca0 0x7ff35015b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.247+0000 7ff34e59c640 1 --2- 172.21.15.5:0/4106695959 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3500b1650 0x7ff35009b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.247+0000 7ff34dd9b640 1 -- 172.21.15.5:0/4106695959 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3500b1650 msgr2=0x7ff35009b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.247+0000 7ff34dd9b640 1 --2- 172.21.15.5:0/4106695959 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3500b1650 0x7ff35009b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.247+0000 7ff34dd9b640 1 -- 172.21.15.5:0/4106695959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff350159ca0 msgr2=0x7ff35015b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.247+0000 7ff34dd9b640 1 --2- 172.21.15.5:0/4106695959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff350159ca0 0x7ff35015b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.247+0000 7ff34dd9b640 1 -- 172.21.15.5:0/4106695959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff35009c110 con 0x7ff3500b23f0 2024-05-20T18:27:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.247+0000 7ff34ed9d640 1 --2- 172.21.15.5:0/4106695959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff350159ca0 0x7ff35015b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:27:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.247+0000 7ff34dd9b640 1 --2- 172.21.15.5:0/4106695959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3500b23f0 0x7ff35014b820 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff34400e8a0 tx=0x7ff34400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.248+0000 7ff33effd640 1 -- 172.21.15.5:0/4106695959 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff344017070 con 0x7ff3500b23f0 2024-05-20T18:27:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.248+0000 7ff33effd640 1 -- 172.21.15.5:0/4106695959 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff34400c940 con 0x7ff3500b23f0 2024-05-20T18:27:04.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.248+0000 7ff354dce640 1 -- 172.21.15.5:0/4106695959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff35009c3f0 con 0x7ff3500b23f0 2024-05-20T18:27:04.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.248+0000 7ff33effd640 1 -- 172.21.15.5:0/4106695959 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34400cd30 con 0x7ff3500b23f0 2024-05-20T18:27:04.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.248+0000 7ff354dce640 1 -- 172.21.15.5:0/4106695959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff35009c9b0 con 0x7ff3500b23f0 2024-05-20T18:27:04.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.249+0000 7ff354dce640 1 -- 172.21.15.5:0/4106695959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff350075b00 con 0x7ff3500b23f0 2024-05-20T18:27:04.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.251+0000 7ff33effd640 1 -- 172.21.15.5:0/4106695959 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7ff34401a070 con 0x7ff3500b23f0 2024-05-20T18:27:04.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.251+0000 7ff33effd640 1 --2- 172.21.15.5:0/4106695959 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff33006bda0 0x7ff33006e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:04.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.251+0000 7ff33effd640 1 -- 172.21.15.5:0/4106695959 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7ff34408ab50 con 0x7ff3500b23f0 2024-05-20T18:27:04.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.251+0000 7ff34e59c640 1 --2- 172.21.15.5:0/4106695959 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff33006bda0 0x7ff33006e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:04.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.252+0000 7ff34e59c640 1 --2- 172.21.15.5:0/4106695959 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff33006bda0 0x7ff33006e2b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff340005880 tx=0x7ff340009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:04.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.256+0000 7ff33effd640 1 -- 172.21.15.5:0/4106695959 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff3440592e0 con 0x7ff3500b23f0 2024-05-20T18:27:04.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.444+0000 7ff354dce640 1 -- 172.21.15.5:0/4106695959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff35009be60 con 0x7ff3500b23f0 2024-05-20T18:27:04.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.445+0000 7ff33effd640 1 -- 172.21.15.5:0/4106695959 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12777 (secure 0 0 0) 0x7ff34405d5e0 con 0x7ff3500b23f0 2024-05-20T18:27:04.444 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:04.444 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":16,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:27:03.752573+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:27:04.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.447+0000 7ff354dce640 1 -- 172.21.15.5:0/4106695959 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff33006bda0 msgr2=0x7ff33006e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:04.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.447+0000 7ff354dce640 1 --2- 172.21.15.5:0/4106695959 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff33006bda0 0x7ff33006e2b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff340005880 tx=0x7ff340009040 comp rx=0 tx=0).stop 2024-05-20T18:27:04.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.447+0000 7ff354dce640 1 -- 172.21.15.5:0/4106695959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3500b23f0 msgr2=0x7ff35014b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:04.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.447+0000 7ff354dce640 1 --2- 172.21.15.5:0/4106695959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3500b23f0 0x7ff35014b820 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff34400e8a0 tx=0x7ff34400ed60 comp rx=0 tx=0).stop 2024-05-20T18:27:04.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.447+0000 7ff354dce640 1 -- 172.21.15.5:0/4106695959 shutdown_connections 2024-05-20T18:27:04.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.447+0000 7ff354dce640 1 --2- 172.21.15.5:0/4106695959 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff33006bda0 0x7ff33006e2b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:04.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.447+0000 7ff354dce640 1 --2- 172.21.15.5:0/4106695959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff350159ca0 0x7ff35015b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:04.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.447+0000 7ff354dce640 1 --2- 172.21.15.5:0/4106695959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3500b23f0 0x7ff35014b820 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:04.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.447+0000 7ff354dce640 1 --2- 172.21.15.5:0/4106695959 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3500b1650 0x7ff35009b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:04.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.447+0000 7ff354dce640 1 -- 172.21.15.5:0/4106695959 >> 172.21.15.5:0/4106695959 conn(0x7ff3500b6620 msgr2=0x7ff3500af920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:04.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.447+0000 7ff354dce640 1 -- 172.21.15.5:0/4106695959 shutdown_connections 2024-05-20T18:27:04.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.447+0000 7ff354dce640 1 -- 172.21.15.5:0/4106695959 wait complete. 2024-05-20T18:27:04.457 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:27:04.457 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:27:04.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.577+0000 7f0651efe640 1 Processor -- start 2024-05-20T18:27:04.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.577+0000 7f0651efe640 1 -- start start 2024-05-20T18:27:04.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.579+0000 7f0651efe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f064c162820 0x7f064c0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:04.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.579+0000 7f0651efe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f064c0b3e80 0x7f064c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:04.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.579+0000 7f0651efe640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f064c06dd20 con 0x7f064c06d910 2024-05-20T18:27:04.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.579+0000 7f0651efe640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f064c0d0630 con 0x7f064c162820 2024-05-20T18:27:04.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.579+0000 7f0651efe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f064c0d39c0 con 0x7f064c0b3e80 2024-05-20T18:27:04.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.579+0000 7f064b7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f064c162820 0x7f064c0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:04.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.579+0000 7f064b7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f064c162820 0x7f064c0b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45916/0 (socket says 172.21.15.5:45916) 2024-05-20T18:27:04.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.579+0000 7f064b7fe640 1 -- 172.21.15.5:0/4236335028 learned_addr learned my addr 172.21.15.5:0/4236335028 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:04.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.579+0000 7f064affd640 1 --2- 172.21.15.5:0/4236335028 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f064c0b3e80 0x7f064c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:04.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.579+0000 7f064b7fe640 1 -- 172.21.15.5:0/4236335028 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f064c0b3e80 msgr2=0x7f064c06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:04.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.579+0000 7f064b7fe640 1 --2- 172.21.15.5:0/4236335028 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f064c0b3e80 0x7f064c06d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:04.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.579+0000 7f064b7fe640 1 -- 172.21.15.5:0/4236335028 >> v1:172.21.15.63:6789/0 conn(0x7f064c06d910 legacy=0x7f064c0d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:04.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.579+0000 7f064b7fe640 1 -- 172.21.15.5:0/4236335028 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f064c0d3ca0 con 0x7f064c162820 2024-05-20T18:27:04.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.580+0000 7f064b7fe640 1 --2- 172.21.15.5:0/4236335028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f064c162820 0x7f064c0b3940 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f063c009e50 tx=0x7f063c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ddc85dd9a7d488 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:04.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.580+0000 7f064a7fc640 1 -- 172.21.15.5:0/4236335028 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c036070 con 0x7f064c162820 2024-05-20T18:27:04.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.580+0000 7f064a7fc640 1 -- 172.21.15.5:0/4236335028 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f063c031600 con 0x7f064c162820 2024-05-20T18:27:04.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.580+0000 7f064a7fc640 1 -- 172.21.15.5:0/4236335028 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c0319d0 con 0x7f064c162820 2024-05-20T18:27:04.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.581+0000 7f0651efe640 1 -- 172.21.15.5:0/4236335028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f064c162820 msgr2=0x7f064c0b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:04.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.581+0000 7f0651efe640 1 --2- 172.21.15.5:0/4236335028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f064c162820 0x7f064c0b3940 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f063c009e50 tx=0x7f063c02a710 comp rx=0 tx=0).stop 2024-05-20T18:27:04.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.581+0000 7f0651efe640 1 -- 172.21.15.5:0/4236335028 shutdown_connections 2024-05-20T18:27:04.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.581+0000 7f0651efe640 1 --2- 172.21.15.5:0/4236335028 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f064c0b3e80 0x7f064c06d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:04.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.581+0000 7f0651efe640 1 --2- 172.21.15.5:0/4236335028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f064c162820 0x7f064c0b3940 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:04.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.581+0000 7f0651efe640 1 -- 172.21.15.5:0/4236335028 >> 172.21.15.5:0/4236335028 conn(0x7f064c0d4bd0 msgr2=0x7f064c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:04.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.581+0000 7f0651efe640 1 -- 172.21.15.5:0/4236335028 shutdown_connections 2024-05-20T18:27:04.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.581+0000 7f0651efe640 1 -- 172.21.15.5:0/4236335028 wait complete. 2024-05-20T18:27:04.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.583+0000 7f0651efe640 1 Processor -- start 2024-05-20T18:27:04.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.583+0000 7f0651efe640 1 -- start start 2024-05-20T18:27:04.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.584+0000 7f0651efe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f064c06d910 0x7f064c160000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:04.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.584+0000 7f0651efe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f064c0b3e80 0x7f064c157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:04.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.584+0000 7f0651efe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f064c162820 0x7f064c0b4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:04.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.584+0000 7f0651efe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f064c0d0300 con 0x7f064c162820 2024-05-20T18:27:04.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.584+0000 7f0651efe640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f064c0d4260 con 0x7f064c0b3e80 2024-05-20T18:27:04.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.584+0000 7f0651efe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f064c0b91a0 con 0x7f064c06d910 2024-05-20T18:27:04.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.584+0000 7f064bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f064c162820 0x7f064c0b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:04.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.584+0000 7f064b7fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f064c06d910 0x7f064c160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:04.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.584+0000 7f064affd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f064c0b3e80 0x7f064c157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:04.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.584+0000 7f064bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f064c162820 0x7f064c0b4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45918/0 (socket says 172.21.15.5:45918) 2024-05-20T18:27:04.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.584+0000 7f064affd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f064c0b3e80 0x7f064c157070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:49700/0 (socket says 172.21.15.5:49700) 2024-05-20T18:27:04.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.584+0000 7f064b7fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f064c06d910 0x7f064c160000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:53960/0 (socket says 172.21.15.5:53960) 2024-05-20T18:27:04.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.584+0000 7f064bfff640 1 -- 172.21.15.5:0/1435292079 learned_addr learned my addr 172.21.15.5:0/1435292079 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:04.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.585+0000 7f064bfff640 1 -- 172.21.15.5:0/1435292079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f064c06d910 msgr2=0x7f064c160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:04.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.585+0000 7f064bfff640 1 --2- 172.21.15.5:0/1435292079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f064c06d910 0x7f064c160000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:04.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.585+0000 7f064bfff640 1 -- 172.21.15.5:0/1435292079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f064c0b3e80 msgr2=0x7f064c157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:04.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.585+0000 7f064bfff640 1 --2- 172.21.15.5:0/1435292079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f064c0b3e80 0x7f064c157070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:04.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.585+0000 7f064bfff640 1 -- 172.21.15.5:0/1435292079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f064c160ba0 con 0x7f064c162820 2024-05-20T18:27:04.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.585+0000 7f064bfff640 1 --2- 172.21.15.5:0/1435292079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f064c162820 0x7f064c0b4630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f064400c3c0 tx=0x7f064400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:04.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.585+0000 7f0643fff640 1 -- 172.21.15.5:0/1435292079 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0644017070 con 0x7f064c162820 2024-05-20T18:27:04.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.585+0000 7f0643fff640 1 -- 172.21.15.5:0/1435292079 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06440052d0 con 0x7f064c162820 2024-05-20T18:27:04.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.585+0000 7f0651efe640 1 -- 172.21.15.5:0/1435292079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f064c1f56b0 con 0x7f064c162820 2024-05-20T18:27:04.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.585+0000 7f0643fff640 1 -- 172.21.15.5:0/1435292079 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0644003830 con 0x7f064c162820 2024-05-20T18:27:04.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.585+0000 7f0651efe640 1 -- 172.21.15.5:0/1435292079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f064c1f5950 con 0x7f064c162820 2024-05-20T18:27:04.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.586+0000 7f0651efe640 1 -- 172.21.15.5:0/1435292079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f064c160630 con 0x7f064c162820 2024-05-20T18:27:04.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.587+0000 7f0643fff640 1 -- 172.21.15.5:0/1435292079 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f064401a070 con 0x7f064c162820 2024-05-20T18:27:04.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.587+0000 7f0643fff640 1 --2- 172.21.15.5:0/1435292079 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f062c06bda0 0x7f062c06e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:04.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.587+0000 7f0643fff640 1 -- 172.21.15.5:0/1435292079 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f0644089d80 con 0x7f064c162820 2024-05-20T18:27:04.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.587+0000 7f064b7fe640 1 --2- 172.21.15.5:0/1435292079 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f062c06bda0 0x7f062c06e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:04.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.588+0000 7f064b7fe640 1 --2- 172.21.15.5:0/1435292079 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f062c06bda0 0x7f062c06e2b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f063c033000 tx=0x7f063c009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:04.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.592+0000 7f0643fff640 1 -- 172.21.15.5:0/1435292079 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f064c06dd20 con 0x7f064c162820 2024-05-20T18:27:04.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.778+0000 7f0651efe640 1 -- 172.21.15.5:0/1435292079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f064c075b00 con 0x7f064c162820 2024-05-20T18:27:04.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.779+0000 7f0643fff640 1 -- 172.21.15.5:0/1435292079 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7f0644058480 con 0x7f064c162820 2024-05-20T18:27:04.778 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:27:04.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.781+0000 7f0651efe640 1 -- 172.21.15.5:0/1435292079 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f062c06bda0 msgr2=0x7f062c06e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:04.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.781+0000 7f0651efe640 1 --2- 172.21.15.5:0/1435292079 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f062c06bda0 0x7f062c06e2b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f063c033000 tx=0x7f063c009000 comp rx=0 tx=0).stop 2024-05-20T18:27:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.781+0000 7f0651efe640 1 -- 172.21.15.5:0/1435292079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f064c162820 msgr2=0x7f064c0b4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.781+0000 7f0651efe640 1 --2- 172.21.15.5:0/1435292079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f064c162820 0x7f064c0b4630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f064400c3c0 tx=0x7f064400c7c0 comp rx=0 tx=0).stop 2024-05-20T18:27:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.781+0000 7f0651efe640 1 -- 172.21.15.5:0/1435292079 shutdown_connections 2024-05-20T18:27:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.781+0000 7f0651efe640 1 --2- 172.21.15.5:0/1435292079 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f062c06bda0 0x7f062c06e2b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.781+0000 7f0651efe640 1 --2- 172.21.15.5:0/1435292079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f064c162820 0x7f064c0b4630 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.781+0000 7f0651efe640 1 --2- 172.21.15.5:0/1435292079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f064c0b3e80 0x7f064c157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.781+0000 7f0651efe640 1 --2- 172.21.15.5:0/1435292079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f064c06d910 0x7f064c160000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.781+0000 7f0651efe640 1 -- 172.21.15.5:0/1435292079 >> 172.21.15.5:0/1435292079 conn(0x7f064c0d4bd0 msgr2=0x7f064c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.781+0000 7f0651efe640 1 -- 172.21.15.5:0/1435292079 shutdown_connections 2024-05-20T18:27:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.781+0000 7f0651efe640 1 -- 172.21.15.5:0/1435292079 wait complete. 2024-05-20T18:27:04.792 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-20T18:27:04.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.909+0000 7f957c74b640 1 Processor -- start 2024-05-20T18:27:04.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.909+0000 7f957c74b640 1 -- start start 2024-05-20T18:27:04.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.910+0000 7f957c74b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f95740b6620 0x7f95740ba3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:04.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.910+0000 7f957c74b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9574162ce0 0x7f95740b8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:04.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.910+0000 7f957c74b640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95741634a0 con 0x7f95740b85b0 2024-05-20T18:27:04.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.910+0000 7f957c74b640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f95740d3480 con 0x7f9574162ce0 2024-05-20T18:27:04.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.910+0000 7f957c74b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9574163780 con 0x7f95740b6620 2024-05-20T18:27:04.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.910+0000 7f957a4e7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f95740b6620 0x7f95740ba3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:04.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.911+0000 7f957a4e7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f95740b6620 0x7f95740ba3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:58460/0 (socket says 172.21.15.5:58460) 2024-05-20T18:27:04.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.911+0000 7f957a4e7640 1 -- 172.21.15.5:0/3567544562 learned_addr learned my addr 172.21.15.5:0/3567544562 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:04.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.911+0000 7f9579ce6640 1 --2- 172.21.15.5:0/3567544562 >> v2:172.21.15.63:3300/0 conn(0x7f9574162ce0 0x7f95740b8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:04.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.911+0000 7f957a4e7640 1 -- 172.21.15.5:0/3567544562 >> v2:172.21.15.63:3300/0 conn(0x7f9574162ce0 msgr2=0x7f95740b8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:04.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.911+0000 7f957a4e7640 1 --2- 172.21.15.5:0/3567544562 >> v2:172.21.15.63:3300/0 conn(0x7f9574162ce0 0x7f95740b8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:04.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.911+0000 7f957a4e7640 1 -- 172.21.15.5:0/3567544562 >> v1:172.21.15.63:6789/0 conn(0x7f95740b85b0 legacy=0x7f95740d3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:04.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.911+0000 7f957a4e7640 1 -- 172.21.15.5:0/3567544562 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9574163a60 con 0x7f95740b6620 2024-05-20T18:27:04.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.911+0000 7f9579ce6640 1 --2- 172.21.15.5:0/3567544562 >> v2:172.21.15.63:3300/0 conn(0x7f9574162ce0 0x7f95740b8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:04.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.912+0000 7f957a4e7640 1 --2- 172.21.15.5:0/3567544562 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f95740b6620 0x7f95740ba3c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9570009e50 tx=0x7f957002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b1bc813ad6a7fa7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:04.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.912+0000 7f95794e5640 1 -- 172.21.15.5:0/3567544562 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9570036070 con 0x7f95740b6620 2024-05-20T18:27:04.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.912+0000 7f95794e5640 1 -- 172.21.15.5:0/3567544562 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9570031600 con 0x7f95740b6620 2024-05-20T18:27:04.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.912+0000 7f95794e5640 1 -- 172.21.15.5:0/3567544562 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95700319d0 con 0x7f95740b6620 2024-05-20T18:27:04.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.912+0000 7f957c74b640 1 -- 172.21.15.5:0/3567544562 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f95740b6620 msgr2=0x7f95740ba3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:04.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.912+0000 7f957c74b640 1 --2- 172.21.15.5:0/3567544562 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f95740b6620 0x7f95740ba3c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9570009e50 tx=0x7f957002a710 comp rx=0 tx=0).stop 2024-05-20T18:27:04.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.913+0000 7f957c74b640 1 -- 172.21.15.5:0/3567544562 shutdown_connections 2024-05-20T18:27:04.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.913+0000 7f957c74b640 1 --2- 172.21.15.5:0/3567544562 >> v2:172.21.15.63:3300/0 conn(0x7f9574162ce0 0x7f95740b8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:04.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.913+0000 7f957c74b640 1 --2- 172.21.15.5:0/3567544562 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f95740b6620 0x7f95740ba3c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:04.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.913+0000 7f957c74b640 1 -- 172.21.15.5:0/3567544562 >> 172.21.15.5:0/3567544562 conn(0x7f957406d240 msgr2=0x7f957406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:04.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.913+0000 7f957c74b640 1 -- 172.21.15.5:0/3567544562 shutdown_connections 2024-05-20T18:27:04.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.913+0000 7f957c74b640 1 -- 172.21.15.5:0/3567544562 wait complete. 2024-05-20T18:27:04.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.914+0000 7f957c74b640 1 Processor -- start 2024-05-20T18:27:04.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.914+0000 7f957c74b640 1 -- start start 2024-05-20T18:27:04.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.916+0000 7f957c74b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95740b6620 0x7f9574156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:04.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.916+0000 7f957c74b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95740b85b0 0x7f95740cd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:04.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.916+0000 7f957c74b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9574162ce0 0x7f95741ea4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:04.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.916+0000 7f957c74b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f95741630f0 con 0x7f95740b85b0 2024-05-20T18:27:04.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.916+0000 7f957c74b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f95740d4300 con 0x7f95740b6620 2024-05-20T18:27:04.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.916+0000 7f957c74b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f957415da30 con 0x7f9574162ce0 2024-05-20T18:27:04.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.916+0000 7f957a4e7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95740b6620 0x7f9574156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:04.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.916+0000 7f9579ce6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95740b85b0 0x7f95740cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:04.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.916+0000 7f957a4e7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95740b6620 0x7f9574156d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:56944/0 (socket says 172.21.15.5:56944) 2024-05-20T18:27:04.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.916+0000 7f957a4e7640 1 -- 172.21.15.5:0/4087713975 learned_addr learned my addr 172.21.15.5:0/4087713975 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:04.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.916+0000 7f957ace8640 1 --2- 172.21.15.5:0/4087713975 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9574162ce0 0x7f95741ea4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:04.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.916+0000 7f957ace8640 1 -- 172.21.15.5:0/4087713975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95740b6620 msgr2=0x7f9574156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:04.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.916+0000 7f957ace8640 1 --2- 172.21.15.5:0/4087713975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95740b6620 0x7f9574156d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:04.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.916+0000 7f957ace8640 1 -- 172.21.15.5:0/4087713975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95740b85b0 msgr2=0x7f95740cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:04.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.916+0000 7f957ace8640 1 --2- 172.21.15.5:0/4087713975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95740b85b0 0x7f95740cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:04.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.916+0000 7f957ace8640 1 -- 172.21.15.5:0/4087713975 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9574157a80 con 0x7f9574162ce0 2024-05-20T18:27:04.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.916+0000 7f957ace8640 1 --2- 172.21.15.5:0/4087713975 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9574162ce0 0x7f95741ea4e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9574068040 tx=0x7f956c00c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:04.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.917+0000 7f956affd640 1 -- 172.21.15.5:0/4087713975 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c017070 con 0x7f9574162ce0 2024-05-20T18:27:04.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.917+0000 7f956affd640 1 -- 172.21.15.5:0/4087713975 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f956c0052d0 con 0x7f9574162ce0 2024-05-20T18:27:04.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.917+0000 7f956affd640 1 -- 172.21.15.5:0/4087713975 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c003830 con 0x7f9574162ce0 2024-05-20T18:27:04.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.917+0000 7f957c74b640 1 -- 172.21.15.5:0/4087713975 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9574157f80 con 0x7f9574162ce0 2024-05-20T18:27:04.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.917+0000 7f957c74b640 1 -- 172.21.15.5:0/4087713975 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9574158540 con 0x7f9574162ce0 2024-05-20T18:27:04.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.918+0000 7f957c74b640 1 -- 172.21.15.5:0/4087713975 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9574075b00 con 0x7f9574162ce0 2024-05-20T18:27:04.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.918+0000 7f956affd640 1 -- 172.21.15.5:0/4087713975 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f956c01a070 con 0x7f9574162ce0 2024-05-20T18:27:04.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.919+0000 7f956affd640 1 --2- 172.21.15.5:0/4087713975 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f955806bda0 0x7f955806e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:04.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.919+0000 7f957a4e7640 1 --2- 172.21.15.5:0/4087713975 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f955806bda0 0x7f955806e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:04.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.919+0000 7f956affd640 1 -- 172.21.15.5:0/4087713975 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f956c089bb0 con 0x7f9574162ce0 2024-05-20T18:27:04.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.919+0000 7f957a4e7640 1 --2- 172.21.15.5:0/4087713975 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f955806bda0 0x7f955806e2b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f957002a750 tx=0x7f9570009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:04.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:04.924+0000 7f956affd640 1 -- 172.21.15.5:0/4087713975 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f956c058370 con 0x7f9574162ce0 2024-05-20T18:27:05.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.112+0000 7f957c74b640 1 -- 172.21.15.5:0/4087713975 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f95740d2fa0 con 0x7f9574162ce0 2024-05-20T18:27:05.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.112+0000 7f956affd640 1 -- 172.21.15.5:0/4087713975 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f956c05c670 con 0x7f9574162ce0 2024-05-20T18:27:05.111 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:05.112 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-05-20T18:27:05.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:27:05.112 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-05-20T18:27:05.112 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-05-20T18:27:05.112 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:27:05.112 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-05-20T18:27:05.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.114+0000 7f957c74b640 1 -- 172.21.15.5:0/4087713975 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f955806bda0 msgr2=0x7f955806e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:05.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.114+0000 7f957c74b640 1 --2- 172.21.15.5:0/4087713975 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f955806bda0 0x7f955806e2b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f957002a750 tx=0x7f9570009000 comp rx=0 tx=0).stop 2024-05-20T18:27:05.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.114+0000 7f957c74b640 1 -- 172.21.15.5:0/4087713975 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9574162ce0 msgr2=0x7f95741ea4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:05.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.114+0000 7f957c74b640 1 --2- 172.21.15.5:0/4087713975 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9574162ce0 0x7f95741ea4e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9574068040 tx=0x7f956c00c820 comp rx=0 tx=0).stop 2024-05-20T18:27:05.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.115+0000 7f957c74b640 1 -- 172.21.15.5:0/4087713975 shutdown_connections 2024-05-20T18:27:05.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.115+0000 7f957c74b640 1 --2- 172.21.15.5:0/4087713975 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f955806bda0 0x7f955806e2b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:05.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.115+0000 7f957c74b640 1 --2- 172.21.15.5:0/4087713975 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9574162ce0 0x7f95741ea4e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:05.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.115+0000 7f957c74b640 1 --2- 172.21.15.5:0/4087713975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95740b85b0 0x7f95740cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:05.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.115+0000 7f957c74b640 1 --2- 172.21.15.5:0/4087713975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95740b6620 0x7f9574156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:05.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.115+0000 7f957c74b640 1 -- 172.21.15.5:0/4087713975 >> 172.21.15.5:0/4087713975 conn(0x7f957406d240 msgr2=0x7f9574160b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:05.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.115+0000 7f957c74b640 1 -- 172.21.15.5:0/4087713975 shutdown_connections 2024-05-20T18:27:05.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.115+0000 7f957c74b640 1 -- 172.21.15.5:0/4087713975 wait complete. 2024-05-20T18:27:05.125 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-20T18:27:05.125 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-20T18:27:05.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.244+0000 7fbdec784640 1 Processor -- start 2024-05-20T18:27:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.244+0000 7fbdec784640 1 -- start start 2024-05-20T18:27:05.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.246+0000 7fbdec784640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbde40b3860 0x7fbde40d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.246+0000 7fbdec784640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbde408bd20 0x7fbde408c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.246+0000 7fbdec784640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbde408c670 con 0x7fbde406d870 2024-05-20T18:27:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.246+0000 7fbdec784640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fbde40d0450 con 0x7fbde40b3860 2024-05-20T18:27:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.246+0000 7fbdec784640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbde40d3b10 con 0x7fbde408bd20 2024-05-20T18:27:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.246+0000 7fbdea520640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbde40b3860 0x7fbde40d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.246+0000 7fbdea520640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbde40b3860 0x7fbde40d45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44550/0 (socket says 172.21.15.5:44550) 2024-05-20T18:27:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.246+0000 7fbdea520640 1 -- 172.21.15.5:0/3591166248 learned_addr learned my addr 172.21.15.5:0/3591166248 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.246+0000 7fbdead21640 1 --2- 172.21.15.5:0/3591166248 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbde408bd20 0x7fbde408c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.246+0000 7fbdea520640 1 -- 172.21.15.5:0/3591166248 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbde408bd20 msgr2=0x7fbde408c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.246+0000 7fbde951e640 1 -- 172.21.15.5:0/3591166248 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2450394671 0 0) 0x7fbdd8001110 con 0x7fbde406d870 2024-05-20T18:27:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.246+0000 7fbdea520640 1 --2- 172.21.15.5:0/3591166248 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbde408bd20 0x7fbde408c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.246+0000 7fbdea520640 1 -- 172.21.15.5:0/3591166248 >> v1:172.21.15.63:6789/0 conn(0x7fbde406d870 legacy=0x7fbde415f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.246+0000 7fbdea520640 1 -- 172.21.15.5:0/3591166248 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde40d3df0 con 0x7fbde40b3860 2024-05-20T18:27:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.247+0000 7fbdea520640 1 --2- 172.21.15.5:0/3591166248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde40b3860 0x7fbde40d45f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4009e50 tx=0x7fbdd402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee8b9ec4669e3920 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.247+0000 7fbde951e640 1 -- 172.21.15.5:0/3591166248 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4036070 con 0x7fbde40b3860 2024-05-20T18:27:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.247+0000 7fbde951e640 1 -- 172.21.15.5:0/3591166248 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdd4031600 con 0x7fbde40b3860 2024-05-20T18:27:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.247+0000 7fbde951e640 1 -- 172.21.15.5:0/3591166248 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd40319d0 con 0x7fbde40b3860 2024-05-20T18:27:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.248+0000 7fbdec784640 1 -- 172.21.15.5:0/3591166248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde40b3860 msgr2=0x7fbde40d45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.248+0000 7fbdec784640 1 --2- 172.21.15.5:0/3591166248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde40b3860 0x7fbde40d45f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4009e50 tx=0x7fbdd402a5f0 comp rx=0 tx=0).stop 2024-05-20T18:27:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.248+0000 7fbdec784640 1 -- 172.21.15.5:0/3591166248 shutdown_connections 2024-05-20T18:27:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.248+0000 7fbdec784640 1 --2- 172.21.15.5:0/3591166248 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbde408bd20 0x7fbde408c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.248+0000 7fbdec784640 1 --2- 172.21.15.5:0/3591166248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde40b3860 0x7fbde40d45f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.248+0000 7fbdec784640 1 -- 172.21.15.5:0/3591166248 >> 172.21.15.5:0/3591166248 conn(0x7fbde40b6d10 msgr2=0x7fbde406d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.248+0000 7fbdec784640 1 -- 172.21.15.5:0/3591166248 shutdown_connections 2024-05-20T18:27:05.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.248+0000 7fbdec784640 1 -- 172.21.15.5:0/3591166248 wait complete. 2024-05-20T18:27:05.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.250+0000 7fbdec784640 1 Processor -- start 2024-05-20T18:27:05.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.250+0000 7fbdec784640 1 -- start start 2024-05-20T18:27:05.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.251+0000 7fbdec784640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde406d870 0x7fbde40cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:05.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.251+0000 7fbdec784640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbde408bd20 0x7fbde41f5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:05.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.251+0000 7fbdec784640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbde40b3860 0x7fbde415a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:05.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.251+0000 7fbdec784640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbde40b81b0 con 0x7fbde406d870 2024-05-20T18:27:05.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.251+0000 7fbdec784640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbde40d0730 con 0x7fbde40b3860 2024-05-20T18:27:05.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.251+0000 7fbdec784640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbde40b8330 con 0x7fbde408bd20 2024-05-20T18:27:05.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.251+0000 7fbdea520640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde406d870 0x7fbde40cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:05.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.251+0000 7fbdea520640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde406d870 0x7fbde40cd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44564/0 (socket says 172.21.15.5:44564) 2024-05-20T18:27:05.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.251+0000 7fbdea520640 1 -- 172.21.15.5:0/937340666 learned_addr learned my addr 172.21.15.5:0/937340666 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:05.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.251+0000 7fbdead21640 1 --2- 172.21.15.5:0/937340666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbde40b3860 0x7fbde415a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:05.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.251+0000 7fbde9d1f640 1 --2- 172.21.15.5:0/937340666 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbde408bd20 0x7fbde41f5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:05.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.252+0000 7fbdea520640 1 -- 172.21.15.5:0/937340666 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbde408bd20 msgr2=0x7fbde41f5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:05.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.252+0000 7fbdea520640 1 --2- 172.21.15.5:0/937340666 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbde408bd20 0x7fbde41f5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:05.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.252+0000 7fbdea520640 1 -- 172.21.15.5:0/937340666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbde40b3860 msgr2=0x7fbde415a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:05.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.252+0000 7fbdea520640 1 --2- 172.21.15.5:0/937340666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbde40b3860 0x7fbde415a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:05.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.252+0000 7fbdea520640 1 -- 172.21.15.5:0/937340666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde415a780 con 0x7fbde406d870 2024-05-20T18:27:05.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.252+0000 7fbdea520640 1 --2- 172.21.15.5:0/937340666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde406d870 0x7fbde40cd220 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4034040 tx=0x7fbdd4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:05.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.252+0000 7fbde2ffd640 1 -- 172.21.15.5:0/937340666 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4036050 con 0x7fbde406d870 2024-05-20T18:27:05.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.252+0000 7fbde2ffd640 1 -- 172.21.15.5:0/937340666 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdd4037040 con 0x7fbde406d870 2024-05-20T18:27:05.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.252+0000 7fbdec784640 1 -- 172.21.15.5:0/937340666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde40cb4a0 con 0x7fbde406d870 2024-05-20T18:27:05.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.252+0000 7fbde2ffd640 1 -- 172.21.15.5:0/937340666 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4009510 con 0x7fbde406d870 2024-05-20T18:27:05.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.252+0000 7fbdec784640 1 -- 172.21.15.5:0/937340666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde40cf050 con 0x7fbde406d870 2024-05-20T18:27:05.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.253+0000 7fbdec784640 1 -- 172.21.15.5:0/937340666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbde40690b0 con 0x7fbde406d870 2024-05-20T18:27:05.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.254+0000 7fbde2ffd640 1 -- 172.21.15.5:0/937340666 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7fbdd4044020 con 0x7fbde406d870 2024-05-20T18:27:05.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.254+0000 7fbde2ffd640 1 --2- 172.21.15.5:0/937340666 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbdd006bda0 0x7fbdd006e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:05.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.254+0000 7fbde2ffd640 1 -- 172.21.15.5:0/937340666 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fbdd40aaf70 con 0x7fbde406d870 2024-05-20T18:27:05.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.254+0000 7fbde9d1f640 1 --2- 172.21.15.5:0/937340666 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbdd006bda0 0x7fbdd006e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:05.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.255+0000 7fbde9d1f640 1 --2- 172.21.15.5:0/937340666 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbdd006bda0 0x7fbdd006e2b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbde41f8ae0 tx=0x7fbdd8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:05.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.259+0000 7fbde2ffd640 1 -- 172.21.15.5:0/937340666 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbdd404b020 con 0x7fbde406d870 2024-05-20T18:27:05.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.444+0000 7fbdec784640 1 -- 172.21.15.5:0/937340666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fbde40d3670 con 0x7fbde406d870 2024-05-20T18:27:05.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.498+0000 7fbde2ffd640 1 -- 172.21.15.5:0/937340666 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbdd4005d10 con 0x7fbde406d870 2024-05-20T18:27:05.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.764+0000 7fbde2ffd640 1 -- 172.21.15.5:0/937340666 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7fbdd4092b00 con 0x7fbde406d870 2024-05-20T18:27:05.764 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-05-20T18:27:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.767+0000 7fbdec784640 1 -- 172.21.15.5:0/937340666 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbdd006bda0 msgr2=0x7fbdd006e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.767+0000 7fbdec784640 1 --2- 172.21.15.5:0/937340666 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbdd006bda0 0x7fbdd006e2b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbde41f8ae0 tx=0x7fbdd8009040 comp rx=0 tx=0).stop 2024-05-20T18:27:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.767+0000 7fbdec784640 1 -- 172.21.15.5:0/937340666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde406d870 msgr2=0x7fbde40cd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.767+0000 7fbdec784640 1 --2- 172.21.15.5:0/937340666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde406d870 0x7fbde40cd220 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4034040 tx=0x7fbdd4004a40 comp rx=0 tx=0).stop 2024-05-20T18:27:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.767+0000 7fbdec784640 1 -- 172.21.15.5:0/937340666 shutdown_connections 2024-05-20T18:27:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.767+0000 7fbdec784640 1 --2- 172.21.15.5:0/937340666 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbdd006bda0 0x7fbdd006e2b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.767+0000 7fbdec784640 1 --2- 172.21.15.5:0/937340666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbde40b3860 0x7fbde415a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:05.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.767+0000 7fbdec784640 1 --2- 172.21.15.5:0/937340666 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbde408bd20 0x7fbde41f5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:05.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.767+0000 7fbdec784640 1 --2- 172.21.15.5:0/937340666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde406d870 0x7fbde40cd220 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:05.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.767+0000 7fbdec784640 1 -- 172.21.15.5:0/937340666 >> 172.21.15.5:0/937340666 conn(0x7fbde40b6d10 msgr2=0x7fbde415d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:05.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.767+0000 7fbdec784640 1 -- 172.21.15.5:0/937340666 shutdown_connections 2024-05-20T18:27:05.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.767+0000 7fbdec784640 1 -- 172.21.15.5:0/937340666 wait complete. 2024-05-20T18:27:05.783 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-20T18:27:05.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.906+0000 7fbc2f9b5640 1 Processor -- start 2024-05-20T18:27:05.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.907+0000 7fbc2f9b5640 1 -- start start 2024-05-20T18:27:05.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.908+0000 7fbc2f9b5640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbc2808c4c0 0x7fbc280b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:05.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.908+0000 7fbc2f9b5640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc280d2890 con 0x7fbc280d46e0 2024-05-20T18:27:05.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.908+0000 7fbc2f9b5640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc280d2b70 con 0x7fbc2815f320 2024-05-20T18:27:05.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.908+0000 7fbc2f9b5640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fbc280d2e50 con 0x7fbc2808c4c0 2024-05-20T18:27:05.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.908+0000 7fbc2d751640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbc280d46e0 0x7fbc2808bdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44298/0 (socket says 172.21.15.5:44298) 2024-05-20T18:27:05.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.908+0000 7fbc2d751640 1 -- 172.21.15.5:0/943395989 learned_addr learned my addr 172.21.15.5:0/943395989 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:05.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.908+0000 7fbc2cf50640 1 --2- 172.21.15.5:0/943395989 >> v2:172.21.15.63:3300/0 conn(0x7fbc2808c4c0 0x7fbc280b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:05.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.908+0000 7fbc1ffff640 1 -- 172.21.15.5:0/943395989 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3133852232 0 0) 0x7fbc280d2890 con 0x7fbc280d46e0 2024-05-20T18:27:05.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.908+0000 7fbc1ffff640 1 -- 172.21.15.5:0/943395989 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc14003520 con 0x7fbc280d46e0 2024-05-20T18:27:05.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.909+0000 7fbc1ffff640 1 -- 172.21.15.5:0/943395989 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3044801501 0 0) 0x7fbc14003520 con 0x7fbc280d46e0 2024-05-20T18:27:05.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.909+0000 7fbc1ffff640 1 -- 172.21.15.5:0/943395989 >> v2:172.21.15.63:3300/0 conn(0x7fbc2808c4c0 msgr2=0x7fbc280b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:05.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.909+0000 7fbc1ffff640 1 --2- 172.21.15.5:0/943395989 >> v2:172.21.15.63:3300/0 conn(0x7fbc2808c4c0 0x7fbc280b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:05.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.909+0000 7fbc1ffff640 1 -- 172.21.15.5:0/943395989 >> v1:172.21.15.63:6789/0 conn(0x7fbc2815f320 legacy=0x7fbc280b8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:05.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.909+0000 7fbc1ffff640 1 -- 172.21.15.5:0/943395989 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc280d3130 con 0x7fbc280d46e0 2024-05-20T18:27:05.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.909+0000 7fbc1ffff640 1 -- 172.21.15.5:0/943395989 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7fbc24001670 con 0x7fbc280d46e0 2024-05-20T18:27:05.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.909+0000 7fbc1ffff640 1 -- 172.21.15.5:0/943395989 >> v1:172.21.15.5:6789/0 conn(0x7fbc280d46e0 legacy=0x7fbc2808bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:05.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.909+0000 7fbc1ffff640 1 --2- 172.21.15.5:0/943395989 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbc140039b0 0x7fbc14003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:05.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.909+0000 7fbc1ffff640 1 --2- 172.21.15.5:0/943395989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc14004800 0x7fbc14004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:05.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.909+0000 7fbc1ffff640 1 --2- 172.21.15.5:0/943395989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc140051c0 0x7fbc1400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:05.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.909+0000 7fbc1ffff640 1 -- 172.21.15.5:0/943395989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbc280d2e50 con 0x7fbc140051c0 2024-05-20T18:27:05.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.909+0000 7fbc1ffff640 1 -- 172.21.15.5:0/943395989 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1400c160 con 0x7fbc14004800 2024-05-20T18:27:05.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.909+0000 7fbc1ffff640 1 -- 172.21.15.5:0/943395989 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbc1400c360 con 0x7fbc140039b0 2024-05-20T18:27:05.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.909+0000 7fbc2df52640 1 --2- 172.21.15.5:0/943395989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc140051c0 0x7fbc1400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:05.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.909+0000 7fbc2cf50640 1 --2- 172.21.15.5:0/943395989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc14004800 0x7fbc14004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:05.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.909+0000 7fbc2d751640 1 --2- 172.21.15.5:0/943395989 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbc140039b0 0x7fbc14003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:05.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.909+0000 7fbc2df52640 1 -- 172.21.15.5:0/943395989 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbc140039b0 msgr2=0x7fbc14003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:05.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.909+0000 7fbc2df52640 1 --2- 172.21.15.5:0/943395989 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbc140039b0 0x7fbc14003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:05.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.909+0000 7fbc2df52640 1 -- 172.21.15.5:0/943395989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc14004800 msgr2=0x7fbc14004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:05.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.909+0000 7fbc2df52640 1 --2- 172.21.15.5:0/943395989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc14004800 0x7fbc14004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:05.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.909+0000 7fbc2df52640 1 -- 172.21.15.5:0/943395989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc14003520 con 0x7fbc140051c0 2024-05-20T18:27:05.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.910+0000 7fbc2df52640 1 -- 172.21.15.5:0/943395989 reap_dead start 2024-05-20T18:27:05.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.910+0000 7fbc2df52640 1 --2- 172.21.15.5:0/943395989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc140051c0 0x7fbc1400b9c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbc20001e50 tx=0x7fbc14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba2826d25ba2757 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:05.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.910+0000 7fbc1ffff640 1 -- 172.21.15.5:0/943395989 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc20001a70 con 0x7fbc140051c0 2024-05-20T18:27:05.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.910+0000 7fbc1ffff640 1 -- 172.21.15.5:0/943395989 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc2002adc0 con 0x7fbc140051c0 2024-05-20T18:27:05.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.910+0000 7fbc1ffff640 1 -- 172.21.15.5:0/943395989 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc20003bb0 con 0x7fbc140051c0 2024-05-20T18:27:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.911+0000 7fbc2f9b5640 1 -- 172.21.15.5:0/943395989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc140051c0 msgr2=0x7fbc1400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.911+0000 7fbc2f9b5640 1 --2- 172.21.15.5:0/943395989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc140051c0 0x7fbc1400b9c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbc20001e50 tx=0x7fbc14005830 comp rx=0 tx=0).stop 2024-05-20T18:27:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.911+0000 7fbc2f9b5640 1 -- 172.21.15.5:0/943395989 shutdown_connections 2024-05-20T18:27:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.911+0000 7fbc2f9b5640 1 --2- 172.21.15.5:0/943395989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc140051c0 0x7fbc1400b9c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:05.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.911+0000 7fbc2f9b5640 1 -- 172.21.15.5:0/943395989 >> 172.21.15.5:0/943395989 conn(0x7fbc280d1cd0 msgr2=0x7fbc280d4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:05.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.911+0000 7fbc2f9b5640 1 -- 172.21.15.5:0/943395989 shutdown_connections 2024-05-20T18:27:05.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.911+0000 7fbc2f9b5640 1 -- 172.21.15.5:0/943395989 wait complete. 2024-05-20T18:27:05.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.913+0000 7fbc2f9b5640 1 Processor -- start 2024-05-20T18:27:05.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.913+0000 7fbc2f9b5640 1 -- start start 2024-05-20T18:27:05.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.914+0000 7fbc2f9b5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc140051c0 0x7fbc28149350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:05.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.914+0000 7fbc2f9b5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc281f5100 0x7fbc28159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:05.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.914+0000 7fbc2f9b5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbc28147200 0x7fbc281598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:05.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.914+0000 7fbc2f9b5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2815f730 con 0x7fbc140051c0 2024-05-20T18:27:05.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.914+0000 7fbc2f9b5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2808ddb0 con 0x7fbc281f5100 2024-05-20T18:27:05.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.914+0000 7fbc2f9b5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbc280d3930 con 0x7fbc28147200 2024-05-20T18:27:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.914+0000 7fbc2d751640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc140051c0 0x7fbc28149350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.914+0000 7fbc2d751640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc140051c0 0x7fbc28149350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44574/0 (socket says 172.21.15.5:44574) 2024-05-20T18:27:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.914+0000 7fbc2d751640 1 -- 172.21.15.5:0/3537705543 learned_addr learned my addr 172.21.15.5:0/3537705543 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.914+0000 7fbc2d751640 1 -- 172.21.15.5:0/3537705543 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbc28147200 msgr2=0x7fbc281598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.914+0000 7fbc2d751640 1 --2- 172.21.15.5:0/3537705543 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbc28147200 0x7fbc281598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.914+0000 7fbc2d751640 1 -- 172.21.15.5:0/3537705543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc281f5100 msgr2=0x7fbc28159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.914+0000 7fbc2d751640 1 --2- 172.21.15.5:0/3537705543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc281f5100 0x7fbc28159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.914+0000 7fbc2d751640 1 -- 172.21.15.5:0/3537705543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc28147950 con 0x7fbc140051c0 2024-05-20T18:27:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.915+0000 7fbc2d751640 1 --2- 172.21.15.5:0/3537705543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc140051c0 0x7fbc28149350 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbc24004f40 tx=0x7fbc240074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.915+0000 7fbc1dffb640 1 -- 172.21.15.5:0/3537705543 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2400f7b0 con 0x7fbc140051c0 2024-05-20T18:27:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.915+0000 7fbc1dffb640 1 -- 172.21.15.5:0/3537705543 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc24018070 con 0x7fbc140051c0 2024-05-20T18:27:05.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.915+0000 7fbc2f9b5640 1 -- 172.21.15.5:0/3537705543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc28147c30 con 0x7fbc140051c0 2024-05-20T18:27:05.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.915+0000 7fbc1dffb640 1 -- 172.21.15.5:0/3537705543 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2400fbd0 con 0x7fbc140051c0 2024-05-20T18:27:05.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.915+0000 7fbc2f9b5640 1 -- 172.21.15.5:0/3537705543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc281481f0 con 0x7fbc140051c0 2024-05-20T18:27:05.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.916+0000 7fbc2f9b5640 1 -- 172.21.15.5:0/3537705543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc280690b0 con 0x7fbc140051c0 2024-05-20T18:27:05.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.917+0000 7fbc1dffb640 1 -- 172.21.15.5:0/3537705543 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbc2401d460 con 0x7fbc140051c0 2024-05-20T18:27:05.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.917+0000 7fbc1dffb640 1 --2- 172.21.15.5:0/3537705543 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbc0c06be60 0x7fbc0c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:05.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.917+0000 7fbc1dffb640 1 -- 172.21.15.5:0/3537705543 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7fbc2408ac40 con 0x7fbc140051c0 2024-05-20T18:27:05.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.917+0000 7fbc2cf50640 1 --2- 172.21.15.5:0/3537705543 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbc0c06be60 0x7fbc0c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:05.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.918+0000 7fbc2cf50640 1 --2- 172.21.15.5:0/3537705543 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbc0c06be60 0x7fbc0c06e370 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbc2815a140 tx=0x7fbc18009320 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:05.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:05.922+0000 7fbc1dffb640 1 -- 172.21.15.5:0/3537705543 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbc240597d0 con 0x7fbc140051c0 2024-05-20T18:27:06.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.106+0000 7fbc2f9b5640 1 -- 172.21.15.5:0/3537705543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fbc280b74d0 con 0x7fbc140051c0 2024-05-20T18:27:06.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.770+0000 7fbc1dffb640 1 -- 172.21.15.5:0/3537705543 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) ==== 171+0+0 (secure 0 0 0) 0x7fbc240590d0 con 0x7fbc140051c0 2024-05-20T18:27:06.769 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-05-20T18:27:06.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.773+0000 7fbc2f9b5640 1 -- 172.21.15.5:0/3537705543 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbc0c06be60 msgr2=0x7fbc0c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:06.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.773+0000 7fbc2f9b5640 1 --2- 172.21.15.5:0/3537705543 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbc0c06be60 0x7fbc0c06e370 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbc2815a140 tx=0x7fbc18009320 comp rx=0 tx=0).stop 2024-05-20T18:27:06.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.774+0000 7fbc2f9b5640 1 -- 172.21.15.5:0/3537705543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc140051c0 msgr2=0x7fbc28149350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:06.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.774+0000 7fbc2f9b5640 1 --2- 172.21.15.5:0/3537705543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc140051c0 0x7fbc28149350 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbc24004f40 tx=0x7fbc240074f0 comp rx=0 tx=0).stop 2024-05-20T18:27:06.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.774+0000 7fbc2f9b5640 1 -- 172.21.15.5:0/3537705543 shutdown_connections 2024-05-20T18:27:06.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.774+0000 7fbc2f9b5640 1 --2- 172.21.15.5:0/3537705543 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbc0c06be60 0x7fbc0c06e370 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:06.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.774+0000 7fbc2f9b5640 1 --2- 172.21.15.5:0/3537705543 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbc28147200 0x7fbc281598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:06.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.774+0000 7fbc2f9b5640 1 --2- 172.21.15.5:0/3537705543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc281f5100 0x7fbc28159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:06.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.774+0000 7fbc2f9b5640 1 --2- 172.21.15.5:0/3537705543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc140051c0 0x7fbc28149350 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:06.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.774+0000 7fbc2f9b5640 1 -- 172.21.15.5:0/3537705543 >> 172.21.15.5:0/3537705543 conn(0x7fbc280d1cd0 msgr2=0x7fbc280b86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:06.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.774+0000 7fbc2f9b5640 1 -- 172.21.15.5:0/3537705543 shutdown_connections 2024-05-20T18:27:06.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.774+0000 7fbc2f9b5640 1 -- 172.21.15.5:0/3537705543 wait complete. 2024-05-20T18:27:06.800 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-20T18:27:06.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.924+0000 7f82fd321640 1 Processor -- start 2024-05-20T18:27:06.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.924+0000 7f82fd321640 1 -- start start 2024-05-20T18:27:06.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.926+0000 7f82fd321640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82f8162820 0x7f82f80b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:06.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.926+0000 7f82fd321640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f82f80b3e80 0x7f82f806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:06.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.926+0000 7f82fd321640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82f806dd20 con 0x7f82f806d910 2024-05-20T18:27:06.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.926+0000 7f82fd321640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f82f80d0630 con 0x7f82f80b3e80 2024-05-20T18:27:06.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.926+0000 7f82fd321640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f82f80d39c0 con 0x7f82f8162820 2024-05-20T18:27:06.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.926+0000 7f82f67fc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f82f80b3e80 0x7f82f806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:06.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.926+0000 7f82f67fc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f82f80b3e80 0x7f82f806d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44590/0 (socket says 172.21.15.5:44590) 2024-05-20T18:27:06.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.926+0000 7f82f67fc640 1 -- 172.21.15.5:0/210453285 learned_addr learned my addr 172.21.15.5:0/210453285 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:06.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.926+0000 7f82f6ffd640 1 --2- 172.21.15.5:0/210453285 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82f8162820 0x7f82f80b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:06.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.926+0000 7f82f67fc640 1 -- 172.21.15.5:0/210453285 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82f8162820 msgr2=0x7f82f80b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:06.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.926+0000 7f82f67fc640 1 --2- 172.21.15.5:0/210453285 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82f8162820 0x7f82f80b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:06.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.926+0000 7f82f67fc640 1 -- 172.21.15.5:0/210453285 >> v1:172.21.15.63:6789/0 conn(0x7f82f806d910 legacy=0x7f82f80d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:06.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.926+0000 7f82f67fc640 1 -- 172.21.15.5:0/210453285 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f80d3ca0 con 0x7f82f80b3e80 2024-05-20T18:27:06.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.926+0000 7f82f6ffd640 1 --2- 172.21.15.5:0/210453285 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82f8162820 0x7f82f80b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:06.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.927+0000 7f82f67fc640 1 --2- 172.21.15.5:0/210453285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f80b3e80 0x7f82f806d3d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009870 tx=0x7f82ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44bb4ab5b55fc610 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:06.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.927+0000 7f82f5ffb640 1 -- 172.21.15.5:0/210453285 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec037070 con 0x7f82f80b3e80 2024-05-20T18:27:06.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.927+0000 7f82f5ffb640 1 -- 172.21.15.5:0/210453285 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82ec032440 con 0x7f82f80b3e80 2024-05-20T18:27:06.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.927+0000 7f82f5ffb640 1 -- 172.21.15.5:0/210453285 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec032810 con 0x7f82f80b3e80 2024-05-20T18:27:06.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.928+0000 7f82fd321640 1 -- 172.21.15.5:0/210453285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f80b3e80 msgr2=0x7f82f806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:06.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.928+0000 7f82fd321640 1 --2- 172.21.15.5:0/210453285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f80b3e80 0x7f82f806d3d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009870 tx=0x7f82ec02a270 comp rx=0 tx=0).stop 2024-05-20T18:27:06.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.928+0000 7f82fd321640 1 -- 172.21.15.5:0/210453285 shutdown_connections 2024-05-20T18:27:06.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.928+0000 7f82fd321640 1 --2- 172.21.15.5:0/210453285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f80b3e80 0x7f82f806d3d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:06.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.928+0000 7f82fd321640 1 --2- 172.21.15.5:0/210453285 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82f8162820 0x7f82f80b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:06.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.928+0000 7f82fd321640 1 -- 172.21.15.5:0/210453285 >> 172.21.15.5:0/210453285 conn(0x7f82f80d4bd0 msgr2=0x7f82f80d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:06.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.928+0000 7f82fd321640 1 -- 172.21.15.5:0/210453285 shutdown_connections 2024-05-20T18:27:06.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.928+0000 7f82fd321640 1 -- 172.21.15.5:0/210453285 wait complete. 2024-05-20T18:27:06.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.930+0000 7f82fd321640 1 Processor -- start 2024-05-20T18:27:06.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.930+0000 7f82fd321640 1 -- start start 2024-05-20T18:27:06.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.931+0000 7f82fd321640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82f806d910 0x7f82f80ce500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:06.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.931+0000 7f82fd321640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f80b3e80 0x7f82f8148af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:06.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.931+0000 7f82fd321640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82f8162820 0x7f82f8149030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:06.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.931+0000 7f82fd321640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f82f80d0300 con 0x7f82f80b3e80 2024-05-20T18:27:06.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.931+0000 7f82fd321640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f82f80d4260 con 0x7f82f8162820 2024-05-20T18:27:06.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.931+0000 7f82fd321640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f82f80b91a0 con 0x7f82f806d910 2024-05-20T18:27:06.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.931+0000 7f82f67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f80b3e80 0x7f82f8148af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:06.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.931+0000 7f82f67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f80b3e80 0x7f82f8148af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44602/0 (socket says 172.21.15.5:44602) 2024-05-20T18:27:06.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.931+0000 7f82f67fc640 1 -- 172.21.15.5:0/1376437467 learned_addr learned my addr 172.21.15.5:0/1376437467 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:06.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.931+0000 7f82f6ffd640 1 --2- 172.21.15.5:0/1376437467 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82f806d910 0x7f82f80ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:06.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.931+0000 7f82f77fe640 1 --2- 172.21.15.5:0/1376437467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82f8162820 0x7f82f8149030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:06.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.931+0000 7f82f67fc640 1 -- 172.21.15.5:0/1376437467 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82f806d910 msgr2=0x7f82f80ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:06.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.931+0000 7f82f67fc640 1 --2- 172.21.15.5:0/1376437467 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82f806d910 0x7f82f80ce500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:06.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.931+0000 7f82f67fc640 1 -- 172.21.15.5:0/1376437467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82f8162820 msgr2=0x7f82f8149030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:06.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.931+0000 7f82f67fc640 1 --2- 172.21.15.5:0/1376437467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82f8162820 0x7f82f8149030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:06.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.931+0000 7f82f67fc640 1 -- 172.21.15.5:0/1376437467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f80cea40 con 0x7f82f80b3e80 2024-05-20T18:27:06.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.932+0000 7f82f67fc640 1 --2- 172.21.15.5:0/1376437467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f80b3e80 0x7f82f8148af0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f82ec000c00 tx=0x7f82ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:06.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.932+0000 7f82df7fe640 1 -- 172.21.15.5:0/1376437467 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec041020 con 0x7f82f80b3e80 2024-05-20T18:27:06.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.932+0000 7f82df7fe640 1 -- 172.21.15.5:0/1376437467 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82ec0306c0 con 0x7f82f80b3e80 2024-05-20T18:27:06.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.932+0000 7f82df7fe640 1 -- 172.21.15.5:0/1376437467 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec037040 con 0x7f82f80b3e80 2024-05-20T18:27:06.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.932+0000 7f82fd321640 1 -- 172.21.15.5:0/1376437467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f8146b00 con 0x7f82f80b3e80 2024-05-20T18:27:06.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.932+0000 7f82fd321640 1 -- 172.21.15.5:0/1376437467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f81470c0 con 0x7f82f80b3e80 2024-05-20T18:27:06.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.933+0000 7f82fd321640 1 -- 172.21.15.5:0/1376437467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f8075b00 con 0x7f82f80b3e80 2024-05-20T18:27:06.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.934+0000 7f82df7fe640 1 -- 172.21.15.5:0/1376437467 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f82ec044020 con 0x7f82f80b3e80 2024-05-20T18:27:06.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.934+0000 7f82df7fe640 1 --2- 172.21.15.5:0/1376437467 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f82e406be60 0x7f82e406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:06.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.934+0000 7f82df7fe640 1 -- 172.21.15.5:0/1376437467 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f82ec0ab1b0 con 0x7f82f80b3e80 2024-05-20T18:27:06.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.934+0000 7f82f6ffd640 1 --2- 172.21.15.5:0/1376437467 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f82e406be60 0x7f82e406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:06.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.935+0000 7f82f6ffd640 1 --2- 172.21.15.5:0/1376437467 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f82e406be60 0x7f82e406e370 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009fd0 tx=0x7f82e80097f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:06.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:06.939+0000 7f82df7fe640 1 -- 172.21.15.5:0/1376437467 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f82ec04b070 con 0x7f82f80b3e80 2024-05-20T18:27:07.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.162+0000 7f82fd321640 1 -- 172.21.15.5:0/1376437467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f82f806c530 con 0x7f82f80b3e80 2024-05-20T18:27:07.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.772+0000 7f82df7fe640 1 -- 172.21.15.5:0/1376437467 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f82f80cea40 con 0x7f82f80b3e80 2024-05-20T18:27:07.771 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 2 and data pool 3 2024-05-20T18:27:07.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.777+0000 7f82fd321640 1 -- 172.21.15.5:0/1376437467 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f82e406be60 msgr2=0x7f82e406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:07.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.777+0000 7f82fd321640 1 --2- 172.21.15.5:0/1376437467 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f82e406be60 0x7f82e406e370 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009fd0 tx=0x7f82e80097f0 comp rx=0 tx=0).stop 2024-05-20T18:27:07.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.777+0000 7f82fd321640 1 -- 172.21.15.5:0/1376437467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f80b3e80 msgr2=0x7f82f8148af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:07.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.777+0000 7f82fd321640 1 --2- 172.21.15.5:0/1376437467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f80b3e80 0x7f82f8148af0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f82ec000c00 tx=0x7f82ec0046c0 comp rx=0 tx=0).stop 2024-05-20T18:27:07.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.778+0000 7f82fd321640 1 -- 172.21.15.5:0/1376437467 shutdown_connections 2024-05-20T18:27:07.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.778+0000 7f82fd321640 1 --2- 172.21.15.5:0/1376437467 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f82e406be60 0x7f82e406e370 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:07.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.778+0000 7f82fd321640 1 --2- 172.21.15.5:0/1376437467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82f8162820 0x7f82f8149030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:07.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.778+0000 7f82fd321640 1 --2- 172.21.15.5:0/1376437467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f80b3e80 0x7f82f8148af0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:07.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.778+0000 7f82fd321640 1 --2- 172.21.15.5:0/1376437467 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82f806d910 0x7f82f80ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:07.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.778+0000 7f82fd321640 1 -- 172.21.15.5:0/1376437467 >> 172.21.15.5:0/1376437467 conn(0x7f82f80d4bd0 msgr2=0x7f82f8160eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:07.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.780+0000 7f82fd321640 1 -- 172.21.15.5:0/1376437467 shutdown_connections 2024-05-20T18:27:07.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.780+0000 7f82fd321640 1 -- 172.21.15.5:0/1376437467 wait complete. 2024-05-20T18:27:07.812 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-20T18:27:07.812 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-20T18:27:07.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.937+0000 7fa2fa614640 1 Processor -- start 2024-05-20T18:27:07.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.937+0000 7fa2fa614640 1 -- start start 2024-05-20T18:27:07.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.939+0000 7fa2fa614640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2f4163950 0x7fa2f40b66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:07.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.939+0000 7fa2fa614640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2f40b6ff0 con 0x7fa2f40d3c60 2024-05-20T18:27:07.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.939+0000 7fa2fa614640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2f4160410 con 0x7fa2f40b6be0 2024-05-20T18:27:07.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.939+0000 7fa2fa614640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa2f41606f0 con 0x7fa2f4163950 2024-05-20T18:27:07.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.939+0000 7fa2f8bb1640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa2f40d3c60 0x7fa2f415f160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44306/0 (socket says 172.21.15.5:44306) 2024-05-20T18:27:07.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.939+0000 7fa2f8bb1640 1 -- 172.21.15.5:0/1305259071 learned_addr learned my addr 172.21.15.5:0/1305259071 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:07.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.939+0000 7fa2f3fff640 1 --2- 172.21.15.5:0/1305259071 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2f4163950 0x7fa2f40b66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:07.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.939+0000 7fa2f37fe640 1 -- 172.21.15.5:0/1305259071 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1658242342 0 0) 0x7fa2ec001490 con 0x7fa2f40d3c60 2024-05-20T18:27:07.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.939+0000 7fa2f37fe640 1 -- 172.21.15.5:0/1305259071 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2e0003520 con 0x7fa2f40d3c60 2024-05-20T18:27:07.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.939+0000 7fa2f37fe640 1 -- 172.21.15.5:0/1305259071 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1621146286 0 0) 0x7fa2f4160410 con 0x7fa2f40b6be0 2024-05-20T18:27:07.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.939+0000 7fa2f37fe640 1 -- 172.21.15.5:0/1305259071 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2ec001490 con 0x7fa2f40b6be0 2024-05-20T18:27:07.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.939+0000 7fa2f37fe640 1 -- 172.21.15.5:0/1305259071 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4162766463 0 0) 0x7fa2e0003520 con 0x7fa2f40d3c60 2024-05-20T18:27:07.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.940+0000 7fa2f3fff640 1 -- 172.21.15.5:0/1305259071 >> v1:172.21.15.63:6789/0 conn(0x7fa2f40b6be0 legacy=0x7fa2f40d3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:07.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.940+0000 7fa2f3fff640 1 -- 172.21.15.5:0/1305259071 >> v1:172.21.15.5:6789/0 conn(0x7fa2f40d3c60 legacy=0x7fa2f415f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:07.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.940+0000 7fa2f3fff640 1 -- 172.21.15.5:0/1305259071 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f4162e70 con 0x7fa2f4163950 2024-05-20T18:27:07.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.940+0000 7fa2f3fff640 1 --2- 172.21.15.5:0/1305259071 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2f4163950 0x7fa2f40b66a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40097d0 tx=0x7fa2e402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=607d33daf5bb77e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:07.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.940+0000 7fa2f37fe640 1 -- 172.21.15.5:0/1305259071 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4037070 con 0x7fa2f4163950 2024-05-20T18:27:07.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.941+0000 7fa2f37fe640 1 -- 172.21.15.5:0/1305259071 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2e402ac30 con 0x7fa2f4163950 2024-05-20T18:27:07.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.941+0000 7fa2f37fe640 1 -- 172.21.15.5:0/1305259071 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec001bc0 con 0x7fa2f4163950 2024-05-20T18:27:07.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.941+0000 7fa2fa614640 1 -- 172.21.15.5:0/1305259071 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2f4163950 msgr2=0x7fa2f40b66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:07.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.941+0000 7fa2fa614640 1 --2- 172.21.15.5:0/1305259071 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2f4163950 0x7fa2f40b66a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40097d0 tx=0x7fa2e402a1a0 comp rx=0 tx=0).stop 2024-05-20T18:27:07.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.941+0000 7fa2fa614640 1 -- 172.21.15.5:0/1305259071 shutdown_connections 2024-05-20T18:27:07.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.941+0000 7fa2fa614640 1 --2- 172.21.15.5:0/1305259071 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2f4163950 0x7fa2f40b66a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:07.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.941+0000 7fa2fa614640 1 -- 172.21.15.5:0/1305259071 >> 172.21.15.5:0/1305259071 conn(0x7fa2f40d16a0 msgr2=0x7fa2f40d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:07.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.941+0000 7fa2fa614640 1 -- 172.21.15.5:0/1305259071 shutdown_connections 2024-05-20T18:27:07.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.941+0000 7fa2fa614640 1 -- 172.21.15.5:0/1305259071 wait complete. 2024-05-20T18:27:07.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.943+0000 7fa2fa614640 1 Processor -- start 2024-05-20T18:27:07.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.943+0000 7fa2fa614640 1 -- start start 2024-05-20T18:27:07.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.944+0000 7fa2fa614640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2f40b6be0 0x7fa2f41e8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:07.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.945+0000 7fa2fa614640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2f40d3c60 0x7fa2f414d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:07.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.945+0000 7fa2fa614640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2f4163950 0x7fa2f40cb100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:07.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.945+0000 7fa2fa614640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f40d4070 con 0x7fa2f40b6be0 2024-05-20T18:27:07.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.945+0000 7fa2fa614640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f4163430 con 0x7fa2f40d3c60 2024-05-20T18:27:07.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.945+0000 7fa2fa614640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa2f40d41f0 con 0x7fa2f4163950 2024-05-20T18:27:07.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.945+0000 7fa2f3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2f40b6be0 0x7fa2f41e8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:07.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.945+0000 7fa2f3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2f40b6be0 0x7fa2f41e8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44606/0 (socket says 172.21.15.5:44606) 2024-05-20T18:27:07.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.945+0000 7fa2f3fff640 1 -- 172.21.15.5:0/3391491318 learned_addr learned my addr 172.21.15.5:0/3391491318 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:07.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.945+0000 7fa2ebfff640 1 --2- 172.21.15.5:0/3391491318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2f40d3c60 0x7fa2f414d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:07.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.945+0000 7fa2f8bb1640 1 --2- 172.21.15.5:0/3391491318 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2f4163950 0x7fa2f40cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:07.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.945+0000 7fa2f3fff640 1 -- 172.21.15.5:0/3391491318 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2f4163950 msgr2=0x7fa2f40cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:07.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.945+0000 7fa2f3fff640 1 --2- 172.21.15.5:0/3391491318 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2f4163950 0x7fa2f40cb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:07.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.945+0000 7fa2f3fff640 1 -- 172.21.15.5:0/3391491318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2f40d3c60 msgr2=0x7fa2f414d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:07.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.945+0000 7fa2f3fff640 1 --2- 172.21.15.5:0/3391491318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2f40d3c60 0x7fa2f414d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:07.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.945+0000 7fa2f3fff640 1 -- 172.21.15.5:0/3391491318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f41e9a50 con 0x7fa2f40b6be0 2024-05-20T18:27:07.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.945+0000 7fa2f3fff640 1 --2- 172.21.15.5:0/3391491318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2f40b6be0 0x7fa2f41e8f50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4009720 tx=0x7fa2e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:07.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.945+0000 7fa2f17fa640 1 -- 172.21.15.5:0/3391491318 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4041070 con 0x7fa2f40b6be0 2024-05-20T18:27:07.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.945+0000 7fa2f17fa640 1 -- 172.21.15.5:0/3391491318 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2e403c660 con 0x7fa2f40b6be0 2024-05-20T18:27:07.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.945+0000 7fa2f17fa640 1 -- 172.21.15.5:0/3391491318 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4037040 con 0x7fa2f40b6be0 2024-05-20T18:27:07.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.945+0000 7fa2fa614640 1 -- 172.21.15.5:0/3391491318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2f41e9d30 con 0x7fa2f40b6be0 2024-05-20T18:27:07.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.946+0000 7fa2fa614640 1 -- 172.21.15.5:0/3391491318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2f41ea2f0 con 0x7fa2f40b6be0 2024-05-20T18:27:07.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.947+0000 7fa2fa614640 1 -- 172.21.15.5:0/3391491318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2f40690b0 con 0x7fa2f40b6be0 2024-05-20T18:27:07.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.947+0000 7fa2f17fa640 1 -- 172.21.15.5:0/3391491318 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa2e4044070 con 0x7fa2f40b6be0 2024-05-20T18:27:07.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.948+0000 7fa2f17fa640 1 --2- 172.21.15.5:0/3391491318 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa2d406be30 0x7fa2d406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:07.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.948+0000 7fa2ebfff640 1 --2- 172.21.15.5:0/3391491318 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa2d406be30 0x7fa2d406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:07.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.948+0000 7fa2f17fa640 1 -- 172.21.15.5:0/3391491318 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa2e40ab300 con 0x7fa2f40b6be0 2024-05-20T18:27:07.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.948+0000 7fa2ebfff640 1 --2- 172.21.15.5:0/3391491318 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa2d406be30 0x7fa2d406e340 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa2f406d3a0 tx=0x7fa2dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:07.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:07.953+0000 7fa2f17fa640 1 -- 172.21.15.5:0/3391491318 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa2e4079660 con 0x7fa2f40b6be0 2024-05-20T18:27:08.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.144+0000 7fa2fa614640 1 -- 172.21.15.5:0/3391491318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fa2f415a7a0 con 0x7fa2f40b6be0 2024-05-20T18:27:08.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.787+0000 7fa2f17fa640 1 -- 172.21.15.5:0/3391491318 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) ==== 157+0+0 (secure 0 0 0) 0x7fa2e407d960 con 0x7fa2f40b6be0 2024-05-20T18:27:08.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.791+0000 7fa2fa614640 1 -- 172.21.15.5:0/3391491318 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa2d406be30 msgr2=0x7fa2d406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:08.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.791+0000 7fa2fa614640 1 --2- 172.21.15.5:0/3391491318 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa2d406be30 0x7fa2d406e340 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa2f406d3a0 tx=0x7fa2dc009040 comp rx=0 tx=0).stop 2024-05-20T18:27:08.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.791+0000 7fa2fa614640 1 -- 172.21.15.5:0/3391491318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2f40b6be0 msgr2=0x7fa2f41e8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:08.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.791+0000 7fa2fa614640 1 --2- 172.21.15.5:0/3391491318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2f40b6be0 0x7fa2f41e8f50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4009720 tx=0x7fa2e40046c0 comp rx=0 tx=0).stop 2024-05-20T18:27:08.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.791+0000 7fa2fa614640 1 -- 172.21.15.5:0/3391491318 shutdown_connections 2024-05-20T18:27:08.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.791+0000 7fa2fa614640 1 --2- 172.21.15.5:0/3391491318 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa2d406be30 0x7fa2d406e340 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:08.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.791+0000 7fa2fa614640 1 --2- 172.21.15.5:0/3391491318 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2f4163950 0x7fa2f40cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:08.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.791+0000 7fa2fa614640 1 --2- 172.21.15.5:0/3391491318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2f40d3c60 0x7fa2f414d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:08.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.791+0000 7fa2fa614640 1 --2- 172.21.15.5:0/3391491318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2f40b6be0 0x7fa2f41e8f50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:08.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.791+0000 7fa2fa614640 1 -- 172.21.15.5:0/3391491318 >> 172.21.15.5:0/3391491318 conn(0x7fa2f40d16a0 msgr2=0x7fa2f40d3550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:08.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.791+0000 7fa2fa614640 1 -- 172.21.15.5:0/3391491318 shutdown_connections 2024-05-20T18:27:08.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.792+0000 7fa2fa614640 1 -- 172.21.15.5:0/3391491318 wait complete. 2024-05-20T18:27:08.806 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-20T18:27:08.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.936+0000 7f970cebc640 1 Processor -- start 2024-05-20T18:27:08.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.936+0000 7f970cebc640 1 -- start start 2024-05-20T18:27:08.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.938+0000 7f970cebc640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f970815cde0 0x7f97080d4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:08.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.938+0000 7f970cebc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97080d5200 0x7f970815fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:08.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.938+0000 7f970cebc640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f970815d1f0 con 0x7f970815f7a0 2024-05-20T18:27:08.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.938+0000 7f970cebc640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f97080d3bc0 con 0x7f970815cde0 2024-05-20T18:27:08.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.938+0000 7f970cebc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f97081602b0 con 0x7f97080d5200 2024-05-20T18:27:08.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.938+0000 7f970659c640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f970815cde0 0x7f97080d4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:08.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.938+0000 7f9706d9d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97080d5200 0x7f970815fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:08.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.938+0000 7f9706d9d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97080d5200 0x7f970815fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:58576/0 (socket says 172.21.15.5:58576) 2024-05-20T18:27:08.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.938+0000 7f970659c640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f970815cde0 0x7f97080d4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:57004/0 (socket says 172.21.15.5:57004) 2024-05-20T18:27:08.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.938+0000 7f9705d9b640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f970815f7a0 0x7f97080d4af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:50662/0 (socket says 172.21.15.5:50662) 2024-05-20T18:27:08.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.938+0000 7f9706d9d640 1 -- 172.21.15.5:0/1147799751 learned_addr learned my addr 172.21.15.5:0/1147799751 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:08.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.938+0000 7f970559a640 1 -- 172.21.15.5:0/1147799751 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2575737764 0 0) 0x7f970815d1f0 con 0x7f970815f7a0 2024-05-20T18:27:08.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.939+0000 7f970559a640 1 -- 172.21.15.5:0/1147799751 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96f0003520 con 0x7f970815f7a0 2024-05-20T18:27:08.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.939+0000 7f9706d9d640 1 -- 172.21.15.5:0/1147799751 >> v2:172.21.15.63:3300/0 conn(0x7f970815cde0 msgr2=0x7f97080d4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:08.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.939+0000 7f9706d9d640 1 --2- 172.21.15.5:0/1147799751 >> v2:172.21.15.63:3300/0 conn(0x7f970815cde0 0x7f97080d4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:08.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.939+0000 7f9706d9d640 1 -- 172.21.15.5:0/1147799751 >> v1:172.21.15.63:6789/0 conn(0x7f970815f7a0 legacy=0x7f97080d4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:08.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.939+0000 7f9706d9d640 1 -- 172.21.15.5:0/1147799751 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9708160590 con 0x7f97080d5200 2024-05-20T18:27:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.939+0000 7f9706d9d640 1 --2- 172.21.15.5:0/1147799751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97080d5200 0x7f970815fbb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9700009870 tx=0x7f970002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b3d59e8a4bf4a7b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.940+0000 7f970559a640 1 -- 172.21.15.5:0/1147799751 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9700037070 con 0x7f97080d5200 2024-05-20T18:27:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.940+0000 7f970559a640 1 -- 172.21.15.5:0/1147799751 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9700032440 con 0x7f97080d5200 2024-05-20T18:27:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.940+0000 7f970559a640 1 -- 172.21.15.5:0/1147799751 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9700032810 con 0x7f97080d5200 2024-05-20T18:27:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.940+0000 7f970cebc640 1 -- 172.21.15.5:0/1147799751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97080d5200 msgr2=0x7f970815fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.940+0000 7f970cebc640 1 --2- 172.21.15.5:0/1147799751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97080d5200 0x7f970815fbb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9700009870 tx=0x7f970002a270 comp rx=0 tx=0).stop 2024-05-20T18:27:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.940+0000 7f970cebc640 1 -- 172.21.15.5:0/1147799751 shutdown_connections 2024-05-20T18:27:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.940+0000 7f970cebc640 1 --2- 172.21.15.5:0/1147799751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97080d5200 0x7f970815fbb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.940+0000 7f970cebc640 1 --2- 172.21.15.5:0/1147799751 >> v2:172.21.15.63:3300/0 conn(0x7f970815cde0 0x7f97080d4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.940+0000 7f970cebc640 1 -- 172.21.15.5:0/1147799751 >> 172.21.15.5:0/1147799751 conn(0x7f970806b520 msgr2=0x7f970806b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.940+0000 7f970cebc640 1 -- 172.21.15.5:0/1147799751 shutdown_connections 2024-05-20T18:27:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.940+0000 7f970cebc640 1 -- 172.21.15.5:0/1147799751 wait complete. 2024-05-20T18:27:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.942+0000 7f970cebc640 1 Processor -- start 2024-05-20T18:27:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.942+0000 7f970cebc640 1 -- start start 2024-05-20T18:27:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.944+0000 7f970cebc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f970815cde0 0x7f97080acea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.944+0000 7f970cebc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f970815f7a0 0x7f97080b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.944+0000 7f970cebc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97080ae480 0x7f97080ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.944+0000 7f970cebc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f97080d4590 con 0x7f970815cde0 2024-05-20T18:27:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.944+0000 7f970cebc640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f97080d31a0 con 0x7f970815f7a0 2024-05-20T18:27:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.944+0000 7f970cebc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f97080d4710 con 0x7f97080ae480 2024-05-20T18:27:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.944+0000 7f970659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f970815cde0 0x7f97080acea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.944+0000 7f970659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f970815cde0 0x7f97080acea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44616/0 (socket says 172.21.15.5:44616) 2024-05-20T18:27:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.944+0000 7f970659c640 1 -- 172.21.15.5:0/768960997 learned_addr learned my addr 172.21.15.5:0/768960997 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.944+0000 7f9705d9b640 1 --2- 172.21.15.5:0/768960997 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f970815f7a0 0x7f97080b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:08.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.944+0000 7f9706d9d640 1 --2- 172.21.15.5:0/768960997 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97080ae480 0x7f97080ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:08.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.944+0000 7f970659c640 1 -- 172.21.15.5:0/768960997 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97080ae480 msgr2=0x7f97080ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:08.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.944+0000 7f970659c640 1 --2- 172.21.15.5:0/768960997 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97080ae480 0x7f97080ae890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:08.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.944+0000 7f970659c640 1 -- 172.21.15.5:0/768960997 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f970815f7a0 msgr2=0x7f97080b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:08.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.944+0000 7f970659c640 1 --2- 172.21.15.5:0/768960997 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f970815f7a0 0x7f97080b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:08.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.944+0000 7f970659c640 1 -- 172.21.15.5:0/768960997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97080abe00 con 0x7f970815cde0 2024-05-20T18:27:08.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.944+0000 7f970659c640 1 --2- 172.21.15.5:0/768960997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f970815cde0 0x7f97080acea0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f96f800efc0 tx=0x7f96f800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:08.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.944+0000 7f96f6ffd640 1 -- 172.21.15.5:0/768960997 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f8018020 con 0x7f970815cde0 2024-05-20T18:27:08.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.944+0000 7f96f6ffd640 1 -- 172.21.15.5:0/768960997 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96f8014480 con 0x7f970815cde0 2024-05-20T18:27:08.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.944+0000 7f970cebc640 1 -- 172.21.15.5:0/768960997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97080ac0e0 con 0x7f970815cde0 2024-05-20T18:27:08.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.945+0000 7f96f6ffd640 1 -- 172.21.15.5:0/768960997 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f8014850 con 0x7f970815cde0 2024-05-20T18:27:08.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.945+0000 7f970cebc640 1 -- 172.21.15.5:0/768960997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97080ac6a0 con 0x7f970815cde0 2024-05-20T18:27:08.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.945+0000 7f970cebc640 1 -- 172.21.15.5:0/768960997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97080690b0 con 0x7f970815cde0 2024-05-20T18:27:08.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.946+0000 7f96f6ffd640 1 -- 172.21.15.5:0/768960997 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f96f8005d10 con 0x7f970815cde0 2024-05-20T18:27:08.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.947+0000 7f96f6ffd640 1 --2- 172.21.15.5:0/768960997 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f96e806be60 0x7f96e806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:08.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.947+0000 7f9705d9b640 1 --2- 172.21.15.5:0/768960997 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f96e806be60 0x7f96e806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:08.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.947+0000 7f96f6ffd640 1 -- 172.21.15.5:0/768960997 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 6007+0+0 (secure 0 0 0) 0x7f96f808bcc0 con 0x7f970815cde0 2024-05-20T18:27:08.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.947+0000 7f9705d9b640 1 --2- 172.21.15.5:0/768960997 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f96e806be60 0x7f96e806e370 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f970815f1c0 tx=0x7f96fc007cf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:08.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:08.951+0000 7f96f6ffd640 1 -- 172.21.15.5:0/768960997 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f96f8022030 con 0x7f970815cde0 2024-05-20T18:27:09.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:09.135+0000 7f970cebc640 1 -- 172.21.15.5:0/768960997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f970815f220 con 0x7f970815cde0 2024-05-20T18:27:10.544 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:27:10.547 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:27:10.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.799+0000 7f96f6ffd640 1 -- 172.21.15.5:0/768960997 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) ==== 228+0+0 (secure 0 0 0) 0x7f96f8010040 con 0x7f970815cde0 2024-05-20T18:27:10.799 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data_ec' created 2024-05-20T18:27:10.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.803+0000 7f970cebc640 1 -- 172.21.15.5:0/768960997 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f96e806be60 msgr2=0x7f96e806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.803+0000 7f970cebc640 1 --2- 172.21.15.5:0/768960997 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f96e806be60 0x7f96e806e370 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f970815f1c0 tx=0x7f96fc007cf0 comp rx=0 tx=0).stop 2024-05-20T18:27:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.803+0000 7f970cebc640 1 -- 172.21.15.5:0/768960997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f970815cde0 msgr2=0x7f97080acea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.803+0000 7f970cebc640 1 --2- 172.21.15.5:0/768960997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f970815cde0 0x7f97080acea0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f96f800efc0 tx=0x7f96f800c490 comp rx=0 tx=0).stop 2024-05-20T18:27:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.803+0000 7f970cebc640 1 -- 172.21.15.5:0/768960997 shutdown_connections 2024-05-20T18:27:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.803+0000 7f970cebc640 1 --2- 172.21.15.5:0/768960997 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f96e806be60 0x7f96e806e370 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.803+0000 7f970cebc640 1 --2- 172.21.15.5:0/768960997 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97080ae480 0x7f97080ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.803+0000 7f970cebc640 1 --2- 172.21.15.5:0/768960997 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f970815f7a0 0x7f97080b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.803+0000 7f970cebc640 1 --2- 172.21.15.5:0/768960997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f970815cde0 0x7f97080acea0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.803+0000 7f970cebc640 1 -- 172.21.15.5:0/768960997 >> 172.21.15.5:0/768960997 conn(0x7f970806b520 msgr2=0x7f97080b36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.804+0000 7f970cebc640 1 -- 172.21.15.5:0/768960997 shutdown_connections 2024-05-20T18:27:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.804+0000 7f970cebc640 1 -- 172.21.15.5:0/768960997 wait complete. 2024-05-20T18:27:10.823 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-20T18:27:10.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.949+0000 7f47c55ff640 1 Processor -- start 2024-05-20T18:27:10.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.949+0000 7f47c55ff640 1 -- start start 2024-05-20T18:27:10.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.951+0000 7f47c55ff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f47c00b3d00 0x7f47c00d3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:10.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.951+0000 7f47c55ff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47c00d1940 0x7f47c00cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:10.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.951+0000 7f47c55ff640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47c015f910 con 0x7f47c00b8530 2024-05-20T18:27:10.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.951+0000 7f47c55ff640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f47c0163d80 con 0x7f47c00b3d00 2024-05-20T18:27:10.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.951+0000 7f47c55ff640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f47c00cf380 con 0x7f47c00d1940 2024-05-20T18:27:10.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.951+0000 7f47beffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f47c00b3d00 0x7f47c00d3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:10.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.951+0000 7f47beffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f47c00b3d00 0x7f47c00d3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44618/0 (socket says 172.21.15.5:44618) 2024-05-20T18:27:10.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.951+0000 7f47be7fc640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f47c00b8530 0x7f47c00d1230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44318/0 (socket says 172.21.15.5:44318) 2024-05-20T18:27:10.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.951+0000 7f47beffd640 1 -- 172.21.15.5:0/1468961170 learned_addr learned my addr 172.21.15.5:0/1468961170 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.951+0000 7f47bf7fe640 1 --2- 172.21.15.5:0/1468961170 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47c00d1940 0x7f47c00cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.951+0000 7f47beffd640 1 -- 172.21.15.5:0/1468961170 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47c00d1940 msgr2=0x7f47c00cfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.951+0000 7f47beffd640 1 --2- 172.21.15.5:0/1468961170 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47c00d1940 0x7f47c00cfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.951+0000 7f47beffd640 1 -- 172.21.15.5:0/1468961170 >> v1:172.21.15.5:6789/0 conn(0x7f47c00b8530 legacy=0x7f47c00d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.951+0000 7f47beffd640 1 -- 172.21.15.5:0/1468961170 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c00cf660 con 0x7f47c00b3d00 2024-05-20T18:27:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.951+0000 7f47bdffb640 1 -- 172.21.15.5:0/1468961170 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 352402186 0 0) 0x7f47c015f910 con 0x7f47c00b8530 2024-05-20T18:27:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.952+0000 7f47beffd640 1 --2- 172.21.15.5:0/1468961170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c00b3d00 0x7f47c00d3a50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009870 tx=0x7f47b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e3d099339aea0e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.952+0000 7f47bdffb640 1 -- 172.21.15.5:0/1468961170 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b0037070 con 0x7f47c00b3d00 2024-05-20T18:27:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.952+0000 7f47bdffb640 1 -- 172.21.15.5:0/1468961170 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47b0032440 con 0x7f47c00b3d00 2024-05-20T18:27:10.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.952+0000 7f47bdffb640 1 -- 172.21.15.5:0/1468961170 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b0032810 con 0x7f47c00b3d00 2024-05-20T18:27:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.953+0000 7f47c55ff640 1 -- 172.21.15.5:0/1468961170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c00b3d00 msgr2=0x7f47c00d3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.953+0000 7f47c55ff640 1 --2- 172.21.15.5:0/1468961170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c00b3d00 0x7f47c00d3a50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009870 tx=0x7f47b002a270 comp rx=0 tx=0).stop 2024-05-20T18:27:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.953+0000 7f47c55ff640 1 -- 172.21.15.5:0/1468961170 shutdown_connections 2024-05-20T18:27:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.953+0000 7f47c55ff640 1 --2- 172.21.15.5:0/1468961170 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47c00d1940 0x7f47c00cfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.953+0000 7f47c55ff640 1 --2- 172.21.15.5:0/1468961170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c00b3d00 0x7f47c00d3a50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:10.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.953+0000 7f47c55ff640 1 -- 172.21.15.5:0/1468961170 >> 172.21.15.5:0/1468961170 conn(0x7f47c00b66f0 msgr2=0x7f47c00b6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.953+0000 7f47c55ff640 1 -- 172.21.15.5:0/1468961170 shutdown_connections 2024-05-20T18:27:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.953+0000 7f47c55ff640 1 -- 172.21.15.5:0/1468961170 wait complete. 2024-05-20T18:27:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.955+0000 7f47c55ff640 1 Processor -- start 2024-05-20T18:27:10.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.955+0000 7f47c55ff640 1 -- start start 2024-05-20T18:27:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.956+0000 7f47c55ff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47c00b3d00 0x7f47c014a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.956+0000 7f47c55ff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c00b8530 0x7f47c01f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.956+0000 7f47c55ff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47c00d1940 0x7f47c00ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.956+0000 7f47c55ff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f47c00ba970 con 0x7f47c00b8530 2024-05-20T18:27:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.956+0000 7f47c55ff640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f47c0163a30 con 0x7f47c00b3d00 2024-05-20T18:27:10.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.956+0000 7f47c55ff640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f47c00baaf0 con 0x7f47c00d1940 2024-05-20T18:27:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.956+0000 7f47be7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c00b8530 0x7f47c01f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.956+0000 7f47be7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c00b8530 0x7f47c01f52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44628/0 (socket says 172.21.15.5:44628) 2024-05-20T18:27:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.956+0000 7f47be7fc640 1 -- 172.21.15.5:0/3947471018 learned_addr learned my addr 172.21.15.5:0/3947471018 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.956+0000 7f47bf7fe640 1 --2- 172.21.15.5:0/3947471018 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47c00d1940 0x7f47c00ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.956+0000 7f47beffd640 1 --2- 172.21.15.5:0/3947471018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47c00b3d00 0x7f47c014a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.956+0000 7f47be7fc640 1 -- 172.21.15.5:0/3947471018 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47c00d1940 msgr2=0x7f47c00ab4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.956+0000 7f47be7fc640 1 --2- 172.21.15.5:0/3947471018 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47c00d1940 0x7f47c00ab4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.956+0000 7f47be7fc640 1 -- 172.21.15.5:0/3947471018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47c00b3d00 msgr2=0x7f47c014a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.956+0000 7f47be7fc640 1 --2- 172.21.15.5:0/3947471018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47c00b3d00 0x7f47c014a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.956+0000 7f47be7fc640 1 -- 172.21.15.5:0/3947471018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c014d5b0 con 0x7f47c00b8530 2024-05-20T18:27:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.957+0000 7f47be7fc640 1 --2- 172.21.15.5:0/3947471018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c00b8530 0x7f47c01f52f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f47b400c460 tx=0x7f47b400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.957+0000 7f47a77fe640 1 -- 172.21.15.5:0/3947471018 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b4007480 con 0x7f47c00b8530 2024-05-20T18:27:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.957+0000 7f47a77fe640 1 -- 172.21.15.5:0/3947471018 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47b4007620 con 0x7f47c00b8530 2024-05-20T18:27:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.957+0000 7f47c55ff640 1 -- 172.21.15.5:0/3947471018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c0160970 con 0x7f47c00b8530 2024-05-20T18:27:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.957+0000 7f47a77fe640 1 -- 172.21.15.5:0/3947471018 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b40059e0 con 0x7f47c00b8530 2024-05-20T18:27:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.957+0000 7f47c55ff640 1 -- 172.21.15.5:0/3947471018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c0160f30 con 0x7f47c00b8530 2024-05-20T18:27:10.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.958+0000 7f47c55ff640 1 -- 172.21.15.5:0/3947471018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47c0075980 con 0x7f47c00b8530 2024-05-20T18:27:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.959+0000 7f47a77fe640 1 -- 172.21.15.5:0/3947471018 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f47b4005b80 con 0x7f47c00b8530 2024-05-20T18:27:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.959+0000 7f47a77fe640 1 --2- 172.21.15.5:0/3947471018 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f47ac06be30 0x7f47ac06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.959+0000 7f47a77fe640 1 -- 172.21.15.5:0/3947471018 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7f47b408add0 con 0x7f47c00b8530 2024-05-20T18:27:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.959+0000 7f47beffd640 1 --2- 172.21.15.5:0/3947471018 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f47ac06be30 0x7f47ac06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:10.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.960+0000 7f47beffd640 1 --2- 172.21.15.5:0/3947471018 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f47ac06be30 0x7f47ac06e340 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f47b0034000 tx=0x7f47b002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:10.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:10.964+0000 7f47a77fe640 1 -- 172.21.15.5:0/3947471018 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f47b40593d0 con 0x7f47c00b8530 2024-05-20T18:27:11.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.149+0000 7f47c55ff640 1 -- 172.21.15.5:0/3947471018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f47c00d3370 con 0x7f47c00b8530 2024-05-20T18:27:11.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.802+0000 7f47a77fe640 1 -- 172.21.15.5:0/3947471018 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) ==== 169+0+0 (secure 0 0 0) 0x7f47b408c070 con 0x7f47c00b8530 2024-05-20T18:27:11.801 INFO:teuthology.orchestra.run.smithi005.stderr:set pool 4 allow_ec_overwrites to true 2024-05-20T18:27:11.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.805+0000 7f47c55ff640 1 -- 172.21.15.5:0/3947471018 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f47ac06be30 msgr2=0x7f47ac06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:11.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.805+0000 7f47c55ff640 1 --2- 172.21.15.5:0/3947471018 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f47ac06be30 0x7f47ac06e340 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f47b0034000 tx=0x7f47b002e000 comp rx=0 tx=0).stop 2024-05-20T18:27:11.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.805+0000 7f47c55ff640 1 -- 172.21.15.5:0/3947471018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c00b8530 msgr2=0x7f47c01f52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:11.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.805+0000 7f47c55ff640 1 --2- 172.21.15.5:0/3947471018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c00b8530 0x7f47c01f52f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f47b400c460 tx=0x7f47b400c920 comp rx=0 tx=0).stop 2024-05-20T18:27:11.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.806+0000 7f47c55ff640 1 -- 172.21.15.5:0/3947471018 shutdown_connections 2024-05-20T18:27:11.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.806+0000 7f47c55ff640 1 --2- 172.21.15.5:0/3947471018 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f47ac06be30 0x7f47ac06e340 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:11.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.806+0000 7f47c55ff640 1 --2- 172.21.15.5:0/3947471018 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47c00d1940 0x7f47c00ab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:11.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.806+0000 7f47c55ff640 1 --2- 172.21.15.5:0/3947471018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c00b8530 0x7f47c01f52f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:11.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.806+0000 7f47c55ff640 1 --2- 172.21.15.5:0/3947471018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47c00b3d00 0x7f47c014a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:11.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.806+0000 7f47c55ff640 1 -- 172.21.15.5:0/3947471018 >> 172.21.15.5:0/3947471018 conn(0x7f47c00b66f0 msgr2=0x7f47c00b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:11.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.806+0000 7f47c55ff640 1 -- 172.21.15.5:0/3947471018 shutdown_connections 2024-05-20T18:27:11.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.807+0000 7f47c55ff640 1 -- 172.21.15.5:0/3947471018 wait complete. 2024-05-20T18:27:11.841 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-20T18:27:11.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.963+0000 7f3bb3bce640 1 Processor -- start 2024-05-20T18:27:11.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.963+0000 7f3bb3bce640 1 -- start start 2024-05-20T18:27:11.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.964+0000 7f3bb3bce640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3bac162820 0x7f3bac0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:11.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.964+0000 7f3bb3bce640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bac06dae0 0x7f3bac0b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:11.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.964+0000 7f3bb3bce640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bac0d0630 con 0x7f3bac0b3e80 2024-05-20T18:27:11.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.964+0000 7f3bb3bce640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f3bac0d39c0 con 0x7f3bac162820 2024-05-20T18:27:11.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.964+0000 7f3bb3bce640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3bac0d3ca0 con 0x7f3bac06dae0 2024-05-20T18:27:11.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.965+0000 7f3bb1169640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3bac0b3e80 0x7f3bac06d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44334/0 (socket says 172.21.15.5:44334) 2024-05-20T18:27:11.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.965+0000 7f3bb1169640 1 -- 172.21.15.5:0/1415826306 learned_addr learned my addr 172.21.15.5:0/1415826306 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:11.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.965+0000 7f3bb216b640 1 --2- 172.21.15.5:0/1415826306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bac06dae0 0x7f3bac0b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:11.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.965+0000 7f3bb196a640 1 --2- 172.21.15.5:0/1415826306 >> v2:172.21.15.63:3300/0 conn(0x7f3bac162820 0x7f3bac0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:11.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.965+0000 7f3bb0968640 1 -- 172.21.15.5:0/1415826306 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1620358787 0 0) 0x7f3bac0d0630 con 0x7f3bac0b3e80 2024-05-20T18:27:11.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.965+0000 7f3bb0968640 1 -- 172.21.15.5:0/1415826306 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b98003520 con 0x7f3bac0b3e80 2024-05-20T18:27:11.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.965+0000 7f3bb216b640 1 -- 172.21.15.5:0/1415826306 >> v2:172.21.15.63:3300/0 conn(0x7f3bac162820 msgr2=0x7f3bac0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:11.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.965+0000 7f3bb216b640 1 --2- 172.21.15.5:0/1415826306 >> v2:172.21.15.63:3300/0 conn(0x7f3bac162820 0x7f3bac0b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:11.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.965+0000 7f3bb216b640 1 -- 172.21.15.5:0/1415826306 >> v1:172.21.15.5:6789/0 conn(0x7f3bac0b3e80 legacy=0x7f3bac06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:11.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.965+0000 7f3bb216b640 1 -- 172.21.15.5:0/1415826306 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bac0d3f80 con 0x7f3bac06dae0 2024-05-20T18:27:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.966+0000 7f3bb216b640 1 --2- 172.21.15.5:0/1415826306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bac06dae0 0x7f3bac0b8190 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40097a0 tx=0x7f3ba402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66644b7d6ba0bc0d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.966+0000 7f3bb0968640 1 -- 172.21.15.5:0/1415826306 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4037070 con 0x7f3bac06dae0 2024-05-20T18:27:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.966+0000 7f3bb0968640 1 -- 172.21.15.5:0/1415826306 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ba4032440 con 0x7f3bac06dae0 2024-05-20T18:27:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.966+0000 7f3bb0968640 1 -- 172.21.15.5:0/1415826306 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba40327f0 con 0x7f3bac06dae0 2024-05-20T18:27:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.967+0000 7f3bb3bce640 1 -- 172.21.15.5:0/1415826306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bac06dae0 msgr2=0x7f3bac0b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.967+0000 7f3bb3bce640 1 --2- 172.21.15.5:0/1415826306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bac06dae0 0x7f3bac0b8190 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40097a0 tx=0x7f3ba402a180 comp rx=0 tx=0).stop 2024-05-20T18:27:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.967+0000 7f3bb3bce640 1 -- 172.21.15.5:0/1415826306 shutdown_connections 2024-05-20T18:27:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.967+0000 7f3bb3bce640 1 --2- 172.21.15.5:0/1415826306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bac06dae0 0x7f3bac0b8190 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.967+0000 7f3bb3bce640 1 --2- 172.21.15.5:0/1415826306 >> v2:172.21.15.63:3300/0 conn(0x7f3bac162820 0x7f3bac0b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.967+0000 7f3bb3bce640 1 -- 172.21.15.5:0/1415826306 >> 172.21.15.5:0/1415826306 conn(0x7f3bac0d4bd0 msgr2=0x7f3bac0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.967+0000 7f3bb3bce640 1 -- 172.21.15.5:0/1415826306 shutdown_connections 2024-05-20T18:27:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.967+0000 7f3bb3bce640 1 -- 172.21.15.5:0/1415826306 wait complete. 2024-05-20T18:27:11.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.969+0000 7f3bb3bce640 1 Processor -- start 2024-05-20T18:27:11.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.969+0000 7f3bb3bce640 1 -- start start 2024-05-20T18:27:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.970+0000 7f3bb3bce640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bac0b3e80 0x7f3bac14ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.970+0000 7f3bb3bce640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bac162820 0x7f3bac0b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.970+0000 7f3bb3bce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bac0b5900 0x7f3bac1f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.970+0000 7f3bb3bce640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac0d4780 con 0x7f3bac0b5900 2024-05-20T18:27:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.970+0000 7f3bb3bce640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac0d0300 con 0x7f3bac162820 2024-05-20T18:27:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.970+0000 7f3bb3bce640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3bac0d4900 con 0x7f3bac0b3e80 2024-05-20T18:27:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.970+0000 7f3bb216b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bac0b5900 0x7f3bac1f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.970+0000 7f3bb216b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bac0b5900 0x7f3bac1f50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44638/0 (socket says 172.21.15.5:44638) 2024-05-20T18:27:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.970+0000 7f3bb216b640 1 -- 172.21.15.5:0/1205340670 learned_addr learned my addr 172.21.15.5:0/1205340670 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.970+0000 7f3bb196a640 1 --2- 172.21.15.5:0/1205340670 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bac0b3e80 0x7f3bac14ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.970+0000 7f3bb1169640 1 --2- 172.21.15.5:0/1205340670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bac162820 0x7f3bac0b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.970+0000 7f3bb216b640 1 -- 172.21.15.5:0/1205340670 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bac0b3e80 msgr2=0x7f3bac14ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.971+0000 7f3bb216b640 1 --2- 172.21.15.5:0/1205340670 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bac0b3e80 0x7f3bac14ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.971+0000 7f3bb216b640 1 -- 172.21.15.5:0/1205340670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bac162820 msgr2=0x7f3bac0b53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.971+0000 7f3bb216b640 1 --2- 172.21.15.5:0/1205340670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bac162820 0x7f3bac0b53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.971+0000 7f3bb216b640 1 -- 172.21.15.5:0/1205340670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bac1581d0 con 0x7f3bac0b5900 2024-05-20T18:27:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.971+0000 7f3bb1169640 1 --2- 172.21.15.5:0/1205340670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bac162820 0x7f3bac0b53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:11.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.971+0000 7f3bb196a640 1 --2- 172.21.15.5:0/1205340670 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bac0b3e80 0x7f3bac14ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:11.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.971+0000 7f3bb216b640 1 --2- 172.21.15.5:0/1205340670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bac0b5900 0x7f3bac1f50f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4035040 tx=0x7f3ba40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:11.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.971+0000 7f3ba27fc640 1 -- 172.21.15.5:0/1205340670 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4037040 con 0x7f3bac0b5900 2024-05-20T18:27:11.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.971+0000 7f3ba27fc640 1 -- 172.21.15.5:0/1205340670 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ba4042070 con 0x7f3bac0b5900 2024-05-20T18:27:11.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.971+0000 7f3bb3bce640 1 -- 172.21.15.5:0/1205340670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bac0b5d10 con 0x7f3bac0b5900 2024-05-20T18:27:11.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.971+0000 7f3ba27fc640 1 -- 172.21.15.5:0/1205340670 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba40044e0 con 0x7f3bac0b5900 2024-05-20T18:27:11.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.971+0000 7f3bb3bce640 1 -- 172.21.15.5:0/1205340670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bac15a9c0 con 0x7f3bac0b5900 2024-05-20T18:27:11.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.972+0000 7f3bb3bce640 1 -- 172.21.15.5:0/1205340670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bac075b00 con 0x7f3bac0b5900 2024-05-20T18:27:11.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.973+0000 7f3ba27fc640 1 -- 172.21.15.5:0/1205340670 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3ba4032830 con 0x7f3bac0b5900 2024-05-20T18:27:11.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.973+0000 7f3ba27fc640 1 --2- 172.21.15.5:0/1205340670 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3b9006be30 0x7f3b9006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:11.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.973+0000 7f3ba27fc640 1 -- 172.21.15.5:0/1205340670 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7f3ba40ab970 con 0x7f3bac0b5900 2024-05-20T18:27:11.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.973+0000 7f3bb196a640 1 --2- 172.21.15.5:0/1205340670 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3b9006be30 0x7f3b9006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:11.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.974+0000 7f3bb196a640 1 --2- 172.21.15.5:0/1205340670 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3b9006be30 0x7f3b9006e340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8007440 tx=0x7f3ba800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:11.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:11.978+0000 7f3ba27fc640 1 -- 172.21.15.5:0/1205340670 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3bac0b5d10 con 0x7f3bac0b5900 2024-05-20T18:27:12.109 INFO:tasks.ceph.mds.b.smithi063.stderr:2024-05-20T18:27:12.110+0000 7f25b7bf3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T18:27:12.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.206+0000 7f3bb3bce640 1 -- 172.21.15.5:0/1205340670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f3bac15c4a0 con 0x7f3bac0b5900 2024-05-20T18:27:12.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.827+0000 7f3ba27fc640 1 -- 172.21.15.5:0/1205340670 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7f3bac1581d0 con 0x7f3bac0b5900 2024-05-20T18:27:12.826 INFO:teuthology.orchestra.run.smithi005.stderr:added data pool 4 to fsmap 2024-05-20T18:27:12.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.830+0000 7f3bb3bce640 1 -- 172.21.15.5:0/1205340670 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3b9006be30 msgr2=0x7f3b9006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:12.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.830+0000 7f3bb3bce640 1 --2- 172.21.15.5:0/1205340670 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3b9006be30 0x7f3b9006e340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8007440 tx=0x7f3ba800b040 comp rx=0 tx=0).stop 2024-05-20T18:27:12.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.830+0000 7f3bb3bce640 1 -- 172.21.15.5:0/1205340670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bac0b5900 msgr2=0x7f3bac1f50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:12.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.830+0000 7f3bb3bce640 1 --2- 172.21.15.5:0/1205340670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bac0b5900 0x7f3bac1f50f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4035040 tx=0x7f3ba40046c0 comp rx=0 tx=0).stop 2024-05-20T18:27:12.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.830+0000 7f3bb3bce640 1 -- 172.21.15.5:0/1205340670 shutdown_connections 2024-05-20T18:27:12.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.830+0000 7f3bb3bce640 1 --2- 172.21.15.5:0/1205340670 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3b9006be30 0x7f3b9006e340 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:12.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.830+0000 7f3bb3bce640 1 --2- 172.21.15.5:0/1205340670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bac0b5900 0x7f3bac1f50f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:12.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.830+0000 7f3bb3bce640 1 --2- 172.21.15.5:0/1205340670 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bac162820 0x7f3bac0b53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:12.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.830+0000 7f3bb3bce640 1 --2- 172.21.15.5:0/1205340670 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3bac0b3e80 0x7f3bac14ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:12.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.830+0000 7f3bb3bce640 1 -- 172.21.15.5:0/1205340670 >> 172.21.15.5:0/1205340670 conn(0x7f3bac0d4bd0 msgr2=0x7f3bac15edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:12.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.830+0000 7f3bb3bce640 1 -- 172.21.15.5:0/1205340670 shutdown_connections 2024-05-20T18:27:12.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.830+0000 7f3bb3bce640 1 -- 172.21.15.5:0/1205340670 wait complete. 2024-05-20T18:27:12.844 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:27:12.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.966+0000 7fa8eae1b640 1 Processor -- start 2024-05-20T18:27:12.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.966+0000 7fa8eae1b640 1 -- start start 2024-05-20T18:27:12.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.968+0000 7fa8eae1b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa8e4162820 0x7fa8e40b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.968+0000 7fa8eae1b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa8e40b3e80 0x7fa8e406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.968+0000 7fa8eae1b640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8e406dd20 con 0x7fa8e406d910 2024-05-20T18:27:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.968+0000 7fa8eae1b640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fa8e40d0630 con 0x7fa8e4162820 2024-05-20T18:27:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.968+0000 7fa8eae1b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa8e40d39c0 con 0x7fa8e40b3e80 2024-05-20T18:27:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.968+0000 7fa8e8bb7640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa8e4162820 0x7fa8e40b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.968+0000 7fa8e8bb7640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa8e4162820 0x7fa8e40b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44654/0 (socket says 172.21.15.5:44654) 2024-05-20T18:27:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.968+0000 7fa8e8bb7640 1 -- 172.21.15.5:0/1492243790 learned_addr learned my addr 172.21.15.5:0/1492243790 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.968+0000 7fa8e3fff640 1 --2- 172.21.15.5:0/1492243790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa8e40b3e80 0x7fa8e406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.968+0000 7fa8e8bb7640 1 -- 172.21.15.5:0/1492243790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa8e40b3e80 msgr2=0x7fa8e406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.968+0000 7fa8e8bb7640 1 --2- 172.21.15.5:0/1492243790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa8e40b3e80 0x7fa8e406d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.968+0000 7fa8e8bb7640 1 -- 172.21.15.5:0/1492243790 >> v1:172.21.15.5:6789/0 conn(0x7fa8e406d910 legacy=0x7fa8e40d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:12.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.968+0000 7fa8e8bb7640 1 -- 172.21.15.5:0/1492243790 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e40d3ca0 con 0x7fa8e4162820 2024-05-20T18:27:12.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.968+0000 7fa8e37fe640 1 -- 172.21.15.5:0/1492243790 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1595020285 0 0) 0x7fa8dc001490 con 0x7fa8e406d910 2024-05-20T18:27:12.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.968+0000 7fa8e3fff640 1 --2- 172.21.15.5:0/1492243790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa8e40b3e80 0x7fa8e406d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:12.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.969+0000 7fa8e8bb7640 1 --2- 172.21.15.5:0/1492243790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e4162820 0x7fa8e40b3940 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4009870 tx=0x7fa8d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8877290bec7558fe server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:12.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.969+0000 7fa8e37fe640 1 -- 172.21.15.5:0/1492243790 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4037070 con 0x7fa8e4162820 2024-05-20T18:27:12.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.970+0000 7fa8e37fe640 1 -- 172.21.15.5:0/1492243790 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8d4032440 con 0x7fa8e4162820 2024-05-20T18:27:12.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.970+0000 7fa8e37fe640 1 -- 172.21.15.5:0/1492243790 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4032810 con 0x7fa8e4162820 2024-05-20T18:27:12.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.970+0000 7fa8eae1b640 1 -- 172.21.15.5:0/1492243790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e4162820 msgr2=0x7fa8e40b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:12.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.970+0000 7fa8eae1b640 1 --2- 172.21.15.5:0/1492243790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e4162820 0x7fa8e40b3940 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4009870 tx=0x7fa8d402a270 comp rx=0 tx=0).stop 2024-05-20T18:27:12.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.971+0000 7fa8eae1b640 1 -- 172.21.15.5:0/1492243790 shutdown_connections 2024-05-20T18:27:12.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.971+0000 7fa8eae1b640 1 --2- 172.21.15.5:0/1492243790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa8e40b3e80 0x7fa8e406d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:12.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.971+0000 7fa8eae1b640 1 --2- 172.21.15.5:0/1492243790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e4162820 0x7fa8e40b3940 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:12.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.971+0000 7fa8eae1b640 1 -- 172.21.15.5:0/1492243790 >> 172.21.15.5:0/1492243790 conn(0x7fa8e40d4bd0 msgr2=0x7fa8e40d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:12.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.971+0000 7fa8eae1b640 1 -- 172.21.15.5:0/1492243790 shutdown_connections 2024-05-20T18:27:12.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.971+0000 7fa8eae1b640 1 -- 172.21.15.5:0/1492243790 wait complete. 2024-05-20T18:27:12.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.973+0000 7fa8eae1b640 1 Processor -- start 2024-05-20T18:27:12.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.973+0000 7fa8eae1b640 1 -- start start 2024-05-20T18:27:12.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.974+0000 7fa8eae1b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e406d910 0x7fa8e415ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:12.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.974+0000 7fa8eae1b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa8e40b3e80 0x7fa8e4158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.974+0000 7fa8eae1b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8e4162820 0x7fa8e415b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.974+0000 7fa8eae1b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e40d0300 con 0x7fa8e406d910 2024-05-20T18:27:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.974+0000 7fa8eae1b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e40d4260 con 0x7fa8e4162820 2024-05-20T18:27:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.974+0000 7fa8eae1b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa8e40b91a0 con 0x7fa8e40b3e80 2024-05-20T18:27:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.974+0000 7fa8e8bb7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e406d910 0x7fa8e415ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.974+0000 7fa8e8bb7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e406d910 0x7fa8e415ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44670/0 (socket says 172.21.15.5:44670) 2024-05-20T18:27:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.974+0000 7fa8e8bb7640 1 -- 172.21.15.5:0/4253912099 learned_addr learned my addr 172.21.15.5:0/4253912099 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.975+0000 7fa8e8bb7640 1 -- 172.21.15.5:0/4253912099 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa8e40b3e80 msgr2=0x7fa8e4158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.975+0000 7fa8e8bb7640 1 --2- 172.21.15.5:0/4253912099 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa8e40b3e80 0x7fa8e4158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.975+0000 7fa8e8bb7640 1 -- 172.21.15.5:0/4253912099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8e4162820 msgr2=0x7fa8e415b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.975+0000 7fa8e8bb7640 1 --2- 172.21.15.5:0/4253912099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8e4162820 0x7fa8e415b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.975+0000 7fa8e8bb7640 1 -- 172.21.15.5:0/4253912099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e40b4630 con 0x7fa8e406d910 2024-05-20T18:27:12.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.975+0000 7fa8e8bb7640 1 --2- 172.21.15.5:0/4253912099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e406d910 0x7fa8e415ab70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4009870 tx=0x7fa8d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:12.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.975+0000 7fa8e17fa640 1 -- 172.21.15.5:0/4253912099 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4041020 con 0x7fa8e406d910 2024-05-20T18:27:12.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.975+0000 7fa8e17fa640 1 -- 172.21.15.5:0/4253912099 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8d40306c0 con 0x7fa8e406d910 2024-05-20T18:27:12.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.975+0000 7fa8eae1b640 1 -- 172.21.15.5:0/4253912099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8e4158c50 con 0x7fa8e406d910 2024-05-20T18:27:12.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.975+0000 7fa8e17fa640 1 -- 172.21.15.5:0/4253912099 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4037040 con 0x7fa8e406d910 2024-05-20T18:27:12.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.975+0000 7fa8eae1b640 1 -- 172.21.15.5:0/4253912099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8e41f67f0 con 0x7fa8e406d910 2024-05-20T18:27:12.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.976+0000 7fa8eae1b640 1 -- 172.21.15.5:0/4253912099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8e4075b00 con 0x7fa8e406d910 2024-05-20T18:27:12.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.977+0000 7fa8e17fa640 1 -- 172.21.15.5:0/4253912099 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa8d4044020 con 0x7fa8e406d910 2024-05-20T18:27:12.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.977+0000 7fa8e17fa640 1 --2- 172.21.15.5:0/4253912099 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa8d006be30 0x7fa8d006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:12.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.977+0000 7fa8e17fa640 1 -- 172.21.15.5:0/4253912099 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa8d40ab6b0 con 0x7fa8e406d910 2024-05-20T18:27:12.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.977+0000 7fa8e3fff640 1 --2- 172.21.15.5:0/4253912099 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa8d006be30 0x7fa8d006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:12.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.978+0000 7fa8e3fff640 1 --2- 172.21.15.5:0/4253912099 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa8d006be30 0x7fa8d006e340 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8006fd0 tx=0x7fa8d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:12.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:12.982+0000 7fa8e17fa640 1 -- 172.21.15.5:0/4253912099 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa8d404b070 con 0x7fa8e406d910 2024-05-20T18:27:13.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.203+0000 7fa8eae1b640 1 -- 172.21.15.5:0/4253912099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8e4068770 con 0x7fa8e406d910 2024-05-20T18:27:13.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.204+0000 7fa8e17fa640 1 -- 172.21.15.5:0/4253912099 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5168 (secure 0 0 0) 0x7fa8d4079c90 con 0x7fa8e406d910 2024-05-20T18:27:13.203 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:13.203 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2982954341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2982954341},{"type":"v1","addr":"172.21.15.5:6836","nonce":2982954341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4032850001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":4032850001},{"type":"v1","addr":"172.21.15.5:6837","nonce":4032850001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6833/424580378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":424580378},{"type":"v1","addr":"172.21.15.63:6833","nonce":424580378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:27:07.164414+0000","modified":"2024-05-20T18:27:12.208433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/3249423677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3249423677},{"type":"v1","addr":"172.21.15.63:6835","nonce":3249423677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-05-20T18:27:13.204 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 7 2024-05-20T18:27:13.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.206+0000 7fa8eae1b640 1 -- 172.21.15.5:0/4253912099 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa8d006be30 msgr2=0x7fa8d006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:13.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.206+0000 7fa8eae1b640 1 --2- 172.21.15.5:0/4253912099 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa8d006be30 0x7fa8d006e340 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8006fd0 tx=0x7fa8d8008040 comp rx=0 tx=0).stop 2024-05-20T18:27:13.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.206+0000 7fa8eae1b640 1 -- 172.21.15.5:0/4253912099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e406d910 msgr2=0x7fa8e415ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:13.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.206+0000 7fa8eae1b640 1 --2- 172.21.15.5:0/4253912099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e406d910 0x7fa8e415ab70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4009870 tx=0x7fa8d40046c0 comp rx=0 tx=0).stop 2024-05-20T18:27:13.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.206+0000 7fa8eae1b640 1 -- 172.21.15.5:0/4253912099 shutdown_connections 2024-05-20T18:27:13.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.206+0000 7fa8eae1b640 1 --2- 172.21.15.5:0/4253912099 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa8d006be30 0x7fa8d006e340 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:13.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.206+0000 7fa8eae1b640 1 --2- 172.21.15.5:0/4253912099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8e4162820 0x7fa8e415b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:13.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.206+0000 7fa8eae1b640 1 --2- 172.21.15.5:0/4253912099 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa8e40b3e80 0x7fa8e4158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:13.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.206+0000 7fa8eae1b640 1 --2- 172.21.15.5:0/4253912099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e406d910 0x7fa8e415ab70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:13.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.206+0000 7fa8eae1b640 1 -- 172.21.15.5:0/4253912099 >> 172.21.15.5:0/4253912099 conn(0x7fa8e40d4bd0 msgr2=0x7fa8e40d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:13.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.206+0000 7fa8eae1b640 1 -- 172.21.15.5:0/4253912099 shutdown_connections 2024-05-20T18:27:13.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.206+0000 7fa8eae1b640 1 -- 172.21.15.5:0/4253912099 wait complete. 2024-05-20T18:27:13.216 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:27:13.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.332+0000 7fc6426fe640 1 Processor -- start 2024-05-20T18:27:13.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.332+0000 7fc6426fe640 1 -- start start 2024-05-20T18:27:13.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.333+0000 7fc6426fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc63c0b6620 0x7fc63c0ba3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:13.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.333+0000 7fc6426fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc63c162ce0 0x7fc63c0b8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:13.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.333+0000 7fc6426fe640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc63c1634a0 con 0x7fc63c0b85b0 2024-05-20T18:27:13.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.333+0000 7fc6426fe640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fc63c0d3480 con 0x7fc63c0b6620 2024-05-20T18:27:13.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.333+0000 7fc6426fe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc63c163780 con 0x7fc63c162ce0 2024-05-20T18:27:13.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.333+0000 7fc63bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc63c0b6620 0x7fc63c0ba3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:13.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.333+0000 7fc63bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc63c0b6620 0x7fc63c0ba3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44684/0 (socket says 172.21.15.5:44684) 2024-05-20T18:27:13.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.333+0000 7fc63bfff640 1 -- 172.21.15.5:0/3352755499 learned_addr learned my addr 172.21.15.5:0/3352755499 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:13.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.333+0000 7fc63b7fe640 1 --2- 172.21.15.5:0/3352755499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc63c162ce0 0x7fc63c0b8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:13.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.333+0000 7fc63bfff640 1 -- 172.21.15.5:0/3352755499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc63c162ce0 msgr2=0x7fc63c0b8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:13.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.333+0000 7fc63bfff640 1 --2- 172.21.15.5:0/3352755499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc63c162ce0 0x7fc63c0b8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:13.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.334+0000 7fc63bfff640 1 -- 172.21.15.5:0/3352755499 >> v1:172.21.15.63:6789/0 conn(0x7fc63c0b85b0 legacy=0x7fc63c0d3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:13.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.334+0000 7fc63bfff640 1 -- 172.21.15.5:0/3352755499 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63c163a60 con 0x7fc63c0b6620 2024-05-20T18:27:13.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.334+0000 7fc63bfff640 1 --2- 172.21.15.5:0/3352755499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc63c0b6620 0x7fc63c0ba3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc62c009870 tx=0x7fc62c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9135fbc11970e948 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:13.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.334+0000 7fc63affd640 1 -- 172.21.15.5:0/3352755499 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c037070 con 0x7fc63c0b6620 2024-05-20T18:27:13.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.334+0000 7fc63affd640 1 -- 172.21.15.5:0/3352755499 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc62c032440 con 0x7fc63c0b6620 2024-05-20T18:27:13.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.335+0000 7fc63affd640 1 -- 172.21.15.5:0/3352755499 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c032810 con 0x7fc63c0b6620 2024-05-20T18:27:13.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.335+0000 7fc6426fe640 1 -- 172.21.15.5:0/3352755499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc63c0b6620 msgr2=0x7fc63c0ba3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:13.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.335+0000 7fc6426fe640 1 --2- 172.21.15.5:0/3352755499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc63c0b6620 0x7fc63c0ba3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc62c009870 tx=0x7fc62c02a270 comp rx=0 tx=0).stop 2024-05-20T18:27:13.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.335+0000 7fc6426fe640 1 -- 172.21.15.5:0/3352755499 shutdown_connections 2024-05-20T18:27:13.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.335+0000 7fc6426fe640 1 --2- 172.21.15.5:0/3352755499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc63c162ce0 0x7fc63c0b8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:13.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.335+0000 7fc6426fe640 1 --2- 172.21.15.5:0/3352755499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc63c0b6620 0x7fc63c0ba3c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:13.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.335+0000 7fc6426fe640 1 -- 172.21.15.5:0/3352755499 >> 172.21.15.5:0/3352755499 conn(0x7fc63c06d240 msgr2=0x7fc63c06d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:13.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.335+0000 7fc6426fe640 1 -- 172.21.15.5:0/3352755499 shutdown_connections 2024-05-20T18:27:13.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.335+0000 7fc6426fe640 1 -- 172.21.15.5:0/3352755499 wait complete. 2024-05-20T18:27:13.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.337+0000 7fc6426fe640 1 Processor -- start 2024-05-20T18:27:13.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.337+0000 7fc6426fe640 1 -- start start 2024-05-20T18:27:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.339+0000 7fc6426fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc63c0b6620 0x7fc63c1f95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.339+0000 7fc6426fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc63c0b85b0 0x7fc63c1ea8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.339+0000 7fc6426fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc63c162ce0 0x7fc63c1eae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.339+0000 7fc6426fe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c1630f0 con 0x7fc63c0b85b0 2024-05-20T18:27:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.339+0000 7fc6426fe640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c0d4300 con 0x7fc63c0b6620 2024-05-20T18:27:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.339+0000 7fc6426fe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc63c15cc50 con 0x7fc63c162ce0 2024-05-20T18:27:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.339+0000 7fc63b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc63c0b85b0 0x7fc63c1ea8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.339+0000 7fc63b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc63c0b85b0 0x7fc63c1ea8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44700/0 (socket says 172.21.15.5:44700) 2024-05-20T18:27:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.339+0000 7fc63b7fe640 1 -- 172.21.15.5:0/1613631801 learned_addr learned my addr 172.21.15.5:0/1613631801 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.339+0000 7fc640c9b640 1 --2- 172.21.15.5:0/1613631801 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc63c162ce0 0x7fc63c1eae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.339+0000 7fc63bfff640 1 --2- 172.21.15.5:0/1613631801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc63c0b6620 0x7fc63c1f95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.339+0000 7fc63b7fe640 1 -- 172.21.15.5:0/1613631801 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc63c162ce0 msgr2=0x7fc63c1eae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.339+0000 7fc63b7fe640 1 --2- 172.21.15.5:0/1613631801 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc63c162ce0 0x7fc63c1eae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.339+0000 7fc63b7fe640 1 -- 172.21.15.5:0/1613631801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc63c0b6620 msgr2=0x7fc63c1f95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.339+0000 7fc63b7fe640 1 --2- 172.21.15.5:0/1613631801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc63c0b6620 0x7fc63c1f95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.339+0000 7fc63b7fe640 1 -- 172.21.15.5:0/1613631801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63c15a9d0 con 0x7fc63c0b85b0 2024-05-20T18:27:13.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.339+0000 7fc640c9b640 1 --2- 172.21.15.5:0/1613631801 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc63c162ce0 0x7fc63c1eae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:13.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.339+0000 7fc63bfff640 1 --2- 172.21.15.5:0/1613631801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc63c0b6620 0x7fc63c1f95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:13.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.339+0000 7fc63b7fe640 1 --2- 172.21.15.5:0/1613631801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc63c0b85b0 0x7fc63c1ea8d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc63000d5f0 tx=0x7fc63000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:13.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.339+0000 7fc638ff9640 1 -- 172.21.15.5:0/1613631801 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6300125f0 con 0x7fc63c0b85b0 2024-05-20T18:27:13.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.340+0000 7fc638ff9640 1 -- 172.21.15.5:0/1613631801 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc630019070 con 0x7fc63c0b85b0 2024-05-20T18:27:13.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.340+0000 7fc6426fe640 1 -- 172.21.15.5:0/1613631801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc63c1f9af0 con 0x7fc63c0b85b0 2024-05-20T18:27:13.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.340+0000 7fc638ff9640 1 -- 172.21.15.5:0/1613631801 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6300129d0 con 0x7fc63c0b85b0 2024-05-20T18:27:13.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.340+0000 7fc6426fe640 1 -- 172.21.15.5:0/1613631801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc63c15b5c0 con 0x7fc63c0b85b0 2024-05-20T18:27:13.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.340+0000 7fc6426fe640 1 -- 172.21.15.5:0/1613631801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc63c157e70 con 0x7fc63c0b85b0 2024-05-20T18:27:13.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.341+0000 7fc638ff9640 1 -- 172.21.15.5:0/1613631801 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc630012b70 con 0x7fc63c0b85b0 2024-05-20T18:27:13.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.342+0000 7fc638ff9640 1 --2- 172.21.15.5:0/1613631801 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc62806be30 0x7fc62806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:13.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.342+0000 7fc63bfff640 1 --2- 172.21.15.5:0/1613631801 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc62806be30 0x7fc62806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:13.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.342+0000 7fc638ff9640 1 -- 172.21.15.5:0/1613631801 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc63008b5d0 con 0x7fc63c0b85b0 2024-05-20T18:27:13.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.342+0000 7fc63bfff640 1 --2- 172.21.15.5:0/1613631801 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc62806be30 0x7fc62806e340 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc62c034000 tx=0x7fc62c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:13.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.346+0000 7fc638ff9640 1 -- 172.21.15.5:0/1613631801 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc63c1f9af0 con 0x7fc63c0b85b0 2024-05-20T18:27:13.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.533+0000 7fc6426fe640 1 -- 172.21.15.5:0/1613631801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc63c075b00 con 0x7fc63c0b85b0 2024-05-20T18:27:13.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.534+0000 7fc638ff9640 1 -- 172.21.15.5:0/1613631801 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18407 (secure 0 0 0) 0x7fc63c15a9d0 con 0x7fc63c0b85b0 2024-05-20T18:27:13.533 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:13.534 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:27:12.208211+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:27:05.476596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T18:27:06.137762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T18:27:09.829920+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:27:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.536+0000 7fc6426fe640 1 -- 172.21.15.5:0/1613631801 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc62806be30 msgr2=0x7fc62806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.536+0000 7fc6426fe640 1 --2- 172.21.15.5:0/1613631801 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc62806be30 0x7fc62806e340 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc62c034000 tx=0x7fc62c02e000 comp rx=0 tx=0).stop 2024-05-20T18:27:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.536+0000 7fc6426fe640 1 -- 172.21.15.5:0/1613631801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc63c0b85b0 msgr2=0x7fc63c1ea8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.536+0000 7fc6426fe640 1 --2- 172.21.15.5:0/1613631801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc63c0b85b0 0x7fc63c1ea8d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc63000d5f0 tx=0x7fc63000dab0 comp rx=0 tx=0).stop 2024-05-20T18:27:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.536+0000 7fc6426fe640 1 -- 172.21.15.5:0/1613631801 shutdown_connections 2024-05-20T18:27:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.536+0000 7fc6426fe640 1 --2- 172.21.15.5:0/1613631801 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc62806be30 0x7fc62806e340 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.536+0000 7fc6426fe640 1 --2- 172.21.15.5:0/1613631801 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc63c162ce0 0x7fc63c1eae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.536+0000 7fc6426fe640 1 --2- 172.21.15.5:0/1613631801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc63c0b85b0 0x7fc63c1ea8d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.536+0000 7fc6426fe640 1 --2- 172.21.15.5:0/1613631801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc63c0b6620 0x7fc63c1f95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:13.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.536+0000 7fc6426fe640 1 -- 172.21.15.5:0/1613631801 >> 172.21.15.5:0/1613631801 conn(0x7fc63c06d240 msgr2=0x7fc63c0ab560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:13.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.537+0000 7fc6426fe640 1 -- 172.21.15.5:0/1613631801 shutdown_connections 2024-05-20T18:27:13.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.537+0000 7fc6426fe640 1 -- 172.21.15.5:0/1613631801 wait complete. 2024-05-20T18:27:13.547 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:27:13.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.664+0000 7fa5a398f640 1 Processor -- start 2024-05-20T18:27:13.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.664+0000 7fa5a398f640 1 -- start start 2024-05-20T18:27:13.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.665+0000 7fa5a398f640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa59c162820 0x7fa59c0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:13.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.665+0000 7fa5a398f640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa59c0d0630 con 0x7fa59c06dae0 2024-05-20T18:27:13.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.665+0000 7fa5a398f640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa59c0d39c0 con 0x7fa59c0b3e80 2024-05-20T18:27:13.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.665+0000 7fa5a398f640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa59c0d3ca0 con 0x7fa59c162820 2024-05-20T18:27:13.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.666+0000 7fa5a1f2c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa59c06dae0 0x7fa59c0b8190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44350/0 (socket says 172.21.15.5:44350) 2024-05-20T18:27:13.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.666+0000 7fa5a1f2c640 1 -- 172.21.15.5:0/1348051853 learned_addr learned my addr 172.21.15.5:0/1348051853 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:13.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.666+0000 7fa5a172b640 1 --2- 172.21.15.5:0/1348051853 >> v2:172.21.15.63:3300/0 conn(0x7fa59c162820 0x7fa59c0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:13.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.666+0000 7fa593fff640 1 -- 172.21.15.5:0/1348051853 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2411736981 0 0) 0x7fa59c0d0630 con 0x7fa59c06dae0 2024-05-20T18:27:13.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.666+0000 7fa593fff640 1 -- 172.21.15.5:0/1348051853 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa588003520 con 0x7fa59c06dae0 2024-05-20T18:27:13.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.666+0000 7fa593fff640 1 -- 172.21.15.5:0/1348051853 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1222530032 0 0) 0x7fa59c0d39c0 con 0x7fa59c0b3e80 2024-05-20T18:27:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.666+0000 7fa593fff640 1 -- 172.21.15.5:0/1348051853 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa59c0d0630 con 0x7fa59c0b3e80 2024-05-20T18:27:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.666+0000 7fa593fff640 1 -- 172.21.15.5:0/1348051853 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3027383582 0 0) 0x7fa588003520 con 0x7fa59c06dae0 2024-05-20T18:27:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.666+0000 7fa593fff640 1 -- 172.21.15.5:0/1348051853 >> v2:172.21.15.63:3300/0 conn(0x7fa59c162820 msgr2=0x7fa59c0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.666+0000 7fa593fff640 1 --2- 172.21.15.5:0/1348051853 >> v2:172.21.15.63:3300/0 conn(0x7fa59c162820 0x7fa59c0b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.666+0000 7fa593fff640 1 -- 172.21.15.5:0/1348051853 >> v1:172.21.15.63:6789/0 conn(0x7fa59c0b3e80 legacy=0x7fa59c06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.666+0000 7fa593fff640 1 -- 172.21.15.5:0/1348051853 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59c0d3f80 con 0x7fa59c06dae0 2024-05-20T18:27:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.666+0000 7fa593fff640 1 -- 172.21.15.5:0/1348051853 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7fa5940019f0 con 0x7fa59c06dae0 2024-05-20T18:27:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.666+0000 7fa593fff640 1 -- 172.21.15.5:0/1348051853 >> v1:172.21.15.5:6789/0 conn(0x7fa59c06dae0 legacy=0x7fa59c0b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.666+0000 7fa593fff640 1 --2- 172.21.15.5:0/1348051853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa588003b40 0x7fa588003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.666+0000 7fa593fff640 1 --2- 172.21.15.5:0/1348051853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa588004800 0x7fa588004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.666+0000 7fa593fff640 1 --2- 172.21.15.5:0/1348051853 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5880051a0 0x7fa58800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.666+0000 7fa593fff640 1 -- 172.21.15.5:0/1348051853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c0d3ca0 con 0x7fa588003b40 2024-05-20T18:27:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.666+0000 7fa593fff640 1 -- 172.21.15.5:0/1348051853 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa58800c140 con 0x7fa588004800 2024-05-20T18:27:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.666+0000 7fa593fff640 1 -- 172.21.15.5:0/1348051853 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa58800c340 con 0x7fa5880051a0 2024-05-20T18:27:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.666+0000 7fa5a172b640 1 --2- 172.21.15.5:0/1348051853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa588003b40 0x7fa588003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.666+0000 7fa5a0f2a640 1 --2- 172.21.15.5:0/1348051853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa588004800 0x7fa588004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.667+0000 7fa5a1f2c640 1 --2- 172.21.15.5:0/1348051853 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5880051a0 0x7fa58800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.667+0000 7fa5a172b640 1 -- 172.21.15.5:0/1348051853 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5880051a0 msgr2=0x7fa58800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.667+0000 7fa5a172b640 1 --2- 172.21.15.5:0/1348051853 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5880051a0 0x7fa58800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.667+0000 7fa5a172b640 1 -- 172.21.15.5:0/1348051853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa588004800 msgr2=0x7fa588004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.667+0000 7fa5a172b640 1 --2- 172.21.15.5:0/1348051853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa588004800 0x7fa588004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.667+0000 7fa5a1f2c640 1 -- 172.21.15.5:0/1348051853 reap_dead start 2024-05-20T18:27:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.667+0000 7fa5a172b640 1 -- 172.21.15.5:0/1348051853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa588003520 con 0x7fa588003b40 2024-05-20T18:27:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.667+0000 7fa5a0f2a640 1 --2- 172.21.15.5:0/1348051853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa588004800 0x7fa588004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.667+0000 7fa5a172b640 1 --2- 172.21.15.5:0/1348051853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa588003b40 0x7fa588003f90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa5880057e0 tx=0x7fa588005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de9bf5d627fe284 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.668+0000 7fa593fff640 1 -- 172.21.15.5:0/1348051853 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa598036070 con 0x7fa588003b40 2024-05-20T18:27:13.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.668+0000 7fa593fff640 1 -- 172.21.15.5:0/1348051853 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa598031400 con 0x7fa588003b40 2024-05-20T18:27:13.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.668+0000 7fa593fff640 1 -- 172.21.15.5:0/1348051853 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa598009690 con 0x7fa588003b40 2024-05-20T18:27:13.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.668+0000 7fa5a398f640 1 -- 172.21.15.5:0/1348051853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa588003b40 msgr2=0x7fa588003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:13.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.668+0000 7fa5a398f640 1 --2- 172.21.15.5:0/1348051853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa588003b40 0x7fa588003f90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa5880057e0 tx=0x7fa588005830 comp rx=0 tx=0).stop 2024-05-20T18:27:13.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.668+0000 7fa5a398f640 1 -- 172.21.15.5:0/1348051853 shutdown_connections 2024-05-20T18:27:13.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.668+0000 7fa5a398f640 1 --2- 172.21.15.5:0/1348051853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa588003b40 0x7fa588003f90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:13.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.668+0000 7fa5a398f640 1 -- 172.21.15.5:0/1348051853 >> 172.21.15.5:0/1348051853 conn(0x7fa59c0d4bd0 msgr2=0x7fa59c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:13.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.668+0000 7fa5a398f640 1 -- 172.21.15.5:0/1348051853 shutdown_connections 2024-05-20T18:27:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.668+0000 7fa5a398f640 1 -- 172.21.15.5:0/1348051853 wait complete. 2024-05-20T18:27:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.670+0000 7fa5a398f640 1 Processor -- start 2024-05-20T18:27:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.670+0000 7fa5a398f640 1 -- start start 2024-05-20T18:27:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.671+0000 7fa5a398f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa588003b40 0x7fa59c1464f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.671+0000 7fa5a398f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa59c154b50 0x7fa59c0cde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.671+0000 7fa5a398f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa59c0ce370 0x7fa59c0c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.672+0000 7fa5a398f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c0d4780 con 0x7fa59c0ce370 2024-05-20T18:27:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.672+0000 7fa5a398f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c0d0300 con 0x7fa59c154b50 2024-05-20T18:27:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.672+0000 7fa5a398f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa59c0d4900 con 0x7fa588003b40 2024-05-20T18:27:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.672+0000 7fa5a1f2c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa59c0ce370 0x7fa59c0c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.672+0000 7fa5a1f2c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa59c0ce370 0x7fa59c0c8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44724/0 (socket says 172.21.15.5:44724) 2024-05-20T18:27:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.672+0000 7fa5a1f2c640 1 -- 172.21.15.5:0/4125807165 learned_addr learned my addr 172.21.15.5:0/4125807165 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.672+0000 7fa5a0f2a640 1 --2- 172.21.15.5:0/4125807165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa59c154b50 0x7fa59c0cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.672+0000 7fa5a172b640 1 --2- 172.21.15.5:0/4125807165 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa588003b40 0x7fa59c1464f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.672+0000 7fa5a1f2c640 1 -- 172.21.15.5:0/4125807165 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa588003b40 msgr2=0x7fa59c1464f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.672+0000 7fa5a1f2c640 1 --2- 172.21.15.5:0/4125807165 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa588003b40 0x7fa59c1464f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.672+0000 7fa5a1f2c640 1 -- 172.21.15.5:0/4125807165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa59c154b50 msgr2=0x7fa59c0cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.672+0000 7fa5a1f2c640 1 --2- 172.21.15.5:0/4125807165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa59c154b50 0x7fa59c0cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.672+0000 7fa5a1f2c640 1 -- 172.21.15.5:0/4125807165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59c0ce780 con 0x7fa59c0ce370 2024-05-20T18:27:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.672+0000 7fa5a1f2c640 1 --2- 172.21.15.5:0/4125807165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa59c0ce370 0x7fa59c0c8700 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0cf670 tx=0x7fa59400e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.672+0000 7fa591ffb640 1 -- 172.21.15.5:0/4125807165 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59400eb70 con 0x7fa59c0ce370 2024-05-20T18:27:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.672+0000 7fa591ffb640 1 -- 172.21.15.5:0/4125807165 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa59400ed50 con 0x7fa59c0ce370 2024-05-20T18:27:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.672+0000 7fa5a398f640 1 -- 172.21.15.5:0/4125807165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa59c0cabd0 con 0x7fa59c0ce370 2024-05-20T18:27:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.672+0000 7fa591ffb640 1 -- 172.21.15.5:0/4125807165 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5940146c0 con 0x7fa59c0ce370 2024-05-20T18:27:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.673+0000 7fa5a398f640 1 -- 172.21.15.5:0/4125807165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa59c0cb190 con 0x7fa59c0ce370 2024-05-20T18:27:13.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.674+0000 7fa5a398f640 1 -- 172.21.15.5:0/4125807165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa59c075b00 con 0x7fa59c0ce370 2024-05-20T18:27:13.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.674+0000 7fa591ffb640 1 -- 172.21.15.5:0/4125807165 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa594014860 con 0x7fa59c0ce370 2024-05-20T18:27:13.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.675+0000 7fa591ffb640 1 --2- 172.21.15.5:0/4125807165 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa58006be60 0x7fa58006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:13.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.675+0000 7fa5a172b640 1 --2- 172.21.15.5:0/4125807165 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa58006be60 0x7fa58006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:13.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.675+0000 7fa591ffb640 1 -- 172.21.15.5:0/4125807165 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa59408af90 con 0x7fa59c0ce370 2024-05-20T18:27:13.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.675+0000 7fa5a172b640 1 --2- 172.21.15.5:0/4125807165 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa58006be60 0x7fa58006e370 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa598033000 tx=0x7fa598008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:13.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.679+0000 7fa591ffb640 1 -- 172.21.15.5:0/4125807165 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa59c0ce780 con 0x7fa59c0ce370 2024-05-20T18:27:13.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.870+0000 7fa5a398f640 1 -- 172.21.15.5:0/4125807165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa59c15f900 con 0x7fa59c0ce370 2024-05-20T18:27:13.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.872+0000 7fa591ffb640 1 -- 172.21.15.5:0/4125807165 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18394 (secure 0 0 0) 0x7fa594059600 con 0x7fa59c0ce370 2024-05-20T18:27:13.871 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:13.871 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:27:13.817633+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:27:05.476596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T18:27:06.137762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T18:27:09.829920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:27:13.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.874+0000 7fa5a398f640 1 -- 172.21.15.5:0/4125807165 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa58006be60 msgr2=0x7fa58006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:13.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.874+0000 7fa5a398f640 1 --2- 172.21.15.5:0/4125807165 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa58006be60 0x7fa58006e370 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa598033000 tx=0x7fa598008030 comp rx=0 tx=0).stop 2024-05-20T18:27:13.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.874+0000 7fa5a398f640 1 -- 172.21.15.5:0/4125807165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa59c0ce370 msgr2=0x7fa59c0c8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:13.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.874+0000 7fa5a398f640 1 --2- 172.21.15.5:0/4125807165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa59c0ce370 0x7fa59c0c8700 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0cf670 tx=0x7fa59400e160 comp rx=0 tx=0).stop 2024-05-20T18:27:13.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.874+0000 7fa5a398f640 1 -- 172.21.15.5:0/4125807165 shutdown_connections 2024-05-20T18:27:13.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.874+0000 7fa5a398f640 1 --2- 172.21.15.5:0/4125807165 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa58006be60 0x7fa58006e370 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:13.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.874+0000 7fa5a398f640 1 --2- 172.21.15.5:0/4125807165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa59c0ce370 0x7fa59c0c8700 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:13.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.874+0000 7fa5a398f640 1 --2- 172.21.15.5:0/4125807165 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa59c154b50 0x7fa59c0cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:13.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.874+0000 7fa5a398f640 1 --2- 172.21.15.5:0/4125807165 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa588003b40 0x7fa59c1464f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:13.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.874+0000 7fa5a398f640 1 -- 172.21.15.5:0/4125807165 >> 172.21.15.5:0/4125807165 conn(0x7fa59c0d4bd0 msgr2=0x7fa59c0b4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:13.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.874+0000 7fa5a398f640 1 -- 172.21.15.5:0/4125807165 shutdown_connections 2024-05-20T18:27:13.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:13.874+0000 7fa5a398f640 1 -- 172.21.15.5:0/4125807165 wait complete. 2024-05-20T18:27:13.901 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:27:14.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.017+0000 7fbf792ca640 1 Processor -- start 2024-05-20T18:27:14.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.017+0000 7fbf792ca640 1 -- start start 2024-05-20T18:27:14.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.018+0000 7fbf792ca640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf7406d3d0 0x7fbf7406d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:14.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.018+0000 7fbf792ca640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf7406dd20 con 0x7fbf74162820 2024-05-20T18:27:14.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.018+0000 7fbf792ca640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf740d0630 con 0x7fbf740b8190 2024-05-20T18:27:14.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.018+0000 7fbf792ca640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbf740d39c0 con 0x7fbf7406d3d0 2024-05-20T18:27:14.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.018+0000 7fbf72ffd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbf74162820 0x7fbf740b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44354/0 (socket says 172.21.15.5:44354) 2024-05-20T18:27:14.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.018+0000 7fbf72ffd640 1 -- 172.21.15.5:0/717973743 learned_addr learned my addr 172.21.15.5:0/717973743 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:14.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.019+0000 7fbf71ffb640 1 -- 172.21.15.5:0/717973743 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 167441898 0 0) 0x7fbf64001110 con 0x7fbf74162820 2024-05-20T18:27:14.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.019+0000 7fbf727fc640 1 --2- 172.21.15.5:0/717973743 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf7406d3d0 0x7fbf7406d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:14.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.019+0000 7fbf71ffb640 1 -- 172.21.15.5:0/717973743 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf5c003520 con 0x7fbf74162820 2024-05-20T18:27:14.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.019+0000 7fbf71ffb640 1 -- 172.21.15.5:0/717973743 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3197138321 0 0) 0x7fbf5c003520 con 0x7fbf74162820 2024-05-20T18:27:14.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.019+0000 7fbf71ffb640 1 -- 172.21.15.5:0/717973743 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf7406d3d0 msgr2=0x7fbf7406d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:14.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.019+0000 7fbf71ffb640 1 --2- 172.21.15.5:0/717973743 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf7406d3d0 0x7fbf7406d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:14.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.019+0000 7fbf71ffb640 1 -- 172.21.15.5:0/717973743 >> v1:172.21.15.63:6789/0 conn(0x7fbf740b8190 legacy=0x7fbf740b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:14.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.019+0000 7fbf71ffb640 1 -- 172.21.15.5:0/717973743 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf740d3ca0 con 0x7fbf74162820 2024-05-20T18:27:14.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.019+0000 7fbf71ffb640 1 -- 172.21.15.5:0/717973743 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7fbf64001840 con 0x7fbf74162820 2024-05-20T18:27:14.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.019+0000 7fbf71ffb640 1 -- 172.21.15.5:0/717973743 >> v1:172.21.15.5:6789/0 conn(0x7fbf74162820 legacy=0x7fbf740b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:14.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.019+0000 7fbf71ffb640 1 --2- 172.21.15.5:0/717973743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5c0039e0 0x7fbf5c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:14.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.019+0000 7fbf71ffb640 1 --2- 172.21.15.5:0/717973743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf5c004800 0x7fbf5c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:14.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.019+0000 7fbf71ffb640 1 --2- 172.21.15.5:0/717973743 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf7406d3d0 0x7fbf5c005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:14.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.019+0000 7fbf71ffb640 1 -- 172.21.15.5:0/717973743 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbf740d39c0 con 0x7fbf5c0039e0 2024-05-20T18:27:14.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.019+0000 7fbf71ffb640 1 -- 172.21.15.5:0/717973743 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c00ba70 con 0x7fbf5c004800 2024-05-20T18:27:14.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.019+0000 7fbf71ffb640 1 -- 172.21.15.5:0/717973743 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbf5c00bc70 con 0x7fbf7406d3d0 2024-05-20T18:27:14.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.019+0000 7fbf72ffd640 1 --2- 172.21.15.5:0/717973743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5c0039e0 0x7fbf5c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:14.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.019+0000 7fbf727fc640 1 --2- 172.21.15.5:0/717973743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf5c004800 0x7fbf5c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:14.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.019+0000 7fbf737fe640 1 --2- 172.21.15.5:0/717973743 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf7406d3d0 0x7fbf5c005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:14.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.020+0000 7fbf72ffd640 1 -- 172.21.15.5:0/717973743 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf7406d3d0 msgr2=0x7fbf5c005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:14.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.020+0000 7fbf72ffd640 1 --2- 172.21.15.5:0/717973743 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf7406d3d0 0x7fbf5c005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:14.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.020+0000 7fbf72ffd640 1 -- 172.21.15.5:0/717973743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf5c004800 msgr2=0x7fbf5c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:14.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.020+0000 7fbf72ffd640 1 --2- 172.21.15.5:0/717973743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf5c004800 0x7fbf5c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:14.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.020+0000 7fbf72ffd640 1 -- 172.21.15.5:0/717973743 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf5c003520 con 0x7fbf5c0039e0 2024-05-20T18:27:14.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.020+0000 7fbf72ffd640 1 --2- 172.21.15.5:0/717973743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5c0039e0 0x7fbf5c003e10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbf6400c4c0 tx=0x7fbf5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595ce8dbe00545db server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:14.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.020+0000 7fbf71ffb640 1 -- 172.21.15.5:0/717973743 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6400c0e0 con 0x7fbf5c0039e0 2024-05-20T18:27:14.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.020+0000 7fbf71ffb640 1 -- 172.21.15.5:0/717973743 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf64034070 con 0x7fbf5c0039e0 2024-05-20T18:27:14.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.020+0000 7fbf71ffb640 1 -- 172.21.15.5:0/717973743 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64005d10 con 0x7fbf5c0039e0 2024-05-20T18:27:14.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.021+0000 7fbf792ca640 1 -- 172.21.15.5:0/717973743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5c0039e0 msgr2=0x7fbf5c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:14.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.021+0000 7fbf792ca640 1 --2- 172.21.15.5:0/717973743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5c0039e0 0x7fbf5c003e10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbf6400c4c0 tx=0x7fbf5c005830 comp rx=0 tx=0).stop 2024-05-20T18:27:14.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.021+0000 7fbf737fe640 1 -- 172.21.15.5:0/717973743 reap_dead start 2024-05-20T18:27:14.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.021+0000 7fbf792ca640 1 -- 172.21.15.5:0/717973743 shutdown_connections 2024-05-20T18:27:14.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.021+0000 7fbf792ca640 1 -- 172.21.15.5:0/717973743 >> 172.21.15.5:0/717973743 conn(0x7fbf740d4bd0 msgr2=0x7fbf740d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:14.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.021+0000 7fbf792ca640 1 -- 172.21.15.5:0/717973743 shutdown_connections 2024-05-20T18:27:14.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.021+0000 7fbf792ca640 1 -- 172.21.15.5:0/717973743 wait complete. 2024-05-20T18:27:14.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.023+0000 7fbf792ca640 1 Processor -- start 2024-05-20T18:27:14.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.023+0000 7fbf792ca640 1 -- start start 2024-05-20T18:27:14.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.024+0000 7fbf792ca640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf5c0039e0 0x7fbf7415bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:14.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.024+0000 7fbf792ca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf740cb3d0 0x7fbf740cde50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:14.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.024+0000 7fbf792ca640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf741550e0 0x7fbf740ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.024+0000 7fbf792ca640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbf740d0300 con 0x7fbf740cb3d0 2024-05-20T18:27:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.024+0000 7fbf792ca640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbf740d4260 con 0x7fbf5c0039e0 2024-05-20T18:27:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.024+0000 7fbf792ca640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbf740b4050 con 0x7fbf741550e0 2024-05-20T18:27:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.025+0000 7fbf727fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf740cb3d0 0x7fbf740cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.025+0000 7fbf727fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf740cb3d0 0x7fbf740cde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44746/0 (socket says 172.21.15.5:44746) 2024-05-20T18:27:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.025+0000 7fbf727fc640 1 -- 172.21.15.5:0/1131831899 learned_addr learned my addr 172.21.15.5:0/1131831899 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.025+0000 7fbf72ffd640 1 --2- 172.21.15.5:0/1131831899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf5c0039e0 0x7fbf7415bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.025+0000 7fbf737fe640 1 --2- 172.21.15.5:0/1131831899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf741550e0 0x7fbf740ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.025+0000 7fbf727fc640 1 -- 172.21.15.5:0/1131831899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf741550e0 msgr2=0x7fbf740ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.025+0000 7fbf727fc640 1 --2- 172.21.15.5:0/1131831899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf741550e0 0x7fbf740ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.025+0000 7fbf727fc640 1 -- 172.21.15.5:0/1131831899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf5c0039e0 msgr2=0x7fbf7415bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.025+0000 7fbf727fc640 1 --2- 172.21.15.5:0/1131831899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf5c0039e0 0x7fbf7415bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:14.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.025+0000 7fbf727fc640 1 -- 172.21.15.5:0/1131831899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf741577d0 con 0x7fbf740cb3d0 2024-05-20T18:27:14.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.025+0000 7fbf72ffd640 1 --2- 172.21.15.5:0/1131831899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf5c0039e0 0x7fbf7415bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:14.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.025+0000 7fbf737fe640 1 --2- 172.21.15.5:0/1131831899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf741550e0 0x7fbf740ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:14.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.025+0000 7fbf727fc640 1 --2- 172.21.15.5:0/1131831899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf740cb3d0 0x7fbf740cde50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbf680118e0 tx=0x7fbf68011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:14.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.025+0000 7fbf5b7fe640 1 -- 172.21.15.5:0/1131831899 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6800faf0 con 0x7fbf740cb3d0 2024-05-20T18:27:14.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.025+0000 7fbf5b7fe640 1 -- 172.21.15.5:0/1131831899 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf6800fc90 con 0x7fbf740cb3d0 2024-05-20T18:27:14.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.025+0000 7fbf5b7fe640 1 -- 172.21.15.5:0/1131831899 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf68014440 con 0x7fbf740cb3d0 2024-05-20T18:27:14.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.025+0000 7fbf792ca640 1 -- 172.21.15.5:0/1131831899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf7415c6f0 con 0x7fbf740cb3d0 2024-05-20T18:27:14.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.025+0000 7fbf792ca640 1 -- 172.21.15.5:0/1131831899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf741592c0 con 0x7fbf740cb3d0 2024-05-20T18:27:14.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.026+0000 7fbf792ca640 1 -- 172.21.15.5:0/1131831899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf74075b00 con 0x7fbf740cb3d0 2024-05-20T18:27:14.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.027+0000 7fbf5b7fe640 1 -- 172.21.15.5:0/1131831899 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbf68012040 con 0x7fbf740cb3d0 2024-05-20T18:27:14.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.027+0000 7fbf5b7fe640 1 --2- 172.21.15.5:0/1131831899 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf6006be30 0x7fbf6006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:14.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.028+0000 7fbf5b7fe640 1 -- 172.21.15.5:0/1131831899 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fbf6808b930 con 0x7fbf740cb3d0 2024-05-20T18:27:14.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.028+0000 7fbf72ffd640 1 --2- 172.21.15.5:0/1131831899 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf6006be30 0x7fbf6006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:14.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.028+0000 7fbf72ffd640 1 --2- 172.21.15.5:0/1131831899 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf6006be30 0x7fbf6006e340 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbf6400c4c0 tx=0x7fbf64009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:14.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.032+0000 7fbf5b7fe640 1 -- 172.21.15.5:0/1131831899 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbf68019030 con 0x7fbf740cb3d0 2024-05-20T18:27:14.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.256+0000 7fbf792ca640 1 -- 172.21.15.5:0/1131831899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf7415fae0 con 0x7fbf740cb3d0 2024-05-20T18:27:14.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.257+0000 7fbf5b7fe640 1 -- 172.21.15.5:0/1131831899 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5168 (secure 0 0 0) 0x7fbf6801b300 con 0x7fbf740cb3d0 2024-05-20T18:27:14.256 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:14.257 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2982954341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2982954341},{"type":"v1","addr":"172.21.15.5:6836","nonce":2982954341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4032850001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":4032850001},{"type":"v1","addr":"172.21.15.5:6837","nonce":4032850001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6833/424580378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":424580378},{"type":"v1","addr":"172.21.15.63:6833","nonce":424580378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:27:07.164414+0000","modified":"2024-05-20T18:27:12.208433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/3249423677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3249423677},{"type":"v1","addr":"172.21.15.63:6835","nonce":3249423677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-05-20T18:27:14.257 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 7 2024-05-20T18:27:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.259+0000 7fbf792ca640 1 -- 172.21.15.5:0/1131831899 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf6006be30 msgr2=0x7fbf6006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.259+0000 7fbf792ca640 1 --2- 172.21.15.5:0/1131831899 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf6006be30 0x7fbf6006e340 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbf6400c4c0 tx=0x7fbf64009030 comp rx=0 tx=0).stop 2024-05-20T18:27:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.259+0000 7fbf792ca640 1 -- 172.21.15.5:0/1131831899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf740cb3d0 msgr2=0x7fbf740cde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.259+0000 7fbf792ca640 1 --2- 172.21.15.5:0/1131831899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf740cb3d0 0x7fbf740cde50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbf680118e0 tx=0x7fbf68011da0 comp rx=0 tx=0).stop 2024-05-20T18:27:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.259+0000 7fbf792ca640 1 -- 172.21.15.5:0/1131831899 shutdown_connections 2024-05-20T18:27:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.259+0000 7fbf792ca640 1 --2- 172.21.15.5:0/1131831899 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf6006be30 0x7fbf6006e340 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.259+0000 7fbf792ca640 1 --2- 172.21.15.5:0/1131831899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf741550e0 0x7fbf740ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.259+0000 7fbf792ca640 1 --2- 172.21.15.5:0/1131831899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf740cb3d0 0x7fbf740cde50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.259+0000 7fbf792ca640 1 --2- 172.21.15.5:0/1131831899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf5c0039e0 0x7fbf7415bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.259+0000 7fbf792ca640 1 -- 172.21.15.5:0/1131831899 >> 172.21.15.5:0/1131831899 conn(0x7fbf740d4bd0 msgr2=0x7fbf740b5130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.259+0000 7fbf792ca640 1 -- 172.21.15.5:0/1131831899 shutdown_connections 2024-05-20T18:27:14.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.260+0000 7fbf792ca640 1 -- 172.21.15.5:0/1131831899 wait complete. 2024-05-20T18:27:14.270 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:27:14.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.385+0000 7feeebcca640 1 Processor -- start 2024-05-20T18:27:14.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.385+0000 7feeebcca640 1 -- start start 2024-05-20T18:27:14.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.387+0000 7feeebcca640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feee4163a10 0x7feee40d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:14.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.387+0000 7feeebcca640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7feee40d14b0 0x7feee40d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:14.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.387+0000 7feeebcca640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feee40d0370 con 0x7feee40b4010 2024-05-20T18:27:14.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.387+0000 7feeebcca640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7feee40d3ca0 con 0x7feee40d14b0 2024-05-20T18:27:14.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.387+0000 7feeebcca640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7feee40d0650 con 0x7feee4163a10 2024-05-20T18:27:14.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.387+0000 7feeea267640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7feee40d14b0 0x7feee40d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:14.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.387+0000 7feeea267640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7feee40d14b0 0x7feee40d18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:57106/0 (socket says 172.21.15.5:57106) 2024-05-20T18:27:14.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.387+0000 7feee9265640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7feee40b4010 0x7feee40cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:50684/0 (socket says 172.21.15.5:50684) 2024-05-20T18:27:14.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.387+0000 7feee9a66640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feee4163a10 0x7feee40d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:14.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.387+0000 7feeea267640 1 -- 172.21.15.5:0/433110129 learned_addr learned my addr 172.21.15.5:0/433110129 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:14.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.388+0000 7feee9a66640 1 -- 172.21.15.5:0/433110129 >> v2:172.21.15.63:3300/0 conn(0x7feee40d14b0 msgr2=0x7feee40d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:14.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.388+0000 7feee9a66640 1 --2- 172.21.15.5:0/433110129 >> v2:172.21.15.63:3300/0 conn(0x7feee40d14b0 0x7feee40d18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:14.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.388+0000 7feee9a66640 1 -- 172.21.15.5:0/433110129 >> v1:172.21.15.63:6789/0 conn(0x7feee40b4010 legacy=0x7feee40cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:14.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.388+0000 7feee9a66640 1 -- 172.21.15.5:0/433110129 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee40d3650 con 0x7feee4163a10 2024-05-20T18:27:14.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.388+0000 7feeea267640 1 --2- 172.21.15.5:0/433110129 >> v2:172.21.15.63:3300/0 conn(0x7feee40d14b0 0x7feee40d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:27:14.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.388+0000 7feee9a66640 1 --2- 172.21.15.5:0/433110129 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feee4163a10 0x7feee40d4130 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7feee00097a0 tx=0x7feee002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88e89b691ac5ff54 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:14.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.389+0000 7feee8a64640 1 -- 172.21.15.5:0/433110129 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee0037070 con 0x7feee4163a10 2024-05-20T18:27:14.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.389+0000 7feee8a64640 1 -- 172.21.15.5:0/433110129 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feee0032440 con 0x7feee4163a10 2024-05-20T18:27:14.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.389+0000 7feee8a64640 1 -- 172.21.15.5:0/433110129 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee0032810 con 0x7feee4163a10 2024-05-20T18:27:14.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.389+0000 7feeebcca640 1 -- 172.21.15.5:0/433110129 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feee4163a10 msgr2=0x7feee40d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:14.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.389+0000 7feeebcca640 1 --2- 172.21.15.5:0/433110129 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feee4163a10 0x7feee40d4130 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7feee00097a0 tx=0x7feee002a180 comp rx=0 tx=0).stop 2024-05-20T18:27:14.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.389+0000 7feeebcca640 1 -- 172.21.15.5:0/433110129 shutdown_connections 2024-05-20T18:27:14.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.389+0000 7feeebcca640 1 --2- 172.21.15.5:0/433110129 >> v2:172.21.15.63:3300/0 conn(0x7feee40d14b0 0x7feee40d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:14.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.389+0000 7feeebcca640 1 --2- 172.21.15.5:0/433110129 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feee4163a10 0x7feee40d4130 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:14.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.389+0000 7feeebcca640 1 -- 172.21.15.5:0/433110129 >> 172.21.15.5:0/433110129 conn(0x7feee406d240 msgr2=0x7feee406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:14.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.389+0000 7feeebcca640 1 -- 172.21.15.5:0/433110129 shutdown_connections 2024-05-20T18:27:14.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.389+0000 7feeebcca640 1 -- 172.21.15.5:0/433110129 wait complete. 2024-05-20T18:27:14.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.391+0000 7feeebcca640 1 Processor -- start 2024-05-20T18:27:14.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.391+0000 7feeebcca640 1 -- start start 2024-05-20T18:27:14.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.392+0000 7feeebcca640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feee40b4010 0x7feee4155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:14.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.393+0000 7feeebcca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feee40d14b0 0x7feee41554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:14.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.393+0000 7feeebcca640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feee4163a10 0x7feee41580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:14.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.393+0000 7feeebcca640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feee40b8210 con 0x7feee40d14b0 2024-05-20T18:27:14.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.393+0000 7feeebcca640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feee40d1e00 con 0x7feee40b4010 2024-05-20T18:27:14.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.393+0000 7feeebcca640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7feee40b8390 con 0x7feee4163a10 2024-05-20T18:27:14.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.393+0000 7feee9265640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feee40d14b0 0x7feee41554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:14.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.393+0000 7feee9265640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feee40d14b0 0x7feee41554b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44750/0 (socket says 172.21.15.5:44750) 2024-05-20T18:27:14.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.393+0000 7feee9265640 1 -- 172.21.15.5:0/3152719164 learned_addr learned my addr 172.21.15.5:0/3152719164 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:14.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.393+0000 7feeea267640 1 --2- 172.21.15.5:0/3152719164 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feee4163a10 0x7feee41580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:14.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.393+0000 7feee9a66640 1 --2- 172.21.15.5:0/3152719164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feee40b4010 0x7feee4155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:14.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.393+0000 7feee9265640 1 -- 172.21.15.5:0/3152719164 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feee4163a10 msgr2=0x7feee41580b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:14.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.393+0000 7feee9265640 1 --2- 172.21.15.5:0/3152719164 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feee4163a10 0x7feee41580b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:14.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.393+0000 7feee9265640 1 -- 172.21.15.5:0/3152719164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feee40b4010 msgr2=0x7feee4155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:14.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.393+0000 7feee9265640 1 --2- 172.21.15.5:0/3152719164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feee40b4010 0x7feee4155e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:14.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.393+0000 7feee9265640 1 -- 172.21.15.5:0/3152719164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee4158d10 con 0x7feee40d14b0 2024-05-20T18:27:14.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.393+0000 7feee9265640 1 --2- 172.21.15.5:0/3152719164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feee40d14b0 0x7feee41554b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7feed400c390 tx=0x7feed400c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:14.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.393+0000 7feeda7fc640 1 -- 172.21.15.5:0/3152719164 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed40074e0 con 0x7feee40d14b0 2024-05-20T18:27:14.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.393+0000 7feeda7fc640 1 -- 172.21.15.5:0/3152719164 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feed40047e0 con 0x7feee40d14b0 2024-05-20T18:27:14.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.393+0000 7feeebcca640 1 -- 172.21.15.5:0/3152719164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee4158ff0 con 0x7feee40d14b0 2024-05-20T18:27:14.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.393+0000 7feeda7fc640 1 -- 172.21.15.5:0/3152719164 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed4004bd0 con 0x7feee40d14b0 2024-05-20T18:27:14.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.394+0000 7feeebcca640 1 -- 172.21.15.5:0/3152719164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee415bdc0 con 0x7feee40d14b0 2024-05-20T18:27:14.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.394+0000 7feeebcca640 1 -- 172.21.15.5:0/3152719164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feee41591c0 con 0x7feee40d14b0 2024-05-20T18:27:14.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.395+0000 7feeda7fc640 1 -- 172.21.15.5:0/3152719164 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feed401a020 con 0x7feee40d14b0 2024-05-20T18:27:14.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.396+0000 7feeda7fc640 1 --2- 172.21.15.5:0/3152719164 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feec806be30 0x7feec806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:14.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.396+0000 7feeda7fc640 1 -- 172.21.15.5:0/3152719164 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7feed408a770 con 0x7feee40d14b0 2024-05-20T18:27:14.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.396+0000 7feee9a66640 1 --2- 172.21.15.5:0/3152719164 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feec806be30 0x7feec806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:14.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.396+0000 7feee9a66640 1 --2- 172.21.15.5:0/3152719164 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feec806be30 0x7feec806e340 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7feee0034000 tx=0x7feee002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:14.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.400+0000 7feeda7fc640 1 -- 172.21.15.5:0/3152719164 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7feed4058700 con 0x7feee40d14b0 2024-05-20T18:27:14.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.583+0000 7feeebcca640 1 -- 172.21.15.5:0/3152719164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feee4075b00 con 0x7feee40d14b0 2024-05-20T18:27:14.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.584+0000 7feeda7fc640 1 -- 172.21.15.5:0/3152719164 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18394 (secure 0 0 0) 0x7feed405ca00 con 0x7feee40d14b0 2024-05-20T18:27:14.584 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:14.584 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:27:13.817633+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:27:05.476596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T18:27:06.137762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T18:27:09.829920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:27:14.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.586+0000 7feeebcca640 1 -- 172.21.15.5:0/3152719164 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feec806be30 msgr2=0x7feec806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:14.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.586+0000 7feeebcca640 1 --2- 172.21.15.5:0/3152719164 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feec806be30 0x7feec806e340 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7feee0034000 tx=0x7feee002e000 comp rx=0 tx=0).stop 2024-05-20T18:27:14.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.586+0000 7feeebcca640 1 -- 172.21.15.5:0/3152719164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feee40d14b0 msgr2=0x7feee41554b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:14.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.586+0000 7feeebcca640 1 --2- 172.21.15.5:0/3152719164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feee40d14b0 0x7feee41554b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7feed400c390 tx=0x7feed400c790 comp rx=0 tx=0).stop 2024-05-20T18:27:14.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.586+0000 7feeebcca640 1 -- 172.21.15.5:0/3152719164 shutdown_connections 2024-05-20T18:27:14.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.586+0000 7feeebcca640 1 --2- 172.21.15.5:0/3152719164 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feec806be30 0x7feec806e340 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:14.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.586+0000 7feeebcca640 1 --2- 172.21.15.5:0/3152719164 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feee4163a10 0x7feee41580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:14.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.586+0000 7feeebcca640 1 --2- 172.21.15.5:0/3152719164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feee40d14b0 0x7feee41554b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:14.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.586+0000 7feeebcca640 1 --2- 172.21.15.5:0/3152719164 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feee40b4010 0x7feee4155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:14.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.586+0000 7feeebcca640 1 -- 172.21.15.5:0/3152719164 >> 172.21.15.5:0/3152719164 conn(0x7feee406d240 msgr2=0x7feee415f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:14.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.586+0000 7feeebcca640 1 -- 172.21.15.5:0/3152719164 shutdown_connections 2024-05-20T18:27:14.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:14.586+0000 7feeebcca640 1 -- 172.21.15.5:0/3152719164 wait complete. 2024-05-20T18:27:14.999 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T18:27:07.164414+0000', 'modified': '2024-05-20T18:27:12.208433+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4279}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4279': {'gid': 4279, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.63:6835/3249423677', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6834', 'nonce': 3249423677}, {'type': 'v1', 'addr': '172.21.15.63:6835', 'nonce': 3249423677}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4279, 'qdb_cluster': [4279]} 2024-05-20T18:27:14.999 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-20T18:27:15.000 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-20T18:27:15.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.118+0000 7eff212fe640 1 Processor -- start 2024-05-20T18:27:15.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.118+0000 7eff212fe640 1 -- start start 2024-05-20T18:27:15.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.120+0000 7eff212fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7eff1c15f220 0x7eff1c0d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:15.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.120+0000 7eff212fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7eff1c0cfae0 0x7eff1c0ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:15.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.120+0000 7eff212fe640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff1c0cda20 con 0x7eff1c0d0e00 2024-05-20T18:27:15.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.120+0000 7eff212fe640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7eff1c0cdd00 con 0x7eff1c0cfae0 2024-05-20T18:27:15.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.120+0000 7eff212fe640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7eff1c0cdfe0 con 0x7eff1c15f220 2024-05-20T18:27:15.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.120+0000 7eff1b7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7eff1c0cfae0 0x7eff1c0ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:15.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.120+0000 7eff1a7fc640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7eff1c0d0e00 0x7eff1c0cf3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36474/0 (socket says 172.21.15.5:36474) 2024-05-20T18:27:15.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.120+0000 7eff1a7fc640 1 -- 172.21.15.5:0/947564402 learned_addr learned my addr 172.21.15.5:0/947564402 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:15.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.120+0000 7eff1affd640 1 --2- 172.21.15.5:0/947564402 >> v2:172.21.15.63:3300/0 conn(0x7eff1c15f220 0x7eff1c0d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:15.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.120+0000 7eff1b7fe640 1 -- 172.21.15.5:0/947564402 >> v2:172.21.15.63:3300/0 conn(0x7eff1c15f220 msgr2=0x7eff1c0d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:15.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.120+0000 7eff1b7fe640 1 --2- 172.21.15.5:0/947564402 >> v2:172.21.15.63:3300/0 conn(0x7eff1c15f220 0x7eff1c0d08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.120+0000 7eff1b7fe640 1 -- 172.21.15.5:0/947564402 >> v1:172.21.15.5:6789/0 conn(0x7eff1c0d0e00 legacy=0x7eff1c0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:15.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.120+0000 7eff1b7fe640 1 -- 172.21.15.5:0/947564402 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1c0ce2c0 con 0x7eff1c0cfae0 2024-05-20T18:27:15.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.120+0000 7eff19ffb640 1 -- 172.21.15.5:0/947564402 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 586412084 0 0) 0x7eff1c0cda20 con 0x7eff1c0d0e00 2024-05-20T18:27:15.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.121+0000 7eff1b7fe640 1 --2- 172.21.15.5:0/947564402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff1c0cfae0 0x7eff1c0ce7e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7eff1c06e810 tx=0x7eff1402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5397fb62effddb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:15.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.121+0000 7eff19ffb640 1 -- 172.21.15.5:0/947564402 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff14007740 con 0x7eff1c0cfae0 2024-05-20T18:27:15.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.121+0000 7eff19ffb640 1 -- 172.21.15.5:0/947564402 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff140078e0 con 0x7eff1c0cfae0 2024-05-20T18:27:15.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.121+0000 7eff19ffb640 1 -- 172.21.15.5:0/947564402 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff14007cb0 con 0x7eff1c0cfae0 2024-05-20T18:27:15.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.122+0000 7eff212fe640 1 -- 172.21.15.5:0/947564402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff1c0cfae0 msgr2=0x7eff1c0ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:15.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.122+0000 7eff212fe640 1 --2- 172.21.15.5:0/947564402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff1c0cfae0 0x7eff1c0ce7e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7eff1c06e810 tx=0x7eff1402a5f0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.122+0000 7eff212fe640 1 -- 172.21.15.5:0/947564402 shutdown_connections 2024-05-20T18:27:15.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.122+0000 7eff212fe640 1 --2- 172.21.15.5:0/947564402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff1c0cfae0 0x7eff1c0ce7e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.122+0000 7eff212fe640 1 --2- 172.21.15.5:0/947564402 >> v2:172.21.15.63:3300/0 conn(0x7eff1c15f220 0x7eff1c0d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.122+0000 7eff212fe640 1 -- 172.21.15.5:0/947564402 >> 172.21.15.5:0/947564402 conn(0x7eff1c0b44d0 msgr2=0x7eff1c0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:15.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.122+0000 7eff212fe640 1 -- 172.21.15.5:0/947564402 shutdown_connections 2024-05-20T18:27:15.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.122+0000 7eff212fe640 1 -- 172.21.15.5:0/947564402 wait complete. 2024-05-20T18:27:15.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.124+0000 7eff212fe640 1 Processor -- start 2024-05-20T18:27:15.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.124+0000 7eff212fe640 1 -- start start 2024-05-20T18:27:15.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.125+0000 7eff212fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff1c0d0e00 0x7eff1c160340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:15.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.125+0000 7eff212fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff1c15f220 0x7eff1c0b7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:15.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.125+0000 7eff212fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7eff1c160880 0x7eff1c0b7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:15.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.125+0000 7eff212fe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c15f7b0 con 0x7eff1c0d0e00 2024-05-20T18:27:15.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.125+0000 7eff212fe640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c15f630 con 0x7eff1c15f220 2024-05-20T18:27:15.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.125+0000 7eff212fe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7eff1c0ced20 con 0x7eff1c160880 2024-05-20T18:27:15.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.125+0000 7eff1affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff1c0d0e00 0x7eff1c160340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:15.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.125+0000 7eff1affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff1c0d0e00 0x7eff1c160340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54408/0 (socket says 172.21.15.5:54408) 2024-05-20T18:27:15.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.125+0000 7eff1affd640 1 -- 172.21.15.5:0/1589400022 learned_addr learned my addr 172.21.15.5:0/1589400022 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:15.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.126+0000 7eff1affd640 1 -- 172.21.15.5:0/1589400022 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7eff1c160880 msgr2=0x7eff1c0b7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:15.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.126+0000 7eff1a7fc640 1 --2- 172.21.15.5:0/1589400022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff1c15f220 0x7eff1c0b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:15.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.126+0000 7eff1affd640 1 --2- 172.21.15.5:0/1589400022 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7eff1c160880 0x7eff1c0b7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.126+0000 7eff1affd640 1 -- 172.21.15.5:0/1589400022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff1c15f220 msgr2=0x7eff1c0b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:15.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.126+0000 7eff1affd640 1 --2- 172.21.15.5:0/1589400022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff1c15f220 0x7eff1c0b7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.126+0000 7eff1affd640 1 -- 172.21.15.5:0/1589400022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1c161950 con 0x7eff1c0d0e00 2024-05-20T18:27:15.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.126+0000 7eff1affd640 1 --2- 172.21.15.5:0/1589400022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff1c0d0e00 0x7eff1c160340 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00d5d0 tx=0x7eff0c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:15.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.126+0000 7eff037fe640 1 -- 172.21.15.5:0/1589400022 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c004240 con 0x7eff1c0d0e00 2024-05-20T18:27:15.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.126+0000 7eff037fe640 1 -- 172.21.15.5:0/1589400022 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff0c0043e0 con 0x7eff1c0d0e00 2024-05-20T18:27:15.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.126+0000 7eff212fe640 1 -- 172.21.15.5:0/1589400022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff1c0b7000 con 0x7eff1c0d0e00 2024-05-20T18:27:15.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.126+0000 7eff037fe640 1 -- 172.21.15.5:0/1589400022 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c003830 con 0x7eff1c0d0e00 2024-05-20T18:27:15.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.126+0000 7eff212fe640 1 -- 172.21.15.5:0/1589400022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff1c161670 con 0x7eff1c0d0e00 2024-05-20T18:27:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.128+0000 7eff037fe640 1 -- 172.21.15.5:0/1589400022 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7eff0c0039d0 con 0x7eff1c0d0e00 2024-05-20T18:27:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.128+0000 7eff037fe640 1 --2- 172.21.15.5:0/1589400022 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7eff0806be30 0x7eff0806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.128+0000 7eff037fe640 1 -- 172.21.15.5:0/1589400022 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7eff0c08b4f0 con 0x7eff1c0d0e00 2024-05-20T18:27:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.128+0000 7eff1a7fc640 1 --2- 172.21.15.5:0/1589400022 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7eff0806be30 0x7eff0806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.129+0000 7eff1a7fc640 1 --2- 172.21.15.5:0/1589400022 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7eff0806be30 0x7eff0806e340 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0b4f10 tx=0x7eff10009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.129+0000 7eff281a6740 1 Processor -- start 2024-05-20T18:27:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.130+0000 7eff281a6740 1 -- start start 2024-05-20T18:27:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.131+0000 7eff281a6740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ee476270 0x55f8ee841700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.131+0000 7eff281a6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8ee44d3b0 0x55f8ee83e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.131+0000 7eff281a6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8ee455590 0x55f8ee817360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.131+0000 7eff281a6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f8ee77a4d0 con 0x55f8ee455590 2024-05-20T18:27:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.131+0000 7eff281a6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f8ee7223e0 con 0x55f8ee44d3b0 2024-05-20T18:27:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.131+0000 7eff281a6740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55f8ee776490 con 0x55f8ee476270 2024-05-20T18:27:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.131+0000 7eff1b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8ee455590 0x55f8ee817360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.131+0000 7eff1b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8ee455590 0x55f8ee817360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54424/0 (socket says 172.21.15.5:54424) 2024-05-20T18:27:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.131+0000 7eff1b7fe640 1 -- 172.21.15.5:0/713725000 learned_addr learned my addr 172.21.15.5:0/713725000 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.131+0000 7eff1affd640 1 --2- 172.21.15.5:0/713725000 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ee476270 0x55f8ee841700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.131+0000 7eff1b7fe640 1 -- 172.21.15.5:0/713725000 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ee476270 msgr2=0x55f8ee841700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.131+0000 7eff1b7fe640 1 --2- 172.21.15.5:0/713725000 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ee476270 0x55f8ee841700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.131+0000 7eff1b7fe640 1 -- 172.21.15.5:0/713725000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8ee44d3b0 msgr2=0x55f8ee83e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.131+0000 7eff1a7fc640 1 --2- 172.21.15.5:0/713725000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8ee44d3b0 0x55f8ee83e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.131+0000 7eff1b7fe640 1 --2- 172.21.15.5:0/713725000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8ee44d3b0 0x55f8ee83e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.131+0000 7eff1b7fe640 1 -- 172.21.15.5:0/713725000 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8ee48e2d0 con 0x55f8ee455590 2024-05-20T18:27:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.131+0000 7eff1a7fc640 1 --2- 172.21.15.5:0/713725000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8ee44d3b0 0x55f8ee83e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.132+0000 7eff1b7fe640 1 --2- 172.21.15.5:0/713725000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8ee455590 0x55f8ee817360 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7eff14009e50 tx=0x7eff1402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91d2e687131aec7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.132+0000 7eff00ff9640 1 -- 172.21.15.5:0/713725000 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1403f070 con 0x55f8ee455590 2024-05-20T18:27:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.132+0000 7eff00ff9640 1 -- 172.21.15.5:0/713725000 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff14007740 con 0x55f8ee455590 2024-05-20T18:27:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.132+0000 7eff00ff9640 1 -- 172.21.15.5:0/713725000 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff14007af0 con 0x55f8ee455590 2024-05-20T18:27:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.133+0000 7eff281a6740 1 -- 172.21.15.5:0/713725000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8ee455590 msgr2=0x55f8ee817360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.133+0000 7eff281a6740 1 --2- 172.21.15.5:0/713725000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8ee455590 0x55f8ee817360 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7eff14009e50 tx=0x7eff1402c000 comp rx=0 tx=0).stop 2024-05-20T18:27:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.133+0000 7eff281a6740 1 -- 172.21.15.5:0/713725000 shutdown_connections 2024-05-20T18:27:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.133+0000 7eff281a6740 1 --2- 172.21.15.5:0/713725000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8ee455590 0x55f8ee817360 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.133+0000 7eff281a6740 1 --2- 172.21.15.5:0/713725000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8ee44d3b0 0x55f8ee83e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.133+0000 7eff281a6740 1 --2- 172.21.15.5:0/713725000 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ee476270 0x55f8ee841700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.133+0000 7eff281a6740 1 -- 172.21.15.5:0/713725000 >> 172.21.15.5:0/713725000 conn(0x55f8ee46a3c0 msgr2=0x55f8ee816af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.133+0000 7eff281a6740 1 -- 172.21.15.5:0/713725000 shutdown_connections 2024-05-20T18:27:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.133+0000 7eff281a6740 1 -- 172.21.15.5:0/713725000 wait complete. 2024-05-20T18:27:15.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.134+0000 7eff281a6740 1 -- start start 2024-05-20T18:27:15.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.134+0000 7eff281a6740 1 Processor -- start 2024-05-20T18:27:15.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.135+0000 7eff281a6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8ee44d3b0 0x55f8ee8389a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.135+0000 7eff281a6740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ee455590 0x55f8ee8425b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.135+0000 7eff281a6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8ee476270 0x55f8ee842af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.135+0000 7eff281a6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f8ee7358e0 con 0x55f8ee44d3b0 2024-05-20T18:27:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.135+0000 7eff281a6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f8ee85b150 con 0x55f8ee476270 2024-05-20T18:27:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.135+0000 7eff281a6740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55f8ee83ac60 con 0x55f8ee455590 2024-05-20T18:27:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.135+0000 7eff1affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8ee44d3b0 0x55f8ee8389a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.135+0000 7eff281a6740 10 client.0 _ll_register_callbacks cb 0x55f8ee7029c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:27:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.135+0000 7eff1affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8ee44d3b0 0x55f8ee8389a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54438/0 (socket says 172.21.15.5:54438) 2024-05-20T18:27:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.135+0000 7eff1affd640 1 -- 172.21.15.5:0/59263828 learned_addr learned my addr 172.21.15.5:0/59263828 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.135+0000 7eff1b7fe640 1 --2- 172.21.15.5:0/59263828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8ee476270 0x55f8ee842af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.135+0000 7eff1affd640 1 -- 172.21.15.5:0/59263828 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ee455590 msgr2=0x55f8ee8425b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.135+0000 7eff1a7fc640 1 --2- 172.21.15.5:0/59263828 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ee455590 0x55f8ee8425b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.135+0000 7eff1affd640 1 --2- 172.21.15.5:0/59263828 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ee455590 0x55f8ee8425b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.135+0000 7eff1affd640 1 -- 172.21.15.5:0/59263828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8ee476270 msgr2=0x55f8ee842af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.135+0000 7eff1affd640 1 --2- 172.21.15.5:0/59263828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8ee476270 0x55f8ee842af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.136+0000 7eff1affd640 1 -- 172.21.15.5:0/59263828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8ee5586b0 con 0x55f8ee44d3b0 2024-05-20T18:27:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.136+0000 7eff1b7fe640 1 --2- 172.21.15.5:0/59263828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8ee476270 0x55f8ee842af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.136+0000 7eff1affd640 1 --2- 172.21.15.5:0/59263828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8ee44d3b0 0x55f8ee8389a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7eff0c061b10 tx=0x7eff0c059bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4559935ee24f428c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.136+0000 7efef9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:27:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.136+0000 7efef9ffb640 1 -- 172.21.15.5:0/59263828 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c071030 con 0x55f8ee44d3b0 2024-05-20T18:27:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.136+0000 7efef9ffb640 1 -- 172.21.15.5:0/59263828 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7eff0c003cc0 con 0x55f8ee44d3b0 2024-05-20T18:27:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.136+0000 7eff281a6740 1 -- 172.21.15.5:0/59263828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff14007740 con 0x55f8ee44d3b0 2024-05-20T18:27:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.136+0000 7efef9ffb640 1 -- 172.21.15.5:0/59263828 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7eff0c062040 con 0x55f8ee44d3b0 2024-05-20T18:27:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.136+0000 7eff281a6740 10 client.4398 fetch_fsmap learned FSMap version 7 2024-05-20T18:27:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.136+0000 7eff281a6740 1 -- 172.21.15.5:0/59263828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55f8ee85ae90 con 0x55f8ee44d3b0 2024-05-20T18:27:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.136+0000 7efef9ffb640 1 -- 172.21.15.5:0/59263828 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff14007740 con 0x55f8ee44d3b0 2024-05-20T18:27:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.136+0000 7efef9ffb640 1 -- 172.21.15.5:0/59263828 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7eff0c013070 con 0x55f8ee44d3b0 2024-05-20T18:27:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.136+0000 7eff281a6740 10 client.4398 fetch_fsmap finished waiting for FSMap version 7 2024-05-20T18:27:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.136+0000 7eff281a6740 10 client.4398 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-20T18:27:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.136+0000 7eff281a6740 20 client.4398 populate_metadata read hostname 'smithi005' 2024-05-20T18:27:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.136+0000 7eff281a6740 1 --2- 172.21.15.5:0/59263828 >> [v2:172.21.15.63:6834/3249423677,v1:172.21.15.63:6835/3249423677] conn(0x55f8ee434e40 0x55f8ee879300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.136+0000 7eff281a6740 4 client.4398 mds_command: new command op to 4279 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:27:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.136+0000 7eff281a6740 1 -- 172.21.15.5:0/59263828 --> [v2:172.21.15.63:6834/3249423677,v1:172.21.15.63:6835/3249423677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8ee787c00 con 0x55f8ee434e40 2024-05-20T18:27:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.137+0000 7eff1a7fc640 1 --2- 172.21.15.5:0/59263828 >> [v2:172.21.15.63:6834/3249423677,v1:172.21.15.63:6835/3249423677] conn(0x55f8ee434e40 0x55f8ee879300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.137+0000 7eff1a7fc640 1 --2- 172.21.15.5:0/59263828 >> [v2:172.21.15.63:6834/3249423677,v1:172.21.15.63:6835/3249423677] conn(0x55f8ee434e40 0x55f8ee879300 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5379511135c351c server_cookie=336d2bb2baebf2cb in_seq=0 out_seq=0 2024-05-20T18:27:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.137+0000 7efef9ffb640 10 client.4398 ms_handle_connect on v2:172.21.15.63:6834/3249423677 2024-05-20T18:27:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.138+0000 7efef9ffb640 1 -- 172.21.15.5:0/59263828 <== mds.0 v2:172.21.15.63:6834/3249423677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f8ee787c00 con 0x55f8ee434e40 2024-05-20T18:27:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.138+0000 7efef9ffb640 10 client.4398 handle_command_reply: tid=0 2024-05-20T18:27:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.138+0000 7eff281a6740 1 client.4398 shutdown 2024-05-20T18:27:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.138+0000 7eff281a6740 1 -- 172.21.15.5:0/59263828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8ee44d3b0 msgr2=0x55f8ee8389a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.138+0000 7eff281a6740 1 --2- 172.21.15.5:0/59263828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8ee44d3b0 0x55f8ee8389a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7eff0c061b10 tx=0x7eff0c059bd0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.138+0000 7eff281a6740 1 -- 172.21.15.5:0/59263828 shutdown_connections 2024-05-20T18:27:15.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.138+0000 7eff281a6740 1 --2- 172.21.15.5:0/59263828 >> [v2:172.21.15.63:6834/3249423677,v1:172.21.15.63:6835/3249423677] conn(0x55f8ee434e40 0x55f8ee879300 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.138+0000 7eff281a6740 1 --2- 172.21.15.5:0/59263828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8ee476270 0x55f8ee842af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.138+0000 7eff281a6740 1 --2- 172.21.15.5:0/59263828 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ee455590 0x55f8ee8425b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.138+0000 7efef9ffb640 0 client.4398 ms_handle_reset on v2:172.21.15.63:6834/3249423677 2024-05-20T18:27:15.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.138+0000 7eff281a6740 1 --2- 172.21.15.5:0/59263828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8ee44d3b0 0x55f8ee8389a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.138+0000 7eff281a6740 1 -- 172.21.15.5:0/59263828 >> 172.21.15.5:0/59263828 conn(0x55f8ee46a3c0 msgr2=0x55f8ee810e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:15.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.138+0000 7eff281a6740 1 -- 172.21.15.5:0/59263828 shutdown_connections 2024-05-20T18:27:15.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.138+0000 7eff281a6740 1 -- 172.21.15.5:0/59263828 wait complete. 2024-05-20T18:27:15.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.138+0000 7eff281a6740 20 client.4398 trim_cache size 0 max 16384 2024-05-20T18:27:15.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.157+0000 7eff281a6740 1 Processor -- start 2024-05-20T18:27:15.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.157+0000 7eff281a6740 1 -- start start 2024-05-20T18:27:15.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.158+0000 7eff281a6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8ee44d3b0 0x55f8ee8521f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:15.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.158+0000 7eff281a6740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ee455590 0x55f8ee852750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:15.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.158+0000 7eff281a6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8ee476270 0x55f8ee8d3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:15.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.158+0000 7eff281a6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c202100 con 0x55f8ee44d3b0 2024-05-20T18:27:15.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.158+0000 7eff281a6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c201460 con 0x55f8ee476270 2024-05-20T18:27:15.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.158+0000 7eff281a6740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7eff1c201ab0 con 0x55f8ee455590 2024-05-20T18:27:15.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.159+0000 7eff1affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8ee44d3b0 0x55f8ee8521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:15.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.159+0000 7eff1affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8ee44d3b0 0x55f8ee8521f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54454/0 (socket says 172.21.15.5:54454) 2024-05-20T18:27:15.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.159+0000 7eff1affd640 1 -- 172.21.15.5:0/2303013916 learned_addr learned my addr 172.21.15.5:0/2303013916 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:15.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.159+0000 7eff1b7fe640 1 --2- 172.21.15.5:0/2303013916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8ee476270 0x55f8ee8d3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:15.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.159+0000 7eff1a7fc640 1 --2- 172.21.15.5:0/2303013916 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ee455590 0x55f8ee852750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:15.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.159+0000 7eff1affd640 1 -- 172.21.15.5:0/2303013916 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ee455590 msgr2=0x55f8ee852750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:15.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.159+0000 7eff1affd640 1 --2- 172.21.15.5:0/2303013916 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ee455590 0x55f8ee852750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.159+0000 7eff1affd640 1 -- 172.21.15.5:0/2303013916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8ee476270 msgr2=0x55f8ee8d3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:15.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.159+0000 7eff1affd640 1 --2- 172.21.15.5:0/2303013916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8ee476270 0x55f8ee8d3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.159+0000 7eff1affd640 1 -- 172.21.15.5:0/2303013916 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8ee842e50 con 0x55f8ee44d3b0 2024-05-20T18:27:15.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.159+0000 7eff1b7fe640 1 --2- 172.21.15.5:0/2303013916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8ee476270 0x55f8ee8d3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:15.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.159+0000 7eff1a7fc640 1 --2- 172.21.15.5:0/2303013916 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ee455590 0x55f8ee852750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:15.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.159+0000 7eff1affd640 1 --2- 172.21.15.5:0/2303013916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8ee44d3b0 0x55f8ee8521f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7eff0c08b810 tx=0x7eff0c0557a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47564ea9c95e12b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:15.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.159+0000 7efefbfff640 1 -- 172.21.15.5:0/2303013916 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c071040 con 0x55f8ee44d3b0 2024-05-20T18:27:15.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.159+0000 7efefbfff640 1 -- 172.21.15.5:0/2303013916 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff0c003cc0 con 0x55f8ee44d3b0 2024-05-20T18:27:15.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.159+0000 7efefbfff640 1 -- 172.21.15.5:0/2303013916 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c078340 con 0x55f8ee44d3b0 2024-05-20T18:27:15.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.160+0000 7eff281a6740 1 -- 172.21.15.5:0/2303013916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8ee44d3b0 msgr2=0x55f8ee8521f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:15.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.160+0000 7eff281a6740 1 --2- 172.21.15.5:0/2303013916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8ee44d3b0 0x55f8ee8521f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7eff0c08b810 tx=0x7eff0c0557a0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.160+0000 7eff281a6740 1 -- 172.21.15.5:0/2303013916 shutdown_connections 2024-05-20T18:27:15.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.160+0000 7eff281a6740 1 --2- 172.21.15.5:0/2303013916 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8ee476270 0x55f8ee8d3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.160+0000 7eff281a6740 1 --2- 172.21.15.5:0/2303013916 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ee455590 0x55f8ee852750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.160+0000 7eff281a6740 1 --2- 172.21.15.5:0/2303013916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8ee44d3b0 0x55f8ee8521f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.160+0000 7eff281a6740 1 -- 172.21.15.5:0/2303013916 >> 172.21.15.5:0/2303013916 conn(0x55f8ee46a3c0 msgr2=0x55f8ee83fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:15.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.160+0000 7eff281a6740 1 -- 172.21.15.5:0/2303013916 shutdown_connections 2024-05-20T18:27:15.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.160+0000 7eff281a6740 1 -- 172.21.15.5:0/2303013916 wait complete. 2024-05-20T18:27:15.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.161+0000 7eff281a6740 1 -- start start 2024-05-20T18:27:15.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.161+0000 7eff281a6740 1 Processor -- start 2024-05-20T18:27:15.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.162+0000 7eff281a6740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ee44d3b0 0x55f8ee862f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:15.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.162+0000 7eff281a6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8ee455590 0x55f8ee837a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:15.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.162+0000 7eff281a6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8ee476270 0x55f8ee8751d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:15.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.162+0000 7eff281a6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f8ee83a580 con 0x55f8ee476270 2024-05-20T18:27:15.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.162+0000 7eff281a6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f8ee83a100 con 0x55f8ee455590 2024-05-20T18:27:15.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.162+0000 7eff281a6740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55f8ee840b80 con 0x55f8ee44d3b0 2024-05-20T18:27:15.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.162+0000 7eff281a6740 10 client.0 _ll_register_callbacks cb 0x7eff0c006e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:27:15.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.162+0000 7eff1b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8ee476270 0x55f8ee8751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:15.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.162+0000 7eff1b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8ee476270 0x55f8ee8751d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54460/0 (socket says 172.21.15.5:54460) 2024-05-20T18:27:15.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.162+0000 7eff1b7fe640 1 -- 172.21.15.5:0/3141278887 learned_addr learned my addr 172.21.15.5:0/3141278887 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:15.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.162+0000 7eff1a7fc640 1 --2- 172.21.15.5:0/3141278887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8ee455590 0x55f8ee837a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:15.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.162+0000 7eff1b7fe640 1 -- 172.21.15.5:0/3141278887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ee44d3b0 msgr2=0x55f8ee862f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:15.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.162+0000 7eff1affd640 1 --2- 172.21.15.5:0/3141278887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ee44d3b0 0x55f8ee862f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:15.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.163+0000 7eff1b7fe640 1 --2- 172.21.15.5:0/3141278887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ee44d3b0 0x55f8ee862f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.163+0000 7eff1b7fe640 1 -- 172.21.15.5:0/3141278887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8ee455590 msgr2=0x55f8ee837a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:15.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.163+0000 7eff1b7fe640 1 --2- 172.21.15.5:0/3141278887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8ee455590 0x55f8ee837a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.163+0000 7eff1b7fe640 1 -- 172.21.15.5:0/3141278887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8ee8cfce0 con 0x55f8ee476270 2024-05-20T18:27:15.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.163+0000 7eff1a7fc640 1 --2- 172.21.15.5:0/3141278887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8ee455590 0x55f8ee837a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:15.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.163+0000 7eff1b7fe640 1 --2- 172.21.15.5:0/3141278887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8ee476270 0x55f8ee8751d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7eff14000f80 tx=0x7eff14004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94f37fbfa8eddb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:15.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.163+0000 7efef9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:27:15.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.163+0000 7efef9ffb640 1 -- 172.21.15.5:0/3141278887 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff14051060 con 0x55f8ee476270 2024-05-20T18:27:15.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.163+0000 7efef9ffb640 1 -- 172.21.15.5:0/3141278887 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7eff14004500 con 0x55f8ee476270 2024-05-20T18:27:15.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.163+0000 7eff281a6740 1 -- 172.21.15.5:0/3141278887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff0c003cc0 con 0x55f8ee476270 2024-05-20T18:27:15.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.163+0000 7efef9ffb640 1 -- 172.21.15.5:0/3141278887 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7eff14057050 con 0x55f8ee476270 2024-05-20T18:27:15.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.163+0000 7eff281a6740 10 client.4404 fetch_fsmap learned FSMap version 7 2024-05-20T18:27:15.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.163+0000 7eff281a6740 1 -- 172.21.15.5:0/3141278887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55f8ee8d05c0 con 0x55f8ee476270 2024-05-20T18:27:15.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.163+0000 7efef9ffb640 1 -- 172.21.15.5:0/3141278887 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff0c003cc0 con 0x55f8ee476270 2024-05-20T18:27:15.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.163+0000 7efef9ffb640 1 -- 172.21.15.5:0/3141278887 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7eff1405a020 con 0x55f8ee476270 2024-05-20T18:27:15.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.163+0000 7eff281a6740 10 client.4404 fetch_fsmap finished waiting for FSMap version 7 2024-05-20T18:27:15.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.163+0000 7eff281a6740 10 client.4404 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-20T18:27:15.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.163+0000 7eff281a6740 20 client.4404 populate_metadata read hostname 'smithi005' 2024-05-20T18:27:15.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.163+0000 7eff281a6740 1 --2- 172.21.15.5:0/3141278887 >> [v2:172.21.15.63:6834/3249423677,v1:172.21.15.63:6835/3249423677] conn(0x55f8ee875870 0x55f8ee843330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:15.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.163+0000 7eff281a6740 4 client.4404 mds_command: new command op to 4279 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T18:27:15.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.163+0000 7eff281a6740 1 -- 172.21.15.5:0/3141278887 --> [v2:172.21.15.63:6834/3249423677,v1:172.21.15.63:6835/3249423677] -- command(tid 0: {"prefix": "status"}) -- 0x55f8ee812060 con 0x55f8ee875870 2024-05-20T18:27:15.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.164+0000 7eff1affd640 1 --2- 172.21.15.5:0/3141278887 >> [v2:172.21.15.63:6834/3249423677,v1:172.21.15.63:6835/3249423677] conn(0x55f8ee875870 0x55f8ee843330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:15.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.164+0000 7eff1affd640 1 --2- 172.21.15.5:0/3141278887 >> [v2:172.21.15.63:6834/3249423677,v1:172.21.15.63:6835/3249423677] conn(0x55f8ee875870 0x55f8ee843330 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a1455c871c6c1dc server_cookie=7087d62189d8c5db in_seq=0 out_seq=0 2024-05-20T18:27:15.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.164+0000 7efef9ffb640 10 client.4404 ms_handle_connect on v2:172.21.15.63:6834/3249423677 2024-05-20T18:27:15.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.164+0000 7efef9ffb640 1 -- 172.21.15.5:0/3141278887 <== mds.0 v2:172.21.15.63:6834/3249423677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55f8ee812060 con 0x55f8ee875870 2024-05-20T18:27:15.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.164+0000 7efef9ffb640 10 client.4404 handle_command_reply: tid=0 2024-05-20T18:27:15.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.164+0000 7eff281a6740 1 client.4404 shutdown 2024-05-20T18:27:15.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.164+0000 7eff281a6740 1 -- 172.21.15.5:0/3141278887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8ee476270 msgr2=0x55f8ee8751d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:15.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.164+0000 7eff281a6740 1 --2- 172.21.15.5:0/3141278887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8ee476270 0x55f8ee8751d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7eff14000f80 tx=0x7eff14004ab0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.164+0000 7eff281a6740 1 -- 172.21.15.5:0/3141278887 shutdown_connections 2024-05-20T18:27:15.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.164+0000 7eff281a6740 1 --2- 172.21.15.5:0/3141278887 >> [v2:172.21.15.63:6834/3249423677,v1:172.21.15.63:6835/3249423677] conn(0x55f8ee875870 0x55f8ee843330 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.164+0000 7eff281a6740 1 --2- 172.21.15.5:0/3141278887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8ee476270 0x55f8ee8751d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.164+0000 7eff281a6740 1 --2- 172.21.15.5:0/3141278887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8ee455590 0x55f8ee837a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.164+0000 7efef9ffb640 0 client.4404 ms_handle_reset on v2:172.21.15.63:6834/3249423677 2024-05-20T18:27:15.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.164+0000 7eff281a6740 1 --2- 172.21.15.5:0/3141278887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ee44d3b0 0x55f8ee862f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.164+0000 7eff281a6740 1 -- 172.21.15.5:0/3141278887 >> 172.21.15.5:0/3141278887 conn(0x55f8ee46a3c0 msgr2=0x55f8ee8111b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:15.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.165+0000 7eff281a6740 1 -- 172.21.15.5:0/3141278887 shutdown_connections 2024-05-20T18:27:15.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.165+0000 7eff281a6740 1 -- 172.21.15.5:0/3141278887 wait complete. 2024-05-20T18:27:15.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.165+0000 7eff281a6740 20 client.4404 trim_cache size 0 max 16384 2024-05-20T18:27:15.171 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:27:15.172 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "399ab250-fd3c-47fa-a1c8-25e8149c9cac", 2024-05-20T18:27:15.172 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-20T18:27:15.172 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 4279, 2024-05-20T18:27:15.172 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-20T18:27:15.172 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-20T18:27:15.172 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-20T18:27:15.172 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 7.3886465560000003, 2024-05-20T18:27:15.172 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 7, 2024-05-20T18:27:15.172 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 26, 2024-05-20T18:27:15.172 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 25, 2024-05-20T18:27:15.172 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 14.053165392 2024-05-20T18:27:15.173 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:27:15.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.165+0000 7eff212fe640 1 -- 172.21.15.5:0/1589400022 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7eff0806be30 msgr2=0x7eff0806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:15.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.165+0000 7eff212fe640 1 --2- 172.21.15.5:0/1589400022 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7eff0806be30 0x7eff0806e340 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0b4f10 tx=0x7eff10009040 comp rx=0 tx=0).stop 2024-05-20T18:27:15.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.165+0000 7eff212fe640 1 -- 172.21.15.5:0/1589400022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff1c0d0e00 msgr2=0x7eff1c160340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:15.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.165+0000 7eff212fe640 1 --2- 172.21.15.5:0/1589400022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff1c0d0e00 0x7eff1c160340 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00d5d0 tx=0x7eff0c00da90 comp rx=0 tx=0).stop 2024-05-20T18:27:15.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.165+0000 7eff212fe640 1 -- 172.21.15.5:0/1589400022 shutdown_connections 2024-05-20T18:27:15.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.165+0000 7eff212fe640 1 --2- 172.21.15.5:0/1589400022 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7eff0806be30 0x7eff0806e340 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.165+0000 7eff212fe640 1 --2- 172.21.15.5:0/1589400022 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7eff1c160880 0x7eff1c0b7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.165+0000 7eff212fe640 1 --2- 172.21.15.5:0/1589400022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff1c15f220 0x7eff1c0b7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.165+0000 7eff212fe640 1 --2- 172.21.15.5:0/1589400022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff1c0d0e00 0x7eff1c160340 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.165+0000 7eff212fe640 1 -- 172.21.15.5:0/1589400022 >> 172.21.15.5:0/1589400022 conn(0x7eff1c0b44d0 msgr2=0x7eff1c0b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:15.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.165+0000 7eff212fe640 1 -- 172.21.15.5:0/1589400022 shutdown_connections 2024-05-20T18:27:15.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.165+0000 7eff212fe640 1 -- 172.21.15.5:0/1589400022 wait complete. 2024-05-20T18:27:15.176 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-20T18:27:15.176 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-20T18:27:15.176 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-20T18:27:15.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.291+0000 7f839f8af640 1 Processor -- start 2024-05-20T18:27:15.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.291+0000 7f839f8af640 1 -- start start 2024-05-20T18:27:15.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.293+0000 7f839f8af640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f839815f220 0x7f83980d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:15.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.293+0000 7f839f8af640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f83980cfae0 0x7f83980ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:15.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.293+0000 7f839f8af640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83980cda20 con 0x7f83980d0e00 2024-05-20T18:27:15.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.293+0000 7f839f8af640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f83980cdd00 con 0x7f83980cfae0 2024-05-20T18:27:15.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.293+0000 7f839f8af640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f83980cdfe0 con 0x7f839815f220 2024-05-20T18:27:15.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.293+0000 7f839de4c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f83980cfae0 0x7f83980ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:15.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.293+0000 7f839ce4a640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f83980d0e00 0x7f83980cf3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36490/0 (socket says 172.21.15.5:36490) 2024-05-20T18:27:15.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.293+0000 7f839de4c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f83980cfae0 0x7f83980ce7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54470/0 (socket says 172.21.15.5:54470) 2024-05-20T18:27:15.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.293+0000 7f839ce4a640 1 -- 172.21.15.5:0/3891391010 learned_addr learned my addr 172.21.15.5:0/3891391010 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:15.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.293+0000 7f839d64b640 1 --2- 172.21.15.5:0/3891391010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f839815f220 0x7f83980d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:15.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.293+0000 7f839de4c640 1 -- 172.21.15.5:0/3891391010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f839815f220 msgr2=0x7f83980d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:15.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.293+0000 7f839de4c640 1 --2- 172.21.15.5:0/3891391010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f839815f220 0x7f83980d08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.293+0000 7f839de4c640 1 -- 172.21.15.5:0/3891391010 >> v1:172.21.15.5:6789/0 conn(0x7f83980d0e00 legacy=0x7f83980cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:15.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.293+0000 7f839de4c640 1 -- 172.21.15.5:0/3891391010 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83980ce2c0 con 0x7f83980cfae0 2024-05-20T18:27:15.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.293+0000 7f838ffff640 1 -- 172.21.15.5:0/3891391010 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 783431049 0 0) 0x7f83980cda20 con 0x7f83980d0e00 2024-05-20T18:27:15.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.294+0000 7f839de4c640 1 --2- 172.21.15.5:0/3891391010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83980cfae0 0x7f83980ce7e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f83900097a0 tx=0x7f839002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e18a3eb0f02ae2a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:15.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.294+0000 7f838ffff640 1 -- 172.21.15.5:0/3891391010 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8390037070 con 0x7f83980cfae0 2024-05-20T18:27:15.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.294+0000 7f838ffff640 1 -- 172.21.15.5:0/3891391010 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f839002ae50 con 0x7f83980cfae0 2024-05-20T18:27:15.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.294+0000 7f838ffff640 1 -- 172.21.15.5:0/3891391010 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8390032660 con 0x7f83980cfae0 2024-05-20T18:27:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.295+0000 7f839f8af640 1 -- 172.21.15.5:0/3891391010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83980cfae0 msgr2=0x7f83980ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.295+0000 7f839f8af640 1 --2- 172.21.15.5:0/3891391010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83980cfae0 0x7f83980ce7e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f83900097a0 tx=0x7f839002a100 comp rx=0 tx=0).stop 2024-05-20T18:27:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.295+0000 7f839f8af640 1 -- 172.21.15.5:0/3891391010 shutdown_connections 2024-05-20T18:27:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.295+0000 7f839f8af640 1 --2- 172.21.15.5:0/3891391010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83980cfae0 0x7f83980ce7e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.295+0000 7f839f8af640 1 --2- 172.21.15.5:0/3891391010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f839815f220 0x7f83980d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.295+0000 7f839f8af640 1 -- 172.21.15.5:0/3891391010 >> 172.21.15.5:0/3891391010 conn(0x7f83980b44d0 msgr2=0x7f83980b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.295+0000 7f839f8af640 1 -- 172.21.15.5:0/3891391010 shutdown_connections 2024-05-20T18:27:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.295+0000 7f839f8af640 1 -- 172.21.15.5:0/3891391010 wait complete. 2024-05-20T18:27:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.297+0000 7f839f8af640 1 Processor -- start 2024-05-20T18:27:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.297+0000 7f839f8af640 1 -- start start 2024-05-20T18:27:15.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.298+0000 7f839f8af640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83980d0e00 0x7f83980b5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:15.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.298+0000 7f839f8af640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f839815f220 0x7f83981f7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:15.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.298+0000 7f839f8af640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83981f81b0 0x7f83981f85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:15.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.298+0000 7f839f8af640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f839815f7b0 con 0x7f83981f81b0 2024-05-20T18:27:15.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.298+0000 7f839f8af640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f839815f630 con 0x7f83980d0e00 2024-05-20T18:27:15.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.298+0000 7f839f8af640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f83980ced20 con 0x7f839815f220 2024-05-20T18:27:15.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.299+0000 7f839de4c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83981f81b0 0x7f83981f85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:15.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.299+0000 7f839de4c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83981f81b0 0x7f83981f85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54476/0 (socket says 172.21.15.5:54476) 2024-05-20T18:27:15.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.299+0000 7f839de4c640 1 -- 172.21.15.5:0/2833457054 learned_addr learned my addr 172.21.15.5:0/2833457054 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:15.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.299+0000 7f839d64b640 1 --2- 172.21.15.5:0/2833457054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83980d0e00 0x7f83980b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:15.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.299+0000 7f839ce4a640 1 --2- 172.21.15.5:0/2833457054 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f839815f220 0x7f83981f7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:15.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.299+0000 7f839de4c640 1 -- 172.21.15.5:0/2833457054 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f839815f220 msgr2=0x7f83981f7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:15.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.299+0000 7f839de4c640 1 --2- 172.21.15.5:0/2833457054 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f839815f220 0x7f83981f7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.299+0000 7f839de4c640 1 -- 172.21.15.5:0/2833457054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83980d0e00 msgr2=0x7f83980b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:15.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.299+0000 7f839de4c640 1 --2- 172.21.15.5:0/2833457054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83980d0e00 0x7f83980b5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.299+0000 7f839de4c640 1 -- 172.21.15.5:0/2833457054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83980b8260 con 0x7f83981f81b0 2024-05-20T18:27:15.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.299+0000 7f839d64b640 1 --2- 172.21.15.5:0/2833457054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83980d0e00 0x7f83980b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:15.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.299+0000 7f839de4c640 1 --2- 172.21.15.5:0/2833457054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83981f81b0 0x7f83981f85c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8390035040 tx=0x7f83900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:15.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.299+0000 7f838dffb640 1 -- 172.21.15.5:0/2833457054 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8390041070 con 0x7f83981f81b0 2024-05-20T18:27:15.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.299+0000 7f838dffb640 1 -- 172.21.15.5:0/2833457054 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8390032e10 con 0x7f83981f81b0 2024-05-20T18:27:15.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.299+0000 7f839f8af640 1 -- 172.21.15.5:0/2833457054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83980b8540 con 0x7f83981f81b0 2024-05-20T18:27:15.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.299+0000 7f838dffb640 1 -- 172.21.15.5:0/2833457054 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8390037040 con 0x7f83981f81b0 2024-05-20T18:27:15.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.300+0000 7f839f8af640 1 -- 172.21.15.5:0/2833457054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83980c7140 con 0x7f83981f81b0 2024-05-20T18:27:15.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.300+0000 7f839f8af640 1 -- 172.21.15.5:0/2833457054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83980b7fb0 con 0x7f83981f81b0 2024-05-20T18:27:15.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.301+0000 7f838dffb640 1 -- 172.21.15.5:0/2833457054 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8390044070 con 0x7f83981f81b0 2024-05-20T18:27:15.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.302+0000 7f838dffb640 1 --2- 172.21.15.5:0/2833457054 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f837c06be60 0x7f837c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:15.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.302+0000 7f838dffb640 1 -- 172.21.15.5:0/2833457054 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f83900ab460 con 0x7f83981f81b0 2024-05-20T18:27:15.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.302+0000 7f839d64b640 1 --2- 172.21.15.5:0/2833457054 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f837c06be60 0x7f837c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:15.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.302+0000 7f839d64b640 1 --2- 172.21.15.5:0/2833457054 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f837c06be60 0x7f837c06e370 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f83980c64c0 tx=0x7f83940039d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:15.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.306+0000 7f838dffb640 1 -- 172.21.15.5:0/2833457054 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8390079a40 con 0x7f83981f81b0 2024-05-20T18:27:15.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.531+0000 7f839f8af640 1 -- 172.21.15.5:0/2833457054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8398075b00 con 0x7f83981f81b0 2024-05-20T18:27:15.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.531+0000 7f838dffb640 1 -- 172.21.15.5:0/2833457054 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f8390079340 con 0x7f83981f81b0 2024-05-20T18:27:15.531 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-20T18:27:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.534+0000 7f839f8af640 1 -- 172.21.15.5:0/2833457054 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f837c06be60 msgr2=0x7f837c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.534+0000 7f839f8af640 1 --2- 172.21.15.5:0/2833457054 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f837c06be60 0x7f837c06e370 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f83980c64c0 tx=0x7f83940039d0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.534+0000 7f839f8af640 1 -- 172.21.15.5:0/2833457054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83981f81b0 msgr2=0x7f83981f85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.534+0000 7f839f8af640 1 --2- 172.21.15.5:0/2833457054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83981f81b0 0x7f83981f85c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8390035040 tx=0x7f83900046c0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.534+0000 7f839f8af640 1 -- 172.21.15.5:0/2833457054 shutdown_connections 2024-05-20T18:27:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.534+0000 7f839f8af640 1 --2- 172.21.15.5:0/2833457054 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f837c06be60 0x7f837c06e370 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.534+0000 7f839f8af640 1 --2- 172.21.15.5:0/2833457054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83981f81b0 0x7f83981f85c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.534+0000 7f839f8af640 1 --2- 172.21.15.5:0/2833457054 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f839815f220 0x7f83981f7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.534+0000 7f839f8af640 1 --2- 172.21.15.5:0/2833457054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83980d0e00 0x7f83980b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:15.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.534+0000 7f839f8af640 1 -- 172.21.15.5:0/2833457054 >> 172.21.15.5:0/2833457054 conn(0x7f83980b44d0 msgr2=0x7f83980b3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:15.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.534+0000 7f839f8af640 1 -- 172.21.15.5:0/2833457054 shutdown_connections 2024-05-20T18:27:15.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15.534+0000 7f839f8af640 1 -- 172.21.15.5:0/2833457054 wait complete. 2024-05-20T18:27:15.544 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-20T18:27:15.544 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-20T18:27:15.544 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-05-20T18:27:15.544 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-20T18:27:15.544 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-20T18:27:15.545 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-20T18:27:15.545 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-20T18:27:15.545 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-20T18:27:15.545 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-20T18:27:15.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:15.545 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-05-20T18:27:15.561 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T18:27:15.561 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T18:27:15.561 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T18:27:15.562 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:15.562 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-20T18:27:15.562 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:15.562 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:27:15.562 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-05-20T18:27:15.562 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-05-20T18:27:15.562 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:27:15.562 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-05-20T18:27:15.562 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-05-20T18:27:15.562 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-20T18:27:15.562 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:27:15.562 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-05-20T18:27:15.563 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-20T18:27:15.563 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42663sec preferred_lft 42663sec 2024-05-20T18:27:15.563 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-05-20T18:27:15.563 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:15.563 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:27:15.563 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:27:15.563 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-05-20T18:27:15.564 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-20T18:27:15.564 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:27:15.564 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-20T18:27:15.564 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add name ceph-brx type bridge 2024-05-20T18:27:15.564 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr flush dev ceph-brx 2024-05-20T18:27:15.564 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set ceph-brx up 2024-05-20T18:27:15.564 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-20T18:27:15.564 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-20T18:27:15.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:27:15.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:15.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:15.762 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-20T18:27:15.789 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-20T18:27:15.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:15.791 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-05-20T18:27:15.855 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-05-20T18:27:15.855 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-20T18:27:15.855 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-20T18:27:15.855 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-20T18:27:15.856 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-20T18:27:15.856 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:27:15.856 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-20T18:27:15.856 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-20T18:27:15.856 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-20T18:27:15.856 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-20T18:27:15.856 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-20T18:27:15.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:27:16.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:16.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:16.164 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-20T18:27:16.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:16.183 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list-id 2024-05-20T18:27:16.240 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:27:16.240 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-20T18:27:16.241 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-20T18:27:16.241 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-20T18:27:16.241 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-20T18:27:16.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:27:16.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:16.367 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-20T18:27:16.368 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:27:16.368 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-20T18:27:16.368 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-20T18:27:16.368 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-20T18:27:16.368 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-20T18:27:16.368 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-20T18:27:16.368 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-20T18:27:16.368 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-20T18:27:16.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:27:16.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:16.525 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:27:16.525 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-20T18:27:16.525 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-05-20T18:27:16.525 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set dev brx.0 master ceph-brx 2024-05-20T18:27:16.526 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-20T18:27:16.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:27:16.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:16.607 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-20T18:27:16.608 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:27:16.608 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-20T18:27:16.624 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-20T18:27:16.624 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:27:16.625 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-20T18:27:16.681 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2024-05-20T18:27:16.747 DEBUG:teuthology.orchestra.run.smithi005:> cat /proc/self/mounts | awk '{print $2}' 2024-05-20T18:27:16.804 INFO:teuthology.orchestra.run.smithi005.stdout:/proc 2024-05-20T18:27:16.805 INFO:teuthology.orchestra.run.smithi005.stdout:/sys 2024-05-20T18:27:16.805 INFO:teuthology.orchestra.run.smithi005.stdout:/dev 2024-05-20T18:27:16.805 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/security 2024-05-20T18:27:16.805 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/shm 2024-05-20T18:27:16.805 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/pts 2024-05-20T18:27:16.805 INFO:teuthology.orchestra.run.smithi005.stdout:/run 2024-05-20T18:27:16.805 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup 2024-05-20T18:27:16.805 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/pstore 2024-05-20T18:27:16.805 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/bpf 2024-05-20T18:27:16.805 INFO:teuthology.orchestra.run.smithi005.stdout:/ 2024-05-20T18:27:16.805 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/selinux 2024-05-20T18:27:16.806 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2024-05-20T18:27:16.806 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/mqueue 2024-05-20T18:27:16.806 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/hugepages 2024-05-20T18:27:16.806 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/debug 2024-05-20T18:27:16.806 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/tracing 2024-05-20T18:27:16.806 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/fuse/connections 2024-05-20T18:27:16.806 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/config 2024-05-20T18:27:16.806 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-sysctl.service 2024-05-20T18:27:16.806 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-20T18:27:16.806 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-20T18:27:16.806 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/nfs/rpc_pipefs 2024-05-20T18:27:16.806 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/1000 2024-05-20T18:27:16.806 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph 2024-05-20T18:27:16.807 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-0 2024-05-20T18:27:16.807 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-1 2024-05-20T18:27:16.807 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-2 2024-05-20T18:27:16.807 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-3 2024-05-20T18:27:16.807 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns 2024-05-20T18:27:16.807 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-20T18:27:16.807 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-20T18:27:16.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:16.807 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2024-05-20T18:27:16.861 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-20T18:27:16.861 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-20T18:27:16.903 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2024-05-20T18:27:16.928 DEBUG:teuthology.orchestra.run.smithi005:> cat /proc/self/mounts | awk '{print $2}' 2024-05-20T18:27:16.986 INFO:teuthology.orchestra.run.smithi005.stdout:/proc 2024-05-20T18:27:16.986 INFO:teuthology.orchestra.run.smithi005.stdout:/sys 2024-05-20T18:27:16.986 INFO:teuthology.orchestra.run.smithi005.stdout:/dev 2024-05-20T18:27:16.986 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/security 2024-05-20T18:27:16.987 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/shm 2024-05-20T18:27:16.987 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/pts 2024-05-20T18:27:16.987 INFO:teuthology.orchestra.run.smithi005.stdout:/run 2024-05-20T18:27:16.987 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup 2024-05-20T18:27:16.987 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/pstore 2024-05-20T18:27:16.987 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/bpf 2024-05-20T18:27:16.987 INFO:teuthology.orchestra.run.smithi005.stdout:/ 2024-05-20T18:27:16.987 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/selinux 2024-05-20T18:27:16.987 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2024-05-20T18:27:16.987 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/mqueue 2024-05-20T18:27:16.987 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/hugepages 2024-05-20T18:27:16.987 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/debug 2024-05-20T18:27:16.987 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/tracing 2024-05-20T18:27:16.987 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/fuse/connections 2024-05-20T18:27:16.988 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/config 2024-05-20T18:27:16.988 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-sysctl.service 2024-05-20T18:27:16.988 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-20T18:27:16.988 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-20T18:27:16.988 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/nfs/rpc_pipefs 2024-05-20T18:27:16.988 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/1000 2024-05-20T18:27:16.988 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph 2024-05-20T18:27:16.988 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-0 2024-05-20T18:27:16.988 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-1 2024-05-20T18:27:16.988 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-2 2024-05-20T18:27:16.988 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-3 2024-05-20T18:27:16.988 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns 2024-05-20T18:27:16.988 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-20T18:27:16.989 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-20T18:27:16.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:16.990 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2024-05-20T18:27:17.007 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi005.stderr:2024-05-20T18:27:17.007+0000 7f120a572480 -1 init, newargv = 0x55935a6ab760 newargc=15 2024-05-20T18:27:17.007 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi005.stderr:ceph-fuse[43837]: starting ceph client 2024-05-20T18:27:17.014 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi005.stderr:ceph-fuse[43837]: starting fuse 2024-05-20T18:27:17.023 INFO:teuthology.orchestra.run.smithi005.stdout:42 2024-05-20T18:27:17.023 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-05-20T18:27:17.023 DEBUG:teuthology.orchestra.run.smithi005:> sudo stdin-killer -- python3 -c ' 2024-05-20T18:27:17.024 DEBUG:teuthology.orchestra.run.smithi005:> import glob 2024-05-20T18:27:17.024 DEBUG:teuthology.orchestra.run.smithi005:> import re 2024-05-20T18:27:17.024 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:27:17.024 DEBUG:teuthology.orchestra.run.smithi005:> import subprocess 2024-05-20T18:27:17.024 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:27:17.024 DEBUG:teuthology.orchestra.run.smithi005:> def _find_admin_socket(client_name): 2024-05-20T18:27:17.024 DEBUG:teuthology.orchestra.run.smithi005:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-20T18:27:17.024 DEBUG:teuthology.orchestra.run.smithi005:> files = glob.glob(asok_path) 2024-05-20T18:27:17.024 DEBUG:teuthology.orchestra.run.smithi005:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-20T18:27:17.024 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:27:17.024 DEBUG:teuthology.orchestra.run.smithi005:> # Given a non-glob path, it better be there 2024-05-20T18:27:17.024 DEBUG:teuthology.orchestra.run.smithi005:> if "*" not in asok_path: 2024-05-20T18:27:17.024 DEBUG:teuthology.orchestra.run.smithi005:> assert(len(files) == 1) 2024-05-20T18:27:17.024 DEBUG:teuthology.orchestra.run.smithi005:> return files[0] 2024-05-20T18:27:17.024 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:27:17.024 DEBUG:teuthology.orchestra.run.smithi005:> for f in files: 2024-05-20T18:27:17.025 DEBUG:teuthology.orchestra.run.smithi005:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-20T18:27:17.025 DEBUG:teuthology.orchestra.run.smithi005:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-20T18:27:17.025 DEBUG:teuthology.orchestra.run.smithi005:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-20T18:27:17.025 DEBUG:teuthology.orchestra.run.smithi005:> contents = proc_f.read() 2024-05-20T18:27:17.025 DEBUG:teuthology.orchestra.run.smithi005:> if mountpoint in contents: 2024-05-20T18:27:17.025 DEBUG:teuthology.orchestra.run.smithi005:> return f 2024-05-20T18:27:17.025 DEBUG:teuthology.orchestra.run.smithi005:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-20T18:27:17.025 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:27:17.025 DEBUG:teuthology.orchestra.run.smithi005:> print(_find_admin_socket("client.admin")) 2024-05-20T18:27:17.025 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:27:17.145 INFO:teuthology.orchestra.run.smithi005.stdout:/var/run/ceph/ceph-client.admin.43837.asok 2024-05-20T18:27:17.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:17.156 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.43837.asok 2024-05-20T18:27:17.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:17.156 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.43837.asok status 2024-05-20T18:27:17.305 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:27:17.305 INFO:teuthology.orchestra.run.smithi005.stdout: "metadata": { 2024-05-20T18:27:17.305 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_sha1": "d2b507c724ef6c8295ee0a35d185cdab167fd61d", 2024-05-20T18:27:17.305 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_version": "ceph version 19.0.0-2744-gd2b507c7 (d2b507c724ef6c8295ee0a35d185cdab167fd61d) squid (dev)", 2024-05-20T18:27:17.305 INFO:teuthology.orchestra.run.smithi005.stdout: "entity_id": "admin", 2024-05-20T18:27:17.306 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": "smithi005", 2024-05-20T18:27:17.306 INFO:teuthology.orchestra.run.smithi005.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-20T18:27:17.306 INFO:teuthology.orchestra.run.smithi005.stdout: "pid": "43837", 2024-05-20T18:27:17.306 INFO:teuthology.orchestra.run.smithi005.stdout: "root": "/" 2024-05-20T18:27:17.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:27:17.306 INFO:teuthology.orchestra.run.smithi005.stdout: "dentry_count": 0, 2024-05-20T18:27:17.306 INFO:teuthology.orchestra.run.smithi005.stdout: "dentry_pinned_count": 0, 2024-05-20T18:27:17.306 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 4419, 2024-05-20T18:27:17.306 INFO:teuthology.orchestra.run.smithi005.stdout: "inst": { 2024-05-20T18:27:17.306 INFO:teuthology.orchestra.run.smithi005.stdout: "name": { 2024-05-20T18:27:17.306 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "client", 2024-05-20T18:27:17.306 INFO:teuthology.orchestra.run.smithi005.stdout: "num": 4419 2024-05-20T18:27:17.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:27:17.307 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": { 2024-05-20T18:27:17.307 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-20T18:27:17.307 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "192.168.0.1:0", 2024-05-20T18:27:17.307 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 439882589 2024-05-20T18:27:17.307 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:27:17.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:27:17.307 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": { 2024-05-20T18:27:17.307 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-20T18:27:17.307 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "192.168.0.1:0", 2024-05-20T18:27:17.307 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 439882589 2024-05-20T18:27:17.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:27:17.307 INFO:teuthology.orchestra.run.smithi005.stdout: "inst_str": "client.4419 192.168.0.1:0/439882589", 2024-05-20T18:27:17.307 INFO:teuthology.orchestra.run.smithi005.stdout: "addr_str": "192.168.0.1:0/439882589", 2024-05-20T18:27:17.307 INFO:teuthology.orchestra.run.smithi005.stdout: "inode_count": 1, 2024-05-20T18:27:17.308 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_epoch": 7, 2024-05-20T18:27:17.308 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_epoch": 26, 2024-05-20T18:27:17.308 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_epoch_barrier": 0, 2024-05-20T18:27:17.308 INFO:teuthology.orchestra.run.smithi005.stdout: "blocklisted": false, 2024-05-20T18:27:17.308 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs" 2024-05-20T18:27:17.308 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:27:17.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:17.314 DEBUG:teuthology.orchestra.run.smithi005:> stat --file-system '--printf=%T 2024-05-20T18:27:17.314 DEBUG:teuthology.orchestra.run.smithi005:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-20T18:27:17.330 INFO:teuthology.orchestra.run.smithi005.stdout:fuseblk 2024-05-20T18:27:17.330 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-20T18:27:17.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:17.331 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-20T18:27:17.398 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-20T18:27:17.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T18:27:17.511 INFO:teuthology.orchestra.run.smithi005.stdout:# file: . 2024-05-20T18:27:17.511 INFO:teuthology.orchestra.run.smithi005.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-20T18:27:17.511 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:17.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:17.517 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:27:17.517 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:27:17.534 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:27:17.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.689+0000 7f6fcd596640 1 Processor -- start 2024-05-20T18:27:17.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.689+0000 7f6fcd596640 1 -- start start 2024-05-20T18:27:17.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.691+0000 7f6fcd596640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6fc806d9a0 0x7f6fc80d1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:17.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.691+0000 7f6fcd596640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6fc815d940 0x7f6fc815dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.691+0000 7f6fcd596640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fc815e6b0 con 0x7f6fc80b2f10 2024-05-20T18:27:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.691+0000 7f6fcd596640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f6fc80d2400 con 0x7f6fc815d940 2024-05-20T18:27:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.691+0000 7f6fcd596640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f6fc80d1a30 con 0x7f6fc806d9a0 2024-05-20T18:27:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.691+0000 7f6fc77fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6fc815d940 0x7f6fc815dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.691+0000 7f6fc77fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6fc815d940 0x7f6fc815dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54482/0 (socket says 172.21.15.5:54482) 2024-05-20T18:27:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.691+0000 7f6fc77fe640 1 -- 172.21.15.5:0/1336122239 learned_addr learned my addr 172.21.15.5:0/1336122239 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.691+0000 7f6fc67fc640 1 --2- 172.21.15.5:0/1336122239 >> v2:172.21.15.63:3300/0 conn(0x7f6fc806d9a0 0x7f6fc80d1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.691+0000 7f6fc77fe640 1 -- 172.21.15.5:0/1336122239 >> v2:172.21.15.63:3300/0 conn(0x7f6fc806d9a0 msgr2=0x7f6fc80d1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.691+0000 7f6fc77fe640 1 --2- 172.21.15.5:0/1336122239 >> v2:172.21.15.63:3300/0 conn(0x7f6fc806d9a0 0x7f6fc80d1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.691+0000 7f6fc77fe640 1 -- 172.21.15.5:0/1336122239 >> v1:172.21.15.63:6789/0 conn(0x7f6fc80b2f10 legacy=0x7f6fc806d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.691+0000 7f6fc77fe640 1 -- 172.21.15.5:0/1336122239 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc80d1100 con 0x7f6fc815d940 2024-05-20T18:27:17.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.692+0000 7f6fc77fe640 1 --2- 172.21.15.5:0/1336122239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fc815d940 0x7f6fc815dd50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6fc809b680 tx=0x7f6fc002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0fa918ea1118e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:17.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.692+0000 7f6fc5ffb640 1 -- 172.21.15.5:0/1336122239 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0037070 con 0x7f6fc815d940 2024-05-20T18:27:17.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.692+0000 7f6fc5ffb640 1 -- 172.21.15.5:0/1336122239 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fc0032440 con 0x7f6fc815d940 2024-05-20T18:27:17.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.692+0000 7f6fc5ffb640 1 -- 172.21.15.5:0/1336122239 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0032810 con 0x7f6fc815d940 2024-05-20T18:27:17.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.693+0000 7f6fcd596640 1 -- 172.21.15.5:0/1336122239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fc815d940 msgr2=0x7f6fc815dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:17.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.693+0000 7f6fcd596640 1 --2- 172.21.15.5:0/1336122239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fc815d940 0x7f6fc815dd50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6fc809b680 tx=0x7f6fc002a1e0 comp rx=0 tx=0).stop 2024-05-20T18:27:17.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.693+0000 7f6fcd596640 1 -- 172.21.15.5:0/1336122239 shutdown_connections 2024-05-20T18:27:17.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.693+0000 7f6fcd596640 1 --2- 172.21.15.5:0/1336122239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fc815d940 0x7f6fc815dd50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:17.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.693+0000 7f6fcd596640 1 --2- 172.21.15.5:0/1336122239 >> v2:172.21.15.63:3300/0 conn(0x7f6fc806d9a0 0x7f6fc80d1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:17.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.693+0000 7f6fcd596640 1 -- 172.21.15.5:0/1336122239 >> 172.21.15.5:0/1336122239 conn(0x7f6fc80b6670 msgr2=0x7f6fc80b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:17.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.693+0000 7f6fcd596640 1 -- 172.21.15.5:0/1336122239 shutdown_connections 2024-05-20T18:27:17.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.693+0000 7f6fcd596640 1 -- 172.21.15.5:0/1336122239 wait complete. 2024-05-20T18:27:17.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.696+0000 7f6fcd596640 1 Processor -- start 2024-05-20T18:27:17.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.696+0000 7f6fcd596640 1 -- start start 2024-05-20T18:27:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.696+0000 7f6fcd596640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6fc806d9a0 0x7f6fc8156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.696+0000 7f6fcd596640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fc80b2f10 0x7f6fc81535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.696+0000 7f6fcd596640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fc815d940 0x7f6fc809e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.696+0000 7f6fcd596640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc80d16c0 con 0x7f6fc80b2f10 2024-05-20T18:27:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.696+0000 7f6fcd596640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc80b3320 con 0x7f6fc815d940 2024-05-20T18:27:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.696+0000 7f6fcd596640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6fc815e290 con 0x7f6fc806d9a0 2024-05-20T18:27:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.696+0000 7f6fc67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fc80b2f10 0x7f6fc81535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.696+0000 7f6fc67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fc80b2f10 0x7f6fc81535f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54488/0 (socket says 172.21.15.5:54488) 2024-05-20T18:27:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.696+0000 7f6fc67fc640 1 -- 172.21.15.5:0/2819795667 learned_addr learned my addr 172.21.15.5:0/2819795667 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.696+0000 7f6fc77fe640 1 --2- 172.21.15.5:0/2819795667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fc815d940 0x7f6fc809e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.696+0000 7f6fc6ffd640 1 --2- 172.21.15.5:0/2819795667 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6fc806d9a0 0x7f6fc8156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.697+0000 7f6fc67fc640 1 -- 172.21.15.5:0/2819795667 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6fc806d9a0 msgr2=0x7f6fc8156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.697+0000 7f6fc67fc640 1 --2- 172.21.15.5:0/2819795667 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6fc806d9a0 0x7f6fc8156540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.697+0000 7f6fc67fc640 1 -- 172.21.15.5:0/2819795667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fc815d940 msgr2=0x7f6fc809e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.697+0000 7f6fc67fc640 1 --2- 172.21.15.5:0/2819795667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fc815d940 0x7f6fc809e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.697+0000 7f6fc67fc640 1 -- 172.21.15.5:0/2819795667 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc81f8fe0 con 0x7f6fc80b2f10 2024-05-20T18:27:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.697+0000 7f6fc67fc640 1 --2- 172.21.15.5:0/2819795667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fc80b2f10 0x7f6fc81535f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000e840 tx=0x7f6fb000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.697+0000 7f6fbf7fe640 1 -- 172.21.15.5:0/2819795667 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0017070 con 0x7f6fc80b2f10 2024-05-20T18:27:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.697+0000 7f6fbf7fe640 1 -- 172.21.15.5:0/2819795667 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fb000c940 con 0x7f6fc80b2f10 2024-05-20T18:27:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.697+0000 7f6fbf7fe640 1 -- 172.21.15.5:0/2819795667 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb000cd30 con 0x7f6fc80b2f10 2024-05-20T18:27:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.697+0000 7f6fcd596640 1 -- 172.21.15.5:0/2819795667 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fc81f9280 con 0x7f6fc80b2f10 2024-05-20T18:27:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.697+0000 7f6fcd596640 1 -- 172.21.15.5:0/2819795667 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fc81f9840 con 0x7f6fc80b2f10 2024-05-20T18:27:17.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.698+0000 7f6fcd596640 1 -- 172.21.15.5:0/2819795667 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fc809f860 con 0x7f6fc80b2f10 2024-05-20T18:27:17.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.699+0000 7f6fbf7fe640 1 -- 172.21.15.5:0/2819795667 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6fb001a070 con 0x7f6fc80b2f10 2024-05-20T18:27:17.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.699+0000 7f6fbf7fe640 1 --2- 172.21.15.5:0/2819795667 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6fa806be30 0x7f6fa806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:17.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.699+0000 7f6fbf7fe640 1 -- 172.21.15.5:0/2819795667 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f6fb008b570 con 0x7f6fc80b2f10 2024-05-20T18:27:17.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.699+0000 7f6fc6ffd640 1 --2- 172.21.15.5:0/2819795667 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6fa806be30 0x7f6fa806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:17.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.700+0000 7f6fc6ffd640 1 --2- 172.21.15.5:0/2819795667 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6fa806be30 0x7f6fa806e340 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6fb80012e0 tx=0x7f6fb8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:17.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.704+0000 7f6fbf7fe640 1 -- 172.21.15.5:0/2819795667 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6fb0059460 con 0x7f6fc80b2f10 2024-05-20T18:27:17.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.894+0000 7f6fcd596640 1 -- 172.21.15.5:0/2819795667 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6fc8075b00 con 0x7f6fc80b2f10 2024-05-20T18:27:17.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.895+0000 7f6fbf7fe640 1 -- 172.21.15.5:0/2819795667 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18394 (secure 0 0 0) 0x7f6fb005d760 con 0x7f6fc80b2f10 2024-05-20T18:27:17.894 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:17.894 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:27:13.817633+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:27:05.476596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T18:27:06.137762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T18:27:09.829920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:27:17.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.897+0000 7f6fcd596640 1 -- 172.21.15.5:0/2819795667 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6fa806be30 msgr2=0x7f6fa806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:17.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.897+0000 7f6fcd596640 1 --2- 172.21.15.5:0/2819795667 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6fa806be30 0x7f6fa806e340 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6fb80012e0 tx=0x7f6fb8009040 comp rx=0 tx=0).stop 2024-05-20T18:27:17.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.897+0000 7f6fcd596640 1 -- 172.21.15.5:0/2819795667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fc80b2f10 msgr2=0x7f6fc81535f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:17.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.897+0000 7f6fcd596640 1 --2- 172.21.15.5:0/2819795667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fc80b2f10 0x7f6fc81535f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000e840 tx=0x7f6fb000ed00 comp rx=0 tx=0).stop 2024-05-20T18:27:17.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.897+0000 7f6fcd596640 1 -- 172.21.15.5:0/2819795667 shutdown_connections 2024-05-20T18:27:17.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.897+0000 7f6fcd596640 1 --2- 172.21.15.5:0/2819795667 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6fa806be30 0x7f6fa806e340 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:17.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.897+0000 7f6fcd596640 1 --2- 172.21.15.5:0/2819795667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fc815d940 0x7f6fc809e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:17.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.897+0000 7f6fcd596640 1 --2- 172.21.15.5:0/2819795667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fc80b2f10 0x7f6fc81535f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:17.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.897+0000 7f6fcd596640 1 --2- 172.21.15.5:0/2819795667 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6fc806d9a0 0x7f6fc8156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:17.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.897+0000 7f6fcd596640 1 -- 172.21.15.5:0/2819795667 >> 172.21.15.5:0/2819795667 conn(0x7f6fc80b6670 msgr2=0x7f6fc80b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:17.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.897+0000 7f6fcd596640 1 -- 172.21.15.5:0/2819795667 shutdown_connections 2024-05-20T18:27:17.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:17.897+0000 7f6fcd596640 1 -- 172.21.15.5:0/2819795667 wait complete. 2024-05-20T18:27:17.907 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:27:05.476596+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:27:06.137762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T18:27:09.829920+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T18:27:17.908 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:27:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.027+0000 7f6716dce640 1 Processor -- start 2024-05-20T18:27:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.027+0000 7f6716dce640 1 -- start start 2024-05-20T18:27:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.028+0000 7f6716dce640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f67100af330 0x7f67100af740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.028+0000 7f6716dce640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f67100b2560 0x7f67100d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.028+0000 7f6716dce640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f671006c840 con 0x7f67100d1610 2024-05-20T18:27:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.028+0000 7f6716dce640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f67100d20d0 con 0x7f67100b2560 2024-05-20T18:27:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.028+0000 7f6716dce640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f671006cae0 con 0x7f67100af330 2024-05-20T18:27:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.028+0000 7f671536b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f67100b2560 0x7f67100d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.029+0000 7f671536b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f67100b2560 0x7f67100d1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54492/0 (socket says 172.21.15.5:54492) 2024-05-20T18:27:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.029+0000 7f670ffff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f67100d1610 0x7f67100b1e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36500/0 (socket says 172.21.15.5:36500) 2024-05-20T18:27:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.029+0000 7f671536b640 1 -- 172.21.15.5:0/2803631300 learned_addr learned my addr 172.21.15.5:0/2803631300 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.029+0000 7f6714b6a640 1 --2- 172.21.15.5:0/2803631300 >> v2:172.21.15.63:3300/0 conn(0x7f67100af330 0x7f67100af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:18.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.029+0000 7f671536b640 1 -- 172.21.15.5:0/2803631300 >> v2:172.21.15.63:3300/0 conn(0x7f67100af330 msgr2=0x7f67100af740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:18.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.029+0000 7f671536b640 1 --2- 172.21.15.5:0/2803631300 >> v2:172.21.15.63:3300/0 conn(0x7f67100af330 0x7f67100af740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:18.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.029+0000 7f671536b640 1 -- 172.21.15.5:0/2803631300 >> v1:172.21.15.5:6789/0 conn(0x7f67100d1610 legacy=0x7f67100b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:18.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.029+0000 7f671536b640 1 -- 172.21.15.5:0/2803631300 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671006cdc0 con 0x7f67100b2560 2024-05-20T18:27:18.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.029+0000 7f670f7fe640 1 -- 172.21.15.5:0/2803631300 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2947801615 0 0) 0x7f671006c840 con 0x7f67100d1610 2024-05-20T18:27:18.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.029+0000 7f6714b6a640 1 --2- 172.21.15.5:0/2803631300 >> v2:172.21.15.63:3300/0 conn(0x7f67100af330 0x7f67100af740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:18.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.030+0000 7f671536b640 1 --2- 172.21.15.5:0/2803631300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67100b2560 0x7f67100d1a20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f67100729c0 tx=0x7f670802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2872344a2230da server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:18.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.030+0000 7f670f7fe640 1 -- 172.21.15.5:0/2803631300 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6708036070 con 0x7f67100b2560 2024-05-20T18:27:18.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.030+0000 7f670f7fe640 1 -- 172.21.15.5:0/2803631300 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6708031600 con 0x7f67100b2560 2024-05-20T18:27:18.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.030+0000 7f670f7fe640 1 -- 172.21.15.5:0/2803631300 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67080319d0 con 0x7f67100b2560 2024-05-20T18:27:18.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.030+0000 7f6716dce640 1 -- 172.21.15.5:0/2803631300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67100b2560 msgr2=0x7f67100d1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:18.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.030+0000 7f6716dce640 1 --2- 172.21.15.5:0/2803631300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67100b2560 0x7f67100d1a20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f67100729c0 tx=0x7f670802a6a0 comp rx=0 tx=0).stop 2024-05-20T18:27:18.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.031+0000 7f6716dce640 1 -- 172.21.15.5:0/2803631300 shutdown_connections 2024-05-20T18:27:18.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.031+0000 7f6716dce640 1 --2- 172.21.15.5:0/2803631300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67100b2560 0x7f67100d1a20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:18.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.031+0000 7f6716dce640 1 --2- 172.21.15.5:0/2803631300 >> v2:172.21.15.63:3300/0 conn(0x7f67100af330 0x7f67100af740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:18.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.031+0000 7f6716dce640 1 -- 172.21.15.5:0/2803631300 >> 172.21.15.5:0/2803631300 conn(0x7f67100b6670 msgr2=0x7f67100b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:18.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.031+0000 7f6716dce640 1 -- 172.21.15.5:0/2803631300 shutdown_connections 2024-05-20T18:27:18.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.031+0000 7f6716dce640 1 -- 172.21.15.5:0/2803631300 wait complete. 2024-05-20T18:27:18.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.032+0000 7f6716dce640 1 Processor -- start 2024-05-20T18:27:18.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.032+0000 7f6716dce640 1 -- start start 2024-05-20T18:27:18.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.034+0000 7f6716dce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67100af330 0x7f67101464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:18.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.034+0000 7f6716dce640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67100b2560 0x7f671006f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.034+0000 7f6716dce640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67100d1610 0x7f6710152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.034+0000 7f6716dce640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6710159d30 con 0x7f67100af330 2024-05-20T18:27:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.034+0000 7f6716dce640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f671015d350 con 0x7f67100b2560 2024-05-20T18:27:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.034+0000 7f6716dce640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6710159eb0 con 0x7f67100d1610 2024-05-20T18:27:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.034+0000 7f6714b6a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67100af330 0x7f67101464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.034+0000 7f6714b6a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67100af330 0x7f67101464b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54494/0 (socket says 172.21.15.5:54494) 2024-05-20T18:27:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.034+0000 7f6714b6a640 1 -- 172.21.15.5:0/702747612 learned_addr learned my addr 172.21.15.5:0/702747612 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.034+0000 7f6714b6a640 1 -- 172.21.15.5:0/702747612 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67100d1610 msgr2=0x7f6710152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.034+0000 7f671536b640 1 --2- 172.21.15.5:0/702747612 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67100d1610 0x7f6710152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.034+0000 7f6714b6a640 1 --2- 172.21.15.5:0/702747612 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67100d1610 0x7f6710152c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.034+0000 7f6714b6a640 1 -- 172.21.15.5:0/702747612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67100b2560 msgr2=0x7f671006f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.034+0000 7f670ffff640 1 --2- 172.21.15.5:0/702747612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67100b2560 0x7f671006f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.034+0000 7f6714b6a640 1 --2- 172.21.15.5:0/702747612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67100b2560 0x7f671006f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.034+0000 7f6714b6a640 1 -- 172.21.15.5:0/702747612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6710070130 con 0x7f67100af330 2024-05-20T18:27:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.035+0000 7f6714b6a640 1 --2- 172.21.15.5:0/702747612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67100af330 0x7f67101464b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f670400fa40 tx=0x7f670400ff00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.035+0000 7f670d7fa640 1 -- 172.21.15.5:0/702747612 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6704007c90 con 0x7f67100af330 2024-05-20T18:27:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.035+0000 7f670d7fa640 1 -- 172.21.15.5:0/702747612 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6704007e30 con 0x7f67100af330 2024-05-20T18:27:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.035+0000 7f670d7fa640 1 -- 172.21.15.5:0/702747612 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6704011670 con 0x7f67100af330 2024-05-20T18:27:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.035+0000 7f6716dce640 1 -- 172.21.15.5:0/702747612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6710070410 con 0x7f67100af330 2024-05-20T18:27:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.035+0000 7f6716dce640 1 -- 172.21.15.5:0/702747612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67100709d0 con 0x7f67100af330 2024-05-20T18:27:18.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.036+0000 7f6716dce640 1 -- 172.21.15.5:0/702747612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6710075b00 con 0x7f67100af330 2024-05-20T18:27:18.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.036+0000 7f670d7fa640 1 -- 172.21.15.5:0/702747612 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6704019020 con 0x7f67100af330 2024-05-20T18:27:18.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.037+0000 7f670d7fa640 1 --2- 172.21.15.5:0/702747612 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f66f406be30 0x7f66f406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:18.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.037+0000 7f670d7fa640 1 -- 172.21.15.5:0/702747612 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f670408d460 con 0x7f67100af330 2024-05-20T18:27:18.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.037+0000 7f670ffff640 1 --2- 172.21.15.5:0/702747612 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f66f406be30 0x7f66f406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:18.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.037+0000 7f670ffff640 1 --2- 172.21.15.5:0/702747612 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f66f406be30 0x7f66f406e340 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f67100d1090 tx=0x7f66fc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:18.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.042+0000 7f670d7fa640 1 -- 172.21.15.5:0/702747612 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f670405ba30 con 0x7f67100af330 2024-05-20T18:27:18.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.225+0000 7f6716dce640 1 -- 172.21.15.5:0/702747612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f671015d760 con 0x7f67100af330 2024-05-20T18:27:18.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.225+0000 7f670d7fa640 1 -- 172.21.15.5:0/702747612 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f670405b330 con 0x7f67100af330 2024-05-20T18:27:18.224 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:27:18.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.227+0000 7f6716dce640 1 -- 172.21.15.5:0/702747612 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f66f406be30 msgr2=0x7f66f406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:18.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.227+0000 7f6716dce640 1 --2- 172.21.15.5:0/702747612 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f66f406be30 0x7f66f406e340 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f67100d1090 tx=0x7f66fc009040 comp rx=0 tx=0).stop 2024-05-20T18:27:18.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.227+0000 7f6716dce640 1 -- 172.21.15.5:0/702747612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67100af330 msgr2=0x7f67101464b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:18.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.227+0000 7f6716dce640 1 --2- 172.21.15.5:0/702747612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67100af330 0x7f67101464b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f670400fa40 tx=0x7f670400ff00 comp rx=0 tx=0).stop 2024-05-20T18:27:18.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.227+0000 7f6716dce640 1 -- 172.21.15.5:0/702747612 shutdown_connections 2024-05-20T18:27:18.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.227+0000 7f6716dce640 1 --2- 172.21.15.5:0/702747612 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f66f406be30 0x7f66f406e340 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:18.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.227+0000 7f6716dce640 1 --2- 172.21.15.5:0/702747612 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67100d1610 0x7f6710152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:18.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.227+0000 7f6716dce640 1 --2- 172.21.15.5:0/702747612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67100b2560 0x7f671006f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:18.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.227+0000 7f6716dce640 1 --2- 172.21.15.5:0/702747612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67100af330 0x7f67101464b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:18.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.227+0000 7f6716dce640 1 -- 172.21.15.5:0/702747612 >> 172.21.15.5:0/702747612 conn(0x7f67100b6670 msgr2=0x7f6710158fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:18.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.228+0000 7f6716dce640 1 -- 172.21.15.5:0/702747612 shutdown_connections 2024-05-20T18:27:18.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.228+0000 7f6716dce640 1 -- 172.21.15.5:0/702747612 wait complete. 2024-05-20T18:27:18.237 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:27:18.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.356+0000 7f671c4e9640 1 Processor -- start 2024-05-20T18:27:18.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.356+0000 7f671c4e9640 1 -- start start 2024-05-20T18:27:18.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.358+0000 7f671c4e9640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f67140b4010 0x7f67140d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:18.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.358+0000 7f671c4e9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f67140d14b0 0x7f67140d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:18.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.358+0000 7f671c4e9640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67140d0370 con 0x7f6714163a10 2024-05-20T18:27:18.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.358+0000 7f671c4e9640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f67140d3ca0 con 0x7f67140d14b0 2024-05-20T18:27:18.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.358+0000 7f671c4e9640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f67140d0650 con 0x7f67140b4010 2024-05-20T18:27:18.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.358+0000 7f671aa86640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f67140d14b0 0x7f67140d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:18.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.358+0000 7f671aa86640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f67140d14b0 0x7f67140d18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54496/0 (socket says 172.21.15.5:54496) 2024-05-20T18:27:18.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.358+0000 7f671aa86640 1 -- 172.21.15.5:0/222931954 learned_addr learned my addr 172.21.15.5:0/222931954 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:18.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.358+0000 7f6719a84640 1 --2- 172.21.15.5:0/222931954 >> v2:172.21.15.63:3300/0 conn(0x7f67140b4010 0x7f67140d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:18.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.358+0000 7f671aa86640 1 -- 172.21.15.5:0/222931954 >> v2:172.21.15.63:3300/0 conn(0x7f67140b4010 msgr2=0x7f67140d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:18.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.358+0000 7f671aa86640 1 --2- 172.21.15.5:0/222931954 >> v2:172.21.15.63:3300/0 conn(0x7f67140b4010 0x7f67140d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:18.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.358+0000 7f671aa86640 1 -- 172.21.15.5:0/222931954 >> v1:172.21.15.63:6789/0 conn(0x7f6714163a10 legacy=0x7f67140cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:18.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.359+0000 7f671aa86640 1 -- 172.21.15.5:0/222931954 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67140d3650 con 0x7f67140d14b0 2024-05-20T18:27:18.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.359+0000 7f671aa86640 1 --2- 172.21.15.5:0/222931954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67140d14b0 0x7f67140d18c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f67140b7f80 tx=0x7f670c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bbd6933b197cc34 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:18.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.360+0000 7f6719283640 1 -- 172.21.15.5:0/222931954 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c037070 con 0x7f67140d14b0 2024-05-20T18:27:18.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.360+0000 7f6719283640 1 -- 172.21.15.5:0/222931954 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f670c032440 con 0x7f67140d14b0 2024-05-20T18:27:18.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.360+0000 7f6719283640 1 -- 172.21.15.5:0/222931954 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c032810 con 0x7f67140d14b0 2024-05-20T18:27:18.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.360+0000 7f671c4e9640 1 -- 172.21.15.5:0/222931954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67140d14b0 msgr2=0x7f67140d18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:18.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.360+0000 7f671c4e9640 1 --2- 172.21.15.5:0/222931954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67140d14b0 0x7f67140d18c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f67140b7f80 tx=0x7f670c02a270 comp rx=0 tx=0).stop 2024-05-20T18:27:18.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.360+0000 7f671c4e9640 1 -- 172.21.15.5:0/222931954 shutdown_connections 2024-05-20T18:27:18.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.360+0000 7f671c4e9640 1 --2- 172.21.15.5:0/222931954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67140d14b0 0x7f67140d18c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:18.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.360+0000 7f671c4e9640 1 --2- 172.21.15.5:0/222931954 >> v2:172.21.15.63:3300/0 conn(0x7f67140b4010 0x7f67140d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:18.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.360+0000 7f671c4e9640 1 -- 172.21.15.5:0/222931954 >> 172.21.15.5:0/222931954 conn(0x7f671406d240 msgr2=0x7f671406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:18.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.360+0000 7f671c4e9640 1 -- 172.21.15.5:0/222931954 shutdown_connections 2024-05-20T18:27:18.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.360+0000 7f671c4e9640 1 -- 172.21.15.5:0/222931954 wait complete. 2024-05-20T18:27:18.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.362+0000 7f671c4e9640 1 Processor -- start 2024-05-20T18:27:18.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.362+0000 7f671c4e9640 1 -- start start 2024-05-20T18:27:18.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.364+0000 7f671c4e9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67140b4010 0x7f6714157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:18.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.364+0000 7f671c4e9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67140d14b0 0x7f67140ce260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:18.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.364+0000 7f671c4e9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6714163a10 0x7f6714148ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:18.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.364+0000 7f671c4e9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f67140b8210 con 0x7f67140b4010 2024-05-20T18:27:18.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.364+0000 7f671c4e9640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f67140d1e00 con 0x7f6714163a10 2024-05-20T18:27:18.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.364+0000 7f671c4e9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f67140b8390 con 0x7f67140d14b0 2024-05-20T18:27:18.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.364+0000 7f671a285640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67140b4010 0x7f6714157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:18.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.364+0000 7f671a285640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67140b4010 0x7f6714157bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54508/0 (socket says 172.21.15.5:54508) 2024-05-20T18:27:18.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.364+0000 7f671a285640 1 -- 172.21.15.5:0/2376983379 learned_addr learned my addr 172.21.15.5:0/2376983379 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:18.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.364+0000 7f6719a84640 1 --2- 172.21.15.5:0/2376983379 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67140d14b0 0x7f67140ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:18.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.364+0000 7f671aa86640 1 --2- 172.21.15.5:0/2376983379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6714163a10 0x7f6714148ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:18.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.364+0000 7f671a285640 1 -- 172.21.15.5:0/2376983379 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67140d14b0 msgr2=0x7f67140ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:18.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.364+0000 7f671a285640 1 --2- 172.21.15.5:0/2376983379 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67140d14b0 0x7f67140ce260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:18.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.364+0000 7f671a285640 1 -- 172.21.15.5:0/2376983379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6714163a10 msgr2=0x7f6714148ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:18.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.364+0000 7f671a285640 1 --2- 172.21.15.5:0/2376983379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6714163a10 0x7f6714148ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:18.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.364+0000 7f671a285640 1 -- 172.21.15.5:0/2376983379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6714158110 con 0x7f67140b4010 2024-05-20T18:27:18.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.364+0000 7f6719a84640 1 --2- 172.21.15.5:0/2376983379 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67140d14b0 0x7f67140ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:18.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.364+0000 7f671a285640 1 --2- 172.21.15.5:0/2376983379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67140b4010 0x7f6714157bd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f671000c6b0 tx=0x7f671000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:18.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.364+0000 7f670affd640 1 -- 172.21.15.5:0/2376983379 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6710016020 con 0x7f67140b4010 2024-05-20T18:27:18.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.364+0000 7f670affd640 1 -- 172.21.15.5:0/2376983379 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6710006d50 con 0x7f67140b4010 2024-05-20T18:27:18.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.364+0000 7f671c4e9640 1 -- 172.21.15.5:0/2376983379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f671415a750 con 0x7f67140b4010 2024-05-20T18:27:18.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.364+0000 7f670affd640 1 -- 172.21.15.5:0/2376983379 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6710003bb0 con 0x7f67140b4010 2024-05-20T18:27:18.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.365+0000 7f671c4e9640 1 -- 172.21.15.5:0/2376983379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f671415ad10 con 0x7f67140b4010 2024-05-20T18:27:18.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.365+0000 7f671c4e9640 1 -- 172.21.15.5:0/2376983379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6714163830 con 0x7f67140b4010 2024-05-20T18:27:18.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.366+0000 7f670affd640 1 -- 172.21.15.5:0/2376983379 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f67100050d0 con 0x7f67140b4010 2024-05-20T18:27:18.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.366+0000 7f670affd640 1 --2- 172.21.15.5:0/2376983379 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f66f806be30 0x7f66f806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:18.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.367+0000 7f6719a84640 1 --2- 172.21.15.5:0/2376983379 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f66f806be30 0x7f66f806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:18.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.367+0000 7f670affd640 1 -- 172.21.15.5:0/2376983379 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f671008ae70 con 0x7f67140b4010 2024-05-20T18:27:18.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.367+0000 7f6719a84640 1 --2- 172.21.15.5:0/2376983379 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f66f806be30 0x7f66f806e340 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f67140b7270 tx=0x7f6704008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:18.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.371+0000 7f670affd640 1 -- 172.21.15.5:0/2376983379 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6714163830 con 0x7f67140b4010 2024-05-20T18:27:18.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.554+0000 7f671c4e9640 1 -- 172.21.15.5:0/2376983379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f67140d23e0 con 0x7f67140b4010 2024-05-20T18:27:18.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.554+0000 7f670affd640 1 -- 172.21.15.5:0/2376983379 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f67140d23e0 con 0x7f67140b4010 2024-05-20T18:27:18.552 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:27:18.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.556+0000 7f671c4e9640 1 -- 172.21.15.5:0/2376983379 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f66f806be30 msgr2=0x7f66f806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:18.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.556+0000 7f671c4e9640 1 --2- 172.21.15.5:0/2376983379 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f66f806be30 0x7f66f806e340 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f67140b7270 tx=0x7f6704008040 comp rx=0 tx=0).stop 2024-05-20T18:27:18.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.556+0000 7f671c4e9640 1 -- 172.21.15.5:0/2376983379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67140b4010 msgr2=0x7f6714157bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:18.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.556+0000 7f671c4e9640 1 --2- 172.21.15.5:0/2376983379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67140b4010 0x7f6714157bd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f671000c6b0 tx=0x7f671000cb70 comp rx=0 tx=0).stop 2024-05-20T18:27:18.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.556+0000 7f671c4e9640 1 -- 172.21.15.5:0/2376983379 shutdown_connections 2024-05-20T18:27:18.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.556+0000 7f671c4e9640 1 --2- 172.21.15.5:0/2376983379 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f66f806be30 0x7f66f806e340 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:18.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.556+0000 7f671c4e9640 1 --2- 172.21.15.5:0/2376983379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6714163a10 0x7f6714148ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:18.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.556+0000 7f671c4e9640 1 --2- 172.21.15.5:0/2376983379 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67140d14b0 0x7f67140ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:18.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.556+0000 7f671c4e9640 1 --2- 172.21.15.5:0/2376983379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67140b4010 0x7f6714157bd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:18.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.556+0000 7f671c4e9640 1 -- 172.21.15.5:0/2376983379 >> 172.21.15.5:0/2376983379 conn(0x7f671406d240 msgr2=0x7f671415f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:18.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.556+0000 7f671c4e9640 1 -- 172.21.15.5:0/2376983379 shutdown_connections 2024-05-20T18:27:18.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.556+0000 7f671c4e9640 1 -- 172.21.15.5:0/2376983379 wait complete. 2024-05-20T18:27:18.565 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:27:18.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.684+0000 7f60c4138640 1 Processor -- start 2024-05-20T18:27:18.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.685+0000 7f60c4138640 1 -- start start 2024-05-20T18:27:18.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.686+0000 7f60c4138640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f60bc0b6620 0x7f60bc0ba3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:18.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.686+0000 7f60c4138640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f60bc162ce0 0x7f60bc0b8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:18.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.686+0000 7f60c4138640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60bc1634a0 con 0x7f60bc0b85b0 2024-05-20T18:27:18.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.686+0000 7f60c4138640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f60bc0d3480 con 0x7f60bc0b6620 2024-05-20T18:27:18.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.686+0000 7f60c4138640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f60bc163780 con 0x7f60bc162ce0 2024-05-20T18:27:18.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.686+0000 7f60c1ed4640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f60bc0b6620 0x7f60bc0ba3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:18.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.686+0000 7f60c26d5640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f60bc0b85b0 0x7f60bc0d3bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36514/0 (socket says 172.21.15.5:36514) 2024-05-20T18:27:18.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.686+0000 7f60c26d5640 1 -- 172.21.15.5:0/2550479750 learned_addr learned my addr 172.21.15.5:0/2550479750 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:18.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.686+0000 7f60c16d3640 1 --2- 172.21.15.5:0/2550479750 >> v2:172.21.15.63:3300/0 conn(0x7f60bc162ce0 0x7f60bc0b8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:18.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.687+0000 7f60c1ed4640 1 -- 172.21.15.5:0/2550479750 >> v2:172.21.15.63:3300/0 conn(0x7f60bc162ce0 msgr2=0x7f60bc0b8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:18.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.687+0000 7f60c1ed4640 1 --2- 172.21.15.5:0/2550479750 >> v2:172.21.15.63:3300/0 conn(0x7f60bc162ce0 0x7f60bc0b8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.687+0000 7f60c1ed4640 1 -- 172.21.15.5:0/2550479750 >> v1:172.21.15.5:6789/0 conn(0x7f60bc0b85b0 legacy=0x7f60bc0d3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.687+0000 7f60c1ed4640 1 -- 172.21.15.5:0/2550479750 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60bc163a60 con 0x7f60bc0b6620 2024-05-20T18:27:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.687+0000 7f60c0ed2640 1 -- 172.21.15.5:0/2550479750 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 866309441 0 0) 0x7f60bc1634a0 con 0x7f60bc0b85b0 2024-05-20T18:27:18.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.687+0000 7f60c1ed4640 1 --2- 172.21.15.5:0/2550479750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60bc0b6620 0x7f60bc0ba3c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f60bc0b7790 tx=0x7f60ac02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d348e79561bc86b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:18.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.687+0000 7f60c0ed2640 1 -- 172.21.15.5:0/2550479750 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac007740 con 0x7f60bc0b6620 2024-05-20T18:27:18.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.688+0000 7f60c0ed2640 1 -- 172.21.15.5:0/2550479750 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60ac0078e0 con 0x7f60bc0b6620 2024-05-20T18:27:18.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.688+0000 7f60c0ed2640 1 -- 172.21.15.5:0/2550479750 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac007cb0 con 0x7f60bc0b6620 2024-05-20T18:27:18.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.688+0000 7f60c4138640 1 -- 172.21.15.5:0/2550479750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60bc0b6620 msgr2=0x7f60bc0ba3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:18.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.688+0000 7f60c4138640 1 --2- 172.21.15.5:0/2550479750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60bc0b6620 0x7f60bc0ba3c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f60bc0b7790 tx=0x7f60ac02a680 comp rx=0 tx=0).stop 2024-05-20T18:27:18.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.688+0000 7f60c4138640 1 -- 172.21.15.5:0/2550479750 shutdown_connections 2024-05-20T18:27:18.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.688+0000 7f60c4138640 1 --2- 172.21.15.5:0/2550479750 >> v2:172.21.15.63:3300/0 conn(0x7f60bc162ce0 0x7f60bc0b8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:18.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.688+0000 7f60c4138640 1 --2- 172.21.15.5:0/2550479750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60bc0b6620 0x7f60bc0ba3c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:18.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.688+0000 7f60c4138640 1 -- 172.21.15.5:0/2550479750 >> 172.21.15.5:0/2550479750 conn(0x7f60bc06d240 msgr2=0x7f60bc06d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:18.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.688+0000 7f60c4138640 1 -- 172.21.15.5:0/2550479750 shutdown_connections 2024-05-20T18:27:18.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.688+0000 7f60c4138640 1 -- 172.21.15.5:0/2550479750 wait complete. 2024-05-20T18:27:18.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.690+0000 7f60c4138640 1 Processor -- start 2024-05-20T18:27:18.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.690+0000 7f60c4138640 1 -- start start 2024-05-20T18:27:18.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.691+0000 7f60c4138640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60bc0b6620 0x7f60bc0cc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:18.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.691+0000 7f60c4138640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60bc0b85b0 0x7f60bc0ce500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:18.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.691+0000 7f60c4138640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60bc162ce0 0x7f60bc0ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:18.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.691+0000 7f60c4138640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc1630f0 con 0x7f60bc0b6620 2024-05-20T18:27:18.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.691+0000 7f60c4138640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc0d4300 con 0x7f60bc0b85b0 2024-05-20T18:27:18.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.691+0000 7f60c4138640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f60bc0cccc0 con 0x7f60bc162ce0 2024-05-20T18:27:18.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.692+0000 7f60c1ed4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60bc0b6620 0x7f60bc0cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:18.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.692+0000 7f60c1ed4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60bc0b6620 0x7f60bc0cc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54536/0 (socket says 172.21.15.5:54536) 2024-05-20T18:27:18.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.692+0000 7f60c1ed4640 1 -- 172.21.15.5:0/3735434328 learned_addr learned my addr 172.21.15.5:0/3735434328 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:18.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.692+0000 7f60c16d3640 1 --2- 172.21.15.5:0/3735434328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60bc0b85b0 0x7f60bc0ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:18.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.692+0000 7f60c26d5640 1 --2- 172.21.15.5:0/3735434328 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60bc162ce0 0x7f60bc0ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:18.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.692+0000 7f60c1ed4640 1 -- 172.21.15.5:0/3735434328 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60bc162ce0 msgr2=0x7f60bc0ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:18.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.692+0000 7f60c1ed4640 1 --2- 172.21.15.5:0/3735434328 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60bc162ce0 0x7f60bc0ca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:18.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.692+0000 7f60c1ed4640 1 -- 172.21.15.5:0/3735434328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60bc0b85b0 msgr2=0x7f60bc0ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:18.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.692+0000 7f60c1ed4640 1 --2- 172.21.15.5:0/3735434328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60bc0b85b0 0x7f60bc0ce500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:18.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.692+0000 7f60c1ed4640 1 -- 172.21.15.5:0/3735434328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60bc0cadd0 con 0x7f60bc0b6620 2024-05-20T18:27:18.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.692+0000 7f60c16d3640 1 --2- 172.21.15.5:0/3735434328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60bc0b85b0 0x7f60bc0ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:18.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.692+0000 7f60c1ed4640 1 --2- 172.21.15.5:0/3735434328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60bc0b6620 0x7f60bc0cc350 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f60ac009e80 tx=0x7f60ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:18.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.692+0000 7f60ba7fc640 1 -- 172.21.15.5:0/3735434328 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac03f070 con 0x7f60bc0b6620 2024-05-20T18:27:18.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.692+0000 7f60ba7fc640 1 -- 172.21.15.5:0/3735434328 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60ac031c40 con 0x7f60bc0b6620 2024-05-20T18:27:18.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.692+0000 7f60c4138640 1 -- 172.21.15.5:0/3735434328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60bc15cb50 con 0x7f60bc0b6620 2024-05-20T18:27:18.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.692+0000 7f60ba7fc640 1 -- 172.21.15.5:0/3735434328 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac044460 con 0x7f60bc0b6620 2024-05-20T18:27:18.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.692+0000 7f60c4138640 1 -- 172.21.15.5:0/3735434328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60bc15d0d0 con 0x7f60bc0b6620 2024-05-20T18:27:18.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.693+0000 7f60c4138640 1 -- 172.21.15.5:0/3735434328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60bc075b00 con 0x7f60bc0b6620 2024-05-20T18:27:18.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.694+0000 7f60ba7fc640 1 -- 172.21.15.5:0/3735434328 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f60ac03b410 con 0x7f60bc0b6620 2024-05-20T18:27:18.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.694+0000 7f60ba7fc640 1 --2- 172.21.15.5:0/3735434328 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f60a806be30 0x7f60a806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:18.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.694+0000 7f60ba7fc640 1 -- 172.21.15.5:0/3735434328 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f60ac0acb40 con 0x7f60bc0b6620 2024-05-20T18:27:18.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.695+0000 7f60c16d3640 1 --2- 172.21.15.5:0/3735434328 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f60a806be30 0x7f60a806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:18.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.695+0000 7f60c16d3640 1 --2- 172.21.15.5:0/3735434328 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f60a806be30 0x7f60a806e340 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f60b0006fd0 tx=0x7f60b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:18.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.699+0000 7f60ba7fc640 1 -- 172.21.15.5:0/3735434328 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f60ac07b110 con 0x7f60bc0b6620 2024-05-20T18:27:18.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.882+0000 7f60c4138640 1 -- 172.21.15.5:0/3735434328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f60bc0d2fa0 con 0x7f60bc0b6620 2024-05-20T18:27:18.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.883+0000 7f60ba7fc640 1 -- 172.21.15.5:0/3735434328 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f60ac07aa10 con 0x7f60bc0b6620 2024-05-20T18:27:18.881 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:27:18.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.885+0000 7f60c4138640 1 -- 172.21.15.5:0/3735434328 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f60a806be30 msgr2=0x7f60a806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:18.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.885+0000 7f60c4138640 1 --2- 172.21.15.5:0/3735434328 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f60a806be30 0x7f60a806e340 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f60b0006fd0 tx=0x7f60b0008040 comp rx=0 tx=0).stop 2024-05-20T18:27:18.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.885+0000 7f60c4138640 1 -- 172.21.15.5:0/3735434328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60bc0b6620 msgr2=0x7f60bc0cc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:18.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.885+0000 7f60c4138640 1 --2- 172.21.15.5:0/3735434328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60bc0b6620 0x7f60bc0cc350 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f60ac009e80 tx=0x7f60ac004a40 comp rx=0 tx=0).stop 2024-05-20T18:27:18.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.885+0000 7f60c4138640 1 -- 172.21.15.5:0/3735434328 shutdown_connections 2024-05-20T18:27:18.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.885+0000 7f60c4138640 1 --2- 172.21.15.5:0/3735434328 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f60a806be30 0x7f60a806e340 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:18.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.885+0000 7f60c4138640 1 --2- 172.21.15.5:0/3735434328 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60bc162ce0 0x7f60bc0ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:18.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.885+0000 7f60c4138640 1 --2- 172.21.15.5:0/3735434328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60bc0b85b0 0x7f60bc0ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:18.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.885+0000 7f60c4138640 1 --2- 172.21.15.5:0/3735434328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60bc0b6620 0x7f60bc0cc350 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:18.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.885+0000 7f60c4138640 1 -- 172.21.15.5:0/3735434328 >> 172.21.15.5:0/3735434328 conn(0x7f60bc06d240 msgr2=0x7f60bc0b8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:18.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.885+0000 7f60c4138640 1 -- 172.21.15.5:0/3735434328 shutdown_connections 2024-05-20T18:27:18.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:18.885+0000 7f60c4138640 1 -- 172.21.15.5:0/3735434328 wait complete. 2024-05-20T18:27:18.894 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T18:27:19.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.014+0000 7f273ebaa640 1 Processor -- start 2024-05-20T18:27:19.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.014+0000 7f273ebaa640 1 -- start start 2024-05-20T18:27:19.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.015+0000 7f273ebaa640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2738161350 0x7f2738161760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:19.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.015+0000 7f273ebaa640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27380d1830 0x7f27380d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:19.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.015+0000 7f273ebaa640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2738160b90 con 0x7f273806b520 2024-05-20T18:27:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.015+0000 7f273ebaa640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f27380d2180 con 0x7f2738161350 2024-05-20T18:27:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.015+0000 7f273ebaa640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f273815de60 con 0x7f27380d1830 2024-05-20T18:27:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.016+0000 7f273c946640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f273806b520 0x7f273815d540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36520/0 (socket says 172.21.15.5:36520) 2024-05-20T18:27:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.016+0000 7f273c946640 1 -- 172.21.15.5:0/2793857992 learned_addr learned my addr 172.21.15.5:0/2793857992 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.016+0000 7f2737fff640 1 --2- 172.21.15.5:0/2793857992 >> v2:172.21.15.5:3300/0 conn(0x7f2738161350 0x7f2738161760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.016+0000 7f273d147640 1 --2- 172.21.15.5:0/2793857992 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27380d1830 0x7f27380d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.016+0000 7f2737fff640 1 -- 172.21.15.5:0/2793857992 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27380d1830 msgr2=0x7f27380d1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.016+0000 7f2737fff640 1 --2- 172.21.15.5:0/2793857992 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27380d1830 0x7f27380d1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.016+0000 7f2737fff640 1 -- 172.21.15.5:0/2793857992 >> v1:172.21.15.5:6789/0 conn(0x7f273806b520 legacy=0x7f273815d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.016+0000 7f2737fff640 1 -- 172.21.15.5:0/2793857992 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f273815e140 con 0x7f2738161350 2024-05-20T18:27:19.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.016+0000 7f27377fe640 1 -- 172.21.15.5:0/2793857992 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 757031387 0 0) 0x7f2738160b90 con 0x7f273806b520 2024-05-20T18:27:19.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.017+0000 7f2737fff640 1 --2- 172.21.15.5:0/2793857992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2738161350 0x7f2738161760 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f27240097a0 tx=0x7f272402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163245e3244bb509 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:19.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.017+0000 7f27377fe640 1 -- 172.21.15.5:0/2793857992 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2724037070 con 0x7f2738161350 2024-05-20T18:27:19.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.017+0000 7f27377fe640 1 -- 172.21.15.5:0/2793857992 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2724032440 con 0x7f2738161350 2024-05-20T18:27:19.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.017+0000 7f27377fe640 1 -- 172.21.15.5:0/2793857992 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2724032810 con 0x7f2738161350 2024-05-20T18:27:19.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.017+0000 7f273ebaa640 1 -- 172.21.15.5:0/2793857992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2738161350 msgr2=0x7f2738161760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:19.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.017+0000 7f273ebaa640 1 --2- 172.21.15.5:0/2793857992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2738161350 0x7f2738161760 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f27240097a0 tx=0x7f272402a180 comp rx=0 tx=0).stop 2024-05-20T18:27:19.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.018+0000 7f273ebaa640 1 -- 172.21.15.5:0/2793857992 shutdown_connections 2024-05-20T18:27:19.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.018+0000 7f273ebaa640 1 --2- 172.21.15.5:0/2793857992 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27380d1830 0x7f27380d1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:19.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.018+0000 7f273ebaa640 1 --2- 172.21.15.5:0/2793857992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2738161350 0x7f2738161760 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:19.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.018+0000 7f273ebaa640 1 -- 172.21.15.5:0/2793857992 >> 172.21.15.5:0/2793857992 conn(0x7f27380b6670 msgr2=0x7f27380b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:19.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.018+0000 7f273ebaa640 1 -- 172.21.15.5:0/2793857992 shutdown_connections 2024-05-20T18:27:19.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.018+0000 7f273ebaa640 1 -- 172.21.15.5:0/2793857992 wait complete. 2024-05-20T18:27:19.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.019+0000 7f273ebaa640 1 Processor -- start 2024-05-20T18:27:19.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.019+0000 7f273ebaa640 1 -- start start 2024-05-20T18:27:19.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.021+0000 7f273ebaa640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f273806b520 0x7f2738142410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:19.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.021+0000 7f273ebaa640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27380d1830 0x7f273815cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:19.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.021+0000 7f273ebaa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2738161350 0x7f273815d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:19.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.021+0000 7f273ebaa640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2738143740 con 0x7f2738161350 2024-05-20T18:27:19.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.021+0000 7f273ebaa640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2738160e30 con 0x7f27380d1830 2024-05-20T18:27:19.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.021+0000 7f273ebaa640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f273815e970 con 0x7f273806b520 2024-05-20T18:27:19.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.021+0000 7f273d147640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2738161350 0x7f273815d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:19.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.021+0000 7f273d147640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2738161350 0x7f273815d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54540/0 (socket says 172.21.15.5:54540) 2024-05-20T18:27:19.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.021+0000 7f273d147640 1 -- 172.21.15.5:0/1421854379 learned_addr learned my addr 172.21.15.5:0/1421854379 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:19.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.021+0000 7f273c946640 1 --2- 172.21.15.5:0/1421854379 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f273806b520 0x7f2738142410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:19.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.021+0000 7f2737fff640 1 --2- 172.21.15.5:0/1421854379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27380d1830 0x7f273815cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:19.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.021+0000 7f273d147640 1 -- 172.21.15.5:0/1421854379 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f273806b520 msgr2=0x7f2738142410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:19.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.021+0000 7f273d147640 1 --2- 172.21.15.5:0/1421854379 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f273806b520 0x7f2738142410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:19.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.021+0000 7f273d147640 1 -- 172.21.15.5:0/1421854379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27380d1830 msgr2=0x7f273815cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:19.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.021+0000 7f273d147640 1 --2- 172.21.15.5:0/1421854379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27380d1830 0x7f273815cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:19.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.021+0000 7f273d147640 1 -- 172.21.15.5:0/1421854379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27380b3380 con 0x7f2738161350 2024-05-20T18:27:19.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.022+0000 7f273d147640 1 --2- 172.21.15.5:0/1421854379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2738161350 0x7f273815d100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2730009cb0 tx=0x7f273000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:19.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.022+0000 7f27357fa640 1 -- 172.21.15.5:0/1421854379 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2730018020 con 0x7f2738161350 2024-05-20T18:27:19.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.022+0000 7f27357fa640 1 -- 172.21.15.5:0/1421854379 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2730014480 con 0x7f2738161350 2024-05-20T18:27:19.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.022+0000 7f273ebaa640 1 -- 172.21.15.5:0/1421854379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27381f3660 con 0x7f2738161350 2024-05-20T18:27:19.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.022+0000 7f27357fa640 1 -- 172.21.15.5:0/1421854379 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2730014870 con 0x7f2738161350 2024-05-20T18:27:19.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.022+0000 7f273ebaa640 1 -- 172.21.15.5:0/1421854379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27381f3be0 con 0x7f2738161350 2024-05-20T18:27:19.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.023+0000 7f273ebaa640 1 -- 172.21.15.5:0/1421854379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2738075b00 con 0x7f2738161350 2024-05-20T18:27:19.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.023+0000 7f27357fa640 1 -- 172.21.15.5:0/1421854379 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2730005d10 con 0x7f2738161350 2024-05-20T18:27:19.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.024+0000 7f27357fa640 1 --2- 172.21.15.5:0/1421854379 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f271c06be60 0x7f271c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:19.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.024+0000 7f27357fa640 1 -- 172.21.15.5:0/1421854379 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f273008bdb0 con 0x7f2738161350 2024-05-20T18:27:19.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.024+0000 7f273c946640 1 --2- 172.21.15.5:0/1421854379 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f271c06be60 0x7f271c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:19.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.024+0000 7f273c946640 1 --2- 172.21.15.5:0/1421854379 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f271c06be60 0x7f271c06e370 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f272c000f60 tx=0x7f272c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:19.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.029+0000 7f27357fa640 1 -- 172.21.15.5:0/1421854379 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2730022030 con 0x7f2738161350 2024-05-20T18:27:19.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.210+0000 7f273ebaa640 1 -- 172.21.15.5:0/1421854379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2738142d10 con 0x7f2738161350 2024-05-20T18:27:19.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.210+0000 7f27357fa640 1 -- 172.21.15.5:0/1421854379 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f27380b3380 con 0x7f2738161350 2024-05-20T18:27:19.209 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:27:19.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.212+0000 7f273ebaa640 1 -- 172.21.15.5:0/1421854379 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f271c06be60 msgr2=0x7f271c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.212+0000 7f273ebaa640 1 --2- 172.21.15.5:0/1421854379 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f271c06be60 0x7f271c06e370 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f272c000f60 tx=0x7f272c009040 comp rx=0 tx=0).stop 2024-05-20T18:27:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.212+0000 7f273ebaa640 1 -- 172.21.15.5:0/1421854379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2738161350 msgr2=0x7f273815d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.212+0000 7f273ebaa640 1 --2- 172.21.15.5:0/1421854379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2738161350 0x7f273815d100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2730009cb0 tx=0x7f273000c490 comp rx=0 tx=0).stop 2024-05-20T18:27:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.212+0000 7f273ebaa640 1 -- 172.21.15.5:0/1421854379 shutdown_connections 2024-05-20T18:27:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.212+0000 7f273ebaa640 1 --2- 172.21.15.5:0/1421854379 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f271c06be60 0x7f271c06e370 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.212+0000 7f273ebaa640 1 --2- 172.21.15.5:0/1421854379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2738161350 0x7f273815d100 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.212+0000 7f273ebaa640 1 --2- 172.21.15.5:0/1421854379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27380d1830 0x7f273815cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.212+0000 7f273ebaa640 1 --2- 172.21.15.5:0/1421854379 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f273806b520 0x7f2738142410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.212+0000 7f273ebaa640 1 -- 172.21.15.5:0/1421854379 >> 172.21.15.5:0/1421854379 conn(0x7f27380b6670 msgr2=0x7f273809a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.212+0000 7f273ebaa640 1 -- 172.21.15.5:0/1421854379 shutdown_connections 2024-05-20T18:27:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.212+0000 7f273ebaa640 1 -- 172.21.15.5:0/1421854379 wait complete. 2024-05-20T18:27:19.222 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T18:27:19.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.339+0000 7f1c72737640 1 Processor -- start 2024-05-20T18:27:19.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.339+0000 7f1c72737640 1 -- start start 2024-05-20T18:27:19.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.341+0000 7f1c72737640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1c6c162820 0x7f1c6c0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:19.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.341+0000 7f1c72737640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c6c0d0630 con 0x7f1c6c0b3e80 2024-05-20T18:27:19.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.341+0000 7f1c72737640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c6c0d39c0 con 0x7f1c6c06dae0 2024-05-20T18:27:19.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.341+0000 7f1c72737640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1c6c0d3ca0 con 0x7f1c6c162820 2024-05-20T18:27:19.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.341+0000 7f1c63fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1c6c0b3e80 0x7f1c6c06d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36524/0 (socket says 172.21.15.5:36524) 2024-05-20T18:27:19.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.341+0000 7f1c63fff640 1 -- 172.21.15.5:0/2281231534 learned_addr learned my addr 172.21.15.5:0/2281231534 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:19.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.341+0000 7f1c6bfff640 1 --2- 172.21.15.5:0/2281231534 >> v2:172.21.15.63:3300/0 conn(0x7f1c6c162820 0x7f1c6c0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:19.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.341+0000 7f1c6b7fe640 1 -- 172.21.15.5:0/2281231534 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1612513670 0 0) 0x7f1c6c0d0630 con 0x7f1c6c0b3e80 2024-05-20T18:27:19.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.341+0000 7f1c6b7fe640 1 -- 172.21.15.5:0/2281231534 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c58003520 con 0x7f1c6c0b3e80 2024-05-20T18:27:19.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.342+0000 7f1c6b7fe640 1 -- 172.21.15.5:0/2281231534 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 349285405 0 0) 0x7f1c58003520 con 0x7f1c6c0b3e80 2024-05-20T18:27:19.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.342+0000 7f1c6b7fe640 1 -- 172.21.15.5:0/2281231534 >> v2:172.21.15.63:3300/0 conn(0x7f1c6c162820 msgr2=0x7f1c6c0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:19.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.342+0000 7f1c6b7fe640 1 --2- 172.21.15.5:0/2281231534 >> v2:172.21.15.63:3300/0 conn(0x7f1c6c162820 0x7f1c6c0b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:19.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.342+0000 7f1c6b7fe640 1 -- 172.21.15.5:0/2281231534 >> v1:172.21.15.63:6789/0 conn(0x7f1c6c06dae0 legacy=0x7f1c6c0b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:19.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.342+0000 7f1c6b7fe640 1 -- 172.21.15.5:0/2281231534 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c6c0d3f80 con 0x7f1c6c0b3e80 2024-05-20T18:27:19.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.342+0000 7f1c6b7fe640 1 -- 172.21.15.5:0/2281231534 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f1c54001670 con 0x7f1c6c0b3e80 2024-05-20T18:27:19.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.342+0000 7f1c6b7fe640 1 -- 172.21.15.5:0/2281231534 >> v1:172.21.15.5:6789/0 conn(0x7f1c6c0b3e80 legacy=0x7f1c6c06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:19.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.342+0000 7f1c6b7fe640 1 --2- 172.21.15.5:0/2281231534 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c580039b0 0x7f1c58003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:19.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.342+0000 7f1c6b7fe640 1 --2- 172.21.15.5:0/2281231534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c58004800 0x7f1c58004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:19.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.342+0000 7f1c6b7fe640 1 --2- 172.21.15.5:0/2281231534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c580051c0 0x7f1c5800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:19.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.342+0000 7f1c6b7fe640 1 -- 172.21.15.5:0/2281231534 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c0d3ca0 con 0x7f1c58004800 2024-05-20T18:27:19.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.342+0000 7f1c6b7fe640 1 -- 172.21.15.5:0/2281231534 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5800c160 con 0x7f1c580051c0 2024-05-20T18:27:19.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.342+0000 7f1c6b7fe640 1 -- 172.21.15.5:0/2281231534 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1c5800c360 con 0x7f1c580039b0 2024-05-20T18:27:19.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.342+0000 7f1c63fff640 1 --2- 172.21.15.5:0/2281231534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c58004800 0x7f1c58004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:19.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.342+0000 7f1c6bfff640 1 --2- 172.21.15.5:0/2281231534 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c580039b0 0x7f1c58003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:19.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.342+0000 7f1c70cd4640 1 --2- 172.21.15.5:0/2281231534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c580051c0 0x7f1c5800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:19.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.342+0000 7f1c63fff640 1 -- 172.21.15.5:0/2281231534 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c580039b0 msgr2=0x7f1c58003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:19.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.342+0000 7f1c63fff640 1 --2- 172.21.15.5:0/2281231534 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c580039b0 0x7f1c58003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:19.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.342+0000 7f1c63fff640 1 -- 172.21.15.5:0/2281231534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c580051c0 msgr2=0x7f1c5800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:19.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.342+0000 7f1c63fff640 1 --2- 172.21.15.5:0/2281231534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c580051c0 0x7f1c5800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:19.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.342+0000 7f1c63fff640 1 -- 172.21.15.5:0/2281231534 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c58003520 con 0x7f1c58004800 2024-05-20T18:27:19.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.342+0000 7f1c70cd4640 1 -- 172.21.15.5:0/2281231534 reap_dead start 2024-05-20T18:27:19.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.343+0000 7f1c63fff640 1 --2- 172.21.15.5:0/2281231534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c58004800 0x7f1c58004c10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400c240 tx=0x7f1c58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d03ee530e179b943 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:19.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.343+0000 7f1c6b7fe640 1 -- 172.21.15.5:0/2281231534 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54007cf0 con 0x7f1c58004800 2024-05-20T18:27:19.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.343+0000 7f1c6b7fe640 1 -- 172.21.15.5:0/2281231534 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c540058d0 con 0x7f1c58004800 2024-05-20T18:27:19.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.343+0000 7f1c6b7fe640 1 -- 172.21.15.5:0/2281231534 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54005ca0 con 0x7f1c58004800 2024-05-20T18:27:19.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.344+0000 7f1c72737640 1 -- 172.21.15.5:0/2281231534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c58004800 msgr2=0x7f1c58004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:19.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.344+0000 7f1c72737640 1 --2- 172.21.15.5:0/2281231534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c58004800 0x7f1c58004c10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400c240 tx=0x7f1c58005830 comp rx=0 tx=0).stop 2024-05-20T18:27:19.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.344+0000 7f1c72737640 1 -- 172.21.15.5:0/2281231534 shutdown_connections 2024-05-20T18:27:19.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.344+0000 7f1c72737640 1 --2- 172.21.15.5:0/2281231534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c58004800 0x7f1c58004c10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:19.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.344+0000 7f1c72737640 1 -- 172.21.15.5:0/2281231534 >> 172.21.15.5:0/2281231534 conn(0x7f1c6c0d4bd0 msgr2=0x7f1c6c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:19.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.344+0000 7f1c72737640 1 -- 172.21.15.5:0/2281231534 shutdown_connections 2024-05-20T18:27:19.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.344+0000 7f1c72737640 1 -- 172.21.15.5:0/2281231534 wait complete. 2024-05-20T18:27:19.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.346+0000 7f1c72737640 1 Processor -- start 2024-05-20T18:27:19.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.346+0000 7f1c72737640 1 -- start start 2024-05-20T18:27:19.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.347+0000 7f1c72737640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c58004800 0x7f1c6c0cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:19.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.347+0000 7f1c72737640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c6c0cb430 0x7f1c6c0c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:19.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.347+0000 7f1c72737640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c6c0cdec0 0x7f1c6c0ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.347+0000 7f1c72737640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c0d4780 con 0x7f1c58004800 2024-05-20T18:27:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.347+0000 7f1c72737640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c0d0300 con 0x7f1c6c0cb430 2024-05-20T18:27:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.347+0000 7f1c72737640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1c6c0d4900 con 0x7f1c6c0cdec0 2024-05-20T18:27:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.347+0000 7f1c6bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c58004800 0x7f1c6c0cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.347+0000 7f1c6bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c58004800 0x7f1c6c0cc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54560/0 (socket says 172.21.15.5:54560) 2024-05-20T18:27:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.347+0000 7f1c6bfff640 1 -- 172.21.15.5:0/3733033653 learned_addr learned my addr 172.21.15.5:0/3733033653 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.347+0000 7f1c63fff640 1 --2- 172.21.15.5:0/3733033653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c6c0cb430 0x7f1c6c0c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.347+0000 7f1c70cd4640 1 --2- 172.21.15.5:0/3733033653 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c6c0cdec0 0x7f1c6c0ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.347+0000 7f1c6bfff640 1 -- 172.21.15.5:0/3733033653 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c6c0cdec0 msgr2=0x7f1c6c0ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.348+0000 7f1c6bfff640 1 --2- 172.21.15.5:0/3733033653 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c6c0cdec0 0x7f1c6c0ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.348+0000 7f1c6bfff640 1 -- 172.21.15.5:0/3733033653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c6c0cb430 msgr2=0x7f1c6c0c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.348+0000 7f1c6bfff640 1 --2- 172.21.15.5:0/3733033653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c6c0cb430 0x7f1c6c0c6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.348+0000 7f1c6bfff640 1 -- 172.21.15.5:0/3733033653 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c6c0c8780 con 0x7f1c58004800 2024-05-20T18:27:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.348+0000 7f1c70cd4640 1 --2- 172.21.15.5:0/3733033653 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c6c0cdec0 0x7f1c6c0ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.348+0000 7f1c6bfff640 1 --2- 172.21.15.5:0/3733033653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c58004800 0x7f1c6c0cc670 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c0118c0 tx=0x7f1c5c011d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.348+0000 7f1c697fa640 1 -- 172.21.15.5:0/3733033653 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c019070 con 0x7f1c58004800 2024-05-20T18:27:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.348+0000 7f1c697fa640 1 -- 172.21.15.5:0/3733033653 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c5c004240 con 0x7f1c58004800 2024-05-20T18:27:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.348+0000 7f1c72737640 1 -- 172.21.15.5:0/3733033653 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c6c0ccf70 con 0x7f1c58004800 2024-05-20T18:27:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.348+0000 7f1c697fa640 1 -- 172.21.15.5:0/3733033653 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c00f820 con 0x7f1c58004800 2024-05-20T18:27:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.348+0000 7f1c72737640 1 -- 172.21.15.5:0/3733033653 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c6c0cd250 con 0x7f1c58004800 2024-05-20T18:27:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.349+0000 7f1c72737640 1 -- 172.21.15.5:0/3733033653 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c6c075b00 con 0x7f1c58004800 2024-05-20T18:27:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.350+0000 7f1c697fa640 1 -- 172.21.15.5:0/3733033653 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c5c01c070 con 0x7f1c58004800 2024-05-20T18:27:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.350+0000 7f1c697fa640 1 --2- 172.21.15.5:0/3733033653 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c4c06be60 0x7f1c4c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.350+0000 7f1c697fa640 1 -- 172.21.15.5:0/3733033653 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f1c5c08b610 con 0x7f1c58004800 2024-05-20T18:27:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.350+0000 7f1c63fff640 1 --2- 172.21.15.5:0/3733033653 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c4c06be60 0x7f1c4c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.351+0000 7f1c63fff640 1 --2- 172.21.15.5:0/3733033653 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c4c06be60 0x7f1c4c06e370 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1c54033000 tx=0x7f1c54009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:19.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.355+0000 7f1c697fa640 1 -- 172.21.15.5:0/3733033653 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1c5c059c80 con 0x7f1c58004800 2024-05-20T18:27:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.550+0000 7f1c72737640 1 -- 172.21.15.5:0/3733033653 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1c6c0b4720 con 0x7f1c58004800 2024-05-20T18:27:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.550+0000 7f1c697fa640 1 -- 172.21.15.5:0/3733033653 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f1c5c059580 con 0x7f1c58004800 2024-05-20T18:27:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-05-20T18:27:19.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.552+0000 7f1c72737640 1 -- 172.21.15.5:0/3733033653 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c4c06be60 msgr2=0x7f1c4c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:19.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.552+0000 7f1c72737640 1 --2- 172.21.15.5:0/3733033653 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c4c06be60 0x7f1c4c06e370 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1c54033000 tx=0x7f1c54009030 comp rx=0 tx=0).stop 2024-05-20T18:27:19.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.552+0000 7f1c72737640 1 -- 172.21.15.5:0/3733033653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c58004800 msgr2=0x7f1c6c0cc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:19.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.552+0000 7f1c72737640 1 --2- 172.21.15.5:0/3733033653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c58004800 0x7f1c6c0cc670 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c0118c0 tx=0x7f1c5c011d80 comp rx=0 tx=0).stop 2024-05-20T18:27:19.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.552+0000 7f1c72737640 1 -- 172.21.15.5:0/3733033653 shutdown_connections 2024-05-20T18:27:19.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.552+0000 7f1c72737640 1 --2- 172.21.15.5:0/3733033653 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c4c06be60 0x7f1c4c06e370 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:19.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.552+0000 7f1c72737640 1 --2- 172.21.15.5:0/3733033653 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c6c0cdec0 0x7f1c6c0ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:19.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.552+0000 7f1c72737640 1 --2- 172.21.15.5:0/3733033653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c6c0cb430 0x7f1c6c0c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:19.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.552+0000 7f1c72737640 1 --2- 172.21.15.5:0/3733033653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c58004800 0x7f1c6c0cc670 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:19.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.552+0000 7f1c72737640 1 -- 172.21.15.5:0/3733033653 >> 172.21.15.5:0/3733033653 conn(0x7f1c6c0d4bd0 msgr2=0x7f1c6c1e7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:19.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.553+0000 7f1c72737640 1 -- 172.21.15.5:0/3733033653 shutdown_connections 2024-05-20T18:27:19.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.553+0000 7f1c72737640 1 -- 172.21.15.5:0/3733033653 wait complete. 2024-05-20T18:27:19.562 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:27:19.562 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:27:19.578 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:27:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.735+0000 7f76b39aa640 1 Processor -- start 2024-05-20T18:27:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.735+0000 7f76b39aa640 1 -- start start 2024-05-20T18:27:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.736+0000 7f76b39aa640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76ac162820 0x7f76ac0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.736+0000 7f76b39aa640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76ac0d0630 con 0x7f76ac06dae0 2024-05-20T18:27:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.736+0000 7f76b39aa640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76ac0d39c0 con 0x7f76ac0b3e80 2024-05-20T18:27:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.736+0000 7f76b39aa640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f76ac0d3ca0 con 0x7f76ac162820 2024-05-20T18:27:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.737+0000 7f76b1f47640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f76ac06dae0 0x7f76ac0b8190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36540/0 (socket says 172.21.15.5:36540) 2024-05-20T18:27:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.737+0000 7f76b1f47640 1 -- 172.21.15.5:0/2204733381 learned_addr learned my addr 172.21.15.5:0/2204733381 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.737+0000 7f76b1746640 1 --2- 172.21.15.5:0/2204733381 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76ac162820 0x7f76ac0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.737+0000 7f76a3fff640 1 -- 172.21.15.5:0/2204733381 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3923473108 0 0) 0x7f76ac0d0630 con 0x7f76ac06dae0 2024-05-20T18:27:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.737+0000 7f76a3fff640 1 -- 172.21.15.5:0/2204733381 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7698003520 con 0x7f76ac06dae0 2024-05-20T18:27:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.737+0000 7f76a3fff640 1 -- 172.21.15.5:0/2204733381 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2380186708 0 0) 0x7f76ac0d39c0 con 0x7f76ac0b3e80 2024-05-20T18:27:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.737+0000 7f76b1746640 1 -- 172.21.15.5:0/2204733381 >> v1:172.21.15.63:6789/0 conn(0x7f76ac0b3e80 legacy=0x7f76ac06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.737+0000 7f76b1746640 1 -- 172.21.15.5:0/2204733381 >> v1:172.21.15.5:6789/0 conn(0x7f76ac06dae0 legacy=0x7f76ac0b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.737+0000 7f76b1746640 1 -- 172.21.15.5:0/2204733381 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76ac0d3f80 con 0x7f76ac162820 2024-05-20T18:27:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.738+0000 7f76b1746640 1 --2- 172.21.15.5:0/2204733381 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76ac162820 0x7f76ac0b3940 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f76a80097d0 tx=0x7f76a802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d20240a2dbbd8006 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.738+0000 7f76a3fff640 1 -- 172.21.15.5:0/2204733381 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a8037070 con 0x7f76ac162820 2024-05-20T18:27:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.738+0000 7f76a3fff640 1 -- 172.21.15.5:0/2204733381 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a8032440 con 0x7f76ac162820 2024-05-20T18:27:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.738+0000 7f76a3fff640 1 -- 172.21.15.5:0/2204733381 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a8032810 con 0x7f76ac162820 2024-05-20T18:27:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.739+0000 7f76b39aa640 1 -- 172.21.15.5:0/2204733381 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76ac162820 msgr2=0x7f76ac0b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:19.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.739+0000 7f76b39aa640 1 --2- 172.21.15.5:0/2204733381 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76ac162820 0x7f76ac0b3940 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f76a80097d0 tx=0x7f76a802a210 comp rx=0 tx=0).stop 2024-05-20T18:27:19.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.739+0000 7f76b39aa640 1 -- 172.21.15.5:0/2204733381 shutdown_connections 2024-05-20T18:27:19.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.739+0000 7f76b39aa640 1 --2- 172.21.15.5:0/2204733381 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76ac162820 0x7f76ac0b3940 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:19.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.739+0000 7f76b39aa640 1 -- 172.21.15.5:0/2204733381 >> 172.21.15.5:0/2204733381 conn(0x7f76ac0d4bd0 msgr2=0x7f76ac0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:19.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.739+0000 7f76b39aa640 1 -- 172.21.15.5:0/2204733381 shutdown_connections 2024-05-20T18:27:19.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.739+0000 7f76b39aa640 1 -- 172.21.15.5:0/2204733381 wait complete. 2024-05-20T18:27:19.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.741+0000 7f76b39aa640 1 Processor -- start 2024-05-20T18:27:19.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.741+0000 7f76b39aa640 1 -- start start 2024-05-20T18:27:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.742+0000 7f76b39aa640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76ac0b3e80 0x7f76ac0b6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.742+0000 7f76b39aa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76ac162820 0x7f76ac14cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:19.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.742+0000 7f76b39aa640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76ac15c290 0x7f76ac15c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:19.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.742+0000 7f76b39aa640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac0d4780 con 0x7f76ac162820 2024-05-20T18:27:19.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.742+0000 7f76b39aa640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac0d0300 con 0x7f76ac0b3e80 2024-05-20T18:27:19.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.742+0000 7f76b39aa640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f76ac0d4900 con 0x7f76ac15c290 2024-05-20T18:27:19.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.742+0000 7f76b0f45640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76ac162820 0x7f76ac14cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:19.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.742+0000 7f76b0f45640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76ac162820 0x7f76ac14cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54570/0 (socket says 172.21.15.5:54570) 2024-05-20T18:27:19.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.742+0000 7f76b0f45640 1 -- 172.21.15.5:0/2991426268 learned_addr learned my addr 172.21.15.5:0/2991426268 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:19.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.742+0000 7f76b1746640 1 --2- 172.21.15.5:0/2991426268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76ac0b3e80 0x7f76ac0b6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:19.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.742+0000 7f76b1f47640 1 --2- 172.21.15.5:0/2991426268 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76ac15c290 0x7f76ac15c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:19.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.742+0000 7f76b0f45640 1 -- 172.21.15.5:0/2991426268 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76ac15c290 msgr2=0x7f76ac15c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:19.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.742+0000 7f76b0f45640 1 --2- 172.21.15.5:0/2991426268 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76ac15c290 0x7f76ac15c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:19.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.742+0000 7f76b0f45640 1 -- 172.21.15.5:0/2991426268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76ac0b3e80 msgr2=0x7f76ac0b6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:19.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.742+0000 7f76b0f45640 1 --2- 172.21.15.5:0/2991426268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76ac0b3e80 0x7f76ac0b6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:19.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.742+0000 7f76b0f45640 1 -- 172.21.15.5:0/2991426268 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76ac15cbe0 con 0x7f76ac162820 2024-05-20T18:27:19.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.742+0000 7f76b1f47640 1 --2- 172.21.15.5:0/2991426268 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76ac15c290 0x7f76ac15c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:19.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.743+0000 7f76b0f45640 1 --2- 172.21.15.5:0/2991426268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76ac162820 0x7f76ac14cf60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f769c00c390 tx=0x7f769c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:19.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.743+0000 7f76a1ffb640 1 -- 172.21.15.5:0/2991426268 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c017070 con 0x7f76ac162820 2024-05-20T18:27:19.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.743+0000 7f76a1ffb640 1 -- 172.21.15.5:0/2991426268 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f769c0035f0 con 0x7f76ac162820 2024-05-20T18:27:19.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.743+0000 7f76b39aa640 1 -- 172.21.15.5:0/2991426268 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76ac156f80 con 0x7f76ac162820 2024-05-20T18:27:19.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.743+0000 7f76a1ffb640 1 -- 172.21.15.5:0/2991426268 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c0039e0 con 0x7f76ac162820 2024-05-20T18:27:19.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.743+0000 7f76b39aa640 1 -- 172.21.15.5:0/2991426268 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76ac157540 con 0x7f76ac162820 2024-05-20T18:27:19.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.744+0000 7f76b39aa640 1 -- 172.21.15.5:0/2991426268 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76ac156cf0 con 0x7f76ac162820 2024-05-20T18:27:19.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.745+0000 7f76a1ffb640 1 -- 172.21.15.5:0/2991426268 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f769c01a070 con 0x7f76ac162820 2024-05-20T18:27:19.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.745+0000 7f76a1ffb640 1 --2- 172.21.15.5:0/2991426268 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f769006be60 0x7f769006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:19.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.745+0000 7f76a1ffb640 1 -- 172.21.15.5:0/2991426268 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f769c08a5c0 con 0x7f76ac162820 2024-05-20T18:27:19.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.745+0000 7f76b1746640 1 --2- 172.21.15.5:0/2991426268 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f769006be60 0x7f769006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:19.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.746+0000 7f76b1746640 1 --2- 172.21.15.5:0/2991426268 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f769006be60 0x7f769006e370 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f76a8034000 tx=0x7f76a802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:19.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.750+0000 7f76a1ffb640 1 -- 172.21.15.5:0/2991426268 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f76ac15cbe0 con 0x7f76ac162820 2024-05-20T18:27:19.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.938+0000 7f76b39aa640 1 -- 172.21.15.5:0/2991426268 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76ac162cc0 con 0x7f76ac162820 2024-05-20T18:27:19.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.939+0000 7f76a1ffb640 1 -- 172.21.15.5:0/2991426268 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18394 (secure 0 0 0) 0x7f76ac162cc0 con 0x7f76ac162820 2024-05-20T18:27:19.938 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:19.938 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:27:13.817633+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:27:05.476596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T18:27:06.137762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T18:27:09.829920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:27:19.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.942+0000 7f76b39aa640 1 -- 172.21.15.5:0/2991426268 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f769006be60 msgr2=0x7f769006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:19.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.942+0000 7f76b39aa640 1 --2- 172.21.15.5:0/2991426268 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f769006be60 0x7f769006e370 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f76a8034000 tx=0x7f76a802e000 comp rx=0 tx=0).stop 2024-05-20T18:27:19.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.942+0000 7f76b39aa640 1 -- 172.21.15.5:0/2991426268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76ac162820 msgr2=0x7f76ac14cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:19.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.942+0000 7f76b39aa640 1 --2- 172.21.15.5:0/2991426268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76ac162820 0x7f76ac14cf60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f769c00c390 tx=0x7f769c00c850 comp rx=0 tx=0).stop 2024-05-20T18:27:19.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.942+0000 7f76b39aa640 1 -- 172.21.15.5:0/2991426268 shutdown_connections 2024-05-20T18:27:19.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.942+0000 7f76b39aa640 1 --2- 172.21.15.5:0/2991426268 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f769006be60 0x7f769006e370 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:19.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.942+0000 7f76b39aa640 1 --2- 172.21.15.5:0/2991426268 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76ac15c290 0x7f76ac15c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:19.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.942+0000 7f76b39aa640 1 --2- 172.21.15.5:0/2991426268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76ac162820 0x7f76ac14cf60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:19.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.942+0000 7f76b39aa640 1 --2- 172.21.15.5:0/2991426268 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76ac0b3e80 0x7f76ac0b6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:19.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.942+0000 7f76b39aa640 1 -- 172.21.15.5:0/2991426268 >> 172.21.15.5:0/2991426268 conn(0x7f76ac0d4bd0 msgr2=0x7f76ac15bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:19.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.942+0000 7f76b39aa640 1 -- 172.21.15.5:0/2991426268 shutdown_connections 2024-05-20T18:27:19.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:19.942+0000 7f76b39aa640 1 -- 172.21.15.5:0/2991426268 wait complete. 2024-05-20T18:27:19.951 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:27:05.476596+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:27:06.137762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T18:27:09.829920+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T18:27:19.952 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:27:20.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.070+0000 7f9e74aab640 1 Processor -- start 2024-05-20T18:27:20.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.070+0000 7f9e74aab640 1 -- start start 2024-05-20T18:27:20.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.072+0000 7f9e74aab640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9e7015f220 0x7f9e700d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:20.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.072+0000 7f9e74aab640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9e700d0e00 0x7f9e700cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:20.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.072+0000 7f9e74aab640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e700cf910 0x7f9e700ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:20.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.072+0000 7f9e74aab640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f9e700cfd20 con 0x7f9e7015f220 2024-05-20T18:27:20.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.072+0000 7f9e74aab640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f9e700cda20 con 0x7f9e700d0e00 2024-05-20T18:27:20.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.072+0000 7f9e74aab640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9e700cdd00 con 0x7f9e700cf910 2024-05-20T18:27:20.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.072+0000 7f9e6e59c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9e7015f220 0x7f9e700d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:20.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.072+0000 7f9e6e59c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9e7015f220 0x7f9e700d08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54584/0 (socket says 172.21.15.5:54584) 2024-05-20T18:27:20.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.072+0000 7f9e6e59c640 1 -- 172.21.15.5:0/3243476821 learned_addr learned my addr 172.21.15.5:0/3243476821 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:20.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.072+0000 7f9e6dd9b640 1 --2- 172.21.15.5:0/3243476821 >> v2:172.21.15.63:3300/0 conn(0x7f9e700d0e00 0x7f9e700cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:20.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.072+0000 7f9e6ed9d640 1 --2- 172.21.15.5:0/3243476821 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e700cf910 0x7f9e700ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:20.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.072+0000 7f9e6e59c640 1 -- 172.21.15.5:0/3243476821 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e700cf910 msgr2=0x7f9e700ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:20.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.072+0000 7f9e6e59c640 1 --2- 172.21.15.5:0/3243476821 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e700cf910 0x7f9e700ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:20.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.072+0000 7f9e6e59c640 1 -- 172.21.15.5:0/3243476821 >> v2:172.21.15.63:3300/0 conn(0x7f9e700d0e00 msgr2=0x7f9e700cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:20.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.072+0000 7f9e6e59c640 1 --2- 172.21.15.5:0/3243476821 >> v2:172.21.15.63:3300/0 conn(0x7f9e700d0e00 0x7f9e700cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:20.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.072+0000 7f9e6e59c640 1 -- 172.21.15.5:0/3243476821 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e700cdfe0 con 0x7f9e7015f220 2024-05-20T18:27:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.073+0000 7f9e6e59c640 1 --2- 172.21.15.5:0/3243476821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e7015f220 0x7f9e700d08c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9e70068350 tx=0x7f9e6802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ba559877ab7726 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.073+0000 7f9e6d59a640 1 -- 172.21.15.5:0/3243476821 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e68037070 con 0x7f9e7015f220 2024-05-20T18:27:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.073+0000 7f9e6d59a640 1 -- 172.21.15.5:0/3243476821 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e68032440 con 0x7f9e7015f220 2024-05-20T18:27:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.073+0000 7f9e6d59a640 1 -- 172.21.15.5:0/3243476821 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e68032810 con 0x7f9e7015f220 2024-05-20T18:27:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.074+0000 7f9e74aab640 1 -- 172.21.15.5:0/3243476821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e7015f220 msgr2=0x7f9e700d08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.074+0000 7f9e74aab640 1 --2- 172.21.15.5:0/3243476821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e7015f220 0x7f9e700d08c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9e70068350 tx=0x7f9e6802a140 comp rx=0 tx=0).stop 2024-05-20T18:27:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.074+0000 7f9e74aab640 1 -- 172.21.15.5:0/3243476821 shutdown_connections 2024-05-20T18:27:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.074+0000 7f9e74aab640 1 --2- 172.21.15.5:0/3243476821 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e700cf910 0x7f9e700ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.074+0000 7f9e74aab640 1 --2- 172.21.15.5:0/3243476821 >> v2:172.21.15.63:3300/0 conn(0x7f9e700d0e00 0x7f9e700cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.074+0000 7f9e74aab640 1 --2- 172.21.15.5:0/3243476821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e7015f220 0x7f9e700d08c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:20.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.074+0000 7f9e74aab640 1 -- 172.21.15.5:0/3243476821 >> 172.21.15.5:0/3243476821 conn(0x7f9e700b44d0 msgr2=0x7f9e700b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:20.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.074+0000 7f9e74aab640 1 -- 172.21.15.5:0/3243476821 shutdown_connections 2024-05-20T18:27:20.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.074+0000 7f9e74aab640 1 -- 172.21.15.5:0/3243476821 wait complete. 2024-05-20T18:27:20.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.076+0000 7f9e74aab640 1 Processor -- start 2024-05-20T18:27:20.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.076+0000 7f9e74aab640 1 -- start start 2024-05-20T18:27:20.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.078+0000 7f9e74aab640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e700cf910 0x7f9e700ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:20.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.078+0000 7f9e74aab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e700d0e00 0x7f9e701f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:20.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.078+0000 7f9e74aab640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e7015f220 0x7f9e7015da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:20.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.078+0000 7f9e74aab640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9e700cd800 con 0x7f9e700d0e00 2024-05-20T18:27:20.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.078+0000 7f9e74aab640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9e700cd680 con 0x7f9e7015f220 2024-05-20T18:27:20.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.078+0000 7f9e74aab640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9e7015f630 con 0x7f9e700cf910 2024-05-20T18:27:20.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.078+0000 7f9e6dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e700d0e00 0x7f9e701f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:20.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.078+0000 7f9e6dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e700d0e00 0x7f9e701f5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54598/0 (socket says 172.21.15.5:54598) 2024-05-20T18:27:20.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.078+0000 7f9e6dd9b640 1 -- 172.21.15.5:0/2307664150 learned_addr learned my addr 172.21.15.5:0/2307664150 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:20.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.078+0000 7f9e6e59c640 1 --2- 172.21.15.5:0/2307664150 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e700cf910 0x7f9e700ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:20.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.078+0000 7f9e6ed9d640 1 --2- 172.21.15.5:0/2307664150 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e7015f220 0x7f9e7015da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:20.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.078+0000 7f9e6dd9b640 1 -- 172.21.15.5:0/2307664150 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e700cf910 msgr2=0x7f9e700ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:20.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.078+0000 7f9e6dd9b640 1 --2- 172.21.15.5:0/2307664150 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e700cf910 0x7f9e700ca140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:20.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.078+0000 7f9e6dd9b640 1 -- 172.21.15.5:0/2307664150 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e7015f220 msgr2=0x7f9e7015da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:20.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.078+0000 7f9e6dd9b640 1 --2- 172.21.15.5:0/2307664150 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e7015f220 0x7f9e7015da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:20.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.078+0000 7f9e6dd9b640 1 -- 172.21.15.5:0/2307664150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e701618a0 con 0x7f9e700d0e00 2024-05-20T18:27:20.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.078+0000 7f9e6dd9b640 1 --2- 172.21.15.5:0/2307664150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e700d0e00 0x7f9e701f5cb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9e5800e8a0 tx=0x7f9e5800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:20.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.078+0000 7f9e66ffd640 1 -- 172.21.15.5:0/2307664150 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e58017070 con 0x7f9e700d0e00 2024-05-20T18:27:20.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.078+0000 7f9e66ffd640 1 -- 172.21.15.5:0/2307664150 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e5800ee90 con 0x7f9e700d0e00 2024-05-20T18:27:20.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.078+0000 7f9e74aab640 1 -- 172.21.15.5:0/2307664150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e70161b80 con 0x7f9e700d0e00 2024-05-20T18:27:20.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.078+0000 7f9e66ffd640 1 -- 172.21.15.5:0/2307664150 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5800cb60 con 0x7f9e700d0e00 2024-05-20T18:27:20.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.079+0000 7f9e74aab640 1 -- 172.21.15.5:0/2307664150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e70162140 con 0x7f9e700d0e00 2024-05-20T18:27:20.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.080+0000 7f9e74aab640 1 -- 172.21.15.5:0/2307664150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e700cf670 con 0x7f9e700d0e00 2024-05-20T18:27:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.080+0000 7f9e66ffd640 1 -- 172.21.15.5:0/2307664150 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9e5801a070 con 0x7f9e700d0e00 2024-05-20T18:27:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.080+0000 7f9e66ffd640 1 --2- 172.21.15.5:0/2307664150 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9e5006be60 0x7f9e5006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.081+0000 7f9e66ffd640 1 -- 172.21.15.5:0/2307664150 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f9e5808b2a0 con 0x7f9e700d0e00 2024-05-20T18:27:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.081+0000 7f9e6e59c640 1 --2- 172.21.15.5:0/2307664150 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9e5006be60 0x7f9e5006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.081+0000 7f9e6e59c640 1 --2- 172.21.15.5:0/2307664150 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9e5006be60 0x7f9e5006e370 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9e68034000 tx=0x7f9e6802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:20.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.085+0000 7f9e66ffd640 1 -- 172.21.15.5:0/2307664150 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9e700cf670 con 0x7f9e700d0e00 2024-05-20T18:27:20.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.266+0000 7f9e74aab640 1 -- 172.21.15.5:0/2307664150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9e70075b00 con 0x7f9e700d0e00 2024-05-20T18:27:20.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.267+0000 7f9e66ffd640 1 -- 172.21.15.5:0/2307664150 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f9e580592c0 con 0x7f9e700d0e00 2024-05-20T18:27:20.266 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:27:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.269+0000 7f9e74aab640 1 -- 172.21.15.5:0/2307664150 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9e5006be60 msgr2=0x7f9e5006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.269+0000 7f9e74aab640 1 --2- 172.21.15.5:0/2307664150 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9e5006be60 0x7f9e5006e370 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9e68034000 tx=0x7f9e6802e000 comp rx=0 tx=0).stop 2024-05-20T18:27:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.269+0000 7f9e74aab640 1 -- 172.21.15.5:0/2307664150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e700d0e00 msgr2=0x7f9e701f5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.269+0000 7f9e74aab640 1 --2- 172.21.15.5:0/2307664150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e700d0e00 0x7f9e701f5cb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9e5800e8a0 tx=0x7f9e5800ed60 comp rx=0 tx=0).stop 2024-05-20T18:27:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.269+0000 7f9e74aab640 1 -- 172.21.15.5:0/2307664150 shutdown_connections 2024-05-20T18:27:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.269+0000 7f9e74aab640 1 --2- 172.21.15.5:0/2307664150 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9e5006be60 0x7f9e5006e370 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.269+0000 7f9e74aab640 1 --2- 172.21.15.5:0/2307664150 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e7015f220 0x7f9e7015da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:20.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.269+0000 7f9e74aab640 1 --2- 172.21.15.5:0/2307664150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e700d0e00 0x7f9e701f5cb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:20.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.269+0000 7f9e74aab640 1 --2- 172.21.15.5:0/2307664150 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e700cf910 0x7f9e700ca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:20.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.269+0000 7f9e74aab640 1 -- 172.21.15.5:0/2307664150 >> 172.21.15.5:0/2307664150 conn(0x7f9e700b44d0 msgr2=0x7f9e700d13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:20.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.269+0000 7f9e74aab640 1 -- 172.21.15.5:0/2307664150 shutdown_connections 2024-05-20T18:27:20.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.269+0000 7f9e74aab640 1 -- 172.21.15.5:0/2307664150 wait complete. 2024-05-20T18:27:20.279 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:27:20.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.398+0000 7f86fe972640 1 Processor -- start 2024-05-20T18:27:20.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.399+0000 7f86fe972640 1 -- start start 2024-05-20T18:27:20.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.400+0000 7f86fe972640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f86f8163a10 0x7f86f80d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:20.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.400+0000 7f86fe972640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86f80d14b0 0x7f86f80d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:20.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.400+0000 7f86fe972640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86f80d0370 con 0x7f86f80b4010 2024-05-20T18:27:20.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.400+0000 7f86fe972640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f86f80d3ca0 con 0x7f86f8163a10 2024-05-20T18:27:20.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.400+0000 7f86fe972640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f86f80d0650 con 0x7f86f80d14b0 2024-05-20T18:27:20.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.400+0000 7f86f7fff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f86f8163a10 0x7f86f80d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:20.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.400+0000 7f86fcf0f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86f80d14b0 0x7f86f80d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:20.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.400+0000 7f86f77fe640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f86f80b4010 0x7f86f80cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:57312/0 (socket says 172.21.15.5:57312) 2024-05-20T18:27:20.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.400+0000 7f86f7fff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f86f8163a10 0x7f86f80d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:55792/0 (socket says 172.21.15.5:55792) 2024-05-20T18:27:20.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.400+0000 7f86f77fe640 1 -- 172.21.15.5:0/1174149587 learned_addr learned my addr 172.21.15.5:0/1174149587 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:20.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.401+0000 7f86fcf0f640 1 -- 172.21.15.5:0/1174149587 >> v2:172.21.15.63:3300/0 conn(0x7f86f8163a10 msgr2=0x7f86f80d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:20.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.401+0000 7f86fcf0f640 1 --2- 172.21.15.5:0/1174149587 >> v2:172.21.15.63:3300/0 conn(0x7f86f8163a10 0x7f86f80d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:20.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.401+0000 7f86fcf0f640 1 -- 172.21.15.5:0/1174149587 >> v1:172.21.15.63:6789/0 conn(0x7f86f80b4010 legacy=0x7f86f80cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:20.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.401+0000 7f86fcf0f640 1 -- 172.21.15.5:0/1174149587 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f80d3650 con 0x7f86f80d14b0 2024-05-20T18:27:20.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.401+0000 7f86fcf0f640 1 --2- 172.21.15.5:0/1174149587 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86f80d14b0 0x7f86f80d18c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009d90 tx=0x7f86f002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0beff4e53ddbaae server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:20.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.402+0000 7f86f6ffd640 1 -- 172.21.15.5:0/1174149587 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0007740 con 0x7f86f80d14b0 2024-05-20T18:27:20.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.402+0000 7f86f6ffd640 1 -- 172.21.15.5:0/1174149587 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86f00078e0 con 0x7f86f80d14b0 2024-05-20T18:27:20.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.402+0000 7f86f6ffd640 1 -- 172.21.15.5:0/1174149587 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0007cb0 con 0x7f86f80d14b0 2024-05-20T18:27:20.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.402+0000 7f86fe972640 1 -- 172.21.15.5:0/1174149587 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86f80d14b0 msgr2=0x7f86f80d18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:20.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.402+0000 7f86fe972640 1 --2- 172.21.15.5:0/1174149587 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86f80d14b0 0x7f86f80d18c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009d90 tx=0x7f86f002a660 comp rx=0 tx=0).stop 2024-05-20T18:27:20.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.402+0000 7f86fe972640 1 -- 172.21.15.5:0/1174149587 shutdown_connections 2024-05-20T18:27:20.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.402+0000 7f86fe972640 1 --2- 172.21.15.5:0/1174149587 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86f80d14b0 0x7f86f80d18c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:20.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.402+0000 7f86fe972640 1 --2- 172.21.15.5:0/1174149587 >> v2:172.21.15.63:3300/0 conn(0x7f86f8163a10 0x7f86f80d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:20.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.402+0000 7f86fe972640 1 -- 172.21.15.5:0/1174149587 >> 172.21.15.5:0/1174149587 conn(0x7f86f806d240 msgr2=0x7f86f806d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:20.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.402+0000 7f86fe972640 1 -- 172.21.15.5:0/1174149587 shutdown_connections 2024-05-20T18:27:20.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.402+0000 7f86fe972640 1 -- 172.21.15.5:0/1174149587 wait complete. 2024-05-20T18:27:20.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.404+0000 7f86fe972640 1 Processor -- start 2024-05-20T18:27:20.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.404+0000 7f86fe972640 1 -- start start 2024-05-20T18:27:20.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.406+0000 7f86fe972640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86f80b4010 0x7f86f80aea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:20.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.406+0000 7f86fe972640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86f80d14b0 0x7f86f80ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:20.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.406+0000 7f86fe972640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86f8163a10 0x7f86f80ade30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:20.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.406+0000 7f86fe972640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f86f80b8210 con 0x7f86f8163a10 2024-05-20T18:27:20.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.406+0000 7f86fe972640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f86f80d1e00 con 0x7f86f80d14b0 2024-05-20T18:27:20.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.406+0000 7f86fe972640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f86f80b8390 con 0x7f86f80b4010 2024-05-20T18:27:20.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.406+0000 7f86fcf0f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86f8163a10 0x7f86f80ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:20.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.406+0000 7f86fcf0f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86f8163a10 0x7f86f80ade30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54614/0 (socket says 172.21.15.5:54614) 2024-05-20T18:27:20.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.406+0000 7f86fcf0f640 1 -- 172.21.15.5:0/3701154217 learned_addr learned my addr 172.21.15.5:0/3701154217 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:20.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.406+0000 7f86f7fff640 1 --2- 172.21.15.5:0/3701154217 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86f80b4010 0x7f86f80aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:20.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.406+0000 7f86f77fe640 1 --2- 172.21.15.5:0/3701154217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86f80d14b0 0x7f86f80ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:20.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.406+0000 7f86fcf0f640 1 -- 172.21.15.5:0/3701154217 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86f80b4010 msgr2=0x7f86f80aea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:20.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.406+0000 7f86fcf0f640 1 --2- 172.21.15.5:0/3701154217 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86f80b4010 0x7f86f80aea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:20.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.406+0000 7f86fcf0f640 1 -- 172.21.15.5:0/3701154217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86f80d14b0 msgr2=0x7f86f80ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:20.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.406+0000 7f86fcf0f640 1 --2- 172.21.15.5:0/3701154217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86f80d14b0 0x7f86f80ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:20.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.406+0000 7f86fcf0f640 1 -- 172.21.15.5:0/3701154217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f80aefb0 con 0x7f86f8163a10 2024-05-20T18:27:20.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.406+0000 7f86fcf0f640 1 --2- 172.21.15.5:0/3701154217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86f8163a10 0x7f86f80ade30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009eb0 tx=0x7f86f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:20.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.407+0000 7f86f4ff9640 1 -- 172.21.15.5:0/3701154217 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f003f070 con 0x7f86f8163a10 2024-05-20T18:27:20.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.407+0000 7f86f4ff9640 1 -- 172.21.15.5:0/3701154217 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86f0031c60 con 0x7f86f8163a10 2024-05-20T18:27:20.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.407+0000 7f86fe972640 1 -- 172.21.15.5:0/3701154217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f81f5cb0 con 0x7f86f8163a10 2024-05-20T18:27:20.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.407+0000 7f86f4ff9640 1 -- 172.21.15.5:0/3701154217 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0009450 con 0x7f86f8163a10 2024-05-20T18:27:20.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.407+0000 7f86fe972640 1 -- 172.21.15.5:0/3701154217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f81f6270 con 0x7f86f8163a10 2024-05-20T18:27:20.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.408+0000 7f86fe972640 1 -- 172.21.15.5:0/3701154217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86f81f5a20 con 0x7f86f8163a10 2024-05-20T18:27:20.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.408+0000 7f86f4ff9640 1 -- 172.21.15.5:0/3701154217 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f86f003b410 con 0x7f86f8163a10 2024-05-20T18:27:20.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.409+0000 7f86f4ff9640 1 --2- 172.21.15.5:0/3701154217 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f86e406be60 0x7f86e406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:20.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.409+0000 7f86f4ff9640 1 -- 172.21.15.5:0/3701154217 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f86f00ac4d0 con 0x7f86f8163a10 2024-05-20T18:27:20.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.409+0000 7f86f7fff640 1 --2- 172.21.15.5:0/3701154217 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f86e406be60 0x7f86e406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:20.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.409+0000 7f86f7fff640 1 --2- 172.21.15.5:0/3701154217 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f86e406be60 0x7f86e406e370 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f86f80d3020 tx=0x7f86e8009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:20.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.413+0000 7f86f4ff9640 1 -- 172.21.15.5:0/3701154217 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f86f007a4e0 con 0x7f86f8163a10 2024-05-20T18:27:20.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.602+0000 7f86fe972640 1 -- 172.21.15.5:0/3701154217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f86f8075b00 con 0x7f86f8163a10 2024-05-20T18:27:20.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.602+0000 7f86f4ff9640 1 -- 172.21.15.5:0/3701154217 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f86f007e7e0 con 0x7f86f8163a10 2024-05-20T18:27:20.602 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:27:20.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.604+0000 7f86fe972640 1 -- 172.21.15.5:0/3701154217 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f86e406be60 msgr2=0x7f86e406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:20.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.604+0000 7f86fe972640 1 --2- 172.21.15.5:0/3701154217 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f86e406be60 0x7f86e406e370 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f86f80d3020 tx=0x7f86e8009190 comp rx=0 tx=0).stop 2024-05-20T18:27:20.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.604+0000 7f86fe972640 1 -- 172.21.15.5:0/3701154217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86f8163a10 msgr2=0x7f86f80ade30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:20.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.604+0000 7f86fe972640 1 --2- 172.21.15.5:0/3701154217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86f8163a10 0x7f86f80ade30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009eb0 tx=0x7f86f0004a40 comp rx=0 tx=0).stop 2024-05-20T18:27:20.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.604+0000 7f86fe972640 1 -- 172.21.15.5:0/3701154217 shutdown_connections 2024-05-20T18:27:20.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.605+0000 7f86fe972640 1 --2- 172.21.15.5:0/3701154217 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f86e406be60 0x7f86e406e370 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:20.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.605+0000 7f86fe972640 1 --2- 172.21.15.5:0/3701154217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86f8163a10 0x7f86f80ade30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:20.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.605+0000 7f86fe972640 1 --2- 172.21.15.5:0/3701154217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86f80d14b0 0x7f86f80ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:20.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.605+0000 7f86fe972640 1 --2- 172.21.15.5:0/3701154217 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86f80b4010 0x7f86f80aea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:20.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.605+0000 7f86fe972640 1 -- 172.21.15.5:0/3701154217 >> 172.21.15.5:0/3701154217 conn(0x7f86f806d240 msgr2=0x7f86f80b49c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:20.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.605+0000 7f86fe972640 1 -- 172.21.15.5:0/3701154217 shutdown_connections 2024-05-20T18:27:20.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.605+0000 7f86fe972640 1 -- 172.21.15.5:0/3701154217 wait complete. 2024-05-20T18:27:20.615 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:27:20.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.733+0000 7f1498dd5640 1 Processor -- start 2024-05-20T18:27:20.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.733+0000 7f1498dd5640 1 -- start start 2024-05-20T18:27:20.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.735+0000 7f1498dd5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f149406cf00 0x7f149406d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:20.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.735+0000 7f1498dd5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f14940d2780 0x7f14940d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:20.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.735+0000 7f1498dd5640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f149408d2a0 con 0x7f14940d4400 2024-05-20T18:27:20.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.735+0000 7f1498dd5640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f14940d3f70 con 0x7f14940d2780 2024-05-20T18:27:20.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.735+0000 7f1498dd5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f149406d850 con 0x7f149406cf00 2024-05-20T18:27:20.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.735+0000 7f1492d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f14940d2780 0x7f14940d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:20.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.735+0000 7f149259c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f14940d4400 0x7f149415f440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36542/0 (socket says 172.21.15.5:36542) 2024-05-20T18:27:20.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.735+0000 7f149259c640 1 -- 172.21.15.5:0/2838044079 learned_addr learned my addr 172.21.15.5:0/2838044079 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:20.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.735+0000 7f1491d9b640 1 --2- 172.21.15.5:0/2838044079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f149406cf00 0x7f149406d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:20.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.735+0000 7f1492d9d640 1 -- 172.21.15.5:0/2838044079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f149406cf00 msgr2=0x7f149406d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:20.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.735+0000 7f1492d9d640 1 --2- 172.21.15.5:0/2838044079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f149406cf00 0x7f149406d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:20.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.735+0000 7f1492d9d640 1 -- 172.21.15.5:0/2838044079 >> v1:172.21.15.5:6789/0 conn(0x7f14940d4400 legacy=0x7f149415f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:20.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.735+0000 7f1492d9d640 1 -- 172.21.15.5:0/2838044079 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14940d30d0 con 0x7f14940d2780 2024-05-20T18:27:20.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.735+0000 7f149159a640 1 -- 172.21.15.5:0/2838044079 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1190051898 0 0) 0x7f148c001110 con 0x7f14940d4400 2024-05-20T18:27:20.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.736+0000 7f1492d9d640 1 --2- 172.21.15.5:0/2838044079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14940d2780 0x7f14940d2b90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1484009d90 tx=0x7f148402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc72ed5188acc0ac server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:20.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.736+0000 7f149159a640 1 -- 172.21.15.5:0/2838044079 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1484007740 con 0x7f14940d2780 2024-05-20T18:27:20.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.736+0000 7f149159a640 1 -- 172.21.15.5:0/2838044079 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14840078e0 con 0x7f14940d2780 2024-05-20T18:27:20.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.736+0000 7f149159a640 1 -- 172.21.15.5:0/2838044079 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1484007cb0 con 0x7f14940d2780 2024-05-20T18:27:20.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.737+0000 7f1498dd5640 1 -- 172.21.15.5:0/2838044079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14940d2780 msgr2=0x7f14940d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:20.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.737+0000 7f1498dd5640 1 --2- 172.21.15.5:0/2838044079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14940d2780 0x7f14940d2b90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1484009d90 tx=0x7f148402a6f0 comp rx=0 tx=0).stop 2024-05-20T18:27:20.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.737+0000 7f1498dd5640 1 -- 172.21.15.5:0/2838044079 shutdown_connections 2024-05-20T18:27:20.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.737+0000 7f1498dd5640 1 --2- 172.21.15.5:0/2838044079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14940d2780 0x7f14940d2b90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:20.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.737+0000 7f1498dd5640 1 --2- 172.21.15.5:0/2838044079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f149406cf00 0x7f149406d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:20.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.737+0000 7f1498dd5640 1 -- 172.21.15.5:0/2838044079 >> 172.21.15.5:0/2838044079 conn(0x7f14940d4bb0 msgr2=0x7f14940d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:20.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.737+0000 7f1498dd5640 1 -- 172.21.15.5:0/2838044079 shutdown_connections 2024-05-20T18:27:20.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.737+0000 7f1498dd5640 1 -- 172.21.15.5:0/2838044079 wait complete. 2024-05-20T18:27:20.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.739+0000 7f1498dd5640 1 Processor -- start 2024-05-20T18:27:20.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.739+0000 7f1498dd5640 1 -- start start 2024-05-20T18:27:20.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.740+0000 7f1498dd5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f149406cf00 0x7f14941487d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:20.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.740+0000 7f1498dd5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14940d2780 0x7f14940abf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:20.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.740+0000 7f1498dd5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14940d4400 0x7f14941f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:20.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.740+0000 7f1498dd5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f14940d3640 con 0x7f14940d4400 2024-05-20T18:27:20.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.740+0000 7f1498dd5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f14940d3360 con 0x7f14940d2780 2024-05-20T18:27:20.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.740+0000 7f1498dd5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f14940d3920 con 0x7f149406cf00 2024-05-20T18:27:20.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.740+0000 7f1492d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14940d4400 0x7f14941f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:20.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.740+0000 7f1492d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14940d4400 0x7f14941f4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54634/0 (socket says 172.21.15.5:54634) 2024-05-20T18:27:20.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.740+0000 7f1492d9d640 1 -- 172.21.15.5:0/3589859856 learned_addr learned my addr 172.21.15.5:0/3589859856 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:20.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.741+0000 7f1491d9b640 1 --2- 172.21.15.5:0/3589859856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14940d2780 0x7f14940abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:20.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.741+0000 7f149259c640 1 --2- 172.21.15.5:0/3589859856 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f149406cf00 0x7f14941487d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:20.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.741+0000 7f1492d9d640 1 -- 172.21.15.5:0/3589859856 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f149406cf00 msgr2=0x7f14941487d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:20.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.741+0000 7f1492d9d640 1 --2- 172.21.15.5:0/3589859856 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f149406cf00 0x7f14941487d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:20.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.741+0000 7f1492d9d640 1 -- 172.21.15.5:0/3589859856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14940d2780 msgr2=0x7f14940abf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:20.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.741+0000 7f1492d9d640 1 --2- 172.21.15.5:0/3589859856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14940d2780 0x7f14940abf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:20.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.741+0000 7f1492d9d640 1 -- 172.21.15.5:0/3589859856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1494148e60 con 0x7f14940d4400 2024-05-20T18:27:20.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.741+0000 7f1492d9d640 1 --2- 172.21.15.5:0/3589859856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14940d4400 0x7f14941f4e70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1484009eb0 tx=0x7f1484004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:20.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.741+0000 7f148affd640 1 -- 172.21.15.5:0/3589859856 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148403f070 con 0x7f14940d4400 2024-05-20T18:27:20.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.741+0000 7f148affd640 1 -- 172.21.15.5:0/3589859856 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1484031bc0 con 0x7f14940d4400 2024-05-20T18:27:20.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.741+0000 7f1498dd5640 1 -- 172.21.15.5:0/3589859856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1494149140 con 0x7f14940d4400 2024-05-20T18:27:20.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.741+0000 7f148affd640 1 -- 172.21.15.5:0/3589859856 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1484009450 con 0x7f14940d4400 2024-05-20T18:27:20.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.741+0000 7f1498dd5640 1 -- 172.21.15.5:0/3589859856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1494149700 con 0x7f14940d4400 2024-05-20T18:27:20.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.742+0000 7f1498dd5640 1 -- 172.21.15.5:0/3589859856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14940735a0 con 0x7f14940d4400 2024-05-20T18:27:20.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.743+0000 7f148affd640 1 -- 172.21.15.5:0/3589859856 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f148403b410 con 0x7f14940d4400 2024-05-20T18:27:20.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.743+0000 7f148affd640 1 --2- 172.21.15.5:0/3589859856 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f147406be30 0x7f147406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:20.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.743+0000 7f148affd640 1 -- 172.21.15.5:0/3589859856 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f14840ac9f0 con 0x7f14940d4400 2024-05-20T18:27:20.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.743+0000 7f149259c640 1 --2- 172.21.15.5:0/3589859856 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f147406be30 0x7f147406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:20.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.744+0000 7f149259c640 1 --2- 172.21.15.5:0/3589859856 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f147406be30 0x7f147406e340 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f148c000f60 tx=0x7f148c007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:20.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.748+0000 7f148affd640 1 -- 172.21.15.5:0/3589859856 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f14940735a0 con 0x7f14940d4400 2024-05-20T18:27:20.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.932+0000 7f1498dd5640 1 -- 172.21.15.5:0/3589859856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f149406dc90 con 0x7f14940d4400 2024-05-20T18:27:20.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.933+0000 7f148affd640 1 -- 172.21.15.5:0/3589859856 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f148407aa10 con 0x7f14940d4400 2024-05-20T18:27:20.932 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:27:20.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.934+0000 7f1498dd5640 1 -- 172.21.15.5:0/3589859856 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f147406be30 msgr2=0x7f147406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:20.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.934+0000 7f1498dd5640 1 --2- 172.21.15.5:0/3589859856 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f147406be30 0x7f147406e340 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f148c000f60 tx=0x7f148c007c00 comp rx=0 tx=0).stop 2024-05-20T18:27:20.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.934+0000 7f1498dd5640 1 -- 172.21.15.5:0/3589859856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14940d4400 msgr2=0x7f14941f4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:20.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.934+0000 7f1498dd5640 1 --2- 172.21.15.5:0/3589859856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14940d4400 0x7f14941f4e70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1484009eb0 tx=0x7f1484004a40 comp rx=0 tx=0).stop 2024-05-20T18:27:20.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.934+0000 7f1498dd5640 1 -- 172.21.15.5:0/3589859856 shutdown_connections 2024-05-20T18:27:20.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.934+0000 7f1498dd5640 1 --2- 172.21.15.5:0/3589859856 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f147406be30 0x7f147406e340 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:20.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.934+0000 7f1498dd5640 1 --2- 172.21.15.5:0/3589859856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14940d4400 0x7f14941f4e70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:20.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.934+0000 7f1498dd5640 1 --2- 172.21.15.5:0/3589859856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14940d2780 0x7f14940abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:20.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.934+0000 7f1498dd5640 1 --2- 172.21.15.5:0/3589859856 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f149406cf00 0x7f14941487d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:20.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.934+0000 7f1498dd5640 1 -- 172.21.15.5:0/3589859856 >> 172.21.15.5:0/3589859856 conn(0x7f14940d4bb0 msgr2=0x7f14940d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:20.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.935+0000 7f1498dd5640 1 -- 172.21.15.5:0/3589859856 shutdown_connections 2024-05-20T18:27:20.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:20.935+0000 7f1498dd5640 1 -- 172.21.15.5:0/3589859856 wait complete. 2024-05-20T18:27:20.945 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T18:27:21.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.064+0000 7fc3992de640 1 Processor -- start 2024-05-20T18:27:21.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.065+0000 7fc3992de640 1 -- start start 2024-05-20T18:27:21.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.066+0000 7fc3992de640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc394163a10 0x7fc3940d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:21.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.066+0000 7fc3992de640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3940b4010 0x7fc3940cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:21.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.066+0000 7fc3992de640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc3940d14b0 0x7fc3940d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:21.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.066+0000 7fc3992de640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fc3940d3ca0 con 0x7fc3940d14b0 2024-05-20T18:27:21.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.066+0000 7fc3992de640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc3940d0370 con 0x7fc394163a10 2024-05-20T18:27:21.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.066+0000 7fc3992de640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc3940d0650 con 0x7fc3940b4010 2024-05-20T18:27:21.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.066+0000 7fc3937fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc3940d14b0 0x7fc3940d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:21.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.066+0000 7fc3937fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc3940d14b0 0x7fc3940d18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54636/0 (socket says 172.21.15.5:54636) 2024-05-20T18:27:21.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.066+0000 7fc3937fe640 1 -- 172.21.15.5:0/777135055 learned_addr learned my addr 172.21.15.5:0/777135055 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:21.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.066+0000 7fc392ffd640 1 --2- 172.21.15.5:0/777135055 >> v2:172.21.15.63:3300/0 conn(0x7fc394163a10 0x7fc3940d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:21.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.066+0000 7fc3927fc640 1 --2- 172.21.15.5:0/777135055 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3940b4010 0x7fc3940cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:21.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.066+0000 7fc3937fe640 1 -- 172.21.15.5:0/777135055 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3940b4010 msgr2=0x7fc3940cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:21.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.066+0000 7fc3937fe640 1 --2- 172.21.15.5:0/777135055 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3940b4010 0x7fc3940cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:21.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.066+0000 7fc3937fe640 1 -- 172.21.15.5:0/777135055 >> v2:172.21.15.63:3300/0 conn(0x7fc394163a10 msgr2=0x7fc3940d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:21.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.066+0000 7fc3937fe640 1 --2- 172.21.15.5:0/777135055 >> v2:172.21.15.63:3300/0 conn(0x7fc394163a10 0x7fc3940d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:21.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.066+0000 7fc3937fe640 1 -- 172.21.15.5:0/777135055 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3940d3650 con 0x7fc3940d14b0 2024-05-20T18:27:21.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.066+0000 7fc392ffd640 1 --2- 172.21.15.5:0/777135055 >> v2:172.21.15.63:3300/0 conn(0x7fc394163a10 0x7fc3940d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:21.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.067+0000 7fc3937fe640 1 --2- 172.21.15.5:0/777135055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3940d14b0 0x7fc3940d18c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc3940cf1c0 tx=0x7fc38c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d548068f8230bf3e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:21.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.067+0000 7fc391ffb640 1 -- 172.21.15.5:0/777135055 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38c037070 con 0x7fc3940d14b0 2024-05-20T18:27:21.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.067+0000 7fc391ffb640 1 -- 172.21.15.5:0/777135055 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc38c032440 con 0x7fc3940d14b0 2024-05-20T18:27:21.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.067+0000 7fc391ffb640 1 -- 172.21.15.5:0/777135055 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38c032810 con 0x7fc3940d14b0 2024-05-20T18:27:21.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.068+0000 7fc3992de640 1 -- 172.21.15.5:0/777135055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3940d14b0 msgr2=0x7fc3940d18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:21.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.068+0000 7fc3992de640 1 --2- 172.21.15.5:0/777135055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3940d14b0 0x7fc3940d18c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc3940cf1c0 tx=0x7fc38c02a1e0 comp rx=0 tx=0).stop 2024-05-20T18:27:21.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.068+0000 7fc3992de640 1 -- 172.21.15.5:0/777135055 shutdown_connections 2024-05-20T18:27:21.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.068+0000 7fc3992de640 1 --2- 172.21.15.5:0/777135055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3940d14b0 0x7fc3940d18c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:21.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.068+0000 7fc3992de640 1 --2- 172.21.15.5:0/777135055 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3940b4010 0x7fc3940cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:21.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.068+0000 7fc3992de640 1 --2- 172.21.15.5:0/777135055 >> v2:172.21.15.63:3300/0 conn(0x7fc394163a10 0x7fc3940d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:21.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.068+0000 7fc3992de640 1 -- 172.21.15.5:0/777135055 >> 172.21.15.5:0/777135055 conn(0x7fc39406d240 msgr2=0x7fc39406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:21.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.068+0000 7fc3992de640 1 -- 172.21.15.5:0/777135055 shutdown_connections 2024-05-20T18:27:21.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.068+0000 7fc3992de640 1 -- 172.21.15.5:0/777135055 wait complete. 2024-05-20T18:27:21.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.070+0000 7fc3992de640 1 Processor -- start 2024-05-20T18:27:21.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.070+0000 7fc3992de640 1 -- start start 2024-05-20T18:27:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.071+0000 7fc3992de640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3940b4010 0x7fc3940adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.071+0000 7fc3992de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3940d14b0 0x7fc3940b3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.071+0000 7fc3992de640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc394163a10 0x7fc3941eac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.071+0000 7fc3992de640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc3940d1e00 con 0x7fc3940d14b0 2024-05-20T18:27:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.071+0000 7fc3992de640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc3940d0040 con 0x7fc394163a10 2024-05-20T18:27:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.071+0000 7fc3992de640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc3940b8210 con 0x7fc3940b4010 2024-05-20T18:27:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.071+0000 7fc3927fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3940d14b0 0x7fc3940b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.071+0000 7fc3927fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3940d14b0 0x7fc3940b3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54644/0 (socket says 172.21.15.5:54644) 2024-05-20T18:27:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.072+0000 7fc3927fc640 1 -- 172.21.15.5:0/2814645245 learned_addr learned my addr 172.21.15.5:0/2814645245 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.072+0000 7fc3937fe640 1 --2- 172.21.15.5:0/2814645245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc394163a10 0x7fc3941eac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.072+0000 7fc392ffd640 1 --2- 172.21.15.5:0/2814645245 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3940b4010 0x7fc3940adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.072+0000 7fc3927fc640 1 -- 172.21.15.5:0/2814645245 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3940b4010 msgr2=0x7fc3940adeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.072+0000 7fc3927fc640 1 --2- 172.21.15.5:0/2814645245 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3940b4010 0x7fc3940adeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.072+0000 7fc3927fc640 1 -- 172.21.15.5:0/2814645245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc394163a10 msgr2=0x7fc3941eac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.072+0000 7fc3927fc640 1 --2- 172.21.15.5:0/2814645245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc394163a10 0x7fc3941eac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.072+0000 7fc3927fc640 1 -- 172.21.15.5:0/2814645245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3940ac690 con 0x7fc3940d14b0 2024-05-20T18:27:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.072+0000 7fc3927fc640 1 --2- 172.21.15.5:0/2814645245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3940d14b0 0x7fc3940b3140 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc38800d830 tx=0x7fc38800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.072+0000 7fc37b7fe640 1 -- 172.21.15.5:0/2814645245 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc388018030 con 0x7fc3940d14b0 2024-05-20T18:27:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.072+0000 7fc37b7fe640 1 -- 172.21.15.5:0/2814645245 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc38800be10 con 0x7fc3940d14b0 2024-05-20T18:27:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.072+0000 7fc3992de640 1 -- 172.21.15.5:0/2814645245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3940ac970 con 0x7fc3940d14b0 2024-05-20T18:27:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.072+0000 7fc37b7fe640 1 -- 172.21.15.5:0/2814645245 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3880098b0 con 0x7fc3940d14b0 2024-05-20T18:27:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.072+0000 7fc3992de640 1 -- 172.21.15.5:0/2814645245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3940acf30 con 0x7fc3940d14b0 2024-05-20T18:27:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.073+0000 7fc3992de640 1 -- 172.21.15.5:0/2814645245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc394075b00 con 0x7fc3940d14b0 2024-05-20T18:27:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.074+0000 7fc37b7fe640 1 -- 172.21.15.5:0/2814645245 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc38801b070 con 0x7fc3940d14b0 2024-05-20T18:27:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.074+0000 7fc37b7fe640 1 --2- 172.21.15.5:0/2814645245 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc38006be60 0x7fc38006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.074+0000 7fc37b7fe640 1 -- 172.21.15.5:0/2814645245 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc38808b360 con 0x7fc3940d14b0 2024-05-20T18:27:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.074+0000 7fc392ffd640 1 --2- 172.21.15.5:0/2814645245 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc38006be60 0x7fc38006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:21.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.075+0000 7fc392ffd640 1 --2- 172.21.15.5:0/2814645245 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc38006be60 0x7fc38006e370 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc3940d3020 tx=0x7fc384009870 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:21.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.079+0000 7fc37b7fe640 1 -- 172.21.15.5:0/2814645245 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc388059250 con 0x7fc3940d14b0 2024-05-20T18:27:21.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.263+0000 7fc3992de640 1 -- 172.21.15.5:0/2814645245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc3940ac420 con 0x7fc3940d14b0 2024-05-20T18:27:21.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.263+0000 7fc37b7fe640 1 -- 172.21.15.5:0/2814645245 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fc38805d550 con 0x7fc3940d14b0 2024-05-20T18:27:21.264 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:27:21.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.265+0000 7fc3992de640 1 -- 172.21.15.5:0/2814645245 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc38006be60 msgr2=0x7fc38006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:21.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.265+0000 7fc3992de640 1 --2- 172.21.15.5:0/2814645245 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc38006be60 0x7fc38006e370 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc3940d3020 tx=0x7fc384009870 comp rx=0 tx=0).stop 2024-05-20T18:27:21.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.265+0000 7fc3992de640 1 -- 172.21.15.5:0/2814645245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3940d14b0 msgr2=0x7fc3940b3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:21.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.265+0000 7fc3992de640 1 --2- 172.21.15.5:0/2814645245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3940d14b0 0x7fc3940b3140 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc38800d830 tx=0x7fc38800dcf0 comp rx=0 tx=0).stop 2024-05-20T18:27:21.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.265+0000 7fc3992de640 1 -- 172.21.15.5:0/2814645245 shutdown_connections 2024-05-20T18:27:21.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.265+0000 7fc3992de640 1 --2- 172.21.15.5:0/2814645245 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc38006be60 0x7fc38006e370 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:21.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.265+0000 7fc3992de640 1 --2- 172.21.15.5:0/2814645245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc394163a10 0x7fc3941eac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:21.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.265+0000 7fc3992de640 1 --2- 172.21.15.5:0/2814645245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3940d14b0 0x7fc3940b3140 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:21.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.265+0000 7fc3992de640 1 --2- 172.21.15.5:0/2814645245 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3940b4010 0x7fc3940adeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:21.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.265+0000 7fc3992de640 1 -- 172.21.15.5:0/2814645245 >> 172.21.15.5:0/2814645245 conn(0x7fc39406d240 msgr2=0x7fc3940ba310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:21.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.266+0000 7fc3992de640 1 -- 172.21.15.5:0/2814645245 shutdown_connections 2024-05-20T18:27:21.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.266+0000 7fc3992de640 1 -- 172.21.15.5:0/2814645245 wait complete. 2024-05-20T18:27:21.277 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T18:27:21.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.394+0000 7f3cf419c640 1 Processor -- start 2024-05-20T18:27:21.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.394+0000 7f3cf419c640 1 -- start start 2024-05-20T18:27:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.395+0000 7f3cf419c640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3cec162820 0x7f3cec0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.395+0000 7f3cf419c640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cec0d0630 con 0x7f3cec0b3e80 2024-05-20T18:27:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.395+0000 7f3cf419c640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cec0d39c0 con 0x7f3cec06dae0 2024-05-20T18:27:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.395+0000 7f3cf419c640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f3cec0d3ca0 con 0x7f3cec162820 2024-05-20T18:27:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.396+0000 7f3cf1737640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3cec0b3e80 0x7f3cec06d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36544/0 (socket says 172.21.15.5:36544) 2024-05-20T18:27:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.396+0000 7f3cf1737640 1 -- 172.21.15.5:0/1708603336 learned_addr learned my addr 172.21.15.5:0/1708603336 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.396+0000 7f3cf1f38640 1 --2- 172.21.15.5:0/1708603336 >> v2:172.21.15.63:3300/0 conn(0x7f3cec162820 0x7f3cec0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.396+0000 7f3cf0f36640 1 -- 172.21.15.5:0/1708603336 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 391202299 0 0) 0x7f3cec0d0630 con 0x7f3cec0b3e80 2024-05-20T18:27:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.396+0000 7f3cf0f36640 1 -- 172.21.15.5:0/1708603336 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3cd8003520 con 0x7f3cec0b3e80 2024-05-20T18:27:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.396+0000 7f3cf1f38640 1 -- 172.21.15.5:0/1708603336 >> v1:172.21.15.63:6789/0 conn(0x7f3cec06dae0 legacy=0x7f3cec0b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.396+0000 7f3cf1f38640 1 -- 172.21.15.5:0/1708603336 >> v1:172.21.15.5:6789/0 conn(0x7f3cec0b3e80 legacy=0x7f3cec06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.396+0000 7f3cf1f38640 1 -- 172.21.15.5:0/1708603336 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cec0d3f80 con 0x7f3cec162820 2024-05-20T18:27:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.397+0000 7f3cf1f38640 1 --2- 172.21.15.5:0/1708603336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3cec162820 0x7f3cec0b3940 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3ce80097a0 tx=0x7f3ce802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad9a954d49184e85 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.397+0000 7f3cf0f36640 1 -- 172.21.15.5:0/1708603336 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8037070 con 0x7f3cec162820 2024-05-20T18:27:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.397+0000 7f3cf0f36640 1 -- 172.21.15.5:0/1708603336 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ce8032440 con 0x7f3cec162820 2024-05-20T18:27:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.397+0000 7f3cf0f36640 1 -- 172.21.15.5:0/1708603336 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce80327f0 con 0x7f3cec162820 2024-05-20T18:27:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.398+0000 7f3cf419c640 1 -- 172.21.15.5:0/1708603336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3cec162820 msgr2=0x7f3cec0b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.398+0000 7f3cf419c640 1 --2- 172.21.15.5:0/1708603336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3cec162820 0x7f3cec0b3940 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3ce80097a0 tx=0x7f3ce802a230 comp rx=0 tx=0).stop 2024-05-20T18:27:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.398+0000 7f3cf419c640 1 -- 172.21.15.5:0/1708603336 shutdown_connections 2024-05-20T18:27:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.398+0000 7f3cf419c640 1 --2- 172.21.15.5:0/1708603336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3cec162820 0x7f3cec0b3940 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.398+0000 7f3cf419c640 1 -- 172.21.15.5:0/1708603336 >> 172.21.15.5:0/1708603336 conn(0x7f3cec0d4bd0 msgr2=0x7f3cec0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.398+0000 7f3cf419c640 1 -- 172.21.15.5:0/1708603336 shutdown_connections 2024-05-20T18:27:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.398+0000 7f3cf419c640 1 -- 172.21.15.5:0/1708603336 wait complete. 2024-05-20T18:27:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.400+0000 7f3cf419c640 1 Processor -- start 2024-05-20T18:27:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.400+0000 7f3cf419c640 1 -- start start 2024-05-20T18:27:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.401+0000 7f3cf419c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cec0b3e80 0x7f3cec0c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.401+0000 7f3cf419c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3cec162820 0x7f3cec14f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.401+0000 7f3cf419c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3cec0b4da0 0x7f3cec0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.401+0000 7f3cf419c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec0d4780 con 0x7f3cec0b3e80 2024-05-20T18:27:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.401+0000 7f3cf419c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec0d0300 con 0x7f3cec0b4da0 2024-05-20T18:27:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.401+0000 7f3cf419c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3cec0d4900 con 0x7f3cec162820 2024-05-20T18:27:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.401+0000 7f3cf1f38640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cec0b3e80 0x7f3cec0c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.401+0000 7f3cf1f38640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cec0b3e80 0x7f3cec0c9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54656/0 (socket says 172.21.15.5:54656) 2024-05-20T18:27:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.401+0000 7f3cf1f38640 1 -- 172.21.15.5:0/3534189851 learned_addr learned my addr 172.21.15.5:0/3534189851 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.401+0000 7f3cf2739640 1 --2- 172.21.15.5:0/3534189851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3cec0b4da0 0x7f3cec0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.401+0000 7f3cf1737640 1 --2- 172.21.15.5:0/3534189851 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3cec162820 0x7f3cec14f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.401+0000 7f3cf1f38640 1 -- 172.21.15.5:0/3534189851 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3cec162820 msgr2=0x7f3cec14f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.401+0000 7f3cf1f38640 1 --2- 172.21.15.5:0/3534189851 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3cec162820 0x7f3cec14f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.401+0000 7f3cf1f38640 1 -- 172.21.15.5:0/3534189851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3cec0b4da0 msgr2=0x7f3cec0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.401+0000 7f3cf1f38640 1 --2- 172.21.15.5:0/3534189851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3cec0b4da0 0x7f3cec0b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.401+0000 7f3cf1f38640 1 -- 172.21.15.5:0/3534189851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cec0cd2b0 con 0x7f3cec0b3e80 2024-05-20T18:27:21.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.402+0000 7f3cf2739640 1 --2- 172.21.15.5:0/3534189851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3cec0b4da0 0x7f3cec0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:21.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.402+0000 7f3cf1737640 1 --2- 172.21.15.5:0/3534189851 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3cec162820 0x7f3cec14f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:21.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.402+0000 7f3cf1f38640 1 --2- 172.21.15.5:0/3534189851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cec0b3e80 0x7f3cec0c9f40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3ce80097a0 tx=0x7f3ce80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:21.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.402+0000 7f3ce27fc640 1 -- 172.21.15.5:0/3534189851 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8037050 con 0x7f3cec0b3e80 2024-05-20T18:27:21.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.402+0000 7f3ce27fc640 1 -- 172.21.15.5:0/3534189851 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ce8038040 con 0x7f3cec0b3e80 2024-05-20T18:27:21.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.402+0000 7f3cf419c640 1 -- 172.21.15.5:0/3534189851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cec14f550 con 0x7f3cec0b3e80 2024-05-20T18:27:21.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.402+0000 7f3ce27fc640 1 -- 172.21.15.5:0/3534189851 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8004470 con 0x7f3cec0b3e80 2024-05-20T18:27:21.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.402+0000 7f3cf419c640 1 -- 172.21.15.5:0/3534189851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cec0cd520 con 0x7f3cec0b3e80 2024-05-20T18:27:21.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.403+0000 7f3cf419c640 1 -- 172.21.15.5:0/3534189851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cec075b00 con 0x7f3cec0b3e80 2024-05-20T18:27:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.404+0000 7f3ce27fc640 1 -- 172.21.15.5:0/3534189851 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3ce803c6d0 con 0x7f3cec0b3e80 2024-05-20T18:27:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.404+0000 7f3ce27fc640 1 --2- 172.21.15.5:0/3534189851 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3cd006be60 0x7f3cd006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.404+0000 7f3ce27fc640 1 -- 172.21.15.5:0/3534189851 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f3ce80abc00 con 0x7f3cec0b3e80 2024-05-20T18:27:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.404+0000 7f3cf1737640 1 --2- 172.21.15.5:0/3534189851 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3cd006be60 0x7f3cd006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.405+0000 7f3cf1737640 1 --2- 172.21.15.5:0/3534189851 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3cd006be60 0x7f3cd006e370 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc007fd0 tx=0x7f3cdc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:21.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.409+0000 7f3ce27fc640 1 -- 172.21.15.5:0/3534189851 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3ce80b2080 con 0x7f3cec0b3e80 2024-05-20T18:27:21.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.606+0000 7f3cf419c640 1 -- 172.21.15.5:0/3534189851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3cec0b3620 con 0x7f3cec0b3e80 2024-05-20T18:27:21.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.606+0000 7f3ce27fc640 1 -- 172.21.15.5:0/3534189851 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f3cec14f550 con 0x7f3cec0b3e80 2024-05-20T18:27:21.605 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-05-20T18:27:21.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.608+0000 7f3cf419c640 1 -- 172.21.15.5:0/3534189851 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3cd006be60 msgr2=0x7f3cd006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:21.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.608+0000 7f3cf419c640 1 --2- 172.21.15.5:0/3534189851 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3cd006be60 0x7f3cd006e370 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc007fd0 tx=0x7f3cdc009040 comp rx=0 tx=0).stop 2024-05-20T18:27:21.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.608+0000 7f3cf419c640 1 -- 172.21.15.5:0/3534189851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cec0b3e80 msgr2=0x7f3cec0c9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:21.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.608+0000 7f3cf419c640 1 --2- 172.21.15.5:0/3534189851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cec0b3e80 0x7f3cec0c9f40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3ce80097a0 tx=0x7f3ce80046c0 comp rx=0 tx=0).stop 2024-05-20T18:27:21.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.608+0000 7f3cf419c640 1 -- 172.21.15.5:0/3534189851 shutdown_connections 2024-05-20T18:27:21.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.608+0000 7f3cf419c640 1 --2- 172.21.15.5:0/3534189851 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3cd006be60 0x7f3cd006e370 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:21.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.608+0000 7f3cf419c640 1 --2- 172.21.15.5:0/3534189851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3cec0b4da0 0x7f3cec0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:21.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.608+0000 7f3cf419c640 1 --2- 172.21.15.5:0/3534189851 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3cec162820 0x7f3cec14f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:21.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.608+0000 7f3cf419c640 1 --2- 172.21.15.5:0/3534189851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cec0b3e80 0x7f3cec0c9f40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:21.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.608+0000 7f3cf419c640 1 -- 172.21.15.5:0/3534189851 >> 172.21.15.5:0/3534189851 conn(0x7f3cec0d4bd0 msgr2=0x7f3cec0b8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:21.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.608+0000 7f3cf419c640 1 -- 172.21.15.5:0/3534189851 shutdown_connections 2024-05-20T18:27:21.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:21.608+0000 7f3cf419c640 1 -- 172.21.15.5:0/3534189851 wait complete. 2024-05-20T18:27:21.619 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi005.front.sepia.ceph.com... 2024-05-20T18:27:21.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:21.619 DEBUG:teuthology.orchestra.run.smithi005:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-20T18:27:21.643 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi005.stderr:ceph-fuse[43837]: fuse finished with error 0 and tester_r 0 2024-05-20T18:27:21.645 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:27:27.746 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:27:27.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:27.747 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-20T18:27:27.779 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:27:27.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.949+0000 7f0f92957640 1 Processor -- start 2024-05-20T18:27:27.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.949+0000 7f0f92957640 1 -- start start 2024-05-20T18:27:27.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.950+0000 7f0f92957640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0f8c06d3d0 0x7f0f8c06d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:27.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.950+0000 7f0f92957640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0f8c0b8190 0x7f0f8c0b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:27.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.950+0000 7f0f92957640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f8c06dd20 con 0x7f0f8c162820 2024-05-20T18:27:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.950+0000 7f0f92957640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f0f8c0d0630 con 0x7f0f8c06d3d0 2024-05-20T18:27:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.950+0000 7f0f92957640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0f8c0d39c0 con 0x7f0f8c0b8190 2024-05-20T18:27:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.950+0000 7f0f8b7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0f8c06d3d0 0x7f0f8c06d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.950+0000 7f0f8b7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0f8c06d3d0 0x7f0f8c06d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58900/0 (socket says 172.21.15.5:58900) 2024-05-20T18:27:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.950+0000 7f0f8b7fe640 1 -- 172.21.15.5:0/627852517 learned_addr learned my addr 172.21.15.5:0/627852517 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.951+0000 7f0f90ef4640 1 --2- 172.21.15.5:0/627852517 >> v2:172.21.15.63:3300/0 conn(0x7f0f8c0b8190 0x7f0f8c0b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.951+0000 7f0f8b7fe640 1 -- 172.21.15.5:0/627852517 >> v2:172.21.15.63:3300/0 conn(0x7f0f8c0b8190 msgr2=0x7f0f8c0b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.951+0000 7f0f8b7fe640 1 --2- 172.21.15.5:0/627852517 >> v2:172.21.15.63:3300/0 conn(0x7f0f8c0b8190 0x7f0f8c0b85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.951+0000 7f0f8b7fe640 1 -- 172.21.15.5:0/627852517 >> v1:172.21.15.63:6789/0 conn(0x7f0f8c162820 legacy=0x7f0f8c0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.951+0000 7f0f8b7fe640 1 -- 172.21.15.5:0/627852517 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f8c0d3ca0 con 0x7f0f8c06d3d0 2024-05-20T18:27:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.951+0000 7f0f90ef4640 1 --2- 172.21.15.5:0/627852517 >> v2:172.21.15.63:3300/0 conn(0x7f0f8c0b8190 0x7f0f8c0b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.951+0000 7f0f8b7fe640 1 --2- 172.21.15.5:0/627852517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f8c06d3d0 0x7f0f8c06d7e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c08e070 tx=0x7f0f8002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d6436772713363 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:27.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.952+0000 7f0f8affd640 1 -- 172.21.15.5:0/627852517 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80037070 con 0x7f0f8c06d3d0 2024-05-20T18:27:27.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.952+0000 7f0f8affd640 1 -- 172.21.15.5:0/627852517 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f80032440 con 0x7f0f8c06d3d0 2024-05-20T18:27:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.952+0000 7f0f8affd640 1 -- 172.21.15.5:0/627852517 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80032810 con 0x7f0f8c06d3d0 2024-05-20T18:27:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.952+0000 7f0f92957640 1 -- 172.21.15.5:0/627852517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f8c06d3d0 msgr2=0x7f0f8c06d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.952+0000 7f0f92957640 1 --2- 172.21.15.5:0/627852517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f8c06d3d0 0x7f0f8c06d7e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c08e070 tx=0x7f0f8002a270 comp rx=0 tx=0).stop 2024-05-20T18:27:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.952+0000 7f0f92957640 1 -- 172.21.15.5:0/627852517 shutdown_connections 2024-05-20T18:27:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.952+0000 7f0f92957640 1 --2- 172.21.15.5:0/627852517 >> v2:172.21.15.63:3300/0 conn(0x7f0f8c0b8190 0x7f0f8c0b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.952+0000 7f0f92957640 1 --2- 172.21.15.5:0/627852517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f8c06d3d0 0x7f0f8c06d7e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.952+0000 7f0f92957640 1 -- 172.21.15.5:0/627852517 >> 172.21.15.5:0/627852517 conn(0x7f0f8c0d4bd0 msgr2=0x7f0f8c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.953+0000 7f0f92957640 1 -- 172.21.15.5:0/627852517 shutdown_connections 2024-05-20T18:27:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.953+0000 7f0f92957640 1 -- 172.21.15.5:0/627852517 wait complete. 2024-05-20T18:27:27.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.954+0000 7f0f92957640 1 Processor -- start 2024-05-20T18:27:27.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.955+0000 7f0f92957640 1 -- start start 2024-05-20T18:27:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.956+0000 7f0f92957640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f8c06d3d0 0x7f0f8c152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.956+0000 7f0f92957640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f8c0b8190 0x7f0f8c152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.956+0000 7f0f92957640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f8c162820 0x7f0f8c0ab5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.956+0000 7f0f92957640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8c0d0300 con 0x7f0f8c0b8190 2024-05-20T18:27:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.956+0000 7f0f92957640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8c0d4260 con 0x7f0f8c162820 2024-05-20T18:27:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.956+0000 7f0f92957640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0f8c0b4050 con 0x7f0f8c06d3d0 2024-05-20T18:27:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.956+0000 7f0f8b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f8c0b8190 0x7f0f8c152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.956+0000 7f0f8b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f8c0b8190 0x7f0f8c152a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58902/0 (socket says 172.21.15.5:58902) 2024-05-20T18:27:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.956+0000 7f0f8b7fe640 1 -- 172.21.15.5:0/3384173084 learned_addr learned my addr 172.21.15.5:0/3384173084 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.956+0000 7f0f8bfff640 1 --2- 172.21.15.5:0/3384173084 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f8c06d3d0 0x7f0f8c152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.956+0000 7f0f90ef4640 1 --2- 172.21.15.5:0/3384173084 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f8c162820 0x7f0f8c0ab5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.956+0000 7f0f8b7fe640 1 -- 172.21.15.5:0/3384173084 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f8c06d3d0 msgr2=0x7f0f8c152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.956+0000 7f0f8b7fe640 1 --2- 172.21.15.5:0/3384173084 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f8c06d3d0 0x7f0f8c152020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.956+0000 7f0f8b7fe640 1 -- 172.21.15.5:0/3384173084 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f8c162820 msgr2=0x7f0f8c0ab5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.956+0000 7f0f8b7fe640 1 --2- 172.21.15.5:0/3384173084 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f8c162820 0x7f0f8c0ab5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.956+0000 7f0f8b7fe640 1 -- 172.21.15.5:0/3384173084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f8c152560 con 0x7f0f8c0b8190 2024-05-20T18:27:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.956+0000 7f0f90ef4640 1 --2- 172.21.15.5:0/3384173084 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f8c162820 0x7f0f8c0ab5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.956+0000 7f0f8bfff640 1 --2- 172.21.15.5:0/3384173084 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f8c06d3d0 0x7f0f8c152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.957+0000 7f0f8b7fe640 1 --2- 172.21.15.5:0/3384173084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f8c0b8190 0x7f0f8c152a90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0f8002a2b0 tx=0x7f0f800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.957+0000 7f0f88ff9640 1 -- 172.21.15.5:0/3384173084 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80037040 con 0x7f0f8c0b8190 2024-05-20T18:27:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.957+0000 7f0f88ff9640 1 -- 172.21.15.5:0/3384173084 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f80042070 con 0x7f0f8c0b8190 2024-05-20T18:27:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.957+0000 7f0f92957640 1 -- 172.21.15.5:0/3384173084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f8c0c9e70 con 0x7f0f8c0b8190 2024-05-20T18:27:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.957+0000 7f0f88ff9640 1 -- 172.21.15.5:0/3384173084 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80030bb0 con 0x7f0f8c0b8190 2024-05-20T18:27:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.957+0000 7f0f92957640 1 -- 172.21.15.5:0/3384173084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f8c0ca430 con 0x7f0f8c0b8190 2024-05-20T18:27:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.958+0000 7f0f92957640 1 -- 172.21.15.5:0/3384173084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f8c075b00 con 0x7f0f8c0b8190 2024-05-20T18:27:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.959+0000 7f0f88ff9640 1 -- 172.21.15.5:0/3384173084 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0f80047460 con 0x7f0f8c0b8190 2024-05-20T18:27:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.959+0000 7f0f88ff9640 1 --2- 172.21.15.5:0/3384173084 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0f7806be30 0x7f0f7806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.960+0000 7f0f8bfff640 1 --2- 172.21.15.5:0/3384173084 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0f7806be30 0x7f0f7806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.960+0000 7f0f88ff9640 1 -- 172.21.15.5:0/3384173084 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0f800abe70 con 0x7f0f8c0b8190 2024-05-20T18:27:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.960+0000 7f0f8bfff640 1 --2- 172.21.15.5:0/3384173084 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0f7806be30 0x7f0f7806e340 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c001660 tx=0x7f0f7c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:27.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:27.964+0000 7f0f88ff9640 1 -- 172.21.15.5:0/3384173084 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0f8007a450 con 0x7f0f8c0b8190 2024-05-20T18:27:28.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.151+0000 7f0f92957640 1 -- 172.21.15.5:0/3384173084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f8c161e60 con 0x7f0f8c0b8190 2024-05-20T18:27:28.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.152+0000 7f0f88ff9640 1 -- 172.21.15.5:0/3384173084 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18394 (secure 0 0 0) 0x7f0f80079d50 con 0x7f0f8c0b8190 2024-05-20T18:27:28.151 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:28.152 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:27:13.817633+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:27:05.476596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T18:27:06.137762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T18:27:09.829920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:27:28.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.154+0000 7f0f92957640 1 -- 172.21.15.5:0/3384173084 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0f7806be30 msgr2=0x7f0f7806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:28.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.154+0000 7f0f92957640 1 --2- 172.21.15.5:0/3384173084 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0f7806be30 0x7f0f7806e340 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c001660 tx=0x7f0f7c009040 comp rx=0 tx=0).stop 2024-05-20T18:27:28.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.154+0000 7f0f92957640 1 -- 172.21.15.5:0/3384173084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f8c0b8190 msgr2=0x7f0f8c152a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:28.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.154+0000 7f0f92957640 1 --2- 172.21.15.5:0/3384173084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f8c0b8190 0x7f0f8c152a90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0f8002a2b0 tx=0x7f0f800046c0 comp rx=0 tx=0).stop 2024-05-20T18:27:28.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.154+0000 7f0f92957640 1 -- 172.21.15.5:0/3384173084 shutdown_connections 2024-05-20T18:27:28.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.154+0000 7f0f92957640 1 --2- 172.21.15.5:0/3384173084 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0f7806be30 0x7f0f7806e340 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:28.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.154+0000 7f0f92957640 1 --2- 172.21.15.5:0/3384173084 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f8c162820 0x7f0f8c0ab5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:28.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.154+0000 7f0f92957640 1 --2- 172.21.15.5:0/3384173084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f8c0b8190 0x7f0f8c152a90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:28.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.154+0000 7f0f92957640 1 --2- 172.21.15.5:0/3384173084 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f8c06d3d0 0x7f0f8c152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:28.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.154+0000 7f0f92957640 1 -- 172.21.15.5:0/3384173084 >> 172.21.15.5:0/3384173084 conn(0x7f0f8c0d4bd0 msgr2=0x7f0f8c0cf6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:28.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.154+0000 7f0f92957640 1 -- 172.21.15.5:0/3384173084 shutdown_connections 2024-05-20T18:27:28.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.154+0000 7f0f92957640 1 -- 172.21.15.5:0/3384173084 wait complete. 2024-05-20T18:27:28.165 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:27:28.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.281+0000 7f46fdd77640 1 Processor -- start 2024-05-20T18:27:28.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.281+0000 7f46fdd77640 1 -- start start 2024-05-20T18:27:28.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.282+0000 7f46fdd77640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46f806cf00 0x7f46f806d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:28.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.282+0000 7f46fdd77640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46f808d2a0 con 0x7f46f80d4400 2024-05-20T18:27:28.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.282+0000 7f46fdd77640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46f806d850 con 0x7f46f80d2780 2024-05-20T18:27:28.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.282+0000 7f46fdd77640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f46f80d3f70 con 0x7f46f806cf00 2024-05-20T18:27:28.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.283+0000 7f46f77fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f46f80d4400 0x7f46f815f440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38564/0 (socket says 172.21.15.5:38564) 2024-05-20T18:27:28.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.283+0000 7f46f77fe640 1 -- 172.21.15.5:0/1085489781 learned_addr learned my addr 172.21.15.5:0/1085489781 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:28.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.283+0000 7f46f6ffd640 1 --2- 172.21.15.5:0/1085489781 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46f806cf00 0x7f46f806d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:28.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.283+0000 7f46f67fc640 1 -- 172.21.15.5:0/1085489781 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 262682850 0 0) 0x7f46e8001110 con 0x7f46f80d4400 2024-05-20T18:27:28.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.283+0000 7f46f67fc640 1 -- 172.21.15.5:0/1085489781 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46e0003520 con 0x7f46f80d4400 2024-05-20T18:27:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.283+0000 7f46f67fc640 1 -- 172.21.15.5:0/1085489781 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2165803387 0 0) 0x7f46f0001490 con 0x7f46f80d2780 2024-05-20T18:27:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.283+0000 7f46f67fc640 1 -- 172.21.15.5:0/1085489781 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46e8001110 con 0x7f46f80d2780 2024-05-20T18:27:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.283+0000 7f46f67fc640 1 -- 172.21.15.5:0/1085489781 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1684253417 0 0) 0x7f46e0003520 con 0x7f46f80d4400 2024-05-20T18:27:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.283+0000 7f46f67fc640 1 -- 172.21.15.5:0/1085489781 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46f806cf00 msgr2=0x7f46f806d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.283+0000 7f46f67fc640 1 --2- 172.21.15.5:0/1085489781 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46f806cf00 0x7f46f806d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.283+0000 7f46f67fc640 1 -- 172.21.15.5:0/1085489781 >> v1:172.21.15.63:6789/0 conn(0x7f46f80d2780 legacy=0x7f46f80d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.283+0000 7f46f67fc640 1 -- 172.21.15.5:0/1085489781 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f80d4810 con 0x7f46f80d4400 2024-05-20T18:27:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.283+0000 7f46f67fc640 1 -- 172.21.15.5:0/1085489781 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f46e8001840 con 0x7f46f80d4400 2024-05-20T18:27:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.283+0000 7f46f67fc640 1 -- 172.21.15.5:0/1085489781 >> v1:172.21.15.5:6789/0 conn(0x7f46f80d4400 legacy=0x7f46f815f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.283+0000 7f46f67fc640 1 --2- 172.21.15.5:0/1085489781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46e0003b70 0x7f46e0003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.283+0000 7f46f67fc640 1 --2- 172.21.15.5:0/1085489781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e0004800 0x7f46e0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.283+0000 7f46f67fc640 1 --2- 172.21.15.5:0/1085489781 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46f806cf00 0x7f46e0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.283+0000 7f46f67fc640 1 -- 172.21.15.5:0/1085489781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f46f80d3f70 con 0x7f46e0003b70 2024-05-20T18:27:28.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.283+0000 7f46f67fc640 1 -- 172.21.15.5:0/1085489781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f46e000baf0 con 0x7f46e0004800 2024-05-20T18:27:28.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.283+0000 7f46f67fc640 1 -- 172.21.15.5:0/1085489781 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f46e000bcf0 con 0x7f46f806cf00 2024-05-20T18:27:28.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.284+0000 7f46f77fe640 1 --2- 172.21.15.5:0/1085489781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46e0003b70 0x7f46e0003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:28.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.284+0000 7f46f6ffd640 1 --2- 172.21.15.5:0/1085489781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e0004800 0x7f46e0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:28.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.284+0000 7f46f7fff640 1 --2- 172.21.15.5:0/1085489781 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46f806cf00 0x7f46e0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:28.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.284+0000 7f46f77fe640 1 -- 172.21.15.5:0/1085489781 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46f806cf00 msgr2=0x7f46e0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:28.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.284+0000 7f46f77fe640 1 --2- 172.21.15.5:0/1085489781 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46f806cf00 0x7f46e0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:28.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.284+0000 7f46f77fe640 1 -- 172.21.15.5:0/1085489781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e0004800 msgr2=0x7f46e0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:28.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.284+0000 7f46f77fe640 1 --2- 172.21.15.5:0/1085489781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e0004800 0x7f46e0004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:28.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.284+0000 7f46f77fe640 1 -- 172.21.15.5:0/1085489781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e0003520 con 0x7f46e0003b70 2024-05-20T18:27:28.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.285+0000 7f46f77fe640 1 --2- 172.21.15.5:0/1085489781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46e0003b70 0x7f46e0003fe0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f46e8001eb0 tx=0x7f46e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=514174457b6c8689 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:28.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.285+0000 7f46f67fc640 1 -- 172.21.15.5:0/1085489781 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8035070 con 0x7f46e0003b70 2024-05-20T18:27:28.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.285+0000 7f46f67fc640 1 -- 172.21.15.5:0/1085489781 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e802c040 con 0x7f46e0003b70 2024-05-20T18:27:28.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.285+0000 7f46f67fc640 1 -- 172.21.15.5:0/1085489781 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e800c0e0 con 0x7f46e0003b70 2024-05-20T18:27:28.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.285+0000 7f46fdd77640 1 -- 172.21.15.5:0/1085489781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46e0003b70 msgr2=0x7f46e0003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:28.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.285+0000 7f46fdd77640 1 --2- 172.21.15.5:0/1085489781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46e0003b70 0x7f46e0003fe0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f46e8001eb0 tx=0x7f46e0005830 comp rx=0 tx=0).stop 2024-05-20T18:27:28.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.285+0000 7f46f7fff640 1 -- 172.21.15.5:0/1085489781 reap_dead start 2024-05-20T18:27:28.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.285+0000 7f46fdd77640 1 -- 172.21.15.5:0/1085489781 shutdown_connections 2024-05-20T18:27:28.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.285+0000 7f46fdd77640 1 -- 172.21.15.5:0/1085489781 >> 172.21.15.5:0/1085489781 conn(0x7f46f80d4bb0 msgr2=0x7f46f80d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:28.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.285+0000 7f46fdd77640 1 -- 172.21.15.5:0/1085489781 shutdown_connections 2024-05-20T18:27:28.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.285+0000 7f46fdd77640 1 -- 172.21.15.5:0/1085489781 wait complete. 2024-05-20T18:27:28.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.287+0000 7f46fdd77640 1 Processor -- start 2024-05-20T18:27:28.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.287+0000 7f46fdd77640 1 -- start start 2024-05-20T18:27:28.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.289+0000 7f46fdd77640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46e0003b70 0x7f46f80ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:28.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.289+0000 7f46fdd77640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46f81e8ab0 0x7f46f815dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:28.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.289+0000 7f46fdd77640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46f8162030 0x7f46f8162440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:28.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.289+0000 7f46fdd77640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f46f80d3920 con 0x7f46f8162030 2024-05-20T18:27:28.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.289+0000 7f46fdd77640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f46f80d3640 con 0x7f46f81e8ab0 2024-05-20T18:27:28.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.289+0000 7f46fdd77640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f46f80d3c00 con 0x7f46e0003b70 2024-05-20T18:27:28.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.289+0000 7f46f7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46f8162030 0x7f46f8162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:28.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.289+0000 7f46f7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46f8162030 0x7f46f8162440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58926/0 (socket says 172.21.15.5:58926) 2024-05-20T18:27:28.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.289+0000 7f46f7fff640 1 -- 172.21.15.5:0/867087876 learned_addr learned my addr 172.21.15.5:0/867087876 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:28.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.289+0000 7f46f77fe640 1 --2- 172.21.15.5:0/867087876 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46e0003b70 0x7f46f80ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:28.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.289+0000 7f46f6ffd640 1 --2- 172.21.15.5:0/867087876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46f81e8ab0 0x7f46f815dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:28.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.289+0000 7f46f7fff640 1 -- 172.21.15.5:0/867087876 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46e0003b70 msgr2=0x7f46f80ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:28.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.289+0000 7f46f7fff640 1 --2- 172.21.15.5:0/867087876 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46e0003b70 0x7f46f80ca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:28.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.289+0000 7f46f7fff640 1 -- 172.21.15.5:0/867087876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46f81e8ab0 msgr2=0x7f46f815dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:28.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.289+0000 7f46f7fff640 1 --2- 172.21.15.5:0/867087876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46f81e8ab0 0x7f46f815dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:28.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.289+0000 7f46f7fff640 1 -- 172.21.15.5:0/867087876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f8162980 con 0x7f46f8162030 2024-05-20T18:27:28.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.289+0000 7f46f7fff640 1 --2- 172.21.15.5:0/867087876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46f8162030 0x7f46f8162440 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f46f808e040 tx=0x7f46f000ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:28.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.290+0000 7f46dffff640 1 -- 172.21.15.5:0/867087876 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f0003b80 con 0x7f46f8162030 2024-05-20T18:27:28.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.290+0000 7f46dffff640 1 -- 172.21.15.5:0/867087876 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46f0018070 con 0x7f46f8162030 2024-05-20T18:27:28.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.290+0000 7f46dffff640 1 -- 172.21.15.5:0/867087876 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f00050d0 con 0x7f46f8162030 2024-05-20T18:27:28.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.290+0000 7f46fdd77640 1 -- 172.21.15.5:0/867087876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f81e7cf0 con 0x7f46f8162030 2024-05-20T18:27:28.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.290+0000 7f46fdd77640 1 -- 172.21.15.5:0/867087876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f81e82b0 con 0x7f46f8162030 2024-05-20T18:27:28.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.291+0000 7f46fdd77640 1 -- 172.21.15.5:0/867087876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46f80735a0 con 0x7f46f8162030 2024-05-20T18:27:28.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.291+0000 7f46dffff640 1 -- 172.21.15.5:0/867087876 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f46f0005270 con 0x7f46f8162030 2024-05-20T18:27:28.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.292+0000 7f46dffff640 1 --2- 172.21.15.5:0/867087876 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f46e406be30 0x7f46e406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:28.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.292+0000 7f46dffff640 1 -- 172.21.15.5:0/867087876 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f46f008adb0 con 0x7f46f8162030 2024-05-20T18:27:28.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.292+0000 7f46f77fe640 1 --2- 172.21.15.5:0/867087876 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f46e406be30 0x7f46e406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:28.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.292+0000 7f46f77fe640 1 --2- 172.21.15.5:0/867087876 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f46e406be30 0x7f46e406e340 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f46e8032000 tx=0x7f46e8009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:28.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.296+0000 7f46dffff640 1 -- 172.21.15.5:0/867087876 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f46f80735a0 con 0x7f46f8162030 2024-05-20T18:27:28.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.483+0000 7f46fdd77640 1 -- 172.21.15.5:0/867087876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f46f8075b00 con 0x7f46f8162030 2024-05-20T18:27:28.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.484+0000 7f46dffff640 1 -- 172.21.15.5:0/867087876 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18394 (secure 0 0 0) 0x7f46f8162980 con 0x7f46f8162030 2024-05-20T18:27:28.484 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:28.484 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:27:13.817633+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:27:05.476596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T18:27:06.137762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T18:27:09.829920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:27:28.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.486+0000 7f46fdd77640 1 -- 172.21.15.5:0/867087876 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f46e406be30 msgr2=0x7f46e406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:28.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.486+0000 7f46fdd77640 1 --2- 172.21.15.5:0/867087876 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f46e406be30 0x7f46e406e340 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f46e8032000 tx=0x7f46e8009030 comp rx=0 tx=0).stop 2024-05-20T18:27:28.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.486+0000 7f46fdd77640 1 -- 172.21.15.5:0/867087876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46f8162030 msgr2=0x7f46f8162440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:28.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.486+0000 7f46fdd77640 1 --2- 172.21.15.5:0/867087876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46f8162030 0x7f46f8162440 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f46f808e040 tx=0x7f46f000ea60 comp rx=0 tx=0).stop 2024-05-20T18:27:28.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.486+0000 7f46fdd77640 1 -- 172.21.15.5:0/867087876 shutdown_connections 2024-05-20T18:27:28.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.486+0000 7f46fdd77640 1 --2- 172.21.15.5:0/867087876 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f46e406be30 0x7f46e406e340 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:28.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.486+0000 7f46fdd77640 1 --2- 172.21.15.5:0/867087876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46f8162030 0x7f46f8162440 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:28.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.486+0000 7f46fdd77640 1 --2- 172.21.15.5:0/867087876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46f81e8ab0 0x7f46f815dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:28.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.486+0000 7f46fdd77640 1 --2- 172.21.15.5:0/867087876 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46e0003b70 0x7f46f80ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:28.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.486+0000 7f46fdd77640 1 -- 172.21.15.5:0/867087876 >> 172.21.15.5:0/867087876 conn(0x7f46f80d4bb0 msgr2=0x7f46f80cdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:28.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.487+0000 7f46fdd77640 1 -- 172.21.15.5:0/867087876 shutdown_connections 2024-05-20T18:27:28.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.487+0000 7f46fdd77640 1 -- 172.21.15.5:0/867087876 wait complete. 2024-05-20T18:27:28.498 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-20T18:27:28.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.616+0000 7f04bcae9640 1 Processor -- start 2024-05-20T18:27:28.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.616+0000 7f04bcae9640 1 -- start start 2024-05-20T18:27:28.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.618+0000 7f04bcae9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f04b806d3d0 0x7f04b806d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:28.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.618+0000 7f04bcae9640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f04b80b8190 0x7f04b80b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:28.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.618+0000 7f04bcae9640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04b806dd20 con 0x7f04b8162820 2024-05-20T18:27:28.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.618+0000 7f04bcae9640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f04b80d0630 con 0x7f04b806d3d0 2024-05-20T18:27:28.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.618+0000 7f04bcae9640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f04b80d39c0 con 0x7f04b80b8190 2024-05-20T18:27:28.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.618+0000 7f04b5d9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f04b806d3d0 0x7f04b806d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:28.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.618+0000 7f04b5d9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f04b806d3d0 0x7f04b806d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58938/0 (socket says 172.21.15.5:58938) 2024-05-20T18:27:28.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.618+0000 7f04b5d9b640 1 -- 172.21.15.5:0/404166971 learned_addr learned my addr 172.21.15.5:0/404166971 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:28.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.618+0000 7f04b6d9d640 1 --2- 172.21.15.5:0/404166971 >> v2:172.21.15.63:3300/0 conn(0x7f04b80b8190 0x7f04b80b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:28.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.618+0000 7f04b5d9b640 1 -- 172.21.15.5:0/404166971 >> v2:172.21.15.63:3300/0 conn(0x7f04b80b8190 msgr2=0x7f04b80b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:28.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.618+0000 7f04b5d9b640 1 --2- 172.21.15.5:0/404166971 >> v2:172.21.15.63:3300/0 conn(0x7f04b80b8190 0x7f04b80b85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:28.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.618+0000 7f04b5d9b640 1 -- 172.21.15.5:0/404166971 >> v1:172.21.15.63:6789/0 conn(0x7f04b8162820 legacy=0x7f04b80b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:28.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.618+0000 7f04b5d9b640 1 -- 172.21.15.5:0/404166971 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b80d3ca0 con 0x7f04b806d3d0 2024-05-20T18:27:28.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.619+0000 7f04b5d9b640 1 --2- 172.21.15.5:0/404166971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04b806d3d0 0x7f04b806d7e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f04b808e070 tx=0x7f04a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad637436034e7c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:28.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.619+0000 7f04b559a640 1 -- 172.21.15.5:0/404166971 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a0037070 con 0x7f04b806d3d0 2024-05-20T18:27:28.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.619+0000 7f04b559a640 1 -- 172.21.15.5:0/404166971 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04a0032440 con 0x7f04b806d3d0 2024-05-20T18:27:28.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.619+0000 7f04b559a640 1 -- 172.21.15.5:0/404166971 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a0032810 con 0x7f04b806d3d0 2024-05-20T18:27:28.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.620+0000 7f04bcae9640 1 -- 172.21.15.5:0/404166971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04b806d3d0 msgr2=0x7f04b806d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:28.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.620+0000 7f04bcae9640 1 --2- 172.21.15.5:0/404166971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04b806d3d0 0x7f04b806d7e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f04b808e070 tx=0x7f04a002a270 comp rx=0 tx=0).stop 2024-05-20T18:27:28.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.620+0000 7f04bcae9640 1 -- 172.21.15.5:0/404166971 shutdown_connections 2024-05-20T18:27:28.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.620+0000 7f04bcae9640 1 --2- 172.21.15.5:0/404166971 >> v2:172.21.15.63:3300/0 conn(0x7f04b80b8190 0x7f04b80b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:28.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.620+0000 7f04bcae9640 1 --2- 172.21.15.5:0/404166971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04b806d3d0 0x7f04b806d7e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:28.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.620+0000 7f04bcae9640 1 -- 172.21.15.5:0/404166971 >> 172.21.15.5:0/404166971 conn(0x7f04b80d4bd0 msgr2=0x7f04b80d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:28.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.620+0000 7f04bcae9640 1 -- 172.21.15.5:0/404166971 shutdown_connections 2024-05-20T18:27:28.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.620+0000 7f04bcae9640 1 -- 172.21.15.5:0/404166971 wait complete. 2024-05-20T18:27:28.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.622+0000 7f04bcae9640 1 Processor -- start 2024-05-20T18:27:28.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.622+0000 7f04bcae9640 1 -- start start 2024-05-20T18:27:28.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.623+0000 7f04bcae9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04b806d3d0 0x7f04b814e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:28.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.623+0000 7f04bcae9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04b80b8190 0x7f04b80b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:28.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.623+0000 7f04bcae9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04b8162820 0x7f04b814afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:28.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.623+0000 7f04bcae9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f04b80d0300 con 0x7f04b80b8190 2024-05-20T18:27:28.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.623+0000 7f04bcae9640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f04b80d4260 con 0x7f04b8162820 2024-05-20T18:27:28.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.623+0000 7f04bcae9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f04b80b4050 con 0x7f04b806d3d0 2024-05-20T18:27:28.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.623+0000 7f04b5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04b80b8190 0x7f04b80b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:28.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.623+0000 7f04b5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04b80b8190 0x7f04b80b5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58942/0 (socket says 172.21.15.5:58942) 2024-05-20T18:27:28.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.623+0000 7f04b5d9b640 1 -- 172.21.15.5:0/3312703452 learned_addr learned my addr 172.21.15.5:0/3312703452 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:28.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.624+0000 7f04b659c640 1 --2- 172.21.15.5:0/3312703452 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04b806d3d0 0x7f04b814e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:28.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.624+0000 7f04b6d9d640 1 --2- 172.21.15.5:0/3312703452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04b8162820 0x7f04b814afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:28.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.624+0000 7f04b5d9b640 1 -- 172.21.15.5:0/3312703452 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04b806d3d0 msgr2=0x7f04b814e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:28.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.624+0000 7f04b5d9b640 1 --2- 172.21.15.5:0/3312703452 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04b806d3d0 0x7f04b814e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:28.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.624+0000 7f04b5d9b640 1 -- 172.21.15.5:0/3312703452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04b8162820 msgr2=0x7f04b814afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:28.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.624+0000 7f04b5d9b640 1 --2- 172.21.15.5:0/3312703452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04b8162820 0x7f04b814afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:28.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.624+0000 7f04b5d9b640 1 -- 172.21.15.5:0/3312703452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b814b4f0 con 0x7f04b80b8190 2024-05-20T18:27:28.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.624+0000 7f04b5d9b640 1 --2- 172.21.15.5:0/3312703452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04b80b8190 0x7f04b80b5d60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f04a0000c00 tx=0x7f04a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:28.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.624+0000 7f04aeffd640 1 -- 172.21.15.5:0/3312703452 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a0037040 con 0x7f04b80b8190 2024-05-20T18:27:28.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.624+0000 7f04aeffd640 1 -- 172.21.15.5:0/3312703452 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04a0042070 con 0x7f04b80b8190 2024-05-20T18:27:28.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.624+0000 7f04bcae9640 1 -- 172.21.15.5:0/3312703452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04b80b6600 con 0x7f04b80b8190 2024-05-20T18:27:28.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.624+0000 7f04aeffd640 1 -- 172.21.15.5:0/3312703452 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a0030880 con 0x7f04b80b8190 2024-05-20T18:27:28.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.624+0000 7f04bcae9640 1 -- 172.21.15.5:0/3312703452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04b80b6bc0 con 0x7f04b80b8190 2024-05-20T18:27:28.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.625+0000 7f04bcae9640 1 -- 172.21.15.5:0/3312703452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04b8075b00 con 0x7f04b80b8190 2024-05-20T18:27:28.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.626+0000 7f04aeffd640 1 -- 172.21.15.5:0/3312703452 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f04a0030a20 con 0x7f04b80b8190 2024-05-20T18:27:28.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.626+0000 7f04aeffd640 1 --2- 172.21.15.5:0/3312703452 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f049806be30 0x7f049806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:28.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.626+0000 7f04aeffd640 1 -- 172.21.15.5:0/3312703452 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f04a00abad0 con 0x7f04b80b8190 2024-05-20T18:27:28.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.626+0000 7f04b659c640 1 --2- 172.21.15.5:0/3312703452 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f049806be30 0x7f049806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:28.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.627+0000 7f04b659c640 1 --2- 172.21.15.5:0/3312703452 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f049806be30 0x7f049806e340 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f04b0007fd0 tx=0x7f04b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:28.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.631+0000 7f04aeffd640 1 -- 172.21.15.5:0/3312703452 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f04a007a0b0 con 0x7f04b80b8190 2024-05-20T18:27:28.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.857+0000 7f04bcae9640 1 -- 172.21.15.5:0/3312703452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f04b808dab0 con 0x7f04b80b8190 2024-05-20T18:27:28.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.909+0000 7f04aeffd640 1 -- 172.21.15.5:0/3312703452 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7f04a00799b0 con 0x7f04b80b8190 2024-05-20T18:27:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.912+0000 7f04bcae9640 1 -- 172.21.15.5:0/3312703452 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f049806be30 msgr2=0x7f049806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.912+0000 7f04bcae9640 1 --2- 172.21.15.5:0/3312703452 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f049806be30 0x7f049806e340 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f04b0007fd0 tx=0x7f04b0009040 comp rx=0 tx=0).stop 2024-05-20T18:27:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.912+0000 7f04bcae9640 1 -- 172.21.15.5:0/3312703452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04b80b8190 msgr2=0x7f04b80b5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.912+0000 7f04bcae9640 1 --2- 172.21.15.5:0/3312703452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04b80b8190 0x7f04b80b5d60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f04a0000c00 tx=0x7f04a00046c0 comp rx=0 tx=0).stop 2024-05-20T18:27:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.912+0000 7f04bcae9640 1 -- 172.21.15.5:0/3312703452 shutdown_connections 2024-05-20T18:27:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.912+0000 7f04bcae9640 1 --2- 172.21.15.5:0/3312703452 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f049806be30 0x7f049806e340 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.912+0000 7f04bcae9640 1 --2- 172.21.15.5:0/3312703452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04b8162820 0x7f04b814afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.912+0000 7f04bcae9640 1 --2- 172.21.15.5:0/3312703452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04b80b8190 0x7f04b80b5d60 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.912+0000 7f04bcae9640 1 --2- 172.21.15.5:0/3312703452 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04b806d3d0 0x7f04b814e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.912+0000 7f04bcae9640 1 -- 172.21.15.5:0/3312703452 >> 172.21.15.5:0/3312703452 conn(0x7f04b80d4bd0 msgr2=0x7f04b815e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.912+0000 7f04bcae9640 1 -- 172.21.15.5:0/3312703452 shutdown_connections 2024-05-20T18:27:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:28.912+0000 7f04bcae9640 1 -- 172.21.15.5:0/3312703452 wait complete. 2024-05-20T18:27:28.922 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-05-20T18:27:28.922 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-20T18:27:29.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.041+0000 7f2c50621640 1 Processor -- start 2024-05-20T18:27:29.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.041+0000 7f2c50621640 1 -- start start 2024-05-20T18:27:29.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.043+0000 7f2c50621640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2c4815f700 0x7f2c4815fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:29.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.043+0000 7f2c50621640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c48160430 con 0x7f2c4806c180 2024-05-20T18:27:29.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.043+0000 7f2c50621640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c4806b6a0 con 0x7f2c480af030 2024-05-20T18:27:29.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.043+0000 7f2c50621640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2c4806bcf0 con 0x7f2c4815f700 2024-05-20T18:27:29.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.043+0000 7f2c4ebbe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2c4806c180 0x7f2c4806c590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38568/0 (socket says 172.21.15.5:38568) 2024-05-20T18:27:29.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.043+0000 7f2c4ebbe640 1 -- 172.21.15.5:0/2839758436 learned_addr learned my addr 172.21.15.5:0/2839758436 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:29.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.043+0000 7f2c4dbbc640 1 --2- 172.21.15.5:0/2839758436 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2c4815f700 0x7f2c4815fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:29.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.043+0000 7f2c4d3bb640 1 -- 172.21.15.5:0/2839758436 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2162039121 0 0) 0x7f2c48160430 con 0x7f2c4806c180 2024-05-20T18:27:29.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.043+0000 7f2c4d3bb640 1 -- 172.21.15.5:0/2839758436 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c34003520 con 0x7f2c4806c180 2024-05-20T18:27:29.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.043+0000 7f2c4d3bb640 1 -- 172.21.15.5:0/2839758436 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1555742170 0 0) 0x7f2c4806b6a0 con 0x7f2c480af030 2024-05-20T18:27:29.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.043+0000 7f2c4d3bb640 1 -- 172.21.15.5:0/2839758436 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c48160430 con 0x7f2c480af030 2024-05-20T18:27:29.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.043+0000 7f2c4d3bb640 1 -- 172.21.15.5:0/2839758436 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 932233845 0 0) 0x7f2c34003520 con 0x7f2c4806c180 2024-05-20T18:27:29.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.043+0000 7f2c4d3bb640 1 -- 172.21.15.5:0/2839758436 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2c4815f700 msgr2=0x7f2c4815fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:29.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.043+0000 7f2c4d3bb640 1 --2- 172.21.15.5:0/2839758436 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2c4815f700 0x7f2c4815fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:29.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.043+0000 7f2c4d3bb640 1 -- 172.21.15.5:0/2839758436 >> v1:172.21.15.63:6789/0 conn(0x7f2c480af030 legacy=0x7f2c480af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:29.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.043+0000 7f2c4d3bb640 1 -- 172.21.15.5:0/2839758436 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4806b980 con 0x7f2c4806c180 2024-05-20T18:27:29.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.043+0000 7f2c4d3bb640 1 -- 172.21.15.5:0/2839758436 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f2c400019f0 con 0x7f2c4806c180 2024-05-20T18:27:29.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.044+0000 7f2c4d3bb640 1 -- 172.21.15.5:0/2839758436 >> v1:172.21.15.5:6789/0 conn(0x7f2c4806c180 legacy=0x7f2c4806c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:29.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.044+0000 7f2c4d3bb640 1 --2- 172.21.15.5:0/2839758436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c34003b70 0x7f2c34003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:29.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.044+0000 7f2c4d3bb640 1 --2- 172.21.15.5:0/2839758436 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2c4815f700 0x7f2c34004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:29.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.044+0000 7f2c4d3bb640 1 --2- 172.21.15.5:0/2839758436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c34004d40 0x7f2c34005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:29.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.044+0000 7f2c4d3bb640 1 -- 172.21.15.5:0/2839758436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4806bcf0 con 0x7f2c34003b70 2024-05-20T18:27:29.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.044+0000 7f2c4d3bb640 1 -- 172.21.15.5:0/2839758436 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3400bac0 con 0x7f2c34004d40 2024-05-20T18:27:29.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.044+0000 7f2c4d3bb640 1 -- 172.21.15.5:0/2839758436 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2c3400bcc0 con 0x7f2c4815f700 2024-05-20T18:27:29.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.044+0000 7f2c4e3bd640 1 --2- 172.21.15.5:0/2839758436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c34003b70 0x7f2c34003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:29.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.044+0000 7f2c4ebbe640 1 --2- 172.21.15.5:0/2839758436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c34004d40 0x7f2c34005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:29.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.044+0000 7f2c4dbbc640 1 --2- 172.21.15.5:0/2839758436 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2c4815f700 0x7f2c34004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:29.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.044+0000 7f2c4e3bd640 1 -- 172.21.15.5:0/2839758436 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2c4815f700 msgr2=0x7f2c34004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:29.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.044+0000 7f2c4e3bd640 1 --2- 172.21.15.5:0/2839758436 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2c4815f700 0x7f2c34004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:29.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.044+0000 7f2c4e3bd640 1 -- 172.21.15.5:0/2839758436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c34004d40 msgr2=0x7f2c34005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:29.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.044+0000 7f2c4e3bd640 1 --2- 172.21.15.5:0/2839758436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c34004d40 0x7f2c34005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:29.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.044+0000 7f2c4e3bd640 1 -- 172.21.15.5:0/2839758436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c34003520 con 0x7f2c34003b70 2024-05-20T18:27:29.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.044+0000 7f2c4ebbe640 1 --2- 172.21.15.5:0/2839758436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c34004d40 0x7f2c34005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:29.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.044+0000 7f2c4dbbc640 1 --2- 172.21.15.5:0/2839758436 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2c4815f700 0x7f2c34004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:29.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.045+0000 7f2c4e3bd640 1 --2- 172.21.15.5:0/2839758436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c34003b70 0x7f2c34003fc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2c440018a0 tx=0x7f2c34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f93f87fd8efb70b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:29.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.045+0000 7f2c4d3bb640 1 -- 172.21.15.5:0/2839758436 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c440014f0 con 0x7f2c34003b70 2024-05-20T18:27:29.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.045+0000 7f2c4d3bb640 1 -- 172.21.15.5:0/2839758436 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c440052d0 con 0x7f2c34003b70 2024-05-20T18:27:29.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.045+0000 7f2c4d3bb640 1 -- 172.21.15.5:0/2839758436 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4402ac60 con 0x7f2c34003b70 2024-05-20T18:27:29.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.046+0000 7f2c50621640 1 -- 172.21.15.5:0/2839758436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c34003b70 msgr2=0x7f2c34003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:29.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.046+0000 7f2c50621640 1 --2- 172.21.15.5:0/2839758436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c34003b70 0x7f2c34003fc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2c440018a0 tx=0x7f2c34005830 comp rx=0 tx=0).stop 2024-05-20T18:27:29.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.046+0000 7f2c4ebbe640 1 -- 172.21.15.5:0/2839758436 reap_dead start 2024-05-20T18:27:29.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.046+0000 7f2c50621640 1 -- 172.21.15.5:0/2839758436 shutdown_connections 2024-05-20T18:27:29.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.046+0000 7f2c50621640 1 -- 172.21.15.5:0/2839758436 >> 172.21.15.5:0/2839758436 conn(0x7f2c480b6620 msgr2=0x7f2c480b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:29.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.046+0000 7f2c50621640 1 -- 172.21.15.5:0/2839758436 shutdown_connections 2024-05-20T18:27:29.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.046+0000 7f2c50621640 1 -- 172.21.15.5:0/2839758436 wait complete. 2024-05-20T18:27:29.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.047+0000 7f2c50621640 1 Processor -- start 2024-05-20T18:27:29.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.048+0000 7f2c50621640 1 -- start start 2024-05-20T18:27:29.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.049+0000 7f2c50621640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c34003b70 0x7f2c4815bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:29.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.049+0000 7f2c50621640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c48156ab0 0x7f2c4814e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:29.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.049+0000 7f2c50621640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2c48153930 0x7f2c48159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:29.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.049+0000 7f2c50621640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4815fc90 con 0x7f2c34003b70 2024-05-20T18:27:29.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.049+0000 7f2c50621640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4815fb10 con 0x7f2c48156ab0 2024-05-20T18:27:29.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.049+0000 7f2c50621640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2c4815fe10 con 0x7f2c48153930 2024-05-20T18:27:29.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.049+0000 7f2c4e3bd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c34003b70 0x7f2c4815bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:29.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.049+0000 7f2c4e3bd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c34003b70 0x7f2c4815bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58958/0 (socket says 172.21.15.5:58958) 2024-05-20T18:27:29.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.049+0000 7f2c4e3bd640 1 -- 172.21.15.5:0/1456391426 learned_addr learned my addr 172.21.15.5:0/1456391426 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:29.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.049+0000 7f2c4dbbc640 1 --2- 172.21.15.5:0/1456391426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c48156ab0 0x7f2c4814e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:29.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.049+0000 7f2c4ebbe640 1 --2- 172.21.15.5:0/1456391426 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2c48153930 0x7f2c48159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:29.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.049+0000 7f2c4e3bd640 1 -- 172.21.15.5:0/1456391426 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2c48153930 msgr2=0x7f2c48159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:29.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.049+0000 7f2c4e3bd640 1 --2- 172.21.15.5:0/1456391426 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2c48153930 0x7f2c48159cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:29.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.049+0000 7f2c4e3bd640 1 -- 172.21.15.5:0/1456391426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c48156ab0 msgr2=0x7f2c4814e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:29.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.049+0000 7f2c4e3bd640 1 --2- 172.21.15.5:0/1456391426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c48156ab0 0x7f2c4814e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:29.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.049+0000 7f2c4e3bd640 1 -- 172.21.15.5:0/1456391426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4815c590 con 0x7f2c34003b70 2024-05-20T18:27:29.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.050+0000 7f2c4e3bd640 1 --2- 172.21.15.5:0/1456391426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c34003b70 0x7f2c4815bcb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2c44005040 tx=0x7f2c440012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:29.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.050+0000 7f2c3effd640 1 -- 172.21.15.5:0/1456391426 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4403f020 con 0x7f2c34003b70 2024-05-20T18:27:29.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.050+0000 7f2c3effd640 1 -- 172.21.15.5:0/1456391426 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c44005480 con 0x7f2c34003b70 2024-05-20T18:27:29.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.050+0000 7f2c50621640 1 -- 172.21.15.5:0/1456391426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c4815c870 con 0x7f2c34003b70 2024-05-20T18:27:29.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.050+0000 7f2c3effd640 1 -- 172.21.15.5:0/1456391426 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c44037040 con 0x7f2c34003b70 2024-05-20T18:27:29.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.050+0000 7f2c50621640 1 -- 172.21.15.5:0/1456391426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c48156f20 con 0x7f2c34003b70 2024-05-20T18:27:29.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.051+0000 7f2c50621640 1 -- 172.21.15.5:0/1456391426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c48075b00 con 0x7f2c34003b70 2024-05-20T18:27:29.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.051+0000 7f2c3effd640 1 -- 172.21.15.5:0/1456391426 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2c44042020 con 0x7f2c34003b70 2024-05-20T18:27:29.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.052+0000 7f2c3effd640 1 --2- 172.21.15.5:0/1456391426 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2c2c06be30 0x7f2c2c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:29.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.052+0000 7f2c3effd640 1 -- 172.21.15.5:0/1456391426 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2c440aac70 con 0x7f2c34003b70 2024-05-20T18:27:29.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.052+0000 7f2c4dbbc640 1 --2- 172.21.15.5:0/1456391426 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2c2c06be30 0x7f2c2c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:29.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.052+0000 7f2c4dbbc640 1 --2- 172.21.15.5:0/1456391426 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2c2c06be30 0x7f2c2c06e340 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2c481588d0 tx=0x7f2c38008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:29.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.057+0000 7f2c3effd640 1 -- 172.21.15.5:0/1456391426 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2c44049030 con 0x7f2c34003b70 2024-05-20T18:27:29.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.287+0000 7f2c50621640 1 -- 172.21.15.5:0/1456391426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f2c48157150 con 0x7f2c34003b70 2024-05-20T18:27:29.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.910+0000 7f2c3effd640 1 -- 172.21.15.5:0/1456391426 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f2c44078c90 con 0x7f2c34003b70 2024-05-20T18:27:29.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.913+0000 7f2c50621640 1 -- 172.21.15.5:0/1456391426 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2c2c06be30 msgr2=0x7f2c2c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:29.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.913+0000 7f2c50621640 1 --2- 172.21.15.5:0/1456391426 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2c2c06be30 0x7f2c2c06e340 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2c481588d0 tx=0x7f2c38008040 comp rx=0 tx=0).stop 2024-05-20T18:27:29.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.913+0000 7f2c50621640 1 -- 172.21.15.5:0/1456391426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c34003b70 msgr2=0x7f2c4815bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:29.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.913+0000 7f2c50621640 1 --2- 172.21.15.5:0/1456391426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c34003b70 0x7f2c4815bcb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2c44005040 tx=0x7f2c440012d0 comp rx=0 tx=0).stop 2024-05-20T18:27:29.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.913+0000 7f2c50621640 1 -- 172.21.15.5:0/1456391426 shutdown_connections 2024-05-20T18:27:29.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.913+0000 7f2c50621640 1 --2- 172.21.15.5:0/1456391426 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2c2c06be30 0x7f2c2c06e340 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:29.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.913+0000 7f2c50621640 1 --2- 172.21.15.5:0/1456391426 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2c48153930 0x7f2c48159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:29.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.913+0000 7f2c50621640 1 --2- 172.21.15.5:0/1456391426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c48156ab0 0x7f2c4814e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:29.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.913+0000 7f2c50621640 1 --2- 172.21.15.5:0/1456391426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c34003b70 0x7f2c4815bcb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:29.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.913+0000 7f2c50621640 1 -- 172.21.15.5:0/1456391426 >> 172.21.15.5:0/1456391426 conn(0x7f2c480b6620 msgr2=0x7f2c480b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:29.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.914+0000 7f2c50621640 1 -- 172.21.15.5:0/1456391426 shutdown_connections 2024-05-20T18:27:29.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:29.914+0000 7f2c50621640 1 -- 172.21.15.5:0/1456391426 wait complete. 2024-05-20T18:27:29.926 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-20T18:27:29.926 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:27:30.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.044+0000 7fbac84e9640 1 Processor -- start 2024-05-20T18:27:30.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.045+0000 7fbac84e9640 1 -- start start 2024-05-20T18:27:30.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.046+0000 7fbac84e9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbac0162820 0x7fbac00b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:30.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.046+0000 7fbac84e9640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbac00b3e80 0x7fbac006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:30.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.046+0000 7fbac84e9640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbac006dd20 con 0x7fbac006d910 2024-05-20T18:27:30.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.046+0000 7fbac84e9640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fbac00d0630 con 0x7fbac0162820 2024-05-20T18:27:30.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.046+0000 7fbac84e9640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fbac00d39c0 con 0x7fbac00b3e80 2024-05-20T18:27:30.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.046+0000 7fbac6285640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbac0162820 0x7fbac00b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:30.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.046+0000 7fbac6285640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbac0162820 0x7fbac00b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58960/0 (socket says 172.21.15.5:58960) 2024-05-20T18:27:30.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.046+0000 7fbac6285640 1 -- 172.21.15.5:0/4232521714 learned_addr learned my addr 172.21.15.5:0/4232521714 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:30.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.046+0000 7fbac5a84640 1 --2- 172.21.15.5:0/4232521714 >> v2:172.21.15.63:3300/0 conn(0x7fbac00b3e80 0x7fbac006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:30.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.047+0000 7fbac6285640 1 -- 172.21.15.5:0/4232521714 >> v2:172.21.15.63:3300/0 conn(0x7fbac00b3e80 msgr2=0x7fbac006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:30.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.047+0000 7fbac6285640 1 --2- 172.21.15.5:0/4232521714 >> v2:172.21.15.63:3300/0 conn(0x7fbac00b3e80 0x7fbac006d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:30.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.047+0000 7fbac6285640 1 -- 172.21.15.5:0/4232521714 >> v1:172.21.15.63:6789/0 conn(0x7fbac006d910 legacy=0x7fbac00d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:30.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.047+0000 7fbac6285640 1 -- 172.21.15.5:0/4232521714 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac00d3ca0 con 0x7fbac0162820 2024-05-20T18:27:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.047+0000 7fbac6285640 1 --2- 172.21.15.5:0/4232521714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbac0162820 0x7fbac00b3940 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbac00d1230 tx=0x7fbabc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ce4e86c7d59a66 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.048+0000 7fbac5283640 1 -- 172.21.15.5:0/4232521714 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc037070 con 0x7fbac0162820 2024-05-20T18:27:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.048+0000 7fbac5283640 1 -- 172.21.15.5:0/4232521714 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbabc032440 con 0x7fbac0162820 2024-05-20T18:27:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.048+0000 7fbac5283640 1 -- 172.21.15.5:0/4232521714 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc032810 con 0x7fbac0162820 2024-05-20T18:27:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.048+0000 7fbac84e9640 1 -- 172.21.15.5:0/4232521714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbac0162820 msgr2=0x7fbac00b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.048+0000 7fbac84e9640 1 --2- 172.21.15.5:0/4232521714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbac0162820 0x7fbac00b3940 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbac00d1230 tx=0x7fbabc02a270 comp rx=0 tx=0).stop 2024-05-20T18:27:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.048+0000 7fbac84e9640 1 -- 172.21.15.5:0/4232521714 shutdown_connections 2024-05-20T18:27:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.048+0000 7fbac84e9640 1 --2- 172.21.15.5:0/4232521714 >> v2:172.21.15.63:3300/0 conn(0x7fbac00b3e80 0x7fbac006d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.048+0000 7fbac84e9640 1 --2- 172.21.15.5:0/4232521714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbac0162820 0x7fbac00b3940 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.048+0000 7fbac84e9640 1 -- 172.21.15.5:0/4232521714 >> 172.21.15.5:0/4232521714 conn(0x7fbac00d4bd0 msgr2=0x7fbac00d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:30.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.048+0000 7fbac84e9640 1 -- 172.21.15.5:0/4232521714 shutdown_connections 2024-05-20T18:27:30.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.048+0000 7fbac84e9640 1 -- 172.21.15.5:0/4232521714 wait complete. 2024-05-20T18:27:30.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.050+0000 7fbac84e9640 1 Processor -- start 2024-05-20T18:27:30.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.050+0000 7fbac84e9640 1 -- start start 2024-05-20T18:27:30.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.051+0000 7fbac84e9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbac006d910 0x7fbac0151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:30.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.051+0000 7fbac84e9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbac00b3e80 0x7fbac01f85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:30.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.051+0000 7fbac84e9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbac0162820 0x7fbac015fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:30.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.052+0000 7fbac84e9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbac00d0300 con 0x7fbac00b3e80 2024-05-20T18:27:30.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.052+0000 7fbac84e9640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbac00d4260 con 0x7fbac0162820 2024-05-20T18:27:30.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.052+0000 7fbac84e9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbac00b91a0 con 0x7fbac006d910 2024-05-20T18:27:30.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.052+0000 7fbac5a84640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbac00b3e80 0x7fbac01f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:30.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.052+0000 7fbac5a84640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbac00b3e80 0x7fbac01f85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58968/0 (socket says 172.21.15.5:58968) 2024-05-20T18:27:30.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.052+0000 7fbac5a84640 1 -- 172.21.15.5:0/151106503 learned_addr learned my addr 172.21.15.5:0/151106503 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:30.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.052+0000 7fbac6a86640 1 --2- 172.21.15.5:0/151106503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbac0162820 0x7fbac015fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:30.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.052+0000 7fbac6285640 1 --2- 172.21.15.5:0/151106503 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbac006d910 0x7fbac0151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:30.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.052+0000 7fbac5a84640 1 -- 172.21.15.5:0/151106503 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbac006d910 msgr2=0x7fbac0151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:30.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.052+0000 7fbac5a84640 1 --2- 172.21.15.5:0/151106503 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbac006d910 0x7fbac0151130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:30.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.052+0000 7fbac5a84640 1 -- 172.21.15.5:0/151106503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbac0162820 msgr2=0x7fbac015fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:30.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.052+0000 7fbac5a84640 1 --2- 172.21.15.5:0/151106503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbac0162820 0x7fbac015fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:30.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.052+0000 7fbac5a84640 1 -- 172.21.15.5:0/151106503 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac01f9290 con 0x7fbac00b3e80 2024-05-20T18:27:30.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.052+0000 7fbac6a86640 1 --2- 172.21.15.5:0/151106503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbac0162820 0x7fbac015fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:30.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.052+0000 7fbac5a84640 1 --2- 172.21.15.5:0/151106503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbac00b3e80 0x7fbac01f85d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbab000e8a0 tx=0x7fbab000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:30.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.052+0000 7fbab6ffd640 1 -- 172.21.15.5:0/151106503 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab0017070 con 0x7fbac00b3e80 2024-05-20T18:27:30.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.052+0000 7fbab6ffd640 1 -- 172.21.15.5:0/151106503 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbab000c940 con 0x7fbac00b3e80 2024-05-20T18:27:30.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.052+0000 7fbac84e9640 1 -- 172.21.15.5:0/151106503 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbac00cbed0 con 0x7fbac00b3e80 2024-05-20T18:27:30.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.052+0000 7fbab6ffd640 1 -- 172.21.15.5:0/151106503 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab000cd30 con 0x7fbac00b3e80 2024-05-20T18:27:30.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.052+0000 7fbac84e9640 1 -- 172.21.15.5:0/151106503 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbac00cda30 con 0x7fbac00b3e80 2024-05-20T18:27:30.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.053+0000 7fbac84e9640 1 -- 172.21.15.5:0/151106503 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbac01f9000 con 0x7fbac00b3e80 2024-05-20T18:27:30.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.054+0000 7fbab6ffd640 1 -- 172.21.15.5:0/151106503 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbab001a070 con 0x7fbac00b3e80 2024-05-20T18:27:30.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.054+0000 7fbab6ffd640 1 --2- 172.21.15.5:0/151106503 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbaa406be60 0x7fbaa406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:30.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.055+0000 7fbab6ffd640 1 -- 172.21.15.5:0/151106503 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fbab008b440 con 0x7fbac00b3e80 2024-05-20T18:27:30.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.055+0000 7fbac6285640 1 --2- 172.21.15.5:0/151106503 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbaa406be60 0x7fbaa406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:30.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.055+0000 7fbac6285640 1 --2- 172.21.15.5:0/151106503 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbaa406be60 0x7fbaa406e370 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbabc034000 tx=0x7fbabc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:30.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.059+0000 7fbab6ffd640 1 -- 172.21.15.5:0/151106503 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbab0059a20 con 0x7fbac00b3e80 2024-05-20T18:27:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.280+0000 7fbac84e9640 1 -- 172.21.15.5:0/151106503 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbac0075b00 con 0x7fbac00b3e80 2024-05-20T18:27:30.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.281+0000 7fbab6ffd640 1 -- 172.21.15.5:0/151106503 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5168 (secure 0 0 0) 0x7fbab0059320 con 0x7fbac00b3e80 2024-05-20T18:27:30.280 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:30.281 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2982954341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2982954341},{"type":"v1","addr":"172.21.15.5:6836","nonce":2982954341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4032850001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":4032850001},{"type":"v1","addr":"172.21.15.5:6837","nonce":4032850001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6833/424580378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":424580378},{"type":"v1","addr":"172.21.15.63:6833","nonce":424580378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:27:07.164414+0000","modified":"2024-05-20T18:27:29.288681+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/3249423677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3249423677},{"type":"v1","addr":"172.21.15.63:6835","nonce":3249423677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-05-20T18:27:30.281 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 9 2024-05-20T18:27:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.283+0000 7fbac84e9640 1 -- 172.21.15.5:0/151106503 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbaa406be60 msgr2=0x7fbaa406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.283+0000 7fbac84e9640 1 --2- 172.21.15.5:0/151106503 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbaa406be60 0x7fbaa406e370 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbabc034000 tx=0x7fbabc02e000 comp rx=0 tx=0).stop 2024-05-20T18:27:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.283+0000 7fbac84e9640 1 -- 172.21.15.5:0/151106503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbac00b3e80 msgr2=0x7fbac01f85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.283+0000 7fbac84e9640 1 --2- 172.21.15.5:0/151106503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbac00b3e80 0x7fbac01f85d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbab000e8a0 tx=0x7fbab000ed60 comp rx=0 tx=0).stop 2024-05-20T18:27:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.283+0000 7fbac84e9640 1 -- 172.21.15.5:0/151106503 shutdown_connections 2024-05-20T18:27:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.283+0000 7fbac84e9640 1 --2- 172.21.15.5:0/151106503 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbaa406be60 0x7fbaa406e370 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.283+0000 7fbac84e9640 1 --2- 172.21.15.5:0/151106503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbac0162820 0x7fbac015fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.283+0000 7fbac84e9640 1 --2- 172.21.15.5:0/151106503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbac00b3e80 0x7fbac01f85d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.283+0000 7fbac84e9640 1 --2- 172.21.15.5:0/151106503 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbac006d910 0x7fbac0151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.283+0000 7fbac84e9640 1 -- 172.21.15.5:0/151106503 >> 172.21.15.5:0/151106503 conn(0x7fbac00d4bd0 msgr2=0x7fbac00d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.283+0000 7fbac84e9640 1 -- 172.21.15.5:0/151106503 shutdown_connections 2024-05-20T18:27:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.283+0000 7fbac84e9640 1 -- 172.21.15.5:0/151106503 wait complete. 2024-05-20T18:27:30.294 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:27:30.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.409+0000 7fc0f93ea640 1 Processor -- start 2024-05-20T18:27:30.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.409+0000 7fc0f93ea640 1 -- start start 2024-05-20T18:27:30.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.410+0000 7fc0f93ea640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc0f40b6620 0x7fc0f40ba3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:30.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.410+0000 7fc0f93ea640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0f41634a0 con 0x7fc0f4162ce0 2024-05-20T18:27:30.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.410+0000 7fc0f93ea640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0f4163780 con 0x7fc0f40d3bf0 2024-05-20T18:27:30.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.410+0000 7fc0f93ea640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc0f40d3480 con 0x7fc0f40b6620 2024-05-20T18:27:30.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.410+0000 7fc0f27fc640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc0f4162ce0 0x7fc0f40b8070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38572/0 (socket says 172.21.15.5:38572) 2024-05-20T18:27:30.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.410+0000 7fc0f27fc640 1 -- 172.21.15.5:0/2378644516 learned_addr learned my addr 172.21.15.5:0/2378644516 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:30.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.410+0000 7fc0f2ffd640 1 --2- 172.21.15.5:0/2378644516 >> v2:172.21.15.63:3300/0 conn(0x7fc0f40b6620 0x7fc0f40ba3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:30.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.411+0000 7fc0f1ffb640 1 -- 172.21.15.5:0/2378644516 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3837936212 0 0) 0x7fc0f41634a0 con 0x7fc0f4162ce0 2024-05-20T18:27:30.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.411+0000 7fc0f1ffb640 1 -- 172.21.15.5:0/2378644516 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0dc003520 con 0x7fc0f4162ce0 2024-05-20T18:27:30.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.411+0000 7fc0f1ffb640 1 -- 172.21.15.5:0/2378644516 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 424201844 0 0) 0x7fc0f4163780 con 0x7fc0f40d3bf0 2024-05-20T18:27:30.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.411+0000 7fc0f1ffb640 1 -- 172.21.15.5:0/2378644516 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0f41634a0 con 0x7fc0f40d3bf0 2024-05-20T18:27:30.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.411+0000 7fc0f2ffd640 1 -- 172.21.15.5:0/2378644516 >> v1:172.21.15.63:6789/0 conn(0x7fc0f40d3bf0 legacy=0x7fc0f40d4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:30.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.411+0000 7fc0f1ffb640 1 -- 172.21.15.5:0/2378644516 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2453691160 0 0) 0x7fc0dc003520 con 0x7fc0f4162ce0 2024-05-20T18:27:30.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.411+0000 7fc0f2ffd640 1 -- 172.21.15.5:0/2378644516 >> v1:172.21.15.5:6789/0 conn(0x7fc0f4162ce0 legacy=0x7fc0f40b8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:30.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.411+0000 7fc0f2ffd640 1 -- 172.21.15.5:0/2378644516 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f4163a60 con 0x7fc0f40b6620 2024-05-20T18:27:30.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.412+0000 7fc0f2ffd640 1 --2- 172.21.15.5:0/2378644516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0f40b6620 0x7fc0f40ba3c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009d90 tx=0x7fc0e402a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91ce9e56e0c2ab41 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:30.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.412+0000 7fc0f1ffb640 1 -- 172.21.15.5:0/2378644516 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8001670 con 0x7fc0f40b6620 2024-05-20T18:27:30.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.412+0000 7fc0f1ffb640 1 -- 172.21.15.5:0/2378644516 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0e4007540 con 0x7fc0f40b6620 2024-05-20T18:27:30.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.412+0000 7fc0f1ffb640 1 -- 172.21.15.5:0/2378644516 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4007910 con 0x7fc0f40b6620 2024-05-20T18:27:30.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.412+0000 7fc0f93ea640 1 -- 172.21.15.5:0/2378644516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0f40b6620 msgr2=0x7fc0f40ba3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:30.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.412+0000 7fc0f93ea640 1 --2- 172.21.15.5:0/2378644516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0f40b6620 0x7fc0f40ba3c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009d90 tx=0x7fc0e402a680 comp rx=0 tx=0).stop 2024-05-20T18:27:30.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.413+0000 7fc0f93ea640 1 -- 172.21.15.5:0/2378644516 shutdown_connections 2024-05-20T18:27:30.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.413+0000 7fc0f93ea640 1 --2- 172.21.15.5:0/2378644516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0f40b6620 0x7fc0f40ba3c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:30.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.413+0000 7fc0f93ea640 1 -- 172.21.15.5:0/2378644516 >> 172.21.15.5:0/2378644516 conn(0x7fc0f406d240 msgr2=0x7fc0f406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:30.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.413+0000 7fc0f93ea640 1 -- 172.21.15.5:0/2378644516 shutdown_connections 2024-05-20T18:27:30.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.413+0000 7fc0f93ea640 1 -- 172.21.15.5:0/2378644516 wait complete. 2024-05-20T18:27:30.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.414+0000 7fc0f93ea640 1 Processor -- start 2024-05-20T18:27:30.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.415+0000 7fc0f93ea640 1 -- start start 2024-05-20T18:27:30.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.416+0000 7fc0f93ea640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0f40b6620 0x7fc0f415c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:30.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.416+0000 7fc0f93ea640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0f40d3bf0 0x7fc0f415cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:30.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.416+0000 7fc0f93ea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0f4162ce0 0x7fc0f4145110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:30.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.416+0000 7fc0f93ea640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f41630f0 con 0x7fc0f4162ce0 2024-05-20T18:27:30.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.416+0000 7fc0f93ea640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f40b8780 con 0x7fc0f40b6620 2024-05-20T18:27:30.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.416+0000 7fc0f93ea640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc0f4145d90 con 0x7fc0f40d3bf0 2024-05-20T18:27:30.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.416+0000 7fc0f37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0f4162ce0 0x7fc0f4145110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:30.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.416+0000 7fc0f37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0f4162ce0 0x7fc0f4145110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58984/0 (socket says 172.21.15.5:58984) 2024-05-20T18:27:30.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.416+0000 7fc0f37fe640 1 -- 172.21.15.5:0/1835639584 learned_addr learned my addr 172.21.15.5:0/1835639584 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:30.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.416+0000 7fc0f2ffd640 1 --2- 172.21.15.5:0/1835639584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0f40b6620 0x7fc0f415c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:30.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.416+0000 7fc0f27fc640 1 --2- 172.21.15.5:0/1835639584 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0f40d3bf0 0x7fc0f415cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:30.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.416+0000 7fc0f37fe640 1 -- 172.21.15.5:0/1835639584 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0f40d3bf0 msgr2=0x7fc0f415cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:30.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.416+0000 7fc0f37fe640 1 --2- 172.21.15.5:0/1835639584 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0f40d3bf0 0x7fc0f415cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:30.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.416+0000 7fc0f37fe640 1 -- 172.21.15.5:0/1835639584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0f40b6620 msgr2=0x7fc0f415c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:30.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.416+0000 7fc0f37fe640 1 --2- 172.21.15.5:0/1835639584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0f40b6620 0x7fc0f415c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:30.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.416+0000 7fc0f37fe640 1 -- 172.21.15.5:0/1835639584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f415d0f0 con 0x7fc0f4162ce0 2024-05-20T18:27:30.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.416+0000 7fc0f27fc640 1 --2- 172.21.15.5:0/1835639584 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0f40d3bf0 0x7fc0f415cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:30.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.417+0000 7fc0f37fe640 1 --2- 172.21.15.5:0/1835639584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0f4162ce0 0x7fc0f4145110 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00c3f0 tx=0x7fc0ec001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:30.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.417+0000 7fc0db7fe640 1 -- 172.21.15.5:0/1835639584 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec017070 con 0x7fc0f4162ce0 2024-05-20T18:27:30.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.417+0000 7fc0db7fe640 1 -- 172.21.15.5:0/1835639584 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0ec0052d0 con 0x7fc0f4162ce0 2024-05-20T18:27:30.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.417+0000 7fc0f93ea640 1 -- 172.21.15.5:0/1835639584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0f415b2d0 con 0x7fc0f4162ce0 2024-05-20T18:27:30.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.417+0000 7fc0db7fe640 1 -- 172.21.15.5:0/1835639584 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec003830 con 0x7fc0f4162ce0 2024-05-20T18:27:30.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.417+0000 7fc0f93ea640 1 -- 172.21.15.5:0/1835639584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0f415b890 con 0x7fc0f4162ce0 2024-05-20T18:27:30.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.418+0000 7fc0f93ea640 1 -- 172.21.15.5:0/1835639584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0f4075b00 con 0x7fc0f4162ce0 2024-05-20T18:27:30.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.418+0000 7fc0db7fe640 1 -- 172.21.15.5:0/1835639584 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc0ec01a070 con 0x7fc0f4162ce0 2024-05-20T18:27:30.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.419+0000 7fc0db7fe640 1 --2- 172.21.15.5:0/1835639584 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc0e006be30 0x7fc0e006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:30.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.419+0000 7fc0f2ffd640 1 --2- 172.21.15.5:0/1835639584 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc0e006be30 0x7fc0e006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:30.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.419+0000 7fc0db7fe640 1 -- 172.21.15.5:0/1835639584 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc0ec08a470 con 0x7fc0f4162ce0 2024-05-20T18:27:30.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.419+0000 7fc0f2ffd640 1 --2- 172.21.15.5:0/1835639584 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc0e006be30 0x7fc0e006e340 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4034000 tx=0x7fc0e402f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:30.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.424+0000 7fc0db7fe640 1 -- 172.21.15.5:0/1835639584 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc0ec0583f0 con 0x7fc0f4162ce0 2024-05-20T18:27:30.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.608+0000 7fc0f93ea640 1 -- 172.21.15.5:0/1835639584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0f40d2fa0 con 0x7fc0f4162ce0 2024-05-20T18:27:30.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.610+0000 7fc0db7fe640 1 -- 172.21.15.5:0/1835639584 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18394 (secure 0 0 0) 0x7fc0ec05c6f0 con 0x7fc0f4162ce0 2024-05-20T18:27:30.609 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:30.609 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:27:13.817633+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:27:05.476596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T18:27:06.137762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T18:27:09.829920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:27:30.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.611+0000 7fc0f93ea640 1 -- 172.21.15.5:0/1835639584 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc0e006be30 msgr2=0x7fc0e006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:30.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.612+0000 7fc0f93ea640 1 --2- 172.21.15.5:0/1835639584 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc0e006be30 0x7fc0e006e340 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4034000 tx=0x7fc0e402f000 comp rx=0 tx=0).stop 2024-05-20T18:27:30.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.612+0000 7fc0f93ea640 1 -- 172.21.15.5:0/1835639584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0f4162ce0 msgr2=0x7fc0f4145110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:30.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.612+0000 7fc0f93ea640 1 --2- 172.21.15.5:0/1835639584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0f4162ce0 0x7fc0f4145110 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00c3f0 tx=0x7fc0ec001e70 comp rx=0 tx=0).stop 2024-05-20T18:27:30.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.612+0000 7fc0f93ea640 1 -- 172.21.15.5:0/1835639584 shutdown_connections 2024-05-20T18:27:30.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.612+0000 7fc0f93ea640 1 --2- 172.21.15.5:0/1835639584 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc0e006be30 0x7fc0e006e340 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:30.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.612+0000 7fc0f93ea640 1 --2- 172.21.15.5:0/1835639584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0f4162ce0 0x7fc0f4145110 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:30.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.612+0000 7fc0f93ea640 1 --2- 172.21.15.5:0/1835639584 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0f40d3bf0 0x7fc0f415cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:30.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.612+0000 7fc0f93ea640 1 --2- 172.21.15.5:0/1835639584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0f40b6620 0x7fc0f415c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:30.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.612+0000 7fc0f93ea640 1 -- 172.21.15.5:0/1835639584 >> 172.21.15.5:0/1835639584 conn(0x7fc0f406d240 msgr2=0x7fc0f40b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:30.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.612+0000 7fc0f93ea640 1 -- 172.21.15.5:0/1835639584 shutdown_connections 2024-05-20T18:27:30.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.612+0000 7fc0f93ea640 1 -- 172.21.15.5:0/1835639584 wait complete. 2024-05-20T18:27:30.623 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-20T18:27:30.623 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T18:27:30.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.739+0000 7f060eac3640 1 Processor -- start 2024-05-20T18:27:30.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.739+0000 7f060eac3640 1 -- start start 2024-05-20T18:27:30.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.740+0000 7f060eac3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f06080d10e0 0x7f06080cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:30.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.740+0000 7f060eac3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06080d14f0 0x7f06080cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:30.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.741+0000 7f060eac3640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f06080cd4d0 0x7f06080cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:30.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.741+0000 7f060eac3640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f06080d1ac0 con 0x7f06080d10e0 2024-05-20T18:27:30.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.741+0000 7f060eac3640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f06080d1da0 con 0x7f06080cd4d0 2024-05-20T18:27:30.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.741+0000 7f060eac3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f06080d2080 con 0x7f06080d14f0 2024-05-20T18:27:30.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.741+0000 7f060c85f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f06080d10e0 0x7f06080cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:30.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.741+0000 7f060c85f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f06080d10e0 0x7f06080cf140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58986/0 (socket says 172.21.15.5:58986) 2024-05-20T18:27:30.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.741+0000 7f060c85f640 1 -- 172.21.15.5:0/1624645560 learned_addr learned my addr 172.21.15.5:0/1624645560 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:30.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.741+0000 7f0607fff640 1 --2- 172.21.15.5:0/1624645560 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06080d14f0 0x7f06080cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:30.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.741+0000 7f060d060640 1 --2- 172.21.15.5:0/1624645560 >> v2:172.21.15.63:3300/0 conn(0x7f06080cd4d0 0x7f06080cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:30.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.741+0000 7f060c85f640 1 -- 172.21.15.5:0/1624645560 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06080d14f0 msgr2=0x7f06080cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:30.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.741+0000 7f060c85f640 1 --2- 172.21.15.5:0/1624645560 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06080d14f0 0x7f06080cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:30.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.741+0000 7f060c85f640 1 -- 172.21.15.5:0/1624645560 >> v2:172.21.15.63:3300/0 conn(0x7f06080cd4d0 msgr2=0x7f06080cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:30.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.741+0000 7f060c85f640 1 --2- 172.21.15.5:0/1624645560 >> v2:172.21.15.63:3300/0 conn(0x7f06080cd4d0 0x7f06080cd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:30.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.741+0000 7f060c85f640 1 -- 172.21.15.5:0/1624645560 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06080d2360 con 0x7f06080d10e0 2024-05-20T18:27:30.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.742+0000 7f060c85f640 1 --2- 172.21.15.5:0/1624645560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06080d10e0 0x7f06080cf140 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f06080b8850 tx=0x7f05f802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c87834d9f5ce3323 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:30.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.742+0000 7f06077fe640 1 -- 172.21.15.5:0/1624645560 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f8037070 con 0x7f06080d10e0 2024-05-20T18:27:30.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.742+0000 7f06077fe640 1 -- 172.21.15.5:0/1624645560 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05f8032440 con 0x7f06080d10e0 2024-05-20T18:27:30.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.742+0000 7f06077fe640 1 -- 172.21.15.5:0/1624645560 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f8032810 con 0x7f06080d10e0 2024-05-20T18:27:30.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.743+0000 7f060eac3640 1 -- 172.21.15.5:0/1624645560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06080d10e0 msgr2=0x7f06080cf140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:30.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.743+0000 7f060eac3640 1 --2- 172.21.15.5:0/1624645560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06080d10e0 0x7f06080cf140 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f06080b8850 tx=0x7f05f802a1e0 comp rx=0 tx=0).stop 2024-05-20T18:27:30.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.743+0000 7f060eac3640 1 -- 172.21.15.5:0/1624645560 shutdown_connections 2024-05-20T18:27:30.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.743+0000 7f060eac3640 1 --2- 172.21.15.5:0/1624645560 >> v2:172.21.15.63:3300/0 conn(0x7f06080cd4d0 0x7f06080cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:30.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.743+0000 7f060eac3640 1 --2- 172.21.15.5:0/1624645560 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06080d14f0 0x7f06080cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:30.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.743+0000 7f060eac3640 1 --2- 172.21.15.5:0/1624645560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06080d10e0 0x7f06080cf140 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:30.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.743+0000 7f060eac3640 1 -- 172.21.15.5:0/1624645560 >> 172.21.15.5:0/1624645560 conn(0x7f06080b44d0 msgr2=0x7f06080b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:30.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.743+0000 7f060eac3640 1 -- 172.21.15.5:0/1624645560 shutdown_connections 2024-05-20T18:27:30.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.743+0000 7f060eac3640 1 -- 172.21.15.5:0/1624645560 wait complete. 2024-05-20T18:27:30.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.744+0000 7f060eac3640 1 Processor -- start 2024-05-20T18:27:30.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.745+0000 7f060eac3640 1 -- start start 2024-05-20T18:27:30.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.746+0000 7f060eac3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06080cd4d0 0x7f0608151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:30.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.746+0000 7f060eac3640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06080d10e0 0x7f060806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:30.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.746+0000 7f060eac3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06080d14f0 0x7f060815edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:30.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.746+0000 7f060eac3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f06080b8440 con 0x7f06080cd4d0 2024-05-20T18:27:30.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.746+0000 7f060eac3640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f06080b82c0 con 0x7f06080d10e0 2024-05-20T18:27:30.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.746+0000 7f060eac3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f06080709f0 con 0x7f06080d14f0 2024-05-20T18:27:30.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.746+0000 7f060c85f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06080cd4d0 0x7f0608151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:30.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.746+0000 7f060c85f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06080cd4d0 0x7f0608151ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59000/0 (socket says 172.21.15.5:59000) 2024-05-20T18:27:30.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.746+0000 7f060c85f640 1 -- 172.21.15.5:0/724299244 learned_addr learned my addr 172.21.15.5:0/724299244 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:30.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.746+0000 7f060d060640 1 --2- 172.21.15.5:0/724299244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06080d14f0 0x7f060815edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:30.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.746+0000 7f0607fff640 1 --2- 172.21.15.5:0/724299244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06080d10e0 0x7f060806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:30.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.746+0000 7f060c85f640 1 -- 172.21.15.5:0/724299244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06080d14f0 msgr2=0x7f060815edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:30.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.746+0000 7f060c85f640 1 --2- 172.21.15.5:0/724299244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06080d14f0 0x7f060815edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:30.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.746+0000 7f060c85f640 1 -- 172.21.15.5:0/724299244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06080d10e0 msgr2=0x7f060806da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:30.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.746+0000 7f060c85f640 1 --2- 172.21.15.5:0/724299244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06080d10e0 0x7f060806da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:30.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.746+0000 7f060c85f640 1 -- 172.21.15.5:0/724299244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06081f6800 con 0x7f06080cd4d0 2024-05-20T18:27:30.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.747+0000 7f060c85f640 1 --2- 172.21.15.5:0/724299244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06080cd4d0 0x7f0608151ea0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f05f8000c00 tx=0x7f05f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:30.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.747+0000 7f06057fa640 1 -- 172.21.15.5:0/724299244 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f8041020 con 0x7f06080cd4d0 2024-05-20T18:27:30.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.747+0000 7f06057fa640 1 -- 172.21.15.5:0/724299244 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05f803d620 con 0x7f06080cd4d0 2024-05-20T18:27:30.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.747+0000 7f060eac3640 1 -- 172.21.15.5:0/724299244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06080c9d20 con 0x7f06080cd4d0 2024-05-20T18:27:30.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.747+0000 7f06057fa640 1 -- 172.21.15.5:0/724299244 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f8037040 con 0x7f06080cd4d0 2024-05-20T18:27:30.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.747+0000 7f060eac3640 1 -- 172.21.15.5:0/724299244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06080ca2e0 con 0x7f06080cd4d0 2024-05-20T18:27:30.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.748+0000 7f060eac3640 1 -- 172.21.15.5:0/724299244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0608075b00 con 0x7f06080cd4d0 2024-05-20T18:27:30.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.749+0000 7f06057fa640 1 -- 172.21.15.5:0/724299244 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f05f8044020 con 0x7f06080cd4d0 2024-05-20T18:27:30.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.749+0000 7f06057fa640 1 --2- 172.21.15.5:0/724299244 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f05e806be60 0x7f05e806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:30.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.749+0000 7f06057fa640 1 -- 172.21.15.5:0/724299244 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f05f80ab910 con 0x7f06080cd4d0 2024-05-20T18:27:30.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.749+0000 7f0607fff640 1 --2- 172.21.15.5:0/724299244 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f05e806be60 0x7f05e806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:30.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.750+0000 7f0607fff640 1 --2- 172.21.15.5:0/724299244 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f05e806be60 0x7f05e806e370 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f06080c9380 tx=0x7f05f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:30.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.754+0000 7f06057fa640 1 -- 172.21.15.5:0/724299244 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f05f804b030 con 0x7f06080cd4d0 2024-05-20T18:27:30.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.936+0000 7f060eac3640 1 -- 172.21.15.5:0/724299244 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f060815d2e0 con 0x7f05e806be60 2024-05-20T18:27:30.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.945+0000 7f06057fa640 1 -- 172.21.15.5:0/724299244 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332558 (secure 0 0 0) 0x7f060815d2e0 con 0x7f05e806be60 2024-05-20T18:27:30.948 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-20T18:27:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.949+0000 7f060eac3640 1 -- 172.21.15.5:0/724299244 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f05e806be60 msgr2=0x7f05e806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.949+0000 7f060eac3640 1 --2- 172.21.15.5:0/724299244 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f05e806be60 0x7f05e806e370 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f06080c9380 tx=0x7f05f0008040 comp rx=0 tx=0).stop 2024-05-20T18:27:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.949+0000 7f060eac3640 1 -- 172.21.15.5:0/724299244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06080cd4d0 msgr2=0x7f0608151ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.949+0000 7f060eac3640 1 --2- 172.21.15.5:0/724299244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06080cd4d0 0x7f0608151ea0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f05f8000c00 tx=0x7f05f80046c0 comp rx=0 tx=0).stop 2024-05-20T18:27:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.949+0000 7f060eac3640 1 -- 172.21.15.5:0/724299244 shutdown_connections 2024-05-20T18:27:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.949+0000 7f060eac3640 1 --2- 172.21.15.5:0/724299244 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f05e806be60 0x7f05e806e370 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:30.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.949+0000 7f060eac3640 1 --2- 172.21.15.5:0/724299244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06080d14f0 0x7f060815edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:30.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.949+0000 7f060eac3640 1 --2- 172.21.15.5:0/724299244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06080d10e0 0x7f060806da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:30.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.949+0000 7f060eac3640 1 --2- 172.21.15.5:0/724299244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06080cd4d0 0x7f0608151ea0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:30.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.949+0000 7f060eac3640 1 -- 172.21.15.5:0/724299244 >> 172.21.15.5:0/724299244 conn(0x7f06080b44d0 msgr2=0x7f06080b2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:30.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.949+0000 7f060eac3640 1 -- 172.21.15.5:0/724299244 shutdown_connections 2024-05-20T18:27:30.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:30.949+0000 7f060eac3640 1 -- 172.21.15.5:0/724299244 wait complete. 2024-05-20T18:27:30.960 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-05-20T18:27:30.578435+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236216,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738504,"statfs":{"total":773094113280,"available":772852228096,"internally_reserved":0,"allocated":3481600,"data_stored":2086528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003039"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048716+0000","last_change":"2024-05-20T18:27:06.794819+0000","last_active":"2024-05-20T18:27:14.048716+0000","last_peered":"2024-05-20T18:27:14.048716+0000","last_clean":"2024-05-20T18:27:14.048716+0000","last_became_active":"2024-05-20T18:27:06.794505+0000","last_became_peered":"2024-05-20T18:27:06.794505+0000","last_unstale":"2024-05-20T18:27:14.048716+0000","last_undegraded":"2024-05-20T18:27:14.048716+0000","last_fullsized":"2024-05-20T18:27:14.048716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:20:52.095853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821927+0000","last_change":"2024-05-20T18:27:07.802838+0000","last_active":"2024-05-20T18:27:13.821927+0000","last_peered":"2024-05-20T18:27:13.821927+0000","last_clean":"2024-05-20T18:27:13.821927+0000","last_became_active":"2024-05-20T18:27:07.802468+0000","last_became_peered":"2024-05-20T18:27:07.802468+0000","last_unstale":"2024-05-20T18:27:13.821927+0000","last_undegraded":"2024-05-20T18:27:13.821927+0000","last_fullsized":"2024-05-20T18:27:13.821927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:12.006827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821103+0000","last_change":"2024-05-20T18:27:12.469613+0000","last_active":"2024-05-20T18:27:13.821103+0000","last_peered":"2024-05-20T18:27:13.821103+0000","last_clean":"2024-05-20T18:27:13.821103+0000","last_became_active":"2024-05-20T18:27:12.469161+0000","last_became_peered":"2024-05-20T18:27:12.469161+0000","last_unstale":"2024-05-20T18:27:13.821103+0000","last_undegraded":"2024-05-20T18:27:13.821103+0000","last_fullsized":"2024-05-20T18:27:13.821103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:32:50.234917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.822083+0000","last_change":"2024-05-20T18:27:06.799063+0000","last_active":"2024-05-20T18:27:13.822083+0000","last_peered":"2024-05-20T18:27:13.822083+0000","last_clean":"2024-05-20T18:27:13.822083+0000","last_became_active":"2024-05-20T18:27:06.798689+0000","last_became_peered":"2024-05-20T18:27:06.798689+0000","last_unstale":"2024-05-20T18:27:13.822083+0000","last_undegraded":"2024-05-20T18:27:13.822083+0000","last_fullsized":"2024-05-20T18:27:13.822083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:52:44.943426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048397+0000","last_change":"2024-05-20T18:27:07.805002+0000","last_active":"2024-05-20T18:27:14.048397+0000","last_peered":"2024-05-20T18:27:14.048397+0000","last_clean":"2024-05-20T18:27:14.048397+0000","last_became_active":"2024-05-20T18:27:07.804655+0000","last_became_peered":"2024-05-20T18:27:07.804655+0000","last_unstale":"2024-05-20T18:27:14.048397+0000","last_undegraded":"2024-05-20T18:27:14.048397+0000","last_fullsized":"2024-05-20T18:27:14.048397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:52:57.891709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047788+0000","last_change":"2024-05-20T18:27:11.833803+0000","last_active":"2024-05-20T18:27:14.047788+0000","last_peered":"2024-05-20T18:27:14.047788+0000","last_clean":"2024-05-20T18:27:14.047788+0000","last_became_active":"2024-05-20T18:27:11.833241+0000","last_became_peered":"2024-05-20T18:27:11.833241+0000","last_unstale":"2024-05-20T18:27:14.047788+0000","last_undegraded":"2024-05-20T18:27:14.047788+0000","last_fullsized":"2024-05-20T18:27:14.047788+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:25:03.246597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049511+0000","last_change":"2024-05-20T18:27:06.806008+0000","last_active":"2024-05-20T18:27:14.049511+0000","last_peered":"2024-05-20T18:27:14.049511+0000","last_clean":"2024-05-20T18:27:14.049511+0000","last_became_active":"2024-05-20T18:27:06.805632+0000","last_became_peered":"2024-05-20T18:27:06.805632+0000","last_unstale":"2024-05-20T18:27:14.049511+0000","last_undegraded":"2024-05-20T18:27:14.049511+0000","last_fullsized":"2024-05-20T18:27:14.049511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:11:07.831277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825124+0000","last_change":"2024-05-20T18:27:07.812390+0000","last_active":"2024-05-20T18:27:13.825124+0000","last_peered":"2024-05-20T18:27:13.825124+0000","last_clean":"2024-05-20T18:27:13.825124+0000","last_became_active":"2024-05-20T18:27:07.811667+0000","last_became_peered":"2024-05-20T18:27:07.811667+0000","last_unstale":"2024-05-20T18:27:13.825124+0000","last_undegraded":"2024-05-20T18:27:13.825124+0000","last_fullsized":"2024-05-20T18:27:13.825124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:29:33.478403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048201+0000","last_change":"2024-05-20T18:27:11.833494+0000","last_active":"2024-05-20T18:27:14.048201+0000","last_peered":"2024-05-20T18:27:14.048201+0000","last_clean":"2024-05-20T18:27:14.048201+0000","last_became_active":"2024-05-20T18:27:11.832999+0000","last_became_peered":"2024-05-20T18:27:11.832999+0000","last_unstale":"2024-05-20T18:27:14.048201+0000","last_undegraded":"2024-05-20T18:27:14.048201+0000","last_fullsized":"2024-05-20T18:27:14.048201+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:08:42.465927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049057+0000","last_change":"2024-05-20T18:27:06.798023+0000","last_active":"2024-05-20T18:27:14.049057+0000","last_peered":"2024-05-20T18:27:14.049057+0000","last_clean":"2024-05-20T18:27:14.049057+0000","last_became_active":"2024-05-20T18:27:06.797674+0000","last_became_peered":"2024-05-20T18:27:06.797674+0000","last_unstale":"2024-05-20T18:27:14.049057+0000","last_undegraded":"2024-05-20T18:27:14.049057+0000","last_fullsized":"2024-05-20T18:27:14.049057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:43:31.154358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824776+0000","last_change":"2024-05-20T18:27:07.811508+0000","last_active":"2024-05-20T18:27:13.824776+0000","last_peered":"2024-05-20T18:27:13.824776+0000","last_clean":"2024-05-20T18:27:13.824776+0000","last_became_active":"2024-05-20T18:27:07.811217+0000","last_became_peered":"2024-05-20T18:27:07.811217+0000","last_unstale":"2024-05-20T18:27:13.824776+0000","last_undegraded":"2024-05-20T18:27:13.824776+0000","last_fullsized":"2024-05-20T18:27:13.824776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:01:46.623171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047874+0000","last_change":"2024-05-20T18:27:11.827280+0000","last_active":"2024-05-20T18:27:14.047874+0000","last_peered":"2024-05-20T18:27:14.047874+0000","last_clean":"2024-05-20T18:27:14.047874+0000","last_became_active":"2024-05-20T18:27:11.826770+0000","last_became_peered":"2024-05-20T18:27:11.826770+0000","last_unstale":"2024-05-20T18:27:14.047874+0000","last_undegraded":"2024-05-20T18:27:14.047874+0000","last_fullsized":"2024-05-20T18:27:14.047874+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:16:41.968388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825487+0000","last_change":"2024-05-20T18:27:07.791795+0000","last_active":"2024-05-20T18:27:13.825487+0000","last_peered":"2024-05-20T18:27:13.825487+0000","last_clean":"2024-05-20T18:27:13.825487+0000","last_became_active":"2024-05-20T18:27:07.791422+0000","last_became_peered":"2024-05-20T18:27:07.791422+0000","last_unstale":"2024-05-20T18:27:13.825487+0000","last_undegraded":"2024-05-20T18:27:13.825487+0000","last_fullsized":"2024-05-20T18:27:13.825487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:40:08.698691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049335+0000","last_change":"2024-05-20T18:27:06.799475+0000","last_active":"2024-05-20T18:27:14.049335+0000","last_peered":"2024-05-20T18:27:14.049335+0000","last_clean":"2024-05-20T18:27:14.049335+0000","last_became_active":"2024-05-20T18:27:06.798540+0000","last_became_peered":"2024-05-20T18:27:06.798540+0000","last_unstale":"2024-05-20T18:27:14.049335+0000","last_undegraded":"2024-05-20T18:27:14.049335+0000","last_fullsized":"2024-05-20T18:27:14.049335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:14:22.040824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048392+0000","last_change":"2024-05-20T18:27:11.826856+0000","last_active":"2024-05-20T18:27:14.048392+0000","last_peered":"2024-05-20T18:27:14.048392+0000","last_clean":"2024-05-20T18:27:14.048392+0000","last_became_active":"2024-05-20T18:27:11.826205+0000","last_became_peered":"2024-05-20T18:27:11.826205+0000","last_unstale":"2024-05-20T18:27:14.048392+0000","last_undegraded":"2024-05-20T18:27:14.048392+0000","last_fullsized":"2024-05-20T18:27:14.048392+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:34:21.648495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821544+0000","last_change":"2024-05-20T18:27:07.811973+0000","last_active":"2024-05-20T18:27:13.821544+0000","last_peered":"2024-05-20T18:27:13.821544+0000","last_clean":"2024-05-20T18:27:13.821544+0000","last_became_active":"2024-05-20T18:27:07.811617+0000","last_became_peered":"2024-05-20T18:27:07.811617+0000","last_unstale":"2024-05-20T18:27:13.821544+0000","last_undegraded":"2024-05-20T18:27:13.821544+0000","last_fullsized":"2024-05-20T18:27:13.821544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:52:22.903884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048965+0000","last_change":"2024-05-20T18:27:06.792524+0000","last_active":"2024-05-20T18:27:14.048965+0000","last_peered":"2024-05-20T18:27:14.048965+0000","last_clean":"2024-05-20T18:27:14.048965+0000","last_became_active":"2024-05-20T18:27:06.792104+0000","last_became_peered":"2024-05-20T18:27:06.792104+0000","last_unstale":"2024-05-20T18:27:14.048965+0000","last_undegraded":"2024-05-20T18:27:14.048965+0000","last_fullsized":"2024-05-20T18:27:14.048965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:10:27.276400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048408+0000","last_change":"2024-05-20T18:27:11.833928+0000","last_active":"2024-05-20T18:27:14.048408+0000","last_peered":"2024-05-20T18:27:14.048408+0000","last_clean":"2024-05-20T18:27:14.048408+0000","last_became_active":"2024-05-20T18:27:11.832887+0000","last_became_peered":"2024-05-20T18:27:11.832887+0000","last_unstale":"2024-05-20T18:27:14.048408+0000","last_undegraded":"2024-05-20T18:27:14.048408+0000","last_fullsized":"2024-05-20T18:27:14.048408+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:38:58.503399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821356+0000","last_change":"2024-05-20T18:27:07.788711+0000","last_active":"2024-05-20T18:27:13.821356+0000","last_peered":"2024-05-20T18:27:13.821356+0000","last_clean":"2024-05-20T18:27:13.821356+0000","last_became_active":"2024-05-20T18:27:07.788402+0000","last_became_peered":"2024-05-20T18:27:07.788402+0000","last_unstale":"2024-05-20T18:27:13.821356+0000","last_undegraded":"2024-05-20T18:27:13.821356+0000","last_fullsized":"2024-05-20T18:27:13.821356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:34:15.614124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048633+0000","last_change":"2024-05-20T18:27:06.803611+0000","last_active":"2024-05-20T18:27:14.048633+0000","last_peered":"2024-05-20T18:27:14.048633+0000","last_clean":"2024-05-20T18:27:14.048633+0000","last_became_active":"2024-05-20T18:27:06.803176+0000","last_became_peered":"2024-05-20T18:27:06.803176+0000","last_unstale":"2024-05-20T18:27:14.048633+0000","last_undegraded":"2024-05-20T18:27:14.048633+0000","last_fullsized":"2024-05-20T18:27:14.048633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:52:44.943426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049451+0000","last_change":"2024-05-20T18:27:12.466975+0000","last_active":"2024-05-20T18:27:14.049451+0000","last_peered":"2024-05-20T18:27:14.049451+0000","last_clean":"2024-05-20T18:27:14.049451+0000","last_became_active":"2024-05-20T18:27:12.466473+0000","last_became_peered":"2024-05-20T18:27:12.466473+0000","last_unstale":"2024-05-20T18:27:14.049451+0000","last_undegraded":"2024-05-20T18:27:14.049451+0000","last_fullsized":"2024-05-20T18:27:14.049451+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:55:23.896898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048689+0000","last_change":"2024-05-20T18:27:07.786091+0000","last_active":"2024-05-20T18:27:14.048689+0000","last_peered":"2024-05-20T18:27:14.048689+0000","last_clean":"2024-05-20T18:27:14.048689+0000","last_became_active":"2024-05-20T18:27:07.785704+0000","last_became_peered":"2024-05-20T18:27:07.785704+0000","last_unstale":"2024-05-20T18:27:14.048689+0000","last_undegraded":"2024-05-20T18:27:14.048689+0000","last_fullsized":"2024-05-20T18:27:14.048689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:06:32.450870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047619+0000","last_change":"2024-05-20T18:27:06.799668+0000","last_active":"2024-05-20T18:27:14.047619+0000","last_peered":"2024-05-20T18:27:14.047619+0000","last_clean":"2024-05-20T18:27:14.047619+0000","last_became_active":"2024-05-20T18:27:06.799002+0000","last_became_peered":"2024-05-20T18:27:06.799002+0000","last_unstale":"2024-05-20T18:27:14.047619+0000","last_undegraded":"2024-05-20T18:27:14.047619+0000","last_fullsized":"2024-05-20T18:27:14.047619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:06:31.445366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048146+0000","last_change":"2024-05-20T18:27:11.834133+0000","last_active":"2024-05-20T18:27:14.048146+0000","last_peered":"2024-05-20T18:27:14.048146+0000","last_clean":"2024-05-20T18:27:14.048146+0000","last_became_active":"2024-05-20T18:27:11.832891+0000","last_became_peered":"2024-05-20T18:27:11.832891+0000","last_unstale":"2024-05-20T18:27:14.048146+0000","last_undegraded":"2024-05-20T18:27:14.048146+0000","last_fullsized":"2024-05-20T18:27:14.048146+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:55:36.987878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049426+0000","last_change":"2024-05-20T18:27:07.790534+0000","last_active":"2024-05-20T18:27:14.049426+0000","last_peered":"2024-05-20T18:27:14.049426+0000","last_clean":"2024-05-20T18:27:14.049426+0000","last_became_active":"2024-05-20T18:27:07.790192+0000","last_became_peered":"2024-05-20T18:27:07.790192+0000","last_unstale":"2024-05-20T18:27:14.049426+0000","last_undegraded":"2024-05-20T18:27:14.049426+0000","last_fullsized":"2024-05-20T18:27:14.049426+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:12.006827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.826184+0000","last_change":"2024-05-20T18:27:06.794221+0000","last_active":"2024-05-20T18:27:13.826184+0000","last_peered":"2024-05-20T18:27:13.826184+0000","last_clean":"2024-05-20T18:27:13.826184+0000","last_became_active":"2024-05-20T18:27:06.793895+0000","last_became_peered":"2024-05-20T18:27:06.793895+0000","last_unstale":"2024-05-20T18:27:13.826184+0000","last_undegraded":"2024-05-20T18:27:13.826184+0000","last_fullsized":"2024-05-20T18:27:13.826184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:24:55.001924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048453+0000","last_change":"2024-05-20T18:27:12.471660+0000","last_active":"2024-05-20T18:27:14.048453+0000","last_peered":"2024-05-20T18:27:14.048453+0000","last_clean":"2024-05-20T18:27:14.048453+0000","last_became_active":"2024-05-20T18:27:12.470460+0000","last_became_peered":"2024-05-20T18:27:12.470460+0000","last_unstale":"2024-05-20T18:27:14.048453+0000","last_undegraded":"2024-05-20T18:27:14.048453+0000","last_fullsized":"2024-05-20T18:27:14.048453+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:53:56.893940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.822216+0000","last_change":"2024-05-20T18:27:07.799399+0000","last_active":"2024-05-20T18:27:13.822216+0000","last_peered":"2024-05-20T18:27:13.822216+0000","last_clean":"2024-05-20T18:27:13.822216+0000","last_became_active":"2024-05-20T18:27:07.799017+0000","last_became_peered":"2024-05-20T18:27:07.799017+0000","last_unstale":"2024-05-20T18:27:13.822216+0000","last_undegraded":"2024-05-20T18:27:13.822216+0000","last_fullsized":"2024-05-20T18:27:13.822216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:08:51.463251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049547+0000","last_change":"2024-05-20T18:27:06.794477+0000","last_active":"2024-05-20T18:27:14.049547+0000","last_peered":"2024-05-20T18:27:14.049547+0000","last_clean":"2024-05-20T18:27:14.049547+0000","last_became_active":"2024-05-20T18:27:06.794130+0000","last_became_peered":"2024-05-20T18:27:06.794130+0000","last_unstale":"2024-05-20T18:27:14.049547+0000","last_undegraded":"2024-05-20T18:27:14.049547+0000","last_fullsized":"2024-05-20T18:27:14.049547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:43:31.154358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821030+0000","last_change":"2024-05-20T18:27:12.468521+0000","last_active":"2024-05-20T18:27:13.821030+0000","last_peered":"2024-05-20T18:27:13.821030+0000","last_clean":"2024-05-20T18:27:13.821030+0000","last_became_active":"2024-05-20T18:27:12.467880+0000","last_became_peered":"2024-05-20T18:27:12.467880+0000","last_unstale":"2024-05-20T18:27:13.821030+0000","last_undegraded":"2024-05-20T18:27:13.821030+0000","last_fullsized":"2024-05-20T18:27:13.821030+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:48:16.180009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048938+0000","last_change":"2024-05-20T18:27:07.784489+0000","last_active":"2024-05-20T18:27:14.048938+0000","last_peered":"2024-05-20T18:27:14.048938+0000","last_clean":"2024-05-20T18:27:14.048938+0000","last_became_active":"2024-05-20T18:27:07.783968+0000","last_became_peered":"2024-05-20T18:27:07.783968+0000","last_unstale":"2024-05-20T18:27:14.048938+0000","last_undegraded":"2024-05-20T18:27:14.048938+0000","last_fullsized":"2024-05-20T18:27:14.048938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:07:15.764912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049643+0000","last_change":"2024-05-20T18:27:06.805626+0000","last_active":"2024-05-20T18:27:14.049643+0000","last_peered":"2024-05-20T18:27:14.049643+0000","last_clean":"2024-05-20T18:27:14.049643+0000","last_became_active":"2024-05-20T18:27:06.805267+0000","last_became_peered":"2024-05-20T18:27:06.805267+0000","last_unstale":"2024-05-20T18:27:14.049643+0000","last_undegraded":"2024-05-20T18:27:14.049643+0000","last_fullsized":"2024-05-20T18:27:14.049643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:18:18.528418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048076+0000","last_change":"2024-05-20T18:27:12.465690+0000","last_active":"2024-05-20T18:27:14.048076+0000","last_peered":"2024-05-20T18:27:14.048076+0000","last_clean":"2024-05-20T18:27:14.048076+0000","last_became_active":"2024-05-20T18:27:12.465196+0000","last_became_peered":"2024-05-20T18:27:12.465196+0000","last_unstale":"2024-05-20T18:27:14.048076+0000","last_undegraded":"2024-05-20T18:27:14.048076+0000","last_fullsized":"2024-05-20T18:27:14.048076+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:14:31.209452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825227+0000","last_change":"2024-05-20T18:27:07.806673+0000","last_active":"2024-05-20T18:27:13.825227+0000","last_peered":"2024-05-20T18:27:13.825227+0000","last_clean":"2024-05-20T18:27:13.825227+0000","last_became_active":"2024-05-20T18:27:07.805764+0000","last_became_peered":"2024-05-20T18:27:07.805764+0000","last_unstale":"2024-05-20T18:27:13.825227+0000","last_undegraded":"2024-05-20T18:27:13.825227+0000","last_fullsized":"2024-05-20T18:27:13.825227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:16:37.938695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049063+0000","last_change":"2024-05-20T18:27:06.796398+0000","last_active":"2024-05-20T18:27:14.049063+0000","last_peered":"2024-05-20T18:27:14.049063+0000","last_clean":"2024-05-20T18:27:14.049063+0000","last_became_active":"2024-05-20T18:27:06.795981+0000","last_became_peered":"2024-05-20T18:27:06.795981+0000","last_unstale":"2024-05-20T18:27:14.049063+0000","last_undegraded":"2024-05-20T18:27:14.049063+0000","last_fullsized":"2024-05-20T18:27:14.049063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:11:07.831277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048435+0000","last_change":"2024-05-20T18:27:11.830530+0000","last_active":"2024-05-20T18:27:14.048435+0000","last_peered":"2024-05-20T18:27:14.048435+0000","last_clean":"2024-05-20T18:27:14.048435+0000","last_became_active":"2024-05-20T18:27:11.828707+0000","last_became_peered":"2024-05-20T18:27:11.828707+0000","last_unstale":"2024-05-20T18:27:14.048435+0000","last_undegraded":"2024-05-20T18:27:14.048435+0000","last_fullsized":"2024-05-20T18:27:14.048435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:14:31.209452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048613+0000","last_change":"2024-05-20T18:27:07.802587+0000","last_active":"2024-05-20T18:27:14.048613+0000","last_peered":"2024-05-20T18:27:14.048613+0000","last_clean":"2024-05-20T18:27:14.048613+0000","last_became_active":"2024-05-20T18:27:07.802284+0000","last_became_peered":"2024-05-20T18:27:07.802284+0000","last_unstale":"2024-05-20T18:27:14.048613+0000","last_undegraded":"2024-05-20T18:27:14.048613+0000","last_fullsized":"2024-05-20T18:27:14.048613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:18:19.533923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825123+0000","last_change":"2024-05-20T18:27:06.795278+0000","last_active":"2024-05-20T18:27:13.825123+0000","last_peered":"2024-05-20T18:27:13.825123+0000","last_clean":"2024-05-20T18:27:13.825123+0000","last_became_active":"2024-05-20T18:27:06.794895+0000","last_became_peered":"2024-05-20T18:27:06.794895+0000","last_unstale":"2024-05-20T18:27:13.825123+0000","last_undegraded":"2024-05-20T18:27:13.825123+0000","last_fullsized":"2024-05-20T18:27:13.825123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:52:44.943426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048956+0000","last_change":"2024-05-20T18:27:11.833463+0000","last_active":"2024-05-20T18:27:14.048956+0000","last_peered":"2024-05-20T18:27:14.048956+0000","last_clean":"2024-05-20T18:27:14.048956+0000","last_became_active":"2024-05-20T18:27:11.832954+0000","last_became_peered":"2024-05-20T18:27:11.832954+0000","last_unstale":"2024-05-20T18:27:14.048956+0000","last_undegraded":"2024-05-20T18:27:14.048956+0000","last_fullsized":"2024-05-20T18:27:14.048956+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:34:21.648495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048483+0000","last_change":"2024-05-20T18:27:07.785038+0000","last_active":"2024-05-20T18:27:14.048483+0000","last_peered":"2024-05-20T18:27:14.048483+0000","last_clean":"2024-05-20T18:27:14.048483+0000","last_became_active":"2024-05-20T18:27:07.783927+0000","last_became_peered":"2024-05-20T18:27:07.783927+0000","last_unstale":"2024-05-20T18:27:14.048483+0000","last_undegraded":"2024-05-20T18:27:14.048483+0000","last_fullsized":"2024-05-20T18:27:14.048483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:12.006827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047552+0000","last_change":"2024-05-20T18:27:06.789785+0000","last_active":"2024-05-20T18:27:14.047552+0000","last_peered":"2024-05-20T18:27:14.047552+0000","last_clean":"2024-05-20T18:27:14.047552+0000","last_became_active":"2024-05-20T18:27:06.789421+0000","last_became_peered":"2024-05-20T18:27:06.789421+0000","last_unstale":"2024-05-20T18:27:14.047552+0000","last_undegraded":"2024-05-20T18:27:14.047552+0000","last_fullsized":"2024-05-20T18:27:14.047552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:24:55.001924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049327+0000","last_change":"2024-05-20T18:27:11.833707+0000","last_active":"2024-05-20T18:27:14.049327+0000","last_peered":"2024-05-20T18:27:14.049327+0000","last_clean":"2024-05-20T18:27:14.049327+0000","last_became_active":"2024-05-20T18:27:11.832619+0000","last_became_peered":"2024-05-20T18:27:11.832619+0000","last_unstale":"2024-05-20T18:27:14.049327+0000","last_undegraded":"2024-05-20T18:27:14.049327+0000","last_fullsized":"2024-05-20T18:27:14.049327+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:53:56.893940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047552+0000","last_change":"2024-05-20T18:27:07.799471+0000","last_active":"2024-05-20T18:27:14.047552+0000","last_peered":"2024-05-20T18:27:14.047552+0000","last_clean":"2024-05-20T18:27:14.047552+0000","last_became_active":"2024-05-20T18:27:07.799089+0000","last_became_peered":"2024-05-20T18:27:07.799089+0000","last_unstale":"2024-05-20T18:27:14.047552+0000","last_undegraded":"2024-05-20T18:27:14.047552+0000","last_fullsized":"2024-05-20T18:27:14.047552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:39:20.381356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047626+0000","last_change":"2024-05-20T18:27:06.799374+0000","last_active":"2024-05-20T18:27:14.047626+0000","last_peered":"2024-05-20T18:27:14.047626+0000","last_clean":"2024-05-20T18:27:14.047626+0000","last_became_active":"2024-05-20T18:27:06.798552+0000","last_became_peered":"2024-05-20T18:27:06.798552+0000","last_unstale":"2024-05-20T18:27:14.047626+0000","last_undegraded":"2024-05-20T18:27:14.047626+0000","last_fullsized":"2024-05-20T18:27:14.047626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:52:44.943426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049339+0000","last_change":"2024-05-20T18:27:12.469418+0000","last_active":"2024-05-20T18:27:14.049339+0000","last_peered":"2024-05-20T18:27:14.049339+0000","last_clean":"2024-05-20T18:27:14.049339+0000","last_became_active":"2024-05-20T18:27:12.468923+0000","last_became_peered":"2024-05-20T18:27:12.468923+0000","last_unstale":"2024-05-20T18:27:14.049339+0000","last_undegraded":"2024-05-20T18:27:14.049339+0000","last_fullsized":"2024-05-20T18:27:14.049339+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:55:23.896898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821455+0000","last_change":"2024-05-20T18:27:07.804772+0000","last_active":"2024-05-20T18:27:13.821455+0000","last_peered":"2024-05-20T18:27:13.821455+0000","last_clean":"2024-05-20T18:27:13.821455+0000","last_became_active":"2024-05-20T18:27:07.804443+0000","last_became_peered":"2024-05-20T18:27:07.804443+0000","last_unstale":"2024-05-20T18:27:13.821455+0000","last_undegraded":"2024-05-20T18:27:13.821455+0000","last_fullsized":"2024-05-20T18:27:13.821455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:14:17.791789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824615+0000","last_change":"2024-05-20T18:27:06.790984+0000","last_active":"2024-05-20T18:27:13.824615+0000","last_peered":"2024-05-20T18:27:13.824615+0000","last_clean":"2024-05-20T18:27:13.824615+0000","last_became_active":"2024-05-20T18:27:06.790620+0000","last_became_peered":"2024-05-20T18:27:06.790620+0000","last_unstale":"2024-05-20T18:27:13.824615+0000","last_undegraded":"2024-05-20T18:27:13.824615+0000","last_fullsized":"2024-05-20T18:27:13.824615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:10:27.276400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":179,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047791+0000","last_change":"2024-05-20T18:27:03.765694+0000","last_active":"2024-05-20T18:27:14.047791+0000","last_peered":"2024-05-20T18:27:14.047791+0000","last_clean":"2024-05-20T18:27:14.047791+0000","last_became_active":"2024-05-20T18:27:03.764695+0000","last_became_peered":"2024-05-20T18:27:03.764695+0000","last_unstale":"2024-05-20T18:27:14.047791+0000","last_undegraded":"2024-05-20T18:27:14.047791+0000","last_fullsized":"2024-05-20T18:27:14.047791+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_clean_scrub_stamp":"2024-05-20T18:27:02.750853+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:30:04.407487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824624+0000","last_change":"2024-05-20T18:27:12.467983+0000","last_active":"2024-05-20T18:27:13.824624+0000","last_peered":"2024-05-20T18:27:13.824624+0000","last_clean":"2024-05-20T18:27:13.824624+0000","last_became_active":"2024-05-20T18:27:12.467420+0000","last_became_peered":"2024-05-20T18:27:12.467420+0000","last_unstale":"2024-05-20T18:27:13.824624+0000","last_undegraded":"2024-05-20T18:27:13.824624+0000","last_fullsized":"2024-05-20T18:27:13.824624+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:01:50.652864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049235+0000","last_change":"2024-05-20T18:27:07.814589+0000","last_active":"2024-05-20T18:27:14.049235+0000","last_peered":"2024-05-20T18:27:14.049235+0000","last_clean":"2024-05-20T18:27:14.049235+0000","last_became_active":"2024-05-20T18:27:07.814281+0000","last_became_peered":"2024-05-20T18:27:07.814281+0000","last_unstale":"2024-05-20T18:27:14.049235+0000","last_undegraded":"2024-05-20T18:27:14.049235+0000","last_fullsized":"2024-05-20T18:27:14.049235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:29:33.478403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047483+0000","last_change":"2024-05-20T18:27:06.798075+0000","last_active":"2024-05-20T18:27:14.047483+0000","last_peered":"2024-05-20T18:27:14.047483+0000","last_clean":"2024-05-20T18:27:14.047483+0000","last_became_active":"2024-05-20T18:27:06.797696+0000","last_became_peered":"2024-05-20T18:27:06.797696+0000","last_unstale":"2024-05-20T18:27:14.047483+0000","last_undegraded":"2024-05-20T18:27:14.047483+0000","last_fullsized":"2024-05-20T18:27:14.047483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:06:31.445366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825521+0000","last_change":"2024-05-20T18:27:12.465857+0000","last_active":"2024-05-20T18:27:13.825521+0000","last_peered":"2024-05-20T18:27:13.825521+0000","last_clean":"2024-05-20T18:27:13.825521+0000","last_became_active":"2024-05-20T18:27:12.465296+0000","last_became_peered":"2024-05-20T18:27:12.465296+0000","last_unstale":"2024-05-20T18:27:13.825521+0000","last_undegraded":"2024-05-20T18:27:13.825521+0000","last_fullsized":"2024-05-20T18:27:13.825521+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:25:03.246597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825312+0000","last_change":"2024-05-20T18:27:07.808534+0000","last_active":"2024-05-20T18:27:13.825312+0000","last_peered":"2024-05-20T18:27:13.825312+0000","last_clean":"2024-05-20T18:27:13.825312+0000","last_became_active":"2024-05-20T18:27:07.807813+0000","last_became_peered":"2024-05-20T18:27:07.807813+0000","last_unstale":"2024-05-20T18:27:13.825312+0000","last_undegraded":"2024-05-20T18:27:13.825312+0000","last_fullsized":"2024-05-20T18:27:13.825312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:08:51.463251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049093+0000","last_change":"2024-05-20T18:27:06.793443+0000","last_active":"2024-05-20T18:27:14.049093+0000","last_peered":"2024-05-20T18:27:14.049093+0000","last_clean":"2024-05-20T18:27:14.049093+0000","last_became_active":"2024-05-20T18:27:06.793037+0000","last_became_peered":"2024-05-20T18:27:06.793037+0000","last_unstale":"2024-05-20T18:27:14.049093+0000","last_undegraded":"2024-05-20T18:27:14.049093+0000","last_fullsized":"2024-05-20T18:27:14.049093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:06:31.445366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048364+0000","last_change":"2024-05-20T18:27:11.828038+0000","last_active":"2024-05-20T18:27:14.048364+0000","last_peered":"2024-05-20T18:27:14.048364+0000","last_clean":"2024-05-20T18:27:14.048364+0000","last_became_active":"2024-05-20T18:27:11.827482+0000","last_became_peered":"2024-05-20T18:27:11.827482+0000","last_unstale":"2024-05-20T18:27:14.048364+0000","last_undegraded":"2024-05-20T18:27:14.048364+0000","last_fullsized":"2024-05-20T18:27:14.048364+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:38:58.503399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049301+0000","last_change":"2024-05-20T18:27:07.809779+0000","last_active":"2024-05-20T18:27:14.049301+0000","last_peered":"2024-05-20T18:27:14.049301+0000","last_clean":"2024-05-20T18:27:14.049301+0000","last_became_active":"2024-05-20T18:27:07.809458+0000","last_became_peered":"2024-05-20T18:27:07.809458+0000","last_unstale":"2024-05-20T18:27:14.049301+0000","last_undegraded":"2024-05-20T18:27:14.049301+0000","last_fullsized":"2024-05-20T18:27:14.049301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:55:32.958186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047310+0000","last_change":"2024-05-20T18:27:06.788828+0000","last_active":"2024-05-20T18:27:14.047310+0000","last_peered":"2024-05-20T18:27:14.047310+0000","last_clean":"2024-05-20T18:27:14.047310+0000","last_became_active":"2024-05-20T18:27:06.788142+0000","last_became_peered":"2024-05-20T18:27:06.788142+0000","last_unstale":"2024-05-20T18:27:14.047310+0000","last_undegraded":"2024-05-20T18:27:14.047310+0000","last_fullsized":"2024-05-20T18:27:14.047310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:52:44.943426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.823921+0000","last_change":"2024-05-20T18:27:12.466818+0000","last_active":"2024-05-20T18:27:13.823921+0000","last_peered":"2024-05-20T18:27:13.823921+0000","last_clean":"2024-05-20T18:27:13.823921+0000","last_became_active":"2024-05-20T18:27:12.466228+0000","last_became_peered":"2024-05-20T18:27:12.466228+0000","last_unstale":"2024-05-20T18:27:13.823921+0000","last_undegraded":"2024-05-20T18:27:13.823921+0000","last_fullsized":"2024-05-20T18:27:13.823921+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:55:23.896898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049485+0000","last_change":"2024-05-20T18:27:07.808881+0000","last_active":"2024-05-20T18:27:14.049485+0000","last_peered":"2024-05-20T18:27:14.049485+0000","last_clean":"2024-05-20T18:27:14.049485+0000","last_became_active":"2024-05-20T18:27:07.808550+0000","last_became_peered":"2024-05-20T18:27:07.808550+0000","last_unstale":"2024-05-20T18:27:14.049485+0000","last_undegraded":"2024-05-20T18:27:14.049485+0000","last_fullsized":"2024-05-20T18:27:14.049485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:01:46.623171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048816+0000","last_change":"2024-05-20T18:27:06.797126+0000","last_active":"2024-05-20T18:27:14.048816+0000","last_peered":"2024-05-20T18:27:14.048816+0000","last_clean":"2024-05-20T18:27:14.048816+0000","last_became_active":"2024-05-20T18:27:06.796764+0000","last_became_peered":"2024-05-20T18:27:06.796764+0000","last_unstale":"2024-05-20T18:27:14.048816+0000","last_undegraded":"2024-05-20T18:27:14.048816+0000","last_fullsized":"2024-05-20T18:27:14.048816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:59:18.905646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047675+0000","last_change":"2024-05-20T18:27:11.827350+0000","last_active":"2024-05-20T18:27:14.047675+0000","last_peered":"2024-05-20T18:27:14.047675+0000","last_clean":"2024-05-20T18:27:14.047675+0000","last_became_active":"2024-05-20T18:27:11.826656+0000","last_became_peered":"2024-05-20T18:27:11.826656+0000","last_unstale":"2024-05-20T18:27:14.047675+0000","last_undegraded":"2024-05-20T18:27:14.047675+0000","last_fullsized":"2024-05-20T18:27:14.047675+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:01:50.652864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048684+0000","last_change":"2024-05-20T18:27:07.809732+0000","last_active":"2024-05-20T18:27:14.048684+0000","last_peered":"2024-05-20T18:27:14.048684+0000","last_clean":"2024-05-20T18:27:14.048684+0000","last_became_active":"2024-05-20T18:27:07.809412+0000","last_became_peered":"2024-05-20T18:27:07.809412+0000","last_unstale":"2024-05-20T18:27:14.048684+0000","last_undegraded":"2024-05-20T18:27:14.048684+0000","last_fullsized":"2024-05-20T18:27:14.048684+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:14:03.712607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.046974+0000","last_change":"2024-05-20T18:27:06.787105+0000","last_active":"2024-05-20T18:27:14.046974+0000","last_peered":"2024-05-20T18:27:14.046974+0000","last_clean":"2024-05-20T18:27:14.046974+0000","last_became_active":"2024-05-20T18:27:06.786759+0000","last_became_peered":"2024-05-20T18:27:06.786759+0000","last_unstale":"2024-05-20T18:27:14.046974+0000","last_undegraded":"2024-05-20T18:27:14.046974+0000","last_fullsized":"2024-05-20T18:27:14.046974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:10:27.276400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049140+0000","last_change":"2024-05-20T18:27:11.834554+0000","last_active":"2024-05-20T18:27:14.049140+0000","last_peered":"2024-05-20T18:27:14.049140+0000","last_clean":"2024-05-20T18:27:14.049140+0000","last_became_active":"2024-05-20T18:27:11.833955+0000","last_became_peered":"2024-05-20T18:27:11.833955+0000","last_unstale":"2024-05-20T18:27:14.049140+0000","last_undegraded":"2024-05-20T18:27:14.049140+0000","last_fullsized":"2024-05-20T18:27:14.049140+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:53:56.893940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825845+0000","last_change":"2024-05-20T18:27:07.814023+0000","last_active":"2024-05-20T18:27:13.825845+0000","last_peered":"2024-05-20T18:27:13.825845+0000","last_clean":"2024-05-20T18:27:13.825845+0000","last_became_active":"2024-05-20T18:27:07.813679+0000","last_became_peered":"2024-05-20T18:27:07.813679+0000","last_unstale":"2024-05-20T18:27:13.825845+0000","last_undegraded":"2024-05-20T18:27:13.825845+0000","last_fullsized":"2024-05-20T18:27:13.825845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:10:20.947601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825319+0000","last_change":"2024-05-20T18:27:06.787006+0000","last_active":"2024-05-20T18:27:13.825319+0000","last_peered":"2024-05-20T18:27:13.825319+0000","last_clean":"2024-05-20T18:27:13.825319+0000","last_became_active":"2024-05-20T18:27:06.786565+0000","last_became_peered":"2024-05-20T18:27:06.786565+0000","last_unstale":"2024-05-20T18:27:13.825319+0000","last_undegraded":"2024-05-20T18:27:13.825319+0000","last_fullsized":"2024-05-20T18:27:13.825319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:10:27.276400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.822189+0000","last_change":"2024-05-20T18:27:12.468307+0000","last_active":"2024-05-20T18:27:13.822189+0000","last_peered":"2024-05-20T18:27:13.822189+0000","last_clean":"2024-05-20T18:27:13.822189+0000","last_became_active":"2024-05-20T18:27:12.467856+0000","last_became_peered":"2024-05-20T18:27:12.467856+0000","last_unstale":"2024-05-20T18:27:13.822189+0000","last_undegraded":"2024-05-20T18:27:13.822189+0000","last_fullsized":"2024-05-20T18:27:13.822189+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:08:26.960099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049060+0000","last_change":"2024-05-20T18:27:07.809032+0000","last_active":"2024-05-20T18:27:14.049060+0000","last_peered":"2024-05-20T18:27:14.049060+0000","last_clean":"2024-05-20T18:27:14.049060+0000","last_became_active":"2024-05-20T18:27:07.808708+0000","last_became_peered":"2024-05-20T18:27:07.808708+0000","last_unstale":"2024-05-20T18:27:14.049060+0000","last_undegraded":"2024-05-20T18:27:14.049060+0000","last_fullsized":"2024-05-20T18:27:14.049060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:08:51.463251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825345+0000","last_change":"2024-05-20T18:27:06.794995+0000","last_active":"2024-05-20T18:27:13.825345+0000","last_peered":"2024-05-20T18:27:13.825345+0000","last_clean":"2024-05-20T18:27:13.825345+0000","last_became_active":"2024-05-20T18:27:06.794662+0000","last_became_peered":"2024-05-20T18:27:06.794662+0000","last_unstale":"2024-05-20T18:27:13.825345+0000","last_undegraded":"2024-05-20T18:27:13.825345+0000","last_fullsized":"2024-05-20T18:27:13.825345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:06:31.445366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821770+0000","last_change":"2024-05-20T18:27:12.465482+0000","last_active":"2024-05-20T18:27:13.821770+0000","last_peered":"2024-05-20T18:27:13.821770+0000","last_clean":"2024-05-20T18:27:13.821770+0000","last_became_active":"2024-05-20T18:27:12.464905+0000","last_became_peered":"2024-05-20T18:27:12.464905+0000","last_unstale":"2024-05-20T18:27:13.821770+0000","last_undegraded":"2024-05-20T18:27:13.821770+0000","last_fullsized":"2024-05-20T18:27:13.821770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:16:04.416155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048114+0000","last_change":"2024-05-20T18:27:07.807096+0000","last_active":"2024-05-20T18:27:14.048114+0000","last_peered":"2024-05-20T18:27:14.048114+0000","last_clean":"2024-05-20T18:27:14.048114+0000","last_became_active":"2024-05-20T18:27:07.806625+0000","last_became_peered":"2024-05-20T18:27:07.806625+0000","last_unstale":"2024-05-20T18:27:14.048114+0000","last_undegraded":"2024-05-20T18:27:14.048114+0000","last_fullsized":"2024-05-20T18:27:14.048114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:15:10.022458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048546+0000","last_change":"2024-05-20T18:27:06.785919+0000","last_active":"2024-05-20T18:27:14.048546+0000","last_peered":"2024-05-20T18:27:14.048546+0000","last_clean":"2024-05-20T18:27:14.048546+0000","last_became_active":"2024-05-20T18:27:06.785587+0000","last_became_peered":"2024-05-20T18:27:06.785587+0000","last_unstale":"2024-05-20T18:27:14.048546+0000","last_undegraded":"2024-05-20T18:27:14.048546+0000","last_fullsized":"2024-05-20T18:27:14.048546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:10:44.980209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824622+0000","last_change":"2024-05-20T18:27:12.466586+0000","last_active":"2024-05-20T18:27:13.824622+0000","last_peered":"2024-05-20T18:27:13.824622+0000","last_clean":"2024-05-20T18:27:13.824622+0000","last_became_active":"2024-05-20T18:27:12.466076+0000","last_became_peered":"2024-05-20T18:27:12.466076+0000","last_unstale":"2024-05-20T18:27:13.824622+0000","last_undegraded":"2024-05-20T18:27:13.824622+0000","last_fullsized":"2024-05-20T18:27:13.824622+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:18:23.718914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049203+0000","last_change":"2024-05-20T18:27:07.805541+0000","last_active":"2024-05-20T18:27:14.049203+0000","last_peered":"2024-05-20T18:27:14.049203+0000","last_clean":"2024-05-20T18:27:14.049203+0000","last_became_active":"2024-05-20T18:27:07.805223+0000","last_became_peered":"2024-05-20T18:27:07.805223+0000","last_unstale":"2024-05-20T18:27:14.049203+0000","last_undegraded":"2024-05-20T18:27:14.049203+0000","last_fullsized":"2024-05-20T18:27:14.049203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:52:22.903884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":33,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:17.511072+0000","last_change":"2024-05-20T18:27:06.788980+0000","last_active":"2024-05-20T18:27:17.511072+0000","last_peered":"2024-05-20T18:27:17.511072+0000","last_clean":"2024-05-20T18:27:17.511072+0000","last_became_active":"2024-05-20T18:27:06.788477+0000","last_became_peered":"2024-05-20T18:27:06.788477+0000","last_unstale":"2024-05-20T18:27:17.511072+0000","last_undegraded":"2024-05-20T18:27:17.511072+0000","last_fullsized":"2024-05-20T18:27:17.511072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:43:31.154358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048876+0000","last_change":"2024-05-20T18:27:11.829159+0000","last_active":"2024-05-20T18:27:14.048876+0000","last_peered":"2024-05-20T18:27:14.048876+0000","last_clean":"2024-05-20T18:27:14.048876+0000","last_became_active":"2024-05-20T18:27:11.826918+0000","last_became_peered":"2024-05-20T18:27:11.826918+0000","last_unstale":"2024-05-20T18:27:14.048876+0000","last_undegraded":"2024-05-20T18:27:14.048876+0000","last_fullsized":"2024-05-20T18:27:14.048876+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:38:58.503399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048691+0000","last_change":"2024-05-20T18:27:07.791122+0000","last_active":"2024-05-20T18:27:14.048691+0000","last_peered":"2024-05-20T18:27:14.048691+0000","last_clean":"2024-05-20T18:27:14.048691+0000","last_became_active":"2024-05-20T18:27:07.790777+0000","last_became_peered":"2024-05-20T18:27:07.790777+0000","last_unstale":"2024-05-20T18:27:14.048691+0000","last_undegraded":"2024-05-20T18:27:14.048691+0000","last_fullsized":"2024-05-20T18:27:14.048691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:11:08.836782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048983+0000","last_change":"2024-05-20T18:27:06.799123+0000","last_active":"2024-05-20T18:27:14.048983+0000","last_peered":"2024-05-20T18:27:14.048983+0000","last_clean":"2024-05-20T18:27:14.048983+0000","last_became_active":"2024-05-20T18:27:06.798681+0000","last_became_peered":"2024-05-20T18:27:06.798681+0000","last_unstale":"2024-05-20T18:27:14.048983+0000","last_undegraded":"2024-05-20T18:27:14.048983+0000","last_fullsized":"2024-05-20T18:27:14.048983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:18:18.528418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048322+0000","last_change":"2024-05-20T18:27:11.823730+0000","last_active":"2024-05-20T18:27:14.048322+0000","last_peered":"2024-05-20T18:27:14.048322+0000","last_clean":"2024-05-20T18:27:14.048322+0000","last_became_active":"2024-05-20T18:27:11.823226+0000","last_became_peered":"2024-05-20T18:27:11.823226+0000","last_unstale":"2024-05-20T18:27:14.048322+0000","last_undegraded":"2024-05-20T18:27:14.048322+0000","last_fullsized":"2024-05-20T18:27:14.048322+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:14:31.209452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048598+0000","last_change":"2024-05-20T18:27:07.804871+0000","last_active":"2024-05-20T18:27:14.048598+0000","last_peered":"2024-05-20T18:27:14.048598+0000","last_clean":"2024-05-20T18:27:14.048598+0000","last_became_active":"2024-05-20T18:27:07.804518+0000","last_became_peered":"2024-05-20T18:27:07.804518+0000","last_unstale":"2024-05-20T18:27:14.048598+0000","last_undegraded":"2024-05-20T18:27:14.048598+0000","last_fullsized":"2024-05-20T18:27:14.048598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:14:03.712607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824274+0000","last_change":"2024-05-20T18:27:06.795936+0000","last_active":"2024-05-20T18:27:13.824274+0000","last_peered":"2024-05-20T18:27:13.824274+0000","last_clean":"2024-05-20T18:27:13.824274+0000","last_became_active":"2024-05-20T18:27:06.795616+0000","last_became_peered":"2024-05-20T18:27:06.795616+0000","last_unstale":"2024-05-20T18:27:13.824274+0000","last_undegraded":"2024-05-20T18:27:13.824274+0000","last_fullsized":"2024-05-20T18:27:13.824274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:24:55.001924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821242+0000","last_change":"2024-05-20T18:27:11.827063+0000","last_active":"2024-05-20T18:27:13.821242+0000","last_peered":"2024-05-20T18:27:13.821242+0000","last_clean":"2024-05-20T18:27:13.821242+0000","last_became_active":"2024-05-20T18:27:11.825223+0000","last_became_peered":"2024-05-20T18:27:11.825223+0000","last_unstale":"2024-05-20T18:27:13.821242+0000","last_undegraded":"2024-05-20T18:27:13.821242+0000","last_fullsized":"2024-05-20T18:27:13.821242+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:01:18.112337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824776+0000","last_change":"2024-05-20T18:27:07.810225+0000","last_active":"2024-05-20T18:27:13.824776+0000","last_peered":"2024-05-20T18:27:13.824776+0000","last_clean":"2024-05-20T18:27:13.824776+0000","last_became_active":"2024-05-20T18:27:07.809511+0000","last_became_peered":"2024-05-20T18:27:07.809511+0000","last_unstale":"2024-05-20T18:27:13.824776+0000","last_undegraded":"2024-05-20T18:27:13.824776+0000","last_fullsized":"2024-05-20T18:27:13.824776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:24:59.216904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.826020+0000","last_change":"2024-05-20T18:27:06.796334+0000","last_active":"2024-05-20T18:27:13.826020+0000","last_peered":"2024-05-20T18:27:13.826020+0000","last_clean":"2024-05-20T18:27:13.826020+0000","last_became_active":"2024-05-20T18:27:06.795897+0000","last_became_peered":"2024-05-20T18:27:06.795897+0000","last_unstale":"2024-05-20T18:27:13.826020+0000","last_undegraded":"2024-05-20T18:27:13.826020+0000","last_fullsized":"2024-05-20T18:27:13.826020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:06:31.445366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821208+0000","last_change":"2024-05-20T18:27:12.465588+0000","last_active":"2024-05-20T18:27:13.821208+0000","last_peered":"2024-05-20T18:27:13.821208+0000","last_clean":"2024-05-20T18:27:13.821208+0000","last_became_active":"2024-05-20T18:27:12.465170+0000","last_became_peered":"2024-05-20T18:27:12.465170+0000","last_unstale":"2024-05-20T18:27:13.821208+0000","last_undegraded":"2024-05-20T18:27:13.821208+0000","last_fullsized":"2024-05-20T18:27:13.821208+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:36:54.997159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048936+0000","last_change":"2024-05-20T18:27:12.469946+0000","last_active":"2024-05-20T18:27:14.048936+0000","last_peered":"2024-05-20T18:27:14.048936+0000","last_clean":"2024-05-20T18:27:14.048936+0000","last_became_active":"2024-05-20T18:27:12.469526+0000","last_became_peered":"2024-05-20T18:27:12.469526+0000","last_unstale":"2024-05-20T18:27:14.048936+0000","last_undegraded":"2024-05-20T18:27:14.048936+0000","last_fullsized":"2024-05-20T18:27:14.048936+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:34:21.648495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048409+0000","last_change":"2024-05-20T18:27:07.788636+0000","last_active":"2024-05-20T18:27:14.048409+0000","last_peered":"2024-05-20T18:27:14.048409+0000","last_clean":"2024-05-20T18:27:14.048409+0000","last_became_active":"2024-05-20T18:27:07.788307+0000","last_became_peered":"2024-05-20T18:27:07.788307+0000","last_unstale":"2024-05-20T18:27:14.048409+0000","last_undegraded":"2024-05-20T18:27:14.048409+0000","last_fullsized":"2024-05-20T18:27:14.048409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:10:20.947601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824264+0000","last_change":"2024-05-20T18:27:06.797781+0000","last_active":"2024-05-20T18:27:13.824264+0000","last_peered":"2024-05-20T18:27:13.824264+0000","last_clean":"2024-05-20T18:27:13.824264+0000","last_became_active":"2024-05-20T18:27:06.797415+0000","last_became_peered":"2024-05-20T18:27:06.797415+0000","last_unstale":"2024-05-20T18:27:13.824264+0000","last_undegraded":"2024-05-20T18:27:13.824264+0000","last_fullsized":"2024-05-20T18:27:13.824264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:18:18.528418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825975+0000","last_change":"2024-05-20T18:27:11.827561+0000","last_active":"2024-05-20T18:27:13.825975+0000","last_peered":"2024-05-20T18:27:13.825975+0000","last_clean":"2024-05-20T18:27:13.825975+0000","last_became_active":"2024-05-20T18:27:11.826925+0000","last_became_peered":"2024-05-20T18:27:11.826925+0000","last_unstale":"2024-05-20T18:27:13.825975+0000","last_undegraded":"2024-05-20T18:27:13.825975+0000","last_fullsized":"2024-05-20T18:27:13.825975+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:16:41.968388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821278+0000","last_change":"2024-05-20T18:27:07.789976+0000","last_active":"2024-05-20T18:27:13.821278+0000","last_peered":"2024-05-20T18:27:13.821278+0000","last_clean":"2024-05-20T18:27:13.821278+0000","last_became_active":"2024-05-20T18:27:07.789679+0000","last_became_peered":"2024-05-20T18:27:07.789679+0000","last_unstale":"2024-05-20T18:27:13.821278+0000","last_undegraded":"2024-05-20T18:27:13.821278+0000","last_fullsized":"2024-05-20T18:27:13.821278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:52:57.891709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821366+0000","last_change":"2024-05-20T18:27:06.794988+0000","last_active":"2024-05-20T18:27:13.821366+0000","last_peered":"2024-05-20T18:27:13.821366+0000","last_clean":"2024-05-20T18:27:13.821366+0000","last_became_active":"2024-05-20T18:27:06.794630+0000","last_became_peered":"2024-05-20T18:27:06.794630+0000","last_unstale":"2024-05-20T18:27:13.821366+0000","last_undegraded":"2024-05-20T18:27:13.821366+0000","last_fullsized":"2024-05-20T18:27:13.821366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:24:55.001924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825689+0000","last_change":"2024-05-20T18:27:06.785041+0000","last_active":"2024-05-20T18:27:13.825689+0000","last_peered":"2024-05-20T18:27:13.825689+0000","last_clean":"2024-05-20T18:27:13.825689+0000","last_became_active":"2024-05-20T18:27:06.784673+0000","last_became_peered":"2024-05-20T18:27:06.784673+0000","last_unstale":"2024-05-20T18:27:13.825689+0000","last_undegraded":"2024-05-20T18:27:13.825689+0000","last_fullsized":"2024-05-20T18:27:13.825689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:52:44.943426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048466+0000","last_change":"2024-05-20T18:27:07.790855+0000","last_active":"2024-05-20T18:27:14.048466+0000","last_peered":"2024-05-20T18:27:14.048466+0000","last_clean":"2024-05-20T18:27:14.048466+0000","last_became_active":"2024-05-20T18:27:07.790370+0000","last_became_peered":"2024-05-20T18:27:07.790370+0000","last_unstale":"2024-05-20T18:27:14.048466+0000","last_undegraded":"2024-05-20T18:27:14.048466+0000","last_fullsized":"2024-05-20T18:27:14.048466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:07:15.764912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047264+0000","last_change":"2024-05-20T18:27:11.828935+0000","last_active":"2024-05-20T18:27:14.047264+0000","last_peered":"2024-05-20T18:27:14.047264+0000","last_clean":"2024-05-20T18:27:14.047264+0000","last_became_active":"2024-05-20T18:27:11.828516+0000","last_became_peered":"2024-05-20T18:27:11.828516+0000","last_unstale":"2024-05-20T18:27:14.047264+0000","last_undegraded":"2024-05-20T18:27:14.047264+0000","last_fullsized":"2024-05-20T18:27:14.047264+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:15:14.052151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821617+0000","last_change":"2024-05-20T18:27:06.803606+0000","last_active":"2024-05-20T18:27:13.821617+0000","last_peered":"2024-05-20T18:27:13.821617+0000","last_clean":"2024-05-20T18:27:13.821617+0000","last_became_active":"2024-05-20T18:27:06.803340+0000","last_became_peered":"2024-05-20T18:27:06.803340+0000","last_unstale":"2024-05-20T18:27:13.821617+0000","last_undegraded":"2024-05-20T18:27:13.821617+0000","last_fullsized":"2024-05-20T18:27:13.821617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:10:27.276400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048051+0000","last_change":"2024-05-20T18:27:07.789834+0000","last_active":"2024-05-20T18:27:14.048051+0000","last_peered":"2024-05-20T18:27:14.048051+0000","last_clean":"2024-05-20T18:27:14.048051+0000","last_became_active":"2024-05-20T18:27:07.789472+0000","last_became_peered":"2024-05-20T18:27:07.789472+0000","last_unstale":"2024-05-20T18:27:14.048051+0000","last_undegraded":"2024-05-20T18:27:14.048051+0000","last_fullsized":"2024-05-20T18:27:14.048051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:52:45.948931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821874+0000","last_change":"2024-05-20T18:27:11.825859+0000","last_active":"2024-05-20T18:27:13.821874+0000","last_peered":"2024-05-20T18:27:13.821874+0000","last_clean":"2024-05-20T18:27:13.821874+0000","last_became_active":"2024-05-20T18:27:11.825093+0000","last_became_peered":"2024-05-20T18:27:11.825093+0000","last_unstale":"2024-05-20T18:27:13.821874+0000","last_undegraded":"2024-05-20T18:27:13.821874+0000","last_fullsized":"2024-05-20T18:27:13.821874+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:02:46.122485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825718+0000","last_change":"2024-05-20T18:27:07.793263+0000","last_active":"2024-05-20T18:27:13.825718+0000","last_peered":"2024-05-20T18:27:13.825718+0000","last_clean":"2024-05-20T18:27:13.825718+0000","last_became_active":"2024-05-20T18:27:07.792916+0000","last_became_peered":"2024-05-20T18:27:07.792916+0000","last_unstale":"2024-05-20T18:27:13.825718+0000","last_undegraded":"2024-05-20T18:27:13.825718+0000","last_fullsized":"2024-05-20T18:27:13.825718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:15:10.022458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047433+0000","last_change":"2024-05-20T18:27:07.805541+0000","last_active":"2024-05-20T18:27:14.047433+0000","last_peered":"2024-05-20T18:27:14.047433+0000","last_clean":"2024-05-20T18:27:14.047433+0000","last_became_active":"2024-05-20T18:27:07.805160+0000","last_became_peered":"2024-05-20T18:27:07.805160+0000","last_unstale":"2024-05-20T18:27:14.047433+0000","last_undegraded":"2024-05-20T18:27:14.047433+0000","last_fullsized":"2024-05-20T18:27:14.047433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:14:23.046328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824975+0000","last_change":"2024-05-20T18:27:07.804547+0000","last_active":"2024-05-20T18:27:13.824975+0000","last_peered":"2024-05-20T18:27:13.824975+0000","last_clean":"2024-05-20T18:27:13.824975+0000","last_became_active":"2024-05-20T18:27:07.804242+0000","last_became_peered":"2024-05-20T18:27:07.804242+0000","last_unstale":"2024-05-20T18:27:13.824975+0000","last_undegraded":"2024-05-20T18:27:13.824975+0000","last_fullsized":"2024-05-20T18:27:13.824975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:55:24.560518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048436+0000","last_change":"2024-05-20T18:27:07.790683+0000","last_active":"2024-05-20T18:27:14.048436+0000","last_peered":"2024-05-20T18:27:14.048436+0000","last_clean":"2024-05-20T18:27:14.048436+0000","last_became_active":"2024-05-20T18:27:07.789978+0000","last_became_peered":"2024-05-20T18:27:07.789978+0000","last_unstale":"2024-05-20T18:27:14.048436+0000","last_undegraded":"2024-05-20T18:27:14.048436+0000","last_fullsized":"2024-05-20T18:27:14.048436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:40:08.698691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821852+0000","last_change":"2024-05-20T18:27:07.791924+0000","last_active":"2024-05-20T18:27:13.821852+0000","last_peered":"2024-05-20T18:27:13.821852+0000","last_clean":"2024-05-20T18:27:13.821852+0000","last_became_active":"2024-05-20T18:27:07.791622+0000","last_became_peered":"2024-05-20T18:27:07.791622+0000","last_unstale":"2024-05-20T18:27:13.821852+0000","last_undegraded":"2024-05-20T18:27:13.821852+0000","last_fullsized":"2024-05-20T18:27:13.821852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:07:15.764912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048748+0000","last_change":"2024-05-20T18:27:07.789116+0000","last_active":"2024-05-20T18:27:14.048748+0000","last_peered":"2024-05-20T18:27:14.048748+0000","last_clean":"2024-05-20T18:27:14.048748+0000","last_became_active":"2024-05-20T18:27:07.787855+0000","last_became_peered":"2024-05-20T18:27:07.787855+0000","last_unstale":"2024-05-20T18:27:14.048748+0000","last_undegraded":"2024-05-20T18:27:14.048748+0000","last_fullsized":"2024-05-20T18:27:14.048748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:52:57.891709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825914+0000","last_change":"2024-05-20T18:27:07.793074+0000","last_active":"2024-05-20T18:27:13.825914+0000","last_peered":"2024-05-20T18:27:13.825914+0000","last_clean":"2024-05-20T18:27:13.825914+0000","last_became_active":"2024-05-20T18:27:07.792754+0000","last_became_peered":"2024-05-20T18:27:07.792754+0000","last_unstale":"2024-05-20T18:27:13.825914+0000","last_undegraded":"2024-05-20T18:27:13.825914+0000","last_fullsized":"2024-05-20T18:27:13.825914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:12.006827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.822159+0000","last_change":"2024-05-20T18:27:07.791208+0000","last_active":"2024-05-20T18:27:13.822159+0000","last_peered":"2024-05-20T18:27:13.822159+0000","last_clean":"2024-05-20T18:27:13.822159+0000","last_became_active":"2024-05-20T18:27:07.790864+0000","last_became_peered":"2024-05-20T18:27:07.790864+0000","last_unstale":"2024-05-20T18:27:13.822159+0000","last_undegraded":"2024-05-20T18:27:13.822159+0000","last_fullsized":"2024-05-20T18:27:13.822159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:19:31.649320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824356+0000","last_change":"2024-05-20T18:27:07.808305+0000","last_active":"2024-05-20T18:27:13.824356+0000","last_peered":"2024-05-20T18:27:13.824356+0000","last_clean":"2024-05-20T18:27:13.824356+0000","last_became_active":"2024-05-20T18:27:07.807877+0000","last_became_peered":"2024-05-20T18:27:07.807877+0000","last_unstale":"2024-05-20T18:27:13.824356+0000","last_undegraded":"2024-05-20T18:27:13.824356+0000","last_fullsized":"2024-05-20T18:27:13.824356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:14:03.712607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821322+0000","last_change":"2024-05-20T18:27:07.791872+0000","last_active":"2024-05-20T18:27:13.821322+0000","last_peered":"2024-05-20T18:27:13.821322+0000","last_clean":"2024-05-20T18:27:13.821322+0000","last_became_active":"2024-05-20T18:27:07.791570+0000","last_became_peered":"2024-05-20T18:27:07.791570+0000","last_unstale":"2024-05-20T18:27:13.821322+0000","last_undegraded":"2024-05-20T18:27:13.821322+0000","last_fullsized":"2024-05-20T18:27:13.821322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:29:33.478403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824943+0000","last_change":"2024-05-20T18:27:07.811057+0000","last_active":"2024-05-20T18:27:13.824943+0000","last_peered":"2024-05-20T18:27:13.824943+0000","last_clean":"2024-05-20T18:27:13.824943+0000","last_became_active":"2024-05-20T18:27:07.810728+0000","last_became_peered":"2024-05-20T18:27:07.810728+0000","last_unstale":"2024-05-20T18:27:13.824943+0000","last_undegraded":"2024-05-20T18:27:13.824943+0000","last_fullsized":"2024-05-20T18:27:13.824943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:19:31.649320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048182+0000","last_change":"2024-05-20T18:27:07.795559+0000","last_active":"2024-05-20T18:27:14.048182+0000","last_peered":"2024-05-20T18:27:14.048182+0000","last_clean":"2024-05-20T18:27:14.048182+0000","last_became_active":"2024-05-20T18:27:07.795228+0000","last_became_peered":"2024-05-20T18:27:07.795228+0000","last_unstale":"2024-05-20T18:27:14.048182+0000","last_undegraded":"2024-05-20T18:27:14.048182+0000","last_fullsized":"2024-05-20T18:27:14.048182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:10:20.947601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.826124+0000","last_change":"2024-05-20T18:27:07.795052+0000","last_active":"2024-05-20T18:27:13.826124+0000","last_peered":"2024-05-20T18:27:13.826124+0000","last_clean":"2024-05-20T18:27:13.826124+0000","last_became_active":"2024-05-20T18:27:07.794704+0000","last_became_peered":"2024-05-20T18:27:07.794704+0000","last_unstale":"2024-05-20T18:27:13.826124+0000","last_undegraded":"2024-05-20T18:27:13.826124+0000","last_fullsized":"2024-05-20T18:27:13.826124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:34:15.614124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049515+0000","last_change":"2024-05-20T18:27:07.807441+0000","last_active":"2024-05-20T18:27:14.049515+0000","last_peered":"2024-05-20T18:27:14.049515+0000","last_clean":"2024-05-20T18:27:14.049515+0000","last_became_active":"2024-05-20T18:27:07.807095+0000","last_became_peered":"2024-05-20T18:27:07.807095+0000","last_unstale":"2024-05-20T18:27:14.049515+0000","last_undegraded":"2024-05-20T18:27:14.049515+0000","last_fullsized":"2024-05-20T18:27:14.049515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:24:59.216904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047301+0000","last_change":"2024-05-20T18:27:07.801623+0000","last_active":"2024-05-20T18:27:14.047301+0000","last_peered":"2024-05-20T18:27:14.047301+0000","last_clean":"2024-05-20T18:27:14.047301+0000","last_became_active":"2024-05-20T18:27:07.801303+0000","last_became_peered":"2024-05-20T18:27:07.801303+0000","last_unstale":"2024-05-20T18:27:14.047301+0000","last_undegraded":"2024-05-20T18:27:14.047301+0000","last_fullsized":"2024-05-20T18:27:14.047301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:29:19.667394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824621+0000","last_change":"2024-05-20T18:27:07.809861+0000","last_active":"2024-05-20T18:27:13.824621+0000","last_peered":"2024-05-20T18:27:13.824621+0000","last_clean":"2024-05-20T18:27:13.824621+0000","last_became_active":"2024-05-20T18:27:07.809523+0000","last_became_peered":"2024-05-20T18:27:07.809523+0000","last_unstale":"2024-05-20T18:27:13.824621+0000","last_undegraded":"2024-05-20T18:27:13.824621+0000","last_fullsized":"2024-05-20T18:27:13.824621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:55:32.958186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.823933+0000","last_change":"2024-05-20T18:27:07.814234+0000","last_active":"2024-05-20T18:27:13.823933+0000","last_peered":"2024-05-20T18:27:13.823933+0000","last_clean":"2024-05-20T18:27:13.823933+0000","last_became_active":"2024-05-20T18:27:07.813168+0000","last_became_peered":"2024-05-20T18:27:07.813168+0000","last_unstale":"2024-05-20T18:27:13.823933+0000","last_undegraded":"2024-05-20T18:27:13.823933+0000","last_fullsized":"2024-05-20T18:27:13.823933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:12.006827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048520+0000","last_change":"2024-05-20T18:27:07.787849+0000","last_active":"2024-05-20T18:27:14.048520+0000","last_peered":"2024-05-20T18:27:14.048520+0000","last_clean":"2024-05-20T18:27:14.048520+0000","last_became_active":"2024-05-20T18:27:07.787406+0000","last_became_peered":"2024-05-20T18:27:07.787406+0000","last_unstale":"2024-05-20T18:27:14.048520+0000","last_undegraded":"2024-05-20T18:27:14.048520+0000","last_fullsized":"2024-05-20T18:27:14.048520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:34:15.614124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047987+0000","last_change":"2024-05-20T18:27:07.795141+0000","last_active":"2024-05-20T18:27:14.047987+0000","last_peered":"2024-05-20T18:27:14.047987+0000","last_clean":"2024-05-20T18:27:14.047987+0000","last_became_active":"2024-05-20T18:27:07.793487+0000","last_became_peered":"2024-05-20T18:27:07.793487+0000","last_unstale":"2024-05-20T18:27:14.047987+0000","last_undegraded":"2024-05-20T18:27:14.047987+0000","last_fullsized":"2024-05-20T18:27:14.047987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:32:42.769509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821496+0000","last_change":"2024-05-20T18:27:07.802751+0000","last_active":"2024-05-20T18:27:13.821496+0000","last_peered":"2024-05-20T18:27:13.821496+0000","last_clean":"2024-05-20T18:27:13.821496+0000","last_became_active":"2024-05-20T18:27:07.802412+0000","last_became_peered":"2024-05-20T18:27:07.802412+0000","last_unstale":"2024-05-20T18:27:13.821496+0000","last_undegraded":"2024-05-20T18:27:13.821496+0000","last_fullsized":"2024-05-20T18:27:13.821496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:14:03.712607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048564+0000","last_change":"2024-05-20T18:27:07.796408+0000","last_active":"2024-05-20T18:27:14.048564+0000","last_peered":"2024-05-20T18:27:14.048564+0000","last_clean":"2024-05-20T18:27:14.048564+0000","last_became_active":"2024-05-20T18:27:07.795961+0000","last_became_peered":"2024-05-20T18:27:07.795961+0000","last_unstale":"2024-05-20T18:27:14.048564+0000","last_undegraded":"2024-05-20T18:27:14.048564+0000","last_fullsized":"2024-05-20T18:27:14.048564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:43:32.159862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824841+0000","last_change":"2024-05-20T18:27:07.806684+0000","last_active":"2024-05-20T18:27:13.824841+0000","last_peered":"2024-05-20T18:27:13.824841+0000","last_clean":"2024-05-20T18:27:13.824841+0000","last_became_active":"2024-05-20T18:27:07.806320+0000","last_became_peered":"2024-05-20T18:27:07.806320+0000","last_unstale":"2024-05-20T18:27:13.824841+0000","last_undegraded":"2024-05-20T18:27:13.824841+0000","last_fullsized":"2024-05-20T18:27:13.824841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:52:57.891709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825792+0000","last_change":"2024-05-20T18:27:07.805587+0000","last_active":"2024-05-20T18:27:13.825792+0000","last_peered":"2024-05-20T18:27:13.825792+0000","last_clean":"2024-05-20T18:27:13.825792+0000","last_became_active":"2024-05-20T18:27:07.804971+0000","last_became_peered":"2024-05-20T18:27:07.804971+0000","last_unstale":"2024-05-20T18:27:13.825792+0000","last_undegraded":"2024-05-20T18:27:13.825792+0000","last_fullsized":"2024-05-20T18:27:13.825792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:07:15.764912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049422+0000","last_change":"2024-05-20T18:27:07.815867+0000","last_active":"2024-05-20T18:27:14.049422+0000","last_peered":"2024-05-20T18:27:14.049422+0000","last_clean":"2024-05-20T18:27:14.049422+0000","last_became_active":"2024-05-20T18:27:07.815421+0000","last_became_peered":"2024-05-20T18:27:07.815421+0000","last_unstale":"2024-05-20T18:27:14.049422+0000","last_undegraded":"2024-05-20T18:27:14.049422+0000","last_fullsized":"2024-05-20T18:27:14.049422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:16:37.938695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821166+0000","last_change":"2024-05-20T18:27:07.810206+0000","last_active":"2024-05-20T18:27:13.821166+0000","last_peered":"2024-05-20T18:27:13.821166+0000","last_clean":"2024-05-20T18:27:13.821166+0000","last_became_active":"2024-05-20T18:27:07.809846+0000","last_became_peered":"2024-05-20T18:27:07.809846+0000","last_unstale":"2024-05-20T18:27:13.821166+0000","last_undegraded":"2024-05-20T18:27:13.821166+0000","last_fullsized":"2024-05-20T18:27:13.821166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:10:20.947601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048623+0000","last_change":"2024-05-20T18:27:07.813420+0000","last_active":"2024-05-20T18:27:14.048623+0000","last_peered":"2024-05-20T18:27:14.048623+0000","last_clean":"2024-05-20T18:27:14.048623+0000","last_became_active":"2024-05-20T18:27:07.813093+0000","last_became_peered":"2024-05-20T18:27:07.813093+0000","last_unstale":"2024-05-20T18:27:14.048623+0000","last_undegraded":"2024-05-20T18:27:14.048623+0000","last_fullsized":"2024-05-20T18:27:14.048623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:19:31.649320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824831+0000","last_change":"2024-05-20T18:27:07.812561+0000","last_active":"2024-05-20T18:27:13.824831+0000","last_peered":"2024-05-20T18:27:13.824831+0000","last_clean":"2024-05-20T18:27:13.824831+0000","last_became_active":"2024-05-20T18:27:07.812264+0000","last_became_peered":"2024-05-20T18:27:07.812264+0000","last_unstale":"2024-05-20T18:27:13.824831+0000","last_undegraded":"2024-05-20T18:27:13.824831+0000","last_fullsized":"2024-05-20T18:27:13.824831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:52:22.903884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048783+0000","last_change":"2024-05-20T18:27:07.804441+0000","last_active":"2024-05-20T18:27:14.048783+0000","last_peered":"2024-05-20T18:27:14.048783+0000","last_clean":"2024-05-20T18:27:14.048783+0000","last_became_active":"2024-05-20T18:27:07.804126+0000","last_became_peered":"2024-05-20T18:27:07.804126+0000","last_unstale":"2024-05-20T18:27:14.048783+0000","last_undegraded":"2024-05-20T18:27:14.048783+0000","last_fullsized":"2024-05-20T18:27:14.048783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:14:17.791789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825082+0000","last_change":"2024-05-20T18:27:07.808819+0000","last_active":"2024-05-20T18:27:13.825082+0000","last_peered":"2024-05-20T18:27:13.825082+0000","last_clean":"2024-05-20T18:27:13.825082+0000","last_became_active":"2024-05-20T18:27:07.808231+0000","last_became_peered":"2024-05-20T18:27:07.808231+0000","last_unstale":"2024-05-20T18:27:13.825082+0000","last_undegraded":"2024-05-20T18:27:13.825082+0000","last_fullsized":"2024-05-20T18:27:13.825082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:14:17.791789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048097+0000","last_change":"2024-05-20T18:27:07.793960+0000","last_active":"2024-05-20T18:27:14.048097+0000","last_peered":"2024-05-20T18:27:14.048097+0000","last_clean":"2024-05-20T18:27:14.048097+0000","last_became_active":"2024-05-20T18:27:07.793594+0000","last_became_peered":"2024-05-20T18:27:07.793594+0000","last_unstale":"2024-05-20T18:27:14.048097+0000","last_undegraded":"2024-05-20T18:27:14.048097+0000","last_fullsized":"2024-05-20T18:27:14.048097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:34:15.614124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049410+0000","last_change":"2024-05-20T18:27:07.790606+0000","last_active":"2024-05-20T18:27:14.049410+0000","last_peered":"2024-05-20T18:27:14.049410+0000","last_clean":"2024-05-20T18:27:14.049410+0000","last_became_active":"2024-05-20T18:27:07.790227+0000","last_became_peered":"2024-05-20T18:27:07.790227+0000","last_unstale":"2024-05-20T18:27:14.049410+0000","last_undegraded":"2024-05-20T18:27:14.049410+0000","last_fullsized":"2024-05-20T18:27:14.049410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:14:03.712607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048973+0000","last_change":"2024-05-20T18:27:07.816207+0000","last_active":"2024-05-20T18:27:14.048973+0000","last_peered":"2024-05-20T18:27:14.048973+0000","last_clean":"2024-05-20T18:27:14.048973+0000","last_became_active":"2024-05-20T18:27:07.815556+0000","last_became_peered":"2024-05-20T18:27:07.815556+0000","last_unstale":"2024-05-20T18:27:14.048973+0000","last_undegraded":"2024-05-20T18:27:14.048973+0000","last_fullsized":"2024-05-20T18:27:14.048973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:55:24.560518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":876544,"data_stored":703014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":115906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":112154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":112190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":290816,"data_stored":115906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":278528,"data_stored":112064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":286720,"data_stored":112224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":884736,"data_stored":703070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T18:27:30.965 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T18:27:31.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.116+0000 7f60d2b02640 1 Processor -- start 2024-05-20T18:27:31.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.116+0000 7f60d2b02640 1 -- start start 2024-05-20T18:27:31.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.117+0000 7f60d2b02640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60cc15d530 0x7f60cc15aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:31.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.117+0000 7f60d2b02640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f60cc0b1700 0x7f60cc0b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:31.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.117+0000 7f60d2b02640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60cc0cd580 con 0x7f60cc070850 2024-05-20T18:27:31.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.117+0000 7f60d2b02640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f60cc0ce190 con 0x7f60cc0b1700 2024-05-20T18:27:31.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.117+0000 7f60d2b02640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f60cc0cd860 con 0x7f60cc15d530 2024-05-20T18:27:31.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.118+0000 7f60c3fff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f60cc0b1700 0x7f60cc0b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:31.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.118+0000 7f60d089e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60cc15d530 0x7f60cc15aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:31.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.118+0000 7f60d089e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60cc15d530 0x7f60cc15aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:46104/0 (socket says 172.21.15.5:46104) 2024-05-20T18:27:31.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.118+0000 7f60c3fff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f60cc0b1700 0x7f60cc0b1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:53312/0 (socket says 172.21.15.5:53312) 2024-05-20T18:27:31.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.118+0000 7f60d089e640 1 -- 172.21.15.5:0/1017663624 learned_addr learned my addr 172.21.15.5:0/1017663624 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:31.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.118+0000 7f60d089e640 1 -- 172.21.15.5:0/1017663624 >> v2:172.21.15.63:3300/0 conn(0x7f60cc0b1700 msgr2=0x7f60cc0b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:31.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.118+0000 7f60d089e640 1 --2- 172.21.15.5:0/1017663624 >> v2:172.21.15.63:3300/0 conn(0x7f60cc0b1700 0x7f60cc0b1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:31.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.118+0000 7f60d089e640 1 -- 172.21.15.5:0/1017663624 >> v1:172.21.15.63:6789/0 conn(0x7f60cc070850 legacy=0x7f60cc070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:31.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.118+0000 7f60d089e640 1 -- 172.21.15.5:0/1017663624 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60cc0cdb40 con 0x7f60cc15d530 2024-05-20T18:27:31.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.118+0000 7f60c3fff640 1 --2- 172.21.15.5:0/1017663624 >> v2:172.21.15.63:3300/0 conn(0x7f60cc0b1700 0x7f60cc0b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:27:31.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.119+0000 7f60d089e640 1 --2- 172.21.15.5:0/1017663624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60cc15d530 0x7f60cc15aaa0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009e50 tx=0x7f60c802a730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5af20e1a2731070c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:31.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.119+0000 7f60c37fe640 1 -- 172.21.15.5:0/1017663624 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c8036070 con 0x7f60cc15d530 2024-05-20T18:27:31.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.119+0000 7f60c37fe640 1 -- 172.21.15.5:0/1017663624 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60c8031600 con 0x7f60cc15d530 2024-05-20T18:27:31.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.119+0000 7f60c37fe640 1 -- 172.21.15.5:0/1017663624 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c80319d0 con 0x7f60cc15d530 2024-05-20T18:27:31.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.120+0000 7f60d2b02640 1 -- 172.21.15.5:0/1017663624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60cc15d530 msgr2=0x7f60cc15aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:31.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.120+0000 7f60d2b02640 1 --2- 172.21.15.5:0/1017663624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60cc15d530 0x7f60cc15aaa0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009e50 tx=0x7f60c802a730 comp rx=0 tx=0).stop 2024-05-20T18:27:31.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.120+0000 7f60d2b02640 1 -- 172.21.15.5:0/1017663624 shutdown_connections 2024-05-20T18:27:31.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.120+0000 7f60d2b02640 1 --2- 172.21.15.5:0/1017663624 >> v2:172.21.15.63:3300/0 conn(0x7f60cc0b1700 0x7f60cc0b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:31.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.120+0000 7f60d2b02640 1 --2- 172.21.15.5:0/1017663624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60cc15d530 0x7f60cc15aaa0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:31.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.120+0000 7f60d2b02640 1 -- 172.21.15.5:0/1017663624 >> 172.21.15.5:0/1017663624 conn(0x7f60cc0d29d0 msgr2=0x7f60cc0d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:31.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.120+0000 7f60d2b02640 1 -- 172.21.15.5:0/1017663624 shutdown_connections 2024-05-20T18:27:31.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.120+0000 7f60d2b02640 1 -- 172.21.15.5:0/1017663624 wait complete. 2024-05-20T18:27:31.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.122+0000 7f60d2b02640 1 Processor -- start 2024-05-20T18:27:31.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.122+0000 7f60d2b02640 1 -- start start 2024-05-20T18:27:31.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.123+0000 7f60d2b02640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60cc070850 0x7f60cc148e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:31.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.123+0000 7f60d2b02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60cc0b1700 0x7f60cc0d1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:31.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.123+0000 7f60d2b02640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60cc15d530 0x7f60cc0d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:31.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.123+0000 7f60d2b02640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc0b2050 con 0x7f60cc0b1700 2024-05-20T18:27:31.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.123+0000 7f60d2b02640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc0cd230 con 0x7f60cc15d530 2024-05-20T18:27:31.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.123+0000 7f60d2b02640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f60cc071370 con 0x7f60cc070850 2024-05-20T18:27:31.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.123+0000 7f60c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60cc0b1700 0x7f60cc0d1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:31.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.124+0000 7f60c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60cc0b1700 0x7f60cc0d1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59002/0 (socket says 172.21.15.5:59002) 2024-05-20T18:27:31.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.124+0000 7f60c3fff640 1 -- 172.21.15.5:0/401636588 learned_addr learned my addr 172.21.15.5:0/401636588 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:31.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.124+0000 7f60d109f640 1 --2- 172.21.15.5:0/401636588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60cc15d530 0x7f60cc0d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:31.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.124+0000 7f60d089e640 1 --2- 172.21.15.5:0/401636588 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60cc070850 0x7f60cc148e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.124+0000 7f60c3fff640 1 -- 172.21.15.5:0/401636588 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60cc070850 msgr2=0x7f60cc148e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.124+0000 7f60c3fff640 1 --2- 172.21.15.5:0/401636588 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60cc070850 0x7f60cc148e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.124+0000 7f60c3fff640 1 -- 172.21.15.5:0/401636588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60cc15d530 msgr2=0x7f60cc0d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.124+0000 7f60c3fff640 1 --2- 172.21.15.5:0/401636588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60cc15d530 0x7f60cc0d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.124+0000 7f60c3fff640 1 -- 172.21.15.5:0/401636588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60cc149380 con 0x7f60cc0b1700 2024-05-20T18:27:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.124+0000 7f60c3fff640 1 --2- 172.21.15.5:0/401636588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60cc0b1700 0x7f60cc0d1860 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f60bc00e900 tx=0x7f60bc00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.124+0000 7f60c17fa640 1 -- 172.21.15.5:0/401636588 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc017070 con 0x7f60cc0b1700 2024-05-20T18:27:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.124+0000 7f60c17fa640 1 -- 172.21.15.5:0/401636588 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60bc00ca00 con 0x7f60cc0b1700 2024-05-20T18:27:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.124+0000 7f60d2b02640 1 -- 172.21.15.5:0/401636588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60cc0c8750 con 0x7f60cc0b1700 2024-05-20T18:27:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.124+0000 7f60c17fa640 1 -- 172.21.15.5:0/401636588 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc00cdf0 con 0x7f60cc0b1700 2024-05-20T18:27:31.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.124+0000 7f60d2b02640 1 -- 172.21.15.5:0/401636588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60cc0c8d10 con 0x7f60cc0b1700 2024-05-20T18:27:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.126+0000 7f60d2b02640 1 -- 172.21.15.5:0/401636588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60cc075b00 con 0x7f60cc0b1700 2024-05-20T18:27:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.126+0000 7f60c17fa640 1 -- 172.21.15.5:0/401636588 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f60bc01a070 con 0x7f60cc0b1700 2024-05-20T18:27:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.126+0000 7f60c17fa640 1 --2- 172.21.15.5:0/401636588 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f60b806be30 0x7f60b806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.126+0000 7f60c17fa640 1 -- 172.21.15.5:0/401636588 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f60bc08adc0 con 0x7f60cc0b1700 2024-05-20T18:27:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.126+0000 7f60d089e640 1 --2- 172.21.15.5:0/401636588 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f60b806be30 0x7f60b806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.127+0000 7f60d089e640 1 --2- 172.21.15.5:0/401636588 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f60b806be30 0x7f60b806e340 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f60c8033000 tx=0x7f60c8009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.131+0000 7f60c17fa640 1 -- 172.21.15.5:0/401636588 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f60cc149380 con 0x7f60cc0b1700 2024-05-20T18:27:31.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.312+0000 7f60d2b02640 1 -- 172.21.15.5:0/401636588 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f60cc067f60 con 0x7f60b806be30 2024-05-20T18:27:31.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.320+0000 7f60c17fa640 1 -- 172.21.15.5:0/401636588 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332558 (secure 0 0 0) 0x7f60cc067f60 con 0x7f60b806be30 2024-05-20T18:27:31.324 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:31.325 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-20T18:27:31.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.324+0000 7f60d2b02640 1 -- 172.21.15.5:0/401636588 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f60b806be30 msgr2=0x7f60b806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:31.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.324+0000 7f60d2b02640 1 --2- 172.21.15.5:0/401636588 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f60b806be30 0x7f60b806e340 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f60c8033000 tx=0x7f60c8009000 comp rx=0 tx=0).stop 2024-05-20T18:27:31.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.324+0000 7f60d2b02640 1 -- 172.21.15.5:0/401636588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60cc0b1700 msgr2=0x7f60cc0d1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:31.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.324+0000 7f60d2b02640 1 --2- 172.21.15.5:0/401636588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60cc0b1700 0x7f60cc0d1860 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f60bc00e900 tx=0x7f60bc00edc0 comp rx=0 tx=0).stop 2024-05-20T18:27:31.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.324+0000 7f60d2b02640 1 -- 172.21.15.5:0/401636588 shutdown_connections 2024-05-20T18:27:31.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.324+0000 7f60d2b02640 1 --2- 172.21.15.5:0/401636588 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f60b806be30 0x7f60b806e340 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:31.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.324+0000 7f60d2b02640 1 --2- 172.21.15.5:0/401636588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60cc15d530 0x7f60cc0d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:31.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.324+0000 7f60d2b02640 1 --2- 172.21.15.5:0/401636588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60cc0b1700 0x7f60cc0d1860 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:31.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.324+0000 7f60d2b02640 1 --2- 172.21.15.5:0/401636588 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60cc070850 0x7f60cc148e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:31.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.324+0000 7f60d2b02640 1 -- 172.21.15.5:0/401636588 >> 172.21.15.5:0/401636588 conn(0x7f60cc0d29d0 msgr2=0x7f60cc15f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:31.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.324+0000 7f60d2b02640 1 -- 172.21.15.5:0/401636588 shutdown_connections 2024-05-20T18:27:31.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.324+0000 7f60d2b02640 1 -- 172.21.15.5:0/401636588 wait complete. 2024-05-20T18:27:31.335 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-05-20T18:27:30.578435+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236216,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738504,"statfs":{"total":773094113280,"available":772852228096,"internally_reserved":0,"allocated":3481600,"data_stored":2086528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003039"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048716+0000","last_change":"2024-05-20T18:27:06.794819+0000","last_active":"2024-05-20T18:27:14.048716+0000","last_peered":"2024-05-20T18:27:14.048716+0000","last_clean":"2024-05-20T18:27:14.048716+0000","last_became_active":"2024-05-20T18:27:06.794505+0000","last_became_peered":"2024-05-20T18:27:06.794505+0000","last_unstale":"2024-05-20T18:27:14.048716+0000","last_undegraded":"2024-05-20T18:27:14.048716+0000","last_fullsized":"2024-05-20T18:27:14.048716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:20:52.095853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821927+0000","last_change":"2024-05-20T18:27:07.802838+0000","last_active":"2024-05-20T18:27:13.821927+0000","last_peered":"2024-05-20T18:27:13.821927+0000","last_clean":"2024-05-20T18:27:13.821927+0000","last_became_active":"2024-05-20T18:27:07.802468+0000","last_became_peered":"2024-05-20T18:27:07.802468+0000","last_unstale":"2024-05-20T18:27:13.821927+0000","last_undegraded":"2024-05-20T18:27:13.821927+0000","last_fullsized":"2024-05-20T18:27:13.821927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:12.006827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821103+0000","last_change":"2024-05-20T18:27:12.469613+0000","last_active":"2024-05-20T18:27:13.821103+0000","last_peered":"2024-05-20T18:27:13.821103+0000","last_clean":"2024-05-20T18:27:13.821103+0000","last_became_active":"2024-05-20T18:27:12.469161+0000","last_became_peered":"2024-05-20T18:27:12.469161+0000","last_unstale":"2024-05-20T18:27:13.821103+0000","last_undegraded":"2024-05-20T18:27:13.821103+0000","last_fullsized":"2024-05-20T18:27:13.821103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:32:50.234917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.822083+0000","last_change":"2024-05-20T18:27:06.799063+0000","last_active":"2024-05-20T18:27:13.822083+0000","last_peered":"2024-05-20T18:27:13.822083+0000","last_clean":"2024-05-20T18:27:13.822083+0000","last_became_active":"2024-05-20T18:27:06.798689+0000","last_became_peered":"2024-05-20T18:27:06.798689+0000","last_unstale":"2024-05-20T18:27:13.822083+0000","last_undegraded":"2024-05-20T18:27:13.822083+0000","last_fullsized":"2024-05-20T18:27:13.822083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:52:44.943426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048397+0000","last_change":"2024-05-20T18:27:07.805002+0000","last_active":"2024-05-20T18:27:14.048397+0000","last_peered":"2024-05-20T18:27:14.048397+0000","last_clean":"2024-05-20T18:27:14.048397+0000","last_became_active":"2024-05-20T18:27:07.804655+0000","last_became_peered":"2024-05-20T18:27:07.804655+0000","last_unstale":"2024-05-20T18:27:14.048397+0000","last_undegraded":"2024-05-20T18:27:14.048397+0000","last_fullsized":"2024-05-20T18:27:14.048397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:52:57.891709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047788+0000","last_change":"2024-05-20T18:27:11.833803+0000","last_active":"2024-05-20T18:27:14.047788+0000","last_peered":"2024-05-20T18:27:14.047788+0000","last_clean":"2024-05-20T18:27:14.047788+0000","last_became_active":"2024-05-20T18:27:11.833241+0000","last_became_peered":"2024-05-20T18:27:11.833241+0000","last_unstale":"2024-05-20T18:27:14.047788+0000","last_undegraded":"2024-05-20T18:27:14.047788+0000","last_fullsized":"2024-05-20T18:27:14.047788+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:25:03.246597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049511+0000","last_change":"2024-05-20T18:27:06.806008+0000","last_active":"2024-05-20T18:27:14.049511+0000","last_peered":"2024-05-20T18:27:14.049511+0000","last_clean":"2024-05-20T18:27:14.049511+0000","last_became_active":"2024-05-20T18:27:06.805632+0000","last_became_peered":"2024-05-20T18:27:06.805632+0000","last_unstale":"2024-05-20T18:27:14.049511+0000","last_undegraded":"2024-05-20T18:27:14.049511+0000","last_fullsized":"2024-05-20T18:27:14.049511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:11:07.831277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825124+0000","last_change":"2024-05-20T18:27:07.812390+0000","last_active":"2024-05-20T18:27:13.825124+0000","last_peered":"2024-05-20T18:27:13.825124+0000","last_clean":"2024-05-20T18:27:13.825124+0000","last_became_active":"2024-05-20T18:27:07.811667+0000","last_became_peered":"2024-05-20T18:27:07.811667+0000","last_unstale":"2024-05-20T18:27:13.825124+0000","last_undegraded":"2024-05-20T18:27:13.825124+0000","last_fullsized":"2024-05-20T18:27:13.825124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:29:33.478403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048201+0000","last_change":"2024-05-20T18:27:11.833494+0000","last_active":"2024-05-20T18:27:14.048201+0000","last_peered":"2024-05-20T18:27:14.048201+0000","last_clean":"2024-05-20T18:27:14.048201+0000","last_became_active":"2024-05-20T18:27:11.832999+0000","last_became_peered":"2024-05-20T18:27:11.832999+0000","last_unstale":"2024-05-20T18:27:14.048201+0000","last_undegraded":"2024-05-20T18:27:14.048201+0000","last_fullsized":"2024-05-20T18:27:14.048201+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:08:42.465927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049057+0000","last_change":"2024-05-20T18:27:06.798023+0000","last_active":"2024-05-20T18:27:14.049057+0000","last_peered":"2024-05-20T18:27:14.049057+0000","last_clean":"2024-05-20T18:27:14.049057+0000","last_became_active":"2024-05-20T18:27:06.797674+0000","last_became_peered":"2024-05-20T18:27:06.797674+0000","last_unstale":"2024-05-20T18:27:14.049057+0000","last_undegraded":"2024-05-20T18:27:14.049057+0000","last_fullsized":"2024-05-20T18:27:14.049057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:43:31.154358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824776+0000","last_change":"2024-05-20T18:27:07.811508+0000","last_active":"2024-05-20T18:27:13.824776+0000","last_peered":"2024-05-20T18:27:13.824776+0000","last_clean":"2024-05-20T18:27:13.824776+0000","last_became_active":"2024-05-20T18:27:07.811217+0000","last_became_peered":"2024-05-20T18:27:07.811217+0000","last_unstale":"2024-05-20T18:27:13.824776+0000","last_undegraded":"2024-05-20T18:27:13.824776+0000","last_fullsized":"2024-05-20T18:27:13.824776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:01:46.623171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047874+0000","last_change":"2024-05-20T18:27:11.827280+0000","last_active":"2024-05-20T18:27:14.047874+0000","last_peered":"2024-05-20T18:27:14.047874+0000","last_clean":"2024-05-20T18:27:14.047874+0000","last_became_active":"2024-05-20T18:27:11.826770+0000","last_became_peered":"2024-05-20T18:27:11.826770+0000","last_unstale":"2024-05-20T18:27:14.047874+0000","last_undegraded":"2024-05-20T18:27:14.047874+0000","last_fullsized":"2024-05-20T18:27:14.047874+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:16:41.968388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825487+0000","last_change":"2024-05-20T18:27:07.791795+0000","last_active":"2024-05-20T18:27:13.825487+0000","last_peered":"2024-05-20T18:27:13.825487+0000","last_clean":"2024-05-20T18:27:13.825487+0000","last_became_active":"2024-05-20T18:27:07.791422+0000","last_became_peered":"2024-05-20T18:27:07.791422+0000","last_unstale":"2024-05-20T18:27:13.825487+0000","last_undegraded":"2024-05-20T18:27:13.825487+0000","last_fullsized":"2024-05-20T18:27:13.825487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:40:08.698691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049335+0000","last_change":"2024-05-20T18:27:06.799475+0000","last_active":"2024-05-20T18:27:14.049335+0000","last_peered":"2024-05-20T18:27:14.049335+0000","last_clean":"2024-05-20T18:27:14.049335+0000","last_became_active":"2024-05-20T18:27:06.798540+0000","last_became_peered":"2024-05-20T18:27:06.798540+0000","last_unstale":"2024-05-20T18:27:14.049335+0000","last_undegraded":"2024-05-20T18:27:14.049335+0000","last_fullsized":"2024-05-20T18:27:14.049335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:14:22.040824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048392+0000","last_change":"2024-05-20T18:27:11.826856+0000","last_active":"2024-05-20T18:27:14.048392+0000","last_peered":"2024-05-20T18:27:14.048392+0000","last_clean":"2024-05-20T18:27:14.048392+0000","last_became_active":"2024-05-20T18:27:11.826205+0000","last_became_peered":"2024-05-20T18:27:11.826205+0000","last_unstale":"2024-05-20T18:27:14.048392+0000","last_undegraded":"2024-05-20T18:27:14.048392+0000","last_fullsized":"2024-05-20T18:27:14.048392+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:34:21.648495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821544+0000","last_change":"2024-05-20T18:27:07.811973+0000","last_active":"2024-05-20T18:27:13.821544+0000","last_peered":"2024-05-20T18:27:13.821544+0000","last_clean":"2024-05-20T18:27:13.821544+0000","last_became_active":"2024-05-20T18:27:07.811617+0000","last_became_peered":"2024-05-20T18:27:07.811617+0000","last_unstale":"2024-05-20T18:27:13.821544+0000","last_undegraded":"2024-05-20T18:27:13.821544+0000","last_fullsized":"2024-05-20T18:27:13.821544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:52:22.903884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048965+0000","last_change":"2024-05-20T18:27:06.792524+0000","last_active":"2024-05-20T18:27:14.048965+0000","last_peered":"2024-05-20T18:27:14.048965+0000","last_clean":"2024-05-20T18:27:14.048965+0000","last_became_active":"2024-05-20T18:27:06.792104+0000","last_became_peered":"2024-05-20T18:27:06.792104+0000","last_unstale":"2024-05-20T18:27:14.048965+0000","last_undegraded":"2024-05-20T18:27:14.048965+0000","last_fullsized":"2024-05-20T18:27:14.048965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:10:27.276400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048408+0000","last_change":"2024-05-20T18:27:11.833928+0000","last_active":"2024-05-20T18:27:14.048408+0000","last_peered":"2024-05-20T18:27:14.048408+0000","last_clean":"2024-05-20T18:27:14.048408+0000","last_became_active":"2024-05-20T18:27:11.832887+0000","last_became_peered":"2024-05-20T18:27:11.832887+0000","last_unstale":"2024-05-20T18:27:14.048408+0000","last_undegraded":"2024-05-20T18:27:14.048408+0000","last_fullsized":"2024-05-20T18:27:14.048408+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:38:58.503399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821356+0000","last_change":"2024-05-20T18:27:07.788711+0000","last_active":"2024-05-20T18:27:13.821356+0000","last_peered":"2024-05-20T18:27:13.821356+0000","last_clean":"2024-05-20T18:27:13.821356+0000","last_became_active":"2024-05-20T18:27:07.788402+0000","last_became_peered":"2024-05-20T18:27:07.788402+0000","last_unstale":"2024-05-20T18:27:13.821356+0000","last_undegraded":"2024-05-20T18:27:13.821356+0000","last_fullsized":"2024-05-20T18:27:13.821356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:34:15.614124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048633+0000","last_change":"2024-05-20T18:27:06.803611+0000","last_active":"2024-05-20T18:27:14.048633+0000","last_peered":"2024-05-20T18:27:14.048633+0000","last_clean":"2024-05-20T18:27:14.048633+0000","last_became_active":"2024-05-20T18:27:06.803176+0000","last_became_peered":"2024-05-20T18:27:06.803176+0000","last_unstale":"2024-05-20T18:27:14.048633+0000","last_undegraded":"2024-05-20T18:27:14.048633+0000","last_fullsized":"2024-05-20T18:27:14.048633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:52:44.943426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049451+0000","last_change":"2024-05-20T18:27:12.466975+0000","last_active":"2024-05-20T18:27:14.049451+0000","last_peered":"2024-05-20T18:27:14.049451+0000","last_clean":"2024-05-20T18:27:14.049451+0000","last_became_active":"2024-05-20T18:27:12.466473+0000","last_became_peered":"2024-05-20T18:27:12.466473+0000","last_unstale":"2024-05-20T18:27:14.049451+0000","last_undegraded":"2024-05-20T18:27:14.049451+0000","last_fullsized":"2024-05-20T18:27:14.049451+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:55:23.896898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048689+0000","last_change":"2024-05-20T18:27:07.786091+0000","last_active":"2024-05-20T18:27:14.048689+0000","last_peered":"2024-05-20T18:27:14.048689+0000","last_clean":"2024-05-20T18:27:14.048689+0000","last_became_active":"2024-05-20T18:27:07.785704+0000","last_became_peered":"2024-05-20T18:27:07.785704+0000","last_unstale":"2024-05-20T18:27:14.048689+0000","last_undegraded":"2024-05-20T18:27:14.048689+0000","last_fullsized":"2024-05-20T18:27:14.048689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:06:32.450870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047619+0000","last_change":"2024-05-20T18:27:06.799668+0000","last_active":"2024-05-20T18:27:14.047619+0000","last_peered":"2024-05-20T18:27:14.047619+0000","last_clean":"2024-05-20T18:27:14.047619+0000","last_became_active":"2024-05-20T18:27:06.799002+0000","last_became_peered":"2024-05-20T18:27:06.799002+0000","last_unstale":"2024-05-20T18:27:14.047619+0000","last_undegraded":"2024-05-20T18:27:14.047619+0000","last_fullsized":"2024-05-20T18:27:14.047619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:06:31.445366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048146+0000","last_change":"2024-05-20T18:27:11.834133+0000","last_active":"2024-05-20T18:27:14.048146+0000","last_peered":"2024-05-20T18:27:14.048146+0000","last_clean":"2024-05-20T18:27:14.048146+0000","last_became_active":"2024-05-20T18:27:11.832891+0000","last_became_peered":"2024-05-20T18:27:11.832891+0000","last_unstale":"2024-05-20T18:27:14.048146+0000","last_undegraded":"2024-05-20T18:27:14.048146+0000","last_fullsized":"2024-05-20T18:27:14.048146+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:55:36.987878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049426+0000","last_change":"2024-05-20T18:27:07.790534+0000","last_active":"2024-05-20T18:27:14.049426+0000","last_peered":"2024-05-20T18:27:14.049426+0000","last_clean":"2024-05-20T18:27:14.049426+0000","last_became_active":"2024-05-20T18:27:07.790192+0000","last_became_peered":"2024-05-20T18:27:07.790192+0000","last_unstale":"2024-05-20T18:27:14.049426+0000","last_undegraded":"2024-05-20T18:27:14.049426+0000","last_fullsized":"2024-05-20T18:27:14.049426+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:12.006827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.826184+0000","last_change":"2024-05-20T18:27:06.794221+0000","last_active":"2024-05-20T18:27:13.826184+0000","last_peered":"2024-05-20T18:27:13.826184+0000","last_clean":"2024-05-20T18:27:13.826184+0000","last_became_active":"2024-05-20T18:27:06.793895+0000","last_became_peered":"2024-05-20T18:27:06.793895+0000","last_unstale":"2024-05-20T18:27:13.826184+0000","last_undegraded":"2024-05-20T18:27:13.826184+0000","last_fullsized":"2024-05-20T18:27:13.826184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:24:55.001924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048453+0000","last_change":"2024-05-20T18:27:12.471660+0000","last_active":"2024-05-20T18:27:14.048453+0000","last_peered":"2024-05-20T18:27:14.048453+0000","last_clean":"2024-05-20T18:27:14.048453+0000","last_became_active":"2024-05-20T18:27:12.470460+0000","last_became_peered":"2024-05-20T18:27:12.470460+0000","last_unstale":"2024-05-20T18:27:14.048453+0000","last_undegraded":"2024-05-20T18:27:14.048453+0000","last_fullsized":"2024-05-20T18:27:14.048453+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:53:56.893940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.822216+0000","last_change":"2024-05-20T18:27:07.799399+0000","last_active":"2024-05-20T18:27:13.822216+0000","last_peered":"2024-05-20T18:27:13.822216+0000","last_clean":"2024-05-20T18:27:13.822216+0000","last_became_active":"2024-05-20T18:27:07.799017+0000","last_became_peered":"2024-05-20T18:27:07.799017+0000","last_unstale":"2024-05-20T18:27:13.822216+0000","last_undegraded":"2024-05-20T18:27:13.822216+0000","last_fullsized":"2024-05-20T18:27:13.822216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:08:51.463251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049547+0000","last_change":"2024-05-20T18:27:06.794477+0000","last_active":"2024-05-20T18:27:14.049547+0000","last_peered":"2024-05-20T18:27:14.049547+0000","last_clean":"2024-05-20T18:27:14.049547+0000","last_became_active":"2024-05-20T18:27:06.794130+0000","last_became_peered":"2024-05-20T18:27:06.794130+0000","last_unstale":"2024-05-20T18:27:14.049547+0000","last_undegraded":"2024-05-20T18:27:14.049547+0000","last_fullsized":"2024-05-20T18:27:14.049547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:43:31.154358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821030+0000","last_change":"2024-05-20T18:27:12.468521+0000","last_active":"2024-05-20T18:27:13.821030+0000","last_peered":"2024-05-20T18:27:13.821030+0000","last_clean":"2024-05-20T18:27:13.821030+0000","last_became_active":"2024-05-20T18:27:12.467880+0000","last_became_peered":"2024-05-20T18:27:12.467880+0000","last_unstale":"2024-05-20T18:27:13.821030+0000","last_undegraded":"2024-05-20T18:27:13.821030+0000","last_fullsized":"2024-05-20T18:27:13.821030+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:48:16.180009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048938+0000","last_change":"2024-05-20T18:27:07.784489+0000","last_active":"2024-05-20T18:27:14.048938+0000","last_peered":"2024-05-20T18:27:14.048938+0000","last_clean":"2024-05-20T18:27:14.048938+0000","last_became_active":"2024-05-20T18:27:07.783968+0000","last_became_peered":"2024-05-20T18:27:07.783968+0000","last_unstale":"2024-05-20T18:27:14.048938+0000","last_undegraded":"2024-05-20T18:27:14.048938+0000","last_fullsized":"2024-05-20T18:27:14.048938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:07:15.764912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049643+0000","last_change":"2024-05-20T18:27:06.805626+0000","last_active":"2024-05-20T18:27:14.049643+0000","last_peered":"2024-05-20T18:27:14.049643+0000","last_clean":"2024-05-20T18:27:14.049643+0000","last_became_active":"2024-05-20T18:27:06.805267+0000","last_became_peered":"2024-05-20T18:27:06.805267+0000","last_unstale":"2024-05-20T18:27:14.049643+0000","last_undegraded":"2024-05-20T18:27:14.049643+0000","last_fullsized":"2024-05-20T18:27:14.049643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:18:18.528418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048076+0000","last_change":"2024-05-20T18:27:12.465690+0000","last_active":"2024-05-20T18:27:14.048076+0000","last_peered":"2024-05-20T18:27:14.048076+0000","last_clean":"2024-05-20T18:27:14.048076+0000","last_became_active":"2024-05-20T18:27:12.465196+0000","last_became_peered":"2024-05-20T18:27:12.465196+0000","last_unstale":"2024-05-20T18:27:14.048076+0000","last_undegraded":"2024-05-20T18:27:14.048076+0000","last_fullsized":"2024-05-20T18:27:14.048076+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:14:31.209452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825227+0000","last_change":"2024-05-20T18:27:07.806673+0000","last_active":"2024-05-20T18:27:13.825227+0000","last_peered":"2024-05-20T18:27:13.825227+0000","last_clean":"2024-05-20T18:27:13.825227+0000","last_became_active":"2024-05-20T18:27:07.805764+0000","last_became_peered":"2024-05-20T18:27:07.805764+0000","last_unstale":"2024-05-20T18:27:13.825227+0000","last_undegraded":"2024-05-20T18:27:13.825227+0000","last_fullsized":"2024-05-20T18:27:13.825227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:16:37.938695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049063+0000","last_change":"2024-05-20T18:27:06.796398+0000","last_active":"2024-05-20T18:27:14.049063+0000","last_peered":"2024-05-20T18:27:14.049063+0000","last_clean":"2024-05-20T18:27:14.049063+0000","last_became_active":"2024-05-20T18:27:06.795981+0000","last_became_peered":"2024-05-20T18:27:06.795981+0000","last_unstale":"2024-05-20T18:27:14.049063+0000","last_undegraded":"2024-05-20T18:27:14.049063+0000","last_fullsized":"2024-05-20T18:27:14.049063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:11:07.831277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048435+0000","last_change":"2024-05-20T18:27:11.830530+0000","last_active":"2024-05-20T18:27:14.048435+0000","last_peered":"2024-05-20T18:27:14.048435+0000","last_clean":"2024-05-20T18:27:14.048435+0000","last_became_active":"2024-05-20T18:27:11.828707+0000","last_became_peered":"2024-05-20T18:27:11.828707+0000","last_unstale":"2024-05-20T18:27:14.048435+0000","last_undegraded":"2024-05-20T18:27:14.048435+0000","last_fullsized":"2024-05-20T18:27:14.048435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:14:31.209452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048613+0000","last_change":"2024-05-20T18:27:07.802587+0000","last_active":"2024-05-20T18:27:14.048613+0000","last_peered":"2024-05-20T18:27:14.048613+0000","last_clean":"2024-05-20T18:27:14.048613+0000","last_became_active":"2024-05-20T18:27:07.802284+0000","last_became_peered":"2024-05-20T18:27:07.802284+0000","last_unstale":"2024-05-20T18:27:14.048613+0000","last_undegraded":"2024-05-20T18:27:14.048613+0000","last_fullsized":"2024-05-20T18:27:14.048613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:18:19.533923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825123+0000","last_change":"2024-05-20T18:27:06.795278+0000","last_active":"2024-05-20T18:27:13.825123+0000","last_peered":"2024-05-20T18:27:13.825123+0000","last_clean":"2024-05-20T18:27:13.825123+0000","last_became_active":"2024-05-20T18:27:06.794895+0000","last_became_peered":"2024-05-20T18:27:06.794895+0000","last_unstale":"2024-05-20T18:27:13.825123+0000","last_undegraded":"2024-05-20T18:27:13.825123+0000","last_fullsized":"2024-05-20T18:27:13.825123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:52:44.943426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048956+0000","last_change":"2024-05-20T18:27:11.833463+0000","last_active":"2024-05-20T18:27:14.048956+0000","last_peered":"2024-05-20T18:27:14.048956+0000","last_clean":"2024-05-20T18:27:14.048956+0000","last_became_active":"2024-05-20T18:27:11.832954+0000","last_became_peered":"2024-05-20T18:27:11.832954+0000","last_unstale":"2024-05-20T18:27:14.048956+0000","last_undegraded":"2024-05-20T18:27:14.048956+0000","last_fullsized":"2024-05-20T18:27:14.048956+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:34:21.648495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048483+0000","last_change":"2024-05-20T18:27:07.785038+0000","last_active":"2024-05-20T18:27:14.048483+0000","last_peered":"2024-05-20T18:27:14.048483+0000","last_clean":"2024-05-20T18:27:14.048483+0000","last_became_active":"2024-05-20T18:27:07.783927+0000","last_became_peered":"2024-05-20T18:27:07.783927+0000","last_unstale":"2024-05-20T18:27:14.048483+0000","last_undegraded":"2024-05-20T18:27:14.048483+0000","last_fullsized":"2024-05-20T18:27:14.048483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:12.006827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047552+0000","last_change":"2024-05-20T18:27:06.789785+0000","last_active":"2024-05-20T18:27:14.047552+0000","last_peered":"2024-05-20T18:27:14.047552+0000","last_clean":"2024-05-20T18:27:14.047552+0000","last_became_active":"2024-05-20T18:27:06.789421+0000","last_became_peered":"2024-05-20T18:27:06.789421+0000","last_unstale":"2024-05-20T18:27:14.047552+0000","last_undegraded":"2024-05-20T18:27:14.047552+0000","last_fullsized":"2024-05-20T18:27:14.047552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:24:55.001924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049327+0000","last_change":"2024-05-20T18:27:11.833707+0000","last_active":"2024-05-20T18:27:14.049327+0000","last_peered":"2024-05-20T18:27:14.049327+0000","last_clean":"2024-05-20T18:27:14.049327+0000","last_became_active":"2024-05-20T18:27:11.832619+0000","last_became_peered":"2024-05-20T18:27:11.832619+0000","last_unstale":"2024-05-20T18:27:14.049327+0000","last_undegraded":"2024-05-20T18:27:14.049327+0000","last_fullsized":"2024-05-20T18:27:14.049327+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:53:56.893940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047552+0000","last_change":"2024-05-20T18:27:07.799471+0000","last_active":"2024-05-20T18:27:14.047552+0000","last_peered":"2024-05-20T18:27:14.047552+0000","last_clean":"2024-05-20T18:27:14.047552+0000","last_became_active":"2024-05-20T18:27:07.799089+0000","last_became_peered":"2024-05-20T18:27:07.799089+0000","last_unstale":"2024-05-20T18:27:14.047552+0000","last_undegraded":"2024-05-20T18:27:14.047552+0000","last_fullsized":"2024-05-20T18:27:14.047552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:39:20.381356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047626+0000","last_change":"2024-05-20T18:27:06.799374+0000","last_active":"2024-05-20T18:27:14.047626+0000","last_peered":"2024-05-20T18:27:14.047626+0000","last_clean":"2024-05-20T18:27:14.047626+0000","last_became_active":"2024-05-20T18:27:06.798552+0000","last_became_peered":"2024-05-20T18:27:06.798552+0000","last_unstale":"2024-05-20T18:27:14.047626+0000","last_undegraded":"2024-05-20T18:27:14.047626+0000","last_fullsized":"2024-05-20T18:27:14.047626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:52:44.943426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049339+0000","last_change":"2024-05-20T18:27:12.469418+0000","last_active":"2024-05-20T18:27:14.049339+0000","last_peered":"2024-05-20T18:27:14.049339+0000","last_clean":"2024-05-20T18:27:14.049339+0000","last_became_active":"2024-05-20T18:27:12.468923+0000","last_became_peered":"2024-05-20T18:27:12.468923+0000","last_unstale":"2024-05-20T18:27:14.049339+0000","last_undegraded":"2024-05-20T18:27:14.049339+0000","last_fullsized":"2024-05-20T18:27:14.049339+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:55:23.896898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821455+0000","last_change":"2024-05-20T18:27:07.804772+0000","last_active":"2024-05-20T18:27:13.821455+0000","last_peered":"2024-05-20T18:27:13.821455+0000","last_clean":"2024-05-20T18:27:13.821455+0000","last_became_active":"2024-05-20T18:27:07.804443+0000","last_became_peered":"2024-05-20T18:27:07.804443+0000","last_unstale":"2024-05-20T18:27:13.821455+0000","last_undegraded":"2024-05-20T18:27:13.821455+0000","last_fullsized":"2024-05-20T18:27:13.821455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:14:17.791789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824615+0000","last_change":"2024-05-20T18:27:06.790984+0000","last_active":"2024-05-20T18:27:13.824615+0000","last_peered":"2024-05-20T18:27:13.824615+0000","last_clean":"2024-05-20T18:27:13.824615+0000","last_became_active":"2024-05-20T18:27:06.790620+0000","last_became_peered":"2024-05-20T18:27:06.790620+0000","last_unstale":"2024-05-20T18:27:13.824615+0000","last_undegraded":"2024-05-20T18:27:13.824615+0000","last_fullsized":"2024-05-20T18:27:13.824615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:10:27.276400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":179,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047791+0000","last_change":"2024-05-20T18:27:03.765694+0000","last_active":"2024-05-20T18:27:14.047791+0000","last_peered":"2024-05-20T18:27:14.047791+0000","last_clean":"2024-05-20T18:27:14.047791+0000","last_became_active":"2024-05-20T18:27:03.764695+0000","last_became_peered":"2024-05-20T18:27:03.764695+0000","last_unstale":"2024-05-20T18:27:14.047791+0000","last_undegraded":"2024-05-20T18:27:14.047791+0000","last_fullsized":"2024-05-20T18:27:14.047791+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_clean_scrub_stamp":"2024-05-20T18:27:02.750853+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:30:04.407487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824624+0000","last_change":"2024-05-20T18:27:12.467983+0000","last_active":"2024-05-20T18:27:13.824624+0000","last_peered":"2024-05-20T18:27:13.824624+0000","last_clean":"2024-05-20T18:27:13.824624+0000","last_became_active":"2024-05-20T18:27:12.467420+0000","last_became_peered":"2024-05-20T18:27:12.467420+0000","last_unstale":"2024-05-20T18:27:13.824624+0000","last_undegraded":"2024-05-20T18:27:13.824624+0000","last_fullsized":"2024-05-20T18:27:13.824624+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:01:50.652864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049235+0000","last_change":"2024-05-20T18:27:07.814589+0000","last_active":"2024-05-20T18:27:14.049235+0000","last_peered":"2024-05-20T18:27:14.049235+0000","last_clean":"2024-05-20T18:27:14.049235+0000","last_became_active":"2024-05-20T18:27:07.814281+0000","last_became_peered":"2024-05-20T18:27:07.814281+0000","last_unstale":"2024-05-20T18:27:14.049235+0000","last_undegraded":"2024-05-20T18:27:14.049235+0000","last_fullsized":"2024-05-20T18:27:14.049235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:29:33.478403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047483+0000","last_change":"2024-05-20T18:27:06.798075+0000","last_active":"2024-05-20T18:27:14.047483+0000","last_peered":"2024-05-20T18:27:14.047483+0000","last_clean":"2024-05-20T18:27:14.047483+0000","last_became_active":"2024-05-20T18:27:06.797696+0000","last_became_peered":"2024-05-20T18:27:06.797696+0000","last_unstale":"2024-05-20T18:27:14.047483+0000","last_undegraded":"2024-05-20T18:27:14.047483+0000","last_fullsized":"2024-05-20T18:27:14.047483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:06:31.445366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825521+0000","last_change":"2024-05-20T18:27:12.465857+0000","last_active":"2024-05-20T18:27:13.825521+0000","last_peered":"2024-05-20T18:27:13.825521+0000","last_clean":"2024-05-20T18:27:13.825521+0000","last_became_active":"2024-05-20T18:27:12.465296+0000","last_became_peered":"2024-05-20T18:27:12.465296+0000","last_unstale":"2024-05-20T18:27:13.825521+0000","last_undegraded":"2024-05-20T18:27:13.825521+0000","last_fullsized":"2024-05-20T18:27:13.825521+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:25:03.246597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825312+0000","last_change":"2024-05-20T18:27:07.808534+0000","last_active":"2024-05-20T18:27:13.825312+0000","last_peered":"2024-05-20T18:27:13.825312+0000","last_clean":"2024-05-20T18:27:13.825312+0000","last_became_active":"2024-05-20T18:27:07.807813+0000","last_became_peered":"2024-05-20T18:27:07.807813+0000","last_unstale":"2024-05-20T18:27:13.825312+0000","last_undegraded":"2024-05-20T18:27:13.825312+0000","last_fullsized":"2024-05-20T18:27:13.825312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:08:51.463251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049093+0000","last_change":"2024-05-20T18:27:06.793443+0000","last_active":"2024-05-20T18:27:14.049093+0000","last_peered":"2024-05-20T18:27:14.049093+0000","last_clean":"2024-05-20T18:27:14.049093+0000","last_became_active":"2024-05-20T18:27:06.793037+0000","last_became_peered":"2024-05-20T18:27:06.793037+0000","last_unstale":"2024-05-20T18:27:14.049093+0000","last_undegraded":"2024-05-20T18:27:14.049093+0000","last_fullsized":"2024-05-20T18:27:14.049093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:06:31.445366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048364+0000","last_change":"2024-05-20T18:27:11.828038+0000","last_active":"2024-05-20T18:27:14.048364+0000","last_peered":"2024-05-20T18:27:14.048364+0000","last_clean":"2024-05-20T18:27:14.048364+0000","last_became_active":"2024-05-20T18:27:11.827482+0000","last_became_peered":"2024-05-20T18:27:11.827482+0000","last_unstale":"2024-05-20T18:27:14.048364+0000","last_undegraded":"2024-05-20T18:27:14.048364+0000","last_fullsized":"2024-05-20T18:27:14.048364+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:38:58.503399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049301+0000","last_change":"2024-05-20T18:27:07.809779+0000","last_active":"2024-05-20T18:27:14.049301+0000","last_peered":"2024-05-20T18:27:14.049301+0000","last_clean":"2024-05-20T18:27:14.049301+0000","last_became_active":"2024-05-20T18:27:07.809458+0000","last_became_peered":"2024-05-20T18:27:07.809458+0000","last_unstale":"2024-05-20T18:27:14.049301+0000","last_undegraded":"2024-05-20T18:27:14.049301+0000","last_fullsized":"2024-05-20T18:27:14.049301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:55:32.958186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047310+0000","last_change":"2024-05-20T18:27:06.788828+0000","last_active":"2024-05-20T18:27:14.047310+0000","last_peered":"2024-05-20T18:27:14.047310+0000","last_clean":"2024-05-20T18:27:14.047310+0000","last_became_active":"2024-05-20T18:27:06.788142+0000","last_became_peered":"2024-05-20T18:27:06.788142+0000","last_unstale":"2024-05-20T18:27:14.047310+0000","last_undegraded":"2024-05-20T18:27:14.047310+0000","last_fullsized":"2024-05-20T18:27:14.047310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:52:44.943426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.823921+0000","last_change":"2024-05-20T18:27:12.466818+0000","last_active":"2024-05-20T18:27:13.823921+0000","last_peered":"2024-05-20T18:27:13.823921+0000","last_clean":"2024-05-20T18:27:13.823921+0000","last_became_active":"2024-05-20T18:27:12.466228+0000","last_became_peered":"2024-05-20T18:27:12.466228+0000","last_unstale":"2024-05-20T18:27:13.823921+0000","last_undegraded":"2024-05-20T18:27:13.823921+0000","last_fullsized":"2024-05-20T18:27:13.823921+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:55:23.896898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049485+0000","last_change":"2024-05-20T18:27:07.808881+0000","last_active":"2024-05-20T18:27:14.049485+0000","last_peered":"2024-05-20T18:27:14.049485+0000","last_clean":"2024-05-20T18:27:14.049485+0000","last_became_active":"2024-05-20T18:27:07.808550+0000","last_became_peered":"2024-05-20T18:27:07.808550+0000","last_unstale":"2024-05-20T18:27:14.049485+0000","last_undegraded":"2024-05-20T18:27:14.049485+0000","last_fullsized":"2024-05-20T18:27:14.049485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:01:46.623171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048816+0000","last_change":"2024-05-20T18:27:06.797126+0000","last_active":"2024-05-20T18:27:14.048816+0000","last_peered":"2024-05-20T18:27:14.048816+0000","last_clean":"2024-05-20T18:27:14.048816+0000","last_became_active":"2024-05-20T18:27:06.796764+0000","last_became_peered":"2024-05-20T18:27:06.796764+0000","last_unstale":"2024-05-20T18:27:14.048816+0000","last_undegraded":"2024-05-20T18:27:14.048816+0000","last_fullsized":"2024-05-20T18:27:14.048816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:59:18.905646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047675+0000","last_change":"2024-05-20T18:27:11.827350+0000","last_active":"2024-05-20T18:27:14.047675+0000","last_peered":"2024-05-20T18:27:14.047675+0000","last_clean":"2024-05-20T18:27:14.047675+0000","last_became_active":"2024-05-20T18:27:11.826656+0000","last_became_peered":"2024-05-20T18:27:11.826656+0000","last_unstale":"2024-05-20T18:27:14.047675+0000","last_undegraded":"2024-05-20T18:27:14.047675+0000","last_fullsized":"2024-05-20T18:27:14.047675+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:01:50.652864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048684+0000","last_change":"2024-05-20T18:27:07.809732+0000","last_active":"2024-05-20T18:27:14.048684+0000","last_peered":"2024-05-20T18:27:14.048684+0000","last_clean":"2024-05-20T18:27:14.048684+0000","last_became_active":"2024-05-20T18:27:07.809412+0000","last_became_peered":"2024-05-20T18:27:07.809412+0000","last_unstale":"2024-05-20T18:27:14.048684+0000","last_undegraded":"2024-05-20T18:27:14.048684+0000","last_fullsized":"2024-05-20T18:27:14.048684+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:14:03.712607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.046974+0000","last_change":"2024-05-20T18:27:06.787105+0000","last_active":"2024-05-20T18:27:14.046974+0000","last_peered":"2024-05-20T18:27:14.046974+0000","last_clean":"2024-05-20T18:27:14.046974+0000","last_became_active":"2024-05-20T18:27:06.786759+0000","last_became_peered":"2024-05-20T18:27:06.786759+0000","last_unstale":"2024-05-20T18:27:14.046974+0000","last_undegraded":"2024-05-20T18:27:14.046974+0000","last_fullsized":"2024-05-20T18:27:14.046974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:10:27.276400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049140+0000","last_change":"2024-05-20T18:27:11.834554+0000","last_active":"2024-05-20T18:27:14.049140+0000","last_peered":"2024-05-20T18:27:14.049140+0000","last_clean":"2024-05-20T18:27:14.049140+0000","last_became_active":"2024-05-20T18:27:11.833955+0000","last_became_peered":"2024-05-20T18:27:11.833955+0000","last_unstale":"2024-05-20T18:27:14.049140+0000","last_undegraded":"2024-05-20T18:27:14.049140+0000","last_fullsized":"2024-05-20T18:27:14.049140+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:53:56.893940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825845+0000","last_change":"2024-05-20T18:27:07.814023+0000","last_active":"2024-05-20T18:27:13.825845+0000","last_peered":"2024-05-20T18:27:13.825845+0000","last_clean":"2024-05-20T18:27:13.825845+0000","last_became_active":"2024-05-20T18:27:07.813679+0000","last_became_peered":"2024-05-20T18:27:07.813679+0000","last_unstale":"2024-05-20T18:27:13.825845+0000","last_undegraded":"2024-05-20T18:27:13.825845+0000","last_fullsized":"2024-05-20T18:27:13.825845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:10:20.947601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825319+0000","last_change":"2024-05-20T18:27:06.787006+0000","last_active":"2024-05-20T18:27:13.825319+0000","last_peered":"2024-05-20T18:27:13.825319+0000","last_clean":"2024-05-20T18:27:13.825319+0000","last_became_active":"2024-05-20T18:27:06.786565+0000","last_became_peered":"2024-05-20T18:27:06.786565+0000","last_unstale":"2024-05-20T18:27:13.825319+0000","last_undegraded":"2024-05-20T18:27:13.825319+0000","last_fullsized":"2024-05-20T18:27:13.825319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:10:27.276400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.822189+0000","last_change":"2024-05-20T18:27:12.468307+0000","last_active":"2024-05-20T18:27:13.822189+0000","last_peered":"2024-05-20T18:27:13.822189+0000","last_clean":"2024-05-20T18:27:13.822189+0000","last_became_active":"2024-05-20T18:27:12.467856+0000","last_became_peered":"2024-05-20T18:27:12.467856+0000","last_unstale":"2024-05-20T18:27:13.822189+0000","last_undegraded":"2024-05-20T18:27:13.822189+0000","last_fullsized":"2024-05-20T18:27:13.822189+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:08:26.960099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049060+0000","last_change":"2024-05-20T18:27:07.809032+0000","last_active":"2024-05-20T18:27:14.049060+0000","last_peered":"2024-05-20T18:27:14.049060+0000","last_clean":"2024-05-20T18:27:14.049060+0000","last_became_active":"2024-05-20T18:27:07.808708+0000","last_became_peered":"2024-05-20T18:27:07.808708+0000","last_unstale":"2024-05-20T18:27:14.049060+0000","last_undegraded":"2024-05-20T18:27:14.049060+0000","last_fullsized":"2024-05-20T18:27:14.049060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:08:51.463251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825345+0000","last_change":"2024-05-20T18:27:06.794995+0000","last_active":"2024-05-20T18:27:13.825345+0000","last_peered":"2024-05-20T18:27:13.825345+0000","last_clean":"2024-05-20T18:27:13.825345+0000","last_became_active":"2024-05-20T18:27:06.794662+0000","last_became_peered":"2024-05-20T18:27:06.794662+0000","last_unstale":"2024-05-20T18:27:13.825345+0000","last_undegraded":"2024-05-20T18:27:13.825345+0000","last_fullsized":"2024-05-20T18:27:13.825345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:06:31.445366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821770+0000","last_change":"2024-05-20T18:27:12.465482+0000","last_active":"2024-05-20T18:27:13.821770+0000","last_peered":"2024-05-20T18:27:13.821770+0000","last_clean":"2024-05-20T18:27:13.821770+0000","last_became_active":"2024-05-20T18:27:12.464905+0000","last_became_peered":"2024-05-20T18:27:12.464905+0000","last_unstale":"2024-05-20T18:27:13.821770+0000","last_undegraded":"2024-05-20T18:27:13.821770+0000","last_fullsized":"2024-05-20T18:27:13.821770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:16:04.416155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048114+0000","last_change":"2024-05-20T18:27:07.807096+0000","last_active":"2024-05-20T18:27:14.048114+0000","last_peered":"2024-05-20T18:27:14.048114+0000","last_clean":"2024-05-20T18:27:14.048114+0000","last_became_active":"2024-05-20T18:27:07.806625+0000","last_became_peered":"2024-05-20T18:27:07.806625+0000","last_unstale":"2024-05-20T18:27:14.048114+0000","last_undegraded":"2024-05-20T18:27:14.048114+0000","last_fullsized":"2024-05-20T18:27:14.048114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:15:10.022458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048546+0000","last_change":"2024-05-20T18:27:06.785919+0000","last_active":"2024-05-20T18:27:14.048546+0000","last_peered":"2024-05-20T18:27:14.048546+0000","last_clean":"2024-05-20T18:27:14.048546+0000","last_became_active":"2024-05-20T18:27:06.785587+0000","last_became_peered":"2024-05-20T18:27:06.785587+0000","last_unstale":"2024-05-20T18:27:14.048546+0000","last_undegraded":"2024-05-20T18:27:14.048546+0000","last_fullsized":"2024-05-20T18:27:14.048546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:10:44.980209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824622+0000","last_change":"2024-05-20T18:27:12.466586+0000","last_active":"2024-05-20T18:27:13.824622+0000","last_peered":"2024-05-20T18:27:13.824622+0000","last_clean":"2024-05-20T18:27:13.824622+0000","last_became_active":"2024-05-20T18:27:12.466076+0000","last_became_peered":"2024-05-20T18:27:12.466076+0000","last_unstale":"2024-05-20T18:27:13.824622+0000","last_undegraded":"2024-05-20T18:27:13.824622+0000","last_fullsized":"2024-05-20T18:27:13.824622+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:18:23.718914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049203+0000","last_change":"2024-05-20T18:27:07.805541+0000","last_active":"2024-05-20T18:27:14.049203+0000","last_peered":"2024-05-20T18:27:14.049203+0000","last_clean":"2024-05-20T18:27:14.049203+0000","last_became_active":"2024-05-20T18:27:07.805223+0000","last_became_peered":"2024-05-20T18:27:07.805223+0000","last_unstale":"2024-05-20T18:27:14.049203+0000","last_undegraded":"2024-05-20T18:27:14.049203+0000","last_fullsized":"2024-05-20T18:27:14.049203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:52:22.903884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":33,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:17.511072+0000","last_change":"2024-05-20T18:27:06.788980+0000","last_active":"2024-05-20T18:27:17.511072+0000","last_peered":"2024-05-20T18:27:17.511072+0000","last_clean":"2024-05-20T18:27:17.511072+0000","last_became_active":"2024-05-20T18:27:06.788477+0000","last_became_peered":"2024-05-20T18:27:06.788477+0000","last_unstale":"2024-05-20T18:27:17.511072+0000","last_undegraded":"2024-05-20T18:27:17.511072+0000","last_fullsized":"2024-05-20T18:27:17.511072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:43:31.154358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048876+0000","last_change":"2024-05-20T18:27:11.829159+0000","last_active":"2024-05-20T18:27:14.048876+0000","last_peered":"2024-05-20T18:27:14.048876+0000","last_clean":"2024-05-20T18:27:14.048876+0000","last_became_active":"2024-05-20T18:27:11.826918+0000","last_became_peered":"2024-05-20T18:27:11.826918+0000","last_unstale":"2024-05-20T18:27:14.048876+0000","last_undegraded":"2024-05-20T18:27:14.048876+0000","last_fullsized":"2024-05-20T18:27:14.048876+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:38:58.503399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048691+0000","last_change":"2024-05-20T18:27:07.791122+0000","last_active":"2024-05-20T18:27:14.048691+0000","last_peered":"2024-05-20T18:27:14.048691+0000","last_clean":"2024-05-20T18:27:14.048691+0000","last_became_active":"2024-05-20T18:27:07.790777+0000","last_became_peered":"2024-05-20T18:27:07.790777+0000","last_unstale":"2024-05-20T18:27:14.048691+0000","last_undegraded":"2024-05-20T18:27:14.048691+0000","last_fullsized":"2024-05-20T18:27:14.048691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:11:08.836782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048983+0000","last_change":"2024-05-20T18:27:06.799123+0000","last_active":"2024-05-20T18:27:14.048983+0000","last_peered":"2024-05-20T18:27:14.048983+0000","last_clean":"2024-05-20T18:27:14.048983+0000","last_became_active":"2024-05-20T18:27:06.798681+0000","last_became_peered":"2024-05-20T18:27:06.798681+0000","last_unstale":"2024-05-20T18:27:14.048983+0000","last_undegraded":"2024-05-20T18:27:14.048983+0000","last_fullsized":"2024-05-20T18:27:14.048983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:18:18.528418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048322+0000","last_change":"2024-05-20T18:27:11.823730+0000","last_active":"2024-05-20T18:27:14.048322+0000","last_peered":"2024-05-20T18:27:14.048322+0000","last_clean":"2024-05-20T18:27:14.048322+0000","last_became_active":"2024-05-20T18:27:11.823226+0000","last_became_peered":"2024-05-20T18:27:11.823226+0000","last_unstale":"2024-05-20T18:27:14.048322+0000","last_undegraded":"2024-05-20T18:27:14.048322+0000","last_fullsized":"2024-05-20T18:27:14.048322+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:14:31.209452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048598+0000","last_change":"2024-05-20T18:27:07.804871+0000","last_active":"2024-05-20T18:27:14.048598+0000","last_peered":"2024-05-20T18:27:14.048598+0000","last_clean":"2024-05-20T18:27:14.048598+0000","last_became_active":"2024-05-20T18:27:07.804518+0000","last_became_peered":"2024-05-20T18:27:07.804518+0000","last_unstale":"2024-05-20T18:27:14.048598+0000","last_undegraded":"2024-05-20T18:27:14.048598+0000","last_fullsized":"2024-05-20T18:27:14.048598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:14:03.712607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824274+0000","last_change":"2024-05-20T18:27:06.795936+0000","last_active":"2024-05-20T18:27:13.824274+0000","last_peered":"2024-05-20T18:27:13.824274+0000","last_clean":"2024-05-20T18:27:13.824274+0000","last_became_active":"2024-05-20T18:27:06.795616+0000","last_became_peered":"2024-05-20T18:27:06.795616+0000","last_unstale":"2024-05-20T18:27:13.824274+0000","last_undegraded":"2024-05-20T18:27:13.824274+0000","last_fullsized":"2024-05-20T18:27:13.824274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:24:55.001924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821242+0000","last_change":"2024-05-20T18:27:11.827063+0000","last_active":"2024-05-20T18:27:13.821242+0000","last_peered":"2024-05-20T18:27:13.821242+0000","last_clean":"2024-05-20T18:27:13.821242+0000","last_became_active":"2024-05-20T18:27:11.825223+0000","last_became_peered":"2024-05-20T18:27:11.825223+0000","last_unstale":"2024-05-20T18:27:13.821242+0000","last_undegraded":"2024-05-20T18:27:13.821242+0000","last_fullsized":"2024-05-20T18:27:13.821242+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:01:18.112337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824776+0000","last_change":"2024-05-20T18:27:07.810225+0000","last_active":"2024-05-20T18:27:13.824776+0000","last_peered":"2024-05-20T18:27:13.824776+0000","last_clean":"2024-05-20T18:27:13.824776+0000","last_became_active":"2024-05-20T18:27:07.809511+0000","last_became_peered":"2024-05-20T18:27:07.809511+0000","last_unstale":"2024-05-20T18:27:13.824776+0000","last_undegraded":"2024-05-20T18:27:13.824776+0000","last_fullsized":"2024-05-20T18:27:13.824776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:24:59.216904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.826020+0000","last_change":"2024-05-20T18:27:06.796334+0000","last_active":"2024-05-20T18:27:13.826020+0000","last_peered":"2024-05-20T18:27:13.826020+0000","last_clean":"2024-05-20T18:27:13.826020+0000","last_became_active":"2024-05-20T18:27:06.795897+0000","last_became_peered":"2024-05-20T18:27:06.795897+0000","last_unstale":"2024-05-20T18:27:13.826020+0000","last_undegraded":"2024-05-20T18:27:13.826020+0000","last_fullsized":"2024-05-20T18:27:13.826020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:06:31.445366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821208+0000","last_change":"2024-05-20T18:27:12.465588+0000","last_active":"2024-05-20T18:27:13.821208+0000","last_peered":"2024-05-20T18:27:13.821208+0000","last_clean":"2024-05-20T18:27:13.821208+0000","last_became_active":"2024-05-20T18:27:12.465170+0000","last_became_peered":"2024-05-20T18:27:12.465170+0000","last_unstale":"2024-05-20T18:27:13.821208+0000","last_undegraded":"2024-05-20T18:27:13.821208+0000","last_fullsized":"2024-05-20T18:27:13.821208+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:36:54.997159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048936+0000","last_change":"2024-05-20T18:27:12.469946+0000","last_active":"2024-05-20T18:27:14.048936+0000","last_peered":"2024-05-20T18:27:14.048936+0000","last_clean":"2024-05-20T18:27:14.048936+0000","last_became_active":"2024-05-20T18:27:12.469526+0000","last_became_peered":"2024-05-20T18:27:12.469526+0000","last_unstale":"2024-05-20T18:27:14.048936+0000","last_undegraded":"2024-05-20T18:27:14.048936+0000","last_fullsized":"2024-05-20T18:27:14.048936+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:34:21.648495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048409+0000","last_change":"2024-05-20T18:27:07.788636+0000","last_active":"2024-05-20T18:27:14.048409+0000","last_peered":"2024-05-20T18:27:14.048409+0000","last_clean":"2024-05-20T18:27:14.048409+0000","last_became_active":"2024-05-20T18:27:07.788307+0000","last_became_peered":"2024-05-20T18:27:07.788307+0000","last_unstale":"2024-05-20T18:27:14.048409+0000","last_undegraded":"2024-05-20T18:27:14.048409+0000","last_fullsized":"2024-05-20T18:27:14.048409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:10:20.947601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824264+0000","last_change":"2024-05-20T18:27:06.797781+0000","last_active":"2024-05-20T18:27:13.824264+0000","last_peered":"2024-05-20T18:27:13.824264+0000","last_clean":"2024-05-20T18:27:13.824264+0000","last_became_active":"2024-05-20T18:27:06.797415+0000","last_became_peered":"2024-05-20T18:27:06.797415+0000","last_unstale":"2024-05-20T18:27:13.824264+0000","last_undegraded":"2024-05-20T18:27:13.824264+0000","last_fullsized":"2024-05-20T18:27:13.824264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:18:18.528418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825975+0000","last_change":"2024-05-20T18:27:11.827561+0000","last_active":"2024-05-20T18:27:13.825975+0000","last_peered":"2024-05-20T18:27:13.825975+0000","last_clean":"2024-05-20T18:27:13.825975+0000","last_became_active":"2024-05-20T18:27:11.826925+0000","last_became_peered":"2024-05-20T18:27:11.826925+0000","last_unstale":"2024-05-20T18:27:13.825975+0000","last_undegraded":"2024-05-20T18:27:13.825975+0000","last_fullsized":"2024-05-20T18:27:13.825975+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:16:41.968388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821278+0000","last_change":"2024-05-20T18:27:07.789976+0000","last_active":"2024-05-20T18:27:13.821278+0000","last_peered":"2024-05-20T18:27:13.821278+0000","last_clean":"2024-05-20T18:27:13.821278+0000","last_became_active":"2024-05-20T18:27:07.789679+0000","last_became_peered":"2024-05-20T18:27:07.789679+0000","last_unstale":"2024-05-20T18:27:13.821278+0000","last_undegraded":"2024-05-20T18:27:13.821278+0000","last_fullsized":"2024-05-20T18:27:13.821278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:52:57.891709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821366+0000","last_change":"2024-05-20T18:27:06.794988+0000","last_active":"2024-05-20T18:27:13.821366+0000","last_peered":"2024-05-20T18:27:13.821366+0000","last_clean":"2024-05-20T18:27:13.821366+0000","last_became_active":"2024-05-20T18:27:06.794630+0000","last_became_peered":"2024-05-20T18:27:06.794630+0000","last_unstale":"2024-05-20T18:27:13.821366+0000","last_undegraded":"2024-05-20T18:27:13.821366+0000","last_fullsized":"2024-05-20T18:27:13.821366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:24:55.001924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825689+0000","last_change":"2024-05-20T18:27:06.785041+0000","last_active":"2024-05-20T18:27:13.825689+0000","last_peered":"2024-05-20T18:27:13.825689+0000","last_clean":"2024-05-20T18:27:13.825689+0000","last_became_active":"2024-05-20T18:27:06.784673+0000","last_became_peered":"2024-05-20T18:27:06.784673+0000","last_unstale":"2024-05-20T18:27:13.825689+0000","last_undegraded":"2024-05-20T18:27:13.825689+0000","last_fullsized":"2024-05-20T18:27:13.825689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:52:44.943426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048466+0000","last_change":"2024-05-20T18:27:07.790855+0000","last_active":"2024-05-20T18:27:14.048466+0000","last_peered":"2024-05-20T18:27:14.048466+0000","last_clean":"2024-05-20T18:27:14.048466+0000","last_became_active":"2024-05-20T18:27:07.790370+0000","last_became_peered":"2024-05-20T18:27:07.790370+0000","last_unstale":"2024-05-20T18:27:14.048466+0000","last_undegraded":"2024-05-20T18:27:14.048466+0000","last_fullsized":"2024-05-20T18:27:14.048466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:07:15.764912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047264+0000","last_change":"2024-05-20T18:27:11.828935+0000","last_active":"2024-05-20T18:27:14.047264+0000","last_peered":"2024-05-20T18:27:14.047264+0000","last_clean":"2024-05-20T18:27:14.047264+0000","last_became_active":"2024-05-20T18:27:11.828516+0000","last_became_peered":"2024-05-20T18:27:11.828516+0000","last_unstale":"2024-05-20T18:27:14.047264+0000","last_undegraded":"2024-05-20T18:27:14.047264+0000","last_fullsized":"2024-05-20T18:27:14.047264+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:15:14.052151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821617+0000","last_change":"2024-05-20T18:27:06.803606+0000","last_active":"2024-05-20T18:27:13.821617+0000","last_peered":"2024-05-20T18:27:13.821617+0000","last_clean":"2024-05-20T18:27:13.821617+0000","last_became_active":"2024-05-20T18:27:06.803340+0000","last_became_peered":"2024-05-20T18:27:06.803340+0000","last_unstale":"2024-05-20T18:27:13.821617+0000","last_undegraded":"2024-05-20T18:27:13.821617+0000","last_fullsized":"2024-05-20T18:27:13.821617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:10:27.276400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048051+0000","last_change":"2024-05-20T18:27:07.789834+0000","last_active":"2024-05-20T18:27:14.048051+0000","last_peered":"2024-05-20T18:27:14.048051+0000","last_clean":"2024-05-20T18:27:14.048051+0000","last_became_active":"2024-05-20T18:27:07.789472+0000","last_became_peered":"2024-05-20T18:27:07.789472+0000","last_unstale":"2024-05-20T18:27:14.048051+0000","last_undegraded":"2024-05-20T18:27:14.048051+0000","last_fullsized":"2024-05-20T18:27:14.048051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:52:45.948931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821874+0000","last_change":"2024-05-20T18:27:11.825859+0000","last_active":"2024-05-20T18:27:13.821874+0000","last_peered":"2024-05-20T18:27:13.821874+0000","last_clean":"2024-05-20T18:27:13.821874+0000","last_became_active":"2024-05-20T18:27:11.825093+0000","last_became_peered":"2024-05-20T18:27:11.825093+0000","last_unstale":"2024-05-20T18:27:13.821874+0000","last_undegraded":"2024-05-20T18:27:13.821874+0000","last_fullsized":"2024-05-20T18:27:13.821874+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:02:46.122485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825718+0000","last_change":"2024-05-20T18:27:07.793263+0000","last_active":"2024-05-20T18:27:13.825718+0000","last_peered":"2024-05-20T18:27:13.825718+0000","last_clean":"2024-05-20T18:27:13.825718+0000","last_became_active":"2024-05-20T18:27:07.792916+0000","last_became_peered":"2024-05-20T18:27:07.792916+0000","last_unstale":"2024-05-20T18:27:13.825718+0000","last_undegraded":"2024-05-20T18:27:13.825718+0000","last_fullsized":"2024-05-20T18:27:13.825718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:15:10.022458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047433+0000","last_change":"2024-05-20T18:27:07.805541+0000","last_active":"2024-05-20T18:27:14.047433+0000","last_peered":"2024-05-20T18:27:14.047433+0000","last_clean":"2024-05-20T18:27:14.047433+0000","last_became_active":"2024-05-20T18:27:07.805160+0000","last_became_peered":"2024-05-20T18:27:07.805160+0000","last_unstale":"2024-05-20T18:27:14.047433+0000","last_undegraded":"2024-05-20T18:27:14.047433+0000","last_fullsized":"2024-05-20T18:27:14.047433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:14:23.046328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824975+0000","last_change":"2024-05-20T18:27:07.804547+0000","last_active":"2024-05-20T18:27:13.824975+0000","last_peered":"2024-05-20T18:27:13.824975+0000","last_clean":"2024-05-20T18:27:13.824975+0000","last_became_active":"2024-05-20T18:27:07.804242+0000","last_became_peered":"2024-05-20T18:27:07.804242+0000","last_unstale":"2024-05-20T18:27:13.824975+0000","last_undegraded":"2024-05-20T18:27:13.824975+0000","last_fullsized":"2024-05-20T18:27:13.824975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:55:24.560518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048436+0000","last_change":"2024-05-20T18:27:07.790683+0000","last_active":"2024-05-20T18:27:14.048436+0000","last_peered":"2024-05-20T18:27:14.048436+0000","last_clean":"2024-05-20T18:27:14.048436+0000","last_became_active":"2024-05-20T18:27:07.789978+0000","last_became_peered":"2024-05-20T18:27:07.789978+0000","last_unstale":"2024-05-20T18:27:14.048436+0000","last_undegraded":"2024-05-20T18:27:14.048436+0000","last_fullsized":"2024-05-20T18:27:14.048436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:40:08.698691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821852+0000","last_change":"2024-05-20T18:27:07.791924+0000","last_active":"2024-05-20T18:27:13.821852+0000","last_peered":"2024-05-20T18:27:13.821852+0000","last_clean":"2024-05-20T18:27:13.821852+0000","last_became_active":"2024-05-20T18:27:07.791622+0000","last_became_peered":"2024-05-20T18:27:07.791622+0000","last_unstale":"2024-05-20T18:27:13.821852+0000","last_undegraded":"2024-05-20T18:27:13.821852+0000","last_fullsized":"2024-05-20T18:27:13.821852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:07:15.764912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048748+0000","last_change":"2024-05-20T18:27:07.789116+0000","last_active":"2024-05-20T18:27:14.048748+0000","last_peered":"2024-05-20T18:27:14.048748+0000","last_clean":"2024-05-20T18:27:14.048748+0000","last_became_active":"2024-05-20T18:27:07.787855+0000","last_became_peered":"2024-05-20T18:27:07.787855+0000","last_unstale":"2024-05-20T18:27:14.048748+0000","last_undegraded":"2024-05-20T18:27:14.048748+0000","last_fullsized":"2024-05-20T18:27:14.048748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:52:57.891709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825914+0000","last_change":"2024-05-20T18:27:07.793074+0000","last_active":"2024-05-20T18:27:13.825914+0000","last_peered":"2024-05-20T18:27:13.825914+0000","last_clean":"2024-05-20T18:27:13.825914+0000","last_became_active":"2024-05-20T18:27:07.792754+0000","last_became_peered":"2024-05-20T18:27:07.792754+0000","last_unstale":"2024-05-20T18:27:13.825914+0000","last_undegraded":"2024-05-20T18:27:13.825914+0000","last_fullsized":"2024-05-20T18:27:13.825914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:12.006827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.822159+0000","last_change":"2024-05-20T18:27:07.791208+0000","last_active":"2024-05-20T18:27:13.822159+0000","last_peered":"2024-05-20T18:27:13.822159+0000","last_clean":"2024-05-20T18:27:13.822159+0000","last_became_active":"2024-05-20T18:27:07.790864+0000","last_became_peered":"2024-05-20T18:27:07.790864+0000","last_unstale":"2024-05-20T18:27:13.822159+0000","last_undegraded":"2024-05-20T18:27:13.822159+0000","last_fullsized":"2024-05-20T18:27:13.822159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:19:31.649320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824356+0000","last_change":"2024-05-20T18:27:07.808305+0000","last_active":"2024-05-20T18:27:13.824356+0000","last_peered":"2024-05-20T18:27:13.824356+0000","last_clean":"2024-05-20T18:27:13.824356+0000","last_became_active":"2024-05-20T18:27:07.807877+0000","last_became_peered":"2024-05-20T18:27:07.807877+0000","last_unstale":"2024-05-20T18:27:13.824356+0000","last_undegraded":"2024-05-20T18:27:13.824356+0000","last_fullsized":"2024-05-20T18:27:13.824356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:14:03.712607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821322+0000","last_change":"2024-05-20T18:27:07.791872+0000","last_active":"2024-05-20T18:27:13.821322+0000","last_peered":"2024-05-20T18:27:13.821322+0000","last_clean":"2024-05-20T18:27:13.821322+0000","last_became_active":"2024-05-20T18:27:07.791570+0000","last_became_peered":"2024-05-20T18:27:07.791570+0000","last_unstale":"2024-05-20T18:27:13.821322+0000","last_undegraded":"2024-05-20T18:27:13.821322+0000","last_fullsized":"2024-05-20T18:27:13.821322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:29:33.478403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824943+0000","last_change":"2024-05-20T18:27:07.811057+0000","last_active":"2024-05-20T18:27:13.824943+0000","last_peered":"2024-05-20T18:27:13.824943+0000","last_clean":"2024-05-20T18:27:13.824943+0000","last_became_active":"2024-05-20T18:27:07.810728+0000","last_became_peered":"2024-05-20T18:27:07.810728+0000","last_unstale":"2024-05-20T18:27:13.824943+0000","last_undegraded":"2024-05-20T18:27:13.824943+0000","last_fullsized":"2024-05-20T18:27:13.824943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:19:31.649320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048182+0000","last_change":"2024-05-20T18:27:07.795559+0000","last_active":"2024-05-20T18:27:14.048182+0000","last_peered":"2024-05-20T18:27:14.048182+0000","last_clean":"2024-05-20T18:27:14.048182+0000","last_became_active":"2024-05-20T18:27:07.795228+0000","last_became_peered":"2024-05-20T18:27:07.795228+0000","last_unstale":"2024-05-20T18:27:14.048182+0000","last_undegraded":"2024-05-20T18:27:14.048182+0000","last_fullsized":"2024-05-20T18:27:14.048182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:10:20.947601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.826124+0000","last_change":"2024-05-20T18:27:07.795052+0000","last_active":"2024-05-20T18:27:13.826124+0000","last_peered":"2024-05-20T18:27:13.826124+0000","last_clean":"2024-05-20T18:27:13.826124+0000","last_became_active":"2024-05-20T18:27:07.794704+0000","last_became_peered":"2024-05-20T18:27:07.794704+0000","last_unstale":"2024-05-20T18:27:13.826124+0000","last_undegraded":"2024-05-20T18:27:13.826124+0000","last_fullsized":"2024-05-20T18:27:13.826124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:34:15.614124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049515+0000","last_change":"2024-05-20T18:27:07.807441+0000","last_active":"2024-05-20T18:27:14.049515+0000","last_peered":"2024-05-20T18:27:14.049515+0000","last_clean":"2024-05-20T18:27:14.049515+0000","last_became_active":"2024-05-20T18:27:07.807095+0000","last_became_peered":"2024-05-20T18:27:07.807095+0000","last_unstale":"2024-05-20T18:27:14.049515+0000","last_undegraded":"2024-05-20T18:27:14.049515+0000","last_fullsized":"2024-05-20T18:27:14.049515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:24:59.216904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047301+0000","last_change":"2024-05-20T18:27:07.801623+0000","last_active":"2024-05-20T18:27:14.047301+0000","last_peered":"2024-05-20T18:27:14.047301+0000","last_clean":"2024-05-20T18:27:14.047301+0000","last_became_active":"2024-05-20T18:27:07.801303+0000","last_became_peered":"2024-05-20T18:27:07.801303+0000","last_unstale":"2024-05-20T18:27:14.047301+0000","last_undegraded":"2024-05-20T18:27:14.047301+0000","last_fullsized":"2024-05-20T18:27:14.047301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:29:19.667394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824621+0000","last_change":"2024-05-20T18:27:07.809861+0000","last_active":"2024-05-20T18:27:13.824621+0000","last_peered":"2024-05-20T18:27:13.824621+0000","last_clean":"2024-05-20T18:27:13.824621+0000","last_became_active":"2024-05-20T18:27:07.809523+0000","last_became_peered":"2024-05-20T18:27:07.809523+0000","last_unstale":"2024-05-20T18:27:13.824621+0000","last_undegraded":"2024-05-20T18:27:13.824621+0000","last_fullsized":"2024-05-20T18:27:13.824621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:55:32.958186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.823933+0000","last_change":"2024-05-20T18:27:07.814234+0000","last_active":"2024-05-20T18:27:13.823933+0000","last_peered":"2024-05-20T18:27:13.823933+0000","last_clean":"2024-05-20T18:27:13.823933+0000","last_became_active":"2024-05-20T18:27:07.813168+0000","last_became_peered":"2024-05-20T18:27:07.813168+0000","last_unstale":"2024-05-20T18:27:13.823933+0000","last_undegraded":"2024-05-20T18:27:13.823933+0000","last_fullsized":"2024-05-20T18:27:13.823933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:12.006827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048520+0000","last_change":"2024-05-20T18:27:07.787849+0000","last_active":"2024-05-20T18:27:14.048520+0000","last_peered":"2024-05-20T18:27:14.048520+0000","last_clean":"2024-05-20T18:27:14.048520+0000","last_became_active":"2024-05-20T18:27:07.787406+0000","last_became_peered":"2024-05-20T18:27:07.787406+0000","last_unstale":"2024-05-20T18:27:14.048520+0000","last_undegraded":"2024-05-20T18:27:14.048520+0000","last_fullsized":"2024-05-20T18:27:14.048520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:34:15.614124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047987+0000","last_change":"2024-05-20T18:27:07.795141+0000","last_active":"2024-05-20T18:27:14.047987+0000","last_peered":"2024-05-20T18:27:14.047987+0000","last_clean":"2024-05-20T18:27:14.047987+0000","last_became_active":"2024-05-20T18:27:07.793487+0000","last_became_peered":"2024-05-20T18:27:07.793487+0000","last_unstale":"2024-05-20T18:27:14.047987+0000","last_undegraded":"2024-05-20T18:27:14.047987+0000","last_fullsized":"2024-05-20T18:27:14.047987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:32:42.769509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821496+0000","last_change":"2024-05-20T18:27:07.802751+0000","last_active":"2024-05-20T18:27:13.821496+0000","last_peered":"2024-05-20T18:27:13.821496+0000","last_clean":"2024-05-20T18:27:13.821496+0000","last_became_active":"2024-05-20T18:27:07.802412+0000","last_became_peered":"2024-05-20T18:27:07.802412+0000","last_unstale":"2024-05-20T18:27:13.821496+0000","last_undegraded":"2024-05-20T18:27:13.821496+0000","last_fullsized":"2024-05-20T18:27:13.821496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:14:03.712607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048564+0000","last_change":"2024-05-20T18:27:07.796408+0000","last_active":"2024-05-20T18:27:14.048564+0000","last_peered":"2024-05-20T18:27:14.048564+0000","last_clean":"2024-05-20T18:27:14.048564+0000","last_became_active":"2024-05-20T18:27:07.795961+0000","last_became_peered":"2024-05-20T18:27:07.795961+0000","last_unstale":"2024-05-20T18:27:14.048564+0000","last_undegraded":"2024-05-20T18:27:14.048564+0000","last_fullsized":"2024-05-20T18:27:14.048564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:43:32.159862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824841+0000","last_change":"2024-05-20T18:27:07.806684+0000","last_active":"2024-05-20T18:27:13.824841+0000","last_peered":"2024-05-20T18:27:13.824841+0000","last_clean":"2024-05-20T18:27:13.824841+0000","last_became_active":"2024-05-20T18:27:07.806320+0000","last_became_peered":"2024-05-20T18:27:07.806320+0000","last_unstale":"2024-05-20T18:27:13.824841+0000","last_undegraded":"2024-05-20T18:27:13.824841+0000","last_fullsized":"2024-05-20T18:27:13.824841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:52:57.891709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825792+0000","last_change":"2024-05-20T18:27:07.805587+0000","last_active":"2024-05-20T18:27:13.825792+0000","last_peered":"2024-05-20T18:27:13.825792+0000","last_clean":"2024-05-20T18:27:13.825792+0000","last_became_active":"2024-05-20T18:27:07.804971+0000","last_became_peered":"2024-05-20T18:27:07.804971+0000","last_unstale":"2024-05-20T18:27:13.825792+0000","last_undegraded":"2024-05-20T18:27:13.825792+0000","last_fullsized":"2024-05-20T18:27:13.825792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:07:15.764912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049422+0000","last_change":"2024-05-20T18:27:07.815867+0000","last_active":"2024-05-20T18:27:14.049422+0000","last_peered":"2024-05-20T18:27:14.049422+0000","last_clean":"2024-05-20T18:27:14.049422+0000","last_became_active":"2024-05-20T18:27:07.815421+0000","last_became_peered":"2024-05-20T18:27:07.815421+0000","last_unstale":"2024-05-20T18:27:14.049422+0000","last_undegraded":"2024-05-20T18:27:14.049422+0000","last_fullsized":"2024-05-20T18:27:14.049422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:16:37.938695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821166+0000","last_change":"2024-05-20T18:27:07.810206+0000","last_active":"2024-05-20T18:27:13.821166+0000","last_peered":"2024-05-20T18:27:13.821166+0000","last_clean":"2024-05-20T18:27:13.821166+0000","last_became_active":"2024-05-20T18:27:07.809846+0000","last_became_peered":"2024-05-20T18:27:07.809846+0000","last_unstale":"2024-05-20T18:27:13.821166+0000","last_undegraded":"2024-05-20T18:27:13.821166+0000","last_fullsized":"2024-05-20T18:27:13.821166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:10:20.947601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048623+0000","last_change":"2024-05-20T18:27:07.813420+0000","last_active":"2024-05-20T18:27:14.048623+0000","last_peered":"2024-05-20T18:27:14.048623+0000","last_clean":"2024-05-20T18:27:14.048623+0000","last_became_active":"2024-05-20T18:27:07.813093+0000","last_became_peered":"2024-05-20T18:27:07.813093+0000","last_unstale":"2024-05-20T18:27:14.048623+0000","last_undegraded":"2024-05-20T18:27:14.048623+0000","last_fullsized":"2024-05-20T18:27:14.048623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:19:31.649320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824831+0000","last_change":"2024-05-20T18:27:07.812561+0000","last_active":"2024-05-20T18:27:13.824831+0000","last_peered":"2024-05-20T18:27:13.824831+0000","last_clean":"2024-05-20T18:27:13.824831+0000","last_became_active":"2024-05-20T18:27:07.812264+0000","last_became_peered":"2024-05-20T18:27:07.812264+0000","last_unstale":"2024-05-20T18:27:13.824831+0000","last_undegraded":"2024-05-20T18:27:13.824831+0000","last_fullsized":"2024-05-20T18:27:13.824831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:52:22.903884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048783+0000","last_change":"2024-05-20T18:27:07.804441+0000","last_active":"2024-05-20T18:27:14.048783+0000","last_peered":"2024-05-20T18:27:14.048783+0000","last_clean":"2024-05-20T18:27:14.048783+0000","last_became_active":"2024-05-20T18:27:07.804126+0000","last_became_peered":"2024-05-20T18:27:07.804126+0000","last_unstale":"2024-05-20T18:27:14.048783+0000","last_undegraded":"2024-05-20T18:27:14.048783+0000","last_fullsized":"2024-05-20T18:27:14.048783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:14:17.791789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825082+0000","last_change":"2024-05-20T18:27:07.808819+0000","last_active":"2024-05-20T18:27:13.825082+0000","last_peered":"2024-05-20T18:27:13.825082+0000","last_clean":"2024-05-20T18:27:13.825082+0000","last_became_active":"2024-05-20T18:27:07.808231+0000","last_became_peered":"2024-05-20T18:27:07.808231+0000","last_unstale":"2024-05-20T18:27:13.825082+0000","last_undegraded":"2024-05-20T18:27:13.825082+0000","last_fullsized":"2024-05-20T18:27:13.825082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:14:17.791789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048097+0000","last_change":"2024-05-20T18:27:07.793960+0000","last_active":"2024-05-20T18:27:14.048097+0000","last_peered":"2024-05-20T18:27:14.048097+0000","last_clean":"2024-05-20T18:27:14.048097+0000","last_became_active":"2024-05-20T18:27:07.793594+0000","last_became_peered":"2024-05-20T18:27:07.793594+0000","last_unstale":"2024-05-20T18:27:14.048097+0000","last_undegraded":"2024-05-20T18:27:14.048097+0000","last_fullsized":"2024-05-20T18:27:14.048097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:34:15.614124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049410+0000","last_change":"2024-05-20T18:27:07.790606+0000","last_active":"2024-05-20T18:27:14.049410+0000","last_peered":"2024-05-20T18:27:14.049410+0000","last_clean":"2024-05-20T18:27:14.049410+0000","last_became_active":"2024-05-20T18:27:07.790227+0000","last_became_peered":"2024-05-20T18:27:07.790227+0000","last_unstale":"2024-05-20T18:27:14.049410+0000","last_undegraded":"2024-05-20T18:27:14.049410+0000","last_fullsized":"2024-05-20T18:27:14.049410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:14:03.712607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048973+0000","last_change":"2024-05-20T18:27:07.816207+0000","last_active":"2024-05-20T18:27:14.048973+0000","last_peered":"2024-05-20T18:27:14.048973+0000","last_clean":"2024-05-20T18:27:14.048973+0000","last_became_active":"2024-05-20T18:27:07.815556+0000","last_became_peered":"2024-05-20T18:27:07.815556+0000","last_unstale":"2024-05-20T18:27:14.048973+0000","last_undegraded":"2024-05-20T18:27:14.048973+0000","last_fullsized":"2024-05-20T18:27:14.048973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:55:24.560518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":876544,"data_stored":703014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":115906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":112154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":112190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":290816,"data_stored":115906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":278528,"data_stored":112064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":286720,"data_stored":112224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":884736,"data_stored":703070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T18:27:31.340 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-20T18:27:31.340 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-20T18:27:31.340 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-20T18:27:31.341 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-20T18:27:31.341 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-20T18:27:31.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.490+0000 7ff3ba13e640 1 Processor -- start 2024-05-20T18:27:31.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.490+0000 7ff3ba13e640 1 -- start start 2024-05-20T18:27:31.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.492+0000 7ff3ba13e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff3b40b6620 0x7ff3b40ba3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:31.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.492+0000 7ff3ba13e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff3b40d3bf0 0x7ff3b40d4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:31.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.492+0000 7ff3ba13e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3b40d4540 con 0x7ff3b4162ce0 2024-05-20T18:27:31.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.492+0000 7ff3ba13e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7ff3b40d3480 con 0x7ff3b40b6620 2024-05-20T18:27:31.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.492+0000 7ff3ba13e640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff3b41634a0 con 0x7ff3b40d3bf0 2024-05-20T18:27:31.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.492+0000 7ff3b37fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff3b40b6620 0x7ff3b40ba3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:31.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.492+0000 7ff3b37fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff3b40b6620 0x7ff3b40ba3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59008/0 (socket says 172.21.15.5:59008) 2024-05-20T18:27:31.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.492+0000 7ff3b37fe640 1 -- 172.21.15.5:0/4181170561 learned_addr learned my addr 172.21.15.5:0/4181170561 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:31.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.492+0000 7ff3b3fff640 1 --2- 172.21.15.5:0/4181170561 >> v2:172.21.15.63:3300/0 conn(0x7ff3b40d3bf0 0x7ff3b40d4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:31.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.492+0000 7ff3b37fe640 1 -- 172.21.15.5:0/4181170561 >> v2:172.21.15.63:3300/0 conn(0x7ff3b40d3bf0 msgr2=0x7ff3b40d4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:31.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.492+0000 7ff3b37fe640 1 --2- 172.21.15.5:0/4181170561 >> v2:172.21.15.63:3300/0 conn(0x7ff3b40d3bf0 0x7ff3b40d4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:31.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.492+0000 7ff3b37fe640 1 -- 172.21.15.5:0/4181170561 >> v1:172.21.15.63:6789/0 conn(0x7ff3b4162ce0 legacy=0x7ff3b40b8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:31.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.492+0000 7ff3b37fe640 1 -- 172.21.15.5:0/4181170561 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3b4163780 con 0x7ff3b40b6620 2024-05-20T18:27:31.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.493+0000 7ff3b37fe640 1 --2- 172.21.15.5:0/4181170561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3b40b6620 0x7ff3b40ba3c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40cf1a0 tx=0x7ff3a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa167c4b4c1e62fd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:31.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.493+0000 7ff3b27fc640 1 -- 172.21.15.5:0/4181170561 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4037070 con 0x7ff3b40b6620 2024-05-20T18:27:31.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.493+0000 7ff3b27fc640 1 -- 172.21.15.5:0/4181170561 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3a4032440 con 0x7ff3b40b6620 2024-05-20T18:27:31.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.493+0000 7ff3b27fc640 1 -- 172.21.15.5:0/4181170561 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4032810 con 0x7ff3b40b6620 2024-05-20T18:27:31.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.494+0000 7ff3ba13e640 1 -- 172.21.15.5:0/4181170561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3b40b6620 msgr2=0x7ff3b40ba3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:31.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.494+0000 7ff3ba13e640 1 --2- 172.21.15.5:0/4181170561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3b40b6620 0x7ff3b40ba3c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40cf1a0 tx=0x7ff3a402a270 comp rx=0 tx=0).stop 2024-05-20T18:27:31.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.494+0000 7ff3ba13e640 1 -- 172.21.15.5:0/4181170561 shutdown_connections 2024-05-20T18:27:31.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.494+0000 7ff3ba13e640 1 --2- 172.21.15.5:0/4181170561 >> v2:172.21.15.63:3300/0 conn(0x7ff3b40d3bf0 0x7ff3b40d4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:31.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.494+0000 7ff3ba13e640 1 --2- 172.21.15.5:0/4181170561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3b40b6620 0x7ff3b40ba3c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:31.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.494+0000 7ff3ba13e640 1 -- 172.21.15.5:0/4181170561 >> 172.21.15.5:0/4181170561 conn(0x7ff3b406d240 msgr2=0x7ff3b406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:31.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.494+0000 7ff3ba13e640 1 -- 172.21.15.5:0/4181170561 shutdown_connections 2024-05-20T18:27:31.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.494+0000 7ff3ba13e640 1 -- 172.21.15.5:0/4181170561 wait complete. 2024-05-20T18:27:31.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.496+0000 7ff3ba13e640 1 Processor -- start 2024-05-20T18:27:31.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.496+0000 7ff3ba13e640 1 -- start start 2024-05-20T18:27:31.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.497+0000 7ff3ba13e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3b40b6620 0x7ff3b40c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:31.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.497+0000 7ff3ba13e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3b40d3bf0 0x7ff3b41f6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:31.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.497+0000 7ff3ba13e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3b4162ce0 0x7ff3b41f6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:31.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.497+0000 7ff3ba13e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b41630f0 con 0x7ff3b4162ce0 2024-05-20T18:27:31.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.497+0000 7ff3ba13e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b40d36d0 con 0x7ff3b40d3bf0 2024-05-20T18:27:31.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.497+0000 7ff3ba13e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff3b41f7aa0 con 0x7ff3b40b6620 2024-05-20T18:27:31.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.497+0000 7ff3b3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3b4162ce0 0x7ff3b41f6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:31.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.498+0000 7ff3b3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3b4162ce0 0x7ff3b41f6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59012/0 (socket says 172.21.15.5:59012) 2024-05-20T18:27:31.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.498+0000 7ff3b3fff640 1 -- 172.21.15.5:0/1781399898 learned_addr learned my addr 172.21.15.5:0/1781399898 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:31.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.498+0000 7ff3b37fe640 1 --2- 172.21.15.5:0/1781399898 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3b40b6620 0x7ff3b40c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:31.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.498+0000 7ff3b2ffd640 1 --2- 172.21.15.5:0/1781399898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3b40d3bf0 0x7ff3b41f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:31.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.498+0000 7ff3b3fff640 1 -- 172.21.15.5:0/1781399898 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3b40b6620 msgr2=0x7ff3b40c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:31.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.498+0000 7ff3b3fff640 1 --2- 172.21.15.5:0/1781399898 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3b40b6620 0x7ff3b40c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:31.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.498+0000 7ff3b3fff640 1 -- 172.21.15.5:0/1781399898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3b40d3bf0 msgr2=0x7ff3b41f6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:31.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.498+0000 7ff3b3fff640 1 --2- 172.21.15.5:0/1781399898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3b40d3bf0 0x7ff3b41f6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:31.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.498+0000 7ff3b3fff640 1 -- 172.21.15.5:0/1781399898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3b40c8c30 con 0x7ff3b4162ce0 2024-05-20T18:27:31.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.498+0000 7ff3b3fff640 1 --2- 172.21.15.5:0/1781399898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3b4162ce0 0x7ff3b41f6d10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00efc0 tx=0x7ff3ac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:31.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.498+0000 7ff39bfff640 1 -- 172.21.15.5:0/1781399898 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac018020 con 0x7ff3b4162ce0 2024-05-20T18:27:31.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.498+0000 7ff39bfff640 1 -- 172.21.15.5:0/1781399898 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3ac014480 con 0x7ff3b4162ce0 2024-05-20T18:27:31.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.498+0000 7ff39bfff640 1 -- 172.21.15.5:0/1781399898 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac009e60 con 0x7ff3b4162ce0 2024-05-20T18:27:31.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.498+0000 7ff3ba13e640 1 -- 172.21.15.5:0/1781399898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3b40c8f10 con 0x7ff3b4162ce0 2024-05-20T18:27:31.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.498+0000 7ff3ba13e640 1 -- 172.21.15.5:0/1781399898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3b40c94d0 con 0x7ff3b4162ce0 2024-05-20T18:27:31.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.499+0000 7ff3ba13e640 1 -- 172.21.15.5:0/1781399898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3b4075b00 con 0x7ff3b4162ce0 2024-05-20T18:27:31.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.500+0000 7ff39bfff640 1 -- 172.21.15.5:0/1781399898 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff3ac005d10 con 0x7ff3b4162ce0 2024-05-20T18:27:31.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.500+0000 7ff39bfff640 1 --2- 172.21.15.5:0/1781399898 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff3a006be60 0x7ff3a006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:31.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.501+0000 7ff39bfff640 1 -- 172.21.15.5:0/1781399898 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff3ac08beb0 con 0x7ff3b4162ce0 2024-05-20T18:27:31.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.501+0000 7ff3b37fe640 1 --2- 172.21.15.5:0/1781399898 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff3a006be60 0x7ff3a006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:31.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.501+0000 7ff3b37fe640 1 --2- 172.21.15.5:0/1781399898 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff3a006be60 0x7ff3a006e370 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4034000 tx=0x7ff3a402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:31.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.505+0000 7ff39bfff640 1 -- 172.21.15.5:0/1781399898 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff3ac022030 con 0x7ff3b4162ce0 2024-05-20T18:27:31.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.738+0000 7ff3ba13e640 1 -- 172.21.15.5:0/1781399898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7ff3b40c89c0 con 0x7ff3b4162ce0 2024-05-20T18:27:31.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.742+0000 7ff39bfff640 1 -- 172.21.15.5:0/1781399898 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+172930 (secure 0 0 0) 0x7ff3ac010040 con 0x7ff3b4162ce0 2024-05-20T18:27:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:31.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.744+0000 7ff3ba13e640 1 -- 172.21.15.5:0/1781399898 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff3a006be60 msgr2=0x7ff3a006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:31.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.744+0000 7ff3ba13e640 1 --2- 172.21.15.5:0/1781399898 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff3a006be60 0x7ff3a006e370 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4034000 tx=0x7ff3a402e000 comp rx=0 tx=0).stop 2024-05-20T18:27:31.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.744+0000 7ff3ba13e640 1 -- 172.21.15.5:0/1781399898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3b4162ce0 msgr2=0x7ff3b41f6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:31.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.744+0000 7ff3ba13e640 1 --2- 172.21.15.5:0/1781399898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3b4162ce0 0x7ff3b41f6d10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00efc0 tx=0x7ff3ac00c490 comp rx=0 tx=0).stop 2024-05-20T18:27:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.744+0000 7ff3ba13e640 1 -- 172.21.15.5:0/1781399898 shutdown_connections 2024-05-20T18:27:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.744+0000 7ff3ba13e640 1 --2- 172.21.15.5:0/1781399898 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff3a006be60 0x7ff3a006e370 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.744+0000 7ff3ba13e640 1 --2- 172.21.15.5:0/1781399898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3b4162ce0 0x7ff3b41f6d10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.744+0000 7ff3ba13e640 1 --2- 172.21.15.5:0/1781399898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3b40d3bf0 0x7ff3b41f6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.744+0000 7ff3ba13e640 1 --2- 172.21.15.5:0/1781399898 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3b40b6620 0x7ff3b40c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.744+0000 7ff3ba13e640 1 -- 172.21.15.5:0/1781399898 >> 172.21.15.5:0/1781399898 conn(0x7ff3b406d240 msgr2=0x7ff3b40d4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.744+0000 7ff3ba13e640 1 -- 172.21.15.5:0/1781399898 shutdown_connections 2024-05-20T18:27:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.744+0000 7ff3ba13e640 1 -- 172.21.15.5:0/1781399898 wait complete. 2024-05-20T18:27:31.755 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3330161497},{"type":"v1","addr":"172.21.15.5:6833","nonce":3330161497}]},"active_addr":"172.21.15.5:6833/3330161497","active_change":"2024-05-20T18:26:54.565859+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4115,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":275207151}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":3092091909}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":2965423623}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":432990047}]}]} 2024-05-20T18:27:31.759 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-20T18:27:31.759 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-20T18:27:31.760 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:27:31.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.879+0000 7f2f54237640 1 Processor -- start 2024-05-20T18:27:31.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.880+0000 7f2f54237640 1 -- start start 2024-05-20T18:27:31.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.881+0000 7f2f54237640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2f4c0b4010 0x7f2f4c0d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:31.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.881+0000 7f2f54237640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f4c0d14b0 0x7f2f4c0d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:31.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.881+0000 7f2f54237640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f4c0d0370 con 0x7f2f4c163a10 2024-05-20T18:27:31.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.881+0000 7f2f54237640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f2f4c0d3ca0 con 0x7f2f4c0b4010 2024-05-20T18:27:31.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.881+0000 7f2f54237640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2f4c0d0650 con 0x7f2f4c0d14b0 2024-05-20T18:27:31.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.881+0000 7f2f517d2640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2f4c0b4010 0x7f2f4c0d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:31.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.881+0000 7f2f517d2640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2f4c0b4010 0x7f2f4c0d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59018/0 (socket says 172.21.15.5:59018) 2024-05-20T18:27:31.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.881+0000 7f2f517d2640 1 -- 172.21.15.5:0/2490841464 learned_addr learned my addr 172.21.15.5:0/2490841464 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:31.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.881+0000 7f2f51fd3640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2f4c163a10 0x7f2f4c0cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38584/0 (socket says 172.21.15.5:38584) 2024-05-20T18:27:31.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.881+0000 7f2f527d4640 1 --2- 172.21.15.5:0/2490841464 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f4c0d14b0 0x7f2f4c0d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:31.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.881+0000 7f2f517d2640 1 -- 172.21.15.5:0/2490841464 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f4c0d14b0 msgr2=0x7f2f4c0d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:31.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.881+0000 7f2f517d2640 1 --2- 172.21.15.5:0/2490841464 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f4c0d14b0 0x7f2f4c0d18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:31.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.881+0000 7f2f517d2640 1 -- 172.21.15.5:0/2490841464 >> v1:172.21.15.5:6789/0 conn(0x7f2f4c163a10 legacy=0x7f2f4c0cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:31.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.881+0000 7f2f517d2640 1 -- 172.21.15.5:0/2490841464 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f4c0d3650 con 0x7f2f4c0b4010 2024-05-20T18:27:31.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.881+0000 7f2f50fd1640 1 -- 172.21.15.5:0/2490841464 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1774714037 0 0) 0x7f2f4c0d0370 con 0x7f2f4c163a10 2024-05-20T18:27:31.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.882+0000 7f2f517d2640 1 --2- 172.21.15.5:0/2490841464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f4c0b4010 0x7f2f4c0d4130 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009870 tx=0x7f2f3c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5865afaa964a680e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:31.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.882+0000 7f2f50fd1640 1 -- 172.21.15.5:0/2490841464 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c037070 con 0x7f2f4c0b4010 2024-05-20T18:27:31.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.882+0000 7f2f50fd1640 1 -- 172.21.15.5:0/2490841464 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f3c032440 con 0x7f2f4c0b4010 2024-05-20T18:27:31.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.882+0000 7f2f50fd1640 1 -- 172.21.15.5:0/2490841464 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c032810 con 0x7f2f4c0b4010 2024-05-20T18:27:31.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.883+0000 7f2f54237640 1 -- 172.21.15.5:0/2490841464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f4c0b4010 msgr2=0x7f2f4c0d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:31.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.883+0000 7f2f54237640 1 --2- 172.21.15.5:0/2490841464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f4c0b4010 0x7f2f4c0d4130 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009870 tx=0x7f2f3c02a270 comp rx=0 tx=0).stop 2024-05-20T18:27:31.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.883+0000 7f2f54237640 1 -- 172.21.15.5:0/2490841464 shutdown_connections 2024-05-20T18:27:31.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.883+0000 7f2f54237640 1 --2- 172.21.15.5:0/2490841464 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f4c0d14b0 0x7f2f4c0d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:31.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.883+0000 7f2f54237640 1 --2- 172.21.15.5:0/2490841464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f4c0b4010 0x7f2f4c0d4130 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:31.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.883+0000 7f2f54237640 1 -- 172.21.15.5:0/2490841464 >> 172.21.15.5:0/2490841464 conn(0x7f2f4c06d240 msgr2=0x7f2f4c06d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:31.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.883+0000 7f2f54237640 1 -- 172.21.15.5:0/2490841464 shutdown_connections 2024-05-20T18:27:31.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.883+0000 7f2f54237640 1 -- 172.21.15.5:0/2490841464 wait complete. 2024-05-20T18:27:31.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.885+0000 7f2f54237640 1 Processor -- start 2024-05-20T18:27:31.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.885+0000 7f2f54237640 1 -- start start 2024-05-20T18:27:31.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.886+0000 7f2f54237640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f4c0b4010 0x7f2f4c153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:31.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.886+0000 7f2f54237640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f4c0d14b0 0x7f2f4c160820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:31.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.886+0000 7f2f54237640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f4c163a10 0x7f2f4c160d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:31.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.886+0000 7f2f54237640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c0b8210 con 0x7f2f4c0b4010 2024-05-20T18:27:31.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.886+0000 7f2f54237640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c0d1e00 con 0x7f2f4c163a10 2024-05-20T18:27:31.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.886+0000 7f2f54237640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2f4c0b8390 con 0x7f2f4c0d14b0 2024-05-20T18:27:31.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.886+0000 7f2f51fd3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f4c0b4010 0x7f2f4c153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:31.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.886+0000 7f2f51fd3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f4c0b4010 0x7f2f4c153600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59022/0 (socket says 172.21.15.5:59022) 2024-05-20T18:27:31.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.886+0000 7f2f51fd3640 1 -- 172.21.15.5:0/1354653545 learned_addr learned my addr 172.21.15.5:0/1354653545 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:31.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.887+0000 7f2f517d2640 1 --2- 172.21.15.5:0/1354653545 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f4c0d14b0 0x7f2f4c160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:31.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.887+0000 7f2f527d4640 1 --2- 172.21.15.5:0/1354653545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f4c163a10 0x7f2f4c160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:31.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.887+0000 7f2f51fd3640 1 -- 172.21.15.5:0/1354653545 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f4c0d14b0 msgr2=0x7f2f4c160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:31.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.887+0000 7f2f51fd3640 1 --2- 172.21.15.5:0/1354653545 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f4c0d14b0 0x7f2f4c160820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:31.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.887+0000 7f2f51fd3640 1 -- 172.21.15.5:0/1354653545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f4c163a10 msgr2=0x7f2f4c160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:31.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.887+0000 7f2f51fd3640 1 --2- 172.21.15.5:0/1354653545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f4c163a10 0x7f2f4c160d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:31.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.887+0000 7f2f51fd3640 1 -- 172.21.15.5:0/1354653545 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f4c15fa60 con 0x7f2f4c0b4010 2024-05-20T18:27:31.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.887+0000 7f2f527d4640 1 --2- 172.21.15.5:0/1354653545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f4c163a10 0x7f2f4c160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:31.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.887+0000 7f2f517d2640 1 --2- 172.21.15.5:0/1354653545 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f4c0d14b0 0x7f2f4c160820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:31.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.887+0000 7f2f51fd3640 1 --2- 172.21.15.5:0/1354653545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f4c0b4010 0x7f2f4c153600 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2f4800c7b0 tx=0x7f2f4800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.887+0000 7f2f427fc640 1 -- 172.21.15.5:0/1354653545 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f48018070 con 0x7f2f4c0b4010 2024-05-20T18:27:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.887+0000 7f2f427fc640 1 -- 172.21.15.5:0/1354653545 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f4800a640 con 0x7f2f4c0b4010 2024-05-20T18:27:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.887+0000 7f2f54237640 1 -- 172.21.15.5:0/1354653545 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f4c15fd40 con 0x7f2f4c0b4010 2024-05-20T18:27:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.887+0000 7f2f427fc640 1 -- 172.21.15.5:0/1354653545 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4800aa30 con 0x7f2f4c0b4010 2024-05-20T18:27:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.887+0000 7f2f54237640 1 -- 172.21.15.5:0/1354653545 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f4c1540d0 con 0x7f2f4c0b4010 2024-05-20T18:27:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.888+0000 7f2f54237640 1 -- 172.21.15.5:0/1354653545 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f4c075b00 con 0x7f2f4c0b4010 2024-05-20T18:27:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.889+0000 7f2f427fc640 1 -- 172.21.15.5:0/1354653545 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2f4801b070 con 0x7f2f4c0b4010 2024-05-20T18:27:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.889+0000 7f2f427fc640 1 --2- 172.21.15.5:0/1354653545 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2f3006be60 0x7f2f3006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.889+0000 7f2f427fc640 1 -- 172.21.15.5:0/1354653545 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2f4808a830 con 0x7f2f4c0b4010 2024-05-20T18:27:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.889+0000 7f2f517d2640 1 --2- 172.21.15.5:0/1354653545 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2f3006be60 0x7f2f3006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.890+0000 7f2f517d2640 1 --2- 172.21.15.5:0/1354653545 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2f3006be60 0x7f2f3006e370 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c034000 tx=0x7f2f3c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:31.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:31.894+0000 7f2f427fc640 1 -- 172.21.15.5:0/1354653545 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2f48058e10 con 0x7f2f4c0b4010 2024-05-20T18:27:32.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.078+0000 7f2f54237640 1 -- 172.21.15.5:0/1354653545 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f4c0d3050 con 0x7f2f4c0b4010 2024-05-20T18:27:32.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.079+0000 7f2f427fc640 1 -- 172.21.15.5:0/1354653545 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18394 (secure 0 0 0) 0x7f2f48058710 con 0x7f2f4c0b4010 2024-05-20T18:27:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:32.079 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:27:13.817633+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:27:05.476596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T18:27:06.137762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T18:27:09.829920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:27:32.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.081+0000 7f2f54237640 1 -- 172.21.15.5:0/1354653545 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2f3006be60 msgr2=0x7f2f3006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.081+0000 7f2f54237640 1 --2- 172.21.15.5:0/1354653545 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2f3006be60 0x7f2f3006e370 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c034000 tx=0x7f2f3c02e000 comp rx=0 tx=0).stop 2024-05-20T18:27:32.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.081+0000 7f2f54237640 1 -- 172.21.15.5:0/1354653545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f4c0b4010 msgr2=0x7f2f4c153600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.081+0000 7f2f54237640 1 --2- 172.21.15.5:0/1354653545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f4c0b4010 0x7f2f4c153600 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2f4800c7b0 tx=0x7f2f4800cc70 comp rx=0 tx=0).stop 2024-05-20T18:27:32.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.081+0000 7f2f54237640 1 -- 172.21.15.5:0/1354653545 shutdown_connections 2024-05-20T18:27:32.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.081+0000 7f2f54237640 1 --2- 172.21.15.5:0/1354653545 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2f3006be60 0x7f2f3006e370 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.081+0000 7f2f54237640 1 --2- 172.21.15.5:0/1354653545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f4c163a10 0x7f2f4c160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.081+0000 7f2f54237640 1 --2- 172.21.15.5:0/1354653545 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f4c0d14b0 0x7f2f4c160820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.081+0000 7f2f54237640 1 --2- 172.21.15.5:0/1354653545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f4c0b4010 0x7f2f4c153600 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.081+0000 7f2f54237640 1 -- 172.21.15.5:0/1354653545 >> 172.21.15.5:0/1354653545 conn(0x7f2f4c06d240 msgr2=0x7f2f4c06d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:32.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.081+0000 7f2f54237640 1 -- 172.21.15.5:0/1354653545 shutdown_connections 2024-05-20T18:27:32.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.081+0000 7f2f54237640 1 -- 172.21.15.5:0/1354653545 wait complete. 2024-05-20T18:27:32.092 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-20T18:27:32.093 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:27:32.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.210+0000 7fc1a58e9640 1 Processor -- start 2024-05-20T18:27:32.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.210+0000 7fc1a58e9640 1 -- start start 2024-05-20T18:27:32.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.211+0000 7fc1a58e9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc1a006d9a0 0x7fc1a00d1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.211+0000 7fc1a58e9640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc1a015d940 0x7fc1a015dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.211+0000 7fc1a58e9640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1a015e6b0 con 0x7fc1a00b2f10 2024-05-20T18:27:32.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.211+0000 7fc1a58e9640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fc1a00d2400 con 0x7fc1a006d9a0 2024-05-20T18:27:32.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.211+0000 7fc1a58e9640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc1a00d1a30 con 0x7fc1a015d940 2024-05-20T18:27:32.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.211+0000 7fc19e7fc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc1a006d9a0 0x7fc1a00d1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.211+0000 7fc19e7fc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc1a006d9a0 0x7fc1a00d1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59034/0 (socket says 172.21.15.5:59034) 2024-05-20T18:27:32.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.211+0000 7fc19e7fc640 1 -- 172.21.15.5:0/1109253148 learned_addr learned my addr 172.21.15.5:0/1109253148 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:32.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.211+0000 7fc19f7fe640 1 --2- 172.21.15.5:0/1109253148 >> v2:172.21.15.63:3300/0 conn(0x7fc1a015d940 0x7fc1a015dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.212+0000 7fc19e7fc640 1 -- 172.21.15.5:0/1109253148 >> v2:172.21.15.63:3300/0 conn(0x7fc1a015d940 msgr2=0x7fc1a015dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:32.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.212+0000 7fc19e7fc640 1 --2- 172.21.15.5:0/1109253148 >> v2:172.21.15.63:3300/0 conn(0x7fc1a015d940 0x7fc1a015dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.212+0000 7fc19e7fc640 1 -- 172.21.15.5:0/1109253148 >> v1:172.21.15.63:6789/0 conn(0x7fc1a00b2f10 legacy=0x7fc1a006d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.212+0000 7fc19e7fc640 1 -- 172.21.15.5:0/1109253148 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a00d1100 con 0x7fc1a006d9a0 2024-05-20T18:27:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.212+0000 7fc19e7fc640 1 --2- 172.21.15.5:0/1109253148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a006d9a0 0x7fc1a00d1ec0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0073460 tx=0x7fc19402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cff29fa543fae15 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.213+0000 7fc19dffb640 1 -- 172.21.15.5:0/1109253148 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194004030 con 0x7fc1a006d9a0 2024-05-20T18:27:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.213+0000 7fc19dffb640 1 -- 172.21.15.5:0/1109253148 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1940041d0 con 0x7fc1a006d9a0 2024-05-20T18:27:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.213+0000 7fc19dffb640 1 -- 172.21.15.5:0/1109253148 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194003610 con 0x7fc1a006d9a0 2024-05-20T18:27:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.213+0000 7fc1a58e9640 1 -- 172.21.15.5:0/1109253148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a006d9a0 msgr2=0x7fc1a00d1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.213+0000 7fc1a58e9640 1 --2- 172.21.15.5:0/1109253148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a006d9a0 0x7fc1a00d1ec0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0073460 tx=0x7fc19402a1c0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.213+0000 7fc1a58e9640 1 -- 172.21.15.5:0/1109253148 shutdown_connections 2024-05-20T18:27:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.213+0000 7fc1a58e9640 1 --2- 172.21.15.5:0/1109253148 >> v2:172.21.15.63:3300/0 conn(0x7fc1a015d940 0x7fc1a015dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.213+0000 7fc1a58e9640 1 --2- 172.21.15.5:0/1109253148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a006d9a0 0x7fc1a00d1ec0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.213+0000 7fc1a58e9640 1 -- 172.21.15.5:0/1109253148 >> 172.21.15.5:0/1109253148 conn(0x7fc1a00b6670 msgr2=0x7fc1a00b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.213+0000 7fc1a58e9640 1 -- 172.21.15.5:0/1109253148 shutdown_connections 2024-05-20T18:27:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.213+0000 7fc1a58e9640 1 -- 172.21.15.5:0/1109253148 wait complete. 2024-05-20T18:27:32.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.215+0000 7fc1a58e9640 1 Processor -- start 2024-05-20T18:27:32.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.215+0000 7fc1a58e9640 1 -- start start 2024-05-20T18:27:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.216+0000 7fc1a58e9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1a006d9a0 0x7fc1a01441c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.216+0000 7fc1a58e9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a00b2f10 0x7fc1a01f50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.217+0000 7fc1a58e9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1a015d940 0x7fc1a006f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.217+0000 7fc1a58e9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a00d16c0 con 0x7fc1a00b2f10 2024-05-20T18:27:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.217+0000 7fc1a58e9640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a00b3320 con 0x7fc1a006d9a0 2024-05-20T18:27:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.217+0000 7fc1a58e9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc1a015e290 con 0x7fc1a015d940 2024-05-20T18:27:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.217+0000 7fc19e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a00b2f10 0x7fc1a01f50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.217+0000 7fc19e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a00b2f10 0x7fc1a01f50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59040/0 (socket says 172.21.15.5:59040) 2024-05-20T18:27:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.217+0000 7fc19e7fc640 1 -- 172.21.15.5:0/3295935482 learned_addr learned my addr 172.21.15.5:0/3295935482 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.217+0000 7fc19f7fe640 1 --2- 172.21.15.5:0/3295935482 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1a015d940 0x7fc1a006f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.217+0000 7fc19effd640 1 --2- 172.21.15.5:0/3295935482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1a006d9a0 0x7fc1a01441c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.217+0000 7fc19e7fc640 1 -- 172.21.15.5:0/3295935482 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1a015d940 msgr2=0x7fc1a006f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.217+0000 7fc19e7fc640 1 --2- 172.21.15.5:0/3295935482 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1a015d940 0x7fc1a006f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.217+0000 7fc19e7fc640 1 -- 172.21.15.5:0/3295935482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1a006d9a0 msgr2=0x7fc1a01441c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.217+0000 7fc19e7fc640 1 --2- 172.21.15.5:0/3295935482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1a006d9a0 0x7fc1a01441c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.217+0000 7fc19e7fc640 1 -- 172.21.15.5:0/3295935482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a01f5d20 con 0x7fc1a00b2f10 2024-05-20T18:27:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.217+0000 7fc19e7fc640 1 --2- 172.21.15.5:0/3295935482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a00b2f10 0x7fc1a01f50e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc19402a200 tx=0x7fc1940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.217+0000 7fc187fff640 1 -- 172.21.15.5:0/3295935482 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194041020 con 0x7fc1a00b2f10 2024-05-20T18:27:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.217+0000 7fc187fff640 1 -- 172.21.15.5:0/3295935482 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc194003c50 con 0x7fc1a00b2f10 2024-05-20T18:27:32.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.217+0000 7fc1a58e9640 1 -- 172.21.15.5:0/3295935482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a01f6000 con 0x7fc1a00b2f10 2024-05-20T18:27:32.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.217+0000 7fc187fff640 1 -- 172.21.15.5:0/3295935482 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194004030 con 0x7fc1a00b2f10 2024-05-20T18:27:32.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.218+0000 7fc1a58e9640 1 -- 172.21.15.5:0/3295935482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a00af4e0 con 0x7fc1a00b2f10 2024-05-20T18:27:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.219+0000 7fc1a58e9640 1 -- 172.21.15.5:0/3295935482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1a01f5a90 con 0x7fc1a00b2f10 2024-05-20T18:27:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.219+0000 7fc187fff640 1 -- 172.21.15.5:0/3295935482 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc194044020 con 0x7fc1a00b2f10 2024-05-20T18:27:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.220+0000 7fc187fff640 1 --2- 172.21.15.5:0/3295935482 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc18c06be60 0x7fc18c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.220+0000 7fc187fff640 1 -- 172.21.15.5:0/3295935482 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc1940abde0 con 0x7fc1a00b2f10 2024-05-20T18:27:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.220+0000 7fc19effd640 1 --2- 172.21.15.5:0/3295935482 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc18c06be60 0x7fc18c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.221+0000 7fc19effd640 1 --2- 172.21.15.5:0/3295935482 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc18c06be60 0x7fc18c06e370 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc1900012e0 tx=0x7fc190009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.225+0000 7fc187fff640 1 -- 172.21.15.5:0/3295935482 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc19404b030 con 0x7fc1a00b2f10 2024-05-20T18:27:32.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.408+0000 7fc1a58e9640 1 -- 172.21.15.5:0/3295935482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1a0075b00 con 0x7fc1a00b2f10 2024-05-20T18:27:32.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.410+0000 7fc187fff640 1 -- 172.21.15.5:0/3295935482 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18394 (secure 0 0 0) 0x7fc19407a3c0 con 0x7fc1a00b2f10 2024-05-20T18:27:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:32.409 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:27:13.817633+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:27:05.476596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T18:27:06.137762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T18:27:09.829920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:27:32.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.412+0000 7fc1a58e9640 1 -- 172.21.15.5:0/3295935482 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc18c06be60 msgr2=0x7fc18c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.412+0000 7fc1a58e9640 1 --2- 172.21.15.5:0/3295935482 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc18c06be60 0x7fc18c06e370 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc1900012e0 tx=0x7fc190009040 comp rx=0 tx=0).stop 2024-05-20T18:27:32.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.412+0000 7fc1a58e9640 1 -- 172.21.15.5:0/3295935482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a00b2f10 msgr2=0x7fc1a01f50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.412+0000 7fc1a58e9640 1 --2- 172.21.15.5:0/3295935482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a00b2f10 0x7fc1a01f50e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc19402a200 tx=0x7fc1940046c0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.412+0000 7fc1a58e9640 1 -- 172.21.15.5:0/3295935482 shutdown_connections 2024-05-20T18:27:32.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.412+0000 7fc1a58e9640 1 --2- 172.21.15.5:0/3295935482 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc18c06be60 0x7fc18c06e370 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.412+0000 7fc1a58e9640 1 --2- 172.21.15.5:0/3295935482 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1a015d940 0x7fc1a006f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.412+0000 7fc1a58e9640 1 --2- 172.21.15.5:0/3295935482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a00b2f10 0x7fc1a01f50e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.412+0000 7fc1a58e9640 1 --2- 172.21.15.5:0/3295935482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1a006d9a0 0x7fc1a01441c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.412+0000 7fc1a58e9640 1 -- 172.21.15.5:0/3295935482 >> 172.21.15.5:0/3295935482 conn(0x7fc1a00b6670 msgr2=0x7fc1a006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:32.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.412+0000 7fc1a58e9640 1 -- 172.21.15.5:0/3295935482 shutdown_connections 2024-05-20T18:27:32.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.412+0000 7fc1a58e9640 1 -- 172.21.15.5:0/3295935482 wait complete. 2024-05-20T18:27:32.424 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-20T18:27:32.424 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-20T18:27:32.425 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-20T18:27:32.425 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-20T18:27:32.426 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-20T18:27:32.426 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-20T18:27:32.427 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-20T18:27:32.427 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-20T18:27:32.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.657+0000 7fe3759ff640 1 Processor -- start 2024-05-20T18:27:32.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.657+0000 7fe3759ff640 1 -- start start 2024-05-20T18:27:32.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.658+0000 7fd636e6c640 1 Processor -- start 2024-05-20T18:27:32.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.658+0000 7fc20d573640 1 Processor -- start 2024-05-20T18:27:32.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.658+0000 7fc20d573640 1 -- start start 2024-05-20T18:27:32.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.659+0000 7f440905f640 1 Processor -- start 2024-05-20T18:27:32.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.658+0000 7fd636e6c640 1 -- start start 2024-05-20T18:27:32.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.659+0000 7f7290b17640 1 Processor -- start 2024-05-20T18:27:32.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.659+0000 7fe3759ff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe3700b64f0 0x7fe37006b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.659+0000 7fe3759ff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe370165ff0 0x7fe370166400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.659+0000 7fe3759ff640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3700d1010 con 0x7fe3700b1cb0 2024-05-20T18:27:32.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.659+0000 7fe3759ff640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe37006bf20 con 0x7fe370165ff0 2024-05-20T18:27:32.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.660+0000 7fe3759ff640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe3700d12f0 con 0x7fe3700b64f0 2024-05-20T18:27:32.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.659+0000 7f7290b17640 1 -- start start 2024-05-20T18:27:32.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.660+0000 7fe36f7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe370165ff0 0x7fe370166400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.659+0000 7f440905f640 1 -- start start 2024-05-20T18:27:32.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.660+0000 7fe36f7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe370165ff0 0x7fe370166400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59052/0 (socket says 172.21.15.5:59052) 2024-05-20T18:27:32.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.660+0000 7fe36f7fe640 1 -- 172.21.15.5:0/2582633447 learned_addr learned my addr 172.21.15.5:0/2582633447 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:32.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.660+0000 7fe36e7fc640 1 --1- 172.21.15.5:0/2582633447 >> v1:172.21.15.5:6789/0 conn(0x7fe3700b1cb0 0x7fe3700b20c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38596/0 (socket says 172.21.15.5:38596) 2024-05-20T18:27:32.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.660+0000 7fe36effd640 1 --2- 172.21.15.5:0/2582633447 >> v2:172.21.15.63:3300/0 conn(0x7fe3700b64f0 0x7fe37006b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.661+0000 7fe36f7fe640 1 -- 172.21.15.5:0/2582633447 >> v2:172.21.15.63:3300/0 conn(0x7fe3700b64f0 msgr2=0x7fe37006b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:32.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.661+0000 7fe36f7fe640 1 --2- 172.21.15.5:0/2582633447 >> v2:172.21.15.63:3300/0 conn(0x7fe3700b64f0 0x7fe37006b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.661+0000 7fe36f7fe640 1 -- 172.21.15.5:0/2582633447 >> v1:172.21.15.5:6789/0 conn(0x7fe3700b1cb0 legacy=0x7fe3700b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.661+0000 7fe36f7fe640 1 -- 172.21.15.5:0/2582633447 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3700d15d0 con 0x7fe370165ff0 2024-05-20T18:27:32.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.661+0000 7fc20d573640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc20003a500 0x7fc2000e1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.661+0000 7fc20d573640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc200034130 0x7fc200038500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.661+0000 7fd636e6c640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd63016c4c0 0x7fd63016e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.661+0000 7fd636e6c640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6300b0fd0 con 0x7fd630167dd0 2024-05-20T18:27:32.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.661+0000 7fd636e6c640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6300b1290 con 0x7fd6300c48f0 2024-05-20T18:27:32.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.661+0000 7fd636e6c640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd63009b500 con 0x7fd63016c4c0 2024-05-20T18:27:32.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.661+0000 7fc20d573640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc200050920 con 0x7fc200038a40 2024-05-20T18:27:32.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.661+0000 7fc20d573640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc200038e50 con 0x7fc20003a500 2024-05-20T18:27:32.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.661+0000 7fd634c08640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd630167dd0 0x7fd6300b20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38608/0 (socket says 172.21.15.5:38608) 2024-05-20T18:27:32.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.661+0000 7fd634c08640 1 -- 172.21.15.5:0/904407953 learned_addr learned my addr 172.21.15.5:0/904407953 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:32.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.661+0000 7fe2a359e640 1 Processor -- start 2024-05-20T18:27:32.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.661+0000 7fd635409640 1 --2- 172.21.15.5:0/904407953 >> v2:172.21.15.63:3300/0 conn(0x7fd63016c4c0 0x7fd63016e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.662+0000 7f440905f640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f43fc04f370 0x7f43fc033aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.662+0000 7f440905f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f43fc033fe0 0x7f43fc038370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.662+0000 7fd62f7fe640 1 -- 172.21.15.5:0/904407953 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3694113362 0 0) 0x7fd6300b1290 con 0x7fd6300c48f0 2024-05-20T18:27:32.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.662+0000 7f4402d9d640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f43fc052fe0 0x7f43fc007350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:57446/0 (socket says 172.21.15.5:57446) 2024-05-20T18:27:32.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.662+0000 7fd62f7fe640 1 -- 172.21.15.5:0/904407953 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd618003520 con 0x7fd6300c48f0 2024-05-20T18:27:32.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.662+0000 7fd635409640 1 -- 172.21.15.5:0/904407953 >> v1:172.21.15.63:6789/0 conn(0x7fd6300c48f0 legacy=0x7fd6300c4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.662+0000 7fd635409640 1 -- 172.21.15.5:0/904407953 >> v1:172.21.15.5:6789/0 conn(0x7fd630167dd0 legacy=0x7fd6300b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.662+0000 7fe2a359e640 1 -- start start 2024-05-20T18:27:32.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.662+0000 7fd635409640 1 -- 172.21.15.5:0/904407953 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6300b1570 con 0x7fd63016c4c0 2024-05-20T18:27:32.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.662+0000 7f7290b17640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f728c0cfe10 0x7f728c163600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.662+0000 7f7290b17640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f728c167b90 0x7f728c06c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.662+0000 7f7290b17640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f728c06ce80 con 0x7f728c06ca70 2024-05-20T18:27:32.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.662+0000 7f7290b17640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f728c0b54f0 con 0x7f728c0cfe10 2024-05-20T18:27:32.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.662+0000 7f7290b17640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f728c06f110 con 0x7f728c167b90 2024-05-20T18:27:32.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.663+0000 7fe36f7fe640 1 --2- 172.21.15.5:0/2582633447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe370165ff0 0x7fe370166400 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe370073a50 tx=0x7fe36802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf3bc778de9759e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.662+0000 7f728bfff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f728c06ca70 0x7f728c0ce5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38620/0 (socket says 172.21.15.5:38620) 2024-05-20T18:27:32.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.662+0000 7f728bfff640 1 -- 172.21.15.5:0/22769698 learned_addr learned my addr 172.21.15.5:0/22769698 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:32.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.663+0000 7f728b7fe640 1 --2- 172.21.15.5:0/22769698 >> v2:172.21.15.63:3300/0 conn(0x7f728c0cfe10 0x7f728c163600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.661+0000 7fc20d573640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc200034540 con 0x7fc200034130 2024-05-20T18:27:32.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.662+0000 7fc207fff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc20003a500 0x7fc2000e1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.662+0000 7fc207fff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc20003a500 0x7fc2000e1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:53388/0 (socket says 172.21.15.5:53388) 2024-05-20T18:27:32.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.662+0000 7fc207fff640 1 -- 172.21.15.5:0/4192651247 learned_addr learned my addr 172.21.15.5:0/4192651247 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:32.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.663+0000 7fc2077fe640 1 --2- 172.21.15.5:0/4192651247 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc200034130 0x7fc200038500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.661+0000 7f8883fff640 1 Processor -- start 2024-05-20T18:27:32.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.661+0000 7f8883fff640 1 -- start start 2024-05-20T18:27:32.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.662+0000 7f440259c640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f43fc04f370 0x7f43fc033aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.662+0000 7f440259c640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f43fc04f370 0x7f43fc033aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:53400/0 (socket says 172.21.15.5:53400) 2024-05-20T18:27:32.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.662+0000 7f440359e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f43fc033fe0 0x7f43fc038370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.663+0000 7f4402d9d640 1 -- 172.21.15.5:0/963324593 learned_addr learned my addr 172.21.15.5:0/963324593 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:32.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.663+0000 7f440905f640 1 -- 172.21.15.5:0/963324593 --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43fc04f780 con 0x7f43fc052fe0 2024-05-20T18:27:32.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.663+0000 7f728b7fe640 1 -- 172.21.15.5:0/22769698 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f728c167b90 msgr2=0x7f728c06c530 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T18:27:32.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.663+0000 7f728b7fe640 1 --2- 172.21.15.5:0/22769698 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f728c167b90 0x7f728c06c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.663+0000 7f728b7fe640 1 -- 172.21.15.5:0/22769698 >> v1:172.21.15.5:6789/0 conn(0x7f728c06ca70 legacy=0x7f728c0ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.663+0000 7f440905f640 1 -- 172.21.15.5:0/963324593 --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f43fc0533f0 con 0x7f43fc033fe0 2024-05-20T18:27:32.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.663+0000 7f440905f640 1 -- 172.21.15.5:0/963324593 --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f43fc0343f0 con 0x7f43fc04f370 2024-05-20T18:27:32.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.663+0000 7fe36dffb640 1 -- 172.21.15.5:0/2582633447 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368036070 con 0x7fe370165ff0 2024-05-20T18:27:32.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.663+0000 7fc206ffd640 1 -- 172.21.15.5:0/4192651247 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3020607539 0 0) 0x7fc1f4001110 con 0x7fc200038a40 2024-05-20T18:27:32.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.663+0000 7fc206ffd640 1 -- 172.21.15.5:0/4192651247 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1f8003520 con 0x7fc200038a40 2024-05-20T18:27:32.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.663+0000 7f728b7fe640 1 -- 172.21.15.5:0/22769698 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f728c0b48e0 con 0x7f728c0cfe10 2024-05-20T18:27:32.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.663+0000 7fe36dffb640 1 -- 172.21.15.5:0/2582633447 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe368031600 con 0x7fe370165ff0 2024-05-20T18:27:32.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.663+0000 7fe36dffb640 1 -- 172.21.15.5:0/2582633447 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3680319d0 con 0x7fe370165ff0 2024-05-20T18:27:32.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.663+0000 7fd635409640 1 --2- 172.21.15.5:0/904407953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd63016c4c0 0x7fd63016e8e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd6200097a0 tx=0x7fd62002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a256857f336f6f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.664+0000 7fd62f7fe640 1 -- 172.21.15.5:0/904407953 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd620037070 con 0x7fd63016c4c0 2024-05-20T18:27:32.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.664+0000 7fd62f7fe640 1 -- 172.21.15.5:0/904407953 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd620032440 con 0x7fd63016c4c0 2024-05-20T18:27:32.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.664+0000 7f8883fff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f887c033950 0x7f887c033d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.664+0000 7f8883fff640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f887c0342a0 con 0x7f887c048320 2024-05-20T18:27:32.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.664+0000 7f8883fff640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f887c048020 con 0x7f887c04f340 2024-05-20T18:27:32.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.664+0000 7f8883fff640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f887c053210 con 0x7f887c033950 2024-05-20T18:27:32.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.664+0000 7fd62f7fe640 1 -- 172.21.15.5:0/904407953 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6200327f0 con 0x7fd63016c4c0 2024-05-20T18:27:32.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.664+0000 7f8882ffd640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f887c04f340 0x7f887c0e7b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:57482/0 (socket says 172.21.15.5:57482) 2024-05-20T18:27:32.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.664+0000 7f8882ffd640 1 -- 172.21.15.5:0/1259825745 learned_addr learned my addr 172.21.15.5:0/1259825745 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:32.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.665+0000 7f88837fe640 1 --2- 172.21.15.5:0/1259825745 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f887c033950 0x7f887c033d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.664+0000 7fe2a359e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe29c04f330 0x7fe29c033a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.664+0000 7fe2a359e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe29c033fa0 0x7fe29c038330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.664+0000 7fe2a359e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe29c04f740 con 0x7fe29c052fa0 2024-05-20T18:27:32.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.664+0000 7fe2a359e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe29c0533b0 con 0x7fe29c033fa0 2024-05-20T18:27:32.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.664+0000 7fe2a359e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe29c0343b0 con 0x7fe29c04f330 2024-05-20T18:27:32.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.664+0000 7fe2a259c640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fe29c052fa0 0x7fe29c007310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:57468/0 (socket says 172.21.15.5:57468) 2024-05-20T18:27:32.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.664+0000 7fe2a259c640 1 -- 172.21.15.5:0/3419890307 learned_addr learned my addr 172.21.15.5:0/3419890307 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:32.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.664+0000 7f440359e640 1 -- 172.21.15.5:0/963324593 >> v2:172.21.15.63:3300/0 conn(0x7f43fc04f370 msgr2=0x7f43fc033aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:32.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.664+0000 7f440359e640 1 --2- 172.21.15.5:0/963324593 >> v2:172.21.15.63:3300/0 conn(0x7f43fc04f370 0x7f43fc033aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.664+0000 7f440359e640 1 -- 172.21.15.5:0/963324593 >> v1:172.21.15.63:6789/0 conn(0x7f43fc052fe0 legacy=0x7f43fc007350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:32.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.664+0000 7f440359e640 1 -- 172.21.15.5:0/963324593 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43fc007a60 con 0x7f43fc033fe0 2024-05-20T18:27:32.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.664+0000 7fe3759ff640 1 -- 172.21.15.5:0/2582633447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe370165ff0 msgr2=0x7fe370166400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.664+0000 7fe3759ff640 1 --2- 172.21.15.5:0/2582633447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe370165ff0 0x7fe370166400 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe370073a50 tx=0x7fe36802a6a0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.665+0000 7fe3759ff640 1 -- 172.21.15.5:0/2582633447 shutdown_connections 2024-05-20T18:27:32.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.665+0000 7fe3759ff640 1 --2- 172.21.15.5:0/2582633447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe370165ff0 0x7fe370166400 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.665+0000 7fe3759ff640 1 --2- 172.21.15.5:0/2582633447 >> v2:172.21.15.63:3300/0 conn(0x7fe3700b64f0 0x7fe37006b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.665+0000 7fe3759ff640 1 -- 172.21.15.5:0/2582633447 >> 172.21.15.5:0/2582633447 conn(0x7fe37009a4f0 msgr2=0x7fe3700b5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:32.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.665+0000 7fe3759ff640 1 -- 172.21.15.5:0/2582633447 shutdown_connections 2024-05-20T18:27:32.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.665+0000 7fe3759ff640 1 -- 172.21.15.5:0/2582633447 wait complete. 2024-05-20T18:27:32.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.665+0000 7f8881ffb640 1 -- 172.21.15.5:0/1259825745 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2248401313 0 0) 0x7f887c048020 con 0x7f887c04f340 2024-05-20T18:27:32.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.665+0000 7fd636e6c640 1 -- 172.21.15.5:0/904407953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd63016c4c0 msgr2=0x7fd63016e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.665+0000 7fd636e6c640 1 --2- 172.21.15.5:0/904407953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd63016c4c0 0x7fd63016e8e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd6200097a0 tx=0x7fd62002a230 comp rx=0 tx=0).stop 2024-05-20T18:27:32.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.665+0000 7fe2a1d9b640 1 --2- 172.21.15.5:0/3419890307 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe29c04f330 0x7fe29c033a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.665+0000 7fe2a159a640 1 -- 172.21.15.5:0/3419890307 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3702125529 0 0) 0x7fe29c04f740 con 0x7fe29c052fa0 2024-05-20T18:27:32.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.665+0000 7fe2a159a640 1 -- 172.21.15.5:0/3419890307 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe28c003520 con 0x7fe29c052fa0 2024-05-20T18:27:32.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.665+0000 7fd636e6c640 1 -- 172.21.15.5:0/904407953 shutdown_connections 2024-05-20T18:27:32.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.665+0000 7fd636e6c640 1 --2- 172.21.15.5:0/904407953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd63016c4c0 0x7fd63016e8e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.665+0000 7f8881ffb640 1 -- 172.21.15.5:0/1259825745 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f886c003520 con 0x7f887c04f340 2024-05-20T18:27:32.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.665+0000 7fd636e6c640 1 -- 172.21.15.5:0/904407953 >> 172.21.15.5:0/904407953 conn(0x7fd6300b6620 msgr2=0x7fd6300b6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:32.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.664+0000 7fc207fff640 1 -- 172.21.15.5:0/4192651247 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc200034130 msgr2=0x7fc200038500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:32.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.664+0000 7fc207fff640 1 --2- 172.21.15.5:0/4192651247 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc200034130 0x7fc200038500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.664+0000 7fc207fff640 1 -- 172.21.15.5:0/4192651247 >> v1:172.21.15.63:6789/0 conn(0x7fc200038a40 legacy=0x7fc2000e5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.664+0000 7fc207fff640 1 -- 172.21.15.5:0/4192651247 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc20003aa90 con 0x7fc20003a500 2024-05-20T18:27:32.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.665+0000 7f8881ffb640 1 -- 172.21.15.5:0/1259825745 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 258711643 0 0) 0x7f8878001110 con 0x7f887c048320 2024-05-20T18:27:32.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.665+0000 7f728b7fe640 1 --2- 172.21.15.5:0/22769698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f728c0cfe10 0x7f728c163600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f727c0097a0 tx=0x7f727c02a0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f9568f115e552f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.665+0000 7f8881ffb640 1 -- 172.21.15.5:0/1259825745 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f887c048020 con 0x7f887c048320 2024-05-20T18:27:32.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.665+0000 7fc207fff640 1 --2- 172.21.15.5:0/4192651247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc20003a500 0x7fc2000e1690 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc20804fc40 tx=0x7fc2080a6020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97158bb1b2b8ddf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.665+0000 7f88837fe640 1 -- 172.21.15.5:0/1259825745 >> v1:172.21.15.63:6789/0 conn(0x7f887c04f340 legacy=0x7f887c0e7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.665+0000 7f88837fe640 1 -- 172.21.15.5:0/1259825745 >> v1:172.21.15.5:6789/0 conn(0x7f887c048320 legacy=0x7f887c0382b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.665+0000 7f88837fe640 1 -- 172.21.15.5:0/1259825745 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f887c006180 con 0x7f887c033950 2024-05-20T18:27:32.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.666+0000 7fc206ffd640 1 -- 172.21.15.5:0/4192651247 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2080b3070 con 0x7fc20003a500 2024-05-20T18:27:32.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.666+0000 7fc206ffd640 1 -- 172.21.15.5:0/4192651247 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2080a6c20 con 0x7fc20003a500 2024-05-20T18:27:32.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.666+0000 7fc206ffd640 1 -- 172.21.15.5:0/4192651247 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2080ae640 con 0x7fc20003a500 2024-05-20T18:27:32.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.666+0000 7f728a7fc640 1 -- 172.21.15.5:0/22769698 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c037070 con 0x7f728c0cfe10 2024-05-20T18:27:32.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.666+0000 7f728a7fc640 1 -- 172.21.15.5:0/22769698 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f727c02ad30 con 0x7f728c0cfe10 2024-05-20T18:27:32.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.666+0000 7f728a7fc640 1 -- 172.21.15.5:0/22769698 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c032660 con 0x7f728c0cfe10 2024-05-20T18:27:32.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.665+0000 7fd636e6c640 1 -- 172.21.15.5:0/904407953 shutdown_connections 2024-05-20T18:27:32.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.666+0000 7fd636e6c640 1 -- 172.21.15.5:0/904407953 wait complete. 2024-05-20T18:27:32.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.665+0000 7fe2a2d9d640 1 --2- 172.21.15.5:0/3419890307 >> v2:172.21.15.5:3300/0 conn(0x7fe29c033fa0 0x7fe29c038330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.666+0000 7fe2a1d9b640 1 -- 172.21.15.5:0/3419890307 >> v2:172.21.15.5:3300/0 conn(0x7fe29c033fa0 msgr2=0x7fe29c038330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:32.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.666+0000 7fe2a1d9b640 1 --2- 172.21.15.5:0/3419890307 >> v2:172.21.15.5:3300/0 conn(0x7fe29c033fa0 0x7fe29c038330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.666+0000 7fe2a1d9b640 1 -- 172.21.15.5:0/3419890307 >> v1:172.21.15.63:6789/0 conn(0x7fe29c052fa0 legacy=0x7fe29c007310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.666+0000 7fe2a1d9b640 1 -- 172.21.15.5:0/3419890307 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe29c007a20 con 0x7fe29c04f330 2024-05-20T18:27:32.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.666+0000 7f88837fe640 1 --2- 172.21.15.5:0/1259825745 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f887c033950 0x7f887c033d60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f88740097d0 tx=0x7f887402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f7d733b6c3c7c4e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.667+0000 7f440359e640 1 --2- 172.21.15.5:0/963324593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43fc033fe0 0x7f43fc038370 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f43fc055e20 tx=0x7f43f802a0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef937d5bba07d89 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.667+0000 7fccfb59e640 1 Processor -- start 2024-05-20T18:27:32.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.667+0000 7f7290b17640 1 -- 172.21.15.5:0/22769698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f728c0cfe10 msgr2=0x7f728c163600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.667+0000 7f7290b17640 1 --2- 172.21.15.5:0/22769698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f728c0cfe10 0x7f728c163600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f727c0097a0 tx=0x7f727c02a0a0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.667+0000 7fccfb59e640 1 -- start start 2024-05-20T18:27:32.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.667+0000 7f8881ffb640 1 -- 172.21.15.5:0/1259825745 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874037070 con 0x7f887c033950 2024-05-20T18:27:32.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.667+0000 7f8881ffb640 1 -- 172.21.15.5:0/1259825745 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8874032440 con 0x7f887c033950 2024-05-20T18:27:32.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.667+0000 7f8881ffb640 1 -- 172.21.15.5:0/1259825745 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874032810 con 0x7f887c033950 2024-05-20T18:27:32.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.667+0000 7f7290b17640 1 -- 172.21.15.5:0/22769698 shutdown_connections 2024-05-20T18:27:32.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.667+0000 7f7290b17640 1 --2- 172.21.15.5:0/22769698 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f728c167b90 0x7f728c06c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.667+0000 7f7290b17640 1 --2- 172.21.15.5:0/22769698 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f728c0cfe10 0x7f728c163600 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.667+0000 7f7290b17640 1 -- 172.21.15.5:0/22769698 >> 172.21.15.5:0/22769698 conn(0x7f728c0721d0 msgr2=0x7f728c0725e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:32.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.667+0000 7f7290b17640 1 -- 172.21.15.5:0/22769698 shutdown_connections 2024-05-20T18:27:32.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.667+0000 7f7290b17640 1 -- 172.21.15.5:0/22769698 wait complete. 2024-05-20T18:27:32.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.667+0000 7fe2a1d9b640 1 --2- 172.21.15.5:0/3419890307 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe29c04f330 0x7fe29c033a60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe2980097a0 tx=0x7fe29802a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=771dd8eab4a2b75c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.667+0000 7fe3759ff640 1 Processor -- start 2024-05-20T18:27:32.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.667+0000 7f4401d9b640 1 -- 172.21.15.5:0/963324593 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f8037070 con 0x7f43fc033fe0 2024-05-20T18:27:32.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.668+0000 7f4401d9b640 1 -- 172.21.15.5:0/963324593 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43f802adf0 con 0x7f43fc033fe0 2024-05-20T18:27:32.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.668+0000 7f4401d9b640 1 -- 172.21.15.5:0/963324593 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f8032660 con 0x7f43fc033fe0 2024-05-20T18:27:32.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.668+0000 7fc20d573640 1 -- 172.21.15.5:0/4192651247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc20003a500 msgr2=0x7fc2000e1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.668+0000 7fc20d573640 1 --2- 172.21.15.5:0/4192651247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc20003a500 0x7fc2000e1690 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc20804fc40 tx=0x7fc2080a6020 comp rx=0 tx=0).stop 2024-05-20T18:27:32.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.668+0000 7fe2a159a640 1 -- 172.21.15.5:0/3419890307 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe298037070 con 0x7fe29c04f330 2024-05-20T18:27:32.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.668+0000 7fe2a159a640 1 -- 172.21.15.5:0/3419890307 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe29802aaf0 con 0x7fe29c04f330 2024-05-20T18:27:32.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.668+0000 7fe3759ff640 1 -- start start 2024-05-20T18:27:32.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.668+0000 7fe2a159a640 1 -- 172.21.15.5:0/3419890307 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe294001670 con 0x7fe29c04f330 2024-05-20T18:27:32.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.668+0000 7fc20d573640 1 -- 172.21.15.5:0/4192651247 shutdown_connections 2024-05-20T18:27:32.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.668+0000 7fc20d573640 1 --2- 172.21.15.5:0/4192651247 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc200034130 0x7fc200038500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.668+0000 7fc20d573640 1 --2- 172.21.15.5:0/4192651247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc20003a500 0x7fc2000e1690 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.668+0000 7fc20d573640 1 -- 172.21.15.5:0/4192651247 >> 172.21.15.5:0/4192651247 conn(0x7fc200054ce0 msgr2=0x7fc2000550f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:32.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.668+0000 7f8883fff640 1 -- 172.21.15.5:0/1259825745 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f887c033950 msgr2=0x7f887c033d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.668+0000 7f8883fff640 1 --2- 172.21.15.5:0/1259825745 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f887c033950 0x7f887c033d60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f88740097d0 tx=0x7f887402a210 comp rx=0 tx=0).stop 2024-05-20T18:27:32.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.668+0000 7fc20d573640 1 -- 172.21.15.5:0/4192651247 shutdown_connections 2024-05-20T18:27:32.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.668+0000 7f8883fff640 1 -- 172.21.15.5:0/1259825745 shutdown_connections 2024-05-20T18:27:32.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.668+0000 7f8883fff640 1 --2- 172.21.15.5:0/1259825745 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f887c033950 0x7f887c033d60 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.668+0000 7f8883fff640 1 -- 172.21.15.5:0/1259825745 >> 172.21.15.5:0/1259825745 conn(0x7f887c054cf0 msgr2=0x7f887c055100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:32.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.669+0000 7f8883fff640 1 -- 172.21.15.5:0/1259825745 shutdown_connections 2024-05-20T18:27:32.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.669+0000 7f440905f640 1 -- 172.21.15.5:0/963324593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43fc033fe0 msgr2=0x7f43fc038370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.669+0000 7fe2a359e640 1 -- 172.21.15.5:0/3419890307 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe29c04f330 msgr2=0x7fe29c033a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.669+0000 7fe2a359e640 1 --2- 172.21.15.5:0/3419890307 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe29c04f330 0x7fe29c033a60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe2980097a0 tx=0x7fe29802a060 comp rx=0 tx=0).stop 2024-05-20T18:27:32.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.669+0000 7fe2a359e640 1 -- 172.21.15.5:0/3419890307 shutdown_connections 2024-05-20T18:27:32.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.669+0000 7fe2a359e640 1 --2- 172.21.15.5:0/3419890307 >> v2:172.21.15.5:3300/0 conn(0x7fe29c033fa0 0x7fe29c038330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.669+0000 7fe2a359e640 1 --2- 172.21.15.5:0/3419890307 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe29c04f330 0x7fe29c033a60 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe2980097a0 tx=0x7fe29802a060 comp rx=0 tx=0).stop 2024-05-20T18:27:32.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.669+0000 7fe2a359e640 1 -- 172.21.15.5:0/3419890307 >> 172.21.15.5:0/3419890307 conn(0x7fe29c054ce0 msgr2=0x7fe29c0550f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:32.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.669+0000 7f440905f640 1 --2- 172.21.15.5:0/963324593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43fc033fe0 0x7f43fc038370 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f43fc055e20 tx=0x7f43f802a0e0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.669+0000 7fe2a359e640 1 -- 172.21.15.5:0/3419890307 shutdown_connections 2024-05-20T18:27:32.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.669+0000 7fe2a359e640 1 -- 172.21.15.5:0/3419890307 wait complete. 2024-05-20T18:27:32.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.670+0000 7f440905f640 1 -- 172.21.15.5:0/963324593 shutdown_connections 2024-05-20T18:27:32.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.670+0000 7f440905f640 1 --2- 172.21.15.5:0/963324593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43fc033fe0 0x7f43fc038370 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.670+0000 7f440905f640 1 --2- 172.21.15.5:0/963324593 >> v2:172.21.15.63:3300/0 conn(0x7f43fc04f370 0x7f43fc033aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.669+0000 7fccfb59e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fccf4052fb0 0x7fccf40e3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.669+0000 7fccfb59e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccf40501e0 con 0x7fccf4054090 2024-05-20T18:27:32.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.669+0000 7fccfb59e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccf40504c0 con 0x7fccf4054560 2024-05-20T18:27:32.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.669+0000 7fccfb59e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fccf40507a0 con 0x7fccf4052fb0 2024-05-20T18:27:32.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.670+0000 7fccfa59c640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fccf4054560 0x7fccf4053980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:57492/0 (socket says 172.21.15.5:57492) 2024-05-20T18:27:32.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.670+0000 7fccfa59c640 1 -- 172.21.15.5:0/691123236 learned_addr learned my addr 172.21.15.5:0/691123236 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:32.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.670+0000 7f440905f640 1 -- 172.21.15.5:0/963324593 >> 172.21.15.5:0/963324593 conn(0x7f43fc054d20 msgr2=0x7f43fc055130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:32.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.670+0000 7fccfad9d640 1 --2- 172.21.15.5:0/691123236 >> v2:172.21.15.5:3300/0 conn(0x7fccf4052fb0 0x7fccf40e3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.669+0000 7f8883fff640 1 -- 172.21.15.5:0/1259825745 wait complete. 2024-05-20T18:27:32.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.669+0000 7fd636e6c640 1 Processor -- start 2024-05-20T18:27:32.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.669+0000 7fd636e6c640 1 -- start start 2024-05-20T18:27:32.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.670+0000 7fccfad9d640 1 -- 172.21.15.5:0/691123236 >> v1:172.21.15.63:6789/0 conn(0x7fccf4054560 legacy=0x7fccf4053980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.670+0000 7fccfad9d640 1 -- 172.21.15.5:0/691123236 >> v1:172.21.15.5:6789/0 conn(0x7fccf4054090 legacy=0x7fccf40068e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T18:27:32.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.670+0000 7fccfad9d640 1 -- 172.21.15.5:0/691123236 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf40060e0 con 0x7fccf4052fb0 2024-05-20T18:27:32.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.668+0000 7fc20d573640 1 -- 172.21.15.5:0/4192651247 wait complete. 2024-05-20T18:27:32.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.671+0000 7fc20d573640 1 Processor -- start 2024-05-20T18:27:32.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.672+0000 7f440905f640 1 -- 172.21.15.5:0/963324593 shutdown_connections 2024-05-20T18:27:32.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.672+0000 7fc20d573640 1 -- start start 2024-05-20T18:27:32.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.670+0000 7fe3759ff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3700b1cb0 0x7fe37009c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.670+0000 7fe3759ff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3700b64f0 0x7fe3701595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.670+0000 7fe3759ff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe370165ff0 0x7fe37012d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.670+0000 7fe3759ff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe3700cb550 con 0x7fe3700b64f0 2024-05-20T18:27:32.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.670+0000 7fe3759ff640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe3700b0a70 con 0x7fe3700b1cb0 2024-05-20T18:27:32.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.670+0000 7fe3759ff640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe3700cb6d0 con 0x7fe370165ff0 2024-05-20T18:27:32.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.670+0000 7fe36e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3700b64f0 0x7fe3701595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.670+0000 7fe36e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3700b64f0 0x7fe3701595d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59090/0 (socket says 172.21.15.5:59090) 2024-05-20T18:27:32.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.670+0000 7fe36e7fc640 1 -- 172.21.15.5:0/544665762 learned_addr learned my addr 172.21.15.5:0/544665762 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:32.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.670+0000 7fe36f7fe640 1 --2- 172.21.15.5:0/544665762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe370165ff0 0x7fe37012d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.671+0000 7fe36e7fc640 1 -- 172.21.15.5:0/544665762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe370165ff0 msgr2=0x7fe37012d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.671+0000 7fe36e7fc640 1 --2- 172.21.15.5:0/544665762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe370165ff0 0x7fe37012d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.671+0000 7fe36e7fc640 1 -- 172.21.15.5:0/544665762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3700b1cb0 msgr2=0x7fe37009c510 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-20T18:27:32.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.671+0000 7fe36e7fc640 1 --2- 172.21.15.5:0/544665762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3700b1cb0 0x7fe37009c510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.671+0000 7fe36e7fc640 1 -- 172.21.15.5:0/544665762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe370159b10 con 0x7fe3700b64f0 2024-05-20T18:27:32.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.671+0000 7fe36e7fc640 1 --2- 172.21.15.5:0/544665762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3700b64f0 0x7fe3701595d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe36000c3e0 tx=0x7fe36000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.672+0000 7fe357fff640 1 -- 172.21.15.5:0/544665762 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3600074e0 con 0x7fe3700b64f0 2024-05-20T18:27:32.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.672+0000 7fe357fff640 1 -- 172.21.15.5:0/544665762 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe360007680 con 0x7fe3700b64f0 2024-05-20T18:27:32.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.672+0000 7fe357fff640 1 -- 172.21.15.5:0/544665762 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3600049e0 con 0x7fe3700b64f0 2024-05-20T18:27:32.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.672+0000 7fe3759ff640 1 -- 172.21.15.5:0/544665762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37009af40 con 0x7fe3700b64f0 2024-05-20T18:27:32.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.672+0000 7fe3759ff640 1 -- 172.21.15.5:0/544665762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe37009b750 con 0x7fe3700b64f0 2024-05-20T18:27:32.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.670+0000 7f7290b17640 1 Processor -- start 2024-05-20T18:27:32.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.671+0000 7f7290b17640 1 -- start start 2024-05-20T18:27:32.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.673+0000 7fd636e6c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6300c48f0 0x7fd630166540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.673+0000 7fd636e6c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd630167dd0 0x7fd63015cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.673+0000 7fd636e6c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd63016c4c0 0x7fd63015dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.673+0000 7fd636e6c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd630163a00 con 0x7fd630167dd0 2024-05-20T18:27:32.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.673+0000 7fd636e6c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd6300b2990 con 0x7fd63016c4c0 2024-05-20T18:27:32.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.673+0000 7fd636e6c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd6300b1d70 con 0x7fd6300c48f0 2024-05-20T18:27:32.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.673+0000 7f7290b17640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f728c06ca70 0x7f728c165520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.672+0000 7f440905f640 1 -- 172.21.15.5:0/963324593 wait complete. 2024-05-20T18:27:32.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.673+0000 7f7290b17640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f728c0cfe10 0x7f728c1603c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.673+0000 7f7290b17640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f728c167b90 0x7f728c160900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.673+0000 7f7290b17640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f728c0b5180 con 0x7f728c0cfe10 2024-05-20T18:27:32.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.673+0000 7f7290b17640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f728c0b4ea0 con 0x7f728c06ca70 2024-05-20T18:27:32.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.673+0000 7f7290b17640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f728c0b5980 con 0x7f728c167b90 2024-05-20T18:27:32.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.672+0000 7fccfad9d640 1 --2- 172.21.15.5:0/691123236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf4052fb0 0x7fccf40e3950 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fccfc0504d0 tx=0x7fccfc0a6060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d73910d06a3f165 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.672+0000 7fccf959a640 1 -- 172.21.15.5:0/691123236 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc0b3070 con 0x7fccf4052fb0 2024-05-20T18:27:32.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.673+0000 7fccf959a640 1 -- 172.21.15.5:0/691123236 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccfc0a6c60 con 0x7fccf4052fb0 2024-05-20T18:27:32.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.673+0000 7fccf959a640 1 -- 172.21.15.5:0/691123236 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc0ae680 con 0x7fccf4052fb0 2024-05-20T18:27:32.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.673+0000 7f728b7fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f728c06ca70 0x7f728c165520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.673+0000 7f728b7fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f728c06ca70 0x7f728c165520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:53438/0 (socket says 172.21.15.5:53438) 2024-05-20T18:27:32.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.673+0000 7f728b7fe640 1 -- 172.21.15.5:0/3038902163 learned_addr learned my addr 172.21.15.5:0/3038902163 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:32.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.674+0000 7f728affd640 1 --2- 172.21.15.5:0/3038902163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f728c0cfe10 0x7f728c1603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.674+0000 7f728bfff640 1 --2- 172.21.15.5:0/3038902163 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f728c167b90 0x7f728c160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.672+0000 7fe2a359e640 1 Processor -- start 2024-05-20T18:27:32.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.672+0000 7fe2a359e640 1 -- start start 2024-05-20T18:27:32.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.674+0000 7fe3759ff640 1 -- 172.21.15.5:0/544665762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe370075980 con 0x7fe3700b64f0 2024-05-20T18:27:32.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.674+0000 7fd634c08640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6300c48f0 0x7fd630166540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.672+0000 7f8883fff640 1 Processor -- start 2024-05-20T18:27:32.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.673+0000 7f8883fff640 1 -- start start 2024-05-20T18:27:32.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.674+0000 7f728affd640 1 -- 172.21.15.5:0/3038902163 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f728c167b90 msgr2=0x7f728c160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.674+0000 7f728affd640 1 --2- 172.21.15.5:0/3038902163 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f728c167b90 0x7f728c160900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.674+0000 7f728affd640 1 -- 172.21.15.5:0/3038902163 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f728c06ca70 msgr2=0x7f728c165520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.674+0000 7f728affd640 1 --2- 172.21.15.5:0/3038902163 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f728c06ca70 0x7f728c165520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.674+0000 7f728affd640 1 -- 172.21.15.5:0/3038902163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f728c1eccb0 con 0x7f728c0cfe10 2024-05-20T18:27:32.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.674+0000 7fd635409640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd63016c4c0 0x7fd63015dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.674+0000 7fd635409640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd63016c4c0 0x7fd63015dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:53454/0 (socket says 172.21.15.5:53454) 2024-05-20T18:27:32.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.674+0000 7fd635409640 1 -- 172.21.15.5:0/2552678454 learned_addr learned my addr 172.21.15.5:0/2552678454 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:32.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.674+0000 7f728affd640 1 --2- 172.21.15.5:0/3038902163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f728c0cfe10 0x7f728c1603c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f728000b3e0 tx=0x7f728000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7fd62ffff640 1 --2- 172.21.15.5:0/2552678454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd630167dd0 0x7fd63015cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7fd62ffff640 1 -- 172.21.15.5:0/2552678454 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6300c48f0 msgr2=0x7fd630166540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7fd62ffff640 1 --2- 172.21.15.5:0/2552678454 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6300c48f0 0x7fd630166540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7fd62ffff640 1 -- 172.21.15.5:0/2552678454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd63016c4c0 msgr2=0x7fd63015dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7fd62ffff640 1 --2- 172.21.15.5:0/2552678454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd63016c4c0 0x7fd63015dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7fd62ffff640 1 -- 172.21.15.5:0/2552678454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd630163b80 con 0x7fd630167dd0 2024-05-20T18:27:32.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7fd62ffff640 1 --2- 172.21.15.5:0/2552678454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd630167dd0 0x7fd63015cce0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd62800c6b0 tx=0x7fd62800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7f7273fff640 1 -- 172.21.15.5:0/3038902163 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7280015020 con 0x7f728c0cfe10 2024-05-20T18:27:32.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7f7290b17640 1 -- 172.21.15.5:0/3038902163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f728c1ecf90 con 0x7f728c0cfe10 2024-05-20T18:27:32.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7f7290b17640 1 -- 172.21.15.5:0/3038902163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f728c1ed550 con 0x7f728c0cfe10 2024-05-20T18:27:32.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7f7273fff640 1 -- 172.21.15.5:0/3038902163 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7280011830 con 0x7f728c0cfe10 2024-05-20T18:27:32.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7f440905f640 1 Processor -- start 2024-05-20T18:27:32.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7fe2a359e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe29c033fa0 0x7fe29c0e2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7fe2a359e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe29c052fa0 0x7fe29c0e3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7fe2a359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe29c0438a0 0x7fe29c0da0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7fe2a359e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c006550 con 0x7fe29c0438a0 2024-05-20T18:27:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7fe2a359e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c006270 con 0x7fe29c052fa0 2024-05-20T18:27:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7fe2a359e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe29c006830 con 0x7fe29c033fa0 2024-05-20T18:27:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7f8883fff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f887c033950 0x7f887c17cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7f8883fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f887c048320 0x7f887c186d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7f8883fff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f887c04f340 0x7f887c0e5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7f8883fff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f887c006ab0 con 0x7f887c048320 2024-05-20T18:27:32.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7f8883fff640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f887c047dd0 con 0x7f887c033950 2024-05-20T18:27:32.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7f8883fff640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f887c006740 con 0x7f887c04f340 2024-05-20T18:27:32.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7fccfb59e640 1 -- 172.21.15.5:0/691123236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf4052fb0 msgr2=0x7fccf40e3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7fccfb59e640 1 --2- 172.21.15.5:0/691123236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf4052fb0 0x7fccf40e3950 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fccfc0504d0 tx=0x7fccfc0a6060 comp rx=0 tx=0).stop 2024-05-20T18:27:32.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7f8882ffd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f887c033950 0x7f887c17cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7f8882ffd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f887c033950 0x7f887c17cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:53468/0 (socket says 172.21.15.5:53468) 2024-05-20T18:27:32.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7f8882ffd640 1 -- 172.21.15.5:0/3588628824 learned_addr learned my addr 172.21.15.5:0/3588628824 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:32.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7f88837fe640 1 --2- 172.21.15.5:0/3588628824 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f887c04f340 0x7f887c0e5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7fccfb59e640 1 -- 172.21.15.5:0/691123236 shutdown_connections 2024-05-20T18:27:32.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7fccfb59e640 1 --2- 172.21.15.5:0/691123236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf4052fb0 0x7fccf40e3950 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7fccfb59e640 1 -- 172.21.15.5:0/691123236 >> 172.21.15.5:0/691123236 conn(0x7fccf4054cf0 msgr2=0x7fccf4055100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:32.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.676+0000 7fccfb59e640 1 -- 172.21.15.5:0/691123236 shutdown_connections 2024-05-20T18:27:32.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.676+0000 7fccfb59e640 1 -- 172.21.15.5:0/691123236 wait complete. 2024-05-20T18:27:32.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.676+0000 7fd62d7fa640 1 -- 172.21.15.5:0/2552678454 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd628003b70 con 0x7fd630167dd0 2024-05-20T18:27:32.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.676+0000 7fd636e6c640 1 -- 172.21.15.5:0/2552678454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6300af330 con 0x7fd630167dd0 2024-05-20T18:27:32.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.676+0000 7fd636e6c640 1 -- 172.21.15.5:0/2552678454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd63015f600 con 0x7fd630167dd0 2024-05-20T18:27:32.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.676+0000 7fe2a259c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe29c033fa0 0x7fe29c0e2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.676+0000 7fe2a259c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe29c033fa0 0x7fe29c0e2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:46224/0 (socket says 172.21.15.5:46224) 2024-05-20T18:27:32.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.676+0000 7fe2a259c640 1 -- 172.21.15.5:0/4205473635 learned_addr learned my addr 172.21.15.5:0/4205473635 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:32.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.676+0000 7fe2a1d9b640 1 --2- 172.21.15.5:0/4205473635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe29c052fa0 0x7fe29c0e3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.676+0000 7f8882ffd640 1 -- 172.21.15.5:0/3588628824 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f887c04f340 msgr2=0x7f887c0e5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.676+0000 7f8882ffd640 1 --2- 172.21.15.5:0/3588628824 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f887c04f340 0x7f887c0e5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.676+0000 7f8882ffd640 1 -- 172.21.15.5:0/3588628824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f887c048320 msgr2=0x7f887c186d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T18:27:32.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.676+0000 7f88827fc640 1 --2- 172.21.15.5:0/3588628824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f887c048320 0x7f887c186d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.676+0000 7fc20d573640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc200034130 0x7fc2000dcca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.677+0000 7fe2a2d9d640 1 --2- 172.21.15.5:0/4205473635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe29c0438a0 0x7fe29c0da0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.677+0000 7fe2a259c640 1 -- 172.21.15.5:0/4205473635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe29c052fa0 msgr2=0x7fe29c0e3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.677+0000 7fe2a259c640 1 --2- 172.21.15.5:0/4205473635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe29c052fa0 0x7fe29c0e3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.677+0000 7fe2a259c640 1 -- 172.21.15.5:0/4205473635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe29c0438a0 msgr2=0x7fe29c0da0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.677+0000 7fe2a259c640 1 --2- 172.21.15.5:0/4205473635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe29c0438a0 0x7fe29c0da0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.677+0000 7fe2a259c640 1 -- 172.21.15.5:0/4205473635 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe29c0440b0 con 0x7fe29c033fa0 2024-05-20T18:27:32.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.677+0000 7fc20d573640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc200038a40 0x7fc200186b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.677+0000 7fc20d573640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc20003a500 0x7fc2000e5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.677+0000 7fc207fff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc200034130 0x7fc2000dcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.677+0000 7fc207fff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc200034130 0x7fc2000dcca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:46242/0 (socket says 172.21.15.5:46242) 2024-05-20T18:27:32.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.677+0000 7fc207fff640 1 -- 172.21.15.5:0/2831288730 learned_addr learned my addr 172.21.15.5:0/2831288730 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:32.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.677+0000 7fc20d573640 1 -- 172.21.15.5:0/2831288730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc200033da0 con 0x7fc20003a500 2024-05-20T18:27:32.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.677+0000 7fc20d573640 1 -- 172.21.15.5:0/2831288730 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc200050e80 con 0x7fc200038a40 2024-05-20T18:27:32.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.677+0000 7fc20d573640 1 -- 172.21.15.5:0/2831288730 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc200033f20 con 0x7fc200034130 2024-05-20T18:27:32.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.677+0000 7fe2a259c640 1 --2- 172.21.15.5:0/4205473635 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe29c033fa0 0x7fe29c0e2f10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe29400c7b0 tx=0x7fe29400cc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.677+0000 7f7273fff640 1 -- 172.21.15.5:0/3038902163 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7280011be0 con 0x7f728c0cfe10 2024-05-20T18:27:32.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.676+0000 7fe357fff640 1 -- 172.21.15.5:0/544665762 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe36001a020 con 0x7fe3700b64f0 2024-05-20T18:27:32.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.675+0000 7f440905f640 1 -- start start 2024-05-20T18:27:32.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.678+0000 7f440905f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43fc033fe0 0x7f43fc0e1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.678+0000 7f440905f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43fc04f370 0x7f43fc0dd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.677+0000 7fe357fff640 1 --2- 172.21.15.5:0/544665762 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe35c06be30 0x7fe35c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.677+0000 7fe357fff640 1 -- 172.21.15.5:0/544665762 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe36008a970 con 0x7fe3700b64f0 2024-05-20T18:27:32.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.678+0000 7f440905f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43fc052fe0 0x7f43fc0e36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.677+0000 7fe357fff640 1 --2- 172.21.15.5:0/544665762 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x7fe35c075570 0x7fe35c077a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.677+0000 7fe357fff640 1 -- 172.21.15.5:0/544665762 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe36000ae50 con 0x7fe35c075570 2024-05-20T18:27:32.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.676+0000 7f8882ffd640 1 --2- 172.21.15.5:0/3588628824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f887c048320 0x7f887c186d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.678+0000 7f8882ffd640 1 -- 172.21.15.5:0/3588628824 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f887c187290 con 0x7f887c033950 2024-05-20T18:27:32.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.677+0000 7fe36f7fe640 1 --2- 172.21.15.5:0/544665762 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x7fe35c075570 0x7fe35c077a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.678+0000 7f88827fc640 1 --2- 172.21.15.5:0/3588628824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f887c048320 0x7f887c186d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:32.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.678+0000 7fe36effd640 1 --2- 172.21.15.5:0/544665762 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe35c06be30 0x7fe35c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.679+0000 7f7290b17640 1 -- 172.21.15.5:0/3038902163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f728c075980 con 0x7f728c0cfe10 2024-05-20T18:27:32.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.678+0000 7f440259c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43fc04f370 0x7f43fc0dd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.678+0000 7f440259c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43fc04f370 0x7f43fc0dd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:46246/0 (socket says 172.21.15.5:46246) 2024-05-20T18:27:32.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.678+0000 7f4402d9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43fc033fe0 0x7f43fc0e1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.678+0000 7f4402d9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43fc033fe0 0x7f43fc0e1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:53486/0 (socket says 172.21.15.5:53486) 2024-05-20T18:27:32.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.678+0000 7f440359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43fc052fe0 0x7f43fc0e36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.679+0000 7f440359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43fc052fe0 0x7f43fc0e36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59142/0 (socket says 172.21.15.5:59142) 2024-05-20T18:27:32.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.678+0000 7fc20cd72640 1 --2- 172.21.15.5:0/2831288730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc20003a500 0x7fc2000e5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.678+0000 7f8882ffd640 1 --2- 172.21.15.5:0/3588628824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f887c033950 0x7f887c17cdc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f888402d830 tx=0x7f8884074f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.679+0000 7f440259c640 1 -- 172.21.15.5:0/4162966454 learned_addr learned my addr 172.21.15.5:0/4162966454 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:32.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.677+0000 7fd62d7fa640 1 -- 172.21.15.5:0/2552678454 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd628003d10 con 0x7fd630167dd0 2024-05-20T18:27:32.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.677+0000 7fd62d7fa640 1 -- 172.21.15.5:0/2552678454 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6280052d0 con 0x7fd630167dd0 2024-05-20T18:27:32.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.679+0000 7fc207fff640 1 -- 172.21.15.5:0/2831288730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc200038a40 msgr2=0x7fc200186b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.679+0000 7fc207fff640 1 --2- 172.21.15.5:0/2831288730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc200038a40 0x7fc200186b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.679+0000 7fc207fff640 1 -- 172.21.15.5:0/2831288730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc20003a500 msgr2=0x7fc2000e5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.679+0000 7f440905f640 1 -- 172.21.15.5:0/4162966454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f43fc006590 con 0x7f43fc052fe0 2024-05-20T18:27:32.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.679+0000 7f440905f640 1 -- 172.21.15.5:0/4162966454 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f43fc0062f0 con 0x7f43fc033fe0 2024-05-20T18:27:32.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.679+0000 7f440905f640 1 -- 172.21.15.5:0/4162966454 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f43fc006870 con 0x7f43fc04f370 2024-05-20T18:27:32.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.679+0000 7fc207fff640 1 --2- 172.21.15.5:0/2831288730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc20003a500 0x7fc2000e5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.679+0000 7fc207fff640 1 -- 172.21.15.5:0/2831288730 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2000dd1e0 con 0x7fc200034130 2024-05-20T18:27:32.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.679+0000 7f886b7fe640 1 -- 172.21.15.5:0/3588628824 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888408bd50 con 0x7f887c033950 2024-05-20T18:27:32.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.679+0000 7f886b7fe640 1 -- 172.21.15.5:0/3588628824 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f888408f020 con 0x7f887c033950 2024-05-20T18:27:32.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.679+0000 7f886b7fe640 1 -- 172.21.15.5:0/3588628824 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888406b760 con 0x7f887c033950 2024-05-20T18:27:32.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.679+0000 7f8883fff640 1 -- 172.21.15.5:0/3588628824 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f887c187490 con 0x7f887c033950 2024-05-20T18:27:32.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.679+0000 7f8883fff640 1 -- 172.21.15.5:0/3588628824 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f887c0e9930 con 0x7f887c033950 2024-05-20T18:27:32.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.678+0000 7fe36effd640 1 --2- 172.21.15.5:0/544665762 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe35c06be30 0x7fe35c06e340 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe364009930 tx=0x7fe364009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.679+0000 7fc20cd72640 1 --2- 172.21.15.5:0/2831288730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc20003a500 0x7fc2000e5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:32.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.680+0000 7f7273fff640 1 -- 172.21.15.5:0/3038902163 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7280018020 con 0x7f728c0cfe10 2024-05-20T18:27:32.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.680+0000 7f7273fff640 1 --2- 172.21.15.5:0/3038902163 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f727806be30 0x7f727806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.679+0000 7f4402d9d640 1 -- 172.21.15.5:0/4162966454 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43fc04f370 msgr2=0x7f43fc0dd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.679+0000 7f4402d9d640 1 --2- 172.21.15.5:0/4162966454 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43fc04f370 0x7f43fc0dd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.679+0000 7f4402d9d640 1 -- 172.21.15.5:0/4162966454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43fc052fe0 msgr2=0x7f43fc0e36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.679+0000 7f4402d9d640 1 --2- 172.21.15.5:0/4162966454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43fc052fe0 0x7f43fc0e36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.679+0000 7f4402d9d640 1 -- 172.21.15.5:0/4162966454 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43fc0e1be0 con 0x7f43fc033fe0 2024-05-20T18:27:32.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.680+0000 7f4402d9d640 1 --2- 172.21.15.5:0/4162966454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43fc033fe0 0x7f43fc0e1200 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4404050f00 tx=0x7f4404074f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.680+0000 7fc207fff640 1 --2- 172.21.15.5:0/2831288730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc200034130 0x7fc2000dcca0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc2080652c0 tx=0x7fc2080696a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.680+0000 7f8883fff640 1 -- 172.21.15.5:0/3588628824 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f887c0054a0 con 0x7f887c033950 2024-05-20T18:27:32.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.679+0000 7fe36f7fe640 1 --2- 172.21.15.5:0/544665762 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x7fe35c075570 0x7fe35c077a00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.680+0000 7f43eb7fe640 1 -- 172.21.15.5:0/4162966454 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4404073150 con 0x7f43fc033fe0 2024-05-20T18:27:32.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.680+0000 7f43eb7fe640 1 -- 172.21.15.5:0/4162966454 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f440408f020 con 0x7f43fc033fe0 2024-05-20T18:27:32.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.680+0000 7f43eb7fe640 1 -- 172.21.15.5:0/4162966454 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4404073550 con 0x7f43fc033fe0 2024-05-20T18:27:32.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.680+0000 7f440905f640 1 -- 172.21.15.5:0/4162966454 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43fc0e1ec0 con 0x7f43fc033fe0 2024-05-20T18:27:32.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.680+0000 7f440905f640 1 -- 172.21.15.5:0/4162966454 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43fc0e2480 con 0x7f43fc033fe0 2024-05-20T18:27:32.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.681+0000 7fe357fff640 1 -- 172.21.15.5:0/544665762 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe360054a50 con 0x7fe3700b64f0 2024-05-20T18:27:32.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.681+0000 7fe357fff640 1 -- 172.21.15.5:0/544665762 <== osd.3 v2:172.21.15.5:6824/1164130599 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe36000ae50 con 0x7fe35c075570 2024-05-20T18:27:32.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.680+0000 7fe292ffd640 1 -- 172.21.15.5:0/4205473635 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe294018070 con 0x7fe29c033fa0 2024-05-20T18:27:32.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.680+0000 7fe292ffd640 1 -- 172.21.15.5:0/4205473635 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2940074e0 con 0x7fe29c033fa0 2024-05-20T18:27:32.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.681+0000 7fe292ffd640 1 -- 172.21.15.5:0/4205473635 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29400ce60 con 0x7fe29c033fa0 2024-05-20T18:27:32.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.681+0000 7f886b7fe640 1 -- 172.21.15.5:0/3588628824 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f888406b900 con 0x7f887c033950 2024-05-20T18:27:32.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.681+0000 7f728b7fe640 1 --2- 172.21.15.5:0/3038902163 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f727806be30 0x7f727806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.681+0000 7f7273fff640 1 -- 172.21.15.5:0/3038902163 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f728008b130 con 0x7f728c0cfe10 2024-05-20T18:27:32.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.681+0000 7f7273fff640 1 --2- 172.21.15.5:0/3038902163 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x7f7278075600 0x7f7278077a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.682+0000 7f7273fff640 1 -- 172.21.15.5:0/3038902163 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7280011830 con 0x7f7278075600 2024-05-20T18:27:32.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.682+0000 7f7273fff640 1 -- 172.21.15.5:0/3038902163 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7280092050 con 0x7f728c0cfe10 2024-05-20T18:27:32.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.680+0000 7fccfb59e640 1 Processor -- start 2024-05-20T18:27:32.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.680+0000 7fccfb59e640 1 -- start start 2024-05-20T18:27:32.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.682+0000 7fc204ff9640 1 -- 172.21.15.5:0/2831288730 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2080b3050 con 0x7fc200034130 2024-05-20T18:27:32.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.682+0000 7f43e97fa640 1 -- 172.21.15.5:0/4162966454 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f43fc0054d0 con 0x7f43fc033fe0 2024-05-20T18:27:32.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.679+0000 7fd636e6c640 1 -- 172.21.15.5:0/2552678454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd610001030 con 0x7fd630167dd0 2024-05-20T18:27:32.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.680+0000 7fd62d7fa640 1 -- 172.21.15.5:0/2552678454 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd628005470 con 0x7fd630167dd0 2024-05-20T18:27:32.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.681+0000 7fd62d7fa640 1 --2- 172.21.15.5:0/2552678454 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd61c06be30 0x7fd61c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.681+0000 7fd62d7fa640 1 -- 172.21.15.5:0/2552678454 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd62808b080 con 0x7fd630167dd0 2024-05-20T18:27:32.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.681+0000 7fd62d7fa640 1 --2- 172.21.15.5:0/2552678454 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x7fd61c075570 0x7fd61c077a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.681+0000 7fd62d7fa640 1 -- 172.21.15.5:0/2552678454 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd628015ec0 con 0x7fd61c075570 2024-05-20T18:27:32.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.681+0000 7fd634c08640 1 --2- 172.21.15.5:0/2552678454 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd61c06be30 0x7fd61c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.682+0000 7fd62d7fa640 1 -- 172.21.15.5:0/2552678454 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd628092050 con 0x7fd630167dd0 2024-05-20T18:27:32.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.682+0000 7fc204ff9640 1 -- 172.21.15.5:0/2831288730 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2080b4040 con 0x7fc200034130 2024-05-20T18:27:32.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.682+0000 7fc204ff9640 1 -- 172.21.15.5:0/2831288730 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2080b8610 con 0x7fc200034130 2024-05-20T18:27:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.682+0000 7fe2a359e640 1 -- 172.21.15.5:0/4205473635 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe29c044390 con 0x7fe29c033fa0 2024-05-20T18:27:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.682+0000 7fe2a359e640 1 -- 172.21.15.5:0/4205473635 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe29c0d4d40 con 0x7fe29c033fa0 2024-05-20T18:27:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.682+0000 7fccfb59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf4052fb0 0x7fccf41803d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.682+0000 7fccfb59e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fccf4054560 0x7fccf40d9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.682+0000 7fccfb59e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fccf40da400 0x7fccf40e6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.682+0000 7fccfb59e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fccf404fd50 con 0x7fccf4052fb0 2024-05-20T18:27:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.682+0000 7fccfb59e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fccf4038340 con 0x7fccf4054560 2024-05-20T18:27:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.682+0000 7fccfb59e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fccf404fed0 con 0x7fccf40da400 2024-05-20T18:27:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.682+0000 7fd635409640 1 --2- 172.21.15.5:0/2552678454 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x7fd61c075570 0x7fd61c077a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.682+0000 7f728bfff640 1 --2- 172.21.15.5:0/3038902163 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x7f7278075600 0x7f7278077a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.682+0000 7fc20d573640 1 -- 172.21.15.5:0/2831288730 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2000e2880 con 0x7fc200034130 2024-05-20T18:27:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.682+0000 7fc20d573640 1 -- 172.21.15.5:0/2831288730 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2000e2e40 con 0x7fc200034130 2024-05-20T18:27:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.682+0000 7f886b7fe640 1 --2- 172.21.15.5:0/3588628824 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f887006bf00 0x7f887006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.682+0000 7f886b7fe640 1 -- 172.21.15.5:0/3588628824 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f8884105d80 con 0x7f887c033950 2024-05-20T18:27:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.682+0000 7f886b7fe640 1 --2- 172.21.15.5:0/3588628824 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x7f88700756d0 0x7f8870077b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.682+0000 7f886b7fe640 1 -- 172.21.15.5:0/3588628824 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8870078230 con 0x7f88700756d0 2024-05-20T18:27:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.682+0000 7f886b7fe640 1 -- 172.21.15.5:0/3588628824 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f88841083e0 con 0x7f887c033950 2024-05-20T18:27:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.683+0000 7f88827fc640 1 --2- 172.21.15.5:0/3588628824 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f887006bf00 0x7f887006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.682+0000 7fccfad9d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fccf40da400 0x7fccf40e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.684+0000 7f88837fe640 1 --2- 172.21.15.5:0/3588628824 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x7f88700756d0 0x7f8870077b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.684+0000 7f88827fc640 1 --2- 172.21.15.5:0/3588628824 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f887006bf00 0x7f887006e410 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8878007fd0 tx=0x7f8878009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.682+0000 7fccfad9d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fccf40da400 0x7fccf40e6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:46250/0 (socket says 172.21.15.5:46250) 2024-05-20T18:27:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.682+0000 7fccfad9d640 1 -- 172.21.15.5:0/1054935186 learned_addr learned my addr 172.21.15.5:0/1054935186 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.682+0000 7fccf9d9b640 1 --2- 172.21.15.5:0/1054935186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fccf4054560 0x7fccf40d9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.683+0000 7fccfad9d640 1 -- 172.21.15.5:0/1054935186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fccf4054560 msgr2=0x7fccf40d9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.683+0000 7fccfad9d640 1 --2- 172.21.15.5:0/1054935186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fccf4054560 0x7fccf40d9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.683+0000 7fccfad9d640 1 -- 172.21.15.5:0/1054935186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf4052fb0 msgr2=0x7fccf41803d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.683+0000 7fccfad9d640 1 --2- 172.21.15.5:0/1054935186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf4052fb0 0x7fccf41803d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.683+0000 7fccfad9d640 1 -- 172.21.15.5:0/1054935186 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf4186490 con 0x7fccf40da400 2024-05-20T18:27:32.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.683+0000 7fccfad9d640 1 --2- 172.21.15.5:0/1054935186 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fccf40da400 0x7fccf40e6100 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fccfc064fc0 tx=0x7fccfc074640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.683+0000 7fcceaffd640 1 -- 172.21.15.5:0/1054935186 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc0b3040 con 0x7fccf40da400 2024-05-20T18:27:32.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.683+0000 7fcceaffd640 1 -- 172.21.15.5:0/1054935186 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccfc0b8660 con 0x7fccf40da400 2024-05-20T18:27:32.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.684+0000 7fcceaffd640 1 -- 172.21.15.5:0/1054935186 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc0b8a10 con 0x7fccf40da400 2024-05-20T18:27:32.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.684+0000 7fc20d573640 1 -- 172.21.15.5:0/2831288730 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc2000e2630 con 0x7fc200034130 2024-05-20T18:27:32.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.683+0000 7fd635409640 1 --2- 172.21.15.5:0/2552678454 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x7fd61c075570 0x7fd61c077a00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.684+0000 7fd634c08640 1 --2- 172.21.15.5:0/2552678454 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd61c06be30 0x7fd61c06e340 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd624007fd0 tx=0x7fd624009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.684+0000 7fd62d7fa640 1 -- 172.21.15.5:0/2552678454 <== osd.0 v2:172.21.15.5:6808/688093005 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd628015ec0 con 0x7fd61c075570 2024-05-20T18:27:32.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.684+0000 7fccfb59e640 1 -- 172.21.15.5:0/1054935186 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf4186760 con 0x7fccf40da400 2024-05-20T18:27:32.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.684+0000 7fccfb59e640 1 -- 172.21.15.5:0/1054935186 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf4186d20 con 0x7fccf40da400 2024-05-20T18:27:32.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.684+0000 7f728b7fe640 1 --2- 172.21.15.5:0/3038902163 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f727806be30 0x7f727806e340 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f727c035040 tx=0x7f727c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.684+0000 7fe2a359e640 1 -- 172.21.15.5:0/4205473635 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe29c005490 con 0x7fe29c033fa0 2024-05-20T18:27:32.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.684+0000 7f728bfff640 1 --2- 172.21.15.5:0/3038902163 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x7f7278075600 0x7f7278077a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.685+0000 7fe292ffd640 1 -- 172.21.15.5:0/4205473635 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe29401b070 con 0x7fe29c033fa0 2024-05-20T18:27:32.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.685+0000 7fc204ff9640 1 -- 172.21.15.5:0/2831288730 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc2080c1070 con 0x7fc200034130 2024-05-20T18:27:32.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.686+0000 7fe292ffd640 1 --2- 172.21.15.5:0/4205473635 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe28406beb0 0x7fe28406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.686+0000 7f7273fff640 1 -- 172.21.15.5:0/3038902163 <== osd.1 v2:172.21.15.5:6815/550664725 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7280011830 con 0x7f7278075600 2024-05-20T18:27:32.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.684+0000 7f43eb7fe640 1 -- 172.21.15.5:0/4162966454 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f44040736f0 con 0x7f43fc033fe0 2024-05-20T18:27:32.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.684+0000 7f43eb7fe640 1 --2- 172.21.15.5:0/4162966454 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f43f006bf00 0x7f43f006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.685+0000 7f440259c640 1 --2- 172.21.15.5:0/4162966454 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f43f006bf00 0x7f43f006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.685+0000 7f440259c640 1 --2- 172.21.15.5:0/4162966454 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f43f006bf00 0x7f43f006e410 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f43fc0e10b0 tx=0x7f43f4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.686+0000 7f43eb7fe640 1 -- 172.21.15.5:0/4162966454 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f4404105d30 con 0x7f43fc033fe0 2024-05-20T18:27:32.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.686+0000 7f43eb7fe640 1 --2- 172.21.15.5:0/4162966454 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x7f43f0075640 0x7f43f0077ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.686+0000 7f43eb7fe640 1 -- 172.21.15.5:0/4162966454 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f43f00781a0 con 0x7f43f0075640 2024-05-20T18:27:32.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.686+0000 7f43eb7fe640 1 -- 172.21.15.5:0/4162966454 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4404108390 con 0x7f43fc033fe0 2024-05-20T18:27:32.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.686+0000 7fc204ff9640 1 --2- 172.21.15.5:0/2831288730 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc1ec06be80 0x7fc1ec06e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.686+0000 7fe292ffd640 1 -- 172.21.15.5:0/4205473635 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe294056850 con 0x7fe29c033fa0 2024-05-20T18:27:32.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.686+0000 7fe292ffd640 1 --2- 172.21.15.5:0/4205473635 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x7fe2840762d0 0x7fe2840787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.686+0000 7fe292ffd640 1 -- 172.21.15.5:0/4205473635 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe2940074e0 con 0x7fe2840762d0 2024-05-20T18:27:32.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.686+0000 7fc204ff9640 1 -- 172.21.15.5:0/2831288730 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc208126930 con 0x7fc200034130 2024-05-20T18:27:32.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.686+0000 7fc2077fe640 1 --2- 172.21.15.5:0/2831288730 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc1ec06be80 0x7fc1ec06e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.686+0000 7fccfb59e640 1 -- 172.21.15.5:0/1054935186 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fccf40054a0 con 0x7fccf40da400 2024-05-20T18:27:32.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.686+0000 7fe2a1d9b640 1 --2- 172.21.15.5:0/4205473635 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe28406beb0 0x7fe28406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.686+0000 7fc204ff9640 1 --2- 172.21.15.5:0/2831288730 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x7fc1ec075640 0x7fc1ec077ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.687+0000 7f88837fe640 1 --2- 172.21.15.5:0/3588628824 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x7f88700756d0 0x7f8870077b60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.687+0000 7fe292ffd640 1 -- 172.21.15.5:0/4205473635 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe2940551e0 con 0x7fe29c033fa0 2024-05-20T18:27:32.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.686+0000 7fc204ff9640 1 -- 172.21.15.5:0/2831288730 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc2080aaec0 con 0x7fc1ec075640 2024-05-20T18:27:32.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.687+0000 7fc204ff9640 1 -- 172.21.15.5:0/2831288730 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc20812b2a0 con 0x7fc200034130 2024-05-20T18:27:32.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.687+0000 7fe2a2d9d640 1 --2- 172.21.15.5:0/4205473635 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x7fe2840762d0 0x7fe2840787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.687+0000 7f440359e640 1 --2- 172.21.15.5:0/4162966454 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x7f43f0075640 0x7f43f0077ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.687+0000 7fe2a1d9b640 1 --2- 172.21.15.5:0/4205473635 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe28406beb0 0x7fe28406e3c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe298034000 tx=0x7fe29802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.687+0000 7fc2077fe640 1 --2- 172.21.15.5:0/2831288730 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc1ec06be80 0x7fc1ec06e390 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc006a90 tx=0x7fc1fc006a20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.687+0000 7fcceaffd640 1 -- 172.21.15.5:0/1054935186 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fccfc0b6070 con 0x7fccf40da400 2024-05-20T18:27:32.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.687+0000 7fe2a2d9d640 1 --2- 172.21.15.5:0/4205473635 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x7fe2840762d0 0x7fe2840787c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.688+0000 7f886b7fe640 1 -- 172.21.15.5:0/3588628824 <== osd.7 v2:172.21.15.63:6824/4136124451 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8870078230 con 0x7f88700756d0 2024-05-20T18:27:32.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.688+0000 7fc20cd72640 1 --2- 172.21.15.5:0/2831288730 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x7fc1ec075640 0x7fc1ec077ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.688+0000 7fcceaffd640 1 --2- 172.21.15.5:0/1054935186 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fccdc06be80 0x7fccdc06e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.688+0000 7f440359e640 1 --2- 172.21.15.5:0/4162966454 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x7f43f0075640 0x7f43f0077ad0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.688+0000 7fcceaffd640 1 -- 172.21.15.5:0/1054935186 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fccfc127170 con 0x7fccf40da400 2024-05-20T18:27:32.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.688+0000 7fccfa59c640 1 --2- 172.21.15.5:0/1054935186 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fccdc06be80 0x7fccdc06e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.688+0000 7fcceaffd640 1 --2- 172.21.15.5:0/1054935186 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x7fccdc075650 0x7fccdc077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:32.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.689+0000 7fcceaffd640 1 -- 172.21.15.5:0/1054935186 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fccfc062f90 con 0x7fccdc075650 2024-05-20T18:27:32.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.689+0000 7fcceaffd640 1 -- 172.21.15.5:0/1054935186 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fccfc12e050 con 0x7fccf40da400 2024-05-20T18:27:32.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.689+0000 7fc20cd72640 1 --2- 172.21.15.5:0/2831288730 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x7fc1ec075640 0x7fc1ec077ad0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.689+0000 7fccf9d9b640 1 --2- 172.21.15.5:0/1054935186 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x7fccdc075650 0x7fccdc077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:32.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.689+0000 7fe292ffd640 1 -- 172.21.15.5:0/4205473635 <== osd.4 v2:172.21.15.63:6800/3673324406 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe2940074e0 con 0x7fe2840762d0 2024-05-20T18:27:32.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.689+0000 7fccfa59c640 1 --2- 172.21.15.5:0/1054935186 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fccdc06be80 0x7fccdc06e390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fccec0015a0 tx=0x7fccec007a40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.689+0000 7f43eb7fe640 1 -- 172.21.15.5:0/4162966454 <== osd.2 v2:172.21.15.5:6800/921679764 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f43f00781a0 con 0x7f43f0075640 2024-05-20T18:27:32.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.690+0000 7fccf9d9b640 1 --2- 172.21.15.5:0/1054935186 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x7fccdc075650 0x7fccdc077ae0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:32.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.690+0000 7fc204ff9640 1 -- 172.21.15.5:0/2831288730 <== osd.6 v2:172.21.15.63:6816/3384565140 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc2080aaec0 con 0x7fc1ec075640 2024-05-20T18:27:32.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.690+0000 7fcceaffd640 1 -- 172.21.15.5:0/1054935186 <== osd.5 v2:172.21.15.63:6808/1616066286 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fccfc062f90 con 0x7fccdc075650 2024-05-20T18:27:32.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.713+0000 7fe3759ff640 1 -- 172.21.15.5:0/544665762 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe37009b220 con 0x7fe35c075570 2024-05-20T18:27:32.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.714+0000 7fe357fff640 1 -- 172.21.15.5:0/544665762 <== osd.3 v2:172.21.15.5:6824/1164130599 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe37009b220 con 0x7fe35c075570 2024-05-20T18:27:32.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.714+0000 7fe355ffb640 1 -- 172.21.15.5:0/544665762 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x7fe35c075570 msgr2=0x7fe35c077a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.714+0000 7fe355ffb640 1 --2- 172.21.15.5:0/544665762 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x7fe35c075570 0x7fe35c077a00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.714+0000 7fe355ffb640 1 -- 172.21.15.5:0/544665762 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe35c06be30 msgr2=0x7fe35c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.715+0000 7fe355ffb640 1 --2- 172.21.15.5:0/544665762 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe35c06be30 0x7fe35c06e340 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe364009930 tx=0x7fe364009190 comp rx=0 tx=0).stop 2024-05-20T18:27:32.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.715+0000 7fe355ffb640 1 -- 172.21.15.5:0/544665762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3700b64f0 msgr2=0x7fe3701595d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.715+0000 7fe355ffb640 1 --2- 172.21.15.5:0/544665762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3700b64f0 0x7fe3701595d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe36000c3e0 tx=0x7fe36000c8a0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.715+0000 7fe36f7fe640 1 -- 172.21.15.5:0/544665762 reap_dead start 2024-05-20T18:27:32.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.715+0000 7fe355ffb640 1 -- 172.21.15.5:0/544665762 shutdown_connections 2024-05-20T18:27:32.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.715+0000 7fe355ffb640 1 -- 172.21.15.5:0/544665762 >> 172.21.15.5:0/544665762 conn(0x7fe37009a4f0 msgr2=0x7fe3700d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:32.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.715+0000 7fe355ffb640 1 -- 172.21.15.5:0/544665762 shutdown_connections 2024-05-20T18:27:32.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.717+0000 7fe355ffb640 1 -- 172.21.15.5:0/544665762 wait complete. 2024-05-20T18:27:32.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.717+0000 7fd636e6c640 1 -- 172.21.15.5:0/2552678454 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd610002d50 con 0x7fd61c075570 2024-05-20T18:27:32.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.718+0000 7fd62d7fa640 1 -- 172.21.15.5:0/2552678454 <== osd.0 v2:172.21.15.5:6808/688093005 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd610002d50 con 0x7fd61c075570 2024-05-20T18:27:32.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.718+0000 7f7290b17640 1 -- 172.21.15.5:0/3038902163 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f728c165b20 con 0x7f7278075600 2024-05-20T18:27:32.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.719+0000 7fd636e6c640 1 -- 172.21.15.5:0/2552678454 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x7fd61c075570 msgr2=0x7fd61c077a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.719+0000 7fd636e6c640 1 --2- 172.21.15.5:0/2552678454 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x7fd61c075570 0x7fd61c077a00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.719+0000 7f7273fff640 1 -- 172.21.15.5:0/3038902163 <== osd.1 v2:172.21.15.5:6815/550664725 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f728c165b20 con 0x7f7278075600 2024-05-20T18:27:32.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.719+0000 7fe2a359e640 1 -- 172.21.15.5:0/4205473635 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe29c007560 con 0x7fe2840762d0 2024-05-20T18:27:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.720+0000 7fd636e6c640 1 -- 172.21.15.5:0/2552678454 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd61c06be30 msgr2=0x7fd61c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.720+0000 7fd636e6c640 1 --2- 172.21.15.5:0/2552678454 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd61c06be30 0x7fd61c06e340 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd624007fd0 tx=0x7fd624009040 comp rx=0 tx=0).stop 2024-05-20T18:27:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.720+0000 7f7271ffb640 1 -- 172.21.15.5:0/3038902163 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x7f7278075600 msgr2=0x7f7278077a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.720+0000 7f7271ffb640 1 --2- 172.21.15.5:0/3038902163 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x7f7278075600 0x7f7278077a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.720+0000 7fd636e6c640 1 -- 172.21.15.5:0/2552678454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd630167dd0 msgr2=0x7fd63015cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.720+0000 7fd636e6c640 1 --2- 172.21.15.5:0/2552678454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd630167dd0 0x7fd63015cce0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd62800c6b0 tx=0x7fd62800cb70 comp rx=0 tx=0).stop 2024-05-20T18:27:32.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.720+0000 7f7271ffb640 1 -- 172.21.15.5:0/3038902163 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f727806be30 msgr2=0x7f727806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.720+0000 7f7271ffb640 1 --2- 172.21.15.5:0/3038902163 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f727806be30 0x7f727806e340 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f727c035040 tx=0x7f727c02e000 comp rx=0 tx=0).stop 2024-05-20T18:27:32.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.720+0000 7f7271ffb640 1 -- 172.21.15.5:0/3038902163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f728c0cfe10 msgr2=0x7f728c1603c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.720+0000 7f7271ffb640 1 --2- 172.21.15.5:0/3038902163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f728c0cfe10 0x7f728c1603c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f728000b3e0 tx=0x7f728000b8a0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.720+0000 7f728bfff640 1 -- 172.21.15.5:0/3038902163 reap_dead start 2024-05-20T18:27:32.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.720+0000 7f7271ffb640 1 -- 172.21.15.5:0/3038902163 shutdown_connections 2024-05-20T18:27:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.720+0000 7f7271ffb640 1 -- 172.21.15.5:0/3038902163 >> 172.21.15.5:0/3038902163 conn(0x7f728c0721d0 msgr2=0x7f728c0725e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.720+0000 7fe292ffd640 1 -- 172.21.15.5:0/4205473635 <== osd.4 v2:172.21.15.63:6800/3673324406 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe29c007560 con 0x7fe2840762d0 2024-05-20T18:27:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.720+0000 7fd635409640 1 -- 172.21.15.5:0/2552678454 reap_dead start 2024-05-20T18:27:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.720+0000 7fd636e6c640 1 -- 172.21.15.5:0/2552678454 shutdown_connections 2024-05-20T18:27:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.720+0000 7fd636e6c640 1 -- 172.21.15.5:0/2552678454 >> 172.21.15.5:0/2552678454 conn(0x7fd6300b6620 msgr2=0x7fd6300d1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:32.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.721+0000 7fd636e6c640 1 -- 172.21.15.5:0/2552678454 shutdown_connections 2024-05-20T18:27:32.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.721+0000 7fd636e6c640 1 -- 172.21.15.5:0/2552678454 wait complete. 2024-05-20T18:27:32.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.721+0000 7f7271ffb640 1 -- 172.21.15.5:0/3038902163 shutdown_connections 2024-05-20T18:27:32.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.721+0000 7f8883fff640 1 -- 172.21.15.5:0/3588628824 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f887c0df1e0 con 0x7f88700756d0 2024-05-20T18:27:32.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.721+0000 7f7271ffb640 1 -- 172.21.15.5:0/3038902163 wait complete. 2024-05-20T18:27:32.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.721+0000 7fe2a359e640 1 -- 172.21.15.5:0/4205473635 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x7fe2840762d0 msgr2=0x7fe2840787c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.721+0000 7fe2a359e640 1 --2- 172.21.15.5:0/4205473635 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x7fe2840762d0 0x7fe2840787c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.722+0000 7fe2a359e640 1 -- 172.21.15.5:0/4205473635 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe28406beb0 msgr2=0x7fe28406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.722+0000 7fe2a359e640 1 --2- 172.21.15.5:0/4205473635 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe28406beb0 0x7fe28406e3c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe298034000 tx=0x7fe29802e000 comp rx=0 tx=0).stop 2024-05-20T18:27:32.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.722+0000 7fe2a359e640 1 -- 172.21.15.5:0/4205473635 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe29c033fa0 msgr2=0x7fe29c0e2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.722+0000 7fe2a359e640 1 --2- 172.21.15.5:0/4205473635 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe29c033fa0 0x7fe29c0e2f10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe29400c7b0 tx=0x7fe29400cc70 comp rx=0 tx=0).stop 2024-05-20T18:27:32.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.722+0000 7fe2a2d9d640 1 -- 172.21.15.5:0/4205473635 reap_dead start 2024-05-20T18:27:32.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.722+0000 7f886b7fe640 1 -- 172.21.15.5:0/3588628824 <== osd.7 v2:172.21.15.63:6824/4136124451 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f887c0df1e0 con 0x7f88700756d0 2024-05-20T18:27:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.722+0000 7fe2a359e640 1 -- 172.21.15.5:0/4205473635 shutdown_connections 2024-05-20T18:27:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.722+0000 7fe2a359e640 1 -- 172.21.15.5:0/4205473635 >> 172.21.15.5:0/4205473635 conn(0x7fe29c054ce0 msgr2=0x7fe29c0382f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.722+0000 7fe2a359e640 1 -- 172.21.15.5:0/4205473635 shutdown_connections 2024-05-20T18:27:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.722+0000 7fe2a359e640 1 -- 172.21.15.5:0/4205473635 wait complete. 2024-05-20T18:27:32.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.722+0000 7f43e97fa640 1 -- 172.21.15.5:0/4162966454 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f43fc03a5f0 con 0x7f43f0075640 2024-05-20T18:27:32.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.723+0000 7f8883fff640 1 -- 172.21.15.5:0/3588628824 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x7f88700756d0 msgr2=0x7f8870077b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.723+0000 7f8883fff640 1 --2- 172.21.15.5:0/3588628824 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x7f88700756d0 0x7f8870077b60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.723+0000 7f8883fff640 1 -- 172.21.15.5:0/3588628824 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f887006bf00 msgr2=0x7f887006e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.723+0000 7f8883fff640 1 --2- 172.21.15.5:0/3588628824 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f887006bf00 0x7f887006e410 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8878007fd0 tx=0x7f8878009040 comp rx=0 tx=0).stop 2024-05-20T18:27:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.723+0000 7f8883fff640 1 -- 172.21.15.5:0/3588628824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f887c033950 msgr2=0x7f887c17cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.723+0000 7f8883fff640 1 --2- 172.21.15.5:0/3588628824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f887c033950 0x7f887c17cdc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f888402d830 tx=0x7f8884074f20 comp rx=0 tx=0).stop 2024-05-20T18:27:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.723+0000 7f88837fe640 1 -- 172.21.15.5:0/3588628824 reap_dead start 2024-05-20T18:27:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.723+0000 7f8883fff640 1 -- 172.21.15.5:0/3588628824 shutdown_connections 2024-05-20T18:27:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.723+0000 7f8883fff640 1 -- 172.21.15.5:0/3588628824 >> 172.21.15.5:0/3588628824 conn(0x7f887c054cf0 msgr2=0x7f887c0390a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.723+0000 7f43eb7fe640 1 -- 172.21.15.5:0/4162966454 <== osd.2 v2:172.21.15.5:6800/921679764 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f43fc03a5f0 con 0x7f43f0075640 2024-05-20T18:27:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.723+0000 7f8883fff640 1 -- 172.21.15.5:0/3588628824 shutdown_connections 2024-05-20T18:27:32.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.723+0000 7f8883fff640 1 -- 172.21.15.5:0/3588628824 wait complete. 2024-05-20T18:27:32.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.724+0000 7f43e97fa640 1 -- 172.21.15.5:0/4162966454 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x7f43f0075640 msgr2=0x7f43f0077ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.724+0000 7f43e97fa640 1 --2- 172.21.15.5:0/4162966454 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x7f43f0075640 0x7f43f0077ad0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.724+0000 7fc20d573640 1 -- 172.21.15.5:0/2831288730 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc200005490 con 0x7fc1ec075640 2024-05-20T18:27:32.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.724+0000 7fccfb59e640 1 -- 172.21.15.5:0/1054935186 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fccf4037c60 con 0x7fccdc075650 2024-05-20T18:27:32.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.724+0000 7f43e97fa640 1 -- 172.21.15.5:0/4162966454 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f43f006bf00 msgr2=0x7f43f006e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.724+0000 7f43e97fa640 1 --2- 172.21.15.5:0/4162966454 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f43f006bf00 0x7f43f006e410 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f43fc0e10b0 tx=0x7f43f4009190 comp rx=0 tx=0).stop 2024-05-20T18:27:32.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.724+0000 7fc204ff9640 1 -- 172.21.15.5:0/2831288730 <== osd.6 v2:172.21.15.63:6816/3384565140 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc200005490 con 0x7fc1ec075640 2024-05-20T18:27:32.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.725+0000 7f43e97fa640 1 -- 172.21.15.5:0/4162966454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43fc033fe0 msgr2=0x7f43fc0e1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.725+0000 7f43e97fa640 1 --2- 172.21.15.5:0/4162966454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43fc033fe0 0x7f43fc0e1200 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4404050f00 tx=0x7f4404074f20 comp rx=0 tx=0).stop 2024-05-20T18:27:32.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.725+0000 7f440359e640 1 -- 172.21.15.5:0/4162966454 reap_dead start 2024-05-20T18:27:32.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.725+0000 7f43e97fa640 1 -- 172.21.15.5:0/4162966454 shutdown_connections 2024-05-20T18:27:32.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.725+0000 7f43e97fa640 1 -- 172.21.15.5:0/4162966454 >> 172.21.15.5:0/4162966454 conn(0x7f43fc054d20 msgr2=0x7f43fc055130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:32.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.725+0000 7fcceaffd640 1 -- 172.21.15.5:0/1054935186 <== osd.5 v2:172.21.15.63:6808/1616066286 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fccf4037c60 con 0x7fccdc075650 2024-05-20T18:27:32.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.725+0000 7f43e97fa640 1 -- 172.21.15.5:0/4162966454 shutdown_connections 2024-05-20T18:27:32.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.725+0000 7f43e97fa640 1 -- 172.21.15.5:0/4162966454 wait complete. 2024-05-20T18:27:32.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.725+0000 7fc20d573640 1 -- 172.21.15.5:0/2831288730 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x7fc1ec075640 msgr2=0x7fc1ec077ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.725+0000 7fc20d573640 1 --2- 172.21.15.5:0/2831288730 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x7fc1ec075640 0x7fc1ec077ad0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.726+0000 7fc20d573640 1 -- 172.21.15.5:0/2831288730 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc1ec06be80 msgr2=0x7fc1ec06e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.726+0000 7fc20d573640 1 --2- 172.21.15.5:0/2831288730 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc1ec06be80 0x7fc1ec06e390 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc006a90 tx=0x7fc1fc006a20 comp rx=0 tx=0).stop 2024-05-20T18:27:32.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.726+0000 7fc20d573640 1 -- 172.21.15.5:0/2831288730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc200034130 msgr2=0x7fc2000dcca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.726+0000 7fc20d573640 1 --2- 172.21.15.5:0/2831288730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc200034130 0x7fc2000dcca0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc2080652c0 tx=0x7fc2080696a0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.726+0000 7fc20cd72640 1 -- 172.21.15.5:0/2831288730 reap_dead start 2024-05-20T18:27:32.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.725+0000 7fccfb59e640 1 -- 172.21.15.5:0/1054935186 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x7fccdc075650 msgr2=0x7fccdc077ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.726+0000 7fccfb59e640 1 --2- 172.21.15.5:0/1054935186 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x7fccdc075650 0x7fccdc077ae0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:32.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.726+0000 7fc20d573640 1 -- 172.21.15.5:0/2831288730 shutdown_connections 2024-05-20T18:27:32.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.726+0000 7fc20d573640 1 -- 172.21.15.5:0/2831288730 >> 172.21.15.5:0/2831288730 conn(0x7fc200054ce0 msgr2=0x7fc2000df710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:32.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.726+0000 7fccfb59e640 1 -- 172.21.15.5:0/1054935186 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fccdc06be80 msgr2=0x7fccdc06e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.726+0000 7fccfb59e640 1 --2- 172.21.15.5:0/1054935186 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fccdc06be80 0x7fccdc06e390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fccec0015a0 tx=0x7fccec007a40 comp rx=0 tx=0).stop 2024-05-20T18:27:32.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.726+0000 7fccfb59e640 1 -- 172.21.15.5:0/1054935186 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fccf40da400 msgr2=0x7fccf40e6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:32.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.726+0000 7fccfb59e640 1 --2- 172.21.15.5:0/1054935186 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fccf40da400 0x7fccf40e6100 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fccfc064fc0 tx=0x7fccfc074640 comp rx=0 tx=0).stop 2024-05-20T18:27:32.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.726+0000 7fc20d573640 1 -- 172.21.15.5:0/2831288730 shutdown_connections 2024-05-20T18:27:32.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.727+0000 7fccfad9d640 1 -- 172.21.15.5:0/1054935186 reap_dead start 2024-05-20T18:27:32.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.727+0000 7fccfb59e640 1 -- 172.21.15.5:0/1054935186 shutdown_connections 2024-05-20T18:27:32.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.727+0000 7fccfb59e640 1 -- 172.21.15.5:0/1054935186 >> 172.21.15.5:0/1054935186 conn(0x7fccf4054cf0 msgr2=0x7fccf40e1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:32.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.727+0000 7fc20d573640 1 -- 172.21.15.5:0/2831288730 wait complete. 2024-05-20T18:27:32.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.727+0000 7fccfb59e640 1 -- 172.21.15.5:0/1054935186 shutdown_connections 2024-05-20T18:27:32.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:32.727+0000 7fccfb59e640 1 -- 172.21.15.5:0/1054935186 wait complete. 2024-05-20T18:27:32.777 INFO:teuthology.orchestra.run.smithi005.stdout:55834574858 2024-05-20T18:27:32.777 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-20T18:27:32.778 INFO:teuthology.orchestra.run.smithi005.stdout:55834574858 2024-05-20T18:27:32.778 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-20T18:27:32.779 INFO:teuthology.orchestra.run.smithi005.stdout:55834574857 2024-05-20T18:27:32.779 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-20T18:27:32.780 INFO:teuthology.orchestra.run.smithi005.stdout:55834574856 2024-05-20T18:27:32.780 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-20T18:27:32.781 INFO:teuthology.orchestra.run.smithi005.stdout:55834574858 2024-05-20T18:27:32.781 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-20T18:27:32.782 INFO:teuthology.orchestra.run.smithi005.stdout:55834574856 2024-05-20T18:27:32.782 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-20T18:27:32.783 INFO:teuthology.orchestra.run.smithi005.stdout:55834574856 2024-05-20T18:27:32.783 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-20T18:27:32.784 INFO:teuthology.orchestra.run.smithi005.stdout:55834574857 2024-05-20T18:27:32.784 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-20T18:27:33.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.037+0000 7fe7d98ca640 1 Processor -- start 2024-05-20T18:27:33.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.037+0000 7fe7d98ca640 1 -- start start 2024-05-20T18:27:33.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.040+0000 7fe7d98ca640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe7d4143620 0x7fe7d4161320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.040+0000 7fe7d98ca640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7d415dd50 con 0x7fe7d406b590 2024-05-20T18:27:33.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.040+0000 7fe7d98ca640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7d415e030 con 0x7fe7d40d18b0 2024-05-20T18:27:33.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.040+0000 7fe7d98ca640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe7d415e310 con 0x7fe7d4143620 2024-05-20T18:27:33.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.040+0000 7fe7d27fc640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe7d406b590 0x7fe7d415d440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38632/0 (socket says 172.21.15.5:38632) 2024-05-20T18:27:33.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.040+0000 7fe7d27fc640 1 -- 172.21.15.5:0/3256606561 learned_addr learned my addr 172.21.15.5:0/3256606561 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:33.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.040+0000 7fe7d2ffd640 1 --2- 172.21.15.5:0/3256606561 >> v2:172.21.15.63:3300/0 conn(0x7fe7d4143620 0x7fe7d4161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.040+0000 7fbc4102b640 1 Processor -- start 2024-05-20T18:27:33.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.040+0000 7f7f1e0a7640 1 Processor -- start 2024-05-20T18:27:33.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.040+0000 7fbc4102b640 1 -- start start 2024-05-20T18:27:33.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.041+0000 7fe7d1ffb640 1 -- 172.21.15.5:0/3256606561 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3357423166 0 0) 0x7fe7d415dd50 con 0x7fe7d406b590 2024-05-20T18:27:33.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.041+0000 7fe7d1ffb640 1 -- 172.21.15.5:0/3256606561 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7bc003520 con 0x7fe7d406b590 2024-05-20T18:27:33.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.041+0000 7f7a62692640 1 Processor -- start 2024-05-20T18:27:33.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.041+0000 7fe7d1ffb640 1 -- 172.21.15.5:0/3256606561 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2398169280 0 0) 0x7fe7d415e030 con 0x7fe7d40d18b0 2024-05-20T18:27:33.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.041+0000 7fe7d1ffb640 1 -- 172.21.15.5:0/3256606561 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7d415dd50 con 0x7fe7d40d18b0 2024-05-20T18:27:33.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.041+0000 7f8aee934640 1 Processor -- start 2024-05-20T18:27:33.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.041+0000 7fe7d1ffb640 1 -- 172.21.15.5:0/3256606561 <== mon.1 v1:172.21.15.63:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2262968597 0 0) 0x7fe7d415dd50 con 0x7fe7d40d18b0 2024-05-20T18:27:33.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.041+0000 7fe7d1ffb640 1 -- 172.21.15.5:0/3256606561 >> v2:172.21.15.63:3300/0 conn(0x7fe7d4143620 msgr2=0x7fe7d4161320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:33.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.041+0000 7fe7d1ffb640 1 --2- 172.21.15.5:0/3256606561 >> v2:172.21.15.63:3300/0 conn(0x7fe7d4143620 0x7fe7d4161320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.041+0000 7fe7d1ffb640 1 -- 172.21.15.5:0/3256606561 >> v1:172.21.15.5:6789/0 conn(0x7fe7d406b590 legacy=0x7fe7d415d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.041+0000 7fe7d1ffb640 1 -- 172.21.15.5:0/3256606561 --> v1:172.21.15.63:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d4160b20 con 0x7fe7d40d18b0 2024-05-20T18:27:33.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.041+0000 7fe7d1ffb640 1 -- 172.21.15.5:0/3256606561 <== mon.1 v1:172.21.15.63:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7fe7cc0019f0 con 0x7fe7d40d18b0 2024-05-20T18:27:33.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.041+0000 7fe7d1ffb640 1 -- 172.21.15.5:0/3256606561 >> v1:172.21.15.63:6789/0 conn(0x7fe7d40d18b0 legacy=0x7fe7d40d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.041+0000 7fe7d1ffb640 1 --2- 172.21.15.5:0/3256606561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7bc004ba0 0x7fe7bc004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.041+0000 7fe7d1ffb640 1 --2- 172.21.15.5:0/3256606561 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7bc0037a0 0x7fe7bc003bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.041+0000 7fe7d1ffb640 1 --2- 172.21.15.5:0/3256606561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7bc004200 0x7fe7bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.041+0000 7fe7d1ffb640 1 -- 172.21.15.5:0/3256606561 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d415e310 con 0x7fe7bc004ba0 2024-05-20T18:27:33.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.041+0000 7fe7d1ffb640 1 -- 172.21.15.5:0/3256606561 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc00c1c0 con 0x7fe7bc004200 2024-05-20T18:27:33.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.041+0000 7fe7d1ffb640 1 -- 172.21.15.5:0/3256606561 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe7bc00c3c0 con 0x7fe7bc0037a0 2024-05-20T18:27:33.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.042+0000 7fe7d37fe640 1 --2- 172.21.15.5:0/3256606561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7bc004200 0x7fe7bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.042+0000 7fe7d27fc640 1 --2- 172.21.15.5:0/3256606561 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7bc0037a0 0x7fe7bc003bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.042+0000 7fe7d2ffd640 1 --2- 172.21.15.5:0/3256606561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7bc004ba0 0x7fe7bc004ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.041+0000 7fe6e7fff640 1 Processor -- start 2024-05-20T18:27:33.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.041+0000 7fe6e7fff640 1 -- start start 2024-05-20T18:27:33.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.042+0000 7fe7d2ffd640 1 -- 172.21.15.5:0/3256606561 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7bc0037a0 msgr2=0x7fe7bc003bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:33.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.042+0000 7fe7d2ffd640 1 --2- 172.21.15.5:0/3256606561 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7bc0037a0 0x7fe7bc003bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.043+0000 7fe7d2ffd640 1 -- 172.21.15.5:0/3256606561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7bc004200 msgr2=0x7fe7bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:33.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.043+0000 7fe7d2ffd640 1 --2- 172.21.15.5:0/3256606561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7bc004200 0x7fe7bc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.043+0000 7fe7d2ffd640 1 -- 172.21.15.5:0/3256606561 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d4160b20 con 0x7fe7bc004ba0 2024-05-20T18:27:33.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.043+0000 7fe7d37fe640 1 --2- 172.21.15.5:0/3256606561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7bc004200 0x7fe7bc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:27:33.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.043+0000 7fe7d37fe640 1 -- 172.21.15.5:0/3256606561 reap_dead start 2024-05-20T18:27:33.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.041+0000 7f7a62692640 1 -- start start 2024-05-20T18:27:33.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.043+0000 7fbc4102b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbc3c0b4b90 0x7fbc3c0cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.043+0000 7fbc4102b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbc3c1559e0 0x7fbc3c0ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.041+0000 7f7f1e0a7640 1 -- start start 2024-05-20T18:27:33.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.043+0000 7f7f1e0a7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7f10050640 0x7f7f100d98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.043+0000 7f7f1e0a7640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7f10050a50 0x7f7f100d9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.043+0000 7f7f1e0a7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f1004f340 0x7f7f1004f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.043+0000 7f7f1e0a7640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f7f1004fc90 con 0x7f7f10050a50 2024-05-20T18:27:33.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.043+0000 7f7f1e0a7640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7f100e0be0 con 0x7f7f10050640 2024-05-20T18:27:33.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.043+0000 7f7f1e0a7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7f10053880 con 0x7f7f1004f340 2024-05-20T18:27:33.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.043+0000 7fbc4102b640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc3c15b770 con 0x7fbc3c0ccd40 2024-05-20T18:27:33.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.043+0000 7fbc4102b640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fbc3c0b5050 con 0x7fbc3c0b4b90 2024-05-20T18:27:33.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.043+0000 7fbc3bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbc3c0b4b90 0x7fbc3c0cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.043+0000 7f7f1c8a4640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7f10050a50 0x7f7f100d9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.044+0000 7f7f1c8a4640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7f10050a50 0x7f7f100d9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59154/0 (socket says 172.21.15.5:59154) 2024-05-20T18:27:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.044+0000 7f7f1c8a4640 1 -- 172.21.15.5:0/3486564699 learned_addr learned my addr 172.21.15.5:0/3486564699 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.044+0000 7fbc3bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbc3c0b4b90 0x7fbc3c0cfce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59156/0 (socket says 172.21.15.5:59156) 2024-05-20T18:27:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.044+0000 7fbc3bfff640 1 -- 172.21.15.5:0/3081338324 learned_addr learned my addr 172.21.15.5:0/3081338324 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.044+0000 7fbc3b7fe640 1 --2- 172.21.15.5:0/3081338324 >> v2:172.21.15.63:3300/0 conn(0x7fbc3c1559e0 0x7fbc3c0ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.044+0000 7fbc4102b640 1 -- 172.21.15.5:0/3081338324 --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fbc3c0b5330 con 0x7fbc3c1559e0 2024-05-20T18:27:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.044+0000 7fd4d24ca640 1 Processor -- start 2024-05-20T18:27:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.043+0000 7f7a62692640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7a5c06b520 0x7f7a5c166fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.043+0000 7f7a62692640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7a5c1674f0 0x7f7a5c06f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.043+0000 7f7a62692640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a5c167900 con 0x7f7a5c146530 2024-05-20T18:27:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.043+0000 7f7a62692640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f7a5c0b43d0 con 0x7f7a5c1674f0 2024-05-20T18:27:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.043+0000 7f7a62692640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7a5c0b37c0 con 0x7f7a5c06b520 2024-05-20T18:27:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.044+0000 7f7a5b7fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7a5c06b520 0x7f7a5c166fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.044+0000 7f7a5b7fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7a5c06b520 0x7f7a5c166fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:53552/0 (socket says 172.21.15.5:53552) 2024-05-20T18:27:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.044+0000 7f7a5b7fe640 1 -- 172.21.15.5:0/2834704588 learned_addr learned my addr 172.21.15.5:0/2834704588 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.044+0000 7f7a60c2f640 1 --2- 172.21.15.5:0/2834704588 >> v2:172.21.15.5:3300/0 conn(0x7f7a5c1674f0 0x7f7a5c06f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.044+0000 7fe6e7fff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6e00e3780 0x7fe6e0050330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.044+0000 7f7f1d8a6640 1 --2- 172.21.15.5:0/3486564699 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f1004f340 0x7f7f1004f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.044+0000 7f7f1d0a5640 1 --2- 172.21.15.5:0/3486564699 >> v2:172.21.15.63:3300/0 conn(0x7f7f10050640 0x7f7f100d98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.045+0000 7fe7d2ffd640 1 --2- 172.21.15.5:0/3256606561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7bc004ba0 0x7fe7bc004ff0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe7d406c3d0 tx=0x7fe7bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=143d855455031e57 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:33.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.044+0000 7fe6e7fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe6e0050c80 0x7fe6e0007480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.044+0000 7fe6e7fff640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6e004f920 con 0x7fe6e0050870 2024-05-20T18:27:33.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.044+0000 7fe6e7fff640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe6e0053ca0 con 0x7fe6e0050c80 2024-05-20T18:27:33.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.044+0000 7fe6e7fff640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e0054b00 con 0x7fe6e00e3780 2024-05-20T18:27:33.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.044+0000 7fe6e77fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe6e0050c80 0x7fe6e0007480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.044+0000 7fe6e77fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe6e0050c80 0x7fe6e0007480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59170/0 (socket says 172.21.15.5:59170) 2024-05-20T18:27:33.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.044+0000 7fe6e77fe640 1 -- 172.21.15.5:0/83701518 learned_addr learned my addr 172.21.15.5:0/83701518 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:33.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.045+0000 7fe6e6ffd640 1 --2- 172.21.15.5:0/83701518 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6e00e3780 0x7fe6e0050330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.045+0000 7fe6e5ffb640 1 -- 172.21.15.5:0/83701518 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3355943630 0 0) 0x7fe6e004f920 con 0x7fe6e0050870 2024-05-20T18:27:33.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.045+0000 7fe6e5ffb640 1 -- 172.21.15.5:0/83701518 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe6d0003520 con 0x7fe6e0050870 2024-05-20T18:27:33.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.045+0000 7fe7d1ffb640 1 -- 172.21.15.5:0/3256606561 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4036070 con 0x7fe7bc004ba0 2024-05-20T18:27:33.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.045+0000 7fe7d1ffb640 1 -- 172.21.15.5:0/3256606561 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7c4031400 con 0x7fe7bc004ba0 2024-05-20T18:27:33.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.045+0000 7fe7d1ffb640 1 -- 172.21.15.5:0/3256606561 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4009690 con 0x7fe7bc004ba0 2024-05-20T18:27:33.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.044+0000 7f8aee934640 1 -- start start 2024-05-20T18:27:33.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.045+0000 7f7a60c2f640 1 -- 172.21.15.5:0/2834704588 >> v2:172.21.15.63:3300/0 conn(0x7f7a5c06b520 msgr2=0x7f7a5c166fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:33.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.045+0000 7f7a60c2f640 1 --2- 172.21.15.5:0/2834704588 >> v2:172.21.15.63:3300/0 conn(0x7f7a5c06b520 0x7f7a5c166fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.045+0000 7f7a60c2f640 1 -- 172.21.15.5:0/2834704588 >> v1:172.21.15.5:6789/0 conn(0x7f7a5c146530 legacy=0x7f7a5c0b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.045+0000 7f7a5affd640 1 -- 172.21.15.5:0/2834704588 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3140599698 0 0) 0x7f7a50001110 con 0x7f7a5c146530 2024-05-20T18:27:33.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.045+0000 7f7a60c2f640 1 -- 172.21.15.5:0/2834704588 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a5c0b3aa0 con 0x7f7a5c1674f0 2024-05-20T18:27:33.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.045+0000 7f7f1c8a4640 1 -- 172.21.15.5:0/3486564699 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f1004f340 msgr2=0x7f7f1004f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:33.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.045+0000 7f7f1c8a4640 1 --2- 172.21.15.5:0/3486564699 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f1004f340 0x7f7f1004f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.045+0000 7f7f1c8a4640 1 -- 172.21.15.5:0/3486564699 >> v2:172.21.15.63:3300/0 conn(0x7f7f10050640 msgr2=0x7f7f100d98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:33.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.045+0000 7f7f1c8a4640 1 --2- 172.21.15.5:0/3486564699 >> v2:172.21.15.63:3300/0 conn(0x7f7f10050640 0x7f7f100d98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.045+0000 7f7f1c8a4640 1 -- 172.21.15.5:0/3486564699 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f10053b60 con 0x7f7f10050a50 2024-05-20T18:27:33.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.046+0000 7fe6e6ffd640 1 -- 172.21.15.5:0/83701518 >> v2:172.21.15.5:3300/0 conn(0x7fe6e0050c80 msgr2=0x7fe6e0007480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:33.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.046+0000 7fe6e6ffd640 1 --2- 172.21.15.5:0/83701518 >> v2:172.21.15.5:3300/0 conn(0x7fe6e0050c80 0x7fe6e0007480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.046+0000 7fe6e6ffd640 1 -- 172.21.15.5:0/83701518 >> v1:172.21.15.5:6789/0 conn(0x7fe6e0050870 legacy=0x7fe6e00dfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.046+0000 7fe6e6ffd640 1 -- 172.21.15.5:0/83701518 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e0053ef0 con 0x7fe6e00e3780 2024-05-20T18:27:33.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.044+0000 7fbc3affd640 1 -- 172.21.15.5:0/3081338324 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1136262850 0 0) 0x7fbc3c15b770 con 0x7fbc3c0ccd40 2024-05-20T18:27:33.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.044+0000 7fbc3affd640 1 -- 172.21.15.5:0/3081338324 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc28003520 con 0x7fbc3c0ccd40 2024-05-20T18:27:33.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.044+0000 7fbc3bfff640 1 -- 172.21.15.5:0/3081338324 >> v2:172.21.15.63:3300/0 conn(0x7fbc3c1559e0 msgr2=0x7fbc3c0ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:33.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.044+0000 7fbc3bfff640 1 --2- 172.21.15.5:0/3081338324 >> v2:172.21.15.63:3300/0 conn(0x7fbc3c1559e0 0x7fbc3c0ce5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.044+0000 7fbc3bfff640 1 -- 172.21.15.5:0/3081338324 >> v1:172.21.15.63:6789/0 conn(0x7fbc3c0ccd40 legacy=0x7fbc3c06c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.045+0000 7fbc3bfff640 1 -- 172.21.15.5:0/3081338324 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc3c0b5610 con 0x7fbc3c0b4b90 2024-05-20T18:27:33.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.046+0000 7fbc3bfff640 1 --2- 172.21.15.5:0/3081338324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc3c0b4b90 0x7fbc3c0cfce0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c0735c0 tx=0x7fbc2c02a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9eefcd90a7bf43b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:33.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.046+0000 7f7a60c2f640 1 --2- 172.21.15.5:0/2834704588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a5c1674f0 0x7f7a5c06f680 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c06e2c0 tx=0x7f7a5402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c8133719ab08d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:33.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.046+0000 7fe7d98ca640 1 -- 172.21.15.5:0/3256606561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7bc004ba0 msgr2=0x7fe7bc004ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.046+0000 7fe7d98ca640 1 --2- 172.21.15.5:0/3256606561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7bc004ba0 0x7fe7bc004ff0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe7d406c3d0 tx=0x7fe7bc005830 comp rx=0 tx=0).stop 2024-05-20T18:27:33.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.044+0000 7fd4d24ca640 1 -- start start 2024-05-20T18:27:33.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.046+0000 7fd4d24ca640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd4cc0d14f0 0x7fd4cc0b6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.047+0000 7fd4d24ca640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4cc0d1ac0 con 0x7fd4cc0d10e0 2024-05-20T18:27:33.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.047+0000 7fd4d24ca640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4cc0d1da0 con 0x7fd4cc0cf850 2024-05-20T18:27:33.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.047+0000 7fe7d98ca640 1 -- 172.21.15.5:0/3256606561 shutdown_connections 2024-05-20T18:27:33.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.047+0000 7f7a5affd640 1 -- 172.21.15.5:0/2834704588 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a54007740 con 0x7f7a5c1674f0 2024-05-20T18:27:33.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.047+0000 7fd4d24ca640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd4cc0d2080 con 0x7fd4cc0d14f0 2024-05-20T18:27:33.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.047+0000 7f8aee934640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8ae00383b0 0x7f8ae00e48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.047+0000 7fe7d98ca640 1 --2- 172.21.15.5:0/3256606561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7bc004ba0 0x7fe7bc004ff0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.047+0000 7fe7d98ca640 1 -- 172.21.15.5:0/3256606561 >> 172.21.15.5:0/3256606561 conn(0x7fe7d4073770 msgr2=0x7fe7d40b6720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:33.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.047+0000 7f8aee934640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8ae0057b60 0x7f8ae0054050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.047+0000 7fd4cb7fe640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fd4cc0cf850 0x7fd4cc0b6200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:57512/0 (socket says 172.21.15.5:57512) 2024-05-20T18:27:33.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.047+0000 7fd4cb7fe640 1 -- 172.21.15.5:0/986907650 learned_addr learned my addr 172.21.15.5:0/986907650 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:33.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.047+0000 7fd4cbfff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd4cc0d10e0 0x7fd4cc0cf140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38646/0 (socket says 172.21.15.5:38646) 2024-05-20T18:27:33.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.047+0000 7f8aee934640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ae0054590 0x7f8ae00e7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.047+0000 7f8aee934640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f8ae00549a0 con 0x7f8ae0057b60 2024-05-20T18:27:33.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.047+0000 7fbc3affd640 1 -- 172.21.15.5:0/3081338324 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c037070 con 0x7fbc3c0b4b90 2024-05-20T18:27:33.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.047+0000 7f8aee934640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8ae0058b50 con 0x7f8ae00383b0 2024-05-20T18:27:33.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.047+0000 7f8aee934640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8ae00582b0 con 0x7f8ae0054590 2024-05-20T18:27:33.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.047+0000 7fd4d0a67640 1 --2- 172.21.15.5:0/986907650 >> v2:172.21.15.63:3300/0 conn(0x7fd4cc0d14f0 0x7fd4cc0b6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.047+0000 7f8aed131640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8ae0057b60 0x7f8ae0054050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.047+0000 7f8aed131640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8ae0057b60 0x7f8ae0054050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59176/0 (socket says 172.21.15.5:59176) 2024-05-20T18:27:33.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.047+0000 7f8aed131640 1 -- 172.21.15.5:0/2459437338 learned_addr learned my addr 172.21.15.5:0/2459437338 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:33.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.047+0000 7fbc3affd640 1 -- 172.21.15.5:0/3081338324 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc2c032440 con 0x7fbc3c0b4b90 2024-05-20T18:27:33.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.047+0000 7fe6e6ffd640 1 --2- 172.21.15.5:0/83701518 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6e00e3780 0x7fe6e0050330 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8009790 tx=0x7fe6d802a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=535c733867b3662 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:33.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.047+0000 7fbc3affd640 1 -- 172.21.15.5:0/3081338324 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c0327f0 con 0x7fbc3c0b4b90 2024-05-20T18:27:33.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.047+0000 7f7a5affd640 1 -- 172.21.15.5:0/2834704588 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a540078e0 con 0x7f7a5c1674f0 2024-05-20T18:27:33.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.047+0000 7f7f1c8a4640 1 --2- 172.21.15.5:0/3486564699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f10050a50 0x7f7f100d9e30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7f100531a0 tx=0x7f7f0802e5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3e2d96a54f8d6d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:33.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.047+0000 7fe7d98ca640 1 -- 172.21.15.5:0/3256606561 shutdown_connections 2024-05-20T18:27:33.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.047+0000 7fe7d98ca640 1 -- 172.21.15.5:0/3256606561 wait complete. 2024-05-20T18:27:33.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.047+0000 7f8aed932640 1 --2- 172.21.15.5:0/2459437338 >> v2:172.21.15.63:3300/0 conn(0x7f8ae00383b0 0x7f8ae00e48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.047+0000 7f8aee133640 1 --2- 172.21.15.5:0/2459437338 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ae0054590 0x7f8ae00e7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.048+0000 7fe6e5ffb640 1 -- 172.21.15.5:0/83701518 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8037070 con 0x7fe6e00e3780 2024-05-20T18:27:33.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.048+0000 7fe6e5ffb640 1 -- 172.21.15.5:0/83701518 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6d8032440 con 0x7fe6e00e3780 2024-05-20T18:27:33.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.048+0000 7fe6e5ffb640 1 -- 172.21.15.5:0/83701518 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8032810 con 0x7fe6e00e3780 2024-05-20T18:27:33.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.048+0000 7f8aed131640 1 -- 172.21.15.5:0/2459437338 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ae0054590 msgr2=0x7f8ae00e7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:33.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.048+0000 7f8aed131640 1 --2- 172.21.15.5:0/2459437338 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ae0054590 0x7f8ae00e7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.048+0000 7f8aed131640 1 -- 172.21.15.5:0/2459437338 >> v2:172.21.15.63:3300/0 conn(0x7f8ae00383b0 msgr2=0x7f8ae00e48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:33.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.048+0000 7fd4caffd640 1 -- 172.21.15.5:0/986907650 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1337149765 0 0) 0x7fd4cc0d1ac0 con 0x7fd4cc0d10e0 2024-05-20T18:27:33.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.048+0000 7fd4caffd640 1 -- 172.21.15.5:0/986907650 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4b4003520 con 0x7fd4cc0d10e0 2024-05-20T18:27:33.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.048+0000 7f8aed131640 1 --2- 172.21.15.5:0/2459437338 >> v2:172.21.15.63:3300/0 conn(0x7f8ae00383b0 0x7f8ae00e48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.048+0000 7f8aed131640 1 -- 172.21.15.5:0/2459437338 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae0058590 con 0x7f8ae0057b60 2024-05-20T18:27:33.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.048+0000 7f8aee133640 1 --2- 172.21.15.5:0/2459437338 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ae0054590 0x7f8ae00e7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:27:33.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.048+0000 7fd4caffd640 1 -- 172.21.15.5:0/986907650 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2367763874 0 0) 0x7fd4b4003520 con 0x7fd4cc0d10e0 2024-05-20T18:27:33.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.048+0000 7fd4caffd640 1 -- 172.21.15.5:0/986907650 >> v2:172.21.15.63:3300/0 conn(0x7fd4cc0d14f0 msgr2=0x7fd4cc0b6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:33.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.048+0000 7fd4caffd640 1 --2- 172.21.15.5:0/986907650 >> v2:172.21.15.63:3300/0 conn(0x7fd4cc0d14f0 0x7fd4cc0b6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.048+0000 7fd4caffd640 1 -- 172.21.15.5:0/986907650 >> v1:172.21.15.63:6789/0 conn(0x7fd4cc0cf850 legacy=0x7fd4cc0b6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.048+0000 7fd4caffd640 1 -- 172.21.15.5:0/986907650 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4cc0d2360 con 0x7fd4cc0d10e0 2024-05-20T18:27:33.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.048+0000 7f7a62692640 1 -- 172.21.15.5:0/2834704588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a5c1674f0 msgr2=0x7f7a5c06f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.048+0000 7f7a62692640 1 --2- 172.21.15.5:0/2834704588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a5c1674f0 0x7f7a5c06f680 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c06e2c0 tx=0x7f7a5402a6a0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.048+0000 7fbc4102b640 1 -- 172.21.15.5:0/3081338324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc3c0b4b90 msgr2=0x7fbc3c0cfce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.049+0000 7fbc4102b640 1 --2- 172.21.15.5:0/3081338324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc3c0b4b90 0x7fbc3c0cfce0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c0735c0 tx=0x7fbc2c02a290 comp rx=0 tx=0).stop 2024-05-20T18:27:33.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.049+0000 7f8aed932640 1 --2- 172.21.15.5:0/2459437338 >> v2:172.21.15.63:3300/0 conn(0x7f8ae00383b0 0x7f8ae00e48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:27:33.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.049+0000 7f7a62692640 1 -- 172.21.15.5:0/2834704588 shutdown_connections 2024-05-20T18:27:33.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.049+0000 7f7a62692640 1 --2- 172.21.15.5:0/2834704588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a5c1674f0 0x7f7a5c06f680 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.049+0000 7f7a62692640 1 --2- 172.21.15.5:0/2834704588 >> v2:172.21.15.63:3300/0 conn(0x7f7a5c06b520 0x7f7a5c166fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.049+0000 7f7a62692640 1 -- 172.21.15.5:0/2834704588 >> 172.21.15.5:0/2834704588 conn(0x7f7a5c0d0a30 msgr2=0x7f7a5c0d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:33.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.049+0000 7fd4caffd640 1 -- 172.21.15.5:0/986907650 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7fd4c0001650 con 0x7fd4cc0d10e0 2024-05-20T18:27:33.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.049+0000 7fd4caffd640 1 -- 172.21.15.5:0/986907650 >> v1:172.21.15.5:6789/0 conn(0x7fd4cc0d10e0 legacy=0x7fd4cc0cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.049+0000 7f7f0ffff640 1 -- 172.21.15.5:0/3486564699 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0803a070 con 0x7f7f10050a50 2024-05-20T18:27:33.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.049+0000 7f7f0ffff640 1 -- 172.21.15.5:0/3486564699 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f08033510 con 0x7f7f10050a50 2024-05-20T18:27:33.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.049+0000 7f7f0ffff640 1 -- 172.21.15.5:0/3486564699 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f080338e0 con 0x7f7f10050a50 2024-05-20T18:27:33.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.049+0000 7f7a62692640 1 -- 172.21.15.5:0/2834704588 shutdown_connections 2024-05-20T18:27:33.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.049+0000 7f8aed131640 1 --2- 172.21.15.5:0/2459437338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ae0057b60 0x7f8ae0054050 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8000b60 tx=0x7f8ae80a60d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f817fe1f7dfd6439 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:33.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.049+0000 7fd4caffd640 1 --2- 172.21.15.5:0/986907650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4b40039d0 0x7fd4b4003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.049+0000 7fd4caffd640 1 --2- 172.21.15.5:0/986907650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4b4004800 0x7fd4b4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.049+0000 7fd4caffd640 1 --2- 172.21.15.5:0/986907650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4b4005250 0x7fd4b400fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.049+0000 7fd4caffd640 1 -- 172.21.15.5:0/986907650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc0d2080 con 0x7fd4b40039d0 2024-05-20T18:27:33.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.049+0000 7fd4caffd640 1 -- 172.21.15.5:0/986907650 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b4010680 con 0x7fd4b4004800 2024-05-20T18:27:33.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.049+0000 7fd4caffd640 1 -- 172.21.15.5:0/986907650 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd4b4010880 con 0x7fd4b4005250 2024-05-20T18:27:33.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.049+0000 7f8aec930640 1 -- 172.21.15.5:0/2459437338 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae80b3070 con 0x7f8ae0057b60 2024-05-20T18:27:33.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.049+0000 7fe6e7fff640 1 -- 172.21.15.5:0/83701518 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6e00e3780 msgr2=0x7fe6e0050330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.049+0000 7f8aec930640 1 -- 172.21.15.5:0/2459437338 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae80a6cd0 con 0x7f8ae0057b60 2024-05-20T18:27:33.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.049+0000 7f8aec930640 1 -- 172.21.15.5:0/2459437338 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae80ae640 con 0x7f8ae0057b60 2024-05-20T18:27:33.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.049+0000 7fe6e7fff640 1 --2- 172.21.15.5:0/83701518 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6e00e3780 0x7fe6e0050330 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8009790 tx=0x7fe6d802a1a0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.049+0000 7fd4cb7fe640 1 --2- 172.21.15.5:0/986907650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4b4004800 0x7fd4b4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.050+0000 7fbc4102b640 1 -- 172.21.15.5:0/3081338324 shutdown_connections 2024-05-20T18:27:33.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.050+0000 7fe6e7fff640 1 -- 172.21.15.5:0/83701518 shutdown_connections 2024-05-20T18:27:33.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.050+0000 7fe6e7fff640 1 --2- 172.21.15.5:0/83701518 >> v2:172.21.15.5:3300/0 conn(0x7fe6e0050c80 0x7fe6e0007480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.050+0000 7fe6e7fff640 1 --2- 172.21.15.5:0/83701518 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6e00e3780 0x7fe6e0050330 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.050+0000 7fbc4102b640 1 --2- 172.21.15.5:0/3081338324 >> v2:172.21.15.63:3300/0 conn(0x7fbc3c1559e0 0x7fbc3c0ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.050+0000 7fe6e7fff640 1 -- 172.21.15.5:0/83701518 >> 172.21.15.5:0/83701518 conn(0x7fe6e000aa70 msgr2=0x7fe6e000ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:33.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.050+0000 7fbc4102b640 1 --2- 172.21.15.5:0/3081338324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc3c0b4b90 0x7fbc3c0cfce0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.050+0000 7fd4cbfff640 1 --2- 172.21.15.5:0/986907650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4b40039d0 0x7fd4b4003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.050+0000 7fd4d0a67640 1 --2- 172.21.15.5:0/986907650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4b4005250 0x7fd4b400fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.050+0000 7fd4cb7fe640 1 -- 172.21.15.5:0/986907650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4b4005250 msgr2=0x7fd4b400fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:33.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.050+0000 7fd4cb7fe640 1 --2- 172.21.15.5:0/986907650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4b4005250 0x7fd4b400fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.050+0000 7fd4cb7fe640 1 -- 172.21.15.5:0/986907650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4b40039d0 msgr2=0x7fd4b4003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:33.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.050+0000 7fd4cb7fe640 1 --2- 172.21.15.5:0/986907650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4b40039d0 0x7fd4b4003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.050+0000 7fd4cb7fe640 1 -- 172.21.15.5:0/986907650 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4cc0d2360 con 0x7fd4b4004800 2024-05-20T18:27:33.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.050+0000 7fd4d0a67640 1 -- 172.21.15.5:0/986907650 reap_dead start 2024-05-20T18:27:33.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.050+0000 7fe6e7fff640 1 -- 172.21.15.5:0/83701518 shutdown_connections 2024-05-20T18:27:33.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.050+0000 7f7f1e0a7640 1 -- 172.21.15.5:0/3486564699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f10050a50 msgr2=0x7f7f100d9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.050+0000 7f7f1e0a7640 1 --2- 172.21.15.5:0/3486564699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f10050a50 0x7f7f100d9e30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7f100531a0 tx=0x7f7f0802e5e0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.049+0000 7f7a62692640 1 -- 172.21.15.5:0/2834704588 wait complete. 2024-05-20T18:27:33.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.050+0000 7fbc4102b640 1 -- 172.21.15.5:0/3081338324 >> 172.21.15.5:0/3081338324 conn(0x7fbc3c0721d0 msgr2=0x7fbc3c0725e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:33.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.050+0000 7fe6e7fff640 1 -- 172.21.15.5:0/83701518 wait complete. 2024-05-20T18:27:33.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.050+0000 7f7f1e0a7640 1 -- 172.21.15.5:0/3486564699 shutdown_connections 2024-05-20T18:27:33.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.050+0000 7f7f1e0a7640 1 --2- 172.21.15.5:0/3486564699 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f1004f340 0x7f7f1004f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.050+0000 7f7f1e0a7640 1 --2- 172.21.15.5:0/3486564699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f10050a50 0x7f7f100d9e30 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.050+0000 7f7f1e0a7640 1 --2- 172.21.15.5:0/3486564699 >> v2:172.21.15.63:3300/0 conn(0x7f7f10050640 0x7f7f100d98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.050+0000 7f7f1e0a7640 1 -- 172.21.15.5:0/3486564699 >> 172.21.15.5:0/3486564699 conn(0x7f7f10054cf0 msgr2=0x7f7f10055100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:33.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.051+0000 7fe7d98ca640 1 Processor -- start 2024-05-20T18:27:33.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.051+0000 7f8aee934640 1 -- 172.21.15.5:0/2459437338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ae0057b60 msgr2=0x7f8ae0054050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.051+0000 7f8aee934640 1 --2- 172.21.15.5:0/2459437338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ae0057b60 0x7f8ae0054050 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8000b60 tx=0x7f8ae80a60d0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.051+0000 7fbc4102b640 1 -- 172.21.15.5:0/3081338324 shutdown_connections 2024-05-20T18:27:33.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.051+0000 7fbc4102b640 1 -- 172.21.15.5:0/3081338324 wait complete. 2024-05-20T18:27:33.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.051+0000 7fe7d98ca640 1 -- start start 2024-05-20T18:27:33.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.051+0000 7f7f1e0a7640 1 -- 172.21.15.5:0/3486564699 shutdown_connections 2024-05-20T18:27:33.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.051+0000 7f7f1e0a7640 1 -- 172.21.15.5:0/3486564699 wait complete. 2024-05-20T18:27:33.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.051+0000 7fd4cb7fe640 1 --2- 172.21.15.5:0/986907650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4b4004800 0x7fd4b4004c10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc001810 tx=0x7fd4b4005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e2e0da36a4b4e6e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:33.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.051+0000 7f8aee934640 1 -- 172.21.15.5:0/2459437338 shutdown_connections 2024-05-20T18:27:33.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.051+0000 7f8aee934640 1 --2- 172.21.15.5:0/2459437338 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ae0054590 0x7f8ae00e7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.051+0000 7f8aee934640 1 --2- 172.21.15.5:0/2459437338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ae0057b60 0x7f8ae0054050 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.051+0000 7f8aee934640 1 --2- 172.21.15.5:0/2459437338 >> v2:172.21.15.63:3300/0 conn(0x7f8ae00383b0 0x7f8ae00e48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.051+0000 7f8aee934640 1 -- 172.21.15.5:0/2459437338 >> 172.21.15.5:0/2459437338 conn(0x7f8ae003ac00 msgr2=0x7f8ae003b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:33.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.052+0000 7f8aee934640 1 -- 172.21.15.5:0/2459437338 shutdown_connections 2024-05-20T18:27:33.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.052+0000 7f7a62692640 1 Processor -- start 2024-05-20T18:27:33.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.052+0000 7f8aee934640 1 -- 172.21.15.5:0/2459437338 wait complete. 2024-05-20T18:27:33.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.052+0000 7f7a62692640 1 -- start start 2024-05-20T18:27:33.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.052+0000 7fd4caffd640 1 -- 172.21.15.5:0/986907650 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc001460 con 0x7fd4b4004800 2024-05-20T18:27:33.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.052+0000 7fd4caffd640 1 -- 172.21.15.5:0/986907650 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4bc02e070 con 0x7fd4b4004800 2024-05-20T18:27:33.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.052+0000 7fd4caffd640 1 -- 172.21.15.5:0/986907650 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc02add0 con 0x7fd4b4004800 2024-05-20T18:27:33.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.054+0000 7fe7d98ca640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7bc004ba0 0x7fe7d413fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.054+0000 7fd4d24ca640 1 -- 172.21.15.5:0/986907650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4b4004800 msgr2=0x7fd4b4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.054+0000 7fd4d24ca640 1 --2- 172.21.15.5:0/986907650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4b4004800 0x7fd4b4004c10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc001810 tx=0x7fd4b4005830 comp rx=0 tx=0).stop 2024-05-20T18:27:33.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.054+0000 7fe7d98ca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7d415bbf0 0x7fe7d415a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.054+0000 7fe6e7fff640 1 Processor -- start 2024-05-20T18:27:33.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.054+0000 7f7f1e0a7640 1 Processor -- start 2024-05-20T18:27:33.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.054+0000 7fe7d2ffd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7bc004ba0 0x7fe7d413fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.054+0000 7fe7d2ffd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7bc004ba0 0x7fe7d413fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:46318/0 (socket says 172.21.15.5:46318) 2024-05-20T18:27:33.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.054+0000 7fe6e7fff640 1 -- start start 2024-05-20T18:27:33.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.054+0000 7f7a62692640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a5c06b520 0x7f7a5c15cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.054+0000 7f7a62692640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a5c1674f0 0x7f7a5c15d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.054+0000 7fbc4102b640 1 Processor -- start 2024-05-20T18:27:33.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.054+0000 7fe7d27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7d415bbf0 0x7fe7d415a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.054+0000 7f7a62692640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a5c1654d0 0x7f7a5c15d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.054+0000 7fe7d27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7d415bbf0 0x7fe7d415a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59200/0 (socket says 172.21.15.5:59200) 2024-05-20T18:27:33.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.054+0000 7f7a62692640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c0b5490 con 0x7f7a5c1674f0 2024-05-20T18:27:33.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.054+0000 7f7a62692640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c0b4060 con 0x7f7a5c1654d0 2024-05-20T18:27:33.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.054+0000 7fe7d2ffd640 1 -- 172.21.15.5:0/3264278588 learned_addr learned my addr 172.21.15.5:0/3264278588 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:33.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.054+0000 7f7a5bfff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a5c06b520 0x7f7a5c15cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.055+0000 7f7a62692640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7a5c167c70 con 0x7f7a5c06b520 2024-05-20T18:27:33.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.055+0000 7f7a5bfff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a5c06b520 0x7f7a5c15cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:46322/0 (socket says 172.21.15.5:46322) 2024-05-20T18:27:33.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.055+0000 7fe7d98ca640 1 --2- 172.21.15.5:0/3264278588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7d4070740 0x7fe7d4070b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.055+0000 7f7a5bfff640 1 -- 172.21.15.5:0/2216779244 learned_addr learned my addr 172.21.15.5:0/2216779244 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:33.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.055+0000 7f7a60c2f640 1 --2- 172.21.15.5:0/2216779244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a5c1654d0 0x7f7a5c15d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.054+0000 7f8aee934640 1 Processor -- start 2024-05-20T18:27:33.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.055+0000 7f8aee934640 1 -- start start 2024-05-20T18:27:33.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.055+0000 7fd4d24ca640 1 -- 172.21.15.5:0/986907650 shutdown_connections 2024-05-20T18:27:33.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.055+0000 7fd4d24ca640 1 --2- 172.21.15.5:0/986907650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4b4004800 0x7fd4b4004c10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.055+0000 7fd4d24ca640 1 -- 172.21.15.5:0/986907650 >> 172.21.15.5:0/986907650 conn(0x7fd4cc0b44d0 msgr2=0x7fd4cc0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:33.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.055+0000 7fe7d98ca640 1 -- 172.21.15.5:0/3264278588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d40d1e40 con 0x7fe7d415bbf0 2024-05-20T18:27:33.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.055+0000 7fe7d98ca640 1 -- 172.21.15.5:0/3264278588 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d40d1cc0 con 0x7fe7d4070740 2024-05-20T18:27:33.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.055+0000 7fe7d37fe640 1 --2- 172.21.15.5:0/3264278588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7d4070740 0x7fe7d4070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.055+0000 7fe7d98ca640 1 -- 172.21.15.5:0/3264278588 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe7d40d1fc0 con 0x7fe7bc004ba0 2024-05-20T18:27:33.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.055+0000 7f7a5b7fe640 1 --2- 172.21.15.5:0/2216779244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a5c1674f0 0x7f7a5c15d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.055+0000 7f7a60c2f640 1 -- 172.21.15.5:0/2216779244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a5c06b520 msgr2=0x7f7a5c15cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.055+0000 7f7a60c2f640 1 --2- 172.21.15.5:0/2216779244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a5c06b520 0x7f7a5c15cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.055+0000 7f7a60c2f640 1 -- 172.21.15.5:0/2216779244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a5c1674f0 msgr2=0x7f7a5c15d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.055+0000 7f7a60c2f640 1 --2- 172.21.15.5:0/2216779244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a5c1674f0 0x7f7a5c15d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.055+0000 7f7a60c2f640 1 -- 172.21.15.5:0/2216779244 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a5c15de70 con 0x7f7a5c1654d0 2024-05-20T18:27:33.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.056+0000 7f7a60c2f640 1 --2- 172.21.15.5:0/2216779244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a5c1654d0 0x7f7a5c15d5b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c06e520 tx=0x7f7a54004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:33.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.055+0000 7fe7d27fc640 1 -- 172.21.15.5:0/3264278588 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7bc004ba0 msgr2=0x7fe7d413fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.055+0000 7fe7d27fc640 1 --2- 172.21.15.5:0/3264278588 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7bc004ba0 0x7fe7d413fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.055+0000 7fe7d27fc640 1 -- 172.21.15.5:0/3264278588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7d4070740 msgr2=0x7fe7d4070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.055+0000 7fe7d27fc640 1 --2- 172.21.15.5:0/3264278588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7d4070740 0x7fe7d4070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.056+0000 7fe7d27fc640 1 -- 172.21.15.5:0/3264278588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d4071090 con 0x7fe7d415bbf0 2024-05-20T18:27:33.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.056+0000 7fbc4102b640 1 -- start start 2024-05-20T18:27:33.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.056+0000 7fe7d27fc640 1 --2- 172.21.15.5:0/3264278588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7d415bbf0 0x7fe7d415a5a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe7d41609e0 tx=0x7fe7c8001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:33.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.056+0000 7fe7d2ffd640 1 --2- 172.21.15.5:0/3264278588 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7bc004ba0 0x7fe7d413fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:33.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.054+0000 7f7f1e0a7640 1 -- start start 2024-05-20T18:27:33.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.056+0000 7f7f1e0a7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f1004f340 0x7f7f100dcde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.056+0000 7fe7bbfff640 1 -- 172.21.15.5:0/3264278588 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c800f470 con 0x7fe7d415bbf0 2024-05-20T18:27:33.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.056+0000 7f7f1e0a7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f10050640 0x7f7f1004ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.056+0000 7fe7bbfff640 1 -- 172.21.15.5:0/3264278588 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7c8018070 con 0x7fe7d415bbf0 2024-05-20T18:27:33.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.056+0000 7f7f1d0a5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f1004f340 0x7f7f100dcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.057+0000 7f7f1d0a5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f1004f340 0x7f7f100dcde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:46332/0 (socket says 172.21.15.5:46332) 2024-05-20T18:27:33.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.056+0000 7f7f1e0a7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f100dacf0 0x7f7f10173260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.056+0000 7fe7bbfff640 1 -- 172.21.15.5:0/3264278588 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c800f810 con 0x7fe7d415bbf0 2024-05-20T18:27:33.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.057+0000 7fe7d98ca640 1 -- 172.21.15.5:0/3264278588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d4140280 con 0x7fe7d415bbf0 2024-05-20T18:27:33.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.057+0000 7f8aee934640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ae00383b0 0x7f8ae00526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.057+0000 7f7f1c8a4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f10050640 0x7f7f1004ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.057+0000 7f7f1c8a4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f10050640 0x7f7f1004ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:53614/0 (socket says 172.21.15.5:53614) 2024-05-20T18:27:33.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.057+0000 7f7f1d8a6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f100dacf0 0x7f7f10173260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.057+0000 7fe7d98ca640 1 -- 172.21.15.5:0/3264278588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d415b110 con 0x7fe7d415bbf0 2024-05-20T18:27:33.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.057+0000 7f7f1d8a6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f100dacf0 0x7f7f10173260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59214/0 (socket says 172.21.15.5:59214) 2024-05-20T18:27:33.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.057+0000 7f7f1d0a5640 1 -- 172.21.15.5:0/1448392202 learned_addr learned my addr 172.21.15.5:0/1448392202 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:33.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.057+0000 7f7f1e0a7640 1 -- 172.21.15.5:0/1448392202 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7f10054640 con 0x7f7f100dacf0 2024-05-20T18:27:33.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.057+0000 7f7f1e0a7640 1 -- 172.21.15.5:0/1448392202 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7f10054120 con 0x7f7f10050640 2024-05-20T18:27:33.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.057+0000 7f7f1e0a7640 1 -- 172.21.15.5:0/1448392202 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7f100d9110 con 0x7f7f1004f340 2024-05-20T18:27:33.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.058+0000 7f7f1c8a4640 1 -- 172.21.15.5:0/1448392202 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f1004f340 msgr2=0x7f7f100dcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.058+0000 7f7f1c8a4640 1 --2- 172.21.15.5:0/1448392202 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f1004f340 0x7f7f100dcde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.058+0000 7f7f1c8a4640 1 -- 172.21.15.5:0/1448392202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f100dacf0 msgr2=0x7f7f10173260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.058+0000 7f7f1c8a4640 1 --2- 172.21.15.5:0/1448392202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f100dacf0 0x7f7f10173260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.058+0000 7f7f1c8a4640 1 -- 172.21.15.5:0/1448392202 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f100db500 con 0x7f7f10050640 2024-05-20T18:27:33.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.058+0000 7f7f1d8a6640 1 --2- 172.21.15.5:0/1448392202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f100dacf0 0x7f7f10173260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:27:33.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.056+0000 7f7a58ff9640 1 -- 172.21.15.5:0/2216779244 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5403f070 con 0x7f7a5c1654d0 2024-05-20T18:27:33.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.056+0000 7f7a58ff9640 1 -- 172.21.15.5:0/2216779244 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a54044660 con 0x7f7a5c1654d0 2024-05-20T18:27:33.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.056+0000 7f7a58ff9640 1 -- 172.21.15.5:0/2216779244 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a54044a30 con 0x7f7a5c1654d0 2024-05-20T18:27:33.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.057+0000 7f7a62692640 1 -- 172.21.15.5:0/2216779244 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a5c165cb0 con 0x7f7a5c1654d0 2024-05-20T18:27:33.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.058+0000 7f7a62692640 1 -- 172.21.15.5:0/2216779244 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a5c166270 con 0x7f7a5c1654d0 2024-05-20T18:27:33.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.058+0000 7f8aee934640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ae0054590 0x7f8ae00e1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.058+0000 7f8aee934640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ae0057b60 0x7f8ae004f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.058+0000 7f8aee934640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae00387c0 con 0x7f8ae00383b0 2024-05-20T18:27:33.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.058+0000 7f8aed131640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ae0054590 0x7f8ae00e1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.058+0000 7f8aed131640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ae0054590 0x7f8ae00e1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:46346/0 (socket says 172.21.15.5:46346) 2024-05-20T18:27:33.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.058+0000 7f8aed131640 1 -- 172.21.15.5:0/1330388298 learned_addr learned my addr 172.21.15.5:0/1330388298 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:33.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.058+0000 7f8aee934640 1 -- 172.21.15.5:0/1330388298 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae0006180 con 0x7f8ae0057b60 2024-05-20T18:27:33.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.058+0000 7f8aee934640 1 -- 172.21.15.5:0/1330388298 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8ae0057f70 con 0x7f8ae0054590 2024-05-20T18:27:33.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.055+0000 7fd4d24ca640 1 -- 172.21.15.5:0/986907650 shutdown_connections 2024-05-20T18:27:33.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.055+0000 7fd4d24ca640 1 -- 172.21.15.5:0/986907650 wait complete. 2024-05-20T18:27:33.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.058+0000 7fd4d24ca640 1 Processor -- start 2024-05-20T18:27:33.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.058+0000 7fbc4102b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc3c0ccd40 0x7fbc3c15a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.058+0000 7fbc4102b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbc3c1559e0 0x7fbc3c1542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.058+0000 7fbc4102b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc3c1576a0 0x7fbc3c09b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.058+0000 7fbc4102b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c0cec80 con 0x7fbc3c1576a0 2024-05-20T18:27:33.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.058+0000 7fbc4102b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c0ceb00 con 0x7fbc3c0ccd40 2024-05-20T18:27:33.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.058+0000 7fbc4102b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbc3c06cc40 con 0x7fbc3c1559e0 2024-05-20T18:27:33.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.058+0000 7f48ce348640 1 Processor -- start 2024-05-20T18:27:33.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.058+0000 7f8aed932640 1 --2- 172.21.15.5:0/1330388298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ae00383b0 0x7f8ae00526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.059+0000 7fe7d98ca640 1 -- 172.21.15.5:0/3264278588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7d406caa0 con 0x7fe7d415bbf0 2024-05-20T18:27:33.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.059+0000 7f8aed131640 1 -- 172.21.15.5:0/1330388298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ae0057b60 msgr2=0x7f8ae004f360 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T18:27:33.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.058+0000 7f48ce348640 1 -- start start 2024-05-20T18:27:33.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.059+0000 7f8aed131640 1 --2- 172.21.15.5:0/1330388298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ae0057b60 0x7f8ae004f360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.059+0000 7f8aed131640 1 -- 172.21.15.5:0/1330388298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ae00383b0 msgr2=0x7f8ae00526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.059+0000 7f8aed131640 1 --2- 172.21.15.5:0/1330388298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ae00383b0 0x7f8ae00526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.059+0000 7f8aed131640 1 -- 172.21.15.5:0/1330388298 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae004dd50 con 0x7f8ae0054590 2024-05-20T18:27:33.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.059+0000 7f7f1c8a4640 1 --2- 172.21.15.5:0/1448392202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f10050640 0x7f7f1004ea60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000bdc0 tx=0x7f7f080046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:33.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.059+0000 7fd4d24ca640 1 -- start start 2024-05-20T18:27:33.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.060+0000 7fbc3bfff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc3c0ccd40 0x7fbc3c15a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.060+0000 7fbc3bfff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc3c0ccd40 0x7fbc3c15a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:53622/0 (socket says 172.21.15.5:53622) 2024-05-20T18:27:33.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.060+0000 7fbc3bfff640 1 -- 172.21.15.5:0/1640342193 learned_addr learned my addr 172.21.15.5:0/1640342193 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:33.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.060+0000 7f8aed131640 1 --2- 172.21.15.5:0/1330388298 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ae0054590 0x7f8ae00e1d10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8000b60 tx=0x7f8ae8074640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:33.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.060+0000 7f7a58ff9640 1 -- 172.21.15.5:0/2216779244 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7a54007750 con 0x7f7a5c1654d0 2024-05-20T18:27:33.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.059+0000 7fe6e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6e0050870 0x7fe6e00482f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.059+0000 7fe6e7fff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6e0050c80 0x7fe6e00cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.059+0000 7fe6e7fff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6e00e3780 0x7fe6e00cda20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.059+0000 7fe6e7fff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e0054790 con 0x7fe6e0050870 2024-05-20T18:27:33.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.059+0000 7fe6e7fff640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e00544b0 con 0x7fe6e00e3780 2024-05-20T18:27:33.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.059+0000 7fe6e7fff640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e0051090 con 0x7fe6e0050c80 2024-05-20T18:27:33.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.059+0000 7fe6e67fc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6e0050c80 0x7fe6e00cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.059+0000 7fe6e67fc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6e0050c80 0x7fe6e00cd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:46360/0 (socket says 172.21.15.5:46360) 2024-05-20T18:27:33.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.059+0000 7fe6e67fc640 1 -- 172.21.15.5:0/4124350059 learned_addr learned my addr 172.21.15.5:0/4124350059 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:33.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.060+0000 7fe6e67fc640 1 -- 172.21.15.5:0/4124350059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6e00e3780 msgr2=0x7fe6e00cda20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T18:27:33.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.060+0000 7fe6e67fc640 1 --2- 172.21.15.5:0/4124350059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6e00e3780 0x7fe6e00cda20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.060+0000 7fe6e67fc640 1 -- 172.21.15.5:0/4124350059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6e0050870 msgr2=0x7fe6e00482f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-20T18:27:33.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.060+0000 7fe6e67fc640 1 --2- 172.21.15.5:0/4124350059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6e0050870 0x7fe6e00482f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.060+0000 7fe6e67fc640 1 -- 172.21.15.5:0/4124350059 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e00d36a0 con 0x7fe6e0050c80 2024-05-20T18:27:33.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.060+0000 7f7a62692640 1 -- 172.21.15.5:0/2216779244 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a5c075980 con 0x7f7a5c1654d0 2024-05-20T18:27:33.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.060+0000 7fbc3b7fe640 1 --2- 172.21.15.5:0/1640342193 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbc3c1559e0 0x7fbc3c1542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.060+0000 7f7f0dffb640 1 -- 172.21.15.5:0/1448392202 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f08043070 con 0x7f7f10050640 2024-05-20T18:27:33.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.060+0000 7fe6e67fc640 1 --2- 172.21.15.5:0/4124350059 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6e0050c80 0x7fe6e00cd4e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8050f00 tx=0x7fe6e8074f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:33.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.060+0000 7f7f0dffb640 1 -- 172.21.15.5:0/1448392202 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f0803e450 con 0x7f7f10050640 2024-05-20T18:27:33.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7f7f0dffb640 1 -- 172.21.15.5:0/1448392202 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0803a070 con 0x7f7f10050640 2024-05-20T18:27:33.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.060+0000 7f7a58ff9640 1 --2- 172.21.15.5:0/2216779244 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7a4806beb0 0x7f7a4806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7f7a58ff9640 1 -- 172.21.15.5:0/2216779244 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f7a540ac910 con 0x7f7a5c1654d0 2024-05-20T18:27:33.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7f7a5bfff640 1 --2- 172.21.15.5:0/2216779244 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7a4806beb0 0x7f7a4806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.062+0000 7f7a5bfff640 1 --2- 172.21.15.5:0/2216779244 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7a4806beb0 0x7f7a4806e3c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7a50007fd0 tx=0x7f7a50009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:33.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.060+0000 7f8ade7fc640 1 -- 172.21.15.5:0/1330388298 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae80bb070 con 0x7f8ae0054590 2024-05-20T18:27:33.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.060+0000 7f8ade7fc640 1 -- 172.21.15.5:0/1330388298 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae80b7410 con 0x7f8ae0054590 2024-05-20T18:27:33.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7f8ade7fc640 1 -- 172.21.15.5:0/1330388298 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae80b3040 con 0x7f8ae0054590 2024-05-20T18:27:33.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7f8aee934640 1 -- 172.21.15.5:0/1330388298 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ae004e030 con 0x7f8ae0054590 2024-05-20T18:27:33.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.062+0000 7f8aee934640 1 -- 172.21.15.5:0/1330388298 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ae004e5f0 con 0x7f8ae0054590 2024-05-20T18:27:33.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7fd4d24ca640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4b4004800 0x7fd4cc1fd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7fd4d24ca640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4cc0cc440 0x7fd4cc15a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7fd4d24ca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4cc0cc850 0x7fd4cc15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7fd4d24ca640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc0b8440 con 0x7fd4cc0cc850 2024-05-20T18:27:33.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7fd4d24ca640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc0b82c0 con 0x7fd4b4004800 2024-05-20T18:27:33.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7fd4d24ca640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd4cc0709f0 con 0x7fd4cc0cc440 2024-05-20T18:27:33.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7fd4cb7fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4cc0cc440 0x7fd4cc15a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7fd4cb7fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4cc0cc440 0x7fd4cc15a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:46392/0 (socket says 172.21.15.5:46392) 2024-05-20T18:27:33.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7fd4cb7fe640 1 -- 172.21.15.5:0/2837601474 learned_addr learned my addr 172.21.15.5:0/2837601474 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:33.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7fd4cbfff640 1 --2- 172.21.15.5:0/2837601474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4b4004800 0x7fd4cc1fd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.062+0000 7fd4d0a67640 1 --2- 172.21.15.5:0/2837601474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4cc0cc850 0x7fd4cc15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.062+0000 7fd4cb7fe640 1 -- 172.21.15.5:0/2837601474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4b4004800 msgr2=0x7fd4cc1fd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.062+0000 7fd4cb7fe640 1 --2- 172.21.15.5:0/2837601474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4b4004800 0x7fd4cc1fd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.062+0000 7fd4cb7fe640 1 -- 172.21.15.5:0/2837601474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4cc0cc850 msgr2=0x7fd4cc15ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.062+0000 7fd4cb7fe640 1 --2- 172.21.15.5:0/2837601474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4cc0cc850 0x7fd4cc15ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.062+0000 7fd4cb7fe640 1 -- 172.21.15.5:0/2837601474 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4cc15e860 con 0x7fd4cc0cc440 2024-05-20T18:27:33.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.062+0000 7fd4d0a67640 1 --2- 172.21.15.5:0/2837601474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4cc0cc850 0x7fd4cc15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:33.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.060+0000 7fe7bbfff640 1 -- 172.21.15.5:0/3264278588 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe7c800f9b0 con 0x7fe7d415bbf0 2024-05-20T18:27:33.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7fe7bbfff640 1 --2- 172.21.15.5:0/3264278588 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe7c006bf30 0x7fe7c006e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7fe7bbfff640 1 -- 172.21.15.5:0/3264278588 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe7c808ade0 con 0x7fe7d415bbf0 2024-05-20T18:27:33.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.060+0000 7fbc4082a640 1 --2- 172.21.15.5:0/1640342193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc3c1576a0 0x7fbc3c09b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7fbc4082a640 1 -- 172.21.15.5:0/1640342193 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbc3c1559e0 msgr2=0x7fbc3c1542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7fbc4082a640 1 --2- 172.21.15.5:0/1640342193 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbc3c1559e0 0x7fbc3c1542d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7fbc4082a640 1 -- 172.21.15.5:0/1640342193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc3c0ccd40 msgr2=0x7fbc3c15a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7fbc4082a640 1 --2- 172.21.15.5:0/1640342193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc3c0ccd40 0x7fbc3c15a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7fbc4082a640 1 -- 172.21.15.5:0/1640342193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc3c159720 con 0x7fbc3c1576a0 2024-05-20T18:27:33.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7f48ce348640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48c8163600 0x7f48c806c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7f48ce348640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f48c806ca70 0x7f48c806e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7f48ccb45640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48c8163600 0x7f48c806c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7f48ccb45640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48c8163600 0x7f48c806c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:46382/0 (socket says 172.21.15.5:46382) 2024-05-20T18:27:33.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7f48ccb45640 1 -- 172.21.15.5:0/2765380179 learned_addr learned my addr 172.21.15.5:0/2765380179 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:33.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7f48ce348640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48c806ce80 con 0x7f48c80cfe10 2024-05-20T18:27:33.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7f48ce348640 1 -- 172.21.15.5:0/2765380179 --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f48c80b54f0 con 0x7f48c806ca70 2024-05-20T18:27:33.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7f48cdb47640 1 --2- 172.21.15.5:0/2765380179 >> v2:172.21.15.5:3300/0 conn(0x7f48c806ca70 0x7f48c806e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7f48ce348640 1 -- 172.21.15.5:0/2765380179 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f48c8163a10 con 0x7f48c8163600 2024-05-20T18:27:33.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.062+0000 7f48ccb45640 1 -- 172.21.15.5:0/2765380179 >> v2:172.21.15.5:3300/0 conn(0x7f48c806ca70 msgr2=0x7f48c806e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:33.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.062+0000 7f48ccb45640 1 --2- 172.21.15.5:0/2765380179 >> v2:172.21.15.5:3300/0 conn(0x7f48c806ca70 0x7f48c806e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.062+0000 7f48ccb45640 1 -- 172.21.15.5:0/2765380179 >> v1:172.21.15.63:6789/0 conn(0x7f48c80cfe10 legacy=0x7f48c8167b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.062+0000 7f48ccb45640 1 -- 172.21.15.5:0/2765380179 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c80b48e0 con 0x7f48c8163600 2024-05-20T18:27:33.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7fe6cffff640 1 -- 172.21.15.5:0/4124350059 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e808bc90 con 0x7fe6e0050c80 2024-05-20T18:27:33.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7fe6cffff640 1 -- 172.21.15.5:0/4124350059 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6e808f020 con 0x7fe6e0050c80 2024-05-20T18:27:33.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.061+0000 7fe6cffff640 1 -- 172.21.15.5:0/4124350059 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e806c8a0 con 0x7fe6e0050c80 2024-05-20T18:27:33.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.062+0000 7fe6e7fff640 1 -- 172.21.15.5:0/4124350059 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e00d3980 con 0x7fe6e0050c80 2024-05-20T18:27:33.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.062+0000 7fe6e7fff640 1 -- 172.21.15.5:0/4124350059 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e00d3f40 con 0x7fe6e0050c80 2024-05-20T18:27:33.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.062+0000 7f7f1e0a7640 1 -- 172.21.15.5:0/1448392202 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f100db7e0 con 0x7f7f10050640 2024-05-20T18:27:33.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.063+0000 7f7f1e0a7640 1 -- 172.21.15.5:0/1448392202 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f100d78b0 con 0x7f7f10050640 2024-05-20T18:27:33.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.063+0000 7fbc4082a640 1 --2- 172.21.15.5:0/1640342193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc3c1576a0 0x7fbc3c09b540 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbc34002180 tx=0x7fbc3400edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:33.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.062+0000 7fe7d2ffd640 1 --2- 172.21.15.5:0/3264278588 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe7c006bf30 0x7fe7c006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.063+0000 7fd4cb7fe640 1 --2- 172.21.15.5:0/2837601474 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4cc0cc440 0x7fd4cc15a7d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc001930 tx=0x7fd4bc001120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:33.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.064+0000 7f8aee934640 1 -- 172.21.15.5:0/1330388298 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ae00054a0 con 0x7f8ae0054590 2024-05-20T18:27:33.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.064+0000 7fd4c8ff9640 1 -- 172.21.15.5:0/2837601474 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc004bf0 con 0x7fd4cc0cc440 2024-05-20T18:27:33.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.064+0000 7fe7d2ffd640 1 --2- 172.21.15.5:0/3264278588 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe7c006bf30 0x7fe7c006e440 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4009b80 tx=0x7fe7c4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:33.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.064+0000 7fd4c8ff9640 1 -- 172.21.15.5:0/2837601474 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4bc004d90 con 0x7fd4cc0cc440 2024-05-20T18:27:33.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.063+0000 7fe6e7fff640 1 -- 172.21.15.5:0/4124350059 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6e00058f0 con 0x7fe6e0050c80 2024-05-20T18:27:33.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.063+0000 7f48ccb45640 1 --2- 172.21.15.5:0/2765380179 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48c8163600 0x7f48c806c530 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f48b8009870 tx=0x7f48b802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36b7336e1046e338 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:33.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.064+0000 7f48bffff640 1 -- 172.21.15.5:0/2765380179 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b8037070 con 0x7f48c8163600 2024-05-20T18:27:33.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.064+0000 7f48bffff640 1 -- 172.21.15.5:0/2765380179 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48b8032440 con 0x7f48c8163600 2024-05-20T18:27:33.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.064+0000 7f48bffff640 1 -- 172.21.15.5:0/2765380179 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b8032810 con 0x7f48c8163600 2024-05-20T18:27:33.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.064+0000 7f7f1e0a7640 1 -- 172.21.15.5:0/1448392202 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f100054a0 con 0x7f7f10050640 2024-05-20T18:27:33.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.064+0000 7fe6cffff640 1 -- 172.21.15.5:0/4124350059 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe6e806ca40 con 0x7fe6e0050c80 2024-05-20T18:27:33.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.064+0000 7f8ade7fc640 1 -- 172.21.15.5:0/1330388298 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8ae80be070 con 0x7f8ae0054590 2024-05-20T18:27:33.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.064+0000 7fd4d24ca640 1 -- 172.21.15.5:0/2837601474 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4cc15eb40 con 0x7fd4cc0cc440 2024-05-20T18:27:33.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.064+0000 7fd4d24ca640 1 -- 172.21.15.5:0/2837601474 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4cc0c9b00 con 0x7fd4cc0cc440 2024-05-20T18:27:33.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.064+0000 7fd4c8ff9640 1 -- 172.21.15.5:0/2837601474 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc007270 con 0x7fd4cc0cc440 2024-05-20T18:27:33.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.065+0000 7f48ce348640 1 -- 172.21.15.5:0/2765380179 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48c8163600 msgr2=0x7f48c806c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.065+0000 7f48ce348640 1 --2- 172.21.15.5:0/2765380179 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48c8163600 0x7f48c806c530 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f48b8009870 tx=0x7f48b802a270 comp rx=0 tx=0).stop 2024-05-20T18:27:33.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.065+0000 7f48ce348640 1 -- 172.21.15.5:0/2765380179 shutdown_connections 2024-05-20T18:27:33.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.065+0000 7f48ce348640 1 --2- 172.21.15.5:0/2765380179 >> v2:172.21.15.5:3300/0 conn(0x7f48c806ca70 0x7f48c806e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.065+0000 7f48ce348640 1 --2- 172.21.15.5:0/2765380179 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48c8163600 0x7f48c806c530 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.065+0000 7fe6cffff640 1 --2- 172.21.15.5:0/4124350059 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe6d406be80 0x7fe6d406e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.065+0000 7f48ce348640 1 -- 172.21.15.5:0/2765380179 >> 172.21.15.5:0/2765380179 conn(0x7f48c80721d0 msgr2=0x7f48c80725e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:33.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.065+0000 7f8ade7fc640 1 --2- 172.21.15.5:0/1330388298 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8acc06bf00 0x7f8acc06e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.065+0000 7fe6cffff640 1 -- 172.21.15.5:0/4124350059 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe6e8105db0 con 0x7fe6e0050c80 2024-05-20T18:27:33.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.065+0000 7f48ce348640 1 -- 172.21.15.5:0/2765380179 shutdown_connections 2024-05-20T18:27:33.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.065+0000 7fe6e6ffd640 1 --2- 172.21.15.5:0/4124350059 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe6d406be80 0x7fe6d406e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.065+0000 7f48ce348640 1 -- 172.21.15.5:0/2765380179 wait complete. 2024-05-20T18:27:33.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.065+0000 7f8ade7fc640 1 -- 172.21.15.5:0/1330388298 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f8ae8126d50 con 0x7f8ae0054590 2024-05-20T18:27:33.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.066+0000 7f8aed932640 1 --2- 172.21.15.5:0/1330388298 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8acc06bf00 0x7f8acc06e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.066+0000 7fd4d24ca640 1 -- 172.21.15.5:0/2837601474 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4cc075b00 con 0x7fd4cc0cc440 2024-05-20T18:27:33.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.066+0000 7fe6e6ffd640 1 --2- 172.21.15.5:0/4124350059 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe6d406be80 0x7fe6d406e390 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8034000 tx=0x7fe6d802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.066+0000 7f8aed932640 1 --2- 172.21.15.5:0/1330388298 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8acc06bf00 0x7f8acc06e410 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009580 tx=0x7f8ad8009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.066+0000 7f7a58ff9640 1 -- 172.21.15.5:0/2216779244 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7a5407a880 con 0x7f7a5c1654d0 2024-05-20T18:27:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.068+0000 7f48ce348640 1 Processor -- start 2024-05-20T18:27:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.068+0000 7f48ce348640 1 -- start start 2024-05-20T18:27:33.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.068+0000 7f7f0dffb640 1 -- 172.21.15.5:0/1448392202 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7f08046070 con 0x7f7f10050640 2024-05-20T18:27:33.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.068+0000 7f7f0dffb640 1 --2- 172.21.15.5:0/1448392202 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7efc06bf00 0x7f7efc06e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.068+0000 7fbc38ff9640 1 -- 172.21.15.5:0/1640342193 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3400cc60 con 0x7fbc3c1576a0 2024-05-20T18:27:33.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.068+0000 7f7f0dffb640 1 -- 172.21.15.5:0/1448392202 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f7f080ab010 con 0x7f7f10050640 2024-05-20T18:27:33.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.069+0000 7f7f1d0a5640 1 --2- 172.21.15.5:0/1448392202 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7efc06bf00 0x7f7efc06e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.069+0000 7fbc38ff9640 1 -- 172.21.15.5:0/1640342193 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc3400ce20 con 0x7fbc3c1576a0 2024-05-20T18:27:33.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.069+0000 7fbc4102b640 1 -- 172.21.15.5:0/1640342193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc3c159a00 con 0x7fbc3c1576a0 2024-05-20T18:27:33.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.069+0000 7fe7bbfff640 1 -- 172.21.15.5:0/3264278588 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe7d406caa0 con 0x7fe7d415bbf0 2024-05-20T18:27:33.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.069+0000 7f48ce348640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48c806ca70 0x7f48c8164780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.069+0000 7f48ce348640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48c80cfe10 0x7f48c8164cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.069+0000 7f7f1d0a5640 1 --2- 172.21.15.5:0/1448392202 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7efc06bf00 0x7f7efc06e410 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400aac0 tx=0x7f7f14008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:33.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.069+0000 7fbc38ff9640 1 -- 172.21.15.5:0/1640342193 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc34003bb0 con 0x7fbc3c1576a0 2024-05-20T18:27:33.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.069+0000 7f48ce348640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48c8163600 0x7f48c8165200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.069+0000 7fbc4102b640 1 -- 172.21.15.5:0/1640342193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc3c09c080 con 0x7fbc3c1576a0 2024-05-20T18:27:33.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.069+0000 7f48ce348640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f48c80b5180 con 0x7f48c8163600 2024-05-20T18:27:33.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.069+0000 7f48ce348640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f48c80b4ea0 con 0x7f48c80cfe10 2024-05-20T18:27:33.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.069+0000 7f48cd346640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48c806ca70 0x7f48c8164780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.069+0000 7f48cd346640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48c806ca70 0x7f48c8164780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:46404/0 (socket says 172.21.15.5:46404) 2024-05-20T18:27:33.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.070+0000 7f48cd346640 1 -- 172.21.15.5:0/1837851741 learned_addr learned my addr 172.21.15.5:0/1837851741 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:33.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.070+0000 7f48ce348640 1 -- 172.21.15.5:0/1837851741 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f48c80b5980 con 0x7f48c806ca70 2024-05-20T18:27:33.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.070+0000 7f48cdb47640 1 --2- 172.21.15.5:0/1837851741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48c8163600 0x7f48c8165200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.070+0000 7f48ccb45640 1 --2- 172.21.15.5:0/1837851741 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48c80cfe10 0x7f48c8164cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.070+0000 7f48cdb47640 1 -- 172.21.15.5:0/1837851741 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48c806ca70 msgr2=0x7f48c8164780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.070+0000 7f48cdb47640 1 --2- 172.21.15.5:0/1837851741 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48c806ca70 0x7f48c8164780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.070+0000 7f48cdb47640 1 -- 172.21.15.5:0/1837851741 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48c80cfe10 msgr2=0x7f48c8164cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.070+0000 7f48cdb47640 1 --2- 172.21.15.5:0/1837851741 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48c80cfe10 0x7f48c8164cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.070+0000 7f48cdb47640 1 -- 172.21.15.5:0/1837851741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c809e360 con 0x7f48c8163600 2024-05-20T18:27:33.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.070+0000 7f48cd346640 1 --2- 172.21.15.5:0/1837851741 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48c806ca70 0x7f48c8164780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:27:33.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.070+0000 7f48ccb45640 1 --2- 172.21.15.5:0/1837851741 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48c80cfe10 0x7f48c8164cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:33.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.070+0000 7f48cdb47640 1 --2- 172.21.15.5:0/1837851741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48c8163600 0x7f48c8165200 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f48c400e7c0 tx=0x7f48c400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:33.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.071+0000 7f48bdffb640 1 -- 172.21.15.5:0/1837851741 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c4017070 con 0x7f48c8163600 2024-05-20T18:27:33.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.071+0000 7fbc4102b640 1 -- 172.21.15.5:0/1640342193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc3c154810 con 0x7fbc3c1576a0 2024-05-20T18:27:33.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.071+0000 7f48ce348640 1 -- 172.21.15.5:0/1837851741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48c809e080 con 0x7f48c8163600 2024-05-20T18:27:33.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.071+0000 7fe6cffff640 1 -- 172.21.15.5:0/4124350059 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe6e8099030 con 0x7fe6e0050c80 2024-05-20T18:27:33.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.071+0000 7f48ce348640 1 -- 172.21.15.5:0/1837851741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48c8163f80 con 0x7f48c8163600 2024-05-20T18:27:33.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.071+0000 7f48bdffb640 1 -- 172.21.15.5:0/1837851741 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48c400c730 con 0x7f48c8163600 2024-05-20T18:27:33.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.071+0000 7f48bdffb640 1 -- 172.21.15.5:0/1837851741 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c4017070 con 0x7f48c8163600 2024-05-20T18:27:33.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.071+0000 7fbc38ff9640 1 -- 172.21.15.5:0/1640342193 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbc34013770 con 0x7fbc3c1576a0 2024-05-20T18:27:33.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.072+0000 7f7f0dffb640 1 -- 172.21.15.5:0/1448392202 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7f0804d080 con 0x7f7f10050640 2024-05-20T18:27:33.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.072+0000 7fbc38ff9640 1 --2- 172.21.15.5:0/1640342193 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbc1c06be80 0x7fbc1c06e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.072+0000 7fbc38ff9640 1 -- 172.21.15.5:0/1640342193 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fbc3408f460 con 0x7fbc3c1576a0 2024-05-20T18:27:33.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.073+0000 7f48ce348640 1 -- 172.21.15.5:0/1837851741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48a8005290 con 0x7f48c8163600 2024-05-20T18:27:33.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.073+0000 7fbc3bfff640 1 --2- 172.21.15.5:0/1640342193 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbc1c06be80 0x7fbc1c06e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.074+0000 7fbc3bfff640 1 --2- 172.21.15.5:0/1640342193 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbc1c06be80 0x7fbc1c06e390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009160 tx=0x7fbc2c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:33.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.074+0000 7f48bdffb640 1 -- 172.21.15.5:0/1837851741 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f48c401a070 con 0x7f48c8163600 2024-05-20T18:27:33.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.075+0000 7f48bdffb640 1 --2- 172.21.15.5:0/1837851741 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f48ac06c030 0x7f48ac06e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.075+0000 7f48bdffb640 1 -- 172.21.15.5:0/1837851741 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f48c408a6a0 con 0x7f48c8163600 2024-05-20T18:27:33.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.075+0000 7f48cd346640 1 --2- 172.21.15.5:0/1837851741 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f48ac06c030 0x7f48ac06e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.075+0000 7f8ade7fc640 1 -- 172.21.15.5:0/1330388298 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8ae80f4cc0 con 0x7f8ae0054590 2024-05-20T18:27:33.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.076+0000 7f48cd346640 1 --2- 172.21.15.5:0/1837851741 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f48ac06c030 0x7f48ac06e540 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f48c000f1d0 tx=0x7f48c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:33.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.077+0000 7fd4c8ff9640 1 -- 172.21.15.5:0/2837601474 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd4bc030050 con 0x7fd4cc0cc440 2024-05-20T18:27:33.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.078+0000 7fd4c8ff9640 1 --2- 172.21.15.5:0/2837601474 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd4b806be80 0x7fd4b806e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:33.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.078+0000 7fd4c8ff9640 1 -- 172.21.15.5:0/2837601474 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd4bc0ab2f0 con 0x7fd4cc0cc440 2024-05-20T18:27:33.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.078+0000 7fd4cbfff640 1 --2- 172.21.15.5:0/2837601474 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd4b806be80 0x7fd4b806e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:33.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.079+0000 7fd4cbfff640 1 --2- 172.21.15.5:0/2837601474 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd4b806be80 0x7fd4b806e390 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0001eb0 tx=0x7fd4c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:33.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.080+0000 7fbc38ff9640 1 -- 172.21.15.5:0/1640342193 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbc3c154810 con 0x7fbc3c1576a0 2024-05-20T18:27:33.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.081+0000 7fd4c8ff9640 1 -- 172.21.15.5:0/2837601474 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd4bc04b020 con 0x7fd4cc0cc440 2024-05-20T18:27:33.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.089+0000 7f48bdffb640 1 -- 172.21.15.5:0/1837851741 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f48c40586c0 con 0x7f48c8163600 2024-05-20T18:27:33.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.496+0000 7fe7d98ca640 1 -- 172.21.15.5:0/3264278588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fe7d4075b00 con 0x7fe7d415bbf0 2024-05-20T18:27:33.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.497+0000 7fe7bbfff640 1 -- 172.21.15.5:0/3264278588 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe7d4071090 con 0x7fe7d415bbf0 2024-05-20T18:27:33.496 INFO:teuthology.orchestra.run.smithi005.stdout:55834574857 2024-05-20T18:27:33.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.500+0000 7f7a62692640 1 -- 172.21.15.5:0/2216779244 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f7a5c06f460 con 0x7f7a5c1654d0 2024-05-20T18:27:33.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.501+0000 7f7a58ff9640 1 -- 172.21.15.5:0/2216779244 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7a5407eb80 con 0x7f7a5c1654d0 2024-05-20T18:27:33.500 INFO:teuthology.orchestra.run.smithi005.stdout:55834574857 2024-05-20T18:27:33.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.500+0000 7fe7b9ffb640 1 -- 172.21.15.5:0/3264278588 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe7c006bf30 msgr2=0x7fe7c006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.500+0000 7fe7b9ffb640 1 --2- 172.21.15.5:0/3264278588 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe7c006bf30 0x7fe7c006e440 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4009b80 tx=0x7fe7c4008030 comp rx=0 tx=0).stop 2024-05-20T18:27:33.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.500+0000 7fe7b9ffb640 1 -- 172.21.15.5:0/3264278588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7d415bbf0 msgr2=0x7fe7d415a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.500+0000 7fe7b9ffb640 1 --2- 172.21.15.5:0/3264278588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7d415bbf0 0x7fe7d415a5a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe7d41609e0 tx=0x7fe7c8001970 comp rx=0 tx=0).stop 2024-05-20T18:27:33.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.500+0000 7fe7b9ffb640 1 -- 172.21.15.5:0/3264278588 shutdown_connections 2024-05-20T18:27:33.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.500+0000 7fe7b9ffb640 1 --2- 172.21.15.5:0/3264278588 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe7c006bf30 0x7fe7c006e440 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4009b80 tx=0x7fe7c4008030 comp rx=0 tx=0).stop 2024-05-20T18:27:33.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.500+0000 7fe7b9ffb640 1 --2- 172.21.15.5:0/3264278588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7d4070740 0x7fe7d4070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.500+0000 7fe7b9ffb640 1 --2- 172.21.15.5:0/3264278588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7d415bbf0 0x7fe7d415a5a0 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe7d41609e0 tx=0x7fe7c8001970 comp rx=0 tx=0).stop 2024-05-20T18:27:33.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.500+0000 7fe7b9ffb640 1 --2- 172.21.15.5:0/3264278588 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7bc004ba0 0x7fe7d413fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.500+0000 7fe7b9ffb640 1 -- 172.21.15.5:0/3264278588 >> 172.21.15.5:0/3264278588 conn(0x7fe7d4073770 msgr2=0x7fe7d40b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:33.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.501+0000 7fe7b9ffb640 1 -- 172.21.15.5:0/3264278588 shutdown_connections 2024-05-20T18:27:33.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.501+0000 7fe7b9ffb640 1 -- 172.21.15.5:0/3264278588 wait complete. 2024-05-20T18:27:33.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.504+0000 7f7a427fc640 1 -- 172.21.15.5:0/2216779244 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7a4806beb0 msgr2=0x7f7a4806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.504+0000 7f7a427fc640 1 --2- 172.21.15.5:0/2216779244 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7a4806beb0 0x7f7a4806e3c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7a50007fd0 tx=0x7f7a50009040 comp rx=0 tx=0).stop 2024-05-20T18:27:33.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.504+0000 7f7a427fc640 1 -- 172.21.15.5:0/2216779244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a5c1654d0 msgr2=0x7f7a5c15d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.504+0000 7f7a427fc640 1 --2- 172.21.15.5:0/2216779244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a5c1654d0 0x7f7a5c15d5b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c06e520 tx=0x7f7a54004a40 comp rx=0 tx=0).stop 2024-05-20T18:27:33.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.504+0000 7f7a427fc640 1 -- 172.21.15.5:0/2216779244 shutdown_connections 2024-05-20T18:27:33.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.504+0000 7f7a427fc640 1 --2- 172.21.15.5:0/2216779244 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7a4806beb0 0x7f7a4806e3c0 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7a50007fd0 tx=0x7f7a50009040 comp rx=0 tx=0).stop 2024-05-20T18:27:33.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.504+0000 7f7a427fc640 1 --2- 172.21.15.5:0/2216779244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a5c1654d0 0x7f7a5c15d5b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.504+0000 7f7a427fc640 1 --2- 172.21.15.5:0/2216779244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a5c1674f0 0x7f7a5c15d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.504+0000 7f7a427fc640 1 --2- 172.21.15.5:0/2216779244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a5c06b520 0x7f7a5c15cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.504+0000 7f7a427fc640 1 -- 172.21.15.5:0/2216779244 >> 172.21.15.5:0/2216779244 conn(0x7f7a5c0d0a30 msgr2=0x7f7a5c144af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:33.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.505+0000 7f7a427fc640 1 -- 172.21.15.5:0/2216779244 shutdown_connections 2024-05-20T18:27:33.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.505+0000 7f7a427fc640 1 -- 172.21.15.5:0/2216779244 wait complete. 2024-05-20T18:27:33.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.511+0000 7f7f1e0a7640 1 -- 172.21.15.5:0/1448392202 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f7f100db290 con 0x7f7f10050640 2024-05-20T18:27:33.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.511+0000 7fe6e7fff640 1 -- 172.21.15.5:0/4124350059 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fe6e0053a80 con 0x7fe6e0050c80 2024-05-20T18:27:33.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.512+0000 7f7f0dffb640 1 -- 172.21.15.5:0/1448392202 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7f08079010 con 0x7f7f10050640 2024-05-20T18:27:33.511 INFO:teuthology.orchestra.run.smithi005.stdout:55834574857 2024-05-20T18:27:33.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.512+0000 7fe6cffff640 1 -- 172.21.15.5:0/4124350059 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe6e8092320 con 0x7fe6e0050c80 2024-05-20T18:27:33.512 INFO:teuthology.orchestra.run.smithi005.stdout:55834574856 2024-05-20T18:27:33.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.515+0000 7f8aee934640 1 -- 172.21.15.5:0/1330388298 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f8ae00e6ed0 con 0x7f8ae0054590 2024-05-20T18:27:33.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.515+0000 7f7f1e0a7640 1 -- 172.21.15.5:0/1448392202 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7efc06bf00 msgr2=0x7f7efc06e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.515+0000 7f7f1e0a7640 1 --2- 172.21.15.5:0/1448392202 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7efc06bf00 0x7f7efc06e410 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400aac0 tx=0x7f7f14008040 comp rx=0 tx=0).stop 2024-05-20T18:27:33.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.515+0000 7f7f1e0a7640 1 -- 172.21.15.5:0/1448392202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f10050640 msgr2=0x7f7f1004ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.515+0000 7f7f1e0a7640 1 --2- 172.21.15.5:0/1448392202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f10050640 0x7f7f1004ea60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000bdc0 tx=0x7f7f080046c0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.516+0000 7fbc4102b640 1 -- 172.21.15.5:0/1640342193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fbc3c09c2d0 con 0x7fbc3c1576a0 2024-05-20T18:27:33.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.516+0000 7f7f1e0a7640 1 -- 172.21.15.5:0/1448392202 shutdown_connections 2024-05-20T18:27:33.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.516+0000 7f7f1e0a7640 1 --2- 172.21.15.5:0/1448392202 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7efc06bf00 0x7f7efc06e410 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.516+0000 7f7f1e0a7640 1 --2- 172.21.15.5:0/1448392202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f100dacf0 0x7f7f10173260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.516+0000 7f7f1e0a7640 1 --2- 172.21.15.5:0/1448392202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f10050640 0x7f7f1004ea60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.516+0000 7f7f1e0a7640 1 --2- 172.21.15.5:0/1448392202 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f1004f340 0x7f7f100dcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.516+0000 7f7f1e0a7640 1 -- 172.21.15.5:0/1448392202 >> 172.21.15.5:0/1448392202 conn(0x7f7f10054cf0 msgr2=0x7f7f10035680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:33.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.516+0000 7f7f1e0a7640 1 -- 172.21.15.5:0/1448392202 shutdown_connections 2024-05-20T18:27:33.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.516+0000 7f7f1e0a7640 1 -- 172.21.15.5:0/1448392202 wait complete. 2024-05-20T18:27:33.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.516+0000 7f8ade7fc640 1 -- 172.21.15.5:0/1330388298 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8ae80f8fc0 con 0x7f8ae0054590 2024-05-20T18:27:33.517 INFO:teuthology.orchestra.run.smithi005.stdout:55834574856 2024-05-20T18:27:33.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.516+0000 7fbc38ff9640 1 -- 172.21.15.5:0/1640342193 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbc3c154810 con 0x7fbc3c1576a0 2024-05-20T18:27:33.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.516+0000 7fe6e7fff640 1 -- 172.21.15.5:0/4124350059 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe6d406be80 msgr2=0x7fe6d406e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.516+0000 7fe6e7fff640 1 --2- 172.21.15.5:0/4124350059 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe6d406be80 0x7fe6d406e390 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8034000 tx=0x7fe6d802e000 comp rx=0 tx=0).stop 2024-05-20T18:27:33.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.516+0000 7fe6e7fff640 1 -- 172.21.15.5:0/4124350059 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6e0050c80 msgr2=0x7fe6e00cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.516+0000 7fe6e7fff640 1 --2- 172.21.15.5:0/4124350059 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6e0050c80 0x7fe6e00cd4e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8050f00 tx=0x7fe6e8074f20 comp rx=0 tx=0).stop 2024-05-20T18:27:33.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.517+0000 7fe6e7fff640 1 -- 172.21.15.5:0/4124350059 shutdown_connections 2024-05-20T18:27:33.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.517+0000 7fe6e7fff640 1 --2- 172.21.15.5:0/4124350059 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe6d406be80 0x7fe6d406e390 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.517+0000 7fe6e7fff640 1 --2- 172.21.15.5:0/4124350059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6e00e3780 0x7fe6e00cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.517+0000 7fe6e7fff640 1 --2- 172.21.15.5:0/4124350059 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6e0050c80 0x7fe6e00cd4e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.517+0000 7fe6e7fff640 1 --2- 172.21.15.5:0/4124350059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6e0050870 0x7fe6e00482f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.517+0000 7fe6e7fff640 1 -- 172.21.15.5:0/4124350059 >> 172.21.15.5:0/4124350059 conn(0x7fe6e000aa70 msgr2=0x7fe6e003a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:33.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.517+0000 7fe6e7fff640 1 -- 172.21.15.5:0/4124350059 shutdown_connections 2024-05-20T18:27:33.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.517+0000 7fe6e7fff640 1 -- 172.21.15.5:0/4124350059 wait complete. 2024-05-20T18:27:33.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.520+0000 7f8acbfff640 1 -- 172.21.15.5:0/1330388298 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8acc06bf00 msgr2=0x7f8acc06e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.520+0000 7f8acbfff640 1 --2- 172.21.15.5:0/1330388298 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8acc06bf00 0x7f8acc06e410 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009580 tx=0x7f8ad8009190 comp rx=0 tx=0).stop 2024-05-20T18:27:33.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.520+0000 7f8acbfff640 1 -- 172.21.15.5:0/1330388298 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ae0054590 msgr2=0x7f8ae00e1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.520+0000 7f8acbfff640 1 --2- 172.21.15.5:0/1330388298 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ae0054590 0x7f8ae00e1d10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8000b60 tx=0x7f8ae8074640 comp rx=0 tx=0).stop 2024-05-20T18:27:33.520 INFO:teuthology.orchestra.run.smithi005.stdout:55834574855 2024-05-20T18:27:33.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.520+0000 7fd4d24ca640 1 -- 172.21.15.5:0/2837601474 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fd4cc15e5f0 con 0x7fd4cc0cc440 2024-05-20T18:27:33.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.521+0000 7fd4c8ff9640 1 -- 172.21.15.5:0/2837601474 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd4bc04b020 con 0x7fd4cc0cc440 2024-05-20T18:27:33.521 INFO:teuthology.orchestra.run.smithi005.stdout:55834574855 2024-05-20T18:27:33.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.521+0000 7f8acbfff640 1 -- 172.21.15.5:0/1330388298 shutdown_connections 2024-05-20T18:27:33.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.521+0000 7f8acbfff640 1 --2- 172.21.15.5:0/1330388298 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8acc06bf00 0x7f8acc06e410 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.521+0000 7f8acbfff640 1 --2- 172.21.15.5:0/1330388298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ae0057b60 0x7f8ae004f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.521+0000 7f8acbfff640 1 --2- 172.21.15.5:0/1330388298 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ae0054590 0x7f8ae00e1d10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.523 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.7 2024-05-20T18:27:33.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.522+0000 7f8acbfff640 1 --2- 172.21.15.5:0/1330388298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ae00383b0 0x7f8ae00526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.522+0000 7f8acbfff640 1 -- 172.21.15.5:0/1330388298 >> 172.21.15.5:0/1330388298 conn(0x7f8ae003ac00 msgr2=0x7f8ae0004940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:33.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.522+0000 7f8acbfff640 1 -- 172.21.15.5:0/1330388298 shutdown_connections 2024-05-20T18:27:33.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.522+0000 7f8acbfff640 1 -- 172.21.15.5:0/1330388298 wait complete. 2024-05-20T18:27:33.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.522+0000 7fbc4102b640 1 -- 172.21.15.5:0/1640342193 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbc1c06be80 msgr2=0x7fbc1c06e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.522+0000 7fbc4102b640 1 --2- 172.21.15.5:0/1640342193 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbc1c06be80 0x7fbc1c06e390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009160 tx=0x7fbc2c02e000 comp rx=0 tx=0).stop 2024-05-20T18:27:33.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.522+0000 7fbc4102b640 1 -- 172.21.15.5:0/1640342193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc3c1576a0 msgr2=0x7fbc3c09b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.522+0000 7fbc4102b640 1 --2- 172.21.15.5:0/1640342193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc3c1576a0 0x7fbc3c09b540 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbc34002180 tx=0x7fbc3400edd0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.523+0000 7fbc4102b640 1 -- 172.21.15.5:0/1640342193 shutdown_connections 2024-05-20T18:27:33.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.523+0000 7fbc4102b640 1 --2- 172.21.15.5:0/1640342193 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbc1c06be80 0x7fbc1c06e390 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.523+0000 7fbc4102b640 1 --2- 172.21.15.5:0/1640342193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc3c1576a0 0x7fbc3c09b540 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.523+0000 7fbc4102b640 1 --2- 172.21.15.5:0/1640342193 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbc3c1559e0 0x7fbc3c1542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.523+0000 7fbc4102b640 1 --2- 172.21.15.5:0/1640342193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc3c0ccd40 0x7fbc3c15a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.523+0000 7fbc4102b640 1 -- 172.21.15.5:0/1640342193 >> 172.21.15.5:0/1640342193 conn(0x7fbc3c0721d0 msgr2=0x7fbc3c06fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:33.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.523+0000 7fbc4102b640 1 -- 172.21.15.5:0/1640342193 shutdown_connections 2024-05-20T18:27:33.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.523+0000 7fbc4102b640 1 -- 172.21.15.5:0/1640342193 wait complete. 2024-05-20T18:27:33.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.524+0000 7fd4d24ca640 1 -- 172.21.15.5:0/2837601474 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd4b806be80 msgr2=0x7fd4b806e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.524+0000 7fd4d24ca640 1 --2- 172.21.15.5:0/2837601474 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd4b806be80 0x7fd4b806e390 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0001eb0 tx=0x7fd4c0009040 comp rx=0 tx=0).stop 2024-05-20T18:27:33.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.524+0000 7fd4d24ca640 1 -- 172.21.15.5:0/2837601474 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4cc0cc440 msgr2=0x7fd4cc15a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.524+0000 7fd4d24ca640 1 --2- 172.21.15.5:0/2837601474 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4cc0cc440 0x7fd4cc15a7d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc001930 tx=0x7fd4bc001120 comp rx=0 tx=0).stop 2024-05-20T18:27:33.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.524+0000 7fd4d24ca640 1 -- 172.21.15.5:0/2837601474 shutdown_connections 2024-05-20T18:27:33.527 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.3 2024-05-20T18:27:33.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.524+0000 7fd4d24ca640 1 --2- 172.21.15.5:0/2837601474 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd4b806be80 0x7fd4b806e390 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.524+0000 7fd4d24ca640 1 --2- 172.21.15.5:0/2837601474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4cc0cc850 0x7fd4cc15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.524+0000 7fd4d24ca640 1 --2- 172.21.15.5:0/2837601474 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4cc0cc440 0x7fd4cc15a7d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.524+0000 7fd4d24ca640 1 --2- 172.21.15.5:0/2837601474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4b4004800 0x7fd4cc1fd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.525+0000 7fd4d24ca640 1 -- 172.21.15.5:0/2837601474 >> 172.21.15.5:0/2837601474 conn(0x7fd4cc0b44d0 msgr2=0x7fd4cc0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:33.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.525+0000 7fd4d24ca640 1 -- 172.21.15.5:0/2837601474 shutdown_connections 2024-05-20T18:27:33.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.526+0000 7fd4d24ca640 1 -- 172.21.15.5:0/2837601474 wait complete. 2024-05-20T18:27:33.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.527+0000 7f48ce348640 1 -- 172.21.15.5:0/1837851741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f48a8005850 con 0x7f48c8163600 2024-05-20T18:27:33.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.528+0000 7f48bdffb640 1 -- 172.21.15.5:0/1837851741 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f48c405c9c0 con 0x7f48c8163600 2024-05-20T18:27:33.529 INFO:teuthology.orchestra.run.smithi005.stdout:55834574855 2024-05-20T18:27:33.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.532+0000 7f48ce348640 1 -- 172.21.15.5:0/1837851741 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f48ac06c030 msgr2=0x7f48ac06e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.532+0000 7f48ce348640 1 --2- 172.21.15.5:0/1837851741 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f48ac06c030 0x7f48ac06e540 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f48c000f1d0 tx=0x7f48c0009040 comp rx=0 tx=0).stop 2024-05-20T18:27:33.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.532+0000 7f48ce348640 1 -- 172.21.15.5:0/1837851741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48c8163600 msgr2=0x7f48c8165200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:33.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.532+0000 7f48ce348640 1 --2- 172.21.15.5:0/1837851741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48c8163600 0x7f48c8165200 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f48c400e7c0 tx=0x7f48c400ec80 comp rx=0 tx=0).stop 2024-05-20T18:27:33.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.532+0000 7f48ce348640 1 -- 172.21.15.5:0/1837851741 shutdown_connections 2024-05-20T18:27:33.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.532+0000 7f48ce348640 1 --2- 172.21.15.5:0/1837851741 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f48ac06c030 0x7f48ac06e540 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.532+0000 7f48ce348640 1 --2- 172.21.15.5:0/1837851741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48c8163600 0x7f48c8165200 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.532+0000 7f48ce348640 1 --2- 172.21.15.5:0/1837851741 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48c80cfe10 0x7f48c8164cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.532+0000 7f48ce348640 1 --2- 172.21.15.5:0/1837851741 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48c806ca70 0x7f48c8164780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:33.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.532+0000 7f48ce348640 1 -- 172.21.15.5:0/1837851741 >> 172.21.15.5:0/1837851741 conn(0x7f48c80721d0 msgr2=0x7f48c806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:33.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.532+0000 7f48ce348640 1 -- 172.21.15.5:0/1837851741 shutdown_connections 2024-05-20T18:27:33.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:33.532+0000 7f48ce348640 1 -- 172.21.15.5:0/1837851741 wait complete. 2024-05-20T18:27:33.538 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.1 2024-05-20T18:27:33.540 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.2 2024-05-20T18:27:33.540 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.0 2024-05-20T18:27:33.542 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.4 2024-05-20T18:27:33.543 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.5 2024-05-20T18:27:33.548 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.6 2024-05-20T18:27:34.524 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-20T18:27:34.527 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-20T18:27:34.538 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-20T18:27:34.541 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-20T18:27:34.541 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-20T18:27:34.543 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-20T18:27:34.544 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-20T18:27:34.549 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-20T18:27:34.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.722+0000 7ff0ecbd5640 1 Processor -- start 2024-05-20T18:27:34.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.722+0000 7ff0ecbd5640 1 -- start start 2024-05-20T18:27:34.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.725+0000 7f15d2ee9640 1 Processor -- start 2024-05-20T18:27:34.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.727+0000 7ff0ecbd5640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff0e806c530 0x7ff0e806c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.727+0000 7ff0ecbd5640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0e806ce80 con 0x7ff0e80d0100 2024-05-20T18:27:34.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.727+0000 7ff0ecbd5640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0e815bbd0 con 0x7ff0e8072d40 2024-05-20T18:27:34.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.727+0000 7ff0ecbd5640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff0e815c420 con 0x7ff0e806c530 2024-05-20T18:27:34.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.728+0000 7ff0e6d9d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff0e806c530 0x7ff0e806c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.728+0000 7ff0e6d9d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff0e806c530 0x7ff0e806c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:41414/0 (socket says 172.21.15.5:41414) 2024-05-20T18:27:34.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.728+0000 7ff0e6d9d640 1 -- 172.21.15.5:0/4289893602 learned_addr learned my addr 172.21.15.5:0/4289893602 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:34.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.726+0000 7f15d2ee9640 1 -- start start 2024-05-20T18:27:34.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.730+0000 7f15d2ee9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15c4050670 0x7f15c40d9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.730+0000 7f15d2ee9640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f15c4050a80 0x7f15c40d9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.730+0000 7f15d2ee9640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15c40e0c10 con 0x7f15c4055840 2024-05-20T18:27:34.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.730+0000 7f15d2ee9640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f15c4007120 con 0x7f15c4050a80 2024-05-20T18:27:34.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.730+0000 7f15d2ee9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f15c40538b0 con 0x7f15c4050670 2024-05-20T18:27:34.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.730+0000 7ff0e559a640 1 -- 172.21.15.5:0/4289893602 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1946995224 0 0) 0x7ff0d8001490 con 0x7ff0e80d0100 2024-05-20T18:27:34.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.730+0000 7ff0e559a640 1 -- 172.21.15.5:0/4289893602 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0c8003520 con 0x7ff0e80d0100 2024-05-20T18:27:34.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.730+0000 7f15d1486640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f15c4055840 0x7f15c40dfe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:45108/0 (socket says 172.21.15.5:45108) 2024-05-20T18:27:34.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.730+0000 7f15d1486640 1 -- 172.21.15.5:0/1217313420 learned_addr learned my addr 172.21.15.5:0/1217313420 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:34.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.730+0000 7f15cbfff640 1 --2- 172.21.15.5:0/1217313420 >> v2:172.21.15.63:3300/0 conn(0x7f15c4050a80 0x7f15c40d9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.731+0000 7f15cb7fe640 1 -- 172.21.15.5:0/1217313420 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3383594975 0 0) 0x7f15c40e0c10 con 0x7f15c4055840 2024-05-20T18:27:34.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.731+0000 7f15cb7fe640 1 -- 172.21.15.5:0/1217313420 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15bc003520 con 0x7f15c4055840 2024-05-20T18:27:34.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.730+0000 7ff0e6d9d640 1 -- 172.21.15.5:0/4289893602 >> v1:172.21.15.63:6789/0 conn(0x7ff0e8072d40 legacy=0x7ff0e815b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:34.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.730+0000 7ff0e6d9d640 1 -- 172.21.15.5:0/4289893602 >> v1:172.21.15.5:6789/0 conn(0x7ff0e80d0100 legacy=0x7ff0e80b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:34.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.730+0000 7ff0e6d9d640 1 -- 172.21.15.5:0/4289893602 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e815be60 con 0x7ff0e806c530 2024-05-20T18:27:34.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.732+0000 7f15d0c85640 1 --2- 172.21.15.5:0/1217313420 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15c4050670 0x7f15c40d9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.732+0000 7f15cb7fe640 1 -- 172.21.15.5:0/1217313420 <== mon.1 v1:172.21.15.63:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2248813557 0 0) 0x7f15bc003520 con 0x7f15c4055840 2024-05-20T18:27:34.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.732+0000 7ff0e6d9d640 1 --2- 172.21.15.5:0/4289893602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0e806c530 0x7ff0e806c940 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00098a0 tx=0x7ff0e002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=892bc7100d8712df server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:34.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.733+0000 7ff0e559a640 1 -- 172.21.15.5:0/4289893602 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0037070 con 0x7ff0e806c530 2024-05-20T18:27:34.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.732+0000 7f15cbfff640 1 -- 172.21.15.5:0/1217313420 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15c4050670 msgr2=0x7f15c40d9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:34.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.732+0000 7f15cbfff640 1 --2- 172.21.15.5:0/1217313420 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15c4050670 0x7f15c40d9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.732+0000 7f15cbfff640 1 -- 172.21.15.5:0/1217313420 >> v1:172.21.15.63:6789/0 conn(0x7f15c4055840 legacy=0x7f15c40dfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:34.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.732+0000 7f15cbfff640 1 -- 172.21.15.5:0/1217313420 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c4053b90 con 0x7f15c4050a80 2024-05-20T18:27:34.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.733+0000 7ff0e559a640 1 -- 172.21.15.5:0/4289893602 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0e0032440 con 0x7ff0e806c530 2024-05-20T18:27:34.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.734+0000 7f15cbfff640 1 --2- 172.21.15.5:0/1217313420 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15c4050a80 0x7f15c40d9e60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f15b80097a0 tx=0x7f15b802a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a184be879de4720 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:34.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.734+0000 7ff0ecbd5640 1 -- 172.21.15.5:0/4289893602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0e806c530 msgr2=0x7ff0e806c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:34.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.735+0000 7ff0e559a640 1 -- 172.21.15.5:0/4289893602 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0037070 con 0x7ff0e806c530 2024-05-20T18:27:34.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.735+0000 7ff0ecbd5640 1 --2- 172.21.15.5:0/4289893602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0e806c530 0x7ff0e806c940 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00098a0 tx=0x7ff0e002a270 comp rx=0 tx=0).stop 2024-05-20T18:27:34.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.735+0000 7f15d0c85640 1 --2- 172.21.15.5:0/1217313420 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15c4050670 0x7f15c40d9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:34.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.735+0000 7ff0ecbd5640 1 -- 172.21.15.5:0/4289893602 shutdown_connections 2024-05-20T18:27:34.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.735+0000 7ff0ecbd5640 1 --2- 172.21.15.5:0/4289893602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0e806c530 0x7ff0e806c940 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.735+0000 7ff0ecbd5640 1 -- 172.21.15.5:0/4289893602 >> 172.21.15.5:0/4289893602 conn(0x7ff0e8072220 msgr2=0x7ff0e8072630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:34.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.735+0000 7ff0ecbd5640 1 -- 172.21.15.5:0/4289893602 shutdown_connections 2024-05-20T18:27:34.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.737+0000 7f15cb7fe640 1 -- 172.21.15.5:0/1217313420 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b8037070 con 0x7f15c4050a80 2024-05-20T18:27:34.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.737+0000 7f15cb7fe640 1 -- 172.21.15.5:0/1217313420 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15cc074750 con 0x7f15c4050a80 2024-05-20T18:27:34.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.735+0000 7ff0ecbd5640 1 -- 172.21.15.5:0/4289893602 wait complete. 2024-05-20T18:27:34.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.738+0000 7f15d2ee9640 1 -- 172.21.15.5:0/1217313420 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15c4050a80 msgr2=0x7f15c40d9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:34.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.738+0000 7f15d2ee9640 1 --2- 172.21.15.5:0/1217313420 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15c4050a80 0x7f15c40d9e60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f15b80097a0 tx=0x7f15b802a040 comp rx=0 tx=0).stop 2024-05-20T18:27:34.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.738+0000 7ff0ecbd5640 1 Processor -- start 2024-05-20T18:27:34.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.739+0000 7f15d2ee9640 1 -- 172.21.15.5:0/1217313420 shutdown_connections 2024-05-20T18:27:34.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.739+0000 7f15d2ee9640 1 --2- 172.21.15.5:0/1217313420 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15c4050a80 0x7f15c40d9e60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.739+0000 7f15d2ee9640 1 --2- 172.21.15.5:0/1217313420 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15c4050670 0x7f15c40d9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.739+0000 7f15d2ee9640 1 -- 172.21.15.5:0/1217313420 >> 172.21.15.5:0/1217313420 conn(0x7f15c4054d20 msgr2=0x7f15c4055130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:34.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.739+0000 7ff0ecbd5640 1 -- start start 2024-05-20T18:27:34.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.741+0000 7ff0ecbd5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0e806c530 0x7ff0e80ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.741+0000 7ff0ecbd5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0e8072d40 0x7ff0e8202250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.742+0000 7ff0e659c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0e806c530 0x7ff0e80ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.742+0000 7ff0e659c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0e806c530 0x7ff0e80ceed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:41438/0 (socket says 172.21.15.5:41438) 2024-05-20T18:27:34.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.742+0000 7ff0e5d9b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0e8072d40 0x7ff0e8202250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.742+0000 7ff0e5d9b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0e8072d40 0x7ff0e8202250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:37062/0 (socket says 172.21.15.5:37062) 2024-05-20T18:27:34.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.742+0000 7ff0ecbd5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0e80d0100 0x7ff0e8161320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.742+0000 7ff0e659c640 1 -- 172.21.15.5:0/2120729887 learned_addr learned my addr 172.21.15.5:0/2120729887 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:34.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.743+0000 7ff0e6d9d640 1 --2- 172.21.15.5:0/2120729887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0e80d0100 0x7ff0e8161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.743+0000 7ff0ecbd5640 1 -- 172.21.15.5:0/2120729887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e80b3770 con 0x7ff0e80d0100 2024-05-20T18:27:34.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.743+0000 7ff0ecbd5640 1 -- 172.21.15.5:0/2120729887 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e80b3ae0 con 0x7ff0e806c530 2024-05-20T18:27:34.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.743+0000 7ff0ecbd5640 1 -- 172.21.15.5:0/2120729887 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff0e80b4680 con 0x7ff0e8072d40 2024-05-20T18:27:34.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.743+0000 7ff0e659c640 1 -- 172.21.15.5:0/2120729887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0e8072d40 msgr2=0x7ff0e8202250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:34.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.743+0000 7ff0e659c640 1 --2- 172.21.15.5:0/2120729887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0e8072d40 0x7ff0e8202250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.743+0000 7ff0e659c640 1 -- 172.21.15.5:0/2120729887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0e80d0100 msgr2=0x7ff0e8161320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:34.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.743+0000 7ff0e659c640 1 --2- 172.21.15.5:0/2120729887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0e80d0100 0x7ff0e8161320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.743+0000 7ff0e659c640 1 -- 172.21.15.5:0/2120729887 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e80cf410 con 0x7ff0e806c530 2024-05-20T18:27:34.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.744+0000 7ff0e659c640 1 --2- 172.21.15.5:0/2120729887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0e806c530 0x7ff0e80ceed0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff0d000c400 tx=0x7ff0d000c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:34.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.744+0000 7ff0e6d9d640 1 --2- 172.21.15.5:0/2120729887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0e80d0100 0x7ff0e8161320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:34.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.744+0000 7ff0deffd640 1 -- 172.21.15.5:0/2120729887 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d0017070 con 0x7ff0e806c530 2024-05-20T18:27:34.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.745+0000 7ff0ecbd5640 1 -- 172.21.15.5:0/2120729887 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0e8145d20 con 0x7ff0e806c530 2024-05-20T18:27:34.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.739+0000 7f15d2ee9640 1 -- 172.21.15.5:0/1217313420 shutdown_connections 2024-05-20T18:27:34.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.739+0000 7f15d2ee9640 1 -- 172.21.15.5:0/1217313420 wait complete. 2024-05-20T18:27:34.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.742+0000 7f15d2ee9640 1 Processor -- start 2024-05-20T18:27:34.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.742+0000 7f15d2ee9640 1 -- start start 2024-05-20T18:27:34.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.745+0000 7f15d2ee9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15c4050670 0x7f15c4196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.745+0000 7f15d2ee9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15c4055840 0x7f15c40dc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.745+0000 7ff0ecbd5640 1 -- 172.21.15.5:0/2120729887 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0e81462e0 con 0x7ff0e806c530 2024-05-20T18:27:34.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.745+0000 7f15d2ee9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15c40d8230 0x7f15c40dcba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.745+0000 7f15cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15c4055840 0x7f15c40dc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.745+0000 7f15cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15c4055840 0x7f15c40dc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52794/0 (socket says 172.21.15.5:52794) 2024-05-20T18:27:34.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.746+0000 7f15cbfff640 1 -- 172.21.15.5:0/2953615906 learned_addr learned my addr 172.21.15.5:0/2953615906 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:34.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.746+0000 7f15d1486640 1 --2- 172.21.15.5:0/2953615906 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15c40d8230 0x7f15c40dcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.746+0000 7f15d2ee9640 1 -- 172.21.15.5:0/2953615906 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f15c4054670 con 0x7f15c4055840 2024-05-20T18:27:34.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.746+0000 7f15d2ee9640 1 -- 172.21.15.5:0/2953615906 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f15c4054150 con 0x7f15c4050670 2024-05-20T18:27:34.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.746+0000 7f15d2ee9640 1 -- 172.21.15.5:0/2953615906 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f15c4197ea0 con 0x7f15c40d8230 2024-05-20T18:27:34.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.746+0000 7f15d1486640 1 -- 172.21.15.5:0/2953615906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15c4050670 msgr2=0x7f15c4196a90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-20T18:27:34.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.746+0000 7f15d1486640 1 --2- 172.21.15.5:0/2953615906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15c4050670 0x7f15c4196a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.746+0000 7f15d1486640 1 -- 172.21.15.5:0/2953615906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15c4055840 msgr2=0x7f15c40dc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:34.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.746+0000 7f15d1486640 1 --2- 172.21.15.5:0/2953615906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15c4055840 0x7f15c40dc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.746+0000 7f15d1486640 1 -- 172.21.15.5:0/2953615906 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c419a8b0 con 0x7f15c40d8230 2024-05-20T18:27:34.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.747+0000 7f15d1486640 1 --2- 172.21.15.5:0/2953615906 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15c40d8230 0x7f15c40dcba0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f15cc050f00 tx=0x7f15cc074f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:34.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.747+0000 7f15c97fa640 1 -- 172.21.15.5:0/2953615906 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc06b520 con 0x7f15c40d8230 2024-05-20T18:27:34.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.747+0000 7f15d2ee9640 1 -- 172.21.15.5:0/2953615906 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15c419ab90 con 0x7f15c40d8230 2024-05-20T18:27:34.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.749+0000 7ff0deffd640 1 -- 172.21.15.5:0/2120729887 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0d00052d0 con 0x7ff0e806c530 2024-05-20T18:27:34.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.749+0000 7ff0deffd640 1 -- 172.21.15.5:0/2120729887 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d0003830 con 0x7ff0e806c530 2024-05-20T18:27:34.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.749+0000 7ff0deffd640 1 -- 172.21.15.5:0/2120729887 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff0d001a070 con 0x7ff0e806c530 2024-05-20T18:27:34.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.750+0000 7ff0deffd640 1 --2- 172.21.15.5:0/2120729887 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff0cc06bf30 0x7ff0cc06e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.750+0000 7ff0e5d9b640 1 --2- 172.21.15.5:0/2120729887 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff0cc06bf30 0x7ff0cc06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.747+0000 7f15d2ee9640 1 -- 172.21.15.5:0/2953615906 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f15c419b150 con 0x7f15c40d8230 2024-05-20T18:27:34.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.751+0000 7ff0e5d9b640 1 --2- 172.21.15.5:0/2120729887 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff0cc06bf30 0x7ff0cc06e440 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff0e806ca80 tx=0x7ff0d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:34.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.751+0000 7ff0deffd640 1 -- 172.21.15.5:0/2120729887 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff0d008a5e0 con 0x7ff0e806c530 2024-05-20T18:27:34.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.752+0000 7f15c97fa640 1 -- 172.21.15.5:0/2953615906 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15cc06b6c0 con 0x7f15c40d8230 2024-05-20T18:27:34.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.752+0000 7ff0ecbd5640 1 -- 172.21.15.5:0/2120729887 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0c0005290 con 0x7ff0e806c530 2024-05-20T18:27:34.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.752+0000 7f15c97fa640 1 -- 172.21.15.5:0/2953615906 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc06bb30 con 0x7f15c40d8230 2024-05-20T18:27:34.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.752+0000 7f15c97fa640 1 -- 172.21.15.5:0/2953615906 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f15cc084430 con 0x7f15c40d8230 2024-05-20T18:27:34.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.753+0000 7f15c97fa640 1 --2- 172.21.15.5:0/2953615906 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f15bc06c200 0x7f15bc06e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.753+0000 7f15d0c85640 1 --2- 172.21.15.5:0/2953615906 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f15bc06c200 0x7f15bc06e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.754+0000 7f15c97fa640 1 -- 172.21.15.5:0/2953615906 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f15cc0f5580 con 0x7f15c40d8230 2024-05-20T18:27:34.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.754+0000 7f15d0c85640 1 --2- 172.21.15.5:0/2953615906 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f15bc06c200 0x7f15bc06e710 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f15c0006fd0 tx=0x7f15c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:34.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.755+0000 7f15d2ee9640 1 -- 172.21.15.5:0/2953615906 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15ac005290 con 0x7f15c40d8230 2024-05-20T18:27:34.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.757+0000 7ff0deffd640 1 -- 172.21.15.5:0/2120729887 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff0d00585e0 con 0x7ff0e806c530 2024-05-20T18:27:34.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.760+0000 7f15c97fa640 1 -- 172.21.15.5:0/2953615906 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f15cc0c3580 con 0x7f15c40d8230 2024-05-20T18:27:34.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.793+0000 7f4485181640 1 Processor -- start 2024-05-20T18:27:34.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.793+0000 7f4485181640 1 -- start start 2024-05-20T18:27:34.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.794+0000 7fa5ce921640 1 Processor -- start 2024-05-20T18:27:34.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.794+0000 7fa5ce921640 1 -- start start 2024-05-20T18:27:34.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.795+0000 7f4485181640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f4478048320 0x7f44780382b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.795+0000 7f4485181640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4478048020 con 0x7f447804f340 2024-05-20T18:27:34.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.795+0000 7f4485181640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4478006180 con 0x7f4478033950 2024-05-20T18:27:34.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.795+0000 7f4485181640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f4478053210 con 0x7f4478048320 2024-05-20T18:27:34.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.796+0000 7f447ffff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f447804f340 0x7f44780e7b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:40968/0 (socket says 172.21.15.5:40968) 2024-05-20T18:27:34.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.796+0000 7f447ffff640 1 -- 172.21.15.5:0/1439647110 learned_addr learned my addr 172.21.15.5:0/1439647110 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:34.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.796+0000 7f447f7fe640 1 --2- 172.21.15.5:0/1439647110 >> v2:172.21.15.63:3300/0 conn(0x7f4478048320 0x7f44780382b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.796+0000 7fa5ce921640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5c00544f0 0x7fa5c00506c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.796+0000 7fa5ce921640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa5c00e33a0 0x7fa5c00e37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.796+0000 7fa5ce921640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa5c00dfdb0 0x7fa5c00e01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.796+0000 7fa5ce921640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fa5c0054060 con 0x7fa5c00dfdb0 2024-05-20T18:27:34.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.796+0000 7fa5ce921640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa5c00e3cf0 con 0x7fa5c00e33a0 2024-05-20T18:27:34.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.796+0000 7fa5ce921640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa5c0053a30 con 0x7fa5c00544f0 2024-05-20T18:27:34.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.797+0000 7fa5c77fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa5c00e33a0 0x7fa5c00e37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.797+0000 7fa5c7fff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5c00544f0 0x7fa5c00506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.797+0000 7fa5c7fff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5c00544f0 0x7fa5c00506c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:37070/0 (socket says 172.21.15.5:37070) 2024-05-20T18:27:34.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.797+0000 7fa5c7fff640 1 -- 172.21.15.5:0/2675254128 learned_addr learned my addr 172.21.15.5:0/2675254128 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:34.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.797+0000 7fa5ccebe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa5c00dfdb0 0x7fa5c00e01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.797+0000 7fa5c7fff640 1 -- 172.21.15.5:0/2675254128 >> v2:172.21.15.63:3300/0 conn(0x7fa5c00e33a0 msgr2=0x7fa5c00e37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:34.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.797+0000 7fa5c7fff640 1 --2- 172.21.15.5:0/2675254128 >> v2:172.21.15.63:3300/0 conn(0x7fa5c00e33a0 0x7fa5c00e37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.797+0000 7fa5c7fff640 1 -- 172.21.15.5:0/2675254128 >> v2:172.21.15.5:3300/0 conn(0x7fa5c00dfdb0 msgr2=0x7fa5c00e01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:34.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.797+0000 7fa5c7fff640 1 --2- 172.21.15.5:0/2675254128 >> v2:172.21.15.5:3300/0 conn(0x7fa5c00dfdb0 0x7fa5c00e01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.797+0000 7fa5c7fff640 1 -- 172.21.15.5:0/2675254128 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c00e0ac0 con 0x7fa5c00544f0 2024-05-20T18:27:34.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.797+0000 7f447effd640 1 -- 172.21.15.5:0/1439647110 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3687261802 0 0) 0x7f4478048020 con 0x7f447804f340 2024-05-20T18:27:34.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.797+0000 7f447f7fe640 1 -- 172.21.15.5:0/1439647110 >> v1:172.21.15.63:6789/0 conn(0x7f4478033950 legacy=0x7f4478033d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:34.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.797+0000 7f447f7fe640 1 -- 172.21.15.5:0/1439647110 >> v1:172.21.15.5:6789/0 conn(0x7f447804f340 legacy=0x7f44780e7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:34.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.797+0000 7f447f7fe640 1 -- 172.21.15.5:0/1439647110 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4478006460 con 0x7f4478048320 2024-05-20T18:27:34.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.798+0000 7f94bf59e640 1 Processor -- start 2024-05-20T18:27:34.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.798+0000 7f15b9bb9640 1 Processor -- start 2024-05-20T18:27:34.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.798+0000 7fa5c7fff640 1 --2- 172.21.15.5:0/2675254128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5c00544f0 0x7fa5c00506c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0037910 tx=0x7fa5bc02a700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aac62699c30a713b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:34.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.798+0000 7f447f7fe640 1 --2- 172.21.15.5:0/1439647110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4478048320 0x7f44780382b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f446c0097a0 tx=0x7f446c02a160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=910a7652c7950d50 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:34.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.798+0000 7f94bf59e640 1 -- start start 2024-05-20T18:27:34.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.799+0000 7fa5c6ffd640 1 -- 172.21.15.5:0/2675254128 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc036070 con 0x7fa5c00544f0 2024-05-20T18:27:34.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.799+0000 7f447effd640 1 -- 172.21.15.5:0/1439647110 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c037070 con 0x7f4478048320 2024-05-20T18:27:34.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.799+0000 7fa5c6ffd640 1 -- 172.21.15.5:0/2675254128 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5bc00e5a0 con 0x7fa5c00544f0 2024-05-20T18:27:34.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.799+0000 7f447effd640 1 -- 172.21.15.5:0/1439647110 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f446c032440 con 0x7f4478048320 2024-05-20T18:27:34.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.799+0000 7fa5c6ffd640 1 -- 172.21.15.5:0/2675254128 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc00c950 con 0x7fa5c00544f0 2024-05-20T18:27:34.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.799+0000 7f447effd640 1 -- 172.21.15.5:0/1439647110 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c032850 con 0x7f4478048320 2024-05-20T18:27:34.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.798+0000 7f15b9bb9640 1 -- start start 2024-05-20T18:27:34.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.800+0000 7f4485181640 1 -- 172.21.15.5:0/1439647110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4478048320 msgr2=0x7f44780382b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:34.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.800+0000 7f4485181640 1 --2- 172.21.15.5:0/1439647110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4478048320 0x7f44780382b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f446c0097a0 tx=0x7f446c02a160 comp rx=0 tx=0).stop 2024-05-20T18:27:34.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.801+0000 7f4485181640 1 -- 172.21.15.5:0/1439647110 shutdown_connections 2024-05-20T18:27:34.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.801+0000 7f4485181640 1 --2- 172.21.15.5:0/1439647110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4478048320 0x7f44780382b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.801+0000 7f4485181640 1 -- 172.21.15.5:0/1439647110 >> 172.21.15.5:0/1439647110 conn(0x7f4478054cf0 msgr2=0x7f4478055100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:34.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.801+0000 7f773bb77640 1 Processor -- start 2024-05-20T18:27:34.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.801+0000 7f94bf59e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f94b8052fb0 0x7f94b804f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.801+0000 7f94bf59e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94b80e8230 con 0x7f94b8034180 2024-05-20T18:27:34.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.801+0000 7f773bb77640 1 -- start start 2024-05-20T18:27:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.801+0000 7f94bf59e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94b8006280 con 0x7f94b8007320 2024-05-20T18:27:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.801+0000 7f94bf59e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f94b80533c0 con 0x7f94b8052fb0 2024-05-20T18:27:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.801+0000 7f15b9bb9640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f15b4072cf0 0x7f15b415c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.801+0000 7f15b9bb9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f15b40ccb10 0x7f15b406c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.801+0000 7f15b9bb9640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15b406ce80 con 0x7f15b406ca70 2024-05-20T18:27:34.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.801+0000 7f15b9bb9640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f15b40ccf20 con 0x7f15b40ccb10 2024-05-20T18:27:34.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.801+0000 7f15b9bb9640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f15b4070070 con 0x7f15b4072cf0 2024-05-20T18:27:34.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.801+0000 7f4485181640 1 -- 172.21.15.5:0/1439647110 shutdown_connections 2024-05-20T18:27:34.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.801+0000 7f15b3fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f15b40ccb10 0x7f15b406c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.801+0000 7f15b8bb7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f15b4072cf0 0x7f15b415c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.801+0000 7f15b3fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f15b40ccb10 0x7f15b406c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52806/0 (socket says 172.21.15.5:52806) 2024-05-20T18:27:34.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.802+0000 7f15b3fff640 1 -- 172.21.15.5:0/1047839403 learned_addr learned my addr 172.21.15.5:0/1047839403 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:34.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.801+0000 7f15b93b8640 1 --1- 172.21.15.5:0/1047839403 >> v1:172.21.15.63:6789/0 conn(0x7f15b406ca70 0x7f15b41517c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:45128/0 (socket says 172.21.15.5:45128) 2024-05-20T18:27:34.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.802+0000 7f94bed9d640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f94b8034180 0x7f94b8038340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:40972/0 (socket says 172.21.15.5:40972) 2024-05-20T18:27:34.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.802+0000 7f94bed9d640 1 -- 172.21.15.5:0/3917416127 learned_addr learned my addr 172.21.15.5:0/3917416127 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:34.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.802+0000 7fa5ce921640 1 -- 172.21.15.5:0/2675254128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5c00544f0 msgr2=0x7fa5c00506c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:34.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.802+0000 7fa5ce921640 1 --2- 172.21.15.5:0/2675254128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5c00544f0 0x7fa5c00506c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0037910 tx=0x7fa5bc02a700 comp rx=0 tx=0).stop 2024-05-20T18:27:34.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.802+0000 7f94be59c640 1 --2- 172.21.15.5:0/3917416127 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f94b8052fb0 0x7f94b804f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.802+0000 7fa5ce921640 1 -- 172.21.15.5:0/2675254128 shutdown_connections 2024-05-20T18:27:34.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.802+0000 7fa5ce921640 1 --2- 172.21.15.5:0/2675254128 >> v2:172.21.15.5:3300/0 conn(0x7fa5c00dfdb0 0x7fa5c00e01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.802+0000 7fa5ce921640 1 --2- 172.21.15.5:0/2675254128 >> v2:172.21.15.63:3300/0 conn(0x7fa5c00e33a0 0x7fa5c00e37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.802+0000 7fa5ce921640 1 --2- 172.21.15.5:0/2675254128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5c00544f0 0x7fa5c00506c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.802+0000 7fa5ce921640 1 -- 172.21.15.5:0/2675254128 >> 172.21.15.5:0/2675254128 conn(0x7fa5c0054d20 msgr2=0x7fa5c0055130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:34.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.802+0000 7fa5ce921640 1 -- 172.21.15.5:0/2675254128 shutdown_connections 2024-05-20T18:27:34.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.802+0000 7fa5ce921640 1 -- 172.21.15.5:0/2675254128 wait complete. 2024-05-20T18:27:34.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.802+0000 7fd0e4ddb640 1 Processor -- start 2024-05-20T18:27:34.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.802+0000 7f15b8bb7640 1 -- 172.21.15.5:0/1047839403 >> v2:172.21.15.5:3300/0 conn(0x7f15b40ccb10 msgr2=0x7f15b406c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:34.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.802+0000 7f15b8bb7640 1 --2- 172.21.15.5:0/1047839403 >> v2:172.21.15.5:3300/0 conn(0x7f15b40ccb10 0x7f15b406c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.802+0000 7f15b8bb7640 1 -- 172.21.15.5:0/1047839403 >> v1:172.21.15.63:6789/0 conn(0x7f15b406ca70 legacy=0x7f15b41517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:34.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.802+0000 7f15b8bb7640 1 -- 172.21.15.5:0/1047839403 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b406f110 con 0x7f15b4072cf0 2024-05-20T18:27:34.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.803+0000 7f4485181640 1 -- 172.21.15.5:0/1439647110 wait complete. 2024-05-20T18:27:34.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.803+0000 7f773bb77640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f772c0382c0 0x7f772c006420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.803+0000 7f773bb77640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f772c057940 con 0x7f772c0e47f0 2024-05-20T18:27:34.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.803+0000 7f773bb77640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f772c057c20 con 0x7f772c0386d0 2024-05-20T18:27:34.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.803+0000 7f773bb77640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f772c03c1c0 con 0x7f772c0382c0 2024-05-20T18:27:34.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.803+0000 7fd0e4ddb640 1 -- start start 2024-05-20T18:27:34.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.802+0000 7f94be59c640 1 -- 172.21.15.5:0/3917416127 >> v1:172.21.15.63:6789/0 conn(0x7f94b8007320 legacy=0x7f94b8033a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:34.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.802+0000 7f94be59c640 1 -- 172.21.15.5:0/3917416127 >> v1:172.21.15.5:6789/0 conn(0x7f94b8034180 legacy=0x7f94b8038340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:34.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.804+0000 7f15b8bb7640 1 --2- 172.21.15.5:0/1047839403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15b4072cf0 0x7f15b415c010 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f15b40713e0 tx=0x7f15a802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15d4534475fc7758 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:34.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.802+0000 7f94be59c640 1 -- 172.21.15.5:0/3917416127 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b8006560 con 0x7f94b8052fb0 2024-05-20T18:27:34.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.804+0000 7f94be59c640 1 --2- 172.21.15.5:0/3917416127 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f94b8052fb0 0x7f94b804f340 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f94b8053540 tx=0x7f94c00a5f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bce439e2e39ab390 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:34.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.804+0000 7f7739913640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f772c0e47f0 0x7f772c037bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:40984/0 (socket says 172.21.15.5:40984) 2024-05-20T18:27:34.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.804+0000 7f7739913640 1 -- 172.21.15.5:0/3158220776 learned_addr learned my addr 172.21.15.5:0/3158220776 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:34.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.804+0000 7f15b37fe640 1 -- 172.21.15.5:0/1047839403 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a8037070 con 0x7f15b4072cf0 2024-05-20T18:27:34.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.804+0000 7f15b37fe640 1 -- 172.21.15.5:0/1047839403 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15a802ae10 con 0x7f15b4072cf0 2024-05-20T18:27:34.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.804+0000 7f15b37fe640 1 -- 172.21.15.5:0/1047839403 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a8032660 con 0x7f15b4072cf0 2024-05-20T18:27:34.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.804+0000 7f94bd59a640 1 -- 172.21.15.5:0/3917416127 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c00b3070 con 0x7f94b8052fb0 2024-05-20T18:27:34.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.805+0000 7fa5ce921640 1 Processor -- start 2024-05-20T18:27:34.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.806+0000 7f15b9bb9640 1 -- 172.21.15.5:0/1047839403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15b4072cf0 msgr2=0x7f15b415c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:34.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.804+0000 7f7738911640 1 -- 172.21.15.5:0/3158220776 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4248089149 0 0) 0x7f772c057940 con 0x7f772c0e47f0 2024-05-20T18:27:34.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.804+0000 7f7738911640 1 -- 172.21.15.5:0/3158220776 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7720003520 con 0x7f772c0e47f0 2024-05-20T18:27:34.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.804+0000 7f7738911640 1 -- 172.21.15.5:0/3158220776 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1712256526 0 0) 0x7f772c057c20 con 0x7f772c0386d0 2024-05-20T18:27:34.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.804+0000 7f7738911640 1 -- 172.21.15.5:0/3158220776 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f772c057940 con 0x7f772c0386d0 2024-05-20T18:27:34.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.805+0000 7f7738911640 1 -- 172.21.15.5:0/3158220776 <== mon.1 v1:172.21.15.63:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2963107509 0 0) 0x7f772c057940 con 0x7f772c0386d0 2024-05-20T18:27:34.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.805+0000 7f7738911640 1 -- 172.21.15.5:0/3158220776 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f772c0382c0 msgr2=0x7f772c006420 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-20T18:27:34.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.805+0000 7f7738911640 1 --2- 172.21.15.5:0/3158220776 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f772c0382c0 0x7f772c006420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.805+0000 7f7738911640 1 -- 172.21.15.5:0/3158220776 >> v1:172.21.15.5:6789/0 conn(0x7f772c0e47f0 legacy=0x7f772c037bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:34.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.805+0000 7f7738911640 1 -- 172.21.15.5:0/3158220776 --> v1:172.21.15.63:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f772c057f00 con 0x7f772c0386d0 2024-05-20T18:27:34.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.805+0000 7f7738911640 1 -- 172.21.15.5:0/3158220776 <== mon.1 v1:172.21.15.63:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f7734074750 con 0x7f772c0386d0 2024-05-20T18:27:34.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.805+0000 7f7738911640 1 -- 172.21.15.5:0/3158220776 >> v1:172.21.15.63:6789/0 conn(0x7f772c0386d0 legacy=0x7f772c006960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:34.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.805+0000 7f7738911640 1 --2- 172.21.15.5:0/3158220776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7720003b70 0x7f7720003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.805+0000 7f7738911640 1 --2- 172.21.15.5:0/3158220776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7720004800 0x7f7720004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.805+0000 7f7738911640 1 --2- 172.21.15.5:0/3158220776 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7720005350 0x7f77200079a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.805+0000 7f7738911640 1 -- 172.21.15.5:0/3158220776 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f772c03c1c0 con 0x7f7720004800 2024-05-20T18:27:34.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.805+0000 7f7738911640 1 -- 172.21.15.5:0/3158220776 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7720008190 con 0x7f7720003b70 2024-05-20T18:27:34.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.805+0000 7f7738911640 1 -- 172.21.15.5:0/3158220776 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7720008390 con 0x7f7720005350 2024-05-20T18:27:34.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.805+0000 7f773a114640 1 --2- 172.21.15.5:0/3158220776 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7720005350 0x7f77200079a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.806+0000 7f7739913640 1 --2- 172.21.15.5:0/3158220776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7720003b70 0x7f7720003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.806+0000 7f773a114640 1 -- 172.21.15.5:0/3158220776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7720003b70 msgr2=0x7f7720003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:34.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.806+0000 7f773a114640 1 --2- 172.21.15.5:0/3158220776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7720003b70 0x7f7720003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.806+0000 7f773a114640 1 -- 172.21.15.5:0/3158220776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7720004800 msgr2=0x7f7720004c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-20T18:27:34.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.806+0000 7f773a114640 1 --2- 172.21.15.5:0/3158220776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7720004800 0x7f7720004c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.806+0000 7f773a114640 1 -- 172.21.15.5:0/3158220776 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f772c057f00 con 0x7f7720005350 2024-05-20T18:27:34.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.806+0000 7fd0e4ddb640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd0d80e3db0 0x7fd0d80e7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.806+0000 7f15b9bb9640 1 --2- 172.21.15.5:0/1047839403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15b4072cf0 0x7f15b415c010 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f15b40713e0 tx=0x7f15a802a100 comp rx=0 tx=0).stop 2024-05-20T18:27:34.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.805+0000 7f94bd59a640 1 -- 172.21.15.5:0/3917416127 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94c00a6b30 con 0x7f94b8052fb0 2024-05-20T18:27:34.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.806+0000 7f94bd59a640 1 -- 172.21.15.5:0/3917416127 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c00b3070 con 0x7f94b8052fb0 2024-05-20T18:27:34.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.806+0000 7fd0e4ddb640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd0d80e73a0 0x7fd0d8057e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.806+0000 7f94bf59e640 1 -- 172.21.15.5:0/3917416127 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f94b8052fb0 msgr2=0x7f94b804f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:34.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.806+0000 7f94bf59e640 1 --2- 172.21.15.5:0/3917416127 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f94b8052fb0 0x7f94b804f340 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f94b8053540 tx=0x7f94c00a5f30 comp rx=0 tx=0).stop 2024-05-20T18:27:34.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.806+0000 7f4485181640 1 Processor -- start 2024-05-20T18:27:34.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.806+0000 7fd0df7fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd0d80e3db0 0x7fd0d80e7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.806+0000 7fd0df7fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd0d80e3db0 0x7fd0d80e7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:41500/0 (socket says 172.21.15.5:41500) 2024-05-20T18:27:34.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.806+0000 7fd0df7fe640 1 -- 172.21.15.5:0/3297005607 learned_addr learned my addr 172.21.15.5:0/3297005607 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:34.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.806+0000 7f94bf59e640 1 -- 172.21.15.5:0/3917416127 shutdown_connections 2024-05-20T18:27:34.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.806+0000 7f94bf59e640 1 --2- 172.21.15.5:0/3917416127 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f94b8052fb0 0x7f94b804f340 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f94b8053540 tx=0x7f94c00a5f30 comp rx=0 tx=0).stop 2024-05-20T18:27:34.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.806+0000 7f94bf59e640 1 -- 172.21.15.5:0/3917416127 >> 172.21.15.5:0/3917416127 conn(0x7f94b8054cf0 msgr2=0x7f94b8055100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:34.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.806+0000 7fd0deffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd0d80e73a0 0x7fd0d8057e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.807+0000 7fd0e4ddb640 1 --2- 172.21.15.5:0/3297005607 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0d8058340 0x7fd0d8037b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.806+0000 7fa5ce921640 1 -- start start 2024-05-20T18:27:34.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.807+0000 7f4485181640 1 -- start start 2024-05-20T18:27:34.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.807+0000 7fd0e4ddb640 1 -- 172.21.15.5:0/3297005607 --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fd0d8058750 con 0x7fd0d80e73a0 2024-05-20T18:27:34.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.807+0000 7f94bf59e640 1 -- 172.21.15.5:0/3917416127 shutdown_connections 2024-05-20T18:27:34.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.807+0000 7f94bf59e640 1 -- 172.21.15.5:0/3917416127 wait complete. 2024-05-20T18:27:34.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.807+0000 7fd0e4ddb640 1 -- 172.21.15.5:0/3297005607 --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd0d8006450 con 0x7fd0d80e3db0 2024-05-20T18:27:34.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.807+0000 7f15b9bb9640 1 -- 172.21.15.5:0/1047839403 shutdown_connections 2024-05-20T18:27:34.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.807+0000 7f15b9bb9640 1 --2- 172.21.15.5:0/1047839403 >> v2:172.21.15.5:3300/0 conn(0x7f15b40ccb10 0x7f15b406c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.807+0000 7f15b9bb9640 1 --2- 172.21.15.5:0/1047839403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15b4072cf0 0x7f15b415c010 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.807+0000 7fd0dffff640 1 --2- 172.21.15.5:0/3297005607 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0d8058340 0x7fd0d8037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.807+0000 7f15b9bb9640 1 -- 172.21.15.5:0/1047839403 >> 172.21.15.5:0/1047839403 conn(0x7f15b40721d0 msgr2=0x7f15b40725e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:34.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.807+0000 7fd0e4ddb640 1 -- 172.21.15.5:0/3297005607 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd0d80e4320 con 0x7fd0d8058340 2024-05-20T18:27:34.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.807+0000 7f773a114640 1 --2- 172.21.15.5:0/3158220776 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7720005350 0x7f77200079a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f773406a030 tx=0x7f7720005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88e8af4b24f864a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:34.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.807+0000 7fd0df7fe640 1 -- 172.21.15.5:0/3297005607 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0d8058340 msgr2=0x7fd0d8037b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:34.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.807+0000 7fd0df7fe640 1 --2- 172.21.15.5:0/3297005607 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0d8058340 0x7fd0d8037b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.807+0000 7fd0df7fe640 1 -- 172.21.15.5:0/3297005607 >> v2:172.21.15.5:3300/0 conn(0x7fd0d80e73a0 msgr2=0x7fd0d8057e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:34.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.807+0000 7fd0df7fe640 1 --2- 172.21.15.5:0/3297005607 >> v2:172.21.15.5:3300/0 conn(0x7fd0d80e73a0 0x7fd0d8057e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.807+0000 7fd0df7fe640 1 -- 172.21.15.5:0/3297005607 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d80e4600 con 0x7fd0d80e3db0 2024-05-20T18:27:34.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.807+0000 7fd0dffff640 1 --2- 172.21.15.5:0/3297005607 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0d8058340 0x7fd0d8037b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:34.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.807+0000 7f15b9bb9640 1 -- 172.21.15.5:0/1047839403 shutdown_connections 2024-05-20T18:27:34.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.808+0000 7f15b9bb9640 1 -- 172.21.15.5:0/1047839403 wait complete. 2024-05-20T18:27:34.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.808+0000 7f7738911640 1 -- 172.21.15.5:0/3158220776 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77340b2070 con 0x7f7720005350 2024-05-20T18:27:34.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.808+0000 7f7738911640 1 -- 172.21.15.5:0/3158220776 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77340ad620 con 0x7f7720005350 2024-05-20T18:27:34.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.808+0000 7f7738911640 1 -- 172.21.15.5:0/3158220776 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77340690b0 con 0x7f7720005350 2024-05-20T18:27:34.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.809+0000 7fd0df7fe640 1 --2- 172.21.15.5:0/3297005607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0d80e3db0 0x7fd0d80e7c90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8011800 tx=0x7fd0d402a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=228b2f91359399a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:34.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.808+0000 7fa5ce921640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5c00544f0 0x7fa5c00d2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.808+0000 7fa5c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5c00544f0 0x7fa5c00d2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.808+0000 7fa5c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5c00544f0 0x7fa5c00d2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52836/0 (socket says 172.21.15.5:52836) 2024-05-20T18:27:34.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.808+0000 7fa5c7fff640 1 -- 172.21.15.5:0/111229794 learned_addr learned my addr 172.21.15.5:0/111229794 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:34.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.808+0000 7fa5ce921640 1 --2- 172.21.15.5:0/111229794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5c00dfdb0 0x7fa5c004ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.808+0000 7fa5ce921640 1 --2- 172.21.15.5:0/111229794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5c00e33a0 0x7fa5c00da590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.808+0000 7fa5ce921640 1 -- 172.21.15.5:0/111229794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c00e0880 con 0x7fa5c00544f0 2024-05-20T18:27:34.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.808+0000 7fa5ce921640 1 -- 172.21.15.5:0/111229794 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c00e0700 con 0x7fa5c00dfdb0 2024-05-20T18:27:34.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.808+0000 7fa5ce921640 1 -- 172.21.15.5:0/111229794 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa5c0050c00 con 0x7fa5c00e33a0 2024-05-20T18:27:34.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.809+0000 7fa5ccebe640 1 --2- 172.21.15.5:0/111229794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5c00e33a0 0x7fa5c00da590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.809+0000 7fd0de7fc640 1 -- 172.21.15.5:0/3297005607 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4037070 con 0x7fd0d80e3db0 2024-05-20T18:27:34.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.809+0000 7fd0de7fc640 1 -- 172.21.15.5:0/3297005607 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0d4032440 con 0x7fd0d80e3db0 2024-05-20T18:27:34.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.809+0000 7f773bb77640 1 -- 172.21.15.5:0/3158220776 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7720005350 msgr2=0x7f77200079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:34.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.809+0000 7f773bb77640 1 --2- 172.21.15.5:0/3158220776 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7720005350 0x7f77200079a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f773406a030 tx=0x7f7720005830 comp rx=0 tx=0).stop 2024-05-20T18:27:34.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.809+0000 7f773a114640 1 -- 172.21.15.5:0/3158220776 reap_dead start 2024-05-20T18:27:34.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.809+0000 7fa5c7fff640 1 -- 172.21.15.5:0/111229794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5c00e33a0 msgr2=0x7fa5c00da590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:34.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.809+0000 7fa5c7fff640 1 --2- 172.21.15.5:0/111229794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5c00e33a0 0x7fa5c00da590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.809+0000 7fa5c7fff640 1 -- 172.21.15.5:0/111229794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5c00dfdb0 msgr2=0x7fa5c004ace0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-20T18:27:34.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.809+0000 7fa5c7fff640 1 --2- 172.21.15.5:0/111229794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5c00dfdb0 0x7fa5c004ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.809+0000 7fa5c7fff640 1 -- 172.21.15.5:0/111229794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c00d60f0 con 0x7fa5c00544f0 2024-05-20T18:27:34.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.810+0000 7fd0de7fc640 1 -- 172.21.15.5:0/3297005607 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4032810 con 0x7fd0d80e3db0 2024-05-20T18:27:34.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.810+0000 7f94bf59e640 1 Processor -- start 2024-05-20T18:27:34.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.810+0000 7f773bb77640 1 -- 172.21.15.5:0/3158220776 shutdown_connections 2024-05-20T18:27:34.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.810+0000 7f773bb77640 1 -- 172.21.15.5:0/3158220776 >> 172.21.15.5:0/3158220776 conn(0x7f772c059140 msgr2=0x7f772c059550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:34.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.810+0000 7f94bf59e640 1 -- start start 2024-05-20T18:27:34.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.810+0000 7fa5c7fff640 1 --2- 172.21.15.5:0/111229794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5c00544f0 0x7fa5c00d2350 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00ec80 tx=0x7fa5bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:34.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.810+0000 7f4485181640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4478033950 0x7f44780e9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.810+0000 7fa5c4ff9640 1 -- 172.21.15.5:0/111229794 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc036040 con 0x7fa5c00544f0 2024-05-20T18:27:34.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.810+0000 7fa5ce921640 1 -- 172.21.15.5:0/111229794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c004b220 con 0x7fa5c00544f0 2024-05-20T18:27:34.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.810+0000 7fa5c4ff9640 1 -- 172.21.15.5:0/111229794 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5bc041070 con 0x7fa5c00544f0 2024-05-20T18:27:34.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.810+0000 7fa5c4ff9640 1 -- 172.21.15.5:0/111229794 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc005c70 con 0x7fa5c00544f0 2024-05-20T18:27:34.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.810+0000 7fa5ce921640 1 -- 172.21.15.5:0/111229794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c00d6670 con 0x7fa5c00544f0 2024-05-20T18:27:34.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.810+0000 7f4485181640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4478048320 0x7f447817de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.810+0000 7f4485181640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f447804f340 0x7f44780e5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.810+0000 7f4485181640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4478006ab0 con 0x7f4478033950 2024-05-20T18:27:34.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.810+0000 7f4485181640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4478047dd0 con 0x7f447804f340 2024-05-20T18:27:34.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.810+0000 7f4485181640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f44780387f0 con 0x7f4478048320 2024-05-20T18:27:34.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.810+0000 7f4484980640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f447804f340 0x7f44780e5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.810+0000 7f4484980640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f447804f340 0x7f44780e5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:41512/0 (socket says 172.21.15.5:41512) 2024-05-20T18:27:34.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.810+0000 7f4484980640 1 -- 172.21.15.5:0/2382549770 learned_addr learned my addr 172.21.15.5:0/2382549770 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:34.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.811+0000 7f447f7fe640 1 --2- 172.21.15.5:0/2382549770 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4478048320 0x7f447817de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.811+0000 7f15b9bb9640 1 Processor -- start 2024-05-20T18:27:34.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.811+0000 7f15b9bb9640 1 -- start start 2024-05-20T18:27:34.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.811+0000 7fd0e4ddb640 1 -- 172.21.15.5:0/3297005607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0d80e3db0 msgr2=0x7fd0d80e7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:34.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.811+0000 7fd0e4ddb640 1 --2- 172.21.15.5:0/3297005607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0d80e3db0 0x7fd0d80e7c90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8011800 tx=0x7fd0d402a1e0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.812+0000 7fd0e4ddb640 1 -- 172.21.15.5:0/3297005607 shutdown_connections 2024-05-20T18:27:34.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.811+0000 7f447ffff640 1 --2- 172.21.15.5:0/2382549770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4478033950 0x7f44780e9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.812+0000 7fd0e4ddb640 1 --2- 172.21.15.5:0/3297005607 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0d8058340 0x7fd0d8037b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.812+0000 7fd0e4ddb640 1 --2- 172.21.15.5:0/3297005607 >> v2:172.21.15.5:3300/0 conn(0x7fd0d80e73a0 0x7fd0d8057e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.811+0000 7f447ffff640 1 -- 172.21.15.5:0/2382549770 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4478048320 msgr2=0x7f447817de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:34.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.812+0000 7fd0e4ddb640 1 --2- 172.21.15.5:0/3297005607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0d80e3db0 0x7fd0d80e7c90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.812+0000 7fd0e4ddb640 1 -- 172.21.15.5:0/3297005607 >> 172.21.15.5:0/3297005607 conn(0x7fd0d803ac60 msgr2=0x7fd0d803b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:34.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.811+0000 7f447ffff640 1 --2- 172.21.15.5:0/2382549770 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4478048320 0x7f447817de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.811+0000 7f447ffff640 1 -- 172.21.15.5:0/2382549770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f447804f340 msgr2=0x7f44780e5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:34.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.811+0000 7f447ffff640 1 --2- 172.21.15.5:0/2382549770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f447804f340 0x7f44780e5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.811+0000 7f447ffff640 1 -- 172.21.15.5:0/2382549770 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44780e9c60 con 0x7f4478033950 2024-05-20T18:27:34.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.812+0000 7f447ffff640 1 --2- 172.21.15.5:0/2382549770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4478033950 0x7f44780e9410 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f44780516d0 tx=0x7f447400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:34.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.812+0000 7fa5ce921640 1 -- 172.21.15.5:0/111229794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c00054d0 con 0x7fa5c00544f0 2024-05-20T18:27:34.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.812+0000 7f94bf59e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94b8007320 0x7f94b80d8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.812+0000 7f94bf59e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f94b817e010 0x7f94b816fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.812+0000 7f94bf59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94b81703a0 0x7f94b80e36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.812+0000 7f773bb77640 1 -- 172.21.15.5:0/3158220776 shutdown_connections 2024-05-20T18:27:34.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.812+0000 7f94bf59e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f94b8006e00 con 0x7f94b81703a0 2024-05-20T18:27:34.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.812+0000 7f94bf59e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f94b8006b20 con 0x7f94b8007320 2024-05-20T18:27:34.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.812+0000 7f94bf59e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f94b8007730 con 0x7f94b817e010 2024-05-20T18:27:34.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.812+0000 7f94be59c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94b8007320 0x7f94b80d8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.812+0000 7f94be59c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94b8007320 0x7f94b80d8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:41514/0 (socket says 172.21.15.5:41514) 2024-05-20T18:27:34.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.812+0000 7f94be59c640 1 -- 172.21.15.5:0/1603290546 learned_addr learned my addr 172.21.15.5:0/1603290546 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:34.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.813+0000 7f447cff9640 1 -- 172.21.15.5:0/2382549770 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44740074e0 con 0x7f4478033950 2024-05-20T18:27:34.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.813+0000 7f773bb77640 1 -- 172.21.15.5:0/3158220776 wait complete. 2024-05-20T18:27:34.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.813+0000 7fd0e4ddb640 1 -- 172.21.15.5:0/3297005607 shutdown_connections 2024-05-20T18:27:34.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.812+0000 7f94bed9d640 1 --2- 172.21.15.5:0/1603290546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94b81703a0 0x7f94b80e36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.812+0000 7f94bdd9b640 1 --2- 172.21.15.5:0/1603290546 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f94b817e010 0x7f94b816fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.813+0000 7f447cff9640 1 -- 172.21.15.5:0/2382549770 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44740047e0 con 0x7f4478033950 2024-05-20T18:27:34.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.813+0000 7f94bdd9b640 1 -- 172.21.15.5:0/1603290546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94b8007320 msgr2=0x7f94b80d8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:34.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.813+0000 7f94bdd9b640 1 --2- 172.21.15.5:0/1603290546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94b8007320 0x7f94b80d8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.813+0000 7f94bdd9b640 1 -- 172.21.15.5:0/1603290546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94b81703a0 msgr2=0x7f94b80e36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:34.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.813+0000 7f94bdd9b640 1 --2- 172.21.15.5:0/1603290546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94b81703a0 0x7f94b80e36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.813+0000 7f94bdd9b640 1 -- 172.21.15.5:0/1603290546 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b80e21d0 con 0x7f94b817e010 2024-05-20T18:27:34.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.813+0000 7f94bdd9b640 1 --2- 172.21.15.5:0/1603290546 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f94b817e010 0x7f94b816fe60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f94b800bf20 tx=0x7f94b400c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:34.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.814+0000 7f94aeffd640 1 -- 172.21.15.5:0/1603290546 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b4005220 con 0x7f94b817e010 2024-05-20T18:27:34.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.814+0000 7f94aeffd640 1 -- 172.21.15.5:0/1603290546 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94b40053c0 con 0x7f94b817e010 2024-05-20T18:27:34.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.814+0000 7f94aeffd640 1 -- 172.21.15.5:0/1603290546 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b4007250 con 0x7f94b817e010 2024-05-20T18:27:34.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.814+0000 7f94bf59e640 1 -- 172.21.15.5:0/1603290546 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94b80e2640 con 0x7f94b817e010 2024-05-20T18:27:34.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.814+0000 7f94bf59e640 1 -- 172.21.15.5:0/1603290546 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f94b80e2c00 con 0x7f94b817e010 2024-05-20T18:27:34.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.814+0000 7f15b9bb9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15b406ca70 0x7f15b413ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.814+0000 7f15b9bb9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15b4072cf0 0x7f15b406e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.814+0000 7f15b8bb7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15b406ca70 0x7f15b413ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.814+0000 7f15b8bb7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15b406ca70 0x7f15b413ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:37128/0 (socket says 172.21.15.5:37128) 2024-05-20T18:27:34.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.814+0000 7f15b9bb9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15b40ccb10 0x7f15b406e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.814+0000 7f15b3fff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15b4072cf0 0x7f15b406e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.814+0000 7f15b3fff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15b4072cf0 0x7f15b406e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:41518/0 (socket says 172.21.15.5:41518) 2024-05-20T18:27:34.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.814+0000 7f15b8bb7640 1 -- 172.21.15.5:0/2579676819 learned_addr learned my addr 172.21.15.5:0/2579676819 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:34.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.814+0000 7f15b9bb9640 1 -- 172.21.15.5:0/2579676819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f15b40cea40 con 0x7f15b40ccb10 2024-05-20T18:27:34.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.815+0000 7f15b9bb9640 1 -- 172.21.15.5:0/2579676819 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f15b40cf090 con 0x7f15b4072cf0 2024-05-20T18:27:34.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.815+0000 7f15b9bb9640 1 -- 172.21.15.5:0/2579676819 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f15b40ced20 con 0x7f15b406ca70 2024-05-20T18:27:34.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.815+0000 7f15b93b8640 1 --2- 172.21.15.5:0/2579676819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15b40ccb10 0x7f15b406e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.815+0000 7f15b93b8640 1 -- 172.21.15.5:0/2579676819 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15b406ca70 msgr2=0x7f15b413ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:34.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.815+0000 7f15b93b8640 1 --2- 172.21.15.5:0/2579676819 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15b406ca70 0x7f15b413ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.815+0000 7f15b93b8640 1 -- 172.21.15.5:0/2579676819 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15b4072cf0 msgr2=0x7f15b406e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:34.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.815+0000 7f15b93b8640 1 --2- 172.21.15.5:0/2579676819 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15b4072cf0 0x7f15b406e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.815+0000 7f15b93b8640 1 -- 172.21.15.5:0/2579676819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b41404b0 con 0x7f15b40ccb10 2024-05-20T18:27:34.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.814+0000 7f4485181640 1 -- 172.21.15.5:0/2382549770 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44780e9f40 con 0x7f4478033950 2024-05-20T18:27:34.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.814+0000 7f4485181640 1 -- 172.21.15.5:0/2382549770 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44780ea500 con 0x7f4478033950 2024-05-20T18:27:34.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.813+0000 7fd0e4ddb640 1 -- 172.21.15.5:0/3297005607 wait complete. 2024-05-20T18:27:34.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.816+0000 7f447cff9640 1 -- 172.21.15.5:0/2382549770 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4474004c10 con 0x7f4478033950 2024-05-20T18:27:34.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.816+0000 7f15b93b8640 1 --2- 172.21.15.5:0/2579676819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15b40ccb10 0x7f15b406e640 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00ff70 tx=0x7f15ac00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:34.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.816+0000 7fa5c4ff9640 1 -- 172.21.15.5:0/111229794 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa5bc00e5a0 con 0x7fa5c00544f0 2024-05-20T18:27:34.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.816+0000 7f446a7fc640 1 -- 172.21.15.5:0/2382549770 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4478038550 con 0x7f4478033950 2024-05-20T18:27:34.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.816+0000 7fd0e4ddb640 1 Processor -- start 2024-05-20T18:27:34.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.816+0000 7f773bb77640 1 Processor -- start 2024-05-20T18:27:34.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.817+0000 7fa5c4ff9640 1 --2- 172.21.15.5:0/111229794 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa5ac06be30 0x7fa5ac06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.817+0000 7fd0e4ddb640 1 -- start start 2024-05-20T18:27:34.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.817+0000 7fa5c4ff9640 1 -- 172.21.15.5:0/111229794 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa5bc0ac870 con 0x7fa5c00544f0 2024-05-20T18:27:34.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.817+0000 7f773bb77640 1 -- start start 2024-05-20T18:27:34.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.817+0000 7f94bf59e640 1 -- 172.21.15.5:0/1603290546 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94a4005290 con 0x7f94b817e010 2024-05-20T18:27:34.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.817+0000 7fa5c77fe640 1 --2- 172.21.15.5:0/111229794 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa5ac06be30 0x7fa5ac06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.818+0000 7f94aeffd640 1 -- 172.21.15.5:0/1603290546 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f94b4019020 con 0x7f94b817e010 2024-05-20T18:27:34.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.819+0000 7f773bb77640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7720005350 0x7f772c0d5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.819+0000 7f773bb77640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f772c03e200 0x7f772c03e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.819+0000 7f773bb77640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f772c0e5f60 0x7f772c0e6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.819+0000 7fa5c77fe640 1 --2- 172.21.15.5:0/111229794 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa5ac06be30 0x7fa5ac06e340 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0049d20 tx=0x7fa5b4013040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:34.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.819+0000 7f773bb77640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f772c0e4c00 con 0x7f772c0e5f60 2024-05-20T18:27:34.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.819+0000 7f773bb77640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f772c0584c0 con 0x7f7720005350 2024-05-20T18:27:34.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.819+0000 7f773bb77640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f772c04ea50 con 0x7f772c03e200 2024-05-20T18:27:34.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.819+0000 7f94aeffd640 1 --2- 172.21.15.5:0/1603290546 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f94a006be80 0x7f94a006e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.819+0000 7f94aeffd640 1 -- 172.21.15.5:0/1603290546 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f94b408a240 con 0x7f94b817e010 2024-05-20T18:27:34.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.819+0000 7f773a114640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f772c0e5f60 0x7f772c0e6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.819+0000 7f773a114640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f772c0e5f60 0x7f772c0e6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52876/0 (socket says 172.21.15.5:52876) 2024-05-20T18:27:34.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.819+0000 7f773a114640 1 -- 172.21.15.5:0/3127755484 learned_addr learned my addr 172.21.15.5:0/3127755484 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:34.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.819+0000 7f7739112640 1 --2- 172.21.15.5:0/3127755484 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f772c03e200 0x7f772c03e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.819+0000 7f94be59c640 1 --2- 172.21.15.5:0/1603290546 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f94a006be80 0x7f94a006e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.819+0000 7fd0e4ddb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0d8058340 0x7fd0d804a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.819+0000 7fd0e4ddb640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0d80e3db0 0x7fd0d80d42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.819+0000 7f773a114640 1 -- 172.21.15.5:0/3127755484 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f772c03e200 msgr2=0x7f772c03e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:34.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.819+0000 7f773a114640 1 --2- 172.21.15.5:0/3127755484 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f772c03e200 0x7f772c03e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.819+0000 7f773a114640 1 -- 172.21.15.5:0/3127755484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7720005350 msgr2=0x7f772c0d5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:34.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.819+0000 7f773a114640 1 --2- 172.21.15.5:0/3127755484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7720005350 0x7f772c0d5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.819+0000 7f773a114640 1 -- 172.21.15.5:0/3127755484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f772c0e68b0 con 0x7f772c0e5f60 2024-05-20T18:27:34.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.820+0000 7fd0df7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0d8058340 0x7fd0d804a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.820+0000 7fd0df7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0d8058340 0x7fd0d804a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52882/0 (socket says 172.21.15.5:52882) 2024-05-20T18:27:34.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.820+0000 7f7739112640 1 --2- 172.21.15.5:0/3127755484 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f772c03e200 0x7f772c03e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:34.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.820+0000 7fd0df7fe640 1 -- 172.21.15.5:0/3579487692 learned_addr learned my addr 172.21.15.5:0/3579487692 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:34.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.820+0000 7fd0e4ddb640 1 --2- 172.21.15.5:0/3579487692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0d80e73a0 0x7fd0d80e2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.820+0000 7f773a114640 1 --2- 172.21.15.5:0/3127755484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f772c0e5f60 0x7f772c0e6370 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7734069ea0 tx=0x7f77340696a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:34.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.820+0000 7fd0e4ddb640 1 -- 172.21.15.5:0/3579487692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d80e81d0 con 0x7fd0d8058340 2024-05-20T18:27:34.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.820+0000 7fd0deffd640 1 --2- 172.21.15.5:0/3579487692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0d80e3db0 0x7fd0d80d42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.820+0000 7fd0e4ddb640 1 -- 172.21.15.5:0/3579487692 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d80e4bc0 con 0x7fd0d80e73a0 2024-05-20T18:27:34.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.820+0000 7fd0dffff640 1 --2- 172.21.15.5:0/3579487692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0d80e73a0 0x7fd0d80e2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.820+0000 7fd0e4ddb640 1 -- 172.21.15.5:0/3579487692 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd0d8058a80 con 0x7fd0d80e3db0 2024-05-20T18:27:34.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.820+0000 7f94be59c640 1 --2- 172.21.15.5:0/1603290546 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f94a006be80 0x7f94a006e390 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f94c0050c70 tx=0x7f94c0061f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:34.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.820+0000 7fd0df7fe640 1 -- 172.21.15.5:0/3579487692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0d80e3db0 msgr2=0x7fd0d80d42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:34.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.820+0000 7fd0df7fe640 1 --2- 172.21.15.5:0/3579487692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0d80e3db0 0x7fd0d80d42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.820+0000 7fd0df7fe640 1 -- 172.21.15.5:0/3579487692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0d80e73a0 msgr2=0x7fd0d80e2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:34.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.820+0000 7fd0df7fe640 1 --2- 172.21.15.5:0/3579487692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0d80e73a0 0x7fd0d80e2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:34.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.820+0000 7fd0df7fe640 1 -- 172.21.15.5:0/3579487692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d80d21a0 con 0x7fd0d8058340 2024-05-20T18:27:34.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.821+0000 7fd0df7fe640 1 --2- 172.21.15.5:0/3579487692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0d8058340 0x7fd0d804a740 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4009960 tx=0x7fd0d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:34.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.822+0000 7fa5c4ff9640 1 -- 172.21.15.5:0/111229794 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa5bc07b070 con 0x7fa5c00544f0 2024-05-20T18:27:34.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.822+0000 7f15b17fa640 1 -- 172.21.15.5:0/2579676819 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac00d0d0 con 0x7f15b40ccb10 2024-05-20T18:27:34.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.822+0000 7f15b9bb9640 1 -- 172.21.15.5:0/2579676819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b414f390 con 0x7f15b40ccb10 2024-05-20T18:27:34.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.822+0000 7f772a7fc640 1 -- 172.21.15.5:0/3127755484 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77340b2040 con 0x7f772c0e5f60 2024-05-20T18:27:34.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.823+0000 7f15b9bb9640 1 -- 172.21.15.5:0/2579676819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b41539e0 con 0x7f15b40ccb10 2024-05-20T18:27:34.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.823+0000 7f772a7fc640 1 -- 172.21.15.5:0/3127755484 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77340bf070 con 0x7f772c0e5f60 2024-05-20T18:27:34.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.823+0000 7f772a7fc640 1 -- 172.21.15.5:0/3127755484 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77340690b0 con 0x7f772c0e5f60 2024-05-20T18:27:34.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.823+0000 7f15b17fa640 1 -- 172.21.15.5:0/2579676819 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15ac00d270 con 0x7f15b40ccb10 2024-05-20T18:27:34.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.823+0000 7f773bb77640 1 -- 172.21.15.5:0/3127755484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f772c03c930 con 0x7f772c0e5f60 2024-05-20T18:27:34.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.823+0000 7f773bb77640 1 -- 172.21.15.5:0/3127755484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f772c03cef0 con 0x7f772c0e5f60 2024-05-20T18:27:34.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.823+0000 7fd0cbfff640 1 -- 172.21.15.5:0/3579487692 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4037040 con 0x7fd0d8058340 2024-05-20T18:27:34.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.823+0000 7fd0cbfff640 1 -- 172.21.15.5:0/3579487692 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0d4042070 con 0x7fd0d8058340 2024-05-20T18:27:34.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.823+0000 7fd0cbfff640 1 -- 172.21.15.5:0/3579487692 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4030bd0 con 0x7fd0d8058340 2024-05-20T18:27:34.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.823+0000 7fd0e4ddb640 1 -- 172.21.15.5:0/3579487692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0d80d2480 con 0x7fd0d8058340 2024-05-20T18:27:34.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.824+0000 7fd0e4ddb640 1 -- 172.21.15.5:0/3579487692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0d8047b00 con 0x7fd0d8058340 2024-05-20T18:27:34.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.824+0000 7f447cff9640 1 -- 172.21.15.5:0/2382549770 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4474011070 con 0x7f4478033950 2024-05-20T18:27:34.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.825+0000 7f15b17fa640 1 -- 172.21.15.5:0/2579676819 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac00b680 con 0x7f15b40ccb10 2024-05-20T18:27:34.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.825+0000 7f94aeffd640 1 -- 172.21.15.5:0/1603290546 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f94b4058890 con 0x7f94b817e010 2024-05-20T18:27:34.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.826+0000 7f15b9bb9640 1 -- 172.21.15.5:0/2579676819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1594005290 con 0x7f15b40ccb10 2024-05-20T18:27:34.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.826+0000 7fd0c9ffb640 1 -- 172.21.15.5:0/3579487692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0d80058f0 con 0x7fd0d8058340 2024-05-20T18:27:34.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.826+0000 7f447cff9640 1 --2- 172.21.15.5:0/2382549770 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f446406bf00 0x7f446406e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.826+0000 7f447cff9640 1 -- 172.21.15.5:0/2382549770 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f447408b3d0 con 0x7f4478033950 2024-05-20T18:27:34.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.827+0000 7f7717fff640 1 -- 172.21.15.5:0/3127755484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f772c007bd0 con 0x7f772c0e5f60 2024-05-20T18:27:34.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.827+0000 7f447f7fe640 1 --2- 172.21.15.5:0/2382549770 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f446406bf00 0x7f446406e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.828+0000 7f447f7fe640 1 --2- 172.21.15.5:0/2382549770 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f446406bf00 0x7f446406e410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f446c034000 tx=0x7f446c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:34.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.833+0000 7f447cff9640 1 -- 172.21.15.5:0/2382549770 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4478038550 con 0x7f4478033950 2024-05-20T18:27:34.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.835+0000 7f772a7fc640 1 -- 172.21.15.5:0/3127755484 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f77340b8470 con 0x7f772c0e5f60 2024-05-20T18:27:34.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.835+0000 7f772a7fc640 1 --2- 172.21.15.5:0/3127755484 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f771806be60 0x7f771806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.835+0000 7f7739913640 1 --2- 172.21.15.5:0/3127755484 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f771806be60 0x7f771806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.836+0000 7f7739913640 1 --2- 172.21.15.5:0/3127755484 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f771806be60 0x7f771806e370 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7730001390 tx=0x7f7730001220 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:34.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.836+0000 7fd0cbfff640 1 -- 172.21.15.5:0/3579487692 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd0d4047460 con 0x7fd0d8058340 2024-05-20T18:27:34.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.837+0000 7fd0cbfff640 1 --2- 172.21.15.5:0/3579487692 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd0c406be80 0x7fd0c406e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.837+0000 7f772a7fc640 1 -- 172.21.15.5:0/3127755484 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f77340f3890 con 0x7f772c0e5f60 2024-05-20T18:27:34.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.837+0000 7fd0deffd640 1 --2- 172.21.15.5:0/3579487692 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd0c406be80 0x7fd0c406e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.838+0000 7fd0deffd640 1 --2- 172.21.15.5:0/3579487692 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd0c406be80 0x7fd0c406e390 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8000f40 tx=0x7fd0cc009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:34.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.838+0000 7f15b17fa640 1 -- 172.21.15.5:0/2579676819 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f15ac00b820 con 0x7f15b40ccb10 2024-05-20T18:27:34.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.838+0000 7fd0cbfff640 1 -- 172.21.15.5:0/3579487692 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd0d40787c0 con 0x7fd0d8058340 2024-05-20T18:27:34.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.839+0000 7f15b17fa640 1 --2- 172.21.15.5:0/2579676819 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f159806be30 0x7f159806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:34.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.839+0000 7f15b8bb7640 1 --2- 172.21.15.5:0/2579676819 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f159806be30 0x7f159806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:34.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.840+0000 7f15b17fa640 1 -- 172.21.15.5:0/2579676819 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f15ac08f520 con 0x7f15b40ccb10 2024-05-20T18:27:34.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.840+0000 7f15b8bb7640 1 --2- 172.21.15.5:0/2579676819 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f159806be30 0x7f159806e340 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f15a8000c00 tx=0x7f15a802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:34.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.847+0000 7f772a7fc640 1 -- 172.21.15.5:0/3127755484 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f772c007bd0 con 0x7f772c0e5f60 2024-05-20T18:27:34.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.857+0000 7fd0cbfff640 1 -- 172.21.15.5:0/3579487692 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd0d407a720 con 0x7fd0d8058340 2024-05-20T18:27:34.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:34.868+0000 7f15b17fa640 1 -- 172.21.15.5:0/2579676819 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f15ac05d4a0 con 0x7f15b40ccb10 2024-05-20T18:27:35.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.140+0000 7f15d2ee9640 1 -- 172.21.15.5:0/2953615906 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f15ac005580 con 0x7f15c40d8230 2024-05-20T18:27:35.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.140+0000 7f15c97fa640 1 -- 172.21.15.5:0/2953615906 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f15cc0c7880 con 0x7f15c40d8230 2024-05-20T18:27:35.140 INFO:teuthology.orchestra.run.smithi005.stdout:55834574858 2024-05-20T18:27:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.144+0000 7f15b6ffd640 1 -- 172.21.15.5:0/2953615906 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f15bc06c200 msgr2=0x7f15bc06e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.144+0000 7f15b6ffd640 1 --2- 172.21.15.5:0/2953615906 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f15bc06c200 0x7f15bc06e710 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f15c0006fd0 tx=0x7f15c0008040 comp rx=0 tx=0).stop 2024-05-20T18:27:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.144+0000 7f15b6ffd640 1 -- 172.21.15.5:0/2953615906 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15c40d8230 msgr2=0x7f15c40dcba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.144+0000 7f15b6ffd640 1 --2- 172.21.15.5:0/2953615906 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15c40d8230 0x7f15c40dcba0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f15cc050f00 tx=0x7f15cc074f20 comp rx=0 tx=0).stop 2024-05-20T18:27:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.145+0000 7f15b6ffd640 1 -- 172.21.15.5:0/2953615906 shutdown_connections 2024-05-20T18:27:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.145+0000 7f15b6ffd640 1 --2- 172.21.15.5:0/2953615906 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f15bc06c200 0x7f15bc06e710 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.145+0000 7f15b6ffd640 1 --2- 172.21.15.5:0/2953615906 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15c40d8230 0x7f15c40dcba0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.145+0000 7f15b6ffd640 1 --2- 172.21.15.5:0/2953615906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15c4055840 0x7f15c40dc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.145+0000 7f15b6ffd640 1 --2- 172.21.15.5:0/2953615906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15c4050670 0x7f15c4196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.145+0000 7f15b6ffd640 1 -- 172.21.15.5:0/2953615906 >> 172.21.15.5:0/2953615906 conn(0x7f15c4054d20 msgr2=0x7f15c4035f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.145+0000 7f15b6ffd640 1 -- 172.21.15.5:0/2953615906 shutdown_connections 2024-05-20T18:27:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.145+0000 7f15b6ffd640 1 -- 172.21.15.5:0/2953615906 wait complete. 2024-05-20T18:27:35.163 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.7 2024-05-20T18:27:35.164 DEBUG:teuthology.parallel:result is None 2024-05-20T18:27:35.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.180+0000 7ff0ecbd5640 1 -- 172.21.15.5:0/2120729887 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7ff0c0005580 con 0x7ff0e806c530 2024-05-20T18:27:35.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.180+0000 7ff0deffd640 1 -- 172.21.15.5:0/2120729887 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff0d005c8e0 con 0x7ff0e806c530 2024-05-20T18:27:35.180 INFO:teuthology.orchestra.run.smithi005.stdout:55834574858 2024-05-20T18:27:35.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.183+0000 7ff0ecbd5640 1 -- 172.21.15.5:0/2120729887 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff0cc06bf30 msgr2=0x7ff0cc06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:35.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.183+0000 7ff0ecbd5640 1 --2- 172.21.15.5:0/2120729887 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff0cc06bf30 0x7ff0cc06e440 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff0e806ca80 tx=0x7ff0d8009040 comp rx=0 tx=0).stop 2024-05-20T18:27:35.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.183+0000 7ff0ecbd5640 1 -- 172.21.15.5:0/2120729887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0e806c530 msgr2=0x7ff0e80ceed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:35.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.183+0000 7ff0ecbd5640 1 --2- 172.21.15.5:0/2120729887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0e806c530 0x7ff0e80ceed0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff0d000c400 tx=0x7ff0d000c8c0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.183+0000 7ff0ecbd5640 1 -- 172.21.15.5:0/2120729887 shutdown_connections 2024-05-20T18:27:35.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.183+0000 7ff0ecbd5640 1 --2- 172.21.15.5:0/2120729887 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff0cc06bf30 0x7ff0cc06e440 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.183+0000 7ff0ecbd5640 1 --2- 172.21.15.5:0/2120729887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0e80d0100 0x7ff0e8161320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.184+0000 7ff0ecbd5640 1 --2- 172.21.15.5:0/2120729887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0e8072d40 0x7ff0e8202250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.184+0000 7ff0ecbd5640 1 --2- 172.21.15.5:0/2120729887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0e806c530 0x7ff0e80ceed0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.184+0000 7ff0ecbd5640 1 -- 172.21.15.5:0/2120729887 >> 172.21.15.5:0/2120729887 conn(0x7ff0e8072220 msgr2=0x7ff0e8147b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:35.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.184+0000 7ff0ecbd5640 1 -- 172.21.15.5:0/2120729887 shutdown_connections 2024-05-20T18:27:35.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.184+0000 7ff0ecbd5640 1 -- 172.21.15.5:0/2120729887 wait complete. 2024-05-20T18:27:35.200 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.3 2024-05-20T18:27:35.200 DEBUG:teuthology.parallel:result is None 2024-05-20T18:27:35.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.214+0000 7fd0c9ffb640 1 -- 172.21.15.5:0/3579487692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fd0d803e830 con 0x7fd0d8058340 2024-05-20T18:27:35.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.214+0000 7fd0cbfff640 1 -- 172.21.15.5:0/3579487692 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd0d407ea20 con 0x7fd0d8058340 2024-05-20T18:27:35.214 INFO:teuthology.orchestra.run.smithi005.stdout:55834574856 2024-05-20T18:27:35.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.216+0000 7fd0c9ffb640 1 -- 172.21.15.5:0/3579487692 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd0c406be80 msgr2=0x7fd0c406e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:35.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.216+0000 7fd0c9ffb640 1 --2- 172.21.15.5:0/3579487692 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd0c406be80 0x7fd0c406e390 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8000f40 tx=0x7fd0cc009190 comp rx=0 tx=0).stop 2024-05-20T18:27:35.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.217+0000 7fd0c9ffb640 1 -- 172.21.15.5:0/3579487692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0d8058340 msgr2=0x7fd0d804a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:35.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.217+0000 7fd0c9ffb640 1 --2- 172.21.15.5:0/3579487692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0d8058340 0x7fd0d804a740 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4009960 tx=0x7fd0d40046c0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.217+0000 7fd0c9ffb640 1 -- 172.21.15.5:0/3579487692 shutdown_connections 2024-05-20T18:27:35.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.217+0000 7fd0c9ffb640 1 --2- 172.21.15.5:0/3579487692 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd0c406be80 0x7fd0c406e390 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.217+0000 7fd0c9ffb640 1 --2- 172.21.15.5:0/3579487692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0d80e73a0 0x7fd0d80e2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.217+0000 7fd0c9ffb640 1 --2- 172.21.15.5:0/3579487692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0d80e3db0 0x7fd0d80d42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.217+0000 7fd0c9ffb640 1 --2- 172.21.15.5:0/3579487692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0d8058340 0x7fd0d804a740 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.217+0000 7fd0c9ffb640 1 -- 172.21.15.5:0/3579487692 >> 172.21.15.5:0/3579487692 conn(0x7fd0d803ac60 msgr2=0x7fd0d8005d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:35.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.217+0000 7fd0c9ffb640 1 -- 172.21.15.5:0/3579487692 shutdown_connections 2024-05-20T18:27:35.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.217+0000 7fd0c9ffb640 1 -- 172.21.15.5:0/3579487692 wait complete. 2024-05-20T18:27:35.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.221+0000 7f94bf59e640 1 -- 172.21.15.5:0/1603290546 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f94a4005580 con 0x7f94b817e010 2024-05-20T18:27:35.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.222+0000 7f94aeffd640 1 -- 172.21.15.5:0/1603290546 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f94b4058190 con 0x7f94b817e010 2024-05-20T18:27:35.221 INFO:teuthology.orchestra.run.smithi005.stdout:55834574857 2024-05-20T18:27:35.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.224+0000 7f94bf59e640 1 -- 172.21.15.5:0/1603290546 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f94a006be80 msgr2=0x7f94a006e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:35.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.224+0000 7f94bf59e640 1 --2- 172.21.15.5:0/1603290546 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f94a006be80 0x7f94a006e390 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f94c0050c70 tx=0x7f94c0061f90 comp rx=0 tx=0).stop 2024-05-20T18:27:35.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.224+0000 7f94bf59e640 1 -- 172.21.15.5:0/1603290546 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f94b817e010 msgr2=0x7f94b816fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:35.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.224+0000 7f94bf59e640 1 --2- 172.21.15.5:0/1603290546 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f94b817e010 0x7f94b816fe60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f94b800bf20 tx=0x7f94b400c820 comp rx=0 tx=0).stop 2024-05-20T18:27:35.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.224+0000 7f94bf59e640 1 -- 172.21.15.5:0/1603290546 shutdown_connections 2024-05-20T18:27:35.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.224+0000 7f94bf59e640 1 --2- 172.21.15.5:0/1603290546 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f94a006be80 0x7f94a006e390 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.224+0000 7f94bf59e640 1 --2- 172.21.15.5:0/1603290546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94b81703a0 0x7f94b80e36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.224+0000 7f94bf59e640 1 --2- 172.21.15.5:0/1603290546 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f94b817e010 0x7f94b816fe60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.224+0000 7f94bf59e640 1 --2- 172.21.15.5:0/1603290546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94b8007320 0x7f94b80d8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.224+0000 7f94bf59e640 1 -- 172.21.15.5:0/1603290546 >> 172.21.15.5:0/1603290546 conn(0x7f94b8054cf0 msgr2=0x7f94b8039860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:35.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.224+0000 7f94bf59e640 1 -- 172.21.15.5:0/1603290546 shutdown_connections 2024-05-20T18:27:35.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.224+0000 7f94bf59e640 1 -- 172.21.15.5:0/1603290546 wait complete. 2024-05-20T18:27:35.231 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2024-05-20T18:27:35.231 DEBUG:teuthology.parallel:result is None 2024-05-20T18:27:35.237 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.2 2024-05-20T18:27:35.237 DEBUG:teuthology.parallel:result is None 2024-05-20T18:27:35.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.254+0000 7fa5ce921640 1 -- 172.21.15.5:0/111229794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fa5c0000f20 con 0x7fa5c00544f0 2024-05-20T18:27:35.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.255+0000 7fa5c4ff9640 1 -- 172.21.15.5:0/111229794 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa5bc07a970 con 0x7fa5c00544f0 2024-05-20T18:27:35.254 INFO:teuthology.orchestra.run.smithi005.stdout:55834574858 2024-05-20T18:27:35.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.258+0000 7fa5ce921640 1 -- 172.21.15.5:0/111229794 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa5ac06be30 msgr2=0x7fa5ac06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:35.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.258+0000 7fa5ce921640 1 --2- 172.21.15.5:0/111229794 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa5ac06be30 0x7fa5ac06e340 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0049d20 tx=0x7fa5b4013040 comp rx=0 tx=0).stop 2024-05-20T18:27:35.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.258+0000 7fa5ce921640 1 -- 172.21.15.5:0/111229794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5c00544f0 msgr2=0x7fa5c00d2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:35.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.258+0000 7fa5ce921640 1 --2- 172.21.15.5:0/111229794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5c00544f0 0x7fa5c00d2350 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00ec80 tx=0x7fa5bc0046c0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.258+0000 7fa5ce921640 1 -- 172.21.15.5:0/111229794 shutdown_connections 2024-05-20T18:27:35.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.258+0000 7fa5ce921640 1 --2- 172.21.15.5:0/111229794 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa5ac06be30 0x7fa5ac06e340 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.258+0000 7fa5ce921640 1 --2- 172.21.15.5:0/111229794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5c00e33a0 0x7fa5c00da590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.258+0000 7fa5ce921640 1 --2- 172.21.15.5:0/111229794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5c00dfdb0 0x7fa5c004ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.258+0000 7fa5ce921640 1 --2- 172.21.15.5:0/111229794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5c00544f0 0x7fa5c00d2350 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.258+0000 7fa5ce921640 1 -- 172.21.15.5:0/111229794 >> 172.21.15.5:0/111229794 conn(0x7fa5c0054d20 msgr2=0x7fa5c00deb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:35.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.258+0000 7fa5ce921640 1 -- 172.21.15.5:0/111229794 shutdown_connections 2024-05-20T18:27:35.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.258+0000 7fa5ce921640 1 -- 172.21.15.5:0/111229794 wait complete. 2024-05-20T18:27:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.266+0000 7f446a7fc640 1 -- 172.21.15.5:0/2382549770 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f44780054a0 con 0x7f4478033950 2024-05-20T18:27:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.267+0000 7f447cff9640 1 -- 172.21.15.5:0/2382549770 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4474059990 con 0x7f4478033950 2024-05-20T18:27:35.266 INFO:teuthology.orchestra.run.smithi005.stdout:55834574856 2024-05-20T18:27:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.269+0000 7f446a7fc640 1 -- 172.21.15.5:0/2382549770 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f446406bf00 msgr2=0x7f446406e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.269+0000 7f446a7fc640 1 --2- 172.21.15.5:0/2382549770 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f446406bf00 0x7f446406e410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f446c034000 tx=0x7f446c02e000 comp rx=0 tx=0).stop 2024-05-20T18:27:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.269+0000 7f446a7fc640 1 -- 172.21.15.5:0/2382549770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4478033950 msgr2=0x7f44780e9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.269+0000 7f446a7fc640 1 --2- 172.21.15.5:0/2382549770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4478033950 0x7f44780e9410 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f44780516d0 tx=0x7f447400c870 comp rx=0 tx=0).stop 2024-05-20T18:27:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.269+0000 7f446a7fc640 1 -- 172.21.15.5:0/2382549770 shutdown_connections 2024-05-20T18:27:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.269+0000 7f446a7fc640 1 --2- 172.21.15.5:0/2382549770 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f446406bf00 0x7f446406e410 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.269+0000 7f446a7fc640 1 --2- 172.21.15.5:0/2382549770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f447804f340 0x7f44780e5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.269+0000 7f446a7fc640 1 --2- 172.21.15.5:0/2382549770 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4478048320 0x7f447817de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.269+0000 7f446a7fc640 1 --2- 172.21.15.5:0/2382549770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4478033950 0x7f44780e9410 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.269+0000 7f446a7fc640 1 -- 172.21.15.5:0/2382549770 >> 172.21.15.5:0/2382549770 conn(0x7f4478054cf0 msgr2=0x7f44780367c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.269+0000 7f446a7fc640 1 -- 172.21.15.5:0/2382549770 shutdown_connections 2024-05-20T18:27:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.269+0000 7f446a7fc640 1 -- 172.21.15.5:0/2382549770 wait complete. 2024-05-20T18:27:35.270 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.1 2024-05-20T18:27:35.270 DEBUG:teuthology.parallel:result is None 2024-05-20T18:27:35.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.275+0000 7f7717fff640 1 -- 172.21.15.5:0/3127755484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f772c03ecc0 con 0x7f772c0e5f60 2024-05-20T18:27:35.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.275+0000 7f772a7fc640 1 -- 172.21.15.5:0/3127755484 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f772c0e68b0 con 0x7f772c0e5f60 2024-05-20T18:27:35.275 INFO:teuthology.orchestra.run.smithi005.stdout:55834574856 2024-05-20T18:27:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.277+0000 7f15b9bb9640 1 -- 172.21.15.5:0/2579676819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f1594005850 con 0x7f15b40ccb10 2024-05-20T18:27:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.277+0000 7f7717fff640 1 -- 172.21.15.5:0/3127755484 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f771806be60 msgr2=0x7f771806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.277+0000 7f7717fff640 1 --2- 172.21.15.5:0/3127755484 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f771806be60 0x7f771806e370 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7730001390 tx=0x7f7730001220 comp rx=0 tx=0).stop 2024-05-20T18:27:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.278+0000 7f7717fff640 1 -- 172.21.15.5:0/3127755484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f772c0e5f60 msgr2=0x7f772c0e6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.278+0000 7f7717fff640 1 --2- 172.21.15.5:0/3127755484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f772c0e5f60 0x7f772c0e6370 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7734069ea0 tx=0x7f77340696a0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.278+0000 7f7717fff640 1 -- 172.21.15.5:0/3127755484 shutdown_connections 2024-05-20T18:27:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.278+0000 7f7717fff640 1 --2- 172.21.15.5:0/3127755484 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f771806be60 0x7f771806e370 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.278+0000 7f7717fff640 1 --2- 172.21.15.5:0/3127755484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f772c0e5f60 0x7f772c0e6370 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.278+0000 7f7717fff640 1 --2- 172.21.15.5:0/3127755484 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f772c03e200 0x7f772c03e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.278+0000 7f7717fff640 1 --2- 172.21.15.5:0/3127755484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7720005350 0x7f772c0d5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.278+0000 7f7717fff640 1 -- 172.21.15.5:0/3127755484 >> 172.21.15.5:0/3127755484 conn(0x7f772c059140 msgr2=0x7f772c0dd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:35.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.278+0000 7f15b17fa640 1 -- 172.21.15.5:0/2579676819 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f15ac0617a0 con 0x7f15b40ccb10 2024-05-20T18:27:35.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.278+0000 7f7717fff640 1 -- 172.21.15.5:0/3127755484 shutdown_connections 2024-05-20T18:27:35.279 INFO:teuthology.orchestra.run.smithi005.stdout:55834574857 2024-05-20T18:27:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.278+0000 7f7717fff640 1 -- 172.21.15.5:0/3127755484 wait complete. 2024-05-20T18:27:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.280+0000 7f15b9bb9640 1 -- 172.21.15.5:0/2579676819 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f159806be30 msgr2=0x7f159806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.280+0000 7f15b9bb9640 1 --2- 172.21.15.5:0/2579676819 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f159806be30 0x7f159806e340 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f15a8000c00 tx=0x7f15a802e000 comp rx=0 tx=0).stop 2024-05-20T18:27:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.280+0000 7f15b9bb9640 1 -- 172.21.15.5:0/2579676819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15b40ccb10 msgr2=0x7f15b406e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.280+0000 7f15b9bb9640 1 --2- 172.21.15.5:0/2579676819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15b40ccb10 0x7f15b406e640 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00ff70 tx=0x7f15ac00de40 comp rx=0 tx=0).stop 2024-05-20T18:27:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.281+0000 7f15b9bb9640 1 -- 172.21.15.5:0/2579676819 shutdown_connections 2024-05-20T18:27:35.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.281+0000 7f15b9bb9640 1 --2- 172.21.15.5:0/2579676819 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f159806be30 0x7f159806e340 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.281+0000 7f15b9bb9640 1 --2- 172.21.15.5:0/2579676819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15b40ccb10 0x7f15b406e640 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.281+0000 7f15b9bb9640 1 --2- 172.21.15.5:0/2579676819 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15b4072cf0 0x7f15b406e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.281+0000 7f15b9bb9640 1 --2- 172.21.15.5:0/2579676819 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15b406ca70 0x7f15b413ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.281+0000 7f15b9bb9640 1 -- 172.21.15.5:0/2579676819 >> 172.21.15.5:0/2579676819 conn(0x7f15b40721d0 msgr2=0x7f15b41520d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:35.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.281+0000 7f15b9bb9640 1 -- 172.21.15.5:0/2579676819 shutdown_connections 2024-05-20T18:27:35.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.281+0000 7f15b9bb9640 1 -- 172.21.15.5:0/2579676819 wait complete. 2024-05-20T18:27:35.282 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.4 2024-05-20T18:27:35.282 DEBUG:teuthology.parallel:result is None 2024-05-20T18:27:35.293 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.6 2024-05-20T18:27:35.293 DEBUG:teuthology.parallel:result is None 2024-05-20T18:27:35.298 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.0 2024-05-20T18:27:35.299 DEBUG:teuthology.parallel:result is None 2024-05-20T18:27:35.299 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-20T18:27:35.299 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T18:27:35.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.455+0000 7f6eea422640 1 Processor -- start 2024-05-20T18:27:35.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.455+0000 7f6eea422640 1 -- start start 2024-05-20T18:27:35.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.456+0000 7f6eea422640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ee40d10e0 0x7f6ee40cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:35.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.456+0000 7f6eea422640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6ee40b7210 0x7f6ee40cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:35.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.456+0000 7f6eea422640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ee40d1ac0 con 0x7f6ee40d14f0 2024-05-20T18:27:35.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.456+0000 7f6eea422640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f6ee40d1da0 con 0x7f6ee40b7210 2024-05-20T18:27:35.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.456+0000 7f6eea422640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6ee40d2080 con 0x7f6ee40d10e0 2024-05-20T18:27:35.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.456+0000 7f6ee89bf640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6ee40b7210 0x7f6ee40cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:35.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.456+0000 7f6ee89bf640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6ee40b7210 0x7f6ee40cf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52896/0 (socket says 172.21.15.5:52896) 2024-05-20T18:27:35.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.456+0000 7f6edbfff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f6ee40d14f0 0x7f6ee40cd4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:40998/0 (socket says 172.21.15.5:40998) 2024-05-20T18:27:35.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.456+0000 7f6ee89bf640 1 -- 172.21.15.5:0/128459127 learned_addr learned my addr 172.21.15.5:0/128459127 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:35.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.456+0000 7f6ee3fff640 1 --2- 172.21.15.5:0/128459127 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ee40d10e0 0x7f6ee40cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:35.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.457+0000 7f6ee89bf640 1 -- 172.21.15.5:0/128459127 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ee40d10e0 msgr2=0x7f6ee40cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:35.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.457+0000 7f6ee89bf640 1 --2- 172.21.15.5:0/128459127 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ee40d10e0 0x7f6ee40cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.457+0000 7f6ee89bf640 1 -- 172.21.15.5:0/128459127 >> v1:172.21.15.5:6789/0 conn(0x7f6ee40d14f0 legacy=0x7f6ee40cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:35.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.457+0000 7f6ee89bf640 1 -- 172.21.15.5:0/128459127 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee40d2360 con 0x7f6ee40b7210 2024-05-20T18:27:35.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.457+0000 7f6ee37fe640 1 -- 172.21.15.5:0/128459127 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3202877380 0 0) 0x7f6ee40d1ac0 con 0x7f6ee40d14f0 2024-05-20T18:27:35.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.457+0000 7f6ee89bf640 1 --2- 172.21.15.5:0/128459127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ee40b7210 0x7f6ee40cf680 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6edc009e50 tx=0x7f6edc02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ea104da60262c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:35.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.458+0000 7f6ee37fe640 1 -- 172.21.15.5:0/128459127 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc036070 con 0x7f6ee40b7210 2024-05-20T18:27:35.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.458+0000 7f6ee37fe640 1 -- 172.21.15.5:0/128459127 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6edc031600 con 0x7f6ee40b7210 2024-05-20T18:27:35.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.458+0000 7f6ee37fe640 1 -- 172.21.15.5:0/128459127 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc0319d0 con 0x7f6ee40b7210 2024-05-20T18:27:35.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.458+0000 7f6eea422640 1 -- 172.21.15.5:0/128459127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ee40b7210 msgr2=0x7f6ee40cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:35.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.458+0000 7f6eea422640 1 --2- 172.21.15.5:0/128459127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ee40b7210 0x7f6ee40cf680 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6edc009e50 tx=0x7f6edc02a730 comp rx=0 tx=0).stop 2024-05-20T18:27:35.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.458+0000 7f6eea422640 1 -- 172.21.15.5:0/128459127 shutdown_connections 2024-05-20T18:27:35.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.458+0000 7f6eea422640 1 --2- 172.21.15.5:0/128459127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ee40b7210 0x7f6ee40cf680 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.458+0000 7f6eea422640 1 --2- 172.21.15.5:0/128459127 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ee40d10e0 0x7f6ee40cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.458+0000 7f6eea422640 1 -- 172.21.15.5:0/128459127 >> 172.21.15.5:0/128459127 conn(0x7f6ee40b44d0 msgr2=0x7f6ee40b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:35.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.458+0000 7f6eea422640 1 -- 172.21.15.5:0/128459127 shutdown_connections 2024-05-20T18:27:35.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.458+0000 7f6eea422640 1 -- 172.21.15.5:0/128459127 wait complete. 2024-05-20T18:27:35.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.460+0000 7f6eea422640 1 Processor -- start 2024-05-20T18:27:35.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.460+0000 7f6eea422640 1 -- start start 2024-05-20T18:27:35.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.461+0000 7f6eea422640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ee40b7210 0x7f6ee4152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:35.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.461+0000 7f6eea422640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ee40d10e0 0x7f6ee4155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:35.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.461+0000 7f6eea422640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ee40d14f0 0x7f6ee41561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:35.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.461+0000 7f6eea422640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee40b8440 con 0x7f6ee40d10e0 2024-05-20T18:27:35.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.462+0000 7f6eea422640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee40b82c0 con 0x7f6ee40d14f0 2024-05-20T18:27:35.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.462+0000 7f6eea422640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6ee40709f0 con 0x7f6ee40b7210 2024-05-20T18:27:35.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.462+0000 7f6edbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ee40d10e0 0x7f6ee4155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:35.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.462+0000 7f6edbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ee40d10e0 0x7f6ee4155c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52906/0 (socket says 172.21.15.5:52906) 2024-05-20T18:27:35.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.462+0000 7f6edbfff640 1 -- 172.21.15.5:0/2770516702 learned_addr learned my addr 172.21.15.5:0/2770516702 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:35.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.462+0000 7f6ee3fff640 1 --2- 172.21.15.5:0/2770516702 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ee40b7210 0x7f6ee4152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:35.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.462+0000 7f6ee89bf640 1 --2- 172.21.15.5:0/2770516702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ee40d14f0 0x7f6ee41561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:35.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.462+0000 7f6edbfff640 1 -- 172.21.15.5:0/2770516702 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ee40b7210 msgr2=0x7f6ee4152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:35.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.462+0000 7f6edbfff640 1 --2- 172.21.15.5:0/2770516702 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ee40b7210 0x7f6ee4152ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.462+0000 7f6edbfff640 1 -- 172.21.15.5:0/2770516702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ee40d14f0 msgr2=0x7f6ee41561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:35.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.462+0000 7f6edbfff640 1 --2- 172.21.15.5:0/2770516702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ee40d14f0 0x7f6ee41561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.462+0000 7f6edbfff640 1 -- 172.21.15.5:0/2770516702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee41552c0 con 0x7f6ee40d10e0 2024-05-20T18:27:35.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.462+0000 7f6edbfff640 1 --2- 172.21.15.5:0/2770516702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ee40d10e0 0x7f6ee4155c60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00c390 tx=0x7f6ecc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:35.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.462+0000 7f6ee17fa640 1 -- 172.21.15.5:0/2770516702 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc0074e0 con 0x7f6ee40d10e0 2024-05-20T18:27:35.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.462+0000 7f6ee17fa640 1 -- 172.21.15.5:0/2770516702 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ecc007680 con 0x7f6ee40d10e0 2024-05-20T18:27:35.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.462+0000 7f6eea422640 1 -- 172.21.15.5:0/2770516702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ee4154220 con 0x7f6ee40d10e0 2024-05-20T18:27:35.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.462+0000 7f6ee17fa640 1 -- 172.21.15.5:0/2770516702 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc004a00 con 0x7f6ee40d10e0 2024-05-20T18:27:35.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.462+0000 7f6eea422640 1 -- 172.21.15.5:0/2770516702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ee41547e0 con 0x7f6ee40d10e0 2024-05-20T18:27:35.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.463+0000 7f6eea422640 1 -- 172.21.15.5:0/2770516702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ee4075b00 con 0x7f6ee40d10e0 2024-05-20T18:27:35.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.464+0000 7f6ee17fa640 1 -- 172.21.15.5:0/2770516702 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6ecc011070 con 0x7f6ee40d10e0 2024-05-20T18:27:35.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.464+0000 7f6ee17fa640 1 --2- 172.21.15.5:0/2770516702 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6ec406be30 0x7f6ec406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:35.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.464+0000 7f6ee17fa640 1 -- 172.21.15.5:0/2770516702 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f6ecc08aca0 con 0x7f6ee40d10e0 2024-05-20T18:27:35.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.464+0000 7f6ee3fff640 1 --2- 172.21.15.5:0/2770516702 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6ec406be30 0x7f6ec406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:35.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.465+0000 7f6ee3fff640 1 --2- 172.21.15.5:0/2770516702 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6ec406be30 0x7f6ec406e340 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4009aa0 tx=0x7f6ed4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:35.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.469+0000 7f6ee17fa640 1 -- 172.21.15.5:0/2770516702 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6ecc058c30 con 0x7f6ee40d10e0 2024-05-20T18:27:35.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.649+0000 7f6eea422640 1 -- 172.21.15.5:0/2770516702 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6ee41531e0 con 0x7f6ec406be30 2024-05-20T18:27:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.658+0000 7f6ee17fa640 1 -- 172.21.15.5:0/2770516702 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332533 (secure 0 0 0) 0x7f6ee41531e0 con 0x7f6ec406be30 2024-05-20T18:27:35.660 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:35.662 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-20T18:27:35.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.661+0000 7f6eea422640 1 -- 172.21.15.5:0/2770516702 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6ec406be30 msgr2=0x7f6ec406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:35.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.661+0000 7f6eea422640 1 --2- 172.21.15.5:0/2770516702 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6ec406be30 0x7f6ec406e340 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4009aa0 tx=0x7f6ed4009190 comp rx=0 tx=0).stop 2024-05-20T18:27:35.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.661+0000 7f6eea422640 1 -- 172.21.15.5:0/2770516702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ee40d10e0 msgr2=0x7f6ee4155c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:35.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.661+0000 7f6eea422640 1 --2- 172.21.15.5:0/2770516702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ee40d10e0 0x7f6ee4155c60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00c390 tx=0x7f6ecc00c850 comp rx=0 tx=0).stop 2024-05-20T18:27:35.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.661+0000 7f6eea422640 1 -- 172.21.15.5:0/2770516702 shutdown_connections 2024-05-20T18:27:35.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.661+0000 7f6eea422640 1 --2- 172.21.15.5:0/2770516702 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6ec406be30 0x7f6ec406e340 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.661+0000 7f6eea422640 1 --2- 172.21.15.5:0/2770516702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ee40d14f0 0x7f6ee41561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.661+0000 7f6eea422640 1 --2- 172.21.15.5:0/2770516702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ee40d10e0 0x7f6ee4155c60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.661+0000 7f6eea422640 1 --2- 172.21.15.5:0/2770516702 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ee40b7210 0x7f6ee4152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.661+0000 7f6eea422640 1 -- 172.21.15.5:0/2770516702 >> 172.21.15.5:0/2770516702 conn(0x7f6ee40b44d0 msgr2=0x7f6ee415c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:35.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.661+0000 7f6eea422640 1 -- 172.21.15.5:0/2770516702 shutdown_connections 2024-05-20T18:27:35.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.661+0000 7f6eea422640 1 -- 172.21.15.5:0/2770516702 wait complete. 2024-05-20T18:27:35.672 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-05-20T18:27:34.580839+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236216,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738504,"statfs":{"total":773094113280,"available":772852228096,"internally_reserved":0,"allocated":3481600,"data_stored":2086528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004641"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048716+0000","last_change":"2024-05-20T18:27:06.794819+0000","last_active":"2024-05-20T18:27:14.048716+0000","last_peered":"2024-05-20T18:27:14.048716+0000","last_clean":"2024-05-20T18:27:14.048716+0000","last_became_active":"2024-05-20T18:27:06.794505+0000","last_became_peered":"2024-05-20T18:27:06.794505+0000","last_unstale":"2024-05-20T18:27:14.048716+0000","last_undegraded":"2024-05-20T18:27:14.048716+0000","last_fullsized":"2024-05-20T18:27:14.048716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:20:52.095853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821927+0000","last_change":"2024-05-20T18:27:07.802838+0000","last_active":"2024-05-20T18:27:13.821927+0000","last_peered":"2024-05-20T18:27:13.821927+0000","last_clean":"2024-05-20T18:27:13.821927+0000","last_became_active":"2024-05-20T18:27:07.802468+0000","last_became_peered":"2024-05-20T18:27:07.802468+0000","last_unstale":"2024-05-20T18:27:13.821927+0000","last_undegraded":"2024-05-20T18:27:13.821927+0000","last_fullsized":"2024-05-20T18:27:13.821927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:12.006827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821103+0000","last_change":"2024-05-20T18:27:12.469613+0000","last_active":"2024-05-20T18:27:13.821103+0000","last_peered":"2024-05-20T18:27:13.821103+0000","last_clean":"2024-05-20T18:27:13.821103+0000","last_became_active":"2024-05-20T18:27:12.469161+0000","last_became_peered":"2024-05-20T18:27:12.469161+0000","last_unstale":"2024-05-20T18:27:13.821103+0000","last_undegraded":"2024-05-20T18:27:13.821103+0000","last_fullsized":"2024-05-20T18:27:13.821103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:32:50.234917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.822083+0000","last_change":"2024-05-20T18:27:06.799063+0000","last_active":"2024-05-20T18:27:13.822083+0000","last_peered":"2024-05-20T18:27:13.822083+0000","last_clean":"2024-05-20T18:27:13.822083+0000","last_became_active":"2024-05-20T18:27:06.798689+0000","last_became_peered":"2024-05-20T18:27:06.798689+0000","last_unstale":"2024-05-20T18:27:13.822083+0000","last_undegraded":"2024-05-20T18:27:13.822083+0000","last_fullsized":"2024-05-20T18:27:13.822083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:52:44.943426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048397+0000","last_change":"2024-05-20T18:27:07.805002+0000","last_active":"2024-05-20T18:27:14.048397+0000","last_peered":"2024-05-20T18:27:14.048397+0000","last_clean":"2024-05-20T18:27:14.048397+0000","last_became_active":"2024-05-20T18:27:07.804655+0000","last_became_peered":"2024-05-20T18:27:07.804655+0000","last_unstale":"2024-05-20T18:27:14.048397+0000","last_undegraded":"2024-05-20T18:27:14.048397+0000","last_fullsized":"2024-05-20T18:27:14.048397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:52:57.891709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047788+0000","last_change":"2024-05-20T18:27:11.833803+0000","last_active":"2024-05-20T18:27:14.047788+0000","last_peered":"2024-05-20T18:27:14.047788+0000","last_clean":"2024-05-20T18:27:14.047788+0000","last_became_active":"2024-05-20T18:27:11.833241+0000","last_became_peered":"2024-05-20T18:27:11.833241+0000","last_unstale":"2024-05-20T18:27:14.047788+0000","last_undegraded":"2024-05-20T18:27:14.047788+0000","last_fullsized":"2024-05-20T18:27:14.047788+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:25:03.246597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049511+0000","last_change":"2024-05-20T18:27:06.806008+0000","last_active":"2024-05-20T18:27:14.049511+0000","last_peered":"2024-05-20T18:27:14.049511+0000","last_clean":"2024-05-20T18:27:14.049511+0000","last_became_active":"2024-05-20T18:27:06.805632+0000","last_became_peered":"2024-05-20T18:27:06.805632+0000","last_unstale":"2024-05-20T18:27:14.049511+0000","last_undegraded":"2024-05-20T18:27:14.049511+0000","last_fullsized":"2024-05-20T18:27:14.049511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:11:07.831277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825124+0000","last_change":"2024-05-20T18:27:07.812390+0000","last_active":"2024-05-20T18:27:13.825124+0000","last_peered":"2024-05-20T18:27:13.825124+0000","last_clean":"2024-05-20T18:27:13.825124+0000","last_became_active":"2024-05-20T18:27:07.811667+0000","last_became_peered":"2024-05-20T18:27:07.811667+0000","last_unstale":"2024-05-20T18:27:13.825124+0000","last_undegraded":"2024-05-20T18:27:13.825124+0000","last_fullsized":"2024-05-20T18:27:13.825124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:29:33.478403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048201+0000","last_change":"2024-05-20T18:27:11.833494+0000","last_active":"2024-05-20T18:27:14.048201+0000","last_peered":"2024-05-20T18:27:14.048201+0000","last_clean":"2024-05-20T18:27:14.048201+0000","last_became_active":"2024-05-20T18:27:11.832999+0000","last_became_peered":"2024-05-20T18:27:11.832999+0000","last_unstale":"2024-05-20T18:27:14.048201+0000","last_undegraded":"2024-05-20T18:27:14.048201+0000","last_fullsized":"2024-05-20T18:27:14.048201+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:08:42.465927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049057+0000","last_change":"2024-05-20T18:27:06.798023+0000","last_active":"2024-05-20T18:27:14.049057+0000","last_peered":"2024-05-20T18:27:14.049057+0000","last_clean":"2024-05-20T18:27:14.049057+0000","last_became_active":"2024-05-20T18:27:06.797674+0000","last_became_peered":"2024-05-20T18:27:06.797674+0000","last_unstale":"2024-05-20T18:27:14.049057+0000","last_undegraded":"2024-05-20T18:27:14.049057+0000","last_fullsized":"2024-05-20T18:27:14.049057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:43:31.154358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824776+0000","last_change":"2024-05-20T18:27:07.811508+0000","last_active":"2024-05-20T18:27:13.824776+0000","last_peered":"2024-05-20T18:27:13.824776+0000","last_clean":"2024-05-20T18:27:13.824776+0000","last_became_active":"2024-05-20T18:27:07.811217+0000","last_became_peered":"2024-05-20T18:27:07.811217+0000","last_unstale":"2024-05-20T18:27:13.824776+0000","last_undegraded":"2024-05-20T18:27:13.824776+0000","last_fullsized":"2024-05-20T18:27:13.824776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:01:46.623171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047874+0000","last_change":"2024-05-20T18:27:11.827280+0000","last_active":"2024-05-20T18:27:14.047874+0000","last_peered":"2024-05-20T18:27:14.047874+0000","last_clean":"2024-05-20T18:27:14.047874+0000","last_became_active":"2024-05-20T18:27:11.826770+0000","last_became_peered":"2024-05-20T18:27:11.826770+0000","last_unstale":"2024-05-20T18:27:14.047874+0000","last_undegraded":"2024-05-20T18:27:14.047874+0000","last_fullsized":"2024-05-20T18:27:14.047874+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:16:41.968388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825487+0000","last_change":"2024-05-20T18:27:07.791795+0000","last_active":"2024-05-20T18:27:13.825487+0000","last_peered":"2024-05-20T18:27:13.825487+0000","last_clean":"2024-05-20T18:27:13.825487+0000","last_became_active":"2024-05-20T18:27:07.791422+0000","last_became_peered":"2024-05-20T18:27:07.791422+0000","last_unstale":"2024-05-20T18:27:13.825487+0000","last_undegraded":"2024-05-20T18:27:13.825487+0000","last_fullsized":"2024-05-20T18:27:13.825487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:40:08.698691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049335+0000","last_change":"2024-05-20T18:27:06.799475+0000","last_active":"2024-05-20T18:27:14.049335+0000","last_peered":"2024-05-20T18:27:14.049335+0000","last_clean":"2024-05-20T18:27:14.049335+0000","last_became_active":"2024-05-20T18:27:06.798540+0000","last_became_peered":"2024-05-20T18:27:06.798540+0000","last_unstale":"2024-05-20T18:27:14.049335+0000","last_undegraded":"2024-05-20T18:27:14.049335+0000","last_fullsized":"2024-05-20T18:27:14.049335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:14:22.040824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048392+0000","last_change":"2024-05-20T18:27:11.826856+0000","last_active":"2024-05-20T18:27:14.048392+0000","last_peered":"2024-05-20T18:27:14.048392+0000","last_clean":"2024-05-20T18:27:14.048392+0000","last_became_active":"2024-05-20T18:27:11.826205+0000","last_became_peered":"2024-05-20T18:27:11.826205+0000","last_unstale":"2024-05-20T18:27:14.048392+0000","last_undegraded":"2024-05-20T18:27:14.048392+0000","last_fullsized":"2024-05-20T18:27:14.048392+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:34:21.648495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821544+0000","last_change":"2024-05-20T18:27:07.811973+0000","last_active":"2024-05-20T18:27:13.821544+0000","last_peered":"2024-05-20T18:27:13.821544+0000","last_clean":"2024-05-20T18:27:13.821544+0000","last_became_active":"2024-05-20T18:27:07.811617+0000","last_became_peered":"2024-05-20T18:27:07.811617+0000","last_unstale":"2024-05-20T18:27:13.821544+0000","last_undegraded":"2024-05-20T18:27:13.821544+0000","last_fullsized":"2024-05-20T18:27:13.821544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:52:22.903884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048965+0000","last_change":"2024-05-20T18:27:06.792524+0000","last_active":"2024-05-20T18:27:14.048965+0000","last_peered":"2024-05-20T18:27:14.048965+0000","last_clean":"2024-05-20T18:27:14.048965+0000","last_became_active":"2024-05-20T18:27:06.792104+0000","last_became_peered":"2024-05-20T18:27:06.792104+0000","last_unstale":"2024-05-20T18:27:14.048965+0000","last_undegraded":"2024-05-20T18:27:14.048965+0000","last_fullsized":"2024-05-20T18:27:14.048965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:10:27.276400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048408+0000","last_change":"2024-05-20T18:27:11.833928+0000","last_active":"2024-05-20T18:27:14.048408+0000","last_peered":"2024-05-20T18:27:14.048408+0000","last_clean":"2024-05-20T18:27:14.048408+0000","last_became_active":"2024-05-20T18:27:11.832887+0000","last_became_peered":"2024-05-20T18:27:11.832887+0000","last_unstale":"2024-05-20T18:27:14.048408+0000","last_undegraded":"2024-05-20T18:27:14.048408+0000","last_fullsized":"2024-05-20T18:27:14.048408+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:38:58.503399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821356+0000","last_change":"2024-05-20T18:27:07.788711+0000","last_active":"2024-05-20T18:27:13.821356+0000","last_peered":"2024-05-20T18:27:13.821356+0000","last_clean":"2024-05-20T18:27:13.821356+0000","last_became_active":"2024-05-20T18:27:07.788402+0000","last_became_peered":"2024-05-20T18:27:07.788402+0000","last_unstale":"2024-05-20T18:27:13.821356+0000","last_undegraded":"2024-05-20T18:27:13.821356+0000","last_fullsized":"2024-05-20T18:27:13.821356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:34:15.614124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048633+0000","last_change":"2024-05-20T18:27:06.803611+0000","last_active":"2024-05-20T18:27:14.048633+0000","last_peered":"2024-05-20T18:27:14.048633+0000","last_clean":"2024-05-20T18:27:14.048633+0000","last_became_active":"2024-05-20T18:27:06.803176+0000","last_became_peered":"2024-05-20T18:27:06.803176+0000","last_unstale":"2024-05-20T18:27:14.048633+0000","last_undegraded":"2024-05-20T18:27:14.048633+0000","last_fullsized":"2024-05-20T18:27:14.048633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:52:44.943426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049451+0000","last_change":"2024-05-20T18:27:12.466975+0000","last_active":"2024-05-20T18:27:14.049451+0000","last_peered":"2024-05-20T18:27:14.049451+0000","last_clean":"2024-05-20T18:27:14.049451+0000","last_became_active":"2024-05-20T18:27:12.466473+0000","last_became_peered":"2024-05-20T18:27:12.466473+0000","last_unstale":"2024-05-20T18:27:14.049451+0000","last_undegraded":"2024-05-20T18:27:14.049451+0000","last_fullsized":"2024-05-20T18:27:14.049451+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:55:23.896898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048689+0000","last_change":"2024-05-20T18:27:07.786091+0000","last_active":"2024-05-20T18:27:14.048689+0000","last_peered":"2024-05-20T18:27:14.048689+0000","last_clean":"2024-05-20T18:27:14.048689+0000","last_became_active":"2024-05-20T18:27:07.785704+0000","last_became_peered":"2024-05-20T18:27:07.785704+0000","last_unstale":"2024-05-20T18:27:14.048689+0000","last_undegraded":"2024-05-20T18:27:14.048689+0000","last_fullsized":"2024-05-20T18:27:14.048689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:06:32.450870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047619+0000","last_change":"2024-05-20T18:27:06.799668+0000","last_active":"2024-05-20T18:27:14.047619+0000","last_peered":"2024-05-20T18:27:14.047619+0000","last_clean":"2024-05-20T18:27:14.047619+0000","last_became_active":"2024-05-20T18:27:06.799002+0000","last_became_peered":"2024-05-20T18:27:06.799002+0000","last_unstale":"2024-05-20T18:27:14.047619+0000","last_undegraded":"2024-05-20T18:27:14.047619+0000","last_fullsized":"2024-05-20T18:27:14.047619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:06:31.445366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048146+0000","last_change":"2024-05-20T18:27:11.834133+0000","last_active":"2024-05-20T18:27:14.048146+0000","last_peered":"2024-05-20T18:27:14.048146+0000","last_clean":"2024-05-20T18:27:14.048146+0000","last_became_active":"2024-05-20T18:27:11.832891+0000","last_became_peered":"2024-05-20T18:27:11.832891+0000","last_unstale":"2024-05-20T18:27:14.048146+0000","last_undegraded":"2024-05-20T18:27:14.048146+0000","last_fullsized":"2024-05-20T18:27:14.048146+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:55:36.987878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049426+0000","last_change":"2024-05-20T18:27:07.790534+0000","last_active":"2024-05-20T18:27:14.049426+0000","last_peered":"2024-05-20T18:27:14.049426+0000","last_clean":"2024-05-20T18:27:14.049426+0000","last_became_active":"2024-05-20T18:27:07.790192+0000","last_became_peered":"2024-05-20T18:27:07.790192+0000","last_unstale":"2024-05-20T18:27:14.049426+0000","last_undegraded":"2024-05-20T18:27:14.049426+0000","last_fullsized":"2024-05-20T18:27:14.049426+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:12.006827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.826184+0000","last_change":"2024-05-20T18:27:06.794221+0000","last_active":"2024-05-20T18:27:13.826184+0000","last_peered":"2024-05-20T18:27:13.826184+0000","last_clean":"2024-05-20T18:27:13.826184+0000","last_became_active":"2024-05-20T18:27:06.793895+0000","last_became_peered":"2024-05-20T18:27:06.793895+0000","last_unstale":"2024-05-20T18:27:13.826184+0000","last_undegraded":"2024-05-20T18:27:13.826184+0000","last_fullsized":"2024-05-20T18:27:13.826184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:24:55.001924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048453+0000","last_change":"2024-05-20T18:27:12.471660+0000","last_active":"2024-05-20T18:27:14.048453+0000","last_peered":"2024-05-20T18:27:14.048453+0000","last_clean":"2024-05-20T18:27:14.048453+0000","last_became_active":"2024-05-20T18:27:12.470460+0000","last_became_peered":"2024-05-20T18:27:12.470460+0000","last_unstale":"2024-05-20T18:27:14.048453+0000","last_undegraded":"2024-05-20T18:27:14.048453+0000","last_fullsized":"2024-05-20T18:27:14.048453+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:53:56.893940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.822216+0000","last_change":"2024-05-20T18:27:07.799399+0000","last_active":"2024-05-20T18:27:13.822216+0000","last_peered":"2024-05-20T18:27:13.822216+0000","last_clean":"2024-05-20T18:27:13.822216+0000","last_became_active":"2024-05-20T18:27:07.799017+0000","last_became_peered":"2024-05-20T18:27:07.799017+0000","last_unstale":"2024-05-20T18:27:13.822216+0000","last_undegraded":"2024-05-20T18:27:13.822216+0000","last_fullsized":"2024-05-20T18:27:13.822216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:08:51.463251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049547+0000","last_change":"2024-05-20T18:27:06.794477+0000","last_active":"2024-05-20T18:27:14.049547+0000","last_peered":"2024-05-20T18:27:14.049547+0000","last_clean":"2024-05-20T18:27:14.049547+0000","last_became_active":"2024-05-20T18:27:06.794130+0000","last_became_peered":"2024-05-20T18:27:06.794130+0000","last_unstale":"2024-05-20T18:27:14.049547+0000","last_undegraded":"2024-05-20T18:27:14.049547+0000","last_fullsized":"2024-05-20T18:27:14.049547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:43:31.154358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821030+0000","last_change":"2024-05-20T18:27:12.468521+0000","last_active":"2024-05-20T18:27:13.821030+0000","last_peered":"2024-05-20T18:27:13.821030+0000","last_clean":"2024-05-20T18:27:13.821030+0000","last_became_active":"2024-05-20T18:27:12.467880+0000","last_became_peered":"2024-05-20T18:27:12.467880+0000","last_unstale":"2024-05-20T18:27:13.821030+0000","last_undegraded":"2024-05-20T18:27:13.821030+0000","last_fullsized":"2024-05-20T18:27:13.821030+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:48:16.180009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048938+0000","last_change":"2024-05-20T18:27:07.784489+0000","last_active":"2024-05-20T18:27:14.048938+0000","last_peered":"2024-05-20T18:27:14.048938+0000","last_clean":"2024-05-20T18:27:14.048938+0000","last_became_active":"2024-05-20T18:27:07.783968+0000","last_became_peered":"2024-05-20T18:27:07.783968+0000","last_unstale":"2024-05-20T18:27:14.048938+0000","last_undegraded":"2024-05-20T18:27:14.048938+0000","last_fullsized":"2024-05-20T18:27:14.048938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:07:15.764912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049643+0000","last_change":"2024-05-20T18:27:06.805626+0000","last_active":"2024-05-20T18:27:14.049643+0000","last_peered":"2024-05-20T18:27:14.049643+0000","last_clean":"2024-05-20T18:27:14.049643+0000","last_became_active":"2024-05-20T18:27:06.805267+0000","last_became_peered":"2024-05-20T18:27:06.805267+0000","last_unstale":"2024-05-20T18:27:14.049643+0000","last_undegraded":"2024-05-20T18:27:14.049643+0000","last_fullsized":"2024-05-20T18:27:14.049643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:18:18.528418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048076+0000","last_change":"2024-05-20T18:27:12.465690+0000","last_active":"2024-05-20T18:27:14.048076+0000","last_peered":"2024-05-20T18:27:14.048076+0000","last_clean":"2024-05-20T18:27:14.048076+0000","last_became_active":"2024-05-20T18:27:12.465196+0000","last_became_peered":"2024-05-20T18:27:12.465196+0000","last_unstale":"2024-05-20T18:27:14.048076+0000","last_undegraded":"2024-05-20T18:27:14.048076+0000","last_fullsized":"2024-05-20T18:27:14.048076+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:14:31.209452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825227+0000","last_change":"2024-05-20T18:27:07.806673+0000","last_active":"2024-05-20T18:27:13.825227+0000","last_peered":"2024-05-20T18:27:13.825227+0000","last_clean":"2024-05-20T18:27:13.825227+0000","last_became_active":"2024-05-20T18:27:07.805764+0000","last_became_peered":"2024-05-20T18:27:07.805764+0000","last_unstale":"2024-05-20T18:27:13.825227+0000","last_undegraded":"2024-05-20T18:27:13.825227+0000","last_fullsized":"2024-05-20T18:27:13.825227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:16:37.938695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049063+0000","last_change":"2024-05-20T18:27:06.796398+0000","last_active":"2024-05-20T18:27:14.049063+0000","last_peered":"2024-05-20T18:27:14.049063+0000","last_clean":"2024-05-20T18:27:14.049063+0000","last_became_active":"2024-05-20T18:27:06.795981+0000","last_became_peered":"2024-05-20T18:27:06.795981+0000","last_unstale":"2024-05-20T18:27:14.049063+0000","last_undegraded":"2024-05-20T18:27:14.049063+0000","last_fullsized":"2024-05-20T18:27:14.049063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:11:07.831277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048435+0000","last_change":"2024-05-20T18:27:11.830530+0000","last_active":"2024-05-20T18:27:14.048435+0000","last_peered":"2024-05-20T18:27:14.048435+0000","last_clean":"2024-05-20T18:27:14.048435+0000","last_became_active":"2024-05-20T18:27:11.828707+0000","last_became_peered":"2024-05-20T18:27:11.828707+0000","last_unstale":"2024-05-20T18:27:14.048435+0000","last_undegraded":"2024-05-20T18:27:14.048435+0000","last_fullsized":"2024-05-20T18:27:14.048435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:14:31.209452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048613+0000","last_change":"2024-05-20T18:27:07.802587+0000","last_active":"2024-05-20T18:27:14.048613+0000","last_peered":"2024-05-20T18:27:14.048613+0000","last_clean":"2024-05-20T18:27:14.048613+0000","last_became_active":"2024-05-20T18:27:07.802284+0000","last_became_peered":"2024-05-20T18:27:07.802284+0000","last_unstale":"2024-05-20T18:27:14.048613+0000","last_undegraded":"2024-05-20T18:27:14.048613+0000","last_fullsized":"2024-05-20T18:27:14.048613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:18:19.533923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825123+0000","last_change":"2024-05-20T18:27:06.795278+0000","last_active":"2024-05-20T18:27:13.825123+0000","last_peered":"2024-05-20T18:27:13.825123+0000","last_clean":"2024-05-20T18:27:13.825123+0000","last_became_active":"2024-05-20T18:27:06.794895+0000","last_became_peered":"2024-05-20T18:27:06.794895+0000","last_unstale":"2024-05-20T18:27:13.825123+0000","last_undegraded":"2024-05-20T18:27:13.825123+0000","last_fullsized":"2024-05-20T18:27:13.825123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:52:44.943426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048956+0000","last_change":"2024-05-20T18:27:11.833463+0000","last_active":"2024-05-20T18:27:14.048956+0000","last_peered":"2024-05-20T18:27:14.048956+0000","last_clean":"2024-05-20T18:27:14.048956+0000","last_became_active":"2024-05-20T18:27:11.832954+0000","last_became_peered":"2024-05-20T18:27:11.832954+0000","last_unstale":"2024-05-20T18:27:14.048956+0000","last_undegraded":"2024-05-20T18:27:14.048956+0000","last_fullsized":"2024-05-20T18:27:14.048956+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:34:21.648495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048483+0000","last_change":"2024-05-20T18:27:07.785038+0000","last_active":"2024-05-20T18:27:14.048483+0000","last_peered":"2024-05-20T18:27:14.048483+0000","last_clean":"2024-05-20T18:27:14.048483+0000","last_became_active":"2024-05-20T18:27:07.783927+0000","last_became_peered":"2024-05-20T18:27:07.783927+0000","last_unstale":"2024-05-20T18:27:14.048483+0000","last_undegraded":"2024-05-20T18:27:14.048483+0000","last_fullsized":"2024-05-20T18:27:14.048483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:12.006827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047552+0000","last_change":"2024-05-20T18:27:06.789785+0000","last_active":"2024-05-20T18:27:14.047552+0000","last_peered":"2024-05-20T18:27:14.047552+0000","last_clean":"2024-05-20T18:27:14.047552+0000","last_became_active":"2024-05-20T18:27:06.789421+0000","last_became_peered":"2024-05-20T18:27:06.789421+0000","last_unstale":"2024-05-20T18:27:14.047552+0000","last_undegraded":"2024-05-20T18:27:14.047552+0000","last_fullsized":"2024-05-20T18:27:14.047552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:24:55.001924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049327+0000","last_change":"2024-05-20T18:27:11.833707+0000","last_active":"2024-05-20T18:27:14.049327+0000","last_peered":"2024-05-20T18:27:14.049327+0000","last_clean":"2024-05-20T18:27:14.049327+0000","last_became_active":"2024-05-20T18:27:11.832619+0000","last_became_peered":"2024-05-20T18:27:11.832619+0000","last_unstale":"2024-05-20T18:27:14.049327+0000","last_undegraded":"2024-05-20T18:27:14.049327+0000","last_fullsized":"2024-05-20T18:27:14.049327+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:53:56.893940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047552+0000","last_change":"2024-05-20T18:27:07.799471+0000","last_active":"2024-05-20T18:27:14.047552+0000","last_peered":"2024-05-20T18:27:14.047552+0000","last_clean":"2024-05-20T18:27:14.047552+0000","last_became_active":"2024-05-20T18:27:07.799089+0000","last_became_peered":"2024-05-20T18:27:07.799089+0000","last_unstale":"2024-05-20T18:27:14.047552+0000","last_undegraded":"2024-05-20T18:27:14.047552+0000","last_fullsized":"2024-05-20T18:27:14.047552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:39:20.381356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047626+0000","last_change":"2024-05-20T18:27:06.799374+0000","last_active":"2024-05-20T18:27:14.047626+0000","last_peered":"2024-05-20T18:27:14.047626+0000","last_clean":"2024-05-20T18:27:14.047626+0000","last_became_active":"2024-05-20T18:27:06.798552+0000","last_became_peered":"2024-05-20T18:27:06.798552+0000","last_unstale":"2024-05-20T18:27:14.047626+0000","last_undegraded":"2024-05-20T18:27:14.047626+0000","last_fullsized":"2024-05-20T18:27:14.047626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:52:44.943426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049339+0000","last_change":"2024-05-20T18:27:12.469418+0000","last_active":"2024-05-20T18:27:14.049339+0000","last_peered":"2024-05-20T18:27:14.049339+0000","last_clean":"2024-05-20T18:27:14.049339+0000","last_became_active":"2024-05-20T18:27:12.468923+0000","last_became_peered":"2024-05-20T18:27:12.468923+0000","last_unstale":"2024-05-20T18:27:14.049339+0000","last_undegraded":"2024-05-20T18:27:14.049339+0000","last_fullsized":"2024-05-20T18:27:14.049339+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:55:23.896898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821455+0000","last_change":"2024-05-20T18:27:07.804772+0000","last_active":"2024-05-20T18:27:13.821455+0000","last_peered":"2024-05-20T18:27:13.821455+0000","last_clean":"2024-05-20T18:27:13.821455+0000","last_became_active":"2024-05-20T18:27:07.804443+0000","last_became_peered":"2024-05-20T18:27:07.804443+0000","last_unstale":"2024-05-20T18:27:13.821455+0000","last_undegraded":"2024-05-20T18:27:13.821455+0000","last_fullsized":"2024-05-20T18:27:13.821455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:14:17.791789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824615+0000","last_change":"2024-05-20T18:27:06.790984+0000","last_active":"2024-05-20T18:27:13.824615+0000","last_peered":"2024-05-20T18:27:13.824615+0000","last_clean":"2024-05-20T18:27:13.824615+0000","last_became_active":"2024-05-20T18:27:06.790620+0000","last_became_peered":"2024-05-20T18:27:06.790620+0000","last_unstale":"2024-05-20T18:27:13.824615+0000","last_undegraded":"2024-05-20T18:27:13.824615+0000","last_fullsized":"2024-05-20T18:27:13.824615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:10:27.276400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":179,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047791+0000","last_change":"2024-05-20T18:27:03.765694+0000","last_active":"2024-05-20T18:27:14.047791+0000","last_peered":"2024-05-20T18:27:14.047791+0000","last_clean":"2024-05-20T18:27:14.047791+0000","last_became_active":"2024-05-20T18:27:03.764695+0000","last_became_peered":"2024-05-20T18:27:03.764695+0000","last_unstale":"2024-05-20T18:27:14.047791+0000","last_undegraded":"2024-05-20T18:27:14.047791+0000","last_fullsized":"2024-05-20T18:27:14.047791+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_clean_scrub_stamp":"2024-05-20T18:27:02.750853+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:30:04.407487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824624+0000","last_change":"2024-05-20T18:27:12.467983+0000","last_active":"2024-05-20T18:27:13.824624+0000","last_peered":"2024-05-20T18:27:13.824624+0000","last_clean":"2024-05-20T18:27:13.824624+0000","last_became_active":"2024-05-20T18:27:12.467420+0000","last_became_peered":"2024-05-20T18:27:12.467420+0000","last_unstale":"2024-05-20T18:27:13.824624+0000","last_undegraded":"2024-05-20T18:27:13.824624+0000","last_fullsized":"2024-05-20T18:27:13.824624+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:01:50.652864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049235+0000","last_change":"2024-05-20T18:27:07.814589+0000","last_active":"2024-05-20T18:27:14.049235+0000","last_peered":"2024-05-20T18:27:14.049235+0000","last_clean":"2024-05-20T18:27:14.049235+0000","last_became_active":"2024-05-20T18:27:07.814281+0000","last_became_peered":"2024-05-20T18:27:07.814281+0000","last_unstale":"2024-05-20T18:27:14.049235+0000","last_undegraded":"2024-05-20T18:27:14.049235+0000","last_fullsized":"2024-05-20T18:27:14.049235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:29:33.478403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047483+0000","last_change":"2024-05-20T18:27:06.798075+0000","last_active":"2024-05-20T18:27:14.047483+0000","last_peered":"2024-05-20T18:27:14.047483+0000","last_clean":"2024-05-20T18:27:14.047483+0000","last_became_active":"2024-05-20T18:27:06.797696+0000","last_became_peered":"2024-05-20T18:27:06.797696+0000","last_unstale":"2024-05-20T18:27:14.047483+0000","last_undegraded":"2024-05-20T18:27:14.047483+0000","last_fullsized":"2024-05-20T18:27:14.047483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:06:31.445366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825521+0000","last_change":"2024-05-20T18:27:12.465857+0000","last_active":"2024-05-20T18:27:13.825521+0000","last_peered":"2024-05-20T18:27:13.825521+0000","last_clean":"2024-05-20T18:27:13.825521+0000","last_became_active":"2024-05-20T18:27:12.465296+0000","last_became_peered":"2024-05-20T18:27:12.465296+0000","last_unstale":"2024-05-20T18:27:13.825521+0000","last_undegraded":"2024-05-20T18:27:13.825521+0000","last_fullsized":"2024-05-20T18:27:13.825521+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:25:03.246597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825312+0000","last_change":"2024-05-20T18:27:07.808534+0000","last_active":"2024-05-20T18:27:13.825312+0000","last_peered":"2024-05-20T18:27:13.825312+0000","last_clean":"2024-05-20T18:27:13.825312+0000","last_became_active":"2024-05-20T18:27:07.807813+0000","last_became_peered":"2024-05-20T18:27:07.807813+0000","last_unstale":"2024-05-20T18:27:13.825312+0000","last_undegraded":"2024-05-20T18:27:13.825312+0000","last_fullsized":"2024-05-20T18:27:13.825312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:08:51.463251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049093+0000","last_change":"2024-05-20T18:27:06.793443+0000","last_active":"2024-05-20T18:27:14.049093+0000","last_peered":"2024-05-20T18:27:14.049093+0000","last_clean":"2024-05-20T18:27:14.049093+0000","last_became_active":"2024-05-20T18:27:06.793037+0000","last_became_peered":"2024-05-20T18:27:06.793037+0000","last_unstale":"2024-05-20T18:27:14.049093+0000","last_undegraded":"2024-05-20T18:27:14.049093+0000","last_fullsized":"2024-05-20T18:27:14.049093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:06:31.445366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048364+0000","last_change":"2024-05-20T18:27:11.828038+0000","last_active":"2024-05-20T18:27:14.048364+0000","last_peered":"2024-05-20T18:27:14.048364+0000","last_clean":"2024-05-20T18:27:14.048364+0000","last_became_active":"2024-05-20T18:27:11.827482+0000","last_became_peered":"2024-05-20T18:27:11.827482+0000","last_unstale":"2024-05-20T18:27:14.048364+0000","last_undegraded":"2024-05-20T18:27:14.048364+0000","last_fullsized":"2024-05-20T18:27:14.048364+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:38:58.503399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049301+0000","last_change":"2024-05-20T18:27:07.809779+0000","last_active":"2024-05-20T18:27:14.049301+0000","last_peered":"2024-05-20T18:27:14.049301+0000","last_clean":"2024-05-20T18:27:14.049301+0000","last_became_active":"2024-05-20T18:27:07.809458+0000","last_became_peered":"2024-05-20T18:27:07.809458+0000","last_unstale":"2024-05-20T18:27:14.049301+0000","last_undegraded":"2024-05-20T18:27:14.049301+0000","last_fullsized":"2024-05-20T18:27:14.049301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:55:32.958186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047310+0000","last_change":"2024-05-20T18:27:06.788828+0000","last_active":"2024-05-20T18:27:14.047310+0000","last_peered":"2024-05-20T18:27:14.047310+0000","last_clean":"2024-05-20T18:27:14.047310+0000","last_became_active":"2024-05-20T18:27:06.788142+0000","last_became_peered":"2024-05-20T18:27:06.788142+0000","last_unstale":"2024-05-20T18:27:14.047310+0000","last_undegraded":"2024-05-20T18:27:14.047310+0000","last_fullsized":"2024-05-20T18:27:14.047310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:52:44.943426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.823921+0000","last_change":"2024-05-20T18:27:12.466818+0000","last_active":"2024-05-20T18:27:13.823921+0000","last_peered":"2024-05-20T18:27:13.823921+0000","last_clean":"2024-05-20T18:27:13.823921+0000","last_became_active":"2024-05-20T18:27:12.466228+0000","last_became_peered":"2024-05-20T18:27:12.466228+0000","last_unstale":"2024-05-20T18:27:13.823921+0000","last_undegraded":"2024-05-20T18:27:13.823921+0000","last_fullsized":"2024-05-20T18:27:13.823921+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:55:23.896898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049485+0000","last_change":"2024-05-20T18:27:07.808881+0000","last_active":"2024-05-20T18:27:14.049485+0000","last_peered":"2024-05-20T18:27:14.049485+0000","last_clean":"2024-05-20T18:27:14.049485+0000","last_became_active":"2024-05-20T18:27:07.808550+0000","last_became_peered":"2024-05-20T18:27:07.808550+0000","last_unstale":"2024-05-20T18:27:14.049485+0000","last_undegraded":"2024-05-20T18:27:14.049485+0000","last_fullsized":"2024-05-20T18:27:14.049485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:01:46.623171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048816+0000","last_change":"2024-05-20T18:27:06.797126+0000","last_active":"2024-05-20T18:27:14.048816+0000","last_peered":"2024-05-20T18:27:14.048816+0000","last_clean":"2024-05-20T18:27:14.048816+0000","last_became_active":"2024-05-20T18:27:06.796764+0000","last_became_peered":"2024-05-20T18:27:06.796764+0000","last_unstale":"2024-05-20T18:27:14.048816+0000","last_undegraded":"2024-05-20T18:27:14.048816+0000","last_fullsized":"2024-05-20T18:27:14.048816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:59:18.905646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047675+0000","last_change":"2024-05-20T18:27:11.827350+0000","last_active":"2024-05-20T18:27:14.047675+0000","last_peered":"2024-05-20T18:27:14.047675+0000","last_clean":"2024-05-20T18:27:14.047675+0000","last_became_active":"2024-05-20T18:27:11.826656+0000","last_became_peered":"2024-05-20T18:27:11.826656+0000","last_unstale":"2024-05-20T18:27:14.047675+0000","last_undegraded":"2024-05-20T18:27:14.047675+0000","last_fullsized":"2024-05-20T18:27:14.047675+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:01:50.652864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048684+0000","last_change":"2024-05-20T18:27:07.809732+0000","last_active":"2024-05-20T18:27:14.048684+0000","last_peered":"2024-05-20T18:27:14.048684+0000","last_clean":"2024-05-20T18:27:14.048684+0000","last_became_active":"2024-05-20T18:27:07.809412+0000","last_became_peered":"2024-05-20T18:27:07.809412+0000","last_unstale":"2024-05-20T18:27:14.048684+0000","last_undegraded":"2024-05-20T18:27:14.048684+0000","last_fullsized":"2024-05-20T18:27:14.048684+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:14:03.712607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.046974+0000","last_change":"2024-05-20T18:27:06.787105+0000","last_active":"2024-05-20T18:27:14.046974+0000","last_peered":"2024-05-20T18:27:14.046974+0000","last_clean":"2024-05-20T18:27:14.046974+0000","last_became_active":"2024-05-20T18:27:06.786759+0000","last_became_peered":"2024-05-20T18:27:06.786759+0000","last_unstale":"2024-05-20T18:27:14.046974+0000","last_undegraded":"2024-05-20T18:27:14.046974+0000","last_fullsized":"2024-05-20T18:27:14.046974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:10:27.276400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049140+0000","last_change":"2024-05-20T18:27:11.834554+0000","last_active":"2024-05-20T18:27:14.049140+0000","last_peered":"2024-05-20T18:27:14.049140+0000","last_clean":"2024-05-20T18:27:14.049140+0000","last_became_active":"2024-05-20T18:27:11.833955+0000","last_became_peered":"2024-05-20T18:27:11.833955+0000","last_unstale":"2024-05-20T18:27:14.049140+0000","last_undegraded":"2024-05-20T18:27:14.049140+0000","last_fullsized":"2024-05-20T18:27:14.049140+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:53:56.893940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825845+0000","last_change":"2024-05-20T18:27:07.814023+0000","last_active":"2024-05-20T18:27:13.825845+0000","last_peered":"2024-05-20T18:27:13.825845+0000","last_clean":"2024-05-20T18:27:13.825845+0000","last_became_active":"2024-05-20T18:27:07.813679+0000","last_became_peered":"2024-05-20T18:27:07.813679+0000","last_unstale":"2024-05-20T18:27:13.825845+0000","last_undegraded":"2024-05-20T18:27:13.825845+0000","last_fullsized":"2024-05-20T18:27:13.825845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:10:20.947601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825319+0000","last_change":"2024-05-20T18:27:06.787006+0000","last_active":"2024-05-20T18:27:13.825319+0000","last_peered":"2024-05-20T18:27:13.825319+0000","last_clean":"2024-05-20T18:27:13.825319+0000","last_became_active":"2024-05-20T18:27:06.786565+0000","last_became_peered":"2024-05-20T18:27:06.786565+0000","last_unstale":"2024-05-20T18:27:13.825319+0000","last_undegraded":"2024-05-20T18:27:13.825319+0000","last_fullsized":"2024-05-20T18:27:13.825319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:10:27.276400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.822189+0000","last_change":"2024-05-20T18:27:12.468307+0000","last_active":"2024-05-20T18:27:13.822189+0000","last_peered":"2024-05-20T18:27:13.822189+0000","last_clean":"2024-05-20T18:27:13.822189+0000","last_became_active":"2024-05-20T18:27:12.467856+0000","last_became_peered":"2024-05-20T18:27:12.467856+0000","last_unstale":"2024-05-20T18:27:13.822189+0000","last_undegraded":"2024-05-20T18:27:13.822189+0000","last_fullsized":"2024-05-20T18:27:13.822189+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:08:26.960099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049060+0000","last_change":"2024-05-20T18:27:07.809032+0000","last_active":"2024-05-20T18:27:14.049060+0000","last_peered":"2024-05-20T18:27:14.049060+0000","last_clean":"2024-05-20T18:27:14.049060+0000","last_became_active":"2024-05-20T18:27:07.808708+0000","last_became_peered":"2024-05-20T18:27:07.808708+0000","last_unstale":"2024-05-20T18:27:14.049060+0000","last_undegraded":"2024-05-20T18:27:14.049060+0000","last_fullsized":"2024-05-20T18:27:14.049060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:08:51.463251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825345+0000","last_change":"2024-05-20T18:27:06.794995+0000","last_active":"2024-05-20T18:27:13.825345+0000","last_peered":"2024-05-20T18:27:13.825345+0000","last_clean":"2024-05-20T18:27:13.825345+0000","last_became_active":"2024-05-20T18:27:06.794662+0000","last_became_peered":"2024-05-20T18:27:06.794662+0000","last_unstale":"2024-05-20T18:27:13.825345+0000","last_undegraded":"2024-05-20T18:27:13.825345+0000","last_fullsized":"2024-05-20T18:27:13.825345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:06:31.445366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821770+0000","last_change":"2024-05-20T18:27:12.465482+0000","last_active":"2024-05-20T18:27:13.821770+0000","last_peered":"2024-05-20T18:27:13.821770+0000","last_clean":"2024-05-20T18:27:13.821770+0000","last_became_active":"2024-05-20T18:27:12.464905+0000","last_became_peered":"2024-05-20T18:27:12.464905+0000","last_unstale":"2024-05-20T18:27:13.821770+0000","last_undegraded":"2024-05-20T18:27:13.821770+0000","last_fullsized":"2024-05-20T18:27:13.821770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:16:04.416155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048114+0000","last_change":"2024-05-20T18:27:07.807096+0000","last_active":"2024-05-20T18:27:14.048114+0000","last_peered":"2024-05-20T18:27:14.048114+0000","last_clean":"2024-05-20T18:27:14.048114+0000","last_became_active":"2024-05-20T18:27:07.806625+0000","last_became_peered":"2024-05-20T18:27:07.806625+0000","last_unstale":"2024-05-20T18:27:14.048114+0000","last_undegraded":"2024-05-20T18:27:14.048114+0000","last_fullsized":"2024-05-20T18:27:14.048114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:15:10.022458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048546+0000","last_change":"2024-05-20T18:27:06.785919+0000","last_active":"2024-05-20T18:27:14.048546+0000","last_peered":"2024-05-20T18:27:14.048546+0000","last_clean":"2024-05-20T18:27:14.048546+0000","last_became_active":"2024-05-20T18:27:06.785587+0000","last_became_peered":"2024-05-20T18:27:06.785587+0000","last_unstale":"2024-05-20T18:27:14.048546+0000","last_undegraded":"2024-05-20T18:27:14.048546+0000","last_fullsized":"2024-05-20T18:27:14.048546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:10:44.980209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824622+0000","last_change":"2024-05-20T18:27:12.466586+0000","last_active":"2024-05-20T18:27:13.824622+0000","last_peered":"2024-05-20T18:27:13.824622+0000","last_clean":"2024-05-20T18:27:13.824622+0000","last_became_active":"2024-05-20T18:27:12.466076+0000","last_became_peered":"2024-05-20T18:27:12.466076+0000","last_unstale":"2024-05-20T18:27:13.824622+0000","last_undegraded":"2024-05-20T18:27:13.824622+0000","last_fullsized":"2024-05-20T18:27:13.824622+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:18:23.718914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049203+0000","last_change":"2024-05-20T18:27:07.805541+0000","last_active":"2024-05-20T18:27:14.049203+0000","last_peered":"2024-05-20T18:27:14.049203+0000","last_clean":"2024-05-20T18:27:14.049203+0000","last_became_active":"2024-05-20T18:27:07.805223+0000","last_became_peered":"2024-05-20T18:27:07.805223+0000","last_unstale":"2024-05-20T18:27:14.049203+0000","last_undegraded":"2024-05-20T18:27:14.049203+0000","last_fullsized":"2024-05-20T18:27:14.049203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:52:22.903884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":33,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:17.511072+0000","last_change":"2024-05-20T18:27:06.788980+0000","last_active":"2024-05-20T18:27:17.511072+0000","last_peered":"2024-05-20T18:27:17.511072+0000","last_clean":"2024-05-20T18:27:17.511072+0000","last_became_active":"2024-05-20T18:27:06.788477+0000","last_became_peered":"2024-05-20T18:27:06.788477+0000","last_unstale":"2024-05-20T18:27:17.511072+0000","last_undegraded":"2024-05-20T18:27:17.511072+0000","last_fullsized":"2024-05-20T18:27:17.511072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:43:31.154358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048876+0000","last_change":"2024-05-20T18:27:11.829159+0000","last_active":"2024-05-20T18:27:14.048876+0000","last_peered":"2024-05-20T18:27:14.048876+0000","last_clean":"2024-05-20T18:27:14.048876+0000","last_became_active":"2024-05-20T18:27:11.826918+0000","last_became_peered":"2024-05-20T18:27:11.826918+0000","last_unstale":"2024-05-20T18:27:14.048876+0000","last_undegraded":"2024-05-20T18:27:14.048876+0000","last_fullsized":"2024-05-20T18:27:14.048876+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:38:58.503399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048691+0000","last_change":"2024-05-20T18:27:07.791122+0000","last_active":"2024-05-20T18:27:14.048691+0000","last_peered":"2024-05-20T18:27:14.048691+0000","last_clean":"2024-05-20T18:27:14.048691+0000","last_became_active":"2024-05-20T18:27:07.790777+0000","last_became_peered":"2024-05-20T18:27:07.790777+0000","last_unstale":"2024-05-20T18:27:14.048691+0000","last_undegraded":"2024-05-20T18:27:14.048691+0000","last_fullsized":"2024-05-20T18:27:14.048691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:11:08.836782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048983+0000","last_change":"2024-05-20T18:27:06.799123+0000","last_active":"2024-05-20T18:27:14.048983+0000","last_peered":"2024-05-20T18:27:14.048983+0000","last_clean":"2024-05-20T18:27:14.048983+0000","last_became_active":"2024-05-20T18:27:06.798681+0000","last_became_peered":"2024-05-20T18:27:06.798681+0000","last_unstale":"2024-05-20T18:27:14.048983+0000","last_undegraded":"2024-05-20T18:27:14.048983+0000","last_fullsized":"2024-05-20T18:27:14.048983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:18:18.528418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048322+0000","last_change":"2024-05-20T18:27:11.823730+0000","last_active":"2024-05-20T18:27:14.048322+0000","last_peered":"2024-05-20T18:27:14.048322+0000","last_clean":"2024-05-20T18:27:14.048322+0000","last_became_active":"2024-05-20T18:27:11.823226+0000","last_became_peered":"2024-05-20T18:27:11.823226+0000","last_unstale":"2024-05-20T18:27:14.048322+0000","last_undegraded":"2024-05-20T18:27:14.048322+0000","last_fullsized":"2024-05-20T18:27:14.048322+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:14:31.209452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048598+0000","last_change":"2024-05-20T18:27:07.804871+0000","last_active":"2024-05-20T18:27:14.048598+0000","last_peered":"2024-05-20T18:27:14.048598+0000","last_clean":"2024-05-20T18:27:14.048598+0000","last_became_active":"2024-05-20T18:27:07.804518+0000","last_became_peered":"2024-05-20T18:27:07.804518+0000","last_unstale":"2024-05-20T18:27:14.048598+0000","last_undegraded":"2024-05-20T18:27:14.048598+0000","last_fullsized":"2024-05-20T18:27:14.048598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:14:03.712607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824274+0000","last_change":"2024-05-20T18:27:06.795936+0000","last_active":"2024-05-20T18:27:13.824274+0000","last_peered":"2024-05-20T18:27:13.824274+0000","last_clean":"2024-05-20T18:27:13.824274+0000","last_became_active":"2024-05-20T18:27:06.795616+0000","last_became_peered":"2024-05-20T18:27:06.795616+0000","last_unstale":"2024-05-20T18:27:13.824274+0000","last_undegraded":"2024-05-20T18:27:13.824274+0000","last_fullsized":"2024-05-20T18:27:13.824274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:24:55.001924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821242+0000","last_change":"2024-05-20T18:27:11.827063+0000","last_active":"2024-05-20T18:27:13.821242+0000","last_peered":"2024-05-20T18:27:13.821242+0000","last_clean":"2024-05-20T18:27:13.821242+0000","last_became_active":"2024-05-20T18:27:11.825223+0000","last_became_peered":"2024-05-20T18:27:11.825223+0000","last_unstale":"2024-05-20T18:27:13.821242+0000","last_undegraded":"2024-05-20T18:27:13.821242+0000","last_fullsized":"2024-05-20T18:27:13.821242+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:01:18.112337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824776+0000","last_change":"2024-05-20T18:27:07.810225+0000","last_active":"2024-05-20T18:27:13.824776+0000","last_peered":"2024-05-20T18:27:13.824776+0000","last_clean":"2024-05-20T18:27:13.824776+0000","last_became_active":"2024-05-20T18:27:07.809511+0000","last_became_peered":"2024-05-20T18:27:07.809511+0000","last_unstale":"2024-05-20T18:27:13.824776+0000","last_undegraded":"2024-05-20T18:27:13.824776+0000","last_fullsized":"2024-05-20T18:27:13.824776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:24:59.216904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.826020+0000","last_change":"2024-05-20T18:27:06.796334+0000","last_active":"2024-05-20T18:27:13.826020+0000","last_peered":"2024-05-20T18:27:13.826020+0000","last_clean":"2024-05-20T18:27:13.826020+0000","last_became_active":"2024-05-20T18:27:06.795897+0000","last_became_peered":"2024-05-20T18:27:06.795897+0000","last_unstale":"2024-05-20T18:27:13.826020+0000","last_undegraded":"2024-05-20T18:27:13.826020+0000","last_fullsized":"2024-05-20T18:27:13.826020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:06:31.445366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821208+0000","last_change":"2024-05-20T18:27:12.465588+0000","last_active":"2024-05-20T18:27:13.821208+0000","last_peered":"2024-05-20T18:27:13.821208+0000","last_clean":"2024-05-20T18:27:13.821208+0000","last_became_active":"2024-05-20T18:27:12.465170+0000","last_became_peered":"2024-05-20T18:27:12.465170+0000","last_unstale":"2024-05-20T18:27:13.821208+0000","last_undegraded":"2024-05-20T18:27:13.821208+0000","last_fullsized":"2024-05-20T18:27:13.821208+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:36:54.997159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048936+0000","last_change":"2024-05-20T18:27:12.469946+0000","last_active":"2024-05-20T18:27:14.048936+0000","last_peered":"2024-05-20T18:27:14.048936+0000","last_clean":"2024-05-20T18:27:14.048936+0000","last_became_active":"2024-05-20T18:27:12.469526+0000","last_became_peered":"2024-05-20T18:27:12.469526+0000","last_unstale":"2024-05-20T18:27:14.048936+0000","last_undegraded":"2024-05-20T18:27:14.048936+0000","last_fullsized":"2024-05-20T18:27:14.048936+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:34:21.648495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048409+0000","last_change":"2024-05-20T18:27:07.788636+0000","last_active":"2024-05-20T18:27:14.048409+0000","last_peered":"2024-05-20T18:27:14.048409+0000","last_clean":"2024-05-20T18:27:14.048409+0000","last_became_active":"2024-05-20T18:27:07.788307+0000","last_became_peered":"2024-05-20T18:27:07.788307+0000","last_unstale":"2024-05-20T18:27:14.048409+0000","last_undegraded":"2024-05-20T18:27:14.048409+0000","last_fullsized":"2024-05-20T18:27:14.048409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:10:20.947601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824264+0000","last_change":"2024-05-20T18:27:06.797781+0000","last_active":"2024-05-20T18:27:13.824264+0000","last_peered":"2024-05-20T18:27:13.824264+0000","last_clean":"2024-05-20T18:27:13.824264+0000","last_became_active":"2024-05-20T18:27:06.797415+0000","last_became_peered":"2024-05-20T18:27:06.797415+0000","last_unstale":"2024-05-20T18:27:13.824264+0000","last_undegraded":"2024-05-20T18:27:13.824264+0000","last_fullsized":"2024-05-20T18:27:13.824264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:18:18.528418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825975+0000","last_change":"2024-05-20T18:27:11.827561+0000","last_active":"2024-05-20T18:27:13.825975+0000","last_peered":"2024-05-20T18:27:13.825975+0000","last_clean":"2024-05-20T18:27:13.825975+0000","last_became_active":"2024-05-20T18:27:11.826925+0000","last_became_peered":"2024-05-20T18:27:11.826925+0000","last_unstale":"2024-05-20T18:27:13.825975+0000","last_undegraded":"2024-05-20T18:27:13.825975+0000","last_fullsized":"2024-05-20T18:27:13.825975+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:16:41.968388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821278+0000","last_change":"2024-05-20T18:27:07.789976+0000","last_active":"2024-05-20T18:27:13.821278+0000","last_peered":"2024-05-20T18:27:13.821278+0000","last_clean":"2024-05-20T18:27:13.821278+0000","last_became_active":"2024-05-20T18:27:07.789679+0000","last_became_peered":"2024-05-20T18:27:07.789679+0000","last_unstale":"2024-05-20T18:27:13.821278+0000","last_undegraded":"2024-05-20T18:27:13.821278+0000","last_fullsized":"2024-05-20T18:27:13.821278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:52:57.891709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821366+0000","last_change":"2024-05-20T18:27:06.794988+0000","last_active":"2024-05-20T18:27:13.821366+0000","last_peered":"2024-05-20T18:27:13.821366+0000","last_clean":"2024-05-20T18:27:13.821366+0000","last_became_active":"2024-05-20T18:27:06.794630+0000","last_became_peered":"2024-05-20T18:27:06.794630+0000","last_unstale":"2024-05-20T18:27:13.821366+0000","last_undegraded":"2024-05-20T18:27:13.821366+0000","last_fullsized":"2024-05-20T18:27:13.821366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:24:55.001924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825689+0000","last_change":"2024-05-20T18:27:06.785041+0000","last_active":"2024-05-20T18:27:13.825689+0000","last_peered":"2024-05-20T18:27:13.825689+0000","last_clean":"2024-05-20T18:27:13.825689+0000","last_became_active":"2024-05-20T18:27:06.784673+0000","last_became_peered":"2024-05-20T18:27:06.784673+0000","last_unstale":"2024-05-20T18:27:13.825689+0000","last_undegraded":"2024-05-20T18:27:13.825689+0000","last_fullsized":"2024-05-20T18:27:13.825689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:52:44.943426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048466+0000","last_change":"2024-05-20T18:27:07.790855+0000","last_active":"2024-05-20T18:27:14.048466+0000","last_peered":"2024-05-20T18:27:14.048466+0000","last_clean":"2024-05-20T18:27:14.048466+0000","last_became_active":"2024-05-20T18:27:07.790370+0000","last_became_peered":"2024-05-20T18:27:07.790370+0000","last_unstale":"2024-05-20T18:27:14.048466+0000","last_undegraded":"2024-05-20T18:27:14.048466+0000","last_fullsized":"2024-05-20T18:27:14.048466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:07:15.764912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047264+0000","last_change":"2024-05-20T18:27:11.828935+0000","last_active":"2024-05-20T18:27:14.047264+0000","last_peered":"2024-05-20T18:27:14.047264+0000","last_clean":"2024-05-20T18:27:14.047264+0000","last_became_active":"2024-05-20T18:27:11.828516+0000","last_became_peered":"2024-05-20T18:27:11.828516+0000","last_unstale":"2024-05-20T18:27:14.047264+0000","last_undegraded":"2024-05-20T18:27:14.047264+0000","last_fullsized":"2024-05-20T18:27:14.047264+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:15:14.052151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821617+0000","last_change":"2024-05-20T18:27:06.803606+0000","last_active":"2024-05-20T18:27:13.821617+0000","last_peered":"2024-05-20T18:27:13.821617+0000","last_clean":"2024-05-20T18:27:13.821617+0000","last_became_active":"2024-05-20T18:27:06.803340+0000","last_became_peered":"2024-05-20T18:27:06.803340+0000","last_unstale":"2024-05-20T18:27:13.821617+0000","last_undegraded":"2024-05-20T18:27:13.821617+0000","last_fullsized":"2024-05-20T18:27:13.821617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:10:27.276400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048051+0000","last_change":"2024-05-20T18:27:07.789834+0000","last_active":"2024-05-20T18:27:14.048051+0000","last_peered":"2024-05-20T18:27:14.048051+0000","last_clean":"2024-05-20T18:27:14.048051+0000","last_became_active":"2024-05-20T18:27:07.789472+0000","last_became_peered":"2024-05-20T18:27:07.789472+0000","last_unstale":"2024-05-20T18:27:14.048051+0000","last_undegraded":"2024-05-20T18:27:14.048051+0000","last_fullsized":"2024-05-20T18:27:14.048051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:52:45.948931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821874+0000","last_change":"2024-05-20T18:27:11.825859+0000","last_active":"2024-05-20T18:27:13.821874+0000","last_peered":"2024-05-20T18:27:13.821874+0000","last_clean":"2024-05-20T18:27:13.821874+0000","last_became_active":"2024-05-20T18:27:11.825093+0000","last_became_peered":"2024-05-20T18:27:11.825093+0000","last_unstale":"2024-05-20T18:27:13.821874+0000","last_undegraded":"2024-05-20T18:27:13.821874+0000","last_fullsized":"2024-05-20T18:27:13.821874+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:02:46.122485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825718+0000","last_change":"2024-05-20T18:27:07.793263+0000","last_active":"2024-05-20T18:27:13.825718+0000","last_peered":"2024-05-20T18:27:13.825718+0000","last_clean":"2024-05-20T18:27:13.825718+0000","last_became_active":"2024-05-20T18:27:07.792916+0000","last_became_peered":"2024-05-20T18:27:07.792916+0000","last_unstale":"2024-05-20T18:27:13.825718+0000","last_undegraded":"2024-05-20T18:27:13.825718+0000","last_fullsized":"2024-05-20T18:27:13.825718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:15:10.022458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047433+0000","last_change":"2024-05-20T18:27:07.805541+0000","last_active":"2024-05-20T18:27:14.047433+0000","last_peered":"2024-05-20T18:27:14.047433+0000","last_clean":"2024-05-20T18:27:14.047433+0000","last_became_active":"2024-05-20T18:27:07.805160+0000","last_became_peered":"2024-05-20T18:27:07.805160+0000","last_unstale":"2024-05-20T18:27:14.047433+0000","last_undegraded":"2024-05-20T18:27:14.047433+0000","last_fullsized":"2024-05-20T18:27:14.047433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:14:23.046328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824975+0000","last_change":"2024-05-20T18:27:07.804547+0000","last_active":"2024-05-20T18:27:13.824975+0000","last_peered":"2024-05-20T18:27:13.824975+0000","last_clean":"2024-05-20T18:27:13.824975+0000","last_became_active":"2024-05-20T18:27:07.804242+0000","last_became_peered":"2024-05-20T18:27:07.804242+0000","last_unstale":"2024-05-20T18:27:13.824975+0000","last_undegraded":"2024-05-20T18:27:13.824975+0000","last_fullsized":"2024-05-20T18:27:13.824975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:55:24.560518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048436+0000","last_change":"2024-05-20T18:27:07.790683+0000","last_active":"2024-05-20T18:27:14.048436+0000","last_peered":"2024-05-20T18:27:14.048436+0000","last_clean":"2024-05-20T18:27:14.048436+0000","last_became_active":"2024-05-20T18:27:07.789978+0000","last_became_peered":"2024-05-20T18:27:07.789978+0000","last_unstale":"2024-05-20T18:27:14.048436+0000","last_undegraded":"2024-05-20T18:27:14.048436+0000","last_fullsized":"2024-05-20T18:27:14.048436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:40:08.698691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821852+0000","last_change":"2024-05-20T18:27:07.791924+0000","last_active":"2024-05-20T18:27:13.821852+0000","last_peered":"2024-05-20T18:27:13.821852+0000","last_clean":"2024-05-20T18:27:13.821852+0000","last_became_active":"2024-05-20T18:27:07.791622+0000","last_became_peered":"2024-05-20T18:27:07.791622+0000","last_unstale":"2024-05-20T18:27:13.821852+0000","last_undegraded":"2024-05-20T18:27:13.821852+0000","last_fullsized":"2024-05-20T18:27:13.821852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:07:15.764912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048748+0000","last_change":"2024-05-20T18:27:07.789116+0000","last_active":"2024-05-20T18:27:14.048748+0000","last_peered":"2024-05-20T18:27:14.048748+0000","last_clean":"2024-05-20T18:27:14.048748+0000","last_became_active":"2024-05-20T18:27:07.787855+0000","last_became_peered":"2024-05-20T18:27:07.787855+0000","last_unstale":"2024-05-20T18:27:14.048748+0000","last_undegraded":"2024-05-20T18:27:14.048748+0000","last_fullsized":"2024-05-20T18:27:14.048748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:52:57.891709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825914+0000","last_change":"2024-05-20T18:27:07.793074+0000","last_active":"2024-05-20T18:27:13.825914+0000","last_peered":"2024-05-20T18:27:13.825914+0000","last_clean":"2024-05-20T18:27:13.825914+0000","last_became_active":"2024-05-20T18:27:07.792754+0000","last_became_peered":"2024-05-20T18:27:07.792754+0000","last_unstale":"2024-05-20T18:27:13.825914+0000","last_undegraded":"2024-05-20T18:27:13.825914+0000","last_fullsized":"2024-05-20T18:27:13.825914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:12.006827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.822159+0000","last_change":"2024-05-20T18:27:07.791208+0000","last_active":"2024-05-20T18:27:13.822159+0000","last_peered":"2024-05-20T18:27:13.822159+0000","last_clean":"2024-05-20T18:27:13.822159+0000","last_became_active":"2024-05-20T18:27:07.790864+0000","last_became_peered":"2024-05-20T18:27:07.790864+0000","last_unstale":"2024-05-20T18:27:13.822159+0000","last_undegraded":"2024-05-20T18:27:13.822159+0000","last_fullsized":"2024-05-20T18:27:13.822159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:19:31.649320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824356+0000","last_change":"2024-05-20T18:27:07.808305+0000","last_active":"2024-05-20T18:27:13.824356+0000","last_peered":"2024-05-20T18:27:13.824356+0000","last_clean":"2024-05-20T18:27:13.824356+0000","last_became_active":"2024-05-20T18:27:07.807877+0000","last_became_peered":"2024-05-20T18:27:07.807877+0000","last_unstale":"2024-05-20T18:27:13.824356+0000","last_undegraded":"2024-05-20T18:27:13.824356+0000","last_fullsized":"2024-05-20T18:27:13.824356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:14:03.712607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821322+0000","last_change":"2024-05-20T18:27:07.791872+0000","last_active":"2024-05-20T18:27:13.821322+0000","last_peered":"2024-05-20T18:27:13.821322+0000","last_clean":"2024-05-20T18:27:13.821322+0000","last_became_active":"2024-05-20T18:27:07.791570+0000","last_became_peered":"2024-05-20T18:27:07.791570+0000","last_unstale":"2024-05-20T18:27:13.821322+0000","last_undegraded":"2024-05-20T18:27:13.821322+0000","last_fullsized":"2024-05-20T18:27:13.821322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:29:33.478403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824943+0000","last_change":"2024-05-20T18:27:07.811057+0000","last_active":"2024-05-20T18:27:13.824943+0000","last_peered":"2024-05-20T18:27:13.824943+0000","last_clean":"2024-05-20T18:27:13.824943+0000","last_became_active":"2024-05-20T18:27:07.810728+0000","last_became_peered":"2024-05-20T18:27:07.810728+0000","last_unstale":"2024-05-20T18:27:13.824943+0000","last_undegraded":"2024-05-20T18:27:13.824943+0000","last_fullsized":"2024-05-20T18:27:13.824943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:19:31.649320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048182+0000","last_change":"2024-05-20T18:27:07.795559+0000","last_active":"2024-05-20T18:27:14.048182+0000","last_peered":"2024-05-20T18:27:14.048182+0000","last_clean":"2024-05-20T18:27:14.048182+0000","last_became_active":"2024-05-20T18:27:07.795228+0000","last_became_peered":"2024-05-20T18:27:07.795228+0000","last_unstale":"2024-05-20T18:27:14.048182+0000","last_undegraded":"2024-05-20T18:27:14.048182+0000","last_fullsized":"2024-05-20T18:27:14.048182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:10:20.947601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.826124+0000","last_change":"2024-05-20T18:27:07.795052+0000","last_active":"2024-05-20T18:27:13.826124+0000","last_peered":"2024-05-20T18:27:13.826124+0000","last_clean":"2024-05-20T18:27:13.826124+0000","last_became_active":"2024-05-20T18:27:07.794704+0000","last_became_peered":"2024-05-20T18:27:07.794704+0000","last_unstale":"2024-05-20T18:27:13.826124+0000","last_undegraded":"2024-05-20T18:27:13.826124+0000","last_fullsized":"2024-05-20T18:27:13.826124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:34:15.614124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049515+0000","last_change":"2024-05-20T18:27:07.807441+0000","last_active":"2024-05-20T18:27:14.049515+0000","last_peered":"2024-05-20T18:27:14.049515+0000","last_clean":"2024-05-20T18:27:14.049515+0000","last_became_active":"2024-05-20T18:27:07.807095+0000","last_became_peered":"2024-05-20T18:27:07.807095+0000","last_unstale":"2024-05-20T18:27:14.049515+0000","last_undegraded":"2024-05-20T18:27:14.049515+0000","last_fullsized":"2024-05-20T18:27:14.049515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:24:59.216904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047301+0000","last_change":"2024-05-20T18:27:07.801623+0000","last_active":"2024-05-20T18:27:14.047301+0000","last_peered":"2024-05-20T18:27:14.047301+0000","last_clean":"2024-05-20T18:27:14.047301+0000","last_became_active":"2024-05-20T18:27:07.801303+0000","last_became_peered":"2024-05-20T18:27:07.801303+0000","last_unstale":"2024-05-20T18:27:14.047301+0000","last_undegraded":"2024-05-20T18:27:14.047301+0000","last_fullsized":"2024-05-20T18:27:14.047301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:29:19.667394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824621+0000","last_change":"2024-05-20T18:27:07.809861+0000","last_active":"2024-05-20T18:27:13.824621+0000","last_peered":"2024-05-20T18:27:13.824621+0000","last_clean":"2024-05-20T18:27:13.824621+0000","last_became_active":"2024-05-20T18:27:07.809523+0000","last_became_peered":"2024-05-20T18:27:07.809523+0000","last_unstale":"2024-05-20T18:27:13.824621+0000","last_undegraded":"2024-05-20T18:27:13.824621+0000","last_fullsized":"2024-05-20T18:27:13.824621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:55:32.958186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.823933+0000","last_change":"2024-05-20T18:27:07.814234+0000","last_active":"2024-05-20T18:27:13.823933+0000","last_peered":"2024-05-20T18:27:13.823933+0000","last_clean":"2024-05-20T18:27:13.823933+0000","last_became_active":"2024-05-20T18:27:07.813168+0000","last_became_peered":"2024-05-20T18:27:07.813168+0000","last_unstale":"2024-05-20T18:27:13.823933+0000","last_undegraded":"2024-05-20T18:27:13.823933+0000","last_fullsized":"2024-05-20T18:27:13.823933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:12.006827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048520+0000","last_change":"2024-05-20T18:27:07.787849+0000","last_active":"2024-05-20T18:27:14.048520+0000","last_peered":"2024-05-20T18:27:14.048520+0000","last_clean":"2024-05-20T18:27:14.048520+0000","last_became_active":"2024-05-20T18:27:07.787406+0000","last_became_peered":"2024-05-20T18:27:07.787406+0000","last_unstale":"2024-05-20T18:27:14.048520+0000","last_undegraded":"2024-05-20T18:27:14.048520+0000","last_fullsized":"2024-05-20T18:27:14.048520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:34:15.614124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047987+0000","last_change":"2024-05-20T18:27:07.795141+0000","last_active":"2024-05-20T18:27:14.047987+0000","last_peered":"2024-05-20T18:27:14.047987+0000","last_clean":"2024-05-20T18:27:14.047987+0000","last_became_active":"2024-05-20T18:27:07.793487+0000","last_became_peered":"2024-05-20T18:27:07.793487+0000","last_unstale":"2024-05-20T18:27:14.047987+0000","last_undegraded":"2024-05-20T18:27:14.047987+0000","last_fullsized":"2024-05-20T18:27:14.047987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:32:42.769509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821496+0000","last_change":"2024-05-20T18:27:07.802751+0000","last_active":"2024-05-20T18:27:13.821496+0000","last_peered":"2024-05-20T18:27:13.821496+0000","last_clean":"2024-05-20T18:27:13.821496+0000","last_became_active":"2024-05-20T18:27:07.802412+0000","last_became_peered":"2024-05-20T18:27:07.802412+0000","last_unstale":"2024-05-20T18:27:13.821496+0000","last_undegraded":"2024-05-20T18:27:13.821496+0000","last_fullsized":"2024-05-20T18:27:13.821496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:14:03.712607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048564+0000","last_change":"2024-05-20T18:27:07.796408+0000","last_active":"2024-05-20T18:27:14.048564+0000","last_peered":"2024-05-20T18:27:14.048564+0000","last_clean":"2024-05-20T18:27:14.048564+0000","last_became_active":"2024-05-20T18:27:07.795961+0000","last_became_peered":"2024-05-20T18:27:07.795961+0000","last_unstale":"2024-05-20T18:27:14.048564+0000","last_undegraded":"2024-05-20T18:27:14.048564+0000","last_fullsized":"2024-05-20T18:27:14.048564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:43:32.159862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824841+0000","last_change":"2024-05-20T18:27:07.806684+0000","last_active":"2024-05-20T18:27:13.824841+0000","last_peered":"2024-05-20T18:27:13.824841+0000","last_clean":"2024-05-20T18:27:13.824841+0000","last_became_active":"2024-05-20T18:27:07.806320+0000","last_became_peered":"2024-05-20T18:27:07.806320+0000","last_unstale":"2024-05-20T18:27:13.824841+0000","last_undegraded":"2024-05-20T18:27:13.824841+0000","last_fullsized":"2024-05-20T18:27:13.824841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:52:57.891709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825792+0000","last_change":"2024-05-20T18:27:07.805587+0000","last_active":"2024-05-20T18:27:13.825792+0000","last_peered":"2024-05-20T18:27:13.825792+0000","last_clean":"2024-05-20T18:27:13.825792+0000","last_became_active":"2024-05-20T18:27:07.804971+0000","last_became_peered":"2024-05-20T18:27:07.804971+0000","last_unstale":"2024-05-20T18:27:13.825792+0000","last_undegraded":"2024-05-20T18:27:13.825792+0000","last_fullsized":"2024-05-20T18:27:13.825792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:07:15.764912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049422+0000","last_change":"2024-05-20T18:27:07.815867+0000","last_active":"2024-05-20T18:27:14.049422+0000","last_peered":"2024-05-20T18:27:14.049422+0000","last_clean":"2024-05-20T18:27:14.049422+0000","last_became_active":"2024-05-20T18:27:07.815421+0000","last_became_peered":"2024-05-20T18:27:07.815421+0000","last_unstale":"2024-05-20T18:27:14.049422+0000","last_undegraded":"2024-05-20T18:27:14.049422+0000","last_fullsized":"2024-05-20T18:27:14.049422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:16:37.938695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821166+0000","last_change":"2024-05-20T18:27:07.810206+0000","last_active":"2024-05-20T18:27:13.821166+0000","last_peered":"2024-05-20T18:27:13.821166+0000","last_clean":"2024-05-20T18:27:13.821166+0000","last_became_active":"2024-05-20T18:27:07.809846+0000","last_became_peered":"2024-05-20T18:27:07.809846+0000","last_unstale":"2024-05-20T18:27:13.821166+0000","last_undegraded":"2024-05-20T18:27:13.821166+0000","last_fullsized":"2024-05-20T18:27:13.821166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:10:20.947601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048623+0000","last_change":"2024-05-20T18:27:07.813420+0000","last_active":"2024-05-20T18:27:14.048623+0000","last_peered":"2024-05-20T18:27:14.048623+0000","last_clean":"2024-05-20T18:27:14.048623+0000","last_became_active":"2024-05-20T18:27:07.813093+0000","last_became_peered":"2024-05-20T18:27:07.813093+0000","last_unstale":"2024-05-20T18:27:14.048623+0000","last_undegraded":"2024-05-20T18:27:14.048623+0000","last_fullsized":"2024-05-20T18:27:14.048623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:19:31.649320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824831+0000","last_change":"2024-05-20T18:27:07.812561+0000","last_active":"2024-05-20T18:27:13.824831+0000","last_peered":"2024-05-20T18:27:13.824831+0000","last_clean":"2024-05-20T18:27:13.824831+0000","last_became_active":"2024-05-20T18:27:07.812264+0000","last_became_peered":"2024-05-20T18:27:07.812264+0000","last_unstale":"2024-05-20T18:27:13.824831+0000","last_undegraded":"2024-05-20T18:27:13.824831+0000","last_fullsized":"2024-05-20T18:27:13.824831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:52:22.903884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048783+0000","last_change":"2024-05-20T18:27:07.804441+0000","last_active":"2024-05-20T18:27:14.048783+0000","last_peered":"2024-05-20T18:27:14.048783+0000","last_clean":"2024-05-20T18:27:14.048783+0000","last_became_active":"2024-05-20T18:27:07.804126+0000","last_became_peered":"2024-05-20T18:27:07.804126+0000","last_unstale":"2024-05-20T18:27:14.048783+0000","last_undegraded":"2024-05-20T18:27:14.048783+0000","last_fullsized":"2024-05-20T18:27:14.048783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:14:17.791789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825082+0000","last_change":"2024-05-20T18:27:07.808819+0000","last_active":"2024-05-20T18:27:13.825082+0000","last_peered":"2024-05-20T18:27:13.825082+0000","last_clean":"2024-05-20T18:27:13.825082+0000","last_became_active":"2024-05-20T18:27:07.808231+0000","last_became_peered":"2024-05-20T18:27:07.808231+0000","last_unstale":"2024-05-20T18:27:13.825082+0000","last_undegraded":"2024-05-20T18:27:13.825082+0000","last_fullsized":"2024-05-20T18:27:13.825082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:14:17.791789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048097+0000","last_change":"2024-05-20T18:27:07.793960+0000","last_active":"2024-05-20T18:27:14.048097+0000","last_peered":"2024-05-20T18:27:14.048097+0000","last_clean":"2024-05-20T18:27:14.048097+0000","last_became_active":"2024-05-20T18:27:07.793594+0000","last_became_peered":"2024-05-20T18:27:07.793594+0000","last_unstale":"2024-05-20T18:27:14.048097+0000","last_undegraded":"2024-05-20T18:27:14.048097+0000","last_fullsized":"2024-05-20T18:27:14.048097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:34:15.614124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049410+0000","last_change":"2024-05-20T18:27:07.790606+0000","last_active":"2024-05-20T18:27:14.049410+0000","last_peered":"2024-05-20T18:27:14.049410+0000","last_clean":"2024-05-20T18:27:14.049410+0000","last_became_active":"2024-05-20T18:27:07.790227+0000","last_became_peered":"2024-05-20T18:27:07.790227+0000","last_unstale":"2024-05-20T18:27:14.049410+0000","last_undegraded":"2024-05-20T18:27:14.049410+0000","last_fullsized":"2024-05-20T18:27:14.049410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:14:03.712607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048973+0000","last_change":"2024-05-20T18:27:07.816207+0000","last_active":"2024-05-20T18:27:14.048973+0000","last_peered":"2024-05-20T18:27:14.048973+0000","last_clean":"2024-05-20T18:27:14.048973+0000","last_became_active":"2024-05-20T18:27:07.815556+0000","last_became_peered":"2024-05-20T18:27:07.815556+0000","last_unstale":"2024-05-20T18:27:14.048973+0000","last_undegraded":"2024-05-20T18:27:14.048973+0000","last_fullsized":"2024-05-20T18:27:14.048973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:55:24.560518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":876544,"data_stored":703014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":115906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":112154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":112190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":290816,"data_stored":115906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":278528,"data_stored":112064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":286720,"data_stored":112224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":884736,"data_stored":703070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T18:27:35.677 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T18:27:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.828+0000 7f9351fd5640 1 Processor -- start 2024-05-20T18:27:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.829+0000 7f9351fd5640 1 -- start start 2024-05-20T18:27:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.830+0000 7f9351fd5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f934c071630 0x7f934c15f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.830+0000 7f9351fd5640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f934c0cd1b0 0x7f934c06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.830+0000 7f9351fd5640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f934c0ce2b0 con 0x7f934c070850 2024-05-20T18:27:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.830+0000 7f9351fd5640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f934c15f790 con 0x7f934c0cd1b0 2024-05-20T18:27:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.830+0000 7f9351fd5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f934c06e590 con 0x7f934c071630 2024-05-20T18:27:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.830+0000 7f934b7fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f934c071630 0x7f934c15f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.830+0000 7f934affd640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f934c0cd1b0 0x7f934c06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.830+0000 7f934affd640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f934c0cd1b0 0x7f934c06d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:41540/0 (socket says 172.21.15.5:41540) 2024-05-20T18:27:35.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.830+0000 7f934b7fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f934c071630 0x7f934c15f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:37162/0 (socket says 172.21.15.5:37162) 2024-05-20T18:27:35.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.830+0000 7f934affd640 1 -- 172.21.15.5:0/2185204392 learned_addr learned my addr 172.21.15.5:0/2185204392 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:35.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.831+0000 7f934b7fe640 1 -- 172.21.15.5:0/2185204392 >> v2:172.21.15.63:3300/0 conn(0x7f934c0cd1b0 msgr2=0x7f934c06d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:35.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.831+0000 7f934b7fe640 1 --2- 172.21.15.5:0/2185204392 >> v2:172.21.15.63:3300/0 conn(0x7f934c0cd1b0 0x7f934c06d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.831+0000 7f934b7fe640 1 -- 172.21.15.5:0/2185204392 >> v1:172.21.15.63:6789/0 conn(0x7f934c070850 legacy=0x7f934c070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:35.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.831+0000 7f934b7fe640 1 -- 172.21.15.5:0/2185204392 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f934c06cce0 con 0x7f934c071630 2024-05-20T18:27:35.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.831+0000 7f934affd640 1 --2- 172.21.15.5:0/2185204392 >> v2:172.21.15.63:3300/0 conn(0x7f934c0cd1b0 0x7f934c06d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:27:35.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.832+0000 7f934b7fe640 1 --2- 172.21.15.5:0/2185204392 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f934c071630 0x7f934c15f250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f933c0097a0 tx=0x7f933c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1346eab75c9c14f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:35.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.832+0000 7f934a7fc640 1 -- 172.21.15.5:0/2185204392 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933c037070 con 0x7f934c071630 2024-05-20T18:27:35.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.832+0000 7f934a7fc640 1 -- 172.21.15.5:0/2185204392 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f933c02ae50 con 0x7f934c071630 2024-05-20T18:27:35.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.832+0000 7f934a7fc640 1 -- 172.21.15.5:0/2185204392 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933c032660 con 0x7f934c071630 2024-05-20T18:27:35.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.833+0000 7f9351fd5640 1 -- 172.21.15.5:0/2185204392 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f934c071630 msgr2=0x7f934c15f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:35.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.833+0000 7f9351fd5640 1 --2- 172.21.15.5:0/2185204392 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f934c071630 0x7f934c15f250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f933c0097a0 tx=0x7f933c02a100 comp rx=0 tx=0).stop 2024-05-20T18:27:35.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.833+0000 7f9351fd5640 1 -- 172.21.15.5:0/2185204392 shutdown_connections 2024-05-20T18:27:35.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.833+0000 7f9351fd5640 1 --2- 172.21.15.5:0/2185204392 >> v2:172.21.15.63:3300/0 conn(0x7f934c0cd1b0 0x7f934c06d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.833+0000 7f9351fd5640 1 --2- 172.21.15.5:0/2185204392 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f934c071630 0x7f934c15f250 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.833+0000 7f9351fd5640 1 -- 172.21.15.5:0/2185204392 >> 172.21.15.5:0/2185204392 conn(0x7f934c0d29d0 msgr2=0x7f934c0d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:35.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.833+0000 7f9351fd5640 1 -- 172.21.15.5:0/2185204392 shutdown_connections 2024-05-20T18:27:35.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.833+0000 7f9351fd5640 1 -- 172.21.15.5:0/2185204392 wait complete. 2024-05-20T18:27:35.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.835+0000 7f9351fd5640 1 Processor -- start 2024-05-20T18:27:35.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.835+0000 7f9351fd5640 1 -- start start 2024-05-20T18:27:35.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.836+0000 7f9351fd5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f934c070850 0x7f934c1f4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:35.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.836+0000 7f9351fd5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f934c071630 0x7f934c1e5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.836+0000 7f9351fd5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f934c0cd1b0 0x7f934c0b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.836+0000 7f9351fd5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f934c06d2a0 con 0x7f934c0cd1b0 2024-05-20T18:27:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.836+0000 7f9351fd5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f934c06c990 con 0x7f934c070850 2024-05-20T18:27:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.836+0000 7f9351fd5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f934c06dd00 con 0x7f934c071630 2024-05-20T18:27:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.836+0000 7f934bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f934c0cd1b0 0x7f934c0b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.836+0000 7f934bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f934c0cd1b0 0x7f934c0b7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52914/0 (socket says 172.21.15.5:52914) 2024-05-20T18:27:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.836+0000 7f934bfff640 1 -- 172.21.15.5:0/789206999 learned_addr learned my addr 172.21.15.5:0/789206999 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.836+0000 7f934affd640 1 --2- 172.21.15.5:0/789206999 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f934c071630 0x7f934c1e5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.836+0000 7f934b7fe640 1 --2- 172.21.15.5:0/789206999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f934c070850 0x7f934c1f4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.837+0000 7f934bfff640 1 -- 172.21.15.5:0/789206999 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f934c071630 msgr2=0x7f934c1e5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.837+0000 7f934bfff640 1 --2- 172.21.15.5:0/789206999 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f934c071630 0x7f934c1e5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.837+0000 7f934bfff640 1 -- 172.21.15.5:0/789206999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f934c070850 msgr2=0x7f934c1f4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.837+0000 7f934bfff640 1 --2- 172.21.15.5:0/789206999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f934c070850 0x7f934c1f4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.837+0000 7f934bfff640 1 -- 172.21.15.5:0/789206999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f934c0b68b0 con 0x7f934c0cd1b0 2024-05-20T18:27:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.837+0000 7f934bfff640 1 --2- 172.21.15.5:0/789206999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f934c0cd1b0 0x7f934c0b7950 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f934400c680 tx=0x7f934400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.837+0000 7f9333fff640 1 -- 172.21.15.5:0/789206999 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9344003b70 con 0x7f934c0cd1b0 2024-05-20T18:27:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.837+0000 7f9333fff640 1 -- 172.21.15.5:0/789206999 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9344003d10 con 0x7f934c0cd1b0 2024-05-20T18:27:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.837+0000 7f9351fd5640 1 -- 172.21.15.5:0/789206999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f934c0b6b90 con 0x7f934c0cd1b0 2024-05-20T18:27:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.837+0000 7f9333fff640 1 -- 172.21.15.5:0/789206999 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9344005310 con 0x7f934c0cd1b0 2024-05-20T18:27:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.837+0000 7f9351fd5640 1 -- 172.21.15.5:0/789206999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f934c0b7150 con 0x7f934c0cd1b0 2024-05-20T18:27:35.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.838+0000 7f9351fd5640 1 -- 172.21.15.5:0/789206999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f934c075b00 con 0x7f934c0cd1b0 2024-05-20T18:27:35.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.839+0000 7f9333fff640 1 -- 172.21.15.5:0/789206999 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f93440054b0 con 0x7f934c0cd1b0 2024-05-20T18:27:35.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.839+0000 7f9333fff640 1 --2- 172.21.15.5:0/789206999 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f933806be30 0x7f933806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:35.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.839+0000 7f9333fff640 1 -- 172.21.15.5:0/789206999 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f934408aef0 con 0x7f934c0cd1b0 2024-05-20T18:27:35.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.839+0000 7f934b7fe640 1 --2- 172.21.15.5:0/789206999 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f933806be30 0x7f933806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:35.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.840+0000 7f934b7fe640 1 --2- 172.21.15.5:0/789206999 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f933806be30 0x7f933806e340 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f933c034000 tx=0x7f933c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:35.844+0000 7f9333fff640 1 -- 172.21.15.5:0/789206999 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9344058e80 con 0x7f934c0cd1b0 2024-05-20T18:27:36.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.025+0000 7f9351fd5640 1 -- 172.21.15.5:0/789206999 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f934c0b6660 con 0x7f933806be30 2024-05-20T18:27:36.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.033+0000 7f9333fff640 1 -- 172.21.15.5:0/789206999 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332533 (secure 0 0 0) 0x7f933c093310 con 0x7f933806be30 2024-05-20T18:27:36.036 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:36.038 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-20T18:27:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.037+0000 7f9351fd5640 1 -- 172.21.15.5:0/789206999 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f933806be30 msgr2=0x7f933806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.037+0000 7f9351fd5640 1 --2- 172.21.15.5:0/789206999 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f933806be30 0x7f933806e340 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f933c034000 tx=0x7f933c02e000 comp rx=0 tx=0).stop 2024-05-20T18:27:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.037+0000 7f9351fd5640 1 -- 172.21.15.5:0/789206999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f934c0cd1b0 msgr2=0x7f934c0b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.037+0000 7f9351fd5640 1 --2- 172.21.15.5:0/789206999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f934c0cd1b0 0x7f934c0b7950 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f934400c680 tx=0x7f934400cb40 comp rx=0 tx=0).stop 2024-05-20T18:27:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.037+0000 7f9351fd5640 1 -- 172.21.15.5:0/789206999 shutdown_connections 2024-05-20T18:27:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.037+0000 7f9351fd5640 1 --2- 172.21.15.5:0/789206999 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f933806be30 0x7f933806e340 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.037+0000 7f9351fd5640 1 --2- 172.21.15.5:0/789206999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f934c0cd1b0 0x7f934c0b7950 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.037+0000 7f9351fd5640 1 --2- 172.21.15.5:0/789206999 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f934c071630 0x7f934c1e5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.037+0000 7f9351fd5640 1 --2- 172.21.15.5:0/789206999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f934c070850 0x7f934c1f4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.037+0000 7f9351fd5640 1 -- 172.21.15.5:0/789206999 >> 172.21.15.5:0/789206999 conn(0x7f934c0d29d0 msgr2=0x7f934c0d0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.037+0000 7f9351fd5640 1 -- 172.21.15.5:0/789206999 shutdown_connections 2024-05-20T18:27:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.037+0000 7f9351fd5640 1 -- 172.21.15.5:0/789206999 wait complete. 2024-05-20T18:27:36.048 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-05-20T18:27:34.580839+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236216,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738504,"statfs":{"total":773094113280,"available":772852228096,"internally_reserved":0,"allocated":3481600,"data_stored":2086528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004641"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048716+0000","last_change":"2024-05-20T18:27:06.794819+0000","last_active":"2024-05-20T18:27:14.048716+0000","last_peered":"2024-05-20T18:27:14.048716+0000","last_clean":"2024-05-20T18:27:14.048716+0000","last_became_active":"2024-05-20T18:27:06.794505+0000","last_became_peered":"2024-05-20T18:27:06.794505+0000","last_unstale":"2024-05-20T18:27:14.048716+0000","last_undegraded":"2024-05-20T18:27:14.048716+0000","last_fullsized":"2024-05-20T18:27:14.048716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:20:52.095853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821927+0000","last_change":"2024-05-20T18:27:07.802838+0000","last_active":"2024-05-20T18:27:13.821927+0000","last_peered":"2024-05-20T18:27:13.821927+0000","last_clean":"2024-05-20T18:27:13.821927+0000","last_became_active":"2024-05-20T18:27:07.802468+0000","last_became_peered":"2024-05-20T18:27:07.802468+0000","last_unstale":"2024-05-20T18:27:13.821927+0000","last_undegraded":"2024-05-20T18:27:13.821927+0000","last_fullsized":"2024-05-20T18:27:13.821927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:12.006827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821103+0000","last_change":"2024-05-20T18:27:12.469613+0000","last_active":"2024-05-20T18:27:13.821103+0000","last_peered":"2024-05-20T18:27:13.821103+0000","last_clean":"2024-05-20T18:27:13.821103+0000","last_became_active":"2024-05-20T18:27:12.469161+0000","last_became_peered":"2024-05-20T18:27:12.469161+0000","last_unstale":"2024-05-20T18:27:13.821103+0000","last_undegraded":"2024-05-20T18:27:13.821103+0000","last_fullsized":"2024-05-20T18:27:13.821103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:32:50.234917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.822083+0000","last_change":"2024-05-20T18:27:06.799063+0000","last_active":"2024-05-20T18:27:13.822083+0000","last_peered":"2024-05-20T18:27:13.822083+0000","last_clean":"2024-05-20T18:27:13.822083+0000","last_became_active":"2024-05-20T18:27:06.798689+0000","last_became_peered":"2024-05-20T18:27:06.798689+0000","last_unstale":"2024-05-20T18:27:13.822083+0000","last_undegraded":"2024-05-20T18:27:13.822083+0000","last_fullsized":"2024-05-20T18:27:13.822083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:52:44.943426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048397+0000","last_change":"2024-05-20T18:27:07.805002+0000","last_active":"2024-05-20T18:27:14.048397+0000","last_peered":"2024-05-20T18:27:14.048397+0000","last_clean":"2024-05-20T18:27:14.048397+0000","last_became_active":"2024-05-20T18:27:07.804655+0000","last_became_peered":"2024-05-20T18:27:07.804655+0000","last_unstale":"2024-05-20T18:27:14.048397+0000","last_undegraded":"2024-05-20T18:27:14.048397+0000","last_fullsized":"2024-05-20T18:27:14.048397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:52:57.891709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047788+0000","last_change":"2024-05-20T18:27:11.833803+0000","last_active":"2024-05-20T18:27:14.047788+0000","last_peered":"2024-05-20T18:27:14.047788+0000","last_clean":"2024-05-20T18:27:14.047788+0000","last_became_active":"2024-05-20T18:27:11.833241+0000","last_became_peered":"2024-05-20T18:27:11.833241+0000","last_unstale":"2024-05-20T18:27:14.047788+0000","last_undegraded":"2024-05-20T18:27:14.047788+0000","last_fullsized":"2024-05-20T18:27:14.047788+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:25:03.246597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049511+0000","last_change":"2024-05-20T18:27:06.806008+0000","last_active":"2024-05-20T18:27:14.049511+0000","last_peered":"2024-05-20T18:27:14.049511+0000","last_clean":"2024-05-20T18:27:14.049511+0000","last_became_active":"2024-05-20T18:27:06.805632+0000","last_became_peered":"2024-05-20T18:27:06.805632+0000","last_unstale":"2024-05-20T18:27:14.049511+0000","last_undegraded":"2024-05-20T18:27:14.049511+0000","last_fullsized":"2024-05-20T18:27:14.049511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:11:07.831277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825124+0000","last_change":"2024-05-20T18:27:07.812390+0000","last_active":"2024-05-20T18:27:13.825124+0000","last_peered":"2024-05-20T18:27:13.825124+0000","last_clean":"2024-05-20T18:27:13.825124+0000","last_became_active":"2024-05-20T18:27:07.811667+0000","last_became_peered":"2024-05-20T18:27:07.811667+0000","last_unstale":"2024-05-20T18:27:13.825124+0000","last_undegraded":"2024-05-20T18:27:13.825124+0000","last_fullsized":"2024-05-20T18:27:13.825124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:29:33.478403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048201+0000","last_change":"2024-05-20T18:27:11.833494+0000","last_active":"2024-05-20T18:27:14.048201+0000","last_peered":"2024-05-20T18:27:14.048201+0000","last_clean":"2024-05-20T18:27:14.048201+0000","last_became_active":"2024-05-20T18:27:11.832999+0000","last_became_peered":"2024-05-20T18:27:11.832999+0000","last_unstale":"2024-05-20T18:27:14.048201+0000","last_undegraded":"2024-05-20T18:27:14.048201+0000","last_fullsized":"2024-05-20T18:27:14.048201+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:08:42.465927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049057+0000","last_change":"2024-05-20T18:27:06.798023+0000","last_active":"2024-05-20T18:27:14.049057+0000","last_peered":"2024-05-20T18:27:14.049057+0000","last_clean":"2024-05-20T18:27:14.049057+0000","last_became_active":"2024-05-20T18:27:06.797674+0000","last_became_peered":"2024-05-20T18:27:06.797674+0000","last_unstale":"2024-05-20T18:27:14.049057+0000","last_undegraded":"2024-05-20T18:27:14.049057+0000","last_fullsized":"2024-05-20T18:27:14.049057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:43:31.154358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824776+0000","last_change":"2024-05-20T18:27:07.811508+0000","last_active":"2024-05-20T18:27:13.824776+0000","last_peered":"2024-05-20T18:27:13.824776+0000","last_clean":"2024-05-20T18:27:13.824776+0000","last_became_active":"2024-05-20T18:27:07.811217+0000","last_became_peered":"2024-05-20T18:27:07.811217+0000","last_unstale":"2024-05-20T18:27:13.824776+0000","last_undegraded":"2024-05-20T18:27:13.824776+0000","last_fullsized":"2024-05-20T18:27:13.824776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:01:46.623171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047874+0000","last_change":"2024-05-20T18:27:11.827280+0000","last_active":"2024-05-20T18:27:14.047874+0000","last_peered":"2024-05-20T18:27:14.047874+0000","last_clean":"2024-05-20T18:27:14.047874+0000","last_became_active":"2024-05-20T18:27:11.826770+0000","last_became_peered":"2024-05-20T18:27:11.826770+0000","last_unstale":"2024-05-20T18:27:14.047874+0000","last_undegraded":"2024-05-20T18:27:14.047874+0000","last_fullsized":"2024-05-20T18:27:14.047874+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:16:41.968388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825487+0000","last_change":"2024-05-20T18:27:07.791795+0000","last_active":"2024-05-20T18:27:13.825487+0000","last_peered":"2024-05-20T18:27:13.825487+0000","last_clean":"2024-05-20T18:27:13.825487+0000","last_became_active":"2024-05-20T18:27:07.791422+0000","last_became_peered":"2024-05-20T18:27:07.791422+0000","last_unstale":"2024-05-20T18:27:13.825487+0000","last_undegraded":"2024-05-20T18:27:13.825487+0000","last_fullsized":"2024-05-20T18:27:13.825487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:40:08.698691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049335+0000","last_change":"2024-05-20T18:27:06.799475+0000","last_active":"2024-05-20T18:27:14.049335+0000","last_peered":"2024-05-20T18:27:14.049335+0000","last_clean":"2024-05-20T18:27:14.049335+0000","last_became_active":"2024-05-20T18:27:06.798540+0000","last_became_peered":"2024-05-20T18:27:06.798540+0000","last_unstale":"2024-05-20T18:27:14.049335+0000","last_undegraded":"2024-05-20T18:27:14.049335+0000","last_fullsized":"2024-05-20T18:27:14.049335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:14:22.040824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048392+0000","last_change":"2024-05-20T18:27:11.826856+0000","last_active":"2024-05-20T18:27:14.048392+0000","last_peered":"2024-05-20T18:27:14.048392+0000","last_clean":"2024-05-20T18:27:14.048392+0000","last_became_active":"2024-05-20T18:27:11.826205+0000","last_became_peered":"2024-05-20T18:27:11.826205+0000","last_unstale":"2024-05-20T18:27:14.048392+0000","last_undegraded":"2024-05-20T18:27:14.048392+0000","last_fullsized":"2024-05-20T18:27:14.048392+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:34:21.648495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821544+0000","last_change":"2024-05-20T18:27:07.811973+0000","last_active":"2024-05-20T18:27:13.821544+0000","last_peered":"2024-05-20T18:27:13.821544+0000","last_clean":"2024-05-20T18:27:13.821544+0000","last_became_active":"2024-05-20T18:27:07.811617+0000","last_became_peered":"2024-05-20T18:27:07.811617+0000","last_unstale":"2024-05-20T18:27:13.821544+0000","last_undegraded":"2024-05-20T18:27:13.821544+0000","last_fullsized":"2024-05-20T18:27:13.821544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:52:22.903884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048965+0000","last_change":"2024-05-20T18:27:06.792524+0000","last_active":"2024-05-20T18:27:14.048965+0000","last_peered":"2024-05-20T18:27:14.048965+0000","last_clean":"2024-05-20T18:27:14.048965+0000","last_became_active":"2024-05-20T18:27:06.792104+0000","last_became_peered":"2024-05-20T18:27:06.792104+0000","last_unstale":"2024-05-20T18:27:14.048965+0000","last_undegraded":"2024-05-20T18:27:14.048965+0000","last_fullsized":"2024-05-20T18:27:14.048965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:10:27.276400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048408+0000","last_change":"2024-05-20T18:27:11.833928+0000","last_active":"2024-05-20T18:27:14.048408+0000","last_peered":"2024-05-20T18:27:14.048408+0000","last_clean":"2024-05-20T18:27:14.048408+0000","last_became_active":"2024-05-20T18:27:11.832887+0000","last_became_peered":"2024-05-20T18:27:11.832887+0000","last_unstale":"2024-05-20T18:27:14.048408+0000","last_undegraded":"2024-05-20T18:27:14.048408+0000","last_fullsized":"2024-05-20T18:27:14.048408+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:38:58.503399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821356+0000","last_change":"2024-05-20T18:27:07.788711+0000","last_active":"2024-05-20T18:27:13.821356+0000","last_peered":"2024-05-20T18:27:13.821356+0000","last_clean":"2024-05-20T18:27:13.821356+0000","last_became_active":"2024-05-20T18:27:07.788402+0000","last_became_peered":"2024-05-20T18:27:07.788402+0000","last_unstale":"2024-05-20T18:27:13.821356+0000","last_undegraded":"2024-05-20T18:27:13.821356+0000","last_fullsized":"2024-05-20T18:27:13.821356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:34:15.614124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048633+0000","last_change":"2024-05-20T18:27:06.803611+0000","last_active":"2024-05-20T18:27:14.048633+0000","last_peered":"2024-05-20T18:27:14.048633+0000","last_clean":"2024-05-20T18:27:14.048633+0000","last_became_active":"2024-05-20T18:27:06.803176+0000","last_became_peered":"2024-05-20T18:27:06.803176+0000","last_unstale":"2024-05-20T18:27:14.048633+0000","last_undegraded":"2024-05-20T18:27:14.048633+0000","last_fullsized":"2024-05-20T18:27:14.048633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:52:44.943426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049451+0000","last_change":"2024-05-20T18:27:12.466975+0000","last_active":"2024-05-20T18:27:14.049451+0000","last_peered":"2024-05-20T18:27:14.049451+0000","last_clean":"2024-05-20T18:27:14.049451+0000","last_became_active":"2024-05-20T18:27:12.466473+0000","last_became_peered":"2024-05-20T18:27:12.466473+0000","last_unstale":"2024-05-20T18:27:14.049451+0000","last_undegraded":"2024-05-20T18:27:14.049451+0000","last_fullsized":"2024-05-20T18:27:14.049451+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:55:23.896898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048689+0000","last_change":"2024-05-20T18:27:07.786091+0000","last_active":"2024-05-20T18:27:14.048689+0000","last_peered":"2024-05-20T18:27:14.048689+0000","last_clean":"2024-05-20T18:27:14.048689+0000","last_became_active":"2024-05-20T18:27:07.785704+0000","last_became_peered":"2024-05-20T18:27:07.785704+0000","last_unstale":"2024-05-20T18:27:14.048689+0000","last_undegraded":"2024-05-20T18:27:14.048689+0000","last_fullsized":"2024-05-20T18:27:14.048689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:06:32.450870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047619+0000","last_change":"2024-05-20T18:27:06.799668+0000","last_active":"2024-05-20T18:27:14.047619+0000","last_peered":"2024-05-20T18:27:14.047619+0000","last_clean":"2024-05-20T18:27:14.047619+0000","last_became_active":"2024-05-20T18:27:06.799002+0000","last_became_peered":"2024-05-20T18:27:06.799002+0000","last_unstale":"2024-05-20T18:27:14.047619+0000","last_undegraded":"2024-05-20T18:27:14.047619+0000","last_fullsized":"2024-05-20T18:27:14.047619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:06:31.445366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048146+0000","last_change":"2024-05-20T18:27:11.834133+0000","last_active":"2024-05-20T18:27:14.048146+0000","last_peered":"2024-05-20T18:27:14.048146+0000","last_clean":"2024-05-20T18:27:14.048146+0000","last_became_active":"2024-05-20T18:27:11.832891+0000","last_became_peered":"2024-05-20T18:27:11.832891+0000","last_unstale":"2024-05-20T18:27:14.048146+0000","last_undegraded":"2024-05-20T18:27:14.048146+0000","last_fullsized":"2024-05-20T18:27:14.048146+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:55:36.987878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049426+0000","last_change":"2024-05-20T18:27:07.790534+0000","last_active":"2024-05-20T18:27:14.049426+0000","last_peered":"2024-05-20T18:27:14.049426+0000","last_clean":"2024-05-20T18:27:14.049426+0000","last_became_active":"2024-05-20T18:27:07.790192+0000","last_became_peered":"2024-05-20T18:27:07.790192+0000","last_unstale":"2024-05-20T18:27:14.049426+0000","last_undegraded":"2024-05-20T18:27:14.049426+0000","last_fullsized":"2024-05-20T18:27:14.049426+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:12.006827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.826184+0000","last_change":"2024-05-20T18:27:06.794221+0000","last_active":"2024-05-20T18:27:13.826184+0000","last_peered":"2024-05-20T18:27:13.826184+0000","last_clean":"2024-05-20T18:27:13.826184+0000","last_became_active":"2024-05-20T18:27:06.793895+0000","last_became_peered":"2024-05-20T18:27:06.793895+0000","last_unstale":"2024-05-20T18:27:13.826184+0000","last_undegraded":"2024-05-20T18:27:13.826184+0000","last_fullsized":"2024-05-20T18:27:13.826184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:24:55.001924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048453+0000","last_change":"2024-05-20T18:27:12.471660+0000","last_active":"2024-05-20T18:27:14.048453+0000","last_peered":"2024-05-20T18:27:14.048453+0000","last_clean":"2024-05-20T18:27:14.048453+0000","last_became_active":"2024-05-20T18:27:12.470460+0000","last_became_peered":"2024-05-20T18:27:12.470460+0000","last_unstale":"2024-05-20T18:27:14.048453+0000","last_undegraded":"2024-05-20T18:27:14.048453+0000","last_fullsized":"2024-05-20T18:27:14.048453+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:53:56.893940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.822216+0000","last_change":"2024-05-20T18:27:07.799399+0000","last_active":"2024-05-20T18:27:13.822216+0000","last_peered":"2024-05-20T18:27:13.822216+0000","last_clean":"2024-05-20T18:27:13.822216+0000","last_became_active":"2024-05-20T18:27:07.799017+0000","last_became_peered":"2024-05-20T18:27:07.799017+0000","last_unstale":"2024-05-20T18:27:13.822216+0000","last_undegraded":"2024-05-20T18:27:13.822216+0000","last_fullsized":"2024-05-20T18:27:13.822216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:08:51.463251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049547+0000","last_change":"2024-05-20T18:27:06.794477+0000","last_active":"2024-05-20T18:27:14.049547+0000","last_peered":"2024-05-20T18:27:14.049547+0000","last_clean":"2024-05-20T18:27:14.049547+0000","last_became_active":"2024-05-20T18:27:06.794130+0000","last_became_peered":"2024-05-20T18:27:06.794130+0000","last_unstale":"2024-05-20T18:27:14.049547+0000","last_undegraded":"2024-05-20T18:27:14.049547+0000","last_fullsized":"2024-05-20T18:27:14.049547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:43:31.154358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821030+0000","last_change":"2024-05-20T18:27:12.468521+0000","last_active":"2024-05-20T18:27:13.821030+0000","last_peered":"2024-05-20T18:27:13.821030+0000","last_clean":"2024-05-20T18:27:13.821030+0000","last_became_active":"2024-05-20T18:27:12.467880+0000","last_became_peered":"2024-05-20T18:27:12.467880+0000","last_unstale":"2024-05-20T18:27:13.821030+0000","last_undegraded":"2024-05-20T18:27:13.821030+0000","last_fullsized":"2024-05-20T18:27:13.821030+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:48:16.180009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048938+0000","last_change":"2024-05-20T18:27:07.784489+0000","last_active":"2024-05-20T18:27:14.048938+0000","last_peered":"2024-05-20T18:27:14.048938+0000","last_clean":"2024-05-20T18:27:14.048938+0000","last_became_active":"2024-05-20T18:27:07.783968+0000","last_became_peered":"2024-05-20T18:27:07.783968+0000","last_unstale":"2024-05-20T18:27:14.048938+0000","last_undegraded":"2024-05-20T18:27:14.048938+0000","last_fullsized":"2024-05-20T18:27:14.048938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:07:15.764912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049643+0000","last_change":"2024-05-20T18:27:06.805626+0000","last_active":"2024-05-20T18:27:14.049643+0000","last_peered":"2024-05-20T18:27:14.049643+0000","last_clean":"2024-05-20T18:27:14.049643+0000","last_became_active":"2024-05-20T18:27:06.805267+0000","last_became_peered":"2024-05-20T18:27:06.805267+0000","last_unstale":"2024-05-20T18:27:14.049643+0000","last_undegraded":"2024-05-20T18:27:14.049643+0000","last_fullsized":"2024-05-20T18:27:14.049643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:18:18.528418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048076+0000","last_change":"2024-05-20T18:27:12.465690+0000","last_active":"2024-05-20T18:27:14.048076+0000","last_peered":"2024-05-20T18:27:14.048076+0000","last_clean":"2024-05-20T18:27:14.048076+0000","last_became_active":"2024-05-20T18:27:12.465196+0000","last_became_peered":"2024-05-20T18:27:12.465196+0000","last_unstale":"2024-05-20T18:27:14.048076+0000","last_undegraded":"2024-05-20T18:27:14.048076+0000","last_fullsized":"2024-05-20T18:27:14.048076+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:14:31.209452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825227+0000","last_change":"2024-05-20T18:27:07.806673+0000","last_active":"2024-05-20T18:27:13.825227+0000","last_peered":"2024-05-20T18:27:13.825227+0000","last_clean":"2024-05-20T18:27:13.825227+0000","last_became_active":"2024-05-20T18:27:07.805764+0000","last_became_peered":"2024-05-20T18:27:07.805764+0000","last_unstale":"2024-05-20T18:27:13.825227+0000","last_undegraded":"2024-05-20T18:27:13.825227+0000","last_fullsized":"2024-05-20T18:27:13.825227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:16:37.938695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049063+0000","last_change":"2024-05-20T18:27:06.796398+0000","last_active":"2024-05-20T18:27:14.049063+0000","last_peered":"2024-05-20T18:27:14.049063+0000","last_clean":"2024-05-20T18:27:14.049063+0000","last_became_active":"2024-05-20T18:27:06.795981+0000","last_became_peered":"2024-05-20T18:27:06.795981+0000","last_unstale":"2024-05-20T18:27:14.049063+0000","last_undegraded":"2024-05-20T18:27:14.049063+0000","last_fullsized":"2024-05-20T18:27:14.049063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:11:07.831277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048435+0000","last_change":"2024-05-20T18:27:11.830530+0000","last_active":"2024-05-20T18:27:14.048435+0000","last_peered":"2024-05-20T18:27:14.048435+0000","last_clean":"2024-05-20T18:27:14.048435+0000","last_became_active":"2024-05-20T18:27:11.828707+0000","last_became_peered":"2024-05-20T18:27:11.828707+0000","last_unstale":"2024-05-20T18:27:14.048435+0000","last_undegraded":"2024-05-20T18:27:14.048435+0000","last_fullsized":"2024-05-20T18:27:14.048435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:14:31.209452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048613+0000","last_change":"2024-05-20T18:27:07.802587+0000","last_active":"2024-05-20T18:27:14.048613+0000","last_peered":"2024-05-20T18:27:14.048613+0000","last_clean":"2024-05-20T18:27:14.048613+0000","last_became_active":"2024-05-20T18:27:07.802284+0000","last_became_peered":"2024-05-20T18:27:07.802284+0000","last_unstale":"2024-05-20T18:27:14.048613+0000","last_undegraded":"2024-05-20T18:27:14.048613+0000","last_fullsized":"2024-05-20T18:27:14.048613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:18:19.533923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825123+0000","last_change":"2024-05-20T18:27:06.795278+0000","last_active":"2024-05-20T18:27:13.825123+0000","last_peered":"2024-05-20T18:27:13.825123+0000","last_clean":"2024-05-20T18:27:13.825123+0000","last_became_active":"2024-05-20T18:27:06.794895+0000","last_became_peered":"2024-05-20T18:27:06.794895+0000","last_unstale":"2024-05-20T18:27:13.825123+0000","last_undegraded":"2024-05-20T18:27:13.825123+0000","last_fullsized":"2024-05-20T18:27:13.825123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:52:44.943426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048956+0000","last_change":"2024-05-20T18:27:11.833463+0000","last_active":"2024-05-20T18:27:14.048956+0000","last_peered":"2024-05-20T18:27:14.048956+0000","last_clean":"2024-05-20T18:27:14.048956+0000","last_became_active":"2024-05-20T18:27:11.832954+0000","last_became_peered":"2024-05-20T18:27:11.832954+0000","last_unstale":"2024-05-20T18:27:14.048956+0000","last_undegraded":"2024-05-20T18:27:14.048956+0000","last_fullsized":"2024-05-20T18:27:14.048956+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:34:21.648495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048483+0000","last_change":"2024-05-20T18:27:07.785038+0000","last_active":"2024-05-20T18:27:14.048483+0000","last_peered":"2024-05-20T18:27:14.048483+0000","last_clean":"2024-05-20T18:27:14.048483+0000","last_became_active":"2024-05-20T18:27:07.783927+0000","last_became_peered":"2024-05-20T18:27:07.783927+0000","last_unstale":"2024-05-20T18:27:14.048483+0000","last_undegraded":"2024-05-20T18:27:14.048483+0000","last_fullsized":"2024-05-20T18:27:14.048483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:12.006827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047552+0000","last_change":"2024-05-20T18:27:06.789785+0000","last_active":"2024-05-20T18:27:14.047552+0000","last_peered":"2024-05-20T18:27:14.047552+0000","last_clean":"2024-05-20T18:27:14.047552+0000","last_became_active":"2024-05-20T18:27:06.789421+0000","last_became_peered":"2024-05-20T18:27:06.789421+0000","last_unstale":"2024-05-20T18:27:14.047552+0000","last_undegraded":"2024-05-20T18:27:14.047552+0000","last_fullsized":"2024-05-20T18:27:14.047552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:24:55.001924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049327+0000","last_change":"2024-05-20T18:27:11.833707+0000","last_active":"2024-05-20T18:27:14.049327+0000","last_peered":"2024-05-20T18:27:14.049327+0000","last_clean":"2024-05-20T18:27:14.049327+0000","last_became_active":"2024-05-20T18:27:11.832619+0000","last_became_peered":"2024-05-20T18:27:11.832619+0000","last_unstale":"2024-05-20T18:27:14.049327+0000","last_undegraded":"2024-05-20T18:27:14.049327+0000","last_fullsized":"2024-05-20T18:27:14.049327+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:53:56.893940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047552+0000","last_change":"2024-05-20T18:27:07.799471+0000","last_active":"2024-05-20T18:27:14.047552+0000","last_peered":"2024-05-20T18:27:14.047552+0000","last_clean":"2024-05-20T18:27:14.047552+0000","last_became_active":"2024-05-20T18:27:07.799089+0000","last_became_peered":"2024-05-20T18:27:07.799089+0000","last_unstale":"2024-05-20T18:27:14.047552+0000","last_undegraded":"2024-05-20T18:27:14.047552+0000","last_fullsized":"2024-05-20T18:27:14.047552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:39:20.381356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047626+0000","last_change":"2024-05-20T18:27:06.799374+0000","last_active":"2024-05-20T18:27:14.047626+0000","last_peered":"2024-05-20T18:27:14.047626+0000","last_clean":"2024-05-20T18:27:14.047626+0000","last_became_active":"2024-05-20T18:27:06.798552+0000","last_became_peered":"2024-05-20T18:27:06.798552+0000","last_unstale":"2024-05-20T18:27:14.047626+0000","last_undegraded":"2024-05-20T18:27:14.047626+0000","last_fullsized":"2024-05-20T18:27:14.047626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:52:44.943426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049339+0000","last_change":"2024-05-20T18:27:12.469418+0000","last_active":"2024-05-20T18:27:14.049339+0000","last_peered":"2024-05-20T18:27:14.049339+0000","last_clean":"2024-05-20T18:27:14.049339+0000","last_became_active":"2024-05-20T18:27:12.468923+0000","last_became_peered":"2024-05-20T18:27:12.468923+0000","last_unstale":"2024-05-20T18:27:14.049339+0000","last_undegraded":"2024-05-20T18:27:14.049339+0000","last_fullsized":"2024-05-20T18:27:14.049339+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:55:23.896898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821455+0000","last_change":"2024-05-20T18:27:07.804772+0000","last_active":"2024-05-20T18:27:13.821455+0000","last_peered":"2024-05-20T18:27:13.821455+0000","last_clean":"2024-05-20T18:27:13.821455+0000","last_became_active":"2024-05-20T18:27:07.804443+0000","last_became_peered":"2024-05-20T18:27:07.804443+0000","last_unstale":"2024-05-20T18:27:13.821455+0000","last_undegraded":"2024-05-20T18:27:13.821455+0000","last_fullsized":"2024-05-20T18:27:13.821455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:14:17.791789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824615+0000","last_change":"2024-05-20T18:27:06.790984+0000","last_active":"2024-05-20T18:27:13.824615+0000","last_peered":"2024-05-20T18:27:13.824615+0000","last_clean":"2024-05-20T18:27:13.824615+0000","last_became_active":"2024-05-20T18:27:06.790620+0000","last_became_peered":"2024-05-20T18:27:06.790620+0000","last_unstale":"2024-05-20T18:27:13.824615+0000","last_undegraded":"2024-05-20T18:27:13.824615+0000","last_fullsized":"2024-05-20T18:27:13.824615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:10:27.276400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":179,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047791+0000","last_change":"2024-05-20T18:27:03.765694+0000","last_active":"2024-05-20T18:27:14.047791+0000","last_peered":"2024-05-20T18:27:14.047791+0000","last_clean":"2024-05-20T18:27:14.047791+0000","last_became_active":"2024-05-20T18:27:03.764695+0000","last_became_peered":"2024-05-20T18:27:03.764695+0000","last_unstale":"2024-05-20T18:27:14.047791+0000","last_undegraded":"2024-05-20T18:27:14.047791+0000","last_fullsized":"2024-05-20T18:27:14.047791+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_clean_scrub_stamp":"2024-05-20T18:27:02.750853+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:30:04.407487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824624+0000","last_change":"2024-05-20T18:27:12.467983+0000","last_active":"2024-05-20T18:27:13.824624+0000","last_peered":"2024-05-20T18:27:13.824624+0000","last_clean":"2024-05-20T18:27:13.824624+0000","last_became_active":"2024-05-20T18:27:12.467420+0000","last_became_peered":"2024-05-20T18:27:12.467420+0000","last_unstale":"2024-05-20T18:27:13.824624+0000","last_undegraded":"2024-05-20T18:27:13.824624+0000","last_fullsized":"2024-05-20T18:27:13.824624+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:01:50.652864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049235+0000","last_change":"2024-05-20T18:27:07.814589+0000","last_active":"2024-05-20T18:27:14.049235+0000","last_peered":"2024-05-20T18:27:14.049235+0000","last_clean":"2024-05-20T18:27:14.049235+0000","last_became_active":"2024-05-20T18:27:07.814281+0000","last_became_peered":"2024-05-20T18:27:07.814281+0000","last_unstale":"2024-05-20T18:27:14.049235+0000","last_undegraded":"2024-05-20T18:27:14.049235+0000","last_fullsized":"2024-05-20T18:27:14.049235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:29:33.478403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047483+0000","last_change":"2024-05-20T18:27:06.798075+0000","last_active":"2024-05-20T18:27:14.047483+0000","last_peered":"2024-05-20T18:27:14.047483+0000","last_clean":"2024-05-20T18:27:14.047483+0000","last_became_active":"2024-05-20T18:27:06.797696+0000","last_became_peered":"2024-05-20T18:27:06.797696+0000","last_unstale":"2024-05-20T18:27:14.047483+0000","last_undegraded":"2024-05-20T18:27:14.047483+0000","last_fullsized":"2024-05-20T18:27:14.047483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:06:31.445366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825521+0000","last_change":"2024-05-20T18:27:12.465857+0000","last_active":"2024-05-20T18:27:13.825521+0000","last_peered":"2024-05-20T18:27:13.825521+0000","last_clean":"2024-05-20T18:27:13.825521+0000","last_became_active":"2024-05-20T18:27:12.465296+0000","last_became_peered":"2024-05-20T18:27:12.465296+0000","last_unstale":"2024-05-20T18:27:13.825521+0000","last_undegraded":"2024-05-20T18:27:13.825521+0000","last_fullsized":"2024-05-20T18:27:13.825521+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:25:03.246597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825312+0000","last_change":"2024-05-20T18:27:07.808534+0000","last_active":"2024-05-20T18:27:13.825312+0000","last_peered":"2024-05-20T18:27:13.825312+0000","last_clean":"2024-05-20T18:27:13.825312+0000","last_became_active":"2024-05-20T18:27:07.807813+0000","last_became_peered":"2024-05-20T18:27:07.807813+0000","last_unstale":"2024-05-20T18:27:13.825312+0000","last_undegraded":"2024-05-20T18:27:13.825312+0000","last_fullsized":"2024-05-20T18:27:13.825312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:08:51.463251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049093+0000","last_change":"2024-05-20T18:27:06.793443+0000","last_active":"2024-05-20T18:27:14.049093+0000","last_peered":"2024-05-20T18:27:14.049093+0000","last_clean":"2024-05-20T18:27:14.049093+0000","last_became_active":"2024-05-20T18:27:06.793037+0000","last_became_peered":"2024-05-20T18:27:06.793037+0000","last_unstale":"2024-05-20T18:27:14.049093+0000","last_undegraded":"2024-05-20T18:27:14.049093+0000","last_fullsized":"2024-05-20T18:27:14.049093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:06:31.445366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048364+0000","last_change":"2024-05-20T18:27:11.828038+0000","last_active":"2024-05-20T18:27:14.048364+0000","last_peered":"2024-05-20T18:27:14.048364+0000","last_clean":"2024-05-20T18:27:14.048364+0000","last_became_active":"2024-05-20T18:27:11.827482+0000","last_became_peered":"2024-05-20T18:27:11.827482+0000","last_unstale":"2024-05-20T18:27:14.048364+0000","last_undegraded":"2024-05-20T18:27:14.048364+0000","last_fullsized":"2024-05-20T18:27:14.048364+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:38:58.503399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049301+0000","last_change":"2024-05-20T18:27:07.809779+0000","last_active":"2024-05-20T18:27:14.049301+0000","last_peered":"2024-05-20T18:27:14.049301+0000","last_clean":"2024-05-20T18:27:14.049301+0000","last_became_active":"2024-05-20T18:27:07.809458+0000","last_became_peered":"2024-05-20T18:27:07.809458+0000","last_unstale":"2024-05-20T18:27:14.049301+0000","last_undegraded":"2024-05-20T18:27:14.049301+0000","last_fullsized":"2024-05-20T18:27:14.049301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:55:32.958186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047310+0000","last_change":"2024-05-20T18:27:06.788828+0000","last_active":"2024-05-20T18:27:14.047310+0000","last_peered":"2024-05-20T18:27:14.047310+0000","last_clean":"2024-05-20T18:27:14.047310+0000","last_became_active":"2024-05-20T18:27:06.788142+0000","last_became_peered":"2024-05-20T18:27:06.788142+0000","last_unstale":"2024-05-20T18:27:14.047310+0000","last_undegraded":"2024-05-20T18:27:14.047310+0000","last_fullsized":"2024-05-20T18:27:14.047310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:52:44.943426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.823921+0000","last_change":"2024-05-20T18:27:12.466818+0000","last_active":"2024-05-20T18:27:13.823921+0000","last_peered":"2024-05-20T18:27:13.823921+0000","last_clean":"2024-05-20T18:27:13.823921+0000","last_became_active":"2024-05-20T18:27:12.466228+0000","last_became_peered":"2024-05-20T18:27:12.466228+0000","last_unstale":"2024-05-20T18:27:13.823921+0000","last_undegraded":"2024-05-20T18:27:13.823921+0000","last_fullsized":"2024-05-20T18:27:13.823921+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:55:23.896898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049485+0000","last_change":"2024-05-20T18:27:07.808881+0000","last_active":"2024-05-20T18:27:14.049485+0000","last_peered":"2024-05-20T18:27:14.049485+0000","last_clean":"2024-05-20T18:27:14.049485+0000","last_became_active":"2024-05-20T18:27:07.808550+0000","last_became_peered":"2024-05-20T18:27:07.808550+0000","last_unstale":"2024-05-20T18:27:14.049485+0000","last_undegraded":"2024-05-20T18:27:14.049485+0000","last_fullsized":"2024-05-20T18:27:14.049485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:01:46.623171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048816+0000","last_change":"2024-05-20T18:27:06.797126+0000","last_active":"2024-05-20T18:27:14.048816+0000","last_peered":"2024-05-20T18:27:14.048816+0000","last_clean":"2024-05-20T18:27:14.048816+0000","last_became_active":"2024-05-20T18:27:06.796764+0000","last_became_peered":"2024-05-20T18:27:06.796764+0000","last_unstale":"2024-05-20T18:27:14.048816+0000","last_undegraded":"2024-05-20T18:27:14.048816+0000","last_fullsized":"2024-05-20T18:27:14.048816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:59:18.905646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047675+0000","last_change":"2024-05-20T18:27:11.827350+0000","last_active":"2024-05-20T18:27:14.047675+0000","last_peered":"2024-05-20T18:27:14.047675+0000","last_clean":"2024-05-20T18:27:14.047675+0000","last_became_active":"2024-05-20T18:27:11.826656+0000","last_became_peered":"2024-05-20T18:27:11.826656+0000","last_unstale":"2024-05-20T18:27:14.047675+0000","last_undegraded":"2024-05-20T18:27:14.047675+0000","last_fullsized":"2024-05-20T18:27:14.047675+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:01:50.652864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048684+0000","last_change":"2024-05-20T18:27:07.809732+0000","last_active":"2024-05-20T18:27:14.048684+0000","last_peered":"2024-05-20T18:27:14.048684+0000","last_clean":"2024-05-20T18:27:14.048684+0000","last_became_active":"2024-05-20T18:27:07.809412+0000","last_became_peered":"2024-05-20T18:27:07.809412+0000","last_unstale":"2024-05-20T18:27:14.048684+0000","last_undegraded":"2024-05-20T18:27:14.048684+0000","last_fullsized":"2024-05-20T18:27:14.048684+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:14:03.712607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.046974+0000","last_change":"2024-05-20T18:27:06.787105+0000","last_active":"2024-05-20T18:27:14.046974+0000","last_peered":"2024-05-20T18:27:14.046974+0000","last_clean":"2024-05-20T18:27:14.046974+0000","last_became_active":"2024-05-20T18:27:06.786759+0000","last_became_peered":"2024-05-20T18:27:06.786759+0000","last_unstale":"2024-05-20T18:27:14.046974+0000","last_undegraded":"2024-05-20T18:27:14.046974+0000","last_fullsized":"2024-05-20T18:27:14.046974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:10:27.276400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049140+0000","last_change":"2024-05-20T18:27:11.834554+0000","last_active":"2024-05-20T18:27:14.049140+0000","last_peered":"2024-05-20T18:27:14.049140+0000","last_clean":"2024-05-20T18:27:14.049140+0000","last_became_active":"2024-05-20T18:27:11.833955+0000","last_became_peered":"2024-05-20T18:27:11.833955+0000","last_unstale":"2024-05-20T18:27:14.049140+0000","last_undegraded":"2024-05-20T18:27:14.049140+0000","last_fullsized":"2024-05-20T18:27:14.049140+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:53:56.893940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825845+0000","last_change":"2024-05-20T18:27:07.814023+0000","last_active":"2024-05-20T18:27:13.825845+0000","last_peered":"2024-05-20T18:27:13.825845+0000","last_clean":"2024-05-20T18:27:13.825845+0000","last_became_active":"2024-05-20T18:27:07.813679+0000","last_became_peered":"2024-05-20T18:27:07.813679+0000","last_unstale":"2024-05-20T18:27:13.825845+0000","last_undegraded":"2024-05-20T18:27:13.825845+0000","last_fullsized":"2024-05-20T18:27:13.825845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:10:20.947601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825319+0000","last_change":"2024-05-20T18:27:06.787006+0000","last_active":"2024-05-20T18:27:13.825319+0000","last_peered":"2024-05-20T18:27:13.825319+0000","last_clean":"2024-05-20T18:27:13.825319+0000","last_became_active":"2024-05-20T18:27:06.786565+0000","last_became_peered":"2024-05-20T18:27:06.786565+0000","last_unstale":"2024-05-20T18:27:13.825319+0000","last_undegraded":"2024-05-20T18:27:13.825319+0000","last_fullsized":"2024-05-20T18:27:13.825319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:10:27.276400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.822189+0000","last_change":"2024-05-20T18:27:12.468307+0000","last_active":"2024-05-20T18:27:13.822189+0000","last_peered":"2024-05-20T18:27:13.822189+0000","last_clean":"2024-05-20T18:27:13.822189+0000","last_became_active":"2024-05-20T18:27:12.467856+0000","last_became_peered":"2024-05-20T18:27:12.467856+0000","last_unstale":"2024-05-20T18:27:13.822189+0000","last_undegraded":"2024-05-20T18:27:13.822189+0000","last_fullsized":"2024-05-20T18:27:13.822189+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:08:26.960099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049060+0000","last_change":"2024-05-20T18:27:07.809032+0000","last_active":"2024-05-20T18:27:14.049060+0000","last_peered":"2024-05-20T18:27:14.049060+0000","last_clean":"2024-05-20T18:27:14.049060+0000","last_became_active":"2024-05-20T18:27:07.808708+0000","last_became_peered":"2024-05-20T18:27:07.808708+0000","last_unstale":"2024-05-20T18:27:14.049060+0000","last_undegraded":"2024-05-20T18:27:14.049060+0000","last_fullsized":"2024-05-20T18:27:14.049060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:08:51.463251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825345+0000","last_change":"2024-05-20T18:27:06.794995+0000","last_active":"2024-05-20T18:27:13.825345+0000","last_peered":"2024-05-20T18:27:13.825345+0000","last_clean":"2024-05-20T18:27:13.825345+0000","last_became_active":"2024-05-20T18:27:06.794662+0000","last_became_peered":"2024-05-20T18:27:06.794662+0000","last_unstale":"2024-05-20T18:27:13.825345+0000","last_undegraded":"2024-05-20T18:27:13.825345+0000","last_fullsized":"2024-05-20T18:27:13.825345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:06:31.445366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821770+0000","last_change":"2024-05-20T18:27:12.465482+0000","last_active":"2024-05-20T18:27:13.821770+0000","last_peered":"2024-05-20T18:27:13.821770+0000","last_clean":"2024-05-20T18:27:13.821770+0000","last_became_active":"2024-05-20T18:27:12.464905+0000","last_became_peered":"2024-05-20T18:27:12.464905+0000","last_unstale":"2024-05-20T18:27:13.821770+0000","last_undegraded":"2024-05-20T18:27:13.821770+0000","last_fullsized":"2024-05-20T18:27:13.821770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:16:04.416155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048114+0000","last_change":"2024-05-20T18:27:07.807096+0000","last_active":"2024-05-20T18:27:14.048114+0000","last_peered":"2024-05-20T18:27:14.048114+0000","last_clean":"2024-05-20T18:27:14.048114+0000","last_became_active":"2024-05-20T18:27:07.806625+0000","last_became_peered":"2024-05-20T18:27:07.806625+0000","last_unstale":"2024-05-20T18:27:14.048114+0000","last_undegraded":"2024-05-20T18:27:14.048114+0000","last_fullsized":"2024-05-20T18:27:14.048114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:15:10.022458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048546+0000","last_change":"2024-05-20T18:27:06.785919+0000","last_active":"2024-05-20T18:27:14.048546+0000","last_peered":"2024-05-20T18:27:14.048546+0000","last_clean":"2024-05-20T18:27:14.048546+0000","last_became_active":"2024-05-20T18:27:06.785587+0000","last_became_peered":"2024-05-20T18:27:06.785587+0000","last_unstale":"2024-05-20T18:27:14.048546+0000","last_undegraded":"2024-05-20T18:27:14.048546+0000","last_fullsized":"2024-05-20T18:27:14.048546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:10:44.980209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824622+0000","last_change":"2024-05-20T18:27:12.466586+0000","last_active":"2024-05-20T18:27:13.824622+0000","last_peered":"2024-05-20T18:27:13.824622+0000","last_clean":"2024-05-20T18:27:13.824622+0000","last_became_active":"2024-05-20T18:27:12.466076+0000","last_became_peered":"2024-05-20T18:27:12.466076+0000","last_unstale":"2024-05-20T18:27:13.824622+0000","last_undegraded":"2024-05-20T18:27:13.824622+0000","last_fullsized":"2024-05-20T18:27:13.824622+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:18:23.718914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049203+0000","last_change":"2024-05-20T18:27:07.805541+0000","last_active":"2024-05-20T18:27:14.049203+0000","last_peered":"2024-05-20T18:27:14.049203+0000","last_clean":"2024-05-20T18:27:14.049203+0000","last_became_active":"2024-05-20T18:27:07.805223+0000","last_became_peered":"2024-05-20T18:27:07.805223+0000","last_unstale":"2024-05-20T18:27:14.049203+0000","last_undegraded":"2024-05-20T18:27:14.049203+0000","last_fullsized":"2024-05-20T18:27:14.049203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:52:22.903884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":33,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:17.511072+0000","last_change":"2024-05-20T18:27:06.788980+0000","last_active":"2024-05-20T18:27:17.511072+0000","last_peered":"2024-05-20T18:27:17.511072+0000","last_clean":"2024-05-20T18:27:17.511072+0000","last_became_active":"2024-05-20T18:27:06.788477+0000","last_became_peered":"2024-05-20T18:27:06.788477+0000","last_unstale":"2024-05-20T18:27:17.511072+0000","last_undegraded":"2024-05-20T18:27:17.511072+0000","last_fullsized":"2024-05-20T18:27:17.511072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:43:31.154358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048876+0000","last_change":"2024-05-20T18:27:11.829159+0000","last_active":"2024-05-20T18:27:14.048876+0000","last_peered":"2024-05-20T18:27:14.048876+0000","last_clean":"2024-05-20T18:27:14.048876+0000","last_became_active":"2024-05-20T18:27:11.826918+0000","last_became_peered":"2024-05-20T18:27:11.826918+0000","last_unstale":"2024-05-20T18:27:14.048876+0000","last_undegraded":"2024-05-20T18:27:14.048876+0000","last_fullsized":"2024-05-20T18:27:14.048876+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:38:58.503399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048691+0000","last_change":"2024-05-20T18:27:07.791122+0000","last_active":"2024-05-20T18:27:14.048691+0000","last_peered":"2024-05-20T18:27:14.048691+0000","last_clean":"2024-05-20T18:27:14.048691+0000","last_became_active":"2024-05-20T18:27:07.790777+0000","last_became_peered":"2024-05-20T18:27:07.790777+0000","last_unstale":"2024-05-20T18:27:14.048691+0000","last_undegraded":"2024-05-20T18:27:14.048691+0000","last_fullsized":"2024-05-20T18:27:14.048691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:11:08.836782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048983+0000","last_change":"2024-05-20T18:27:06.799123+0000","last_active":"2024-05-20T18:27:14.048983+0000","last_peered":"2024-05-20T18:27:14.048983+0000","last_clean":"2024-05-20T18:27:14.048983+0000","last_became_active":"2024-05-20T18:27:06.798681+0000","last_became_peered":"2024-05-20T18:27:06.798681+0000","last_unstale":"2024-05-20T18:27:14.048983+0000","last_undegraded":"2024-05-20T18:27:14.048983+0000","last_fullsized":"2024-05-20T18:27:14.048983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:18:18.528418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048322+0000","last_change":"2024-05-20T18:27:11.823730+0000","last_active":"2024-05-20T18:27:14.048322+0000","last_peered":"2024-05-20T18:27:14.048322+0000","last_clean":"2024-05-20T18:27:14.048322+0000","last_became_active":"2024-05-20T18:27:11.823226+0000","last_became_peered":"2024-05-20T18:27:11.823226+0000","last_unstale":"2024-05-20T18:27:14.048322+0000","last_undegraded":"2024-05-20T18:27:14.048322+0000","last_fullsized":"2024-05-20T18:27:14.048322+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:14:31.209452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048598+0000","last_change":"2024-05-20T18:27:07.804871+0000","last_active":"2024-05-20T18:27:14.048598+0000","last_peered":"2024-05-20T18:27:14.048598+0000","last_clean":"2024-05-20T18:27:14.048598+0000","last_became_active":"2024-05-20T18:27:07.804518+0000","last_became_peered":"2024-05-20T18:27:07.804518+0000","last_unstale":"2024-05-20T18:27:14.048598+0000","last_undegraded":"2024-05-20T18:27:14.048598+0000","last_fullsized":"2024-05-20T18:27:14.048598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:14:03.712607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824274+0000","last_change":"2024-05-20T18:27:06.795936+0000","last_active":"2024-05-20T18:27:13.824274+0000","last_peered":"2024-05-20T18:27:13.824274+0000","last_clean":"2024-05-20T18:27:13.824274+0000","last_became_active":"2024-05-20T18:27:06.795616+0000","last_became_peered":"2024-05-20T18:27:06.795616+0000","last_unstale":"2024-05-20T18:27:13.824274+0000","last_undegraded":"2024-05-20T18:27:13.824274+0000","last_fullsized":"2024-05-20T18:27:13.824274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:24:55.001924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821242+0000","last_change":"2024-05-20T18:27:11.827063+0000","last_active":"2024-05-20T18:27:13.821242+0000","last_peered":"2024-05-20T18:27:13.821242+0000","last_clean":"2024-05-20T18:27:13.821242+0000","last_became_active":"2024-05-20T18:27:11.825223+0000","last_became_peered":"2024-05-20T18:27:11.825223+0000","last_unstale":"2024-05-20T18:27:13.821242+0000","last_undegraded":"2024-05-20T18:27:13.821242+0000","last_fullsized":"2024-05-20T18:27:13.821242+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:01:18.112337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824776+0000","last_change":"2024-05-20T18:27:07.810225+0000","last_active":"2024-05-20T18:27:13.824776+0000","last_peered":"2024-05-20T18:27:13.824776+0000","last_clean":"2024-05-20T18:27:13.824776+0000","last_became_active":"2024-05-20T18:27:07.809511+0000","last_became_peered":"2024-05-20T18:27:07.809511+0000","last_unstale":"2024-05-20T18:27:13.824776+0000","last_undegraded":"2024-05-20T18:27:13.824776+0000","last_fullsized":"2024-05-20T18:27:13.824776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:24:59.216904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.826020+0000","last_change":"2024-05-20T18:27:06.796334+0000","last_active":"2024-05-20T18:27:13.826020+0000","last_peered":"2024-05-20T18:27:13.826020+0000","last_clean":"2024-05-20T18:27:13.826020+0000","last_became_active":"2024-05-20T18:27:06.795897+0000","last_became_peered":"2024-05-20T18:27:06.795897+0000","last_unstale":"2024-05-20T18:27:13.826020+0000","last_undegraded":"2024-05-20T18:27:13.826020+0000","last_fullsized":"2024-05-20T18:27:13.826020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:06:31.445366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821208+0000","last_change":"2024-05-20T18:27:12.465588+0000","last_active":"2024-05-20T18:27:13.821208+0000","last_peered":"2024-05-20T18:27:13.821208+0000","last_clean":"2024-05-20T18:27:13.821208+0000","last_became_active":"2024-05-20T18:27:12.465170+0000","last_became_peered":"2024-05-20T18:27:12.465170+0000","last_unstale":"2024-05-20T18:27:13.821208+0000","last_undegraded":"2024-05-20T18:27:13.821208+0000","last_fullsized":"2024-05-20T18:27:13.821208+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:36:54.997159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048936+0000","last_change":"2024-05-20T18:27:12.469946+0000","last_active":"2024-05-20T18:27:14.048936+0000","last_peered":"2024-05-20T18:27:14.048936+0000","last_clean":"2024-05-20T18:27:14.048936+0000","last_became_active":"2024-05-20T18:27:12.469526+0000","last_became_peered":"2024-05-20T18:27:12.469526+0000","last_unstale":"2024-05-20T18:27:14.048936+0000","last_undegraded":"2024-05-20T18:27:14.048936+0000","last_fullsized":"2024-05-20T18:27:14.048936+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:34:21.648495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048409+0000","last_change":"2024-05-20T18:27:07.788636+0000","last_active":"2024-05-20T18:27:14.048409+0000","last_peered":"2024-05-20T18:27:14.048409+0000","last_clean":"2024-05-20T18:27:14.048409+0000","last_became_active":"2024-05-20T18:27:07.788307+0000","last_became_peered":"2024-05-20T18:27:07.788307+0000","last_unstale":"2024-05-20T18:27:14.048409+0000","last_undegraded":"2024-05-20T18:27:14.048409+0000","last_fullsized":"2024-05-20T18:27:14.048409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:10:20.947601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824264+0000","last_change":"2024-05-20T18:27:06.797781+0000","last_active":"2024-05-20T18:27:13.824264+0000","last_peered":"2024-05-20T18:27:13.824264+0000","last_clean":"2024-05-20T18:27:13.824264+0000","last_became_active":"2024-05-20T18:27:06.797415+0000","last_became_peered":"2024-05-20T18:27:06.797415+0000","last_unstale":"2024-05-20T18:27:13.824264+0000","last_undegraded":"2024-05-20T18:27:13.824264+0000","last_fullsized":"2024-05-20T18:27:13.824264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:18:18.528418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825975+0000","last_change":"2024-05-20T18:27:11.827561+0000","last_active":"2024-05-20T18:27:13.825975+0000","last_peered":"2024-05-20T18:27:13.825975+0000","last_clean":"2024-05-20T18:27:13.825975+0000","last_became_active":"2024-05-20T18:27:11.826925+0000","last_became_peered":"2024-05-20T18:27:11.826925+0000","last_unstale":"2024-05-20T18:27:13.825975+0000","last_undegraded":"2024-05-20T18:27:13.825975+0000","last_fullsized":"2024-05-20T18:27:13.825975+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:16:41.968388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821278+0000","last_change":"2024-05-20T18:27:07.789976+0000","last_active":"2024-05-20T18:27:13.821278+0000","last_peered":"2024-05-20T18:27:13.821278+0000","last_clean":"2024-05-20T18:27:13.821278+0000","last_became_active":"2024-05-20T18:27:07.789679+0000","last_became_peered":"2024-05-20T18:27:07.789679+0000","last_unstale":"2024-05-20T18:27:13.821278+0000","last_undegraded":"2024-05-20T18:27:13.821278+0000","last_fullsized":"2024-05-20T18:27:13.821278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:52:57.891709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821366+0000","last_change":"2024-05-20T18:27:06.794988+0000","last_active":"2024-05-20T18:27:13.821366+0000","last_peered":"2024-05-20T18:27:13.821366+0000","last_clean":"2024-05-20T18:27:13.821366+0000","last_became_active":"2024-05-20T18:27:06.794630+0000","last_became_peered":"2024-05-20T18:27:06.794630+0000","last_unstale":"2024-05-20T18:27:13.821366+0000","last_undegraded":"2024-05-20T18:27:13.821366+0000","last_fullsized":"2024-05-20T18:27:13.821366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:24:55.001924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825689+0000","last_change":"2024-05-20T18:27:06.785041+0000","last_active":"2024-05-20T18:27:13.825689+0000","last_peered":"2024-05-20T18:27:13.825689+0000","last_clean":"2024-05-20T18:27:13.825689+0000","last_became_active":"2024-05-20T18:27:06.784673+0000","last_became_peered":"2024-05-20T18:27:06.784673+0000","last_unstale":"2024-05-20T18:27:13.825689+0000","last_undegraded":"2024-05-20T18:27:13.825689+0000","last_fullsized":"2024-05-20T18:27:13.825689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:52:44.943426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048466+0000","last_change":"2024-05-20T18:27:07.790855+0000","last_active":"2024-05-20T18:27:14.048466+0000","last_peered":"2024-05-20T18:27:14.048466+0000","last_clean":"2024-05-20T18:27:14.048466+0000","last_became_active":"2024-05-20T18:27:07.790370+0000","last_became_peered":"2024-05-20T18:27:07.790370+0000","last_unstale":"2024-05-20T18:27:14.048466+0000","last_undegraded":"2024-05-20T18:27:14.048466+0000","last_fullsized":"2024-05-20T18:27:14.048466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:07:15.764912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047264+0000","last_change":"2024-05-20T18:27:11.828935+0000","last_active":"2024-05-20T18:27:14.047264+0000","last_peered":"2024-05-20T18:27:14.047264+0000","last_clean":"2024-05-20T18:27:14.047264+0000","last_became_active":"2024-05-20T18:27:11.828516+0000","last_became_peered":"2024-05-20T18:27:11.828516+0000","last_unstale":"2024-05-20T18:27:14.047264+0000","last_undegraded":"2024-05-20T18:27:14.047264+0000","last_fullsized":"2024-05-20T18:27:14.047264+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:15:14.052151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821617+0000","last_change":"2024-05-20T18:27:06.803606+0000","last_active":"2024-05-20T18:27:13.821617+0000","last_peered":"2024-05-20T18:27:13.821617+0000","last_clean":"2024-05-20T18:27:13.821617+0000","last_became_active":"2024-05-20T18:27:06.803340+0000","last_became_peered":"2024-05-20T18:27:06.803340+0000","last_unstale":"2024-05-20T18:27:13.821617+0000","last_undegraded":"2024-05-20T18:27:13.821617+0000","last_fullsized":"2024-05-20T18:27:13.821617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:05.760859+0000","last_clean_scrub_stamp":"2024-05-20T18:27:05.760859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:10:27.276400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048051+0000","last_change":"2024-05-20T18:27:07.789834+0000","last_active":"2024-05-20T18:27:14.048051+0000","last_peered":"2024-05-20T18:27:14.048051+0000","last_clean":"2024-05-20T18:27:14.048051+0000","last_became_active":"2024-05-20T18:27:07.789472+0000","last_became_peered":"2024-05-20T18:27:07.789472+0000","last_unstale":"2024-05-20T18:27:14.048051+0000","last_undegraded":"2024-05-20T18:27:14.048051+0000","last_fullsized":"2024-05-20T18:27:14.048051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:52:45.948931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821874+0000","last_change":"2024-05-20T18:27:11.825859+0000","last_active":"2024-05-20T18:27:13.821874+0000","last_peered":"2024-05-20T18:27:13.821874+0000","last_clean":"2024-05-20T18:27:13.821874+0000","last_became_active":"2024-05-20T18:27:11.825093+0000","last_became_peered":"2024-05-20T18:27:11.825093+0000","last_unstale":"2024-05-20T18:27:13.821874+0000","last_undegraded":"2024-05-20T18:27:13.821874+0000","last_fullsized":"2024-05-20T18:27:13.821874+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:10.796056+0000","last_clean_scrub_stamp":"2024-05-20T18:27:10.796056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:02:46.122485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825718+0000","last_change":"2024-05-20T18:27:07.793263+0000","last_active":"2024-05-20T18:27:13.825718+0000","last_peered":"2024-05-20T18:27:13.825718+0000","last_clean":"2024-05-20T18:27:13.825718+0000","last_became_active":"2024-05-20T18:27:07.792916+0000","last_became_peered":"2024-05-20T18:27:07.792916+0000","last_unstale":"2024-05-20T18:27:13.825718+0000","last_undegraded":"2024-05-20T18:27:13.825718+0000","last_fullsized":"2024-05-20T18:27:13.825718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:15:10.022458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047433+0000","last_change":"2024-05-20T18:27:07.805541+0000","last_active":"2024-05-20T18:27:14.047433+0000","last_peered":"2024-05-20T18:27:14.047433+0000","last_clean":"2024-05-20T18:27:14.047433+0000","last_became_active":"2024-05-20T18:27:07.805160+0000","last_became_peered":"2024-05-20T18:27:07.805160+0000","last_unstale":"2024-05-20T18:27:14.047433+0000","last_undegraded":"2024-05-20T18:27:14.047433+0000","last_fullsized":"2024-05-20T18:27:14.047433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:14:23.046328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824975+0000","last_change":"2024-05-20T18:27:07.804547+0000","last_active":"2024-05-20T18:27:13.824975+0000","last_peered":"2024-05-20T18:27:13.824975+0000","last_clean":"2024-05-20T18:27:13.824975+0000","last_became_active":"2024-05-20T18:27:07.804242+0000","last_became_peered":"2024-05-20T18:27:07.804242+0000","last_unstale":"2024-05-20T18:27:13.824975+0000","last_undegraded":"2024-05-20T18:27:13.824975+0000","last_fullsized":"2024-05-20T18:27:13.824975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:55:24.560518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048436+0000","last_change":"2024-05-20T18:27:07.790683+0000","last_active":"2024-05-20T18:27:14.048436+0000","last_peered":"2024-05-20T18:27:14.048436+0000","last_clean":"2024-05-20T18:27:14.048436+0000","last_became_active":"2024-05-20T18:27:07.789978+0000","last_became_peered":"2024-05-20T18:27:07.789978+0000","last_unstale":"2024-05-20T18:27:14.048436+0000","last_undegraded":"2024-05-20T18:27:14.048436+0000","last_fullsized":"2024-05-20T18:27:14.048436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:40:08.698691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821852+0000","last_change":"2024-05-20T18:27:07.791924+0000","last_active":"2024-05-20T18:27:13.821852+0000","last_peered":"2024-05-20T18:27:13.821852+0000","last_clean":"2024-05-20T18:27:13.821852+0000","last_became_active":"2024-05-20T18:27:07.791622+0000","last_became_peered":"2024-05-20T18:27:07.791622+0000","last_unstale":"2024-05-20T18:27:13.821852+0000","last_undegraded":"2024-05-20T18:27:13.821852+0000","last_fullsized":"2024-05-20T18:27:13.821852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:07:15.764912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048748+0000","last_change":"2024-05-20T18:27:07.789116+0000","last_active":"2024-05-20T18:27:14.048748+0000","last_peered":"2024-05-20T18:27:14.048748+0000","last_clean":"2024-05-20T18:27:14.048748+0000","last_became_active":"2024-05-20T18:27:07.787855+0000","last_became_peered":"2024-05-20T18:27:07.787855+0000","last_unstale":"2024-05-20T18:27:14.048748+0000","last_undegraded":"2024-05-20T18:27:14.048748+0000","last_fullsized":"2024-05-20T18:27:14.048748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:52:57.891709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825914+0000","last_change":"2024-05-20T18:27:07.793074+0000","last_active":"2024-05-20T18:27:13.825914+0000","last_peered":"2024-05-20T18:27:13.825914+0000","last_clean":"2024-05-20T18:27:13.825914+0000","last_became_active":"2024-05-20T18:27:07.792754+0000","last_became_peered":"2024-05-20T18:27:07.792754+0000","last_unstale":"2024-05-20T18:27:13.825914+0000","last_undegraded":"2024-05-20T18:27:13.825914+0000","last_fullsized":"2024-05-20T18:27:13.825914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:12.006827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.822159+0000","last_change":"2024-05-20T18:27:07.791208+0000","last_active":"2024-05-20T18:27:13.822159+0000","last_peered":"2024-05-20T18:27:13.822159+0000","last_clean":"2024-05-20T18:27:13.822159+0000","last_became_active":"2024-05-20T18:27:07.790864+0000","last_became_peered":"2024-05-20T18:27:07.790864+0000","last_unstale":"2024-05-20T18:27:13.822159+0000","last_undegraded":"2024-05-20T18:27:13.822159+0000","last_fullsized":"2024-05-20T18:27:13.822159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:19:31.649320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824356+0000","last_change":"2024-05-20T18:27:07.808305+0000","last_active":"2024-05-20T18:27:13.824356+0000","last_peered":"2024-05-20T18:27:13.824356+0000","last_clean":"2024-05-20T18:27:13.824356+0000","last_became_active":"2024-05-20T18:27:07.807877+0000","last_became_peered":"2024-05-20T18:27:07.807877+0000","last_unstale":"2024-05-20T18:27:13.824356+0000","last_undegraded":"2024-05-20T18:27:13.824356+0000","last_fullsized":"2024-05-20T18:27:13.824356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:14:03.712607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821322+0000","last_change":"2024-05-20T18:27:07.791872+0000","last_active":"2024-05-20T18:27:13.821322+0000","last_peered":"2024-05-20T18:27:13.821322+0000","last_clean":"2024-05-20T18:27:13.821322+0000","last_became_active":"2024-05-20T18:27:07.791570+0000","last_became_peered":"2024-05-20T18:27:07.791570+0000","last_unstale":"2024-05-20T18:27:13.821322+0000","last_undegraded":"2024-05-20T18:27:13.821322+0000","last_fullsized":"2024-05-20T18:27:13.821322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:29:33.478403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824943+0000","last_change":"2024-05-20T18:27:07.811057+0000","last_active":"2024-05-20T18:27:13.824943+0000","last_peered":"2024-05-20T18:27:13.824943+0000","last_clean":"2024-05-20T18:27:13.824943+0000","last_became_active":"2024-05-20T18:27:07.810728+0000","last_became_peered":"2024-05-20T18:27:07.810728+0000","last_unstale":"2024-05-20T18:27:13.824943+0000","last_undegraded":"2024-05-20T18:27:13.824943+0000","last_fullsized":"2024-05-20T18:27:13.824943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:19:31.649320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048182+0000","last_change":"2024-05-20T18:27:07.795559+0000","last_active":"2024-05-20T18:27:14.048182+0000","last_peered":"2024-05-20T18:27:14.048182+0000","last_clean":"2024-05-20T18:27:14.048182+0000","last_became_active":"2024-05-20T18:27:07.795228+0000","last_became_peered":"2024-05-20T18:27:07.795228+0000","last_unstale":"2024-05-20T18:27:14.048182+0000","last_undegraded":"2024-05-20T18:27:14.048182+0000","last_fullsized":"2024-05-20T18:27:14.048182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:10:20.947601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.826124+0000","last_change":"2024-05-20T18:27:07.795052+0000","last_active":"2024-05-20T18:27:13.826124+0000","last_peered":"2024-05-20T18:27:13.826124+0000","last_clean":"2024-05-20T18:27:13.826124+0000","last_became_active":"2024-05-20T18:27:07.794704+0000","last_became_peered":"2024-05-20T18:27:07.794704+0000","last_unstale":"2024-05-20T18:27:13.826124+0000","last_undegraded":"2024-05-20T18:27:13.826124+0000","last_fullsized":"2024-05-20T18:27:13.826124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:34:15.614124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049515+0000","last_change":"2024-05-20T18:27:07.807441+0000","last_active":"2024-05-20T18:27:14.049515+0000","last_peered":"2024-05-20T18:27:14.049515+0000","last_clean":"2024-05-20T18:27:14.049515+0000","last_became_active":"2024-05-20T18:27:07.807095+0000","last_became_peered":"2024-05-20T18:27:07.807095+0000","last_unstale":"2024-05-20T18:27:14.049515+0000","last_undegraded":"2024-05-20T18:27:14.049515+0000","last_fullsized":"2024-05-20T18:27:14.049515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:24:59.216904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047301+0000","last_change":"2024-05-20T18:27:07.801623+0000","last_active":"2024-05-20T18:27:14.047301+0000","last_peered":"2024-05-20T18:27:14.047301+0000","last_clean":"2024-05-20T18:27:14.047301+0000","last_became_active":"2024-05-20T18:27:07.801303+0000","last_became_peered":"2024-05-20T18:27:07.801303+0000","last_unstale":"2024-05-20T18:27:14.047301+0000","last_undegraded":"2024-05-20T18:27:14.047301+0000","last_fullsized":"2024-05-20T18:27:14.047301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:29:19.667394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824621+0000","last_change":"2024-05-20T18:27:07.809861+0000","last_active":"2024-05-20T18:27:13.824621+0000","last_peered":"2024-05-20T18:27:13.824621+0000","last_clean":"2024-05-20T18:27:13.824621+0000","last_became_active":"2024-05-20T18:27:07.809523+0000","last_became_peered":"2024-05-20T18:27:07.809523+0000","last_unstale":"2024-05-20T18:27:13.824621+0000","last_undegraded":"2024-05-20T18:27:13.824621+0000","last_fullsized":"2024-05-20T18:27:13.824621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:55:32.958186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.823933+0000","last_change":"2024-05-20T18:27:07.814234+0000","last_active":"2024-05-20T18:27:13.823933+0000","last_peered":"2024-05-20T18:27:13.823933+0000","last_clean":"2024-05-20T18:27:13.823933+0000","last_became_active":"2024-05-20T18:27:07.813168+0000","last_became_peered":"2024-05-20T18:27:07.813168+0000","last_unstale":"2024-05-20T18:27:13.823933+0000","last_undegraded":"2024-05-20T18:27:13.823933+0000","last_fullsized":"2024-05-20T18:27:13.823933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:12.006827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048520+0000","last_change":"2024-05-20T18:27:07.787849+0000","last_active":"2024-05-20T18:27:14.048520+0000","last_peered":"2024-05-20T18:27:14.048520+0000","last_clean":"2024-05-20T18:27:14.048520+0000","last_became_active":"2024-05-20T18:27:07.787406+0000","last_became_peered":"2024-05-20T18:27:07.787406+0000","last_unstale":"2024-05-20T18:27:14.048520+0000","last_undegraded":"2024-05-20T18:27:14.048520+0000","last_fullsized":"2024-05-20T18:27:14.048520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:34:15.614124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.047987+0000","last_change":"2024-05-20T18:27:07.795141+0000","last_active":"2024-05-20T18:27:14.047987+0000","last_peered":"2024-05-20T18:27:14.047987+0000","last_clean":"2024-05-20T18:27:14.047987+0000","last_became_active":"2024-05-20T18:27:07.793487+0000","last_became_peered":"2024-05-20T18:27:07.793487+0000","last_unstale":"2024-05-20T18:27:14.047987+0000","last_undegraded":"2024-05-20T18:27:14.047987+0000","last_fullsized":"2024-05-20T18:27:14.047987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:32:42.769509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821496+0000","last_change":"2024-05-20T18:27:07.802751+0000","last_active":"2024-05-20T18:27:13.821496+0000","last_peered":"2024-05-20T18:27:13.821496+0000","last_clean":"2024-05-20T18:27:13.821496+0000","last_became_active":"2024-05-20T18:27:07.802412+0000","last_became_peered":"2024-05-20T18:27:07.802412+0000","last_unstale":"2024-05-20T18:27:13.821496+0000","last_undegraded":"2024-05-20T18:27:13.821496+0000","last_fullsized":"2024-05-20T18:27:13.821496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:14:03.712607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048564+0000","last_change":"2024-05-20T18:27:07.796408+0000","last_active":"2024-05-20T18:27:14.048564+0000","last_peered":"2024-05-20T18:27:14.048564+0000","last_clean":"2024-05-20T18:27:14.048564+0000","last_became_active":"2024-05-20T18:27:07.795961+0000","last_became_peered":"2024-05-20T18:27:07.795961+0000","last_unstale":"2024-05-20T18:27:14.048564+0000","last_undegraded":"2024-05-20T18:27:14.048564+0000","last_fullsized":"2024-05-20T18:27:14.048564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:43:32.159862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824841+0000","last_change":"2024-05-20T18:27:07.806684+0000","last_active":"2024-05-20T18:27:13.824841+0000","last_peered":"2024-05-20T18:27:13.824841+0000","last_clean":"2024-05-20T18:27:13.824841+0000","last_became_active":"2024-05-20T18:27:07.806320+0000","last_became_peered":"2024-05-20T18:27:07.806320+0000","last_unstale":"2024-05-20T18:27:13.824841+0000","last_undegraded":"2024-05-20T18:27:13.824841+0000","last_fullsized":"2024-05-20T18:27:13.824841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:52:57.891709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825792+0000","last_change":"2024-05-20T18:27:07.805587+0000","last_active":"2024-05-20T18:27:13.825792+0000","last_peered":"2024-05-20T18:27:13.825792+0000","last_clean":"2024-05-20T18:27:13.825792+0000","last_became_active":"2024-05-20T18:27:07.804971+0000","last_became_peered":"2024-05-20T18:27:07.804971+0000","last_unstale":"2024-05-20T18:27:13.825792+0000","last_undegraded":"2024-05-20T18:27:13.825792+0000","last_fullsized":"2024-05-20T18:27:13.825792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:07:15.764912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049422+0000","last_change":"2024-05-20T18:27:07.815867+0000","last_active":"2024-05-20T18:27:14.049422+0000","last_peered":"2024-05-20T18:27:14.049422+0000","last_clean":"2024-05-20T18:27:14.049422+0000","last_became_active":"2024-05-20T18:27:07.815421+0000","last_became_peered":"2024-05-20T18:27:07.815421+0000","last_unstale":"2024-05-20T18:27:14.049422+0000","last_undegraded":"2024-05-20T18:27:14.049422+0000","last_fullsized":"2024-05-20T18:27:14.049422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:16:37.938695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.821166+0000","last_change":"2024-05-20T18:27:07.810206+0000","last_active":"2024-05-20T18:27:13.821166+0000","last_peered":"2024-05-20T18:27:13.821166+0000","last_clean":"2024-05-20T18:27:13.821166+0000","last_became_active":"2024-05-20T18:27:07.809846+0000","last_became_peered":"2024-05-20T18:27:07.809846+0000","last_unstale":"2024-05-20T18:27:13.821166+0000","last_undegraded":"2024-05-20T18:27:13.821166+0000","last_fullsized":"2024-05-20T18:27:13.821166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:10:20.947601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048623+0000","last_change":"2024-05-20T18:27:07.813420+0000","last_active":"2024-05-20T18:27:14.048623+0000","last_peered":"2024-05-20T18:27:14.048623+0000","last_clean":"2024-05-20T18:27:14.048623+0000","last_became_active":"2024-05-20T18:27:07.813093+0000","last_became_peered":"2024-05-20T18:27:07.813093+0000","last_unstale":"2024-05-20T18:27:14.048623+0000","last_undegraded":"2024-05-20T18:27:14.048623+0000","last_fullsized":"2024-05-20T18:27:14.048623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:19:31.649320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.824831+0000","last_change":"2024-05-20T18:27:07.812561+0000","last_active":"2024-05-20T18:27:13.824831+0000","last_peered":"2024-05-20T18:27:13.824831+0000","last_clean":"2024-05-20T18:27:13.824831+0000","last_became_active":"2024-05-20T18:27:07.812264+0000","last_became_peered":"2024-05-20T18:27:07.812264+0000","last_unstale":"2024-05-20T18:27:13.824831+0000","last_undegraded":"2024-05-20T18:27:13.824831+0000","last_fullsized":"2024-05-20T18:27:13.824831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:52:22.903884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048783+0000","last_change":"2024-05-20T18:27:07.804441+0000","last_active":"2024-05-20T18:27:14.048783+0000","last_peered":"2024-05-20T18:27:14.048783+0000","last_clean":"2024-05-20T18:27:14.048783+0000","last_became_active":"2024-05-20T18:27:07.804126+0000","last_became_peered":"2024-05-20T18:27:07.804126+0000","last_unstale":"2024-05-20T18:27:14.048783+0000","last_undegraded":"2024-05-20T18:27:14.048783+0000","last_fullsized":"2024-05-20T18:27:14.048783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:14:17.791789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:13.825082+0000","last_change":"2024-05-20T18:27:07.808819+0000","last_active":"2024-05-20T18:27:13.825082+0000","last_peered":"2024-05-20T18:27:13.825082+0000","last_clean":"2024-05-20T18:27:13.825082+0000","last_became_active":"2024-05-20T18:27:07.808231+0000","last_became_peered":"2024-05-20T18:27:07.808231+0000","last_unstale":"2024-05-20T18:27:13.825082+0000","last_undegraded":"2024-05-20T18:27:13.825082+0000","last_fullsized":"2024-05-20T18:27:13.825082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:14:17.791789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048097+0000","last_change":"2024-05-20T18:27:07.793960+0000","last_active":"2024-05-20T18:27:14.048097+0000","last_peered":"2024-05-20T18:27:14.048097+0000","last_clean":"2024-05-20T18:27:14.048097+0000","last_became_active":"2024-05-20T18:27:07.793594+0000","last_became_peered":"2024-05-20T18:27:07.793594+0000","last_unstale":"2024-05-20T18:27:14.048097+0000","last_undegraded":"2024-05-20T18:27:14.048097+0000","last_fullsized":"2024-05-20T18:27:14.048097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:34:15.614124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.049410+0000","last_change":"2024-05-20T18:27:07.790606+0000","last_active":"2024-05-20T18:27:14.049410+0000","last_peered":"2024-05-20T18:27:14.049410+0000","last_clean":"2024-05-20T18:27:14.049410+0000","last_became_active":"2024-05-20T18:27:07.790227+0000","last_became_peered":"2024-05-20T18:27:07.790227+0000","last_unstale":"2024-05-20T18:27:14.049410+0000","last_undegraded":"2024-05-20T18:27:14.049410+0000","last_fullsized":"2024-05-20T18:27:14.049410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:14:03.712607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-05-20T18:27:14.048973+0000","last_change":"2024-05-20T18:27:07.816207+0000","last_active":"2024-05-20T18:27:14.048973+0000","last_peered":"2024-05-20T18:27:14.048973+0000","last_clean":"2024-05-20T18:27:14.048973+0000","last_became_active":"2024-05-20T18:27:07.815556+0000","last_became_peered":"2024-05-20T18:27:07.815556+0000","last_unstale":"2024-05-20T18:27:14.048973+0000","last_undegraded":"2024-05-20T18:27:14.048973+0000","last_fullsized":"2024-05-20T18:27:14.048973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:06.766363+0000","last_clean_scrub_stamp":"2024-05-20T18:27:06.766363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:55:24.560518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":876544,"data_stored":703014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":115906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":112154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":112190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":290816,"data_stored":115906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":278528,"data_stored":112064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":286720,"data_stored":112224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":884736,"data_stored":703070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T18:27:36.053 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-20T18:27:36.053 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-20T18:27:36.053 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-20T18:27:36.053 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-20T18:27:36.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.203+0000 7f780bc06640 1 Processor -- start 2024-05-20T18:27:36.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.203+0000 7f780bc06640 1 -- start start 2024-05-20T18:27:36.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.204+0000 7f780bc06640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f78040b4070 0x7f78040ba5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:36.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.204+0000 7f780bc06640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f78040d0560 0x7f780406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:36.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.204+0000 7f780bc06640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f780406dd20 con 0x7f780406d910 2024-05-20T18:27:36.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.204+0000 7f780bc06640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f78040b8490 con 0x7f78040d0560 2024-05-20T18:27:36.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.204+0000 7f780bc06640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f780415fa90 con 0x7f78040b4070 2024-05-20T18:27:36.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.205+0000 7f78099a2640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f78040b4070 0x7f78040ba5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.205+0000 7f78091a1640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f78040d0560 0x7f780406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.205+0000 7f78091a1640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f78040d0560 0x7f780406d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:41558/0 (socket says 172.21.15.5:41558) 2024-05-20T18:27:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.205+0000 7f78099a2640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f78040b4070 0x7f78040ba5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:37190/0 (socket says 172.21.15.5:37190) 2024-05-20T18:27:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.205+0000 7f78091a1640 1 -- 172.21.15.5:0/3370645423 learned_addr learned my addr 172.21.15.5:0/3370645423 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.205+0000 7f78091a1640 1 -- 172.21.15.5:0/3370645423 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f78040b4070 msgr2=0x7f78040ba5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.205+0000 7f78091a1640 1 --2- 172.21.15.5:0/3370645423 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f78040b4070 0x7f78040ba5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.205+0000 7f78091a1640 1 -- 172.21.15.5:0/3370645423 >> v1:172.21.15.63:6789/0 conn(0x7f780406d910 legacy=0x7f78040d27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.205+0000 7f78091a1640 1 -- 172.21.15.5:0/3370645423 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78040d3f00 con 0x7f78040d0560 2024-05-20T18:27:36.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.206+0000 7f78091a1640 1 --2- 172.21.15.5:0/3370645423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78040d0560 0x7f780406d3d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f77f4009870 tx=0x7f77f402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b4c347a22e3a1a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:36.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.206+0000 7f78089a0640 1 -- 172.21.15.5:0/3370645423 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f4037070 con 0x7f78040d0560 2024-05-20T18:27:36.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.206+0000 7f78089a0640 1 -- 172.21.15.5:0/3370645423 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77f4032440 con 0x7f78040d0560 2024-05-20T18:27:36.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.206+0000 7f78089a0640 1 -- 172.21.15.5:0/3370645423 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f4032810 con 0x7f78040d0560 2024-05-20T18:27:36.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.207+0000 7f780bc06640 1 -- 172.21.15.5:0/3370645423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78040d0560 msgr2=0x7f780406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:36.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.207+0000 7f780bc06640 1 --2- 172.21.15.5:0/3370645423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78040d0560 0x7f780406d3d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f77f4009870 tx=0x7f77f402a270 comp rx=0 tx=0).stop 2024-05-20T18:27:36.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.207+0000 7f780bc06640 1 -- 172.21.15.5:0/3370645423 shutdown_connections 2024-05-20T18:27:36.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.207+0000 7f780bc06640 1 --2- 172.21.15.5:0/3370645423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78040d0560 0x7f780406d3d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:36.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.207+0000 7f780bc06640 1 --2- 172.21.15.5:0/3370645423 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f78040b4070 0x7f78040ba5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:36.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.207+0000 7f780bc06640 1 -- 172.21.15.5:0/3370645423 >> 172.21.15.5:0/3370645423 conn(0x7f78040d4c10 msgr2=0x7f78040d5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:36.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.207+0000 7f780bc06640 1 -- 172.21.15.5:0/3370645423 shutdown_connections 2024-05-20T18:27:36.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.207+0000 7f780bc06640 1 -- 172.21.15.5:0/3370645423 wait complete. 2024-05-20T18:27:36.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.209+0000 7f780bc06640 1 Processor -- start 2024-05-20T18:27:36.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.209+0000 7f780bc06640 1 -- start start 2024-05-20T18:27:36.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.210+0000 7f780bc06640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f780406d910 0x7f78040caf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:36.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.210+0000 7f780bc06640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78040b4070 0x7f780415b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:36.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.210+0000 7f780bc06640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78040d0560 0x7f7804161ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.210+0000 7f780bc06640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78040d47a0 con 0x7f78040d0560 2024-05-20T18:27:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.210+0000 7f780bc06640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f78040d44c0 con 0x7f78040b4070 2024-05-20T18:27:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.210+0000 7f780bc06640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f78040bab00 con 0x7f780406d910 2024-05-20T18:27:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.210+0000 7f780a1a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78040d0560 0x7f7804161ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.210+0000 7f780a1a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78040d0560 0x7f7804161ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52930/0 (socket says 172.21.15.5:52930) 2024-05-20T18:27:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.210+0000 7f780a1a3640 1 -- 172.21.15.5:0/3927120043 learned_addr learned my addr 172.21.15.5:0/3927120043 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.210+0000 7f78091a1640 1 --2- 172.21.15.5:0/3927120043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78040b4070 0x7f780415b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.210+0000 7f78099a2640 1 --2- 172.21.15.5:0/3927120043 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f780406d910 0x7f78040caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.210+0000 7f780a1a3640 1 -- 172.21.15.5:0/3927120043 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f780406d910 msgr2=0x7f78040caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.210+0000 7f780a1a3640 1 --2- 172.21.15.5:0/3927120043 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f780406d910 0x7f78040caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.210+0000 7f780a1a3640 1 -- 172.21.15.5:0/3927120043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78040b4070 msgr2=0x7f780415b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.210+0000 7f780a1a3640 1 --2- 172.21.15.5:0/3927120043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78040b4070 0x7f780415b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.210+0000 7f780a1a3640 1 -- 172.21.15.5:0/3927120043 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78040cb450 con 0x7f78040d0560 2024-05-20T18:27:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.211+0000 7f780a1a3640 1 --2- 172.21.15.5:0/3927120043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78040d0560 0x7f7804161ea0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00c760 tx=0x7f77fc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.211+0000 7f77fa7fc640 1 -- 172.21.15.5:0/3927120043 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc003b70 con 0x7f78040d0560 2024-05-20T18:27:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.211+0000 7f77fa7fc640 1 -- 172.21.15.5:0/3927120043 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77fc003d10 con 0x7f78040d0560 2024-05-20T18:27:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.211+0000 7f780bc06640 1 -- 172.21.15.5:0/3927120043 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78040c9e70 con 0x7f78040d0560 2024-05-20T18:27:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.211+0000 7f77fa7fc640 1 -- 172.21.15.5:0/3927120043 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc00aa90 con 0x7f78040d0560 2024-05-20T18:27:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.211+0000 7f780bc06640 1 -- 172.21.15.5:0/3927120043 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78040ca430 con 0x7f78040d0560 2024-05-20T18:27:36.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.212+0000 7f780bc06640 1 -- 172.21.15.5:0/3927120043 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78040690b0 con 0x7f78040d0560 2024-05-20T18:27:36.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.213+0000 7f77fa7fc640 1 -- 172.21.15.5:0/3927120043 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f77fc00ac30 con 0x7f78040d0560 2024-05-20T18:27:36.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.213+0000 7f77fa7fc640 1 --2- 172.21.15.5:0/3927120043 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f77e806be30 0x7f77e806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:36.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.213+0000 7f77fa7fc640 1 -- 172.21.15.5:0/3927120043 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f77fc08afe0 con 0x7f78040d0560 2024-05-20T18:27:36.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.213+0000 7f78099a2640 1 --2- 172.21.15.5:0/3927120043 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f77e806be30 0x7f77e806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:36.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.214+0000 7f78099a2640 1 --2- 172.21.15.5:0/3927120043 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f77e806be30 0x7f77e806e340 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7800009aa0 tx=0x7f7800009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:36.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.217+0000 7f77fa7fc640 1 -- 172.21.15.5:0/3927120043 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f780406dd20 con 0x7f78040d0560 2024-05-20T18:27:36.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.441+0000 7f780bc06640 1 -- 172.21.15.5:0/3927120043 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f780408d2d0 con 0x7f78040d0560 2024-05-20T18:27:36.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.442+0000 7f77fa7fc640 1 -- 172.21.15.5:0/3927120043 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f77fc058f30 con 0x7f78040d0560 2024-05-20T18:27:36.441 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:36.441 INFO:teuthology.orchestra.run.smithi005.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-20T18:27:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.443+0000 7f780bc06640 1 -- 172.21.15.5:0/3927120043 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f77e806be30 msgr2=0x7f77e806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.443+0000 7f780bc06640 1 --2- 172.21.15.5:0/3927120043 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f77e806be30 0x7f77e806e340 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7800009aa0 tx=0x7f7800009190 comp rx=0 tx=0).stop 2024-05-20T18:27:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.444+0000 7f780bc06640 1 -- 172.21.15.5:0/3927120043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78040d0560 msgr2=0x7f7804161ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.444+0000 7f780bc06640 1 --2- 172.21.15.5:0/3927120043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78040d0560 0x7f7804161ea0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00c760 tx=0x7f77fc00cc20 comp rx=0 tx=0).stop 2024-05-20T18:27:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.444+0000 7f780bc06640 1 -- 172.21.15.5:0/3927120043 shutdown_connections 2024-05-20T18:27:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.444+0000 7f780bc06640 1 --2- 172.21.15.5:0/3927120043 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f77e806be30 0x7f77e806e340 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.444+0000 7f780bc06640 1 --2- 172.21.15.5:0/3927120043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78040d0560 0x7f7804161ea0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.444+0000 7f780bc06640 1 --2- 172.21.15.5:0/3927120043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78040b4070 0x7f780415b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.444+0000 7f780bc06640 1 --2- 172.21.15.5:0/3927120043 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f780406d910 0x7f78040caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.444+0000 7f780bc06640 1 -- 172.21.15.5:0/3927120043 >> 172.21.15.5:0/3927120043 conn(0x7f78040d4c10 msgr2=0x7f78040ce440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.444+0000 7f780bc06640 1 -- 172.21.15.5:0/3927120043 shutdown_connections 2024-05-20T18:27:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.444+0000 7f780bc06640 1 -- 172.21.15.5:0/3927120043 wait complete. 2024-05-20T18:27:36.454 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-20T18:27:36.455 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:27:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.571+0000 7fe9bba7a640 1 Processor -- start 2024-05-20T18:27:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.571+0000 7fe9bba7a640 1 -- start start 2024-05-20T18:27:36.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.573+0000 7fe9bba7a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9b4168050 0x7fe9b416a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:36.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.573+0000 7fe9bba7a640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9b40d0630 con 0x7fe9b4162820 2024-05-20T18:27:36.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.573+0000 7fe9bba7a640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9b40d39c0 con 0x7fe9b406d3d0 2024-05-20T18:27:36.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.573+0000 7fe9bba7a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe9b40d3ca0 con 0x7fe9b4168050 2024-05-20T18:27:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.573+0000 7fe9b9816640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe9b4162820 0x7fe9b40b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:41008/0 (socket says 172.21.15.5:41008) 2024-05-20T18:27:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.573+0000 7fe9b9816640 1 -- 172.21.15.5:0/1987921738 learned_addr learned my addr 172.21.15.5:0/1987921738 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.573+0000 7fe9ba017640 1 --2- 172.21.15.5:0/1987921738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9b4168050 0x7fe9b416a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.573+0000 7fe9b8814640 1 -- 172.21.15.5:0/1987921738 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 975691285 0 0) 0x7fe9b40d0630 con 0x7fe9b4162820 2024-05-20T18:27:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.573+0000 7fe9b8814640 1 -- 172.21.15.5:0/1987921738 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe99c003520 con 0x7fe9b4162820 2024-05-20T18:27:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.573+0000 7fe9b8814640 1 -- 172.21.15.5:0/1987921738 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 363299035 0 0) 0x7fe9b40d39c0 con 0x7fe9b406d3d0 2024-05-20T18:27:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.573+0000 7fe9b8814640 1 -- 172.21.15.5:0/1987921738 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9b40d0630 con 0x7fe9b406d3d0 2024-05-20T18:27:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.573+0000 7fe9ba017640 1 -- 172.21.15.5:0/1987921738 >> v1:172.21.15.63:6789/0 conn(0x7fe9b406d3d0 legacy=0x7fe9b406d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.573+0000 7fe9ba017640 1 -- 172.21.15.5:0/1987921738 >> v1:172.21.15.5:6789/0 conn(0x7fe9b4162820 legacy=0x7fe9b40b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.573+0000 7fe9b8814640 1 -- 172.21.15.5:0/1987921738 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 670763183 0 0) 0x7fe99c003520 con 0x7fe9b4162820 2024-05-20T18:27:36.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.574+0000 7fe9ba017640 1 -- 172.21.15.5:0/1987921738 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b40d3f80 con 0x7fe9b4168050 2024-05-20T18:27:36.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.574+0000 7fe9ba017640 1 --2- 172.21.15.5:0/1987921738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9b4168050 0x7fe9b416a470 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac0097d0 tx=0x7fe9ac02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a16fbd0dd80d412f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:36.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.575+0000 7fe9b8814640 1 -- 172.21.15.5:0/1987921738 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac037070 con 0x7fe9b4168050 2024-05-20T18:27:36.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.575+0000 7fe9b8814640 1 -- 172.21.15.5:0/1987921738 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9ac02ac30 con 0x7fe9b4168050 2024-05-20T18:27:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.575+0000 7fe9b8814640 1 -- 172.21.15.5:0/1987921738 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a40019f0 con 0x7fe9b4168050 2024-05-20T18:27:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.575+0000 7fe9bba7a640 1 -- 172.21.15.5:0/1987921738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9b4168050 msgr2=0x7fe9b416a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.575+0000 7fe9bba7a640 1 --2- 172.21.15.5:0/1987921738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9b4168050 0x7fe9b416a470 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac0097d0 tx=0x7fe9ac02a1a0 comp rx=0 tx=0).stop 2024-05-20T18:27:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.575+0000 7fe9bba7a640 1 -- 172.21.15.5:0/1987921738 shutdown_connections 2024-05-20T18:27:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.575+0000 7fe9bba7a640 1 --2- 172.21.15.5:0/1987921738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9b4168050 0x7fe9b416a470 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.575+0000 7fe9bba7a640 1 -- 172.21.15.5:0/1987921738 >> 172.21.15.5:0/1987921738 conn(0x7fe9b40d4bd0 msgr2=0x7fe9b40d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.575+0000 7fe9bba7a640 1 -- 172.21.15.5:0/1987921738 shutdown_connections 2024-05-20T18:27:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.575+0000 7fe9bba7a640 1 -- 172.21.15.5:0/1987921738 wait complete. 2024-05-20T18:27:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.577+0000 7fe9bba7a640 1 Processor -- start 2024-05-20T18:27:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.577+0000 7fe9bba7a640 1 -- start start 2024-05-20T18:27:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.578+0000 7fe9bba7a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9b406d3d0 0x7fe9b415f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.579+0000 7fe9bba7a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9b4162820 0x7fe9b4150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.579+0000 7fe9bba7a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9b4168050 0x7fe9b415fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.579+0000 7fe9bba7a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b40b91a0 con 0x7fe9b4168050 2024-05-20T18:27:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.579+0000 7fe9bba7a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b40d0300 con 0x7fe9b4162820 2024-05-20T18:27:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.579+0000 7fe9bba7a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe9b40b4050 con 0x7fe9b406d3d0 2024-05-20T18:27:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.579+0000 7fe9ba017640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9b4168050 0x7fe9b415fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.579+0000 7fe9ba017640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9b4168050 0x7fe9b415fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52946/0 (socket says 172.21.15.5:52946) 2024-05-20T18:27:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.579+0000 7fe9ba017640 1 -- 172.21.15.5:0/109013089 learned_addr learned my addr 172.21.15.5:0/109013089 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.579+0000 7fe9b9816640 1 --2- 172.21.15.5:0/109013089 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9b406d3d0 0x7fe9b415f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.579+0000 7fe9b9015640 1 --2- 172.21.15.5:0/109013089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9b4162820 0x7fe9b4150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.579+0000 7fe9ba017640 1 -- 172.21.15.5:0/109013089 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9b406d3d0 msgr2=0x7fe9b415f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.579+0000 7fe9ba017640 1 --2- 172.21.15.5:0/109013089 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9b406d3d0 0x7fe9b415f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.579+0000 7fe9ba017640 1 -- 172.21.15.5:0/109013089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9b4162820 msgr2=0x7fe9b4150580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.579+0000 7fe9ba017640 1 --2- 172.21.15.5:0/109013089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9b4162820 0x7fe9b4150580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.579+0000 7fe9ba017640 1 -- 172.21.15.5:0/109013089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b4151bc0 con 0x7fe9b4168050 2024-05-20T18:27:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.579+0000 7fe9ba017640 1 --2- 172.21.15.5:0/109013089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9b4168050 0x7fe9b415fe90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac0097d0 tx=0x7fe9ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.579+0000 7fe9b27fc640 1 -- 172.21.15.5:0/109013089 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac037050 con 0x7fe9b4168050 2024-05-20T18:27:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.580+0000 7fe9b27fc640 1 -- 172.21.15.5:0/109013089 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9ac043070 con 0x7fe9b4168050 2024-05-20T18:27:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.580+0000 7fe9bba7a640 1 -- 172.21.15.5:0/109013089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b4151ea0 con 0x7fe9b4168050 2024-05-20T18:27:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.580+0000 7fe9b27fc640 1 -- 172.21.15.5:0/109013089 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac006e10 con 0x7fe9b4168050 2024-05-20T18:27:36.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.580+0000 7fe9bba7a640 1 -- 172.21.15.5:0/109013089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b4151010 con 0x7fe9b4168050 2024-05-20T18:27:36.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.580+0000 7fe9bba7a640 1 -- 172.21.15.5:0/109013089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9b40d4780 con 0x7fe9b4168050 2024-05-20T18:27:36.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.581+0000 7fe9b27fc640 1 -- 172.21.15.5:0/109013089 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe9ac03e650 con 0x7fe9b4168050 2024-05-20T18:27:36.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.582+0000 7fe9b27fc640 1 --2- 172.21.15.5:0/109013089 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe9a006be30 0x7fe9a006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:36.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.582+0000 7fe9b27fc640 1 -- 172.21.15.5:0/109013089 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe9ac0ab980 con 0x7fe9b4168050 2024-05-20T18:27:36.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.582+0000 7fe9b9816640 1 --2- 172.21.15.5:0/109013089 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe9a006be30 0x7fe9a006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:36.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.582+0000 7fe9b9816640 1 --2- 172.21.15.5:0/109013089 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe9a006be30 0x7fe9a006e340 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4001b60 tx=0x7fe9a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:36.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.586+0000 7fe9b27fc640 1 -- 172.21.15.5:0/109013089 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe9b40d4780 con 0x7fe9b4168050 2024-05-20T18:27:36.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.772+0000 7fe9bba7a640 1 -- 172.21.15.5:0/109013089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9b4075b00 con 0x7fe9b4168050 2024-05-20T18:27:36.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.773+0000 7fe9b27fc640 1 -- 172.21.15.5:0/109013089 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18394 (secure 0 0 0) 0x7fe9ac0799a0 con 0x7fe9b4168050 2024-05-20T18:27:36.773 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:36.773 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:27:13.817633+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:27:05.476596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T18:27:06.137762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T18:27:09.829920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:27:36.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.775+0000 7fe9bba7a640 1 -- 172.21.15.5:0/109013089 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe9a006be30 msgr2=0x7fe9a006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:36.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.775+0000 7fe9bba7a640 1 --2- 172.21.15.5:0/109013089 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe9a006be30 0x7fe9a006e340 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4001b60 tx=0x7fe9a4009040 comp rx=0 tx=0).stop 2024-05-20T18:27:36.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.775+0000 7fe9bba7a640 1 -- 172.21.15.5:0/109013089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9b4168050 msgr2=0x7fe9b415fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:36.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.775+0000 7fe9bba7a640 1 --2- 172.21.15.5:0/109013089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9b4168050 0x7fe9b415fe90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac0097d0 tx=0x7fe9ac0046c0 comp rx=0 tx=0).stop 2024-05-20T18:27:36.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.775+0000 7fe9bba7a640 1 -- 172.21.15.5:0/109013089 shutdown_connections 2024-05-20T18:27:36.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.775+0000 7fe9bba7a640 1 --2- 172.21.15.5:0/109013089 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe9a006be30 0x7fe9a006e340 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:36.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.775+0000 7fe9bba7a640 1 --2- 172.21.15.5:0/109013089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9b4168050 0x7fe9b415fe90 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:36.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.775+0000 7fe9bba7a640 1 --2- 172.21.15.5:0/109013089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9b4162820 0x7fe9b4150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:36.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.775+0000 7fe9bba7a640 1 --2- 172.21.15.5:0/109013089 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9b406d3d0 0x7fe9b415f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:36.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.775+0000 7fe9bba7a640 1 -- 172.21.15.5:0/109013089 >> 172.21.15.5:0/109013089 conn(0x7fe9b40d4bd0 msgr2=0x7fe9b415d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:36.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.776+0000 7fe9bba7a640 1 -- 172.21.15.5:0/109013089 shutdown_connections 2024-05-20T18:27:36.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.776+0000 7fe9bba7a640 1 -- 172.21.15.5:0/109013089 wait complete. 2024-05-20T18:27:36.787 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:27:05.476596+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:27:06.137762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T18:27:09.829920+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T18:27:36.787 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:27:36.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.906+0000 7fae8b691640 1 Processor -- start 2024-05-20T18:27:36.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.907+0000 7fae8b691640 1 -- start start 2024-05-20T18:27:36.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.908+0000 7fae8b691640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fae84163a10 0x7fae840d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:36.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.908+0000 7fae8b691640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae840d14b0 0x7fae840d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:36.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.908+0000 7fae8b691640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae840d0370 con 0x7fae840b4010 2024-05-20T18:27:36.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.908+0000 7fae8b691640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fae840d3ca0 con 0x7fae84163a10 2024-05-20T18:27:36.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.908+0000 7fae8b691640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fae840d0650 con 0x7fae840d14b0 2024-05-20T18:27:36.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.908+0000 7fae8942d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fae84163a10 0x7fae840d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:36.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.908+0000 7fae89c2e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae840d14b0 0x7fae840d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:36.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.908+0000 7fae8942d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fae84163a10 0x7fae840d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:41598/0 (socket says 172.21.15.5:41598) 2024-05-20T18:27:36.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.908+0000 7fae8942d640 1 -- 172.21.15.5:0/2338719526 learned_addr learned my addr 172.21.15.5:0/2338719526 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:36.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.908+0000 7fae88c2c640 1 --1- 172.21.15.5:0/2338719526 >> v1:172.21.15.63:6789/0 conn(0x7fae840b4010 0x7fae840cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:45174/0 (socket says 172.21.15.5:45174) 2024-05-20T18:27:36.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.909+0000 7fae8942d640 1 -- 172.21.15.5:0/2338719526 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae840d14b0 msgr2=0x7fae840d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:36.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.909+0000 7fae8942d640 1 --2- 172.21.15.5:0/2338719526 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae840d14b0 0x7fae840d18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:36.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.909+0000 7fae8942d640 1 -- 172.21.15.5:0/2338719526 >> v1:172.21.15.63:6789/0 conn(0x7fae840b4010 legacy=0x7fae840cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:36.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.909+0000 7fae8942d640 1 -- 172.21.15.5:0/2338719526 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae840d3650 con 0x7fae84163a10 2024-05-20T18:27:36.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.910+0000 7fae8942d640 1 --2- 172.21.15.5:0/2338719526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae84163a10 0x7fae840d4130 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fae80009870 tx=0x7fae8002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c515aaa50e9d37c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:36.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.910+0000 7fae7bfff640 1 -- 172.21.15.5:0/2338719526 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae80037070 con 0x7fae84163a10 2024-05-20T18:27:36.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.910+0000 7fae7bfff640 1 -- 172.21.15.5:0/2338719526 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae80032440 con 0x7fae84163a10 2024-05-20T18:27:36.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.910+0000 7fae7bfff640 1 -- 172.21.15.5:0/2338719526 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae80032810 con 0x7fae84163a10 2024-05-20T18:27:36.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.910+0000 7fae8b691640 1 -- 172.21.15.5:0/2338719526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae84163a10 msgr2=0x7fae840d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:36.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.910+0000 7fae8b691640 1 --2- 172.21.15.5:0/2338719526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae84163a10 0x7fae840d4130 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fae80009870 tx=0x7fae8002a270 comp rx=0 tx=0).stop 2024-05-20T18:27:36.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.911+0000 7fae8b691640 1 -- 172.21.15.5:0/2338719526 shutdown_connections 2024-05-20T18:27:36.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.911+0000 7fae8b691640 1 --2- 172.21.15.5:0/2338719526 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae840d14b0 0x7fae840d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:36.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.911+0000 7fae8b691640 1 --2- 172.21.15.5:0/2338719526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae84163a10 0x7fae840d4130 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:36.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.911+0000 7fae8b691640 1 -- 172.21.15.5:0/2338719526 >> 172.21.15.5:0/2338719526 conn(0x7fae8406d240 msgr2=0x7fae8406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:36.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.911+0000 7fae8b691640 1 -- 172.21.15.5:0/2338719526 shutdown_connections 2024-05-20T18:27:36.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.911+0000 7fae8b691640 1 -- 172.21.15.5:0/2338719526 wait complete. 2024-05-20T18:27:36.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.912+0000 7fae8b691640 1 Processor -- start 2024-05-20T18:27:36.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.913+0000 7fae8b691640 1 -- start start 2024-05-20T18:27:36.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.914+0000 7fae8b691640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae840b4010 0x7fae8415b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:36.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.914+0000 7fae8b691640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae840d14b0 0x7fae84153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:36.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.914+0000 7fae8b691640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae84163a10 0x7fae8415dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:36.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.914+0000 7fae8b691640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fae840b8210 con 0x7fae84163a10 2024-05-20T18:27:36.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.914+0000 7fae8b691640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fae840d1e00 con 0x7fae840b4010 2024-05-20T18:27:36.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.914+0000 7fae8b691640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fae840b8390 con 0x7fae840d14b0 2024-05-20T18:27:36.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.914+0000 7fae89c2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae84163a10 0x7fae8415dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:36.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.914+0000 7fae89c2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae84163a10 0x7fae8415dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52956/0 (socket says 172.21.15.5:52956) 2024-05-20T18:27:36.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.914+0000 7fae89c2e640 1 -- 172.21.15.5:0/3651214895 learned_addr learned my addr 172.21.15.5:0/3651214895 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:36.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.914+0000 7fae8942d640 1 --2- 172.21.15.5:0/3651214895 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae840b4010 0x7fae8415b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:36.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.914+0000 7fae88c2c640 1 --2- 172.21.15.5:0/3651214895 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae840d14b0 0x7fae84153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:36.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.914+0000 7fae89c2e640 1 -- 172.21.15.5:0/3651214895 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae840d14b0 msgr2=0x7fae84153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:36.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.914+0000 7fae89c2e640 1 --2- 172.21.15.5:0/3651214895 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae840d14b0 0x7fae84153940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:36.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.914+0000 7fae89c2e640 1 -- 172.21.15.5:0/3651214895 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae840b4010 msgr2=0x7fae8415b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:36.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.914+0000 7fae89c2e640 1 --2- 172.21.15.5:0/3651214895 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae840b4010 0x7fae8415b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:36.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.914+0000 7fae89c2e640 1 -- 172.21.15.5:0/3651214895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8415a790 con 0x7fae84163a10 2024-05-20T18:27:36.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.915+0000 7fae89c2e640 1 --2- 172.21.15.5:0/3651214895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae84163a10 0x7fae8415dd30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00efc0 tx=0x7fae7c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:36.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.915+0000 7fae79ffb640 1 -- 172.21.15.5:0/3651214895 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c018020 con 0x7fae84163a10 2024-05-20T18:27:36.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.915+0000 7fae79ffb640 1 -- 172.21.15.5:0/3651214895 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae7c014480 con 0x7fae84163a10 2024-05-20T18:27:36.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.915+0000 7fae8b691640 1 -- 172.21.15.5:0/3651214895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae841596f0 con 0x7fae84163a10 2024-05-20T18:27:36.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.915+0000 7fae79ffb640 1 -- 172.21.15.5:0/3651214895 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c009e60 con 0x7fae84163a10 2024-05-20T18:27:36.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.915+0000 7fae8b691640 1 -- 172.21.15.5:0/3651214895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae84159cb0 con 0x7fae84163a10 2024-05-20T18:27:36.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.916+0000 7fae8b691640 1 -- 172.21.15.5:0/3651214895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae84075b00 con 0x7fae84163a10 2024-05-20T18:27:36.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.917+0000 7fae79ffb640 1 -- 172.21.15.5:0/3651214895 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fae7c005d10 con 0x7fae84163a10 2024-05-20T18:27:36.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.917+0000 7fae79ffb640 1 --2- 172.21.15.5:0/3651214895 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae6806be60 0x7fae6806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:36.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.918+0000 7fae8942d640 1 --2- 172.21.15.5:0/3651214895 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae6806be60 0x7fae6806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:36.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.918+0000 7fae79ffb640 1 -- 172.21.15.5:0/3651214895 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fae7c08be60 con 0x7fae84163a10 2024-05-20T18:27:36.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.918+0000 7fae8942d640 1 --2- 172.21.15.5:0/3651214895 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae6806be60 0x7fae6806e370 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fae80034000 tx=0x7fae8002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:36.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:36.922+0000 7fae79ffb640 1 -- 172.21.15.5:0/3651214895 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fae7c022030 con 0x7fae84163a10 2024-05-20T18:27:37.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.105+0000 7fae8b691640 1 -- 172.21.15.5:0/3651214895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fae840d3050 con 0x7fae84163a10 2024-05-20T18:27:37.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.106+0000 7fae79ffb640 1 -- 172.21.15.5:0/3651214895 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fae8415a790 con 0x7fae84163a10 2024-05-20T18:27:37.105 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:27:37.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.108+0000 7fae8b691640 1 -- 172.21.15.5:0/3651214895 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae6806be60 msgr2=0x7fae6806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:37.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.108+0000 7fae8b691640 1 --2- 172.21.15.5:0/3651214895 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae6806be60 0x7fae6806e370 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fae80034000 tx=0x7fae8002e000 comp rx=0 tx=0).stop 2024-05-20T18:27:37.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.108+0000 7fae8b691640 1 -- 172.21.15.5:0/3651214895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae84163a10 msgr2=0x7fae8415dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:37.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.108+0000 7fae8b691640 1 --2- 172.21.15.5:0/3651214895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae84163a10 0x7fae8415dd30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00efc0 tx=0x7fae7c00c490 comp rx=0 tx=0).stop 2024-05-20T18:27:37.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.108+0000 7fae8b691640 1 -- 172.21.15.5:0/3651214895 shutdown_connections 2024-05-20T18:27:37.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.108+0000 7fae8b691640 1 --2- 172.21.15.5:0/3651214895 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae6806be60 0x7fae6806e370 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.108+0000 7fae8b691640 1 --2- 172.21.15.5:0/3651214895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae84163a10 0x7fae8415dd30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.108+0000 7fae8b691640 1 --2- 172.21.15.5:0/3651214895 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae840d14b0 0x7fae84153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.108+0000 7fae8b691640 1 --2- 172.21.15.5:0/3651214895 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae840b4010 0x7fae8415b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.108+0000 7fae8b691640 1 -- 172.21.15.5:0/3651214895 >> 172.21.15.5:0/3651214895 conn(0x7fae8406d240 msgr2=0x7fae840b5520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:37.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.108+0000 7fae8b691640 1 -- 172.21.15.5:0/3651214895 shutdown_connections 2024-05-20T18:27:37.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.108+0000 7fae8b691640 1 -- 172.21.15.5:0/3651214895 wait complete. 2024-05-20T18:27:37.119 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:27:37.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.238+0000 7f7b5b444640 1 Processor -- start 2024-05-20T18:27:37.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.238+0000 7f7b5b444640 1 -- start start 2024-05-20T18:27:37.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.240+0000 7f7b5b444640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b54162820 0x7f7b540b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:37.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.240+0000 7f7b5b444640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7b5406dae0 0x7f7b540b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:37.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.240+0000 7f7b5b444640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b540d0630 con 0x7f7b540b3e80 2024-05-20T18:27:37.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.240+0000 7f7b5b444640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f7b540d39c0 con 0x7f7b5406dae0 2024-05-20T18:27:37.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.240+0000 7f7b5b444640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7b540d3ca0 con 0x7f7b54162820 2024-05-20T18:27:37.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.240+0000 7f7b599e1640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7b5406dae0 0x7f7b540b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:37.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.240+0000 7f7b599e1640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7b5406dae0 0x7f7b540b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52966/0 (socket says 172.21.15.5:52966) 2024-05-20T18:27:37.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.240+0000 7f7b599e1640 1 -- 172.21.15.5:0/349708349 learned_addr learned my addr 172.21.15.5:0/349708349 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:37.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.240+0000 7f7b591e0640 1 --2- 172.21.15.5:0/349708349 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b54162820 0x7f7b540b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:37.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.240+0000 7f7b599e1640 1 -- 172.21.15.5:0/349708349 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b54162820 msgr2=0x7f7b540b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:37.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.240+0000 7f7b599e1640 1 --2- 172.21.15.5:0/349708349 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b54162820 0x7f7b540b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.240+0000 7f7b599e1640 1 -- 172.21.15.5:0/349708349 >> v1:172.21.15.63:6789/0 conn(0x7f7b540b3e80 legacy=0x7f7b5406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:37.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.240+0000 7f7b599e1640 1 -- 172.21.15.5:0/349708349 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b540d3f80 con 0x7f7b5406dae0 2024-05-20T18:27:37.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.240+0000 7f7b591e0640 1 --2- 172.21.15.5:0/349708349 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b54162820 0x7f7b540b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:37.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.241+0000 7f7b599e1640 1 --2- 172.21.15.5:0/349708349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b5406dae0 0x7f7b540b8190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c009e50 tx=0x7f7b4c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ae3d70afd0017c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:37.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.241+0000 7f7b4bfff640 1 -- 172.21.15.5:0/349708349 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c036070 con 0x7f7b5406dae0 2024-05-20T18:27:37.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.241+0000 7f7b4bfff640 1 -- 172.21.15.5:0/349708349 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b4c031600 con 0x7f7b5406dae0 2024-05-20T18:27:37.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.241+0000 7f7b4bfff640 1 -- 172.21.15.5:0/349708349 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c0319d0 con 0x7f7b5406dae0 2024-05-20T18:27:37.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.242+0000 7f7b5b444640 1 -- 172.21.15.5:0/349708349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b5406dae0 msgr2=0x7f7b540b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:37.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.242+0000 7f7b5b444640 1 --2- 172.21.15.5:0/349708349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b5406dae0 0x7f7b540b8190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c009e50 tx=0x7f7b4c02a6c0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.242+0000 7f7b5b444640 1 -- 172.21.15.5:0/349708349 shutdown_connections 2024-05-20T18:27:37.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.242+0000 7f7b5b444640 1 --2- 172.21.15.5:0/349708349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b5406dae0 0x7f7b540b8190 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.242+0000 7f7b5b444640 1 --2- 172.21.15.5:0/349708349 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b54162820 0x7f7b540b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.242+0000 7f7b5b444640 1 -- 172.21.15.5:0/349708349 >> 172.21.15.5:0/349708349 conn(0x7f7b540d4bd0 msgr2=0x7f7b540d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:37.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.242+0000 7f7b5b444640 1 -- 172.21.15.5:0/349708349 shutdown_connections 2024-05-20T18:27:37.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.242+0000 7f7b5b444640 1 -- 172.21.15.5:0/349708349 wait complete. 2024-05-20T18:27:37.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.244+0000 7f7b5b444640 1 Processor -- start 2024-05-20T18:27:37.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.244+0000 7f7b5b444640 1 -- start start 2024-05-20T18:27:37.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.245+0000 7f7b5b444640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b540b3e80 0x7f7b540cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:37.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.245+0000 7f7b5b444640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b54162820 0x7f7b5415b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:37.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.245+0000 7f7b5b444640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b5415b650 0x7f7b5414acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:37.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.245+0000 7f7b5b444640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7b540d4780 con 0x7f7b540b3e80 2024-05-20T18:27:37.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.245+0000 7f7b5b444640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7b540d0300 con 0x7f7b54162820 2024-05-20T18:27:37.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.245+0000 7f7b5b444640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7b540d4900 con 0x7f7b5415b650 2024-05-20T18:27:37.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.245+0000 7f7b591e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b540b3e80 0x7f7b540cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:37.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.245+0000 7f7b591e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b540b3e80 0x7f7b540cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52968/0 (socket says 172.21.15.5:52968) 2024-05-20T18:27:37.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.245+0000 7f7b591e0640 1 -- 172.21.15.5:0/2596796248 learned_addr learned my addr 172.21.15.5:0/2596796248 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:37.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.246+0000 7f7b589df640 1 --2- 172.21.15.5:0/2596796248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b54162820 0x7f7b5415b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:37.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.246+0000 7f7b599e1640 1 --2- 172.21.15.5:0/2596796248 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b5415b650 0x7f7b5414acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:37.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.246+0000 7f7b591e0640 1 -- 172.21.15.5:0/2596796248 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b5415b650 msgr2=0x7f7b5414acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:37.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.246+0000 7f7b591e0640 1 --2- 172.21.15.5:0/2596796248 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b5415b650 0x7f7b5414acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.246+0000 7f7b591e0640 1 -- 172.21.15.5:0/2596796248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b54162820 msgr2=0x7f7b5415b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:37.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.246+0000 7f7b591e0640 1 --2- 172.21.15.5:0/2596796248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b54162820 0x7f7b5415b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.246+0000 7f7b591e0640 1 -- 172.21.15.5:0/2596796248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b540b4d80 con 0x7f7b540b3e80 2024-05-20T18:27:37.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.246+0000 7f7b591e0640 1 --2- 172.21.15.5:0/2596796248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b540b3e80 0x7f7b540cb5b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000c7f0 tx=0x7f7b5000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:37.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.246+0000 7f7b49ffb640 1 -- 172.21.15.5:0/2596796248 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b50015070 con 0x7f7b540b3e80 2024-05-20T18:27:37.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.246+0000 7f7b49ffb640 1 -- 172.21.15.5:0/2596796248 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b50007820 con 0x7f7b540b3e80 2024-05-20T18:27:37.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.246+0000 7f7b5b444640 1 -- 172.21.15.5:0/2596796248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b540b5060 con 0x7f7b540b3e80 2024-05-20T18:27:37.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.246+0000 7f7b49ffb640 1 -- 172.21.15.5:0/2596796248 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b50007bd0 con 0x7f7b540b3e80 2024-05-20T18:27:37.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.246+0000 7f7b5b444640 1 -- 172.21.15.5:0/2596796248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b540b4630 con 0x7f7b540b3e80 2024-05-20T18:27:37.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.247+0000 7f7b5b444640 1 -- 172.21.15.5:0/2596796248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b54075b00 con 0x7f7b540b3e80 2024-05-20T18:27:37.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.248+0000 7f7b49ffb640 1 -- 172.21.15.5:0/2596796248 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7b50018070 con 0x7f7b540b3e80 2024-05-20T18:27:37.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.248+0000 7f7b49ffb640 1 --2- 172.21.15.5:0/2596796248 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7b3806be30 0x7f7b3806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:37.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.248+0000 7f7b49ffb640 1 -- 172.21.15.5:0/2596796248 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f7b5008b3b0 con 0x7f7b540b3e80 2024-05-20T18:27:37.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.248+0000 7f7b589df640 1 --2- 172.21.15.5:0/2596796248 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7b3806be30 0x7f7b3806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:37.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.249+0000 7f7b589df640 1 --2- 172.21.15.5:0/2596796248 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7b3806be30 0x7f7b3806e340 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7b541e92f0 tx=0x7f7b44008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:37.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.253+0000 7f7b49ffb640 1 -- 172.21.15.5:0/2596796248 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7b50059980 con 0x7f7b540b3e80 2024-05-20T18:27:37.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.435+0000 7f7b5b444640 1 -- 172.21.15.5:0/2596796248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7b540b48f0 con 0x7f7b540b3e80 2024-05-20T18:27:37.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.435+0000 7f7b49ffb640 1 -- 172.21.15.5:0/2596796248 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f7b50059280 con 0x7f7b540b3e80 2024-05-20T18:27:37.435 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:27:37.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.437+0000 7f7b5b444640 1 -- 172.21.15.5:0/2596796248 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7b3806be30 msgr2=0x7f7b3806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:37.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.437+0000 7f7b5b444640 1 --2- 172.21.15.5:0/2596796248 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7b3806be30 0x7f7b3806e340 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7b541e92f0 tx=0x7f7b44008040 comp rx=0 tx=0).stop 2024-05-20T18:27:37.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.437+0000 7f7b5b444640 1 -- 172.21.15.5:0/2596796248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b540b3e80 msgr2=0x7f7b540cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:37.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.437+0000 7f7b5b444640 1 --2- 172.21.15.5:0/2596796248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b540b3e80 0x7f7b540cb5b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000c7f0 tx=0x7f7b5000ccb0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.437+0000 7f7b5b444640 1 -- 172.21.15.5:0/2596796248 shutdown_connections 2024-05-20T18:27:37.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.437+0000 7f7b5b444640 1 --2- 172.21.15.5:0/2596796248 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7b3806be30 0x7f7b3806e340 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.437+0000 7f7b5b444640 1 --2- 172.21.15.5:0/2596796248 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b5415b650 0x7f7b5414acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.437+0000 7f7b5b444640 1 --2- 172.21.15.5:0/2596796248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b54162820 0x7f7b5415b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.437+0000 7f7b5b444640 1 --2- 172.21.15.5:0/2596796248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b540b3e80 0x7f7b540cb5b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.437+0000 7f7b5b444640 1 -- 172.21.15.5:0/2596796248 >> 172.21.15.5:0/2596796248 conn(0x7f7b540d4bd0 msgr2=0x7f7b540ba3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:37.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.437+0000 7f7b5b444640 1 -- 172.21.15.5:0/2596796248 shutdown_connections 2024-05-20T18:27:37.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.437+0000 7f7b5b444640 1 -- 172.21.15.5:0/2596796248 wait complete. 2024-05-20T18:27:37.448 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:27:37.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.566+0000 7f3d4bc14640 1 Processor -- start 2024-05-20T18:27:37.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.566+0000 7f3d4bc14640 1 -- start start 2024-05-20T18:27:37.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.567+0000 7f3d4bc14640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3d44162820 0x7f3d440b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:37.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.568+0000 7f3d4bc14640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d4406dae0 0x7f3d440b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:37.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.568+0000 7f3d4bc14640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d440d0630 con 0x7f3d440b3e80 2024-05-20T18:27:37.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.568+0000 7f3d4bc14640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f3d440d39c0 con 0x7f3d44162820 2024-05-20T18:27:37.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.568+0000 7f3d4bc14640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3d440d3ca0 con 0x7f3d4406dae0 2024-05-20T18:27:37.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.568+0000 7f3d4a1b1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d4406dae0 0x7f3d440b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:37.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.568+0000 7f3d499b0640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3d44162820 0x7f3d440b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:37.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.568+0000 7f3d4a1b1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d4406dae0 0x7f3d440b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:37254/0 (socket says 172.21.15.5:37254) 2024-05-20T18:27:37.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.568+0000 7f3d4a1b1640 1 -- 172.21.15.5:0/562731510 learned_addr learned my addr 172.21.15.5:0/562731510 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:37.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.568+0000 7f3d4a1b1640 1 -- 172.21.15.5:0/562731510 >> v2:172.21.15.63:3300/0 conn(0x7f3d44162820 msgr2=0x7f3d440b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:37.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.568+0000 7f3d4a1b1640 1 --2- 172.21.15.5:0/562731510 >> v2:172.21.15.63:3300/0 conn(0x7f3d44162820 0x7f3d440b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.568+0000 7f3d4a1b1640 1 -- 172.21.15.5:0/562731510 >> v1:172.21.15.63:6789/0 conn(0x7f3d440b3e80 legacy=0x7f3d4406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:37.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.568+0000 7f3d4a1b1640 1 -- 172.21.15.5:0/562731510 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d440d3f80 con 0x7f3d4406dae0 2024-05-20T18:27:37.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.569+0000 7f3d4a1b1640 1 --2- 172.21.15.5:0/562731510 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d4406dae0 0x7f3d440b8190 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c009870 tx=0x7f3d3c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aff6dd8130b7cbf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:37.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.569+0000 7f3d489ae640 1 -- 172.21.15.5:0/562731510 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c037070 con 0x7f3d4406dae0 2024-05-20T18:27:37.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.569+0000 7f3d489ae640 1 -- 172.21.15.5:0/562731510 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d3c032440 con 0x7f3d4406dae0 2024-05-20T18:27:37.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.569+0000 7f3d489ae640 1 -- 172.21.15.5:0/562731510 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c032810 con 0x7f3d4406dae0 2024-05-20T18:27:37.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.570+0000 7f3d4bc14640 1 -- 172.21.15.5:0/562731510 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d4406dae0 msgr2=0x7f3d440b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:37.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.570+0000 7f3d4bc14640 1 --2- 172.21.15.5:0/562731510 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d4406dae0 0x7f3d440b8190 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c009870 tx=0x7f3d3c02a270 comp rx=0 tx=0).stop 2024-05-20T18:27:37.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.570+0000 7f3d4bc14640 1 -- 172.21.15.5:0/562731510 shutdown_connections 2024-05-20T18:27:37.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.570+0000 7f3d4bc14640 1 --2- 172.21.15.5:0/562731510 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d4406dae0 0x7f3d440b8190 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.570+0000 7f3d4bc14640 1 --2- 172.21.15.5:0/562731510 >> v2:172.21.15.63:3300/0 conn(0x7f3d44162820 0x7f3d440b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.570+0000 7f3d4bc14640 1 -- 172.21.15.5:0/562731510 >> 172.21.15.5:0/562731510 conn(0x7f3d440d4bd0 msgr2=0x7f3d440d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:37.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.570+0000 7f3d4bc14640 1 -- 172.21.15.5:0/562731510 shutdown_connections 2024-05-20T18:27:37.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.570+0000 7f3d4bc14640 1 -- 172.21.15.5:0/562731510 wait complete. 2024-05-20T18:27:37.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.572+0000 7f3d4bc14640 1 Processor -- start 2024-05-20T18:27:37.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.572+0000 7f3d4bc14640 1 -- start start 2024-05-20T18:27:37.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.573+0000 7f3d4bc14640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d440b3e80 0x7f3d440cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:37.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.573+0000 7f3d4bc14640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d44162820 0x7f3d441e7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:37.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.573+0000 7f3d4bc14640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d441e8300 0x7f3d44149a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:37.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.573+0000 7f3d4bc14640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3d440d4780 con 0x7f3d44162820 2024-05-20T18:27:37.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.573+0000 7f3d4bc14640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3d440d0300 con 0x7f3d440b3e80 2024-05-20T18:27:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.573+0000 7f3d4bc14640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3d440d4900 con 0x7f3d441e8300 2024-05-20T18:27:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.573+0000 7f3d491af640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d44162820 0x7f3d441e7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.574+0000 7f3d491af640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d44162820 0x7f3d441e7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52982/0 (socket says 172.21.15.5:52982) 2024-05-20T18:27:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.574+0000 7f3d491af640 1 -- 172.21.15.5:0/413548115 learned_addr learned my addr 172.21.15.5:0/413548115 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.574+0000 7f3d4a1b1640 1 --2- 172.21.15.5:0/413548115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d441e8300 0x7f3d44149a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.574+0000 7f3d499b0640 1 --2- 172.21.15.5:0/413548115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d440b3e80 0x7f3d440cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.574+0000 7f3d491af640 1 -- 172.21.15.5:0/413548115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d441e8300 msgr2=0x7f3d44149a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.574+0000 7f3d491af640 1 --2- 172.21.15.5:0/413548115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d441e8300 0x7f3d44149a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.574+0000 7f3d491af640 1 -- 172.21.15.5:0/413548115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d440b3e80 msgr2=0x7f3d440cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.574+0000 7f3d491af640 1 --2- 172.21.15.5:0/413548115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d440b3e80 0x7f3d440cdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.574+0000 7f3d491af640 1 -- 172.21.15.5:0/413548115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d4415a750 con 0x7f3d44162820 2024-05-20T18:27:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.574+0000 7f3d499b0640 1 --2- 172.21.15.5:0/413548115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d440b3e80 0x7f3d440cdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.574+0000 7f3d491af640 1 --2- 172.21.15.5:0/413548115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d44162820 0x7f3d441e7dc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3d3400c3f0 tx=0x7f3d3400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:37.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.574+0000 7f3d3a7fc640 1 -- 172.21.15.5:0/413548115 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d340052d0 con 0x7f3d44162820 2024-05-20T18:27:37.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.574+0000 7f3d3a7fc640 1 -- 172.21.15.5:0/413548115 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d34005490 con 0x7f3d44162820 2024-05-20T18:27:37.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.574+0000 7f3d4bc14640 1 -- 172.21.15.5:0/413548115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d4415aa30 con 0x7f3d44162820 2024-05-20T18:27:37.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.574+0000 7f3d3a7fc640 1 -- 172.21.15.5:0/413548115 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d34007260 con 0x7f3d44162820 2024-05-20T18:27:37.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.574+0000 7f3d4bc14640 1 -- 172.21.15.5:0/413548115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d4415aff0 con 0x7f3d44162820 2024-05-20T18:27:37.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.575+0000 7f3d4bc14640 1 -- 172.21.15.5:0/413548115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d44075b00 con 0x7f3d44162820 2024-05-20T18:27:37.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.576+0000 7f3d3a7fc640 1 -- 172.21.15.5:0/413548115 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3d34007400 con 0x7f3d44162820 2024-05-20T18:27:37.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.576+0000 7f3d3a7fc640 1 --2- 172.21.15.5:0/413548115 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3d2806be60 0x7f3d2806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:37.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.577+0000 7f3d3a7fc640 1 -- 172.21.15.5:0/413548115 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f3d3408a7a0 con 0x7f3d44162820 2024-05-20T18:27:37.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.577+0000 7f3d499b0640 1 --2- 172.21.15.5:0/413548115 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3d2806be60 0x7f3d2806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:37.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.577+0000 7f3d499b0640 1 --2- 172.21.15.5:0/413548115 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3d2806be60 0x7f3d2806e370 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3d40007440 tx=0x7f3d4000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:37.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.581+0000 7f3d3a7fc640 1 -- 172.21.15.5:0/413548115 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3d34058d00 con 0x7f3d44162820 2024-05-20T18:27:37.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.767+0000 7f3d4bc14640 1 -- 172.21.15.5:0/413548115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3d44161fd0 con 0x7f3d44162820 2024-05-20T18:27:37.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.768+0000 7f3d3a7fc640 1 -- 172.21.15.5:0/413548115 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f3d34058600 con 0x7f3d44162820 2024-05-20T18:27:37.767 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:27:37.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.770+0000 7f3d4bc14640 1 -- 172.21.15.5:0/413548115 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3d2806be60 msgr2=0x7f3d2806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:37.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.770+0000 7f3d4bc14640 1 --2- 172.21.15.5:0/413548115 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3d2806be60 0x7f3d2806e370 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3d40007440 tx=0x7f3d4000b040 comp rx=0 tx=0).stop 2024-05-20T18:27:37.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.770+0000 7f3d4bc14640 1 -- 172.21.15.5:0/413548115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d44162820 msgr2=0x7f3d441e7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:37.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.770+0000 7f3d4bc14640 1 --2- 172.21.15.5:0/413548115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d44162820 0x7f3d441e7dc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3d3400c3f0 tx=0x7f3d3400c8b0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.770+0000 7f3d4bc14640 1 -- 172.21.15.5:0/413548115 shutdown_connections 2024-05-20T18:27:37.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.770+0000 7f3d4bc14640 1 --2- 172.21.15.5:0/413548115 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3d2806be60 0x7f3d2806e370 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.770+0000 7f3d4bc14640 1 --2- 172.21.15.5:0/413548115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d441e8300 0x7f3d44149a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.770+0000 7f3d4bc14640 1 --2- 172.21.15.5:0/413548115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d44162820 0x7f3d441e7dc0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.770+0000 7f3d4bc14640 1 --2- 172.21.15.5:0/413548115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d440b3e80 0x7f3d440cdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.770+0000 7f3d4bc14640 1 -- 172.21.15.5:0/413548115 >> 172.21.15.5:0/413548115 conn(0x7f3d440d4bd0 msgr2=0x7f3d440d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:37.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.770+0000 7f3d4bc14640 1 -- 172.21.15.5:0/413548115 shutdown_connections 2024-05-20T18:27:37.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.770+0000 7f3d4bc14640 1 -- 172.21.15.5:0/413548115 wait complete. 2024-05-20T18:27:37.781 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T18:27:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.899+0000 7f18130bc640 1 Processor -- start 2024-05-20T18:27:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.899+0000 7f18130bc640 1 -- start start 2024-05-20T18:27:37.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.900+0000 7f18130bc640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f180c06d3d0 0x7f180c06d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:37.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.900+0000 7f18130bc640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f180c06dd20 con 0x7f180c0b8190 2024-05-20T18:27:37.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.900+0000 7f18130bc640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f180c0d0630 con 0x7f180c162820 2024-05-20T18:27:37.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.900+0000 7f18130bc640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f180c0d39c0 con 0x7f180c06d3d0 2024-05-20T18:27:37.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.900+0000 7f1811659640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f180c0b8190 0x7f180c0b85a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:41024/0 (socket says 172.21.15.5:41024) 2024-05-20T18:27:37.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.900+0000 7f1811659640 1 -- 172.21.15.5:0/2284522130 learned_addr learned my addr 172.21.15.5:0/2284522130 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:37.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.901+0000 7f1803fff640 1 --2- 172.21.15.5:0/2284522130 >> v2:172.21.15.63:3300/0 conn(0x7f180c06d3d0 0x7f180c06d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:37.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.901+0000 7f18037fe640 1 -- 172.21.15.5:0/2284522130 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2319383270 0 0) 0x7f1804001490 con 0x7f180c0b8190 2024-05-20T18:27:37.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.901+0000 7f18037fe640 1 -- 172.21.15.5:0/2284522130 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17f4003520 con 0x7f180c0b8190 2024-05-20T18:27:37.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.901+0000 7f18037fe640 1 -- 172.21.15.5:0/2284522130 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3532470993 0 0) 0x7f17f4003520 con 0x7f180c0b8190 2024-05-20T18:27:37.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.901+0000 7f18037fe640 1 -- 172.21.15.5:0/2284522130 >> v2:172.21.15.63:3300/0 conn(0x7f180c06d3d0 msgr2=0x7f180c06d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:37.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.901+0000 7f18037fe640 1 --2- 172.21.15.5:0/2284522130 >> v2:172.21.15.63:3300/0 conn(0x7f180c06d3d0 0x7f180c06d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.901+0000 7f18037fe640 1 -- 172.21.15.5:0/2284522130 >> v1:172.21.15.63:6789/0 conn(0x7f180c162820 legacy=0x7f180c0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:37.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.901+0000 7f18037fe640 1 -- 172.21.15.5:0/2284522130 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f180c0d3ca0 con 0x7f180c0b8190 2024-05-20T18:27:37.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.901+0000 7f18037fe640 1 -- 172.21.15.5:0/2284522130 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f1804001bc0 con 0x7f180c0b8190 2024-05-20T18:27:37.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.901+0000 7f18037fe640 1 -- 172.21.15.5:0/2284522130 >> v1:172.21.15.5:6789/0 conn(0x7f180c0b8190 legacy=0x7f180c0b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:37.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.901+0000 7f18037fe640 1 --2- 172.21.15.5:0/2284522130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17f40039b0 0x7f17f4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:37.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.901+0000 7f18037fe640 1 --2- 172.21.15.5:0/2284522130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17f4004800 0x7f17f4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:37.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.901+0000 7f18037fe640 1 --2- 172.21.15.5:0/2284522130 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17f4005150 0x7f17f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:37.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.901+0000 7f18037fe640 1 -- 172.21.15.5:0/2284522130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f180c0d39c0 con 0x7f17f40039b0 2024-05-20T18:27:37.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.901+0000 7f18037fe640 1 -- 172.21.15.5:0/2284522130 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f17f400c0c0 con 0x7f17f4004800 2024-05-20T18:27:37.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.901+0000 7f18037fe640 1 -- 172.21.15.5:0/2284522130 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f17f400c2c0 con 0x7f17f4005150 2024-05-20T18:27:37.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.901+0000 7f1810e58640 1 --2- 172.21.15.5:0/2284522130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17f40039b0 0x7f17f4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:37.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.901+0000 7f1803fff640 1 --2- 172.21.15.5:0/2284522130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17f4004800 0x7f17f4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:37.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.901+0000 7f1811659640 1 --2- 172.21.15.5:0/2284522130 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17f4005150 0x7f17f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:37.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.901+0000 7f1810e58640 1 -- 172.21.15.5:0/2284522130 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17f4005150 msgr2=0x7f17f400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:37.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.901+0000 7f1810e58640 1 --2- 172.21.15.5:0/2284522130 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17f4005150 0x7f17f400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.901+0000 7f1810e58640 1 -- 172.21.15.5:0/2284522130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17f4004800 msgr2=0x7f17f4004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.901+0000 7f1810e58640 1 --2- 172.21.15.5:0/2284522130 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17f4004800 0x7f17f4004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.901+0000 7f1810e58640 1 -- 172.21.15.5:0/2284522130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f4003520 con 0x7f17f40039b0 2024-05-20T18:27:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.901+0000 7f1811659640 1 -- 172.21.15.5:0/2284522130 reap_dead start 2024-05-20T18:27:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.902+0000 7f1810e58640 1 --2- 172.21.15.5:0/2284522130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17f40039b0 0x7f17f4003de0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1808001860 tx=0x7f17f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e1aa260b4cebbf server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.902+0000 7f18037fe640 1 -- 172.21.15.5:0/2284522130 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808037070 con 0x7f17f40039b0 2024-05-20T18:27:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.902+0000 7f18037fe640 1 -- 172.21.15.5:0/2284522130 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f180802ac30 con 0x7f17f40039b0 2024-05-20T18:27:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.902+0000 7f18037fe640 1 -- 172.21.15.5:0/2284522130 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808001430 con 0x7f17f40039b0 2024-05-20T18:27:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.903+0000 7f18130bc640 1 -- 172.21.15.5:0/2284522130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17f40039b0 msgr2=0x7f17f4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.903+0000 7f18130bc640 1 --2- 172.21.15.5:0/2284522130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17f40039b0 0x7f17f4003de0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1808001860 tx=0x7f17f4005830 comp rx=0 tx=0).stop 2024-05-20T18:27:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.903+0000 7f18130bc640 1 -- 172.21.15.5:0/2284522130 shutdown_connections 2024-05-20T18:27:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.903+0000 7f18130bc640 1 --2- 172.21.15.5:0/2284522130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17f40039b0 0x7f17f4003de0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.903+0000 7f18130bc640 1 -- 172.21.15.5:0/2284522130 >> 172.21.15.5:0/2284522130 conn(0x7f180c0d4bd0 msgr2=0x7f180c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.903+0000 7f18130bc640 1 -- 172.21.15.5:0/2284522130 shutdown_connections 2024-05-20T18:27:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.903+0000 7f18130bc640 1 -- 172.21.15.5:0/2284522130 wait complete. 2024-05-20T18:27:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.905+0000 7f18130bc640 1 Processor -- start 2024-05-20T18:27:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.905+0000 7f18130bc640 1 -- start start 2024-05-20T18:27:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.906+0000 7f18130bc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17f40039b0 0x7f180c0ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.906+0000 7f18130bc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f180c0ca820 0x7f180c0cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.906+0000 7f18130bc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f180c1f52e0 0x7f180c1f56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.906+0000 7f18130bc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f180c0d0300 con 0x7f180c0ca820 2024-05-20T18:27:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.906+0000 7f18130bc640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f180c0d4260 con 0x7f17f40039b0 2024-05-20T18:27:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.906+0000 7f18130bc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f180c0d4780 con 0x7f180c1f52e0 2024-05-20T18:27:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.907+0000 7f1803fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f180c0ca820 0x7f180c0cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.907+0000 7f1803fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f180c0ca820 0x7f180c0cac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52992/0 (socket says 172.21.15.5:52992) 2024-05-20T18:27:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.907+0000 7f1803fff640 1 -- 172.21.15.5:0/1740451947 learned_addr learned my addr 172.21.15.5:0/1740451947 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.907+0000 7f1811659640 1 --2- 172.21.15.5:0/1740451947 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f180c1f52e0 0x7f180c1f56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.907+0000 7f1810e58640 1 --2- 172.21.15.5:0/1740451947 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17f40039b0 0x7f180c0ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.907+0000 7f1803fff640 1 -- 172.21.15.5:0/1740451947 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f180c1f52e0 msgr2=0x7f180c1f56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.907+0000 7f1803fff640 1 --2- 172.21.15.5:0/1740451947 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f180c1f52e0 0x7f180c1f56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.907+0000 7f1803fff640 1 -- 172.21.15.5:0/1740451947 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17f40039b0 msgr2=0x7f180c0ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.907+0000 7f1803fff640 1 --2- 172.21.15.5:0/1740451947 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17f40039b0 0x7f180c0ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.907+0000 7f1803fff640 1 -- 172.21.15.5:0/1740451947 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f180c0cc180 con 0x7f180c0ca820 2024-05-20T18:27:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.907+0000 7f1803fff640 1 --2- 172.21.15.5:0/1740451947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f180c0ca820 0x7f180c0cac30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f17fc011990 tx=0x7f17fc011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:37.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.908+0000 7f18017fa640 1 -- 172.21.15.5:0/1740451947 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc00fa30 con 0x7f180c0ca820 2024-05-20T18:27:37.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.908+0000 7f18017fa640 1 -- 172.21.15.5:0/1740451947 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17fc00fbd0 con 0x7f180c0ca820 2024-05-20T18:27:37.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.908+0000 7f18017fa640 1 -- 172.21.15.5:0/1740451947 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc014440 con 0x7f180c0ca820 2024-05-20T18:27:37.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.908+0000 7f18130bc640 1 -- 172.21.15.5:0/1740451947 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f180c0cc460 con 0x7f180c0ca820 2024-05-20T18:27:37.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.908+0000 7f18130bc640 1 -- 172.21.15.5:0/1740451947 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f180c1f6060 con 0x7f180c0ca820 2024-05-20T18:27:37.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.909+0000 7f18130bc640 1 -- 172.21.15.5:0/1740451947 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f180c075b00 con 0x7f180c0ca820 2024-05-20T18:27:37.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.909+0000 7f18017fa640 1 -- 172.21.15.5:0/1740451947 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f17fc012040 con 0x7f180c0ca820 2024-05-20T18:27:37.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.910+0000 7f18017fa640 1 --2- 172.21.15.5:0/1740451947 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f17f806be60 0x7f17f806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:37.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.910+0000 7f18017fa640 1 -- 172.21.15.5:0/1740451947 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f17fc08b640 con 0x7f180c0ca820 2024-05-20T18:27:37.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.910+0000 7f1810e58640 1 --2- 172.21.15.5:0/1740451947 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f17f806be60 0x7f17f806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:37.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.910+0000 7f1810e58640 1 --2- 172.21.15.5:0/1740451947 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f17f806be60 0x7f17f806e370 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1808034000 tx=0x7f1808009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:37.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:37.914+0000 7f18017fa640 1 -- 172.21.15.5:0/1740451947 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f17fc019030 con 0x7f180c0ca820 2024-05-20T18:27:38.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.098+0000 7f18130bc640 1 -- 172.21.15.5:0/1740451947 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f180c0ca0f0 con 0x7f180c0ca820 2024-05-20T18:27:38.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.099+0000 7f18017fa640 1 -- 172.21.15.5:0/1740451947 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f17fc059d40 con 0x7f180c0ca820 2024-05-20T18:27:38.098 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:27:38.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.101+0000 7f18130bc640 1 -- 172.21.15.5:0/1740451947 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f17f806be60 msgr2=0x7f17f806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:38.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.101+0000 7f18130bc640 1 --2- 172.21.15.5:0/1740451947 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f17f806be60 0x7f17f806e370 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1808034000 tx=0x7f1808009030 comp rx=0 tx=0).stop 2024-05-20T18:27:38.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.101+0000 7f18130bc640 1 -- 172.21.15.5:0/1740451947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f180c0ca820 msgr2=0x7f180c0cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:38.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.101+0000 7f18130bc640 1 --2- 172.21.15.5:0/1740451947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f180c0ca820 0x7f180c0cac30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f17fc011990 tx=0x7f17fc011d90 comp rx=0 tx=0).stop 2024-05-20T18:27:38.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.101+0000 7f18130bc640 1 -- 172.21.15.5:0/1740451947 shutdown_connections 2024-05-20T18:27:38.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.101+0000 7f18130bc640 1 --2- 172.21.15.5:0/1740451947 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f17f806be60 0x7f17f806e370 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:38.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.101+0000 7f18130bc640 1 --2- 172.21.15.5:0/1740451947 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f180c1f52e0 0x7f180c1f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:38.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.101+0000 7f18130bc640 1 --2- 172.21.15.5:0/1740451947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f180c0ca820 0x7f180c0cac30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:38.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.101+0000 7f18130bc640 1 --2- 172.21.15.5:0/1740451947 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17f40039b0 0x7f180c0ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:38.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.101+0000 7f18130bc640 1 -- 172.21.15.5:0/1740451947 >> 172.21.15.5:0/1740451947 conn(0x7f180c0d4bd0 msgr2=0x7f180c0b65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:38.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.101+0000 7f18130bc640 1 -- 172.21.15.5:0/1740451947 shutdown_connections 2024-05-20T18:27:38.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.101+0000 7f18130bc640 1 -- 172.21.15.5:0/1740451947 wait complete. 2024-05-20T18:27:38.112 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:27:38.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.227+0000 7ff2b8365640 1 Processor -- start 2024-05-20T18:27:38.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.227+0000 7ff2b8365640 1 -- start start 2024-05-20T18:27:38.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.229+0000 7ff2b8365640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff2b00d4400 0x7ff2b015f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:38.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.229+0000 7ff2b8365640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff2b00d2780 0x7ff2b00d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:38.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.229+0000 7ff2b8365640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2b008d2a0 con 0x7ff2b006cf00 2024-05-20T18:27:38.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.229+0000 7ff2b8365640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7ff2b00d3f70 con 0x7ff2b00d2780 2024-05-20T18:27:38.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.229+0000 7ff2b8365640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff2b00d30d0 con 0x7ff2b00d4400 2024-05-20T18:27:38.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.229+0000 7ff2b6902640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff2b00d2780 0x7ff2b00d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:38.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.229+0000 7ff2b6902640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff2b00d2780 0x7ff2b00d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53002/0 (socket says 172.21.15.5:53002) 2024-05-20T18:27:38.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.229+0000 7ff2b6902640 1 -- 172.21.15.5:0/2320516601 learned_addr learned my addr 172.21.15.5:0/2320516601 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:38.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.229+0000 7ff2b6101640 1 --2- 172.21.15.5:0/2320516601 >> v2:172.21.15.63:3300/0 conn(0x7ff2b00d4400 0x7ff2b015f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:38.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.229+0000 7ff2b6902640 1 -- 172.21.15.5:0/2320516601 >> v2:172.21.15.63:3300/0 conn(0x7ff2b00d4400 msgr2=0x7ff2b015f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:38.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.229+0000 7ff2b6902640 1 --2- 172.21.15.5:0/2320516601 >> v2:172.21.15.63:3300/0 conn(0x7ff2b00d4400 0x7ff2b015f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:38.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.229+0000 7ff2b6902640 1 -- 172.21.15.5:0/2320516601 >> v1:172.21.15.63:6789/0 conn(0x7ff2b006cf00 legacy=0x7ff2b006d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:38.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.229+0000 7ff2b6902640 1 -- 172.21.15.5:0/2320516601 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b00d4810 con 0x7ff2b00d2780 2024-05-20T18:27:38.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.230+0000 7ff2b6902640 1 --2- 172.21.15.5:0/2320516601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b00d2780 0x7ff2b00d2b90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00d5ab0 tx=0x7ff2a802a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf87b7a5106d516 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.230+0000 7ff2b50ff640 1 -- 172.21.15.5:0/2320516601 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8036070 con 0x7ff2b00d2780 2024-05-20T18:27:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.230+0000 7ff2b50ff640 1 -- 172.21.15.5:0/2320516601 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2a8031600 con 0x7ff2b00d2780 2024-05-20T18:27:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.230+0000 7ff2b50ff640 1 -- 172.21.15.5:0/2320516601 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a80319f0 con 0x7ff2b00d2780 2024-05-20T18:27:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.231+0000 7ff2b8365640 1 -- 172.21.15.5:0/2320516601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b00d2780 msgr2=0x7ff2b00d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.231+0000 7ff2b8365640 1 --2- 172.21.15.5:0/2320516601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b00d2780 0x7ff2b00d2b90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00d5ab0 tx=0x7ff2a802a7c0 comp rx=0 tx=0).stop 2024-05-20T18:27:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.231+0000 7ff2b8365640 1 -- 172.21.15.5:0/2320516601 shutdown_connections 2024-05-20T18:27:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.231+0000 7ff2b8365640 1 --2- 172.21.15.5:0/2320516601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b00d2780 0x7ff2b00d2b90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.231+0000 7ff2b8365640 1 --2- 172.21.15.5:0/2320516601 >> v2:172.21.15.63:3300/0 conn(0x7ff2b00d4400 0x7ff2b015f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.231+0000 7ff2b8365640 1 -- 172.21.15.5:0/2320516601 >> 172.21.15.5:0/2320516601 conn(0x7ff2b00d4bb0 msgr2=0x7ff2b00d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.231+0000 7ff2b8365640 1 -- 172.21.15.5:0/2320516601 shutdown_connections 2024-05-20T18:27:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.231+0000 7ff2b8365640 1 -- 172.21.15.5:0/2320516601 wait complete. 2024-05-20T18:27:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.233+0000 7ff2b8365640 1 Processor -- start 2024-05-20T18:27:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.233+0000 7ff2b8365640 1 -- start start 2024-05-20T18:27:38.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.234+0000 7ff2b8365640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b006cf00 0x7ff2b0149c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:38.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.234+0000 7ff2b8365640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2b00d2780 0x7ff2b00ab750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:38.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.234+0000 7ff2b8365640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2b00d4400 0x7ff2b00cd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:38.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.234+0000 7ff2b8365640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b00d3920 con 0x7ff2b006cf00 2024-05-20T18:27:38.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.234+0000 7ff2b8365640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b00d3640 con 0x7ff2b00d2780 2024-05-20T18:27:38.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.234+0000 7ff2b8365640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff2b00d3c00 con 0x7ff2b00d4400 2024-05-20T18:27:38.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.234+0000 7ff2b6101640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b006cf00 0x7ff2b0149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:38.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.234+0000 7ff2b6101640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b006cf00 0x7ff2b0149c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53018/0 (socket says 172.21.15.5:53018) 2024-05-20T18:27:38.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.234+0000 7ff2b6101640 1 -- 172.21.15.5:0/50266898 learned_addr learned my addr 172.21.15.5:0/50266898 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:38.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.234+0000 7ff2b6902640 1 --2- 172.21.15.5:0/50266898 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2b00d4400 0x7ff2b00cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:38.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.234+0000 7ff2b5900640 1 --2- 172.21.15.5:0/50266898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2b00d2780 0x7ff2b00ab750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:38.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.234+0000 7ff2b6101640 1 -- 172.21.15.5:0/50266898 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2b00d4400 msgr2=0x7ff2b00cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:38.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.234+0000 7ff2b6101640 1 --2- 172.21.15.5:0/50266898 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2b00d4400 0x7ff2b00cd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:38.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.234+0000 7ff2b6101640 1 -- 172.21.15.5:0/50266898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2b00d2780 msgr2=0x7ff2b00ab750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:38.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.234+0000 7ff2b6101640 1 --2- 172.21.15.5:0/50266898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2b00d2780 0x7ff2b00ab750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:38.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.234+0000 7ff2b6101640 1 -- 172.21.15.5:0/50266898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b014a570 con 0x7ff2b006cf00 2024-05-20T18:27:38.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.235+0000 7ff2b6101640 1 --2- 172.21.15.5:0/50266898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b006cf00 0x7ff2b0149c00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00e8a0 tx=0x7ff2ac00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:38.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.235+0000 7ff2a6ffd640 1 -- 172.21.15.5:0/50266898 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac017070 con 0x7ff2b006cf00 2024-05-20T18:27:38.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.235+0000 7ff2a6ffd640 1 -- 172.21.15.5:0/50266898 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2ac00c940 con 0x7ff2b006cf00 2024-05-20T18:27:38.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.235+0000 7ff2a6ffd640 1 -- 172.21.15.5:0/50266898 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac00cd30 con 0x7ff2b006cf00 2024-05-20T18:27:38.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.235+0000 7ff2b8365640 1 -- 172.21.15.5:0/50266898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b014a850 con 0x7ff2b006cf00 2024-05-20T18:27:38.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.235+0000 7ff2b8365640 1 -- 172.21.15.5:0/50266898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b01e7fd0 con 0x7ff2b006cf00 2024-05-20T18:27:38.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.236+0000 7ff2b8365640 1 -- 172.21.15.5:0/50266898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2b00735a0 con 0x7ff2b006cf00 2024-05-20T18:27:38.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.236+0000 7ff2a6ffd640 1 -- 172.21.15.5:0/50266898 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff2ac01a070 con 0x7ff2b006cf00 2024-05-20T18:27:38.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.237+0000 7ff2a6ffd640 1 --2- 172.21.15.5:0/50266898 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff29406be30 0x7ff29406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:38.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.237+0000 7ff2b5900640 1 --2- 172.21.15.5:0/50266898 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff29406be30 0x7ff29406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:38.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.237+0000 7ff2a6ffd640 1 -- 172.21.15.5:0/50266898 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff2ac08b3f0 con 0x7ff2b006cf00 2024-05-20T18:27:38.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.238+0000 7ff2b5900640 1 --2- 172.21.15.5:0/50266898 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff29406be30 0x7ff29406e340 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0007fd0 tx=0x7ff2a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:38.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.242+0000 7ff2a6ffd640 1 -- 172.21.15.5:0/50266898 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff2b00735a0 con 0x7ff2b006cf00 2024-05-20T18:27:38.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.462+0000 7ff2b8365640 1 -- 172.21.15.5:0/50266898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2b0075b00 con 0x7ff2b006cf00 2024-05-20T18:27:38.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.463+0000 7ff2a6ffd640 1 -- 172.21.15.5:0/50266898 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5168 (secure 0 0 0) 0x7ff2ac059a60 con 0x7ff2b006cf00 2024-05-20T18:27:38.463 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:38.463 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2982954341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2982954341},{"type":"v1","addr":"172.21.15.5:6836","nonce":2982954341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4032850001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":4032850001},{"type":"v1","addr":"172.21.15.5:6837","nonce":4032850001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6833/424580378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":424580378},{"type":"v1","addr":"172.21.15.63:6833","nonce":424580378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:27:07.164414+0000","modified":"2024-05-20T18:27:29.288681+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/3249423677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3249423677},{"type":"v1","addr":"172.21.15.63:6835","nonce":3249423677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-05-20T18:27:38.465 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 9 2024-05-20T18:27:38.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.465+0000 7ff2b8365640 1 -- 172.21.15.5:0/50266898 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff29406be30 msgr2=0x7ff29406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:38.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.465+0000 7ff2b8365640 1 --2- 172.21.15.5:0/50266898 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff29406be30 0x7ff29406e340 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0007fd0 tx=0x7ff2a0009040 comp rx=0 tx=0).stop 2024-05-20T18:27:38.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.465+0000 7ff2b8365640 1 -- 172.21.15.5:0/50266898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b006cf00 msgr2=0x7ff2b0149c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:38.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.465+0000 7ff2b8365640 1 --2- 172.21.15.5:0/50266898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b006cf00 0x7ff2b0149c00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00e8a0 tx=0x7ff2ac00ed60 comp rx=0 tx=0).stop 2024-05-20T18:27:38.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.465+0000 7ff2b8365640 1 -- 172.21.15.5:0/50266898 shutdown_connections 2024-05-20T18:27:38.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.465+0000 7ff2b8365640 1 --2- 172.21.15.5:0/50266898 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff29406be30 0x7ff29406e340 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:38.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.465+0000 7ff2b8365640 1 --2- 172.21.15.5:0/50266898 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2b00d4400 0x7ff2b00cd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:38.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.465+0000 7ff2b8365640 1 --2- 172.21.15.5:0/50266898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2b00d2780 0x7ff2b00ab750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:38.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.465+0000 7ff2b8365640 1 --2- 172.21.15.5:0/50266898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b006cf00 0x7ff2b0149c00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:38.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.465+0000 7ff2b8365640 1 -- 172.21.15.5:0/50266898 >> 172.21.15.5:0/50266898 conn(0x7ff2b00d4bb0 msgr2=0x7ff2b0160870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:38.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.465+0000 7ff2b8365640 1 -- 172.21.15.5:0/50266898 shutdown_connections 2024-05-20T18:27:38.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.465+0000 7ff2b8365640 1 -- 172.21.15.5:0/50266898 wait complete. 2024-05-20T18:27:38.476 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:27:38.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.593+0000 7f205547a640 1 Processor -- start 2024-05-20T18:27:38.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.593+0000 7f205547a640 1 -- start start 2024-05-20T18:27:38.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.594+0000 7f205547a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f205015d5f0 0x7f20501576d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:38.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.594+0000 7f205547a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f205006c9a0 0x7f20500ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:38.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.594+0000 7f205547a640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f205006b900 con 0x7f20500cec00 2024-05-20T18:27:38.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.594+0000 7f205547a640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f205006bbe0 con 0x7f205015d5f0 2024-05-20T18:27:38.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.594+0000 7f205547a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f205006bec0 con 0x7f205006c9a0 2024-05-20T18:27:38.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.594+0000 7f204effd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f205015d5f0 0x7f20501576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.594+0000 7f204f7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f20500cec00 0x7f205006e3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:41038/0 (socket says 172.21.15.5:41038) 2024-05-20T18:27:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.594+0000 7f204f7fe640 1 -- 172.21.15.5:0/2553342661 learned_addr learned my addr 172.21.15.5:0/2553342661 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.595+0000 7f2047fff640 1 --2- 172.21.15.5:0/2553342661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f205006c9a0 0x7f20500ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.595+0000 7f204effd640 1 -- 172.21.15.5:0/2553342661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f205006c9a0 msgr2=0x7f20500ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.595+0000 7f204effd640 1 --2- 172.21.15.5:0/2553342661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f205006c9a0 0x7f20500ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.595+0000 7f204effd640 1 -- 172.21.15.5:0/2553342661 >> v1:172.21.15.5:6789/0 conn(0x7f20500cec00 legacy=0x7f205006e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.595+0000 7f204e7fc640 1 -- 172.21.15.5:0/2553342661 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3701961450 0 0) 0x7f205006b900 con 0x7f20500cec00 2024-05-20T18:27:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.595+0000 7f204effd640 1 -- 172.21.15.5:0/2553342661 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f205006c1a0 con 0x7f205015d5f0 2024-05-20T18:27:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.596+0000 7f204effd640 1 --2- 172.21.15.5:0/2553342661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f205015d5f0 0x7f20501576d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f20400097a0 tx=0x7f204002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=384f3cd7d30adb7f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.596+0000 7f204e7fc640 1 -- 172.21.15.5:0/2553342661 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040037070 con 0x7f205015d5f0 2024-05-20T18:27:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.596+0000 7f204e7fc640 1 -- 172.21.15.5:0/2553342661 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f204002ae50 con 0x7f205015d5f0 2024-05-20T18:27:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.596+0000 7f204e7fc640 1 -- 172.21.15.5:0/2553342661 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040032660 con 0x7f205015d5f0 2024-05-20T18:27:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.596+0000 7f205547a640 1 -- 172.21.15.5:0/2553342661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f205015d5f0 msgr2=0x7f20501576d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.596+0000 7f205547a640 1 --2- 172.21.15.5:0/2553342661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f205015d5f0 0x7f20501576d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f20400097a0 tx=0x7f204002a100 comp rx=0 tx=0).stop 2024-05-20T18:27:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.597+0000 7f205547a640 1 -- 172.21.15.5:0/2553342661 shutdown_connections 2024-05-20T18:27:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.597+0000 7f205547a640 1 --2- 172.21.15.5:0/2553342661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f205006c9a0 0x7f20500ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.597+0000 7f205547a640 1 --2- 172.21.15.5:0/2553342661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f205015d5f0 0x7f20501576d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.597+0000 7f205547a640 1 -- 172.21.15.5:0/2553342661 >> 172.21.15.5:0/2553342661 conn(0x7f20500cd2a0 msgr2=0x7f20500cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.597+0000 7f205547a640 1 -- 172.21.15.5:0/2553342661 shutdown_connections 2024-05-20T18:27:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.597+0000 7f205547a640 1 -- 172.21.15.5:0/2553342661 wait complete. 2024-05-20T18:27:38.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.598+0000 7f205547a640 1 Processor -- start 2024-05-20T18:27:38.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.598+0000 7f205547a640 1 -- start start 2024-05-20T18:27:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.600+0000 7f205547a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f205006c9a0 0x7f2050154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.600+0000 7f205547a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20500cec00 0x7f2050155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.600+0000 7f205547a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f205015d5f0 0x7f205009c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.600+0000 7f205547a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f205006b520 con 0x7f205015d5f0 2024-05-20T18:27:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.600+0000 7f205547a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f205006cdb0 con 0x7f20500cec00 2024-05-20T18:27:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.600+0000 7f205547a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f205006b6a0 con 0x7f205006c9a0 2024-05-20T18:27:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.600+0000 7f204f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f205015d5f0 0x7f205009c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.600+0000 7f204f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f205015d5f0 0x7f205009c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53026/0 (socket says 172.21.15.5:53026) 2024-05-20T18:27:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.600+0000 7f204f7fe640 1 -- 172.21.15.5:0/2014847010 learned_addr learned my addr 172.21.15.5:0/2014847010 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.600+0000 7f204effd640 1 --2- 172.21.15.5:0/2014847010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f205006c9a0 0x7f2050154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.600+0000 7f2047fff640 1 --2- 172.21.15.5:0/2014847010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20500cec00 0x7f2050155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.600+0000 7f204f7fe640 1 -- 172.21.15.5:0/2014847010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f205006c9a0 msgr2=0x7f2050154de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.600+0000 7f204f7fe640 1 --2- 172.21.15.5:0/2014847010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f205006c9a0 0x7f2050154de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.600+0000 7f204f7fe640 1 -- 172.21.15.5:0/2014847010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20500cec00 msgr2=0x7f2050155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.600+0000 7f204f7fe640 1 --2- 172.21.15.5:0/2014847010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20500cec00 0x7f2050155320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.600+0000 7f204f7fe640 1 -- 172.21.15.5:0/2014847010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20501567b0 con 0x7f205015d5f0 2024-05-20T18:27:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.600+0000 7f204f7fe640 1 --2- 172.21.15.5:0/2014847010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f205015d5f0 0x7f205009c9b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f204800c730 tx=0x7f204800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.600+0000 7f20477fe640 1 -- 172.21.15.5:0/2014847010 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2048018070 con 0x7f205015d5f0 2024-05-20T18:27:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.600+0000 7f20477fe640 1 -- 172.21.15.5:0/2014847010 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f204800ce40 con 0x7f205015d5f0 2024-05-20T18:27:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.601+0000 7f205547a640 1 -- 172.21.15.5:0/2014847010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2050151780 con 0x7f205015d5f0 2024-05-20T18:27:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.601+0000 7f20477fe640 1 -- 172.21.15.5:0/2014847010 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204800a880 con 0x7f205015d5f0 2024-05-20T18:27:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.601+0000 7f205547a640 1 -- 172.21.15.5:0/2014847010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2050155e80 con 0x7f205015d5f0 2024-05-20T18:27:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.602+0000 7f205547a640 1 -- 172.21.15.5:0/2014847010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2050075b00 con 0x7f205015d5f0 2024-05-20T18:27:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.602+0000 7f20477fe640 1 -- 172.21.15.5:0/2014847010 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f204801b070 con 0x7f205015d5f0 2024-05-20T18:27:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.603+0000 7f20477fe640 1 --2- 172.21.15.5:0/2014847010 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f203006be30 0x7f203006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.603+0000 7f204effd640 1 --2- 172.21.15.5:0/2014847010 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f203006be30 0x7f203006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.603+0000 7f20477fe640 1 -- 172.21.15.5:0/2014847010 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f204808a740 con 0x7f205015d5f0 2024-05-20T18:27:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.603+0000 7f204effd640 1 --2- 172.21.15.5:0/2014847010 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f203006be30 0x7f203006e340 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2040034000 tx=0x7f204002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:38.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.607+0000 7f20477fe640 1 -- 172.21.15.5:0/2014847010 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2048058d20 con 0x7f205015d5f0 2024-05-20T18:27:38.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.793+0000 7f205547a640 1 -- 172.21.15.5:0/2014847010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f20500cff40 con 0x7f205015d5f0 2024-05-20T18:27:38.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.795+0000 7f20477fe640 1 -- 172.21.15.5:0/2014847010 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18394 (secure 0 0 0) 0x7f2048058620 con 0x7f205015d5f0 2024-05-20T18:27:38.794 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:38.794 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:27:13.817633+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:27:05.476596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T18:27:06.137762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T18:27:09.829920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:27:38.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.797+0000 7f205547a640 1 -- 172.21.15.5:0/2014847010 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f203006be30 msgr2=0x7f203006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:38.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.797+0000 7f205547a640 1 --2- 172.21.15.5:0/2014847010 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f203006be30 0x7f203006e340 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2040034000 tx=0x7f204002e000 comp rx=0 tx=0).stop 2024-05-20T18:27:38.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.797+0000 7f205547a640 1 -- 172.21.15.5:0/2014847010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f205015d5f0 msgr2=0x7f205009c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:38.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.797+0000 7f205547a640 1 --2- 172.21.15.5:0/2014847010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f205015d5f0 0x7f205009c9b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f204800c730 tx=0x7f204800cbf0 comp rx=0 tx=0).stop 2024-05-20T18:27:38.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.797+0000 7f205547a640 1 -- 172.21.15.5:0/2014847010 shutdown_connections 2024-05-20T18:27:38.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.797+0000 7f205547a640 1 --2- 172.21.15.5:0/2014847010 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f203006be30 0x7f203006e340 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:38.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.797+0000 7f205547a640 1 --2- 172.21.15.5:0/2014847010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f205015d5f0 0x7f205009c9b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:38.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.797+0000 7f205547a640 1 --2- 172.21.15.5:0/2014847010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20500cec00 0x7f2050155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:38.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.797+0000 7f205547a640 1 --2- 172.21.15.5:0/2014847010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f205006c9a0 0x7f2050154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:38.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.797+0000 7f205547a640 1 -- 172.21.15.5:0/2014847010 >> 172.21.15.5:0/2014847010 conn(0x7f20500cd2a0 msgr2=0x7f205015ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:38.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.797+0000 7f205547a640 1 -- 172.21.15.5:0/2014847010 shutdown_connections 2024-05-20T18:27:38.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.797+0000 7f205547a640 1 -- 172.21.15.5:0/2014847010 wait complete. 2024-05-20T18:27:38.808 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:27:05.476596+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:27:06.137762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T18:27:09.829920+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T18:27:38.809 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:27:38.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.927+0000 7fe787916640 1 Processor -- start 2024-05-20T18:27:38.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.927+0000 7fe787916640 1 -- start start 2024-05-20T18:27:38.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.929+0000 7fe787916640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe780162820 0x7fe7800b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:38.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.929+0000 7fe787916640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe78006dae0 0x7fe7800b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:38.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.929+0000 7fe787916640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7800d0630 con 0x7fe7800b3e80 2024-05-20T18:27:38.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.929+0000 7fe787916640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe7800d39c0 con 0x7fe78006dae0 2024-05-20T18:27:38.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.929+0000 7fe787916640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe7800d3ca0 con 0x7fe780162820 2024-05-20T18:27:38.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.929+0000 7fe785eb3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe78006dae0 0x7fe7800b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:38.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.929+0000 7fe784eb1640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe7800b3e80 0x7fe78006d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:41048/0 (socket says 172.21.15.5:41048) 2024-05-20T18:27:38.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.929+0000 7fe784eb1640 1 -- 172.21.15.5:0/1280767664 learned_addr learned my addr 172.21.15.5:0/1280767664 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:38.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.929+0000 7fe7856b2640 1 --2- 172.21.15.5:0/1280767664 >> v2:172.21.15.63:3300/0 conn(0x7fe780162820 0x7fe7800b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:38.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.929+0000 7fe785eb3640 1 -- 172.21.15.5:0/1280767664 >> v2:172.21.15.63:3300/0 conn(0x7fe780162820 msgr2=0x7fe7800b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:38.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.929+0000 7fe785eb3640 1 --2- 172.21.15.5:0/1280767664 >> v2:172.21.15.63:3300/0 conn(0x7fe780162820 0x7fe7800b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:38.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.929+0000 7fe785eb3640 1 -- 172.21.15.5:0/1280767664 >> v1:172.21.15.5:6789/0 conn(0x7fe7800b3e80 legacy=0x7fe78006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:38.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.929+0000 7fe785eb3640 1 -- 172.21.15.5:0/1280767664 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7800d3f80 con 0x7fe78006dae0 2024-05-20T18:27:38.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.929+0000 7fe777fff640 1 -- 172.21.15.5:0/1280767664 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1222047373 0 0) 0x7fe7800d0630 con 0x7fe7800b3e80 2024-05-20T18:27:38.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.930+0000 7fe785eb3640 1 --2- 172.21.15.5:0/1280767664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe78006dae0 0x7fe7800b8190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe7800b7740 tx=0x7fe77802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed580e1c2ab02daf server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:38.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.930+0000 7fe777fff640 1 -- 172.21.15.5:0/1280767664 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe778037070 con 0x7fe78006dae0 2024-05-20T18:27:38.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.930+0000 7fe777fff640 1 -- 172.21.15.5:0/1280767664 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe778032440 con 0x7fe78006dae0 2024-05-20T18:27:38.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.930+0000 7fe777fff640 1 -- 172.21.15.5:0/1280767664 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe778032810 con 0x7fe78006dae0 2024-05-20T18:27:38.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.931+0000 7fe787916640 1 -- 172.21.15.5:0/1280767664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe78006dae0 msgr2=0x7fe7800b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:38.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.931+0000 7fe787916640 1 --2- 172.21.15.5:0/1280767664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe78006dae0 0x7fe7800b8190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe7800b7740 tx=0x7fe77802a1a0 comp rx=0 tx=0).stop 2024-05-20T18:27:38.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.931+0000 7fe787916640 1 -- 172.21.15.5:0/1280767664 shutdown_connections 2024-05-20T18:27:38.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.931+0000 7fe787916640 1 --2- 172.21.15.5:0/1280767664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe78006dae0 0x7fe7800b8190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:38.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.931+0000 7fe787916640 1 --2- 172.21.15.5:0/1280767664 >> v2:172.21.15.63:3300/0 conn(0x7fe780162820 0x7fe7800b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:38.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.931+0000 7fe787916640 1 -- 172.21.15.5:0/1280767664 >> 172.21.15.5:0/1280767664 conn(0x7fe7800d4bd0 msgr2=0x7fe7800d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:38.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.931+0000 7fe787916640 1 -- 172.21.15.5:0/1280767664 shutdown_connections 2024-05-20T18:27:38.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.931+0000 7fe787916640 1 -- 172.21.15.5:0/1280767664 wait complete. 2024-05-20T18:27:38.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.933+0000 7fe787916640 1 Processor -- start 2024-05-20T18:27:38.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.933+0000 7fe787916640 1 -- start start 2024-05-20T18:27:38.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.934+0000 7fe787916640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7800b3e80 0x7fe780147270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:38.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.934+0000 7fe787916640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe780162820 0x7fe780155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:38.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.934+0000 7fe787916640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7801477b0 0x7fe780155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:38.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.934+0000 7fe787916640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe7800d4780 con 0x7fe7801477b0 2024-05-20T18:27:38.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.934+0000 7fe787916640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe7800d0300 con 0x7fe7800b3e80 2024-05-20T18:27:38.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.934+0000 7fe787916640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe7800d4900 con 0x7fe780162820 2024-05-20T18:27:38.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.934+0000 7fe785eb3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7801477b0 0x7fe780155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:38.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.934+0000 7fe785eb3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7801477b0 0x7fe780155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53042/0 (socket says 172.21.15.5:53042) 2024-05-20T18:27:38.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.934+0000 7fe785eb3640 1 -- 172.21.15.5:0/3416283282 learned_addr learned my addr 172.21.15.5:0/3416283282 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:38.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.935+0000 7fe7856b2640 1 --2- 172.21.15.5:0/3416283282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7800b3e80 0x7fe780147270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:38.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.935+0000 7fe784eb1640 1 --2- 172.21.15.5:0/3416283282 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe780162820 0x7fe780155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:38.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.935+0000 7fe785eb3640 1 -- 172.21.15.5:0/3416283282 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe780162820 msgr2=0x7fe780155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:38.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.935+0000 7fe785eb3640 1 --2- 172.21.15.5:0/3416283282 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe780162820 0x7fe780155aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:38.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.935+0000 7fe785eb3640 1 -- 172.21.15.5:0/3416283282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7800b3e80 msgr2=0x7fe780147270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:38.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.935+0000 7fe785eb3640 1 --2- 172.21.15.5:0/3416283282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7800b3e80 0x7fe780147270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:38.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.935+0000 7fe785eb3640 1 -- 172.21.15.5:0/3416283282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe78015f4c0 con 0x7fe7801477b0 2024-05-20T18:27:38.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.935+0000 7fe785eb3640 1 --2- 172.21.15.5:0/3416283282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7801477b0 0x7fe780155fe0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe778035040 tx=0x7fe7780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:38.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.935+0000 7fe775ffb640 1 -- 172.21.15.5:0/3416283282 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe778037040 con 0x7fe7801477b0 2024-05-20T18:27:38.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.935+0000 7fe775ffb640 1 -- 172.21.15.5:0/3416283282 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe778042070 con 0x7fe7801477b0 2024-05-20T18:27:38.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.935+0000 7fe787916640 1 -- 172.21.15.5:0/3416283282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe78015f7a0 con 0x7fe7801477b0 2024-05-20T18:27:38.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.935+0000 7fe775ffb640 1 -- 172.21.15.5:0/3416283282 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7780308a0 con 0x7fe7801477b0 2024-05-20T18:27:38.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.936+0000 7fe787916640 1 -- 172.21.15.5:0/3416283282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe78015fd60 con 0x7fe7801477b0 2024-05-20T18:27:38.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.936+0000 7fe787916640 1 -- 172.21.15.5:0/3416283282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7801e9300 con 0x7fe7801477b0 2024-05-20T18:27:38.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.939+0000 7fe775ffb640 1 -- 172.21.15.5:0/3416283282 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe778030a40 con 0x7fe7801477b0 2024-05-20T18:27:38.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.939+0000 7fe775ffb640 1 --2- 172.21.15.5:0/3416283282 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe76406be30 0x7fe76406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:38.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.939+0000 7fe775ffb640 1 -- 172.21.15.5:0/3416283282 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe7780ac680 con 0x7fe7801477b0 2024-05-20T18:27:38.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.940+0000 7fe7856b2640 1 --2- 172.21.15.5:0/3416283282 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe76406be30 0x7fe76406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:38.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.940+0000 7fe7856b2640 1 --2- 172.21.15.5:0/3416283282 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe76406be30 0x7fe76406e340 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe77c0073f0 tx=0x7fe77c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:38.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:38.943+0000 7fe775ffb640 1 -- 172.21.15.5:0/3416283282 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe7801e9300 con 0x7fe7801477b0 2024-05-20T18:27:39.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.128+0000 7fe787916640 1 -- 172.21.15.5:0/3416283282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe780075b00 con 0x7fe7801477b0 2024-05-20T18:27:39.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.128+0000 7fe775ffb640 1 -- 172.21.15.5:0/3416283282 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fe77804f380 con 0x7fe7801477b0 2024-05-20T18:27:39.128 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:27:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.130+0000 7fe787916640 1 -- 172.21.15.5:0/3416283282 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe76406be30 msgr2=0x7fe76406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.130+0000 7fe787916640 1 --2- 172.21.15.5:0/3416283282 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe76406be30 0x7fe76406e340 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe77c0073f0 tx=0x7fe77c00b040 comp rx=0 tx=0).stop 2024-05-20T18:27:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.130+0000 7fe787916640 1 -- 172.21.15.5:0/3416283282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7801477b0 msgr2=0x7fe780155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.130+0000 7fe787916640 1 --2- 172.21.15.5:0/3416283282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7801477b0 0x7fe780155fe0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe778035040 tx=0x7fe7780046c0 comp rx=0 tx=0).stop 2024-05-20T18:27:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.130+0000 7fe787916640 1 -- 172.21.15.5:0/3416283282 shutdown_connections 2024-05-20T18:27:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.130+0000 7fe787916640 1 --2- 172.21.15.5:0/3416283282 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe76406be30 0x7fe76406e340 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.130+0000 7fe787916640 1 --2- 172.21.15.5:0/3416283282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7801477b0 0x7fe780155fe0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.130+0000 7fe787916640 1 --2- 172.21.15.5:0/3416283282 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe780162820 0x7fe780155aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.130+0000 7fe787916640 1 --2- 172.21.15.5:0/3416283282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7800b3e80 0x7fe780147270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.130+0000 7fe787916640 1 -- 172.21.15.5:0/3416283282 >> 172.21.15.5:0/3416283282 conn(0x7fe7800d4bd0 msgr2=0x7fe78015c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.130+0000 7fe787916640 1 -- 172.21.15.5:0/3416283282 shutdown_connections 2024-05-20T18:27:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.130+0000 7fe787916640 1 -- 172.21.15.5:0/3416283282 wait complete. 2024-05-20T18:27:39.141 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:27:39.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.259+0000 7ff2ec1c3640 1 Processor -- start 2024-05-20T18:27:39.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.259+0000 7ff2ec1c3640 1 -- start start 2024-05-20T18:27:39.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.261+0000 7ff2ec1c3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2e4162820 0x7ff2e40b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:39.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.261+0000 7ff2ec1c3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff2e406dae0 0x7ff2e40b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:39.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.261+0000 7ff2ec1c3640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2e40d0630 con 0x7ff2e40b3e80 2024-05-20T18:27:39.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.261+0000 7ff2ec1c3640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7ff2e40d39c0 con 0x7ff2e406dae0 2024-05-20T18:27:39.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.261+0000 7ff2ec1c3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff2e40d3ca0 con 0x7ff2e4162820 2024-05-20T18:27:39.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.261+0000 7ff2ea760640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff2e406dae0 0x7ff2e40b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:39.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.261+0000 7ff2e975e640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff2e40b3e80 0x7ff2e406d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:41050/0 (socket says 172.21.15.5:41050) 2024-05-20T18:27:39.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.261+0000 7ff2ea760640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff2e406dae0 0x7ff2e40b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53046/0 (socket says 172.21.15.5:53046) 2024-05-20T18:27:39.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.261+0000 7ff2e975e640 1 -- 172.21.15.5:0/3830195910 learned_addr learned my addr 172.21.15.5:0/3830195910 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:39.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.261+0000 7ff2e9f5f640 1 --2- 172.21.15.5:0/3830195910 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2e4162820 0x7ff2e40b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:39.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.261+0000 7ff2ea760640 1 -- 172.21.15.5:0/3830195910 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2e4162820 msgr2=0x7ff2e40b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:39.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.261+0000 7ff2ea760640 1 --2- 172.21.15.5:0/3830195910 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2e4162820 0x7ff2e40b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:39.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.261+0000 7ff2ea760640 1 -- 172.21.15.5:0/3830195910 >> v1:172.21.15.5:6789/0 conn(0x7ff2e40b3e80 legacy=0x7ff2e406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:39.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.261+0000 7ff2e8f5d640 1 -- 172.21.15.5:0/3830195910 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 23226919 0 0) 0x7ff2e40d0630 con 0x7ff2e40b3e80 2024-05-20T18:27:39.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.261+0000 7ff2ea760640 1 -- 172.21.15.5:0/3830195910 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e40d3f80 con 0x7ff2e406dae0 2024-05-20T18:27:39.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.262+0000 7ff2ea760640 1 --2- 172.21.15.5:0/3830195910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2e406dae0 0x7ff2e40b8190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009d90 tx=0x7ff2d402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e742d8c50e43374c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:39.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.262+0000 7ff2e8f5d640 1 -- 172.21.15.5:0/3830195910 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4007740 con 0x7ff2e406dae0 2024-05-20T18:27:39.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.262+0000 7ff2e8f5d640 1 -- 172.21.15.5:0/3830195910 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2d40078e0 con 0x7ff2e406dae0 2024-05-20T18:27:39.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.262+0000 7ff2e8f5d640 1 -- 172.21.15.5:0/3830195910 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4007cb0 con 0x7ff2e406dae0 2024-05-20T18:27:39.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.263+0000 7ff2ec1c3640 1 -- 172.21.15.5:0/3830195910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2e406dae0 msgr2=0x7ff2e40b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:39.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.263+0000 7ff2ec1c3640 1 --2- 172.21.15.5:0/3830195910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2e406dae0 0x7ff2e40b8190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009d90 tx=0x7ff2d402a660 comp rx=0 tx=0).stop 2024-05-20T18:27:39.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.263+0000 7ff2ec1c3640 1 -- 172.21.15.5:0/3830195910 shutdown_connections 2024-05-20T18:27:39.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.263+0000 7ff2ec1c3640 1 --2- 172.21.15.5:0/3830195910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2e406dae0 0x7ff2e40b8190 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:39.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.263+0000 7ff2ec1c3640 1 --2- 172.21.15.5:0/3830195910 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2e4162820 0x7ff2e40b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:39.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.263+0000 7ff2ec1c3640 1 -- 172.21.15.5:0/3830195910 >> 172.21.15.5:0/3830195910 conn(0x7ff2e40d4bd0 msgr2=0x7ff2e40d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:39.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.263+0000 7ff2ec1c3640 1 -- 172.21.15.5:0/3830195910 shutdown_connections 2024-05-20T18:27:39.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.263+0000 7ff2ec1c3640 1 -- 172.21.15.5:0/3830195910 wait complete. 2024-05-20T18:27:39.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.265+0000 7ff2ec1c3640 1 Processor -- start 2024-05-20T18:27:39.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.265+0000 7ff2ec1c3640 1 -- start start 2024-05-20T18:27:39.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.266+0000 7ff2ec1c3640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2e40b3e80 0x7ff2e40cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:39.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.266+0000 7ff2ec1c3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2e4162820 0x7ff2e40cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:39.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.266+0000 7ff2ec1c3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2e40ce4f0 0x7ff2e40c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:39.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.266+0000 7ff2ec1c3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e40d4780 con 0x7ff2e4162820 2024-05-20T18:27:39.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.266+0000 7ff2ec1c3640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e40d0300 con 0x7ff2e40b3e80 2024-05-20T18:27:39.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.266+0000 7ff2ec1c3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff2e40d4900 con 0x7ff2e40ce4f0 2024-05-20T18:27:39.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.266+0000 7ff2e975e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2e4162820 0x7ff2e40cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:39.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.266+0000 7ff2e975e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2e4162820 0x7ff2e40cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53052/0 (socket says 172.21.15.5:53052) 2024-05-20T18:27:39.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.266+0000 7ff2e975e640 1 -- 172.21.15.5:0/669088595 learned_addr learned my addr 172.21.15.5:0/669088595 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:39.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.267+0000 7ff2ea760640 1 --2- 172.21.15.5:0/669088595 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2e40ce4f0 0x7ff2e40c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:39.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.267+0000 7ff2e9f5f640 1 --2- 172.21.15.5:0/669088595 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2e40b3e80 0x7ff2e40cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:39.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.267+0000 7ff2e975e640 1 -- 172.21.15.5:0/669088595 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2e40ce4f0 msgr2=0x7ff2e40c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:39.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.267+0000 7ff2e975e640 1 --2- 172.21.15.5:0/669088595 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2e40ce4f0 0x7ff2e40c9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:39.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.267+0000 7ff2e975e640 1 -- 172.21.15.5:0/669088595 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2e40b3e80 msgr2=0x7ff2e40cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:39.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.267+0000 7ff2e975e640 1 --2- 172.21.15.5:0/669088595 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2e40b3e80 0x7ff2e40cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:39.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.267+0000 7ff2e975e640 1 -- 172.21.15.5:0/669088595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e40cc190 con 0x7ff2e4162820 2024-05-20T18:27:39.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.267+0000 7ff2e9f5f640 1 --2- 172.21.15.5:0/669088595 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2e40b3e80 0x7ff2e40cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:39.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.267+0000 7ff2e975e640 1 --2- 172.21.15.5:0/669088595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2e4162820 0x7ff2e40cdfb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff2d800c440 tx=0x7ff2d800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:39.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.267+0000 7ff2e27fc640 1 -- 172.21.15.5:0/669088595 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8018070 con 0x7ff2e4162820 2024-05-20T18:27:39.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.267+0000 7ff2e27fc640 1 -- 172.21.15.5:0/669088595 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2d80074e0 con 0x7ff2e4162820 2024-05-20T18:27:39.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.267+0000 7ff2ec1c3640 1 -- 172.21.15.5:0/669088595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2e40cc6a0 con 0x7ff2e4162820 2024-05-20T18:27:39.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.267+0000 7ff2e27fc640 1 -- 172.21.15.5:0/669088595 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8004a20 con 0x7ff2e4162820 2024-05-20T18:27:39.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.267+0000 7ff2ec1c3640 1 -- 172.21.15.5:0/669088595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2e40ccc60 con 0x7ff2e4162820 2024-05-20T18:27:39.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.268+0000 7ff2ec1c3640 1 -- 172.21.15.5:0/669088595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2e4075b00 con 0x7ff2e4162820 2024-05-20T18:27:39.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.269+0000 7ff2e27fc640 1 -- 172.21.15.5:0/669088595 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff2d801b070 con 0x7ff2e4162820 2024-05-20T18:27:39.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.269+0000 7ff2e27fc640 1 --2- 172.21.15.5:0/669088595 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff2d006be60 0x7ff2d006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:39.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.269+0000 7ff2e27fc640 1 -- 172.21.15.5:0/669088595 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff2d808a460 con 0x7ff2e4162820 2024-05-20T18:27:39.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.269+0000 7ff2e9f5f640 1 --2- 172.21.15.5:0/669088595 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff2d006be60 0x7ff2d006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:39.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.270+0000 7ff2e9f5f640 1 --2- 172.21.15.5:0/669088595 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff2d006be60 0x7ff2d006e370 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc009a20 tx=0x7ff2dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:39.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.274+0000 7ff2e27fc640 1 -- 172.21.15.5:0/669088595 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff2d8058a40 con 0x7ff2e4162820 2024-05-20T18:27:39.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.456+0000 7ff2ec1c3640 1 -- 172.21.15.5:0/669088595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff2e40cc430 con 0x7ff2e4162820 2024-05-20T18:27:39.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.456+0000 7ff2e27fc640 1 -- 172.21.15.5:0/669088595 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7ff2d8058340 con 0x7ff2e4162820 2024-05-20T18:27:39.455 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:27:39.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.458+0000 7ff2ec1c3640 1 -- 172.21.15.5:0/669088595 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff2d006be60 msgr2=0x7ff2d006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:39.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.458+0000 7ff2ec1c3640 1 --2- 172.21.15.5:0/669088595 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff2d006be60 0x7ff2d006e370 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc009a20 tx=0x7ff2dc008040 comp rx=0 tx=0).stop 2024-05-20T18:27:39.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.458+0000 7ff2ec1c3640 1 -- 172.21.15.5:0/669088595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2e4162820 msgr2=0x7ff2e40cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:39.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.458+0000 7ff2ec1c3640 1 --2- 172.21.15.5:0/669088595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2e4162820 0x7ff2e40cdfb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff2d800c440 tx=0x7ff2d800c900 comp rx=0 tx=0).stop 2024-05-20T18:27:39.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.458+0000 7ff2ec1c3640 1 -- 172.21.15.5:0/669088595 shutdown_connections 2024-05-20T18:27:39.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.458+0000 7ff2ec1c3640 1 --2- 172.21.15.5:0/669088595 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff2d006be60 0x7ff2d006e370 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:39.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.458+0000 7ff2ec1c3640 1 --2- 172.21.15.5:0/669088595 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff2e40ce4f0 0x7ff2e40c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:39.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.458+0000 7ff2ec1c3640 1 --2- 172.21.15.5:0/669088595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2e4162820 0x7ff2e40cdfb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:39.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.458+0000 7ff2ec1c3640 1 --2- 172.21.15.5:0/669088595 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2e40b3e80 0x7ff2e40cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:39.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.458+0000 7ff2ec1c3640 1 -- 172.21.15.5:0/669088595 >> 172.21.15.5:0/669088595 conn(0x7ff2e40d4bd0 msgr2=0x7ff2e40ba1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:39.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.458+0000 7ff2ec1c3640 1 -- 172.21.15.5:0/669088595 shutdown_connections 2024-05-20T18:27:39.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.458+0000 7ff2ec1c3640 1 -- 172.21.15.5:0/669088595 wait complete. 2024-05-20T18:27:39.469 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:27:39.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.587+0000 7ff186110640 1 Processor -- start 2024-05-20T18:27:39.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.587+0000 7ff186110640 1 -- start start 2024-05-20T18:27:39.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.589+0000 7ff186110640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff180162820 0x7ff1800b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:39.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.589+0000 7ff186110640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1800b3e80 0x7ff18006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:39.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.589+0000 7ff186110640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff18006dd20 con 0x7ff18006d910 2024-05-20T18:27:39.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.589+0000 7ff186110640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff1800d0630 con 0x7ff180162820 2024-05-20T18:27:39.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.589+0000 7ff186110640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff1800d39c0 con 0x7ff1800b3e80 2024-05-20T18:27:39.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.589+0000 7ff17f7fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff180162820 0x7ff1800b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:39.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.589+0000 7ff17effd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1800b3e80 0x7ff18006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:39.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.589+0000 7ff17effd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1800b3e80 0x7ff18006d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:37348/0 (socket says 172.21.15.5:37348) 2024-05-20T18:27:39.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.589+0000 7ff17f7fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff180162820 0x7ff1800b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:41696/0 (socket says 172.21.15.5:41696) 2024-05-20T18:27:39.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.589+0000 7ff17effd640 1 -- 172.21.15.5:0/23752155 learned_addr learned my addr 172.21.15.5:0/23752155 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:39.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.590+0000 7ff17e7fc640 1 -- 172.21.15.5:0/23752155 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 359615066 0 0) 0x7ff178001490 con 0x7ff18006d910 2024-05-20T18:27:39.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.590+0000 7ff17effd640 1 -- 172.21.15.5:0/23752155 >> v2:172.21.15.63:3300/0 conn(0x7ff180162820 msgr2=0x7ff1800b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:39.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.590+0000 7ff17effd640 1 --2- 172.21.15.5:0/23752155 >> v2:172.21.15.63:3300/0 conn(0x7ff180162820 0x7ff1800b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:39.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.590+0000 7ff17effd640 1 -- 172.21.15.5:0/23752155 >> v1:172.21.15.63:6789/0 conn(0x7ff18006d910 legacy=0x7ff1800d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:39.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.590+0000 7ff17effd640 1 -- 172.21.15.5:0/23752155 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1800d3ca0 con 0x7ff1800b3e80 2024-05-20T18:27:39.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.590+0000 7ff17effd640 1 --2- 172.21.15.5:0/23752155 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1800b3e80 0x7ff18006d3d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff174009870 tx=0x7ff17402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=749573482d32eaa4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:39.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.591+0000 7ff17e7fc640 1 -- 172.21.15.5:0/23752155 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff174037070 con 0x7ff1800b3e80 2024-05-20T18:27:39.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.591+0000 7ff17e7fc640 1 -- 172.21.15.5:0/23752155 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff174032440 con 0x7ff1800b3e80 2024-05-20T18:27:39.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.591+0000 7ff17e7fc640 1 -- 172.21.15.5:0/23752155 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff174032810 con 0x7ff1800b3e80 2024-05-20T18:27:39.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.591+0000 7ff186110640 1 -- 172.21.15.5:0/23752155 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1800b3e80 msgr2=0x7ff18006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:39.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.591+0000 7ff186110640 1 --2- 172.21.15.5:0/23752155 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1800b3e80 0x7ff18006d3d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff174009870 tx=0x7ff17402a270 comp rx=0 tx=0).stop 2024-05-20T18:27:39.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.591+0000 7ff186110640 1 -- 172.21.15.5:0/23752155 shutdown_connections 2024-05-20T18:27:39.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.591+0000 7ff186110640 1 --2- 172.21.15.5:0/23752155 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1800b3e80 0x7ff18006d3d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:39.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.591+0000 7ff186110640 1 --2- 172.21.15.5:0/23752155 >> v2:172.21.15.63:3300/0 conn(0x7ff180162820 0x7ff1800b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:39.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.591+0000 7ff186110640 1 -- 172.21.15.5:0/23752155 >> 172.21.15.5:0/23752155 conn(0x7ff1800d4bd0 msgr2=0x7ff1800d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:39.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.591+0000 7ff186110640 1 -- 172.21.15.5:0/23752155 shutdown_connections 2024-05-20T18:27:39.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.591+0000 7ff186110640 1 -- 172.21.15.5:0/23752155 wait complete. 2024-05-20T18:27:39.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.593+0000 7ff186110640 1 Processor -- start 2024-05-20T18:27:39.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.593+0000 7ff186110640 1 -- start start 2024-05-20T18:27:39.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.595+0000 7ff186110640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff18006d910 0x7ff1800cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:39.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.595+0000 7ff186110640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1800b3e80 0x7ff1800ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:39.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.595+0000 7ff186110640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff180162820 0x7ff1800caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:39.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.595+0000 7ff186110640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff1800d0300 con 0x7ff18006d910 2024-05-20T18:27:39.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.595+0000 7ff186110640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff1800d4260 con 0x7ff1800b3e80 2024-05-20T18:27:39.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.595+0000 7ff186110640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff1800b91a0 con 0x7ff180162820 2024-05-20T18:27:39.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.595+0000 7ff17f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff18006d910 0x7ff1800cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:39.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.595+0000 7ff17f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff18006d910 0x7ff1800cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53054/0 (socket says 172.21.15.5:53054) 2024-05-20T18:27:39.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.595+0000 7ff17f7fe640 1 -- 172.21.15.5:0/2095935554 learned_addr learned my addr 172.21.15.5:0/2095935554 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:39.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.595+0000 7ff17effd640 1 --2- 172.21.15.5:0/2095935554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1800b3e80 0x7ff1800ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:39.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.595+0000 7ff17ffff640 1 --2- 172.21.15.5:0/2095935554 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff180162820 0x7ff1800caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:39.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.595+0000 7ff17f7fe640 1 -- 172.21.15.5:0/2095935554 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff180162820 msgr2=0x7ff1800caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:39.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.595+0000 7ff17f7fe640 1 --2- 172.21.15.5:0/2095935554 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff180162820 0x7ff1800caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:39.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.595+0000 7ff17f7fe640 1 -- 172.21.15.5:0/2095935554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1800b3e80 msgr2=0x7ff1800ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:39.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.595+0000 7ff17f7fe640 1 --2- 172.21.15.5:0/2095935554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1800b3e80 0x7ff1800ca510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:39.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.595+0000 7ff17f7fe640 1 -- 172.21.15.5:0/2095935554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1800ce260 con 0x7ff18006d910 2024-05-20T18:27:39.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.595+0000 7ff17ffff640 1 --2- 172.21.15.5:0/2095935554 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff180162820 0x7ff1800caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:39.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.595+0000 7ff17f7fe640 1 --2- 172.21.15.5:0/2095935554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff18006d910 0x7ff1800cdd20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff17000e8a0 tx=0x7ff17000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:39.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.596+0000 7ff167fff640 1 -- 172.21.15.5:0/2095935554 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff170017070 con 0x7ff18006d910 2024-05-20T18:27:39.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.596+0000 7ff167fff640 1 -- 172.21.15.5:0/2095935554 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff17000c940 con 0x7ff18006d910 2024-05-20T18:27:39.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.596+0000 7ff167fff640 1 -- 172.21.15.5:0/2095935554 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17000cd30 con 0x7ff18006d910 2024-05-20T18:27:39.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.596+0000 7ff186110640 1 -- 172.21.15.5:0/2095935554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1800ce6e0 con 0x7ff18006d910 2024-05-20T18:27:39.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.596+0000 7ff186110640 1 -- 172.21.15.5:0/2095935554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1800c9a30 con 0x7ff18006d910 2024-05-20T18:27:39.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.597+0000 7ff186110640 1 -- 172.21.15.5:0/2095935554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff180075b00 con 0x7ff18006d910 2024-05-20T18:27:39.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.597+0000 7ff167fff640 1 -- 172.21.15.5:0/2095935554 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff17001a070 con 0x7ff18006d910 2024-05-20T18:27:39.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.598+0000 7ff167fff640 1 --2- 172.21.15.5:0/2095935554 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff16c06be30 0x7ff16c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:39.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.598+0000 7ff167fff640 1 -- 172.21.15.5:0/2095935554 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff17008b3b0 con 0x7ff18006d910 2024-05-20T18:27:39.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.598+0000 7ff17effd640 1 --2- 172.21.15.5:0/2095935554 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff16c06be30 0x7ff16c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:39.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.598+0000 7ff17effd640 1 --2- 172.21.15.5:0/2095935554 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff16c06be30 0x7ff16c06e340 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff174034000 tx=0x7ff17402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:39.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.603+0000 7ff167fff640 1 -- 172.21.15.5:0/2095935554 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff170059a20 con 0x7ff18006d910 2024-05-20T18:27:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.785+0000 7ff186110640 1 -- 172.21.15.5:0/2095935554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff1800d5390 con 0x7ff18006d910 2024-05-20T18:27:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.785+0000 7ff167fff640 1 -- 172.21.15.5:0/2095935554 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7ff170059320 con 0x7ff18006d910 2024-05-20T18:27:39.785 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:27:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.788+0000 7ff186110640 1 -- 172.21.15.5:0/2095935554 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff16c06be30 msgr2=0x7ff16c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.788+0000 7ff186110640 1 --2- 172.21.15.5:0/2095935554 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff16c06be30 0x7ff16c06e340 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff174034000 tx=0x7ff17402e000 comp rx=0 tx=0).stop 2024-05-20T18:27:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.788+0000 7ff186110640 1 -- 172.21.15.5:0/2095935554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff18006d910 msgr2=0x7ff1800cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.788+0000 7ff186110640 1 --2- 172.21.15.5:0/2095935554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff18006d910 0x7ff1800cdd20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff17000e8a0 tx=0x7ff17000ed60 comp rx=0 tx=0).stop 2024-05-20T18:27:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.788+0000 7ff186110640 1 -- 172.21.15.5:0/2095935554 shutdown_connections 2024-05-20T18:27:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.788+0000 7ff186110640 1 --2- 172.21.15.5:0/2095935554 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff16c06be30 0x7ff16c06e340 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.788+0000 7ff186110640 1 --2- 172.21.15.5:0/2095935554 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff180162820 0x7ff1800caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.788+0000 7ff186110640 1 --2- 172.21.15.5:0/2095935554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1800b3e80 0x7ff1800ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.788+0000 7ff186110640 1 --2- 172.21.15.5:0/2095935554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff18006d910 0x7ff1800cdd20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.788+0000 7ff186110640 1 -- 172.21.15.5:0/2095935554 >> 172.21.15.5:0/2095935554 conn(0x7ff1800d4bd0 msgr2=0x7ff180160eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.788+0000 7ff186110640 1 -- 172.21.15.5:0/2095935554 shutdown_connections 2024-05-20T18:27:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.788+0000 7ff186110640 1 -- 172.21.15.5:0/2095935554 wait complete. 2024-05-20T18:27:39.799 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T18:27:39.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.918+0000 7f75880de640 1 Processor -- start 2024-05-20T18:27:39.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.918+0000 7f75880de640 1 -- start start 2024-05-20T18:27:39.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.920+0000 7f75880de640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7580163a10 0x7f75800d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:39.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.920+0000 7f75880de640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75800d0370 con 0x7f75800b4010 2024-05-20T18:27:39.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.920+0000 7f75880de640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75800d0650 con 0x7f75800d14b0 2024-05-20T18:27:39.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.920+0000 7f75880de640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f75800d3ca0 con 0x7f7580163a10 2024-05-20T18:27:39.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.920+0000 7f7585e7a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7580163a10 0x7f75800d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:39.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.920+0000 7f7585679640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f75800b4010 0x7f75800cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:41054/0 (socket says 172.21.15.5:41054) 2024-05-20T18:27:39.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.920+0000 7f7585679640 1 -- 172.21.15.5:0/1245403045 learned_addr learned my addr 172.21.15.5:0/1245403045 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:39.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.920+0000 7f7585e7a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7580163a10 0x7f75800d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53068/0 (socket says 172.21.15.5:53068) 2024-05-20T18:27:39.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.920+0000 7f7585e7a640 1 -- 172.21.15.5:0/1245403045 >> v1:172.21.15.63:6789/0 conn(0x7f75800d14b0 legacy=0x7f75800d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:39.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.920+0000 7f7585e7a640 1 -- 172.21.15.5:0/1245403045 >> v1:172.21.15.5:6789/0 conn(0x7f75800b4010 legacy=0x7f75800cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:39.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.920+0000 7f7585e7a640 1 -- 172.21.15.5:0/1245403045 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75800d3650 con 0x7f7580163a10 2024-05-20T18:27:39.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.920+0000 7f7584e78640 1 -- 172.21.15.5:0/1245403045 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 988199367 0 0) 0x7f75800d0370 con 0x7f75800b4010 2024-05-20T18:27:39.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.921+0000 7f7585e7a640 1 --2- 172.21.15.5:0/1245403045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7580163a10 0x7f75800d4130 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f757c0097a0 tx=0x7f757c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef9a3e6d30e28a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:39.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.921+0000 7f7584e78640 1 -- 172.21.15.5:0/1245403045 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c037070 con 0x7f7580163a10 2024-05-20T18:27:39.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.921+0000 7f7584e78640 1 -- 172.21.15.5:0/1245403045 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f757c032440 con 0x7f7580163a10 2024-05-20T18:27:39.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.921+0000 7f7584e78640 1 -- 172.21.15.5:0/1245403045 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c032810 con 0x7f7580163a10 2024-05-20T18:27:39.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.922+0000 7f75880de640 1 -- 172.21.15.5:0/1245403045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7580163a10 msgr2=0x7f75800d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:39.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.922+0000 7f75880de640 1 --2- 172.21.15.5:0/1245403045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7580163a10 0x7f75800d4130 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f757c0097a0 tx=0x7f757c02a230 comp rx=0 tx=0).stop 2024-05-20T18:27:39.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.922+0000 7f75880de640 1 -- 172.21.15.5:0/1245403045 shutdown_connections 2024-05-20T18:27:39.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.922+0000 7f75880de640 1 --2- 172.21.15.5:0/1245403045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7580163a10 0x7f75800d4130 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:39.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.922+0000 7f75880de640 1 -- 172.21.15.5:0/1245403045 >> 172.21.15.5:0/1245403045 conn(0x7f758006d240 msgr2=0x7f758006d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:39.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.922+0000 7f75880de640 1 -- 172.21.15.5:0/1245403045 shutdown_connections 2024-05-20T18:27:39.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.922+0000 7f75880de640 1 -- 172.21.15.5:0/1245403045 wait complete. 2024-05-20T18:27:39.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.924+0000 7f75880de640 1 Processor -- start 2024-05-20T18:27:39.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.924+0000 7f75880de640 1 -- start start 2024-05-20T18:27:39.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.925+0000 7f75880de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75800b4010 0x7f7580156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:39.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.925+0000 7f75880de640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75800d14b0 0x7f7580154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:39.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.925+0000 7f75880de640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7580163a10 0x7f7580157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:39.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.925+0000 7f75880de640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f75800b8390 con 0x7f75800b4010 2024-05-20T18:27:39.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.925+0000 7f75880de640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f75800b8210 con 0x7f75800d14b0 2024-05-20T18:27:39.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.925+0000 7f75880de640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f75800b8510 con 0x7f7580163a10 2024-05-20T18:27:39.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.925+0000 7f7585e7a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75800b4010 0x7f7580156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:39.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.925+0000 7f7585e7a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75800b4010 0x7f7580156530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53074/0 (socket says 172.21.15.5:53074) 2024-05-20T18:27:39.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.925+0000 7f7585e7a640 1 -- 172.21.15.5:0/2049662193 learned_addr learned my addr 172.21.15.5:0/2049662193 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:39.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.925+0000 7f758667b640 1 --2- 172.21.15.5:0/2049662193 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7580163a10 0x7f7580157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:39.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.925+0000 7f7585679640 1 --2- 172.21.15.5:0/2049662193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75800d14b0 0x7f7580154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:39.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.925+0000 7f7585e7a640 1 -- 172.21.15.5:0/2049662193 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7580163a10 msgr2=0x7f7580157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:39.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.925+0000 7f7585e7a640 1 --2- 172.21.15.5:0/2049662193 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7580163a10 0x7f7580157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:39.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.925+0000 7f7585e7a640 1 -- 172.21.15.5:0/2049662193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75800d14b0 msgr2=0x7f7580154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:39.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.925+0000 7f7585e7a640 1 --2- 172.21.15.5:0/2049662193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75800d14b0 0x7f7580154010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:39.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.925+0000 7f7585e7a640 1 -- 172.21.15.5:0/2049662193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75801554f0 con 0x7f75800b4010 2024-05-20T18:27:39.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.925+0000 7f758667b640 1 --2- 172.21.15.5:0/2049662193 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7580163a10 0x7f7580157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:39.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.926+0000 7f7585e7a640 1 --2- 172.21.15.5:0/2049662193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75800b4010 0x7f7580156530 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f757c0097a0 tx=0x7f757c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:39.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.926+0000 7f75767fc640 1 -- 172.21.15.5:0/2049662193 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c037050 con 0x7f75800b4010 2024-05-20T18:27:39.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.926+0000 7f75767fc640 1 -- 172.21.15.5:0/2049662193 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f757c038040 con 0x7f75800b4010 2024-05-20T18:27:39.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.926+0000 7f75880de640 1 -- 172.21.15.5:0/2049662193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7580154550 con 0x7f75800b4010 2024-05-20T18:27:39.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.926+0000 7f75767fc640 1 -- 172.21.15.5:0/2049662193 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c004490 con 0x7f75800b4010 2024-05-20T18:27:39.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.926+0000 7f75880de640 1 -- 172.21.15.5:0/2049662193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7580155a50 con 0x7f75800b4010 2024-05-20T18:27:39.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.927+0000 7f75880de640 1 -- 172.21.15.5:0/2049662193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7580075b00 con 0x7f75800b4010 2024-05-20T18:27:39.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.928+0000 7f75767fc640 1 -- 172.21.15.5:0/2049662193 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f757c030900 con 0x7f75800b4010 2024-05-20T18:27:39.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.928+0000 7f75767fc640 1 --2- 172.21.15.5:0/2049662193 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f756406be30 0x7f756406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:39.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.928+0000 7f75767fc640 1 -- 172.21.15.5:0/2049662193 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f757c0ac0e0 con 0x7f75800b4010 2024-05-20T18:27:39.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.928+0000 7f7585679640 1 --2- 172.21.15.5:0/2049662193 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f756406be30 0x7f756406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:39.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.929+0000 7f7585679640 1 --2- 172.21.15.5:0/2049662193 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f756406be30 0x7f756406e340 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f758015ecf0 tx=0x7f7570009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:39.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:39.933+0000 7f75767fc640 1 -- 172.21.15.5:0/2049662193 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7580154550 con 0x7f75800b4010 2024-05-20T18:27:40.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.116+0000 7f75880de640 1 -- 172.21.15.5:0/2049662193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f75800d3050 con 0x7f75800b4010 2024-05-20T18:27:40.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.117+0000 7f75767fc640 1 -- 172.21.15.5:0/2049662193 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f75801554f0 con 0x7f75800b4010 2024-05-20T18:27:40.117 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:27:40.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.119+0000 7f75880de640 1 -- 172.21.15.5:0/2049662193 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f756406be30 msgr2=0x7f756406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:40.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.119+0000 7f75880de640 1 --2- 172.21.15.5:0/2049662193 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f756406be30 0x7f756406e340 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f758015ecf0 tx=0x7f7570009040 comp rx=0 tx=0).stop 2024-05-20T18:27:40.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.119+0000 7f75880de640 1 -- 172.21.15.5:0/2049662193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75800b4010 msgr2=0x7f7580156530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:40.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.119+0000 7f75880de640 1 --2- 172.21.15.5:0/2049662193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75800b4010 0x7f7580156530 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f757c0097a0 tx=0x7f757c0046c0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.119+0000 7f75880de640 1 -- 172.21.15.5:0/2049662193 shutdown_connections 2024-05-20T18:27:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.119+0000 7f75880de640 1 --2- 172.21.15.5:0/2049662193 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f756406be30 0x7f756406e340 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.119+0000 7f75880de640 1 --2- 172.21.15.5:0/2049662193 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7580163a10 0x7f7580157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.119+0000 7f75880de640 1 --2- 172.21.15.5:0/2049662193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75800d14b0 0x7f7580154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.119+0000 7f75880de640 1 --2- 172.21.15.5:0/2049662193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75800b4010 0x7f7580156530 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.119+0000 7f75880de640 1 -- 172.21.15.5:0/2049662193 >> 172.21.15.5:0/2049662193 conn(0x7f758006d240 msgr2=0x7f75800b46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:40.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.119+0000 7f75880de640 1 -- 172.21.15.5:0/2049662193 shutdown_connections 2024-05-20T18:27:40.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.119+0000 7f75880de640 1 -- 172.21.15.5:0/2049662193 wait complete. 2024-05-20T18:27:40.131 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:27:40.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.247+0000 7f2e98c0d640 1 Processor -- start 2024-05-20T18:27:40.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.247+0000 7f2e98c0d640 1 -- start start 2024-05-20T18:27:40.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.248+0000 7f2e98c0d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2e94162820 0x7f2e940b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:40.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.248+0000 7f2e98c0d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2e940b3e80 0x7f2e9406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:40.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.248+0000 7f2e98c0d640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e9406dd20 con 0x7f2e9406d910 2024-05-20T18:27:40.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.248+0000 7f2e98c0d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f2e940d0630 con 0x7f2e940b3e80 2024-05-20T18:27:40.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.248+0000 7f2e98c0d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2e940d39c0 con 0x7f2e94162820 2024-05-20T18:27:40.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.249+0000 7f2e8bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2e940b3e80 0x7f2e9406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:40.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.249+0000 7f2e8bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2e940b3e80 0x7f2e9406d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53088/0 (socket says 172.21.15.5:53088) 2024-05-20T18:27:40.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.249+0000 7f2e8bfff640 1 -- 172.21.15.5:0/3288966592 learned_addr learned my addr 172.21.15.5:0/3288966592 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:40.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.249+0000 7f2e9259c640 1 --2- 172.21.15.5:0/3288966592 >> v2:172.21.15.63:3300/0 conn(0x7f2e94162820 0x7f2e940b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:40.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.249+0000 7f2e8bfff640 1 -- 172.21.15.5:0/3288966592 >> v2:172.21.15.63:3300/0 conn(0x7f2e94162820 msgr2=0x7f2e940b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:40.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.249+0000 7f2e8bfff640 1 --2- 172.21.15.5:0/3288966592 >> v2:172.21.15.63:3300/0 conn(0x7f2e94162820 0x7f2e940b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.249+0000 7f2e8bfff640 1 -- 172.21.15.5:0/3288966592 >> v1:172.21.15.63:6789/0 conn(0x7f2e9406d910 legacy=0x7f2e940d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:40.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.249+0000 7f2e8bfff640 1 -- 172.21.15.5:0/3288966592 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e940d3ca0 con 0x7f2e940b3e80 2024-05-20T18:27:40.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.249+0000 7f2e9259c640 1 --2- 172.21.15.5:0/3288966592 >> v2:172.21.15.63:3300/0 conn(0x7f2e94162820 0x7f2e940b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:40.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.250+0000 7f2e8bfff640 1 --2- 172.21.15.5:0/3288966592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e940b3e80 0x7f2e9406d3d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2e940b7740 tx=0x7f2e7c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b31ca9ab7463da server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:40.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.250+0000 7f2e91d9b640 1 -- 172.21.15.5:0/3288966592 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c037070 con 0x7f2e940b3e80 2024-05-20T18:27:40.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.250+0000 7f2e91d9b640 1 -- 172.21.15.5:0/3288966592 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e7c032440 con 0x7f2e940b3e80 2024-05-20T18:27:40.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.250+0000 7f2e91d9b640 1 -- 172.21.15.5:0/3288966592 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c032810 con 0x7f2e940b3e80 2024-05-20T18:27:40.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.251+0000 7f2e98c0d640 1 -- 172.21.15.5:0/3288966592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e940b3e80 msgr2=0x7f2e9406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:40.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.251+0000 7f2e98c0d640 1 --2- 172.21.15.5:0/3288966592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e940b3e80 0x7f2e9406d3d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2e940b7740 tx=0x7f2e7c02a270 comp rx=0 tx=0).stop 2024-05-20T18:27:40.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.251+0000 7f2e98c0d640 1 -- 172.21.15.5:0/3288966592 shutdown_connections 2024-05-20T18:27:40.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.251+0000 7f2e98c0d640 1 --2- 172.21.15.5:0/3288966592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e940b3e80 0x7f2e9406d3d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.251+0000 7f2e98c0d640 1 --2- 172.21.15.5:0/3288966592 >> v2:172.21.15.63:3300/0 conn(0x7f2e94162820 0x7f2e940b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.251+0000 7f2e98c0d640 1 -- 172.21.15.5:0/3288966592 >> 172.21.15.5:0/3288966592 conn(0x7f2e940d4bd0 msgr2=0x7f2e940d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:40.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.251+0000 7f2e98c0d640 1 -- 172.21.15.5:0/3288966592 shutdown_connections 2024-05-20T18:27:40.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.251+0000 7f2e98c0d640 1 -- 172.21.15.5:0/3288966592 wait complete. 2024-05-20T18:27:40.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.253+0000 7f2e98c0d640 1 Processor -- start 2024-05-20T18:27:40.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.253+0000 7f2e98c0d640 1 -- start start 2024-05-20T18:27:40.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.254+0000 7f2e98c0d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e9406d910 0x7f2e9414e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:40.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.254+0000 7f2e98c0d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e940b3e80 0x7f2e941f89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:40.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.254+0000 7f2e98c0d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e94162820 0x7f2e941f8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:40.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.254+0000 7f2e98c0d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2e940d0300 con 0x7f2e9406d910 2024-05-20T18:27:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.254+0000 7f2e98c0d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2e940d4260 con 0x7f2e940b3e80 2024-05-20T18:27:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.254+0000 7f2e98c0d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2e940b91a0 con 0x7f2e94162820 2024-05-20T18:27:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.254+0000 7f2e9259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e9406d910 0x7f2e9414e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.254+0000 7f2e9259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e9406d910 0x7f2e9414e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53098/0 (socket says 172.21.15.5:53098) 2024-05-20T18:27:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.254+0000 7f2e9259c640 1 -- 172.21.15.5:0/2464501934 learned_addr learned my addr 172.21.15.5:0/2464501934 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.254+0000 7f2e92d9d640 1 --2- 172.21.15.5:0/2464501934 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e94162820 0x7f2e941f8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.254+0000 7f2e8bfff640 1 --2- 172.21.15.5:0/2464501934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e940b3e80 0x7f2e941f89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.254+0000 7f2e9259c640 1 -- 172.21.15.5:0/2464501934 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e94162820 msgr2=0x7f2e941f8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.254+0000 7f2e9259c640 1 --2- 172.21.15.5:0/2464501934 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e94162820 0x7f2e941f8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.254+0000 7f2e9259c640 1 -- 172.21.15.5:0/2464501934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e940b3e80 msgr2=0x7f2e941f89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.254+0000 7f2e9259c640 1 --2- 172.21.15.5:0/2464501934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e940b3e80 0x7f2e941f89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.254+0000 7f2e9259c640 1 -- 172.21.15.5:0/2464501934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e9414e640 con 0x7f2e9406d910 2024-05-20T18:27:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.254+0000 7f2e8bfff640 1 --2- 172.21.15.5:0/2464501934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e940b3e80 0x7f2e941f89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:40.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.255+0000 7f2e9259c640 1 --2- 172.21.15.5:0/2464501934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e9406d910 0x7f2e9414e100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2e8400d800 tx=0x7f2e8400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:40.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.255+0000 7f2e8affd640 1 -- 172.21.15.5:0/2464501934 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e84004030 con 0x7f2e9406d910 2024-05-20T18:27:40.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.255+0000 7f2e8affd640 1 -- 172.21.15.5:0/2464501934 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e840041d0 con 0x7f2e9406d910 2024-05-20T18:27:40.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.255+0000 7f2e98c0d640 1 -- 172.21.15.5:0/2464501934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e940cbed0 con 0x7f2e9406d910 2024-05-20T18:27:40.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.255+0000 7f2e8affd640 1 -- 172.21.15.5:0/2464501934 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e840096a0 con 0x7f2e9406d910 2024-05-20T18:27:40.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.255+0000 7f2e98c0d640 1 -- 172.21.15.5:0/2464501934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e940cc490 con 0x7f2e9406d910 2024-05-20T18:27:40.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.256+0000 7f2e98c0d640 1 -- 172.21.15.5:0/2464501934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e94072fc0 con 0x7f2e9406d910 2024-05-20T18:27:40.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.257+0000 7f2e8affd640 1 -- 172.21.15.5:0/2464501934 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2e84009840 con 0x7f2e9406d910 2024-05-20T18:27:40.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.257+0000 7f2e8affd640 1 --2- 172.21.15.5:0/2464501934 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2e7406be30 0x7f2e7406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:40.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.257+0000 7f2e8affd640 1 -- 172.21.15.5:0/2464501934 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2e8408b3e0 con 0x7f2e9406d910 2024-05-20T18:27:40.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.257+0000 7f2e8bfff640 1 --2- 172.21.15.5:0/2464501934 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2e7406be30 0x7f2e7406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:40.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.258+0000 7f2e8bfff640 1 --2- 172.21.15.5:0/2464501934 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2e7406be30 0x7f2e7406e340 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c034000 tx=0x7f2e7c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:40.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.262+0000 7f2e8affd640 1 -- 172.21.15.5:0/2464501934 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2e94072fc0 con 0x7f2e9406d910 2024-05-20T18:27:40.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.484+0000 7f2e98c0d640 1 -- 172.21.15.5:0/2464501934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e94075b00 con 0x7f2e9406d910 2024-05-20T18:27:40.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.484+0000 7f2e8affd640 1 -- 172.21.15.5:0/2464501934 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5168 (secure 0 0 0) 0x7f2e84059a50 con 0x7f2e9406d910 2024-05-20T18:27:40.484 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:40.484 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2982954341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2982954341},{"type":"v1","addr":"172.21.15.5:6836","nonce":2982954341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4032850001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":4032850001},{"type":"v1","addr":"172.21.15.5:6837","nonce":4032850001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6833/424580378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":424580378},{"type":"v1","addr":"172.21.15.63:6833","nonce":424580378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:27:07.164414+0000","modified":"2024-05-20T18:27:29.288681+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/3249423677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3249423677},{"type":"v1","addr":"172.21.15.63:6835","nonce":3249423677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-05-20T18:27:40.484 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 9 2024-05-20T18:27:40.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.486+0000 7f2e98c0d640 1 -- 172.21.15.5:0/2464501934 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2e7406be30 msgr2=0x7f2e7406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:40.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.486+0000 7f2e98c0d640 1 --2- 172.21.15.5:0/2464501934 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2e7406be30 0x7f2e7406e340 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c034000 tx=0x7f2e7c02e000 comp rx=0 tx=0).stop 2024-05-20T18:27:40.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.486+0000 7f2e98c0d640 1 -- 172.21.15.5:0/2464501934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e9406d910 msgr2=0x7f2e9414e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:40.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.486+0000 7f2e98c0d640 1 --2- 172.21.15.5:0/2464501934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e9406d910 0x7f2e9414e100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2e8400d800 tx=0x7f2e8400dcc0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.486+0000 7f2e98c0d640 1 -- 172.21.15.5:0/2464501934 shutdown_connections 2024-05-20T18:27:40.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.486+0000 7f2e98c0d640 1 --2- 172.21.15.5:0/2464501934 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2e7406be30 0x7f2e7406e340 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.486+0000 7f2e98c0d640 1 --2- 172.21.15.5:0/2464501934 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e94162820 0x7f2e941f8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.486+0000 7f2e98c0d640 1 --2- 172.21.15.5:0/2464501934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e940b3e80 0x7f2e941f89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.486+0000 7f2e98c0d640 1 --2- 172.21.15.5:0/2464501934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e9406d910 0x7f2e9414e100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.486+0000 7f2e98c0d640 1 -- 172.21.15.5:0/2464501934 >> 172.21.15.5:0/2464501934 conn(0x7f2e940d4bd0 msgr2=0x7f2e940d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:40.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.486+0000 7f2e98c0d640 1 -- 172.21.15.5:0/2464501934 shutdown_connections 2024-05-20T18:27:40.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.486+0000 7f2e98c0d640 1 -- 172.21.15.5:0/2464501934 wait complete. 2024-05-20T18:27:40.497 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:27:40.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.613+0000 7f4033e91640 1 Processor -- start 2024-05-20T18:27:40.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.613+0000 7f4033e91640 1 -- start start 2024-05-20T18:27:40.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.614+0000 7f4033e91640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f402c0b6d90 0x7f402c15f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:40.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.614+0000 7f4033e91640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f402c0d3220 0x7f402c0d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:40.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.614+0000 7f4033e91640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f402c0d0550 con 0x7f402c0cf250 2024-05-20T18:27:40.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.614+0000 7f4033e91640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f402c15f9a0 con 0x7f402c0b6d90 2024-05-20T18:27:40.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.614+0000 7f4033e91640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f402c0d0b10 con 0x7f402c0d3220 2024-05-20T18:27:40.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.614+0000 7f403142c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f402c0b6d90 0x7f402c15f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:40.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.614+0000 7f403142c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f402c0b6d90 0x7f402c15f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53110/0 (socket says 172.21.15.5:53110) 2024-05-20T18:27:40.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.614+0000 7f403142c640 1 -- 172.21.15.5:0/1748208171 learned_addr learned my addr 172.21.15.5:0/1748208171 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:40.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.615+0000 7f403242e640 1 --2- 172.21.15.5:0/1748208171 >> v2:172.21.15.63:3300/0 conn(0x7f402c0d3220 0x7f402c0d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:40.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.615+0000 7f403142c640 1 -- 172.21.15.5:0/1748208171 >> v2:172.21.15.63:3300/0 conn(0x7f402c0d3220 msgr2=0x7f402c0d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:40.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.615+0000 7f403142c640 1 --2- 172.21.15.5:0/1748208171 >> v2:172.21.15.63:3300/0 conn(0x7f402c0d3220 0x7f402c0d3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.615+0000 7f403142c640 1 -- 172.21.15.5:0/1748208171 >> v1:172.21.15.63:6789/0 conn(0x7f402c0cf250 legacy=0x7f402c0b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:40.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.615+0000 7f403142c640 1 -- 172.21.15.5:0/1748208171 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402c0d0830 con 0x7f402c0b6d90 2024-05-20T18:27:40.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.616+0000 7f403142c640 1 --2- 172.21.15.5:0/1748208171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f402c0b6d90 0x7f402c15f460 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f402c0d11c0 tx=0x7f401c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caec573318f3ae61 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:40.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.616+0000 7f4030c2b640 1 -- 172.21.15.5:0/1748208171 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c037070 con 0x7f402c0b6d90 2024-05-20T18:27:40.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.616+0000 7f4030c2b640 1 -- 172.21.15.5:0/1748208171 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f401c032440 con 0x7f402c0b6d90 2024-05-20T18:27:40.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.616+0000 7f4030c2b640 1 -- 172.21.15.5:0/1748208171 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c032810 con 0x7f402c0b6d90 2024-05-20T18:27:40.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.616+0000 7f4033e91640 1 -- 172.21.15.5:0/1748208171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f402c0b6d90 msgr2=0x7f402c15f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:40.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.616+0000 7f4033e91640 1 --2- 172.21.15.5:0/1748208171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f402c0b6d90 0x7f402c15f460 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f402c0d11c0 tx=0x7f401c02a230 comp rx=0 tx=0).stop 2024-05-20T18:27:40.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.616+0000 7f4033e91640 1 -- 172.21.15.5:0/1748208171 shutdown_connections 2024-05-20T18:27:40.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.616+0000 7f4033e91640 1 --2- 172.21.15.5:0/1748208171 >> v2:172.21.15.63:3300/0 conn(0x7f402c0d3220 0x7f402c0d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.616+0000 7f4033e91640 1 --2- 172.21.15.5:0/1748208171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f402c0b6d90 0x7f402c15f460 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.616+0000 7f4033e91640 1 -- 172.21.15.5:0/1748208171 >> 172.21.15.5:0/1748208171 conn(0x7f402c0d1860 msgr2=0x7f402c0d1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:40.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.617+0000 7f4033e91640 1 -- 172.21.15.5:0/1748208171 shutdown_connections 2024-05-20T18:27:40.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.617+0000 7f4033e91640 1 -- 172.21.15.5:0/1748208171 wait complete. 2024-05-20T18:27:40.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.618+0000 7f4033e91640 1 Processor -- start 2024-05-20T18:27:40.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.618+0000 7f4033e91640 1 -- start start 2024-05-20T18:27:40.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.620+0000 7f4033e91640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f402c0cf250 0x7f402c1f6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:40.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.620+0000 7f4033e91640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f402c0d3220 0x7f402c160f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:40.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.620+0000 7f4033e91640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f402c1614d0 0x7f402c0cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:40.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.620+0000 7f4033e91640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f402c0d2d00 con 0x7f402c1614d0 2024-05-20T18:27:40.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.620+0000 7f4033e91640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f402c0d27d0 con 0x7f402c0d3220 2024-05-20T18:27:40.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.620+0000 7f4033e91640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f402c0d0140 con 0x7f402c0cf250 2024-05-20T18:27:40.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.620+0000 7f403242e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f402c1614d0 0x7f402c0cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:40.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.620+0000 7f403242e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f402c1614d0 0x7f402c0cab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53112/0 (socket says 172.21.15.5:53112) 2024-05-20T18:27:40.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.620+0000 7f403242e640 1 -- 172.21.15.5:0/4248506304 learned_addr learned my addr 172.21.15.5:0/4248506304 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:40.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.620+0000 7f403142c640 1 --2- 172.21.15.5:0/4248506304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f402c0d3220 0x7f402c160f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:40.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.620+0000 7f4031c2d640 1 --2- 172.21.15.5:0/4248506304 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f402c0cf250 0x7f402c1f6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:40.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.620+0000 7f403242e640 1 -- 172.21.15.5:0/4248506304 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f402c0cf250 msgr2=0x7f402c1f6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:40.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.620+0000 7f403242e640 1 --2- 172.21.15.5:0/4248506304 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f402c0cf250 0x7f402c1f6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.620+0000 7f403242e640 1 -- 172.21.15.5:0/4248506304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f402c0d3220 msgr2=0x7f402c160f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:40.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.620+0000 7f403242e640 1 --2- 172.21.15.5:0/4248506304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f402c0d3220 0x7f402c160f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.620+0000 7f403242e640 1 -- 172.21.15.5:0/4248506304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402c161d10 con 0x7f402c1614d0 2024-05-20T18:27:40.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.620+0000 7f403242e640 1 --2- 172.21.15.5:0/4248506304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f402c1614d0 0x7f402c0cab30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4024007e60 tx=0x7f402400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:40.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.621+0000 7f40227fc640 1 -- 172.21.15.5:0/4248506304 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4024014420 con 0x7f402c1614d0 2024-05-20T18:27:40.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.621+0000 7f40227fc640 1 -- 172.21.15.5:0/4248506304 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4024019070 con 0x7f402c1614d0 2024-05-20T18:27:40.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.621+0000 7f4033e91640 1 -- 172.21.15.5:0/4248506304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f402c161ff0 con 0x7f402c1614d0 2024-05-20T18:27:40.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.621+0000 7f40227fc640 1 -- 172.21.15.5:0/4248506304 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4024014820 con 0x7f402c1614d0 2024-05-20T18:27:40.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.621+0000 7f4033e91640 1 -- 172.21.15.5:0/4248506304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f402c1625b0 con 0x7f402c1614d0 2024-05-20T18:27:40.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.622+0000 7f4033e91640 1 -- 172.21.15.5:0/4248506304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f402c0d1d00 con 0x7f402c1614d0 2024-05-20T18:27:40.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.622+0000 7f40227fc640 1 -- 172.21.15.5:0/4248506304 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f40240149c0 con 0x7f402c1614d0 2024-05-20T18:27:40.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.623+0000 7f40227fc640 1 --2- 172.21.15.5:0/4248506304 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f401006be30 0x7f401006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:40.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.623+0000 7f40227fc640 1 -- 172.21.15.5:0/4248506304 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f402408b880 con 0x7f402c1614d0 2024-05-20T18:27:40.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.623+0000 7f4031c2d640 1 --2- 172.21.15.5:0/4248506304 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f401006be30 0x7f401006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:40.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.623+0000 7f4031c2d640 1 --2- 172.21.15.5:0/4248506304 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f401006be30 0x7f401006e340 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4028006fd0 tx=0x7f4028009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:40.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.627+0000 7f40227fc640 1 -- 172.21.15.5:0/4248506304 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f402c0d1d00 con 0x7f402c1614d0 2024-05-20T18:27:40.738 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:27:40.741 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:27:40.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.813+0000 7f4033e91640 1 -- 172.21.15.5:0/4248506304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f402c161aa0 con 0x7f402c1614d0 2024-05-20T18:27:40.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.815+0000 7f40227fc640 1 -- 172.21.15.5:0/4248506304 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18394 (secure 0 0 0) 0x7f40240600c0 con 0x7f402c1614d0 2024-05-20T18:27:40.814 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:40.814 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:27:13.817633+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:27:05.476596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T18:27:06.137762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T18:27:09.829920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:27:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.817+0000 7f4033e91640 1 -- 172.21.15.5:0/4248506304 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f401006be30 msgr2=0x7f401006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.817+0000 7f4033e91640 1 --2- 172.21.15.5:0/4248506304 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f401006be30 0x7f401006e340 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4028006fd0 tx=0x7f4028009060 comp rx=0 tx=0).stop 2024-05-20T18:27:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.817+0000 7f4033e91640 1 -- 172.21.15.5:0/4248506304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f402c1614d0 msgr2=0x7f402c0cab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.817+0000 7f4033e91640 1 --2- 172.21.15.5:0/4248506304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f402c1614d0 0x7f402c0cab30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4024007e60 tx=0x7f402400e4b0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.817+0000 7f4033e91640 1 -- 172.21.15.5:0/4248506304 shutdown_connections 2024-05-20T18:27:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.817+0000 7f4033e91640 1 --2- 172.21.15.5:0/4248506304 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f401006be30 0x7f401006e340 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.817+0000 7f4033e91640 1 --2- 172.21.15.5:0/4248506304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f402c1614d0 0x7f402c0cab30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.817+0000 7f4033e91640 1 --2- 172.21.15.5:0/4248506304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f402c0d3220 0x7f402c160f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.817+0000 7f4033e91640 1 --2- 172.21.15.5:0/4248506304 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f402c0cf250 0x7f402c1f6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.817+0000 7f4033e91640 1 -- 172.21.15.5:0/4248506304 >> 172.21.15.5:0/4248506304 conn(0x7f402c0d1860 msgr2=0x7f402c0b6900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:40.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.817+0000 7f4033e91640 1 -- 172.21.15.5:0/4248506304 shutdown_connections 2024-05-20T18:27:40.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.817+0000 7f4033e91640 1 -- 172.21.15.5:0/4248506304 wait complete. 2024-05-20T18:27:40.828 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:27:40.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.984+0000 7f94da557640 1 Processor -- start 2024-05-20T18:27:40.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.984+0000 7f94da557640 1 -- start start 2024-05-20T18:27:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.986+0000 7f94da557640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f94d40cd710 0x7f94d4162e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.986+0000 7f94da557640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f94d406d6b0 0x7f94d415f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.986+0000 7f94da557640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f94d40cf3d0 0x7f94d40cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.986+0000 7f94da557640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f94d40d0ba0 con 0x7f94d406d6b0 2024-05-20T18:27:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.986+0000 7f94da557640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f94d40cfd20 con 0x7f94d40cf3d0 2024-05-20T18:27:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.986+0000 7f94da557640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f94d406c8c0 con 0x7f94d40cd710 2024-05-20T18:27:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.986+0000 7f94d37fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f94d406d6b0 0x7f94d415f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.986+0000 7f94d37fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f94d406d6b0 0x7f94d415f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53120/0 (socket says 172.21.15.5:53120) 2024-05-20T18:27:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.986+0000 7f94d37fe640 1 -- 172.21.15.5:0/4072029650 learned_addr learned my addr 172.21.15.5:0/4072029650 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.986+0000 7f94d8af4640 1 --2- 172.21.15.5:0/4072029650 >> v2:172.21.15.63:3300/0 conn(0x7f94d40cf3d0 0x7f94d40cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.986+0000 7f94d3fff640 1 --2- 172.21.15.5:0/4072029650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f94d40cd710 0x7f94d4162e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.986+0000 7f94d37fe640 1 -- 172.21.15.5:0/4072029650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f94d40cd710 msgr2=0x7f94d4162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:40.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.986+0000 7f94d37fe640 1 --2- 172.21.15.5:0/4072029650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f94d40cd710 0x7f94d4162e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.986+0000 7f94d37fe640 1 -- 172.21.15.5:0/4072029650 >> v2:172.21.15.63:3300/0 conn(0x7f94d40cf3d0 msgr2=0x7f94d40cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:40.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.986+0000 7f94d37fe640 1 --2- 172.21.15.5:0/4072029650 >> v2:172.21.15.63:3300/0 conn(0x7f94d40cf3d0 0x7f94d40cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.986+0000 7f94d37fe640 1 -- 172.21.15.5:0/4072029650 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d406dac0 con 0x7f94d406d6b0 2024-05-20T18:27:40.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.987+0000 7f94d37fe640 1 --2- 172.21.15.5:0/4072029650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94d406d6b0 0x7f94d415f2b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f94d40b2370 tx=0x7f94c802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ac9df724beca12 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:40.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.987+0000 7f94d2ffd640 1 -- 172.21.15.5:0/4072029650 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c8037070 con 0x7f94d406d6b0 2024-05-20T18:27:40.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.987+0000 7f94d2ffd640 1 -- 172.21.15.5:0/4072029650 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94c8032440 con 0x7f94d406d6b0 2024-05-20T18:27:40.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.987+0000 7f94d2ffd640 1 -- 172.21.15.5:0/4072029650 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c8032810 con 0x7f94d406d6b0 2024-05-20T18:27:40.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.988+0000 7f94da557640 1 -- 172.21.15.5:0/4072029650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94d406d6b0 msgr2=0x7f94d415f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:40.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.988+0000 7f94da557640 1 --2- 172.21.15.5:0/4072029650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94d406d6b0 0x7f94d415f2b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f94d40b2370 tx=0x7f94c802a1e0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.988+0000 7f94da557640 1 -- 172.21.15.5:0/4072029650 shutdown_connections 2024-05-20T18:27:40.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.988+0000 7f94da557640 1 --2- 172.21.15.5:0/4072029650 >> v2:172.21.15.63:3300/0 conn(0x7f94d40cf3d0 0x7f94d40cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.988+0000 7f94da557640 1 --2- 172.21.15.5:0/4072029650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94d406d6b0 0x7f94d415f2b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.988+0000 7f94da557640 1 --2- 172.21.15.5:0/4072029650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f94d40cd710 0x7f94d4162e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.988+0000 7f94da557640 1 -- 172.21.15.5:0/4072029650 >> 172.21.15.5:0/4072029650 conn(0x7f94d40b44d0 msgr2=0x7f94d40b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:40.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.988+0000 7f94da557640 1 -- 172.21.15.5:0/4072029650 shutdown_connections 2024-05-20T18:27:40.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.988+0000 7f94da557640 1 -- 172.21.15.5:0/4072029650 wait complete. 2024-05-20T18:27:40.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.990+0000 7f94da557640 1 Processor -- start 2024-05-20T18:27:40.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.990+0000 7f94da557640 1 -- start start 2024-05-20T18:27:40.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.991+0000 7f94da557640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f94d406d6b0 0x7f94d415b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:40.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.991+0000 7f94da557640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94d40cd710 0x7f94d41ea920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:40.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.991+0000 7f94da557640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94d40cf3d0 0x7f94d40c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:40.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.991+0000 7f94da557640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f94d41639a0 con 0x7f94d40cf3d0 2024-05-20T18:27:40.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.991+0000 7f94da557640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f94d41636c0 con 0x7f94d40cd710 2024-05-20T18:27:40.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.991+0000 7f94da557640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f94d4163c80 con 0x7f94d406d6b0 2024-05-20T18:27:40.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.991+0000 7f94d8af4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94d40cf3d0 0x7f94d40c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:40.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.991+0000 7f94d8af4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94d40cf3d0 0x7f94d40c8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53128/0 (socket says 172.21.15.5:53128) 2024-05-20T18:27:40.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.991+0000 7f94d8af4640 1 -- 172.21.15.5:0/734221552 learned_addr learned my addr 172.21.15.5:0/734221552 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:40.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.991+0000 7f94d3fff640 1 --2- 172.21.15.5:0/734221552 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f94d406d6b0 0x7f94d415b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:40.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.991+0000 7f94d37fe640 1 --2- 172.21.15.5:0/734221552 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94d40cd710 0x7f94d41ea920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:40.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.992+0000 7f94d8af4640 1 -- 172.21.15.5:0/734221552 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f94d406d6b0 msgr2=0x7f94d415b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:40.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.992+0000 7f94d8af4640 1 --2- 172.21.15.5:0/734221552 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f94d406d6b0 0x7f94d415b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.992+0000 7f94d8af4640 1 -- 172.21.15.5:0/734221552 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94d40cd710 msgr2=0x7f94d41ea920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:40.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.992+0000 7f94d8af4640 1 --2- 172.21.15.5:0/734221552 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94d40cd710 0x7f94d41ea920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:40.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.992+0000 7f94d8af4640 1 -- 172.21.15.5:0/734221552 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d41eae60 con 0x7f94d40cf3d0 2024-05-20T18:27:40.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.992+0000 7f94d8af4640 1 --2- 172.21.15.5:0/734221552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94d40cf3d0 0x7f94d40c8c60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f94cc007d70 tx=0x7f94cc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:40.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.992+0000 7f94d0ff9640 1 -- 172.21.15.5:0/734221552 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc014420 con 0x7f94d40cf3d0 2024-05-20T18:27:40.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.992+0000 7f94d0ff9640 1 -- 172.21.15.5:0/734221552 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94cc019070 con 0x7f94d40cf3d0 2024-05-20T18:27:40.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.992+0000 7f94da557640 1 -- 172.21.15.5:0/734221552 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94d415df80 con 0x7f94d40cf3d0 2024-05-20T18:27:40.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.992+0000 7f94d0ff9640 1 -- 172.21.15.5:0/734221552 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc014820 con 0x7f94d40cf3d0 2024-05-20T18:27:40.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.992+0000 7f94da557640 1 -- 172.21.15.5:0/734221552 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94d41e9e40 con 0x7f94d40cf3d0 2024-05-20T18:27:40.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.993+0000 7f94da557640 1 -- 172.21.15.5:0/734221552 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94d4075b00 con 0x7f94d40cf3d0 2024-05-20T18:27:40.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.994+0000 7f94d0ff9640 1 -- 172.21.15.5:0/734221552 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f94cc0149c0 con 0x7f94d40cf3d0 2024-05-20T18:27:40.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.994+0000 7f94d0ff9640 1 --2- 172.21.15.5:0/734221552 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f94c006be60 0x7f94c006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:40.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.994+0000 7f94d0ff9640 1 -- 172.21.15.5:0/734221552 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f94cc08b8e0 con 0x7f94d40cf3d0 2024-05-20T18:27:40.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.994+0000 7f94d3fff640 1 --2- 172.21.15.5:0/734221552 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f94c006be60 0x7f94c006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:40.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.995+0000 7f94d3fff640 1 --2- 172.21.15.5:0/734221552 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f94c006be60 0x7f94c006e370 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f94d4162d60 tx=0x7f94c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:40.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:40.999+0000 7f94d0ff9640 1 -- 172.21.15.5:0/734221552 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f94cc0597f0 con 0x7f94d40cf3d0 2024-05-20T18:27:41.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.227+0000 7f94da557640 1 -- 172.21.15.5:0/734221552 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94d40c94d0 con 0x7f94d40cf3d0 2024-05-20T18:27:41.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.228+0000 7f94d0ff9640 1 -- 172.21.15.5:0/734221552 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5168 (secure 0 0 0) 0x7f94cc05daf0 con 0x7f94d40cf3d0 2024-05-20T18:27:41.227 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:41.227 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2982954341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2982954341},{"type":"v1","addr":"172.21.15.5:6836","nonce":2982954341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4032850001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":4032850001},{"type":"v1","addr":"172.21.15.5:6837","nonce":4032850001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6833/424580378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":424580378},{"type":"v1","addr":"172.21.15.63:6833","nonce":424580378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:27:07.164414+0000","modified":"2024-05-20T18:27:29.288681+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/3249423677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3249423677},{"type":"v1","addr":"172.21.15.63:6835","nonce":3249423677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-05-20T18:27:41.227 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 9 2024-05-20T18:27:41.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.230+0000 7f94da557640 1 -- 172.21.15.5:0/734221552 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f94c006be60 msgr2=0x7f94c006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:41.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.230+0000 7f94da557640 1 --2- 172.21.15.5:0/734221552 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f94c006be60 0x7f94c006e370 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f94d4162d60 tx=0x7f94c4008040 comp rx=0 tx=0).stop 2024-05-20T18:27:41.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.230+0000 7f94da557640 1 -- 172.21.15.5:0/734221552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94d40cf3d0 msgr2=0x7f94d40c8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:41.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.230+0000 7f94da557640 1 --2- 172.21.15.5:0/734221552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94d40cf3d0 0x7f94d40c8c60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f94cc007d70 tx=0x7f94cc00e4b0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.230+0000 7f94da557640 1 -- 172.21.15.5:0/734221552 shutdown_connections 2024-05-20T18:27:41.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.230+0000 7f94da557640 1 --2- 172.21.15.5:0/734221552 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f94c006be60 0x7f94c006e370 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.230+0000 7f94da557640 1 --2- 172.21.15.5:0/734221552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94d40cf3d0 0x7f94d40c8c60 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.230+0000 7f94da557640 1 --2- 172.21.15.5:0/734221552 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94d40cd710 0x7f94d41ea920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.230+0000 7f94da557640 1 --2- 172.21.15.5:0/734221552 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f94d406d6b0 0x7f94d415b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.230+0000 7f94da557640 1 -- 172.21.15.5:0/734221552 >> 172.21.15.5:0/734221552 conn(0x7f94d40b44d0 msgr2=0x7f94d40b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:41.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.230+0000 7f94da557640 1 -- 172.21.15.5:0/734221552 shutdown_connections 2024-05-20T18:27:41.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.230+0000 7f94da557640 1 -- 172.21.15.5:0/734221552 wait complete. 2024-05-20T18:27:41.643 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T18:27:07.164414+0000', 'modified': '2024-05-20T18:27:29.288681+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4279}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4279': {'gid': 4279, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.63:6835/3249423677', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6834', 'nonce': 3249423677}, {'type': 'v1', 'addr': '172.21.15.63:6835', 'nonce': 3249423677}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4279, 'qdb_cluster': [4279]} 2024-05-20T18:27:41.644 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-20T18:27:41.644 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-20T18:27:41.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.763+0000 7f1f66b2f640 1 Processor -- start 2024-05-20T18:27:41.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.764+0000 7f1f66b2f640 1 -- start start 2024-05-20T18:27:41.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.765+0000 7f1f66b2f640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1f60162820 0x7f1f600b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:41.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.765+0000 7f1f66b2f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1f600b3e80 0x7f1f6006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:41.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.765+0000 7f1f66b2f640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f6006dd20 con 0x7f1f6006d910 2024-05-20T18:27:41.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.765+0000 7f1f66b2f640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f1f600d0630 con 0x7f1f600b3e80 2024-05-20T18:27:41.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.765+0000 7f1f66b2f640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1f600d39c0 con 0x7f1f60162820 2024-05-20T18:27:41.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.765+0000 7f1f5ffff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1f600b3e80 0x7f1f6006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:41.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.765+0000 7f1f5ffff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1f600b3e80 0x7f1f6006d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53136/0 (socket says 172.21.15.5:53136) 2024-05-20T18:27:41.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.765+0000 7f1f650cc640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1f6006d910 0x7f1f600d27a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:41060/0 (socket says 172.21.15.5:41060) 2024-05-20T18:27:41.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.765+0000 7f1f5ffff640 1 -- 172.21.15.5:0/604532131 learned_addr learned my addr 172.21.15.5:0/604532131 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:41.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.765+0000 7f1f648cb640 1 --2- 172.21.15.5:0/604532131 >> v2:172.21.15.63:3300/0 conn(0x7f1f60162820 0x7f1f600b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:41.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.765+0000 7f1f5ffff640 1 -- 172.21.15.5:0/604532131 >> v2:172.21.15.63:3300/0 conn(0x7f1f60162820 msgr2=0x7f1f600b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:41.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.765+0000 7f1f5ffff640 1 --2- 172.21.15.5:0/604532131 >> v2:172.21.15.63:3300/0 conn(0x7f1f60162820 0x7f1f600b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.765+0000 7f1f5ffff640 1 -- 172.21.15.5:0/604532131 >> v1:172.21.15.5:6789/0 conn(0x7f1f6006d910 legacy=0x7f1f600d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:41.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.765+0000 7f1f5ffff640 1 -- 172.21.15.5:0/604532131 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f600d3ca0 con 0x7f1f600b3e80 2024-05-20T18:27:41.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.765+0000 7f1f5f7fe640 1 -- 172.21.15.5:0/604532131 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 231933490 0 0) 0x7f1f58001490 con 0x7f1f6006d910 2024-05-20T18:27:41.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.766+0000 7f1f5ffff640 1 --2- 172.21.15.5:0/604532131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f600b3e80 0x7f1f6006d3d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1f600b7740 tx=0x7f1f5402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b31880b7cef62216 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:41.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.766+0000 7f1f5f7fe640 1 -- 172.21.15.5:0/604532131 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f54037070 con 0x7f1f600b3e80 2024-05-20T18:27:41.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.766+0000 7f1f5f7fe640 1 -- 172.21.15.5:0/604532131 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f54032440 con 0x7f1f600b3e80 2024-05-20T18:27:41.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.766+0000 7f1f5f7fe640 1 -- 172.21.15.5:0/604532131 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f54032810 con 0x7f1f600b3e80 2024-05-20T18:27:41.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.767+0000 7f1f66b2f640 1 -- 172.21.15.5:0/604532131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f600b3e80 msgr2=0x7f1f6006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:41.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.767+0000 7f1f66b2f640 1 --2- 172.21.15.5:0/604532131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f600b3e80 0x7f1f6006d3d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1f600b7740 tx=0x7f1f5402a270 comp rx=0 tx=0).stop 2024-05-20T18:27:41.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.767+0000 7f1f66b2f640 1 -- 172.21.15.5:0/604532131 shutdown_connections 2024-05-20T18:27:41.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.767+0000 7f1f66b2f640 1 --2- 172.21.15.5:0/604532131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f600b3e80 0x7f1f6006d3d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.767+0000 7f1f66b2f640 1 --2- 172.21.15.5:0/604532131 >> v2:172.21.15.63:3300/0 conn(0x7f1f60162820 0x7f1f600b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.767+0000 7f1f66b2f640 1 -- 172.21.15.5:0/604532131 >> 172.21.15.5:0/604532131 conn(0x7f1f600d4bd0 msgr2=0x7f1f600d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:41.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.767+0000 7f1f66b2f640 1 -- 172.21.15.5:0/604532131 shutdown_connections 2024-05-20T18:27:41.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.767+0000 7f1f66b2f640 1 -- 172.21.15.5:0/604532131 wait complete. 2024-05-20T18:27:41.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.769+0000 7f1f66b2f640 1 Processor -- start 2024-05-20T18:27:41.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.769+0000 7f1f66b2f640 1 -- start start 2024-05-20T18:27:41.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.770+0000 7f1f66b2f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f6006d910 0x7f1f601f9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:41.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.770+0000 7f1f66b2f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f600b3e80 0x7f1f60150e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:41.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.770+0000 7f1f66b2f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f60162820 0x7f1f600cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:41.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.770+0000 7f1f66b2f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1f600d0300 con 0x7f1f60162820 2024-05-20T18:27:41.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.770+0000 7f1f66b2f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1f600d4260 con 0x7f1f600b3e80 2024-05-20T18:27:41.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.770+0000 7f1f66b2f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1f600b91a0 con 0x7f1f6006d910 2024-05-20T18:27:41.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.771+0000 7f1f650cc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f60162820 0x7f1f600cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:41.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.771+0000 7f1f650cc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f60162820 0x7f1f600cad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53150/0 (socket says 172.21.15.5:53150) 2024-05-20T18:27:41.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.771+0000 7f1f650cc640 1 -- 172.21.15.5:0/2909172630 learned_addr learned my addr 172.21.15.5:0/2909172630 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:41.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.771+0000 7f1f5ffff640 1 --2- 172.21.15.5:0/2909172630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f600b3e80 0x7f1f60150e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:41.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.771+0000 7f1f648cb640 1 --2- 172.21.15.5:0/2909172630 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f6006d910 0x7f1f601f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:41.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.771+0000 7f1f650cc640 1 -- 172.21.15.5:0/2909172630 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f6006d910 msgr2=0x7f1f601f9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:41.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.771+0000 7f1f650cc640 1 --2- 172.21.15.5:0/2909172630 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f6006d910 0x7f1f601f9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.771+0000 7f1f650cc640 1 -- 172.21.15.5:0/2909172630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f600b3e80 msgr2=0x7f1f60150e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:41.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.771+0000 7f1f650cc640 1 --2- 172.21.15.5:0/2909172630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f600b3e80 0x7f1f60150e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.771+0000 7f1f650cc640 1 -- 172.21.15.5:0/2909172630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f60151940 con 0x7f1f60162820 2024-05-20T18:27:41.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.771+0000 7f1f648cb640 1 --2- 172.21.15.5:0/2909172630 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f6006d910 0x7f1f601f9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:41.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.771+0000 7f1f650cc640 1 --2- 172.21.15.5:0/2909172630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f60162820 0x7f1f600cad10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1f5800c700 tx=0x7f1f5800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:41.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.771+0000 7f1f5d7fa640 1 -- 172.21.15.5:0/2909172630 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f58018070 con 0x7f1f60162820 2024-05-20T18:27:41.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.771+0000 7f1f5d7fa640 1 -- 172.21.15.5:0/2909172630 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f5800ce10 con 0x7f1f60162820 2024-05-20T18:27:41.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.771+0000 7f1f66b2f640 1 -- 172.21.15.5:0/2909172630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f60151c20 con 0x7f1f60162820 2024-05-20T18:27:41.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.771+0000 7f1f5d7fa640 1 -- 172.21.15.5:0/2909172630 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5800a880 con 0x7f1f60162820 2024-05-20T18:27:41.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.771+0000 7f1f66b2f640 1 -- 172.21.15.5:0/2909172630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f601f9200 con 0x7f1f60162820 2024-05-20T18:27:41.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.774+0000 7f1f5d7fa640 1 -- 172.21.15.5:0/2909172630 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1f5801b070 con 0x7f1f60162820 2024-05-20T18:27:41.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.774+0000 7f1f5d7fa640 1 --2- 172.21.15.5:0/2909172630 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1f4c06be30 0x7f1f4c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:41.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.774+0000 7f1f5d7fa640 1 -- 172.21.15.5:0/2909172630 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f1f5808a7d0 con 0x7f1f60162820 2024-05-20T18:27:41.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.774+0000 7f1f648cb640 1 --2- 172.21.15.5:0/2909172630 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1f4c06be30 0x7f1f4c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:41.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.775+0000 7f1f648cb640 1 --2- 172.21.15.5:0/2909172630 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1f4c06be30 0x7f1f4c06e340 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1f50009a20 tx=0x7f1f50008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:41.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.775+0000 7f1f6d9d5740 1 Processor -- start 2024-05-20T18:27:41.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.775+0000 7f1f6d9d5740 1 -- start start 2024-05-20T18:27:41.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.776+0000 7f1f6d9d5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e3806270 0x55d0e3be2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:41.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.776+0000 7f1f6d9d5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d0e37dd3b0 0x55d0e3ba5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:41.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.776+0000 7f1f6d9d5740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d0e37e5590 0x55d0e3bed350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:41.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.776+0000 7f1f6d9d5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d0e3b0a4d0 con 0x55d0e3806270 2024-05-20T18:27:41.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.776+0000 7f1f6d9d5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d0e3ab23e0 con 0x55d0e37dd3b0 2024-05-20T18:27:41.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.776+0000 7f1f6d9d5740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55d0e3b06490 con 0x55d0e37e5590 2024-05-20T18:27:41.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.776+0000 7f1f648cb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e3806270 0x55d0e3be2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:41.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.776+0000 7f1f648cb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e3806270 0x55d0e3be2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53160/0 (socket says 172.21.15.5:53160) 2024-05-20T18:27:41.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.777+0000 7f1f648cb640 1 -- 172.21.15.5:0/490650375 learned_addr learned my addr 172.21.15.5:0/490650375 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:41.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.777+0000 7f1f5ffff640 1 --2- 172.21.15.5:0/490650375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d0e37dd3b0 0x55d0e3ba5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:41.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.777+0000 7f1f650cc640 1 --2- 172.21.15.5:0/490650375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d0e37e5590 0x55d0e3bed350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:41.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.777+0000 7f1f648cb640 1 -- 172.21.15.5:0/490650375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d0e37e5590 msgr2=0x55d0e3bed350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:41.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.777+0000 7f1f648cb640 1 --2- 172.21.15.5:0/490650375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d0e37e5590 0x55d0e3bed350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.777+0000 7f1f648cb640 1 -- 172.21.15.5:0/490650375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d0e37dd3b0 msgr2=0x55d0e3ba5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:41.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.777+0000 7f1f648cb640 1 --2- 172.21.15.5:0/490650375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d0e37dd3b0 0x55d0e3ba5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.777+0000 7f1f648cb640 1 -- 172.21.15.5:0/490650375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0e381e2d0 con 0x55d0e3806270 2024-05-20T18:27:41.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.777+0000 7f1f648cb640 1 --2- 172.21.15.5:0/490650375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e3806270 0x55d0e3be2b60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1f50004260 tx=0x7f1f50013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22da57a84df8f59c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:41.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.777+0000 7f1f467fc640 1 -- 172.21.15.5:0/490650375 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f50027070 con 0x55d0e3806270 2024-05-20T18:27:41.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.777+0000 7f1f467fc640 1 -- 172.21.15.5:0/490650375 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f50003a20 con 0x55d0e3806270 2024-05-20T18:27:41.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.777+0000 7f1f467fc640 1 -- 172.21.15.5:0/490650375 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5000f040 con 0x55d0e3806270 2024-05-20T18:27:41.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.778+0000 7f1f6d9d5740 1 -- 172.21.15.5:0/490650375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e3806270 msgr2=0x55d0e3be2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:41.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.778+0000 7f1f6d9d5740 1 --2- 172.21.15.5:0/490650375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e3806270 0x55d0e3be2b60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1f50004260 tx=0x7f1f50013000 comp rx=0 tx=0).stop 2024-05-20T18:27:41.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.778+0000 7f1f6d9d5740 1 -- 172.21.15.5:0/490650375 shutdown_connections 2024-05-20T18:27:41.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.778+0000 7f1f6d9d5740 1 --2- 172.21.15.5:0/490650375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d0e37e5590 0x55d0e3bed350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.778+0000 7f1f6d9d5740 1 --2- 172.21.15.5:0/490650375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d0e37dd3b0 0x55d0e3ba5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.778+0000 7f1f6d9d5740 1 --2- 172.21.15.5:0/490650375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e3806270 0x55d0e3be2b60 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.778+0000 7f1f6d9d5740 1 -- 172.21.15.5:0/490650375 >> 172.21.15.5:0/490650375 conn(0x55d0e37fa3c0 msgr2=0x55d0e3ba6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:41.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.778+0000 7f1f6d9d5740 1 -- 172.21.15.5:0/490650375 shutdown_connections 2024-05-20T18:27:41.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.778+0000 7f1f6d9d5740 1 -- 172.21.15.5:0/490650375 wait complete. 2024-05-20T18:27:41.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.779+0000 7f1f6d9d5740 1 -- start start 2024-05-20T18:27:41.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.779+0000 7f1f6d9d5740 1 Processor -- start 2024-05-20T18:27:41.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.780+0000 7f1f6d9d5740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d0e37dd3b0 0x55d0e3bd2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:41.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.780+0000 7f1f6d9d5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e37e5590 0x55d0e3bd4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:41.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.780+0000 7f1f6d9d5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d0e3806270 0x55d0e3bfa960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:41.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.780+0000 7f1f6d9d5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d0e3ac58e0 con 0x55d0e37e5590 2024-05-20T18:27:41.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.780+0000 7f1f6d9d5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d0e3ba1b20 con 0x55d0e3806270 2024-05-20T18:27:41.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.780+0000 7f1f6d9d5740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55d0e3beab40 con 0x55d0e37dd3b0 2024-05-20T18:27:41.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.780+0000 7f1f6d9d5740 10 client.0 _ll_register_callbacks cb 0x55d0e3a929c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:27:41.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.780+0000 7f1f5ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e37e5590 0x55d0e3bd4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:41.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.781+0000 7f1f5ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e37e5590 0x55d0e3bd4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53176/0 (socket says 172.21.15.5:53176) 2024-05-20T18:27:41.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.781+0000 7f1f5ffff640 1 -- 172.21.15.5:0/2905350053 learned_addr learned my addr 172.21.15.5:0/2905350053 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:41.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.781+0000 7f1f650cc640 1 --2- 172.21.15.5:0/2905350053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d0e3806270 0x55d0e3bfa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:41.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.781+0000 7f1f648cb640 1 --2- 172.21.15.5:0/2905350053 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d0e37dd3b0 0x55d0e3bd2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:41.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.781+0000 7f1f5ffff640 1 -- 172.21.15.5:0/2905350053 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d0e37dd3b0 msgr2=0x55d0e3bd2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:41.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.781+0000 7f1f5ffff640 1 --2- 172.21.15.5:0/2905350053 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d0e37dd3b0 0x55d0e3bd2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.781+0000 7f1f5ffff640 1 -- 172.21.15.5:0/2905350053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d0e3806270 msgr2=0x55d0e3bfa960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:41.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.781+0000 7f1f5ffff640 1 --2- 172.21.15.5:0/2905350053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d0e3806270 0x55d0e3bfa960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.781+0000 7f1f5ffff640 1 -- 172.21.15.5:0/2905350053 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0e38e86b0 con 0x55d0e37e5590 2024-05-20T18:27:41.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.781+0000 7f1f650cc640 1 --2- 172.21.15.5:0/2905350053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d0e3806270 0x55d0e3bfa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:41.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.781+0000 7f1f5ffff640 1 --2- 172.21.15.5:0/2905350053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e37e5590 0x55d0e3bd4740 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1f54009960 tx=0x7f1f54004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4639e46becd2dd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:41.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.781+0000 7f1f3f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:27:41.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.781+0000 7f1f3f7fe640 1 -- 172.21.15.5:0/2905350053 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f54042070 con 0x55d0e37e5590 2024-05-20T18:27:41.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.781+0000 7f1f3f7fe640 1 -- 172.21.15.5:0/2905350053 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f1f54032dc0 con 0x55d0e37e5590 2024-05-20T18:27:41.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.781+0000 7f1f6d9d5740 1 -- 172.21.15.5:0/2905350053 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f50003a20 con 0x55d0e37e5590 2024-05-20T18:27:41.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.781+0000 7f1f3f7fe640 1 -- 172.21.15.5:0/2905350053 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f1f5404d050 con 0x55d0e37e5590 2024-05-20T18:27:41.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.781+0000 7f1f6d9d5740 10 client.4812 fetch_fsmap learned FSMap version 9 2024-05-20T18:27:41.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.781+0000 7f1f6d9d5740 1 -- 172.21.15.5:0/2905350053 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55d0e3bea5c0 con 0x55d0e37e5590 2024-05-20T18:27:41.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.782+0000 7f1f3f7fe640 1 -- 172.21.15.5:0/2905350053 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f50003a20 con 0x55d0e37e5590 2024-05-20T18:27:41.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.782+0000 7f1f3f7fe640 1 -- 172.21.15.5:0/2905350053 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f1f54050020 con 0x55d0e37e5590 2024-05-20T18:27:41.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.782+0000 7f1f6d9d5740 10 client.4812 fetch_fsmap finished waiting for FSMap version 9 2024-05-20T18:27:41.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.782+0000 7f1f6d9d5740 10 client.4812 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-20T18:27:41.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.782+0000 7f1f6d9d5740 20 client.4812 populate_metadata read hostname 'smithi005' 2024-05-20T18:27:41.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.782+0000 7f1f6d9d5740 1 --2- 172.21.15.5:0/2905350053 >> [v2:172.21.15.63:6834/3249423677,v1:172.21.15.63:6835/3249423677] conn(0x55d0e37c4e40 0x55d0e3c12360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:41.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.782+0000 7f1f6d9d5740 4 client.4812 mds_command: new command op to 4279 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:27:41.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.782+0000 7f1f6d9d5740 1 -- 172.21.15.5:0/2905350053 --> [v2:172.21.15.63:6834/3249423677,v1:172.21.15.63:6835/3249423677] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0e3b17c00 con 0x55d0e37c4e40 2024-05-20T18:27:41.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.782+0000 7f1f648cb640 1 --2- 172.21.15.5:0/2905350053 >> [v2:172.21.15.63:6834/3249423677,v1:172.21.15.63:6835/3249423677] conn(0x55d0e37c4e40 0x55d0e3c12360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:41.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.782+0000 7f1f648cb640 1 --2- 172.21.15.5:0/2905350053 >> [v2:172.21.15.63:6834/3249423677,v1:172.21.15.63:6835/3249423677] conn(0x55d0e37c4e40 0x55d0e3c12360 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a21e9f3488549629 server_cookie=23f5b1ac1ef107e7 in_seq=0 out_seq=0 2024-05-20T18:27:41.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.782+0000 7f1f3f7fe640 10 client.4812 ms_handle_connect on v2:172.21.15.63:6834/3249423677 2024-05-20T18:27:41.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.783+0000 7f1f3f7fe640 1 -- 172.21.15.5:0/2905350053 <== mds.0 v2:172.21.15.63:6834/3249423677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d0e3b17c00 con 0x55d0e37c4e40 2024-05-20T18:27:41.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.783+0000 7f1f3f7fe640 10 client.4812 handle_command_reply: tid=0 2024-05-20T18:27:41.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.783+0000 7f1f6d9d5740 1 client.4812 shutdown 2024-05-20T18:27:41.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.783+0000 7f1f6d9d5740 1 -- 172.21.15.5:0/2905350053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e37e5590 msgr2=0x55d0e3bd4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:41.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.783+0000 7f1f6d9d5740 1 --2- 172.21.15.5:0/2905350053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e37e5590 0x55d0e3bd4740 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1f54009960 tx=0x7f1f54004730 comp rx=0 tx=0).stop 2024-05-20T18:27:41.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.784+0000 7f1f6d9d5740 1 -- 172.21.15.5:0/2905350053 shutdown_connections 2024-05-20T18:27:41.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.784+0000 7f1f6d9d5740 1 --2- 172.21.15.5:0/2905350053 >> [v2:172.21.15.63:6834/3249423677,v1:172.21.15.63:6835/3249423677] conn(0x55d0e37c4e40 0x55d0e3c12360 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.784+0000 7f1f6d9d5740 1 --2- 172.21.15.5:0/2905350053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d0e3806270 0x55d0e3bfa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.784+0000 7f1f3f7fe640 0 client.4812 ms_handle_reset on v2:172.21.15.63:6834/3249423677 2024-05-20T18:27:41.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.784+0000 7f1f6d9d5740 1 --2- 172.21.15.5:0/2905350053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e37e5590 0x55d0e3bd4740 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.784+0000 7f1f6d9d5740 1 --2- 172.21.15.5:0/2905350053 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d0e37dd3b0 0x55d0e3bd2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.784+0000 7f1f6d9d5740 1 -- 172.21.15.5:0/2905350053 >> 172.21.15.5:0/2905350053 conn(0x55d0e37fa3c0 msgr2=0x55d0e3ba7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:41.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.784+0000 7f1f6d9d5740 1 -- 172.21.15.5:0/2905350053 shutdown_connections 2024-05-20T18:27:41.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.784+0000 7f1f6d9d5740 1 -- 172.21.15.5:0/2905350053 wait complete. 2024-05-20T18:27:41.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.784+0000 7f1f6d9d5740 20 client.4812 trim_cache size 0 max 16384 2024-05-20T18:27:41.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.803+0000 7f1f6d9d5740 1 Processor -- start 2024-05-20T18:27:41.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.803+0000 7f1f6d9d5740 1 -- start start 2024-05-20T18:27:41.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.804+0000 7f1f6d9d5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d0e37dd3b0 0x55d0e3ba88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:41.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.804+0000 7f1f6d9d5740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d0e37e5590 0x55d0e3ba7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:41.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.804+0000 7f1f6d9d5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e3806270 0x55d0e3bd25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:41.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.804+0000 7f1f6d9d5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1f60070d30 con 0x55d0e3806270 2024-05-20T18:27:41.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.804+0000 7f1f6d9d5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1f600b5000 con 0x55d0e37dd3b0 2024-05-20T18:27:41.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.804+0000 7f1f6d9d5740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1f600706e0 con 0x55d0e37e5590 2024-05-20T18:27:41.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.804+0000 7f1f650cc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e3806270 0x55d0e3bd25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:41.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.804+0000 7f1f650cc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e3806270 0x55d0e3bd25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53192/0 (socket says 172.21.15.5:53192) 2024-05-20T18:27:41.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.804+0000 7f1f650cc640 1 -- 172.21.15.5:0/3429927830 learned_addr learned my addr 172.21.15.5:0/3429927830 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:41.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.804+0000 7f1f5ffff640 1 --2- 172.21.15.5:0/3429927830 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d0e37e5590 0x55d0e3ba7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:41.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.804+0000 7f1f648cb640 1 --2- 172.21.15.5:0/3429927830 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d0e37dd3b0 0x55d0e3ba88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:41.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.804+0000 7f1f650cc640 1 -- 172.21.15.5:0/3429927830 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d0e37e5590 msgr2=0x55d0e3ba7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:41.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.804+0000 7f1f650cc640 1 --2- 172.21.15.5:0/3429927830 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d0e37e5590 0x55d0e3ba7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.804+0000 7f1f650cc640 1 -- 172.21.15.5:0/3429927830 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d0e37dd3b0 msgr2=0x55d0e3ba88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:41.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.804+0000 7f1f650cc640 1 --2- 172.21.15.5:0/3429927830 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d0e37dd3b0 0x55d0e3ba88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.804+0000 7f1f650cc640 1 -- 172.21.15.5:0/3429927830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0e3ba7e50 con 0x55d0e3806270 2024-05-20T18:27:41.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.804+0000 7f1f648cb640 1 --2- 172.21.15.5:0/3429927830 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d0e37dd3b0 0x55d0e3ba88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:41.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.805+0000 7f1f650cc640 1 --2- 172.21.15.5:0/3429927830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e3806270 0x55d0e3bd25b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1f58020030 tx=0x7f1f5808a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f532c5efdbfad506 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:41.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.805+0000 7f1f45ffb640 1 -- 172.21.15.5:0/3429927830 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f58061320 con 0x55d0e3806270 2024-05-20T18:27:41.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.805+0000 7f1f45ffb640 1 -- 172.21.15.5:0/3429927830 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f580569b0 con 0x55d0e3806270 2024-05-20T18:27:41.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.805+0000 7f1f45ffb640 1 -- 172.21.15.5:0/3429927830 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5806cbd0 con 0x55d0e3806270 2024-05-20T18:27:41.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.805+0000 7f1f6d9d5740 1 -- 172.21.15.5:0/3429927830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e3806270 msgr2=0x55d0e3bd25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:41.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.805+0000 7f1f6d9d5740 1 --2- 172.21.15.5:0/3429927830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e3806270 0x55d0e3bd25b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1f58020030 tx=0x7f1f5808a040 comp rx=0 tx=0).stop 2024-05-20T18:27:41.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.805+0000 7f1f6d9d5740 1 -- 172.21.15.5:0/3429927830 shutdown_connections 2024-05-20T18:27:41.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.805+0000 7f1f6d9d5740 1 --2- 172.21.15.5:0/3429927830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e3806270 0x55d0e3bd25b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.805+0000 7f1f6d9d5740 1 --2- 172.21.15.5:0/3429927830 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d0e37e5590 0x55d0e3ba7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.805+0000 7f1f6d9d5740 1 --2- 172.21.15.5:0/3429927830 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d0e37dd3b0 0x55d0e3ba88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.805+0000 7f1f6d9d5740 1 -- 172.21.15.5:0/3429927830 >> 172.21.15.5:0/3429927830 conn(0x55d0e37fa3c0 msgr2=0x55d0e3bfb370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:41.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.805+0000 7f1f6d9d5740 1 -- 172.21.15.5:0/3429927830 shutdown_connections 2024-05-20T18:27:41.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.805+0000 7f1f6d9d5740 1 -- 172.21.15.5:0/3429927830 wait complete. 2024-05-20T18:27:41.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.806+0000 7f1f6d9d5740 1 -- start start 2024-05-20T18:27:41.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.806+0000 7f1f6d9d5740 1 Processor -- start 2024-05-20T18:27:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.808+0000 7f1f6d9d5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e37dd3b0 0x55d0e3c02c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.808+0000 7f1f6d9d5740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d0e37e5590 0x55d0e3c03180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.808+0000 7f1f6d9d5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d0e3806270 0x55d0e3bcd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.808+0000 7f1f6d9d5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d0e3bd5060 con 0x55d0e37dd3b0 2024-05-20T18:27:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.808+0000 7f1f6d9d5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d0e3bea8e0 con 0x55d0e3806270 2024-05-20T18:27:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.808+0000 7f1f6d9d5740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55d0e3bca840 con 0x55d0e37e5590 2024-05-20T18:27:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.808+0000 7f1f6d9d5740 10 client.0 _ll_register_callbacks cb 0x7f1f54035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:27:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.808+0000 7f1f648cb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e37dd3b0 0x55d0e3c02c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.808+0000 7f1f648cb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e37dd3b0 0x55d0e3c02c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53202/0 (socket says 172.21.15.5:53202) 2024-05-20T18:27:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.808+0000 7f1f648cb640 1 -- 172.21.15.5:0/151268471 learned_addr learned my addr 172.21.15.5:0/151268471 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.808+0000 7f1f5ffff640 1 --2- 172.21.15.5:0/151268471 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d0e37e5590 0x55d0e3c03180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.808+0000 7f1f650cc640 1 --2- 172.21.15.5:0/151268471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d0e3806270 0x55d0e3bcd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.808+0000 7f1f648cb640 1 -- 172.21.15.5:0/151268471 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d0e37e5590 msgr2=0x55d0e3c03180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.808+0000 7f1f648cb640 1 --2- 172.21.15.5:0/151268471 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d0e37e5590 0x55d0e3c03180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.808+0000 7f1f648cb640 1 -- 172.21.15.5:0/151268471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d0e3806270 msgr2=0x55d0e3bcd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.808+0000 7f1f648cb640 1 --2- 172.21.15.5:0/151268471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d0e3806270 0x55d0e3bcd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.808+0000 7f1f648cb640 1 -- 172.21.15.5:0/151268471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0e3bf91f0 con 0x55d0e37dd3b0 2024-05-20T18:27:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.808+0000 7f1f650cc640 1 --2- 172.21.15.5:0/151268471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d0e3806270 0x55d0e3bcd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.809+0000 7f1f648cb640 1 --2- 172.21.15.5:0/151268471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e37dd3b0 0x55d0e3c02c40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1f50025040 tx=0x7f1f500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854e2815871c67c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:41.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.809+0000 7f1f3f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:27:41.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.809+0000 7f1f3f7fe640 1 -- 172.21.15.5:0/151268471 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f50036030 con 0x55d0e37dd3b0 2024-05-20T18:27:41.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.809+0000 7f1f6d9d5740 1 -- 172.21.15.5:0/151268471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f580569b0 con 0x55d0e37dd3b0 2024-05-20T18:27:41.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.809+0000 7f1f3f7fe640 1 -- 172.21.15.5:0/151268471 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f1f500039a0 con 0x55d0e37dd3b0 2024-05-20T18:27:41.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.809+0000 7f1f3f7fe640 1 -- 172.21.15.5:0/151268471 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f1f50042050 con 0x55d0e37dd3b0 2024-05-20T18:27:41.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.809+0000 7f1f6d9d5740 10 client.4818 fetch_fsmap learned FSMap version 9 2024-05-20T18:27:41.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.809+0000 7f1f6d9d5740 1 -- 172.21.15.5:0/151268471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55d0e3ba1bf0 con 0x55d0e37dd3b0 2024-05-20T18:27:41.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.809+0000 7f1f3f7fe640 1 -- 172.21.15.5:0/151268471 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f580569b0 con 0x55d0e37dd3b0 2024-05-20T18:27:41.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.809+0000 7f1f3f7fe640 1 -- 172.21.15.5:0/151268471 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f1f50045020 con 0x55d0e37dd3b0 2024-05-20T18:27:41.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.809+0000 7f1f6d9d5740 10 client.4818 fetch_fsmap finished waiting for FSMap version 9 2024-05-20T18:27:41.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.809+0000 7f1f6d9d5740 10 client.4818 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-20T18:27:41.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.809+0000 7f1f6d9d5740 20 client.4818 populate_metadata read hostname 'smithi005' 2024-05-20T18:27:41.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.809+0000 7f1f6d9d5740 1 --2- 172.21.15.5:0/151268471 >> [v2:172.21.15.63:6834/3249423677,v1:172.21.15.63:6835/3249423677] conn(0x55d0e3bcdf20 0x55d0e3bd25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:41.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.809+0000 7f1f6d9d5740 4 client.4818 mds_command: new command op to 4279 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T18:27:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.809+0000 7f1f6d9d5740 1 -- 172.21.15.5:0/151268471 --> [v2:172.21.15.63:6834/3249423677,v1:172.21.15.63:6835/3249423677] -- command(tid 0: {"prefix": "status"}) -- 0x55d0e3b11f50 con 0x55d0e3bcdf20 2024-05-20T18:27:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.810+0000 7f1f5ffff640 1 --2- 172.21.15.5:0/151268471 >> [v2:172.21.15.63:6834/3249423677,v1:172.21.15.63:6835/3249423677] conn(0x55d0e3bcdf20 0x55d0e3bd25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.810+0000 7f1f5ffff640 1 --2- 172.21.15.5:0/151268471 >> [v2:172.21.15.63:6834/3249423677,v1:172.21.15.63:6835/3249423677] conn(0x55d0e3bcdf20 0x55d0e3bd25b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7328ee54bbcc21f1 server_cookie=8c7ab6b566fa0c9b in_seq=0 out_seq=0 2024-05-20T18:27:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.810+0000 7f1f3f7fe640 10 client.4818 ms_handle_connect on v2:172.21.15.63:6834/3249423677 2024-05-20T18:27:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.810+0000 7f1f3f7fe640 1 -- 172.21.15.5:0/151268471 <== mds.0 v2:172.21.15.63:6834/3249423677 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55d0e3b11f50 con 0x55d0e3bcdf20 2024-05-20T18:27:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.810+0000 7f1f3f7fe640 10 client.4818 handle_command_reply: tid=0 2024-05-20T18:27:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.810+0000 7f1f6d9d5740 1 client.4818 shutdown 2024-05-20T18:27:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.810+0000 7f1f6d9d5740 1 -- 172.21.15.5:0/151268471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e37dd3b0 msgr2=0x55d0e3c02c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.810+0000 7f1f6d9d5740 1 --2- 172.21.15.5:0/151268471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e37dd3b0 0x55d0e3c02c40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1f50025040 tx=0x7f1f500046c0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.810+0000 7f1f6d9d5740 1 -- 172.21.15.5:0/151268471 shutdown_connections 2024-05-20T18:27:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.810+0000 7f1f6d9d5740 1 --2- 172.21.15.5:0/151268471 >> [v2:172.21.15.63:6834/3249423677,v1:172.21.15.63:6835/3249423677] conn(0x55d0e3bcdf20 0x55d0e3bd25b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.810+0000 7f1f6d9d5740 1 --2- 172.21.15.5:0/151268471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d0e3806270 0x55d0e3bcd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.810+0000 7f1f6d9d5740 1 --2- 172.21.15.5:0/151268471 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d0e37e5590 0x55d0e3c03180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.810+0000 7f1f6d9d5740 1 --2- 172.21.15.5:0/151268471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e37dd3b0 0x55d0e3c02c40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.810+0000 7f1f3f7fe640 0 client.4818 ms_handle_reset on v2:172.21.15.63:6834/3249423677 2024-05-20T18:27:41.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.810+0000 7f1f6d9d5740 1 -- 172.21.15.5:0/151268471 >> 172.21.15.5:0/151268471 conn(0x55d0e37fa3c0 msgr2=0x55d0e3ba86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:41.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.811+0000 7f1f6d9d5740 1 -- 172.21.15.5:0/151268471 shutdown_connections 2024-05-20T18:27:41.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.811+0000 7f1f6d9d5740 1 -- 172.21.15.5:0/151268471 wait complete. 2024-05-20T18:27:41.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.811+0000 7f1f6d9d5740 20 client.4818 trim_cache size 0 max 16384 2024-05-20T18:27:41.815 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:27:41.815 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "399ab250-fd3c-47fa-a1c8-25e8149c9cac", 2024-05-20T18:27:41.815 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-20T18:27:41.815 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 4279, 2024-05-20T18:27:41.815 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-20T18:27:41.816 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-20T18:27:41.816 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-20T18:27:41.816 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 34.034536740999997, 2024-05-20T18:27:41.816 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 9, 2024-05-20T18:27:41.816 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 26, 2024-05-20T18:27:41.816 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 26, 2024-05-20T18:27:41.816 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 40.699056495999997 2024-05-20T18:27:41.816 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:27:41.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.811+0000 7f1f66b2f640 1 -- 172.21.15.5:0/2909172630 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1f4c06be30 msgr2=0x7f1f4c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:41.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.811+0000 7f1f66b2f640 1 --2- 172.21.15.5:0/2909172630 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1f4c06be30 0x7f1f4c06e340 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1f50009a20 tx=0x7f1f50008040 comp rx=0 tx=0).stop 2024-05-20T18:27:41.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.811+0000 7f1f66b2f640 1 -- 172.21.15.5:0/2909172630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f60162820 msgr2=0x7f1f600cad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:41.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.811+0000 7f1f66b2f640 1 --2- 172.21.15.5:0/2909172630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f60162820 0x7f1f600cad10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1f5800c700 tx=0x7f1f5800cbc0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.811+0000 7f1f66b2f640 1 -- 172.21.15.5:0/2909172630 shutdown_connections 2024-05-20T18:27:41.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.811+0000 7f1f66b2f640 1 --2- 172.21.15.5:0/2909172630 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1f4c06be30 0x7f1f4c06e340 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.811+0000 7f1f66b2f640 1 --2- 172.21.15.5:0/2909172630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f60162820 0x7f1f600cad10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.811+0000 7f1f66b2f640 1 --2- 172.21.15.5:0/2909172630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f600b3e80 0x7f1f60150e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.811+0000 7f1f66b2f640 1 --2- 172.21.15.5:0/2909172630 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f6006d910 0x7f1f601f9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:41.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.811+0000 7f1f66b2f640 1 -- 172.21.15.5:0/2909172630 >> 172.21.15.5:0/2909172630 conn(0x7f1f600d4bd0 msgr2=0x7f1f600b8190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:41.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.811+0000 7f1f66b2f640 1 -- 172.21.15.5:0/2909172630 shutdown_connections 2024-05-20T18:27:41.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:41.811+0000 7f1f66b2f640 1 -- 172.21.15.5:0/2909172630 wait complete. 2024-05-20T18:27:41.822 INFO:teuthology.run_tasks:Running task sequential... 2024-05-20T18:27:41.835 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-20T18:27:41.835 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-20T18:27:41.835 INFO:teuthology.task.print:Enabling mgr modules 2024-05-20T18:27:41.835 INFO:teuthology.run_tasks:Running task kclient... 2024-05-20T18:27:41.900 INFO:tasks.kclient:Mounting kernel clients... 2024-05-20T18:27:41.900 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-05-20T18:27:41.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:41.900 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-20T18:27:41.918 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-20T18:27:41.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:41.919 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-20T18:27:41.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:41.987 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-05-20T18:27:42.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:42.082 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-20T18:27:42.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:42.100 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-05-20T18:27:42.171 INFO:teuthology.orchestra.run.smithi005.stderr:Cannot find device "ceph-brx" 2024-05-20T18:27:42.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T18:27:42.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:42.173 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-05-20T18:27:42.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:42.194 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link delete ceph-brx 2024-05-20T18:27:42.268 INFO:teuthology.orchestra.run.smithi063.stderr:Cannot find device "ceph-brx" 2024-05-20T18:27:42.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T18:27:42.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:42.270 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-05-20T18:27:42.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:42.329 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link delete ceph-brx 2024-05-20T18:27:42.401 INFO:teuthology.orchestra.run.smithi063.stderr:Cannot find device "ceph-brx" 2024-05-20T18:27:42.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T18:27:42.403 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-05-20T18:27:42.403 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-20T18:27:42.403 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-20T18:27:42.403 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-20T18:27:42.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.519+0000 7f12f7179640 1 Processor -- start 2024-05-20T18:27:42.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.519+0000 7f12f7179640 1 -- start start 2024-05-20T18:27:42.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.521+0000 7f12f7179640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f12f015d5f0 0x7f12f01576d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:42.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.521+0000 7f12f7179640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12f006c9a0 0x7f12f00ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:42.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.521+0000 7f12f7179640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12f006b900 con 0x7f12f00cec00 2024-05-20T18:27:42.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.521+0000 7f12f7179640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f12f006bbe0 con 0x7f12f015d5f0 2024-05-20T18:27:42.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.521+0000 7f12f7179640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f12f006bec0 con 0x7f12f006c9a0 2024-05-20T18:27:42.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.521+0000 7f12f4f15640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f12f015d5f0 0x7f12f01576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:42.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.521+0000 7f12effff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12f006c9a0 0x7f12f00ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:42.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.521+0000 7f12f5716640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f12f00cec00 0x7f12f006e3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:45260/0 (socket says 172.21.15.5:45260) 2024-05-20T18:27:42.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.521+0000 7f12f4f15640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f12f015d5f0 0x7f12f01576d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:41850/0 (socket says 172.21.15.5:41850) 2024-05-20T18:27:42.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.521+0000 7f12f5716640 1 -- 172.21.15.5:0/56490684 learned_addr learned my addr 172.21.15.5:0/56490684 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:42.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.521+0000 7f12effff640 1 -- 172.21.15.5:0/56490684 >> v2:172.21.15.63:3300/0 conn(0x7f12f015d5f0 msgr2=0x7f12f01576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:42.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.521+0000 7f12effff640 1 --2- 172.21.15.5:0/56490684 >> v2:172.21.15.63:3300/0 conn(0x7f12f015d5f0 0x7f12f01576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:42.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.521+0000 7f12effff640 1 -- 172.21.15.5:0/56490684 >> v1:172.21.15.63:6789/0 conn(0x7f12f00cec00 legacy=0x7f12f006e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:42.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.521+0000 7f12ef7fe640 1 -- 172.21.15.5:0/56490684 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3356425393 0 0) 0x7f12f006b900 con 0x7f12f00cec00 2024-05-20T18:27:42.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.522+0000 7f12effff640 1 -- 172.21.15.5:0/56490684 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12f006c1a0 con 0x7f12f006c9a0 2024-05-20T18:27:42.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.522+0000 7f12f4f15640 1 --2- 172.21.15.5:0/56490684 >> v2:172.21.15.63:3300/0 conn(0x7f12f015d5f0 0x7f12f01576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:27:42.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.522+0000 7f12effff640 1 --2- 172.21.15.5:0/56490684 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12f006c9a0 0x7f12f00ce6c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f12e40097a0 tx=0x7f12e402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ab9ea20bc8ea8bc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:42.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.522+0000 7f12ef7fe640 1 -- 172.21.15.5:0/56490684 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e4037070 con 0x7f12f006c9a0 2024-05-20T18:27:42.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.523+0000 7f12ef7fe640 1 -- 172.21.15.5:0/56490684 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12e402ae50 con 0x7f12f006c9a0 2024-05-20T18:27:42.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.523+0000 7f12ef7fe640 1 -- 172.21.15.5:0/56490684 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e4032660 con 0x7f12f006c9a0 2024-05-20T18:27:42.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.523+0000 7f12f7179640 1 -- 172.21.15.5:0/56490684 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12f006c9a0 msgr2=0x7f12f00ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:42.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.523+0000 7f12f7179640 1 --2- 172.21.15.5:0/56490684 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12f006c9a0 0x7f12f00ce6c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f12e40097a0 tx=0x7f12e402a100 comp rx=0 tx=0).stop 2024-05-20T18:27:42.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.523+0000 7f12f7179640 1 -- 172.21.15.5:0/56490684 shutdown_connections 2024-05-20T18:27:42.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.523+0000 7f12f7179640 1 --2- 172.21.15.5:0/56490684 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12f006c9a0 0x7f12f00ce6c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:42.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.523+0000 7f12f7179640 1 --2- 172.21.15.5:0/56490684 >> v2:172.21.15.63:3300/0 conn(0x7f12f015d5f0 0x7f12f01576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:42.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.523+0000 7f12f7179640 1 -- 172.21.15.5:0/56490684 >> 172.21.15.5:0/56490684 conn(0x7f12f00cd2a0 msgr2=0x7f12f00cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:42.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.523+0000 7f12f7179640 1 -- 172.21.15.5:0/56490684 shutdown_connections 2024-05-20T18:27:42.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.523+0000 7f12f7179640 1 -- 172.21.15.5:0/56490684 wait complete. 2024-05-20T18:27:42.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.525+0000 7f12f7179640 1 Processor -- start 2024-05-20T18:27:42.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.525+0000 7f12f7179640 1 -- start start 2024-05-20T18:27:42.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.526+0000 7f12f7179640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12f006c9a0 0x7f12f015f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:42.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.526+0000 7f12f7179640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12f00cec00 0x7f12f013e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:42.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.526+0000 7f12f7179640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f015d5f0 0x7f12f01563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:42.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.526+0000 7f12f7179640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12f006b520 con 0x7f12f015d5f0 2024-05-20T18:27:42.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.526+0000 7f12f7179640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f12f006cdb0 con 0x7f12f006c9a0 2024-05-20T18:27:42.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.526+0000 7f12f7179640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f12f006b6a0 con 0x7f12f00cec00 2024-05-20T18:27:42.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.527+0000 7f12f5716640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f015d5f0 0x7f12f01563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:42.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.527+0000 7f12f5716640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f015d5f0 0x7f12f01563e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53204/0 (socket says 172.21.15.5:53204) 2024-05-20T18:27:42.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.527+0000 7f12f5716640 1 -- 172.21.15.5:0/2408266749 learned_addr learned my addr 172.21.15.5:0/2408266749 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:42.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.527+0000 7f12f4f15640 1 --2- 172.21.15.5:0/2408266749 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12f006c9a0 0x7f12f015f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:42.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.527+0000 7f12effff640 1 --2- 172.21.15.5:0/2408266749 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12f00cec00 0x7f12f013e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:42.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.527+0000 7f12f5716640 1 -- 172.21.15.5:0/2408266749 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12f00cec00 msgr2=0x7f12f013e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:42.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.527+0000 7f12f5716640 1 --2- 172.21.15.5:0/2408266749 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12f00cec00 0x7f12f013e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:42.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.527+0000 7f12f5716640 1 -- 172.21.15.5:0/2408266749 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12f006c9a0 msgr2=0x7f12f015f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:42.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.527+0000 7f12f5716640 1 --2- 172.21.15.5:0/2408266749 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12f006c9a0 0x7f12f015f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:42.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.527+0000 7f12f5716640 1 -- 172.21.15.5:0/2408266749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12f015ea70 con 0x7f12f015d5f0 2024-05-20T18:27:42.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.527+0000 7f12f5716640 1 --2- 172.21.15.5:0/2408266749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f015d5f0 0x7f12f01563e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f12e800c700 tx=0x7f12e800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:42.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.527+0000 7f12ed7fa640 1 -- 172.21.15.5:0/2408266749 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e8018070 con 0x7f12f015d5f0 2024-05-20T18:27:42.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.527+0000 7f12ed7fa640 1 -- 172.21.15.5:0/2408266749 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12e800ce10 con 0x7f12f015d5f0 2024-05-20T18:27:42.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.527+0000 7f12f7179640 1 -- 172.21.15.5:0/2408266749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12f0155900 con 0x7f12f015d5f0 2024-05-20T18:27:42.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.527+0000 7f12ed7fa640 1 -- 172.21.15.5:0/2408266749 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e800a860 con 0x7f12f015d5f0 2024-05-20T18:27:42.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.527+0000 7f12f7179640 1 -- 172.21.15.5:0/2408266749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12f013eef0 con 0x7f12f015d5f0 2024-05-20T18:27:42.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.528+0000 7f12f7179640 1 -- 172.21.15.5:0/2408266749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12f0155730 con 0x7f12f015d5f0 2024-05-20T18:27:42.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.529+0000 7f12ed7fa640 1 -- 172.21.15.5:0/2408266749 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f12e801b070 con 0x7f12f015d5f0 2024-05-20T18:27:42.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.530+0000 7f12ed7fa640 1 --2- 172.21.15.5:0/2408266749 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f12dc06be30 0x7f12dc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:42.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.530+0000 7f12f4f15640 1 --2- 172.21.15.5:0/2408266749 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f12dc06be30 0x7f12dc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:42.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.530+0000 7f12ed7fa640 1 -- 172.21.15.5:0/2408266749 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f12e808a730 con 0x7f12f015d5f0 2024-05-20T18:27:42.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.530+0000 7f12f4f15640 1 --2- 172.21.15.5:0/2408266749 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f12dc06be30 0x7f12dc06e340 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f12f00b0490 tx=0x7f12e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:42.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.534+0000 7f12ed7fa640 1 -- 172.21.15.5:0/2408266749 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f12e8058e30 con 0x7f12f015d5f0 2024-05-20T18:27:42.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.759+0000 7f12f7179640 1 -- 172.21.15.5:0/2408266749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f12f006f620 con 0x7f12f015d5f0 2024-05-20T18:27:42.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.759+0000 7f12ed7fa640 1 -- 172.21.15.5:0/2408266749 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f12f006f620 con 0x7f12f015d5f0 2024-05-20T18:27:42.759 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-20T18:27:42.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.762+0000 7f12f7179640 1 -- 172.21.15.5:0/2408266749 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f12dc06be30 msgr2=0x7f12dc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:42.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.762+0000 7f12f7179640 1 --2- 172.21.15.5:0/2408266749 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f12dc06be30 0x7f12dc06e340 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f12f00b0490 tx=0x7f12e0008040 comp rx=0 tx=0).stop 2024-05-20T18:27:42.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.762+0000 7f12f7179640 1 -- 172.21.15.5:0/2408266749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f015d5f0 msgr2=0x7f12f01563e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:42.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.762+0000 7f12f7179640 1 --2- 172.21.15.5:0/2408266749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f015d5f0 0x7f12f01563e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f12e800c700 tx=0x7f12e800cbc0 comp rx=0 tx=0).stop 2024-05-20T18:27:42.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.762+0000 7f12f7179640 1 -- 172.21.15.5:0/2408266749 shutdown_connections 2024-05-20T18:27:42.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.762+0000 7f12f7179640 1 --2- 172.21.15.5:0/2408266749 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f12dc06be30 0x7f12dc06e340 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:42.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.762+0000 7f12f7179640 1 --2- 172.21.15.5:0/2408266749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f015d5f0 0x7f12f01563e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:42.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.762+0000 7f12f7179640 1 --2- 172.21.15.5:0/2408266749 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12f00cec00 0x7f12f013e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:42.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.762+0000 7f12f7179640 1 --2- 172.21.15.5:0/2408266749 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12f006c9a0 0x7f12f015f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:42.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.762+0000 7f12f7179640 1 -- 172.21.15.5:0/2408266749 >> 172.21.15.5:0/2408266749 conn(0x7f12f00cd2a0 msgr2=0x7f12f0153420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:42.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.762+0000 7f12f7179640 1 -- 172.21.15.5:0/2408266749 shutdown_connections 2024-05-20T18:27:42.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42.762+0000 7f12f7179640 1 -- 172.21.15.5:0/2408266749 wait complete. 2024-05-20T18:27:42.772 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-20T18:27:42.772 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-20T18:27:42.773 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-05-20T18:27:42.773 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-20T18:27:42.773 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-20T18:27:42.773 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-20T18:27:42.773 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-20T18:27:42.773 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-20T18:27:42.773 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:27:42.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:42.773 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-05-20T18:27:42.790 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T18:27:42.790 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T18:27:42.790 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T18:27:42.790 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:42.790 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-20T18:27:42.790 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:42.791 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:27:42.791 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-05-20T18:27:42.791 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-05-20T18:27:42.791 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:27:42.791 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-05-20T18:27:42.791 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-05-20T18:27:42.791 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-20T18:27:42.791 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:27:42.791 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-05-20T18:27:42.791 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-20T18:27:42.792 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42635sec preferred_lft 42635sec 2024-05-20T18:27:42.792 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-05-20T18:27:42.792 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:42.792 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:27:42.792 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:27:42.792 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-05-20T18:27:42.792 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-20T18:27:42.793 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:27:42.793 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-20T18:27:42.793 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add name ceph-brx type bridge 2024-05-20T18:27:42.793 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr flush dev ceph-brx 2024-05-20T18:27:42.793 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set ceph-brx up 2024-05-20T18:27:42.793 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-20T18:27:42.793 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-20T18:27:42.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:27:42.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:42.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:42.933 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-20T18:27:42.958 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-20T18:27:42.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:42.960 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-05-20T18:27:43.017 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-05-20T18:27:43.017 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-20T18:27:43.017 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-20T18:27:43.017 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-20T18:27:43.018 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-20T18:27:43.018 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:27:43.018 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-20T18:27:43.018 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-20T18:27:43.018 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-20T18:27:43.018 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-20T18:27:43.018 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-20T18:27:43.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:27:43.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:43.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:43.153 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-20T18:27:43.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:43.170 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list-id 2024-05-20T18:27:43.226 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:27:43.226 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-20T18:27:43.227 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T18:27:43.227 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-20T18:27:43.227 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-20T18:27:43.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:27:43.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:43.352 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-20T18:27:43.352 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:27:43.353 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-20T18:27:43.353 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-20T18:27:43.353 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-20T18:27:43.353 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-20T18:27:43.353 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-20T18:27:43.353 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-20T18:27:43.353 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-20T18:27:43.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:27:43.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:43.492 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:27:43.492 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-20T18:27:43.492 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-05-20T18:27:43.492 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set dev brx.0 master ceph-brx 2024-05-20T18:27:43.492 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-20T18:27:43.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:27:43.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:43.574 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:27:43.574 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-20T18:27:43.591 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:27:43.591 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:27:43.591 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:27:43.643 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-20T18:27:43.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:43.644 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:27:43.741 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:27:43.741 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:27:43.741 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:27:43.856 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:27:43.856 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:27:43.856 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:27:43.856 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:27:43.856 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:27:43.863 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:27:43.863 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:27:43.863 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:27:43.864 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:27:43.864 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:27:43.864 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790" 2024-05-20T18:27:43.864 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790:/ 2024-05-20T18:27:43.864 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=399ab250-fd3c-47fa-a1c8-25e8149c9cac" will pass to kernel 2024-05-20T18:27:43.864 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-20T18:27:43.864 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-05-20T18:27:43.864 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-20T18:27:43.865 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-05-20T18:27:43.865 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-20T18:27:43.866 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-20T18:27:43.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:43.867 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:27:43.896 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-20T18:27:43.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T18:27:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:43.995 INFO:teuthology.orchestra.run.smithi005.stdout:client4824 2024-05-20T18:27:43.995 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:27:43.996 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:27:44.012 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-20T18:27:44.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T18:27:44.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:44.110 INFO:teuthology.orchestra.run.smithi005.stdout:client4824 2024-05-20T18:27:44.111 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-20T18:27:44.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T18:27:44.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:44.168 INFO:teuthology.orchestra.run.smithi005.stdout:client4824 2024-05-20T18:27:44.168 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/399ab250-fd3c-47fa-a1c8-25e8149c9cac.client4824/status') 2024-05-20T18:27:44.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:27:44.229 INFO:teuthology.orchestra.run.smithi005.stdout:instance: client.4824 (3)192.168.0.1:0/3305522055 2024-05-20T18:27:44.229 INFO:teuthology.orchestra.run.smithi005.stdout:blocklisted: false 2024-05-20T18:27:44.229 INFO:teuthology.orchestra.run.smithi005.stderr:0+1 records in 2024-05-20T18:27:44.229 INFO:teuthology.orchestra.run.smithi005.stderr:0+1 records out 2024-05-20T18:27:44.229 INFO:teuthology.orchestra.run.smithi005.stderr:69 bytes copied, 2.7779e-05 s, 2.5 MB/s 2024-05-20T18:27:44.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:44.236 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-05-20T18:27:44.237 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-20T18:27:44.237 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-20T18:27:44.237 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-20T18:27:44.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.353+0000 7f28bb377640 1 Processor -- start 2024-05-20T18:27:44.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.353+0000 7f28bb377640 1 -- start start 2024-05-20T18:27:44.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.354+0000 7f28bb377640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f28b40b1e10 0x7f28b415aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:44.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.354+0000 7f28bb377640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f28b4070850 0x7f28b4070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:44.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.354+0000 7f28bb377640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28b40cd580 con 0x7f28b415d530 2024-05-20T18:27:44.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.354+0000 7f28bb377640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f28b40ce190 con 0x7f28b4070850 2024-05-20T18:27:44.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.354+0000 7f28bb377640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f28b40cd860 con 0x7f28b40b1e10 2024-05-20T18:27:44.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.355+0000 7f28b9914640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f28b4070850 0x7f28b4070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:44.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.355+0000 7f28b9914640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f28b4070850 0x7f28b4070c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53218/0 (socket says 172.21.15.5:53218) 2024-05-20T18:27:44.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.355+0000 7f28b9914640 1 -- 172.21.15.5:0/2526663538 learned_addr learned my addr 172.21.15.5:0/2526663538 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:44.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.355+0000 7f28b9113640 1 --1- 172.21.15.5:0/2526663538 >> v1:172.21.15.5:6789/0 conn(0x7f28b415d530 0x7f28b40b1700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:41064/0 (socket says 172.21.15.5:41064) 2024-05-20T18:27:44.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.355+0000 7f28b8912640 1 --2- 172.21.15.5:0/2526663538 >> v2:172.21.15.63:3300/0 conn(0x7f28b40b1e10 0x7f28b415aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:44.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.355+0000 7f28b9914640 1 -- 172.21.15.5:0/2526663538 >> v2:172.21.15.63:3300/0 conn(0x7f28b40b1e10 msgr2=0x7f28b415aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:44.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.355+0000 7f28b9914640 1 --2- 172.21.15.5:0/2526663538 >> v2:172.21.15.63:3300/0 conn(0x7f28b40b1e10 0x7f28b415aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:44.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.355+0000 7f28b9914640 1 -- 172.21.15.5:0/2526663538 >> v1:172.21.15.5:6789/0 conn(0x7f28b415d530 legacy=0x7f28b40b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:44.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.355+0000 7f28b9914640 1 -- 172.21.15.5:0/2526663538 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b40cdb40 con 0x7f28b4070850 2024-05-20T18:27:44.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.355+0000 7f28abfff640 1 -- 172.21.15.5:0/2526663538 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2616416814 0 0) 0x7f28b40cd580 con 0x7f28b415d530 2024-05-20T18:27:44.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.356+0000 7f28b9914640 1 --2- 172.21.15.5:0/2526663538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28b4070850 0x7f28b4070c60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f28b40d3ad0 tx=0x7f28ac02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c67158b6dd2c644b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.356+0000 7f28abfff640 1 -- 172.21.15.5:0/2526663538 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac036070 con 0x7f28b4070850 2024-05-20T18:27:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.356+0000 7f28abfff640 1 -- 172.21.15.5:0/2526663538 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28ac031600 con 0x7f28b4070850 2024-05-20T18:27:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.356+0000 7f28abfff640 1 -- 172.21.15.5:0/2526663538 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac0319d0 con 0x7f28b4070850 2024-05-20T18:27:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.357+0000 7f28bb377640 1 -- 172.21.15.5:0/2526663538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28b4070850 msgr2=0x7f28b4070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.357+0000 7f28bb377640 1 --2- 172.21.15.5:0/2526663538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28b4070850 0x7f28b4070c60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f28b40d3ad0 tx=0x7f28ac02a730 comp rx=0 tx=0).stop 2024-05-20T18:27:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.357+0000 7f28bb377640 1 -- 172.21.15.5:0/2526663538 shutdown_connections 2024-05-20T18:27:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.357+0000 7f28bb377640 1 --2- 172.21.15.5:0/2526663538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28b4070850 0x7f28b4070c60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.357+0000 7f28bb377640 1 --2- 172.21.15.5:0/2526663538 >> v2:172.21.15.63:3300/0 conn(0x7f28b40b1e10 0x7f28b415aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.357+0000 7f28bb377640 1 -- 172.21.15.5:0/2526663538 >> 172.21.15.5:0/2526663538 conn(0x7f28b40d29d0 msgr2=0x7f28b40d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.357+0000 7f28bb377640 1 -- 172.21.15.5:0/2526663538 shutdown_connections 2024-05-20T18:27:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.357+0000 7f28bb377640 1 -- 172.21.15.5:0/2526663538 wait complete. 2024-05-20T18:27:44.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.359+0000 7f28bb377640 1 Processor -- start 2024-05-20T18:27:44.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.359+0000 7f28bb377640 1 -- start start 2024-05-20T18:27:44.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.360+0000 7f28bb377640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28b4070850 0x7f28b4148a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:44.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.360+0000 7f28bb377640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28b40b1e10 0x7f28b41f6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:44.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.361+0000 7f28bb377640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f28b415d530 0x7f28b41f69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:44.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.361+0000 7f28bb377640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f28b40711a0 con 0x7f28b4070850 2024-05-20T18:27:44.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.361+0000 7f28bb377640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f28b40cd230 con 0x7f28b40b1e10 2024-05-20T18:27:44.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.361+0000 7f28bb377640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f28b4071320 con 0x7f28b415d530 2024-05-20T18:27:44.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.361+0000 7f28b9113640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28b4070850 0x7f28b4148a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:44.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.361+0000 7f28b9113640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28b4070850 0x7f28b4148a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53222/0 (socket says 172.21.15.5:53222) 2024-05-20T18:27:44.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.361+0000 7f28b9113640 1 -- 172.21.15.5:0/1329823732 learned_addr learned my addr 172.21.15.5:0/1329823732 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:44.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.361+0000 7f28b8912640 1 --2- 172.21.15.5:0/1329823732 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28b40b1e10 0x7f28b41f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:44.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.361+0000 7f28b9914640 1 --2- 172.21.15.5:0/1329823732 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f28b415d530 0x7f28b41f69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:44.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.361+0000 7f28b9113640 1 -- 172.21.15.5:0/1329823732 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f28b415d530 msgr2=0x7f28b41f69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:44.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.361+0000 7f28b9113640 1 --2- 172.21.15.5:0/1329823732 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f28b415d530 0x7f28b41f69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:44.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.361+0000 7f28b9113640 1 -- 172.21.15.5:0/1329823732 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28b40b1e10 msgr2=0x7f28b41f6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:44.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.361+0000 7f28b9113640 1 --2- 172.21.15.5:0/1329823732 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28b40b1e10 0x7f28b41f6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:44.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.361+0000 7f28b9113640 1 -- 172.21.15.5:0/1329823732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b41f7220 con 0x7f28b4070850 2024-05-20T18:27:44.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.361+0000 7f28b9113640 1 --2- 172.21.15.5:0/1329823732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28b4070850 0x7f28b4148a80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f28b000c410 tx=0x7f28b000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:44.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.361+0000 7f28a9ffb640 1 -- 172.21.15.5:0/1329823732 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b00074e0 con 0x7f28b4070850 2024-05-20T18:27:44.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.361+0000 7f28a9ffb640 1 -- 172.21.15.5:0/1329823732 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28b00047e0 con 0x7f28b4070850 2024-05-20T18:27:44.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.361+0000 7f28bb377640 1 -- 172.21.15.5:0/1329823732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b414b2d0 con 0x7f28b4070850 2024-05-20T18:27:44.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.361+0000 7f28a9ffb640 1 -- 172.21.15.5:0/1329823732 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b0004bd0 con 0x7f28b4070850 2024-05-20T18:27:44.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.362+0000 7f28bb377640 1 -- 172.21.15.5:0/1329823732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b414b890 con 0x7f28b4070850 2024-05-20T18:27:44.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.363+0000 7f28bb377640 1 -- 172.21.15.5:0/1329823732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b4075b00 con 0x7f28b4070850 2024-05-20T18:27:44.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.363+0000 7f28a9ffb640 1 -- 172.21.15.5:0/1329823732 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f28b0011070 con 0x7f28b4070850 2024-05-20T18:27:44.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.364+0000 7f28a9ffb640 1 --2- 172.21.15.5:0/1329823732 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f289806be60 0x7f289806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:44.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.364+0000 7f28a9ffb640 1 -- 172.21.15.5:0/1329823732 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f28b008acc0 con 0x7f28b4070850 2024-05-20T18:27:44.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.364+0000 7f28b8912640 1 --2- 172.21.15.5:0/1329823732 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f289806be60 0x7f289806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:44.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.364+0000 7f28b8912640 1 --2- 172.21.15.5:0/1329823732 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f289806be60 0x7f289806e370 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f28b40b8050 tx=0x7f28a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:44.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.369+0000 7f28a9ffb640 1 -- 172.21.15.5:0/1329823732 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f28b0058c50 con 0x7f28b4070850 2024-05-20T18:27:44.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.591+0000 7f28bb377640 1 -- 172.21.15.5:0/1329823732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f28b40ce930 con 0x7f28b4070850 2024-05-20T18:27:44.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.591+0000 7f28a9ffb640 1 -- 172.21.15.5:0/1329823732 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f28b005cf50 con 0x7f28b4070850 2024-05-20T18:27:44.590 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-20T18:27:44.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.593+0000 7f28bb377640 1 -- 172.21.15.5:0/1329823732 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f289806be60 msgr2=0x7f289806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.593+0000 7f28bb377640 1 --2- 172.21.15.5:0/1329823732 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f289806be60 0x7f289806e370 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f28b40b8050 tx=0x7f28a4008040 comp rx=0 tx=0).stop 2024-05-20T18:27:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.593+0000 7f28bb377640 1 -- 172.21.15.5:0/1329823732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28b4070850 msgr2=0x7f28b4148a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.593+0000 7f28bb377640 1 --2- 172.21.15.5:0/1329823732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28b4070850 0x7f28b4148a80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f28b000c410 tx=0x7f28b000c8d0 comp rx=0 tx=0).stop 2024-05-20T18:27:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.593+0000 7f28bb377640 1 -- 172.21.15.5:0/1329823732 shutdown_connections 2024-05-20T18:27:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.593+0000 7f28bb377640 1 --2- 172.21.15.5:0/1329823732 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f289806be60 0x7f289806e370 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.593+0000 7f28bb377640 1 --2- 172.21.15.5:0/1329823732 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f28b415d530 0x7f28b41f69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.593+0000 7f28bb377640 1 --2- 172.21.15.5:0/1329823732 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28b40b1e10 0x7f28b41f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.593+0000 7f28bb377640 1 --2- 172.21.15.5:0/1329823732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28b4070850 0x7f28b4148a80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.593+0000 7f28bb377640 1 -- 172.21.15.5:0/1329823732 >> 172.21.15.5:0/1329823732 conn(0x7f28b40d29d0 msgr2=0x7f28b40b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.593+0000 7f28bb377640 1 -- 172.21.15.5:0/1329823732 shutdown_connections 2024-05-20T18:27:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44.593+0000 7f28bb377640 1 -- 172.21.15.5:0/1329823732 wait complete. 2024-05-20T18:27:44.604 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-20T18:27:44.604 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-20T18:27:44.604 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-05-20T18:27:44.604 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-05-20T18:27:44.604 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-20T18:27:44.604 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-20T18:27:44.604 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-20T18:27:44.604 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-20T18:27:44.604 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-05-20T18:27:44.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:44.605 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-05-20T18:27:44.621 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T18:27:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T18:27:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T18:27:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-20T18:27:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:44.621 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:27:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-05-20T18:27:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-05-20T18:27:44.621 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:27:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-05-20T18:27:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-05-20T18:27:44.622 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-20T18:27:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:27:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-05-20T18:27:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-20T18:27:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42634sec preferred_lft 42634sec 2024-05-20T18:27:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-05-20T18:27:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:44.622 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:27:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:27:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-05-20T18:27:44.622 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-20T18:27:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:27:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-20T18:27:44.623 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:44.623 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::4474:eaff:fe07:4f9a/64 scope link 2024-05-20T18:27:44.623 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:44.623 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:27:44.623 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T18:27:44.623 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ce4:f4ff:fe5e:7906/64 scope link tentative 2024-05-20T18:27:44.623 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:44.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:44.623 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-20T18:27:44.685 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-20T18:27:44.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:44.687 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-05-20T18:27:44.743 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-05-20T18:27:44.744 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-20T18:27:44.744 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-20T18:27:44.744 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-20T18:27:44.744 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-20T18:27:44.744 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:27:44.744 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-20T18:27:44.744 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-20T18:27:44.745 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-20T18:27:44.745 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-20T18:27:44.745 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-20T18:27:44.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:27:44.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:44.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:44.874 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-20T18:27:44.891 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-20T18:27:44.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:44.892 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list-id 2024-05-20T18:27:44.948 INFO:teuthology.orchestra.run.smithi005.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-05-20T18:27:44.948 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:27:44.948 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-20T18:27:44.948 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T18:27:44.948 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-05-20T18:27:44.949 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-20T18:27:45.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:27:45.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:45.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:45.069 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-20T18:27:45.094 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T18:27:45.095 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T18:27:45.095 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T18:27:45.095 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:45.095 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-20T18:27:45.095 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:45.095 INFO:teuthology.orchestra.run.smithi005.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-20T18:27:45.095 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 5e:e7:2d:8b:38:50 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-20T18:27:45.095 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-20T18:27:45.095 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:45.095 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::5ce7:2dff:fe8b:3850/64 scope link tentative 2024-05-20T18:27:45.095 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:45.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:45.098 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-20T18:27:45.162 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T18:27:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T18:27:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T18:27:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-20T18:27:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:45.163 INFO:teuthology.orchestra.run.smithi005.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-20T18:27:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 5e:e7:2d:8b:38:50 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-20T18:27:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-20T18:27:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::5ce7:2dff:fe8b:3850/64 scope link tentative 2024-05-20T18:27:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:45.165 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-05-20T18:27:45.165 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:27:45.165 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-20T18:27:45.165 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-05-20T18:27:45.165 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-20T18:27:45.165 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-05-20T18:27:45.165 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-05-20T18:27:45.165 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-05-20T18:27:45.165 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-20T18:27:45.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:27:45.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:45.323 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:27:45.323 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-20T18:27:45.323 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.1 up 2024-05-20T18:27:45.323 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set dev brx.1 master ceph-brx 2024-05-20T18:27:45.323 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-20T18:27:45.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:27:45.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:45.405 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:27:45.405 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-05-20T18:27:45.421 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-05-20T18:27:45.422 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:27:45.422 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-05-20T18:27:45.478 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-20T18:27:45.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:45.478 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-20T18:27:45.575 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-20T18:27:45.575 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-05-20T18:27:45.576 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:27:45.586 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-20T18:27:45.586 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-05-20T18:27:45.586 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:27:45.586 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:27:45.587 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:27:45.593 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-20T18:27:45.593 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-05-20T18:27:45.593 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:27:45.593 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:27:45.593 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:27:45.593 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790" 2024-05-20T18:27:45.594 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790:/ 2024-05-20T18:27:45.594 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=399ab250-fd3c-47fa-a1c8-25e8149c9cac" will pass to kernel 2024-05-20T18:27:45.594 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-05-20T18:27:45.594 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-05-20T18:27:45.594 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-20T18:27:45.594 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-05-20T18:27:45.594 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-20T18:27:45.595 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-20T18:27:45.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:45.596 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-05-20T18:27:45.622 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-20T18:27:45.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T18:27:45.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:45.720 INFO:teuthology.orchestra.run.smithi005.stdout:client4813 2024-05-20T18:27:45.721 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:27:45.721 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:27:45.737 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-20T18:27:45.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T18:27:45.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:45.835 INFO:teuthology.orchestra.run.smithi005.stdout:client4813 2024-05-20T18:27:45.836 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-20T18:27:45.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T18:27:45.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:45.894 INFO:teuthology.orchestra.run.smithi005.stdout:client4813 2024-05-20T18:27:45.894 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/399ab250-fd3c-47fa-a1c8-25e8149c9cac.client4813/status') 2024-05-20T18:27:45.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:27:45.955 INFO:teuthology.orchestra.run.smithi005.stdout:instance: client.4813 (3)172.21.15.5:0/3035861559 2024-05-20T18:27:45.955 INFO:teuthology.orchestra.run.smithi005.stdout:blocklisted: false 2024-05-20T18:27:45.955 INFO:teuthology.orchestra.run.smithi005.stderr:0+1 records in 2024-05-20T18:27:45.956 INFO:teuthology.orchestra.run.smithi005.stderr:0+1 records out 2024-05-20T18:27:45.956 INFO:teuthology.orchestra.run.smithi005.stderr:69 bytes copied, 2.686e-05 s, 2.6 MB/s 2024-05-20T18:27:45.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:45.963 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-05-20T18:27:45.963 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-20T18:27:45.963 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-20T18:27:45.963 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-20T18:27:46.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.079+0000 7fb37ea3d640 1 Processor -- start 2024-05-20T18:27:46.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.079+0000 7fb37ea3d640 1 -- start start 2024-05-20T18:27:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.080+0000 7fb37ea3d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb378162820 0x7fb3780b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.080+0000 7fb37ea3d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb3780b3e80 0x7fb37806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.080+0000 7fb37ea3d640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb37806dd20 con 0x7fb37806d910 2024-05-20T18:27:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.080+0000 7fb37ea3d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fb3780d0630 con 0x7fb378162820 2024-05-20T18:27:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.080+0000 7fb37ea3d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb3780d39c0 con 0x7fb3780b3e80 2024-05-20T18:27:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.080+0000 7fb377fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb378162820 0x7fb3780b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.080+0000 7fb377fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb378162820 0x7fb3780b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35490/0 (socket says 172.21.15.5:35490) 2024-05-20T18:27:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.080+0000 7fb377fff640 1 -- 172.21.15.5:0/1721991472 learned_addr learned my addr 172.21.15.5:0/1721991472 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.080+0000 7fb3777fe640 1 --2- 172.21.15.5:0/1721991472 >> v2:172.21.15.63:3300/0 conn(0x7fb3780b3e80 0x7fb37806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.080+0000 7fb377fff640 1 -- 172.21.15.5:0/1721991472 >> v2:172.21.15.63:3300/0 conn(0x7fb3780b3e80 msgr2=0x7fb37806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.080+0000 7fb377fff640 1 --2- 172.21.15.5:0/1721991472 >> v2:172.21.15.63:3300/0 conn(0x7fb3780b3e80 0x7fb37806d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.080+0000 7fb377fff640 1 -- 172.21.15.5:0/1721991472 >> v1:172.21.15.63:6789/0 conn(0x7fb37806d910 legacy=0x7fb3780d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.081+0000 7fb377fff640 1 -- 172.21.15.5:0/1721991472 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3780d3ca0 con 0x7fb378162820 2024-05-20T18:27:46.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.081+0000 7fb377fff640 1 --2- 172.21.15.5:0/1721991472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378162820 0x7fb3780b3940 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb3780d1230 tx=0x7fb36802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf101417f4f0e81c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:46.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.081+0000 7fb376ffd640 1 -- 172.21.15.5:0/1721991472 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb368037070 con 0x7fb378162820 2024-05-20T18:27:46.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.081+0000 7fb376ffd640 1 -- 172.21.15.5:0/1721991472 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb368032440 con 0x7fb378162820 2024-05-20T18:27:46.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.081+0000 7fb376ffd640 1 -- 172.21.15.5:0/1721991472 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb368032810 con 0x7fb378162820 2024-05-20T18:27:46.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.082+0000 7fb37ea3d640 1 -- 172.21.15.5:0/1721991472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378162820 msgr2=0x7fb3780b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:46.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.082+0000 7fb37ea3d640 1 --2- 172.21.15.5:0/1721991472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378162820 0x7fb3780b3940 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb3780d1230 tx=0x7fb36802a270 comp rx=0 tx=0).stop 2024-05-20T18:27:46.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.082+0000 7fb37ea3d640 1 -- 172.21.15.5:0/1721991472 shutdown_connections 2024-05-20T18:27:46.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.082+0000 7fb37ea3d640 1 --2- 172.21.15.5:0/1721991472 >> v2:172.21.15.63:3300/0 conn(0x7fb3780b3e80 0x7fb37806d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:46.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.082+0000 7fb37ea3d640 1 --2- 172.21.15.5:0/1721991472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378162820 0x7fb3780b3940 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:46.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.082+0000 7fb37ea3d640 1 -- 172.21.15.5:0/1721991472 >> 172.21.15.5:0/1721991472 conn(0x7fb3780d4bd0 msgr2=0x7fb3780d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:46.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.082+0000 7fb37ea3d640 1 -- 172.21.15.5:0/1721991472 shutdown_connections 2024-05-20T18:27:46.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.082+0000 7fb37ea3d640 1 -- 172.21.15.5:0/1721991472 wait complete. 2024-05-20T18:27:46.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.084+0000 7fb37ea3d640 1 Processor -- start 2024-05-20T18:27:46.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.084+0000 7fb37ea3d640 1 -- start start 2024-05-20T18:27:46.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.085+0000 7fb37ea3d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb37806d910 0x7fb378151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:46.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.085+0000 7fb37ea3d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb3780b3e80 0x7fb3781f85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:46.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.086+0000 7fb37ea3d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378162820 0x7fb37815fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:46.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.086+0000 7fb37ea3d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb3780d0300 con 0x7fb378162820 2024-05-20T18:27:46.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.086+0000 7fb37ea3d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb3780d4260 con 0x7fb37806d910 2024-05-20T18:27:46.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.086+0000 7fb37ea3d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb3780b91a0 con 0x7fb3780b3e80 2024-05-20T18:27:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.086+0000 7fb37cfda640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378162820 0x7fb37815fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.086+0000 7fb37cfda640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378162820 0x7fb37815fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35504/0 (socket says 172.21.15.5:35504) 2024-05-20T18:27:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.086+0000 7fb37cfda640 1 -- 172.21.15.5:0/3090626317 learned_addr learned my addr 172.21.15.5:0/3090626317 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.086+0000 7fb377fff640 1 --2- 172.21.15.5:0/3090626317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb37806d910 0x7fb378151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.086+0000 7fb3777fe640 1 --2- 172.21.15.5:0/3090626317 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb3780b3e80 0x7fb3781f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.086+0000 7fb37cfda640 1 -- 172.21.15.5:0/3090626317 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb3780b3e80 msgr2=0x7fb3781f85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.086+0000 7fb37cfda640 1 --2- 172.21.15.5:0/3090626317 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb3780b3e80 0x7fb3781f85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.086+0000 7fb37cfda640 1 -- 172.21.15.5:0/3090626317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb37806d910 msgr2=0x7fb378151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.086+0000 7fb37cfda640 1 --2- 172.21.15.5:0/3090626317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb37806d910 0x7fb378151130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.086+0000 7fb37cfda640 1 -- 172.21.15.5:0/3090626317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3781f9290 con 0x7fb378162820 2024-05-20T18:27:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.086+0000 7fb37cfda640 1 --2- 172.21.15.5:0/3090626317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378162820 0x7fb37815fcc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb37000c6e0 tx=0x7fb37000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.086+0000 7fb374ff9640 1 -- 172.21.15.5:0/3090626317 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370003b70 con 0x7fb378162820 2024-05-20T18:27:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.086+0000 7fb374ff9640 1 -- 172.21.15.5:0/3090626317 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb370003d30 con 0x7fb378162820 2024-05-20T18:27:46.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.086+0000 7fb374ff9640 1 -- 172.21.15.5:0/3090626317 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370005310 con 0x7fb378162820 2024-05-20T18:27:46.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.086+0000 7fb37ea3d640 1 -- 172.21.15.5:0/3090626317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3780cbed0 con 0x7fb378162820 2024-05-20T18:27:46.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.087+0000 7fb37ea3d640 1 -- 172.21.15.5:0/3090626317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3780cda30 con 0x7fb378162820 2024-05-20T18:27:46.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.087+0000 7fb37ea3d640 1 -- 172.21.15.5:0/3090626317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3781f9000 con 0x7fb378162820 2024-05-20T18:27:46.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.088+0000 7fb374ff9640 1 -- 172.21.15.5:0/3090626317 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb3700054b0 con 0x7fb378162820 2024-05-20T18:27:46.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.088+0000 7fb374ff9640 1 --2- 172.21.15.5:0/3090626317 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb36406be60 0x7fb36406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:46.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.089+0000 7fb374ff9640 1 -- 172.21.15.5:0/3090626317 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fb37008afe0 con 0x7fb378162820 2024-05-20T18:27:46.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.089+0000 7fb377fff640 1 --2- 172.21.15.5:0/3090626317 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb36406be60 0x7fb36406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:46.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.089+0000 7fb377fff640 1 --2- 172.21.15.5:0/3090626317 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb36406be60 0x7fb36406e370 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb368034000 tx=0x7fb36802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:46.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.093+0000 7fb374ff9640 1 -- 172.21.15.5:0/3090626317 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb37806dd20 con 0x7fb378162820 2024-05-20T18:27:46.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.317+0000 7fb37ea3d640 1 -- 172.21.15.5:0/3090626317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb378075b00 con 0x7fb378162820 2024-05-20T18:27:46.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.318+0000 7fb374ff9640 1 -- 172.21.15.5:0/3090626317 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fb3700595c0 con 0x7fb378162820 2024-05-20T18:27:46.317 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-20T18:27:46.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.320+0000 7fb37ea3d640 1 -- 172.21.15.5:0/3090626317 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb36406be60 msgr2=0x7fb36406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:46.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.320+0000 7fb37ea3d640 1 --2- 172.21.15.5:0/3090626317 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb36406be60 0x7fb36406e370 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb368034000 tx=0x7fb36802e000 comp rx=0 tx=0).stop 2024-05-20T18:27:46.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.320+0000 7fb37ea3d640 1 -- 172.21.15.5:0/3090626317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378162820 msgr2=0x7fb37815fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:46.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.320+0000 7fb37ea3d640 1 --2- 172.21.15.5:0/3090626317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378162820 0x7fb37815fcc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb37000c6e0 tx=0x7fb37000cba0 comp rx=0 tx=0).stop 2024-05-20T18:27:46.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.320+0000 7fb37ea3d640 1 -- 172.21.15.5:0/3090626317 shutdown_connections 2024-05-20T18:27:46.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.320+0000 7fb37ea3d640 1 --2- 172.21.15.5:0/3090626317 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb36406be60 0x7fb36406e370 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:46.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.320+0000 7fb37ea3d640 1 --2- 172.21.15.5:0/3090626317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378162820 0x7fb37815fcc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:46.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.320+0000 7fb37ea3d640 1 --2- 172.21.15.5:0/3090626317 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb3780b3e80 0x7fb3781f85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:46.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.320+0000 7fb37ea3d640 1 --2- 172.21.15.5:0/3090626317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb37806d910 0x7fb378151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:46.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.320+0000 7fb37ea3d640 1 -- 172.21.15.5:0/3090626317 >> 172.21.15.5:0/3090626317 conn(0x7fb3780d4bd0 msgr2=0x7fb3780d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:46.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.320+0000 7fb37ea3d640 1 -- 172.21.15.5:0/3090626317 shutdown_connections 2024-05-20T18:27:46.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:46.320+0000 7fb37ea3d640 1 -- 172.21.15.5:0/3090626317 wait complete. 2024-05-20T18:27:46.331 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-20T18:27:46.331 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-20T18:27:46.331 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi063.front.sepia.ceph.com 2024-05-20T18:27:46.331 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-20T18:27:46.331 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-20T18:27:46.331 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-20T18:27:46.331 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-20T18:27:46.331 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-20T18:27:46.332 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-20T18:27:46.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:46.332 DEBUG:teuthology.orchestra.run.smithi063:> ip addr 2024-05-20T18:27:46.352 INFO:teuthology.orchestra.run.smithi063.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T18:27:46.352 INFO:teuthology.orchestra.run.smithi063.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T18:27:46.352 INFO:teuthology.orchestra.run.smithi063.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T18:27:46.352 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:46.353 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 ::1/128 scope host 2024-05-20T18:27:46.353 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:46.353 INFO:teuthology.orchestra.run.smithi063.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:27:46.353 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:a8:b4:76 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:27:46.353 INFO:teuthology.orchestra.run.smithi063.stdout: altname enp7s0f0 2024-05-20T18:27:46.353 INFO:teuthology.orchestra.run.smithi063.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:27:46.353 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:a8:b4:77 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:27:46.353 INFO:teuthology.orchestra.run.smithi063.stdout: altname enp7s0f1 2024-05-20T18:27:46.353 INFO:teuthology.orchestra.run.smithi063.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:27:46.353 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:b6:f8:d6 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:27:46.353 INFO:teuthology.orchestra.run.smithi063.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-20T18:27:46.353 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:b6:f8:d7 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:27:46.353 INFO:teuthology.orchestra.run.smithi063.stdout: inet 172.21.15.63/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-20T18:27:46.354 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft 42651sec preferred_lft 42651sec 2024-05-20T18:27:46.354 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 fe80::ec4:7aff:feb6:f8d7/64 scope link 2024-05-20T18:27:46.354 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:46.354 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-20T18:27:46.354 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:27:46.354 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-20T18:27:46.354 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link add name ceph-brx type bridge 2024-05-20T18:27:46.354 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip addr flush dev ceph-brx 2024-05-20T18:27:46.354 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set ceph-brx up 2024-05-20T18:27:46.354 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-20T18:27:46.355 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-20T18:27:46.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:27:46.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:46.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:46.579 DEBUG:teuthology.orchestra.run.smithi063:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-20T18:27:46.605 INFO:teuthology.orchestra.run.smithi063.stdout:1 2024-05-20T18:27:46.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:46.607 DEBUG:teuthology.orchestra.run.smithi063:> route 2024-05-20T18:27:46.681 INFO:teuthology.orchestra.run.smithi063.stdout:Kernel IP routing table 2024-05-20T18:27:46.681 INFO:teuthology.orchestra.run.smithi063.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-20T18:27:46.681 INFO:teuthology.orchestra.run.smithi063.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-20T18:27:46.681 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-20T18:27:46.681 INFO:teuthology.orchestra.run.smithi063.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-20T18:27:46.682 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:27:46.682 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-20T18:27:46.682 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-20T18:27:46.682 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-20T18:27:46.682 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-20T18:27:46.682 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-20T18:27:46.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:27:46.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:46.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:46.955 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-05-20T18:27:46.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:46.971 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list-id 2024-05-20T18:27:47.027 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:27:47.027 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-20T18:27:47.027 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-20T18:27:47.027 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-20T18:27:47.027 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-20T18:27:47.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:27:47.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:47.147 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-20T18:27:47.148 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:27:47.148 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-20T18:27:47.148 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-20T18:27:47.148 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-20T18:27:47.148 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-20T18:27:47.148 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-20T18:27:47.148 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-20T18:27:47.148 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-20T18:27:47.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:27:47.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:47.280 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:27:47.280 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-20T18:27:47.280 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set brx.0 up 2024-05-20T18:27:47.280 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set dev brx.0 master ceph-brx 2024-05-20T18:27:47.280 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-20T18:27:47.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:27:47.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:47.360 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:27:47.360 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-20T18:27:47.376 INFO:teuthology.orchestra.run.smithi063.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-20T18:27:47.376 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:27:47.376 DEBUG:teuthology.orchestra.run.smithi063:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-20T18:27:47.432 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-20T18:27:47.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:47.432 DEBUG:teuthology.orchestra.run.smithi063:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-20T18:27:47.536 INFO:teuthology.orchestra.run.smithi063.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-20T18:27:47.536 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-20T18:27:47.536 INFO:teuthology.orchestra.run.smithi063.stdout:invalid new device string format 2024-05-20T18:27:47.659 INFO:teuthology.orchestra.run.smithi063.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-20T18:27:47.659 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-20T18:27:47.660 INFO:teuthology.orchestra.run.smithi063.stdout:invalid new device string format 2024-05-20T18:27:47.660 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:27:47.660 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:27:47.665 INFO:teuthology.orchestra.run.smithi063.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-20T18:27:47.665 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-20T18:27:47.665 INFO:teuthology.orchestra.run.smithi063.stdout:invalid new device string format 2024-05-20T18:27:47.665 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:27:47.666 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:27:47.666 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790" 2024-05-20T18:27:47.666 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790:/ 2024-05-20T18:27:47.666 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=399ab250-fd3c-47fa-a1c8-25e8149c9cac" will pass to kernel 2024-05-20T18:27:47.666 INFO:teuthology.orchestra.run.smithi063.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-05-20T18:27:47.667 INFO:teuthology.orchestra.run.smithi063.stdout: You just mounted a file system that supports labels which does not 2024-05-20T18:27:47.667 INFO:teuthology.orchestra.run.smithi063.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-20T18:27:47.667 INFO:teuthology.orchestra.run.smithi063.stdout: applications will generate AVC messages and not be allowed access to 2024-05-20T18:27:47.667 INFO:teuthology.orchestra.run.smithi063.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-20T18:27:47.669 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-20T18:27:47.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:47.669 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-20T18:27:47.695 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-20T18:27:47.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T18:27:47.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:47.794 INFO:teuthology.orchestra.run.smithi063.stdout:client4825 2024-05-20T18:27:47.794 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:27:47.794 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:27:47.812 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-20T18:27:47.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T18:27:47.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:47.910 INFO:teuthology.orchestra.run.smithi063.stdout:client4825 2024-05-20T18:27:47.910 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-20T18:27:47.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T18:27:47.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:47.968 INFO:teuthology.orchestra.run.smithi063.stdout:client4825 2024-05-20T18:27:47.968 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/399ab250-fd3c-47fa-a1c8-25e8149c9cac.client4825/status') 2024-05-20T18:27:48.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:27:48.029 INFO:teuthology.orchestra.run.smithi063.stdout:instance: client.4825 (3)192.168.0.1:0/1759895445 2024-05-20T18:27:48.029 INFO:teuthology.orchestra.run.smithi063.stdout:blocklisted: false 2024-05-20T18:27:48.029 INFO:teuthology.orchestra.run.smithi063.stderr:0+1 records in 2024-05-20T18:27:48.029 INFO:teuthology.orchestra.run.smithi063.stderr:0+1 records out 2024-05-20T18:27:48.029 INFO:teuthology.orchestra.run.smithi063.stderr:69 bytes copied, 2.4527e-05 s, 2.8 MB/s 2024-05-20T18:27:48.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:48.037 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-05-20T18:27:48.037 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-20T18:27:48.037 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-20T18:27:48.037 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-20T18:27:48.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.167+0000 7f1df7b57640 1 Processor -- start 2024-05-20T18:27:48.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.167+0000 7f1df7b57640 1 -- start start 2024-05-20T18:27:48.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.169+0000 7f1df7b57640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1df00b6620 0x7f1df00ba3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:48.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.169+0000 7f1df7b57640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1df0162ce0 0x7f1df00b8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:48.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.169+0000 7f1df7b57640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1df00b85b0 0x7f1df00d3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:48.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.169+0000 7f1df7b57640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f1df00d3480 con 0x7f1df0162ce0 2024-05-20T18:27:48.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.169+0000 7f1df7b57640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1df01634a0 con 0x7f1df00b6620 2024-05-20T18:27:48.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.169+0000 7f1df7b57640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1df0163780 con 0x7f1df00b85b0 2024-05-20T18:27:48.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.169+0000 7f1df50f2640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1df0162ce0 0x7f1df00b8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:48.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.169+0000 7f1df50f2640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1df0162ce0 0x7f1df00b8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35512/0 (socket says 172.21.15.5:35512) 2024-05-20T18:27:48.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.169+0000 7f1df50f2640 1 -- 172.21.15.5:0/1259344873 learned_addr learned my addr 172.21.15.5:0/1259344873 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:48.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.169+0000 7f1df58f3640 1 --2- 172.21.15.5:0/1259344873 >> v2:172.21.15.63:3300/0 conn(0x7f1df00b6620 0x7f1df00ba3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:48.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.169+0000 7f1df60f4640 1 --2- 172.21.15.5:0/1259344873 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1df00b85b0 0x7f1df00d3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:48.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.169+0000 7f1df50f2640 1 -- 172.21.15.5:0/1259344873 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1df00b85b0 msgr2=0x7f1df00d3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:48.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.169+0000 7f1df50f2640 1 --2- 172.21.15.5:0/1259344873 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1df00b85b0 0x7f1df00d3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:48.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.169+0000 7f1df50f2640 1 -- 172.21.15.5:0/1259344873 >> v2:172.21.15.63:3300/0 conn(0x7f1df00b6620 msgr2=0x7f1df00ba3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:48.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.169+0000 7f1df50f2640 1 --2- 172.21.15.5:0/1259344873 >> v2:172.21.15.63:3300/0 conn(0x7f1df00b6620 0x7f1df00ba3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:48.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.169+0000 7f1df50f2640 1 -- 172.21.15.5:0/1259344873 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df0163a60 con 0x7f1df0162ce0 2024-05-20T18:27:48.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.170+0000 7f1df50f2640 1 --2- 172.21.15.5:0/1259344873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1df0162ce0 0x7f1df00b8070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1df008db90 tx=0x7f1de002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a639277a4bbb8ef5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:48.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.170+0000 7f1df48f1640 1 -- 172.21.15.5:0/1259344873 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de0037070 con 0x7f1df0162ce0 2024-05-20T18:27:48.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.170+0000 7f1df48f1640 1 -- 172.21.15.5:0/1259344873 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1de0032440 con 0x7f1df0162ce0 2024-05-20T18:27:48.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.170+0000 7f1df48f1640 1 -- 172.21.15.5:0/1259344873 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de0032810 con 0x7f1df0162ce0 2024-05-20T18:27:48.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.171+0000 7f1df7b57640 1 -- 172.21.15.5:0/1259344873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1df0162ce0 msgr2=0x7f1df00b8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.171+0000 7f1df7b57640 1 --2- 172.21.15.5:0/1259344873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1df0162ce0 0x7f1df00b8070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1df008db90 tx=0x7f1de002a1e0 comp rx=0 tx=0).stop 2024-05-20T18:27:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.171+0000 7f1df7b57640 1 -- 172.21.15.5:0/1259344873 shutdown_connections 2024-05-20T18:27:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.171+0000 7f1df7b57640 1 --2- 172.21.15.5:0/1259344873 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1df00b85b0 0x7f1df00d3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.171+0000 7f1df7b57640 1 --2- 172.21.15.5:0/1259344873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1df0162ce0 0x7f1df00b8070 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.171+0000 7f1df7b57640 1 --2- 172.21.15.5:0/1259344873 >> v2:172.21.15.63:3300/0 conn(0x7f1df00b6620 0x7f1df00ba3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.171+0000 7f1df7b57640 1 -- 172.21.15.5:0/1259344873 >> 172.21.15.5:0/1259344873 conn(0x7f1df006d240 msgr2=0x7f1df006d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.171+0000 7f1df7b57640 1 -- 172.21.15.5:0/1259344873 shutdown_connections 2024-05-20T18:27:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.171+0000 7f1df7b57640 1 -- 172.21.15.5:0/1259344873 wait complete. 2024-05-20T18:27:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.173+0000 7f1df7b57640 1 Processor -- start 2024-05-20T18:27:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.173+0000 7f1df7b57640 1 -- start start 2024-05-20T18:27:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.174+0000 7f1df7b57640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1df00b6620 0x7f1df00ce310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.174+0000 7f1df7b57640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1df00b85b0 0x7f1df014c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.174+0000 7f1df7b57640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1df0162ce0 0x7f1df014c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.174+0000 7f1df7b57640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1df01630f0 con 0x7f1df0162ce0 2024-05-20T18:27:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.174+0000 7f1df7b57640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1df00d4130 con 0x7f1df00b85b0 2024-05-20T18:27:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.174+0000 7f1df7b57640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1df00b28e0 con 0x7f1df00b6620 2024-05-20T18:27:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.175+0000 7f1df60f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1df0162ce0 0x7f1df014c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.175+0000 7f1df60f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1df0162ce0 0x7f1df014c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35524/0 (socket says 172.21.15.5:35524) 2024-05-20T18:27:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.175+0000 7f1df60f4640 1 -- 172.21.15.5:0/4212879689 learned_addr learned my addr 172.21.15.5:0/4212879689 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.175+0000 7f1df50f2640 1 --2- 172.21.15.5:0/4212879689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1df00b85b0 0x7f1df014c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.175+0000 7f1df58f3640 1 --2- 172.21.15.5:0/4212879689 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1df00b6620 0x7f1df00ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.175+0000 7f1df60f4640 1 -- 172.21.15.5:0/4212879689 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1df00b6620 msgr2=0x7f1df00ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.175+0000 7f1df60f4640 1 --2- 172.21.15.5:0/4212879689 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1df00b6620 0x7f1df00ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:48.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.175+0000 7f1df60f4640 1 -- 172.21.15.5:0/4212879689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1df00b85b0 msgr2=0x7f1df014c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:48.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.175+0000 7f1df60f4640 1 --2- 172.21.15.5:0/4212879689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1df00b85b0 0x7f1df014c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:48.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.175+0000 7f1df60f4640 1 -- 172.21.15.5:0/4212879689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df014d110 con 0x7f1df0162ce0 2024-05-20T18:27:48.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.175+0000 7f1df50f2640 1 --2- 172.21.15.5:0/4212879689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1df00b85b0 0x7f1df014c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:27:48.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.175+0000 7f1df60f4640 1 --2- 172.21.15.5:0/4212879689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1df0162ce0 0x7f1df014c940 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1de800ef90 tx=0x7f1de800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:48.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.175+0000 7f1de67fc640 1 -- 172.21.15.5:0/4212879689 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de8018020 con 0x7f1df0162ce0 2024-05-20T18:27:48.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.175+0000 7f1de67fc640 1 -- 172.21.15.5:0/4212879689 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1de8014480 con 0x7f1df0162ce0 2024-05-20T18:27:48.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.175+0000 7f1df7b57640 1 -- 172.21.15.5:0/4212879689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1df014d3f0 con 0x7f1df0162ce0 2024-05-20T18:27:48.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.175+0000 7f1de67fc640 1 -- 172.21.15.5:0/4212879689 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de8009e60 con 0x7f1df0162ce0 2024-05-20T18:27:48.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.175+0000 7f1df7b57640 1 -- 172.21.15.5:0/4212879689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1df00b0da0 con 0x7f1df0162ce0 2024-05-20T18:27:48.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.177+0000 7f1df7b57640 1 -- 172.21.15.5:0/4212879689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1df0075b00 con 0x7f1df0162ce0 2024-05-20T18:27:48.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.177+0000 7f1de67fc640 1 -- 172.21.15.5:0/4212879689 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1de8005d10 con 0x7f1df0162ce0 2024-05-20T18:27:48.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.177+0000 7f1de67fc640 1 --2- 172.21.15.5:0/4212879689 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1dd406be60 0x7f1dd406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:48.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.178+0000 7f1df58f3640 1 --2- 172.21.15.5:0/4212879689 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1dd406be60 0x7f1dd406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.178+0000 7f1de67fc640 1 -- 172.21.15.5:0/4212879689 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f1de808bdb0 con 0x7f1df0162ce0 2024-05-20T18:27:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.178+0000 7f1df58f3640 1 --2- 172.21.15.5:0/4212879689 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1dd406be60 0x7f1dd406e370 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009a20 tx=0x7f1dec009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.182+0000 7f1de67fc640 1 -- 172.21.15.5:0/4212879689 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1de8090070 con 0x7f1df0162ce0 2024-05-20T18:27:48.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.404+0000 7f1df7b57640 1 -- 172.21.15.5:0/4212879689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1df014cea0 con 0x7f1df0162ce0 2024-05-20T18:27:48.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.404+0000 7f1de67fc640 1 -- 172.21.15.5:0/4212879689 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f1de8022030 con 0x7f1df0162ce0 2024-05-20T18:27:48.404 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-20T18:27:48.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.406+0000 7f1df7b57640 1 -- 172.21.15.5:0/4212879689 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1dd406be60 msgr2=0x7f1dd406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:48.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.406+0000 7f1df7b57640 1 --2- 172.21.15.5:0/4212879689 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1dd406be60 0x7f1dd406e370 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009a20 tx=0x7f1dec009190 comp rx=0 tx=0).stop 2024-05-20T18:27:48.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.406+0000 7f1df7b57640 1 -- 172.21.15.5:0/4212879689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1df0162ce0 msgr2=0x7f1df014c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:48.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.406+0000 7f1df7b57640 1 --2- 172.21.15.5:0/4212879689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1df0162ce0 0x7f1df014c940 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1de800ef90 tx=0x7f1de800c490 comp rx=0 tx=0).stop 2024-05-20T18:27:48.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.406+0000 7f1df7b57640 1 -- 172.21.15.5:0/4212879689 shutdown_connections 2024-05-20T18:27:48.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.406+0000 7f1df7b57640 1 --2- 172.21.15.5:0/4212879689 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1dd406be60 0x7f1dd406e370 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:48.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.406+0000 7f1df7b57640 1 --2- 172.21.15.5:0/4212879689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1df0162ce0 0x7f1df014c940 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:48.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.406+0000 7f1df7b57640 1 --2- 172.21.15.5:0/4212879689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1df00b85b0 0x7f1df014c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:48.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.406+0000 7f1df7b57640 1 --2- 172.21.15.5:0/4212879689 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1df00b6620 0x7f1df00ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:48.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.407+0000 7f1df7b57640 1 -- 172.21.15.5:0/4212879689 >> 172.21.15.5:0/4212879689 conn(0x7f1df006d240 msgr2=0x7f1df0160f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:48.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.407+0000 7f1df7b57640 1 -- 172.21.15.5:0/4212879689 shutdown_connections 2024-05-20T18:27:48.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:48.407+0000 7f1df7b57640 1 -- 172.21.15.5:0/4212879689 wait complete. 2024-05-20T18:27:48.417 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-20T18:27:48.417 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-20T18:27:48.417 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi063.front.sepia.ceph.com 2024-05-20T18:27:48.418 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-05-20T18:27:48.418 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-20T18:27:48.418 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-20T18:27:48.418 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-20T18:27:48.418 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-20T18:27:48.418 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-05-20T18:27:48.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:48.418 DEBUG:teuthology.orchestra.run.smithi063:> ip addr 2024-05-20T18:27:48.435 INFO:teuthology.orchestra.run.smithi063.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T18:27:48.436 INFO:teuthology.orchestra.run.smithi063.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T18:27:48.436 INFO:teuthology.orchestra.run.smithi063.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T18:27:48.436 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:48.436 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 ::1/128 scope host 2024-05-20T18:27:48.436 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:48.436 INFO:teuthology.orchestra.run.smithi063.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:27:48.436 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:a8:b4:76 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:27:48.436 INFO:teuthology.orchestra.run.smithi063.stdout: altname enp7s0f0 2024-05-20T18:27:48.436 INFO:teuthology.orchestra.run.smithi063.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:27:48.436 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:a8:b4:77 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:27:48.436 INFO:teuthology.orchestra.run.smithi063.stdout: altname enp7s0f1 2024-05-20T18:27:48.436 INFO:teuthology.orchestra.run.smithi063.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:27:48.437 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:b6:f8:d6 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:27:48.437 INFO:teuthology.orchestra.run.smithi063.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-20T18:27:48.437 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:b6:f8:d7 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:27:48.437 INFO:teuthology.orchestra.run.smithi063.stdout: inet 172.21.15.63/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-20T18:27:48.437 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft 42648sec preferred_lft 42648sec 2024-05-20T18:27:48.437 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 fe80::ec4:7aff:feb6:f8d7/64 scope link 2024-05-20T18:27:48.437 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:48.437 INFO:teuthology.orchestra.run.smithi063.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-20T18:27:48.437 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 6e:3f:39:97:52:70 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:27:48.437 INFO:teuthology.orchestra.run.smithi063.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-20T18:27:48.437 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:48.437 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 fe80::103e:5ff:fedf:41f6/64 scope link 2024-05-20T18:27:48.437 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:48.437 INFO:teuthology.orchestra.run.smithi063.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:27:48.438 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 6e:3f:39:97:52:70 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-20T18:27:48.438 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 fe80::6c3f:39ff:fe97:5270/64 scope link tentative 2024-05-20T18:27:48.438 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:48.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:48.438 DEBUG:teuthology.orchestra.run.smithi063:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-20T18:27:48.501 INFO:teuthology.orchestra.run.smithi063.stdout:1 2024-05-20T18:27:48.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:48.503 DEBUG:teuthology.orchestra.run.smithi063:> route 2024-05-20T18:27:48.560 INFO:teuthology.orchestra.run.smithi063.stdout:Kernel IP routing table 2024-05-20T18:27:48.560 INFO:teuthology.orchestra.run.smithi063.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-20T18:27:48.561 INFO:teuthology.orchestra.run.smithi063.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-20T18:27:48.561 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-20T18:27:48.561 INFO:teuthology.orchestra.run.smithi063.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-20T18:27:48.561 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:27:48.561 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-20T18:27:48.561 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-20T18:27:48.561 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-20T18:27:48.561 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-20T18:27:48.562 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-20T18:27:48.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:27:48.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:48.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:48.690 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-05-20T18:27:48.706 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-20T18:27:48.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:48.707 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list-id 2024-05-20T18:27:48.763 INFO:teuthology.orchestra.run.smithi063.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-05-20T18:27:48.763 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:27:48.763 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-20T18:27:48.764 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-20T18:27:48.764 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-05-20T18:27:48.764 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-20T18:27:48.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:27:48.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:48.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:48.883 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-20T18:27:48.911 INFO:teuthology.orchestra.run.smithi063.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T18:27:48.912 INFO:teuthology.orchestra.run.smithi063.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T18:27:48.912 INFO:teuthology.orchestra.run.smithi063.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T18:27:48.912 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:48.912 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 ::1/128 scope host 2024-05-20T18:27:48.912 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:48.912 INFO:teuthology.orchestra.run.smithi063.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-20T18:27:48.913 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether d2:a9:72:c5:70:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-20T18:27:48.913 INFO:teuthology.orchestra.run.smithi063.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-20T18:27:48.913 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:48.913 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 fe80::d0a9:72ff:fec5:705a/64 scope link 2024-05-20T18:27:48.913 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:48.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:48.914 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-20T18:27:48.978 INFO:teuthology.orchestra.run.smithi063.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T18:27:48.978 INFO:teuthology.orchestra.run.smithi063.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T18:27:48.978 INFO:teuthology.orchestra.run.smithi063.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T18:27:48.978 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:48.978 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 ::1/128 scope host 2024-05-20T18:27:48.978 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:48.978 INFO:teuthology.orchestra.run.smithi063.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-20T18:27:48.979 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether d2:a9:72:c5:70:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-20T18:27:48.979 INFO:teuthology.orchestra.run.smithi063.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-20T18:27:48.979 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:48.979 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 fe80::d0a9:72ff:fec5:705a/64 scope link 2024-05-20T18:27:48.979 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:27:48.980 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-05-20T18:27:48.980 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:27:48.980 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-20T18:27:48.980 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-05-20T18:27:48.980 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-20T18:27:48.980 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-05-20T18:27:48.980 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-05-20T18:27:48.981 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-05-20T18:27:48.981 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-20T18:27:49.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:27:49.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:49.138 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:27:49.138 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-20T18:27:49.138 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set brx.1 up 2024-05-20T18:27:49.138 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set dev brx.1 master ceph-brx 2024-05-20T18:27:49.138 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-20T18:27:49.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:27:49.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:49.219 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:27:49.219 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-05-20T18:27:49.235 INFO:teuthology.orchestra.run.smithi063.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-05-20T18:27:49.235 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:27:49.236 DEBUG:teuthology.orchestra.run.smithi063:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-05-20T18:27:49.291 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-20T18:27:49.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:49.291 DEBUG:teuthology.orchestra.run.smithi063:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-20T18:27:49.386 INFO:teuthology.orchestra.run.smithi063.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-20T18:27:49.387 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-05-20T18:27:49.387 INFO:teuthology.orchestra.run.smithi063.stdout:invalid new device string format 2024-05-20T18:27:49.397 INFO:teuthology.orchestra.run.smithi063.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-20T18:27:49.397 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-05-20T18:27:49.398 INFO:teuthology.orchestra.run.smithi063.stdout:invalid new device string format 2024-05-20T18:27:49.398 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:27:49.398 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:27:49.403 INFO:teuthology.orchestra.run.smithi063.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-20T18:27:49.404 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-05-20T18:27:49.404 INFO:teuthology.orchestra.run.smithi063.stdout:invalid new device string format 2024-05-20T18:27:49.404 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:27:49.404 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:27:49.404 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790" 2024-05-20T18:27:49.404 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790:/ 2024-05-20T18:27:49.404 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=399ab250-fd3c-47fa-a1c8-25e8149c9cac" will pass to kernel 2024-05-20T18:27:49.405 INFO:teuthology.orchestra.run.smithi063.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-05-20T18:27:49.405 INFO:teuthology.orchestra.run.smithi063.stdout: You just mounted a file system that supports labels which does not 2024-05-20T18:27:49.405 INFO:teuthology.orchestra.run.smithi063.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-20T18:27:49.405 INFO:teuthology.orchestra.run.smithi063.stdout: applications will generate AVC messages and not be allowed access to 2024-05-20T18:27:49.405 INFO:teuthology.orchestra.run.smithi063.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-20T18:27:49.406 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-20T18:27:49.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:27:49.406 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-05-20T18:27:49.432 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-20T18:27:49.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T18:27:49.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:49.531 INFO:teuthology.orchestra.run.smithi063.stdout:client4834 2024-05-20T18:27:49.531 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:27:49.531 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:27:49.548 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-20T18:27:49.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T18:27:49.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:49.645 INFO:teuthology.orchestra.run.smithi063.stdout:client4834 2024-05-20T18:27:49.646 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-20T18:27:49.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T18:27:49.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:49.704 INFO:teuthology.orchestra.run.smithi063.stdout:client4834 2024-05-20T18:27:49.704 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/399ab250-fd3c-47fa-a1c8-25e8149c9cac.client4834/status') 2024-05-20T18:27:49.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:27:49.766 INFO:teuthology.orchestra.run.smithi063.stdout:instance: client.4834 (3)192.168.0.2:0/139557474 2024-05-20T18:27:49.766 INFO:teuthology.orchestra.run.smithi063.stdout:blocklisted: false 2024-05-20T18:27:49.766 INFO:teuthology.orchestra.run.smithi063.stderr:0+1 records in 2024-05-20T18:27:49.766 INFO:teuthology.orchestra.run.smithi063.stderr:0+1 records out 2024-05-20T18:27:49.766 INFO:teuthology.orchestra.run.smithi063.stderr:68 bytes copied, 2.5472e-05 s, 2.7 MB/s 2024-05-20T18:27:49.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-20T18:27:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:27:49.774 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-05-20T18:27:49.792 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:27:49.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.923+0000 7ff49e0f0640 1 Processor -- start 2024-05-20T18:27:49.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.923+0000 7ff49e0f0640 1 -- start start 2024-05-20T18:27:49.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.924+0000 7ff49e0f0640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff4981576d0 0x7ff49806c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:49.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.924+0000 7ff49e0f0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff4980cedd0 0x7ff49806e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:49.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.924+0000 7ff49e0f0640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff498157ae0 con 0x7ff49815d5f0 2024-05-20T18:27:49.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.924+0000 7ff49e0f0640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff49806cee0 con 0x7ff4981576d0 2024-05-20T18:27:49.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.924+0000 7ff49e0f0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff49806b900 con 0x7ff4980cedd0 2024-05-20T18:27:49.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.924+0000 7ff4977fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff49815d5f0 0x7ff4980ce6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59218/0 (socket says 172.21.15.5:59218) 2024-05-20T18:27:49.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.924+0000 7ff4977fe640 1 -- 172.21.15.5:0/3399479777 learned_addr learned my addr 172.21.15.5:0/3399479777 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:49.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.924+0000 7ff497fff640 1 --2- 172.21.15.5:0/3399479777 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff4980cedd0 0x7ff49806e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:49.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.924+0000 7ff496ffd640 1 --2- 172.21.15.5:0/3399479777 >> v2:172.21.15.63:3300/0 conn(0x7ff4981576d0 0x7ff49806c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:49.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.925+0000 7ff4967fc640 1 -- 172.21.15.5:0/3399479777 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2089278747 0 0) 0x7ff498157ae0 con 0x7ff49815d5f0 2024-05-20T18:27:49.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.925+0000 7ff4967fc640 1 -- 172.21.15.5:0/3399479777 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff480003520 con 0x7ff49815d5f0 2024-05-20T18:27:49.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.925+0000 7ff496ffd640 1 -- 172.21.15.5:0/3399479777 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff4980cedd0 msgr2=0x7ff49806e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:49.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.925+0000 7ff496ffd640 1 --2- 172.21.15.5:0/3399479777 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff4980cedd0 0x7ff49806e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:49.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.925+0000 7ff496ffd640 1 -- 172.21.15.5:0/3399479777 >> v1:172.21.15.5:6789/0 conn(0x7ff49815d5f0 legacy=0x7ff4980ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:49.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.925+0000 7ff496ffd640 1 -- 172.21.15.5:0/3399479777 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49806bbe0 con 0x7ff4981576d0 2024-05-20T18:27:49.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.925+0000 7ff497fff640 1 --2- 172.21.15.5:0/3399479777 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff4980cedd0 0x7ff49806e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:27:49.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.926+0000 7ff496ffd640 1 --2- 172.21.15.5:0/3399479777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4981576d0 0x7ff49806c9a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0097a0 tx=0x7ff48c02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afeb2f3eb32b6534 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:49.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.926+0000 7ff4967fc640 1 -- 172.21.15.5:0/3399479777 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c037070 con 0x7ff4981576d0 2024-05-20T18:27:49.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.926+0000 7ff4967fc640 1 -- 172.21.15.5:0/3399479777 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff48c02ae50 con 0x7ff4981576d0 2024-05-20T18:27:49.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.926+0000 7ff4967fc640 1 -- 172.21.15.5:0/3399479777 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c032660 con 0x7ff4981576d0 2024-05-20T18:27:49.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.926+0000 7ff49e0f0640 1 -- 172.21.15.5:0/3399479777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4981576d0 msgr2=0x7ff49806c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:49.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.926+0000 7ff49e0f0640 1 --2- 172.21.15.5:0/3399479777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4981576d0 0x7ff49806c9a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0097a0 tx=0x7ff48c02a100 comp rx=0 tx=0).stop 2024-05-20T18:27:49.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.926+0000 7ff49e0f0640 1 -- 172.21.15.5:0/3399479777 shutdown_connections 2024-05-20T18:27:49.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.926+0000 7ff49e0f0640 1 --2- 172.21.15.5:0/3399479777 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff4980cedd0 0x7ff49806e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:49.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.926+0000 7ff49e0f0640 1 --2- 172.21.15.5:0/3399479777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4981576d0 0x7ff49806c9a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:49.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.927+0000 7ff49e0f0640 1 -- 172.21.15.5:0/3399479777 >> 172.21.15.5:0/3399479777 conn(0x7ff4980cd2a0 msgr2=0x7ff4980cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:49.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.927+0000 7ff49e0f0640 1 -- 172.21.15.5:0/3399479777 shutdown_connections 2024-05-20T18:27:49.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.927+0000 7ff49e0f0640 1 -- 172.21.15.5:0/3399479777 wait complete. 2024-05-20T18:27:49.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.928+0000 7ff49e0f0640 1 Processor -- start 2024-05-20T18:27:49.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.929+0000 7ff49e0f0640 1 -- start start 2024-05-20T18:27:49.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.930+0000 7ff49e0f0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4980cedd0 0x7ff49813d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:49.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.930+0000 7ff49e0f0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff4981576d0 0x7ff49814fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:49.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.930+0000 7ff49e0f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49815d5f0 0x7ff49814db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:49.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.930+0000 7ff49e0f0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff49806c480 con 0x7ff49815d5f0 2024-05-20T18:27:49.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.930+0000 7ff49e0f0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff49806c1a0 con 0x7ff4980cedd0 2024-05-20T18:27:49.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.930+0000 7ff49e0f0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff4980cf1e0 con 0x7ff4981576d0 2024-05-20T18:27:49.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.930+0000 7ff497fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49815d5f0 0x7ff49814db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:49.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.930+0000 7ff497fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49815d5f0 0x7ff49814db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35532/0 (socket says 172.21.15.5:35532) 2024-05-20T18:27:49.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.930+0000 7ff497fff640 1 -- 172.21.15.5:0/2423050553 learned_addr learned my addr 172.21.15.5:0/2423050553 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:49.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.930+0000 7ff496ffd640 1 --2- 172.21.15.5:0/2423050553 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff4981576d0 0x7ff49814fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:49.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.930+0000 7ff4977fe640 1 --2- 172.21.15.5:0/2423050553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4980cedd0 0x7ff49813d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:49.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.931+0000 7ff497fff640 1 -- 172.21.15.5:0/2423050553 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff4981576d0 msgr2=0x7ff49814fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:49.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.931+0000 7ff497fff640 1 --2- 172.21.15.5:0/2423050553 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff4981576d0 0x7ff49814fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:49.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.931+0000 7ff497fff640 1 -- 172.21.15.5:0/2423050553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4980cedd0 msgr2=0x7ff49813d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:49.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.931+0000 7ff497fff640 1 --2- 172.21.15.5:0/2423050553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4980cedd0 0x7ff49813d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:49.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.931+0000 7ff497fff640 1 -- 172.21.15.5:0/2423050553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49813d9e0 con 0x7ff49815d5f0 2024-05-20T18:27:49.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.931+0000 7ff496ffd640 1 --2- 172.21.15.5:0/2423050553 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff4981576d0 0x7ff49814fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:27:49.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.931+0000 7ff4977fe640 1 --2- 172.21.15.5:0/2423050553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4980cedd0 0x7ff49813d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:27:49.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.931+0000 7ff497fff640 1 --2- 172.21.15.5:0/2423050553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49815d5f0 0x7ff49814db70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff490009e10 tx=0x7ff49000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:49.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.931+0000 7ff47ffff640 1 -- 172.21.15.5:0/2423050553 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff490018020 con 0x7ff49815d5f0 2024-05-20T18:27:49.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.931+0000 7ff47ffff640 1 -- 172.21.15.5:0/2423050553 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff490014480 con 0x7ff49815d5f0 2024-05-20T18:27:49.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.931+0000 7ff49e0f0640 1 -- 172.21.15.5:0/2423050553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff498150a60 con 0x7ff49815d5f0 2024-05-20T18:27:49.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.931+0000 7ff47ffff640 1 -- 172.21.15.5:0/2423050553 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff490014850 con 0x7ff49815d5f0 2024-05-20T18:27:49.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.931+0000 7ff49e0f0640 1 -- 172.21.15.5:0/2423050553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4980ada40 con 0x7ff49815d5f0 2024-05-20T18:27:49.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.932+0000 7ff49e0f0640 1 -- 172.21.15.5:0/2423050553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4981507d0 con 0x7ff49815d5f0 2024-05-20T18:27:49.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.933+0000 7ff47ffff640 1 -- 172.21.15.5:0/2423050553 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff490005d10 con 0x7ff49815d5f0 2024-05-20T18:27:49.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.933+0000 7ff47ffff640 1 --2- 172.21.15.5:0/2423050553 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff48406be60 0x7ff48406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:49.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.933+0000 7ff47ffff640 1 -- 172.21.15.5:0/2423050553 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff49008bd90 con 0x7ff49815d5f0 2024-05-20T18:27:49.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.933+0000 7ff4977fe640 1 --2- 172.21.15.5:0/2423050553 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff48406be60 0x7ff48406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:49.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.934+0000 7ff4977fe640 1 --2- 172.21.15.5:0/2423050553 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff48406be60 0x7ff48406e370 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff488005880 tx=0x7ff488009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:49.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:49.938+0000 7ff47ffff640 1 -- 172.21.15.5:0/2423050553 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff490022030 con 0x7ff49815d5f0 2024-05-20T18:27:50.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.122+0000 7ff49e0f0640 1 -- 172.21.15.5:0/2423050553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff49813dbb0 con 0x7ff49815d5f0 2024-05-20T18:27:50.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.123+0000 7ff47ffff640 1 -- 172.21.15.5:0/2423050553 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18394 (secure 0 0 0) 0x7ff490010040 con 0x7ff49815d5f0 2024-05-20T18:27:50.122 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:50.123 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:27:13.817633+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:27:05.476596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T18:27:06.137762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T18:27:09.829920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:27:50.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.125+0000 7ff49e0f0640 1 -- 172.21.15.5:0/2423050553 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff48406be60 msgr2=0x7ff48406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.125+0000 7ff49e0f0640 1 --2- 172.21.15.5:0/2423050553 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff48406be60 0x7ff48406e370 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff488005880 tx=0x7ff488009040 comp rx=0 tx=0).stop 2024-05-20T18:27:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.125+0000 7ff49e0f0640 1 -- 172.21.15.5:0/2423050553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49815d5f0 msgr2=0x7ff49814db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.125+0000 7ff49e0f0640 1 --2- 172.21.15.5:0/2423050553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49815d5f0 0x7ff49814db70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff490009e10 tx=0x7ff49000c490 comp rx=0 tx=0).stop 2024-05-20T18:27:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.125+0000 7ff49e0f0640 1 -- 172.21.15.5:0/2423050553 shutdown_connections 2024-05-20T18:27:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.125+0000 7ff49e0f0640 1 --2- 172.21.15.5:0/2423050553 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff48406be60 0x7ff48406e370 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.125+0000 7ff49e0f0640 1 --2- 172.21.15.5:0/2423050553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49815d5f0 0x7ff49814db70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.125+0000 7ff49e0f0640 1 --2- 172.21.15.5:0/2423050553 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff4981576d0 0x7ff49814fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.125+0000 7ff49e0f0640 1 --2- 172.21.15.5:0/2423050553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4980cedd0 0x7ff49813d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.125+0000 7ff49e0f0640 1 -- 172.21.15.5:0/2423050553 >> 172.21.15.5:0/2423050553 conn(0x7ff4980cd2a0 msgr2=0x7ff4980b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.125+0000 7ff49e0f0640 1 -- 172.21.15.5:0/2423050553 shutdown_connections 2024-05-20T18:27:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.125+0000 7ff49e0f0640 1 -- 172.21.15.5:0/2423050553 wait complete. 2024-05-20T18:27:50.136 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:27:05.476596+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:27:06.137762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T18:27:09.829920+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T18:27:50.137 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:27:50.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.256+0000 7f92497aa640 1 Processor -- start 2024-05-20T18:27:50.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.256+0000 7f92497aa640 1 -- start start 2024-05-20T18:27:50.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.257+0000 7f92497aa640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f92440d1f40 0x7f9244161760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:50.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.257+0000 7f92497aa640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9244160b90 con 0x7f9244161350 2024-05-20T18:27:50.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.257+0000 7f92497aa640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f924415de60 con 0x7f924406b520 2024-05-20T18:27:50.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.257+0000 7f92497aa640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f924415e140 con 0x7f92440d1f40 2024-05-20T18:27:50.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.257+0000 7f92427fc640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f9244161350 0x7f92440d1830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59226/0 (socket says 172.21.15.5:59226) 2024-05-20T18:27:50.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.257+0000 7f92427fc640 1 -- 172.21.15.5:0/71776643 learned_addr learned my addr 172.21.15.5:0/71776643 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:50.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.257+0000 7f92437fe640 1 --2- 172.21.15.5:0/71776643 >> v2:172.21.15.63:3300/0 conn(0x7f92440d1f40 0x7f9244161760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:50.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.257+0000 7f9241ffb640 1 -- 172.21.15.5:0/71776643 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3657346237 0 0) 0x7f9244160b90 con 0x7f9244161350 2024-05-20T18:27:50.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.257+0000 7f9241ffb640 1 -- 172.21.15.5:0/71776643 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9230003520 con 0x7f9244161350 2024-05-20T18:27:50.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.258+0000 7f9241ffb640 1 -- 172.21.15.5:0/71776643 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 594194667 0 0) 0x7f9230003520 con 0x7f9244161350 2024-05-20T18:27:50.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.258+0000 7f9241ffb640 1 -- 172.21.15.5:0/71776643 >> v2:172.21.15.63:3300/0 conn(0x7f92440d1f40 msgr2=0x7f9244161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:50.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.258+0000 7f9241ffb640 1 --2- 172.21.15.5:0/71776643 >> v2:172.21.15.63:3300/0 conn(0x7f92440d1f40 0x7f9244161760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:50.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.258+0000 7f9241ffb640 1 -- 172.21.15.5:0/71776643 >> v1:172.21.15.63:6789/0 conn(0x7f924406b520 legacy=0x7f924415d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:50.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.258+0000 7f9241ffb640 1 -- 172.21.15.5:0/71776643 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924415e420 con 0x7f9244161350 2024-05-20T18:27:50.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.258+0000 7f9241ffb640 1 -- 172.21.15.5:0/71776643 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f922c0019f0 con 0x7f9244161350 2024-05-20T18:27:50.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.258+0000 7f9241ffb640 1 -- 172.21.15.5:0/71776643 >> v1:172.21.15.5:6789/0 conn(0x7f9244161350 legacy=0x7f92440d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:50.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.258+0000 7f9241ffb640 1 --2- 172.21.15.5:0/71776643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92300039b0 0x7f9230003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:50.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.258+0000 7f9241ffb640 1 --2- 172.21.15.5:0/71776643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9230004800 0x7f9230004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:50.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.258+0000 7f9241ffb640 1 --2- 172.21.15.5:0/71776643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f92300051c0 0x7f923000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:50.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.258+0000 7f9241ffb640 1 -- 172.21.15.5:0/71776643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f924415e140 con 0x7f9230004800 2024-05-20T18:27:50.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.258+0000 7f9241ffb640 1 -- 172.21.15.5:0/71776643 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f923000c160 con 0x7f92300039b0 2024-05-20T18:27:50.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.258+0000 7f9241ffb640 1 -- 172.21.15.5:0/71776643 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f923000c360 con 0x7f92300051c0 2024-05-20T18:27:50.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.258+0000 7f92427fc640 1 --2- 172.21.15.5:0/71776643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9230004800 0x7f9230004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:50.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.258+0000 7f9242ffd640 1 --2- 172.21.15.5:0/71776643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92300039b0 0x7f9230003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:50.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.258+0000 7f92437fe640 1 --2- 172.21.15.5:0/71776643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f92300051c0 0x7f923000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:50.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.258+0000 7f9242ffd640 1 -- 172.21.15.5:0/71776643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f92300051c0 msgr2=0x7f923000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:50.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.258+0000 7f9242ffd640 1 --2- 172.21.15.5:0/71776643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f92300051c0 0x7f923000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:50.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.258+0000 7f9242ffd640 1 -- 172.21.15.5:0/71776643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9230004800 msgr2=0x7f9230004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:50.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.258+0000 7f9242ffd640 1 --2- 172.21.15.5:0/71776643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9230004800 0x7f9230004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:50.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.258+0000 7f9242ffd640 1 -- 172.21.15.5:0/71776643 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9230003520 con 0x7f92300039b0 2024-05-20T18:27:50.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.258+0000 7f92427fc640 1 --2- 172.21.15.5:0/71776643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9230004800 0x7f9230004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:27:50.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.258+0000 7f92437fe640 1 -- 172.21.15.5:0/71776643 reap_dead start 2024-05-20T18:27:50.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.259+0000 7f9242ffd640 1 --2- 172.21.15.5:0/71776643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92300039b0 0x7f9230003e20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f924406e130 tx=0x7f9230005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f42b2294a72dc904 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:50.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.259+0000 7f9241ffb640 1 -- 172.21.15.5:0/71776643 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9234001370 con 0x7f92300039b0 2024-05-20T18:27:50.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.259+0000 7f9241ffb640 1 -- 172.21.15.5:0/71776643 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92340052d0 con 0x7f92300039b0 2024-05-20T18:27:50.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.259+0000 7f9241ffb640 1 -- 172.21.15.5:0/71776643 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923402aac0 con 0x7f92300039b0 2024-05-20T18:27:50.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.260+0000 7f92497aa640 1 -- 172.21.15.5:0/71776643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92300039b0 msgr2=0x7f9230003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:50.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.260+0000 7f92497aa640 1 --2- 172.21.15.5:0/71776643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92300039b0 0x7f9230003e20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f924406e130 tx=0x7f9230005830 comp rx=0 tx=0).stop 2024-05-20T18:27:50.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.260+0000 7f92497aa640 1 -- 172.21.15.5:0/71776643 shutdown_connections 2024-05-20T18:27:50.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.260+0000 7f92497aa640 1 --2- 172.21.15.5:0/71776643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92300039b0 0x7f9230003e20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:50.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.260+0000 7f92497aa640 1 -- 172.21.15.5:0/71776643 >> 172.21.15.5:0/71776643 conn(0x7f92440b6670 msgr2=0x7f92440b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:50.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.260+0000 7f92497aa640 1 -- 172.21.15.5:0/71776643 shutdown_connections 2024-05-20T18:27:50.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.260+0000 7f92497aa640 1 -- 172.21.15.5:0/71776643 wait complete. 2024-05-20T18:27:50.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.262+0000 7f92497aa640 1 Processor -- start 2024-05-20T18:27:50.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.262+0000 7f92497aa640 1 -- start start 2024-05-20T18:27:50.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.263+0000 7f92497aa640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f92300039b0 0x7f92441f67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:50.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.263+0000 7f92497aa640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f924413f1a0 0x7f92441581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:50.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.263+0000 7f92497aa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9244153ba0 0x7f92441f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:50.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.263+0000 7f92497aa640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9244161760 con 0x7f9244153ba0 2024-05-20T18:27:50.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.263+0000 7f92497aa640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9244143740 con 0x7f924413f1a0 2024-05-20T18:27:50.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.263+0000 7f92497aa640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f92441618e0 con 0x7f92300039b0 2024-05-20T18:27:50.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.263+0000 7f92437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9244153ba0 0x7f92441f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:50.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.263+0000 7f92437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9244153ba0 0x7f92441f38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35558/0 (socket says 172.21.15.5:35558) 2024-05-20T18:27:50.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.264+0000 7f92437fe640 1 -- 172.21.15.5:0/1893261025 learned_addr learned my addr 172.21.15.5:0/1893261025 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:50.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.264+0000 7f92427fc640 1 --2- 172.21.15.5:0/1893261025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f924413f1a0 0x7f92441581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:50.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.264+0000 7f9242ffd640 1 --2- 172.21.15.5:0/1893261025 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f92300039b0 0x7f92441f67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:50.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.264+0000 7f92437fe640 1 -- 172.21.15.5:0/1893261025 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f92300039b0 msgr2=0x7f92441f67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:50.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.264+0000 7f92437fe640 1 --2- 172.21.15.5:0/1893261025 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f92300039b0 0x7f92441f67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:50.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.264+0000 7f92437fe640 1 -- 172.21.15.5:0/1893261025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f924413f1a0 msgr2=0x7f92441581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:50.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.264+0000 7f92437fe640 1 --2- 172.21.15.5:0/1893261025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f924413f1a0 0x7f92441581e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:50.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.264+0000 7f92437fe640 1 -- 172.21.15.5:0/1893261025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92441f6e70 con 0x7f9244153ba0 2024-05-20T18:27:50.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.264+0000 7f92437fe640 1 --2- 172.21.15.5:0/1893261025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9244153ba0 0x7f92441f38f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f924406bed0 tx=0x7f923c011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:50.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.264+0000 7f923b7fe640 1 -- 172.21.15.5:0/1893261025 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923c004240 con 0x7f9244153ba0 2024-05-20T18:27:50.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.264+0000 7f923b7fe640 1 -- 172.21.15.5:0/1893261025 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f923c01c070 con 0x7f9244153ba0 2024-05-20T18:27:50.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.264+0000 7f92497aa640 1 -- 172.21.15.5:0/1893261025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92441f7150 con 0x7f9244153ba0 2024-05-20T18:27:50.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.264+0000 7f923b7fe640 1 -- 172.21.15.5:0/1893261025 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923c00f710 con 0x7f9244153ba0 2024-05-20T18:27:50.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.264+0000 7f92497aa640 1 -- 172.21.15.5:0/1893261025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92441f33d0 con 0x7f9244153ba0 2024-05-20T18:27:50.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.265+0000 7f92497aa640 1 -- 172.21.15.5:0/1893261025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92441f7670 con 0x7f9244153ba0 2024-05-20T18:27:50.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.266+0000 7f923b7fe640 1 -- 172.21.15.5:0/1893261025 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f923c012040 con 0x7f9244153ba0 2024-05-20T18:27:50.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.267+0000 7f923b7fe640 1 --2- 172.21.15.5:0/1893261025 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f922406be30 0x7f922406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:50.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.267+0000 7f9242ffd640 1 --2- 172.21.15.5:0/1893261025 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f922406be30 0x7f922406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:50.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.267+0000 7f923b7fe640 1 -- 172.21.15.5:0/1893261025 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f923c08b110 con 0x7f9244153ba0 2024-05-20T18:27:50.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.267+0000 7f9242ffd640 1 --2- 172.21.15.5:0/1893261025 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f922406be30 0x7f922406e340 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9234032000 tx=0x7f923402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:50.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.271+0000 7f923b7fe640 1 -- 172.21.15.5:0/1893261025 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f92441f7670 con 0x7f9244153ba0 2024-05-20T18:27:50.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.456+0000 7f92497aa640 1 -- 172.21.15.5:0/1893261025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f92441f31c0 con 0x7f9244153ba0 2024-05-20T18:27:50.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.457+0000 7f923b7fe640 1 -- 172.21.15.5:0/1893261025 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f923c059650 con 0x7f9244153ba0 2024-05-20T18:27:50.456 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:27:50.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.459+0000 7f92497aa640 1 -- 172.21.15.5:0/1893261025 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f922406be30 msgr2=0x7f922406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:50.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.459+0000 7f92497aa640 1 --2- 172.21.15.5:0/1893261025 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f922406be30 0x7f922406e340 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9234032000 tx=0x7f923402e000 comp rx=0 tx=0).stop 2024-05-20T18:27:50.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.459+0000 7f92497aa640 1 -- 172.21.15.5:0/1893261025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9244153ba0 msgr2=0x7f92441f38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.459+0000 7f92497aa640 1 --2- 172.21.15.5:0/1893261025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9244153ba0 0x7f92441f38f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f924406bed0 tx=0x7f923c011cf0 comp rx=0 tx=0).stop 2024-05-20T18:27:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.459+0000 7f92497aa640 1 -- 172.21.15.5:0/1893261025 shutdown_connections 2024-05-20T18:27:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.459+0000 7f92497aa640 1 --2- 172.21.15.5:0/1893261025 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f922406be30 0x7f922406e340 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.459+0000 7f92497aa640 1 --2- 172.21.15.5:0/1893261025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9244153ba0 0x7f92441f38f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.459+0000 7f92497aa640 1 --2- 172.21.15.5:0/1893261025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f924413f1a0 0x7f92441581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.459+0000 7f92497aa640 1 --2- 172.21.15.5:0/1893261025 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f92300039b0 0x7f92441f67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.459+0000 7f92497aa640 1 -- 172.21.15.5:0/1893261025 >> 172.21.15.5:0/1893261025 conn(0x7f92440b6670 msgr2=0x7f924415cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.459+0000 7f92497aa640 1 -- 172.21.15.5:0/1893261025 shutdown_connections 2024-05-20T18:27:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.459+0000 7f92497aa640 1 -- 172.21.15.5:0/1893261025 wait complete. 2024-05-20T18:27:50.470 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:27:50.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.589+0000 7f08a7757640 1 Processor -- start 2024-05-20T18:27:50.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.590+0000 7f08a7757640 1 -- start start 2024-05-20T18:27:50.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.591+0000 7f08a7757640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f08a00b4010 0x7f08a00d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:50.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.591+0000 7f08a7757640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f08a00d14b0 0x7f08a00d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:50.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.591+0000 7f08a7757640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08a00d0370 con 0x7f08a0163a10 2024-05-20T18:27:50.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.591+0000 7f08a7757640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f08a00d3ca0 con 0x7f08a00b4010 2024-05-20T18:27:50.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.591+0000 7f08a7757640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f08a00d0650 con 0x7f08a00d14b0 2024-05-20T18:27:50.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.591+0000 7f08a4cf2640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f08a00b4010 0x7f08a00d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:50.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.591+0000 7f08a4cf2640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f08a00b4010 0x7f08a00d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35566/0 (socket says 172.21.15.5:35566) 2024-05-20T18:27:50.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.591+0000 7f08a4cf2640 1 -- 172.21.15.5:0/1790632846 learned_addr learned my addr 172.21.15.5:0/1790632846 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:50.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.591+0000 7f08a5cf4640 1 --2- 172.21.15.5:0/1790632846 >> v2:172.21.15.63:3300/0 conn(0x7f08a00d14b0 0x7f08a00d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:50.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.591+0000 7f08a4cf2640 1 -- 172.21.15.5:0/1790632846 >> v2:172.21.15.63:3300/0 conn(0x7f08a00d14b0 msgr2=0x7f08a00d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:50.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.591+0000 7f08a4cf2640 1 --2- 172.21.15.5:0/1790632846 >> v2:172.21.15.63:3300/0 conn(0x7f08a00d14b0 0x7f08a00d18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:50.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.591+0000 7f08a4cf2640 1 -- 172.21.15.5:0/1790632846 >> v1:172.21.15.63:6789/0 conn(0x7f08a0163a10 legacy=0x7f08a00cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:50.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.592+0000 7f08a4cf2640 1 -- 172.21.15.5:0/1790632846 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a00d3650 con 0x7f08a00b4010 2024-05-20T18:27:50.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.592+0000 7f08a4cf2640 1 --2- 172.21.15.5:0/1790632846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08a00b4010 0x7f08a00d4130 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f08a00b4590 tx=0x7f089002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c4ca58486e2017 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:50.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.592+0000 7f0897fff640 1 -- 172.21.15.5:0/1790632846 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0890037070 con 0x7f08a00b4010 2024-05-20T18:27:50.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.592+0000 7f0897fff640 1 -- 172.21.15.5:0/1790632846 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0890032440 con 0x7f08a00b4010 2024-05-20T18:27:50.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.592+0000 7f0897fff640 1 -- 172.21.15.5:0/1790632846 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0890032810 con 0x7f08a00b4010 2024-05-20T18:27:50.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.593+0000 7f08a7757640 1 -- 172.21.15.5:0/1790632846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08a00b4010 msgr2=0x7f08a00d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:50.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.593+0000 7f08a7757640 1 --2- 172.21.15.5:0/1790632846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08a00b4010 0x7f08a00d4130 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f08a00b4590 tx=0x7f089002a270 comp rx=0 tx=0).stop 2024-05-20T18:27:50.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.593+0000 7f08a7757640 1 -- 172.21.15.5:0/1790632846 shutdown_connections 2024-05-20T18:27:50.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.593+0000 7f08a7757640 1 --2- 172.21.15.5:0/1790632846 >> v2:172.21.15.63:3300/0 conn(0x7f08a00d14b0 0x7f08a00d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:50.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.593+0000 7f08a7757640 1 --2- 172.21.15.5:0/1790632846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08a00b4010 0x7f08a00d4130 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:50.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.593+0000 7f08a7757640 1 -- 172.21.15.5:0/1790632846 >> 172.21.15.5:0/1790632846 conn(0x7f08a006d240 msgr2=0x7f08a006d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:50.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.593+0000 7f08a7757640 1 -- 172.21.15.5:0/1790632846 shutdown_connections 2024-05-20T18:27:50.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.593+0000 7f08a7757640 1 -- 172.21.15.5:0/1790632846 wait complete. 2024-05-20T18:27:50.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.595+0000 7f08a7757640 1 Processor -- start 2024-05-20T18:27:50.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.595+0000 7f08a7757640 1 -- start start 2024-05-20T18:27:50.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.596+0000 7f08a7757640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08a00b4010 0x7f08a01f5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:50.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.596+0000 7f08a7757640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08a00d14b0 0x7f08a01f57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:50.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.596+0000 7f08a7757640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08a0163a10 0x7f08a0158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:50.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.596+0000 7f08a7757640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08a00b8210 con 0x7f08a00d14b0 2024-05-20T18:27:50.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.596+0000 7f08a7757640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f08a00d1e00 con 0x7f08a00b4010 2024-05-20T18:27:50.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.596+0000 7f08a7757640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f08a00b8390 con 0x7f08a0163a10 2024-05-20T18:27:50.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.596+0000 7f08a4cf2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08a00d14b0 0x7f08a01f57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:50.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.596+0000 7f08a4cf2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08a00d14b0 0x7f08a01f57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35580/0 (socket says 172.21.15.5:35580) 2024-05-20T18:27:50.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.597+0000 7f08a4cf2640 1 -- 172.21.15.5:0/2592833 learned_addr learned my addr 172.21.15.5:0/2592833 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:50.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.597+0000 7f08a54f3640 1 --2- 172.21.15.5:0/2592833 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08a00b4010 0x7f08a01f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:50.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.597+0000 7f08a5cf4640 1 --2- 172.21.15.5:0/2592833 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08a0163a10 0x7f08a0158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:50.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.597+0000 7f08a4cf2640 1 -- 172.21.15.5:0/2592833 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08a0163a10 msgr2=0x7f08a0158360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:50.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.597+0000 7f08a4cf2640 1 --2- 172.21.15.5:0/2592833 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08a0163a10 0x7f08a0158360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:50.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.597+0000 7f08a4cf2640 1 -- 172.21.15.5:0/2592833 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08a00b4010 msgr2=0x7f08a01f5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:50.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.597+0000 7f08a4cf2640 1 --2- 172.21.15.5:0/2592833 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08a00b4010 0x7f08a01f5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:50.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.597+0000 7f08a4cf2640 1 -- 172.21.15.5:0/2592833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a015c0c0 con 0x7f08a00d14b0 2024-05-20T18:27:50.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.597+0000 7f08a54f3640 1 --2- 172.21.15.5:0/2592833 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08a00b4010 0x7f08a01f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:50.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.597+0000 7f08a4cf2640 1 --2- 172.21.15.5:0/2592833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08a00d14b0 0x7f08a01f57a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f089002a2b0 tx=0x7f08900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:50.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.597+0000 7f0895ffb640 1 -- 172.21.15.5:0/2592833 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0890037040 con 0x7f08a00d14b0 2024-05-20T18:27:50.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.597+0000 7f0895ffb640 1 -- 172.21.15.5:0/2592833 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0890042070 con 0x7f08a00d14b0 2024-05-20T18:27:50.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.597+0000 7f0895ffb640 1 -- 172.21.15.5:0/2592833 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0890030880 con 0x7f08a00d14b0 2024-05-20T18:27:50.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.597+0000 7f08a7757640 1 -- 172.21.15.5:0/2592833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08a015c3a0 con 0x7f08a00d14b0 2024-05-20T18:27:50.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.598+0000 7f08a7757640 1 -- 172.21.15.5:0/2592833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08a015c960 con 0x7f08a00d14b0 2024-05-20T18:27:50.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.598+0000 7f08a7757640 1 -- 172.21.15.5:0/2592833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08a015be30 con 0x7f08a00d14b0 2024-05-20T18:27:50.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.599+0000 7f0895ffb640 1 -- 172.21.15.5:0/2592833 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0890030a20 con 0x7f08a00d14b0 2024-05-20T18:27:50.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.600+0000 7f0895ffb640 1 --2- 172.21.15.5:0/2592833 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f088406be30 0x7f088406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:50.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.600+0000 7f0895ffb640 1 -- 172.21.15.5:0/2592833 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f08900abad0 con 0x7f08a00d14b0 2024-05-20T18:27:50.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.600+0000 7f08a54f3640 1 --2- 172.21.15.5:0/2592833 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f088406be30 0x7f088406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:50.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.600+0000 7f08a54f3640 1 --2- 172.21.15.5:0/2592833 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f088406be30 0x7f088406e340 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f08a00d3020 tx=0x7f089c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:50.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.604+0000 7f0895ffb640 1 -- 172.21.15.5:0/2592833 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f089007a0b0 con 0x7f08a00d14b0 2024-05-20T18:27:50.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.787+0000 7f08a7757640 1 -- 172.21.15.5:0/2592833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f08a0075b00 con 0x7f08a00d14b0 2024-05-20T18:27:50.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.787+0000 7f0895ffb640 1 -- 172.21.15.5:0/2592833 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f08900799b0 con 0x7f08a00d14b0 2024-05-20T18:27:50.787 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:27:50.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.789+0000 7f08a7757640 1 -- 172.21.15.5:0/2592833 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f088406be30 msgr2=0x7f088406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:50.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.789+0000 7f08a7757640 1 --2- 172.21.15.5:0/2592833 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f088406be30 0x7f088406e340 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f08a00d3020 tx=0x7f089c009040 comp rx=0 tx=0).stop 2024-05-20T18:27:50.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.789+0000 7f08a7757640 1 -- 172.21.15.5:0/2592833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08a00d14b0 msgr2=0x7f08a01f57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:50.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.789+0000 7f08a7757640 1 --2- 172.21.15.5:0/2592833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08a00d14b0 0x7f08a01f57a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f089002a2b0 tx=0x7f08900046c0 comp rx=0 tx=0).stop 2024-05-20T18:27:50.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.789+0000 7f08a7757640 1 -- 172.21.15.5:0/2592833 shutdown_connections 2024-05-20T18:27:50.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.789+0000 7f08a7757640 1 --2- 172.21.15.5:0/2592833 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f088406be30 0x7f088406e340 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:50.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.789+0000 7f08a7757640 1 --2- 172.21.15.5:0/2592833 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08a0163a10 0x7f08a0158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:50.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.789+0000 7f08a7757640 1 --2- 172.21.15.5:0/2592833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08a00d14b0 0x7f08a01f57a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:50.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.789+0000 7f08a7757640 1 --2- 172.21.15.5:0/2592833 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08a00b4010 0x7f08a01f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:50.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.789+0000 7f08a7757640 1 -- 172.21.15.5:0/2592833 >> 172.21.15.5:0/2592833 conn(0x7f08a006d240 msgr2=0x7f08a015f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:50.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.789+0000 7f08a7757640 1 -- 172.21.15.5:0/2592833 shutdown_connections 2024-05-20T18:27:50.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.789+0000 7f08a7757640 1 -- 172.21.15.5:0/2592833 wait complete. 2024-05-20T18:27:50.800 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:27:50.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.918+0000 7fa47497e640 1 Processor -- start 2024-05-20T18:27:50.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.918+0000 7fa47497e640 1 -- start start 2024-05-20T18:27:50.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.920+0000 7fa47497e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa47006d3d0 0x7fa47006d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:50.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.920+0000 7fa47497e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa47006dd20 con 0x7fa470162820 2024-05-20T18:27:50.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.920+0000 7fa47497e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4700d0630 con 0x7fa4700b8190 2024-05-20T18:27:50.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.920+0000 7fa47497e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fa4700d39c0 con 0x7fa47006d3d0 2024-05-20T18:27:50.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.920+0000 7fa46e59c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa470162820 0x7fa4700b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59236/0 (socket says 172.21.15.5:59236) 2024-05-20T18:27:50.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.920+0000 7fa46e59c640 1 -- 172.21.15.5:0/3376820402 learned_addr learned my addr 172.21.15.5:0/3376820402 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:50.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.920+0000 7fa46dd9b640 1 --2- 172.21.15.5:0/3376820402 >> v2:172.21.15.5:3300/0 conn(0x7fa47006d3d0 0x7fa47006d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:50.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.920+0000 7fa46dd9b640 1 -- 172.21.15.5:0/3376820402 >> v1:172.21.15.63:6789/0 conn(0x7fa4700b8190 legacy=0x7fa4700b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:50.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.920+0000 7fa46dd9b640 1 -- 172.21.15.5:0/3376820402 >> v1:172.21.15.5:6789/0 conn(0x7fa470162820 legacy=0x7fa4700b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:50.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.920+0000 7fa46dd9b640 1 -- 172.21.15.5:0/3376820402 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4700d3ca0 con 0x7fa47006d3d0 2024-05-20T18:27:50.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.921+0000 7fa46dd9b640 1 --2- 172.21.15.5:0/3376820402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa47006d3d0 0x7fa47006d7e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa4580097a0 tx=0x7fa45802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7669e50e6afef51 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:50.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.921+0000 7fa46d59a640 1 -- 172.21.15.5:0/3376820402 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa458037070 con 0x7fa47006d3d0 2024-05-20T18:27:50.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.921+0000 7fa46d59a640 1 -- 172.21.15.5:0/3376820402 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa458032440 con 0x7fa47006d3d0 2024-05-20T18:27:50.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.921+0000 7fa46d59a640 1 -- 172.21.15.5:0/3376820402 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa458032810 con 0x7fa47006d3d0 2024-05-20T18:27:50.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.922+0000 7fa47497e640 1 -- 172.21.15.5:0/3376820402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa47006d3d0 msgr2=0x7fa47006d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:50.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.922+0000 7fa47497e640 1 --2- 172.21.15.5:0/3376820402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa47006d3d0 0x7fa47006d7e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa4580097a0 tx=0x7fa45802a1e0 comp rx=0 tx=0).stop 2024-05-20T18:27:50.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.922+0000 7fa47497e640 1 -- 172.21.15.5:0/3376820402 shutdown_connections 2024-05-20T18:27:50.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.922+0000 7fa47497e640 1 --2- 172.21.15.5:0/3376820402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa47006d3d0 0x7fa47006d7e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:50.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.922+0000 7fa47497e640 1 -- 172.21.15.5:0/3376820402 >> 172.21.15.5:0/3376820402 conn(0x7fa4700d4bd0 msgr2=0x7fa4700d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:50.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.922+0000 7fa47497e640 1 -- 172.21.15.5:0/3376820402 shutdown_connections 2024-05-20T18:27:50.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.922+0000 7fa47497e640 1 -- 172.21.15.5:0/3376820402 wait complete. 2024-05-20T18:27:50.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.924+0000 7fa47497e640 1 Processor -- start 2024-05-20T18:27:50.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.924+0000 7fa47497e640 1 -- start start 2024-05-20T18:27:50.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.925+0000 7fa47497e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa47006d3d0 0x7fa47014f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:50.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.925+0000 7fa47497e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4700b8190 0x7fa4701e9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:50.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.925+0000 7fa47497e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa470162820 0x7fa4700b95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:50.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.925+0000 7fa47497e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4700d0300 con 0x7fa47006d3d0 2024-05-20T18:27:50.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.925+0000 7fa47497e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa4700d42e0 con 0x7fa470162820 2024-05-20T18:27:50.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.925+0000 7fa47497e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa4700b4050 con 0x7fa4700b8190 2024-05-20T18:27:50.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.925+0000 7fa46e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa47006d3d0 0x7fa47014f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:50.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.926+0000 7fa46e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa47006d3d0 0x7fa47014f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35588/0 (socket says 172.21.15.5:35588) 2024-05-20T18:27:50.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.926+0000 7fa46e59c640 1 -- 172.21.15.5:0/3251057882 learned_addr learned my addr 172.21.15.5:0/3251057882 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:50.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.926+0000 7fa46ed9d640 1 --2- 172.21.15.5:0/3251057882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa470162820 0x7fa4700b95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:50.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.926+0000 7fa46dd9b640 1 --2- 172.21.15.5:0/3251057882 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4700b8190 0x7fa4701e9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:50.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.926+0000 7fa46e59c640 1 -- 172.21.15.5:0/3251057882 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4700b8190 msgr2=0x7fa4701e9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:50.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.926+0000 7fa46e59c640 1 --2- 172.21.15.5:0/3251057882 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4700b8190 0x7fa4701e9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:50.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.926+0000 7fa46e59c640 1 -- 172.21.15.5:0/3251057882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa470162820 msgr2=0x7fa4700b95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:50.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.926+0000 7fa46e59c640 1 --2- 172.21.15.5:0/3251057882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa470162820 0x7fa4700b95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:50.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.926+0000 7fa46e59c640 1 -- 172.21.15.5:0/3251057882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4701e9bd0 con 0x7fa47006d3d0 2024-05-20T18:27:50.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.926+0000 7fa46e59c640 1 --2- 172.21.15.5:0/3251057882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa47006d3d0 0x7fa47014f6f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa46000caa0 tx=0x7fa46000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:50.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.926+0000 7fa466ffd640 1 -- 172.21.15.5:0/3251057882 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460017070 con 0x7fa47006d3d0 2024-05-20T18:27:50.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.926+0000 7fa466ffd640 1 -- 172.21.15.5:0/3251057882 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa46000ac00 con 0x7fa47006d3d0 2024-05-20T18:27:50.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.926+0000 7fa466ffd640 1 -- 172.21.15.5:0/3251057882 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460012400 con 0x7fa47006d3d0 2024-05-20T18:27:50.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.926+0000 7fa47497e640 1 -- 172.21.15.5:0/3251057882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa47014fc30 con 0x7fa47006d3d0 2024-05-20T18:27:50.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.926+0000 7fa47497e640 1 -- 172.21.15.5:0/3251057882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa470150240 con 0x7fa47006d3d0 2024-05-20T18:27:50.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.927+0000 7fa47497e640 1 -- 172.21.15.5:0/3251057882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa47006db40 con 0x7fa47006d3d0 2024-05-20T18:27:50.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.928+0000 7fa466ffd640 1 -- 172.21.15.5:0/3251057882 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa46001a070 con 0x7fa47006d3d0 2024-05-20T18:27:50.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.928+0000 7fa466ffd640 1 --2- 172.21.15.5:0/3251057882 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa45006be30 0x7fa45006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:50.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.928+0000 7fa466ffd640 1 -- 172.21.15.5:0/3251057882 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa46008af70 con 0x7fa47006d3d0 2024-05-20T18:27:50.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.928+0000 7fa46dd9b640 1 --2- 172.21.15.5:0/3251057882 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa45006be30 0x7fa45006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:50.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.929+0000 7fa46dd9b640 1 --2- 172.21.15.5:0/3251057882 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa45006be30 0x7fa45006e340 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa458034000 tx=0x7fa45802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:50.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:50.933+0000 7fa466ffd640 1 -- 172.21.15.5:0/3251057882 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa47006db40 con 0x7fa47006d3d0 2024-05-20T18:27:51.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.115+0000 7fa47497e640 1 -- 172.21.15.5:0/3251057882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa470075b00 con 0x7fa47006d3d0 2024-05-20T18:27:51.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.115+0000 7fa466ffd640 1 -- 172.21.15.5:0/3251057882 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fa47014fc30 con 0x7fa47006d3d0 2024-05-20T18:27:51.115 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:27:51.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.118+0000 7fa47497e640 1 -- 172.21.15.5:0/3251057882 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa45006be30 msgr2=0x7fa45006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:51.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.118+0000 7fa47497e640 1 --2- 172.21.15.5:0/3251057882 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa45006be30 0x7fa45006e340 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa458034000 tx=0x7fa45802e000 comp rx=0 tx=0).stop 2024-05-20T18:27:51.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.118+0000 7fa47497e640 1 -- 172.21.15.5:0/3251057882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa47006d3d0 msgr2=0x7fa47014f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:51.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.118+0000 7fa47497e640 1 --2- 172.21.15.5:0/3251057882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa47006d3d0 0x7fa47014f6f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa46000caa0 tx=0x7fa46000cf60 comp rx=0 tx=0).stop 2024-05-20T18:27:51.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.118+0000 7fa47497e640 1 -- 172.21.15.5:0/3251057882 shutdown_connections 2024-05-20T18:27:51.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.118+0000 7fa47497e640 1 --2- 172.21.15.5:0/3251057882 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa45006be30 0x7fa45006e340 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:51.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.118+0000 7fa47497e640 1 --2- 172.21.15.5:0/3251057882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa470162820 0x7fa4700b95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:51.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.118+0000 7fa47497e640 1 --2- 172.21.15.5:0/3251057882 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4700b8190 0x7fa4701e9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:51.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.118+0000 7fa47497e640 1 --2- 172.21.15.5:0/3251057882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa47006d3d0 0x7fa47014f6f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:51.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.118+0000 7fa47497e640 1 -- 172.21.15.5:0/3251057882 >> 172.21.15.5:0/3251057882 conn(0x7fa4700d4bd0 msgr2=0x7fa470160eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:51.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.118+0000 7fa47497e640 1 -- 172.21.15.5:0/3251057882 shutdown_connections 2024-05-20T18:27:51.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.118+0000 7fa47497e640 1 -- 172.21.15.5:0/3251057882 wait complete. 2024-05-20T18:27:51.128 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T18:27:51.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.248+0000 7f65f5673640 1 Processor -- start 2024-05-20T18:27:51.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.248+0000 7f65f5673640 1 -- start start 2024-05-20T18:27:51.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.249+0000 7f65f5673640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f65f00d10e0 0x7f65f00cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:51.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.249+0000 7f65f5673640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65f00ce2f0 con 0x7f65f00b7210 2024-05-20T18:27:51.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.249+0000 7f65f5673640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65f00d1ac0 con 0x7f65f00d14f0 2024-05-20T18:27:51.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.249+0000 7f65f5673640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f65f00d1da0 con 0x7f65f00d10e0 2024-05-20T18:27:51.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.249+0000 7f65ef7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f65f00b7210 0x7f65f00cdbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59250/0 (socket says 172.21.15.5:59250) 2024-05-20T18:27:51.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.250+0000 7f65ef7fe640 1 -- 172.21.15.5:0/719861415 learned_addr learned my addr 172.21.15.5:0/719861415 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:51.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.250+0000 7f65eeffd640 1 --2- 172.21.15.5:0/719861415 >> v2:172.21.15.63:3300/0 conn(0x7f65f00d10e0 0x7f65f00cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:51.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.250+0000 7f65edffb640 1 -- 172.21.15.5:0/719861415 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2398269979 0 0) 0x7f65f00ce2f0 con 0x7f65f00b7210 2024-05-20T18:27:51.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.250+0000 7f65edffb640 1 -- 172.21.15.5:0/719861415 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65d8003520 con 0x7f65f00b7210 2024-05-20T18:27:51.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.250+0000 7f65edffb640 1 -- 172.21.15.5:0/719861415 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1951703649 0 0) 0x7f65d8003520 con 0x7f65f00b7210 2024-05-20T18:27:51.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.250+0000 7f65edffb640 1 -- 172.21.15.5:0/719861415 >> v2:172.21.15.63:3300/0 conn(0x7f65f00d10e0 msgr2=0x7f65f00cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:51.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.250+0000 7f65edffb640 1 --2- 172.21.15.5:0/719861415 >> v2:172.21.15.63:3300/0 conn(0x7f65f00d10e0 0x7f65f00cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:51.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.250+0000 7f65edffb640 1 -- 172.21.15.5:0/719861415 >> v1:172.21.15.63:6789/0 conn(0x7f65f00d14f0 legacy=0x7f65f00cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:51.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.250+0000 7f65edffb640 1 -- 172.21.15.5:0/719861415 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f00d2080 con 0x7f65f00b7210 2024-05-20T18:27:51.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.250+0000 7f65edffb640 1 -- 172.21.15.5:0/719861415 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f65e8001670 con 0x7f65f00b7210 2024-05-20T18:27:51.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.250+0000 7f65edffb640 1 -- 172.21.15.5:0/719861415 >> v1:172.21.15.5:6789/0 conn(0x7f65f00b7210 legacy=0x7f65f00cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:51.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.250+0000 7f65edffb640 1 --2- 172.21.15.5:0/719861415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65d80039b0 0x7f65d8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:51.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.250+0000 7f65edffb640 1 --2- 172.21.15.5:0/719861415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d8004800 0x7f65d8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:51.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.250+0000 7f65edffb640 1 --2- 172.21.15.5:0/719861415 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65d80051c0 0x7f65d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:51.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.250+0000 7f65edffb640 1 -- 172.21.15.5:0/719861415 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f65f00d1da0 con 0x7f65d8004800 2024-05-20T18:27:51.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.250+0000 7f65edffb640 1 -- 172.21.15.5:0/719861415 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f65d800c160 con 0x7f65d80039b0 2024-05-20T18:27:51.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.250+0000 7f65edffb640 1 -- 172.21.15.5:0/719861415 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f65d800c360 con 0x7f65d80051c0 2024-05-20T18:27:51.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.250+0000 7f65ee7fc640 1 --2- 172.21.15.5:0/719861415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d8004800 0x7f65d8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:51.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.250+0000 7f65eeffd640 1 --2- 172.21.15.5:0/719861415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65d80039b0 0x7f65d8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:51.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.250+0000 7f65ef7fe640 1 --2- 172.21.15.5:0/719861415 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65d80051c0 0x7f65d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:51.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.251+0000 7f65ee7fc640 1 -- 172.21.15.5:0/719861415 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65d80051c0 msgr2=0x7f65d800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:51.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.251+0000 7f65ee7fc640 1 --2- 172.21.15.5:0/719861415 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65d80051c0 0x7f65d800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:51.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.251+0000 7f65ee7fc640 1 -- 172.21.15.5:0/719861415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65d80039b0 msgr2=0x7f65d8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:51.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.251+0000 7f65ee7fc640 1 --2- 172.21.15.5:0/719861415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65d80039b0 0x7f65d8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:51.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.251+0000 7f65ee7fc640 1 -- 172.21.15.5:0/719861415 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d8003520 con 0x7f65d8004800 2024-05-20T18:27:51.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.251+0000 7f65ef7fe640 1 --2- 172.21.15.5:0/719861415 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65d80051c0 0x7f65d800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:51.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.251+0000 7f65eeffd640 1 --2- 172.21.15.5:0/719861415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65d80039b0 0x7f65d8003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:51.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.251+0000 7f65ef7fe640 1 -- 172.21.15.5:0/719861415 reap_dead start 2024-05-20T18:27:51.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.251+0000 7f65ee7fc640 1 --2- 172.21.15.5:0/719861415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d8004800 0x7f65d8004c10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f65e4001750 tx=0x7f65d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b23a23694506cfc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:51.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.251+0000 7f65edffb640 1 -- 172.21.15.5:0/719861415 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4001370 con 0x7f65d8004800 2024-05-20T18:27:51.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.252+0000 7f65edffb640 1 -- 172.21.15.5:0/719861415 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65e40052d0 con 0x7f65d8004800 2024-05-20T18:27:51.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.252+0000 7f65edffb640 1 -- 172.21.15.5:0/719861415 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e402abc0 con 0x7f65d8004800 2024-05-20T18:27:51.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.252+0000 7f65f5673640 1 -- 172.21.15.5:0/719861415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d8004800 msgr2=0x7f65d8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:51.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.252+0000 7f65f5673640 1 --2- 172.21.15.5:0/719861415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d8004800 0x7f65d8004c10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f65e4001750 tx=0x7f65d8005830 comp rx=0 tx=0).stop 2024-05-20T18:27:51.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.252+0000 7f65f5673640 1 -- 172.21.15.5:0/719861415 shutdown_connections 2024-05-20T18:27:51.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.252+0000 7f65f5673640 1 --2- 172.21.15.5:0/719861415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d8004800 0x7f65d8004c10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:51.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.252+0000 7f65f5673640 1 -- 172.21.15.5:0/719861415 >> 172.21.15.5:0/719861415 conn(0x7f65f00b44d0 msgr2=0x7f65f00b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:51.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.252+0000 7f65f5673640 1 -- 172.21.15.5:0/719861415 shutdown_connections 2024-05-20T18:27:51.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.252+0000 7f65f5673640 1 -- 172.21.15.5:0/719861415 wait complete. 2024-05-20T18:27:51.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.254+0000 7f65f5673640 1 Processor -- start 2024-05-20T18:27:51.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.254+0000 7f65f5673640 1 -- start start 2024-05-20T18:27:51.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.255+0000 7f65f5673640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65d8004800 0x7f65f01f31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:51.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.255+0000 7f65f5673640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65f00ca180 0x7f65f00ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:51.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.255+0000 7f65f5673640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65f00cbfa0 0x7f65f00cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:51.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.255+0000 7f65f5673640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f65f00b82c0 con 0x7f65f00cbfa0 2024-05-20T18:27:51.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.255+0000 7f65f5673640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f65f00d2640 con 0x7f65f00ca180 2024-05-20T18:27:51.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.256+0000 7f65f5673640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f65f00b8440 con 0x7f65d8004800 2024-05-20T18:27:51.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.256+0000 7f65ef7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65f00cbfa0 0x7f65f00cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:51.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.256+0000 7f65ef7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65f00cbfa0 0x7f65f00cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35604/0 (socket says 172.21.15.5:35604) 2024-05-20T18:27:51.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.256+0000 7f65ef7fe640 1 -- 172.21.15.5:0/2842475077 learned_addr learned my addr 172.21.15.5:0/2842475077 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:51.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.256+0000 7f65eeffd640 1 --2- 172.21.15.5:0/2842475077 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65d8004800 0x7f65f01f31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:51.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.256+0000 7f65ee7fc640 1 --2- 172.21.15.5:0/2842475077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65f00ca180 0x7f65f00ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:51.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.256+0000 7f65ef7fe640 1 -- 172.21.15.5:0/2842475077 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65d8004800 msgr2=0x7f65f01f31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:51.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.256+0000 7f65ef7fe640 1 --2- 172.21.15.5:0/2842475077 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65d8004800 0x7f65f01f31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:51.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.256+0000 7f65ef7fe640 1 -- 172.21.15.5:0/2842475077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65f00ca180 msgr2=0x7f65f00ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:51.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.256+0000 7f65ef7fe640 1 --2- 172.21.15.5:0/2842475077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65f00ca180 0x7f65f00ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:51.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.256+0000 7f65ef7fe640 1 -- 172.21.15.5:0/2842475077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f00cc8f0 con 0x7f65f00cbfa0 2024-05-20T18:27:51.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.256+0000 7f65ef7fe640 1 --2- 172.21.15.5:0/2842475077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65f00cbfa0 0x7f65f00cc3b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f65e8000c00 tx=0x7f65e800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:51.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.256+0000 7f65d77fe640 1 -- 172.21.15.5:0/2842475077 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e800c5b0 con 0x7f65f00cbfa0 2024-05-20T18:27:51.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.256+0000 7f65d77fe640 1 -- 172.21.15.5:0/2842475077 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65e8018070 con 0x7f65f00cbfa0 2024-05-20T18:27:51.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.256+0000 7f65d77fe640 1 -- 172.21.15.5:0/2842475077 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e800c970 con 0x7f65f00cbfa0 2024-05-20T18:27:51.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.256+0000 7f65f5673640 1 -- 172.21.15.5:0/2842475077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f01556d0 con 0x7f65f00cbfa0 2024-05-20T18:27:51.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.256+0000 7f65f5673640 1 -- 172.21.15.5:0/2842475077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f0155c90 con 0x7f65f00cbfa0 2024-05-20T18:27:51.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.258+0000 7f65f5673640 1 -- 172.21.15.5:0/2842475077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65f00b48e0 con 0x7f65f00cbfa0 2024-05-20T18:27:51.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.258+0000 7f65d77fe640 1 -- 172.21.15.5:0/2842475077 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f65e800cb10 con 0x7f65f00cbfa0 2024-05-20T18:27:51.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.259+0000 7f65d77fe640 1 --2- 172.21.15.5:0/2842475077 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f65dc06be60 0x7f65dc06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:51.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.259+0000 7f65eeffd640 1 --2- 172.21.15.5:0/2842475077 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f65dc06be60 0x7f65dc06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:51.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.259+0000 7f65d77fe640 1 -- 172.21.15.5:0/2842475077 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f65e808abc0 con 0x7f65f00cbfa0 2024-05-20T18:27:51.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.259+0000 7f65eeffd640 1 --2- 172.21.15.5:0/2842475077 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f65dc06be60 0x7f65dc06e370 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f65e0007540 tx=0x7f65e000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:51.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.263+0000 7f65d77fe640 1 -- 172.21.15.5:0/2842475077 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f65f00b48e0 con 0x7f65f00cbfa0 2024-05-20T18:27:51.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.447+0000 7f65f5673640 1 -- 172.21.15.5:0/2842475077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f65f0075b00 con 0x7f65f00cbfa0 2024-05-20T18:27:51.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.447+0000 7f65d77fe640 1 -- 172.21.15.5:0/2842475077 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f65f00cc8f0 con 0x7f65f00cbfa0 2024-05-20T18:27:51.447 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:27:51.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.449+0000 7f65f5673640 1 -- 172.21.15.5:0/2842475077 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f65dc06be60 msgr2=0x7f65dc06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:51.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.449+0000 7f65f5673640 1 --2- 172.21.15.5:0/2842475077 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f65dc06be60 0x7f65dc06e370 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f65e0007540 tx=0x7f65e000b040 comp rx=0 tx=0).stop 2024-05-20T18:27:51.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.449+0000 7f65f5673640 1 -- 172.21.15.5:0/2842475077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65f00cbfa0 msgr2=0x7f65f00cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:51.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.449+0000 7f65f5673640 1 --2- 172.21.15.5:0/2842475077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65f00cbfa0 0x7f65f00cc3b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f65e8000c00 tx=0x7f65e800eb90 comp rx=0 tx=0).stop 2024-05-20T18:27:51.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.449+0000 7f65f5673640 1 -- 172.21.15.5:0/2842475077 shutdown_connections 2024-05-20T18:27:51.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.450+0000 7f65f5673640 1 --2- 172.21.15.5:0/2842475077 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f65dc06be60 0x7f65dc06e370 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:51.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.450+0000 7f65f5673640 1 --2- 172.21.15.5:0/2842475077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65f00cbfa0 0x7f65f00cc3b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:51.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.450+0000 7f65f5673640 1 --2- 172.21.15.5:0/2842475077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65f00ca180 0x7f65f00ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:51.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.450+0000 7f65f5673640 1 --2- 172.21.15.5:0/2842475077 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65d8004800 0x7f65f01f31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:51.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.450+0000 7f65f5673640 1 -- 172.21.15.5:0/2842475077 >> 172.21.15.5:0/2842475077 conn(0x7f65f00b44d0 msgr2=0x7f65f015ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:51.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.450+0000 7f65f5673640 1 -- 172.21.15.5:0/2842475077 shutdown_connections 2024-05-20T18:27:51.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.450+0000 7f65f5673640 1 -- 172.21.15.5:0/2842475077 wait complete. 2024-05-20T18:27:51.461 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:27:51.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.576+0000 7f4819537640 1 Processor -- start 2024-05-20T18:27:51.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.576+0000 7f4819537640 1 -- start start 2024-05-20T18:27:51.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.578+0000 7f4819537640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48140b6620 0x7f48140ba3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:51.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.578+0000 7f4819537640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f48140d3bf0 0x7f48140d4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:51.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.578+0000 7f4819537640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48140d4540 con 0x7f4814162ce0 2024-05-20T18:27:51.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.578+0000 7f4819537640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f48140d3480 con 0x7f48140d3bf0 2024-05-20T18:27:51.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.578+0000 7f4819537640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f48141634a0 con 0x7f48140b6620 2024-05-20T18:27:51.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.578+0000 7f48137fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f48140d3bf0 0x7f48140d4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:51.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.578+0000 7f48137fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f48140d3bf0 0x7f48140d4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35608/0 (socket says 172.21.15.5:35608) 2024-05-20T18:27:51.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.578+0000 7f48137fe640 1 -- 172.21.15.5:0/215124550 learned_addr learned my addr 172.21.15.5:0/215124550 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:51.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.578+0000 7f4812ffd640 1 --2- 172.21.15.5:0/215124550 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48140b6620 0x7f48140ba3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:51.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.578+0000 7f48137fe640 1 -- 172.21.15.5:0/215124550 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48140b6620 msgr2=0x7f48140ba3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:51.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.578+0000 7f48137fe640 1 --2- 172.21.15.5:0/215124550 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48140b6620 0x7f48140ba3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:51.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.578+0000 7f48137fe640 1 -- 172.21.15.5:0/215124550 >> v1:172.21.15.63:6789/0 conn(0x7f4814162ce0 legacy=0x7f48140b8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:51.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.578+0000 7f48137fe640 1 -- 172.21.15.5:0/215124550 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4814163780 con 0x7f48140d3bf0 2024-05-20T18:27:51.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.579+0000 7f48137fe640 1 --2- 172.21.15.5:0/215124550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48140d3bf0 0x7f48140d4000 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f480c009870 tx=0x7f480c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290602056ec61378 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:51.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.579+0000 7f4811ffb640 1 -- 172.21.15.5:0/215124550 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c037070 con 0x7f48140d3bf0 2024-05-20T18:27:51.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.579+0000 7f4811ffb640 1 -- 172.21.15.5:0/215124550 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f480c032440 con 0x7f48140d3bf0 2024-05-20T18:27:51.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.579+0000 7f4811ffb640 1 -- 172.21.15.5:0/215124550 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c032810 con 0x7f48140d3bf0 2024-05-20T18:27:51.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.580+0000 7f4819537640 1 -- 172.21.15.5:0/215124550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48140d3bf0 msgr2=0x7f48140d4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:51.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.580+0000 7f4819537640 1 --2- 172.21.15.5:0/215124550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48140d3bf0 0x7f48140d4000 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f480c009870 tx=0x7f480c02a270 comp rx=0 tx=0).stop 2024-05-20T18:27:51.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.580+0000 7f4819537640 1 -- 172.21.15.5:0/215124550 shutdown_connections 2024-05-20T18:27:51.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.580+0000 7f4819537640 1 --2- 172.21.15.5:0/215124550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48140d3bf0 0x7f48140d4000 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:51.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.580+0000 7f4819537640 1 --2- 172.21.15.5:0/215124550 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48140b6620 0x7f48140ba3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:51.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.580+0000 7f4819537640 1 -- 172.21.15.5:0/215124550 >> 172.21.15.5:0/215124550 conn(0x7f481406d240 msgr2=0x7f481406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:51.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.580+0000 7f4819537640 1 -- 172.21.15.5:0/215124550 shutdown_connections 2024-05-20T18:27:51.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.580+0000 7f4819537640 1 -- 172.21.15.5:0/215124550 wait complete. 2024-05-20T18:27:51.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.582+0000 7f4819537640 1 Processor -- start 2024-05-20T18:27:51.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.582+0000 7f4819537640 1 -- start start 2024-05-20T18:27:51.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.583+0000 7f4819537640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48140b6620 0x7f48141529b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:51.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.583+0000 7f4819537640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48140d3bf0 0x7f48140abf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:51.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.583+0000 7f4819537640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4814162ce0 0x7f48140afc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:51.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.583+0000 7f4819537640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f48141630f0 con 0x7f4814162ce0 2024-05-20T18:27:51.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.583+0000 7f4819537640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f48140d36d0 con 0x7f48140d3bf0 2024-05-20T18:27:51.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.583+0000 7f4819537640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f48140ad4e0 con 0x7f48140b6620 2024-05-20T18:27:51.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.583+0000 7f48137fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4814162ce0 0x7f48140afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:51.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.583+0000 7f48137fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4814162ce0 0x7f48140afc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35616/0 (socket says 172.21.15.5:35616) 2024-05-20T18:27:51.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.583+0000 7f48137fe640 1 -- 172.21.15.5:0/866726323 learned_addr learned my addr 172.21.15.5:0/866726323 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:51.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.583+0000 7f4812ffd640 1 --2- 172.21.15.5:0/866726323 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48140b6620 0x7f48141529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:51.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.583+0000 7f48127fc640 1 --2- 172.21.15.5:0/866726323 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48140d3bf0 0x7f48140abf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:51.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.584+0000 7f48137fe640 1 -- 172.21.15.5:0/866726323 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48140b6620 msgr2=0x7f48141529b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:51.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.584+0000 7f48137fe640 1 --2- 172.21.15.5:0/866726323 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48140b6620 0x7f48141529b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:51.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.584+0000 7f48137fe640 1 -- 172.21.15.5:0/866726323 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48140d3bf0 msgr2=0x7f48140abf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:51.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.584+0000 7f48137fe640 1 --2- 172.21.15.5:0/866726323 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48140d3bf0 0x7f48140abf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.584+0000 7f48137fe640 1 -- 172.21.15.5:0/866726323 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4814153460 con 0x7f4814162ce0 2024-05-20T18:27:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.584+0000 7f48137fe640 1 --2- 172.21.15.5:0/866726323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4814162ce0 0x7f48140afc60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f480c009870 tx=0x7f480c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.584+0000 7f47fb7fe640 1 -- 172.21.15.5:0/866726323 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c041020 con 0x7f4814162ce0 2024-05-20T18:27:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.584+0000 7f47fb7fe640 1 -- 172.21.15.5:0/866726323 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f480c03d640 con 0x7f4814162ce0 2024-05-20T18:27:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.584+0000 7f4819537640 1 -- 172.21.15.5:0/866726323 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48140b3710 con 0x7f4814162ce0 2024-05-20T18:27:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.584+0000 7f47fb7fe640 1 -- 172.21.15.5:0/866726323 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c037040 con 0x7f4814162ce0 2024-05-20T18:27:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.584+0000 7f4819537640 1 -- 172.21.15.5:0/866726323 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48140acb50 con 0x7f4814162ce0 2024-05-20T18:27:51.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.585+0000 7f4819537640 1 -- 172.21.15.5:0/866726323 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4814075b00 con 0x7f4814162ce0 2024-05-20T18:27:51.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.586+0000 7f47fb7fe640 1 -- 172.21.15.5:0/866726323 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f480c044020 con 0x7f4814162ce0 2024-05-20T18:27:51.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.586+0000 7f47fb7fe640 1 --2- 172.21.15.5:0/866726323 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f480006be30 0x7f480006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:51.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.587+0000 7f47fb7fe640 1 -- 172.21.15.5:0/866726323 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f480c0ab8d0 con 0x7f4814162ce0 2024-05-20T18:27:51.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.587+0000 7f4812ffd640 1 --2- 172.21.15.5:0/866726323 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f480006be30 0x7f480006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:51.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.587+0000 7f4812ffd640 1 --2- 172.21.15.5:0/866726323 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f480006be30 0x7f480006e340 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4804011340 tx=0x7f480400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:51.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.591+0000 7f47fb7fe640 1 -- 172.21.15.5:0/866726323 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f480c04b030 con 0x7f4814162ce0 2024-05-20T18:27:51.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.777+0000 7f4819537640 1 -- 172.21.15.5:0/866726323 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f48140d2fa0 con 0x7f4814162ce0 2024-05-20T18:27:51.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.779+0000 7f47fb7fe640 1 -- 172.21.15.5:0/866726323 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18394 (secure 0 0 0) 0x7f4814153460 con 0x7f4814162ce0 2024-05-20T18:27:51.778 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:51.778 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:27:13.817633+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:27:05.476596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T18:27:06.137762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T18:27:09.829920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:27:51.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.780+0000 7f4819537640 1 -- 172.21.15.5:0/866726323 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f480006be30 msgr2=0x7f480006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:51.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.780+0000 7f4819537640 1 --2- 172.21.15.5:0/866726323 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f480006be30 0x7f480006e340 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4804011340 tx=0x7f480400b040 comp rx=0 tx=0).stop 2024-05-20T18:27:51.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.780+0000 7f4819537640 1 -- 172.21.15.5:0/866726323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4814162ce0 msgr2=0x7f48140afc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:51.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.780+0000 7f4819537640 1 --2- 172.21.15.5:0/866726323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4814162ce0 0x7f48140afc60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f480c009870 tx=0x7f480c0046c0 comp rx=0 tx=0).stop 2024-05-20T18:27:51.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.781+0000 7f4819537640 1 -- 172.21.15.5:0/866726323 shutdown_connections 2024-05-20T18:27:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.781+0000 7f4819537640 1 --2- 172.21.15.5:0/866726323 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f480006be30 0x7f480006e340 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.781+0000 7f4819537640 1 --2- 172.21.15.5:0/866726323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4814162ce0 0x7f48140afc60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.781+0000 7f4819537640 1 --2- 172.21.15.5:0/866726323 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48140d3bf0 0x7f48140abf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.781+0000 7f4819537640 1 --2- 172.21.15.5:0/866726323 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48140b6620 0x7f48141529b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.781+0000 7f4819537640 1 -- 172.21.15.5:0/866726323 >> 172.21.15.5:0/866726323 conn(0x7f481406d240 msgr2=0x7f48140b8030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.781+0000 7f4819537640 1 -- 172.21.15.5:0/866726323 shutdown_connections 2024-05-20T18:27:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.781+0000 7f4819537640 1 -- 172.21.15.5:0/866726323 wait complete. 2024-05-20T18:27:51.792 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:27:05.476596+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:27:06.137762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T18:27:09.829920+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T18:27:51.792 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:27:51.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.912+0000 7f8397785640 1 Processor -- start 2024-05-20T18:27:51.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.912+0000 7f8397785640 1 -- start start 2024-05-20T18:27:51.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.913+0000 7f8397785640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f839006b520 0x7f839006cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:51.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.913+0000 7f8397785640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f839006eab0 0x7f839015d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:51.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.913+0000 7f8397785640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f839006ba20 con 0x7f8390157770 2024-05-20T18:27:51.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.913+0000 7f8397785640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f839006c630 con 0x7f839006eab0 2024-05-20T18:27:51.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.913+0000 7f8397785640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f839006bd00 con 0x7f839006b520 2024-05-20T18:27:51.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.914+0000 7f8394d20640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8390157770 0x7f839006e3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59254/0 (socket says 172.21.15.5:59254) 2024-05-20T18:27:51.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.914+0000 7f8394d20640 1 -- 172.21.15.5:0/2190438570 learned_addr learned my addr 172.21.15.5:0/2190438570 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:51.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.914+0000 7f8395d22640 1 --2- 172.21.15.5:0/2190438570 >> v2:172.21.15.63:3300/0 conn(0x7f839006eab0 0x7f839015d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:51.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.914+0000 7f8395521640 1 --2- 172.21.15.5:0/2190438570 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f839006b520 0x7f839006cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:51.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.914+0000 7f8383fff640 1 -- 172.21.15.5:0/2190438570 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3534686809 0 0) 0x7f839006ba20 con 0x7f8390157770 2024-05-20T18:27:51.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.914+0000 7f8383fff640 1 -- 172.21.15.5:0/2190438570 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f837c003520 con 0x7f8390157770 2024-05-20T18:27:51.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.914+0000 7f8383fff640 1 -- 172.21.15.5:0/2190438570 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4265886067 0 0) 0x7f837c003520 con 0x7f8390157770 2024-05-20T18:27:51.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.914+0000 7f8383fff640 1 -- 172.21.15.5:0/2190438570 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f839006b520 msgr2=0x7f839006cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:51.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.914+0000 7f8383fff640 1 --2- 172.21.15.5:0/2190438570 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f839006b520 0x7f839006cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:51.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.914+0000 7f8383fff640 1 -- 172.21.15.5:0/2190438570 >> v2:172.21.15.63:3300/0 conn(0x7f839006eab0 msgr2=0x7f839015d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:51.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.914+0000 7f8383fff640 1 --2- 172.21.15.5:0/2190438570 >> v2:172.21.15.63:3300/0 conn(0x7f839006eab0 0x7f839015d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:51.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.914+0000 7f8383fff640 1 -- 172.21.15.5:0/2190438570 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839006bfe0 con 0x7f8390157770 2024-05-20T18:27:51.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.914+0000 7f8383fff640 1 -- 172.21.15.5:0/2190438570 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f8384001670 con 0x7f8390157770 2024-05-20T18:27:51.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.914+0000 7f8383fff640 1 -- 172.21.15.5:0/2190438570 >> v1:172.21.15.5:6789/0 conn(0x7f8390157770 legacy=0x7f839006e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:51.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.914+0000 7f8383fff640 1 --2- 172.21.15.5:0/2190438570 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f837c0039b0 0x7f837c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:51.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.914+0000 7f8383fff640 1 --2- 172.21.15.5:0/2190438570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f837c004800 0x7f837c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:51.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.914+0000 7f8383fff640 1 --2- 172.21.15.5:0/2190438570 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f837c005150 0x7f837c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:51.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.914+0000 7f8383fff640 1 -- 172.21.15.5:0/2190438570 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f839006c630 con 0x7f837c004800 2024-05-20T18:27:51.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.914+0000 7f8383fff640 1 -- 172.21.15.5:0/2190438570 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f839006bd00 con 0x7f837c0039b0 2024-05-20T18:27:51.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.914+0000 7f8383fff640 1 -- 172.21.15.5:0/2190438570 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f837c00bf80 con 0x7f837c005150 2024-05-20T18:27:51.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.914+0000 7f8395521640 1 --2- 172.21.15.5:0/2190438570 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f839006b520 0x7f839006cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:27:51.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.914+0000 7f8395d22640 1 --2- 172.21.15.5:0/2190438570 >> v2:172.21.15.63:3300/0 conn(0x7f839006eab0 0x7f839015d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:27:51.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.914+0000 7f8394d20640 1 --2- 172.21.15.5:0/2190438570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f837c004800 0x7f837c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:51.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.915+0000 7f8395d22640 1 --2- 172.21.15.5:0/2190438570 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f837c005150 0x7f837c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:51.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.915+0000 7f8395521640 1 --2- 172.21.15.5:0/2190438570 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f837c0039b0 0x7f837c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:51.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.915+0000 7f8394d20640 1 -- 172.21.15.5:0/2190438570 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f837c005150 msgr2=0x7f837c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:51.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.915+0000 7f8394d20640 1 --2- 172.21.15.5:0/2190438570 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f837c005150 0x7f837c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:51.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.915+0000 7f8394d20640 1 -- 172.21.15.5:0/2190438570 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f837c0039b0 msgr2=0x7f837c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:51.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.915+0000 7f8394d20640 1 --2- 172.21.15.5:0/2190438570 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f837c0039b0 0x7f837c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:51.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.915+0000 7f8394d20640 1 -- 172.21.15.5:0/2190438570 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f837c003520 con 0x7f837c004800 2024-05-20T18:27:51.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.915+0000 7f8394d20640 1 --2- 172.21.15.5:0/2190438570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f837c004800 0x7f837c004c10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f838400c240 tx=0x7f837c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf95215cb1310d97 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:51.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.916+0000 7f8383fff640 1 -- 172.21.15.5:0/2190438570 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838402e070 con 0x7f837c004800 2024-05-20T18:27:51.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.916+0000 7f8383fff640 1 -- 172.21.15.5:0/2190438570 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8384005af0 con 0x7f837c004800 2024-05-20T18:27:51.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.916+0000 7f8383fff640 1 -- 172.21.15.5:0/2190438570 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8384007d30 con 0x7f837c004800 2024-05-20T18:27:51.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.916+0000 7f8397785640 1 -- 172.21.15.5:0/2190438570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f837c004800 msgr2=0x7f837c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:51.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.916+0000 7f8397785640 1 --2- 172.21.15.5:0/2190438570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f837c004800 0x7f837c004c10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f838400c240 tx=0x7f837c005830 comp rx=0 tx=0).stop 2024-05-20T18:27:51.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.916+0000 7f8395d22640 1 -- 172.21.15.5:0/2190438570 reap_dead start 2024-05-20T18:27:51.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.916+0000 7f8397785640 1 -- 172.21.15.5:0/2190438570 shutdown_connections 2024-05-20T18:27:51.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.916+0000 7f8397785640 1 -- 172.21.15.5:0/2190438570 >> 172.21.15.5:0/2190438570 conn(0x7f83900cd2a0 msgr2=0x7f83900cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:51.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.916+0000 7f8397785640 1 -- 172.21.15.5:0/2190438570 shutdown_connections 2024-05-20T18:27:51.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.916+0000 7f8397785640 1 -- 172.21.15.5:0/2190438570 wait complete. 2024-05-20T18:27:51.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.918+0000 7f8397785640 1 Processor -- start 2024-05-20T18:27:51.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.918+0000 7f8397785640 1 -- start start 2024-05-20T18:27:51.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.919+0000 7f8397785640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f837c004800 0x7f83901493f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:51.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.920+0000 7f8397785640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8390157b80 0x7f83901f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:51.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.920+0000 7f8397785640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83901f2fd0 0x7f8390159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:51.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.920+0000 7f8397785640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f839006f0c0 con 0x7f837c004800 2024-05-20T18:27:51.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.920+0000 7f8397785640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f839006ef40 con 0x7f83901f2fd0 2024-05-20T18:27:51.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.920+0000 7f8397785640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f839006cac0 con 0x7f8390157b80 2024-05-20T18:27:51.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.920+0000 7f8395521640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f837c004800 0x7f83901493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:51.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.920+0000 7f8395521640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f837c004800 0x7f83901493f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35626/0 (socket says 172.21.15.5:35626) 2024-05-20T18:27:51.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.920+0000 7f8395521640 1 -- 172.21.15.5:0/1931974886 learned_addr learned my addr 172.21.15.5:0/1931974886 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:51.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.920+0000 7f8394d20640 1 --2- 172.21.15.5:0/1931974886 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8390157b80 0x7f83901f2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:51.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.920+0000 7f8395d22640 1 --2- 172.21.15.5:0/1931974886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83901f2fd0 0x7f8390159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:51.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.920+0000 7f8395521640 1 -- 172.21.15.5:0/1931974886 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8390157b80 msgr2=0x7f83901f2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:51.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.920+0000 7f8395521640 1 --2- 172.21.15.5:0/1931974886 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8390157b80 0x7f83901f2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:51.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.920+0000 7f8395521640 1 -- 172.21.15.5:0/1931974886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83901f2fd0 msgr2=0x7f8390159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:51.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.920+0000 7f8395521640 1 --2- 172.21.15.5:0/1931974886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83901f2fd0 0x7f8390159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:51.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.920+0000 7f8395521640 1 -- 172.21.15.5:0/1931974886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83900ad830 con 0x7f837c004800 2024-05-20T18:27:51.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.920+0000 7f8395d22640 1 --2- 172.21.15.5:0/1931974886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83901f2fd0 0x7f8390159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:51.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.920+0000 7f8395521640 1 --2- 172.21.15.5:0/1931974886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f837c004800 0x7f83901493f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f838c011600 tx=0x7f838c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:51.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.920+0000 7f8381ffb640 1 -- 172.21.15.5:0/1931974886 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c00f8c0 con 0x7f837c004800 2024-05-20T18:27:51.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.920+0000 7f8381ffb640 1 -- 172.21.15.5:0/1931974886 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f838c00fa60 con 0x7f837c004800 2024-05-20T18:27:51.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.920+0000 7f8397785640 1 -- 172.21.15.5:0/1931974886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83900b10d0 con 0x7f837c004800 2024-05-20T18:27:51.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.920+0000 7f8381ffb640 1 -- 172.21.15.5:0/1931974886 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c00fe10 con 0x7f837c004800 2024-05-20T18:27:51.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.921+0000 7f8397785640 1 -- 172.21.15.5:0/1931974886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83901f3930 con 0x7f837c004800 2024-05-20T18:27:51.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.922+0000 7f8397785640 1 -- 172.21.15.5:0/1931974886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8390075b00 con 0x7f837c004800 2024-05-20T18:27:51.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.922+0000 7f8381ffb640 1 -- 172.21.15.5:0/1931974886 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f838c015460 con 0x7f837c004800 2024-05-20T18:27:51.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.923+0000 7f8381ffb640 1 --2- 172.21.15.5:0/1931974886 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f837406be30 0x7f837406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:51.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.923+0000 7f8381ffb640 1 -- 172.21.15.5:0/1931974886 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f838c08b820 con 0x7f837c004800 2024-05-20T18:27:51.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.923+0000 7f8394d20640 1 --2- 172.21.15.5:0/1931974886 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f837406be30 0x7f837406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:51.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.923+0000 7f8394d20640 1 --2- 172.21.15.5:0/1931974886 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f837406be30 0x7f837406e340 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8384033000 tx=0x7f8384009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:51.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:51.927+0000 7f8381ffb640 1 -- 172.21.15.5:0/1931974886 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f838c019030 con 0x7f837c004800 2024-05-20T18:27:52.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.112+0000 7f8397785640 1 -- 172.21.15.5:0/1931974886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f83900b1c90 con 0x7f837c004800 2024-05-20T18:27:52.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.112+0000 7f8381ffb640 1 -- 172.21.15.5:0/1931974886 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f838c059e00 con 0x7f837c004800 2024-05-20T18:27:52.112 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:27:52.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.114+0000 7f8397785640 1 -- 172.21.15.5:0/1931974886 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f837406be30 msgr2=0x7f837406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:52.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.114+0000 7f8397785640 1 --2- 172.21.15.5:0/1931974886 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f837406be30 0x7f837406e340 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8384033000 tx=0x7f8384009030 comp rx=0 tx=0).stop 2024-05-20T18:27:52.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.115+0000 7f8397785640 1 -- 172.21.15.5:0/1931974886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f837c004800 msgr2=0x7f83901493f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:52.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.115+0000 7f8397785640 1 --2- 172.21.15.5:0/1931974886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f837c004800 0x7f83901493f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f838c011600 tx=0x7f838c011ac0 comp rx=0 tx=0).stop 2024-05-20T18:27:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.115+0000 7f8397785640 1 -- 172.21.15.5:0/1931974886 shutdown_connections 2024-05-20T18:27:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.115+0000 7f8397785640 1 --2- 172.21.15.5:0/1931974886 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f837406be30 0x7f837406e340 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.115+0000 7f8397785640 1 --2- 172.21.15.5:0/1931974886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83901f2fd0 0x7f8390159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.115+0000 7f8397785640 1 --2- 172.21.15.5:0/1931974886 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8390157b80 0x7f83901f2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.115+0000 7f8397785640 1 --2- 172.21.15.5:0/1931974886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f837c004800 0x7f83901493f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.115+0000 7f8397785640 1 -- 172.21.15.5:0/1931974886 >> 172.21.15.5:0/1931974886 conn(0x7f83900cd2a0 msgr2=0x7f839015f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.115+0000 7f8397785640 1 -- 172.21.15.5:0/1931974886 shutdown_connections 2024-05-20T18:27:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.115+0000 7f8397785640 1 -- 172.21.15.5:0/1931974886 wait complete. 2024-05-20T18:27:52.125 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:27:52.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.244+0000 7fcbc0a73640 1 Processor -- start 2024-05-20T18:27:52.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.244+0000 7fcbc0a73640 1 -- start start 2024-05-20T18:27:52.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.246+0000 7fcbc0a73640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbbc162820 0x7fcbbc0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:52.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.246+0000 7fcbc0a73640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fcbbc06dae0 0x7fcbbc0b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:52.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.246+0000 7fcbc0a73640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbbc0d0630 con 0x7fcbbc0b3e80 2024-05-20T18:27:52.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.246+0000 7fcbc0a73640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fcbbc0d39c0 con 0x7fcbbc06dae0 2024-05-20T18:27:52.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.246+0000 7fcbc0a73640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcbbc0d3ca0 con 0x7fcbbc162820 2024-05-20T18:27:52.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.246+0000 7fcbb9d9b640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fcbbc0b3e80 0x7fcbbc06d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59266/0 (socket says 172.21.15.5:59266) 2024-05-20T18:27:52.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.246+0000 7fcbb9d9b640 1 -- 172.21.15.5:0/3572637994 learned_addr learned my addr 172.21.15.5:0/3572637994 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:52.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.246+0000 7fcbbad9d640 1 --2- 172.21.15.5:0/3572637994 >> v2:172.21.15.63:3300/0 conn(0x7fcbbc06dae0 0x7fcbbc0b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:52.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.246+0000 7fcbba59c640 1 --2- 172.21.15.5:0/3572637994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbbc162820 0x7fcbbc0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:52.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.246+0000 7fcbb959a640 1 -- 172.21.15.5:0/3572637994 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2691748067 0 0) 0x7fcbbc0d0630 con 0x7fcbbc0b3e80 2024-05-20T18:27:52.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.246+0000 7fcbb959a640 1 -- 172.21.15.5:0/3572637994 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcba8003520 con 0x7fcbbc0b3e80 2024-05-20T18:27:52.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.246+0000 7fcbb959a640 1 -- 172.21.15.5:0/3572637994 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1036027262 0 0) 0x7fcba8003520 con 0x7fcbbc0b3e80 2024-05-20T18:27:52.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.247+0000 7fcbb959a640 1 -- 172.21.15.5:0/3572637994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbbc162820 msgr2=0x7fcbbc0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:52.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.247+0000 7fcbb959a640 1 --2- 172.21.15.5:0/3572637994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbbc162820 0x7fcbbc0b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:52.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.247+0000 7fcbb959a640 1 -- 172.21.15.5:0/3572637994 >> v2:172.21.15.63:3300/0 conn(0x7fcbbc06dae0 msgr2=0x7fcbbc0b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:52.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.247+0000 7fcbb959a640 1 --2- 172.21.15.5:0/3572637994 >> v2:172.21.15.63:3300/0 conn(0x7fcbbc06dae0 0x7fcbbc0b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:52.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.247+0000 7fcbb959a640 1 -- 172.21.15.5:0/3572637994 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbbc0d3f80 con 0x7fcbbc0b3e80 2024-05-20T18:27:52.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.247+0000 7fcbb959a640 1 -- 172.21.15.5:0/3572637994 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7fcba4001670 con 0x7fcbbc0b3e80 2024-05-20T18:27:52.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.247+0000 7fcbb959a640 1 -- 172.21.15.5:0/3572637994 >> v1:172.21.15.5:6789/0 conn(0x7fcbbc0b3e80 legacy=0x7fcbbc06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:52.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.247+0000 7fcbb959a640 1 --2- 172.21.15.5:0/3572637994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba80039b0 0x7fcba8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:52.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.247+0000 7fcbb959a640 1 --2- 172.21.15.5:0/3572637994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbbc162820 0x7fcba8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:52.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.247+0000 7fcbb959a640 1 --2- 172.21.15.5:0/3572637994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba8004d40 0x7fcba8005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:52.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.247+0000 7fcbb959a640 1 -- 172.21.15.5:0/3572637994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc0d39c0 con 0x7fcba8004d40 2024-05-20T18:27:52.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.247+0000 7fcbb959a640 1 -- 172.21.15.5:0/3572637994 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc0d3ca0 con 0x7fcba80039b0 2024-05-20T18:27:52.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.247+0000 7fcbb959a640 1 -- 172.21.15.5:0/3572637994 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcba800b9c0 con 0x7fcbbc162820 2024-05-20T18:27:52.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.247+0000 7fcbbad9d640 1 --2- 172.21.15.5:0/3572637994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba8004d40 0x7fcba8005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:52.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.247+0000 7fcbb9d9b640 1 --2- 172.21.15.5:0/3572637994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbbc162820 0x7fcba8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:52.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.247+0000 7fcbba59c640 1 --2- 172.21.15.5:0/3572637994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba80039b0 0x7fcba8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:52.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.247+0000 7fcbbad9d640 1 -- 172.21.15.5:0/3572637994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbbc162820 msgr2=0x7fcba8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:52.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.247+0000 7fcbbad9d640 1 --2- 172.21.15.5:0/3572637994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbbc162820 0x7fcba8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:52.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.247+0000 7fcbbad9d640 1 -- 172.21.15.5:0/3572637994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba80039b0 msgr2=0x7fcba8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:52.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.247+0000 7fcbbad9d640 1 --2- 172.21.15.5:0/3572637994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba80039b0 0x7fcba8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:52.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.247+0000 7fcbbad9d640 1 -- 172.21.15.5:0/3572637994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba8003520 con 0x7fcba8004d40 2024-05-20T18:27:52.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.247+0000 7fcbba59c640 1 --2- 172.21.15.5:0/3572637994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba80039b0 0x7fcba8003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:27:52.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.248+0000 7fcbbad9d640 1 --2- 172.21.15.5:0/3572637994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba8004d40 0x7fcba8005150 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009fc0 tx=0x7fcba8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=490dacac1d513f40 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:52.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.248+0000 7fcbb959a640 1 -- 172.21.15.5:0/3572637994 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac009c10 con 0x7fcba8004d40 2024-05-20T18:27:52.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.248+0000 7fcbb959a640 1 -- 172.21.15.5:0/3572637994 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbac036070 con 0x7fcba8004d40 2024-05-20T18:27:52.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.248+0000 7fcbb959a640 1 -- 172.21.15.5:0/3572637994 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac02d070 con 0x7fcba8004d40 2024-05-20T18:27:52.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.249+0000 7fcbc0a73640 1 -- 172.21.15.5:0/3572637994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba8004d40 msgr2=0x7fcba8005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:52.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.249+0000 7fcbc0a73640 1 --2- 172.21.15.5:0/3572637994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba8004d40 0x7fcba8005150 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009fc0 tx=0x7fcba8005830 comp rx=0 tx=0).stop 2024-05-20T18:27:52.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.249+0000 7fcbbad9d640 1 -- 172.21.15.5:0/3572637994 reap_dead start 2024-05-20T18:27:52.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.249+0000 7fcbc0a73640 1 -- 172.21.15.5:0/3572637994 shutdown_connections 2024-05-20T18:27:52.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.249+0000 7fcbc0a73640 1 -- 172.21.15.5:0/3572637994 >> 172.21.15.5:0/3572637994 conn(0x7fcbbc0d4bd0 msgr2=0x7fcbbc0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:52.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.249+0000 7fcbc0a73640 1 -- 172.21.15.5:0/3572637994 shutdown_connections 2024-05-20T18:27:52.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.249+0000 7fcbc0a73640 1 -- 172.21.15.5:0/3572637994 wait complete. 2024-05-20T18:27:52.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.251+0000 7fcbc0a73640 1 Processor -- start 2024-05-20T18:27:52.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.251+0000 7fcbc0a73640 1 -- start start 2024-05-20T18:27:52.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.252+0000 7fcbc0a73640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba8004d40 0x7fcbbc157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:52.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.252+0000 7fcbc0a73640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbbc155ce0 0x7fcbbc0cea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:52.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.252+0000 7fcbc0a73640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbbc1f8900 0x7fcbbc1f8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:52.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.252+0000 7fcbc0a73640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc0d4780 con 0x7fcbbc155ce0 2024-05-20T18:27:52.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.252+0000 7fcbc0a73640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc0d0300 con 0x7fcba8004d40 2024-05-20T18:27:52.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.252+0000 7fcbc0a73640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcbbc0d4900 con 0x7fcbbc1f8900 2024-05-20T18:27:52.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.252+0000 7fcbb9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbbc155ce0 0x7fcbbc0cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:52.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.252+0000 7fcbb9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbbc155ce0 0x7fcbbc0cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35642/0 (socket says 172.21.15.5:35642) 2024-05-20T18:27:52.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.252+0000 7fcbb9d9b640 1 -- 172.21.15.5:0/495915482 learned_addr learned my addr 172.21.15.5:0/495915482 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:52.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.253+0000 7fcbbad9d640 1 --2- 172.21.15.5:0/495915482 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbbc1f8900 0x7fcbbc1f8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:52.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.253+0000 7fcbba59c640 1 --2- 172.21.15.5:0/495915482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba8004d40 0x7fcbbc157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:52.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.253+0000 7fcbb9d9b640 1 -- 172.21.15.5:0/495915482 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbbc1f8900 msgr2=0x7fcbbc1f8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:52.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.253+0000 7fcbb9d9b640 1 --2- 172.21.15.5:0/495915482 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbbc1f8900 0x7fcbbc1f8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:52.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.253+0000 7fcbb9d9b640 1 -- 172.21.15.5:0/495915482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba8004d40 msgr2=0x7fcbbc157f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:52.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.253+0000 7fcbb9d9b640 1 --2- 172.21.15.5:0/495915482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba8004d40 0x7fcbbc157f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:52.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.253+0000 7fcbb9d9b640 1 -- 172.21.15.5:0/495915482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbbc1f9250 con 0x7fcbbc155ce0 2024-05-20T18:27:52.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.253+0000 7fcbba59c640 1 --2- 172.21.15.5:0/495915482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba8004d40 0x7fcbbc157f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:52.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.253+0000 7fcbb9d9b640 1 --2- 172.21.15.5:0/495915482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbbc155ce0 0x7fcbbc0cea20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcba4007340 tx=0x7fcba4007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:52.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.253+0000 7fcbb2ffd640 1 -- 172.21.15.5:0/495915482 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba400f8e0 con 0x7fcbbc155ce0 2024-05-20T18:27:52.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.253+0000 7fcbb2ffd640 1 -- 172.21.15.5:0/495915482 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcba4018070 con 0x7fcbbc155ce0 2024-05-20T18:27:52.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.253+0000 7fcbb2ffd640 1 -- 172.21.15.5:0/495915482 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba400fca0 con 0x7fcbbc155ce0 2024-05-20T18:27:52.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.253+0000 7fcbc0a73640 1 -- 172.21.15.5:0/495915482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbbc0cef60 con 0x7fcbbc155ce0 2024-05-20T18:27:52.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.253+0000 7fcbc0a73640 1 -- 172.21.15.5:0/495915482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbbc158ba0 con 0x7fcbbc155ce0 2024-05-20T18:27:52.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.254+0000 7fcbc0a73640 1 -- 172.21.15.5:0/495915482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbbc158630 con 0x7fcbbc155ce0 2024-05-20T18:27:52.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.255+0000 7fcbb2ffd640 1 -- 172.21.15.5:0/495915482 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcba401d460 con 0x7fcbbc155ce0 2024-05-20T18:27:52.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.255+0000 7fcbb2ffd640 1 --2- 172.21.15.5:0/495915482 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcb9c06be30 0x7fcb9c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:52.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.255+0000 7fcbb2ffd640 1 -- 172.21.15.5:0/495915482 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fcba408b370 con 0x7fcbbc155ce0 2024-05-20T18:27:52.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.255+0000 7fcbba59c640 1 --2- 172.21.15.5:0/495915482 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcb9c06be30 0x7fcb9c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:52.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.256+0000 7fcbba59c640 1 --2- 172.21.15.5:0/495915482 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcb9c06be30 0x7fcb9c06e340 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4009be0 tx=0x7fcbb4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:52.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.260+0000 7fcbb2ffd640 1 -- 172.21.15.5:0/495915482 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fcbbc1f9250 con 0x7fcbbc155ce0 2024-05-20T18:27:52.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.445+0000 7fcbc0a73640 1 -- 172.21.15.5:0/495915482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcbbc162c30 con 0x7fcbbc155ce0 2024-05-20T18:27:52.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.445+0000 7fcbb2ffd640 1 -- 172.21.15.5:0/495915482 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fcbbc162c30 con 0x7fcbbc155ce0 2024-05-20T18:27:52.445 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:27:52.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.448+0000 7fcbc0a73640 1 -- 172.21.15.5:0/495915482 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcb9c06be30 msgr2=0x7fcb9c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:52.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.448+0000 7fcbc0a73640 1 --2- 172.21.15.5:0/495915482 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcb9c06be30 0x7fcb9c06e340 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4009be0 tx=0x7fcbb4008040 comp rx=0 tx=0).stop 2024-05-20T18:27:52.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.448+0000 7fcbc0a73640 1 -- 172.21.15.5:0/495915482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbbc155ce0 msgr2=0x7fcbbc0cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:52.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.448+0000 7fcbc0a73640 1 --2- 172.21.15.5:0/495915482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbbc155ce0 0x7fcbbc0cea20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcba4007340 tx=0x7fcba4007800 comp rx=0 tx=0).stop 2024-05-20T18:27:52.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.448+0000 7fcbc0a73640 1 -- 172.21.15.5:0/495915482 shutdown_connections 2024-05-20T18:27:52.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.448+0000 7fcbc0a73640 1 --2- 172.21.15.5:0/495915482 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcb9c06be30 0x7fcb9c06e340 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:52.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.448+0000 7fcbc0a73640 1 --2- 172.21.15.5:0/495915482 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbbc1f8900 0x7fcbbc1f8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:52.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.448+0000 7fcbc0a73640 1 --2- 172.21.15.5:0/495915482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbbc155ce0 0x7fcbbc0cea20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:52.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.448+0000 7fcbc0a73640 1 --2- 172.21.15.5:0/495915482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba8004d40 0x7fcbbc157f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:52.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.448+0000 7fcbc0a73640 1 -- 172.21.15.5:0/495915482 >> 172.21.15.5:0/495915482 conn(0x7fcbbc0d4bd0 msgr2=0x7fcbbc06da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:52.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.448+0000 7fcbc0a73640 1 -- 172.21.15.5:0/495915482 shutdown_connections 2024-05-20T18:27:52.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.448+0000 7fcbc0a73640 1 -- 172.21.15.5:0/495915482 wait complete. 2024-05-20T18:27:52.459 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:27:52.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.577+0000 7fe9778b8640 1 Processor -- start 2024-05-20T18:27:52.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.577+0000 7fe9778b8640 1 -- start start 2024-05-20T18:27:52.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.579+0000 7fe9778b8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe970162820 0x7fe9700b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:52.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.579+0000 7fe9778b8640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe97006dae0 0x7fe9700b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:52.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.579+0000 7fe9778b8640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9700d0630 con 0x7fe9700b3e80 2024-05-20T18:27:52.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.579+0000 7fe9778b8640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe9700d39c0 con 0x7fe97006dae0 2024-05-20T18:27:52.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.579+0000 7fe9778b8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe9700d3ca0 con 0x7fe970162820 2024-05-20T18:27:52.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.579+0000 7fe975e55640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe97006dae0 0x7fe9700b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:52.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.579+0000 7fe975e55640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe97006dae0 0x7fe9700b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35658/0 (socket says 172.21.15.5:35658) 2024-05-20T18:27:52.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.579+0000 7fe975e55640 1 -- 172.21.15.5:0/1570936206 learned_addr learned my addr 172.21.15.5:0/1570936206 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:52.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.579+0000 7fe975654640 1 --2- 172.21.15.5:0/1570936206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe970162820 0x7fe9700b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:52.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.579+0000 7fe975e55640 1 -- 172.21.15.5:0/1570936206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe970162820 msgr2=0x7fe9700b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:52.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.579+0000 7fe975e55640 1 --2- 172.21.15.5:0/1570936206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe970162820 0x7fe9700b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:52.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.579+0000 7fe975e55640 1 -- 172.21.15.5:0/1570936206 >> v1:172.21.15.63:6789/0 conn(0x7fe9700b3e80 legacy=0x7fe97006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:52.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.579+0000 7fe975e55640 1 -- 172.21.15.5:0/1570936206 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9700d3f80 con 0x7fe97006dae0 2024-05-20T18:27:52.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.580+0000 7fe975e55640 1 --2- 172.21.15.5:0/1570936206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe97006dae0 0x7fe9700b8190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe968009870 tx=0x7fe96802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea9cd8761448005 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:52.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.580+0000 7fe96ffff640 1 -- 172.21.15.5:0/1570936206 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe968037070 con 0x7fe97006dae0 2024-05-20T18:27:52.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.580+0000 7fe96ffff640 1 -- 172.21.15.5:0/1570936206 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe968032440 con 0x7fe97006dae0 2024-05-20T18:27:52.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.580+0000 7fe96ffff640 1 -- 172.21.15.5:0/1570936206 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe968032810 con 0x7fe97006dae0 2024-05-20T18:27:52.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.581+0000 7fe9778b8640 1 -- 172.21.15.5:0/1570936206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe97006dae0 msgr2=0x7fe9700b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:52.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.581+0000 7fe9778b8640 1 --2- 172.21.15.5:0/1570936206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe97006dae0 0x7fe9700b8190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe968009870 tx=0x7fe96802a270 comp rx=0 tx=0).stop 2024-05-20T18:27:52.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.581+0000 7fe9778b8640 1 -- 172.21.15.5:0/1570936206 shutdown_connections 2024-05-20T18:27:52.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.581+0000 7fe9778b8640 1 --2- 172.21.15.5:0/1570936206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe97006dae0 0x7fe9700b8190 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:52.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.581+0000 7fe9778b8640 1 --2- 172.21.15.5:0/1570936206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe970162820 0x7fe9700b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:52.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.581+0000 7fe9778b8640 1 -- 172.21.15.5:0/1570936206 >> 172.21.15.5:0/1570936206 conn(0x7fe9700d4bd0 msgr2=0x7fe9700d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:52.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.581+0000 7fe9778b8640 1 -- 172.21.15.5:0/1570936206 shutdown_connections 2024-05-20T18:27:52.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.581+0000 7fe9778b8640 1 -- 172.21.15.5:0/1570936206 wait complete. 2024-05-20T18:27:52.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.583+0000 7fe9778b8640 1 Processor -- start 2024-05-20T18:27:52.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.583+0000 7fe9778b8640 1 -- start start 2024-05-20T18:27:52.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.584+0000 7fe9778b8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9700b3e80 0x7fe9701f5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:52.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.584+0000 7fe9778b8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe970162820 0x7fe9701f8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:52.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.584+0000 7fe9778b8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9701f93c0 0x7fe9700caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:52.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.584+0000 7fe9778b8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe9700d4780 con 0x7fe970162820 2024-05-20T18:27:52.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.584+0000 7fe9778b8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe9700d0300 con 0x7fe9700b3e80 2024-05-20T18:27:52.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.584+0000 7fe9778b8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe9700d4900 con 0x7fe9701f93c0 2024-05-20T18:27:52.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.584+0000 7fe974e53640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe970162820 0x7fe9701f8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:52.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.585+0000 7fe974e53640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe970162820 0x7fe9701f8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35670/0 (socket says 172.21.15.5:35670) 2024-05-20T18:27:52.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.585+0000 7fe974e53640 1 -- 172.21.15.5:0/1178627423 learned_addr learned my addr 172.21.15.5:0/1178627423 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:52.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.585+0000 7fe975654640 1 --2- 172.21.15.5:0/1178627423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9700b3e80 0x7fe9701f5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:52.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.585+0000 7fe975e55640 1 --2- 172.21.15.5:0/1178627423 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9701f93c0 0x7fe9700caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:52.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.585+0000 7fe974e53640 1 -- 172.21.15.5:0/1178627423 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9701f93c0 msgr2=0x7fe9700caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:52.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.585+0000 7fe974e53640 1 --2- 172.21.15.5:0/1178627423 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9701f93c0 0x7fe9700caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:52.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.585+0000 7fe974e53640 1 -- 172.21.15.5:0/1178627423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9700b3e80 msgr2=0x7fe9701f5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:52.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.585+0000 7fe974e53640 1 --2- 172.21.15.5:0/1178627423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9700b3e80 0x7fe9701f5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:52.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.585+0000 7fe974e53640 1 -- 172.21.15.5:0/1178627423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9701f97d0 con 0x7fe970162820 2024-05-20T18:27:52.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.585+0000 7fe974e53640 1 --2- 172.21.15.5:0/1178627423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe970162820 0x7fe9701f8e80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe96400c470 tx=0x7fe96400c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:52.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.585+0000 7fe96dffb640 1 -- 172.21.15.5:0/1178627423 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9640052d0 con 0x7fe970162820 2024-05-20T18:27:52.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.585+0000 7fe96dffb640 1 -- 172.21.15.5:0/1178627423 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe964005490 con 0x7fe970162820 2024-05-20T18:27:52.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.585+0000 7fe9778b8640 1 -- 172.21.15.5:0/1178627423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9700c9ca0 con 0x7fe970162820 2024-05-20T18:27:52.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.585+0000 7fe96dffb640 1 -- 172.21.15.5:0/1178627423 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964007260 con 0x7fe970162820 2024-05-20T18:27:52.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.586+0000 7fe9778b8640 1 -- 172.21.15.5:0/1178627423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9700ca260 con 0x7fe970162820 2024-05-20T18:27:52.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.586+0000 7fe9778b8640 1 -- 172.21.15.5:0/1178627423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe970075b00 con 0x7fe970162820 2024-05-20T18:27:52.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.587+0000 7fe96dffb640 1 -- 172.21.15.5:0/1178627423 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe964007400 con 0x7fe970162820 2024-05-20T18:27:52.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.587+0000 7fe96dffb640 1 --2- 172.21.15.5:0/1178627423 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe95c06be30 0x7fe95c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:52.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.587+0000 7fe96dffb640 1 -- 172.21.15.5:0/1178627423 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe96408a5c0 con 0x7fe970162820 2024-05-20T18:27:52.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.587+0000 7fe975654640 1 --2- 172.21.15.5:0/1178627423 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe95c06be30 0x7fe95c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:52.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.588+0000 7fe975654640 1 --2- 172.21.15.5:0/1178627423 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe95c06be30 0x7fe95c06e340 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe960011340 tx=0x7fe96000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:52.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.592+0000 7fe96dffb640 1 -- 172.21.15.5:0/1178627423 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe9701f97d0 con 0x7fe970162820 2024-05-20T18:27:52.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.773+0000 7fe9778b8640 1 -- 172.21.15.5:0/1178627423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe97015d6f0 con 0x7fe970162820 2024-05-20T18:27:52.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.774+0000 7fe96dffb640 1 -- 172.21.15.5:0/1178627423 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fe964058550 con 0x7fe970162820 2024-05-20T18:27:52.773 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:27:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.776+0000 7fe9778b8640 1 -- 172.21.15.5:0/1178627423 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe95c06be30 msgr2=0x7fe95c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.776+0000 7fe9778b8640 1 --2- 172.21.15.5:0/1178627423 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe95c06be30 0x7fe95c06e340 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe960011340 tx=0x7fe96000b040 comp rx=0 tx=0).stop 2024-05-20T18:27:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.776+0000 7fe9778b8640 1 -- 172.21.15.5:0/1178627423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe970162820 msgr2=0x7fe9701f8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.776+0000 7fe9778b8640 1 --2- 172.21.15.5:0/1178627423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe970162820 0x7fe9701f8e80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe96400c470 tx=0x7fe96400c930 comp rx=0 tx=0).stop 2024-05-20T18:27:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.776+0000 7fe9778b8640 1 -- 172.21.15.5:0/1178627423 shutdown_connections 2024-05-20T18:27:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.776+0000 7fe9778b8640 1 --2- 172.21.15.5:0/1178627423 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe95c06be30 0x7fe95c06e340 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.776+0000 7fe9778b8640 1 --2- 172.21.15.5:0/1178627423 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9701f93c0 0x7fe9700caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.776+0000 7fe9778b8640 1 --2- 172.21.15.5:0/1178627423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe970162820 0x7fe9701f8e80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.776+0000 7fe9778b8640 1 --2- 172.21.15.5:0/1178627423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9700b3e80 0x7fe9701f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.776+0000 7fe9778b8640 1 -- 172.21.15.5:0/1178627423 >> 172.21.15.5:0/1178627423 conn(0x7fe9700d4bd0 msgr2=0x7fe97015ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.776+0000 7fe9778b8640 1 -- 172.21.15.5:0/1178627423 shutdown_connections 2024-05-20T18:27:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.776+0000 7fe9778b8640 1 -- 172.21.15.5:0/1178627423 wait complete. 2024-05-20T18:27:52.793 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T18:27:52.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.915+0000 7fd0a490f640 1 Processor -- start 2024-05-20T18:27:52.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.915+0000 7fd0a490f640 1 -- start start 2024-05-20T18:27:52.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.917+0000 7fd0a490f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd0a006d3d0 0x7fd0a006d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:52.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.917+0000 7fd0a490f640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd0a00b8190 0x7fd0a00b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:52.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.917+0000 7fd0a490f640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0a006dd20 con 0x7fd0a0162820 2024-05-20T18:27:52.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.917+0000 7fd0a490f640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fd0a00d0630 con 0x7fd0a006d3d0 2024-05-20T18:27:52.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.917+0000 7fd0a490f640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd0a00d39c0 con 0x7fd0a00b8190 2024-05-20T18:27:52.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.917+0000 7fd09dd9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd0a006d3d0 0x7fd0a006d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:52.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.917+0000 7fd09dd9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd0a006d3d0 0x7fd0a006d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35676/0 (socket says 172.21.15.5:35676) 2024-05-20T18:27:52.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.917+0000 7fd09dd9b640 1 -- 172.21.15.5:0/1743618271 learned_addr learned my addr 172.21.15.5:0/1743618271 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:52.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.917+0000 7fd09ed9d640 1 --2- 172.21.15.5:0/1743618271 >> v2:172.21.15.63:3300/0 conn(0x7fd0a00b8190 0x7fd0a00b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:52.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.917+0000 7fd09dd9b640 1 -- 172.21.15.5:0/1743618271 >> v2:172.21.15.63:3300/0 conn(0x7fd0a00b8190 msgr2=0x7fd0a00b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:52.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.917+0000 7fd09dd9b640 1 --2- 172.21.15.5:0/1743618271 >> v2:172.21.15.63:3300/0 conn(0x7fd0a00b8190 0x7fd0a00b85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:52.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.917+0000 7fd09dd9b640 1 -- 172.21.15.5:0/1743618271 >> v1:172.21.15.63:6789/0 conn(0x7fd0a0162820 legacy=0x7fd0a00b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:52.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.917+0000 7fd09dd9b640 1 -- 172.21.15.5:0/1743618271 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a00d3ca0 con 0x7fd0a006d3d0 2024-05-20T18:27:52.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.917+0000 7fd09ed9d640 1 --2- 172.21.15.5:0/1743618271 >> v2:172.21.15.63:3300/0 conn(0x7fd0a00b8190 0x7fd0a00b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:52.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.918+0000 7fd09dd9b640 1 --2- 172.21.15.5:0/1743618271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a006d3d0 0x7fd0a006d7e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd0a008e070 tx=0x7fd09402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42fc9fda22d40e53 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:52.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.918+0000 7fd09d59a640 1 -- 172.21.15.5:0/1743618271 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd094037070 con 0x7fd0a006d3d0 2024-05-20T18:27:52.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.918+0000 7fd09d59a640 1 -- 172.21.15.5:0/1743618271 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd094032440 con 0x7fd0a006d3d0 2024-05-20T18:27:52.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.918+0000 7fd09d59a640 1 -- 172.21.15.5:0/1743618271 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd094032810 con 0x7fd0a006d3d0 2024-05-20T18:27:52.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.919+0000 7fd0a490f640 1 -- 172.21.15.5:0/1743618271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a006d3d0 msgr2=0x7fd0a006d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:52.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.919+0000 7fd0a490f640 1 --2- 172.21.15.5:0/1743618271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a006d3d0 0x7fd0a006d7e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd0a008e070 tx=0x7fd09402a270 comp rx=0 tx=0).stop 2024-05-20T18:27:52.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.919+0000 7fd0a490f640 1 -- 172.21.15.5:0/1743618271 shutdown_connections 2024-05-20T18:27:52.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.919+0000 7fd0a490f640 1 --2- 172.21.15.5:0/1743618271 >> v2:172.21.15.63:3300/0 conn(0x7fd0a00b8190 0x7fd0a00b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:52.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.919+0000 7fd0a490f640 1 --2- 172.21.15.5:0/1743618271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a006d3d0 0x7fd0a006d7e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:52.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.919+0000 7fd0a490f640 1 -- 172.21.15.5:0/1743618271 >> 172.21.15.5:0/1743618271 conn(0x7fd0a00d4bd0 msgr2=0x7fd0a00d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:52.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.919+0000 7fd0a490f640 1 -- 172.21.15.5:0/1743618271 shutdown_connections 2024-05-20T18:27:52.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.919+0000 7fd0a490f640 1 -- 172.21.15.5:0/1743618271 wait complete. 2024-05-20T18:27:52.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.921+0000 7fd0a490f640 1 Processor -- start 2024-05-20T18:27:52.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.921+0000 7fd0a490f640 1 -- start start 2024-05-20T18:27:52.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.922+0000 7fd0a490f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0a006d3d0 0x7fd0a00b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:52.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.922+0000 7fd0a490f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0a00b8190 0x7fd0a01f8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:52.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.922+0000 7fd0a490f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a0162820 0x7fd0a01f9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:52.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.922+0000 7fd0a490f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a00d0300 con 0x7fd0a0162820 2024-05-20T18:27:52.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.922+0000 7fd0a490f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a00d4260 con 0x7fd0a00b8190 2024-05-20T18:27:52.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.922+0000 7fd0a490f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd0a00b4050 con 0x7fd0a006d3d0 2024-05-20T18:27:52.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.922+0000 7fd09ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a0162820 0x7fd0a01f9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:52.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.922+0000 7fd09ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a0162820 0x7fd0a01f9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35692/0 (socket says 172.21.15.5:35692) 2024-05-20T18:27:52.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.922+0000 7fd09ed9d640 1 -- 172.21.15.5:0/2003368419 learned_addr learned my addr 172.21.15.5:0/2003368419 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:52.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.923+0000 7fd09dd9b640 1 --2- 172.21.15.5:0/2003368419 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0a00b8190 0x7fd0a01f8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:52.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.923+0000 7fd09ed9d640 1 -- 172.21.15.5:0/2003368419 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0a006d3d0 msgr2=0x7fd0a00b7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:52.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.923+0000 7fd09ed9d640 1 --2- 172.21.15.5:0/2003368419 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0a006d3d0 0x7fd0a00b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:52.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.923+0000 7fd09ed9d640 1 -- 172.21.15.5:0/2003368419 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0a00b8190 msgr2=0x7fd0a01f8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:52.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.923+0000 7fd09ed9d640 1 --2- 172.21.15.5:0/2003368419 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0a00b8190 0x7fd0a01f8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:52.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.923+0000 7fd09ed9d640 1 -- 172.21.15.5:0/2003368419 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a00ca970 con 0x7fd0a0162820 2024-05-20T18:27:52.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.923+0000 7fd09ed9d640 1 --2- 172.21.15.5:0/2003368419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a0162820 0x7fd0a01f9050 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd098007e20 tx=0x7fd09800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:52.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.923+0000 7fd08effd640 1 -- 172.21.15.5:0/2003368419 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09801a070 con 0x7fd0a0162820 2024-05-20T18:27:52.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.923+0000 7fd08effd640 1 -- 172.21.15.5:0/2003368419 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd098003b90 con 0x7fd0a0162820 2024-05-20T18:27:52.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.923+0000 7fd0a490f640 1 -- 172.21.15.5:0/2003368419 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a00cac50 con 0x7fd0a0162820 2024-05-20T18:27:52.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.923+0000 7fd08effd640 1 -- 172.21.15.5:0/2003368419 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09800de80 con 0x7fd0a0162820 2024-05-20T18:27:52.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.923+0000 7fd0a490f640 1 -- 172.21.15.5:0/2003368419 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a01f68f0 con 0x7fd0a0162820 2024-05-20T18:27:52.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.924+0000 7fd0a490f640 1 -- 172.21.15.5:0/2003368419 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a00ca6e0 con 0x7fd0a0162820 2024-05-20T18:27:52.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.925+0000 7fd08effd640 1 -- 172.21.15.5:0/2003368419 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd09801d070 con 0x7fd0a0162820 2024-05-20T18:27:52.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.925+0000 7fd08effd640 1 --2- 172.21.15.5:0/2003368419 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd08006be30 0x7fd08006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:52.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.925+0000 7fd08effd640 1 -- 172.21.15.5:0/2003368419 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd09808b450 con 0x7fd0a0162820 2024-05-20T18:27:52.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.925+0000 7fd09e59c640 1 --2- 172.21.15.5:0/2003368419 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd08006be30 0x7fd08006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:52.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.926+0000 7fd09e59c640 1 --2- 172.21.15.5:0/2003368419 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd08006be30 0x7fd08006e340 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd090000f60 tx=0x7fd0900079d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:52.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:52.930+0000 7fd08effd640 1 -- 172.21.15.5:0/2003368419 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd098010070 con 0x7fd0a0162820 2024-05-20T18:27:53.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.116+0000 7fd0a490f640 1 -- 172.21.15.5:0/2003368419 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd0a01f66e0 con 0x7fd0a0162820 2024-05-20T18:27:53.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.116+0000 7fd08effd640 1 -- 172.21.15.5:0/2003368419 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fd098059ac0 con 0x7fd0a0162820 2024-05-20T18:27:53.116 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:27:53.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.118+0000 7fd0a490f640 1 -- 172.21.15.5:0/2003368419 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd08006be30 msgr2=0x7fd08006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:53.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.118+0000 7fd0a490f640 1 --2- 172.21.15.5:0/2003368419 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd08006be30 0x7fd08006e340 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd090000f60 tx=0x7fd0900079d0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.118+0000 7fd0a490f640 1 -- 172.21.15.5:0/2003368419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a0162820 msgr2=0x7fd0a01f9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:53.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.118+0000 7fd0a490f640 1 --2- 172.21.15.5:0/2003368419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a0162820 0x7fd0a01f9050 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd098007e20 tx=0x7fd09800d4a0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.118+0000 7fd0a490f640 1 -- 172.21.15.5:0/2003368419 shutdown_connections 2024-05-20T18:27:53.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.118+0000 7fd0a490f640 1 --2- 172.21.15.5:0/2003368419 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd08006be30 0x7fd08006e340 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.118+0000 7fd0a490f640 1 --2- 172.21.15.5:0/2003368419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a0162820 0x7fd0a01f9050 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.118+0000 7fd0a490f640 1 --2- 172.21.15.5:0/2003368419 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0a00b8190 0x7fd0a01f8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.118+0000 7fd0a490f640 1 --2- 172.21.15.5:0/2003368419 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0a006d3d0 0x7fd0a00b7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.118+0000 7fd0a490f640 1 -- 172.21.15.5:0/2003368419 >> 172.21.15.5:0/2003368419 conn(0x7fd0a00d4bd0 msgr2=0x7fd0a015f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:53.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.118+0000 7fd0a490f640 1 -- 172.21.15.5:0/2003368419 shutdown_connections 2024-05-20T18:27:53.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.118+0000 7fd0a490f640 1 -- 172.21.15.5:0/2003368419 wait complete. 2024-05-20T18:27:53.129 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:27:53.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.247+0000 7fe0b8db7640 1 Processor -- start 2024-05-20T18:27:53.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.247+0000 7fe0b8db7640 1 -- start start 2024-05-20T18:27:53.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.248+0000 7fe0b8db7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0b4163a10 0x7fe0b40d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:53.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.248+0000 7fe0b8db7640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe0b40d14b0 0x7fe0b40d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:53.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.248+0000 7fe0b8db7640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0b40d0370 con 0x7fe0b40b4010 2024-05-20T18:27:53.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.248+0000 7fe0b8db7640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe0b40d3ca0 con 0x7fe0b40d14b0 2024-05-20T18:27:53.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.248+0000 7fe0b8db7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe0b40d0650 con 0x7fe0b4163a10 2024-05-20T18:27:53.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.248+0000 7fe0b2d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe0b40d14b0 0x7fe0b40d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:53.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.248+0000 7fe0b2d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe0b40d14b0 0x7fe0b40d18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35700/0 (socket says 172.21.15.5:35700) 2024-05-20T18:27:53.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.248+0000 7fe0b2d9d640 1 -- 172.21.15.5:0/1567598433 learned_addr learned my addr 172.21.15.5:0/1567598433 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:53.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.249+0000 7fe0b259c640 1 --2- 172.21.15.5:0/1567598433 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0b4163a10 0x7fe0b40d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:53.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.249+0000 7fe0b2d9d640 1 -- 172.21.15.5:0/1567598433 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0b4163a10 msgr2=0x7fe0b40d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:53.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.249+0000 7fe0b2d9d640 1 --2- 172.21.15.5:0/1567598433 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0b4163a10 0x7fe0b40d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.249+0000 7fe0b2d9d640 1 -- 172.21.15.5:0/1567598433 >> v1:172.21.15.63:6789/0 conn(0x7fe0b40b4010 legacy=0x7fe0b40cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:53.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.249+0000 7fe0b2d9d640 1 -- 172.21.15.5:0/1567598433 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0b40d3650 con 0x7fe0b40d14b0 2024-05-20T18:27:53.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.249+0000 7fe0b2d9d640 1 --2- 172.21.15.5:0/1567598433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b40d14b0 0x7fe0b40d18c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac009e50 tx=0x7fe0ac02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae97a1a80f99ae3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:53.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.249+0000 7fe0b1d9b640 1 -- 172.21.15.5:0/1567598433 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac036070 con 0x7fe0b40d14b0 2024-05-20T18:27:53.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.250+0000 7fe0b1d9b640 1 -- 172.21.15.5:0/1567598433 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0ac031600 con 0x7fe0b40d14b0 2024-05-20T18:27:53.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.250+0000 7fe0b1d9b640 1 -- 172.21.15.5:0/1567598433 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac0319d0 con 0x7fe0b40d14b0 2024-05-20T18:27:53.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.250+0000 7fe0b8db7640 1 -- 172.21.15.5:0/1567598433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b40d14b0 msgr2=0x7fe0b40d18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:53.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.250+0000 7fe0b8db7640 1 --2- 172.21.15.5:0/1567598433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b40d14b0 0x7fe0b40d18c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac009e50 tx=0x7fe0ac02a6c0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.250+0000 7fe0b8db7640 1 -- 172.21.15.5:0/1567598433 shutdown_connections 2024-05-20T18:27:53.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.250+0000 7fe0b8db7640 1 --2- 172.21.15.5:0/1567598433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b40d14b0 0x7fe0b40d18c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.250+0000 7fe0b8db7640 1 --2- 172.21.15.5:0/1567598433 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0b4163a10 0x7fe0b40d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.250+0000 7fe0b8db7640 1 -- 172.21.15.5:0/1567598433 >> 172.21.15.5:0/1567598433 conn(0x7fe0b406d240 msgr2=0x7fe0b406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:53.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.250+0000 7fe0b8db7640 1 -- 172.21.15.5:0/1567598433 shutdown_connections 2024-05-20T18:27:53.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.250+0000 7fe0b8db7640 1 -- 172.21.15.5:0/1567598433 wait complete. 2024-05-20T18:27:53.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.252+0000 7fe0b8db7640 1 Processor -- start 2024-05-20T18:27:53.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.252+0000 7fe0b8db7640 1 -- start start 2024-05-20T18:27:53.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.254+0000 7fe0b8db7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0b40b4010 0x7fe0b41422a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:53.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.254+0000 7fe0b8db7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0b40d14b0 0x7fe0b415a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:53.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.254+0000 7fe0b8db7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b4163a10 0x7fe0b415a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:53.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.254+0000 7fe0b8db7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b40b8210 con 0x7fe0b4163a10 2024-05-20T18:27:53.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.254+0000 7fe0b8db7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b40d1e00 con 0x7fe0b40b4010 2024-05-20T18:27:53.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.254+0000 7fe0b8db7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe0b40b8390 con 0x7fe0b40d14b0 2024-05-20T18:27:53.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.254+0000 7fe0b2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b4163a10 0x7fe0b415a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:53.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.254+0000 7fe0b2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b4163a10 0x7fe0b415a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35708/0 (socket says 172.21.15.5:35708) 2024-05-20T18:27:53.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.254+0000 7fe0b2d9d640 1 -- 172.21.15.5:0/3233599874 learned_addr learned my addr 172.21.15.5:0/3233599874 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:53.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.254+0000 7fe0b259c640 1 --2- 172.21.15.5:0/3233599874 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0b40b4010 0x7fe0b41422a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:53.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.254+0000 7fe0a9d9b640 1 --2- 172.21.15.5:0/3233599874 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0b40d14b0 0x7fe0b415a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:53.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.254+0000 7fe0b2d9d640 1 -- 172.21.15.5:0/3233599874 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0b40d14b0 msgr2=0x7fe0b415a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:53.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.254+0000 7fe0b2d9d640 1 --2- 172.21.15.5:0/3233599874 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0b40d14b0 0x7fe0b415a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.254+0000 7fe0b2d9d640 1 -- 172.21.15.5:0/3233599874 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0b40b4010 msgr2=0x7fe0b41422a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:53.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.254+0000 7fe0b2d9d640 1 --2- 172.21.15.5:0/3233599874 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0b40b4010 0x7fe0b41422a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.254+0000 7fe0b2d9d640 1 -- 172.21.15.5:0/3233599874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0b4156090 con 0x7fe0b4163a10 2024-05-20T18:27:53.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.254+0000 7fe0b2d9d640 1 --2- 172.21.15.5:0/3233599874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b4163a10 0x7fe0b415a900 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac034040 tx=0x7fe0ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:53.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.255+0000 7fe0ab7fe640 1 -- 172.21.15.5:0/3233599874 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac036050 con 0x7fe0b4163a10 2024-05-20T18:27:53.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.255+0000 7fe0ab7fe640 1 -- 172.21.15.5:0/3233599874 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0ac042070 con 0x7fe0b4163a10 2024-05-20T18:27:53.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.255+0000 7fe0b8db7640 1 -- 172.21.15.5:0/3233599874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0b41427e0 con 0x7fe0b4163a10 2024-05-20T18:27:53.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.255+0000 7fe0ab7fe640 1 -- 172.21.15.5:0/3233599874 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac009510 con 0x7fe0b4163a10 2024-05-20T18:27:53.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.255+0000 7fe0b8db7640 1 -- 172.21.15.5:0/3233599874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0b4142d40 con 0x7fe0b4163a10 2024-05-20T18:27:53.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.256+0000 7fe0b8db7640 1 -- 172.21.15.5:0/3233599874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0b4075b00 con 0x7fe0b4163a10 2024-05-20T18:27:53.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.256+0000 7fe0ab7fe640 1 -- 172.21.15.5:0/3233599874 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe0ac007bf0 con 0x7fe0b4163a10 2024-05-20T18:27:53.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.257+0000 7fe0ab7fe640 1 --2- 172.21.15.5:0/3233599874 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe09406be60 0x7fe09406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:53.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.257+0000 7fe0ab7fe640 1 -- 172.21.15.5:0/3233599874 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe0ac0ac1d0 con 0x7fe0b4163a10 2024-05-20T18:27:53.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.257+0000 7fe0b259c640 1 --2- 172.21.15.5:0/3233599874 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe09406be60 0x7fe09406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:53.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.257+0000 7fe0b259c640 1 --2- 172.21.15.5:0/3233599874 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe09406be60 0x7fe09406e370 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4006fd0 tx=0x7fe0a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:53.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.261+0000 7fe0ab7fe640 1 -- 172.21.15.5:0/3233599874 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe0ac045360 con 0x7fe0b4163a10 2024-05-20T18:27:53.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.446+0000 7fe0b8db7640 1 -- 172.21.15.5:0/3233599874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe0b40d3050 con 0x7fe0b4163a10 2024-05-20T18:27:53.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.448+0000 7fe0ab7fe640 1 -- 172.21.15.5:0/3233599874 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18394 (secure 0 0 0) 0x7fe0ac07a7b0 con 0x7fe0b4163a10 2024-05-20T18:27:53.447 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:53.447 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:27:13.817633+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:27:05.476596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T18:27:06.137762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T18:27:09.829920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:27:53.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.449+0000 7fe0b8db7640 1 -- 172.21.15.5:0/3233599874 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe09406be60 msgr2=0x7fe09406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:53.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.449+0000 7fe0b8db7640 1 --2- 172.21.15.5:0/3233599874 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe09406be60 0x7fe09406e370 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4006fd0 tx=0x7fe0a4008040 comp rx=0 tx=0).stop 2024-05-20T18:27:53.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.449+0000 7fe0b8db7640 1 -- 172.21.15.5:0/3233599874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b4163a10 msgr2=0x7fe0b415a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:53.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.449+0000 7fe0b8db7640 1 --2- 172.21.15.5:0/3233599874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b4163a10 0x7fe0b415a900 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac034040 tx=0x7fe0ac004a40 comp rx=0 tx=0).stop 2024-05-20T18:27:53.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.450+0000 7fe0b8db7640 1 -- 172.21.15.5:0/3233599874 shutdown_connections 2024-05-20T18:27:53.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.450+0000 7fe0b8db7640 1 --2- 172.21.15.5:0/3233599874 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe09406be60 0x7fe09406e370 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.450+0000 7fe0b8db7640 1 --2- 172.21.15.5:0/3233599874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b4163a10 0x7fe0b415a900 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.450+0000 7fe0b8db7640 1 --2- 172.21.15.5:0/3233599874 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0b40d14b0 0x7fe0b415a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.450+0000 7fe0b8db7640 1 --2- 172.21.15.5:0/3233599874 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0b40b4010 0x7fe0b41422a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.450+0000 7fe0b8db7640 1 -- 172.21.15.5:0/3233599874 >> 172.21.15.5:0/3233599874 conn(0x7fe0b406d240 msgr2=0x7fe0b40b4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:53.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.450+0000 7fe0b8db7640 1 -- 172.21.15.5:0/3233599874 shutdown_connections 2024-05-20T18:27:53.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.450+0000 7fe0b8db7640 1 -- 172.21.15.5:0/3233599874 wait complete. 2024-05-20T18:27:53.461 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:27:05.476596+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:27:06.137762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T18:27:09.829920+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T18:27:53.461 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:27:53.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.579+0000 7ff094237640 1 Processor -- start 2024-05-20T18:27:53.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.579+0000 7ff094237640 1 -- start start 2024-05-20T18:27:53.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.581+0000 7ff094237640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff08c06d3d0 0x7ff08c06d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:53.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.581+0000 7ff094237640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff08c0b8190 0x7ff08c0b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:53.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.581+0000 7ff094237640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff08c06dd20 con 0x7ff08c162820 2024-05-20T18:27:53.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.581+0000 7ff094237640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7ff08c0d0630 con 0x7ff08c06d3d0 2024-05-20T18:27:53.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.581+0000 7ff094237640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff08c0d39c0 con 0x7ff08c0b8190 2024-05-20T18:27:53.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.581+0000 7ff0917d2640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff08c06d3d0 0x7ff08c06d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:53.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.581+0000 7ff091fd3640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff08c162820 0x7ff08c0b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59268/0 (socket says 172.21.15.5:59268) 2024-05-20T18:27:53.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.581+0000 7ff0917d2640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff08c06d3d0 0x7ff08c06d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35710/0 (socket says 172.21.15.5:35710) 2024-05-20T18:27:53.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.581+0000 7ff091fd3640 1 -- 172.21.15.5:0/3411151962 learned_addr learned my addr 172.21.15.5:0/3411151962 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:53.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.581+0000 7ff0927d4640 1 --2- 172.21.15.5:0/3411151962 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff08c0b8190 0x7ff08c0b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:53.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.581+0000 7ff0917d2640 1 -- 172.21.15.5:0/3411151962 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff08c0b8190 msgr2=0x7ff08c0b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:53.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.581+0000 7ff0917d2640 1 --2- 172.21.15.5:0/3411151962 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff08c0b8190 0x7ff08c0b85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.581+0000 7ff0917d2640 1 -- 172.21.15.5:0/3411151962 >> v1:172.21.15.5:6789/0 conn(0x7ff08c162820 legacy=0x7ff08c0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:53.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.581+0000 7ff0917d2640 1 -- 172.21.15.5:0/3411151962 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08c0d3ca0 con 0x7ff08c06d3d0 2024-05-20T18:27:53.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.581+0000 7ff0927d4640 1 --2- 172.21.15.5:0/3411151962 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff08c0b8190 0x7ff08c0b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:53.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.581+0000 7ff090fd1640 1 -- 172.21.15.5:0/3411151962 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1951284803 0 0) 0x7ff07c001490 con 0x7ff08c162820 2024-05-20T18:27:53.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.582+0000 7ff0917d2640 1 --2- 172.21.15.5:0/3411151962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff08c06d3d0 0x7ff08c06d7e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff0800097a0 tx=0x7ff08002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c067a32aa682c9d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:53.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.582+0000 7ff090fd1640 1 -- 172.21.15.5:0/3411151962 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff080037070 con 0x7ff08c06d3d0 2024-05-20T18:27:53.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.582+0000 7ff090fd1640 1 -- 172.21.15.5:0/3411151962 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff080032440 con 0x7ff08c06d3d0 2024-05-20T18:27:53.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.582+0000 7ff090fd1640 1 -- 172.21.15.5:0/3411151962 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff080032810 con 0x7ff08c06d3d0 2024-05-20T18:27:53.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.583+0000 7ff094237640 1 -- 172.21.15.5:0/3411151962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff08c06d3d0 msgr2=0x7ff08c06d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:53.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.583+0000 7ff094237640 1 --2- 172.21.15.5:0/3411151962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff08c06d3d0 0x7ff08c06d7e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff0800097a0 tx=0x7ff08002a180 comp rx=0 tx=0).stop 2024-05-20T18:27:53.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.583+0000 7ff094237640 1 -- 172.21.15.5:0/3411151962 shutdown_connections 2024-05-20T18:27:53.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.583+0000 7ff094237640 1 --2- 172.21.15.5:0/3411151962 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff08c0b8190 0x7ff08c0b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.583+0000 7ff094237640 1 --2- 172.21.15.5:0/3411151962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff08c06d3d0 0x7ff08c06d7e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.583+0000 7ff094237640 1 -- 172.21.15.5:0/3411151962 >> 172.21.15.5:0/3411151962 conn(0x7ff08c0d4bd0 msgr2=0x7ff08c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.583+0000 7ff094237640 1 -- 172.21.15.5:0/3411151962 shutdown_connections 2024-05-20T18:27:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.583+0000 7ff094237640 1 -- 172.21.15.5:0/3411151962 wait complete. 2024-05-20T18:27:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.585+0000 7ff094237640 1 Processor -- start 2024-05-20T18:27:53.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.585+0000 7ff094237640 1 -- start start 2024-05-20T18:27:53.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.586+0000 7ff094237640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff08c06d3d0 0x7ff08c0c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:53.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.586+0000 7ff094237640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff08c0b8190 0x7ff08c0b87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:53.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.586+0000 7ff094237640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff08c162820 0x7ff08c159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:53.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.586+0000 7ff094237640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c0d0300 con 0x7ff08c0b8190 2024-05-20T18:27:53.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.586+0000 7ff094237640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c0d4260 con 0x7ff08c162820 2024-05-20T18:27:53.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.586+0000 7ff094237640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff08c0b4050 con 0x7ff08c06d3d0 2024-05-20T18:27:53.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.586+0000 7ff0917d2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff08c0b8190 0x7ff08c0b87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:53.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.586+0000 7ff0917d2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff08c0b8190 0x7ff08c0b87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35712/0 (socket says 172.21.15.5:35712) 2024-05-20T18:27:53.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.586+0000 7ff0917d2640 1 -- 172.21.15.5:0/1700848061 learned_addr learned my addr 172.21.15.5:0/1700848061 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:53.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.587+0000 7ff0927d4640 1 --2- 172.21.15.5:0/1700848061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff08c162820 0x7ff08c159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:53.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.587+0000 7ff091fd3640 1 --2- 172.21.15.5:0/1700848061 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff08c06d3d0 0x7ff08c0c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:53.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.587+0000 7ff0917d2640 1 -- 172.21.15.5:0/1700848061 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff08c06d3d0 msgr2=0x7ff08c0c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:53.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.587+0000 7ff0917d2640 1 --2- 172.21.15.5:0/1700848061 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff08c06d3d0 0x7ff08c0c9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.587+0000 7ff0917d2640 1 -- 172.21.15.5:0/1700848061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff08c162820 msgr2=0x7ff08c159be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:53.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.587+0000 7ff0917d2640 1 --2- 172.21.15.5:0/1700848061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff08c162820 0x7ff08c159be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.587+0000 7ff0917d2640 1 -- 172.21.15.5:0/1700848061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08c0c88a0 con 0x7ff08c0b8190 2024-05-20T18:27:53.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.587+0000 7ff0927d4640 1 --2- 172.21.15.5:0/1700848061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff08c162820 0x7ff08c159be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:53.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.587+0000 7ff0917d2640 1 --2- 172.21.15.5:0/1700848061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff08c0b8190 0x7ff08c0b87e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff0800098c0 tx=0x7ff0800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:53.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.587+0000 7ff08a7fc640 1 -- 172.21.15.5:0/1700848061 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff080037040 con 0x7ff08c0b8190 2024-05-20T18:27:53.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.587+0000 7ff08a7fc640 1 -- 172.21.15.5:0/1700848061 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff080042070 con 0x7ff08c0b8190 2024-05-20T18:27:53.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.587+0000 7ff094237640 1 -- 172.21.15.5:0/1700848061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff08c0c8b40 con 0x7ff08c0b8190 2024-05-20T18:27:53.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.587+0000 7ff08a7fc640 1 -- 172.21.15.5:0/1700848061 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff080030bb0 con 0x7ff08c0b8190 2024-05-20T18:27:53.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.587+0000 7ff094237640 1 -- 172.21.15.5:0/1700848061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff08c0c9100 con 0x7ff08c0b8190 2024-05-20T18:27:53.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.588+0000 7ff094237640 1 -- 172.21.15.5:0/1700848061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff08c075b00 con 0x7ff08c0b8190 2024-05-20T18:27:53.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.589+0000 7ff08a7fc640 1 -- 172.21.15.5:0/1700848061 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff080047460 con 0x7ff08c0b8190 2024-05-20T18:27:53.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.590+0000 7ff08a7fc640 1 --2- 172.21.15.5:0/1700848061 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff07806be60 0x7ff07806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:53.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.590+0000 7ff08a7fc640 1 -- 172.21.15.5:0/1700848061 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff0800abb60 con 0x7ff08c0b8190 2024-05-20T18:27:53.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.590+0000 7ff091fd3640 1 --2- 172.21.15.5:0/1700848061 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff07806be60 0x7ff07806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:53.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.590+0000 7ff091fd3640 1 --2- 172.21.15.5:0/1700848061 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff07806be60 0x7ff07806e370 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff07c007ed0 tx=0x7ff07c007e60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:53.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.594+0000 7ff08a7fc640 1 -- 172.21.15.5:0/1700848061 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff080032e20 con 0x7ff08c0b8190 2024-05-20T18:27:53.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.778+0000 7ff094237640 1 -- 172.21.15.5:0/1700848061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff08c0cb340 con 0x7ff08c0b8190 2024-05-20T18:27:53.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.778+0000 7ff08a7fc640 1 -- 172.21.15.5:0/1700848061 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7ff08007a1d0 con 0x7ff08c0b8190 2024-05-20T18:27:53.778 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:27:53.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.780+0000 7ff094237640 1 -- 172.21.15.5:0/1700848061 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff07806be60 msgr2=0x7ff07806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:53.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.780+0000 7ff094237640 1 --2- 172.21.15.5:0/1700848061 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff07806be60 0x7ff07806e370 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff07c007ed0 tx=0x7ff07c007e60 comp rx=0 tx=0).stop 2024-05-20T18:27:53.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.780+0000 7ff094237640 1 -- 172.21.15.5:0/1700848061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff08c0b8190 msgr2=0x7ff08c0b87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:53.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.781+0000 7ff094237640 1 --2- 172.21.15.5:0/1700848061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff08c0b8190 0x7ff08c0b87e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff0800098c0 tx=0x7ff0800046c0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.781+0000 7ff094237640 1 -- 172.21.15.5:0/1700848061 shutdown_connections 2024-05-20T18:27:53.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.781+0000 7ff094237640 1 --2- 172.21.15.5:0/1700848061 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff07806be60 0x7ff07806e370 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.781+0000 7ff094237640 1 --2- 172.21.15.5:0/1700848061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff08c162820 0x7ff08c159be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.781+0000 7ff094237640 1 --2- 172.21.15.5:0/1700848061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff08c0b8190 0x7ff08c0b87e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.781+0000 7ff094237640 1 --2- 172.21.15.5:0/1700848061 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff08c06d3d0 0x7ff08c0c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.781+0000 7ff094237640 1 -- 172.21.15.5:0/1700848061 >> 172.21.15.5:0/1700848061 conn(0x7ff08c0d4bd0 msgr2=0x7ff08c15fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:53.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.781+0000 7ff094237640 1 -- 172.21.15.5:0/1700848061 shutdown_connections 2024-05-20T18:27:53.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.781+0000 7ff094237640 1 -- 172.21.15.5:0/1700848061 wait complete. 2024-05-20T18:27:53.791 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:27:53.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.910+0000 7fb2d897e640 1 Processor -- start 2024-05-20T18:27:53.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.910+0000 7fb2d897e640 1 -- start start 2024-05-20T18:27:53.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.911+0000 7fb2d897e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb2d4163a10 0x7fb2d40d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:53.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.911+0000 7fb2d897e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2d40b4010 0x7fb2d40cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:53.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.911+0000 7fb2d897e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb2d40d14b0 0x7fb2d40d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:53.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.911+0000 7fb2d897e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fb2d40d3ca0 con 0x7fb2d40d14b0 2024-05-20T18:27:53.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.911+0000 7fb2d897e640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb2d40d0370 con 0x7fb2d4163a10 2024-05-20T18:27:53.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.911+0000 7fb2d897e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb2d40d0650 con 0x7fb2d40b4010 2024-05-20T18:27:53.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.911+0000 7fb2d2d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb2d40d14b0 0x7fb2d40d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:53.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.911+0000 7fb2d2d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb2d40d14b0 0x7fb2d40d18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35718/0 (socket says 172.21.15.5:35718) 2024-05-20T18:27:53.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.911+0000 7fb2d2d9d640 1 -- 172.21.15.5:0/2085084505 learned_addr learned my addr 172.21.15.5:0/2085084505 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:53.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.912+0000 7fb2d259c640 1 --2- 172.21.15.5:0/2085084505 >> v2:172.21.15.63:3300/0 conn(0x7fb2d4163a10 0x7fb2d40d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:53.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.912+0000 7fb2c9d9b640 1 --2- 172.21.15.5:0/2085084505 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2d40b4010 0x7fb2d40cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:53.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.912+0000 7fb2d2d9d640 1 -- 172.21.15.5:0/2085084505 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2d40b4010 msgr2=0x7fb2d40cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:53.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.912+0000 7fb2d2d9d640 1 --2- 172.21.15.5:0/2085084505 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2d40b4010 0x7fb2d40cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.912+0000 7fb2d2d9d640 1 -- 172.21.15.5:0/2085084505 >> v2:172.21.15.63:3300/0 conn(0x7fb2d4163a10 msgr2=0x7fb2d40d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:53.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.912+0000 7fb2d2d9d640 1 --2- 172.21.15.5:0/2085084505 >> v2:172.21.15.63:3300/0 conn(0x7fb2d4163a10 0x7fb2d40d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.912+0000 7fb2d2d9d640 1 -- 172.21.15.5:0/2085084505 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d40d3650 con 0x7fb2d40d14b0 2024-05-20T18:27:53.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.913+0000 7fb2d2d9d640 1 --2- 172.21.15.5:0/2085084505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2d40d14b0 0x7fb2d40d18c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40cf1c0 tx=0x7fb2cc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5160048028102711 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:53.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.913+0000 7fb2d1d9b640 1 -- 172.21.15.5:0/2085084505 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc036070 con 0x7fb2d40d14b0 2024-05-20T18:27:53.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.913+0000 7fb2d1d9b640 1 -- 172.21.15.5:0/2085084505 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2cc031600 con 0x7fb2d40d14b0 2024-05-20T18:27:53.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.913+0000 7fb2d1d9b640 1 -- 172.21.15.5:0/2085084505 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc0319d0 con 0x7fb2d40d14b0 2024-05-20T18:27:53.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.913+0000 7fb2d897e640 1 -- 172.21.15.5:0/2085084505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2d40d14b0 msgr2=0x7fb2d40d18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:53.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.913+0000 7fb2d897e640 1 --2- 172.21.15.5:0/2085084505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2d40d14b0 0x7fb2d40d18c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40cf1c0 tx=0x7fb2cc02a6a0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.913+0000 7fb2d897e640 1 -- 172.21.15.5:0/2085084505 shutdown_connections 2024-05-20T18:27:53.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.913+0000 7fb2d897e640 1 --2- 172.21.15.5:0/2085084505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2d40d14b0 0x7fb2d40d18c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.913+0000 7fb2d897e640 1 --2- 172.21.15.5:0/2085084505 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2d40b4010 0x7fb2d40cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.913+0000 7fb2d897e640 1 --2- 172.21.15.5:0/2085084505 >> v2:172.21.15.63:3300/0 conn(0x7fb2d4163a10 0x7fb2d40d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.913+0000 7fb2d897e640 1 -- 172.21.15.5:0/2085084505 >> 172.21.15.5:0/2085084505 conn(0x7fb2d406d240 msgr2=0x7fb2d406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:53.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.913+0000 7fb2d897e640 1 -- 172.21.15.5:0/2085084505 shutdown_connections 2024-05-20T18:27:53.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.913+0000 7fb2d897e640 1 -- 172.21.15.5:0/2085084505 wait complete. 2024-05-20T18:27:53.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.916+0000 7fb2d897e640 1 Processor -- start 2024-05-20T18:27:53.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.916+0000 7fb2d897e640 1 -- start start 2024-05-20T18:27:53.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.917+0000 7fb2d897e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2d40b4010 0x7fb2d40adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:53.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.917+0000 7fb2d897e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2d40d14b0 0x7fb2d40b3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:53.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.917+0000 7fb2d897e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2d4163a10 0x7fb2d41eac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:53.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.917+0000 7fb2d897e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d40d1e00 con 0x7fb2d40d14b0 2024-05-20T18:27:53.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.917+0000 7fb2d897e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d40d0040 con 0x7fb2d40b4010 2024-05-20T18:27:53.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.917+0000 7fb2d897e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb2d40b8210 con 0x7fb2d4163a10 2024-05-20T18:27:53.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.917+0000 7fb2c9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2d40d14b0 0x7fb2d40b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:53.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.917+0000 7fb2c9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2d40d14b0 0x7fb2d40b3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35724/0 (socket says 172.21.15.5:35724) 2024-05-20T18:27:53.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.917+0000 7fb2c9d9b640 1 -- 172.21.15.5:0/1440455665 learned_addr learned my addr 172.21.15.5:0/1440455665 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:53.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.917+0000 7fb2d259c640 1 --2- 172.21.15.5:0/1440455665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2d40b4010 0x7fb2d40adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:53.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.917+0000 7fb2d2d9d640 1 --2- 172.21.15.5:0/1440455665 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2d4163a10 0x7fb2d41eac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:53.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.917+0000 7fb2c9d9b640 1 -- 172.21.15.5:0/1440455665 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2d4163a10 msgr2=0x7fb2d41eac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:53.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.917+0000 7fb2c9d9b640 1 --2- 172.21.15.5:0/1440455665 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2d4163a10 0x7fb2d41eac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.917+0000 7fb2c9d9b640 1 -- 172.21.15.5:0/1440455665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2d40b4010 msgr2=0x7fb2d40adeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:53.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.917+0000 7fb2c9d9b640 1 --2- 172.21.15.5:0/1440455665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2d40b4010 0x7fb2d40adeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:53.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.917+0000 7fb2c9d9b640 1 -- 172.21.15.5:0/1440455665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d40ac690 con 0x7fb2d40d14b0 2024-05-20T18:27:53.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.918+0000 7fb2c9d9b640 1 --2- 172.21.15.5:0/1440455665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2d40d14b0 0x7fb2d40b3140 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00d830 tx=0x7fb2bc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:53.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.918+0000 7fb2cb7fe640 1 -- 172.21.15.5:0/1440455665 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc018030 con 0x7fb2d40d14b0 2024-05-20T18:27:53.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.918+0000 7fb2cb7fe640 1 -- 172.21.15.5:0/1440455665 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2bc00be10 con 0x7fb2d40d14b0 2024-05-20T18:27:53.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.918+0000 7fb2d897e640 1 -- 172.21.15.5:0/1440455665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2d40ac970 con 0x7fb2d40d14b0 2024-05-20T18:27:53.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.918+0000 7fb2cb7fe640 1 -- 172.21.15.5:0/1440455665 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc0098b0 con 0x7fb2d40d14b0 2024-05-20T18:27:53.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.918+0000 7fb2d897e640 1 -- 172.21.15.5:0/1440455665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2d40acf30 con 0x7fb2d40d14b0 2024-05-20T18:27:53.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.919+0000 7fb2d897e640 1 -- 172.21.15.5:0/1440455665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2d4075b00 con 0x7fb2d40d14b0 2024-05-20T18:27:53.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.919+0000 7fb2cb7fe640 1 -- 172.21.15.5:0/1440455665 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb2bc01b070 con 0x7fb2d40d14b0 2024-05-20T18:27:53.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.920+0000 7fb2cb7fe640 1 --2- 172.21.15.5:0/1440455665 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb2b406be60 0x7fb2b406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:53.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.920+0000 7fb2cb7fe640 1 -- 172.21.15.5:0/1440455665 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fb2bc08b2c0 con 0x7fb2d40d14b0 2024-05-20T18:27:53.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.920+0000 7fb2d259c640 1 --2- 172.21.15.5:0/1440455665 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb2b406be60 0x7fb2b406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:53.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.921+0000 7fb2d259c640 1 --2- 172.21.15.5:0/1440455665 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb2b406be60 0x7fb2b406e370 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40d3020 tx=0x7fb2c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:53.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:53.925+0000 7fb2cb7fe640 1 -- 172.21.15.5:0/1440455665 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb2bc059250 con 0x7fb2d40d14b0 2024-05-20T18:27:54.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.107+0000 7fb2d897e640 1 -- 172.21.15.5:0/1440455665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb2d40ac420 con 0x7fb2d40d14b0 2024-05-20T18:27:54.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.108+0000 7fb2cb7fe640 1 -- 172.21.15.5:0/1440455665 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fb2bc05d550 con 0x7fb2d40d14b0 2024-05-20T18:27:54.107 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:27:54.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.110+0000 7fb2d897e640 1 -- 172.21.15.5:0/1440455665 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb2b406be60 msgr2=0x7fb2b406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:54.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.110+0000 7fb2d897e640 1 --2- 172.21.15.5:0/1440455665 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb2b406be60 0x7fb2b406e370 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40d3020 tx=0x7fb2c4008040 comp rx=0 tx=0).stop 2024-05-20T18:27:54.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.110+0000 7fb2d897e640 1 -- 172.21.15.5:0/1440455665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2d40d14b0 msgr2=0x7fb2d40b3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:54.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.110+0000 7fb2d897e640 1 --2- 172.21.15.5:0/1440455665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2d40d14b0 0x7fb2d40b3140 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00d830 tx=0x7fb2bc00dcf0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.110+0000 7fb2d897e640 1 -- 172.21.15.5:0/1440455665 shutdown_connections 2024-05-20T18:27:54.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.110+0000 7fb2d897e640 1 --2- 172.21.15.5:0/1440455665 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb2b406be60 0x7fb2b406e370 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.110+0000 7fb2d897e640 1 --2- 172.21.15.5:0/1440455665 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2d4163a10 0x7fb2d41eac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.110+0000 7fb2d897e640 1 --2- 172.21.15.5:0/1440455665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2d40d14b0 0x7fb2d40b3140 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.110+0000 7fb2d897e640 1 --2- 172.21.15.5:0/1440455665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2d40b4010 0x7fb2d40adeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.110+0000 7fb2d897e640 1 -- 172.21.15.5:0/1440455665 >> 172.21.15.5:0/1440455665 conn(0x7fb2d406d240 msgr2=0x7fb2d40ba310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:54.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.110+0000 7fb2d897e640 1 -- 172.21.15.5:0/1440455665 shutdown_connections 2024-05-20T18:27:54.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.110+0000 7fb2d897e640 1 -- 172.21.15.5:0/1440455665 wait complete. 2024-05-20T18:27:54.121 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:27:54.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.240+0000 7f0e5a456640 1 Processor -- start 2024-05-20T18:27:54.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.240+0000 7f0e5a456640 1 -- start start 2024-05-20T18:27:54.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.242+0000 7f0e5a456640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e540cf250 0x7f0e5415f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:54.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.242+0000 7f0e5a456640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0e540b6680 0x7f0e540b6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:54.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.242+0000 7f0e5a456640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e540b6fd0 con 0x7f0e54163020 2024-05-20T18:27:54.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.242+0000 7f0e5a456640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0e5415f9a0 con 0x7f0e540b6680 2024-05-20T18:27:54.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.242+0000 7f0e5a456640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0e540d0b10 con 0x7f0e540cf250 2024-05-20T18:27:54.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.242+0000 7f0e589f3640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0e54163020 0x7f0e54163430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59272/0 (socket says 172.21.15.5:59272) 2024-05-20T18:27:54.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.242+0000 7f0e589f3640 1 -- 172.21.15.5:0/1902462069 learned_addr learned my addr 172.21.15.5:0/1902462069 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:54.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.242+0000 7f0e537fe640 1 --2- 172.21.15.5:0/1902462069 >> v2:172.21.15.63:3300/0 conn(0x7f0e540b6680 0x7f0e540b6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:54.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.242+0000 7f0e53fff640 1 --2- 172.21.15.5:0/1902462069 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e540cf250 0x7f0e5415f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:54.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.242+0000 7f0e52ffd640 1 -- 172.21.15.5:0/1902462069 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 245560898 0 0) 0x7f0e4c001490 con 0x7f0e54163020 2024-05-20T18:27:54.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.242+0000 7f0e52ffd640 1 -- 172.21.15.5:0/1902462069 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e3c003520 con 0x7f0e54163020 2024-05-20T18:27:54.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.243+0000 7f0e52ffd640 1 -- 172.21.15.5:0/1902462069 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3378071855 0 0) 0x7f0e3c003520 con 0x7f0e54163020 2024-05-20T18:27:54.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.243+0000 7f0e52ffd640 1 -- 172.21.15.5:0/1902462069 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e540cf250 msgr2=0x7f0e5415f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:54.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.243+0000 7f0e52ffd640 1 --2- 172.21.15.5:0/1902462069 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e540cf250 0x7f0e5415f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.243+0000 7f0e52ffd640 1 -- 172.21.15.5:0/1902462069 >> v2:172.21.15.63:3300/0 conn(0x7f0e540b6680 msgr2=0x7f0e540b6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:54.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.243+0000 7f0e52ffd640 1 --2- 172.21.15.5:0/1902462069 >> v2:172.21.15.63:3300/0 conn(0x7f0e540b6680 0x7f0e540b6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.243+0000 7f0e52ffd640 1 -- 172.21.15.5:0/1902462069 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e540d0550 con 0x7f0e54163020 2024-05-20T18:27:54.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.243+0000 7f0e52ffd640 1 -- 172.21.15.5:0/1902462069 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f0e4c001bc0 con 0x7f0e54163020 2024-05-20T18:27:54.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.243+0000 7f0e52ffd640 1 -- 172.21.15.5:0/1902462069 >> v1:172.21.15.5:6789/0 conn(0x7f0e54163020 legacy=0x7f0e54163430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:54.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.243+0000 7f0e52ffd640 1 --2- 172.21.15.5:0/1902462069 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e540cf250 0x7f0e3c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:54.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.243+0000 7f0e52ffd640 1 --2- 172.21.15.5:0/1902462069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e3c003f30 0x7f0e3c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:54.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.243+0000 7f0e52ffd640 1 --2- 172.21.15.5:0/1902462069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e3c004d40 0x7f0e3c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:54.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.243+0000 7f0e52ffd640 1 -- 172.21.15.5:0/1902462069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5415f9a0 con 0x7f0e3c003f30 2024-05-20T18:27:54.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.243+0000 7f0e52ffd640 1 -- 172.21.15.5:0/1902462069 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c00ba70 con 0x7f0e3c004d40 2024-05-20T18:27:54.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.243+0000 7f0e52ffd640 1 -- 172.21.15.5:0/1902462069 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0e3c00bc40 con 0x7f0e540cf250 2024-05-20T18:27:54.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.243+0000 7f0e537fe640 1 --2- 172.21.15.5:0/1902462069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e3c003f30 0x7f0e3c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:54.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.243+0000 7f0e53fff640 1 --2- 172.21.15.5:0/1902462069 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e540cf250 0x7f0e3c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:54.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.243+0000 7f0e589f3640 1 --2- 172.21.15.5:0/1902462069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e3c004d40 0x7f0e3c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:54.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.243+0000 7f0e537fe640 1 -- 172.21.15.5:0/1902462069 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e540cf250 msgr2=0x7f0e3c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:54.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.243+0000 7f0e537fe640 1 --2- 172.21.15.5:0/1902462069 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e540cf250 0x7f0e3c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.243+0000 7f0e537fe640 1 -- 172.21.15.5:0/1902462069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e3c004d40 msgr2=0x7f0e3c005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:54.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.243+0000 7f0e537fe640 1 --2- 172.21.15.5:0/1902462069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e3c004d40 0x7f0e3c005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.243+0000 7f0e537fe640 1 -- 172.21.15.5:0/1902462069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e3c003520 con 0x7f0e3c003f30 2024-05-20T18:27:54.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.244+0000 7f0e537fe640 1 --2- 172.21.15.5:0/1902462069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e3c003f30 0x7f0e3c004800 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009ac0 tx=0x7f0e3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=768d391a090c915d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:54.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.244+0000 7f0e52ffd640 1 -- 172.21.15.5:0/1902462069 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e48036070 con 0x7f0e3c003f30 2024-05-20T18:27:54.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.244+0000 7f0e52ffd640 1 -- 172.21.15.5:0/1902462069 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e48031400 con 0x7f0e3c003f30 2024-05-20T18:27:54.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.244+0000 7f0e52ffd640 1 -- 172.21.15.5:0/1902462069 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e48009690 con 0x7f0e3c003f30 2024-05-20T18:27:54.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.245+0000 7f0e5a456640 1 -- 172.21.15.5:0/1902462069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e3c003f30 msgr2=0x7f0e3c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:54.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.245+0000 7f0e5a456640 1 --2- 172.21.15.5:0/1902462069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e3c003f30 0x7f0e3c004800 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009ac0 tx=0x7f0e3c005830 comp rx=0 tx=0).stop 2024-05-20T18:27:54.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.245+0000 7f0e589f3640 1 -- 172.21.15.5:0/1902462069 reap_dead start 2024-05-20T18:27:54.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.245+0000 7f0e5a456640 1 -- 172.21.15.5:0/1902462069 shutdown_connections 2024-05-20T18:27:54.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.245+0000 7f0e5a456640 1 -- 172.21.15.5:0/1902462069 >> 172.21.15.5:0/1902462069 conn(0x7f0e540d1860 msgr2=0x7f0e540d1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:54.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.245+0000 7f0e5a456640 1 -- 172.21.15.5:0/1902462069 shutdown_connections 2024-05-20T18:27:54.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.245+0000 7f0e5a456640 1 -- 172.21.15.5:0/1902462069 wait complete. 2024-05-20T18:27:54.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.247+0000 7f0e5a456640 1 Processor -- start 2024-05-20T18:27:54.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.247+0000 7f0e5a456640 1 -- start start 2024-05-20T18:27:54.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.248+0000 7f0e5a456640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e3c003f30 0x7f0e540cd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:54.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.248+0000 7f0e5a456640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e540ca4d0 0x7f0e540c8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.248+0000 7f0e5a456640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e541f8670 0x7f0e5415cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.248+0000 7f0e5a456640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0e540d2a20 con 0x7f0e540ca4d0 2024-05-20T18:27:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.248+0000 7f0e5a456640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0e540d27d0 con 0x7f0e541f8670 2024-05-20T18:27:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.248+0000 7f0e5a456640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0e540d2d00 con 0x7f0e3c003f30 2024-05-20T18:27:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.248+0000 7f0e537fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e540ca4d0 0x7f0e540c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.248+0000 7f0e537fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e540ca4d0 0x7f0e540c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35744/0 (socket says 172.21.15.5:35744) 2024-05-20T18:27:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.248+0000 7f0e537fe640 1 -- 172.21.15.5:0/641917216 learned_addr learned my addr 172.21.15.5:0/641917216 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.248+0000 7f0e589f3640 1 --2- 172.21.15.5:0/641917216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e541f8670 0x7f0e5415cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.248+0000 7f0e53fff640 1 --2- 172.21.15.5:0/641917216 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e3c003f30 0x7f0e540cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.249+0000 7f0e537fe640 1 -- 172.21.15.5:0/641917216 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e3c003f30 msgr2=0x7f0e540cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.249+0000 7f0e537fe640 1 --2- 172.21.15.5:0/641917216 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e3c003f30 0x7f0e540cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.249+0000 7f0e537fe640 1 -- 172.21.15.5:0/641917216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e541f8670 msgr2=0x7f0e5415cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:54.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.249+0000 7f0e537fe640 1 --2- 172.21.15.5:0/641917216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e541f8670 0x7f0e5415cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.249+0000 7f0e537fe640 1 -- 172.21.15.5:0/641917216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e540c8f60 con 0x7f0e540ca4d0 2024-05-20T18:27:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.249+0000 7f0e537fe640 1 --2- 172.21.15.5:0/641917216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e540ca4d0 0x7f0e540c8710 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0e48034040 tx=0x7f0e48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.249+0000 7f0e50ff9640 1 -- 172.21.15.5:0/641917216 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e48040020 con 0x7f0e540ca4d0 2024-05-20T18:27:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.249+0000 7f0e50ff9640 1 -- 172.21.15.5:0/641917216 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e48003c60 con 0x7f0e540ca4d0 2024-05-20T18:27:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.249+0000 7f0e50ff9640 1 -- 172.21.15.5:0/641917216 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e48036040 con 0x7f0e540ca4d0 2024-05-20T18:27:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.249+0000 7f0e5a456640 1 -- 172.21.15.5:0/641917216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e541f75d0 con 0x7f0e540ca4d0 2024-05-20T18:27:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.249+0000 7f0e5a456640 1 -- 172.21.15.5:0/641917216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e541f7b90 con 0x7f0e540ca4d0 2024-05-20T18:27:54.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.250+0000 7f0e5a456640 1 -- 172.21.15.5:0/641917216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e54075b00 con 0x7f0e540ca4d0 2024-05-20T18:27:54.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.251+0000 7f0e50ff9640 1 -- 172.21.15.5:0/641917216 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0e48043020 con 0x7f0e540ca4d0 2024-05-20T18:27:54.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.251+0000 7f0e50ff9640 1 --2- 172.21.15.5:0/641917216 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0e4006be30 0x7f0e4006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:54.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.252+0000 7f0e50ff9640 1 -- 172.21.15.5:0/641917216 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0e480ab710 con 0x7f0e540ca4d0 2024-05-20T18:27:54.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.252+0000 7f0e53fff640 1 --2- 172.21.15.5:0/641917216 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0e4006be30 0x7f0e4006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:54.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.252+0000 7f0e53fff640 1 --2- 172.21.15.5:0/641917216 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0e4006be30 0x7f0e4006e340 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009b20 tx=0x7f0e44009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:54.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.256+0000 7f0e50ff9640 1 -- 172.21.15.5:0/641917216 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0e4804a030 con 0x7f0e540ca4d0 2024-05-20T18:27:54.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.438+0000 7f0e5a456640 1 -- 172.21.15.5:0/641917216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0e540b7440 con 0x7f0e540ca4d0 2024-05-20T18:27:54.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.438+0000 7f0e50ff9640 1 -- 172.21.15.5:0/641917216 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f0e480044e0 con 0x7f0e540ca4d0 2024-05-20T18:27:54.438 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:27:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.440+0000 7f0e5a456640 1 -- 172.21.15.5:0/641917216 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0e4006be30 msgr2=0x7f0e4006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.440+0000 7f0e5a456640 1 --2- 172.21.15.5:0/641917216 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0e4006be30 0x7f0e4006e340 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009b20 tx=0x7f0e44009190 comp rx=0 tx=0).stop 2024-05-20T18:27:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.440+0000 7f0e5a456640 1 -- 172.21.15.5:0/641917216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e540ca4d0 msgr2=0x7f0e540c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.440+0000 7f0e5a456640 1 --2- 172.21.15.5:0/641917216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e540ca4d0 0x7f0e540c8710 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0e48034040 tx=0x7f0e48004730 comp rx=0 tx=0).stop 2024-05-20T18:27:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.440+0000 7f0e5a456640 1 -- 172.21.15.5:0/641917216 shutdown_connections 2024-05-20T18:27:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.440+0000 7f0e5a456640 1 --2- 172.21.15.5:0/641917216 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0e4006be30 0x7f0e4006e340 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.440+0000 7f0e5a456640 1 --2- 172.21.15.5:0/641917216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e541f8670 0x7f0e5415cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.440+0000 7f0e5a456640 1 --2- 172.21.15.5:0/641917216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e540ca4d0 0x7f0e540c8710 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.440+0000 7f0e5a456640 1 --2- 172.21.15.5:0/641917216 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e3c003f30 0x7f0e540cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.440+0000 7f0e5a456640 1 -- 172.21.15.5:0/641917216 >> 172.21.15.5:0/641917216 conn(0x7f0e540d1860 msgr2=0x7f0e541624d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.440+0000 7f0e5a456640 1 -- 172.21.15.5:0/641917216 shutdown_connections 2024-05-20T18:27:54.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.440+0000 7f0e5a456640 1 -- 172.21.15.5:0/641917216 wait complete. 2024-05-20T18:27:54.451 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T18:27:54.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.570+0000 7fed557b7640 1 Processor -- start 2024-05-20T18:27:54.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.570+0000 7fed557b7640 1 -- start start 2024-05-20T18:27:54.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.571+0000 7fed557b7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fed5015d530 0x7fed5015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:54.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.571+0000 7fed557b7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed500b1700 0x7fed500b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:54.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.571+0000 7fed557b7640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed500cd580 con 0x7fed50070850 2024-05-20T18:27:54.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.571+0000 7fed557b7640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fed500ce190 con 0x7fed5015d530 2024-05-20T18:27:54.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.571+0000 7fed557b7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fed500cd860 con 0x7fed500b1700 2024-05-20T18:27:54.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.571+0000 7fed4f7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fed50070850 0x7fed50070c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59280/0 (socket says 172.21.15.5:59280) 2024-05-20T18:27:54.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.571+0000 7fed4f7fe640 1 -- 172.21.15.5:0/868265794 learned_addr learned my addr 172.21.15.5:0/868265794 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:54.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.571+0000 7fed4effd640 1 --2- 172.21.15.5:0/868265794 >> v2:172.21.15.63:3300/0 conn(0x7fed5015d530 0x7fed5015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:54.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.571+0000 7fed4e7fc640 1 --2- 172.21.15.5:0/868265794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed500b1700 0x7fed500b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:54.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.571+0000 7fed4dffb640 1 -- 172.21.15.5:0/868265794 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4233643291 0 0) 0x7fed500cd580 con 0x7fed50070850 2024-05-20T18:27:54.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.572+0000 7fed4dffb640 1 -- 172.21.15.5:0/868265794 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed3c003520 con 0x7fed50070850 2024-05-20T18:27:54.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.572+0000 7fed4effd640 1 -- 172.21.15.5:0/868265794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed500b1700 msgr2=0x7fed500b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:54.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.572+0000 7fed4effd640 1 --2- 172.21.15.5:0/868265794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed500b1700 0x7fed500b1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.572+0000 7fed4effd640 1 -- 172.21.15.5:0/868265794 >> v1:172.21.15.5:6789/0 conn(0x7fed50070850 legacy=0x7fed50070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:54.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.572+0000 7fed4effd640 1 -- 172.21.15.5:0/868265794 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed500cdb40 con 0x7fed5015d530 2024-05-20T18:27:54.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.572+0000 7fed4effd640 1 --2- 172.21.15.5:0/868265794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed5015d530 0x7fed5015aaa0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fed48009d90 tx=0x7fed4802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6d13290a55c3dc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:54.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.573+0000 7fed4dffb640 1 -- 172.21.15.5:0/868265794 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed48007740 con 0x7fed5015d530 2024-05-20T18:27:54.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.573+0000 7fed4dffb640 1 -- 172.21.15.5:0/868265794 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed480078e0 con 0x7fed5015d530 2024-05-20T18:27:54.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.573+0000 7fed4dffb640 1 -- 172.21.15.5:0/868265794 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed48007c90 con 0x7fed5015d530 2024-05-20T18:27:54.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.573+0000 7fed557b7640 1 -- 172.21.15.5:0/868265794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed5015d530 msgr2=0x7fed5015aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:54.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.573+0000 7fed557b7640 1 --2- 172.21.15.5:0/868265794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed5015d530 0x7fed5015aaa0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fed48009d90 tx=0x7fed4802a660 comp rx=0 tx=0).stop 2024-05-20T18:27:54.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.573+0000 7fed557b7640 1 -- 172.21.15.5:0/868265794 shutdown_connections 2024-05-20T18:27:54.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.573+0000 7fed557b7640 1 --2- 172.21.15.5:0/868265794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed500b1700 0x7fed500b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.573+0000 7fed557b7640 1 --2- 172.21.15.5:0/868265794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed5015d530 0x7fed5015aaa0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.573+0000 7fed557b7640 1 -- 172.21.15.5:0/868265794 >> 172.21.15.5:0/868265794 conn(0x7fed500d29d0 msgr2=0x7fed500d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:54.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.573+0000 7fed557b7640 1 -- 172.21.15.5:0/868265794 shutdown_connections 2024-05-20T18:27:54.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.573+0000 7fed557b7640 1 -- 172.21.15.5:0/868265794 wait complete. 2024-05-20T18:27:54.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.575+0000 7fed557b7640 1 Processor -- start 2024-05-20T18:27:54.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.575+0000 7fed557b7640 1 -- start start 2024-05-20T18:27:54.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.577+0000 7fed557b7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed50070850 0x7fed501550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:54.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.577+0000 7fed557b7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed500b1700 0x7fed500ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:54.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.577+0000 7fed557b7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed5015d530 0x7fed500b6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:54.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.577+0000 7fed557b7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fed500b2050 con 0x7fed50070850 2024-05-20T18:27:54.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.577+0000 7fed557b7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fed500cd230 con 0x7fed500b1700 2024-05-20T18:27:54.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.577+0000 7fed557b7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fed50071370 con 0x7fed5015d530 2024-05-20T18:27:54.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.577+0000 7fed4effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed50070850 0x7fed501550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:54.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.577+0000 7fed4effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed50070850 0x7fed501550e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35754/0 (socket says 172.21.15.5:35754) 2024-05-20T18:27:54.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.577+0000 7fed4effd640 1 -- 172.21.15.5:0/1582822363 learned_addr learned my addr 172.21.15.5:0/1582822363 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:54.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.577+0000 7fed4e7fc640 1 --2- 172.21.15.5:0/1582822363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed500b1700 0x7fed500ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:54.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.577+0000 7fed4f7fe640 1 --2- 172.21.15.5:0/1582822363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed5015d530 0x7fed500b6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:54.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.577+0000 7fed4effd640 1 -- 172.21.15.5:0/1582822363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed5015d530 msgr2=0x7fed500b6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:54.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.577+0000 7fed4effd640 1 --2- 172.21.15.5:0/1582822363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed5015d530 0x7fed500b6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.577+0000 7fed4effd640 1 -- 172.21.15.5:0/1582822363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed500b1700 msgr2=0x7fed500ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:54.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.577+0000 7fed4effd640 1 --2- 172.21.15.5:0/1582822363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed500b1700 0x7fed500ca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.577+0000 7fed4effd640 1 -- 172.21.15.5:0/1582822363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed500cae00 con 0x7fed50070850 2024-05-20T18:27:54.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.577+0000 7fed4f7fe640 1 --2- 172.21.15.5:0/1582822363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed5015d530 0x7fed500b6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:54.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.577+0000 7fed4effd640 1 --2- 172.21.15.5:0/1582822363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed50070850 0x7fed501550e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fed4802ac50 tx=0x7fed48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:54.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.578+0000 7fed477fe640 1 -- 172.21.15.5:0/1582822363 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4803f070 con 0x7fed50070850 2024-05-20T18:27:54.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.578+0000 7fed477fe640 1 -- 172.21.15.5:0/1582822363 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed48031c40 con 0x7fed50070850 2024-05-20T18:27:54.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.578+0000 7fed557b7640 1 -- 172.21.15.5:0/1582822363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed500cb0e0 con 0x7fed50070850 2024-05-20T18:27:54.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.578+0000 7fed477fe640 1 -- 172.21.15.5:0/1582822363 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed48009450 con 0x7fed50070850 2024-05-20T18:27:54.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.578+0000 7fed557b7640 1 -- 172.21.15.5:0/1582822363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed5006e030 con 0x7fed50070850 2024-05-20T18:27:54.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.579+0000 7fed557b7640 1 -- 172.21.15.5:0/1582822363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed50075b00 con 0x7fed50070850 2024-05-20T18:27:54.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.579+0000 7fed477fe640 1 -- 172.21.15.5:0/1582822363 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fed4803b410 con 0x7fed50070850 2024-05-20T18:27:54.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.580+0000 7fed477fe640 1 --2- 172.21.15.5:0/1582822363 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fed3006be30 0x7fed3006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:54.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.580+0000 7fed4e7fc640 1 --2- 172.21.15.5:0/1582822363 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fed3006be30 0x7fed3006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:54.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.580+0000 7fed477fe640 1 -- 172.21.15.5:0/1582822363 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fed480ac980 con 0x7fed50070850 2024-05-20T18:27:54.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.581+0000 7fed4e7fc640 1 --2- 172.21.15.5:0/1582822363 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fed3006be30 0x7fed3006e340 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fed5015f040 tx=0x7fed38009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:54.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.585+0000 7fed477fe640 1 -- 172.21.15.5:0/1582822363 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fed4807aec0 con 0x7fed50070850 2024-05-20T18:27:54.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.766+0000 7fed557b7640 1 -- 172.21.15.5:0/1582822363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fed500ce930 con 0x7fed50070850 2024-05-20T18:27:54.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.767+0000 7fed477fe640 1 -- 172.21.15.5:0/1582822363 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fed4807a7c0 con 0x7fed50070850 2024-05-20T18:27:54.766 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:27:54.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.768+0000 7fed557b7640 1 -- 172.21.15.5:0/1582822363 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fed3006be30 msgr2=0x7fed3006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:54.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.768+0000 7fed557b7640 1 --2- 172.21.15.5:0/1582822363 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fed3006be30 0x7fed3006e340 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fed5015f040 tx=0x7fed38009190 comp rx=0 tx=0).stop 2024-05-20T18:27:54.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.768+0000 7fed557b7640 1 -- 172.21.15.5:0/1582822363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed50070850 msgr2=0x7fed501550e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:54.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.768+0000 7fed557b7640 1 --2- 172.21.15.5:0/1582822363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed50070850 0x7fed501550e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fed4802ac50 tx=0x7fed48004a40 comp rx=0 tx=0).stop 2024-05-20T18:27:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.768+0000 7fed557b7640 1 -- 172.21.15.5:0/1582822363 shutdown_connections 2024-05-20T18:27:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.768+0000 7fed557b7640 1 --2- 172.21.15.5:0/1582822363 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fed3006be30 0x7fed3006e340 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.768+0000 7fed557b7640 1 --2- 172.21.15.5:0/1582822363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed5015d530 0x7fed500b6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.768+0000 7fed557b7640 1 --2- 172.21.15.5:0/1582822363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed500b1700 0x7fed500ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.768+0000 7fed557b7640 1 --2- 172.21.15.5:0/1582822363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed50070850 0x7fed501550e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.768+0000 7fed557b7640 1 -- 172.21.15.5:0/1582822363 >> 172.21.15.5:0/1582822363 conn(0x7fed500d29d0 msgr2=0x7fed500c5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.769+0000 7fed557b7640 1 -- 172.21.15.5:0/1582822363 shutdown_connections 2024-05-20T18:27:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.769+0000 7fed557b7640 1 -- 172.21.15.5:0/1582822363 wait complete. 2024-05-20T18:27:54.779 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:27:54.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.896+0000 7f98ec177640 1 Processor -- start 2024-05-20T18:27:54.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.896+0000 7f98ec177640 1 -- start start 2024-05-20T18:27:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.898+0000 7f98ec177640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f98e415d5f0 0x7f98e41576d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.898+0000 7f98ec177640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98e406c9a0 0x7f98e40ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.898+0000 7f98ec177640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f98e40cec00 0x7f98e406e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.898+0000 7f98ec177640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f98e406b900 con 0x7f98e40cec00 2024-05-20T18:27:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.898+0000 7f98ec177640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f98e406bbe0 con 0x7f98e415d5f0 2024-05-20T18:27:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.898+0000 7f98ec177640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f98e406bec0 con 0x7f98e406c9a0 2024-05-20T18:27:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.898+0000 7f98ea714640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f98e40cec00 0x7f98e406e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.898+0000 7f98ea714640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f98e40cec00 0x7f98e406e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50920/0 (socket says 172.21.15.5:50920) 2024-05-20T18:27:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.898+0000 7f98ea714640 1 -- 172.21.15.5:0/3882281576 learned_addr learned my addr 172.21.15.5:0/3882281576 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.898+0000 7f98e9f13640 1 --2- 172.21.15.5:0/3882281576 >> v2:172.21.15.63:3300/0 conn(0x7f98e415d5f0 0x7f98e41576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.898+0000 7f98e9712640 1 --2- 172.21.15.5:0/3882281576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98e406c9a0 0x7f98e40ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.898+0000 7f98ea714640 1 -- 172.21.15.5:0/3882281576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98e406c9a0 msgr2=0x7f98e40ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.898+0000 7f98ea714640 1 --2- 172.21.15.5:0/3882281576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98e406c9a0 0x7f98e40ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.898+0000 7f98ea714640 1 -- 172.21.15.5:0/3882281576 >> v2:172.21.15.63:3300/0 conn(0x7f98e415d5f0 msgr2=0x7f98e41576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.898+0000 7f98ea714640 1 --2- 172.21.15.5:0/3882281576 >> v2:172.21.15.63:3300/0 conn(0x7f98e415d5f0 0x7f98e41576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.898+0000 7f98ea714640 1 -- 172.21.15.5:0/3882281576 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e406c1a0 con 0x7f98e40cec00 2024-05-20T18:27:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.898+0000 7f98e9712640 1 --2- 172.21.15.5:0/3882281576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98e406c9a0 0x7f98e40ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.898+0000 7f98e9f13640 1 --2- 172.21.15.5:0/3882281576 >> v2:172.21.15.63:3300/0 conn(0x7f98e415d5f0 0x7f98e41576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.899+0000 7f98ea714640 1 --2- 172.21.15.5:0/3882281576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98e40cec00 0x7f98e406e3a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f98e406d110 tx=0x7f98dc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a026890e03487983 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.899+0000 7f98e8f11640 1 -- 172.21.15.5:0/3882281576 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc036070 con 0x7f98e40cec00 2024-05-20T18:27:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.899+0000 7f98e8f11640 1 -- 172.21.15.5:0/3882281576 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98dc031600 con 0x7f98e40cec00 2024-05-20T18:27:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.899+0000 7f98e8f11640 1 -- 172.21.15.5:0/3882281576 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc0319d0 con 0x7f98e40cec00 2024-05-20T18:27:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.900+0000 7f98ec177640 1 -- 172.21.15.5:0/3882281576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98e40cec00 msgr2=0x7f98e406e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.900+0000 7f98ec177640 1 --2- 172.21.15.5:0/3882281576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98e40cec00 0x7f98e406e3a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f98e406d110 tx=0x7f98dc02a610 comp rx=0 tx=0).stop 2024-05-20T18:27:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.900+0000 7f98ec177640 1 -- 172.21.15.5:0/3882281576 shutdown_connections 2024-05-20T18:27:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.900+0000 7f98ec177640 1 --2- 172.21.15.5:0/3882281576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98e40cec00 0x7f98e406e3a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.900+0000 7f98ec177640 1 --2- 172.21.15.5:0/3882281576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98e406c9a0 0x7f98e40ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.900+0000 7f98ec177640 1 --2- 172.21.15.5:0/3882281576 >> v2:172.21.15.63:3300/0 conn(0x7f98e415d5f0 0x7f98e41576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.900+0000 7f98ec177640 1 -- 172.21.15.5:0/3882281576 >> 172.21.15.5:0/3882281576 conn(0x7f98e40cd2a0 msgr2=0x7f98e40cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.900+0000 7f98ec177640 1 -- 172.21.15.5:0/3882281576 shutdown_connections 2024-05-20T18:27:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.900+0000 7f98ec177640 1 -- 172.21.15.5:0/3882281576 wait complete. 2024-05-20T18:27:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.902+0000 7f98ec177640 1 Processor -- start 2024-05-20T18:27:54.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.902+0000 7f98ec177640 1 -- start start 2024-05-20T18:27:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.903+0000 7f98ec177640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98e406c9a0 0x7f98e415e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.903+0000 7f98ec177640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98e40cec00 0x7f98e4155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.903+0000 7f98ec177640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98e415d5f0 0x7f98e4142dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.903+0000 7f98ec177640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f98e406b520 con 0x7f98e415d5f0 2024-05-20T18:27:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.903+0000 7f98ec177640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f98e406cdb0 con 0x7f98e40cec00 2024-05-20T18:27:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.903+0000 7f98ec177640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f98e406b6a0 con 0x7f98e406c9a0 2024-05-20T18:27:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.903+0000 7f98ea714640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98e415d5f0 0x7f98e4142dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.903+0000 7f98ea714640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98e415d5f0 0x7f98e4142dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50922/0 (socket says 172.21.15.5:50922) 2024-05-20T18:27:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.903+0000 7f98ea714640 1 -- 172.21.15.5:0/38778214 learned_addr learned my addr 172.21.15.5:0/38778214 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.903+0000 7f98e9712640 1 --2- 172.21.15.5:0/38778214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98e40cec00 0x7f98e4155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.903+0000 7f98e9f13640 1 --2- 172.21.15.5:0/38778214 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98e406c9a0 0x7f98e415e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.903+0000 7f98ea714640 1 -- 172.21.15.5:0/38778214 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98e406c9a0 msgr2=0x7f98e415e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.903+0000 7f98ea714640 1 --2- 172.21.15.5:0/38778214 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98e406c9a0 0x7f98e415e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.903+0000 7f98ea714640 1 -- 172.21.15.5:0/38778214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98e40cec00 msgr2=0x7f98e4155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.903+0000 7f98ea714640 1 --2- 172.21.15.5:0/38778214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98e40cec00 0x7f98e4155da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.903+0000 7f98ea714640 1 -- 172.21.15.5:0/38778214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e41e6b50 con 0x7f98e415d5f0 2024-05-20T18:27:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.904+0000 7f98ea714640 1 --2- 172.21.15.5:0/38778214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98e415d5f0 0x7f98e4142dd0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009f40 tx=0x7f98dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.904+0000 7f98da7fc640 1 -- 172.21.15.5:0/38778214 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc036050 con 0x7f98e415d5f0 2024-05-20T18:27:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.904+0000 7f98da7fc640 1 -- 172.21.15.5:0/38778214 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98dc042070 con 0x7f98e415d5f0 2024-05-20T18:27:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.904+0000 7f98ec177640 1 -- 172.21.15.5:0/38778214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e41e6e30 con 0x7f98e415d5f0 2024-05-20T18:27:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.904+0000 7f98da7fc640 1 -- 172.21.15.5:0/38778214 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc009510 con 0x7f98e415d5f0 2024-05-20T18:27:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.904+0000 7f98ec177640 1 -- 172.21.15.5:0/38778214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e41e7910 con 0x7f98e415d5f0 2024-05-20T18:27:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.905+0000 7f98ec177640 1 -- 172.21.15.5:0/38778214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98e4156790 con 0x7f98e415d5f0 2024-05-20T18:27:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.906+0000 7f98da7fc640 1 -- 172.21.15.5:0/38778214 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f98dc007bd0 con 0x7f98e415d5f0 2024-05-20T18:27:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.906+0000 7f98da7fc640 1 --2- 172.21.15.5:0/38778214 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f98c806be30 0x7f98c806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.906+0000 7f98e9f13640 1 --2- 172.21.15.5:0/38778214 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f98c806be30 0x7f98c806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.906+0000 7f98da7fc640 1 -- 172.21.15.5:0/38778214 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f98dc0abf80 con 0x7f98e415d5f0 2024-05-20T18:27:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.907+0000 7f98e9f13640 1 --2- 172.21.15.5:0/38778214 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f98c806be30 0x7f98c806e340 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f98e40b0490 tx=0x7f98e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:54.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:54.911+0000 7f98da7fc640 1 -- 172.21.15.5:0/38778214 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f98dc079fa0 con 0x7f98e415d5f0 2024-05-20T18:27:55.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.095+0000 7f98ec177640 1 -- 172.21.15.5:0/38778214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f98e4075b00 con 0x7f98e415d5f0 2024-05-20T18:27:55.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.097+0000 7f98da7fc640 1 -- 172.21.15.5:0/38778214 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18394 (secure 0 0 0) 0x7f98dc07e2a0 con 0x7f98e415d5f0 2024-05-20T18:27:55.096 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:55.096 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:27:13.817633+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:27:05.476596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T18:27:06.137762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T18:27:09.829920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:27:55.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.099+0000 7f98ec177640 1 -- 172.21.15.5:0/38778214 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f98c806be30 msgr2=0x7f98c806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:55.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.099+0000 7f98ec177640 1 --2- 172.21.15.5:0/38778214 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f98c806be30 0x7f98c806e340 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f98e40b0490 tx=0x7f98e0008040 comp rx=0 tx=0).stop 2024-05-20T18:27:55.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.099+0000 7f98ec177640 1 -- 172.21.15.5:0/38778214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98e415d5f0 msgr2=0x7f98e4142dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:55.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.099+0000 7f98ec177640 1 --2- 172.21.15.5:0/38778214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98e415d5f0 0x7f98e4142dd0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009f40 tx=0x7f98dc004a40 comp rx=0 tx=0).stop 2024-05-20T18:27:55.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.099+0000 7f98ec177640 1 -- 172.21.15.5:0/38778214 shutdown_connections 2024-05-20T18:27:55.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.099+0000 7f98ec177640 1 --2- 172.21.15.5:0/38778214 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f98c806be30 0x7f98c806e340 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:55.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.099+0000 7f98ec177640 1 --2- 172.21.15.5:0/38778214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98e415d5f0 0x7f98e4142dd0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:55.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.099+0000 7f98ec177640 1 --2- 172.21.15.5:0/38778214 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98e40cec00 0x7f98e4155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:55.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.099+0000 7f98ec177640 1 --2- 172.21.15.5:0/38778214 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98e406c9a0 0x7f98e415e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:55.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.099+0000 7f98ec177640 1 -- 172.21.15.5:0/38778214 >> 172.21.15.5:0/38778214 conn(0x7f98e40cd2a0 msgr2=0x7f98e4154510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:55.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.099+0000 7f98ec177640 1 -- 172.21.15.5:0/38778214 shutdown_connections 2024-05-20T18:27:55.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.099+0000 7f98ec177640 1 -- 172.21.15.5:0/38778214 wait complete. 2024-05-20T18:27:55.110 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:27:05.476596+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:27:06.137762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T18:27:09.829920+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T18:27:55.111 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:27:55.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.235+0000 7f62d2c3d640 1 Processor -- start 2024-05-20T18:27:55.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.235+0000 7f62d2c3d640 1 -- start start 2024-05-20T18:27:55.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.237+0000 7f62d2c3d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62cc0d10e0 0x7f62cc0cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:55.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.237+0000 7f62d2c3d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f62cc0b7210 0x7f62cc0cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:55.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.237+0000 7f62d2c3d640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62cc0d1ac0 con 0x7f62cc0d14f0 2024-05-20T18:27:55.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.237+0000 7f62d2c3d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f62cc0d1da0 con 0x7f62cc0b7210 2024-05-20T18:27:55.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.237+0000 7f62d2c3d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f62cc0d2080 con 0x7f62cc0d10e0 2024-05-20T18:27:55.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.237+0000 7f62d11da640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f62cc0b7210 0x7f62cc0cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:55.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.237+0000 7f62d11da640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f62cc0b7210 0x7f62cc0cf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50926/0 (socket says 172.21.15.5:50926) 2024-05-20T18:27:55.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.237+0000 7f62d11da640 1 -- 172.21.15.5:0/2780905832 learned_addr learned my addr 172.21.15.5:0/2780905832 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:55.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.237+0000 7f62d09d9640 1 --2- 172.21.15.5:0/2780905832 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62cc0d10e0 0x7f62cc0cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:55.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.237+0000 7f62d11da640 1 -- 172.21.15.5:0/2780905832 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62cc0d10e0 msgr2=0x7f62cc0cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:55.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.237+0000 7f62d11da640 1 --2- 172.21.15.5:0/2780905832 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62cc0d10e0 0x7f62cc0cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:55.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.237+0000 7f62d11da640 1 -- 172.21.15.5:0/2780905832 >> v1:172.21.15.63:6789/0 conn(0x7f62cc0d14f0 legacy=0x7f62cc0cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:55.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.237+0000 7f62d11da640 1 -- 172.21.15.5:0/2780905832 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62cc0d2360 con 0x7f62cc0b7210 2024-05-20T18:27:55.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.237+0000 7f62d09d9640 1 --2- 172.21.15.5:0/2780905832 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62cc0d10e0 0x7f62cc0cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:55.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.238+0000 7f62d11da640 1 --2- 172.21.15.5:0/2780905832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62cc0b7210 0x7f62cc0cf680 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f62c4009e50 tx=0x7f62c402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc623ed67a91600d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:55.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.238+0000 7f62cb7fe640 1 -- 172.21.15.5:0/2780905832 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c4036070 con 0x7f62cc0b7210 2024-05-20T18:27:55.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.238+0000 7f62cb7fe640 1 -- 172.21.15.5:0/2780905832 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62c4031600 con 0x7f62cc0b7210 2024-05-20T18:27:55.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.238+0000 7f62cb7fe640 1 -- 172.21.15.5:0/2780905832 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c40319d0 con 0x7f62cc0b7210 2024-05-20T18:27:55.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.239+0000 7f62d2c3d640 1 -- 172.21.15.5:0/2780905832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62cc0b7210 msgr2=0x7f62cc0cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:55.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.239+0000 7f62d2c3d640 1 --2- 172.21.15.5:0/2780905832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62cc0b7210 0x7f62cc0cf680 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f62c4009e50 tx=0x7f62c402a6d0 comp rx=0 tx=0).stop 2024-05-20T18:27:55.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.239+0000 7f62d2c3d640 1 -- 172.21.15.5:0/2780905832 shutdown_connections 2024-05-20T18:27:55.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.239+0000 7f62d2c3d640 1 --2- 172.21.15.5:0/2780905832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62cc0b7210 0x7f62cc0cf680 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:55.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.239+0000 7f62d2c3d640 1 --2- 172.21.15.5:0/2780905832 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62cc0d10e0 0x7f62cc0cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:55.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.239+0000 7f62d2c3d640 1 -- 172.21.15.5:0/2780905832 >> 172.21.15.5:0/2780905832 conn(0x7f62cc0b44d0 msgr2=0x7f62cc0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:55.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.239+0000 7f62d2c3d640 1 -- 172.21.15.5:0/2780905832 shutdown_connections 2024-05-20T18:27:55.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.239+0000 7f62d2c3d640 1 -- 172.21.15.5:0/2780905832 wait complete. 2024-05-20T18:27:55.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.241+0000 7f62d2c3d640 1 Processor -- start 2024-05-20T18:27:55.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.241+0000 7f62d2c3d640 1 -- start start 2024-05-20T18:27:55.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.242+0000 7f62d2c3d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62cc0b7210 0x7f62cc150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:55.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.242+0000 7f62d2c3d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62cc0d10e0 0x7f62cc1f3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:55.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.242+0000 7f62d2c3d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62cc0d14f0 0x7f62cc1f4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:55.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.242+0000 7f62d2c3d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc0b8440 con 0x7f62cc0b7210 2024-05-20T18:27:55.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.242+0000 7f62d2c3d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc0b82c0 con 0x7f62cc0d10e0 2024-05-20T18:27:55.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.242+0000 7f62d2c3d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f62cc0709f0 con 0x7f62cc0d14f0 2024-05-20T18:27:55.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.242+0000 7f62d09d9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62cc0b7210 0x7f62cc150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:55.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.242+0000 7f62d09d9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62cc0b7210 0x7f62cc150f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50932/0 (socket says 172.21.15.5:50932) 2024-05-20T18:27:55.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.242+0000 7f62d09d9640 1 -- 172.21.15.5:0/1821768610 learned_addr learned my addr 172.21.15.5:0/1821768610 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:55.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.243+0000 7f62cbfff640 1 --2- 172.21.15.5:0/1821768610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62cc0d10e0 0x7f62cc1f3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:55.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.243+0000 7f62d11da640 1 --2- 172.21.15.5:0/1821768610 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62cc0d14f0 0x7f62cc1f4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:55.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.243+0000 7f62d09d9640 1 -- 172.21.15.5:0/1821768610 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62cc0d14f0 msgr2=0x7f62cc1f4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:55.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.243+0000 7f62d09d9640 1 --2- 172.21.15.5:0/1821768610 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62cc0d14f0 0x7f62cc1f4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:55.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.243+0000 7f62d09d9640 1 -- 172.21.15.5:0/1821768610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62cc0d10e0 msgr2=0x7f62cc1f3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:55.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.243+0000 7f62d09d9640 1 --2- 172.21.15.5:0/1821768610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62cc0d10e0 0x7f62cc1f3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:55.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.243+0000 7f62d09d9640 1 -- 172.21.15.5:0/1821768610 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62cc1f4950 con 0x7f62cc0b7210 2024-05-20T18:27:55.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.243+0000 7f62cbfff640 1 --2- 172.21.15.5:0/1821768610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62cc0d10e0 0x7f62cc1f3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:55.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.243+0000 7f62d09d9640 1 --2- 172.21.15.5:0/1821768610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62cc0b7210 0x7f62cc150f40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f62bc00c7f0 tx=0x7f62bc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:55.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.243+0000 7f62c97fa640 1 -- 172.21.15.5:0/1821768610 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62bc015070 con 0x7f62cc0b7210 2024-05-20T18:27:55.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.243+0000 7f62c97fa640 1 -- 172.21.15.5:0/1821768610 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62bc0077c0 con 0x7f62cc0b7210 2024-05-20T18:27:55.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.243+0000 7f62d2c3d640 1 -- 172.21.15.5:0/1821768610 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62cc1f4c30 con 0x7f62cc0b7210 2024-05-20T18:27:55.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.243+0000 7f62c97fa640 1 -- 172.21.15.5:0/1821768610 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62bc007b70 con 0x7f62cc0b7210 2024-05-20T18:27:55.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.243+0000 7f62d2c3d640 1 -- 172.21.15.5:0/1821768610 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62cc1f51f0 con 0x7f62cc0b7210 2024-05-20T18:27:55.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.244+0000 7f62d2c3d640 1 -- 172.21.15.5:0/1821768610 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62cc075b00 con 0x7f62cc0b7210 2024-05-20T18:27:55.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.245+0000 7f62c97fa640 1 -- 172.21.15.5:0/1821768610 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f62bc018070 con 0x7f62cc0b7210 2024-05-20T18:27:55.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.245+0000 7f62c97fa640 1 --2- 172.21.15.5:0/1821768610 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f62b806be30 0x7f62b806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:55.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.246+0000 7f62c97fa640 1 -- 172.21.15.5:0/1821768610 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f62bc08b270 con 0x7f62cc0b7210 2024-05-20T18:27:55.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.246+0000 7f62cbfff640 1 --2- 172.21.15.5:0/1821768610 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f62b806be30 0x7f62b806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:55.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.246+0000 7f62cbfff640 1 --2- 172.21.15.5:0/1821768610 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f62b806be30 0x7f62b806e340 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0ab020 tx=0x7f62c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:55.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.250+0000 7f62c97fa640 1 -- 172.21.15.5:0/1821768610 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f62bc059700 con 0x7f62cc0b7210 2024-05-20T18:27:55.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.433+0000 7f62d2c3d640 1 -- 172.21.15.5:0/1821768610 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f62cc15d1e0 con 0x7f62cc0b7210 2024-05-20T18:27:55.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.433+0000 7f62c97fa640 1 -- 172.21.15.5:0/1821768610 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f62bc059000 con 0x7f62cc0b7210 2024-05-20T18:27:55.433 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:27:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.435+0000 7f62d2c3d640 1 -- 172.21.15.5:0/1821768610 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f62b806be30 msgr2=0x7f62b806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.435+0000 7f62d2c3d640 1 --2- 172.21.15.5:0/1821768610 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f62b806be30 0x7f62b806e340 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0ab020 tx=0x7f62c0008040 comp rx=0 tx=0).stop 2024-05-20T18:27:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.435+0000 7f62d2c3d640 1 -- 172.21.15.5:0/1821768610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62cc0b7210 msgr2=0x7f62cc150f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:55.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.435+0000 7f62d2c3d640 1 --2- 172.21.15.5:0/1821768610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62cc0b7210 0x7f62cc150f40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f62bc00c7f0 tx=0x7f62bc00ccb0 comp rx=0 tx=0).stop 2024-05-20T18:27:55.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.436+0000 7f62d2c3d640 1 -- 172.21.15.5:0/1821768610 shutdown_connections 2024-05-20T18:27:55.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.436+0000 7f62d2c3d640 1 --2- 172.21.15.5:0/1821768610 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f62b806be30 0x7f62b806e340 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:55.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.436+0000 7f62d2c3d640 1 --2- 172.21.15.5:0/1821768610 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62cc0d14f0 0x7f62cc1f4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:55.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.436+0000 7f62d2c3d640 1 --2- 172.21.15.5:0/1821768610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62cc0d10e0 0x7f62cc1f3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:55.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.436+0000 7f62d2c3d640 1 --2- 172.21.15.5:0/1821768610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62cc0b7210 0x7f62cc150f40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:55.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.436+0000 7f62d2c3d640 1 -- 172.21.15.5:0/1821768610 >> 172.21.15.5:0/1821768610 conn(0x7f62cc0b44d0 msgr2=0x7f62cc0d29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:55.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.436+0000 7f62d2c3d640 1 -- 172.21.15.5:0/1821768610 shutdown_connections 2024-05-20T18:27:55.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.436+0000 7f62d2c3d640 1 -- 172.21.15.5:0/1821768610 wait complete. 2024-05-20T18:27:55.447 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:27:55.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.567+0000 7f2ecf6d7640 1 Processor -- start 2024-05-20T18:27:55.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.567+0000 7f2ecf6d7640 1 -- start start 2024-05-20T18:27:55.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.568+0000 7f2ecf6d7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2ec815f490 0x7f2ec8163900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:55.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.568+0000 7f2ecf6d7640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2ec80cd5d0 0x7f2ec80b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:55.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.568+0000 7f2ecf6d7640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ec80b4db0 con 0x7f2ec80b49a0 2024-05-20T18:27:55.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.568+0000 7f2ecf6d7640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f2ec8163e40 con 0x7f2ec80cd5d0 2024-05-20T18:27:55.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.568+0000 7f2ecf6d7640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2ec806c5b0 con 0x7f2ec815f490 2024-05-20T18:27:55.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.569+0000 7f2eccc72640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2ec80cd5d0 0x7f2ec80b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:55.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.569+0000 7f2eccc72640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2ec80cd5d0 0x7f2ec80b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50936/0 (socket says 172.21.15.5:50936) 2024-05-20T18:27:55.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.569+0000 7f2eccc72640 1 -- 172.21.15.5:0/2026040381 learned_addr learned my addr 172.21.15.5:0/2026040381 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:55.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.569+0000 7f2ecd473640 1 --2- 172.21.15.5:0/2026040381 >> v2:172.21.15.63:3300/0 conn(0x7f2ec815f490 0x7f2ec8163900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:55.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.569+0000 7f2eccc72640 1 -- 172.21.15.5:0/2026040381 >> v2:172.21.15.63:3300/0 conn(0x7f2ec815f490 msgr2=0x7f2ec8163900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:55.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.569+0000 7f2eccc72640 1 --2- 172.21.15.5:0/2026040381 >> v2:172.21.15.63:3300/0 conn(0x7f2ec815f490 0x7f2ec8163900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:55.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.569+0000 7f2eccc72640 1 -- 172.21.15.5:0/2026040381 >> v1:172.21.15.63:6789/0 conn(0x7f2ec80b49a0 legacy=0x7f2ec80708d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:55.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.569+0000 7f2eccc72640 1 -- 172.21.15.5:0/2026040381 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec80ce950 con 0x7f2ec80cd5d0 2024-05-20T18:27:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.570+0000 7f2eccc72640 1 --2- 172.21.15.5:0/2026040381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ec80cd5d0 0x7f2ec80b4460 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2ec808baf0 tx=0x7f2eb802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18d8e3502b3a1c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.570+0000 7f2ebffff640 1 -- 172.21.15.5:0/2026040381 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8037070 con 0x7f2ec80cd5d0 2024-05-20T18:27:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.570+0000 7f2ebffff640 1 -- 172.21.15.5:0/2026040381 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2eb8032440 con 0x7f2ec80cd5d0 2024-05-20T18:27:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.570+0000 7f2ebffff640 1 -- 172.21.15.5:0/2026040381 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8032810 con 0x7f2ec80cd5d0 2024-05-20T18:27:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.571+0000 7f2ecf6d7640 1 -- 172.21.15.5:0/2026040381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ec80cd5d0 msgr2=0x7f2ec80b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.571+0000 7f2ecf6d7640 1 --2- 172.21.15.5:0/2026040381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ec80cd5d0 0x7f2ec80b4460 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2ec808baf0 tx=0x7f2eb802a270 comp rx=0 tx=0).stop 2024-05-20T18:27:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.571+0000 7f2ecf6d7640 1 -- 172.21.15.5:0/2026040381 shutdown_connections 2024-05-20T18:27:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.571+0000 7f2ecf6d7640 1 --2- 172.21.15.5:0/2026040381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ec80cd5d0 0x7f2ec80b4460 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.571+0000 7f2ecf6d7640 1 --2- 172.21.15.5:0/2026040381 >> v2:172.21.15.63:3300/0 conn(0x7f2ec815f490 0x7f2ec8163900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.571+0000 7f2ecf6d7640 1 -- 172.21.15.5:0/2026040381 >> 172.21.15.5:0/2026040381 conn(0x7f2ec80cf460 msgr2=0x7f2ec80cf870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.571+0000 7f2ecf6d7640 1 -- 172.21.15.5:0/2026040381 shutdown_connections 2024-05-20T18:27:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.571+0000 7f2ecf6d7640 1 -- 172.21.15.5:0/2026040381 wait complete. 2024-05-20T18:27:55.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.573+0000 7f2ecf6d7640 1 Processor -- start 2024-05-20T18:27:55.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.573+0000 7f2ecf6d7640 1 -- start start 2024-05-20T18:27:55.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.574+0000 7f2ecf6d7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ec80b49a0 0x7f2ec80c7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:55.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.574+0000 7f2ecf6d7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ec80cd5d0 0x7f2ec81589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:55.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.574+0000 7f2ecf6d7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ec815f490 0x7f2ec8156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:55.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.574+0000 7f2ecf6d7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec80ce390 con 0x7f2ec80cd5d0 2024-05-20T18:27:55.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.574+0000 7f2ecf6d7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec80ce0b0 con 0x7f2ec80b49a0 2024-05-20T18:27:55.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.574+0000 7f2ecf6d7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2ec80ce670 con 0x7f2ec815f490 2024-05-20T18:27:55.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.574+0000 7f2eccc72640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ec80cd5d0 0x7f2ec81589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:55.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.574+0000 7f2eccc72640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ec80cd5d0 0x7f2ec81589c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50944/0 (socket says 172.21.15.5:50944) 2024-05-20T18:27:55.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.574+0000 7f2eccc72640 1 -- 172.21.15.5:0/1237843126 learned_addr learned my addr 172.21.15.5:0/1237843126 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:55.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.575+0000 7f2ecd473640 1 --2- 172.21.15.5:0/1237843126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ec80b49a0 0x7f2ec80c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:55.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.575+0000 7f2ecdc74640 1 --2- 172.21.15.5:0/1237843126 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ec815f490 0x7f2ec8156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:55.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.575+0000 7f2eccc72640 1 -- 172.21.15.5:0/1237843126 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ec815f490 msgr2=0x7f2ec8156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:55.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.575+0000 7f2eccc72640 1 --2- 172.21.15.5:0/1237843126 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ec815f490 0x7f2ec8156800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:55.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.575+0000 7f2eccc72640 1 -- 172.21.15.5:0/1237843126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ec80b49a0 msgr2=0x7f2ec80c7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:55.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.575+0000 7f2eccc72640 1 --2- 172.21.15.5:0/1237843126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ec80b49a0 0x7f2ec80c7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:55.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.575+0000 7f2eccc72640 1 -- 172.21.15.5:0/1237843126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec80ca610 con 0x7f2ec80cd5d0 2024-05-20T18:27:55.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.575+0000 7f2ecd473640 1 --2- 172.21.15.5:0/1237843126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ec80b49a0 0x7f2ec80c7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:55.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.575+0000 7f2ecdc74640 1 --2- 172.21.15.5:0/1237843126 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ec815f490 0x7f2ec8156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:55.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.575+0000 7f2eccc72640 1 --2- 172.21.15.5:0/1237843126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ec80cd5d0 0x7f2ec81589c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2eb802a2b0 tx=0x7f2eb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:55.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.575+0000 7f2ebdffb640 1 -- 172.21.15.5:0/1237843126 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8037040 con 0x7f2ec80cd5d0 2024-05-20T18:27:55.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.575+0000 7f2ebdffb640 1 -- 172.21.15.5:0/1237843126 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2eb8042070 con 0x7f2ec80cd5d0 2024-05-20T18:27:55.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.575+0000 7f2ecf6d7640 1 -- 172.21.15.5:0/1237843126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec80ca8f0 con 0x7f2ec80cd5d0 2024-05-20T18:27:55.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.575+0000 7f2ebdffb640 1 -- 172.21.15.5:0/1237843126 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8030880 con 0x7f2ec80cd5d0 2024-05-20T18:27:55.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.576+0000 7f2ecf6d7640 1 -- 172.21.15.5:0/1237843126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec80caeb0 con 0x7f2ec80cd5d0 2024-05-20T18:27:55.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.577+0000 7f2ecf6d7640 1 -- 172.21.15.5:0/1237843126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ec80d24d0 con 0x7f2ec80cd5d0 2024-05-20T18:27:55.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.577+0000 7f2ebdffb640 1 -- 172.21.15.5:0/1237843126 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2eb8030a20 con 0x7f2ec80cd5d0 2024-05-20T18:27:55.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.578+0000 7f2ebdffb640 1 --2- 172.21.15.5:0/1237843126 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2eac06be30 0x7f2eac06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:55.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.578+0000 7f2ebdffb640 1 -- 172.21.15.5:0/1237843126 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7f2eb80abba0 con 0x7f2ec80cd5d0 2024-05-20T18:27:55.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.578+0000 7f2ecd473640 1 --2- 172.21.15.5:0/1237843126 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2eac06be30 0x7f2eac06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:55.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.579+0000 7f2ecd473640 1 --2- 172.21.15.5:0/1237843126 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2eac06be30 0x7f2eac06e340 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4009af0 tx=0x7f2ec4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.583+0000 7f2ebdffb640 1 -- 172.21.15.5:0/1237843126 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2ec80d24d0 con 0x7f2ec80cd5d0 2024-05-20T18:27:55.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.768+0000 7f2ecf6d7640 1 -- 172.21.15.5:0/1237843126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2ec80ca3a0 con 0x7f2ec80cd5d0 2024-05-20T18:27:55.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.768+0000 7f2ebdffb640 1 -- 172.21.15.5:0/1237843126 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f2eb807a0b0 con 0x7f2ec80cd5d0 2024-05-20T18:27:55.768 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:27:55.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.770+0000 7f2ecf6d7640 1 -- 172.21.15.5:0/1237843126 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2eac06be30 msgr2=0x7f2eac06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:55.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.771+0000 7f2ecf6d7640 1 --2- 172.21.15.5:0/1237843126 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2eac06be30 0x7f2eac06e340 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4009af0 tx=0x7f2ec4008040 comp rx=0 tx=0).stop 2024-05-20T18:27:55.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.771+0000 7f2ecf6d7640 1 -- 172.21.15.5:0/1237843126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ec80cd5d0 msgr2=0x7f2ec81589c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:55.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.771+0000 7f2ecf6d7640 1 --2- 172.21.15.5:0/1237843126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ec80cd5d0 0x7f2ec81589c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2eb802a2b0 tx=0x7f2eb80046c0 comp rx=0 tx=0).stop 2024-05-20T18:27:55.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.771+0000 7f2ecf6d7640 1 -- 172.21.15.5:0/1237843126 shutdown_connections 2024-05-20T18:27:55.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.771+0000 7f2ecf6d7640 1 --2- 172.21.15.5:0/1237843126 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2eac06be30 0x7f2eac06e340 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:55.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.771+0000 7f2ecf6d7640 1 --2- 172.21.15.5:0/1237843126 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ec815f490 0x7f2ec8156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:55.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.771+0000 7f2ecf6d7640 1 --2- 172.21.15.5:0/1237843126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ec80cd5d0 0x7f2ec81589c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:55.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.771+0000 7f2ecf6d7640 1 --2- 172.21.15.5:0/1237843126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ec80b49a0 0x7f2ec80c7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:55.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.771+0000 7f2ecf6d7640 1 -- 172.21.15.5:0/1237843126 >> 172.21.15.5:0/1237843126 conn(0x7f2ec80cf460 msgr2=0x7f2ec815b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:55.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.771+0000 7f2ecf6d7640 1 -- 172.21.15.5:0/1237843126 shutdown_connections 2024-05-20T18:27:55.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.771+0000 7f2ecf6d7640 1 -- 172.21.15.5:0/1237843126 wait complete. 2024-05-20T18:27:55.781 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:27:55.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.901+0000 7fbb60165640 1 Processor -- start 2024-05-20T18:27:55.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.901+0000 7fbb60165640 1 -- start start 2024-05-20T18:27:55.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.903+0000 7fbb60165640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbb58168050 0x7fbb5816a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:55.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.903+0000 7fbb60165640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb580d0630 con 0x7fbb58162820 2024-05-20T18:27:55.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.903+0000 7fbb60165640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb580d39c0 con 0x7fbb5806d3d0 2024-05-20T18:27:55.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.903+0000 7fbb60165640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fbb580d3ca0 con 0x7fbb58168050 2024-05-20T18:27:55.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.903+0000 7fbb5e702640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbb58168050 0x7fbb5816a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:55.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.903+0000 7fbb5e702640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbb58168050 0x7fbb5816a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50958/0 (socket says 172.21.15.5:50958) 2024-05-20T18:27:55.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.903+0000 7fbb5df01640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbb58162820 0x7fbb580b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53944/0 (socket says 172.21.15.5:53944) 2024-05-20T18:27:55.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.903+0000 7fbb5e702640 1 -- 172.21.15.5:0/2161659858 learned_addr learned my addr 172.21.15.5:0/2161659858 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:55.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.903+0000 7fbb5e702640 1 -- 172.21.15.5:0/2161659858 >> v1:172.21.15.63:6789/0 conn(0x7fbb5806d3d0 legacy=0x7fbb5806d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:55.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.903+0000 7fbb5e702640 1 -- 172.21.15.5:0/2161659858 >> v1:172.21.15.5:6789/0 conn(0x7fbb58162820 legacy=0x7fbb580b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:55.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.903+0000 7fbb5e702640 1 -- 172.21.15.5:0/2161659858 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb580d3f80 con 0x7fbb58168050 2024-05-20T18:27:55.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.903+0000 7fbb5ceff640 1 -- 172.21.15.5:0/2161659858 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3013502450 0 0) 0x7fbb580d0630 con 0x7fbb58162820 2024-05-20T18:27:55.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.904+0000 7fbb5e702640 1 --2- 172.21.15.5:0/2161659858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb58168050 0x7fbb5816a470 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009870 tx=0x7fbb5002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=609f02aa4c556e7e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:55.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.904+0000 7fbb5ceff640 1 -- 172.21.15.5:0/2161659858 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb50037070 con 0x7fbb58168050 2024-05-20T18:27:55.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.904+0000 7fbb5ceff640 1 -- 172.21.15.5:0/2161659858 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb50032440 con 0x7fbb58168050 2024-05-20T18:27:55.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.904+0000 7fbb5ceff640 1 -- 172.21.15.5:0/2161659858 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb50032810 con 0x7fbb58168050 2024-05-20T18:27:55.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.905+0000 7fbb60165640 1 -- 172.21.15.5:0/2161659858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb58168050 msgr2=0x7fbb5816a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:55.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.905+0000 7fbb60165640 1 --2- 172.21.15.5:0/2161659858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb58168050 0x7fbb5816a470 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009870 tx=0x7fbb5002a2a0 comp rx=0 tx=0).stop 2024-05-20T18:27:55.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.905+0000 7fbb60165640 1 -- 172.21.15.5:0/2161659858 shutdown_connections 2024-05-20T18:27:55.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.905+0000 7fbb60165640 1 --2- 172.21.15.5:0/2161659858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb58168050 0x7fbb5816a470 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:55.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.905+0000 7fbb60165640 1 -- 172.21.15.5:0/2161659858 >> 172.21.15.5:0/2161659858 conn(0x7fbb580d4bd0 msgr2=0x7fbb580d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:55.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.905+0000 7fbb60165640 1 -- 172.21.15.5:0/2161659858 shutdown_connections 2024-05-20T18:27:55.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.905+0000 7fbb60165640 1 -- 172.21.15.5:0/2161659858 wait complete. 2024-05-20T18:27:55.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.907+0000 7fbb60165640 1 Processor -- start 2024-05-20T18:27:55.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.907+0000 7fbb60165640 1 -- start start 2024-05-20T18:27:55.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.908+0000 7fbb60165640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb5806d3d0 0x7fbb581557a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:55.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.908+0000 7fbb60165640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb58162820 0x7fbb58154b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:55.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.908+0000 7fbb60165640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb58168050 0x7fbb5815ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:55.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.908+0000 7fbb60165640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbb580b91a0 con 0x7fbb58162820 2024-05-20T18:27:55.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.908+0000 7fbb60165640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbb580d0300 con 0x7fbb58168050 2024-05-20T18:27:55.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.908+0000 7fbb60165640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbb580b4050 con 0x7fbb5806d3d0 2024-05-20T18:27:55.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.908+0000 7fbb5d700640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb58162820 0x7fbb58154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:55.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.908+0000 7fbb5d700640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb58162820 0x7fbb58154b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50960/0 (socket says 172.21.15.5:50960) 2024-05-20T18:27:55.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.908+0000 7fbb5d700640 1 -- 172.21.15.5:0/1936354369 learned_addr learned my addr 172.21.15.5:0/1936354369 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:55.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.908+0000 7fbb5e702640 1 --2- 172.21.15.5:0/1936354369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb58168050 0x7fbb5815ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:55.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.908+0000 7fbb5df01640 1 --2- 172.21.15.5:0/1936354369 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb5806d3d0 0x7fbb581557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:55.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.908+0000 7fbb5d700640 1 -- 172.21.15.5:0/1936354369 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb5806d3d0 msgr2=0x7fbb581557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:55.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.908+0000 7fbb5d700640 1 --2- 172.21.15.5:0/1936354369 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb5806d3d0 0x7fbb581557a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:55.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.908+0000 7fbb5d700640 1 -- 172.21.15.5:0/1936354369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb58168050 msgr2=0x7fbb5815ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:55.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.908+0000 7fbb5d700640 1 --2- 172.21.15.5:0/1936354369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb58168050 0x7fbb5815ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:55.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.908+0000 7fbb5d700640 1 -- 172.21.15.5:0/1936354369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb580b4640 con 0x7fbb58162820 2024-05-20T18:27:55.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.909+0000 7fbb5e702640 1 --2- 172.21.15.5:0/1936354369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb58168050 0x7fbb5815ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:55.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.909+0000 7fbb5df01640 1 --2- 172.21.15.5:0/1936354369 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb5806d3d0 0x7fbb581557a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:55.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.909+0000 7fbb5d700640 1 --2- 172.21.15.5:0/1936354369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb58162820 0x7fbb58154b70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c00b3f0 tx=0x7fbb4c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:55.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.909+0000 7fbb567fc640 1 -- 172.21.15.5:0/1936354369 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c015020 con 0x7fbb58162820 2024-05-20T18:27:55.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.909+0000 7fbb567fc640 1 -- 172.21.15.5:0/1936354369 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb4c011830 con 0x7fbb58162820 2024-05-20T18:27:55.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.909+0000 7fbb60165640 1 -- 172.21.15.5:0/1936354369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb580b4920 con 0x7fbb58162820 2024-05-20T18:27:55.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.909+0000 7fbb567fc640 1 -- 172.21.15.5:0/1936354369 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c011c20 con 0x7fbb58162820 2024-05-20T18:27:55.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.909+0000 7fbb60165640 1 -- 172.21.15.5:0/1936354369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb5815dcb0 con 0x7fbb58162820 2024-05-20T18:27:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.910+0000 7fbb60165640 1 -- 172.21.15.5:0/1936354369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb58075b00 con 0x7fbb58162820 2024-05-20T18:27:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.911+0000 7fbb567fc640 1 -- 172.21.15.5:0/1936354369 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbb4c018020 con 0x7fbb58162820 2024-05-20T18:27:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.911+0000 7fbb567fc640 1 --2- 172.21.15.5:0/1936354369 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbb4406be60 0x7fbb4406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.911+0000 7fbb567fc640 1 -- 172.21.15.5:0/1936354369 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6743+0+0 (secure 0 0 0) 0x7fbb4c08afa0 con 0x7fbb58162820 2024-05-20T18:27:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.911+0000 7fbb5df01640 1 --2- 172.21.15.5:0/1936354369 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbb4406be60 0x7fbb4406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.912+0000 7fbb5df01640 1 --2- 172.21.15.5:0/1936354369 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbb4406be60 0x7fbb4406e370 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbb48007fd0 tx=0x7fbb48009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:55.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:55.916+0000 7fbb567fc640 1 -- 172.21.15.5:0/1936354369 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbb4c01f030 con 0x7fbb58162820 2024-05-20T18:27:56.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.105+0000 7fbb60165640 1 -- 172.21.15.5:0/1936354369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbb580b3620 con 0x7fbb58162820 2024-05-20T18:27:56.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.105+0000 7fbb567fc640 1 -- 172.21.15.5:0/1936354369 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fbb4c0172c0 con 0x7fbb58162820 2024-05-20T18:27:56.105 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:27:56.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.107+0000 7fbb60165640 1 -- 172.21.15.5:0/1936354369 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbb4406be60 msgr2=0x7fbb4406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:56.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.107+0000 7fbb60165640 1 --2- 172.21.15.5:0/1936354369 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbb4406be60 0x7fbb4406e370 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbb48007fd0 tx=0x7fbb48009040 comp rx=0 tx=0).stop 2024-05-20T18:27:56.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.107+0000 7fbb60165640 1 -- 172.21.15.5:0/1936354369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb58162820 msgr2=0x7fbb58154b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:56.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.107+0000 7fbb60165640 1 --2- 172.21.15.5:0/1936354369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb58162820 0x7fbb58154b70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c00b3f0 tx=0x7fbb4c00b8b0 comp rx=0 tx=0).stop 2024-05-20T18:27:56.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.108+0000 7fbb60165640 1 -- 172.21.15.5:0/1936354369 shutdown_connections 2024-05-20T18:27:56.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.108+0000 7fbb60165640 1 --2- 172.21.15.5:0/1936354369 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbb4406be60 0x7fbb4406e370 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:56.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.108+0000 7fbb60165640 1 --2- 172.21.15.5:0/1936354369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb58168050 0x7fbb5815ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:56.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.108+0000 7fbb60165640 1 --2- 172.21.15.5:0/1936354369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb58162820 0x7fbb58154b70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:56.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.108+0000 7fbb60165640 1 --2- 172.21.15.5:0/1936354369 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb5806d3d0 0x7fbb581557a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:56.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.108+0000 7fbb60165640 1 -- 172.21.15.5:0/1936354369 >> 172.21.15.5:0/1936354369 conn(0x7fbb580d4bd0 msgr2=0x7fbb581610f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:56.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.108+0000 7fbb60165640 1 -- 172.21.15.5:0/1936354369 shutdown_connections 2024-05-20T18:27:56.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.108+0000 7fbb60165640 1 -- 172.21.15.5:0/1936354369 wait complete. 2024-05-20T18:27:56.119 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T18:27:56.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.242+0000 7f749fa51640 1 Processor -- start 2024-05-20T18:27:56.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.243+0000 7f749fa51640 1 -- start start 2024-05-20T18:27:56.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.244+0000 7f749fa51640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7498162820 0x7f74980b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:56.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.244+0000 7f749fa51640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f749806dae0 0x7f74980b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:56.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.244+0000 7f749fa51640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74980d0630 con 0x7f74980b3e80 2024-05-20T18:27:56.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.244+0000 7f749fa51640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f74980d39c0 con 0x7f7498162820 2024-05-20T18:27:56.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.244+0000 7f749fa51640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f74980d3ca0 con 0x7f749806dae0 2024-05-20T18:27:56.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.244+0000 7f749d7ed640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7498162820 0x7f74980b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:56.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.244+0000 7f749cfec640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f74980b3e80 0x7f749806d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53946/0 (socket says 172.21.15.5:53946) 2024-05-20T18:27:56.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.244+0000 7f749cfec640 1 -- 172.21.15.5:0/2596222988 learned_addr learned my addr 172.21.15.5:0/2596222988 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:56.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.244+0000 7f749d7ed640 1 --2- 172.21.15.5:0/2596222988 >> v2:172.21.15.5:3300/0 conn(0x7f7498162820 0x7f74980b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50976/0 (socket says 172.21.15.5:50976) 2024-05-20T18:27:56.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.244+0000 7f749d7ed640 1 -- 172.21.15.5:0/2596222988 >> v2:172.21.15.63:3300/0 conn(0x7f749806dae0 msgr2=0x7f74980b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:56.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.244+0000 7f749dfee640 1 --2- 172.21.15.5:0/2596222988 >> v2:172.21.15.63:3300/0 conn(0x7f749806dae0 0x7f74980b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:56.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.244+0000 7f749d7ed640 1 --2- 172.21.15.5:0/2596222988 >> v2:172.21.15.63:3300/0 conn(0x7f749806dae0 0x7f74980b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:56.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.244+0000 7f749d7ed640 1 -- 172.21.15.5:0/2596222988 >> v1:172.21.15.5:6789/0 conn(0x7f74980b3e80 legacy=0x7f749806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:56.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.244+0000 7f748ffff640 1 -- 172.21.15.5:0/2596222988 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2720094603 0 0) 0x7f74980d0630 con 0x7f74980b3e80 2024-05-20T18:27:56.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.244+0000 7f749d7ed640 1 -- 172.21.15.5:0/2596222988 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74980d3f80 con 0x7f7498162820 2024-05-20T18:27:56.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.244+0000 7f749dfee640 1 --2- 172.21.15.5:0/2596222988 >> v2:172.21.15.63:3300/0 conn(0x7f749806dae0 0x7f74980b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:56.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.245+0000 7f749d7ed640 1 --2- 172.21.15.5:0/2596222988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7498162820 0x7f74980b3940 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f749808e070 tx=0x7f749402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c65ceb25e333854c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:56.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.245+0000 7f748ffff640 1 -- 172.21.15.5:0/2596222988 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7494037070 con 0x7f7498162820 2024-05-20T18:27:56.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.245+0000 7f748ffff640 1 -- 172.21.15.5:0/2596222988 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7494032440 con 0x7f7498162820 2024-05-20T18:27:56.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.245+0000 7f748ffff640 1 -- 172.21.15.5:0/2596222988 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7494032810 con 0x7f7498162820 2024-05-20T18:27:56.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.246+0000 7f749fa51640 1 -- 172.21.15.5:0/2596222988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7498162820 msgr2=0x7f74980b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:56.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.246+0000 7f749fa51640 1 --2- 172.21.15.5:0/2596222988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7498162820 0x7f74980b3940 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f749808e070 tx=0x7f749402a1a0 comp rx=0 tx=0).stop 2024-05-20T18:27:56.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.246+0000 7f749fa51640 1 -- 172.21.15.5:0/2596222988 shutdown_connections 2024-05-20T18:27:56.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.246+0000 7f749fa51640 1 --2- 172.21.15.5:0/2596222988 >> v2:172.21.15.63:3300/0 conn(0x7f749806dae0 0x7f74980b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:56.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.246+0000 7f749fa51640 1 --2- 172.21.15.5:0/2596222988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7498162820 0x7f74980b3940 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:56.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.246+0000 7f749fa51640 1 -- 172.21.15.5:0/2596222988 >> 172.21.15.5:0/2596222988 conn(0x7f74980d4bd0 msgr2=0x7f74980d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:56.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.246+0000 7f749fa51640 1 -- 172.21.15.5:0/2596222988 shutdown_connections 2024-05-20T18:27:56.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.246+0000 7f749fa51640 1 -- 172.21.15.5:0/2596222988 wait complete. 2024-05-20T18:27:56.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.248+0000 7f749fa51640 1 Processor -- start 2024-05-20T18:27:56.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.248+0000 7f749fa51640 1 -- start start 2024-05-20T18:27:56.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.249+0000 7f749fa51640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f74980b3e80 0x7f7498155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:56.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.249+0000 7f749fa51640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7498162820 0x7f74980b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:56.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.249+0000 7f749fa51640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74980cb2a0 0x7f749815f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:56.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.249+0000 7f749fa51640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f74980d4780 con 0x7f74980cb2a0 2024-05-20T18:27:56.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.249+0000 7f749fa51640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f74980d0300 con 0x7f7498162820 2024-05-20T18:27:56.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.249+0000 7f749fa51640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f74980d4900 con 0x7f74980b3e80 2024-05-20T18:27:56.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.249+0000 7f749dfee640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74980cb2a0 0x7f749815f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:56.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.249+0000 7f749dfee640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74980cb2a0 0x7f749815f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50992/0 (socket says 172.21.15.5:50992) 2024-05-20T18:27:56.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.249+0000 7f749dfee640 1 -- 172.21.15.5:0/152634376 learned_addr learned my addr 172.21.15.5:0/152634376 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:56.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.250+0000 7f749d7ed640 1 --2- 172.21.15.5:0/152634376 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f74980b3e80 0x7f7498155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:56.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.250+0000 7f749cfec640 1 --2- 172.21.15.5:0/152634376 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7498162820 0x7f74980b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:56.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.250+0000 7f749dfee640 1 -- 172.21.15.5:0/152634376 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f74980b3e80 msgr2=0x7f7498155e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:56.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.250+0000 7f749dfee640 1 --2- 172.21.15.5:0/152634376 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f74980b3e80 0x7f7498155e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:56.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.250+0000 7f749dfee640 1 -- 172.21.15.5:0/152634376 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7498162820 msgr2=0x7f74980b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:56.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.250+0000 7f749dfee640 1 --2- 172.21.15.5:0/152634376 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7498162820 0x7f74980b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:56.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.250+0000 7f749dfee640 1 -- 172.21.15.5:0/152634376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74980c8a10 con 0x7f74980cb2a0 2024-05-20T18:27:56.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.250+0000 7f749d7ed640 1 --2- 172.21.15.5:0/152634376 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f74980b3e80 0x7f7498155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:56.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.250+0000 7f749dfee640 1 --2- 172.21.15.5:0/152634376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74980cb2a0 0x7f749815f9a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f749000cd80 tx=0x7f749000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:56.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.250+0000 7f748dffb640 1 -- 172.21.15.5:0/152634376 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749000f680 con 0x7f74980cb2a0 2024-05-20T18:27:56.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.250+0000 7f748dffb640 1 -- 172.21.15.5:0/152634376 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f749000f840 con 0x7f74980cb2a0 2024-05-20T18:27:56.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.250+0000 7f749fa51640 1 -- 172.21.15.5:0/152634376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74980ca200 con 0x7f74980cb2a0 2024-05-20T18:27:56.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.250+0000 7f748dffb640 1 -- 172.21.15.5:0/152634376 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749000fc30 con 0x7f74980cb2a0 2024-05-20T18:27:56.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.250+0000 7f749fa51640 1 -- 172.21.15.5:0/152634376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74980ca7c0 con 0x7f74980cb2a0 2024-05-20T18:27:56.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.251+0000 7f749fa51640 1 -- 172.21.15.5:0/152634376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7498156370 con 0x7f74980cb2a0 2024-05-20T18:27:56.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.252+0000 7f748dffb640 1 -- 172.21.15.5:0/152634376 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f74900213f0 con 0x7f74980cb2a0 2024-05-20T18:27:56.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.252+0000 7f748dffb640 1 --2- 172.21.15.5:0/152634376 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f747c06be60 0x7f747c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:56.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.253+0000 7f748dffb640 1 -- 172.21.15.5:0/152634376 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7f749008bf40 con 0x7f74980cb2a0 2024-05-20T18:27:56.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.253+0000 7f749d7ed640 1 --2- 172.21.15.5:0/152634376 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f747c06be60 0x7f747c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:56.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.253+0000 7f749d7ed640 1 --2- 172.21.15.5:0/152634376 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f747c06be60 0x7f747c06e370 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7494034000 tx=0x7f749402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:56.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.257+0000 7f748dffb640 1 -- 172.21.15.5:0/152634376 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7498156370 con 0x7f74980cb2a0 2024-05-20T18:27:56.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.440+0000 7f749fa51640 1 -- 172.21.15.5:0/152634376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f74980c9f90 con 0x7f74980cb2a0 2024-05-20T18:27:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.440+0000 7f748dffb640 1 -- 172.21.15.5:0/152634376 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f7490014030 con 0x7f74980cb2a0 2024-05-20T18:27:56.440 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:27:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.442+0000 7f749fa51640 1 -- 172.21.15.5:0/152634376 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f747c06be60 msgr2=0x7f747c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.442+0000 7f749fa51640 1 --2- 172.21.15.5:0/152634376 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f747c06be60 0x7f747c06e370 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7494034000 tx=0x7f749402e000 comp rx=0 tx=0).stop 2024-05-20T18:27:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.442+0000 7f749fa51640 1 -- 172.21.15.5:0/152634376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74980cb2a0 msgr2=0x7f749815f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.442+0000 7f749fa51640 1 --2- 172.21.15.5:0/152634376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74980cb2a0 0x7f749815f9a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f749000cd80 tx=0x7f749000a650 comp rx=0 tx=0).stop 2024-05-20T18:27:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.442+0000 7f749fa51640 1 -- 172.21.15.5:0/152634376 shutdown_connections 2024-05-20T18:27:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.442+0000 7f749fa51640 1 --2- 172.21.15.5:0/152634376 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f747c06be60 0x7f747c06e370 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.442+0000 7f749fa51640 1 --2- 172.21.15.5:0/152634376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74980cb2a0 0x7f749815f9a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.442+0000 7f749fa51640 1 --2- 172.21.15.5:0/152634376 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7498162820 0x7f74980b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.442+0000 7f749fa51640 1 --2- 172.21.15.5:0/152634376 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f74980b3e80 0x7f7498155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.442+0000 7f749fa51640 1 -- 172.21.15.5:0/152634376 >> 172.21.15.5:0/152634376 conn(0x7f74980d4bd0 msgr2=0x7f749815c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.442+0000 7f749fa51640 1 -- 172.21.15.5:0/152634376 shutdown_connections 2024-05-20T18:27:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.443+0000 7f749fa51640 1 -- 172.21.15.5:0/152634376 wait complete. 2024-05-20T18:27:56.477 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-20T18:27:56.478 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:27:56.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.594+0000 7fafbb516640 1 Processor -- start 2024-05-20T18:27:56.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.594+0000 7fafbb516640 1 -- start start 2024-05-20T18:27:56.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.595+0000 7fafbb516640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fafb4163a10 0x7fafb40d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:56.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.595+0000 7fafbb516640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fafb40d14b0 0x7fafb40d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:56.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.595+0000 7fafbb516640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafb40d0370 con 0x7fafb40b4010 2024-05-20T18:27:56.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.595+0000 7fafbb516640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fafb40d3ca0 con 0x7fafb4163a10 2024-05-20T18:27:56.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.595+0000 7fafbb516640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fafb40d0650 con 0x7fafb40d14b0 2024-05-20T18:27:56.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.595+0000 7fafb92b2640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fafb4163a10 0x7fafb40d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:56.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.596+0000 7fafb92b2640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fafb4163a10 0x7fafb40d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51006/0 (socket says 172.21.15.5:51006) 2024-05-20T18:27:56.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.596+0000 7fafb92b2640 1 -- 172.21.15.5:0/3104488136 learned_addr learned my addr 172.21.15.5:0/3104488136 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:56.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.596+0000 7fafb9ab3640 1 --2- 172.21.15.5:0/3104488136 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fafb40d14b0 0x7fafb40d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:56.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.596+0000 7fafb92b2640 1 -- 172.21.15.5:0/3104488136 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fafb40d14b0 msgr2=0x7fafb40d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:56.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.596+0000 7fafb92b2640 1 --2- 172.21.15.5:0/3104488136 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fafb40d14b0 0x7fafb40d18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:56.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.596+0000 7fafb92b2640 1 -- 172.21.15.5:0/3104488136 >> v1:172.21.15.63:6789/0 conn(0x7fafb40b4010 legacy=0x7fafb40cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:56.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.596+0000 7fafb92b2640 1 -- 172.21.15.5:0/3104488136 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb40d3650 con 0x7fafb4163a10 2024-05-20T18:27:56.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.597+0000 7fafb92b2640 1 --2- 172.21.15.5:0/3104488136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafb4163a10 0x7fafb40d4130 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009870 tx=0x7fafb002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554b70a94276ab96 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:56.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.597+0000 7fafabfff640 1 -- 172.21.15.5:0/3104488136 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0037070 con 0x7fafb4163a10 2024-05-20T18:27:56.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.597+0000 7fafabfff640 1 -- 172.21.15.5:0/3104488136 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafb0032440 con 0x7fafb4163a10 2024-05-20T18:27:56.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.597+0000 7fafabfff640 1 -- 172.21.15.5:0/3104488136 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0032810 con 0x7fafb4163a10 2024-05-20T18:27:56.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.597+0000 7fafbb516640 1 -- 172.21.15.5:0/3104488136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafb4163a10 msgr2=0x7fafb40d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:56.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.597+0000 7fafbb516640 1 --2- 172.21.15.5:0/3104488136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafb4163a10 0x7fafb40d4130 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009870 tx=0x7fafb002a270 comp rx=0 tx=0).stop 2024-05-20T18:27:56.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.598+0000 7fafbb516640 1 -- 172.21.15.5:0/3104488136 shutdown_connections 2024-05-20T18:27:56.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.598+0000 7fafbb516640 1 --2- 172.21.15.5:0/3104488136 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fafb40d14b0 0x7fafb40d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:56.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.598+0000 7fafbb516640 1 --2- 172.21.15.5:0/3104488136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafb4163a10 0x7fafb40d4130 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:56.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.598+0000 7fafbb516640 1 -- 172.21.15.5:0/3104488136 >> 172.21.15.5:0/3104488136 conn(0x7fafb406d240 msgr2=0x7fafb406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:56.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.598+0000 7fafbb516640 1 -- 172.21.15.5:0/3104488136 shutdown_connections 2024-05-20T18:27:56.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.598+0000 7fafbb516640 1 -- 172.21.15.5:0/3104488136 wait complete. 2024-05-20T18:27:56.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.599+0000 7fafbb516640 1 Processor -- start 2024-05-20T18:27:56.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.599+0000 7fafbb516640 1 -- start start 2024-05-20T18:27:56.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.601+0000 7fafbb516640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafb40b4010 0x7fafb415b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:56.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.601+0000 7fafbb516640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafb40d14b0 0x7fafb4154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:56.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.601+0000 7fafbb516640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fafb4163a10 0x7fafb415a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:56.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.601+0000 7fafbb516640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fafb40b8210 con 0x7fafb40b4010 2024-05-20T18:27:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.601+0000 7fafbb516640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fafb40d1e00 con 0x7fafb40d14b0 2024-05-20T18:27:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.601+0000 7fafbb516640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fafb40b8390 con 0x7fafb4163a10 2024-05-20T18:27:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.601+0000 7fafb92b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafb40b4010 0x7fafb415b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.601+0000 7fafb92b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafb40b4010 0x7fafb415b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51010/0 (socket says 172.21.15.5:51010) 2024-05-20T18:27:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.601+0000 7fafb92b2640 1 -- 172.21.15.5:0/1427015212 learned_addr learned my addr 172.21.15.5:0/1427015212 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.601+0000 7fafb8ab1640 1 --2- 172.21.15.5:0/1427015212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafb40d14b0 0x7fafb4154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.601+0000 7fafb9ab3640 1 --2- 172.21.15.5:0/1427015212 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fafb4163a10 0x7fafb415a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.601+0000 7fafb92b2640 1 -- 172.21.15.5:0/1427015212 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fafb4163a10 msgr2=0x7fafb415a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.601+0000 7fafb92b2640 1 --2- 172.21.15.5:0/1427015212 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fafb4163a10 0x7fafb415a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:56.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.601+0000 7fafb92b2640 1 -- 172.21.15.5:0/1427015212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafb40d14b0 msgr2=0x7fafb4154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:56.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.601+0000 7fafb92b2640 1 --2- 172.21.15.5:0/1427015212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafb40d14b0 0x7fafb4154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:56.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.601+0000 7fafb92b2640 1 -- 172.21.15.5:0/1427015212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb4160100 con 0x7fafb40b4010 2024-05-20T18:27:56.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.601+0000 7fafb8ab1640 1 --2- 172.21.15.5:0/1427015212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafb40d14b0 0x7fafb4154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:56.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.601+0000 7fafb9ab3640 1 --2- 172.21.15.5:0/1427015212 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fafb4163a10 0x7fafb415a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:56.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.601+0000 7fafb92b2640 1 --2- 172.21.15.5:0/1427015212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafb40b4010 0x7fafb415b100 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fafb0000c00 tx=0x7fafb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:56.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.602+0000 7fafa9ffb640 1 -- 172.21.15.5:0/1427015212 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0041020 con 0x7fafb40b4010 2024-05-20T18:27:56.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.602+0000 7fafa9ffb640 1 -- 172.21.15.5:0/1427015212 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafb00306c0 con 0x7fafb40b4010 2024-05-20T18:27:56.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.602+0000 7fafa9ffb640 1 -- 172.21.15.5:0/1427015212 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0037040 con 0x7fafb40b4010 2024-05-20T18:27:56.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.602+0000 7fafbb516640 1 -- 172.21.15.5:0/1427015212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb4157420 con 0x7fafb40b4010 2024-05-20T18:27:56.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.602+0000 7fafbb516640 1 -- 172.21.15.5:0/1427015212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb41603c0 con 0x7fafb40b4010 2024-05-20T18:27:56.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.603+0000 7fafbb516640 1 -- 172.21.15.5:0/1427015212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafb4075b00 con 0x7fafb40b4010 2024-05-20T18:27:56.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.604+0000 7fafa9ffb640 1 -- 172.21.15.5:0/1427015212 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fafb0044020 con 0x7fafb40b4010 2024-05-20T18:27:56.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.604+0000 7fafa9ffb640 1 --2- 172.21.15.5:0/1427015212 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faf9806be60 0x7faf9806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:56.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.604+0000 7fafa9ffb640 1 -- 172.21.15.5:0/1427015212 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7fafb00abae0 con 0x7fafb40b4010 2024-05-20T18:27:56.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.604+0000 7fafb8ab1640 1 --2- 172.21.15.5:0/1427015212 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faf9806be60 0x7faf9806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:56.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.605+0000 7fafb8ab1640 1 --2- 172.21.15.5:0/1427015212 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faf9806be60 0x7faf9806e370 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fafb415a0a0 tx=0x7fafa4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:56.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.609+0000 7fafa9ffb640 1 -- 172.21.15.5:0/1427015212 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fafb004b030 con 0x7fafb40b4010 2024-05-20T18:27:56.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.803+0000 7fafbb516640 1 -- 172.21.15.5:0/1427015212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fafb40d3050 con 0x7fafb40b4010 2024-05-20T18:27:56.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.804+0000 7fafa9ffb640 1 -- 172.21.15.5:0/1427015212 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18800 (secure 0 0 0) 0x7fafb007a080 con 0x7fafb40b4010 2024-05-20T18:27:56.804 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:27:56.804 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:27:56.213938+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:27:05.476596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T18:27:06.137762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T18:27:09.829920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:27:56.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.806+0000 7fafbb516640 1 -- 172.21.15.5:0/1427015212 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faf9806be60 msgr2=0x7faf9806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.806+0000 7fafbb516640 1 --2- 172.21.15.5:0/1427015212 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faf9806be60 0x7faf9806e370 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fafb415a0a0 tx=0x7fafa4009040 comp rx=0 tx=0).stop 2024-05-20T18:27:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.806+0000 7fafbb516640 1 -- 172.21.15.5:0/1427015212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafb40b4010 msgr2=0x7fafb415b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.806+0000 7fafbb516640 1 --2- 172.21.15.5:0/1427015212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafb40b4010 0x7fafb415b100 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fafb0000c00 tx=0x7fafb00046c0 comp rx=0 tx=0).stop 2024-05-20T18:27:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.807+0000 7fafbb516640 1 -- 172.21.15.5:0/1427015212 shutdown_connections 2024-05-20T18:27:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.807+0000 7fafbb516640 1 --2- 172.21.15.5:0/1427015212 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faf9806be60 0x7faf9806e370 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.807+0000 7fafbb516640 1 --2- 172.21.15.5:0/1427015212 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fafb4163a10 0x7fafb415a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.807+0000 7fafbb516640 1 --2- 172.21.15.5:0/1427015212 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafb40d14b0 0x7fafb4154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.807+0000 7fafbb516640 1 --2- 172.21.15.5:0/1427015212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafb40b4010 0x7fafb415b100 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.807+0000 7fafbb516640 1 -- 172.21.15.5:0/1427015212 >> 172.21.15.5:0/1427015212 conn(0x7fafb406d240 msgr2=0x7fafb415ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.807+0000 7fafbb516640 1 -- 172.21.15.5:0/1427015212 shutdown_connections 2024-05-20T18:27:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.807+0000 7fafbb516640 1 -- 172.21.15.5:0/1427015212 wait complete. 2024-05-20T18:27:56.818 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:27:05.476596+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:27:06.137762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T18:27:09.829920+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T18:27:56.818 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:27:56.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.936+0000 7f9ff4632640 1 Processor -- start 2024-05-20T18:27:56.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.936+0000 7f9ff4632640 1 -- start start 2024-05-20T18:27:56.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.937+0000 7f9ff4632640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9fec06cf00 0x7f9fec06d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:56.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.938+0000 7f9ff4632640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9fec0d2780 0x7f9fec0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:56.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.938+0000 7f9ff4632640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fec08d2a0 con 0x7f9fec0d4400 2024-05-20T18:27:56.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.938+0000 7f9ff4632640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f9fec0d3f70 con 0x7f9fec06cf00 2024-05-20T18:27:56.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.938+0000 7f9ff4632640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f9fec06d850 con 0x7f9fec0d2780 2024-05-20T18:27:56.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.938+0000 7f9ff1bcd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9fec06cf00 0x7f9fec06d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:56.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.938+0000 7f9ff23ce640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f9fec0d4400 0x7f9fec15f440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53962/0 (socket says 172.21.15.5:53962) 2024-05-20T18:27:56.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.938+0000 7f9ff23ce640 1 -- 172.21.15.5:0/3476225069 learned_addr learned my addr 172.21.15.5:0/3476225069 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:56.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.938+0000 7f9ff2bcf640 1 --2- 172.21.15.5:0/3476225069 >> v2:172.21.15.63:3300/0 conn(0x7f9fec0d2780 0x7f9fec0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:56.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.938+0000 7f9ff1bcd640 1 -- 172.21.15.5:0/3476225069 >> v2:172.21.15.63:3300/0 conn(0x7f9fec0d2780 msgr2=0x7f9fec0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:56.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.938+0000 7f9ff1bcd640 1 --2- 172.21.15.5:0/3476225069 >> v2:172.21.15.63:3300/0 conn(0x7f9fec0d2780 0x7f9fec0d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:56.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.938+0000 7f9ff1bcd640 1 -- 172.21.15.5:0/3476225069 >> v1:172.21.15.5:6789/0 conn(0x7f9fec0d4400 legacy=0x7f9fec15f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:56.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.938+0000 7f9ff1bcd640 1 -- 172.21.15.5:0/3476225069 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fec0d30d0 con 0x7f9fec06cf00 2024-05-20T18:27:56.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.938+0000 7f9ff13cc640 1 -- 172.21.15.5:0/3476225069 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2446873407 0 0) 0x7f9fe8001490 con 0x7f9fec0d4400 2024-05-20T18:27:56.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.938+0000 7f9ff2bcf640 1 --2- 172.21.15.5:0/3476225069 >> v2:172.21.15.63:3300/0 conn(0x7f9fec0d2780 0x7f9fec0d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:56.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.939+0000 7f9ff1bcd640 1 --2- 172.21.15.5:0/3476225069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fec06cf00 0x7f9fec06d310 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9fec08e020 tx=0x7f9fdc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3fde12f71aaeab server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:56.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.939+0000 7f9ff13cc640 1 -- 172.21.15.5:0/3476225069 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc037070 con 0x7f9fec06cf00 2024-05-20T18:27:56.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.939+0000 7f9ff13cc640 1 -- 172.21.15.5:0/3476225069 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fdc032440 con 0x7f9fec06cf00 2024-05-20T18:27:56.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.939+0000 7f9ff13cc640 1 -- 172.21.15.5:0/3476225069 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc032810 con 0x7f9fec06cf00 2024-05-20T18:27:56.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.940+0000 7f9ff4632640 1 -- 172.21.15.5:0/3476225069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fec06cf00 msgr2=0x7f9fec06d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:56.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.940+0000 7f9ff4632640 1 --2- 172.21.15.5:0/3476225069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fec06cf00 0x7f9fec06d310 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9fec08e020 tx=0x7f9fdc02a230 comp rx=0 tx=0).stop 2024-05-20T18:27:56.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.940+0000 7f9ff4632640 1 -- 172.21.15.5:0/3476225069 shutdown_connections 2024-05-20T18:27:56.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.940+0000 7f9ff4632640 1 --2- 172.21.15.5:0/3476225069 >> v2:172.21.15.63:3300/0 conn(0x7f9fec0d2780 0x7f9fec0d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:56.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.940+0000 7f9ff4632640 1 --2- 172.21.15.5:0/3476225069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fec06cf00 0x7f9fec06d310 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:56.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.940+0000 7f9ff4632640 1 -- 172.21.15.5:0/3476225069 >> 172.21.15.5:0/3476225069 conn(0x7f9fec0d4bb0 msgr2=0x7f9fec0d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:56.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.940+0000 7f9ff4632640 1 -- 172.21.15.5:0/3476225069 shutdown_connections 2024-05-20T18:27:56.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.940+0000 7f9ff4632640 1 -- 172.21.15.5:0/3476225069 wait complete. 2024-05-20T18:27:56.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.942+0000 7f9ff4632640 1 Processor -- start 2024-05-20T18:27:56.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.942+0000 7f9ff4632640 1 -- start start 2024-05-20T18:27:56.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.943+0000 7f9ff4632640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fec06cf00 0x7f9fec153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:56.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.943+0000 7f9ff4632640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fec0d2780 0x7f9fec153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:56.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.943+0000 7f9ff4632640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fec0d4400 0x7f9fec158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:56.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.943+0000 7f9ff4632640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9fec0d3640 con 0x7f9fec0d4400 2024-05-20T18:27:56.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.943+0000 7f9ff4632640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9fec0d3360 con 0x7f9fec0d2780 2024-05-20T18:27:56.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.943+0000 7f9ff4632640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9fec0d3920 con 0x7f9fec06cf00 2024-05-20T18:27:56.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.943+0000 7f9ff2bcf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fec0d4400 0x7f9fec158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:56.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.943+0000 7f9ff2bcf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fec0d4400 0x7f9fec158640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51026/0 (socket says 172.21.15.5:51026) 2024-05-20T18:27:56.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.943+0000 7f9ff2bcf640 1 -- 172.21.15.5:0/375918573 learned_addr learned my addr 172.21.15.5:0/375918573 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:56.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.943+0000 7f9ff1bcd640 1 --2- 172.21.15.5:0/375918573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fec0d2780 0x7f9fec153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:56.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.943+0000 7f9ff23ce640 1 --2- 172.21.15.5:0/375918573 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fec06cf00 0x7f9fec153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:56.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.944+0000 7f9ff2bcf640 1 -- 172.21.15.5:0/375918573 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fec06cf00 msgr2=0x7f9fec153a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:56.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.944+0000 7f9ff2bcf640 1 --2- 172.21.15.5:0/375918573 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fec06cf00 0x7f9fec153a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:56.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.944+0000 7f9ff2bcf640 1 -- 172.21.15.5:0/375918573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fec0d2780 msgr2=0x7f9fec153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:56.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.944+0000 7f9ff2bcf640 1 --2- 172.21.15.5:0/375918573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fec0d2780 0x7f9fec153f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:56.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.944+0000 7f9ff2bcf640 1 -- 172.21.15.5:0/375918573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fec152360 con 0x7f9fec0d4400 2024-05-20T18:27:56.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.944+0000 7f9ff23ce640 1 --2- 172.21.15.5:0/375918573 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fec06cf00 0x7f9fec153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:56.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.944+0000 7f9ff2bcf640 1 --2- 172.21.15.5:0/375918573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fec0d4400 0x7f9fec158640 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400d800 tx=0x7f9fe400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:56.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.944+0000 7f9fe2ffd640 1 -- 172.21.15.5:0/375918573 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4004030 con 0x7f9fec0d4400 2024-05-20T18:27:56.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.944+0000 7f9fe2ffd640 1 -- 172.21.15.5:0/375918573 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fe40041d0 con 0x7f9fec0d4400 2024-05-20T18:27:56.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.944+0000 7f9ff4632640 1 -- 172.21.15.5:0/375918573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fec152990 con 0x7f9fec0d4400 2024-05-20T18:27:56.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.944+0000 7f9fe2ffd640 1 -- 172.21.15.5:0/375918573 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe40096a0 con 0x7f9fec0d4400 2024-05-20T18:27:56.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.944+0000 7f9ff4632640 1 -- 172.21.15.5:0/375918573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fec152f50 con 0x7f9fec0d4400 2024-05-20T18:27:56.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.945+0000 7f9ff4632640 1 -- 172.21.15.5:0/375918573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fec0735a0 con 0x7f9fec0d4400 2024-05-20T18:27:56.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.946+0000 7f9fe2ffd640 1 -- 172.21.15.5:0/375918573 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9fe4009840 con 0x7f9fec0d4400 2024-05-20T18:27:56.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.946+0000 7f9fe2ffd640 1 --2- 172.21.15.5:0/375918573 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9fd006be30 0x7f9fd006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:56.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.946+0000 7f9fe2ffd640 1 -- 172.21.15.5:0/375918573 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7f9fe408b520 con 0x7f9fec0d4400 2024-05-20T18:27:56.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.946+0000 7f9ff23ce640 1 --2- 172.21.15.5:0/375918573 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9fd006be30 0x7f9fd006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:56.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.947+0000 7f9ff23ce640 1 --2- 172.21.15.5:0/375918573 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9fd006be30 0x7f9fd006e340 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9fe80012e0 tx=0x7f9fe8005b90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:56.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:56.951+0000 7f9fe2ffd640 1 -- 172.21.15.5:0/375918573 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9fec0735a0 con 0x7f9fec0d4400 2024-05-20T18:27:57.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.135+0000 7f9ff4632640 1 -- 172.21.15.5:0/375918573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9fec075b00 con 0x7f9fec0d4400 2024-05-20T18:27:57.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.136+0000 7f9fe2ffd640 1 -- 172.21.15.5:0/375918573 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f9fec152360 con 0x7f9fec0d4400 2024-05-20T18:27:57.135 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:27:57.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.137+0000 7f9ff4632640 1 -- 172.21.15.5:0/375918573 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9fd006be30 msgr2=0x7f9fd006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:57.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.137+0000 7f9ff4632640 1 --2- 172.21.15.5:0/375918573 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9fd006be30 0x7f9fd006e340 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9fe80012e0 tx=0x7f9fe8005b90 comp rx=0 tx=0).stop 2024-05-20T18:27:57.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.138+0000 7f9ff4632640 1 -- 172.21.15.5:0/375918573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fec0d4400 msgr2=0x7f9fec158640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:57.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.138+0000 7f9ff4632640 1 --2- 172.21.15.5:0/375918573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fec0d4400 0x7f9fec158640 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400d800 tx=0x7f9fe400dcc0 comp rx=0 tx=0).stop 2024-05-20T18:27:57.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.138+0000 7f9ff4632640 1 -- 172.21.15.5:0/375918573 shutdown_connections 2024-05-20T18:27:57.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.138+0000 7f9ff4632640 1 --2- 172.21.15.5:0/375918573 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9fd006be30 0x7f9fd006e340 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:57.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.138+0000 7f9ff4632640 1 --2- 172.21.15.5:0/375918573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fec0d4400 0x7f9fec158640 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:57.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.138+0000 7f9ff4632640 1 --2- 172.21.15.5:0/375918573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9fec0d2780 0x7f9fec153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:57.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.138+0000 7f9ff4632640 1 --2- 172.21.15.5:0/375918573 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9fec06cf00 0x7f9fec153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:57.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.138+0000 7f9ff4632640 1 -- 172.21.15.5:0/375918573 >> 172.21.15.5:0/375918573 conn(0x7f9fec0d4bb0 msgr2=0x7f9fec161f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:57.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.138+0000 7f9ff4632640 1 -- 172.21.15.5:0/375918573 shutdown_connections 2024-05-20T18:27:57.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.138+0000 7f9ff4632640 1 -- 172.21.15.5:0/375918573 wait complete. 2024-05-20T18:27:57.148 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:27:57.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.267+0000 7f9649ca2640 1 Processor -- start 2024-05-20T18:27:57.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.267+0000 7f9649ca2640 1 -- start start 2024-05-20T18:27:57.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.269+0000 7f9649ca2640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f964406ca80 0x7f964415f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:57.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.269+0000 7f9649ca2640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96440ae320 0x7f964406bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:57.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.269+0000 7f9649ca2640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96440b1b80 con 0x7f96440ae730 2024-05-20T18:27:57.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.269+0000 7f9649ca2640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f964406b750 con 0x7f964406ca80 2024-05-20T18:27:57.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.269+0000 7f9649ca2640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f96440b12e0 con 0x7f96440ae320 2024-05-20T18:27:57.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.269+0000 7f96437fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f964406ca80 0x7f964415f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:57.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.269+0000 7f96437fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f964406ca80 0x7f964415f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51038/0 (socket says 172.21.15.5:51038) 2024-05-20T18:27:57.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.269+0000 7f9643fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f96440ae730 0x7f964406c120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53978/0 (socket says 172.21.15.5:53978) 2024-05-20T18:27:57.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.269+0000 7f96437fe640 1 -- 172.21.15.5:0/4250018605 learned_addr learned my addr 172.21.15.5:0/4250018605 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:57.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.269+0000 7f9642ffd640 1 --2- 172.21.15.5:0/4250018605 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96440ae320 0x7f964406bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:57.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.269+0000 7f96437fe640 1 -- 172.21.15.5:0/4250018605 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96440ae320 msgr2=0x7f964406bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:57.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.269+0000 7f96437fe640 1 --2- 172.21.15.5:0/4250018605 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96440ae320 0x7f964406bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:57.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.269+0000 7f96437fe640 1 -- 172.21.15.5:0/4250018605 >> v1:172.21.15.5:6789/0 conn(0x7f96440ae730 legacy=0x7f964406c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:57.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.269+0000 7f96437fe640 1 -- 172.21.15.5:0/4250018605 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96440b15c0 con 0x7f964406ca80 2024-05-20T18:27:57.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.269+0000 7f96427fc640 1 -- 172.21.15.5:0/4250018605 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2023384741 0 0) 0x7f963c001490 con 0x7f96440ae730 2024-05-20T18:27:57.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.270+0000 7f96437fe640 1 --2- 172.21.15.5:0/4250018605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f964406ca80 0x7f964415f280 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9634009870 tx=0x7f963402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa7e1f49d2a1a01 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:57.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.270+0000 7f96427fc640 1 -- 172.21.15.5:0/4250018605 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9634037070 con 0x7f964406ca80 2024-05-20T18:27:57.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.270+0000 7f96427fc640 1 -- 172.21.15.5:0/4250018605 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9634032440 con 0x7f964406ca80 2024-05-20T18:27:57.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.270+0000 7f96427fc640 1 -- 172.21.15.5:0/4250018605 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9634032810 con 0x7f964406ca80 2024-05-20T18:27:57.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.271+0000 7f9649ca2640 1 -- 172.21.15.5:0/4250018605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f964406ca80 msgr2=0x7f964415f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:57.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.271+0000 7f9649ca2640 1 --2- 172.21.15.5:0/4250018605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f964406ca80 0x7f964415f280 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9634009870 tx=0x7f963402a270 comp rx=0 tx=0).stop 2024-05-20T18:27:57.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.271+0000 7f9649ca2640 1 -- 172.21.15.5:0/4250018605 shutdown_connections 2024-05-20T18:27:57.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.271+0000 7f9649ca2640 1 --2- 172.21.15.5:0/4250018605 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96440ae320 0x7f964406bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:57.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.271+0000 7f9649ca2640 1 --2- 172.21.15.5:0/4250018605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f964406ca80 0x7f964415f280 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:57.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.271+0000 7f9649ca2640 1 -- 172.21.15.5:0/4250018605 >> 172.21.15.5:0/4250018605 conn(0x7f9644072ed0 msgr2=0x7f96440b6080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:57.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.271+0000 7f9649ca2640 1 -- 172.21.15.5:0/4250018605 shutdown_connections 2024-05-20T18:27:57.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.271+0000 7f9649ca2640 1 -- 172.21.15.5:0/4250018605 wait complete. 2024-05-20T18:27:57.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.273+0000 7f9649ca2640 1 Processor -- start 2024-05-20T18:27:57.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.273+0000 7f9649ca2640 1 -- start start 2024-05-20T18:27:57.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.274+0000 7f9649ca2640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f964406ca80 0x7f96441527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:57.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.274+0000 7f9649ca2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96440ae320 0x7f9644153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:57.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.274+0000 7f9649ca2640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96440ae730 0x7f96441541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:57.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.274+0000 7f9649ca2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f96440d0c10 con 0x7f96440ae320 2024-05-20T18:27:57.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.274+0000 7f9649ca2640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9644074e00 con 0x7f96440ae730 2024-05-20T18:27:57.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.274+0000 7f9649ca2640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f96440d0d90 con 0x7f964406ca80 2024-05-20T18:27:57.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.274+0000 7f9642ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96440ae320 0x7f9644153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:57.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.274+0000 7f9642ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96440ae320 0x7f9644153cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51052/0 (socket says 172.21.15.5:51052) 2024-05-20T18:27:57.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.274+0000 7f9642ffd640 1 -- 172.21.15.5:0/1314035471 learned_addr learned my addr 172.21.15.5:0/1314035471 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:57.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.274+0000 7f96437fe640 1 --2- 172.21.15.5:0/1314035471 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f964406ca80 0x7f96441527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:57.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.274+0000 7f9643fff640 1 --2- 172.21.15.5:0/1314035471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96440ae730 0x7f96441541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:57.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.275+0000 7f9642ffd640 1 -- 172.21.15.5:0/1314035471 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f964406ca80 msgr2=0x7f96441527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:57.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.275+0000 7f9642ffd640 1 --2- 172.21.15.5:0/1314035471 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f964406ca80 0x7f96441527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:57.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.275+0000 7f9642ffd640 1 -- 172.21.15.5:0/1314035471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96440ae730 msgr2=0x7f96441541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:57.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.275+0000 7f9642ffd640 1 --2- 172.21.15.5:0/1314035471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96440ae730 0x7f96441541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:57.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.275+0000 7f9642ffd640 1 -- 172.21.15.5:0/1314035471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f964414fe90 con 0x7f96440ae320 2024-05-20T18:27:57.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.275+0000 7f9643fff640 1 --2- 172.21.15.5:0/1314035471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96440ae730 0x7f96441541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:57.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.275+0000 7f9642ffd640 1 --2- 172.21.15.5:0/1314035471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96440ae320 0x7f9644153cb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f963800e8a0 tx=0x7f963800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:57.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.275+0000 7f962bfff640 1 -- 172.21.15.5:0/1314035471 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9638017070 con 0x7f96440ae320 2024-05-20T18:27:57.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.275+0000 7f962bfff640 1 -- 172.21.15.5:0/1314035471 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f963800c940 con 0x7f96440ae320 2024-05-20T18:27:57.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.275+0000 7f962bfff640 1 -- 172.21.15.5:0/1314035471 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963800cd30 con 0x7f96440ae320 2024-05-20T18:27:57.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.275+0000 7f9649ca2640 1 -- 172.21.15.5:0/1314035471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9644150130 con 0x7f96440ae320 2024-05-20T18:27:57.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.275+0000 7f9649ca2640 1 -- 172.21.15.5:0/1314035471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96441506f0 con 0x7f96440ae320 2024-05-20T18:27:57.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.276+0000 7f9649ca2640 1 -- 172.21.15.5:0/1314035471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9644075b00 con 0x7f96440ae320 2024-05-20T18:27:57.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.277+0000 7f962bfff640 1 -- 172.21.15.5:0/1314035471 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f963801a070 con 0x7f96440ae320 2024-05-20T18:27:57.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.277+0000 7f962bfff640 1 --2- 172.21.15.5:0/1314035471 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f963006be30 0x7f963006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:57.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.277+0000 7f962bfff640 1 -- 172.21.15.5:0/1314035471 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7f963808b110 con 0x7f96440ae320 2024-05-20T18:27:57.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.277+0000 7f96437fe640 1 --2- 172.21.15.5:0/1314035471 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f963006be30 0x7f963006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:57.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.278+0000 7f96437fe640 1 --2- 172.21.15.5:0/1314035471 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f963006be30 0x7f963006e340 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9634034000 tx=0x7f963402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:57.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.282+0000 7f962bfff640 1 -- 172.21.15.5:0/1314035471 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9638058fd0 con 0x7f96440ae320 2024-05-20T18:27:57.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.465+0000 7f9649ca2640 1 -- 172.21.15.5:0/1314035471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f96441629d0 con 0x7f96440ae320 2024-05-20T18:27:57.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.465+0000 7f962bfff640 1 -- 172.21.15.5:0/1314035471 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f963805d2d0 con 0x7f96440ae320 2024-05-20T18:27:57.465 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:27:57.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.468+0000 7f9649ca2640 1 -- 172.21.15.5:0/1314035471 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f963006be30 msgr2=0x7f963006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:57.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.468+0000 7f9649ca2640 1 --2- 172.21.15.5:0/1314035471 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f963006be30 0x7f963006e340 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9634034000 tx=0x7f963402e000 comp rx=0 tx=0).stop 2024-05-20T18:27:57.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.468+0000 7f9649ca2640 1 -- 172.21.15.5:0/1314035471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96440ae320 msgr2=0x7f9644153cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:57.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.468+0000 7f9649ca2640 1 --2- 172.21.15.5:0/1314035471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96440ae320 0x7f9644153cb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f963800e8a0 tx=0x7f963800ed60 comp rx=0 tx=0).stop 2024-05-20T18:27:57.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.468+0000 7f9649ca2640 1 -- 172.21.15.5:0/1314035471 shutdown_connections 2024-05-20T18:27:57.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.468+0000 7f9649ca2640 1 --2- 172.21.15.5:0/1314035471 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f963006be30 0x7f963006e340 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:57.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.468+0000 7f9649ca2640 1 --2- 172.21.15.5:0/1314035471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96440ae730 0x7f96441541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:57.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.468+0000 7f9649ca2640 1 --2- 172.21.15.5:0/1314035471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96440ae320 0x7f9644153cb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:57.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.468+0000 7f9649ca2640 1 --2- 172.21.15.5:0/1314035471 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f964406ca80 0x7f96441527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:57.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.468+0000 7f9649ca2640 1 -- 172.21.15.5:0/1314035471 >> 172.21.15.5:0/1314035471 conn(0x7f9644072ed0 msgr2=0x7f96440b0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:57.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.468+0000 7f9649ca2640 1 -- 172.21.15.5:0/1314035471 shutdown_connections 2024-05-20T18:27:57.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.468+0000 7f9649ca2640 1 -- 172.21.15.5:0/1314035471 wait complete. 2024-05-20T18:27:57.479 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:27:57.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.597+0000 7fab78d9d640 1 Processor -- start 2024-05-20T18:27:57.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.598+0000 7fab78d9d640 1 -- start start 2024-05-20T18:27:57.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.599+0000 7fab78d9d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fab740cf860 0x7fab740b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:57.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.599+0000 7fab78d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fab74163520 0x7fab740d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:57.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.599+0000 7fab78d9d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab741605a0 con 0x7fab74163930 2024-05-20T18:27:57.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.599+0000 7fab78d9d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fab7415ffe0 con 0x7fab74163520 2024-05-20T18:27:57.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.599+0000 7fab78d9d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fab741602c0 con 0x7fab740cf860 2024-05-20T18:27:57.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.599+0000 7fab72ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fab74163520 0x7fab740d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:57.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.599+0000 7fab73fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fab74163930 0x7fab740d3140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53988/0 (socket says 172.21.15.5:53988) 2024-05-20T18:27:57.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.599+0000 7fab73fff640 1 -- 172.21.15.5:0/4278552314 learned_addr learned my addr 172.21.15.5:0/4278552314 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:57.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.599+0000 7fab72ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fab74163520 0x7fab740d12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51066/0 (socket says 172.21.15.5:51066) 2024-05-20T18:27:57.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.599+0000 7fab737fe640 1 --2- 172.21.15.5:0/4278552314 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fab740cf860 0x7fab740b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:57.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.599+0000 7fab72ffd640 1 -- 172.21.15.5:0/4278552314 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fab740cf860 msgr2=0x7fab740b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:57.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.599+0000 7fab72ffd640 1 --2- 172.21.15.5:0/4278552314 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fab740cf860 0x7fab740b3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:57.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.599+0000 7fab72ffd640 1 -- 172.21.15.5:0/4278552314 >> v1:172.21.15.5:6789/0 conn(0x7fab74163930 legacy=0x7fab740d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:57.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.599+0000 7fab72ffd640 1 -- 172.21.15.5:0/4278552314 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab74162d20 con 0x7fab74163520 2024-05-20T18:27:57.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.599+0000 7fab727fc640 1 -- 172.21.15.5:0/4278552314 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 392309071 0 0) 0x7fab741605a0 con 0x7fab74163930 2024-05-20T18:27:57.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.600+0000 7fab72ffd640 1 --2- 172.21.15.5:0/4278552314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab74163520 0x7fab740d12a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fab680097a0 tx=0x7fab6802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4014a8ad184b14b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:57.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.600+0000 7fab727fc640 1 -- 172.21.15.5:0/4278552314 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab68037070 con 0x7fab74163520 2024-05-20T18:27:57.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.600+0000 7fab727fc640 1 -- 172.21.15.5:0/4278552314 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab6802ae50 con 0x7fab74163520 2024-05-20T18:27:57.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.600+0000 7fab727fc640 1 -- 172.21.15.5:0/4278552314 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab68032660 con 0x7fab74163520 2024-05-20T18:27:57.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.601+0000 7fab78d9d640 1 -- 172.21.15.5:0/4278552314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab74163520 msgr2=0x7fab740d12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:57.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.601+0000 7fab78d9d640 1 --2- 172.21.15.5:0/4278552314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab74163520 0x7fab740d12a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fab680097a0 tx=0x7fab6802a100 comp rx=0 tx=0).stop 2024-05-20T18:27:57.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.601+0000 7fab78d9d640 1 -- 172.21.15.5:0/4278552314 shutdown_connections 2024-05-20T18:27:57.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.601+0000 7fab78d9d640 1 --2- 172.21.15.5:0/4278552314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab74163520 0x7fab740d12a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:57.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.601+0000 7fab78d9d640 1 --2- 172.21.15.5:0/4278552314 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fab740cf860 0x7fab740b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:57.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.601+0000 7fab78d9d640 1 -- 172.21.15.5:0/4278552314 >> 172.21.15.5:0/4278552314 conn(0x7fab7406dbb0 msgr2=0x7fab7406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:57.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.601+0000 7fab78d9d640 1 -- 172.21.15.5:0/4278552314 shutdown_connections 2024-05-20T18:27:57.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.601+0000 7fab78d9d640 1 -- 172.21.15.5:0/4278552314 wait complete. 2024-05-20T18:27:57.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.603+0000 7fab78d9d640 1 Processor -- start 2024-05-20T18:27:57.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.603+0000 7fab78d9d640 1 -- start start 2024-05-20T18:27:57.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.604+0000 7fab78d9d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fab740cf860 0x7fab740ce220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:57.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.604+0000 7fab78d9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab74163520 0x7fab740b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:57.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.604+0000 7fab78d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab74163930 0x7fab74161510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:57.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.604+0000 7fab78d9d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fab740d39d0 con 0x7fab74163930 2024-05-20T18:27:57.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.604+0000 7fab78d9d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fab740d3850 con 0x7fab74163520 2024-05-20T18:27:57.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.604+0000 7fab78d9d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fab740d3b50 con 0x7fab740cf860 2024-05-20T18:27:57.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.605+0000 7fab73fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab74163930 0x7fab74161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:57.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.605+0000 7fab73fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab74163930 0x7fab74161510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51070/0 (socket says 172.21.15.5:51070) 2024-05-20T18:27:57.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.605+0000 7fab73fff640 1 -- 172.21.15.5:0/3181409002 learned_addr learned my addr 172.21.15.5:0/3181409002 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:57.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.605+0000 7fab72ffd640 1 --2- 172.21.15.5:0/3181409002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab74163520 0x7fab740b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:57.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.605+0000 7fab737fe640 1 --2- 172.21.15.5:0/3181409002 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fab740cf860 0x7fab740ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:57.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.605+0000 7fab73fff640 1 -- 172.21.15.5:0/3181409002 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fab740cf860 msgr2=0x7fab740ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:57.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.605+0000 7fab73fff640 1 --2- 172.21.15.5:0/3181409002 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fab740cf860 0x7fab740ce220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:57.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.605+0000 7fab73fff640 1 -- 172.21.15.5:0/3181409002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab74163520 msgr2=0x7fab740b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:57.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.605+0000 7fab73fff640 1 --2- 172.21.15.5:0/3181409002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab74163520 0x7fab740b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:57.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.605+0000 7fab73fff640 1 -- 172.21.15.5:0/3181409002 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab740cd320 con 0x7fab74163930 2024-05-20T18:27:57.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.605+0000 7fab72ffd640 1 --2- 172.21.15.5:0/3181409002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab74163520 0x7fab740b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:57.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.605+0000 7fab737fe640 1 --2- 172.21.15.5:0/3181409002 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fab740cf860 0x7fab740ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:27:57.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.605+0000 7fab73fff640 1 --2- 172.21.15.5:0/3181409002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab74163930 0x7fab74161510 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00c780 tx=0x7fab6c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:57.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.605+0000 7fab5bfff640 1 -- 172.21.15.5:0/3181409002 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c018070 con 0x7fab74163930 2024-05-20T18:27:57.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.605+0000 7fab5bfff640 1 -- 172.21.15.5:0/3181409002 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab6c00ce30 con 0x7fab74163930 2024-05-20T18:27:57.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.605+0000 7fab78d9d640 1 -- 172.21.15.5:0/3181409002 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab740cd600 con 0x7fab74163930 2024-05-20T18:27:57.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.605+0000 7fab5bfff640 1 -- 172.21.15.5:0/3181409002 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c00a840 con 0x7fab74163930 2024-05-20T18:27:57.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.605+0000 7fab78d9d640 1 -- 172.21.15.5:0/3181409002 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab740b4520 con 0x7fab74163930 2024-05-20T18:27:57.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.606+0000 7fab78d9d640 1 -- 172.21.15.5:0/3181409002 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab74075980 con 0x7fab74163930 2024-05-20T18:27:57.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.607+0000 7fab5bfff640 1 -- 172.21.15.5:0/3181409002 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fab6c01b070 con 0x7fab74163930 2024-05-20T18:27:57.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.607+0000 7fab5bfff640 1 --2- 172.21.15.5:0/3181409002 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fab6006be30 0x7fab6006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:57.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.608+0000 7fab5bfff640 1 -- 172.21.15.5:0/3181409002 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7fab6c08a850 con 0x7fab74163930 2024-05-20T18:27:57.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.608+0000 7fab737fe640 1 --2- 172.21.15.5:0/3181409002 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fab6006be30 0x7fab6006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:57.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.608+0000 7fab737fe640 1 --2- 172.21.15.5:0/3181409002 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fab6006be30 0x7fab6006e340 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fab740d11f0 tx=0x7fab64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:57.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.612+0000 7fab5bfff640 1 -- 172.21.15.5:0/3181409002 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fab6c058d50 con 0x7fab74163930 2024-05-20T18:27:57.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.796+0000 7fab78d9d640 1 -- 172.21.15.5:0/3181409002 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fab740d2f20 con 0x7fab74163930 2024-05-20T18:27:57.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.797+0000 7fab5bfff640 1 -- 172.21.15.5:0/3181409002 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7fab6c058650 con 0x7fab74163930 2024-05-20T18:27:57.798 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:27:57.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.799+0000 7fab78d9d640 1 -- 172.21.15.5:0/3181409002 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fab6006be30 msgr2=0x7fab6006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:57.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.799+0000 7fab78d9d640 1 --2- 172.21.15.5:0/3181409002 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fab6006be30 0x7fab6006e340 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fab740d11f0 tx=0x7fab64008040 comp rx=0 tx=0).stop 2024-05-20T18:27:57.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.799+0000 7fab78d9d640 1 -- 172.21.15.5:0/3181409002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab74163930 msgr2=0x7fab74161510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:57.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.799+0000 7fab78d9d640 1 --2- 172.21.15.5:0/3181409002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab74163930 0x7fab74161510 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00c780 tx=0x7fab6c00cc40 comp rx=0 tx=0).stop 2024-05-20T18:27:57.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.799+0000 7fab78d9d640 1 -- 172.21.15.5:0/3181409002 shutdown_connections 2024-05-20T18:27:57.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.799+0000 7fab78d9d640 1 --2- 172.21.15.5:0/3181409002 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fab6006be30 0x7fab6006e340 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:57.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.799+0000 7fab78d9d640 1 --2- 172.21.15.5:0/3181409002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab74163930 0x7fab74161510 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:57.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.799+0000 7fab78d9d640 1 --2- 172.21.15.5:0/3181409002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fab74163520 0x7fab740b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:57.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.799+0000 7fab78d9d640 1 --2- 172.21.15.5:0/3181409002 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fab740cf860 0x7fab740ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:57.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.799+0000 7fab78d9d640 1 -- 172.21.15.5:0/3181409002 >> 172.21.15.5:0/3181409002 conn(0x7fab7406dbb0 msgr2=0x7fab740b8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:57.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.799+0000 7fab78d9d640 1 -- 172.21.15.5:0/3181409002 shutdown_connections 2024-05-20T18:27:57.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.799+0000 7fab78d9d640 1 -- 172.21.15.5:0/3181409002 wait complete. 2024-05-20T18:27:57.809 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T18:27:57.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.928+0000 7f0cbb39d640 1 Processor -- start 2024-05-20T18:27:57.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.929+0000 7f0cbb39d640 1 -- start start 2024-05-20T18:27:57.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.930+0000 7f0cbb39d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cb41599a0 0x7f0cb40b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:57.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.930+0000 7f0cbb39d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0cb406d980 0x7f0cb409a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:57.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.930+0000 7f0cbb39d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cb409ae40 con 0x7f0cb409aa30 2024-05-20T18:27:57.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.930+0000 7f0cbb39d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0cb41607a0 con 0x7f0cb406d980 2024-05-20T18:27:57.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.930+0000 7f0cbb39d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0cb409b2d0 con 0x7f0cb41599a0 2024-05-20T18:27:57.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.930+0000 7f0cb993a640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0cb409aa30 0x7f0cb40d18b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53996/0 (socket says 172.21.15.5:53996) 2024-05-20T18:27:57.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.930+0000 7f0cb993a640 1 -- 172.21.15.5:0/3990547336 learned_addr learned my addr 172.21.15.5:0/3990547336 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:57.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.930+0000 7f0cb9139640 1 --2- 172.21.15.5:0/3990547336 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cb41599a0 0x7f0cb40b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:57.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.931+0000 7f0cb3fff640 1 -- 172.21.15.5:0/3990547336 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1306322549 0 0) 0x7f0ca4001490 con 0x7f0cb409aa30 2024-05-20T18:27:57.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.931+0000 7f0cb8938640 1 --2- 172.21.15.5:0/3990547336 >> v2:172.21.15.63:3300/0 conn(0x7f0cb406d980 0x7f0cb409a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:57.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.931+0000 7f0cb3fff640 1 -- 172.21.15.5:0/3990547336 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c9c003520 con 0x7f0cb409aa30 2024-05-20T18:27:57.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.931+0000 7f0cb3fff640 1 -- 172.21.15.5:0/3990547336 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2469713692 0 0) 0x7f0c9c003520 con 0x7f0cb409aa30 2024-05-20T18:27:57.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.931+0000 7f0cb3fff640 1 -- 172.21.15.5:0/3990547336 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cb41599a0 msgr2=0x7f0cb40b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:57.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.931+0000 7f0cb3fff640 1 --2- 172.21.15.5:0/3990547336 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cb41599a0 0x7f0cb40b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:57.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.931+0000 7f0cb3fff640 1 -- 172.21.15.5:0/3990547336 >> v2:172.21.15.63:3300/0 conn(0x7f0cb406d980 msgr2=0x7f0cb409a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:57.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.931+0000 7f0cb3fff640 1 --2- 172.21.15.5:0/3990547336 >> v2:172.21.15.63:3300/0 conn(0x7f0cb406d980 0x7f0cb409a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:57.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.931+0000 7f0cb3fff640 1 -- 172.21.15.5:0/3990547336 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb415fc20 con 0x7f0cb409aa30 2024-05-20T18:27:57.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.931+0000 7f0cb3fff640 1 -- 172.21.15.5:0/3990547336 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f0ca4001bc0 con 0x7f0cb409aa30 2024-05-20T18:27:57.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.931+0000 7f0cb3fff640 1 -- 172.21.15.5:0/3990547336 >> v1:172.21.15.5:6789/0 conn(0x7f0cb409aa30 legacy=0x7f0cb40d18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:57.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.931+0000 7f0cb3fff640 1 --2- 172.21.15.5:0/3990547336 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cb41599a0 0x7f0c9c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:57.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.931+0000 7f0cb3fff640 1 --2- 172.21.15.5:0/3990547336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c9c003f30 0x7f0c9c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:57.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.931+0000 7f0cb3fff640 1 --2- 172.21.15.5:0/3990547336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c9c004d40 0x7f0c9c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:57.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.931+0000 7f0cb3fff640 1 -- 172.21.15.5:0/3990547336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb41607a0 con 0x7f0c9c004d40 2024-05-20T18:27:57.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.931+0000 7f0cb3fff640 1 -- 172.21.15.5:0/3990547336 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb409b2d0 con 0x7f0c9c003f30 2024-05-20T18:27:57.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.931+0000 7f0cb3fff640 1 -- 172.21.15.5:0/3990547336 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0c9c00bac0 con 0x7f0cb41599a0 2024-05-20T18:27:57.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.931+0000 7f0cb993a640 1 --2- 172.21.15.5:0/3990547336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c9c004d40 0x7f0c9c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:57.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.931+0000 7f0cb8938640 1 --2- 172.21.15.5:0/3990547336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c9c003f30 0x7f0c9c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:57.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.931+0000 7f0cb9139640 1 --2- 172.21.15.5:0/3990547336 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cb41599a0 0x7f0c9c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:57.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.931+0000 7f0cb993a640 1 -- 172.21.15.5:0/3990547336 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cb41599a0 msgr2=0x7f0c9c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:57.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.931+0000 7f0cb993a640 1 --2- 172.21.15.5:0/3990547336 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cb41599a0 0x7f0c9c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:57.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.931+0000 7f0cb993a640 1 -- 172.21.15.5:0/3990547336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c9c003f30 msgr2=0x7f0c9c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:57.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.931+0000 7f0cb993a640 1 --2- 172.21.15.5:0/3990547336 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c9c003f30 0x7f0c9c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:57.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.931+0000 7f0cb993a640 1 -- 172.21.15.5:0/3990547336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c9c003520 con 0x7f0c9c004d40 2024-05-20T18:27:57.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.932+0000 7f0cb993a640 1 --2- 172.21.15.5:0/3990547336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c9c004d40 0x7f0c9c005150 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0ca400aa50 tx=0x7f0c9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22d11ba111d1850 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:57.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.932+0000 7f0cb3fff640 1 -- 172.21.15.5:0/3990547336 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca400a670 con 0x7f0c9c004d40 2024-05-20T18:27:57.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.932+0000 7f0cb3fff640 1 -- 172.21.15.5:0/3990547336 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ca402f070 con 0x7f0c9c004d40 2024-05-20T18:27:57.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.932+0000 7f0cb3fff640 1 -- 172.21.15.5:0/3990547336 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4034470 con 0x7f0c9c004d40 2024-05-20T18:27:57.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.933+0000 7f0cbb39d640 1 -- 172.21.15.5:0/3990547336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c9c004d40 msgr2=0x7f0c9c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:57.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.933+0000 7f0cbb39d640 1 --2- 172.21.15.5:0/3990547336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c9c004d40 0x7f0c9c005150 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0ca400aa50 tx=0x7f0c9c005830 comp rx=0 tx=0).stop 2024-05-20T18:27:57.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.933+0000 7f0cb993a640 1 -- 172.21.15.5:0/3990547336 reap_dead start 2024-05-20T18:27:57.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.933+0000 7f0cbb39d640 1 -- 172.21.15.5:0/3990547336 shutdown_connections 2024-05-20T18:27:57.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.933+0000 7f0cbb39d640 1 -- 172.21.15.5:0/3990547336 >> 172.21.15.5:0/3990547336 conn(0x7f0cb40b6670 msgr2=0x7f0cb40b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:57.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.933+0000 7f0cbb39d640 1 -- 172.21.15.5:0/3990547336 shutdown_connections 2024-05-20T18:27:57.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.933+0000 7f0cbb39d640 1 -- 172.21.15.5:0/3990547336 wait complete. 2024-05-20T18:27:57.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.935+0000 7f0cbb39d640 1 Processor -- start 2024-05-20T18:27:57.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.935+0000 7f0cbb39d640 1 -- start start 2024-05-20T18:27:57.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.936+0000 7f0cbb39d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cb4162000 0x7f0cb4162410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:57.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.936+0000 7f0cbb39d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cb415aeb0 0x7f0cb415b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:57.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.936+0000 7f0cbb39d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb406c570 0x7f0cb406c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:57.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.936+0000 7f0cbb39d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb41604c0 con 0x7f0cb406c570 2024-05-20T18:27:57.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.936+0000 7f0cbb39d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb41601e0 con 0x7f0cb415aeb0 2024-05-20T18:27:57.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.936+0000 7f0cbb39d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0cb406dd90 con 0x7f0cb4162000 2024-05-20T18:27:57.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.936+0000 7f0cb993a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb406c570 0x7f0cb406c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:57.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.936+0000 7f0cb993a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb406c570 0x7f0cb406c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51094/0 (socket says 172.21.15.5:51094) 2024-05-20T18:27:57.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.936+0000 7f0cb993a640 1 -- 172.21.15.5:0/2713505079 learned_addr learned my addr 172.21.15.5:0/2713505079 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:57.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.936+0000 7f0cb9139640 1 --2- 172.21.15.5:0/2713505079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cb4162000 0x7f0cb4162410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:57.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.937+0000 7f0cb8938640 1 --2- 172.21.15.5:0/2713505079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cb415aeb0 0x7f0cb415b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:57.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.937+0000 7f0cb993a640 1 -- 172.21.15.5:0/2713505079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cb4162000 msgr2=0x7f0cb4162410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:57.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.937+0000 7f0cb993a640 1 --2- 172.21.15.5:0/2713505079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cb4162000 0x7f0cb4162410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:57.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.937+0000 7f0cb993a640 1 -- 172.21.15.5:0/2713505079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cb415aeb0 msgr2=0x7f0cb415b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:57.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.937+0000 7f0cb993a640 1 --2- 172.21.15.5:0/2713505079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cb415aeb0 0x7f0cb415b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:57.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.937+0000 7f0cb993a640 1 -- 172.21.15.5:0/2713505079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb406cec0 con 0x7f0cb406c570 2024-05-20T18:27:57.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.937+0000 7f0cb8938640 1 --2- 172.21.15.5:0/2713505079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cb415aeb0 0x7f0cb415b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:27:57.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.937+0000 7f0cb993a640 1 --2- 172.21.15.5:0/2713505079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb406c570 0x7f0cb406c980 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0ca400aa50 tx=0x7f0cb415fdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:57.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.937+0000 7f0cb1ffb640 1 -- 172.21.15.5:0/2713505079 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca400a670 con 0x7f0cb406c570 2024-05-20T18:27:57.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.937+0000 7f0cb1ffb640 1 -- 172.21.15.5:0/2713505079 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ca4001bb0 con 0x7f0cb406c570 2024-05-20T18:27:57.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.937+0000 7f0cbb39d640 1 -- 172.21.15.5:0/2713505079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb4159e50 con 0x7f0cb406c570 2024-05-20T18:27:57.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.937+0000 7f0cb1ffb640 1 -- 172.21.15.5:0/2713505079 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4034870 con 0x7f0cb406c570 2024-05-20T18:27:57.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.937+0000 7f0cbb39d640 1 -- 172.21.15.5:0/2713505079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb415a3d0 con 0x7f0cb406c570 2024-05-20T18:27:57.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.939+0000 7f0cbb39d640 1 -- 172.21.15.5:0/2713505079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cb4075b00 con 0x7f0cb406c570 2024-05-20T18:27:57.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.939+0000 7f0cb1ffb640 1 -- 172.21.15.5:0/2713505079 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0ca4046060 con 0x7f0cb406c570 2024-05-20T18:27:57.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.940+0000 7f0cb1ffb640 1 --2- 172.21.15.5:0/2713505079 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0ca006be60 0x7f0ca006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:57.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.940+0000 7f0cb1ffb640 1 -- 172.21.15.5:0/2713505079 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7f0ca40aaf90 con 0x7f0cb406c570 2024-05-20T18:27:57.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.940+0000 7f0cb9139640 1 --2- 172.21.15.5:0/2713505079 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0ca006be60 0x7f0ca006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:57.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.940+0000 7f0cb9139640 1 --2- 172.21.15.5:0/2713505079 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0ca006be60 0x7f0ca006e370 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0cb40b2f70 tx=0x7f0cac009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:57.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:57.944+0000 7f0cb1ffb640 1 -- 172.21.15.5:0/2713505079 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0cb406cec0 con 0x7f0cb406c570 2024-05-20T18:27:58.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.125+0000 7f0cbb39d640 1 -- 172.21.15.5:0/2713505079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0cb40af860 con 0x7f0cb406c570 2024-05-20T18:27:58.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.125+0000 7f0cb1ffb640 1 -- 172.21.15.5:0/2713505079 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f0cb409ae40 con 0x7f0cb406c570 2024-05-20T18:27:58.125 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:27:58.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.127+0000 7f0cbb39d640 1 -- 172.21.15.5:0/2713505079 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0ca006be60 msgr2=0x7f0ca006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:58.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.127+0000 7f0cbb39d640 1 --2- 172.21.15.5:0/2713505079 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0ca006be60 0x7f0ca006e370 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0cb40b2f70 tx=0x7f0cac009190 comp rx=0 tx=0).stop 2024-05-20T18:27:58.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.127+0000 7f0cbb39d640 1 -- 172.21.15.5:0/2713505079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb406c570 msgr2=0x7f0cb406c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:58.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.127+0000 7f0cbb39d640 1 --2- 172.21.15.5:0/2713505079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb406c570 0x7f0cb406c980 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0ca400aa50 tx=0x7f0cb415fdf0 comp rx=0 tx=0).stop 2024-05-20T18:27:58.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.127+0000 7f0cbb39d640 1 -- 172.21.15.5:0/2713505079 shutdown_connections 2024-05-20T18:27:58.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.127+0000 7f0cbb39d640 1 --2- 172.21.15.5:0/2713505079 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0ca006be60 0x7f0ca006e370 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:58.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.127+0000 7f0cbb39d640 1 --2- 172.21.15.5:0/2713505079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb406c570 0x7f0cb406c980 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:58.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.127+0000 7f0cbb39d640 1 --2- 172.21.15.5:0/2713505079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0cb415aeb0 0x7f0cb415b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:58.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.127+0000 7f0cbb39d640 1 --2- 172.21.15.5:0/2713505079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0cb4162000 0x7f0cb4162410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:58.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.127+0000 7f0cbb39d640 1 -- 172.21.15.5:0/2713505079 >> 172.21.15.5:0/2713505079 conn(0x7f0cb40b6670 msgr2=0x7f0cb40700c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:58.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.128+0000 7f0cbb39d640 1 -- 172.21.15.5:0/2713505079 shutdown_connections 2024-05-20T18:27:58.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.128+0000 7f0cbb39d640 1 -- 172.21.15.5:0/2713505079 wait complete. 2024-05-20T18:27:58.139 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-05-20T18:27:58.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.256+0000 7f262904b640 1 Processor -- start 2024-05-20T18:27:58.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.256+0000 7f262904b640 1 -- start start 2024-05-20T18:27:58.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.257+0000 7f262904b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2624162820 0x7f26240b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:58.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.257+0000 7f262904b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f262406dae0 0x7f26240b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:58.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.257+0000 7f262904b640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26240d0630 con 0x7f26240b3e80 2024-05-20T18:27:58.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.257+0000 7f262904b640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f26240d39c0 con 0x7f2624162820 2024-05-20T18:27:58.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.257+0000 7f262904b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f26240d3ca0 con 0x7f262406dae0 2024-05-20T18:27:58.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.258+0000 7f262359e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f262406dae0 0x7f26240b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:58.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.258+0000 7f2622d9d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2624162820 0x7f26240b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:58.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.258+0000 7f262359e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f262406dae0 0x7f26240b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:59534/0 (socket says 172.21.15.5:59534) 2024-05-20T18:27:58.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.258+0000 7f2622d9d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2624162820 0x7f26240b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:48116/0 (socket says 172.21.15.5:48116) 2024-05-20T18:27:58.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.258+0000 7f262359e640 1 -- 172.21.15.5:0/1332062927 learned_addr learned my addr 172.21.15.5:0/1332062927 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:58.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.258+0000 7f261a59c640 1 --1- 172.21.15.5:0/1332062927 >> v1:172.21.15.63:6789/0 conn(0x7f26240b3e80 0x7f262406d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:54076/0 (socket says 172.21.15.5:54076) 2024-05-20T18:27:58.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.258+0000 7f262359e640 1 -- 172.21.15.5:0/1332062927 >> v2:172.21.15.63:3300/0 conn(0x7f2624162820 msgr2=0x7f26240b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:58.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.258+0000 7f262359e640 1 --2- 172.21.15.5:0/1332062927 >> v2:172.21.15.63:3300/0 conn(0x7f2624162820 0x7f26240b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:58.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.258+0000 7f262359e640 1 -- 172.21.15.5:0/1332062927 >> v1:172.21.15.63:6789/0 conn(0x7f26240b3e80 legacy=0x7f262406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:58.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.258+0000 7f262359e640 1 -- 172.21.15.5:0/1332062927 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26240d3f80 con 0x7f262406dae0 2024-05-20T18:27:58.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.259+0000 7f262359e640 1 --2- 172.21.15.5:0/1332062927 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f262406dae0 0x7f26240b8190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f261c009e50 tx=0x7f261c02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dc43662c28e2ade server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:58.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.259+0000 7f262259c640 1 -- 172.21.15.5:0/1332062927 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c036070 con 0x7f262406dae0 2024-05-20T18:27:58.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.259+0000 7f262259c640 1 -- 172.21.15.5:0/1332062927 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f261c031600 con 0x7f262406dae0 2024-05-20T18:27:58.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.259+0000 7f262259c640 1 -- 172.21.15.5:0/1332062927 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c0319d0 con 0x7f262406dae0 2024-05-20T18:27:58.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.260+0000 7f262904b640 1 -- 172.21.15.5:0/1332062927 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f262406dae0 msgr2=0x7f26240b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:58.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.260+0000 7f262904b640 1 --2- 172.21.15.5:0/1332062927 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f262406dae0 0x7f26240b8190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f261c009e50 tx=0x7f261c02a710 comp rx=0 tx=0).stop 2024-05-20T18:27:58.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.260+0000 7f262904b640 1 -- 172.21.15.5:0/1332062927 shutdown_connections 2024-05-20T18:27:58.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.260+0000 7f262904b640 1 --2- 172.21.15.5:0/1332062927 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f262406dae0 0x7f26240b8190 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:58.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.260+0000 7f262904b640 1 --2- 172.21.15.5:0/1332062927 >> v2:172.21.15.63:3300/0 conn(0x7f2624162820 0x7f26240b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:58.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.260+0000 7f262904b640 1 -- 172.21.15.5:0/1332062927 >> 172.21.15.5:0/1332062927 conn(0x7f26240d4bd0 msgr2=0x7f26240d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:58.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.260+0000 7f262904b640 1 -- 172.21.15.5:0/1332062927 shutdown_connections 2024-05-20T18:27:58.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.260+0000 7f262904b640 1 -- 172.21.15.5:0/1332062927 wait complete. 2024-05-20T18:27:58.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.262+0000 7f262904b640 1 Processor -- start 2024-05-20T18:27:58.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.262+0000 7f262904b640 1 -- start start 2024-05-20T18:27:58.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.263+0000 7f262904b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26240b3e80 0x7f262414c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:58.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.263+0000 7f262904b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2624162820 0x7f262414cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:58.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.263+0000 7f262904b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26240c8700 0x7f2624152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:58.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.263+0000 7f262904b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f26240d4780 con 0x7f26240c8700 2024-05-20T18:27:58.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.263+0000 7f262904b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f26240d0300 con 0x7f26240b3e80 2024-05-20T18:27:58.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.263+0000 7f262904b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f26240d4900 con 0x7f2624162820 2024-05-20T18:27:58.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.264+0000 7f262359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26240c8700 0x7f2624152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:58.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.264+0000 7f262359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26240c8700 0x7f2624152860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51108/0 (socket says 172.21.15.5:51108) 2024-05-20T18:27:58.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.264+0000 7f262359e640 1 -- 172.21.15.5:0/1730472811 learned_addr learned my addr 172.21.15.5:0/1730472811 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:58.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.264+0000 7f261a59c640 1 --2- 172.21.15.5:0/1730472811 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2624162820 0x7f262414cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:58.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.264+0000 7f2622d9d640 1 --2- 172.21.15.5:0/1730472811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26240b3e80 0x7f262414c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:58.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.264+0000 7f262359e640 1 -- 172.21.15.5:0/1730472811 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2624162820 msgr2=0x7f262414cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:58.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.264+0000 7f262359e640 1 --2- 172.21.15.5:0/1730472811 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2624162820 0x7f262414cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:58.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.264+0000 7f262359e640 1 -- 172.21.15.5:0/1730472811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26240b3e80 msgr2=0x7f262414c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:58.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.264+0000 7f262359e640 1 --2- 172.21.15.5:0/1730472811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26240b3e80 0x7f262414c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:58.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.264+0000 7f262359e640 1 -- 172.21.15.5:0/1730472811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26241502f0 con 0x7f26240c8700 2024-05-20T18:27:58.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.264+0000 7f262359e640 1 --2- 172.21.15.5:0/1730472811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26240c8700 0x7f2624152860 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f261c009f70 tx=0x7f261c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:58.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.264+0000 7f261bfff640 1 -- 172.21.15.5:0/1730472811 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c03f020 con 0x7f26240c8700 2024-05-20T18:27:58.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.264+0000 7f261bfff640 1 -- 172.21.15.5:0/1730472811 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f261c037040 con 0x7f26240c8700 2024-05-20T18:27:58.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.264+0000 7f262904b640 1 -- 172.21.15.5:0/1730472811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26240c8e30 con 0x7f26240c8700 2024-05-20T18:27:58.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.264+0000 7f261bfff640 1 -- 172.21.15.5:0/1730472811 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c036070 con 0x7f26240c8700 2024-05-20T18:27:58.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.264+0000 7f262904b640 1 -- 172.21.15.5:0/1730472811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26240c93f0 con 0x7f26240c8700 2024-05-20T18:27:58.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.266+0000 7f262904b640 1 -- 172.21.15.5:0/1730472811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26240c8ba0 con 0x7f26240c8700 2024-05-20T18:27:58.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.266+0000 7f261bfff640 1 -- 172.21.15.5:0/1730472811 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f261c042020 con 0x7f26240c8700 2024-05-20T18:27:58.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.266+0000 7f261bfff640 1 --2- 172.21.15.5:0/1730472811 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f260406be60 0x7f260406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:58.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.267+0000 7f261bfff640 1 -- 172.21.15.5:0/1730472811 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7f261c0ab5b0 con 0x7f26240c8700 2024-05-20T18:27:58.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.267+0000 7f2622d9d640 1 --2- 172.21.15.5:0/1730472811 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f260406be60 0x7f260406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:58.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.267+0000 7f2622d9d640 1 --2- 172.21.15.5:0/1730472811 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f260406be60 0x7f260406e370 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2614009aa0 tx=0x7f2614009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:58.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.271+0000 7f261bfff640 1 -- 172.21.15.5:0/1730472811 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f261c049030 con 0x7f26240c8700 2024-05-20T18:27:58.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:58.524+0000 7f262904b640 1 -- 172.21.15.5:0/1730472811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7f262406df90 con 0x7f26240c8700 2024-05-20T18:27:59.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.230+0000 7f261bfff640 1 -- 172.21.15.5:0/1730472811 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v65) ==== 145+0+0 (secure 0 0 0) 0x7f261c079470 con 0x7f26240c8700 2024-05-20T18:27:59.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.233+0000 7f262904b640 1 -- 172.21.15.5:0/1730472811 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f260406be60 msgr2=0x7f260406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:59.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.233+0000 7f262904b640 1 --2- 172.21.15.5:0/1730472811 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f260406be60 0x7f260406e370 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2614009aa0 tx=0x7f2614009190 comp rx=0 tx=0).stop 2024-05-20T18:27:59.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.233+0000 7f262904b640 1 -- 172.21.15.5:0/1730472811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26240c8700 msgr2=0x7f2624152860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:59.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.233+0000 7f262904b640 1 --2- 172.21.15.5:0/1730472811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26240c8700 0x7f2624152860 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f261c009f70 tx=0x7f261c004a40 comp rx=0 tx=0).stop 2024-05-20T18:27:59.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.233+0000 7f262904b640 1 -- 172.21.15.5:0/1730472811 shutdown_connections 2024-05-20T18:27:59.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.233+0000 7f262904b640 1 --2- 172.21.15.5:0/1730472811 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f260406be60 0x7f260406e370 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:59.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.233+0000 7f262904b640 1 --2- 172.21.15.5:0/1730472811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26240c8700 0x7f2624152860 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:59.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.233+0000 7f262904b640 1 --2- 172.21.15.5:0/1730472811 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2624162820 0x7f262414cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:59.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.233+0000 7f262904b640 1 --2- 172.21.15.5:0/1730472811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26240b3e80 0x7f262414c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:59.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.233+0000 7f262904b640 1 -- 172.21.15.5:0/1730472811 >> 172.21.15.5:0/1730472811 conn(0x7f26240d4bd0 msgr2=0x7f26240d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:59.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.233+0000 7f262904b640 1 -- 172.21.15.5:0/1730472811 shutdown_connections 2024-05-20T18:27:59.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.233+0000 7f262904b640 1 -- 172.21.15.5:0/1730472811 wait complete. 2024-05-20T18:27:59.247 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-20T18:27:59.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.372+0000 7fc663e56640 1 Processor -- start 2024-05-20T18:27:59.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.373+0000 7fc663e56640 1 -- start start 2024-05-20T18:27:59.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.374+0000 7fc663e56640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc65c0d10e0 0x7fc65c0cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:59.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.374+0000 7fc663e56640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc65c0d14f0 0x7fc65c0cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:59.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.374+0000 7fc663e56640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc65c0cd4d0 0x7fc65c0cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:59.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.374+0000 7fc663e56640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fc65c0d1ac0 con 0x7fc65c0cd4d0 2024-05-20T18:27:59.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.374+0000 7fc663e56640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc65c0d1da0 con 0x7fc65c0d14f0 2024-05-20T18:27:59.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.374+0000 7fc663e56640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc65c0d2080 con 0x7fc65c0d10e0 2024-05-20T18:27:59.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.374+0000 7fc6623f3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc65c0cd4d0 0x7fc65c0cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:59.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.374+0000 7fc6623f3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc65c0cd4d0 0x7fc65c0cd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51116/0 (socket says 172.21.15.5:51116) 2024-05-20T18:27:59.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.374+0000 7fc6623f3640 1 -- 172.21.15.5:0/4053798867 learned_addr learned my addr 172.21.15.5:0/4053798867 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:59.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.374+0000 7fc661bf2640 1 --2- 172.21.15.5:0/4053798867 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc65c0d10e0 0x7fc65c0cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:59.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.374+0000 7fc6613f1640 1 --2- 172.21.15.5:0/4053798867 >> v2:172.21.15.63:3300/0 conn(0x7fc65c0d14f0 0x7fc65c0cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:59.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.375+0000 7fc6623f3640 1 -- 172.21.15.5:0/4053798867 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc65c0d10e0 msgr2=0x7fc65c0cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:59.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.375+0000 7fc6623f3640 1 --2- 172.21.15.5:0/4053798867 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc65c0d10e0 0x7fc65c0cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:59.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.375+0000 7fc6623f3640 1 -- 172.21.15.5:0/4053798867 >> v2:172.21.15.63:3300/0 conn(0x7fc65c0d14f0 msgr2=0x7fc65c0cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:59.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.375+0000 7fc6623f3640 1 --2- 172.21.15.5:0/4053798867 >> v2:172.21.15.63:3300/0 conn(0x7fc65c0d14f0 0x7fc65c0cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:59.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.375+0000 7fc6623f3640 1 -- 172.21.15.5:0/4053798867 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65c0d2360 con 0x7fc65c0cd4d0 2024-05-20T18:27:59.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.375+0000 7fc6623f3640 1 --2- 172.21.15.5:0/4053798867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc65c0cd4d0 0x7fc65c0cd8e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc65c068370 tx=0x7fc65402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5fd0267b39b3a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:59.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.375+0000 7fc660bf0640 1 -- 172.21.15.5:0/4053798867 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc654036070 con 0x7fc65c0cd4d0 2024-05-20T18:27:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.376+0000 7fc660bf0640 1 -- 172.21.15.5:0/4053798867 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc654031600 con 0x7fc65c0cd4d0 2024-05-20T18:27:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.376+0000 7fc660bf0640 1 -- 172.21.15.5:0/4053798867 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6540319d0 con 0x7fc65c0cd4d0 2024-05-20T18:27:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.376+0000 7fc663e56640 1 -- 172.21.15.5:0/4053798867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc65c0cd4d0 msgr2=0x7fc65c0cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.376+0000 7fc663e56640 1 --2- 172.21.15.5:0/4053798867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc65c0cd4d0 0x7fc65c0cd8e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc65c068370 tx=0x7fc65402a6a0 comp rx=0 tx=0).stop 2024-05-20T18:27:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.376+0000 7fc663e56640 1 -- 172.21.15.5:0/4053798867 shutdown_connections 2024-05-20T18:27:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.376+0000 7fc663e56640 1 --2- 172.21.15.5:0/4053798867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc65c0cd4d0 0x7fc65c0cd8e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.376+0000 7fc663e56640 1 --2- 172.21.15.5:0/4053798867 >> v2:172.21.15.63:3300/0 conn(0x7fc65c0d14f0 0x7fc65c0cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.376+0000 7fc663e56640 1 --2- 172.21.15.5:0/4053798867 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc65c0d10e0 0x7fc65c0cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.376+0000 7fc663e56640 1 -- 172.21.15.5:0/4053798867 >> 172.21.15.5:0/4053798867 conn(0x7fc65c0b44d0 msgr2=0x7fc65c0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.376+0000 7fc663e56640 1 -- 172.21.15.5:0/4053798867 shutdown_connections 2024-05-20T18:27:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.376+0000 7fc663e56640 1 -- 172.21.15.5:0/4053798867 wait complete. 2024-05-20T18:27:59.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.378+0000 7fc663e56640 1 Processor -- start 2024-05-20T18:27:59.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.378+0000 7fc663e56640 1 -- start start 2024-05-20T18:27:59.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.379+0000 7fc663e56640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc65c0cd4d0 0x7fc65c1564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:59.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.379+0000 7fc663e56640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc65c0d10e0 0x7fc65c159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:59.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.379+0000 7fc663e56640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc65c0d14f0 0x7fc65c159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:59.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.380+0000 7fc663e56640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c0b8440 con 0x7fc65c0d14f0 2024-05-20T18:27:59.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.380+0000 7fc663e56640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c0b82c0 con 0x7fc65c0d10e0 2024-05-20T18:27:59.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.380+0000 7fc663e56640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc65c0709f0 con 0x7fc65c0cd4d0 2024-05-20T18:27:59.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.380+0000 7fc6623f3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc65c0d14f0 0x7fc65c159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:59.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.380+0000 7fc6623f3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc65c0d14f0 0x7fc65c159a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51126/0 (socket says 172.21.15.5:51126) 2024-05-20T18:27:59.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.380+0000 7fc6623f3640 1 -- 172.21.15.5:0/2128101244 learned_addr learned my addr 172.21.15.5:0/2128101244 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:59.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.380+0000 7fc661bf2640 1 --2- 172.21.15.5:0/2128101244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc65c0cd4d0 0x7fc65c1564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:59.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.380+0000 7fc6613f1640 1 --2- 172.21.15.5:0/2128101244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc65c0d10e0 0x7fc65c159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:59.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.380+0000 7fc6623f3640 1 -- 172.21.15.5:0/2128101244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc65c0cd4d0 msgr2=0x7fc65c1564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:59.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.380+0000 7fc6623f3640 1 --2- 172.21.15.5:0/2128101244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc65c0cd4d0 0x7fc65c1564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:59.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.380+0000 7fc6623f3640 1 -- 172.21.15.5:0/2128101244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc65c0d10e0 msgr2=0x7fc65c159530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:59.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.380+0000 7fc6623f3640 1 --2- 172.21.15.5:0/2128101244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc65c0d10e0 0x7fc65c159530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:59.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.380+0000 7fc6623f3640 1 -- 172.21.15.5:0/2128101244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65c159120 con 0x7fc65c0d14f0 2024-05-20T18:27:59.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.380+0000 7fc6623f3640 1 --2- 172.21.15.5:0/2128101244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc65c0d14f0 0x7fc65c159a70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc65402ac90 tx=0x7fc654004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:59.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.380+0000 7fc6527fc640 1 -- 172.21.15.5:0/2128101244 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc654036050 con 0x7fc65c0d14f0 2024-05-20T18:27:59.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.380+0000 7fc6527fc640 1 -- 172.21.15.5:0/2128101244 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc654042070 con 0x7fc65c0d14f0 2024-05-20T18:27:59.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.380+0000 7fc663e56640 1 -- 172.21.15.5:0/2128101244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc65c0c9880 con 0x7fc65c0d14f0 2024-05-20T18:27:59.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.380+0000 7fc6527fc640 1 -- 172.21.15.5:0/2128101244 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc654009510 con 0x7fc65c0d14f0 2024-05-20T18:27:59.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.380+0000 7fc663e56640 1 -- 172.21.15.5:0/2128101244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc65c0c9e40 con 0x7fc65c0d14f0 2024-05-20T18:27:59.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.381+0000 7fc663e56640 1 -- 172.21.15.5:0/2128101244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc65c158e90 con 0x7fc65c0d14f0 2024-05-20T18:27:59.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.382+0000 7fc6527fc640 1 -- 172.21.15.5:0/2128101244 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc654007bf0 con 0x7fc65c0d14f0 2024-05-20T18:27:59.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.383+0000 7fc6527fc640 1 --2- 172.21.15.5:0/2128101244 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc64006be60 0x7fc64006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:59.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.383+0000 7fc661bf2640 1 --2- 172.21.15.5:0/2128101244 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc64006be60 0x7fc64006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:59.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.383+0000 7fc6527fc640 1 -- 172.21.15.5:0/2128101244 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7fc6540abee0 con 0x7fc65c0d14f0 2024-05-20T18:27:59.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.383+0000 7fc661bf2640 1 --2- 172.21.15.5:0/2128101244 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc64006be60 0x7fc64006e370 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc658009aa0 tx=0x7fc658009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:59.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.387+0000 7fc6527fc640 1 -- 172.21.15.5:0/2128101244 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc654079f50 con 0x7fc65c0d14f0 2024-05-20T18:27:59.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.574+0000 7fc663e56640 1 -- 172.21.15.5:0/2128101244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc65c075b00 con 0x7fc65c0d14f0 2024-05-20T18:27:59.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.578+0000 7fc6527fc640 1 -- 172.21.15.5:0/2128101244 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7fc65407e250 con 0x7fc65c0d14f0 2024-05-20T18:27:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.580+0000 7fc663e56640 1 -- 172.21.15.5:0/2128101244 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc64006be60 msgr2=0x7fc64006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.580+0000 7fc663e56640 1 --2- 172.21.15.5:0/2128101244 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc64006be60 0x7fc64006e370 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc658009aa0 tx=0x7fc658009190 comp rx=0 tx=0).stop 2024-05-20T18:27:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.580+0000 7fc663e56640 1 -- 172.21.15.5:0/2128101244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc65c0d14f0 msgr2=0x7fc65c159a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.580+0000 7fc663e56640 1 --2- 172.21.15.5:0/2128101244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc65c0d14f0 0x7fc65c159a70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc65402ac90 tx=0x7fc654004a40 comp rx=0 tx=0).stop 2024-05-20T18:27:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.580+0000 7fc663e56640 1 -- 172.21.15.5:0/2128101244 shutdown_connections 2024-05-20T18:27:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.580+0000 7fc663e56640 1 --2- 172.21.15.5:0/2128101244 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc64006be60 0x7fc64006e370 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.580+0000 7fc663e56640 1 --2- 172.21.15.5:0/2128101244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc65c0d14f0 0x7fc65c159a70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.580+0000 7fc663e56640 1 --2- 172.21.15.5:0/2128101244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc65c0d10e0 0x7fc65c159530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.580+0000 7fc663e56640 1 --2- 172.21.15.5:0/2128101244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc65c0cd4d0 0x7fc65c1564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.580+0000 7fc663e56640 1 -- 172.21.15.5:0/2128101244 >> 172.21.15.5:0/2128101244 conn(0x7fc65c0b44d0 msgr2=0x7fc65c0b1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.580+0000 7fc663e56640 1 -- 172.21.15.5:0/2128101244 shutdown_connections 2024-05-20T18:27:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.580+0000 7fc663e56640 1 -- 172.21.15.5:0/2128101244 wait complete. 2024-05-20T18:27:59.591 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-20T18:27:59.591 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T18:27:59.639 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T18:27:59.689 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-20T18:27:59.689 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:27:59.690 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:27:59.706 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:27:59.706 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:27:59.763 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:27:59.764 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:27:59.820 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:27:59.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.976+0000 7f0a24b77640 1 Processor -- start 2024-05-20T18:27:59.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.976+0000 7f0a24b77640 1 -- start start 2024-05-20T18:27:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.977+0000 7f0a24b77640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0a200d10e0 0x7f0a200cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.977+0000 7f0a24b77640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0a200d14f0 0x7f0a200cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.977+0000 7f0a24b77640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a200d1ac0 con 0x7f0a200cd4d0 2024-05-20T18:27:59.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.977+0000 7f0a24b77640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f0a200d1da0 con 0x7f0a200d10e0 2024-05-20T18:27:59.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.977+0000 7f0a24b77640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0a200d2080 con 0x7f0a200d14f0 2024-05-20T18:27:59.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.978+0000 7f0a1e59c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0a200d10e0 0x7f0a200cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:59.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.978+0000 7f0a1ed9d640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0a200cd4d0 0x7f0a200cd8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53998/0 (socket says 172.21.15.5:53998) 2024-05-20T18:27:59.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.978+0000 7f0a1ed9d640 1 -- 172.21.15.5:0/723565518 learned_addr learned my addr 172.21.15.5:0/723565518 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:27:59.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.978+0000 7f0a17fff640 1 --2- 172.21.15.5:0/723565518 >> v2:172.21.15.63:3300/0 conn(0x7f0a200d14f0 0x7f0a200cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:59.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.978+0000 7f0a1e59c640 1 -- 172.21.15.5:0/723565518 >> v2:172.21.15.63:3300/0 conn(0x7f0a200d14f0 msgr2=0x7f0a200cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:27:59.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.978+0000 7f0a1e59c640 1 --2- 172.21.15.5:0/723565518 >> v2:172.21.15.63:3300/0 conn(0x7f0a200d14f0 0x7f0a200cf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:59.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.978+0000 7f0a1e59c640 1 -- 172.21.15.5:0/723565518 >> v1:172.21.15.5:6789/0 conn(0x7f0a200cd4d0 legacy=0x7f0a200cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:59.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.978+0000 7f0a1dd9b640 1 -- 172.21.15.5:0/723565518 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1284103874 0 0) 0x7f0a200d1ac0 con 0x7f0a200cd4d0 2024-05-20T18:27:59.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.978+0000 7f0a1e59c640 1 -- 172.21.15.5:0/723565518 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a200d2360 con 0x7f0a200d10e0 2024-05-20T18:27:59.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.979+0000 7f0a1e59c640 1 --2- 172.21.15.5:0/723565518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a200d10e0 0x7f0a200cf140 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0a2008bee0 tx=0x7f0a1002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1a94f597e952ab server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:59.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.979+0000 7f0a1dd9b640 1 -- 172.21.15.5:0/723565518 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a10037070 con 0x7f0a200d10e0 2024-05-20T18:27:59.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.979+0000 7f0a1dd9b640 1 -- 172.21.15.5:0/723565518 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a10032440 con 0x7f0a200d10e0 2024-05-20T18:27:59.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.979+0000 7f0a1dd9b640 1 -- 172.21.15.5:0/723565518 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a10032810 con 0x7f0a200d10e0 2024-05-20T18:27:59.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.979+0000 7f0a24b77640 1 -- 172.21.15.5:0/723565518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a200d10e0 msgr2=0x7f0a200cf140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:59.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.980+0000 7f0a24b77640 1 --2- 172.21.15.5:0/723565518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a200d10e0 0x7f0a200cf140 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0a2008bee0 tx=0x7f0a1002a1a0 comp rx=0 tx=0).stop 2024-05-20T18:27:59.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.980+0000 7f0a24b77640 1 -- 172.21.15.5:0/723565518 shutdown_connections 2024-05-20T18:27:59.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.980+0000 7f0a24b77640 1 --2- 172.21.15.5:0/723565518 >> v2:172.21.15.63:3300/0 conn(0x7f0a200d14f0 0x7f0a200cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:59.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.980+0000 7f0a24b77640 1 --2- 172.21.15.5:0/723565518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a200d10e0 0x7f0a200cf140 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:59.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.980+0000 7f0a24b77640 1 -- 172.21.15.5:0/723565518 >> 172.21.15.5:0/723565518 conn(0x7f0a200b44d0 msgr2=0x7f0a200b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:27:59.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.980+0000 7f0a24b77640 1 -- 172.21.15.5:0/723565518 shutdown_connections 2024-05-20T18:27:59.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.980+0000 7f0a24b77640 1 -- 172.21.15.5:0/723565518 wait complete. 2024-05-20T18:27:59.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.982+0000 7f0a24b77640 1 Processor -- start 2024-05-20T18:27:59.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.982+0000 7f0a24b77640 1 -- start start 2024-05-20T18:27:59.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.983+0000 7f0a24b77640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a200cd4d0 0x7f0a200aa090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:59.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.983+0000 7f0a24b77640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0a200d10e0 0x7f0a200b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:59.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.983+0000 7f0a24b77640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a200d14f0 0x7f0a20155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:59.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.983+0000 7f0a24b77640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0a200b8440 con 0x7f0a200d14f0 2024-05-20T18:27:59.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.983+0000 7f0a24b77640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0a200b82c0 con 0x7f0a200cd4d0 2024-05-20T18:27:59.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.983+0000 7f0a24b77640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0a200709f0 con 0x7f0a200d10e0 2024-05-20T18:27:59.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.983+0000 7f0a1ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a200d14f0 0x7f0a20155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:59.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.983+0000 7f0a1ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a200d14f0 0x7f0a20155940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51146/0 (socket says 172.21.15.5:51146) 2024-05-20T18:27:59.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.983+0000 7f0a1ed9d640 1 -- 172.21.15.5:0/3820656039 learned_addr learned my addr 172.21.15.5:0/3820656039 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:27:59.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.983+0000 7f0a1e59c640 1 --2- 172.21.15.5:0/3820656039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a200cd4d0 0x7f0a200aa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.983+0000 7f0a17fff640 1 --2- 172.21.15.5:0/3820656039 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0a200d10e0 0x7f0a200b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.983+0000 7f0a1ed9d640 1 -- 172.21.15.5:0/3820656039 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0a200d10e0 msgr2=0x7f0a200b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.983+0000 7f0a1ed9d640 1 --2- 172.21.15.5:0/3820656039 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0a200d10e0 0x7f0a200b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.983+0000 7f0a1ed9d640 1 -- 172.21.15.5:0/3820656039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a200cd4d0 msgr2=0x7f0a200aa090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:27:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.983+0000 7f0a1ed9d640 1 --2- 172.21.15.5:0/3820656039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a200cd4d0 0x7f0a200aa090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:27:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.983+0000 7f0a1ed9d640 1 -- 172.21.15.5:0/3820656039 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a20155e80 con 0x7f0a200d14f0 2024-05-20T18:27:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.984+0000 7f0a1ed9d640 1 --2- 172.21.15.5:0/3820656039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a200d14f0 0x7f0a20155940 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800c700 tx=0x7f0a1800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.984+0000 7f0a16ffd640 1 -- 172.21.15.5:0/3820656039 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a18018070 con 0x7f0a200d14f0 2024-05-20T18:27:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.984+0000 7f0a16ffd640 1 -- 172.21.15.5:0/3820656039 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a1800ce10 con 0x7f0a200d14f0 2024-05-20T18:27:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.984+0000 7f0a16ffd640 1 -- 172.21.15.5:0/3820656039 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1800a880 con 0x7f0a200d14f0 2024-05-20T18:27:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.984+0000 7f0a24b77640 1 -- 172.21.15.5:0/3820656039 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a200c9880 con 0x7f0a200d14f0 2024-05-20T18:27:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.984+0000 7f0a24b77640 1 -- 172.21.15.5:0/3820656039 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a200c9e40 con 0x7f0a200d14f0 2024-05-20T18:27:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.985+0000 7f0a24b77640 1 -- 172.21.15.5:0/3820656039 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a20075b00 con 0x7f0a200d14f0 2024-05-20T18:27:59.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.985+0000 7f0a16ffd640 1 -- 172.21.15.5:0/3820656039 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0a1801b070 con 0x7f0a200d14f0 2024-05-20T18:27:59.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.986+0000 7f0a16ffd640 1 --2- 172.21.15.5:0/3820656039 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0a0006be60 0x7f0a0006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:27:59.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.986+0000 7f0a16ffd640 1 -- 172.21.15.5:0/3820656039 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7f0a1808a680 con 0x7f0a200d14f0 2024-05-20T18:27:59.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.986+0000 7f0a1e59c640 1 --2- 172.21.15.5:0/3820656039 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0a0006be60 0x7f0a0006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:27:59.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.986+0000 7f0a1e59c640 1 --2- 172.21.15.5:0/3820656039 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0a0006be60 0x7f0a0006e370 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0a10034000 tx=0x7f0a1002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:27:59.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:27:59.990+0000 7f0a16ffd640 1 -- 172.21.15.5:0/3820656039 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0a180585d0 con 0x7f0a200d14f0 2024-05-20T18:28:00.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.175+0000 7f0a24b77640 1 -- 172.21.15.5:0/3820656039 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0a200aa6b0 con 0x7f0a200d14f0 2024-05-20T18:28:00.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.176+0000 7f0a16ffd640 1 -- 172.21.15.5:0/3820656039 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18800 (secure 0 0 0) 0x7f0a1805c8d0 con 0x7f0a200d14f0 2024-05-20T18:28:00.176 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:00.176 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:27:56.213938+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:27:05.476596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T18:27:06.137762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T18:27:09.829920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:28:00.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.178+0000 7f0a24b77640 1 -- 172.21.15.5:0/3820656039 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0a0006be60 msgr2=0x7f0a0006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:00.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.178+0000 7f0a24b77640 1 --2- 172.21.15.5:0/3820656039 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0a0006be60 0x7f0a0006e370 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0a10034000 tx=0x7f0a1002e000 comp rx=0 tx=0).stop 2024-05-20T18:28:00.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.178+0000 7f0a24b77640 1 -- 172.21.15.5:0/3820656039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a200d14f0 msgr2=0x7f0a20155940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:00.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.178+0000 7f0a24b77640 1 --2- 172.21.15.5:0/3820656039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a200d14f0 0x7f0a20155940 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800c700 tx=0x7f0a1800cbc0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.178+0000 7f0a24b77640 1 -- 172.21.15.5:0/3820656039 shutdown_connections 2024-05-20T18:28:00.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.178+0000 7f0a24b77640 1 --2- 172.21.15.5:0/3820656039 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0a0006be60 0x7f0a0006e370 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.178+0000 7f0a24b77640 1 --2- 172.21.15.5:0/3820656039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a200d14f0 0x7f0a20155940 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.178+0000 7f0a24b77640 1 --2- 172.21.15.5:0/3820656039 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0a200d10e0 0x7f0a200b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.178+0000 7f0a24b77640 1 --2- 172.21.15.5:0/3820656039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a200cd4d0 0x7f0a200aa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.178+0000 7f0a24b77640 1 -- 172.21.15.5:0/3820656039 >> 172.21.15.5:0/3820656039 conn(0x7f0a200b44d0 msgr2=0x7f0a200cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:00.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.178+0000 7f0a24b77640 1 -- 172.21.15.5:0/3820656039 shutdown_connections 2024-05-20T18:28:00.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.178+0000 7f0a24b77640 1 -- 172.21.15.5:0/3820656039 wait complete. 2024-05-20T18:28:00.189 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:27:05.476596+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:27:06.137762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T18:27:09.829920+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T18:28:00.190 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:28:00.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.311+0000 7fbf3257e640 1 Processor -- start 2024-05-20T18:28:00.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.311+0000 7fbf3257e640 1 -- start start 2024-05-20T18:28:00.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.313+0000 7fbf3257e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbf2c162820 0x7fbf2c0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:00.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.313+0000 7fbf3257e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbf2c06dae0 0x7fbf2c0b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:00.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.313+0000 7fbf3257e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf2c0d0630 con 0x7fbf2c0b3e80 2024-05-20T18:28:00.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.313+0000 7fbf3257e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fbf2c0d39c0 con 0x7fbf2c06dae0 2024-05-20T18:28:00.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.313+0000 7fbf3257e640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fbf2c0d3ca0 con 0x7fbf2c162820 2024-05-20T18:28:00.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.313+0000 7fbf30b1b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbf2c06dae0 0x7fbf2c0b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:00.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.313+0000 7fbf30b1b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbf2c06dae0 0x7fbf2c0b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51160/0 (socket says 172.21.15.5:51160) 2024-05-20T18:28:00.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.313+0000 7fbf30b1b640 1 -- 172.21.15.5:0/2931339096 learned_addr learned my addr 172.21.15.5:0/2931339096 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:00.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.313+0000 7fbf2bfff640 1 --2- 172.21.15.5:0/2931339096 >> v2:172.21.15.63:3300/0 conn(0x7fbf2c162820 0x7fbf2c0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:00.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.313+0000 7fbf30b1b640 1 -- 172.21.15.5:0/2931339096 >> v2:172.21.15.63:3300/0 conn(0x7fbf2c162820 msgr2=0x7fbf2c0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:00.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.313+0000 7fbf30b1b640 1 --2- 172.21.15.5:0/2931339096 >> v2:172.21.15.63:3300/0 conn(0x7fbf2c162820 0x7fbf2c0b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.313+0000 7fbf30b1b640 1 -- 172.21.15.5:0/2931339096 >> v1:172.21.15.63:6789/0 conn(0x7fbf2c0b3e80 legacy=0x7fbf2c06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:00.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.313+0000 7fbf30b1b640 1 -- 172.21.15.5:0/2931339096 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2c0d3f80 con 0x7fbf2c06dae0 2024-05-20T18:28:00.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.314+0000 7fbf30b1b640 1 --2- 172.21.15.5:0/2931339096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf2c06dae0 0x7fbf2c0b8190 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c0b7740 tx=0x7fbf2402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5fd6609e69b4ed server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:00.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.314+0000 7fbf2b7fe640 1 -- 172.21.15.5:0/2931339096 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf24036070 con 0x7fbf2c06dae0 2024-05-20T18:28:00.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.314+0000 7fbf2b7fe640 1 -- 172.21.15.5:0/2931339096 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf24031600 con 0x7fbf2c06dae0 2024-05-20T18:28:00.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.314+0000 7fbf2b7fe640 1 -- 172.21.15.5:0/2931339096 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf240319d0 con 0x7fbf2c06dae0 2024-05-20T18:28:00.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.315+0000 7fbf3257e640 1 -- 172.21.15.5:0/2931339096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf2c06dae0 msgr2=0x7fbf2c0b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:00.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.315+0000 7fbf3257e640 1 --2- 172.21.15.5:0/2931339096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf2c06dae0 0x7fbf2c0b8190 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c0b7740 tx=0x7fbf2402a6f0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.315+0000 7fbf3257e640 1 -- 172.21.15.5:0/2931339096 shutdown_connections 2024-05-20T18:28:00.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.315+0000 7fbf3257e640 1 --2- 172.21.15.5:0/2931339096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf2c06dae0 0x7fbf2c0b8190 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.315+0000 7fbf3257e640 1 --2- 172.21.15.5:0/2931339096 >> v2:172.21.15.63:3300/0 conn(0x7fbf2c162820 0x7fbf2c0b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.315+0000 7fbf3257e640 1 -- 172.21.15.5:0/2931339096 >> 172.21.15.5:0/2931339096 conn(0x7fbf2c0d4bd0 msgr2=0x7fbf2c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:00.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.315+0000 7fbf3257e640 1 -- 172.21.15.5:0/2931339096 shutdown_connections 2024-05-20T18:28:00.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.315+0000 7fbf3257e640 1 -- 172.21.15.5:0/2931339096 wait complete. 2024-05-20T18:28:00.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.317+0000 7fbf3257e640 1 Processor -- start 2024-05-20T18:28:00.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.317+0000 7fbf3257e640 1 -- start start 2024-05-20T18:28:00.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.318+0000 7fbf3257e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf2c0b3e80 0x7fbf2c0b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:00.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.318+0000 7fbf3257e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf2c162820 0x7fbf2c0b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:00.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.318+0000 7fbf3257e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf2c0b6b30 0x7fbf2c0cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:00.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.318+0000 7fbf3257e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c0d4780 con 0x7fbf2c162820 2024-05-20T18:28:00.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.318+0000 7fbf3257e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c0d0300 con 0x7fbf2c0b6b30 2024-05-20T18:28:00.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.318+0000 7fbf3257e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbf2c0d4900 con 0x7fbf2c0b3e80 2024-05-20T18:28:00.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.318+0000 7fbf23fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf2c162820 0x7fbf2c0b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:00.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.318+0000 7fbf23fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf2c162820 0x7fbf2c0b65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51176/0 (socket says 172.21.15.5:51176) 2024-05-20T18:28:00.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.318+0000 7fbf23fff640 1 -- 172.21.15.5:0/1780610702 learned_addr learned my addr 172.21.15.5:0/1780610702 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:00.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.319+0000 7fbf30b1b640 1 --2- 172.21.15.5:0/1780610702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf2c0b6b30 0x7fbf2c0cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:00.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.319+0000 7fbf2bfff640 1 --2- 172.21.15.5:0/1780610702 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf2c0b3e80 0x7fbf2c0b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:00.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.319+0000 7fbf23fff640 1 -- 172.21.15.5:0/1780610702 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf2c0b3e80 msgr2=0x7fbf2c0b59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:00.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.319+0000 7fbf23fff640 1 --2- 172.21.15.5:0/1780610702 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf2c0b3e80 0x7fbf2c0b59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.319+0000 7fbf23fff640 1 -- 172.21.15.5:0/1780610702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf2c0b6b30 msgr2=0x7fbf2c0cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:00.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.319+0000 7fbf23fff640 1 --2- 172.21.15.5:0/1780610702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf2c0b6b30 0x7fbf2c0cbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.319+0000 7fbf23fff640 1 -- 172.21.15.5:0/1780610702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2c0b4900 con 0x7fbf2c162820 2024-05-20T18:28:00.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.319+0000 7fbf30b1b640 1 --2- 172.21.15.5:0/1780610702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf2c0b6b30 0x7fbf2c0cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:00.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.319+0000 7fbf23fff640 1 --2- 172.21.15.5:0/1780610702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf2c162820 0x7fbf2c0b65f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400b410 tx=0x7fbf1400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:00.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.319+0000 7fbf297fa640 1 -- 172.21.15.5:0/1780610702 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf14015020 con 0x7fbf2c162820 2024-05-20T18:28:00.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.319+0000 7fbf297fa640 1 -- 172.21.15.5:0/1780610702 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf14011830 con 0x7fbf2c162820 2024-05-20T18:28:00.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.319+0000 7fbf3257e640 1 -- 172.21.15.5:0/1780610702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf2c0b4be0 con 0x7fbf2c162820 2024-05-20T18:28:00.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.319+0000 7fbf297fa640 1 -- 172.21.15.5:0/1780610702 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf14011c20 con 0x7fbf2c162820 2024-05-20T18:28:00.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.319+0000 7fbf3257e640 1 -- 172.21.15.5:0/1780610702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf2c0b51a0 con 0x7fbf2c162820 2024-05-20T18:28:00.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.320+0000 7fbf3257e640 1 -- 172.21.15.5:0/1780610702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf2c075b00 con 0x7fbf2c162820 2024-05-20T18:28:00.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.321+0000 7fbf297fa640 1 -- 172.21.15.5:0/1780610702 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbf14018020 con 0x7fbf2c162820 2024-05-20T18:28:00.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.321+0000 7fbf297fa640 1 --2- 172.21.15.5:0/1780610702 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf0c06be60 0x7fbf0c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:00.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.321+0000 7fbf297fa640 1 -- 172.21.15.5:0/1780610702 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7fbf1408ad40 con 0x7fbf2c162820 2024-05-20T18:28:00.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.321+0000 7fbf2bfff640 1 --2- 172.21.15.5:0/1780610702 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf0c06be60 0x7fbf0c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:00.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.322+0000 7fbf2bfff640 1 --2- 172.21.15.5:0/1780610702 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf0c06be60 0x7fbf0c06e370 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c006fd0 tx=0x7fbf1c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:00.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.326+0000 7fbf297fa640 1 -- 172.21.15.5:0/1780610702 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbf1401f070 con 0x7fbf2c162820 2024-05-20T18:28:00.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.512+0000 7fbf3257e640 1 -- 172.21.15.5:0/1780610702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbf2c0b4690 con 0x7fbf2c162820 2024-05-20T18:28:00.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.512+0000 7fbf297fa640 1 -- 172.21.15.5:0/1780610702 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7fbf14059250 con 0x7fbf2c162820 2024-05-20T18:28:00.512 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:28:00.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.514+0000 7fbf3257e640 1 -- 172.21.15.5:0/1780610702 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf0c06be60 msgr2=0x7fbf0c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:00.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.514+0000 7fbf3257e640 1 --2- 172.21.15.5:0/1780610702 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf0c06be60 0x7fbf0c06e370 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c006fd0 tx=0x7fbf1c008040 comp rx=0 tx=0).stop 2024-05-20T18:28:00.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.514+0000 7fbf3257e640 1 -- 172.21.15.5:0/1780610702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf2c162820 msgr2=0x7fbf2c0b65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:00.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.514+0000 7fbf3257e640 1 --2- 172.21.15.5:0/1780610702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf2c162820 0x7fbf2c0b65f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400b410 tx=0x7fbf1400b8d0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.515+0000 7fbf3257e640 1 -- 172.21.15.5:0/1780610702 shutdown_connections 2024-05-20T18:28:00.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.515+0000 7fbf3257e640 1 --2- 172.21.15.5:0/1780610702 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf0c06be60 0x7fbf0c06e370 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.515+0000 7fbf3257e640 1 --2- 172.21.15.5:0/1780610702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf2c0b6b30 0x7fbf2c0cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.515+0000 7fbf3257e640 1 --2- 172.21.15.5:0/1780610702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf2c162820 0x7fbf2c0b65f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.515+0000 7fbf3257e640 1 --2- 172.21.15.5:0/1780610702 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf2c0b3e80 0x7fbf2c0b59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.515+0000 7fbf3257e640 1 -- 172.21.15.5:0/1780610702 >> 172.21.15.5:0/1780610702 conn(0x7fbf2c0d4bd0 msgr2=0x7fbf2c15bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:00.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.515+0000 7fbf3257e640 1 -- 172.21.15.5:0/1780610702 shutdown_connections 2024-05-20T18:28:00.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.515+0000 7fbf3257e640 1 -- 172.21.15.5:0/1780610702 wait complete. 2024-05-20T18:28:00.526 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:28:00.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.645+0000 7f1f515b5640 1 Processor -- start 2024-05-20T18:28:00.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.645+0000 7f1f515b5640 1 -- start start 2024-05-20T18:28:00.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.646+0000 7f1f515b5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f4c15cf50 0x7f1f4c0b1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:00.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.646+0000 7f1f515b5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1f4c0d1400 0x7f1f4c160b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:00.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.646+0000 7f1f515b5640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f4c0d18d0 con 0x7f1f4c0b1eb0 2024-05-20T18:28:00.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.646+0000 7f1f515b5640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f1f4c0c9400 con 0x7f1f4c0d1400 2024-05-20T18:28:00.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.646+0000 7f1f515b5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1f4c0c8f70 con 0x7f1f4c15cf50 2024-05-20T18:28:00.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.646+0000 7f1f4b7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1f4c0d1400 0x7f1f4c160b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:00.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.647+0000 7f1f4b7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1f4c0d1400 0x7f1f4c160b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51182/0 (socket says 172.21.15.5:51182) 2024-05-20T18:28:00.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.647+0000 7f1f4b7fe640 1 -- 172.21.15.5:0/2143391922 learned_addr learned my addr 172.21.15.5:0/2143391922 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:00.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.647+0000 7f1f4affd640 1 --2- 172.21.15.5:0/2143391922 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f4c15cf50 0x7f1f4c0b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:00.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.647+0000 7f1f4b7fe640 1 -- 172.21.15.5:0/2143391922 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f4c15cf50 msgr2=0x7f1f4c0b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:00.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.647+0000 7f1f4b7fe640 1 --2- 172.21.15.5:0/2143391922 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f4c15cf50 0x7f1f4c0b1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.647+0000 7f1f4b7fe640 1 -- 172.21.15.5:0/2143391922 >> v1:172.21.15.63:6789/0 conn(0x7f1f4c0b1eb0 legacy=0x7f1f4c0d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:00.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.647+0000 7f1f4b7fe640 1 -- 172.21.15.5:0/2143391922 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4c0b22c0 con 0x7f1f4c0d1400 2024-05-20T18:28:00.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.648+0000 7f1f4b7fe640 1 --2- 172.21.15.5:0/2143391922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f4c0d1400 0x7f1f4c160b20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1f44009e50 tx=0x7f1f4402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=284445d9137ae854 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:00.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.648+0000 7f1f4a7fc640 1 -- 172.21.15.5:0/2143391922 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f44036070 con 0x7f1f4c0d1400 2024-05-20T18:28:00.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.648+0000 7f1f4a7fc640 1 -- 172.21.15.5:0/2143391922 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f44031600 con 0x7f1f4c0d1400 2024-05-20T18:28:00.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.648+0000 7f1f4a7fc640 1 -- 172.21.15.5:0/2143391922 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f440319d0 con 0x7f1f4c0d1400 2024-05-20T18:28:00.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.648+0000 7f1f515b5640 1 -- 172.21.15.5:0/2143391922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f4c0d1400 msgr2=0x7f1f4c160b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:00.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.648+0000 7f1f515b5640 1 --2- 172.21.15.5:0/2143391922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f4c0d1400 0x7f1f4c160b20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1f44009e50 tx=0x7f1f4402a710 comp rx=0 tx=0).stop 2024-05-20T18:28:00.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.649+0000 7f1f515b5640 1 -- 172.21.15.5:0/2143391922 shutdown_connections 2024-05-20T18:28:00.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.649+0000 7f1f515b5640 1 --2- 172.21.15.5:0/2143391922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f4c0d1400 0x7f1f4c160b20 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.649+0000 7f1f515b5640 1 --2- 172.21.15.5:0/2143391922 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f4c15cf50 0x7f1f4c0b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.649+0000 7f1f515b5640 1 -- 172.21.15.5:0/2143391922 >> 172.21.15.5:0/2143391922 conn(0x7f1f4c06f2b0 msgr2=0x7f1f4c0b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:00.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.649+0000 7f1f515b5640 1 -- 172.21.15.5:0/2143391922 shutdown_connections 2024-05-20T18:28:00.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.649+0000 7f1f515b5640 1 -- 172.21.15.5:0/2143391922 wait complete. 2024-05-20T18:28:00.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.650+0000 7f1f515b5640 1 Processor -- start 2024-05-20T18:28:00.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.650+0000 7f1f515b5640 1 -- start start 2024-05-20T18:28:00.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.652+0000 7f1f515b5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f4c0b1eb0 0x7f1f4c0b6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:00.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.652+0000 7f1f515b5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f4c0d1400 0x7f1f4c1f77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:00.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.652+0000 7f1f515b5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f4c15cf50 0x7f1f4c149960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:00.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.652+0000 7f1f515b5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4c0c8c00 con 0x7f1f4c0d1400 2024-05-20T18:28:00.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.652+0000 7f1f515b5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4c0c8920 con 0x7f1f4c15cf50 2024-05-20T18:28:00.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.652+0000 7f1f515b5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1f4c15d360 con 0x7f1f4c0b1eb0 2024-05-20T18:28:00.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.652+0000 7f1f43fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f4c0d1400 0x7f1f4c1f77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:00.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.652+0000 7f1f43fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f4c0d1400 0x7f1f4c1f77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51188/0 (socket says 172.21.15.5:51188) 2024-05-20T18:28:00.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.652+0000 7f1f43fff640 1 -- 172.21.15.5:0/3820150647 learned_addr learned my addr 172.21.15.5:0/3820150647 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:00.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.652+0000 7f1f4b7fe640 1 --2- 172.21.15.5:0/3820150647 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f4c15cf50 0x7f1f4c149960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:00.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.652+0000 7f1f4affd640 1 --2- 172.21.15.5:0/3820150647 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f4c0b1eb0 0x7f1f4c0b6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:00.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.652+0000 7f1f43fff640 1 -- 172.21.15.5:0/3820150647 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f4c0b1eb0 msgr2=0x7f1f4c0b6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:00.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.652+0000 7f1f43fff640 1 --2- 172.21.15.5:0/3820150647 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f4c0b1eb0 0x7f1f4c0b6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.652+0000 7f1f43fff640 1 -- 172.21.15.5:0/3820150647 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f4c15cf50 msgr2=0x7f1f4c149960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:00.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.652+0000 7f1f43fff640 1 --2- 172.21.15.5:0/3820150647 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f4c15cf50 0x7f1f4c149960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.652+0000 7f1f43fff640 1 -- 172.21.15.5:0/3820150647 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4c1f7d10 con 0x7f1f4c0d1400 2024-05-20T18:28:00.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.652+0000 7f1f4affd640 1 --2- 172.21.15.5:0/3820150647 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f4c0b1eb0 0x7f1f4c0b6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:00.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.652+0000 7f1f43fff640 1 --2- 172.21.15.5:0/3820150647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f4c0d1400 0x7f1f4c1f77d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400c3f0 tx=0x7f1f3400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:00.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.652+0000 7f1f437fe640 1 -- 172.21.15.5:0/3820150647 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f340052d0 con 0x7f1f4c0d1400 2024-05-20T18:28:00.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.653+0000 7f1f437fe640 1 -- 172.21.15.5:0/3820150647 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f34005490 con 0x7f1f4c0d1400 2024-05-20T18:28:00.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.653+0000 7f1f515b5640 1 -- 172.21.15.5:0/3820150647 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f4c0cb7c0 con 0x7f1f4c0d1400 2024-05-20T18:28:00.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.653+0000 7f1f437fe640 1 -- 172.21.15.5:0/3820150647 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34007260 con 0x7f1f4c0d1400 2024-05-20T18:28:00.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.653+0000 7f1f515b5640 1 -- 172.21.15.5:0/3820150647 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f4c149160 con 0x7f1f4c0d1400 2024-05-20T18:28:00.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.654+0000 7f1f515b5640 1 -- 172.21.15.5:0/3820150647 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f4c075b00 con 0x7f1f4c0d1400 2024-05-20T18:28:00.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.654+0000 7f1f437fe640 1 -- 172.21.15.5:0/3820150647 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1f34007400 con 0x7f1f4c0d1400 2024-05-20T18:28:00.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.655+0000 7f1f437fe640 1 --2- 172.21.15.5:0/3820150647 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1f2c06be30 0x7f1f2c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:00.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.655+0000 7f1f437fe640 1 -- 172.21.15.5:0/3820150647 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7f1f3408a690 con 0x7f1f4c0d1400 2024-05-20T18:28:00.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.655+0000 7f1f4affd640 1 --2- 172.21.15.5:0/3820150647 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1f2c06be30 0x7f1f2c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:00.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.655+0000 7f1f4affd640 1 --2- 172.21.15.5:0/3820150647 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1f2c06be30 0x7f1f2c06e340 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c006fd0 tx=0x7f1f3c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:00.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.659+0000 7f1f437fe640 1 -- 172.21.15.5:0/3820150647 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1f4c1f7d10 con 0x7f1f4c0d1400 2024-05-20T18:28:00.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.842+0000 7f1f515b5640 1 -- 172.21.15.5:0/3820150647 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1f4c0cb510 con 0x7f1f4c0d1400 2024-05-20T18:28:00.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.842+0000 7f1f437fe640 1 -- 172.21.15.5:0/3820150647 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f1f4c0d18d0 con 0x7f1f4c0d1400 2024-05-20T18:28:00.841 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:28:00.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.844+0000 7f1f515b5640 1 -- 172.21.15.5:0/3820150647 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1f2c06be30 msgr2=0x7f1f2c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:00.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.844+0000 7f1f515b5640 1 --2- 172.21.15.5:0/3820150647 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1f2c06be30 0x7f1f2c06e340 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c006fd0 tx=0x7f1f3c008040 comp rx=0 tx=0).stop 2024-05-20T18:28:00.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.844+0000 7f1f515b5640 1 -- 172.21.15.5:0/3820150647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f4c0d1400 msgr2=0x7f1f4c1f77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:00.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.844+0000 7f1f515b5640 1 --2- 172.21.15.5:0/3820150647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f4c0d1400 0x7f1f4c1f77d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400c3f0 tx=0x7f1f3400c8b0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.844+0000 7f1f515b5640 1 -- 172.21.15.5:0/3820150647 shutdown_connections 2024-05-20T18:28:00.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.844+0000 7f1f515b5640 1 --2- 172.21.15.5:0/3820150647 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1f2c06be30 0x7f1f2c06e340 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.844+0000 7f1f515b5640 1 --2- 172.21.15.5:0/3820150647 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f4c15cf50 0x7f1f4c149960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.844+0000 7f1f515b5640 1 --2- 172.21.15.5:0/3820150647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f4c0d1400 0x7f1f4c1f77d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.844+0000 7f1f515b5640 1 --2- 172.21.15.5:0/3820150647 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f4c0b1eb0 0x7f1f4c0b6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.844+0000 7f1f515b5640 1 -- 172.21.15.5:0/3820150647 >> 172.21.15.5:0/3820150647 conn(0x7f1f4c06f2b0 msgr2=0x7f1f4c0712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:00.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.844+0000 7f1f515b5640 1 -- 172.21.15.5:0/3820150647 shutdown_connections 2024-05-20T18:28:00.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.844+0000 7f1f515b5640 1 -- 172.21.15.5:0/3820150647 wait complete. 2024-05-20T18:28:00.855 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:28:00.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.973+0000 7fd8704bc640 1 Processor -- start 2024-05-20T18:28:00.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.973+0000 7fd8704bc640 1 -- start start 2024-05-20T18:28:00.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.975+0000 7fd8704bc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd8680af330 0x7fd8680af740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.975+0000 7fd8704bc640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd8680b2560 0x7fd8680d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.975+0000 7fd8704bc640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd86806c840 con 0x7fd8680d1610 2024-05-20T18:28:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.975+0000 7fd8704bc640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fd8680d20d0 con 0x7fd8680af330 2024-05-20T18:28:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.975+0000 7fd8704bc640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd86806cae0 con 0x7fd8680b2560 2024-05-20T18:28:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.975+0000 7fd86e258640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd8680af330 0x7fd8680af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.975+0000 7fd86e258640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd8680af330 0x7fd8680af740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51190/0 (socket says 172.21.15.5:51190) 2024-05-20T18:28:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.975+0000 7fd86da57640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd8680d1610 0x7fd8680b1e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:54000/0 (socket says 172.21.15.5:54000) 2024-05-20T18:28:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.975+0000 7fd86e258640 1 -- 172.21.15.5:0/2376339438 learned_addr learned my addr 172.21.15.5:0/2376339438 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.975+0000 7fd86ea59640 1 --2- 172.21.15.5:0/2376339438 >> v2:172.21.15.63:3300/0 conn(0x7fd8680b2560 0x7fd8680d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.975+0000 7fd86e258640 1 -- 172.21.15.5:0/2376339438 >> v2:172.21.15.63:3300/0 conn(0x7fd8680b2560 msgr2=0x7fd8680d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.975+0000 7fd86e258640 1 --2- 172.21.15.5:0/2376339438 >> v2:172.21.15.63:3300/0 conn(0x7fd8680b2560 0x7fd8680d1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.975+0000 7fd86e258640 1 -- 172.21.15.5:0/2376339438 >> v1:172.21.15.5:6789/0 conn(0x7fd8680d1610 legacy=0x7fd8680b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.975+0000 7fd86e258640 1 -- 172.21.15.5:0/2376339438 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd86806cdc0 con 0x7fd8680af330 2024-05-20T18:28:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.975+0000 7fd86ea59640 1 --2- 172.21.15.5:0/2376339438 >> v2:172.21.15.63:3300/0 conn(0x7fd8680b2560 0x7fd8680d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.976+0000 7fd86e258640 1 --2- 172.21.15.5:0/2376339438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8680af330 0x7fd8680af740 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd868074d20 tx=0x7fd86002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4620cdf911a479 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.976+0000 7fd86d256640 1 -- 172.21.15.5:0/2376339438 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd860037070 con 0x7fd8680af330 2024-05-20T18:28:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.976+0000 7fd86d256640 1 -- 172.21.15.5:0/2376339438 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd860032440 con 0x7fd8680af330 2024-05-20T18:28:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.976+0000 7fd86d256640 1 -- 172.21.15.5:0/2376339438 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd860032810 con 0x7fd8680af330 2024-05-20T18:28:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.977+0000 7fd8704bc640 1 -- 172.21.15.5:0/2376339438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8680af330 msgr2=0x7fd8680af740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.977+0000 7fd8704bc640 1 --2- 172.21.15.5:0/2376339438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8680af330 0x7fd8680af740 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd868074d20 tx=0x7fd86002a1e0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.977+0000 7fd8704bc640 1 -- 172.21.15.5:0/2376339438 shutdown_connections 2024-05-20T18:28:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.977+0000 7fd8704bc640 1 --2- 172.21.15.5:0/2376339438 >> v2:172.21.15.63:3300/0 conn(0x7fd8680b2560 0x7fd8680d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.977+0000 7fd8704bc640 1 --2- 172.21.15.5:0/2376339438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8680af330 0x7fd8680af740 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.977+0000 7fd8704bc640 1 -- 172.21.15.5:0/2376339438 >> 172.21.15.5:0/2376339438 conn(0x7fd8680b6670 msgr2=0x7fd8680b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.977+0000 7fd8704bc640 1 -- 172.21.15.5:0/2376339438 shutdown_connections 2024-05-20T18:28:00.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.977+0000 7fd8704bc640 1 -- 172.21.15.5:0/2376339438 wait complete. 2024-05-20T18:28:00.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.979+0000 7fd8704bc640 1 Processor -- start 2024-05-20T18:28:00.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.979+0000 7fd8704bc640 1 -- start start 2024-05-20T18:28:00.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.980+0000 7fd8704bc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8680af330 0x7fd86815bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:00.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.980+0000 7fd8704bc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8680b2560 0x7fd86815dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:00.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.980+0000 7fd8704bc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8680d1610 0x7fd868155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:00.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.980+0000 7fd8704bc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd868159d30 con 0x7fd8680af330 2024-05-20T18:28:00.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.980+0000 7fd8704bc640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd86815d350 con 0x7fd8680d1610 2024-05-20T18:28:00.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.980+0000 7fd8704bc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd868159eb0 con 0x7fd8680b2560 2024-05-20T18:28:00.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.980+0000 7fd86e258640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8680af330 0x7fd86815bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:00.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.980+0000 7fd86e258640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8680af330 0x7fd86815bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51202/0 (socket says 172.21.15.5:51202) 2024-05-20T18:28:00.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.980+0000 7fd86e258640 1 -- 172.21.15.5:0/297858060 learned_addr learned my addr 172.21.15.5:0/297858060 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:00.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.980+0000 7fd86ea59640 1 --2- 172.21.15.5:0/297858060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8680d1610 0x7fd868155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:00.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.980+0000 7fd86da57640 1 --2- 172.21.15.5:0/297858060 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8680b2560 0x7fd86815dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:00.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.981+0000 7fd86e258640 1 -- 172.21.15.5:0/297858060 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8680b2560 msgr2=0x7fd86815dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:00.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.981+0000 7fd86e258640 1 --2- 172.21.15.5:0/297858060 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8680b2560 0x7fd86815dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.981+0000 7fd86e258640 1 -- 172.21.15.5:0/297858060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8680d1610 msgr2=0x7fd868155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:00.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.981+0000 7fd86e258640 1 --2- 172.21.15.5:0/297858060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8680d1610 0x7fd868155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:00.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.981+0000 7fd86e258640 1 -- 172.21.15.5:0/297858060 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd868152f20 con 0x7fd8680af330 2024-05-20T18:28:00.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.981+0000 7fd86e258640 1 --2- 172.21.15.5:0/297858060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8680af330 0x7fd86815bc00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd860000c00 tx=0x7fd8600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:00.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.981+0000 7fd866ffd640 1 -- 172.21.15.5:0/297858060 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd860037040 con 0x7fd8680af330 2024-05-20T18:28:00.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.981+0000 7fd866ffd640 1 -- 172.21.15.5:0/297858060 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd860042070 con 0x7fd8680af330 2024-05-20T18:28:00.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.981+0000 7fd866ffd640 1 -- 172.21.15.5:0/297858060 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd860030bb0 con 0x7fd8680af330 2024-05-20T18:28:00.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.981+0000 7fd8704bc640 1 -- 172.21.15.5:0/297858060 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd868153200 con 0x7fd8680af330 2024-05-20T18:28:00.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.981+0000 7fd8704bc640 1 -- 172.21.15.5:0/297858060 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8681537c0 con 0x7fd8680af330 2024-05-20T18:28:00.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.982+0000 7fd8704bc640 1 -- 172.21.15.5:0/297858060 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd868075b00 con 0x7fd8680af330 2024-05-20T18:28:00.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.983+0000 7fd866ffd640 1 -- 172.21.15.5:0/297858060 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd860047460 con 0x7fd8680af330 2024-05-20T18:28:00.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.983+0000 7fd866ffd640 1 --2- 172.21.15.5:0/297858060 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd85406be60 0x7fd85406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:00.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.983+0000 7fd866ffd640 1 -- 172.21.15.5:0/297858060 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7fd8600abe10 con 0x7fd8680af330 2024-05-20T18:28:00.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.983+0000 7fd86da57640 1 --2- 172.21.15.5:0/297858060 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd85406be60 0x7fd85406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:00.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.984+0000 7fd86da57640 1 --2- 172.21.15.5:0/297858060 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd85406be60 0x7fd85406e370 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd868156e00 tx=0x7fd85c007e20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:00.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:00.988+0000 7fd866ffd640 1 -- 172.21.15.5:0/297858060 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd860079cd0 con 0x7fd8680af330 2024-05-20T18:28:01.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.171+0000 7fd8704bc640 1 -- 172.21.15.5:0/297858060 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd8680b1670 con 0x7fd8680af330 2024-05-20T18:28:01.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.171+0000 7fd866ffd640 1 -- 172.21.15.5:0/297858060 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7fd86007dfd0 con 0x7fd8680af330 2024-05-20T18:28:01.170 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:28:01.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.173+0000 7fd8704bc640 1 -- 172.21.15.5:0/297858060 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd85406be60 msgr2=0x7fd85406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:01.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.173+0000 7fd8704bc640 1 --2- 172.21.15.5:0/297858060 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd85406be60 0x7fd85406e370 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd868156e00 tx=0x7fd85c007e20 comp rx=0 tx=0).stop 2024-05-20T18:28:01.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.173+0000 7fd8704bc640 1 -- 172.21.15.5:0/297858060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8680af330 msgr2=0x7fd86815bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:01.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.173+0000 7fd8704bc640 1 --2- 172.21.15.5:0/297858060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8680af330 0x7fd86815bc00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd860000c00 tx=0x7fd8600046c0 comp rx=0 tx=0).stop 2024-05-20T18:28:01.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.173+0000 7fd8704bc640 1 -- 172.21.15.5:0/297858060 shutdown_connections 2024-05-20T18:28:01.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.173+0000 7fd8704bc640 1 --2- 172.21.15.5:0/297858060 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd85406be60 0x7fd85406e370 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:01.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.173+0000 7fd8704bc640 1 --2- 172.21.15.5:0/297858060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8680d1610 0x7fd868155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:01.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.173+0000 7fd8704bc640 1 --2- 172.21.15.5:0/297858060 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8680b2560 0x7fd86815dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:01.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.173+0000 7fd8704bc640 1 --2- 172.21.15.5:0/297858060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8680af330 0x7fd86815bc00 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:01.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.173+0000 7fd8704bc640 1 -- 172.21.15.5:0/297858060 >> 172.21.15.5:0/297858060 conn(0x7fd8680b6670 msgr2=0x7fd86806d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:01.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.173+0000 7fd8704bc640 1 -- 172.21.15.5:0/297858060 shutdown_connections 2024-05-20T18:28:01.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.173+0000 7fd8704bc640 1 -- 172.21.15.5:0/297858060 wait complete. 2024-05-20T18:28:01.184 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T18:28:01.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.303+0000 7f4b739f5640 1 Processor -- start 2024-05-20T18:28:01.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.303+0000 7f4b739f5640 1 -- start start 2024-05-20T18:28:01.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.305+0000 7f4b739f5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4b6c162820 0x7f4b6c0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:01.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.305+0000 7f4b739f5640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f4b6c0b3e80 0x7f4b6c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:01.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.305+0000 7f4b739f5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b6c06d910 0x7f4b6c0d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:01.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.305+0000 7f4b739f5640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f4b6c06dd20 con 0x7f4b6c162820 2024-05-20T18:28:01.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.305+0000 7f4b739f5640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f4b6c0d0630 con 0x7f4b6c0b3e80 2024-05-20T18:28:01.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.305+0000 7f4b739f5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4b6c0d39c0 con 0x7f4b6c06d910 2024-05-20T18:28:01.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.305+0000 7f4b71791640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4b6c162820 0x7f4b6c0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:01.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.305+0000 7f4b71791640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4b6c162820 0x7f4b6c0b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51208/0 (socket says 172.21.15.5:51208) 2024-05-20T18:28:01.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.305+0000 7f4b71791640 1 -- 172.21.15.5:0/3452686757 learned_addr learned my addr 172.21.15.5:0/3452686757 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:01.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.305+0000 7f4b70f90640 1 --2- 172.21.15.5:0/3452686757 >> v2:172.21.15.63:3300/0 conn(0x7f4b6c0b3e80 0x7f4b6c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:01.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.305+0000 7f4b71f92640 1 --2- 172.21.15.5:0/3452686757 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b6c06d910 0x7f4b6c0d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:01.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.305+0000 7f4b71791640 1 -- 172.21.15.5:0/3452686757 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b6c06d910 msgr2=0x7f4b6c0d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:01.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.305+0000 7f4b71791640 1 --2- 172.21.15.5:0/3452686757 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b6c06d910 0x7f4b6c0d27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:01.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.305+0000 7f4b71791640 1 -- 172.21.15.5:0/3452686757 >> v2:172.21.15.63:3300/0 conn(0x7f4b6c0b3e80 msgr2=0x7f4b6c06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:01.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.305+0000 7f4b71791640 1 --2- 172.21.15.5:0/3452686757 >> v2:172.21.15.63:3300/0 conn(0x7f4b6c0b3e80 0x7f4b6c06d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:01.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.305+0000 7f4b71791640 1 -- 172.21.15.5:0/3452686757 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b6c0d3ca0 con 0x7f4b6c162820 2024-05-20T18:28:01.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.305+0000 7f4b71f92640 1 --2- 172.21.15.5:0/3452686757 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b6c06d910 0x7f4b6c0d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:01.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.306+0000 7f4b71791640 1 --2- 172.21.15.5:0/3452686757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b6c162820 0x7f4b6c0b3940 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c0d1230 tx=0x7f4b6802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe347be756971e4e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:01.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.306+0000 7f4b63fff640 1 -- 172.21.15.5:0/3452686757 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b68037070 con 0x7f4b6c162820 2024-05-20T18:28:01.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.306+0000 7f4b63fff640 1 -- 172.21.15.5:0/3452686757 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b68032440 con 0x7f4b6c162820 2024-05-20T18:28:01.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.306+0000 7f4b63fff640 1 -- 172.21.15.5:0/3452686757 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b68032810 con 0x7f4b6c162820 2024-05-20T18:28:01.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.307+0000 7f4b739f5640 1 -- 172.21.15.5:0/3452686757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b6c162820 msgr2=0x7f4b6c0b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:01.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.307+0000 7f4b739f5640 1 --2- 172.21.15.5:0/3452686757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b6c162820 0x7f4b6c0b3940 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c0d1230 tx=0x7f4b6802a1e0 comp rx=0 tx=0).stop 2024-05-20T18:28:01.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.307+0000 7f4b739f5640 1 -- 172.21.15.5:0/3452686757 shutdown_connections 2024-05-20T18:28:01.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.307+0000 7f4b739f5640 1 --2- 172.21.15.5:0/3452686757 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b6c06d910 0x7f4b6c0d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:01.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.307+0000 7f4b739f5640 1 --2- 172.21.15.5:0/3452686757 >> v2:172.21.15.63:3300/0 conn(0x7f4b6c0b3e80 0x7f4b6c06d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:01.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.307+0000 7f4b739f5640 1 --2- 172.21.15.5:0/3452686757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b6c162820 0x7f4b6c0b3940 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:01.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.307+0000 7f4b739f5640 1 -- 172.21.15.5:0/3452686757 >> 172.21.15.5:0/3452686757 conn(0x7f4b6c0d4bd0 msgr2=0x7f4b6c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:01.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.307+0000 7f4b739f5640 1 -- 172.21.15.5:0/3452686757 shutdown_connections 2024-05-20T18:28:01.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.307+0000 7f4b739f5640 1 -- 172.21.15.5:0/3452686757 wait complete. 2024-05-20T18:28:01.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.309+0000 7f4b739f5640 1 Processor -- start 2024-05-20T18:28:01.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.309+0000 7f4b739f5640 1 -- start start 2024-05-20T18:28:01.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.310+0000 7f4b739f5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b6c06d910 0x7f4b6c1582d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:01.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.310+0000 7f4b739f5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b6c0b3e80 0x7f4b6c14ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:01.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.310+0000 7f4b739f5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b6c162820 0x7f4b6c1f5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:01.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.310+0000 7f4b739f5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c0d0300 con 0x7f4b6c06d910 2024-05-20T18:28:01.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.310+0000 7f4b739f5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c0d4260 con 0x7f4b6c162820 2024-05-20T18:28:01.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.310+0000 7f4b739f5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4b6c0d4780 con 0x7f4b6c0b3e80 2024-05-20T18:28:01.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.310+0000 7f4b71791640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b6c06d910 0x7f4b6c1582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:01.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.310+0000 7f4b71791640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b6c06d910 0x7f4b6c1582d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51212/0 (socket says 172.21.15.5:51212) 2024-05-20T18:28:01.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.310+0000 7f4b71791640 1 -- 172.21.15.5:0/2544688794 learned_addr learned my addr 172.21.15.5:0/2544688794 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:01.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.310+0000 7f4b71f92640 1 --2- 172.21.15.5:0/2544688794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b6c162820 0x7f4b6c1f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:01.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.310+0000 7f4b71791640 1 -- 172.21.15.5:0/2544688794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b6c0b3e80 msgr2=0x7f4b6c14ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:01.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.310+0000 7f4b71791640 1 --2- 172.21.15.5:0/2544688794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b6c0b3e80 0x7f4b6c14ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:01.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.310+0000 7f4b71791640 1 -- 172.21.15.5:0/2544688794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b6c162820 msgr2=0x7f4b6c1f5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:01.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.310+0000 7f4b71791640 1 --2- 172.21.15.5:0/2544688794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b6c162820 0x7f4b6c1f5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:01.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.310+0000 7f4b71791640 1 -- 172.21.15.5:0/2544688794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b6c158dc0 con 0x7f4b6c06d910 2024-05-20T18:28:01.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.311+0000 7f4b71791640 1 --2- 172.21.15.5:0/2544688794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b6c06d910 0x7f4b6c1582d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4b6802a220 tx=0x7f4b680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:01.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.311+0000 7f4b61ffb640 1 -- 172.21.15.5:0/2544688794 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b68041020 con 0x7f4b6c06d910 2024-05-20T18:28:01.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.311+0000 7f4b61ffb640 1 -- 172.21.15.5:0/2544688794 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b680306c0 con 0x7f4b6c06d910 2024-05-20T18:28:01.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.311+0000 7f4b739f5640 1 -- 172.21.15.5:0/2544688794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b6c1590a0 con 0x7f4b6c06d910 2024-05-20T18:28:01.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.311+0000 7f4b61ffb640 1 -- 172.21.15.5:0/2544688794 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b68037040 con 0x7f4b6c06d910 2024-05-20T18:28:01.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.311+0000 7f4b739f5640 1 -- 172.21.15.5:0/2544688794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b6c159660 con 0x7f4b6c06d910 2024-05-20T18:28:01.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.312+0000 7f4b739f5640 1 -- 172.21.15.5:0/2544688794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b6c075b00 con 0x7f4b6c06d910 2024-05-20T18:28:01.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.313+0000 7f4b61ffb640 1 -- 172.21.15.5:0/2544688794 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4b68044020 con 0x7f4b6c06d910 2024-05-20T18:28:01.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.313+0000 7f4b61ffb640 1 --2- 172.21.15.5:0/2544688794 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4b5006be30 0x7f4b5006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:01.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.313+0000 7f4b61ffb640 1 -- 172.21.15.5:0/2544688794 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7f4b680abab0 con 0x7f4b6c06d910 2024-05-20T18:28:01.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.313+0000 7f4b70f90640 1 --2- 172.21.15.5:0/2544688794 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4b5006be30 0x7f4b5006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:01.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.314+0000 7f4b70f90640 1 --2- 172.21.15.5:0/2544688794 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4b5006be30 0x7f4b5006e340 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c006fd0 tx=0x7f4b5c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:01.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.318+0000 7f4b61ffb640 1 -- 172.21.15.5:0/2544688794 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4b6804b030 con 0x7f4b6c06d910 2024-05-20T18:28:01.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.499+0000 7f4b739f5640 1 -- 172.21.15.5:0/2544688794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f4b6c0b3950 con 0x7f4b6c06d910 2024-05-20T18:28:01.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.500+0000 7f4b61ffb640 1 -- 172.21.15.5:0/2544688794 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f4b6807a050 con 0x7f4b6c06d910 2024-05-20T18:28:01.499 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:28:01.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.502+0000 7f4b739f5640 1 -- 172.21.15.5:0/2544688794 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4b5006be30 msgr2=0x7f4b5006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:01.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.502+0000 7f4b739f5640 1 --2- 172.21.15.5:0/2544688794 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4b5006be30 0x7f4b5006e340 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c006fd0 tx=0x7f4b5c008040 comp rx=0 tx=0).stop 2024-05-20T18:28:01.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.502+0000 7f4b739f5640 1 -- 172.21.15.5:0/2544688794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b6c06d910 msgr2=0x7f4b6c1582d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:01.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.502+0000 7f4b739f5640 1 --2- 172.21.15.5:0/2544688794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b6c06d910 0x7f4b6c1582d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4b6802a220 tx=0x7f4b680046c0 comp rx=0 tx=0).stop 2024-05-20T18:28:01.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.502+0000 7f4b739f5640 1 -- 172.21.15.5:0/2544688794 shutdown_connections 2024-05-20T18:28:01.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.502+0000 7f4b739f5640 1 --2- 172.21.15.5:0/2544688794 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4b5006be30 0x7f4b5006e340 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:01.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.502+0000 7f4b739f5640 1 --2- 172.21.15.5:0/2544688794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b6c162820 0x7f4b6c1f5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:01.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.502+0000 7f4b739f5640 1 --2- 172.21.15.5:0/2544688794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b6c0b3e80 0x7f4b6c14ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:01.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.502+0000 7f4b739f5640 1 --2- 172.21.15.5:0/2544688794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b6c06d910 0x7f4b6c1582d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:01.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.502+0000 7f4b739f5640 1 -- 172.21.15.5:0/2544688794 >> 172.21.15.5:0/2544688794 conn(0x7f4b6c0d4bd0 msgr2=0x7f4b6c0cb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:01.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.502+0000 7f4b739f5640 1 -- 172.21.15.5:0/2544688794 shutdown_connections 2024-05-20T18:28:01.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.502+0000 7f4b739f5640 1 -- 172.21.15.5:0/2544688794 wait complete. 2024-05-20T18:28:01.513 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T18:28:01.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.630+0000 7f572060d640 1 Processor -- start 2024-05-20T18:28:01.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.630+0000 7f572060d640 1 -- start start 2024-05-20T18:28:01.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.631+0000 7f572060d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f57180cd710 0x7f5718162e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:01.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.631+0000 7f572060d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f571806d6b0 0x7f571815f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:01.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.631+0000 7f572060d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f57180cf3d0 0x7f57180cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:01.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.631+0000 7f572060d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f57180d0ba0 con 0x7f571806d6b0 2024-05-20T18:28:01.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.631+0000 7f572060d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f57180cfd20 con 0x7f57180cd710 2024-05-20T18:28:01.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.631+0000 7f572060d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f571806c8c0 con 0x7f57180cf3d0 2024-05-20T18:28:01.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.631+0000 7f571dba8640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f571806d6b0 0x7f571815f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:01.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.631+0000 7f571dba8640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f571806d6b0 0x7f571815f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51222/0 (socket says 172.21.15.5:51222) 2024-05-20T18:28:01.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.631+0000 7f571dba8640 1 -- 172.21.15.5:0/3879264633 learned_addr learned my addr 172.21.15.5:0/3879264633 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:01.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.631+0000 7f571ebaa640 1 --2- 172.21.15.5:0/3879264633 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f57180cf3d0 0x7f57180cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:01.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.631+0000 7f571e3a9640 1 --2- 172.21.15.5:0/3879264633 >> v2:172.21.15.63:3300/0 conn(0x7f57180cd710 0x7f5718162e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:01.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.632+0000 7f571dba8640 1 -- 172.21.15.5:0/3879264633 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f57180cf3d0 msgr2=0x7f57180cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:01.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.632+0000 7f571dba8640 1 --2- 172.21.15.5:0/3879264633 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f57180cf3d0 0x7f57180cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:01.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.632+0000 7f571dba8640 1 -- 172.21.15.5:0/3879264633 >> v2:172.21.15.63:3300/0 conn(0x7f57180cd710 msgr2=0x7f5718162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:01.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.632+0000 7f571dba8640 1 --2- 172.21.15.5:0/3879264633 >> v2:172.21.15.63:3300/0 conn(0x7f57180cd710 0x7f5718162e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:01.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.632+0000 7f571dba8640 1 -- 172.21.15.5:0/3879264633 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f571806dac0 con 0x7f571806d6b0 2024-05-20T18:28:01.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.632+0000 7f571e3a9640 1 --2- 172.21.15.5:0/3879264633 >> v2:172.21.15.63:3300/0 conn(0x7f57180cd710 0x7f5718162e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:01.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.632+0000 7f571ebaa640 1 --2- 172.21.15.5:0/3879264633 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f57180cf3d0 0x7f57180cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:01.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.632+0000 7f571dba8640 1 --2- 172.21.15.5:0/3879264633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f571806d6b0 0x7f571815f2b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f571806dc90 tx=0x7f570c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d4a09bf5a79ba1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:01.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.633+0000 7f571d3a7640 1 -- 172.21.15.5:0/3879264633 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570c037070 con 0x7f571806d6b0 2024-05-20T18:28:01.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.633+0000 7f571d3a7640 1 -- 172.21.15.5:0/3879264633 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f570c032440 con 0x7f571806d6b0 2024-05-20T18:28:01.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.633+0000 7f571d3a7640 1 -- 172.21.15.5:0/3879264633 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570c032810 con 0x7f571806d6b0 2024-05-20T18:28:01.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.633+0000 7f572060d640 1 -- 172.21.15.5:0/3879264633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f571806d6b0 msgr2=0x7f571815f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:01.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.633+0000 7f572060d640 1 --2- 172.21.15.5:0/3879264633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f571806d6b0 0x7f571815f2b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f571806dc90 tx=0x7f570c02a1c0 comp rx=0 tx=0).stop 2024-05-20T18:28:01.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.633+0000 7f572060d640 1 -- 172.21.15.5:0/3879264633 shutdown_connections 2024-05-20T18:28:01.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.633+0000 7f572060d640 1 --2- 172.21.15.5:0/3879264633 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f57180cf3d0 0x7f57180cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:01.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.633+0000 7f572060d640 1 --2- 172.21.15.5:0/3879264633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f571806d6b0 0x7f571815f2b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:01.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.633+0000 7f572060d640 1 --2- 172.21.15.5:0/3879264633 >> v2:172.21.15.63:3300/0 conn(0x7f57180cd710 0x7f5718162e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:01.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.633+0000 7f572060d640 1 -- 172.21.15.5:0/3879264633 >> 172.21.15.5:0/3879264633 conn(0x7f57180b44d0 msgr2=0x7f57180b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:01.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.633+0000 7f572060d640 1 -- 172.21.15.5:0/3879264633 shutdown_connections 2024-05-20T18:28:01.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.633+0000 7f572060d640 1 -- 172.21.15.5:0/3879264633 wait complete. 2024-05-20T18:28:01.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.635+0000 7f572060d640 1 Processor -- start 2024-05-20T18:28:01.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.635+0000 7f572060d640 1 -- start start 2024-05-20T18:28:01.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.637+0000 7f572060d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f571806d6b0 0x7f5718157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:01.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.637+0000 7f572060d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57180cd710 0x7f5718156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:01.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.637+0000 7f572060d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57180cf3d0 0x7f5718160b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:01.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.637+0000 7f572060d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f57181639a0 con 0x7f57180cf3d0 2024-05-20T18:28:01.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.637+0000 7f572060d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f57181636c0 con 0x7f57180cd710 2024-05-20T18:28:01.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.637+0000 7f572060d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5718163c80 con 0x7f571806d6b0 2024-05-20T18:28:01.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.637+0000 7f571ebaa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57180cf3d0 0x7f5718160b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:01.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.637+0000 7f571ebaa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57180cf3d0 0x7f5718160b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51228/0 (socket says 172.21.15.5:51228) 2024-05-20T18:28:01.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.637+0000 7f571ebaa640 1 -- 172.21.15.5:0/1879780566 learned_addr learned my addr 172.21.15.5:0/1879780566 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:01.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.637+0000 7f571dba8640 1 --2- 172.21.15.5:0/1879780566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57180cd710 0x7f5718156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:01.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.637+0000 7f571e3a9640 1 --2- 172.21.15.5:0/1879780566 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f571806d6b0 0x7f5718157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:01.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.637+0000 7f571ebaa640 1 -- 172.21.15.5:0/1879780566 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f571806d6b0 msgr2=0x7f5718157170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:01.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.637+0000 7f571ebaa640 1 --2- 172.21.15.5:0/1879780566 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f571806d6b0 0x7f5718157170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:01.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.637+0000 7f571ebaa640 1 -- 172.21.15.5:0/1879780566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57180cd710 msgr2=0x7f5718156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:01.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.637+0000 7f571ebaa640 1 --2- 172.21.15.5:0/1879780566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57180cd710 0x7f5718156510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:01.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.637+0000 7f571ebaa640 1 -- 172.21.15.5:0/1879780566 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57180cafd0 con 0x7f57180cf3d0 2024-05-20T18:28:01.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.637+0000 7f571ebaa640 1 --2- 172.21.15.5:0/1879780566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57180cf3d0 0x7f5718160b60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f571000c5d0 tx=0x7f571000ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:01.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.637+0000 7f5716ffd640 1 -- 172.21.15.5:0/1879780566 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5710015070 con 0x7f57180cf3d0 2024-05-20T18:28:01.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.637+0000 7f5716ffd640 1 -- 172.21.15.5:0/1879780566 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5710004240 con 0x7f57180cf3d0 2024-05-20T18:28:01.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.637+0000 7f572060d640 1 -- 172.21.15.5:0/1879780566 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57180c7f20 con 0x7f57180cf3d0 2024-05-20T18:28:01.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.638+0000 7f5716ffd640 1 -- 172.21.15.5:0/1879780566 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5710003610 con 0x7f57180cf3d0 2024-05-20T18:28:01.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.638+0000 7f572060d640 1 -- 172.21.15.5:0/1879780566 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5718154a80 con 0x7f57180cf3d0 2024-05-20T18:28:01.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.638+0000 7f572060d640 1 -- 172.21.15.5:0/1879780566 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5718154ce0 con 0x7f57180cf3d0 2024-05-20T18:28:01.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.639+0000 7f5716ffd640 1 -- 172.21.15.5:0/1879780566 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f57100037b0 con 0x7f57180cf3d0 2024-05-20T18:28:01.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.640+0000 7f5716ffd640 1 --2- 172.21.15.5:0/1879780566 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f570406be60 0x7f570406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:01.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.640+0000 7f5716ffd640 1 -- 172.21.15.5:0/1879780566 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7f571008afa0 con 0x7f57180cf3d0 2024-05-20T18:28:01.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.640+0000 7f571e3a9640 1 --2- 172.21.15.5:0/1879780566 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f570406be60 0x7f570406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:01.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.640+0000 7f571e3a9640 1 --2- 172.21.15.5:0/1879780566 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f570406be60 0x7f570406e370 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5718162d60 tx=0x7f5708007740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:01.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.644+0000 7f5716ffd640 1 -- 172.21.15.5:0/1879780566 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5710058ef0 con 0x7f57180cf3d0 2024-05-20T18:28:01.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.841+0000 7f572060d640 1 -- 172.21.15.5:0/1879780566 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f57180cad60 con 0x7f57180cf3d0 2024-05-20T18:28:01.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.842+0000 7f5716ffd640 1 -- 172.21.15.5:0/1879780566 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7f571005d1f0 con 0x7f57180cf3d0 2024-05-20T18:28:01.841 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-05-20T18:28:01.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.844+0000 7f572060d640 1 -- 172.21.15.5:0/1879780566 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f570406be60 msgr2=0x7f570406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:01.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.844+0000 7f572060d640 1 --2- 172.21.15.5:0/1879780566 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f570406be60 0x7f570406e370 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5718162d60 tx=0x7f5708007740 comp rx=0 tx=0).stop 2024-05-20T18:28:01.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.844+0000 7f572060d640 1 -- 172.21.15.5:0/1879780566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57180cf3d0 msgr2=0x7f5718160b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:01.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.844+0000 7f572060d640 1 --2- 172.21.15.5:0/1879780566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57180cf3d0 0x7f5718160b60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f571000c5d0 tx=0x7f571000ca90 comp rx=0 tx=0).stop 2024-05-20T18:28:01.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.844+0000 7f572060d640 1 -- 172.21.15.5:0/1879780566 shutdown_connections 2024-05-20T18:28:01.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.844+0000 7f572060d640 1 --2- 172.21.15.5:0/1879780566 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f570406be60 0x7f570406e370 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:01.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.844+0000 7f572060d640 1 --2- 172.21.15.5:0/1879780566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57180cf3d0 0x7f5718160b60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:01.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.844+0000 7f572060d640 1 --2- 172.21.15.5:0/1879780566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57180cd710 0x7f5718156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:01.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.844+0000 7f572060d640 1 --2- 172.21.15.5:0/1879780566 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f571806d6b0 0x7f5718157170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:01.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.844+0000 7f572060d640 1 -- 172.21.15.5:0/1879780566 >> 172.21.15.5:0/1879780566 conn(0x7f57180b44d0 msgr2=0x7f571815a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:01.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.844+0000 7f572060d640 1 -- 172.21.15.5:0/1879780566 shutdown_connections 2024-05-20T18:28:01.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:01.844+0000 7f572060d640 1 -- 172.21.15.5:0/1879780566 wait complete. 2024-05-20T18:28:01.855 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-20T18:28:01.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:28:01.855 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-05-20T18:28:01.882 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:28:01.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:28:01.882 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-20T18:28:01.938 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:28:01.939 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:28:01.959 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:28:01.959 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:28:02.018 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:28:02.018 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:28:02.077 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:28:02.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.193+0000 7fe45b25f640 1 Processor -- start 2024-05-20T18:28:02.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.193+0000 7fe45b25f640 1 -- start start 2024-05-20T18:28:02.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.194+0000 7fe45b25f640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe4540cf250 0x7fe45415f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:02.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.194+0000 7fe45b25f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4540d3220 0x7fe4540d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:02.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.194+0000 7fe45b25f640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4540d0550 con 0x7fe4540b6680 2024-05-20T18:28:02.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.194+0000 7fe45b25f640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe45415f9a0 con 0x7fe4540cf250 2024-05-20T18:28:02.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.194+0000 7fe45b25f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe4540d0b10 con 0x7fe4540d3220 2024-05-20T18:28:02.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.194+0000 7fe458ffb640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe4540cf250 0x7fe45415f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:02.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.194+0000 7fe4597fc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4540d3220 0x7fe4540d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:02.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.195+0000 7fe453fff640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fe4540b6680 0x7fe4540b6a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:54104/0 (socket says 172.21.15.5:54104) 2024-05-20T18:28:02.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.195+0000 7fe453fff640 1 -- 172.21.15.5:0/1873995911 learned_addr learned my addr 172.21.15.5:0/1873995911 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:02.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.195+0000 7fe4537fe640 1 -- 172.21.15.5:0/1873995911 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4016017151 0 0) 0x7fe4540d0550 con 0x7fe4540b6680 2024-05-20T18:28:02.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.195+0000 7fe4537fe640 1 -- 172.21.15.5:0/1873995911 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe440003520 con 0x7fe4540b6680 2024-05-20T18:28:02.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.195+0000 7fe4597fc640 1 -- 172.21.15.5:0/1873995911 >> v2:172.21.15.63:3300/0 conn(0x7fe4540cf250 msgr2=0x7fe45415f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:02.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.195+0000 7fe4597fc640 1 --2- 172.21.15.5:0/1873995911 >> v2:172.21.15.63:3300/0 conn(0x7fe4540cf250 0x7fe45415f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:02.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.195+0000 7fe4597fc640 1 -- 172.21.15.5:0/1873995911 >> v1:172.21.15.63:6789/0 conn(0x7fe4540b6680 legacy=0x7fe4540b6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:02.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.195+0000 7fe4597fc640 1 -- 172.21.15.5:0/1873995911 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4540d0830 con 0x7fe4540d3220 2024-05-20T18:28:02.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.195+0000 7fe458ffb640 1 --2- 172.21.15.5:0/1873995911 >> v2:172.21.15.63:3300/0 conn(0x7fe4540cf250 0x7fe45415f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:28:02.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.196+0000 7fe4597fc640 1 --2- 172.21.15.5:0/1873995911 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4540d3220 0x7fe4540d3630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe44c0097a0 tx=0x7fe44c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6758197ef4850df6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:02.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.196+0000 7fe4537fe640 1 -- 172.21.15.5:0/1873995911 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c037070 con 0x7fe4540d3220 2024-05-20T18:28:02.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.196+0000 7fe4537fe640 1 -- 172.21.15.5:0/1873995911 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe44c032440 con 0x7fe4540d3220 2024-05-20T18:28:02.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.196+0000 7fe4537fe640 1 -- 172.21.15.5:0/1873995911 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c032810 con 0x7fe4540d3220 2024-05-20T18:28:02.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.197+0000 7fe45b25f640 1 -- 172.21.15.5:0/1873995911 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4540d3220 msgr2=0x7fe4540d3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:02.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.197+0000 7fe45b25f640 1 --2- 172.21.15.5:0/1873995911 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4540d3220 0x7fe4540d3630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe44c0097a0 tx=0x7fe44c02a180 comp rx=0 tx=0).stop 2024-05-20T18:28:02.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.197+0000 7fe45b25f640 1 -- 172.21.15.5:0/1873995911 shutdown_connections 2024-05-20T18:28:02.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.197+0000 7fe45b25f640 1 --2- 172.21.15.5:0/1873995911 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4540d3220 0x7fe4540d3630 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:02.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.197+0000 7fe45b25f640 1 --2- 172.21.15.5:0/1873995911 >> v2:172.21.15.63:3300/0 conn(0x7fe4540cf250 0x7fe45415f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:02.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.197+0000 7fe45b25f640 1 -- 172.21.15.5:0/1873995911 >> 172.21.15.5:0/1873995911 conn(0x7fe4540d1860 msgr2=0x7fe4540d1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:02.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.197+0000 7fe45b25f640 1 -- 172.21.15.5:0/1873995911 shutdown_connections 2024-05-20T18:28:02.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.197+0000 7fe45b25f640 1 -- 172.21.15.5:0/1873995911 wait complete. 2024-05-20T18:28:02.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.199+0000 7fe45b25f640 1 Processor -- start 2024-05-20T18:28:02.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.199+0000 7fe45b25f640 1 -- start start 2024-05-20T18:28:02.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.200+0000 7fe45b25f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4540b6680 0x7fe45414d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:02.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.200+0000 7fe45b25f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4540cf250 0x7fe4540c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:02.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.200+0000 7fe45b25f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4540d3220 0x7fe454161d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:02.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.200+0000 7fe45b25f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe4540d2d00 con 0x7fe4540b6680 2024-05-20T18:28:02.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.200+0000 7fe45b25f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe4540d27d0 con 0x7fe4540cf250 2024-05-20T18:28:02.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.200+0000 7fe45b25f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe4540d0140 con 0x7fe4540d3220 2024-05-20T18:28:02.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.200+0000 7fe458ffb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4540b6680 0x7fe45414d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:02.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.200+0000 7fe458ffb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4540b6680 0x7fe45414d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51240/0 (socket says 172.21.15.5:51240) 2024-05-20T18:28:02.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.200+0000 7fe458ffb640 1 -- 172.21.15.5:0/4222876317 learned_addr learned my addr 172.21.15.5:0/4222876317 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:02.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.201+0000 7fe4597fc640 1 --2- 172.21.15.5:0/4222876317 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4540d3220 0x7fe454161d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:02.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.201+0000 7fe453fff640 1 --2- 172.21.15.5:0/4222876317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4540cf250 0x7fe4540c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:02.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.201+0000 7fe458ffb640 1 -- 172.21.15.5:0/4222876317 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4540d3220 msgr2=0x7fe454161d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:02.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.201+0000 7fe458ffb640 1 --2- 172.21.15.5:0/4222876317 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4540d3220 0x7fe454161d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:02.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.201+0000 7fe458ffb640 1 -- 172.21.15.5:0/4222876317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4540cf250 msgr2=0x7fe4540c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:02.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.201+0000 7fe458ffb640 1 --2- 172.21.15.5:0/4222876317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4540cf250 0x7fe4540c83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:02.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.201+0000 7fe458ffb640 1 -- 172.21.15.5:0/4222876317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe45415fb20 con 0x7fe4540b6680 2024-05-20T18:28:02.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.201+0000 7fe4597fc640 1 --2- 172.21.15.5:0/4222876317 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4540d3220 0x7fe454161d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:02.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.201+0000 7fe453fff640 1 --2- 172.21.15.5:0/4222876317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4540cf250 0x7fe4540c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:02.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.201+0000 7fe458ffb640 1 --2- 172.21.15.5:0/4222876317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4540b6680 0x7fe45414d090 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe43c00e990 tx=0x7fe43c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:02.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.201+0000 7fe4517fa640 1 -- 172.21.15.5:0/4222876317 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c017070 con 0x7fe4540b6680 2024-05-20T18:28:02.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.201+0000 7fe4517fa640 1 -- 172.21.15.5:0/4222876317 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe43c00c980 con 0x7fe4540b6680 2024-05-20T18:28:02.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.201+0000 7fe45b25f640 1 -- 172.21.15.5:0/4222876317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4540c8e20 con 0x7fe4540b6680 2024-05-20T18:28:02.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.201+0000 7fe4517fa640 1 -- 172.21.15.5:0/4222876317 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43c00cd50 con 0x7fe4540b6680 2024-05-20T18:28:02.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.201+0000 7fe45b25f640 1 -- 172.21.15.5:0/4222876317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4540c93e0 con 0x7fe4540b6680 2024-05-20T18:28:02.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.202+0000 7fe45b25f640 1 -- 172.21.15.5:0/4222876317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4540b52c0 con 0x7fe4540b6680 2024-05-20T18:28:02.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.203+0000 7fe4517fa640 1 -- 172.21.15.5:0/4222876317 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe43c01a070 con 0x7fe4540b6680 2024-05-20T18:28:02.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.203+0000 7fe4517fa640 1 --2- 172.21.15.5:0/4222876317 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe43406be30 0x7fe43406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:02.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.203+0000 7fe4517fa640 1 -- 172.21.15.5:0/4222876317 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7fe43c08b500 con 0x7fe4540b6680 2024-05-20T18:28:02.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.204+0000 7fe453fff640 1 --2- 172.21.15.5:0/4222876317 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe43406be30 0x7fe43406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:02.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.204+0000 7fe453fff640 1 --2- 172.21.15.5:0/4222876317 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe43406be30 0x7fe43406e340 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe444005c00 tx=0x7fe444009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:02.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.208+0000 7fe4517fa640 1 -- 172.21.15.5:0/4222876317 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe4540b52c0 con 0x7fe4540b6680 2024-05-20T18:28:02.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.393+0000 7fe45b25f640 1 -- 172.21.15.5:0/4222876317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe454075b00 con 0x7fe4540b6680 2024-05-20T18:28:02.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.394+0000 7fe4517fa640 1 -- 172.21.15.5:0/4222876317 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18800 (secure 0 0 0) 0x7fe43c059a00 con 0x7fe4540b6680 2024-05-20T18:28:02.394 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:02.395 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:27:56.213938+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:27:05.476596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T18:27:06.137762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T18:27:09.829920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:28:02.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.396+0000 7fe45b25f640 1 -- 172.21.15.5:0/4222876317 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe43406be30 msgr2=0x7fe43406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:02.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.396+0000 7fe45b25f640 1 --2- 172.21.15.5:0/4222876317 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe43406be30 0x7fe43406e340 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe444005c00 tx=0x7fe444009040 comp rx=0 tx=0).stop 2024-05-20T18:28:02.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.396+0000 7fe45b25f640 1 -- 172.21.15.5:0/4222876317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4540b6680 msgr2=0x7fe45414d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:02.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.396+0000 7fe45b25f640 1 --2- 172.21.15.5:0/4222876317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4540b6680 0x7fe45414d090 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe43c00e990 tx=0x7fe43c00ee50 comp rx=0 tx=0).stop 2024-05-20T18:28:02.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.396+0000 7fe45b25f640 1 -- 172.21.15.5:0/4222876317 shutdown_connections 2024-05-20T18:28:02.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.396+0000 7fe45b25f640 1 --2- 172.21.15.5:0/4222876317 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe43406be30 0x7fe43406e340 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:02.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.396+0000 7fe45b25f640 1 --2- 172.21.15.5:0/4222876317 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4540d3220 0x7fe454161d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:02.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.396+0000 7fe45b25f640 1 --2- 172.21.15.5:0/4222876317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4540cf250 0x7fe4540c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:02.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.396+0000 7fe45b25f640 1 --2- 172.21.15.5:0/4222876317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4540b6680 0x7fe45414d090 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:02.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.396+0000 7fe45b25f640 1 -- 172.21.15.5:0/4222876317 >> 172.21.15.5:0/4222876317 conn(0x7fe4540d1860 msgr2=0x7fe4540b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:02.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.396+0000 7fe45b25f640 1 -- 172.21.15.5:0/4222876317 shutdown_connections 2024-05-20T18:28:02.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.396+0000 7fe45b25f640 1 -- 172.21.15.5:0/4222876317 wait complete. 2024-05-20T18:28:02.407 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:27:05.476596+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:27:06.137762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T18:27:09.829920+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T18:28:02.408 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:28:02.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.527+0000 7f1103267640 1 Processor -- start 2024-05-20T18:28:02.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.527+0000 7f1103267640 1 -- start start 2024-05-20T18:28:02.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.528+0000 7f1103267640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f10fc15d530 0x7f10fc15aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:02.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.528+0000 7f1103267640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f10fc070850 0x7f10fc070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:02.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.528+0000 7f1103267640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10fc0cd580 con 0x7f10fc0b1700 2024-05-20T18:28:02.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.528+0000 7f1103267640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f10fc0ce190 con 0x7f10fc15d530 2024-05-20T18:28:02.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.528+0000 7f1103267640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f10fc0cd860 con 0x7f10fc070850 2024-05-20T18:28:02.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.529+0000 7f1101003640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f10fc15d530 0x7f10fc15aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:02.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.529+0000 7f1100802640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f10fc0b1700 0x7f10fc0b1b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:54012/0 (socket says 172.21.15.5:54012) 2024-05-20T18:28:02.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.529+0000 7f1100802640 1 -- 172.21.15.5:0/3350130528 learned_addr learned my addr 172.21.15.5:0/3350130528 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:02.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.529+0000 7f1101804640 1 --2- 172.21.15.5:0/3350130528 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f10fc070850 0x7f10fc070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:02.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.529+0000 7f1101003640 1 -- 172.21.15.5:0/3350130528 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f10fc070850 msgr2=0x7f10fc070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:02.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.529+0000 7f1101003640 1 --2- 172.21.15.5:0/3350130528 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f10fc070850 0x7f10fc070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:02.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.529+0000 7f1101003640 1 -- 172.21.15.5:0/3350130528 >> v1:172.21.15.5:6789/0 conn(0x7f10fc0b1700 legacy=0x7f10fc0b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:02.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.529+0000 7f10f3fff640 1 -- 172.21.15.5:0/3350130528 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3844446520 0 0) 0x7f10fc0cd580 con 0x7f10fc0b1700 2024-05-20T18:28:02.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.529+0000 7f1101003640 1 -- 172.21.15.5:0/3350130528 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10fc0cdb40 con 0x7f10fc15d530 2024-05-20T18:28:02.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.530+0000 7f1101003640 1 --2- 172.21.15.5:0/3350130528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10fc15d530 0x7f10fc15aaa0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f10f80097a0 tx=0x7f10f802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ddfd6dccde570e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:02.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.530+0000 7f10f3fff640 1 -- 172.21.15.5:0/3350130528 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f8037070 con 0x7f10fc15d530 2024-05-20T18:28:02.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.530+0000 7f10f3fff640 1 -- 172.21.15.5:0/3350130528 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10f8032440 con 0x7f10fc15d530 2024-05-20T18:28:02.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.530+0000 7f10f3fff640 1 -- 172.21.15.5:0/3350130528 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f8032810 con 0x7f10fc15d530 2024-05-20T18:28:02.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.531+0000 7f1103267640 1 -- 172.21.15.5:0/3350130528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10fc15d530 msgr2=0x7f10fc15aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:02.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.531+0000 7f1103267640 1 --2- 172.21.15.5:0/3350130528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10fc15d530 0x7f10fc15aaa0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f10f80097a0 tx=0x7f10f802a180 comp rx=0 tx=0).stop 2024-05-20T18:28:02.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.531+0000 7f1103267640 1 -- 172.21.15.5:0/3350130528 shutdown_connections 2024-05-20T18:28:02.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.531+0000 7f1103267640 1 --2- 172.21.15.5:0/3350130528 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f10fc070850 0x7f10fc070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:02.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.531+0000 7f1103267640 1 --2- 172.21.15.5:0/3350130528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10fc15d530 0x7f10fc15aaa0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:02.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.531+0000 7f1103267640 1 -- 172.21.15.5:0/3350130528 >> 172.21.15.5:0/3350130528 conn(0x7f10fc0d29d0 msgr2=0x7f10fc0d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:02.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.531+0000 7f1103267640 1 -- 172.21.15.5:0/3350130528 shutdown_connections 2024-05-20T18:28:02.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.531+0000 7f1103267640 1 -- 172.21.15.5:0/3350130528 wait complete. 2024-05-20T18:28:02.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.533+0000 7f1103267640 1 Processor -- start 2024-05-20T18:28:02.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.533+0000 7f1103267640 1 -- start start 2024-05-20T18:28:02.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.534+0000 7f1103267640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10fc070850 0x7f10fc159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:02.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.534+0000 7f1103267640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f10fc0b1700 0x7f10fc1f5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:02.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.534+0000 7f1103267640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f10fc15d530 0x7f10fc1f5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:02.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.534+0000 7f1103267640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc0711a0 con 0x7f10fc070850 2024-05-20T18:28:02.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.534+0000 7f1103267640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc0cd230 con 0x7f10fc15d530 2024-05-20T18:28:02.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.534+0000 7f1103267640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f10fc071320 con 0x7f10fc0b1700 2024-05-20T18:28:02.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.534+0000 7f1101003640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10fc070850 0x7f10fc159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:02.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.534+0000 7f1101003640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10fc070850 0x7f10fc159bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51266/0 (socket says 172.21.15.5:51266) 2024-05-20T18:28:02.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.534+0000 7f1101003640 1 -- 172.21.15.5:0/1011947663 learned_addr learned my addr 172.21.15.5:0/1011947663 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:02.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.534+0000 7f1100802640 1 --2- 172.21.15.5:0/1011947663 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f10fc0b1700 0x7f10fc1f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:02.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.534+0000 7f1101804640 1 --2- 172.21.15.5:0/1011947663 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f10fc15d530 0x7f10fc1f5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.535+0000 7f1101003640 1 -- 172.21.15.5:0/1011947663 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f10fc0b1700 msgr2=0x7f10fc1f5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.535+0000 7f1101003640 1 --2- 172.21.15.5:0/1011947663 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f10fc0b1700 0x7f10fc1f5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.535+0000 7f1101003640 1 -- 172.21.15.5:0/1011947663 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f10fc15d530 msgr2=0x7f10fc1f5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.535+0000 7f1101003640 1 --2- 172.21.15.5:0/1011947663 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f10fc15d530 0x7f10fc1f5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.535+0000 7f1101003640 1 -- 172.21.15.5:0/1011947663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10fc0c6fb0 con 0x7f10fc070850 2024-05-20T18:28:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.535+0000 7f1101804640 1 --2- 172.21.15.5:0/1011947663 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f10fc15d530 0x7f10fc1f5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.535+0000 7f1101003640 1 --2- 172.21.15.5:0/1011947663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10fc070850 0x7f10fc159bb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f10f802a770 tx=0x7f10f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.535+0000 7f10f1ffb640 1 -- 172.21.15.5:0/1011947663 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f8037040 con 0x7f10fc070850 2024-05-20T18:28:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.535+0000 7f10f1ffb640 1 -- 172.21.15.5:0/1011947663 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10f8042070 con 0x7f10fc070850 2024-05-20T18:28:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.535+0000 7f10f1ffb640 1 -- 172.21.15.5:0/1011947663 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f8030940 con 0x7f10fc070850 2024-05-20T18:28:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.535+0000 7f1103267640 1 -- 172.21.15.5:0/1011947663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10fc15a0f0 con 0x7f10fc070850 2024-05-20T18:28:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.535+0000 7f1103267640 1 -- 172.21.15.5:0/1011947663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10fc0c7530 con 0x7f10fc070850 2024-05-20T18:28:02.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.537+0000 7f1103267640 1 -- 172.21.15.5:0/1011947663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10fc075b00 con 0x7f10fc070850 2024-05-20T18:28:02.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.537+0000 7f10f1ffb640 1 -- 172.21.15.5:0/1011947663 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f10f8030ae0 con 0x7f10fc070850 2024-05-20T18:28:02.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.538+0000 7f10f1ffb640 1 --2- 172.21.15.5:0/1011947663 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f10e006be60 0x7f10e006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:02.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.538+0000 7f10f1ffb640 1 -- 172.21.15.5:0/1011947663 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7f10f80abc00 con 0x7f10fc070850 2024-05-20T18:28:02.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.538+0000 7f1100802640 1 --2- 172.21.15.5:0/1011947663 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f10e006be60 0x7f10e006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:02.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.538+0000 7f1100802640 1 --2- 172.21.15.5:0/1011947663 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f10e006be60 0x7f10e006e370 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f10ec007fd0 tx=0x7f10ec007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:02.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.542+0000 7f10f1ffb640 1 -- 172.21.15.5:0/1011947663 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f10fc0c6fb0 con 0x7f10fc070850 2024-05-20T18:28:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.724+0000 7f1103267640 1 -- 172.21.15.5:0/1011947663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f10fc0b8450 con 0x7f10fc070850 2024-05-20T18:28:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.725+0000 7f10f1ffb640 1 -- 172.21.15.5:0/1011947663 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f10f807a070 con 0x7f10fc070850 2024-05-20T18:28:02.724 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:28:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.727+0000 7f1103267640 1 -- 172.21.15.5:0/1011947663 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f10e006be60 msgr2=0x7f10e006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:02.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.727+0000 7f1103267640 1 --2- 172.21.15.5:0/1011947663 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f10e006be60 0x7f10e006e370 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f10ec007fd0 tx=0x7f10ec007de0 comp rx=0 tx=0).stop 2024-05-20T18:28:02.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.727+0000 7f1103267640 1 -- 172.21.15.5:0/1011947663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10fc070850 msgr2=0x7f10fc159bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:02.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.727+0000 7f1103267640 1 --2- 172.21.15.5:0/1011947663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10fc070850 0x7f10fc159bb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f10f802a770 tx=0x7f10f80046c0 comp rx=0 tx=0).stop 2024-05-20T18:28:02.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.727+0000 7f1103267640 1 -- 172.21.15.5:0/1011947663 shutdown_connections 2024-05-20T18:28:02.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.727+0000 7f1103267640 1 --2- 172.21.15.5:0/1011947663 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f10e006be60 0x7f10e006e370 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:02.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.727+0000 7f1103267640 1 --2- 172.21.15.5:0/1011947663 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f10fc15d530 0x7f10fc1f5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:02.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.727+0000 7f1103267640 1 --2- 172.21.15.5:0/1011947663 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f10fc0b1700 0x7f10fc1f5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:02.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.727+0000 7f1103267640 1 --2- 172.21.15.5:0/1011947663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10fc070850 0x7f10fc159bb0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:02.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.727+0000 7f1103267640 1 -- 172.21.15.5:0/1011947663 >> 172.21.15.5:0/1011947663 conn(0x7f10fc0d29d0 msgr2=0x7f10fc0b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:02.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.727+0000 7f1103267640 1 -- 172.21.15.5:0/1011947663 shutdown_connections 2024-05-20T18:28:02.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.727+0000 7f1103267640 1 -- 172.21.15.5:0/1011947663 wait complete. 2024-05-20T18:28:02.738 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:28:02.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.858+0000 7f23183b5640 1 Processor -- start 2024-05-20T18:28:02.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.858+0000 7f23183b5640 1 -- start start 2024-05-20T18:28:02.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.859+0000 7f23183b5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f231009ac00 0x7f23100d18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:02.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.859+0000 7f23183b5640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23101607a0 con 0x7f23100b1cc0 2024-05-20T18:28:02.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.859+0000 7f23183b5640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f231009b2d0 con 0x7f23101599a0 2024-05-20T18:28:02.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.859+0000 7f23183b5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f231015fc20 con 0x7f231009ac00 2024-05-20T18:28:02.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.859+0000 7f2315950640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f23100b1cc0 0x7f231009a4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:54024/0 (socket says 172.21.15.5:54024) 2024-05-20T18:28:02.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.859+0000 7f2315950640 1 -- 172.21.15.5:0/943910353 learned_addr learned my addr 172.21.15.5:0/943910353 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:02.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.860+0000 7f2316952640 1 --2- 172.21.15.5:0/943910353 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f231009ac00 0x7f23100d18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:02.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.860+0000 7f231514f640 1 -- 172.21.15.5:0/943910353 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3314626421 0 0) 0x7f23101607a0 con 0x7f23100b1cc0 2024-05-20T18:28:02.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.860+0000 7f231514f640 1 -- 172.21.15.5:0/943910353 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22fc003520 con 0x7f23100b1cc0 2024-05-20T18:28:02.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.860+0000 7f231514f640 1 -- 172.21.15.5:0/943910353 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 612332968 0 0) 0x7f22fc003520 con 0x7f23100b1cc0 2024-05-20T18:28:02.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.860+0000 7f231514f640 1 -- 172.21.15.5:0/943910353 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f231009ac00 msgr2=0x7f23100d18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:02.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.860+0000 7f231514f640 1 --2- 172.21.15.5:0/943910353 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f231009ac00 0x7f23100d18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:02.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.860+0000 7f231514f640 1 -- 172.21.15.5:0/943910353 >> v1:172.21.15.63:6789/0 conn(0x7f23101599a0 legacy=0x7f231006d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:02.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.860+0000 7f231514f640 1 -- 172.21.15.5:0/943910353 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f231015ff00 con 0x7f23100b1cc0 2024-05-20T18:28:02.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.860+0000 7f231514f640 1 -- 172.21.15.5:0/943910353 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f2300001670 con 0x7f23100b1cc0 2024-05-20T18:28:02.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.860+0000 7f231514f640 1 -- 172.21.15.5:0/943910353 >> v1:172.21.15.5:6789/0 conn(0x7f23100b1cc0 legacy=0x7f231009a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:02.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.860+0000 7f231514f640 1 --2- 172.21.15.5:0/943910353 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22fc0039e0 0x7f22fc003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:02.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.860+0000 7f231514f640 1 --2- 172.21.15.5:0/943910353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22fc004370 0x7f22fc0048c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:02.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.860+0000 7f231514f640 1 --2- 172.21.15.5:0/943910353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22fc0050c0 0x7f22fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:02.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.860+0000 7f231514f640 1 -- 172.21.15.5:0/943910353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f231015fc20 con 0x7f22fc0050c0 2024-05-20T18:28:02.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.860+0000 7f231514f640 1 -- 172.21.15.5:0/943910353 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f22fc00c010 con 0x7f22fc004370 2024-05-20T18:28:02.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.860+0000 7f231514f640 1 -- 172.21.15.5:0/943910353 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f22fc00c210 con 0x7f22fc0039e0 2024-05-20T18:28:02.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.860+0000 7f2316952640 1 --2- 172.21.15.5:0/943910353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22fc0050c0 0x7f22fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:02.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.860+0000 7f2316151640 1 --2- 172.21.15.5:0/943910353 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22fc0039e0 0x7f22fc003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:02.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.860+0000 7f2315950640 1 --2- 172.21.15.5:0/943910353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22fc004370 0x7f22fc0048c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:02.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.861+0000 7f2316952640 1 -- 172.21.15.5:0/943910353 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22fc0039e0 msgr2=0x7f22fc003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:02.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.861+0000 7f2316952640 1 --2- 172.21.15.5:0/943910353 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22fc0039e0 0x7f22fc003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:02.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.861+0000 7f2316952640 1 -- 172.21.15.5:0/943910353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22fc004370 msgr2=0x7f22fc0048c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:02.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.861+0000 7f2316952640 1 --2- 172.21.15.5:0/943910353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22fc004370 0x7f22fc0048c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:02.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.861+0000 7f2316952640 1 -- 172.21.15.5:0/943910353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22fc003520 con 0x7f22fc0050c0 2024-05-20T18:28:02.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.861+0000 7f2316952640 1 --2- 172.21.15.5:0/943910353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22fc0050c0 0x7f22fc00b9c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2308009a90 tx=0x7f22fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73081522632881b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:02.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.861+0000 7f231514f640 1 -- 172.21.15.5:0/943910353 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308036070 con 0x7f22fc0050c0 2024-05-20T18:28:02.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.861+0000 7f231514f640 1 -- 172.21.15.5:0/943910353 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2308031400 con 0x7f22fc0050c0 2024-05-20T18:28:02.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.861+0000 7f231514f640 1 -- 172.21.15.5:0/943910353 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308009690 con 0x7f22fc0050c0 2024-05-20T18:28:02.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.862+0000 7f23183b5640 1 -- 172.21.15.5:0/943910353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22fc0050c0 msgr2=0x7f22fc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:02.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.862+0000 7f23183b5640 1 --2- 172.21.15.5:0/943910353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22fc0050c0 0x7f22fc00b9c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2308009a90 tx=0x7f22fc005830 comp rx=0 tx=0).stop 2024-05-20T18:28:02.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.862+0000 7f2316952640 1 -- 172.21.15.5:0/943910353 reap_dead start 2024-05-20T18:28:02.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.862+0000 7f23183b5640 1 -- 172.21.15.5:0/943910353 shutdown_connections 2024-05-20T18:28:02.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.862+0000 7f23183b5640 1 -- 172.21.15.5:0/943910353 >> 172.21.15.5:0/943910353 conn(0x7f23100b6670 msgr2=0x7f23100b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:02.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.862+0000 7f23183b5640 1 -- 172.21.15.5:0/943910353 shutdown_connections 2024-05-20T18:28:02.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.863+0000 7f23183b5640 1 -- 172.21.15.5:0/943910353 wait complete. 2024-05-20T18:28:02.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.864+0000 7f23183b5640 1 Processor -- start 2024-05-20T18:28:02.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.864+0000 7f23183b5640 1 -- start start 2024-05-20T18:28:02.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.866+0000 7f23183b5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22fc0050c0 0x7f2310146290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:02.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.866+0000 7f23183b5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2310151e30 0x7f231006c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:02.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.866+0000 7f23183b5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2310152240 0x7f231006cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:02.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.866+0000 7f23183b5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f231015f7f0 con 0x7f22fc0050c0 2024-05-20T18:28:02.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.866+0000 7f23183b5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f23101604c0 con 0x7f2310152240 2024-05-20T18:28:02.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.866+0000 7f23183b5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f231015f970 con 0x7f2310151e30 2024-05-20T18:28:02.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.866+0000 7f2316151640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22fc0050c0 0x7f2310146290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:02.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.866+0000 7f2316151640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22fc0050c0 0x7f2310146290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51284/0 (socket says 172.21.15.5:51284) 2024-05-20T18:28:02.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.866+0000 7f2316151640 1 -- 172.21.15.5:0/4011844379 learned_addr learned my addr 172.21.15.5:0/4011844379 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:02.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.866+0000 7f2315950640 1 --2- 172.21.15.5:0/4011844379 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2310151e30 0x7f231006c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:02.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.866+0000 7f2316952640 1 --2- 172.21.15.5:0/4011844379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2310152240 0x7f231006cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:02.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.866+0000 7f2316151640 1 -- 172.21.15.5:0/4011844379 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2310151e30 msgr2=0x7f231006c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:02.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.866+0000 7f2316151640 1 --2- 172.21.15.5:0/4011844379 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2310151e30 0x7f231006c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:02.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.866+0000 7f2316151640 1 -- 172.21.15.5:0/4011844379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2310152240 msgr2=0x7f231006cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:02.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.866+0000 7f2316151640 1 --2- 172.21.15.5:0/4011844379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2310152240 0x7f231006cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:02.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.866+0000 7f2316151640 1 -- 172.21.15.5:0/4011844379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23101f5220 con 0x7f22fc0050c0 2024-05-20T18:28:02.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.866+0000 7f2316151640 1 --2- 172.21.15.5:0/4011844379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22fc0050c0 0x7f2310146290 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f230c00fb60 tx=0x7f230c00ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:02.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.866+0000 7f2306ffd640 1 -- 172.21.15.5:0/4011844379 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c003b80 con 0x7f22fc0050c0 2024-05-20T18:28:02.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.866+0000 7f2306ffd640 1 -- 172.21.15.5:0/4011844379 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f230c018070 con 0x7f22fc0050c0 2024-05-20T18:28:02.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.866+0000 7f23183b5640 1 -- 172.21.15.5:0/4011844379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23101f54e0 con 0x7f22fc0050c0 2024-05-20T18:28:02.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.866+0000 7f2306ffd640 1 -- 172.21.15.5:0/4011844379 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c00d7c0 con 0x7f22fc0050c0 2024-05-20T18:28:02.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.866+0000 7f23183b5640 1 -- 172.21.15.5:0/4011844379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23101f5aa0 con 0x7f22fc0050c0 2024-05-20T18:28:02.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.867+0000 7f23183b5640 1 -- 172.21.15.5:0/4011844379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2310075b00 con 0x7f22fc0050c0 2024-05-20T18:28:02.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.868+0000 7f2306ffd640 1 -- 172.21.15.5:0/4011844379 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f230c00d960 con 0x7f22fc0050c0 2024-05-20T18:28:02.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.868+0000 7f2306ffd640 1 --2- 172.21.15.5:0/4011844379 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f22f406be60 0x7f22f406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:02.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.869+0000 7f2306ffd640 1 -- 172.21.15.5:0/4011844379 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7f230c08b3a0 con 0x7f22fc0050c0 2024-05-20T18:28:02.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.869+0000 7f2315950640 1 --2- 172.21.15.5:0/4011844379 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f22f406be60 0x7f22f406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:02.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.869+0000 7f2315950640 1 --2- 172.21.15.5:0/4011844379 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f22f406be60 0x7f22f406e370 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f23100b4820 tx=0x7f23000078f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:02.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:02.873+0000 7f2306ffd640 1 -- 172.21.15.5:0/4011844379 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f230c0598a0 con 0x7f22fc0050c0 2024-05-20T18:28:03.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.055+0000 7f23183b5640 1 -- 172.21.15.5:0/4011844379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f23101591a0 con 0x7f22fc0050c0 2024-05-20T18:28:03.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.055+0000 7f2306ffd640 1 -- 172.21.15.5:0/4011844379 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f230c0591a0 con 0x7f22fc0050c0 2024-05-20T18:28:03.055 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:28:03.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.057+0000 7f23183b5640 1 -- 172.21.15.5:0/4011844379 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f22f406be60 msgr2=0x7f22f406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:03.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.057+0000 7f23183b5640 1 --2- 172.21.15.5:0/4011844379 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f22f406be60 0x7f22f406e370 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f23100b4820 tx=0x7f23000078f0 comp rx=0 tx=0).stop 2024-05-20T18:28:03.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.057+0000 7f23183b5640 1 -- 172.21.15.5:0/4011844379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22fc0050c0 msgr2=0x7f2310146290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:03.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.057+0000 7f23183b5640 1 --2- 172.21.15.5:0/4011844379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22fc0050c0 0x7f2310146290 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f230c00fb60 tx=0x7f230c00ff60 comp rx=0 tx=0).stop 2024-05-20T18:28:03.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.057+0000 7f23183b5640 1 -- 172.21.15.5:0/4011844379 shutdown_connections 2024-05-20T18:28:03.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.057+0000 7f23183b5640 1 --2- 172.21.15.5:0/4011844379 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f22f406be60 0x7f22f406e370 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:03.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.057+0000 7f23183b5640 1 --2- 172.21.15.5:0/4011844379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2310152240 0x7f231006cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:03.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.057+0000 7f23183b5640 1 --2- 172.21.15.5:0/4011844379 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2310151e30 0x7f231006c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:03.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.057+0000 7f23183b5640 1 --2- 172.21.15.5:0/4011844379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22fc0050c0 0x7f2310146290 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:03.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.057+0000 7f23183b5640 1 -- 172.21.15.5:0/4011844379 >> 172.21.15.5:0/4011844379 conn(0x7f23100b6670 msgr2=0x7f2310067b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:03.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.057+0000 7f23183b5640 1 -- 172.21.15.5:0/4011844379 shutdown_connections 2024-05-20T18:28:03.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.057+0000 7f23183b5640 1 -- 172.21.15.5:0/4011844379 wait complete. 2024-05-20T18:28:03.068 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:28:03.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.191+0000 7f6005ed7640 1 Processor -- start 2024-05-20T18:28:03.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.191+0000 7f6005ed7640 1 -- start start 2024-05-20T18:28:03.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.192+0000 7f6005ed7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f60000cd680 0x7f600015f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:03.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.192+0000 7f6005ed7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f600006c6c0 0x7f600006cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:03.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.192+0000 7f6005ed7640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f600006d0f0 con 0x7f60000cf3d0 2024-05-20T18:28:03.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.192+0000 7f6005ed7640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f60000cda90 con 0x7f60000cd680 2024-05-20T18:28:03.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.192+0000 7f6005ed7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f600015f760 con 0x7f600006c6c0 2024-05-20T18:28:03.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.192+0000 7f5fff7fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f60000cd680 0x7f600015f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:03.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.192+0000 7f5ff6ffd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f600006c6c0 0x7f600006cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:03.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.192+0000 7f5fff7fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f60000cd680 0x7f600015f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:48292/0 (socket says 172.21.15.5:48292) 2024-05-20T18:28:03.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.192+0000 7f5fff7fe640 1 -- 172.21.15.5:0/3285152265 learned_addr learned my addr 172.21.15.5:0/3285152265 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.193+0000 7f5fff7fe640 1 -- 172.21.15.5:0/3285152265 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f600006c6c0 msgr2=0x7f600006cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.193+0000 7f5fff7fe640 1 --2- 172.21.15.5:0/3285152265 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f600006c6c0 0x7f600006cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.193+0000 7f5fff7fe640 1 -- 172.21.15.5:0/3285152265 >> v1:172.21.15.63:6789/0 conn(0x7f60000cf3d0 legacy=0x7f60000cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.193+0000 7f5fff7fe640 1 -- 172.21.15.5:0/3285152265 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f600006d3d0 con 0x7f60000cd680 2024-05-20T18:28:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.193+0000 7f5ffeffd640 1 -- 172.21.15.5:0/3285152265 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2740933092 0 0) 0x7f600006d0f0 con 0x7f60000cf3d0 2024-05-20T18:28:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.194+0000 7f5fff7fe640 1 --2- 172.21.15.5:0/3285152265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60000cd680 0x7f600015f220 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009870 tx=0x7f5ff002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=915c23eaab05c1ee server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.194+0000 7f5ffeffd640 1 -- 172.21.15.5:0/3285152265 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0037070 con 0x7f60000cd680 2024-05-20T18:28:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.194+0000 7f5ffeffd640 1 -- 172.21.15.5:0/3285152265 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ff0032440 con 0x7f60000cd680 2024-05-20T18:28:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.194+0000 7f5ffeffd640 1 -- 172.21.15.5:0/3285152265 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0032810 con 0x7f60000cd680 2024-05-20T18:28:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.194+0000 7f6005ed7640 1 -- 172.21.15.5:0/3285152265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60000cd680 msgr2=0x7f600015f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.194+0000 7f6005ed7640 1 --2- 172.21.15.5:0/3285152265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60000cd680 0x7f600015f220 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009870 tx=0x7f5ff002a1c0 comp rx=0 tx=0).stop 2024-05-20T18:28:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.194+0000 7f6005ed7640 1 -- 172.21.15.5:0/3285152265 shutdown_connections 2024-05-20T18:28:03.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.194+0000 7f6005ed7640 1 --2- 172.21.15.5:0/3285152265 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f600006c6c0 0x7f600006cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:03.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.194+0000 7f6005ed7640 1 --2- 172.21.15.5:0/3285152265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60000cd680 0x7f600015f220 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:03.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.194+0000 7f6005ed7640 1 -- 172.21.15.5:0/3285152265 >> 172.21.15.5:0/3285152265 conn(0x7f60000b44d0 msgr2=0x7f60000b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:03.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.194+0000 7f6005ed7640 1 -- 172.21.15.5:0/3285152265 shutdown_connections 2024-05-20T18:28:03.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.194+0000 7f6005ed7640 1 -- 172.21.15.5:0/3285152265 wait complete. 2024-05-20T18:28:03.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.196+0000 7f6005ed7640 1 Processor -- start 2024-05-20T18:28:03.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.196+0000 7f6005ed7640 1 -- start start 2024-05-20T18:28:03.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.198+0000 7f6005ed7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f600006c6c0 0x7f6000151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:03.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.198+0000 7f6005ed7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60000cd680 0x7f60000c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:03.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.198+0000 7f6005ed7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60000cf3d0 0x7f600015deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:03.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.198+0000 7f6005ed7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f600006d990 con 0x7f60000cf3d0 2024-05-20T18:28:03.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.198+0000 7f6005ed7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f60000d0b40 con 0x7f600006c6c0 2024-05-20T18:28:03.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.198+0000 7f6005ed7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f60000d2460 con 0x7f60000cd680 2024-05-20T18:28:03.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.198+0000 7f5ffffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60000cf3d0 0x7f600015deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:03.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.198+0000 7f5ffffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60000cf3d0 0x7f600015deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51296/0 (socket says 172.21.15.5:51296) 2024-05-20T18:28:03.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.198+0000 7f5ffffff640 1 -- 172.21.15.5:0/833765281 learned_addr learned my addr 172.21.15.5:0/833765281 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:03.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.198+0000 7f5fff7fe640 1 --2- 172.21.15.5:0/833765281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f600006c6c0 0x7f6000151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:03.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.198+0000 7f5ff6ffd640 1 --2- 172.21.15.5:0/833765281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60000cd680 0x7f60000c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:03.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.198+0000 7f5ffffff640 1 -- 172.21.15.5:0/833765281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60000cd680 msgr2=0x7f60000c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:03.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.198+0000 7f5ffffff640 1 --2- 172.21.15.5:0/833765281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60000cd680 0x7f60000c8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:03.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.198+0000 7f5ffffff640 1 -- 172.21.15.5:0/833765281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f600006c6c0 msgr2=0x7f6000151f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:03.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.198+0000 7f5ffffff640 1 --2- 172.21.15.5:0/833765281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f600006c6c0 0x7f6000151f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:03.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.198+0000 7f5ffffff640 1 -- 172.21.15.5:0/833765281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60001609f0 con 0x7f60000cf3d0 2024-05-20T18:28:03.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.198+0000 7f5ff6ffd640 1 --2- 172.21.15.5:0/833765281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60000cd680 0x7f60000c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:03.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.198+0000 7f5ffffff640 1 --2- 172.21.15.5:0/833765281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60000cf3d0 0x7f600015deb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5ff800c780 tx=0x7f5ff800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:03.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.199+0000 7f5ffcff9640 1 -- 172.21.15.5:0/833765281 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8003b70 con 0x7f60000cf3d0 2024-05-20T18:28:03.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.199+0000 7f5ffcff9640 1 -- 172.21.15.5:0/833765281 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ff8003d10 con 0x7f60000cf3d0 2024-05-20T18:28:03.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.199+0000 7f6005ed7640 1 -- 172.21.15.5:0/833765281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6000160cd0 con 0x7f60000cf3d0 2024-05-20T18:28:03.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.199+0000 7f5ffcff9640 1 -- 172.21.15.5:0/833765281 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8005270 con 0x7f60000cf3d0 2024-05-20T18:28:03.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.199+0000 7f6005ed7640 1 -- 172.21.15.5:0/833765281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6000161290 con 0x7f60000cf3d0 2024-05-20T18:28:03.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.200+0000 7f6005ed7640 1 -- 172.21.15.5:0/833765281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6000075b00 con 0x7f60000cf3d0 2024-05-20T18:28:03.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.200+0000 7f5ffcff9640 1 -- 172.21.15.5:0/833765281 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5ff8005410 con 0x7f60000cf3d0 2024-05-20T18:28:03.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.201+0000 7f5ffcff9640 1 --2- 172.21.15.5:0/833765281 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5fe006be30 0x7f5fe006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:03.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.201+0000 7f5fff7fe640 1 --2- 172.21.15.5:0/833765281 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5fe006be30 0x7f5fe006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:03.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.201+0000 7f5ffcff9640 1 -- 172.21.15.5:0/833765281 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7f5ff808b210 con 0x7f60000cf3d0 2024-05-20T18:28:03.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.201+0000 7f5fff7fe640 1 --2- 172.21.15.5:0/833765281 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5fe006be30 0x7f5fe006e340 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0034000 tx=0x7f5ff002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:03.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.205+0000 7f5ffcff9640 1 -- 172.21.15.5:0/833765281 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5ff8059680 con 0x7f60000cf3d0 2024-05-20T18:28:03.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.390+0000 7f6005ed7640 1 -- 172.21.15.5:0/833765281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f60000b8940 con 0x7f60000cf3d0 2024-05-20T18:28:03.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.390+0000 7f5ffcff9640 1 -- 172.21.15.5:0/833765281 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7f5ff8058f80 con 0x7f60000cf3d0 2024-05-20T18:28:03.389 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:28:03.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.392+0000 7f6005ed7640 1 -- 172.21.15.5:0/833765281 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5fe006be30 msgr2=0x7f5fe006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:03.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.392+0000 7f6005ed7640 1 --2- 172.21.15.5:0/833765281 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5fe006be30 0x7f5fe006e340 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0034000 tx=0x7f5ff002e000 comp rx=0 tx=0).stop 2024-05-20T18:28:03.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.392+0000 7f6005ed7640 1 -- 172.21.15.5:0/833765281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60000cf3d0 msgr2=0x7f600015deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:03.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.392+0000 7f6005ed7640 1 --2- 172.21.15.5:0/833765281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60000cf3d0 0x7f600015deb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5ff800c780 tx=0x7f5ff800cb80 comp rx=0 tx=0).stop 2024-05-20T18:28:03.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.392+0000 7f6005ed7640 1 -- 172.21.15.5:0/833765281 shutdown_connections 2024-05-20T18:28:03.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.392+0000 7f6005ed7640 1 --2- 172.21.15.5:0/833765281 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5fe006be30 0x7f5fe006e340 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:03.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.392+0000 7f6005ed7640 1 --2- 172.21.15.5:0/833765281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60000cf3d0 0x7f600015deb0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:03.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.392+0000 7f6005ed7640 1 --2- 172.21.15.5:0/833765281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60000cd680 0x7f60000c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:03.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.392+0000 7f6005ed7640 1 --2- 172.21.15.5:0/833765281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f600006c6c0 0x7f6000151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:03.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.392+0000 7f6005ed7640 1 -- 172.21.15.5:0/833765281 >> 172.21.15.5:0/833765281 conn(0x7f60000b44d0 msgr2=0x7f60000cebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:03.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.392+0000 7f6005ed7640 1 -- 172.21.15.5:0/833765281 shutdown_connections 2024-05-20T18:28:03.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.392+0000 7f6005ed7640 1 -- 172.21.15.5:0/833765281 wait complete. 2024-05-20T18:28:03.403 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T18:28:03.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.522+0000 7f675b6f7640 1 Processor -- start 2024-05-20T18:28:03.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.522+0000 7f675b6f7640 1 -- start start 2024-05-20T18:28:03.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.523+0000 7f675b6f7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6754163a10 0x7f67540d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:03.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.523+0000 7f675b6f7640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67540d0370 con 0x7f67540d14b0 2024-05-20T18:28:03.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.523+0000 7f675b6f7640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67540d0650 con 0x7f67540b4010 2024-05-20T18:28:03.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.523+0000 7f675b6f7640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f67540d3ca0 con 0x7f6754163a10 2024-05-20T18:28:03.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.523+0000 7f6759c94640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f67540d14b0 0x7f67540d18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:54030/0 (socket says 172.21.15.5:54030) 2024-05-20T18:28:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.523+0000 7f6759c94640 1 -- 172.21.15.5:0/2050984199 learned_addr learned my addr 172.21.15.5:0/2050984199 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.524+0000 7f6759493640 1 --2- 172.21.15.5:0/2050984199 >> v2:172.21.15.63:3300/0 conn(0x7f6754163a10 0x7f67540d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.524+0000 7f6747fff640 1 -- 172.21.15.5:0/2050984199 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1628403200 0 0) 0x7f67540d0370 con 0x7f67540d14b0 2024-05-20T18:28:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.524+0000 7f6747fff640 1 -- 172.21.15.5:0/2050984199 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6740003520 con 0x7f67540d14b0 2024-05-20T18:28:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.524+0000 7f6759493640 1 -- 172.21.15.5:0/2050984199 >> v1:172.21.15.63:6789/0 conn(0x7f67540b4010 legacy=0x7f67540cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.524+0000 7f6747fff640 1 -- 172.21.15.5:0/2050984199 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1934727897 0 0) 0x7f67540d0650 con 0x7f67540b4010 2024-05-20T18:28:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.524+0000 7f6759493640 1 -- 172.21.15.5:0/2050984199 >> v1:172.21.15.5:6789/0 conn(0x7f67540d14b0 legacy=0x7f67540d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.524+0000 7f6759493640 1 -- 172.21.15.5:0/2050984199 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67540d3650 con 0x7f6754163a10 2024-05-20T18:28:03.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.525+0000 7f6759493640 1 --2- 172.21.15.5:0/2050984199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6754163a10 0x7f67540d4130 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f67500097a0 tx=0x7f675002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b22258a11163e70 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:03.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.525+0000 7f6747fff640 1 -- 172.21.15.5:0/2050984199 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750037070 con 0x7f6754163a10 2024-05-20T18:28:03.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.525+0000 7f6747fff640 1 -- 172.21.15.5:0/2050984199 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6750032440 con 0x7f6754163a10 2024-05-20T18:28:03.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.525+0000 7f6747fff640 1 -- 172.21.15.5:0/2050984199 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750032810 con 0x7f6754163a10 2024-05-20T18:28:03.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.526+0000 7f675b6f7640 1 -- 172.21.15.5:0/2050984199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6754163a10 msgr2=0x7f67540d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:03.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.526+0000 7f675b6f7640 1 --2- 172.21.15.5:0/2050984199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6754163a10 0x7f67540d4130 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f67500097a0 tx=0x7f675002a230 comp rx=0 tx=0).stop 2024-05-20T18:28:03.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.526+0000 7f675b6f7640 1 -- 172.21.15.5:0/2050984199 shutdown_connections 2024-05-20T18:28:03.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.526+0000 7f675b6f7640 1 --2- 172.21.15.5:0/2050984199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6754163a10 0x7f67540d4130 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:03.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.526+0000 7f675b6f7640 1 -- 172.21.15.5:0/2050984199 >> 172.21.15.5:0/2050984199 conn(0x7f675406d240 msgr2=0x7f675406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:03.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.526+0000 7f675b6f7640 1 -- 172.21.15.5:0/2050984199 shutdown_connections 2024-05-20T18:28:03.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.526+0000 7f675b6f7640 1 -- 172.21.15.5:0/2050984199 wait complete. 2024-05-20T18:28:03.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.528+0000 7f675b6f7640 1 Processor -- start 2024-05-20T18:28:03.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.528+0000 7f675b6f7640 1 -- start start 2024-05-20T18:28:03.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.529+0000 7f675b6f7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67540b4010 0x7f6754157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:03.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.529+0000 7f675b6f7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67540d14b0 0x7f67540ce320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.529+0000 7f675b6f7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6754163a10 0x7f67541f8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.529+0000 7f675b6f7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f67540b8390 con 0x7f67540d14b0 2024-05-20T18:28:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.529+0000 7f675b6f7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f67540b8210 con 0x7f67540b4010 2024-05-20T18:28:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.529+0000 7f675b6f7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f67540b8510 con 0x7f6754163a10 2024-05-20T18:28:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.529+0000 7f6758c92640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67540d14b0 0x7f67540ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.529+0000 7f6758c92640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67540d14b0 0x7f67540ce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51312/0 (socket says 172.21.15.5:51312) 2024-05-20T18:28:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.529+0000 7f6758c92640 1 -- 172.21.15.5:0/2605069228 learned_addr learned my addr 172.21.15.5:0/2605069228 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.529+0000 7f6759c94640 1 --2- 172.21.15.5:0/2605069228 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6754163a10 0x7f67541f8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.529+0000 7f6759493640 1 --2- 172.21.15.5:0/2605069228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67540b4010 0x7f6754157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.530+0000 7f6758c92640 1 -- 172.21.15.5:0/2605069228 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6754163a10 msgr2=0x7f67541f8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.530+0000 7f6758c92640 1 --2- 172.21.15.5:0/2605069228 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6754163a10 0x7f67541f8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.530+0000 7f6758c92640 1 -- 172.21.15.5:0/2605069228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67540b4010 msgr2=0x7f6754157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.530+0000 7f6758c92640 1 --2- 172.21.15.5:0/2605069228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67540b4010 0x7f6754157290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:03.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.530+0000 7f6758c92640 1 -- 172.21.15.5:0/2605069228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67540ce860 con 0x7f67540d14b0 2024-05-20T18:28:03.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.530+0000 7f6758c92640 1 --2- 172.21.15.5:0/2605069228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67540d14b0 0x7f67540ce320 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f674800c440 tx=0x7f674800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:03.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.530+0000 7f6745ffb640 1 -- 172.21.15.5:0/2605069228 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6748017070 con 0x7f67540d14b0 2024-05-20T18:28:03.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.530+0000 7f6745ffb640 1 -- 172.21.15.5:0/2605069228 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67480035f0 con 0x7f67540d14b0 2024-05-20T18:28:03.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.530+0000 7f675b6f7640 1 -- 172.21.15.5:0/2605069228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6754154cf0 con 0x7f67540d14b0 2024-05-20T18:28:03.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.530+0000 7f6745ffb640 1 -- 172.21.15.5:0/2605069228 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67480039e0 con 0x7f67540d14b0 2024-05-20T18:28:03.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.530+0000 7f675b6f7640 1 -- 172.21.15.5:0/2605069228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67541552b0 con 0x7f67540d14b0 2024-05-20T18:28:03.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.531+0000 7f675b6f7640 1 -- 172.21.15.5:0/2605069228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6754075b00 con 0x7f67540d14b0 2024-05-20T18:28:03.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.532+0000 7f6745ffb640 1 -- 172.21.15.5:0/2605069228 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f674801a070 con 0x7f67540d14b0 2024-05-20T18:28:03.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.532+0000 7f6745ffb640 1 --2- 172.21.15.5:0/2605069228 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f673806be60 0x7f673806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:03.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.532+0000 7f6745ffb640 1 -- 172.21.15.5:0/2605069228 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7f674808a5c0 con 0x7f67540d14b0 2024-05-20T18:28:03.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.532+0000 7f6759493640 1 --2- 172.21.15.5:0/2605069228 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f673806be60 0x7f673806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:03.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.533+0000 7f6759493640 1 --2- 172.21.15.5:0/2605069228 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f673806be60 0x7f673806e370 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6750034000 tx=0x7f675002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:03.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.537+0000 7f6745ffb640 1 -- 172.21.15.5:0/2605069228 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f67540ce860 con 0x7f67540d14b0 2024-05-20T18:28:03.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.720+0000 7f675b6f7640 1 -- 172.21.15.5:0/2605069228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f67540d3050 con 0x7f67540d14b0 2024-05-20T18:28:03.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.721+0000 7f6745ffb640 1 -- 172.21.15.5:0/2605069228 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f6748058510 con 0x7f67540d14b0 2024-05-20T18:28:03.721 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:28:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.723+0000 7f675b6f7640 1 -- 172.21.15.5:0/2605069228 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f673806be60 msgr2=0x7f673806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.723+0000 7f675b6f7640 1 --2- 172.21.15.5:0/2605069228 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f673806be60 0x7f673806e370 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6750034000 tx=0x7f675002e000 comp rx=0 tx=0).stop 2024-05-20T18:28:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.723+0000 7f675b6f7640 1 -- 172.21.15.5:0/2605069228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67540d14b0 msgr2=0x7f67540ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.723+0000 7f675b6f7640 1 --2- 172.21.15.5:0/2605069228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67540d14b0 0x7f67540ce320 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f674800c440 tx=0x7f674800c900 comp rx=0 tx=0).stop 2024-05-20T18:28:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.723+0000 7f675b6f7640 1 -- 172.21.15.5:0/2605069228 shutdown_connections 2024-05-20T18:28:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.723+0000 7f675b6f7640 1 --2- 172.21.15.5:0/2605069228 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f673806be60 0x7f673806e370 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.723+0000 7f675b6f7640 1 --2- 172.21.15.5:0/2605069228 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6754163a10 0x7f67541f8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.723+0000 7f675b6f7640 1 --2- 172.21.15.5:0/2605069228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67540d14b0 0x7f67540ce320 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.723+0000 7f675b6f7640 1 --2- 172.21.15.5:0/2605069228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67540b4010 0x7f6754157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.723+0000 7f675b6f7640 1 -- 172.21.15.5:0/2605069228 >> 172.21.15.5:0/2605069228 conn(0x7f675406d240 msgr2=0x7f675415fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:03.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.723+0000 7f675b6f7640 1 -- 172.21.15.5:0/2605069228 shutdown_connections 2024-05-20T18:28:03.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.723+0000 7f675b6f7640 1 -- 172.21.15.5:0/2605069228 wait complete. 2024-05-20T18:28:03.734 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T18:28:03.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.853+0000 7f194e7d5640 1 Processor -- start 2024-05-20T18:28:03.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.854+0000 7f194e7d5640 1 -- start start 2024-05-20T18:28:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.855+0000 7f194e7d5640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1948162820 0x7f19480b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.855+0000 7f194e7d5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19480b3e80 0x7f194806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.855+0000 7f194e7d5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f194806d910 0x7f19480d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.855+0000 7f194e7d5640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f194806dd20 con 0x7f194806d910 2024-05-20T18:28:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.855+0000 7f194e7d5640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f19480d0630 con 0x7f1948162820 2024-05-20T18:28:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.855+0000 7f194e7d5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f19480d39c0 con 0x7f19480b3e80 2024-05-20T18:28:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.855+0000 7f194cd72640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f194806d910 0x7f19480d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.855+0000 7f194cd72640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f194806d910 0x7f19480d27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51314/0 (socket says 172.21.15.5:51314) 2024-05-20T18:28:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.855+0000 7f194cd72640 1 -- 172.21.15.5:0/1113095805 learned_addr learned my addr 172.21.15.5:0/1113095805 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.855+0000 7f19477fe640 1 --2- 172.21.15.5:0/1113095805 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19480b3e80 0x7f194806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.855+0000 7f1947fff640 1 --2- 172.21.15.5:0/1113095805 >> v2:172.21.15.63:3300/0 conn(0x7f1948162820 0x7f19480b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:03.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.855+0000 7f194cd72640 1 -- 172.21.15.5:0/1113095805 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19480b3e80 msgr2=0x7f194806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:03.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.855+0000 7f194cd72640 1 --2- 172.21.15.5:0/1113095805 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19480b3e80 0x7f194806d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:03.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.855+0000 7f194cd72640 1 -- 172.21.15.5:0/1113095805 >> v2:172.21.15.63:3300/0 conn(0x7f1948162820 msgr2=0x7f19480b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:03.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.855+0000 7f194cd72640 1 --2- 172.21.15.5:0/1113095805 >> v2:172.21.15.63:3300/0 conn(0x7f1948162820 0x7f19480b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:03.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.855+0000 7f194cd72640 1 -- 172.21.15.5:0/1113095805 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19480d3ca0 con 0x7f194806d910 2024-05-20T18:28:03.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.856+0000 7f194cd72640 1 --2- 172.21.15.5:0/1113095805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f194806d910 0x7f19480d27a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f19480b7740 tx=0x7f194002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e8188591a720f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:03.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.856+0000 7f1946ffd640 1 -- 172.21.15.5:0/1113095805 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1940036070 con 0x7f194806d910 2024-05-20T18:28:03.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.856+0000 7f1946ffd640 1 -- 172.21.15.5:0/1113095805 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1940031600 con 0x7f194806d910 2024-05-20T18:28:03.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.856+0000 7f1946ffd640 1 -- 172.21.15.5:0/1113095805 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19400319d0 con 0x7f194806d910 2024-05-20T18:28:03.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.857+0000 7f194e7d5640 1 -- 172.21.15.5:0/1113095805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f194806d910 msgr2=0x7f19480d27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:03.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.857+0000 7f194e7d5640 1 --2- 172.21.15.5:0/1113095805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f194806d910 0x7f19480d27a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f19480b7740 tx=0x7f194002a6a0 comp rx=0 tx=0).stop 2024-05-20T18:28:03.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.857+0000 7f194e7d5640 1 -- 172.21.15.5:0/1113095805 shutdown_connections 2024-05-20T18:28:03.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.857+0000 7f194e7d5640 1 --2- 172.21.15.5:0/1113095805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f194806d910 0x7f19480d27a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:03.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.857+0000 7f194e7d5640 1 --2- 172.21.15.5:0/1113095805 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19480b3e80 0x7f194806d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:03.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.857+0000 7f194e7d5640 1 --2- 172.21.15.5:0/1113095805 >> v2:172.21.15.63:3300/0 conn(0x7f1948162820 0x7f19480b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:03.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.857+0000 7f194e7d5640 1 -- 172.21.15.5:0/1113095805 >> 172.21.15.5:0/1113095805 conn(0x7f19480d4bd0 msgr2=0x7f19480d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:03.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.857+0000 7f194e7d5640 1 -- 172.21.15.5:0/1113095805 shutdown_connections 2024-05-20T18:28:03.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.857+0000 7f194e7d5640 1 -- 172.21.15.5:0/1113095805 wait complete. 2024-05-20T18:28:03.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.859+0000 7f194e7d5640 1 Processor -- start 2024-05-20T18:28:03.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.859+0000 7f194e7d5640 1 -- start start 2024-05-20T18:28:03.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.860+0000 7f194e7d5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f194806d910 0x7f19480cd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:03.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.860+0000 7f194e7d5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19480b3e80 0x7f19480cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:03.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.860+0000 7f194e7d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1948162820 0x7f19480b66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:03.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.860+0000 7f194e7d5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f19480d0300 con 0x7f1948162820 2024-05-20T18:28:03.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.860+0000 7f194e7d5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f19480d4260 con 0x7f194806d910 2024-05-20T18:28:03.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.860+0000 7f194e7d5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f19480d4780 con 0x7f19480b3e80 2024-05-20T18:28:03.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.861+0000 7f194cd72640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1948162820 0x7f19480b66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:03.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.861+0000 7f194cd72640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1948162820 0x7f19480b66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51316/0 (socket says 172.21.15.5:51316) 2024-05-20T18:28:03.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.861+0000 7f194cd72640 1 -- 172.21.15.5:0/3420808893 learned_addr learned my addr 172.21.15.5:0/3420808893 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:03.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.861+0000 7f1947fff640 1 --2- 172.21.15.5:0/3420808893 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f194806d910 0x7f19480cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:03.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.861+0000 7f19477fe640 1 --2- 172.21.15.5:0/3420808893 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19480b3e80 0x7f19480cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:03.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.861+0000 7f194cd72640 1 -- 172.21.15.5:0/3420808893 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19480b3e80 msgr2=0x7f19480cbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:03.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.861+0000 7f194cd72640 1 --2- 172.21.15.5:0/3420808893 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19480b3e80 0x7f19480cbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:03.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.861+0000 7f194cd72640 1 -- 172.21.15.5:0/3420808893 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f194806d910 msgr2=0x7f19480cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:03.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.861+0000 7f194cd72640 1 --2- 172.21.15.5:0/3420808893 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f194806d910 0x7f19480cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:03.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.861+0000 7f194cd72640 1 -- 172.21.15.5:0/3420808893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19480cc660 con 0x7f1948162820 2024-05-20T18:28:03.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.861+0000 7f1947fff640 1 --2- 172.21.15.5:0/3420808893 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f194806d910 0x7f19480cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:03.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.861+0000 7f19477fe640 1 --2- 172.21.15.5:0/3420808893 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19480b3e80 0x7f19480cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:03.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.861+0000 7f194cd72640 1 --2- 172.21.15.5:0/3420808893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1948162820 0x7f19480b66d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1940034040 tx=0x7f1940004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:03.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.861+0000 7f1944ff9640 1 -- 172.21.15.5:0/3420808893 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1940036050 con 0x7f1948162820 2024-05-20T18:28:03.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.861+0000 7f1944ff9640 1 -- 172.21.15.5:0/3420808893 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1940042070 con 0x7f1948162820 2024-05-20T18:28:03.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.861+0000 7f194e7d5640 1 -- 172.21.15.5:0/3420808893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19480cc940 con 0x7f1948162820 2024-05-20T18:28:03.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.861+0000 7f1944ff9640 1 -- 172.21.15.5:0/3420808893 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1940009510 con 0x7f1948162820 2024-05-20T18:28:03.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.861+0000 7f194e7d5640 1 -- 172.21.15.5:0/3420808893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19480ccf00 con 0x7f1948162820 2024-05-20T18:28:03.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.862+0000 7f194e7d5640 1 -- 172.21.15.5:0/3420808893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1948075b00 con 0x7f1948162820 2024-05-20T18:28:03.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.863+0000 7f1944ff9640 1 -- 172.21.15.5:0/3420808893 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1940007bf0 con 0x7f1948162820 2024-05-20T18:28:03.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.864+0000 7f1944ff9640 1 --2- 172.21.15.5:0/3420808893 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f193406be60 0x7f193406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:03.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.864+0000 7f1947fff640 1 --2- 172.21.15.5:0/3420808893 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f193406be60 0x7f193406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:03.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.864+0000 7f1944ff9640 1 -- 172.21.15.5:0/3420808893 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7f19400ac030 con 0x7f1948162820 2024-05-20T18:28:03.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.864+0000 7f1947fff640 1 --2- 172.21.15.5:0/3420808893 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f193406be60 0x7f193406e370 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1938009a20 tx=0x7f1938008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:03.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:03.868+0000 7f1944ff9640 1 -- 172.21.15.5:0/3420808893 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f194007a540 con 0x7f1948162820 2024-05-20T18:28:04.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.065+0000 7f194e7d5640 1 -- 172.21.15.5:0/3420808893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f19480cc3f0 con 0x7f1948162820 2024-05-20T18:28:04.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.065+0000 7f1944ff9640 1 -- 172.21.15.5:0/3420808893 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7f1940079e40 con 0x7f1948162820 2024-05-20T18:28:04.065 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-05-20T18:28:04.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.067+0000 7f194e7d5640 1 -- 172.21.15.5:0/3420808893 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f193406be60 msgr2=0x7f193406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:04.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.067+0000 7f194e7d5640 1 --2- 172.21.15.5:0/3420808893 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f193406be60 0x7f193406e370 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1938009a20 tx=0x7f1938008040 comp rx=0 tx=0).stop 2024-05-20T18:28:04.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.067+0000 7f194e7d5640 1 -- 172.21.15.5:0/3420808893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1948162820 msgr2=0x7f19480b66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:04.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.067+0000 7f194e7d5640 1 --2- 172.21.15.5:0/3420808893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1948162820 0x7f19480b66d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1940034040 tx=0x7f1940004a40 comp rx=0 tx=0).stop 2024-05-20T18:28:04.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.067+0000 7f194e7d5640 1 -- 172.21.15.5:0/3420808893 shutdown_connections 2024-05-20T18:28:04.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.067+0000 7f194e7d5640 1 --2- 172.21.15.5:0/3420808893 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f193406be60 0x7f193406e370 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:04.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.067+0000 7f194e7d5640 1 --2- 172.21.15.5:0/3420808893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1948162820 0x7f19480b66d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:04.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.067+0000 7f194e7d5640 1 --2- 172.21.15.5:0/3420808893 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19480b3e80 0x7f19480cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:04.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.067+0000 7f194e7d5640 1 --2- 172.21.15.5:0/3420808893 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f194806d910 0x7f19480cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:04.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.067+0000 7f194e7d5640 1 -- 172.21.15.5:0/3420808893 >> 172.21.15.5:0/3420808893 conn(0x7f19480d4bd0 msgr2=0x7f19480d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:04.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.067+0000 7f194e7d5640 1 -- 172.21.15.5:0/3420808893 shutdown_connections 2024-05-20T18:28:04.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.067+0000 7f194e7d5640 1 -- 172.21.15.5:0/3420808893 wait complete. 2024-05-20T18:28:04.078 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-20T18:28:04.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:28:04.079 DEBUG:teuthology.orchestra.run.smithi063:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-05-20T18:28:04.114 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi063.front.sepia.ceph.com 2024-05-20T18:28:04.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:28:04.114 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-20T18:28:04.182 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:28:04.182 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:28:04.200 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:28:04.200 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:28:04.258 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:28:04.258 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:28:04.315 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:28:04.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.474+0000 7fb01a851640 1 Processor -- start 2024-05-20T18:28:04.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.474+0000 7fb01a851640 1 -- start start 2024-05-20T18:28:04.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.475+0000 7fb01a851640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb014162820 0x7fb0140b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:04.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.475+0000 7fb01a851640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb01406dae0 0x7fb0140b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:04.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.475+0000 7fb01a851640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0140d0630 con 0x7fb0140b3e80 2024-05-20T18:28:04.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.475+0000 7fb01a851640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb0140d39c0 con 0x7fb01406dae0 2024-05-20T18:28:04.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.475+0000 7fb01a851640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb0140d3ca0 con 0x7fb014162820 2024-05-20T18:28:04.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.475+0000 7fb0137fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb0140b3e80 0x7fb01406d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:54038/0 (socket says 172.21.15.5:54038) 2024-05-20T18:28:04.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.475+0000 7fb0137fe640 1 -- 172.21.15.5:0/1409397458 learned_addr learned my addr 172.21.15.5:0/1409397458 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:04.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.476+0000 7fb018dee640 1 --2- 172.21.15.5:0/1409397458 >> v2:172.21.15.63:3300/0 conn(0x7fb01406dae0 0x7fb0140b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:04.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.476+0000 7fb013fff640 1 --2- 172.21.15.5:0/1409397458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb014162820 0x7fb0140b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:04.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.476+0000 7fb012ffd640 1 -- 172.21.15.5:0/1409397458 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1612909260 0 0) 0x7fb0140d0630 con 0x7fb0140b3e80 2024-05-20T18:28:04.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.476+0000 7fb012ffd640 1 -- 172.21.15.5:0/1409397458 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faffc003520 con 0x7fb0140b3e80 2024-05-20T18:28:04.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.476+0000 7fb012ffd640 1 -- 172.21.15.5:0/1409397458 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2701514732 0 0) 0x7faffc003520 con 0x7fb0140b3e80 2024-05-20T18:28:04.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.476+0000 7fb012ffd640 1 -- 172.21.15.5:0/1409397458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb014162820 msgr2=0x7fb0140b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:04.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.476+0000 7fb012ffd640 1 --2- 172.21.15.5:0/1409397458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb014162820 0x7fb0140b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:04.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.476+0000 7fb012ffd640 1 -- 172.21.15.5:0/1409397458 >> v2:172.21.15.63:3300/0 conn(0x7fb01406dae0 msgr2=0x7fb0140b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:04.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.476+0000 7fb012ffd640 1 --2- 172.21.15.5:0/1409397458 >> v2:172.21.15.63:3300/0 conn(0x7fb01406dae0 0x7fb0140b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:04.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.476+0000 7fb012ffd640 1 -- 172.21.15.5:0/1409397458 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0140d3f80 con 0x7fb0140b3e80 2024-05-20T18:28:04.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.476+0000 7fb012ffd640 1 -- 172.21.15.5:0/1409397458 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7fb008001670 con 0x7fb0140b3e80 2024-05-20T18:28:04.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.476+0000 7fb012ffd640 1 -- 172.21.15.5:0/1409397458 >> v1:172.21.15.5:6789/0 conn(0x7fb0140b3e80 legacy=0x7fb01406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:04.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.476+0000 7fb012ffd640 1 --2- 172.21.15.5:0/1409397458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faffc0039b0 0x7faffc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:04.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.476+0000 7fb012ffd640 1 --2- 172.21.15.5:0/1409397458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faffc004800 0x7faffc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:04.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.476+0000 7fb012ffd640 1 --2- 172.21.15.5:0/1409397458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb014162820 0x7faffc005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:04.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.476+0000 7fb012ffd640 1 -- 172.21.15.5:0/1409397458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb0140d39c0 con 0x7faffc0039b0 2024-05-20T18:28:04.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.476+0000 7fb012ffd640 1 -- 172.21.15.5:0/1409397458 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb0140d3ca0 con 0x7faffc004800 2024-05-20T18:28:04.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.476+0000 7fb012ffd640 1 -- 172.21.15.5:0/1409397458 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faffc00b9c0 con 0x7fb014162820 2024-05-20T18:28:04.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.476+0000 7fb013fff640 1 --2- 172.21.15.5:0/1409397458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faffc0039b0 0x7faffc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:04.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.476+0000 7fb0137fe640 1 --2- 172.21.15.5:0/1409397458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faffc004800 0x7faffc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:04.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.476+0000 7fb018dee640 1 --2- 172.21.15.5:0/1409397458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb014162820 0x7faffc005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:04.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.476+0000 7fb013fff640 1 -- 172.21.15.5:0/1409397458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb014162820 msgr2=0x7faffc005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:04.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.476+0000 7fb013fff640 1 --2- 172.21.15.5:0/1409397458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb014162820 0x7faffc005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:04.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.476+0000 7fb013fff640 1 -- 172.21.15.5:0/1409397458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faffc004800 msgr2=0x7faffc004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:04.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.476+0000 7fb013fff640 1 --2- 172.21.15.5:0/1409397458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faffc004800 0x7faffc004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:04.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.476+0000 7fb013fff640 1 -- 172.21.15.5:0/1409397458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faffc003520 con 0x7faffc0039b0 2024-05-20T18:28:04.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.477+0000 7fb0137fe640 1 --2- 172.21.15.5:0/1409397458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faffc004800 0x7faffc004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:04.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.477+0000 7fb018dee640 1 --2- 172.21.15.5:0/1409397458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb014162820 0x7faffc005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:04.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.477+0000 7fb013fff640 1 --2- 172.21.15.5:0/1409397458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faffc0039b0 0x7faffc003e20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb004009ac0 tx=0x7faffc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcfe977ce91f0cf server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:04.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.477+0000 7fb012ffd640 1 -- 172.21.15.5:0/1409397458 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb004036070 con 0x7faffc0039b0 2024-05-20T18:28:04.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.477+0000 7fb012ffd640 1 -- 172.21.15.5:0/1409397458 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb004031400 con 0x7faffc0039b0 2024-05-20T18:28:04.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.477+0000 7fb012ffd640 1 -- 172.21.15.5:0/1409397458 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb004009690 con 0x7faffc0039b0 2024-05-20T18:28:04.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.478+0000 7fb01a851640 1 -- 172.21.15.5:0/1409397458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faffc0039b0 msgr2=0x7faffc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:04.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.478+0000 7fb01a851640 1 --2- 172.21.15.5:0/1409397458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faffc0039b0 0x7faffc003e20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb004009ac0 tx=0x7faffc005830 comp rx=0 tx=0).stop 2024-05-20T18:28:04.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.478+0000 7fb018dee640 1 -- 172.21.15.5:0/1409397458 reap_dead start 2024-05-20T18:28:04.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.478+0000 7fb01a851640 1 -- 172.21.15.5:0/1409397458 shutdown_connections 2024-05-20T18:28:04.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.478+0000 7fb01a851640 1 -- 172.21.15.5:0/1409397458 >> 172.21.15.5:0/1409397458 conn(0x7fb0140d4bd0 msgr2=0x7fb0140d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:04.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.478+0000 7fb01a851640 1 -- 172.21.15.5:0/1409397458 shutdown_connections 2024-05-20T18:28:04.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.478+0000 7fb01a851640 1 -- 172.21.15.5:0/1409397458 wait complete. 2024-05-20T18:28:04.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.480+0000 7fb01a851640 1 Processor -- start 2024-05-20T18:28:04.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.480+0000 7fb01a851640 1 -- start start 2024-05-20T18:28:04.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.481+0000 7fb01a851640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faffc0039b0 0x7fb0140b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:04.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.481+0000 7fb01a851640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb014157990 0x7fb0141f81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:04.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.481+0000 7fb01a851640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0141f8710 0x7fb0140cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:04.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.481+0000 7fb01a851640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb0140d4780 con 0x7faffc0039b0 2024-05-20T18:28:04.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.481+0000 7fb01a851640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb0140d0300 con 0x7fb014157990 2024-05-20T18:28:04.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.481+0000 7fb01a851640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb0140d4900 con 0x7fb0141f8710 2024-05-20T18:28:04.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.482+0000 7fb013fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faffc0039b0 0x7fb0140b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:04.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.482+0000 7fb013fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faffc0039b0 0x7fb0140b47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51342/0 (socket says 172.21.15.5:51342) 2024-05-20T18:28:04.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.482+0000 7fb013fff640 1 -- 172.21.15.5:0/2692846116 learned_addr learned my addr 172.21.15.5:0/2692846116 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:04.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.482+0000 7fb0137fe640 1 --2- 172.21.15.5:0/2692846116 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb014157990 0x7fb0141f81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:04.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.482+0000 7fb018dee640 1 --2- 172.21.15.5:0/2692846116 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0141f8710 0x7fb0140cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:04.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.482+0000 7fb013fff640 1 -- 172.21.15.5:0/2692846116 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0141f8710 msgr2=0x7fb0140cbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:04.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.482+0000 7fb013fff640 1 --2- 172.21.15.5:0/2692846116 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0141f8710 0x7fb0140cbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:04.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.482+0000 7fb013fff640 1 -- 172.21.15.5:0/2692846116 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb014157990 msgr2=0x7fb0141f81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:04.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.482+0000 7fb013fff640 1 --2- 172.21.15.5:0/2692846116 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb014157990 0x7fb0141f81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:04.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.482+0000 7fb013fff640 1 -- 172.21.15.5:0/2692846116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0141f8b20 con 0x7faffc0039b0 2024-05-20T18:28:04.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.482+0000 7fb013fff640 1 --2- 172.21.15.5:0/2692846116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faffc0039b0 0x7fb0140b47f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb004009ac0 tx=0x7fb004004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:04.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.482+0000 7fb010ff9640 1 -- 172.21.15.5:0/2692846116 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb004040020 con 0x7faffc0039b0 2024-05-20T18:28:04.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.482+0000 7fb010ff9640 1 -- 172.21.15.5:0/2692846116 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb004003940 con 0x7faffc0039b0 2024-05-20T18:28:04.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.482+0000 7fb010ff9640 1 -- 172.21.15.5:0/2692846116 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb004036040 con 0x7faffc0039b0 2024-05-20T18:28:04.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.482+0000 7fb01a851640 1 -- 172.21.15.5:0/2692846116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0141f4a10 con 0x7faffc0039b0 2024-05-20T18:28:04.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.482+0000 7fb01a851640 1 -- 172.21.15.5:0/2692846116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0140b54b0 con 0x7faffc0039b0 2024-05-20T18:28:04.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.483+0000 7fb01a851640 1 -- 172.21.15.5:0/2692846116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0140d00e0 con 0x7faffc0039b0 2024-05-20T18:28:04.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.484+0000 7fb010ff9640 1 -- 172.21.15.5:0/2692846116 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb004043020 con 0x7faffc0039b0 2024-05-20T18:28:04.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.484+0000 7fb010ff9640 1 --2- 172.21.15.5:0/2692846116 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb00006be60 0x7fb00006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:04.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.485+0000 7fb0137fe640 1 --2- 172.21.15.5:0/2692846116 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb00006be60 0x7fb00006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:04.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.485+0000 7fb010ff9640 1 -- 172.21.15.5:0/2692846116 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7fb0040ab660 con 0x7faffc0039b0 2024-05-20T18:28:04.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.485+0000 7fb0137fe640 1 --2- 172.21.15.5:0/2692846116 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb00006be60 0x7fb00006e370 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb0140b64c0 tx=0x7fb008009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:04.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.489+0000 7fb010ff9640 1 -- 172.21.15.5:0/2692846116 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb0140d00e0 con 0x7faffc0039b0 2024-05-20T18:28:04.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.674+0000 7fb01a851640 1 -- 172.21.15.5:0/2692846116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb014075b00 con 0x7faffc0039b0 2024-05-20T18:28:04.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.675+0000 7fb010ff9640 1 -- 172.21.15.5:0/2692846116 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18800 (secure 0 0 0) 0x7fb00404a070 con 0x7faffc0039b0 2024-05-20T18:28:04.675 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:04.675 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:27:56.213938+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:27:05.476596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T18:27:06.137762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T18:27:09.829920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:28:04.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.677+0000 7fb01a851640 1 -- 172.21.15.5:0/2692846116 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb00006be60 msgr2=0x7fb00006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:04.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.677+0000 7fb01a851640 1 --2- 172.21.15.5:0/2692846116 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb00006be60 0x7fb00006e370 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb0140b64c0 tx=0x7fb008009040 comp rx=0 tx=0).stop 2024-05-20T18:28:04.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.677+0000 7fb01a851640 1 -- 172.21.15.5:0/2692846116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faffc0039b0 msgr2=0x7fb0140b47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:04.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.677+0000 7fb01a851640 1 --2- 172.21.15.5:0/2692846116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faffc0039b0 0x7fb0140b47f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb004009ac0 tx=0x7fb004004730 comp rx=0 tx=0).stop 2024-05-20T18:28:04.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.677+0000 7fb01a851640 1 -- 172.21.15.5:0/2692846116 shutdown_connections 2024-05-20T18:28:04.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.677+0000 7fb01a851640 1 --2- 172.21.15.5:0/2692846116 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb00006be60 0x7fb00006e370 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:04.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.677+0000 7fb01a851640 1 --2- 172.21.15.5:0/2692846116 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0141f8710 0x7fb0140cbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:04.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.677+0000 7fb01a851640 1 --2- 172.21.15.5:0/2692846116 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb014157990 0x7fb0141f81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:04.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.677+0000 7fb01a851640 1 --2- 172.21.15.5:0/2692846116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faffc0039b0 0x7fb0140b47f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:04.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.677+0000 7fb01a851640 1 -- 172.21.15.5:0/2692846116 >> 172.21.15.5:0/2692846116 conn(0x7fb0140d4bd0 msgr2=0x7fb01415e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:04.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.677+0000 7fb01a851640 1 -- 172.21.15.5:0/2692846116 shutdown_connections 2024-05-20T18:28:04.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.677+0000 7fb01a851640 1 -- 172.21.15.5:0/2692846116 wait complete. 2024-05-20T18:28:04.689 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:27:05.476596+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:27:06.137762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T18:27:09.829920+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T18:28:04.689 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:28:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.808+0000 7f7ef63d5640 1 Processor -- start 2024-05-20T18:28:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.808+0000 7f7ef63d5640 1 -- start start 2024-05-20T18:28:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.810+0000 7f7ef63d5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ef00cfae0 0x7f7ef0162e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.810+0000 7f7ef63d5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7ef006d6b0 0x7f7ef015f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.810+0000 7f7ef63d5640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ef006c8c0 con 0x7f7ef00cd710 2024-05-20T18:28:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.810+0000 7f7ef63d5640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f7ef00d0ba0 con 0x7f7ef006d6b0 2024-05-20T18:28:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.810+0000 7f7ef63d5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7ef006dac0 con 0x7f7ef00cfae0 2024-05-20T18:28:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.810+0000 7f7ef4972640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7ef006d6b0 0x7f7ef015f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.810+0000 7f7eeffff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7ef00cd710 0x7f7ef00cf3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35614/0 (socket says 172.21.15.5:35614) 2024-05-20T18:28:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.810+0000 7f7eeffff640 1 -- 172.21.15.5:0/171521032 learned_addr learned my addr 172.21.15.5:0/171521032 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.810+0000 7f7eef7fe640 1 --2- 172.21.15.5:0/171521032 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ef00cfae0 0x7f7ef0162e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.810+0000 7f7ef4972640 1 -- 172.21.15.5:0/171521032 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ef00cfae0 msgr2=0x7f7ef0162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.810+0000 7f7ef4972640 1 --2- 172.21.15.5:0/171521032 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ef00cfae0 0x7f7ef0162e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.810+0000 7f7ef4972640 1 -- 172.21.15.5:0/171521032 >> v1:172.21.15.5:6789/0 conn(0x7f7ef00cd710 legacy=0x7f7ef00cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:04.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.810+0000 7f7ef4972640 1 -- 172.21.15.5:0/171521032 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef01633e0 con 0x7f7ef006d6b0 2024-05-20T18:28:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.811+0000 7f7ef4972640 1 --2- 172.21.15.5:0/171521032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ef006d6b0 0x7f7ef015f2b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8009d90 tx=0x7f7ee802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67f528e28103534 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.811+0000 7f7eeeffd640 1 -- 172.21.15.5:0/171521032 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8007740 con 0x7f7ef006d6b0 2024-05-20T18:28:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.811+0000 7f7eeeffd640 1 -- 172.21.15.5:0/171521032 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee80078e0 con 0x7f7ef006d6b0 2024-05-20T18:28:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.811+0000 7f7eeeffd640 1 -- 172.21.15.5:0/171521032 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8007cb0 con 0x7f7ef006d6b0 2024-05-20T18:28:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.812+0000 7f7ef63d5640 1 -- 172.21.15.5:0/171521032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ef006d6b0 msgr2=0x7f7ef015f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:04.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.812+0000 7f7ef63d5640 1 --2- 172.21.15.5:0/171521032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ef006d6b0 0x7f7ef015f2b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8009d90 tx=0x7f7ee802a660 comp rx=0 tx=0).stop 2024-05-20T18:28:04.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.812+0000 7f7ef63d5640 1 -- 172.21.15.5:0/171521032 shutdown_connections 2024-05-20T18:28:04.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.812+0000 7f7ef63d5640 1 --2- 172.21.15.5:0/171521032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ef006d6b0 0x7f7ef015f2b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:04.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.812+0000 7f7ef63d5640 1 --2- 172.21.15.5:0/171521032 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ef00cfae0 0x7f7ef0162e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:04.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.812+0000 7f7ef63d5640 1 -- 172.21.15.5:0/171521032 >> 172.21.15.5:0/171521032 conn(0x7f7ef00b44d0 msgr2=0x7f7ef00b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:04.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.812+0000 7f7ef63d5640 1 -- 172.21.15.5:0/171521032 shutdown_connections 2024-05-20T18:28:04.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.812+0000 7f7ef63d5640 1 -- 172.21.15.5:0/171521032 wait complete. 2024-05-20T18:28:04.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.814+0000 7f7ef63d5640 1 Processor -- start 2024-05-20T18:28:04.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.814+0000 7f7ef63d5640 1 -- start start 2024-05-20T18:28:04.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.815+0000 7f7ef63d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ef006d6b0 0x7f7ef00ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:04.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.815+0000 7f7ef63d5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ef00cd710 0x7f7ef0158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:04.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.815+0000 7f7ef63d5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ef00b34e0 0x7f7ef00b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:04.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.815+0000 7f7ef63d5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef0163c80 con 0x7f7ef006d6b0 2024-05-20T18:28:04.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.815+0000 7f7ef63d5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef01639a0 con 0x7f7ef00cd710 2024-05-20T18:28:04.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.815+0000 7f7ef63d5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7ef00d1030 con 0x7f7ef00b34e0 2024-05-20T18:28:04.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.815+0000 7f7eeffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ef006d6b0 0x7f7ef00ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:04.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.816+0000 7f7eeffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ef006d6b0 0x7f7ef00ca4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45966/0 (socket says 172.21.15.5:45966) 2024-05-20T18:28:04.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.816+0000 7f7eeffff640 1 -- 172.21.15.5:0/244711500 learned_addr learned my addr 172.21.15.5:0/244711500 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:04.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.816+0000 7f7ef4972640 1 --2- 172.21.15.5:0/244711500 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ef00b34e0 0x7f7ef00b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.816+0000 7f7eef7fe640 1 --2- 172.21.15.5:0/244711500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ef00cd710 0x7f7ef0158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.816+0000 7f7eeffff640 1 -- 172.21.15.5:0/244711500 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ef00b34e0 msgr2=0x7f7ef00b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.816+0000 7f7eeffff640 1 --2- 172.21.15.5:0/244711500 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ef00b34e0 0x7f7ef00b38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.816+0000 7f7eeffff640 1 -- 172.21.15.5:0/244711500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ef00cd710 msgr2=0x7f7ef0158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.816+0000 7f7eeffff640 1 --2- 172.21.15.5:0/244711500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ef00cd710 0x7f7ef0158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.816+0000 7f7eeffff640 1 -- 172.21.15.5:0/244711500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef00b3e30 con 0x7f7ef006d6b0 2024-05-20T18:28:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.816+0000 7f7eeffff640 1 --2- 172.21.15.5:0/244711500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ef006d6b0 0x7f7ef00ca4c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7ee000c470 tx=0x7f7ee000c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.816+0000 7f7eecff9640 1 -- 172.21.15.5:0/244711500 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0017070 con 0x7f7ef006d6b0 2024-05-20T18:28:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.816+0000 7f7eecff9640 1 -- 172.21.15.5:0/244711500 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee0005d00 con 0x7f7ef006d6b0 2024-05-20T18:28:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.816+0000 7f7eecff9640 1 -- 172.21.15.5:0/244711500 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0007310 con 0x7f7ef006d6b0 2024-05-20T18:28:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.816+0000 7f7ef63d5640 1 -- 172.21.15.5:0/244711500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef015f620 con 0x7f7ef006d6b0 2024-05-20T18:28:04.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.816+0000 7f7ef63d5640 1 -- 172.21.15.5:0/244711500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef015fbe0 con 0x7f7ef006d6b0 2024-05-20T18:28:04.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.817+0000 7f7ef63d5640 1 -- 172.21.15.5:0/244711500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ef0075b00 con 0x7f7ef006d6b0 2024-05-20T18:28:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.818+0000 7f7eecff9640 1 -- 172.21.15.5:0/244711500 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7ee001a070 con 0x7f7ef006d6b0 2024-05-20T18:28:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.818+0000 7f7eecff9640 1 --2- 172.21.15.5:0/244711500 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7edc06be30 0x7f7edc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.819+0000 7f7eecff9640 1 -- 172.21.15.5:0/244711500 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7f7ee008a4d0 con 0x7f7ef006d6b0 2024-05-20T18:28:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.819+0000 7f7eef7fe640 1 --2- 172.21.15.5:0/244711500 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7edc06be30 0x7f7edc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.819+0000 7f7eef7fe640 1 --2- 172.21.15.5:0/244711500 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7edc06be30 0x7f7edc06e340 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7ef015ab10 tx=0x7f7ee4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:04.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:04.823+0000 7f7eecff9640 1 -- 172.21.15.5:0/244711500 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7ef00b3e30 con 0x7f7ef006d6b0 2024-05-20T18:28:05.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.008+0000 7f7ef63d5640 1 -- 172.21.15.5:0/244711500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7ef00c8e00 con 0x7f7ef006d6b0 2024-05-20T18:28:05.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.008+0000 7f7eecff9640 1 -- 172.21.15.5:0/244711500 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f7ee0058420 con 0x7f7ef006d6b0 2024-05-20T18:28:05.008 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:28:05.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.011+0000 7f7ef63d5640 1 -- 172.21.15.5:0/244711500 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7edc06be30 msgr2=0x7f7edc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:05.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.011+0000 7f7ef63d5640 1 --2- 172.21.15.5:0/244711500 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7edc06be30 0x7f7edc06e340 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7ef015ab10 tx=0x7f7ee4009190 comp rx=0 tx=0).stop 2024-05-20T18:28:05.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.011+0000 7f7ef63d5640 1 -- 172.21.15.5:0/244711500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ef006d6b0 msgr2=0x7f7ef00ca4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:05.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.011+0000 7f7ef63d5640 1 --2- 172.21.15.5:0/244711500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ef006d6b0 0x7f7ef00ca4c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7ee000c470 tx=0x7f7ee000c930 comp rx=0 tx=0).stop 2024-05-20T18:28:05.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.011+0000 7f7ef63d5640 1 -- 172.21.15.5:0/244711500 shutdown_connections 2024-05-20T18:28:05.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.011+0000 7f7ef63d5640 1 --2- 172.21.15.5:0/244711500 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7edc06be30 0x7f7edc06e340 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:05.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.011+0000 7f7ef63d5640 1 --2- 172.21.15.5:0/244711500 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ef00b34e0 0x7f7ef00b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:05.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.011+0000 7f7ef63d5640 1 --2- 172.21.15.5:0/244711500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ef00cd710 0x7f7ef0158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:05.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.011+0000 7f7ef63d5640 1 --2- 172.21.15.5:0/244711500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ef006d6b0 0x7f7ef00ca4c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:05.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.011+0000 7f7ef63d5640 1 -- 172.21.15.5:0/244711500 >> 172.21.15.5:0/244711500 conn(0x7f7ef00b44d0 msgr2=0x7f7ef00d1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:05.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.011+0000 7f7ef63d5640 1 -- 172.21.15.5:0/244711500 shutdown_connections 2024-05-20T18:28:05.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.011+0000 7f7ef63d5640 1 -- 172.21.15.5:0/244711500 wait complete. 2024-05-20T18:28:05.022 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:28:05.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.141+0000 7f127554c640 1 Processor -- start 2024-05-20T18:28:05.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.141+0000 7f127554c640 1 -- start start 2024-05-20T18:28:05.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.143+0000 7f127554c640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1270162820 0x7f12700b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:05.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.143+0000 7f127554c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f12700b3e80 0x7f127006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:05.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.143+0000 7f127554c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f127006d910 0x7f12700d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:05.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.143+0000 7f127554c640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f127006dd20 con 0x7f12700b3e80 2024-05-20T18:28:05.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.143+0000 7f127554c640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f12700d0630 con 0x7f1270162820 2024-05-20T18:28:05.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.143+0000 7f127554c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f12700d39c0 con 0x7f127006d910 2024-05-20T18:28:05.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.143+0000 7f1267fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f12700b3e80 0x7f127006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:05.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.143+0000 7f1267fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f12700b3e80 0x7f127006d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45974/0 (socket says 172.21.15.5:45974) 2024-05-20T18:28:05.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.143+0000 7f1267fff640 1 -- 172.21.15.5:0/2263542192 learned_addr learned my addr 172.21.15.5:0/2263542192 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:05.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.143+0000 7f126effd640 1 --2- 172.21.15.5:0/2263542192 >> v2:172.21.15.63:3300/0 conn(0x7f1270162820 0x7f12700b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:05.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.143+0000 7f126f7fe640 1 --2- 172.21.15.5:0/2263542192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f127006d910 0x7f12700d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:05.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.143+0000 7f1267fff640 1 -- 172.21.15.5:0/2263542192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f127006d910 msgr2=0x7f12700d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:05.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.143+0000 7f1267fff640 1 --2- 172.21.15.5:0/2263542192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f127006d910 0x7f12700d27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:05.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.143+0000 7f1267fff640 1 -- 172.21.15.5:0/2263542192 >> v2:172.21.15.63:3300/0 conn(0x7f1270162820 msgr2=0x7f12700b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:05.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.143+0000 7f1267fff640 1 --2- 172.21.15.5:0/2263542192 >> v2:172.21.15.63:3300/0 conn(0x7f1270162820 0x7f12700b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:05.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.143+0000 7f1267fff640 1 -- 172.21.15.5:0/2263542192 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12700d3ca0 con 0x7f12700b3e80 2024-05-20T18:28:05.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.144+0000 7f1267fff640 1 --2- 172.21.15.5:0/2263542192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12700b3e80 0x7f127006d3d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f12700b7740 tx=0x7f126002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16730438d868bc28 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:05.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.144+0000 7f126e7fc640 1 -- 172.21.15.5:0/2263542192 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1260037070 con 0x7f12700b3e80 2024-05-20T18:28:05.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.144+0000 7f126e7fc640 1 -- 172.21.15.5:0/2263542192 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1260032440 con 0x7f12700b3e80 2024-05-20T18:28:05.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.144+0000 7f126e7fc640 1 -- 172.21.15.5:0/2263542192 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1260032810 con 0x7f12700b3e80 2024-05-20T18:28:05.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.145+0000 7f127554c640 1 -- 172.21.15.5:0/2263542192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12700b3e80 msgr2=0x7f127006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:05.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.145+0000 7f127554c640 1 --2- 172.21.15.5:0/2263542192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12700b3e80 0x7f127006d3d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f12700b7740 tx=0x7f126002a1e0 comp rx=0 tx=0).stop 2024-05-20T18:28:05.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.145+0000 7f127554c640 1 -- 172.21.15.5:0/2263542192 shutdown_connections 2024-05-20T18:28:05.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.145+0000 7f127554c640 1 --2- 172.21.15.5:0/2263542192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f127006d910 0x7f12700d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:05.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.145+0000 7f127554c640 1 --2- 172.21.15.5:0/2263542192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12700b3e80 0x7f127006d3d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:05.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.145+0000 7f127554c640 1 --2- 172.21.15.5:0/2263542192 >> v2:172.21.15.63:3300/0 conn(0x7f1270162820 0x7f12700b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:05.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.145+0000 7f127554c640 1 -- 172.21.15.5:0/2263542192 >> 172.21.15.5:0/2263542192 conn(0x7f12700d4bd0 msgr2=0x7f12700d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:05.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.145+0000 7f127554c640 1 -- 172.21.15.5:0/2263542192 shutdown_connections 2024-05-20T18:28:05.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.145+0000 7f127554c640 1 -- 172.21.15.5:0/2263542192 wait complete. 2024-05-20T18:28:05.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.147+0000 7f127554c640 1 Processor -- start 2024-05-20T18:28:05.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.147+0000 7f127554c640 1 -- start start 2024-05-20T18:28:05.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.148+0000 7f127554c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f127006d910 0x7f12701e7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:05.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.148+0000 7f127554c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12700b3e80 0x7f127014a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:05.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.148+0000 7f127554c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1270162820 0x7f127014cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:05.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.148+0000 7f127554c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12700d0300 con 0x7f127006d910 2024-05-20T18:28:05.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.148+0000 7f127554c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f12700d4260 con 0x7f1270162820 2024-05-20T18:28:05.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.148+0000 7f127554c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f12700d4780 con 0x7f12700b3e80 2024-05-20T18:28:05.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.148+0000 7f126effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f127006d910 0x7f12701e7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:05.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.148+0000 7f126effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f127006d910 0x7f12701e7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45976/0 (socket says 172.21.15.5:45976) 2024-05-20T18:28:05.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.148+0000 7f126effd640 1 -- 172.21.15.5:0/2579390943 learned_addr learned my addr 172.21.15.5:0/2579390943 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:05.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.149+0000 7f126effd640 1 -- 172.21.15.5:0/2579390943 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12700b3e80 msgr2=0x7f127014a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:05.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.149+0000 7f126effd640 1 --2- 172.21.15.5:0/2579390943 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12700b3e80 0x7f127014a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.149+0000 7f126f7fe640 1 --2- 172.21.15.5:0/2579390943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1270162820 0x7f127014cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.149+0000 7f126effd640 1 -- 172.21.15.5:0/2579390943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1270162820 msgr2=0x7f127014cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.149+0000 7f126effd640 1 --2- 172.21.15.5:0/2579390943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1270162820 0x7f127014cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.149+0000 7f126effd640 1 -- 172.21.15.5:0/2579390943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f127014ac20 con 0x7f127006d910 2024-05-20T18:28:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.149+0000 7f126effd640 1 --2- 172.21.15.5:0/2579390943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f127006d910 0x7f12701e7b50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f125800e870 tx=0x7f125800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.149+0000 7f12677fe640 1 -- 172.21.15.5:0/2579390943 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1258017070 con 0x7f127006d910 2024-05-20T18:28:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.149+0000 7f12677fe640 1 -- 172.21.15.5:0/2579390943 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f125800c940 con 0x7f127006d910 2024-05-20T18:28:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.149+0000 7f127554c640 1 -- 172.21.15.5:0/2579390943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f127014b240 con 0x7f127006d910 2024-05-20T18:28:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.149+0000 7f12677fe640 1 -- 172.21.15.5:0/2579390943 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125800cd30 con 0x7f127006d910 2024-05-20T18:28:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.149+0000 7f127554c640 1 -- 172.21.15.5:0/2579390943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f127014b800 con 0x7f127006d910 2024-05-20T18:28:05.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.150+0000 7f127554c640 1 -- 172.21.15.5:0/2579390943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1270075b00 con 0x7f127006d910 2024-05-20T18:28:05.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.151+0000 7f12677fe640 1 -- 172.21.15.5:0/2579390943 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f125801a070 con 0x7f127006d910 2024-05-20T18:28:05.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.151+0000 7f12677fe640 1 --2- 172.21.15.5:0/2579390943 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f125006be60 0x7f125006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:05.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.152+0000 7f12677fe640 1 -- 172.21.15.5:0/2579390943 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7f125808b1c0 con 0x7f127006d910 2024-05-20T18:28:05.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.152+0000 7f1267fff640 1 --2- 172.21.15.5:0/2579390943 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f125006be60 0x7f125006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:05.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.152+0000 7f1267fff640 1 --2- 172.21.15.5:0/2579390943 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f125006be60 0x7f125006e370 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1260034000 tx=0x7f126002c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:05.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.156+0000 7f12677fe640 1 -- 172.21.15.5:0/2579390943 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f12580596c0 con 0x7f127006d910 2024-05-20T18:28:05.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.340+0000 7f127554c640 1 -- 172.21.15.5:0/2579390943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f12700cb6c0 con 0x7f127006d910 2024-05-20T18:28:05.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.340+0000 7f12677fe640 1 -- 172.21.15.5:0/2579390943 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f1258058fc0 con 0x7f127006d910 2024-05-20T18:28:05.340 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:28:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.342+0000 7f127554c640 1 -- 172.21.15.5:0/2579390943 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f125006be60 msgr2=0x7f125006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.342+0000 7f127554c640 1 --2- 172.21.15.5:0/2579390943 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f125006be60 0x7f125006e370 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1260034000 tx=0x7f126002c000 comp rx=0 tx=0).stop 2024-05-20T18:28:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.342+0000 7f127554c640 1 -- 172.21.15.5:0/2579390943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f127006d910 msgr2=0x7f12701e7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.342+0000 7f127554c640 1 --2- 172.21.15.5:0/2579390943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f127006d910 0x7f12701e7b50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f125800e870 tx=0x7f125800ed30 comp rx=0 tx=0).stop 2024-05-20T18:28:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.342+0000 7f127554c640 1 -- 172.21.15.5:0/2579390943 shutdown_connections 2024-05-20T18:28:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.342+0000 7f127554c640 1 --2- 172.21.15.5:0/2579390943 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f125006be60 0x7f125006e370 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:05.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.342+0000 7f127554c640 1 --2- 172.21.15.5:0/2579390943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1270162820 0x7f127014cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:05.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.342+0000 7f127554c640 1 --2- 172.21.15.5:0/2579390943 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12700b3e80 0x7f127014a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:05.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.342+0000 7f127554c640 1 --2- 172.21.15.5:0/2579390943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f127006d910 0x7f12701e7b50 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:05.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.342+0000 7f127554c640 1 -- 172.21.15.5:0/2579390943 >> 172.21.15.5:0/2579390943 conn(0x7f12700d4bd0 msgr2=0x7f12700b8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:05.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.342+0000 7f127554c640 1 -- 172.21.15.5:0/2579390943 shutdown_connections 2024-05-20T18:28:05.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.343+0000 7f127554c640 1 -- 172.21.15.5:0/2579390943 wait complete. 2024-05-20T18:28:05.353 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:28:05.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.471+0000 7f18a623d640 1 Processor -- start 2024-05-20T18:28:05.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.471+0000 7f18a623d640 1 -- start start 2024-05-20T18:28:05.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.473+0000 7f18a623d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f18a00d4400 0x7f18a015f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:05.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.473+0000 7f18a623d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f18a006cf00 0x7f18a006d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:05.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.473+0000 7f18a623d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18a008d2a0 con 0x7f18a00d2780 2024-05-20T18:28:05.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.473+0000 7f18a623d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f18a00d3f70 con 0x7f18a00d4400 2024-05-20T18:28:05.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.473+0000 7f18a623d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f18a006d850 con 0x7f18a006cf00 2024-05-20T18:28:05.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.473+0000 7f189ffff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f18a00d2780 0x7f18a00d2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35624/0 (socket says 172.21.15.5:35624) 2024-05-20T18:28:05.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.473+0000 7f189ffff640 1 -- 172.21.15.5:0/3899186166 learned_addr learned my addr 172.21.15.5:0/3899186166 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:05.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.473+0000 7f189f7fe640 1 --2- 172.21.15.5:0/3899186166 >> v2:172.21.15.63:3300/0 conn(0x7f18a00d4400 0x7f18a015f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:05.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.473+0000 7f189effd640 1 --2- 172.21.15.5:0/3899186166 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f18a006cf00 0x7f18a006d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:05.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.473+0000 7f189e7fc640 1 -- 172.21.15.5:0/3899186166 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2872912907 0 0) 0x7f1898001490 con 0x7f18a00d2780 2024-05-20T18:28:05.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.473+0000 7f189e7fc640 1 -- 172.21.15.5:0/3899186166 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1888003520 con 0x7f18a00d2780 2024-05-20T18:28:05.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.473+0000 7f189effd640 1 -- 172.21.15.5:0/3899186166 >> v2:172.21.15.63:3300/0 conn(0x7f18a00d4400 msgr2=0x7f18a015f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:05.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.473+0000 7f189effd640 1 --2- 172.21.15.5:0/3899186166 >> v2:172.21.15.63:3300/0 conn(0x7f18a00d4400 0x7f18a015f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:05.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.473+0000 7f189effd640 1 -- 172.21.15.5:0/3899186166 >> v1:172.21.15.5:6789/0 conn(0x7f18a00d2780 legacy=0x7f18a00d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:05.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.473+0000 7f189effd640 1 -- 172.21.15.5:0/3899186166 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a00d4810 con 0x7f18a006cf00 2024-05-20T18:28:05.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.473+0000 7f189f7fe640 1 --2- 172.21.15.5:0/3899186166 >> v2:172.21.15.63:3300/0 conn(0x7f18a00d4400 0x7f18a015f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:28:05.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.474+0000 7f189effd640 1 --2- 172.21.15.5:0/3899186166 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f18a006cf00 0x7f18a006d310 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f18940097d0 tx=0x7f189402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=709441246832c12 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:05.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.474+0000 7f189e7fc640 1 -- 172.21.15.5:0/3899186166 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894037070 con 0x7f18a006cf00 2024-05-20T18:28:05.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.475+0000 7f189e7fc640 1 -- 172.21.15.5:0/3899186166 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1894032440 con 0x7f18a006cf00 2024-05-20T18:28:05.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.475+0000 7f189e7fc640 1 -- 172.21.15.5:0/3899186166 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894032810 con 0x7f18a006cf00 2024-05-20T18:28:05.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.475+0000 7f18a623d640 1 -- 172.21.15.5:0/3899186166 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f18a006cf00 msgr2=0x7f18a006d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:05.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.475+0000 7f18a623d640 1 --2- 172.21.15.5:0/3899186166 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f18a006cf00 0x7f18a006d310 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f18940097d0 tx=0x7f189402a250 comp rx=0 tx=0).stop 2024-05-20T18:28:05.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.475+0000 7f18a623d640 1 -- 172.21.15.5:0/3899186166 shutdown_connections 2024-05-20T18:28:05.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.475+0000 7f18a623d640 1 --2- 172.21.15.5:0/3899186166 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f18a006cf00 0x7f18a006d310 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:05.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.475+0000 7f18a623d640 1 --2- 172.21.15.5:0/3899186166 >> v2:172.21.15.63:3300/0 conn(0x7f18a00d4400 0x7f18a015f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:05.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.475+0000 7f18a623d640 1 -- 172.21.15.5:0/3899186166 >> 172.21.15.5:0/3899186166 conn(0x7f18a00d4bb0 msgr2=0x7f18a00d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:05.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.475+0000 7f18a623d640 1 -- 172.21.15.5:0/3899186166 shutdown_connections 2024-05-20T18:28:05.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.475+0000 7f18a623d640 1 -- 172.21.15.5:0/3899186166 wait complete. 2024-05-20T18:28:05.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.477+0000 7f18a623d640 1 Processor -- start 2024-05-20T18:28:05.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.477+0000 7f18a623d640 1 -- start start 2024-05-20T18:28:05.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.479+0000 7f18a623d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18a006cf00 0x7f18a015e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:05.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.479+0000 7f18a623d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18a00d2780 0x7f18a015cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:05.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.479+0000 7f18a623d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f18a00d4400 0x7f18a00b9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:05.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.479+0000 7f18a623d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f18a00d3920 con 0x7f18a006cf00 2024-05-20T18:28:05.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.479+0000 7f18a623d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f18a00d3640 con 0x7f18a00d2780 2024-05-20T18:28:05.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.479+0000 7f18a623d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f18a00d3c00 con 0x7f18a00d4400 2024-05-20T18:28:05.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.479+0000 7f189f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18a006cf00 0x7f18a015e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:05.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.479+0000 7f189f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18a006cf00 0x7f18a015e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45984/0 (socket says 172.21.15.5:45984) 2024-05-20T18:28:05.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.479+0000 7f189f7fe640 1 -- 172.21.15.5:0/4133404551 learned_addr learned my addr 172.21.15.5:0/4133404551 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:05.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.479+0000 7f189effd640 1 --2- 172.21.15.5:0/4133404551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18a00d2780 0x7f18a015cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:05.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.479+0000 7f189ffff640 1 --2- 172.21.15.5:0/4133404551 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f18a00d4400 0x7f18a00b9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:05.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.479+0000 7f189f7fe640 1 -- 172.21.15.5:0/4133404551 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f18a00d4400 msgr2=0x7f18a00b9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:05.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.479+0000 7f189f7fe640 1 --2- 172.21.15.5:0/4133404551 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f18a00d4400 0x7f18a00b9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:05.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.479+0000 7f189f7fe640 1 -- 172.21.15.5:0/4133404551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18a00d2780 msgr2=0x7f18a015cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:05.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.479+0000 7f189f7fe640 1 --2- 172.21.15.5:0/4133404551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18a00d2780 0x7f18a015cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:05.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.479+0000 7f189f7fe640 1 -- 172.21.15.5:0/4133404551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a0155db0 con 0x7f18a006cf00 2024-05-20T18:28:05.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.479+0000 7f189f7fe640 1 --2- 172.21.15.5:0/4133404551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18a006cf00 0x7f18a015e7b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f189000e960 tx=0x7f189000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:05.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.479+0000 7f1887fff640 1 -- 172.21.15.5:0/4133404551 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890017070 con 0x7f18a006cf00 2024-05-20T18:28:05.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.479+0000 7f1887fff640 1 -- 172.21.15.5:0/4133404551 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f189000ca00 con 0x7f18a006cf00 2024-05-20T18:28:05.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.480+0000 7f1887fff640 1 -- 172.21.15.5:0/4133404551 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189000cdd0 con 0x7f18a006cf00 2024-05-20T18:28:05.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.480+0000 7f18a623d640 1 -- 172.21.15.5:0/4133404551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18a0156090 con 0x7f18a006cf00 2024-05-20T18:28:05.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.480+0000 7f18a623d640 1 -- 172.21.15.5:0/4133404551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18a0156650 con 0x7f18a006cf00 2024-05-20T18:28:05.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.481+0000 7f18a623d640 1 -- 172.21.15.5:0/4133404551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18a00735a0 con 0x7f18a006cf00 2024-05-20T18:28:05.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.481+0000 7f1887fff640 1 -- 172.21.15.5:0/4133404551 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f189001a070 con 0x7f18a006cf00 2024-05-20T18:28:05.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.481+0000 7f1887fff640 1 --2- 172.21.15.5:0/4133404551 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f188c06be30 0x7f188c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:05.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.482+0000 7f1887fff640 1 -- 172.21.15.5:0/4133404551 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7f189008b070 con 0x7f18a006cf00 2024-05-20T18:28:05.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.482+0000 7f189effd640 1 --2- 172.21.15.5:0/4133404551 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f188c06be30 0x7f188c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:05.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.482+0000 7f189effd640 1 --2- 172.21.15.5:0/4133404551 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f188c06be30 0x7f188c06e340 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1894034000 tx=0x7f189402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:05.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.486+0000 7f1887fff640 1 -- 172.21.15.5:0/4133404551 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f18a00735a0 con 0x7f18a006cf00 2024-05-20T18:28:05.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.668+0000 7f18a623d640 1 -- 172.21.15.5:0/4133404551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f18a0075b00 con 0x7f18a006cf00 2024-05-20T18:28:05.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.668+0000 7f1887fff640 1 -- 172.21.15.5:0/4133404551 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7f1890058fc0 con 0x7f18a006cf00 2024-05-20T18:28:05.668 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:28:05.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.670+0000 7f18a623d640 1 -- 172.21.15.5:0/4133404551 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f188c06be30 msgr2=0x7f188c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:05.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.670+0000 7f18a623d640 1 --2- 172.21.15.5:0/4133404551 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f188c06be30 0x7f188c06e340 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1894034000 tx=0x7f189402e000 comp rx=0 tx=0).stop 2024-05-20T18:28:05.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.670+0000 7f18a623d640 1 -- 172.21.15.5:0/4133404551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18a006cf00 msgr2=0x7f18a015e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:05.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.670+0000 7f18a623d640 1 --2- 172.21.15.5:0/4133404551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18a006cf00 0x7f18a015e7b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f189000e960 tx=0x7f189000ee20 comp rx=0 tx=0).stop 2024-05-20T18:28:05.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.671+0000 7f18a623d640 1 -- 172.21.15.5:0/4133404551 shutdown_connections 2024-05-20T18:28:05.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.671+0000 7f18a623d640 1 --2- 172.21.15.5:0/4133404551 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f188c06be30 0x7f188c06e340 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:05.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.671+0000 7f18a623d640 1 --2- 172.21.15.5:0/4133404551 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f18a00d4400 0x7f18a00b9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:05.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.671+0000 7f18a623d640 1 --2- 172.21.15.5:0/4133404551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18a00d2780 0x7f18a015cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:05.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.671+0000 7f18a623d640 1 --2- 172.21.15.5:0/4133404551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18a006cf00 0x7f18a015e7b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:05.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.671+0000 7f18a623d640 1 -- 172.21.15.5:0/4133404551 >> 172.21.15.5:0/4133404551 conn(0x7f18a00d4bb0 msgr2=0x7f18a0161e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:05.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.671+0000 7f18a623d640 1 -- 172.21.15.5:0/4133404551 shutdown_connections 2024-05-20T18:28:05.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.671+0000 7f18a623d640 1 -- 172.21.15.5:0/4133404551 wait complete. 2024-05-20T18:28:05.681 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T18:28:05.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.800+0000 7fd428965640 1 Processor -- start 2024-05-20T18:28:05.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.800+0000 7fd428965640 1 -- start start 2024-05-20T18:28:05.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.801+0000 7fd428965640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd4240afad0 0x7fd4240b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:05.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.801+0000 7fd428965640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd42415d160 0x7fd4240b5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:05.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.801+0000 7fd428965640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4240b3aa0 con 0x7fd4240afee0 2024-05-20T18:28:05.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.801+0000 7fd428965640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd4240b3850 con 0x7fd4240afad0 2024-05-20T18:28:05.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.801+0000 7fd428965640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd4240703e0 con 0x7fd42415d160 2024-05-20T18:28:05.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.801+0000 7fd419d9b640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd4240afee0 0x7fd42415ca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35632/0 (socket says 172.21.15.5:35632) 2024-05-20T18:28:05.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.801+0000 7fd419d9b640 1 -- 172.21.15.5:0/1780189427 learned_addr learned my addr 172.21.15.5:0/1780189427 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:05.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.802+0000 7fd42259c640 1 --2- 172.21.15.5:0/1780189427 >> v2:172.21.15.63:3300/0 conn(0x7fd4240afad0 0x7fd4240b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:05.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.802+0000 7fd422d9d640 1 --2- 172.21.15.5:0/1780189427 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd42415d160 0x7fd4240b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:05.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.802+0000 7fd421d9b640 1 -- 172.21.15.5:0/1780189427 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1326555414 0 0) 0x7fd4240b3aa0 con 0x7fd4240afee0 2024-05-20T18:28:05.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.802+0000 7fd421d9b640 1 -- 172.21.15.5:0/1780189427 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd410003520 con 0x7fd4240afee0 2024-05-20T18:28:05.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.802+0000 7fd42259c640 1 -- 172.21.15.5:0/1780189427 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd42415d160 msgr2=0x7fd4240b5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:05.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.802+0000 7fd42259c640 1 --2- 172.21.15.5:0/1780189427 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd42415d160 0x7fd4240b5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:05.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.802+0000 7fd42259c640 1 -- 172.21.15.5:0/1780189427 >> v1:172.21.15.5:6789/0 conn(0x7fd4240afee0 legacy=0x7fd42415ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:05.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.802+0000 7fd42259c640 1 -- 172.21.15.5:0/1780189427 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4240b3d80 con 0x7fd4240afad0 2024-05-20T18:28:05.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.802+0000 7fd422d9d640 1 --2- 172.21.15.5:0/1780189427 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd42415d160 0x7fd4240b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:28:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.803+0000 7fd42259c640 1 --2- 172.21.15.5:0/1780189427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4240afad0 0x7fd4240b51e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd4140097a0 tx=0x7fd41402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89d703fcdf33f711 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:05.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.803+0000 7fd421d9b640 1 -- 172.21.15.5:0/1780189427 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd414037070 con 0x7fd4240afad0 2024-05-20T18:28:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.803+0000 7fd421d9b640 1 -- 172.21.15.5:0/1780189427 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd41402ae50 con 0x7fd4240afad0 2024-05-20T18:28:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.803+0000 7fd421d9b640 1 -- 172.21.15.5:0/1780189427 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd414032660 con 0x7fd4240afad0 2024-05-20T18:28:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.804+0000 7fd428965640 1 -- 172.21.15.5:0/1780189427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4240afad0 msgr2=0x7fd4240b51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.804+0000 7fd428965640 1 --2- 172.21.15.5:0/1780189427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4240afad0 0x7fd4240b51e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd4140097a0 tx=0x7fd41402a100 comp rx=0 tx=0).stop 2024-05-20T18:28:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.804+0000 7fd428965640 1 -- 172.21.15.5:0/1780189427 shutdown_connections 2024-05-20T18:28:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.804+0000 7fd428965640 1 --2- 172.21.15.5:0/1780189427 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd42415d160 0x7fd4240b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.804+0000 7fd428965640 1 --2- 172.21.15.5:0/1780189427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4240afad0 0x7fd4240b51e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.804+0000 7fd428965640 1 -- 172.21.15.5:0/1780189427 >> 172.21.15.5:0/1780189427 conn(0x7fd4240cd2a0 msgr2=0x7fd4240cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.804+0000 7fd428965640 1 -- 172.21.15.5:0/1780189427 shutdown_connections 2024-05-20T18:28:05.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.804+0000 7fd428965640 1 -- 172.21.15.5:0/1780189427 wait complete. 2024-05-20T18:28:05.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.806+0000 7fd428965640 1 Processor -- start 2024-05-20T18:28:05.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.806+0000 7fd428965640 1 -- start start 2024-05-20T18:28:05.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.807+0000 7fd428965640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4240afad0 0x7fd42406c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:05.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.807+0000 7fd428965640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4240afee0 0x7fd4241516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:05.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.807+0000 7fd428965640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd42415d160 0x7fd42414def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:05.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.807+0000 7fd428965640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd4240b4860 con 0x7fd4240afee0 2024-05-20T18:28:05.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.807+0000 7fd428965640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd4240b4340 con 0x7fd42415d160 2024-05-20T18:28:05.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.807+0000 7fd428965640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd4241f57e0 con 0x7fd4240afad0 2024-05-20T18:28:05.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.807+0000 7fd419d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4240afee0 0x7fd4241516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:05.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.807+0000 7fd419d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4240afee0 0x7fd4241516f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46000/0 (socket says 172.21.15.5:46000) 2024-05-20T18:28:05.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.807+0000 7fd419d9b640 1 -- 172.21.15.5:0/781194049 learned_addr learned my addr 172.21.15.5:0/781194049 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:05.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.808+0000 7fd422d9d640 1 --2- 172.21.15.5:0/781194049 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd42415d160 0x7fd42414def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:05.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.808+0000 7fd42259c640 1 --2- 172.21.15.5:0/781194049 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4240afad0 0x7fd42406c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:05.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.808+0000 7fd419d9b640 1 -- 172.21.15.5:0/781194049 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4240afad0 msgr2=0x7fd42406c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:05.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.808+0000 7fd419d9b640 1 --2- 172.21.15.5:0/781194049 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4240afad0 0x7fd42406c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:05.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.808+0000 7fd419d9b640 1 -- 172.21.15.5:0/781194049 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd42415d160 msgr2=0x7fd42414def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:05.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.808+0000 7fd419d9b640 1 --2- 172.21.15.5:0/781194049 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd42415d160 0x7fd42414def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:05.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.808+0000 7fd419d9b640 1 -- 172.21.15.5:0/781194049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4241f4430 con 0x7fd4240afee0 2024-05-20T18:28:05.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.808+0000 7fd419d9b640 1 --2- 172.21.15.5:0/781194049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4240afee0 0x7fd4241516f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00c390 tx=0x7fd40c00c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:05.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.808+0000 7fd41b7fe640 1 -- 172.21.15.5:0/781194049 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c017070 con 0x7fd4240afee0 2024-05-20T18:28:05.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.808+0000 7fd41b7fe640 1 -- 172.21.15.5:0/781194049 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd40c005ce0 con 0x7fd4240afee0 2024-05-20T18:28:05.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.808+0000 7fd428965640 1 -- 172.21.15.5:0/781194049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4241f3670 con 0x7fd4240afee0 2024-05-20T18:28:05.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.808+0000 7fd41b7fe640 1 -- 172.21.15.5:0/781194049 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c0072f0 con 0x7fd4240afee0 2024-05-20T18:28:05.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.808+0000 7fd428965640 1 -- 172.21.15.5:0/781194049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4241f3c30 con 0x7fd4240afee0 2024-05-20T18:28:05.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.809+0000 7fd428965640 1 -- 172.21.15.5:0/781194049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd424075b00 con 0x7fd4240afee0 2024-05-20T18:28:05.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.810+0000 7fd41b7fe640 1 -- 172.21.15.5:0/781194049 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd40c01a070 con 0x7fd4240afee0 2024-05-20T18:28:05.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.811+0000 7fd41b7fe640 1 --2- 172.21.15.5:0/781194049 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd40406be30 0x7fd40406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:05.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.811+0000 7fd42259c640 1 --2- 172.21.15.5:0/781194049 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd40406be30 0x7fd40406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:05.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.811+0000 7fd41b7fe640 1 -- 172.21.15.5:0/781194049 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7fd40c08a5b0 con 0x7fd4240afee0 2024-05-20T18:28:05.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.811+0000 7fd42259c640 1 --2- 172.21.15.5:0/781194049 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd40406be30 0x7fd40406e340 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd414034000 tx=0x7fd41402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:05.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:05.815+0000 7fd41b7fe640 1 -- 172.21.15.5:0/781194049 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd40c058470 con 0x7fd4240afee0 2024-05-20T18:28:06.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.001+0000 7fd428965640 1 -- 172.21.15.5:0/781194049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd4240ce1f0 con 0x7fd4240afee0 2024-05-20T18:28:06.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.001+0000 7fd41b7fe640 1 -- 172.21.15.5:0/781194049 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7fd40c05c770 con 0x7fd4240afee0 2024-05-20T18:28:06.001 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:28:06.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.003+0000 7fd428965640 1 -- 172.21.15.5:0/781194049 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd40406be30 msgr2=0x7fd40406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:06.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.003+0000 7fd428965640 1 --2- 172.21.15.5:0/781194049 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd40406be30 0x7fd40406e340 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd414034000 tx=0x7fd41402e000 comp rx=0 tx=0).stop 2024-05-20T18:28:06.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.003+0000 7fd428965640 1 -- 172.21.15.5:0/781194049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4240afee0 msgr2=0x7fd4241516f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:06.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.003+0000 7fd428965640 1 --2- 172.21.15.5:0/781194049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4240afee0 0x7fd4241516f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00c390 tx=0x7fd40c00c790 comp rx=0 tx=0).stop 2024-05-20T18:28:06.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.003+0000 7fd428965640 1 -- 172.21.15.5:0/781194049 shutdown_connections 2024-05-20T18:28:06.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.003+0000 7fd428965640 1 --2- 172.21.15.5:0/781194049 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd40406be30 0x7fd40406e340 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:06.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.003+0000 7fd428965640 1 --2- 172.21.15.5:0/781194049 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd42415d160 0x7fd42414def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:06.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.003+0000 7fd428965640 1 --2- 172.21.15.5:0/781194049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4240afee0 0x7fd4241516f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:06.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.003+0000 7fd428965640 1 --2- 172.21.15.5:0/781194049 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4240afad0 0x7fd42406c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:06.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.003+0000 7fd428965640 1 -- 172.21.15.5:0/781194049 >> 172.21.15.5:0/781194049 conn(0x7fd4240cd2a0 msgr2=0x7fd4240cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:06.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.003+0000 7fd428965640 1 -- 172.21.15.5:0/781194049 shutdown_connections 2024-05-20T18:28:06.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.003+0000 7fd428965640 1 -- 172.21.15.5:0/781194049 wait complete. 2024-05-20T18:28:06.015 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T18:28:06.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.131+0000 7fb254bd5640 1 Processor -- start 2024-05-20T18:28:06.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.131+0000 7fb254bd5640 1 -- start start 2024-05-20T18:28:06.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.133+0000 7fb254bd5640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb250162820 0x7fb2500b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:06.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.133+0000 7fb254bd5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb2500b3e80 0x7fb25006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:06.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.133+0000 7fb254bd5640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb25006dd20 con 0x7fb25006d910 2024-05-20T18:28:06.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.133+0000 7fb254bd5640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fb2500d0630 con 0x7fb2500b3e80 2024-05-20T18:28:06.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.133+0000 7fb254bd5640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb2500d39c0 con 0x7fb250162820 2024-05-20T18:28:06.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.133+0000 7fb24dd9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb2500b3e80 0x7fb25006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:06.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.133+0000 7fb24ed9d640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb25006d910 0x7fb2500d27a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35644/0 (socket says 172.21.15.5:35644) 2024-05-20T18:28:06.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.133+0000 7fb24dd9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb2500b3e80 0x7fb25006d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46016/0 (socket says 172.21.15.5:46016) 2024-05-20T18:28:06.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.133+0000 7fb24ed9d640 1 -- 172.21.15.5:0/3073819891 learned_addr learned my addr 172.21.15.5:0/3073819891 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:06.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.133+0000 7fb24e59c640 1 --2- 172.21.15.5:0/3073819891 >> v2:172.21.15.63:3300/0 conn(0x7fb250162820 0x7fb2500b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:06.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.133+0000 7fb24dd9b640 1 -- 172.21.15.5:0/3073819891 >> v2:172.21.15.63:3300/0 conn(0x7fb250162820 msgr2=0x7fb2500b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:06.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.133+0000 7fb24dd9b640 1 --2- 172.21.15.5:0/3073819891 >> v2:172.21.15.63:3300/0 conn(0x7fb250162820 0x7fb2500b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:06.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.133+0000 7fb24dd9b640 1 -- 172.21.15.5:0/3073819891 >> v1:172.21.15.5:6789/0 conn(0x7fb25006d910 legacy=0x7fb2500d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:06.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.133+0000 7fb24dd9b640 1 -- 172.21.15.5:0/3073819891 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2500d3ca0 con 0x7fb2500b3e80 2024-05-20T18:28:06.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.133+0000 7fb24d59a640 1 -- 172.21.15.5:0/3073819891 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 826210512 0 0) 0x7fb240001110 con 0x7fb25006d910 2024-05-20T18:28:06.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.133+0000 7fb24e59c640 1 --2- 172.21.15.5:0/3073819891 >> v2:172.21.15.63:3300/0 conn(0x7fb250162820 0x7fb2500b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:06.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.134+0000 7fb24dd9b640 1 --2- 172.21.15.5:0/3073819891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2500b3e80 0x7fb25006d3d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb2500b7740 tx=0x7fb23802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ae47bd6b17197a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:06.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.134+0000 7fb24d59a640 1 -- 172.21.15.5:0/3073819891 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb238037070 con 0x7fb2500b3e80 2024-05-20T18:28:06.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.134+0000 7fb24d59a640 1 -- 172.21.15.5:0/3073819891 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb238032440 con 0x7fb2500b3e80 2024-05-20T18:28:06.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.134+0000 7fb24d59a640 1 -- 172.21.15.5:0/3073819891 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb238032810 con 0x7fb2500b3e80 2024-05-20T18:28:06.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.135+0000 7fb254bd5640 1 -- 172.21.15.5:0/3073819891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2500b3e80 msgr2=0x7fb25006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:06.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.135+0000 7fb254bd5640 1 --2- 172.21.15.5:0/3073819891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2500b3e80 0x7fb25006d3d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb2500b7740 tx=0x7fb23802a1a0 comp rx=0 tx=0).stop 2024-05-20T18:28:06.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.135+0000 7fb254bd5640 1 -- 172.21.15.5:0/3073819891 shutdown_connections 2024-05-20T18:28:06.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.135+0000 7fb254bd5640 1 --2- 172.21.15.5:0/3073819891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2500b3e80 0x7fb25006d3d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:06.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.135+0000 7fb254bd5640 1 --2- 172.21.15.5:0/3073819891 >> v2:172.21.15.63:3300/0 conn(0x7fb250162820 0x7fb2500b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:06.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.135+0000 7fb254bd5640 1 -- 172.21.15.5:0/3073819891 >> 172.21.15.5:0/3073819891 conn(0x7fb2500d4bd0 msgr2=0x7fb2500d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:06.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.135+0000 7fb254bd5640 1 -- 172.21.15.5:0/3073819891 shutdown_connections 2024-05-20T18:28:06.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.135+0000 7fb254bd5640 1 -- 172.21.15.5:0/3073819891 wait complete. 2024-05-20T18:28:06.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.137+0000 7fb254bd5640 1 Processor -- start 2024-05-20T18:28:06.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.137+0000 7fb254bd5640 1 -- start start 2024-05-20T18:28:06.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.138+0000 7fb254bd5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb25006d910 0x7fb25014ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:06.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.138+0000 7fb254bd5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2500b3e80 0x7fb2500b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:06.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.138+0000 7fb254bd5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb250162820 0x7fb25014d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:06.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.138+0000 7fb254bd5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb2500d0300 con 0x7fb250162820 2024-05-20T18:28:06.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.138+0000 7fb254bd5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb2500d4260 con 0x7fb25006d910 2024-05-20T18:28:06.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.138+0000 7fb254bd5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb2500b91a0 con 0x7fb2500b3e80 2024-05-20T18:28:06.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.138+0000 7fb24ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb250162820 0x7fb25014d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:06.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.139+0000 7fb24ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb250162820 0x7fb25014d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46026/0 (socket says 172.21.15.5:46026) 2024-05-20T18:28:06.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.139+0000 7fb24ed9d640 1 -- 172.21.15.5:0/1711326235 learned_addr learned my addr 172.21.15.5:0/1711326235 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:06.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.139+0000 7fb24e59c640 1 --2- 172.21.15.5:0/1711326235 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb25006d910 0x7fb25014ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:06.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.139+0000 7fb24dd9b640 1 --2- 172.21.15.5:0/1711326235 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2500b3e80 0x7fb2500b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:06.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.139+0000 7fb24ed9d640 1 -- 172.21.15.5:0/1711326235 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2500b3e80 msgr2=0x7fb2500b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:06.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.139+0000 7fb24ed9d640 1 --2- 172.21.15.5:0/1711326235 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2500b3e80 0x7fb2500b5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:06.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.139+0000 7fb24ed9d640 1 -- 172.21.15.5:0/1711326235 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb25006d910 msgr2=0x7fb25014ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:06.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.139+0000 7fb24ed9d640 1 --2- 172.21.15.5:0/1711326235 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb25006d910 0x7fb25014ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:06.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.139+0000 7fb24ed9d640 1 -- 172.21.15.5:0/1711326235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb25014b150 con 0x7fb250162820 2024-05-20T18:28:06.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.139+0000 7fb24ed9d640 1 --2- 172.21.15.5:0/1711326235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb250162820 0x7fb25014d4f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb24000c440 tx=0x7fb24000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:06.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.139+0000 7fb246ffd640 1 -- 172.21.15.5:0/1711326235 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2400074e0 con 0x7fb250162820 2024-05-20T18:28:06.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.139+0000 7fb246ffd640 1 -- 172.21.15.5:0/1711326235 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2400047e0 con 0x7fb250162820 2024-05-20T18:28:06.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.139+0000 7fb254bd5640 1 -- 172.21.15.5:0/1711326235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2500b69e0 con 0x7fb250162820 2024-05-20T18:28:06.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.139+0000 7fb246ffd640 1 -- 172.21.15.5:0/1711326235 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb240004bd0 con 0x7fb250162820 2024-05-20T18:28:06.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.139+0000 7fb254bd5640 1 -- 172.21.15.5:0/1711326235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb25014b8c0 con 0x7fb250162820 2024-05-20T18:28:06.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.140+0000 7fb254bd5640 1 -- 172.21.15.5:0/1711326235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb250075b00 con 0x7fb250162820 2024-05-20T18:28:06.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.141+0000 7fb246ffd640 1 -- 172.21.15.5:0/1711326235 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb240011070 con 0x7fb250162820 2024-05-20T18:28:06.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.141+0000 7fb246ffd640 1 --2- 172.21.15.5:0/1711326235 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb23006be60 0x7fb23006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:06.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.142+0000 7fb246ffd640 1 -- 172.21.15.5:0/1711326235 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7fb24008ae30 con 0x7fb250162820 2024-05-20T18:28:06.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.142+0000 7fb24e59c640 1 --2- 172.21.15.5:0/1711326235 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb23006be60 0x7fb23006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:06.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.142+0000 7fb24e59c640 1 --2- 172.21.15.5:0/1711326235 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb23006be60 0x7fb23006e370 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb24800f500 tx=0x7fb248007770 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:06.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.146+0000 7fb246ffd640 1 -- 172.21.15.5:0/1711326235 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb25006dd20 con 0x7fb250162820 2024-05-20T18:28:06.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.342+0000 7fb254bd5640 1 -- 172.21.15.5:0/1711326235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb25015d4b0 con 0x7fb250162820 2024-05-20T18:28:06.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.343+0000 7fb246ffd640 1 -- 172.21.15.5:0/1711326235 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7fb240058c50 con 0x7fb250162820 2024-05-20T18:28:06.342 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-05-20T18:28:06.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.344+0000 7fb254bd5640 1 -- 172.21.15.5:0/1711326235 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb23006be60 msgr2=0x7fb23006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:06.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.344+0000 7fb254bd5640 1 --2- 172.21.15.5:0/1711326235 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb23006be60 0x7fb23006e370 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb24800f500 tx=0x7fb248007770 comp rx=0 tx=0).stop 2024-05-20T18:28:06.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.344+0000 7fb254bd5640 1 -- 172.21.15.5:0/1711326235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb250162820 msgr2=0x7fb25014d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:06.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.344+0000 7fb254bd5640 1 --2- 172.21.15.5:0/1711326235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb250162820 0x7fb25014d4f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb24000c440 tx=0x7fb24000c900 comp rx=0 tx=0).stop 2024-05-20T18:28:06.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.345+0000 7fb254bd5640 1 -- 172.21.15.5:0/1711326235 shutdown_connections 2024-05-20T18:28:06.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.345+0000 7fb254bd5640 1 --2- 172.21.15.5:0/1711326235 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb23006be60 0x7fb23006e370 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:06.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.345+0000 7fb254bd5640 1 --2- 172.21.15.5:0/1711326235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb250162820 0x7fb25014d4f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:06.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.345+0000 7fb254bd5640 1 --2- 172.21.15.5:0/1711326235 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2500b3e80 0x7fb2500b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:06.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.345+0000 7fb254bd5640 1 --2- 172.21.15.5:0/1711326235 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb25006d910 0x7fb25014ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:06.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.345+0000 7fb254bd5640 1 -- 172.21.15.5:0/1711326235 >> 172.21.15.5:0/1711326235 conn(0x7fb2500d4bd0 msgr2=0x7fb250160eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:06.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.345+0000 7fb254bd5640 1 -- 172.21.15.5:0/1711326235 shutdown_connections 2024-05-20T18:28:06.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.345+0000 7fb254bd5640 1 -- 172.21.15.5:0/1711326235 wait complete. 2024-05-20T18:28:06.355 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-05-20T18:28:06.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:28:06.356 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-05-20T18:28:06.382 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:28:06.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:28:06.383 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-20T18:28:06.440 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:28:06.440 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:28:06.460 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:28:06.460 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:28:06.519 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:28:06.520 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:28:06.580 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:28:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.696+0000 7fdf922f0640 1 Processor -- start 2024-05-20T18:28:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.696+0000 7fdf922f0640 1 -- start start 2024-05-20T18:28:06.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.697+0000 7fdf922f0640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdf8c162820 0x7fdf8c0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:06.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.697+0000 7fdf922f0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf8c06dae0 0x7fdf8c0b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:06.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.697+0000 7fdf922f0640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf8c0d0630 con 0x7fdf8c0b3e80 2024-05-20T18:28:06.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.697+0000 7fdf922f0640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fdf8c0d39c0 con 0x7fdf8c162820 2024-05-20T18:28:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.697+0000 7fdf922f0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdf8c0d3ca0 con 0x7fdf8c06dae0 2024-05-20T18:28:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.697+0000 7fdf8bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdf8c162820 0x7fdf8c0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.697+0000 7fdf83fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fdf8c0b3e80 0x7fdf8c06d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35656/0 (socket says 172.21.15.5:35656) 2024-05-20T18:28:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.697+0000 7fdf83fff640 1 -- 172.21.15.5:0/3081525277 learned_addr learned my addr 172.21.15.5:0/3081525277 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.697+0000 7fdf8bfff640 1 --2- 172.21.15.5:0/3081525277 >> v2:172.21.15.5:3300/0 conn(0x7fdf8c162820 0x7fdf8c0b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46030/0 (socket says 172.21.15.5:46030) 2024-05-20T18:28:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.698+0000 7fdf8bfff640 1 -- 172.21.15.5:0/3081525277 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf8c06dae0 msgr2=0x7fdf8c0b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.698+0000 7fdf8bfff640 1 --2- 172.21.15.5:0/3081525277 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf8c06dae0 0x7fdf8c0b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.698+0000 7fdf8bfff640 1 -- 172.21.15.5:0/3081525277 >> v1:172.21.15.5:6789/0 conn(0x7fdf8c0b3e80 legacy=0x7fdf8c06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.698+0000 7fdf8bfff640 1 -- 172.21.15.5:0/3081525277 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf8c0d3f80 con 0x7fdf8c162820 2024-05-20T18:28:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.698+0000 7fdf8b7fe640 1 -- 172.21.15.5:0/3081525277 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4184083979 0 0) 0x7fdf8c0d0630 con 0x7fdf8c0b3e80 2024-05-20T18:28:06.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.698+0000 7fdf8bfff640 1 --2- 172.21.15.5:0/3081525277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf8c162820 0x7fdf8c0b3940 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0097a0 tx=0x7fdf7c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d2d810078674f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:06.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.698+0000 7fdf8b7fe640 1 -- 172.21.15.5:0/3081525277 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c037070 con 0x7fdf8c162820 2024-05-20T18:28:06.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.699+0000 7fdf8b7fe640 1 -- 172.21.15.5:0/3081525277 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf7c032440 con 0x7fdf8c162820 2024-05-20T18:28:06.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.699+0000 7fdf8b7fe640 1 -- 172.21.15.5:0/3081525277 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c032810 con 0x7fdf8c162820 2024-05-20T18:28:06.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.699+0000 7fdf922f0640 1 -- 172.21.15.5:0/3081525277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf8c162820 msgr2=0x7fdf8c0b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:06.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.699+0000 7fdf922f0640 1 --2- 172.21.15.5:0/3081525277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf8c162820 0x7fdf8c0b3940 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0097a0 tx=0x7fdf7c02a1a0 comp rx=0 tx=0).stop 2024-05-20T18:28:06.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.699+0000 7fdf922f0640 1 -- 172.21.15.5:0/3081525277 shutdown_connections 2024-05-20T18:28:06.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.699+0000 7fdf922f0640 1 --2- 172.21.15.5:0/3081525277 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf8c06dae0 0x7fdf8c0b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:06.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.699+0000 7fdf922f0640 1 --2- 172.21.15.5:0/3081525277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf8c162820 0x7fdf8c0b3940 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:06.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.699+0000 7fdf922f0640 1 -- 172.21.15.5:0/3081525277 >> 172.21.15.5:0/3081525277 conn(0x7fdf8c0d4bd0 msgr2=0x7fdf8c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:06.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.699+0000 7fdf922f0640 1 -- 172.21.15.5:0/3081525277 shutdown_connections 2024-05-20T18:28:06.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.699+0000 7fdf922f0640 1 -- 172.21.15.5:0/3081525277 wait complete. 2024-05-20T18:28:06.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.701+0000 7fdf922f0640 1 Processor -- start 2024-05-20T18:28:06.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.701+0000 7fdf922f0640 1 -- start start 2024-05-20T18:28:06.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.702+0000 7fdf922f0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf8c0b3e80 0x7fdf8c0cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:06.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.702+0000 7fdf922f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf8c162820 0x7fdf8c0b9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:06.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.702+0000 7fdf922f0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf8c1563a0 0x7fdf8c0b9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:06.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.702+0000 7fdf922f0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8c0d4780 con 0x7fdf8c162820 2024-05-20T18:28:06.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.702+0000 7fdf922f0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8c0d0300 con 0x7fdf8c0b3e80 2024-05-20T18:28:06.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.702+0000 7fdf922f0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdf8c0d4900 con 0x7fdf8c1563a0 2024-05-20T18:28:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.703+0000 7fdf83fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf8c162820 0x7fdf8c0b9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.703+0000 7fdf83fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf8c162820 0x7fdf8c0b9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46034/0 (socket says 172.21.15.5:46034) 2024-05-20T18:28:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.703+0000 7fdf83fff640 1 -- 172.21.15.5:0/1106785471 learned_addr learned my addr 172.21.15.5:0/1106785471 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.703+0000 7fdf9088d640 1 --2- 172.21.15.5:0/1106785471 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf8c1563a0 0x7fdf8c0b9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.703+0000 7fdf8bfff640 1 --2- 172.21.15.5:0/1106785471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf8c0b3e80 0x7fdf8c0cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.703+0000 7fdf83fff640 1 -- 172.21.15.5:0/1106785471 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf8c1563a0 msgr2=0x7fdf8c0b9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.703+0000 7fdf83fff640 1 --2- 172.21.15.5:0/1106785471 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf8c1563a0 0x7fdf8c0b9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.703+0000 7fdf83fff640 1 -- 172.21.15.5:0/1106785471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf8c0b3e80 msgr2=0x7fdf8c0cbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.703+0000 7fdf83fff640 1 --2- 172.21.15.5:0/1106785471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf8c0b3e80 0x7fdf8c0cbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.703+0000 7fdf83fff640 1 -- 172.21.15.5:0/1106785471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf8c0b86b0 con 0x7fdf8c162820 2024-05-20T18:28:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.703+0000 7fdf8bfff640 1 --2- 172.21.15.5:0/1106785471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf8c0b3e80 0x7fdf8c0cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.703+0000 7fdf83fff640 1 --2- 172.21.15.5:0/1106785471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf8c162820 0x7fdf8c0b9750 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdf7400c460 tx=0x7fdf7400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.703+0000 7fdf897fa640 1 -- 172.21.15.5:0/1106785471 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf74018070 con 0x7fdf8c162820 2024-05-20T18:28:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.703+0000 7fdf897fa640 1 -- 172.21.15.5:0/1106785471 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf74007500 con 0x7fdf8c162820 2024-05-20T18:28:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.703+0000 7fdf922f0640 1 -- 172.21.15.5:0/1106785471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf8c0b8990 con 0x7fdf8c162820 2024-05-20T18:28:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.703+0000 7fdf897fa640 1 -- 172.21.15.5:0/1106785471 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf74004a20 con 0x7fdf8c162820 2024-05-20T18:28:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.703+0000 7fdf922f0640 1 -- 172.21.15.5:0/1106785471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf8c0b8f50 con 0x7fdf8c162820 2024-05-20T18:28:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.704+0000 7fdf922f0640 1 -- 172.21.15.5:0/1106785471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf8c0730e0 con 0x7fdf8c162820 2024-05-20T18:28:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.705+0000 7fdf897fa640 1 -- 172.21.15.5:0/1106785471 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdf7401b070 con 0x7fdf8c162820 2024-05-20T18:28:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.706+0000 7fdf897fa640 1 --2- 172.21.15.5:0/1106785471 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdf6c06be60 0x7fdf6c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.706+0000 7fdf8bfff640 1 --2- 172.21.15.5:0/1106785471 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdf6c06be60 0x7fdf6c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.706+0000 7fdf897fa640 1 -- 172.21.15.5:0/1106785471 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7fdf7408a4c0 con 0x7fdf8c162820 2024-05-20T18:28:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.706+0000 7fdf8bfff640 1 --2- 172.21.15.5:0/1106785471 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdf6c06be60 0x7fdf6c06e370 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c034000 tx=0x7fdf7c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:06.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.710+0000 7fdf897fa640 1 -- 172.21.15.5:0/1106785471 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fdf8c0730e0 con 0x7fdf8c162820 2024-05-20T18:28:06.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.895+0000 7fdf922f0640 1 -- 172.21.15.5:0/1106785471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf8c0ca770 con 0x7fdf8c162820 2024-05-20T18:28:06.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.896+0000 7fdf897fa640 1 -- 172.21.15.5:0/1106785471 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18800 (secure 0 0 0) 0x7fdf74058380 con 0x7fdf8c162820 2024-05-20T18:28:06.895 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:06.896 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:27:56.213938+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:27:05.476596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T18:27:06.137762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T18:27:09.829920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:28:06.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.898+0000 7fdf922f0640 1 -- 172.21.15.5:0/1106785471 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdf6c06be60 msgr2=0x7fdf6c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:06.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.898+0000 7fdf922f0640 1 --2- 172.21.15.5:0/1106785471 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdf6c06be60 0x7fdf6c06e370 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c034000 tx=0x7fdf7c02e000 comp rx=0 tx=0).stop 2024-05-20T18:28:06.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.898+0000 7fdf922f0640 1 -- 172.21.15.5:0/1106785471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf8c162820 msgr2=0x7fdf8c0b9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:06.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.898+0000 7fdf922f0640 1 --2- 172.21.15.5:0/1106785471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf8c162820 0x7fdf8c0b9750 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdf7400c460 tx=0x7fdf7400c920 comp rx=0 tx=0).stop 2024-05-20T18:28:06.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.898+0000 7fdf922f0640 1 -- 172.21.15.5:0/1106785471 shutdown_connections 2024-05-20T18:28:06.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.898+0000 7fdf922f0640 1 --2- 172.21.15.5:0/1106785471 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdf6c06be60 0x7fdf6c06e370 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:06.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.898+0000 7fdf922f0640 1 --2- 172.21.15.5:0/1106785471 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf8c1563a0 0x7fdf8c0b9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:06.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.898+0000 7fdf922f0640 1 --2- 172.21.15.5:0/1106785471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf8c162820 0x7fdf8c0b9750 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:06.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.898+0000 7fdf922f0640 1 --2- 172.21.15.5:0/1106785471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf8c0b3e80 0x7fdf8c0cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:06.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.898+0000 7fdf922f0640 1 -- 172.21.15.5:0/1106785471 >> 172.21.15.5:0/1106785471 conn(0x7fdf8c0d4bd0 msgr2=0x7fdf8c15d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:06.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.898+0000 7fdf922f0640 1 -- 172.21.15.5:0/1106785471 shutdown_connections 2024-05-20T18:28:06.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:06.898+0000 7fdf922f0640 1 -- 172.21.15.5:0/1106785471 wait complete. 2024-05-20T18:28:06.909 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:27:05.476596+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:27:06.137762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T18:27:09.829920+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T18:28:06.910 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:28:07.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.028+0000 7f3d57907640 1 Processor -- start 2024-05-20T18:28:07.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.028+0000 7f3d57907640 1 -- start start 2024-05-20T18:28:07.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.030+0000 7f3d57907640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d500cf850 0x7f3d500b6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:07.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.030+0000 7f3d57907640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3d500d14f0 0x7f3d500b6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:07.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.030+0000 7f3d57907640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d500d1ac0 con 0x7f3d500d10e0 2024-05-20T18:28:07.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.030+0000 7f3d57907640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f3d500d1da0 con 0x7f3d500d14f0 2024-05-20T18:28:07.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.030+0000 7f3d57907640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3d500d2080 con 0x7f3d500cf850 2024-05-20T18:28:07.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.030+0000 7f3d55ea4640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3d500d14f0 0x7f3d500b6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:07.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.030+0000 7f3d55ea4640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3d500d14f0 0x7f3d500b6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46046/0 (socket says 172.21.15.5:46046) 2024-05-20T18:28:07.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.030+0000 7f3d55ea4640 1 -- 172.21.15.5:0/187542876 learned_addr learned my addr 172.21.15.5:0/187542876 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:07.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.030+0000 7f3d54ea2640 1 --2- 172.21.15.5:0/187542876 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d500cf850 0x7f3d500b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:07.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.030+0000 7f3d55ea4640 1 -- 172.21.15.5:0/187542876 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d500cf850 msgr2=0x7f3d500b6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:07.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.030+0000 7f3d55ea4640 1 --2- 172.21.15.5:0/187542876 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d500cf850 0x7f3d500b6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.030+0000 7f3d55ea4640 1 -- 172.21.15.5:0/187542876 >> v1:172.21.15.63:6789/0 conn(0x7f3d500d10e0 legacy=0x7f3d500cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:07.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.030+0000 7f3d55ea4640 1 -- 172.21.15.5:0/187542876 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d500d2360 con 0x7f3d500d14f0 2024-05-20T18:28:07.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.031+0000 7f3d55ea4640 1 --2- 172.21.15.5:0/187542876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d500d14f0 0x7f3d500b6740 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3d40009e50 tx=0x7f3d4002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=641fa732a3b6db0b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:07.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.031+0000 7f3d4ffff640 1 -- 172.21.15.5:0/187542876 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d40036070 con 0x7f3d500d14f0 2024-05-20T18:28:07.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.031+0000 7f3d4ffff640 1 -- 172.21.15.5:0/187542876 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d40031600 con 0x7f3d500d14f0 2024-05-20T18:28:07.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.031+0000 7f3d4ffff640 1 -- 172.21.15.5:0/187542876 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d400319d0 con 0x7f3d500d14f0 2024-05-20T18:28:07.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.032+0000 7f3d57907640 1 -- 172.21.15.5:0/187542876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d500d14f0 msgr2=0x7f3d500b6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:07.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.032+0000 7f3d57907640 1 --2- 172.21.15.5:0/187542876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d500d14f0 0x7f3d500b6740 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3d40009e50 tx=0x7f3d4002a710 comp rx=0 tx=0).stop 2024-05-20T18:28:07.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.032+0000 7f3d57907640 1 -- 172.21.15.5:0/187542876 shutdown_connections 2024-05-20T18:28:07.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.032+0000 7f3d57907640 1 --2- 172.21.15.5:0/187542876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d500d14f0 0x7f3d500b6740 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.032+0000 7f3d57907640 1 --2- 172.21.15.5:0/187542876 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d500cf850 0x7f3d500b6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.032+0000 7f3d57907640 1 -- 172.21.15.5:0/187542876 >> 172.21.15.5:0/187542876 conn(0x7f3d500b44d0 msgr2=0x7f3d500b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:07.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.032+0000 7f3d57907640 1 -- 172.21.15.5:0/187542876 shutdown_connections 2024-05-20T18:28:07.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.032+0000 7f3d57907640 1 -- 172.21.15.5:0/187542876 wait complete. 2024-05-20T18:28:07.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.034+0000 7f3d57907640 1 Processor -- start 2024-05-20T18:28:07.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.034+0000 7f3d57907640 1 -- start start 2024-05-20T18:28:07.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.035+0000 7f3d57907640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d500cf850 0x7f3d501f6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:07.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.035+0000 7f3d57907640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d500d10e0 0x7f3d501e62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:07.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.035+0000 7f3d57907640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d500d14f0 0x7f3d501e7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:07.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.035+0000 7f3d57907640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3d500b8440 con 0x7f3d500d14f0 2024-05-20T18:28:07.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.035+0000 7f3d57907640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3d500b82c0 con 0x7f3d500d10e0 2024-05-20T18:28:07.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.035+0000 7f3d57907640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3d500709f0 con 0x7f3d500cf850 2024-05-20T18:28:07.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.035+0000 7f3d55ea4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d500d14f0 0x7f3d501e7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:07.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.035+0000 7f3d55ea4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d500d14f0 0x7f3d501e7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46058/0 (socket says 172.21.15.5:46058) 2024-05-20T18:28:07.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.035+0000 7f3d55ea4640 1 -- 172.21.15.5:0/1576580532 learned_addr learned my addr 172.21.15.5:0/1576580532 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:07.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.036+0000 7f3d54ea2640 1 --2- 172.21.15.5:0/1576580532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d500d10e0 0x7f3d501e62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:07.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.036+0000 7f3d556a3640 1 --2- 172.21.15.5:0/1576580532 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d500cf850 0x7f3d501f6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:07.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.036+0000 7f3d55ea4640 1 -- 172.21.15.5:0/1576580532 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d500cf850 msgr2=0x7f3d501f6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:07.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.036+0000 7f3d55ea4640 1 --2- 172.21.15.5:0/1576580532 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d500cf850 0x7f3d501f6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.036+0000 7f3d55ea4640 1 -- 172.21.15.5:0/1576580532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d500d10e0 msgr2=0x7f3d501e62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:07.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.036+0000 7f3d55ea4640 1 --2- 172.21.15.5:0/1576580532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d500d10e0 0x7f3d501e62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.036+0000 7f3d55ea4640 1 -- 172.21.15.5:0/1576580532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d501e6d20 con 0x7f3d500d14f0 2024-05-20T18:28:07.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.036+0000 7f3d55ea4640 1 --2- 172.21.15.5:0/1576580532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d500d14f0 0x7f3d501e7550 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3d40034040 tx=0x7f3d40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:07.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.036+0000 7f3d4dffb640 1 -- 172.21.15.5:0/1576580532 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d40036050 con 0x7f3d500d14f0 2024-05-20T18:28:07.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.036+0000 7f3d4dffb640 1 -- 172.21.15.5:0/1576580532 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d40042070 con 0x7f3d500d14f0 2024-05-20T18:28:07.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.036+0000 7f3d4dffb640 1 -- 172.21.15.5:0/1576580532 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d40009510 con 0x7f3d500d14f0 2024-05-20T18:28:07.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.036+0000 7f3d57907640 1 -- 172.21.15.5:0/1576580532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d500c9b10 con 0x7f3d500d14f0 2024-05-20T18:28:07.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.036+0000 7f3d57907640 1 -- 172.21.15.5:0/1576580532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d500ca0d0 con 0x7f3d500d14f0 2024-05-20T18:28:07.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.037+0000 7f3d57907640 1 -- 172.21.15.5:0/1576580532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d501606c0 con 0x7f3d500d14f0 2024-05-20T18:28:07.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.038+0000 7f3d4dffb640 1 -- 172.21.15.5:0/1576580532 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3d40007bb0 con 0x7f3d500d14f0 2024-05-20T18:28:07.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.039+0000 7f3d4dffb640 1 --2- 172.21.15.5:0/1576580532 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3d3c06be60 0x7f3d3c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:07.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.039+0000 7f3d556a3640 1 --2- 172.21.15.5:0/1576580532 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3d3c06be60 0x7f3d3c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:07.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.039+0000 7f3d4dffb640 1 -- 172.21.15.5:0/1576580532 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7f3d400ac190 con 0x7f3d500d14f0 2024-05-20T18:28:07.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.039+0000 7f3d556a3640 1 --2- 172.21.15.5:0/1576580532 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3d3c06be60 0x7f3d3c06e370 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3d48007fd0 tx=0x7f3d48009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:07.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.043+0000 7f3d4dffb640 1 -- 172.21.15.5:0/1576580532 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3d501606c0 con 0x7f3d500d14f0 2024-05-20T18:28:07.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.227+0000 7f3d57907640 1 -- 172.21.15.5:0/1576580532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3d500b4e10 con 0x7f3d500d14f0 2024-05-20T18:28:07.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.227+0000 7f3d4dffb640 1 -- 172.21.15.5:0/1576580532 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f3d500b4e10 con 0x7f3d500d14f0 2024-05-20T18:28:07.227 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:28:07.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.229+0000 7f3d57907640 1 -- 172.21.15.5:0/1576580532 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3d3c06be60 msgr2=0x7f3d3c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:07.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.229+0000 7f3d57907640 1 --2- 172.21.15.5:0/1576580532 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3d3c06be60 0x7f3d3c06e370 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3d48007fd0 tx=0x7f3d48009040 comp rx=0 tx=0).stop 2024-05-20T18:28:07.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.229+0000 7f3d57907640 1 -- 172.21.15.5:0/1576580532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d500d14f0 msgr2=0x7f3d501e7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:07.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.229+0000 7f3d57907640 1 --2- 172.21.15.5:0/1576580532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d500d14f0 0x7f3d501e7550 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3d40034040 tx=0x7f3d40004a40 comp rx=0 tx=0).stop 2024-05-20T18:28:07.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.229+0000 7f3d57907640 1 -- 172.21.15.5:0/1576580532 shutdown_connections 2024-05-20T18:28:07.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.229+0000 7f3d57907640 1 --2- 172.21.15.5:0/1576580532 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3d3c06be60 0x7f3d3c06e370 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.229+0000 7f3d57907640 1 --2- 172.21.15.5:0/1576580532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d500d14f0 0x7f3d501e7550 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.229+0000 7f3d57907640 1 --2- 172.21.15.5:0/1576580532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d500d10e0 0x7f3d501e62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.229+0000 7f3d57907640 1 --2- 172.21.15.5:0/1576580532 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d500cf850 0x7f3d501f6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.229+0000 7f3d57907640 1 -- 172.21.15.5:0/1576580532 >> 172.21.15.5:0/1576580532 conn(0x7f3d500b44d0 msgr2=0x7f3d5015ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:07.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.229+0000 7f3d57907640 1 -- 172.21.15.5:0/1576580532 shutdown_connections 2024-05-20T18:28:07.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.229+0000 7f3d57907640 1 -- 172.21.15.5:0/1576580532 wait complete. 2024-05-20T18:28:07.240 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:28:07.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.360+0000 7f6244ce2640 1 Processor -- start 2024-05-20T18:28:07.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.360+0000 7f6244ce2640 1 -- start start 2024-05-20T18:28:07.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.362+0000 7f6244ce2640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f62400ba3c0 0x7f62400b8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:07.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.362+0000 7f6244ce2640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62401634a0 con 0x7f62400b6620 2024-05-20T18:28:07.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.362+0000 7f6244ce2640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6240163780 con 0x7f62400b85b0 2024-05-20T18:28:07.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.362+0000 7f6244ce2640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f62400d3480 con 0x7f62400ba3c0 2024-05-20T18:28:07.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.362+0000 7f623e59c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f62400b6620 0x7f6240162ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35670/0 (socket says 172.21.15.5:35670) 2024-05-20T18:28:07.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.362+0000 7f623e59c640 1 -- 172.21.15.5:0/1562158959 learned_addr learned my addr 172.21.15.5:0/1562158959 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:07.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.362+0000 7f623dd9b640 1 --2- 172.21.15.5:0/1562158959 >> v2:172.21.15.63:3300/0 conn(0x7f62400ba3c0 0x7f62400b8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:07.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.362+0000 7f623d59a640 1 -- 172.21.15.5:0/1562158959 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 504640206 0 0) 0x7f62401634a0 con 0x7f62400b6620 2024-05-20T18:28:07.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.362+0000 7f623d59a640 1 -- 172.21.15.5:0/1562158959 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6228003520 con 0x7f62400b6620 2024-05-20T18:28:07.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.362+0000 7f623d59a640 1 -- 172.21.15.5:0/1562158959 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1449679923 0 0) 0x7f6228003520 con 0x7f62400b6620 2024-05-20T18:28:07.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.362+0000 7f623d59a640 1 -- 172.21.15.5:0/1562158959 >> v2:172.21.15.63:3300/0 conn(0x7f62400ba3c0 msgr2=0x7f62400b8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:07.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.362+0000 7f623d59a640 1 --2- 172.21.15.5:0/1562158959 >> v2:172.21.15.63:3300/0 conn(0x7f62400ba3c0 0x7f62400b8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.362+0000 7f623d59a640 1 -- 172.21.15.5:0/1562158959 >> v1:172.21.15.63:6789/0 conn(0x7f62400b85b0 legacy=0x7f62400d3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:07.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.362+0000 7f623d59a640 1 -- 172.21.15.5:0/1562158959 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6240163a60 con 0x7f62400b6620 2024-05-20T18:28:07.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.362+0000 7f623d59a640 1 -- 172.21.15.5:0/1562158959 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f6230001670 con 0x7f62400b6620 2024-05-20T18:28:07.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.362+0000 7f623d59a640 1 -- 172.21.15.5:0/1562158959 >> v1:172.21.15.5:6789/0 conn(0x7f62400b6620 legacy=0x7f6240162ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:07.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.362+0000 7f623d59a640 1 --2- 172.21.15.5:0/1562158959 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62280039b0 0x7f6228003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:07.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.362+0000 7f623d59a640 1 --2- 172.21.15.5:0/1562158959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6228004800 0x7f6228004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.362+0000 7f623d59a640 1 --2- 172.21.15.5:0/1562158959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62280051c0 0x7f622800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.363+0000 7f623d59a640 1 -- 172.21.15.5:0/1562158959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f62400d3480 con 0x7f62280051c0 2024-05-20T18:28:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.363+0000 7f623d59a640 1 -- 172.21.15.5:0/1562158959 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f622800c160 con 0x7f6228004800 2024-05-20T18:28:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.363+0000 7f623d59a640 1 -- 172.21.15.5:0/1562158959 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f622800c360 con 0x7f62280039b0 2024-05-20T18:28:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.363+0000 7f623ed9d640 1 --2- 172.21.15.5:0/1562158959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62280051c0 0x7f622800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.363+0000 7f623dd9b640 1 --2- 172.21.15.5:0/1562158959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6228004800 0x7f6228004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.363+0000 7f623e59c640 1 --2- 172.21.15.5:0/1562158959 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62280039b0 0x7f6228003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.363+0000 7f623ed9d640 1 -- 172.21.15.5:0/1562158959 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62280039b0 msgr2=0x7f6228003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.363+0000 7f623ed9d640 1 --2- 172.21.15.5:0/1562158959 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62280039b0 0x7f6228003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.363+0000 7f623ed9d640 1 -- 172.21.15.5:0/1562158959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6228004800 msgr2=0x7f6228004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.363+0000 7f623ed9d640 1 --2- 172.21.15.5:0/1562158959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6228004800 0x7f6228004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.363+0000 7f623ed9d640 1 -- 172.21.15.5:0/1562158959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6228003520 con 0x7f62280051c0 2024-05-20T18:28:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.364+0000 7f623ed9d640 1 -- 172.21.15.5:0/1562158959 reap_dead start 2024-05-20T18:28:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.364+0000 7f623ed9d640 1 --2- 172.21.15.5:0/1562158959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62280051c0 0x7f622800b9c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f624008def0 tx=0x7f6228005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2268a18afce979c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:07.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.364+0000 7f623d59a640 1 -- 172.21.15.5:0/1562158959 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6238001a70 con 0x7f62280051c0 2024-05-20T18:28:07.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.364+0000 7f623d59a640 1 -- 172.21.15.5:0/1562158959 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f623802ad40 con 0x7f62280051c0 2024-05-20T18:28:07.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.364+0000 7f623d59a640 1 -- 172.21.15.5:0/1562158959 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6238003bb0 con 0x7f62280051c0 2024-05-20T18:28:07.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.364+0000 7f6244ce2640 1 -- 172.21.15.5:0/1562158959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62280051c0 msgr2=0x7f622800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:07.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.364+0000 7f6244ce2640 1 --2- 172.21.15.5:0/1562158959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62280051c0 0x7f622800b9c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f624008def0 tx=0x7f6228005830 comp rx=0 tx=0).stop 2024-05-20T18:28:07.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.365+0000 7f6244ce2640 1 -- 172.21.15.5:0/1562158959 shutdown_connections 2024-05-20T18:28:07.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.365+0000 7f6244ce2640 1 --2- 172.21.15.5:0/1562158959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62280051c0 0x7f622800b9c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.365+0000 7f6244ce2640 1 -- 172.21.15.5:0/1562158959 >> 172.21.15.5:0/1562158959 conn(0x7f624006d240 msgr2=0x7f624006d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:07.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.365+0000 7f6244ce2640 1 -- 172.21.15.5:0/1562158959 shutdown_connections 2024-05-20T18:28:07.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.365+0000 7f6244ce2640 1 -- 172.21.15.5:0/1562158959 wait complete. 2024-05-20T18:28:07.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.367+0000 7f6244ce2640 1 Processor -- start 2024-05-20T18:28:07.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.367+0000 7f6244ce2640 1 -- start start 2024-05-20T18:28:07.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.368+0000 7f6244ce2640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62280051c0 0x7f624015a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:07.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.368+0000 7f6244ce2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62400d4d60 0x7f62400d5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:07.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.368+0000 7f6244ce2640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f624015d360 0x7f624015d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:07.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.368+0000 7f6244ce2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f62400b81f0 con 0x7f62400d4d60 2024-05-20T18:28:07.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.368+0000 7f6244ce2640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f62400b8070 con 0x7f624015d360 2024-05-20T18:28:07.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.368+0000 7f6244ce2640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f62400b8370 con 0x7f62280051c0 2024-05-20T18:28:07.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.368+0000 7f623dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62400d4d60 0x7f62400d5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:07.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.368+0000 7f623dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62400d4d60 0x7f62400d5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46074/0 (socket says 172.21.15.5:46074) 2024-05-20T18:28:07.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.368+0000 7f623dd9b640 1 -- 172.21.15.5:0/4201719684 learned_addr learned my addr 172.21.15.5:0/4201719684 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:07.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.368+0000 7f623e59c640 1 --2- 172.21.15.5:0/4201719684 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62280051c0 0x7f624015a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:07.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.368+0000 7f623ed9d640 1 --2- 172.21.15.5:0/4201719684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f624015d360 0x7f624015d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:07.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.368+0000 7f623dd9b640 1 -- 172.21.15.5:0/4201719684 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62280051c0 msgr2=0x7f624015a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:07.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.368+0000 7f623dd9b640 1 --2- 172.21.15.5:0/4201719684 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62280051c0 0x7f624015a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.368+0000 7f623dd9b640 1 -- 172.21.15.5:0/4201719684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f624015d360 msgr2=0x7f624015d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:07.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.368+0000 7f623dd9b640 1 --2- 172.21.15.5:0/4201719684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f624015d360 0x7f624015d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.368+0000 7f623dd9b640 1 -- 172.21.15.5:0/4201719684 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62401f9030 con 0x7f62400d4d60 2024-05-20T18:28:07.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.369+0000 7f623dd9b640 1 --2- 172.21.15.5:0/4201719684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62400d4d60 0x7f62400d5170 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f62340118e0 tx=0x7f6234011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:07.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.369+0000 7f622effd640 1 -- 172.21.15.5:0/4201719684 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623400fa30 con 0x7f62400d4d60 2024-05-20T18:28:07.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.369+0000 7f622effd640 1 -- 172.21.15.5:0/4201719684 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f623400fbd0 con 0x7f62400d4d60 2024-05-20T18:28:07.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.369+0000 7f6244ce2640 1 -- 172.21.15.5:0/4201719684 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62401f9310 con 0x7f62400d4d60 2024-05-20T18:28:07.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.369+0000 7f622effd640 1 -- 172.21.15.5:0/4201719684 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6234014440 con 0x7f62400d4d60 2024-05-20T18:28:07.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.369+0000 7f6244ce2640 1 -- 172.21.15.5:0/4201719684 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f624015b1f0 con 0x7f62400d4d60 2024-05-20T18:28:07.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.370+0000 7f6244ce2640 1 -- 172.21.15.5:0/4201719684 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62401f8da0 con 0x7f62400d4d60 2024-05-20T18:28:07.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.370+0000 7f622effd640 1 -- 172.21.15.5:0/4201719684 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6234012040 con 0x7f62400d4d60 2024-05-20T18:28:07.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.371+0000 7f622effd640 1 --2- 172.21.15.5:0/4201719684 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f622006be30 0x7f622006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:07.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.371+0000 7f622effd640 1 -- 172.21.15.5:0/4201719684 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7f623408b7c0 con 0x7f62400d4d60 2024-05-20T18:28:07.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.371+0000 7f623e59c640 1 --2- 172.21.15.5:0/4201719684 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f622006be30 0x7f622006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:07.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.371+0000 7f623e59c640 1 --2- 172.21.15.5:0/4201719684 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f622006be30 0x7f622006e340 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f62300015d0 tx=0x7f62300078f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:07.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.376+0000 7f622effd640 1 -- 172.21.15.5:0/4201719684 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6234019030 con 0x7f62400d4d60 2024-05-20T18:28:07.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.559+0000 7f6244ce2640 1 -- 172.21.15.5:0/4201719684 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6240075b00 con 0x7f62400d4d60 2024-05-20T18:28:07.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.560+0000 7f622effd640 1 -- 172.21.15.5:0/4201719684 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f6234059d60 con 0x7f62400d4d60 2024-05-20T18:28:07.559 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:28:07.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.561+0000 7f6244ce2640 1 -- 172.21.15.5:0/4201719684 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f622006be30 msgr2=0x7f622006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:07.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.561+0000 7f6244ce2640 1 --2- 172.21.15.5:0/4201719684 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f622006be30 0x7f622006e340 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f62300015d0 tx=0x7f62300078f0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.561+0000 7f6244ce2640 1 -- 172.21.15.5:0/4201719684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62400d4d60 msgr2=0x7f62400d5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:07.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.561+0000 7f6244ce2640 1 --2- 172.21.15.5:0/4201719684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62400d4d60 0x7f62400d5170 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f62340118e0 tx=0x7f6234011ce0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.561+0000 7f6244ce2640 1 -- 172.21.15.5:0/4201719684 shutdown_connections 2024-05-20T18:28:07.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.561+0000 7f6244ce2640 1 --2- 172.21.15.5:0/4201719684 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f622006be30 0x7f622006e340 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.561+0000 7f6244ce2640 1 --2- 172.21.15.5:0/4201719684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f624015d360 0x7f624015d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.561+0000 7f6244ce2640 1 --2- 172.21.15.5:0/4201719684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62400d4d60 0x7f62400d5170 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.561+0000 7f6244ce2640 1 --2- 172.21.15.5:0/4201719684 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62280051c0 0x7f624015a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.561+0000 7f6244ce2640 1 -- 172.21.15.5:0/4201719684 >> 172.21.15.5:0/4201719684 conn(0x7f624006d240 msgr2=0x7f624006d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:07.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.562+0000 7f6244ce2640 1 -- 172.21.15.5:0/4201719684 shutdown_connections 2024-05-20T18:28:07.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.562+0000 7f6244ce2640 1 -- 172.21.15.5:0/4201719684 wait complete. 2024-05-20T18:28:07.572 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:28:07.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.691+0000 7f5c61e21640 1 Processor -- start 2024-05-20T18:28:07.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.691+0000 7f5c61e21640 1 -- start start 2024-05-20T18:28:07.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.693+0000 7f5c61e21640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5c5c162820 0x7f5c5c0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:07.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.693+0000 7f5c61e21640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c5c0b3e80 0x7f5c5c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:07.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.693+0000 7f5c61e21640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5c5c06d910 0x7f5c5c0d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:07.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.693+0000 7f5c61e21640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f5c5c06dd20 con 0x7f5c5c06d910 2024-05-20T18:28:07.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.693+0000 7f5c61e21640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f5c5c0d0630 con 0x7f5c5c162820 2024-05-20T18:28:07.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.693+0000 7f5c61e21640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5c5c0d39c0 con 0x7f5c5c0b3e80 2024-05-20T18:28:07.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.693+0000 7f5c5bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5c5c06d910 0x7f5c5c0d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:07.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.693+0000 7f5c5bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5c5c06d910 0x7f5c5c0d27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46090/0 (socket says 172.21.15.5:46090) 2024-05-20T18:28:07.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.693+0000 7f5c5bfff640 1 -- 172.21.15.5:0/3312934979 learned_addr learned my addr 172.21.15.5:0/3312934979 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:07.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.693+0000 7f5c5b7fe640 1 --2- 172.21.15.5:0/3312934979 >> v2:172.21.15.63:3300/0 conn(0x7f5c5c162820 0x7f5c5c0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:07.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.693+0000 7f5c52ffd640 1 --2- 172.21.15.5:0/3312934979 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c5c0b3e80 0x7f5c5c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:07.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.693+0000 7f5c5bfff640 1 -- 172.21.15.5:0/3312934979 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c5c0b3e80 msgr2=0x7f5c5c06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:07.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.693+0000 7f5c5bfff640 1 --2- 172.21.15.5:0/3312934979 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c5c0b3e80 0x7f5c5c06d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.693+0000 7f5c5bfff640 1 -- 172.21.15.5:0/3312934979 >> v2:172.21.15.63:3300/0 conn(0x7f5c5c162820 msgr2=0x7f5c5c0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:07.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.693+0000 7f5c5bfff640 1 --2- 172.21.15.5:0/3312934979 >> v2:172.21.15.63:3300/0 conn(0x7f5c5c162820 0x7f5c5c0b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.693+0000 7f5c5bfff640 1 -- 172.21.15.5:0/3312934979 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5c0d3ca0 con 0x7f5c5c06d910 2024-05-20T18:28:07.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.694+0000 7f5c5bfff640 1 --2- 172.21.15.5:0/3312934979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c5c06d910 0x7f5c5c0d27a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c0b7740 tx=0x7f5c5402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317fd4da38dfcb16 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:07.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.694+0000 7f5c5affd640 1 -- 172.21.15.5:0/3312934979 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c54036070 con 0x7f5c5c06d910 2024-05-20T18:28:07.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.694+0000 7f5c5affd640 1 -- 172.21.15.5:0/3312934979 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c54031600 con 0x7f5c5c06d910 2024-05-20T18:28:07.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.694+0000 7f5c5affd640 1 -- 172.21.15.5:0/3312934979 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c540319d0 con 0x7f5c5c06d910 2024-05-20T18:28:07.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.695+0000 7f5c61e21640 1 -- 172.21.15.5:0/3312934979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c5c06d910 msgr2=0x7f5c5c0d27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:07.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.695+0000 7f5c61e21640 1 --2- 172.21.15.5:0/3312934979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c5c06d910 0x7f5c5c0d27a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c0b7740 tx=0x7f5c5402a6a0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.695+0000 7f5c61e21640 1 -- 172.21.15.5:0/3312934979 shutdown_connections 2024-05-20T18:28:07.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.695+0000 7f5c61e21640 1 --2- 172.21.15.5:0/3312934979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c5c06d910 0x7f5c5c0d27a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.695+0000 7f5c61e21640 1 --2- 172.21.15.5:0/3312934979 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c5c0b3e80 0x7f5c5c06d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.695+0000 7f5c61e21640 1 --2- 172.21.15.5:0/3312934979 >> v2:172.21.15.63:3300/0 conn(0x7f5c5c162820 0x7f5c5c0b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.695+0000 7f5c61e21640 1 -- 172.21.15.5:0/3312934979 >> 172.21.15.5:0/3312934979 conn(0x7f5c5c0d4bd0 msgr2=0x7f5c5c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:07.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.695+0000 7f5c61e21640 1 -- 172.21.15.5:0/3312934979 shutdown_connections 2024-05-20T18:28:07.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.695+0000 7f5c61e21640 1 -- 172.21.15.5:0/3312934979 wait complete. 2024-05-20T18:28:07.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.697+0000 7f5c61e21640 1 Processor -- start 2024-05-20T18:28:07.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.697+0000 7f5c61e21640 1 -- start start 2024-05-20T18:28:07.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.698+0000 7f5c61e21640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c5c06d910 0x7f5c5c14cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:07.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.698+0000 7f5c61e21640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c5c0b3e80 0x7f5c5c15b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:07.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.698+0000 7f5c61e21640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c5c162820 0x7f5c5c15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:07.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.698+0000 7f5c61e21640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c0d0300 con 0x7f5c5c162820 2024-05-20T18:28:07.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.698+0000 7f5c61e21640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c0d4260 con 0x7f5c5c06d910 2024-05-20T18:28:07.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.698+0000 7f5c61e21640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5c5c0d4780 con 0x7f5c5c0b3e80 2024-05-20T18:28:07.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.698+0000 7f5c5bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c5c162820 0x7f5c5c15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:07.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.698+0000 7f5c5bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c5c162820 0x7f5c5c15b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46104/0 (socket says 172.21.15.5:46104) 2024-05-20T18:28:07.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.698+0000 7f5c5bfff640 1 -- 172.21.15.5:0/4143040095 learned_addr learned my addr 172.21.15.5:0/4143040095 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:07.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.699+0000 7f5c52ffd640 1 --2- 172.21.15.5:0/4143040095 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c5c0b3e80 0x7f5c5c15b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:07.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.699+0000 7f5c5b7fe640 1 --2- 172.21.15.5:0/4143040095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c5c06d910 0x7f5c5c14cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:07.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.699+0000 7f5c5bfff640 1 -- 172.21.15.5:0/4143040095 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c5c0b3e80 msgr2=0x7f5c5c15b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:07.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.699+0000 7f5c5bfff640 1 --2- 172.21.15.5:0/4143040095 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c5c0b3e80 0x7f5c5c15b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.699+0000 7f5c5bfff640 1 -- 172.21.15.5:0/4143040095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c5c06d910 msgr2=0x7f5c5c14cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:07.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.699+0000 7f5c5bfff640 1 --2- 172.21.15.5:0/4143040095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c5c06d910 0x7f5c5c14cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.699+0000 7f5c5bfff640 1 -- 172.21.15.5:0/4143040095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5c0b44c0 con 0x7f5c5c162820 2024-05-20T18:28:07.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.699+0000 7f5c5b7fe640 1 --2- 172.21.15.5:0/4143040095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c5c06d910 0x7f5c5c14cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:07.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.699+0000 7f5c52ffd640 1 --2- 172.21.15.5:0/4143040095 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c5c0b3e80 0x7f5c5c15b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:07.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.699+0000 7f5c5bfff640 1 --2- 172.21.15.5:0/4143040095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c5c162820 0x7f5c5c15b6a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5c54009f40 tx=0x7f5c54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:07.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.699+0000 7f5c58ff9640 1 -- 172.21.15.5:0/4143040095 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5403f020 con 0x7f5c5c162820 2024-05-20T18:28:07.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.699+0000 7f5c58ff9640 1 -- 172.21.15.5:0/4143040095 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c54037040 con 0x7f5c5c162820 2024-05-20T18:28:07.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.699+0000 7f5c58ff9640 1 -- 172.21.15.5:0/4143040095 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c54036070 con 0x7f5c5c162820 2024-05-20T18:28:07.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.699+0000 7f5c61e21640 1 -- 172.21.15.5:0/4143040095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c5c0b47a0 con 0x7f5c5c162820 2024-05-20T18:28:07.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.699+0000 7f5c61e21640 1 -- 172.21.15.5:0/4143040095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c5c0b4d60 con 0x7f5c5c162820 2024-05-20T18:28:07.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.700+0000 7f5c61e21640 1 -- 172.21.15.5:0/4143040095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c5c075b00 con 0x7f5c5c162820 2024-05-20T18:28:07.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.701+0000 7f5c58ff9640 1 -- 172.21.15.5:0/4143040095 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5c54042020 con 0x7f5c5c162820 2024-05-20T18:28:07.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.702+0000 7f5c58ff9640 1 --2- 172.21.15.5:0/4143040095 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5c3c06be30 0x7f5c3c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:07.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.702+0000 7f5c5b7fe640 1 --2- 172.21.15.5:0/4143040095 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5c3c06be30 0x7f5c3c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:07.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.702+0000 7f5c58ff9640 1 -- 172.21.15.5:0/4143040095 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7f5c540aba40 con 0x7f5c5c162820 2024-05-20T18:28:07.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.702+0000 7f5c5b7fe640 1 --2- 172.21.15.5:0/4143040095 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5c3c06be30 0x7f5c3c06e340 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009af0 tx=0x7f5c4c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:07.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.706+0000 7f5c58ff9640 1 -- 172.21.15.5:0/4143040095 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5c54049020 con 0x7f5c5c162820 2024-05-20T18:28:07.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.892+0000 7f5c61e21640 1 -- 172.21.15.5:0/4143040095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5c5c1eac50 con 0x7f5c5c162820 2024-05-20T18:28:07.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.893+0000 7f5c58ff9640 1 -- 172.21.15.5:0/4143040095 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7f5c54079900 con 0x7f5c5c162820 2024-05-20T18:28:07.892 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:28:07.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.895+0000 7f5c61e21640 1 -- 172.21.15.5:0/4143040095 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5c3c06be30 msgr2=0x7f5c3c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:07.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.895+0000 7f5c61e21640 1 --2- 172.21.15.5:0/4143040095 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5c3c06be30 0x7f5c3c06e340 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009af0 tx=0x7f5c4c008040 comp rx=0 tx=0).stop 2024-05-20T18:28:07.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.895+0000 7f5c61e21640 1 -- 172.21.15.5:0/4143040095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c5c162820 msgr2=0x7f5c5c15b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:07.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.895+0000 7f5c61e21640 1 --2- 172.21.15.5:0/4143040095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c5c162820 0x7f5c5c15b6a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5c54009f40 tx=0x7f5c54004a40 comp rx=0 tx=0).stop 2024-05-20T18:28:07.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.895+0000 7f5c61e21640 1 -- 172.21.15.5:0/4143040095 shutdown_connections 2024-05-20T18:28:07.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.895+0000 7f5c61e21640 1 --2- 172.21.15.5:0/4143040095 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5c3c06be30 0x7f5c3c06e340 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.895+0000 7f5c61e21640 1 --2- 172.21.15.5:0/4143040095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c5c162820 0x7f5c5c15b6a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.895+0000 7f5c61e21640 1 --2- 172.21.15.5:0/4143040095 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c5c0b3e80 0x7f5c5c15b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.895+0000 7f5c61e21640 1 --2- 172.21.15.5:0/4143040095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c5c06d910 0x7f5c5c14cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:07.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.895+0000 7f5c61e21640 1 -- 172.21.15.5:0/4143040095 >> 172.21.15.5:0/4143040095 conn(0x7f5c5c0d4bd0 msgr2=0x7f5c5c0ba460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:07.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.895+0000 7f5c61e21640 1 -- 172.21.15.5:0/4143040095 shutdown_connections 2024-05-20T18:28:07.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:07.895+0000 7f5c61e21640 1 -- 172.21.15.5:0/4143040095 wait complete. 2024-05-20T18:28:07.906 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T18:28:08.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.023+0000 7fe412021640 1 Processor -- start 2024-05-20T18:28:08.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.023+0000 7fe412021640 1 -- start start 2024-05-20T18:28:08.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.024+0000 7fe412021640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe40c06d3d0 0x7fe40c06d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:08.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.024+0000 7fe412021640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe40c06dd20 con 0x7fe40c162820 2024-05-20T18:28:08.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.024+0000 7fe412021640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe40c0d0630 con 0x7fe40c0b8190 2024-05-20T18:28:08.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.024+0000 7fe412021640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe40c0d39c0 con 0x7fe40c06d3d0 2024-05-20T18:28:08.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.025+0000 7fe403fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe40c06d3d0 0x7fe40c06d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:08.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.025+0000 7fe40b7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe40c162820 0x7fe40c0b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35678/0 (socket says 172.21.15.5:35678) 2024-05-20T18:28:08.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.025+0000 7fe403fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe40c06d3d0 0x7fe40c06d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46120/0 (socket says 172.21.15.5:46120) 2024-05-20T18:28:08.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.025+0000 7fe40b7fe640 1 -- 172.21.15.5:0/2710366403 learned_addr learned my addr 172.21.15.5:0/2710366403 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:08.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.025+0000 7fe403fff640 1 -- 172.21.15.5:0/2710366403 >> v1:172.21.15.63:6789/0 conn(0x7fe40c0b8190 legacy=0x7fe40c0b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:08.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.025+0000 7fe403fff640 1 -- 172.21.15.5:0/2710366403 >> v1:172.21.15.5:6789/0 conn(0x7fe40c162820 legacy=0x7fe40c0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:08.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.025+0000 7fe403fff640 1 -- 172.21.15.5:0/2710366403 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe40c0d3ca0 con 0x7fe40c06d3d0 2024-05-20T18:28:08.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.025+0000 7fe40affd640 1 -- 172.21.15.5:0/2710366403 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 621585448 0 0) 0x7fe3fc001110 con 0x7fe40c162820 2024-05-20T18:28:08.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.026+0000 7fe403fff640 1 --2- 172.21.15.5:0/2710366403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe40c06d3d0 0x7fe40c06d7e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe3f40097a0 tx=0x7fe3f402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c0e5658d28a675 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:08.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.026+0000 7fe40affd640 1 -- 172.21.15.5:0/2710366403 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4037070 con 0x7fe40c06d3d0 2024-05-20T18:28:08.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.026+0000 7fe40affd640 1 -- 172.21.15.5:0/2710366403 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3f4032440 con 0x7fe40c06d3d0 2024-05-20T18:28:08.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.026+0000 7fe40affd640 1 -- 172.21.15.5:0/2710366403 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4032810 con 0x7fe40c06d3d0 2024-05-20T18:28:08.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.027+0000 7fe412021640 1 -- 172.21.15.5:0/2710366403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe40c06d3d0 msgr2=0x7fe40c06d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:08.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.027+0000 7fe412021640 1 --2- 172.21.15.5:0/2710366403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe40c06d3d0 0x7fe40c06d7e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe3f40097a0 tx=0x7fe3f402a230 comp rx=0 tx=0).stop 2024-05-20T18:28:08.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.027+0000 7fe412021640 1 -- 172.21.15.5:0/2710366403 shutdown_connections 2024-05-20T18:28:08.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.027+0000 7fe412021640 1 --2- 172.21.15.5:0/2710366403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe40c06d3d0 0x7fe40c06d7e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:08.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.027+0000 7fe412021640 1 -- 172.21.15.5:0/2710366403 >> 172.21.15.5:0/2710366403 conn(0x7fe40c0d4bd0 msgr2=0x7fe40c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:08.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.027+0000 7fe412021640 1 -- 172.21.15.5:0/2710366403 shutdown_connections 2024-05-20T18:28:08.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.027+0000 7fe412021640 1 -- 172.21.15.5:0/2710366403 wait complete. 2024-05-20T18:28:08.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.028+0000 7fe412021640 1 Processor -- start 2024-05-20T18:28:08.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.029+0000 7fe412021640 1 -- start start 2024-05-20T18:28:08.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.030+0000 7fe412021640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe40c06d3d0 0x7fe40c1543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:08.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.030+0000 7fe412021640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe40c0b8190 0x7fe40c0b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:08.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.030+0000 7fe412021640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe40c162820 0x7fe40c0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:08.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.030+0000 7fe412021640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c0d0300 con 0x7fe40c0b8190 2024-05-20T18:28:08.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.030+0000 7fe412021640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c0d42e0 con 0x7fe40c162820 2024-05-20T18:28:08.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.030+0000 7fe412021640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe40c0b4050 con 0x7fe40c06d3d0 2024-05-20T18:28:08.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.030+0000 7fe403fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe40c0b8190 0x7fe40c0b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:08.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.030+0000 7fe403fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe40c0b8190 0x7fe40c0b56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46126/0 (socket says 172.21.15.5:46126) 2024-05-20T18:28:08.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.030+0000 7fe403fff640 1 -- 172.21.15.5:0/2444146510 learned_addr learned my addr 172.21.15.5:0/2444146510 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:08.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.030+0000 7fe40b7fe640 1 --2- 172.21.15.5:0/2444146510 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe40c06d3d0 0x7fe40c1543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:08.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.030+0000 7fe40bfff640 1 --2- 172.21.15.5:0/2444146510 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe40c162820 0x7fe40c0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:08.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.030+0000 7fe403fff640 1 -- 172.21.15.5:0/2444146510 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe40c06d3d0 msgr2=0x7fe40c1543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:08.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.030+0000 7fe403fff640 1 --2- 172.21.15.5:0/2444146510 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe40c06d3d0 0x7fe40c1543f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:08.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.030+0000 7fe403fff640 1 -- 172.21.15.5:0/2444146510 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe40c162820 msgr2=0x7fe40c0c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:08.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.030+0000 7fe403fff640 1 --2- 172.21.15.5:0/2444146510 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe40c162820 0x7fe40c0c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:08.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.030+0000 7fe403fff640 1 -- 172.21.15.5:0/2444146510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe40c0b6060 con 0x7fe40c0b8190 2024-05-20T18:28:08.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.031+0000 7fe403fff640 1 --2- 172.21.15.5:0/2444146510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe40c0b8190 0x7fe40c0b56c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4000c00 tx=0x7fe3f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:08.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.031+0000 7fe408ff9640 1 -- 172.21.15.5:0/2444146510 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4037040 con 0x7fe40c0b8190 2024-05-20T18:28:08.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.031+0000 7fe408ff9640 1 -- 172.21.15.5:0/2444146510 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3f4042070 con 0x7fe40c0b8190 2024-05-20T18:28:08.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.031+0000 7fe408ff9640 1 -- 172.21.15.5:0/2444146510 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4030a70 con 0x7fe40c0b8190 2024-05-20T18:28:08.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.031+0000 7fe412021640 1 -- 172.21.15.5:0/2444146510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe40c0b6340 con 0x7fe40c0b8190 2024-05-20T18:28:08.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.031+0000 7fe412021640 1 -- 172.21.15.5:0/2444146510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe40c0b6900 con 0x7fe40c0b8190 2024-05-20T18:28:08.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.032+0000 7fe412021640 1 -- 172.21.15.5:0/2444146510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe40c075b00 con 0x7fe40c0b8190 2024-05-20T18:28:08.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.032+0000 7fe408ff9640 1 -- 172.21.15.5:0/2444146510 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe3f4030c10 con 0x7fe40c0b8190 2024-05-20T18:28:08.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.033+0000 7fe408ff9640 1 --2- 172.21.15.5:0/2444146510 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe3ec06be60 0x7fe3ec06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:08.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.033+0000 7fe40b7fe640 1 --2- 172.21.15.5:0/2444146510 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe3ec06be60 0x7fe3ec06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:08.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.033+0000 7fe408ff9640 1 -- 172.21.15.5:0/2444146510 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7fe3f40abc80 con 0x7fe40c0b8190 2024-05-20T18:28:08.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.034+0000 7fe40b7fe640 1 --2- 172.21.15.5:0/2444146510 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe3ec06be60 0x7fe3ec06e370 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc000f60 tx=0x7fe3fc007e40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:08.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.038+0000 7fe408ff9640 1 -- 172.21.15.5:0/2444146510 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe3f40b1020 con 0x7fe40c0b8190 2024-05-20T18:28:08.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.223+0000 7fe412021640 1 -- 172.21.15.5:0/2444146510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe40c15d610 con 0x7fe40c0b8190 2024-05-20T18:28:08.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.223+0000 7fe408ff9640 1 -- 172.21.15.5:0/2444146510 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7fe3f4079b20 con 0x7fe40c0b8190 2024-05-20T18:28:08.223 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:28:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.226+0000 7fe412021640 1 -- 172.21.15.5:0/2444146510 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe3ec06be60 msgr2=0x7fe3ec06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.226+0000 7fe412021640 1 --2- 172.21.15.5:0/2444146510 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe3ec06be60 0x7fe3ec06e370 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc000f60 tx=0x7fe3fc007e40 comp rx=0 tx=0).stop 2024-05-20T18:28:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.226+0000 7fe412021640 1 -- 172.21.15.5:0/2444146510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe40c0b8190 msgr2=0x7fe40c0b56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.226+0000 7fe412021640 1 --2- 172.21.15.5:0/2444146510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe40c0b8190 0x7fe40c0b56c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4000c00 tx=0x7fe3f40046c0 comp rx=0 tx=0).stop 2024-05-20T18:28:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.226+0000 7fe412021640 1 -- 172.21.15.5:0/2444146510 shutdown_connections 2024-05-20T18:28:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.226+0000 7fe412021640 1 --2- 172.21.15.5:0/2444146510 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe3ec06be60 0x7fe3ec06e370 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:08.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.226+0000 7fe412021640 1 --2- 172.21.15.5:0/2444146510 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe40c162820 0x7fe40c0c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:08.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.226+0000 7fe412021640 1 --2- 172.21.15.5:0/2444146510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe40c0b8190 0x7fe40c0b56c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:08.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.226+0000 7fe412021640 1 --2- 172.21.15.5:0/2444146510 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe40c06d3d0 0x7fe40c1543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:08.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.226+0000 7fe412021640 1 -- 172.21.15.5:0/2444146510 >> 172.21.15.5:0/2444146510 conn(0x7fe40c0d4bd0 msgr2=0x7fe40c15fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:08.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.226+0000 7fe412021640 1 -- 172.21.15.5:0/2444146510 shutdown_connections 2024-05-20T18:28:08.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.226+0000 7fe412021640 1 -- 172.21.15.5:0/2444146510 wait complete. 2024-05-20T18:28:08.237 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T18:28:08.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.353+0000 7fe19ba22640 1 Processor -- start 2024-05-20T18:28:08.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.354+0000 7fe19ba22640 1 -- start start 2024-05-20T18:28:08.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.355+0000 7fe19ba22640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe194162820 0x7fe1940b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:08.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.355+0000 7fe19ba22640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe1940b3e80 0x7fe19406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:08.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.355+0000 7fe19ba22640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe19406dd20 con 0x7fe19406d910 2024-05-20T18:28:08.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.355+0000 7fe19ba22640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe1940d0630 con 0x7fe194162820 2024-05-20T18:28:08.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.355+0000 7fe19ba22640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe1940d39c0 con 0x7fe1940b3e80 2024-05-20T18:28:08.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.355+0000 7fe1997be640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe194162820 0x7fe1940b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:08.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.355+0000 7fe199fbf640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe19406d910 0x7fe1940d27a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35684/0 (socket says 172.21.15.5:35684) 2024-05-20T18:28:08.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.355+0000 7fe199fbf640 1 -- 172.21.15.5:0/4103090270 learned_addr learned my addr 172.21.15.5:0/4103090270 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:08.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.355+0000 7fe198fbd640 1 --2- 172.21.15.5:0/4103090270 >> v2:172.21.15.63:3300/0 conn(0x7fe1940b3e80 0x7fe19406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:08.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.355+0000 7fe1997be640 1 -- 172.21.15.5:0/4103090270 >> v2:172.21.15.63:3300/0 conn(0x7fe1940b3e80 msgr2=0x7fe19406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:08.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.355+0000 7fe1997be640 1 --2- 172.21.15.5:0/4103090270 >> v2:172.21.15.63:3300/0 conn(0x7fe1940b3e80 0x7fe19406d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:08.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.355+0000 7fe1997be640 1 -- 172.21.15.5:0/4103090270 >> v1:172.21.15.5:6789/0 conn(0x7fe19406d910 legacy=0x7fe1940d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:08.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.355+0000 7fe18bfff640 1 -- 172.21.15.5:0/4103090270 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4172711496 0 0) 0x7fe18c001490 con 0x7fe19406d910 2024-05-20T18:28:08.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.355+0000 7fe1997be640 1 -- 172.21.15.5:0/4103090270 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1940d3ca0 con 0x7fe194162820 2024-05-20T18:28:08.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.355+0000 7fe198fbd640 1 --2- 172.21.15.5:0/4103090270 >> v2:172.21.15.63:3300/0 conn(0x7fe1940b3e80 0x7fe19406d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:08.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.356+0000 7fe1997be640 1 --2- 172.21.15.5:0/4103090270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe194162820 0x7fe1940b3940 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe1940d1230 tx=0x7fe19002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04663dd7975f888 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:08.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.356+0000 7fe18bfff640 1 -- 172.21.15.5:0/4103090270 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe190037070 con 0x7fe194162820 2024-05-20T18:28:08.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.356+0000 7fe18bfff640 1 -- 172.21.15.5:0/4103090270 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe190032440 con 0x7fe194162820 2024-05-20T18:28:08.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.356+0000 7fe18bfff640 1 -- 172.21.15.5:0/4103090270 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe190032810 con 0x7fe194162820 2024-05-20T18:28:08.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.357+0000 7fe19ba22640 1 -- 172.21.15.5:0/4103090270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe194162820 msgr2=0x7fe1940b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:08.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.357+0000 7fe19ba22640 1 --2- 172.21.15.5:0/4103090270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe194162820 0x7fe1940b3940 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe1940d1230 tx=0x7fe19002a1a0 comp rx=0 tx=0).stop 2024-05-20T18:28:08.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.357+0000 7fe19ba22640 1 -- 172.21.15.5:0/4103090270 shutdown_connections 2024-05-20T18:28:08.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.357+0000 7fe19ba22640 1 --2- 172.21.15.5:0/4103090270 >> v2:172.21.15.63:3300/0 conn(0x7fe1940b3e80 0x7fe19406d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:08.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.357+0000 7fe19ba22640 1 --2- 172.21.15.5:0/4103090270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe194162820 0x7fe1940b3940 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:08.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.357+0000 7fe19ba22640 1 -- 172.21.15.5:0/4103090270 >> 172.21.15.5:0/4103090270 conn(0x7fe1940d4bd0 msgr2=0x7fe1940d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:08.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.357+0000 7fe19ba22640 1 -- 172.21.15.5:0/4103090270 shutdown_connections 2024-05-20T18:28:08.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.357+0000 7fe19ba22640 1 -- 172.21.15.5:0/4103090270 wait complete. 2024-05-20T18:28:08.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.359+0000 7fe19ba22640 1 Processor -- start 2024-05-20T18:28:08.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.359+0000 7fe19ba22640 1 -- start start 2024-05-20T18:28:08.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.360+0000 7fe19ba22640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe19406d910 0x7fe1940ce500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:08.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.360+0000 7fe19ba22640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1940b3e80 0x7fe194148c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:08.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.360+0000 7fe19ba22640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe194162820 0x7fe194149150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:08.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.360+0000 7fe19ba22640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1940d0300 con 0x7fe1940b3e80 2024-05-20T18:28:08.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.360+0000 7fe19ba22640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe1940d4260 con 0x7fe19406d910 2024-05-20T18:28:08.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.360+0000 7fe19ba22640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe1940b91a0 con 0x7fe194162820 2024-05-20T18:28:08.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.361+0000 7fe198fbd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1940b3e80 0x7fe194148c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:08.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.361+0000 7fe198fbd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1940b3e80 0x7fe194148c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46138/0 (socket says 172.21.15.5:46138) 2024-05-20T18:28:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.361+0000 7fe198fbd640 1 -- 172.21.15.5:0/2463158030 learned_addr learned my addr 172.21.15.5:0/2463158030 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.361+0000 7fe1997be640 1 --2- 172.21.15.5:0/2463158030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe19406d910 0x7fe1940ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.361+0000 7fe199fbf640 1 --2- 172.21.15.5:0/2463158030 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe194162820 0x7fe194149150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.361+0000 7fe198fbd640 1 -- 172.21.15.5:0/2463158030 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe194162820 msgr2=0x7fe194149150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.361+0000 7fe198fbd640 1 --2- 172.21.15.5:0/2463158030 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe194162820 0x7fe194149150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.361+0000 7fe198fbd640 1 -- 172.21.15.5:0/2463158030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe19406d910 msgr2=0x7fe1940ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.361+0000 7fe198fbd640 1 --2- 172.21.15.5:0/2463158030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe19406d910 0x7fe1940ce500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.361+0000 7fe198fbd640 1 -- 172.21.15.5:0/2463158030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1940cea40 con 0x7fe1940b3e80 2024-05-20T18:28:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.361+0000 7fe198fbd640 1 --2- 172.21.15.5:0/2463158030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1940b3e80 0x7fe194148c10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe18400d800 tx=0x7fe18400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.361+0000 7fe189ffb640 1 -- 172.21.15.5:0/2463158030 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe184004030 con 0x7fe1940b3e80 2024-05-20T18:28:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.361+0000 7fe189ffb640 1 -- 172.21.15.5:0/2463158030 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1840041d0 con 0x7fe1940b3e80 2024-05-20T18:28:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.361+0000 7fe189ffb640 1 -- 172.21.15.5:0/2463158030 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1840096a0 con 0x7fe1940b3e80 2024-05-20T18:28:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.361+0000 7fe19ba22640 1 -- 172.21.15.5:0/2463158030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe194146c20 con 0x7fe1940b3e80 2024-05-20T18:28:08.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.362+0000 7fe19ba22640 1 -- 172.21.15.5:0/2463158030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1941471e0 con 0x7fe1940b3e80 2024-05-20T18:28:08.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.363+0000 7fe19ba22640 1 -- 172.21.15.5:0/2463158030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe194075b00 con 0x7fe1940b3e80 2024-05-20T18:28:08.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.363+0000 7fe189ffb640 1 -- 172.21.15.5:0/2463158030 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe184009840 con 0x7fe1940b3e80 2024-05-20T18:28:08.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.364+0000 7fe189ffb640 1 --2- 172.21.15.5:0/2463158030 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe17806be60 0x7fe17806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:08.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.364+0000 7fe189ffb640 1 -- 172.21.15.5:0/2463158030 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7fe18408b3b0 con 0x7fe1940b3e80 2024-05-20T18:28:08.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.364+0000 7fe1997be640 1 --2- 172.21.15.5:0/2463158030 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe17806be60 0x7fe17806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:08.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.364+0000 7fe1997be640 1 --2- 172.21.15.5:0/2463158030 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe17806be60 0x7fe17806e370 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe190034000 tx=0x7fe19002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:08.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.368+0000 7fe189ffb640 1 -- 172.21.15.5:0/2463158030 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe1940cea40 con 0x7fe1940b3e80 2024-05-20T18:28:08.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.572+0000 7fe19ba22640 1 -- 172.21.15.5:0/2463158030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe19406c530 con 0x7fe1940b3e80 2024-05-20T18:28:08.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.572+0000 7fe189ffb640 1 -- 172.21.15.5:0/2463158030 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7fe184059300 con 0x7fe1940b3e80 2024-05-20T18:28:08.572 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-05-20T18:28:08.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.575+0000 7fe19ba22640 1 -- 172.21.15.5:0/2463158030 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe17806be60 msgr2=0x7fe17806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:08.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.575+0000 7fe19ba22640 1 --2- 172.21.15.5:0/2463158030 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe17806be60 0x7fe17806e370 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe190034000 tx=0x7fe19002e000 comp rx=0 tx=0).stop 2024-05-20T18:28:08.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.575+0000 7fe19ba22640 1 -- 172.21.15.5:0/2463158030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1940b3e80 msgr2=0x7fe194148c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:08.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.575+0000 7fe19ba22640 1 --2- 172.21.15.5:0/2463158030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1940b3e80 0x7fe194148c10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe18400d800 tx=0x7fe18400dcc0 comp rx=0 tx=0).stop 2024-05-20T18:28:08.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.575+0000 7fe19ba22640 1 -- 172.21.15.5:0/2463158030 shutdown_connections 2024-05-20T18:28:08.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.575+0000 7fe19ba22640 1 --2- 172.21.15.5:0/2463158030 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe17806be60 0x7fe17806e370 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:08.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.575+0000 7fe19ba22640 1 --2- 172.21.15.5:0/2463158030 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe194162820 0x7fe194149150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:08.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.575+0000 7fe19ba22640 1 --2- 172.21.15.5:0/2463158030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1940b3e80 0x7fe194148c10 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:08.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.575+0000 7fe19ba22640 1 --2- 172.21.15.5:0/2463158030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe19406d910 0x7fe1940ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:08.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.575+0000 7fe19ba22640 1 -- 172.21.15.5:0/2463158030 >> 172.21.15.5:0/2463158030 conn(0x7fe1940d4bd0 msgr2=0x7fe194160eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:08.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.575+0000 7fe19ba22640 1 -- 172.21.15.5:0/2463158030 shutdown_connections 2024-05-20T18:28:08.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.575+0000 7fe19ba22640 1 -- 172.21.15.5:0/2463158030 wait complete. 2024-05-20T18:28:08.586 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-05-20T18:28:08.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:28:08.586 DEBUG:teuthology.orchestra.run.smithi063:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-05-20T18:28:08.619 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi063.front.sepia.ceph.com 2024-05-20T18:28:08.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:28:08.620 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-20T18:28:08.679 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-20T18:28:08.679 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:28:08.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.795+0000 7fa751c08640 1 Processor -- start 2024-05-20T18:28:08.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.795+0000 7fa751c08640 1 -- start start 2024-05-20T18:28:08.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.795+0000 7fa751c08640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa74c15f220 0x7fa74c0d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.796+0000 7fa751c08640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa74c0cfae0 0x7fa74c0ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.796+0000 7fa751c08640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa74c0cda20 con 0x7fa74c0d0e00 2024-05-20T18:28:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.796+0000 7fa751c08640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fa74c0cdd00 con 0x7fa74c0cfae0 2024-05-20T18:28:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.796+0000 7fa751c08640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa74c0cdfe0 con 0x7fa74c15f220 2024-05-20T18:28:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.796+0000 7fa74bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa74c0cfae0 0x7fa74c0ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.796+0000 7fa74bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa74c0cfae0 0x7fa74c0ce7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46152/0 (socket says 172.21.15.5:46152) 2024-05-20T18:28:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.796+0000 7fa74bfff640 1 -- 172.21.15.5:0/1948126205 learned_addr learned my addr 172.21.15.5:0/1948126205 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.796+0000 7fa74b7fe640 1 --2- 172.21.15.5:0/1948126205 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa74c15f220 0x7fa74c0d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.796+0000 7fa74bfff640 1 -- 172.21.15.5:0/1948126205 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa74c15f220 msgr2=0x7fa74c0d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.796+0000 7fa74bfff640 1 --2- 172.21.15.5:0/1948126205 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa74c15f220 0x7fa74c0d08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.796+0000 7fa74bfff640 1 -- 172.21.15.5:0/1948126205 >> v1:172.21.15.63:6789/0 conn(0x7fa74c0d0e00 legacy=0x7fa74c0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.796+0000 7fa74bfff640 1 -- 172.21.15.5:0/1948126205 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa74c0ce2c0 con 0x7fa74c0cfae0 2024-05-20T18:28:08.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.797+0000 7fa74bfff640 1 --2- 172.21.15.5:0/1948126205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa74c0cfae0 0x7fa74c0ce7e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa744009e50 tx=0x7fa74402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed59443749f3d8b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.797+0000 7fa74a7fc640 1 -- 172.21.15.5:0/1948126205 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa744036070 con 0x7fa74c0cfae0 2024-05-20T18:28:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.797+0000 7fa74a7fc640 1 -- 172.21.15.5:0/1948126205 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa744031600 con 0x7fa74c0cfae0 2024-05-20T18:28:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.797+0000 7fa74a7fc640 1 -- 172.21.15.5:0/1948126205 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7440319d0 con 0x7fa74c0cfae0 2024-05-20T18:28:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.798+0000 7fa751c08640 1 -- 172.21.15.5:0/1948126205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa74c0cfae0 msgr2=0x7fa74c0ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.798+0000 7fa751c08640 1 --2- 172.21.15.5:0/1948126205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa74c0cfae0 0x7fa74c0ce7e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa744009e50 tx=0x7fa74402a680 comp rx=0 tx=0).stop 2024-05-20T18:28:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.798+0000 7fa751c08640 1 -- 172.21.15.5:0/1948126205 shutdown_connections 2024-05-20T18:28:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.798+0000 7fa751c08640 1 --2- 172.21.15.5:0/1948126205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa74c0cfae0 0x7fa74c0ce7e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.798+0000 7fa751c08640 1 --2- 172.21.15.5:0/1948126205 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa74c15f220 0x7fa74c0d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.798+0000 7fa751c08640 1 -- 172.21.15.5:0/1948126205 >> 172.21.15.5:0/1948126205 conn(0x7fa74c0b44d0 msgr2=0x7fa74c0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.798+0000 7fa751c08640 1 -- 172.21.15.5:0/1948126205 shutdown_connections 2024-05-20T18:28:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.798+0000 7fa751c08640 1 -- 172.21.15.5:0/1948126205 wait complete. 2024-05-20T18:28:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.800+0000 7fa751c08640 1 Processor -- start 2024-05-20T18:28:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.800+0000 7fa751c08640 1 -- start start 2024-05-20T18:28:08.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.801+0000 7fa751c08640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa74c0d0e00 0x7fa74c151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:08.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.801+0000 7fa751c08640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa74c15f220 0x7fa74c0cb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:08.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.801+0000 7fa751c08640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa74c0b7b40 0x7fa74c0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:08.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.801+0000 7fa751c08640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c15f7b0 con 0x7fa74c0d0e00 2024-05-20T18:28:08.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.801+0000 7fa751c08640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c15f630 con 0x7fa74c0b7b40 2024-05-20T18:28:08.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.801+0000 7fa751c08640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa74c0ced20 con 0x7fa74c15f220 2024-05-20T18:28:08.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.801+0000 7fa74b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa74c0d0e00 0x7fa74c151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:08.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.801+0000 7fa74b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa74c0d0e00 0x7fa74c151e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46158/0 (socket says 172.21.15.5:46158) 2024-05-20T18:28:08.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.801+0000 7fa74b7fe640 1 -- 172.21.15.5:0/1012608380 learned_addr learned my addr 172.21.15.5:0/1012608380 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:08.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.801+0000 7fa74bfff640 1 --2- 172.21.15.5:0/1012608380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa74c0b7b40 0x7fa74c0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:08.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.801+0000 7fa74affd640 1 --2- 172.21.15.5:0/1012608380 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa74c15f220 0x7fa74c0cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:08.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.801+0000 7fa74b7fe640 1 -- 172.21.15.5:0/1012608380 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa74c15f220 msgr2=0x7fa74c0cb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:08.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.802+0000 7fa74b7fe640 1 --2- 172.21.15.5:0/1012608380 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa74c15f220 0x7fa74c0cb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:08.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.802+0000 7fa74b7fe640 1 -- 172.21.15.5:0/1012608380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa74c0b7b40 msgr2=0x7fa74c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:08.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.802+0000 7fa74b7fe640 1 --2- 172.21.15.5:0/1012608380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa74c0b7b40 0x7fa74c0b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:08.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.802+0000 7fa74b7fe640 1 -- 172.21.15.5:0/1012608380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa74c0b6aa0 con 0x7fa74c0d0e00 2024-05-20T18:28:08.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.802+0000 7fa74bfff640 1 --2- 172.21.15.5:0/1012608380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa74c0b7b40 0x7fa74c0b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:08.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.802+0000 7fa74affd640 1 --2- 172.21.15.5:0/1012608380 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa74c15f220 0x7fa74c0cb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:08.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.802+0000 7fa74b7fe640 1 --2- 172.21.15.5:0/1012608380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa74c0d0e00 0x7fa74c151e30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00d600 tx=0x7fa73c00dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:08.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.802+0000 7fa743fff640 1 -- 172.21.15.5:0/1012608380 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c004240 con 0x7fa74c0d0e00 2024-05-20T18:28:08.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.802+0000 7fa743fff640 1 -- 172.21.15.5:0/1012608380 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa73c0043e0 con 0x7fa74c0d0e00 2024-05-20T18:28:08.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.802+0000 7fa751c08640 1 -- 172.21.15.5:0/1012608380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa74c0b6d80 con 0x7fa74c0d0e00 2024-05-20T18:28:08.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.802+0000 7fa743fff640 1 -- 172.21.15.5:0/1012608380 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c003830 con 0x7fa74c0d0e00 2024-05-20T18:28:08.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.802+0000 7fa751c08640 1 -- 172.21.15.5:0/1012608380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa74c0b7340 con 0x7fa74c0d0e00 2024-05-20T18:28:08.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.803+0000 7fa751c08640 1 -- 172.21.15.5:0/1012608380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa74c075b00 con 0x7fa74c0d0e00 2024-05-20T18:28:08.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.804+0000 7fa743fff640 1 -- 172.21.15.5:0/1012608380 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa73c0039d0 con 0x7fa74c0d0e00 2024-05-20T18:28:08.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.804+0000 7fa743fff640 1 --2- 172.21.15.5:0/1012608380 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa72c06be30 0x7fa72c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:08.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.806+0000 7fa74affd640 1 --2- 172.21.15.5:0/1012608380 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa72c06be30 0x7fa72c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:08.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.806+0000 7fa743fff640 1 -- 172.21.15.5:0/1012608380 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7fa73c08b5d0 con 0x7fa74c0d0e00 2024-05-20T18:28:08.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.806+0000 7fa74affd640 1 --2- 172.21.15.5:0/1012608380 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa72c06be30 0x7fa72c06e340 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa74c0c6270 tx=0x7fa734009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:08.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:08.810+0000 7fa743fff640 1 -- 172.21.15.5:0/1012608380 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa73c059ae0 con 0x7fa74c0d0e00 2024-05-20T18:28:09.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.034+0000 7fa751c08640 1 -- 172.21.15.5:0/1012608380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa74c0c62a0 con 0x7fa74c0d0e00 2024-05-20T18:28:09.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.035+0000 7fa743fff640 1 -- 172.21.15.5:0/1012608380 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5168 (secure 0 0 0) 0x7fa73c0593e0 con 0x7fa74c0d0e00 2024-05-20T18:28:09.033 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:09.033 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2982954341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2982954341},{"type":"v1","addr":"172.21.15.5:6836","nonce":2982954341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4032850001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":4032850001},{"type":"v1","addr":"172.21.15.5:6837","nonce":4032850001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6833/424580378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":424580378},{"type":"v1","addr":"172.21.15.63:6833","nonce":424580378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:27:07.164414+0000","modified":"2024-05-20T18:27:29.288681+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/3249423677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3249423677},{"type":"v1","addr":"172.21.15.63:6835","nonce":3249423677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-05-20T18:28:09.034 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 9 2024-05-20T18:28:09.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.037+0000 7fa751c08640 1 -- 172.21.15.5:0/1012608380 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa72c06be30 msgr2=0x7fa72c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:09.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.037+0000 7fa751c08640 1 --2- 172.21.15.5:0/1012608380 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa72c06be30 0x7fa72c06e340 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa74c0c6270 tx=0x7fa734009040 comp rx=0 tx=0).stop 2024-05-20T18:28:09.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.037+0000 7fa751c08640 1 -- 172.21.15.5:0/1012608380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa74c0d0e00 msgr2=0x7fa74c151e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:09.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.037+0000 7fa751c08640 1 --2- 172.21.15.5:0/1012608380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa74c0d0e00 0x7fa74c151e30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00d600 tx=0x7fa73c00dac0 comp rx=0 tx=0).stop 2024-05-20T18:28:09.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.037+0000 7fa751c08640 1 -- 172.21.15.5:0/1012608380 shutdown_connections 2024-05-20T18:28:09.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.037+0000 7fa751c08640 1 --2- 172.21.15.5:0/1012608380 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa72c06be30 0x7fa72c06e340 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:09.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.037+0000 7fa751c08640 1 --2- 172.21.15.5:0/1012608380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa74c0b7b40 0x7fa74c0b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:09.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.037+0000 7fa751c08640 1 --2- 172.21.15.5:0/1012608380 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa74c15f220 0x7fa74c0cb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:09.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.037+0000 7fa751c08640 1 --2- 172.21.15.5:0/1012608380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa74c0d0e00 0x7fa74c151e30 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:09.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.037+0000 7fa751c08640 1 -- 172.21.15.5:0/1012608380 >> 172.21.15.5:0/1012608380 conn(0x7fa74c0b44d0 msgr2=0x7fa74c15e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:09.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.037+0000 7fa751c08640 1 -- 172.21.15.5:0/1012608380 shutdown_connections 2024-05-20T18:28:09.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.037+0000 7fa751c08640 1 -- 172.21.15.5:0/1012608380 wait complete. 2024-05-20T18:28:09.047 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:28:09.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.163+0000 7f91cd244640 1 Processor -- start 2024-05-20T18:28:09.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.163+0000 7f91cd244640 1 -- start start 2024-05-20T18:28:09.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.164+0000 7f91cd244640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f91c80b1e10 0x7f91c815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:09.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.164+0000 7f91cd244640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91c8070850 0x7f91c8070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:09.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.164+0000 7f91cd244640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91c80cd580 con 0x7f91c815d530 2024-05-20T18:28:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.164+0000 7f91cd244640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f91c80ce190 con 0x7f91c80b1e10 2024-05-20T18:28:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.164+0000 7f91cd244640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f91c80cd860 con 0x7f91c8070850 2024-05-20T18:28:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.164+0000 7f91c759e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91c8070850 0x7f91c8070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.164+0000 7f91be59c640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f91c80b1e10 0x7f91c815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.165+0000 7f91c6d9d640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f91c815d530 0x7f91c80b1700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:44898/0 (socket says 172.21.15.5:44898) 2024-05-20T18:28:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.165+0000 7f91c6d9d640 1 -- 172.21.15.5:0/4048520575 learned_addr learned my addr 172.21.15.5:0/4048520575 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.165+0000 7f91c659c640 1 -- 172.21.15.5:0/4048520575 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 791718567 0 0) 0x7f91c80cd580 con 0x7f91c815d530 2024-05-20T18:28:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.165+0000 7f91c659c640 1 -- 172.21.15.5:0/4048520575 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91b4003520 con 0x7f91c815d530 2024-05-20T18:28:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.165+0000 7f91c759e640 1 -- 172.21.15.5:0/4048520575 >> v2:172.21.15.63:3300/0 conn(0x7f91c80b1e10 msgr2=0x7f91c815aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.165+0000 7f91c759e640 1 --2- 172.21.15.5:0/4048520575 >> v2:172.21.15.63:3300/0 conn(0x7f91c80b1e10 0x7f91c815aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.165+0000 7f91c759e640 1 -- 172.21.15.5:0/4048520575 >> v1:172.21.15.63:6789/0 conn(0x7f91c815d530 legacy=0x7f91c80b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.165+0000 7f91c759e640 1 -- 172.21.15.5:0/4048520575 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c80cdb40 con 0x7f91c8070850 2024-05-20T18:28:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.165+0000 7f91be59c640 1 --2- 172.21.15.5:0/4048520575 >> v2:172.21.15.63:3300/0 conn(0x7f91c80b1e10 0x7f91c815aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:28:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.166+0000 7f91c759e640 1 --2- 172.21.15.5:0/4048520575 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91c8070850 0x7f91c8070c60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009d90 tx=0x7f91c002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8570d9d8521ad4db server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:09.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.166+0000 7f91c659c640 1 -- 172.21.15.5:0/4048520575 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0007740 con 0x7f91c8070850 2024-05-20T18:28:09.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.166+0000 7f91c659c640 1 -- 172.21.15.5:0/4048520575 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91c00078e0 con 0x7f91c8070850 2024-05-20T18:28:09.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.166+0000 7f91c659c640 1 -- 172.21.15.5:0/4048520575 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0007cb0 con 0x7f91c8070850 2024-05-20T18:28:09.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.167+0000 7f91cd244640 1 -- 172.21.15.5:0/4048520575 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91c8070850 msgr2=0x7f91c8070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:09.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.167+0000 7f91cd244640 1 --2- 172.21.15.5:0/4048520575 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91c8070850 0x7f91c8070c60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009d90 tx=0x7f91c002a660 comp rx=0 tx=0).stop 2024-05-20T18:28:09.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.167+0000 7f91cd244640 1 -- 172.21.15.5:0/4048520575 shutdown_connections 2024-05-20T18:28:09.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.167+0000 7f91cd244640 1 --2- 172.21.15.5:0/4048520575 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91c8070850 0x7f91c8070c60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:09.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.167+0000 7f91cd244640 1 --2- 172.21.15.5:0/4048520575 >> v2:172.21.15.63:3300/0 conn(0x7f91c80b1e10 0x7f91c815aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:09.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.167+0000 7f91cd244640 1 -- 172.21.15.5:0/4048520575 >> 172.21.15.5:0/4048520575 conn(0x7f91c80d29d0 msgr2=0x7f91c80d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:09.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.167+0000 7f91cd244640 1 -- 172.21.15.5:0/4048520575 shutdown_connections 2024-05-20T18:28:09.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.167+0000 7f91cd244640 1 -- 172.21.15.5:0/4048520575 wait complete. 2024-05-20T18:28:09.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.169+0000 7f91cd244640 1 Processor -- start 2024-05-20T18:28:09.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.169+0000 7f91cd244640 1 -- start start 2024-05-20T18:28:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.170+0000 7f91cd244640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91c8070850 0x7f91c80c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.170+0000 7f91cd244640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91c80b1e10 0x7f91c8159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.170+0000 7f91cd244640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91c815d530 0x7f91c80b6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.170+0000 7f91cd244640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f91c80711a0 con 0x7f91c815d530 2024-05-20T18:28:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.170+0000 7f91cd244640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f91c80cd230 con 0x7f91c8070850 2024-05-20T18:28:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.170+0000 7f91cd244640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f91c8071320 con 0x7f91c80b1e10 2024-05-20T18:28:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.170+0000 7f91c759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91c815d530 0x7f91c80b6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.170+0000 7f91c759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91c815d530 0x7f91c80b6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46166/0 (socket says 172.21.15.5:46166) 2024-05-20T18:28:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.170+0000 7f91c759e640 1 -- 172.21.15.5:0/995734255 learned_addr learned my addr 172.21.15.5:0/995734255 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.171+0000 7f91be59c640 1 --2- 172.21.15.5:0/995734255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91c80b1e10 0x7f91c8159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.171+0000 7f91c6d9d640 1 --2- 172.21.15.5:0/995734255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91c8070850 0x7f91c80c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.171+0000 7f91c759e640 1 -- 172.21.15.5:0/995734255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91c80b1e10 msgr2=0x7f91c8159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.171+0000 7f91c759e640 1 --2- 172.21.15.5:0/995734255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91c80b1e10 0x7f91c8159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.171+0000 7f91c759e640 1 -- 172.21.15.5:0/995734255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91c8070850 msgr2=0x7f91c80c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.171+0000 7f91c759e640 1 --2- 172.21.15.5:0/995734255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91c8070850 0x7f91c80c6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.171+0000 7f91c759e640 1 -- 172.21.15.5:0/995734255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c81f5da0 con 0x7f91c815d530 2024-05-20T18:28:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.171+0000 7f91c759e640 1 --2- 172.21.15.5:0/995734255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91c815d530 0x7f91c80b6e60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f91c002ac50 tx=0x7f91c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.171+0000 7f91bffff640 1 -- 172.21.15.5:0/995734255 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c003f070 con 0x7f91c815d530 2024-05-20T18:28:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.171+0000 7f91bffff640 1 -- 172.21.15.5:0/995734255 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91c0031bc0 con 0x7f91c815d530 2024-05-20T18:28:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.171+0000 7f91cd244640 1 -- 172.21.15.5:0/995734255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91c81f6080 con 0x7f91c815d530 2024-05-20T18:28:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.171+0000 7f91bffff640 1 -- 172.21.15.5:0/995734255 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0009450 con 0x7f91c815d530 2024-05-20T18:28:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.171+0000 7f91cd244640 1 -- 172.21.15.5:0/995734255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91c81f6640 con 0x7f91c815d530 2024-05-20T18:28:09.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.172+0000 7f91cd244640 1 -- 172.21.15.5:0/995734255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91c8075b00 con 0x7f91c815d530 2024-05-20T18:28:09.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.173+0000 7f91bffff640 1 -- 172.21.15.5:0/995734255 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f91c0044460 con 0x7f91c815d530 2024-05-20T18:28:09.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.173+0000 7f91bffff640 1 --2- 172.21.15.5:0/995734255 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f91a806be30 0x7f91a806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:09.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.174+0000 7f91bffff640 1 -- 172.21.15.5:0/995734255 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7f91c00ac970 con 0x7f91c815d530 2024-05-20T18:28:09.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.174+0000 7f91c6d9d640 1 --2- 172.21.15.5:0/995734255 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f91a806be30 0x7f91a806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:09.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.174+0000 7f91c6d9d640 1 --2- 172.21.15.5:0/995734255 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f91a806be30 0x7f91a806e340 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f91b8005880 tx=0x7f91b8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:09.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.178+0000 7f91bffff640 1 -- 172.21.15.5:0/995734255 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f91c007a8c0 con 0x7f91c815d530 2024-05-20T18:28:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.364+0000 7f91cd244640 1 -- 172.21.15.5:0/995734255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91c80c6ae0 con 0x7f91c815d530 2024-05-20T18:28:09.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.365+0000 7f91bffff640 1 -- 172.21.15.5:0/995734255 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18800 (secure 0 0 0) 0x7f91c007ebc0 con 0x7f91c815d530 2024-05-20T18:28:09.364 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:09.364 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:27:56.213938+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:27:05.476596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T18:27:06.137762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T18:27:09.829920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:28:09.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.367+0000 7f91cd244640 1 -- 172.21.15.5:0/995734255 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f91a806be30 msgr2=0x7f91a806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:09.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.367+0000 7f91cd244640 1 --2- 172.21.15.5:0/995734255 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f91a806be30 0x7f91a806e340 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f91b8005880 tx=0x7f91b8009040 comp rx=0 tx=0).stop 2024-05-20T18:28:09.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.367+0000 7f91cd244640 1 -- 172.21.15.5:0/995734255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91c815d530 msgr2=0x7f91c80b6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:09.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.367+0000 7f91cd244640 1 --2- 172.21.15.5:0/995734255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91c815d530 0x7f91c80b6e60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f91c002ac50 tx=0x7f91c0004a40 comp rx=0 tx=0).stop 2024-05-20T18:28:09.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.367+0000 7f91cd244640 1 -- 172.21.15.5:0/995734255 shutdown_connections 2024-05-20T18:28:09.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.367+0000 7f91cd244640 1 --2- 172.21.15.5:0/995734255 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f91a806be30 0x7f91a806e340 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:09.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.367+0000 7f91cd244640 1 --2- 172.21.15.5:0/995734255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91c815d530 0x7f91c80b6e60 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:09.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.367+0000 7f91cd244640 1 --2- 172.21.15.5:0/995734255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91c80b1e10 0x7f91c8159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:09.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.367+0000 7f91cd244640 1 --2- 172.21.15.5:0/995734255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91c8070850 0x7f91c80c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:09.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.367+0000 7f91cd244640 1 -- 172.21.15.5:0/995734255 >> 172.21.15.5:0/995734255 conn(0x7f91c80d29d0 msgr2=0x7f91c815d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:09.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.367+0000 7f91cd244640 1 -- 172.21.15.5:0/995734255 shutdown_connections 2024-05-20T18:28:09.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.367+0000 7f91cd244640 1 -- 172.21.15.5:0/995734255 wait complete. 2024-05-20T18:28:09.378 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:27:05.476596+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:27:06.137762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-20T18:27:09.829920+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-20T18:28:09.379 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:28:09.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.499+0000 7f2061244640 1 Processor -- start 2024-05-20T18:28:09.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.499+0000 7f2061244640 1 -- start start 2024-05-20T18:28:09.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.500+0000 7f2061244640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f205c06d3d0 0x7f205c06d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:09.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.500+0000 7f2061244640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f205c0b8190 0x7f205c0b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:09.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.500+0000 7f2061244640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f205c06dd20 con 0x7f205c162820 2024-05-20T18:28:09.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.500+0000 7f2061244640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f205c0d0630 con 0x7f205c0b8190 2024-05-20T18:28:09.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.500+0000 7f2061244640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f205c0d39c0 con 0x7f205c06d3d0 2024-05-20T18:28:09.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.501+0000 7f205b59e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f205c0b8190 0x7f205c0b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:09.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.501+0000 7f205b59e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f205c0b8190 0x7f205c0b85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46180/0 (socket says 172.21.15.5:46180) 2024-05-20T18:28:09.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.501+0000 7f205b59e640 1 -- 172.21.15.5:0/276798661 learned_addr learned my addr 172.21.15.5:0/276798661 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:09.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.501+0000 7f205a59c640 1 --2- 172.21.15.5:0/276798661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f205c06d3d0 0x7f205c06d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:09.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.501+0000 7f205b59e640 1 -- 172.21.15.5:0/276798661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f205c06d3d0 msgr2=0x7f205c06d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:09.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.501+0000 7f205b59e640 1 --2- 172.21.15.5:0/276798661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f205c06d3d0 0x7f205c06d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:09.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.501+0000 7f205b59e640 1 -- 172.21.15.5:0/276798661 >> v1:172.21.15.63:6789/0 conn(0x7f205c162820 legacy=0x7f205c0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:09.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.501+0000 7f205b59e640 1 -- 172.21.15.5:0/276798661 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f205c0d3ca0 con 0x7f205c0b8190 2024-05-20T18:28:09.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.502+0000 7f205b59e640 1 --2- 172.21.15.5:0/276798661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f205c0b8190 0x7f205c0b85a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2054009870 tx=0x7f205402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a76e495fd573228 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:09.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.502+0000 7f2059d9b640 1 -- 172.21.15.5:0/276798661 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2054037070 con 0x7f205c0b8190 2024-05-20T18:28:09.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.502+0000 7f2059d9b640 1 -- 172.21.15.5:0/276798661 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2054032440 con 0x7f205c0b8190 2024-05-20T18:28:09.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.502+0000 7f2059d9b640 1 -- 172.21.15.5:0/276798661 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2054032810 con 0x7f205c0b8190 2024-05-20T18:28:09.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.503+0000 7f2061244640 1 -- 172.21.15.5:0/276798661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f205c0b8190 msgr2=0x7f205c0b85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:09.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.503+0000 7f2061244640 1 --2- 172.21.15.5:0/276798661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f205c0b8190 0x7f205c0b85a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2054009870 tx=0x7f205402a270 comp rx=0 tx=0).stop 2024-05-20T18:28:09.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.503+0000 7f2061244640 1 -- 172.21.15.5:0/276798661 shutdown_connections 2024-05-20T18:28:09.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.503+0000 7f2061244640 1 --2- 172.21.15.5:0/276798661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f205c0b8190 0x7f205c0b85a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:09.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.503+0000 7f2061244640 1 --2- 172.21.15.5:0/276798661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f205c06d3d0 0x7f205c06d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:09.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.503+0000 7f2061244640 1 -- 172.21.15.5:0/276798661 >> 172.21.15.5:0/276798661 conn(0x7f205c0d4bd0 msgr2=0x7f205c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:09.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.503+0000 7f2061244640 1 -- 172.21.15.5:0/276798661 shutdown_connections 2024-05-20T18:28:09.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.503+0000 7f2061244640 1 -- 172.21.15.5:0/276798661 wait complete. 2024-05-20T18:28:09.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.505+0000 7f2061244640 1 Processor -- start 2024-05-20T18:28:09.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.505+0000 7f2061244640 1 -- start start 2024-05-20T18:28:09.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.506+0000 7f2061244640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f205c06d3d0 0x7f205c0c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:09.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.506+0000 7f2061244640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f205c0b8190 0x7f205c0c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:09.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.506+0000 7f2061244640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f205c162820 0x7f205c14af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:09.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.506+0000 7f2061244640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f205c0d0300 con 0x7f205c0b8190 2024-05-20T18:28:09.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.506+0000 7f2061244640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f205c0d4260 con 0x7f205c06d3d0 2024-05-20T18:28:09.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.506+0000 7f2061244640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f205c0b4050 con 0x7f205c162820 2024-05-20T18:28:09.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.506+0000 7f205a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f205c0b8190 0x7f205c0c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:09.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.506+0000 7f205a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f205c0b8190 0x7f205c0c8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46184/0 (socket says 172.21.15.5:46184) 2024-05-20T18:28:09.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.506+0000 7f205a59c640 1 -- 172.21.15.5:0/3591566453 learned_addr learned my addr 172.21.15.5:0/3591566453 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:09.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.506+0000 7f205b59e640 1 --2- 172.21.15.5:0/3591566453 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f205c162820 0x7f205c14af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:09.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.506+0000 7f205ad9d640 1 --2- 172.21.15.5:0/3591566453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f205c06d3d0 0x7f205c0c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:09.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.507+0000 7f205a59c640 1 -- 172.21.15.5:0/3591566453 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f205c162820 msgr2=0x7f205c14af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:09.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.507+0000 7f205a59c640 1 --2- 172.21.15.5:0/3591566453 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f205c162820 0x7f205c14af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:09.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.507+0000 7f205a59c640 1 -- 172.21.15.5:0/3591566453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f205c06d3d0 msgr2=0x7f205c0c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:09.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.507+0000 7f205a59c640 1 --2- 172.21.15.5:0/3591566453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f205c06d3d0 0x7f205c0c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:09.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.507+0000 7f205a59c640 1 -- 172.21.15.5:0/3591566453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f205c0cb000 con 0x7f205c0b8190 2024-05-20T18:28:09.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.507+0000 7f205ad9d640 1 --2- 172.21.15.5:0/3591566453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f205c06d3d0 0x7f205c0c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:09.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.507+0000 7f205a59c640 1 --2- 172.21.15.5:0/3591566453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f205c0b8190 0x7f205c0c8ac0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f205000e8a0 tx=0x7f205000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:09.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.507+0000 7f20437fe640 1 -- 172.21.15.5:0/3591566453 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2050017070 con 0x7f205c0b8190 2024-05-20T18:28:09.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.507+0000 7f20437fe640 1 -- 172.21.15.5:0/3591566453 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f205000c8e0 con 0x7f205c0b8190 2024-05-20T18:28:09.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.507+0000 7f20437fe640 1 -- 172.21.15.5:0/3591566453 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205000ccd0 con 0x7f205c0b8190 2024-05-20T18:28:09.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.507+0000 7f2061244640 1 -- 172.21.15.5:0/3591566453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f205c0cb2e0 con 0x7f205c0b8190 2024-05-20T18:28:09.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.507+0000 7f2061244640 1 -- 172.21.15.5:0/3591566453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f205c14b480 con 0x7f205c0b8190 2024-05-20T18:28:09.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.508+0000 7f2061244640 1 -- 172.21.15.5:0/3591566453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f205c075b00 con 0x7f205c0b8190 2024-05-20T18:28:09.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.509+0000 7f20437fe640 1 -- 172.21.15.5:0/3591566453 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f205001a070 con 0x7f205c0b8190 2024-05-20T18:28:09.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.509+0000 7f20437fe640 1 --2- 172.21.15.5:0/3591566453 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f204806be30 0x7f204806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:09.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.509+0000 7f20437fe640 1 -- 172.21.15.5:0/3591566453 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7f205008b530 con 0x7f205c0b8190 2024-05-20T18:28:09.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.509+0000 7f205ad9d640 1 --2- 172.21.15.5:0/3591566453 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f204806be30 0x7f204806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:09.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.510+0000 7f205ad9d640 1 --2- 172.21.15.5:0/3591566453 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f204806be30 0x7f204806e340 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f204c0012e0 tx=0x7f204c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:09.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.514+0000 7f20437fe640 1 -- 172.21.15.5:0/3591566453 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2050059a40 con 0x7f205c0b8190 2024-05-20T18:28:09.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.695+0000 7f2061244640 1 -- 172.21.15.5:0/3591566453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f205c06e1c0 con 0x7f205c0b8190 2024-05-20T18:28:09.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.696+0000 7f20437fe640 1 -- 172.21.15.5:0/3591566453 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f2050059340 con 0x7f205c0b8190 2024-05-20T18:28:09.695 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:28:09.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.698+0000 7f2061244640 1 -- 172.21.15.5:0/3591566453 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f204806be30 msgr2=0x7f204806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:09.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.698+0000 7f2061244640 1 --2- 172.21.15.5:0/3591566453 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f204806be30 0x7f204806e340 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f204c0012e0 tx=0x7f204c009040 comp rx=0 tx=0).stop 2024-05-20T18:28:09.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.698+0000 7f2061244640 1 -- 172.21.15.5:0/3591566453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f205c0b8190 msgr2=0x7f205c0c8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:09.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.698+0000 7f2061244640 1 --2- 172.21.15.5:0/3591566453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f205c0b8190 0x7f205c0c8ac0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f205000e8a0 tx=0x7f205000ed60 comp rx=0 tx=0).stop 2024-05-20T18:28:09.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.698+0000 7f2061244640 1 -- 172.21.15.5:0/3591566453 shutdown_connections 2024-05-20T18:28:09.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.698+0000 7f2061244640 1 --2- 172.21.15.5:0/3591566453 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f204806be30 0x7f204806e340 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:09.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.698+0000 7f2061244640 1 --2- 172.21.15.5:0/3591566453 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f205c162820 0x7f205c14af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:09.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.698+0000 7f2061244640 1 --2- 172.21.15.5:0/3591566453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f205c0b8190 0x7f205c0c8ac0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:09.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.698+0000 7f2061244640 1 --2- 172.21.15.5:0/3591566453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f205c06d3d0 0x7f205c0c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:09.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.698+0000 7f2061244640 1 -- 172.21.15.5:0/3591566453 >> 172.21.15.5:0/3591566453 conn(0x7f205c0d4bd0 msgr2=0x7f205c0ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:09.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.698+0000 7f2061244640 1 -- 172.21.15.5:0/3591566453 shutdown_connections 2024-05-20T18:28:09.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.698+0000 7f2061244640 1 -- 172.21.15.5:0/3591566453 wait complete. 2024-05-20T18:28:09.708 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:28:09.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.826+0000 7f294f667640 1 Processor -- start 2024-05-20T18:28:09.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.827+0000 7f294f667640 1 -- start start 2024-05-20T18:28:09.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.828+0000 7f294f667640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f29480afad0 0x7f29480b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:09.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.828+0000 7f294f667640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29480afee0 0x7f29480b5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:09.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.828+0000 7f294f667640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29480b3aa0 con 0x7f294815ca50 2024-05-20T18:28:09.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.828+0000 7f294f667640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f29480b3850 con 0x7f29480afad0 2024-05-20T18:28:09.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.828+0000 7f294f667640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f29480703e0 con 0x7f29480afee0 2024-05-20T18:28:09.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.829+0000 7f294dc04640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f294815ca50 0x7f294815ce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:44920/0 (socket says 172.21.15.5:44920) 2024-05-20T18:28:09.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.829+0000 7f294dc04640 1 -- 172.21.15.5:0/2587326357 learned_addr learned my addr 172.21.15.5:0/2587326357 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:09.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.829+0000 7f294d403640 1 --2- 172.21.15.5:0/2587326357 >> v2:172.21.15.63:3300/0 conn(0x7f29480afad0 0x7f29480b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:09.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.829+0000 7f294cc02640 1 --2- 172.21.15.5:0/2587326357 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29480afee0 0x7f29480b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:09.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.829+0000 7f2947fff640 1 -- 172.21.15.5:0/2587326357 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2173582640 0 0) 0x7f29480b3aa0 con 0x7f294815ca50 2024-05-20T18:28:09.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.829+0000 7f2947fff640 1 -- 172.21.15.5:0/2587326357 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2930003520 con 0x7f294815ca50 2024-05-20T18:28:09.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.829+0000 7f294cc02640 1 -- 172.21.15.5:0/2587326357 >> v2:172.21.15.63:3300/0 conn(0x7f29480afad0 msgr2=0x7f29480b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:09.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.829+0000 7f2947fff640 1 -- 172.21.15.5:0/2587326357 <== mon.1 v1:172.21.15.63:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 56541788 0 0) 0x7f2930003520 con 0x7f294815ca50 2024-05-20T18:28:09.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.829+0000 7f294cc02640 1 --2- 172.21.15.5:0/2587326357 >> v2:172.21.15.63:3300/0 conn(0x7f29480afad0 0x7f29480b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:09.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.829+0000 7f294cc02640 1 -- 172.21.15.5:0/2587326357 >> v1:172.21.15.63:6789/0 conn(0x7f294815ca50 legacy=0x7f294815ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:09.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.829+0000 7f294cc02640 1 -- 172.21.15.5:0/2587326357 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29480b3d80 con 0x7f29480afee0 2024-05-20T18:28:09.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.829+0000 7f294d403640 1 --2- 172.21.15.5:0/2587326357 >> v2:172.21.15.63:3300/0 conn(0x7f29480afad0 0x7f29480b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:28:09.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.830+0000 7f294cc02640 1 --2- 172.21.15.5:0/2587326357 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29480afee0 0x7f29480b5720 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f293c0097a0 tx=0x7f293c02a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a756926e2bf756a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:09.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.830+0000 7f2947fff640 1 -- 172.21.15.5:0/2587326357 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c037070 con 0x7f29480afee0 2024-05-20T18:28:09.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.830+0000 7f2947fff640 1 -- 172.21.15.5:0/2587326357 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f293c02ab10 con 0x7f29480afee0 2024-05-20T18:28:09.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.830+0000 7f2947fff640 1 -- 172.21.15.5:0/2587326357 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29400019f0 con 0x7f29480afee0 2024-05-20T18:28:09.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.831+0000 7f294f667640 1 -- 172.21.15.5:0/2587326357 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29480afee0 msgr2=0x7f29480b5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:09.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.831+0000 7f294f667640 1 --2- 172.21.15.5:0/2587326357 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29480afee0 0x7f29480b5720 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f293c0097a0 tx=0x7f293c02a080 comp rx=0 tx=0).stop 2024-05-20T18:28:09.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.831+0000 7f294f667640 1 -- 172.21.15.5:0/2587326357 shutdown_connections 2024-05-20T18:28:09.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.831+0000 7f294f667640 1 --2- 172.21.15.5:0/2587326357 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29480afee0 0x7f29480b5720 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:09.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.831+0000 7f294f667640 1 --2- 172.21.15.5:0/2587326357 >> v2:172.21.15.63:3300/0 conn(0x7f29480afad0 0x7f29480b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:09.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.831+0000 7f294f667640 1 -- 172.21.15.5:0/2587326357 >> 172.21.15.5:0/2587326357 conn(0x7f29480cd2a0 msgr2=0x7f29480cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:09.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.831+0000 7f294f667640 1 -- 172.21.15.5:0/2587326357 shutdown_connections 2024-05-20T18:28:09.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.831+0000 7f294f667640 1 -- 172.21.15.5:0/2587326357 wait complete. 2024-05-20T18:28:09.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.833+0000 7f294f667640 1 Processor -- start 2024-05-20T18:28:09.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.833+0000 7f294f667640 1 -- start start 2024-05-20T18:28:09.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.834+0000 7f294f667640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29480afad0 0x7f29481f75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:09.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.834+0000 7f294f667640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29480afee0 0x7f294814b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:09.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.834+0000 7f294f667640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f294815ca50 0x7f29481e7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:09.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.834+0000 7f294f667640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f29480b4860 con 0x7f29480afad0 2024-05-20T18:28:09.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.834+0000 7f294f667640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f29480b4340 con 0x7f294815ca50 2024-05-20T18:28:09.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.834+0000 7f294f667640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f29481f5380 con 0x7f29480afee0 2024-05-20T18:28:09.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.834+0000 7f294d403640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29480afad0 0x7f29481f75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:09.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.835+0000 7f294d403640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29480afad0 0x7f29481f75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46198/0 (socket says 172.21.15.5:46198) 2024-05-20T18:28:09.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.835+0000 7f294d403640 1 -- 172.21.15.5:0/152588658 learned_addr learned my addr 172.21.15.5:0/152588658 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:09.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.835+0000 7f294dc04640 1 --2- 172.21.15.5:0/152588658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f294815ca50 0x7f29481e7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:09.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.835+0000 7f294cc02640 1 --2- 172.21.15.5:0/152588658 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29480afee0 0x7f294814b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:09.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.835+0000 7f294d403640 1 -- 172.21.15.5:0/152588658 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29480afee0 msgr2=0x7f294814b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:09.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.835+0000 7f294d403640 1 --2- 172.21.15.5:0/152588658 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29480afee0 0x7f294814b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:09.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.835+0000 7f294d403640 1 -- 172.21.15.5:0/152588658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f294815ca50 msgr2=0x7f29481e7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:09.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.835+0000 7f294d403640 1 --2- 172.21.15.5:0/152588658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f294815ca50 0x7f29481e7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:09.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.835+0000 7f294d403640 1 -- 172.21.15.5:0/152588658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29481f4390 con 0x7f29480afad0 2024-05-20T18:28:09.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.835+0000 7f294d403640 1 --2- 172.21.15.5:0/152588658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29480afad0 0x7f29481f75a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f293800e910 tx=0x7f293800edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:09.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.835+0000 7f2945ffb640 1 -- 172.21.15.5:0/152588658 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2938017070 con 0x7f29480afad0 2024-05-20T18:28:09.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.835+0000 7f2945ffb640 1 -- 172.21.15.5:0/152588658 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f293800ca00 con 0x7f29480afad0 2024-05-20T18:28:09.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.835+0000 7f294f667640 1 -- 172.21.15.5:0/152588658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29481f3e10 con 0x7f29480afad0 2024-05-20T18:28:09.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.835+0000 7f2945ffb640 1 -- 172.21.15.5:0/152588658 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293800cdb0 con 0x7f29480afad0 2024-05-20T18:28:09.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.836+0000 7f294f667640 1 -- 172.21.15.5:0/152588658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29481f4100 con 0x7f29480afad0 2024-05-20T18:28:09.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.837+0000 7f294f667640 1 -- 172.21.15.5:0/152588658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2948075b00 con 0x7f29480afad0 2024-05-20T18:28:09.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.837+0000 7f2945ffb640 1 -- 172.21.15.5:0/152588658 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f293801a070 con 0x7f29480afad0 2024-05-20T18:28:09.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.838+0000 7f2945ffb640 1 --2- 172.21.15.5:0/152588658 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f293406be30 0x7f293406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:09.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.838+0000 7f2945ffb640 1 -- 172.21.15.5:0/152588658 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7f293808b250 con 0x7f29480afad0 2024-05-20T18:28:09.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.838+0000 7f294cc02640 1 --2- 172.21.15.5:0/152588658 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f293406be30 0x7f293406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:09.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.838+0000 7f294cc02640 1 --2- 172.21.15.5:0/152588658 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f293406be30 0x7f293406e340 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f293c034000 tx=0x7f293c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:09.842+0000 7f2945ffb640 1 -- 172.21.15.5:0/152588658 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f29481f3e10 con 0x7f29480afad0 2024-05-20T18:28:10.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.024+0000 7f294f667640 1 -- 172.21.15.5:0/152588658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f29480b49e0 con 0x7f29480afad0 2024-05-20T18:28:10.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.024+0000 7f2945ffb640 1 -- 172.21.15.5:0/152588658 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f29380596d0 con 0x7f29480afad0 2024-05-20T18:28:10.023 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:28:10.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.026+0000 7f294f667640 1 -- 172.21.15.5:0/152588658 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f293406be30 msgr2=0x7f293406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:10.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.026+0000 7f294f667640 1 --2- 172.21.15.5:0/152588658 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f293406be30 0x7f293406e340 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f293c034000 tx=0x7f293c02e000 comp rx=0 tx=0).stop 2024-05-20T18:28:10.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.026+0000 7f294f667640 1 -- 172.21.15.5:0/152588658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29480afad0 msgr2=0x7f29481f75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:10.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.026+0000 7f294f667640 1 --2- 172.21.15.5:0/152588658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29480afad0 0x7f29481f75a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f293800e910 tx=0x7f293800edd0 comp rx=0 tx=0).stop 2024-05-20T18:28:10.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.027+0000 7f294f667640 1 -- 172.21.15.5:0/152588658 shutdown_connections 2024-05-20T18:28:10.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.027+0000 7f294f667640 1 --2- 172.21.15.5:0/152588658 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f293406be30 0x7f293406e340 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:10.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.027+0000 7f294f667640 1 --2- 172.21.15.5:0/152588658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f294815ca50 0x7f29481e7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:10.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.027+0000 7f294f667640 1 --2- 172.21.15.5:0/152588658 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29480afee0 0x7f294814b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:10.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.027+0000 7f294f667640 1 --2- 172.21.15.5:0/152588658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29480afad0 0x7f29481f75a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:10.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.027+0000 7f294f667640 1 -- 172.21.15.5:0/152588658 >> 172.21.15.5:0/152588658 conn(0x7f29480cd2a0 msgr2=0x7f2948157e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:10.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.027+0000 7f294f667640 1 -- 172.21.15.5:0/152588658 shutdown_connections 2024-05-20T18:28:10.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.027+0000 7f294f667640 1 -- 172.21.15.5:0/152588658 wait complete. 2024-05-20T18:28:10.037 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:28:10.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.156+0000 7fbb7e921640 1 Processor -- start 2024-05-20T18:28:10.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.156+0000 7fbb7e921640 1 -- start start 2024-05-20T18:28:10.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.158+0000 7fbb7e921640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbb7806d3d0 0x7fbb7806d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:10.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.158+0000 7fbb7e921640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb780b8190 0x7fbb780b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:10.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.158+0000 7fbb7e921640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb7806dd20 con 0x7fbb78162820 2024-05-20T18:28:10.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.158+0000 7fbb7e921640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fbb780d0630 con 0x7fbb7806d3d0 2024-05-20T18:28:10.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.158+0000 7fbb7e921640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbb780d39c0 con 0x7fbb780b8190 2024-05-20T18:28:10.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.158+0000 7fbb777fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbb7806d3d0 0x7fbb7806d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:10.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.158+0000 7fbb777fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbb7806d3d0 0x7fbb7806d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46212/0 (socket says 172.21.15.5:46212) 2024-05-20T18:28:10.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.158+0000 7fbb777fe640 1 -- 172.21.15.5:0/1201027787 learned_addr learned my addr 172.21.15.5:0/1201027787 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:10.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.158+0000 7fbb7cebe640 1 --2- 172.21.15.5:0/1201027787 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb780b8190 0x7fbb780b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:10.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.158+0000 7fbb777fe640 1 -- 172.21.15.5:0/1201027787 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb780b8190 msgr2=0x7fbb780b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:10.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.158+0000 7fbb777fe640 1 --2- 172.21.15.5:0/1201027787 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb780b8190 0x7fbb780b85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:10.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.158+0000 7fbb777fe640 1 -- 172.21.15.5:0/1201027787 >> v1:172.21.15.63:6789/0 conn(0x7fbb78162820 legacy=0x7fbb780b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:10.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.158+0000 7fbb777fe640 1 -- 172.21.15.5:0/1201027787 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb780d3ca0 con 0x7fbb7806d3d0 2024-05-20T18:28:10.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.159+0000 7fbb777fe640 1 --2- 172.21.15.5:0/1201027787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb7806d3d0 0x7fbb7806d7e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009870 tx=0x7fbb6c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47608a7108fc22a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:10.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.159+0000 7fbb76ffd640 1 -- 172.21.15.5:0/1201027787 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c037070 con 0x7fbb7806d3d0 2024-05-20T18:28:10.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.159+0000 7fbb76ffd640 1 -- 172.21.15.5:0/1201027787 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb6c032440 con 0x7fbb7806d3d0 2024-05-20T18:28:10.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.159+0000 7fbb76ffd640 1 -- 172.21.15.5:0/1201027787 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c032810 con 0x7fbb7806d3d0 2024-05-20T18:28:10.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.160+0000 7fbb7e921640 1 -- 172.21.15.5:0/1201027787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb7806d3d0 msgr2=0x7fbb7806d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:10.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.160+0000 7fbb7e921640 1 --2- 172.21.15.5:0/1201027787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb7806d3d0 0x7fbb7806d7e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009870 tx=0x7fbb6c02a210 comp rx=0 tx=0).stop 2024-05-20T18:28:10.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.160+0000 7fbb7e921640 1 -- 172.21.15.5:0/1201027787 shutdown_connections 2024-05-20T18:28:10.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.160+0000 7fbb7e921640 1 --2- 172.21.15.5:0/1201027787 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb780b8190 0x7fbb780b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:10.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.160+0000 7fbb7e921640 1 --2- 172.21.15.5:0/1201027787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb7806d3d0 0x7fbb7806d7e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:10.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.160+0000 7fbb7e921640 1 -- 172.21.15.5:0/1201027787 >> 172.21.15.5:0/1201027787 conn(0x7fbb780d4bd0 msgr2=0x7fbb780d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:10.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.160+0000 7fbb7e921640 1 -- 172.21.15.5:0/1201027787 shutdown_connections 2024-05-20T18:28:10.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.160+0000 7fbb7e921640 1 -- 172.21.15.5:0/1201027787 wait complete. 2024-05-20T18:28:10.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.162+0000 7fbb7e921640 1 Processor -- start 2024-05-20T18:28:10.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.162+0000 7fbb7e921640 1 -- start start 2024-05-20T18:28:10.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.163+0000 7fbb7e921640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb7806d3d0 0x7fbb780cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:10.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.163+0000 7fbb7e921640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb780b8190 0x7fbb7814dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:10.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.163+0000 7fbb7e921640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb78162820 0x7fbb780b57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:10.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.163+0000 7fbb7e921640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbb780d0300 con 0x7fbb7806d3d0 2024-05-20T18:28:10.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.163+0000 7fbb7e921640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbb780d4260 con 0x7fbb78162820 2024-05-20T18:28:10.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.163+0000 7fbb7e921640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbb780b4050 con 0x7fbb780b8190 2024-05-20T18:28:10.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.163+0000 7fbb77fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb7806d3d0 0x7fbb780cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:10.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.163+0000 7fbb77fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb7806d3d0 0x7fbb780cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46220/0 (socket says 172.21.15.5:46220) 2024-05-20T18:28:10.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.163+0000 7fbb77fff640 1 -- 172.21.15.5:0/3542626892 learned_addr learned my addr 172.21.15.5:0/3542626892 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.163+0000 7fbb7cebe640 1 --2- 172.21.15.5:0/3542626892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb78162820 0x7fbb780b57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.163+0000 7fbb777fe640 1 --2- 172.21.15.5:0/3542626892 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb780b8190 0x7fbb7814dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.164+0000 7fbb77fff640 1 -- 172.21.15.5:0/3542626892 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb780b8190 msgr2=0x7fbb7814dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.164+0000 7fbb77fff640 1 --2- 172.21.15.5:0/3542626892 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb780b8190 0x7fbb7814dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.164+0000 7fbb77fff640 1 -- 172.21.15.5:0/3542626892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb78162820 msgr2=0x7fbb780b57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.164+0000 7fbb77fff640 1 --2- 172.21.15.5:0/3542626892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb78162820 0x7fbb780b57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.164+0000 7fbb77fff640 1 -- 172.21.15.5:0/3542626892 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb7814e520 con 0x7fbb7806d3d0 2024-05-20T18:28:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.164+0000 7fbb77fff640 1 --2- 172.21.15.5:0/3542626892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb7806d3d0 0x7fbb780cc640 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800b420 tx=0x7fbb6800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.164+0000 7fbb74ff9640 1 -- 172.21.15.5:0/3542626892 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb68011830 con 0x7fbb7806d3d0 2024-05-20T18:28:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.164+0000 7fbb74ff9640 1 -- 172.21.15.5:0/3542626892 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb68016070 con 0x7fbb7806d3d0 2024-05-20T18:28:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.164+0000 7fbb7e921640 1 -- 172.21.15.5:0/3542626892 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb780ce100 con 0x7fbb7806d3d0 2024-05-20T18:28:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.164+0000 7fbb74ff9640 1 -- 172.21.15.5:0/3542626892 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb68011c10 con 0x7fbb7806d3d0 2024-05-20T18:28:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.164+0000 7fbb7e921640 1 -- 172.21.15.5:0/3542626892 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb780ce6c0 con 0x7fbb7806d3d0 2024-05-20T18:28:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.165+0000 7fbb7e921640 1 -- 172.21.15.5:0/3542626892 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb78075b00 con 0x7fbb7806d3d0 2024-05-20T18:28:10.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.165+0000 7fbb74ff9640 1 -- 172.21.15.5:0/3542626892 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbb6801b460 con 0x7fbb7806d3d0 2024-05-20T18:28:10.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.166+0000 7fbb74ff9640 1 --2- 172.21.15.5:0/3542626892 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbb6406be30 0x7fbb6406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:10.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.166+0000 7fbb74ff9640 1 -- 172.21.15.5:0/3542626892 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7fbb6808ad20 con 0x7fbb7806d3d0 2024-05-20T18:28:10.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.166+0000 7fbb777fe640 1 --2- 172.21.15.5:0/3542626892 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbb6406be30 0x7fbb6406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:10.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.167+0000 7fbb777fe640 1 --2- 172.21.15.5:0/3542626892 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbb6406be30 0x7fbb6406e340 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c034000 tx=0x7fbb6c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:10.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.171+0000 7fbb74ff9640 1 -- 172.21.15.5:0/3542626892 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbb68020020 con 0x7fbb7806d3d0 2024-05-20T18:28:10.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.358+0000 7fbb7e921640 1 -- 172.21.15.5:0/3542626892 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbb780676e0 con 0x7fbb7806d3d0 2024-05-20T18:28:10.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.358+0000 7fbb74ff9640 1 -- 172.21.15.5:0/3542626892 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7fbb680592c0 con 0x7fbb7806d3d0 2024-05-20T18:28:10.357 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:28:10.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.360+0000 7fbb7e921640 1 -- 172.21.15.5:0/3542626892 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbb6406be30 msgr2=0x7fbb6406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:10.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.360+0000 7fbb7e921640 1 --2- 172.21.15.5:0/3542626892 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbb6406be30 0x7fbb6406e340 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c034000 tx=0x7fbb6c02e000 comp rx=0 tx=0).stop 2024-05-20T18:28:10.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.360+0000 7fbb7e921640 1 -- 172.21.15.5:0/3542626892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb7806d3d0 msgr2=0x7fbb780cc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:10.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.360+0000 7fbb7e921640 1 --2- 172.21.15.5:0/3542626892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb7806d3d0 0x7fbb780cc640 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800b420 tx=0x7fbb6800b8e0 comp rx=0 tx=0).stop 2024-05-20T18:28:10.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.360+0000 7fbb7e921640 1 -- 172.21.15.5:0/3542626892 shutdown_connections 2024-05-20T18:28:10.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.360+0000 7fbb7e921640 1 --2- 172.21.15.5:0/3542626892 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbb6406be30 0x7fbb6406e340 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:10.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.360+0000 7fbb7e921640 1 --2- 172.21.15.5:0/3542626892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb78162820 0x7fbb780b57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:10.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.360+0000 7fbb7e921640 1 --2- 172.21.15.5:0/3542626892 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb780b8190 0x7fbb7814dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:10.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.360+0000 7fbb7e921640 1 --2- 172.21.15.5:0/3542626892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb7806d3d0 0x7fbb780cc640 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:10.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.360+0000 7fbb7e921640 1 -- 172.21.15.5:0/3542626892 >> 172.21.15.5:0/3542626892 conn(0x7fbb780d4bd0 msgr2=0x7fbb78160eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:10.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.361+0000 7fbb7e921640 1 -- 172.21.15.5:0/3542626892 shutdown_connections 2024-05-20T18:28:10.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.361+0000 7fbb7e921640 1 -- 172.21.15.5:0/3542626892 wait complete. 2024-05-20T18:28:10.371 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-20T18:28:10.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.491+0000 7f88bed31640 1 Processor -- start 2024-05-20T18:28:10.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.491+0000 7f88bed31640 1 -- start start 2024-05-20T18:28:10.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.492+0000 7f88bed31640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f88b8162820 0x7f88b80b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:10.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.492+0000 7f88bed31640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88b806dae0 0x7f88b80b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:10.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.492+0000 7f88bed31640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88b80d0630 con 0x7f88b80b3e80 2024-05-20T18:28:10.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.492+0000 7f88bed31640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f88b80d39c0 con 0x7f88b8162820 2024-05-20T18:28:10.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.492+0000 7f88bed31640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f88b80d3ca0 con 0x7f88b806dae0 2024-05-20T18:28:10.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.493+0000 7f88affff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f88b80b3e80 0x7f88b806d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35696/0 (socket says 172.21.15.5:35696) 2024-05-20T18:28:10.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.493+0000 7f88affff640 1 -- 172.21.15.5:0/3909604345 learned_addr learned my addr 172.21.15.5:0/3909604345 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:10.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.493+0000 7f88bcacd640 1 --2- 172.21.15.5:0/3909604345 >> v2:172.21.15.63:3300/0 conn(0x7f88b8162820 0x7f88b80b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:10.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.493+0000 7f88bd2ce640 1 --2- 172.21.15.5:0/3909604345 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88b806dae0 0x7f88b80b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:10.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.493+0000 7f88af7fe640 1 -- 172.21.15.5:0/3909604345 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1482574345 0 0) 0x7f88b80d0630 con 0x7f88b80b3e80 2024-05-20T18:28:10.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.493+0000 7f88af7fe640 1 -- 172.21.15.5:0/3909604345 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88a0003520 con 0x7f88b80b3e80 2024-05-20T18:28:10.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.493+0000 7f88bcacd640 1 -- 172.21.15.5:0/3909604345 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88b806dae0 msgr2=0x7f88b80b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:10.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.493+0000 7f88bcacd640 1 --2- 172.21.15.5:0/3909604345 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88b806dae0 0x7f88b80b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:10.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.493+0000 7f88bcacd640 1 -- 172.21.15.5:0/3909604345 >> v1:172.21.15.5:6789/0 conn(0x7f88b80b3e80 legacy=0x7f88b806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:10.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.493+0000 7f88bcacd640 1 -- 172.21.15.5:0/3909604345 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b80d3f80 con 0x7f88b8162820 2024-05-20T18:28:10.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.493+0000 7f88bd2ce640 1 --2- 172.21.15.5:0/3909604345 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88b806dae0 0x7f88b80b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:28:10.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.494+0000 7f88bcacd640 1 --2- 172.21.15.5:0/3909604345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88b8162820 0x7f88b80b3940 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f88b4009d90 tx=0x7f88b402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=378619202b9fa126 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:10.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.494+0000 7f88af7fe640 1 -- 172.21.15.5:0/3909604345 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b4007740 con 0x7f88b8162820 2024-05-20T18:28:10.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.494+0000 7f88af7fe640 1 -- 172.21.15.5:0/3909604345 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88b40078e0 con 0x7f88b8162820 2024-05-20T18:28:10.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.494+0000 7f88af7fe640 1 -- 172.21.15.5:0/3909604345 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b4007c90 con 0x7f88b8162820 2024-05-20T18:28:10.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.495+0000 7f88bed31640 1 -- 172.21.15.5:0/3909604345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88b8162820 msgr2=0x7f88b80b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:10.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.495+0000 7f88bed31640 1 --2- 172.21.15.5:0/3909604345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88b8162820 0x7f88b80b3940 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f88b4009d90 tx=0x7f88b402a660 comp rx=0 tx=0).stop 2024-05-20T18:28:10.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.495+0000 7f88bed31640 1 -- 172.21.15.5:0/3909604345 shutdown_connections 2024-05-20T18:28:10.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.495+0000 7f88bed31640 1 --2- 172.21.15.5:0/3909604345 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88b806dae0 0x7f88b80b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:10.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.495+0000 7f88bed31640 1 --2- 172.21.15.5:0/3909604345 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88b8162820 0x7f88b80b3940 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:10.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.495+0000 7f88bed31640 1 -- 172.21.15.5:0/3909604345 >> 172.21.15.5:0/3909604345 conn(0x7f88b80d4bd0 msgr2=0x7f88b80d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:10.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.495+0000 7f88bed31640 1 -- 172.21.15.5:0/3909604345 shutdown_connections 2024-05-20T18:28:10.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.495+0000 7f88bed31640 1 -- 172.21.15.5:0/3909604345 wait complete. 2024-05-20T18:28:10.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.497+0000 7f88bed31640 1 Processor -- start 2024-05-20T18:28:10.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.497+0000 7f88bed31640 1 -- start start 2024-05-20T18:28:10.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.498+0000 7f88bed31640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88b80b3e80 0x7f88b815aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:10.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.498+0000 7f88bed31640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88b8162820 0x7f88b80cb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:10.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.498+0000 7f88bed31640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88b81f4ee0 0x7f88b80b6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:10.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.498+0000 7f88bed31640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f88b80d4780 con 0x7f88b80b3e80 2024-05-20T18:28:10.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.498+0000 7f88bed31640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f88b80d0300 con 0x7f88b8162820 2024-05-20T18:28:10.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.498+0000 7f88bed31640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f88b80d4900 con 0x7f88b81f4ee0 2024-05-20T18:28:10.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.498+0000 7f88bcacd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88b80b3e80 0x7f88b815aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:10.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.498+0000 7f88bcacd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88b80b3e80 0x7f88b815aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46230/0 (socket says 172.21.15.5:46230) 2024-05-20T18:28:10.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.498+0000 7f88bcacd640 1 -- 172.21.15.5:0/1533269539 learned_addr learned my addr 172.21.15.5:0/1533269539 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:10.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.499+0000 7f88affff640 1 --2- 172.21.15.5:0/1533269539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88b8162820 0x7f88b80cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:10.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.499+0000 7f88bd2ce640 1 --2- 172.21.15.5:0/1533269539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88b81f4ee0 0x7f88b80b6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:10.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.499+0000 7f88bcacd640 1 -- 172.21.15.5:0/1533269539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88b81f4ee0 msgr2=0x7f88b80b6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:10.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.499+0000 7f88bcacd640 1 --2- 172.21.15.5:0/1533269539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88b81f4ee0 0x7f88b80b6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:10.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.499+0000 7f88bcacd640 1 -- 172.21.15.5:0/1533269539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88b8162820 msgr2=0x7f88b80cb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:10.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.499+0000 7f88bcacd640 1 --2- 172.21.15.5:0/1533269539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88b8162820 0x7f88b80cb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:10.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.499+0000 7f88bcacd640 1 -- 172.21.15.5:0/1533269539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b80cb880 con 0x7f88b80b3e80 2024-05-20T18:28:10.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.499+0000 7f88bd2ce640 1 --2- 172.21.15.5:0/1533269539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88b81f4ee0 0x7f88b80b6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:10.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.499+0000 7f88affff640 1 --2- 172.21.15.5:0/1533269539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88b8162820 0x7f88b80cb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:10.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.499+0000 7f88bcacd640 1 --2- 172.21.15.5:0/1533269539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88b80b3e80 0x7f88b815aad0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f88b402ac50 tx=0x7f88b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:10.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.499+0000 7f88ad7fa640 1 -- 172.21.15.5:0/1533269539 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b403f070 con 0x7f88b80b3e80 2024-05-20T18:28:10.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.499+0000 7f88ad7fa640 1 -- 172.21.15.5:0/1533269539 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88b4031c40 con 0x7f88b80b3e80 2024-05-20T18:28:10.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.499+0000 7f88bed31640 1 -- 172.21.15.5:0/1533269539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b80b5ce0 con 0x7f88b80b3e80 2024-05-20T18:28:10.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.499+0000 7f88ad7fa640 1 -- 172.21.15.5:0/1533269539 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b4009450 con 0x7f88b80b3e80 2024-05-20T18:28:10.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.499+0000 7f88bed31640 1 -- 172.21.15.5:0/1533269539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b80b62a0 con 0x7f88b80b3e80 2024-05-20T18:28:10.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.500+0000 7f88bed31640 1 -- 172.21.15.5:0/1533269539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88b8075b00 con 0x7f88b80b3e80 2024-05-20T18:28:10.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.501+0000 7f88ad7fa640 1 -- 172.21.15.5:0/1533269539 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f88b403b410 con 0x7f88b80b3e80 2024-05-20T18:28:10.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.501+0000 7f88ad7fa640 1 --2- 172.21.15.5:0/1533269539 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f88a406be30 0x7f88a406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:10.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.501+0000 7f88ad7fa640 1 -- 172.21.15.5:0/1533269539 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7f88b40acac0 con 0x7f88b80b3e80 2024-05-20T18:28:10.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.501+0000 7f88affff640 1 --2- 172.21.15.5:0/1533269539 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f88a406be30 0x7f88a406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:10.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.502+0000 7f88affff640 1 --2- 172.21.15.5:0/1533269539 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f88a406be30 0x7f88a406e340 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f88b815f900 tx=0x7f88a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:10.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.506+0000 7f88ad7fa640 1 -- 172.21.15.5:0/1533269539 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f88b407aa10 con 0x7f88b80b3e80 2024-05-20T18:28:10.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.692+0000 7f88bed31640 1 -- 172.21.15.5:0/1533269539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f88b8161230 con 0x7f88b80b3e80 2024-05-20T18:28:10.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.693+0000 7f88ad7fa640 1 -- 172.21.15.5:0/1533269539 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f88b407ed10 con 0x7f88b80b3e80 2024-05-20T18:28:10.694 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:28:10.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.695+0000 7f88bed31640 1 -- 172.21.15.5:0/1533269539 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f88a406be30 msgr2=0x7f88a406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:10.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.695+0000 7f88bed31640 1 --2- 172.21.15.5:0/1533269539 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f88a406be30 0x7f88a406e340 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f88b815f900 tx=0x7f88a8009040 comp rx=0 tx=0).stop 2024-05-20T18:28:10.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.695+0000 7f88bed31640 1 -- 172.21.15.5:0/1533269539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88b80b3e80 msgr2=0x7f88b815aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:10.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.695+0000 7f88bed31640 1 --2- 172.21.15.5:0/1533269539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88b80b3e80 0x7f88b815aad0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f88b402ac50 tx=0x7f88b4004a40 comp rx=0 tx=0).stop 2024-05-20T18:28:10.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.695+0000 7f88bed31640 1 -- 172.21.15.5:0/1533269539 shutdown_connections 2024-05-20T18:28:10.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.695+0000 7f88bed31640 1 --2- 172.21.15.5:0/1533269539 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f88a406be30 0x7f88a406e340 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:10.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.695+0000 7f88bed31640 1 --2- 172.21.15.5:0/1533269539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88b81f4ee0 0x7f88b80b6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:10.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.695+0000 7f88bed31640 1 --2- 172.21.15.5:0/1533269539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88b8162820 0x7f88b80cb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:10.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.695+0000 7f88bed31640 1 --2- 172.21.15.5:0/1533269539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88b80b3e80 0x7f88b815aad0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:10.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.695+0000 7f88bed31640 1 -- 172.21.15.5:0/1533269539 >> 172.21.15.5:0/1533269539 conn(0x7f88b80d4bd0 msgr2=0x7f88b8161a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:10.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.695+0000 7f88bed31640 1 -- 172.21.15.5:0/1533269539 shutdown_connections 2024-05-20T18:28:10.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.695+0000 7f88bed31640 1 -- 172.21.15.5:0/1533269539 wait complete. 2024-05-20T18:28:10.705 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:28:10.791 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:28:10.793 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:28:10.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.822+0000 7fd20d196640 1 Processor -- start 2024-05-20T18:28:10.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.823+0000 7fd20d196640 1 -- start start 2024-05-20T18:28:10.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.824+0000 7fd20d196640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd2080b25d0 0x7fd2080b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:10.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.824+0000 7fd20d196640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd20815d870 con 0x7fd20806c530 2024-05-20T18:28:10.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.824+0000 7fd20d196640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd20815daf0 con 0x7fd20815d240 2024-05-20T18:28:10.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.824+0000 7fd20d196640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd20815d650 con 0x7fd2080b25d0 2024-05-20T18:28:10.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.824+0000 7fd20759e640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd20806c530 0x7fd20806c940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35700/0 (socket says 172.21.15.5:35700) 2024-05-20T18:28:10.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.824+0000 7fd20759e640 1 -- 172.21.15.5:0/1952463905 learned_addr learned my addr 172.21.15.5:0/1952463905 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:10.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.824+0000 7fd20659c640 1 --2- 172.21.15.5:0/1952463905 >> v2:172.21.15.63:3300/0 conn(0x7fd2080b25d0 0x7fd2080b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:10.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.824+0000 7fd205d9b640 1 -- 172.21.15.5:0/1952463905 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3133031265 0 0) 0x7fd200001490 con 0x7fd20806c530 2024-05-20T18:28:10.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.824+0000 7fd205d9b640 1 -- 172.21.15.5:0/1952463905 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1f0003520 con 0x7fd20806c530 2024-05-20T18:28:10.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.825+0000 7fd20659c640 1 -- 172.21.15.5:0/1952463905 >> v1:172.21.15.63:6789/0 conn(0x7fd20815d240 legacy=0x7fd20815e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:10.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.825+0000 7fd20659c640 1 -- 172.21.15.5:0/1952463905 >> v1:172.21.15.5:6789/0 conn(0x7fd20806c530 legacy=0x7fd20806c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:10.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.825+0000 7fd20659c640 1 -- 172.21.15.5:0/1952463905 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd20815ddd0 con 0x7fd2080b25d0 2024-05-20T18:28:10.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.825+0000 7fd20659c640 1 --2- 172.21.15.5:0/1952463905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2080b25d0 0x7fd2080b0c60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc0097a0 tx=0x7fd1fc02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cc917c4194c869c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:10.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.826+0000 7fd205d9b640 1 -- 172.21.15.5:0/1952463905 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc037070 con 0x7fd2080b25d0 2024-05-20T18:28:10.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.826+0000 7fd205d9b640 1 -- 172.21.15.5:0/1952463905 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1fc032440 con 0x7fd2080b25d0 2024-05-20T18:28:10.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.826+0000 7fd205d9b640 1 -- 172.21.15.5:0/1952463905 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc032810 con 0x7fd2080b25d0 2024-05-20T18:28:10.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.826+0000 7fd20d196640 1 -- 172.21.15.5:0/1952463905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2080b25d0 msgr2=0x7fd2080b0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:10.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.826+0000 7fd20d196640 1 --2- 172.21.15.5:0/1952463905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2080b25d0 0x7fd2080b0c60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc0097a0 tx=0x7fd1fc02a230 comp rx=0 tx=0).stop 2024-05-20T18:28:10.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.826+0000 7fd20d196640 1 -- 172.21.15.5:0/1952463905 shutdown_connections 2024-05-20T18:28:10.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.826+0000 7fd20d196640 1 --2- 172.21.15.5:0/1952463905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2080b25d0 0x7fd2080b0c60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:10.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.826+0000 7fd20d196640 1 -- 172.21.15.5:0/1952463905 >> 172.21.15.5:0/1952463905 conn(0x7fd208072ed0 msgr2=0x7fd2080b6080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:10.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.826+0000 7fd20d196640 1 -- 172.21.15.5:0/1952463905 shutdown_connections 2024-05-20T18:28:10.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.826+0000 7fd20d196640 1 -- 172.21.15.5:0/1952463905 wait complete. 2024-05-20T18:28:10.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.828+0000 7fd20d196640 1 Processor -- start 2024-05-20T18:28:10.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.829+0000 7fd20d196640 1 -- start start 2024-05-20T18:28:10.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.830+0000 7fd20d196640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd20806c530 0x7fd2081f8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:10.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.830+0000 7fd20d196640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2080b25d0 0x7fd2081f9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:10.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.830+0000 7fd20d196640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd20815d240 0x7fd20809dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:10.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.830+0000 7fd20d196640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd2080b29e0 con 0x7fd20815d240 2024-05-20T18:28:10.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.830+0000 7fd20d196640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd2080b11a0 con 0x7fd20806c530 2024-05-20T18:28:10.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.830+0000 7fd20d196640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd20815ece0 con 0x7fd2080b25d0 2024-05-20T18:28:10.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.830+0000 7fd20759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd20815d240 0x7fd20809dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:10.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.830+0000 7fd20759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd20815d240 0x7fd20809dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46238/0 (socket says 172.21.15.5:46238) 2024-05-20T18:28:10.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.830+0000 7fd20759e640 1 -- 172.21.15.5:0/1551472101 learned_addr learned my addr 172.21.15.5:0/1551472101 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:10.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.830+0000 7fd206d9d640 1 --2- 172.21.15.5:0/1551472101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd20806c530 0x7fd2081f8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:10.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.830+0000 7fd20659c640 1 --2- 172.21.15.5:0/1551472101 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2080b25d0 0x7fd2081f9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:10.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.830+0000 7fd20759e640 1 -- 172.21.15.5:0/1551472101 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2080b25d0 msgr2=0x7fd2081f9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:10.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.830+0000 7fd20759e640 1 --2- 172.21.15.5:0/1551472101 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2080b25d0 0x7fd2081f9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:10.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.830+0000 7fd20759e640 1 -- 172.21.15.5:0/1551472101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd20806c530 msgr2=0x7fd2081f8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:10.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.830+0000 7fd20759e640 1 --2- 172.21.15.5:0/1551472101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd20806c530 0x7fd2081f8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:10.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.830+0000 7fd20759e640 1 -- 172.21.15.5:0/1551472101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2080cb6a0 con 0x7fd20815d240 2024-05-20T18:28:10.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.830+0000 7fd20759e640 1 --2- 172.21.15.5:0/1551472101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd20815d240 0x7fd20809dd50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd20000cad0 tx=0x7fd20000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:10.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.831+0000 7fd1ef7fe640 1 -- 172.21.15.5:0/1551472101 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd200017070 con 0x7fd20815d240 2024-05-20T18:28:10.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.831+0000 7fd1ef7fe640 1 -- 172.21.15.5:0/1551472101 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd20000ac30 con 0x7fd20815d240 2024-05-20T18:28:10.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.831+0000 7fd20d196640 1 -- 172.21.15.5:0/1551472101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2081f9ad0 con 0x7fd20815d240 2024-05-20T18:28:10.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.831+0000 7fd1ef7fe640 1 -- 172.21.15.5:0/1551472101 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd200012400 con 0x7fd20815d240 2024-05-20T18:28:10.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.831+0000 7fd20d196640 1 -- 172.21.15.5:0/1551472101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2081f9db0 con 0x7fd20815d240 2024-05-20T18:28:10.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.832+0000 7fd20d196640 1 -- 172.21.15.5:0/1551472101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd208075b00 con 0x7fd20815d240 2024-05-20T18:28:10.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.833+0000 7fd1ef7fe640 1 -- 172.21.15.5:0/1551472101 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd20001a070 con 0x7fd20815d240 2024-05-20T18:28:10.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.833+0000 7fd1ef7fe640 1 --2- 172.21.15.5:0/1551472101 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd1f406be60 0x7fd1f406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:10.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.833+0000 7fd1ef7fe640 1 -- 172.21.15.5:0/1551472101 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7fd20008af10 con 0x7fd20815d240 2024-05-20T18:28:10.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.833+0000 7fd206d9d640 1 --2- 172.21.15.5:0/1551472101 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd1f406be60 0x7fd1f406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:10.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.834+0000 7fd206d9d640 1 --2- 172.21.15.5:0/1551472101 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd1f406be60 0x7fd1f406e370 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd1f80016e0 tx=0x7fd1f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:10.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:10.838+0000 7fd1ef7fe640 1 -- 172.21.15.5:0/1551472101 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd20815d870 con 0x7fd20815d240 2024-05-20T18:28:11.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.060+0000 7fd20d196640 1 -- 172.21.15.5:0/1551472101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2080ca9d0 con 0x7fd20815d240 2024-05-20T18:28:11.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.061+0000 7fd1ef7fe640 1 -- 172.21.15.5:0/1551472101 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5168 (secure 0 0 0) 0x7fd200058e60 con 0x7fd20815d240 2024-05-20T18:28:11.059 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:11.059 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2982954341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2982954341},{"type":"v1","addr":"172.21.15.5:6836","nonce":2982954341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4032850001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":4032850001},{"type":"v1","addr":"172.21.15.5:6837","nonce":4032850001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6833/424580378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":424580378},{"type":"v1","addr":"172.21.15.63:6833","nonce":424580378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:27:07.164414+0000","modified":"2024-05-20T18:27:29.288681+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/3249423677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3249423677},{"type":"v1","addr":"172.21.15.63:6835","nonce":3249423677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-05-20T18:28:11.060 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 9 2024-05-20T18:28:11.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.063+0000 7fd20d196640 1 -- 172.21.15.5:0/1551472101 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd1f406be60 msgr2=0x7fd1f406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:11.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.063+0000 7fd20d196640 1 --2- 172.21.15.5:0/1551472101 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd1f406be60 0x7fd1f406e370 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd1f80016e0 tx=0x7fd1f8009040 comp rx=0 tx=0).stop 2024-05-20T18:28:11.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.063+0000 7fd20d196640 1 -- 172.21.15.5:0/1551472101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd20815d240 msgr2=0x7fd20809dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:11.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.063+0000 7fd20d196640 1 --2- 172.21.15.5:0/1551472101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd20815d240 0x7fd20809dd50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd20000cad0 tx=0x7fd20000cf90 comp rx=0 tx=0).stop 2024-05-20T18:28:11.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.063+0000 7fd20d196640 1 -- 172.21.15.5:0/1551472101 shutdown_connections 2024-05-20T18:28:11.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.063+0000 7fd20d196640 1 --2- 172.21.15.5:0/1551472101 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd1f406be60 0x7fd1f406e370 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:11.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.063+0000 7fd20d196640 1 --2- 172.21.15.5:0/1551472101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd20815d240 0x7fd20809dd50 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:11.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.063+0000 7fd20d196640 1 --2- 172.21.15.5:0/1551472101 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2080b25d0 0x7fd2081f9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:11.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.063+0000 7fd20d196640 1 --2- 172.21.15.5:0/1551472101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd20806c530 0x7fd2081f8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:11.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.063+0000 7fd20d196640 1 -- 172.21.15.5:0/1551472101 >> 172.21.15.5:0/1551472101 conn(0x7fd208072ed0 msgr2=0x7fd2080b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:11.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.063+0000 7fd20d196640 1 -- 172.21.15.5:0/1551472101 shutdown_connections 2024-05-20T18:28:11.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.063+0000 7fd20d196640 1 -- 172.21.15.5:0/1551472101 wait complete. 2024-05-20T18:28:11.073 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:28:11.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.188+0000 7f591cae9640 1 Processor -- start 2024-05-20T18:28:11.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.188+0000 7f591cae9640 1 -- start start 2024-05-20T18:28:11.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.190+0000 7f591cae9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5918162820 0x7f59180b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:11.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.190+0000 7f591cae9640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f591806dae0 0x7f59180b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:11.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.190+0000 7f591cae9640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59180d0630 con 0x7f59180b3e80 2024-05-20T18:28:11.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.190+0000 7f591cae9640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f59180d39c0 con 0x7f5918162820 2024-05-20T18:28:11.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.190+0000 7f591cae9640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f59180d3ca0 con 0x7f591806dae0 2024-05-20T18:28:11.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.190+0000 7f591659c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5918162820 0x7f59180b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:11.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.190+0000 7f591659c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5918162820 0x7f59180b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46248/0 (socket says 172.21.15.5:46248) 2024-05-20T18:28:11.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.190+0000 7f591659c640 1 -- 172.21.15.5:0/1515671924 learned_addr learned my addr 172.21.15.5:0/1515671924 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:11.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.190+0000 7f5916d9d640 1 --2- 172.21.15.5:0/1515671924 >> v2:172.21.15.63:3300/0 conn(0x7f591806dae0 0x7f59180b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:11.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.190+0000 7f591659c640 1 -- 172.21.15.5:0/1515671924 >> v2:172.21.15.63:3300/0 conn(0x7f591806dae0 msgr2=0x7f59180b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:11.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.190+0000 7f591659c640 1 --2- 172.21.15.5:0/1515671924 >> v2:172.21.15.63:3300/0 conn(0x7f591806dae0 0x7f59180b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:11.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.191+0000 7f591659c640 1 -- 172.21.15.5:0/1515671924 >> v1:172.21.15.63:6789/0 conn(0x7f59180b3e80 legacy=0x7f591806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:11.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.191+0000 7f591659c640 1 -- 172.21.15.5:0/1515671924 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59180d3f80 con 0x7f5918162820 2024-05-20T18:28:11.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.191+0000 7f591659c640 1 --2- 172.21.15.5:0/1515671924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5918162820 0x7f59180b3940 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f591808e070 tx=0x7f590802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa4c86da633adc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:11.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.191+0000 7f591559a640 1 -- 172.21.15.5:0/1515671924 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5908037070 con 0x7f5918162820 2024-05-20T18:28:11.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.191+0000 7f591559a640 1 -- 172.21.15.5:0/1515671924 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5908032440 con 0x7f5918162820 2024-05-20T18:28:11.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.191+0000 7f591559a640 1 -- 172.21.15.5:0/1515671924 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5908032810 con 0x7f5918162820 2024-05-20T18:28:11.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.192+0000 7f591cae9640 1 -- 172.21.15.5:0/1515671924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5918162820 msgr2=0x7f59180b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:11.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.192+0000 7f591cae9640 1 --2- 172.21.15.5:0/1515671924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5918162820 0x7f59180b3940 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f591808e070 tx=0x7f590802a230 comp rx=0 tx=0).stop 2024-05-20T18:28:11.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.192+0000 7f591cae9640 1 -- 172.21.15.5:0/1515671924 shutdown_connections 2024-05-20T18:28:11.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.192+0000 7f591cae9640 1 --2- 172.21.15.5:0/1515671924 >> v2:172.21.15.63:3300/0 conn(0x7f591806dae0 0x7f59180b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:11.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.192+0000 7f591cae9640 1 --2- 172.21.15.5:0/1515671924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5918162820 0x7f59180b3940 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:11.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.192+0000 7f591cae9640 1 -- 172.21.15.5:0/1515671924 >> 172.21.15.5:0/1515671924 conn(0x7f59180d4bd0 msgr2=0x7f59180d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:11.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.192+0000 7f591cae9640 1 -- 172.21.15.5:0/1515671924 shutdown_connections 2024-05-20T18:28:11.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.192+0000 7f591cae9640 1 -- 172.21.15.5:0/1515671924 wait complete. 2024-05-20T18:28:11.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.194+0000 7f591cae9640 1 Processor -- start 2024-05-20T18:28:11.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.194+0000 7f591cae9640 1 -- start start 2024-05-20T18:28:11.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.195+0000 7f591cae9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59180b3e80 0x7f5918150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:11.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.196+0000 7f591cae9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5918162820 0x7f59180cb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:11.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.196+0000 7f591cae9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59180cb780 0x7f59180c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:11.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.196+0000 7f591cae9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f59180d4780 con 0x7f59180cb780 2024-05-20T18:28:11.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.196+0000 7f591cae9640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f59180d0300 con 0x7f5918162820 2024-05-20T18:28:11.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.196+0000 7f591cae9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f59180d4900 con 0x7f59180b3e80 2024-05-20T18:28:11.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.196+0000 7f5916d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59180cb780 0x7f59180c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:11.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.196+0000 7f5916d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59180cb780 0x7f59180c8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46258/0 (socket says 172.21.15.5:46258) 2024-05-20T18:28:11.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.196+0000 7f5916d9d640 1 -- 172.21.15.5:0/4219322778 learned_addr learned my addr 172.21.15.5:0/4219322778 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:11.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.196+0000 7f591659c640 1 --2- 172.21.15.5:0/4219322778 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59180b3e80 0x7f5918150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:11.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.196+0000 7f5915d9b640 1 --2- 172.21.15.5:0/4219322778 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5918162820 0x7f59180cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:11.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.196+0000 7f5916d9d640 1 -- 172.21.15.5:0/4219322778 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59180b3e80 msgr2=0x7f5918150230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:11.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.196+0000 7f5916d9d640 1 --2- 172.21.15.5:0/4219322778 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59180b3e80 0x7f5918150230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:11.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.196+0000 7f5916d9d640 1 -- 172.21.15.5:0/4219322778 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5918162820 msgr2=0x7f59180cb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:11.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.196+0000 7f5916d9d640 1 --2- 172.21.15.5:0/4219322778 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5918162820 0x7f59180cb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:11.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.196+0000 7f5916d9d640 1 -- 172.21.15.5:0/4219322778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59180cdeb0 con 0x7f59180cb780 2024-05-20T18:28:11.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.196+0000 7f5916d9d640 1 --2- 172.21.15.5:0/4219322778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59180cb780 0x7f59180c8820 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f591000cc60 tx=0x7f591000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:11.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.196+0000 7f590effd640 1 -- 172.21.15.5:0/4219322778 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5910003990 con 0x7f59180cb780 2024-05-20T18:28:11.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.196+0000 7f590effd640 1 -- 172.21.15.5:0/4219322778 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5910003b30 con 0x7f59180cb780 2024-05-20T18:28:11.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.196+0000 7f591cae9640 1 -- 172.21.15.5:0/4219322778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59180ce190 con 0x7f59180cb780 2024-05-20T18:28:11.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.196+0000 7f590effd640 1 -- 172.21.15.5:0/4219322778 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5910004020 con 0x7f59180cb780 2024-05-20T18:28:11.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.196+0000 7f591cae9640 1 -- 172.21.15.5:0/4219322778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59180ce750 con 0x7f59180cb780 2024-05-20T18:28:11.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.198+0000 7f591cae9640 1 -- 172.21.15.5:0/4219322778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5918075b00 con 0x7f59180cb780 2024-05-20T18:28:11.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.198+0000 7f590effd640 1 -- 172.21.15.5:0/4219322778 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f59100041c0 con 0x7f59180cb780 2024-05-20T18:28:11.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.199+0000 7f590effd640 1 --2- 172.21.15.5:0/4219322778 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f58f806be30 0x7f58f806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:11.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.199+0000 7f590effd640 1 -- 172.21.15.5:0/4219322778 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7f591008b790 con 0x7f59180cb780 2024-05-20T18:28:11.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.199+0000 7f591659c640 1 --2- 172.21.15.5:0/4219322778 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f58f806be30 0x7f58f806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:11.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.199+0000 7f591659c640 1 --2- 172.21.15.5:0/4219322778 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f58f806be30 0x7f58f806e340 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5908034000 tx=0x7f590802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:11.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.203+0000 7f590effd640 1 -- 172.21.15.5:0/4219322778 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5910014030 con 0x7f59180cb780 2024-05-20T18:28:11.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.391+0000 7f591cae9640 1 -- 172.21.15.5:0/4219322778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59180b3850 con 0x7f59180cb780 2024-05-20T18:28:11.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.392+0000 7f590effd640 1 -- 172.21.15.5:0/4219322778 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18800 (secure 0 0 0) 0x7f5910059ca0 con 0x7f59180cb780 2024-05-20T18:28:11.392 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:11.392 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:27:56.213938+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:27:05.476596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T18:27:06.137762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T18:27:09.829920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:28:11.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.394+0000 7f591cae9640 1 -- 172.21.15.5:0/4219322778 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f58f806be30 msgr2=0x7f58f806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:11.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.394+0000 7f591cae9640 1 --2- 172.21.15.5:0/4219322778 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f58f806be30 0x7f58f806e340 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5908034000 tx=0x7f590802e000 comp rx=0 tx=0).stop 2024-05-20T18:28:11.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.394+0000 7f591cae9640 1 -- 172.21.15.5:0/4219322778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59180cb780 msgr2=0x7f59180c8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:11.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.394+0000 7f591cae9640 1 --2- 172.21.15.5:0/4219322778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59180cb780 0x7f59180c8820 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f591000cc60 tx=0x7f591000a5c0 comp rx=0 tx=0).stop 2024-05-20T18:28:11.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.394+0000 7f591cae9640 1 -- 172.21.15.5:0/4219322778 shutdown_connections 2024-05-20T18:28:11.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.394+0000 7f591cae9640 1 --2- 172.21.15.5:0/4219322778 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f58f806be30 0x7f58f806e340 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:11.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.394+0000 7f591cae9640 1 --2- 172.21.15.5:0/4219322778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59180cb780 0x7f59180c8820 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:11.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.394+0000 7f591cae9640 1 --2- 172.21.15.5:0/4219322778 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5918162820 0x7f59180cb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:11.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.394+0000 7f591cae9640 1 --2- 172.21.15.5:0/4219322778 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59180b3e80 0x7f5918150230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:11.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.394+0000 7f591cae9640 1 -- 172.21.15.5:0/4219322778 >> 172.21.15.5:0/4219322778 conn(0x7f59180d4bd0 msgr2=0x7f591815e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:11.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.394+0000 7f591cae9640 1 -- 172.21.15.5:0/4219322778 shutdown_connections 2024-05-20T18:28:11.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.395+0000 7f591cae9640 1 -- 172.21.15.5:0/4219322778 wait complete. 2024-05-20T18:28:11.406 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-20T18:28:11.406 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:28:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.522+0000 7f7c1963d640 1 Processor -- start 2024-05-20T18:28:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.522+0000 7f7c1963d640 1 -- start start 2024-05-20T18:28:11.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.523+0000 7f7c1963d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7c1415d530 0x7f7c1415aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:11.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.523+0000 7f7c1963d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7c140b1700 0x7f7c140b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:11.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.523+0000 7f7c1963d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c140cd580 con 0x7f7c14070850 2024-05-20T18:28:11.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.523+0000 7f7c1963d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f7c140ce190 con 0x7f7c140b1700 2024-05-20T18:28:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.523+0000 7f7c1963d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7c140cd860 con 0x7f7c1415d530 2024-05-20T18:28:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.523+0000 7f7c0a7fc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7c140b1700 0x7f7c140b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.523+0000 7f7c0a7fc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7c140b1700 0x7f7c140b1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46268/0 (socket says 172.21.15.5:46268) 2024-05-20T18:28:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.523+0000 7f7c137fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7c14070850 0x7f7c14070c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35706/0 (socket says 172.21.15.5:35706) 2024-05-20T18:28:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.523+0000 7f7c0a7fc640 1 -- 172.21.15.5:0/142332215 learned_addr learned my addr 172.21.15.5:0/142332215 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.523+0000 7f7c12ffd640 1 --2- 172.21.15.5:0/142332215 >> v2:172.21.15.63:3300/0 conn(0x7f7c1415d530 0x7f7c1415aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.524+0000 7f7c127fc640 1 -- 172.21.15.5:0/142332215 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2737287257 0 0) 0x7f7c140cd580 con 0x7f7c14070850 2024-05-20T18:28:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.524+0000 7f7c127fc640 1 -- 172.21.15.5:0/142332215 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c00003520 con 0x7f7c14070850 2024-05-20T18:28:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.524+0000 7f7c0a7fc640 1 -- 172.21.15.5:0/142332215 >> v2:172.21.15.63:3300/0 conn(0x7f7c1415d530 msgr2=0x7f7c1415aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.524+0000 7f7c0a7fc640 1 --2- 172.21.15.5:0/142332215 >> v2:172.21.15.63:3300/0 conn(0x7f7c1415d530 0x7f7c1415aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.524+0000 7f7c0a7fc640 1 -- 172.21.15.5:0/142332215 >> v1:172.21.15.5:6789/0 conn(0x7f7c14070850 legacy=0x7f7c14070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.524+0000 7f7c0a7fc640 1 -- 172.21.15.5:0/142332215 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c140cdb40 con 0x7f7c140b1700 2024-05-20T18:28:11.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.524+0000 7f7c0a7fc640 1 --2- 172.21.15.5:0/142332215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c140b1700 0x7f7c140b1b10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7c1404f790 tx=0x7f7bfc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ed9a343497dd56 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.525+0000 7f7c127fc640 1 -- 172.21.15.5:0/142332215 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc037070 con 0x7f7c140b1700 2024-05-20T18:28:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.525+0000 7f7c127fc640 1 -- 172.21.15.5:0/142332215 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bfc032440 con 0x7f7c140b1700 2024-05-20T18:28:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.525+0000 7f7c127fc640 1 -- 172.21.15.5:0/142332215 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc032810 con 0x7f7c140b1700 2024-05-20T18:28:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.525+0000 7f7c1963d640 1 -- 172.21.15.5:0/142332215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c140b1700 msgr2=0x7f7c140b1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.525+0000 7f7c1963d640 1 --2- 172.21.15.5:0/142332215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c140b1700 0x7f7c140b1b10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7c1404f790 tx=0x7f7bfc02a270 comp rx=0 tx=0).stop 2024-05-20T18:28:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.525+0000 7f7c1963d640 1 -- 172.21.15.5:0/142332215 shutdown_connections 2024-05-20T18:28:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.525+0000 7f7c1963d640 1 --2- 172.21.15.5:0/142332215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c140b1700 0x7f7c140b1b10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.525+0000 7f7c1963d640 1 --2- 172.21.15.5:0/142332215 >> v2:172.21.15.63:3300/0 conn(0x7f7c1415d530 0x7f7c1415aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.525+0000 7f7c1963d640 1 -- 172.21.15.5:0/142332215 >> 172.21.15.5:0/142332215 conn(0x7f7c140d29d0 msgr2=0x7f7c140d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.525+0000 7f7c1963d640 1 -- 172.21.15.5:0/142332215 shutdown_connections 2024-05-20T18:28:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.525+0000 7f7c1963d640 1 -- 172.21.15.5:0/142332215 wait complete. 2024-05-20T18:28:11.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.527+0000 7f7c1963d640 1 Processor -- start 2024-05-20T18:28:11.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.527+0000 7f7c1963d640 1 -- start start 2024-05-20T18:28:11.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.529+0000 7f7c1963d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c14070850 0x7f7c141556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:11.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.529+0000 7f7c1963d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c140b1700 0x7f7c140c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:11.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.529+0000 7f7c1963d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c1415d530 0x7f7c1415bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:11.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.529+0000 7f7c1963d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7c140b2050 con 0x7f7c140b1700 2024-05-20T18:28:11.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.529+0000 7f7c1963d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7c140cd230 con 0x7f7c14070850 2024-05-20T18:28:11.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.529+0000 7f7c1963d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7c14071370 con 0x7f7c1415d530 2024-05-20T18:28:11.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.529+0000 7f7c0a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c140b1700 0x7f7c140c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:11.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.529+0000 7f7c0a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c140b1700 0x7f7c140c6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46280/0 (socket says 172.21.15.5:46280) 2024-05-20T18:28:11.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.529+0000 7f7c12ffd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c14070850 0x7f7c141556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:11.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.529+0000 7f7c0a7fc640 1 -- 172.21.15.5:0/3474101386 learned_addr learned my addr 172.21.15.5:0/3474101386 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:11.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.529+0000 7f7c137fe640 1 --2- 172.21.15.5:0/3474101386 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c1415d530 0x7f7c1415bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:11.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.529+0000 7f7c0a7fc640 1 -- 172.21.15.5:0/3474101386 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c1415d530 msgr2=0x7f7c1415bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:11.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.529+0000 7f7c0a7fc640 1 --2- 172.21.15.5:0/3474101386 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c1415d530 0x7f7c1415bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:11.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.529+0000 7f7c0a7fc640 1 -- 172.21.15.5:0/3474101386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c14070850 msgr2=0x7f7c141556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:11.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.529+0000 7f7c0a7fc640 1 --2- 172.21.15.5:0/3474101386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c14070850 0x7f7c141556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:11.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.529+0000 7f7c0a7fc640 1 -- 172.21.15.5:0/3474101386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c1415b550 con 0x7f7c140b1700 2024-05-20T18:28:11.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.529+0000 7f7c0a7fc640 1 --2- 172.21.15.5:0/3474101386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c140b1700 0x7f7c140c6500 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc02a2b0 tx=0x7f7bfc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:11.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.530+0000 7f7c0bfff640 1 -- 172.21.15.5:0/3474101386 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc041020 con 0x7f7c140b1700 2024-05-20T18:28:11.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.530+0000 7f7c0bfff640 1 -- 172.21.15.5:0/3474101386 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bfc03d620 con 0x7f7c140b1700 2024-05-20T18:28:11.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.530+0000 7f7c1963d640 1 -- 172.21.15.5:0/3474101386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c141565c0 con 0x7f7c140b1700 2024-05-20T18:28:11.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.530+0000 7f7c0bfff640 1 -- 172.21.15.5:0/3474101386 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc037040 con 0x7f7c140b1700 2024-05-20T18:28:11.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.530+0000 7f7c1963d640 1 -- 172.21.15.5:0/3474101386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c140cc8b0 con 0x7f7c140b1700 2024-05-20T18:28:11.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.531+0000 7f7c1963d640 1 -- 172.21.15.5:0/3474101386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c14075b00 con 0x7f7c140b1700 2024-05-20T18:28:11.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.531+0000 7f7c0bfff640 1 -- 172.21.15.5:0/3474101386 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7bfc044020 con 0x7f7c140b1700 2024-05-20T18:28:11.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.532+0000 7f7c0bfff640 1 --2- 172.21.15.5:0/3474101386 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7bf406be30 0x7f7bf406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:11.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.532+0000 7f7c0bfff640 1 -- 172.21.15.5:0/3474101386 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7f7bfc0ab8d0 con 0x7f7c140b1700 2024-05-20T18:28:11.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.532+0000 7f7c12ffd640 1 --2- 172.21.15.5:0/3474101386 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7bf406be30 0x7f7bf406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:11.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.532+0000 7f7c12ffd640 1 --2- 172.21.15.5:0/3474101386 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7bf406be30 0x7f7bf406e340 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7c140d29a0 tx=0x7f7c04009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:11.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.536+0000 7f7c0bfff640 1 -- 172.21.15.5:0/3474101386 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7bfc04b030 con 0x7f7c140b1700 2024-05-20T18:28:11.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.757+0000 7f7c1963d640 1 -- 172.21.15.5:0/3474101386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c1415a450 con 0x7f7c140b1700 2024-05-20T18:28:11.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.757+0000 7f7c0bfff640 1 -- 172.21.15.5:0/3474101386 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5168 (secure 0 0 0) 0x7f7c141565c0 con 0x7f7c140b1700 2024-05-20T18:28:11.757 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:11.757 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2982954341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2982954341},{"type":"v1","addr":"172.21.15.5:6836","nonce":2982954341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4032850001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":4032850001},{"type":"v1","addr":"172.21.15.5:6837","nonce":4032850001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6833/424580378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":424580378},{"type":"v1","addr":"172.21.15.63:6833","nonce":424580378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:27:07.164414+0000","modified":"2024-05-20T18:27:29.288681+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/3249423677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3249423677},{"type":"v1","addr":"172.21.15.63:6835","nonce":3249423677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-05-20T18:28:11.757 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 9 2024-05-20T18:28:11.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.759+0000 7f7c1963d640 1 -- 172.21.15.5:0/3474101386 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7bf406be30 msgr2=0x7f7bf406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:11.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.759+0000 7f7c1963d640 1 --2- 172.21.15.5:0/3474101386 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7bf406be30 0x7f7bf406e340 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7c140d29a0 tx=0x7f7c04009190 comp rx=0 tx=0).stop 2024-05-20T18:28:11.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.759+0000 7f7c1963d640 1 -- 172.21.15.5:0/3474101386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c140b1700 msgr2=0x7f7c140c6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:11.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.759+0000 7f7c1963d640 1 --2- 172.21.15.5:0/3474101386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c140b1700 0x7f7c140c6500 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc02a2b0 tx=0x7f7bfc0046c0 comp rx=0 tx=0).stop 2024-05-20T18:28:11.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.759+0000 7f7c1963d640 1 -- 172.21.15.5:0/3474101386 shutdown_connections 2024-05-20T18:28:11.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.759+0000 7f7c1963d640 1 --2- 172.21.15.5:0/3474101386 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7bf406be30 0x7f7bf406e340 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:11.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.759+0000 7f7c1963d640 1 --2- 172.21.15.5:0/3474101386 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c1415d530 0x7f7c1415bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:11.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.759+0000 7f7c1963d640 1 --2- 172.21.15.5:0/3474101386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c140b1700 0x7f7c140c6500 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:11.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.759+0000 7f7c1963d640 1 --2- 172.21.15.5:0/3474101386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c14070850 0x7f7c141556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:11.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.759+0000 7f7c1963d640 1 -- 172.21.15.5:0/3474101386 >> 172.21.15.5:0/3474101386 conn(0x7f7c140d29d0 msgr2=0x7f7c140c5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:11.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.759+0000 7f7c1963d640 1 -- 172.21.15.5:0/3474101386 shutdown_connections 2024-05-20T18:28:11.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.759+0000 7f7c1963d640 1 -- 172.21.15.5:0/3474101386 wait complete. 2024-05-20T18:28:11.770 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:28:11.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.888+0000 7fcfae014640 1 Processor -- start 2024-05-20T18:28:11.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.888+0000 7fcfae014640 1 -- start start 2024-05-20T18:28:11.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.889+0000 7fcfae014640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa809ac00 0x7fcfa80d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:11.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.889+0000 7fcfae014640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfa8160700 con 0x7fcfa81602f0 2024-05-20T18:28:11.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.889+0000 7fcfae014640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfa80af9d0 con 0x7fcfa806da80 2024-05-20T18:28:11.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.889+0000 7fcfae014640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcfa8159da0 con 0x7fcfa809ac00 2024-05-20T18:28:11.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.890+0000 7fcfa6ffd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fcfa81602f0 0x7fcfa809a4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35712/0 (socket says 172.21.15.5:35712) 2024-05-20T18:28:11.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.890+0000 7fcfa6ffd640 1 -- 172.21.15.5:0/2059387348 learned_addr learned my addr 172.21.15.5:0/2059387348 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:11.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.890+0000 7fcfa7fff640 1 --2- 172.21.15.5:0/2059387348 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa809ac00 0x7fcfa80d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:11.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.890+0000 7fcfa67fc640 1 -- 172.21.15.5:0/2059387348 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4256483513 0 0) 0x7fcf9c001290 con 0x7fcfa81602f0 2024-05-20T18:28:11.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.890+0000 7fcfa67fc640 1 -- 172.21.15.5:0/2059387348 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf90003520 con 0x7fcfa81602f0 2024-05-20T18:28:11.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.890+0000 7fcfa67fc640 1 -- 172.21.15.5:0/2059387348 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2464046837 0 0) 0x7fcfa80af9d0 con 0x7fcfa806da80 2024-05-20T18:28:11.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.890+0000 7fcfa67fc640 1 -- 172.21.15.5:0/2059387348 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf9c001290 con 0x7fcfa806da80 2024-05-20T18:28:11.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.890+0000 7fcfa7fff640 1 -- 172.21.15.5:0/2059387348 >> v1:172.21.15.63:6789/0 conn(0x7fcfa806da80 legacy=0x7fcfa80b1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:11.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.890+0000 7fcfa7fff640 1 -- 172.21.15.5:0/2059387348 >> v1:172.21.15.5:6789/0 conn(0x7fcfa81602f0 legacy=0x7fcfa809a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:11.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.890+0000 7fcfa7fff640 1 -- 172.21.15.5:0/2059387348 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa80af410 con 0x7fcfa809ac00 2024-05-20T18:28:11.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.891+0000 7fcfa7fff640 1 --2- 172.21.15.5:0/2059387348 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa809ac00 0x7fcfa80d1c40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009dc0 tx=0x7fcfa002a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d896ca1dc50acc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:11.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.891+0000 7fcfa67fc640 1 -- 172.21.15.5:0/2059387348 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa002ae60 con 0x7fcfa809ac00 2024-05-20T18:28:11.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.891+0000 7fcfa67fc640 1 -- 172.21.15.5:0/2059387348 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfa0007740 con 0x7fcfa809ac00 2024-05-20T18:28:11.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.891+0000 7fcfa67fc640 1 -- 172.21.15.5:0/2059387348 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0007b10 con 0x7fcfa809ac00 2024-05-20T18:28:11.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.892+0000 7fcfae014640 1 -- 172.21.15.5:0/2059387348 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa809ac00 msgr2=0x7fcfa80d1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:11.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.892+0000 7fcfae014640 1 --2- 172.21.15.5:0/2059387348 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa809ac00 0x7fcfa80d1c40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009dc0 tx=0x7fcfa002a640 comp rx=0 tx=0).stop 2024-05-20T18:28:11.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.892+0000 7fcfae014640 1 -- 172.21.15.5:0/2059387348 shutdown_connections 2024-05-20T18:28:11.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.892+0000 7fcfae014640 1 --2- 172.21.15.5:0/2059387348 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa809ac00 0x7fcfa80d1c40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:11.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.892+0000 7fcfae014640 1 -- 172.21.15.5:0/2059387348 >> 172.21.15.5:0/2059387348 conn(0x7fcfa80b6670 msgr2=0x7fcfa80b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:11.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.892+0000 7fcfae014640 1 -- 172.21.15.5:0/2059387348 shutdown_connections 2024-05-20T18:28:11.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.892+0000 7fcfae014640 1 -- 172.21.15.5:0/2059387348 wait complete. 2024-05-20T18:28:11.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.894+0000 7fcfae014640 1 Processor -- start 2024-05-20T18:28:11.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.894+0000 7fcfae014640 1 -- start start 2024-05-20T18:28:11.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.895+0000 7fcfae014640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa806da80 0x7fcfa8148a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:11.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.895+0000 7fcfae014640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfa81602f0 0x7fcfa815e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:11.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.895+0000 7fcfae014640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfa81583e0 0x7fcfa814b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:11.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.895+0000 7fcfae014640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa80d17b0 con 0x7fcfa81602f0 2024-05-20T18:28:11.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.895+0000 7fcfae014640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa80af1c0 con 0x7fcfa81583e0 2024-05-20T18:28:11.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.895+0000 7fcfae014640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcfa80b2390 con 0x7fcfa806da80 2024-05-20T18:28:11.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.895+0000 7fcfa6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfa81602f0 0x7fcfa815e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:11.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.895+0000 7fcfa6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfa81602f0 0x7fcfa815e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46282/0 (socket says 172.21.15.5:46282) 2024-05-20T18:28:11.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.895+0000 7fcfa6ffd640 1 -- 172.21.15.5:0/2263150976 learned_addr learned my addr 172.21.15.5:0/2263150976 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:11.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.895+0000 7fcfa77fe640 1 --2- 172.21.15.5:0/2263150976 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa806da80 0x7fcfa8148a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.895+0000 7fcfa7fff640 1 --2- 172.21.15.5:0/2263150976 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfa81583e0 0x7fcfa814b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.896+0000 7fcfa6ffd640 1 -- 172.21.15.5:0/2263150976 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa806da80 msgr2=0x7fcfa8148a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.896+0000 7fcfa6ffd640 1 --2- 172.21.15.5:0/2263150976 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa806da80 0x7fcfa8148a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.896+0000 7fcfa6ffd640 1 -- 172.21.15.5:0/2263150976 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfa81583e0 msgr2=0x7fcfa814b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.896+0000 7fcfa6ffd640 1 --2- 172.21.15.5:0/2263150976 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfa81583e0 0x7fcfa814b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.896+0000 7fcfa6ffd640 1 -- 172.21.15.5:0/2263150976 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa809dc60 con 0x7fcfa81602f0 2024-05-20T18:28:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.896+0000 7fcfa77fe640 1 --2- 172.21.15.5:0/2263150976 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa806da80 0x7fcfa8148a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:28:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.896+0000 7fcfa6ffd640 1 --2- 172.21.15.5:0/2263150976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfa81602f0 0x7fcfa815e630 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00c730 tx=0x7fcf9c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.896+0000 7fcf8ffff640 1 -- 172.21.15.5:0/2263150976 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c017070 con 0x7fcfa81602f0 2024-05-20T18:28:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.896+0000 7fcf8ffff640 1 -- 172.21.15.5:0/2263150976 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf9c007290 con 0x7fcfa81602f0 2024-05-20T18:28:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.896+0000 7fcfae014640 1 -- 172.21.15.5:0/2263150976 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa809df40 con 0x7fcfa81602f0 2024-05-20T18:28:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.896+0000 7fcf8ffff640 1 -- 172.21.15.5:0/2263150976 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c0057a0 con 0x7fcfa81602f0 2024-05-20T18:28:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.896+0000 7fcfae014640 1 -- 172.21.15.5:0/2263150976 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa809e500 con 0x7fcfa81602f0 2024-05-20T18:28:11.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.897+0000 7fcfae014640 1 -- 172.21.15.5:0/2263150976 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfa8075b00 con 0x7fcfa81602f0 2024-05-20T18:28:11.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.898+0000 7fcf8ffff640 1 -- 172.21.15.5:0/2263150976 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcf9c01a070 con 0x7fcfa81602f0 2024-05-20T18:28:11.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.898+0000 7fcf8ffff640 1 --2- 172.21.15.5:0/2263150976 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcf9406be30 0x7fcf9406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:11.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.899+0000 7fcf8ffff640 1 -- 172.21.15.5:0/2263150976 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7fcf9c08a750 con 0x7fcfa81602f0 2024-05-20T18:28:11.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.899+0000 7fcfa77fe640 1 --2- 172.21.15.5:0/2263150976 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcf9406be30 0x7fcf9406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:11.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.899+0000 7fcfa77fe640 1 --2- 172.21.15.5:0/2263150976 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcf9406be30 0x7fcf9406e340 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcf98007fd0 tx=0x7fcf98009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:11.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:11.903+0000 7fcf8ffff640 1 -- 172.21.15.5:0/2263150976 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fcfa8160700 con 0x7fcfa81602f0 2024-05-20T18:28:12.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.125+0000 7fcfae014640 1 -- 172.21.15.5:0/2263150976 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcfa815eda0 con 0x7fcfa81602f0 2024-05-20T18:28:12.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.126+0000 7fcf8ffff640 1 -- 172.21.15.5:0/2263150976 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5168 (secure 0 0 0) 0x7fcfa815eda0 con 0x7fcfa81602f0 2024-05-20T18:28:12.125 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:12.125 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2982954341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2982954341},{"type":"v1","addr":"172.21.15.5:6836","nonce":2982954341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4032850001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":4032850001},{"type":"v1","addr":"172.21.15.5:6837","nonce":4032850001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6833/424580378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":424580378},{"type":"v1","addr":"172.21.15.63:6833","nonce":424580378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:27:07.164414+0000","modified":"2024-05-20T18:27:29.288681+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/3249423677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3249423677},{"type":"v1","addr":"172.21.15.63:6835","nonce":3249423677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-05-20T18:28:12.126 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 9 2024-05-20T18:28:12.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.128+0000 7fcfae014640 1 -- 172.21.15.5:0/2263150976 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcf9406be30 msgr2=0x7fcf9406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:12.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.128+0000 7fcfae014640 1 --2- 172.21.15.5:0/2263150976 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcf9406be30 0x7fcf9406e340 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcf98007fd0 tx=0x7fcf98009040 comp rx=0 tx=0).stop 2024-05-20T18:28:12.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.128+0000 7fcfae014640 1 -- 172.21.15.5:0/2263150976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfa81602f0 msgr2=0x7fcfa815e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:12.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.128+0000 7fcfae014640 1 --2- 172.21.15.5:0/2263150976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfa81602f0 0x7fcfa815e630 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00c730 tx=0x7fcf9c00cbf0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.128+0000 7fcfae014640 1 -- 172.21.15.5:0/2263150976 shutdown_connections 2024-05-20T18:28:12.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.128+0000 7fcfae014640 1 --2- 172.21.15.5:0/2263150976 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcf9406be30 0x7fcf9406e340 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.128+0000 7fcfae014640 1 --2- 172.21.15.5:0/2263150976 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfa81583e0 0x7fcfa814b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.128+0000 7fcfae014640 1 --2- 172.21.15.5:0/2263150976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfa81602f0 0x7fcfa815e630 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.128+0000 7fcfae014640 1 --2- 172.21.15.5:0/2263150976 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa806da80 0x7fcfa8148a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.128+0000 7fcfae014640 1 -- 172.21.15.5:0/2263150976 >> 172.21.15.5:0/2263150976 conn(0x7fcfa80b6670 msgr2=0x7fcfa80b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:12.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.128+0000 7fcfae014640 1 -- 172.21.15.5:0/2263150976 shutdown_connections 2024-05-20T18:28:12.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.128+0000 7fcfae014640 1 -- 172.21.15.5:0/2263150976 wait complete. 2024-05-20T18:28:12.139 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:28:12.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.255+0000 7f4530f85640 1 Processor -- start 2024-05-20T18:28:12.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.255+0000 7f4530f85640 1 -- start start 2024-05-20T18:28:12.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.256+0000 7f4530f85640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f452c162820 0x7f452c0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:12.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.257+0000 7f4530f85640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f452c0b3e80 0x7f452c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:12.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.257+0000 7f4530f85640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f452c06dd20 con 0x7f452c06d910 2024-05-20T18:28:12.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.257+0000 7f4530f85640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f452c0d0630 con 0x7f452c0b3e80 2024-05-20T18:28:12.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.257+0000 7f4530f85640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f452c0d39c0 con 0x7f452c162820 2024-05-20T18:28:12.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.257+0000 7f4529d9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f452c0b3e80 0x7f452c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:12.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.257+0000 7f4529d9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f452c0b3e80 0x7f452c06d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46294/0 (socket says 172.21.15.5:46294) 2024-05-20T18:28:12.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.257+0000 7f4529d9b640 1 -- 172.21.15.5:0/208370981 learned_addr learned my addr 172.21.15.5:0/208370981 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:12.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.257+0000 7f452ad9d640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f452c06d910 0x7f452c0d27a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35722/0 (socket says 172.21.15.5:35722) 2024-05-20T18:28:12.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.257+0000 7f452a59c640 1 --2- 172.21.15.5:0/208370981 >> v2:172.21.15.63:3300/0 conn(0x7f452c162820 0x7f452c0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:12.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.257+0000 7f4529d9b640 1 -- 172.21.15.5:0/208370981 >> v2:172.21.15.63:3300/0 conn(0x7f452c162820 msgr2=0x7f452c0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:12.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.257+0000 7f4529d9b640 1 --2- 172.21.15.5:0/208370981 >> v2:172.21.15.63:3300/0 conn(0x7f452c162820 0x7f452c0b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.257+0000 7f4529d9b640 1 -- 172.21.15.5:0/208370981 >> v1:172.21.15.5:6789/0 conn(0x7f452c06d910 legacy=0x7f452c0d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:12.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.257+0000 7f452959a640 1 -- 172.21.15.5:0/208370981 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3001892564 0 0) 0x7f4524001490 con 0x7f452c06d910 2024-05-20T18:28:12.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.257+0000 7f4529d9b640 1 -- 172.21.15.5:0/208370981 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452c0d3ca0 con 0x7f452c0b3e80 2024-05-20T18:28:12.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.258+0000 7f4529d9b640 1 --2- 172.21.15.5:0/208370981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f452c0b3e80 0x7f452c06d3d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f452c0b7740 tx=0x7f452002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa57749f1e11428 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:12.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.258+0000 7f452959a640 1 -- 172.21.15.5:0/208370981 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520037070 con 0x7f452c0b3e80 2024-05-20T18:28:12.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.258+0000 7f452959a640 1 -- 172.21.15.5:0/208370981 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4520032440 con 0x7f452c0b3e80 2024-05-20T18:28:12.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.258+0000 7f452959a640 1 -- 172.21.15.5:0/208370981 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520032810 con 0x7f452c0b3e80 2024-05-20T18:28:12.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.259+0000 7f4530f85640 1 -- 172.21.15.5:0/208370981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f452c0b3e80 msgr2=0x7f452c06d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:12.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.259+0000 7f4530f85640 1 --2- 172.21.15.5:0/208370981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f452c0b3e80 0x7f452c06d3d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f452c0b7740 tx=0x7f452002a270 comp rx=0 tx=0).stop 2024-05-20T18:28:12.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.259+0000 7f4530f85640 1 -- 172.21.15.5:0/208370981 shutdown_connections 2024-05-20T18:28:12.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.259+0000 7f4530f85640 1 --2- 172.21.15.5:0/208370981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f452c0b3e80 0x7f452c06d3d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.259+0000 7f4530f85640 1 --2- 172.21.15.5:0/208370981 >> v2:172.21.15.63:3300/0 conn(0x7f452c162820 0x7f452c0b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.259+0000 7f4530f85640 1 -- 172.21.15.5:0/208370981 >> 172.21.15.5:0/208370981 conn(0x7f452c0d4bd0 msgr2=0x7f452c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:12.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.259+0000 7f4530f85640 1 -- 172.21.15.5:0/208370981 shutdown_connections 2024-05-20T18:28:12.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.259+0000 7f4530f85640 1 -- 172.21.15.5:0/208370981 wait complete. 2024-05-20T18:28:12.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.261+0000 7f4530f85640 1 Processor -- start 2024-05-20T18:28:12.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.261+0000 7f4530f85640 1 -- start start 2024-05-20T18:28:12.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.262+0000 7f4530f85640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f452c06d910 0x7f452c15ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:12.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.262+0000 7f4530f85640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f452c0b3e80 0x7f452c1540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:12.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.262+0000 7f4530f85640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f452c162820 0x7f452c0cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:12.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.262+0000 7f4530f85640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f452c0d0300 con 0x7f452c162820 2024-05-20T18:28:12.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.262+0000 7f4530f85640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f452c0d4260 con 0x7f452c06d910 2024-05-20T18:28:12.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.262+0000 7f4530f85640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f452c0b91a0 con 0x7f452c0b3e80 2024-05-20T18:28:12.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.262+0000 7f452ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f452c162820 0x7f452c0cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:12.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.262+0000 7f452ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f452c162820 0x7f452c0cd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46298/0 (socket says 172.21.15.5:46298) 2024-05-20T18:28:12.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.262+0000 7f452ad9d640 1 -- 172.21.15.5:0/3031669162 learned_addr learned my addr 172.21.15.5:0/3031669162 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:12.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.262+0000 7f452a59c640 1 --2- 172.21.15.5:0/3031669162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f452c06d910 0x7f452c15ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:12.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.262+0000 7f4529d9b640 1 --2- 172.21.15.5:0/3031669162 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f452c0b3e80 0x7f452c1540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:12.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.263+0000 7f452ad9d640 1 -- 172.21.15.5:0/3031669162 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f452c0b3e80 msgr2=0x7f452c1540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:12.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.263+0000 7f452ad9d640 1 --2- 172.21.15.5:0/3031669162 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f452c0b3e80 0x7f452c1540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.263+0000 7f452ad9d640 1 -- 172.21.15.5:0/3031669162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f452c06d910 msgr2=0x7f452c15ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:12.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.263+0000 7f452ad9d640 1 --2- 172.21.15.5:0/3031669162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f452c06d910 0x7f452c15ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.263+0000 7f452ad9d640 1 -- 172.21.15.5:0/3031669162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452c0c9110 con 0x7f452c162820 2024-05-20T18:28:12.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.263+0000 7f452ad9d640 1 --2- 172.21.15.5:0/3031669162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f452c162820 0x7f452c0cd110 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f452400c730 tx=0x7f452400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:12.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.263+0000 7f451affd640 1 -- 172.21.15.5:0/3031669162 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4524018070 con 0x7f452c162820 2024-05-20T18:28:12.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.263+0000 7f451affd640 1 -- 172.21.15.5:0/3031669162 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f452400ce40 con 0x7f452c162820 2024-05-20T18:28:12.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.263+0000 7f4530f85640 1 -- 172.21.15.5:0/3031669162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f452c0c93f0 con 0x7f452c162820 2024-05-20T18:28:12.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.263+0000 7f451affd640 1 -- 172.21.15.5:0/3031669162 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452400a860 con 0x7f452c162820 2024-05-20T18:28:12.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.263+0000 7f4530f85640 1 -- 172.21.15.5:0/3031669162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f452c0b58e0 con 0x7f452c162820 2024-05-20T18:28:12.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.264+0000 7f4530f85640 1 -- 172.21.15.5:0/3031669162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f452c075b00 con 0x7f452c162820 2024-05-20T18:28:12.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.265+0000 7f451affd640 1 -- 172.21.15.5:0/3031669162 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f452401b070 con 0x7f452c162820 2024-05-20T18:28:12.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.265+0000 7f451affd640 1 --2- 172.21.15.5:0/3031669162 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f450c06be30 0x7f450c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:12.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.265+0000 7f451affd640 1 -- 172.21.15.5:0/3031669162 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7f452408a870 con 0x7f452c162820 2024-05-20T18:28:12.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.265+0000 7f452a59c640 1 --2- 172.21.15.5:0/3031669162 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f450c06be30 0x7f450c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:12.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.266+0000 7f452a59c640 1 --2- 172.21.15.5:0/3031669162 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f450c06be30 0x7f450c06e340 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f451c009aa0 tx=0x7f451c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:12.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.270+0000 7f451affd640 1 -- 172.21.15.5:0/3031669162 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f452c06dd20 con 0x7f452c162820 2024-05-20T18:28:12.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.460+0000 7f4530f85640 1 -- 172.21.15.5:0/3031669162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f452c15b240 con 0x7f452c162820 2024-05-20T18:28:12.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.461+0000 7f451affd640 1 -- 172.21.15.5:0/3031669162 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18800 (secure 0 0 0) 0x7f4524058730 con 0x7f452c162820 2024-05-20T18:28:12.461 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:12.461 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:27:56.213938+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:27:05.476596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-20T18:27:06.137762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-20T18:27:09.829920+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:28:12.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.463+0000 7f4530f85640 1 -- 172.21.15.5:0/3031669162 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f450c06be30 msgr2=0x7f450c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:12.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.463+0000 7f4530f85640 1 --2- 172.21.15.5:0/3031669162 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f450c06be30 0x7f450c06e340 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f451c009aa0 tx=0x7f451c009190 comp rx=0 tx=0).stop 2024-05-20T18:28:12.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.463+0000 7f4530f85640 1 -- 172.21.15.5:0/3031669162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f452c162820 msgr2=0x7f452c0cd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:12.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.463+0000 7f4530f85640 1 --2- 172.21.15.5:0/3031669162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f452c162820 0x7f452c0cd110 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f452400c730 tx=0x7f452400cbf0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.464+0000 7f4530f85640 1 -- 172.21.15.5:0/3031669162 shutdown_connections 2024-05-20T18:28:12.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.464+0000 7f4530f85640 1 --2- 172.21.15.5:0/3031669162 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f450c06be30 0x7f450c06e340 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.464+0000 7f4530f85640 1 --2- 172.21.15.5:0/3031669162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f452c162820 0x7f452c0cd110 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.464+0000 7f4530f85640 1 --2- 172.21.15.5:0/3031669162 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f452c0b3e80 0x7f452c1540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.464+0000 7f4530f85640 1 --2- 172.21.15.5:0/3031669162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f452c06d910 0x7f452c15ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.464+0000 7f4530f85640 1 -- 172.21.15.5:0/3031669162 >> 172.21.15.5:0/3031669162 conn(0x7f452c0d4bd0 msgr2=0x7f452c0ba4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:12.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.464+0000 7f4530f85640 1 -- 172.21.15.5:0/3031669162 shutdown_connections 2024-05-20T18:28:12.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.464+0000 7f4530f85640 1 -- 172.21.15.5:0/3031669162 wait complete. 2024-05-20T18:28:12.475 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-20T18:28:12.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.592+0000 7f96ac109640 1 Processor -- start 2024-05-20T18:28:12.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.593+0000 7f96ac109640 1 -- start start 2024-05-20T18:28:12.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.594+0000 7f96ac109640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f96a40b6620 0x7f96a40ba3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:12.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.594+0000 7f96ac109640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96a40d3bf0 0x7f96a40d4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:12.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.594+0000 7f96ac109640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96a40d4540 con 0x7f96a4162ce0 2024-05-20T18:28:12.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.594+0000 7f96ac109640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f96a40d3480 con 0x7f96a40b6620 2024-05-20T18:28:12.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.594+0000 7f96ac109640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f96a41634a0 con 0x7f96a40d3bf0 2024-05-20T18:28:12.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.594+0000 7f96a9ea5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f96a40b6620 0x7f96a40ba3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:12.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.594+0000 7f96a96a4640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f96a4162ce0 0x7f96a40b8070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35734/0 (socket says 172.21.15.5:35734) 2024-05-20T18:28:12.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.594+0000 7f96a96a4640 1 -- 172.21.15.5:0/974130110 learned_addr learned my addr 172.21.15.5:0/974130110 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:12.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.594+0000 7f96a9ea5640 1 -- 172.21.15.5:0/974130110 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96a40d3bf0 msgr2=0x7f96a40d4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:12.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.594+0000 7f96a9ea5640 1 --2- 172.21.15.5:0/974130110 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96a40d3bf0 0x7f96a40d4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.594+0000 7f96a9ea5640 1 -- 172.21.15.5:0/974130110 >> v1:172.21.15.5:6789/0 conn(0x7f96a4162ce0 legacy=0x7f96a40b8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:12.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.594+0000 7f96a9ea5640 1 -- 172.21.15.5:0/974130110 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a4163780 con 0x7f96a40b6620 2024-05-20T18:28:12.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.594+0000 7f96a8ea3640 1 -- 172.21.15.5:0/974130110 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1526423013 0 0) 0x7f9694001110 con 0x7f96a4162ce0 2024-05-20T18:28:12.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.595+0000 7f96a9ea5640 1 --2- 172.21.15.5:0/974130110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96a40b6620 0x7f96a40ba3c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f96a00097a0 tx=0x7f96a002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce598905f70311e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:12.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.595+0000 7f96a8ea3640 1 -- 172.21.15.5:0/974130110 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0037070 con 0x7f96a40b6620 2024-05-20T18:28:12.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.595+0000 7f96a8ea3640 1 -- 172.21.15.5:0/974130110 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a0032440 con 0x7f96a40b6620 2024-05-20T18:28:12.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.595+0000 7f96a8ea3640 1 -- 172.21.15.5:0/974130110 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0032810 con 0x7f96a40b6620 2024-05-20T18:28:12.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.596+0000 7f96ac109640 1 -- 172.21.15.5:0/974130110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96a40b6620 msgr2=0x7f96a40ba3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:12.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.596+0000 7f96ac109640 1 --2- 172.21.15.5:0/974130110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96a40b6620 0x7f96a40ba3c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f96a00097a0 tx=0x7f96a002a1a0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.596+0000 7f96ac109640 1 -- 172.21.15.5:0/974130110 shutdown_connections 2024-05-20T18:28:12.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.596+0000 7f96ac109640 1 --2- 172.21.15.5:0/974130110 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96a40d3bf0 0x7f96a40d4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.596+0000 7f96ac109640 1 --2- 172.21.15.5:0/974130110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96a40b6620 0x7f96a40ba3c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.596+0000 7f96ac109640 1 -- 172.21.15.5:0/974130110 >> 172.21.15.5:0/974130110 conn(0x7f96a406d240 msgr2=0x7f96a406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:12.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.596+0000 7f96ac109640 1 -- 172.21.15.5:0/974130110 shutdown_connections 2024-05-20T18:28:12.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.596+0000 7f96ac109640 1 -- 172.21.15.5:0/974130110 wait complete. 2024-05-20T18:28:12.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.598+0000 7f96ac109640 1 Processor -- start 2024-05-20T18:28:12.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.598+0000 7f96ac109640 1 -- start start 2024-05-20T18:28:12.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.599+0000 7f96ac109640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96a40b6620 0x7f96a41500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:12.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.599+0000 7f96ac109640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96a40d3bf0 0x7f96a40c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:12.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.599+0000 7f96ac109640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96a4162ce0 0x7f96a40d4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:12.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.599+0000 7f96ac109640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41630f0 con 0x7f96a40d3bf0 2024-05-20T18:28:12.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.599+0000 7f96ac109640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f96a40d36d0 con 0x7f96a4162ce0 2024-05-20T18:28:12.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.599+0000 7f96ac109640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f96a40cb460 con 0x7f96a40b6620 2024-05-20T18:28:12.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.599+0000 7f96a96a4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96a40d3bf0 0x7f96a40c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:12.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.599+0000 7f96a96a4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96a40d3bf0 0x7f96a40c9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46314/0 (socket says 172.21.15.5:46314) 2024-05-20T18:28:12.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.599+0000 7f96a96a4640 1 -- 172.21.15.5:0/1821939190 learned_addr learned my addr 172.21.15.5:0/1821939190 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:12.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.599+0000 7f96a9ea5640 1 --2- 172.21.15.5:0/1821939190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96a40b6620 0x7f96a41500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:12.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.600+0000 7f96aa6a6640 1 --2- 172.21.15.5:0/1821939190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96a4162ce0 0x7f96a40d4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:12.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.600+0000 7f96a96a4640 1 -- 172.21.15.5:0/1821939190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96a40b6620 msgr2=0x7f96a41500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:12.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.600+0000 7f96a96a4640 1 --2- 172.21.15.5:0/1821939190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96a40b6620 0x7f96a41500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.600+0000 7f96a96a4640 1 -- 172.21.15.5:0/1821939190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96a4162ce0 msgr2=0x7f96a40d4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:12.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.600+0000 7f96a96a4640 1 --2- 172.21.15.5:0/1821939190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96a4162ce0 0x7f96a40d4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.600+0000 7f96a96a4640 1 -- 172.21.15.5:0/1821939190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a41f6ab0 con 0x7f96a40d3bf0 2024-05-20T18:28:12.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.600+0000 7f96a96a4640 1 --2- 172.21.15.5:0/1821939190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96a40d3bf0 0x7f96a40c9da0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f969400c430 tx=0x7f969400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:12.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.600+0000 7f969a7fc640 1 -- 172.21.15.5:0/1821939190 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96940074e0 con 0x7f96a40d3bf0 2024-05-20T18:28:12.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.600+0000 7f969a7fc640 1 -- 172.21.15.5:0/1821939190 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96940047e0 con 0x7f96a40d3bf0 2024-05-20T18:28:12.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.600+0000 7f96ac109640 1 -- 172.21.15.5:0/1821939190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a41f6fe0 con 0x7f96a40d3bf0 2024-05-20T18:28:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.600+0000 7f969a7fc640 1 -- 172.21.15.5:0/1821939190 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694004bd0 con 0x7f96a40d3bf0 2024-05-20T18:28:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.600+0000 7f96ac109640 1 -- 172.21.15.5:0/1821939190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a41f75a0 con 0x7f96a40d3bf0 2024-05-20T18:28:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.601+0000 7f96ac109640 1 -- 172.21.15.5:0/1821939190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96a41f6d30 con 0x7f96a40d3bf0 2024-05-20T18:28:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.602+0000 7f969a7fc640 1 -- 172.21.15.5:0/1821939190 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9694011070 con 0x7f96a40d3bf0 2024-05-20T18:28:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.602+0000 7f969a7fc640 1 --2- 172.21.15.5:0/1821939190 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f968806be30 0x7f968806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.602+0000 7f969a7fc640 1 -- 172.21.15.5:0/1821939190 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6743+0+0 (secure 0 0 0) 0x7f969408ade0 con 0x7f96a40d3bf0 2024-05-20T18:28:12.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.602+0000 7f96a9ea5640 1 --2- 172.21.15.5:0/1821939190 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f968806be30 0x7f968806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:12.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.603+0000 7f96a9ea5640 1 --2- 172.21.15.5:0/1821939190 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f968806be30 0x7f968806e340 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f96a0034000 tx=0x7f96a002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:12.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.607+0000 7f969a7fc640 1 -- 172.21.15.5:0/1821939190 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f96a40d4540 con 0x7f96a40d3bf0 2024-05-20T18:28:12.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.831+0000 7f96ac109640 1 -- 172.21.15.5:0/1821939190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f96a4075b00 con 0x7f96a40d3bf0 2024-05-20T18:28:12.831 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-20T18:28:12.832+0000 7fd1cf53a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-20T18:28:12.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.833+0000 7f969a7fc640 1 -- 172.21.15.5:0/1821939190 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7f9694058ca0 con 0x7f96a40d3bf0 2024-05-20T18:28:12.834 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-20T18:28:12.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.836+0000 7f96ac109640 1 -- 172.21.15.5:0/1821939190 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f968806be30 msgr2=0x7f968806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:12.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.836+0000 7f96ac109640 1 --2- 172.21.15.5:0/1821939190 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f968806be30 0x7f968806e340 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f96a0034000 tx=0x7f96a002e000 comp rx=0 tx=0).stop 2024-05-20T18:28:12.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.836+0000 7f96ac109640 1 -- 172.21.15.5:0/1821939190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96a40d3bf0 msgr2=0x7f96a40c9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:12.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.836+0000 7f96ac109640 1 --2- 172.21.15.5:0/1821939190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96a40d3bf0 0x7f96a40c9da0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f969400c430 tx=0x7f969400c8f0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.836+0000 7f96ac109640 1 -- 172.21.15.5:0/1821939190 shutdown_connections 2024-05-20T18:28:12.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.836+0000 7f96ac109640 1 --2- 172.21.15.5:0/1821939190 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f968806be30 0x7f968806e340 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.836+0000 7f96ac109640 1 --2- 172.21.15.5:0/1821939190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96a4162ce0 0x7f96a40d4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.836+0000 7f96ac109640 1 --2- 172.21.15.5:0/1821939190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96a40d3bf0 0x7f96a40c9da0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.836+0000 7f96ac109640 1 --2- 172.21.15.5:0/1821939190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96a40b6620 0x7f96a41500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.836+0000 7f96ac109640 1 -- 172.21.15.5:0/1821939190 >> 172.21.15.5:0/1821939190 conn(0x7f96a406d240 msgr2=0x7f96a4160c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:12.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.836+0000 7f96ac109640 1 -- 172.21.15.5:0/1821939190 shutdown_connections 2024-05-20T18:28:12.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.836+0000 7f96ac109640 1 -- 172.21.15.5:0/1821939190 wait complete. 2024-05-20T18:28:12.840 INFO:tasks.ceph.mds.b.smithi063.stderr: -5586> 2024-05-20T18:27:12.110+0000 7f25b7bf3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T18:28:12.847 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-20T18:28:12.899 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-20T18:28:12.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.965+0000 7fb3bfcfe640 1 Processor -- start 2024-05-20T18:28:12.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.965+0000 7fb3bfcfe640 1 -- start start 2024-05-20T18:28:12.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.967+0000 7fb3bfcfe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb3b8162820 0x7fb3b80b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:12.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.967+0000 7fb3bfcfe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb3b806dae0 0x7fb3b80b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:12.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.967+0000 7fb3bfcfe640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3b80d0630 con 0x7fb3b80b3e80 2024-05-20T18:28:12.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.967+0000 7fb3bfcfe640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fb3b80d39c0 con 0x7fb3b806dae0 2024-05-20T18:28:12.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.967+0000 7fb3bfcfe640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb3b80d3ca0 con 0x7fb3b8162820 2024-05-20T18:28:12.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.967+0000 7fb3be29b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb3b806dae0 0x7fb3b80b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.967+0000 7fb3be29b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb3b806dae0 0x7fb3b80b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46322/0 (socket says 172.21.15.5:46322) 2024-05-20T18:28:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.967+0000 7fb3bda9a640 1 --2- 172.21.15.5:0/3950828043 >> v2:172.21.15.63:3300/0 conn(0x7fb3b8162820 0x7fb3b80b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.967+0000 7fb3be29b640 1 -- 172.21.15.5:0/3950828043 learned_addr learned my addr 172.21.15.5:0/3950828043 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.967+0000 7fb3bda9a640 1 -- 172.21.15.5:0/3950828043 >> v2:172.21.15.5:3300/0 conn(0x7fb3b806dae0 msgr2=0x7fb3b80b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.967+0000 7fb3bda9a640 1 --2- 172.21.15.5:0/3950828043 >> v2:172.21.15.5:3300/0 conn(0x7fb3b806dae0 0x7fb3b80b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.967+0000 7fb3bda9a640 1 -- 172.21.15.5:0/3950828043 >> v1:172.21.15.63:6789/0 conn(0x7fb3b80b3e80 legacy=0x7fb3b806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.967+0000 7fb3bda9a640 1 -- 172.21.15.5:0/3950828043 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b80d3f80 con 0x7fb3b8162820 2024-05-20T18:28:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.967+0000 7fb3bca98640 1 -- 172.21.15.5:0/3950828043 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2762168548 0 0) 0x7fb3b80d0630 con 0x7fb3b80b3e80 2024-05-20T18:28:12.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.968+0000 7fb3bda9a640 1 --2- 172.21.15.5:0/3950828043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3b8162820 0x7fb3b80b3940 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80b7740 tx=0x7fb3b002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d57a605ad4b1d6c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:12.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.968+0000 7fb3bca98640 1 -- 172.21.15.5:0/3950828043 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0037070 con 0x7fb3b8162820 2024-05-20T18:28:12.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.968+0000 7fb3bca98640 1 -- 172.21.15.5:0/3950828043 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3b0032440 con 0x7fb3b8162820 2024-05-20T18:28:12.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.968+0000 7fb3bca98640 1 -- 172.21.15.5:0/3950828043 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0032810 con 0x7fb3b8162820 2024-05-20T18:28:12.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.969+0000 7fb3bfcfe640 1 -- 172.21.15.5:0/3950828043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3b8162820 msgr2=0x7fb3b80b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:12.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.969+0000 7fb3bfcfe640 1 --2- 172.21.15.5:0/3950828043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3b8162820 0x7fb3b80b3940 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80b7740 tx=0x7fb3b002a1a0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.969+0000 7fb3bfcfe640 1 -- 172.21.15.5:0/3950828043 shutdown_connections 2024-05-20T18:28:12.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.969+0000 7fb3bfcfe640 1 --2- 172.21.15.5:0/3950828043 >> v2:172.21.15.5:3300/0 conn(0x7fb3b806dae0 0x7fb3b80b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.969+0000 7fb3bfcfe640 1 --2- 172.21.15.5:0/3950828043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3b8162820 0x7fb3b80b3940 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.969+0000 7fb3bfcfe640 1 -- 172.21.15.5:0/3950828043 >> 172.21.15.5:0/3950828043 conn(0x7fb3b80d4bd0 msgr2=0x7fb3b80d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:12.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.969+0000 7fb3bfcfe640 1 -- 172.21.15.5:0/3950828043 shutdown_connections 2024-05-20T18:28:12.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.969+0000 7fb3bfcfe640 1 -- 172.21.15.5:0/3950828043 wait complete. 2024-05-20T18:28:12.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.971+0000 7fb3bfcfe640 1 Processor -- start 2024-05-20T18:28:12.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.971+0000 7fb3bfcfe640 1 -- start start 2024-05-20T18:28:12.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.972+0000 7fb3bfcfe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3b80b3e80 0x7fb3b80cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:12.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.972+0000 7fb3bfcfe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b8162820 0x7fb3b80b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:12.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.972+0000 7fb3bfcfe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb3b8159c10 0x7fb3b80b7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:12.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.972+0000 7fb3bfcfe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b80d4780 con 0x7fb3b8162820 2024-05-20T18:28:12.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.972+0000 7fb3bfcfe640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b80d0300 con 0x7fb3b80b3e80 2024-05-20T18:28:12.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.972+0000 7fb3bfcfe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb3b80d4900 con 0x7fb3b8159c10 2024-05-20T18:28:12.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.972+0000 7fb3bd299640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b8162820 0x7fb3b80b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:12.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.973+0000 7fb3bd299640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b8162820 0x7fb3b80b6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46326/0 (socket says 172.21.15.5:46326) 2024-05-20T18:28:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.973+0000 7fb3bd299640 1 -- 172.21.15.5:0/121395202 learned_addr learned my addr 172.21.15.5:0/121395202 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.973+0000 7fb3bda9a640 1 --2- 172.21.15.5:0/121395202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3b80b3e80 0x7fb3b80cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.973+0000 7fb3be29b640 1 --2- 172.21.15.5:0/121395202 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb3b8159c10 0x7fb3b80b7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.973+0000 7fb3bd299640 1 -- 172.21.15.5:0/121395202 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb3b8159c10 msgr2=0x7fb3b80b7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.973+0000 7fb3bd299640 1 --2- 172.21.15.5:0/121395202 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb3b8159c10 0x7fb3b80b7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.973+0000 7fb3bd299640 1 -- 172.21.15.5:0/121395202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3b80b3e80 msgr2=0x7fb3b80cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.973+0000 7fb3bd299640 1 --2- 172.21.15.5:0/121395202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3b80b3e80 0x7fb3b80cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.973+0000 7fb3bd299640 1 -- 172.21.15.5:0/121395202 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b80b4f40 con 0x7fb3b8162820 2024-05-20T18:28:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.973+0000 7fb3bd299640 1 --2- 172.21.15.5:0/121395202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b8162820 0x7fb3b80b6af0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00c410 tx=0x7fb3ac00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.973+0000 7fb3b67fc640 1 -- 172.21.15.5:0/121395202 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac017070 con 0x7fb3b8162820 2024-05-20T18:28:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.973+0000 7fb3b67fc640 1 -- 172.21.15.5:0/121395202 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3ac0035f0 con 0x7fb3b8162820 2024-05-20T18:28:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.973+0000 7fb3bfcfe640 1 -- 172.21.15.5:0/121395202 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b80b5220 con 0x7fb3b8162820 2024-05-20T18:28:12.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.973+0000 7fb3b67fc640 1 -- 172.21.15.5:0/121395202 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac0039e0 con 0x7fb3b8162820 2024-05-20T18:28:12.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.973+0000 7fb3bfcfe640 1 -- 172.21.15.5:0/121395202 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b80b57e0 con 0x7fb3b8162820 2024-05-20T18:28:12.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.974+0000 7fb3bfcfe640 1 -- 172.21.15.5:0/121395202 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3b8075b00 con 0x7fb3b8162820 2024-05-20T18:28:12.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.975+0000 7fb3b67fc640 1 -- 172.21.15.5:0/121395202 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb3ac01a070 con 0x7fb3b8162820 2024-05-20T18:28:12.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.976+0000 7fb3b67fc640 1 --2- 172.21.15.5:0/121395202 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb3a406be30 0x7fb3a406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:12.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.976+0000 7fb3bda9a640 1 --2- 172.21.15.5:0/121395202 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb3a406be30 0x7fb3a406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:12.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.976+0000 7fb3b67fc640 1 -- 172.21.15.5:0/121395202 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6829+0+0 (secure 0 0 0) 0x7fb3ac08a670 con 0x7fb3b8162820 2024-05-20T18:28:12.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.976+0000 7fb3bda9a640 1 --2- 172.21.15.5:0/121395202 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb3a406be30 0x7fb3a406e340 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0034000 tx=0x7fb3b002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:12.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:12.981+0000 7fb3b67fc640 1 -- 172.21.15.5:0/121395202 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb3ac058570 con 0x7fb3b8162820 2024-05-20T18:28:13.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.202+0000 7fb3bfcfe640 1 -- 172.21.15.5:0/121395202 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb3b80b3850 con 0x7fb3b8162820 2024-05-20T18:28:13.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.835+0000 7fb3b67fc640 1 -- 172.21.15.5:0/121395202 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7fb3ac05c870 con 0x7fb3b8162820 2024-05-20T18:28:13.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.837+0000 7fb3bfcfe640 1 -- 172.21.15.5:0/121395202 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb3a406be30 msgr2=0x7fb3a406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:13.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.837+0000 7fb3bfcfe640 1 --2- 172.21.15.5:0/121395202 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb3a406be30 0x7fb3a406e340 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0034000 tx=0x7fb3b002e000 comp rx=0 tx=0).stop 2024-05-20T18:28:13.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.837+0000 7fb3bfcfe640 1 -- 172.21.15.5:0/121395202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b8162820 msgr2=0x7fb3b80b6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:13.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.837+0000 7fb3bfcfe640 1 --2- 172.21.15.5:0/121395202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b8162820 0x7fb3b80b6af0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00c410 tx=0x7fb3ac00c8d0 comp rx=0 tx=0).stop 2024-05-20T18:28:13.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.837+0000 7fb3bfcfe640 1 -- 172.21.15.5:0/121395202 shutdown_connections 2024-05-20T18:28:13.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.837+0000 7fb3bfcfe640 1 --2- 172.21.15.5:0/121395202 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb3a406be30 0x7fb3a406e340 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:13.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.837+0000 7fb3bfcfe640 1 --2- 172.21.15.5:0/121395202 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb3b8159c10 0x7fb3b80b7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:13.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.837+0000 7fb3bfcfe640 1 --2- 172.21.15.5:0/121395202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b8162820 0x7fb3b80b6af0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:13.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.837+0000 7fb3bfcfe640 1 --2- 172.21.15.5:0/121395202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb3b80b3e80 0x7fb3b80cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:13.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.837+0000 7fb3bfcfe640 1 -- 172.21.15.5:0/121395202 >> 172.21.15.5:0/121395202 conn(0x7fb3b80d4bd0 msgr2=0x7fb3b81615c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:13.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.837+0000 7fb3bfcfe640 1 -- 172.21.15.5:0/121395202 shutdown_connections 2024-05-20T18:28:13.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.837+0000 7fb3bfcfe640 1 -- 172.21.15.5:0/121395202 wait complete. 2024-05-20T18:28:13.851 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-20T18:28:13.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.976+0000 7f059cfaa640 1 Processor -- start 2024-05-20T18:28:13.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.976+0000 7f059cfaa640 1 -- start start 2024-05-20T18:28:13.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.977+0000 7f059cfaa640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f05980b0000 0x7f05980d1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:13.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.977+0000 7f059cfaa640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05980aeb50 con 0x7f05980afbf0 2024-05-20T18:28:13.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.977+0000 7f059cfaa640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05980aee30 con 0x7f05980b3a30 2024-05-20T18:28:13.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.977+0000 7f059cfaa640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f059815f920 con 0x7f05980b0000 2024-05-20T18:28:13.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.977+0000 7f0596d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f05980b0000 0x7f05980d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:13.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.977+0000 7f0595d9b640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f05980afbf0 0x7f05980d0930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35744/0 (socket says 172.21.15.5:35744) 2024-05-20T18:28:13.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.977+0000 7f0595d9b640 1 -- 172.21.15.5:0/2171969802 learned_addr learned my addr 172.21.15.5:0/2171969802 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:13.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.978+0000 7f0596d9d640 1 -- 172.21.15.5:0/2171969802 >> v1:172.21.15.63:6789/0 conn(0x7f05980b3a30 legacy=0x7f059806b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:13.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.978+0000 7f0596d9d640 1 -- 172.21.15.5:0/2171969802 >> v1:172.21.15.5:6789/0 conn(0x7f05980afbf0 legacy=0x7f05980d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:13.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.978+0000 7f0596d9d640 1 -- 172.21.15.5:0/2171969802 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05980af110 con 0x7f05980b0000 2024-05-20T18:28:13.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.978+0000 7f059559a640 1 -- 172.21.15.5:0/2171969802 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3647672563 0 0) 0x7f05980aeb50 con 0x7f05980afbf0 2024-05-20T18:28:13.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.978+0000 7f0596d9d640 1 --2- 172.21.15.5:0/2171969802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05980b0000 0x7f05980d1040 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0590009d90 tx=0x7f059002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d31d63d713103 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:13.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.979+0000 7f059559a640 1 -- 172.21.15.5:0/2171969802 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590007740 con 0x7f05980b0000 2024-05-20T18:28:13.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.979+0000 7f059559a640 1 -- 172.21.15.5:0/2171969802 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05900078e0 con 0x7f05980b0000 2024-05-20T18:28:13.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.979+0000 7f059559a640 1 -- 172.21.15.5:0/2171969802 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590007cb0 con 0x7f05980b0000 2024-05-20T18:28:13.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.979+0000 7f059cfaa640 1 -- 172.21.15.5:0/2171969802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05980b0000 msgr2=0x7f05980d1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:13.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.979+0000 7f059cfaa640 1 --2- 172.21.15.5:0/2171969802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05980b0000 0x7f05980d1040 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0590009d90 tx=0x7f059002a6f0 comp rx=0 tx=0).stop 2024-05-20T18:28:13.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.979+0000 7f059cfaa640 1 -- 172.21.15.5:0/2171969802 shutdown_connections 2024-05-20T18:28:13.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.979+0000 7f059cfaa640 1 --2- 172.21.15.5:0/2171969802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05980b0000 0x7f05980d1040 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:13.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.979+0000 7f059cfaa640 1 -- 172.21.15.5:0/2171969802 >> 172.21.15.5:0/2171969802 conn(0x7f05980b6630 msgr2=0x7f05980b6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:13.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.979+0000 7f059cfaa640 1 -- 172.21.15.5:0/2171969802 shutdown_connections 2024-05-20T18:28:13.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.980+0000 7f059cfaa640 1 -- 172.21.15.5:0/2171969802 wait complete. 2024-05-20T18:28:13.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.981+0000 7f059cfaa640 1 Processor -- start 2024-05-20T18:28:13.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.982+0000 7f059cfaa640 1 -- start start 2024-05-20T18:28:13.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.983+0000 7f059cfaa640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05980afbf0 0x7f0598152d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:13.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.983+0000 7f059cfaa640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05980b0000 0x7f0598161cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:13.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.983+0000 7f059cfaa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05980b3a30 0x7f0598162ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:13.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.983+0000 7f059cfaa640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f05980af6d0 con 0x7f05980b3a30 2024-05-20T18:28:13.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.983+0000 7f059cfaa640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f05980b3e40 con 0x7f05980afbf0 2024-05-20T18:28:13.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.983+0000 7f059cfaa640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f059806bc30 con 0x7f05980b0000 2024-05-20T18:28:13.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.983+0000 7f0596d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05980b3a30 0x7f0598162ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:13.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.983+0000 7f0596d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05980b3a30 0x7f0598162ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46350/0 (socket says 172.21.15.5:46350) 2024-05-20T18:28:13.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.983+0000 7f0596d9d640 1 -- 172.21.15.5:0/1035679236 learned_addr learned my addr 172.21.15.5:0/1035679236 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:13.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.983+0000 7f0595d9b640 1 --2- 172.21.15.5:0/1035679236 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05980b0000 0x7f0598161cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:13.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.983+0000 7f059659c640 1 --2- 172.21.15.5:0/1035679236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05980afbf0 0x7f0598152d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:13.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.983+0000 7f0596d9d640 1 -- 172.21.15.5:0/1035679236 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05980b0000 msgr2=0x7f0598161cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.983+0000 7f0596d9d640 1 --2- 172.21.15.5:0/1035679236 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05980b0000 0x7f0598161cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.983+0000 7f0596d9d640 1 -- 172.21.15.5:0/1035679236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05980afbf0 msgr2=0x7f0598152d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.983+0000 7f0596d9d640 1 --2- 172.21.15.5:0/1035679236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05980afbf0 0x7f0598152d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.983+0000 7f0596d9d640 1 -- 172.21.15.5:0/1035679236 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05981532c0 con 0x7f05980b3a30 2024-05-20T18:28:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.984+0000 7f0596d9d640 1 --2- 172.21.15.5:0/1035679236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05980b3a30 0x7f0598162ce0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0590009d90 tx=0x7f0590004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.984+0000 7f0586ffd640 1 -- 172.21.15.5:0/1035679236 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059003f070 con 0x7f05980b3a30 2024-05-20T18:28:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.984+0000 7f0586ffd640 1 -- 172.21.15.5:0/1035679236 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0590031c90 con 0x7f05980b3a30 2024-05-20T18:28:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.984+0000 7f0586ffd640 1 -- 172.21.15.5:0/1035679236 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590043630 con 0x7f05980b3a30 2024-05-20T18:28:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.984+0000 7f059cfaa640 1 -- 172.21.15.5:0/1035679236 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f059809f540 con 0x7f05980b3a30 2024-05-20T18:28:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.984+0000 7f059cfaa640 1 -- 172.21.15.5:0/1035679236 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f059809b8e0 con 0x7f05980b3a30 2024-05-20T18:28:13.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.985+0000 7f059cfaa640 1 -- 172.21.15.5:0/1035679236 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05980690b0 con 0x7f05980b3a30 2024-05-20T18:28:13.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.986+0000 7f0586ffd640 1 -- 172.21.15.5:0/1035679236 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f059003b410 con 0x7f05980b3a30 2024-05-20T18:28:13.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.986+0000 7f0586ffd640 1 --2- 172.21.15.5:0/1035679236 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f057806be30 0x7f057806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:13.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.986+0000 7f0586ffd640 1 -- 172.21.15.5:0/1035679236 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6829+0+0 (secure 0 0 0) 0x7f05900ac990 con 0x7f05980b3a30 2024-05-20T18:28:13.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.986+0000 7f059659c640 1 --2- 172.21.15.5:0/1035679236 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f057806be30 0x7f057806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:13.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.987+0000 7f059659c640 1 --2- 172.21.15.5:0/1035679236 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f057806be30 0x7f057806e340 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0588000f60 tx=0x7f0588009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:13.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:13.991+0000 7f0586ffd640 1 -- 172.21.15.5:0/1035679236 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f05981532c0 con 0x7f05980b3a30 2024-05-20T18:28:14.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.173+0000 7f059cfaa640 1 -- 172.21.15.5:0/1035679236 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0598163ba0 con 0x7f05980b3a30 2024-05-20T18:28:14.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.836+0000 7f0586ffd640 1 -- 172.21.15.5:0/1035679236 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v30) ==== 181+0+0 (secure 0 0 0) 0x7f059007a8c0 con 0x7f05980b3a30 2024-05-20T18:28:14.836 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-05-20T18:28:14.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.840+0000 7f059cfaa640 1 -- 172.21.15.5:0/1035679236 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f057806be30 msgr2=0x7f057806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:14.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.840+0000 7f059cfaa640 1 --2- 172.21.15.5:0/1035679236 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f057806be30 0x7f057806e340 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0588000f60 tx=0x7f0588009040 comp rx=0 tx=0).stop 2024-05-20T18:28:14.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.840+0000 7f059cfaa640 1 -- 172.21.15.5:0/1035679236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05980b3a30 msgr2=0x7f0598162ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:14.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.840+0000 7f059cfaa640 1 --2- 172.21.15.5:0/1035679236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05980b3a30 0x7f0598162ce0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0590009d90 tx=0x7f0590004a40 comp rx=0 tx=0).stop 2024-05-20T18:28:14.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.840+0000 7f059cfaa640 1 -- 172.21.15.5:0/1035679236 shutdown_connections 2024-05-20T18:28:14.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.840+0000 7f059cfaa640 1 --2- 172.21.15.5:0/1035679236 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f057806be30 0x7f057806e340 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:14.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.840+0000 7f059cfaa640 1 --2- 172.21.15.5:0/1035679236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05980b3a30 0x7f0598162ce0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:14.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.840+0000 7f059cfaa640 1 --2- 172.21.15.5:0/1035679236 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05980b0000 0x7f0598161cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:14.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.840+0000 7f059cfaa640 1 --2- 172.21.15.5:0/1035679236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05980afbf0 0x7f0598152d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:14.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.840+0000 7f059cfaa640 1 -- 172.21.15.5:0/1035679236 >> 172.21.15.5:0/1035679236 conn(0x7f05980b6630 msgr2=0x7f05980d1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:14.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.840+0000 7f059cfaa640 1 -- 172.21.15.5:0/1035679236 shutdown_connections 2024-05-20T18:28:14.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.840+0000 7f059cfaa640 1 -- 172.21.15.5:0/1035679236 wait complete. 2024-05-20T18:28:14.854 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-20T18:28:14.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.980+0000 7fbb55ed7640 1 Processor -- start 2024-05-20T18:28:14.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.980+0000 7fbb55ed7640 1 -- start start 2024-05-20T18:28:14.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.981+0000 7fbb55ed7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb5008c540 0x7fbb5015d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:14.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.981+0000 7fbb55ed7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbb5008cdd0 0x7fbb500cd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:14.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.981+0000 7fbb55ed7640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb500d0910 con 0x7fbb500b82f0 2024-05-20T18:28:14.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.981+0000 7fbb55ed7640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fbb500d0bf0 con 0x7fbb5008cdd0 2024-05-20T18:28:14.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.981+0000 7fbb55ed7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbb500d0ed0 con 0x7fbb5008c540 2024-05-20T18:28:14.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.981+0000 7fbb4effd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb5008c540 0x7fbb5015d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:14.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.981+0000 7fbb4ffff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbb5008cdd0 0x7fbb500cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:14.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.982+0000 7fbb4ffff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbb5008cdd0 0x7fbb500cd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:45734/0 (socket says 172.21.15.5:45734) 2024-05-20T18:28:14.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.982+0000 7fbb4effd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb5008c540 0x7fbb5015d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:55894/0 (socket says 172.21.15.5:55894) 2024-05-20T18:28:14.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.982+0000 7fbb4effd640 1 -- 172.21.15.5:0/1996309044 learned_addr learned my addr 172.21.15.5:0/1996309044 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:14.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.982+0000 7fbb4e7fc640 1 -- 172.21.15.5:0/1996309044 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3937077017 0 0) 0x7fbb500d0910 con 0x7fbb500b82f0 2024-05-20T18:28:14.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.982+0000 7fbb4e7fc640 1 -- 172.21.15.5:0/1996309044 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb38003520 con 0x7fbb500b82f0 2024-05-20T18:28:14.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.982+0000 7fbb4effd640 1 -- 172.21.15.5:0/1996309044 >> v2:172.21.15.63:3300/0 conn(0x7fbb5008cdd0 msgr2=0x7fbb500cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:14.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.982+0000 7fbb4effd640 1 --2- 172.21.15.5:0/1996309044 >> v2:172.21.15.63:3300/0 conn(0x7fbb5008cdd0 0x7fbb500cd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:14.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.982+0000 7fbb4effd640 1 -- 172.21.15.5:0/1996309044 >> v1:172.21.15.63:6789/0 conn(0x7fbb500b82f0 legacy=0x7fbb5008be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:14.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.982+0000 7fbb4effd640 1 -- 172.21.15.5:0/1996309044 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb500d11b0 con 0x7fbb5008c540 2024-05-20T18:28:14.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.982+0000 7fbb4ffff640 1 --2- 172.21.15.5:0/1996309044 >> v2:172.21.15.63:3300/0 conn(0x7fbb5008cdd0 0x7fbb500cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:28:14.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.983+0000 7fbb4effd640 1 --2- 172.21.15.5:0/1996309044 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb5008c540 0x7fbb5015d9b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbb44009870 tx=0x7fbb4402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d240befa6874cc11 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:14.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.983+0000 7fbb4e7fc640 1 -- 172.21.15.5:0/1996309044 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb44037070 con 0x7fbb5008c540 2024-05-20T18:28:14.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.983+0000 7fbb4e7fc640 1 -- 172.21.15.5:0/1996309044 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb44032440 con 0x7fbb5008c540 2024-05-20T18:28:14.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.983+0000 7fbb4e7fc640 1 -- 172.21.15.5:0/1996309044 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb44032810 con 0x7fbb5008c540 2024-05-20T18:28:14.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.984+0000 7fbb55ed7640 1 -- 172.21.15.5:0/1996309044 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb5008c540 msgr2=0x7fbb5015d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:14.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.984+0000 7fbb55ed7640 1 --2- 172.21.15.5:0/1996309044 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb5008c540 0x7fbb5015d9b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbb44009870 tx=0x7fbb4402a1c0 comp rx=0 tx=0).stop 2024-05-20T18:28:14.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.984+0000 7fbb55ed7640 1 -- 172.21.15.5:0/1996309044 shutdown_connections 2024-05-20T18:28:14.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.984+0000 7fbb55ed7640 1 --2- 172.21.15.5:0/1996309044 >> v2:172.21.15.63:3300/0 conn(0x7fbb5008cdd0 0x7fbb500cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:14.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.984+0000 7fbb55ed7640 1 --2- 172.21.15.5:0/1996309044 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb5008c540 0x7fbb5015d9b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:14.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.984+0000 7fbb55ed7640 1 -- 172.21.15.5:0/1996309044 >> 172.21.15.5:0/1996309044 conn(0x7fbb500b44c0 msgr2=0x7fbb500b48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:14.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.984+0000 7fbb55ed7640 1 -- 172.21.15.5:0/1996309044 shutdown_connections 2024-05-20T18:28:14.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.984+0000 7fbb55ed7640 1 -- 172.21.15.5:0/1996309044 wait complete. 2024-05-20T18:28:14.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.986+0000 7fbb55ed7640 1 Processor -- start 2024-05-20T18:28:14.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.986+0000 7fbb55ed7640 1 -- start start 2024-05-20T18:28:14.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.987+0000 7fbb55ed7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb5008cdd0 0x7fbb501500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:14.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.987+0000 7fbb55ed7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb500b82f0 0x7fbb500cec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:14.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.987+0000 7fbb55ed7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb500b5ea0 0x7fbb500b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:14.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.987+0000 7fbb55ed7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbb500d1b30 con 0x7fbb500b5ea0 2024-05-20T18:28:14.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.987+0000 7fbb55ed7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbb500d19b0 con 0x7fbb5008cdd0 2024-05-20T18:28:14.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.987+0000 7fbb55ed7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbb500cdc60 con 0x7fbb500b82f0 2024-05-20T18:28:14.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.987+0000 7fbb4ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb500b5ea0 0x7fbb500b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:14.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.987+0000 7fbb4ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb500b5ea0 0x7fbb500b62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37004/0 (socket says 172.21.15.5:37004) 2024-05-20T18:28:14.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.987+0000 7fbb4ffff640 1 -- 172.21.15.5:0/125410762 learned_addr learned my addr 172.21.15.5:0/125410762 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:14.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.988+0000 7fbb4effd640 1 --2- 172.21.15.5:0/125410762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb500b82f0 0x7fbb500cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:14.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.988+0000 7fbb4f7fe640 1 --2- 172.21.15.5:0/125410762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb5008cdd0 0x7fbb501500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:14.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.988+0000 7fbb4ffff640 1 -- 172.21.15.5:0/125410762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb500b82f0 msgr2=0x7fbb500cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:14.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.988+0000 7fbb4ffff640 1 --2- 172.21.15.5:0/125410762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb500b82f0 0x7fbb500cec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:14.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.988+0000 7fbb4ffff640 1 -- 172.21.15.5:0/125410762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb5008cdd0 msgr2=0x7fbb501500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:14.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.988+0000 7fbb4ffff640 1 --2- 172.21.15.5:0/125410762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb5008cdd0 0x7fbb501500d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:14.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.988+0000 7fbb4ffff640 1 -- 172.21.15.5:0/125410762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5015ee20 con 0x7fbb500b5ea0 2024-05-20T18:28:14.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.988+0000 7fbb4ffff640 1 --2- 172.21.15.5:0/125410762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb500b5ea0 0x7fbb500b62b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbb48009e00 tx=0x7fbb4800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:14.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.988+0000 7fbb37fff640 1 -- 172.21.15.5:0/125410762 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb48018020 con 0x7fbb500b5ea0 2024-05-20T18:28:14.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.988+0000 7fbb37fff640 1 -- 172.21.15.5:0/125410762 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb48014480 con 0x7fbb500b5ea0 2024-05-20T18:28:14.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.988+0000 7fbb55ed7640 1 -- 172.21.15.5:0/125410762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb500b70b0 con 0x7fbb500b5ea0 2024-05-20T18:28:14.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.988+0000 7fbb37fff640 1 -- 172.21.15.5:0/125410762 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb48014870 con 0x7fbb500b5ea0 2024-05-20T18:28:14.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.988+0000 7fbb55ed7640 1 -- 172.21.15.5:0/125410762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb500cf150 con 0x7fbb500b5ea0 2024-05-20T18:28:14.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.989+0000 7fbb55ed7640 1 -- 172.21.15.5:0/125410762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb500690b0 con 0x7fbb500b5ea0 2024-05-20T18:28:14.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.990+0000 7fbb37fff640 1 -- 172.21.15.5:0/125410762 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbb48005d10 con 0x7fbb500b5ea0 2024-05-20T18:28:14.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.990+0000 7fbb37fff640 1 --2- 172.21.15.5:0/125410762 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbb3c06be30 0x7fbb3c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:14.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.990+0000 7fbb37fff640 1 -- 172.21.15.5:0/125410762 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6391+0+0 (secure 0 0 0) 0x7fbb4808bea0 con 0x7fbb500b5ea0 2024-05-20T18:28:14.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.991+0000 7fbb4f7fe640 1 --2- 172.21.15.5:0/125410762 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbb3c06be30 0x7fbb3c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:14.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.991+0000 7fbb4f7fe640 1 --2- 172.21.15.5:0/125410762 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbb3c06be30 0x7fbb3c06e340 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbb40000fb0 tx=0x7fbb40009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:14.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:14.995+0000 7fbb37fff640 1 -- 172.21.15.5:0/125410762 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbb48022030 con 0x7fbb500b5ea0 2024-05-20T18:28:15.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.179+0000 7fbb55ed7640 1 -- 172.21.15.5:0/125410762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbb500b53d0 con 0x7fbb500b5ea0 2024-05-20T18:28:15.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.841+0000 7fbb37fff640 1 -- 172.21.15.5:0/125410762 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v31) ==== 169+0+0 (secure 0 0 0) 0x7fbb48010040 con 0x7fbb500b5ea0 2024-05-20T18:28:15.841 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-05-20T18:28:15.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.846+0000 7fbb55ed7640 1 -- 172.21.15.5:0/125410762 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbb3c06be30 msgr2=0x7fbb3c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:15.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.846+0000 7fbb55ed7640 1 --2- 172.21.15.5:0/125410762 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbb3c06be30 0x7fbb3c06e340 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbb40000fb0 tx=0x7fbb40009040 comp rx=0 tx=0).stop 2024-05-20T18:28:15.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.846+0000 7fbb55ed7640 1 -- 172.21.15.5:0/125410762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb500b5ea0 msgr2=0x7fbb500b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:15.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.846+0000 7fbb55ed7640 1 --2- 172.21.15.5:0/125410762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb500b5ea0 0x7fbb500b62b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbb48009e00 tx=0x7fbb4800c490 comp rx=0 tx=0).stop 2024-05-20T18:28:15.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.846+0000 7fbb55ed7640 1 -- 172.21.15.5:0/125410762 shutdown_connections 2024-05-20T18:28:15.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.846+0000 7fbb55ed7640 1 --2- 172.21.15.5:0/125410762 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbb3c06be30 0x7fbb3c06e340 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:15.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.846+0000 7fbb55ed7640 1 --2- 172.21.15.5:0/125410762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb500b5ea0 0x7fbb500b62b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:15.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.846+0000 7fbb55ed7640 1 --2- 172.21.15.5:0/125410762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb500b82f0 0x7fbb500cec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:15.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.846+0000 7fbb55ed7640 1 --2- 172.21.15.5:0/125410762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb5008cdd0 0x7fbb501500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:15.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.846+0000 7fbb55ed7640 1 -- 172.21.15.5:0/125410762 >> 172.21.15.5:0/125410762 conn(0x7fbb500b44c0 msgr2=0x7fbb500af620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:15.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.846+0000 7fbb55ed7640 1 -- 172.21.15.5:0/125410762 shutdown_connections 2024-05-20T18:28:15.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.846+0000 7fbb55ed7640 1 -- 172.21.15.5:0/125410762 wait complete. 2024-05-20T18:28:15.861 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-20T18:28:15.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.984+0000 7fb97b384640 1 Processor -- start 2024-05-20T18:28:15.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.984+0000 7fb97b384640 1 -- start start 2024-05-20T18:28:15.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.985+0000 7fb97b384640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb9740d3510 0x7fb9740b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:15.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.985+0000 7fb97b384640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9740cfb80 0x7fb97406cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:15.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.985+0000 7fb97b384640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9740b4300 con 0x7fb9740cf060 2024-05-20T18:28:15.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.985+0000 7fb97b384640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fb97415fac0 con 0x7fb9740d3510 2024-05-20T18:28:15.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.985+0000 7fb97b384640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb97406e090 con 0x7fb9740cfb80 2024-05-20T18:28:15.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.985+0000 7fb979120640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb9740d3510 0x7fb9740b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:15.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.985+0000 7fb97891f640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb9740cf060 0x7fb9740cf470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45740/0 (socket says 172.21.15.5:45740) 2024-05-20T18:28:15.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.985+0000 7fb97891f640 1 -- 172.21.15.5:0/862291201 learned_addr learned my addr 172.21.15.5:0/862291201 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:15.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.985+0000 7fb979120640 1 --2- 172.21.15.5:0/862291201 >> v2:172.21.15.5:3300/0 conn(0x7fb9740d3510 0x7fb9740b3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37012/0 (socket says 172.21.15.5:37012) 2024-05-20T18:28:15.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.985+0000 7fb979921640 1 --2- 172.21.15.5:0/862291201 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9740cfb80 0x7fb97406cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:15.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.986+0000 7fb979120640 1 -- 172.21.15.5:0/862291201 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9740cfb80 msgr2=0x7fb97406cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:15.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.986+0000 7fb979120640 1 --2- 172.21.15.5:0/862291201 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9740cfb80 0x7fb97406cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:15.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.986+0000 7fb979120640 1 -- 172.21.15.5:0/862291201 >> v1:172.21.15.5:6789/0 conn(0x7fb9740cf060 legacy=0x7fb9740cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:15.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.986+0000 7fb973fff640 1 -- 172.21.15.5:0/862291201 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 700665783 0 0) 0x7fb968001110 con 0x7fb9740cf060 2024-05-20T18:28:15.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.986+0000 7fb979120640 1 -- 172.21.15.5:0/862291201 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9740d14a0 con 0x7fb9740d3510 2024-05-20T18:28:15.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.986+0000 7fb979120640 1 --2- 172.21.15.5:0/862291201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9740d3510 0x7fb9740b3dc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0097a0 tx=0x7fb96c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15eea8a57749c87d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:15.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.987+0000 7fb973fff640 1 -- 172.21.15.5:0/862291201 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c037070 con 0x7fb9740d3510 2024-05-20T18:28:15.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.987+0000 7fb973fff640 1 -- 172.21.15.5:0/862291201 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb96c02ae50 con 0x7fb9740d3510 2024-05-20T18:28:15.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.987+0000 7fb973fff640 1 -- 172.21.15.5:0/862291201 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c032660 con 0x7fb9740d3510 2024-05-20T18:28:15.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.987+0000 7fb97b384640 1 -- 172.21.15.5:0/862291201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9740d3510 msgr2=0x7fb9740b3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:15.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.987+0000 7fb97b384640 1 --2- 172.21.15.5:0/862291201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9740d3510 0x7fb9740b3dc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0097a0 tx=0x7fb96c02a100 comp rx=0 tx=0).stop 2024-05-20T18:28:15.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.987+0000 7fb97b384640 1 -- 172.21.15.5:0/862291201 shutdown_connections 2024-05-20T18:28:15.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.987+0000 7fb97b384640 1 --2- 172.21.15.5:0/862291201 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9740cfb80 0x7fb97406cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:15.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.987+0000 7fb97b384640 1 --2- 172.21.15.5:0/862291201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9740d3510 0x7fb9740b3dc0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:15.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.987+0000 7fb97b384640 1 -- 172.21.15.5:0/862291201 >> 172.21.15.5:0/862291201 conn(0x7fb9740d29f0 msgr2=0x7fb9740d2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:15.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.988+0000 7fb97b384640 1 -- 172.21.15.5:0/862291201 shutdown_connections 2024-05-20T18:28:15.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.988+0000 7fb97b384640 1 -- 172.21.15.5:0/862291201 wait complete. 2024-05-20T18:28:15.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.989+0000 7fb97b384640 1 Processor -- start 2024-05-20T18:28:15.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.989+0000 7fb97b384640 1 -- start start 2024-05-20T18:28:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.991+0000 7fb97b384640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9740cf060 0x7fb97415da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.991+0000 7fb97b384640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9740d3510 0x7fb97415ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.991+0000 7fb97b384640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9740b0de0 0x7fb974162ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.991+0000 7fb97b384640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb97406d1e0 con 0x7fb9740b0de0 2024-05-20T18:28:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.991+0000 7fb97b384640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb9740d1af0 con 0x7fb9740d3510 2024-05-20T18:28:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.991+0000 7fb97b384640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb97406d360 con 0x7fb9740cf060 2024-05-20T18:28:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.991+0000 7fb979921640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9740b0de0 0x7fb974162ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.991+0000 7fb979921640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9740b0de0 0x7fb974162ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37026/0 (socket says 172.21.15.5:37026) 2024-05-20T18:28:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.991+0000 7fb979921640 1 -- 172.21.15.5:0/384555583 learned_addr learned my addr 172.21.15.5:0/384555583 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.991+0000 7fb979120640 1 --2- 172.21.15.5:0/384555583 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9740cf060 0x7fb97415da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.991+0000 7fb97891f640 1 --2- 172.21.15.5:0/384555583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9740d3510 0x7fb97415ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.991+0000 7fb979921640 1 -- 172.21.15.5:0/384555583 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9740cf060 msgr2=0x7fb97415da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.991+0000 7fb979921640 1 --2- 172.21.15.5:0/384555583 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9740cf060 0x7fb97415da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.991+0000 7fb979921640 1 -- 172.21.15.5:0/384555583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9740d3510 msgr2=0x7fb97415ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.991+0000 7fb979921640 1 --2- 172.21.15.5:0/384555583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9740d3510 0x7fb97415ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:15.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.991+0000 7fb979921640 1 -- 172.21.15.5:0/384555583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9740b1530 con 0x7fb9740b0de0 2024-05-20T18:28:15.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.991+0000 7fb979921640 1 --2- 172.21.15.5:0/384555583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9740b0de0 0x7fb974162ca0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb96400efc0 tx=0x7fb96400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:15.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.992+0000 7fb971ffb640 1 -- 172.21.15.5:0/384555583 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964018020 con 0x7fb9740b0de0 2024-05-20T18:28:15.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.992+0000 7fb971ffb640 1 -- 172.21.15.5:0/384555583 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb964014480 con 0x7fb9740b0de0 2024-05-20T18:28:15.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.992+0000 7fb97b384640 1 -- 172.21.15.5:0/384555583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9740b1810 con 0x7fb9740b0de0 2024-05-20T18:28:15.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.992+0000 7fb971ffb640 1 -- 172.21.15.5:0/384555583 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964009e60 con 0x7fb9740b0de0 2024-05-20T18:28:15.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.992+0000 7fb97b384640 1 -- 172.21.15.5:0/384555583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9740b1dd0 con 0x7fb9740b0de0 2024-05-20T18:28:15.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.993+0000 7fb97b384640 1 -- 172.21.15.5:0/384555583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9740d2300 con 0x7fb9740b0de0 2024-05-20T18:28:15.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.993+0000 7fb971ffb640 1 -- 172.21.15.5:0/384555583 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb964005d10 con 0x7fb9740b0de0 2024-05-20T18:28:15.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.994+0000 7fb971ffb640 1 --2- 172.21.15.5:0/384555583 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb96006be30 0x7fb96006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:15.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.994+0000 7fb971ffb640 1 -- 172.21.15.5:0/384555583 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5853+0+0 (secure 0 0 0) 0x7fb96408bbd0 con 0x7fb9740b0de0 2024-05-20T18:28:15.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.994+0000 7fb979120640 1 --2- 172.21.15.5:0/384555583 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb96006be30 0x7fb96006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:15.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.994+0000 7fb979120640 1 --2- 172.21.15.5:0/384555583 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb96006be30 0x7fb96006e340 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb96c034000 tx=0x7fb96c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:15.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:15.999+0000 7fb971ffb640 1 -- 172.21.15.5:0/384555583 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb9740d2300 con 0x7fb9740b0de0 2024-05-20T18:28:16.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:16.188+0000 7fb97b384640 1 -- 172.21.15.5:0/384555583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb9740690b0 con 0x7fb9740b0de0 2024-05-20T18:28:16.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:16.845+0000 7fb971ffb640 1 -- 172.21.15.5:0/384555583 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v32) ==== 178+0+0 (secure 0 0 0) 0x7fb964022030 con 0x7fb9740b0de0 2024-05-20T18:28:16.846 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data_ec' removed 2024-05-20T18:28:16.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:16.850+0000 7fb97b384640 1 -- 172.21.15.5:0/384555583 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb96006be30 msgr2=0x7fb96006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:16.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:16.850+0000 7fb97b384640 1 --2- 172.21.15.5:0/384555583 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb96006be30 0x7fb96006e340 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb96c034000 tx=0x7fb96c02e000 comp rx=0 tx=0).stop 2024-05-20T18:28:16.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:16.850+0000 7fb97b384640 1 -- 172.21.15.5:0/384555583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9740b0de0 msgr2=0x7fb974162ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:16.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:16.850+0000 7fb97b384640 1 --2- 172.21.15.5:0/384555583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9740b0de0 0x7fb974162ca0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb96400efc0 tx=0x7fb96400c490 comp rx=0 tx=0).stop 2024-05-20T18:28:16.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:16.851+0000 7fb97b384640 1 -- 172.21.15.5:0/384555583 shutdown_connections 2024-05-20T18:28:16.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:16.851+0000 7fb97b384640 1 --2- 172.21.15.5:0/384555583 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb96006be30 0x7fb96006e340 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:16.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:16.851+0000 7fb97b384640 1 --2- 172.21.15.5:0/384555583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9740b0de0 0x7fb974162ca0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:16.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:16.851+0000 7fb97b384640 1 --2- 172.21.15.5:0/384555583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9740d3510 0x7fb97415ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:16.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:16.851+0000 7fb97b384640 1 --2- 172.21.15.5:0/384555583 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9740cf060 0x7fb97415da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:16.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:16.851+0000 7fb97b384640 1 -- 172.21.15.5:0/384555583 >> 172.21.15.5:0/384555583 conn(0x7fb9740d29f0 msgr2=0x7fb97415e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:16.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:16.851+0000 7fb97b384640 1 -- 172.21.15.5:0/384555583 shutdown_connections 2024-05-20T18:28:16.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:16.851+0000 7fb97b384640 1 -- 172.21.15.5:0/384555583 wait complete. 2024-05-20T18:28:16.868 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-20T18:28:16.869 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-20T18:28:16.869 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-20T18:28:16.869 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:28:16.869 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-20T18:28:16.869 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-20T18:28:16.869 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-20T18:28:16.870 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:28:16.870 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-20T18:28:16.870 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-20T18:28:16.870 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-20T18:28:16.870 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:28:16.870 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-20T18:28:16.870 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-20T18:28:16.871 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-20T18:28:16.871 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:28:16.880 INFO:tasks.ceph.mds.a:Stopped 2024-05-20T18:28:16.880 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-20T18:28:16.881 INFO:tasks.ceph.mds.c:Stopped 2024-05-20T18:28:16.881 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-20T18:28:16.884 INFO:tasks.ceph.mds.d:Stopped 2024-05-20T18:28:16.884 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-20T18:28:16.885 INFO:tasks.ceph.mds.b:Stopped 2024-05-20T18:28:16.885 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-20T18:28:16.923 INFO:tasks.ceph.mds.a:Started 2024-05-20T18:28:16.923 DEBUG:teuthology.parallel:result is None 2024-05-20T18:28:16.923 INFO:tasks.ceph.mds.c:Started 2024-05-20T18:28:16.924 DEBUG:teuthology.parallel:result is None 2024-05-20T18:28:16.927 INFO:tasks.ceph.mds.d:Started 2024-05-20T18:28:16.927 DEBUG:teuthology.parallel:result is None 2024-05-20T18:28:16.928 INFO:tasks.ceph.mds.b:Started 2024-05-20T18:28:16.929 DEBUG:teuthology.parallel:result is None 2024-05-20T18:28:16.929 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-20T18:28:17.025 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-05-20T18:28:17.026 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-05-20T18:28:17.026 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-20T18:28:17.026 INFO:tasks.ceph.mds.d.smithi063.stdout:starting mds.d at 2024-05-20T18:28:17.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.062+0000 7fc7ec738640 1 Processor -- start 2024-05-20T18:28:17.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.063+0000 7fc7ec738640 1 -- start start 2024-05-20T18:28:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.064+0000 7fc7ec738640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7e40b4010 0x7fc7e40d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.064+0000 7fc7ec738640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7e40d0370 con 0x7fc7e4163a10 2024-05-20T18:28:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.064+0000 7fc7ec738640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7e40d0650 con 0x7fc7e40d14b0 2024-05-20T18:28:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.064+0000 7fc7ec738640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e40d3ca0 con 0x7fc7e40b4010 2024-05-20T18:28:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.064+0000 7fc7ea4d4640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc7e4163a10 0x7fc7e40cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45752/0 (socket says 172.21.15.5:45752) 2024-05-20T18:28:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.064+0000 7fc7ea4d4640 1 -- 172.21.15.5:0/2374129525 learned_addr learned my addr 172.21.15.5:0/2374129525 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.065+0000 7fc7e9cd3640 1 --2- 172.21.15.5:0/2374129525 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7e40b4010 0x7fc7e40d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.065+0000 7fc7e94d2640 1 -- 172.21.15.5:0/2374129525 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1513547961 0 0) 0x7fc7e40d0370 con 0x7fc7e4163a10 2024-05-20T18:28:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.065+0000 7fc7e94d2640 1 -- 172.21.15.5:0/2374129525 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7d0003520 con 0x7fc7e4163a10 2024-05-20T18:28:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.065+0000 7fc7e94d2640 1 -- 172.21.15.5:0/2374129525 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2447249150 0 0) 0x7fc7e40d0650 con 0x7fc7e40d14b0 2024-05-20T18:28:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.065+0000 7fc7e94d2640 1 -- 172.21.15.5:0/2374129525 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7e40d0370 con 0x7fc7e40d14b0 2024-05-20T18:28:17.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.065+0000 7fc7e94d2640 1 -- 172.21.15.5:0/2374129525 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2952273856 0 0) 0x7fc7d0003520 con 0x7fc7e4163a10 2024-05-20T18:28:17.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.065+0000 7fc7e94d2640 1 -- 172.21.15.5:0/2374129525 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7e40b4010 msgr2=0x7fc7e40d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:17.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.065+0000 7fc7e94d2640 1 --2- 172.21.15.5:0/2374129525 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7e40b4010 0x7fc7e40d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:17.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.065+0000 7fc7e94d2640 1 -- 172.21.15.5:0/2374129525 >> v1:172.21.15.63:6789/0 conn(0x7fc7e40d14b0 legacy=0x7fc7e40d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:17.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.065+0000 7fc7e94d2640 1 -- 172.21.15.5:0/2374129525 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e40d3650 con 0x7fc7e4163a10 2024-05-20T18:28:17.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.065+0000 7fc7e94d2640 1 -- 172.21.15.5:0/2374129525 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7fc7d8001670 con 0x7fc7e4163a10 2024-05-20T18:28:17.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.065+0000 7fc7e94d2640 1 -- 172.21.15.5:0/2374129525 >> v1:172.21.15.5:6789/0 conn(0x7fc7e4163a10 legacy=0x7fc7e40cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:17.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.065+0000 7fc7e94d2640 1 --2- 172.21.15.5:0/2374129525 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7d0003b70 0x7fc7d0003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:17.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.065+0000 7fc7e94d2640 1 --2- 172.21.15.5:0/2374129525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7d0004800 0x7fc7d0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:17.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.065+0000 7fc7e94d2640 1 --2- 172.21.15.5:0/2374129525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7d0005150 0x7fc7d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:17.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.065+0000 7fc7e94d2640 1 -- 172.21.15.5:0/2374129525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e40d3ca0 con 0x7fc7d0005150 2024-05-20T18:28:17.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.065+0000 7fc7e94d2640 1 -- 172.21.15.5:0/2374129525 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d000c090 con 0x7fc7d0004800 2024-05-20T18:28:17.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.065+0000 7fc7e94d2640 1 -- 172.21.15.5:0/2374129525 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc7d000c290 con 0x7fc7d0003b70 2024-05-20T18:28:17.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.065+0000 7fc7eacd5640 1 --2- 172.21.15.5:0/2374129525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7d0005150 0x7fc7d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:17.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.065+0000 7fc7ea4d4640 1 --2- 172.21.15.5:0/2374129525 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7d0003b70 0x7fc7d0003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:17.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.065+0000 7fc7e9cd3640 1 --2- 172.21.15.5:0/2374129525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7d0004800 0x7fc7d0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:17.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.065+0000 7fc7eacd5640 1 -- 172.21.15.5:0/2374129525 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7d0003b70 msgr2=0x7fc7d0003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:17.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.065+0000 7fc7eacd5640 1 --2- 172.21.15.5:0/2374129525 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7d0003b70 0x7fc7d0003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:17.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.065+0000 7fc7eacd5640 1 -- 172.21.15.5:0/2374129525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7d0004800 msgr2=0x7fc7d0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:17.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.065+0000 7fc7eacd5640 1 --2- 172.21.15.5:0/2374129525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7d0004800 0x7fc7d0004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:17.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.065+0000 7fc7eacd5640 1 -- 172.21.15.5:0/2374129525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7d0003520 con 0x7fc7d0005150 2024-05-20T18:28:17.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.066+0000 7fc7eacd5640 1 --2- 172.21.15.5:0/2374129525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7d0005150 0x7fc7d000b9c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40b45b0 tx=0x7fc7d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6111801c039f07e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:17.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.066+0000 7fc7e94d2640 1 -- 172.21.15.5:0/2374129525 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc001bf0 con 0x7fc7d0005150 2024-05-20T18:28:17.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.066+0000 7fc7e94d2640 1 -- 172.21.15.5:0/2374129525 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7dc003b70 con 0x7fc7d0005150 2024-05-20T18:28:17.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.066+0000 7fc7e94d2640 1 -- 172.21.15.5:0/2374129525 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc0050d0 con 0x7fc7d0005150 2024-05-20T18:28:17.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.067+0000 7fc7ec738640 1 -- 172.21.15.5:0/2374129525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7d0005150 msgr2=0x7fc7d000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:17.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.067+0000 7fc7ec738640 1 --2- 172.21.15.5:0/2374129525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7d0005150 0x7fc7d000b9c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40b45b0 tx=0x7fc7d0005830 comp rx=0 tx=0).stop 2024-05-20T18:28:17.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.067+0000 7fc7eacd5640 1 -- 172.21.15.5:0/2374129525 reap_dead start 2024-05-20T18:28:17.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.067+0000 7fc7ec738640 1 -- 172.21.15.5:0/2374129525 shutdown_connections 2024-05-20T18:28:17.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.067+0000 7fc7ec738640 1 -- 172.21.15.5:0/2374129525 >> 172.21.15.5:0/2374129525 conn(0x7fc7e406d240 msgr2=0x7fc7e406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:17.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.067+0000 7fc7ec738640 1 -- 172.21.15.5:0/2374129525 shutdown_connections 2024-05-20T18:28:17.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.067+0000 7fc7ec738640 1 -- 172.21.15.5:0/2374129525 wait complete. 2024-05-20T18:28:17.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.069+0000 7fc7ec738640 1 Processor -- start 2024-05-20T18:28:17.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.069+0000 7fc7ec738640 1 -- start start 2024-05-20T18:28:17.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.070+0000 7fc7ec738640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7d0005150 0x7fc7e4149de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:17.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.070+0000 7fc7ec738640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7e415a7b0 0x7fc7e40d4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:17.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.070+0000 7fc7ec738640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7e415abc0 0x7fc7e40d5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:17.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.070+0000 7fc7ec738640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e40d42b0 con 0x7fc7e415a7b0 2024-05-20T18:28:17.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.070+0000 7fc7ec738640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e40d4130 con 0x7fc7d0005150 2024-05-20T18:28:17.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.070+0000 7fc7ec738640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e40d4430 con 0x7fc7e415abc0 2024-05-20T18:28:17.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.070+0000 7fc7e9cd3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7e415a7b0 0x7fc7e40d4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:17.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.070+0000 7fc7e9cd3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7e415a7b0 0x7fc7e40d4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37074/0 (socket says 172.21.15.5:37074) 2024-05-20T18:28:17.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.070+0000 7fc7e9cd3640 1 -- 172.21.15.5:0/2392968898 learned_addr learned my addr 172.21.15.5:0/2392968898 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:17.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.071+0000 7fc7eacd5640 1 --2- 172.21.15.5:0/2392968898 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7e415abc0 0x7fc7e40d5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:17.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.071+0000 7fc7ea4d4640 1 --2- 172.21.15.5:0/2392968898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7d0005150 0x7fc7e4149de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:17.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.071+0000 7fc7e9cd3640 1 -- 172.21.15.5:0/2392968898 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7e415abc0 msgr2=0x7fc7e40d5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:17.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.071+0000 7fc7e9cd3640 1 --2- 172.21.15.5:0/2392968898 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7e415abc0 0x7fc7e40d5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:17.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.071+0000 7fc7e9cd3640 1 -- 172.21.15.5:0/2392968898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7d0005150 msgr2=0x7fc7e4149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:17.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.071+0000 7fc7e9cd3640 1 --2- 172.21.15.5:0/2392968898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7d0005150 0x7fc7e4149de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:17.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.071+0000 7fc7e9cd3640 1 -- 172.21.15.5:0/2392968898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e41f8f90 con 0x7fc7e415a7b0 2024-05-20T18:28:17.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.071+0000 7fc7e9cd3640 1 --2- 172.21.15.5:0/2392968898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7e415a7b0 0x7fc7e40d4bc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0011890 tx=0x7fc7e0011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:17.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.071+0000 7fc7d6ffd640 1 -- 172.21.15.5:0/2392968898 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0019070 con 0x7fc7e415a7b0 2024-05-20T18:28:17.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.071+0000 7fc7d6ffd640 1 -- 172.21.15.5:0/2392968898 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7e0004300 con 0x7fc7e415a7b0 2024-05-20T18:28:17.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.071+0000 7fc7d6ffd640 1 -- 172.21.15.5:0/2392968898 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e000fa40 con 0x7fc7e415a7b0 2024-05-20T18:28:17.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.071+0000 7fc7ec738640 1 -- 172.21.15.5:0/2392968898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e41f9270 con 0x7fc7e415a7b0 2024-05-20T18:28:17.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.071+0000 7fc7ec738640 1 -- 172.21.15.5:0/2392968898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e41f9830 con 0x7fc7e415a7b0 2024-05-20T18:28:17.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.072+0000 7fc7ec738640 1 -- 172.21.15.5:0/2392968898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7e4075b00 con 0x7fc7e415a7b0 2024-05-20T18:28:17.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.073+0000 7fc7d6ffd640 1 -- 172.21.15.5:0/2392968898 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc7e001c070 con 0x7fc7e415a7b0 2024-05-20T18:28:17.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.073+0000 7fc7d6ffd640 1 --2- 172.21.15.5:0/2392968898 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc7c806be60 0x7fc7c806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:17.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.073+0000 7fc7d6ffd640 1 -- 172.21.15.5:0/2392968898 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5335+0+0 (secure 0 0 0) 0x7fc7e008b150 con 0x7fc7e415a7b0 2024-05-20T18:28:17.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.073+0000 7fc7ea4d4640 1 --2- 172.21.15.5:0/2392968898 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc7c806be60 0x7fc7c806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:17.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.074+0000 7fc7ea4d4640 1 --2- 172.21.15.5:0/2392968898 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc7c806be60 0x7fc7c806e370 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8001eb0 tx=0x7fc7d8001e40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:17.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.078+0000 7fc7d6ffd640 1 -- 172.21.15.5:0/2392968898 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc7e0059580 con 0x7fc7e415a7b0 2024-05-20T18:28:17.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.280+0000 7fc7ec738640 1 -- 172.21.15.5:0/2392968898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fc7e40d3050 con 0x7fc7e415a7b0 2024-05-20T18:28:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.856+0000 7fc7d6ffd640 1 -- 172.21.15.5:0/2392968898 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v34) ==== 97+0+0 (secure 0 0 0) 0x7fc7e005d880 con 0x7fc7e415a7b0 2024-05-20T18:28:17.855 INFO:teuthology.orchestra.run.smithi005.stderr: removed all blocklist entries 2024-05-20T18:28:17.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.859+0000 7fc7ec738640 1 -- 172.21.15.5:0/2392968898 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc7c806be60 msgr2=0x7fc7c806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:17.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.859+0000 7fc7ec738640 1 --2- 172.21.15.5:0/2392968898 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc7c806be60 0x7fc7c806e370 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8001eb0 tx=0x7fc7d8001e40 comp rx=0 tx=0).stop 2024-05-20T18:28:17.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.859+0000 7fc7ec738640 1 -- 172.21.15.5:0/2392968898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7e415a7b0 msgr2=0x7fc7e40d4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:17.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.859+0000 7fc7ec738640 1 --2- 172.21.15.5:0/2392968898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7e415a7b0 0x7fc7e40d4bc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0011890 tx=0x7fc7e0011d50 comp rx=0 tx=0).stop 2024-05-20T18:28:17.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.859+0000 7fc7ec738640 1 -- 172.21.15.5:0/2392968898 shutdown_connections 2024-05-20T18:28:17.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.859+0000 7fc7ec738640 1 --2- 172.21.15.5:0/2392968898 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc7c806be60 0x7fc7c806e370 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:17.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.859+0000 7fc7ec738640 1 --2- 172.21.15.5:0/2392968898 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7e415abc0 0x7fc7e40d5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:17.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.859+0000 7fc7ec738640 1 --2- 172.21.15.5:0/2392968898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7e415a7b0 0x7fc7e40d4bc0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:17.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.859+0000 7fc7ec738640 1 --2- 172.21.15.5:0/2392968898 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7d0005150 0x7fc7e4149de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:17.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.859+0000 7fc7ec738640 1 -- 172.21.15.5:0/2392968898 >> 172.21.15.5:0/2392968898 conn(0x7fc7e406d240 msgr2=0x7fc7e40cfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:17.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.859+0000 7fc7ec738640 1 -- 172.21.15.5:0/2392968898 shutdown_connections 2024-05-20T18:28:17.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.859+0000 7fc7ec738640 1 -- 172.21.15.5:0/2392968898 wait complete. 2024-05-20T18:28:17.873 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-20T18:28:17.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.994+0000 7f086f5f5640 1 Processor -- start 2024-05-20T18:28:17.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.994+0000 7f086f5f5640 1 -- start start 2024-05-20T18:28:17.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.995+0000 7f086f5f5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0868162820 0x7f08680b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:17.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.995+0000 7f086f5f5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08680b3e80 0x7f086806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:17.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.995+0000 7f086f5f5640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f086806dd20 con 0x7f086806d910 2024-05-20T18:28:17.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.995+0000 7f086f5f5640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f08680d0630 con 0x7f0868162820 2024-05-20T18:28:17.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.995+0000 7f086f5f5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f08680d39c0 con 0x7f08680b3e80 2024-05-20T18:28:17.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.995+0000 7f086d391640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0868162820 0x7f08680b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:17.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.996+0000 7f086d391640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0868162820 0x7f08680b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37088/0 (socket says 172.21.15.5:37088) 2024-05-20T18:28:17.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.996+0000 7f086db92640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f086806d910 0x7f08680d27a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45756/0 (socket says 172.21.15.5:45756) 2024-05-20T18:28:17.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.996+0000 7f086d391640 1 -- 172.21.15.5:0/3918519107 learned_addr learned my addr 172.21.15.5:0/3918519107 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:17.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.996+0000 7f086cb90640 1 --2- 172.21.15.5:0/3918519107 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08680b3e80 0x7f086806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:17.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.996+0000 7f086d391640 1 -- 172.21.15.5:0/3918519107 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08680b3e80 msgr2=0x7f086806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:17.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.996+0000 7f086d391640 1 --2- 172.21.15.5:0/3918519107 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08680b3e80 0x7f086806d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:17.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.996+0000 7f086d391640 1 -- 172.21.15.5:0/3918519107 >> v1:172.21.15.5:6789/0 conn(0x7f086806d910 legacy=0x7f08680d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:17.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.996+0000 7f085bfff640 1 -- 172.21.15.5:0/3918519107 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 440000654 0 0) 0x7f0860001490 con 0x7f086806d910 2024-05-20T18:28:17.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.996+0000 7f086d391640 1 -- 172.21.15.5:0/3918519107 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08680d3ca0 con 0x7f0868162820 2024-05-20T18:28:17.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.997+0000 7f086d391640 1 --2- 172.21.15.5:0/3918519107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0868162820 0x7f08680b3940 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0864009870 tx=0x7f086402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c901edfe19086c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:17.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.997+0000 7f085bfff640 1 -- 172.21.15.5:0/3918519107 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0864037070 con 0x7f0868162820 2024-05-20T18:28:17.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.997+0000 7f085bfff640 1 -- 172.21.15.5:0/3918519107 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0864032440 con 0x7f0868162820 2024-05-20T18:28:17.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.997+0000 7f085bfff640 1 -- 172.21.15.5:0/3918519107 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0864032810 con 0x7f0868162820 2024-05-20T18:28:17.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.997+0000 7f086f5f5640 1 -- 172.21.15.5:0/3918519107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0868162820 msgr2=0x7f08680b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:17.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.997+0000 7f086f5f5640 1 --2- 172.21.15.5:0/3918519107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0868162820 0x7f08680b3940 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0864009870 tx=0x7f086402a270 comp rx=0 tx=0).stop 2024-05-20T18:28:17.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.998+0000 7f086f5f5640 1 -- 172.21.15.5:0/3918519107 shutdown_connections 2024-05-20T18:28:17.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.998+0000 7f086f5f5640 1 --2- 172.21.15.5:0/3918519107 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08680b3e80 0x7f086806d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:17.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.998+0000 7f086f5f5640 1 --2- 172.21.15.5:0/3918519107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0868162820 0x7f08680b3940 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:17.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.998+0000 7f086f5f5640 1 -- 172.21.15.5:0/3918519107 >> 172.21.15.5:0/3918519107 conn(0x7f08680d4bd0 msgr2=0x7f08680d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:17.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.998+0000 7f086f5f5640 1 -- 172.21.15.5:0/3918519107 shutdown_connections 2024-05-20T18:28:17.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.998+0000 7f086f5f5640 1 -- 172.21.15.5:0/3918519107 wait complete. 2024-05-20T18:28:17.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.999+0000 7f086f5f5640 1 Processor -- start 2024-05-20T18:28:17.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:17.999+0000 7f086f5f5640 1 -- start start 2024-05-20T18:28:18.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.001+0000 7f086f5f5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f086806d910 0x7f08680b4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:18.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.001+0000 7f086f5f5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08680b3e80 0x7f08680b6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:18.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.001+0000 7f086f5f5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0868162820 0x7f08680b6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:18.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.001+0000 7f086f5f5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08680d0300 con 0x7f08680b3e80 2024-05-20T18:28:18.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.001+0000 7f086f5f5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f08680d4260 con 0x7f086806d910 2024-05-20T18:28:18.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.001+0000 7f086f5f5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f08680b91a0 con 0x7f0868162820 2024-05-20T18:28:18.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.001+0000 7f086cb90640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08680b3e80 0x7f08680b6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:18.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.001+0000 7f086cb90640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08680b3e80 0x7f08680b6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37092/0 (socket says 172.21.15.5:37092) 2024-05-20T18:28:18.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.001+0000 7f086cb90640 1 -- 172.21.15.5:0/2277342718 learned_addr learned my addr 172.21.15.5:0/2277342718 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:18.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.001+0000 7f086db92640 1 --2- 172.21.15.5:0/2277342718 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0868162820 0x7f08680b6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:18.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.001+0000 7f086d391640 1 --2- 172.21.15.5:0/2277342718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f086806d910 0x7f08680b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:18.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.001+0000 7f086cb90640 1 -- 172.21.15.5:0/2277342718 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0868162820 msgr2=0x7f08680b6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:18.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.001+0000 7f086cb90640 1 --2- 172.21.15.5:0/2277342718 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0868162820 0x7f08680b6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:18.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.001+0000 7f086cb90640 1 -- 172.21.15.5:0/2277342718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f086806d910 msgr2=0x7f08680b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:18.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.001+0000 7f086cb90640 1 --2- 172.21.15.5:0/2277342718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f086806d910 0x7f08680b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:18.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.001+0000 7f086cb90640 1 -- 172.21.15.5:0/2277342718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08680cd780 con 0x7f08680b3e80 2024-05-20T18:28:18.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.001+0000 7f086d391640 1 --2- 172.21.15.5:0/2277342718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f086806d910 0x7f08680b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:18.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.001+0000 7f086cb90640 1 --2- 172.21.15.5:0/2277342718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08680b3e80 0x7f08680b6920 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f085c00d650 tx=0x7f085c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:18.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.002+0000 7f0859ffb640 1 -- 172.21.15.5:0/2277342718 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c00b660 con 0x7f08680b3e80 2024-05-20T18:28:18.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.002+0000 7f0859ffb640 1 -- 172.21.15.5:0/2277342718 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f085c019070 con 0x7f08680b3e80 2024-05-20T18:28:18.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.002+0000 7f086f5f5640 1 -- 172.21.15.5:0/2277342718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08680cda60 con 0x7f08680b3e80 2024-05-20T18:28:18.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.002+0000 7f0859ffb640 1 -- 172.21.15.5:0/2277342718 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c00ba40 con 0x7f08680b3e80 2024-05-20T18:28:18.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.002+0000 7f086f5f5640 1 -- 172.21.15.5:0/2277342718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08680ce540 con 0x7f08680b3e80 2024-05-20T18:28:18.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.003+0000 7f086f5f5640 1 -- 172.21.15.5:0/2277342718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0868075b00 con 0x7f08680b3e80 2024-05-20T18:28:18.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.003+0000 7f0859ffb640 1 -- 172.21.15.5:0/2277342718 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f085c00bbe0 con 0x7f08680b3e80 2024-05-20T18:28:18.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.004+0000 7f0859ffb640 1 --2- 172.21.15.5:0/2277342718 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f084c06be60 0x7f084c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:18.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.004+0000 7f0859ffb640 1 -- 172.21.15.5:0/2277342718 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f085c08b050 con 0x7f08680b3e80 2024-05-20T18:28:18.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.004+0000 7f086d391640 1 --2- 172.21.15.5:0/2277342718 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f084c06be60 0x7f084c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:18.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.005+0000 7f086d391640 1 --2- 172.21.15.5:0/2277342718 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f084c06be60 0x7f084c06e370 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0864034000 tx=0x7f086402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:18.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.009+0000 7f0859ffb640 1 -- 172.21.15.5:0/2277342718 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f085c059b30 con 0x7f08680b3e80 2024-05-20T18:28:18.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.261+0000 7f086f5f5640 1 -- 172.21.15.5:0/2277342718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f08680b3620 con 0x7f08680b3e80 2024-05-20T18:28:18.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.261+0000 7f0859ffb640 1 -- 172.21.15.5:0/2277342718 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7f085c059430 con 0x7f08680b3e80 2024-05-20T18:28:18.261 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:18.261 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:28:18.261 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_dump": [ 2024-05-20T18:28:18.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:18.261 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.a", 2024-05-20T18:28:18.261 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtm9lSeCBAA/PpiEqojk6ueJo60tU4/iA==", 2024-05-20T18:28:18.261 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:18.262 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:28:18.262 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:28:18.262 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:28:18.262 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:28:18.262 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:18.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:18.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:18.262 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.b", 2024-05-20T18:28:18.262 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmdov0FRAACkWAR/0zaCDjgnkHnoHwgw==", 2024-05-20T18:28:18.262 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:18.262 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:28:18.262 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:28:18.263 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:28:18.263 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:28:18.263 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:18.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:18.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:18.263 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.c", 2024-05-20T18:28:18.263 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmpEBlEBAAdgxDXj9x38VSOizUgdFpYA==", 2024-05-20T18:28:18.263 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:18.263 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:28:18.263 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:28:18.263 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:28:18.263 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:28:18.263 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:18.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:18.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:18.264 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.d", 2024-05-20T18:28:18.264 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtm0Gy9HRAAAkPqcBLwxTwuWmCi7aad1A==", 2024-05-20T18:28:18.264 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:18.264 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:28:18.264 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:28:18.264 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:28:18.264 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:28:18.264 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:18.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:18.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:18.264 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.0", 2024-05-20T18:28:18.264 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDHlUtmWFGzBRAA81qN37dlHf6QDZ1FgwbJ9w==", 2024-05-20T18:28:18.264 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:18.265 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:28:18.265 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:28:18.265 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:28:18.265 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:18.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:18.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:18.265 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.1", 2024-05-20T18:28:18.265 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDKlUtmBMKsBBAAD4tTTvetME6oxwugTubSKQ==", 2024-05-20T18:28:18.265 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:18.265 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:28:18.265 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:28:18.265 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:28:18.265 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:18.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:18.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:18.266 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.2", 2024-05-20T18:28:18.266 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDNlUtm4Rm/AxAAIGWTUxb+LII09Rd+h5cJ8Q==", 2024-05-20T18:28:18.266 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:18.266 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:28:18.266 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:28:18.266 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:28:18.266 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:18.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:18.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:18.266 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.3", 2024-05-20T18:28:18.266 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDQlUtmWVxZAxAAB/w6k4WJ58ulEr9zatCRfw==", 2024-05-20T18:28:18.267 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:18.267 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:28:18.267 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:28:18.267 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:28:18.267 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:18.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:18.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:18.267 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.4", 2024-05-20T18:28:18.267 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDWlUtmIyokCRAAolOiYVCYaNTQZb1PjtR6fg==", 2024-05-20T18:28:18.267 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:18.267 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:28:18.267 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:28:18.267 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:28:18.267 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:18.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:18.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:18.268 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.5", 2024-05-20T18:28:18.268 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZlUtmSf6mCBAAw73nwq/zzt9kIXZymZh93A==", 2024-05-20T18:28:18.268 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:18.268 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:28:18.268 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:28:18.268 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:28:18.268 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:18.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:18.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:18.268 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.6", 2024-05-20T18:28:18.268 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDclUtmwwhGCBAA94OBEtj9h7cmCAddYMonaA==", 2024-05-20T18:28:18.268 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:18.269 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:28:18.269 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:28:18.269 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:28:18.269 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:18.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:18.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:18.269 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.7", 2024-05-20T18:28:18.269 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDflUtm9wefBxAAvCZTBzRu05koFhc+pH5I8w==", 2024-05-20T18:28:18.269 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:18.269 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:28:18.269 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:28:18.269 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:28:18.269 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:18.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:18.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:18.270 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.0", 2024-05-20T18:28:18.270 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmevaaIhAATIwyXzr0vGT38ZmcMqR2gA==", 2024-05-20T18:28:18.270 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:18.270 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:28:18.270 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-20T18:28:18.270 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-20T18:28:18.270 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-20T18:28:18.270 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:18.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:18.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:18.270 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.1", 2024-05-20T18:28:18.271 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmK7NfKhAADaXl0uVP1Q1gUOlPsstnzg==", 2024-05-20T18:28:18.271 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:18.271 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:28:18.271 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-20T18:28:18.271 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-20T18:28:18.271 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-20T18:28:18.271 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:18.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:18.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:18.271 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.2", 2024-05-20T18:28:18.271 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmMJF1JhAAeM+HseqvgvKgG8EnIsZuhg==", 2024-05-20T18:28:18.271 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:18.271 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:28:18.271 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-20T18:28:18.272 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-20T18:28:18.272 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-20T18:28:18.272 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:18.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:18.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:18.272 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.3", 2024-05-20T18:28:18.272 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmodA7LhAAUI2v0npB6HHs2n1g3KdTXA==", 2024-05-20T18:28:18.272 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:18.272 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:28:18.272 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-20T18:28:18.272 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-20T18:28:18.272 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-20T18:28:18.272 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:18.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:18.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:18.273 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.admin", 2024-05-20T18:28:18.273 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDElUtmZacGJRAAoyr9XNPBr9r0DSU53qCHSQ==", 2024-05-20T18:28:18.273 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:18.273 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-20T18:28:18.273 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:28:18.273 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:28:18.273 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:28:18.273 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:18.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:18.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:18.273 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.bootstrap-mds", 2024-05-20T18:28:18.273 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDqlUtmVw39DRAAF4d+ux+qw5tdscny5PgF+g==", 2024-05-20T18:28:18.274 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:18.274 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile bootstrap-mds" 2024-05-20T18:28:18.274 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:18.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:18.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:18.274 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.bootstrap-mgr", 2024-05-20T18:28:18.274 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDqlUtmIxj9DRAATG0SH/IpuCoKrKmRIzFn2g==", 2024-05-20T18:28:18.274 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:18.274 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile bootstrap-mgr" 2024-05-20T18:28:18.274 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:18.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:18.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:18.274 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.bootstrap-osd", 2024-05-20T18:28:18.274 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDqlUtmwSb9DRAAOlRoY8A1rRsCWOqO/bs48g==", 2024-05-20T18:28:18.275 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:18.275 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile bootstrap-osd" 2024-05-20T18:28:18.275 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:18.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:18.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:18.275 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.bootstrap-rbd", 2024-05-20T18:28:18.275 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDqlUtmUS/9DRAA1bQ2xZSU3iLhNlWRGrGZSA==", 2024-05-20T18:28:18.275 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:18.275 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile bootstrap-rbd" 2024-05-20T18:28:18.275 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:18.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:18.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:18.275 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-05-20T18:28:18.276 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDqlUtmXDf9DRAAgKX5XeMbOiq78z2YTffayw==", 2024-05-20T18:28:18.276 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:18.276 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-05-20T18:28:18.276 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:18.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:18.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:18.276 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.bootstrap-rgw", 2024-05-20T18:28:18.276 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDqlUtmMz/9DRAAxRGqgLUV7vapYqPMljAQJA==", 2024-05-20T18:28:18.276 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:18.276 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile bootstrap-rgw" 2024-05-20T18:28:18.276 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:18.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:18.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:18.276 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.x", 2024-05-20T18:28:18.277 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmPgq0BBAAAljNeQDsSau0wH2Lh2KdBw==", 2024-05-20T18:28:18.277 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:18.277 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-20T18:28:18.277 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-05-20T18:28:18.277 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:28:18.277 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:18.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:18.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:18.277 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.y", 2024-05-20T18:28:18.277 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDElUtm6Xv3OhAA0doOWTC73jy5rOYm7Vf68Q==", 2024-05-20T18:28:18.277 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:18.277 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-20T18:28:18.277 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-05-20T18:28:18.277 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:28:18.278 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:18.278 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:18.278 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-20T18:28:18.278 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:28:18.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.263+0000 7f086f5f5640 1 -- 172.21.15.5:0/2277342718 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f084c06be60 msgr2=0x7f084c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:18.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.263+0000 7f086f5f5640 1 --2- 172.21.15.5:0/2277342718 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f084c06be60 0x7f084c06e370 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0864034000 tx=0x7f086402e000 comp rx=0 tx=0).stop 2024-05-20T18:28:18.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.263+0000 7f086f5f5640 1 -- 172.21.15.5:0/2277342718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08680b3e80 msgr2=0x7f08680b6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:18.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.263+0000 7f086f5f5640 1 --2- 172.21.15.5:0/2277342718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08680b3e80 0x7f08680b6920 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f085c00d650 tx=0x7f085c00db10 comp rx=0 tx=0).stop 2024-05-20T18:28:18.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.263+0000 7f086f5f5640 1 -- 172.21.15.5:0/2277342718 shutdown_connections 2024-05-20T18:28:18.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.263+0000 7f086f5f5640 1 --2- 172.21.15.5:0/2277342718 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f084c06be60 0x7f084c06e370 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:18.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.263+0000 7f086f5f5640 1 --2- 172.21.15.5:0/2277342718 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0868162820 0x7f08680b6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:18.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.263+0000 7f086f5f5640 1 --2- 172.21.15.5:0/2277342718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08680b3e80 0x7f08680b6920 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:18.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.263+0000 7f086f5f5640 1 --2- 172.21.15.5:0/2277342718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f086806d910 0x7f08680b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:18.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.263+0000 7f086f5f5640 1 -- 172.21.15.5:0/2277342718 >> 172.21.15.5:0/2277342718 conn(0x7f08680d4bd0 msgr2=0x7f08680b9640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:18.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.263+0000 7f086f5f5640 1 -- 172.21.15.5:0/2277342718 shutdown_connections 2024-05-20T18:28:18.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.263+0000 7f086f5f5640 1 -- 172.21.15.5:0/2277342718 wait complete. 2024-05-20T18:28:18.280 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-05-20T18:28:18.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.396+0000 7f0dead79640 1 Processor -- start 2024-05-20T18:28:18.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.396+0000 7f0dead79640 1 -- start start 2024-05-20T18:28:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.398+0000 7f0dead79640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0de406d3d0 0x7f0de406d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.398+0000 7f0dead79640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0de40b8190 0x7f0de40b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.398+0000 7f0dead79640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0de406dd20 con 0x7f0de4162820 2024-05-20T18:28:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.398+0000 7f0dead79640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f0de40d0630 con 0x7f0de406d3d0 2024-05-20T18:28:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.398+0000 7f0dead79640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0de40d39c0 con 0x7f0de40b8190 2024-05-20T18:28:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.398+0000 7f0de3fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0de406d3d0 0x7f0de406d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.398+0000 7f0de8b15640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0de4162820 0x7f0de40b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45758/0 (socket says 172.21.15.5:45758) 2024-05-20T18:28:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.398+0000 7f0de8b15640 1 -- 172.21.15.5:0/2870301540 learned_addr learned my addr 172.21.15.5:0/2870301540 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.398+0000 7f0de9316640 1 --2- 172.21.15.5:0/2870301540 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0de40b8190 0x7f0de40b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.398+0000 7f0de3fff640 1 -- 172.21.15.5:0/2870301540 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0de40b8190 msgr2=0x7f0de40b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.398+0000 7f0de3fff640 1 --2- 172.21.15.5:0/2870301540 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0de40b8190 0x7f0de40b85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:18.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.398+0000 7f0de3fff640 1 -- 172.21.15.5:0/2870301540 >> v1:172.21.15.5:6789/0 conn(0x7f0de4162820 legacy=0x7f0de40b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:18.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.398+0000 7f0de3fff640 1 -- 172.21.15.5:0/2870301540 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de40d3ca0 con 0x7f0de406d3d0 2024-05-20T18:28:18.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.398+0000 7f0de37fe640 1 -- 172.21.15.5:0/2870301540 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2581070438 0 0) 0x7f0dd4001110 con 0x7f0de4162820 2024-05-20T18:28:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.399+0000 7f0de3fff640 1 --2- 172.21.15.5:0/2870301540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de406d3d0 0x7f0de406d7e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0dd80097a0 tx=0x7f0dd802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5caf7318929e75e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.399+0000 7f0de37fe640 1 -- 172.21.15.5:0/2870301540 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8037070 con 0x7f0de406d3d0 2024-05-20T18:28:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.399+0000 7f0de37fe640 1 -- 172.21.15.5:0/2870301540 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dd8032440 con 0x7f0de406d3d0 2024-05-20T18:28:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.399+0000 7f0de37fe640 1 -- 172.21.15.5:0/2870301540 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8032810 con 0x7f0de406d3d0 2024-05-20T18:28:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.400+0000 7f0dead79640 1 -- 172.21.15.5:0/2870301540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de406d3d0 msgr2=0x7f0de406d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.400+0000 7f0dead79640 1 --2- 172.21.15.5:0/2870301540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de406d3d0 0x7f0de406d7e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0dd80097a0 tx=0x7f0dd802a180 comp rx=0 tx=0).stop 2024-05-20T18:28:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.400+0000 7f0dead79640 1 -- 172.21.15.5:0/2870301540 shutdown_connections 2024-05-20T18:28:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.400+0000 7f0dead79640 1 --2- 172.21.15.5:0/2870301540 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0de40b8190 0x7f0de40b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.400+0000 7f0dead79640 1 --2- 172.21.15.5:0/2870301540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de406d3d0 0x7f0de406d7e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.400+0000 7f0dead79640 1 -- 172.21.15.5:0/2870301540 >> 172.21.15.5:0/2870301540 conn(0x7f0de40d4bd0 msgr2=0x7f0de40d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.400+0000 7f0dead79640 1 -- 172.21.15.5:0/2870301540 shutdown_connections 2024-05-20T18:28:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.400+0000 7f0dead79640 1 -- 172.21.15.5:0/2870301540 wait complete. 2024-05-20T18:28:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.402+0000 7f0dead79640 1 Processor -- start 2024-05-20T18:28:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.402+0000 7f0dead79640 1 -- start start 2024-05-20T18:28:18.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.403+0000 7f0dead79640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0de406d3d0 0x7f0de40ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:18.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.403+0000 7f0dead79640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de40b8190 0x7f0de41f5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:18.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.403+0000 7f0dead79640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0de4162820 0x7f0de415fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:18.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.403+0000 7f0dead79640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0de40d0300 con 0x7f0de40b8190 2024-05-20T18:28:18.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.403+0000 7f0dead79640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0de40d4260 con 0x7f0de4162820 2024-05-20T18:28:18.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.403+0000 7f0dead79640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0de40b4050 con 0x7f0de406d3d0 2024-05-20T18:28:18.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.403+0000 7f0de3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de40b8190 0x7f0de41f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:18.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.403+0000 7f0de3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de40b8190 0x7f0de41f5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37112/0 (socket says 172.21.15.5:37112) 2024-05-20T18:28:18.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.403+0000 7f0de3fff640 1 -- 172.21.15.5:0/1737510781 learned_addr learned my addr 172.21.15.5:0/1737510781 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:18.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.403+0000 7f0de9316640 1 --2- 172.21.15.5:0/1737510781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0de4162820 0x7f0de415fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:18.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.404+0000 7f0de8b15640 1 --2- 172.21.15.5:0/1737510781 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0de406d3d0 0x7f0de40ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:18.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.404+0000 7f0de3fff640 1 -- 172.21.15.5:0/1737510781 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0de406d3d0 msgr2=0x7f0de40ca9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:18.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.404+0000 7f0de3fff640 1 --2- 172.21.15.5:0/1737510781 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0de406d3d0 0x7f0de40ca9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.404+0000 7f0de3fff640 1 -- 172.21.15.5:0/1737510781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0de4162820 msgr2=0x7f0de415fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.404+0000 7f0de3fff640 1 --2- 172.21.15.5:0/1737510781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0de4162820 0x7f0de415fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.404+0000 7f0de3fff640 1 -- 172.21.15.5:0/1737510781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de40cbe20 con 0x7f0de40b8190 2024-05-20T18:28:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.404+0000 7f0de3fff640 1 --2- 172.21.15.5:0/1737510781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de40b8190 0x7f0de41f5170 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8000c00 tx=0x7f0dd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.404+0000 7f0de17fa640 1 -- 172.21.15.5:0/1737510781 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8037040 con 0x7f0de40b8190 2024-05-20T18:28:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.404+0000 7f0de17fa640 1 -- 172.21.15.5:0/1737510781 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dd8042070 con 0x7f0de40b8190 2024-05-20T18:28:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.404+0000 7f0dead79640 1 -- 172.21.15.5:0/1737510781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0de40cb1f0 con 0x7f0de40b8190 2024-05-20T18:28:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.404+0000 7f0de17fa640 1 -- 172.21.15.5:0/1737510781 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8030bb0 con 0x7f0de40b8190 2024-05-20T18:28:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.404+0000 7f0dead79640 1 -- 172.21.15.5:0/1737510781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0de40cb7b0 con 0x7f0de40b8190 2024-05-20T18:28:18.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.405+0000 7f0dead79640 1 -- 172.21.15.5:0/1737510781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0de4075b00 con 0x7f0de40b8190 2024-05-20T18:28:18.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.406+0000 7f0de17fa640 1 -- 172.21.15.5:0/1737510781 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0dd8047460 con 0x7f0de40b8190 2024-05-20T18:28:18.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.406+0000 7f0de17fa640 1 --2- 172.21.15.5:0/1737510781 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0dd006be30 0x7f0dd006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:18.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.406+0000 7f0de17fa640 1 -- 172.21.15.5:0/1737510781 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f0dd80ab870 con 0x7f0de40b8190 2024-05-20T18:28:18.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.406+0000 7f0de8b15640 1 --2- 172.21.15.5:0/1737510781 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0dd006be30 0x7f0dd006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:18.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.407+0000 7f0de8b15640 1 --2- 172.21.15.5:0/1737510781 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0dd006be30 0x7f0dd006e340 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4000f60 tx=0x7f0dd4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:18.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.411+0000 7f0de17fa640 1 -- 172.21.15.5:0/1737510781 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0dd8032e20 con 0x7f0de40b8190 2024-05-20T18:28:18.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.664+0000 7f0dead79640 1 -- 172.21.15.5:0/1737510781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f0de40caf80 con 0x7f0de40b8190 2024-05-20T18:28:18.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.667+0000 7f0de17fa640 1 -- 172.21.15.5:0/1737510781 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f0dd807a3e0 con 0x7f0de40b8190 2024-05-20T18:28:18.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.669+0000 7f0dead79640 1 -- 172.21.15.5:0/1737510781 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0dd006be30 msgr2=0x7f0dd006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:18.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.669+0000 7f0dead79640 1 --2- 172.21.15.5:0/1737510781 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0dd006be30 0x7f0dd006e340 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4000f60 tx=0x7f0dd4009040 comp rx=0 tx=0).stop 2024-05-20T18:28:18.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.669+0000 7f0dead79640 1 -- 172.21.15.5:0/1737510781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de40b8190 msgr2=0x7f0de41f5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:18.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.669+0000 7f0dead79640 1 --2- 172.21.15.5:0/1737510781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de40b8190 0x7f0de41f5170 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8000c00 tx=0x7f0dd80046c0 comp rx=0 tx=0).stop 2024-05-20T18:28:18.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.669+0000 7f0dead79640 1 -- 172.21.15.5:0/1737510781 shutdown_connections 2024-05-20T18:28:18.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.669+0000 7f0dead79640 1 --2- 172.21.15.5:0/1737510781 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0dd006be30 0x7f0dd006e340 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:18.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.669+0000 7f0dead79640 1 --2- 172.21.15.5:0/1737510781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0de4162820 0x7f0de415fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:18.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.669+0000 7f0dead79640 1 --2- 172.21.15.5:0/1737510781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0de40b8190 0x7f0de41f5170 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:18.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.669+0000 7f0dead79640 1 --2- 172.21.15.5:0/1737510781 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0de406d3d0 0x7f0de40ca9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:18.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.669+0000 7f0dead79640 1 -- 172.21.15.5:0/1737510781 >> 172.21.15.5:0/1737510781 conn(0x7f0de40d4bd0 msgr2=0x7f0de415f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:18.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.669+0000 7f0dead79640 1 -- 172.21.15.5:0/1737510781 shutdown_connections 2024-05-20T18:28:18.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.669+0000 7f0dead79640 1 -- 172.21.15.5:0/1737510781 wait complete. 2024-05-20T18:28:18.683 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-05-20T18:28:18.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.806+0000 7fb06e698640 1 Processor -- start 2024-05-20T18:28:18.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.806+0000 7fb06e698640 1 -- start start 2024-05-20T18:28:18.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.807+0000 7fb06e698640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb0680d1780 0x7fb0680d1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:18.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.807+0000 7fb06e698640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb06815d870 con 0x7fb0680b17f0 2024-05-20T18:28:18.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.807+0000 7fb06e698640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0680d20d0 con 0x7fb06815d460 2024-05-20T18:28:18.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.807+0000 7fb06e698640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fb06815dae0 con 0x7fb0680d1780 2024-05-20T18:28:18.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.807+0000 7fb05ffff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb0680d1780 0x7fb0680d1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:18.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.807+0000 7fb06cc35640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb0680b17f0 0x7fb0680b1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45762/0 (socket says 172.21.15.5:45762) 2024-05-20T18:28:18.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.807+0000 7fb05ffff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb0680d1780 0x7fb0680d1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37114/0 (socket says 172.21.15.5:37114) 2024-05-20T18:28:18.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.807+0000 7fb06cc35640 1 -- 172.21.15.5:0/3411557985 learned_addr learned my addr 172.21.15.5:0/3411557985 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:18.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.808+0000 7fb05ffff640 1 -- 172.21.15.5:0/3411557985 >> v1:172.21.15.63:6789/0 conn(0x7fb06815d460 legacy=0x7fb06806b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:18.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.808+0000 7fb05ffff640 1 -- 172.21.15.5:0/3411557985 >> v1:172.21.15.5:6789/0 conn(0x7fb0680b17f0 legacy=0x7fb0680b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:18.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.808+0000 7fb05ffff640 1 -- 172.21.15.5:0/3411557985 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06815ddc0 con 0x7fb0680d1780 2024-05-20T18:28:18.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.808+0000 7fb05ffff640 1 --2- 172.21.15.5:0/3411557985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0680d1780 0x7fb0680d1b90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb0500097a0 tx=0x7fb05002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=924a7b13e6853162 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:18.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.809+0000 7fb0677fe640 1 -- 172.21.15.5:0/3411557985 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb050037070 con 0x7fb0680d1780 2024-05-20T18:28:18.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.809+0000 7fb0677fe640 1 -- 172.21.15.5:0/3411557985 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb050032440 con 0x7fb0680d1780 2024-05-20T18:28:18.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.809+0000 7fb0677fe640 1 -- 172.21.15.5:0/3411557985 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb050032810 con 0x7fb0680d1780 2024-05-20T18:28:18.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.809+0000 7fb06e698640 1 -- 172.21.15.5:0/3411557985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0680d1780 msgr2=0x7fb0680d1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:18.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.809+0000 7fb06e698640 1 --2- 172.21.15.5:0/3411557985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0680d1780 0x7fb0680d1b90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb0500097a0 tx=0x7fb05002a1a0 comp rx=0 tx=0).stop 2024-05-20T18:28:18.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.809+0000 7fb06e698640 1 -- 172.21.15.5:0/3411557985 shutdown_connections 2024-05-20T18:28:18.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.809+0000 7fb06e698640 1 --2- 172.21.15.5:0/3411557985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0680d1780 0x7fb0680d1b90 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:18.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.809+0000 7fb06e698640 1 -- 172.21.15.5:0/3411557985 >> 172.21.15.5:0/3411557985 conn(0x7fb0680b6620 msgr2=0x7fb0680b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:18.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.809+0000 7fb06e698640 1 -- 172.21.15.5:0/3411557985 shutdown_connections 2024-05-20T18:28:18.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.809+0000 7fb06e698640 1 -- 172.21.15.5:0/3411557985 wait complete. 2024-05-20T18:28:18.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.811+0000 7fb06e698640 1 Processor -- start 2024-05-20T18:28:18.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.811+0000 7fb06e698640 1 -- start start 2024-05-20T18:28:18.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.812+0000 7fb06e698640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0680b17f0 0x7fb06809f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:18.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.812+0000 7fb06e698640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0680d1780 0x7fb06809b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:18.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.813+0000 7fb06e698640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb06815d460 0x7fb0680cfd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:18.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.813+0000 7fb06e698640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb06815e380 con 0x7fb06815d460 2024-05-20T18:28:18.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.813+0000 7fb06e698640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb0680b12d0 con 0x7fb0680b17f0 2024-05-20T18:28:18.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.813+0000 7fb06e698640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb0680b2310 con 0x7fb0680d1780 2024-05-20T18:28:18.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.813+0000 7fb06cc35640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb06815d460 0x7fb0680cfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:18.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.813+0000 7fb06cc35640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb06815d460 0x7fb0680cfd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37122/0 (socket says 172.21.15.5:37122) 2024-05-20T18:28:18.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.813+0000 7fb06cc35640 1 -- 172.21.15.5:0/330124244 learned_addr learned my addr 172.21.15.5:0/330124244 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:18.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.813+0000 7fb05ffff640 1 --2- 172.21.15.5:0/330124244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0680d1780 0x7fb06809b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:18.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.813+0000 7fb067fff640 1 --2- 172.21.15.5:0/330124244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0680b17f0 0x7fb06809f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:18.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.813+0000 7fb06cc35640 1 -- 172.21.15.5:0/330124244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0680d1780 msgr2=0x7fb06809b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:18.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.813+0000 7fb06cc35640 1 --2- 172.21.15.5:0/330124244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0680d1780 0x7fb06809b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:18.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.813+0000 7fb06cc35640 1 -- 172.21.15.5:0/330124244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0680b17f0 msgr2=0x7fb06809f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:18.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.813+0000 7fb06cc35640 1 --2- 172.21.15.5:0/330124244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0680b17f0 0x7fb06809f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:18.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.813+0000 7fb06cc35640 1 -- 172.21.15.5:0/330124244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06809c940 con 0x7fb06815d460 2024-05-20T18:28:18.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.813+0000 7fb06cc35640 1 --2- 172.21.15.5:0/330124244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb06815d460 0x7fb0680cfd00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb06000cad0 tx=0x7fb06000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:18.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.813+0000 7fb0657fa640 1 -- 172.21.15.5:0/330124244 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060017070 con 0x7fb06815d460 2024-05-20T18:28:18.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.813+0000 7fb0657fa640 1 -- 172.21.15.5:0/330124244 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb06000ac30 con 0x7fb06815d460 2024-05-20T18:28:18.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.813+0000 7fb06e698640 1 -- 172.21.15.5:0/330124244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb06809c380 con 0x7fb06815d460 2024-05-20T18:28:18.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.813+0000 7fb0657fa640 1 -- 172.21.15.5:0/330124244 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060012400 con 0x7fb06815d460 2024-05-20T18:28:18.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.813+0000 7fb06e698640 1 -- 172.21.15.5:0/330124244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0681f4890 con 0x7fb06815d460 2024-05-20T18:28:18.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.815+0000 7fb06e698640 1 -- 172.21.15.5:0/330124244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb06809c0f0 con 0x7fb06815d460 2024-05-20T18:28:18.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.815+0000 7fb0657fa640 1 -- 172.21.15.5:0/330124244 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb06001a070 con 0x7fb06815d460 2024-05-20T18:28:18.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.816+0000 7fb0657fa640 1 --2- 172.21.15.5:0/330124244 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb04806be60 0x7fb04806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:18.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.816+0000 7fb0657fa640 1 -- 172.21.15.5:0/330124244 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7fb06008a840 con 0x7fb06815d460 2024-05-20T18:28:18.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.816+0000 7fb067fff640 1 --2- 172.21.15.5:0/330124244 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb04806be60 0x7fb04806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:18.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.816+0000 7fb067fff640 1 --2- 172.21.15.5:0/330124244 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb04806be60 0x7fb04806e370 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb06806c760 tx=0x7fb058009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:18.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:18.821+0000 7fb0657fa640 1 -- 172.21.15.5:0/330124244 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb060058cd0 con 0x7fb06815d460 2024-05-20T18:28:19.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.078+0000 7fb06e698640 1 -- 172.21.15.5:0/330124244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fb06809f960 con 0x7fb06815d460 2024-05-20T18:28:19.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.081+0000 7fb0657fa640 1 -- 172.21.15.5:0/330124244 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7fb06005cfd0 con 0x7fb06815d460 2024-05-20T18:28:19.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.083+0000 7fb06e698640 1 -- 172.21.15.5:0/330124244 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb04806be60 msgr2=0x7fb04806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:19.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.083+0000 7fb06e698640 1 --2- 172.21.15.5:0/330124244 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb04806be60 0x7fb04806e370 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb06806c760 tx=0x7fb058009040 comp rx=0 tx=0).stop 2024-05-20T18:28:19.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.083+0000 7fb06e698640 1 -- 172.21.15.5:0/330124244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb06815d460 msgr2=0x7fb0680cfd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:19.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.083+0000 7fb06e698640 1 --2- 172.21.15.5:0/330124244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb06815d460 0x7fb0680cfd00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb06000cad0 tx=0x7fb06000cf90 comp rx=0 tx=0).stop 2024-05-20T18:28:19.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.083+0000 7fb06e698640 1 -- 172.21.15.5:0/330124244 shutdown_connections 2024-05-20T18:28:19.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.083+0000 7fb06e698640 1 --2- 172.21.15.5:0/330124244 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb04806be60 0x7fb04806e370 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:19.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.083+0000 7fb06e698640 1 --2- 172.21.15.5:0/330124244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb06815d460 0x7fb0680cfd00 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:19.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.083+0000 7fb06e698640 1 --2- 172.21.15.5:0/330124244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0680d1780 0x7fb06809b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:19.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.083+0000 7fb06e698640 1 --2- 172.21.15.5:0/330124244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0680b17f0 0x7fb06809f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:19.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.083+0000 7fb06e698640 1 -- 172.21.15.5:0/330124244 >> 172.21.15.5:0/330124244 conn(0x7fb0680b6620 msgr2=0x7fb0680b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:19.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.083+0000 7fb06e698640 1 -- 172.21.15.5:0/330124244 shutdown_connections 2024-05-20T18:28:19.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.083+0000 7fb06e698640 1 -- 172.21.15.5:0/330124244 wait complete. 2024-05-20T18:28:19.094 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-05-20T18:28:19.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.210+0000 7f90dbd96640 1 Processor -- start 2024-05-20T18:28:19.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.210+0000 7f90dbd96640 1 -- start start 2024-05-20T18:28:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.211+0000 7f90dbd96640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90d40cd710 0x7f90d4162e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.211+0000 7f90dbd96640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f90d406d6b0 0x7f90d415f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.212+0000 7f90dbd96640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90d406c8c0 con 0x7f90d40cf3d0 2024-05-20T18:28:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.212+0000 7f90dbd96640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f90d40d0ba0 con 0x7f90d406d6b0 2024-05-20T18:28:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.212+0000 7f90dbd96640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f90d406dac0 con 0x7f90d40cd710 2024-05-20T18:28:19.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.212+0000 7f90d9b32640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90d40cd710 0x7f90d4162e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:19.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.212+0000 7f90d9331640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f90d406d6b0 0x7f90d415f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:19.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.212+0000 7f90d9b32640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90d40cd710 0x7f90d4162e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:56028/0 (socket says 172.21.15.5:56028) 2024-05-20T18:28:19.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.212+0000 7f90d9b32640 1 -- 172.21.15.5:0/3006766600 learned_addr learned my addr 172.21.15.5:0/3006766600 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:19.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.212+0000 7f90d9331640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f90d406d6b0 0x7f90d415f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:45852/0 (socket says 172.21.15.5:45852) 2024-05-20T18:28:19.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.212+0000 7f90d9b32640 1 -- 172.21.15.5:0/3006766600 >> v2:172.21.15.63:3300/0 conn(0x7f90d406d6b0 msgr2=0x7f90d415f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:19.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.212+0000 7f90d9b32640 1 --2- 172.21.15.5:0/3006766600 >> v2:172.21.15.63:3300/0 conn(0x7f90d406d6b0 0x7f90d415f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:19.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.212+0000 7f90d9b32640 1 -- 172.21.15.5:0/3006766600 >> v1:172.21.15.63:6789/0 conn(0x7f90d40cf3d0 legacy=0x7f90d40cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:19.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.212+0000 7f90d9b32640 1 -- 172.21.15.5:0/3006766600 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d41633e0 con 0x7f90d40cd710 2024-05-20T18:28:19.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.212+0000 7f90d9331640 1 --2- 172.21.15.5:0/3006766600 >> v2:172.21.15.63:3300/0 conn(0x7f90d406d6b0 0x7f90d415f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:28:19.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.213+0000 7f90d9b32640 1 --2- 172.21.15.5:0/3006766600 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90d40cd710 0x7f90d4162e00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f90d40d0d20 tx=0x7f90d002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85569546d96c8e31 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:19.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.213+0000 7f90d8b30640 1 -- 172.21.15.5:0/3006766600 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0037070 con 0x7f90d40cd710 2024-05-20T18:28:19.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.213+0000 7f90d8b30640 1 -- 172.21.15.5:0/3006766600 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90d0032440 con 0x7f90d40cd710 2024-05-20T18:28:19.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.213+0000 7f90d8b30640 1 -- 172.21.15.5:0/3006766600 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0032810 con 0x7f90d40cd710 2024-05-20T18:28:19.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.214+0000 7f90dbd96640 1 -- 172.21.15.5:0/3006766600 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90d40cd710 msgr2=0x7f90d4162e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:19.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.214+0000 7f90dbd96640 1 --2- 172.21.15.5:0/3006766600 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90d40cd710 0x7f90d4162e00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f90d40d0d20 tx=0x7f90d002a250 comp rx=0 tx=0).stop 2024-05-20T18:28:19.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.214+0000 7f90dbd96640 1 -- 172.21.15.5:0/3006766600 shutdown_connections 2024-05-20T18:28:19.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.214+0000 7f90dbd96640 1 --2- 172.21.15.5:0/3006766600 >> v2:172.21.15.63:3300/0 conn(0x7f90d406d6b0 0x7f90d415f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:19.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.214+0000 7f90dbd96640 1 --2- 172.21.15.5:0/3006766600 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90d40cd710 0x7f90d4162e00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:19.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.214+0000 7f90dbd96640 1 -- 172.21.15.5:0/3006766600 >> 172.21.15.5:0/3006766600 conn(0x7f90d40b44d0 msgr2=0x7f90d40b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:19.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.214+0000 7f90dbd96640 1 -- 172.21.15.5:0/3006766600 shutdown_connections 2024-05-20T18:28:19.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.214+0000 7f90dbd96640 1 -- 172.21.15.5:0/3006766600 wait complete. 2024-05-20T18:28:19.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.216+0000 7f90dbd96640 1 Processor -- start 2024-05-20T18:28:19.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.216+0000 7f90dbd96640 1 -- start start 2024-05-20T18:28:19.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.217+0000 7f90dbd96640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90d406d6b0 0x7f90d41569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:19.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.217+0000 7f90dbd96640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90d40cd710 0x7f90d40cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:19.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.217+0000 7f90dbd96640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90d40cf3d0 0x7f90d4158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:19.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.217+0000 7f90dbd96640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90d4163c80 con 0x7f90d40cd710 2024-05-20T18:28:19.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.217+0000 7f90dbd96640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f90d41639a0 con 0x7f90d406d6b0 2024-05-20T18:28:19.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.217+0000 7f90dbd96640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f90d40d1030 con 0x7f90d40cf3d0 2024-05-20T18:28:19.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.217+0000 7f90d9331640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90d40cd710 0x7f90d40cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.217+0000 7f90d9331640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90d40cd710 0x7f90d40cc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37134/0 (socket says 172.21.15.5:37134) 2024-05-20T18:28:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.217+0000 7f90d9331640 1 -- 172.21.15.5:0/3896599868 learned_addr learned my addr 172.21.15.5:0/3896599868 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.217+0000 7f90d9b32640 1 --2- 172.21.15.5:0/3896599868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90d406d6b0 0x7f90d41569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.217+0000 7f90da333640 1 --2- 172.21.15.5:0/3896599868 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90d40cf3d0 0x7f90d4158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.218+0000 7f90d9331640 1 -- 172.21.15.5:0/3896599868 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90d40cf3d0 msgr2=0x7f90d4158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.218+0000 7f90d9331640 1 --2- 172.21.15.5:0/3896599868 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90d40cf3d0 0x7f90d4158a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.218+0000 7f90d9331640 1 -- 172.21.15.5:0/3896599868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90d406d6b0 msgr2=0x7f90d41569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.218+0000 7f90d9331640 1 --2- 172.21.15.5:0/3896599868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90d406d6b0 0x7f90d41569c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.218+0000 7f90d9331640 1 -- 172.21.15.5:0/3896599868 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d41f5670 con 0x7f90d40cd710 2024-05-20T18:28:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.218+0000 7f90d9331640 1 --2- 172.21.15.5:0/3896599868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90d40cd710 0x7f90d40cc6b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f90c400e8d0 tx=0x7f90c400ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.218+0000 7f90ca7fc640 1 -- 172.21.15.5:0/3896599868 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c4017070 con 0x7f90d40cd710 2024-05-20T18:28:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.218+0000 7f90ca7fc640 1 -- 172.21.15.5:0/3896599868 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90c400c940 con 0x7f90d40cd710 2024-05-20T18:28:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.218+0000 7f90dbd96640 1 -- 172.21.15.5:0/3896599868 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d40ccbf0 con 0x7f90d40cd710 2024-05-20T18:28:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.218+0000 7f90ca7fc640 1 -- 172.21.15.5:0/3896599868 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c400cd30 con 0x7f90d40cd710 2024-05-20T18:28:19.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.218+0000 7f90dbd96640 1 -- 172.21.15.5:0/3896599868 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d40c9e10 con 0x7f90d40cd710 2024-05-20T18:28:19.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.219+0000 7f90dbd96640 1 -- 172.21.15.5:0/3896599868 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90d40cce70 con 0x7f90d40cd710 2024-05-20T18:28:19.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.220+0000 7f90ca7fc640 1 -- 172.21.15.5:0/3896599868 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f90c401a070 con 0x7f90d40cd710 2024-05-20T18:28:19.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.220+0000 7f90ca7fc640 1 --2- 172.21.15.5:0/3896599868 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f90b806be60 0x7f90b806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:19.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.221+0000 7f90ca7fc640 1 -- 172.21.15.5:0/3896599868 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f90c408a6f0 con 0x7f90d40cd710 2024-05-20T18:28:19.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.221+0000 7f90d9b32640 1 --2- 172.21.15.5:0/3896599868 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f90b806be60 0x7f90b806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:19.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.221+0000 7f90d9b32640 1 --2- 172.21.15.5:0/3896599868 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f90b806be60 0x7f90b806e370 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f90d0034000 tx=0x7f90d002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:19.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.225+0000 7f90ca7fc640 1 -- 172.21.15.5:0/3896599868 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f90c4058b80 con 0x7f90d40cd710 2024-05-20T18:28:19.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.481+0000 7f90dbd96640 1 -- 172.21.15.5:0/3896599868 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f90d4075b00 con 0x7f90d40cd710 2024-05-20T18:28:19.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.484+0000 7f90ca7fc640 1 -- 172.21.15.5:0/3896599868 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f90c405ce80 con 0x7f90d40cd710 2024-05-20T18:28:19.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.486+0000 7f90dbd96640 1 -- 172.21.15.5:0/3896599868 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f90b806be60 msgr2=0x7f90b806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:19.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.486+0000 7f90dbd96640 1 --2- 172.21.15.5:0/3896599868 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f90b806be60 0x7f90b806e370 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f90d0034000 tx=0x7f90d002e000 comp rx=0 tx=0).stop 2024-05-20T18:28:19.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.486+0000 7f90dbd96640 1 -- 172.21.15.5:0/3896599868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90d40cd710 msgr2=0x7f90d40cc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:19.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.486+0000 7f90dbd96640 1 --2- 172.21.15.5:0/3896599868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90d40cd710 0x7f90d40cc6b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f90c400e8d0 tx=0x7f90c400ed90 comp rx=0 tx=0).stop 2024-05-20T18:28:19.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.486+0000 7f90dbd96640 1 -- 172.21.15.5:0/3896599868 shutdown_connections 2024-05-20T18:28:19.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.486+0000 7f90dbd96640 1 --2- 172.21.15.5:0/3896599868 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f90b806be60 0x7f90b806e370 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:19.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.486+0000 7f90dbd96640 1 --2- 172.21.15.5:0/3896599868 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90d40cf3d0 0x7f90d4158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:19.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.486+0000 7f90dbd96640 1 --2- 172.21.15.5:0/3896599868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90d40cd710 0x7f90d40cc6b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:19.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.486+0000 7f90dbd96640 1 --2- 172.21.15.5:0/3896599868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90d406d6b0 0x7f90d41569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:19.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.486+0000 7f90dbd96640 1 -- 172.21.15.5:0/3896599868 >> 172.21.15.5:0/3896599868 conn(0x7f90d40b44d0 msgr2=0x7f90d40b23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:19.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.486+0000 7f90dbd96640 1 -- 172.21.15.5:0/3896599868 shutdown_connections 2024-05-20T18:28:19.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.486+0000 7f90dbd96640 1 -- 172.21.15.5:0/3896599868 wait complete. 2024-05-20T18:28:19.497 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-05-20T18:28:19.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.614+0000 7fb45f39d640 1 Processor -- start 2024-05-20T18:28:19.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.614+0000 7fb45f39d640 1 -- start start 2024-05-20T18:28:19.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.615+0000 7fb45f39d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb4580cf860 0x7fb4580b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.615+0000 7fb45f39d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb458163520 0x7fb4580d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.615+0000 7fb45f39d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb458163930 0x7fb4580d17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.615+0000 7fb45f39d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fb4581605a0 con 0x7fb458163930 2024-05-20T18:28:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.615+0000 7fb45f39d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb45815ffe0 con 0x7fb458163520 2024-05-20T18:28:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.615+0000 7fb45f39d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb4581602c0 con 0x7fb4580cf860 2024-05-20T18:28:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.616+0000 7fb45eb9c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb458163930 0x7fb4580d17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.616+0000 7fb45eb9c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb458163930 0x7fb4580d17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37150/0 (socket says 172.21.15.5:37150) 2024-05-20T18:28:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.616+0000 7fb45eb9c640 1 -- 172.21.15.5:0/2534988094 learned_addr learned my addr 172.21.15.5:0/2534988094 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.616+0000 7fb45db9a640 1 --2- 172.21.15.5:0/2534988094 >> v2:172.21.15.63:3300/0 conn(0x7fb458163520 0x7fb4580d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.616+0000 7fb45e39b640 1 --2- 172.21.15.5:0/2534988094 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb4580cf860 0x7fb4580b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.616+0000 7fb45eb9c640 1 -- 172.21.15.5:0/2534988094 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb4580cf860 msgr2=0x7fb4580b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.616+0000 7fb45eb9c640 1 --2- 172.21.15.5:0/2534988094 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb4580cf860 0x7fb4580b3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.616+0000 7fb45eb9c640 1 -- 172.21.15.5:0/2534988094 >> v2:172.21.15.63:3300/0 conn(0x7fb458163520 msgr2=0x7fb4580d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.616+0000 7fb45eb9c640 1 --2- 172.21.15.5:0/2534988094 >> v2:172.21.15.63:3300/0 conn(0x7fb458163520 0x7fb4580d12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:19.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.616+0000 7fb45eb9c640 1 -- 172.21.15.5:0/2534988094 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb458162d20 con 0x7fb458163930 2024-05-20T18:28:19.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.616+0000 7fb45db9a640 1 --2- 172.21.15.5:0/2534988094 >> v2:172.21.15.63:3300/0 conn(0x7fb458163520 0x7fb4580d12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:19.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.616+0000 7fb45e39b640 1 --2- 172.21.15.5:0/2534988094 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb4580cf860 0x7fb4580b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:19.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.617+0000 7fb45eb9c640 1 --2- 172.21.15.5:0/2534988094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb458163930 0x7fb4580d17e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb4580b75a0 tx=0x7fb45002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95a22dd93e92e06 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:19.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.617+0000 7fb45d399640 1 -- 172.21.15.5:0/2534988094 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450036070 con 0x7fb458163930 2024-05-20T18:28:19.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.617+0000 7fb45d399640 1 -- 172.21.15.5:0/2534988094 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb450031600 con 0x7fb458163930 2024-05-20T18:28:19.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.617+0000 7fb45d399640 1 -- 172.21.15.5:0/2534988094 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4500319d0 con 0x7fb458163930 2024-05-20T18:28:19.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.618+0000 7fb45f39d640 1 -- 172.21.15.5:0/2534988094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb458163930 msgr2=0x7fb4580d17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:19.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.618+0000 7fb45f39d640 1 --2- 172.21.15.5:0/2534988094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb458163930 0x7fb4580d17e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb4580b75a0 tx=0x7fb45002a610 comp rx=0 tx=0).stop 2024-05-20T18:28:19.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.618+0000 7fb45f39d640 1 -- 172.21.15.5:0/2534988094 shutdown_connections 2024-05-20T18:28:19.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.618+0000 7fb45f39d640 1 --2- 172.21.15.5:0/2534988094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb458163930 0x7fb4580d17e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:19.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.618+0000 7fb45f39d640 1 --2- 172.21.15.5:0/2534988094 >> v2:172.21.15.63:3300/0 conn(0x7fb458163520 0x7fb4580d12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:19.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.618+0000 7fb45f39d640 1 --2- 172.21.15.5:0/2534988094 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb4580cf860 0x7fb4580b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:19.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.618+0000 7fb45f39d640 1 -- 172.21.15.5:0/2534988094 >> 172.21.15.5:0/2534988094 conn(0x7fb45806dbb0 msgr2=0x7fb45806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:19.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.618+0000 7fb45f39d640 1 -- 172.21.15.5:0/2534988094 shutdown_connections 2024-05-20T18:28:19.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.618+0000 7fb45f39d640 1 -- 172.21.15.5:0/2534988094 wait complete. 2024-05-20T18:28:19.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.620+0000 7fb45f39d640 1 Processor -- start 2024-05-20T18:28:19.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.620+0000 7fb45f39d640 1 -- start start 2024-05-20T18:28:19.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.621+0000 7fb45f39d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4580cf860 0x7fb45814a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:19.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.621+0000 7fb45f39d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb458163520 0x7fb4580b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:19.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.621+0000 7fb45f39d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb458163930 0x7fb4580b6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:19.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.621+0000 7fb45f39d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb4580d32c0 con 0x7fb458163520 2024-05-20T18:28:19.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.621+0000 7fb45f39d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb4580d3140 con 0x7fb4580cf860 2024-05-20T18:28:19.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.621+0000 7fb45f39d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb4580d3440 con 0x7fb458163930 2024-05-20T18:28:19.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.621+0000 7fb45db9a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb458163520 0x7fb4580b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:19.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.621+0000 7fb45db9a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb458163520 0x7fb4580b69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37158/0 (socket says 172.21.15.5:37158) 2024-05-20T18:28:19.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.621+0000 7fb45db9a640 1 -- 172.21.15.5:0/2037894699 learned_addr learned my addr 172.21.15.5:0/2037894699 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:19.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.621+0000 7fb45e39b640 1 --2- 172.21.15.5:0/2037894699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4580cf860 0x7fb45814a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:19.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.621+0000 7fb45eb9c640 1 --2- 172.21.15.5:0/2037894699 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb458163930 0x7fb4580b6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:19.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.621+0000 7fb45db9a640 1 -- 172.21.15.5:0/2037894699 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb458163930 msgr2=0x7fb4580b6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:19.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.621+0000 7fb45db9a640 1 --2- 172.21.15.5:0/2037894699 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb458163930 0x7fb4580b6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:19.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.621+0000 7fb45db9a640 1 -- 172.21.15.5:0/2037894699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4580cf860 msgr2=0x7fb45814a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:19.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.621+0000 7fb45db9a640 1 --2- 172.21.15.5:0/2037894699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4580cf860 0x7fb45814a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:19.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.621+0000 7fb45db9a640 1 -- 172.21.15.5:0/2037894699 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4581f6000 con 0x7fb458163520 2024-05-20T18:28:19.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.621+0000 7fb45e39b640 1 --2- 172.21.15.5:0/2037894699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4580cf860 0x7fb45814a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:19.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.622+0000 7fb45db9a640 1 --2- 172.21.15.5:0/2037894699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb458163520 0x7fb4580b69d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00d5c0 tx=0x7fb44c00da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:19.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.622+0000 7fb44affd640 1 -- 172.21.15.5:0/2037894699 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c004030 con 0x7fb458163520 2024-05-20T18:28:19.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.622+0000 7fb44affd640 1 -- 172.21.15.5:0/2037894699 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb44c0041d0 con 0x7fb458163520 2024-05-20T18:28:19.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.622+0000 7fb44affd640 1 -- 172.21.15.5:0/2037894699 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c014620 con 0x7fb458163520 2024-05-20T18:28:19.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.622+0000 7fb45f39d640 1 -- 172.21.15.5:0/2037894699 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4581f62e0 con 0x7fb458163520 2024-05-20T18:28:19.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.622+0000 7fb45f39d640 1 -- 172.21.15.5:0/2037894699 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4580b64b0 con 0x7fb458163520 2024-05-20T18:28:19.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.623+0000 7fb45f39d640 1 -- 172.21.15.5:0/2037894699 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4581f5d50 con 0x7fb458163520 2024-05-20T18:28:19.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.623+0000 7fb44affd640 1 -- 172.21.15.5:0/2037894699 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb44c0147c0 con 0x7fb458163520 2024-05-20T18:28:19.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.624+0000 7fb44affd640 1 --2- 172.21.15.5:0/2037894699 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb43c06be30 0x7fb43c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:19.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.624+0000 7fb44affd640 1 -- 172.21.15.5:0/2037894699 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7fb44c08b450 con 0x7fb458163520 2024-05-20T18:28:19.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.624+0000 7fb45e39b640 1 --2- 172.21.15.5:0/2037894699 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb43c06be30 0x7fb43c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:19.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.624+0000 7fb45e39b640 1 --2- 172.21.15.5:0/2037894699 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb43c06be30 0x7fb43c06e340 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb454006fd0 tx=0x7fb454008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:19.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.629+0000 7fb44affd640 1 -- 172.21.15.5:0/2037894699 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb44c059f30 con 0x7fb458163520 2024-05-20T18:28:19.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.882+0000 7fb45f39d640 1 -- 172.21.15.5:0/2037894699 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fb4581f6800 con 0x7fb458163520 2024-05-20T18:28:19.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.884+0000 7fb44affd640 1 -- 172.21.15.5:0/2037894699 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7fb44c059830 con 0x7fb458163520 2024-05-20T18:28:19.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.886+0000 7fb45f39d640 1 -- 172.21.15.5:0/2037894699 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb43c06be30 msgr2=0x7fb43c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:19.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.886+0000 7fb45f39d640 1 --2- 172.21.15.5:0/2037894699 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb43c06be30 0x7fb43c06e340 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb454006fd0 tx=0x7fb454008040 comp rx=0 tx=0).stop 2024-05-20T18:28:19.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.886+0000 7fb45f39d640 1 -- 172.21.15.5:0/2037894699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb458163520 msgr2=0x7fb4580b69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:19.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.886+0000 7fb45f39d640 1 --2- 172.21.15.5:0/2037894699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb458163520 0x7fb4580b69d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00d5c0 tx=0x7fb44c00da80 comp rx=0 tx=0).stop 2024-05-20T18:28:19.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.886+0000 7fb45f39d640 1 -- 172.21.15.5:0/2037894699 shutdown_connections 2024-05-20T18:28:19.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.886+0000 7fb45f39d640 1 --2- 172.21.15.5:0/2037894699 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb43c06be30 0x7fb43c06e340 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:19.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.886+0000 7fb45f39d640 1 --2- 172.21.15.5:0/2037894699 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb458163930 0x7fb4580b6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:19.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.886+0000 7fb45f39d640 1 --2- 172.21.15.5:0/2037894699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb458163520 0x7fb4580b69d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:19.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.886+0000 7fb45f39d640 1 --2- 172.21.15.5:0/2037894699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4580cf860 0x7fb45814a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:19.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.886+0000 7fb45f39d640 1 -- 172.21.15.5:0/2037894699 >> 172.21.15.5:0/2037894699 conn(0x7fb45806dbb0 msgr2=0x7fb4580b8380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:19.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.886+0000 7fb45f39d640 1 -- 172.21.15.5:0/2037894699 shutdown_connections 2024-05-20T18:28:19.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:19.886+0000 7fb45f39d640 1 -- 172.21.15.5:0/2037894699 wait complete. 2024-05-20T18:28:19.897 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-05-20T18:28:20.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.013+0000 7f812ba61640 1 Processor -- start 2024-05-20T18:28:20.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.013+0000 7f812ba61640 1 -- start start 2024-05-20T18:28:20.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.014+0000 7f812ba61640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81240b17c0 0x7f81240d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:20.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.014+0000 7f812ba61640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f812406c530 0x7f81240af540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:20.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.014+0000 7f812ba61640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f812406cce0 con 0x7f81240d1310 2024-05-20T18:28:20.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.014+0000 7f812ba61640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f812406cab0 con 0x7f812406c530 2024-05-20T18:28:20.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.014+0000 7f812ba61640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f81240b1bd0 con 0x7f81240b17c0 2024-05-20T18:28:20.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.015+0000 7f8129ffe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f812406c530 0x7f81240af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:20.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.015+0000 7f8128ffc640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f81240d1310 0x7f81240aee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45768/0 (socket says 172.21.15.5:45768) 2024-05-20T18:28:20.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.015+0000 7f8128ffc640 1 -- 172.21.15.5:0/1324405013 learned_addr learned my addr 172.21.15.5:0/1324405013 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:20.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.015+0000 7f8129ffe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f812406c530 0x7f81240af540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37172/0 (socket says 172.21.15.5:37172) 2024-05-20T18:28:20.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.015+0000 7f81297fd640 1 --2- 172.21.15.5:0/1324405013 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81240b17c0 0x7f81240d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:20.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.015+0000 7f8129ffe640 1 -- 172.21.15.5:0/1324405013 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81240b17c0 msgr2=0x7f81240d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:20.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.015+0000 7f8129ffe640 1 --2- 172.21.15.5:0/1324405013 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81240b17c0 0x7f81240d0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:20.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.015+0000 7f8129ffe640 1 -- 172.21.15.5:0/1324405013 >> v1:172.21.15.5:6789/0 conn(0x7f81240d1310 legacy=0x7f81240aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:20.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.015+0000 7f8123fff640 1 -- 172.21.15.5:0/1324405013 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 807978037 0 0) 0x7f812406cce0 con 0x7f81240d1310 2024-05-20T18:28:20.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.015+0000 7f8129ffe640 1 -- 172.21.15.5:0/1324405013 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812409ad20 con 0x7f812406c530 2024-05-20T18:28:20.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.016+0000 7f8129ffe640 1 --2- 172.21.15.5:0/1324405013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f812406c530 0x7f81240af540 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f811c0097d0 tx=0x7f811c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215ad2529de1a8dc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:20.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.016+0000 7f8123fff640 1 -- 172.21.15.5:0/1324405013 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c037070 con 0x7f812406c530 2024-05-20T18:28:20.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.016+0000 7f8123fff640 1 -- 172.21.15.5:0/1324405013 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f811c032440 con 0x7f812406c530 2024-05-20T18:28:20.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.016+0000 7f8123fff640 1 -- 172.21.15.5:0/1324405013 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c032810 con 0x7f812406c530 2024-05-20T18:28:20.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.017+0000 7f812ba61640 1 -- 172.21.15.5:0/1324405013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f812406c530 msgr2=0x7f81240af540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:20.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.017+0000 7f812ba61640 1 --2- 172.21.15.5:0/1324405013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f812406c530 0x7f81240af540 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f811c0097d0 tx=0x7f811c02a1c0 comp rx=0 tx=0).stop 2024-05-20T18:28:20.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.017+0000 7f812ba61640 1 -- 172.21.15.5:0/1324405013 shutdown_connections 2024-05-20T18:28:20.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.017+0000 7f812ba61640 1 --2- 172.21.15.5:0/1324405013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f812406c530 0x7f81240af540 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:20.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.017+0000 7f812ba61640 1 --2- 172.21.15.5:0/1324405013 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81240b17c0 0x7f81240d0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:20.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.017+0000 7f812ba61640 1 -- 172.21.15.5:0/1324405013 >> 172.21.15.5:0/1324405013 conn(0x7f8124072ed0 msgr2=0x7f81240b6080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:20.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.017+0000 7f812ba61640 1 -- 172.21.15.5:0/1324405013 shutdown_connections 2024-05-20T18:28:20.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.017+0000 7f812ba61640 1 -- 172.21.15.5:0/1324405013 wait complete. 2024-05-20T18:28:20.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.018+0000 7f812ba61640 1 Processor -- start 2024-05-20T18:28:20.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.019+0000 7f812ba61640 1 -- start start 2024-05-20T18:28:20.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.020+0000 7f812ba61640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f812406c530 0x7f812415c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:20.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.020+0000 7f812ba61640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81240b17c0 0x7f812414fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:20.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.020+0000 7f812ba61640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81240d1310 0x7f8124157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:20.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.020+0000 7f812ba61640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81240d1f50 con 0x7f812406c530 2024-05-20T18:28:20.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.020+0000 7f812ba61640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f81240d1ac0 con 0x7f81240d1310 2024-05-20T18:28:20.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.020+0000 7f812ba61640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8124074d70 con 0x7f81240b17c0 2024-05-20T18:28:20.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.020+0000 7f81297fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f812406c530 0x7f812415c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:20.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.020+0000 7f81297fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f812406c530 0x7f812415c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37174/0 (socket says 172.21.15.5:37174) 2024-05-20T18:28:20.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.020+0000 7f81297fd640 1 -- 172.21.15.5:0/495000009 learned_addr learned my addr 172.21.15.5:0/495000009 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:20.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.020+0000 7f8128ffc640 1 --2- 172.21.15.5:0/495000009 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81240b17c0 0x7f812414fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:20.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.020+0000 7f8129ffe640 1 --2- 172.21.15.5:0/495000009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81240d1310 0x7f8124157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:20.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.020+0000 7f81297fd640 1 -- 172.21.15.5:0/495000009 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81240b17c0 msgr2=0x7f812414fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:20.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.020+0000 7f81297fd640 1 --2- 172.21.15.5:0/495000009 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81240b17c0 0x7f812414fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:20.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.020+0000 7f81297fd640 1 -- 172.21.15.5:0/495000009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81240d1310 msgr2=0x7f8124157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:20.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.020+0000 7f81297fd640 1 --2- 172.21.15.5:0/495000009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81240d1310 0x7f8124157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:20.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.020+0000 7f81297fd640 1 -- 172.21.15.5:0/495000009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8124151ea0 con 0x7f812406c530 2024-05-20T18:28:20.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.021+0000 7f81297fd640 1 --2- 172.21.15.5:0/495000009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f812406c530 0x7f812415c670 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f811400e870 tx=0x7f811400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:20.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.021+0000 7f8121ffb640 1 -- 172.21.15.5:0/495000009 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8114017070 con 0x7f812406c530 2024-05-20T18:28:20.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.021+0000 7f8121ffb640 1 -- 172.21.15.5:0/495000009 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f811400c940 con 0x7f812406c530 2024-05-20T18:28:20.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.021+0000 7f8121ffb640 1 -- 172.21.15.5:0/495000009 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811400cd30 con 0x7f812406c530 2024-05-20T18:28:20.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.021+0000 7f812ba61640 1 -- 172.21.15.5:0/495000009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81241566d0 con 0x7f812406c530 2024-05-20T18:28:20.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.021+0000 7f812ba61640 1 -- 172.21.15.5:0/495000009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8124156c90 con 0x7f812406c530 2024-05-20T18:28:20.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.022+0000 7f812ba61640 1 -- 172.21.15.5:0/495000009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8124075b00 con 0x7f812406c530 2024-05-20T18:28:20.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.023+0000 7f8121ffb640 1 -- 172.21.15.5:0/495000009 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f811401a070 con 0x7f812406c530 2024-05-20T18:28:20.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.023+0000 7f8121ffb640 1 --2- 172.21.15.5:0/495000009 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f811006be60 0x7f811006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:20.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.023+0000 7f8128ffc640 1 --2- 172.21.15.5:0/495000009 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f811006be60 0x7f811006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:20.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.023+0000 7f8121ffb640 1 -- 172.21.15.5:0/495000009 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f811408aa60 con 0x7f812406c530 2024-05-20T18:28:20.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.024+0000 7f8128ffc640 1 --2- 172.21.15.5:0/495000009 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f811006be60 0x7f811006e370 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f81240b0d10 tx=0x7f8118009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:20.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.028+0000 7f8121ffb640 1 -- 172.21.15.5:0/495000009 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8114058f80 con 0x7f812406c530 2024-05-20T18:28:20.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.280+0000 7f812ba61640 1 -- 172.21.15.5:0/495000009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f8124159ef0 con 0x7f812406c530 2024-05-20T18:28:20.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.282+0000 7f8121ffb640 1 -- 172.21.15.5:0/495000009 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7f811405d280 con 0x7f812406c530 2024-05-20T18:28:20.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.284+0000 7f812ba61640 1 -- 172.21.15.5:0/495000009 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f811006be60 msgr2=0x7f811006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.284+0000 7f812ba61640 1 --2- 172.21.15.5:0/495000009 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f811006be60 0x7f811006e370 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f81240b0d10 tx=0x7f8118009040 comp rx=0 tx=0).stop 2024-05-20T18:28:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.284+0000 7f812ba61640 1 -- 172.21.15.5:0/495000009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f812406c530 msgr2=0x7f812415c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.284+0000 7f812ba61640 1 --2- 172.21.15.5:0/495000009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f812406c530 0x7f812415c670 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f811400e870 tx=0x7f811400ed30 comp rx=0 tx=0).stop 2024-05-20T18:28:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.284+0000 7f812ba61640 1 -- 172.21.15.5:0/495000009 shutdown_connections 2024-05-20T18:28:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.284+0000 7f812ba61640 1 --2- 172.21.15.5:0/495000009 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f811006be60 0x7f811006e370 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.284+0000 7f812ba61640 1 --2- 172.21.15.5:0/495000009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81240d1310 0x7f8124157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.285+0000 7f812ba61640 1 --2- 172.21.15.5:0/495000009 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81240b17c0 0x7f812414fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.285+0000 7f812ba61640 1 --2- 172.21.15.5:0/495000009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f812406c530 0x7f812415c670 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.285+0000 7f812ba61640 1 -- 172.21.15.5:0/495000009 >> 172.21.15.5:0/495000009 conn(0x7f8124072ed0 msgr2=0x7f8124155e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.285+0000 7f812ba61640 1 -- 172.21.15.5:0/495000009 shutdown_connections 2024-05-20T18:28:20.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.285+0000 7f812ba61640 1 -- 172.21.15.5:0/495000009 wait complete. 2024-05-20T18:28:20.295 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-05-20T18:28:20.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.412+0000 7fd87feb6640 1 Processor -- start 2024-05-20T18:28:20.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.412+0000 7fd87feb6640 1 -- start start 2024-05-20T18:28:20.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.414+0000 7fd87feb6640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd878162820 0x7fd8780b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:20.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.414+0000 7fd87feb6640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd87806dae0 0x7fd8780b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:20.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.414+0000 7fd87feb6640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8780d0630 con 0x7fd8780b3e80 2024-05-20T18:28:20.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.414+0000 7fd87feb6640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fd8780d39c0 con 0x7fd87806dae0 2024-05-20T18:28:20.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.414+0000 7fd87feb6640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd8780d3ca0 con 0x7fd878162820 2024-05-20T18:28:20.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.414+0000 7fd87e453640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd87806dae0 0x7fd8780b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:20.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.414+0000 7fd87e453640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd87806dae0 0x7fd8780b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37182/0 (socket says 172.21.15.5:37182) 2024-05-20T18:28:20.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.414+0000 7fd87e453640 1 -- 172.21.15.5:0/1471220505 learned_addr learned my addr 172.21.15.5:0/1471220505 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:20.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.414+0000 7fd87dc52640 1 --2- 172.21.15.5:0/1471220505 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd878162820 0x7fd8780b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:20.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.414+0000 7fd87e453640 1 -- 172.21.15.5:0/1471220505 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd878162820 msgr2=0x7fd8780b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:20.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.414+0000 7fd87e453640 1 --2- 172.21.15.5:0/1471220505 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd878162820 0x7fd8780b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:20.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.414+0000 7fd87e453640 1 -- 172.21.15.5:0/1471220505 >> v1:172.21.15.63:6789/0 conn(0x7fd8780b3e80 legacy=0x7fd87806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:20.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.414+0000 7fd87e453640 1 -- 172.21.15.5:0/1471220505 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8780d3f80 con 0x7fd87806dae0 2024-05-20T18:28:20.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.415+0000 7fd87e453640 1 --2- 172.21.15.5:0/1471220505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd87806dae0 0x7fd8780b8190 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd870009e50 tx=0x7fd87002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2abf04239a560be8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:20.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.415+0000 7fd87cc50640 1 -- 172.21.15.5:0/1471220505 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870036070 con 0x7fd87806dae0 2024-05-20T18:28:20.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.415+0000 7fd87cc50640 1 -- 172.21.15.5:0/1471220505 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd870031600 con 0x7fd87806dae0 2024-05-20T18:28:20.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.415+0000 7fd87cc50640 1 -- 172.21.15.5:0/1471220505 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8700319d0 con 0x7fd87806dae0 2024-05-20T18:28:20.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.416+0000 7fd87feb6640 1 -- 172.21.15.5:0/1471220505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd87806dae0 msgr2=0x7fd8780b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:20.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.416+0000 7fd87feb6640 1 --2- 172.21.15.5:0/1471220505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd87806dae0 0x7fd8780b8190 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd870009e50 tx=0x7fd87002a710 comp rx=0 tx=0).stop 2024-05-20T18:28:20.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.416+0000 7fd87feb6640 1 -- 172.21.15.5:0/1471220505 shutdown_connections 2024-05-20T18:28:20.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.416+0000 7fd87feb6640 1 --2- 172.21.15.5:0/1471220505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd87806dae0 0x7fd8780b8190 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:20.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.416+0000 7fd87feb6640 1 --2- 172.21.15.5:0/1471220505 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd878162820 0x7fd8780b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:20.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.416+0000 7fd87feb6640 1 -- 172.21.15.5:0/1471220505 >> 172.21.15.5:0/1471220505 conn(0x7fd8780d4bd0 msgr2=0x7fd8780d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:20.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.416+0000 7fd87feb6640 1 -- 172.21.15.5:0/1471220505 shutdown_connections 2024-05-20T18:28:20.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.416+0000 7fd87feb6640 1 -- 172.21.15.5:0/1471220505 wait complete. 2024-05-20T18:28:20.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.418+0000 7fd87feb6640 1 Processor -- start 2024-05-20T18:28:20.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.418+0000 7fd87feb6640 1 -- start start 2024-05-20T18:28:20.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.419+0000 7fd87feb6640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8780b3e80 0x7fd8781471d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:20.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.419+0000 7fd87feb6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd878162820 0x7fd8780b8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:20.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.419+0000 7fd87feb6640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8780b88b0 0x7fd8781f4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:20.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.419+0000 7fd87feb6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd8780d4780 con 0x7fd878162820 2024-05-20T18:28:20.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.419+0000 7fd87feb6640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd8780d0300 con 0x7fd8780b3e80 2024-05-20T18:28:20.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.419+0000 7fd87feb6640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd8780d4900 con 0x7fd8780b88b0 2024-05-20T18:28:20.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.420+0000 7fd87d451640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd878162820 0x7fd8780b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:20.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.420+0000 7fd87d451640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd878162820 0x7fd8780b8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37198/0 (socket says 172.21.15.5:37198) 2024-05-20T18:28:20.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.420+0000 7fd87d451640 1 -- 172.21.15.5:0/3078736777 learned_addr learned my addr 172.21.15.5:0/3078736777 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:20.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.420+0000 7fd87dc52640 1 --2- 172.21.15.5:0/3078736777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8780b3e80 0x7fd8781471d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:20.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.420+0000 7fd87e453640 1 --2- 172.21.15.5:0/3078736777 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8780b88b0 0x7fd8781f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:20.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.420+0000 7fd87d451640 1 -- 172.21.15.5:0/3078736777 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8780b88b0 msgr2=0x7fd8781f4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:20.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.420+0000 7fd87d451640 1 --2- 172.21.15.5:0/3078736777 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8780b88b0 0x7fd8781f4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:20.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.420+0000 7fd87d451640 1 -- 172.21.15.5:0/3078736777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8780b3e80 msgr2=0x7fd8781471d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:20.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.420+0000 7fd87d451640 1 --2- 172.21.15.5:0/3078736777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8780b3e80 0x7fd8781471d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:20.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.420+0000 7fd87d451640 1 -- 172.21.15.5:0/3078736777 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd878147710 con 0x7fd878162820 2024-05-20T18:28:20.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.420+0000 7fd87d451640 1 --2- 172.21.15.5:0/3078736777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd878162820 0x7fd8780b8370 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd86800c470 tx=0x7fd86800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:20.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.420+0000 7fd86e7fc640 1 -- 172.21.15.5:0/3078736777 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8680052d0 con 0x7fd878162820 2024-05-20T18:28:20.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.420+0000 7fd86e7fc640 1 -- 172.21.15.5:0/3078736777 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd868005490 con 0x7fd878162820 2024-05-20T18:28:20.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.420+0000 7fd87feb6640 1 -- 172.21.15.5:0/3078736777 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8780b7230 con 0x7fd878162820 2024-05-20T18:28:20.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.421+0000 7fd86e7fc640 1 -- 172.21.15.5:0/3078736777 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868007260 con 0x7fd878162820 2024-05-20T18:28:20.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.421+0000 7fd87feb6640 1 -- 172.21.15.5:0/3078736777 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8781f90c0 con 0x7fd878162820 2024-05-20T18:28:20.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.422+0000 7fd87feb6640 1 -- 172.21.15.5:0/3078736777 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd878075b00 con 0x7fd878162820 2024-05-20T18:28:20.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.422+0000 7fd86e7fc640 1 -- 172.21.15.5:0/3078736777 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd868007400 con 0x7fd878162820 2024-05-20T18:28:20.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.423+0000 7fd86e7fc640 1 --2- 172.21.15.5:0/3078736777 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd85c06be60 0x7fd85c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:20.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.423+0000 7fd86e7fc640 1 -- 172.21.15.5:0/3078736777 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7fd86808a000 con 0x7fd878162820 2024-05-20T18:28:20.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.423+0000 7fd87dc52640 1 --2- 172.21.15.5:0/3078736777 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd85c06be60 0x7fd85c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:20.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.423+0000 7fd87dc52640 1 --2- 172.21.15.5:0/3078736777 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd85c06be60 0x7fd85c06e370 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd874006fd0 tx=0x7fd874008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:20.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.427+0000 7fd86e7fc640 1 -- 172.21.15.5:0/3078736777 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd8680585b0 con 0x7fd878162820 2024-05-20T18:28:20.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.685+0000 7fd87feb6640 1 -- 172.21.15.5:0/3078736777 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fd8780b6fc0 con 0x7fd878162820 2024-05-20T18:28:20.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.688+0000 7fd86e7fc640 1 -- 172.21.15.5:0/3078736777 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7fd86805c8b0 con 0x7fd878162820 2024-05-20T18:28:20.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.690+0000 7fd87feb6640 1 -- 172.21.15.5:0/3078736777 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd85c06be60 msgr2=0x7fd85c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:20.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.690+0000 7fd87feb6640 1 --2- 172.21.15.5:0/3078736777 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd85c06be60 0x7fd85c06e370 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd874006fd0 tx=0x7fd874008040 comp rx=0 tx=0).stop 2024-05-20T18:28:20.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.690+0000 7fd87feb6640 1 -- 172.21.15.5:0/3078736777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd878162820 msgr2=0x7fd8780b8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:20.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.690+0000 7fd87feb6640 1 --2- 172.21.15.5:0/3078736777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd878162820 0x7fd8780b8370 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd86800c470 tx=0x7fd86800c930 comp rx=0 tx=0).stop 2024-05-20T18:28:20.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.690+0000 7fd87feb6640 1 -- 172.21.15.5:0/3078736777 shutdown_connections 2024-05-20T18:28:20.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.690+0000 7fd87feb6640 1 --2- 172.21.15.5:0/3078736777 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd85c06be60 0x7fd85c06e370 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:20.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.690+0000 7fd87feb6640 1 --2- 172.21.15.5:0/3078736777 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8780b88b0 0x7fd8781f4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:20.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.690+0000 7fd87feb6640 1 --2- 172.21.15.5:0/3078736777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd878162820 0x7fd8780b8370 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:20.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.690+0000 7fd87feb6640 1 --2- 172.21.15.5:0/3078736777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8780b3e80 0x7fd8781471d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:20.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.690+0000 7fd87feb6640 1 -- 172.21.15.5:0/3078736777 >> 172.21.15.5:0/3078736777 conn(0x7fd8780d4bd0 msgr2=0x7fd87806da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:20.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.690+0000 7fd87feb6640 1 -- 172.21.15.5:0/3078736777 shutdown_connections 2024-05-20T18:28:20.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.690+0000 7fd87feb6640 1 -- 172.21.15.5:0/3078736777 wait complete. 2024-05-20T18:28:20.701 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:28:20.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.816+0000 7f81edb85640 1 Processor -- start 2024-05-20T18:28:20.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.816+0000 7f81edb85640 1 -- start start 2024-05-20T18:28:20.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.818+0000 7f81edb85640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f81e80cd680 0x7f81e815f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:20.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.818+0000 7f81edb85640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81e80cfae0 0x7f81e806cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:20.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.818+0000 7f81edb85640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81e806d0f0 con 0x7f81e806c6c0 2024-05-20T18:28:20.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.818+0000 7f81edb85640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f81e80cda90 con 0x7f81e80cd680 2024-05-20T18:28:20.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.818+0000 7f81edb85640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f81e815f760 con 0x7f81e80cfae0 2024-05-20T18:28:20.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.818+0000 7f81e7fff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81e80cfae0 0x7f81e806cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:20.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.818+0000 7f81e77fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f81e80cd680 0x7f81e815f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:20.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.818+0000 7f81e77fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f81e80cd680 0x7f81e815f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:45918/0 (socket says 172.21.15.5:45918) 2024-05-20T18:28:20.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.818+0000 7f81e7fff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81e80cfae0 0x7f81e806cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:56096/0 (socket says 172.21.15.5:56096) 2024-05-20T18:28:20.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.818+0000 7f81e6ffd640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f81e806c6c0 0x7f81e80cf3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:49474/0 (socket says 172.21.15.5:49474) 2024-05-20T18:28:20.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.818+0000 7f81e77fe640 1 -- 172.21.15.5:0/4287804362 learned_addr learned my addr 172.21.15.5:0/4287804362 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:20.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.819+0000 7f81e77fe640 1 -- 172.21.15.5:0/4287804362 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81e80cfae0 msgr2=0x7f81e806cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:20.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.819+0000 7f81e77fe640 1 --2- 172.21.15.5:0/4287804362 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81e80cfae0 0x7f81e806cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:20.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.819+0000 7f81e77fe640 1 -- 172.21.15.5:0/4287804362 >> v1:172.21.15.63:6789/0 conn(0x7f81e806c6c0 legacy=0x7f81e80cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:20.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.819+0000 7f81e77fe640 1 -- 172.21.15.5:0/4287804362 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e806d3d0 con 0x7f81e80cd680 2024-05-20T18:28:20.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.819+0000 7f81e7fff640 1 --2- 172.21.15.5:0/4287804362 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81e80cfae0 0x7f81e806cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:28:20.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.819+0000 7f81e77fe640 1 --2- 172.21.15.5:0/4287804362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81e80cd680 0x7f81e815f220 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009870 tx=0x7f81d802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fa22b7d21222ce1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:20.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.820+0000 7f81e67fc640 1 -- 172.21.15.5:0/4287804362 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d8037070 con 0x7f81e80cd680 2024-05-20T18:28:20.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.820+0000 7f81e67fc640 1 -- 172.21.15.5:0/4287804362 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81d8032440 con 0x7f81e80cd680 2024-05-20T18:28:20.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.820+0000 7f81e67fc640 1 -- 172.21.15.5:0/4287804362 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d8032810 con 0x7f81e80cd680 2024-05-20T18:28:20.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.820+0000 7f81edb85640 1 -- 172.21.15.5:0/4287804362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81e80cd680 msgr2=0x7f81e815f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:20.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.820+0000 7f81edb85640 1 --2- 172.21.15.5:0/4287804362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81e80cd680 0x7f81e815f220 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009870 tx=0x7f81d802a1c0 comp rx=0 tx=0).stop 2024-05-20T18:28:20.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.820+0000 7f81edb85640 1 -- 172.21.15.5:0/4287804362 shutdown_connections 2024-05-20T18:28:20.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.820+0000 7f81edb85640 1 --2- 172.21.15.5:0/4287804362 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81e80cfae0 0x7f81e806cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:20.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.820+0000 7f81edb85640 1 --2- 172.21.15.5:0/4287804362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81e80cd680 0x7f81e815f220 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:20.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.820+0000 7f81edb85640 1 -- 172.21.15.5:0/4287804362 >> 172.21.15.5:0/4287804362 conn(0x7f81e80b44d0 msgr2=0x7f81e80b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:20.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.820+0000 7f81edb85640 1 -- 172.21.15.5:0/4287804362 shutdown_connections 2024-05-20T18:28:20.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.820+0000 7f81edb85640 1 -- 172.21.15.5:0/4287804362 wait complete. 2024-05-20T18:28:20.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.822+0000 7f81edb85640 1 Processor -- start 2024-05-20T18:28:20.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.822+0000 7f81edb85640 1 -- start start 2024-05-20T18:28:20.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.823+0000 7f81edb85640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81e806c6c0 0x7f81e81ea340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:20.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.823+0000 7f81edb85640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81e80cd680 0x7f81e814f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:20.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.824+0000 7f81edb85640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81e815a540 0x7f81e814fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:20.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.824+0000 7f81edb85640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81e806d990 con 0x7f81e80cd680 2024-05-20T18:28:20.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.824+0000 7f81edb85640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f81e80d0b40 con 0x7f81e806c6c0 2024-05-20T18:28:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.824+0000 7f81edb85640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f81e80d2460 con 0x7f81e815a540 2024-05-20T18:28:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.824+0000 7f81e6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81e80cd680 0x7f81e814f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.824+0000 7f81e6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81e80cd680 0x7f81e814f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37210/0 (socket says 172.21.15.5:37210) 2024-05-20T18:28:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.824+0000 7f81e6ffd640 1 -- 172.21.15.5:0/2558647144 learned_addr learned my addr 172.21.15.5:0/2558647144 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.824+0000 7f81e7fff640 1 --2- 172.21.15.5:0/2558647144 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81e815a540 0x7f81e814fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.824+0000 7f81e77fe640 1 --2- 172.21.15.5:0/2558647144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81e806c6c0 0x7f81e81ea340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.824+0000 7f81e6ffd640 1 -- 172.21.15.5:0/2558647144 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81e815a540 msgr2=0x7f81e814fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.824+0000 7f81e6ffd640 1 --2- 172.21.15.5:0/2558647144 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81e815a540 0x7f81e814fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.824+0000 7f81e6ffd640 1 -- 172.21.15.5:0/2558647144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81e806c6c0 msgr2=0x7f81e81ea340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.824+0000 7f81e6ffd640 1 --2- 172.21.15.5:0/2558647144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81e806c6c0 0x7f81e81ea340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.824+0000 7f81e6ffd640 1 -- 172.21.15.5:0/2558647144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e815b6f0 con 0x7f81e80cd680 2024-05-20T18:28:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.824+0000 7f81e6ffd640 1 --2- 172.21.15.5:0/2558647144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81e80cd680 0x7f81e814f720 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00c3f0 tx=0x7f81dc00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.824+0000 7f81cffff640 1 -- 172.21.15.5:0/2558647144 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81dc0074e0 con 0x7f81e80cd680 2024-05-20T18:28:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.824+0000 7f81cffff640 1 -- 172.21.15.5:0/2558647144 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81dc007680 con 0x7f81e80cd680 2024-05-20T18:28:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.824+0000 7f81edb85640 1 -- 172.21.15.5:0/2558647144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81e81e92a0 con 0x7f81e80cd680 2024-05-20T18:28:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.825+0000 7f81cffff640 1 -- 172.21.15.5:0/2558647144 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81dc004a00 con 0x7f81e80cd680 2024-05-20T18:28:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.825+0000 7f81edb85640 1 -- 172.21.15.5:0/2558647144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81e81e9860 con 0x7f81e80cd680 2024-05-20T18:28:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.826+0000 7f81edb85640 1 -- 172.21.15.5:0/2558647144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81e81e9010 con 0x7f81e80cd680 2024-05-20T18:28:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.826+0000 7f81cffff640 1 -- 172.21.15.5:0/2558647144 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f81dc01a020 con 0x7f81e80cd680 2024-05-20T18:28:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.827+0000 7f81cffff640 1 --2- 172.21.15.5:0/2558647144 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f81d406be60 0x7f81d406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.827+0000 7f81cffff640 1 -- 172.21.15.5:0/2558647144 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f81dc08a330 con 0x7f81e80cd680 2024-05-20T18:28:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.827+0000 7f81e77fe640 1 --2- 172.21.15.5:0/2558647144 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f81d406be60 0x7f81d406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.827+0000 7f81e77fe640 1 --2- 172.21.15.5:0/2558647144 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f81d406be60 0x7f81d406e370 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f81d8034000 tx=0x7f81d802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:20.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:20.831+0000 7f81cffff640 1 -- 172.21.15.5:0/2558647144 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f81dc058e10 con 0x7f81e80cd680 2024-05-20T18:28:21.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.017+0000 7f81edb85640 1 -- 172.21.15.5:0/2558647144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f81e8075b00 con 0x7f81e80cd680 2024-05-20T18:28:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.018+0000 7f81cffff640 1 -- 172.21.15.5:0/2558647144 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+13044 (secure 0 0 0) 0x7f81dc058710 con 0x7f81e80cd680 2024-05-20T18:28:21.017 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:21.017 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":36,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:28:17.864502+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:28:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.020+0000 7f81edb85640 1 -- 172.21.15.5:0/2558647144 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f81d406be60 msgr2=0x7f81d406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.020+0000 7f81edb85640 1 --2- 172.21.15.5:0/2558647144 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f81d406be60 0x7f81d406e370 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f81d8034000 tx=0x7f81d802e000 comp rx=0 tx=0).stop 2024-05-20T18:28:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.020+0000 7f81edb85640 1 -- 172.21.15.5:0/2558647144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81e80cd680 msgr2=0x7f81e814f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:21.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.020+0000 7f81edb85640 1 --2- 172.21.15.5:0/2558647144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81e80cd680 0x7f81e814f720 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00c3f0 tx=0x7f81dc00c7f0 comp rx=0 tx=0).stop 2024-05-20T18:28:21.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.020+0000 7f81edb85640 1 -- 172.21.15.5:0/2558647144 shutdown_connections 2024-05-20T18:28:21.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.020+0000 7f81edb85640 1 --2- 172.21.15.5:0/2558647144 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f81d406be60 0x7f81d406e370 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:21.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.020+0000 7f81edb85640 1 --2- 172.21.15.5:0/2558647144 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81e815a540 0x7f81e814fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:21.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.020+0000 7f81edb85640 1 --2- 172.21.15.5:0/2558647144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81e80cd680 0x7f81e814f720 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:21.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.020+0000 7f81edb85640 1 --2- 172.21.15.5:0/2558647144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81e806c6c0 0x7f81e81ea340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:21.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.020+0000 7f81edb85640 1 -- 172.21.15.5:0/2558647144 >> 172.21.15.5:0/2558647144 conn(0x7f81e80b44d0 msgr2=0x7f81e80b3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:21.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.020+0000 7f81edb85640 1 -- 172.21.15.5:0/2558647144 shutdown_connections 2024-05-20T18:28:21.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.020+0000 7f81edb85640 1 -- 172.21.15.5:0/2558647144 wait complete. 2024-05-20T18:28:21.031 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:28:21.031 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:28:21.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.149+0000 7f7de5977640 1 Processor -- start 2024-05-20T18:28:21.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.149+0000 7f7de5977640 1 -- start start 2024-05-20T18:28:21.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.151+0000 7f7de5977640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7de00af030 0x7f7de00af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:21.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.151+0000 7f7de5977640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7de015f700 0x7f7de015fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:21.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.151+0000 7f7de5977640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7de00af980 con 0x7f7de009a4f0 2024-05-20T18:28:21.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.151+0000 7f7de5977640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7de006bcf0 con 0x7f7de00af030 2024-05-20T18:28:21.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.151+0000 7f7de5977640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7de0160430 con 0x7f7de015f700 2024-05-20T18:28:21.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.151+0000 7f7ddeffd640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7de00af030 0x7f7de00af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:21.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.151+0000 7f7dde7fc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7de015f700 0x7f7de015fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:21.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.151+0000 7f7ddeffd640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7de00af030 0x7f7de00af440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:45948/0 (socket says 172.21.15.5:45948) 2024-05-20T18:28:21.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.151+0000 7f7ddeffd640 1 -- 172.21.15.5:0/2486014048 learned_addr learned my addr 172.21.15.5:0/2486014048 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:21.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.151+0000 7f7ddf7fe640 1 --1- 172.21.15.5:0/2486014048 >> v1:172.21.15.63:6789/0 conn(0x7f7de009a4f0 0x7f7de009a900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:49482/0 (socket says 172.21.15.5:49482) 2024-05-20T18:28:21.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.152+0000 7f7dde7fc640 1 -- 172.21.15.5:0/2486014048 >> v2:172.21.15.63:3300/0 conn(0x7f7de00af030 msgr2=0x7f7de00af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:21.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.152+0000 7f7dde7fc640 1 --2- 172.21.15.5:0/2486014048 >> v2:172.21.15.63:3300/0 conn(0x7f7de00af030 0x7f7de00af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:21.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.152+0000 7f7dde7fc640 1 -- 172.21.15.5:0/2486014048 >> v1:172.21.15.63:6789/0 conn(0x7f7de009a4f0 legacy=0x7f7de009a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:21.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.152+0000 7f7dde7fc640 1 -- 172.21.15.5:0/2486014048 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de006b6a0 con 0x7f7de015f700 2024-05-20T18:28:21.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.152+0000 7f7ddeffd640 1 --2- 172.21.15.5:0/2486014048 >> v2:172.21.15.63:3300/0 conn(0x7f7de00af030 0x7f7de00af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:28:21.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.153+0000 7f7dde7fc640 1 --2- 172.21.15.5:0/2486014048 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7de015f700 0x7f7de015fb10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7dd40097a0 tx=0x7f7dd402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90290da6addc850b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:21.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.153+0000 7f7dddffb640 1 -- 172.21.15.5:0/2486014048 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4037070 con 0x7f7de015f700 2024-05-20T18:28:21.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.153+0000 7f7dddffb640 1 -- 172.21.15.5:0/2486014048 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dd4032440 con 0x7f7de015f700 2024-05-20T18:28:21.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.153+0000 7f7dddffb640 1 -- 172.21.15.5:0/2486014048 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4032810 con 0x7f7de015f700 2024-05-20T18:28:21.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.154+0000 7f7de5977640 1 -- 172.21.15.5:0/2486014048 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7de015f700 msgr2=0x7f7de015fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:21.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.154+0000 7f7de5977640 1 --2- 172.21.15.5:0/2486014048 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7de015f700 0x7f7de015fb10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7dd40097a0 tx=0x7f7dd402a180 comp rx=0 tx=0).stop 2024-05-20T18:28:21.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.154+0000 7f7de5977640 1 -- 172.21.15.5:0/2486014048 shutdown_connections 2024-05-20T18:28:21.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.154+0000 7f7de5977640 1 --2- 172.21.15.5:0/2486014048 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7de015f700 0x7f7de015fb10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:21.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.154+0000 7f7de5977640 1 --2- 172.21.15.5:0/2486014048 >> v2:172.21.15.63:3300/0 conn(0x7f7de00af030 0x7f7de00af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:21.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.154+0000 7f7de5977640 1 -- 172.21.15.5:0/2486014048 >> 172.21.15.5:0/2486014048 conn(0x7f7de00b6620 msgr2=0x7f7de00b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:21.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.154+0000 7f7de5977640 1 -- 172.21.15.5:0/2486014048 shutdown_connections 2024-05-20T18:28:21.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.154+0000 7f7de5977640 1 -- 172.21.15.5:0/2486014048 wait complete. 2024-05-20T18:28:21.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.155+0000 7f7de5977640 1 Processor -- start 2024-05-20T18:28:21.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.156+0000 7f7de5977640 1 -- start start 2024-05-20T18:28:21.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.157+0000 7f7de5977640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7de009a4f0 0x7f7de015e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:21.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.157+0000 7f7de5977640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7de00af030 0x7f7de01607f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.157+0000 7f7de5977640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7de015f700 0x7f7de014c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.157+0000 7f7de5977640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7de0160050 con 0x7f7de009a4f0 2024-05-20T18:28:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.157+0000 7f7de5977640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7de0159aa0 con 0x7f7de015f700 2024-05-20T18:28:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.157+0000 7f7de5977640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7de01601d0 con 0x7f7de00af030 2024-05-20T18:28:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.157+0000 7f7ddeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7de009a4f0 0x7f7de015e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.157+0000 7f7ddeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7de009a4f0 0x7f7de015e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37218/0 (socket says 172.21.15.5:37218) 2024-05-20T18:28:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.157+0000 7f7ddeffd640 1 -- 172.21.15.5:0/3598272549 learned_addr learned my addr 172.21.15.5:0/3598272549 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.157+0000 7f7ddf7fe640 1 --2- 172.21.15.5:0/3598272549 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7de015f700 0x7f7de014c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.157+0000 7f7dde7fc640 1 --2- 172.21.15.5:0/3598272549 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7de00af030 0x7f7de01607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.157+0000 7f7ddeffd640 1 -- 172.21.15.5:0/3598272549 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7de00af030 msgr2=0x7f7de01607f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.157+0000 7f7ddeffd640 1 --2- 172.21.15.5:0/3598272549 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7de00af030 0x7f7de01607f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.157+0000 7f7ddeffd640 1 -- 172.21.15.5:0/3598272549 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7de015f700 msgr2=0x7f7de014c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.157+0000 7f7ddeffd640 1 --2- 172.21.15.5:0/3598272549 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7de015f700 0x7f7de014c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:21.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.157+0000 7f7ddeffd640 1 -- 172.21.15.5:0/3598272549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de014e090 con 0x7f7de009a4f0 2024-05-20T18:28:21.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.158+0000 7f7ddeffd640 1 --2- 172.21.15.5:0/3598272549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7de009a4f0 0x7f7de015e8f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000efc0 tx=0x7f7dd000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:21.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.158+0000 7f7dc7fff640 1 -- 172.21.15.5:0/3598272549 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd000ce00 con 0x7f7de009a4f0 2024-05-20T18:28:21.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.158+0000 7f7dc7fff640 1 -- 172.21.15.5:0/3598272549 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dd0019070 con 0x7f7de009a4f0 2024-05-20T18:28:21.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.158+0000 7f7dc7fff640 1 -- 172.21.15.5:0/3598272549 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd00146a0 con 0x7f7de009a4f0 2024-05-20T18:28:21.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.158+0000 7f7de5977640 1 -- 172.21.15.5:0/3598272549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7de015e5c0 con 0x7f7de009a4f0 2024-05-20T18:28:21.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.158+0000 7f7de5977640 1 -- 172.21.15.5:0/3598272549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7de015e2e0 con 0x7f7de009a4f0 2024-05-20T18:28:21.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.159+0000 7f7de5977640 1 -- 172.21.15.5:0/3598272549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7de006d390 con 0x7f7de009a4f0 2024-05-20T18:28:21.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.159+0000 7f7dc7fff640 1 -- 172.21.15.5:0/3598272549 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7dd0005d10 con 0x7f7de009a4f0 2024-05-20T18:28:21.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.160+0000 7f7dc7fff640 1 --2- 172.21.15.5:0/3598272549 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7dcc06be30 0x7f7dcc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:21.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.160+0000 7f7dc7fff640 1 -- 172.21.15.5:0/3598272549 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f7dd008b040 con 0x7f7de009a4f0 2024-05-20T18:28:21.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.160+0000 7f7dde7fc640 1 --2- 172.21.15.5:0/3598272549 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7dcc06be30 0x7f7dcc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:21.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.160+0000 7f7dde7fc640 1 --2- 172.21.15.5:0/3598272549 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7dcc06be30 0x7f7dcc06e340 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4034000 tx=0x7f7dd402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:21.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.164+0000 7f7dc7fff640 1 -- 172.21.15.5:0/3598272549 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7de006d390 con 0x7f7de009a4f0 2024-05-20T18:28:21.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.349+0000 7f7de5977640 1 -- 172.21.15.5:0/3598272549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7de0075b00 con 0x7f7de009a4f0 2024-05-20T18:28:21.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.350+0000 7f7dc7fff640 1 -- 172.21.15.5:0/3598272549 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7f7de014e090 con 0x7f7de009a4f0 2024-05-20T18:28:21.349 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:28:21.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.352+0000 7f7de5977640 1 -- 172.21.15.5:0/3598272549 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7dcc06be30 msgr2=0x7f7dcc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:21.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.352+0000 7f7de5977640 1 --2- 172.21.15.5:0/3598272549 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7dcc06be30 0x7f7dcc06e340 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4034000 tx=0x7f7dd402e000 comp rx=0 tx=0).stop 2024-05-20T18:28:21.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.352+0000 7f7de5977640 1 -- 172.21.15.5:0/3598272549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7de009a4f0 msgr2=0x7f7de015e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:21.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.352+0000 7f7de5977640 1 --2- 172.21.15.5:0/3598272549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7de009a4f0 0x7f7de015e8f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000efc0 tx=0x7f7dd000c490 comp rx=0 tx=0).stop 2024-05-20T18:28:21.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.352+0000 7f7de5977640 1 -- 172.21.15.5:0/3598272549 shutdown_connections 2024-05-20T18:28:21.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.352+0000 7f7de5977640 1 --2- 172.21.15.5:0/3598272549 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7dcc06be30 0x7f7dcc06e340 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:21.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.352+0000 7f7de5977640 1 --2- 172.21.15.5:0/3598272549 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7de015f700 0x7f7de014c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:21.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.352+0000 7f7de5977640 1 --2- 172.21.15.5:0/3598272549 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7de00af030 0x7f7de01607f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:21.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.352+0000 7f7de5977640 1 --2- 172.21.15.5:0/3598272549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7de009a4f0 0x7f7de015e8f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:21.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.352+0000 7f7de5977640 1 -- 172.21.15.5:0/3598272549 >> 172.21.15.5:0/3598272549 conn(0x7f7de00b6620 msgr2=0x7f7de0158de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:21.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.352+0000 7f7de5977640 1 -- 172.21.15.5:0/3598272549 shutdown_connections 2024-05-20T18:28:21.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.352+0000 7f7de5977640 1 -- 172.21.15.5:0/3598272549 wait complete. 2024-05-20T18:28:21.363 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-20T18:28:21.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.478+0000 7ffa82a51640 1 Processor -- start 2024-05-20T18:28:21.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.478+0000 7ffa82a51640 1 -- start start 2024-05-20T18:28:21.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.479+0000 7ffa82a51640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ffa7c071630 0x7ffa7c15f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:21.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.479+0000 7ffa82a51640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ffa7c0cd1b0 0x7ffa7c06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:21.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.479+0000 7ffa82a51640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa7c0ce2b0 con 0x7ffa7c070850 2024-05-20T18:28:21.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.479+0000 7ffa82a51640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7ffa7c15f790 con 0x7ffa7c071630 2024-05-20T18:28:21.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.479+0000 7ffa82a51640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ffa7c06e590 con 0x7ffa7c0cd1b0 2024-05-20T18:28:21.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.479+0000 7ffa7bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ffa7c071630 0x7ffa7c15f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:21.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.480+0000 7ffa7bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ffa7c071630 0x7ffa7c15f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37234/0 (socket says 172.21.15.5:37234) 2024-05-20T18:28:21.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.480+0000 7ffa7bfff640 1 -- 172.21.15.5:0/1804827947 learned_addr learned my addr 172.21.15.5:0/1804827947 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:21.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.480+0000 7ffa7bfff640 1 -- 172.21.15.5:0/1804827947 >> v2:172.21.15.63:3300/0 conn(0x7ffa7c0cd1b0 msgr2=0x7ffa7c06d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:21.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.480+0000 7ffa7bfff640 1 --2- 172.21.15.5:0/1804827947 >> v2:172.21.15.63:3300/0 conn(0x7ffa7c0cd1b0 0x7ffa7c06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:21.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.480+0000 7ffa7bfff640 1 -- 172.21.15.5:0/1804827947 >> v1:172.21.15.63:6789/0 conn(0x7ffa7c070850 legacy=0x7ffa7c070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:21.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.480+0000 7ffa7bfff640 1 -- 172.21.15.5:0/1804827947 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7c06cce0 con 0x7ffa7c071630 2024-05-20T18:28:21.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.480+0000 7ffa7bfff640 1 --2- 172.21.15.5:0/1804827947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa7c071630 0x7ffa7c15f250 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0ccfa0 tx=0x7ffa6c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea9b565d4efed server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:21.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.481+0000 7ffa7affd640 1 -- 172.21.15.5:0/1804827947 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c037070 con 0x7ffa7c071630 2024-05-20T18:28:21.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.481+0000 7ffa7affd640 1 -- 172.21.15.5:0/1804827947 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa6c032440 con 0x7ffa7c071630 2024-05-20T18:28:21.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.481+0000 7ffa7affd640 1 -- 172.21.15.5:0/1804827947 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c032810 con 0x7ffa7c071630 2024-05-20T18:28:21.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.481+0000 7ffa82a51640 1 -- 172.21.15.5:0/1804827947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa7c071630 msgr2=0x7ffa7c15f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:21.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.481+0000 7ffa82a51640 1 --2- 172.21.15.5:0/1804827947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa7c071630 0x7ffa7c15f250 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0ccfa0 tx=0x7ffa6c02a1a0 comp rx=0 tx=0).stop 2024-05-20T18:28:21.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.481+0000 7ffa82a51640 1 -- 172.21.15.5:0/1804827947 shutdown_connections 2024-05-20T18:28:21.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.481+0000 7ffa82a51640 1 --2- 172.21.15.5:0/1804827947 >> v2:172.21.15.63:3300/0 conn(0x7ffa7c0cd1b0 0x7ffa7c06d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:21.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.481+0000 7ffa82a51640 1 --2- 172.21.15.5:0/1804827947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa7c071630 0x7ffa7c15f250 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:21.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.481+0000 7ffa82a51640 1 -- 172.21.15.5:0/1804827947 >> 172.21.15.5:0/1804827947 conn(0x7ffa7c0d29d0 msgr2=0x7ffa7c0d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:21.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.482+0000 7ffa82a51640 1 -- 172.21.15.5:0/1804827947 shutdown_connections 2024-05-20T18:28:21.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.482+0000 7ffa82a51640 1 -- 172.21.15.5:0/1804827947 wait complete. 2024-05-20T18:28:21.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.483+0000 7ffa82a51640 1 Processor -- start 2024-05-20T18:28:21.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.483+0000 7ffa82a51640 1 -- start start 2024-05-20T18:28:21.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.485+0000 7ffa82a51640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa7c070850 0x7ffa7c1eb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:21.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.485+0000 7ffa82a51640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa7c071630 0x7ffa7c1f5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:21.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.485+0000 7ffa82a51640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa7c0cd1b0 0x7ffa7c15f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:21.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.485+0000 7ffa82a51640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c06d2a0 con 0x7ffa7c070850 2024-05-20T18:28:21.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.485+0000 7ffa82a51640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c06c990 con 0x7ffa7c0cd1b0 2024-05-20T18:28:21.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.485+0000 7ffa82a51640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffa7c06dd00 con 0x7ffa7c071630 2024-05-20T18:28:21.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.485+0000 7ffa7bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa7c070850 0x7ffa7c1eb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:21.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.485+0000 7ffa7bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa7c070850 0x7ffa7c1eb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37240/0 (socket says 172.21.15.5:37240) 2024-05-20T18:28:21.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.485+0000 7ffa7bfff640 1 -- 172.21.15.5:0/1238381160 learned_addr learned my addr 172.21.15.5:0/1238381160 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:21.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.485+0000 7ffa80fee640 1 --2- 172.21.15.5:0/1238381160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa7c0cd1b0 0x7ffa7c15f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:21.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.485+0000 7ffa7b7fe640 1 --2- 172.21.15.5:0/1238381160 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa7c071630 0x7ffa7c1f5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:21.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.485+0000 7ffa7bfff640 1 -- 172.21.15.5:0/1238381160 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa7c071630 msgr2=0x7ffa7c1f5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:21.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.485+0000 7ffa7bfff640 1 --2- 172.21.15.5:0/1238381160 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa7c071630 0x7ffa7c1f5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:21.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.485+0000 7ffa7bfff640 1 -- 172.21.15.5:0/1238381160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa7c0cd1b0 msgr2=0x7ffa7c15f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:21.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.485+0000 7ffa7bfff640 1 --2- 172.21.15.5:0/1238381160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa7c0cd1b0 0x7ffa7c15f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:21.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.485+0000 7ffa7bfff640 1 -- 172.21.15.5:0/1238381160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7c1ea460 con 0x7ffa7c070850 2024-05-20T18:28:21.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.485+0000 7ffa7bfff640 1 --2- 172.21.15.5:0/1238381160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa7c070850 0x7ffa7c1eb500 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c02a1e0 tx=0x7ffa6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:21.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.486+0000 7ffa78ff9640 1 -- 172.21.15.5:0/1238381160 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c041070 con 0x7ffa7c070850 2024-05-20T18:28:21.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.486+0000 7ffa78ff9640 1 -- 172.21.15.5:0/1238381160 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa6c03c870 con 0x7ffa7c070850 2024-05-20T18:28:21.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.486+0000 7ffa82a51640 1 -- 172.21.15.5:0/1238381160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa7c1ea740 con 0x7ffa7c070850 2024-05-20T18:28:21.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.486+0000 7ffa78ff9640 1 -- 172.21.15.5:0/1238381160 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c037040 con 0x7ffa7c070850 2024-05-20T18:28:21.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.486+0000 7ffa82a51640 1 -- 172.21.15.5:0/1238381160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa7c1ead00 con 0x7ffa7c070850 2024-05-20T18:28:21.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.487+0000 7ffa82a51640 1 -- 172.21.15.5:0/1238381160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa7c075b00 con 0x7ffa7c070850 2024-05-20T18:28:21.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.487+0000 7ffa78ff9640 1 -- 172.21.15.5:0/1238381160 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffa6c044070 con 0x7ffa7c070850 2024-05-20T18:28:21.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.488+0000 7ffa78ff9640 1 --2- 172.21.15.5:0/1238381160 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffa6806be30 0x7ffa6806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:21.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.488+0000 7ffa78ff9640 1 -- 172.21.15.5:0/1238381160 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7ffa6c0ab9f0 con 0x7ffa7c070850 2024-05-20T18:28:21.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.488+0000 7ffa7b7fe640 1 --2- 172.21.15.5:0/1238381160 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffa6806be30 0x7ffa6806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:21.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.488+0000 7ffa7b7fe640 1 --2- 172.21.15.5:0/1238381160 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffa6806be30 0x7ffa6806e340 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0b86a0 tx=0x7ffa70008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:21.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.493+0000 7ffa78ff9640 1 -- 172.21.15.5:0/1238381160 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ffa6c07a600 con 0x7ffa7c070850 2024-05-20T18:28:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.679+0000 7ffa82a51640 1 -- 172.21.15.5:0/1238381160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ffa7c071410 con 0x7ffa7c070850 2024-05-20T18:28:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.679+0000 7ffa78ff9640 1 -- 172.21.15.5:0/1238381160 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v36) ==== 84+0+65 (secure 0 0 0) 0x7ffa6c079f00 con 0x7ffa7c070850 2024-05-20T18:28:21.679 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:21.679 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-05-20T18:28:21.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:21.679 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-05-20T18:28:21.679 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-05-20T18:28:21.679 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:21.679 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-05-20T18:28:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.681+0000 7ffa82a51640 1 -- 172.21.15.5:0/1238381160 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffa6806be30 msgr2=0x7ffa6806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.681+0000 7ffa82a51640 1 --2- 172.21.15.5:0/1238381160 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffa6806be30 0x7ffa6806e340 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0b86a0 tx=0x7ffa70008040 comp rx=0 tx=0).stop 2024-05-20T18:28:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.681+0000 7ffa82a51640 1 -- 172.21.15.5:0/1238381160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa7c070850 msgr2=0x7ffa7c1eb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.681+0000 7ffa82a51640 1 --2- 172.21.15.5:0/1238381160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa7c070850 0x7ffa7c1eb500 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c02a1e0 tx=0x7ffa6c0046c0 comp rx=0 tx=0).stop 2024-05-20T18:28:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.681+0000 7ffa82a51640 1 -- 172.21.15.5:0/1238381160 shutdown_connections 2024-05-20T18:28:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.681+0000 7ffa82a51640 1 --2- 172.21.15.5:0/1238381160 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffa6806be30 0x7ffa6806e340 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.681+0000 7ffa82a51640 1 --2- 172.21.15.5:0/1238381160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa7c0cd1b0 0x7ffa7c15f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.681+0000 7ffa82a51640 1 --2- 172.21.15.5:0/1238381160 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa7c071630 0x7ffa7c1f5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.681+0000 7ffa82a51640 1 --2- 172.21.15.5:0/1238381160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa7c070850 0x7ffa7c1eb500 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.681+0000 7ffa82a51640 1 -- 172.21.15.5:0/1238381160 >> 172.21.15.5:0/1238381160 conn(0x7ffa7c0d29d0 msgr2=0x7ffa7c159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.681+0000 7ffa82a51640 1 -- 172.21.15.5:0/1238381160 shutdown_connections 2024-05-20T18:28:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.681+0000 7ffa82a51640 1 -- 172.21.15.5:0/1238381160 wait complete. 2024-05-20T18:28:21.692 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-20T18:28:21.692 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-20T18:28:21.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.811+0000 7f596b07a640 1 Processor -- start 2024-05-20T18:28:21.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.811+0000 7f596b07a640 1 -- start start 2024-05-20T18:28:21.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.813+0000 7f596b07a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f596415d860 0x7f596406f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:21.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.813+0000 7f596b07a640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5964070250 con 0x7f596406fe40 2024-05-20T18:28:21.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.813+0000 7f596b07a640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59640afe40 con 0x7f596409e740 2024-05-20T18:28:21.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.813+0000 7f596b07a640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f596415dc70 con 0x7f596415d860 2024-05-20T18:28:21.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.813+0000 7f5963fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f596415d860 0x7f596406f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:21.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.813+0000 7f5969617640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f596406fe40 0x7f596415c6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45772/0 (socket says 172.21.15.5:45772) 2024-05-20T18:28:21.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.813+0000 7f5969617640 1 -- 172.21.15.5:0/1459044955 learned_addr learned my addr 172.21.15.5:0/1459044955 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:21.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.813+0000 7f5963fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f596415d860 0x7f596406f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37250/0 (socket says 172.21.15.5:37250) 2024-05-20T18:28:21.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.813+0000 7f5963fff640 1 -- 172.21.15.5:0/1459044955 >> v1:172.21.15.63:6789/0 conn(0x7f596409e740 legacy=0x7f59640af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:21.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.813+0000 7f5963fff640 1 -- 172.21.15.5:0/1459044955 >> v1:172.21.15.5:6789/0 conn(0x7f596406fe40 legacy=0x7f596415c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:21.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.813+0000 7f59637fe640 1 -- 172.21.15.5:0/1459044955 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3818615596 0 0) 0x7f595c001490 con 0x7f596406fe40 2024-05-20T18:28:21.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.813+0000 7f5963fff640 1 -- 172.21.15.5:0/1459044955 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59640b00f0 con 0x7f596415d860 2024-05-20T18:28:21.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.814+0000 7f5963fff640 1 --2- 172.21.15.5:0/1459044955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f596415d860 0x7f596406f900 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f59580097d0 tx=0x7f595802c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0ec917ad30db27 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:21.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.814+0000 7f59637fe640 1 -- 172.21.15.5:0/1459044955 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5958039070 con 0x7f596415d860 2024-05-20T18:28:21.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.814+0000 7f59637fe640 1 -- 172.21.15.5:0/1459044955 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5958034640 con 0x7f596415d860 2024-05-20T18:28:21.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.814+0000 7f59637fe640 1 -- 172.21.15.5:0/1459044955 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5958034a10 con 0x7f596415d860 2024-05-20T18:28:21.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.815+0000 7f596b07a640 1 -- 172.21.15.5:0/1459044955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f596415d860 msgr2=0x7f596406f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:21.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.815+0000 7f596b07a640 1 --2- 172.21.15.5:0/1459044955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f596415d860 0x7f596406f900 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f59580097d0 tx=0x7f595802c3d0 comp rx=0 tx=0).stop 2024-05-20T18:28:21.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.815+0000 7f596b07a640 1 -- 172.21.15.5:0/1459044955 shutdown_connections 2024-05-20T18:28:21.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.815+0000 7f596b07a640 1 --2- 172.21.15.5:0/1459044955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f596415d860 0x7f596406f900 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:21.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.815+0000 7f596b07a640 1 -- 172.21.15.5:0/1459044955 >> 172.21.15.5:0/1459044955 conn(0x7f59640d0e10 msgr2=0x7f59640723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:21.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.815+0000 7f596b07a640 1 -- 172.21.15.5:0/1459044955 shutdown_connections 2024-05-20T18:28:21.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.815+0000 7f596b07a640 1 -- 172.21.15.5:0/1459044955 wait complete. 2024-05-20T18:28:21.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.817+0000 7f596b07a640 1 Processor -- start 2024-05-20T18:28:21.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.817+0000 7f596b07a640 1 -- start start 2024-05-20T18:28:21.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.818+0000 7f596b07a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f596406fe40 0x7f596414cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:21.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.818+0000 7f596b07a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f596409e740 0x7f59640b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:21.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.818+0000 7f596b07a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f596415d860 0x7f596415afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:21.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.818+0000 7f596b07a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f59640ade90 con 0x7f596406fe40 2024-05-20T18:28:21.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.818+0000 7f596b07a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f59640adbf0 con 0x7f596409e740 2024-05-20T18:28:21.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.818+0000 7f596b07a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f59640ae170 con 0x7f596415d860 2024-05-20T18:28:21.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.818+0000 7f5968e16640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f596406fe40 0x7f596414cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:21.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.818+0000 7f5968e16640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f596406fe40 0x7f596414cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37258/0 (socket says 172.21.15.5:37258) 2024-05-20T18:28:21.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.818+0000 7f5968e16640 1 -- 172.21.15.5:0/4264328988 learned_addr learned my addr 172.21.15.5:0/4264328988 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:21.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.819+0000 7f5963fff640 1 --2- 172.21.15.5:0/4264328988 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f596409e740 0x7f59640b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:21.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.819+0000 7f5969617640 1 --2- 172.21.15.5:0/4264328988 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f596415d860 0x7f596415afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:21.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.819+0000 7f5968e16640 1 -- 172.21.15.5:0/4264328988 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f596415d860 msgr2=0x7f596415afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:21.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.819+0000 7f5968e16640 1 --2- 172.21.15.5:0/4264328988 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f596415d860 0x7f596415afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:21.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.819+0000 7f5968e16640 1 -- 172.21.15.5:0/4264328988 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f596409e740 msgr2=0x7f59640b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:21.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.819+0000 7f5968e16640 1 --2- 172.21.15.5:0/4264328988 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f596409e740 0x7f59640b0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:21.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.819+0000 7f5968e16640 1 -- 172.21.15.5:0/4264328988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59640ce930 con 0x7f596406fe40 2024-05-20T18:28:21.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.819+0000 7f5969617640 1 --2- 172.21.15.5:0/4264328988 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f596415d860 0x7f596415afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:21.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.819+0000 7f5963fff640 1 --2- 172.21.15.5:0/4264328988 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f596409e740 0x7f59640b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:21.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.819+0000 7f5968e16640 1 --2- 172.21.15.5:0/4264328988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f596406fe40 0x7f596414cde0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f595400c3f0 tx=0x7f595400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:21.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.819+0000 7f59617fa640 1 -- 172.21.15.5:0/4264328988 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5954017070 con 0x7f596406fe40 2024-05-20T18:28:21.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.819+0000 7f59617fa640 1 -- 172.21.15.5:0/4264328988 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59540052d0 con 0x7f596406fe40 2024-05-20T18:28:21.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.819+0000 7f596b07a640 1 -- 172.21.15.5:0/4264328988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f596414d320 con 0x7f596406fe40 2024-05-20T18:28:21.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.819+0000 7f59617fa640 1 -- 172.21.15.5:0/4264328988 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5954003830 con 0x7f596406fe40 2024-05-20T18:28:21.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.819+0000 7f596b07a640 1 -- 172.21.15.5:0/4264328988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59640cf110 con 0x7f596406fe40 2024-05-20T18:28:21.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.820+0000 7f596b07a640 1 -- 172.21.15.5:0/4264328988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59640690b0 con 0x7f596406fe40 2024-05-20T18:28:21.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.821+0000 7f59617fa640 1 -- 172.21.15.5:0/4264328988 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f595401a070 con 0x7f596406fe40 2024-05-20T18:28:21.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.821+0000 7f59617fa640 1 --2- 172.21.15.5:0/4264328988 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f595006be60 0x7f595006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:21.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.821+0000 7f59617fa640 1 -- 172.21.15.5:0/4264328988 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f5954089fc0 con 0x7f596406fe40 2024-05-20T18:28:21.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.821+0000 7f5963fff640 1 --2- 172.21.15.5:0/4264328988 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f595006be60 0x7f595006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:21.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.822+0000 7f5963fff640 1 --2- 172.21.15.5:0/4264328988 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f595006be60 0x7f595006e370 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5958036000 tx=0x7f5958030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:21.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:21.826+0000 7f59617fa640 1 -- 172.21.15.5:0/4264328988 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5954058aa0 con 0x7f596406fe40 2024-05-20T18:28:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:22.011+0000 7f596b07a640 1 -- 172.21.15.5:0/4264328988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f596406f690 con 0x7f596406fe40 2024-05-20T18:28:22.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:22.871+0000 7f59617fa640 1 -- 172.21.15.5:0/4264328988 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v37) ==== 138+0+0 (secure 0 0 0) 0x7f59540583a0 con 0x7f596406fe40 2024-05-20T18:28:22.871 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-05-20T18:28:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:22.874+0000 7f596b07a640 1 -- 172.21.15.5:0/4264328988 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f595006be60 msgr2=0x7f595006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:22.874+0000 7f596b07a640 1 --2- 172.21.15.5:0/4264328988 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f595006be60 0x7f595006e370 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5958036000 tx=0x7f5958030000 comp rx=0 tx=0).stop 2024-05-20T18:28:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:22.874+0000 7f596b07a640 1 -- 172.21.15.5:0/4264328988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f596406fe40 msgr2=0x7f596414cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:22.874+0000 7f596b07a640 1 --2- 172.21.15.5:0/4264328988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f596406fe40 0x7f596414cde0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f595400c3f0 tx=0x7f595400c8b0 comp rx=0 tx=0).stop 2024-05-20T18:28:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:22.874+0000 7f596b07a640 1 -- 172.21.15.5:0/4264328988 shutdown_connections 2024-05-20T18:28:22.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:22.874+0000 7f596b07a640 1 --2- 172.21.15.5:0/4264328988 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f595006be60 0x7f595006e370 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:22.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:22.874+0000 7f596b07a640 1 --2- 172.21.15.5:0/4264328988 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f596415d860 0x7f596415afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:22.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:22.874+0000 7f596b07a640 1 --2- 172.21.15.5:0/4264328988 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f596409e740 0x7f59640b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:22.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:22.874+0000 7f596b07a640 1 --2- 172.21.15.5:0/4264328988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f596406fe40 0x7f596414cde0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:22.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:22.874+0000 7f596b07a640 1 -- 172.21.15.5:0/4264328988 >> 172.21.15.5:0/4264328988 conn(0x7f59640d0e10 msgr2=0x7f5964156d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:22.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:22.875+0000 7f596b07a640 1 -- 172.21.15.5:0/4264328988 shutdown_connections 2024-05-20T18:28:22.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:22.875+0000 7f596b07a640 1 -- 172.21.15.5:0/4264328988 wait complete. 2024-05-20T18:28:22.892 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-20T18:28:23.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.021+0000 7f4410f21640 1 Processor -- start 2024-05-20T18:28:23.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.021+0000 7f4410f21640 1 -- start start 2024-05-20T18:28:23.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.023+0000 7f4410f21640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f440c08c9c0 0x7f440c15f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:23.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.023+0000 7f4410f21640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f440c0cfb20 0x7f440c0cff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:23.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.023+0000 7f4410f21640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f440c0d05e0 con 0x7f440c0b6a40 2024-05-20T18:28:23.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.023+0000 7f4410f21640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f440c0b91b0 con 0x7f440c0cfb20 2024-05-20T18:28:23.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.023+0000 7f4410f21640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f440c08bc90 con 0x7f440c08c9c0 2024-05-20T18:28:23.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.023+0000 7f4409d9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f440c0cfb20 0x7f440c0cff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:23.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.023+0000 7f4409d9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f440c0cfb20 0x7f440c0cff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37260/0 (socket says 172.21.15.5:37260) 2024-05-20T18:28:23.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.023+0000 7f440ad9d640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f440c0b6a40 0x7f440c0b6e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45784/0 (socket says 172.21.15.5:45784) 2024-05-20T18:28:23.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.023+0000 7f4409d9b640 1 -- 172.21.15.5:0/2573334158 learned_addr learned my addr 172.21.15.5:0/2573334158 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:23.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.023+0000 7f440a59c640 1 --2- 172.21.15.5:0/2573334158 >> v2:172.21.15.63:3300/0 conn(0x7f440c08c9c0 0x7f440c15f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:23.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.024+0000 7f4409d9b640 1 -- 172.21.15.5:0/2573334158 >> v2:172.21.15.63:3300/0 conn(0x7f440c08c9c0 msgr2=0x7f440c15f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:23.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.024+0000 7f4409d9b640 1 --2- 172.21.15.5:0/2573334158 >> v2:172.21.15.63:3300/0 conn(0x7f440c08c9c0 0x7f440c15f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:23.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.024+0000 7f4409d9b640 1 -- 172.21.15.5:0/2573334158 >> v1:172.21.15.5:6789/0 conn(0x7f440c0b6a40 legacy=0x7f440c0b6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:23.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.024+0000 7f4409d9b640 1 -- 172.21.15.5:0/2573334158 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f440c08bee0 con 0x7f440c0cfb20 2024-05-20T18:28:23.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.024+0000 7f440959a640 1 -- 172.21.15.5:0/2573334158 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 668732489 0 0) 0x7f440c0d05e0 con 0x7f440c0b6a40 2024-05-20T18:28:23.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.024+0000 7f4409d9b640 1 --2- 172.21.15.5:0/2573334158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f440c0cfb20 0x7f440c0cff30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f440c0cbc60 tx=0x7f440002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad8e15a7e033352 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:23.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.024+0000 7f440959a640 1 -- 172.21.15.5:0/2573334158 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4400037070 con 0x7f440c0cfb20 2024-05-20T18:28:23.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.025+0000 7f440959a640 1 -- 172.21.15.5:0/2573334158 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4400032440 con 0x7f440c0cfb20 2024-05-20T18:28:23.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.025+0000 7f440959a640 1 -- 172.21.15.5:0/2573334158 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4400032810 con 0x7f440c0cfb20 2024-05-20T18:28:23.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.025+0000 7f4410f21640 1 -- 172.21.15.5:0/2573334158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f440c0cfb20 msgr2=0x7f440c0cff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:23.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.025+0000 7f4410f21640 1 --2- 172.21.15.5:0/2573334158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f440c0cfb20 0x7f440c0cff30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f440c0cbc60 tx=0x7f440002a270 comp rx=0 tx=0).stop 2024-05-20T18:28:23.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.025+0000 7f4410f21640 1 -- 172.21.15.5:0/2573334158 shutdown_connections 2024-05-20T18:28:23.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.025+0000 7f4410f21640 1 --2- 172.21.15.5:0/2573334158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f440c0cfb20 0x7f440c0cff30 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:23.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.025+0000 7f4410f21640 1 --2- 172.21.15.5:0/2573334158 >> v2:172.21.15.63:3300/0 conn(0x7f440c08c9c0 0x7f440c15f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:23.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.025+0000 7f4410f21640 1 -- 172.21.15.5:0/2573334158 >> 172.21.15.5:0/2573334158 conn(0x7f440c0d1a80 msgr2=0x7f440c0d1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:23.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.025+0000 7f4410f21640 1 -- 172.21.15.5:0/2573334158 shutdown_connections 2024-05-20T18:28:23.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.025+0000 7f4410f21640 1 -- 172.21.15.5:0/2573334158 wait complete. 2024-05-20T18:28:23.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.027+0000 7f4410f21640 1 Processor -- start 2024-05-20T18:28:23.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.027+0000 7f4410f21640 1 -- start start 2024-05-20T18:28:23.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.029+0000 7f4410f21640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f440c0b6a40 0x7f440c152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:23.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.029+0000 7f4410f21640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f440c0cfb20 0x7f440c153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:23.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.029+0000 7f4410f21640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f440c155a80 0x7f440c155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:23.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.029+0000 7f4410f21640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f440c0d44b0 con 0x7f440c0cfb20 2024-05-20T18:28:23.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.029+0000 7f4410f21640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f440c08c530 con 0x7f440c155a80 2024-05-20T18:28:23.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.029+0000 7f4410f21640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f440c0d4630 con 0x7f440c0b6a40 2024-05-20T18:28:23.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.029+0000 7f4409d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f440c0cfb20 0x7f440c153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:23.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.029+0000 7f4409d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f440c0cfb20 0x7f440c153d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37274/0 (socket says 172.21.15.5:37274) 2024-05-20T18:28:23.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.029+0000 7f4409d9b640 1 -- 172.21.15.5:0/3376853847 learned_addr learned my addr 172.21.15.5:0/3376853847 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:23.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.029+0000 7f440a59c640 1 --2- 172.21.15.5:0/3376853847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f440c0b6a40 0x7f440c152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:23.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.029+0000 7f440ad9d640 1 --2- 172.21.15.5:0/3376853847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f440c155a80 0x7f440c155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:23.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.029+0000 7f4409d9b640 1 -- 172.21.15.5:0/3376853847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f440c0b6a40 msgr2=0x7f440c152c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:23.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.029+0000 7f4409d9b640 1 --2- 172.21.15.5:0/3376853847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f440c0b6a40 0x7f440c152c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:23.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.029+0000 7f4409d9b640 1 -- 172.21.15.5:0/3376853847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f440c155a80 msgr2=0x7f440c155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:23.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.029+0000 7f4409d9b640 1 --2- 172.21.15.5:0/3376853847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f440c155a80 0x7f440c155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:23.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.029+0000 7f4409d9b640 1 -- 172.21.15.5:0/3376853847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f440c153190 con 0x7f440c0cfb20 2024-05-20T18:28:23.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.029+0000 7f440a59c640 1 --2- 172.21.15.5:0/3376853847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f440c0b6a40 0x7f440c152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:23.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.029+0000 7f4409d9b640 1 --2- 172.21.15.5:0/3376853847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f440c0cfb20 0x7f440c153d60 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4400000c00 tx=0x7f44000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:23.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.030+0000 7f43faffd640 1 -- 172.21.15.5:0/3376853847 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4400041020 con 0x7f440c0cfb20 2024-05-20T18:28:23.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.030+0000 7f43faffd640 1 -- 172.21.15.5:0/3376853847 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4400030640 con 0x7f440c0cfb20 2024-05-20T18:28:23.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.030+0000 7f4410f21640 1 -- 172.21.15.5:0/3376853847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f440c156920 con 0x7f440c0cfb20 2024-05-20T18:28:23.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.030+0000 7f43faffd640 1 -- 172.21.15.5:0/3376853847 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4400037040 con 0x7f440c0cfb20 2024-05-20T18:28:23.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.030+0000 7f4410f21640 1 -- 172.21.15.5:0/3376853847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f440c156ee0 con 0x7f440c0cfb20 2024-05-20T18:28:23.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.031+0000 7f4410f21640 1 -- 172.21.15.5:0/3376853847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f440c0690b0 con 0x7f440c0cfb20 2024-05-20T18:28:23.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.032+0000 7f43faffd640 1 -- 172.21.15.5:0/3376853847 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4400044020 con 0x7f440c0cfb20 2024-05-20T18:28:23.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.032+0000 7f43faffd640 1 --2- 172.21.15.5:0/3376853847 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f43ec06be30 0x7f43ec06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:23.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.032+0000 7f43faffd640 1 -- 172.21.15.5:0/3376853847 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5619+0+0 (secure 0 0 0) 0x7f44000ab980 con 0x7f440c0cfb20 2024-05-20T18:28:23.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.032+0000 7f440a59c640 1 --2- 172.21.15.5:0/3376853847 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f43ec06be30 0x7f43ec06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:23.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.033+0000 7f440a59c640 1 --2- 172.21.15.5:0/3376853847 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f43ec06be30 0x7f43ec06e340 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009af0 tx=0x7f43fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:23.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.036+0000 7f43faffd640 1 -- 172.21.15.5:0/3376853847 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f440004b030 con 0x7f440c0cfb20 2024-05-20T18:28:23.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.221+0000 7f4410f21640 1 -- 172.21.15.5:0/3376853847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f440c1566b0 con 0x7f440c0cfb20 2024-05-20T18:28:23.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.873+0000 7f43faffd640 1 -- 172.21.15.5:0/3376853847 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v38) ==== 170+0+0 (secure 0 0 0) 0x7f440c153190 con 0x7f440c0cfb20 2024-05-20T18:28:23.873 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-05-20T18:28:23.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.876+0000 7f4410f21640 1 -- 172.21.15.5:0/3376853847 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f43ec06be30 msgr2=0x7f43ec06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:23.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.876+0000 7f4410f21640 1 --2- 172.21.15.5:0/3376853847 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f43ec06be30 0x7f43ec06e340 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009af0 tx=0x7f43fc008040 comp rx=0 tx=0).stop 2024-05-20T18:28:23.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.877+0000 7f4410f21640 1 -- 172.21.15.5:0/3376853847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f440c0cfb20 msgr2=0x7f440c153d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:23.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.877+0000 7f4410f21640 1 --2- 172.21.15.5:0/3376853847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f440c0cfb20 0x7f440c153d60 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4400000c00 tx=0x7f44000046c0 comp rx=0 tx=0).stop 2024-05-20T18:28:23.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.877+0000 7f4410f21640 1 -- 172.21.15.5:0/3376853847 shutdown_connections 2024-05-20T18:28:23.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.877+0000 7f4410f21640 1 --2- 172.21.15.5:0/3376853847 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f43ec06be30 0x7f43ec06e340 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:23.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.877+0000 7f4410f21640 1 --2- 172.21.15.5:0/3376853847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f440c155a80 0x7f440c155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:23.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.877+0000 7f4410f21640 1 --2- 172.21.15.5:0/3376853847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f440c0cfb20 0x7f440c153d60 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:23.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.877+0000 7f4410f21640 1 --2- 172.21.15.5:0/3376853847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f440c0b6a40 0x7f440c152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:23.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.877+0000 7f4410f21640 1 -- 172.21.15.5:0/3376853847 >> 172.21.15.5:0/3376853847 conn(0x7f440c0d1a80 msgr2=0x7f440c162240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:23.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.877+0000 7f4410f21640 1 -- 172.21.15.5:0/3376853847 shutdown_connections 2024-05-20T18:28:23.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:23.877+0000 7f4410f21640 1 -- 172.21.15.5:0/3376853847 wait complete. 2024-05-20T18:28:23.907 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-20T18:28:24.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.032+0000 7f4320c1b640 1 Processor -- start 2024-05-20T18:28:24.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.032+0000 7f4320c1b640 1 -- start start 2024-05-20T18:28:24.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.034+0000 7f4320c1b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f431c15d460 0x7f431c06b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:24.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.034+0000 7f4320c1b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f431c0b17f0 0x7f431c0b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:24.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.034+0000 7f4320c1b640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f431c15d870 con 0x7f431c0d1780 2024-05-20T18:28:24.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.034+0000 7f4320c1b640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f431c15dae0 con 0x7f431c0b17f0 2024-05-20T18:28:24.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.034+0000 7f4320c1b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f431c15ddc0 con 0x7f431c15d460 2024-05-20T18:28:24.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.034+0000 7f431ad9d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f431c0b17f0 0x7f431c0b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:24.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.034+0000 7f431ad9d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f431c0b17f0 0x7f431c0b1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:46004/0 (socket says 172.21.15.5:46004) 2024-05-20T18:28:24.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.034+0000 7f431a59c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f431c15d460 0x7f431c06b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:24.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.034+0000 7f431ad9d640 1 -- 172.21.15.5:0/3640994969 learned_addr learned my addr 172.21.15.5:0/3640994969 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:24.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.034+0000 7f4319d9b640 1 --1- 172.21.15.5:0/3640994969 >> v1:172.21.15.63:6789/0 conn(0x7f431c0d1780 0x7f431c0d1b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:49506/0 (socket says 172.21.15.5:49506) 2024-05-20T18:28:24.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.035+0000 7f431a59c640 1 -- 172.21.15.5:0/3640994969 >> v2:172.21.15.63:3300/0 conn(0x7f431c0b17f0 msgr2=0x7f431c0b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:24.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.035+0000 7f431a59c640 1 --2- 172.21.15.5:0/3640994969 >> v2:172.21.15.63:3300/0 conn(0x7f431c0b17f0 0x7f431c0b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:24.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.035+0000 7f431a59c640 1 -- 172.21.15.5:0/3640994969 >> v1:172.21.15.63:6789/0 conn(0x7f431c0d1780 legacy=0x7f431c0d1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:24.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.035+0000 7f431a59c640 1 -- 172.21.15.5:0/3640994969 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431c15e0a0 con 0x7f431c15d460 2024-05-20T18:28:24.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.035+0000 7f431ad9d640 1 --2- 172.21.15.5:0/3640994969 >> v2:172.21.15.63:3300/0 conn(0x7f431c0b17f0 0x7f431c0b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:28:24.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.035+0000 7f431a59c640 1 --2- 172.21.15.5:0/3640994969 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f431c15d460 0x7f431c06b870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f430c00dbc0 tx=0x7f430c02e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b9b87f29188cec1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:24.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.036+0000 7f431959a640 1 -- 172.21.15.5:0/3640994969 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c03a070 con 0x7f431c15d460 2024-05-20T18:28:24.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.036+0000 7f431959a640 1 -- 172.21.15.5:0/3640994969 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f430c00b650 con 0x7f431c15d460 2024-05-20T18:28:24.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.036+0000 7f431959a640 1 -- 172.21.15.5:0/3640994969 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c00ba20 con 0x7f431c15d460 2024-05-20T18:28:24.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.036+0000 7f4320c1b640 1 -- 172.21.15.5:0/3640994969 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f431c15d460 msgr2=0x7f431c06b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:24.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.036+0000 7f4320c1b640 1 --2- 172.21.15.5:0/3640994969 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f431c15d460 0x7f431c06b870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f430c00dbc0 tx=0x7f430c02e4b0 comp rx=0 tx=0).stop 2024-05-20T18:28:24.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.036+0000 7f4320c1b640 1 -- 172.21.15.5:0/3640994969 shutdown_connections 2024-05-20T18:28:24.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.036+0000 7f4320c1b640 1 --2- 172.21.15.5:0/3640994969 >> v2:172.21.15.63:3300/0 conn(0x7f431c0b17f0 0x7f431c0b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:24.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.036+0000 7f4320c1b640 1 --2- 172.21.15.5:0/3640994969 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f431c15d460 0x7f431c06b870 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:24.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.036+0000 7f4320c1b640 1 -- 172.21.15.5:0/3640994969 >> 172.21.15.5:0/3640994969 conn(0x7f431c0b6620 msgr2=0x7f431c0b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:24.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.036+0000 7f4320c1b640 1 -- 172.21.15.5:0/3640994969 shutdown_connections 2024-05-20T18:28:24.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.036+0000 7f4320c1b640 1 -- 172.21.15.5:0/3640994969 wait complete. 2024-05-20T18:28:24.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.038+0000 7f4320c1b640 1 Processor -- start 2024-05-20T18:28:24.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.039+0000 7f4320c1b640 1 -- start start 2024-05-20T18:28:24.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.040+0000 7f4320c1b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f431c0b17f0 0x7f431c1f1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:24.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.040+0000 7f4320c1b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f431c0d1780 0x7f431c13b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:24.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.040+0000 7f4320c1b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c15d460 0x7f431c0d0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:24.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.040+0000 7f4320c1b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f431c15e380 con 0x7f431c0d1780 2024-05-20T18:28:24.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.040+0000 7f4320c1b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f431c06bdb0 con 0x7f431c15d460 2024-05-20T18:28:24.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.040+0000 7f4320c1b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f431c0b2140 con 0x7f431c0b17f0 2024-05-20T18:28:24.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.040+0000 7f4319d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f431c0d1780 0x7f431c13b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:24.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.040+0000 7f4319d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f431c0d1780 0x7f431c13b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37278/0 (socket says 172.21.15.5:37278) 2024-05-20T18:28:24.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.040+0000 7f4319d9b640 1 -- 172.21.15.5:0/2899277891 learned_addr learned my addr 172.21.15.5:0/2899277891 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:24.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.040+0000 7f431a59c640 1 --2- 172.21.15.5:0/2899277891 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f431c0b17f0 0x7f431c1f1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:24.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.040+0000 7f431ad9d640 1 --2- 172.21.15.5:0/2899277891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c15d460 0x7f431c0d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:24.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.040+0000 7f4319d9b640 1 -- 172.21.15.5:0/2899277891 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f431c0b17f0 msgr2=0x7f431c1f1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:24.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.040+0000 7f4319d9b640 1 --2- 172.21.15.5:0/2899277891 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f431c0b17f0 0x7f431c1f1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:24.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.040+0000 7f4319d9b640 1 -- 172.21.15.5:0/2899277891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c15d460 msgr2=0x7f431c0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:24.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.040+0000 7f4319d9b640 1 --2- 172.21.15.5:0/2899277891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c15d460 0x7f431c0d0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:24.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.040+0000 7f4319d9b640 1 -- 172.21.15.5:0/2899277891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431c1f1360 con 0x7f431c0d1780 2024-05-20T18:28:24.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.041+0000 7f4319d9b640 1 --2- 172.21.15.5:0/2899277891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f431c0d1780 0x7f431c13b1e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f431c06e1f0 tx=0x7f431000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:24.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.041+0000 7f430affd640 1 -- 172.21.15.5:0/2899277891 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43100074e0 con 0x7f431c0d1780 2024-05-20T18:28:24.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.041+0000 7f430affd640 1 -- 172.21.15.5:0/2899277891 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43100047e0 con 0x7f431c0d1780 2024-05-20T18:28:24.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.041+0000 7f4320c1b640 1 -- 172.21.15.5:0/2899277891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f431c0d0e10 con 0x7f431c0d1780 2024-05-20T18:28:24.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.041+0000 7f430affd640 1 -- 172.21.15.5:0/2899277891 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4310004bd0 con 0x7f431c0d1780 2024-05-20T18:28:24.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.041+0000 7f4320c1b640 1 -- 172.21.15.5:0/2899277891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f431c1f18e0 con 0x7f431c0d1780 2024-05-20T18:28:24.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.042+0000 7f4320c1b640 1 -- 172.21.15.5:0/2899277891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f431c0d0b80 con 0x7f431c0d1780 2024-05-20T18:28:24.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.043+0000 7f430affd640 1 -- 172.21.15.5:0/2899277891 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f431001a020 con 0x7f431c0d1780 2024-05-20T18:28:24.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.043+0000 7f430affd640 1 --2- 172.21.15.5:0/2899277891 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f42fc06be30 0x7f42fc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:24.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.043+0000 7f430affd640 1 -- 172.21.15.5:0/2899277891 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6001+0+0 (secure 0 0 0) 0x7f431008a4d0 con 0x7f431c0d1780 2024-05-20T18:28:24.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.043+0000 7f431a59c640 1 --2- 172.21.15.5:0/2899277891 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f42fc06be30 0x7f42fc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:24.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.044+0000 7f431a59c640 1 --2- 172.21.15.5:0/2899277891 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f42fc06be30 0x7f42fc06e340 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f430c037000 tx=0x7f430c00d000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:24.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.048+0000 7f430affd640 1 -- 172.21.15.5:0/2899277891 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4310058700 con 0x7f431c0d1780 2024-05-20T18:28:24.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.268+0000 7f4320c1b640 1 -- 172.21.15.5:0/2899277891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f431c075b00 con 0x7f431c0d1780 2024-05-20T18:28:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.905+0000 7f430affd640 1 -- 172.21.15.5:0/2899277891 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7f431005ca00 con 0x7f431c0d1780 2024-05-20T18:28:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 5 and data pool 6 2024-05-20T18:28:24.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.913+0000 7f4320c1b640 1 -- 172.21.15.5:0/2899277891 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f42fc06be30 msgr2=0x7f42fc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:24.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.913+0000 7f4320c1b640 1 --2- 172.21.15.5:0/2899277891 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f42fc06be30 0x7f42fc06e340 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f430c037000 tx=0x7f430c00d000 comp rx=0 tx=0).stop 2024-05-20T18:28:24.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.913+0000 7f4320c1b640 1 -- 172.21.15.5:0/2899277891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f431c0d1780 msgr2=0x7f431c13b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:24.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.913+0000 7f4320c1b640 1 --2- 172.21.15.5:0/2899277891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f431c0d1780 0x7f431c13b1e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f431c06e1f0 tx=0x7f431000c760 comp rx=0 tx=0).stop 2024-05-20T18:28:24.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.913+0000 7f4320c1b640 1 -- 172.21.15.5:0/2899277891 shutdown_connections 2024-05-20T18:28:24.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.913+0000 7f4320c1b640 1 --2- 172.21.15.5:0/2899277891 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f42fc06be30 0x7f42fc06e340 secure :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f430c037000 tx=0x7f430c00d000 comp rx=0 tx=0).stop 2024-05-20T18:28:24.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.913+0000 7f4320c1b640 1 --2- 172.21.15.5:0/2899277891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f431c15d460 0x7f431c0d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:24.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.913+0000 7f4320c1b640 1 --2- 172.21.15.5:0/2899277891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f431c0d1780 0x7f431c13b1e0 secure :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f431c06e1f0 tx=0x7f431000c760 comp rx=0 tx=0).stop 2024-05-20T18:28:24.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.913+0000 7f4320c1b640 1 --2- 172.21.15.5:0/2899277891 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f431c0b17f0 0x7f431c1f1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:24.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.913+0000 7f4320c1b640 1 -- 172.21.15.5:0/2899277891 >> 172.21.15.5:0/2899277891 conn(0x7f431c0b6620 msgr2=0x7f431c13ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:24.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.914+0000 7f4320c1b640 1 -- 172.21.15.5:0/2899277891 shutdown_connections 2024-05-20T18:28:24.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:24.914+0000 7f4320c1b640 1 -- 172.21.15.5:0/2899277891 wait complete. 2024-05-20T18:28:24.937 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:28:25.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.057+0000 7fbf4b47a640 1 Processor -- start 2024-05-20T18:28:25.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.057+0000 7fbf4b47a640 1 -- start start 2024-05-20T18:28:25.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.058+0000 7fbf4b47a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbf4415d530 0x7fbf4415aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:25.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.058+0000 7fbf4b47a640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf440cd580 con 0x7fbf440b1700 2024-05-20T18:28:25.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.058+0000 7fbf4b47a640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf440cd860 con 0x7fbf44070850 2024-05-20T18:28:25.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.058+0000 7fbf4b47a640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fbf440ce190 con 0x7fbf4415d530 2024-05-20T18:28:25.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.058+0000 7fbf49216640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbf4415d530 0x7fbf4415aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:25.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.058+0000 7fbf48a15640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbf440b1700 0x7fbf440b1b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53672/0 (socket says 172.21.15.5:53672) 2024-05-20T18:28:25.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.058+0000 7fbf48a15640 1 -- 172.21.15.5:0/2850402324 learned_addr learned my addr 172.21.15.5:0/2850402324 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:25.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.058+0000 7fbf49216640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbf4415d530 0x7fbf4415aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51522/0 (socket says 172.21.15.5:51522) 2024-05-20T18:28:25.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.059+0000 7fbf49216640 1 -- 172.21.15.5:0/2850402324 >> v1:172.21.15.63:6789/0 conn(0x7fbf44070850 legacy=0x7fbf44070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:25.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.059+0000 7fbf49216640 1 -- 172.21.15.5:0/2850402324 >> v1:172.21.15.5:6789/0 conn(0x7fbf440b1700 legacy=0x7fbf440b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:25.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.059+0000 7fbf49216640 1 -- 172.21.15.5:0/2850402324 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf440cdb40 con 0x7fbf4415d530 2024-05-20T18:28:25.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.059+0000 7fbf43fff640 1 -- 172.21.15.5:0/2850402324 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1576560416 0 0) 0x7fbf440cd580 con 0x7fbf440b1700 2024-05-20T18:28:25.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.059+0000 7fbf49216640 1 --2- 172.21.15.5:0/2850402324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf4415d530 0x7fbf4415aaa0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c0097a0 tx=0x7fbf3c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c3b7e67fae6efa server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:25.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.060+0000 7fbf43fff640 1 -- 172.21.15.5:0/2850402324 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c037070 con 0x7fbf4415d530 2024-05-20T18:28:25.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.060+0000 7fbf43fff640 1 -- 172.21.15.5:0/2850402324 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf3c032440 con 0x7fbf4415d530 2024-05-20T18:28:25.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.060+0000 7fbf43fff640 1 -- 172.21.15.5:0/2850402324 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c032810 con 0x7fbf4415d530 2024-05-20T18:28:25.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.060+0000 7fbf4b47a640 1 -- 172.21.15.5:0/2850402324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf4415d530 msgr2=0x7fbf4415aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:25.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.060+0000 7fbf4b47a640 1 --2- 172.21.15.5:0/2850402324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf4415d530 0x7fbf4415aaa0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c0097a0 tx=0x7fbf3c02a1e0 comp rx=0 tx=0).stop 2024-05-20T18:28:25.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.060+0000 7fbf4b47a640 1 -- 172.21.15.5:0/2850402324 shutdown_connections 2024-05-20T18:28:25.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.060+0000 7fbf4b47a640 1 --2- 172.21.15.5:0/2850402324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf4415d530 0x7fbf4415aaa0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:25.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.060+0000 7fbf4b47a640 1 -- 172.21.15.5:0/2850402324 >> 172.21.15.5:0/2850402324 conn(0x7fbf440d29d0 msgr2=0x7fbf440d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:25.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.061+0000 7fbf4b47a640 1 -- 172.21.15.5:0/2850402324 shutdown_connections 2024-05-20T18:28:25.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.061+0000 7fbf4b47a640 1 -- 172.21.15.5:0/2850402324 wait complete. 2024-05-20T18:28:25.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.062+0000 7fbf4b47a640 1 Processor -- start 2024-05-20T18:28:25.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.062+0000 7fbf4b47a640 1 -- start start 2024-05-20T18:28:25.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.064+0000 7fbf4b47a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf44070850 0x7fbf440b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:25.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.064+0000 7fbf4b47a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf440b1700 0x7fbf440b3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:25.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.064+0000 7fbf4b47a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf4415d530 0x7fbf440c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:25.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.064+0000 7fbf4b47a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbf44071370 con 0x7fbf440b1700 2024-05-20T18:28:25.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.064+0000 7fbf4b47a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbf440cd230 con 0x7fbf44070850 2024-05-20T18:28:25.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.064+0000 7fbf4b47a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbf440714f0 con 0x7fbf4415d530 2024-05-20T18:28:25.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.064+0000 7fbf48a15640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf440b1700 0x7fbf440b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:25.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.064+0000 7fbf48a15640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf440b1700 0x7fbf440b3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51532/0 (socket says 172.21.15.5:51532) 2024-05-20T18:28:25.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.064+0000 7fbf48a15640 1 -- 172.21.15.5:0/1458567319 learned_addr learned my addr 172.21.15.5:0/1458567319 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:25.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.064+0000 7fbf49216640 1 --2- 172.21.15.5:0/1458567319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf44070850 0x7fbf440b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:25.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.064+0000 7fbf49a17640 1 --2- 172.21.15.5:0/1458567319 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf4415d530 0x7fbf440c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:25.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.064+0000 7fbf48a15640 1 -- 172.21.15.5:0/1458567319 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf4415d530 msgr2=0x7fbf440c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:25.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.064+0000 7fbf48a15640 1 --2- 172.21.15.5:0/1458567319 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf4415d530 0x7fbf440c68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:25.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.064+0000 7fbf48a15640 1 -- 172.21.15.5:0/1458567319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf44070850 msgr2=0x7fbf440b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:25.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.064+0000 7fbf48a15640 1 --2- 172.21.15.5:0/1458567319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf44070850 0x7fbf440b31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:25.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.064+0000 7fbf48a15640 1 -- 172.21.15.5:0/1458567319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf440cbf20 con 0x7fbf440b1700 2024-05-20T18:28:25.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.064+0000 7fbf48a15640 1 --2- 172.21.15.5:0/1458567319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf440b1700 0x7fbf440b3700 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbf3800c7c0 tx=0x7fbf3800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:25.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.064+0000 7fbf41ffb640 1 -- 172.21.15.5:0/1458567319 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf38018070 con 0x7fbf440b1700 2024-05-20T18:28:25.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.064+0000 7fbf41ffb640 1 -- 172.21.15.5:0/1458567319 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf3800a640 con 0x7fbf440b1700 2024-05-20T18:28:25.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.065+0000 7fbf4b47a640 1 -- 172.21.15.5:0/1458567319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf440cc200 con 0x7fbf440b1700 2024-05-20T18:28:25.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.065+0000 7fbf41ffb640 1 -- 172.21.15.5:0/1458567319 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3800aa30 con 0x7fbf440b1700 2024-05-20T18:28:25.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.065+0000 7fbf4b47a640 1 -- 172.21.15.5:0/1458567319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf440cc7c0 con 0x7fbf440b1700 2024-05-20T18:28:25.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.066+0000 7fbf4b47a640 1 -- 172.21.15.5:0/1458567319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf44075b00 con 0x7fbf440b1700 2024-05-20T18:28:25.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.066+0000 7fbf41ffb640 1 -- 172.21.15.5:0/1458567319 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbf3801b070 con 0x7fbf440b1700 2024-05-20T18:28:25.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.067+0000 7fbf41ffb640 1 --2- 172.21.15.5:0/1458567319 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf3006be60 0x7fbf3006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:25.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.067+0000 7fbf49216640 1 --2- 172.21.15.5:0/1458567319 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf3006be60 0x7fbf3006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:25.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.067+0000 7fbf41ffb640 1 -- 172.21.15.5:0/1458567319 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fbf3808a610 con 0x7fbf440b1700 2024-05-20T18:28:25.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.067+0000 7fbf49216640 1 --2- 172.21.15.5:0/1458567319 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf3006be60 0x7fbf3006e370 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c034000 tx=0x7fbf3c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:25.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.072+0000 7fbf41ffb640 1 -- 172.21.15.5:0/1458567319 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbf38058740 con 0x7fbf440b1700 2024-05-20T18:28:25.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.257+0000 7fbf4b47a640 1 -- 172.21.15.5:0/1458567319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf440d1340 con 0x7fbf440b1700 2024-05-20T18:28:25.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.258+0000 7fbf41ffb640 1 -- 172.21.15.5:0/1458567319 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+16835 (secure 0 0 0) 0x7fbf3805ca40 con 0x7fbf440b1700 2024-05-20T18:28:25.258 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:25.258 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":39,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:28:24.269944+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:28:22.042995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-20T18:28:23.252728+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:28:25.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.260+0000 7fbf4b47a640 1 -- 172.21.15.5:0/1458567319 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf3006be60 msgr2=0x7fbf3006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:25.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.260+0000 7fbf4b47a640 1 --2- 172.21.15.5:0/1458567319 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf3006be60 0x7fbf3006e370 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c034000 tx=0x7fbf3c02e000 comp rx=0 tx=0).stop 2024-05-20T18:28:25.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.260+0000 7fbf4b47a640 1 -- 172.21.15.5:0/1458567319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf440b1700 msgr2=0x7fbf440b3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:25.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.260+0000 7fbf4b47a640 1 --2- 172.21.15.5:0/1458567319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf440b1700 0x7fbf440b3700 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbf3800c7c0 tx=0x7fbf3800cc80 comp rx=0 tx=0).stop 2024-05-20T18:28:25.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.260+0000 7fbf4b47a640 1 -- 172.21.15.5:0/1458567319 shutdown_connections 2024-05-20T18:28:25.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.260+0000 7fbf4b47a640 1 --2- 172.21.15.5:0/1458567319 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf3006be60 0x7fbf3006e370 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:25.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.260+0000 7fbf4b47a640 1 --2- 172.21.15.5:0/1458567319 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf4415d530 0x7fbf440c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:25.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.260+0000 7fbf4b47a640 1 --2- 172.21.15.5:0/1458567319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf440b1700 0x7fbf440b3700 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:25.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.260+0000 7fbf4b47a640 1 --2- 172.21.15.5:0/1458567319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf44070850 0x7fbf440b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:25.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.260+0000 7fbf4b47a640 1 -- 172.21.15.5:0/1458567319 >> 172.21.15.5:0/1458567319 conn(0x7fbf440d29d0 msgr2=0x7fbf440d06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:25.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.260+0000 7fbf4b47a640 1 -- 172.21.15.5:0/1458567319 shutdown_connections 2024-05-20T18:28:25.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.260+0000 7fbf4b47a640 1 -- 172.21.15.5:0/1458567319 wait complete. 2024-05-20T18:28:25.272 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:28:25.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.387+0000 7fcb8188b640 1 Processor -- start 2024-05-20T18:28:25.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.387+0000 7fcb8188b640 1 -- start start 2024-05-20T18:28:25.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.388+0000 7fcb8188b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcb7c15f220 0x7fcb7c0d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:25.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.388+0000 7fcb8188b640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb7c0cda20 con 0x7fcb7c0cfae0 2024-05-20T18:28:25.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.388+0000 7fcb8188b640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb7c0cdd00 con 0x7fcb7c0d0e00 2024-05-20T18:28:25.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.388+0000 7fcb8188b640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fcb7c0cdfe0 con 0x7fcb7c15f220 2024-05-20T18:28:25.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.389+0000 7fcb7affd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcb7c15f220 0x7fcb7c0d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:25.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.389+0000 7fcb7b7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fcb7c0cfae0 0x7fcb7c0ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53686/0 (socket says 172.21.15.5:53686) 2024-05-20T18:28:25.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.389+0000 7fcb7b7fe640 1 -- 172.21.15.5:0/449719424 learned_addr learned my addr 172.21.15.5:0/449719424 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:25.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.389+0000 7fcb7affd640 1 -- 172.21.15.5:0/449719424 >> v1:172.21.15.63:6789/0 conn(0x7fcb7c0d0e00 legacy=0x7fcb7c0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:25.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.389+0000 7fcb7affd640 1 -- 172.21.15.5:0/449719424 >> v1:172.21.15.5:6789/0 conn(0x7fcb7c0cfae0 legacy=0x7fcb7c0ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:25.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.389+0000 7fcb7affd640 1 -- 172.21.15.5:0/449719424 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb7c0ce2c0 con 0x7fcb7c15f220 2024-05-20T18:28:25.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.389+0000 7fcb7a7fc640 1 -- 172.21.15.5:0/449719424 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1390753319 0 0) 0x7fcb7c0cda20 con 0x7fcb7c0cfae0 2024-05-20T18:28:25.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.390+0000 7fcb7affd640 1 --2- 172.21.15.5:0/449719424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb7c15f220 0x7fcb7c0d08c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c0097a0 tx=0x7fcb6c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706ff2ba499354fd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:25.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.390+0000 7fcb7a7fc640 1 -- 172.21.15.5:0/449719424 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c037070 con 0x7fcb7c15f220 2024-05-20T18:28:25.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.390+0000 7fcb7a7fc640 1 -- 172.21.15.5:0/449719424 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb6c032440 con 0x7fcb7c15f220 2024-05-20T18:28:25.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.390+0000 7fcb7a7fc640 1 -- 172.21.15.5:0/449719424 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c032810 con 0x7fcb7c15f220 2024-05-20T18:28:25.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.391+0000 7fcb8188b640 1 -- 172.21.15.5:0/449719424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb7c15f220 msgr2=0x7fcb7c0d08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:25.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.391+0000 7fcb8188b640 1 --2- 172.21.15.5:0/449719424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb7c15f220 0x7fcb7c0d08c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c0097a0 tx=0x7fcb6c02a1a0 comp rx=0 tx=0).stop 2024-05-20T18:28:25.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.391+0000 7fcb8188b640 1 -- 172.21.15.5:0/449719424 shutdown_connections 2024-05-20T18:28:25.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.391+0000 7fcb8188b640 1 --2- 172.21.15.5:0/449719424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb7c15f220 0x7fcb7c0d08c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:25.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.391+0000 7fcb8188b640 1 -- 172.21.15.5:0/449719424 >> 172.21.15.5:0/449719424 conn(0x7fcb7c0b44d0 msgr2=0x7fcb7c0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:25.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.391+0000 7fcb8188b640 1 -- 172.21.15.5:0/449719424 shutdown_connections 2024-05-20T18:28:25.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.391+0000 7fcb8188b640 1 -- 172.21.15.5:0/449719424 wait complete. 2024-05-20T18:28:25.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.393+0000 7fcb8188b640 1 Processor -- start 2024-05-20T18:28:25.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.393+0000 7fcb8188b640 1 -- start start 2024-05-20T18:28:25.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.394+0000 7fcb8188b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb7c0d0e00 0x7fcb7c0b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.394+0000 7fcb8188b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb7c15f220 0x7fcb7c15dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.394+0000 7fcb8188b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcb7c0b3600 0x7fcb7c14c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.394+0000 7fcb8188b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7c15f7b0 con 0x7fcb7c15f220 2024-05-20T18:28:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.394+0000 7fcb8188b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7c15f630 con 0x7fcb7c0d0e00 2024-05-20T18:28:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.394+0000 7fcb8188b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcb7c0ceef0 con 0x7fcb7c0b3600 2024-05-20T18:28:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.394+0000 7fcb73fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb7c15f220 0x7fcb7c15dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.394+0000 7fcb73fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb7c15f220 0x7fcb7c15dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51554/0 (socket says 172.21.15.5:51554) 2024-05-20T18:28:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.394+0000 7fcb73fff640 1 -- 172.21.15.5:0/3153444156 learned_addr learned my addr 172.21.15.5:0/3153444156 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.394+0000 7fcb7affd640 1 --2- 172.21.15.5:0/3153444156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb7c0d0e00 0x7fcb7c0b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.394+0000 7fcb7b7fe640 1 --2- 172.21.15.5:0/3153444156 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcb7c0b3600 0x7fcb7c14c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.394+0000 7fcb73fff640 1 -- 172.21.15.5:0/3153444156 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcb7c0b3600 msgr2=0x7fcb7c14c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.394+0000 7fcb73fff640 1 --2- 172.21.15.5:0/3153444156 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcb7c0b3600 0x7fcb7c14c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.394+0000 7fcb73fff640 1 -- 172.21.15.5:0/3153444156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb7c0d0e00 msgr2=0x7fcb7c0b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:25.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.394+0000 7fcb73fff640 1 --2- 172.21.15.5:0/3153444156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb7c0d0e00 0x7fcb7c0b30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:25.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.394+0000 7fcb73fff640 1 -- 172.21.15.5:0/3153444156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb7c15f930 con 0x7fcb7c15f220 2024-05-20T18:28:25.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.394+0000 7fcb7affd640 1 --2- 172.21.15.5:0/3153444156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb7c0d0e00 0x7fcb7c0b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:25.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.394+0000 7fcb7b7fe640 1 --2- 172.21.15.5:0/3153444156 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcb7c0b3600 0x7fcb7c14c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:25.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.395+0000 7fcb73fff640 1 --2- 172.21.15.5:0/3153444156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb7c15f220 0x7fcb7c15dd30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400c3c0 tx=0x7fcb6400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:25.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.395+0000 7fcb78ff9640 1 -- 172.21.15.5:0/3153444156 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb64017070 con 0x7fcb7c15f220 2024-05-20T18:28:25.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.395+0000 7fcb78ff9640 1 -- 172.21.15.5:0/3153444156 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb640052d0 con 0x7fcb7c15f220 2024-05-20T18:28:25.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.395+0000 7fcb8188b640 1 -- 172.21.15.5:0/3153444156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb7c15ca40 con 0x7fcb7c15f220 2024-05-20T18:28:25.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.395+0000 7fcb78ff9640 1 -- 172.21.15.5:0/3153444156 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb64003830 con 0x7fcb7c15f220 2024-05-20T18:28:25.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.395+0000 7fcb8188b640 1 -- 172.21.15.5:0/3153444156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb7c15d000 con 0x7fcb7c15f220 2024-05-20T18:28:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.396+0000 7fcb8188b640 1 -- 172.21.15.5:0/3153444156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb7c075b00 con 0x7fcb7c15f220 2024-05-20T18:28:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.396+0000 7fcb78ff9640 1 -- 172.21.15.5:0/3153444156 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcb6401a070 con 0x7fcb7c15f220 2024-05-20T18:28:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.397+0000 7fcb78ff9640 1 --2- 172.21.15.5:0/3153444156 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcb5c06be30 0x7fcb5c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.397+0000 7fcb78ff9640 1 -- 172.21.15.5:0/3153444156 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fcb6408a260 con 0x7fcb7c15f220 2024-05-20T18:28:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.397+0000 7fcb7affd640 1 --2- 172.21.15.5:0/3153444156 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcb5c06be30 0x7fcb5c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.398+0000 7fcb7affd640 1 --2- 172.21.15.5:0/3153444156 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcb5c06be30 0x7fcb5c06e340 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c034000 tx=0x7fcb6c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:25.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.402+0000 7fcb78ff9640 1 -- 172.21.15.5:0/3153444156 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fcb64058a80 con 0x7fcb7c15f220 2024-05-20T18:28:25.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.587+0000 7fcb8188b640 1 -- 172.21.15.5:0/3153444156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb7c162c60 con 0x7fcb7c15f220 2024-05-20T18:28:25.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.588+0000 7fcb78ff9640 1 -- 172.21.15.5:0/3153444156 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+16835 (secure 0 0 0) 0x7fcb64058380 con 0x7fcb7c15f220 2024-05-20T18:28:25.588 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:25.588 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":39,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:28:24.269944+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:28:22.042995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-20T18:28:23.252728+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:28:25.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.590+0000 7fcb8188b640 1 -- 172.21.15.5:0/3153444156 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcb5c06be30 msgr2=0x7fcb5c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:25.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.590+0000 7fcb8188b640 1 --2- 172.21.15.5:0/3153444156 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcb5c06be30 0x7fcb5c06e340 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c034000 tx=0x7fcb6c02e000 comp rx=0 tx=0).stop 2024-05-20T18:28:25.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.590+0000 7fcb8188b640 1 -- 172.21.15.5:0/3153444156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb7c15f220 msgr2=0x7fcb7c15dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:25.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.590+0000 7fcb8188b640 1 --2- 172.21.15.5:0/3153444156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb7c15f220 0x7fcb7c15dd30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400c3c0 tx=0x7fcb6400c880 comp rx=0 tx=0).stop 2024-05-20T18:28:25.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.590+0000 7fcb8188b640 1 -- 172.21.15.5:0/3153444156 shutdown_connections 2024-05-20T18:28:25.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.590+0000 7fcb8188b640 1 --2- 172.21.15.5:0/3153444156 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcb5c06be30 0x7fcb5c06e340 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:25.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.590+0000 7fcb8188b640 1 --2- 172.21.15.5:0/3153444156 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcb7c0b3600 0x7fcb7c14c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:25.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.590+0000 7fcb8188b640 1 --2- 172.21.15.5:0/3153444156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb7c15f220 0x7fcb7c15dd30 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:25.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.590+0000 7fcb8188b640 1 --2- 172.21.15.5:0/3153444156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb7c0d0e00 0x7fcb7c0b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:25.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.590+0000 7fcb8188b640 1 -- 172.21.15.5:0/3153444156 >> 172.21.15.5:0/3153444156 conn(0x7fcb7c0b44d0 msgr2=0x7fcb7c0d2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:25.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.591+0000 7fcb8188b640 1 -- 172.21.15.5:0/3153444156 shutdown_connections 2024-05-20T18:28:25.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.591+0000 7fcb8188b640 1 -- 172.21.15.5:0/3153444156 wait complete. 2024-05-20T18:28:25.602 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-20T18:28:25.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.721+0000 7f57ffbce640 1 Processor -- start 2024-05-20T18:28:25.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.721+0000 7f57ffbce640 1 -- start start 2024-05-20T18:28:25.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.722+0000 7f57ffbce640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f57f8161350 0x7f57f8161760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:25.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.722+0000 7f57ffbce640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f57f80d1830 0x7f57f80d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:25.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.722+0000 7f57ffbce640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57f8160b90 con 0x7f57f806b520 2024-05-20T18:28:25.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.722+0000 7f57ffbce640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f57f80d2180 con 0x7f57f80d1830 2024-05-20T18:28:25.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.722+0000 7f57ffbce640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f57f815de60 con 0x7f57f8161350 2024-05-20T18:28:25.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.722+0000 7f57fd96a640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f57f806b520 0x7f57f815d540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53694/0 (socket says 172.21.15.5:53694) 2024-05-20T18:28:25.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.722+0000 7f57fe16b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f57f80d1830 0x7f57f80d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:25.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.722+0000 7f57fd96a640 1 -- 172.21.15.5:0/4269200556 learned_addr learned my addr 172.21.15.5:0/4269200556 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:25.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.723+0000 7f57fd169640 1 --2- 172.21.15.5:0/4269200556 >> v2:172.21.15.63:3300/0 conn(0x7f57f8161350 0x7f57f8161760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:25.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.723+0000 7f57fc968640 1 -- 172.21.15.5:0/4269200556 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3026829563 0 0) 0x7f57f8160b90 con 0x7f57f806b520 2024-05-20T18:28:25.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.723+0000 7f57fc968640 1 -- 172.21.15.5:0/4269200556 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57e4003520 con 0x7f57f806b520 2024-05-20T18:28:25.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.723+0000 7f57fe16b640 1 -- 172.21.15.5:0/4269200556 >> v2:172.21.15.63:3300/0 conn(0x7f57f8161350 msgr2=0x7f57f8161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:25.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.723+0000 7f57fe16b640 1 --2- 172.21.15.5:0/4269200556 >> v2:172.21.15.63:3300/0 conn(0x7f57f8161350 0x7f57f8161760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:25.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.723+0000 7f57fe16b640 1 -- 172.21.15.5:0/4269200556 >> v1:172.21.15.5:6789/0 conn(0x7f57f806b520 legacy=0x7f57f815d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:25.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.723+0000 7f57fe16b640 1 -- 172.21.15.5:0/4269200556 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f815e140 con 0x7f57f80d1830 2024-05-20T18:28:25.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.724+0000 7f57fe16b640 1 --2- 172.21.15.5:0/4269200556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f80d1830 0x7f57f80d1c40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f57f80b7510 tx=0x7f57f002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f003e1215ff556 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:25.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.724+0000 7f57fc968640 1 -- 172.21.15.5:0/4269200556 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0007740 con 0x7f57f80d1830 2024-05-20T18:28:25.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.724+0000 7f57fc968640 1 -- 172.21.15.5:0/4269200556 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f00078e0 con 0x7f57f80d1830 2024-05-20T18:28:25.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.724+0000 7f57fc968640 1 -- 172.21.15.5:0/4269200556 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0007cb0 con 0x7f57f80d1830 2024-05-20T18:28:25.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.724+0000 7f57ffbce640 1 -- 172.21.15.5:0/4269200556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f80d1830 msgr2=0x7f57f80d1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:25.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.724+0000 7f57ffbce640 1 --2- 172.21.15.5:0/4269200556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f80d1830 0x7f57f80d1c40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f57f80b7510 tx=0x7f57f002a680 comp rx=0 tx=0).stop 2024-05-20T18:28:25.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.725+0000 7f57ffbce640 1 -- 172.21.15.5:0/4269200556 shutdown_connections 2024-05-20T18:28:25.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.725+0000 7f57ffbce640 1 --2- 172.21.15.5:0/4269200556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f80d1830 0x7f57f80d1c40 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:25.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.725+0000 7f57ffbce640 1 --2- 172.21.15.5:0/4269200556 >> v2:172.21.15.63:3300/0 conn(0x7f57f8161350 0x7f57f8161760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:25.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.725+0000 7f57ffbce640 1 -- 172.21.15.5:0/4269200556 >> 172.21.15.5:0/4269200556 conn(0x7f57f80b6670 msgr2=0x7f57f80b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:25.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.725+0000 7f57ffbce640 1 -- 172.21.15.5:0/4269200556 shutdown_connections 2024-05-20T18:28:25.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.725+0000 7f57ffbce640 1 -- 172.21.15.5:0/4269200556 wait complete. 2024-05-20T18:28:25.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.726+0000 7f57ffbce640 1 Processor -- start 2024-05-20T18:28:25.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.727+0000 7f57ffbce640 1 -- start start 2024-05-20T18:28:25.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.728+0000 7f57ffbce640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57f806b520 0x7f57f813edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:25.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.728+0000 7f57ffbce640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f57f80d1830 0x7f57f8142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:25.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.728+0000 7f57ffbce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f8161350 0x7f57f80cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:25.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.728+0000 7f57ffbce640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f57f8143740 con 0x7f57f8161350 2024-05-20T18:28:25.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.728+0000 7f57ffbce640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f57f8160e30 con 0x7f57f806b520 2024-05-20T18:28:25.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.728+0000 7f57ffbce640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f57f815e970 con 0x7f57f80d1830 2024-05-20T18:28:25.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.728+0000 7f57fe16b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f8161350 0x7f57f80cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:25.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.728+0000 7f57fe16b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f8161350 0x7f57f80cfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51580/0 (socket says 172.21.15.5:51580) 2024-05-20T18:28:25.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.728+0000 7f57fe16b640 1 -- 172.21.15.5:0/3834138761 learned_addr learned my addr 172.21.15.5:0/3834138761 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:25.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.728+0000 7f57fd169640 1 --2- 172.21.15.5:0/3834138761 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f57f80d1830 0x7f57f8142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:25.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.728+0000 7f57fd96a640 1 --2- 172.21.15.5:0/3834138761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57f806b520 0x7f57f813edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:25.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.728+0000 7f57fe16b640 1 -- 172.21.15.5:0/3834138761 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f57f80d1830 msgr2=0x7f57f8142050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:25.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.728+0000 7f57fe16b640 1 --2- 172.21.15.5:0/3834138761 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f57f80d1830 0x7f57f8142050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:25.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.728+0000 7f57fe16b640 1 -- 172.21.15.5:0/3834138761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57f806b520 msgr2=0x7f57f813edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:25.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.728+0000 7f57fe16b640 1 --2- 172.21.15.5:0/3834138761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57f806b520 0x7f57f813edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:25.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.728+0000 7f57fe16b640 1 -- 172.21.15.5:0/3834138761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f8154160 con 0x7f57f8161350 2024-05-20T18:28:25.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.729+0000 7f57fe16b640 1 --2- 172.21.15.5:0/3834138761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f8161350 0x7f57f80cfdb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f57f002ac70 tx=0x7f57f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:25.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.729+0000 7f57ee7fc640 1 -- 172.21.15.5:0/3834138761 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f003f070 con 0x7f57f8161350 2024-05-20T18:28:25.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.729+0000 7f57ee7fc640 1 -- 172.21.15.5:0/3834138761 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f0031bc0 con 0x7f57f8161350 2024-05-20T18:28:25.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.729+0000 7f57ffbce640 1 -- 172.21.15.5:0/3834138761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57f8154440 con 0x7f57f8161350 2024-05-20T18:28:25.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.729+0000 7f57ee7fc640 1 -- 172.21.15.5:0/3834138761 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0009450 con 0x7f57f8161350 2024-05-20T18:28:25.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.729+0000 7f57ffbce640 1 -- 172.21.15.5:0/3834138761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57f8154a00 con 0x7f57f8161350 2024-05-20T18:28:25.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.730+0000 7f57ffbce640 1 -- 172.21.15.5:0/3834138761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57f8075b00 con 0x7f57f8161350 2024-05-20T18:28:25.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.731+0000 7f57ee7fc640 1 -- 172.21.15.5:0/3834138761 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f57f003b410 con 0x7f57f8161350 2024-05-20T18:28:25.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.731+0000 7f57ee7fc640 1 --2- 172.21.15.5:0/3834138761 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f57dc06be60 0x7f57dc06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:25.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.731+0000 7f57ee7fc640 1 -- 172.21.15.5:0/3834138761 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f57f00ac590 con 0x7f57f8161350 2024-05-20T18:28:25.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.731+0000 7f57fd96a640 1 --2- 172.21.15.5:0/3834138761 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f57dc06be60 0x7f57dc06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:25.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.731+0000 7f57fd96a640 1 --2- 172.21.15.5:0/3834138761 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f57dc06be60 0x7f57dc06e370 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f57f4001200 tx=0x7f57f4000bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:25.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.736+0000 7f57ee7fc640 1 -- 172.21.15.5:0/3834138761 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f57f007a750 con 0x7f57f8161350 2024-05-20T18:28:25.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:25.960+0000 7f57ffbce640 1 -- 172.21.15.5:0/3834138761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f57f80b1090 con 0x7f57f8161350 2024-05-20T18:28:26.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:26.914+0000 7f57ee7fc640 1 -- 172.21.15.5:0/3834138761 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f57f007ea50 con 0x7f57f8161350 2024-05-20T18:28:26.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:26.917+0000 7f57ffbce640 1 -- 172.21.15.5:0/3834138761 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f57dc06be60 msgr2=0x7f57dc06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:26.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:26.917+0000 7f57ffbce640 1 --2- 172.21.15.5:0/3834138761 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f57dc06be60 0x7f57dc06e370 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f57f4001200 tx=0x7f57f4000bc0 comp rx=0 tx=0).stop 2024-05-20T18:28:26.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:26.917+0000 7f57ffbce640 1 -- 172.21.15.5:0/3834138761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f8161350 msgr2=0x7f57f80cfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:26.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:26.917+0000 7f57ffbce640 1 --2- 172.21.15.5:0/3834138761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f8161350 0x7f57f80cfdb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f57f002ac70 tx=0x7f57f0004a40 comp rx=0 tx=0).stop 2024-05-20T18:28:26.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:26.917+0000 7f57ffbce640 1 -- 172.21.15.5:0/3834138761 shutdown_connections 2024-05-20T18:28:26.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:26.917+0000 7f57ffbce640 1 --2- 172.21.15.5:0/3834138761 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f57dc06be60 0x7f57dc06e370 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:26.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:26.917+0000 7f57ffbce640 1 --2- 172.21.15.5:0/3834138761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f8161350 0x7f57f80cfdb0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:26.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:26.917+0000 7f57ffbce640 1 --2- 172.21.15.5:0/3834138761 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f57f80d1830 0x7f57f8142050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:26.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:26.917+0000 7f57ffbce640 1 --2- 172.21.15.5:0/3834138761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57f806b520 0x7f57f813edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:26.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:26.917+0000 7f57ffbce640 1 -- 172.21.15.5:0/3834138761 >> 172.21.15.5:0/3834138761 conn(0x7f57f80b6670 msgr2=0x7f57f8158f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:26.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:26.917+0000 7f57ffbce640 1 -- 172.21.15.5:0/3834138761 shutdown_connections 2024-05-20T18:28:26.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:26.917+0000 7f57ffbce640 1 -- 172.21.15.5:0/3834138761 wait complete. 2024-05-20T18:28:26.931 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-20T18:28:26.931 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-20T18:28:27.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.057+0000 7f0eb1432640 1 Processor -- start 2024-05-20T18:28:27.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.057+0000 7f0eb1432640 1 -- start start 2024-05-20T18:28:27.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.058+0000 7f0eb1432640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0eac163a10 0x7f0eac0d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:27.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.058+0000 7f0eb1432640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0eac0d0370 con 0x7f0eac0b4010 2024-05-20T18:28:27.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.058+0000 7f0eb1432640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0eac0d0650 con 0x7f0eac0d14b0 2024-05-20T18:28:27.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.058+0000 7f0eb1432640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0eac0d3ca0 con 0x7f0eac163a10 2024-05-20T18:28:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.058+0000 7f0ea3fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0eac0b4010 0x7f0eac0cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53696/0 (socket says 172.21.15.5:53696) 2024-05-20T18:28:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.058+0000 7f0ea3fff640 1 -- 172.21.15.5:0/1700174079 learned_addr learned my addr 172.21.15.5:0/1700174079 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.059+0000 7f0eaaffd640 1 --2- 172.21.15.5:0/1700174079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0eac163a10 0x7f0eac0d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.059+0000 7f0eaa7fc640 1 -- 172.21.15.5:0/1700174079 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2689043030 0 0) 0x7f0eac0d0370 con 0x7f0eac0b4010 2024-05-20T18:28:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.059+0000 7f0eaa7fc640 1 -- 172.21.15.5:0/1700174079 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e98003520 con 0x7f0eac0b4010 2024-05-20T18:28:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.059+0000 7f0eaa7fc640 1 -- 172.21.15.5:0/1700174079 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1452479270 0 0) 0x7f0eac0d0650 con 0x7f0eac0d14b0 2024-05-20T18:28:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.059+0000 7f0eaa7fc640 1 -- 172.21.15.5:0/1700174079 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0eac0d0370 con 0x7f0eac0d14b0 2024-05-20T18:28:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.059+0000 7f0eaaffd640 1 -- 172.21.15.5:0/1700174079 >> v1:172.21.15.63:6789/0 conn(0x7f0eac0d14b0 legacy=0x7f0eac0d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.059+0000 7f0eaaffd640 1 -- 172.21.15.5:0/1700174079 >> v1:172.21.15.5:6789/0 conn(0x7f0eac0b4010 legacy=0x7f0eac0cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.059+0000 7f0eaaffd640 1 -- 172.21.15.5:0/1700174079 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eac0d3650 con 0x7f0eac163a10 2024-05-20T18:28:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.060+0000 7f0eaaffd640 1 --2- 172.21.15.5:0/1700174079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0eac163a10 0x7f0eac0d4130 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0097d0 tx=0x7f0e9c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89b83e7d973ea196 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.060+0000 7f0eaa7fc640 1 -- 172.21.15.5:0/1700174079 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c037070 con 0x7f0eac163a10 2024-05-20T18:28:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.060+0000 7f0eaa7fc640 1 -- 172.21.15.5:0/1700174079 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e9c032440 con 0x7f0eac163a10 2024-05-20T18:28:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.060+0000 7f0eaa7fc640 1 -- 172.21.15.5:0/1700174079 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c032810 con 0x7f0eac163a10 2024-05-20T18:28:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.061+0000 7f0eb1432640 1 -- 172.21.15.5:0/1700174079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0eac163a10 msgr2=0x7f0eac0d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.061+0000 7f0eb1432640 1 --2- 172.21.15.5:0/1700174079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0eac163a10 0x7f0eac0d4130 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0097d0 tx=0x7f0e9c02a210 comp rx=0 tx=0).stop 2024-05-20T18:28:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.061+0000 7f0eb1432640 1 -- 172.21.15.5:0/1700174079 shutdown_connections 2024-05-20T18:28:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.061+0000 7f0eb1432640 1 --2- 172.21.15.5:0/1700174079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0eac163a10 0x7f0eac0d4130 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.061+0000 7f0eb1432640 1 -- 172.21.15.5:0/1700174079 >> 172.21.15.5:0/1700174079 conn(0x7f0eac06d240 msgr2=0x7f0eac06d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.061+0000 7f0eb1432640 1 -- 172.21.15.5:0/1700174079 shutdown_connections 2024-05-20T18:28:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.061+0000 7f0eb1432640 1 -- 172.21.15.5:0/1700174079 wait complete. 2024-05-20T18:28:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.063+0000 7f0eb1432640 1 Processor -- start 2024-05-20T18:28:27.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.063+0000 7f0eb1432640 1 -- start start 2024-05-20T18:28:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.064+0000 7f0eb1432640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0eac0b4010 0x7f0eac15a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.064+0000 7f0eb1432640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0eac0d14b0 0x7f0eac1f5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.064+0000 7f0eb1432640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0eac163a10 0x7f0eac154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.064+0000 7f0eb1432640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac0b8390 con 0x7f0eac163a10 2024-05-20T18:28:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.064+0000 7f0eb1432640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac0b8210 con 0x7f0eac0b4010 2024-05-20T18:28:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.064+0000 7f0eb1432640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0eac0b8510 con 0x7f0eac0d14b0 2024-05-20T18:28:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.064+0000 7f0eab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0eac163a10 0x7f0eac154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.064+0000 7f0eab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0eac163a10 0x7f0eac154d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51588/0 (socket says 172.21.15.5:51588) 2024-05-20T18:28:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.064+0000 7f0eab7fe640 1 -- 172.21.15.5:0/369290672 learned_addr learned my addr 172.21.15.5:0/369290672 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.064+0000 7f0eaaffd640 1 --2- 172.21.15.5:0/369290672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0eac0b4010 0x7f0eac15a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.064+0000 7f0ea3fff640 1 --2- 172.21.15.5:0/369290672 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0eac0d14b0 0x7f0eac1f5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.064+0000 7f0eab7fe640 1 -- 172.21.15.5:0/369290672 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0eac0d14b0 msgr2=0x7f0eac1f5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.064+0000 7f0eab7fe640 1 --2- 172.21.15.5:0/369290672 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0eac0d14b0 0x7f0eac1f5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.064+0000 7f0eab7fe640 1 -- 172.21.15.5:0/369290672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0eac0b4010 msgr2=0x7f0eac15a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.064+0000 7f0eab7fe640 1 --2- 172.21.15.5:0/369290672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0eac0b4010 0x7f0eac15a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.064+0000 7f0eab7fe640 1 -- 172.21.15.5:0/369290672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eac144ee0 con 0x7f0eac163a10 2024-05-20T18:28:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.065+0000 7f0eab7fe640 1 --2- 172.21.15.5:0/369290672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0eac163a10 0x7f0eac154d60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400c6e0 tx=0x7f0ea400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.065+0000 7f0ea37fe640 1 -- 172.21.15.5:0/369290672 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4003b70 con 0x7f0eac163a10 2024-05-20T18:28:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.065+0000 7f0ea37fe640 1 -- 172.21.15.5:0/369290672 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ea4003d10 con 0x7f0eac163a10 2024-05-20T18:28:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.065+0000 7f0eb1432640 1 -- 172.21.15.5:0/369290672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eac1451c0 con 0x7f0eac163a10 2024-05-20T18:28:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.065+0000 7f0ea37fe640 1 -- 172.21.15.5:0/369290672 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea40052d0 con 0x7f0eac163a10 2024-05-20T18:28:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.065+0000 7f0eb1432640 1 -- 172.21.15.5:0/369290672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eac145780 con 0x7f0eac163a10 2024-05-20T18:28:27.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.066+0000 7f0eb1432640 1 -- 172.21.15.5:0/369290672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eac144c50 con 0x7f0eac163a10 2024-05-20T18:28:27.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.066+0000 7f0ea37fe640 1 -- 172.21.15.5:0/369290672 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0ea4005470 con 0x7f0eac163a10 2024-05-20T18:28:27.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.067+0000 7f0ea37fe640 1 --2- 172.21.15.5:0/369290672 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0e8c06be60 0x7f0e8c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:27.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.067+0000 7f0eaaffd640 1 --2- 172.21.15.5:0/369290672 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0e8c06be60 0x7f0e8c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:27.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.067+0000 7f0ea37fe640 1 -- 172.21.15.5:0/369290672 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f0ea408ad70 con 0x7f0eac163a10 2024-05-20T18:28:27.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.067+0000 7f0eaaffd640 1 --2- 172.21.15.5:0/369290672 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0e8c06be60 0x7f0e8c06e370 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c034000 tx=0x7f0e9c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:27.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.072+0000 7f0ea37fe640 1 -- 172.21.15.5:0/369290672 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0ea4058f40 con 0x7f0eac163a10 2024-05-20T18:28:27.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.295+0000 7f0eb1432640 1 -- 172.21.15.5:0/369290672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f0eac06db80 con 0x7f0eac163a10 2024-05-20T18:28:27.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.916+0000 7f0ea37fe640 1 -- 172.21.15.5:0/369290672 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7f0eac06db80 con 0x7f0eac163a10 2024-05-20T18:28:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.918+0000 7f0eb1432640 1 -- 172.21.15.5:0/369290672 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0e8c06be60 msgr2=0x7f0e8c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.918+0000 7f0eb1432640 1 --2- 172.21.15.5:0/369290672 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0e8c06be60 0x7f0e8c06e370 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c034000 tx=0x7f0e9c02e000 comp rx=0 tx=0).stop 2024-05-20T18:28:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.918+0000 7f0eb1432640 1 -- 172.21.15.5:0/369290672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0eac163a10 msgr2=0x7f0eac154d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:27.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.918+0000 7f0eb1432640 1 --2- 172.21.15.5:0/369290672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0eac163a10 0x7f0eac154d60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400c6e0 tx=0x7f0ea400cba0 comp rx=0 tx=0).stop 2024-05-20T18:28:27.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.918+0000 7f0eb1432640 1 -- 172.21.15.5:0/369290672 shutdown_connections 2024-05-20T18:28:27.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.918+0000 7f0eb1432640 1 --2- 172.21.15.5:0/369290672 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0e8c06be60 0x7f0e8c06e370 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:27.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.918+0000 7f0eb1432640 1 --2- 172.21.15.5:0/369290672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0eac163a10 0x7f0eac154d60 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:27.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.918+0000 7f0eb1432640 1 --2- 172.21.15.5:0/369290672 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0eac0d14b0 0x7f0eac1f5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:27.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.918+0000 7f0eb1432640 1 --2- 172.21.15.5:0/369290672 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0eac0b4010 0x7f0eac15a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:27.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.918+0000 7f0eb1432640 1 -- 172.21.15.5:0/369290672 >> 172.21.15.5:0/369290672 conn(0x7f0eac06d240 msgr2=0x7f0eac15f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:27.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.919+0000 7f0eb1432640 1 -- 172.21.15.5:0/369290672 shutdown_connections 2024-05-20T18:28:27.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:27.919+0000 7f0eb1432640 1 -- 172.21.15.5:0/369290672 wait complete. 2024-05-20T18:28:27.932 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-20T18:28:27.932 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:28:28.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.055+0000 7f433c57e640 1 Processor -- start 2024-05-20T18:28:28.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.055+0000 7f433c57e640 1 -- start start 2024-05-20T18:28:28.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.056+0000 7f433c57e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f4334163a10 0x7f43340d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:28.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.056+0000 7f433c57e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43340b4010 0x7f43340cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:28.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.056+0000 7f433c57e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f43340d14b0 0x7f43340d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:28.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.056+0000 7f433c57e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f43340d3ca0 con 0x7f43340d14b0 2024-05-20T18:28:28.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.056+0000 7f433c57e640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f43340d0370 con 0x7f4334163a10 2024-05-20T18:28:28.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.056+0000 7f433c57e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f43340d0650 con 0x7f43340b4010 2024-05-20T18:28:28.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.056+0000 7f433ab1b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f43340d14b0 0x7f43340d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:28.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.057+0000 7f433ab1b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f43340d14b0 0x7f43340d18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51592/0 (socket says 172.21.15.5:51592) 2024-05-20T18:28:28.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.057+0000 7f433ab1b640 1 -- 172.21.15.5:0/2829994354 learned_addr learned my addr 172.21.15.5:0/2829994354 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:28.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.057+0000 7f4339b19640 1 --2- 172.21.15.5:0/2829994354 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43340b4010 0x7f43340cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:28.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.057+0000 7f433a31a640 1 --2- 172.21.15.5:0/2829994354 >> v2:172.21.15.63:3300/0 conn(0x7f4334163a10 0x7f43340d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:28.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.057+0000 7f433ab1b640 1 -- 172.21.15.5:0/2829994354 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43340b4010 msgr2=0x7f43340cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:28.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.057+0000 7f433ab1b640 1 --2- 172.21.15.5:0/2829994354 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43340b4010 0x7f43340cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:28.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.057+0000 7f433ab1b640 1 -- 172.21.15.5:0/2829994354 >> v2:172.21.15.63:3300/0 conn(0x7f4334163a10 msgr2=0x7f43340d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:28.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.057+0000 7f433ab1b640 1 --2- 172.21.15.5:0/2829994354 >> v2:172.21.15.63:3300/0 conn(0x7f4334163a10 0x7f43340d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:28.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.057+0000 7f433ab1b640 1 -- 172.21.15.5:0/2829994354 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43340d3650 con 0x7f43340d14b0 2024-05-20T18:28:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.058+0000 7f433ab1b640 1 --2- 172.21.15.5:0/2829994354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43340d14b0 0x7f43340d18c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f43340cf1c0 tx=0x7f432c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4739c71f279d52c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.058+0000 7f4339318640 1 -- 172.21.15.5:0/2829994354 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c036070 con 0x7f43340d14b0 2024-05-20T18:28:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.058+0000 7f4339318640 1 -- 172.21.15.5:0/2829994354 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f432c031600 con 0x7f43340d14b0 2024-05-20T18:28:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.058+0000 7f4339318640 1 -- 172.21.15.5:0/2829994354 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c0319d0 con 0x7f43340d14b0 2024-05-20T18:28:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.059+0000 7f433c57e640 1 -- 172.21.15.5:0/2829994354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43340d14b0 msgr2=0x7f43340d18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.059+0000 7f433c57e640 1 --2- 172.21.15.5:0/2829994354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43340d14b0 0x7f43340d18c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f43340cf1c0 tx=0x7f432c02a6a0 comp rx=0 tx=0).stop 2024-05-20T18:28:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.059+0000 7f433c57e640 1 -- 172.21.15.5:0/2829994354 shutdown_connections 2024-05-20T18:28:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.059+0000 7f433c57e640 1 --2- 172.21.15.5:0/2829994354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43340d14b0 0x7f43340d18c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.059+0000 7f433c57e640 1 --2- 172.21.15.5:0/2829994354 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43340b4010 0x7f43340cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.059+0000 7f433c57e640 1 --2- 172.21.15.5:0/2829994354 >> v2:172.21.15.63:3300/0 conn(0x7f4334163a10 0x7f43340d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.059+0000 7f433c57e640 1 -- 172.21.15.5:0/2829994354 >> 172.21.15.5:0/2829994354 conn(0x7f433406d240 msgr2=0x7f433406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.059+0000 7f433c57e640 1 -- 172.21.15.5:0/2829994354 shutdown_connections 2024-05-20T18:28:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.059+0000 7f433c57e640 1 -- 172.21.15.5:0/2829994354 wait complete. 2024-05-20T18:28:28.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.061+0000 7f433c57e640 1 Processor -- start 2024-05-20T18:28:28.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.061+0000 7f433c57e640 1 -- start start 2024-05-20T18:28:28.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.062+0000 7f433c57e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43340b4010 0x7f43340d4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.062+0000 7f433c57e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43340d14b0 0x7f433414f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.062+0000 7f433c57e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4334163a10 0x7f43340ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.062+0000 7f433c57e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f43340d1e00 con 0x7f43340b4010 2024-05-20T18:28:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.062+0000 7f433c57e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f43340d0040 con 0x7f43340d14b0 2024-05-20T18:28:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.062+0000 7f433c57e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f43340b8210 con 0x7f4334163a10 2024-05-20T18:28:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.062+0000 7f433a31a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43340b4010 0x7f43340d4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.062+0000 7f433a31a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43340b4010 0x7f43340d4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51604/0 (socket says 172.21.15.5:51604) 2024-05-20T18:28:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.062+0000 7f433a31a640 1 -- 172.21.15.5:0/2476229635 learned_addr learned my addr 172.21.15.5:0/2476229635 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.062+0000 7f433ab1b640 1 --2- 172.21.15.5:0/2476229635 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4334163a10 0x7f43340ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.062+0000 7f4339b19640 1 --2- 172.21.15.5:0/2476229635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43340d14b0 0x7f433414f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.062+0000 7f433a31a640 1 -- 172.21.15.5:0/2476229635 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4334163a10 msgr2=0x7f43340ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.062+0000 7f433a31a640 1 --2- 172.21.15.5:0/2476229635 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4334163a10 0x7f43340ce0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.062+0000 7f433a31a640 1 -- 172.21.15.5:0/2476229635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43340d14b0 msgr2=0x7f433414f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.062+0000 7f433a31a640 1 --2- 172.21.15.5:0/2476229635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43340d14b0 0x7f433414f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.062+0000 7f433a31a640 1 -- 172.21.15.5:0/2476229635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4334150040 con 0x7f43340b4010 2024-05-20T18:28:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.063+0000 7f433a31a640 1 --2- 172.21.15.5:0/2476229635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43340b4010 0x7f43340d4d50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f433000e840 tx=0x7f433000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.063+0000 7f432affd640 1 -- 172.21.15.5:0/2476229635 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330017070 con 0x7f43340b4010 2024-05-20T18:28:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.063+0000 7f432affd640 1 -- 172.21.15.5:0/2476229635 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f433000c940 con 0x7f43340b4010 2024-05-20T18:28:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.063+0000 7f433c57e640 1 -- 172.21.15.5:0/2476229635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f433414d490 con 0x7f43340b4010 2024-05-20T18:28:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.063+0000 7f432affd640 1 -- 172.21.15.5:0/2476229635 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433000cd10 con 0x7f43340b4010 2024-05-20T18:28:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.063+0000 7f433c57e640 1 -- 172.21.15.5:0/2476229635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f433414eff0 con 0x7f43340b4010 2024-05-20T18:28:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.064+0000 7f433c57e640 1 -- 172.21.15.5:0/2476229635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4334075b00 con 0x7f43340b4010 2024-05-20T18:28:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.065+0000 7f432affd640 1 -- 172.21.15.5:0/2476229635 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f433001a070 con 0x7f43340b4010 2024-05-20T18:28:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.065+0000 7f432affd640 1 --2- 172.21.15.5:0/2476229635 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f431806be30 0x7f431806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.065+0000 7f432affd640 1 -- 172.21.15.5:0/2476229635 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f433008b160 con 0x7f43340b4010 2024-05-20T18:28:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.065+0000 7f4339b19640 1 --2- 172.21.15.5:0/2476229635 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f431806be30 0x7f431806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.066+0000 7f4339b19640 1 --2- 172.21.15.5:0/2476229635 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f431806be30 0x7f431806e340 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f43341604e0 tx=0x7f4324008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.070+0000 7f432affd640 1 -- 172.21.15.5:0/2476229635 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4330059a10 con 0x7f43340b4010 2024-05-20T18:28:28.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.290+0000 7f433c57e640 1 -- 172.21.15.5:0/2476229635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43340d3050 con 0x7f43340b4010 2024-05-20T18:28:28.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.291+0000 7f432affd640 1 -- 172.21.15.5:0/2476229635 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5175 (secure 0 0 0) 0x7f4330059310 con 0x7f43340b4010 2024-05-20T18:28:28.291 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:28.291 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1458170078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1458170078},{"type":"v1","addr":"172.21.15.63:6834","nonce":1458170078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4916,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/2601750663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":2601750663},{"type":"v1","addr":"172.21.15.63:6835","nonce":2601750663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5313,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/1913030219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1913030219},{"type":"v1","addr":"172.21.15.5:6836","nonce":1913030219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:28:24.270282+0000","modified":"2024-05-20T18:28:27.296534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5316},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5316":{"gid":5316,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6837/2355805105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":2355805105},{"type":"v1","addr":"172.21.15.5:6837","nonce":2355805105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5316,"qdb_cluster":[5316]},"id":2}]} 2024-05-20T18:28:28.291 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 17 2024-05-20T18:28:28.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.293+0000 7f433c57e640 1 -- 172.21.15.5:0/2476229635 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f431806be30 msgr2=0x7f431806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:28.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.293+0000 7f433c57e640 1 --2- 172.21.15.5:0/2476229635 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f431806be30 0x7f431806e340 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f43341604e0 tx=0x7f4324008040 comp rx=0 tx=0).stop 2024-05-20T18:28:28.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.293+0000 7f433c57e640 1 -- 172.21.15.5:0/2476229635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43340b4010 msgr2=0x7f43340d4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:28.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.293+0000 7f433c57e640 1 --2- 172.21.15.5:0/2476229635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43340b4010 0x7f43340d4d50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f433000e840 tx=0x7f433000ed00 comp rx=0 tx=0).stop 2024-05-20T18:28:28.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.293+0000 7f433c57e640 1 -- 172.21.15.5:0/2476229635 shutdown_connections 2024-05-20T18:28:28.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.293+0000 7f433c57e640 1 --2- 172.21.15.5:0/2476229635 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f431806be30 0x7f431806e340 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:28.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.293+0000 7f433c57e640 1 --2- 172.21.15.5:0/2476229635 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4334163a10 0x7f43340ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:28.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.293+0000 7f433c57e640 1 --2- 172.21.15.5:0/2476229635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43340d14b0 0x7f433414f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:28.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.293+0000 7f433c57e640 1 --2- 172.21.15.5:0/2476229635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43340b4010 0x7f43340d4d50 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:28.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.293+0000 7f433c57e640 1 -- 172.21.15.5:0/2476229635 >> 172.21.15.5:0/2476229635 conn(0x7f433406d240 msgr2=0x7f43340ba3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:28.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.293+0000 7f433c57e640 1 -- 172.21.15.5:0/2476229635 shutdown_connections 2024-05-20T18:28:28.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.293+0000 7f433c57e640 1 -- 172.21.15.5:0/2476229635 wait complete. 2024-05-20T18:28:28.304 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:28:28.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.419+0000 7f4cf59b5640 1 Processor -- start 2024-05-20T18:28:28.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.419+0000 7f4cf59b5640 1 -- start start 2024-05-20T18:28:28.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.420+0000 7f4cf59b5640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f4cf0163a10 0x7f4cf00d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:28.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.420+0000 7f4cf59b5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4cf00b4010 0x7f4cf00cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:28.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.420+0000 7f4cf59b5640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cf00d0370 con 0x7f4cf00d14b0 2024-05-20T18:28:28.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.420+0000 7f4cf59b5640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f4cf00d3ca0 con 0x7f4cf0163a10 2024-05-20T18:28:28.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.420+0000 7f4cf59b5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4cf00d0650 con 0x7f4cf00b4010 2024-05-20T18:28:28.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.420+0000 7f4cef7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f4cf00d14b0 0x7f4cf00d18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53706/0 (socket says 172.21.15.5:53706) 2024-05-20T18:28:28.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.420+0000 7f4cef7fe640 1 -- 172.21.15.5:0/997675321 learned_addr learned my addr 172.21.15.5:0/997675321 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:28.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.420+0000 7f4ce67fc640 1 --2- 172.21.15.5:0/997675321 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4cf00b4010 0x7f4cf00cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:28.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.420+0000 7f4ceeffd640 1 --2- 172.21.15.5:0/997675321 >> v2:172.21.15.63:3300/0 conn(0x7f4cf0163a10 0x7f4cf00d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:28.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.421+0000 7f4cee7fc640 1 -- 172.21.15.5:0/997675321 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2380286607 0 0) 0x7f4cf00d0370 con 0x7f4cf00d14b0 2024-05-20T18:28:28.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.421+0000 7f4cee7fc640 1 -- 172.21.15.5:0/997675321 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4cdc003520 con 0x7f4cf00d14b0 2024-05-20T18:28:28.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.421+0000 7f4cee7fc640 1 -- 172.21.15.5:0/997675321 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1902546289 0 0) 0x7f4cdc003520 con 0x7f4cf00d14b0 2024-05-20T18:28:28.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.421+0000 7f4cee7fc640 1 -- 172.21.15.5:0/997675321 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4cf00b4010 msgr2=0x7f4cf00cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:28.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.421+0000 7f4cee7fc640 1 --2- 172.21.15.5:0/997675321 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4cf00b4010 0x7f4cf00cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:28.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.421+0000 7f4cee7fc640 1 -- 172.21.15.5:0/997675321 >> v2:172.21.15.63:3300/0 conn(0x7f4cf0163a10 msgr2=0x7f4cf00d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:28.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.421+0000 7f4cee7fc640 1 --2- 172.21.15.5:0/997675321 >> v2:172.21.15.63:3300/0 conn(0x7f4cf0163a10 0x7f4cf00d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:28.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.421+0000 7f4cee7fc640 1 -- 172.21.15.5:0/997675321 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf00d3650 con 0x7f4cf00d14b0 2024-05-20T18:28:28.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.421+0000 7f4cee7fc640 1 -- 172.21.15.5:0/997675321 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f4ce80019f0 con 0x7f4cf00d14b0 2024-05-20T18:28:28.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.421+0000 7f4cee7fc640 1 -- 172.21.15.5:0/997675321 >> v1:172.21.15.5:6789/0 conn(0x7f4cf00d14b0 legacy=0x7f4cf00d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:28.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.421+0000 7f4cee7fc640 1 --2- 172.21.15.5:0/997675321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4cdc0039b0 0x7f4cdc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:28.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.421+0000 7f4cee7fc640 1 --2- 172.21.15.5:0/997675321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cdc004800 0x7f4cdc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:28.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.421+0000 7f4cee7fc640 1 --2- 172.21.15.5:0/997675321 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4cf00b4010 0x7f4cdc005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:28.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.421+0000 7f4cee7fc640 1 -- 172.21.15.5:0/997675321 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf00d3ca0 con 0x7f4cdc004800 2024-05-20T18:28:28.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.421+0000 7f4cee7fc640 1 -- 172.21.15.5:0/997675321 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf00d0650 con 0x7f4cdc0039b0 2024-05-20T18:28:28.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.421+0000 7f4cee7fc640 1 -- 172.21.15.5:0/997675321 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4cdc00b9c0 con 0x7f4cf00b4010 2024-05-20T18:28:28.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.421+0000 7f4ceeffd640 1 --2- 172.21.15.5:0/997675321 >> v2:172.21.15.63:3300/0 conn(0x7f4cf0163a10 0x7f4cf00d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:28:28.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.421+0000 7f4ce67fc640 1 --2- 172.21.15.5:0/997675321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cdc004800 0x7f4cdc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:28.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.421+0000 7f4cef7fe640 1 --2- 172.21.15.5:0/997675321 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4cf00b4010 0x7f4cdc005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:28.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.421+0000 7f4ceeffd640 1 --2- 172.21.15.5:0/997675321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4cdc0039b0 0x7f4cdc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:28.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.421+0000 7f4ce67fc640 1 -- 172.21.15.5:0/997675321 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4cf00b4010 msgr2=0x7f4cdc005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:28.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.421+0000 7f4ce67fc640 1 --2- 172.21.15.5:0/997675321 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4cf00b4010 0x7f4cdc005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:28.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.421+0000 7f4ce67fc640 1 -- 172.21.15.5:0/997675321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4cdc0039b0 msgr2=0x7f4cdc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:28.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.421+0000 7f4ce67fc640 1 --2- 172.21.15.5:0/997675321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4cdc0039b0 0x7f4cdc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:28.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.421+0000 7f4ce67fc640 1 -- 172.21.15.5:0/997675321 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cdc003520 con 0x7f4cdc004800 2024-05-20T18:28:28.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.421+0000 7f4cef7fe640 1 --2- 172.21.15.5:0/997675321 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4cf00b4010 0x7f4cdc005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:28.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.422+0000 7f4ce67fc640 1 --2- 172.21.15.5:0/997675321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cdc004800 0x7f4cdc004c10 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8009ac0 tx=0x7f4cdc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=603cf7dd047767d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:28.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.422+0000 7f4cee7fc640 1 -- 172.21.15.5:0/997675321 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8036070 con 0x7f4cdc004800 2024-05-20T18:28:28.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.422+0000 7f4cee7fc640 1 -- 172.21.15.5:0/997675321 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cd8031400 con 0x7f4cdc004800 2024-05-20T18:28:28.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.422+0000 7f4cee7fc640 1 -- 172.21.15.5:0/997675321 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8009690 con 0x7f4cdc004800 2024-05-20T18:28:28.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.423+0000 7f4cf59b5640 1 -- 172.21.15.5:0/997675321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cdc004800 msgr2=0x7f4cdc004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:28.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.423+0000 7f4cf59b5640 1 --2- 172.21.15.5:0/997675321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cdc004800 0x7f4cdc004c10 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8009ac0 tx=0x7f4cdc005830 comp rx=0 tx=0).stop 2024-05-20T18:28:28.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.423+0000 7f4cef7fe640 1 -- 172.21.15.5:0/997675321 reap_dead start 2024-05-20T18:28:28.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.423+0000 7f4cf59b5640 1 -- 172.21.15.5:0/997675321 shutdown_connections 2024-05-20T18:28:28.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.423+0000 7f4cf59b5640 1 -- 172.21.15.5:0/997675321 >> 172.21.15.5:0/997675321 conn(0x7f4cf006d240 msgr2=0x7f4cf006d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:28.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.423+0000 7f4cf59b5640 1 -- 172.21.15.5:0/997675321 shutdown_connections 2024-05-20T18:28:28.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.423+0000 7f4cf59b5640 1 -- 172.21.15.5:0/997675321 wait complete. 2024-05-20T18:28:28.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.425+0000 7f4cf59b5640 1 Processor -- start 2024-05-20T18:28:28.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.425+0000 7f4cf59b5640 1 -- start start 2024-05-20T18:28:28.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.426+0000 7f4cf59b5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cdc004800 0x7f4cf015a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:28.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.426+0000 7f4cf59b5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4cf01540f0 0x7f4cf00cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:28.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.426+0000 7f4cf59b5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4cf00b4f70 0x7f4cf00b5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:28.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.426+0000 7f4cf59b5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf00cfc80 con 0x7f4cdc004800 2024-05-20T18:28:28.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.426+0000 7f4cf59b5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf00cfb00 con 0x7f4cf00b4f70 2024-05-20T18:28:28.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.426+0000 7f4cf59b5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4cf00cfe00 con 0x7f4cf01540f0 2024-05-20T18:28:28.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.426+0000 7f4ceeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cdc004800 0x7f4cf015a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:28.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.426+0000 7f4ceeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cdc004800 0x7f4cf015a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51626/0 (socket says 172.21.15.5:51626) 2024-05-20T18:28:28.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.426+0000 7f4ceeffd640 1 -- 172.21.15.5:0/2369581215 learned_addr learned my addr 172.21.15.5:0/2369581215 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:28.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.427+0000 7f4cef7fe640 1 --2- 172.21.15.5:0/2369581215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4cf00b4f70 0x7f4cf00b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:28.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.427+0000 7f4ce67fc640 1 --2- 172.21.15.5:0/2369581215 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4cf01540f0 0x7f4cf00cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:28.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.427+0000 7f4ceeffd640 1 -- 172.21.15.5:0/2369581215 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4cf01540f0 msgr2=0x7f4cf00cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:28.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.427+0000 7f4ceeffd640 1 --2- 172.21.15.5:0/2369581215 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4cf01540f0 0x7f4cf00cddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:28.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.427+0000 7f4ceeffd640 1 -- 172.21.15.5:0/2369581215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4cf00b4f70 msgr2=0x7f4cf00b5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:28.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.427+0000 7f4ceeffd640 1 --2- 172.21.15.5:0/2369581215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4cf00b4f70 0x7f4cf00b5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:28.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.427+0000 7f4ceeffd640 1 -- 172.21.15.5:0/2369581215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf00b47e0 con 0x7f4cdc004800 2024-05-20T18:28:28.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.427+0000 7f4cef7fe640 1 --2- 172.21.15.5:0/2369581215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4cf00b4f70 0x7f4cf00b5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:28.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.427+0000 7f4ceeffd640 1 --2- 172.21.15.5:0/2369581215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cdc004800 0x7f4cf015a470 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0011900 tx=0x7f4ce0011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:28.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.427+0000 7f4cecff9640 1 -- 172.21.15.5:0/2369581215 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce000fbd0 con 0x7f4cdc004800 2024-05-20T18:28:28.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.427+0000 7f4cecff9640 1 -- 172.21.15.5:0/2369581215 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce000fd70 con 0x7f4cdc004800 2024-05-20T18:28:28.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.427+0000 7f4cf59b5640 1 -- 172.21.15.5:0/2369581215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf015adc0 con 0x7f4cdc004800 2024-05-20T18:28:28.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.427+0000 7f4cecff9640 1 -- 172.21.15.5:0/2369581215 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce00156a0 con 0x7f4cdc004800 2024-05-20T18:28:28.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.427+0000 7f4cf59b5640 1 -- 172.21.15.5:0/2369581215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf00b4a50 con 0x7f4cdc004800 2024-05-20T18:28:28.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.428+0000 7f4cf59b5640 1 -- 172.21.15.5:0/2369581215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cf0075b00 con 0x7f4cdc004800 2024-05-20T18:28:28.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.429+0000 7f4cecff9640 1 -- 172.21.15.5:0/2369581215 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4ce0015840 con 0x7f4cdc004800 2024-05-20T18:28:28.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.429+0000 7f4cecff9640 1 --2- 172.21.15.5:0/2369581215 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4cd006be30 0x7f4cd006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:28.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.429+0000 7f4cecff9640 1 -- 172.21.15.5:0/2369581215 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f4ce008bb70 con 0x7f4cdc004800 2024-05-20T18:28:28.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.429+0000 7f4ce67fc640 1 --2- 172.21.15.5:0/2369581215 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4cd006be30 0x7f4cd006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:28.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.430+0000 7f4ce67fc640 1 --2- 172.21.15.5:0/2369581215 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4cd006be30 0x7f4cd006e340 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8033000 tx=0x7f4cd8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:28.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.434+0000 7f4cecff9640 1 -- 172.21.15.5:0/2369581215 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4ce0019030 con 0x7f4cdc004800 2024-05-20T18:28:28.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.617+0000 7f4cf59b5640 1 -- 172.21.15.5:0/2369581215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4cf00d3050 con 0x7f4cdc004800 2024-05-20T18:28:28.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.618+0000 7f4cecff9640 1 -- 172.21.15.5:0/2369581215 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16822 (secure 0 0 0) 0x7f4ce005a390 con 0x7f4cdc004800 2024-05-20T18:28:28.619 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:28.619 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":40,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:28:25.902315+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:28:22.042995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-20T18:28:23.252728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:28:28.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.620+0000 7f4cf59b5640 1 -- 172.21.15.5:0/2369581215 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4cd006be30 msgr2=0x7f4cd006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:28.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.620+0000 7f4cf59b5640 1 --2- 172.21.15.5:0/2369581215 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4cd006be30 0x7f4cd006e340 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8033000 tx=0x7f4cd8008030 comp rx=0 tx=0).stop 2024-05-20T18:28:28.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.620+0000 7f4cf59b5640 1 -- 172.21.15.5:0/2369581215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cdc004800 msgr2=0x7f4cf015a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:28.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.620+0000 7f4cf59b5640 1 --2- 172.21.15.5:0/2369581215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cdc004800 0x7f4cf015a470 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0011900 tx=0x7f4ce0011dc0 comp rx=0 tx=0).stop 2024-05-20T18:28:28.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.620+0000 7f4cf59b5640 1 -- 172.21.15.5:0/2369581215 shutdown_connections 2024-05-20T18:28:28.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.620+0000 7f4cf59b5640 1 --2- 172.21.15.5:0/2369581215 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4cd006be30 0x7f4cd006e340 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:28.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.620+0000 7f4cf59b5640 1 --2- 172.21.15.5:0/2369581215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4cf00b4f70 0x7f4cf00b5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:28.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.620+0000 7f4cf59b5640 1 --2- 172.21.15.5:0/2369581215 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4cf01540f0 0x7f4cf00cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:28.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.620+0000 7f4cf59b5640 1 --2- 172.21.15.5:0/2369581215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cdc004800 0x7f4cf015a470 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:28.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.620+0000 7f4cf59b5640 1 -- 172.21.15.5:0/2369581215 >> 172.21.15.5:0/2369581215 conn(0x7f4cf006d240 msgr2=0x7f4cf006d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:28.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.621+0000 7f4cf59b5640 1 -- 172.21.15.5:0/2369581215 shutdown_connections 2024-05-20T18:28:28.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.621+0000 7f4cf59b5640 1 -- 172.21.15.5:0/2369581215 wait complete. 2024-05-20T18:28:28.632 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-20T18:28:28.632 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T18:28:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.747+0000 7fa724fb7640 1 Processor -- start 2024-05-20T18:28:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.747+0000 7fa724fb7640 1 -- start start 2024-05-20T18:28:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.749+0000 7fa724fb7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa72006d3d0 0x7fa72006d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.749+0000 7fa724fb7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa7200b8190 0x7fa7200b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.749+0000 7fa724fb7640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa72006dd20 con 0x7fa720162820 2024-05-20T18:28:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.749+0000 7fa724fb7640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa7200d0630 con 0x7fa7200b8190 2024-05-20T18:28:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.749+0000 7fa724fb7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa7200d39c0 con 0x7fa72006d3d0 2024-05-20T18:28:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.749+0000 7fa71e59c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa720162820 0x7fa7200b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53722/0 (socket says 172.21.15.5:53722) 2024-05-20T18:28:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.749+0000 7fa71e59c640 1 -- 172.21.15.5:0/1420920347 learned_addr learned my addr 172.21.15.5:0/1420920347 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.749+0000 7fa71ed9d640 1 --2- 172.21.15.5:0/1420920347 >> v2:172.21.15.63:3300/0 conn(0x7fa7200b8190 0x7fa7200b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.749+0000 7fa717fff640 1 --2- 172.21.15.5:0/1420920347 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa72006d3d0 0x7fa72006d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.749+0000 7fa71dd9b640 1 -- 172.21.15.5:0/1420920347 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1904337049 0 0) 0x7fa710001110 con 0x7fa720162820 2024-05-20T18:28:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.749+0000 7fa71dd9b640 1 -- 172.21.15.5:0/1420920347 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa70c003520 con 0x7fa720162820 2024-05-20T18:28:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.750+0000 7fa71ed9d640 1 -- 172.21.15.5:0/1420920347 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa72006d3d0 msgr2=0x7fa72006d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.750+0000 7fa71ed9d640 1 --2- 172.21.15.5:0/1420920347 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa72006d3d0 0x7fa72006d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.750+0000 7fa71ed9d640 1 -- 172.21.15.5:0/1420920347 >> v1:172.21.15.5:6789/0 conn(0x7fa720162820 legacy=0x7fa7200b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.750+0000 7fa71dd9b640 1 -- 172.21.15.5:0/1420920347 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3475824866 0 0) 0x7fa70c003520 con 0x7fa720162820 2024-05-20T18:28:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.750+0000 7fa71ed9d640 1 -- 172.21.15.5:0/1420920347 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7200d3ca0 con 0x7fa7200b8190 2024-05-20T18:28:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.750+0000 7fa717fff640 1 --2- 172.21.15.5:0/1420920347 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa72006d3d0 0x7fa72006d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:28:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.750+0000 7fa71ed9d640 1 --2- 172.21.15.5:0/1420920347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7200b8190 0x7fa7200b85a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa718009d90 tx=0x7fa71802a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=387b6fc67de1a01d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.751+0000 7fa71dd9b640 1 -- 172.21.15.5:0/1420920347 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa710001840 con 0x7fa7200b8190 2024-05-20T18:28:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.751+0000 7fa71dd9b640 1 -- 172.21.15.5:0/1420920347 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa718007540 con 0x7fa7200b8190 2024-05-20T18:28:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.751+0000 7fa71dd9b640 1 -- 172.21.15.5:0/1420920347 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7180078f0 con 0x7fa7200b8190 2024-05-20T18:28:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.751+0000 7fa724fb7640 1 -- 172.21.15.5:0/1420920347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7200b8190 msgr2=0x7fa7200b85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.751+0000 7fa724fb7640 1 --2- 172.21.15.5:0/1420920347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7200b8190 0x7fa7200b85a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa718009d90 tx=0x7fa71802a5f0 comp rx=0 tx=0).stop 2024-05-20T18:28:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.751+0000 7fa724fb7640 1 -- 172.21.15.5:0/1420920347 shutdown_connections 2024-05-20T18:28:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.751+0000 7fa724fb7640 1 --2- 172.21.15.5:0/1420920347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7200b8190 0x7fa7200b85a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.751+0000 7fa724fb7640 1 --2- 172.21.15.5:0/1420920347 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa72006d3d0 0x7fa72006d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.751+0000 7fa724fb7640 1 -- 172.21.15.5:0/1420920347 >> 172.21.15.5:0/1420920347 conn(0x7fa7200d4bd0 msgr2=0x7fa7200d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.751+0000 7fa724fb7640 1 -- 172.21.15.5:0/1420920347 shutdown_connections 2024-05-20T18:28:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.751+0000 7fa724fb7640 1 -- 172.21.15.5:0/1420920347 wait complete. 2024-05-20T18:28:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.753+0000 7fa724fb7640 1 Processor -- start 2024-05-20T18:28:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.753+0000 7fa724fb7640 1 -- start start 2024-05-20T18:28:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.755+0000 7fa724fb7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa72006d3d0 0x7fa720144d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.755+0000 7fa724fb7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7200b8190 0x7fa7201474e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.755+0000 7fa724fb7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa720162820 0x7fa720147a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.755+0000 7fa724fb7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7200d0300 con 0x7fa720162820 2024-05-20T18:28:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.755+0000 7fa724fb7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa7200d4260 con 0x7fa7200b8190 2024-05-20T18:28:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.755+0000 7fa724fb7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa7200b4050 con 0x7fa72006d3d0 2024-05-20T18:28:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.755+0000 7fa71ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa720162820 0x7fa720147a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.755+0000 7fa71ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa720162820 0x7fa720147a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51636/0 (socket says 172.21.15.5:51636) 2024-05-20T18:28:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.755+0000 7fa71ed9d640 1 -- 172.21.15.5:0/685425598 learned_addr learned my addr 172.21.15.5:0/685425598 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.755+0000 7fa71e59c640 1 --2- 172.21.15.5:0/685425598 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa72006d3d0 0x7fa720144d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.755+0000 7fa717fff640 1 --2- 172.21.15.5:0/685425598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7200b8190 0x7fa7201474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.755+0000 7fa71ed9d640 1 -- 172.21.15.5:0/685425598 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa72006d3d0 msgr2=0x7fa720144d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.755+0000 7fa71ed9d640 1 --2- 172.21.15.5:0/685425598 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa72006d3d0 0x7fa720144d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.755+0000 7fa71ed9d640 1 -- 172.21.15.5:0/685425598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7200b8190 msgr2=0x7fa7201474e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.755+0000 7fa71ed9d640 1 --2- 172.21.15.5:0/685425598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7200b8190 0x7fa7201474e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.755+0000 7fa71ed9d640 1 -- 172.21.15.5:0/685425598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7201456a0 con 0x7fa720162820 2024-05-20T18:28:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.755+0000 7fa71ed9d640 1 --2- 172.21.15.5:0/685425598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa720162820 0x7fa720147a20 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa71802abe0 tx=0x7fa718004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.756+0000 7fa716ffd640 1 -- 172.21.15.5:0/685425598 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71803f070 con 0x7fa720162820 2024-05-20T18:28:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.756+0000 7fa716ffd640 1 -- 172.21.15.5:0/685425598 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa718031c40 con 0x7fa720162820 2024-05-20T18:28:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.756+0000 7fa724fb7640 1 -- 172.21.15.5:0/685425598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa720145980 con 0x7fa720162820 2024-05-20T18:28:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.756+0000 7fa716ffd640 1 -- 172.21.15.5:0/685425598 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa718009450 con 0x7fa720162820 2024-05-20T18:28:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.756+0000 7fa724fb7640 1 -- 172.21.15.5:0/685425598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa720158900 con 0x7fa720162820 2024-05-20T18:28:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.757+0000 7fa724fb7640 1 -- 172.21.15.5:0/685425598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa720075b00 con 0x7fa720162820 2024-05-20T18:28:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.757+0000 7fa716ffd640 1 -- 172.21.15.5:0/685425598 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa718043410 con 0x7fa720162820 2024-05-20T18:28:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.758+0000 7fa716ffd640 1 --2- 172.21.15.5:0/685425598 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa70006be30 0x7fa70006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.758+0000 7fa71e59c640 1 --2- 172.21.15.5:0/685425598 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa70006be30 0x7fa70006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.758+0000 7fa716ffd640 1 -- 172.21.15.5:0/685425598 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fa7180ac180 con 0x7fa720162820 2024-05-20T18:28:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.759+0000 7fa71e59c640 1 --2- 172.21.15.5:0/685425598 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa70006be30 0x7fa70006e340 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa710005880 tx=0x7fa710001230 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:28.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.763+0000 7fa716ffd640 1 -- 172.21.15.5:0/685425598 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa71807a350 con 0x7fa720162820 2024-05-20T18:28:28.867 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-20T18:28:28.868+0000 7f4335486640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T18:28:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.946+0000 7fa724fb7640 1 -- 172.21.15.5:0/685425598 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa720158ba0 con 0x7fa70006be30 2024-05-20T18:28:29.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.952+0000 7fa716ffd640 1 -- 172.21.15.5:0/685425598 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251957 (secure 0 0 0) 0x7fa720158ba0 con 0x7fa70006be30 2024-05-20T18:28:29.349 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:29.350 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-20T18:28:29.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.955+0000 7fa724fb7640 1 -- 172.21.15.5:0/685425598 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa70006be30 msgr2=0x7fa70006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.955+0000 7fa724fb7640 1 --2- 172.21.15.5:0/685425598 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa70006be30 0x7fa70006e340 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa710005880 tx=0x7fa710001230 comp rx=0 tx=0).stop 2024-05-20T18:28:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.955+0000 7fa724fb7640 1 -- 172.21.15.5:0/685425598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa720162820 msgr2=0x7fa720147a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.955+0000 7fa724fb7640 1 --2- 172.21.15.5:0/685425598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa720162820 0x7fa720147a20 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa71802abe0 tx=0x7fa718004a40 comp rx=0 tx=0).stop 2024-05-20T18:28:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.956+0000 7fa724fb7640 1 -- 172.21.15.5:0/685425598 shutdown_connections 2024-05-20T18:28:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.956+0000 7fa724fb7640 1 --2- 172.21.15.5:0/685425598 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa70006be30 0x7fa70006e340 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.956+0000 7fa724fb7640 1 --2- 172.21.15.5:0/685425598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa720162820 0x7fa720147a20 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.956+0000 7fa724fb7640 1 --2- 172.21.15.5:0/685425598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7200b8190 0x7fa7201474e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.956+0000 7fa724fb7640 1 --2- 172.21.15.5:0/685425598 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa72006d3d0 0x7fa720144d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.956+0000 7fa724fb7640 1 -- 172.21.15.5:0/685425598 >> 172.21.15.5:0/685425598 conn(0x7fa7200d4bd0 msgr2=0x7fa7200d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.956+0000 7fa724fb7640 1 -- 172.21.15.5:0/685425598 shutdown_connections 2024-05-20T18:28:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:28.956+0000 7fa724fb7640 1 -- 172.21.15.5:0/685425598 wait complete. 2024-05-20T18:28:29.354 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-05-20T18:28:28.593261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":141,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237628,"kb_used_data":4792,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737092,"statfs":{"total":773094113280,"available":772850782208,"internally_reserved":0,"allocated":4907008,"data_stored":2773732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.720384"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.911701+0000","last_change":"2024-05-20T18:28:24.925237+0000","last_active":"2024-05-20T18:28:25.911701+0000","last_peered":"2024-05-20T18:28:25.911701+0000","last_clean":"2024-05-20T18:28:25.911701+0000","last_became_active":"2024-05-20T18:28:24.924745+0000","last_became_peered":"2024-05-20T18:28:24.924745+0000","last_unstale":"2024-05-20T18:28:25.911701+0000","last_undegraded":"2024-05-20T18:28:25.911701+0000","last_fullsized":"2024-05-20T18:28:25.911701+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:29:37.746005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.911441+0000","last_change":"2024-05-20T18:28:24.911232+0000","last_active":"2024-05-20T18:28:25.911441+0000","last_peered":"2024-05-20T18:28:25.911441+0000","last_clean":"2024-05-20T18:28:25.911441+0000","last_became_active":"2024-05-20T18:28:24.910872+0000","last_became_peered":"2024-05-20T18:28:24.910872+0000","last_unstale":"2024-05-20T18:28:25.911441+0000","last_undegraded":"2024-05-20T18:28:25.911441+0000","last_fullsized":"2024-05-20T18:28:25.911441+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:46:14.154986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.057806+0000","last_change":"2024-05-20T18:28:24.923691+0000","last_active":"2024-05-20T18:28:26.057806+0000","last_peered":"2024-05-20T18:28:26.057806+0000","last_clean":"2024-05-20T18:28:26.057806+0000","last_became_active":"2024-05-20T18:28:24.923139+0000","last_became_peered":"2024-05-20T18:28:24.923139+0000","last_unstale":"2024-05-20T18:28:26.057806+0000","last_undegraded":"2024-05-20T18:28:26.057806+0000","last_fullsized":"2024-05-20T18:28:26.057806+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:22:53.319670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.913425+0000","last_change":"2024-05-20T18:28:24.916947+0000","last_active":"2024-05-20T18:28:25.913425+0000","last_peered":"2024-05-20T18:28:25.913425+0000","last_clean":"2024-05-20T18:28:25.913425+0000","last_became_active":"2024-05-20T18:28:24.916270+0000","last_became_peered":"2024-05-20T18:28:24.916270+0000","last_unstale":"2024-05-20T18:28:25.913425+0000","last_undegraded":"2024-05-20T18:28:25.913425+0000","last_fullsized":"2024-05-20T18:28:25.913425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:43:52.611770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.913266+0000","last_change":"2024-05-20T18:28:24.918503+0000","last_active":"2024-05-20T18:28:25.913266+0000","last_peered":"2024-05-20T18:28:25.913266+0000","last_clean":"2024-05-20T18:28:25.913266+0000","last_became_active":"2024-05-20T18:28:24.918170+0000","last_became_peered":"2024-05-20T18:28:24.918170+0000","last_unstale":"2024-05-20T18:28:25.913266+0000","last_undegraded":"2024-05-20T18:28:25.913266+0000","last_fullsized":"2024-05-20T18:28:25.913266+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:46:00.161250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058004+0000","last_change":"2024-05-20T18:28:24.920538+0000","last_active":"2024-05-20T18:28:26.058004+0000","last_peered":"2024-05-20T18:28:26.058004+0000","last_clean":"2024-05-20T18:28:26.058004+0000","last_became_active":"2024-05-20T18:28:24.920217+0000","last_became_peered":"2024-05-20T18:28:24.920217+0000","last_unstale":"2024-05-20T18:28:26.058004+0000","last_undegraded":"2024-05-20T18:28:26.058004+0000","last_fullsized":"2024-05-20T18:28:26.058004+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:50:54.323312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.916072+0000","last_change":"2024-05-20T18:28:24.918154+0000","last_active":"2024-05-20T18:28:25.916072+0000","last_peered":"2024-05-20T18:28:25.916072+0000","last_clean":"2024-05-20T18:28:25.916072+0000","last_became_active":"2024-05-20T18:28:24.917825+0000","last_became_peered":"2024-05-20T18:28:24.917825+0000","last_unstale":"2024-05-20T18:28:25.916072+0000","last_undegraded":"2024-05-20T18:28:25.916072+0000","last_fullsized":"2024-05-20T18:28:25.916072+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:11:46.939770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.910898+0000","last_change":"2024-05-20T18:28:24.916670+0000","last_active":"2024-05-20T18:28:25.910898+0000","last_peered":"2024-05-20T18:28:25.910898+0000","last_clean":"2024-05-20T18:28:25.910898+0000","last_became_active":"2024-05-20T18:28:24.916345+0000","last_became_peered":"2024-05-20T18:28:24.916345+0000","last_unstale":"2024-05-20T18:28:25.910898+0000","last_undegraded":"2024-05-20T18:28:25.910898+0000","last_fullsized":"2024-05-20T18:28:25.910898+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:22:10.801800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.057961+0000","last_change":"2024-05-20T18:28:24.914944+0000","last_active":"2024-05-20T18:28:26.057961+0000","last_peered":"2024-05-20T18:28:26.057961+0000","last_clean":"2024-05-20T18:28:26.057961+0000","last_became_active":"2024-05-20T18:28:24.914400+0000","last_became_peered":"2024-05-20T18:28:24.914400+0000","last_unstale":"2024-05-20T18:28:26.057961+0000","last_undegraded":"2024-05-20T18:28:26.057961+0000","last_fullsized":"2024-05-20T18:28:26.057961+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:43:12.609193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.916880+0000","last_change":"2024-05-20T18:28:24.916672+0000","last_active":"2024-05-20T18:28:25.916880+0000","last_peered":"2024-05-20T18:28:25.916880+0000","last_clean":"2024-05-20T18:28:25.916880+0000","last_became_active":"2024-05-20T18:28:24.913171+0000","last_became_peered":"2024-05-20T18:28:24.913171+0000","last_unstale":"2024-05-20T18:28:25.916880+0000","last_undegraded":"2024-05-20T18:28:25.916880+0000","last_fullsized":"2024-05-20T18:28:25.916880+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:43:20.094774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.917161+0000","last_change":"2024-05-20T18:28:24.918125+0000","last_active":"2024-05-20T18:28:25.917161+0000","last_peered":"2024-05-20T18:28:25.917161+0000","last_clean":"2024-05-20T18:28:25.917161+0000","last_became_active":"2024-05-20T18:28:24.917782+0000","last_became_peered":"2024-05-20T18:28:24.917782+0000","last_unstale":"2024-05-20T18:28:25.917161+0000","last_undegraded":"2024-05-20T18:28:25.917161+0000","last_fullsized":"2024-05-20T18:28:25.917161+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:00:56.383920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.057708+0000","last_change":"2024-05-20T18:28:24.917036+0000","last_active":"2024-05-20T18:28:26.057708+0000","last_peered":"2024-05-20T18:28:26.057708+0000","last_clean":"2024-05-20T18:28:26.057708+0000","last_became_active":"2024-05-20T18:28:24.916642+0000","last_became_peered":"2024-05-20T18:28:24.916642+0000","last_unstale":"2024-05-20T18:28:26.057708+0000","last_undegraded":"2024-05-20T18:28:26.057708+0000","last_fullsized":"2024-05-20T18:28:26.057708+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:17:13.880808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.916456+0000","last_change":"2024-05-20T18:28:24.922811+0000","last_active":"2024-05-20T18:28:25.916456+0000","last_peered":"2024-05-20T18:28:25.916456+0000","last_clean":"2024-05-20T18:28:25.916456+0000","last_became_active":"2024-05-20T18:28:24.922479+0000","last_became_peered":"2024-05-20T18:28:24.922479+0000","last_unstale":"2024-05-20T18:28:25.916456+0000","last_undegraded":"2024-05-20T18:28:25.916456+0000","last_fullsized":"2024-05-20T18:28:25.916456+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:40:49.558398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.915861+0000","last_change":"2024-05-20T18:28:24.911637+0000","last_active":"2024-05-20T18:28:25.915861+0000","last_peered":"2024-05-20T18:28:25.915861+0000","last_clean":"2024-05-20T18:28:25.915861+0000","last_became_active":"2024-05-20T18:28:24.911164+0000","last_became_peered":"2024-05-20T18:28:24.911164+0000","last_unstale":"2024-05-20T18:28:25.915861+0000","last_undegraded":"2024-05-20T18:28:25.915861+0000","last_fullsized":"2024-05-20T18:28:25.915861+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:11:47.463102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.916507+0000","last_change":"2024-05-20T18:28:24.923073+0000","last_active":"2024-05-20T18:28:25.916507+0000","last_peered":"2024-05-20T18:28:25.916507+0000","last_clean":"2024-05-20T18:28:25.916507+0000","last_became_active":"2024-05-20T18:28:24.921935+0000","last_became_peered":"2024-05-20T18:28:24.921935+0000","last_unstale":"2024-05-20T18:28:25.916507+0000","last_undegraded":"2024-05-20T18:28:25.916507+0000","last_fullsized":"2024-05-20T18:28:25.916507+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:55:36.297837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.057900+0000","last_change":"2024-05-20T18:28:24.924179+0000","last_active":"2024-05-20T18:28:26.057900+0000","last_peered":"2024-05-20T18:28:26.057900+0000","last_clean":"2024-05-20T18:28:26.057900+0000","last_became_active":"2024-05-20T18:28:24.923839+0000","last_became_peered":"2024-05-20T18:28:24.923839+0000","last_unstale":"2024-05-20T18:28:26.057900+0000","last_undegraded":"2024-05-20T18:28:26.057900+0000","last_fullsized":"2024-05-20T18:28:26.057900+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:50:54.323312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058609+0000","last_change":"2024-05-20T18:28:24.923447+0000","last_active":"2024-05-20T18:28:26.058609+0000","last_peered":"2024-05-20T18:28:26.058609+0000","last_clean":"2024-05-20T18:28:26.058609+0000","last_became_active":"2024-05-20T18:28:24.923124+0000","last_became_peered":"2024-05-20T18:28:24.923124+0000","last_unstale":"2024-05-20T18:28:26.058609+0000","last_undegraded":"2024-05-20T18:28:26.058609+0000","last_fullsized":"2024-05-20T18:28:26.058609+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:27:23.187550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.918463+0000","last_change":"2024-05-20T18:28:24.913022+0000","last_active":"2024-05-20T18:28:25.918463+0000","last_peered":"2024-05-20T18:28:25.918463+0000","last_clean":"2024-05-20T18:28:25.918463+0000","last_became_active":"2024-05-20T18:28:24.912708+0000","last_became_peered":"2024-05-20T18:28:24.912708+0000","last_unstale":"2024-05-20T18:28:25.918463+0000","last_undegraded":"2024-05-20T18:28:25.918463+0000","last_fullsized":"2024-05-20T18:28:25.918463+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:32:35.134081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.913519+0000","last_change":"2024-05-20T18:28:24.914891+0000","last_active":"2024-05-20T18:28:25.913519+0000","last_peered":"2024-05-20T18:28:25.913519+0000","last_clean":"2024-05-20T18:28:25.913519+0000","last_became_active":"2024-05-20T18:28:24.914577+0000","last_became_peered":"2024-05-20T18:28:24.914577+0000","last_unstale":"2024-05-20T18:28:25.913519+0000","last_undegraded":"2024-05-20T18:28:25.913519+0000","last_fullsized":"2024-05-20T18:28:25.913519+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:39:42.086357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.918297+0000","last_change":"2024-05-20T18:28:24.909832+0000","last_active":"2024-05-20T18:28:25.918297+0000","last_peered":"2024-05-20T18:28:25.918297+0000","last_clean":"2024-05-20T18:28:25.918297+0000","last_became_active":"2024-05-20T18:28:24.909506+0000","last_became_peered":"2024-05-20T18:28:24.909506+0000","last_unstale":"2024-05-20T18:28:25.918297+0000","last_undegraded":"2024-05-20T18:28:25.918297+0000","last_fullsized":"2024-05-20T18:28:25.918297+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:27:02.499774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.916648+0000","last_change":"2024-05-20T18:28:24.914062+0000","last_active":"2024-05-20T18:28:25.916648+0000","last_peered":"2024-05-20T18:28:25.916648+0000","last_clean":"2024-05-20T18:28:25.916648+0000","last_became_active":"2024-05-20T18:28:24.913723+0000","last_became_peered":"2024-05-20T18:28:24.913723+0000","last_unstale":"2024-05-20T18:28:25.916648+0000","last_undegraded":"2024-05-20T18:28:25.916648+0000","last_fullsized":"2024-05-20T18:28:25.916648+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:02:17.059963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.916368+0000","last_change":"2024-05-20T18:28:24.915215+0000","last_active":"2024-05-20T18:28:25.916368+0000","last_peered":"2024-05-20T18:28:25.916368+0000","last_clean":"2024-05-20T18:28:25.916368+0000","last_became_active":"2024-05-20T18:28:24.914879+0000","last_became_peered":"2024-05-20T18:28:24.914879+0000","last_unstale":"2024-05-20T18:28:25.916368+0000","last_undegraded":"2024-05-20T18:28:25.916368+0000","last_fullsized":"2024-05-20T18:28:25.916368+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:17:13.880808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.057993+0000","last_change":"2024-05-20T18:28:24.924961+0000","last_active":"2024-05-20T18:28:26.057993+0000","last_peered":"2024-05-20T18:28:26.057993+0000","last_clean":"2024-05-20T18:28:26.057993+0000","last_became_active":"2024-05-20T18:28:24.924589+0000","last_became_peered":"2024-05-20T18:28:24.924589+0000","last_unstale":"2024-05-20T18:28:26.057993+0000","last_undegraded":"2024-05-20T18:28:26.057993+0000","last_fullsized":"2024-05-20T18:28:26.057993+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:17:13.880808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.913114+0000","last_change":"2024-05-20T18:28:24.917948+0000","last_active":"2024-05-20T18:28:25.913114+0000","last_peered":"2024-05-20T18:28:25.913114+0000","last_clean":"2024-05-20T18:28:25.913114+0000","last_became_active":"2024-05-20T18:28:24.917642+0000","last_became_peered":"2024-05-20T18:28:24.917642+0000","last_unstale":"2024-05-20T18:28:25.913114+0000","last_undegraded":"2024-05-20T18:28:25.913114+0000","last_fullsized":"2024-05-20T18:28:25.913114+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:52:18.085709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.918648+0000","last_change":"2024-05-20T18:28:24.916506+0000","last_active":"2024-05-20T18:28:25.918648+0000","last_peered":"2024-05-20T18:28:25.918648+0000","last_clean":"2024-05-20T18:28:25.918648+0000","last_became_active":"2024-05-20T18:28:24.916098+0000","last_became_peered":"2024-05-20T18:28:24.916098+0000","last_unstale":"2024-05-20T18:28:25.918648+0000","last_undegraded":"2024-05-20T18:28:25.918648+0000","last_fullsized":"2024-05-20T18:28:25.918648+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:35:24.942545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058384+0000","last_change":"2024-05-20T18:28:24.919467+0000","last_active":"2024-05-20T18:28:26.058384+0000","last_peered":"2024-05-20T18:28:26.058384+0000","last_clean":"2024-05-20T18:28:26.058384+0000","last_became_active":"2024-05-20T18:28:24.919105+0000","last_became_peered":"2024-05-20T18:28:24.919105+0000","last_unstale":"2024-05-20T18:28:26.058384+0000","last_undegraded":"2024-05-20T18:28:26.058384+0000","last_fullsized":"2024-05-20T18:28:26.058384+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:11:59.707265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.918481+0000","last_change":"2024-05-20T18:28:24.912602+0000","last_active":"2024-05-20T18:28:25.918481+0000","last_peered":"2024-05-20T18:28:25.918481+0000","last_clean":"2024-05-20T18:28:25.918481+0000","last_became_active":"2024-05-20T18:28:24.912228+0000","last_became_peered":"2024-05-20T18:28:24.912228+0000","last_unstale":"2024-05-20T18:28:25.918481+0000","last_undegraded":"2024-05-20T18:28:25.918481+0000","last_fullsized":"2024-05-20T18:28:25.918481+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:53:48.597638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.917148+0000","last_change":"2024-05-20T18:28:24.916263+0000","last_active":"2024-05-20T18:28:25.917148+0000","last_peered":"2024-05-20T18:28:25.917148+0000","last_clean":"2024-05-20T18:28:25.917148+0000","last_became_active":"2024-05-20T18:28:24.914253+0000","last_became_peered":"2024-05-20T18:28:24.914253+0000","last_unstale":"2024-05-20T18:28:25.917148+0000","last_undegraded":"2024-05-20T18:28:25.917148+0000","last_fullsized":"2024-05-20T18:28:25.917148+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:26:11.273568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.918344+0000","last_change":"2024-05-20T18:28:24.910327+0000","last_active":"2024-05-20T18:28:25.918344+0000","last_peered":"2024-05-20T18:28:25.918344+0000","last_clean":"2024-05-20T18:28:25.918344+0000","last_became_active":"2024-05-20T18:28:24.909988+0000","last_became_peered":"2024-05-20T18:28:24.909988+0000","last_unstale":"2024-05-20T18:28:25.918344+0000","last_undegraded":"2024-05-20T18:28:25.918344+0000","last_fullsized":"2024-05-20T18:28:25.918344+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:07:11.841682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.918184+0000","last_change":"2024-05-20T18:28:24.910812+0000","last_active":"2024-05-20T18:28:25.918184+0000","last_peered":"2024-05-20T18:28:25.918184+0000","last_clean":"2024-05-20T18:28:25.918184+0000","last_became_active":"2024-05-20T18:28:24.910471+0000","last_became_peered":"2024-05-20T18:28:24.910471+0000","last_unstale":"2024-05-20T18:28:25.918184+0000","last_undegraded":"2024-05-20T18:28:25.918184+0000","last_fullsized":"2024-05-20T18:28:25.918184+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:25:24.281662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058119+0000","last_change":"2024-05-20T18:28:24.926360+0000","last_active":"2024-05-20T18:28:26.058119+0000","last_peered":"2024-05-20T18:28:26.058119+0000","last_clean":"2024-05-20T18:28:26.058119+0000","last_became_active":"2024-05-20T18:28:24.925985+0000","last_became_peered":"2024-05-20T18:28:24.925985+0000","last_unstale":"2024-05-20T18:28:26.058119+0000","last_undegraded":"2024-05-20T18:28:26.058119+0000","last_fullsized":"2024-05-20T18:28:26.058119+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:11:59.707265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.057970+0000","last_change":"2024-05-20T18:28:24.911082+0000","last_active":"2024-05-20T18:28:26.057970+0000","last_peered":"2024-05-20T18:28:26.057970+0000","last_clean":"2024-05-20T18:28:26.057970+0000","last_became_active":"2024-05-20T18:28:24.910748+0000","last_became_peered":"2024-05-20T18:28:24.910748+0000","last_unstale":"2024-05-20T18:28:26.057970+0000","last_undegraded":"2024-05-20T18:28:26.057970+0000","last_fullsized":"2024-05-20T18:28:26.057970+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:11:47.463102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058266+0000","last_change":"2024-05-20T18:28:24.923491+0000","last_active":"2024-05-20T18:28:26.058266+0000","last_peered":"2024-05-20T18:28:26.058266+0000","last_clean":"2024-05-20T18:28:26.058266+0000","last_became_active":"2024-05-20T18:28:24.923111+0000","last_became_peered":"2024-05-20T18:28:24.923111+0000","last_unstale":"2024-05-20T18:28:26.058266+0000","last_undegraded":"2024-05-20T18:28:26.058266+0000","last_fullsized":"2024-05-20T18:28:26.058266+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:43:12.609193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.918609+0000","last_change":"2024-05-20T18:28:23.890621+0000","last_active":"2024-05-20T18:28:25.918609+0000","last_peered":"2024-05-20T18:28:25.918609+0000","last_clean":"2024-05-20T18:28:25.918609+0000","last_became_active":"2024-05-20T18:28:23.890215+0000","last_became_peered":"2024-05-20T18:28:23.890215+0000","last_unstale":"2024-05-20T18:28:25.918609+0000","last_undegraded":"2024-05-20T18:28:25.918609+0000","last_fullsized":"2024-05-20T18:28:25.918609+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:52:17.081034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058432+0000","last_change":"2024-05-20T18:28:24.920530+0000","last_active":"2024-05-20T18:28:26.058432+0000","last_peered":"2024-05-20T18:28:26.058432+0000","last_clean":"2024-05-20T18:28:26.058432+0000","last_became_active":"2024-05-20T18:28:24.920187+0000","last_became_peered":"2024-05-20T18:28:24.920187+0000","last_unstale":"2024-05-20T18:28:26.058432+0000","last_undegraded":"2024-05-20T18:28:26.058432+0000","last_fullsized":"2024-05-20T18:28:26.058432+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:22:53.319670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.057939+0000","last_change":"2024-05-20T18:28:23.901392+0000","last_active":"2024-05-20T18:28:26.057939+0000","last_peered":"2024-05-20T18:28:26.057939+0000","last_clean":"2024-05-20T18:28:26.057939+0000","last_became_active":"2024-05-20T18:28:23.901035+0000","last_became_peered":"2024-05-20T18:28:23.901035+0000","last_unstale":"2024-05-20T18:28:26.057939+0000","last_undegraded":"2024-05-20T18:28:26.057939+0000","last_fullsized":"2024-05-20T18:28:26.057939+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:43:51.607095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.917369+0000","last_change":"2024-05-20T18:28:24.912716+0000","last_active":"2024-05-20T18:28:25.917369+0000","last_peered":"2024-05-20T18:28:25.917369+0000","last_clean":"2024-05-20T18:28:25.917369+0000","last_became_active":"2024-05-20T18:28:24.912394+0000","last_became_peered":"2024-05-20T18:28:24.912394+0000","last_unstale":"2024-05-20T18:28:25.917369+0000","last_undegraded":"2024-05-20T18:28:25.917369+0000","last_fullsized":"2024-05-20T18:28:25.917369+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:43:12.609193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058486+0000","last_change":"2024-05-20T18:28:23.903027+0000","last_active":"2024-05-20T18:28:26.058486+0000","last_peered":"2024-05-20T18:28:26.058486+0000","last_clean":"2024-05-20T18:28:26.058486+0000","last_became_active":"2024-05-20T18:28:23.902322+0000","last_became_peered":"2024-05-20T18:28:23.902322+0000","last_unstale":"2024-05-20T18:28:26.058486+0000","last_undegraded":"2024-05-20T18:28:26.058486+0000","last_fullsized":"2024-05-20T18:28:26.058486+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:39:41.081682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.911597+0000","last_change":"2024-05-20T18:28:24.908701+0000","last_active":"2024-05-20T18:28:25.911597+0000","last_peered":"2024-05-20T18:28:25.911597+0000","last_clean":"2024-05-20T18:28:25.911597+0000","last_became_active":"2024-05-20T18:28:24.908438+0000","last_became_peered":"2024-05-20T18:28:24.908438+0000","last_unstale":"2024-05-20T18:28:25.911597+0000","last_undegraded":"2024-05-20T18:28:25.911597+0000","last_fullsized":"2024-05-20T18:28:25.911597+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:13:00.979960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.916835+0000","last_change":"2024-05-20T18:28:23.885560+0000","last_active":"2024-05-20T18:28:25.916835+0000","last_peered":"2024-05-20T18:28:25.916835+0000","last_clean":"2024-05-20T18:28:25.916835+0000","last_became_active":"2024-05-20T18:28:23.885157+0000","last_became_peered":"2024-05-20T18:28:23.885157+0000","last_unstale":"2024-05-20T18:28:25.916835+0000","last_undegraded":"2024-05-20T18:28:25.916835+0000","last_fullsized":"2024-05-20T18:28:25.916835+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:55:23.129684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.057786+0000","last_change":"2024-05-20T18:28:24.915518+0000","last_active":"2024-05-20T18:28:26.057786+0000","last_peered":"2024-05-20T18:28:26.057786+0000","last_clean":"2024-05-20T18:28:26.057786+0000","last_became_active":"2024-05-20T18:28:24.914816+0000","last_became_peered":"2024-05-20T18:28:24.914816+0000","last_unstale":"2024-05-20T18:28:26.057786+0000","last_undegraded":"2024-05-20T18:28:26.057786+0000","last_fullsized":"2024-05-20T18:28:26.057786+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:11:59.707265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.916704+0000","last_change":"2024-05-20T18:28:23.897800+0000","last_active":"2024-05-20T18:28:25.916704+0000","last_peered":"2024-05-20T18:28:25.916704+0000","last_clean":"2024-05-20T18:28:25.916704+0000","last_became_active":"2024-05-20T18:28:23.897473+0000","last_became_peered":"2024-05-20T18:28:23.897473+0000","last_unstale":"2024-05-20T18:28:25.916704+0000","last_undegraded":"2024-05-20T18:28:25.916704+0000","last_fullsized":"2024-05-20T18:28:25.916704+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:53:47.592963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.916291+0000","last_change":"2024-05-20T18:28:24.910962+0000","last_active":"2024-05-20T18:28:25.916291+0000","last_peered":"2024-05-20T18:28:25.916291+0000","last_clean":"2024-05-20T18:28:25.916291+0000","last_became_active":"2024-05-20T18:28:24.910471+0000","last_became_peered":"2024-05-20T18:28:24.910471+0000","last_unstale":"2024-05-20T18:28:25.916291+0000","last_undegraded":"2024-05-20T18:28:25.916291+0000","last_fullsized":"2024-05-20T18:28:25.916291+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:18:47.935602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058066+0000","last_change":"2024-05-20T18:28:23.898365+0000","last_active":"2024-05-20T18:28:26.058066+0000","last_peered":"2024-05-20T18:28:26.058066+0000","last_clean":"2024-05-20T18:28:26.058066+0000","last_became_active":"2024-05-20T18:28:23.897996+0000","last_became_peered":"2024-05-20T18:28:23.897996+0000","last_unstale":"2024-05-20T18:28:26.058066+0000","last_undegraded":"2024-05-20T18:28:26.058066+0000","last_fullsized":"2024-05-20T18:28:26.058066+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:45:59.156575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.916729+0000","last_change":"2024-05-20T18:28:24.918574+0000","last_active":"2024-05-20T18:28:25.916729+0000","last_peered":"2024-05-20T18:28:25.916729+0000","last_clean":"2024-05-20T18:28:25.916729+0000","last_became_active":"2024-05-20T18:28:24.918163+0000","last_became_peered":"2024-05-20T18:28:24.918163+0000","last_unstale":"2024-05-20T18:28:25.916729+0000","last_undegraded":"2024-05-20T18:28:25.916729+0000","last_fullsized":"2024-05-20T18:28:25.916729+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:11:59.707265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058382+0000","last_change":"2024-05-20T18:28:23.898055+0000","last_active":"2024-05-20T18:28:26.058382+0000","last_peered":"2024-05-20T18:28:26.058382+0000","last_clean":"2024-05-20T18:28:26.058382+0000","last_became_active":"2024-05-20T18:28:23.897583+0000","last_became_peered":"2024-05-20T18:28:23.897583+0000","last_unstale":"2024-05-20T18:28:26.058382+0000","last_undegraded":"2024-05-20T18:28:26.058382+0000","last_fullsized":"2024-05-20T18:28:26.058382+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:26:56.937862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.911028+0000","last_change":"2024-05-20T18:28:24.911448+0000","last_active":"2024-05-20T18:28:25.911028+0000","last_peered":"2024-05-20T18:28:25.911028+0000","last_clean":"2024-05-20T18:28:25.911028+0000","last_became_active":"2024-05-20T18:28:24.911103+0000","last_became_peered":"2024-05-20T18:28:24.911103+0000","last_unstale":"2024-05-20T18:28:25.911028+0000","last_undegraded":"2024-05-20T18:28:25.911028+0000","last_fullsized":"2024-05-20T18:28:25.911028+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:13:25.533126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.915693+0000","last_change":"2024-05-20T18:28:23.887743+0000","last_active":"2024-05-20T18:28:25.915693+0000","last_peered":"2024-05-20T18:28:25.915693+0000","last_clean":"2024-05-20T18:28:25.915693+0000","last_became_active":"2024-05-20T18:28:23.887384+0000","last_became_peered":"2024-05-20T18:28:23.887384+0000","last_unstale":"2024-05-20T18:28:25.915693+0000","last_undegraded":"2024-05-20T18:28:25.915693+0000","last_fullsized":"2024-05-20T18:28:25.915693+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:26:56.937862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.913388+0000","last_change":"2024-05-20T18:28:24.917239+0000","last_active":"2024-05-20T18:28:25.913388+0000","last_peered":"2024-05-20T18:28:25.913388+0000","last_clean":"2024-05-20T18:28:25.913388+0000","last_became_active":"2024-05-20T18:28:24.916648+0000","last_became_peered":"2024-05-20T18:28:24.916648+0000","last_unstale":"2024-05-20T18:28:25.913388+0000","last_undegraded":"2024-05-20T18:28:25.913388+0000","last_fullsized":"2024-05-20T18:28:25.913388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:26:57.942537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.911701+0000","last_change":"2024-05-20T18:28:23.895233+0000","last_active":"2024-05-20T18:28:25.911701+0000","last_peered":"2024-05-20T18:28:25.911701+0000","last_clean":"2024-05-20T18:28:25.911701+0000","last_became_active":"2024-05-20T18:28:23.894890+0000","last_became_peered":"2024-05-20T18:28:23.894890+0000","last_unstale":"2024-05-20T18:28:25.911701+0000","last_undegraded":"2024-05-20T18:28:25.911701+0000","last_fullsized":"2024-05-20T18:28:25.911701+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:40:48.553723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058464+0000","last_change":"2024-05-20T18:28:24.918894+0000","last_active":"2024-05-20T18:28:26.058464+0000","last_peered":"2024-05-20T18:28:26.058464+0000","last_clean":"2024-05-20T18:28:26.058464+0000","last_became_active":"2024-05-20T18:28:24.918100+0000","last_became_peered":"2024-05-20T18:28:24.918100+0000","last_unstale":"2024-05-20T18:28:26.058464+0000","last_undegraded":"2024-05-20T18:28:26.058464+0000","last_fullsized":"2024-05-20T18:28:26.058464+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:11:46.939770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.917307+0000","last_change":"2024-05-20T18:28:23.893401+0000","last_active":"2024-05-20T18:28:25.917307+0000","last_peered":"2024-05-20T18:28:25.917307+0000","last_clean":"2024-05-20T18:28:25.917307+0000","last_became_active":"2024-05-20T18:28:23.893018+0000","last_became_peered":"2024-05-20T18:28:23.893018+0000","last_unstale":"2024-05-20T18:28:25.917307+0000","last_undegraded":"2024-05-20T18:28:25.917307+0000","last_fullsized":"2024-05-20T18:28:25.917307+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:25:23.276987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058014+0000","last_change":"2024-05-20T18:28:24.915342+0000","last_active":"2024-05-20T18:28:26.058014+0000","last_peered":"2024-05-20T18:28:26.058014+0000","last_clean":"2024-05-20T18:28:26.058014+0000","last_became_active":"2024-05-20T18:28:24.914997+0000","last_became_peered":"2024-05-20T18:28:24.914997+0000","last_unstale":"2024-05-20T18:28:26.058014+0000","last_undegraded":"2024-05-20T18:28:26.058014+0000","last_fullsized":"2024-05-20T18:28:26.058014+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:46:07.023527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.917285+0000","last_change":"2024-05-20T18:28:23.898547+0000","last_active":"2024-05-20T18:28:25.917285+0000","last_peered":"2024-05-20T18:28:25.917285+0000","last_clean":"2024-05-20T18:28:25.917285+0000","last_became_active":"2024-05-20T18:28:23.898206+0000","last_became_peered":"2024-05-20T18:28:23.898206+0000","last_unstale":"2024-05-20T18:28:25.917285+0000","last_undegraded":"2024-05-20T18:28:25.917285+0000","last_fullsized":"2024-05-20T18:28:25.917285+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:32:34.129406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.916643+0000","last_change":"2024-05-20T18:28:24.910052+0000","last_active":"2024-05-20T18:28:25.916643+0000","last_peered":"2024-05-20T18:28:25.916643+0000","last_clean":"2024-05-20T18:28:25.916643+0000","last_became_active":"2024-05-20T18:28:24.909709+0000","last_became_peered":"2024-05-20T18:28:24.909709+0000","last_unstale":"2024-05-20T18:28:25.916643+0000","last_undegraded":"2024-05-20T18:28:25.916643+0000","last_fullsized":"2024-05-20T18:28:25.916643+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:33:28.408090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.911489+0000","last_change":"2024-05-20T18:28:23.901816+0000","last_active":"2024-05-20T18:28:25.911489+0000","last_peered":"2024-05-20T18:28:25.911489+0000","last_clean":"2024-05-20T18:28:25.911489+0000","last_became_active":"2024-05-20T18:28:23.901465+0000","last_became_peered":"2024-05-20T18:28:23.901465+0000","last_unstale":"2024-05-20T18:28:25.911489+0000","last_undegraded":"2024-05-20T18:28:25.911489+0000","last_fullsized":"2024-05-20T18:28:25.911489+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:27:22.182875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.057560+0000","last_change":"2024-05-20T18:28:24.915240+0000","last_active":"2024-05-20T18:28:26.057560+0000","last_peered":"2024-05-20T18:28:26.057560+0000","last_clean":"2024-05-20T18:28:26.057560+0000","last_became_active":"2024-05-20T18:28:24.914374+0000","last_became_peered":"2024-05-20T18:28:24.914374+0000","last_unstale":"2024-05-20T18:28:26.057560+0000","last_undegraded":"2024-05-20T18:28:26.057560+0000","last_fullsized":"2024-05-20T18:28:26.057560+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:11:47.463102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.911294+0000","last_change":"2024-05-20T18:28:23.893914+0000","last_active":"2024-05-20T18:28:25.911294+0000","last_peered":"2024-05-20T18:28:25.911294+0000","last_clean":"2024-05-20T18:28:25.911294+0000","last_became_active":"2024-05-20T18:28:23.893589+0000","last_became_peered":"2024-05-20T18:28:23.893589+0000","last_unstale":"2024-05-20T18:28:25.911294+0000","last_undegraded":"2024-05-20T18:28:25.911294+0000","last_fullsized":"2024-05-20T18:28:25.911294+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:18:46.930927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.910960+0000","last_change":"2024-05-20T18:28:24.922679+0000","last_active":"2024-05-20T18:28:25.910960+0000","last_peered":"2024-05-20T18:28:25.910960+0000","last_clean":"2024-05-20T18:28:25.910960+0000","last_became_active":"2024-05-20T18:28:24.922314+0000","last_became_peered":"2024-05-20T18:28:24.922314+0000","last_unstale":"2024-05-20T18:28:25.910960+0000","last_undegraded":"2024-05-20T18:28:25.910960+0000","last_fullsized":"2024-05-20T18:28:25.910960+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:51:45.606238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058130+0000","last_change":"2024-05-20T18:28:23.899428+0000","last_active":"2024-05-20T18:28:26.058130+0000","last_peered":"2024-05-20T18:28:26.058130+0000","last_clean":"2024-05-20T18:28:26.058130+0000","last_became_active":"2024-05-20T18:28:23.899078+0000","last_became_peered":"2024-05-20T18:28:23.899078+0000","last_unstale":"2024-05-20T18:28:26.058130+0000","last_undegraded":"2024-05-20T18:28:26.058130+0000","last_fullsized":"2024-05-20T18:28:26.058130+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:43:51.607095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058076+0000","last_change":"2024-05-20T18:28:24.919882+0000","last_active":"2024-05-20T18:28:26.058076+0000","last_peered":"2024-05-20T18:28:26.058076+0000","last_clean":"2024-05-20T18:28:26.058076+0000","last_became_active":"2024-05-20T18:28:24.919529+0000","last_became_peered":"2024-05-20T18:28:24.919529+0000","last_unstale":"2024-05-20T18:28:26.058076+0000","last_undegraded":"2024-05-20T18:28:26.058076+0000","last_fullsized":"2024-05-20T18:28:26.058076+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:33:28.408090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.917930+0000","last_change":"2024-05-20T18:28:23.891290+0000","last_active":"2024-05-20T18:28:25.917930+0000","last_peered":"2024-05-20T18:28:25.917930+0000","last_clean":"2024-05-20T18:28:25.917930+0000","last_became_active":"2024-05-20T18:28:23.890931+0000","last_became_peered":"2024-05-20T18:28:23.890931+0000","last_unstale":"2024-05-20T18:28:25.917930+0000","last_undegraded":"2024-05-20T18:28:25.917930+0000","last_fullsized":"2024-05-20T18:28:25.917930+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:04:14.459707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.911160+0000","last_change":"2024-05-20T18:28:24.921845+0000","last_active":"2024-05-20T18:28:25.911160+0000","last_peered":"2024-05-20T18:28:25.911160+0000","last_clean":"2024-05-20T18:28:25.911160+0000","last_became_active":"2024-05-20T18:28:24.921539+0000","last_became_peered":"2024-05-20T18:28:24.921539+0000","last_unstale":"2024-05-20T18:28:25.911160+0000","last_undegraded":"2024-05-20T18:28:25.911160+0000","last_fullsized":"2024-05-20T18:28:25.911160+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:20:49.429451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":22,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.057968+0000","last_change":"2024-05-20T18:28:23.899387+0000","last_active":"2024-05-20T18:28:26.057968+0000","last_peered":"2024-05-20T18:28:26.057968+0000","last_clean":"2024-05-20T18:28:26.057968+0000","last_became_active":"2024-05-20T18:28:23.898884+0000","last_became_peered":"2024-05-20T18:28:23.898884+0000","last_unstale":"2024-05-20T18:28:26.057968+0000","last_undegraded":"2024-05-20T18:28:26.057968+0000","last_fullsized":"2024-05-20T18:28:26.057968+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:50:09.909370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.911313+0000","last_change":"2024-05-20T18:28:24.908025+0000","last_active":"2024-05-20T18:28:25.911313+0000","last_peered":"2024-05-20T18:28:25.911313+0000","last_clean":"2024-05-20T18:28:25.911313+0000","last_became_active":"2024-05-20T18:28:24.907747+0000","last_became_peered":"2024-05-20T18:28:24.907747+0000","last_unstale":"2024-05-20T18:28:25.911313+0000","last_undegraded":"2024-05-20T18:28:25.911313+0000","last_fullsized":"2024-05-20T18:28:25.911313+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:17:36.795124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.917038+0000","last_change":"2024-05-20T18:28:23.897609+0000","last_active":"2024-05-20T18:28:25.917038+0000","last_peered":"2024-05-20T18:28:25.917038+0000","last_clean":"2024-05-20T18:28:25.917038+0000","last_became_active":"2024-05-20T18:28:23.897267+0000","last_became_peered":"2024-05-20T18:28:23.897267+0000","last_unstale":"2024-05-20T18:28:25.917038+0000","last_undegraded":"2024-05-20T18:28:25.917038+0000","last_fullsized":"2024-05-20T18:28:25.917038+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:55:23.129684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.917133+0000","last_change":"2024-05-20T18:28:24.926018+0000","last_active":"2024-05-20T18:28:25.917133+0000","last_peered":"2024-05-20T18:28:25.917133+0000","last_clean":"2024-05-20T18:28:25.917133+0000","last_became_active":"2024-05-20T18:28:24.925674+0000","last_became_peered":"2024-05-20T18:28:24.925674+0000","last_unstale":"2024-05-20T18:28:25.917133+0000","last_undegraded":"2024-05-20T18:28:25.917133+0000","last_fullsized":"2024-05-20T18:28:25.917133+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:49:05.545604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.917335+0000","last_change":"2024-05-20T18:28:23.895801+0000","last_active":"2024-05-20T18:28:25.917335+0000","last_peered":"2024-05-20T18:28:25.917335+0000","last_clean":"2024-05-20T18:28:25.917335+0000","last_became_active":"2024-05-20T18:28:23.895446+0000","last_became_peered":"2024-05-20T18:28:23.895446+0000","last_unstale":"2024-05-20T18:28:25.917335+0000","last_undegraded":"2024-05-20T18:28:25.917335+0000","last_fullsized":"2024-05-20T18:28:25.917335+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:43:51.607095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.917157+0000","last_change":"2024-05-20T18:28:24.905847+0000","last_active":"2024-05-20T18:28:25.917157+0000","last_peered":"2024-05-20T18:28:25.917157+0000","last_clean":"2024-05-20T18:28:25.917157+0000","last_became_active":"2024-05-20T18:28:24.905496+0000","last_became_peered":"2024-05-20T18:28:24.905496+0000","last_unstale":"2024-05-20T18:28:25.917157+0000","last_undegraded":"2024-05-20T18:28:25.917157+0000","last_fullsized":"2024-05-20T18:28:25.917157+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:22:53.319670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.913472+0000","last_change":"2024-05-20T18:28:23.899625+0000","last_active":"2024-05-20T18:28:25.913472+0000","last_peered":"2024-05-20T18:28:25.913472+0000","last_clean":"2024-05-20T18:28:25.913472+0000","last_became_active":"2024-05-20T18:28:23.899293+0000","last_became_peered":"2024-05-20T18:28:23.899293+0000","last_unstale":"2024-05-20T18:28:25.913472+0000","last_undegraded":"2024-05-20T18:28:25.913472+0000","last_fullsized":"2024-05-20T18:28:25.913472+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:09:39.031471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.913372+0000","last_change":"2024-05-20T18:28:24.918997+0000","last_active":"2024-05-20T18:28:25.913372+0000","last_peered":"2024-05-20T18:28:25.913372+0000","last_clean":"2024-05-20T18:28:25.913372+0000","last_became_active":"2024-05-20T18:28:24.918622+0000","last_became_peered":"2024-05-20T18:28:24.918622+0000","last_unstale":"2024-05-20T18:28:25.913372+0000","last_undegraded":"2024-05-20T18:28:25.913372+0000","last_fullsized":"2024-05-20T18:28:25.913372+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:50:10.914045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058199+0000","last_change":"2024-05-20T18:28:23.900260+0000","last_active":"2024-05-20T18:28:26.058199+0000","last_peered":"2024-05-20T18:28:26.058199+0000","last_clean":"2024-05-20T18:28:26.058199+0000","last_became_active":"2024-05-20T18:28:23.898939+0000","last_became_peered":"2024-05-20T18:28:23.898939+0000","last_unstale":"2024-05-20T18:28:26.058199+0000","last_undegraded":"2024-05-20T18:28:26.058199+0000","last_fullsized":"2024-05-20T18:28:26.058199+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:55:23.129684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058224+0000","last_change":"2024-05-20T18:28:24.926391+0000","last_active":"2024-05-20T18:28:26.058224+0000","last_peered":"2024-05-20T18:28:26.058224+0000","last_clean":"2024-05-20T18:28:26.058224+0000","last_became_active":"2024-05-20T18:28:24.926050+0000","last_became_peered":"2024-05-20T18:28:24.926050+0000","last_unstale":"2024-05-20T18:28:26.058224+0000","last_undegraded":"2024-05-20T18:28:26.058224+0000","last_fullsized":"2024-05-20T18:28:26.058224+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:22:53.319670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.913232+0000","last_change":"2024-05-20T18:28:23.896073+0000","last_active":"2024-05-20T18:28:25.913232+0000","last_peered":"2024-05-20T18:28:25.913232+0000","last_clean":"2024-05-20T18:28:25.913232+0000","last_became_active":"2024-05-20T18:28:23.895727+0000","last_became_peered":"2024-05-20T18:28:23.895727+0000","last_unstale":"2024-05-20T18:28:25.913232+0000","last_undegraded":"2024-05-20T18:28:25.913232+0000","last_fullsized":"2024-05-20T18:28:25.913232+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:02:30.183710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.057633+0000","last_change":"2024-05-20T18:28:24.914653+0000","last_active":"2024-05-20T18:28:26.057633+0000","last_peered":"2024-05-20T18:28:26.057633+0000","last_clean":"2024-05-20T18:28:26.057633+0000","last_became_active":"2024-05-20T18:28:24.914207+0000","last_became_peered":"2024-05-20T18:28:24.914207+0000","last_unstale":"2024-05-20T18:28:26.057633+0000","last_undegraded":"2024-05-20T18:28:26.057633+0000","last_fullsized":"2024-05-20T18:28:26.057633+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:50:54.323312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.913145+0000","last_change":"2024-05-20T18:28:23.887872+0000","last_active":"2024-05-20T18:28:25.913145+0000","last_peered":"2024-05-20T18:28:25.913145+0000","last_clean":"2024-05-20T18:28:25.913145+0000","last_became_active":"2024-05-20T18:28:23.887519+0000","last_became_peered":"2024-05-20T18:28:23.887519+0000","last_unstale":"2024-05-20T18:28:25.913145+0000","last_undegraded":"2024-05-20T18:28:25.913145+0000","last_fullsized":"2024-05-20T18:28:25.913145+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:38:07.068531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.916499+0000","last_change":"2024-05-20T18:28:24.916171+0000","last_active":"2024-05-20T18:28:25.916499+0000","last_peered":"2024-05-20T18:28:25.916499+0000","last_clean":"2024-05-20T18:28:25.916499+0000","last_became_active":"2024-05-20T18:28:24.913433+0000","last_became_peered":"2024-05-20T18:28:24.913433+0000","last_unstale":"2024-05-20T18:28:25.916499+0000","last_undegraded":"2024-05-20T18:28:25.916499+0000","last_fullsized":"2024-05-20T18:28:25.916499+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:07:28.502495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058153+0000","last_change":"2024-05-20T18:28:23.898491+0000","last_active":"2024-05-20T18:28:26.058153+0000","last_peered":"2024-05-20T18:28:26.058153+0000","last_clean":"2024-05-20T18:28:26.058153+0000","last_became_active":"2024-05-20T18:28:23.898167+0000","last_became_peered":"2024-05-20T18:28:23.898167+0000","last_unstale":"2024-05-20T18:28:26.058153+0000","last_undegraded":"2024-05-20T18:28:26.058153+0000","last_fullsized":"2024-05-20T18:28:26.058153+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:43:51.607095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.911214+0000","last_change":"2024-05-20T18:28:24.910463+0000","last_active":"2024-05-20T18:28:25.911214+0000","last_peered":"2024-05-20T18:28:25.911214+0000","last_clean":"2024-05-20T18:28:25.911214+0000","last_became_active":"2024-05-20T18:28:24.909954+0000","last_became_peered":"2024-05-20T18:28:24.909954+0000","last_unstale":"2024-05-20T18:28:25.911214+0000","last_undegraded":"2024-05-20T18:28:25.911214+0000","last_fullsized":"2024-05-20T18:28:25.911214+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:30:33.575837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.913050+0000","last_change":"2024-05-20T18:28:23.892500+0000","last_active":"2024-05-20T18:28:25.913050+0000","last_peered":"2024-05-20T18:28:25.913050+0000","last_clean":"2024-05-20T18:28:25.913050+0000","last_became_active":"2024-05-20T18:28:23.892137+0000","last_became_peered":"2024-05-20T18:28:23.892137+0000","last_unstale":"2024-05-20T18:28:25.913050+0000","last_undegraded":"2024-05-20T18:28:25.913050+0000","last_fullsized":"2024-05-20T18:28:25.913050+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:17:16.487528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058210+0000","last_change":"2024-05-20T18:28:24.918164+0000","last_active":"2024-05-20T18:28:26.058210+0000","last_peered":"2024-05-20T18:28:26.058210+0000","last_clean":"2024-05-20T18:28:26.058210+0000","last_became_active":"2024-05-20T18:28:24.917832+0000","last_became_peered":"2024-05-20T18:28:24.917832+0000","last_unstale":"2024-05-20T18:28:26.058210+0000","last_undegraded":"2024-05-20T18:28:26.058210+0000","last_fullsized":"2024-05-20T18:28:26.058210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:33:28.408090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.917206+0000","last_change":"2024-05-20T18:28:23.899440+0000","last_active":"2024-05-20T18:28:25.917206+0000","last_peered":"2024-05-20T18:28:25.917206+0000","last_clean":"2024-05-20T18:28:25.917206+0000","last_became_active":"2024-05-20T18:28:23.899129+0000","last_became_peered":"2024-05-20T18:28:23.899129+0000","last_unstale":"2024-05-20T18:28:25.917206+0000","last_undegraded":"2024-05-20T18:28:25.917206+0000","last_fullsized":"2024-05-20T18:28:25.917206+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:27:01.495099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058559+0000","last_change":"2024-05-20T18:28:24.919200+0000","last_active":"2024-05-20T18:28:26.058559+0000","last_peered":"2024-05-20T18:28:26.058559+0000","last_clean":"2024-05-20T18:28:26.058559+0000","last_became_active":"2024-05-20T18:28:24.918571+0000","last_became_peered":"2024-05-20T18:28:24.918571+0000","last_unstale":"2024-05-20T18:28:26.058559+0000","last_undegraded":"2024-05-20T18:28:26.058559+0000","last_fullsized":"2024-05-20T18:28:26.058559+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:18:47.935602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.915154+0000","last_change":"2024-05-20T18:28:23.893151+0000","last_active":"2024-05-20T18:28:25.915154+0000","last_peered":"2024-05-20T18:28:25.915154+0000","last_clean":"2024-05-20T18:28:25.915154+0000","last_became_active":"2024-05-20T18:28:23.892806+0000","last_became_peered":"2024-05-20T18:28:23.892806+0000","last_unstale":"2024-05-20T18:28:25.915154+0000","last_undegraded":"2024-05-20T18:28:25.915154+0000","last_fullsized":"2024-05-20T18:28:25.915154+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:03:58.193858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.911516+0000","last_change":"2024-05-20T18:28:24.910137+0000","last_active":"2024-05-20T18:28:25.911516+0000","last_peered":"2024-05-20T18:28:25.911516+0000","last_clean":"2024-05-20T18:28:25.911516+0000","last_became_active":"2024-05-20T18:28:24.909763+0000","last_became_peered":"2024-05-20T18:28:24.909763+0000","last_unstale":"2024-05-20T18:28:25.911516+0000","last_undegraded":"2024-05-20T18:28:25.911516+0000","last_fullsized":"2024-05-20T18:28:25.911516+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:37:34.646258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.057804+0000","last_change":"2024-05-20T18:28:23.898332+0000","last_active":"2024-05-20T18:28:26.057804+0000","last_peered":"2024-05-20T18:28:26.057804+0000","last_clean":"2024-05-20T18:28:26.057804+0000","last_became_active":"2024-05-20T18:28:23.897949+0000","last_became_peered":"2024-05-20T18:28:23.897949+0000","last_unstale":"2024-05-20T18:28:26.057804+0000","last_undegraded":"2024-05-20T18:28:26.057804+0000","last_fullsized":"2024-05-20T18:28:26.057804+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:26:56.937862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.057744+0000","last_change":"2024-05-20T18:28:24.917396+0000","last_active":"2024-05-20T18:28:26.057744+0000","last_peered":"2024-05-20T18:28:26.057744+0000","last_clean":"2024-05-20T18:28:26.057744+0000","last_became_active":"2024-05-20T18:28:24.917094+0000","last_became_peered":"2024-05-20T18:28:24.917094+0000","last_unstale":"2024-05-20T18:28:26.057744+0000","last_undegraded":"2024-05-20T18:28:26.057744+0000","last_fullsized":"2024-05-20T18:28:26.057744+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:33:28.408090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058081+0000","last_change":"2024-05-20T18:28:23.895060+0000","last_active":"2024-05-20T18:28:26.058081+0000","last_peered":"2024-05-20T18:28:26.058081+0000","last_clean":"2024-05-20T18:28:26.058081+0000","last_became_active":"2024-05-20T18:28:23.894690+0000","last_became_peered":"2024-05-20T18:28:23.894690+0000","last_unstale":"2024-05-20T18:28:26.058081+0000","last_undegraded":"2024-05-20T18:28:26.058081+0000","last_fullsized":"2024-05-20T18:28:26.058081+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:39:41.081682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058087+0000","last_change":"2024-05-20T18:28:24.914948+0000","last_active":"2024-05-20T18:28:26.058087+0000","last_peered":"2024-05-20T18:28:26.058087+0000","last_clean":"2024-05-20T18:28:26.058087+0000","last_became_active":"2024-05-20T18:28:24.914621+0000","last_became_peered":"2024-05-20T18:28:24.914621+0000","last_unstale":"2024-05-20T18:28:26.058087+0000","last_undegraded":"2024-05-20T18:28:26.058087+0000","last_fullsized":"2024-05-20T18:28:26.058087+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:17:13.880808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.918125+0000","last_change":"2024-05-20T18:28:23.897978+0000","last_active":"2024-05-20T18:28:25.918125+0000","last_peered":"2024-05-20T18:28:25.918125+0000","last_clean":"2024-05-20T18:28:25.918125+0000","last_became_active":"2024-05-20T18:28:23.897611+0000","last_became_peered":"2024-05-20T18:28:23.897611+0000","last_unstale":"2024-05-20T18:28:25.918125+0000","last_undegraded":"2024-05-20T18:28:25.918125+0000","last_fullsized":"2024-05-20T18:28:25.918125+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:21:52.115216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.916072+0000","last_change":"2024-05-20T18:28:24.914362+0000","last_active":"2024-05-20T18:28:25.916072+0000","last_peered":"2024-05-20T18:28:25.916072+0000","last_clean":"2024-05-20T18:28:25.916072+0000","last_became_active":"2024-05-20T18:28:24.914044+0000","last_became_peered":"2024-05-20T18:28:24.914044+0000","last_unstale":"2024-05-20T18:28:25.916072+0000","last_undegraded":"2024-05-20T18:28:25.916072+0000","last_fullsized":"2024-05-20T18:28:25.916072+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:50:54.323312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.915767+0000","last_change":"2024-05-20T18:28:23.889191+0000","last_active":"2024-05-20T18:28:25.915767+0000","last_peered":"2024-05-20T18:28:25.915767+0000","last_clean":"2024-05-20T18:28:25.915767+0000","last_became_active":"2024-05-20T18:28:23.888846+0000","last_became_peered":"2024-05-20T18:28:23.888846+0000","last_unstale":"2024-05-20T18:28:25.915767+0000","last_undegraded":"2024-05-20T18:28:25.915767+0000","last_fullsized":"2024-05-20T18:28:25.915767+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:39:41.081682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058439+0000","last_change":"2024-05-20T18:28:24.918976+0000","last_active":"2024-05-20T18:28:26.058439+0000","last_peered":"2024-05-20T18:28:26.058439+0000","last_clean":"2024-05-20T18:28:26.058439+0000","last_became_active":"2024-05-20T18:28:24.918545+0000","last_became_peered":"2024-05-20T18:28:24.918545+0000","last_unstale":"2024-05-20T18:28:26.058439+0000","last_undegraded":"2024-05-20T18:28:26.058439+0000","last_fullsized":"2024-05-20T18:28:26.058439+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:46:07.023527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.910931+0000","last_change":"2024-05-20T18:28:23.898195+0000","last_active":"2024-05-20T18:28:25.910931+0000","last_peered":"2024-05-20T18:28:25.910931+0000","last_clean":"2024-05-20T18:28:25.910931+0000","last_became_active":"2024-05-20T18:28:23.897826+0000","last_became_peered":"2024-05-20T18:28:23.897826+0000","last_unstale":"2024-05-20T18:28:25.910931+0000","last_undegraded":"2024-05-20T18:28:25.910931+0000","last_fullsized":"2024-05-20T18:28:25.910931+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:11:45.935095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.916373+0000","last_change":"2024-05-20T18:28:24.921307+0000","last_active":"2024-05-20T18:28:25.916373+0000","last_peered":"2024-05-20T18:28:25.916373+0000","last_clean":"2024-05-20T18:28:25.916373+0000","last_became_active":"2024-05-20T18:28:24.920842+0000","last_became_peered":"2024-05-20T18:28:24.920842+0000","last_unstale":"2024-05-20T18:28:25.916373+0000","last_undegraded":"2024-05-20T18:28:25.916373+0000","last_fullsized":"2024-05-20T18:28:25.916373+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:27:23.187550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058437+0000","last_change":"2024-05-20T18:28:23.899326+0000","last_active":"2024-05-20T18:28:26.058437+0000","last_peered":"2024-05-20T18:28:26.058437+0000","last_clean":"2024-05-20T18:28:26.058437+0000","last_became_active":"2024-05-20T18:28:23.898977+0000","last_became_peered":"2024-05-20T18:28:23.898977+0000","last_unstale":"2024-05-20T18:28:26.058437+0000","last_undegraded":"2024-05-20T18:28:26.058437+0000","last_fullsized":"2024-05-20T18:28:26.058437+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:26:56.937862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":205,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058210+0000","last_change":"2024-05-20T18:27:03.765694+0000","last_active":"2024-05-20T18:28:26.058210+0000","last_peered":"2024-05-20T18:28:26.058210+0000","last_clean":"2024-05-20T18:28:26.058210+0000","last_became_active":"2024-05-20T18:27:03.764695+0000","last_became_peered":"2024-05-20T18:27:03.764695+0000","last_unstale":"2024-05-20T18:28:26.058210+0000","last_undegraded":"2024-05-20T18:28:26.058210+0000","last_fullsized":"2024-05-20T18:28:26.058210+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_clean_scrub_stamp":"2024-05-20T18:27:02.750853+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:30:04.407487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":787226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":202450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":195288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":202472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31656,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340184,"statfs":{"total":96636764160,"available":96604348416,"internally_reserved":0,"allocated":479232,"data_stored":203514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":454656,"data_stored":195246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":454656,"data_stored":195266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1060864,"data_stored":792270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T18:28:29.358 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T18:28:29.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.476+0000 7f28cb0e7640 1 Processor -- start 2024-05-20T18:28:29.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.476+0000 7f28cb0e7640 1 -- start start 2024-05-20T18:28:29.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.478+0000 7f28cb0e7640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f28c40b4010 0x7f28c40d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:29.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.478+0000 7f28cb0e7640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28c40d0370 con 0x7f28c40d14b0 2024-05-20T18:28:29.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.478+0000 7f28cb0e7640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28c40d0650 con 0x7f28c4163a10 2024-05-20T18:28:29.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.478+0000 7f28cb0e7640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f28c40d3ca0 con 0x7f28c40b4010 2024-05-20T18:28:29.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.478+0000 7f28c3fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f28c40b4010 0x7f28c40d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:29.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.478+0000 7f28c9684640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f28c40d14b0 0x7f28c40d18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53728/0 (socket says 172.21.15.5:53728) 2024-05-20T18:28:29.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.478+0000 7f28c9684640 1 -- 172.21.15.5:0/4176675987 learned_addr learned my addr 172.21.15.5:0/4176675987 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:29.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.478+0000 7f28c37fe640 1 -- 172.21.15.5:0/4176675987 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3577016234 0 0) 0x7f28c40d0370 con 0x7f28c40d14b0 2024-05-20T18:28:29.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.478+0000 7f28c37fe640 1 -- 172.21.15.5:0/4176675987 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28b4003520 con 0x7f28c40d14b0 2024-05-20T18:28:29.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.478+0000 7f28c3fff640 1 -- 172.21.15.5:0/4176675987 >> v1:172.21.15.63:6789/0 conn(0x7f28c4163a10 legacy=0x7f28c40cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:29.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.478+0000 7f28c37fe640 1 -- 172.21.15.5:0/4176675987 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 631388767 0 0) 0x7f28c40d0650 con 0x7f28c4163a10 2024-05-20T18:28:29.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.478+0000 7f28c3fff640 1 -- 172.21.15.5:0/4176675987 >> v1:172.21.15.5:6789/0 conn(0x7f28c40d14b0 legacy=0x7f28c40d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:29.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.478+0000 7f28c3fff640 1 -- 172.21.15.5:0/4176675987 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c40d3650 con 0x7f28c40b4010 2024-05-20T18:28:29.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.479+0000 7f28c3fff640 1 --2- 172.21.15.5:0/4176675987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28c40b4010 0x7f28c40d4130 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f28b00097a0 tx=0x7f28b002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf611e6704a491eb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:29.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.479+0000 7f28c37fe640 1 -- 172.21.15.5:0/4176675987 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b0037070 con 0x7f28c40b4010 2024-05-20T18:28:29.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.479+0000 7f28c37fe640 1 -- 172.21.15.5:0/4176675987 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28b0032440 con 0x7f28c40b4010 2024-05-20T18:28:29.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.479+0000 7f28c37fe640 1 -- 172.21.15.5:0/4176675987 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b0032810 con 0x7f28c40b4010 2024-05-20T18:28:29.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.480+0000 7f28cb0e7640 1 -- 172.21.15.5:0/4176675987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28c40b4010 msgr2=0x7f28c40d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:29.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.480+0000 7f28cb0e7640 1 --2- 172.21.15.5:0/4176675987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28c40b4010 0x7f28c40d4130 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f28b00097a0 tx=0x7f28b002a230 comp rx=0 tx=0).stop 2024-05-20T18:28:29.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.480+0000 7f28cb0e7640 1 -- 172.21.15.5:0/4176675987 shutdown_connections 2024-05-20T18:28:29.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.480+0000 7f28cb0e7640 1 --2- 172.21.15.5:0/4176675987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28c40b4010 0x7f28c40d4130 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:29.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.480+0000 7f28cb0e7640 1 -- 172.21.15.5:0/4176675987 >> 172.21.15.5:0/4176675987 conn(0x7f28c406d240 msgr2=0x7f28c406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:29.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.480+0000 7f28cb0e7640 1 -- 172.21.15.5:0/4176675987 shutdown_connections 2024-05-20T18:28:29.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.480+0000 7f28cb0e7640 1 -- 172.21.15.5:0/4176675987 wait complete. 2024-05-20T18:28:29.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.482+0000 7f28cb0e7640 1 Processor -- start 2024-05-20T18:28:29.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.482+0000 7f28cb0e7640 1 -- start start 2024-05-20T18:28:29.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.483+0000 7f28cb0e7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28c40b4010 0x7f28c41599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:29.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.483+0000 7f28cb0e7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28c40d14b0 0x7f28c4159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:29.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.483+0000 7f28cb0e7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f28c4163a10 0x7f28c41f5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:29.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.483+0000 7f28cb0e7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f28c40b8390 con 0x7f28c40b4010 2024-05-20T18:28:29.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.483+0000 7f28cb0e7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f28c40b8210 con 0x7f28c40d14b0 2024-05-20T18:28:29.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.483+0000 7f28cb0e7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f28c40b8510 con 0x7f28c4163a10 2024-05-20T18:28:29.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.483+0000 7f28c8e83640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28c40b4010 0x7f28c41599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:29.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.483+0000 7f28c8e83640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28c40b4010 0x7f28c41599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51654/0 (socket says 172.21.15.5:51654) 2024-05-20T18:28:29.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.483+0000 7f28c8e83640 1 -- 172.21.15.5:0/3288576436 learned_addr learned my addr 172.21.15.5:0/3288576436 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:29.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.484+0000 7f28c3fff640 1 --2- 172.21.15.5:0/3288576436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28c40d14b0 0x7f28c4159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:29.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.484+0000 7f28c9684640 1 --2- 172.21.15.5:0/3288576436 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f28c4163a10 0x7f28c41f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:29.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.484+0000 7f28c8e83640 1 -- 172.21.15.5:0/3288576436 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f28c4163a10 msgr2=0x7f28c41f5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:29.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.484+0000 7f28c8e83640 1 --2- 172.21.15.5:0/3288576436 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f28c4163a10 0x7f28c41f5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:29.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.484+0000 7f28c8e83640 1 -- 172.21.15.5:0/3288576436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28c40d14b0 msgr2=0x7f28c4159f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:29.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.484+0000 7f28c8e83640 1 --2- 172.21.15.5:0/3288576436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28c40d14b0 0x7f28c4159f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:29.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.484+0000 7f28c8e83640 1 -- 172.21.15.5:0/3288576436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c41553c0 con 0x7f28c40b4010 2024-05-20T18:28:29.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.484+0000 7f28c8e83640 1 --2- 172.21.15.5:0/3288576436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28c40b4010 0x7f28c41599c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f28b800c3e0 tx=0x7f28b800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:29.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.484+0000 7f28c17fa640 1 -- 172.21.15.5:0/3288576436 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8017070 con 0x7f28c40b4010 2024-05-20T18:28:29.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.484+0000 7f28c17fa640 1 -- 172.21.15.5:0/3288576436 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28b80035f0 con 0x7f28c40b4010 2024-05-20T18:28:29.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.484+0000 7f28cb0e7640 1 -- 172.21.15.5:0/3288576436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28c4155660 con 0x7f28c40b4010 2024-05-20T18:28:29.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.484+0000 7f28c17fa640 1 -- 172.21.15.5:0/3288576436 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b80039c0 con 0x7f28c40b4010 2024-05-20T18:28:29.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.484+0000 7f28cb0e7640 1 -- 172.21.15.5:0/3288576436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28c4155c20 con 0x7f28c40b4010 2024-05-20T18:28:29.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.485+0000 7f28cb0e7640 1 -- 172.21.15.5:0/3288576436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28c4075b00 con 0x7f28c40b4010 2024-05-20T18:28:29.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.486+0000 7f28c17fa640 1 -- 172.21.15.5:0/3288576436 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f28b801a070 con 0x7f28c40b4010 2024-05-20T18:28:29.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.486+0000 7f28c17fa640 1 --2- 172.21.15.5:0/3288576436 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f28a806be60 0x7f28a806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:29.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.486+0000 7f28c17fa640 1 -- 172.21.15.5:0/3288576436 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f28b808a330 con 0x7f28c40b4010 2024-05-20T18:28:29.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.486+0000 7f28c3fff640 1 --2- 172.21.15.5:0/3288576436 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f28a806be60 0x7f28a806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:29.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.487+0000 7f28c3fff640 1 --2- 172.21.15.5:0/3288576436 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f28a806be60 0x7f28a806e370 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f28b0034000 tx=0x7f28b002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:29.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.491+0000 7f28c17fa640 1 -- 172.21.15.5:0/3288576436 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f28b8058470 con 0x7f28c40b4010 2024-05-20T18:28:29.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.671+0000 7f28cb0e7640 1 -- 172.21.15.5:0/3288576436 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f28c40d3050 con 0x7f28a806be60 2024-05-20T18:28:29.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.678+0000 7f28c17fa640 1 -- 172.21.15.5:0/3288576436 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251957 (secure 0 0 0) 0x7f28c40d3050 con 0x7f28a806be60 2024-05-20T18:28:29.679 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:29.681 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-20T18:28:29.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.681+0000 7f28cb0e7640 1 -- 172.21.15.5:0/3288576436 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f28a806be60 msgr2=0x7f28a806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:29.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.681+0000 7f28cb0e7640 1 --2- 172.21.15.5:0/3288576436 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f28a806be60 0x7f28a806e370 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f28b0034000 tx=0x7f28b002e000 comp rx=0 tx=0).stop 2024-05-20T18:28:29.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.681+0000 7f28cb0e7640 1 -- 172.21.15.5:0/3288576436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28c40b4010 msgr2=0x7f28c41599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:29.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.681+0000 7f28cb0e7640 1 --2- 172.21.15.5:0/3288576436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28c40b4010 0x7f28c41599c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f28b800c3e0 tx=0x7f28b800c8a0 comp rx=0 tx=0).stop 2024-05-20T18:28:29.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.681+0000 7f28cb0e7640 1 -- 172.21.15.5:0/3288576436 shutdown_connections 2024-05-20T18:28:29.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.681+0000 7f28cb0e7640 1 --2- 172.21.15.5:0/3288576436 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f28a806be60 0x7f28a806e370 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:29.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.681+0000 7f28cb0e7640 1 --2- 172.21.15.5:0/3288576436 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f28c4163a10 0x7f28c41f5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:29.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.681+0000 7f28cb0e7640 1 --2- 172.21.15.5:0/3288576436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28c40d14b0 0x7f28c4159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:29.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.681+0000 7f28cb0e7640 1 --2- 172.21.15.5:0/3288576436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28c40b4010 0x7f28c41599c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:29.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.681+0000 7f28cb0e7640 1 -- 172.21.15.5:0/3288576436 >> 172.21.15.5:0/3288576436 conn(0x7f28c406d240 msgr2=0x7f28c406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:29.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.681+0000 7f28cb0e7640 1 -- 172.21.15.5:0/3288576436 shutdown_connections 2024-05-20T18:28:29.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.681+0000 7f28cb0e7640 1 -- 172.21.15.5:0/3288576436 wait complete. 2024-05-20T18:28:29.692 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-05-20T18:28:28.593261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":141,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237628,"kb_used_data":4792,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737092,"statfs":{"total":773094113280,"available":772850782208,"internally_reserved":0,"allocated":4907008,"data_stored":2773732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.720384"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.911701+0000","last_change":"2024-05-20T18:28:24.925237+0000","last_active":"2024-05-20T18:28:25.911701+0000","last_peered":"2024-05-20T18:28:25.911701+0000","last_clean":"2024-05-20T18:28:25.911701+0000","last_became_active":"2024-05-20T18:28:24.924745+0000","last_became_peered":"2024-05-20T18:28:24.924745+0000","last_unstale":"2024-05-20T18:28:25.911701+0000","last_undegraded":"2024-05-20T18:28:25.911701+0000","last_fullsized":"2024-05-20T18:28:25.911701+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:29:37.746005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.911441+0000","last_change":"2024-05-20T18:28:24.911232+0000","last_active":"2024-05-20T18:28:25.911441+0000","last_peered":"2024-05-20T18:28:25.911441+0000","last_clean":"2024-05-20T18:28:25.911441+0000","last_became_active":"2024-05-20T18:28:24.910872+0000","last_became_peered":"2024-05-20T18:28:24.910872+0000","last_unstale":"2024-05-20T18:28:25.911441+0000","last_undegraded":"2024-05-20T18:28:25.911441+0000","last_fullsized":"2024-05-20T18:28:25.911441+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:46:14.154986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.057806+0000","last_change":"2024-05-20T18:28:24.923691+0000","last_active":"2024-05-20T18:28:26.057806+0000","last_peered":"2024-05-20T18:28:26.057806+0000","last_clean":"2024-05-20T18:28:26.057806+0000","last_became_active":"2024-05-20T18:28:24.923139+0000","last_became_peered":"2024-05-20T18:28:24.923139+0000","last_unstale":"2024-05-20T18:28:26.057806+0000","last_undegraded":"2024-05-20T18:28:26.057806+0000","last_fullsized":"2024-05-20T18:28:26.057806+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:22:53.319670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.913425+0000","last_change":"2024-05-20T18:28:24.916947+0000","last_active":"2024-05-20T18:28:25.913425+0000","last_peered":"2024-05-20T18:28:25.913425+0000","last_clean":"2024-05-20T18:28:25.913425+0000","last_became_active":"2024-05-20T18:28:24.916270+0000","last_became_peered":"2024-05-20T18:28:24.916270+0000","last_unstale":"2024-05-20T18:28:25.913425+0000","last_undegraded":"2024-05-20T18:28:25.913425+0000","last_fullsized":"2024-05-20T18:28:25.913425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:43:52.611770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.913266+0000","last_change":"2024-05-20T18:28:24.918503+0000","last_active":"2024-05-20T18:28:25.913266+0000","last_peered":"2024-05-20T18:28:25.913266+0000","last_clean":"2024-05-20T18:28:25.913266+0000","last_became_active":"2024-05-20T18:28:24.918170+0000","last_became_peered":"2024-05-20T18:28:24.918170+0000","last_unstale":"2024-05-20T18:28:25.913266+0000","last_undegraded":"2024-05-20T18:28:25.913266+0000","last_fullsized":"2024-05-20T18:28:25.913266+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:46:00.161250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058004+0000","last_change":"2024-05-20T18:28:24.920538+0000","last_active":"2024-05-20T18:28:26.058004+0000","last_peered":"2024-05-20T18:28:26.058004+0000","last_clean":"2024-05-20T18:28:26.058004+0000","last_became_active":"2024-05-20T18:28:24.920217+0000","last_became_peered":"2024-05-20T18:28:24.920217+0000","last_unstale":"2024-05-20T18:28:26.058004+0000","last_undegraded":"2024-05-20T18:28:26.058004+0000","last_fullsized":"2024-05-20T18:28:26.058004+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:50:54.323312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.916072+0000","last_change":"2024-05-20T18:28:24.918154+0000","last_active":"2024-05-20T18:28:25.916072+0000","last_peered":"2024-05-20T18:28:25.916072+0000","last_clean":"2024-05-20T18:28:25.916072+0000","last_became_active":"2024-05-20T18:28:24.917825+0000","last_became_peered":"2024-05-20T18:28:24.917825+0000","last_unstale":"2024-05-20T18:28:25.916072+0000","last_undegraded":"2024-05-20T18:28:25.916072+0000","last_fullsized":"2024-05-20T18:28:25.916072+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:11:46.939770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.910898+0000","last_change":"2024-05-20T18:28:24.916670+0000","last_active":"2024-05-20T18:28:25.910898+0000","last_peered":"2024-05-20T18:28:25.910898+0000","last_clean":"2024-05-20T18:28:25.910898+0000","last_became_active":"2024-05-20T18:28:24.916345+0000","last_became_peered":"2024-05-20T18:28:24.916345+0000","last_unstale":"2024-05-20T18:28:25.910898+0000","last_undegraded":"2024-05-20T18:28:25.910898+0000","last_fullsized":"2024-05-20T18:28:25.910898+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:22:10.801800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.057961+0000","last_change":"2024-05-20T18:28:24.914944+0000","last_active":"2024-05-20T18:28:26.057961+0000","last_peered":"2024-05-20T18:28:26.057961+0000","last_clean":"2024-05-20T18:28:26.057961+0000","last_became_active":"2024-05-20T18:28:24.914400+0000","last_became_peered":"2024-05-20T18:28:24.914400+0000","last_unstale":"2024-05-20T18:28:26.057961+0000","last_undegraded":"2024-05-20T18:28:26.057961+0000","last_fullsized":"2024-05-20T18:28:26.057961+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:43:12.609193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.916880+0000","last_change":"2024-05-20T18:28:24.916672+0000","last_active":"2024-05-20T18:28:25.916880+0000","last_peered":"2024-05-20T18:28:25.916880+0000","last_clean":"2024-05-20T18:28:25.916880+0000","last_became_active":"2024-05-20T18:28:24.913171+0000","last_became_peered":"2024-05-20T18:28:24.913171+0000","last_unstale":"2024-05-20T18:28:25.916880+0000","last_undegraded":"2024-05-20T18:28:25.916880+0000","last_fullsized":"2024-05-20T18:28:25.916880+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:43:20.094774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.917161+0000","last_change":"2024-05-20T18:28:24.918125+0000","last_active":"2024-05-20T18:28:25.917161+0000","last_peered":"2024-05-20T18:28:25.917161+0000","last_clean":"2024-05-20T18:28:25.917161+0000","last_became_active":"2024-05-20T18:28:24.917782+0000","last_became_peered":"2024-05-20T18:28:24.917782+0000","last_unstale":"2024-05-20T18:28:25.917161+0000","last_undegraded":"2024-05-20T18:28:25.917161+0000","last_fullsized":"2024-05-20T18:28:25.917161+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:00:56.383920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.057708+0000","last_change":"2024-05-20T18:28:24.917036+0000","last_active":"2024-05-20T18:28:26.057708+0000","last_peered":"2024-05-20T18:28:26.057708+0000","last_clean":"2024-05-20T18:28:26.057708+0000","last_became_active":"2024-05-20T18:28:24.916642+0000","last_became_peered":"2024-05-20T18:28:24.916642+0000","last_unstale":"2024-05-20T18:28:26.057708+0000","last_undegraded":"2024-05-20T18:28:26.057708+0000","last_fullsized":"2024-05-20T18:28:26.057708+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:17:13.880808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.916456+0000","last_change":"2024-05-20T18:28:24.922811+0000","last_active":"2024-05-20T18:28:25.916456+0000","last_peered":"2024-05-20T18:28:25.916456+0000","last_clean":"2024-05-20T18:28:25.916456+0000","last_became_active":"2024-05-20T18:28:24.922479+0000","last_became_peered":"2024-05-20T18:28:24.922479+0000","last_unstale":"2024-05-20T18:28:25.916456+0000","last_undegraded":"2024-05-20T18:28:25.916456+0000","last_fullsized":"2024-05-20T18:28:25.916456+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:40:49.558398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.915861+0000","last_change":"2024-05-20T18:28:24.911637+0000","last_active":"2024-05-20T18:28:25.915861+0000","last_peered":"2024-05-20T18:28:25.915861+0000","last_clean":"2024-05-20T18:28:25.915861+0000","last_became_active":"2024-05-20T18:28:24.911164+0000","last_became_peered":"2024-05-20T18:28:24.911164+0000","last_unstale":"2024-05-20T18:28:25.915861+0000","last_undegraded":"2024-05-20T18:28:25.915861+0000","last_fullsized":"2024-05-20T18:28:25.915861+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:11:47.463102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.916507+0000","last_change":"2024-05-20T18:28:24.923073+0000","last_active":"2024-05-20T18:28:25.916507+0000","last_peered":"2024-05-20T18:28:25.916507+0000","last_clean":"2024-05-20T18:28:25.916507+0000","last_became_active":"2024-05-20T18:28:24.921935+0000","last_became_peered":"2024-05-20T18:28:24.921935+0000","last_unstale":"2024-05-20T18:28:25.916507+0000","last_undegraded":"2024-05-20T18:28:25.916507+0000","last_fullsized":"2024-05-20T18:28:25.916507+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:55:36.297837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.057900+0000","last_change":"2024-05-20T18:28:24.924179+0000","last_active":"2024-05-20T18:28:26.057900+0000","last_peered":"2024-05-20T18:28:26.057900+0000","last_clean":"2024-05-20T18:28:26.057900+0000","last_became_active":"2024-05-20T18:28:24.923839+0000","last_became_peered":"2024-05-20T18:28:24.923839+0000","last_unstale":"2024-05-20T18:28:26.057900+0000","last_undegraded":"2024-05-20T18:28:26.057900+0000","last_fullsized":"2024-05-20T18:28:26.057900+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:50:54.323312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058609+0000","last_change":"2024-05-20T18:28:24.923447+0000","last_active":"2024-05-20T18:28:26.058609+0000","last_peered":"2024-05-20T18:28:26.058609+0000","last_clean":"2024-05-20T18:28:26.058609+0000","last_became_active":"2024-05-20T18:28:24.923124+0000","last_became_peered":"2024-05-20T18:28:24.923124+0000","last_unstale":"2024-05-20T18:28:26.058609+0000","last_undegraded":"2024-05-20T18:28:26.058609+0000","last_fullsized":"2024-05-20T18:28:26.058609+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:27:23.187550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.918463+0000","last_change":"2024-05-20T18:28:24.913022+0000","last_active":"2024-05-20T18:28:25.918463+0000","last_peered":"2024-05-20T18:28:25.918463+0000","last_clean":"2024-05-20T18:28:25.918463+0000","last_became_active":"2024-05-20T18:28:24.912708+0000","last_became_peered":"2024-05-20T18:28:24.912708+0000","last_unstale":"2024-05-20T18:28:25.918463+0000","last_undegraded":"2024-05-20T18:28:25.918463+0000","last_fullsized":"2024-05-20T18:28:25.918463+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:32:35.134081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.913519+0000","last_change":"2024-05-20T18:28:24.914891+0000","last_active":"2024-05-20T18:28:25.913519+0000","last_peered":"2024-05-20T18:28:25.913519+0000","last_clean":"2024-05-20T18:28:25.913519+0000","last_became_active":"2024-05-20T18:28:24.914577+0000","last_became_peered":"2024-05-20T18:28:24.914577+0000","last_unstale":"2024-05-20T18:28:25.913519+0000","last_undegraded":"2024-05-20T18:28:25.913519+0000","last_fullsized":"2024-05-20T18:28:25.913519+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:39:42.086357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.918297+0000","last_change":"2024-05-20T18:28:24.909832+0000","last_active":"2024-05-20T18:28:25.918297+0000","last_peered":"2024-05-20T18:28:25.918297+0000","last_clean":"2024-05-20T18:28:25.918297+0000","last_became_active":"2024-05-20T18:28:24.909506+0000","last_became_peered":"2024-05-20T18:28:24.909506+0000","last_unstale":"2024-05-20T18:28:25.918297+0000","last_undegraded":"2024-05-20T18:28:25.918297+0000","last_fullsized":"2024-05-20T18:28:25.918297+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:27:02.499774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.916648+0000","last_change":"2024-05-20T18:28:24.914062+0000","last_active":"2024-05-20T18:28:25.916648+0000","last_peered":"2024-05-20T18:28:25.916648+0000","last_clean":"2024-05-20T18:28:25.916648+0000","last_became_active":"2024-05-20T18:28:24.913723+0000","last_became_peered":"2024-05-20T18:28:24.913723+0000","last_unstale":"2024-05-20T18:28:25.916648+0000","last_undegraded":"2024-05-20T18:28:25.916648+0000","last_fullsized":"2024-05-20T18:28:25.916648+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:02:17.059963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.916368+0000","last_change":"2024-05-20T18:28:24.915215+0000","last_active":"2024-05-20T18:28:25.916368+0000","last_peered":"2024-05-20T18:28:25.916368+0000","last_clean":"2024-05-20T18:28:25.916368+0000","last_became_active":"2024-05-20T18:28:24.914879+0000","last_became_peered":"2024-05-20T18:28:24.914879+0000","last_unstale":"2024-05-20T18:28:25.916368+0000","last_undegraded":"2024-05-20T18:28:25.916368+0000","last_fullsized":"2024-05-20T18:28:25.916368+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:17:13.880808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.057993+0000","last_change":"2024-05-20T18:28:24.924961+0000","last_active":"2024-05-20T18:28:26.057993+0000","last_peered":"2024-05-20T18:28:26.057993+0000","last_clean":"2024-05-20T18:28:26.057993+0000","last_became_active":"2024-05-20T18:28:24.924589+0000","last_became_peered":"2024-05-20T18:28:24.924589+0000","last_unstale":"2024-05-20T18:28:26.057993+0000","last_undegraded":"2024-05-20T18:28:26.057993+0000","last_fullsized":"2024-05-20T18:28:26.057993+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:17:13.880808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.913114+0000","last_change":"2024-05-20T18:28:24.917948+0000","last_active":"2024-05-20T18:28:25.913114+0000","last_peered":"2024-05-20T18:28:25.913114+0000","last_clean":"2024-05-20T18:28:25.913114+0000","last_became_active":"2024-05-20T18:28:24.917642+0000","last_became_peered":"2024-05-20T18:28:24.917642+0000","last_unstale":"2024-05-20T18:28:25.913114+0000","last_undegraded":"2024-05-20T18:28:25.913114+0000","last_fullsized":"2024-05-20T18:28:25.913114+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:52:18.085709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.918648+0000","last_change":"2024-05-20T18:28:24.916506+0000","last_active":"2024-05-20T18:28:25.918648+0000","last_peered":"2024-05-20T18:28:25.918648+0000","last_clean":"2024-05-20T18:28:25.918648+0000","last_became_active":"2024-05-20T18:28:24.916098+0000","last_became_peered":"2024-05-20T18:28:24.916098+0000","last_unstale":"2024-05-20T18:28:25.918648+0000","last_undegraded":"2024-05-20T18:28:25.918648+0000","last_fullsized":"2024-05-20T18:28:25.918648+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:35:24.942545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058384+0000","last_change":"2024-05-20T18:28:24.919467+0000","last_active":"2024-05-20T18:28:26.058384+0000","last_peered":"2024-05-20T18:28:26.058384+0000","last_clean":"2024-05-20T18:28:26.058384+0000","last_became_active":"2024-05-20T18:28:24.919105+0000","last_became_peered":"2024-05-20T18:28:24.919105+0000","last_unstale":"2024-05-20T18:28:26.058384+0000","last_undegraded":"2024-05-20T18:28:26.058384+0000","last_fullsized":"2024-05-20T18:28:26.058384+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:11:59.707265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.918481+0000","last_change":"2024-05-20T18:28:24.912602+0000","last_active":"2024-05-20T18:28:25.918481+0000","last_peered":"2024-05-20T18:28:25.918481+0000","last_clean":"2024-05-20T18:28:25.918481+0000","last_became_active":"2024-05-20T18:28:24.912228+0000","last_became_peered":"2024-05-20T18:28:24.912228+0000","last_unstale":"2024-05-20T18:28:25.918481+0000","last_undegraded":"2024-05-20T18:28:25.918481+0000","last_fullsized":"2024-05-20T18:28:25.918481+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:53:48.597638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.917148+0000","last_change":"2024-05-20T18:28:24.916263+0000","last_active":"2024-05-20T18:28:25.917148+0000","last_peered":"2024-05-20T18:28:25.917148+0000","last_clean":"2024-05-20T18:28:25.917148+0000","last_became_active":"2024-05-20T18:28:24.914253+0000","last_became_peered":"2024-05-20T18:28:24.914253+0000","last_unstale":"2024-05-20T18:28:25.917148+0000","last_undegraded":"2024-05-20T18:28:25.917148+0000","last_fullsized":"2024-05-20T18:28:25.917148+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:26:11.273568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.918344+0000","last_change":"2024-05-20T18:28:24.910327+0000","last_active":"2024-05-20T18:28:25.918344+0000","last_peered":"2024-05-20T18:28:25.918344+0000","last_clean":"2024-05-20T18:28:25.918344+0000","last_became_active":"2024-05-20T18:28:24.909988+0000","last_became_peered":"2024-05-20T18:28:24.909988+0000","last_unstale":"2024-05-20T18:28:25.918344+0000","last_undegraded":"2024-05-20T18:28:25.918344+0000","last_fullsized":"2024-05-20T18:28:25.918344+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:07:11.841682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.918184+0000","last_change":"2024-05-20T18:28:24.910812+0000","last_active":"2024-05-20T18:28:25.918184+0000","last_peered":"2024-05-20T18:28:25.918184+0000","last_clean":"2024-05-20T18:28:25.918184+0000","last_became_active":"2024-05-20T18:28:24.910471+0000","last_became_peered":"2024-05-20T18:28:24.910471+0000","last_unstale":"2024-05-20T18:28:25.918184+0000","last_undegraded":"2024-05-20T18:28:25.918184+0000","last_fullsized":"2024-05-20T18:28:25.918184+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:25:24.281662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058119+0000","last_change":"2024-05-20T18:28:24.926360+0000","last_active":"2024-05-20T18:28:26.058119+0000","last_peered":"2024-05-20T18:28:26.058119+0000","last_clean":"2024-05-20T18:28:26.058119+0000","last_became_active":"2024-05-20T18:28:24.925985+0000","last_became_peered":"2024-05-20T18:28:24.925985+0000","last_unstale":"2024-05-20T18:28:26.058119+0000","last_undegraded":"2024-05-20T18:28:26.058119+0000","last_fullsized":"2024-05-20T18:28:26.058119+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:11:59.707265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.057970+0000","last_change":"2024-05-20T18:28:24.911082+0000","last_active":"2024-05-20T18:28:26.057970+0000","last_peered":"2024-05-20T18:28:26.057970+0000","last_clean":"2024-05-20T18:28:26.057970+0000","last_became_active":"2024-05-20T18:28:24.910748+0000","last_became_peered":"2024-05-20T18:28:24.910748+0000","last_unstale":"2024-05-20T18:28:26.057970+0000","last_undegraded":"2024-05-20T18:28:26.057970+0000","last_fullsized":"2024-05-20T18:28:26.057970+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:11:47.463102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058266+0000","last_change":"2024-05-20T18:28:24.923491+0000","last_active":"2024-05-20T18:28:26.058266+0000","last_peered":"2024-05-20T18:28:26.058266+0000","last_clean":"2024-05-20T18:28:26.058266+0000","last_became_active":"2024-05-20T18:28:24.923111+0000","last_became_peered":"2024-05-20T18:28:24.923111+0000","last_unstale":"2024-05-20T18:28:26.058266+0000","last_undegraded":"2024-05-20T18:28:26.058266+0000","last_fullsized":"2024-05-20T18:28:26.058266+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:43:12.609193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.918609+0000","last_change":"2024-05-20T18:28:23.890621+0000","last_active":"2024-05-20T18:28:25.918609+0000","last_peered":"2024-05-20T18:28:25.918609+0000","last_clean":"2024-05-20T18:28:25.918609+0000","last_became_active":"2024-05-20T18:28:23.890215+0000","last_became_peered":"2024-05-20T18:28:23.890215+0000","last_unstale":"2024-05-20T18:28:25.918609+0000","last_undegraded":"2024-05-20T18:28:25.918609+0000","last_fullsized":"2024-05-20T18:28:25.918609+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:52:17.081034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058432+0000","last_change":"2024-05-20T18:28:24.920530+0000","last_active":"2024-05-20T18:28:26.058432+0000","last_peered":"2024-05-20T18:28:26.058432+0000","last_clean":"2024-05-20T18:28:26.058432+0000","last_became_active":"2024-05-20T18:28:24.920187+0000","last_became_peered":"2024-05-20T18:28:24.920187+0000","last_unstale":"2024-05-20T18:28:26.058432+0000","last_undegraded":"2024-05-20T18:28:26.058432+0000","last_fullsized":"2024-05-20T18:28:26.058432+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:22:53.319670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.057939+0000","last_change":"2024-05-20T18:28:23.901392+0000","last_active":"2024-05-20T18:28:26.057939+0000","last_peered":"2024-05-20T18:28:26.057939+0000","last_clean":"2024-05-20T18:28:26.057939+0000","last_became_active":"2024-05-20T18:28:23.901035+0000","last_became_peered":"2024-05-20T18:28:23.901035+0000","last_unstale":"2024-05-20T18:28:26.057939+0000","last_undegraded":"2024-05-20T18:28:26.057939+0000","last_fullsized":"2024-05-20T18:28:26.057939+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:43:51.607095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.917369+0000","last_change":"2024-05-20T18:28:24.912716+0000","last_active":"2024-05-20T18:28:25.917369+0000","last_peered":"2024-05-20T18:28:25.917369+0000","last_clean":"2024-05-20T18:28:25.917369+0000","last_became_active":"2024-05-20T18:28:24.912394+0000","last_became_peered":"2024-05-20T18:28:24.912394+0000","last_unstale":"2024-05-20T18:28:25.917369+0000","last_undegraded":"2024-05-20T18:28:25.917369+0000","last_fullsized":"2024-05-20T18:28:25.917369+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:43:12.609193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058486+0000","last_change":"2024-05-20T18:28:23.903027+0000","last_active":"2024-05-20T18:28:26.058486+0000","last_peered":"2024-05-20T18:28:26.058486+0000","last_clean":"2024-05-20T18:28:26.058486+0000","last_became_active":"2024-05-20T18:28:23.902322+0000","last_became_peered":"2024-05-20T18:28:23.902322+0000","last_unstale":"2024-05-20T18:28:26.058486+0000","last_undegraded":"2024-05-20T18:28:26.058486+0000","last_fullsized":"2024-05-20T18:28:26.058486+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:39:41.081682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.911597+0000","last_change":"2024-05-20T18:28:24.908701+0000","last_active":"2024-05-20T18:28:25.911597+0000","last_peered":"2024-05-20T18:28:25.911597+0000","last_clean":"2024-05-20T18:28:25.911597+0000","last_became_active":"2024-05-20T18:28:24.908438+0000","last_became_peered":"2024-05-20T18:28:24.908438+0000","last_unstale":"2024-05-20T18:28:25.911597+0000","last_undegraded":"2024-05-20T18:28:25.911597+0000","last_fullsized":"2024-05-20T18:28:25.911597+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:13:00.979960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.916835+0000","last_change":"2024-05-20T18:28:23.885560+0000","last_active":"2024-05-20T18:28:25.916835+0000","last_peered":"2024-05-20T18:28:25.916835+0000","last_clean":"2024-05-20T18:28:25.916835+0000","last_became_active":"2024-05-20T18:28:23.885157+0000","last_became_peered":"2024-05-20T18:28:23.885157+0000","last_unstale":"2024-05-20T18:28:25.916835+0000","last_undegraded":"2024-05-20T18:28:25.916835+0000","last_fullsized":"2024-05-20T18:28:25.916835+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:55:23.129684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.057786+0000","last_change":"2024-05-20T18:28:24.915518+0000","last_active":"2024-05-20T18:28:26.057786+0000","last_peered":"2024-05-20T18:28:26.057786+0000","last_clean":"2024-05-20T18:28:26.057786+0000","last_became_active":"2024-05-20T18:28:24.914816+0000","last_became_peered":"2024-05-20T18:28:24.914816+0000","last_unstale":"2024-05-20T18:28:26.057786+0000","last_undegraded":"2024-05-20T18:28:26.057786+0000","last_fullsized":"2024-05-20T18:28:26.057786+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:11:59.707265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.916704+0000","last_change":"2024-05-20T18:28:23.897800+0000","last_active":"2024-05-20T18:28:25.916704+0000","last_peered":"2024-05-20T18:28:25.916704+0000","last_clean":"2024-05-20T18:28:25.916704+0000","last_became_active":"2024-05-20T18:28:23.897473+0000","last_became_peered":"2024-05-20T18:28:23.897473+0000","last_unstale":"2024-05-20T18:28:25.916704+0000","last_undegraded":"2024-05-20T18:28:25.916704+0000","last_fullsized":"2024-05-20T18:28:25.916704+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:53:47.592963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.916291+0000","last_change":"2024-05-20T18:28:24.910962+0000","last_active":"2024-05-20T18:28:25.916291+0000","last_peered":"2024-05-20T18:28:25.916291+0000","last_clean":"2024-05-20T18:28:25.916291+0000","last_became_active":"2024-05-20T18:28:24.910471+0000","last_became_peered":"2024-05-20T18:28:24.910471+0000","last_unstale":"2024-05-20T18:28:25.916291+0000","last_undegraded":"2024-05-20T18:28:25.916291+0000","last_fullsized":"2024-05-20T18:28:25.916291+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:18:47.935602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058066+0000","last_change":"2024-05-20T18:28:23.898365+0000","last_active":"2024-05-20T18:28:26.058066+0000","last_peered":"2024-05-20T18:28:26.058066+0000","last_clean":"2024-05-20T18:28:26.058066+0000","last_became_active":"2024-05-20T18:28:23.897996+0000","last_became_peered":"2024-05-20T18:28:23.897996+0000","last_unstale":"2024-05-20T18:28:26.058066+0000","last_undegraded":"2024-05-20T18:28:26.058066+0000","last_fullsized":"2024-05-20T18:28:26.058066+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:45:59.156575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.916729+0000","last_change":"2024-05-20T18:28:24.918574+0000","last_active":"2024-05-20T18:28:25.916729+0000","last_peered":"2024-05-20T18:28:25.916729+0000","last_clean":"2024-05-20T18:28:25.916729+0000","last_became_active":"2024-05-20T18:28:24.918163+0000","last_became_peered":"2024-05-20T18:28:24.918163+0000","last_unstale":"2024-05-20T18:28:25.916729+0000","last_undegraded":"2024-05-20T18:28:25.916729+0000","last_fullsized":"2024-05-20T18:28:25.916729+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:11:59.707265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058382+0000","last_change":"2024-05-20T18:28:23.898055+0000","last_active":"2024-05-20T18:28:26.058382+0000","last_peered":"2024-05-20T18:28:26.058382+0000","last_clean":"2024-05-20T18:28:26.058382+0000","last_became_active":"2024-05-20T18:28:23.897583+0000","last_became_peered":"2024-05-20T18:28:23.897583+0000","last_unstale":"2024-05-20T18:28:26.058382+0000","last_undegraded":"2024-05-20T18:28:26.058382+0000","last_fullsized":"2024-05-20T18:28:26.058382+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:26:56.937862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.911028+0000","last_change":"2024-05-20T18:28:24.911448+0000","last_active":"2024-05-20T18:28:25.911028+0000","last_peered":"2024-05-20T18:28:25.911028+0000","last_clean":"2024-05-20T18:28:25.911028+0000","last_became_active":"2024-05-20T18:28:24.911103+0000","last_became_peered":"2024-05-20T18:28:24.911103+0000","last_unstale":"2024-05-20T18:28:25.911028+0000","last_undegraded":"2024-05-20T18:28:25.911028+0000","last_fullsized":"2024-05-20T18:28:25.911028+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:13:25.533126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.915693+0000","last_change":"2024-05-20T18:28:23.887743+0000","last_active":"2024-05-20T18:28:25.915693+0000","last_peered":"2024-05-20T18:28:25.915693+0000","last_clean":"2024-05-20T18:28:25.915693+0000","last_became_active":"2024-05-20T18:28:23.887384+0000","last_became_peered":"2024-05-20T18:28:23.887384+0000","last_unstale":"2024-05-20T18:28:25.915693+0000","last_undegraded":"2024-05-20T18:28:25.915693+0000","last_fullsized":"2024-05-20T18:28:25.915693+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:26:56.937862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.913388+0000","last_change":"2024-05-20T18:28:24.917239+0000","last_active":"2024-05-20T18:28:25.913388+0000","last_peered":"2024-05-20T18:28:25.913388+0000","last_clean":"2024-05-20T18:28:25.913388+0000","last_became_active":"2024-05-20T18:28:24.916648+0000","last_became_peered":"2024-05-20T18:28:24.916648+0000","last_unstale":"2024-05-20T18:28:25.913388+0000","last_undegraded":"2024-05-20T18:28:25.913388+0000","last_fullsized":"2024-05-20T18:28:25.913388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:26:57.942537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.911701+0000","last_change":"2024-05-20T18:28:23.895233+0000","last_active":"2024-05-20T18:28:25.911701+0000","last_peered":"2024-05-20T18:28:25.911701+0000","last_clean":"2024-05-20T18:28:25.911701+0000","last_became_active":"2024-05-20T18:28:23.894890+0000","last_became_peered":"2024-05-20T18:28:23.894890+0000","last_unstale":"2024-05-20T18:28:25.911701+0000","last_undegraded":"2024-05-20T18:28:25.911701+0000","last_fullsized":"2024-05-20T18:28:25.911701+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:40:48.553723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058464+0000","last_change":"2024-05-20T18:28:24.918894+0000","last_active":"2024-05-20T18:28:26.058464+0000","last_peered":"2024-05-20T18:28:26.058464+0000","last_clean":"2024-05-20T18:28:26.058464+0000","last_became_active":"2024-05-20T18:28:24.918100+0000","last_became_peered":"2024-05-20T18:28:24.918100+0000","last_unstale":"2024-05-20T18:28:26.058464+0000","last_undegraded":"2024-05-20T18:28:26.058464+0000","last_fullsized":"2024-05-20T18:28:26.058464+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:11:46.939770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.917307+0000","last_change":"2024-05-20T18:28:23.893401+0000","last_active":"2024-05-20T18:28:25.917307+0000","last_peered":"2024-05-20T18:28:25.917307+0000","last_clean":"2024-05-20T18:28:25.917307+0000","last_became_active":"2024-05-20T18:28:23.893018+0000","last_became_peered":"2024-05-20T18:28:23.893018+0000","last_unstale":"2024-05-20T18:28:25.917307+0000","last_undegraded":"2024-05-20T18:28:25.917307+0000","last_fullsized":"2024-05-20T18:28:25.917307+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:25:23.276987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058014+0000","last_change":"2024-05-20T18:28:24.915342+0000","last_active":"2024-05-20T18:28:26.058014+0000","last_peered":"2024-05-20T18:28:26.058014+0000","last_clean":"2024-05-20T18:28:26.058014+0000","last_became_active":"2024-05-20T18:28:24.914997+0000","last_became_peered":"2024-05-20T18:28:24.914997+0000","last_unstale":"2024-05-20T18:28:26.058014+0000","last_undegraded":"2024-05-20T18:28:26.058014+0000","last_fullsized":"2024-05-20T18:28:26.058014+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:46:07.023527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.917285+0000","last_change":"2024-05-20T18:28:23.898547+0000","last_active":"2024-05-20T18:28:25.917285+0000","last_peered":"2024-05-20T18:28:25.917285+0000","last_clean":"2024-05-20T18:28:25.917285+0000","last_became_active":"2024-05-20T18:28:23.898206+0000","last_became_peered":"2024-05-20T18:28:23.898206+0000","last_unstale":"2024-05-20T18:28:25.917285+0000","last_undegraded":"2024-05-20T18:28:25.917285+0000","last_fullsized":"2024-05-20T18:28:25.917285+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:32:34.129406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.916643+0000","last_change":"2024-05-20T18:28:24.910052+0000","last_active":"2024-05-20T18:28:25.916643+0000","last_peered":"2024-05-20T18:28:25.916643+0000","last_clean":"2024-05-20T18:28:25.916643+0000","last_became_active":"2024-05-20T18:28:24.909709+0000","last_became_peered":"2024-05-20T18:28:24.909709+0000","last_unstale":"2024-05-20T18:28:25.916643+0000","last_undegraded":"2024-05-20T18:28:25.916643+0000","last_fullsized":"2024-05-20T18:28:25.916643+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:33:28.408090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.911489+0000","last_change":"2024-05-20T18:28:23.901816+0000","last_active":"2024-05-20T18:28:25.911489+0000","last_peered":"2024-05-20T18:28:25.911489+0000","last_clean":"2024-05-20T18:28:25.911489+0000","last_became_active":"2024-05-20T18:28:23.901465+0000","last_became_peered":"2024-05-20T18:28:23.901465+0000","last_unstale":"2024-05-20T18:28:25.911489+0000","last_undegraded":"2024-05-20T18:28:25.911489+0000","last_fullsized":"2024-05-20T18:28:25.911489+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:27:22.182875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.057560+0000","last_change":"2024-05-20T18:28:24.915240+0000","last_active":"2024-05-20T18:28:26.057560+0000","last_peered":"2024-05-20T18:28:26.057560+0000","last_clean":"2024-05-20T18:28:26.057560+0000","last_became_active":"2024-05-20T18:28:24.914374+0000","last_became_peered":"2024-05-20T18:28:24.914374+0000","last_unstale":"2024-05-20T18:28:26.057560+0000","last_undegraded":"2024-05-20T18:28:26.057560+0000","last_fullsized":"2024-05-20T18:28:26.057560+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:11:47.463102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.911294+0000","last_change":"2024-05-20T18:28:23.893914+0000","last_active":"2024-05-20T18:28:25.911294+0000","last_peered":"2024-05-20T18:28:25.911294+0000","last_clean":"2024-05-20T18:28:25.911294+0000","last_became_active":"2024-05-20T18:28:23.893589+0000","last_became_peered":"2024-05-20T18:28:23.893589+0000","last_unstale":"2024-05-20T18:28:25.911294+0000","last_undegraded":"2024-05-20T18:28:25.911294+0000","last_fullsized":"2024-05-20T18:28:25.911294+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:18:46.930927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.910960+0000","last_change":"2024-05-20T18:28:24.922679+0000","last_active":"2024-05-20T18:28:25.910960+0000","last_peered":"2024-05-20T18:28:25.910960+0000","last_clean":"2024-05-20T18:28:25.910960+0000","last_became_active":"2024-05-20T18:28:24.922314+0000","last_became_peered":"2024-05-20T18:28:24.922314+0000","last_unstale":"2024-05-20T18:28:25.910960+0000","last_undegraded":"2024-05-20T18:28:25.910960+0000","last_fullsized":"2024-05-20T18:28:25.910960+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:51:45.606238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058130+0000","last_change":"2024-05-20T18:28:23.899428+0000","last_active":"2024-05-20T18:28:26.058130+0000","last_peered":"2024-05-20T18:28:26.058130+0000","last_clean":"2024-05-20T18:28:26.058130+0000","last_became_active":"2024-05-20T18:28:23.899078+0000","last_became_peered":"2024-05-20T18:28:23.899078+0000","last_unstale":"2024-05-20T18:28:26.058130+0000","last_undegraded":"2024-05-20T18:28:26.058130+0000","last_fullsized":"2024-05-20T18:28:26.058130+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:43:51.607095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058076+0000","last_change":"2024-05-20T18:28:24.919882+0000","last_active":"2024-05-20T18:28:26.058076+0000","last_peered":"2024-05-20T18:28:26.058076+0000","last_clean":"2024-05-20T18:28:26.058076+0000","last_became_active":"2024-05-20T18:28:24.919529+0000","last_became_peered":"2024-05-20T18:28:24.919529+0000","last_unstale":"2024-05-20T18:28:26.058076+0000","last_undegraded":"2024-05-20T18:28:26.058076+0000","last_fullsized":"2024-05-20T18:28:26.058076+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:33:28.408090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.917930+0000","last_change":"2024-05-20T18:28:23.891290+0000","last_active":"2024-05-20T18:28:25.917930+0000","last_peered":"2024-05-20T18:28:25.917930+0000","last_clean":"2024-05-20T18:28:25.917930+0000","last_became_active":"2024-05-20T18:28:23.890931+0000","last_became_peered":"2024-05-20T18:28:23.890931+0000","last_unstale":"2024-05-20T18:28:25.917930+0000","last_undegraded":"2024-05-20T18:28:25.917930+0000","last_fullsized":"2024-05-20T18:28:25.917930+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:04:14.459707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.911160+0000","last_change":"2024-05-20T18:28:24.921845+0000","last_active":"2024-05-20T18:28:25.911160+0000","last_peered":"2024-05-20T18:28:25.911160+0000","last_clean":"2024-05-20T18:28:25.911160+0000","last_became_active":"2024-05-20T18:28:24.921539+0000","last_became_peered":"2024-05-20T18:28:24.921539+0000","last_unstale":"2024-05-20T18:28:25.911160+0000","last_undegraded":"2024-05-20T18:28:25.911160+0000","last_fullsized":"2024-05-20T18:28:25.911160+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:20:49.429451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":22,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.057968+0000","last_change":"2024-05-20T18:28:23.899387+0000","last_active":"2024-05-20T18:28:26.057968+0000","last_peered":"2024-05-20T18:28:26.057968+0000","last_clean":"2024-05-20T18:28:26.057968+0000","last_became_active":"2024-05-20T18:28:23.898884+0000","last_became_peered":"2024-05-20T18:28:23.898884+0000","last_unstale":"2024-05-20T18:28:26.057968+0000","last_undegraded":"2024-05-20T18:28:26.057968+0000","last_fullsized":"2024-05-20T18:28:26.057968+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:50:09.909370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.911313+0000","last_change":"2024-05-20T18:28:24.908025+0000","last_active":"2024-05-20T18:28:25.911313+0000","last_peered":"2024-05-20T18:28:25.911313+0000","last_clean":"2024-05-20T18:28:25.911313+0000","last_became_active":"2024-05-20T18:28:24.907747+0000","last_became_peered":"2024-05-20T18:28:24.907747+0000","last_unstale":"2024-05-20T18:28:25.911313+0000","last_undegraded":"2024-05-20T18:28:25.911313+0000","last_fullsized":"2024-05-20T18:28:25.911313+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:17:36.795124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.917038+0000","last_change":"2024-05-20T18:28:23.897609+0000","last_active":"2024-05-20T18:28:25.917038+0000","last_peered":"2024-05-20T18:28:25.917038+0000","last_clean":"2024-05-20T18:28:25.917038+0000","last_became_active":"2024-05-20T18:28:23.897267+0000","last_became_peered":"2024-05-20T18:28:23.897267+0000","last_unstale":"2024-05-20T18:28:25.917038+0000","last_undegraded":"2024-05-20T18:28:25.917038+0000","last_fullsized":"2024-05-20T18:28:25.917038+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:55:23.129684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.917133+0000","last_change":"2024-05-20T18:28:24.926018+0000","last_active":"2024-05-20T18:28:25.917133+0000","last_peered":"2024-05-20T18:28:25.917133+0000","last_clean":"2024-05-20T18:28:25.917133+0000","last_became_active":"2024-05-20T18:28:24.925674+0000","last_became_peered":"2024-05-20T18:28:24.925674+0000","last_unstale":"2024-05-20T18:28:25.917133+0000","last_undegraded":"2024-05-20T18:28:25.917133+0000","last_fullsized":"2024-05-20T18:28:25.917133+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:49:05.545604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.917335+0000","last_change":"2024-05-20T18:28:23.895801+0000","last_active":"2024-05-20T18:28:25.917335+0000","last_peered":"2024-05-20T18:28:25.917335+0000","last_clean":"2024-05-20T18:28:25.917335+0000","last_became_active":"2024-05-20T18:28:23.895446+0000","last_became_peered":"2024-05-20T18:28:23.895446+0000","last_unstale":"2024-05-20T18:28:25.917335+0000","last_undegraded":"2024-05-20T18:28:25.917335+0000","last_fullsized":"2024-05-20T18:28:25.917335+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:43:51.607095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.917157+0000","last_change":"2024-05-20T18:28:24.905847+0000","last_active":"2024-05-20T18:28:25.917157+0000","last_peered":"2024-05-20T18:28:25.917157+0000","last_clean":"2024-05-20T18:28:25.917157+0000","last_became_active":"2024-05-20T18:28:24.905496+0000","last_became_peered":"2024-05-20T18:28:24.905496+0000","last_unstale":"2024-05-20T18:28:25.917157+0000","last_undegraded":"2024-05-20T18:28:25.917157+0000","last_fullsized":"2024-05-20T18:28:25.917157+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:22:53.319670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.913472+0000","last_change":"2024-05-20T18:28:23.899625+0000","last_active":"2024-05-20T18:28:25.913472+0000","last_peered":"2024-05-20T18:28:25.913472+0000","last_clean":"2024-05-20T18:28:25.913472+0000","last_became_active":"2024-05-20T18:28:23.899293+0000","last_became_peered":"2024-05-20T18:28:23.899293+0000","last_unstale":"2024-05-20T18:28:25.913472+0000","last_undegraded":"2024-05-20T18:28:25.913472+0000","last_fullsized":"2024-05-20T18:28:25.913472+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:09:39.031471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.913372+0000","last_change":"2024-05-20T18:28:24.918997+0000","last_active":"2024-05-20T18:28:25.913372+0000","last_peered":"2024-05-20T18:28:25.913372+0000","last_clean":"2024-05-20T18:28:25.913372+0000","last_became_active":"2024-05-20T18:28:24.918622+0000","last_became_peered":"2024-05-20T18:28:24.918622+0000","last_unstale":"2024-05-20T18:28:25.913372+0000","last_undegraded":"2024-05-20T18:28:25.913372+0000","last_fullsized":"2024-05-20T18:28:25.913372+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:50:10.914045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058199+0000","last_change":"2024-05-20T18:28:23.900260+0000","last_active":"2024-05-20T18:28:26.058199+0000","last_peered":"2024-05-20T18:28:26.058199+0000","last_clean":"2024-05-20T18:28:26.058199+0000","last_became_active":"2024-05-20T18:28:23.898939+0000","last_became_peered":"2024-05-20T18:28:23.898939+0000","last_unstale":"2024-05-20T18:28:26.058199+0000","last_undegraded":"2024-05-20T18:28:26.058199+0000","last_fullsized":"2024-05-20T18:28:26.058199+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:55:23.129684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058224+0000","last_change":"2024-05-20T18:28:24.926391+0000","last_active":"2024-05-20T18:28:26.058224+0000","last_peered":"2024-05-20T18:28:26.058224+0000","last_clean":"2024-05-20T18:28:26.058224+0000","last_became_active":"2024-05-20T18:28:24.926050+0000","last_became_peered":"2024-05-20T18:28:24.926050+0000","last_unstale":"2024-05-20T18:28:26.058224+0000","last_undegraded":"2024-05-20T18:28:26.058224+0000","last_fullsized":"2024-05-20T18:28:26.058224+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:22:53.319670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.913232+0000","last_change":"2024-05-20T18:28:23.896073+0000","last_active":"2024-05-20T18:28:25.913232+0000","last_peered":"2024-05-20T18:28:25.913232+0000","last_clean":"2024-05-20T18:28:25.913232+0000","last_became_active":"2024-05-20T18:28:23.895727+0000","last_became_peered":"2024-05-20T18:28:23.895727+0000","last_unstale":"2024-05-20T18:28:25.913232+0000","last_undegraded":"2024-05-20T18:28:25.913232+0000","last_fullsized":"2024-05-20T18:28:25.913232+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:02:30.183710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.057633+0000","last_change":"2024-05-20T18:28:24.914653+0000","last_active":"2024-05-20T18:28:26.057633+0000","last_peered":"2024-05-20T18:28:26.057633+0000","last_clean":"2024-05-20T18:28:26.057633+0000","last_became_active":"2024-05-20T18:28:24.914207+0000","last_became_peered":"2024-05-20T18:28:24.914207+0000","last_unstale":"2024-05-20T18:28:26.057633+0000","last_undegraded":"2024-05-20T18:28:26.057633+0000","last_fullsized":"2024-05-20T18:28:26.057633+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:50:54.323312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.913145+0000","last_change":"2024-05-20T18:28:23.887872+0000","last_active":"2024-05-20T18:28:25.913145+0000","last_peered":"2024-05-20T18:28:25.913145+0000","last_clean":"2024-05-20T18:28:25.913145+0000","last_became_active":"2024-05-20T18:28:23.887519+0000","last_became_peered":"2024-05-20T18:28:23.887519+0000","last_unstale":"2024-05-20T18:28:25.913145+0000","last_undegraded":"2024-05-20T18:28:25.913145+0000","last_fullsized":"2024-05-20T18:28:25.913145+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:38:07.068531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.916499+0000","last_change":"2024-05-20T18:28:24.916171+0000","last_active":"2024-05-20T18:28:25.916499+0000","last_peered":"2024-05-20T18:28:25.916499+0000","last_clean":"2024-05-20T18:28:25.916499+0000","last_became_active":"2024-05-20T18:28:24.913433+0000","last_became_peered":"2024-05-20T18:28:24.913433+0000","last_unstale":"2024-05-20T18:28:25.916499+0000","last_undegraded":"2024-05-20T18:28:25.916499+0000","last_fullsized":"2024-05-20T18:28:25.916499+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:07:28.502495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058153+0000","last_change":"2024-05-20T18:28:23.898491+0000","last_active":"2024-05-20T18:28:26.058153+0000","last_peered":"2024-05-20T18:28:26.058153+0000","last_clean":"2024-05-20T18:28:26.058153+0000","last_became_active":"2024-05-20T18:28:23.898167+0000","last_became_peered":"2024-05-20T18:28:23.898167+0000","last_unstale":"2024-05-20T18:28:26.058153+0000","last_undegraded":"2024-05-20T18:28:26.058153+0000","last_fullsized":"2024-05-20T18:28:26.058153+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:43:51.607095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.911214+0000","last_change":"2024-05-20T18:28:24.910463+0000","last_active":"2024-05-20T18:28:25.911214+0000","last_peered":"2024-05-20T18:28:25.911214+0000","last_clean":"2024-05-20T18:28:25.911214+0000","last_became_active":"2024-05-20T18:28:24.909954+0000","last_became_peered":"2024-05-20T18:28:24.909954+0000","last_unstale":"2024-05-20T18:28:25.911214+0000","last_undegraded":"2024-05-20T18:28:25.911214+0000","last_fullsized":"2024-05-20T18:28:25.911214+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:30:33.575837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.913050+0000","last_change":"2024-05-20T18:28:23.892500+0000","last_active":"2024-05-20T18:28:25.913050+0000","last_peered":"2024-05-20T18:28:25.913050+0000","last_clean":"2024-05-20T18:28:25.913050+0000","last_became_active":"2024-05-20T18:28:23.892137+0000","last_became_peered":"2024-05-20T18:28:23.892137+0000","last_unstale":"2024-05-20T18:28:25.913050+0000","last_undegraded":"2024-05-20T18:28:25.913050+0000","last_fullsized":"2024-05-20T18:28:25.913050+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:17:16.487528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058210+0000","last_change":"2024-05-20T18:28:24.918164+0000","last_active":"2024-05-20T18:28:26.058210+0000","last_peered":"2024-05-20T18:28:26.058210+0000","last_clean":"2024-05-20T18:28:26.058210+0000","last_became_active":"2024-05-20T18:28:24.917832+0000","last_became_peered":"2024-05-20T18:28:24.917832+0000","last_unstale":"2024-05-20T18:28:26.058210+0000","last_undegraded":"2024-05-20T18:28:26.058210+0000","last_fullsized":"2024-05-20T18:28:26.058210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:33:28.408090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.917206+0000","last_change":"2024-05-20T18:28:23.899440+0000","last_active":"2024-05-20T18:28:25.917206+0000","last_peered":"2024-05-20T18:28:25.917206+0000","last_clean":"2024-05-20T18:28:25.917206+0000","last_became_active":"2024-05-20T18:28:23.899129+0000","last_became_peered":"2024-05-20T18:28:23.899129+0000","last_unstale":"2024-05-20T18:28:25.917206+0000","last_undegraded":"2024-05-20T18:28:25.917206+0000","last_fullsized":"2024-05-20T18:28:25.917206+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:27:01.495099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058559+0000","last_change":"2024-05-20T18:28:24.919200+0000","last_active":"2024-05-20T18:28:26.058559+0000","last_peered":"2024-05-20T18:28:26.058559+0000","last_clean":"2024-05-20T18:28:26.058559+0000","last_became_active":"2024-05-20T18:28:24.918571+0000","last_became_peered":"2024-05-20T18:28:24.918571+0000","last_unstale":"2024-05-20T18:28:26.058559+0000","last_undegraded":"2024-05-20T18:28:26.058559+0000","last_fullsized":"2024-05-20T18:28:26.058559+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:18:47.935602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.915154+0000","last_change":"2024-05-20T18:28:23.893151+0000","last_active":"2024-05-20T18:28:25.915154+0000","last_peered":"2024-05-20T18:28:25.915154+0000","last_clean":"2024-05-20T18:28:25.915154+0000","last_became_active":"2024-05-20T18:28:23.892806+0000","last_became_peered":"2024-05-20T18:28:23.892806+0000","last_unstale":"2024-05-20T18:28:25.915154+0000","last_undegraded":"2024-05-20T18:28:25.915154+0000","last_fullsized":"2024-05-20T18:28:25.915154+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:03:58.193858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.911516+0000","last_change":"2024-05-20T18:28:24.910137+0000","last_active":"2024-05-20T18:28:25.911516+0000","last_peered":"2024-05-20T18:28:25.911516+0000","last_clean":"2024-05-20T18:28:25.911516+0000","last_became_active":"2024-05-20T18:28:24.909763+0000","last_became_peered":"2024-05-20T18:28:24.909763+0000","last_unstale":"2024-05-20T18:28:25.911516+0000","last_undegraded":"2024-05-20T18:28:25.911516+0000","last_fullsized":"2024-05-20T18:28:25.911516+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:37:34.646258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.057804+0000","last_change":"2024-05-20T18:28:23.898332+0000","last_active":"2024-05-20T18:28:26.057804+0000","last_peered":"2024-05-20T18:28:26.057804+0000","last_clean":"2024-05-20T18:28:26.057804+0000","last_became_active":"2024-05-20T18:28:23.897949+0000","last_became_peered":"2024-05-20T18:28:23.897949+0000","last_unstale":"2024-05-20T18:28:26.057804+0000","last_undegraded":"2024-05-20T18:28:26.057804+0000","last_fullsized":"2024-05-20T18:28:26.057804+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:26:56.937862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.057744+0000","last_change":"2024-05-20T18:28:24.917396+0000","last_active":"2024-05-20T18:28:26.057744+0000","last_peered":"2024-05-20T18:28:26.057744+0000","last_clean":"2024-05-20T18:28:26.057744+0000","last_became_active":"2024-05-20T18:28:24.917094+0000","last_became_peered":"2024-05-20T18:28:24.917094+0000","last_unstale":"2024-05-20T18:28:26.057744+0000","last_undegraded":"2024-05-20T18:28:26.057744+0000","last_fullsized":"2024-05-20T18:28:26.057744+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:33:28.408090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058081+0000","last_change":"2024-05-20T18:28:23.895060+0000","last_active":"2024-05-20T18:28:26.058081+0000","last_peered":"2024-05-20T18:28:26.058081+0000","last_clean":"2024-05-20T18:28:26.058081+0000","last_became_active":"2024-05-20T18:28:23.894690+0000","last_became_peered":"2024-05-20T18:28:23.894690+0000","last_unstale":"2024-05-20T18:28:26.058081+0000","last_undegraded":"2024-05-20T18:28:26.058081+0000","last_fullsized":"2024-05-20T18:28:26.058081+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:39:41.081682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058087+0000","last_change":"2024-05-20T18:28:24.914948+0000","last_active":"2024-05-20T18:28:26.058087+0000","last_peered":"2024-05-20T18:28:26.058087+0000","last_clean":"2024-05-20T18:28:26.058087+0000","last_became_active":"2024-05-20T18:28:24.914621+0000","last_became_peered":"2024-05-20T18:28:24.914621+0000","last_unstale":"2024-05-20T18:28:26.058087+0000","last_undegraded":"2024-05-20T18:28:26.058087+0000","last_fullsized":"2024-05-20T18:28:26.058087+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:17:13.880808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.918125+0000","last_change":"2024-05-20T18:28:23.897978+0000","last_active":"2024-05-20T18:28:25.918125+0000","last_peered":"2024-05-20T18:28:25.918125+0000","last_clean":"2024-05-20T18:28:25.918125+0000","last_became_active":"2024-05-20T18:28:23.897611+0000","last_became_peered":"2024-05-20T18:28:23.897611+0000","last_unstale":"2024-05-20T18:28:25.918125+0000","last_undegraded":"2024-05-20T18:28:25.918125+0000","last_fullsized":"2024-05-20T18:28:25.918125+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:21:52.115216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.916072+0000","last_change":"2024-05-20T18:28:24.914362+0000","last_active":"2024-05-20T18:28:25.916072+0000","last_peered":"2024-05-20T18:28:25.916072+0000","last_clean":"2024-05-20T18:28:25.916072+0000","last_became_active":"2024-05-20T18:28:24.914044+0000","last_became_peered":"2024-05-20T18:28:24.914044+0000","last_unstale":"2024-05-20T18:28:25.916072+0000","last_undegraded":"2024-05-20T18:28:25.916072+0000","last_fullsized":"2024-05-20T18:28:25.916072+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:50:54.323312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.915767+0000","last_change":"2024-05-20T18:28:23.889191+0000","last_active":"2024-05-20T18:28:25.915767+0000","last_peered":"2024-05-20T18:28:25.915767+0000","last_clean":"2024-05-20T18:28:25.915767+0000","last_became_active":"2024-05-20T18:28:23.888846+0000","last_became_peered":"2024-05-20T18:28:23.888846+0000","last_unstale":"2024-05-20T18:28:25.915767+0000","last_undegraded":"2024-05-20T18:28:25.915767+0000","last_fullsized":"2024-05-20T18:28:25.915767+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:39:41.081682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058439+0000","last_change":"2024-05-20T18:28:24.918976+0000","last_active":"2024-05-20T18:28:26.058439+0000","last_peered":"2024-05-20T18:28:26.058439+0000","last_clean":"2024-05-20T18:28:26.058439+0000","last_became_active":"2024-05-20T18:28:24.918545+0000","last_became_peered":"2024-05-20T18:28:24.918545+0000","last_unstale":"2024-05-20T18:28:26.058439+0000","last_undegraded":"2024-05-20T18:28:26.058439+0000","last_fullsized":"2024-05-20T18:28:26.058439+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:46:07.023527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.910931+0000","last_change":"2024-05-20T18:28:23.898195+0000","last_active":"2024-05-20T18:28:25.910931+0000","last_peered":"2024-05-20T18:28:25.910931+0000","last_clean":"2024-05-20T18:28:25.910931+0000","last_became_active":"2024-05-20T18:28:23.897826+0000","last_became_peered":"2024-05-20T18:28:23.897826+0000","last_unstale":"2024-05-20T18:28:25.910931+0000","last_undegraded":"2024-05-20T18:28:25.910931+0000","last_fullsized":"2024-05-20T18:28:25.910931+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:11:45.935095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:25.916373+0000","last_change":"2024-05-20T18:28:24.921307+0000","last_active":"2024-05-20T18:28:25.916373+0000","last_peered":"2024-05-20T18:28:25.916373+0000","last_clean":"2024-05-20T18:28:25.916373+0000","last_became_active":"2024-05-20T18:28:24.920842+0000","last_became_peered":"2024-05-20T18:28:24.920842+0000","last_unstale":"2024-05-20T18:28:25.916373+0000","last_undegraded":"2024-05-20T18:28:25.916373+0000","last_fullsized":"2024-05-20T18:28:25.916373+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:23.872103+0000","last_clean_scrub_stamp":"2024-05-20T18:28:23.872103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:27:23.187550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058437+0000","last_change":"2024-05-20T18:28:23.899326+0000","last_active":"2024-05-20T18:28:26.058437+0000","last_peered":"2024-05-20T18:28:26.058437+0000","last_clean":"2024-05-20T18:28:26.058437+0000","last_became_active":"2024-05-20T18:28:23.898977+0000","last_became_peered":"2024-05-20T18:28:23.898977+0000","last_unstale":"2024-05-20T18:28:26.058437+0000","last_undegraded":"2024-05-20T18:28:26.058437+0000","last_fullsized":"2024-05-20T18:28:26.058437+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:28:22.867428+0000","last_clean_scrub_stamp":"2024-05-20T18:28:22.867428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:26:56.937862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":205,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-20T18:28:26.058210+0000","last_change":"2024-05-20T18:27:03.765694+0000","last_active":"2024-05-20T18:28:26.058210+0000","last_peered":"2024-05-20T18:28:26.058210+0000","last_clean":"2024-05-20T18:28:26.058210+0000","last_became_active":"2024-05-20T18:27:03.764695+0000","last_became_peered":"2024-05-20T18:27:03.764695+0000","last_unstale":"2024-05-20T18:28:26.058210+0000","last_undegraded":"2024-05-20T18:28:26.058210+0000","last_fullsized":"2024-05-20T18:28:26.058210+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_clean_scrub_stamp":"2024-05-20T18:27:02.750853+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:30:04.407487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":787226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":202450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":195288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":202472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31656,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340184,"statfs":{"total":96636764160,"available":96604348416,"internally_reserved":0,"allocated":479232,"data_stored":203514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":454656,"data_stored":195246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":454656,"data_stored":195266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1060864,"data_stored":792270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T18:28:29.695 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-20T18:28:29.696 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-20T18:28:29.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.855+0000 7f0de2ee9640 1 Processor -- start 2024-05-20T18:28:29.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.855+0000 7f0de2ee9640 1 -- start start 2024-05-20T18:28:29.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.856+0000 7f0de2ee9640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0ddc071630 0x7f0ddc15f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:29.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.856+0000 7f0de2ee9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0ddc0cd1b0 0x7f0ddc06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:29.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.856+0000 7f0de2ee9640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ddc0ce2b0 con 0x7f0ddc070850 2024-05-20T18:28:29.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.856+0000 7f0de2ee9640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f0ddc15f790 con 0x7f0ddc0cd1b0 2024-05-20T18:28:29.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.856+0000 7f0de2ee9640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0ddc06e590 con 0x7f0ddc071630 2024-05-20T18:28:29.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.856+0000 7f0ddbfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0ddc0cd1b0 0x7f0ddc06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:29.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.856+0000 7f0de1486640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0ddc070850 0x7f0ddc070c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53742/0 (socket says 172.21.15.5:53742) 2024-05-20T18:28:29.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.857+0000 7f0de1486640 1 -- 172.21.15.5:0/3247263224 learned_addr learned my addr 172.21.15.5:0/3247263224 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:29.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.857+0000 7f0ddbfff640 1 --2- 172.21.15.5:0/3247263224 >> v2:172.21.15.5:3300/0 conn(0x7f0ddc0cd1b0 0x7f0ddc06d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51658/0 (socket says 172.21.15.5:51658) 2024-05-20T18:28:29.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.857+0000 7f0de0c85640 1 --2- 172.21.15.5:0/3247263224 >> v2:172.21.15.63:3300/0 conn(0x7f0ddc071630 0x7f0ddc15f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:29.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.857+0000 7f0ddbfff640 1 -- 172.21.15.5:0/3247263224 >> v2:172.21.15.63:3300/0 conn(0x7f0ddc071630 msgr2=0x7f0ddc15f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:29.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.857+0000 7f0ddbfff640 1 --2- 172.21.15.5:0/3247263224 >> v2:172.21.15.63:3300/0 conn(0x7f0ddc071630 0x7f0ddc15f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:29.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.857+0000 7f0ddbfff640 1 -- 172.21.15.5:0/3247263224 >> v1:172.21.15.5:6789/0 conn(0x7f0ddc070850 legacy=0x7f0ddc070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:29.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.857+0000 7f0ddbfff640 1 -- 172.21.15.5:0/3247263224 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ddc06cce0 con 0x7f0ddc0cd1b0 2024-05-20T18:28:29.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.857+0000 7f0ddb7fe640 1 -- 172.21.15.5:0/3247263224 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1779850044 0 0) 0x7f0ddc0ce2b0 con 0x7f0ddc070850 2024-05-20T18:28:29.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.857+0000 7f0de0c85640 1 --2- 172.21.15.5:0/3247263224 >> v2:172.21.15.63:3300/0 conn(0x7f0ddc071630 0x7f0ddc15f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:29.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.858+0000 7f0ddbfff640 1 --2- 172.21.15.5:0/3247263224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ddc0cd1b0 0x7f0ddc06d7c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc068510 tx=0x7f0dc802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7705c7a4dcd6e707 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:29.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.858+0000 7f0ddb7fe640 1 -- 172.21.15.5:0/3247263224 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8037070 con 0x7f0ddc0cd1b0 2024-05-20T18:28:29.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.858+0000 7f0ddb7fe640 1 -- 172.21.15.5:0/3247263224 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dc8032440 con 0x7f0ddc0cd1b0 2024-05-20T18:28:29.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.858+0000 7f0ddb7fe640 1 -- 172.21.15.5:0/3247263224 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc802ae60 con 0x7f0ddc0cd1b0 2024-05-20T18:28:29.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.858+0000 7f0de2ee9640 1 -- 172.21.15.5:0/3247263224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ddc0cd1b0 msgr2=0x7f0ddc06d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:29.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.858+0000 7f0de2ee9640 1 --2- 172.21.15.5:0/3247263224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ddc0cd1b0 0x7f0ddc06d7c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc068510 tx=0x7f0dc802a110 comp rx=0 tx=0).stop 2024-05-20T18:28:29.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.858+0000 7f0de2ee9640 1 -- 172.21.15.5:0/3247263224 shutdown_connections 2024-05-20T18:28:29.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.858+0000 7f0de2ee9640 1 --2- 172.21.15.5:0/3247263224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ddc0cd1b0 0x7f0ddc06d7c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:29.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.858+0000 7f0de2ee9640 1 --2- 172.21.15.5:0/3247263224 >> v2:172.21.15.63:3300/0 conn(0x7f0ddc071630 0x7f0ddc15f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:29.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.858+0000 7f0de2ee9640 1 -- 172.21.15.5:0/3247263224 >> 172.21.15.5:0/3247263224 conn(0x7f0ddc0d29d0 msgr2=0x7f0ddc0d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:29.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.859+0000 7f0de2ee9640 1 -- 172.21.15.5:0/3247263224 shutdown_connections 2024-05-20T18:28:29.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.859+0000 7f0de2ee9640 1 -- 172.21.15.5:0/3247263224 wait complete. 2024-05-20T18:28:29.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.860+0000 7f0de2ee9640 1 Processor -- start 2024-05-20T18:28:29.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.860+0000 7f0de2ee9640 1 -- start start 2024-05-20T18:28:29.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.862+0000 7f0de2ee9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ddc070850 0x7f0ddc1f2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:29.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.862+0000 7f0de2ee9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ddc071630 0x7f0ddc1f55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:29.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.862+0000 7f0de2ee9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ddc0cd1b0 0x7f0ddc0b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:29.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.862+0000 7f0de2ee9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0ddc06d2a0 con 0x7f0ddc071630 2024-05-20T18:28:29.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.862+0000 7f0de2ee9640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0ddc06c990 con 0x7f0ddc0cd1b0 2024-05-20T18:28:29.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.862+0000 7f0de2ee9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0ddc06dd00 con 0x7f0ddc070850 2024-05-20T18:28:29.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.862+0000 7f0ddbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ddc071630 0x7f0ddc1f55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:29.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.862+0000 7f0ddbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ddc071630 0x7f0ddc1f55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51664/0 (socket says 172.21.15.5:51664) 2024-05-20T18:28:29.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.862+0000 7f0ddbfff640 1 -- 172.21.15.5:0/863959147 learned_addr learned my addr 172.21.15.5:0/863959147 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:29.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.862+0000 7f0de1486640 1 --2- 172.21.15.5:0/863959147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ddc0cd1b0 0x7f0ddc0b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:29.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.862+0000 7f0de0c85640 1 --2- 172.21.15.5:0/863959147 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ddc070850 0x7f0ddc1f2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:29.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.862+0000 7f0ddbfff640 1 -- 172.21.15.5:0/863959147 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ddc070850 msgr2=0x7f0ddc1f2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:29.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.862+0000 7f0ddbfff640 1 --2- 172.21.15.5:0/863959147 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ddc070850 0x7f0ddc1f2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:29.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.862+0000 7f0ddbfff640 1 -- 172.21.15.5:0/863959147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ddc0cd1b0 msgr2=0x7f0ddc0b78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:29.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.862+0000 7f0ddbfff640 1 --2- 172.21.15.5:0/863959147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ddc0cd1b0 0x7f0ddc0b78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:29.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.862+0000 7f0ddbfff640 1 -- 172.21.15.5:0/863959147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ddc1f6610 con 0x7f0ddc071630 2024-05-20T18:28:29.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.863+0000 7f0ddbfff640 1 --2- 172.21.15.5:0/863959147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ddc071630 0x7f0ddc1f55a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009890 tx=0x7f0dc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:29.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.863+0000 7f0dd97fa640 1 -- 172.21.15.5:0/863959147 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8037040 con 0x7f0ddc071630 2024-05-20T18:28:29.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.863+0000 7f0dd97fa640 1 -- 172.21.15.5:0/863959147 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dc8042070 con 0x7f0ddc071630 2024-05-20T18:28:29.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.863+0000 7f0dd97fa640 1 -- 172.21.15.5:0/863959147 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8030bd0 con 0x7f0ddc071630 2024-05-20T18:28:29.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.863+0000 7f0de2ee9640 1 -- 172.21.15.5:0/863959147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ddc0b5610 con 0x7f0ddc071630 2024-05-20T18:28:29.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.863+0000 7f0de2ee9640 1 -- 172.21.15.5:0/863959147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ddc1f62d0 con 0x7f0ddc071630 2024-05-20T18:28:29.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.864+0000 7f0de2ee9640 1 -- 172.21.15.5:0/863959147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ddc1f6040 con 0x7f0ddc071630 2024-05-20T18:28:29.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.865+0000 7f0dd97fa640 1 -- 172.21.15.5:0/863959147 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0dc8047460 con 0x7f0ddc071630 2024-05-20T18:28:29.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.865+0000 7f0dd97fa640 1 --2- 172.21.15.5:0/863959147 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0dc006be60 0x7f0dc006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:29.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.865+0000 7f0de0c85640 1 --2- 172.21.15.5:0/863959147 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0dc006be60 0x7f0dc006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:29.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.865+0000 7f0dd97fa640 1 -- 172.21.15.5:0/863959147 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f0dc80abb20 con 0x7f0ddc071630 2024-05-20T18:28:29.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.866+0000 7f0de0c85640 1 --2- 172.21.15.5:0/863959147 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0dc006be60 0x7f0dc006e370 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0006e80 tx=0x7f0dd0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:29.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:29.870+0000 7f0dd97fa640 1 -- 172.21.15.5:0/863959147 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0dc8032e20 con 0x7f0ddc071630 2024-05-20T18:28:30.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.136+0000 7f0de2ee9640 1 -- 172.21.15.5:0/863959147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f0ddc075b00 con 0x7f0ddc071630 2024-05-20T18:28:30.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.139+0000 7f0dd97fa640 1 -- 172.21.15.5:0/863959147 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f0dc8079c60 con 0x7f0ddc071630 2024-05-20T18:28:30.138 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2024-05-20T18:28:30.138 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDFlUtmevaaIhAATIwyXzr0vGT38ZmcMqR2gA== 2024-05-20T18:28:30.138 INFO:teuthology.orchestra.run.smithi005.stdout: caps mds = "allow" 2024-05-20T18:28:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: caps mon = "allow r" 2024-05-20T18:28:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-20T18:28:30.139 INFO:teuthology.orchestra.run.smithi005.stderr:updated caps for client.0 2024-05-20T18:28:30.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.141+0000 7f0de2ee9640 1 -- 172.21.15.5:0/863959147 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0dc006be60 msgr2=0x7f0dc006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:30.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.141+0000 7f0de2ee9640 1 --2- 172.21.15.5:0/863959147 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0dc006be60 0x7f0dc006e370 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0006e80 tx=0x7f0dd0008040 comp rx=0 tx=0).stop 2024-05-20T18:28:30.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.141+0000 7f0de2ee9640 1 -- 172.21.15.5:0/863959147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ddc071630 msgr2=0x7f0ddc1f55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:30.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.141+0000 7f0de2ee9640 1 --2- 172.21.15.5:0/863959147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ddc071630 0x7f0ddc1f55a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009890 tx=0x7f0dc80046c0 comp rx=0 tx=0).stop 2024-05-20T18:28:30.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.141+0000 7f0de2ee9640 1 -- 172.21.15.5:0/863959147 shutdown_connections 2024-05-20T18:28:30.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.141+0000 7f0de2ee9640 1 --2- 172.21.15.5:0/863959147 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0dc006be60 0x7f0dc006e370 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:30.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.141+0000 7f0de2ee9640 1 --2- 172.21.15.5:0/863959147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ddc0cd1b0 0x7f0ddc0b78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:30.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.141+0000 7f0de2ee9640 1 --2- 172.21.15.5:0/863959147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ddc071630 0x7f0ddc1f55a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:30.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.141+0000 7f0de2ee9640 1 --2- 172.21.15.5:0/863959147 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ddc070850 0x7f0ddc1f2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:30.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.141+0000 7f0de2ee9640 1 -- 172.21.15.5:0/863959147 >> 172.21.15.5:0/863959147 conn(0x7f0ddc0d29d0 msgr2=0x7f0ddc0b2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:30.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.141+0000 7f0de2ee9640 1 -- 172.21.15.5:0/863959147 shutdown_connections 2024-05-20T18:28:30.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.141+0000 7f0de2ee9640 1 -- 172.21.15.5:0/863959147 wait complete. 2024-05-20T18:28:30.152 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:28:30.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.268+0000 7fa09425d640 1 Processor -- start 2024-05-20T18:28:30.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.268+0000 7fa09425d640 1 -- start start 2024-05-20T18:28:30.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.269+0000 7fa09425d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa08c162820 0x7fa08c0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:30.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.269+0000 7fa09425d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa08c0b3e80 0x7fa08c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.269+0000 7fa09425d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa08c06dd20 con 0x7fa08c06d910 2024-05-20T18:28:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.269+0000 7fa09425d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fa08c0d0630 con 0x7fa08c162820 2024-05-20T18:28:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.269+0000 7fa09425d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa08c0d39c0 con 0x7fa08c0b3e80 2024-05-20T18:28:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.270+0000 7fa091ff9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa08c162820 0x7fa08c0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.270+0000 7fa091ff9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa08c162820 0x7fa08c0b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51672/0 (socket says 172.21.15.5:51672) 2024-05-20T18:28:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.270+0000 7fa0927fa640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa08c06d910 0x7fa08c0d27a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53754/0 (socket says 172.21.15.5:53754) 2024-05-20T18:28:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.270+0000 7fa091ff9640 1 -- 172.21.15.5:0/4010092256 learned_addr learned my addr 172.21.15.5:0/4010092256 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.270+0000 7fa0917f8640 1 --2- 172.21.15.5:0/4010092256 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa08c0b3e80 0x7fa08c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.270+0000 7fa091ff9640 1 -- 172.21.15.5:0/4010092256 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa08c0b3e80 msgr2=0x7fa08c06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.270+0000 7fa091ff9640 1 --2- 172.21.15.5:0/4010092256 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa08c0b3e80 0x7fa08c06d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.270+0000 7fa090ff7640 1 -- 172.21.15.5:0/4010092256 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3068377081 0 0) 0x7fa084001490 con 0x7fa08c06d910 2024-05-20T18:28:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.270+0000 7fa091ff9640 1 -- 172.21.15.5:0/4010092256 >> v1:172.21.15.5:6789/0 conn(0x7fa08c06d910 legacy=0x7fa08c0d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:30.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.270+0000 7fa091ff9640 1 -- 172.21.15.5:0/4010092256 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08c0d3ca0 con 0x7fa08c162820 2024-05-20T18:28:30.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.270+0000 7fa0917f8640 1 --2- 172.21.15.5:0/4010092256 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa08c0b3e80 0x7fa08c06d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:30.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.271+0000 7fa091ff9640 1 --2- 172.21.15.5:0/4010092256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa08c162820 0x7fa08c0b3940 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa080009870 tx=0x7fa08002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2c4397a163c678 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:30.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.271+0000 7fa090ff7640 1 -- 172.21.15.5:0/4010092256 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa080037070 con 0x7fa08c162820 2024-05-20T18:28:30.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.271+0000 7fa090ff7640 1 -- 172.21.15.5:0/4010092256 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa080032440 con 0x7fa08c162820 2024-05-20T18:28:30.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.271+0000 7fa090ff7640 1 -- 172.21.15.5:0/4010092256 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa080032810 con 0x7fa08c162820 2024-05-20T18:28:30.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.272+0000 7fa09425d640 1 -- 172.21.15.5:0/4010092256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa08c162820 msgr2=0x7fa08c0b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:30.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.272+0000 7fa09425d640 1 --2- 172.21.15.5:0/4010092256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa08c162820 0x7fa08c0b3940 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa080009870 tx=0x7fa08002a270 comp rx=0 tx=0).stop 2024-05-20T18:28:30.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.272+0000 7fa09425d640 1 -- 172.21.15.5:0/4010092256 shutdown_connections 2024-05-20T18:28:30.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.272+0000 7fa09425d640 1 --2- 172.21.15.5:0/4010092256 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa08c0b3e80 0x7fa08c06d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:30.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.272+0000 7fa09425d640 1 --2- 172.21.15.5:0/4010092256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa08c162820 0x7fa08c0b3940 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:30.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.272+0000 7fa09425d640 1 -- 172.21.15.5:0/4010092256 >> 172.21.15.5:0/4010092256 conn(0x7fa08c0d4bd0 msgr2=0x7fa08c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:30.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.272+0000 7fa09425d640 1 -- 172.21.15.5:0/4010092256 shutdown_connections 2024-05-20T18:28:30.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.272+0000 7fa09425d640 1 -- 172.21.15.5:0/4010092256 wait complete. 2024-05-20T18:28:30.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.274+0000 7fa09425d640 1 Processor -- start 2024-05-20T18:28:30.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.274+0000 7fa09425d640 1 -- start start 2024-05-20T18:28:30.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.275+0000 7fa09425d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa08c06d910 0x7fa08c157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:30.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.275+0000 7fa09425d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa08c0b3e80 0x7fa08c14b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:30.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.275+0000 7fa09425d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa08c162820 0x7fa08c15e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:30.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.275+0000 7fa09425d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c0d0300 con 0x7fa08c0b3e80 2024-05-20T18:28:30.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.275+0000 7fa09425d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c0d4260 con 0x7fa08c06d910 2024-05-20T18:28:30.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.275+0000 7fa09425d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa08c0b91a0 con 0x7fa08c162820 2024-05-20T18:28:30.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.275+0000 7fa0917f8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa08c0b3e80 0x7fa08c14b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:30.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.275+0000 7fa0917f8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa08c0b3e80 0x7fa08c14b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51684/0 (socket says 172.21.15.5:51684) 2024-05-20T18:28:30.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.275+0000 7fa0917f8640 1 -- 172.21.15.5:0/127611644 learned_addr learned my addr 172.21.15.5:0/127611644 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:30.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.275+0000 7fa0927fa640 1 --2- 172.21.15.5:0/127611644 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa08c162820 0x7fa08c15e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:30.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.275+0000 7fa0917f8640 1 -- 172.21.15.5:0/127611644 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa08c162820 msgr2=0x7fa08c15e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:30.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.275+0000 7fa091ff9640 1 --2- 172.21.15.5:0/127611644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa08c06d910 0x7fa08c157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:30.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.275+0000 7fa0917f8640 1 --2- 172.21.15.5:0/127611644 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa08c162820 0x7fa08c15e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:30.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.275+0000 7fa0917f8640 1 -- 172.21.15.5:0/127611644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa08c06d910 msgr2=0x7fa08c157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:30.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.275+0000 7fa0917f8640 1 --2- 172.21.15.5:0/127611644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa08c06d910 0x7fa08c157ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:30.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.275+0000 7fa0917f8640 1 -- 172.21.15.5:0/127611644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08c14b610 con 0x7fa08c0b3e80 2024-05-20T18:28:30.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.276+0000 7fa0917f8640 1 --2- 172.21.15.5:0/127611644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa08c0b3e80 0x7fa08c14b0d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa08800c820 tx=0x7fa08800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:30.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.276+0000 7fa07e7fc640 1 -- 172.21.15.5:0/127611644 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa088015070 con 0x7fa08c0b3e80 2024-05-20T18:28:30.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.276+0000 7fa07e7fc640 1 -- 172.21.15.5:0/127611644 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa088007a30 con 0x7fa08c0b3e80 2024-05-20T18:28:30.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.276+0000 7fa07e7fc640 1 -- 172.21.15.5:0/127611644 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa088007de0 con 0x7fa08c0b3e80 2024-05-20T18:28:30.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.276+0000 7fa09425d640 1 -- 172.21.15.5:0/127611644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa08c158c80 con 0x7fa08c0b3e80 2024-05-20T18:28:30.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.276+0000 7fa09425d640 1 -- 172.21.15.5:0/127611644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa08c0c89e0 con 0x7fa08c0b3e80 2024-05-20T18:28:30.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.277+0000 7fa09425d640 1 -- 172.21.15.5:0/127611644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa08c075b00 con 0x7fa08c0b3e80 2024-05-20T18:28:30.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.278+0000 7fa07e7fc640 1 -- 172.21.15.5:0/127611644 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa088018070 con 0x7fa08c0b3e80 2024-05-20T18:28:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.278+0000 7fa07e7fc640 1 --2- 172.21.15.5:0/127611644 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa07006be60 0x7fa07006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.278+0000 7fa07e7fc640 1 -- 172.21.15.5:0/127611644 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fa08808aee0 con 0x7fa08c0b3e80 2024-05-20T18:28:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.278+0000 7fa091ff9640 1 --2- 172.21.15.5:0/127611644 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa07006be60 0x7fa07006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.279+0000 7fa091ff9640 1 --2- 172.21.15.5:0/127611644 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa07006be60 0x7fa07006e370 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa080034000 tx=0x7fa08002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.283+0000 7fa07e7fc640 1 -- 172.21.15.5:0/127611644 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa088059670 con 0x7fa08c0b3e80 2024-05-20T18:28:30.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.503+0000 7fa09425d640 1 -- 172.21.15.5:0/127611644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa08c067c70 con 0x7fa08c0b3e80 2024-05-20T18:28:30.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.504+0000 7fa07e7fc640 1 -- 172.21.15.5:0/127611644 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5175 (secure 0 0 0) 0x7fa088058f70 con 0x7fa08c0b3e80 2024-05-20T18:28:30.503 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:30.503 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1458170078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1458170078},{"type":"v1","addr":"172.21.15.63:6834","nonce":1458170078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4916,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/2601750663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":2601750663},{"type":"v1","addr":"172.21.15.63:6835","nonce":2601750663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5313,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/1913030219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1913030219},{"type":"v1","addr":"172.21.15.5:6836","nonce":1913030219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:28:24.270282+0000","modified":"2024-05-20T18:28:27.296534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5316},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5316":{"gid":5316,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6837/2355805105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":2355805105},{"type":"v1","addr":"172.21.15.5:6837","nonce":2355805105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5316,"qdb_cluster":[5316]},"id":2}]} 2024-05-20T18:28:30.504 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 17 2024-05-20T18:28:30.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.506+0000 7fa09425d640 1 -- 172.21.15.5:0/127611644 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa07006be60 msgr2=0x7fa07006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:30.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.506+0000 7fa09425d640 1 --2- 172.21.15.5:0/127611644 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa07006be60 0x7fa07006e370 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa080034000 tx=0x7fa08002e000 comp rx=0 tx=0).stop 2024-05-20T18:28:30.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.506+0000 7fa09425d640 1 -- 172.21.15.5:0/127611644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa08c0b3e80 msgr2=0x7fa08c14b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:30.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.506+0000 7fa09425d640 1 --2- 172.21.15.5:0/127611644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa08c0b3e80 0x7fa08c14b0d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa08800c820 tx=0x7fa08800cce0 comp rx=0 tx=0).stop 2024-05-20T18:28:30.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.506+0000 7fa09425d640 1 -- 172.21.15.5:0/127611644 shutdown_connections 2024-05-20T18:28:30.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.506+0000 7fa09425d640 1 --2- 172.21.15.5:0/127611644 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa07006be60 0x7fa07006e370 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:30.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.506+0000 7fa09425d640 1 --2- 172.21.15.5:0/127611644 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa08c162820 0x7fa08c15e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:30.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.506+0000 7fa09425d640 1 --2- 172.21.15.5:0/127611644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa08c0b3e80 0x7fa08c14b0d0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:30.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.506+0000 7fa09425d640 1 --2- 172.21.15.5:0/127611644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa08c06d910 0x7fa08c157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:30.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.506+0000 7fa09425d640 1 -- 172.21.15.5:0/127611644 >> 172.21.15.5:0/127611644 conn(0x7fa08c0d4bd0 msgr2=0x7fa08c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:30.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.506+0000 7fa09425d640 1 -- 172.21.15.5:0/127611644 shutdown_connections 2024-05-20T18:28:30.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:30.506+0000 7fa09425d640 1 -- 172.21.15.5:0/127611644 wait complete. 2024-05-20T18:28:30.920 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T18:28:24.270282+0000', 'modified': '2024-05-20T18:28:27.296534+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5316}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5316': {'gid': 5316, 'name': 'c', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.5:6837/2355805105', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6835', 'nonce': 2355805105}, {'type': 'v1', 'addr': '172.21.15.5:6837', 'nonce': 2355805105}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5316, 'qdb_cluster': [5316]} 2024-05-20T18:28:30.920 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-20T18:28:30.920 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-20T18:28:31.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.041+0000 7f7fdb880640 1 Processor -- start 2024-05-20T18:28:31.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.041+0000 7f7fdb880640 1 -- start start 2024-05-20T18:28:31.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.043+0000 7f7fdb880640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7fd406d3d0 0x7f7fd406d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:31.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.043+0000 7f7fdb880640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fd40b8190 0x7f7fd40b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:31.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.043+0000 7f7fdb880640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fd406dd20 con 0x7f7fd4162820 2024-05-20T18:28:31.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.043+0000 7f7fdb880640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7fd40d0630 con 0x7f7fd406d3d0 2024-05-20T18:28:31.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.043+0000 7f7fdb880640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7fd40d39c0 con 0x7f7fd40b8190 2024-05-20T18:28:31.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.043+0000 7f7fd961c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7fd4162820 0x7f7fd40b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53764/0 (socket says 172.21.15.5:53764) 2024-05-20T18:28:31.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.043+0000 7f7fd961c640 1 -- 172.21.15.5:0/1511687716 learned_addr learned my addr 172.21.15.5:0/1511687716 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:31.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.043+0000 7f7fd8e1b640 1 --2- 172.21.15.5:0/1511687716 >> v2:172.21.15.63:3300/0 conn(0x7f7fd406d3d0 0x7f7fd406d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:31.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.043+0000 7f7fd9e1d640 1 --2- 172.21.15.5:0/1511687716 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fd40b8190 0x7f7fd40b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:31.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.043+0000 7f7fd3fff640 1 -- 172.21.15.5:0/1511687716 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1842722320 0 0) 0x7f7fc4001110 con 0x7f7fd4162820 2024-05-20T18:28:31.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.043+0000 7f7fd3fff640 1 -- 172.21.15.5:0/1511687716 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fbc003520 con 0x7f7fd4162820 2024-05-20T18:28:31.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.044+0000 7f7fd3fff640 1 -- 172.21.15.5:0/1511687716 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 926641468 0 0) 0x7f7fbc003520 con 0x7f7fd4162820 2024-05-20T18:28:31.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.044+0000 7f7fd8e1b640 1 -- 172.21.15.5:0/1511687716 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fd40b8190 msgr2=0x7f7fd40b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:31.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.044+0000 7f7fd8e1b640 1 --2- 172.21.15.5:0/1511687716 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fd40b8190 0x7f7fd40b85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.044+0000 7f7fd8e1b640 1 -- 172.21.15.5:0/1511687716 >> v1:172.21.15.5:6789/0 conn(0x7f7fd4162820 legacy=0x7f7fd40b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:31.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.044+0000 7f7fd8e1b640 1 -- 172.21.15.5:0/1511687716 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd40d3ca0 con 0x7f7fd406d3d0 2024-05-20T18:28:31.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.044+0000 7f7fd9e1d640 1 --2- 172.21.15.5:0/1511687716 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fd40b8190 0x7f7fd40b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:28:31.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.044+0000 7f7fd8e1b640 1 --2- 172.21.15.5:0/1511687716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fd406d3d0 0x7f7fd406d7e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7fc80097a0 tx=0x7f7fc802a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7561df9783b6e8f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:31.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.045+0000 7f7fd3fff640 1 -- 172.21.15.5:0/1511687716 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8037070 con 0x7f7fd406d3d0 2024-05-20T18:28:31.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.045+0000 7f7fd3fff640 1 -- 172.21.15.5:0/1511687716 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fc802aba0 con 0x7f7fd406d3d0 2024-05-20T18:28:31.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.045+0000 7f7fd3fff640 1 -- 172.21.15.5:0/1511687716 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4001840 con 0x7f7fd406d3d0 2024-05-20T18:28:31.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.045+0000 7f7fdb880640 1 -- 172.21.15.5:0/1511687716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fd406d3d0 msgr2=0x7f7fd406d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:31.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.045+0000 7f7fdb880640 1 --2- 172.21.15.5:0/1511687716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fd406d3d0 0x7f7fd406d7e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7fc80097a0 tx=0x7f7fc802a110 comp rx=0 tx=0).stop 2024-05-20T18:28:31.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.045+0000 7f7fdb880640 1 -- 172.21.15.5:0/1511687716 shutdown_connections 2024-05-20T18:28:31.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.045+0000 7f7fdb880640 1 --2- 172.21.15.5:0/1511687716 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fd40b8190 0x7f7fd40b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.045+0000 7f7fdb880640 1 --2- 172.21.15.5:0/1511687716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fd406d3d0 0x7f7fd406d7e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.045+0000 7f7fdb880640 1 -- 172.21.15.5:0/1511687716 >> 172.21.15.5:0/1511687716 conn(0x7f7fd40d4bd0 msgr2=0x7f7fd40d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:31.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.045+0000 7f7fdb880640 1 -- 172.21.15.5:0/1511687716 shutdown_connections 2024-05-20T18:28:31.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.045+0000 7f7fdb880640 1 -- 172.21.15.5:0/1511687716 wait complete. 2024-05-20T18:28:31.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.047+0000 7f7fdb880640 1 Processor -- start 2024-05-20T18:28:31.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.047+0000 7f7fdb880640 1 -- start start 2024-05-20T18:28:31.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.049+0000 7f7fdb880640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fd406d3d0 0x7f7fd40cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:31.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.049+0000 7f7fdb880640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fd40b8190 0x7f7fd40ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:31.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.049+0000 7f7fdb880640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fd4162820 0x7f7fd40ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:31.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.049+0000 7f7fdb880640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd40d0300 con 0x7f7fd4162820 2024-05-20T18:28:31.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.049+0000 7f7fdb880640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd40d4260 con 0x7f7fd40b8190 2024-05-20T18:28:31.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.049+0000 7f7fdb880640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7fd40b4050 con 0x7f7fd406d3d0 2024-05-20T18:28:31.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.049+0000 7f7fd9e1d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fd4162820 0x7f7fd40ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:31.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.049+0000 7f7fd9e1d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fd4162820 0x7f7fd40ce7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51700/0 (socket says 172.21.15.5:51700) 2024-05-20T18:28:31.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.049+0000 7f7fd9e1d640 1 -- 172.21.15.5:0/847816546 learned_addr learned my addr 172.21.15.5:0/847816546 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:31.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.049+0000 7f7fd8e1b640 1 --2- 172.21.15.5:0/847816546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fd40b8190 0x7f7fd40ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:31.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.049+0000 7f7fd961c640 1 --2- 172.21.15.5:0/847816546 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fd406d3d0 0x7f7fd40cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:31.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.049+0000 7f7fd9e1d640 1 -- 172.21.15.5:0/847816546 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fd406d3d0 msgr2=0x7f7fd40cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:31.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.049+0000 7f7fd9e1d640 1 --2- 172.21.15.5:0/847816546 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fd406d3d0 0x7f7fd40cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.049+0000 7f7fd9e1d640 1 -- 172.21.15.5:0/847816546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fd40b8190 msgr2=0x7f7fd40ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:31.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.049+0000 7f7fd9e1d640 1 --2- 172.21.15.5:0/847816546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fd40b8190 0x7f7fd40ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.049+0000 7f7fd9e1d640 1 -- 172.21.15.5:0/847816546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd40cf2a0 con 0x7f7fd4162820 2024-05-20T18:28:31.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.049+0000 7f7fd8e1b640 1 --2- 172.21.15.5:0/847816546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fd40b8190 0x7f7fd40ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:31.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.049+0000 7f7fd9e1d640 1 --2- 172.21.15.5:0/847816546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fd4162820 0x7f7fd40ce7e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc00efc0 tx=0x7f7fcc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:31.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.049+0000 7f7fd1ffb640 1 -- 172.21.15.5:0/847816546 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc018020 con 0x7f7fd4162820 2024-05-20T18:28:31.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.049+0000 7f7fd1ffb640 1 -- 172.21.15.5:0/847816546 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fcc014480 con 0x7f7fd4162820 2024-05-20T18:28:31.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.049+0000 7f7fdb880640 1 -- 172.21.15.5:0/847816546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd40cd200 con 0x7f7fd4162820 2024-05-20T18:28:31.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.050+0000 7f7fd1ffb640 1 -- 172.21.15.5:0/847816546 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc014870 con 0x7f7fd4162820 2024-05-20T18:28:31.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.050+0000 7f7fdb880640 1 -- 172.21.15.5:0/847816546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd40cd7c0 con 0x7f7fd4162820 2024-05-20T18:28:31.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.051+0000 7f7fd1ffb640 1 -- 172.21.15.5:0/847816546 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7fcc005d10 con 0x7f7fd4162820 2024-05-20T18:28:31.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.052+0000 7f7fd1ffb640 1 --2- 172.21.15.5:0/847816546 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7fc006be30 0x7f7fc006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:31.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.052+0000 7f7fd1ffb640 1 -- 172.21.15.5:0/847816546 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f7fcc08bd10 con 0x7f7fd4162820 2024-05-20T18:28:31.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.052+0000 7f7fd961c640 1 --2- 172.21.15.5:0/847816546 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7fc006be30 0x7f7fc006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:31.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.052+0000 7f7fd961c640 1 --2- 172.21.15.5:0/847816546 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7fc006be30 0x7f7fc006e340 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4005880 tx=0x7f7fc4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:31.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.053+0000 7f7fe27fd740 1 Processor -- start 2024-05-20T18:28:31.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.053+0000 7f7fe27fd740 1 -- start start 2024-05-20T18:28:31.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.054+0000 7f7fe27fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e63098270 0x556e63463700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:31.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.054+0000 7f7fe27fd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6306f3b0 0x556e634600a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:31.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.054+0000 7f7fe27fd740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556e63077590 0x556e63439360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:31.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.054+0000 7f7fe27fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556e6339c4d0 con 0x556e63098270 2024-05-20T18:28:31.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.054+0000 7f7fe27fd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556e633443e0 con 0x556e6306f3b0 2024-05-20T18:28:31.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.054+0000 7f7fe27fd740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x556e63398490 con 0x556e63077590 2024-05-20T18:28:31.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.054+0000 7f7fd961c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e63098270 0x556e63463700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:31.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.054+0000 7f7fd961c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e63098270 0x556e63463700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51714/0 (socket says 172.21.15.5:51714) 2024-05-20T18:28:31.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.054+0000 7f7fd961c640 1 -- 172.21.15.5:0/2527909889 learned_addr learned my addr 172.21.15.5:0/2527909889 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:31.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.055+0000 7f7fd9e1d640 1 --2- 172.21.15.5:0/2527909889 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556e63077590 0x556e63439360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:31.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.055+0000 7f7fd8e1b640 1 --2- 172.21.15.5:0/2527909889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6306f3b0 0x556e634600a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:31.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.055+0000 7f7fd961c640 1 -- 172.21.15.5:0/2527909889 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556e63077590 msgr2=0x556e63439360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:31.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.055+0000 7f7fd961c640 1 --2- 172.21.15.5:0/2527909889 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556e63077590 0x556e63439360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.055+0000 7f7fd961c640 1 -- 172.21.15.5:0/2527909889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6306f3b0 msgr2=0x556e634600a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:31.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.055+0000 7f7fd961c640 1 --2- 172.21.15.5:0/2527909889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6306f3b0 0x556e634600a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.055+0000 7f7fd961c640 1 -- 172.21.15.5:0/2527909889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e630b02d0 con 0x556e63098270 2024-05-20T18:28:31.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.055+0000 7f7fd8e1b640 1 --2- 172.21.15.5:0/2527909889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6306f3b0 0x556e634600a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:31.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.055+0000 7f7fd961c640 1 --2- 172.21.15.5:0/2527909889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e63098270 0x556e63463700 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40059d0 tx=0x7f7fc4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97e5c97e7ebeb1c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:31.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.055+0000 7f7fbaffd640 1 -- 172.21.15.5:0/2527909889 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc400e7b0 con 0x556e63098270 2024-05-20T18:28:31.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.055+0000 7f7fbaffd640 1 -- 172.21.15.5:0/2527909889 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fc400e950 con 0x556e63098270 2024-05-20T18:28:31.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.055+0000 7f7fbaffd640 1 -- 172.21.15.5:0/2527909889 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc400ed00 con 0x556e63098270 2024-05-20T18:28:31.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.056+0000 7f7fe27fd740 1 -- 172.21.15.5:0/2527909889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e63098270 msgr2=0x556e63463700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:31.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.056+0000 7f7fe27fd740 1 --2- 172.21.15.5:0/2527909889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e63098270 0x556e63463700 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40059d0 tx=0x7f7fc4011000 comp rx=0 tx=0).stop 2024-05-20T18:28:31.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.056+0000 7f7fe27fd740 1 -- 172.21.15.5:0/2527909889 shutdown_connections 2024-05-20T18:28:31.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.056+0000 7f7fe27fd740 1 --2- 172.21.15.5:0/2527909889 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556e63077590 0x556e63439360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.056+0000 7f7fe27fd740 1 --2- 172.21.15.5:0/2527909889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6306f3b0 0x556e634600a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.056+0000 7f7fe27fd740 1 --2- 172.21.15.5:0/2527909889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e63098270 0x556e63463700 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.056+0000 7f7fe27fd740 1 -- 172.21.15.5:0/2527909889 >> 172.21.15.5:0/2527909889 conn(0x556e6308c3c0 msgr2=0x556e63438af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:31.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.056+0000 7f7fe27fd740 1 -- 172.21.15.5:0/2527909889 shutdown_connections 2024-05-20T18:28:31.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.056+0000 7f7fe27fd740 1 -- 172.21.15.5:0/2527909889 wait complete. 2024-05-20T18:28:31.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.057+0000 7f7fe27fd740 1 -- start start 2024-05-20T18:28:31.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.057+0000 7f7fe27fd740 1 Processor -- start 2024-05-20T18:28:31.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.058+0000 7f7fe27fd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6306f3b0 0x556e6345cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:31.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.059+0000 7f7fe27fd740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556e63077590 0x556e63466500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:31.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.059+0000 7f7fe27fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e63098270 0x556e6347e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:31.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.059+0000 7f7fe27fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556e633578e0 con 0x556e63098270 2024-05-20T18:28:31.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.059+0000 7f7fe27fd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556e6345d8e0 con 0x556e6306f3b0 2024-05-20T18:28:31.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.059+0000 7f7fe27fd740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x556e6345d450 con 0x556e63077590 2024-05-20T18:28:31.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.059+0000 7f7fe27fd740 10 client.0 _ll_register_callbacks cb 0x556e633249c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:28:31.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.059+0000 7f7fd9e1d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e63098270 0x556e6347e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:31.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.059+0000 7f7fd9e1d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e63098270 0x556e6347e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51716/0 (socket says 172.21.15.5:51716) 2024-05-20T18:28:31.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.059+0000 7f7fd9e1d640 1 -- 172.21.15.5:0/2293878753 learned_addr learned my addr 172.21.15.5:0/2293878753 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:31.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.059+0000 7f7fd961c640 1 --2- 172.21.15.5:0/2293878753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6306f3b0 0x556e6345cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:31.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.059+0000 7f7fd8e1b640 1 --2- 172.21.15.5:0/2293878753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556e63077590 0x556e63466500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:31.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.059+0000 7f7fd9e1d640 1 -- 172.21.15.5:0/2293878753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556e63077590 msgr2=0x556e63466500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:31.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.059+0000 7f7fd9e1d640 1 --2- 172.21.15.5:0/2293878753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556e63077590 0x556e63466500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.059+0000 7f7fd9e1d640 1 -- 172.21.15.5:0/2293878753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6306f3b0 msgr2=0x556e6345cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:31.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.059+0000 7f7fd9e1d640 1 --2- 172.21.15.5:0/2293878753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6306f3b0 0x556e6345cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.059+0000 7f7fd9e1d640 1 -- 172.21.15.5:0/2293878753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e6317a6b0 con 0x556e63098270 2024-05-20T18:28:31.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.059+0000 7f7fd9e1d640 1 --2- 172.21.15.5:0/2293878753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e63098270 0x556e6347e0b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc00ef90 tx=0x7f7fcc009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4940663c20ef9314 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:31.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.059+0000 7f7fb3fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:28:31.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.059+0000 7f7fb3fff640 1 -- 172.21.15.5:0/2293878753 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc062a10 con 0x556e63098270 2024-05-20T18:28:31.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.059+0000 7f7fb3fff640 1 -- 172.21.15.5:0/2293878753 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f7fcc024de0 con 0x556e63098270 2024-05-20T18:28:31.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.059+0000 7f7fe27fd740 1 -- 172.21.15.5:0/2293878753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fc400e950 con 0x556e63098270 2024-05-20T18:28:31.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.060+0000 7f7fb3fff640 1 -- 172.21.15.5:0/2293878753 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f7fcc059280 con 0x556e63098270 2024-05-20T18:28:31.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.060+0000 7f7fe27fd740 10 client.5502 fetch_fsmap learned FSMap version 17 2024-05-20T18:28:31.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.060+0000 7f7fe27fd740 1 -- 172.21.15.5:0/2293878753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x556e63465ca0 con 0x556e63098270 2024-05-20T18:28:31.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.060+0000 7f7fb3fff640 1 -- 172.21.15.5:0/2293878753 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fc400e950 con 0x556e63098270 2024-05-20T18:28:31.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.060+0000 7f7fb3fff640 1 -- 172.21.15.5:0/2293878753 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f7fcc059420 con 0x556e63098270 2024-05-20T18:28:31.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.060+0000 7f7fe27fd740 10 client.5502 fetch_fsmap finished waiting for FSMap version 17 2024-05-20T18:28:31.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.060+0000 7f7fe27fd740 10 client.5502 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-20T18:28:31.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.060+0000 7f7fe27fd740 20 client.5502 populate_metadata read hostname 'smithi005' 2024-05-20T18:28:31.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.060+0000 7f7fe27fd740 1 --2- 172.21.15.5:0/2293878753 >> [v2:172.21.15.5:6835/2355805105,v1:172.21.15.5:6837/2355805105] conn(0x556e63056e40 0x556e6345c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:31.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.060+0000 7f7fe27fd740 4 client.5502 mds_command: new command op to 5316 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:28:31.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.060+0000 7f7fe27fd740 1 -- 172.21.15.5:0/2293878753 --> [v2:172.21.15.5:6835/2355805105,v1:172.21.15.5:6837/2355805105] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e633a9c00 con 0x556e63056e40 2024-05-20T18:28:31.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.060+0000 7f7fd961c640 1 --2- 172.21.15.5:0/2293878753 >> [v2:172.21.15.5:6835/2355805105,v1:172.21.15.5:6837/2355805105] conn(0x556e63056e40 0x556e6345c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:31.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.061+0000 7f7fb3fff640 10 client.5502 ms_handle_connect on v2:172.21.15.5:6835/2355805105 2024-05-20T18:28:31.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.061+0000 7f7fd961c640 1 --2- 172.21.15.5:0/2293878753 >> [v2:172.21.15.5:6835/2355805105,v1:172.21.15.5:6837/2355805105] conn(0x556e63056e40 0x556e6345c860 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ded0e544cdb809fd server_cookie=eaf9cc09bfa19a68 in_seq=0 out_seq=0 2024-05-20T18:28:31.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.061+0000 7f7fb3fff640 1 -- 172.21.15.5:0/2293878753 <== mds.0 v2:172.21.15.5:6835/2355805105 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556e633a9c00 con 0x556e63056e40 2024-05-20T18:28:31.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.061+0000 7f7fb3fff640 10 client.5502 handle_command_reply: tid=0 2024-05-20T18:28:31.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.061+0000 7f7fe27fd740 1 client.5502 shutdown 2024-05-20T18:28:31.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.061+0000 7f7fe27fd740 1 -- 172.21.15.5:0/2293878753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e63098270 msgr2=0x556e6347e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:31.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.061+0000 7f7fe27fd740 1 --2- 172.21.15.5:0/2293878753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e63098270 0x556e6347e0b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc00ef90 tx=0x7f7fcc009b70 comp rx=0 tx=0).stop 2024-05-20T18:28:31.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.062+0000 7f7fe27fd740 1 -- 172.21.15.5:0/2293878753 shutdown_connections 2024-05-20T18:28:31.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.062+0000 7f7fe27fd740 1 --2- 172.21.15.5:0/2293878753 >> [v2:172.21.15.5:6835/2355805105,v1:172.21.15.5:6837/2355805105] conn(0x556e63056e40 0x556e6345c860 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.062+0000 7f7fe27fd740 1 --2- 172.21.15.5:0/2293878753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e63098270 0x556e6347e0b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.062+0000 7f7fe27fd740 1 --2- 172.21.15.5:0/2293878753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556e63077590 0x556e63466500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.062+0000 7f7fe27fd740 1 --2- 172.21.15.5:0/2293878753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e6306f3b0 0x556e6345cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.062+0000 7f7fb3fff640 0 client.5502 ms_handle_reset on v2:172.21.15.5:6835/2355805105 2024-05-20T18:28:31.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.062+0000 7f7fe27fd740 1 -- 172.21.15.5:0/2293878753 >> 172.21.15.5:0/2293878753 conn(0x556e6308c3c0 msgr2=0x556e63438af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:31.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.062+0000 7f7fe27fd740 1 -- 172.21.15.5:0/2293878753 shutdown_connections 2024-05-20T18:28:31.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.062+0000 7f7fe27fd740 1 -- 172.21.15.5:0/2293878753 wait complete. 2024-05-20T18:28:31.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.062+0000 7f7fe27fd740 20 client.5502 trim_cache size 0 max 16384 2024-05-20T18:28:31.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.081+0000 7f7fe27fd740 1 Processor -- start 2024-05-20T18:28:31.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.081+0000 7f7fe27fd740 1 -- start start 2024-05-20T18:28:31.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.082+0000 7f7fe27fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e6306f3b0 0x556e63480ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:31.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.082+0000 7f7fe27fd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e63077590 0x556e63480810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:31.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.082+0000 7f7fe27fd740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556e63098270 0x556e63464350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:31.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.082+0000 7f7fe27fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd4201400 con 0x556e6306f3b0 2024-05-20T18:28:31.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.082+0000 7f7fe27fd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd4200760 con 0x556e63077590 2024-05-20T18:28:31.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.082+0000 7f7fe27fd740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7fd4200db0 con 0x556e63098270 2024-05-20T18:28:31.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.082+0000 7f7fd961c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e6306f3b0 0x556e63480ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:31.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.082+0000 7f7fd961c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e6306f3b0 0x556e63480ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51720/0 (socket says 172.21.15.5:51720) 2024-05-20T18:28:31.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.082+0000 7f7fd961c640 1 -- 172.21.15.5:0/1938804530 learned_addr learned my addr 172.21.15.5:0/1938804530 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:31.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.082+0000 7f7fd8e1b640 1 --2- 172.21.15.5:0/1938804530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e63077590 0x556e63480810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:31.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.082+0000 7f7fd9e1d640 1 --2- 172.21.15.5:0/1938804530 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556e63098270 0x556e63464350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:31.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.082+0000 7f7fd961c640 1 -- 172.21.15.5:0/1938804530 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556e63098270 msgr2=0x556e63464350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:31.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.082+0000 7f7fd961c640 1 --2- 172.21.15.5:0/1938804530 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556e63098270 0x556e63464350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.082+0000 7f7fd961c640 1 -- 172.21.15.5:0/1938804530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e63077590 msgr2=0x556e63480810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:31.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.082+0000 7f7fd961c640 1 --2- 172.21.15.5:0/1938804530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e63077590 0x556e63480810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.082+0000 7f7fd961c640 1 -- 172.21.15.5:0/1938804530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e63466960 con 0x556e6306f3b0 2024-05-20T18:28:31.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.083+0000 7f7fd8e1b640 1 --2- 172.21.15.5:0/1938804530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e63077590 0x556e63480810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:31.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.083+0000 7f7fd9e1d640 1 --2- 172.21.15.5:0/1938804530 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556e63098270 0x556e63464350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:31.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.083+0000 7f7fd961c640 1 --2- 172.21.15.5:0/1938804530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e6306f3b0 0x556e63480ed0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x556e63475280 tx=0x7f7fc4007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c826941f9bf02f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:31.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.083+0000 7f7fba7fc640 1 -- 172.21.15.5:0/1938804530 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc400ed80 con 0x556e6306f3b0 2024-05-20T18:28:31.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.083+0000 7f7fba7fc640 1 -- 172.21.15.5:0/1938804530 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fc4027040 con 0x556e6306f3b0 2024-05-20T18:28:31.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.083+0000 7f7fba7fc640 1 -- 172.21.15.5:0/1938804530 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc40303b0 con 0x556e6306f3b0 2024-05-20T18:28:31.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.083+0000 7f7fe27fd740 1 -- 172.21.15.5:0/1938804530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e6306f3b0 msgr2=0x556e63480ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:31.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.083+0000 7f7fe27fd740 1 --2- 172.21.15.5:0/1938804530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e6306f3b0 0x556e63480ed0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x556e63475280 tx=0x7f7fc4007000 comp rx=0 tx=0).stop 2024-05-20T18:28:31.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.084+0000 7f7fe27fd740 1 -- 172.21.15.5:0/1938804530 shutdown_connections 2024-05-20T18:28:31.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.084+0000 7f7fe27fd740 1 --2- 172.21.15.5:0/1938804530 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556e63098270 0x556e63464350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.084+0000 7f7fe27fd740 1 --2- 172.21.15.5:0/1938804530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e63077590 0x556e63480810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.084+0000 7f7fe27fd740 1 --2- 172.21.15.5:0/1938804530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e6306f3b0 0x556e63480ed0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.084+0000 7f7fe27fd740 1 -- 172.21.15.5:0/1938804530 >> 172.21.15.5:0/1938804530 conn(0x556e6308c3c0 msgr2=0x556e63463700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:31.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.084+0000 7f7fe27fd740 1 -- 172.21.15.5:0/1938804530 shutdown_connections 2024-05-20T18:28:31.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.084+0000 7f7fe27fd740 1 -- 172.21.15.5:0/1938804530 wait complete. 2024-05-20T18:28:31.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.084+0000 7f7fe27fd740 1 -- start start 2024-05-20T18:28:31.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.085+0000 7f7fe27fd740 1 Processor -- start 2024-05-20T18:28:31.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.086+0000 7f7fe27fd740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556e6306f3b0 0x556e6345ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:31.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.086+0000 7f7fe27fd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e63077590 0x556e63480c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:31.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.086+0000 7f7fe27fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e63098270 0x556e6347fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:31.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.086+0000 7f7fe27fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556e63466b80 con 0x556e63098270 2024-05-20T18:28:31.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.086+0000 7f7fe27fd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556e6345d620 con 0x556e63077590 2024-05-20T18:28:31.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.086+0000 7f7fe27fd740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x556e63490ad0 con 0x556e6306f3b0 2024-05-20T18:28:31.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.086+0000 7f7fe27fd740 10 client.0 _ll_register_callbacks cb 0x7f7fcc059840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:28:31.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.086+0000 7f7fd9e1d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e63098270 0x556e6347fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:31.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.086+0000 7f7fd9e1d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e63098270 0x556e6347fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51728/0 (socket says 172.21.15.5:51728) 2024-05-20T18:28:31.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.086+0000 7f7fd9e1d640 1 -- 172.21.15.5:0/16721911 learned_addr learned my addr 172.21.15.5:0/16721911 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.086+0000 7f7fd8e1b640 1 --2- 172.21.15.5:0/16721911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e63077590 0x556e63480c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.086+0000 7f7fd961c640 1 --2- 172.21.15.5:0/16721911 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556e6306f3b0 0x556e6345ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.086+0000 7f7fd9e1d640 1 -- 172.21.15.5:0/16721911 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556e6306f3b0 msgr2=0x556e6345ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.086+0000 7f7fd9e1d640 1 --2- 172.21.15.5:0/16721911 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556e6306f3b0 0x556e6345ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.086+0000 7f7fd9e1d640 1 -- 172.21.15.5:0/16721911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e63077590 msgr2=0x556e63480c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.086+0000 7f7fd9e1d640 1 --2- 172.21.15.5:0/16721911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e63077590 0x556e63480c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.086+0000 7f7fd9e1d640 1 -- 172.21.15.5:0/16721911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e6345f260 con 0x556e63098270 2024-05-20T18:28:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.086+0000 7f7fd961c640 1 --2- 172.21.15.5:0/16721911 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556e6306f3b0 0x556e6345ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.086+0000 7f7fd8e1b640 1 --2- 172.21.15.5:0/16721911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e63077590 0x556e63480c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.087+0000 7f7fd9e1d640 1 --2- 172.21.15.5:0/16721911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e63098270 0x556e6347fb10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc009fd0 tx=0x7f7fcc089f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedeaf003feefaaf server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.087+0000 7f7fb3fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:28:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.087+0000 7f7fb3fff640 1 -- 172.21.15.5:0/16721911 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc075070 con 0x556e63098270 2024-05-20T18:28:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.087+0000 7f7fb3fff640 1 -- 172.21.15.5:0/16721911 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f7fcc07a050 con 0x556e63098270 2024-05-20T18:28:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.087+0000 7f7fe27fd740 1 -- 172.21.15.5:0/16721911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fc4027040 con 0x556e63098270 2024-05-20T18:28:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.087+0000 7f7fb3fff640 1 -- 172.21.15.5:0/16721911 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f7fcc06a050 con 0x556e63098270 2024-05-20T18:28:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.087+0000 7f7fe27fd740 10 client.5508 fetch_fsmap learned FSMap version 17 2024-05-20T18:28:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.087+0000 7f7fe27fd740 1 -- 172.21.15.5:0/16721911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x556e6345bab0 con 0x556e63098270 2024-05-20T18:28:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.087+0000 7f7fb3fff640 1 -- 172.21.15.5:0/16721911 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fc4027040 con 0x556e63098270 2024-05-20T18:28:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.087+0000 7f7fb3fff640 1 -- 172.21.15.5:0/16721911 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f7fcc06d020 con 0x556e63098270 2024-05-20T18:28:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.087+0000 7f7fe27fd740 10 client.5508 fetch_fsmap finished waiting for FSMap version 17 2024-05-20T18:28:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.087+0000 7f7fe27fd740 10 client.5508 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-20T18:28:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.087+0000 7f7fe27fd740 20 client.5508 populate_metadata read hostname 'smithi005' 2024-05-20T18:28:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.087+0000 7f7fe27fd740 1 --2- 172.21.15.5:0/16721911 >> [v2:172.21.15.5:6835/2355805105,v1:172.21.15.5:6837/2355805105] conn(0x556e63484cb0 0x556e634850c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.087+0000 7f7fe27fd740 4 client.5508 mds_command: new command op to 5316 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T18:28:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.087+0000 7f7fe27fd740 1 -- 172.21.15.5:0/16721911 --> [v2:172.21.15.5:6835/2355805105,v1:172.21.15.5:6837/2355805105] -- command(tid 0: {"prefix": "status"}) -- 0x556e633a3f50 con 0x556e63484cb0 2024-05-20T18:28:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.087+0000 7f7fd961c640 1 --2- 172.21.15.5:0/16721911 >> [v2:172.21.15.5:6835/2355805105,v1:172.21.15.5:6837/2355805105] conn(0x556e63484cb0 0x556e634850c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:31.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.088+0000 7f7fb3fff640 10 client.5508 ms_handle_connect on v2:172.21.15.5:6835/2355805105 2024-05-20T18:28:31.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.088+0000 7f7fd961c640 1 --2- 172.21.15.5:0/16721911 >> [v2:172.21.15.5:6835/2355805105,v1:172.21.15.5:6837/2355805105] conn(0x556e63484cb0 0x556e634850c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87fb613a716d6170 server_cookie=f31d942be963243b in_seq=0 out_seq=0 2024-05-20T18:28:31.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.088+0000 7f7fb3fff640 1 -- 172.21.15.5:0/16721911 <== mds.0 v2:172.21.15.5:6835/2355805105 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x556e633a3f50 con 0x556e63484cb0 2024-05-20T18:28:31.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.088+0000 7f7fb3fff640 10 client.5508 handle_command_reply: tid=0 2024-05-20T18:28:31.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.088+0000 7f7fe27fd740 1 client.5508 shutdown 2024-05-20T18:28:31.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.088+0000 7f7fe27fd740 1 -- 172.21.15.5:0/16721911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e63098270 msgr2=0x556e6347fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:31.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.088+0000 7f7fe27fd740 1 --2- 172.21.15.5:0/16721911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e63098270 0x556e6347fb10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc009fd0 tx=0x7f7fcc089f80 comp rx=0 tx=0).stop 2024-05-20T18:28:31.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.088+0000 7f7fe27fd740 1 -- 172.21.15.5:0/16721911 shutdown_connections 2024-05-20T18:28:31.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.088+0000 7f7fe27fd740 1 --2- 172.21.15.5:0/16721911 >> [v2:172.21.15.5:6835/2355805105,v1:172.21.15.5:6837/2355805105] conn(0x556e63484cb0 0x556e634850c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.088+0000 7f7fe27fd740 1 --2- 172.21.15.5:0/16721911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e63098270 0x556e6347fb10 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.088+0000 7f7fe27fd740 1 --2- 172.21.15.5:0/16721911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556e63077590 0x556e63480c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.088+0000 7f7fb3fff640 0 client.5508 ms_handle_reset on v2:172.21.15.5:6835/2355805105 2024-05-20T18:28:31.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.088+0000 7f7fe27fd740 1 --2- 172.21.15.5:0/16721911 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556e6306f3b0 0x556e6345ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.088+0000 7f7fe27fd740 1 -- 172.21.15.5:0/16721911 >> 172.21.15.5:0/16721911 conn(0x556e6308c3c0 msgr2=0x556e63438b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:31.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.088+0000 7f7fe27fd740 1 -- 172.21.15.5:0/16721911 shutdown_connections 2024-05-20T18:28:31.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.088+0000 7f7fe27fd740 1 -- 172.21.15.5:0/16721911 wait complete. 2024-05-20T18:28:31.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.089+0000 7f7fe27fd740 20 client.5508 trim_cache size 0 max 16384 2024-05-20T18:28:31.094 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:28:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "399ab250-fd3c-47fa-a1c8-25e8149c9cac", 2024-05-20T18:28:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-20T18:28:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 5316, 2024-05-20T18:28:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-20T18:28:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-20T18:28:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-20T18:28:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 6.1856100959999996, 2024-05-20T18:28:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 17, 2024-05-20T18:28:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 40, 2024-05-20T18:28:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 40, 2024-05-20T18:28:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 14.059984403 2024-05-20T18:28:31.095 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:28:31.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.089+0000 7f7fdb880640 1 -- 172.21.15.5:0/847816546 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7fc006be30 msgr2=0x7f7fc006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:31.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.089+0000 7f7fdb880640 1 --2- 172.21.15.5:0/847816546 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7fc006be30 0x7f7fc006e340 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4005880 tx=0x7f7fc4009040 comp rx=0 tx=0).stop 2024-05-20T18:28:31.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.089+0000 7f7fdb880640 1 -- 172.21.15.5:0/847816546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fd4162820 msgr2=0x7f7fd40ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:31.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.089+0000 7f7fdb880640 1 --2- 172.21.15.5:0/847816546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fd4162820 0x7f7fd40ce7e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc00efc0 tx=0x7f7fcc00c490 comp rx=0 tx=0).stop 2024-05-20T18:28:31.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.089+0000 7f7fdb880640 1 -- 172.21.15.5:0/847816546 shutdown_connections 2024-05-20T18:28:31.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.089+0000 7f7fdb880640 1 --2- 172.21.15.5:0/847816546 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7fc006be30 0x7f7fc006e340 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.089+0000 7f7fdb880640 1 --2- 172.21.15.5:0/847816546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fd4162820 0x7f7fd40ce7e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.089+0000 7f7fdb880640 1 --2- 172.21.15.5:0/847816546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fd40b8190 0x7f7fd40ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.089+0000 7f7fdb880640 1 --2- 172.21.15.5:0/847816546 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fd406d3d0 0x7f7fd40cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.089+0000 7f7fdb880640 1 -- 172.21.15.5:0/847816546 >> 172.21.15.5:0/847816546 conn(0x7f7fd40d4bd0 msgr2=0x7f7fd40b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:31.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.089+0000 7f7fdb880640 1 -- 172.21.15.5:0/847816546 shutdown_connections 2024-05-20T18:28:31.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.089+0000 7f7fdb880640 1 -- 172.21.15.5:0/847816546 wait complete. 2024-05-20T18:28:31.100 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-20T18:28:31.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.215+0000 7ffb07b3e640 1 Processor -- start 2024-05-20T18:28:31.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.216+0000 7ffb07b3e640 1 -- start start 2024-05-20T18:28:31.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.217+0000 7ffb07b3e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ffb0006d3d0 0x7ffb0006d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:31.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.217+0000 7ffb07b3e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb0006dd20 con 0x7ffb00162820 2024-05-20T18:28:31.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.217+0000 7ffb07b3e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb000d0630 con 0x7ffb000b8190 2024-05-20T18:28:31.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.217+0000 7ffb07b3e640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ffb000d39c0 con 0x7ffb0006d3d0 2024-05-20T18:28:31.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.217+0000 7ffb058da640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ffb00162820 0x7ffb000b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53776/0 (socket says 172.21.15.5:53776) 2024-05-20T18:28:31.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.217+0000 7ffb058da640 1 -- 172.21.15.5:0/2387308 learned_addr learned my addr 172.21.15.5:0/2387308 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:31.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.217+0000 7ffb048d8640 1 -- 172.21.15.5:0/2387308 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 700018818 0 0) 0x7ffafc001110 con 0x7ffb00162820 2024-05-20T18:28:31.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.217+0000 7ffb048d8640 1 -- 172.21.15.5:0/2387308 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffaec003520 con 0x7ffb00162820 2024-05-20T18:28:31.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.217+0000 7ffb050d9640 1 --2- 172.21.15.5:0/2387308 >> v2:172.21.15.63:3300/0 conn(0x7ffb0006d3d0 0x7ffb0006d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:31.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.218+0000 7ffb048d8640 1 -- 172.21.15.5:0/2387308 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1301160509 0 0) 0x7ffaec003520 con 0x7ffb00162820 2024-05-20T18:28:31.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.218+0000 7ffb048d8640 1 -- 172.21.15.5:0/2387308 >> v2:172.21.15.63:3300/0 conn(0x7ffb0006d3d0 msgr2=0x7ffb0006d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:31.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.218+0000 7ffb048d8640 1 --2- 172.21.15.5:0/2387308 >> v2:172.21.15.63:3300/0 conn(0x7ffb0006d3d0 0x7ffb0006d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.218+0000 7ffb048d8640 1 -- 172.21.15.5:0/2387308 >> v1:172.21.15.63:6789/0 conn(0x7ffb000b8190 legacy=0x7ffb000b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:31.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.218+0000 7ffb048d8640 1 -- 172.21.15.5:0/2387308 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb000d3ca0 con 0x7ffb00162820 2024-05-20T18:28:31.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.218+0000 7ffb048d8640 1 -- 172.21.15.5:0/2387308 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7ffafc001840 con 0x7ffb00162820 2024-05-20T18:28:31.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.218+0000 7ffb048d8640 1 -- 172.21.15.5:0/2387308 >> v1:172.21.15.5:6789/0 conn(0x7ffb00162820 legacy=0x7ffb000b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:31.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.218+0000 7ffb048d8640 1 --2- 172.21.15.5:0/2387308 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffaec0039b0 0x7ffaec003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:31.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.218+0000 7ffb048d8640 1 --2- 172.21.15.5:0/2387308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaec004800 0x7ffaec004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:31.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.218+0000 7ffb048d8640 1 --2- 172.21.15.5:0/2387308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffaec0051c0 0x7ffaec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:31.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.218+0000 7ffb048d8640 1 -- 172.21.15.5:0/2387308 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffb000d39c0 con 0x7ffaec004800 2024-05-20T18:28:31.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.218+0000 7ffb048d8640 1 -- 172.21.15.5:0/2387308 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec00c160 con 0x7ffaec0051c0 2024-05-20T18:28:31.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.218+0000 7ffb048d8640 1 -- 172.21.15.5:0/2387308 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffaec00c360 con 0x7ffaec0039b0 2024-05-20T18:28:31.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.218+0000 7ffb050d9640 1 --2- 172.21.15.5:0/2387308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaec004800 0x7ffaec004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:31.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.218+0000 7ffb058da640 1 --2- 172.21.15.5:0/2387308 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffaec0039b0 0x7ffaec003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:31.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.218+0000 7ffb060db640 1 --2- 172.21.15.5:0/2387308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffaec0051c0 0x7ffaec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:31.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.218+0000 7ffb050d9640 1 -- 172.21.15.5:0/2387308 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffaec0039b0 msgr2=0x7ffaec003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:31.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.218+0000 7ffb050d9640 1 --2- 172.21.15.5:0/2387308 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffaec0039b0 0x7ffaec003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.218+0000 7ffb050d9640 1 -- 172.21.15.5:0/2387308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffaec0051c0 msgr2=0x7ffaec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:31.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.218+0000 7ffb050d9640 1 --2- 172.21.15.5:0/2387308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffaec0051c0 0x7ffaec00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.218+0000 7ffb050d9640 1 -- 172.21.15.5:0/2387308 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaec003520 con 0x7ffaec004800 2024-05-20T18:28:31.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.218+0000 7ffb060db640 1 -- 172.21.15.5:0/2387308 reap_dead start 2024-05-20T18:28:31.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.218+0000 7ffb058da640 1 --2- 172.21.15.5:0/2387308 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffaec0039b0 0x7ffaec003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:31.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.219+0000 7ffb050d9640 1 --2- 172.21.15.5:0/2387308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaec004800 0x7ffaec004c10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0009ac0 tx=0x7ffaec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a8b9d58fe0beb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:31.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.219+0000 7ffb048d8640 1 -- 172.21.15.5:0/2387308 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0036070 con 0x7ffaec004800 2024-05-20T18:28:31.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.219+0000 7ffb048d8640 1 -- 172.21.15.5:0/2387308 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaf0031400 con 0x7ffaec004800 2024-05-20T18:28:31.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.219+0000 7ffb048d8640 1 -- 172.21.15.5:0/2387308 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0009690 con 0x7ffaec004800 2024-05-20T18:28:31.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.220+0000 7ffb07b3e640 1 -- 172.21.15.5:0/2387308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaec004800 msgr2=0x7ffaec004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:31.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.220+0000 7ffb07b3e640 1 --2- 172.21.15.5:0/2387308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaec004800 0x7ffaec004c10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0009ac0 tx=0x7ffaec005830 comp rx=0 tx=0).stop 2024-05-20T18:28:31.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.220+0000 7ffb07b3e640 1 -- 172.21.15.5:0/2387308 shutdown_connections 2024-05-20T18:28:31.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.220+0000 7ffb07b3e640 1 --2- 172.21.15.5:0/2387308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaec004800 0x7ffaec004c10 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.220+0000 7ffb07b3e640 1 -- 172.21.15.5:0/2387308 >> 172.21.15.5:0/2387308 conn(0x7ffb000d4bd0 msgr2=0x7ffb000d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:31.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.220+0000 7ffb07b3e640 1 -- 172.21.15.5:0/2387308 shutdown_connections 2024-05-20T18:28:31.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.220+0000 7ffb07b3e640 1 -- 172.21.15.5:0/2387308 wait complete. 2024-05-20T18:28:31.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.222+0000 7ffb07b3e640 1 Processor -- start 2024-05-20T18:28:31.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.222+0000 7ffb07b3e640 1 -- start start 2024-05-20T18:28:31.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.223+0000 7ffb07b3e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaec004800 0x7ffb0015e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:31.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.223+0000 7ffb07b3e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb000ccc20 0x7ffb001eace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:31.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.223+0000 7ffb07b3e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb000cd030 0x7ffb001eb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:31.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.223+0000 7ffb07b3e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffb000d0300 con 0x7ffaec004800 2024-05-20T18:28:31.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.223+0000 7ffb07b3e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffb000d4260 con 0x7ffb000cd030 2024-05-20T18:28:31.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.223+0000 7ffb07b3e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffb000d4780 con 0x7ffb000ccc20 2024-05-20T18:28:31.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.223+0000 7ffb058da640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaec004800 0x7ffb0015e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:31.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.223+0000 7ffb058da640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaec004800 0x7ffb0015e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51746/0 (socket says 172.21.15.5:51746) 2024-05-20T18:28:31.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.223+0000 7ffb058da640 1 -- 172.21.15.5:0/2403695928 learned_addr learned my addr 172.21.15.5:0/2403695928 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:31.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.223+0000 7ffb060db640 1 --2- 172.21.15.5:0/2403695928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb000cd030 0x7ffb001eb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:31.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.223+0000 7ffb050d9640 1 --2- 172.21.15.5:0/2403695928 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb000ccc20 0x7ffb001eace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:31.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.224+0000 7ffb058da640 1 -- 172.21.15.5:0/2403695928 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb000ccc20 msgr2=0x7ffb001eace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:31.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.224+0000 7ffb058da640 1 --2- 172.21.15.5:0/2403695928 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb000ccc20 0x7ffb001eace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.224+0000 7ffb058da640 1 -- 172.21.15.5:0/2403695928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb000cd030 msgr2=0x7ffb001eb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:31.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.224+0000 7ffb058da640 1 --2- 172.21.15.5:0/2403695928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb000cd030 0x7ffb001eb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.224+0000 7ffb058da640 1 -- 172.21.15.5:0/2403695928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb001f5310 con 0x7ffaec004800 2024-05-20T18:28:31.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.224+0000 7ffb058da640 1 --2- 172.21.15.5:0/2403695928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaec004800 0x7ffb0015e290 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ffafc005df0 tx=0x7ffafc007420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:31.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.224+0000 7ffaf67fc640 1 -- 172.21.15.5:0/2403695928 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffafc00e740 con 0x7ffaec004800 2024-05-20T18:28:31.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.224+0000 7ffaf67fc640 1 -- 172.21.15.5:0/2403695928 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffafc018070 con 0x7ffaec004800 2024-05-20T18:28:31.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.224+0000 7ffb07b3e640 1 -- 172.21.15.5:0/2403695928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb000cd790 con 0x7ffaec004800 2024-05-20T18:28:31.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.224+0000 7ffaf67fc640 1 -- 172.21.15.5:0/2403695928 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffafc00eae0 con 0x7ffaec004800 2024-05-20T18:28:31.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.224+0000 7ffb07b3e640 1 -- 172.21.15.5:0/2403695928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb000cdd50 con 0x7ffaec004800 2024-05-20T18:28:31.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.225+0000 7ffb07b3e640 1 -- 172.21.15.5:0/2403695928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb00075b00 con 0x7ffaec004800 2024-05-20T18:28:31.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.226+0000 7ffaf67fc640 1 -- 172.21.15.5:0/2403695928 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffafc00ec80 con 0x7ffaec004800 2024-05-20T18:28:31.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.226+0000 7ffaf67fc640 1 --2- 172.21.15.5:0/2403695928 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffae406be30 0x7ffae406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:31.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.226+0000 7ffaf67fc640 1 -- 172.21.15.5:0/2403695928 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7ffafc08afa0 con 0x7ffaec004800 2024-05-20T18:28:31.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.226+0000 7ffb050d9640 1 --2- 172.21.15.5:0/2403695928 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffae406be30 0x7ffae406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:31.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.227+0000 7ffb050d9640 1 --2- 172.21.15.5:0/2403695928 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffae406be30 0x7ffae406e340 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0033000 tx=0x7ffaf0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:31.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.231+0000 7ffaf67fc640 1 -- 172.21.15.5:0/2403695928 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ffb001f5310 con 0x7ffaec004800 2024-05-20T18:28:31.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.454+0000 7ffb07b3e640 1 -- 172.21.15.5:0/2403695928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ffb000cad80 con 0x7ffaec004800 2024-05-20T18:28:31.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.455+0000 7ffaf67fc640 1 -- 172.21.15.5:0/2403695928 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7ffb0006dd20 con 0x7ffaec004800 2024-05-20T18:28:31.454 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-20T18:28:31.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.457+0000 7ffb07b3e640 1 -- 172.21.15.5:0/2403695928 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffae406be30 msgr2=0x7ffae406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:31.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.457+0000 7ffb07b3e640 1 --2- 172.21.15.5:0/2403695928 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffae406be30 0x7ffae406e340 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0033000 tx=0x7ffaf0008030 comp rx=0 tx=0).stop 2024-05-20T18:28:31.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.457+0000 7ffb07b3e640 1 -- 172.21.15.5:0/2403695928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaec004800 msgr2=0x7ffb0015e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:31.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.457+0000 7ffb07b3e640 1 --2- 172.21.15.5:0/2403695928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaec004800 0x7ffb0015e290 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ffafc005df0 tx=0x7ffafc007420 comp rx=0 tx=0).stop 2024-05-20T18:28:31.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.457+0000 7ffb07b3e640 1 -- 172.21.15.5:0/2403695928 shutdown_connections 2024-05-20T18:28:31.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.457+0000 7ffb07b3e640 1 --2- 172.21.15.5:0/2403695928 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffae406be30 0x7ffae406e340 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.457+0000 7ffb07b3e640 1 --2- 172.21.15.5:0/2403695928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb000cd030 0x7ffb001eb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.457+0000 7ffb07b3e640 1 --2- 172.21.15.5:0/2403695928 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb000ccc20 0x7ffb001eace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.457+0000 7ffb07b3e640 1 --2- 172.21.15.5:0/2403695928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaec004800 0x7ffb0015e290 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:31.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.457+0000 7ffb07b3e640 1 -- 172.21.15.5:0/2403695928 >> 172.21.15.5:0/2403695928 conn(0x7ffb000d4bd0 msgr2=0x7ffb00160590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:31.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.457+0000 7ffb07b3e640 1 -- 172.21.15.5:0/2403695928 shutdown_connections 2024-05-20T18:28:31.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31.457+0000 7ffb07b3e640 1 -- 172.21.15.5:0/2403695928 wait complete. 2024-05-20T18:28:31.467 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-20T18:28:31.468 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-20T18:28:31.468 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-05-20T18:28:31.468 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-20T18:28:31.468 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-20T18:28:31.468 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-20T18:28:31.468 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-20T18:28:31.468 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-20T18:28:31.468 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:28:31.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:28:31.468 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-05-20T18:28:31.485 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T18:28:31.485 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T18:28:31.485 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T18:28:31.485 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:28:31.485 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-20T18:28:31.485 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:28:31.486 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:28:31.486 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-05-20T18:28:31.486 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-05-20T18:28:31.486 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:28:31.486 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-05-20T18:28:31.486 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-05-20T18:28:31.486 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-20T18:28:31.486 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:28:31.486 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-05-20T18:28:31.486 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-20T18:28:31.486 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42586sec preferred_lft 42586sec 2024-05-20T18:28:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-05-20T18:28:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:28:31.487 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:28:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:28:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-05-20T18:28:31.487 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-20T18:28:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:28:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-20T18:28:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:28:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::4474:eaff:fe07:4f9a/64 scope link 2024-05-20T18:28:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:28:31.487 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:28:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T18:28:31.488 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ce4:f4ff:fe5e:7906/64 scope link 2024-05-20T18:28:31.488 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:28:31.488 INFO:teuthology.orchestra.run.smithi005.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:28:31.488 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether da:26:04:0e:95:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T18:28:31.488 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::d826:4ff:fe0e:950a/64 scope link 2024-05-20T18:28:31.488 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:28:31.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:28:31.488 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-20T18:28:31.550 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-20T18:28:31.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:28:31.551 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-05-20T18:28:31.608 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-05-20T18:28:31.608 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-20T18:28:31.608 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-20T18:28:31.608 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-20T18:28:31.608 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-20T18:28:31.609 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:28:31.609 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-20T18:28:31.609 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-20T18:28:31.609 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-20T18:28:31.609 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-20T18:28:31.609 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-20T18:28:31.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:28:31.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:28:31.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:28:31.739 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-20T18:28:31.755 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-20T18:28:31.756 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-20T18:28:31.756 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:28:31.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:28:31.756 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-05-20T18:28:31.823 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:28:31.823 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-20T18:28:31.878 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:28:31.878 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:28:31.878 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:28:31.934 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-20T18:28:31.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:28:31.934 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:28:32.030 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:28:32.030 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:28:32.030 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:28:32.041 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:28:32.041 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:28:32.041 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:28:32.041 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:28:32.041 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:28:32.047 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:28:32.047 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:28:32.047 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:28:32.047 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:28:32.047 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:28:32.047 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790" 2024-05-20T18:28:32.048 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790:/ 2024-05-20T18:28:32.048 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=399ab250-fd3c-47fa-a1c8-25e8149c9cac" will pass to kernel 2024-05-20T18:28:32.048 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-20T18:28:32.048 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-05-20T18:28:32.048 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-20T18:28:32.048 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-05-20T18:28:32.048 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-20T18:28:32.049 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-20T18:28:32.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:28:32.050 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:28:32.076 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-20T18:28:32.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T18:28:32.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:28:32.183 INFO:teuthology.orchestra.run.smithi005.stdout:client5042 2024-05-20T18:28:32.184 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:28:32.184 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:28:32.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:28:32.200 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2024-05-20T18:28:32.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:28:32.263 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2024-05-20T18:28:32.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:28:32.322 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2024-05-20T18:28:32.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:28:32.380 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2024-05-20T18:28:32.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:28:32.440 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2024-05-20T18:28:32.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:28:32.498 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-20T18:28:32.557 INFO:teuthology.orchestra.run.smithi005.stdout:./ 2024-05-20T18:28:32.558 INFO:teuthology.orchestra.run.smithi005.stdout:./rfile 2024-05-20T18:28:32.558 INFO:teuthology.orchestra.run.smithi005.stdout:./parentdir 2024-05-20T18:28:32.558 INFO:teuthology.orchestra.run.smithi005.stdout:./parentdir/pfile 2024-05-20T18:28:32.558 INFO:teuthology.orchestra.run.smithi005.stdout:./parentdir/childdir 2024-05-20T18:28:32.558 INFO:teuthology.orchestra.run.smithi005.stdout:./parentdir/childdir/cfile 2024-05-20T18:28:32.559 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:28:32.559 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:28:32.560 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:28:32.560 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:28:32.560 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-20T18:28:32.560 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:28:32.667 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-20T18:28:32.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:28:32.677 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:28:32.677 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:28:32.677 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:28:32.677 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:28:32.677 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2024-05-20T18:28:32.677 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:28:32.784 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627778 2024-05-20T18:28:32.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:28:32.793 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:28:32.793 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:28:32.793 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:28:32.793 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:28:32.794 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2024-05-20T18:28:32.794 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:28:32.901 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627776 2024-05-20T18:28:32.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:28:32.911 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:28:32.911 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:28:32.911 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:28:32.911 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:28:32.911 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2024-05-20T18:28:32.911 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:28:33.019 INFO:teuthology.orchestra.run.smithi005.stdout:1099511628279 2024-05-20T18:28:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:28:33.028 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:28:33.028 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:28:33.028 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:28:33.029 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:28:33.029 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2024-05-20T18:28:33.029 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:28:33.136 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627777 2024-05-20T18:28:33.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:28:33.145 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:28:33.146 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:28:33.146 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:28:33.146 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:28:33.146 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2024-05-20T18:28:33.146 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:28:33.253 INFO:teuthology.orchestra.run.smithi005.stdout:1099511628280 2024-05-20T18:28:33.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:28:33.262 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:28:33.262 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:28:33.319 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:28:33.319 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:28:33.376 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:28:33.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.532+0000 7fd5ea2bc640 1 Processor -- start 2024-05-20T18:28:33.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.532+0000 7fd5ea2bc640 1 -- start start 2024-05-20T18:28:33.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.534+0000 7fd5ea2bc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5e40d10e0 0x7fd5e40cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:33.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.534+0000 7fd5ea2bc640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd5e40d14f0 0x7fd5e40cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:33.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.534+0000 7fd5ea2bc640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5e40d1ac0 con 0x7fd5e40cd4d0 2024-05-20T18:28:33.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.534+0000 7fd5ea2bc640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd5e40d1da0 con 0x7fd5e40d14f0 2024-05-20T18:28:33.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.534+0000 7fd5ea2bc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd5e40d2080 con 0x7fd5e40d10e0 2024-05-20T18:28:33.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.534+0000 7fd5e8859640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd5e40cd4d0 0x7fd5e40cd8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53790/0 (socket says 172.21.15.5:53790) 2024-05-20T18:28:33.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.534+0000 7fd5e8859640 1 -- 172.21.15.5:0/2521798548 learned_addr learned my addr 172.21.15.5:0/2521798548 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:33.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.534+0000 7fd5e37fe640 1 --2- 172.21.15.5:0/2521798548 >> v2:172.21.15.63:3300/0 conn(0x7fd5e40d14f0 0x7fd5e40cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:33.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.534+0000 7fd5e3fff640 1 --2- 172.21.15.5:0/2521798548 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5e40d10e0 0x7fd5e40cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:33.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.534+0000 7fd5e2ffd640 1 -- 172.21.15.5:0/2521798548 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2432632358 0 0) 0x7fd5e40d1ac0 con 0x7fd5e40cd4d0 2024-05-20T18:28:33.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.534+0000 7fd5e2ffd640 1 -- 172.21.15.5:0/2521798548 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5cc003520 con 0x7fd5e40cd4d0 2024-05-20T18:28:33.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.534+0000 7fd5e2ffd640 1 -- 172.21.15.5:0/2521798548 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2432352448 0 0) 0x7fd5cc003520 con 0x7fd5e40cd4d0 2024-05-20T18:28:33.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.534+0000 7fd5e3fff640 1 -- 172.21.15.5:0/2521798548 >> v2:172.21.15.63:3300/0 conn(0x7fd5e40d14f0 msgr2=0x7fd5e40cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:33.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.534+0000 7fd5e3fff640 1 --2- 172.21.15.5:0/2521798548 >> v2:172.21.15.63:3300/0 conn(0x7fd5e40d14f0 0x7fd5e40cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:33.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.534+0000 7fd5e3fff640 1 -- 172.21.15.5:0/2521798548 >> v1:172.21.15.5:6789/0 conn(0x7fd5e40cd4d0 legacy=0x7fd5e40cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:33.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.534+0000 7fd5e3fff640 1 -- 172.21.15.5:0/2521798548 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e40d2360 con 0x7fd5e40d10e0 2024-05-20T18:28:33.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.535+0000 7fd5e37fe640 1 --2- 172.21.15.5:0/2521798548 >> v2:172.21.15.63:3300/0 conn(0x7fd5e40d14f0 0x7fd5e40cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:28:33.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.535+0000 7fd5e3fff640 1 --2- 172.21.15.5:0/2521798548 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5e40d10e0 0x7fd5e40cf140 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd5d40097a0 tx=0x7fd5d402a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bc6f86ab8a4a934 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:33.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.536+0000 7fd5e2ffd640 1 -- 172.21.15.5:0/2521798548 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4037070 con 0x7fd5e40d10e0 2024-05-20T18:28:33.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.536+0000 7fd5e2ffd640 1 -- 172.21.15.5:0/2521798548 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5d402aba0 con 0x7fd5e40d10e0 2024-05-20T18:28:33.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.536+0000 7fd5e2ffd640 1 -- 172.21.15.5:0/2521798548 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc0019f0 con 0x7fd5e40d10e0 2024-05-20T18:28:33.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.536+0000 7fd5ea2bc640 1 -- 172.21.15.5:0/2521798548 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5e40d10e0 msgr2=0x7fd5e40cf140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:33.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.536+0000 7fd5ea2bc640 1 --2- 172.21.15.5:0/2521798548 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5e40d10e0 0x7fd5e40cf140 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd5d40097a0 tx=0x7fd5d402a110 comp rx=0 tx=0).stop 2024-05-20T18:28:33.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.536+0000 7fd5ea2bc640 1 -- 172.21.15.5:0/2521798548 shutdown_connections 2024-05-20T18:28:33.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.536+0000 7fd5ea2bc640 1 --2- 172.21.15.5:0/2521798548 >> v2:172.21.15.63:3300/0 conn(0x7fd5e40d14f0 0x7fd5e40cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:33.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.536+0000 7fd5ea2bc640 1 --2- 172.21.15.5:0/2521798548 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5e40d10e0 0x7fd5e40cf140 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:33.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.536+0000 7fd5ea2bc640 1 -- 172.21.15.5:0/2521798548 >> 172.21.15.5:0/2521798548 conn(0x7fd5e40b44d0 msgr2=0x7fd5e40b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:33.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.536+0000 7fd5ea2bc640 1 -- 172.21.15.5:0/2521798548 shutdown_connections 2024-05-20T18:28:33.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.536+0000 7fd5ea2bc640 1 -- 172.21.15.5:0/2521798548 wait complete. 2024-05-20T18:28:33.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.538+0000 7fd5ea2bc640 1 Processor -- start 2024-05-20T18:28:33.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.538+0000 7fd5ea2bc640 1 -- start start 2024-05-20T18:28:33.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.540+0000 7fd5ea2bc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5e40cd4d0 0x7fd5e41f94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:33.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.540+0000 7fd5ea2bc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5e40d10e0 0x7fd5e4155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.540+0000 7fd5ea2bc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5e40d14f0 0x7fd5e41f8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.540+0000 7fd5ea2bc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e40b8440 con 0x7fd5e40cd4d0 2024-05-20T18:28:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.540+0000 7fd5ea2bc640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e40b82c0 con 0x7fd5e40d14f0 2024-05-20T18:28:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.540+0000 7fd5ea2bc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd5e40709f0 con 0x7fd5e40d10e0 2024-05-20T18:28:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.540+0000 7fd5e3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5e40cd4d0 0x7fd5e41f94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.540+0000 7fd5e3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5e40cd4d0 0x7fd5e41f94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51760/0 (socket says 172.21.15.5:51760) 2024-05-20T18:28:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.540+0000 7fd5e3fff640 1 -- 172.21.15.5:0/3948954064 learned_addr learned my addr 172.21.15.5:0/3948954064 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.540+0000 7fd5e8859640 1 --2- 172.21.15.5:0/3948954064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5e40d14f0 0x7fd5e41f8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.540+0000 7fd5e37fe640 1 --2- 172.21.15.5:0/3948954064 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5e40d10e0 0x7fd5e4155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.540+0000 7fd5e3fff640 1 -- 172.21.15.5:0/3948954064 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5e40d10e0 msgr2=0x7fd5e4155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.540+0000 7fd5e3fff640 1 --2- 172.21.15.5:0/3948954064 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5e40d10e0 0x7fd5e4155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:33.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.540+0000 7fd5e3fff640 1 -- 172.21.15.5:0/3948954064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5e40d14f0 msgr2=0x7fd5e41f8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.540+0000 7fd5e3fff640 1 --2- 172.21.15.5:0/3948954064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5e40d14f0 0x7fd5e41f8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.540+0000 7fd5e3fff640 1 -- 172.21.15.5:0/3948954064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e40aff30 con 0x7fd5e40cd4d0 2024-05-20T18:28:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.540+0000 7fd5e37fe640 1 --2- 172.21.15.5:0/3948954064 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5e40d10e0 0x7fd5e4155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.540+0000 7fd5e8859640 1 --2- 172.21.15.5:0/3948954064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5e40d14f0 0x7fd5e41f8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.540+0000 7fd5e3fff640 1 --2- 172.21.15.5:0/3948954064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5e40cd4d0 0x7fd5e41f94f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd5d40097a0 tx=0x7fd5d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.540+0000 7fd5e0ff9640 1 -- 172.21.15.5:0/3948954064 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4037040 con 0x7fd5e40cd4d0 2024-05-20T18:28:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.541+0000 7fd5e0ff9640 1 -- 172.21.15.5:0/3948954064 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5d4004230 con 0x7fd5e40cd4d0 2024-05-20T18:28:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.541+0000 7fd5ea2bc640 1 -- 172.21.15.5:0/3948954064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e40b0760 con 0x7fd5e40cd4d0 2024-05-20T18:28:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.541+0000 7fd5e0ff9640 1 -- 172.21.15.5:0/3948954064 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4006e10 con 0x7fd5e40cd4d0 2024-05-20T18:28:33.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.541+0000 7fd5ea2bc640 1 -- 172.21.15.5:0/3948954064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e40b0480 con 0x7fd5e40cd4d0 2024-05-20T18:28:33.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.542+0000 7fd5ea2bc640 1 -- 172.21.15.5:0/3948954064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5e4075b00 con 0x7fd5e40cd4d0 2024-05-20T18:28:33.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.542+0000 7fd5e0ff9640 1 -- 172.21.15.5:0/3948954064 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd5d4003610 con 0x7fd5e40cd4d0 2024-05-20T18:28:33.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.543+0000 7fd5e0ff9640 1 --2- 172.21.15.5:0/3948954064 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd5d006be30 0x7fd5d006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:33.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.543+0000 7fd5e0ff9640 1 -- 172.21.15.5:0/3948954064 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fd5d40abdd0 con 0x7fd5e40cd4d0 2024-05-20T18:28:33.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.543+0000 7fd5e37fe640 1 --2- 172.21.15.5:0/3948954064 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd5d006be30 0x7fd5d006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:33.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.543+0000 7fd5e37fe640 1 --2- 172.21.15.5:0/3948954064 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd5d006be30 0x7fd5d006e340 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40cd240 tx=0x7fd5d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:33.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.547+0000 7fd5e0ff9640 1 -- 172.21.15.5:0/3948954064 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd5d404c070 con 0x7fd5e40cd4d0 2024-05-20T18:28:33.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.732+0000 7fd5ea2bc640 1 -- 172.21.15.5:0/3948954064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd5e40b7bc0 con 0x7fd5e40cd4d0 2024-05-20T18:28:33.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.734+0000 7fd5e0ff9640 1 -- 172.21.15.5:0/3948954064 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16822 (secure 0 0 0) 0x7fd5d407a4c0 con 0x7fd5e40cd4d0 2024-05-20T18:28:33.734 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:33.734 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":40,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:28:25.902315+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:28:22.042995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-20T18:28:23.252728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:28:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.736+0000 7fd5ea2bc640 1 -- 172.21.15.5:0/3948954064 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd5d006be30 msgr2=0x7fd5d006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.736+0000 7fd5ea2bc640 1 --2- 172.21.15.5:0/3948954064 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd5d006be30 0x7fd5d006e340 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40cd240 tx=0x7fd5d8008040 comp rx=0 tx=0).stop 2024-05-20T18:28:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.736+0000 7fd5ea2bc640 1 -- 172.21.15.5:0/3948954064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5e40cd4d0 msgr2=0x7fd5e41f94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.736+0000 7fd5ea2bc640 1 --2- 172.21.15.5:0/3948954064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5e40cd4d0 0x7fd5e41f94f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd5d40097a0 tx=0x7fd5d40046c0 comp rx=0 tx=0).stop 2024-05-20T18:28:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.736+0000 7fd5ea2bc640 1 -- 172.21.15.5:0/3948954064 shutdown_connections 2024-05-20T18:28:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.736+0000 7fd5ea2bc640 1 --2- 172.21.15.5:0/3948954064 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd5d006be30 0x7fd5d006e340 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.736+0000 7fd5ea2bc640 1 --2- 172.21.15.5:0/3948954064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5e40d14f0 0x7fd5e41f8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.736+0000 7fd5ea2bc640 1 --2- 172.21.15.5:0/3948954064 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5e40d10e0 0x7fd5e4155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.736+0000 7fd5ea2bc640 1 --2- 172.21.15.5:0/3948954064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5e40cd4d0 0x7fd5e41f94f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:33.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.736+0000 7fd5ea2bc640 1 -- 172.21.15.5:0/3948954064 >> 172.21.15.5:0/3948954064 conn(0x7fd5e40b44d0 msgr2=0x7fd5e415fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.736+0000 7fd5ea2bc640 1 -- 172.21.15.5:0/3948954064 shutdown_connections 2024-05-20T18:28:33.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.736+0000 7fd5ea2bc640 1 -- 172.21.15.5:0/3948954064 wait complete. 2024-05-20T18:28:33.747 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:28:22.042995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:28:23.252728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:28:33.747 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:28:33.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.865+0000 7faec0b4b640 1 Processor -- start 2024-05-20T18:28:33.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.865+0000 7faec0b4b640 1 -- start start 2024-05-20T18:28:33.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.867+0000 7faec0b4b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faebc162820 0x7faebc0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:33.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.867+0000 7faec0b4b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7faebc0b3e80 0x7faebc06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:33.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.867+0000 7faec0b4b640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faebc06dd20 con 0x7faebc06d910 2024-05-20T18:28:33.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.867+0000 7faec0b4b640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7faebc0d0630 con 0x7faebc0b3e80 2024-05-20T18:28:33.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.867+0000 7faec0b4b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faebc0d39c0 con 0x7faebc162820 2024-05-20T18:28:33.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.867+0000 7faeba59c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faebc162820 0x7faebc0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:33.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.867+0000 7faeb9d9b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7faebc0b3e80 0x7faebc06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:33.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.868+0000 7faeba59c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faebc162820 0x7faebc0b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:43296/0 (socket says 172.21.15.5:43296) 2024-05-20T18:28:33.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.868+0000 7faeba59c640 1 -- 172.21.15.5:0/3316456535 learned_addr learned my addr 172.21.15.5:0/3316456535 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:33.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.868+0000 7faeba59c640 1 -- 172.21.15.5:0/3316456535 >> v2:172.21.15.63:3300/0 conn(0x7faebc0b3e80 msgr2=0x7faebc06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:33.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.868+0000 7faeba59c640 1 --2- 172.21.15.5:0/3316456535 >> v2:172.21.15.63:3300/0 conn(0x7faebc0b3e80 0x7faebc06d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:33.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.868+0000 7faeba59c640 1 -- 172.21.15.5:0/3316456535 >> v1:172.21.15.63:6789/0 conn(0x7faebc06d910 legacy=0x7faebc0d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:33.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.868+0000 7faeba59c640 1 -- 172.21.15.5:0/3316456535 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faebc0d3ca0 con 0x7faebc162820 2024-05-20T18:28:33.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.869+0000 7faeba59c640 1 --2- 172.21.15.5:0/3316456535 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faebc162820 0x7faebc0b3940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faeac009e50 tx=0x7faeac02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b82143683f4264f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:33.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.869+0000 7faeb959a640 1 -- 172.21.15.5:0/3316456535 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeac036070 con 0x7faebc162820 2024-05-20T18:28:33.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.869+0000 7faeb959a640 1 -- 172.21.15.5:0/3316456535 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faeac031600 con 0x7faebc162820 2024-05-20T18:28:33.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.869+0000 7faeb959a640 1 -- 172.21.15.5:0/3316456535 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeac0319d0 con 0x7faebc162820 2024-05-20T18:28:33.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.869+0000 7faec0b4b640 1 -- 172.21.15.5:0/3316456535 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faebc162820 msgr2=0x7faebc0b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:33.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.869+0000 7faec0b4b640 1 --2- 172.21.15.5:0/3316456535 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faebc162820 0x7faebc0b3940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faeac009e50 tx=0x7faeac02a710 comp rx=0 tx=0).stop 2024-05-20T18:28:33.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.870+0000 7faec0b4b640 1 -- 172.21.15.5:0/3316456535 shutdown_connections 2024-05-20T18:28:33.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.870+0000 7faec0b4b640 1 --2- 172.21.15.5:0/3316456535 >> v2:172.21.15.63:3300/0 conn(0x7faebc0b3e80 0x7faebc06d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:33.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.870+0000 7faec0b4b640 1 --2- 172.21.15.5:0/3316456535 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faebc162820 0x7faebc0b3940 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:33.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.870+0000 7faec0b4b640 1 -- 172.21.15.5:0/3316456535 >> 172.21.15.5:0/3316456535 conn(0x7faebc0d4bd0 msgr2=0x7faebc0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:33.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.870+0000 7faec0b4b640 1 -- 172.21.15.5:0/3316456535 shutdown_connections 2024-05-20T18:28:33.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.870+0000 7faec0b4b640 1 -- 172.21.15.5:0/3316456535 wait complete. 2024-05-20T18:28:33.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.872+0000 7faec0b4b640 1 Processor -- start 2024-05-20T18:28:33.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.872+0000 7faec0b4b640 1 -- start start 2024-05-20T18:28:33.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.873+0000 7faec0b4b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faebc06d910 0x7faebc1559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:33.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.873+0000 7faec0b4b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faebc0b3e80 0x7faebc157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:33.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.873+0000 7faec0b4b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faebc162820 0x7faebc157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:33.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.873+0000 7faec0b4b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faebc0d0300 con 0x7faebc06d910 2024-05-20T18:28:33.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.873+0000 7faec0b4b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faebc0d4260 con 0x7faebc0b3e80 2024-05-20T18:28:33.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.873+0000 7faec0b4b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faebc0b91a0 con 0x7faebc162820 2024-05-20T18:28:33.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.873+0000 7faeba59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faebc06d910 0x7faebc1559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:33.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.873+0000 7faeba59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faebc06d910 0x7faebc1559b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51766/0 (socket says 172.21.15.5:51766) 2024-05-20T18:28:33.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.873+0000 7faeba59c640 1 -- 172.21.15.5:0/3403360724 learned_addr learned my addr 172.21.15.5:0/3403360724 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:33.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.873+0000 7faeb9d9b640 1 --2- 172.21.15.5:0/3403360724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faebc0b3e80 0x7faebc157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:33.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.873+0000 7faebad9d640 1 --2- 172.21.15.5:0/3403360724 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faebc162820 0x7faebc157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:33.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.873+0000 7faeba59c640 1 -- 172.21.15.5:0/3403360724 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faebc162820 msgr2=0x7faebc157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:33.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.873+0000 7faeba59c640 1 --2- 172.21.15.5:0/3403360724 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faebc162820 0x7faebc157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:33.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.873+0000 7faeba59c640 1 -- 172.21.15.5:0/3403360724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faebc0b3e80 msgr2=0x7faebc157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:33.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.873+0000 7faeba59c640 1 --2- 172.21.15.5:0/3403360724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faebc0b3e80 0x7faebc157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:33.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.873+0000 7faeba59c640 1 -- 172.21.15.5:0/3403360724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faebc156290 con 0x7faebc06d910 2024-05-20T18:28:33.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.874+0000 7faeba59c640 1 --2- 172.21.15.5:0/3403360724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faebc06d910 0x7faebc1559b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7faeac034040 tx=0x7faeac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:33.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.874+0000 7faeaaffd640 1 -- 172.21.15.5:0/3403360724 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeac036050 con 0x7faebc06d910 2024-05-20T18:28:33.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.874+0000 7faeaaffd640 1 -- 172.21.15.5:0/3403360724 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faeac042070 con 0x7faebc06d910 2024-05-20T18:28:33.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.874+0000 7faec0b4b640 1 -- 172.21.15.5:0/3403360724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faebc156570 con 0x7faebc06d910 2024-05-20T18:28:33.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.874+0000 7faeaaffd640 1 -- 172.21.15.5:0/3403360724 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeac009510 con 0x7faebc06d910 2024-05-20T18:28:33.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.874+0000 7faec0b4b640 1 -- 172.21.15.5:0/3403360724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faebc15f120 con 0x7faebc06d910 2024-05-20T18:28:33.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.875+0000 7faec0b4b640 1 -- 172.21.15.5:0/3403360724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faebc075b00 con 0x7faebc06d910 2024-05-20T18:28:33.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.875+0000 7faeaaffd640 1 -- 172.21.15.5:0/3403360724 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faeac007c70 con 0x7faebc06d910 2024-05-20T18:28:33.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.876+0000 7faeaaffd640 1 --2- 172.21.15.5:0/3403360724 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae9c06be30 0x7fae9c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:33.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.876+0000 7faeaaffd640 1 -- 172.21.15.5:0/3403360724 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7faeac0abdf0 con 0x7faebc06d910 2024-05-20T18:28:33.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.876+0000 7faeb9d9b640 1 --2- 172.21.15.5:0/3403360724 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae9c06be30 0x7fae9c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:33.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.877+0000 7faeb9d9b640 1 --2- 172.21.15.5:0/3403360724 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae9c06be30 0x7fae9c06e340 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7faebc155560 tx=0x7faeb0009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:33.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:33.881+0000 7faeaaffd640 1 -- 172.21.15.5:0/3403360724 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7faeac079eb0 con 0x7faebc06d910 2024-05-20T18:28:34.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.066+0000 7faec0b4b640 1 -- 172.21.15.5:0/3403360724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faebc0b8e50 con 0x7faebc06d910 2024-05-20T18:28:34.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.066+0000 7faeaaffd640 1 -- 172.21.15.5:0/3403360724 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) ==== 93+0+10 (secure 0 0 0) 0x7faeac07e1b0 con 0x7faebc06d910 2024-05-20T18:28:34.066 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:28:34.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.068+0000 7faec0b4b640 1 -- 172.21.15.5:0/3403360724 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae9c06be30 msgr2=0x7fae9c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:34.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.068+0000 7faec0b4b640 1 --2- 172.21.15.5:0/3403360724 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae9c06be30 0x7fae9c06e340 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7faebc155560 tx=0x7faeb0009190 comp rx=0 tx=0).stop 2024-05-20T18:28:34.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.068+0000 7faec0b4b640 1 -- 172.21.15.5:0/3403360724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faebc06d910 msgr2=0x7faebc1559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:34.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.068+0000 7faec0b4b640 1 --2- 172.21.15.5:0/3403360724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faebc06d910 0x7faebc1559b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7faeac034040 tx=0x7faeac004a40 comp rx=0 tx=0).stop 2024-05-20T18:28:34.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.068+0000 7faec0b4b640 1 -- 172.21.15.5:0/3403360724 shutdown_connections 2024-05-20T18:28:34.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.068+0000 7faec0b4b640 1 --2- 172.21.15.5:0/3403360724 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae9c06be30 0x7fae9c06e340 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:34.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.068+0000 7faec0b4b640 1 --2- 172.21.15.5:0/3403360724 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faebc162820 0x7faebc157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:34.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.068+0000 7faec0b4b640 1 --2- 172.21.15.5:0/3403360724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faebc0b3e80 0x7faebc157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:34.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.068+0000 7faec0b4b640 1 --2- 172.21.15.5:0/3403360724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faebc06d910 0x7faebc1559b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:34.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.068+0000 7faec0b4b640 1 -- 172.21.15.5:0/3403360724 >> 172.21.15.5:0/3403360724 conn(0x7faebc0d4bd0 msgr2=0x7faebc160eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:34.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.068+0000 7faec0b4b640 1 -- 172.21.15.5:0/3403360724 shutdown_connections 2024-05-20T18:28:34.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.068+0000 7faec0b4b640 1 -- 172.21.15.5:0/3403360724 wait complete. 2024-05-20T18:28:34.079 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:28:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.199+0000 7ffb4a60d640 1 Processor -- start 2024-05-20T18:28:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.199+0000 7ffb4a60d640 1 -- start start 2024-05-20T18:28:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.200+0000 7ffb4a60d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ffb440b1e10 0x7ffb4415aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.200+0000 7ffb4a60d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb44070850 0x7ffb44070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.200+0000 7ffb4a60d640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb440cd580 con 0x7ffb4415d530 2024-05-20T18:28:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.200+0000 7ffb4a60d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ffb440ce190 con 0x7ffb440b1e10 2024-05-20T18:28:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.200+0000 7ffb4a60d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffb440cd860 con 0x7ffb44070850 2024-05-20T18:28:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.201+0000 7ffb48baa640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb44070850 0x7ffb44070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.201+0000 7ffb437fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ffb440b1e10 0x7ffb4415aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.201+0000 7ffb48baa640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb44070850 0x7ffb44070c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:43302/0 (socket says 172.21.15.5:43302) 2024-05-20T18:28:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.201+0000 7ffb48baa640 1 -- 172.21.15.5:0/1889115037 learned_addr learned my addr 172.21.15.5:0/1889115037 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.201+0000 7ffb437fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ffb440b1e10 0x7ffb4415aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:36690/0 (socket says 172.21.15.5:36690) 2024-05-20T18:28:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.201+0000 7ffb42ffd640 1 -- 172.21.15.5:0/1889115037 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 670340899 0 0) 0x7ffb440cd580 con 0x7ffb4415d530 2024-05-20T18:28:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.201+0000 7ffb42ffd640 1 -- 172.21.15.5:0/1889115037 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb2c003520 con 0x7ffb4415d530 2024-05-20T18:28:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.201+0000 7ffb48baa640 1 -- 172.21.15.5:0/1889115037 >> v2:172.21.15.63:3300/0 conn(0x7ffb440b1e10 msgr2=0x7ffb4415aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.201+0000 7ffb48baa640 1 --2- 172.21.15.5:0/1889115037 >> v2:172.21.15.63:3300/0 conn(0x7ffb440b1e10 0x7ffb4415aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.201+0000 7ffb48baa640 1 -- 172.21.15.5:0/1889115037 >> v1:172.21.15.63:6789/0 conn(0x7ffb4415d530 legacy=0x7ffb440b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.201+0000 7ffb48baa640 1 -- 172.21.15.5:0/1889115037 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb440cdb40 con 0x7ffb44070850 2024-05-20T18:28:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.201+0000 7ffb437fe640 1 --2- 172.21.15.5:0/1889115037 >> v2:172.21.15.63:3300/0 conn(0x7ffb440b1e10 0x7ffb4415aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:28:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.202+0000 7ffb48baa640 1 --2- 172.21.15.5:0/1889115037 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb44070850 0x7ffb44070c60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c009e50 tx=0x7ffb3c02a730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76fcf566d6b46712 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.202+0000 7ffb42ffd640 1 -- 172.21.15.5:0/1889115037 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c036070 con 0x7ffb44070850 2024-05-20T18:28:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.202+0000 7ffb42ffd640 1 -- 172.21.15.5:0/1889115037 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb3c031600 con 0x7ffb44070850 2024-05-20T18:28:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.202+0000 7ffb42ffd640 1 -- 172.21.15.5:0/1889115037 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c0319d0 con 0x7ffb44070850 2024-05-20T18:28:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.203+0000 7ffb4a60d640 1 -- 172.21.15.5:0/1889115037 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb44070850 msgr2=0x7ffb44070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.203+0000 7ffb4a60d640 1 --2- 172.21.15.5:0/1889115037 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb44070850 0x7ffb44070c60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c009e50 tx=0x7ffb3c02a730 comp rx=0 tx=0).stop 2024-05-20T18:28:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.203+0000 7ffb4a60d640 1 -- 172.21.15.5:0/1889115037 shutdown_connections 2024-05-20T18:28:34.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.203+0000 7ffb4a60d640 1 --2- 172.21.15.5:0/1889115037 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb44070850 0x7ffb44070c60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:34.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.203+0000 7ffb4a60d640 1 --2- 172.21.15.5:0/1889115037 >> v2:172.21.15.63:3300/0 conn(0x7ffb440b1e10 0x7ffb4415aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:34.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.203+0000 7ffb4a60d640 1 -- 172.21.15.5:0/1889115037 >> 172.21.15.5:0/1889115037 conn(0x7ffb440d29d0 msgr2=0x7ffb440d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:34.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.203+0000 7ffb4a60d640 1 -- 172.21.15.5:0/1889115037 shutdown_connections 2024-05-20T18:28:34.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.203+0000 7ffb4a60d640 1 -- 172.21.15.5:0/1889115037 wait complete. 2024-05-20T18:28:34.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.205+0000 7ffb4a60d640 1 Processor -- start 2024-05-20T18:28:34.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.205+0000 7ffb4a60d640 1 -- start start 2024-05-20T18:28:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.206+0000 7ffb4a60d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb44070850 0x7ffb4415b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.206+0000 7ffb4a60d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb440b1e10 0x7ffb4415b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.206+0000 7ffb4a60d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb4415d530 0x7ffb440caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.206+0000 7ffb4a60d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffb440711a0 con 0x7ffb440b1e10 2024-05-20T18:28:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.206+0000 7ffb4a60d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffb440cd230 con 0x7ffb4415d530 2024-05-20T18:28:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.206+0000 7ffb4a60d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffb44071320 con 0x7ffb44070850 2024-05-20T18:28:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.206+0000 7ffb437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb440b1e10 0x7ffb4415b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.206+0000 7ffb437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb440b1e10 0x7ffb4415b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51782/0 (socket says 172.21.15.5:51782) 2024-05-20T18:28:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.206+0000 7ffb437fe640 1 -- 172.21.15.5:0/1995122162 learned_addr learned my addr 172.21.15.5:0/1995122162 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.206+0000 7ffb48baa640 1 --2- 172.21.15.5:0/1995122162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb4415d530 0x7ffb440caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.206+0000 7ffb43fff640 1 --2- 172.21.15.5:0/1995122162 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb44070850 0x7ffb4415b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.207+0000 7ffb437fe640 1 -- 172.21.15.5:0/1995122162 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb44070850 msgr2=0x7ffb4415b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.207+0000 7ffb437fe640 1 --2- 172.21.15.5:0/1995122162 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb44070850 0x7ffb4415b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.207+0000 7ffb437fe640 1 -- 172.21.15.5:0/1995122162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb4415d530 msgr2=0x7ffb440caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.207+0000 7ffb437fe640 1 --2- 172.21.15.5:0/1995122162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb4415d530 0x7ffb440caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.207+0000 7ffb437fe640 1 -- 172.21.15.5:0/1995122162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb440c69d0 con 0x7ffb440b1e10 2024-05-20T18:28:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.207+0000 7ffb437fe640 1 --2- 172.21.15.5:0/1995122162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb440b1e10 0x7ffb4415b7a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ffb3800e990 tx=0x7ffb3800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.207+0000 7ffb40ff9640 1 -- 172.21.15.5:0/1995122162 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb38017070 con 0x7ffb440b1e10 2024-05-20T18:28:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.207+0000 7ffb40ff9640 1 -- 172.21.15.5:0/1995122162 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb3800c980 con 0x7ffb440b1e10 2024-05-20T18:28:34.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.207+0000 7ffb40ff9640 1 -- 172.21.15.5:0/1995122162 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3800cd50 con 0x7ffb440b1e10 2024-05-20T18:28:34.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.207+0000 7ffb4a60d640 1 -- 172.21.15.5:0/1995122162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb440c6ed0 con 0x7ffb440b1e10 2024-05-20T18:28:34.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.207+0000 7ffb4a60d640 1 -- 172.21.15.5:0/1995122162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb440c7490 con 0x7ffb440b1e10 2024-05-20T18:28:34.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.208+0000 7ffb4a60d640 1 -- 172.21.15.5:0/1995122162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb44075b00 con 0x7ffb440b1e10 2024-05-20T18:28:34.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.209+0000 7ffb40ff9640 1 -- 172.21.15.5:0/1995122162 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffb3801a070 con 0x7ffb440b1e10 2024-05-20T18:28:34.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.209+0000 7ffb40ff9640 1 --2- 172.21.15.5:0/1995122162 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffb3006be60 0x7ffb3006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:34.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.209+0000 7ffb40ff9640 1 -- 172.21.15.5:0/1995122162 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7ffb3808b290 con 0x7ffb440b1e10 2024-05-20T18:28:34.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.209+0000 7ffb43fff640 1 --2- 172.21.15.5:0/1995122162 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffb3006be60 0x7ffb3006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:34.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.210+0000 7ffb43fff640 1 --2- 172.21.15.5:0/1995122162 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffb3006be60 0x7ffb3006e370 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffb440d29a0 tx=0x7ffb34009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.214+0000 7ffb40ff9640 1 -- 172.21.15.5:0/1995122162 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ffb38059980 con 0x7ffb440b1e10 2024-05-20T18:28:34.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.395+0000 7ffb4a60d640 1 -- 172.21.15.5:0/1995122162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ffb440c67b0 con 0x7ffb440b1e10 2024-05-20T18:28:34.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.395+0000 7ffb40ff9640 1 -- 172.21.15.5:0/1995122162 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) ==== 104+0+11 (secure 0 0 0) 0x7ffb38059280 con 0x7ffb440b1e10 2024-05-20T18:28:34.395 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:28:34.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.397+0000 7ffb4a60d640 1 -- 172.21.15.5:0/1995122162 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffb3006be60 msgr2=0x7ffb3006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:34.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.397+0000 7ffb4a60d640 1 --2- 172.21.15.5:0/1995122162 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffb3006be60 0x7ffb3006e370 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffb440d29a0 tx=0x7ffb34009040 comp rx=0 tx=0).stop 2024-05-20T18:28:34.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.397+0000 7ffb4a60d640 1 -- 172.21.15.5:0/1995122162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb440b1e10 msgr2=0x7ffb4415b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:34.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.397+0000 7ffb4a60d640 1 --2- 172.21.15.5:0/1995122162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb440b1e10 0x7ffb4415b7a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ffb3800e990 tx=0x7ffb3800ee50 comp rx=0 tx=0).stop 2024-05-20T18:28:34.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.397+0000 7ffb4a60d640 1 -- 172.21.15.5:0/1995122162 shutdown_connections 2024-05-20T18:28:34.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.397+0000 7ffb4a60d640 1 --2- 172.21.15.5:0/1995122162 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffb3006be60 0x7ffb3006e370 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:34.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.397+0000 7ffb4a60d640 1 --2- 172.21.15.5:0/1995122162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb4415d530 0x7ffb440caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:34.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.397+0000 7ffb4a60d640 1 --2- 172.21.15.5:0/1995122162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb440b1e10 0x7ffb4415b7a0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:34.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.397+0000 7ffb4a60d640 1 --2- 172.21.15.5:0/1995122162 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb44070850 0x7ffb4415b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:34.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.397+0000 7ffb4a60d640 1 -- 172.21.15.5:0/1995122162 >> 172.21.15.5:0/1995122162 conn(0x7ffb440d29d0 msgr2=0x7ffb440b30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:34.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.397+0000 7ffb4a60d640 1 -- 172.21.15.5:0/1995122162 shutdown_connections 2024-05-20T18:28:34.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.397+0000 7ffb4a60d640 1 -- 172.21.15.5:0/1995122162 wait complete. 2024-05-20T18:28:34.409 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:28:34.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.526+0000 7fec4359e640 1 Processor -- start 2024-05-20T18:28:34.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.526+0000 7fec4359e640 1 -- start start 2024-05-20T18:28:34.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.528+0000 7fec4359e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fec440cf860 0x7fec440b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:34.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.528+0000 7fec4359e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec44163520 0x7fec440d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:34.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.528+0000 7fec4359e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec441605a0 con 0x7fec44163930 2024-05-20T18:28:34.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.528+0000 7fec4359e640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fec4415ffe0 con 0x7fec440cf860 2024-05-20T18:28:34.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.528+0000 7fec4359e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fec441602c0 con 0x7fec44163520 2024-05-20T18:28:34.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.528+0000 7fec42d9d640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fec44163930 0x7fec440d3140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53796/0 (socket says 172.21.15.5:53796) 2024-05-20T18:28:34.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.528+0000 7fec42d9d640 1 -- 172.21.15.5:0/3596863656 learned_addr learned my addr 172.21.15.5:0/3596863656 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:34.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.528+0000 7fec4259c640 1 --2- 172.21.15.5:0/3596863656 >> v2:172.21.15.63:3300/0 conn(0x7fec440cf860 0x7fec440b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:34.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.528+0000 7fec41d9b640 1 --2- 172.21.15.5:0/3596863656 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec44163520 0x7fec440d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:34.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.528+0000 7fec4159a640 1 -- 172.21.15.5:0/3596863656 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1316133258 0 0) 0x7fec441605a0 con 0x7fec44163930 2024-05-20T18:28:34.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.528+0000 7fec4159a640 1 -- 172.21.15.5:0/3596863656 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec2c003520 con 0x7fec44163930 2024-05-20T18:28:34.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.529+0000 7fec41d9b640 1 -- 172.21.15.5:0/3596863656 >> v2:172.21.15.63:3300/0 conn(0x7fec440cf860 msgr2=0x7fec440b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:34.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.529+0000 7fec41d9b640 1 --2- 172.21.15.5:0/3596863656 >> v2:172.21.15.63:3300/0 conn(0x7fec440cf860 0x7fec440b3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:34.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.529+0000 7fec41d9b640 1 -- 172.21.15.5:0/3596863656 >> v1:172.21.15.5:6789/0 conn(0x7fec44163930 legacy=0x7fec440d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:34.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.529+0000 7fec41d9b640 1 -- 172.21.15.5:0/3596863656 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec44162d20 con 0x7fec44163520 2024-05-20T18:28:34.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.529+0000 7fec4259c640 1 --2- 172.21.15.5:0/3596863656 >> v2:172.21.15.63:3300/0 conn(0x7fec440cf860 0x7fec440b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:28:34.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.530+0000 7fec41d9b640 1 --2- 172.21.15.5:0/3596863656 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec44163520 0x7fec440d12a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fec380097a0 tx=0x7fec3802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f8cdab912a53cac server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:34.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.530+0000 7fec4159a640 1 -- 172.21.15.5:0/3596863656 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38037070 con 0x7fec44163520 2024-05-20T18:28:34.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.530+0000 7fec4159a640 1 -- 172.21.15.5:0/3596863656 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec3802ae50 con 0x7fec44163520 2024-05-20T18:28:34.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.530+0000 7fec4159a640 1 -- 172.21.15.5:0/3596863656 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38032660 con 0x7fec44163520 2024-05-20T18:28:34.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.530+0000 7fec4359e640 1 -- 172.21.15.5:0/3596863656 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec44163520 msgr2=0x7fec440d12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:34.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.530+0000 7fec4359e640 1 --2- 172.21.15.5:0/3596863656 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec44163520 0x7fec440d12a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fec380097a0 tx=0x7fec3802a100 comp rx=0 tx=0).stop 2024-05-20T18:28:34.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.531+0000 7fec4359e640 1 -- 172.21.15.5:0/3596863656 shutdown_connections 2024-05-20T18:28:34.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.531+0000 7fec4359e640 1 --2- 172.21.15.5:0/3596863656 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec44163520 0x7fec440d12a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:34.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.531+0000 7fec4359e640 1 --2- 172.21.15.5:0/3596863656 >> v2:172.21.15.63:3300/0 conn(0x7fec440cf860 0x7fec440b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:34.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.531+0000 7fec4359e640 1 -- 172.21.15.5:0/3596863656 >> 172.21.15.5:0/3596863656 conn(0x7fec4406dbb0 msgr2=0x7fec4406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:34.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.531+0000 7fec4359e640 1 -- 172.21.15.5:0/3596863656 shutdown_connections 2024-05-20T18:28:34.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.531+0000 7fec4359e640 1 -- 172.21.15.5:0/3596863656 wait complete. 2024-05-20T18:28:34.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.532+0000 7fec4359e640 1 Processor -- start 2024-05-20T18:28:34.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.533+0000 7fec4359e640 1 -- start start 2024-05-20T18:28:34.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.534+0000 7fec4359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec440cf860 0x7fec440b4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:34.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.534+0000 7fec4359e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec44163520 0x7fec440b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:34.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.534+0000 7fec4359e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec44163930 0x7fec441f8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:34.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.534+0000 7fec4359e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fec440d39d0 con 0x7fec440cf860 2024-05-20T18:28:34.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.534+0000 7fec4359e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fec440d3850 con 0x7fec44163930 2024-05-20T18:28:34.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.534+0000 7fec4359e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fec440d3b50 con 0x7fec44163520 2024-05-20T18:28:34.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.534+0000 7fec4259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec440cf860 0x7fec440b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:34.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.534+0000 7fec4259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec440cf860 0x7fec440b4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51784/0 (socket says 172.21.15.5:51784) 2024-05-20T18:28:34.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.534+0000 7fec4259c640 1 -- 172.21.15.5:0/2206931554 learned_addr learned my addr 172.21.15.5:0/2206931554 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:34.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.534+0000 7fec42d9d640 1 --2- 172.21.15.5:0/2206931554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec44163930 0x7fec441f8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:34.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.534+0000 7fec41d9b640 1 --2- 172.21.15.5:0/2206931554 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec44163520 0x7fec440b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:34.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.534+0000 7fec4259c640 1 -- 172.21.15.5:0/2206931554 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec44163520 msgr2=0x7fec440b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:34.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.534+0000 7fec4259c640 1 --2- 172.21.15.5:0/2206931554 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec44163520 0x7fec440b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:34.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.534+0000 7fec4259c640 1 -- 172.21.15.5:0/2206931554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec44163930 msgr2=0x7fec441f8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:34.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.534+0000 7fec4259c640 1 --2- 172.21.15.5:0/2206931554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec44163930 0x7fec441f8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:34.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.534+0000 7fec4259c640 1 -- 172.21.15.5:0/2206931554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec441e8b80 con 0x7fec440cf860 2024-05-20T18:28:34.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.534+0000 7fec41d9b640 1 --2- 172.21.15.5:0/2206931554 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec44163520 0x7fec440b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:34.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.535+0000 7fec4259c640 1 --2- 172.21.15.5:0/2206931554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec440cf860 0x7fec440b4970 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fec34009e60 tx=0x7fec3400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:34.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.535+0000 7fec32ffd640 1 -- 172.21.15.5:0/2206931554 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34018020 con 0x7fec440cf860 2024-05-20T18:28:34.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.535+0000 7fec32ffd640 1 -- 172.21.15.5:0/2206931554 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec34014680 con 0x7fec440cf860 2024-05-20T18:28:34.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.535+0000 7fec4359e640 1 -- 172.21.15.5:0/2206931554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec441e8e60 con 0x7fec440cf860 2024-05-20T18:28:34.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.535+0000 7fec32ffd640 1 -- 172.21.15.5:0/2206931554 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34014a50 con 0x7fec440cf860 2024-05-20T18:28:34.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.535+0000 7fec4359e640 1 -- 172.21.15.5:0/2206931554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec440b55b0 con 0x7fec440cf860 2024-05-20T18:28:34.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.536+0000 7fec4359e640 1 -- 172.21.15.5:0/2206931554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec44075980 con 0x7fec440cf860 2024-05-20T18:28:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.536+0000 7fec32ffd640 1 -- 172.21.15.5:0/2206931554 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fec34005d10 con 0x7fec440cf860 2024-05-20T18:28:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.537+0000 7fec32ffd640 1 --2- 172.21.15.5:0/2206931554 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fec2406be30 0x7fec2406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.537+0000 7fec32ffd640 1 -- 172.21.15.5:0/2206931554 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fec3408bc60 con 0x7fec440cf860 2024-05-20T18:28:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.537+0000 7fec41d9b640 1 --2- 172.21.15.5:0/2206931554 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fec2406be30 0x7fec2406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.537+0000 7fec41d9b640 1 --2- 172.21.15.5:0/2206931554 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fec2406be30 0x7fec2406e340 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fec38034000 tx=0x7fec3802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:34.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.541+0000 7fec32ffd640 1 -- 172.21.15.5:0/2206931554 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fec34022030 con 0x7fec440cf860 2024-05-20T18:28:34.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.725+0000 7fec4359e640 1 -- 172.21.15.5:0/2206931554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fec440d2f20 con 0x7fec440cf860 2024-05-20T18:28:34.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.725+0000 7fec32ffd640 1 -- 172.21.15.5:0/2206931554 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) ==== 100+0+11 (secure 0 0 0) 0x7fec34010040 con 0x7fec440cf860 2024-05-20T18:28:34.726 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:28:34.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.727+0000 7fec4359e640 1 -- 172.21.15.5:0/2206931554 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fec2406be30 msgr2=0x7fec2406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:34.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.727+0000 7fec4359e640 1 --2- 172.21.15.5:0/2206931554 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fec2406be30 0x7fec2406e340 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fec38034000 tx=0x7fec3802e000 comp rx=0 tx=0).stop 2024-05-20T18:28:34.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.727+0000 7fec4359e640 1 -- 172.21.15.5:0/2206931554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec440cf860 msgr2=0x7fec440b4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:34.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.727+0000 7fec4359e640 1 --2- 172.21.15.5:0/2206931554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec440cf860 0x7fec440b4970 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fec34009e60 tx=0x7fec3400c490 comp rx=0 tx=0).stop 2024-05-20T18:28:34.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.727+0000 7fec4359e640 1 -- 172.21.15.5:0/2206931554 shutdown_connections 2024-05-20T18:28:34.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.727+0000 7fec4359e640 1 --2- 172.21.15.5:0/2206931554 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fec2406be30 0x7fec2406e340 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:34.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.727+0000 7fec4359e640 1 --2- 172.21.15.5:0/2206931554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec44163930 0x7fec441f8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:34.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.727+0000 7fec4359e640 1 --2- 172.21.15.5:0/2206931554 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec44163520 0x7fec440b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:34.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.727+0000 7fec4359e640 1 --2- 172.21.15.5:0/2206931554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec440cf860 0x7fec440b4970 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:34.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.727+0000 7fec4359e640 1 -- 172.21.15.5:0/2206931554 >> 172.21.15.5:0/2206931554 conn(0x7fec4406dbb0 msgr2=0x7fec440b8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:34.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.728+0000 7fec4359e640 1 -- 172.21.15.5:0/2206931554 shutdown_connections 2024-05-20T18:28:34.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.728+0000 7fec4359e640 1 -- 172.21.15.5:0/2206931554 wait complete. 2024-05-20T18:28:34.739 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T18:28:34.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.855+0000 7f27cb8ca640 1 Processor -- start 2024-05-20T18:28:34.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.855+0000 7f27cb8ca640 1 -- start start 2024-05-20T18:28:34.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.856+0000 7f27cb8ca640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27c4162820 0x7f27c40b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:34.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.856+0000 7f27cb8ca640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f27c406dae0 0x7f27c40b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:34.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.856+0000 7f27cb8ca640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27c40d0630 con 0x7f27c40b3e80 2024-05-20T18:28:34.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.856+0000 7f27cb8ca640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f27c40d39c0 con 0x7f27c406dae0 2024-05-20T18:28:34.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.856+0000 7f27cb8ca640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f27c40d3ca0 con 0x7f27c4162820 2024-05-20T18:28:34.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.857+0000 7f27c9e67640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f27c406dae0 0x7f27c40b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:34.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.857+0000 7f27c9666640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27c4162820 0x7f27c40b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:34.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.857+0000 7f27c9e67640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f27c406dae0 0x7f27c40b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:34622/0 (socket says 172.21.15.5:34622) 2024-05-20T18:28:34.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.857+0000 7f27c8e65640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f27c40b3e80 0x7f27c406d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:50874/0 (socket says 172.21.15.5:50874) 2024-05-20T18:28:34.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.857+0000 7f27c9666640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27c4162820 0x7f27c40b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:60482/0 (socket says 172.21.15.5:60482) 2024-05-20T18:28:34.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.857+0000 7f27c9e67640 1 -- 172.21.15.5:0/384678918 learned_addr learned my addr 172.21.15.5:0/384678918 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:34.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.857+0000 7f27c9e67640 1 -- 172.21.15.5:0/384678918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27c4162820 msgr2=0x7f27c40b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:34.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.857+0000 7f27c9e67640 1 --2- 172.21.15.5:0/384678918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27c4162820 0x7f27c40b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:34.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.857+0000 7f27c9e67640 1 -- 172.21.15.5:0/384678918 >> v1:172.21.15.63:6789/0 conn(0x7f27c40b3e80 legacy=0x7f27c406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:34.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.857+0000 7f27c9e67640 1 -- 172.21.15.5:0/384678918 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c40d3f80 con 0x7f27c406dae0 2024-05-20T18:28:34.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.857+0000 7f27b7fff640 1 -- 172.21.15.5:0/384678918 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 164471174 0 0) 0x7f27c40d0630 con 0x7f27c40b3e80 2024-05-20T18:28:34.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.857+0000 7f27c9666640 1 --2- 172.21.15.5:0/384678918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27c4162820 0x7f27c40b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:28:34.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.858+0000 7f27c9e67640 1 --2- 172.21.15.5:0/384678918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27c406dae0 0x7f27c40b8190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009e50 tx=0x7f27bc02a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b5def737aafaf9c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:34.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.858+0000 7f27b7fff640 1 -- 172.21.15.5:0/384678918 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27bc036070 con 0x7f27c406dae0 2024-05-20T18:28:34.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.858+0000 7f27b7fff640 1 -- 172.21.15.5:0/384678918 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27bc031600 con 0x7f27c406dae0 2024-05-20T18:28:34.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.858+0000 7f27b7fff640 1 -- 172.21.15.5:0/384678918 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27bc0319d0 con 0x7f27c406dae0 2024-05-20T18:28:34.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.859+0000 7f27cb8ca640 1 -- 172.21.15.5:0/384678918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27c406dae0 msgr2=0x7f27c40b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:34.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.859+0000 7f27cb8ca640 1 --2- 172.21.15.5:0/384678918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27c406dae0 0x7f27c40b8190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009e50 tx=0x7f27bc02a710 comp rx=0 tx=0).stop 2024-05-20T18:28:34.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.859+0000 7f27cb8ca640 1 -- 172.21.15.5:0/384678918 shutdown_connections 2024-05-20T18:28:34.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.859+0000 7f27cb8ca640 1 --2- 172.21.15.5:0/384678918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27c406dae0 0x7f27c40b8190 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:34.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.859+0000 7f27cb8ca640 1 --2- 172.21.15.5:0/384678918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27c4162820 0x7f27c40b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:34.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.859+0000 7f27cb8ca640 1 -- 172.21.15.5:0/384678918 >> 172.21.15.5:0/384678918 conn(0x7f27c40d4bd0 msgr2=0x7f27c40d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:34.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.859+0000 7f27cb8ca640 1 -- 172.21.15.5:0/384678918 shutdown_connections 2024-05-20T18:28:34.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.859+0000 7f27cb8ca640 1 -- 172.21.15.5:0/384678918 wait complete. 2024-05-20T18:28:34.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.861+0000 7f27cb8ca640 1 Processor -- start 2024-05-20T18:28:34.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.861+0000 7f27cb8ca640 1 -- start start 2024-05-20T18:28:34.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.862+0000 7f27cb8ca640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27c40b3e80 0x7f27c41f59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:34.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.862+0000 7f27cb8ca640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27c4162820 0x7f27c41588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:34.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.862+0000 7f27cb8ca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27c41560f0 0x7f27c415ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:34.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.862+0000 7f27cb8ca640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f27c40d4780 con 0x7f27c41560f0 2024-05-20T18:28:34.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.862+0000 7f27cb8ca640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f27c40d0300 con 0x7f27c40b3e80 2024-05-20T18:28:34.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.862+0000 7f27cb8ca640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f27c40d4900 con 0x7f27c4162820 2024-05-20T18:28:34.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.862+0000 7f27c9e67640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27c41560f0 0x7f27c415ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:34.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.862+0000 7f27c9e67640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27c41560f0 0x7f27c415ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48542/0 (socket says 172.21.15.5:48542) 2024-05-20T18:28:34.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.862+0000 7f27c9e67640 1 -- 172.21.15.5:0/4292820149 learned_addr learned my addr 172.21.15.5:0/4292820149 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:34.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.863+0000 7f27c9666640 1 --2- 172.21.15.5:0/4292820149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27c40b3e80 0x7f27c41f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:34.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.863+0000 7f27c8e65640 1 --2- 172.21.15.5:0/4292820149 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27c4162820 0x7f27c41588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:34.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.863+0000 7f27c9e67640 1 -- 172.21.15.5:0/4292820149 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27c4162820 msgr2=0x7f27c41588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:34.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.863+0000 7f27c9e67640 1 --2- 172.21.15.5:0/4292820149 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27c4162820 0x7f27c41588c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:34.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.863+0000 7f27c9e67640 1 -- 172.21.15.5:0/4292820149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27c40b3e80 msgr2=0x7f27c41f59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:34.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.863+0000 7f27c9e67640 1 --2- 172.21.15.5:0/4292820149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27c40b3e80 0x7f27c41f59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:34.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.863+0000 7f27c9e67640 1 -- 172.21.15.5:0/4292820149 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c40c8950 con 0x7f27c41560f0 2024-05-20T18:28:34.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.863+0000 7f27c8e65640 1 --2- 172.21.15.5:0/4292820149 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27c4162820 0x7f27c41588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:34.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.863+0000 7f27c9e67640 1 --2- 172.21.15.5:0/4292820149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27c41560f0 0x7f27c415ffa0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009e50 tx=0x7f27bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:34.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.863+0000 7f27b5ffb640 1 -- 172.21.15.5:0/4292820149 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27bc036050 con 0x7f27c41560f0 2024-05-20T18:28:34.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.863+0000 7f27b5ffb640 1 -- 172.21.15.5:0/4292820149 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27bc042070 con 0x7f27c41560f0 2024-05-20T18:28:34.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.863+0000 7f27cb8ca640 1 -- 172.21.15.5:0/4292820149 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27c40c8c30 con 0x7f27c41560f0 2024-05-20T18:28:34.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.863+0000 7f27b5ffb640 1 -- 172.21.15.5:0/4292820149 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27bc009510 con 0x7f27c41560f0 2024-05-20T18:28:34.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.863+0000 7f27cb8ca640 1 -- 172.21.15.5:0/4292820149 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27c40c91f0 con 0x7f27c41560f0 2024-05-20T18:28:34.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.864+0000 7f27cb8ca640 1 -- 172.21.15.5:0/4292820149 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27c4075b00 con 0x7f27c41560f0 2024-05-20T18:28:34.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.865+0000 7f27b5ffb640 1 -- 172.21.15.5:0/4292820149 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f27bc007bd0 con 0x7f27c41560f0 2024-05-20T18:28:34.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.865+0000 7f27b5ffb640 1 --2- 172.21.15.5:0/4292820149 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f27a806be30 0x7f27a806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:34.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.866+0000 7f27b5ffb640 1 -- 172.21.15.5:0/4292820149 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f27bc0abed0 con 0x7f27c41560f0 2024-05-20T18:28:34.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.866+0000 7f27c9666640 1 --2- 172.21.15.5:0/4292820149 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f27a806be30 0x7f27a806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:34.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.866+0000 7f27c9666640 1 --2- 172.21.15.5:0/4292820149 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f27a806be30 0x7f27a806e340 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009a30 tx=0x7f27b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:34.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:34.870+0000 7f27b5ffb640 1 -- 172.21.15.5:0/4292820149 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f27bc07a660 con 0x7f27c41560f0 2024-05-20T18:28:35.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.067+0000 7f27cb8ca640 1 -- 172.21.15.5:0/4292820149 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f27c415f7a0 con 0x7f27c41560f0 2024-05-20T18:28:35.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.068+0000 7f27b5ffb640 1 -- 172.21.15.5:0/4292820149 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v40) ==== 80+0+0 (secure 0 0 0) 0x7f27bc079f60 con 0x7f27c41560f0 2024-05-20T18:28:35.068 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-05-20T18:28:35.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.070+0000 7f27cb8ca640 1 -- 172.21.15.5:0/4292820149 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f27a806be30 msgr2=0x7f27a806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:35.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.070+0000 7f27cb8ca640 1 --2- 172.21.15.5:0/4292820149 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f27a806be30 0x7f27a806e340 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009a30 tx=0x7f27b8008040 comp rx=0 tx=0).stop 2024-05-20T18:28:35.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.070+0000 7f27cb8ca640 1 -- 172.21.15.5:0/4292820149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27c41560f0 msgr2=0x7f27c415ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:35.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.070+0000 7f27cb8ca640 1 --2- 172.21.15.5:0/4292820149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27c41560f0 0x7f27c415ffa0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009e50 tx=0x7f27bc004a40 comp rx=0 tx=0).stop 2024-05-20T18:28:35.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.070+0000 7f27cb8ca640 1 -- 172.21.15.5:0/4292820149 shutdown_connections 2024-05-20T18:28:35.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.070+0000 7f27cb8ca640 1 --2- 172.21.15.5:0/4292820149 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f27a806be30 0x7f27a806e340 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:35.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.070+0000 7f27cb8ca640 1 --2- 172.21.15.5:0/4292820149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27c41560f0 0x7f27c415ffa0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:35.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.070+0000 7f27cb8ca640 1 --2- 172.21.15.5:0/4292820149 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27c4162820 0x7f27c41588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:35.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.070+0000 7f27cb8ca640 1 --2- 172.21.15.5:0/4292820149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27c40b3e80 0x7f27c41f59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:35.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.070+0000 7f27cb8ca640 1 -- 172.21.15.5:0/4292820149 >> 172.21.15.5:0/4292820149 conn(0x7f27c40d4bd0 msgr2=0x7f27c40ba1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:35.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.070+0000 7f27cb8ca640 1 -- 172.21.15.5:0/4292820149 shutdown_connections 2024-05-20T18:28:35.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.070+0000 7f27cb8ca640 1 -- 172.21.15.5:0/4292820149 wait complete. 2024-05-20T18:28:35.084 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-20T18:28:35.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:28:35.084 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-20T18:28:35.135 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:28:35.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:28:35.135 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-20T18:28:35.160 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:28:35.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.316+0000 7f05d493e640 1 Processor -- start 2024-05-20T18:28:35.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.316+0000 7f05d493e640 1 -- start start 2024-05-20T18:28:35.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.317+0000 7f05d493e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05d015d530 0x7f05d015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:35.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.317+0000 7f05d493e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05d00cd580 con 0x7f05d00b1700 2024-05-20T18:28:35.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.317+0000 7f05d493e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05d00cd860 con 0x7f05d0070850 2024-05-20T18:28:35.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.317+0000 7f05d493e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f05d00ce190 con 0x7f05d015d530 2024-05-20T18:28:35.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.318+0000 7f05cdd9b640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f05d00b1700 0x7f05d00b1b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:52308/0 (socket says 172.21.15.5:52308) 2024-05-20T18:28:35.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.318+0000 7f05cdd9b640 1 -- 172.21.15.5:0/852569832 learned_addr learned my addr 172.21.15.5:0/852569832 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:35.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.318+0000 7f05cd59a640 1 -- 172.21.15.5:0/852569832 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 358664981 0 0) 0x7f05d00cd580 con 0x7f05d00b1700 2024-05-20T18:28:35.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.318+0000 7f05ce59c640 1 --2- 172.21.15.5:0/852569832 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05d015d530 0x7f05d015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:35.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.318+0000 7f05cd59a640 1 -- 172.21.15.5:0/852569832 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05b8003520 con 0x7f05d00b1700 2024-05-20T18:28:35.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.318+0000 7f05cd59a640 1 -- 172.21.15.5:0/852569832 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3857407268 0 0) 0x7f05b8003520 con 0x7f05d00b1700 2024-05-20T18:28:35.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.318+0000 7f05cd59a640 1 -- 172.21.15.5:0/852569832 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05d015d530 msgr2=0x7f05d015aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:35.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.318+0000 7f05cd59a640 1 --2- 172.21.15.5:0/852569832 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05d015d530 0x7f05d015aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:35.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.318+0000 7f05cd59a640 1 -- 172.21.15.5:0/852569832 >> v1:172.21.15.63:6789/0 conn(0x7f05d0070850 legacy=0x7f05d0070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:35.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.318+0000 7f05cd59a640 1 -- 172.21.15.5:0/852569832 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d00cdb40 con 0x7f05d00b1700 2024-05-20T18:28:35.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.318+0000 7f05cd59a640 1 -- 172.21.15.5:0/852569832 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f05c4001670 con 0x7f05d00b1700 2024-05-20T18:28:35.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.318+0000 7f05cd59a640 1 -- 172.21.15.5:0/852569832 >> v1:172.21.15.5:6789/0 conn(0x7f05d00b1700 legacy=0x7f05d00b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:35.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.318+0000 7f05cd59a640 1 --2- 172.21.15.5:0/852569832 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05d015d530 0x7f05b80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:35.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.318+0000 7f05cd59a640 1 --2- 172.21.15.5:0/852569832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05b8003f60 0x7f05b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:35.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.318+0000 7f05cd59a640 1 --2- 172.21.15.5:0/852569832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05b8004d40 0x7f05b8005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:35.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.318+0000 7f05cd59a640 1 -- 172.21.15.5:0/852569832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f05d00ce190 con 0x7f05b8004d40 2024-05-20T18:28:35.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.318+0000 7f05cd59a640 1 -- 172.21.15.5:0/852569832 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f05b800bb50 con 0x7f05b8003f60 2024-05-20T18:28:35.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.318+0000 7f05cd59a640 1 -- 172.21.15.5:0/852569832 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f05b800bd50 con 0x7f05d015d530 2024-05-20T18:28:35.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.318+0000 7f05ced9d640 1 --2- 172.21.15.5:0/852569832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05b8004d40 0x7f05b8005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:35.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.318+0000 7f05ce59c640 1 --2- 172.21.15.5:0/852569832 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05d015d530 0x7f05b80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:35.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.318+0000 7f05cdd9b640 1 --2- 172.21.15.5:0/852569832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05b8003f60 0x7f05b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:35.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.319+0000 7f05ced9d640 1 -- 172.21.15.5:0/852569832 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05d015d530 msgr2=0x7f05b80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:35.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.319+0000 7f05ced9d640 1 --2- 172.21.15.5:0/852569832 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05d015d530 0x7f05b80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:35.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.319+0000 7f05ced9d640 1 -- 172.21.15.5:0/852569832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05b8003f60 msgr2=0x7f05b8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:35.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.319+0000 7f05ced9d640 1 --2- 172.21.15.5:0/852569832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05b8003f60 0x7f05b8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:35.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.319+0000 7f05ced9d640 1 -- 172.21.15.5:0/852569832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b8003520 con 0x7f05b8004d40 2024-05-20T18:28:35.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.319+0000 7f05ced9d640 1 --2- 172.21.15.5:0/852569832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05b8004d40 0x7f05b8005150 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f05d00b8420 tx=0x7f05b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a20ec39069136f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:35.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.320+0000 7f05cd59a640 1 -- 172.21.15.5:0/852569832 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c8001a70 con 0x7f05b8004d40 2024-05-20T18:28:35.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.320+0000 7f05cd59a640 1 -- 172.21.15.5:0/852569832 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05c802ae10 con 0x7f05b8004d40 2024-05-20T18:28:35.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.320+0000 7f05cd59a640 1 -- 172.21.15.5:0/852569832 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c8003bb0 con 0x7f05b8004d40 2024-05-20T18:28:35.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.320+0000 7f05d493e640 1 -- 172.21.15.5:0/852569832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05b8004d40 msgr2=0x7f05b8005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:35.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.320+0000 7f05d493e640 1 --2- 172.21.15.5:0/852569832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05b8004d40 0x7f05b8005150 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f05d00b8420 tx=0x7f05b8005830 comp rx=0 tx=0).stop 2024-05-20T18:28:35.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.320+0000 7f05ced9d640 1 -- 172.21.15.5:0/852569832 reap_dead start 2024-05-20T18:28:35.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.320+0000 7f05d493e640 1 -- 172.21.15.5:0/852569832 shutdown_connections 2024-05-20T18:28:35.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.320+0000 7f05d493e640 1 -- 172.21.15.5:0/852569832 >> 172.21.15.5:0/852569832 conn(0x7f05d00d29d0 msgr2=0x7f05d00d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:35.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.320+0000 7f05d493e640 1 -- 172.21.15.5:0/852569832 shutdown_connections 2024-05-20T18:28:35.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.320+0000 7f05d493e640 1 -- 172.21.15.5:0/852569832 wait complete. 2024-05-20T18:28:35.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.322+0000 7f05d493e640 1 Processor -- start 2024-05-20T18:28:35.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.322+0000 7f05d493e640 1 -- start start 2024-05-20T18:28:35.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.324+0000 7f05d493e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05b8004d40 0x7f05d00b6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:35.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.324+0000 7f05d493e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05d00c6f20 0x7f05d00cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:35.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.324+0000 7f05d493e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05d00c87c0 0x7f05d00c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:35.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.324+0000 7f05d493e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f05d0071370 con 0x7f05b8004d40 2024-05-20T18:28:35.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.324+0000 7f05d493e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f05d00cd230 con 0x7f05d00c87c0 2024-05-20T18:28:35.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.324+0000 7f05d493e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f05d00714f0 con 0x7f05d00c6f20 2024-05-20T18:28:35.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.324+0000 7f05ce59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05b8004d40 0x7f05d00b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:35.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.324+0000 7f05ce59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05b8004d40 0x7f05d00b6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48562/0 (socket says 172.21.15.5:48562) 2024-05-20T18:28:35.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.324+0000 7f05ce59c640 1 -- 172.21.15.5:0/1721589217 learned_addr learned my addr 172.21.15.5:0/1721589217 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:35.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.324+0000 7f05ced9d640 1 --2- 172.21.15.5:0/1721589217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05d00c87c0 0x7f05d00c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:35.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.324+0000 7f05cdd9b640 1 --2- 172.21.15.5:0/1721589217 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05d00c6f20 0x7f05d00cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:35.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.324+0000 7f05ce59c640 1 -- 172.21.15.5:0/1721589217 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05d00c6f20 msgr2=0x7f05d00cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:35.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.324+0000 7f05ce59c640 1 --2- 172.21.15.5:0/1721589217 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05d00c6f20 0x7f05d00cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:35.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.324+0000 7f05ce59c640 1 -- 172.21.15.5:0/1721589217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05d00c87c0 msgr2=0x7f05d00c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:35.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.324+0000 7f05ce59c640 1 --2- 172.21.15.5:0/1721589217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05d00c87c0 0x7f05d00c8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:35.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.324+0000 7f05ce59c640 1 -- 172.21.15.5:0/1721589217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d00c9110 con 0x7f05b8004d40 2024-05-20T18:28:35.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.324+0000 7f05ced9d640 1 --2- 172.21.15.5:0/1721589217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05d00c87c0 0x7f05d00c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:35.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.324+0000 7f05ce59c640 1 --2- 172.21.15.5:0/1721589217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05b8004d40 0x7f05d00b6660 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f05c0011910 tx=0x7f05c0011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:35.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.325+0000 7f05beffd640 1 -- 172.21.15.5:0/1721589217 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c000faf0 con 0x7f05b8004d40 2024-05-20T18:28:35.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.325+0000 7f05beffd640 1 -- 172.21.15.5:0/1721589217 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05c000fc90 con 0x7f05b8004d40 2024-05-20T18:28:35.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.325+0000 7f05d493e640 1 -- 172.21.15.5:0/1721589217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05d00cc710 con 0x7f05b8004d40 2024-05-20T18:28:35.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.325+0000 7f05beffd640 1 -- 172.21.15.5:0/1721589217 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c0014440 con 0x7f05b8004d40 2024-05-20T18:28:35.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.325+0000 7f05d493e640 1 -- 172.21.15.5:0/1721589217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05d00b7270 con 0x7f05b8004d40 2024-05-20T18:28:35.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.326+0000 7f05d493e640 1 -- 172.21.15.5:0/1721589217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05d0075b00 con 0x7f05b8004d40 2024-05-20T18:28:35.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.326+0000 7f05beffd640 1 -- 172.21.15.5:0/1721589217 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f05c0012040 con 0x7f05b8004d40 2024-05-20T18:28:35.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.327+0000 7f05beffd640 1 --2- 172.21.15.5:0/1721589217 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f05b006be30 0x7f05b006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:35.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.327+0000 7f05beffd640 1 -- 172.21.15.5:0/1721589217 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f05c008b750 con 0x7f05b8004d40 2024-05-20T18:28:35.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.327+0000 7f05cdd9b640 1 --2- 172.21.15.5:0/1721589217 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f05b006be30 0x7f05b006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:35.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.328+0000 7f05cdd9b640 1 --2- 172.21.15.5:0/1721589217 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f05b006be30 0x7f05b006e340 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f05d01e9530 tx=0x7f05c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:35.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.332+0000 7f05beffd640 1 -- 172.21.15.5:0/1721589217 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f05d00c9110 con 0x7f05b8004d40 2024-05-20T18:28:35.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.559+0000 7f05d493e640 1 -- 172.21.15.5:0/1721589217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05d00d1790 con 0x7f05b8004d40 2024-05-20T18:28:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.559+0000 7f05beffd640 1 -- 172.21.15.5:0/1721589217 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5175 (secure 0 0 0) 0x7f05c0019030 con 0x7f05b8004d40 2024-05-20T18:28:35.559 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:35.559 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1458170078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1458170078},{"type":"v1","addr":"172.21.15.63:6834","nonce":1458170078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4916,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/2601750663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":2601750663},{"type":"v1","addr":"172.21.15.63:6835","nonce":2601750663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5313,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/1913030219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1913030219},{"type":"v1","addr":"172.21.15.5:6836","nonce":1913030219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:28:24.270282+0000","modified":"2024-05-20T18:28:27.296534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5316},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5316":{"gid":5316,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6837/2355805105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":2355805105},{"type":"v1","addr":"172.21.15.5:6837","nonce":2355805105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5316,"qdb_cluster":[5316]},"id":2}]} 2024-05-20T18:28:35.560 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 17 2024-05-20T18:28:35.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.562+0000 7f05d493e640 1 -- 172.21.15.5:0/1721589217 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f05b006be30 msgr2=0x7f05b006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:35.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.562+0000 7f05d493e640 1 --2- 172.21.15.5:0/1721589217 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f05b006be30 0x7f05b006e340 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f05d01e9530 tx=0x7f05c4009040 comp rx=0 tx=0).stop 2024-05-20T18:28:35.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.562+0000 7f05d493e640 1 -- 172.21.15.5:0/1721589217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05b8004d40 msgr2=0x7f05d00b6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:35.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.562+0000 7f05d493e640 1 --2- 172.21.15.5:0/1721589217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05b8004d40 0x7f05d00b6660 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f05c0011910 tx=0x7f05c0011dd0 comp rx=0 tx=0).stop 2024-05-20T18:28:35.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.562+0000 7f05d493e640 1 -- 172.21.15.5:0/1721589217 shutdown_connections 2024-05-20T18:28:35.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.562+0000 7f05d493e640 1 --2- 172.21.15.5:0/1721589217 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f05b006be30 0x7f05b006e340 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:35.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.562+0000 7f05d493e640 1 --2- 172.21.15.5:0/1721589217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05d00c87c0 0x7f05d00c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:35.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.562+0000 7f05d493e640 1 --2- 172.21.15.5:0/1721589217 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05d00c6f20 0x7f05d00cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:35.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.562+0000 7f05d493e640 1 --2- 172.21.15.5:0/1721589217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05b8004d40 0x7f05d00b6660 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:35.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.562+0000 7f05d493e640 1 -- 172.21.15.5:0/1721589217 >> 172.21.15.5:0/1721589217 conn(0x7f05d00d29d0 msgr2=0x7f05d00d06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:35.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.562+0000 7f05d493e640 1 -- 172.21.15.5:0/1721589217 shutdown_connections 2024-05-20T18:28:35.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.562+0000 7f05d493e640 1 -- 172.21.15.5:0/1721589217 wait complete. 2024-05-20T18:28:35.573 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-05-20T18:28:35.717 INFO:teuthology.orchestra.run.smithi005.stdout:{"message":"","return_code":0} 2024-05-20T18:28:35.717 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-20T18:28:35.718 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:28:35.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.833+0000 7f37b1fce640 1 Processor -- start 2024-05-20T18:28:35.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.833+0000 7f37b1fce640 1 -- start start 2024-05-20T18:28:35.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.834+0000 7f37b1fce640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f37ac15f220 0x7f37ac0cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:35.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.834+0000 7f37b1fce640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37ac0cf910 0x7f37ac0d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:35.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.834+0000 7f37b1fce640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37ac0cfd20 con 0x7f37ac0cd680 2024-05-20T18:28:35.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.834+0000 7f37b1fce640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f37ac0cda90 con 0x7f37ac15f220 2024-05-20T18:28:35.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.834+0000 7f37b1fce640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f37ac06d0f0 con 0x7f37ac0cf910 2024-05-20T18:28:35.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.834+0000 7f37ab7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f37ac0cd680 0x7f37ac06c6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:52324/0 (socket says 172.21.15.5:52324) 2024-05-20T18:28:35.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.834+0000 7f37ab7fe640 1 -- 172.21.15.5:0/1916675683 learned_addr learned my addr 172.21.15.5:0/1916675683 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.835+0000 7f37a3fff640 1 --2- 172.21.15.5:0/1916675683 >> v2:172.21.15.63:3300/0 conn(0x7f37ac15f220 0x7f37ac0cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.835+0000 7f37aaffd640 1 -- 172.21.15.5:0/1916675683 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1648310715 0 0) 0x7f379c001110 con 0x7f37ac0cd680 2024-05-20T18:28:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.835+0000 7f37abfff640 1 --2- 172.21.15.5:0/1916675683 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37ac0cf910 0x7f37ac0d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.835+0000 7f37aaffd640 1 -- 172.21.15.5:0/1916675683 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3798003520 con 0x7f37ac0cd680 2024-05-20T18:28:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.835+0000 7f37aaffd640 1 -- 172.21.15.5:0/1916675683 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4012789635 0 0) 0x7f3798003520 con 0x7f37ac0cd680 2024-05-20T18:28:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.835+0000 7f37aaffd640 1 -- 172.21.15.5:0/1916675683 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37ac0cf910 msgr2=0x7f37ac0d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.835+0000 7f37aaffd640 1 --2- 172.21.15.5:0/1916675683 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37ac0cf910 0x7f37ac0d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.835+0000 7f37aaffd640 1 -- 172.21.15.5:0/1916675683 >> v2:172.21.15.63:3300/0 conn(0x7f37ac15f220 msgr2=0x7f37ac0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.835+0000 7f37aaffd640 1 --2- 172.21.15.5:0/1916675683 >> v2:172.21.15.63:3300/0 conn(0x7f37ac15f220 0x7f37ac0cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.835+0000 7f37aaffd640 1 -- 172.21.15.5:0/1916675683 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ac06d3d0 con 0x7f37ac0cd680 2024-05-20T18:28:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.835+0000 7f37aaffd640 1 -- 172.21.15.5:0/1916675683 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f379c001840 con 0x7f37ac0cd680 2024-05-20T18:28:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.835+0000 7f37aaffd640 1 -- 172.21.15.5:0/1916675683 >> v1:172.21.15.5:6789/0 conn(0x7f37ac0cd680 legacy=0x7f37ac06c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.835+0000 7f37aaffd640 1 --2- 172.21.15.5:0/1916675683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37980039b0 0x7f3798003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.835+0000 7f37aaffd640 1 --2- 172.21.15.5:0/1916675683 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37ac0cf910 0x7f3798004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.835+0000 7f37aaffd640 1 --2- 172.21.15.5:0/1916675683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3798004d40 0x7f3798005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.835+0000 7f37aaffd640 1 -- 172.21.15.5:0/1916675683 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac06d0f0 con 0x7f3798004d40 2024-05-20T18:28:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.835+0000 7f37aaffd640 1 -- 172.21.15.5:0/1916675683 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f379800b9c0 con 0x7f37980039b0 2024-05-20T18:28:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.835+0000 7f37aaffd640 1 -- 172.21.15.5:0/1916675683 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f379800bbc0 con 0x7f37ac0cf910 2024-05-20T18:28:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.835+0000 7f37abfff640 1 --2- 172.21.15.5:0/1916675683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3798004d40 0x7f3798005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.835+0000 7f37ab7fe640 1 --2- 172.21.15.5:0/1916675683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37980039b0 0x7f3798003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.835+0000 7f37a3fff640 1 --2- 172.21.15.5:0/1916675683 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37ac0cf910 0x7f3798004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.836+0000 7f37abfff640 1 -- 172.21.15.5:0/1916675683 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37ac0cf910 msgr2=0x7f3798004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.836+0000 7f37abfff640 1 --2- 172.21.15.5:0/1916675683 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37ac0cf910 0x7f3798004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.836+0000 7f37abfff640 1 -- 172.21.15.5:0/1916675683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37980039b0 msgr2=0x7f3798003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.836+0000 7f37abfff640 1 --2- 172.21.15.5:0/1916675683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37980039b0 0x7f3798003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.836+0000 7f37abfff640 1 -- 172.21.15.5:0/1916675683 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3798003520 con 0x7f3798004d40 2024-05-20T18:28:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.836+0000 7f37abfff640 1 --2- 172.21.15.5:0/1916675683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3798004d40 0x7f3798005150 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f37a4009fc0 tx=0x7f3798005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256140911227868e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:35.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.836+0000 7f37aaffd640 1 -- 172.21.15.5:0/1916675683 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a4034020 con 0x7f3798004d40 2024-05-20T18:28:35.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.837+0000 7f37aaffd640 1 -- 172.21.15.5:0/1916675683 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37a40305f0 con 0x7f3798004d40 2024-05-20T18:28:35.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.837+0000 7f37aaffd640 1 -- 172.21.15.5:0/1916675683 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a4009c10 con 0x7f3798004d40 2024-05-20T18:28:35.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.837+0000 7f37b1fce640 1 -- 172.21.15.5:0/1916675683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3798004d40 msgr2=0x7f3798005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:35.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.837+0000 7f37b1fce640 1 --2- 172.21.15.5:0/1916675683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3798004d40 0x7f3798005150 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f37a4009fc0 tx=0x7f3798005830 comp rx=0 tx=0).stop 2024-05-20T18:28:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.837+0000 7f37abfff640 1 -- 172.21.15.5:0/1916675683 reap_dead start 2024-05-20T18:28:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.837+0000 7f37b1fce640 1 -- 172.21.15.5:0/1916675683 shutdown_connections 2024-05-20T18:28:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.837+0000 7f37b1fce640 1 -- 172.21.15.5:0/1916675683 >> 172.21.15.5:0/1916675683 conn(0x7f37ac0b44d0 msgr2=0x7f37ac0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.837+0000 7f37b1fce640 1 -- 172.21.15.5:0/1916675683 shutdown_connections 2024-05-20T18:28:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.837+0000 7f37b1fce640 1 -- 172.21.15.5:0/1916675683 wait complete. 2024-05-20T18:28:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.839+0000 7f37b1fce640 1 Processor -- start 2024-05-20T18:28:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.839+0000 7f37b1fce640 1 -- start start 2024-05-20T18:28:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.840+0000 7f37b1fce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3798004d40 0x7f37ac0c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.840+0000 7f37b1fce640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37ac161810 0x7f37ac161c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.840+0000 7f37b1fce640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37ac0ca5a0 0x7f37ac0ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.841+0000 7f37b1fce640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac06d990 con 0x7f3798004d40 2024-05-20T18:28:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.841+0000 7f37b1fce640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac0d0b40 con 0x7f37ac0ca5a0 2024-05-20T18:28:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.841+0000 7f37b1fce640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f37ac06cdd0 con 0x7f37ac161810 2024-05-20T18:28:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.841+0000 7f37ab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3798004d40 0x7f37ac0c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.841+0000 7f37ab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3798004d40 0x7f37ac0c8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48588/0 (socket says 172.21.15.5:48588) 2024-05-20T18:28:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.841+0000 7f37ab7fe640 1 -- 172.21.15.5:0/3189850149 learned_addr learned my addr 172.21.15.5:0/3189850149 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.841+0000 7f37a3fff640 1 --2- 172.21.15.5:0/3189850149 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37ac161810 0x7f37ac161c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.841+0000 7f37abfff640 1 --2- 172.21.15.5:0/3189850149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37ac0ca5a0 0x7f37ac0ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.841+0000 7f37ab7fe640 1 -- 172.21.15.5:0/3189850149 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37ac161810 msgr2=0x7f37ac161c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.841+0000 7f37ab7fe640 1 --2- 172.21.15.5:0/3189850149 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37ac161810 0x7f37ac161c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.841+0000 7f37ab7fe640 1 -- 172.21.15.5:0/3189850149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37ac0ca5a0 msgr2=0x7f37ac0ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.841+0000 7f37ab7fe640 1 --2- 172.21.15.5:0/3189850149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37ac0ca5a0 0x7f37ac0ca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.841+0000 7f37ab7fe640 1 -- 172.21.15.5:0/3189850149 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ac1625d0 con 0x7f3798004d40 2024-05-20T18:28:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.841+0000 7f37abfff640 1 --2- 172.21.15.5:0/3189850149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37ac0ca5a0 0x7f37ac0ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.841+0000 7f37ab7fe640 1 --2- 172.21.15.5:0/3189850149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3798004d40 0x7f37ac0c8c60 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f379c007310 tx=0x7f379c0077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.841+0000 7f37a8ff9640 1 -- 172.21.15.5:0/3189850149 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379c00f990 con 0x7f3798004d40 2024-05-20T18:28:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.841+0000 7f37a8ff9640 1 -- 172.21.15.5:0/3189850149 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f379c018070 con 0x7f3798004d40 2024-05-20T18:28:35.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.841+0000 7f37b1fce640 1 -- 172.21.15.5:0/3189850149 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37ac0c9ac0 con 0x7f3798004d40 2024-05-20T18:28:35.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.841+0000 7f37a8ff9640 1 -- 172.21.15.5:0/3189850149 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379c00fdb0 con 0x7f3798004d40 2024-05-20T18:28:35.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.841+0000 7f37b1fce640 1 -- 172.21.15.5:0/3189850149 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37ac0ca080 con 0x7f3798004d40 2024-05-20T18:28:35.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.842+0000 7f37b1fce640 1 -- 172.21.15.5:0/3189850149 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37ac075b00 con 0x7f3798004d40 2024-05-20T18:28:35.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.843+0000 7f37a8ff9640 1 -- 172.21.15.5:0/3189850149 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f379c01d460 con 0x7f3798004d40 2024-05-20T18:28:35.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.843+0000 7f37a8ff9640 1 --2- 172.21.15.5:0/3189850149 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f378c06be30 0x7f378c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:35.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.843+0000 7f37a8ff9640 1 -- 172.21.15.5:0/3189850149 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f379c08b2c0 con 0x7f3798004d40 2024-05-20T18:28:35.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.843+0000 7f37a3fff640 1 --2- 172.21.15.5:0/3189850149 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f378c06be30 0x7f378c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:35.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.844+0000 7f37a3fff640 1 --2- 172.21.15.5:0/3189850149 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f378c06be30 0x7f378c06e340 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f37ac0d3560 tx=0x7f3794008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:35.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:35.848+0000 7f37a8ff9640 1 -- 172.21.15.5:0/3189850149 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f37ac0cfd20 con 0x7f3798004d40 2024-05-20T18:28:36.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:36.073+0000 7f37b1fce640 1 -- 172.21.15.5:0/3189850149 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37ac0b8940 con 0x7f3798004d40 2024-05-20T18:28:36.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:36.074+0000 7f37a8ff9640 1 -- 172.21.15.5:0/3189850149 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5175 (secure 0 0 0) 0x7f379c059a50 con 0x7f3798004d40 2024-05-20T18:28:36.073 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:36.074 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1458170078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1458170078},{"type":"v1","addr":"172.21.15.63:6834","nonce":1458170078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4916,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/2601750663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":2601750663},{"type":"v1","addr":"172.21.15.63:6835","nonce":2601750663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5313,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/1913030219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1913030219},{"type":"v1","addr":"172.21.15.5:6836","nonce":1913030219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:28:24.270282+0000","modified":"2024-05-20T18:28:27.296534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5316},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5316":{"gid":5316,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6837/2355805105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":2355805105},{"type":"v1","addr":"172.21.15.5:6837","nonce":2355805105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5316,"qdb_cluster":[5316]},"id":2}]} 2024-05-20T18:28:36.074 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 17 2024-05-20T18:28:36.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:36.076+0000 7f37b1fce640 1 -- 172.21.15.5:0/3189850149 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f378c06be30 msgr2=0x7f378c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:36.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:36.076+0000 7f37b1fce640 1 --2- 172.21.15.5:0/3189850149 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f378c06be30 0x7f378c06e340 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f37ac0d3560 tx=0x7f3794008040 comp rx=0 tx=0).stop 2024-05-20T18:28:36.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:36.076+0000 7f37b1fce640 1 -- 172.21.15.5:0/3189850149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3798004d40 msgr2=0x7f37ac0c8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:36.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:36.076+0000 7f37b1fce640 1 --2- 172.21.15.5:0/3189850149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3798004d40 0x7f37ac0c8c60 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f379c007310 tx=0x7f379c0077d0 comp rx=0 tx=0).stop 2024-05-20T18:28:36.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:36.076+0000 7f37b1fce640 1 -- 172.21.15.5:0/3189850149 shutdown_connections 2024-05-20T18:28:36.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:36.076+0000 7f37b1fce640 1 --2- 172.21.15.5:0/3189850149 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f378c06be30 0x7f378c06e340 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:36.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:36.076+0000 7f37b1fce640 1 --2- 172.21.15.5:0/3189850149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37ac0ca5a0 0x7f37ac0ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:36.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:36.076+0000 7f37b1fce640 1 --2- 172.21.15.5:0/3189850149 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37ac161810 0x7f37ac161c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:36.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:36.076+0000 7f37b1fce640 1 --2- 172.21.15.5:0/3189850149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3798004d40 0x7f37ac0c8c60 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:36.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:36.076+0000 7f37b1fce640 1 -- 172.21.15.5:0/3189850149 >> 172.21.15.5:0/3189850149 conn(0x7f37ac0b44d0 msgr2=0x7f37ac15c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:36.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:36.076+0000 7f37b1fce640 1 -- 172.21.15.5:0/3189850149 shutdown_connections 2024-05-20T18:28:36.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:36.076+0000 7f37b1fce640 1 -- 172.21.15.5:0/3189850149 wait complete. 2024-05-20T18:28:36.087 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parentdir mytag 2024-05-20T18:28:36.220 INFO:teuthology.orchestra.run.smithi005.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2024-05-20T18:28:36.221 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2024-05-20T18:28:40.852 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:28:40.857 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:28:46.222 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2024-05-20T18:28:46.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.291+0000 7f42fd05cf40 1 Processor -- start 2024-05-20T18:28:46.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.291+0000 7f42fd05cf40 1 -- start start 2024-05-20T18:28:46.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.292+0000 7f42fd05cf40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f8794e3a0 0x564f879ecd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.292+0000 7f42fd05cf40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8794e7b0 0x564f879fad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.292+0000 7f42fd05cf40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f879fcfe0 0x564f879fb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.292+0000 7f42fd05cf40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564f879f9880 con 0x564f8794e7b0 2024-05-20T18:28:46.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.292+0000 7f42fd05cf40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564f878cbc80 con 0x564f879fcfe0 2024-05-20T18:28:46.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.292+0000 7f42fd05cf40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564f87922510 con 0x564f8794e3a0 2024-05-20T18:28:46.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.293+0000 7f42f632a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8794e7b0 0x564f879fad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.293+0000 7f42f632a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8794e7b0 0x564f879fad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33912/0 (socket says 172.21.15.5:33912) 2024-05-20T18:28:46.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.293+0000 7f42f632a640 1 -- 172.21.15.5:0/3301221731 learned_addr learned my addr 172.21.15.5:0/3301221731 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:46.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.293+0000 7f42f732c640 1 --2- 172.21.15.5:0/3301221731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f879fcfe0 0x564f879fb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.293+0000 7f42f6b2b640 1 --2- 172.21.15.5:0/3301221731 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f8794e3a0 0x564f879ecd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.293+0000 7f42f632a640 1 -- 172.21.15.5:0/3301221731 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f8794e3a0 msgr2=0x564f879ecd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:46.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.293+0000 7f42f632a640 1 --2- 172.21.15.5:0/3301221731 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f8794e3a0 0x564f879ecd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.293+0000 7f42f632a640 1 -- 172.21.15.5:0/3301221731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f879fcfe0 msgr2=0x564f879fb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:46.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.293+0000 7f42f632a640 1 --2- 172.21.15.5:0/3301221731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f879fcfe0 0x564f879fb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.293+0000 7f42f632a640 1 -- 172.21.15.5:0/3301221731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f87850cb0 con 0x564f8794e7b0 2024-05-20T18:28:46.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.293+0000 7f42f732c640 1 --2- 172.21.15.5:0/3301221731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f879fcfe0 0x564f879fb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:46.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.293+0000 7f42f632a640 1 --2- 172.21.15.5:0/3301221731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8794e7b0 0x564f879fad40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f42ec00a7a0 tx=0x7f42ec00a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8841f58323ec7aa server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:46.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.293+0000 7f42f5328640 1 -- 172.21.15.5:0/3301221731 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec019070 con 0x564f8794e7b0 2024-05-20T18:28:46.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.293+0000 7f42f5328640 1 -- 172.21.15.5:0/3301221731 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42ec007600 con 0x564f8794e7b0 2024-05-20T18:28:46.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.293+0000 7f42f5328640 1 -- 172.21.15.5:0/3301221731 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec0079d0 con 0x564f8794e7b0 2024-05-20T18:28:46.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.294+0000 7f42fd05cf40 1 -- 172.21.15.5:0/3301221731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8794e7b0 msgr2=0x564f879fad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.294+0000 7f42fd05cf40 1 --2- 172.21.15.5:0/3301221731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8794e7b0 0x564f879fad40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f42ec00a7a0 tx=0x7f42ec00a890 comp rx=0 tx=0).stop 2024-05-20T18:28:46.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.294+0000 7f42fd05cf40 1 -- 172.21.15.5:0/3301221731 shutdown_connections 2024-05-20T18:28:46.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.294+0000 7f42fd05cf40 1 --2- 172.21.15.5:0/3301221731 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f879fcfe0 0x564f879fb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.294+0000 7f42fd05cf40 1 --2- 172.21.15.5:0/3301221731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8794e7b0 0x564f879fad40 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.294+0000 7f42fd05cf40 1 --2- 172.21.15.5:0/3301221731 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f8794e3a0 0x564f879ecd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.294+0000 7f42fd05cf40 1 -- 172.21.15.5:0/3301221731 >> 172.21.15.5:0/3301221731 conn(0x564f87849ac0 msgr2=0x564f8796f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:46.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.294+0000 7f42fd05cf40 1 -- 172.21.15.5:0/3301221731 shutdown_connections 2024-05-20T18:28:46.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.294+0000 7f42fd05cf40 1 -- 172.21.15.5:0/3301221731 wait complete. 2024-05-20T18:28:46.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.295+0000 7f42fd05cf40 1 Processor -- start 2024-05-20T18:28:46.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.295+0000 7f42fd05cf40 1 -- start start 2024-05-20T18:28:46.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.296+0000 7f42fd05cf40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f8794e3a0 0x564f879ea440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.296+0000 7f42fd05cf40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f8794e7b0 0x564f879e9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.296+0000 7f42fd05cf40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f879fcfe0 0x564f879e9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.296+0000 7f42fd05cf40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564f87950380 con 0x564f879fcfe0 2024-05-20T18:28:46.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.296+0000 7f42fd05cf40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564f8796e8e0 con 0x564f8794e7b0 2024-05-20T18:28:46.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.296+0000 7f42fd05cf40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564f87950660 con 0x564f8794e3a0 2024-05-20T18:28:46.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.296+0000 7f42f732c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f879fcfe0 0x564f879e9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.297+0000 7f42f732c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f879fcfe0 0x564f879e9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33922/0 (socket says 172.21.15.5:33922) 2024-05-20T18:28:46.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.297+0000 7f42f732c640 1 -- 172.21.15.5:0/1432541296 learned_addr learned my addr 172.21.15.5:0/1432541296 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:46.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.297+0000 7f42f632a640 1 --2- 172.21.15.5:0/1432541296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f8794e7b0 0x564f879e9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.297+0000 7f42f6b2b640 1 --2- 172.21.15.5:0/1432541296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f8794e3a0 0x564f879ea440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.297+0000 7f42f732c640 1 -- 172.21.15.5:0/1432541296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f8794e3a0 msgr2=0x564f879ea440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.297+0000 7f42f732c640 1 --2- 172.21.15.5:0/1432541296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f8794e3a0 0x564f879ea440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.297+0000 7f42f732c640 1 -- 172.21.15.5:0/1432541296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f8794e7b0 msgr2=0x564f879e9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.297+0000 7f42f732c640 1 --2- 172.21.15.5:0/1432541296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f8794e7b0 0x564f879e9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.297+0000 7f42f732c640 1 -- 172.21.15.5:0/1432541296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f879e5240 con 0x564f879fcfe0 2024-05-20T18:28:46.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.297+0000 7f42f632a640 1 --2- 172.21.15.5:0/1432541296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f8794e7b0 0x564f879e9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:46.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.297+0000 7f42f732c640 1 --2- 172.21.15.5:0/1432541296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f879fcfe0 0x564f879e9950 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f42f000cd60 tx=0x7f42f000a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:46.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.297+0000 7f42dffff640 1 -- 172.21.15.5:0/1432541296 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f0008800 con 0x564f879fcfe0 2024-05-20T18:28:46.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.297+0000 7f42dffff640 1 -- 172.21.15.5:0/1432541296 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42f000ce50 con 0x564f879fcfe0 2024-05-20T18:28:46.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.297+0000 7f42fd05cf40 1 -- 172.21.15.5:0/1432541296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564f879ebdd0 con 0x564f879fcfe0 2024-05-20T18:28:46.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.297+0000 7f42dffff640 1 -- 172.21.15.5:0/1432541296 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f0008bc0 con 0x564f879fcfe0 2024-05-20T18:28:46.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.297+0000 7f42fd05cf40 1 -- 172.21.15.5:0/1432541296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f8796c840 con 0x564f879fcfe0 2024-05-20T18:28:46.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.299+0000 7f42dffff640 1 -- 172.21.15.5:0/1432541296 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f42f001e3f0 con 0x564f879fcfe0 2024-05-20T18:28:46.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.299+0000 7f42dffff640 1 --2- 172.21.15.5:0/1432541296 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f42e406a810 0x7f42e406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.300+0000 7f42f6b2b640 1 --2- 172.21.15.5:0/1432541296 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f42e406a810 0x7f42e406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.300+0000 7f42dffff640 1 -- 172.21.15.5:0/1432541296 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f42f0089d30 con 0x564f879fcfe0 2024-05-20T18:28:46.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.300+0000 7f42fd05cf40 1 --2- 172.21.15.5:0/1432541296 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x564f87a96690 0x564f87a9efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.300+0000 7f42fd05cf40 1 -- 172.21.15.5:0/1432541296 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) -- 0x564f879e5bb0 con 0x564f87a96690 2024-05-20T18:28:46.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.300+0000 7f42f6b2b640 1 --2- 172.21.15.5:0/1432541296 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f42e406a810 0x7f42e406cd20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f42e8056000 tx=0x7f42e805e040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:46.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.300+0000 7f42f632a640 1 --2- 172.21.15.5:0/1432541296 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x564f87a96690 0x564f87a9efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.301+0000 7f42f632a640 1 --2- 172.21.15.5:0/1432541296 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x564f87a96690 0x564f87a9efa0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:46.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.301+0000 7f42f632a640 1 -- 172.21.15.5:0/1432541296 <== osd.7 v2:172.21.15.63:6824/4136124451 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f42ec01c040 con 0x564f87a96690 2024-05-20T18:28:46.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.301+0000 7f42fd05cf40 1 -- 172.21.15.5:0/1432541296 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x564f87a96690 msgr2=0x564f87a9efa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.301+0000 7f42fd05cf40 1 --2- 172.21.15.5:0/1432541296 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x564f87a96690 0x564f87a9efa0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.301+0000 7f42fd05cf40 1 -- 172.21.15.5:0/1432541296 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f42e406a810 msgr2=0x7f42e406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.301+0000 7f42fd05cf40 1 --2- 172.21.15.5:0/1432541296 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f42e406a810 0x7f42e406cd20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f42e8056000 tx=0x7f42e805e040 comp rx=0 tx=0).stop 2024-05-20T18:28:46.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.301+0000 7f42fd05cf40 1 -- 172.21.15.5:0/1432541296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f879fcfe0 msgr2=0x564f879e9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.301+0000 7f42fd05cf40 1 --2- 172.21.15.5:0/1432541296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f879fcfe0 0x564f879e9950 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f42f000cd60 tx=0x7f42f000a750 comp rx=0 tx=0).stop 2024-05-20T18:28:46.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.301+0000 7f42f732c640 1 -- 172.21.15.5:0/1432541296 reap_dead start 2024-05-20T18:28:46.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.301+0000 7f42fd05cf40 1 -- 172.21.15.5:0/1432541296 shutdown_connections 2024-05-20T18:28:46.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.301+0000 7f42fd05cf40 1 -- 172.21.15.5:0/1432541296 >> 172.21.15.5:0/1432541296 conn(0x564f87849ac0 msgr2=0x564f879fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:46.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.301+0000 7f42fd05cf40 1 -- 172.21.15.5:0/1432541296 shutdown_connections 2024-05-20T18:28:46.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.302+0000 7f42fd05cf40 1 -- 172.21.15.5:0/1432541296 wait complete. 2024-05-20T18:28:46.304 INFO:teuthology.orchestra.run.smithi005.stdout:mytag 2024-05-20T18:28:46.305 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2024-05-20T18:28:46.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.365+0000 7fae3151cf40 1 Processor -- start 2024-05-20T18:28:46.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.365+0000 7fae3151cf40 1 -- start start 2024-05-20T18:28:46.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.366+0000 7fae3151cf40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5625aa7bf970 0x5625aa84baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.366+0000 7fae3151cf40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625aa7c15f0 0x5625aa84f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.366+0000 7fae3151cf40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625aa8e1660 0x5625aa7762d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.366+0000 7fae3151cf40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5625aa71fc80 con 0x5625aa8e1660 2024-05-20T18:28:46.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.366+0000 7fae3151cf40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5625aa723cc0 con 0x5625aa7c15f0 2024-05-20T18:28:46.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.366+0000 7fae3151cf40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5625aa7a7550 con 0x5625aa7bf970 2024-05-20T18:28:46.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.366+0000 7fae2ad20640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625aa8e1660 0x5625aa7762d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.366+0000 7fae2ad20640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625aa8e1660 0x5625aa7762d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33928/0 (socket says 172.21.15.5:33928) 2024-05-20T18:28:46.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.366+0000 7fae2ad20640 1 -- 172.21.15.5:0/1311034180 learned_addr learned my addr 172.21.15.5:0/1311034180 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:46.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.366+0000 7fae2a51f640 1 --2- 172.21.15.5:0/1311034180 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5625aa7bf970 0x5625aa84baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.366+0000 7fae2ad20640 1 -- 172.21.15.5:0/1311034180 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5625aa7bf970 msgr2=0x5625aa84baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:46.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.366+0000 7fae23fff640 1 --2- 172.21.15.5:0/1311034180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625aa7c15f0 0x5625aa84f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.366+0000 7fae2ad20640 1 --2- 172.21.15.5:0/1311034180 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5625aa7bf970 0x5625aa84baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.366+0000 7fae2ad20640 1 -- 172.21.15.5:0/1311034180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625aa7c15f0 msgr2=0x5625aa84f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:46.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.366+0000 7fae2ad20640 1 --2- 172.21.15.5:0/1311034180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625aa7c15f0 0x5625aa84f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.366+0000 7fae2ad20640 1 -- 172.21.15.5:0/1311034180 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625aa84bfe0 con 0x5625aa8e1660 2024-05-20T18:28:46.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.366+0000 7fae23fff640 1 --2- 172.21.15.5:0/1311034180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625aa7c15f0 0x5625aa84f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:46.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.367+0000 7fae2ad20640 1 --2- 172.21.15.5:0/1311034180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625aa8e1660 0x5625aa7762d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fae2400a4e0 tx=0x7fae2400a9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15cc54f2dc893f41 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:46.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.367+0000 7fae2951d640 1 -- 172.21.15.5:0/1311034180 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae24015070 con 0x5625aa8e1660 2024-05-20T18:28:46.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.367+0000 7fae2951d640 1 -- 172.21.15.5:0/1311034180 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae2400e730 con 0x5625aa8e1660 2024-05-20T18:28:46.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.367+0000 7fae2951d640 1 -- 172.21.15.5:0/1311034180 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2400eb20 con 0x5625aa8e1660 2024-05-20T18:28:46.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.367+0000 7fae3151cf40 1 -- 172.21.15.5:0/1311034180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625aa8e1660 msgr2=0x5625aa7762d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.367+0000 7fae3151cf40 1 --2- 172.21.15.5:0/1311034180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625aa8e1660 0x5625aa7762d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fae2400a4e0 tx=0x7fae2400a9a0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.367+0000 7fae3151cf40 1 -- 172.21.15.5:0/1311034180 shutdown_connections 2024-05-20T18:28:46.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.367+0000 7fae3151cf40 1 --2- 172.21.15.5:0/1311034180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625aa8e1660 0x5625aa7762d0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.367+0000 7fae3151cf40 1 --2- 172.21.15.5:0/1311034180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625aa7c15f0 0x5625aa84f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.367+0000 7fae3151cf40 1 --2- 172.21.15.5:0/1311034180 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5625aa7bf970 0x5625aa84baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.367+0000 7fae3151cf40 1 -- 172.21.15.5:0/1311034180 >> 172.21.15.5:0/1311034180 conn(0x5625aa69dac0 msgr2=0x5625aa84c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:46.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.367+0000 7fae3151cf40 1 -- 172.21.15.5:0/1311034180 shutdown_connections 2024-05-20T18:28:46.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.367+0000 7fae3151cf40 1 -- 172.21.15.5:0/1311034180 wait complete. 2024-05-20T18:28:46.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.368+0000 7fae3151cf40 1 Processor -- start 2024-05-20T18:28:46.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.368+0000 7fae3151cf40 1 -- start start 2024-05-20T18:28:46.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.370+0000 7fae3151cf40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5625aa7bf970 0x5625aa8e0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.370+0000 7fae3151cf40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625aa7c15f0 0x5625aa846c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.370+0000 7fae3151cf40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625aa8de0f0 0x5625aa8de500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.370+0000 7fae3151cf40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5625aa850b20 con 0x5625aa7c15f0 2024-05-20T18:28:46.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.370+0000 7fae3151cf40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5625aa849430 con 0x5625aa8de0f0 2024-05-20T18:28:46.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.370+0000 7fae3151cf40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5625aa7a80c0 con 0x5625aa7bf970 2024-05-20T18:28:46.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.370+0000 7fae23fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625aa7c15f0 0x5625aa846c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.370+0000 7fae23fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625aa7c15f0 0x5625aa846c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33938/0 (socket says 172.21.15.5:33938) 2024-05-20T18:28:46.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.370+0000 7fae23fff640 1 -- 172.21.15.5:0/878592853 learned_addr learned my addr 172.21.15.5:0/878592853 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:46.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.370+0000 7fae2ad20640 1 --2- 172.21.15.5:0/878592853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625aa8de0f0 0x5625aa8de500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.370+0000 7fae2a51f640 1 --2- 172.21.15.5:0/878592853 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5625aa7bf970 0x5625aa8e0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.370+0000 7fae23fff640 1 -- 172.21.15.5:0/878592853 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5625aa7bf970 msgr2=0x5625aa8e0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.370+0000 7fae23fff640 1 --2- 172.21.15.5:0/878592853 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5625aa7bf970 0x5625aa8e0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.370+0000 7fae23fff640 1 -- 172.21.15.5:0/878592853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625aa8de0f0 msgr2=0x5625aa8de500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.370+0000 7fae23fff640 1 --2- 172.21.15.5:0/878592853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5625aa8de0f0 0x5625aa8de500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.370+0000 7fae23fff640 1 -- 172.21.15.5:0/878592853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625aa8e1150 con 0x5625aa7c15f0 2024-05-20T18:28:46.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.370+0000 7fae23fff640 1 --2- 172.21.15.5:0/878592853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625aa7c15f0 0x5625aa846c40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fae14045d70 tx=0x7fae14002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:46.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.371+0000 7fae237fe640 1 -- 172.21.15.5:0/878592853 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae14065070 con 0x5625aa7c15f0 2024-05-20T18:28:46.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.371+0000 7fae237fe640 1 -- 172.21.15.5:0/878592853 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae14054ab0 con 0x5625aa7c15f0 2024-05-20T18:28:46.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.371+0000 7fae3151cf40 1 -- 172.21.15.5:0/878592853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5625aa7a15d0 con 0x5625aa7c15f0 2024-05-20T18:28:46.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.371+0000 7fae237fe640 1 -- 172.21.15.5:0/878592853 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae14059040 con 0x5625aa7c15f0 2024-05-20T18:28:46.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.371+0000 7fae3151cf40 1 -- 172.21.15.5:0/878592853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625aa8474e0 con 0x5625aa7c15f0 2024-05-20T18:28:46.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.372+0000 7fae237fe640 1 -- 172.21.15.5:0/878592853 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fae14068070 con 0x5625aa7c15f0 2024-05-20T18:28:46.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.373+0000 7fae237fe640 1 --2- 172.21.15.5:0/878592853 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae0c06a840 0x7fae0c06cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.373+0000 7fae2a51f640 1 --2- 172.21.15.5:0/878592853 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae0c06a840 0x7fae0c06cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.373+0000 7fae237fe640 1 -- 172.21.15.5:0/878592853 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fae140cd950 con 0x5625aa7c15f0 2024-05-20T18:28:46.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.373+0000 7fae3151cf40 1 --2- 172.21.15.5:0/878592853 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x5625aa848660 0x5625aa8f2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.373+0000 7fae3151cf40 1 -- 172.21.15.5:0/878592853 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:1 5.7 5:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) -- 0x5625aa84f4c0 con 0x5625aa848660 2024-05-20T18:28:46.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.373+0000 7fae2a51f640 1 --2- 172.21.15.5:0/878592853 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae0c06a840 0x7fae0c06cd50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fae1c000f60 tx=0x7fae1c001670 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:46.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.374+0000 7fae2ad20640 1 --2- 172.21.15.5:0/878592853 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x5625aa848660 0x5625aa8f2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.374+0000 7fae2ad20640 1 --2- 172.21.15.5:0/878592853 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x5625aa848660 0x5625aa8f2940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:46.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.374+0000 7fae2ad20640 1 -- 172.21.15.5:0/878592853 <== osd.5 v2:172.21.15.63:6808/1616066286 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7fae2400e070 con 0x5625aa848660 2024-05-20T18:28:46.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.375+0000 7fae3151cf40 1 -- 172.21.15.5:0/878592853 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x5625aa848660 msgr2=0x5625aa8f2940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.375+0000 7fae3151cf40 1 --2- 172.21.15.5:0/878592853 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x5625aa848660 0x5625aa8f2940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.375+0000 7fae3151cf40 1 -- 172.21.15.5:0/878592853 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae0c06a840 msgr2=0x7fae0c06cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.375+0000 7fae3151cf40 1 --2- 172.21.15.5:0/878592853 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae0c06a840 0x7fae0c06cd50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fae1c000f60 tx=0x7fae1c001670 comp rx=0 tx=0).stop 2024-05-20T18:28:46.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.375+0000 7fae3151cf40 1 -- 172.21.15.5:0/878592853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625aa7c15f0 msgr2=0x5625aa846c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.375+0000 7fae3151cf40 1 --2- 172.21.15.5:0/878592853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625aa7c15f0 0x5625aa846c40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fae14045d70 tx=0x7fae14002fb0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.375+0000 7fae2ad20640 1 -- 172.21.15.5:0/878592853 reap_dead start 2024-05-20T18:28:46.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.375+0000 7fae3151cf40 1 -- 172.21.15.5:0/878592853 shutdown_connections 2024-05-20T18:28:46.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.375+0000 7fae3151cf40 1 -- 172.21.15.5:0/878592853 >> 172.21.15.5:0/878592853 conn(0x5625aa69dac0 msgr2=0x5625aa8e1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:46.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.375+0000 7fae3151cf40 1 -- 172.21.15.5:0/878592853 shutdown_connections 2024-05-20T18:28:46.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.375+0000 7fae3151cf40 1 -- 172.21.15.5:0/878592853 wait complete. 2024-05-20T18:28:46.377 INFO:teuthology.orchestra.run.smithi005.stdout:mytag 2024-05-20T18:28:46.378 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-05-20T18:28:46.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.434+0000 7ff42571ff40 1 Processor -- start 2024-05-20T18:28:46.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.434+0000 7ff42571ff40 1 -- start start 2024-05-20T18:28:46.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.435+0000 7ff42571ff40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559247fa5970 0x55924802fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.435+0000 7ff42571ff40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559248032a30 0x559248034860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.435+0000 7ff42571ff40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559248036010 0x559248034da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.435+0000 7ff42571ff40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559247f8d550 con 0x559248036010 2024-05-20T18:28:46.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.435+0000 7ff42571ff40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559247f05c80 con 0x559248032a30 2024-05-20T18:28:46.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.435+0000 7ff42571ff40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x559248036960 con 0x559247fa5970 2024-05-20T18:28:46.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.436+0000 7ff41ef69640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559248036010 0x559248034da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.436+0000 7ff41ef69640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559248036010 0x559248034da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33968/0 (socket says 172.21.15.5:33968) 2024-05-20T18:28:46.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.436+0000 7ff41ef69640 1 -- 172.21.15.5:0/2322948122 learned_addr learned my addr 172.21.15.5:0/2322948122 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:46.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.436+0000 7ff41e768640 1 --2- 172.21.15.5:0/2322948122 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559247fa5970 0x55924802fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.436+0000 7ff41df67640 1 --2- 172.21.15.5:0/2322948122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559248032a30 0x559248034860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.436+0000 7ff41ef69640 1 -- 172.21.15.5:0/2322948122 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559247fa5970 msgr2=0x55924802fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:46.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.436+0000 7ff41ef69640 1 --2- 172.21.15.5:0/2322948122 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559247fa5970 0x55924802fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.436+0000 7ff41ef69640 1 -- 172.21.15.5:0/2322948122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559248032a30 msgr2=0x559248034860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:46.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.436+0000 7ff41ef69640 1 --2- 172.21.15.5:0/2322948122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559248032a30 0x559248034860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.436+0000 7ff41ef69640 1 -- 172.21.15.5:0/2322948122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559247e8acb0 con 0x559248036010 2024-05-20T18:28:46.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.436+0000 7ff41ef69640 1 --2- 172.21.15.5:0/2322948122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559248036010 0x559248034da0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff4100015f0 tx=0x7ff410001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc91f515d2a734c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:46.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.436+0000 7ff41cf65640 1 -- 172.21.15.5:0/2322948122 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff410004520 con 0x559248036010 2024-05-20T18:28:46.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.436+0000 7ff41cf65640 1 -- 172.21.15.5:0/2322948122 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4100046c0 con 0x559248036010 2024-05-20T18:28:46.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.436+0000 7ff41cf65640 1 -- 172.21.15.5:0/2322948122 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff410004a90 con 0x559248036010 2024-05-20T18:28:46.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.437+0000 7ff42571ff40 1 -- 172.21.15.5:0/2322948122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559248036010 msgr2=0x559248034da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.437+0000 7ff42571ff40 1 --2- 172.21.15.5:0/2322948122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559248036010 0x559248034da0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff4100015f0 tx=0x7ff410001ab0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.437+0000 7ff42571ff40 1 -- 172.21.15.5:0/2322948122 shutdown_connections 2024-05-20T18:28:46.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.437+0000 7ff42571ff40 1 --2- 172.21.15.5:0/2322948122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559248036010 0x559248034da0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.437+0000 7ff42571ff40 1 --2- 172.21.15.5:0/2322948122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559248032a30 0x559248034860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.437+0000 7ff42571ff40 1 --2- 172.21.15.5:0/2322948122 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559247fa5970 0x55924802fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.437+0000 7ff42571ff40 1 -- 172.21.15.5:0/2322948122 >> 172.21.15.5:0/2322948122 conn(0x559247e83ac0 msgr2=0x559247f88bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:46.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.437+0000 7ff42571ff40 1 -- 172.21.15.5:0/2322948122 shutdown_connections 2024-05-20T18:28:46.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.437+0000 7ff42571ff40 1 -- 172.21.15.5:0/2322948122 wait complete. 2024-05-20T18:28:46.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.438+0000 7ff42571ff40 1 Processor -- start 2024-05-20T18:28:46.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.438+0000 7ff42571ff40 1 -- start start 2024-05-20T18:28:46.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.439+0000 7ff42571ff40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559247fa5970 0x55924802c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.439+0000 7ff42571ff40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559248032a30 0x559247f9f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.439+0000 7ff42571ff40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559248036010 0x559247f9f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.439+0000 7ff42571ff40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559247fa6190 con 0x559248036010 2024-05-20T18:28:46.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.439+0000 7ff42571ff40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559247f899e0 con 0x559247fa5970 2024-05-20T18:28:46.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.439+0000 7ff42571ff40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x559247f801f0 con 0x559248032a30 2024-05-20T18:28:46.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.439+0000 7ff41ef69640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559248036010 0x559247f9f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.439+0000 7ff41ef69640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559248036010 0x559247f9f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33970/0 (socket says 172.21.15.5:33970) 2024-05-20T18:28:46.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.439+0000 7ff41ef69640 1 -- 172.21.15.5:0/1890550057 learned_addr learned my addr 172.21.15.5:0/1890550057 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:46.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.440+0000 7ff41e768640 1 --2- 172.21.15.5:0/1890550057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559247fa5970 0x55924802c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.440+0000 7ff41df67640 1 --2- 172.21.15.5:0/1890550057 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559248032a30 0x559247f9f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.440+0000 7ff41ef69640 1 -- 172.21.15.5:0/1890550057 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559248032a30 msgr2=0x559247f9f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.440+0000 7ff41ef69640 1 --2- 172.21.15.5:0/1890550057 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559248032a30 0x559247f9f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.440+0000 7ff41ef69640 1 -- 172.21.15.5:0/1890550057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559247fa5970 msgr2=0x55924802c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.440+0000 7ff41ef69640 1 --2- 172.21.15.5:0/1890550057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559247fa5970 0x55924802c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.440+0000 7ff41ef69640 1 -- 172.21.15.5:0/1890550057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559247f87b50 con 0x559248036010 2024-05-20T18:28:46.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.440+0000 7ff41ef69640 1 --2- 172.21.15.5:0/1890550057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559248036010 0x559247f9f5a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff4100044f0 tx=0x7ff410001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:46.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.440+0000 7ff4177fe640 1 -- 172.21.15.5:0/1890550057 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff410024070 con 0x559248036010 2024-05-20T18:28:46.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.440+0000 7ff4177fe640 1 -- 172.21.15.5:0/1890550057 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff410019040 con 0x559248036010 2024-05-20T18:28:46.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.440+0000 7ff42571ff40 1 -- 172.21.15.5:0/1890550057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559247fa46a0 con 0x559248036010 2024-05-20T18:28:46.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.440+0000 7ff4177fe640 1 -- 172.21.15.5:0/1890550057 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41001f880 con 0x559248036010 2024-05-20T18:28:46.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.440+0000 7ff42571ff40 1 -- 172.21.15.5:0/1890550057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559247fa4c60 con 0x559248036010 2024-05-20T18:28:46.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.442+0000 7ff4177fe640 1 -- 172.21.15.5:0/1890550057 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff410027070 con 0x559248036010 2024-05-20T18:28:46.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.442+0000 7ff4177fe640 1 --2- 172.21.15.5:0/1890550057 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff40006a840 0x7ff40006cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.442+0000 7ff4177fe640 1 -- 172.21.15.5:0/1890550057 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7ff41008b1c0 con 0x559248036010 2024-05-20T18:28:46.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.442+0000 7ff41e768640 1 --2- 172.21.15.5:0/1890550057 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff40006a840 0x7ff40006cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.442+0000 7ff42571ff40 1 --2- 172.21.15.5:0/1890550057 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x55924802b660 0x5592480cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.442+0000 7ff42571ff40 1 -- 172.21.15.5:0/1890550057 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:1 6.36 6:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) -- 0x55924802cb10 con 0x55924802b660 2024-05-20T18:28:46.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.443+0000 7ff41df67640 1 --2- 172.21.15.5:0/1890550057 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x55924802b660 0x5592480cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.443+0000 7ff41e768640 1 --2- 172.21.15.5:0/1890550057 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff40006a840 0x7ff40006cd50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x559247fa8ed0 tx=0x7ff418050000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:46.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.443+0000 7ff41df67640 1 --2- 172.21.15.5:0/1890550057 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x55924802b660 0x5592480cfe60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:46.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.443+0000 7ff41df67640 1 -- 172.21.15.5:0/1890550057 <== osd.5 v2:172.21.15.63:6808/1616066286 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7ff408007870 con 0x55924802b660 2024-05-20T18:28:46.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.444+0000 7ff42571ff40 1 -- 172.21.15.5:0/1890550057 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x55924802b660 msgr2=0x5592480cfe60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.444+0000 7ff42571ff40 1 --2- 172.21.15.5:0/1890550057 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x55924802b660 0x5592480cfe60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.444+0000 7ff42571ff40 1 -- 172.21.15.5:0/1890550057 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff40006a840 msgr2=0x7ff40006cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.444+0000 7ff42571ff40 1 --2- 172.21.15.5:0/1890550057 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff40006a840 0x7ff40006cd50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x559247fa8ed0 tx=0x7ff418050000 comp rx=0 tx=0).stop 2024-05-20T18:28:46.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.444+0000 7ff42571ff40 1 -- 172.21.15.5:0/1890550057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559248036010 msgr2=0x559247f9f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.444+0000 7ff42571ff40 1 --2- 172.21.15.5:0/1890550057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559248036010 0x559247f9f5a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff4100044f0 tx=0x7ff410001250 comp rx=0 tx=0).stop 2024-05-20T18:28:46.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.444+0000 7ff41ef69640 1 -- 172.21.15.5:0/1890550057 reap_dead start 2024-05-20T18:28:46.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.444+0000 7ff42571ff40 1 -- 172.21.15.5:0/1890550057 shutdown_connections 2024-05-20T18:28:46.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.444+0000 7ff42571ff40 1 -- 172.21.15.5:0/1890550057 >> 172.21.15.5:0/1890550057 conn(0x559247e83ac0 msgr2=0x559247f886e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:46.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.444+0000 7ff42571ff40 1 -- 172.21.15.5:0/1890550057 shutdown_connections 2024-05-20T18:28:46.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.444+0000 7ff42571ff40 1 -- 172.21.15.5:0/1890550057 wait complete. 2024-05-20T18:28:46.446 INFO:teuthology.orchestra.run.smithi005.stdout:mytag 2024-05-20T18:28:46.447 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-05-20T18:28:46.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.503+0000 7f0f67114f40 1 Processor -- start 2024-05-20T18:28:46.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.503+0000 7f0f67114f40 1 -- start start 2024-05-20T18:28:46.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.504+0000 7f0f67114f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9a049970 0x55ff9a0d9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.504+0000 7f0f67114f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9a04b5f0 0x55ff9a0d6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.504+0000 7f0f67114f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9a0da010 0x55ff9a0d6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.504+0000 7f0f67114f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ff9a030400 con 0x55ff9a049970 2024-05-20T18:28:46.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.504+0000 7f0f67114f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ff99fa9c80 con 0x55ff9a04b5f0 2024-05-20T18:28:46.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.504+0000 7f0f67114f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55ff9a0306a0 con 0x55ff9a0da010 2024-05-20T18:28:46.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.504+0000 7f0f60be3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9a049970 0x55ff9a0d9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.504+0000 7f0f60be3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9a049970 0x55ff9a0d9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33982/0 (socket says 172.21.15.5:33982) 2024-05-20T18:28:46.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.504+0000 7f0f60be3640 1 -- 172.21.15.5:0/547252228 learned_addr learned my addr 172.21.15.5:0/547252228 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.505+0000 7f0f613e4640 1 --2- 172.21.15.5:0/547252228 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9a0da010 0x55ff9a0d6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.505+0000 7f0f603e2640 1 --2- 172.21.15.5:0/547252228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9a04b5f0 0x55ff9a0d6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.505+0000 7f0f60be3640 1 -- 172.21.15.5:0/547252228 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9a0da010 msgr2=0x55ff9a0d6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.505+0000 7f0f60be3640 1 --2- 172.21.15.5:0/547252228 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9a0da010 0x55ff9a0d6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.505+0000 7f0f60be3640 1 -- 172.21.15.5:0/547252228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9a04b5f0 msgr2=0x55ff9a0d6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.505+0000 7f0f60be3640 1 --2- 172.21.15.5:0/547252228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9a04b5f0 0x55ff9a0d6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.505+0000 7f0f60be3640 1 -- 172.21.15.5:0/547252228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff99f2ecb0 con 0x55ff9a049970 2024-05-20T18:28:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.505+0000 7f0f60be3640 1 --2- 172.21.15.5:0/547252228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9a049970 0x55ff9a0d9020 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0f50054040 tx=0x7f0f50050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14bb9b23fed6c46 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.505+0000 7f0f5f3e0640 1 -- 172.21.15.5:0/547252228 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f50055070 con 0x55ff9a049970 2024-05-20T18:28:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.505+0000 7f0f5f3e0640 1 -- 172.21.15.5:0/547252228 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f50051c20 con 0x55ff9a049970 2024-05-20T18:28:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.505+0000 7f0f5f3e0640 1 -- 172.21.15.5:0/547252228 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f50064430 con 0x55ff9a049970 2024-05-20T18:28:46.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.506+0000 7f0f67114f40 1 -- 172.21.15.5:0/547252228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9a049970 msgr2=0x55ff9a0d9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.506+0000 7f0f67114f40 1 --2- 172.21.15.5:0/547252228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9a049970 0x55ff9a0d9020 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0f50054040 tx=0x7f0f50050000 comp rx=0 tx=0).stop 2024-05-20T18:28:46.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.506+0000 7f0f67114f40 1 -- 172.21.15.5:0/547252228 shutdown_connections 2024-05-20T18:28:46.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.506+0000 7f0f67114f40 1 --2- 172.21.15.5:0/547252228 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9a0da010 0x55ff9a0d6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.506+0000 7f0f67114f40 1 --2- 172.21.15.5:0/547252228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9a04b5f0 0x55ff9a0d6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.506+0000 7f0f67114f40 1 --2- 172.21.15.5:0/547252228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9a049970 0x55ff9a0d9020 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.506+0000 7f0f67114f40 1 -- 172.21.15.5:0/547252228 >> 172.21.15.5:0/547252228 conn(0x55ff99f27ac0 msgr2=0x55ff9a049160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:46.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.506+0000 7f0f67114f40 1 -- 172.21.15.5:0/547252228 shutdown_connections 2024-05-20T18:28:46.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.506+0000 7f0f67114f40 1 -- 172.21.15.5:0/547252228 wait complete. 2024-05-20T18:28:46.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.507+0000 7f0f67114f40 1 Processor -- start 2024-05-20T18:28:46.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.507+0000 7f0f67114f40 1 -- start start 2024-05-20T18:28:46.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.508+0000 7f0f67114f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9a049970 0x55ff9a0d3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.508+0000 7f0f67114f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9a04b5f0 0x55ff9a0d22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.508+0000 7f0f67114f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9a0da010 0x55ff9a0d6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.508+0000 7f0f67114f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ff9a0d2eb0 con 0x55ff9a04b5f0 2024-05-20T18:28:46.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.508+0000 7f0f67114f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ff9a04d670 con 0x55ff9a0da010 2024-05-20T18:28:46.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.508+0000 7f0f67114f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55ff9a171b30 con 0x55ff9a049970 2024-05-20T18:28:46.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.508+0000 7f0f603e2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9a04b5f0 0x55ff9a0d22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.508+0000 7f0f603e2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9a04b5f0 0x55ff9a0d22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33986/0 (socket says 172.21.15.5:33986) 2024-05-20T18:28:46.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.508+0000 7f0f603e2640 1 -- 172.21.15.5:0/2704630854 learned_addr learned my addr 172.21.15.5:0/2704630854 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:46.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.508+0000 7f0f60be3640 1 --2- 172.21.15.5:0/2704630854 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9a049970 0x55ff9a0d3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.508+0000 7f0f613e4640 1 --2- 172.21.15.5:0/2704630854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9a0da010 0x55ff9a0d6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.508+0000 7f0f603e2640 1 -- 172.21.15.5:0/2704630854 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9a049970 msgr2=0x55ff9a0d3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.508+0000 7f0f603e2640 1 --2- 172.21.15.5:0/2704630854 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9a049970 0x55ff9a0d3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.508+0000 7f0f603e2640 1 -- 172.21.15.5:0/2704630854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9a0da010 msgr2=0x55ff9a0d6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.508+0000 7f0f603e2640 1 --2- 172.21.15.5:0/2704630854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9a0da010 0x55ff9a0d6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.508+0000 7f0f603e2640 1 -- 172.21.15.5:0/2704630854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff9a025340 con 0x55ff9a04b5f0 2024-05-20T18:28:46.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.509+0000 7f0f613e4640 1 --2- 172.21.15.5:0/2704630854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9a0da010 0x55ff9a0d6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:46.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.509+0000 7f0f603e2640 1 --2- 172.21.15.5:0/2704630854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9a04b5f0 0x55ff9a0d22c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400dfa0 tx=0x7f0f5400b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:46.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.509+0000 7f0f5e3de640 1 -- 172.21.15.5:0/2704630854 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5400be30 con 0x55ff9a04b5f0 2024-05-20T18:28:46.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.509+0000 7f0f5e3de640 1 -- 172.21.15.5:0/2704630854 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f54016070 con 0x55ff9a04b5f0 2024-05-20T18:28:46.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.509+0000 7f0f67114f40 1 -- 172.21.15.5:0/2704630854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ff9a170320 con 0x55ff9a04b5f0 2024-05-20T18:28:46.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.509+0000 7f0f5e3de640 1 -- 172.21.15.5:0/2704630854 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f540116c0 con 0x55ff9a04b5f0 2024-05-20T18:28:46.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.509+0000 7f0f67114f40 1 -- 172.21.15.5:0/2704630854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff9a0490f0 con 0x55ff9a04b5f0 2024-05-20T18:28:46.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.511+0000 7f0f5e3de640 1 -- 172.21.15.5:0/2704630854 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0f54011860 con 0x55ff9a04b5f0 2024-05-20T18:28:46.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.511+0000 7f0f5e3de640 1 --2- 172.21.15.5:0/2704630854 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0f4c06a840 0x7f0f4c06cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.511+0000 7f0f5e3de640 1 -- 172.21.15.5:0/2704630854 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f0f54089a90 con 0x55ff9a04b5f0 2024-05-20T18:28:46.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.511+0000 7f0f60be3640 1 --2- 172.21.15.5:0/2704630854 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0f4c06a840 0x7f0f4c06cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.511+0000 7f0f67114f40 1 --2- 172.21.15.5:0/2704630854 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x55ff9a0d5220 0x55ff9a0d5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.511+0000 7f0f67114f40 1 -- 172.21.15.5:0/2704630854 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:1 6.19 6:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) -- 0x55ff9a0d5b70 con 0x55ff9a0d5220 2024-05-20T18:28:46.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.512+0000 7f0f60be3640 1 --2- 172.21.15.5:0/2704630854 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0f4c06a840 0x7f0f4c06cd50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0f50062000 tx=0x7f0f50066040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:46.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.512+0000 7f0f613e4640 1 --2- 172.21.15.5:0/2704630854 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x55ff9a0d5220 0x55ff9a0d5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.512+0000 7f0f613e4640 1 --2- 172.21.15.5:0/2704630854 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x55ff9a0d5220 0x55ff9a0d5630 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:46.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.513+0000 7f0f613e4640 1 -- 172.21.15.5:0/2704630854 <== osd.5 v2:172.21.15.63:6808/1616066286 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f0f580096b0 con 0x55ff9a0d5220 2024-05-20T18:28:46.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.513+0000 7f0f67114f40 1 -- 172.21.15.5:0/2704630854 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x55ff9a0d5220 msgr2=0x55ff9a0d5630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.513+0000 7f0f67114f40 1 --2- 172.21.15.5:0/2704630854 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x55ff9a0d5220 0x55ff9a0d5630 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.513+0000 7f0f67114f40 1 -- 172.21.15.5:0/2704630854 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0f4c06a840 msgr2=0x7f0f4c06cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.513+0000 7f0f67114f40 1 --2- 172.21.15.5:0/2704630854 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0f4c06a840 0x7f0f4c06cd50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0f50062000 tx=0x7f0f50066040 comp rx=0 tx=0).stop 2024-05-20T18:28:46.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.513+0000 7f0f67114f40 1 -- 172.21.15.5:0/2704630854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9a04b5f0 msgr2=0x55ff9a0d22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.513+0000 7f0f67114f40 1 --2- 172.21.15.5:0/2704630854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9a04b5f0 0x55ff9a0d22c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400dfa0 tx=0x7f0f5400b5b0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.513+0000 7f0f613e4640 1 -- 172.21.15.5:0/2704630854 reap_dead start 2024-05-20T18:28:46.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.513+0000 7f0f67114f40 1 -- 172.21.15.5:0/2704630854 shutdown_connections 2024-05-20T18:28:46.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.513+0000 7f0f67114f40 1 -- 172.21.15.5:0/2704630854 >> 172.21.15.5:0/2704630854 conn(0x55ff99f27ac0 msgr2=0x55ff9a0d3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:46.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.513+0000 7f0f67114f40 1 -- 172.21.15.5:0/2704630854 shutdown_connections 2024-05-20T18:28:46.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.513+0000 7f0f67114f40 1 -- 172.21.15.5:0/2704630854 wait complete. 2024-05-20T18:28:46.517 INFO:teuthology.orchestra.run.smithi005.stdout:mytag 2024-05-20T18:28:46.518 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-05-20T18:28:46.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.574+0000 7ff30991ff40 1 Processor -- start 2024-05-20T18:28:46.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.574+0000 7ff30991ff40 1 -- start start 2024-05-20T18:28:46.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.576+0000 7ff30991ff40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ad9065970 0x561ad9049ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.576+0000 7ff30991ff40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561ad90f2a30 0x561ad90f4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.576+0000 7ff30991ff40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ad90f6010 0x561ad90488d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.576+0000 7ff30991ff40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561ad90f6960 con 0x561ad9065970 2024-05-20T18:28:46.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.576+0000 7ff30991ff40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561ad904d550 con 0x561ad90f6010 2024-05-20T18:28:46.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.576+0000 7ff30991ff40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x561ad904c400 con 0x561ad90f2a30 2024-05-20T18:28:46.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.576+0000 7ff302ac3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ad9065970 0x561ad9049ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.576+0000 7ff302ac3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ad9065970 0x561ad9049ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33994/0 (socket says 172.21.15.5:33994) 2024-05-20T18:28:46.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.576+0000 7ff302ac3640 1 -- 172.21.15.5:0/2589454843 learned_addr learned my addr 172.21.15.5:0/2589454843 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:46.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.576+0000 7ff3032c4640 1 --2- 172.21.15.5:0/2589454843 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ad90f6010 0x561ad90488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.576+0000 7ff3022c2640 1 --2- 172.21.15.5:0/2589454843 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561ad90f2a30 0x561ad90f4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.576+0000 7ff302ac3640 1 -- 172.21.15.5:0/2589454843 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561ad90f2a30 msgr2=0x561ad90f4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:46.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.576+0000 7ff302ac3640 1 --2- 172.21.15.5:0/2589454843 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561ad90f2a30 0x561ad90f4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.576+0000 7ff302ac3640 1 -- 172.21.15.5:0/2589454843 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ad90f6010 msgr2=0x561ad90488d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:46.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.576+0000 7ff302ac3640 1 --2- 172.21.15.5:0/2589454843 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ad90f6010 0x561ad90488d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.576+0000 7ff302ac3640 1 -- 172.21.15.5:0/2589454843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ad8f5edf0 con 0x561ad9065970 2024-05-20T18:28:46.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.576+0000 7ff3022c2640 1 --2- 172.21.15.5:0/2589454843 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561ad90f2a30 0x561ad90f4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:46.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.576+0000 7ff302ac3640 1 --2- 172.21.15.5:0/2589454843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ad9065970 0x561ad9049ac0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc009810 tx=0x7ff2fc005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ef036a19d8919e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:46.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.576+0000 7ff3012c0640 1 -- 172.21.15.5:0/2589454843 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc019070 con 0x561ad9065970 2024-05-20T18:28:46.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.576+0000 7ff3012c0640 1 -- 172.21.15.5:0/2589454843 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2fc008890 con 0x561ad9065970 2024-05-20T18:28:46.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.576+0000 7ff3012c0640 1 -- 172.21.15.5:0/2589454843 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc008c60 con 0x561ad9065970 2024-05-20T18:28:46.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.577+0000 7ff30991ff40 1 -- 172.21.15.5:0/2589454843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ad9065970 msgr2=0x561ad9049ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.577+0000 7ff30991ff40 1 --2- 172.21.15.5:0/2589454843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ad9065970 0x561ad9049ac0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc009810 tx=0x7ff2fc005a00 comp rx=0 tx=0).stop 2024-05-20T18:28:46.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.577+0000 7ff30991ff40 1 -- 172.21.15.5:0/2589454843 shutdown_connections 2024-05-20T18:28:46.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.577+0000 7ff30991ff40 1 --2- 172.21.15.5:0/2589454843 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ad90f6010 0x561ad90488d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.577+0000 7ff30991ff40 1 --2- 172.21.15.5:0/2589454843 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561ad90f2a30 0x561ad90f4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.577+0000 7ff30991ff40 1 --2- 172.21.15.5:0/2589454843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ad9065970 0x561ad9049ac0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.577+0000 7ff30991ff40 1 -- 172.21.15.5:0/2589454843 >> 172.21.15.5:0/2589454843 conn(0x561ad8f43ac0 msgr2=0x561ad904a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:46.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.577+0000 7ff30991ff40 1 -- 172.21.15.5:0/2589454843 shutdown_connections 2024-05-20T18:28:46.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.577+0000 7ff30991ff40 1 -- 172.21.15.5:0/2589454843 wait complete. 2024-05-20T18:28:46.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.578+0000 7ff30991ff40 1 Processor -- start 2024-05-20T18:28:46.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.578+0000 7ff30991ff40 1 -- start start 2024-05-20T18:28:46.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.579+0000 7ff30991ff40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ad9065970 0x561ad90ec540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.579+0000 7ff30991ff40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561ad90f2a30 0x561ad90f0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.579+0000 7ff30991ff40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ad90f6010 0x561ad9063340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.579+0000 7ff30991ff40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561ad904df70 con 0x561ad90f6010 2024-05-20T18:28:46.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.579+0000 7ff30991ff40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561ad90d8f10 con 0x561ad9065970 2024-05-20T18:28:46.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.579+0000 7ff30991ff40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x561ad90641f0 con 0x561ad90f2a30 2024-05-20T18:28:46.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.579+0000 7ff3032c4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ad90f6010 0x561ad9063340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.579+0000 7ff3032c4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ad90f6010 0x561ad9063340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34000/0 (socket says 172.21.15.5:34000) 2024-05-20T18:28:46.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.579+0000 7ff3032c4640 1 -- 172.21.15.5:0/4106136811 learned_addr learned my addr 172.21.15.5:0/4106136811 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:46.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.580+0000 7ff302ac3640 1 --2- 172.21.15.5:0/4106136811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ad9065970 0x561ad90ec540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.580+0000 7ff3022c2640 1 --2- 172.21.15.5:0/4106136811 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561ad90f2a30 0x561ad90f0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.580+0000 7ff3032c4640 1 -- 172.21.15.5:0/4106136811 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561ad90f2a30 msgr2=0x561ad90f0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.580+0000 7ff3032c4640 1 --2- 172.21.15.5:0/4106136811 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561ad90f2a30 0x561ad90f0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.580+0000 7ff3032c4640 1 -- 172.21.15.5:0/4106136811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ad9065970 msgr2=0x561ad90ec540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.580+0000 7ff3032c4640 1 --2- 172.21.15.5:0/4106136811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561ad9065970 0x561ad90ec540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.580+0000 7ff3032c4640 1 -- 172.21.15.5:0/4106136811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ad8f4acb0 con 0x561ad90f6010 2024-05-20T18:28:46.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.580+0000 7ff3032c4640 1 --2- 172.21.15.5:0/4106136811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ad90f6010 0x561ad9063340 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400a190 tx=0x7ff2f4003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:46.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.580+0000 7ff2fbfff640 1 -- 172.21.15.5:0/4106136811 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4052550 con 0x561ad90f6010 2024-05-20T18:28:46.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.580+0000 7ff2fbfff640 1 -- 172.21.15.5:0/4106136811 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2f4052710 con 0x561ad90f6010 2024-05-20T18:28:46.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.580+0000 7ff30991ff40 1 -- 172.21.15.5:0/4106136811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561ad9048970 con 0x561ad90f6010 2024-05-20T18:28:46.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.580+0000 7ff2fbfff640 1 -- 172.21.15.5:0/4106136811 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4052b00 con 0x561ad90f6010 2024-05-20T18:28:46.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.580+0000 7ff30991ff40 1 -- 172.21.15.5:0/4106136811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ad9048c10 con 0x561ad90f6010 2024-05-20T18:28:46.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.582+0000 7ff2fbfff640 1 -- 172.21.15.5:0/4106136811 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff2f4052ca0 con 0x561ad90f6010 2024-05-20T18:28:46.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.582+0000 7ff2fbfff640 1 --2- 172.21.15.5:0/4106136811 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff2e406a810 0x7ff2e406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.582+0000 7ff2fbfff640 1 -- 172.21.15.5:0/4106136811 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7ff2f40cda90 con 0x561ad90f6010 2024-05-20T18:28:46.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.582+0000 7ff302ac3640 1 --2- 172.21.15.5:0/4106136811 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff2e406a810 0x7ff2e406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.582+0000 7ff30991ff40 1 --2- 172.21.15.5:0/4106136811 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x561ad90f0bc0 0x561ad9190420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.582+0000 7ff30991ff40 1 -- 172.21.15.5:0/4106136811 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:1 6.12 6:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) -- 0x561ad90e97c0 con 0x561ad90f0bc0 2024-05-20T18:28:46.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.583+0000 7ff3022c2640 1 --2- 172.21.15.5:0/4106136811 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x561ad90f0bc0 0x561ad9190420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.583+0000 7ff302ac3640 1 --2- 172.21.15.5:0/4106136811 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff2e406a810 0x7ff2e406cd20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc017040 tx=0x7ff2fc008000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:46.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.583+0000 7ff3022c2640 1 --2- 172.21.15.5:0/4106136811 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x561ad90f0bc0 0x561ad9190420 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:46.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.583+0000 7ff3022c2640 1 -- 172.21.15.5:0/4106136811 <== osd.7 v2:172.21.15.63:6824/4136124451 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7ff2ec0097b0 con 0x561ad90f0bc0 2024-05-20T18:28:46.587 INFO:teuthology.orchestra.run.smithi005.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2024-05-20T18:28:46.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.584+0000 7ff30991ff40 1 -- 172.21.15.5:0/4106136811 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x561ad90f0bc0 msgr2=0x561ad9190420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.584+0000 7ff30991ff40 1 --2- 172.21.15.5:0/4106136811 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x561ad90f0bc0 0x561ad9190420 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.584+0000 7ff30991ff40 1 -- 172.21.15.5:0/4106136811 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff2e406a810 msgr2=0x7ff2e406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.584+0000 7ff30991ff40 1 --2- 172.21.15.5:0/4106136811 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff2e406a810 0x7ff2e406cd20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc017040 tx=0x7ff2fc008000 comp rx=0 tx=0).stop 2024-05-20T18:28:46.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.584+0000 7ff30991ff40 1 -- 172.21.15.5:0/4106136811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ad90f6010 msgr2=0x561ad9063340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.584+0000 7ff30991ff40 1 --2- 172.21.15.5:0/4106136811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ad90f6010 0x561ad9063340 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400a190 tx=0x7ff2f4003330 comp rx=0 tx=0).stop 2024-05-20T18:28:46.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.584+0000 7ff3032c4640 1 -- 172.21.15.5:0/4106136811 reap_dead start 2024-05-20T18:28:46.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.584+0000 7ff30991ff40 1 -- 172.21.15.5:0/4106136811 shutdown_connections 2024-05-20T18:28:46.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.584+0000 7ff30991ff40 1 -- 172.21.15.5:0/4106136811 >> 172.21.15.5:0/4106136811 conn(0x561ad8f43ac0 msgr2=0x561ad90497c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:46.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.584+0000 7ff30991ff40 1 -- 172.21.15.5:0/4106136811 shutdown_connections 2024-05-20T18:28:46.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.584+0000 7ff30991ff40 1 -- 172.21.15.5:0/4106136811 wait complete. 2024-05-20T18:28:46.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T18:28:46.588 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-20T18:28:46.589 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T18:28:46.625 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T18:28:46.675 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-20T18:28:46.675 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T18:28:46.675 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:28:46.675 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:28:46.692 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15663 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T18:28:46.692 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T18:28:46.692 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:28:46.692 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:28:46.711 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19732 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T18:28:46.711 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T18:28:46.711 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:28:46.711 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:28:46.748 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15663 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T18:28:46.748 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T18:28:46.749 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:28:46.749 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:28:46.769 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19732 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T18:28:46.769 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:28:46.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.903+0000 7f0a9c2b6640 1 Processor -- start 2024-05-20T18:28:46.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.903+0000 7f0a9c2b6640 1 -- start start 2024-05-20T18:28:46.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.904+0000 7f0a9c2b6640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0a940b6d90 0x7f0a9415f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.904+0000 7f0a9c2b6640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a940d0550 con 0x7f0a940cf250 2024-05-20T18:28:46.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.904+0000 7f0a9c2b6640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a940d0830 con 0x7f0a940d3220 2024-05-20T18:28:46.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.904+0000 7f0a9c2b6640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f0a9415f9a0 con 0x7f0a940b6d90 2024-05-20T18:28:46.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.904+0000 7f0a99851640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0a940b6d90 0x7f0a9415f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.904+0000 7f0a99851640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0a940b6d90 0x7f0a9415f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34012/0 (socket says 172.21.15.5:34012) 2024-05-20T18:28:46.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.904+0000 7f0a9a052640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0a940cf250 0x7f0a940b6680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:41244/0 (socket says 172.21.15.5:41244) 2024-05-20T18:28:46.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.904+0000 7f0a99851640 1 -- 172.21.15.5:0/2367357297 learned_addr learned my addr 172.21.15.5:0/2367357297 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:46.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.905+0000 7f0a99851640 1 -- 172.21.15.5:0/2367357297 >> v1:172.21.15.63:6789/0 conn(0x7f0a940d3220 legacy=0x7f0a940d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:46.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.905+0000 7f0a99851640 1 -- 172.21.15.5:0/2367357297 >> v1:172.21.15.5:6789/0 conn(0x7f0a940cf250 legacy=0x7f0a940b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.905+0000 7f0a99851640 1 -- 172.21.15.5:0/2367357297 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a940d2a20 con 0x7f0a940b6d90 2024-05-20T18:28:46.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.905+0000 7f0a99050640 1 -- 172.21.15.5:0/2367357297 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2039373327 0 0) 0x7f0a940d0550 con 0x7f0a940cf250 2024-05-20T18:28:46.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.906+0000 7f0a99851640 1 --2- 172.21.15.5:0/2367357297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a940b6d90 0x7f0a9415f460 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009870 tx=0x7f0a8402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=556ea5bad0cbf749 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:46.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.906+0000 7f0a99050640 1 -- 172.21.15.5:0/2367357297 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a84037070 con 0x7f0a940b6d90 2024-05-20T18:28:46.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.906+0000 7f0a99050640 1 -- 172.21.15.5:0/2367357297 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a84032440 con 0x7f0a940b6d90 2024-05-20T18:28:46.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.906+0000 7f0a99050640 1 -- 172.21.15.5:0/2367357297 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a84032810 con 0x7f0a940b6d90 2024-05-20T18:28:46.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.906+0000 7f0a9c2b6640 1 -- 172.21.15.5:0/2367357297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a940b6d90 msgr2=0x7f0a9415f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.906+0000 7f0a9c2b6640 1 --2- 172.21.15.5:0/2367357297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a940b6d90 0x7f0a9415f460 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009870 tx=0x7f0a8402a2a0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.906+0000 7f0a9c2b6640 1 -- 172.21.15.5:0/2367357297 shutdown_connections 2024-05-20T18:28:46.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.906+0000 7f0a9c2b6640 1 --2- 172.21.15.5:0/2367357297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a940b6d90 0x7f0a9415f460 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.906+0000 7f0a9c2b6640 1 -- 172.21.15.5:0/2367357297 >> 172.21.15.5:0/2367357297 conn(0x7f0a940d1860 msgr2=0x7f0a940d1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:46.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.907+0000 7f0a9c2b6640 1 -- 172.21.15.5:0/2367357297 shutdown_connections 2024-05-20T18:28:46.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.907+0000 7f0a9c2b6640 1 -- 172.21.15.5:0/2367357297 wait complete. 2024-05-20T18:28:46.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.908+0000 7f0a9c2b6640 1 Processor -- start 2024-05-20T18:28:46.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.908+0000 7f0a9c2b6640 1 -- start start 2024-05-20T18:28:46.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.910+0000 7f0a9c2b6640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0a940cf250 0x7f0a9414bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.910+0000 7f0a9c2b6640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a940d3220 0x7f0a941e9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.910+0000 7f0a9c2b6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a9415cae0 0x7f0a9415cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.910+0000 7f0a9c2b6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0a940d27d0 con 0x7f0a9415cae0 2024-05-20T18:28:46.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.910+0000 7f0a9c2b6640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0a940d0140 con 0x7f0a940d3220 2024-05-20T18:28:46.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.910+0000 7f0a9c2b6640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0a940d02c0 con 0x7f0a940cf250 2024-05-20T18:28:46.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.910+0000 7f0a9a853640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a9415cae0 0x7f0a9415cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.910+0000 7f0a9a853640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a9415cae0 0x7f0a9415cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34018/0 (socket says 172.21.15.5:34018) 2024-05-20T18:28:46.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.910+0000 7f0a9a853640 1 -- 172.21.15.5:0/3900620870 learned_addr learned my addr 172.21.15.5:0/3900620870 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:46.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.910+0000 7f0a99851640 1 --2- 172.21.15.5:0/3900620870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a940d3220 0x7f0a941e9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.910+0000 7f0a9a052640 1 --2- 172.21.15.5:0/3900620870 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0a940cf250 0x7f0a9414bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.910+0000 7f0a9a853640 1 -- 172.21.15.5:0/3900620870 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0a940cf250 msgr2=0x7f0a9414bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.910+0000 7f0a9a853640 1 --2- 172.21.15.5:0/3900620870 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0a940cf250 0x7f0a9414bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.910+0000 7f0a9a853640 1 -- 172.21.15.5:0/3900620870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a940d3220 msgr2=0x7f0a941e9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:46.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.910+0000 7f0a9a853640 1 --2- 172.21.15.5:0/3900620870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a940d3220 0x7f0a941e9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:46.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.910+0000 7f0a9a853640 1 -- 172.21.15.5:0/3900620870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9415d430 con 0x7f0a9415cae0 2024-05-20T18:28:46.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.910+0000 7f0a99851640 1 --2- 172.21.15.5:0/3900620870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a940d3220 0x7f0a941e9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:46.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.910+0000 7f0a9a052640 1 --2- 172.21.15.5:0/3900620870 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0a940cf250 0x7f0a9414bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:46.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.910+0000 7f0a9a853640 1 --2- 172.21.15.5:0/3900620870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a9415cae0 0x7f0a9415cef0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00b970 tx=0x7f0a8c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:46.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.910+0000 7f0a8affd640 1 -- 172.21.15.5:0/3900620870 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c019070 con 0x7f0a9415cae0 2024-05-20T18:28:46.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.910+0000 7f0a8affd640 1 -- 172.21.15.5:0/3900620870 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a8c00cb40 con 0x7f0a9415cae0 2024-05-20T18:28:46.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.910+0000 7f0a8affd640 1 -- 172.21.15.5:0/3900620870 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c014440 con 0x7f0a9415cae0 2024-05-20T18:28:46.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.910+0000 7f0a9c2b6640 1 -- 172.21.15.5:0/3900620870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a94161b20 con 0x7f0a9415cae0 2024-05-20T18:28:46.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.911+0000 7f0a9c2b6640 1 -- 172.21.15.5:0/3900620870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a941620e0 con 0x7f0a9415cae0 2024-05-20T18:28:46.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.911+0000 7f0a9c2b6640 1 -- 172.21.15.5:0/3900620870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a94161890 con 0x7f0a9415cae0 2024-05-20T18:28:46.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.912+0000 7f0a8affd640 1 -- 172.21.15.5:0/3900620870 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0a8c01c070 con 0x7f0a9415cae0 2024-05-20T18:28:46.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.913+0000 7f0a8affd640 1 --2- 172.21.15.5:0/3900620870 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0a7806be30 0x7f0a7806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:46.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.913+0000 7f0a9a052640 1 --2- 172.21.15.5:0/3900620870 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0a7806be30 0x7f0a7806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:46.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.913+0000 7f0a8affd640 1 -- 172.21.15.5:0/3900620870 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f0a8c08ad40 con 0x7f0a9415cae0 2024-05-20T18:28:46.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.913+0000 7f0a9a052640 1 --2- 172.21.15.5:0/3900620870 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0a7806be30 0x7f0a7806e340 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0a90000f60 tx=0x7f0a90009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:46.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:46.917+0000 7f0a8affd640 1 -- 172.21.15.5:0/3900620870 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0a9415d430 con 0x7f0a9415cae0 2024-05-20T18:28:47.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.141+0000 7f0a9c2b6640 1 -- 172.21.15.5:0/3900620870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a94075b00 con 0x7f0a9415cae0 2024-05-20T18:28:47.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.141+0000 7f0a8affd640 1 -- 172.21.15.5:0/3900620870 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5175 (secure 0 0 0) 0x7f0a8c059430 con 0x7f0a9415cae0 2024-05-20T18:28:47.141 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:47.141 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1458170078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1458170078},{"type":"v1","addr":"172.21.15.63:6834","nonce":1458170078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4916,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/2601750663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":2601750663},{"type":"v1","addr":"172.21.15.63:6835","nonce":2601750663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5313,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/1913030219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1913030219},{"type":"v1","addr":"172.21.15.5:6836","nonce":1913030219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:28:24.270282+0000","modified":"2024-05-20T18:28:27.296534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5316},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5316":{"gid":5316,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6837/2355805105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":2355805105},{"type":"v1","addr":"172.21.15.5:6837","nonce":2355805105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5316,"qdb_cluster":[5316]},"id":2}]} 2024-05-20T18:28:47.141 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 17 2024-05-20T18:28:47.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.143+0000 7f0a9c2b6640 1 -- 172.21.15.5:0/3900620870 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0a7806be30 msgr2=0x7f0a7806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:47.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.143+0000 7f0a9c2b6640 1 --2- 172.21.15.5:0/3900620870 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0a7806be30 0x7f0a7806e340 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0a90000f60 tx=0x7f0a90009040 comp rx=0 tx=0).stop 2024-05-20T18:28:47.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.143+0000 7f0a9c2b6640 1 -- 172.21.15.5:0/3900620870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a9415cae0 msgr2=0x7f0a9415cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:47.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.143+0000 7f0a9c2b6640 1 --2- 172.21.15.5:0/3900620870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a9415cae0 0x7f0a9415cef0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00b970 tx=0x7f0a8c00be30 comp rx=0 tx=0).stop 2024-05-20T18:28:47.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.143+0000 7f0a9c2b6640 1 -- 172.21.15.5:0/3900620870 shutdown_connections 2024-05-20T18:28:47.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.143+0000 7f0a9c2b6640 1 --2- 172.21.15.5:0/3900620870 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0a7806be30 0x7f0a7806e340 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:47.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.143+0000 7f0a9c2b6640 1 --2- 172.21.15.5:0/3900620870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a9415cae0 0x7f0a9415cef0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:47.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.143+0000 7f0a9c2b6640 1 --2- 172.21.15.5:0/3900620870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a940d3220 0x7f0a941e9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:47.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.143+0000 7f0a9c2b6640 1 --2- 172.21.15.5:0/3900620870 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0a940cf250 0x7f0a9414bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:47.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.143+0000 7f0a9c2b6640 1 -- 172.21.15.5:0/3900620870 >> 172.21.15.5:0/3900620870 conn(0x7f0a940d1860 msgr2=0x7f0a9415d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:47.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.143+0000 7f0a9c2b6640 1 -- 172.21.15.5:0/3900620870 shutdown_connections 2024-05-20T18:28:47.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.144+0000 7f0a9c2b6640 1 -- 172.21.15.5:0/3900620870 wait complete. 2024-05-20T18:28:47.155 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:28:47.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.270+0000 7f752b206640 1 Processor -- start 2024-05-20T18:28:47.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.271+0000 7f752b206640 1 -- start start 2024-05-20T18:28:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.272+0000 7f752b206640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7524168050 0x7f752416a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.272+0000 7f752b206640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75240d0630 con 0x7f752406d3d0 2024-05-20T18:28:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.272+0000 7f752b206640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75240d39c0 con 0x7f7524162820 2024-05-20T18:28:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.272+0000 7f752b206640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f75240d3ca0 con 0x7f7524168050 2024-05-20T18:28:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.272+0000 7f7523fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f752406d3d0 0x7f752406d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:41246/0 (socket says 172.21.15.5:41246) 2024-05-20T18:28:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.272+0000 7f7523fff640 1 -- 172.21.15.5:0/2269701282 learned_addr learned my addr 172.21.15.5:0/2269701282 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.272+0000 7f75297a3640 1 --2- 172.21.15.5:0/2269701282 >> v2:172.21.15.63:3300/0 conn(0x7f7524168050 0x7f752416a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.272+0000 7f75237fe640 1 -- 172.21.15.5:0/2269701282 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2573454209 0 0) 0x7f75240d0630 con 0x7f752406d3d0 2024-05-20T18:28:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.272+0000 7f75237fe640 1 -- 172.21.15.5:0/2269701282 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7510003520 con 0x7f752406d3d0 2024-05-20T18:28:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.273+0000 7f75237fe640 1 -- 172.21.15.5:0/2269701282 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 174566391 0 0) 0x7f7510003520 con 0x7f752406d3d0 2024-05-20T18:28:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.273+0000 7f75237fe640 1 -- 172.21.15.5:0/2269701282 >> v2:172.21.15.63:3300/0 conn(0x7f7524168050 msgr2=0x7f752416a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.273+0000 7f75237fe640 1 --2- 172.21.15.5:0/2269701282 >> v2:172.21.15.63:3300/0 conn(0x7f7524168050 0x7f752416a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.273+0000 7f75237fe640 1 -- 172.21.15.5:0/2269701282 >> v1:172.21.15.63:6789/0 conn(0x7f7524162820 legacy=0x7f75240b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.273+0000 7f75237fe640 1 -- 172.21.15.5:0/2269701282 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75240d3f80 con 0x7f752406d3d0 2024-05-20T18:28:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.273+0000 7f75237fe640 1 -- 172.21.15.5:0/2269701282 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f750c001670 con 0x7f752406d3d0 2024-05-20T18:28:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.273+0000 7f75237fe640 1 -- 172.21.15.5:0/2269701282 >> v1:172.21.15.5:6789/0 conn(0x7f752406d3d0 legacy=0x7f752406d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.273+0000 7f75237fe640 1 --2- 172.21.15.5:0/2269701282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75100039b0 0x7f7510003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.273+0000 7f75237fe640 1 --2- 172.21.15.5:0/2269701282 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7510004800 0x7f7510004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.273+0000 7f75237fe640 1 --2- 172.21.15.5:0/2269701282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75100051c0 0x7f751000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.273+0000 7f75237fe640 1 -- 172.21.15.5:0/2269701282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f75240d3ca0 con 0x7f75100039b0 2024-05-20T18:28:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.273+0000 7f75237fe640 1 -- 172.21.15.5:0/2269701282 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f751000c160 con 0x7f75100051c0 2024-05-20T18:28:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.273+0000 7f75237fe640 1 -- 172.21.15.5:0/2269701282 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f751000c360 con 0x7f7510004800 2024-05-20T18:28:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.273+0000 7f7528fa2640 1 --2- 172.21.15.5:0/2269701282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75100039b0 0x7f7510003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.273+0000 7f7523fff640 1 --2- 172.21.15.5:0/2269701282 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7510004800 0x7f7510004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.273+0000 7f75297a3640 1 --2- 172.21.15.5:0/2269701282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75100051c0 0x7f751000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:47.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.273+0000 7f7528fa2640 1 -- 172.21.15.5:0/2269701282 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7510004800 msgr2=0x7f7510004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:47.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.273+0000 7f7528fa2640 1 --2- 172.21.15.5:0/2269701282 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7510004800 0x7f7510004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:47.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.273+0000 7f7528fa2640 1 -- 172.21.15.5:0/2269701282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75100051c0 msgr2=0x7f751000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:47.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.273+0000 7f7528fa2640 1 --2- 172.21.15.5:0/2269701282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75100051c0 0x7f751000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:47.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.273+0000 7f75297a3640 1 -- 172.21.15.5:0/2269701282 reap_dead start 2024-05-20T18:28:47.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.273+0000 7f7528fa2640 1 -- 172.21.15.5:0/2269701282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7510003520 con 0x7f75100039b0 2024-05-20T18:28:47.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.273+0000 7f7523fff640 1 --2- 172.21.15.5:0/2269701282 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7510004800 0x7f7510004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:47.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.274+0000 7f7528fa2640 1 --2- 172.21.15.5:0/2269701282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75100039b0 0x7f7510003e20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f751c001750 tx=0x7f7510005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631c6b67b12eb053 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:47.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.274+0000 7f75237fe640 1 -- 172.21.15.5:0/2269701282 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751c001370 con 0x7f75100039b0 2024-05-20T18:28:47.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.274+0000 7f75237fe640 1 -- 172.21.15.5:0/2269701282 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f751c0052d0 con 0x7f75100039b0 2024-05-20T18:28:47.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.274+0000 7f75237fe640 1 -- 172.21.15.5:0/2269701282 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751c02ab00 con 0x7f75100039b0 2024-05-20T18:28:47.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.275+0000 7f752b206640 1 -- 172.21.15.5:0/2269701282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75100039b0 msgr2=0x7f7510003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:47.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.275+0000 7f752b206640 1 --2- 172.21.15.5:0/2269701282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75100039b0 0x7f7510003e20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f751c001750 tx=0x7f7510005830 comp rx=0 tx=0).stop 2024-05-20T18:28:47.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.275+0000 7f752b206640 1 -- 172.21.15.5:0/2269701282 shutdown_connections 2024-05-20T18:28:47.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.275+0000 7f752b206640 1 --2- 172.21.15.5:0/2269701282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75100039b0 0x7f7510003e20 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.275+0000 7f752b206640 1 -- 172.21.15.5:0/2269701282 >> 172.21.15.5:0/2269701282 conn(0x7f75240d4bd0 msgr2=0x7f75240d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.275+0000 7f752b206640 1 -- 172.21.15.5:0/2269701282 shutdown_connections 2024-05-20T18:28:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.275+0000 7f752b206640 1 -- 172.21.15.5:0/2269701282 wait complete. 2024-05-20T18:28:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.277+0000 7f752b206640 1 Processor -- start 2024-05-20T18:28:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.277+0000 7f752b206640 1 -- start start 2024-05-20T18:28:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.278+0000 7f752b206640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75100039b0 0x7f75240cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.278+0000 7f752b206640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75241503f0 0x7f752414fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.278+0000 7f752b206640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f75240cbc60 0x7f75240cc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.278+0000 7f752b206640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f75240d4780 con 0x7f75100039b0 2024-05-20T18:28:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.278+0000 7f752b206640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f75240d0300 con 0x7f75241503f0 2024-05-20T18:28:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.278+0000 7f752b206640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f75240d4900 con 0x7f75240cbc60 2024-05-20T18:28:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.278+0000 7f7528fa2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75100039b0 0x7f75240cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:47.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.278+0000 7f7528fa2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75100039b0 0x7f75240cab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34050/0 (socket says 172.21.15.5:34050) 2024-05-20T18:28:47.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.278+0000 7f7528fa2640 1 -- 172.21.15.5:0/2063472640 learned_addr learned my addr 172.21.15.5:0/2063472640 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:47.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.278+0000 7f7523fff640 1 --2- 172.21.15.5:0/2063472640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75241503f0 0x7f752414fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:47.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.278+0000 7f75297a3640 1 --2- 172.21.15.5:0/2063472640 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f75240cbc60 0x7f75240cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:47.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.279+0000 7f7528fa2640 1 -- 172.21.15.5:0/2063472640 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f75240cbc60 msgr2=0x7f75240cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:47.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.279+0000 7f7528fa2640 1 --2- 172.21.15.5:0/2063472640 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f75240cbc60 0x7f75240cc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:47.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.279+0000 7f7528fa2640 1 -- 172.21.15.5:0/2063472640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75241503f0 msgr2=0x7f752414fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:47.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.279+0000 7f7528fa2640 1 --2- 172.21.15.5:0/2063472640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75241503f0 0x7f752414fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:47.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.279+0000 7f7528fa2640 1 -- 172.21.15.5:0/2063472640 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75240cc5b0 con 0x7f75100039b0 2024-05-20T18:28:47.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.279+0000 7f7528fa2640 1 --2- 172.21.15.5:0/2063472640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75100039b0 0x7f75240cab30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f751c000c00 tx=0x7f751c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:47.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.279+0000 7f75217fa640 1 -- 172.21.15.5:0/2063472640 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751c037040 con 0x7f75100039b0 2024-05-20T18:28:47.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.279+0000 7f75217fa640 1 -- 172.21.15.5:0/2063472640 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f751c005480 con 0x7f75100039b0 2024-05-20T18:28:47.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.279+0000 7f752b206640 1 -- 172.21.15.5:0/2063472640 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75240c9a90 con 0x7f75100039b0 2024-05-20T18:28:47.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.279+0000 7f75217fa640 1 -- 172.21.15.5:0/2063472640 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751c007a30 con 0x7f75100039b0 2024-05-20T18:28:47.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.279+0000 7f752b206640 1 -- 172.21.15.5:0/2063472640 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75240ca050 con 0x7f75100039b0 2024-05-20T18:28:47.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.280+0000 7f752b206640 1 -- 172.21.15.5:0/2063472640 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7524075b00 con 0x7f75100039b0 2024-05-20T18:28:47.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.281+0000 7f75217fa640 1 -- 172.21.15.5:0/2063472640 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f751c044020 con 0x7f75100039b0 2024-05-20T18:28:47.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.281+0000 7f75217fa640 1 --2- 172.21.15.5:0/2063472640 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f750406be60 0x7f750406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:47.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.281+0000 7f75217fa640 1 -- 172.21.15.5:0/2063472640 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f751c0aadb0 con 0x7f75100039b0 2024-05-20T18:28:47.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.281+0000 7f7523fff640 1 --2- 172.21.15.5:0/2063472640 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f750406be60 0x7f750406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:47.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.282+0000 7f7523fff640 1 --2- 172.21.15.5:0/2063472640 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f750406be60 0x7f750406e370 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f750c001eb0 tx=0x7f750c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:47.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.286+0000 7f75217fa640 1 -- 172.21.15.5:0/2063472640 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f75240cc5b0 con 0x7f75100039b0 2024-05-20T18:28:47.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.473+0000 7f752b206640 1 -- 172.21.15.5:0/2063472640 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f752408dd50 con 0x7f75100039b0 2024-05-20T18:28:47.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.474+0000 7f75217fa640 1 -- 172.21.15.5:0/2063472640 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16822 (secure 0 0 0) 0x7f751c079540 con 0x7f75100039b0 2024-05-20T18:28:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:47.474 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":40,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:28:25.902315+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:28:22.042995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-20T18:28:23.252728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:28:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.476+0000 7f752b206640 1 -- 172.21.15.5:0/2063472640 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f750406be60 msgr2=0x7f750406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.476+0000 7f752b206640 1 --2- 172.21.15.5:0/2063472640 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f750406be60 0x7f750406e370 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f750c001eb0 tx=0x7f750c009040 comp rx=0 tx=0).stop 2024-05-20T18:28:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.476+0000 7f752b206640 1 -- 172.21.15.5:0/2063472640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75100039b0 msgr2=0x7f75240cab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.476+0000 7f752b206640 1 --2- 172.21.15.5:0/2063472640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75100039b0 0x7f75240cab30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f751c000c00 tx=0x7f751c001120 comp rx=0 tx=0).stop 2024-05-20T18:28:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.476+0000 7f752b206640 1 -- 172.21.15.5:0/2063472640 shutdown_connections 2024-05-20T18:28:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.476+0000 7f752b206640 1 --2- 172.21.15.5:0/2063472640 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f750406be60 0x7f750406e370 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.476+0000 7f752b206640 1 --2- 172.21.15.5:0/2063472640 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f75240cbc60 0x7f75240cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.476+0000 7f752b206640 1 --2- 172.21.15.5:0/2063472640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75241503f0 0x7f752414fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.476+0000 7f752b206640 1 --2- 172.21.15.5:0/2063472640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75100039b0 0x7f75240cab30 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.476+0000 7f752b206640 1 -- 172.21.15.5:0/2063472640 >> 172.21.15.5:0/2063472640 conn(0x7f75240d4bd0 msgr2=0x7f7524159a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.476+0000 7f752b206640 1 -- 172.21.15.5:0/2063472640 shutdown_connections 2024-05-20T18:28:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.476+0000 7f752b206640 1 -- 172.21.15.5:0/2063472640 wait complete. 2024-05-20T18:28:47.487 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:28:22.042995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:28:23.252728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:28:47.488 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:28:47.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.606+0000 7f0245d77640 1 Processor -- start 2024-05-20T18:28:47.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.606+0000 7f0245d77640 1 -- start start 2024-05-20T18:28:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.607+0000 7f0245d77640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f02400b6d90 0x7f024015f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.607+0000 7f0245d77640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f02400d3220 0x7f02400d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.607+0000 7f0245d77640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02400d0550 con 0x7f02400cf250 2024-05-20T18:28:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.607+0000 7f0245d77640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f024015f9a0 con 0x7f02400d3220 2024-05-20T18:28:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.607+0000 7f0245d77640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f02400d0b10 con 0x7f02400b6d90 2024-05-20T18:28:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.608+0000 7f023ffff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f02400d3220 0x7f02400d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.608+0000 7f023ffff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f02400d3220 0x7f02400d3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34058/0 (socket says 172.21.15.5:34058) 2024-05-20T18:28:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.608+0000 7f023ffff640 1 -- 172.21.15.5:0/3542123658 learned_addr learned my addr 172.21.15.5:0/3542123658 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.608+0000 7f0237fff640 1 --2- 172.21.15.5:0/3542123658 >> v2:172.21.15.63:3300/0 conn(0x7f02400b6d90 0x7f024015f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.608+0000 7f023ffff640 1 -- 172.21.15.5:0/3542123658 >> v2:172.21.15.63:3300/0 conn(0x7f02400b6d90 msgr2=0x7f024015f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.608+0000 7f023ffff640 1 --2- 172.21.15.5:0/3542123658 >> v2:172.21.15.63:3300/0 conn(0x7f02400b6d90 0x7f024015f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.608+0000 7f023ffff640 1 -- 172.21.15.5:0/3542123658 >> v1:172.21.15.63:6789/0 conn(0x7f02400cf250 legacy=0x7f02400b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.608+0000 7f023ffff640 1 -- 172.21.15.5:0/3542123658 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02400d0830 con 0x7f02400d3220 2024-05-20T18:28:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.609+0000 7f023ffff640 1 --2- 172.21.15.5:0/3542123658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02400d3220 0x7f02400d3630 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f02400d4bb0 tx=0x7f023802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8f385ee94157ac server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.609+0000 7f023effd640 1 -- 172.21.15.5:0/3542123658 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0238036070 con 0x7f02400d3220 2024-05-20T18:28:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.609+0000 7f023effd640 1 -- 172.21.15.5:0/3542123658 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0238031600 con 0x7f02400d3220 2024-05-20T18:28:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.609+0000 7f023effd640 1 -- 172.21.15.5:0/3542123658 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02380319d0 con 0x7f02400d3220 2024-05-20T18:28:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.610+0000 7f0245d77640 1 -- 172.21.15.5:0/3542123658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02400d3220 msgr2=0x7f02400d3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.610+0000 7f0245d77640 1 --2- 172.21.15.5:0/3542123658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02400d3220 0x7f02400d3630 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f02400d4bb0 tx=0x7f023802a730 comp rx=0 tx=0).stop 2024-05-20T18:28:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.610+0000 7f0245d77640 1 -- 172.21.15.5:0/3542123658 shutdown_connections 2024-05-20T18:28:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.610+0000 7f0245d77640 1 --2- 172.21.15.5:0/3542123658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02400d3220 0x7f02400d3630 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.610+0000 7f0245d77640 1 --2- 172.21.15.5:0/3542123658 >> v2:172.21.15.63:3300/0 conn(0x7f02400b6d90 0x7f024015f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.610+0000 7f0245d77640 1 -- 172.21.15.5:0/3542123658 >> 172.21.15.5:0/3542123658 conn(0x7f02400d1860 msgr2=0x7f02400d1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.610+0000 7f0245d77640 1 -- 172.21.15.5:0/3542123658 shutdown_connections 2024-05-20T18:28:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.610+0000 7f0245d77640 1 -- 172.21.15.5:0/3542123658 wait complete. 2024-05-20T18:28:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.611+0000 7f0245d77640 1 Processor -- start 2024-05-20T18:28:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.612+0000 7f0245d77640 1 -- start start 2024-05-20T18:28:47.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.613+0000 7f0245d77640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02400cf250 0x7f0240158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:47.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.613+0000 7f0245d77640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02400d3220 0x7f02400cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:47.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.613+0000 7f0245d77640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02401f8ce0 0x7f02401f90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:47.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.613+0000 7f0245d77640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f02400d2d00 con 0x7f02401f8ce0 2024-05-20T18:28:47.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.613+0000 7f0245d77640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f02400d27d0 con 0x7f02400d3220 2024-05-20T18:28:47.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.613+0000 7f0245d77640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f02400d0140 con 0x7f02400cf250 2024-05-20T18:28:47.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.613+0000 7f023ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02401f8ce0 0x7f02401f90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:47.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.613+0000 7f023ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02401f8ce0 0x7f02401f90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34074/0 (socket says 172.21.15.5:34074) 2024-05-20T18:28:47.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.613+0000 7f023ffff640 1 -- 172.21.15.5:0/3893166894 learned_addr learned my addr 172.21.15.5:0/3893166894 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:47.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.613+0000 7f023f7fe640 1 --2- 172.21.15.5:0/3893166894 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02400cf250 0x7f0240158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:47.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.613+0000 7f0237fff640 1 --2- 172.21.15.5:0/3893166894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02400d3220 0x7f02400cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:47.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.613+0000 7f023ffff640 1 -- 172.21.15.5:0/3893166894 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02400cf250 msgr2=0x7f0240158f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:47.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.613+0000 7f023ffff640 1 --2- 172.21.15.5:0/3893166894 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02400cf250 0x7f0240158f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:47.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.613+0000 7f023ffff640 1 -- 172.21.15.5:0/3893166894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02400d3220 msgr2=0x7f02400cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:47.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.613+0000 7f023ffff640 1 --2- 172.21.15.5:0/3893166894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02400d3220 0x7f02400cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:47.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.613+0000 7f023ffff640 1 -- 172.21.15.5:0/3893166894 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02400c8b30 con 0x7f02401f8ce0 2024-05-20T18:28:47.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.614+0000 7f023ffff640 1 --2- 172.21.15.5:0/3893166894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02401f8ce0 0x7f02401f90f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0238009f40 tx=0x7f0238004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:47.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.614+0000 7f023cff9640 1 -- 172.21.15.5:0/3893166894 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0238036050 con 0x7f02401f8ce0 2024-05-20T18:28:47.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.614+0000 7f023cff9640 1 -- 172.21.15.5:0/3893166894 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0238042070 con 0x7f02401f8ce0 2024-05-20T18:28:47.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.614+0000 7f0245d77640 1 -- 172.21.15.5:0/3893166894 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02400c8e10 con 0x7f02401f8ce0 2024-05-20T18:28:47.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.614+0000 7f023cff9640 1 -- 172.21.15.5:0/3893166894 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0238009510 con 0x7f02401f8ce0 2024-05-20T18:28:47.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.614+0000 7f0245d77640 1 -- 172.21.15.5:0/3893166894 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02401621d0 con 0x7f02401f8ce0 2024-05-20T18:28:47.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.615+0000 7f0245d77640 1 -- 172.21.15.5:0/3893166894 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0240160780 con 0x7f02401f8ce0 2024-05-20T18:28:47.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.615+0000 7f023cff9640 1 -- 172.21.15.5:0/3893166894 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0238007bf0 con 0x7f02401f8ce0 2024-05-20T18:28:47.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.616+0000 7f023cff9640 1 --2- 172.21.15.5:0/3893166894 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f022006be30 0x7f022006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:47.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.616+0000 7f023f7fe640 1 --2- 172.21.15.5:0/3893166894 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f022006be30 0x7f022006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:47.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.616+0000 7f023cff9640 1 -- 172.21.15.5:0/3893166894 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f02380abc80 con 0x7f02401f8ce0 2024-05-20T18:28:47.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.617+0000 7f023f7fe640 1 --2- 172.21.15.5:0/3893166894 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f022006be30 0x7f022006e340 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f02300016e0 tx=0x7f0230009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:47.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.621+0000 7f023cff9640 1 -- 172.21.15.5:0/3893166894 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0238079dc0 con 0x7f02401f8ce0 2024-05-20T18:28:47.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.805+0000 7f0245d77640 1 -- 172.21.15.5:0/3893166894 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0240075b00 con 0x7f02401f8ce0 2024-05-20T18:28:47.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.806+0000 7f023cff9640 1 -- 172.21.15.5:0/3893166894 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) ==== 93+0+10 (secure 0 0 0) 0x7f023807e0c0 con 0x7f02401f8ce0 2024-05-20T18:28:47.806 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:28:47.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.808+0000 7f0245d77640 1 -- 172.21.15.5:0/3893166894 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f022006be30 msgr2=0x7f022006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:47.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.808+0000 7f0245d77640 1 --2- 172.21.15.5:0/3893166894 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f022006be30 0x7f022006e340 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f02300016e0 tx=0x7f0230009040 comp rx=0 tx=0).stop 2024-05-20T18:28:47.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.808+0000 7f0245d77640 1 -- 172.21.15.5:0/3893166894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02401f8ce0 msgr2=0x7f02401f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:47.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.808+0000 7f0245d77640 1 --2- 172.21.15.5:0/3893166894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02401f8ce0 0x7f02401f90f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0238009f40 tx=0x7f0238004a40 comp rx=0 tx=0).stop 2024-05-20T18:28:47.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.808+0000 7f0245d77640 1 -- 172.21.15.5:0/3893166894 shutdown_connections 2024-05-20T18:28:47.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.808+0000 7f0245d77640 1 --2- 172.21.15.5:0/3893166894 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f022006be30 0x7f022006e340 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:47.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.808+0000 7f0245d77640 1 --2- 172.21.15.5:0/3893166894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02401f8ce0 0x7f02401f90f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:47.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.808+0000 7f0245d77640 1 --2- 172.21.15.5:0/3893166894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02400d3220 0x7f02400cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:47.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.808+0000 7f0245d77640 1 --2- 172.21.15.5:0/3893166894 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02400cf250 0x7f0240158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:47.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.808+0000 7f0245d77640 1 -- 172.21.15.5:0/3893166894 >> 172.21.15.5:0/3893166894 conn(0x7f02400d1860 msgr2=0x7f02400d3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:47.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.808+0000 7f0245d77640 1 -- 172.21.15.5:0/3893166894 shutdown_connections 2024-05-20T18:28:47.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.808+0000 7f0245d77640 1 -- 172.21.15.5:0/3893166894 wait complete. 2024-05-20T18:28:47.819 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:28:47.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.938+0000 7f82fe757640 1 Processor -- start 2024-05-20T18:28:47.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.938+0000 7f82fe757640 1 -- start start 2024-05-20T18:28:47.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.939+0000 7f82fe757640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f82f80b1e10 0x7f82f815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:47.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.939+0000 7f82fe757640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82f8070850 0x7f82f8070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:47.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.939+0000 7f82fe757640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82f80cd580 con 0x7f82f815d530 2024-05-20T18:28:47.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.939+0000 7f82fe757640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f82f80ce190 con 0x7f82f80b1e10 2024-05-20T18:28:47.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.939+0000 7f82fe757640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f82f80cd860 con 0x7f82f8070850 2024-05-20T18:28:47.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.939+0000 7f82f77fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f82f80b1e10 0x7f82f815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:47.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.939+0000 7f82f77fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f82f80b1e10 0x7f82f815aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34076/0 (socket says 172.21.15.5:34076) 2024-05-20T18:28:47.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.940+0000 7f82f77fe640 1 -- 172.21.15.5:0/1602402935 learned_addr learned my addr 172.21.15.5:0/1602402935 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:47.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.940+0000 7f82fccf4640 1 --2- 172.21.15.5:0/1602402935 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82f8070850 0x7f82f8070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:47.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.940+0000 7f82f77fe640 1 -- 172.21.15.5:0/1602402935 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82f8070850 msgr2=0x7f82f8070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:47.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.940+0000 7f82f77fe640 1 --2- 172.21.15.5:0/1602402935 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82f8070850 0x7f82f8070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:47.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.940+0000 7f82f77fe640 1 -- 172.21.15.5:0/1602402935 >> v1:172.21.15.63:6789/0 conn(0x7f82f815d530 legacy=0x7f82f80b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:47.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.940+0000 7f82f77fe640 1 -- 172.21.15.5:0/1602402935 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f80cdb40 con 0x7f82f80b1e10 2024-05-20T18:28:47.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.940+0000 7f82fccf4640 1 --2- 172.21.15.5:0/1602402935 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82f8070850 0x7f82f8070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:47.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.941+0000 7f82f77fe640 1 --2- 172.21.15.5:0/1602402935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f80b1e10 0x7f82f815aaa0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009870 tx=0x7f82ec02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=500aac941966f77b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:47.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.941+0000 7f82f6ffd640 1 -- 172.21.15.5:0/1602402935 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec037070 con 0x7f82f80b1e10 2024-05-20T18:28:47.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.941+0000 7f82f6ffd640 1 -- 172.21.15.5:0/1602402935 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82ec032440 con 0x7f82f80b1e10 2024-05-20T18:28:47.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.941+0000 7f82f6ffd640 1 -- 172.21.15.5:0/1602402935 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec032810 con 0x7f82f80b1e10 2024-05-20T18:28:47.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.941+0000 7f82fe757640 1 -- 172.21.15.5:0/1602402935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f80b1e10 msgr2=0x7f82f815aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:47.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.941+0000 7f82fe757640 1 --2- 172.21.15.5:0/1602402935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f80b1e10 0x7f82f815aaa0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009870 tx=0x7f82ec02a210 comp rx=0 tx=0).stop 2024-05-20T18:28:47.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.941+0000 7f82fe757640 1 -- 172.21.15.5:0/1602402935 shutdown_connections 2024-05-20T18:28:47.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.941+0000 7f82fe757640 1 --2- 172.21.15.5:0/1602402935 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82f8070850 0x7f82f8070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:47.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.941+0000 7f82fe757640 1 --2- 172.21.15.5:0/1602402935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f80b1e10 0x7f82f815aaa0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:47.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.941+0000 7f82fe757640 1 -- 172.21.15.5:0/1602402935 >> 172.21.15.5:0/1602402935 conn(0x7f82f80d29d0 msgr2=0x7f82f80d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:47.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.941+0000 7f82fe757640 1 -- 172.21.15.5:0/1602402935 shutdown_connections 2024-05-20T18:28:47.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.942+0000 7f82fe757640 1 -- 172.21.15.5:0/1602402935 wait complete. 2024-05-20T18:28:47.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.943+0000 7f82fe757640 1 Processor -- start 2024-05-20T18:28:47.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.943+0000 7f82fe757640 1 -- start start 2024-05-20T18:28:47.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.945+0000 7f82fe757640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f8070850 0x7f82f81561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:47.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.945+0000 7f82fe757640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82f80b1e10 0x7f82f80c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:47.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.945+0000 7f82fe757640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82f815d530 0x7f82f80c6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:47.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.945+0000 7f82fe757640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f82f80711a0 con 0x7f82f8070850 2024-05-20T18:28:47.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.945+0000 7f82fe757640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f82f80cd230 con 0x7f82f80b1e10 2024-05-20T18:28:47.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.945+0000 7f82fe757640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f82f8071320 con 0x7f82f815d530 2024-05-20T18:28:47.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.945+0000 7f82f7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f8070850 0x7f82f81561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:47.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.945+0000 7f82f7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f8070850 0x7f82f81561a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34088/0 (socket says 172.21.15.5:34088) 2024-05-20T18:28:47.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.945+0000 7f82f7fff640 1 -- 172.21.15.5:0/1402010588 learned_addr learned my addr 172.21.15.5:0/1402010588 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:47.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.945+0000 7f82f77fe640 1 --2- 172.21.15.5:0/1402010588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82f80b1e10 0x7f82f80c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:47.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.945+0000 7f82fccf4640 1 --2- 172.21.15.5:0/1402010588 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82f815d530 0x7f82f80c6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:47.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.945+0000 7f82f7fff640 1 -- 172.21.15.5:0/1402010588 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82f815d530 msgr2=0x7f82f80c6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:47.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.945+0000 7f82f7fff640 1 --2- 172.21.15.5:0/1402010588 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82f815d530 0x7f82f80c6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:47.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.945+0000 7f82f7fff640 1 -- 172.21.15.5:0/1402010588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82f80b1e10 msgr2=0x7f82f80c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:47.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.945+0000 7f82f7fff640 1 --2- 172.21.15.5:0/1402010588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82f80b1e10 0x7f82f80c6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:47.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.945+0000 7f82f7fff640 1 -- 172.21.15.5:0/1402010588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f80ccda0 con 0x7f82f8070850 2024-05-20T18:28:47.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.945+0000 7f82f77fe640 1 --2- 172.21.15.5:0/1402010588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82f80b1e10 0x7f82f80c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:47.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.945+0000 7f82f7fff640 1 --2- 172.21.15.5:0/1402010588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f8070850 0x7f82f81561a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f82e800b3f0 tx=0x7f82e800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:47.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.946+0000 7f82f4ff9640 1 -- 172.21.15.5:0/1402010588 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e8015020 con 0x7f82f8070850 2024-05-20T18:28:47.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.946+0000 7f82f4ff9640 1 -- 172.21.15.5:0/1402010588 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82e8011830 con 0x7f82f8070850 2024-05-20T18:28:47.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.946+0000 7f82fe757640 1 -- 172.21.15.5:0/1402010588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f81566e0 con 0x7f82f8070850 2024-05-20T18:28:47.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.946+0000 7f82f4ff9640 1 -- 172.21.15.5:0/1402010588 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e8011c20 con 0x7f82f8070850 2024-05-20T18:28:47.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.946+0000 7f82fe757640 1 -- 172.21.15.5:0/1402010588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f80ccac0 con 0x7f82f8070850 2024-05-20T18:28:47.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.947+0000 7f82fe757640 1 -- 172.21.15.5:0/1402010588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f8075b00 con 0x7f82f8070850 2024-05-20T18:28:47.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.947+0000 7f82f4ff9640 1 -- 172.21.15.5:0/1402010588 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f82e8018020 con 0x7f82f8070850 2024-05-20T18:28:47.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.948+0000 7f82f4ff9640 1 --2- 172.21.15.5:0/1402010588 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f82e406be30 0x7f82e406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:47.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.948+0000 7f82f4ff9640 1 -- 172.21.15.5:0/1402010588 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f82e808aef0 con 0x7f82f8070850 2024-05-20T18:28:47.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.948+0000 7f82f77fe640 1 --2- 172.21.15.5:0/1402010588 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f82e406be30 0x7f82e406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:47.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.948+0000 7f82f77fe640 1 --2- 172.21.15.5:0/1402010588 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f82e406be30 0x7f82e406e340 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f82ec034000 tx=0x7f82ec02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:47.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:47.952+0000 7f82f4ff9640 1 -- 172.21.15.5:0/1402010588 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f82e801f020 con 0x7f82f8070850 2024-05-20T18:28:48.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.134+0000 7f82fe757640 1 -- 172.21.15.5:0/1402010588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f82f80c5bd0 con 0x7f82f8070850 2024-05-20T18:28:48.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.134+0000 7f82f4ff9640 1 -- 172.21.15.5:0/1402010588 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) ==== 104+0+11 (secure 0 0 0) 0x7f82f81566e0 con 0x7f82f8070850 2024-05-20T18:28:48.134 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:28:48.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.136+0000 7f82fe757640 1 -- 172.21.15.5:0/1402010588 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f82e406be30 msgr2=0x7f82e406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:48.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.136+0000 7f82fe757640 1 --2- 172.21.15.5:0/1402010588 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f82e406be30 0x7f82e406e340 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f82ec034000 tx=0x7f82ec02e000 comp rx=0 tx=0).stop 2024-05-20T18:28:48.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.136+0000 7f82fe757640 1 -- 172.21.15.5:0/1402010588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f8070850 msgr2=0x7f82f81561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:48.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.136+0000 7f82fe757640 1 --2- 172.21.15.5:0/1402010588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f8070850 0x7f82f81561a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f82e800b3f0 tx=0x7f82e800b8b0 comp rx=0 tx=0).stop 2024-05-20T18:28:48.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.136+0000 7f82fe757640 1 -- 172.21.15.5:0/1402010588 shutdown_connections 2024-05-20T18:28:48.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.136+0000 7f82fe757640 1 --2- 172.21.15.5:0/1402010588 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f82e406be30 0x7f82e406e340 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:48.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.136+0000 7f82fe757640 1 --2- 172.21.15.5:0/1402010588 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82f815d530 0x7f82f80c6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:48.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.136+0000 7f82fe757640 1 --2- 172.21.15.5:0/1402010588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82f80b1e10 0x7f82f80c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:48.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.136+0000 7f82fe757640 1 --2- 172.21.15.5:0/1402010588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f8070850 0x7f82f81561a0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:48.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.136+0000 7f82fe757640 1 -- 172.21.15.5:0/1402010588 >> 172.21.15.5:0/1402010588 conn(0x7f82f80d29d0 msgr2=0x7f82f815f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:48.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.136+0000 7f82fe757640 1 -- 172.21.15.5:0/1402010588 shutdown_connections 2024-05-20T18:28:48.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.136+0000 7f82fe757640 1 -- 172.21.15.5:0/1402010588 wait complete. 2024-05-20T18:28:48.147 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:28:48.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.267+0000 7f1ac4916640 1 Processor -- start 2024-05-20T18:28:48.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.267+0000 7f1ac4916640 1 -- start start 2024-05-20T18:28:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.268+0000 7f1ac4916640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1ac0071630 0x7f1ac015f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.268+0000 7f1ac4916640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ac00ce2b0 con 0x7f1ac0070850 2024-05-20T18:28:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.268+0000 7f1ac4916640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ac006cce0 con 0x7f1ac00cd1b0 2024-05-20T18:28:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.268+0000 7f1ac4916640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1ac015f790 con 0x7f1ac0071630 2024-05-20T18:28:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.268+0000 7f1abed9d640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1ac0070850 0x7f1ac0070c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:41260/0 (socket says 172.21.15.5:41260) 2024-05-20T18:28:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.268+0000 7f1abed9d640 1 -- 172.21.15.5:0/3291925063 learned_addr learned my addr 172.21.15.5:0/3291925063 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.269+0000 7f1abe59c640 1 --2- 172.21.15.5:0/3291925063 >> v2:172.21.15.63:3300/0 conn(0x7f1ac0071630 0x7f1ac015f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.269+0000 7f1abdd9b640 1 -- 172.21.15.5:0/3291925063 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1534879895 0 0) 0x7f1ac00ce2b0 con 0x7f1ac0070850 2024-05-20T18:28:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.269+0000 7f1abdd9b640 1 -- 172.21.15.5:0/3291925063 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1aac003520 con 0x7f1ac0070850 2024-05-20T18:28:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.269+0000 7f1abdd9b640 1 -- 172.21.15.5:0/3291925063 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2248690268 0 0) 0x7f1aac003520 con 0x7f1ac0070850 2024-05-20T18:28:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.269+0000 7f1abdd9b640 1 -- 172.21.15.5:0/3291925063 >> v2:172.21.15.63:3300/0 conn(0x7f1ac0071630 msgr2=0x7f1ac015f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.269+0000 7f1abdd9b640 1 --2- 172.21.15.5:0/3291925063 >> v2:172.21.15.63:3300/0 conn(0x7f1ac0071630 0x7f1ac015f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.269+0000 7f1abdd9b640 1 -- 172.21.15.5:0/3291925063 >> v1:172.21.15.63:6789/0 conn(0x7f1ac00cd1b0 legacy=0x7f1ac006d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.269+0000 7f1abdd9b640 1 -- 172.21.15.5:0/3291925063 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac006cfc0 con 0x7f1ac0070850 2024-05-20T18:28:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.269+0000 7f1abdd9b640 1 -- 172.21.15.5:0/3291925063 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f1ab80019f0 con 0x7f1ac0070850 2024-05-20T18:28:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.269+0000 7f1abdd9b640 1 -- 172.21.15.5:0/3291925063 >> v1:172.21.15.5:6789/0 conn(0x7f1ac0070850 legacy=0x7f1ac0070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.269+0000 7f1abdd9b640 1 --2- 172.21.15.5:0/3291925063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1aac0039b0 0x7f1aac003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.269+0000 7f1abdd9b640 1 --2- 172.21.15.5:0/3291925063 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1aac004800 0x7f1aac004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.269+0000 7f1abdd9b640 1 --2- 172.21.15.5:0/3291925063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1aac005250 0x7f1aac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.269+0000 7f1abdd9b640 1 -- 172.21.15.5:0/3291925063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac015f790 con 0x7f1aac0039b0 2024-05-20T18:28:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.269+0000 7f1abdd9b640 1 -- 172.21.15.5:0/3291925063 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1aac00c1e0 con 0x7f1aac005250 2024-05-20T18:28:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.269+0000 7f1abdd9b640 1 -- 172.21.15.5:0/3291925063 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1aac00c3e0 con 0x7f1aac004800 2024-05-20T18:28:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.269+0000 7f1abe59c640 1 --2- 172.21.15.5:0/3291925063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1aac0039b0 0x7f1aac003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.269+0000 7f1abed9d640 1 --2- 172.21.15.5:0/3291925063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1aac005250 0x7f1aac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.269+0000 7f1ab5d9b640 1 --2- 172.21.15.5:0/3291925063 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1aac004800 0x7f1aac004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.269+0000 7f1abe59c640 1 -- 172.21.15.5:0/3291925063 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1aac004800 msgr2=0x7f1aac004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.269+0000 7f1abe59c640 1 --2- 172.21.15.5:0/3291925063 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1aac004800 0x7f1aac004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.270+0000 7f1abe59c640 1 -- 172.21.15.5:0/3291925063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1aac005250 msgr2=0x7f1aac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.270+0000 7f1abe59c640 1 --2- 172.21.15.5:0/3291925063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1aac005250 0x7f1aac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.270+0000 7f1abed9d640 1 -- 172.21.15.5:0/3291925063 reap_dead start 2024-05-20T18:28:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.270+0000 7f1abe59c640 1 -- 172.21.15.5:0/3291925063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aac003520 con 0x7f1aac0039b0 2024-05-20T18:28:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.270+0000 7f1ab5d9b640 1 --2- 172.21.15.5:0/3291925063 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1aac004800 0x7f1aac004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.270+0000 7f1abe59c640 1 --2- 172.21.15.5:0/3291925063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1aac0039b0 0x7f1aac003e20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0009ac0 tx=0x7f1aac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82bde5ecf78f0f29 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.270+0000 7f1abdd9b640 1 -- 172.21.15.5:0/3291925063 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0036070 con 0x7f1aac0039b0 2024-05-20T18:28:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.270+0000 7f1abdd9b640 1 -- 172.21.15.5:0/3291925063 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab0031400 con 0x7f1aac0039b0 2024-05-20T18:28:48.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.270+0000 7f1abdd9b640 1 -- 172.21.15.5:0/3291925063 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0009690 con 0x7f1aac0039b0 2024-05-20T18:28:48.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.271+0000 7f1ac4916640 1 -- 172.21.15.5:0/3291925063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1aac0039b0 msgr2=0x7f1aac003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:48.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.271+0000 7f1ac4916640 1 --2- 172.21.15.5:0/3291925063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1aac0039b0 0x7f1aac003e20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0009ac0 tx=0x7f1aac005830 comp rx=0 tx=0).stop 2024-05-20T18:28:48.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.271+0000 7f1ac4916640 1 -- 172.21.15.5:0/3291925063 shutdown_connections 2024-05-20T18:28:48.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.271+0000 7f1ac4916640 1 --2- 172.21.15.5:0/3291925063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1aac0039b0 0x7f1aac003e20 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:48.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.271+0000 7f1ac4916640 1 -- 172.21.15.5:0/3291925063 >> 172.21.15.5:0/3291925063 conn(0x7f1ac00d29d0 msgr2=0x7f1ac00d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:48.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.271+0000 7f1ac4916640 1 -- 172.21.15.5:0/3291925063 shutdown_connections 2024-05-20T18:28:48.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.271+0000 7f1ac4916640 1 -- 172.21.15.5:0/3291925063 wait complete. 2024-05-20T18:28:48.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.273+0000 7f1ac4916640 1 Processor -- start 2024-05-20T18:28:48.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.273+0000 7f1ac4916640 1 -- start start 2024-05-20T18:28:48.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.275+0000 7f1ac4916640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1aac0039b0 0x7f1ac01f9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:48.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.275+0000 7f1ac4916640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ac015e0a0 0x7f1ac015f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:48.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.275+0000 7f1ac4916640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ac015fe50 0x7f1ac01565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:48.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.275+0000 7f1ac4916640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac006c990 con 0x7f1aac0039b0 2024-05-20T18:28:48.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.275+0000 7f1ac4916640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac015f250 con 0x7f1ac015e0a0 2024-05-20T18:28:48.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.275+0000 7f1ac4916640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1ac01f9ea0 con 0x7f1ac015fe50 2024-05-20T18:28:48.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.275+0000 7f1abe59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1aac0039b0 0x7f1ac01f9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:48.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.275+0000 7f1abe59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1aac0039b0 0x7f1ac01f9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34108/0 (socket says 172.21.15.5:34108) 2024-05-20T18:28:48.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.275+0000 7f1abe59c640 1 -- 172.21.15.5:0/1295370929 learned_addr learned my addr 172.21.15.5:0/1295370929 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:48.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.275+0000 7f1ab5d9b640 1 --2- 172.21.15.5:0/1295370929 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ac015e0a0 0x7f1ac015f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:48.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.275+0000 7f1abed9d640 1 --2- 172.21.15.5:0/1295370929 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ac015fe50 0x7f1ac01565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:48.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.275+0000 7f1abe59c640 1 -- 172.21.15.5:0/1295370929 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ac015fe50 msgr2=0x7f1ac01565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:48.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.275+0000 7f1abe59c640 1 --2- 172.21.15.5:0/1295370929 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ac015fe50 0x7f1ac01565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:48.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.275+0000 7f1abe59c640 1 -- 172.21.15.5:0/1295370929 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ac015e0a0 msgr2=0x7f1ac015f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:48.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.275+0000 7f1abe59c640 1 --2- 172.21.15.5:0/1295370929 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ac015e0a0 0x7f1ac015f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:48.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.275+0000 7f1abe59c640 1 -- 172.21.15.5:0/1295370929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac015d450 con 0x7f1aac0039b0 2024-05-20T18:28:48.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.275+0000 7f1abe59c640 1 --2- 172.21.15.5:0/1295370929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1aac0039b0 0x7f1ac01f9960 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0034040 tx=0x7f1ab0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:48.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.275+0000 7f1ab77fe640 1 -- 172.21.15.5:0/1295370929 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab003f020 con 0x7f1aac0039b0 2024-05-20T18:28:48.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.275+0000 7f1ab77fe640 1 -- 172.21.15.5:0/1295370929 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab0037040 con 0x7f1aac0039b0 2024-05-20T18:28:48.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.276+0000 7f1ac4916640 1 -- 172.21.15.5:0/1295370929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac015d730 con 0x7f1aac0039b0 2024-05-20T18:28:48.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.276+0000 7f1ab77fe640 1 -- 172.21.15.5:0/1295370929 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0036070 con 0x7f1aac0039b0 2024-05-20T18:28:48.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.276+0000 7f1ac4916640 1 -- 172.21.15.5:0/1295370929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac01f9440 con 0x7f1aac0039b0 2024-05-20T18:28:48.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.276+0000 7f1ac4916640 1 -- 172.21.15.5:0/1295370929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ac0075b00 con 0x7f1aac0039b0 2024-05-20T18:28:48.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.277+0000 7f1ab77fe640 1 -- 172.21.15.5:0/1295370929 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1ab0042020 con 0x7f1aac0039b0 2024-05-20T18:28:48.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.278+0000 7f1ab77fe640 1 --2- 172.21.15.5:0/1295370929 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1aa006be30 0x7f1aa006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:48.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.278+0000 7f1ab77fe640 1 -- 172.21.15.5:0/1295370929 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f1ab00ab1b0 con 0x7f1aac0039b0 2024-05-20T18:28:48.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.278+0000 7f1ab5d9b640 1 --2- 172.21.15.5:0/1295370929 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1aa006be30 0x7f1aa006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:48.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.278+0000 7f1ab5d9b640 1 --2- 172.21.15.5:0/1295370929 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1aa006be30 0x7f1aa006e340 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1ac015cc10 tx=0x7f1aa8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:48.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.282+0000 7f1ab77fe640 1 -- 172.21.15.5:0/1295370929 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1ab0049030 con 0x7f1aac0039b0 2024-05-20T18:28:48.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.464+0000 7f1ac4916640 1 -- 172.21.15.5:0/1295370929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1ac0071410 con 0x7f1aac0039b0 2024-05-20T18:28:48.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.464+0000 7f1ab77fe640 1 -- 172.21.15.5:0/1295370929 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) ==== 100+0+11 (secure 0 0 0) 0x7f1ab00799f0 con 0x7f1aac0039b0 2024-05-20T18:28:48.464 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:28:48.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.466+0000 7f1ac4916640 1 -- 172.21.15.5:0/1295370929 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1aa006be30 msgr2=0x7f1aa006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:48.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.466+0000 7f1ac4916640 1 --2- 172.21.15.5:0/1295370929 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1aa006be30 0x7f1aa006e340 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1ac015cc10 tx=0x7f1aa8009040 comp rx=0 tx=0).stop 2024-05-20T18:28:48.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.466+0000 7f1ac4916640 1 -- 172.21.15.5:0/1295370929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1aac0039b0 msgr2=0x7f1ac01f9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:48.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.466+0000 7f1ac4916640 1 --2- 172.21.15.5:0/1295370929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1aac0039b0 0x7f1ac01f9960 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0034040 tx=0x7f1ab0004730 comp rx=0 tx=0).stop 2024-05-20T18:28:48.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.466+0000 7f1ac4916640 1 -- 172.21.15.5:0/1295370929 shutdown_connections 2024-05-20T18:28:48.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.466+0000 7f1ac4916640 1 --2- 172.21.15.5:0/1295370929 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1aa006be30 0x7f1aa006e340 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:48.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.466+0000 7f1ac4916640 1 --2- 172.21.15.5:0/1295370929 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ac015fe50 0x7f1ac01565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:48.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.466+0000 7f1ac4916640 1 --2- 172.21.15.5:0/1295370929 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ac015e0a0 0x7f1ac015f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:48.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.466+0000 7f1ac4916640 1 --2- 172.21.15.5:0/1295370929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1aac0039b0 0x7f1ac01f9960 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:48.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.466+0000 7f1ac4916640 1 -- 172.21.15.5:0/1295370929 >> 172.21.15.5:0/1295370929 conn(0x7f1ac00d29d0 msgr2=0x7f1ac006d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:48.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.466+0000 7f1ac4916640 1 -- 172.21.15.5:0/1295370929 shutdown_connections 2024-05-20T18:28:48.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.466+0000 7f1ac4916640 1 -- 172.21.15.5:0/1295370929 wait complete. 2024-05-20T18:28:48.477 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:28:48.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.593+0000 7fbb37661640 1 Processor -- start 2024-05-20T18:28:48.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.593+0000 7fbb37661640 1 -- start start 2024-05-20T18:28:48.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.594+0000 7fbb37661640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb300cfae0 0x7fbb3016a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:48.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.594+0000 7fbb37661640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb3006d0f0 con 0x7fbb300cd680 2024-05-20T18:28:48.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.594+0000 7fbb37661640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb3006d3d0 con 0x7fbb3015f220 2024-05-20T18:28:48.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.594+0000 7fbb37661640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbb300cda90 con 0x7fbb300cfae0 2024-05-20T18:28:48.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.595+0000 7fbb353fd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbb300cd680 0x7fbb3006c6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:41266/0 (socket says 172.21.15.5:41266) 2024-05-20T18:28:48.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.595+0000 7fbb353fd640 1 -- 172.21.15.5:0/203981151 learned_addr learned my addr 172.21.15.5:0/203981151 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:48.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.595+0000 7fbb35bfe640 1 --2- 172.21.15.5:0/203981151 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb300cfae0 0x7fbb3016a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:48.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.595+0000 7fbb27fff640 1 -- 172.21.15.5:0/203981151 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 946039229 0 0) 0x7fbb3006d0f0 con 0x7fbb300cd680 2024-05-20T18:28:48.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.595+0000 7fbb27fff640 1 -- 172.21.15.5:0/203981151 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb1c003520 con 0x7fbb300cd680 2024-05-20T18:28:48.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.595+0000 7fbb27fff640 1 -- 172.21.15.5:0/203981151 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2130843169 0 0) 0x7fbb3006d3d0 con 0x7fbb3015f220 2024-05-20T18:28:48.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.595+0000 7fbb27fff640 1 -- 172.21.15.5:0/203981151 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb3006d0f0 con 0x7fbb3015f220 2024-05-20T18:28:48.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.595+0000 7fbb35bfe640 1 -- 172.21.15.5:0/203981151 >> v1:172.21.15.63:6789/0 conn(0x7fbb3015f220 legacy=0x7fbb300cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:48.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.595+0000 7fbb27fff640 1 -- 172.21.15.5:0/203981151 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2351594446 0 0) 0x7fbb1c003520 con 0x7fbb300cd680 2024-05-20T18:28:48.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.595+0000 7fbb35bfe640 1 -- 172.21.15.5:0/203981151 >> v1:172.21.15.5:6789/0 conn(0x7fbb300cd680 legacy=0x7fbb3006c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:48.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.595+0000 7fbb35bfe640 1 -- 172.21.15.5:0/203981151 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3006d6b0 con 0x7fbb300cfae0 2024-05-20T18:28:48.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.596+0000 7fbb35bfe640 1 --2- 172.21.15.5:0/203981151 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb300cfae0 0x7fbb3016a210 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbb28009dc0 tx=0x7fbb2802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dec5ef69aebde7a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:48.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.596+0000 7fbb27fff640 1 -- 172.21.15.5:0/203981151 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c001670 con 0x7fbb300cfae0 2024-05-20T18:28:48.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.596+0000 7fbb27fff640 1 -- 172.21.15.5:0/203981151 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb28007540 con 0x7fbb300cfae0 2024-05-20T18:28:48.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.596+0000 7fbb27fff640 1 -- 172.21.15.5:0/203981151 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb280078f0 con 0x7fbb300cfae0 2024-05-20T18:28:48.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.597+0000 7fbb37661640 1 -- 172.21.15.5:0/203981151 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb300cfae0 msgr2=0x7fbb3016a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:48.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.597+0000 7fbb37661640 1 --2- 172.21.15.5:0/203981151 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb300cfae0 0x7fbb3016a210 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbb28009dc0 tx=0x7fbb2802a5d0 comp rx=0 tx=0).stop 2024-05-20T18:28:48.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.597+0000 7fbb37661640 1 -- 172.21.15.5:0/203981151 shutdown_connections 2024-05-20T18:28:48.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.597+0000 7fbb37661640 1 --2- 172.21.15.5:0/203981151 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb300cfae0 0x7fbb3016a210 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:48.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.597+0000 7fbb37661640 1 -- 172.21.15.5:0/203981151 >> 172.21.15.5:0/203981151 conn(0x7fbb300b44d0 msgr2=0x7fbb300b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:48.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.597+0000 7fbb37661640 1 -- 172.21.15.5:0/203981151 shutdown_connections 2024-05-20T18:28:48.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.597+0000 7fbb37661640 1 -- 172.21.15.5:0/203981151 wait complete. 2024-05-20T18:28:48.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.599+0000 7fbb37661640 1 Processor -- start 2024-05-20T18:28:48.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.599+0000 7fbb37661640 1 -- start start 2024-05-20T18:28:48.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.600+0000 7fbb37661640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb300cd680 0x7fbb301622c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:48.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.600+0000 7fbb37661640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb3015f220 0x7fbb301f9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:48.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.600+0000 7fbb37661640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb301514e0 0x7fbb30150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:48.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.600+0000 7fbb37661640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3006cdd0 con 0x7fbb300cd680 2024-05-20T18:28:48.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.600+0000 7fbb37661640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3006d9d0 con 0x7fbb3015f220 2024-05-20T18:28:48.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.600+0000 7fbb37661640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbb3015f630 con 0x7fbb301514e0 2024-05-20T18:28:48.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.600+0000 7fbb353fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb300cd680 0x7fbb301622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:48.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.600+0000 7fbb353fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb300cd680 0x7fbb301622c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34118/0 (socket says 172.21.15.5:34118) 2024-05-20T18:28:48.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.600+0000 7fbb353fd640 1 -- 172.21.15.5:0/24503184 learned_addr learned my addr 172.21.15.5:0/24503184 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:48.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.601+0000 7fbb35bfe640 1 --2- 172.21.15.5:0/24503184 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb301514e0 0x7fbb30150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:48.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.601+0000 7fbb34bfc640 1 --2- 172.21.15.5:0/24503184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb3015f220 0x7fbb301f9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:48.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.601+0000 7fbb353fd640 1 -- 172.21.15.5:0/24503184 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb301514e0 msgr2=0x7fbb30150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:48.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.601+0000 7fbb353fd640 1 --2- 172.21.15.5:0/24503184 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb301514e0 0x7fbb30150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:48.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.601+0000 7fbb353fd640 1 -- 172.21.15.5:0/24503184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb3015f220 msgr2=0x7fbb301f9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:48.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.601+0000 7fbb353fd640 1 --2- 172.21.15.5:0/24503184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb3015f220 0x7fbb301f9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:48.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.601+0000 7fbb353fd640 1 -- 172.21.15.5:0/24503184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb300b7fb0 con 0x7fbb300cd680 2024-05-20T18:28:48.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.601+0000 7fbb34bfc640 1 --2- 172.21.15.5:0/24503184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb3015f220 0x7fbb301f9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:48.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.601+0000 7fbb353fd640 1 --2- 172.21.15.5:0/24503184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb300cd680 0x7fbb301622c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c007fd0 tx=0x7fbb2c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:48.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.601+0000 7fbb25ffb640 1 -- 172.21.15.5:0/24503184 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c017070 con 0x7fbb300cd680 2024-05-20T18:28:48.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.601+0000 7fbb25ffb640 1 -- 172.21.15.5:0/24503184 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb2c005ce0 con 0x7fbb300cd680 2024-05-20T18:28:48.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.601+0000 7fbb37661640 1 -- 172.21.15.5:0/24503184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb300b8290 con 0x7fbb300cd680 2024-05-20T18:28:48.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.601+0000 7fbb25ffb640 1 -- 172.21.15.5:0/24503184 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c007290 con 0x7fbb300cd680 2024-05-20T18:28:48.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.601+0000 7fbb37661640 1 -- 172.21.15.5:0/24503184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb301f8e40 con 0x7fbb300cd680 2024-05-20T18:28:48.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.602+0000 7fbb37661640 1 -- 172.21.15.5:0/24503184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb30075b00 con 0x7fbb300cd680 2024-05-20T18:28:48.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.603+0000 7fbb25ffb640 1 -- 172.21.15.5:0/24503184 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbb2c01a070 con 0x7fbb300cd680 2024-05-20T18:28:48.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.603+0000 7fbb25ffb640 1 --2- 172.21.15.5:0/24503184 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbb1406be30 0x7fbb1406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:48.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.604+0000 7fbb25ffb640 1 -- 172.21.15.5:0/24503184 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fbb2c08a3e0 con 0x7fbb300cd680 2024-05-20T18:28:48.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.604+0000 7fbb34bfc640 1 --2- 172.21.15.5:0/24503184 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbb1406be30 0x7fbb1406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:48.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.604+0000 7fbb34bfc640 1 --2- 172.21.15.5:0/24503184 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbb1406be30 0x7fbb1406e340 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbb3015b410 tx=0x7fbb20009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:48.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.608+0000 7fbb25ffb640 1 -- 172.21.15.5:0/24503184 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbb2c0584a0 con 0x7fbb300cd680 2024-05-20T18:28:48.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.827+0000 7fbb37661640 1 -- 172.21.15.5:0/24503184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb300b8940 con 0x7fbb300cd680 2024-05-20T18:28:48.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.827+0000 7fbb25ffb640 1 -- 172.21.15.5:0/24503184 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5175 (secure 0 0 0) 0x7fbb2c05c7a0 con 0x7fbb300cd680 2024-05-20T18:28:48.827 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:48.827 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1458170078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1458170078},{"type":"v1","addr":"172.21.15.63:6834","nonce":1458170078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4916,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/2601750663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":2601750663},{"type":"v1","addr":"172.21.15.63:6835","nonce":2601750663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5313,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/1913030219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1913030219},{"type":"v1","addr":"172.21.15.5:6836","nonce":1913030219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:28:24.270282+0000","modified":"2024-05-20T18:28:27.296534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5316},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5316":{"gid":5316,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6837/2355805105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":2355805105},{"type":"v1","addr":"172.21.15.5:6837","nonce":2355805105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5316,"qdb_cluster":[5316]},"id":2}]} 2024-05-20T18:28:48.827 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 17 2024-05-20T18:28:48.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.830+0000 7fbb37661640 1 -- 172.21.15.5:0/24503184 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbb1406be30 msgr2=0x7fbb1406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.830+0000 7fbb37661640 1 --2- 172.21.15.5:0/24503184 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbb1406be30 0x7fbb1406e340 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbb3015b410 tx=0x7fbb20009040 comp rx=0 tx=0).stop 2024-05-20T18:28:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.830+0000 7fbb37661640 1 -- 172.21.15.5:0/24503184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb300cd680 msgr2=0x7fbb301622c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.830+0000 7fbb37661640 1 --2- 172.21.15.5:0/24503184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb300cd680 0x7fbb301622c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c007fd0 tx=0x7fbb2c00c9b0 comp rx=0 tx=0).stop 2024-05-20T18:28:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.830+0000 7fbb37661640 1 -- 172.21.15.5:0/24503184 shutdown_connections 2024-05-20T18:28:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.830+0000 7fbb37661640 1 --2- 172.21.15.5:0/24503184 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbb1406be30 0x7fbb1406e340 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.830+0000 7fbb37661640 1 --2- 172.21.15.5:0/24503184 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb301514e0 0x7fbb30150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.830+0000 7fbb37661640 1 --2- 172.21.15.5:0/24503184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb3015f220 0x7fbb301f9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.830+0000 7fbb37661640 1 --2- 172.21.15.5:0/24503184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb300cd680 0x7fbb301622c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.830+0000 7fbb37661640 1 -- 172.21.15.5:0/24503184 >> 172.21.15.5:0/24503184 conn(0x7fbb300b44d0 msgr2=0x7fbb300ce490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.830+0000 7fbb37661640 1 -- 172.21.15.5:0/24503184 shutdown_connections 2024-05-20T18:28:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.830+0000 7fbb37661640 1 -- 172.21.15.5:0/24503184 wait complete. 2024-05-20T18:28:48.841 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:28:48.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.959+0000 7fb0dcfe9640 1 Processor -- start 2024-05-20T18:28:48.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.959+0000 7fb0dcfe9640 1 -- start start 2024-05-20T18:28:48.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.960+0000 7fb0dcfe9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0d815ce20 0x7fb0d81607e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:48.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.960+0000 7fb0dcfe9640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb0d815f620 0x7fb0d815fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:48.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.960+0000 7fb0dcfe9640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0d8160270 con 0x7fb0d80d12a0 2024-05-20T18:28:48.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.960+0000 7fb0dcfe9640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb0d80cfb20 con 0x7fb0d815f620 2024-05-20T18:28:48.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.960+0000 7fb0dcfe9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb0d8160550 con 0x7fb0d815ce20 2024-05-20T18:28:48.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.960+0000 7fb0d77fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0d815ce20 0x7fb0d81607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:48.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.960+0000 7fb0d6ffd640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb0d815f620 0x7fb0d815fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:48.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.960+0000 7fb0d77fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0d815ce20 0x7fb0d81607e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:54542/0 (socket says 172.21.15.5:54542) 2024-05-20T18:28:48.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.961+0000 7fb0d77fe640 1 -- 172.21.15.5:0/25580694 learned_addr learned my addr 172.21.15.5:0/25580694 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:48.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.961+0000 7fb0d6ffd640 1 -- 172.21.15.5:0/25580694 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0d815ce20 msgr2=0x7fb0d81607e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:48.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.961+0000 7fb0d6ffd640 1 --2- 172.21.15.5:0/25580694 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0d815ce20 0x7fb0d81607e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:48.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.961+0000 7fb0d6ffd640 1 -- 172.21.15.5:0/25580694 >> v1:172.21.15.63:6789/0 conn(0x7fb0d80d12a0 legacy=0x7fb0d80d16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:48.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.961+0000 7fb0d6ffd640 1 -- 172.21.15.5:0/25580694 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d80d2d60 con 0x7fb0d815f620 2024-05-20T18:28:48.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.962+0000 7fb0d6ffd640 1 --2- 172.21.15.5:0/25580694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0d815f620 0x7fb0d815fa30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc0097a0 tx=0x7fb0cc02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa6855f2262669f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:48.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.962+0000 7fb0d67fc640 1 -- 172.21.15.5:0/25580694 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc037070 con 0x7fb0d815f620 2024-05-20T18:28:48.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.962+0000 7fb0d67fc640 1 -- 172.21.15.5:0/25580694 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0cc032440 con 0x7fb0d815f620 2024-05-20T18:28:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.962+0000 7fb0d67fc640 1 -- 172.21.15.5:0/25580694 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc032810 con 0x7fb0d815f620 2024-05-20T18:28:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.963+0000 7fb0dcfe9640 1 -- 172.21.15.5:0/25580694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0d815f620 msgr2=0x7fb0d815fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.963+0000 7fb0dcfe9640 1 --2- 172.21.15.5:0/25580694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0d815f620 0x7fb0d815fa30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc0097a0 tx=0x7fb0cc02a180 comp rx=0 tx=0).stop 2024-05-20T18:28:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.963+0000 7fb0dcfe9640 1 -- 172.21.15.5:0/25580694 shutdown_connections 2024-05-20T18:28:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.963+0000 7fb0dcfe9640 1 --2- 172.21.15.5:0/25580694 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0d815f620 0x7fb0d815fa30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.963+0000 7fb0dcfe9640 1 --2- 172.21.15.5:0/25580694 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0d815ce20 0x7fb0d81607e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.963+0000 7fb0dcfe9640 1 -- 172.21.15.5:0/25580694 >> 172.21.15.5:0/25580694 conn(0x7fb0d806dbb0 msgr2=0x7fb0d806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.963+0000 7fb0dcfe9640 1 -- 172.21.15.5:0/25580694 shutdown_connections 2024-05-20T18:28:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.963+0000 7fb0dcfe9640 1 -- 172.21.15.5:0/25580694 wait complete. 2024-05-20T18:28:48.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.964+0000 7fb0dcfe9640 1 Processor -- start 2024-05-20T18:28:48.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.965+0000 7fb0dcfe9640 1 -- start start 2024-05-20T18:28:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.966+0000 7fb0dcfe9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0d80d12a0 0x7fb0d8151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.966+0000 7fb0dcfe9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0d815f620 0x7fb0d80cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.966+0000 7fb0dcfe9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0d8151750 0x7fb0d80cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.966+0000 7fb0dcfe9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d80d3840 con 0x7fb0d815f620 2024-05-20T18:28:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.966+0000 7fb0dcfe9640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d815ff70 con 0x7fb0d8151750 2024-05-20T18:28:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.966+0000 7fb0dcfe9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb0d80d39c0 con 0x7fb0d80d12a0 2024-05-20T18:28:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.966+0000 7fb0d6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0d815f620 0x7fb0d80cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.966+0000 7fb0d6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0d815f620 0x7fb0d80cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34122/0 (socket says 172.21.15.5:34122) 2024-05-20T18:28:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.966+0000 7fb0d6ffd640 1 -- 172.21.15.5:0/4194825940 learned_addr learned my addr 172.21.15.5:0/4194825940 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.966+0000 7fb0d77fe640 1 --2- 172.21.15.5:0/4194825940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0d80d12a0 0x7fb0d8151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.966+0000 7fb0d7fff640 1 --2- 172.21.15.5:0/4194825940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0d8151750 0x7fb0d80cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.966+0000 7fb0d6ffd640 1 -- 172.21.15.5:0/4194825940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0d80d12a0 msgr2=0x7fb0d8151180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.966+0000 7fb0d6ffd640 1 --2- 172.21.15.5:0/4194825940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0d80d12a0 0x7fb0d8151180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.966+0000 7fb0d6ffd640 1 -- 172.21.15.5:0/4194825940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0d8151750 msgr2=0x7fb0d80cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.966+0000 7fb0d6ffd640 1 --2- 172.21.15.5:0/4194825940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0d8151750 0x7fb0d80cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.966+0000 7fb0d6ffd640 1 -- 172.21.15.5:0/4194825940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d80c8660 con 0x7fb0d815f620 2024-05-20T18:28:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.966+0000 7fb0d77fe640 1 --2- 172.21.15.5:0/4194825940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0d80d12a0 0x7fb0d8151180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.967+0000 7fb0d6ffd640 1 --2- 172.21.15.5:0/4194825940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0d815f620 0x7fb0d80cd050 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc0098c0 tx=0x7fb0cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.967+0000 7fb0bffff640 1 -- 172.21.15.5:0/4194825940 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc037040 con 0x7fb0d815f620 2024-05-20T18:28:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.967+0000 7fb0bffff640 1 -- 172.21.15.5:0/4194825940 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0cc030500 con 0x7fb0d815f620 2024-05-20T18:28:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.967+0000 7fb0dcfe9640 1 -- 172.21.15.5:0/4194825940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d80ce6f0 con 0x7fb0d815f620 2024-05-20T18:28:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.967+0000 7fb0bffff640 1 -- 172.21.15.5:0/4194825940 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc0308d0 con 0x7fb0d815f620 2024-05-20T18:28:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.967+0000 7fb0dcfe9640 1 -- 172.21.15.5:0/4194825940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d8156640 con 0x7fb0d815f620 2024-05-20T18:28:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.968+0000 7fb0dcfe9640 1 -- 172.21.15.5:0/4194825940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0d8075980 con 0x7fb0d815f620 2024-05-20T18:28:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.968+0000 7fb0bffff640 1 -- 172.21.15.5:0/4194825940 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb0cc044020 con 0x7fb0d815f620 2024-05-20T18:28:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.969+0000 7fb0bffff640 1 --2- 172.21.15.5:0/4194825940 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb0c406be60 0x7fb0c406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.969+0000 7fb0bffff640 1 -- 172.21.15.5:0/4194825940 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fb0cc0ab080 con 0x7fb0d815f620 2024-05-20T18:28:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.969+0000 7fb0d77fe640 1 --2- 172.21.15.5:0/4194825940 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb0c406be60 0x7fb0c406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:48.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.969+0000 7fb0d77fe640 1 --2- 172.21.15.5:0/4194825940 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb0c406be60 0x7fb0c406e370 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80d1f20 tx=0x7fb0c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:48.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:48.973+0000 7fb0bffff640 1 -- 172.21.15.5:0/4194825940 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb0cc04b020 con 0x7fb0d815f620 2024-05-20T18:28:49.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.158+0000 7fb0dcfe9640 1 -- 172.21.15.5:0/4194825940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb0d80ba9a0 con 0x7fb0d815f620 2024-05-20T18:28:49.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.159+0000 7fb0bffff640 1 -- 172.21.15.5:0/4194825940 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16822 (secure 0 0 0) 0x7fb0cc079930 con 0x7fb0d815f620 2024-05-20T18:28:49.159 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:49.159 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":40,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:28:25.902315+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:28:22.042995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-20T18:28:23.252728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:28:49.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.162+0000 7fb0dcfe9640 1 -- 172.21.15.5:0/4194825940 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb0c406be60 msgr2=0x7fb0c406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:49.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.162+0000 7fb0dcfe9640 1 --2- 172.21.15.5:0/4194825940 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb0c406be60 0x7fb0c406e370 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80d1f20 tx=0x7fb0c8008040 comp rx=0 tx=0).stop 2024-05-20T18:28:49.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.162+0000 7fb0dcfe9640 1 -- 172.21.15.5:0/4194825940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0d815f620 msgr2=0x7fb0d80cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:49.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.162+0000 7fb0dcfe9640 1 --2- 172.21.15.5:0/4194825940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0d815f620 0x7fb0d80cd050 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc0098c0 tx=0x7fb0cc0046c0 comp rx=0 tx=0).stop 2024-05-20T18:28:49.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.162+0000 7fb0dcfe9640 1 -- 172.21.15.5:0/4194825940 shutdown_connections 2024-05-20T18:28:49.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.162+0000 7fb0dcfe9640 1 --2- 172.21.15.5:0/4194825940 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb0c406be60 0x7fb0c406e370 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:49.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.162+0000 7fb0dcfe9640 1 --2- 172.21.15.5:0/4194825940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0d8151750 0x7fb0d80cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:49.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.162+0000 7fb0dcfe9640 1 --2- 172.21.15.5:0/4194825940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0d815f620 0x7fb0d80cd050 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:49.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.162+0000 7fb0dcfe9640 1 --2- 172.21.15.5:0/4194825940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0d80d12a0 0x7fb0d8151180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:49.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.162+0000 7fb0dcfe9640 1 -- 172.21.15.5:0/4194825940 >> 172.21.15.5:0/4194825940 conn(0x7fb0d806dbb0 msgr2=0x7fb0d80cfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:49.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.162+0000 7fb0dcfe9640 1 -- 172.21.15.5:0/4194825940 shutdown_connections 2024-05-20T18:28:49.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.162+0000 7fb0dcfe9640 1 -- 172.21.15.5:0/4194825940 wait complete. 2024-05-20T18:28:49.173 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-20T18:28:49.173 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:28:49.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.289+0000 7fe20133e640 1 Processor -- start 2024-05-20T18:28:49.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.289+0000 7fe20133e640 1 -- start start 2024-05-20T18:28:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.290+0000 7fe20133e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1fc06da80 0x7fe1fc1602f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.290+0000 7fe20133e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe1fc0b1c80 0x7fe1fc09a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.290+0000 7fe20133e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1fc0af9d0 con 0x7fe1fc0b2090 2024-05-20T18:28:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.290+0000 7fe20133e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe1fc159da0 con 0x7fe1fc0b1c80 2024-05-20T18:28:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.290+0000 7fe20133e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe1fc0af1c0 con 0x7fe1fc06da80 2024-05-20T18:28:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.290+0000 7fe1fa7fc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe1fc0b1c80 0x7fe1fc09a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.290+0000 7fe1fa7fc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe1fc0b1c80 0x7fe1fc09a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34126/0 (socket says 172.21.15.5:34126) 2024-05-20T18:28:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.290+0000 7fe1fa7fc640 1 -- 172.21.15.5:0/3454798004 learned_addr learned my addr 172.21.15.5:0/3454798004 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.290+0000 7fe1fb7fe640 1 --1- 172.21.15.5:0/3454798004 >> v1:172.21.15.5:6789/0 conn(0x7fe1fc0b2090 0x7fe1fc0d1c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:41276/0 (socket says 172.21.15.5:41276) 2024-05-20T18:28:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.291+0000 7fe1faffd640 1 --2- 172.21.15.5:0/3454798004 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1fc06da80 0x7fe1fc1602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.291+0000 7fe1fa7fc640 1 -- 172.21.15.5:0/3454798004 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1fc06da80 msgr2=0x7fe1fc1602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.291+0000 7fe1fa7fc640 1 --2- 172.21.15.5:0/3454798004 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1fc06da80 0x7fe1fc1602f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.291+0000 7fe1fa7fc640 1 -- 172.21.15.5:0/3454798004 >> v1:172.21.15.5:6789/0 conn(0x7fe1fc0b2090 legacy=0x7fe1fc0d1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.291+0000 7fe1fa7fc640 1 -- 172.21.15.5:0/3454798004 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1fc0af410 con 0x7fe1fc0b1c80 2024-05-20T18:28:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.291+0000 7fe1f9ffb640 1 -- 172.21.15.5:0/3454798004 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3173577342 0 0) 0x7fe1fc0af9d0 con 0x7fe1fc0b2090 2024-05-20T18:28:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.291+0000 7fe1fa7fc640 1 --2- 172.21.15.5:0/3454798004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1fc0b1c80 0x7fe1fc09a4f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4009850 tx=0x7fe1e402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aaafb3993fc5e4f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.292+0000 7fe1f9ffb640 1 -- 172.21.15.5:0/3454798004 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4004030 con 0x7fe1fc0b1c80 2024-05-20T18:28:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.292+0000 7fe1f9ffb640 1 -- 172.21.15.5:0/3454798004 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1e40041d0 con 0x7fe1fc0b1c80 2024-05-20T18:28:49.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.292+0000 7fe1f9ffb640 1 -- 172.21.15.5:0/3454798004 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4003610 con 0x7fe1fc0b1c80 2024-05-20T18:28:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.292+0000 7fe20133e640 1 -- 172.21.15.5:0/3454798004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1fc0b1c80 msgr2=0x7fe1fc09a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:49.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.292+0000 7fe20133e640 1 --2- 172.21.15.5:0/3454798004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1fc0b1c80 0x7fe1fc09a4f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4009850 tx=0x7fe1e402a1a0 comp rx=0 tx=0).stop 2024-05-20T18:28:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.292+0000 7fe20133e640 1 -- 172.21.15.5:0/3454798004 shutdown_connections 2024-05-20T18:28:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.292+0000 7fe20133e640 1 --2- 172.21.15.5:0/3454798004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1fc0b1c80 0x7fe1fc09a4f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.292+0000 7fe20133e640 1 --2- 172.21.15.5:0/3454798004 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1fc06da80 0x7fe1fc1602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.292+0000 7fe20133e640 1 -- 172.21.15.5:0/3454798004 >> 172.21.15.5:0/3454798004 conn(0x7fe1fc0b6670 msgr2=0x7fe1fc0b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.292+0000 7fe20133e640 1 -- 172.21.15.5:0/3454798004 shutdown_connections 2024-05-20T18:28:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.292+0000 7fe20133e640 1 -- 172.21.15.5:0/3454798004 wait complete. 2024-05-20T18:28:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.294+0000 7fe20133e640 1 Processor -- start 2024-05-20T18:28:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.294+0000 7fe20133e640 1 -- start start 2024-05-20T18:28:49.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.296+0000 7fe20133e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1fc06da80 0x7fe1fc09e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:49.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.296+0000 7fe20133e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1fc0b1c80 0x7fe1fc146390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:49.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.296+0000 7fe20133e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1fc0b2090 0x7fe1fc1e7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:49.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.296+0000 7fe20133e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1fc09b260 con 0x7fe1fc0b1c80 2024-05-20T18:28:49.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.296+0000 7fe20133e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe1fc0d17b0 con 0x7fe1fc06da80 2024-05-20T18:28:49.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.296+0000 7fe20133e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe1fc0d2350 con 0x7fe1fc0b2090 2024-05-20T18:28:49.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.296+0000 7fe1fa7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1fc0b1c80 0x7fe1fc146390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:49.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.296+0000 7fe1fa7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1fc0b1c80 0x7fe1fc146390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34138/0 (socket says 172.21.15.5:34138) 2024-05-20T18:28:49.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.296+0000 7fe1fa7fc640 1 -- 172.21.15.5:0/3477890902 learned_addr learned my addr 172.21.15.5:0/3477890902 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:49.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.296+0000 7fe1faffd640 1 --2- 172.21.15.5:0/3477890902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1fc06da80 0x7fe1fc09e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:49.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.296+0000 7fe1fb7fe640 1 --2- 172.21.15.5:0/3477890902 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1fc0b2090 0x7fe1fc1e7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:49.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.296+0000 7fe1fa7fc640 1 -- 172.21.15.5:0/3477890902 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1fc0b2090 msgr2=0x7fe1fc1e7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:49.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.296+0000 7fe1fa7fc640 1 --2- 172.21.15.5:0/3477890902 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1fc0b2090 0x7fe1fc1e7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:49.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.296+0000 7fe1fa7fc640 1 -- 172.21.15.5:0/3477890902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1fc06da80 msgr2=0x7fe1fc09e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:49.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.296+0000 7fe1fa7fc640 1 --2- 172.21.15.5:0/3477890902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1fc06da80 0x7fe1fc09e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:49.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.296+0000 7fe1fa7fc640 1 -- 172.21.15.5:0/3477890902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1fc09ea60 con 0x7fe1fc0b1c80 2024-05-20T18:28:49.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.296+0000 7fe1faffd640 1 --2- 172.21.15.5:0/3477890902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1fc06da80 0x7fe1fc09e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:49.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.296+0000 7fe1fb7fe640 1 --2- 172.21.15.5:0/3477890902 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1fc0b2090 0x7fe1fc1e7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:49.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.296+0000 7fe1fa7fc640 1 --2- 172.21.15.5:0/3477890902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1fc0b1c80 0x7fe1fc146390 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4034040 tx=0x7fe1e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:49.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.297+0000 7fe1f37fe640 1 -- 172.21.15.5:0/3477890902 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4041030 con 0x7fe1fc0b1c80 2024-05-20T18:28:49.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.297+0000 7fe1f37fe640 1 -- 172.21.15.5:0/3477890902 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1e4003c50 con 0x7fe1fc0b1c80 2024-05-20T18:28:49.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.297+0000 7fe20133e640 1 -- 172.21.15.5:0/3477890902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1fc09ed40 con 0x7fe1fc0b1c80 2024-05-20T18:28:49.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.297+0000 7fe1f37fe640 1 -- 172.21.15.5:0/3477890902 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4003e20 con 0x7fe1fc0b1c80 2024-05-20T18:28:49.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.297+0000 7fe20133e640 1 -- 172.21.15.5:0/3477890902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1fc09f300 con 0x7fe1fc0b1c80 2024-05-20T18:28:49.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.298+0000 7fe20133e640 1 -- 172.21.15.5:0/3477890902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1fc09e7d0 con 0x7fe1fc0b1c80 2024-05-20T18:28:49.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.298+0000 7fe1f37fe640 1 -- 172.21.15.5:0/3477890902 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe1e4044070 con 0x7fe1fc0b1c80 2024-05-20T18:28:49.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.299+0000 7fe1f37fe640 1 --2- 172.21.15.5:0/3477890902 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe1dc06be30 0x7fe1dc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:49.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.299+0000 7fe1f37fe640 1 -- 172.21.15.5:0/3477890902 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fe1e40abc70 con 0x7fe1fc0b1c80 2024-05-20T18:28:49.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.299+0000 7fe1faffd640 1 --2- 172.21.15.5:0/3477890902 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe1dc06be30 0x7fe1dc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:49.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.299+0000 7fe1faffd640 1 --2- 172.21.15.5:0/3477890902 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe1dc06be30 0x7fe1dc06e340 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc0b6ca0 tx=0x7fe1f4007420 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:49.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.304+0000 7fe1f37fe640 1 -- 172.21.15.5:0/3477890902 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe1e407a400 con 0x7fe1fc0b1c80 2024-05-20T18:28:49.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.526+0000 7fe20133e640 1 -- 172.21.15.5:0/3477890902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1fc0b5e40 con 0x7fe1fc0b1c80 2024-05-20T18:28:49.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.527+0000 7fe1f37fe640 1 -- 172.21.15.5:0/3477890902 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5175 (secure 0 0 0) 0x7fe1fc0b5e40 con 0x7fe1fc0b1c80 2024-05-20T18:28:49.527 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:49.527 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1458170078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1458170078},{"type":"v1","addr":"172.21.15.63:6834","nonce":1458170078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4916,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/2601750663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":2601750663},{"type":"v1","addr":"172.21.15.63:6835","nonce":2601750663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5313,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/1913030219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1913030219},{"type":"v1","addr":"172.21.15.5:6836","nonce":1913030219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:28:24.270282+0000","modified":"2024-05-20T18:28:27.296534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5316},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5316":{"gid":5316,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6837/2355805105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":2355805105},{"type":"v1","addr":"172.21.15.5:6837","nonce":2355805105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5316,"qdb_cluster":[5316]},"id":2}]} 2024-05-20T18:28:49.527 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 17 2024-05-20T18:28:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.529+0000 7fe20133e640 1 -- 172.21.15.5:0/3477890902 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe1dc06be30 msgr2=0x7fe1dc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.529+0000 7fe20133e640 1 --2- 172.21.15.5:0/3477890902 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe1dc06be30 0x7fe1dc06e340 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc0b6ca0 tx=0x7fe1f4007420 comp rx=0 tx=0).stop 2024-05-20T18:28:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.529+0000 7fe20133e640 1 -- 172.21.15.5:0/3477890902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1fc0b1c80 msgr2=0x7fe1fc146390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.529+0000 7fe20133e640 1 --2- 172.21.15.5:0/3477890902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1fc0b1c80 0x7fe1fc146390 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4034040 tx=0x7fe1e40046c0 comp rx=0 tx=0).stop 2024-05-20T18:28:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.529+0000 7fe20133e640 1 -- 172.21.15.5:0/3477890902 shutdown_connections 2024-05-20T18:28:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.529+0000 7fe20133e640 1 --2- 172.21.15.5:0/3477890902 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe1dc06be30 0x7fe1dc06e340 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.529+0000 7fe20133e640 1 --2- 172.21.15.5:0/3477890902 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1fc0b2090 0x7fe1fc1e7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.529+0000 7fe20133e640 1 --2- 172.21.15.5:0/3477890902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1fc0b1c80 0x7fe1fc146390 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.529+0000 7fe20133e640 1 --2- 172.21.15.5:0/3477890902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1fc06da80 0x7fe1fc09e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.529+0000 7fe20133e640 1 -- 172.21.15.5:0/3477890902 >> 172.21.15.5:0/3477890902 conn(0x7fe1fc0b6670 msgr2=0x7fe1fc0ae7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.529+0000 7fe20133e640 1 -- 172.21.15.5:0/3477890902 shutdown_connections 2024-05-20T18:28:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.529+0000 7fe20133e640 1 -- 172.21.15.5:0/3477890902 wait complete. 2024-05-20T18:28:49.540 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:28:49.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.655+0000 7fa327357640 1 Processor -- start 2024-05-20T18:28:49.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.656+0000 7fa327357640 1 -- start start 2024-05-20T18:28:49.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.657+0000 7fa327357640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa32015f700 0x7fa32015fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:49.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.657+0000 7fa327357640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa32006c180 0x7fa32006c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:49.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.657+0000 7fa327357640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa320160430 con 0x7fa3200af030 2024-05-20T18:28:49.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.657+0000 7fa327357640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fa32006bcf0 con 0x7fa32006c180 2024-05-20T18:28:49.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.657+0000 7fa327357640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa32006b6a0 con 0x7fa32015f700 2024-05-20T18:28:49.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.657+0000 7fa3250f3640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa3200af030 0x7fa3200af440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:41290/0 (socket says 172.21.15.5:41290) 2024-05-20T18:28:49.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.657+0000 7fa3250f3640 1 -- 172.21.15.5:0/3766994879 learned_addr learned my addr 172.21.15.5:0/3766994879 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:49.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.657+0000 7fa3258f4640 1 --2- 172.21.15.5:0/3766994879 >> v2:172.21.15.5:3300/0 conn(0x7fa32006c180 0x7fa32006c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:49.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.657+0000 7fa3248f2640 1 --2- 172.21.15.5:0/3766994879 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa32015f700 0x7fa32015fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:49.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.658+0000 7fa3258f4640 1 -- 172.21.15.5:0/3766994879 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa32015f700 msgr2=0x7fa32015fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:49.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.658+0000 7fa3258f4640 1 --2- 172.21.15.5:0/3766994879 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa32015f700 0x7fa32015fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:49.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.658+0000 7fa3258f4640 1 -- 172.21.15.5:0/3766994879 >> v1:172.21.15.5:6789/0 conn(0x7fa3200af030 legacy=0x7fa3200af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:49.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.658+0000 7fa3258f4640 1 -- 172.21.15.5:0/3766994879 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa32006b980 con 0x7fa32006c180 2024-05-20T18:28:49.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.658+0000 7fa317fff640 1 -- 172.21.15.5:0/3766994879 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3525530109 0 0) 0x7fa320160430 con 0x7fa3200af030 2024-05-20T18:28:49.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.658+0000 7fa3258f4640 1 --2- 172.21.15.5:0/3766994879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa32006c180 0x7fa32006c590 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa318009d90 tx=0x7fa31802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d64931ed6677f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:49.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.658+0000 7fa317fff640 1 -- 172.21.15.5:0/3766994879 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa318007740 con 0x7fa32006c180 2024-05-20T18:28:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.659+0000 7fa317fff640 1 -- 172.21.15.5:0/3766994879 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3180078e0 con 0x7fa32006c180 2024-05-20T18:28:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.659+0000 7fa317fff640 1 -- 172.21.15.5:0/3766994879 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa318007cb0 con 0x7fa32006c180 2024-05-20T18:28:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.659+0000 7fa327357640 1 -- 172.21.15.5:0/3766994879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa32006c180 msgr2=0x7fa32006c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.659+0000 7fa327357640 1 --2- 172.21.15.5:0/3766994879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa32006c180 0x7fa32006c590 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa318009d90 tx=0x7fa31802a660 comp rx=0 tx=0).stop 2024-05-20T18:28:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.659+0000 7fa327357640 1 -- 172.21.15.5:0/3766994879 shutdown_connections 2024-05-20T18:28:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.659+0000 7fa327357640 1 --2- 172.21.15.5:0/3766994879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa32006c180 0x7fa32006c590 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.659+0000 7fa327357640 1 --2- 172.21.15.5:0/3766994879 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa32015f700 0x7fa32015fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.659+0000 7fa327357640 1 -- 172.21.15.5:0/3766994879 >> 172.21.15.5:0/3766994879 conn(0x7fa3200b6620 msgr2=0x7fa3200b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.659+0000 7fa327357640 1 -- 172.21.15.5:0/3766994879 shutdown_connections 2024-05-20T18:28:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.659+0000 7fa327357640 1 -- 172.21.15.5:0/3766994879 wait complete. 2024-05-20T18:28:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.661+0000 7fa327357640 1 Processor -- start 2024-05-20T18:28:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.661+0000 7fa327357640 1 -- start start 2024-05-20T18:28:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.662+0000 7fa327357640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa32006c180 0x7fa320159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.662+0000 7fa327357640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3200af030 0x7fa320147370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.662+0000 7fa327357640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa32015f700 0x7fa320155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.662+0000 7fa327357640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa3201601d0 con 0x7fa3200af030 2024-05-20T18:28:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.662+0000 7fa327357640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa320160050 con 0x7fa32006c180 2024-05-20T18:28:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.662+0000 7fa327357640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa32006cad0 con 0x7fa32015f700 2024-05-20T18:28:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.662+0000 7fa3248f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3200af030 0x7fa320147370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.663+0000 7fa3248f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3200af030 0x7fa320147370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34170/0 (socket says 172.21.15.5:34170) 2024-05-20T18:28:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.663+0000 7fa3248f2640 1 -- 172.21.15.5:0/3075877250 learned_addr learned my addr 172.21.15.5:0/3075877250 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.663+0000 7fa3258f4640 1 --2- 172.21.15.5:0/3075877250 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa32015f700 0x7fa320155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.663+0000 7fa3250f3640 1 --2- 172.21.15.5:0/3075877250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa32006c180 0x7fa320159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.663+0000 7fa3248f2640 1 -- 172.21.15.5:0/3075877250 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa32015f700 msgr2=0x7fa320155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.663+0000 7fa3248f2640 1 --2- 172.21.15.5:0/3075877250 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa32015f700 0x7fa320155550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.663+0000 7fa3248f2640 1 -- 172.21.15.5:0/3075877250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa32006c180 msgr2=0x7fa320159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.663+0000 7fa3248f2640 1 --2- 172.21.15.5:0/3075877250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa32006c180 0x7fa320159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.663+0000 7fa3248f2640 1 -- 172.21.15.5:0/3075877250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa32015c410 con 0x7fa3200af030 2024-05-20T18:28:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.663+0000 7fa3250f3640 1 --2- 172.21.15.5:0/3075877250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa32006c180 0x7fa320159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.663+0000 7fa3248f2640 1 --2- 172.21.15.5:0/3075877250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3200af030 0x7fa320147370 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa31000d5f0 tx=0x7fa31000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.663+0000 7fa315ffb640 1 -- 172.21.15.5:0/3075877250 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3100125f0 con 0x7fa3200af030 2024-05-20T18:28:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.663+0000 7fa315ffb640 1 -- 172.21.15.5:0/3075877250 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa310019070 con 0x7fa3200af030 2024-05-20T18:28:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.663+0000 7fa327357640 1 -- 172.21.15.5:0/3075877250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa32015c6f0 con 0x7fa3200af030 2024-05-20T18:28:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.663+0000 7fa315ffb640 1 -- 172.21.15.5:0/3075877250 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3100129d0 con 0x7fa3200af030 2024-05-20T18:28:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.664+0000 7fa327357640 1 -- 172.21.15.5:0/3075877250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa320156c70 con 0x7fa3200af030 2024-05-20T18:28:49.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.665+0000 7fa327357640 1 -- 172.21.15.5:0/3075877250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa320075b00 con 0x7fa3200af030 2024-05-20T18:28:49.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.665+0000 7fa315ffb640 1 -- 172.21.15.5:0/3075877250 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa310012b70 con 0x7fa3200af030 2024-05-20T18:28:49.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.666+0000 7fa315ffb640 1 --2- 172.21.15.5:0/3075877250 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa30406be60 0x7fa30406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:49.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.666+0000 7fa315ffb640 1 -- 172.21.15.5:0/3075877250 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fa31008b350 con 0x7fa3200af030 2024-05-20T18:28:49.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.666+0000 7fa3250f3640 1 --2- 172.21.15.5:0/3075877250 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa30406be60 0x7fa30406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:49.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.667+0000 7fa3250f3640 1 --2- 172.21.15.5:0/3075877250 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa30406be60 0x7fa30406e370 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa31c000f60 tx=0x7fa31c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:49.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.671+0000 7fa315ffb640 1 -- 172.21.15.5:0/3075877250 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa310059b70 con 0x7fa3200af030 2024-05-20T18:28:49.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.891+0000 7fa327357640 1 -- 172.21.15.5:0/3075877250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3200d1df0 con 0x7fa3200af030 2024-05-20T18:28:49.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.892+0000 7fa315ffb640 1 -- 172.21.15.5:0/3075877250 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5175 (secure 0 0 0) 0x7fa310059470 con 0x7fa3200af030 2024-05-20T18:28:49.892 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:49.892 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1458170078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1458170078},{"type":"v1","addr":"172.21.15.63:6834","nonce":1458170078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4916,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/2601750663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":2601750663},{"type":"v1","addr":"172.21.15.63:6835","nonce":2601750663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5313,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/1913030219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1913030219},{"type":"v1","addr":"172.21.15.5:6836","nonce":1913030219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:28:24.270282+0000","modified":"2024-05-20T18:28:27.296534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5316},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5316":{"gid":5316,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6837/2355805105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":2355805105},{"type":"v1","addr":"172.21.15.5:6837","nonce":2355805105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5316,"qdb_cluster":[5316]},"id":2}]} 2024-05-20T18:28:49.892 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 17 2024-05-20T18:28:49.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.894+0000 7fa327357640 1 -- 172.21.15.5:0/3075877250 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa30406be60 msgr2=0x7fa30406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:49.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.894+0000 7fa327357640 1 --2- 172.21.15.5:0/3075877250 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa30406be60 0x7fa30406e370 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa31c000f60 tx=0x7fa31c009040 comp rx=0 tx=0).stop 2024-05-20T18:28:49.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.894+0000 7fa327357640 1 -- 172.21.15.5:0/3075877250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3200af030 msgr2=0x7fa320147370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:49.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.894+0000 7fa327357640 1 --2- 172.21.15.5:0/3075877250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3200af030 0x7fa320147370 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa31000d5f0 tx=0x7fa31000dab0 comp rx=0 tx=0).stop 2024-05-20T18:28:49.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.895+0000 7fa327357640 1 -- 172.21.15.5:0/3075877250 shutdown_connections 2024-05-20T18:28:49.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.895+0000 7fa327357640 1 --2- 172.21.15.5:0/3075877250 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa30406be60 0x7fa30406e370 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:49.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.895+0000 7fa327357640 1 --2- 172.21.15.5:0/3075877250 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa32015f700 0x7fa320155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:49.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.895+0000 7fa327357640 1 --2- 172.21.15.5:0/3075877250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3200af030 0x7fa320147370 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:49.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.895+0000 7fa327357640 1 --2- 172.21.15.5:0/3075877250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa32006c180 0x7fa320159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:49.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.895+0000 7fa327357640 1 -- 172.21.15.5:0/3075877250 >> 172.21.15.5:0/3075877250 conn(0x7fa3200b6620 msgr2=0x7fa32006fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:49.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.895+0000 7fa327357640 1 -- 172.21.15.5:0/3075877250 shutdown_connections 2024-05-20T18:28:49.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:49.895+0000 7fa327357640 1 -- 172.21.15.5:0/3075877250 wait complete. 2024-05-20T18:28:49.905 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:28:50.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.020+0000 7f391caab640 1 Processor -- start 2024-05-20T18:28:50.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.020+0000 7f391caab640 1 -- start start 2024-05-20T18:28:50.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.022+0000 7f391caab640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f391806d3d0 0x7f391806d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:50.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.022+0000 7f391caab640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f39180b8190 0x7f39180b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:50.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.022+0000 7f391caab640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f391806dd20 con 0x7f3918162820 2024-05-20T18:28:50.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.022+0000 7f391caab640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f39180d0630 con 0x7f391806d3d0 2024-05-20T18:28:50.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.022+0000 7f391caab640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f39180d39c0 con 0x7f39180b8190 2024-05-20T18:28:50.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.022+0000 7f3915d9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f391806d3d0 0x7f391806d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:50.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.022+0000 7f3915d9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f391806d3d0 0x7f391806d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34182/0 (socket says 172.21.15.5:34182) 2024-05-20T18:28:50.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.022+0000 7f3915d9b640 1 -- 172.21.15.5:0/1246244688 learned_addr learned my addr 172.21.15.5:0/1246244688 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:50.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.022+0000 7f3916d9d640 1 --2- 172.21.15.5:0/1246244688 >> v2:172.21.15.63:3300/0 conn(0x7f39180b8190 0x7f39180b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:50.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.022+0000 7f3915d9b640 1 -- 172.21.15.5:0/1246244688 >> v2:172.21.15.63:3300/0 conn(0x7f39180b8190 msgr2=0x7f39180b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:50.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.022+0000 7f3915d9b640 1 --2- 172.21.15.5:0/1246244688 >> v2:172.21.15.63:3300/0 conn(0x7f39180b8190 0x7f39180b85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:50.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.022+0000 7f3915d9b640 1 -- 172.21.15.5:0/1246244688 >> v1:172.21.15.63:6789/0 conn(0x7f3918162820 legacy=0x7f39180b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:50.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.022+0000 7f3915d9b640 1 -- 172.21.15.5:0/1246244688 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39180d3ca0 con 0x7f391806d3d0 2024-05-20T18:28:50.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.022+0000 7f3916d9d640 1 --2- 172.21.15.5:0/1246244688 >> v2:172.21.15.63:3300/0 conn(0x7f39180b8190 0x7f39180b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:50.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.023+0000 7f3915d9b640 1 --2- 172.21.15.5:0/1246244688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f391806d3d0 0x7f391806d7e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f391808e070 tx=0x7f390c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66aca20284cfcdc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:50.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.023+0000 7f391559a640 1 -- 172.21.15.5:0/1246244688 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c037070 con 0x7f391806d3d0 2024-05-20T18:28:50.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.023+0000 7f391559a640 1 -- 172.21.15.5:0/1246244688 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f390c032440 con 0x7f391806d3d0 2024-05-20T18:28:50.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.023+0000 7f391559a640 1 -- 172.21.15.5:0/1246244688 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c032810 con 0x7f391806d3d0 2024-05-20T18:28:50.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.024+0000 7f391caab640 1 -- 172.21.15.5:0/1246244688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f391806d3d0 msgr2=0x7f391806d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:50.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.024+0000 7f391caab640 1 --2- 172.21.15.5:0/1246244688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f391806d3d0 0x7f391806d7e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f391808e070 tx=0x7f390c02a230 comp rx=0 tx=0).stop 2024-05-20T18:28:50.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.024+0000 7f391caab640 1 -- 172.21.15.5:0/1246244688 shutdown_connections 2024-05-20T18:28:50.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.024+0000 7f391caab640 1 --2- 172.21.15.5:0/1246244688 >> v2:172.21.15.63:3300/0 conn(0x7f39180b8190 0x7f39180b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:50.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.024+0000 7f391caab640 1 --2- 172.21.15.5:0/1246244688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f391806d3d0 0x7f391806d7e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:50.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.024+0000 7f391caab640 1 -- 172.21.15.5:0/1246244688 >> 172.21.15.5:0/1246244688 conn(0x7f39180d4bd0 msgr2=0x7f39180d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:50.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.024+0000 7f391caab640 1 -- 172.21.15.5:0/1246244688 shutdown_connections 2024-05-20T18:28:50.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.024+0000 7f391caab640 1 -- 172.21.15.5:0/1246244688 wait complete. 2024-05-20T18:28:50.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.026+0000 7f391caab640 1 Processor -- start 2024-05-20T18:28:50.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.026+0000 7f391caab640 1 -- start start 2024-05-20T18:28:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.027+0000 7f391caab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f391806d3d0 0x7f39180cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.027+0000 7f391caab640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39180b8190 0x7f39180b6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.027+0000 7f391caab640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3918162820 0x7f3918150000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.027+0000 7f391caab640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39180d0300 con 0x7f391806d3d0 2024-05-20T18:28:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.027+0000 7f391caab640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f39180d4260 con 0x7f39180b8190 2024-05-20T18:28:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.027+0000 7f391caab640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f39180b4050 con 0x7f3918162820 2024-05-20T18:28:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.027+0000 7f391659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f391806d3d0 0x7f39180cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:50.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.027+0000 7f391659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f391806d3d0 0x7f39180cbc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34188/0 (socket says 172.21.15.5:34188) 2024-05-20T18:28:50.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.027+0000 7f391659c640 1 -- 172.21.15.5:0/3695500264 learned_addr learned my addr 172.21.15.5:0/3695500264 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:50.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.028+0000 7f3915d9b640 1 --2- 172.21.15.5:0/3695500264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39180b8190 0x7f39180b6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:50.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.028+0000 7f3916d9d640 1 --2- 172.21.15.5:0/3695500264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3918162820 0x7f3918150000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:50.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.028+0000 7f391659c640 1 -- 172.21.15.5:0/3695500264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3918162820 msgr2=0x7f3918150000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:50.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.028+0000 7f391659c640 1 --2- 172.21.15.5:0/3695500264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3918162820 0x7f3918150000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:50.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.028+0000 7f391659c640 1 -- 172.21.15.5:0/3695500264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39180b8190 msgr2=0x7f39180b6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:50.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.028+0000 7f391659c640 1 --2- 172.21.15.5:0/3695500264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39180b8190 0x7f39180b6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:50.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.028+0000 7f391659c640 1 -- 172.21.15.5:0/3695500264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39180cc320 con 0x7f391806d3d0 2024-05-20T18:28:50.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.028+0000 7f391659c640 1 --2- 172.21.15.5:0/3695500264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f391806d3d0 0x7f39180cbc80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f390800b3e0 tx=0x7f390800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:50.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.028+0000 7f3906ffd640 1 -- 172.21.15.5:0/3695500264 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3908015020 con 0x7f391806d3d0 2024-05-20T18:28:50.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.028+0000 7f3906ffd640 1 -- 172.21.15.5:0/3695500264 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3908011850 con 0x7f391806d3d0 2024-05-20T18:28:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.028+0000 7f391caab640 1 -- 172.21.15.5:0/3695500264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39180cc600 con 0x7f391806d3d0 2024-05-20T18:28:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.028+0000 7f3906ffd640 1 -- 172.21.15.5:0/3695500264 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3908011c40 con 0x7f391806d3d0 2024-05-20T18:28:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.028+0000 7f391caab640 1 -- 172.21.15.5:0/3695500264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39180ccbc0 con 0x7f391806d3d0 2024-05-20T18:28:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.029+0000 7f391caab640 1 -- 172.21.15.5:0/3695500264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f391815ee90 con 0x7f391806d3d0 2024-05-20T18:28:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.030+0000 7f3906ffd640 1 -- 172.21.15.5:0/3695500264 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3908018020 con 0x7f391806d3d0 2024-05-20T18:28:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.030+0000 7f3906ffd640 1 --2- 172.21.15.5:0/3695500264 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f38f806be30 0x7f38f806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.030+0000 7f3906ffd640 1 -- 172.21.15.5:0/3695500264 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f390808b0a0 con 0x7f391806d3d0 2024-05-20T18:28:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.030+0000 7f3915d9b640 1 --2- 172.21.15.5:0/3695500264 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f38f806be30 0x7f38f806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.031+0000 7f3915d9b640 1 --2- 172.21.15.5:0/3695500264 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f38f806be30 0x7f38f806e340 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f390c034000 tx=0x7f390c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:50.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.035+0000 7f3906ffd640 1 -- 172.21.15.5:0/3695500264 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f391815ee90 con 0x7f391806d3d0 2024-05-20T18:28:50.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.225+0000 7f391caab640 1 -- 172.21.15.5:0/3695500264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3918075b00 con 0x7f391806d3d0 2024-05-20T18:28:50.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.226+0000 7f3906ffd640 1 -- 172.21.15.5:0/3695500264 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16822 (secure 0 0 0) 0x7f390801f020 con 0x7f391806d3d0 2024-05-20T18:28:50.226 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:50.226 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":40,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:28:25.902315+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:28:22.042995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-20T18:28:23.252728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:28:50.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.228+0000 7f391caab640 1 -- 172.21.15.5:0/3695500264 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f38f806be30 msgr2=0x7f38f806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:50.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.228+0000 7f391caab640 1 --2- 172.21.15.5:0/3695500264 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f38f806be30 0x7f38f806e340 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f390c034000 tx=0x7f390c02e000 comp rx=0 tx=0).stop 2024-05-20T18:28:50.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.228+0000 7f391caab640 1 -- 172.21.15.5:0/3695500264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f391806d3d0 msgr2=0x7f39180cbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:50.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.228+0000 7f391caab640 1 --2- 172.21.15.5:0/3695500264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f391806d3d0 0x7f39180cbc80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f390800b3e0 tx=0x7f390800b8a0 comp rx=0 tx=0).stop 2024-05-20T18:28:50.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.228+0000 7f391caab640 1 -- 172.21.15.5:0/3695500264 shutdown_connections 2024-05-20T18:28:50.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.228+0000 7f391caab640 1 --2- 172.21.15.5:0/3695500264 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f38f806be30 0x7f38f806e340 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:50.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.228+0000 7f391caab640 1 --2- 172.21.15.5:0/3695500264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3918162820 0x7f3918150000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:50.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.228+0000 7f391caab640 1 --2- 172.21.15.5:0/3695500264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39180b8190 0x7f39180b6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:50.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.228+0000 7f391caab640 1 --2- 172.21.15.5:0/3695500264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f391806d3d0 0x7f39180cbc80 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:50.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.228+0000 7f391caab640 1 -- 172.21.15.5:0/3695500264 >> 172.21.15.5:0/3695500264 conn(0x7f39180d4bd0 msgr2=0x7f391815d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:50.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.228+0000 7f391caab640 1 -- 172.21.15.5:0/3695500264 shutdown_connections 2024-05-20T18:28:50.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.228+0000 7f391caab640 1 -- 172.21.15.5:0/3695500264 wait complete. 2024-05-20T18:28:50.242 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-20T18:28:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.358+0000 7fe67b39d640 1 Processor -- start 2024-05-20T18:28:50.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.358+0000 7fe67b39d640 1 -- start start 2024-05-20T18:28:50.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.359+0000 7fe67b39d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe6740cf860 0x7fe6740b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:50.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.359+0000 7fe67b39d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe674163930 0x7fe6740d3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:50.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.359+0000 7fe67b39d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6741605a0 con 0x7fe674163520 2024-05-20T18:28:50.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.359+0000 7fe67b39d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe67415ffe0 con 0x7fe6740cf860 2024-05-20T18:28:50.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.359+0000 7fe67b39d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe6741602c0 con 0x7fe674163930 2024-05-20T18:28:50.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.360+0000 7fe67a39b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe6740cf860 0x7fe6740b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:50.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.360+0000 7fe67a39b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe6740cf860 0x7fe6740b3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34190/0 (socket says 172.21.15.5:34190) 2024-05-20T18:28:50.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.360+0000 7fe679b9a640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe674163520 0x7fe6740d12a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:41306/0 (socket says 172.21.15.5:41306) 2024-05-20T18:28:50.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.360+0000 7fe67a39b640 1 -- 172.21.15.5:0/2798464987 learned_addr learned my addr 172.21.15.5:0/2798464987 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:50.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.360+0000 7fe67ab9c640 1 --2- 172.21.15.5:0/2798464987 >> v2:172.21.15.63:3300/0 conn(0x7fe674163930 0x7fe6740d3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:50.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.360+0000 7fe67a39b640 1 -- 172.21.15.5:0/2798464987 >> v2:172.21.15.63:3300/0 conn(0x7fe674163930 msgr2=0x7fe6740d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:50.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.360+0000 7fe67a39b640 1 --2- 172.21.15.5:0/2798464987 >> v2:172.21.15.63:3300/0 conn(0x7fe674163930 0x7fe6740d3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:50.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.360+0000 7fe67a39b640 1 -- 172.21.15.5:0/2798464987 >> v1:172.21.15.5:6789/0 conn(0x7fe674163520 legacy=0x7fe6740d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:50.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.360+0000 7fe679399640 1 -- 172.21.15.5:0/2798464987 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3951260246 0 0) 0x7fe6741605a0 con 0x7fe674163520 2024-05-20T18:28:50.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.360+0000 7fe67a39b640 1 -- 172.21.15.5:0/2798464987 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe674162d20 con 0x7fe6740cf860 2024-05-20T18:28:50.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.361+0000 7fe67a39b640 1 --2- 172.21.15.5:0/2798464987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6740cf860 0x7fe6740b3d70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe67406c3a0 tx=0x7fe67002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2dff58caf66945f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:50.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.361+0000 7fe679399640 1 -- 172.21.15.5:0/2798464987 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe670037070 con 0x7fe6740cf860 2024-05-20T18:28:50.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.361+0000 7fe679399640 1 -- 172.21.15.5:0/2798464987 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe670032440 con 0x7fe6740cf860 2024-05-20T18:28:50.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.361+0000 7fe679399640 1 -- 172.21.15.5:0/2798464987 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe670032810 con 0x7fe6740cf860 2024-05-20T18:28:50.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.362+0000 7fe67b39d640 1 -- 172.21.15.5:0/2798464987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6740cf860 msgr2=0x7fe6740b3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:50.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.362+0000 7fe67b39d640 1 --2- 172.21.15.5:0/2798464987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6740cf860 0x7fe6740b3d70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe67406c3a0 tx=0x7fe67002a1e0 comp rx=0 tx=0).stop 2024-05-20T18:28:50.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.362+0000 7fe67b39d640 1 -- 172.21.15.5:0/2798464987 shutdown_connections 2024-05-20T18:28:50.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.362+0000 7fe67b39d640 1 --2- 172.21.15.5:0/2798464987 >> v2:172.21.15.63:3300/0 conn(0x7fe674163930 0x7fe6740d3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:50.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.362+0000 7fe67b39d640 1 --2- 172.21.15.5:0/2798464987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6740cf860 0x7fe6740b3d70 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:50.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.362+0000 7fe67b39d640 1 -- 172.21.15.5:0/2798464987 >> 172.21.15.5:0/2798464987 conn(0x7fe67406dbb0 msgr2=0x7fe67406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:50.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.362+0000 7fe67b39d640 1 -- 172.21.15.5:0/2798464987 shutdown_connections 2024-05-20T18:28:50.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.362+0000 7fe67b39d640 1 -- 172.21.15.5:0/2798464987 wait complete. 2024-05-20T18:28:50.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.363+0000 7fe67b39d640 1 Processor -- start 2024-05-20T18:28:50.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.364+0000 7fe67b39d640 1 -- start start 2024-05-20T18:28:50.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.365+0000 7fe67b39d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6740cf860 0x7fe674157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:50.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.365+0000 7fe67b39d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe674163520 0x7fe6740cd030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:50.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.365+0000 7fe67b39d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe674163930 0x7fe6740cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:50.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.365+0000 7fe67b39d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6740d1b30 con 0x7fe6740cf860 2024-05-20T18:28:50.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.365+0000 7fe67b39d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe6740d19b0 con 0x7fe674163930 2024-05-20T18:28:50.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.365+0000 7fe67b39d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe6740d1cb0 con 0x7fe674163520 2024-05-20T18:28:50.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.365+0000 7fe67a39b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6740cf860 0x7fe674157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:50.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.365+0000 7fe67a39b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6740cf860 0x7fe674157ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34202/0 (socket says 172.21.15.5:34202) 2024-05-20T18:28:50.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.365+0000 7fe67a39b640 1 -- 172.21.15.5:0/1331488273 learned_addr learned my addr 172.21.15.5:0/1331488273 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:50.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.365+0000 7fe679b9a640 1 --2- 172.21.15.5:0/1331488273 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe674163520 0x7fe6740cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:50.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.365+0000 7fe67ab9c640 1 --2- 172.21.15.5:0/1331488273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe674163930 0x7fe6740cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:50.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.365+0000 7fe67a39b640 1 -- 172.21.15.5:0/1331488273 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe674163520 msgr2=0x7fe6740cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:50.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.365+0000 7fe67a39b640 1 --2- 172.21.15.5:0/1331488273 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe674163520 0x7fe6740cd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:50.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.365+0000 7fe67a39b640 1 -- 172.21.15.5:0/1331488273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe674163930 msgr2=0x7fe6740cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:50.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.365+0000 7fe67a39b640 1 --2- 172.21.15.5:0/1331488273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe674163930 0x7fe6740cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:50.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.365+0000 7fe67a39b640 1 -- 172.21.15.5:0/1331488273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe674158370 con 0x7fe6740cf860 2024-05-20T18:28:50.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.365+0000 7fe67ab9c640 1 --2- 172.21.15.5:0/1331488273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe674163930 0x7fe6740cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:50.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.365+0000 7fe67a39b640 1 --2- 172.21.15.5:0/1331488273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6740cf860 0x7fe674157ab0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe670000c00 tx=0x7fe6700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:50.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.366+0000 7fe66affd640 1 -- 172.21.15.5:0/1331488273 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe670041020 con 0x7fe6740cf860 2024-05-20T18:28:50.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.366+0000 7fe66affd640 1 -- 172.21.15.5:0/1331488273 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6700306c0 con 0x7fe6740cf860 2024-05-20T18:28:50.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.366+0000 7fe67b39d640 1 -- 172.21.15.5:0/1331488273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6740b59d0 con 0x7fe6740cf860 2024-05-20T18:28:50.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.366+0000 7fe66affd640 1 -- 172.21.15.5:0/1331488273 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe670037040 con 0x7fe6740cf860 2024-05-20T18:28:50.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.366+0000 7fe67b39d640 1 -- 172.21.15.5:0/1331488273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6740b5f90 con 0x7fe6740cf860 2024-05-20T18:28:50.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.367+0000 7fe67b39d640 1 -- 172.21.15.5:0/1331488273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe674075980 con 0x7fe6740cf860 2024-05-20T18:28:50.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.367+0000 7fe66affd640 1 -- 172.21.15.5:0/1331488273 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe670044020 con 0x7fe6740cf860 2024-05-20T18:28:50.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.368+0000 7fe66affd640 1 --2- 172.21.15.5:0/1331488273 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe65806be60 0x7fe65806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:50.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.368+0000 7fe66affd640 1 -- 172.21.15.5:0/1331488273 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fe6700ab870 con 0x7fe6740cf860 2024-05-20T18:28:50.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.368+0000 7fe679b9a640 1 --2- 172.21.15.5:0/1331488273 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe65806be60 0x7fe65806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:50.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.368+0000 7fe679b9a640 1 --2- 172.21.15.5:0/1331488273 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe65806be60 0x7fe65806e370 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe6741f8d70 tx=0x7fe664007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:50.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.372+0000 7fe66affd640 1 -- 172.21.15.5:0/1331488273 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe67004b030 con 0x7fe6740cf860 2024-05-20T18:28:50.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.591+0000 7fe67b39d640 1 -- 172.21.15.5:0/1331488273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe6740ba2f0 con 0x7fe6740cf860 2024-05-20T18:28:50.591 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-20T18:28:50.592+0000 7fd1cf53a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-20T18:28:50.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.593+0000 7fe66affd640 1 -- 172.21.15.5:0/1331488273 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7fe67007a000 con 0x7fe6740cf860 2024-05-20T18:28:50.593 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-20T18:28:50.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.597+0000 7fe67b39d640 1 -- 172.21.15.5:0/1331488273 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe65806be60 msgr2=0x7fe65806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:50.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.597+0000 7fe67b39d640 1 --2- 172.21.15.5:0/1331488273 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe65806be60 0x7fe65806e370 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe6741f8d70 tx=0x7fe664007de0 comp rx=0 tx=0).stop 2024-05-20T18:28:50.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.597+0000 7fe67b39d640 1 -- 172.21.15.5:0/1331488273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6740cf860 msgr2=0x7fe674157ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:50.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.597+0000 7fe67b39d640 1 --2- 172.21.15.5:0/1331488273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6740cf860 0x7fe674157ab0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe670000c00 tx=0x7fe6700046c0 comp rx=0 tx=0).stop 2024-05-20T18:28:50.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.597+0000 7fe67b39d640 1 -- 172.21.15.5:0/1331488273 shutdown_connections 2024-05-20T18:28:50.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.597+0000 7fe67b39d640 1 --2- 172.21.15.5:0/1331488273 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe65806be60 0x7fe65806e370 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:50.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.597+0000 7fe67b39d640 1 --2- 172.21.15.5:0/1331488273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe674163930 0x7fe6740cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:50.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.597+0000 7fe67b39d640 1 --2- 172.21.15.5:0/1331488273 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe674163520 0x7fe6740cd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:50.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.597+0000 7fe67b39d640 1 --2- 172.21.15.5:0/1331488273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6740cf860 0x7fe674157ab0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:50.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.597+0000 7fe67b39d640 1 -- 172.21.15.5:0/1331488273 >> 172.21.15.5:0/1331488273 conn(0x7fe67406dbb0 msgr2=0x7fe6740d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:50.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.597+0000 7fe67b39d640 1 -- 172.21.15.5:0/1331488273 shutdown_connections 2024-05-20T18:28:50.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.597+0000 7fe67b39d640 1 -- 172.21.15.5:0/1331488273 wait complete. 2024-05-20T18:28:50.600 INFO:tasks.ceph.mds.c.smithi005.stderr: -6582> 2024-05-20T18:28:28.868+0000 7f4335486640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T18:28:50.610 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-20T18:28:50.659 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-05-20T18:28:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.729+0000 7f7b5d451640 1 Processor -- start 2024-05-20T18:28:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.729+0000 7f7b5d451640 1 -- start start 2024-05-20T18:28:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.730+0000 7f7b5d451640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7b58162820 0x7f7b580b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.730+0000 7f7b5d451640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b5806dae0 0x7f7b580b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.730+0000 7f7b5d451640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b580d0630 con 0x7f7b580b3e80 2024-05-20T18:28:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.730+0000 7f7b5d451640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7b580d39c0 con 0x7f7b58162820 2024-05-20T18:28:50.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.730+0000 7f7b5d451640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7b580d3ca0 con 0x7f7b5806dae0 2024-05-20T18:28:50.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.730+0000 7f7b4e7fc640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7b580b3e80 0x7f7b5806d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:41324/0 (socket says 172.21.15.5:41324) 2024-05-20T18:28:50.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.730+0000 7f7b4e7fc640 1 -- 172.21.15.5:0/1352333725 learned_addr learned my addr 172.21.15.5:0/1352333725 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:50.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.730+0000 7f7b56ffd640 1 --2- 172.21.15.5:0/1352333725 >> v2:172.21.15.63:3300/0 conn(0x7f7b58162820 0x7f7b580b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:50.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.730+0000 7f7b577fe640 1 --2- 172.21.15.5:0/1352333725 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b5806dae0 0x7f7b580b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:50.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.731+0000 7f7b567fc640 1 -- 172.21.15.5:0/1352333725 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3184126608 0 0) 0x7f7b580d0630 con 0x7f7b580b3e80 2024-05-20T18:28:50.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.731+0000 7f7b567fc640 1 -- 172.21.15.5:0/1352333725 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b44003520 con 0x7f7b580b3e80 2024-05-20T18:28:50.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.731+0000 7f7b56ffd640 1 -- 172.21.15.5:0/1352333725 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b5806dae0 msgr2=0x7f7b580b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:50.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.731+0000 7f7b56ffd640 1 --2- 172.21.15.5:0/1352333725 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b5806dae0 0x7f7b580b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:50.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.731+0000 7f7b56ffd640 1 -- 172.21.15.5:0/1352333725 >> v1:172.21.15.5:6789/0 conn(0x7f7b580b3e80 legacy=0x7f7b5806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:50.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.731+0000 7f7b56ffd640 1 -- 172.21.15.5:0/1352333725 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b580d3f80 con 0x7f7b58162820 2024-05-20T18:28:50.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.732+0000 7f7b56ffd640 1 --2- 172.21.15.5:0/1352333725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b58162820 0x7f7b580b3940 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7b480097a0 tx=0x7f7b4802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6cf6c1a1c49e9ef server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:50.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.732+0000 7f7b567fc640 1 -- 172.21.15.5:0/1352333725 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b48037070 con 0x7f7b58162820 2024-05-20T18:28:50.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.732+0000 7f7b567fc640 1 -- 172.21.15.5:0/1352333725 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b48032440 con 0x7f7b58162820 2024-05-20T18:28:50.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.732+0000 7f7b567fc640 1 -- 172.21.15.5:0/1352333725 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b48032810 con 0x7f7b58162820 2024-05-20T18:28:50.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.732+0000 7f7b5d451640 1 -- 172.21.15.5:0/1352333725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b58162820 msgr2=0x7f7b580b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:50.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.732+0000 7f7b5d451640 1 --2- 172.21.15.5:0/1352333725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b58162820 0x7f7b580b3940 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7b480097a0 tx=0x7f7b4802a180 comp rx=0 tx=0).stop 2024-05-20T18:28:50.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.733+0000 7f7b5d451640 1 -- 172.21.15.5:0/1352333725 shutdown_connections 2024-05-20T18:28:50.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.733+0000 7f7b5d451640 1 --2- 172.21.15.5:0/1352333725 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b5806dae0 0x7f7b580b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:50.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.733+0000 7f7b5d451640 1 --2- 172.21.15.5:0/1352333725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b58162820 0x7f7b580b3940 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:50.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.733+0000 7f7b5d451640 1 -- 172.21.15.5:0/1352333725 >> 172.21.15.5:0/1352333725 conn(0x7f7b580d4bd0 msgr2=0x7f7b580d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:50.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.733+0000 7f7b5d451640 1 -- 172.21.15.5:0/1352333725 shutdown_connections 2024-05-20T18:28:50.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.733+0000 7f7b5d451640 1 -- 172.21.15.5:0/1352333725 wait complete. 2024-05-20T18:28:50.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.734+0000 7f7b5d451640 1 Processor -- start 2024-05-20T18:28:50.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.735+0000 7f7b5d451640 1 -- start start 2024-05-20T18:28:50.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.736+0000 7f7b5d451640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b580b3e80 0x7f7b581f8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:50.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.736+0000 7f7b5d451640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b58162820 0x7f7b580b6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:50.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.736+0000 7f7b5d451640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b580b9440 0x7f7b580b9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:50.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.736+0000 7f7b5d451640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7b580d4780 con 0x7f7b580b9440 2024-05-20T18:28:50.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.736+0000 7f7b5d451640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7b580d0300 con 0x7f7b580b3e80 2024-05-20T18:28:50.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.736+0000 7f7b5d451640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7b580d4900 con 0x7f7b58162820 2024-05-20T18:28:50.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.736+0000 7f7b577fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b580b9440 0x7f7b580b9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:50.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.736+0000 7f7b577fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b580b9440 0x7f7b580b9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34226/0 (socket says 172.21.15.5:34226) 2024-05-20T18:28:50.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.736+0000 7f7b577fe640 1 -- 172.21.15.5:0/891978702 learned_addr learned my addr 172.21.15.5:0/891978702 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:50.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.736+0000 7f7b4e7fc640 1 --2- 172.21.15.5:0/891978702 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b58162820 0x7f7b580b6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:50.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.736+0000 7f7b56ffd640 1 --2- 172.21.15.5:0/891978702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b580b3e80 0x7f7b581f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:50.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.736+0000 7f7b577fe640 1 -- 172.21.15.5:0/891978702 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b58162820 msgr2=0x7f7b580b6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:50.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.736+0000 7f7b577fe640 1 --2- 172.21.15.5:0/891978702 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b58162820 0x7f7b580b6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:50.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.736+0000 7f7b577fe640 1 -- 172.21.15.5:0/891978702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b580b3e80 msgr2=0x7f7b581f8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:50.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.736+0000 7f7b577fe640 1 --2- 172.21.15.5:0/891978702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b580b3e80 0x7f7b581f8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:50.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.736+0000 7f7b577fe640 1 -- 172.21.15.5:0/891978702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b580b9d90 con 0x7f7b580b9440 2024-05-20T18:28:50.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.737+0000 7f7b577fe640 1 --2- 172.21.15.5:0/891978702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b580b9440 0x7f7b580b9850 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000ef90 tx=0x7f7b5000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:50.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.737+0000 7f7b4ffff640 1 -- 172.21.15.5:0/891978702 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b50018020 con 0x7f7b580b9440 2024-05-20T18:28:50.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.737+0000 7f7b4ffff640 1 -- 172.21.15.5:0/891978702 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b50014480 con 0x7f7b580b9440 2024-05-20T18:28:50.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.737+0000 7f7b4ffff640 1 -- 172.21.15.5:0/891978702 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b50009e60 con 0x7f7b580b9440 2024-05-20T18:28:50.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.737+0000 7f7b5d451640 1 -- 172.21.15.5:0/891978702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b580c96d0 con 0x7f7b580b9440 2024-05-20T18:28:50.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.737+0000 7f7b5d451640 1 -- 172.21.15.5:0/891978702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b580c9c50 con 0x7f7b580b9440 2024-05-20T18:28:50.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.738+0000 7f7b5d451640 1 -- 172.21.15.5:0/891978702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b58068ad0 con 0x7f7b580b9440 2024-05-20T18:28:50.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.739+0000 7f7b4ffff640 1 -- 172.21.15.5:0/891978702 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7b50005d10 con 0x7f7b580b9440 2024-05-20T18:28:50.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.739+0000 7f7b4ffff640 1 --2- 172.21.15.5:0/891978702 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7b3806be30 0x7f7b3806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:50.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.739+0000 7f7b4ffff640 1 -- 172.21.15.5:0/891978702 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f7b5008bd80 con 0x7f7b580b9440 2024-05-20T18:28:50.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.739+0000 7f7b56ffd640 1 --2- 172.21.15.5:0/891978702 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7b3806be30 0x7f7b3806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:50.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.740+0000 7f7b56ffd640 1 --2- 172.21.15.5:0/891978702 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7b3806be30 0x7f7b3806e340 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7b48034000 tx=0x7f7b4802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:50.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.744+0000 7f7b4ffff640 1 -- 172.21.15.5:0/891978702 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7b58068ad0 con 0x7f7b580b9440 2024-05-20T18:28:50.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:50.968+0000 7f7b5d451640 1 -- 172.21.15.5:0/891978702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f7b580b7350 con 0x7f7b580b9440 2024-05-20T18:28:51.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.598+0000 7f7b4ffff640 1 -- 172.21.15.5:0/891978702 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f7b50022030 con 0x7f7b580b9440 2024-05-20T18:28:51.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.601+0000 7f7b5d451640 1 -- 172.21.15.5:0/891978702 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7b3806be30 msgr2=0x7f7b3806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:51.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.601+0000 7f7b5d451640 1 --2- 172.21.15.5:0/891978702 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7b3806be30 0x7f7b3806e340 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7b48034000 tx=0x7f7b4802e000 comp rx=0 tx=0).stop 2024-05-20T18:28:51.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.601+0000 7f7b5d451640 1 -- 172.21.15.5:0/891978702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b580b9440 msgr2=0x7f7b580b9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:51.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.601+0000 7f7b5d451640 1 --2- 172.21.15.5:0/891978702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b580b9440 0x7f7b580b9850 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000ef90 tx=0x7f7b5000c490 comp rx=0 tx=0).stop 2024-05-20T18:28:51.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.601+0000 7f7b5d451640 1 -- 172.21.15.5:0/891978702 shutdown_connections 2024-05-20T18:28:51.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.601+0000 7f7b5d451640 1 --2- 172.21.15.5:0/891978702 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7b3806be30 0x7f7b3806e340 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:51.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.601+0000 7f7b5d451640 1 --2- 172.21.15.5:0/891978702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b580b9440 0x7f7b580b9850 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:51.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.601+0000 7f7b5d451640 1 --2- 172.21.15.5:0/891978702 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b58162820 0x7f7b580b6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:51.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.601+0000 7f7b5d451640 1 --2- 172.21.15.5:0/891978702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b580b3e80 0x7f7b581f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:51.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.601+0000 7f7b5d451640 1 -- 172.21.15.5:0/891978702 >> 172.21.15.5:0/891978702 conn(0x7f7b580d4bd0 msgr2=0x7f7b580ba560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:51.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.601+0000 7f7b5d451640 1 -- 172.21.15.5:0/891978702 shutdown_connections 2024-05-20T18:28:51.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.601+0000 7f7b5d451640 1 -- 172.21.15.5:0/891978702 wait complete. 2024-05-20T18:28:51.614 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-20T18:28:51.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.737+0000 7f352ea21640 1 Processor -- start 2024-05-20T18:28:51.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.737+0000 7f352ea21640 1 -- start start 2024-05-20T18:28:51.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.739+0000 7f352ea21640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35280b5fd0 0x7f3528070f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:51.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.739+0000 7f352ea21640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f352806cc40 0x7f35280b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:51.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.739+0000 7f352ea21640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3528160cb0 con 0x7f352815dfa0 2024-05-20T18:28:51.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.739+0000 7f352ea21640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f3528160f90 con 0x7f352806cc40 2024-05-20T18:28:51.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.739+0000 7f352ea21640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3528161270 con 0x7f35280b5fd0 2024-05-20T18:28:51.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.739+0000 7f35277fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f352815dfa0 0x7f352806c530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:41330/0 (socket says 172.21.15.5:41330) 2024-05-20T18:28:51.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.739+0000 7f35277fe640 1 -- 172.21.15.5:0/2185325297 learned_addr learned my addr 172.21.15.5:0/2185325297 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:51.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.739+0000 7f3526ffd640 1 -- 172.21.15.5:0/2185325297 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1411448334 0 0) 0x7f3528160cb0 con 0x7f352815dfa0 2024-05-20T18:28:51.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.739+0000 7f3526ffd640 1 -- 172.21.15.5:0/2185325297 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3510003520 con 0x7f352815dfa0 2024-05-20T18:28:51.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.739+0000 7f352cfbe640 1 --2- 172.21.15.5:0/2185325297 >> v2:172.21.15.63:3300/0 conn(0x7f352806cc40 0x7f35280b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:51.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.739+0000 7f3527fff640 1 --2- 172.21.15.5:0/2185325297 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35280b5fd0 0x7f3528070f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:51.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.740+0000 7f3526ffd640 1 -- 172.21.15.5:0/2185325297 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3976651198 0 0) 0x7f3510003520 con 0x7f352815dfa0 2024-05-20T18:28:51.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.740+0000 7f3526ffd640 1 -- 172.21.15.5:0/2185325297 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35280b5fd0 msgr2=0x7f3528070f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:51.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.740+0000 7f3526ffd640 1 --2- 172.21.15.5:0/2185325297 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35280b5fd0 0x7f3528070f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:51.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.740+0000 7f3526ffd640 1 -- 172.21.15.5:0/2185325297 >> v2:172.21.15.63:3300/0 conn(0x7f352806cc40 msgr2=0x7f35280b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:51.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.740+0000 7f3526ffd640 1 --2- 172.21.15.5:0/2185325297 >> v2:172.21.15.63:3300/0 conn(0x7f352806cc40 0x7f35280b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:51.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.740+0000 7f3526ffd640 1 -- 172.21.15.5:0/2185325297 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3528161550 con 0x7f352815dfa0 2024-05-20T18:28:51.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.740+0000 7f3526ffd640 1 -- 172.21.15.5:0/2185325297 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f351c001670 con 0x7f352815dfa0 2024-05-20T18:28:51.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.740+0000 7f3526ffd640 1 -- 172.21.15.5:0/2185325297 >> v1:172.21.15.5:6789/0 conn(0x7f352815dfa0 legacy=0x7f352806c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:51.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.740+0000 7f3526ffd640 1 --2- 172.21.15.5:0/2185325297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35100039b0 0x7f3510003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:51.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.740+0000 7f3526ffd640 1 --2- 172.21.15.5:0/2185325297 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35280b5fd0 0x7f3510004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:51.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.740+0000 7f3526ffd640 1 --2- 172.21.15.5:0/2185325297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3510004d40 0x7f3510005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:51.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.740+0000 7f3526ffd640 1 -- 172.21.15.5:0/2185325297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3528160f90 con 0x7f3510004d40 2024-05-20T18:28:51.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.740+0000 7f3526ffd640 1 -- 172.21.15.5:0/2185325297 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3528161270 con 0x7f35100039b0 2024-05-20T18:28:51.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.740+0000 7f3526ffd640 1 -- 172.21.15.5:0/2185325297 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f351000b9c0 con 0x7f35280b5fd0 2024-05-20T18:28:51.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.740+0000 7f352cfbe640 1 --2- 172.21.15.5:0/2185325297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3510004d40 0x7f3510005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:51.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.740+0000 7f3527fff640 1 --2- 172.21.15.5:0/2185325297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35100039b0 0x7f3510003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:51.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.740+0000 7f35277fe640 1 --2- 172.21.15.5:0/2185325297 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35280b5fd0 0x7f3510004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:51.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.740+0000 7f352cfbe640 1 -- 172.21.15.5:0/2185325297 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35280b5fd0 msgr2=0x7f3510004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:51.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.740+0000 7f352cfbe640 1 --2- 172.21.15.5:0/2185325297 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35280b5fd0 0x7f3510004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:51.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.740+0000 7f352cfbe640 1 -- 172.21.15.5:0/2185325297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35100039b0 msgr2=0x7f3510003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:51.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.740+0000 7f352cfbe640 1 --2- 172.21.15.5:0/2185325297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35100039b0 0x7f3510003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:51.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.740+0000 7f352cfbe640 1 -- 172.21.15.5:0/2185325297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3510003520 con 0x7f3510004d40 2024-05-20T18:28:51.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.741+0000 7f352cfbe640 1 --2- 172.21.15.5:0/2185325297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3510004d40 0x7f3510005150 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3528072c00 tx=0x7f3510005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce0ddf46a96b2b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:51.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.741+0000 7f3526ffd640 1 -- 172.21.15.5:0/2185325297 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3520009c10 con 0x7f3510004d40 2024-05-20T18:28:51.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.741+0000 7f3526ffd640 1 -- 172.21.15.5:0/2185325297 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3520036070 con 0x7f3510004d40 2024-05-20T18:28:51.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.741+0000 7f3526ffd640 1 -- 172.21.15.5:0/2185325297 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3520031810 con 0x7f3510004d40 2024-05-20T18:28:51.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.742+0000 7f352ea21640 1 -- 172.21.15.5:0/2185325297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3510004d40 msgr2=0x7f3510005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:51.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.742+0000 7f352ea21640 1 --2- 172.21.15.5:0/2185325297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3510004d40 0x7f3510005150 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3528072c00 tx=0x7f3510005830 comp rx=0 tx=0).stop 2024-05-20T18:28:51.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.742+0000 7f352cfbe640 1 -- 172.21.15.5:0/2185325297 reap_dead start 2024-05-20T18:28:51.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.742+0000 7f352ea21640 1 -- 172.21.15.5:0/2185325297 shutdown_connections 2024-05-20T18:28:51.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.742+0000 7f352ea21640 1 -- 172.21.15.5:0/2185325297 >> 172.21.15.5:0/2185325297 conn(0x7f35280703f0 msgr2=0x7f3528070800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:51.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.742+0000 7f352ea21640 1 -- 172.21.15.5:0/2185325297 shutdown_connections 2024-05-20T18:28:51.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.742+0000 7f352ea21640 1 -- 172.21.15.5:0/2185325297 wait complete. 2024-05-20T18:28:51.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.744+0000 7f352ea21640 1 Processor -- start 2024-05-20T18:28:51.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.744+0000 7f352ea21640 1 -- start start 2024-05-20T18:28:51.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.745+0000 7f352ea21640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3510004d40 0x7f352813d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:51.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.745+0000 7f352ea21640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f352813d930 0x7f352813dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:51.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.745+0000 7f352ea21640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f352813ea00 0x7f352813ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:51.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.745+0000 7f352ea21640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3528071090 con 0x7f3510004d40 2024-05-20T18:28:51.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.745+0000 7f352ea21640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3528070f10 con 0x7f352813ea00 2024-05-20T18:28:51.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.745+0000 7f352ea21640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3528071210 con 0x7f352813d930 2024-05-20T18:28:51.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.745+0000 7f3527fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3510004d40 0x7f352813d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:51.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.745+0000 7f3527fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3510004d40 0x7f352813d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34240/0 (socket says 172.21.15.5:34240) 2024-05-20T18:28:51.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.745+0000 7f3527fff640 1 -- 172.21.15.5:0/2714599171 learned_addr learned my addr 172.21.15.5:0/2714599171 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:51.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.746+0000 7f35277fe640 1 --2- 172.21.15.5:0/2714599171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f352813d930 0x7f352813dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:51.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.746+0000 7f352cfbe640 1 --2- 172.21.15.5:0/2714599171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f352813ea00 0x7f352813ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:51.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.746+0000 7f3527fff640 1 -- 172.21.15.5:0/2714599171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f352813d930 msgr2=0x7f352813dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:51.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.746+0000 7f3527fff640 1 --2- 172.21.15.5:0/2714599171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f352813d930 0x7f352813dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:51.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.746+0000 7f3527fff640 1 -- 172.21.15.5:0/2714599171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f352813ea00 msgr2=0x7f352813ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:51.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.746+0000 7f3527fff640 1 --2- 172.21.15.5:0/2714599171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f352813ea00 0x7f352813ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:51.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.746+0000 7f3527fff640 1 -- 172.21.15.5:0/2714599171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352813e280 con 0x7f3510004d40 2024-05-20T18:28:51.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.746+0000 7f3527fff640 1 --2- 172.21.15.5:0/2714599171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3510004d40 0x7f352813d3f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f35180119a0 tx=0x7f3518011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:51.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.746+0000 7f3524ff9640 1 -- 172.21.15.5:0/2714599171 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351800fa90 con 0x7f3510004d40 2024-05-20T18:28:51.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.746+0000 7f3524ff9640 1 -- 172.21.15.5:0/2714599171 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f351800fc30 con 0x7f3510004d40 2024-05-20T18:28:51.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.746+0000 7f352ea21640 1 -- 172.21.15.5:0/2714599171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f352814f8e0 con 0x7f3510004d40 2024-05-20T18:28:51.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.746+0000 7f3524ff9640 1 -- 172.21.15.5:0/2714599171 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3518014440 con 0x7f3510004d40 2024-05-20T18:28:51.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.746+0000 7f352ea21640 1 -- 172.21.15.5:0/2714599171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f352813cbf0 con 0x7f3510004d40 2024-05-20T18:28:51.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.747+0000 7f352ea21640 1 -- 172.21.15.5:0/2714599171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f352814cd30 con 0x7f3510004d40 2024-05-20T18:28:51.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.748+0000 7f3524ff9640 1 -- 172.21.15.5:0/2714599171 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3518012040 con 0x7f3510004d40 2024-05-20T18:28:51.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.748+0000 7f3524ff9640 1 --2- 172.21.15.5:0/2714599171 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f351406be60 0x7f351406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:51.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.749+0000 7f35277fe640 1 --2- 172.21.15.5:0/2714599171 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f351406be60 0x7f351406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:51.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.749+0000 7f3524ff9640 1 -- 172.21.15.5:0/2714599171 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f351808b7a0 con 0x7f3510004d40 2024-05-20T18:28:51.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.749+0000 7f35277fe640 1 --2- 172.21.15.5:0/2714599171 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f351406be60 0x7f351406e370 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f352815d030 tx=0x7f351c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:51.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.753+0000 7f3524ff9640 1 -- 172.21.15.5:0/2714599171 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3518019030 con 0x7f3510004d40 2024-05-20T18:28:51.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:51.936+0000 7f352ea21640 1 -- 172.21.15.5:0/2714599171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f35280690b0 con 0x7f3510004d40 2024-05-20T18:28:52.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.599+0000 7f3524ff9640 1 -- 172.21.15.5:0/2714599171 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v42) ==== 181+0+0 (secure 0 0 0) 0x7f35180597f0 con 0x7f3510004d40 2024-05-20T18:28:52.599 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-05-20T18:28:52.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.604+0000 7f352ea21640 1 -- 172.21.15.5:0/2714599171 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f351406be60 msgr2=0x7f351406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:52.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.604+0000 7f352ea21640 1 --2- 172.21.15.5:0/2714599171 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f351406be60 0x7f351406e370 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f352815d030 tx=0x7f351c009040 comp rx=0 tx=0).stop 2024-05-20T18:28:52.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.604+0000 7f352ea21640 1 -- 172.21.15.5:0/2714599171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3510004d40 msgr2=0x7f352813d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.604+0000 7f352ea21640 1 --2- 172.21.15.5:0/2714599171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3510004d40 0x7f352813d3f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f35180119a0 tx=0x7f3518011e60 comp rx=0 tx=0).stop 2024-05-20T18:28:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.604+0000 7f352ea21640 1 -- 172.21.15.5:0/2714599171 shutdown_connections 2024-05-20T18:28:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.604+0000 7f352ea21640 1 --2- 172.21.15.5:0/2714599171 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f351406be60 0x7f351406e370 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.604+0000 7f352ea21640 1 --2- 172.21.15.5:0/2714599171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f352813ea00 0x7f352813ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.604+0000 7f352ea21640 1 --2- 172.21.15.5:0/2714599171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f352813d930 0x7f352813dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.604+0000 7f352ea21640 1 --2- 172.21.15.5:0/2714599171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3510004d40 0x7f352813d3f0 secure :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f35180119a0 tx=0x7f3518011e60 comp rx=0 tx=0).stop 2024-05-20T18:28:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.604+0000 7f352ea21640 1 -- 172.21.15.5:0/2714599171 >> 172.21.15.5:0/2714599171 conn(0x7f35280703f0 msgr2=0x7f35280cfdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.604+0000 7f352ea21640 1 -- 172.21.15.5:0/2714599171 shutdown_connections 2024-05-20T18:28:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.604+0000 7f352ea21640 1 -- 172.21.15.5:0/2714599171 wait complete. 2024-05-20T18:28:52.619 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-20T18:28:52.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.745+0000 7fd259d21640 1 Processor -- start 2024-05-20T18:28:52.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.745+0000 7fd259d21640 1 -- start start 2024-05-20T18:28:52.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.746+0000 7fd259d21640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd2540d45f0 0x7fd25406d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:52.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.746+0000 7fd259d21640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd25408bd20 0x7fd25408c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:52.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.746+0000 7fd259d21640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd25408c670 con 0x7fd2540b3860 2024-05-20T18:28:52.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.746+0000 7fd259d21640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd2540d0450 con 0x7fd2540d45f0 2024-05-20T18:28:52.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.746+0000 7fd259d21640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd2540d3b10 con 0x7fd25408bd20 2024-05-20T18:28:52.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.747+0000 7fd252ffd640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd2540d45f0 0x7fd25406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:52.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.747+0000 7fd253fff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd25408bd20 0x7fd25408c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:52.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.747+0000 7fd2537fe640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fd2540b3860 0x7fd25415f230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:39618/0 (socket says 172.21.15.5:39618) 2024-05-20T18:28:52.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.747+0000 7fd2537fe640 1 -- 172.21.15.5:0/2187466580 learned_addr learned my addr 172.21.15.5:0/2187466580 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:52.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.747+0000 7fd253fff640 1 --2- 172.21.15.5:0/2187466580 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd25408bd20 0x7fd25408c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:54654/0 (socket says 172.21.15.5:54654) 2024-05-20T18:28:52.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.747+0000 7fd252ffd640 1 --2- 172.21.15.5:0/2187466580 >> v2:172.21.15.63:3300/0 conn(0x7fd2540d45f0 0x7fd25406d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:45672/0 (socket says 172.21.15.5:45672) 2024-05-20T18:28:52.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.747+0000 7fd253fff640 1 -- 172.21.15.5:0/2187466580 >> v2:172.21.15.63:3300/0 conn(0x7fd2540d45f0 msgr2=0x7fd25406d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:52.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.747+0000 7fd253fff640 1 --2- 172.21.15.5:0/2187466580 >> v2:172.21.15.63:3300/0 conn(0x7fd2540d45f0 0x7fd25406d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:52.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.747+0000 7fd253fff640 1 -- 172.21.15.5:0/2187466580 >> v1:172.21.15.63:6789/0 conn(0x7fd2540b3860 legacy=0x7fd25415f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:52.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.747+0000 7fd253fff640 1 -- 172.21.15.5:0/2187466580 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2540d3df0 con 0x7fd25408bd20 2024-05-20T18:28:52.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.747+0000 7fd252ffd640 1 --2- 172.21.15.5:0/2187466580 >> v2:172.21.15.63:3300/0 conn(0x7fd2540d45f0 0x7fd25406d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:28:52.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.748+0000 7fd253fff640 1 --2- 172.21.15.5:0/2187466580 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd25408bd20 0x7fd25408c130 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd24c009d90 tx=0x7fd24c02a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d226a4ec859e9d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:52.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.748+0000 7fd2527fc640 1 -- 172.21.15.5:0/2187466580 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24c007740 con 0x7fd25408bd20 2024-05-20T18:28:52.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.748+0000 7fd2527fc640 1 -- 172.21.15.5:0/2187466580 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd24c0078e0 con 0x7fd25408bd20 2024-05-20T18:28:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.748+0000 7fd2527fc640 1 -- 172.21.15.5:0/2187466580 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24c007cb0 con 0x7fd25408bd20 2024-05-20T18:28:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.749+0000 7fd259d21640 1 -- 172.21.15.5:0/2187466580 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd25408bd20 msgr2=0x7fd25408c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.749+0000 7fd259d21640 1 --2- 172.21.15.5:0/2187466580 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd25408bd20 0x7fd25408c130 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd24c009d90 tx=0x7fd24c02a530 comp rx=0 tx=0).stop 2024-05-20T18:28:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.749+0000 7fd259d21640 1 -- 172.21.15.5:0/2187466580 shutdown_connections 2024-05-20T18:28:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.749+0000 7fd259d21640 1 --2- 172.21.15.5:0/2187466580 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd25408bd20 0x7fd25408c130 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.749+0000 7fd259d21640 1 --2- 172.21.15.5:0/2187466580 >> v2:172.21.15.63:3300/0 conn(0x7fd2540d45f0 0x7fd25406d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.749+0000 7fd259d21640 1 -- 172.21.15.5:0/2187466580 >> 172.21.15.5:0/2187466580 conn(0x7fd2540b6d10 msgr2=0x7fd25406d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.749+0000 7fd259d21640 1 -- 172.21.15.5:0/2187466580 shutdown_connections 2024-05-20T18:28:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.749+0000 7fd259d21640 1 -- 172.21.15.5:0/2187466580 wait complete. 2024-05-20T18:28:52.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.751+0000 7fd259d21640 1 Processor -- start 2024-05-20T18:28:52.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.751+0000 7fd259d21640 1 -- start start 2024-05-20T18:28:52.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.752+0000 7fd259d21640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd25408bd20 0x7fd2541e4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:52.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.752+0000 7fd259d21640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2540b3860 0x7fd2541f83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:52.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.752+0000 7fd259d21640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2540d45f0 0x7fd2541e9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:52.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.752+0000 7fd259d21640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd2540b81b0 con 0x7fd25408bd20 2024-05-20T18:28:52.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.752+0000 7fd259d21640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd2540d0730 con 0x7fd2540d45f0 2024-05-20T18:28:52.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.752+0000 7fd259d21640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd2540b8330 con 0x7fd2540b3860 2024-05-20T18:28:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.753+0000 7fd2537fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd25408bd20 0x7fd2541e4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.753+0000 7fd2537fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd25408bd20 0x7fd2541e4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34242/0 (socket says 172.21.15.5:34242) 2024-05-20T18:28:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.753+0000 7fd2537fe640 1 -- 172.21.15.5:0/892739465 learned_addr learned my addr 172.21.15.5:0/892739465 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.753+0000 7fd252ffd640 1 --2- 172.21.15.5:0/892739465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2540b3860 0x7fd2541f83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.753+0000 7fd253fff640 1 --2- 172.21.15.5:0/892739465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2540d45f0 0x7fd2541e9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.753+0000 7fd2537fe640 1 -- 172.21.15.5:0/892739465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2540b3860 msgr2=0x7fd2541f83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.753+0000 7fd2537fe640 1 --2- 172.21.15.5:0/892739465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2540b3860 0x7fd2541f83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.753+0000 7fd2537fe640 1 -- 172.21.15.5:0/892739465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2540d45f0 msgr2=0x7fd2541e9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.753+0000 7fd2537fe640 1 --2- 172.21.15.5:0/892739465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2540d45f0 0x7fd2541e9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.753+0000 7fd2537fe640 1 -- 172.21.15.5:0/892739465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2541e9b80 con 0x7fd25408bd20 2024-05-20T18:28:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.753+0000 7fd2537fe640 1 --2- 172.21.15.5:0/892739465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd25408bd20 0x7fd2541e4410 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd24400c470 tx=0x7fd24400c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.753+0000 7fd23bfff640 1 -- 172.21.15.5:0/892739465 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd244017070 con 0x7fd25408bd20 2024-05-20T18:28:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.753+0000 7fd23bfff640 1 -- 172.21.15.5:0/892739465 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd244005ce0 con 0x7fd25408bd20 2024-05-20T18:28:52.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.753+0000 7fd23bfff640 1 -- 172.21.15.5:0/892739465 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2440072d0 con 0x7fd25408bd20 2024-05-20T18:28:52.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.753+0000 7fd259d21640 1 -- 172.21.15.5:0/892739465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd254158670 con 0x7fd25408bd20 2024-05-20T18:28:52.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.753+0000 7fd259d21640 1 -- 172.21.15.5:0/892739465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd254158c30 con 0x7fd25408bd20 2024-05-20T18:28:52.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.755+0000 7fd259d21640 1 -- 172.21.15.5:0/892739465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2540690b0 con 0x7fd25408bd20 2024-05-20T18:28:52.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.755+0000 7fd23bfff640 1 -- 172.21.15.5:0/892739465 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd24401a070 con 0x7fd25408bd20 2024-05-20T18:28:52.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.756+0000 7fd23bfff640 1 --2- 172.21.15.5:0/892739465 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd24006be60 0x7fd24006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:52.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.756+0000 7fd23bfff640 1 -- 172.21.15.5:0/892739465 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5749+0+0 (secure 0 0 0) 0x7fd24408a110 con 0x7fd25408bd20 2024-05-20T18:28:52.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.756+0000 7fd252ffd640 1 --2- 172.21.15.5:0/892739465 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd24006be60 0x7fd24006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:52.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.756+0000 7fd252ffd640 1 --2- 172.21.15.5:0/892739465 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd24006be60 0x7fd24006e370 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd2540b54d0 tx=0x7fd248009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:52.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.760+0000 7fd23bfff640 1 -- 172.21.15.5:0/892739465 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd25408c670 con 0x7fd25408bd20 2024-05-20T18:28:52.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:52.951+0000 7fd259d21640 1 -- 172.21.15.5:0/892739465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd2540b3640 con 0x7fd25408bd20 2024-05-20T18:28:53.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.605+0000 7fd23bfff640 1 -- 172.21.15.5:0/892739465 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v43) ==== 169+0+0 (secure 0 0 0) 0x7fd2440583b0 con 0x7fd25408bd20 2024-05-20T18:28:53.604 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-05-20T18:28:53.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.608+0000 7fd259d21640 1 -- 172.21.15.5:0/892739465 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd24006be60 msgr2=0x7fd24006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:53.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.608+0000 7fd259d21640 1 --2- 172.21.15.5:0/892739465 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd24006be60 0x7fd24006e370 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd2540b54d0 tx=0x7fd248009190 comp rx=0 tx=0).stop 2024-05-20T18:28:53.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.608+0000 7fd259d21640 1 -- 172.21.15.5:0/892739465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd25408bd20 msgr2=0x7fd2541e4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:53.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.608+0000 7fd259d21640 1 --2- 172.21.15.5:0/892739465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd25408bd20 0x7fd2541e4410 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd24400c470 tx=0x7fd24400c930 comp rx=0 tx=0).stop 2024-05-20T18:28:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.609+0000 7fd259d21640 1 -- 172.21.15.5:0/892739465 shutdown_connections 2024-05-20T18:28:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.609+0000 7fd259d21640 1 --2- 172.21.15.5:0/892739465 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd24006be60 0x7fd24006e370 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.609+0000 7fd259d21640 1 --2- 172.21.15.5:0/892739465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2540d45f0 0x7fd2541e9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.609+0000 7fd259d21640 1 --2- 172.21.15.5:0/892739465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2540b3860 0x7fd2541f83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.609+0000 7fd259d21640 1 --2- 172.21.15.5:0/892739465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd25408bd20 0x7fd2541e4410 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.609+0000 7fd259d21640 1 -- 172.21.15.5:0/892739465 >> 172.21.15.5:0/892739465 conn(0x7fd2540b6d10 msgr2=0x7fd2540cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.609+0000 7fd259d21640 1 -- 172.21.15.5:0/892739465 shutdown_connections 2024-05-20T18:28:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.609+0000 7fd259d21640 1 -- 172.21.15.5:0/892739465 wait complete. 2024-05-20T18:28:53.623 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-20T18:28:53.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.743+0000 7f5411c5d640 1 Processor -- start 2024-05-20T18:28:53.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.743+0000 7f5411c5d640 1 -- start start 2024-05-20T18:28:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.744+0000 7f5411c5d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f540c06c640 0x7f540c16a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.744+0000 7f5411c5d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f540c0b4360 con 0x7f540c15f310 2024-05-20T18:28:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.744+0000 7f5411c5d640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f540c06ca50 con 0x7f540c0d0190 2024-05-20T18:28:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.744+0000 7f5411c5d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f540c15f720 con 0x7f540c06c640 2024-05-20T18:28:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.745+0000 7f540bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f540c06c640 0x7f540c16a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.745+0000 7f540affd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f540c15f310 0x7f540c06bf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:41346/0 (socket says 172.21.15.5:41346) 2024-05-20T18:28:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.745+0000 7f540affd640 1 -- 172.21.15.5:0/2741198243 learned_addr learned my addr 172.21.15.5:0/2741198243 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.745+0000 7f540a7fc640 1 -- 172.21.15.5:0/2741198243 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2796805110 0 0) 0x7f540c0b4360 con 0x7f540c15f310 2024-05-20T18:28:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.745+0000 7f540a7fc640 1 -- 172.21.15.5:0/2741198243 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53f4003520 con 0x7f540c15f310 2024-05-20T18:28:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.745+0000 7f540bfff640 1 -- 172.21.15.5:0/2741198243 >> v1:172.21.15.63:6789/0 conn(0x7f540c0d0190 legacy=0x7f540c0ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.745+0000 7f540bfff640 1 -- 172.21.15.5:0/2741198243 >> v1:172.21.15.5:6789/0 conn(0x7f540c15f310 legacy=0x7f540c06bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:53.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.745+0000 7f540bfff640 1 -- 172.21.15.5:0/2741198243 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540c0b39f0 con 0x7f540c06c640 2024-05-20T18:28:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.746+0000 7f540bfff640 1 --2- 172.21.15.5:0/2741198243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f540c06c640 0x7f540c16a1c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5404009d90 tx=0x7f540402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cda63b8582a0525 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.746+0000 7f540a7fc640 1 -- 172.21.15.5:0/2741198243 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5404007740 con 0x7f540c06c640 2024-05-20T18:28:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.746+0000 7f540a7fc640 1 -- 172.21.15.5:0/2741198243 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54040078e0 con 0x7f540c06c640 2024-05-20T18:28:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.746+0000 7f540a7fc640 1 -- 172.21.15.5:0/2741198243 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5404007cb0 con 0x7f540c06c640 2024-05-20T18:28:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.747+0000 7f5411c5d640 1 -- 172.21.15.5:0/2741198243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f540c06c640 msgr2=0x7f540c16a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.747+0000 7f5411c5d640 1 --2- 172.21.15.5:0/2741198243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f540c06c640 0x7f540c16a1c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5404009d90 tx=0x7f540402a660 comp rx=0 tx=0).stop 2024-05-20T18:28:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.747+0000 7f5411c5d640 1 -- 172.21.15.5:0/2741198243 shutdown_connections 2024-05-20T18:28:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.747+0000 7f5411c5d640 1 --2- 172.21.15.5:0/2741198243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f540c06c640 0x7f540c16a1c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.747+0000 7f5411c5d640 1 -- 172.21.15.5:0/2741198243 >> 172.21.15.5:0/2741198243 conn(0x7f540c0cd2a0 msgr2=0x7f540c0cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.747+0000 7f5411c5d640 1 -- 172.21.15.5:0/2741198243 shutdown_connections 2024-05-20T18:28:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.747+0000 7f5411c5d640 1 -- 172.21.15.5:0/2741198243 wait complete. 2024-05-20T18:28:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.749+0000 7f5411c5d640 1 Processor -- start 2024-05-20T18:28:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.749+0000 7f5411c5d640 1 -- start start 2024-05-20T18:28:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.750+0000 7f5411c5d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f540c06c640 0x7f540c1603f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.750+0000 7f5411c5d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f540c0d0190 0x7f540c1545a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.750+0000 7f5411c5d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f540c15f310 0x7f540c159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.750+0000 7f5411c5d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f540c0b3cd0 con 0x7f540c0d0190 2024-05-20T18:28:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.750+0000 7f5411c5d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f540c0b48e0 con 0x7f540c15f310 2024-05-20T18:28:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.750+0000 7f5411c5d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f540c0cec50 con 0x7f540c06c640 2024-05-20T18:28:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.750+0000 7f540affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f540c0d0190 0x7f540c1545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.750+0000 7f540affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f540c0d0190 0x7f540c1545a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34252/0 (socket says 172.21.15.5:34252) 2024-05-20T18:28:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.750+0000 7f540affd640 1 -- 172.21.15.5:0/2723855707 learned_addr learned my addr 172.21.15.5:0/2723855707 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.750+0000 7f540bfff640 1 --2- 172.21.15.5:0/2723855707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f540c15f310 0x7f540c159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.750+0000 7f540b7fe640 1 --2- 172.21.15.5:0/2723855707 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f540c06c640 0x7f540c1603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.750+0000 7f540affd640 1 -- 172.21.15.5:0/2723855707 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f540c06c640 msgr2=0x7f540c1603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.750+0000 7f540affd640 1 --2- 172.21.15.5:0/2723855707 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f540c06c640 0x7f540c1603f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.750+0000 7f540affd640 1 -- 172.21.15.5:0/2723855707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f540c15f310 msgr2=0x7f540c159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.750+0000 7f540affd640 1 --2- 172.21.15.5:0/2723855707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f540c15f310 0x7f540c159650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.751+0000 7f540affd640 1 -- 172.21.15.5:0/2723855707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540c15f950 con 0x7f540c0d0190 2024-05-20T18:28:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.751+0000 7f540bfff640 1 --2- 172.21.15.5:0/2723855707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f540c15f310 0x7f540c159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.751+0000 7f540b7fe640 1 --2- 172.21.15.5:0/2723855707 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f540c06c640 0x7f540c1603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.751+0000 7f540affd640 1 --2- 172.21.15.5:0/2723855707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f540c0d0190 0x7f540c1545a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f540000c430 tx=0x7f540000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.751+0000 7f53f3fff640 1 -- 172.21.15.5:0/2723855707 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5400017070 con 0x7f540c0d0190 2024-05-20T18:28:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.751+0000 7f53f3fff640 1 -- 172.21.15.5:0/2723855707 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5400005ce0 con 0x7f540c0d0190 2024-05-20T18:28:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.751+0000 7f53f3fff640 1 -- 172.21.15.5:0/2723855707 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54000072d0 con 0x7f540c0d0190 2024-05-20T18:28:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.751+0000 7f5411c5d640 1 -- 172.21.15.5:0/2723855707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540c159b90 con 0x7f540c0d0190 2024-05-20T18:28:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.751+0000 7f5411c5d640 1 -- 172.21.15.5:0/2723855707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f540c15fbf0 con 0x7f540c0d0190 2024-05-20T18:28:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.752+0000 7f5411c5d640 1 -- 172.21.15.5:0/2723855707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f540c068140 con 0x7f540c0d0190 2024-05-20T18:28:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.753+0000 7f53f3fff640 1 -- 172.21.15.5:0/2723855707 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f540001a070 con 0x7f540c0d0190 2024-05-20T18:28:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.753+0000 7f53f3fff640 1 --2- 172.21.15.5:0/2723855707 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f53f806be30 0x7f53f806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.753+0000 7f53f3fff640 1 -- 172.21.15.5:0/2723855707 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f540008a080 con 0x7f540c0d0190 2024-05-20T18:28:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.753+0000 7f540b7fe640 1 --2- 172.21.15.5:0/2723855707 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f53f806be30 0x7f53f806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.754+0000 7f540b7fe640 1 --2- 172.21.15.5:0/2723855707 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f53f806be30 0x7f53f806e340 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0015e0 tx=0x7f53fc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:53.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.758+0000 7f53f3fff640 1 -- 172.21.15.5:0/2723855707 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f540c068140 con 0x7f540c0d0190 2024-05-20T18:28:53.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.946+0000 7f5411c5d640 1 -- 172.21.15.5:0/2723855707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f540c075b00 con 0x7f540c0d0190 2024-05-20T18:28:53.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.950+0000 7f53f3fff640 1 -- 172.21.15.5:0/2723855707 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f5400058440 con 0x7f540c0d0190 2024-05-20T18:28:53.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.952+0000 7f5411c5d640 1 -- 172.21.15.5:0/2723855707 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f53f806be30 msgr2=0x7f53f806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:53.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.952+0000 7f5411c5d640 1 --2- 172.21.15.5:0/2723855707 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f53f806be30 0x7f53f806e340 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0015e0 tx=0x7f53fc009040 comp rx=0 tx=0).stop 2024-05-20T18:28:53.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.952+0000 7f5411c5d640 1 -- 172.21.15.5:0/2723855707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f540c0d0190 msgr2=0x7f540c1545a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:53.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.952+0000 7f5411c5d640 1 --2- 172.21.15.5:0/2723855707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f540c0d0190 0x7f540c1545a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f540000c430 tx=0x7f540000c8f0 comp rx=0 tx=0).stop 2024-05-20T18:28:53.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.952+0000 7f5411c5d640 1 -- 172.21.15.5:0/2723855707 shutdown_connections 2024-05-20T18:28:53.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.952+0000 7f5411c5d640 1 --2- 172.21.15.5:0/2723855707 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f53f806be30 0x7f53f806e340 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:53.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.952+0000 7f5411c5d640 1 --2- 172.21.15.5:0/2723855707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f540c15f310 0x7f540c159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:53.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.952+0000 7f5411c5d640 1 --2- 172.21.15.5:0/2723855707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f540c0d0190 0x7f540c1545a0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:53.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.952+0000 7f5411c5d640 1 --2- 172.21.15.5:0/2723855707 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f540c06c640 0x7f540c1603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:53.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.952+0000 7f5411c5d640 1 -- 172.21.15.5:0/2723855707 >> 172.21.15.5:0/2723855707 conn(0x7f540c0cd2a0 msgr2=0x7f540c15bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:53.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.952+0000 7f5411c5d640 1 -- 172.21.15.5:0/2723855707 shutdown_connections 2024-05-20T18:28:53.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:53.952+0000 7f5411c5d640 1 -- 172.21.15.5:0/2723855707 wait complete. 2024-05-20T18:28:53.963 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-05-20T18:28:54.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.078+0000 7efd1288b640 1 Processor -- start 2024-05-20T18:28:54.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.078+0000 7efd1288b640 1 -- start start 2024-05-20T18:28:54.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.080+0000 7efd1288b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efd0c162820 0x7efd0c0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:54.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.080+0000 7efd1288b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7efd0c06dae0 0x7efd0c0b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:54.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.080+0000 7efd1288b640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd0c0d0630 con 0x7efd0c0b3e80 2024-05-20T18:28:54.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.080+0000 7efd1288b640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7efd0c0d39c0 con 0x7efd0c162820 2024-05-20T18:28:54.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.080+0000 7efd1288b640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7efd0c0d3ca0 con 0x7efd0c06dae0 2024-05-20T18:28:54.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.080+0000 7efd0bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efd0c162820 0x7efd0c0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:54.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.080+0000 7efd0bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efd0c162820 0x7efd0c0b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34264/0 (socket says 172.21.15.5:34264) 2024-05-20T18:28:54.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.080+0000 7efd0bfff640 1 -- 172.21.15.5:0/2471381106 learned_addr learned my addr 172.21.15.5:0/2471381106 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:54.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.080+0000 7efd10e28640 1 --2- 172.21.15.5:0/2471381106 >> v2:172.21.15.63:3300/0 conn(0x7efd0c06dae0 0x7efd0c0b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:54.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.080+0000 7efd0bfff640 1 -- 172.21.15.5:0/2471381106 >> v2:172.21.15.63:3300/0 conn(0x7efd0c06dae0 msgr2=0x7efd0c0b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:54.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.080+0000 7efd0bfff640 1 --2- 172.21.15.5:0/2471381106 >> v2:172.21.15.63:3300/0 conn(0x7efd0c06dae0 0x7efd0c0b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:54.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.080+0000 7efd0bfff640 1 -- 172.21.15.5:0/2471381106 >> v1:172.21.15.63:6789/0 conn(0x7efd0c0b3e80 legacy=0x7efd0c06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:54.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.080+0000 7efd0bfff640 1 -- 172.21.15.5:0/2471381106 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0c0d3f80 con 0x7efd0c162820 2024-05-20T18:28:54.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.081+0000 7efd0bfff640 1 --2- 172.21.15.5:0/2471381106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd0c162820 0x7efd0c0b3940 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7efd0c08e070 tx=0x7efcfc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c547c3aec0085f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:54.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.081+0000 7efd0b7fe640 1 -- 172.21.15.5:0/2471381106 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcfc037070 con 0x7efd0c162820 2024-05-20T18:28:54.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.081+0000 7efd0b7fe640 1 -- 172.21.15.5:0/2471381106 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcfc032440 con 0x7efd0c162820 2024-05-20T18:28:54.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.081+0000 7efd0b7fe640 1 -- 172.21.15.5:0/2471381106 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcfc032810 con 0x7efd0c162820 2024-05-20T18:28:54.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.082+0000 7efd1288b640 1 -- 172.21.15.5:0/2471381106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd0c162820 msgr2=0x7efd0c0b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:54.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.082+0000 7efd1288b640 1 --2- 172.21.15.5:0/2471381106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd0c162820 0x7efd0c0b3940 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7efd0c08e070 tx=0x7efcfc02a270 comp rx=0 tx=0).stop 2024-05-20T18:28:54.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.082+0000 7efd1288b640 1 -- 172.21.15.5:0/2471381106 shutdown_connections 2024-05-20T18:28:54.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.082+0000 7efd1288b640 1 --2- 172.21.15.5:0/2471381106 >> v2:172.21.15.63:3300/0 conn(0x7efd0c06dae0 0x7efd0c0b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:54.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.082+0000 7efd1288b640 1 --2- 172.21.15.5:0/2471381106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd0c162820 0x7efd0c0b3940 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:54.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.082+0000 7efd1288b640 1 -- 172.21.15.5:0/2471381106 >> 172.21.15.5:0/2471381106 conn(0x7efd0c0d4bd0 msgr2=0x7efd0c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:54.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.082+0000 7efd1288b640 1 -- 172.21.15.5:0/2471381106 shutdown_connections 2024-05-20T18:28:54.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.082+0000 7efd1288b640 1 -- 172.21.15.5:0/2471381106 wait complete. 2024-05-20T18:28:54.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.084+0000 7efd1288b640 1 Processor -- start 2024-05-20T18:28:54.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.084+0000 7efd1288b640 1 -- start start 2024-05-20T18:28:54.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.085+0000 7efd1288b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd0c0b3e80 0x7efd0c0cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:54.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.085+0000 7efd1288b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd0c162820 0x7efd0c15b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:54.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.085+0000 7efd1288b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd0c0cbc80 0x7efd0c0b64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:54.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.085+0000 7efd1288b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c0d4780 con 0x7efd0c162820 2024-05-20T18:28:54.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.085+0000 7efd1288b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c0d0300 con 0x7efd0c0b3e80 2024-05-20T18:28:54.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.085+0000 7efd1288b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efd0c0d4900 con 0x7efd0c0cbc80 2024-05-20T18:28:54.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.085+0000 7efd03fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd0c162820 0x7efd0c15b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:54.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.085+0000 7efd03fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd0c162820 0x7efd0c15b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34280/0 (socket says 172.21.15.5:34280) 2024-05-20T18:28:54.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.085+0000 7efd03fff640 1 -- 172.21.15.5:0/3337595521 learned_addr learned my addr 172.21.15.5:0/3337595521 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:54.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.085+0000 7efd0bfff640 1 --2- 172.21.15.5:0/3337595521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd0c0b3e80 0x7efd0c0cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:54.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.085+0000 7efd10e28640 1 --2- 172.21.15.5:0/3337595521 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd0c0cbc80 0x7efd0c0b64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:54.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.086+0000 7efd03fff640 1 -- 172.21.15.5:0/3337595521 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd0c0cbc80 msgr2=0x7efd0c0b64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:54.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.086+0000 7efd03fff640 1 --2- 172.21.15.5:0/3337595521 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd0c0cbc80 0x7efd0c0b64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:54.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.086+0000 7efd03fff640 1 -- 172.21.15.5:0/3337595521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd0c0b3e80 msgr2=0x7efd0c0cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:54.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.086+0000 7efd03fff640 1 --2- 172.21.15.5:0/3337595521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd0c0b3e80 0x7efd0c0cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:54.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.086+0000 7efd03fff640 1 -- 172.21.15.5:0/3337595521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0c0cc360 con 0x7efd0c162820 2024-05-20T18:28:54.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.086+0000 7efd10e28640 1 --2- 172.21.15.5:0/3337595521 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd0c0cbc80 0x7efd0c0b64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:54.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.086+0000 7efd0bfff640 1 --2- 172.21.15.5:0/3337595521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd0c0b3e80 0x7efd0c0cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:54.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.086+0000 7efd03fff640 1 --2- 172.21.15.5:0/3337595521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd0c162820 0x7efd0c15b520 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7efcf400c3c0 tx=0x7efcf400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:54.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.086+0000 7efd097fa640 1 -- 172.21.15.5:0/3337595521 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf40052d0 con 0x7efd0c162820 2024-05-20T18:28:54.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.086+0000 7efd097fa640 1 -- 172.21.15.5:0/3337595521 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcf4005490 con 0x7efd0c162820 2024-05-20T18:28:54.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.086+0000 7efd1288b640 1 -- 172.21.15.5:0/3337595521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd0c0ceae0 con 0x7efd0c162820 2024-05-20T18:28:54.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.086+0000 7efd097fa640 1 -- 172.21.15.5:0/3337595521 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf4007260 con 0x7efd0c162820 2024-05-20T18:28:54.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.086+0000 7efd1288b640 1 -- 172.21.15.5:0/3337595521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd0c0cf0a0 con 0x7efd0c162820 2024-05-20T18:28:54.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.087+0000 7efd1288b640 1 -- 172.21.15.5:0/3337595521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd0c0cf2e0 con 0x7efd0c162820 2024-05-20T18:28:54.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.088+0000 7efd097fa640 1 -- 172.21.15.5:0/3337595521 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efcf4007400 con 0x7efd0c162820 2024-05-20T18:28:54.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.088+0000 7efd097fa640 1 --2- 172.21.15.5:0/3337595521 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efcec06be60 0x7efcec06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:54.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.088+0000 7efd097fa640 1 -- 172.21.15.5:0/3337595521 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7efcf408a200 con 0x7efd0c162820 2024-05-20T18:28:54.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.088+0000 7efd0bfff640 1 --2- 172.21.15.5:0/3337595521 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efcec06be60 0x7efcec06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:54.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.089+0000 7efd0bfff640 1 --2- 172.21.15.5:0/3337595521 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efcec06be60 0x7efcec06e370 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efcfc034000 tx=0x7efcfc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:54.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.093+0000 7efd097fa640 1 -- 172.21.15.5:0/3337595521 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7efcf4058c90 con 0x7efd0c162820 2024-05-20T18:28:54.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.361+0000 7efd1288b640 1 -- 172.21.15.5:0/3337595521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7efd0c0cc0f0 con 0x7efd0c162820 2024-05-20T18:28:54.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.606+0000 7efd097fa640 1 -- 172.21.15.5:0/3337595521 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v114) ==== 142+0+0 (secure 0 0 0) 0x7efcf4058590 con 0x7efd0c162820 2024-05-20T18:28:54.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.608+0000 7efd1288b640 1 -- 172.21.15.5:0/3337595521 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efcec06be60 msgr2=0x7efcec06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:54.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.608+0000 7efd1288b640 1 --2- 172.21.15.5:0/3337595521 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efcec06be60 0x7efcec06e370 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efcfc034000 tx=0x7efcfc02e000 comp rx=0 tx=0).stop 2024-05-20T18:28:54.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.608+0000 7efd1288b640 1 -- 172.21.15.5:0/3337595521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd0c162820 msgr2=0x7efd0c15b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:54.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.608+0000 7efd1288b640 1 --2- 172.21.15.5:0/3337595521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd0c162820 0x7efd0c15b520 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7efcf400c3c0 tx=0x7efcf400c880 comp rx=0 tx=0).stop 2024-05-20T18:28:54.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.608+0000 7efd1288b640 1 -- 172.21.15.5:0/3337595521 shutdown_connections 2024-05-20T18:28:54.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.608+0000 7efd1288b640 1 --2- 172.21.15.5:0/3337595521 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efcec06be60 0x7efcec06e370 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:54.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.608+0000 7efd1288b640 1 --2- 172.21.15.5:0/3337595521 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd0c0cbc80 0x7efd0c0b64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:54.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.608+0000 7efd1288b640 1 --2- 172.21.15.5:0/3337595521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd0c162820 0x7efd0c15b520 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:54.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.608+0000 7efd1288b640 1 --2- 172.21.15.5:0/3337595521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd0c0b3e80 0x7efd0c0cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:54.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.608+0000 7efd1288b640 1 -- 172.21.15.5:0/3337595521 >> 172.21.15.5:0/3337595521 conn(0x7efd0c0d4bd0 msgr2=0x7efd0c15fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:54.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.608+0000 7efd1288b640 1 -- 172.21.15.5:0/3337595521 shutdown_connections 2024-05-20T18:28:54.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.608+0000 7efd1288b640 1 -- 172.21.15.5:0/3337595521 wait complete. 2024-05-20T18:28:54.619 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-05-20T18:28:54.620 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2024-05-20T18:28:54.620 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-20T18:28:54.620 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:28:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.735+0000 7f89398a2640 1 Processor -- start 2024-05-20T18:28:54.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.735+0000 7f89398a2640 1 -- start start 2024-05-20T18:28:54.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.736+0000 7f89398a2640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8934162820 0x7f89340b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:54.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.736+0000 7f89398a2640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f89340b3e80 0x7f893406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:54.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.736+0000 7f89398a2640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f893406d910 0x7f89340d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:54.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.736+0000 7f89398a2640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f893406dd20 con 0x7f8934162820 2024-05-20T18:28:54.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.736+0000 7f89398a2640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f89340d0630 con 0x7f89340b3e80 2024-05-20T18:28:54.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.736+0000 7f89398a2640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f89340d39c0 con 0x7f893406d910 2024-05-20T18:28:54.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.736+0000 7f8932ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8934162820 0x7f89340b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:54.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.736+0000 7f8932ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8934162820 0x7f89340b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59566/0 (socket says 172.21.15.5:59566) 2024-05-20T18:28:54.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.736+0000 7f8932ffd640 1 -- 172.21.15.5:0/248297913 learned_addr learned my addr 172.21.15.5:0/248297913 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:54.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.737+0000 7f89337fe640 1 --2- 172.21.15.5:0/248297913 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f893406d910 0x7f89340d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:54.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.737+0000 7f8932ffd640 1 -- 172.21.15.5:0/248297913 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f893406d910 msgr2=0x7f89340d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:54.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.737+0000 7f89327fc640 1 --2- 172.21.15.5:0/248297913 >> v2:172.21.15.63:3300/0 conn(0x7f89340b3e80 0x7f893406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:54.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.737+0000 7f8932ffd640 1 --2- 172.21.15.5:0/248297913 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f893406d910 0x7f89340d27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:54.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.737+0000 7f8932ffd640 1 -- 172.21.15.5:0/248297913 >> v2:172.21.15.63:3300/0 conn(0x7f89340b3e80 msgr2=0x7f893406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:54.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.737+0000 7f8932ffd640 1 --2- 172.21.15.5:0/248297913 >> v2:172.21.15.63:3300/0 conn(0x7f89340b3e80 0x7f893406d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:54.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.737+0000 7f8932ffd640 1 -- 172.21.15.5:0/248297913 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89340d3ca0 con 0x7f8934162820 2024-05-20T18:28:54.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.737+0000 7f89337fe640 1 --2- 172.21.15.5:0/248297913 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f893406d910 0x7f89340d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:54.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.737+0000 7f89327fc640 1 --2- 172.21.15.5:0/248297913 >> v2:172.21.15.63:3300/0 conn(0x7f89340b3e80 0x7f893406d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:54.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.738+0000 7f8932ffd640 1 --2- 172.21.15.5:0/248297913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8934162820 0x7f89340b3940 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f89340d1230 tx=0x7f892402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a33dd53053f9e482 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:54.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.738+0000 7f8931ffb640 1 -- 172.21.15.5:0/248297913 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8924037070 con 0x7f8934162820 2024-05-20T18:28:54.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.738+0000 7f8931ffb640 1 -- 172.21.15.5:0/248297913 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8924032440 con 0x7f8934162820 2024-05-20T18:28:54.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.738+0000 7f8931ffb640 1 -- 172.21.15.5:0/248297913 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8924032810 con 0x7f8934162820 2024-05-20T18:28:54.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.738+0000 7f89398a2640 1 -- 172.21.15.5:0/248297913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8934162820 msgr2=0x7f89340b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:54.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.738+0000 7f89398a2640 1 --2- 172.21.15.5:0/248297913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8934162820 0x7f89340b3940 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f89340d1230 tx=0x7f892402a1e0 comp rx=0 tx=0).stop 2024-05-20T18:28:54.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.738+0000 7f89398a2640 1 -- 172.21.15.5:0/248297913 shutdown_connections 2024-05-20T18:28:54.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.738+0000 7f89398a2640 1 --2- 172.21.15.5:0/248297913 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f893406d910 0x7f89340d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:54.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.738+0000 7f89398a2640 1 --2- 172.21.15.5:0/248297913 >> v2:172.21.15.63:3300/0 conn(0x7f89340b3e80 0x7f893406d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:54.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.738+0000 7f89398a2640 1 --2- 172.21.15.5:0/248297913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8934162820 0x7f89340b3940 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:54.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.738+0000 7f89398a2640 1 -- 172.21.15.5:0/248297913 >> 172.21.15.5:0/248297913 conn(0x7f89340d4bd0 msgr2=0x7f89340d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:54.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.739+0000 7f89398a2640 1 -- 172.21.15.5:0/248297913 shutdown_connections 2024-05-20T18:28:54.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.739+0000 7f89398a2640 1 -- 172.21.15.5:0/248297913 wait complete. 2024-05-20T18:28:54.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.740+0000 7f89398a2640 1 Processor -- start 2024-05-20T18:28:54.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.740+0000 7f89398a2640 1 -- start start 2024-05-20T18:28:54.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.742+0000 7f89398a2640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f893406d910 0x7f8934150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:54.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.742+0000 7f89398a2640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89340b3e80 0x7f8934150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:54.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.742+0000 7f89398a2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8934162820 0x7f89340c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:54.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.742+0000 7f89398a2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f89340d0300 con 0x7f8934162820 2024-05-20T18:28:54.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.742+0000 7f89398a2640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f89340d4260 con 0x7f893406d910 2024-05-20T18:28:54.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.742+0000 7f89398a2640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f89340d4780 con 0x7f89340b3e80 2024-05-20T18:28:54.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.742+0000 7f89337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8934162820 0x7f89340c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:54.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.742+0000 7f89337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8934162820 0x7f89340c8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59582/0 (socket says 172.21.15.5:59582) 2024-05-20T18:28:54.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.742+0000 7f89337fe640 1 -- 172.21.15.5:0/3841246501 learned_addr learned my addr 172.21.15.5:0/3841246501 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:54.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.742+0000 7f89327fc640 1 --2- 172.21.15.5:0/3841246501 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89340b3e80 0x7f8934150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:54.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.742+0000 7f8932ffd640 1 --2- 172.21.15.5:0/3841246501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f893406d910 0x7f8934150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:54.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.742+0000 7f89337fe640 1 -- 172.21.15.5:0/3841246501 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89340b3e80 msgr2=0x7f8934150220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:54.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.742+0000 7f89337fe640 1 --2- 172.21.15.5:0/3841246501 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89340b3e80 0x7f8934150220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:54.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.742+0000 7f89337fe640 1 -- 172.21.15.5:0/3841246501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f893406d910 msgr2=0x7f8934150dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:54.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.742+0000 7f89337fe640 1 --2- 172.21.15.5:0/3841246501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f893406d910 0x7f8934150dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:54.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.742+0000 7f89337fe640 1 -- 172.21.15.5:0/3841246501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893415ccc0 con 0x7f8934162820 2024-05-20T18:28:54.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.743+0000 7f89337fe640 1 --2- 172.21.15.5:0/3841246501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8934162820 0x7f89340c8280 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f892c00ccc0 tx=0x7f892c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:54.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.743+0000 7f891bfff640 1 -- 172.21.15.5:0/3841246501 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c00f680 con 0x7f8934162820 2024-05-20T18:28:54.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.743+0000 7f891bfff640 1 -- 172.21.15.5:0/3841246501 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f892c00f820 con 0x7f8934162820 2024-05-20T18:28:54.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.743+0000 7f89398a2640 1 -- 172.21.15.5:0/3841246501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89340c8e60 con 0x7f8934162820 2024-05-20T18:28:54.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.743+0000 7f891bfff640 1 -- 172.21.15.5:0/3841246501 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c00fc10 con 0x7f8934162820 2024-05-20T18:28:54.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.743+0000 7f89398a2640 1 -- 172.21.15.5:0/3841246501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89340c9420 con 0x7f8934162820 2024-05-20T18:28:54.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.744+0000 7f89398a2640 1 -- 172.21.15.5:0/3841246501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8934075b00 con 0x7f8934162820 2024-05-20T18:28:54.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.744+0000 7f891bfff640 1 -- 172.21.15.5:0/3841246501 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f892c0213f0 con 0x7f8934162820 2024-05-20T18:28:54.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.745+0000 7f891bfff640 1 --2- 172.21.15.5:0/3841246501 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f892006be60 0x7f892006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:54.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.745+0000 7f8932ffd640 1 --2- 172.21.15.5:0/3841246501 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f892006be60 0x7f892006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:54.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.745+0000 7f891bfff640 1 -- 172.21.15.5:0/3841246501 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f892c08b840 con 0x7f8934162820 2024-05-20T18:28:54.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.745+0000 7f8932ffd640 1 --2- 172.21.15.5:0/3841246501 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f892006be60 0x7f892006e370 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8924034000 tx=0x7f892402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:54.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.749+0000 7f891bfff640 1 -- 172.21.15.5:0/3841246501 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f893415ccc0 con 0x7f8934162820 2024-05-20T18:28:54.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.935+0000 7f89398a2640 1 -- 172.21.15.5:0/3841246501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f893415d9d0 con 0x7f8934162820 2024-05-20T18:28:54.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.936+0000 7f891bfff640 1 -- 172.21.15.5:0/3841246501 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+13171 (secure 0 0 0) 0x7f892c014030 con 0x7f8934162820 2024-05-20T18:28:54.936 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:54.936 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":43,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:28:53.601180+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6837/2355805105":"2024-05-21T18:28:50.593365+0000","172.21.15.5:6835/2355805105":"2024-05-21T18:28:50.593365+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:28:54.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.938+0000 7f89398a2640 1 -- 172.21.15.5:0/3841246501 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f892006be60 msgr2=0x7f892006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:54.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.938+0000 7f89398a2640 1 --2- 172.21.15.5:0/3841246501 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f892006be60 0x7f892006e370 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8924034000 tx=0x7f892402e000 comp rx=0 tx=0).stop 2024-05-20T18:28:54.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.938+0000 7f89398a2640 1 -- 172.21.15.5:0/3841246501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8934162820 msgr2=0x7f89340c8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:54.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.938+0000 7f89398a2640 1 --2- 172.21.15.5:0/3841246501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8934162820 0x7f89340c8280 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f892c00ccc0 tx=0x7f892c00a650 comp rx=0 tx=0).stop 2024-05-20T18:28:54.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.939+0000 7f89398a2640 1 -- 172.21.15.5:0/3841246501 shutdown_connections 2024-05-20T18:28:54.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.939+0000 7f89398a2640 1 --2- 172.21.15.5:0/3841246501 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f892006be60 0x7f892006e370 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:54.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.939+0000 7f89398a2640 1 --2- 172.21.15.5:0/3841246501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8934162820 0x7f89340c8280 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:54.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.939+0000 7f89398a2640 1 --2- 172.21.15.5:0/3841246501 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89340b3e80 0x7f8934150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:54.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.939+0000 7f89398a2640 1 --2- 172.21.15.5:0/3841246501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f893406d910 0x7f8934150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:54.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.939+0000 7f89398a2640 1 -- 172.21.15.5:0/3841246501 >> 172.21.15.5:0/3841246501 conn(0x7f89340d4bd0 msgr2=0x7f89340ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:54.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.939+0000 7f89398a2640 1 -- 172.21.15.5:0/3841246501 shutdown_connections 2024-05-20T18:28:54.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:54.939+0000 7f89398a2640 1 -- 172.21.15.5:0/3841246501 wait complete. 2024-05-20T18:28:54.950 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:28:54.951 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:28:55.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.069+0000 7fd0d4251640 1 Processor -- start 2024-05-20T18:28:55.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.069+0000 7fd0d4251640 1 -- start start 2024-05-20T18:28:55.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.070+0000 7fd0d4251640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd0cc0cd680 0x7fd0cc15f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:55.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.070+0000 7fd0d4251640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd0cc06c6c0 0x7fd0cc06cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:55.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.070+0000 7fd0d4251640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0cc0cf3d0 0x7fd0cc0cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:55.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.070+0000 7fd0d4251640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fd0cc0cda90 con 0x7fd0cc06c6c0 2024-05-20T18:28:55.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.070+0000 7fd0d4251640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd0cc15f760 con 0x7fd0cc0cd680 2024-05-20T18:28:55.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.070+0000 7fd0d4251640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd0cc0cfd20 con 0x7fd0cc0cf3d0 2024-05-20T18:28:55.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.070+0000 7fd0d17ec640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd0cc06c6c0 0x7fd0cc06cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:55.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.070+0000 7fd0d17ec640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd0cc06c6c0 0x7fd0cc06cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59594/0 (socket says 172.21.15.5:59594) 2024-05-20T18:28:55.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.070+0000 7fd0d17ec640 1 -- 172.21.15.5:0/1306444088 learned_addr learned my addr 172.21.15.5:0/1306444088 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:55.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.071+0000 7fd0d27ee640 1 --2- 172.21.15.5:0/1306444088 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0cc0cf3d0 0x7fd0cc0cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:55.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.071+0000 7fd0d1fed640 1 --2- 172.21.15.5:0/1306444088 >> v2:172.21.15.63:3300/0 conn(0x7fd0cc0cd680 0x7fd0cc15f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:55.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.071+0000 7fd0d17ec640 1 -- 172.21.15.5:0/1306444088 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0cc0cf3d0 msgr2=0x7fd0cc0cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:55.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.071+0000 7fd0d17ec640 1 --2- 172.21.15.5:0/1306444088 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0cc0cf3d0 0x7fd0cc0cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:55.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.071+0000 7fd0d17ec640 1 -- 172.21.15.5:0/1306444088 >> v2:172.21.15.63:3300/0 conn(0x7fd0cc0cd680 msgr2=0x7fd0cc15f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:55.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.071+0000 7fd0d17ec640 1 --2- 172.21.15.5:0/1306444088 >> v2:172.21.15.63:3300/0 conn(0x7fd0cc0cd680 0x7fd0cc15f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:55.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.071+0000 7fd0d17ec640 1 -- 172.21.15.5:0/1306444088 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0cc06d0f0 con 0x7fd0cc06c6c0 2024-05-20T18:28:55.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.071+0000 7fd0d17ec640 1 --2- 172.21.15.5:0/1306444088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0cc06c6c0 0x7fd0cc06cad0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc08baf0 tx=0x7fd0c002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be7cf57b01d5c4f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:55.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.072+0000 7fd0d0feb640 1 -- 172.21.15.5:0/1306444088 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c0037070 con 0x7fd0cc06c6c0 2024-05-20T18:28:55.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.072+0000 7fd0d0feb640 1 -- 172.21.15.5:0/1306444088 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0c0032440 con 0x7fd0cc06c6c0 2024-05-20T18:28:55.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.072+0000 7fd0d0feb640 1 -- 172.21.15.5:0/1306444088 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c0032810 con 0x7fd0cc06c6c0 2024-05-20T18:28:55.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.072+0000 7fd0d4251640 1 -- 172.21.15.5:0/1306444088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0cc06c6c0 msgr2=0x7fd0cc06cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:55.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.072+0000 7fd0d4251640 1 --2- 172.21.15.5:0/1306444088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0cc06c6c0 0x7fd0cc06cad0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc08baf0 tx=0x7fd0c002a140 comp rx=0 tx=0).stop 2024-05-20T18:28:55.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.072+0000 7fd0d4251640 1 -- 172.21.15.5:0/1306444088 shutdown_connections 2024-05-20T18:28:55.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.072+0000 7fd0d4251640 1 --2- 172.21.15.5:0/1306444088 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0cc0cf3d0 0x7fd0cc0cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:55.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.072+0000 7fd0d4251640 1 --2- 172.21.15.5:0/1306444088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0cc06c6c0 0x7fd0cc06cad0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:55.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.072+0000 7fd0d4251640 1 --2- 172.21.15.5:0/1306444088 >> v2:172.21.15.63:3300/0 conn(0x7fd0cc0cd680 0x7fd0cc15f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:55.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.072+0000 7fd0d4251640 1 -- 172.21.15.5:0/1306444088 >> 172.21.15.5:0/1306444088 conn(0x7fd0cc0b44d0 msgr2=0x7fd0cc0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:55.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.072+0000 7fd0d4251640 1 -- 172.21.15.5:0/1306444088 shutdown_connections 2024-05-20T18:28:55.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.073+0000 7fd0d4251640 1 -- 172.21.15.5:0/1306444088 wait complete. 2024-05-20T18:28:55.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.074+0000 7fd0d4251640 1 Processor -- start 2024-05-20T18:28:55.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.074+0000 7fd0d4251640 1 -- start start 2024-05-20T18:28:55.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.075+0000 7fd0d4251640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0cc06c6c0 0x7fd0cc161080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:55.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.076+0000 7fd0d4251640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0cc0cd680 0x7fd0cc151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:55.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.076+0000 7fd0d4251640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0cc0cf3d0 0x7fd0cc0c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:55.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.076+0000 7fd0d4251640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc0d0b40 con 0x7fd0cc06c6c0 2024-05-20T18:28:55.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.076+0000 7fd0d4251640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc06d450 con 0x7fd0cc0cf3d0 2024-05-20T18:28:55.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.076+0000 7fd0d4251640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd0cc06d990 con 0x7fd0cc0cd680 2024-05-20T18:28:55.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.076+0000 7fd0d1fed640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0cc06c6c0 0x7fd0cc161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:55.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.076+0000 7fd0d1fed640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0cc06c6c0 0x7fd0cc161080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59600/0 (socket says 172.21.15.5:59600) 2024-05-20T18:28:55.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.076+0000 7fd0d1fed640 1 -- 172.21.15.5:0/1045788303 learned_addr learned my addr 172.21.15.5:0/1045788303 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:55.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.076+0000 7fd0d27ee640 1 --2- 172.21.15.5:0/1045788303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0cc0cf3d0 0x7fd0cc0c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:55.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.076+0000 7fd0d17ec640 1 --2- 172.21.15.5:0/1045788303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0cc0cd680 0x7fd0cc151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:55.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.076+0000 7fd0d1fed640 1 -- 172.21.15.5:0/1045788303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0cc0cd680 msgr2=0x7fd0cc151f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:55.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.076+0000 7fd0d1fed640 1 --2- 172.21.15.5:0/1045788303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0cc0cd680 0x7fd0cc151f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:55.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.076+0000 7fd0d1fed640 1 -- 172.21.15.5:0/1045788303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0cc0cf3d0 msgr2=0x7fd0cc0c6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:55.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.076+0000 7fd0d1fed640 1 --2- 172.21.15.5:0/1045788303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0cc0cf3d0 0x7fd0cc0c6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:55.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.076+0000 7fd0d1fed640 1 -- 172.21.15.5:0/1045788303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0cc0c7400 con 0x7fd0cc06c6c0 2024-05-20T18:28:55.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.076+0000 7fd0d27ee640 1 --2- 172.21.15.5:0/1045788303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0cc0cf3d0 0x7fd0cc0c6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:28:55.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.076+0000 7fd0d1fed640 1 --2- 172.21.15.5:0/1045788303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0cc06c6c0 0x7fd0cc161080 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc007ec0 tx=0x7fd0bc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:55.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.076+0000 7fd0ca7fc640 1 -- 172.21.15.5:0/1045788303 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc01a070 con 0x7fd0cc06c6c0 2024-05-20T18:28:55.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.076+0000 7fd0ca7fc640 1 -- 172.21.15.5:0/1045788303 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0bc00ee90 con 0x7fd0cc06c6c0 2024-05-20T18:28:55.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.076+0000 7fd0d4251640 1 -- 172.21.15.5:0/1045788303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0cc0c76d0 con 0x7fd0cc06c6c0 2024-05-20T18:28:55.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.076+0000 7fd0ca7fc640 1 -- 172.21.15.5:0/1045788303 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc003db0 con 0x7fd0cc06c6c0 2024-05-20T18:28:55.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.076+0000 7fd0d4251640 1 -- 172.21.15.5:0/1045788303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0cc0c7c90 con 0x7fd0cc06c6c0 2024-05-20T18:28:55.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.077+0000 7fd0d4251640 1 -- 172.21.15.5:0/1045788303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0cc075b00 con 0x7fd0cc06c6c0 2024-05-20T18:28:55.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.078+0000 7fd0ca7fc640 1 -- 172.21.15.5:0/1045788303 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd0bc01d070 con 0x7fd0cc06c6c0 2024-05-20T18:28:55.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.078+0000 7fd0ca7fc640 1 --2- 172.21.15.5:0/1045788303 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd0b806be60 0x7fd0b806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:55.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.078+0000 7fd0ca7fc640 1 -- 172.21.15.5:0/1045788303 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7fd0bc08ae70 con 0x7fd0cc06c6c0 2024-05-20T18:28:55.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.079+0000 7fd0d17ec640 1 --2- 172.21.15.5:0/1045788303 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd0b806be60 0x7fd0b806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:55.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.079+0000 7fd0d17ec640 1 --2- 172.21.15.5:0/1045788303 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd0b806be60 0x7fd0b806e370 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0034000 tx=0x7fd0c002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:55.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.083+0000 7fd0ca7fc640 1 -- 172.21.15.5:0/1045788303 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd0bc059230 con 0x7fd0cc06c6c0 2024-05-20T18:28:55.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.265+0000 7fd0d4251640 1 -- 172.21.15.5:0/1045788303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd0cc0b8940 con 0x7fd0cc06c6c0 2024-05-20T18:28:55.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.265+0000 7fd0ca7fc640 1 -- 172.21.15.5:0/1045788303 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) ==== 93+0+10 (secure 0 0 0) 0x7fd0bc05d530 con 0x7fd0cc06c6c0 2024-05-20T18:28:55.265 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:28:55.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.267+0000 7fd0d4251640 1 -- 172.21.15.5:0/1045788303 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd0b806be60 msgr2=0x7fd0b806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:55.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.267+0000 7fd0d4251640 1 --2- 172.21.15.5:0/1045788303 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd0b806be60 0x7fd0b806e370 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0034000 tx=0x7fd0c002e000 comp rx=0 tx=0).stop 2024-05-20T18:28:55.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.267+0000 7fd0d4251640 1 -- 172.21.15.5:0/1045788303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0cc06c6c0 msgr2=0x7fd0cc161080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:55.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.267+0000 7fd0d4251640 1 --2- 172.21.15.5:0/1045788303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0cc06c6c0 0x7fd0cc161080 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc007ec0 tx=0x7fd0bc00e4b0 comp rx=0 tx=0).stop 2024-05-20T18:28:55.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.267+0000 7fd0d4251640 1 -- 172.21.15.5:0/1045788303 shutdown_connections 2024-05-20T18:28:55.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.267+0000 7fd0d4251640 1 --2- 172.21.15.5:0/1045788303 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd0b806be60 0x7fd0b806e370 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:55.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.267+0000 7fd0d4251640 1 --2- 172.21.15.5:0/1045788303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0cc0cf3d0 0x7fd0cc0c6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:55.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.267+0000 7fd0d4251640 1 --2- 172.21.15.5:0/1045788303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0cc0cd680 0x7fd0cc151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:55.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.267+0000 7fd0d4251640 1 --2- 172.21.15.5:0/1045788303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0cc06c6c0 0x7fd0cc161080 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:55.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.267+0000 7fd0d4251640 1 -- 172.21.15.5:0/1045788303 >> 172.21.15.5:0/1045788303 conn(0x7fd0cc0b44d0 msgr2=0x7fd0cc0cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:55.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.267+0000 7fd0d4251640 1 -- 172.21.15.5:0/1045788303 shutdown_connections 2024-05-20T18:28:55.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.267+0000 7fd0d4251640 1 -- 172.21.15.5:0/1045788303 wait complete. 2024-05-20T18:28:55.278 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-05-20T18:28:55.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.394+0000 7f04e8f9d640 1 Processor -- start 2024-05-20T18:28:55.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.394+0000 7f04e8f9d640 1 -- start start 2024-05-20T18:28:55.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.395+0000 7f04e8f9d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f04e415d530 0x7f04e415aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:55.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.395+0000 7f04e8f9d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04e4070850 0x7f04e4070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:55.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.395+0000 7f04e8f9d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04e40cd580 con 0x7f04e40b1700 2024-05-20T18:28:55.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.395+0000 7f04e8f9d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f04e40ce190 con 0x7f04e415d530 2024-05-20T18:28:55.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.395+0000 7f04e8f9d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f04e40cd860 con 0x7f04e4070850 2024-05-20T18:28:55.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.395+0000 7f04e1d9b640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f04e40b1700 0x7f04e40b1b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38628/0 (socket says 172.21.15.5:38628) 2024-05-20T18:28:55.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.395+0000 7f04e1d9b640 1 -- 172.21.15.5:0/3393018664 learned_addr learned my addr 172.21.15.5:0/3393018664 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:55.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.395+0000 7f04e2d9d640 1 --2- 172.21.15.5:0/3393018664 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04e4070850 0x7f04e4070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:55.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.395+0000 7f04e259c640 1 --2- 172.21.15.5:0/3393018664 >> v2:172.21.15.63:3300/0 conn(0x7f04e415d530 0x7f04e415aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:55.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.396+0000 7f04e159a640 1 -- 172.21.15.5:0/3393018664 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1580967985 0 0) 0x7f04e40cd580 con 0x7f04e40b1700 2024-05-20T18:28:55.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.396+0000 7f04e159a640 1 -- 172.21.15.5:0/3393018664 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04d0003520 con 0x7f04e40b1700 2024-05-20T18:28:55.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.396+0000 7f04e2d9d640 1 -- 172.21.15.5:0/3393018664 >> v2:172.21.15.63:3300/0 conn(0x7f04e415d530 msgr2=0x7f04e415aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:55.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.396+0000 7f04e2d9d640 1 --2- 172.21.15.5:0/3393018664 >> v2:172.21.15.63:3300/0 conn(0x7f04e415d530 0x7f04e415aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:55.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.396+0000 7f04e2d9d640 1 -- 172.21.15.5:0/3393018664 >> v1:172.21.15.5:6789/0 conn(0x7f04e40b1700 legacy=0x7f04e40b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:55.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.396+0000 7f04e2d9d640 1 -- 172.21.15.5:0/3393018664 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e40cdb40 con 0x7f04e4070850 2024-05-20T18:28:55.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.396+0000 7f04e259c640 1 --2- 172.21.15.5:0/3393018664 >> v2:172.21.15.63:3300/0 conn(0x7f04e415d530 0x7f04e415aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:28:55.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.397+0000 7f04e2d9d640 1 --2- 172.21.15.5:0/3393018664 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04e4070850 0x7f04e4070c60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f04dc009d90 tx=0x7f04dc02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4328bb0ec17fc92d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:55.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.397+0000 7f04e159a640 1 -- 172.21.15.5:0/3393018664 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc007740 con 0x7f04e4070850 2024-05-20T18:28:55.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.397+0000 7f04e159a640 1 -- 172.21.15.5:0/3393018664 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04dc0078e0 con 0x7f04e4070850 2024-05-20T18:28:55.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.397+0000 7f04e159a640 1 -- 172.21.15.5:0/3393018664 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc007cb0 con 0x7f04e4070850 2024-05-20T18:28:55.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.398+0000 7f04e8f9d640 1 -- 172.21.15.5:0/3393018664 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04e4070850 msgr2=0x7f04e4070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:55.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.398+0000 7f04e8f9d640 1 --2- 172.21.15.5:0/3393018664 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04e4070850 0x7f04e4070c60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f04dc009d90 tx=0x7f04dc02a660 comp rx=0 tx=0).stop 2024-05-20T18:28:55.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.398+0000 7f04e8f9d640 1 -- 172.21.15.5:0/3393018664 shutdown_connections 2024-05-20T18:28:55.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.398+0000 7f04e8f9d640 1 --2- 172.21.15.5:0/3393018664 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04e4070850 0x7f04e4070c60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:55.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.398+0000 7f04e8f9d640 1 --2- 172.21.15.5:0/3393018664 >> v2:172.21.15.63:3300/0 conn(0x7f04e415d530 0x7f04e415aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:55.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.398+0000 7f04e8f9d640 1 -- 172.21.15.5:0/3393018664 >> 172.21.15.5:0/3393018664 conn(0x7f04e40d29d0 msgr2=0x7f04e40d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:55.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.398+0000 7f04e8f9d640 1 -- 172.21.15.5:0/3393018664 shutdown_connections 2024-05-20T18:28:55.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.398+0000 7f04e8f9d640 1 -- 172.21.15.5:0/3393018664 wait complete. 2024-05-20T18:28:55.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.400+0000 7f04e8f9d640 1 Processor -- start 2024-05-20T18:28:55.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.400+0000 7f04e8f9d640 1 -- start start 2024-05-20T18:28:55.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.401+0000 7f04e8f9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04e4070850 0x7f04e40b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:55.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.401+0000 7f04e8f9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e40b1700 0x7f04e41f6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:55.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.401+0000 7f04e8f9d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04e415d530 0x7f04e41f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:55.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.401+0000 7f04e8f9d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f04e40711a0 con 0x7f04e40b1700 2024-05-20T18:28:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.401+0000 7f04e8f9d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f04e40cd230 con 0x7f04e4070850 2024-05-20T18:28:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.401+0000 7f04e8f9d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f04e4071320 con 0x7f04e415d530 2024-05-20T18:28:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.401+0000 7f04e1d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e40b1700 0x7f04e41f6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.401+0000 7f04e1d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e40b1700 0x7f04e41f6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59612/0 (socket says 172.21.15.5:59612) 2024-05-20T18:28:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.401+0000 7f04e1d9b640 1 -- 172.21.15.5:0/1055276388 learned_addr learned my addr 172.21.15.5:0/1055276388 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.401+0000 7f04e259c640 1 --2- 172.21.15.5:0/1055276388 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04e4070850 0x7f04e40b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.401+0000 7f04e2d9d640 1 --2- 172.21.15.5:0/1055276388 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04e415d530 0x7f04e41f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.401+0000 7f04e1d9b640 1 -- 172.21.15.5:0/1055276388 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04e415d530 msgr2=0x7f04e41f73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.401+0000 7f04e1d9b640 1 --2- 172.21.15.5:0/1055276388 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04e415d530 0x7f04e41f73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.401+0000 7f04e1d9b640 1 -- 172.21.15.5:0/1055276388 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04e4070850 msgr2=0x7f04e40b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.401+0000 7f04e1d9b640 1 --2- 172.21.15.5:0/1055276388 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04e4070850 0x7f04e40b10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.401+0000 7f04e1d9b640 1 -- 172.21.15.5:0/1055276388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e40c6ce0 con 0x7f04e40b1700 2024-05-20T18:28:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.402+0000 7f04e1d9b640 1 --2- 172.21.15.5:0/1055276388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e40b1700 0x7f04e41f6e60 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00c390 tx=0x7f04cc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:55.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.402+0000 7f04daffd640 1 -- 172.21.15.5:0/1055276388 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc017070 con 0x7f04e40b1700 2024-05-20T18:28:55.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.402+0000 7f04daffd640 1 -- 172.21.15.5:0/1055276388 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04cc005ce0 con 0x7f04e40b1700 2024-05-20T18:28:55.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.402+0000 7f04e8f9d640 1 -- 172.21.15.5:0/1055276388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e40c6fc0 con 0x7f04e40b1700 2024-05-20T18:28:55.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.402+0000 7f04daffd640 1 -- 172.21.15.5:0/1055276388 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc0072f0 con 0x7f04e40b1700 2024-05-20T18:28:55.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.402+0000 7f04e8f9d640 1 -- 172.21.15.5:0/1055276388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e40c7580 con 0x7f04e40b1700 2024-05-20T18:28:55.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.403+0000 7f04e8f9d640 1 -- 172.21.15.5:0/1055276388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e40c6a50 con 0x7f04e40b1700 2024-05-20T18:28:55.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.404+0000 7f04daffd640 1 -- 172.21.15.5:0/1055276388 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f04cc01a070 con 0x7f04e40b1700 2024-05-20T18:28:55.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.404+0000 7f04daffd640 1 --2- 172.21.15.5:0/1055276388 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f04c406be30 0x7f04c406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:55.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.404+0000 7f04daffd640 1 -- 172.21.15.5:0/1055276388 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f04cc08a0f0 con 0x7f04e40b1700 2024-05-20T18:28:55.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.404+0000 7f04e259c640 1 --2- 172.21.15.5:0/1055276388 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f04c406be30 0x7f04c406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:55.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.405+0000 7f04e259c640 1 --2- 172.21.15.5:0/1055276388 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f04c406be30 0x7f04c406e340 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f04e40d29a0 tx=0x7f04d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:55.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.409+0000 7f04daffd640 1 -- 172.21.15.5:0/1055276388 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f04cc058c10 con 0x7f04e40b1700 2024-05-20T18:28:55.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:55.668+0000 7f04e8f9d640 1 -- 172.21.15.5:0/1055276388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7f04e40d1860 con 0x7f04e40b1700 2024-05-20T18:28:56.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.613+0000 7f04daffd640 1 -- 172.21.15.5:0/1055276388 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v116) ==== 152+0+0 (secure 0 0 0) 0x7f04e40d1860 con 0x7f04e40b1700 2024-05-20T18:28:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.616+0000 7f04e8f9d640 1 -- 172.21.15.5:0/1055276388 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f04c406be30 msgr2=0x7f04c406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.616+0000 7f04e8f9d640 1 --2- 172.21.15.5:0/1055276388 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f04c406be30 0x7f04c406e340 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f04e40d29a0 tx=0x7f04d4008040 comp rx=0 tx=0).stop 2024-05-20T18:28:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.616+0000 7f04e8f9d640 1 -- 172.21.15.5:0/1055276388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e40b1700 msgr2=0x7f04e41f6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.616+0000 7f04e8f9d640 1 --2- 172.21.15.5:0/1055276388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e40b1700 0x7f04e41f6e60 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00c390 tx=0x7f04cc00c850 comp rx=0 tx=0).stop 2024-05-20T18:28:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.616+0000 7f04e8f9d640 1 -- 172.21.15.5:0/1055276388 shutdown_connections 2024-05-20T18:28:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.616+0000 7f04e8f9d640 1 --2- 172.21.15.5:0/1055276388 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f04c406be30 0x7f04c406e340 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.616+0000 7f04e8f9d640 1 --2- 172.21.15.5:0/1055276388 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04e415d530 0x7f04e41f73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.616+0000 7f04e8f9d640 1 --2- 172.21.15.5:0/1055276388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e40b1700 0x7f04e41f6e60 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.616+0000 7f04e8f9d640 1 --2- 172.21.15.5:0/1055276388 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04e4070850 0x7f04e40b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.616+0000 7f04e8f9d640 1 -- 172.21.15.5:0/1055276388 >> 172.21.15.5:0/1055276388 conn(0x7f04e40d29d0 msgr2=0x7f04e40b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.616+0000 7f04e8f9d640 1 -- 172.21.15.5:0/1055276388 shutdown_connections 2024-05-20T18:28:56.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.616+0000 7f04e8f9d640 1 -- 172.21.15.5:0/1055276388 wait complete. 2024-05-20T18:28:56.630 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-20T18:28:56.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.755+0000 7f631cd7e640 1 Processor -- start 2024-05-20T18:28:56.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.756+0000 7f631cd7e640 1 -- start start 2024-05-20T18:28:56.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.757+0000 7f631cd7e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6318162820 0x7f63180b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:56.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.757+0000 7f631cd7e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f63180b3e80 0x7f631806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:56.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.757+0000 7f631cd7e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f631806d910 0x7f63180d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:56.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.757+0000 7f631cd7e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f631806dd20 con 0x7f63180b3e80 2024-05-20T18:28:56.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.757+0000 7f631cd7e640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f63180d0630 con 0x7f6318162820 2024-05-20T18:28:56.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.757+0000 7f631cd7e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f63180d39c0 con 0x7f631806d910 2024-05-20T18:28:56.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.757+0000 7f6315d9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f63180b3e80 0x7f631806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:56.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.757+0000 7f6315d9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f63180b3e80 0x7f631806d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59616/0 (socket says 172.21.15.5:59616) 2024-05-20T18:28:56.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.757+0000 7f6315d9b640 1 -- 172.21.15.5:0/784221887 learned_addr learned my addr 172.21.15.5:0/784221887 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:56.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.758+0000 7f631659c640 1 --2- 172.21.15.5:0/784221887 >> v2:172.21.15.63:3300/0 conn(0x7f6318162820 0x7f63180b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:56.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.758+0000 7f6316d9d640 1 --2- 172.21.15.5:0/784221887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f631806d910 0x7f63180d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:56.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.758+0000 7f6315d9b640 1 -- 172.21.15.5:0/784221887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f631806d910 msgr2=0x7f63180d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:56.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.758+0000 7f6315d9b640 1 --2- 172.21.15.5:0/784221887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f631806d910 0x7f63180d27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:56.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.758+0000 7f6315d9b640 1 -- 172.21.15.5:0/784221887 >> v2:172.21.15.63:3300/0 conn(0x7f6318162820 msgr2=0x7f63180b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.758+0000 7f6315d9b640 1 --2- 172.21.15.5:0/784221887 >> v2:172.21.15.63:3300/0 conn(0x7f6318162820 0x7f63180b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.758+0000 7f6315d9b640 1 -- 172.21.15.5:0/784221887 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63180d3ca0 con 0x7f63180b3e80 2024-05-20T18:28:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.758+0000 7f6315d9b640 1 --2- 172.21.15.5:0/784221887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63180b3e80 0x7f631806d3d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f63180b7740 tx=0x7f630002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d40663ee90b59a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.759+0000 7f631559a640 1 -- 172.21.15.5:0/784221887 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6300037070 con 0x7f63180b3e80 2024-05-20T18:28:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.759+0000 7f631559a640 1 -- 172.21.15.5:0/784221887 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6300032440 con 0x7f63180b3e80 2024-05-20T18:28:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.759+0000 7f631559a640 1 -- 172.21.15.5:0/784221887 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6300032810 con 0x7f63180b3e80 2024-05-20T18:28:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.759+0000 7f631cd7e640 1 -- 172.21.15.5:0/784221887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63180b3e80 msgr2=0x7f631806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.759+0000 7f631cd7e640 1 --2- 172.21.15.5:0/784221887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63180b3e80 0x7f631806d3d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f63180b7740 tx=0x7f630002a1e0 comp rx=0 tx=0).stop 2024-05-20T18:28:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.759+0000 7f631cd7e640 1 -- 172.21.15.5:0/784221887 shutdown_connections 2024-05-20T18:28:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.759+0000 7f631cd7e640 1 --2- 172.21.15.5:0/784221887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f631806d910 0x7f63180d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.759+0000 7f631cd7e640 1 --2- 172.21.15.5:0/784221887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63180b3e80 0x7f631806d3d0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.759+0000 7f631cd7e640 1 --2- 172.21.15.5:0/784221887 >> v2:172.21.15.63:3300/0 conn(0x7f6318162820 0x7f63180b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:56.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.759+0000 7f631cd7e640 1 -- 172.21.15.5:0/784221887 >> 172.21.15.5:0/784221887 conn(0x7f63180d4bd0 msgr2=0x7f63180d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:56.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.759+0000 7f631cd7e640 1 -- 172.21.15.5:0/784221887 shutdown_connections 2024-05-20T18:28:56.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.759+0000 7f631cd7e640 1 -- 172.21.15.5:0/784221887 wait complete. 2024-05-20T18:28:56.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.761+0000 7f631cd7e640 1 Processor -- start 2024-05-20T18:28:56.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.761+0000 7f631cd7e640 1 -- start start 2024-05-20T18:28:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.763+0000 7f631cd7e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f631806d910 0x7f63181e7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.763+0000 7f631cd7e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63180b3e80 0x7f631814a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.763+0000 7f631cd7e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6318162820 0x7f631814cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.763+0000 7f631cd7e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f63180d0300 con 0x7f63180b3e80 2024-05-20T18:28:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.763+0000 7f631cd7e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f63180d4260 con 0x7f631806d910 2024-05-20T18:28:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.763+0000 7f631cd7e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f63180d4780 con 0x7f6318162820 2024-05-20T18:28:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.763+0000 7f6315d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63180b3e80 0x7f631814a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.763+0000 7f6315d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63180b3e80 0x7f631814a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59618/0 (socket says 172.21.15.5:59618) 2024-05-20T18:28:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.763+0000 7f6315d9b640 1 -- 172.21.15.5:0/2336078452 learned_addr learned my addr 172.21.15.5:0/2336078452 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.763+0000 7f631659c640 1 --2- 172.21.15.5:0/2336078452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f631806d910 0x7f63181e7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.763+0000 7f6316d9d640 1 --2- 172.21.15.5:0/2336078452 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6318162820 0x7f631814cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.763+0000 7f6315d9b640 1 -- 172.21.15.5:0/2336078452 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6318162820 msgr2=0x7f631814cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.763+0000 7f6315d9b640 1 --2- 172.21.15.5:0/2336078452 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6318162820 0x7f631814cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.763+0000 7f6315d9b640 1 -- 172.21.15.5:0/2336078452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f631806d910 msgr2=0x7f63181e7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.763+0000 7f6315d9b640 1 --2- 172.21.15.5:0/2336078452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f631806d910 0x7f63181e7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.763+0000 7f6315d9b640 1 -- 172.21.15.5:0/2336078452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f631814ac20 con 0x7f63180b3e80 2024-05-20T18:28:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.763+0000 7f6316d9d640 1 --2- 172.21.15.5:0/2336078452 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6318162820 0x7f631814cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.763+0000 7f6315d9b640 1 --2- 172.21.15.5:0/2336078452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63180b3e80 0x7f631814a6e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f630002a220 tx=0x7f63000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.763+0000 7f630effd640 1 -- 172.21.15.5:0/2336078452 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6300041020 con 0x7f63180b3e80 2024-05-20T18:28:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.764+0000 7f630effd640 1 -- 172.21.15.5:0/2336078452 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63000306c0 con 0x7f63180b3e80 2024-05-20T18:28:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.764+0000 7f631cd7e640 1 -- 172.21.15.5:0/2336078452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f631814b240 con 0x7f63180b3e80 2024-05-20T18:28:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.764+0000 7f630effd640 1 -- 172.21.15.5:0/2336078452 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6300037040 con 0x7f63180b3e80 2024-05-20T18:28:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.764+0000 7f631cd7e640 1 -- 172.21.15.5:0/2336078452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f631814b800 con 0x7f63180b3e80 2024-05-20T18:28:56.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.765+0000 7f631cd7e640 1 -- 172.21.15.5:0/2336078452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6318075b00 con 0x7f63180b3e80 2024-05-20T18:28:56.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.765+0000 7f630effd640 1 -- 172.21.15.5:0/2336078452 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6300044020 con 0x7f63180b3e80 2024-05-20T18:28:56.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.766+0000 7f630effd640 1 --2- 172.21.15.5:0/2336078452 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f62f806be60 0x7f62f806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:56.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.766+0000 7f630effd640 1 -- 172.21.15.5:0/2336078452 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f63000ab4c0 con 0x7f63180b3e80 2024-05-20T18:28:56.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.766+0000 7f631659c640 1 --2- 172.21.15.5:0/2336078452 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f62f806be60 0x7f62f806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:56.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.766+0000 7f631659c640 1 --2- 172.21.15.5:0/2336078452 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f62f806be60 0x7f62f806e370 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6308003aa0 tx=0x7f6308009830 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.770+0000 7f630effd640 1 -- 172.21.15.5:0/2336078452 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f630004b030 con 0x7f63180b3e80 2024-05-20T18:28:56.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.953+0000 7f631cd7e640 1 -- 172.21.15.5:0/2336078452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f63180cb6c0 con 0x7f63180b3e80 2024-05-20T18:28:56.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.957+0000 7f630effd640 1 -- 172.21.15.5:0/2336078452 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f630007ee20 con 0x7f63180b3e80 2024-05-20T18:28:56.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.959+0000 7f631cd7e640 1 -- 172.21.15.5:0/2336078452 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f62f806be60 msgr2=0x7f62f806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:56.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.959+0000 7f631cd7e640 1 --2- 172.21.15.5:0/2336078452 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f62f806be60 0x7f62f806e370 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6308003aa0 tx=0x7f6308009830 comp rx=0 tx=0).stop 2024-05-20T18:28:56.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.959+0000 7f631cd7e640 1 -- 172.21.15.5:0/2336078452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63180b3e80 msgr2=0x7f631814a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:56.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.959+0000 7f631cd7e640 1 --2- 172.21.15.5:0/2336078452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63180b3e80 0x7f631814a6e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f630002a220 tx=0x7f63000046c0 comp rx=0 tx=0).stop 2024-05-20T18:28:56.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.959+0000 7f631cd7e640 1 -- 172.21.15.5:0/2336078452 shutdown_connections 2024-05-20T18:28:56.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.959+0000 7f631cd7e640 1 --2- 172.21.15.5:0/2336078452 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f62f806be60 0x7f62f806e370 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:56.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.959+0000 7f631cd7e640 1 --2- 172.21.15.5:0/2336078452 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6318162820 0x7f631814cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:56.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.959+0000 7f631cd7e640 1 --2- 172.21.15.5:0/2336078452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63180b3e80 0x7f631814a6e0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:56.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.959+0000 7f631cd7e640 1 --2- 172.21.15.5:0/2336078452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f631806d910 0x7f63181e7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:56.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.959+0000 7f631cd7e640 1 -- 172.21.15.5:0/2336078452 >> 172.21.15.5:0/2336078452 conn(0x7f63180d4bd0 msgr2=0x7f63180b8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:56.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.959+0000 7f631cd7e640 1 -- 172.21.15.5:0/2336078452 shutdown_connections 2024-05-20T18:28:56.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:56.959+0000 7f631cd7e640 1 -- 172.21.15.5:0/2336078452 wait complete. 2024-05-20T18:28:56.970 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-20T18:28:56.970 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T18:28:57.017 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T18:28:57.062 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-20T18:28:57.062 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:28:57.062 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:28:57.079 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15663 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T18:28:57.079 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:28:57.079 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:28:57.121 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19732 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T18:28:57.122 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:28:57.122 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:28:57.138 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15663 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T18:28:57.138 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:28:57.139 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:28:57.180 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19732 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T18:28:57.181 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-20T18:28:57.181 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:28:57.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.296+0000 7f00f7056640 1 Processor -- start 2024-05-20T18:28:57.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.296+0000 7f00f7056640 1 -- start start 2024-05-20T18:28:57.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.297+0000 7f00f7056640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f00f00cf250 0x7f00f015f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:57.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.297+0000 7f00f7056640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00f00b6680 0x7f00f00b6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:57.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.297+0000 7f00f7056640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00f00b6fd0 con 0x7f00f0163020 2024-05-20T18:28:57.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.297+0000 7f00f7056640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f00f015f9a0 con 0x7f00f00cf250 2024-05-20T18:28:57.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.297+0000 7f00f7056640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f00f00d0b10 con 0x7f00f00b6680 2024-05-20T18:28:57.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.297+0000 7f00f4df2640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f00f00cf250 0x7f00f015f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:57.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.297+0000 7f00f55f3640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f00f0163020 0x7f00f0163430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38634/0 (socket says 172.21.15.5:38634) 2024-05-20T18:28:57.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.297+0000 7f00f55f3640 1 -- 172.21.15.5:0/4108306342 learned_addr learned my addr 172.21.15.5:0/4108306342 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:57.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.298+0000 7f00effff640 1 --2- 172.21.15.5:0/4108306342 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00f00b6680 0x7f00f00b6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:57.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.298+0000 7f00f4df2640 1 -- 172.21.15.5:0/4108306342 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00f00b6680 msgr2=0x7f00f00b6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:57.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.298+0000 7f00f4df2640 1 --2- 172.21.15.5:0/4108306342 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00f00b6680 0x7f00f00b6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:57.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.298+0000 7f00f4df2640 1 -- 172.21.15.5:0/4108306342 >> v1:172.21.15.5:6789/0 conn(0x7f00f0163020 legacy=0x7f00f0163430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:57.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.298+0000 7f00ef7fe640 1 -- 172.21.15.5:0/4108306342 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 733968164 0 0) 0x7f00e8001110 con 0x7f00f0163020 2024-05-20T18:28:57.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.298+0000 7f00f4df2640 1 -- 172.21.15.5:0/4108306342 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f00d0550 con 0x7f00f00cf250 2024-05-20T18:28:57.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.299+0000 7f00f4df2640 1 --2- 172.21.15.5:0/4108306342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00f00cf250 0x7f00f015f460 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f00e0009d90 tx=0x7f00e002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294ad8943c77046f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:57.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.299+0000 7f00ef7fe640 1 -- 172.21.15.5:0/4108306342 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0007740 con 0x7f00f00cf250 2024-05-20T18:28:57.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.299+0000 7f00ef7fe640 1 -- 172.21.15.5:0/4108306342 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00e00078e0 con 0x7f00f00cf250 2024-05-20T18:28:57.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.299+0000 7f00ef7fe640 1 -- 172.21.15.5:0/4108306342 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0007cb0 con 0x7f00f00cf250 2024-05-20T18:28:57.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.299+0000 7f00f7056640 1 -- 172.21.15.5:0/4108306342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00f00cf250 msgr2=0x7f00f015f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:57.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.299+0000 7f00f7056640 1 --2- 172.21.15.5:0/4108306342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00f00cf250 0x7f00f015f460 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f00e0009d90 tx=0x7f00e002a660 comp rx=0 tx=0).stop 2024-05-20T18:28:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.299+0000 7f00f7056640 1 -- 172.21.15.5:0/4108306342 shutdown_connections 2024-05-20T18:28:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.299+0000 7f00f7056640 1 --2- 172.21.15.5:0/4108306342 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00f00b6680 0x7f00f00b6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.299+0000 7f00f7056640 1 --2- 172.21.15.5:0/4108306342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00f00cf250 0x7f00f015f460 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.299+0000 7f00f7056640 1 -- 172.21.15.5:0/4108306342 >> 172.21.15.5:0/4108306342 conn(0x7f00f00d1860 msgr2=0x7f00f00d1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.300+0000 7f00f7056640 1 -- 172.21.15.5:0/4108306342 shutdown_connections 2024-05-20T18:28:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.300+0000 7f00f7056640 1 -- 172.21.15.5:0/4108306342 wait complete. 2024-05-20T18:28:57.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.301+0000 7f00f7056640 1 Processor -- start 2024-05-20T18:28:57.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.301+0000 7f00f7056640 1 -- start start 2024-05-20T18:28:57.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.303+0000 7f00f7056640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00f00b6680 0x7f00f00c8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.303+0000 7f00f7056640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00f00cf250 0x7f00f014a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.303+0000 7f00f7056640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00f0163020 0x7f00f01479a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.303+0000 7f00f7056640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f00f00d2a20 con 0x7f00f00b6680 2024-05-20T18:28:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.303+0000 7f00f7056640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f00f00d27d0 con 0x7f00f00cf250 2024-05-20T18:28:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.303+0000 7f00f7056640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f00f00d2d00 con 0x7f00f0163020 2024-05-20T18:28:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.303+0000 7f00f4df2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00f00b6680 0x7f00f00c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.303+0000 7f00f4df2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00f00b6680 0x7f00f00c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59626/0 (socket says 172.21.15.5:59626) 2024-05-20T18:28:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.303+0000 7f00f4df2640 1 -- 172.21.15.5:0/1601934078 learned_addr learned my addr 172.21.15.5:0/1601934078 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.303+0000 7f00f55f3640 1 --2- 172.21.15.5:0/1601934078 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00f0163020 0x7f00f01479a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.303+0000 7f00effff640 1 --2- 172.21.15.5:0/1601934078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00f00cf250 0x7f00f014a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.303+0000 7f00f4df2640 1 -- 172.21.15.5:0/1601934078 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00f0163020 msgr2=0x7f00f01479a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.303+0000 7f00f4df2640 1 --2- 172.21.15.5:0/1601934078 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00f0163020 0x7f00f01479a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:57.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.303+0000 7f00f4df2640 1 -- 172.21.15.5:0/1601934078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00f00cf250 msgr2=0x7f00f014a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:57.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.303+0000 7f00f4df2640 1 --2- 172.21.15.5:0/1601934078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00f00cf250 0x7f00f014a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:57.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.303+0000 7f00f4df2640 1 -- 172.21.15.5:0/1601934078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f00c8c50 con 0x7f00f00b6680 2024-05-20T18:28:57.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.303+0000 7f00f4df2640 1 --2- 172.21.15.5:0/1601934078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00f00b6680 0x7f00f00c8710 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f00e0009eb0 tx=0x7f00e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:57.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.304+0000 7f00ed7fa640 1 -- 172.21.15.5:0/1601934078 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e003f070 con 0x7f00f00b6680 2024-05-20T18:28:57.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.304+0000 7f00ed7fa640 1 -- 172.21.15.5:0/1601934078 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00e0031c40 con 0x7f00f00b6680 2024-05-20T18:28:57.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.304+0000 7f00ed7fa640 1 -- 172.21.15.5:0/1601934078 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0044460 con 0x7f00f00b6680 2024-05-20T18:28:57.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.304+0000 7f00f7056640 1 -- 172.21.15.5:0/1601934078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f0148670 con 0x7f00f00b6680 2024-05-20T18:28:57.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.304+0000 7f00f7056640 1 -- 172.21.15.5:0/1601934078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f0148c30 con 0x7f00f00b6680 2024-05-20T18:28:57.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.305+0000 7f00f7056640 1 -- 172.21.15.5:0/1601934078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00f0075b00 con 0x7f00f00b6680 2024-05-20T18:28:57.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.305+0000 7f00ed7fa640 1 -- 172.21.15.5:0/1601934078 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f00e003b410 con 0x7f00f00b6680 2024-05-20T18:28:57.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.306+0000 7f00ed7fa640 1 --2- 172.21.15.5:0/1601934078 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f00d006be30 0x7f00d006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:57.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.306+0000 7f00effff640 1 --2- 172.21.15.5:0/1601934078 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f00d006be30 0x7f00d006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:57.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.306+0000 7f00ed7fa640 1 -- 172.21.15.5:0/1601934078 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f00e00ac4e0 con 0x7f00f00b6680 2024-05-20T18:28:57.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.307+0000 7f00effff640 1 --2- 172.21.15.5:0/1601934078 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f00d006be30 0x7f00d006e340 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f00f00b3a90 tx=0x7f00d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:57.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.311+0000 7f00ed7fa640 1 -- 172.21.15.5:0/1601934078 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f00e007a920 con 0x7f00f00b6680 2024-05-20T18:28:57.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.539+0000 7f00f7056640 1 -- 172.21.15.5:0/1601934078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00f00d45d0 con 0x7f00f00b6680 2024-05-20T18:28:57.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.539+0000 7f00ed7fa640 1 -- 172.21.15.5:0/1601934078 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+3860 (secure 0 0 0) 0x7f00e007ec20 con 0x7f00f00b6680 2024-05-20T18:28:57.539 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:57.539 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1458170078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1458170078},{"type":"v1","addr":"172.21.15.63:6834","nonce":1458170078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4916,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/2601750663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":2601750663},{"type":"v1","addr":"172.21.15.63:6835","nonce":2601750663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5313,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/1913030219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1913030219},{"type":"v1","addr":"172.21.15.5:6836","nonce":1913030219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5679,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/143191778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":143191778},{"type":"v1","addr":"172.21.15.5:6837","nonce":143191778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[]} 2024-05-20T18:28:57.540 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 19 2024-05-20T18:28:57.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.541+0000 7f00f7056640 1 -- 172.21.15.5:0/1601934078 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f00d006be30 msgr2=0x7f00d006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.541+0000 7f00f7056640 1 --2- 172.21.15.5:0/1601934078 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f00d006be30 0x7f00d006e340 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f00f00b3a90 tx=0x7f00d8008040 comp rx=0 tx=0).stop 2024-05-20T18:28:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.542+0000 7f00f7056640 1 -- 172.21.15.5:0/1601934078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00f00b6680 msgr2=0x7f00f00c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.542+0000 7f00f7056640 1 --2- 172.21.15.5:0/1601934078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00f00b6680 0x7f00f00c8710 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f00e0009eb0 tx=0x7f00e0004a40 comp rx=0 tx=0).stop 2024-05-20T18:28:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.542+0000 7f00f7056640 1 -- 172.21.15.5:0/1601934078 shutdown_connections 2024-05-20T18:28:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.542+0000 7f00f7056640 1 --2- 172.21.15.5:0/1601934078 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f00d006be30 0x7f00d006e340 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.542+0000 7f00f7056640 1 --2- 172.21.15.5:0/1601934078 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00f0163020 0x7f00f01479a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.542+0000 7f00f7056640 1 --2- 172.21.15.5:0/1601934078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00f00cf250 0x7f00f014a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.542+0000 7f00f7056640 1 --2- 172.21.15.5:0/1601934078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00f00b6680 0x7f00f00c8710 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.542+0000 7f00f7056640 1 -- 172.21.15.5:0/1601934078 >> 172.21.15.5:0/1601934078 conn(0x7f00f00d1860 msgr2=0x7f00f015e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.542+0000 7f00f7056640 1 -- 172.21.15.5:0/1601934078 shutdown_connections 2024-05-20T18:28:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.542+0000 7f00f7056640 1 -- 172.21.15.5:0/1601934078 wait complete. 2024-05-20T18:28:57.553 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-20T18:28:57.553 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-20T18:28:57.553 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-20T18:28:57.553 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:28:57.554 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-20T18:28:57.554 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-20T18:28:57.554 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-20T18:28:57.554 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:28:57.554 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-20T18:28:57.554 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-20T18:28:57.555 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-20T18:28:57.555 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:28:57.555 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-20T18:28:57.555 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-20T18:28:57.555 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-20T18:28:57.555 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:28:57.562 INFO:tasks.ceph.mds.a:Stopped 2024-05-20T18:28:57.563 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-20T18:28:57.566 INFO:tasks.ceph.mds.c:Stopped 2024-05-20T18:28:57.566 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-20T18:28:57.567 INFO:tasks.ceph.mds.b:Stopped 2024-05-20T18:28:57.567 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-20T18:28:57.568 INFO:tasks.ceph.mds.d:Stopped 2024-05-20T18:28:57.568 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-20T18:28:57.608 INFO:tasks.ceph.mds.a:Started 2024-05-20T18:28:57.610 DEBUG:teuthology.parallel:result is None 2024-05-20T18:28:57.611 INFO:tasks.ceph.mds.c:Started 2024-05-20T18:28:57.611 DEBUG:teuthology.parallel:result is None 2024-05-20T18:28:57.612 INFO:tasks.ceph.mds.b:Started 2024-05-20T18:28:57.612 DEBUG:teuthology.parallel:result is None 2024-05-20T18:28:57.613 INFO:tasks.ceph.mds.d:Started 2024-05-20T18:28:57.613 DEBUG:teuthology.parallel:result is None 2024-05-20T18:28:57.613 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-20T18:28:57.699 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-05-20T18:28:57.704 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-05-20T18:28:57.711 INFO:tasks.ceph.mds.d.smithi063.stdout:starting mds.d at 2024-05-20T18:28:57.712 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-20T18:28:57.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.736+0000 7fed2a179640 1 Processor -- start 2024-05-20T18:28:57.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.736+0000 7fed2a179640 1 -- start start 2024-05-20T18:28:57.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.737+0000 7fed2a179640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fed2415d530 0x7fed2415aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:57.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.737+0000 7fed2a179640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed24070850 0x7fed24070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:57.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.737+0000 7fed2a179640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed240cd580 con 0x7fed240b1700 2024-05-20T18:28:57.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.737+0000 7fed2a179640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fed240ce190 con 0x7fed2415d530 2024-05-20T18:28:57.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.737+0000 7fed2a179640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fed240cd860 con 0x7fed24070850 2024-05-20T18:28:57.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.737+0000 7fed1affd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fed240b1700 0x7fed240b1b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38660/0 (socket says 172.21.15.5:38660) 2024-05-20T18:28:57.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.737+0000 7fed1affd640 1 -- 172.21.15.5:0/2272657953 learned_addr learned my addr 172.21.15.5:0/2272657953 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:57.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.737+0000 7fed237fe640 1 --2- 172.21.15.5:0/2272657953 >> v2:172.21.15.63:3300/0 conn(0x7fed2415d530 0x7fed2415aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:57.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.737+0000 7fed23fff640 1 --2- 172.21.15.5:0/2272657953 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed24070850 0x7fed24070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:57.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.737+0000 7fed22ffd640 1 -- 172.21.15.5:0/2272657953 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2624381683 0 0) 0x7fed240cd580 con 0x7fed240b1700 2024-05-20T18:28:57.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.738+0000 7fed22ffd640 1 -- 172.21.15.5:0/2272657953 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed10003520 con 0x7fed240b1700 2024-05-20T18:28:57.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.738+0000 7fed23fff640 1 -- 172.21.15.5:0/2272657953 >> v2:172.21.15.63:3300/0 conn(0x7fed2415d530 msgr2=0x7fed2415aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:57.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.738+0000 7fed23fff640 1 --2- 172.21.15.5:0/2272657953 >> v2:172.21.15.63:3300/0 conn(0x7fed2415d530 0x7fed2415aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:57.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.738+0000 7fed23fff640 1 -- 172.21.15.5:0/2272657953 >> v1:172.21.15.5:6789/0 conn(0x7fed240b1700 legacy=0x7fed240b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:57.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.738+0000 7fed23fff640 1 -- 172.21.15.5:0/2272657953 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed240cdb40 con 0x7fed24070850 2024-05-20T18:28:57.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.738+0000 7fed237fe640 1 --2- 172.21.15.5:0/2272657953 >> v2:172.21.15.63:3300/0 conn(0x7fed2415d530 0x7fed2415aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:28:57.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.738+0000 7fed23fff640 1 --2- 172.21.15.5:0/2272657953 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed24070850 0x7fed24070c60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fed1c009d90 tx=0x7fed1c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae4f2ab2e5beb0bb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:57.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.739+0000 7fed22ffd640 1 -- 172.21.15.5:0/2272657953 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c007740 con 0x7fed24070850 2024-05-20T18:28:57.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.739+0000 7fed22ffd640 1 -- 172.21.15.5:0/2272657953 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed1c0078e0 con 0x7fed24070850 2024-05-20T18:28:57.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.739+0000 7fed22ffd640 1 -- 172.21.15.5:0/2272657953 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c007cb0 con 0x7fed24070850 2024-05-20T18:28:57.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.739+0000 7fed2a179640 1 -- 172.21.15.5:0/2272657953 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed24070850 msgr2=0x7fed24070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:57.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.739+0000 7fed2a179640 1 --2- 172.21.15.5:0/2272657953 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed24070850 0x7fed24070c60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fed1c009d90 tx=0x7fed1c02a660 comp rx=0 tx=0).stop 2024-05-20T18:28:57.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.739+0000 7fed2a179640 1 -- 172.21.15.5:0/2272657953 shutdown_connections 2024-05-20T18:28:57.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.739+0000 7fed2a179640 1 --2- 172.21.15.5:0/2272657953 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed24070850 0x7fed24070c60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:57.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.739+0000 7fed2a179640 1 --2- 172.21.15.5:0/2272657953 >> v2:172.21.15.63:3300/0 conn(0x7fed2415d530 0x7fed2415aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:57.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.739+0000 7fed2a179640 1 -- 172.21.15.5:0/2272657953 >> 172.21.15.5:0/2272657953 conn(0x7fed240d29d0 msgr2=0x7fed240d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:57.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.739+0000 7fed2a179640 1 -- 172.21.15.5:0/2272657953 shutdown_connections 2024-05-20T18:28:57.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.739+0000 7fed2a179640 1 -- 172.21.15.5:0/2272657953 wait complete. 2024-05-20T18:28:57.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.741+0000 7fed2a179640 1 Processor -- start 2024-05-20T18:28:57.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.741+0000 7fed2a179640 1 -- start start 2024-05-20T18:28:57.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.743+0000 7fed2a179640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed24070850 0x7fed241f64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:57.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.743+0000 7fed2a179640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed240b1700 0x7fed241f2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:57.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.743+0000 7fed2a179640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed2415d530 0x7fed2406d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:57.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.743+0000 7fed2a179640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fed240711a0 con 0x7fed2415d530 2024-05-20T18:28:57.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.743+0000 7fed2a179640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fed240cd230 con 0x7fed24070850 2024-05-20T18:28:57.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.743+0000 7fed2a179640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fed24071320 con 0x7fed240b1700 2024-05-20T18:28:57.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.743+0000 7fed23fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed2415d530 0x7fed2406d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:57.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.743+0000 7fed23fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed2415d530 0x7fed2406d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59652/0 (socket says 172.21.15.5:59652) 2024-05-20T18:28:57.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.743+0000 7fed23fff640 1 -- 172.21.15.5:0/1703487964 learned_addr learned my addr 172.21.15.5:0/1703487964 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:57.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.743+0000 7fed237fe640 1 --2- 172.21.15.5:0/1703487964 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed24070850 0x7fed241f64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:57.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.743+0000 7fed1affd640 1 --2- 172.21.15.5:0/1703487964 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed240b1700 0x7fed241f2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:57.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.743+0000 7fed23fff640 1 -- 172.21.15.5:0/1703487964 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed240b1700 msgr2=0x7fed241f2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:57.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.743+0000 7fed23fff640 1 --2- 172.21.15.5:0/1703487964 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed240b1700 0x7fed241f2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:57.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.743+0000 7fed23fff640 1 -- 172.21.15.5:0/1703487964 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed24070850 msgr2=0x7fed241f64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:57.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.743+0000 7fed23fff640 1 --2- 172.21.15.5:0/1703487964 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed24070850 0x7fed241f64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:57.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.743+0000 7fed23fff640 1 -- 172.21.15.5:0/1703487964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed241f69f0 con 0x7fed2415d530 2024-05-20T18:28:57.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.744+0000 7fed23fff640 1 --2- 172.21.15.5:0/1703487964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed2415d530 0x7fed2406d970 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fed1c009eb0 tx=0x7fed1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:57.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.744+0000 7fed20ff9640 1 -- 172.21.15.5:0/1703487964 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c03f070 con 0x7fed2415d530 2024-05-20T18:28:57.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.744+0000 7fed20ff9640 1 -- 172.21.15.5:0/1703487964 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed1c031bc0 con 0x7fed2415d530 2024-05-20T18:28:57.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.744+0000 7fed2a179640 1 -- 172.21.15.5:0/1703487964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2406c8d0 con 0x7fed2415d530 2024-05-20T18:28:57.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.744+0000 7fed20ff9640 1 -- 172.21.15.5:0/1703487964 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c009450 con 0x7fed2415d530 2024-05-20T18:28:57.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.744+0000 7fed2a179640 1 -- 172.21.15.5:0/1703487964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed2406ce90 con 0x7fed2415d530 2024-05-20T18:28:57.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.745+0000 7fed2a179640 1 -- 172.21.15.5:0/1703487964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed240d1860 con 0x7fed2415d530 2024-05-20T18:28:57.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.745+0000 7fed20ff9640 1 -- 172.21.15.5:0/1703487964 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fed1c03b410 con 0x7fed2415d530 2024-05-20T18:28:57.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.746+0000 7fed20ff9640 1 --2- 172.21.15.5:0/1703487964 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fed0406be60 0x7fed0406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:57.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.746+0000 7fed20ff9640 1 -- 172.21.15.5:0/1703487964 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7fed1c0ac2b0 con 0x7fed2415d530 2024-05-20T18:28:57.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.746+0000 7fed237fe640 1 --2- 172.21.15.5:0/1703487964 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fed0406be60 0x7fed0406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:57.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.746+0000 7fed237fe640 1 --2- 172.21.15.5:0/1703487964 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fed0406be60 0x7fed0406e370 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fed14009560 tx=0x7fed14009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:57.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.750+0000 7fed20ff9640 1 -- 172.21.15.5:0/1703487964 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fed240d1860 con 0x7fed2415d530 2024-05-20T18:28:57.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:57.949+0000 7fed2a179640 1 -- 172.21.15.5:0/1703487964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fed24075b00 con 0x7fed2415d530 2024-05-20T18:28:58.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.628+0000 7fed20ff9640 1 -- 172.21.15.5:0/1703487964 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v45) ==== 97+0+0 (secure 0 0 0) 0x7fed1c0ac0d0 con 0x7fed2415d530 2024-05-20T18:28:58.628 INFO:teuthology.orchestra.run.smithi005.stderr: removed all blocklist entries 2024-05-20T18:28:58.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.631+0000 7fed2a179640 1 -- 172.21.15.5:0/1703487964 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fed0406be60 msgr2=0x7fed0406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:58.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.632+0000 7fed2a179640 1 --2- 172.21.15.5:0/1703487964 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fed0406be60 0x7fed0406e370 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fed14009560 tx=0x7fed14009190 comp rx=0 tx=0).stop 2024-05-20T18:28:58.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.632+0000 7fed2a179640 1 -- 172.21.15.5:0/1703487964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed2415d530 msgr2=0x7fed2406d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:58.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.632+0000 7fed2a179640 1 --2- 172.21.15.5:0/1703487964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed2415d530 0x7fed2406d970 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fed1c009eb0 tx=0x7fed1c004a40 comp rx=0 tx=0).stop 2024-05-20T18:28:58.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.632+0000 7fed2a179640 1 -- 172.21.15.5:0/1703487964 shutdown_connections 2024-05-20T18:28:58.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.632+0000 7fed2a179640 1 --2- 172.21.15.5:0/1703487964 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fed0406be60 0x7fed0406e370 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:58.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.632+0000 7fed2a179640 1 --2- 172.21.15.5:0/1703487964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed2415d530 0x7fed2406d970 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:58.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.632+0000 7fed2a179640 1 --2- 172.21.15.5:0/1703487964 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fed240b1700 0x7fed241f2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:58.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.632+0000 7fed2a179640 1 --2- 172.21.15.5:0/1703487964 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed24070850 0x7fed241f64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:58.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.632+0000 7fed2a179640 1 -- 172.21.15.5:0/1703487964 >> 172.21.15.5:0/1703487964 conn(0x7fed240d29d0 msgr2=0x7fed240b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:58.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.632+0000 7fed2a179640 1 -- 172.21.15.5:0/1703487964 shutdown_connections 2024-05-20T18:28:58.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.632+0000 7fed2a179640 1 -- 172.21.15.5:0/1703487964 wait complete. 2024-05-20T18:28:58.646 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-20T18:28:58.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.765+0000 7fb1893cc640 1 Processor -- start 2024-05-20T18:28:58.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.765+0000 7fb1893cc640 1 -- start start 2024-05-20T18:28:58.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.767+0000 7fb1893cc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb184162820 0x7fb1840b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:58.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.767+0000 7fb1893cc640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb1840b3e80 0x7fb18406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:58.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.767+0000 7fb1893cc640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb18406dd20 con 0x7fb18406d910 2024-05-20T18:28:58.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.767+0000 7fb1893cc640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fb1840d0630 con 0x7fb184162820 2024-05-20T18:28:58.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.767+0000 7fb1893cc640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb1840d39c0 con 0x7fb1840b3e80 2024-05-20T18:28:58.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.767+0000 7fb182ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb184162820 0x7fb1840b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:58.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.767+0000 7fb182ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb184162820 0x7fb1840b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59662/0 (socket says 172.21.15.5:59662) 2024-05-20T18:28:58.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.767+0000 7fb182ffd640 1 -- 172.21.15.5:0/1530477225 learned_addr learned my addr 172.21.15.5:0/1530477225 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:58.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.767+0000 7fb182ffd640 1 -- 172.21.15.5:0/1530477225 >> v2:172.21.15.63:3300/0 conn(0x7fb1840b3e80 msgr2=0x7fb18406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:58.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.767+0000 7fb1827fc640 1 --2- 172.21.15.5:0/1530477225 >> v2:172.21.15.63:3300/0 conn(0x7fb1840b3e80 0x7fb18406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:58.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.767+0000 7fb182ffd640 1 --2- 172.21.15.5:0/1530477225 >> v2:172.21.15.63:3300/0 conn(0x7fb1840b3e80 0x7fb18406d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:58.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.767+0000 7fb182ffd640 1 -- 172.21.15.5:0/1530477225 >> v1:172.21.15.63:6789/0 conn(0x7fb18406d910 legacy=0x7fb1840d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:58.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.767+0000 7fb182ffd640 1 -- 172.21.15.5:0/1530477225 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1840d3ca0 con 0x7fb184162820 2024-05-20T18:28:58.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.768+0000 7fb182ffd640 1 --2- 172.21.15.5:0/1530477225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb184162820 0x7fb1840b3940 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb1840d1230 tx=0x7fb17402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42644e1884f66748 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:58.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.768+0000 7fb181ffb640 1 -- 172.21.15.5:0/1530477225 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb174036070 con 0x7fb184162820 2024-05-20T18:28:58.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.768+0000 7fb181ffb640 1 -- 172.21.15.5:0/1530477225 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb174031600 con 0x7fb184162820 2024-05-20T18:28:58.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.768+0000 7fb181ffb640 1 -- 172.21.15.5:0/1530477225 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1740319d0 con 0x7fb184162820 2024-05-20T18:28:58.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.769+0000 7fb1893cc640 1 -- 172.21.15.5:0/1530477225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb184162820 msgr2=0x7fb1840b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:58.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.769+0000 7fb1893cc640 1 --2- 172.21.15.5:0/1530477225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb184162820 0x7fb1840b3940 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb1840d1230 tx=0x7fb17402a6f0 comp rx=0 tx=0).stop 2024-05-20T18:28:58.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.769+0000 7fb1893cc640 1 -- 172.21.15.5:0/1530477225 shutdown_connections 2024-05-20T18:28:58.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.769+0000 7fb1893cc640 1 --2- 172.21.15.5:0/1530477225 >> v2:172.21.15.63:3300/0 conn(0x7fb1840b3e80 0x7fb18406d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:58.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.769+0000 7fb1893cc640 1 --2- 172.21.15.5:0/1530477225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb184162820 0x7fb1840b3940 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:58.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.769+0000 7fb1893cc640 1 -- 172.21.15.5:0/1530477225 >> 172.21.15.5:0/1530477225 conn(0x7fb1840d4bd0 msgr2=0x7fb1840d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:58.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.769+0000 7fb1893cc640 1 -- 172.21.15.5:0/1530477225 shutdown_connections 2024-05-20T18:28:58.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.769+0000 7fb1893cc640 1 -- 172.21.15.5:0/1530477225 wait complete. 2024-05-20T18:28:58.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.771+0000 7fb1893cc640 1 Processor -- start 2024-05-20T18:28:58.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.771+0000 7fb1893cc640 1 -- start start 2024-05-20T18:28:58.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.772+0000 7fb1893cc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb18406d910 0x7fb18414f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:58.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.772+0000 7fb1893cc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1840b3e80 0x7fb1840b6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:58.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.772+0000 7fb1893cc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb184162820 0x7fb1841f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:58.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.772+0000 7fb1893cc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb1840d0300 con 0x7fb184162820 2024-05-20T18:28:58.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.772+0000 7fb1893cc640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb1840d4260 con 0x7fb1840b3e80 2024-05-20T18:28:58.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.772+0000 7fb1893cc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb1840b91a0 con 0x7fb18406d910 2024-05-20T18:28:58.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.772+0000 7fb1837fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb184162820 0x7fb1841f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:58.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.772+0000 7fb1837fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb184162820 0x7fb1841f51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59678/0 (socket says 172.21.15.5:59678) 2024-05-20T18:28:58.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.772+0000 7fb1837fe640 1 -- 172.21.15.5:0/803600269 learned_addr learned my addr 172.21.15.5:0/803600269 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:58.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.772+0000 7fb1827fc640 1 --2- 172.21.15.5:0/803600269 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1840b3e80 0x7fb1840b6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:58.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.772+0000 7fb182ffd640 1 --2- 172.21.15.5:0/803600269 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb18406d910 0x7fb18414f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:58.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.773+0000 7fb1837fe640 1 -- 172.21.15.5:0/803600269 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb18406d910 msgr2=0x7fb18414f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:58.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.773+0000 7fb1837fe640 1 --2- 172.21.15.5:0/803600269 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb18406d910 0x7fb18414f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:58.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.773+0000 7fb1837fe640 1 -- 172.21.15.5:0/803600269 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1840b3e80 msgr2=0x7fb1840b6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:58.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.773+0000 7fb1837fe640 1 --2- 172.21.15.5:0/803600269 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1840b3e80 0x7fb1840b6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:58.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.773+0000 7fb1837fe640 1 -- 172.21.15.5:0/803600269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1840cb4e0 con 0x7fb184162820 2024-05-20T18:28:58.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.773+0000 7fb1837fe640 1 --2- 172.21.15.5:0/803600269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb184162820 0x7fb1841f51b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb17c00b4c0 tx=0x7fb17c00b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:58.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.773+0000 7fb16b7fe640 1 -- 172.21.15.5:0/803600269 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c015020 con 0x7fb184162820 2024-05-20T18:28:58.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.773+0000 7fb16b7fe640 1 -- 172.21.15.5:0/803600269 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb17c011850 con 0x7fb184162820 2024-05-20T18:28:58.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.773+0000 7fb1893cc640 1 -- 172.21.15.5:0/803600269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1840b6270 con 0x7fb184162820 2024-05-20T18:28:58.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.773+0000 7fb16b7fe640 1 -- 172.21.15.5:0/803600269 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c011c40 con 0x7fb184162820 2024-05-20T18:28:58.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.773+0000 7fb1893cc640 1 -- 172.21.15.5:0/803600269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1841f5ce0 con 0x7fb184162820 2024-05-20T18:28:58.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.774+0000 7fb1893cc640 1 -- 172.21.15.5:0/803600269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1841f5f40 con 0x7fb184162820 2024-05-20T18:28:58.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.775+0000 7fb16b7fe640 1 -- 172.21.15.5:0/803600269 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb17c018020 con 0x7fb184162820 2024-05-20T18:28:58.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.775+0000 7fb16b7fe640 1 --2- 172.21.15.5:0/803600269 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb17006be30 0x7fb17006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:58.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.776+0000 7fb16b7fe640 1 -- 172.21.15.5:0/803600269 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5249+0+0 (secure 0 0 0) 0x7fb17c08ac20 con 0x7fb184162820 2024-05-20T18:28:58.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.776+0000 7fb182ffd640 1 --2- 172.21.15.5:0/803600269 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb17006be30 0x7fb17006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:58.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.776+0000 7fb182ffd640 1 --2- 172.21.15.5:0/803600269 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb17006be30 0x7fb17006e340 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb174033000 tx=0x7fb174009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:58.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:58.780+0000 7fb16b7fe640 1 -- 172.21.15.5:0/803600269 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb17c01f020 con 0x7fb184162820 2024-05-20T18:28:59.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.037+0000 7fb1893cc640 1 -- 172.21.15.5:0/803600269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fb1840cb270 con 0x7fb184162820 2024-05-20T18:28:59.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.037+0000 7fb16b7fe640 1 -- 172.21.15.5:0/803600269 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7fb17c058f90 con 0x7fb184162820 2024-05-20T18:28:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:59.037 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:28:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_dump": [ 2024-05-20T18:28:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.a", 2024-05-20T18:28:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtm9lSeCBAA/PpiEqojk6ueJo60tU4/iA==", 2024-05-20T18:28:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:28:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:28:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:28:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:28:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.b", 2024-05-20T18:28:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmdov0FRAACkWAR/0zaCDjgnkHnoHwgw==", 2024-05-20T18:28:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:28:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:28:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:28:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:28:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.c", 2024-05-20T18:28:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmpEBlEBAAdgxDXj9x38VSOizUgdFpYA==", 2024-05-20T18:28:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:28:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:28:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:28:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:28:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.d", 2024-05-20T18:28:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtm0Gy9HRAAAkPqcBLwxTwuWmCi7aad1A==", 2024-05-20T18:28:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:28:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:28:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:28:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:28:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.0", 2024-05-20T18:28:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDHlUtmWFGzBRAA81qN37dlHf6QDZ1FgwbJ9w==", 2024-05-20T18:28:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:28:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:28:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:28:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.1", 2024-05-20T18:28:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDKlUtmBMKsBBAAD4tTTvetME6oxwugTubSKQ==", 2024-05-20T18:28:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:28:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:28:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:28:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.2", 2024-05-20T18:28:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDNlUtm4Rm/AxAAIGWTUxb+LII09Rd+h5cJ8Q==", 2024-05-20T18:28:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:28:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:28:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:28:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.3", 2024-05-20T18:28:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDQlUtmWVxZAxAAB/w6k4WJ58ulEr9zatCRfw==", 2024-05-20T18:28:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:28:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:28:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:28:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.4", 2024-05-20T18:28:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDWlUtmIyokCRAAolOiYVCYaNTQZb1PjtR6fg==", 2024-05-20T18:28:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:28:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:28:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:28:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.5", 2024-05-20T18:28:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZlUtmSf6mCBAAw73nwq/zzt9kIXZymZh93A==", 2024-05-20T18:28:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:28:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:28:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:28:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.6", 2024-05-20T18:28:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDclUtmwwhGCBAA94OBEtj9h7cmCAddYMonaA==", 2024-05-20T18:28:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:28:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:28:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:28:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.7", 2024-05-20T18:28:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDflUtm9wefBxAAvCZTBzRu05koFhc+pH5I8w==", 2024-05-20T18:28:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:28:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:28:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:28:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.0", 2024-05-20T18:28:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmevaaIhAATIwyXzr0vGT38ZmcMqR2gA==", 2024-05-20T18:28:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:28:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow r", 2024-05-20T18:28:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-20T18:28:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.1", 2024-05-20T18:28:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmK7NfKhAADaXl0uVP1Q1gUOlPsstnzg==", 2024-05-20T18:28:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:28:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-20T18:28:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-20T18:28:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-20T18:28:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.2", 2024-05-20T18:28:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmMJF1JhAAeM+HseqvgvKgG8EnIsZuhg==", 2024-05-20T18:28:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:28:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-20T18:28:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-20T18:28:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-20T18:28:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.3", 2024-05-20T18:28:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmodA7LhAAUI2v0npB6HHs2n1g3KdTXA==", 2024-05-20T18:28:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:28:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-20T18:28:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-20T18:28:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-20T18:28:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.admin", 2024-05-20T18:28:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDElUtmZacGJRAAoyr9XNPBr9r0DSU53qCHSQ==", 2024-05-20T18:28:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-20T18:28:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:28:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:28:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:28:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.x", 2024-05-20T18:28:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmPgq0BBAAAljNeQDsSau0wH2Lh2KdBw==", 2024-05-20T18:28:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-20T18:28:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-05-20T18:28:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:28:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:28:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:28:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.y", 2024-05-20T18:28:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDElUtm6Xv3OhAA0doOWTC73jy5rOYm7Vf68Q==", 2024-05-20T18:28:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:28:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-20T18:28:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-05-20T18:28:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:28:59.051 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:59.051 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:28:59.051 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-20T18:28:59.051 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:28:59.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.039+0000 7fb1893cc640 1 -- 172.21.15.5:0/803600269 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb17006be30 msgr2=0x7fb17006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:59.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.039+0000 7fb1893cc640 1 --2- 172.21.15.5:0/803600269 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb17006be30 0x7fb17006e340 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb174033000 tx=0x7fb174009000 comp rx=0 tx=0).stop 2024-05-20T18:28:59.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.039+0000 7fb1893cc640 1 -- 172.21.15.5:0/803600269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb184162820 msgr2=0x7fb1841f51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:59.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.039+0000 7fb1893cc640 1 --2- 172.21.15.5:0/803600269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb184162820 0x7fb1841f51b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb17c00b4c0 tx=0x7fb17c00b980 comp rx=0 tx=0).stop 2024-05-20T18:28:59.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.039+0000 7fb1893cc640 1 -- 172.21.15.5:0/803600269 shutdown_connections 2024-05-20T18:28:59.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.039+0000 7fb1893cc640 1 --2- 172.21.15.5:0/803600269 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb17006be30 0x7fb17006e340 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:59.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.039+0000 7fb1893cc640 1 --2- 172.21.15.5:0/803600269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb184162820 0x7fb1841f51b0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:59.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.039+0000 7fb1893cc640 1 --2- 172.21.15.5:0/803600269 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1840b3e80 0x7fb1840b6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:59.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.039+0000 7fb1893cc640 1 --2- 172.21.15.5:0/803600269 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb18406d910 0x7fb18414f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:59.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.039+0000 7fb1893cc640 1 -- 172.21.15.5:0/803600269 >> 172.21.15.5:0/803600269 conn(0x7fb1840d4bd0 msgr2=0x7fb1840d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:59.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.039+0000 7fb1893cc640 1 -- 172.21.15.5:0/803600269 shutdown_connections 2024-05-20T18:28:59.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.039+0000 7fb1893cc640 1 -- 172.21.15.5:0/803600269 wait complete. 2024-05-20T18:28:59.054 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:28:59.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.168+0000 7f14b4c51640 1 Processor -- start 2024-05-20T18:28:59.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.168+0000 7f14b4c51640 1 -- start start 2024-05-20T18:28:59.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.169+0000 7f14b4c51640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f14b00d17f0 0x7f14b015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:59.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.169+0000 7f14b4c51640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14b00cd580 con 0x7f14b015d530 2024-05-20T18:28:59.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.169+0000 7f14b4c51640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14b00cd860 con 0x7f14b00b1e10 2024-05-20T18:28:59.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.169+0000 7f14b4c51640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f14b00ce190 con 0x7f14b00d17f0 2024-05-20T18:28:59.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.169+0000 7f14aed9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f14b00d17f0 0x7f14b015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:59.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.169+0000 7f14ae59c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f14b015d530 0x7f14b00b1700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38670/0 (socket says 172.21.15.5:38670) 2024-05-20T18:28:59.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.169+0000 7f14aed9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f14b00d17f0 0x7f14b015aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59692/0 (socket says 172.21.15.5:59692) 2024-05-20T18:28:59.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.169+0000 7f14ae59c640 1 -- 172.21.15.5:0/1405190603 learned_addr learned my addr 172.21.15.5:0/1405190603 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:59.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.170+0000 7f14aed9d640 1 -- 172.21.15.5:0/1405190603 >> v1:172.21.15.63:6789/0 conn(0x7f14b00b1e10 legacy=0x7f14b0070850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:59.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.170+0000 7f14aed9d640 1 -- 172.21.15.5:0/1405190603 >> v1:172.21.15.5:6789/0 conn(0x7f14b015d530 legacy=0x7f14b00b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:59.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.170+0000 7f14add9b640 1 -- 172.21.15.5:0/1405190603 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 675556188 0 0) 0x7f14b00cd580 con 0x7f14b015d530 2024-05-20T18:28:59.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.170+0000 7f14aed9d640 1 -- 172.21.15.5:0/1405190603 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b00cdb40 con 0x7f14b00d17f0 2024-05-20T18:28:59.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.170+0000 7f14aed9d640 1 --2- 172.21.15.5:0/1405190603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14b00d17f0 0x7f14b015aaa0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009d90 tx=0x7f14a802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961cd8bda74204cc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:59.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.171+0000 7f14add9b640 1 -- 172.21.15.5:0/1405190603 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8007740 con 0x7f14b00d17f0 2024-05-20T18:28:59.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.171+0000 7f14add9b640 1 -- 172.21.15.5:0/1405190603 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a80078e0 con 0x7f14b00d17f0 2024-05-20T18:28:59.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.171+0000 7f14add9b640 1 -- 172.21.15.5:0/1405190603 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8007cb0 con 0x7f14b00d17f0 2024-05-20T18:28:59.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.171+0000 7f14b4c51640 1 -- 172.21.15.5:0/1405190603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14b00d17f0 msgr2=0x7f14b015aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:59.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.171+0000 7f14b4c51640 1 --2- 172.21.15.5:0/1405190603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14b00d17f0 0x7f14b015aaa0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009d90 tx=0x7f14a802a6f0 comp rx=0 tx=0).stop 2024-05-20T18:28:59.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.171+0000 7f14b4c51640 1 -- 172.21.15.5:0/1405190603 shutdown_connections 2024-05-20T18:28:59.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.171+0000 7f14b4c51640 1 --2- 172.21.15.5:0/1405190603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14b00d17f0 0x7f14b015aaa0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:59.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.171+0000 7f14b4c51640 1 -- 172.21.15.5:0/1405190603 >> 172.21.15.5:0/1405190603 conn(0x7f14b00d29d0 msgr2=0x7f14b00d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:59.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.171+0000 7f14b4c51640 1 -- 172.21.15.5:0/1405190603 shutdown_connections 2024-05-20T18:28:59.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.171+0000 7f14b4c51640 1 -- 172.21.15.5:0/1405190603 wait complete. 2024-05-20T18:28:59.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.173+0000 7f14b4c51640 1 Processor -- start 2024-05-20T18:28:59.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.173+0000 7f14b4c51640 1 -- start start 2024-05-20T18:28:59.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.175+0000 7f14b4c51640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14b00b1e10 0x7f14b00cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:59.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.175+0000 7f14b4c51640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14b00d17f0 0x7f14b0159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:59.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.175+0000 7f14b4c51640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14b015d530 0x7f14b00c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:59.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.175+0000 7f14b4c51640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f14b0070f60 con 0x7f14b00d17f0 2024-05-20T18:28:59.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.175+0000 7f14b4c51640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f14b00cd230 con 0x7f14b00b1e10 2024-05-20T18:28:59.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.175+0000 7f14b4c51640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f14b00710e0 con 0x7f14b015d530 2024-05-20T18:28:59.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.175+0000 7f14a5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14b00d17f0 0x7f14b0159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:59.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.175+0000 7f14a5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14b00d17f0 0x7f14b0159f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59700/0 (socket says 172.21.15.5:59700) 2024-05-20T18:28:59.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.175+0000 7f14a5d9b640 1 -- 172.21.15.5:0/77332538 learned_addr learned my addr 172.21.15.5:0/77332538 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:59.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.175+0000 7f14ae59c640 1 --2- 172.21.15.5:0/77332538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14b00b1e10 0x7f14b00cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:59.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.175+0000 7f14aed9d640 1 --2- 172.21.15.5:0/77332538 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14b015d530 0x7f14b00c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:59.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.175+0000 7f14a5d9b640 1 -- 172.21.15.5:0/77332538 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14b015d530 msgr2=0x7f14b00c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.175+0000 7f14a5d9b640 1 --2- 172.21.15.5:0/77332538 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14b015d530 0x7f14b00c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.175+0000 7f14a5d9b640 1 -- 172.21.15.5:0/77332538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14b00b1e10 msgr2=0x7f14b00cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.175+0000 7f14a5d9b640 1 --2- 172.21.15.5:0/77332538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14b00b1e10 0x7f14b00cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.175+0000 7f14a5d9b640 1 -- 172.21.15.5:0/77332538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b00cbde0 con 0x7f14b00d17f0 2024-05-20T18:28:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.175+0000 7f14a5d9b640 1 --2- 172.21.15.5:0/77332538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14b00d17f0 0x7f14b0159f50 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f149800c400 tx=0x7f149800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.176+0000 7f14a77fe640 1 -- 172.21.15.5:0/77332538 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1498017070 con 0x7f14b00d17f0 2024-05-20T18:28:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.176+0000 7f14a77fe640 1 -- 172.21.15.5:0/77332538 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14980052d0 con 0x7f14b00d17f0 2024-05-20T18:28:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.176+0000 7f14a77fe640 1 -- 172.21.15.5:0/77332538 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1498003830 con 0x7f14b00d17f0 2024-05-20T18:28:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.176+0000 7f14b4c51640 1 -- 172.21.15.5:0/77332538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b00cc0c0 con 0x7f14b00d17f0 2024-05-20T18:28:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.176+0000 7f14b4c51640 1 -- 172.21.15.5:0/77332538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b00a9720 con 0x7f14b00d17f0 2024-05-20T18:28:59.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.177+0000 7f14b4c51640 1 -- 172.21.15.5:0/77332538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14b0075b00 con 0x7f14b00d17f0 2024-05-20T18:28:59.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.178+0000 7f14a77fe640 1 -- 172.21.15.5:0/77332538 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f149801a070 con 0x7f14b00d17f0 2024-05-20T18:28:59.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.178+0000 7f14a77fe640 1 --2- 172.21.15.5:0/77332538 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f149006be30 0x7f149006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:59.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.178+0000 7f14a77fe640 1 -- 172.21.15.5:0/77332538 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5249+0+0 (secure 0 0 0) 0x7f1498089f50 con 0x7f14b00d17f0 2024-05-20T18:28:59.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.178+0000 7f14ae59c640 1 --2- 172.21.15.5:0/77332538 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f149006be30 0x7f149006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:59.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.178+0000 7f14ae59c640 1 --2- 172.21.15.5:0/77332538 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f149006be30 0x7f149006e340 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f14a0000f60 tx=0x7f14a0007e40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.184+0000 7f14a77fe640 1 -- 172.21.15.5:0/77332538 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1498058470 con 0x7f14b00d17f0 2024-05-20T18:28:59.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.373+0000 7f14b4c51640 1 -- 172.21.15.5:0/77332538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f14b00cffe0 con 0x7f14b00d17f0 2024-05-20T18:28:59.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.374+0000 7f14a77fe640 1 -- 172.21.15.5:0/77332538 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+13044 (secure 0 0 0) 0x7f149805c770 con 0x7f14b00d17f0 2024-05-20T18:28:59.373 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:28:59.373 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":47,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:28:58.639628+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:28:59.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.375+0000 7f14b4c51640 1 -- 172.21.15.5:0/77332538 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f149006be30 msgr2=0x7f149006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:59.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.375+0000 7f14b4c51640 1 --2- 172.21.15.5:0/77332538 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f149006be30 0x7f149006e340 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f14a0000f60 tx=0x7f14a0007e40 comp rx=0 tx=0).stop 2024-05-20T18:28:59.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.375+0000 7f14b4c51640 1 -- 172.21.15.5:0/77332538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14b00d17f0 msgr2=0x7f14b0159f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:59.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.375+0000 7f14b4c51640 1 --2- 172.21.15.5:0/77332538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14b00d17f0 0x7f14b0159f50 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f149800c400 tx=0x7f149800c8c0 comp rx=0 tx=0).stop 2024-05-20T18:28:59.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.375+0000 7f14b4c51640 1 -- 172.21.15.5:0/77332538 shutdown_connections 2024-05-20T18:28:59.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.375+0000 7f14b4c51640 1 --2- 172.21.15.5:0/77332538 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f149006be30 0x7f149006e340 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:59.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.375+0000 7f14b4c51640 1 --2- 172.21.15.5:0/77332538 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14b015d530 0x7f14b00c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:59.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.375+0000 7f14b4c51640 1 --2- 172.21.15.5:0/77332538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14b00d17f0 0x7f14b0159f50 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:59.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.375+0000 7f14b4c51640 1 --2- 172.21.15.5:0/77332538 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14b00b1e10 0x7f14b00cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:59.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.376+0000 7f14b4c51640 1 -- 172.21.15.5:0/77332538 >> 172.21.15.5:0/77332538 conn(0x7f14b00d29d0 msgr2=0x7f14b015f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:59.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.376+0000 7f14b4c51640 1 -- 172.21.15.5:0/77332538 shutdown_connections 2024-05-20T18:28:59.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.376+0000 7f14b4c51640 1 -- 172.21.15.5:0/77332538 wait complete. 2024-05-20T18:28:59.387 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:28:59.387 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:28:59.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.506+0000 7fbf2ac6c640 1 Processor -- start 2024-05-20T18:28:59.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.506+0000 7fbf2ac6c640 1 -- start start 2024-05-20T18:28:59.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf2ac6c640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbf24168050 0x7fbf2416a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf2ac6c640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf240d0630 con 0x7fbf24162820 2024-05-20T18:28:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf2ac6c640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf240d39c0 con 0x7fbf2406d3d0 2024-05-20T18:28:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf2ac6c640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fbf240d3ca0 con 0x7fbf24168050 2024-05-20T18:28:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf28a08640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbf24162820 0x7fbf240b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38674/0 (socket says 172.21.15.5:38674) 2024-05-20T18:28:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf28a08640 1 -- 172.21.15.5:0/3148880065 learned_addr learned my addr 172.21.15.5:0/3148880065 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf29209640 1 --2- 172.21.15.5:0/3148880065 >> v2:172.21.15.63:3300/0 conn(0x7fbf24168050 0x7fbf2416a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf237fe640 1 -- 172.21.15.5:0/3148880065 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 583927329 0 0) 0x7fbf240d0630 con 0x7fbf24162820 2024-05-20T18:28:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf237fe640 1 -- 172.21.15.5:0/3148880065 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf10003520 con 0x7fbf24162820 2024-05-20T18:28:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf237fe640 1 -- 172.21.15.5:0/3148880065 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 652628022 0 0) 0x7fbf10003520 con 0x7fbf24162820 2024-05-20T18:28:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf237fe640 1 -- 172.21.15.5:0/3148880065 >> v2:172.21.15.63:3300/0 conn(0x7fbf24168050 msgr2=0x7fbf2416a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf237fe640 1 --2- 172.21.15.5:0/3148880065 >> v2:172.21.15.63:3300/0 conn(0x7fbf24168050 0x7fbf2416a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf237fe640 1 -- 172.21.15.5:0/3148880065 >> v1:172.21.15.63:6789/0 conn(0x7fbf2406d3d0 legacy=0x7fbf2406d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf237fe640 1 -- 172.21.15.5:0/3148880065 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf240d3f80 con 0x7fbf24162820 2024-05-20T18:28:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf237fe640 1 -- 172.21.15.5:0/3148880065 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7fbf14001670 con 0x7fbf24162820 2024-05-20T18:28:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf237fe640 1 -- 172.21.15.5:0/3148880065 >> v1:172.21.15.5:6789/0 conn(0x7fbf24162820 legacy=0x7fbf240b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf237fe640 1 --2- 172.21.15.5:0/3148880065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf100039b0 0x7fbf10003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf237fe640 1 --2- 172.21.15.5:0/3148880065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf10004800 0x7fbf10004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf237fe640 1 --2- 172.21.15.5:0/3148880065 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf100051c0 0x7fbf1000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf237fe640 1 -- 172.21.15.5:0/3148880065 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbf240d3ca0 con 0x7fbf10004800 2024-05-20T18:28:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf237fe640 1 -- 172.21.15.5:0/3148880065 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1000c160 con 0x7fbf100039b0 2024-05-20T18:28:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf237fe640 1 -- 172.21.15.5:0/3148880065 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbf1000c360 con 0x7fbf100051c0 2024-05-20T18:28:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf23fff640 1 --2- 172.21.15.5:0/3148880065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf10004800 0x7fbf10004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf28a08640 1 --2- 172.21.15.5:0/3148880065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf100039b0 0x7fbf10003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf29209640 1 --2- 172.21.15.5:0/3148880065 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf100051c0 0x7fbf1000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf23fff640 1 -- 172.21.15.5:0/3148880065 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf100051c0 msgr2=0x7fbf1000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf23fff640 1 --2- 172.21.15.5:0/3148880065 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf100051c0 0x7fbf1000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf23fff640 1 -- 172.21.15.5:0/3148880065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf100039b0 msgr2=0x7fbf10003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf23fff640 1 --2- 172.21.15.5:0/3148880065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf100039b0 0x7fbf10003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf29209640 1 -- 172.21.15.5:0/3148880065 reap_dead start 2024-05-20T18:28:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.508+0000 7fbf23fff640 1 -- 172.21.15.5:0/3148880065 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf10003520 con 0x7fbf10004800 2024-05-20T18:28:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.509+0000 7fbf23fff640 1 --2- 172.21.15.5:0/3148880065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf10004800 0x7fbf10004c10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c001e50 tx=0x7fbf10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1aa40a4ba8a04b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.509+0000 7fbf237fe640 1 -- 172.21.15.5:0/3148880065 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c001a70 con 0x7fbf10004800 2024-05-20T18:28:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.509+0000 7fbf237fe640 1 -- 172.21.15.5:0/3148880065 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf0c02ae40 con 0x7fbf10004800 2024-05-20T18:28:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.509+0000 7fbf237fe640 1 -- 172.21.15.5:0/3148880065 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c003bb0 con 0x7fbf10004800 2024-05-20T18:28:59.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.510+0000 7fbf2ac6c640 1 -- 172.21.15.5:0/3148880065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf10004800 msgr2=0x7fbf10004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:59.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.510+0000 7fbf2ac6c640 1 --2- 172.21.15.5:0/3148880065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf10004800 0x7fbf10004c10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c001e50 tx=0x7fbf10005830 comp rx=0 tx=0).stop 2024-05-20T18:28:59.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.510+0000 7fbf2ac6c640 1 -- 172.21.15.5:0/3148880065 shutdown_connections 2024-05-20T18:28:59.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.510+0000 7fbf2ac6c640 1 --2- 172.21.15.5:0/3148880065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf10004800 0x7fbf10004c10 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:59.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.510+0000 7fbf2ac6c640 1 -- 172.21.15.5:0/3148880065 >> 172.21.15.5:0/3148880065 conn(0x7fbf240d4bd0 msgr2=0x7fbf240d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:59.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.510+0000 7fbf2ac6c640 1 -- 172.21.15.5:0/3148880065 shutdown_connections 2024-05-20T18:28:59.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.510+0000 7fbf2ac6c640 1 -- 172.21.15.5:0/3148880065 wait complete. 2024-05-20T18:28:59.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.513+0000 7fbf2ac6c640 1 Processor -- start 2024-05-20T18:28:59.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.513+0000 7fbf2ac6c640 1 -- start start 2024-05-20T18:28:59.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.514+0000 7fbf2ac6c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf10004800 0x7fbf24160080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.514+0000 7fbf2ac6c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf24158ad0 0x7fbf24158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.514+0000 7fbf2ac6c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf240cbc60 0x7fbf240cc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.514+0000 7fbf2ac6c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbf240d4780 con 0x7fbf10004800 2024-05-20T18:28:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.514+0000 7fbf2ac6c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbf240d0300 con 0x7fbf24158ad0 2024-05-20T18:28:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.514+0000 7fbf2ac6c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbf240d4900 con 0x7fbf240cbc60 2024-05-20T18:28:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.514+0000 7fbf28a08640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf10004800 0x7fbf24160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.514+0000 7fbf28a08640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf10004800 0x7fbf24160080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59732/0 (socket says 172.21.15.5:59732) 2024-05-20T18:28:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.514+0000 7fbf28a08640 1 -- 172.21.15.5:0/1720190690 learned_addr learned my addr 172.21.15.5:0/1720190690 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.514+0000 7fbf28a08640 1 -- 172.21.15.5:0/1720190690 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf240cbc60 msgr2=0x7fbf240cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.514+0000 7fbf28a08640 1 --2- 172.21.15.5:0/1720190690 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf240cbc60 0x7fbf240cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.514+0000 7fbf28a08640 1 -- 172.21.15.5:0/1720190690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf24158ad0 msgr2=0x7fbf24158ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.514+0000 7fbf28a08640 1 --2- 172.21.15.5:0/1720190690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf24158ad0 0x7fbf24158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.514+0000 7fbf28a08640 1 -- 172.21.15.5:0/1720190690 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2415df60 con 0x7fbf10004800 2024-05-20T18:28:59.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.514+0000 7fbf28a08640 1 --2- 172.21.15.5:0/1720190690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf10004800 0x7fbf24160080 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fbf140072c0 tx=0x7fbf14007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:59.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.514+0000 7fbf217fa640 1 -- 172.21.15.5:0/1720190690 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf14001640 con 0x7fbf10004800 2024-05-20T18:28:59.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.514+0000 7fbf217fa640 1 -- 172.21.15.5:0/1720190690 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf14018070 con 0x7fbf10004800 2024-05-20T18:28:59.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.514+0000 7fbf2ac6c640 1 -- 172.21.15.5:0/1720190690 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf240c8e70 con 0x7fbf10004800 2024-05-20T18:28:59.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.514+0000 7fbf217fa640 1 -- 172.21.15.5:0/1720190690 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1400fc20 con 0x7fbf10004800 2024-05-20T18:28:59.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.514+0000 7fbf2ac6c640 1 -- 172.21.15.5:0/1720190690 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf2415e6b0 con 0x7fbf10004800 2024-05-20T18:28:59.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.515+0000 7fbf2ac6c640 1 -- 172.21.15.5:0/1720190690 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf24075b00 con 0x7fbf10004800 2024-05-20T18:28:59.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.515+0000 7fbf217fa640 1 -- 172.21.15.5:0/1720190690 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbf1401d460 con 0x7fbf10004800 2024-05-20T18:28:59.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.516+0000 7fbf217fa640 1 --2- 172.21.15.5:0/1720190690 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf0406be30 0x7fbf0406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:59.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.516+0000 7fbf217fa640 1 -- 172.21.15.5:0/1720190690 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5249+0+0 (secure 0 0 0) 0x7fbf1408ad30 con 0x7fbf10004800 2024-05-20T18:28:59.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.516+0000 7fbf23fff640 1 --2- 172.21.15.5:0/1720190690 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf0406be30 0x7fbf0406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:59.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.517+0000 7fbf23fff640 1 --2- 172.21.15.5:0/1720190690 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf0406be30 0x7fbf0406e340 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c032000 tx=0x7fbf0c02c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:59.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.521+0000 7fbf217fa640 1 -- 172.21.15.5:0/1720190690 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbf14059810 con 0x7fbf10004800 2024-05-20T18:28:59.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.705+0000 7fbf2ac6c640 1 -- 172.21.15.5:0/1720190690 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbf240d5360 con 0x7fbf10004800 2024-05-20T18:28:59.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.705+0000 7fbf217fa640 1 -- 172.21.15.5:0/1720190690 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) ==== 93+0+10 (secure 0 0 0) 0x7fbf14059110 con 0x7fbf10004800 2024-05-20T18:28:59.705 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:28:59.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.707+0000 7fbf2ac6c640 1 -- 172.21.15.5:0/1720190690 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf0406be30 msgr2=0x7fbf0406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:59.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.707+0000 7fbf2ac6c640 1 --2- 172.21.15.5:0/1720190690 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf0406be30 0x7fbf0406e340 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c032000 tx=0x7fbf0c02c000 comp rx=0 tx=0).stop 2024-05-20T18:28:59.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.707+0000 7fbf2ac6c640 1 -- 172.21.15.5:0/1720190690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf10004800 msgr2=0x7fbf24160080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:59.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.707+0000 7fbf2ac6c640 1 --2- 172.21.15.5:0/1720190690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf10004800 0x7fbf24160080 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fbf140072c0 tx=0x7fbf14007780 comp rx=0 tx=0).stop 2024-05-20T18:28:59.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.708+0000 7fbf2ac6c640 1 -- 172.21.15.5:0/1720190690 shutdown_connections 2024-05-20T18:28:59.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.708+0000 7fbf2ac6c640 1 --2- 172.21.15.5:0/1720190690 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf0406be30 0x7fbf0406e340 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:59.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.708+0000 7fbf2ac6c640 1 --2- 172.21.15.5:0/1720190690 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf240cbc60 0x7fbf240cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:59.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.708+0000 7fbf2ac6c640 1 --2- 172.21.15.5:0/1720190690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf24158ad0 0x7fbf24158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:59.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.708+0000 7fbf2ac6c640 1 --2- 172.21.15.5:0/1720190690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf10004800 0x7fbf24160080 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:59.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.708+0000 7fbf2ac6c640 1 -- 172.21.15.5:0/1720190690 >> 172.21.15.5:0/1720190690 conn(0x7fbf240d4bd0 msgr2=0x7fbf240b3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:59.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.708+0000 7fbf2ac6c640 1 -- 172.21.15.5:0/1720190690 shutdown_connections 2024-05-20T18:28:59.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.708+0000 7fbf2ac6c640 1 -- 172.21.15.5:0/1720190690 wait complete. 2024-05-20T18:28:59.719 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-20T18:28:59.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.834+0000 7fb20338f640 1 Processor -- start 2024-05-20T18:28:59.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.834+0000 7fb20338f640 1 -- start start 2024-05-20T18:28:59.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.836+0000 7fb20338f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb1fc163a10 0x7fb1fc0d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:59.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.836+0000 7fb20338f640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb1fc0d14b0 0x7fb1fc0d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:59.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.836+0000 7fb20338f640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1fc0d0370 con 0x7fb1fc0b4010 2024-05-20T18:28:59.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.836+0000 7fb20338f640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fb1fc0d3ca0 con 0x7fb1fc163a10 2024-05-20T18:28:59.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.836+0000 7fb20338f640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb1fc0d0650 con 0x7fb1fc0d14b0 2024-05-20T18:28:59.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.836+0000 7fb20112b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb1fc163a10 0x7fb1fc0d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:59.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.836+0000 7fb20092a640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb1fc0b4010 0x7fb1fc0cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38686/0 (socket says 172.21.15.5:38686) 2024-05-20T18:28:59.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.836+0000 7fb20092a640 1 -- 172.21.15.5:0/3361268283 learned_addr learned my addr 172.21.15.5:0/3361268283 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:28:59.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.836+0000 7fb20192c640 1 --2- 172.21.15.5:0/3361268283 >> v2:172.21.15.63:3300/0 conn(0x7fb1fc0d14b0 0x7fb1fc0d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:59.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.836+0000 7fb20112b640 1 -- 172.21.15.5:0/3361268283 >> v2:172.21.15.63:3300/0 conn(0x7fb1fc0d14b0 msgr2=0x7fb1fc0d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:28:59.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.836+0000 7fb20112b640 1 --2- 172.21.15.5:0/3361268283 >> v2:172.21.15.63:3300/0 conn(0x7fb1fc0d14b0 0x7fb1fc0d18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:59.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.836+0000 7fb20112b640 1 -- 172.21.15.5:0/3361268283 >> v1:172.21.15.5:6789/0 conn(0x7fb1fc0b4010 legacy=0x7fb1fc0cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:59.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.836+0000 7fb20112b640 1 -- 172.21.15.5:0/3361268283 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1fc0d3650 con 0x7fb1fc163a10 2024-05-20T18:28:59.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.836+0000 7fb1f3fff640 1 -- 172.21.15.5:0/3361268283 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1308645117 0 0) 0x7fb1fc0d0370 con 0x7fb1fc0b4010 2024-05-20T18:28:59.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.836+0000 7fb20192c640 1 --2- 172.21.15.5:0/3361268283 >> v2:172.21.15.63:3300/0 conn(0x7fb1fc0d14b0 0x7fb1fc0d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:59.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.836+0000 7fb20112b640 1 --2- 172.21.15.5:0/3361268283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1fc163a10 0x7fb1fc0d4130 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc0b4590 tx=0x7fb1f802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17d1c52d3ab6c3e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:59.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.837+0000 7fb1f3fff640 1 -- 172.21.15.5:0/3361268283 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8037070 con 0x7fb1fc163a10 2024-05-20T18:28:59.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.837+0000 7fb1f3fff640 1 -- 172.21.15.5:0/3361268283 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1f8032440 con 0x7fb1fc163a10 2024-05-20T18:28:59.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.837+0000 7fb1f3fff640 1 -- 172.21.15.5:0/3361268283 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8032810 con 0x7fb1fc163a10 2024-05-20T18:28:59.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.837+0000 7fb20338f640 1 -- 172.21.15.5:0/3361268283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1fc163a10 msgr2=0x7fb1fc0d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:59.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.837+0000 7fb20338f640 1 --2- 172.21.15.5:0/3361268283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1fc163a10 0x7fb1fc0d4130 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc0b4590 tx=0x7fb1f802a1a0 comp rx=0 tx=0).stop 2024-05-20T18:28:59.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.837+0000 7fb20338f640 1 -- 172.21.15.5:0/3361268283 shutdown_connections 2024-05-20T18:28:59.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.837+0000 7fb20338f640 1 --2- 172.21.15.5:0/3361268283 >> v2:172.21.15.63:3300/0 conn(0x7fb1fc0d14b0 0x7fb1fc0d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:59.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.837+0000 7fb20338f640 1 --2- 172.21.15.5:0/3361268283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1fc163a10 0x7fb1fc0d4130 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:59.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.837+0000 7fb20338f640 1 -- 172.21.15.5:0/3361268283 >> 172.21.15.5:0/3361268283 conn(0x7fb1fc06d240 msgr2=0x7fb1fc06d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:28:59.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.837+0000 7fb20338f640 1 -- 172.21.15.5:0/3361268283 shutdown_connections 2024-05-20T18:28:59.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.837+0000 7fb20338f640 1 -- 172.21.15.5:0/3361268283 wait complete. 2024-05-20T18:28:59.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.839+0000 7fb20338f640 1 Processor -- start 2024-05-20T18:28:59.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.839+0000 7fb20338f640 1 -- start start 2024-05-20T18:28:59.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.840+0000 7fb20338f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1fc0b4010 0x7fb1fc154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:59.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.840+0000 7fb20338f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1fc0d14b0 0x7fb1fc15a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:59.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.841+0000 7fb20338f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1fc163a10 0x7fb1fc15a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:59.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.841+0000 7fb20338f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc0b8210 con 0x7fb1fc0d14b0 2024-05-20T18:28:59.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.841+0000 7fb20338f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc0d1e00 con 0x7fb1fc163a10 2024-05-20T18:28:59.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.841+0000 7fb20338f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb1fc0b8390 con 0x7fb1fc0b4010 2024-05-20T18:28:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.841+0000 7fb20092a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1fc0d14b0 0x7fb1fc15a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.841+0000 7fb20092a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1fc0d14b0 0x7fb1fc15a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59758/0 (socket says 172.21.15.5:59758) 2024-05-20T18:28:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.841+0000 7fb20092a640 1 -- 172.21.15.5:0/2013780970 learned_addr learned my addr 172.21.15.5:0/2013780970 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:28:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.841+0000 7fb20112b640 1 --2- 172.21.15.5:0/2013780970 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1fc0b4010 0x7fb1fc154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.841+0000 7fb20192c640 1 --2- 172.21.15.5:0/2013780970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1fc163a10 0x7fb1fc15a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.841+0000 7fb20092a640 1 -- 172.21.15.5:0/2013780970 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1fc0b4010 msgr2=0x7fb1fc154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.841+0000 7fb20092a640 1 --2- 172.21.15.5:0/2013780970 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1fc0b4010 0x7fb1fc154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.841+0000 7fb20092a640 1 -- 172.21.15.5:0/2013780970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1fc163a10 msgr2=0x7fb1fc15a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:28:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.841+0000 7fb20092a640 1 --2- 172.21.15.5:0/2013780970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1fc163a10 0x7fb1fc15a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:28:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.841+0000 7fb20092a640 1 -- 172.21.15.5:0/2013780970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1fc1ea470 con 0x7fb1fc0d14b0 2024-05-20T18:28:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.841+0000 7fb20112b640 1 --2- 172.21.15.5:0/2013780970 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1fc0b4010 0x7fb1fc154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.841+0000 7fb20192c640 1 --2- 172.21.15.5:0/2013780970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1fc163a10 0x7fb1fc15a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:28:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.841+0000 7fb20092a640 1 --2- 172.21.15.5:0/2013780970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1fc0d14b0 0x7fb1fc15a170 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec00c430 tx=0x7fb1ec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.841+0000 7fb1f1ffb640 1 -- 172.21.15.5:0/2013780970 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec0074e0 con 0x7fb1fc0d14b0 2024-05-20T18:28:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.841+0000 7fb1f1ffb640 1 -- 172.21.15.5:0/2013780970 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1ec0047e0 con 0x7fb1fc0d14b0 2024-05-20T18:28:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.841+0000 7fb1f1ffb640 1 -- 172.21.15.5:0/2013780970 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec004bd0 con 0x7fb1fc0d14b0 2024-05-20T18:28:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.841+0000 7fb20338f640 1 -- 172.21.15.5:0/2013780970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1fc159690 con 0x7fb1fc0d14b0 2024-05-20T18:28:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.842+0000 7fb20338f640 1 -- 172.21.15.5:0/2013780970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1fc154e90 con 0x7fb1fc0d14b0 2024-05-20T18:28:59.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.843+0000 7fb20338f640 1 -- 172.21.15.5:0/2013780970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1fc1ea1e0 con 0x7fb1fc0d14b0 2024-05-20T18:28:59.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.843+0000 7fb1f1ffb640 1 -- 172.21.15.5:0/2013780970 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb1ec011070 con 0x7fb1fc0d14b0 2024-05-20T18:28:59.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.845+0000 7fb1f1ffb640 1 --2- 172.21.15.5:0/2013780970 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb1e006be30 0x7fb1e006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:28:59.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.845+0000 7fb20112b640 1 --2- 172.21.15.5:0/2013780970 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb1e006be30 0x7fb1e006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:28:59.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.845+0000 7fb1f1ffb640 1 -- 172.21.15.5:0/2013780970 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5249+0+0 (secure 0 0 0) 0x7fb1ec08aa70 con 0x7fb1fc0d14b0 2024-05-20T18:28:59.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.845+0000 7fb20112b640 1 --2- 172.21.15.5:0/2013780970 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb1e006be30 0x7fb1e006e340 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8034000 tx=0x7fb1f802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:28:59.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:28:59.849+0000 7fb1f1ffb640 1 -- 172.21.15.5:0/2013780970 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb1ec0594b0 con 0x7fb1fc0d14b0 2024-05-20T18:29:00.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.034+0000 7fb20338f640 1 -- 172.21.15.5:0/2013780970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb1fc1594a0 con 0x7fb1fc0d14b0 2024-05-20T18:29:00.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.035+0000 7fb1f1ffb640 1 -- 172.21.15.5:0/2013780970 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v47) ==== 84+0+65 (secure 0 0 0) 0x7fb1ec058db0 con 0x7fb1fc0d14b0 2024-05-20T18:29:00.034 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:29:00.034 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-05-20T18:29:00.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:29:00.034 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-05-20T18:29:00.034 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-05-20T18:29:00.034 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:29:00.034 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-05-20T18:29:00.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.037+0000 7fb20338f640 1 -- 172.21.15.5:0/2013780970 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb1e006be30 msgr2=0x7fb1e006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:00.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.037+0000 7fb20338f640 1 --2- 172.21.15.5:0/2013780970 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb1e006be30 0x7fb1e006e340 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8034000 tx=0x7fb1f802e000 comp rx=0 tx=0).stop 2024-05-20T18:29:00.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.037+0000 7fb20338f640 1 -- 172.21.15.5:0/2013780970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1fc0d14b0 msgr2=0x7fb1fc15a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:00.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.037+0000 7fb20338f640 1 --2- 172.21.15.5:0/2013780970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1fc0d14b0 0x7fb1fc15a170 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec00c430 tx=0x7fb1ec00c8f0 comp rx=0 tx=0).stop 2024-05-20T18:29:00.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.037+0000 7fb20338f640 1 -- 172.21.15.5:0/2013780970 shutdown_connections 2024-05-20T18:29:00.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.037+0000 7fb20338f640 1 --2- 172.21.15.5:0/2013780970 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb1e006be30 0x7fb1e006e340 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:00.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.037+0000 7fb20338f640 1 --2- 172.21.15.5:0/2013780970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1fc163a10 0x7fb1fc15a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:00.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.037+0000 7fb20338f640 1 --2- 172.21.15.5:0/2013780970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1fc0d14b0 0x7fb1fc15a170 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:00.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.037+0000 7fb20338f640 1 --2- 172.21.15.5:0/2013780970 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1fc0b4010 0x7fb1fc154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:00.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.037+0000 7fb20338f640 1 -- 172.21.15.5:0/2013780970 >> 172.21.15.5:0/2013780970 conn(0x7fb1fc06d240 msgr2=0x7fb1fc15f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:00.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.037+0000 7fb20338f640 1 -- 172.21.15.5:0/2013780970 shutdown_connections 2024-05-20T18:29:00.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.037+0000 7fb20338f640 1 -- 172.21.15.5:0/2013780970 wait complete. 2024-05-20T18:29:00.047 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-20T18:29:00.047 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-20T18:29:00.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.168+0000 7f4982938640 1 Processor -- start 2024-05-20T18:29:00.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.168+0000 7f4982938640 1 -- start start 2024-05-20T18:29:00.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.169+0000 7f4982938640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f497c0b3860 0x7f497c0d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:00.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.169+0000 7f4982938640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f497c06d870 0x7f497c15f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:00.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.169+0000 7f4982938640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f497c0d0450 con 0x7f497c08bd20 2024-05-20T18:29:00.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.169+0000 7f4982938640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f497c0d3b10 con 0x7f497c0b3860 2024-05-20T18:29:00.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.169+0000 7f4982938640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f497c0d3df0 con 0x7f497c06d870 2024-05-20T18:29:00.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.169+0000 7f497bfff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f497c0b3860 0x7f497c0d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:00.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.169+0000 7f4973fff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f497c06d870 0x7f497c15f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:00.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.170+0000 7f497bfff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f497c0b3860 0x7f497c0d45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:34522/0 (socket says 172.21.15.5:34522) 2024-05-20T18:29:00.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.170+0000 7f497bfff640 1 -- 172.21.15.5:0/526392245 learned_addr learned my addr 172.21.15.5:0/526392245 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:00.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.170+0000 7f4980ed5640 1 --1- 172.21.15.5:0/526392245 >> v1:172.21.15.63:6789/0 conn(0x7f497c08bd20 0x7f497c08c130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:43844/0 (socket says 172.21.15.5:43844) 2024-05-20T18:29:00.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.170+0000 7f4973fff640 1 -- 172.21.15.5:0/526392245 >> v2:172.21.15.63:3300/0 conn(0x7f497c0b3860 msgr2=0x7f497c0d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:00.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.170+0000 7f4973fff640 1 --2- 172.21.15.5:0/526392245 >> v2:172.21.15.63:3300/0 conn(0x7f497c0b3860 0x7f497c0d45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:00.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.170+0000 7f4973fff640 1 -- 172.21.15.5:0/526392245 >> v1:172.21.15.63:6789/0 conn(0x7f497c08bd20 legacy=0x7f497c08c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:00.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.170+0000 7f497b7fe640 1 -- 172.21.15.5:0/526392245 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2692301584 0 0) 0x7f497c0d0450 con 0x7f497c08bd20 2024-05-20T18:29:00.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.170+0000 7f4973fff640 1 -- 172.21.15.5:0/526392245 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f497c0d40d0 con 0x7f497c06d870 2024-05-20T18:29:00.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.170+0000 7f497bfff640 1 --2- 172.21.15.5:0/526392245 >> v2:172.21.15.63:3300/0 conn(0x7f497c0b3860 0x7f497c0d45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:29:00.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.171+0000 7f4973fff640 1 --2- 172.21.15.5:0/526392245 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f497c06d870 0x7f497c15f230 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f49640097a0 tx=0x7f496402a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3676c52f853d0965 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:00.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.171+0000 7f497b7fe640 1 -- 172.21.15.5:0/526392245 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4964037070 con 0x7f497c06d870 2024-05-20T18:29:00.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.171+0000 7f497b7fe640 1 -- 172.21.15.5:0/526392245 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f496402adb0 con 0x7f497c06d870 2024-05-20T18:29:00.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.171+0000 7f497b7fe640 1 -- 172.21.15.5:0/526392245 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4964032660 con 0x7f497c06d870 2024-05-20T18:29:00.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.172+0000 7f4982938640 1 -- 172.21.15.5:0/526392245 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f497c06d870 msgr2=0x7f497c15f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:00.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.172+0000 7f4982938640 1 --2- 172.21.15.5:0/526392245 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f497c06d870 0x7f497c15f230 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f49640097a0 tx=0x7f496402a060 comp rx=0 tx=0).stop 2024-05-20T18:29:00.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.172+0000 7f4982938640 1 -- 172.21.15.5:0/526392245 shutdown_connections 2024-05-20T18:29:00.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.172+0000 7f4982938640 1 --2- 172.21.15.5:0/526392245 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f497c06d870 0x7f497c15f230 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:00.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.172+0000 7f4982938640 1 --2- 172.21.15.5:0/526392245 >> v2:172.21.15.63:3300/0 conn(0x7f497c0b3860 0x7f497c0d45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:00.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.172+0000 7f4982938640 1 -- 172.21.15.5:0/526392245 >> 172.21.15.5:0/526392245 conn(0x7f497c0b6d10 msgr2=0x7f497c06d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:00.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.172+0000 7f4982938640 1 -- 172.21.15.5:0/526392245 shutdown_connections 2024-05-20T18:29:00.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.172+0000 7f4982938640 1 -- 172.21.15.5:0/526392245 wait complete. 2024-05-20T18:29:00.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.174+0000 7f4982938640 1 Processor -- start 2024-05-20T18:29:00.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.174+0000 7f4982938640 1 -- start start 2024-05-20T18:29:00.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.175+0000 7f4982938640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f497c06d870 0x7f497c1f8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:00.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.175+0000 7f4982938640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f497c08bd20 0x7f497c148390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:00.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.175+0000 7f4982938640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f497c0b3860 0x7f497c1488d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:00.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.175+0000 7f4982938640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f497c0b81b0 con 0x7f497c0b3860 2024-05-20T18:29:00.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.175+0000 7f4982938640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f497c15f770 con 0x7f497c06d870 2024-05-20T18:29:00.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.175+0000 7f4982938640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f497c0b8330 con 0x7f497c08bd20 2024-05-20T18:29:00.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.175+0000 7f4980ed5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f497c0b3860 0x7f497c1488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:00.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.175+0000 7f4980ed5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f497c0b3860 0x7f497c1488d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59774/0 (socket says 172.21.15.5:59774) 2024-05-20T18:29:00.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.175+0000 7f4980ed5640 1 -- 172.21.15.5:0/656523041 learned_addr learned my addr 172.21.15.5:0/656523041 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:00.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.175+0000 7f4973fff640 1 --2- 172.21.15.5:0/656523041 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f497c08bd20 0x7f497c148390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:00.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.175+0000 7f497bfff640 1 --2- 172.21.15.5:0/656523041 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f497c06d870 0x7f497c1f8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:00.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.176+0000 7f4980ed5640 1 -- 172.21.15.5:0/656523041 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f497c08bd20 msgr2=0x7f497c148390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:00.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.176+0000 7f4980ed5640 1 --2- 172.21.15.5:0/656523041 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f497c08bd20 0x7f497c148390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:00.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.176+0000 7f4980ed5640 1 -- 172.21.15.5:0/656523041 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f497c06d870 msgr2=0x7f497c1f8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:00.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.176+0000 7f4980ed5640 1 --2- 172.21.15.5:0/656523041 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f497c06d870 0x7f497c1f8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:00.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.176+0000 7f4980ed5640 1 -- 172.21.15.5:0/656523041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f497c15a4f0 con 0x7f497c0b3860 2024-05-20T18:29:00.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.176+0000 7f4980ed5640 1 --2- 172.21.15.5:0/656523041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f497c0b3860 0x7f497c1488d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f497400c730 tx=0x7f497400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:00.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.176+0000 7f49797fa640 1 -- 172.21.15.5:0/656523041 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974018070 con 0x7f497c0b3860 2024-05-20T18:29:00.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.176+0000 7f49797fa640 1 -- 172.21.15.5:0/656523041 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f497400ce40 con 0x7f497c0b3860 2024-05-20T18:29:00.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.176+0000 7f4982938640 1 -- 172.21.15.5:0/656523041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f497c1463a0 con 0x7f497c0b3860 2024-05-20T18:29:00.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.176+0000 7f49797fa640 1 -- 172.21.15.5:0/656523041 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497400a880 con 0x7f497c0b3860 2024-05-20T18:29:00.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.176+0000 7f4982938640 1 -- 172.21.15.5:0/656523041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f497c146960 con 0x7f497c0b3860 2024-05-20T18:29:00.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.177+0000 7f4982938640 1 -- 172.21.15.5:0/656523041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f497c0690b0 con 0x7f497c0b3860 2024-05-20T18:29:00.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.178+0000 7f49797fa640 1 -- 172.21.15.5:0/656523041 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f497401b070 con 0x7f497c0b3860 2024-05-20T18:29:00.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.178+0000 7f49797fa640 1 --2- 172.21.15.5:0/656523041 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f495c06be30 0x7f495c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:00.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.178+0000 7f49797fa640 1 -- 172.21.15.5:0/656523041 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5249+0+0 (secure 0 0 0) 0x7f497408a270 con 0x7f497c0b3860 2024-05-20T18:29:00.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.178+0000 7f497bfff640 1 --2- 172.21.15.5:0/656523041 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f495c06be30 0x7f495c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:00.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.179+0000 7f497bfff640 1 --2- 172.21.15.5:0/656523041 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f495c06be30 0x7f495c06e340 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f497c0d15d0 tx=0x7f496c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:00.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.183+0000 7f49797fa640 1 -- 172.21.15.5:0/656523041 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f497c15a4f0 con 0x7f497c0b3860 2024-05-20T18:29:00.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.371+0000 7f4982938640 1 -- 172.21.15.5:0/656523041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f497c0b6150 con 0x7f497c0b3860 2024-05-20T18:29:00.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.635+0000 7f49797fa640 1 -- 172.21.15.5:0/656523041 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v48) ==== 138+0+0 (secure 0 0 0) 0x7f4974058d50 con 0x7f497c0b3860 2024-05-20T18:29:00.634 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-05-20T18:29:00.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.637+0000 7f4982938640 1 -- 172.21.15.5:0/656523041 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f495c06be30 msgr2=0x7f495c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:00.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.637+0000 7f4982938640 1 --2- 172.21.15.5:0/656523041 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f495c06be30 0x7f495c06e340 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f497c0d15d0 tx=0x7f496c008040 comp rx=0 tx=0).stop 2024-05-20T18:29:00.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.637+0000 7f4982938640 1 -- 172.21.15.5:0/656523041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f497c0b3860 msgr2=0x7f497c1488d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:00.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.637+0000 7f4982938640 1 --2- 172.21.15.5:0/656523041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f497c0b3860 0x7f497c1488d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f497400c730 tx=0x7f497400cbf0 comp rx=0 tx=0).stop 2024-05-20T18:29:00.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.638+0000 7f4982938640 1 -- 172.21.15.5:0/656523041 shutdown_connections 2024-05-20T18:29:00.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.638+0000 7f4982938640 1 --2- 172.21.15.5:0/656523041 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f495c06be30 0x7f495c06e340 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:00.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.638+0000 7f4982938640 1 --2- 172.21.15.5:0/656523041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f497c0b3860 0x7f497c1488d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:00.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.638+0000 7f4982938640 1 --2- 172.21.15.5:0/656523041 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f497c08bd20 0x7f497c148390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:00.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.638+0000 7f4982938640 1 --2- 172.21.15.5:0/656523041 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f497c06d870 0x7f497c1f8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:00.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.638+0000 7f4982938640 1 -- 172.21.15.5:0/656523041 >> 172.21.15.5:0/656523041 conn(0x7f497c0b6d10 msgr2=0x7f497c0b8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:00.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.638+0000 7f4982938640 1 -- 172.21.15.5:0/656523041 shutdown_connections 2024-05-20T18:29:00.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.638+0000 7f4982938640 1 -- 172.21.15.5:0/656523041 wait complete. 2024-05-20T18:29:00.652 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-20T18:29:00.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.775+0000 7f52c2afe640 1 Processor -- start 2024-05-20T18:29:00.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.775+0000 7f52c2afe640 1 -- start start 2024-05-20T18:29:00.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.777+0000 7f52c2afe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f52bc0d46e0 0x7f52bc0b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:00.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.777+0000 7f52c2afe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f52bc08c4c0 0x7f52bc0b8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:00.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.777+0000 7f52c2afe640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52bc0d2890 con 0x7f52bc15f320 2024-05-20T18:29:00.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.777+0000 7f52c2afe640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f52bc0d2b70 con 0x7f52bc0d46e0 2024-05-20T18:29:00.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.777+0000 7f52c2afe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f52bc0d2e50 con 0x7f52bc08c4c0 2024-05-20T18:29:00.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.777+0000 7f52c089a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f52bc0d46e0 0x7f52bc0b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:00.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.777+0000 7f52c109b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f52bc08c4c0 0x7f52bc0b8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:00.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.777+0000 7f52c109b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f52bc08c4c0 0x7f52bc0b8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:35682/0 (socket says 172.21.15.5:35682) 2024-05-20T18:29:00.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.777+0000 7f52c089a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f52bc0d46e0 0x7f52bc0b3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:34536/0 (socket says 172.21.15.5:34536) 2024-05-20T18:29:00.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.777+0000 7f52c109b640 1 -- 172.21.15.5:0/3366184043 learned_addr learned my addr 172.21.15.5:0/3366184043 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:00.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.777+0000 7f52bbfff640 1 --1- 172.21.15.5:0/3366184043 >> v1:172.21.15.63:6789/0 conn(0x7f52bc15f320 0x7f52bc08bdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:43858/0 (socket says 172.21.15.5:43858) 2024-05-20T18:29:00.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.778+0000 7f52c109b640 1 -- 172.21.15.5:0/3366184043 >> v2:172.21.15.63:3300/0 conn(0x7f52bc0d46e0 msgr2=0x7f52bc0b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:00.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.778+0000 7f52c109b640 1 --2- 172.21.15.5:0/3366184043 >> v2:172.21.15.63:3300/0 conn(0x7f52bc0d46e0 0x7f52bc0b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:00.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.778+0000 7f52c109b640 1 -- 172.21.15.5:0/3366184043 >> v1:172.21.15.63:6789/0 conn(0x7f52bc15f320 legacy=0x7f52bc08bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:00.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.778+0000 7f52c109b640 1 -- 172.21.15.5:0/3366184043 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52bc0d3130 con 0x7f52bc08c4c0 2024-05-20T18:29:00.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.778+0000 7f52c109b640 1 --2- 172.21.15.5:0/3366184043 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f52bc08c4c0 0x7f52bc0b8310 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f52b4009e50 tx=0x7f52b402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0a09381d9ea030 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:00.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.779+0000 7f52bb7fe640 1 -- 172.21.15.5:0/3366184043 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4036070 con 0x7f52bc08c4c0 2024-05-20T18:29:00.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.779+0000 7f52bb7fe640 1 -- 172.21.15.5:0/3366184043 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b4031600 con 0x7f52bc08c4c0 2024-05-20T18:29:00.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.779+0000 7f52bb7fe640 1 -- 172.21.15.5:0/3366184043 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b40319d0 con 0x7f52bc08c4c0 2024-05-20T18:29:00.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.779+0000 7f52c2afe640 1 -- 172.21.15.5:0/3366184043 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f52bc08c4c0 msgr2=0x7f52bc0b8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:00.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.779+0000 7f52c2afe640 1 --2- 172.21.15.5:0/3366184043 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f52bc08c4c0 0x7f52bc0b8310 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f52b4009e50 tx=0x7f52b402a710 comp rx=0 tx=0).stop 2024-05-20T18:29:00.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.779+0000 7f52c2afe640 1 -- 172.21.15.5:0/3366184043 shutdown_connections 2024-05-20T18:29:00.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.779+0000 7f52c2afe640 1 --2- 172.21.15.5:0/3366184043 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f52bc08c4c0 0x7f52bc0b8310 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:00.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.779+0000 7f52c2afe640 1 --2- 172.21.15.5:0/3366184043 >> v2:172.21.15.63:3300/0 conn(0x7f52bc0d46e0 0x7f52bc0b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:00.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.779+0000 7f52c2afe640 1 -- 172.21.15.5:0/3366184043 >> 172.21.15.5:0/3366184043 conn(0x7f52bc0d1cd0 msgr2=0x7f52bc0d4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:00.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.779+0000 7f52c2afe640 1 -- 172.21.15.5:0/3366184043 shutdown_connections 2024-05-20T18:29:00.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.779+0000 7f52c2afe640 1 -- 172.21.15.5:0/3366184043 wait complete. 2024-05-20T18:29:00.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.781+0000 7f52c2afe640 1 Processor -- start 2024-05-20T18:29:00.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.781+0000 7f52c2afe640 1 -- start start 2024-05-20T18:29:00.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.783+0000 7f52c2afe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52bc0d46e0 0x7f52bc154b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:00.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.783+0000 7f52c2afe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52bc15f320 0x7f52bc151e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:00.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.783+0000 7f52c2afe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f52bc1573c0 0x7f52bc1577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:00.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.783+0000 7f52c2afe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc15f730 con 0x7f52bc15f320 2024-05-20T18:29:00.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.783+0000 7f52c2afe640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc08ddb0 con 0x7f52bc0d46e0 2024-05-20T18:29:00.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.783+0000 7f52c2afe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f52bc0d3930 con 0x7f52bc1573c0 2024-05-20T18:29:00.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.783+0000 7f52bbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52bc15f320 0x7f52bc151e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:00.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.783+0000 7f52bbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52bc15f320 0x7f52bc151e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59790/0 (socket says 172.21.15.5:59790) 2024-05-20T18:29:00.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.783+0000 7f52bbfff640 1 -- 172.21.15.5:0/144173556 learned_addr learned my addr 172.21.15.5:0/144173556 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:00.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.783+0000 7f52c109b640 1 --2- 172.21.15.5:0/144173556 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f52bc1573c0 0x7f52bc1577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:00.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.783+0000 7f52c089a640 1 --2- 172.21.15.5:0/144173556 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52bc0d46e0 0x7f52bc154b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:00.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.783+0000 7f52bbfff640 1 -- 172.21.15.5:0/144173556 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f52bc1573c0 msgr2=0x7f52bc1577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:00.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.783+0000 7f52bbfff640 1 --2- 172.21.15.5:0/144173556 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f52bc1573c0 0x7f52bc1577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:00.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.783+0000 7f52bbfff640 1 -- 172.21.15.5:0/144173556 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52bc0d46e0 msgr2=0x7f52bc154b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:00.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.783+0000 7f52bbfff640 1 --2- 172.21.15.5:0/144173556 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52bc0d46e0 0x7f52bc154b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:00.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.783+0000 7f52bbfff640 1 -- 172.21.15.5:0/144173556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52bc157d10 con 0x7f52bc15f320 2024-05-20T18:29:00.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.783+0000 7f52c089a640 1 --2- 172.21.15.5:0/144173556 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52bc0d46e0 0x7f52bc154b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:29:00.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.783+0000 7f52bbfff640 1 --2- 172.21.15.5:0/144173556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52bc15f320 0x7f52bc151e60 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f52b000c3f0 tx=0x7f52b000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:00.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.784+0000 7f52b97fa640 1 -- 172.21.15.5:0/144173556 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b00074e0 con 0x7f52bc15f320 2024-05-20T18:29:00.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.784+0000 7f52b97fa640 1 -- 172.21.15.5:0/144173556 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b0007680 con 0x7f52bc15f320 2024-05-20T18:29:00.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.784+0000 7f52b97fa640 1 -- 172.21.15.5:0/144173556 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0004a00 con 0x7f52bc15f320 2024-05-20T18:29:00.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.784+0000 7f52c2afe640 1 -- 172.21.15.5:0/144173556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52bc162c10 con 0x7f52bc15f320 2024-05-20T18:29:00.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.784+0000 7f52c2afe640 1 -- 172.21.15.5:0/144173556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52bc162930 con 0x7f52bc15f320 2024-05-20T18:29:00.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.785+0000 7f52c2afe640 1 -- 172.21.15.5:0/144173556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52bc0690b0 con 0x7f52bc15f320 2024-05-20T18:29:00.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.785+0000 7f52b97fa640 1 -- 172.21.15.5:0/144173556 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f52b001a020 con 0x7f52bc15f320 2024-05-20T18:29:00.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.786+0000 7f52b97fa640 1 --2- 172.21.15.5:0/144173556 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f52a806be30 0x7f52a806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:00.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.786+0000 7f52b97fa640 1 -- 172.21.15.5:0/144173556 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5619+0+0 (secure 0 0 0) 0x7f52b008a450 con 0x7f52bc15f320 2024-05-20T18:29:00.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.786+0000 7f52c089a640 1 --2- 172.21.15.5:0/144173556 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f52a806be30 0x7f52a806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:00.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.787+0000 7f52c089a640 1 --2- 172.21.15.5:0/144173556 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f52a806be30 0x7f52a806e340 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f52bc068370 tx=0x7f52ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:00.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.791+0000 7f52b97fa640 1 -- 172.21.15.5:0/144173556 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f52bc157d10 con 0x7f52bc15f320 2024-05-20T18:29:00.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:00.975+0000 7f52c2afe640 1 -- 172.21.15.5:0/144173556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f52bc0b94c0 con 0x7f52bc15f320 2024-05-20T18:29:01.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.637+0000 7f52b97fa640 1 -- 172.21.15.5:0/144173556 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v49) ==== 170+0+0 (secure 0 0 0) 0x7f52b0058e50 con 0x7f52bc15f320 2024-05-20T18:29:01.636 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-05-20T18:29:01.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.641+0000 7f52c2afe640 1 -- 172.21.15.5:0/144173556 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f52a806be30 msgr2=0x7f52a806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:01.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.641+0000 7f52c2afe640 1 --2- 172.21.15.5:0/144173556 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f52a806be30 0x7f52a806e340 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f52bc068370 tx=0x7f52ac008040 comp rx=0 tx=0).stop 2024-05-20T18:29:01.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.641+0000 7f52c2afe640 1 -- 172.21.15.5:0/144173556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52bc15f320 msgr2=0x7f52bc151e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:01.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.641+0000 7f52c2afe640 1 --2- 172.21.15.5:0/144173556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52bc15f320 0x7f52bc151e60 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f52b000c3f0 tx=0x7f52b000c7f0 comp rx=0 tx=0).stop 2024-05-20T18:29:01.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.641+0000 7f52c2afe640 1 -- 172.21.15.5:0/144173556 shutdown_connections 2024-05-20T18:29:01.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.641+0000 7f52c2afe640 1 --2- 172.21.15.5:0/144173556 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f52a806be30 0x7f52a806e340 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:01.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.641+0000 7f52c2afe640 1 --2- 172.21.15.5:0/144173556 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f52bc1573c0 0x7f52bc1577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:01.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.641+0000 7f52c2afe640 1 --2- 172.21.15.5:0/144173556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52bc15f320 0x7f52bc151e60 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:01.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.641+0000 7f52c2afe640 1 --2- 172.21.15.5:0/144173556 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52bc0d46e0 0x7f52bc154b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:01.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.641+0000 7f52c2afe640 1 -- 172.21.15.5:0/144173556 >> 172.21.15.5:0/144173556 conn(0x7f52bc0d1cd0 msgr2=0x7f52bc0b4680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:01.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.643+0000 7f52c2afe640 1 -- 172.21.15.5:0/144173556 shutdown_connections 2024-05-20T18:29:01.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.644+0000 7f52c2afe640 1 -- 172.21.15.5:0/144173556 wait complete. 2024-05-20T18:29:01.674 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-20T18:29:01.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.798+0000 7f714759d640 1 Processor -- start 2024-05-20T18:29:01.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.798+0000 7f714759d640 1 -- start start 2024-05-20T18:29:01.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.799+0000 7f714759d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f71400cfda0 0x7f714015be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:01.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.799+0000 7f714759d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f71400b3af0 0x7f714006f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:01.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.799+0000 7f714759d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f714006c530 0x7f714006c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:01.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.799+0000 7f714759d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f714006ce80 con 0x7f71400cfda0 2024-05-20T18:29:01.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.799+0000 7f714759d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7140072f10 con 0x7f71400b3af0 2024-05-20T18:29:01.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.799+0000 7f714759d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f71400b4260 con 0x7f714006c530 2024-05-20T18:29:01.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.800+0000 7f714659b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f71400cfda0 0x7f714015be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:01.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.800+0000 7f714659b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f71400cfda0 0x7f714015be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59806/0 (socket says 172.21.15.5:59806) 2024-05-20T18:29:01.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.800+0000 7f714659b640 1 -- 172.21.15.5:0/783514672 learned_addr learned my addr 172.21.15.5:0/783514672 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:01.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.800+0000 7f7146d9c640 1 --2- 172.21.15.5:0/783514672 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f714006c530 0x7f714006c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:01.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.800+0000 7f7145d9a640 1 --2- 172.21.15.5:0/783514672 >> v2:172.21.15.63:3300/0 conn(0x7f71400b3af0 0x7f714006f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:01.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.800+0000 7f714659b640 1 -- 172.21.15.5:0/783514672 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f714006c530 msgr2=0x7f714006c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:01.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.800+0000 7f714659b640 1 --2- 172.21.15.5:0/783514672 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f714006c530 0x7f714006c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:01.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.800+0000 7f714659b640 1 -- 172.21.15.5:0/783514672 >> v2:172.21.15.63:3300/0 conn(0x7f71400b3af0 msgr2=0x7f714006f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:01.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.800+0000 7f714659b640 1 --2- 172.21.15.5:0/783514672 >> v2:172.21.15.63:3300/0 conn(0x7f71400b3af0 0x7f714006f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:01.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.800+0000 7f714659b640 1 -- 172.21.15.5:0/783514672 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714006eb10 con 0x7f71400cfda0 2024-05-20T18:29:01.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.800+0000 7f7145d9a640 1 --2- 172.21.15.5:0/783514672 >> v2:172.21.15.63:3300/0 conn(0x7f71400b3af0 0x7f714006f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:29:01.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.801+0000 7f714659b640 1 --2- 172.21.15.5:0/783514672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71400cfda0 0x7f714015be20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f714006dad0 tx=0x7f713c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d0f03dc357f4a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:01.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.801+0000 7f7145599640 1 -- 172.21.15.5:0/783514672 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c037070 con 0x7f71400cfda0 2024-05-20T18:29:01.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.801+0000 7f7145599640 1 -- 172.21.15.5:0/783514672 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f713c032440 con 0x7f71400cfda0 2024-05-20T18:29:01.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.801+0000 7f7145599640 1 -- 172.21.15.5:0/783514672 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c032810 con 0x7f71400cfda0 2024-05-20T18:29:01.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.802+0000 7f714759d640 1 -- 172.21.15.5:0/783514672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71400cfda0 msgr2=0x7f714015be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:01.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.802+0000 7f714759d640 1 --2- 172.21.15.5:0/783514672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71400cfda0 0x7f714015be20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f714006dad0 tx=0x7f713c02a140 comp rx=0 tx=0).stop 2024-05-20T18:29:01.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.802+0000 7f714759d640 1 -- 172.21.15.5:0/783514672 shutdown_connections 2024-05-20T18:29:01.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.802+0000 7f714759d640 1 --2- 172.21.15.5:0/783514672 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f714006c530 0x7f714006c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:01.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.802+0000 7f714759d640 1 --2- 172.21.15.5:0/783514672 >> v2:172.21.15.63:3300/0 conn(0x7f71400b3af0 0x7f714006f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:01.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.802+0000 7f714759d640 1 --2- 172.21.15.5:0/783514672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71400cfda0 0x7f714015be20 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:01.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.802+0000 7f714759d640 1 -- 172.21.15.5:0/783514672 >> 172.21.15.5:0/783514672 conn(0x7f71400721d0 msgr2=0x7f71400725e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:01.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.802+0000 7f714759d640 1 -- 172.21.15.5:0/783514672 shutdown_connections 2024-05-20T18:29:01.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.802+0000 7f714759d640 1 -- 172.21.15.5:0/783514672 wait complete. 2024-05-20T18:29:01.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.803+0000 7f714759d640 1 Processor -- start 2024-05-20T18:29:01.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.804+0000 7f714759d640 1 -- start start 2024-05-20T18:29:01.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.805+0000 7f714759d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f714006c530 0x7f7140150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:01.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.805+0000 7f714759d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f71400b3af0 0x7f714015e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:01.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.805+0000 7f714759d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71400cfda0 0x7f714015eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:01.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.805+0000 7f714759d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f714006f3b0 con 0x7f714006c530 2024-05-20T18:29:01.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.805+0000 7f714759d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f714006f0d0 con 0x7f71400cfda0 2024-05-20T18:29:01.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.805+0000 7f714759d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f71400b3f00 con 0x7f71400b3af0 2024-05-20T18:29:01.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.805+0000 7f714659b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f714006c530 0x7f7140150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:01.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.805+0000 7f714659b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f714006c530 0x7f7140150390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59816/0 (socket says 172.21.15.5:59816) 2024-05-20T18:29:01.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.805+0000 7f714659b640 1 -- 172.21.15.5:0/3586051030 learned_addr learned my addr 172.21.15.5:0/3586051030 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:01.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.805+0000 7f7145d9a640 1 --2- 172.21.15.5:0/3586051030 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f71400b3af0 0x7f714015e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:01.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.805+0000 7f7146d9c640 1 --2- 172.21.15.5:0/3586051030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71400cfda0 0x7f714015eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:01.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.805+0000 7f714659b640 1 -- 172.21.15.5:0/3586051030 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f71400b3af0 msgr2=0x7f714015e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:01.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.805+0000 7f714659b640 1 --2- 172.21.15.5:0/3586051030 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f71400b3af0 0x7f714015e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:01.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.805+0000 7f714659b640 1 -- 172.21.15.5:0/3586051030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71400cfda0 msgr2=0x7f714015eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:01.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.805+0000 7f714659b640 1 --2- 172.21.15.5:0/3586051030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71400cfda0 0x7f714015eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:01.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.805+0000 7f714659b640 1 -- 172.21.15.5:0/3586051030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714015c6f0 con 0x7f714006c530 2024-05-20T18:29:01.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.805+0000 7f714659b640 1 --2- 172.21.15.5:0/3586051030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f714006c530 0x7f7140150390 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f713c000c00 tx=0x7f713c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:01.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.806+0000 7f7136ffd640 1 -- 172.21.15.5:0/3586051030 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c037040 con 0x7f714006c530 2024-05-20T18:29:01.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.806+0000 7f7136ffd640 1 -- 172.21.15.5:0/3586051030 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f713c042070 con 0x7f714006c530 2024-05-20T18:29:01.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.806+0000 7f714759d640 1 -- 172.21.15.5:0/3586051030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f714015c9d0 con 0x7f714006c530 2024-05-20T18:29:01.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.806+0000 7f7136ffd640 1 -- 172.21.15.5:0/3586051030 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c030bb0 con 0x7f714006c530 2024-05-20T18:29:01.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.806+0000 7f714759d640 1 -- 172.21.15.5:0/3586051030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71401530b0 con 0x7f714006c530 2024-05-20T18:29:01.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.807+0000 7f714759d640 1 -- 172.21.15.5:0/3586051030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7140075980 con 0x7f714006c530 2024-05-20T18:29:01.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.808+0000 7f7136ffd640 1 -- 172.21.15.5:0/3586051030 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f713c047460 con 0x7f714006c530 2024-05-20T18:29:01.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.808+0000 7f7136ffd640 1 --2- 172.21.15.5:0/3586051030 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f712406be30 0x7f712406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:01.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.808+0000 7f7145d9a640 1 --2- 172.21.15.5:0/3586051030 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f712406be30 0x7f712406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:01.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.808+0000 7f7136ffd640 1 -- 172.21.15.5:0/3586051030 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6001+0+0 (secure 0 0 0) 0x7f713c0abb20 con 0x7f714006c530 2024-05-20T18:29:01.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.809+0000 7f7145d9a640 1 --2- 172.21.15.5:0/3586051030 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f712406be30 0x7f712406e340 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7140154b20 tx=0x7f7130008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:01.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:01.813+0000 7f7136ffd640 1 -- 172.21.15.5:0/3586051030 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f713c079d50 con 0x7f714006c530 2024-05-20T18:29:02.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.037+0000 7f714759d640 1 -- 172.21.15.5:0/3586051030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f71400ccdf0 con 0x7f714006c530 2024-05-20T18:29:02.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.651+0000 7f7136ffd640 1 -- 172.21.15.5:0/3586051030 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v22) ==== 172+0+0 (secure 0 0 0) 0x7f713c07e050 con 0x7f714006c530 2024-05-20T18:29:02.650 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 7 and data pool 8 2024-05-20T18:29:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.655+0000 7f714759d640 1 -- 172.21.15.5:0/3586051030 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f712406be30 msgr2=0x7f712406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.655+0000 7f714759d640 1 --2- 172.21.15.5:0/3586051030 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f712406be30 0x7f712406e340 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7140154b20 tx=0x7f7130008040 comp rx=0 tx=0).stop 2024-05-20T18:29:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.655+0000 7f714759d640 1 -- 172.21.15.5:0/3586051030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f714006c530 msgr2=0x7f7140150390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.655+0000 7f714759d640 1 --2- 172.21.15.5:0/3586051030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f714006c530 0x7f7140150390 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f713c000c00 tx=0x7f713c0046c0 comp rx=0 tx=0).stop 2024-05-20T18:29:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.656+0000 7f714759d640 1 -- 172.21.15.5:0/3586051030 shutdown_connections 2024-05-20T18:29:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.656+0000 7f714759d640 1 --2- 172.21.15.5:0/3586051030 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f712406be30 0x7f712406e340 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.656+0000 7f714759d640 1 --2- 172.21.15.5:0/3586051030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71400cfda0 0x7f714015eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.656+0000 7f714759d640 1 --2- 172.21.15.5:0/3586051030 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f71400b3af0 0x7f714015e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.656+0000 7f714759d640 1 --2- 172.21.15.5:0/3586051030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f714006c530 0x7f7140150390 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:02.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.656+0000 7f714759d640 1 -- 172.21.15.5:0/3586051030 >> 172.21.15.5:0/3586051030 conn(0x7f71400721d0 msgr2=0x7f71400b0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:02.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.658+0000 7f714759d640 1 -- 172.21.15.5:0/3586051030 shutdown_connections 2024-05-20T18:29:02.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.660+0000 7f714759d640 1 -- 172.21.15.5:0/3586051030 wait complete. 2024-05-20T18:29:02.691 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:29:02.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.812+0000 7fc2a29fe640 1 Processor -- start 2024-05-20T18:29:02.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.812+0000 7fc2a29fe640 1 -- start start 2024-05-20T18:29:02.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.813+0000 7fc2a29fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc29c06d3d0 0x7fc29c06d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:02.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.813+0000 7fc2a29fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc29c0b8190 0x7fc29c0b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:02.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.813+0000 7fc2a29fe640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc29c06dd20 con 0x7fc29c162820 2024-05-20T18:29:02.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.813+0000 7fc2a29fe640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc29c0d0630 con 0x7fc29c0b8190 2024-05-20T18:29:02.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.813+0000 7fc2a29fe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc29c0d39c0 con 0x7fc29c06d3d0 2024-05-20T18:29:02.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.813+0000 7fc2a0f9b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc29c0b8190 0x7fc29c0b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:02.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.814+0000 7fc29bfff640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fc29c162820 0x7fc29c0b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:43874/0 (socket says 172.21.15.5:43874) 2024-05-20T18:29:02.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.814+0000 7fc29bfff640 1 -- 172.21.15.5:0/1093016637 learned_addr learned my addr 172.21.15.5:0/1093016637 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:02.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.814+0000 7fc29b7fe640 1 --2- 172.21.15.5:0/1093016637 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc29c06d3d0 0x7fc29c06d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:02.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.814+0000 7fc2a0f9b640 1 -- 172.21.15.5:0/1093016637 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc29c06d3d0 msgr2=0x7fc29c06d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:02.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.814+0000 7fc2a0f9b640 1 --2- 172.21.15.5:0/1093016637 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc29c06d3d0 0x7fc29c06d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:02.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.814+0000 7fc2a0f9b640 1 -- 172.21.15.5:0/1093016637 >> v1:172.21.15.63:6789/0 conn(0x7fc29c162820 legacy=0x7fc29c0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:02.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.814+0000 7fc29affd640 1 -- 172.21.15.5:0/1093016637 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4014299604 0 0) 0x7fc28c001110 con 0x7fc29c162820 2024-05-20T18:29:02.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.814+0000 7fc2a0f9b640 1 -- 172.21.15.5:0/1093016637 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29c0d3ca0 con 0x7fc29c0b8190 2024-05-20T18:29:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.815+0000 7fc2a0f9b640 1 --2- 172.21.15.5:0/1093016637 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc29c0b8190 0x7fc29c0b85a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc294009d90 tx=0x7fc29402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5a99dd6ced07346 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.815+0000 7fc29affd640 1 -- 172.21.15.5:0/1093016637 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc294007740 con 0x7fc29c0b8190 2024-05-20T18:29:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.815+0000 7fc29affd640 1 -- 172.21.15.5:0/1093016637 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2940078e0 con 0x7fc29c0b8190 2024-05-20T18:29:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.815+0000 7fc29affd640 1 -- 172.21.15.5:0/1093016637 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc294007cb0 con 0x7fc29c0b8190 2024-05-20T18:29:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.816+0000 7fc2a29fe640 1 -- 172.21.15.5:0/1093016637 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc29c0b8190 msgr2=0x7fc29c0b85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.816+0000 7fc2a29fe640 1 --2- 172.21.15.5:0/1093016637 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc29c0b8190 0x7fc29c0b85a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc294009d90 tx=0x7fc29402a660 comp rx=0 tx=0).stop 2024-05-20T18:29:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.816+0000 7fc2a29fe640 1 -- 172.21.15.5:0/1093016637 shutdown_connections 2024-05-20T18:29:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.816+0000 7fc2a29fe640 1 --2- 172.21.15.5:0/1093016637 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc29c0b8190 0x7fc29c0b85a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.816+0000 7fc2a29fe640 1 --2- 172.21.15.5:0/1093016637 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc29c06d3d0 0x7fc29c06d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.816+0000 7fc2a29fe640 1 -- 172.21.15.5:0/1093016637 >> 172.21.15.5:0/1093016637 conn(0x7fc29c0d4bd0 msgr2=0x7fc29c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.816+0000 7fc2a29fe640 1 -- 172.21.15.5:0/1093016637 shutdown_connections 2024-05-20T18:29:02.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.816+0000 7fc2a29fe640 1 -- 172.21.15.5:0/1093016637 wait complete. 2024-05-20T18:29:02.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.818+0000 7fc2a29fe640 1 Processor -- start 2024-05-20T18:29:02.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.818+0000 7fc2a29fe640 1 -- start start 2024-05-20T18:29:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.819+0000 7fc2a29fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc29c06d3d0 0x7fc29c0b6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.819+0000 7fc2a29fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc29c0b8190 0x7fc29c0cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.819+0000 7fc2a29fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc29c162820 0x7fc29c0cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.819+0000 7fc2a29fe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c0d0300 con 0x7fc29c162820 2024-05-20T18:29:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.819+0000 7fc2a29fe640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c0d4260 con 0x7fc29c0b8190 2024-05-20T18:29:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.819+0000 7fc2a29fe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc29c0b4050 con 0x7fc29c06d3d0 2024-05-20T18:29:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.819+0000 7fc2a0f9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc29c162820 0x7fc29c0cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.819+0000 7fc2a0f9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc29c162820 0x7fc29c0cb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59832/0 (socket says 172.21.15.5:59832) 2024-05-20T18:29:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.819+0000 7fc2a0f9b640 1 -- 172.21.15.5:0/4076140245 learned_addr learned my addr 172.21.15.5:0/4076140245 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.819+0000 7fc29bfff640 1 --2- 172.21.15.5:0/4076140245 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc29c06d3d0 0x7fc29c0b6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.820+0000 7fc29b7fe640 1 --2- 172.21.15.5:0/4076140245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc29c0b8190 0x7fc29c0cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.820+0000 7fc2a0f9b640 1 -- 172.21.15.5:0/4076140245 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc29c06d3d0 msgr2=0x7fc29c0b6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.820+0000 7fc2a0f9b640 1 --2- 172.21.15.5:0/4076140245 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc29c06d3d0 0x7fc29c0b6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.820+0000 7fc2a0f9b640 1 -- 172.21.15.5:0/4076140245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc29c0b8190 msgr2=0x7fc29c0cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.820+0000 7fc2a0f9b640 1 --2- 172.21.15.5:0/4076140245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc29c0b8190 0x7fc29c0cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.820+0000 7fc2a0f9b640 1 -- 172.21.15.5:0/4076140245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29c0b5400 con 0x7fc29c162820 2024-05-20T18:29:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.820+0000 7fc29bfff640 1 --2- 172.21.15.5:0/4076140245 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc29c06d3d0 0x7fc29c0b6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:29:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.820+0000 7fc29b7fe640 1 --2- 172.21.15.5:0/4076140245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc29c0b8190 0x7fc29c0cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:29:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.820+0000 7fc2a0f9b640 1 --2- 172.21.15.5:0/4076140245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc29c162820 0x7fc29c0cb250 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc294009eb0 tx=0x7fc294004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.820+0000 7fc298ff9640 1 -- 172.21.15.5:0/4076140245 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29403f070 con 0x7fc29c162820 2024-05-20T18:29:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.820+0000 7fc298ff9640 1 -- 172.21.15.5:0/4076140245 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc294031c40 con 0x7fc29c162820 2024-05-20T18:29:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.820+0000 7fc298ff9640 1 -- 172.21.15.5:0/4076140245 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc294009450 con 0x7fc29c162820 2024-05-20T18:29:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.820+0000 7fc2a29fe640 1 -- 172.21.15.5:0/4076140245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc29c0c90a0 con 0x7fc29c162820 2024-05-20T18:29:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.820+0000 7fc2a29fe640 1 -- 172.21.15.5:0/4076140245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc29c15ac40 con 0x7fc29c162820 2024-05-20T18:29:02.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.821+0000 7fc2a29fe640 1 -- 172.21.15.5:0/4076140245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc29c0c9310 con 0x7fc29c162820 2024-05-20T18:29:02.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.822+0000 7fc298ff9640 1 -- 172.21.15.5:0/4076140245 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc294003990 con 0x7fc29c162820 2024-05-20T18:29:02.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.822+0000 7fc298ff9640 1 --2- 172.21.15.5:0/4076140245 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc27c06be60 0x7fc27c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:02.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.822+0000 7fc298ff9640 1 -- 172.21.15.5:0/4076140245 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7fc2940abd90 con 0x7fc29c162820 2024-05-20T18:29:02.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.823+0000 7fc29bfff640 1 --2- 172.21.15.5:0/4076140245 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc27c06be60 0x7fc27c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:02.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.823+0000 7fc29bfff640 1 --2- 172.21.15.5:0/4076140245 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc27c06be60 0x7fc27c06e370 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc28c000f60 tx=0x7fc28c007ca0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:02.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:02.827+0000 7fc298ff9640 1 -- 172.21.15.5:0/4076140245 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc29407a520 con 0x7fc29c162820 2024-05-20T18:29:03.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.010+0000 7fc2a29fe640 1 -- 172.21.15.5:0/4076140245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc29c075b00 con 0x7fc29c162820 2024-05-20T18:29:03.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.011+0000 7fc298ff9640 1 -- 172.21.15.5:0/4076140245 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+16835 (secure 0 0 0) 0x7fc294079e20 con 0x7fc29c162820 2024-05-20T18:29:03.011 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:29:03.011 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":50,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:29:02.037996+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:29:00.401866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-20T18:29:01.006122+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:29:03.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.013+0000 7fc2a29fe640 1 -- 172.21.15.5:0/4076140245 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc27c06be60 msgr2=0x7fc27c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:03.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.013+0000 7fc2a29fe640 1 --2- 172.21.15.5:0/4076140245 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc27c06be60 0x7fc27c06e370 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc28c000f60 tx=0x7fc28c007ca0 comp rx=0 tx=0).stop 2024-05-20T18:29:03.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.013+0000 7fc2a29fe640 1 -- 172.21.15.5:0/4076140245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc29c162820 msgr2=0x7fc29c0cb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:03.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.013+0000 7fc2a29fe640 1 --2- 172.21.15.5:0/4076140245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc29c162820 0x7fc29c0cb250 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc294009eb0 tx=0x7fc294004a40 comp rx=0 tx=0).stop 2024-05-20T18:29:03.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.013+0000 7fc2a29fe640 1 -- 172.21.15.5:0/4076140245 shutdown_connections 2024-05-20T18:29:03.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.013+0000 7fc2a29fe640 1 --2- 172.21.15.5:0/4076140245 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc27c06be60 0x7fc27c06e370 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:03.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.013+0000 7fc2a29fe640 1 --2- 172.21.15.5:0/4076140245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc29c162820 0x7fc29c0cb250 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:03.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.013+0000 7fc2a29fe640 1 --2- 172.21.15.5:0/4076140245 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc29c0b8190 0x7fc29c0cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:03.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.013+0000 7fc2a29fe640 1 --2- 172.21.15.5:0/4076140245 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc29c06d3d0 0x7fc29c0b6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:03.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.013+0000 7fc2a29fe640 1 -- 172.21.15.5:0/4076140245 >> 172.21.15.5:0/4076140245 conn(0x7fc29c0d4bd0 msgr2=0x7fc29c0ba5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:03.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.013+0000 7fc2a29fe640 1 -- 172.21.15.5:0/4076140245 shutdown_connections 2024-05-20T18:29:03.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.013+0000 7fc2a29fe640 1 -- 172.21.15.5:0/4076140245 wait complete. 2024-05-20T18:29:03.025 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:29:03.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.140+0000 7ffbd8f7e640 1 Processor -- start 2024-05-20T18:29:03.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.140+0000 7ffbd8f7e640 1 -- start start 2024-05-20T18:29:03.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.141+0000 7ffbd8f7e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ffbd415d530 0x7ffbd415aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:03.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.141+0000 7ffbd8f7e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ffbd40b1700 0x7ffbd40b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:03.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.141+0000 7ffbd8f7e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbd40cd580 con 0x7ffbd4070850 2024-05-20T18:29:03.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.141+0000 7ffbd8f7e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7ffbd40ce190 con 0x7ffbd415d530 2024-05-20T18:29:03.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.141+0000 7ffbd8f7e640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ffbd40cd860 con 0x7ffbd40b1700 2024-05-20T18:29:03.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.141+0000 7ffbd259c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ffbd415d530 0x7ffbd415aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:03.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.141+0000 7ffbd259c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ffbd415d530 0x7ffbd415aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59846/0 (socket says 172.21.15.5:59846) 2024-05-20T18:29:03.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.141+0000 7ffbd259c640 1 -- 172.21.15.5:0/1143594105 learned_addr learned my addr 172.21.15.5:0/1143594105 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:03.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.142+0000 7ffbcbfff640 1 --2- 172.21.15.5:0/1143594105 >> v2:172.21.15.63:3300/0 conn(0x7ffbd40b1700 0x7ffbd40b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:03.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.142+0000 7ffbd259c640 1 -- 172.21.15.5:0/1143594105 >> v2:172.21.15.63:3300/0 conn(0x7ffbd40b1700 msgr2=0x7ffbd40b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:03.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.142+0000 7ffbd259c640 1 --2- 172.21.15.5:0/1143594105 >> v2:172.21.15.63:3300/0 conn(0x7ffbd40b1700 0x7ffbd40b1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:03.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.142+0000 7ffbd259c640 1 -- 172.21.15.5:0/1143594105 >> v1:172.21.15.63:6789/0 conn(0x7ffbd4070850 legacy=0x7ffbd4070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:03.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.142+0000 7ffbd259c640 1 -- 172.21.15.5:0/1143594105 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd40cdb40 con 0x7ffbd415d530 2024-05-20T18:29:03.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.142+0000 7ffbd259c640 1 --2- 172.21.15.5:0/1143594105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd415d530 0x7ffbd415aaa0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ffbd40d3ad0 tx=0x7ffbc402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b6de5c68a902d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:03.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.143+0000 7ffbd1d9b640 1 -- 172.21.15.5:0/1143594105 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4037070 con 0x7ffbd415d530 2024-05-20T18:29:03.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.143+0000 7ffbd1d9b640 1 -- 172.21.15.5:0/1143594105 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbc4032440 con 0x7ffbd415d530 2024-05-20T18:29:03.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.143+0000 7ffbd1d9b640 1 -- 172.21.15.5:0/1143594105 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4032810 con 0x7ffbd415d530 2024-05-20T18:29:03.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.143+0000 7ffbd8f7e640 1 -- 172.21.15.5:0/1143594105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd415d530 msgr2=0x7ffbd415aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:03.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.143+0000 7ffbd8f7e640 1 --2- 172.21.15.5:0/1143594105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd415d530 0x7ffbd415aaa0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ffbd40d3ad0 tx=0x7ffbc402a270 comp rx=0 tx=0).stop 2024-05-20T18:29:03.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.143+0000 7ffbd8f7e640 1 -- 172.21.15.5:0/1143594105 shutdown_connections 2024-05-20T18:29:03.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.143+0000 7ffbd8f7e640 1 --2- 172.21.15.5:0/1143594105 >> v2:172.21.15.63:3300/0 conn(0x7ffbd40b1700 0x7ffbd40b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:03.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.143+0000 7ffbd8f7e640 1 --2- 172.21.15.5:0/1143594105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd415d530 0x7ffbd415aaa0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:03.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.143+0000 7ffbd8f7e640 1 -- 172.21.15.5:0/1143594105 >> 172.21.15.5:0/1143594105 conn(0x7ffbd40d29d0 msgr2=0x7ffbd40d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:03.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.143+0000 7ffbd8f7e640 1 -- 172.21.15.5:0/1143594105 shutdown_connections 2024-05-20T18:29:03.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.143+0000 7ffbd8f7e640 1 -- 172.21.15.5:0/1143594105 wait complete. 2024-05-20T18:29:03.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.145+0000 7ffbd8f7e640 1 Processor -- start 2024-05-20T18:29:03.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.145+0000 7ffbd8f7e640 1 -- start start 2024-05-20T18:29:03.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.147+0000 7ffbd8f7e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffbd4070850 0x7ffbd40c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:03.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.147+0000 7ffbd8f7e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd40b1700 0x7ffbd40caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:03.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.147+0000 7ffbd8f7e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbd415d530 0x7ffbd40cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:03.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.147+0000 7ffbd8f7e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd40b2050 con 0x7ffbd40b1700 2024-05-20T18:29:03.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.147+0000 7ffbd8f7e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd40cd230 con 0x7ffbd415d530 2024-05-20T18:29:03.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.147+0000 7ffbd8f7e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffbd4071370 con 0x7ffbd4070850 2024-05-20T18:29:03.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.147+0000 7ffbcbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd40b1700 0x7ffbd40caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:03.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.147+0000 7ffbcbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd40b1700 0x7ffbd40caa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59854/0 (socket says 172.21.15.5:59854) 2024-05-20T18:29:03.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.147+0000 7ffbcbfff640 1 -- 172.21.15.5:0/3940172564 learned_addr learned my addr 172.21.15.5:0/3940172564 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:03.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.147+0000 7ffbd2d9d640 1 --2- 172.21.15.5:0/3940172564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbd415d530 0x7ffbd40cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:03.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.147+0000 7ffbd259c640 1 --2- 172.21.15.5:0/3940172564 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffbd4070850 0x7ffbd40c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:03.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.147+0000 7ffbcbfff640 1 -- 172.21.15.5:0/3940172564 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffbd4070850 msgr2=0x7ffbd40c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:03.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.147+0000 7ffbcbfff640 1 --2- 172.21.15.5:0/3940172564 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffbd4070850 0x7ffbd40c6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:03.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.147+0000 7ffbcbfff640 1 -- 172.21.15.5:0/3940172564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbd415d530 msgr2=0x7ffbd40cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:03.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.147+0000 7ffbcbfff640 1 --2- 172.21.15.5:0/3940172564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbd415d530 0x7ffbd40cb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:03.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.147+0000 7ffbcbfff640 1 -- 172.21.15.5:0/3940172564 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd40c6a40 con 0x7ffbd40b1700 2024-05-20T18:29:03.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.147+0000 7ffbcbfff640 1 --2- 172.21.15.5:0/3940172564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd40b1700 0x7ffbd40caa00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc00e8a0 tx=0x7ffbbc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:03.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.147+0000 7ffbcaffd640 1 -- 172.21.15.5:0/3940172564 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc017070 con 0x7ffbd40b1700 2024-05-20T18:29:03.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.148+0000 7ffbcaffd640 1 -- 172.21.15.5:0/3940172564 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbbc00c940 con 0x7ffbd40b1700 2024-05-20T18:29:03.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.148+0000 7ffbcaffd640 1 -- 172.21.15.5:0/3940172564 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc00cd30 con 0x7ffbd40b1700 2024-05-20T18:29:03.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.148+0000 7ffbd8f7e640 1 -- 172.21.15.5:0/3940172564 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbd40c9c40 con 0x7ffbd40b1700 2024-05-20T18:29:03.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.148+0000 7ffbd8f7e640 1 -- 172.21.15.5:0/3940172564 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbd40ca200 con 0x7ffbd40b1700 2024-05-20T18:29:03.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.148+0000 7ffbd8f7e640 1 -- 172.21.15.5:0/3940172564 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbd4075b00 con 0x7ffbd40b1700 2024-05-20T18:29:03.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.149+0000 7ffbcaffd640 1 -- 172.21.15.5:0/3940172564 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffbbc01a070 con 0x7ffbd40b1700 2024-05-20T18:29:03.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.150+0000 7ffbcaffd640 1 --2- 172.21.15.5:0/3940172564 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffbb406be30 0x7ffbb406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:03.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.150+0000 7ffbcaffd640 1 -- 172.21.15.5:0/3940172564 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7ffbbc08b230 con 0x7ffbd40b1700 2024-05-20T18:29:03.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.150+0000 7ffbd259c640 1 --2- 172.21.15.5:0/3940172564 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffbb406be30 0x7ffbb406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:03.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.150+0000 7ffbd259c640 1 --2- 172.21.15.5:0/3940172564 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffbb406be30 0x7ffbb406e340 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4034000 tx=0x7ffbc402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:03.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.154+0000 7ffbcaffd640 1 -- 172.21.15.5:0/3940172564 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ffbbc059360 con 0x7ffbd40b1700 2024-05-20T18:29:03.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.338+0000 7ffbd8f7e640 1 -- 172.21.15.5:0/3940172564 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffbd40cea50 con 0x7ffbd40b1700 2024-05-20T18:29:03.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.339+0000 7ffbcaffd640 1 -- 172.21.15.5:0/3940172564 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+16835 (secure 0 0 0) 0x7ffbbc05d660 con 0x7ffbd40b1700 2024-05-20T18:29:03.339 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:29:03.339 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":50,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:29:02.037996+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:29:00.401866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-20T18:29:01.006122+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:29:03.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.341+0000 7ffbd8f7e640 1 -- 172.21.15.5:0/3940172564 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffbb406be30 msgr2=0x7ffbb406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:03.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.341+0000 7ffbd8f7e640 1 --2- 172.21.15.5:0/3940172564 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffbb406be30 0x7ffbb406e340 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4034000 tx=0x7ffbc402e000 comp rx=0 tx=0).stop 2024-05-20T18:29:03.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.341+0000 7ffbd8f7e640 1 -- 172.21.15.5:0/3940172564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd40b1700 msgr2=0x7ffbd40caa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:03.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.341+0000 7ffbd8f7e640 1 --2- 172.21.15.5:0/3940172564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd40b1700 0x7ffbd40caa00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc00e8a0 tx=0x7ffbbc00ed60 comp rx=0 tx=0).stop 2024-05-20T18:29:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.341+0000 7ffbd8f7e640 1 -- 172.21.15.5:0/3940172564 shutdown_connections 2024-05-20T18:29:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.341+0000 7ffbd8f7e640 1 --2- 172.21.15.5:0/3940172564 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffbb406be30 0x7ffbb406e340 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.341+0000 7ffbd8f7e640 1 --2- 172.21.15.5:0/3940172564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbd415d530 0x7ffbd40cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.341+0000 7ffbd8f7e640 1 --2- 172.21.15.5:0/3940172564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd40b1700 0x7ffbd40caa00 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.341+0000 7ffbd8f7e640 1 --2- 172.21.15.5:0/3940172564 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffbd4070850 0x7ffbd40c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.341+0000 7ffbd8f7e640 1 -- 172.21.15.5:0/3940172564 >> 172.21.15.5:0/3940172564 conn(0x7ffbd40d29d0 msgr2=0x7ffbd415f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.342+0000 7ffbd8f7e640 1 -- 172.21.15.5:0/3940172564 shutdown_connections 2024-05-20T18:29:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.342+0000 7ffbd8f7e640 1 -- 172.21.15.5:0/3940172564 wait complete. 2024-05-20T18:29:03.353 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-20T18:29:03.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.472+0000 7f5826d9d640 1 Processor -- start 2024-05-20T18:29:03.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.472+0000 7f5826d9d640 1 -- start start 2024-05-20T18:29:03.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.473+0000 7f5826d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f58200cf860 0x7f58200b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:03.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.473+0000 7f5826d9d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5820163520 0x7f58200d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:03.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.473+0000 7f5826d9d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58201605a0 con 0x7f5820163930 2024-05-20T18:29:03.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.473+0000 7f5826d9d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f582015ffe0 con 0x7f58200cf860 2024-05-20T18:29:03.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.473+0000 7f5826d9d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f58201602c0 con 0x7f5820163520 2024-05-20T18:29:03.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.473+0000 7f5825d9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f58200cf860 0x7f58200b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:03.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.473+0000 7f5825d9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f58200cf860 0x7f58200b3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59866/0 (socket says 172.21.15.5:59866) 2024-05-20T18:29:03.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.473+0000 7f582659c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5820163930 0x7f58200d3140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38700/0 (socket says 172.21.15.5:38700) 2024-05-20T18:29:03.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.473+0000 7f5825d9b640 1 -- 172.21.15.5:0/3194338057 learned_addr learned my addr 172.21.15.5:0/3194338057 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:03.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.473+0000 7f582559a640 1 --2- 172.21.15.5:0/3194338057 >> v2:172.21.15.63:3300/0 conn(0x7f5820163520 0x7f58200d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:03.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.474+0000 7f5825d9b640 1 -- 172.21.15.5:0/3194338057 >> v2:172.21.15.63:3300/0 conn(0x7f5820163520 msgr2=0x7f58200d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:03.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.474+0000 7f5825d9b640 1 --2- 172.21.15.5:0/3194338057 >> v2:172.21.15.63:3300/0 conn(0x7f5820163520 0x7f58200d12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:03.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.474+0000 7f5825d9b640 1 -- 172.21.15.5:0/3194338057 >> v1:172.21.15.5:6789/0 conn(0x7f5820163930 legacy=0x7f58200d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:03.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.474+0000 7f5825d9b640 1 -- 172.21.15.5:0/3194338057 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5820162d20 con 0x7f58200cf860 2024-05-20T18:29:03.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.474+0000 7f5824d99640 1 -- 172.21.15.5:0/3194338057 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 522445046 0 0) 0x7f58201605a0 con 0x7f5820163930 2024-05-20T18:29:03.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.474+0000 7f5825d9b640 1 --2- 172.21.15.5:0/3194338057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58200cf860 0x7f58200b3d70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f58200d2c50 tx=0x7f581002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a7fcf1b9015902 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:03.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.475+0000 7f5824d99640 1 -- 172.21.15.5:0/3194338057 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5810036070 con 0x7f58200cf860 2024-05-20T18:29:03.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.475+0000 7f5824d99640 1 -- 172.21.15.5:0/3194338057 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5810031600 con 0x7f58200cf860 2024-05-20T18:29:03.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.475+0000 7f5824d99640 1 -- 172.21.15.5:0/3194338057 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58100319d0 con 0x7f58200cf860 2024-05-20T18:29:03.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.475+0000 7f5826d9d640 1 -- 172.21.15.5:0/3194338057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58200cf860 msgr2=0x7f58200b3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:03.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.475+0000 7f5826d9d640 1 --2- 172.21.15.5:0/3194338057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58200cf860 0x7f58200b3d70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f58200d2c50 tx=0x7f581002a6a0 comp rx=0 tx=0).stop 2024-05-20T18:29:03.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.475+0000 7f5826d9d640 1 -- 172.21.15.5:0/3194338057 shutdown_connections 2024-05-20T18:29:03.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.475+0000 7f5826d9d640 1 --2- 172.21.15.5:0/3194338057 >> v2:172.21.15.63:3300/0 conn(0x7f5820163520 0x7f58200d12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:03.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.475+0000 7f5826d9d640 1 --2- 172.21.15.5:0/3194338057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58200cf860 0x7f58200b3d70 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:03.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.475+0000 7f5826d9d640 1 -- 172.21.15.5:0/3194338057 >> 172.21.15.5:0/3194338057 conn(0x7f582006dbb0 msgr2=0x7f582006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:03.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.475+0000 7f5826d9d640 1 -- 172.21.15.5:0/3194338057 shutdown_connections 2024-05-20T18:29:03.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.475+0000 7f5826d9d640 1 -- 172.21.15.5:0/3194338057 wait complete. 2024-05-20T18:29:03.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.477+0000 7f5826d9d640 1 Processor -- start 2024-05-20T18:29:03.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.477+0000 7f5826d9d640 1 -- start start 2024-05-20T18:29:03.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.479+0000 7f5826d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58200cf860 0x7f58200cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:03.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.479+0000 7f5826d9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5820163520 0x7f5820157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:03.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.479+0000 7f5826d9d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5820163930 0x7f58201575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:03.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.479+0000 7f5826d9d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f58200d39d0 con 0x7f58200cf860 2024-05-20T18:29:03.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.479+0000 7f5826d9d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f58200d3850 con 0x7f5820163520 2024-05-20T18:29:03.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.479+0000 7f5826d9d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f58200d3b50 con 0x7f5820163930 2024-05-20T18:29:03.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.479+0000 7f5825d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58200cf860 0x7f58200cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:03.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.479+0000 7f5825d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58200cf860 0x7f58200cac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59876/0 (socket says 172.21.15.5:59876) 2024-05-20T18:29:03.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.479+0000 7f5825d9b640 1 -- 172.21.15.5:0/388408730 learned_addr learned my addr 172.21.15.5:0/388408730 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:03.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.479+0000 7f582559a640 1 --2- 172.21.15.5:0/388408730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5820163520 0x7f5820157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:03.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.479+0000 7f582659c640 1 --2- 172.21.15.5:0/388408730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5820163930 0x7f58201575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:03.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.479+0000 7f5825d9b640 1 -- 172.21.15.5:0/388408730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5820163930 msgr2=0x7f58201575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:03.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.479+0000 7f5825d9b640 1 --2- 172.21.15.5:0/388408730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5820163930 0x7f58201575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:03.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.479+0000 7f5825d9b640 1 -- 172.21.15.5:0/388408730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5820163520 msgr2=0x7f5820157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:03.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.479+0000 7f5825d9b640 1 --2- 172.21.15.5:0/388408730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5820163520 0x7f5820157090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:03.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.479+0000 7f5825d9b640 1 -- 172.21.15.5:0/388408730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58200ccba0 con 0x7f58200cf860 2024-05-20T18:29:03.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.479+0000 7f5825d9b640 1 --2- 172.21.15.5:0/388408730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58200cf860 0x7f58200cac30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5810034040 tx=0x7f5810004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:03.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.480+0000 7f581e7fc640 1 -- 172.21.15.5:0/388408730 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5810036050 con 0x7f58200cf860 2024-05-20T18:29:03.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.480+0000 7f581e7fc640 1 -- 172.21.15.5:0/388408730 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5810042070 con 0x7f58200cf860 2024-05-20T18:29:03.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.480+0000 7f5826d9d640 1 -- 172.21.15.5:0/388408730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58200cce80 con 0x7f58200cf860 2024-05-20T18:29:03.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.480+0000 7f581e7fc640 1 -- 172.21.15.5:0/388408730 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5810009510 con 0x7f58200cf860 2024-05-20T18:29:03.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.480+0000 7f5826d9d640 1 -- 172.21.15.5:0/388408730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58200cd440 con 0x7f58200cf860 2024-05-20T18:29:03.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.481+0000 7f5826d9d640 1 -- 172.21.15.5:0/388408730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5820075980 con 0x7f58200cf860 2024-05-20T18:29:03.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.481+0000 7f581e7fc640 1 -- 172.21.15.5:0/388408730 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5810007c70 con 0x7f58200cf860 2024-05-20T18:29:03.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.482+0000 7f581e7fc640 1 --2- 172.21.15.5:0/388408730 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f580c06be30 0x7f580c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:03.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.482+0000 7f581e7fc640 1 -- 172.21.15.5:0/388408730 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7f58100abd60 con 0x7f58200cf860 2024-05-20T18:29:03.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.482+0000 7f582559a640 1 --2- 172.21.15.5:0/388408730 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f580c06be30 0x7f580c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:03.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.483+0000 7f582559a640 1 --2- 172.21.15.5:0/388408730 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f580c06be30 0x7f580c06e340 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f58200c8430 tx=0x7f5814009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:03.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.487+0000 7f581e7fc640 1 -- 172.21.15.5:0/388408730 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5810079f30 con 0x7f58200cf860 2024-05-20T18:29:03.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:03.714+0000 7f5826d9d640 1 -- 172.21.15.5:0/388408730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f58200cf290 con 0x7f58200cf860 2024-05-20T18:29:04.631 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-20T18:29:04.632+0000 7f56f54ba640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T18:29:04.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.675+0000 7f581e7fc640 1 -- 172.21.15.5:0/388408730 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f581007e230 con 0x7f58200cf860 2024-05-20T18:29:04.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.678+0000 7f5826d9d640 1 -- 172.21.15.5:0/388408730 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f580c06be30 msgr2=0x7f580c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:04.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.678+0000 7f5826d9d640 1 --2- 172.21.15.5:0/388408730 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f580c06be30 0x7f580c06e340 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f58200c8430 tx=0x7f5814009190 comp rx=0 tx=0).stop 2024-05-20T18:29:04.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.678+0000 7f5826d9d640 1 -- 172.21.15.5:0/388408730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58200cf860 msgr2=0x7f58200cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:04.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.678+0000 7f5826d9d640 1 --2- 172.21.15.5:0/388408730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58200cf860 0x7f58200cac30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5810034040 tx=0x7f5810004a40 comp rx=0 tx=0).stop 2024-05-20T18:29:04.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.678+0000 7f5826d9d640 1 -- 172.21.15.5:0/388408730 shutdown_connections 2024-05-20T18:29:04.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.678+0000 7f5826d9d640 1 --2- 172.21.15.5:0/388408730 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f580c06be30 0x7f580c06e340 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:04.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.678+0000 7f5826d9d640 1 --2- 172.21.15.5:0/388408730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5820163930 0x7f58201575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:04.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.678+0000 7f5826d9d640 1 --2- 172.21.15.5:0/388408730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5820163520 0x7f5820157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:04.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.678+0000 7f5826d9d640 1 --2- 172.21.15.5:0/388408730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58200cf860 0x7f58200cac30 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:04.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.678+0000 7f5826d9d640 1 -- 172.21.15.5:0/388408730 >> 172.21.15.5:0/388408730 conn(0x7f582006dbb0 msgr2=0x7f58200d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:04.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.678+0000 7f5826d9d640 1 -- 172.21.15.5:0/388408730 shutdown_connections 2024-05-20T18:29:04.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.678+0000 7f5826d9d640 1 -- 172.21.15.5:0/388408730 wait complete. 2024-05-20T18:29:04.692 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-20T18:29:04.692 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-20T18:29:04.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.817+0000 7f46af9f5640 1 Processor -- start 2024-05-20T18:29:04.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.818+0000 7f46af9f5640 1 -- start start 2024-05-20T18:29:04.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.819+0000 7f46af9f5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f46a80cd680 0x7f46a815f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:04.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.819+0000 7f46af9f5640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f46a806c6c0 0x7f46a806cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:04.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.819+0000 7f46af9f5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46a80cf3d0 0x7f46a80cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.819+0000 7f46af9f5640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f46a80cda90 con 0x7f46a80cd680 2024-05-20T18:29:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.819+0000 7f46af9f5640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f46a815f760 con 0x7f46a806c6c0 2024-05-20T18:29:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.819+0000 7f46af9f5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f46a80cfd20 con 0x7f46a80cf3d0 2024-05-20T18:29:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.819+0000 7f46ad791640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f46a80cd680 0x7f46a815f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.819+0000 7f46ad791640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f46a80cd680 0x7f46a815f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42750/0 (socket says 172.21.15.5:42750) 2024-05-20T18:29:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.819+0000 7f46ad791640 1 -- 172.21.15.5:0/2333929847 learned_addr learned my addr 172.21.15.5:0/2333929847 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.819+0000 7f46adf92640 1 --2- 172.21.15.5:0/2333929847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46a80cf3d0 0x7f46a80cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.819+0000 7f46acf90640 1 --2- 172.21.15.5:0/2333929847 >> v2:172.21.15.63:3300/0 conn(0x7f46a806c6c0 0x7f46a806cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.819+0000 7f46ad791640 1 -- 172.21.15.5:0/2333929847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46a80cf3d0 msgr2=0x7f46a80cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.819+0000 7f46ad791640 1 --2- 172.21.15.5:0/2333929847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46a80cf3d0 0x7f46a80cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.819+0000 7f46ad791640 1 -- 172.21.15.5:0/2333929847 >> v2:172.21.15.63:3300/0 conn(0x7f46a806c6c0 msgr2=0x7f46a806cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.819+0000 7f46ad791640 1 --2- 172.21.15.5:0/2333929847 >> v2:172.21.15.63:3300/0 conn(0x7f46a806c6c0 0x7f46a806cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.819+0000 7f46ad791640 1 -- 172.21.15.5:0/2333929847 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a806d0f0 con 0x7f46a80cd680 2024-05-20T18:29:04.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.820+0000 7f46ad791640 1 --2- 172.21.15.5:0/2333929847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46a80cd680 0x7f46a815f220 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f46a8070110 tx=0x7f46a402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c09281aaa2c7bc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:04.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.820+0000 7f469ffff640 1 -- 172.21.15.5:0/2333929847 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a4037070 con 0x7f46a80cd680 2024-05-20T18:29:04.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.820+0000 7f469ffff640 1 -- 172.21.15.5:0/2333929847 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46a4032440 con 0x7f46a80cd680 2024-05-20T18:29:04.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.820+0000 7f469ffff640 1 -- 172.21.15.5:0/2333929847 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a4032810 con 0x7f46a80cd680 2024-05-20T18:29:04.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.821+0000 7f46af9f5640 1 -- 172.21.15.5:0/2333929847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46a80cd680 msgr2=0x7f46a815f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:04.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.821+0000 7f46af9f5640 1 --2- 172.21.15.5:0/2333929847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46a80cd680 0x7f46a815f220 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f46a8070110 tx=0x7f46a402a140 comp rx=0 tx=0).stop 2024-05-20T18:29:04.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.821+0000 7f46af9f5640 1 -- 172.21.15.5:0/2333929847 shutdown_connections 2024-05-20T18:29:04.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.821+0000 7f46af9f5640 1 --2- 172.21.15.5:0/2333929847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46a80cf3d0 0x7f46a80cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:04.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.821+0000 7f46af9f5640 1 --2- 172.21.15.5:0/2333929847 >> v2:172.21.15.63:3300/0 conn(0x7f46a806c6c0 0x7f46a806cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:04.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.821+0000 7f46af9f5640 1 --2- 172.21.15.5:0/2333929847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46a80cd680 0x7f46a815f220 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:04.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.821+0000 7f46af9f5640 1 -- 172.21.15.5:0/2333929847 >> 172.21.15.5:0/2333929847 conn(0x7f46a80b44d0 msgr2=0x7f46a80b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:04.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.821+0000 7f46af9f5640 1 -- 172.21.15.5:0/2333929847 shutdown_connections 2024-05-20T18:29:04.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.821+0000 7f46af9f5640 1 -- 172.21.15.5:0/2333929847 wait complete. 2024-05-20T18:29:04.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.823+0000 7f46af9f5640 1 Processor -- start 2024-05-20T18:29:04.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.823+0000 7f46af9f5640 1 -- start start 2024-05-20T18:29:04.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.824+0000 7f46af9f5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46a806c6c0 0x7f46a80b8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:04.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.824+0000 7f46af9f5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46a80cd680 0x7f46a80c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:04.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.824+0000 7f46af9f5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46a80cf3d0 0x7f46a80b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:04.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.824+0000 7f46af9f5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f46a80d0b40 con 0x7f46a80cf3d0 2024-05-20T18:29:04.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.824+0000 7f46af9f5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f46a806d450 con 0x7f46a806c6c0 2024-05-20T18:29:04.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.824+0000 7f46af9f5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f46a806d990 con 0x7f46a80cd680 2024-05-20T18:29:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.824+0000 7f46adf92640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46a80cf3d0 0x7f46a80b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.825+0000 7f46adf92640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46a80cf3d0 0x7f46a80b6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42756/0 (socket says 172.21.15.5:42756) 2024-05-20T18:29:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.825+0000 7f46adf92640 1 -- 172.21.15.5:0/3102179929 learned_addr learned my addr 172.21.15.5:0/3102179929 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.825+0000 7f46acf90640 1 --2- 172.21.15.5:0/3102179929 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46a80cd680 0x7f46a80c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.825+0000 7f46ad791640 1 --2- 172.21.15.5:0/3102179929 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46a806c6c0 0x7f46a80b8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.825+0000 7f46adf92640 1 -- 172.21.15.5:0/3102179929 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46a80cd680 msgr2=0x7f46a80c64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.825+0000 7f46adf92640 1 --2- 172.21.15.5:0/3102179929 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46a80cd680 0x7f46a80c64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.825+0000 7f46adf92640 1 -- 172.21.15.5:0/3102179929 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46a806c6c0 msgr2=0x7f46a80b8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.825+0000 7f46adf92640 1 --2- 172.21.15.5:0/3102179929 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46a806c6c0 0x7f46a80b8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.825+0000 7f46adf92640 1 -- 172.21.15.5:0/3102179929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a80b5970 con 0x7f46a80cf3d0 2024-05-20T18:29:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.825+0000 7f46adf92640 1 --2- 172.21.15.5:0/3102179929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46a80cf3d0 0x7f46a80b6a10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f46a0007e90 tx=0x7f46a000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.825+0000 7f469dffb640 1 -- 172.21.15.5:0/3102179929 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a0014420 con 0x7f46a80cf3d0 2024-05-20T18:29:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.825+0000 7f469dffb640 1 -- 172.21.15.5:0/3102179929 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46a0019070 con 0x7f46a80cf3d0 2024-05-20T18:29:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.825+0000 7f46af9f5640 1 -- 172.21.15.5:0/3102179929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a80b5c50 con 0x7f46a80cf3d0 2024-05-20T18:29:04.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.825+0000 7f469dffb640 1 -- 172.21.15.5:0/3102179929 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a0014800 con 0x7f46a80cf3d0 2024-05-20T18:29:04.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.826+0000 7f46af9f5640 1 -- 172.21.15.5:0/3102179929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a80b6210 con 0x7f46a80cf3d0 2024-05-20T18:29:04.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.826+0000 7f46af9f5640 1 -- 172.21.15.5:0/3102179929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46a8075b00 con 0x7f46a80cf3d0 2024-05-20T18:29:04.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.827+0000 7f469dffb640 1 -- 172.21.15.5:0/3102179929 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f46a00149a0 con 0x7f46a80cf3d0 2024-05-20T18:29:04.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.828+0000 7f469dffb640 1 --2- 172.21.15.5:0/3102179929 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f468c06be30 0x7f468c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:04.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.828+0000 7f469dffb640 1 -- 172.21.15.5:0/3102179929 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f46a008b820 con 0x7f46a80cf3d0 2024-05-20T18:29:04.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.828+0000 7f46ad791640 1 --2- 172.21.15.5:0/3102179929 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f468c06be30 0x7f468c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:04.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.828+0000 7f46ad791640 1 --2- 172.21.15.5:0/3102179929 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f468c06be30 0x7f468c06e340 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f46a4034000 tx=0x7f46a402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:04.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:04.832+0000 7f469dffb640 1 -- 172.21.15.5:0/3102179929 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f46a00598e0 con 0x7f46a80cf3d0 2024-05-20T18:29:05.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.053+0000 7f46af9f5640 1 -- 172.21.15.5:0/3102179929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f46a8152570 con 0x7f46a80cf3d0 2024-05-20T18:29:05.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.676+0000 7f469dffb640 1 -- 172.21.15.5:0/3102179929 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7f46a005dbe0 con 0x7f46a80cf3d0 2024-05-20T18:29:05.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.679+0000 7f46af9f5640 1 -- 172.21.15.5:0/3102179929 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f468c06be30 msgr2=0x7f468c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:05.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.679+0000 7f46af9f5640 1 --2- 172.21.15.5:0/3102179929 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f468c06be30 0x7f468c06e340 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f46a4034000 tx=0x7f46a402e000 comp rx=0 tx=0).stop 2024-05-20T18:29:05.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.679+0000 7f46af9f5640 1 -- 172.21.15.5:0/3102179929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46a80cf3d0 msgr2=0x7f46a80b6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:05.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.679+0000 7f46af9f5640 1 --2- 172.21.15.5:0/3102179929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46a80cf3d0 0x7f46a80b6a10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f46a0007e90 tx=0x7f46a000e4b0 comp rx=0 tx=0).stop 2024-05-20T18:29:05.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.679+0000 7f46af9f5640 1 -- 172.21.15.5:0/3102179929 shutdown_connections 2024-05-20T18:29:05.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.679+0000 7f46af9f5640 1 --2- 172.21.15.5:0/3102179929 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f468c06be30 0x7f468c06e340 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:05.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.679+0000 7f46af9f5640 1 --2- 172.21.15.5:0/3102179929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46a80cf3d0 0x7f46a80b6a10 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:05.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.679+0000 7f46af9f5640 1 --2- 172.21.15.5:0/3102179929 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46a80cd680 0x7f46a80c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:05.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.679+0000 7f46af9f5640 1 --2- 172.21.15.5:0/3102179929 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46a806c6c0 0x7f46a80b8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:05.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.679+0000 7f46af9f5640 1 -- 172.21.15.5:0/3102179929 >> 172.21.15.5:0/3102179929 conn(0x7f46a80b44d0 msgr2=0x7f46a8162c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:05.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.679+0000 7f46af9f5640 1 -- 172.21.15.5:0/3102179929 shutdown_connections 2024-05-20T18:29:05.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.679+0000 7f46af9f5640 1 -- 172.21.15.5:0/3102179929 wait complete. 2024-05-20T18:29:05.693 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-20T18:29:05.693 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:29:05.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.816+0000 7f5b1d165640 1 Processor -- start 2024-05-20T18:29:05.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.816+0000 7f5b1d165640 1 -- start start 2024-05-20T18:29:05.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.817+0000 7f5b1d165640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b1806d9a0 0x7f5b180d1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:05.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.817+0000 7f5b1d165640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5b1815d940 0x7f5b1815dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:05.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.817+0000 7f5b1d165640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b1815e6b0 con 0x7f5b180b2f10 2024-05-20T18:29:05.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.817+0000 7f5b1d165640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f5b180d2400 con 0x7f5b1815d940 2024-05-20T18:29:05.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.817+0000 7f5b1d165640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5b180d1a30 con 0x7f5b1806d9a0 2024-05-20T18:29:05.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.817+0000 7f5b1659c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b1806d9a0 0x7f5b180d1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:05.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.817+0000 7f5b1759e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5b1815d940 0x7f5b1815dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:05.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.818+0000 7f5b1659c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b1806d9a0 0x7f5b180d1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:34990/0 (socket says 172.21.15.5:34990) 2024-05-20T18:29:05.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.818+0000 7f5b16d9d640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f5b180b2f10 0x7f5b1806d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:47144/0 (socket says 172.21.15.5:47144) 2024-05-20T18:29:05.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.818+0000 7f5b1659c640 1 -- 172.21.15.5:0/1094988040 learned_addr learned my addr 172.21.15.5:0/1094988040 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:05.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.818+0000 7f5b1759e640 1 -- 172.21.15.5:0/1094988040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b1806d9a0 msgr2=0x7f5b180d1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:05.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.818+0000 7f5b1759e640 1 --2- 172.21.15.5:0/1094988040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b1806d9a0 0x7f5b180d1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:05.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.818+0000 7f5b1759e640 1 -- 172.21.15.5:0/1094988040 >> v1:172.21.15.63:6789/0 conn(0x7f5b180b2f10 legacy=0x7f5b1806d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:05.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.818+0000 7f5b15d9b640 1 -- 172.21.15.5:0/1094988040 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2271298493 0 0) 0x7f5b1815e6b0 con 0x7f5b180b2f10 2024-05-20T18:29:05.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.818+0000 7f5b1759e640 1 -- 172.21.15.5:0/1094988040 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b180d1100 con 0x7f5b1815d940 2024-05-20T18:29:05.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.818+0000 7f5b1659c640 1 --2- 172.21.15.5:0/1094988040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b1806d9a0 0x7f5b180d1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:29:05.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.819+0000 7f5b1759e640 1 --2- 172.21.15.5:0/1094988040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b1815d940 0x7f5b1815dd50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5b100097a0 tx=0x7f5b1002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3c833b5aa583cb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:05.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.819+0000 7f5b15d9b640 1 -- 172.21.15.5:0/1094988040 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b10037070 con 0x7f5b1815d940 2024-05-20T18:29:05.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.819+0000 7f5b15d9b640 1 -- 172.21.15.5:0/1094988040 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b1002ae50 con 0x7f5b1815d940 2024-05-20T18:29:05.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.819+0000 7f5b15d9b640 1 -- 172.21.15.5:0/1094988040 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b10032660 con 0x7f5b1815d940 2024-05-20T18:29:05.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.820+0000 7f5b1d165640 1 -- 172.21.15.5:0/1094988040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b1815d940 msgr2=0x7f5b1815dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:05.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.820+0000 7f5b1d165640 1 --2- 172.21.15.5:0/1094988040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b1815d940 0x7f5b1815dd50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5b100097a0 tx=0x7f5b1002a100 comp rx=0 tx=0).stop 2024-05-20T18:29:05.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.820+0000 7f5b1d165640 1 -- 172.21.15.5:0/1094988040 shutdown_connections 2024-05-20T18:29:05.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.820+0000 7f5b1d165640 1 --2- 172.21.15.5:0/1094988040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b1815d940 0x7f5b1815dd50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:05.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.820+0000 7f5b1d165640 1 --2- 172.21.15.5:0/1094988040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b1806d9a0 0x7f5b180d1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:05.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.820+0000 7f5b1d165640 1 -- 172.21.15.5:0/1094988040 >> 172.21.15.5:0/1094988040 conn(0x7f5b180b6670 msgr2=0x7f5b180b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:05.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.820+0000 7f5b1d165640 1 -- 172.21.15.5:0/1094988040 shutdown_connections 2024-05-20T18:29:05.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.820+0000 7f5b1d165640 1 -- 172.21.15.5:0/1094988040 wait complete. 2024-05-20T18:29:05.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.822+0000 7f5b1d165640 1 Processor -- start 2024-05-20T18:29:05.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.822+0000 7f5b1d165640 1 -- start start 2024-05-20T18:29:05.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.823+0000 7f5b1d165640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b1806d9a0 0x7f5b180c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:05.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.824+0000 7f5b1d165640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b180b2f10 0x7f5b18150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:05.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.824+0000 7f5b1d165640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b1815d940 0x7f5b1815c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:05.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.824+0000 7f5b1d165640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5b180d16c0 con 0x7f5b180b2f10 2024-05-20T18:29:05.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.824+0000 7f5b1d165640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5b180b3320 con 0x7f5b1806d9a0 2024-05-20T18:29:05.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.824+0000 7f5b1d165640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5b1815e290 con 0x7f5b1815d940 2024-05-20T18:29:05.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.824+0000 7f5b1659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b180b2f10 0x7f5b18150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:05.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.824+0000 7f5b1659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b180b2f10 0x7f5b18150ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42772/0 (socket says 172.21.15.5:42772) 2024-05-20T18:29:05.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.824+0000 7f5b1659c640 1 -- 172.21.15.5:0/2690146949 learned_addr learned my addr 172.21.15.5:0/2690146949 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:05.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.824+0000 7f5b16d9d640 1 --2- 172.21.15.5:0/2690146949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b1806d9a0 0x7f5b180c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:05.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.824+0000 7f5b1759e640 1 --2- 172.21.15.5:0/2690146949 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b1815d940 0x7f5b1815c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:05.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.824+0000 7f5b1659c640 1 -- 172.21.15.5:0/2690146949 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b1815d940 msgr2=0x7f5b1815c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:05.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.824+0000 7f5b1659c640 1 --2- 172.21.15.5:0/2690146949 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b1815d940 0x7f5b1815c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:05.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.824+0000 7f5b1659c640 1 -- 172.21.15.5:0/2690146949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b1806d9a0 msgr2=0x7f5b180c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:05.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.824+0000 7f5b1659c640 1 --2- 172.21.15.5:0/2690146949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b1806d9a0 0x7f5b180c70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:05.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.824+0000 7f5b1659c640 1 -- 172.21.15.5:0/2690146949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b180af440 con 0x7f5b180b2f10 2024-05-20T18:29:05.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.824+0000 7f5b1659c640 1 --2- 172.21.15.5:0/2690146949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b180b2f10 0x7f5b18150ef0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00e9b0 tx=0x7f5b0c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:05.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.824+0000 7f5aff7fe640 1 -- 172.21.15.5:0/2690146949 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c00c9a0 con 0x7f5b180b2f10 2024-05-20T18:29:05.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.824+0000 7f5aff7fe640 1 -- 172.21.15.5:0/2690146949 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b0c00cb40 con 0x7f5b180b2f10 2024-05-20T18:29:05.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.825+0000 7f5b1d165640 1 -- 172.21.15.5:0/2690146949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b180b01c0 con 0x7f5b180b2f10 2024-05-20T18:29:05.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.825+0000 7f5aff7fe640 1 -- 172.21.15.5:0/2690146949 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c005990 con 0x7f5b180b2f10 2024-05-20T18:29:05.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.825+0000 7f5b1d165640 1 -- 172.21.15.5:0/2690146949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b180af6e0 con 0x7f5b180b2f10 2024-05-20T18:29:05.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.825+0000 7f5b1d165640 1 -- 172.21.15.5:0/2690146949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b18075b00 con 0x7f5b180b2f10 2024-05-20T18:29:05.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.826+0000 7f5aff7fe640 1 -- 172.21.15.5:0/2690146949 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5b0c005b30 con 0x7f5b180b2f10 2024-05-20T18:29:05.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.827+0000 7f5aff7fe640 1 --2- 172.21.15.5:0/2690146949 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5b0406be30 0x7f5b0406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:05.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.827+0000 7f5aff7fe640 1 -- 172.21.15.5:0/2690146949 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f5b0c08b2a0 con 0x7f5b180b2f10 2024-05-20T18:29:05.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.827+0000 7f5b16d9d640 1 --2- 172.21.15.5:0/2690146949 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5b0406be30 0x7f5b0406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:05.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.827+0000 7f5b16d9d640 1 --2- 172.21.15.5:0/2690146949 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5b0406be30 0x7f5b0406e340 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5b080012e0 tx=0x7f5b08001680 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:05.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:05.831+0000 7f5aff7fe640 1 -- 172.21.15.5:0/2690146949 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5b0c017020 con 0x7f5b180b2f10 2024-05-20T18:29:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.054+0000 7f5b1d165640 1 -- 172.21.15.5:0/2690146949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b1806e490 con 0x7f5b180b2f10 2024-05-20T18:29:06.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.054+0000 7f5aff7fe640 1 -- 172.21.15.5:0/2690146949 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5175 (secure 0 0 0) 0x7f5b0c08b0c0 con 0x7f5b180b2f10 2024-05-20T18:29:06.054 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:29:06.054 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5207,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3958852803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3958852803},{"type":"v1","addr":"172.21.15.63:6835","nonce":3958852803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5560,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/3020169757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":3020169757},{"type":"v1","addr":"172.21.15.63:6834","nonce":3020169757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2965018210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2965018210},{"type":"v1","addr":"172.21.15.5:6835","nonce":2965018210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:29:02.038442+0000","modified":"2024-05-20T18:29:05.055248+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5751},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5751":{"gid":5751,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6837/3469824363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3469824363},{"type":"v1","addr":"172.21.15.5:6837","nonce":3469824363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5751,"qdb_cluster":[5751]},"id":3}]} 2024-05-20T18:29:06.054 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-05-20T18:29:06.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.057+0000 7f5b1d165640 1 -- 172.21.15.5:0/2690146949 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5b0406be30 msgr2=0x7f5b0406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:06.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.057+0000 7f5b1d165640 1 --2- 172.21.15.5:0/2690146949 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5b0406be30 0x7f5b0406e340 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5b080012e0 tx=0x7f5b08001680 comp rx=0 tx=0).stop 2024-05-20T18:29:06.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.057+0000 7f5b1d165640 1 -- 172.21.15.5:0/2690146949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b180b2f10 msgr2=0x7f5b18150ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:06.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.057+0000 7f5b1d165640 1 --2- 172.21.15.5:0/2690146949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b180b2f10 0x7f5b18150ef0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00e9b0 tx=0x7f5b0c00ee70 comp rx=0 tx=0).stop 2024-05-20T18:29:06.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.057+0000 7f5b1d165640 1 -- 172.21.15.5:0/2690146949 shutdown_connections 2024-05-20T18:29:06.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.057+0000 7f5b1d165640 1 --2- 172.21.15.5:0/2690146949 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5b0406be30 0x7f5b0406e340 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:06.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.057+0000 7f5b1d165640 1 --2- 172.21.15.5:0/2690146949 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5b1815d940 0x7f5b1815c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:06.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.057+0000 7f5b1d165640 1 --2- 172.21.15.5:0/2690146949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b180b2f10 0x7f5b18150ef0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:06.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.057+0000 7f5b1d165640 1 --2- 172.21.15.5:0/2690146949 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b1806d9a0 0x7f5b180c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:06.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.057+0000 7f5b1d165640 1 -- 172.21.15.5:0/2690146949 >> 172.21.15.5:0/2690146949 conn(0x7f5b180b6670 msgr2=0x7f5b180b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:06.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.057+0000 7f5b1d165640 1 -- 172.21.15.5:0/2690146949 shutdown_connections 2024-05-20T18:29:06.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.057+0000 7f5b1d165640 1 -- 172.21.15.5:0/2690146949 wait complete. 2024-05-20T18:29:06.068 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:29:06.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.183+0000 7f51c3fce640 1 Processor -- start 2024-05-20T18:29:06.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.183+0000 7f51c3fce640 1 -- start start 2024-05-20T18:29:06.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.184+0000 7f51c3fce640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51bc0cf850 0x7f51bc0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:06.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.184+0000 7f51c3fce640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f51bc0d14f0 0x7f51bc0b6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:06.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.184+0000 7f51c3fce640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51bc0d1ac0 con 0x7f51bc0d10e0 2024-05-20T18:29:06.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.184+0000 7f51c3fce640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f51bc0d1da0 con 0x7f51bc0d14f0 2024-05-20T18:29:06.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.184+0000 7f51c3fce640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f51bc0d2080 con 0x7f51bc0cf850 2024-05-20T18:29:06.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.184+0000 7f51c1d6a640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f51bc0d10e0 0x7f51bc0cf140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36916/0 (socket says 172.21.15.5:36916) 2024-05-20T18:29:06.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.184+0000 7f51c1d6a640 1 -- 172.21.15.5:0/3826405744 learned_addr learned my addr 172.21.15.5:0/3826405744 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:06.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.185+0000 7f51c1569640 1 --2- 172.21.15.5:0/3826405744 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51bc0cf850 0x7f51bc0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:06.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.185+0000 7f51c256b640 1 --2- 172.21.15.5:0/3826405744 >> v2:172.21.15.63:3300/0 conn(0x7f51bc0d14f0 0x7f51bc0b6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:06.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.185+0000 7f51c0d68640 1 -- 172.21.15.5:0/3826405744 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2898633494 0 0) 0x7f51bc0d1ac0 con 0x7f51bc0d10e0 2024-05-20T18:29:06.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.185+0000 7f51c0d68640 1 -- 172.21.15.5:0/3826405744 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51a8003520 con 0x7f51bc0d10e0 2024-05-20T18:29:06.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.185+0000 7f51c1569640 1 -- 172.21.15.5:0/3826405744 >> v2:172.21.15.63:3300/0 conn(0x7f51bc0d14f0 msgr2=0x7f51bc0b6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:06.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.185+0000 7f51c1569640 1 --2- 172.21.15.5:0/3826405744 >> v2:172.21.15.63:3300/0 conn(0x7f51bc0d14f0 0x7f51bc0b6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:06.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.185+0000 7f51c1569640 1 -- 172.21.15.5:0/3826405744 >> v1:172.21.15.5:6789/0 conn(0x7f51bc0d10e0 legacy=0x7f51bc0cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:06.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.185+0000 7f51c0d68640 1 -- 172.21.15.5:0/3826405744 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 658923239 0 0) 0x7f51a8003520 con 0x7f51bc0d10e0 2024-05-20T18:29:06.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.185+0000 7f51c1569640 1 -- 172.21.15.5:0/3826405744 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51bc0d2360 con 0x7f51bc0cf850 2024-05-20T18:29:06.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.186+0000 7f51c1569640 1 --2- 172.21.15.5:0/3826405744 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51bc0cf850 0x7f51bc0b6200 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f51b00097a0 tx=0x7f51b002a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7637db18c144e1bc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:06.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.186+0000 7f51c0d68640 1 -- 172.21.15.5:0/3826405744 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b0037070 con 0x7f51bc0cf850 2024-05-20T18:29:06.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.186+0000 7f51c0d68640 1 -- 172.21.15.5:0/3826405744 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51b002aba0 con 0x7f51bc0cf850 2024-05-20T18:29:06.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.186+0000 7f51c0d68640 1 -- 172.21.15.5:0/3826405744 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b8001670 con 0x7f51bc0cf850 2024-05-20T18:29:06.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.187+0000 7f51c3fce640 1 -- 172.21.15.5:0/3826405744 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51bc0cf850 msgr2=0x7f51bc0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:06.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.187+0000 7f51c3fce640 1 --2- 172.21.15.5:0/3826405744 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51bc0cf850 0x7f51bc0b6200 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f51b00097a0 tx=0x7f51b002a110 comp rx=0 tx=0).stop 2024-05-20T18:29:06.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.187+0000 7f51c3fce640 1 -- 172.21.15.5:0/3826405744 shutdown_connections 2024-05-20T18:29:06.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.187+0000 7f51c3fce640 1 --2- 172.21.15.5:0/3826405744 >> v2:172.21.15.63:3300/0 conn(0x7f51bc0d14f0 0x7f51bc0b6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:06.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.187+0000 7f51c3fce640 1 --2- 172.21.15.5:0/3826405744 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51bc0cf850 0x7f51bc0b6200 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:06.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.187+0000 7f51c3fce640 1 -- 172.21.15.5:0/3826405744 >> 172.21.15.5:0/3826405744 conn(0x7f51bc0b44d0 msgr2=0x7f51bc0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:06.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.187+0000 7f51c3fce640 1 -- 172.21.15.5:0/3826405744 shutdown_connections 2024-05-20T18:29:06.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.187+0000 7f51c3fce640 1 -- 172.21.15.5:0/3826405744 wait complete. 2024-05-20T18:29:06.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.189+0000 7f51c3fce640 1 Processor -- start 2024-05-20T18:29:06.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.189+0000 7f51c3fce640 1 -- start start 2024-05-20T18:29:06.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.190+0000 7f51c3fce640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51bc0cf850 0x7f51bc0cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:06.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.190+0000 7f51c3fce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51bc0d10e0 0x7f51bc0b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:06.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.190+0000 7f51c3fce640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51bc0d14f0 0x7f51bc152700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:06.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.190+0000 7f51c3fce640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc0b8440 con 0x7f51bc0d10e0 2024-05-20T18:29:06.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.190+0000 7f51c3fce640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc0b82c0 con 0x7f51bc0cf850 2024-05-20T18:29:06.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.190+0000 7f51c3fce640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f51bc0709f0 con 0x7f51bc0d14f0 2024-05-20T18:29:06.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.190+0000 7f51c1569640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51bc0d10e0 0x7f51bc0b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:06.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.190+0000 7f51c1569640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51bc0d10e0 0x7f51bc0b1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42774/0 (socket says 172.21.15.5:42774) 2024-05-20T18:29:06.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.190+0000 7f51c1569640 1 -- 172.21.15.5:0/2145772649 learned_addr learned my addr 172.21.15.5:0/2145772649 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:06.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.190+0000 7f51c256b640 1 --2- 172.21.15.5:0/2145772649 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51bc0d14f0 0x7f51bc152700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:06.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.190+0000 7f51c1d6a640 1 --2- 172.21.15.5:0/2145772649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51bc0cf850 0x7f51bc0cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:06.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.191+0000 7f51c1569640 1 -- 172.21.15.5:0/2145772649 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51bc0d14f0 msgr2=0x7f51bc152700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:06.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.191+0000 7f51c1569640 1 --2- 172.21.15.5:0/2145772649 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51bc0d14f0 0x7f51bc152700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:06.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.191+0000 7f51c1569640 1 -- 172.21.15.5:0/2145772649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51bc0cf850 msgr2=0x7f51bc0cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:06.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.191+0000 7f51c1569640 1 --2- 172.21.15.5:0/2145772649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51bc0cf850 0x7f51bc0cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:06.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.191+0000 7f51c1569640 1 -- 172.21.15.5:0/2145772649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51bc06dfa0 con 0x7f51bc0d10e0 2024-05-20T18:29:06.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.191+0000 7f51c1569640 1 --2- 172.21.15.5:0/2145772649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51bc0d10e0 0x7f51bc0b1010 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f51b0035040 tx=0x7f51b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:06.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.191+0000 7f51ae7fc640 1 -- 172.21.15.5:0/2145772649 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b0041070 con 0x7f51bc0d10e0 2024-05-20T18:29:06.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.191+0000 7f51ae7fc640 1 -- 172.21.15.5:0/2145772649 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51b003c660 con 0x7f51bc0d10e0 2024-05-20T18:29:06.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.191+0000 7f51c3fce640 1 -- 172.21.15.5:0/2145772649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51bc0cc8f0 con 0x7f51bc0d10e0 2024-05-20T18:29:06.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.191+0000 7f51ae7fc640 1 -- 172.21.15.5:0/2145772649 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b0037040 con 0x7f51bc0d10e0 2024-05-20T18:29:06.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.191+0000 7f51c3fce640 1 -- 172.21.15.5:0/2145772649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51bc06d700 con 0x7f51bc0d10e0 2024-05-20T18:29:06.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.192+0000 7f51c3fce640 1 -- 172.21.15.5:0/2145772649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51bc06d190 con 0x7f51bc0d10e0 2024-05-20T18:29:06.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.193+0000 7f51ae7fc640 1 -- 172.21.15.5:0/2145772649 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f51b0044070 con 0x7f51bc0d10e0 2024-05-20T18:29:06.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.193+0000 7f51ae7fc640 1 --2- 172.21.15.5:0/2145772649 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f51a006be30 0x7f51a006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:06.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.193+0000 7f51ae7fc640 1 -- 172.21.15.5:0/2145772649 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f51b00ab970 con 0x7f51bc0d10e0 2024-05-20T18:29:06.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.193+0000 7f51c1d6a640 1 --2- 172.21.15.5:0/2145772649 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f51a006be30 0x7f51a006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:06.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.194+0000 7f51c1d6a640 1 --2- 172.21.15.5:0/2145772649 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f51a006be30 0x7f51a006e340 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f51b8005880 tx=0x7f51b8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:06.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.198+0000 7f51ae7fc640 1 -- 172.21.15.5:0/2145772649 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f51b007a060 con 0x7f51bc0d10e0 2024-05-20T18:29:06.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.384+0000 7f51c3fce640 1 -- 172.21.15.5:0/2145772649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51bc075b00 con 0x7f51bc0d10e0 2024-05-20T18:29:06.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.385+0000 7f51ae7fc640 1 -- 172.21.15.5:0/2145772649 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+16822 (secure 0 0 0) 0x7f51b0079960 con 0x7f51bc0d10e0 2024-05-20T18:29:06.385 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:29:06.385 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":51,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:29:03.665286+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:29:00.401866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-20T18:29:01.006122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:29:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.387+0000 7f51c3fce640 1 -- 172.21.15.5:0/2145772649 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f51a006be30 msgr2=0x7f51a006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.387+0000 7f51c3fce640 1 --2- 172.21.15.5:0/2145772649 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f51a006be30 0x7f51a006e340 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f51b8005880 tx=0x7f51b8009040 comp rx=0 tx=0).stop 2024-05-20T18:29:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.388+0000 7f51c3fce640 1 -- 172.21.15.5:0/2145772649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51bc0d10e0 msgr2=0x7f51bc0b1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.388+0000 7f51c3fce640 1 --2- 172.21.15.5:0/2145772649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51bc0d10e0 0x7f51bc0b1010 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f51b0035040 tx=0x7f51b00046c0 comp rx=0 tx=0).stop 2024-05-20T18:29:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.388+0000 7f51c3fce640 1 -- 172.21.15.5:0/2145772649 shutdown_connections 2024-05-20T18:29:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.388+0000 7f51c3fce640 1 --2- 172.21.15.5:0/2145772649 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f51a006be30 0x7f51a006e340 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.388+0000 7f51c3fce640 1 --2- 172.21.15.5:0/2145772649 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51bc0d14f0 0x7f51bc152700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.388+0000 7f51c3fce640 1 --2- 172.21.15.5:0/2145772649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51bc0d10e0 0x7f51bc0b1010 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.388+0000 7f51c3fce640 1 --2- 172.21.15.5:0/2145772649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51bc0cf850 0x7f51bc0cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.388+0000 7f51c3fce640 1 -- 172.21.15.5:0/2145772649 >> 172.21.15.5:0/2145772649 conn(0x7f51bc0b44d0 msgr2=0x7f51bc0cf140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.388+0000 7f51c3fce640 1 -- 172.21.15.5:0/2145772649 shutdown_connections 2024-05-20T18:29:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.388+0000 7f51c3fce640 1 -- 172.21.15.5:0/2145772649 wait complete. 2024-05-20T18:29:06.400 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-20T18:29:06.400 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T18:29:06.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.515+0000 7f35fd81b640 1 Processor -- start 2024-05-20T18:29:06.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.515+0000 7f35fd81b640 1 -- start start 2024-05-20T18:29:06.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.516+0000 7f35fd81b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f35f815f2e0 0x7f35f80af200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:06.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.516+0000 7f35fd81b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f35f80d1e50 0x7f35f80b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.516+0000 7f35fd81b640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35f806c2f0 con 0x7f35f80af740 2024-05-20T18:29:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.516+0000 7f35fd81b640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f35f806cf00 con 0x7f35f815f2e0 2024-05-20T18:29:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.516+0000 7f35fd81b640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f35f806c5d0 con 0x7f35f80d1e50 2024-05-20T18:29:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.516+0000 7f35f6ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f35f815f2e0 0x7f35f80af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.516+0000 7f35f6ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f35f815f2e0 0x7f35f80af200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42788/0 (socket says 172.21.15.5:42788) 2024-05-20T18:29:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.516+0000 7f35f6ffd640 1 -- 172.21.15.5:0/1576649610 learned_addr learned my addr 172.21.15.5:0/1576649610 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.516+0000 7f35f77fe640 1 --2- 172.21.15.5:0/1576649610 >> v2:172.21.15.63:3300/0 conn(0x7f35f80d1e50 0x7f35f80b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.516+0000 7f35f6ffd640 1 -- 172.21.15.5:0/1576649610 >> v2:172.21.15.63:3300/0 conn(0x7f35f80d1e50 msgr2=0x7f35f80b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.516+0000 7f35f6ffd640 1 --2- 172.21.15.5:0/1576649610 >> v2:172.21.15.63:3300/0 conn(0x7f35f80d1e50 0x7f35f80b0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.517+0000 7f35f6ffd640 1 -- 172.21.15.5:0/1576649610 >> v1:172.21.15.63:6789/0 conn(0x7f35f80af740 legacy=0x7f35f80d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.517+0000 7f35f6ffd640 1 -- 172.21.15.5:0/1576649610 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f806c8b0 con 0x7f35f815f2e0 2024-05-20T18:29:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.517+0000 7f35f6ffd640 1 --2- 172.21.15.5:0/1576649610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35f815f2e0 0x7f35f80af200 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f35f806d080 tx=0x7f35e802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b237fea42d29fc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.517+0000 7f35f5ffb640 1 -- 172.21.15.5:0/1576649610 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e8036070 con 0x7f35f815f2e0 2024-05-20T18:29:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.518+0000 7f35f5ffb640 1 -- 172.21.15.5:0/1576649610 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35e8031600 con 0x7f35f815f2e0 2024-05-20T18:29:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.518+0000 7f35f5ffb640 1 -- 172.21.15.5:0/1576649610 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e80319d0 con 0x7f35f815f2e0 2024-05-20T18:29:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.518+0000 7f35fd81b640 1 -- 172.21.15.5:0/1576649610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35f815f2e0 msgr2=0x7f35f80af200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.518+0000 7f35fd81b640 1 --2- 172.21.15.5:0/1576649610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35f815f2e0 0x7f35f80af200 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f35f806d080 tx=0x7f35e802a730 comp rx=0 tx=0).stop 2024-05-20T18:29:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.518+0000 7f35fd81b640 1 -- 172.21.15.5:0/1576649610 shutdown_connections 2024-05-20T18:29:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.518+0000 7f35fd81b640 1 --2- 172.21.15.5:0/1576649610 >> v2:172.21.15.63:3300/0 conn(0x7f35f80d1e50 0x7f35f80b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.518+0000 7f35fd81b640 1 --2- 172.21.15.5:0/1576649610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35f815f2e0 0x7f35f80af200 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.518+0000 7f35fd81b640 1 -- 172.21.15.5:0/1576649610 >> 172.21.15.5:0/1576649610 conn(0x7f35f8073770 msgr2=0x7f35f80b6720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.518+0000 7f35fd81b640 1 -- 172.21.15.5:0/1576649610 shutdown_connections 2024-05-20T18:29:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.518+0000 7f35fd81b640 1 -- 172.21.15.5:0/1576649610 wait complete. 2024-05-20T18:29:06.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.520+0000 7f35fd81b640 1 Processor -- start 2024-05-20T18:29:06.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.520+0000 7f35fd81b640 1 -- start start 2024-05-20T18:29:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.521+0000 7f35fd81b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35f80af740 0x7f35f80b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.522+0000 7f35fd81b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35f81522d0 0x7f35f81526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.522+0000 7f35fd81b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35f8152c20 0x7f35f81f5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.522+0000 7f35fd81b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f35f806d390 con 0x7f35f81522d0 2024-05-20T18:29:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.522+0000 7f35fd81b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f35f806bee0 con 0x7f35f8152c20 2024-05-20T18:29:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.522+0000 7f35fd81b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f35f8153610 con 0x7f35f80af740 2024-05-20T18:29:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.522+0000 7f35f67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35f81522d0 0x7f35f81526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.522+0000 7f35f67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35f81522d0 0x7f35f81526e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42792/0 (socket says 172.21.15.5:42792) 2024-05-20T18:29:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.522+0000 7f35f67fc640 1 -- 172.21.15.5:0/517683014 learned_addr learned my addr 172.21.15.5:0/517683014 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.522+0000 7f35f77fe640 1 --2- 172.21.15.5:0/517683014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35f8152c20 0x7f35f81f5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.522+0000 7f35f6ffd640 1 --2- 172.21.15.5:0/517683014 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35f80af740 0x7f35f80b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.522+0000 7f35f67fc640 1 -- 172.21.15.5:0/517683014 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35f80af740 msgr2=0x7f35f80b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.522+0000 7f35f67fc640 1 --2- 172.21.15.5:0/517683014 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35f80af740 0x7f35f80b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:06.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.522+0000 7f35f67fc640 1 -- 172.21.15.5:0/517683014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35f8152c20 msgr2=0x7f35f81f5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:06.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.522+0000 7f35f67fc640 1 --2- 172.21.15.5:0/517683014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35f8152c20 0x7f35f81f5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:06.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.522+0000 7f35f67fc640 1 -- 172.21.15.5:0/517683014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f80b1410 con 0x7f35f81522d0 2024-05-20T18:29:06.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.522+0000 7f35f67fc640 1 --2- 172.21.15.5:0/517683014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35f81522d0 0x7f35f81526e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00c3f0 tx=0x7f35ec00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:06.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.522+0000 7f35dffff640 1 -- 172.21.15.5:0/517683014 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec0052d0 con 0x7f35f81522d0 2024-05-20T18:29:06.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.522+0000 7f35dffff640 1 -- 172.21.15.5:0/517683014 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35ec005490 con 0x7f35f81522d0 2024-05-20T18:29:06.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.522+0000 7f35fd81b640 1 -- 172.21.15.5:0/517683014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f80b16f0 con 0x7f35f81522d0 2024-05-20T18:29:06.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.522+0000 7f35dffff640 1 -- 172.21.15.5:0/517683014 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec007260 con 0x7f35f81522d0 2024-05-20T18:29:06.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.522+0000 7f35fd81b640 1 -- 172.21.15.5:0/517683014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f80b1cb0 con 0x7f35f81522d0 2024-05-20T18:29:06.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.523+0000 7f35fd81b640 1 -- 172.21.15.5:0/517683014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35f8075b00 con 0x7f35f81522d0 2024-05-20T18:29:06.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.524+0000 7f35dffff640 1 -- 172.21.15.5:0/517683014 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f35ec007400 con 0x7f35f81522d0 2024-05-20T18:29:06.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.524+0000 7f35dffff640 1 --2- 172.21.15.5:0/517683014 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f35e406be30 0x7f35e406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:06.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.525+0000 7f35dffff640 1 -- 172.21.15.5:0/517683014 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f35ec08a410 con 0x7f35f81522d0 2024-05-20T18:29:06.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.525+0000 7f35f6ffd640 1 --2- 172.21.15.5:0/517683014 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f35e406be30 0x7f35e406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:06.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.525+0000 7f35f6ffd640 1 --2- 172.21.15.5:0/517683014 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f35e406be30 0x7f35e406e340 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f35e8033000 tx=0x7f35e8009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:06.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.529+0000 7f35dffff640 1 -- 172.21.15.5:0/517683014 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f35ec0585e0 con 0x7f35f81522d0 2024-05-20T18:29:06.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.712+0000 7f35fd81b640 1 -- 172.21.15.5:0/517683014 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f35f80b1160 con 0x7f35e406be30 2024-05-20T18:29:06.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.718+0000 7f35dffff640 1 -- 172.21.15.5:0/517683014 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251955 (secure 0 0 0) 0x7f35f80b1160 con 0x7f35e406be30 2024-05-20T18:29:06.720 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:29:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-20T18:29:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.721+0000 7f35fd81b640 1 -- 172.21.15.5:0/517683014 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f35e406be30 msgr2=0x7f35e406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.721+0000 7f35fd81b640 1 --2- 172.21.15.5:0/517683014 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f35e406be30 0x7f35e406e340 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f35e8033000 tx=0x7f35e8009000 comp rx=0 tx=0).stop 2024-05-20T18:29:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.721+0000 7f35fd81b640 1 -- 172.21.15.5:0/517683014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35f81522d0 msgr2=0x7f35f81526e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.721+0000 7f35fd81b640 1 --2- 172.21.15.5:0/517683014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35f81522d0 0x7f35f81526e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00c3f0 tx=0x7f35ec00c8b0 comp rx=0 tx=0).stop 2024-05-20T18:29:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.721+0000 7f35fd81b640 1 -- 172.21.15.5:0/517683014 shutdown_connections 2024-05-20T18:29:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.721+0000 7f35fd81b640 1 --2- 172.21.15.5:0/517683014 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f35e406be30 0x7f35e406e340 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.721+0000 7f35fd81b640 1 --2- 172.21.15.5:0/517683014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35f8152c20 0x7f35f81f5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.721+0000 7f35fd81b640 1 --2- 172.21.15.5:0/517683014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35f81522d0 0x7f35f81526e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.721+0000 7f35fd81b640 1 --2- 172.21.15.5:0/517683014 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35f80af740 0x7f35f80b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.721+0000 7f35fd81b640 1 -- 172.21.15.5:0/517683014 >> 172.21.15.5:0/517683014 conn(0x7f35f8073770 msgr2=0x7f35f806f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.721+0000 7f35fd81b640 1 -- 172.21.15.5:0/517683014 shutdown_connections 2024-05-20T18:29:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.721+0000 7f35fd81b640 1 -- 172.21.15.5:0/517683014 wait complete. 2024-05-20T18:29:06.733 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-05-20T18:29:06.602906+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238780,"kb_used_data":5960,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735940,"statfs":{"total":773094113280,"available":772849602560,"internally_reserved":0,"allocated":6103040,"data_stored":3340596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.967123"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082088+0000","last_change":"2024-05-20T18:29:02.672760+0000","last_active":"2024-05-20T18:29:04.082088+0000","last_peered":"2024-05-20T18:29:04.082088+0000","last_clean":"2024-05-20T18:29:04.082088+0000","last_became_active":"2024-05-20T18:29:02.672419+0000","last_became_peered":"2024-05-20T18:29:02.672419+0000","last_unstale":"2024-05-20T18:29:04.082088+0000","last_undegraded":"2024-05-20T18:29:04.082088+0000","last_fullsized":"2024-05-20T18:29:04.082088+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:40:21.267154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082176+0000","last_change":"2024-05-20T18:29:02.685300+0000","last_active":"2024-05-20T18:29:04.082176+0000","last_peered":"2024-05-20T18:29:04.082176+0000","last_clean":"2024-05-20T18:29:04.082176+0000","last_became_active":"2024-05-20T18:29:02.684942+0000","last_became_peered":"2024-05-20T18:29:02.684942+0000","last_unstale":"2024-05-20T18:29:04.082176+0000","last_undegraded":"2024-05-20T18:29:04.082176+0000","last_fullsized":"2024-05-20T18:29:04.082176+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:52:46.953429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082466+0000","last_change":"2024-05-20T18:29:02.687050+0000","last_active":"2024-05-20T18:29:04.082466+0000","last_peered":"2024-05-20T18:29:04.082466+0000","last_clean":"2024-05-20T18:29:04.082466+0000","last_became_active":"2024-05-20T18:29:02.679862+0000","last_became_peered":"2024-05-20T18:29:02.679862+0000","last_unstale":"2024-05-20T18:29:04.082466+0000","last_undegraded":"2024-05-20T18:29:04.082466+0000","last_fullsized":"2024-05-20T18:29:04.082466+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:25:10.907111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082298+0000","last_change":"2024-05-20T18:29:02.683179+0000","last_active":"2024-05-20T18:29:04.082298+0000","last_peered":"2024-05-20T18:29:04.082298+0000","last_clean":"2024-05-20T18:29:04.082298+0000","last_became_active":"2024-05-20T18:29:02.682757+0000","last_became_peered":"2024-05-20T18:29:02.682757+0000","last_unstale":"2024-05-20T18:29:04.082298+0000","last_undegraded":"2024-05-20T18:29:04.082298+0000","last_fullsized":"2024-05-20T18:29:04.082298+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:19:37.476208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.674194+0000","last_change":"2024-05-20T18:29:02.674694+0000","last_active":"2024-05-20T18:29:03.674194+0000","last_peered":"2024-05-20T18:29:03.674194+0000","last_clean":"2024-05-20T18:29:03.674194+0000","last_became_active":"2024-05-20T18:29:02.674234+0000","last_became_peered":"2024-05-20T18:29:02.674234+0000","last_unstale":"2024-05-20T18:29:03.674194+0000","last_undegraded":"2024-05-20T18:29:03.674194+0000","last_fullsized":"2024-05-20T18:29:03.674194+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:13:38.743236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081570+0000","last_change":"2024-05-20T18:29:02.683447+0000","last_active":"2024-05-20T18:29:04.081570+0000","last_peered":"2024-05-20T18:29:04.081570+0000","last_clean":"2024-05-20T18:29:04.081570+0000","last_became_active":"2024-05-20T18:29:02.680606+0000","last_became_peered":"2024-05-20T18:29:02.680606+0000","last_unstale":"2024-05-20T18:29:04.081570+0000","last_undegraded":"2024-05-20T18:29:04.081570+0000","last_fullsized":"2024-05-20T18:29:04.081570+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:44:18.695885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081646+0000","last_change":"2024-05-20T18:29:02.673604+0000","last_active":"2024-05-20T18:29:04.081646+0000","last_peered":"2024-05-20T18:29:04.081646+0000","last_clean":"2024-05-20T18:29:04.081646+0000","last_became_active":"2024-05-20T18:29:02.673266+0000","last_became_peered":"2024-05-20T18:29:02.673266+0000","last_unstale":"2024-05-20T18:29:04.081646+0000","last_undegraded":"2024-05-20T18:29:04.081646+0000","last_fullsized":"2024-05-20T18:29:04.081646+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:24:27.542753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082368+0000","last_change":"2024-05-20T18:29:02.689510+0000","last_active":"2024-05-20T18:29:04.082368+0000","last_peered":"2024-05-20T18:29:04.082368+0000","last_clean":"2024-05-20T18:29:04.082368+0000","last_became_active":"2024-05-20T18:29:02.688696+0000","last_became_peered":"2024-05-20T18:29:02.688696+0000","last_unstale":"2024-05-20T18:29:04.082368+0000","last_undegraded":"2024-05-20T18:29:04.082368+0000","last_fullsized":"2024-05-20T18:29:04.082368+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:50:27.784534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082182+0000","last_change":"2024-05-20T18:29:02.688179+0000","last_active":"2024-05-20T18:29:04.082182+0000","last_peered":"2024-05-20T18:29:04.082182+0000","last_clean":"2024-05-20T18:29:04.082182+0000","last_became_active":"2024-05-20T18:29:02.687830+0000","last_became_peered":"2024-05-20T18:29:02.687830+0000","last_unstale":"2024-05-20T18:29:04.082182+0000","last_undegraded":"2024-05-20T18:29:04.082182+0000","last_fullsized":"2024-05-20T18:29:04.082182+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:15:29.724415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081435+0000","last_change":"2024-05-20T18:29:02.679932+0000","last_active":"2024-05-20T18:29:04.081435+0000","last_peered":"2024-05-20T18:29:04.081435+0000","last_clean":"2024-05-20T18:29:04.081435+0000","last_became_active":"2024-05-20T18:29:02.679274+0000","last_became_peered":"2024-05-20T18:29:02.679274+0000","last_unstale":"2024-05-20T18:29:04.081435+0000","last_undegraded":"2024-05-20T18:29:04.081435+0000","last_fullsized":"2024-05-20T18:29:04.081435+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:02:28.751921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.672473+0000","last_change":"2024-05-20T18:29:02.678528+0000","last_active":"2024-05-20T18:29:03.672473+0000","last_peered":"2024-05-20T18:29:03.672473+0000","last_clean":"2024-05-20T18:29:03.672473+0000","last_became_active":"2024-05-20T18:29:02.678215+0000","last_became_peered":"2024-05-20T18:29:02.678215+0000","last_unstale":"2024-05-20T18:29:03.672473+0000","last_undegraded":"2024-05-20T18:29:03.672473+0000","last_fullsized":"2024-05-20T18:29:03.672473+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:08:06.265771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082844+0000","last_change":"2024-05-20T18:29:02.680225+0000","last_active":"2024-05-20T18:29:04.082844+0000","last_peered":"2024-05-20T18:29:04.082844+0000","last_clean":"2024-05-20T18:29:04.082844+0000","last_became_active":"2024-05-20T18:29:02.679732+0000","last_became_peered":"2024-05-20T18:29:02.679732+0000","last_unstale":"2024-05-20T18:29:04.082844+0000","last_undegraded":"2024-05-20T18:29:04.082844+0000","last_fullsized":"2024-05-20T18:29:04.082844+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:46:41.204882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082301+0000","last_change":"2024-05-20T18:29:02.683189+0000","last_active":"2024-05-20T18:29:04.082301+0000","last_peered":"2024-05-20T18:29:04.082301+0000","last_clean":"2024-05-20T18:29:04.082301+0000","last_became_active":"2024-05-20T18:29:02.680163+0000","last_became_peered":"2024-05-20T18:29:02.680163+0000","last_unstale":"2024-05-20T18:29:04.082301+0000","last_undegraded":"2024-05-20T18:29:04.082301+0000","last_fullsized":"2024-05-20T18:29:04.082301+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:35:20.045268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082643+0000","last_change":"2024-05-20T18:29:02.682326+0000","last_active":"2024-05-20T18:29:04.082643+0000","last_peered":"2024-05-20T18:29:04.082643+0000","last_clean":"2024-05-20T18:29:04.082643+0000","last_became_active":"2024-05-20T18:29:02.676903+0000","last_became_peered":"2024-05-20T18:29:02.676903+0000","last_unstale":"2024-05-20T18:29:04.082643+0000","last_undegraded":"2024-05-20T18:29:04.082643+0000","last_fullsized":"2024-05-20T18:29:04.082643+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:40:09.123203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081858+0000","last_change":"2024-05-20T18:29:02.678210+0000","last_active":"2024-05-20T18:29:04.081858+0000","last_peered":"2024-05-20T18:29:04.081858+0000","last_clean":"2024-05-20T18:29:04.081858+0000","last_became_active":"2024-05-20T18:29:02.677854+0000","last_became_peered":"2024-05-20T18:29:02.677854+0000","last_unstale":"2024-05-20T18:29:04.081858+0000","last_undegraded":"2024-05-20T18:29:04.081858+0000","last_fullsized":"2024-05-20T18:29:04.081858+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:01:01.162756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081231+0000","last_change":"2024-05-20T18:29:02.675563+0000","last_active":"2024-05-20T18:29:04.081231+0000","last_peered":"2024-05-20T18:29:04.081231+0000","last_clean":"2024-05-20T18:29:04.081231+0000","last_became_active":"2024-05-20T18:29:02.675219+0000","last_became_peered":"2024-05-20T18:29:02.675219+0000","last_unstale":"2024-05-20T18:29:04.081231+0000","last_undegraded":"2024-05-20T18:29:04.081231+0000","last_fullsized":"2024-05-20T18:29:04.081231+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:30:15.509281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082959+0000","last_change":"2024-05-20T18:29:02.674778+0000","last_active":"2024-05-20T18:29:04.082959+0000","last_peered":"2024-05-20T18:29:04.082959+0000","last_clean":"2024-05-20T18:29:04.082959+0000","last_became_active":"2024-05-20T18:29:02.674475+0000","last_became_peered":"2024-05-20T18:29:02.674475+0000","last_unstale":"2024-05-20T18:29:04.082959+0000","last_undegraded":"2024-05-20T18:29:04.082959+0000","last_fullsized":"2024-05-20T18:29:04.082959+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:29:22.046224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082524+0000","last_change":"2024-05-20T18:29:02.685307+0000","last_active":"2024-05-20T18:29:04.082524+0000","last_peered":"2024-05-20T18:29:04.082524+0000","last_clean":"2024-05-20T18:29:04.082524+0000","last_became_active":"2024-05-20T18:29:02.684972+0000","last_became_peered":"2024-05-20T18:29:02.684972+0000","last_unstale":"2024-05-20T18:29:04.082524+0000","last_undegraded":"2024-05-20T18:29:04.082524+0000","last_fullsized":"2024-05-20T18:29:04.082524+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:34:33.698879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082780+0000","last_change":"2024-05-20T18:29:02.688794+0000","last_active":"2024-05-20T18:29:04.082780+0000","last_peered":"2024-05-20T18:29:04.082780+0000","last_clean":"2024-05-20T18:29:04.082780+0000","last_became_active":"2024-05-20T18:29:02.685485+0000","last_became_peered":"2024-05-20T18:29:02.685485+0000","last_unstale":"2024-05-20T18:29:04.082780+0000","last_undegraded":"2024-05-20T18:29:04.082780+0000","last_fullsized":"2024-05-20T18:29:04.082780+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:44:10.748787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.673263+0000","last_change":"2024-05-20T18:29:02.668316+0000","last_active":"2024-05-20T18:29:03.673263+0000","last_peered":"2024-05-20T18:29:03.673263+0000","last_clean":"2024-05-20T18:29:03.673263+0000","last_became_active":"2024-05-20T18:29:02.667979+0000","last_became_peered":"2024-05-20T18:29:02.667979+0000","last_unstale":"2024-05-20T18:29:03.673263+0000","last_undegraded":"2024-05-20T18:29:03.673263+0000","last_fullsized":"2024-05-20T18:29:03.673263+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:46:51.918262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.674260+0000","last_change":"2024-05-20T18:29:02.674706+0000","last_active":"2024-05-20T18:29:03.674260+0000","last_peered":"2024-05-20T18:29:03.674260+0000","last_clean":"2024-05-20T18:29:03.674260+0000","last_became_active":"2024-05-20T18:29:02.674348+0000","last_became_peered":"2024-05-20T18:29:02.674348+0000","last_unstale":"2024-05-20T18:29:03.674260+0000","last_undegraded":"2024-05-20T18:29:03.674260+0000","last_fullsized":"2024-05-20T18:29:03.674260+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:00:42.860723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081497+0000","last_change":"2024-05-20T18:29:02.688526+0000","last_active":"2024-05-20T18:29:04.081497+0000","last_peered":"2024-05-20T18:29:04.081497+0000","last_clean":"2024-05-20T18:29:04.081497+0000","last_became_active":"2024-05-20T18:29:02.688225+0000","last_became_peered":"2024-05-20T18:29:02.688225+0000","last_unstale":"2024-05-20T18:29:04.081497+0000","last_undegraded":"2024-05-20T18:29:04.081497+0000","last_fullsized":"2024-05-20T18:29:04.081497+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:42:59.031072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.672431+0000","last_change":"2024-05-20T18:29:02.673751+0000","last_active":"2024-05-20T18:29:03.672431+0000","last_peered":"2024-05-20T18:29:03.672431+0000","last_clean":"2024-05-20T18:29:03.672431+0000","last_became_active":"2024-05-20T18:29:02.673429+0000","last_became_peered":"2024-05-20T18:29:02.673429+0000","last_unstale":"2024-05-20T18:29:03.672431+0000","last_undegraded":"2024-05-20T18:29:03.672431+0000","last_fullsized":"2024-05-20T18:29:03.672431+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:41:27.321674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081157+0000","last_change":"2024-05-20T18:29:02.687375+0000","last_active":"2024-05-20T18:29:04.081157+0000","last_peered":"2024-05-20T18:29:04.081157+0000","last_clean":"2024-05-20T18:29:04.081157+0000","last_became_active":"2024-05-20T18:29:02.676374+0000","last_became_peered":"2024-05-20T18:29:02.676374+0000","last_unstale":"2024-05-20T18:29:04.081157+0000","last_undegraded":"2024-05-20T18:29:04.081157+0000","last_fullsized":"2024-05-20T18:29:04.081157+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:34:33.698879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.083114+0000","last_change":"2024-05-20T18:29:02.690248+0000","last_active":"2024-05-20T18:29:04.083114+0000","last_peered":"2024-05-20T18:29:04.083114+0000","last_clean":"2024-05-20T18:29:04.083114+0000","last_became_active":"2024-05-20T18:29:02.689929+0000","last_became_peered":"2024-05-20T18:29:02.689929+0000","last_unstale":"2024-05-20T18:29:04.083114+0000","last_undegraded":"2024-05-20T18:29:04.083114+0000","last_fullsized":"2024-05-20T18:29:04.083114+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:55:31.226636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082018+0000","last_change":"2024-05-20T18:29:02.675201+0000","last_active":"2024-05-20T18:29:04.082018+0000","last_peered":"2024-05-20T18:29:04.082018+0000","last_clean":"2024-05-20T18:29:04.082018+0000","last_became_active":"2024-05-20T18:29:02.674605+0000","last_became_peered":"2024-05-20T18:29:02.674605+0000","last_unstale":"2024-05-20T18:29:04.082018+0000","last_undegraded":"2024-05-20T18:29:04.082018+0000","last_fullsized":"2024-05-20T18:29:04.082018+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:04:06.178030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081767+0000","last_change":"2024-05-20T18:29:02.693869+0000","last_active":"2024-05-20T18:29:04.081767+0000","last_peered":"2024-05-20T18:29:04.081767+0000","last_clean":"2024-05-20T18:29:04.081767+0000","last_became_active":"2024-05-20T18:29:02.693562+0000","last_became_peered":"2024-05-20T18:29:02.693562+0000","last_unstale":"2024-05-20T18:29:04.081767+0000","last_undegraded":"2024-05-20T18:29:04.081767+0000","last_fullsized":"2024-05-20T18:29:04.081767+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:50:08.782337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.673315+0000","last_change":"2024-05-20T18:29:02.677234+0000","last_active":"2024-05-20T18:29:03.673315+0000","last_peered":"2024-05-20T18:29:03.673315+0000","last_clean":"2024-05-20T18:29:03.673315+0000","last_became_active":"2024-05-20T18:29:02.673864+0000","last_became_peered":"2024-05-20T18:29:02.673864+0000","last_unstale":"2024-05-20T18:29:03.673315+0000","last_undegraded":"2024-05-20T18:29:03.673315+0000","last_fullsized":"2024-05-20T18:29:03.673315+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:38:12.409535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081429+0000","last_change":"2024-05-20T18:29:02.688564+0000","last_active":"2024-05-20T18:29:04.081429+0000","last_peered":"2024-05-20T18:29:04.081429+0000","last_clean":"2024-05-20T18:29:04.081429+0000","last_became_active":"2024-05-20T18:29:02.688224+0000","last_became_peered":"2024-05-20T18:29:02.688224+0000","last_unstale":"2024-05-20T18:29:04.081429+0000","last_undegraded":"2024-05-20T18:29:04.081429+0000","last_fullsized":"2024-05-20T18:29:04.081429+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:48:55.261971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082431+0000","last_change":"2024-05-20T18:29:02.682079+0000","last_active":"2024-05-20T18:29:04.082431+0000","last_peered":"2024-05-20T18:29:04.082431+0000","last_clean":"2024-05-20T18:29:04.082431+0000","last_became_active":"2024-05-20T18:29:02.681545+0000","last_became_peered":"2024-05-20T18:29:02.681545+0000","last_unstale":"2024-05-20T18:29:04.082431+0000","last_undegraded":"2024-05-20T18:29:04.082431+0000","last_fullsized":"2024-05-20T18:29:04.082431+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:26:40.141288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081815+0000","last_change":"2024-05-20T18:29:02.672518+0000","last_active":"2024-05-20T18:29:04.081815+0000","last_peered":"2024-05-20T18:29:04.081815+0000","last_clean":"2024-05-20T18:29:04.081815+0000","last_became_active":"2024-05-20T18:29:02.672224+0000","last_became_peered":"2024-05-20T18:29:02.672224+0000","last_unstale":"2024-05-20T18:29:04.081815+0000","last_undegraded":"2024-05-20T18:29:04.081815+0000","last_fullsized":"2024-05-20T18:29:04.081815+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:07:06.210696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081588+0000","last_change":"2024-05-20T18:29:02.682072+0000","last_active":"2024-05-20T18:29:04.081588+0000","last_peered":"2024-05-20T18:29:04.081588+0000","last_clean":"2024-05-20T18:29:04.081588+0000","last_became_active":"2024-05-20T18:29:02.681734+0000","last_became_peered":"2024-05-20T18:29:02.681734+0000","last_unstale":"2024-05-20T18:29:04.081588+0000","last_undegraded":"2024-05-20T18:29:04.081588+0000","last_fullsized":"2024-05-20T18:29:04.081588+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:52:46.953429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081575+0000","last_change":"2024-05-20T18:29:02.677541+0000","last_active":"2024-05-20T18:29:04.081575+0000","last_peered":"2024-05-20T18:29:04.081575+0000","last_clean":"2024-05-20T18:29:04.081575+0000","last_became_active":"2024-05-20T18:29:02.677182+0000","last_became_peered":"2024-05-20T18:29:02.677182+0000","last_unstale":"2024-05-20T18:29:04.081575+0000","last_undegraded":"2024-05-20T18:29:04.081575+0000","last_fullsized":"2024-05-20T18:29:04.081575+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:40:21.267154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"50'4","reported_seq":18,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.673132+0000","last_change":"2024-05-20T18:29:01.654415+0000","last_active":"2024-05-20T18:29:03.673132+0000","last_peered":"2024-05-20T18:29:03.673132+0000","last_clean":"2024-05-20T18:29:03.673132+0000","last_became_active":"2024-05-20T18:29:01.654065+0000","last_became_peered":"2024-05-20T18:29:01.654065+0000","last_unstale":"2024-05-20T18:29:03.673132+0000","last_undegraded":"2024-05-20T18:29:03.673132+0000","last_fullsized":"2024-05-20T18:29:03.673132+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:43:56.853477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082429+0000","last_change":"2024-05-20T18:29:02.679961+0000","last_active":"2024-05-20T18:29:04.082429+0000","last_peered":"2024-05-20T18:29:04.082429+0000","last_clean":"2024-05-20T18:29:04.082429+0000","last_became_active":"2024-05-20T18:29:02.679011+0000","last_became_peered":"2024-05-20T18:29:02.679011+0000","last_unstale":"2024-05-20T18:29:04.082429+0000","last_undegraded":"2024-05-20T18:29:04.082429+0000","last_fullsized":"2024-05-20T18:29:04.082429+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:48:55.261971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081509+0000","last_change":"2024-05-20T18:29:01.658924+0000","last_active":"2024-05-20T18:29:04.081509+0000","last_peered":"2024-05-20T18:29:04.081509+0000","last_clean":"2024-05-20T18:29:04.081509+0000","last_became_active":"2024-05-20T18:29:01.658584+0000","last_became_peered":"2024-05-20T18:29:01.658584+0000","last_unstale":"2024-05-20T18:29:04.081509+0000","last_undegraded":"2024-05-20T18:29:04.081509+0000","last_fullsized":"2024-05-20T18:29:04.081509+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:33.142624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.673719+0000","last_change":"2024-05-20T18:29:02.679328+0000","last_active":"2024-05-20T18:29:03.673719+0000","last_peered":"2024-05-20T18:29:03.673719+0000","last_clean":"2024-05-20T18:29:03.673719+0000","last_became_active":"2024-05-20T18:29:02.678993+0000","last_became_peered":"2024-05-20T18:29:02.678993+0000","last_unstale":"2024-05-20T18:29:03.673719+0000","last_undegraded":"2024-05-20T18:29:03.673719+0000","last_fullsized":"2024-05-20T18:29:03.673719+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:30:15.509281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081835+0000","last_change":"2024-05-20T18:29:01.657834+0000","last_active":"2024-05-20T18:29:04.081835+0000","last_peered":"2024-05-20T18:29:04.081835+0000","last_clean":"2024-05-20T18:29:04.081835+0000","last_became_active":"2024-05-20T18:29:01.657409+0000","last_became_peered":"2024-05-20T18:29:01.657409+0000","last_unstale":"2024-05-20T18:29:04.081835+0000","last_undegraded":"2024-05-20T18:29:04.081835+0000","last_fullsized":"2024-05-20T18:29:04.081835+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:25:22.102705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081775+0000","last_change":"2024-05-20T18:29:02.675977+0000","last_active":"2024-05-20T18:29:04.081775+0000","last_peered":"2024-05-20T18:29:04.081775+0000","last_clean":"2024-05-20T18:29:04.081775+0000","last_became_active":"2024-05-20T18:29:02.675528+0000","last_became_peered":"2024-05-20T18:29:02.675528+0000","last_unstale":"2024-05-20T18:29:04.081775+0000","last_undegraded":"2024-05-20T18:29:04.081775+0000","last_fullsized":"2024-05-20T18:29:04.081775+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:42:29.512485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081971+0000","last_change":"2024-05-20T18:29:01.661584+0000","last_active":"2024-05-20T18:29:04.081971+0000","last_peered":"2024-05-20T18:29:04.081971+0000","last_clean":"2024-05-20T18:29:04.081971+0000","last_became_active":"2024-05-20T18:29:01.660902+0000","last_became_peered":"2024-05-20T18:29:01.660902+0000","last_unstale":"2024-05-20T18:29:04.081971+0000","last_undegraded":"2024-05-20T18:29:04.081971+0000","last_fullsized":"2024-05-20T18:29:04.081971+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:00:41.856150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.673579+0000","last_change":"2024-05-20T18:29:02.683305+0000","last_active":"2024-05-20T18:29:03.673579+0000","last_peered":"2024-05-20T18:29:03.673579+0000","last_clean":"2024-05-20T18:29:03.673579+0000","last_became_active":"2024-05-20T18:29:02.682926+0000","last_became_peered":"2024-05-20T18:29:02.682926+0000","last_unstale":"2024-05-20T18:29:03.673579+0000","last_undegraded":"2024-05-20T18:29:03.673579+0000","last_fullsized":"2024-05-20T18:29:03.673579+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:38:12.409535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081488+0000","last_change":"2024-05-20T18:29:01.654206+0000","last_active":"2024-05-20T18:29:04.081488+0000","last_peered":"2024-05-20T18:29:04.081488+0000","last_clean":"2024-05-20T18:29:04.081488+0000","last_became_active":"2024-05-20T18:29:01.653881+0000","last_became_peered":"2024-05-20T18:29:01.653881+0000","last_unstale":"2024-05-20T18:29:04.081488+0000","last_undegraded":"2024-05-20T18:29:04.081488+0000","last_fullsized":"2024-05-20T18:29:04.081488+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:18:13.553828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082384+0000","last_change":"2024-05-20T18:29:02.681641+0000","last_active":"2024-05-20T18:29:04.082384+0000","last_peered":"2024-05-20T18:29:04.082384+0000","last_clean":"2024-05-20T18:29:04.082384+0000","last_became_active":"2024-05-20T18:29:02.681080+0000","last_became_peered":"2024-05-20T18:29:02.681080+0000","last_unstale":"2024-05-20T18:29:04.082384+0000","last_undegraded":"2024-05-20T18:29:04.082384+0000","last_fullsized":"2024-05-20T18:29:04.082384+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:28:13.588490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.672945+0000","last_change":"2024-05-20T18:29:01.654251+0000","last_active":"2024-05-20T18:29:03.672945+0000","last_peered":"2024-05-20T18:29:03.672945+0000","last_clean":"2024-05-20T18:29:03.672945+0000","last_became_active":"2024-05-20T18:29:01.653907+0000","last_became_peered":"2024-05-20T18:29:01.653907+0000","last_unstale":"2024-05-20T18:29:03.672945+0000","last_undegraded":"2024-05-20T18:29:03.672945+0000","last_fullsized":"2024-05-20T18:29:03.672945+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:25:22.102705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082225+0000","last_change":"2024-05-20T18:29:02.674929+0000","last_active":"2024-05-20T18:29:04.082225+0000","last_peered":"2024-05-20T18:29:04.082225+0000","last_clean":"2024-05-20T18:29:04.082225+0000","last_became_active":"2024-05-20T18:29:02.674257+0000","last_became_peered":"2024-05-20T18:29:02.674257+0000","last_unstale":"2024-05-20T18:29:04.082225+0000","last_undegraded":"2024-05-20T18:29:04.082225+0000","last_fullsized":"2024-05-20T18:29:04.082225+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:26:42.466485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.673794+0000","last_change":"2024-05-20T18:29:01.651981+0000","last_active":"2024-05-20T18:29:03.673794+0000","last_peered":"2024-05-20T18:29:03.673794+0000","last_clean":"2024-05-20T18:29:03.673794+0000","last_became_active":"2024-05-20T18:29:01.651648+0000","last_became_peered":"2024-05-20T18:29:01.651648+0000","last_unstale":"2024-05-20T18:29:03.673794+0000","last_undegraded":"2024-05-20T18:29:03.673794+0000","last_fullsized":"2024-05-20T18:29:03.673794+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:11:29.346230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081856+0000","last_change":"2024-05-20T18:29:02.677636+0000","last_active":"2024-05-20T18:29:04.081856+0000","last_peered":"2024-05-20T18:29:04.081856+0000","last_clean":"2024-05-20T18:29:04.081856+0000","last_became_active":"2024-05-20T18:29:02.677237+0000","last_became_peered":"2024-05-20T18:29:02.677237+0000","last_unstale":"2024-05-20T18:29:04.081856+0000","last_undegraded":"2024-05-20T18:29:04.081856+0000","last_fullsized":"2024-05-20T18:29:04.081856+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:32.845836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081106+0000","last_change":"2024-05-20T18:29:01.658231+0000","last_active":"2024-05-20T18:29:04.081106+0000","last_peered":"2024-05-20T18:29:04.081106+0000","last_clean":"2024-05-20T18:29:04.081106+0000","last_became_active":"2024-05-20T18:29:01.657925+0000","last_became_peered":"2024-05-20T18:29:01.657925+0000","last_unstale":"2024-05-20T18:29:04.081106+0000","last_undegraded":"2024-05-20T18:29:04.081106+0000","last_fullsized":"2024-05-20T18:29:04.081106+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:13:37.738664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.672999+0000","last_change":"2024-05-20T18:29:02.687337+0000","last_active":"2024-05-20T18:29:03.672999+0000","last_peered":"2024-05-20T18:29:03.672999+0000","last_clean":"2024-05-20T18:29:03.672999+0000","last_became_active":"2024-05-20T18:29:02.686984+0000","last_became_peered":"2024-05-20T18:29:02.686984+0000","last_unstale":"2024-05-20T18:29:03.672999+0000","last_undegraded":"2024-05-20T18:29:03.672999+0000","last_fullsized":"2024-05-20T18:29:03.672999+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:46:51.918262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081932+0000","last_change":"2024-05-20T18:29:01.659390+0000","last_active":"2024-05-20T18:29:04.081932+0000","last_peered":"2024-05-20T18:29:04.081932+0000","last_clean":"2024-05-20T18:29:04.081932+0000","last_became_active":"2024-05-20T18:29:01.659059+0000","last_became_peered":"2024-05-20T18:29:01.659059+0000","last_unstale":"2024-05-20T18:29:04.081932+0000","last_undegraded":"2024-05-20T18:29:04.081932+0000","last_fullsized":"2024-05-20T18:29:04.081932+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:38:11.404962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082448+0000","last_change":"2024-05-20T18:29:02.668562+0000","last_active":"2024-05-20T18:29:04.082448+0000","last_peered":"2024-05-20T18:29:04.082448+0000","last_clean":"2024-05-20T18:29:04.082448+0000","last_became_active":"2024-05-20T18:29:02.668168+0000","last_became_peered":"2024-05-20T18:29:02.668168+0000","last_unstale":"2024-05-20T18:29:04.082448+0000","last_undegraded":"2024-05-20T18:29:04.082448+0000","last_fullsized":"2024-05-20T18:29:04.082448+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:24:27.542753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.673281+0000","last_change":"2024-05-20T18:29:01.653200+0000","last_active":"2024-05-20T18:29:03.673281+0000","last_peered":"2024-05-20T18:29:03.673281+0000","last_clean":"2024-05-20T18:29:03.673281+0000","last_became_active":"2024-05-20T18:29:01.652885+0000","last_became_peered":"2024-05-20T18:29:01.652885+0000","last_unstale":"2024-05-20T18:29:03.673281+0000","last_undegraded":"2024-05-20T18:29:03.673281+0000","last_fullsized":"2024-05-20T18:29:03.673281+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:17:01.882291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082718+0000","last_change":"2024-05-20T18:29:02.686754+0000","last_active":"2024-05-20T18:29:04.082718+0000","last_peered":"2024-05-20T18:29:04.082718+0000","last_clean":"2024-05-20T18:29:04.082718+0000","last_became_active":"2024-05-20T18:29:02.686348+0000","last_became_peered":"2024-05-20T18:29:02.686348+0000","last_unstale":"2024-05-20T18:29:04.082718+0000","last_undegraded":"2024-05-20T18:29:04.082718+0000","last_fullsized":"2024-05-20T18:29:04.082718+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:17:36.464439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"50'3","reported_seq":17,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082492+0000","last_change":"2024-05-20T18:29:01.663167+0000","last_active":"2024-05-20T18:29:04.082492+0000","last_peered":"2024-05-20T18:29:04.082492+0000","last_clean":"2024-05-20T18:29:04.082492+0000","last_became_active":"2024-05-20T18:29:01.662800+0000","last_became_peered":"2024-05-20T18:29:01.662800+0000","last_unstale":"2024-05-20T18:29:04.082492+0000","last_undegraded":"2024-05-20T18:29:04.082492+0000","last_fullsized":"2024-05-20T18:29:04.082492+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:46:50.913690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.674266+0000","last_change":"2024-05-20T18:29:02.672132+0000","last_active":"2024-05-20T18:29:03.674266+0000","last_peered":"2024-05-20T18:29:03.674266+0000","last_clean":"2024-05-20T18:29:03.674266+0000","last_became_active":"2024-05-20T18:29:02.671790+0000","last_became_peered":"2024-05-20T18:29:02.671790+0000","last_unstale":"2024-05-20T18:29:03.674266+0000","last_undegraded":"2024-05-20T18:29:03.674266+0000","last_fullsized":"2024-05-20T18:29:03.674266+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:30:15.509281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"50'6","reported_seq":20,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082901+0000","last_change":"2024-05-20T18:29:01.661776+0000","last_active":"2024-05-20T18:29:04.082901+0000","last_peered":"2024-05-20T18:29:04.082901+0000","last_clean":"2024-05-20T18:29:04.082901+0000","last_became_active":"2024-05-20T18:29:01.661386+0000","last_became_peered":"2024-05-20T18:29:01.661386+0000","last_unstale":"2024-05-20T18:29:04.082901+0000","last_undegraded":"2024-05-20T18:29:04.082901+0000","last_fullsized":"2024-05-20T18:29:04.082901+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:25:09.902538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081856+0000","last_change":"2024-05-20T18:29:02.678560+0000","last_active":"2024-05-20T18:29:04.081856+0000","last_peered":"2024-05-20T18:29:04.081856+0000","last_clean":"2024-05-20T18:29:04.081856+0000","last_became_active":"2024-05-20T18:29:02.678170+0000","last_became_peered":"2024-05-20T18:29:02.678170+0000","last_unstale":"2024-05-20T18:29:04.081856+0000","last_undegraded":"2024-05-20T18:29:04.081856+0000","last_fullsized":"2024-05-20T18:29:04.081856+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:19:37.476208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081607+0000","last_change":"2024-05-20T18:29:01.656783+0000","last_active":"2024-05-20T18:29:04.081607+0000","last_peered":"2024-05-20T18:29:04.081607+0000","last_clean":"2024-05-20T18:29:04.081607+0000","last_became_active":"2024-05-20T18:29:01.656306+0000","last_became_peered":"2024-05-20T18:29:01.656306+0000","last_unstale":"2024-05-20T18:29:04.081607+0000","last_undegraded":"2024-05-20T18:29:04.081607+0000","last_fullsized":"2024-05-20T18:29:04.081607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:46:50.913690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082743+0000","last_change":"2024-05-20T18:29:02.685760+0000","last_active":"2024-05-20T18:29:04.082743+0000","last_peered":"2024-05-20T18:29:04.082743+0000","last_clean":"2024-05-20T18:29:04.082743+0000","last_became_active":"2024-05-20T18:29:02.685135+0000","last_became_peered":"2024-05-20T18:29:02.685135+0000","last_unstale":"2024-05-20T18:29:04.082743+0000","last_undegraded":"2024-05-20T18:29:04.082743+0000","last_fullsized":"2024-05-20T18:29:04.082743+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:46:20.452501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.673891+0000","last_change":"2024-05-20T18:29:01.659912+0000","last_active":"2024-05-20T18:29:03.673891+0000","last_peered":"2024-05-20T18:29:03.673891+0000","last_clean":"2024-05-20T18:29:03.673891+0000","last_became_active":"2024-05-20T18:29:01.658865+0000","last_became_peered":"2024-05-20T18:29:01.658865+0000","last_unstale":"2024-05-20T18:29:03.673891+0000","last_undegraded":"2024-05-20T18:29:03.673891+0000","last_fullsized":"2024-05-20T18:29:03.673891+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:33.142624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082264+0000","last_change":"2024-05-20T18:29:02.679515+0000","last_active":"2024-05-20T18:29:04.082264+0000","last_peered":"2024-05-20T18:29:04.082264+0000","last_clean":"2024-05-20T18:29:04.082264+0000","last_became_active":"2024-05-20T18:29:02.679183+0000","last_became_peered":"2024-05-20T18:29:02.679183+0000","last_unstale":"2024-05-20T18:29:04.082264+0000","last_undegraded":"2024-05-20T18:29:04.082264+0000","last_fullsized":"2024-05-20T18:29:04.082264+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:15:37.388487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082408+0000","last_change":"2024-05-20T18:29:01.659062+0000","last_active":"2024-05-20T18:29:04.082408+0000","last_peered":"2024-05-20T18:29:04.082408+0000","last_clean":"2024-05-20T18:29:04.082408+0000","last_became_active":"2024-05-20T18:29:01.658658+0000","last_became_peered":"2024-05-20T18:29:01.658658+0000","last_unstale":"2024-05-20T18:29:04.082408+0000","last_undegraded":"2024-05-20T18:29:04.082408+0000","last_fullsized":"2024-05-20T18:29:04.082408+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:18:13.553828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.083022+0000","last_change":"2024-05-20T18:29:02.682398+0000","last_active":"2024-05-20T18:29:04.083022+0000","last_peered":"2024-05-20T18:29:04.083022+0000","last_clean":"2024-05-20T18:29:04.083022+0000","last_became_active":"2024-05-20T18:29:02.682022+0000","last_became_peered":"2024-05-20T18:29:02.682022+0000","last_unstale":"2024-05-20T18:29:04.083022+0000","last_undegraded":"2024-05-20T18:29:04.083022+0000","last_fullsized":"2024-05-20T18:29:04.083022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:35:26.747322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"50'8","reported_seq":22,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082437+0000","last_change":"2024-05-20T18:29:01.663928+0000","last_active":"2024-05-20T18:29:04.082437+0000","last_peered":"2024-05-20T18:29:04.082437+0000","last_clean":"2024-05-20T18:29:04.082437+0000","last_became_active":"2024-05-20T18:29:01.663570+0000","last_became_peered":"2024-05-20T18:29:01.663570+0000","last_unstale":"2024-05-20T18:29:04.082437+0000","last_undegraded":"2024-05-20T18:29:04.082437+0000","last_fullsized":"2024-05-20T18:29:04.082437+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:20:48.010059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082509+0000","last_change":"2024-05-20T18:29:02.678528+0000","last_active":"2024-05-20T18:29:04.082509+0000","last_peered":"2024-05-20T18:29:04.082509+0000","last_clean":"2024-05-20T18:29:04.082509+0000","last_became_active":"2024-05-20T18:29:02.676260+0000","last_became_peered":"2024-05-20T18:29:02.676260+0000","last_unstale":"2024-05-20T18:29:04.082509+0000","last_undegraded":"2024-05-20T18:29:04.082509+0000","last_fullsized":"2024-05-20T18:29:04.082509+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:06:13.229296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.083098+0000","last_change":"2024-05-20T18:29:01.659497+0000","last_active":"2024-05-20T18:29:04.083098+0000","last_peered":"2024-05-20T18:29:04.083098+0000","last_clean":"2024-05-20T18:29:04.083098+0000","last_became_active":"2024-05-20T18:29:01.659189+0000","last_became_peered":"2024-05-20T18:29:01.659189+0000","last_unstale":"2024-05-20T18:29:04.083098+0000","last_undegraded":"2024-05-20T18:29:04.083098+0000","last_fullsized":"2024-05-20T18:29:04.083098+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:31:10.334541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081235+0000","last_change":"2024-05-20T18:29:02.693178+0000","last_active":"2024-05-20T18:29:04.081235+0000","last_peered":"2024-05-20T18:29:04.081235+0000","last_clean":"2024-05-20T18:29:04.081235+0000","last_became_active":"2024-05-20T18:29:02.692824+0000","last_became_peered":"2024-05-20T18:29:02.692824+0000","last_unstale":"2024-05-20T18:29:04.081235+0000","last_undegraded":"2024-05-20T18:29:04.081235+0000","last_fullsized":"2024-05-20T18:29:04.081235+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:00:42.860723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.672481+0000","last_change":"2024-05-20T18:29:01.659031+0000","last_active":"2024-05-20T18:29:03.672481+0000","last_peered":"2024-05-20T18:29:03.672481+0000","last_clean":"2024-05-20T18:29:03.672481+0000","last_became_active":"2024-05-20T18:29:01.658255+0000","last_became_peered":"2024-05-20T18:29:01.658255+0000","last_unstale":"2024-05-20T18:29:03.672481+0000","last_undegraded":"2024-05-20T18:29:03.672481+0000","last_fullsized":"2024-05-20T18:29:03.672481+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:46:43.782231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.673001+0000","last_change":"2024-05-20T18:29:02.683982+0000","last_active":"2024-05-20T18:29:03.673001+0000","last_peered":"2024-05-20T18:29:03.673001+0000","last_clean":"2024-05-20T18:29:03.673001+0000","last_became_active":"2024-05-20T18:29:02.683615+0000","last_became_peered":"2024-05-20T18:29:02.683615+0000","last_unstale":"2024-05-20T18:29:03.673001+0000","last_undegraded":"2024-05-20T18:29:03.673001+0000","last_fullsized":"2024-05-20T18:29:03.673001+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:18:14.558400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.673717+0000","last_change":"2024-05-20T18:29:01.655986+0000","last_active":"2024-05-20T18:29:03.673717+0000","last_peered":"2024-05-20T18:29:03.673717+0000","last_clean":"2024-05-20T18:29:03.673717+0000","last_became_active":"2024-05-20T18:29:01.655641+0000","last_became_peered":"2024-05-20T18:29:01.655641+0000","last_unstale":"2024-05-20T18:29:03.673717+0000","last_undegraded":"2024-05-20T18:29:03.673717+0000","last_fullsized":"2024-05-20T18:29:03.673717+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:26:48.032272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081744+0000","last_change":"2024-05-20T18:29:02.683354+0000","last_active":"2024-05-20T18:29:04.081744+0000","last_peered":"2024-05-20T18:29:04.081744+0000","last_clean":"2024-05-20T18:29:04.081744+0000","last_became_active":"2024-05-20T18:29:02.680578+0000","last_became_peered":"2024-05-20T18:29:02.680578+0000","last_unstale":"2024-05-20T18:29:04.081744+0000","last_undegraded":"2024-05-20T18:29:04.081744+0000","last_fullsized":"2024-05-20T18:29:04.081744+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:56:58.199488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.673340+0000","last_change":"2024-05-20T18:29:01.661629+0000","last_active":"2024-05-20T18:29:03.673340+0000","last_peered":"2024-05-20T18:29:03.673340+0000","last_clean":"2024-05-20T18:29:03.673340+0000","last_became_active":"2024-05-20T18:29:01.661268+0000","last_became_peered":"2024-05-20T18:29:01.661268+0000","last_unstale":"2024-05-20T18:29:03.673340+0000","last_undegraded":"2024-05-20T18:29:03.673340+0000","last_fullsized":"2024-05-20T18:29:03.673340+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:25:22.102705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082072+0000","last_change":"2024-05-20T18:29:02.670089+0000","last_active":"2024-05-20T18:29:04.082072+0000","last_peered":"2024-05-20T18:29:04.082072+0000","last_clean":"2024-05-20T18:29:04.082072+0000","last_became_active":"2024-05-20T18:29:02.669740+0000","last_became_peered":"2024-05-20T18:29:02.669740+0000","last_unstale":"2024-05-20T18:29:04.082072+0000","last_undegraded":"2024-05-20T18:29:04.082072+0000","last_fullsized":"2024-05-20T18:29:04.082072+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:26:40.141288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081811+0000","last_change":"2024-05-20T18:29:01.664131+0000","last_active":"2024-05-20T18:29:04.081811+0000","last_peered":"2024-05-20T18:29:04.081811+0000","last_clean":"2024-05-20T18:29:04.081811+0000","last_became_active":"2024-05-20T18:29:01.663780+0000","last_became_peered":"2024-05-20T18:29:01.663780+0000","last_unstale":"2024-05-20T18:29:04.081811+0000","last_undegraded":"2024-05-20T18:29:04.081811+0000","last_fullsized":"2024-05-20T18:29:04.081811+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:11:29.346230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.673639+0000","last_change":"2024-05-20T18:29:02.661856+0000","last_active":"2024-05-20T18:29:03.673639+0000","last_peered":"2024-05-20T18:29:03.673639+0000","last_clean":"2024-05-20T18:29:03.673639+0000","last_became_active":"2024-05-20T18:29:02.661520+0000","last_became_peered":"2024-05-20T18:29:02.661520+0000","last_unstale":"2024-05-20T18:29:03.673639+0000","last_undegraded":"2024-05-20T18:29:03.673639+0000","last_fullsized":"2024-05-20T18:29:03.673639+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:13:38.743236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081568+0000","last_change":"2024-05-20T18:29:01.665733+0000","last_active":"2024-05-20T18:29:04.081568+0000","last_peered":"2024-05-20T18:29:04.081568+0000","last_clean":"2024-05-20T18:29:04.081568+0000","last_became_active":"2024-05-20T18:29:01.665385+0000","last_became_peered":"2024-05-20T18:29:01.665385+0000","last_unstale":"2024-05-20T18:29:04.081568+0000","last_undegraded":"2024-05-20T18:29:04.081568+0000","last_fullsized":"2024-05-20T18:29:04.081568+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:07:05.206123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082263+0000","last_change":"2024-05-20T18:29:02.688528+0000","last_active":"2024-05-20T18:29:04.082263+0000","last_peered":"2024-05-20T18:29:04.082263+0000","last_clean":"2024-05-20T18:29:04.082263+0000","last_became_active":"2024-05-20T18:29:02.688185+0000","last_became_peered":"2024-05-20T18:29:02.688185+0000","last_unstale":"2024-05-20T18:29:04.082263+0000","last_undegraded":"2024-05-20T18:29:04.082263+0000","last_fullsized":"2024-05-20T18:29:04.082263+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:55:07.462160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081875+0000","last_change":"2024-05-20T18:29:01.655080+0000","last_active":"2024-05-20T18:29:04.081875+0000","last_peered":"2024-05-20T18:29:04.081875+0000","last_clean":"2024-05-20T18:29:04.081875+0000","last_became_active":"2024-05-20T18:29:01.654745+0000","last_became_peered":"2024-05-20T18:29:01.654745+0000","last_unstale":"2024-05-20T18:29:04.081875+0000","last_undegraded":"2024-05-20T18:29:04.081875+0000","last_fullsized":"2024-05-20T18:29:04.081875+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:14:02.291830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082264+0000","last_change":"2024-05-20T18:29:02.689391+0000","last_active":"2024-05-20T18:29:04.082264+0000","last_peered":"2024-05-20T18:29:04.082264+0000","last_clean":"2024-05-20T18:29:04.082264+0000","last_became_active":"2024-05-20T18:29:02.687198+0000","last_became_peered":"2024-05-20T18:29:02.687198+0000","last_unstale":"2024-05-20T18:29:04.082264+0000","last_undegraded":"2024-05-20T18:29:04.082264+0000","last_fullsized":"2024-05-20T18:29:04.082264+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:32.845836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.672053+0000","last_change":"2024-05-20T18:29:01.654572+0000","last_active":"2024-05-20T18:29:03.672053+0000","last_peered":"2024-05-20T18:29:03.672053+0000","last_clean":"2024-05-20T18:29:03.672053+0000","last_became_active":"2024-05-20T18:29:01.654242+0000","last_became_peered":"2024-05-20T18:29:01.654242+0000","last_unstale":"2024-05-20T18:29:03.672053+0000","last_undegraded":"2024-05-20T18:29:03.672053+0000","last_fullsized":"2024-05-20T18:29:03.672053+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:12:23.698473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082361+0000","last_change":"2024-05-20T18:29:02.682283+0000","last_active":"2024-05-20T18:29:04.082361+0000","last_peered":"2024-05-20T18:29:04.082361+0000","last_clean":"2024-05-20T18:29:04.082361+0000","last_became_active":"2024-05-20T18:29:02.681928+0000","last_became_peered":"2024-05-20T18:29:02.681928+0000","last_unstale":"2024-05-20T18:29:04.082361+0000","last_undegraded":"2024-05-20T18:29:04.082361+0000","last_fullsized":"2024-05-20T18:29:04.082361+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:34:33.698879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.673302+0000","last_change":"2024-05-20T18:29:01.659515+0000","last_active":"2024-05-20T18:29:03.673302+0000","last_peered":"2024-05-20T18:29:03.673302+0000","last_clean":"2024-05-20T18:29:03.673302+0000","last_became_active":"2024-05-20T18:29:01.659066+0000","last_became_peered":"2024-05-20T18:29:01.659066+0000","last_unstale":"2024-05-20T18:29:03.673302+0000","last_undegraded":"2024-05-20T18:29:03.673302+0000","last_fullsized":"2024-05-20T18:29:03.673302+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:49:42.304308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081797+0000","last_change":"2024-05-20T18:29:02.677236+0000","last_active":"2024-05-20T18:29:04.081797+0000","last_peered":"2024-05-20T18:29:04.081797+0000","last_clean":"2024-05-20T18:29:04.081797+0000","last_became_active":"2024-05-20T18:29:02.676006+0000","last_became_peered":"2024-05-20T18:29:02.676006+0000","last_unstale":"2024-05-20T18:29:04.081797+0000","last_undegraded":"2024-05-20T18:29:04.081797+0000","last_fullsized":"2024-05-20T18:29:04.081797+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:02:01.847309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082493+0000","last_change":"2024-05-20T18:29:01.658454+0000","last_active":"2024-05-20T18:29:04.082493+0000","last_peered":"2024-05-20T18:29:04.082493+0000","last_clean":"2024-05-20T18:29:04.082493+0000","last_became_active":"2024-05-20T18:29:01.658083+0000","last_became_peered":"2024-05-20T18:29:01.658083+0000","last_unstale":"2024-05-20T18:29:04.082493+0000","last_undegraded":"2024-05-20T18:29:04.082493+0000","last_fullsized":"2024-05-20T18:29:04.082493+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:13:37.738664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081960+0000","last_change":"2024-05-20T18:29:02.675916+0000","last_active":"2024-05-20T18:29:04.081960+0000","last_peered":"2024-05-20T18:29:04.081960+0000","last_clean":"2024-05-20T18:29:04.081960+0000","last_became_active":"2024-05-20T18:29:02.675603+0000","last_became_peered":"2024-05-20T18:29:02.675603+0000","last_unstale":"2024-05-20T18:29:04.081960+0000","last_undegraded":"2024-05-20T18:29:04.081960+0000","last_fullsized":"2024-05-20T18:29:04.081960+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:15:22.834731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081967+0000","last_change":"2024-05-20T18:29:01.647606+0000","last_active":"2024-05-20T18:29:04.081967+0000","last_peered":"2024-05-20T18:29:04.081967+0000","last_clean":"2024-05-20T18:29:04.081967+0000","last_became_active":"2024-05-20T18:29:01.647265+0000","last_became_peered":"2024-05-20T18:29:01.647265+0000","last_unstale":"2024-05-20T18:29:04.081967+0000","last_undegraded":"2024-05-20T18:29:04.081967+0000","last_fullsized":"2024-05-20T18:29:04.081967+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:21:26.188155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081825+0000","last_change":"2024-05-20T18:29:02.691656+0000","last_active":"2024-05-20T18:29:04.081825+0000","last_peered":"2024-05-20T18:29:04.081825+0000","last_clean":"2024-05-20T18:29:04.081825+0000","last_became_active":"2024-05-20T18:29:02.690648+0000","last_became_peered":"2024-05-20T18:29:02.690648+0000","last_unstale":"2024-05-20T18:29:04.081825+0000","last_undegraded":"2024-05-20T18:29:04.081825+0000","last_fullsized":"2024-05-20T18:29:04.081825+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:31:23.977414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082027+0000","last_change":"2024-05-20T18:29:01.663028+0000","last_active":"2024-05-20T18:29:04.082027+0000","last_peered":"2024-05-20T18:29:04.082027+0000","last_clean":"2024-05-20T18:29:04.082027+0000","last_became_active":"2024-05-20T18:29:01.662663+0000","last_became_peered":"2024-05-20T18:29:01.662663+0000","last_unstale":"2024-05-20T18:29:04.082027+0000","last_undegraded":"2024-05-20T18:29:04.082027+0000","last_fullsized":"2024-05-20T18:29:04.082027+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:55:06.457587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081908+0000","last_change":"2024-05-20T18:29:02.678613+0000","last_active":"2024-05-20T18:29:04.081908+0000","last_peered":"2024-05-20T18:29:04.081908+0000","last_clean":"2024-05-20T18:29:04.081908+0000","last_became_active":"2024-05-20T18:29:02.678245+0000","last_became_peered":"2024-05-20T18:29:02.678245+0000","last_unstale":"2024-05-20T18:29:04.081908+0000","last_undegraded":"2024-05-20T18:29:04.081908+0000","last_fullsized":"2024-05-20T18:29:04.081908+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:28:13.588490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081688+0000","last_change":"2024-05-20T18:29:01.658295+0000","last_active":"2024-05-20T18:29:04.081688+0000","last_peered":"2024-05-20T18:29:04.081688+0000","last_clean":"2024-05-20T18:29:04.081688+0000","last_became_active":"2024-05-20T18:29:01.657954+0000","last_became_peered":"2024-05-20T18:29:01.657954+0000","last_unstale":"2024-05-20T18:29:04.081688+0000","last_undegraded":"2024-05-20T18:29:04.081688+0000","last_fullsized":"2024-05-20T18:29:04.081688+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:31:22.972841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082100+0000","last_change":"2024-05-20T18:29:02.683947+0000","last_active":"2024-05-20T18:29:04.082100+0000","last_peered":"2024-05-20T18:29:04.082100+0000","last_clean":"2024-05-20T18:29:04.082100+0000","last_became_active":"2024-05-20T18:29:02.679378+0000","last_became_peered":"2024-05-20T18:29:02.679378+0000","last_unstale":"2024-05-20T18:29:04.082100+0000","last_undegraded":"2024-05-20T18:29:04.082100+0000","last_fullsized":"2024-05-20T18:29:04.082100+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:20:49.014632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.672829+0000","last_change":"2024-05-20T18:29:01.658476+0000","last_active":"2024-05-20T18:29:03.672829+0000","last_peered":"2024-05-20T18:29:03.672829+0000","last_clean":"2024-05-20T18:29:03.672829+0000","last_became_active":"2024-05-20T18:29:01.658139+0000","last_became_peered":"2024-05-20T18:29:01.658139+0000","last_unstale":"2024-05-20T18:29:03.672829+0000","last_undegraded":"2024-05-20T18:29:03.672829+0000","last_fullsized":"2024-05-20T18:29:03.672829+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:11:29.346230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082349+0000","last_change":"2024-05-20T18:29:02.677001+0000","last_active":"2024-05-20T18:29:04.082349+0000","last_peered":"2024-05-20T18:29:04.082349+0000","last_clean":"2024-05-20T18:29:04.082349+0000","last_became_active":"2024-05-20T18:29:02.676626+0000","last_became_peered":"2024-05-20T18:29:02.676626+0000","last_unstale":"2024-05-20T18:29:04.082349+0000","last_undegraded":"2024-05-20T18:29:04.082349+0000","last_fullsized":"2024-05-20T18:29:04.082349+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:47:29.251792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082042+0000","last_change":"2024-05-20T18:29:01.662968+0000","last_active":"2024-05-20T18:29:04.082042+0000","last_peered":"2024-05-20T18:29:04.082042+0000","last_clean":"2024-05-20T18:29:04.082042+0000","last_became_active":"2024-05-20T18:29:01.662519+0000","last_became_peered":"2024-05-20T18:29:01.662519+0000","last_unstale":"2024-05-20T18:29:04.082042+0000","last_undegraded":"2024-05-20T18:29:04.082042+0000","last_fullsized":"2024-05-20T18:29:04.082042+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:02:27.747348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.672999+0000","last_change":"2024-05-20T18:29:02.673465+0000","last_active":"2024-05-20T18:29:03.672999+0000","last_peered":"2024-05-20T18:29:03.672999+0000","last_clean":"2024-05-20T18:29:03.672999+0000","last_became_active":"2024-05-20T18:29:02.673113+0000","last_became_peered":"2024-05-20T18:29:02.673113+0000","last_unstale":"2024-05-20T18:29:03.672999+0000","last_undegraded":"2024-05-20T18:29:03.672999+0000","last_fullsized":"2024-05-20T18:29:03.672999+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:18:14.558400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"51'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082213+0000","last_change":"2024-05-20T18:29:01.656841+0000","last_active":"2024-05-20T18:29:04.082213+0000","last_peered":"2024-05-20T18:29:04.082213+0000","last_clean":"2024-05-20T18:29:04.082213+0000","last_became_active":"2024-05-20T18:29:01.655870+0000","last_became_peered":"2024-05-20T18:29:01.655870+0000","last_unstale":"2024-05-20T18:29:04.082213+0000","last_undegraded":"2024-05-20T18:29:04.082213+0000","last_fullsized":"2024-05-20T18:29:04.082213+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:38:11.404962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":225,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082245+0000","last_change":"2024-05-20T18:27:03.765694+0000","last_active":"2024-05-20T18:29:04.082245+0000","last_peered":"2024-05-20T18:29:04.082245+0000","last_clean":"2024-05-20T18:29:04.082245+0000","last_became_active":"2024-05-20T18:27:03.764695+0000","last_became_peered":"2024-05-20T18:27:03.764695+0000","last_unstale":"2024-05-20T18:29:04.082245+0000","last_undegraded":"2024-05-20T18:29:04.082245+0000","last_fullsized":"2024-05-20T18:29:04.082245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_clean_scrub_stamp":"2024-05-20T18:27:02.750853+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:30:04.407487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28200,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343640,"statfs":{"total":96636764160,"available":96607887360,"internally_reserved":0,"allocated":1208320,"data_stored":859872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":614400,"data_stored":270032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":606208,"data_stored":269380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":610304,"data_stored":269962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31792,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340048,"statfs":{"total":96636764160,"available":96604209152,"internally_reserved":0,"allocated":622592,"data_stored":271014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":618496,"data_stored":270992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":614400,"data_stored":269540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1208320,"data_stored":859804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T18:29:06.737 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T18:29:06.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.888+0000 7fd592965640 1 Processor -- start 2024-05-20T18:29:06.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.888+0000 7fd592965640 1 -- start start 2024-05-20T18:29:06.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.890+0000 7fd592965640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd58c162820 0x7fd58c0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:06.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.890+0000 7fd592965640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd58c0b3e80 0x7fd58c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:06.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.890+0000 7fd592965640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd58c06dd20 con 0x7fd58c06d910 2024-05-20T18:29:06.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.890+0000 7fd592965640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd58c0d0630 con 0x7fd58c0b3e80 2024-05-20T18:29:06.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.890+0000 7fd592965640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd58c0d39c0 con 0x7fd58c162820 2024-05-20T18:29:06.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.890+0000 7fd58b7fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd58c0b3e80 0x7fd58c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:06.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.890+0000 7fd58bfff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd58c162820 0x7fd58c0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:06.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.890+0000 7fd590f02640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fd58c06d910 0x7fd58c0d27a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:47154/0 (socket says 172.21.15.5:47154) 2024-05-20T18:29:06.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.890+0000 7fd58bfff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd58c162820 0x7fd58c0b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:35038/0 (socket says 172.21.15.5:35038) 2024-05-20T18:29:06.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.890+0000 7fd590f02640 1 -- 172.21.15.5:0/2078269817 learned_addr learned my addr 172.21.15.5:0/2078269817 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:06.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.890+0000 7fd58bfff640 1 -- 172.21.15.5:0/2078269817 >> v2:172.21.15.63:3300/0 conn(0x7fd58c0b3e80 msgr2=0x7fd58c06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:06.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.890+0000 7fd58bfff640 1 --2- 172.21.15.5:0/2078269817 >> v2:172.21.15.63:3300/0 conn(0x7fd58c0b3e80 0x7fd58c06d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:06.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.890+0000 7fd58bfff640 1 -- 172.21.15.5:0/2078269817 >> v1:172.21.15.63:6789/0 conn(0x7fd58c06d910 legacy=0x7fd58c0d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:06.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.890+0000 7fd58bfff640 1 -- 172.21.15.5:0/2078269817 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58c0d3ca0 con 0x7fd58c162820 2024-05-20T18:29:06.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.890+0000 7fd58affd640 1 -- 172.21.15.5:0/2078269817 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3276156061 0 0) 0x7fd584001490 con 0x7fd58c06d910 2024-05-20T18:29:06.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.890+0000 7fd58b7fe640 1 --2- 172.21.15.5:0/2078269817 >> v2:172.21.15.63:3300/0 conn(0x7fd58c0b3e80 0x7fd58c06d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:29:06.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.891+0000 7fd58bfff640 1 --2- 172.21.15.5:0/2078269817 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd58c162820 0x7fd58c0b3940 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0097a0 tx=0x7fd57c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e259fe5efb31dec7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:06.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.891+0000 7fd58affd640 1 -- 172.21.15.5:0/2078269817 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c037070 con 0x7fd58c162820 2024-05-20T18:29:06.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.891+0000 7fd58affd640 1 -- 172.21.15.5:0/2078269817 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd57c032440 con 0x7fd58c162820 2024-05-20T18:29:06.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.891+0000 7fd58affd640 1 -- 172.21.15.5:0/2078269817 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c032810 con 0x7fd58c162820 2024-05-20T18:29:06.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.892+0000 7fd592965640 1 -- 172.21.15.5:0/2078269817 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd58c162820 msgr2=0x7fd58c0b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:06.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.892+0000 7fd592965640 1 --2- 172.21.15.5:0/2078269817 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd58c162820 0x7fd58c0b3940 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0097a0 tx=0x7fd57c02a180 comp rx=0 tx=0).stop 2024-05-20T18:29:06.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.892+0000 7fd592965640 1 -- 172.21.15.5:0/2078269817 shutdown_connections 2024-05-20T18:29:06.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.892+0000 7fd592965640 1 --2- 172.21.15.5:0/2078269817 >> v2:172.21.15.63:3300/0 conn(0x7fd58c0b3e80 0x7fd58c06d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:06.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.892+0000 7fd592965640 1 --2- 172.21.15.5:0/2078269817 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd58c162820 0x7fd58c0b3940 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:06.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.892+0000 7fd592965640 1 -- 172.21.15.5:0/2078269817 >> 172.21.15.5:0/2078269817 conn(0x7fd58c0d4bd0 msgr2=0x7fd58c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:06.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.892+0000 7fd592965640 1 -- 172.21.15.5:0/2078269817 shutdown_connections 2024-05-20T18:29:06.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.892+0000 7fd592965640 1 -- 172.21.15.5:0/2078269817 wait complete. 2024-05-20T18:29:06.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.894+0000 7fd592965640 1 Processor -- start 2024-05-20T18:29:06.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.894+0000 7fd592965640 1 -- start start 2024-05-20T18:29:06.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.895+0000 7fd592965640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd58c06d910 0x7fd58c0c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:06.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.895+0000 7fd592965640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd58c0b3e80 0x7fd58c0b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:06.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.895+0000 7fd592965640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd58c162820 0x7fd58c0b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:06.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.895+0000 7fd592965640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd58c0d0300 con 0x7fd58c162820 2024-05-20T18:29:06.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.895+0000 7fd592965640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd58c0d4260 con 0x7fd58c0b3e80 2024-05-20T18:29:06.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.895+0000 7fd592965640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd58c0b91a0 con 0x7fd58c06d910 2024-05-20T18:29:06.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.896+0000 7fd590f02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd58c162820 0x7fd58c0b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:06.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.896+0000 7fd590f02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd58c162820 0x7fd58c0b6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42808/0 (socket says 172.21.15.5:42808) 2024-05-20T18:29:06.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.896+0000 7fd590f02640 1 -- 172.21.15.5:0/2515319897 learned_addr learned my addr 172.21.15.5:0/2515319897 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:06.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.896+0000 7fd58b7fe640 1 --2- 172.21.15.5:0/2515319897 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd58c0b3e80 0x7fd58c0b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:06.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.896+0000 7fd58bfff640 1 --2- 172.21.15.5:0/2515319897 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd58c06d910 0x7fd58c0c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:06.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.896+0000 7fd590f02640 1 -- 172.21.15.5:0/2515319897 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd58c06d910 msgr2=0x7fd58c0c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:06.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.896+0000 7fd590f02640 1 --2- 172.21.15.5:0/2515319897 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd58c06d910 0x7fd58c0c8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:06.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.896+0000 7fd590f02640 1 -- 172.21.15.5:0/2515319897 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd58c0b3e80 msgr2=0x7fd58c0b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:06.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.896+0000 7fd590f02640 1 --2- 172.21.15.5:0/2515319897 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd58c0b3e80 0x7fd58c0b5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:06.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.896+0000 7fd590f02640 1 -- 172.21.15.5:0/2515319897 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58c15ac00 con 0x7fd58c162820 2024-05-20T18:29:06.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.896+0000 7fd590f02640 1 --2- 172.21.15.5:0/2515319897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd58c162820 0x7fd58c0b6180 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd58400c700 tx=0x7fd58400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:06.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.896+0000 7fd588ff9640 1 -- 172.21.15.5:0/2515319897 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd584018070 con 0x7fd58c162820 2024-05-20T18:29:06.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.896+0000 7fd588ff9640 1 -- 172.21.15.5:0/2515319897 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd58400ce10 con 0x7fd58c162820 2024-05-20T18:29:06.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.896+0000 7fd592965640 1 -- 172.21.15.5:0/2515319897 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd58c15b070 con 0x7fd58c162820 2024-05-20T18:29:06.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.896+0000 7fd588ff9640 1 -- 172.21.15.5:0/2515319897 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58400a860 con 0x7fd58c162820 2024-05-20T18:29:06.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.896+0000 7fd592965640 1 -- 172.21.15.5:0/2515319897 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd58c0c94d0 con 0x7fd58c162820 2024-05-20T18:29:06.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.897+0000 7fd592965640 1 -- 172.21.15.5:0/2515319897 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd58c15add0 con 0x7fd58c162820 2024-05-20T18:29:06.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.898+0000 7fd588ff9640 1 -- 172.21.15.5:0/2515319897 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd58400aa00 con 0x7fd58c162820 2024-05-20T18:29:06.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.899+0000 7fd588ff9640 1 --2- 172.21.15.5:0/2515319897 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd57806be60 0x7fd57806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:06.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.899+0000 7fd588ff9640 1 -- 172.21.15.5:0/2515319897 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7fd58408a570 con 0x7fd58c162820 2024-05-20T18:29:06.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.899+0000 7fd58bfff640 1 --2- 172.21.15.5:0/2515319897 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd57806be60 0x7fd57806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:06.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.899+0000 7fd58bfff640 1 --2- 172.21.15.5:0/2515319897 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd57806be60 0x7fd57806e370 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd57c034000 tx=0x7fd57c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:06.903+0000 7fd588ff9640 1 -- 172.21.15.5:0/2515319897 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd58c15add0 con 0x7fd58c162820 2024-05-20T18:29:07.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.084+0000 7fd592965640 1 -- 172.21.15.5:0/2515319897 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd58c075b00 con 0x7fd57806be60 2024-05-20T18:29:07.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.090+0000 7fd588ff9640 1 -- 172.21.15.5:0/2515319897 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251955 (secure 0 0 0) 0x7fd58c075b00 con 0x7fd57806be60 2024-05-20T18:29:07.092 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:29:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-20T18:29:07.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.093+0000 7fd592965640 1 -- 172.21.15.5:0/2515319897 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd57806be60 msgr2=0x7fd57806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:07.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.093+0000 7fd592965640 1 --2- 172.21.15.5:0/2515319897 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd57806be60 0x7fd57806e370 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd57c034000 tx=0x7fd57c02e000 comp rx=0 tx=0).stop 2024-05-20T18:29:07.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.093+0000 7fd592965640 1 -- 172.21.15.5:0/2515319897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd58c162820 msgr2=0x7fd58c0b6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:07.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.093+0000 7fd592965640 1 --2- 172.21.15.5:0/2515319897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd58c162820 0x7fd58c0b6180 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd58400c700 tx=0x7fd58400cbc0 comp rx=0 tx=0).stop 2024-05-20T18:29:07.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.093+0000 7fd592965640 1 -- 172.21.15.5:0/2515319897 shutdown_connections 2024-05-20T18:29:07.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.093+0000 7fd592965640 1 --2- 172.21.15.5:0/2515319897 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd57806be60 0x7fd57806e370 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:07.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.093+0000 7fd592965640 1 --2- 172.21.15.5:0/2515319897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd58c162820 0x7fd58c0b6180 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:07.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.093+0000 7fd592965640 1 --2- 172.21.15.5:0/2515319897 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd58c0b3e80 0x7fd58c0b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:07.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.093+0000 7fd592965640 1 --2- 172.21.15.5:0/2515319897 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd58c06d910 0x7fd58c0c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:07.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.093+0000 7fd592965640 1 -- 172.21.15.5:0/2515319897 >> 172.21.15.5:0/2515319897 conn(0x7fd58c0d4bd0 msgr2=0x7fd58c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:07.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.093+0000 7fd592965640 1 -- 172.21.15.5:0/2515319897 shutdown_connections 2024-05-20T18:29:07.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.093+0000 7fd592965640 1 -- 172.21.15.5:0/2515319897 wait complete. 2024-05-20T18:29:07.104 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-05-20T18:29:06.602906+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238780,"kb_used_data":5960,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735940,"statfs":{"total":773094113280,"available":772849602560,"internally_reserved":0,"allocated":6103040,"data_stored":3340596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.967123"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082088+0000","last_change":"2024-05-20T18:29:02.672760+0000","last_active":"2024-05-20T18:29:04.082088+0000","last_peered":"2024-05-20T18:29:04.082088+0000","last_clean":"2024-05-20T18:29:04.082088+0000","last_became_active":"2024-05-20T18:29:02.672419+0000","last_became_peered":"2024-05-20T18:29:02.672419+0000","last_unstale":"2024-05-20T18:29:04.082088+0000","last_undegraded":"2024-05-20T18:29:04.082088+0000","last_fullsized":"2024-05-20T18:29:04.082088+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:40:21.267154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082176+0000","last_change":"2024-05-20T18:29:02.685300+0000","last_active":"2024-05-20T18:29:04.082176+0000","last_peered":"2024-05-20T18:29:04.082176+0000","last_clean":"2024-05-20T18:29:04.082176+0000","last_became_active":"2024-05-20T18:29:02.684942+0000","last_became_peered":"2024-05-20T18:29:02.684942+0000","last_unstale":"2024-05-20T18:29:04.082176+0000","last_undegraded":"2024-05-20T18:29:04.082176+0000","last_fullsized":"2024-05-20T18:29:04.082176+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:52:46.953429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082466+0000","last_change":"2024-05-20T18:29:02.687050+0000","last_active":"2024-05-20T18:29:04.082466+0000","last_peered":"2024-05-20T18:29:04.082466+0000","last_clean":"2024-05-20T18:29:04.082466+0000","last_became_active":"2024-05-20T18:29:02.679862+0000","last_became_peered":"2024-05-20T18:29:02.679862+0000","last_unstale":"2024-05-20T18:29:04.082466+0000","last_undegraded":"2024-05-20T18:29:04.082466+0000","last_fullsized":"2024-05-20T18:29:04.082466+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:25:10.907111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082298+0000","last_change":"2024-05-20T18:29:02.683179+0000","last_active":"2024-05-20T18:29:04.082298+0000","last_peered":"2024-05-20T18:29:04.082298+0000","last_clean":"2024-05-20T18:29:04.082298+0000","last_became_active":"2024-05-20T18:29:02.682757+0000","last_became_peered":"2024-05-20T18:29:02.682757+0000","last_unstale":"2024-05-20T18:29:04.082298+0000","last_undegraded":"2024-05-20T18:29:04.082298+0000","last_fullsized":"2024-05-20T18:29:04.082298+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:19:37.476208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.674194+0000","last_change":"2024-05-20T18:29:02.674694+0000","last_active":"2024-05-20T18:29:03.674194+0000","last_peered":"2024-05-20T18:29:03.674194+0000","last_clean":"2024-05-20T18:29:03.674194+0000","last_became_active":"2024-05-20T18:29:02.674234+0000","last_became_peered":"2024-05-20T18:29:02.674234+0000","last_unstale":"2024-05-20T18:29:03.674194+0000","last_undegraded":"2024-05-20T18:29:03.674194+0000","last_fullsized":"2024-05-20T18:29:03.674194+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:13:38.743236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081570+0000","last_change":"2024-05-20T18:29:02.683447+0000","last_active":"2024-05-20T18:29:04.081570+0000","last_peered":"2024-05-20T18:29:04.081570+0000","last_clean":"2024-05-20T18:29:04.081570+0000","last_became_active":"2024-05-20T18:29:02.680606+0000","last_became_peered":"2024-05-20T18:29:02.680606+0000","last_unstale":"2024-05-20T18:29:04.081570+0000","last_undegraded":"2024-05-20T18:29:04.081570+0000","last_fullsized":"2024-05-20T18:29:04.081570+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:44:18.695885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081646+0000","last_change":"2024-05-20T18:29:02.673604+0000","last_active":"2024-05-20T18:29:04.081646+0000","last_peered":"2024-05-20T18:29:04.081646+0000","last_clean":"2024-05-20T18:29:04.081646+0000","last_became_active":"2024-05-20T18:29:02.673266+0000","last_became_peered":"2024-05-20T18:29:02.673266+0000","last_unstale":"2024-05-20T18:29:04.081646+0000","last_undegraded":"2024-05-20T18:29:04.081646+0000","last_fullsized":"2024-05-20T18:29:04.081646+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:24:27.542753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082368+0000","last_change":"2024-05-20T18:29:02.689510+0000","last_active":"2024-05-20T18:29:04.082368+0000","last_peered":"2024-05-20T18:29:04.082368+0000","last_clean":"2024-05-20T18:29:04.082368+0000","last_became_active":"2024-05-20T18:29:02.688696+0000","last_became_peered":"2024-05-20T18:29:02.688696+0000","last_unstale":"2024-05-20T18:29:04.082368+0000","last_undegraded":"2024-05-20T18:29:04.082368+0000","last_fullsized":"2024-05-20T18:29:04.082368+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:50:27.784534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082182+0000","last_change":"2024-05-20T18:29:02.688179+0000","last_active":"2024-05-20T18:29:04.082182+0000","last_peered":"2024-05-20T18:29:04.082182+0000","last_clean":"2024-05-20T18:29:04.082182+0000","last_became_active":"2024-05-20T18:29:02.687830+0000","last_became_peered":"2024-05-20T18:29:02.687830+0000","last_unstale":"2024-05-20T18:29:04.082182+0000","last_undegraded":"2024-05-20T18:29:04.082182+0000","last_fullsized":"2024-05-20T18:29:04.082182+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:15:29.724415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081435+0000","last_change":"2024-05-20T18:29:02.679932+0000","last_active":"2024-05-20T18:29:04.081435+0000","last_peered":"2024-05-20T18:29:04.081435+0000","last_clean":"2024-05-20T18:29:04.081435+0000","last_became_active":"2024-05-20T18:29:02.679274+0000","last_became_peered":"2024-05-20T18:29:02.679274+0000","last_unstale":"2024-05-20T18:29:04.081435+0000","last_undegraded":"2024-05-20T18:29:04.081435+0000","last_fullsized":"2024-05-20T18:29:04.081435+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:02:28.751921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.672473+0000","last_change":"2024-05-20T18:29:02.678528+0000","last_active":"2024-05-20T18:29:03.672473+0000","last_peered":"2024-05-20T18:29:03.672473+0000","last_clean":"2024-05-20T18:29:03.672473+0000","last_became_active":"2024-05-20T18:29:02.678215+0000","last_became_peered":"2024-05-20T18:29:02.678215+0000","last_unstale":"2024-05-20T18:29:03.672473+0000","last_undegraded":"2024-05-20T18:29:03.672473+0000","last_fullsized":"2024-05-20T18:29:03.672473+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:08:06.265771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082844+0000","last_change":"2024-05-20T18:29:02.680225+0000","last_active":"2024-05-20T18:29:04.082844+0000","last_peered":"2024-05-20T18:29:04.082844+0000","last_clean":"2024-05-20T18:29:04.082844+0000","last_became_active":"2024-05-20T18:29:02.679732+0000","last_became_peered":"2024-05-20T18:29:02.679732+0000","last_unstale":"2024-05-20T18:29:04.082844+0000","last_undegraded":"2024-05-20T18:29:04.082844+0000","last_fullsized":"2024-05-20T18:29:04.082844+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:46:41.204882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082301+0000","last_change":"2024-05-20T18:29:02.683189+0000","last_active":"2024-05-20T18:29:04.082301+0000","last_peered":"2024-05-20T18:29:04.082301+0000","last_clean":"2024-05-20T18:29:04.082301+0000","last_became_active":"2024-05-20T18:29:02.680163+0000","last_became_peered":"2024-05-20T18:29:02.680163+0000","last_unstale":"2024-05-20T18:29:04.082301+0000","last_undegraded":"2024-05-20T18:29:04.082301+0000","last_fullsized":"2024-05-20T18:29:04.082301+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:35:20.045268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082643+0000","last_change":"2024-05-20T18:29:02.682326+0000","last_active":"2024-05-20T18:29:04.082643+0000","last_peered":"2024-05-20T18:29:04.082643+0000","last_clean":"2024-05-20T18:29:04.082643+0000","last_became_active":"2024-05-20T18:29:02.676903+0000","last_became_peered":"2024-05-20T18:29:02.676903+0000","last_unstale":"2024-05-20T18:29:04.082643+0000","last_undegraded":"2024-05-20T18:29:04.082643+0000","last_fullsized":"2024-05-20T18:29:04.082643+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:40:09.123203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081858+0000","last_change":"2024-05-20T18:29:02.678210+0000","last_active":"2024-05-20T18:29:04.081858+0000","last_peered":"2024-05-20T18:29:04.081858+0000","last_clean":"2024-05-20T18:29:04.081858+0000","last_became_active":"2024-05-20T18:29:02.677854+0000","last_became_peered":"2024-05-20T18:29:02.677854+0000","last_unstale":"2024-05-20T18:29:04.081858+0000","last_undegraded":"2024-05-20T18:29:04.081858+0000","last_fullsized":"2024-05-20T18:29:04.081858+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:01:01.162756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081231+0000","last_change":"2024-05-20T18:29:02.675563+0000","last_active":"2024-05-20T18:29:04.081231+0000","last_peered":"2024-05-20T18:29:04.081231+0000","last_clean":"2024-05-20T18:29:04.081231+0000","last_became_active":"2024-05-20T18:29:02.675219+0000","last_became_peered":"2024-05-20T18:29:02.675219+0000","last_unstale":"2024-05-20T18:29:04.081231+0000","last_undegraded":"2024-05-20T18:29:04.081231+0000","last_fullsized":"2024-05-20T18:29:04.081231+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:30:15.509281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082959+0000","last_change":"2024-05-20T18:29:02.674778+0000","last_active":"2024-05-20T18:29:04.082959+0000","last_peered":"2024-05-20T18:29:04.082959+0000","last_clean":"2024-05-20T18:29:04.082959+0000","last_became_active":"2024-05-20T18:29:02.674475+0000","last_became_peered":"2024-05-20T18:29:02.674475+0000","last_unstale":"2024-05-20T18:29:04.082959+0000","last_undegraded":"2024-05-20T18:29:04.082959+0000","last_fullsized":"2024-05-20T18:29:04.082959+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:29:22.046224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082524+0000","last_change":"2024-05-20T18:29:02.685307+0000","last_active":"2024-05-20T18:29:04.082524+0000","last_peered":"2024-05-20T18:29:04.082524+0000","last_clean":"2024-05-20T18:29:04.082524+0000","last_became_active":"2024-05-20T18:29:02.684972+0000","last_became_peered":"2024-05-20T18:29:02.684972+0000","last_unstale":"2024-05-20T18:29:04.082524+0000","last_undegraded":"2024-05-20T18:29:04.082524+0000","last_fullsized":"2024-05-20T18:29:04.082524+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:34:33.698879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082780+0000","last_change":"2024-05-20T18:29:02.688794+0000","last_active":"2024-05-20T18:29:04.082780+0000","last_peered":"2024-05-20T18:29:04.082780+0000","last_clean":"2024-05-20T18:29:04.082780+0000","last_became_active":"2024-05-20T18:29:02.685485+0000","last_became_peered":"2024-05-20T18:29:02.685485+0000","last_unstale":"2024-05-20T18:29:04.082780+0000","last_undegraded":"2024-05-20T18:29:04.082780+0000","last_fullsized":"2024-05-20T18:29:04.082780+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:44:10.748787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.673263+0000","last_change":"2024-05-20T18:29:02.668316+0000","last_active":"2024-05-20T18:29:03.673263+0000","last_peered":"2024-05-20T18:29:03.673263+0000","last_clean":"2024-05-20T18:29:03.673263+0000","last_became_active":"2024-05-20T18:29:02.667979+0000","last_became_peered":"2024-05-20T18:29:02.667979+0000","last_unstale":"2024-05-20T18:29:03.673263+0000","last_undegraded":"2024-05-20T18:29:03.673263+0000","last_fullsized":"2024-05-20T18:29:03.673263+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:46:51.918262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.674260+0000","last_change":"2024-05-20T18:29:02.674706+0000","last_active":"2024-05-20T18:29:03.674260+0000","last_peered":"2024-05-20T18:29:03.674260+0000","last_clean":"2024-05-20T18:29:03.674260+0000","last_became_active":"2024-05-20T18:29:02.674348+0000","last_became_peered":"2024-05-20T18:29:02.674348+0000","last_unstale":"2024-05-20T18:29:03.674260+0000","last_undegraded":"2024-05-20T18:29:03.674260+0000","last_fullsized":"2024-05-20T18:29:03.674260+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:00:42.860723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081497+0000","last_change":"2024-05-20T18:29:02.688526+0000","last_active":"2024-05-20T18:29:04.081497+0000","last_peered":"2024-05-20T18:29:04.081497+0000","last_clean":"2024-05-20T18:29:04.081497+0000","last_became_active":"2024-05-20T18:29:02.688225+0000","last_became_peered":"2024-05-20T18:29:02.688225+0000","last_unstale":"2024-05-20T18:29:04.081497+0000","last_undegraded":"2024-05-20T18:29:04.081497+0000","last_fullsized":"2024-05-20T18:29:04.081497+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:42:59.031072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.672431+0000","last_change":"2024-05-20T18:29:02.673751+0000","last_active":"2024-05-20T18:29:03.672431+0000","last_peered":"2024-05-20T18:29:03.672431+0000","last_clean":"2024-05-20T18:29:03.672431+0000","last_became_active":"2024-05-20T18:29:02.673429+0000","last_became_peered":"2024-05-20T18:29:02.673429+0000","last_unstale":"2024-05-20T18:29:03.672431+0000","last_undegraded":"2024-05-20T18:29:03.672431+0000","last_fullsized":"2024-05-20T18:29:03.672431+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:41:27.321674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081157+0000","last_change":"2024-05-20T18:29:02.687375+0000","last_active":"2024-05-20T18:29:04.081157+0000","last_peered":"2024-05-20T18:29:04.081157+0000","last_clean":"2024-05-20T18:29:04.081157+0000","last_became_active":"2024-05-20T18:29:02.676374+0000","last_became_peered":"2024-05-20T18:29:02.676374+0000","last_unstale":"2024-05-20T18:29:04.081157+0000","last_undegraded":"2024-05-20T18:29:04.081157+0000","last_fullsized":"2024-05-20T18:29:04.081157+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:34:33.698879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.083114+0000","last_change":"2024-05-20T18:29:02.690248+0000","last_active":"2024-05-20T18:29:04.083114+0000","last_peered":"2024-05-20T18:29:04.083114+0000","last_clean":"2024-05-20T18:29:04.083114+0000","last_became_active":"2024-05-20T18:29:02.689929+0000","last_became_peered":"2024-05-20T18:29:02.689929+0000","last_unstale":"2024-05-20T18:29:04.083114+0000","last_undegraded":"2024-05-20T18:29:04.083114+0000","last_fullsized":"2024-05-20T18:29:04.083114+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:55:31.226636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082018+0000","last_change":"2024-05-20T18:29:02.675201+0000","last_active":"2024-05-20T18:29:04.082018+0000","last_peered":"2024-05-20T18:29:04.082018+0000","last_clean":"2024-05-20T18:29:04.082018+0000","last_became_active":"2024-05-20T18:29:02.674605+0000","last_became_peered":"2024-05-20T18:29:02.674605+0000","last_unstale":"2024-05-20T18:29:04.082018+0000","last_undegraded":"2024-05-20T18:29:04.082018+0000","last_fullsized":"2024-05-20T18:29:04.082018+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:04:06.178030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081767+0000","last_change":"2024-05-20T18:29:02.693869+0000","last_active":"2024-05-20T18:29:04.081767+0000","last_peered":"2024-05-20T18:29:04.081767+0000","last_clean":"2024-05-20T18:29:04.081767+0000","last_became_active":"2024-05-20T18:29:02.693562+0000","last_became_peered":"2024-05-20T18:29:02.693562+0000","last_unstale":"2024-05-20T18:29:04.081767+0000","last_undegraded":"2024-05-20T18:29:04.081767+0000","last_fullsized":"2024-05-20T18:29:04.081767+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:50:08.782337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.673315+0000","last_change":"2024-05-20T18:29:02.677234+0000","last_active":"2024-05-20T18:29:03.673315+0000","last_peered":"2024-05-20T18:29:03.673315+0000","last_clean":"2024-05-20T18:29:03.673315+0000","last_became_active":"2024-05-20T18:29:02.673864+0000","last_became_peered":"2024-05-20T18:29:02.673864+0000","last_unstale":"2024-05-20T18:29:03.673315+0000","last_undegraded":"2024-05-20T18:29:03.673315+0000","last_fullsized":"2024-05-20T18:29:03.673315+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:38:12.409535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081429+0000","last_change":"2024-05-20T18:29:02.688564+0000","last_active":"2024-05-20T18:29:04.081429+0000","last_peered":"2024-05-20T18:29:04.081429+0000","last_clean":"2024-05-20T18:29:04.081429+0000","last_became_active":"2024-05-20T18:29:02.688224+0000","last_became_peered":"2024-05-20T18:29:02.688224+0000","last_unstale":"2024-05-20T18:29:04.081429+0000","last_undegraded":"2024-05-20T18:29:04.081429+0000","last_fullsized":"2024-05-20T18:29:04.081429+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:48:55.261971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082431+0000","last_change":"2024-05-20T18:29:02.682079+0000","last_active":"2024-05-20T18:29:04.082431+0000","last_peered":"2024-05-20T18:29:04.082431+0000","last_clean":"2024-05-20T18:29:04.082431+0000","last_became_active":"2024-05-20T18:29:02.681545+0000","last_became_peered":"2024-05-20T18:29:02.681545+0000","last_unstale":"2024-05-20T18:29:04.082431+0000","last_undegraded":"2024-05-20T18:29:04.082431+0000","last_fullsized":"2024-05-20T18:29:04.082431+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:26:40.141288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081815+0000","last_change":"2024-05-20T18:29:02.672518+0000","last_active":"2024-05-20T18:29:04.081815+0000","last_peered":"2024-05-20T18:29:04.081815+0000","last_clean":"2024-05-20T18:29:04.081815+0000","last_became_active":"2024-05-20T18:29:02.672224+0000","last_became_peered":"2024-05-20T18:29:02.672224+0000","last_unstale":"2024-05-20T18:29:04.081815+0000","last_undegraded":"2024-05-20T18:29:04.081815+0000","last_fullsized":"2024-05-20T18:29:04.081815+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:07:06.210696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081588+0000","last_change":"2024-05-20T18:29:02.682072+0000","last_active":"2024-05-20T18:29:04.081588+0000","last_peered":"2024-05-20T18:29:04.081588+0000","last_clean":"2024-05-20T18:29:04.081588+0000","last_became_active":"2024-05-20T18:29:02.681734+0000","last_became_peered":"2024-05-20T18:29:02.681734+0000","last_unstale":"2024-05-20T18:29:04.081588+0000","last_undegraded":"2024-05-20T18:29:04.081588+0000","last_fullsized":"2024-05-20T18:29:04.081588+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:52:46.953429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081575+0000","last_change":"2024-05-20T18:29:02.677541+0000","last_active":"2024-05-20T18:29:04.081575+0000","last_peered":"2024-05-20T18:29:04.081575+0000","last_clean":"2024-05-20T18:29:04.081575+0000","last_became_active":"2024-05-20T18:29:02.677182+0000","last_became_peered":"2024-05-20T18:29:02.677182+0000","last_unstale":"2024-05-20T18:29:04.081575+0000","last_undegraded":"2024-05-20T18:29:04.081575+0000","last_fullsized":"2024-05-20T18:29:04.081575+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:40:21.267154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"50'4","reported_seq":18,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.673132+0000","last_change":"2024-05-20T18:29:01.654415+0000","last_active":"2024-05-20T18:29:03.673132+0000","last_peered":"2024-05-20T18:29:03.673132+0000","last_clean":"2024-05-20T18:29:03.673132+0000","last_became_active":"2024-05-20T18:29:01.654065+0000","last_became_peered":"2024-05-20T18:29:01.654065+0000","last_unstale":"2024-05-20T18:29:03.673132+0000","last_undegraded":"2024-05-20T18:29:03.673132+0000","last_fullsized":"2024-05-20T18:29:03.673132+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:43:56.853477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082429+0000","last_change":"2024-05-20T18:29:02.679961+0000","last_active":"2024-05-20T18:29:04.082429+0000","last_peered":"2024-05-20T18:29:04.082429+0000","last_clean":"2024-05-20T18:29:04.082429+0000","last_became_active":"2024-05-20T18:29:02.679011+0000","last_became_peered":"2024-05-20T18:29:02.679011+0000","last_unstale":"2024-05-20T18:29:04.082429+0000","last_undegraded":"2024-05-20T18:29:04.082429+0000","last_fullsized":"2024-05-20T18:29:04.082429+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:48:55.261971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081509+0000","last_change":"2024-05-20T18:29:01.658924+0000","last_active":"2024-05-20T18:29:04.081509+0000","last_peered":"2024-05-20T18:29:04.081509+0000","last_clean":"2024-05-20T18:29:04.081509+0000","last_became_active":"2024-05-20T18:29:01.658584+0000","last_became_peered":"2024-05-20T18:29:01.658584+0000","last_unstale":"2024-05-20T18:29:04.081509+0000","last_undegraded":"2024-05-20T18:29:04.081509+0000","last_fullsized":"2024-05-20T18:29:04.081509+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:33.142624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.673719+0000","last_change":"2024-05-20T18:29:02.679328+0000","last_active":"2024-05-20T18:29:03.673719+0000","last_peered":"2024-05-20T18:29:03.673719+0000","last_clean":"2024-05-20T18:29:03.673719+0000","last_became_active":"2024-05-20T18:29:02.678993+0000","last_became_peered":"2024-05-20T18:29:02.678993+0000","last_unstale":"2024-05-20T18:29:03.673719+0000","last_undegraded":"2024-05-20T18:29:03.673719+0000","last_fullsized":"2024-05-20T18:29:03.673719+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:30:15.509281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081835+0000","last_change":"2024-05-20T18:29:01.657834+0000","last_active":"2024-05-20T18:29:04.081835+0000","last_peered":"2024-05-20T18:29:04.081835+0000","last_clean":"2024-05-20T18:29:04.081835+0000","last_became_active":"2024-05-20T18:29:01.657409+0000","last_became_peered":"2024-05-20T18:29:01.657409+0000","last_unstale":"2024-05-20T18:29:04.081835+0000","last_undegraded":"2024-05-20T18:29:04.081835+0000","last_fullsized":"2024-05-20T18:29:04.081835+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:25:22.102705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081775+0000","last_change":"2024-05-20T18:29:02.675977+0000","last_active":"2024-05-20T18:29:04.081775+0000","last_peered":"2024-05-20T18:29:04.081775+0000","last_clean":"2024-05-20T18:29:04.081775+0000","last_became_active":"2024-05-20T18:29:02.675528+0000","last_became_peered":"2024-05-20T18:29:02.675528+0000","last_unstale":"2024-05-20T18:29:04.081775+0000","last_undegraded":"2024-05-20T18:29:04.081775+0000","last_fullsized":"2024-05-20T18:29:04.081775+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:42:29.512485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081971+0000","last_change":"2024-05-20T18:29:01.661584+0000","last_active":"2024-05-20T18:29:04.081971+0000","last_peered":"2024-05-20T18:29:04.081971+0000","last_clean":"2024-05-20T18:29:04.081971+0000","last_became_active":"2024-05-20T18:29:01.660902+0000","last_became_peered":"2024-05-20T18:29:01.660902+0000","last_unstale":"2024-05-20T18:29:04.081971+0000","last_undegraded":"2024-05-20T18:29:04.081971+0000","last_fullsized":"2024-05-20T18:29:04.081971+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:00:41.856150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.673579+0000","last_change":"2024-05-20T18:29:02.683305+0000","last_active":"2024-05-20T18:29:03.673579+0000","last_peered":"2024-05-20T18:29:03.673579+0000","last_clean":"2024-05-20T18:29:03.673579+0000","last_became_active":"2024-05-20T18:29:02.682926+0000","last_became_peered":"2024-05-20T18:29:02.682926+0000","last_unstale":"2024-05-20T18:29:03.673579+0000","last_undegraded":"2024-05-20T18:29:03.673579+0000","last_fullsized":"2024-05-20T18:29:03.673579+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:38:12.409535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081488+0000","last_change":"2024-05-20T18:29:01.654206+0000","last_active":"2024-05-20T18:29:04.081488+0000","last_peered":"2024-05-20T18:29:04.081488+0000","last_clean":"2024-05-20T18:29:04.081488+0000","last_became_active":"2024-05-20T18:29:01.653881+0000","last_became_peered":"2024-05-20T18:29:01.653881+0000","last_unstale":"2024-05-20T18:29:04.081488+0000","last_undegraded":"2024-05-20T18:29:04.081488+0000","last_fullsized":"2024-05-20T18:29:04.081488+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:18:13.553828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082384+0000","last_change":"2024-05-20T18:29:02.681641+0000","last_active":"2024-05-20T18:29:04.082384+0000","last_peered":"2024-05-20T18:29:04.082384+0000","last_clean":"2024-05-20T18:29:04.082384+0000","last_became_active":"2024-05-20T18:29:02.681080+0000","last_became_peered":"2024-05-20T18:29:02.681080+0000","last_unstale":"2024-05-20T18:29:04.082384+0000","last_undegraded":"2024-05-20T18:29:04.082384+0000","last_fullsized":"2024-05-20T18:29:04.082384+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:28:13.588490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.672945+0000","last_change":"2024-05-20T18:29:01.654251+0000","last_active":"2024-05-20T18:29:03.672945+0000","last_peered":"2024-05-20T18:29:03.672945+0000","last_clean":"2024-05-20T18:29:03.672945+0000","last_became_active":"2024-05-20T18:29:01.653907+0000","last_became_peered":"2024-05-20T18:29:01.653907+0000","last_unstale":"2024-05-20T18:29:03.672945+0000","last_undegraded":"2024-05-20T18:29:03.672945+0000","last_fullsized":"2024-05-20T18:29:03.672945+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:25:22.102705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082225+0000","last_change":"2024-05-20T18:29:02.674929+0000","last_active":"2024-05-20T18:29:04.082225+0000","last_peered":"2024-05-20T18:29:04.082225+0000","last_clean":"2024-05-20T18:29:04.082225+0000","last_became_active":"2024-05-20T18:29:02.674257+0000","last_became_peered":"2024-05-20T18:29:02.674257+0000","last_unstale":"2024-05-20T18:29:04.082225+0000","last_undegraded":"2024-05-20T18:29:04.082225+0000","last_fullsized":"2024-05-20T18:29:04.082225+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:26:42.466485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.673794+0000","last_change":"2024-05-20T18:29:01.651981+0000","last_active":"2024-05-20T18:29:03.673794+0000","last_peered":"2024-05-20T18:29:03.673794+0000","last_clean":"2024-05-20T18:29:03.673794+0000","last_became_active":"2024-05-20T18:29:01.651648+0000","last_became_peered":"2024-05-20T18:29:01.651648+0000","last_unstale":"2024-05-20T18:29:03.673794+0000","last_undegraded":"2024-05-20T18:29:03.673794+0000","last_fullsized":"2024-05-20T18:29:03.673794+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:11:29.346230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081856+0000","last_change":"2024-05-20T18:29:02.677636+0000","last_active":"2024-05-20T18:29:04.081856+0000","last_peered":"2024-05-20T18:29:04.081856+0000","last_clean":"2024-05-20T18:29:04.081856+0000","last_became_active":"2024-05-20T18:29:02.677237+0000","last_became_peered":"2024-05-20T18:29:02.677237+0000","last_unstale":"2024-05-20T18:29:04.081856+0000","last_undegraded":"2024-05-20T18:29:04.081856+0000","last_fullsized":"2024-05-20T18:29:04.081856+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:32.845836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081106+0000","last_change":"2024-05-20T18:29:01.658231+0000","last_active":"2024-05-20T18:29:04.081106+0000","last_peered":"2024-05-20T18:29:04.081106+0000","last_clean":"2024-05-20T18:29:04.081106+0000","last_became_active":"2024-05-20T18:29:01.657925+0000","last_became_peered":"2024-05-20T18:29:01.657925+0000","last_unstale":"2024-05-20T18:29:04.081106+0000","last_undegraded":"2024-05-20T18:29:04.081106+0000","last_fullsized":"2024-05-20T18:29:04.081106+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:13:37.738664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.672999+0000","last_change":"2024-05-20T18:29:02.687337+0000","last_active":"2024-05-20T18:29:03.672999+0000","last_peered":"2024-05-20T18:29:03.672999+0000","last_clean":"2024-05-20T18:29:03.672999+0000","last_became_active":"2024-05-20T18:29:02.686984+0000","last_became_peered":"2024-05-20T18:29:02.686984+0000","last_unstale":"2024-05-20T18:29:03.672999+0000","last_undegraded":"2024-05-20T18:29:03.672999+0000","last_fullsized":"2024-05-20T18:29:03.672999+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:46:51.918262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081932+0000","last_change":"2024-05-20T18:29:01.659390+0000","last_active":"2024-05-20T18:29:04.081932+0000","last_peered":"2024-05-20T18:29:04.081932+0000","last_clean":"2024-05-20T18:29:04.081932+0000","last_became_active":"2024-05-20T18:29:01.659059+0000","last_became_peered":"2024-05-20T18:29:01.659059+0000","last_unstale":"2024-05-20T18:29:04.081932+0000","last_undegraded":"2024-05-20T18:29:04.081932+0000","last_fullsized":"2024-05-20T18:29:04.081932+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:38:11.404962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082448+0000","last_change":"2024-05-20T18:29:02.668562+0000","last_active":"2024-05-20T18:29:04.082448+0000","last_peered":"2024-05-20T18:29:04.082448+0000","last_clean":"2024-05-20T18:29:04.082448+0000","last_became_active":"2024-05-20T18:29:02.668168+0000","last_became_peered":"2024-05-20T18:29:02.668168+0000","last_unstale":"2024-05-20T18:29:04.082448+0000","last_undegraded":"2024-05-20T18:29:04.082448+0000","last_fullsized":"2024-05-20T18:29:04.082448+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:24:27.542753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.673281+0000","last_change":"2024-05-20T18:29:01.653200+0000","last_active":"2024-05-20T18:29:03.673281+0000","last_peered":"2024-05-20T18:29:03.673281+0000","last_clean":"2024-05-20T18:29:03.673281+0000","last_became_active":"2024-05-20T18:29:01.652885+0000","last_became_peered":"2024-05-20T18:29:01.652885+0000","last_unstale":"2024-05-20T18:29:03.673281+0000","last_undegraded":"2024-05-20T18:29:03.673281+0000","last_fullsized":"2024-05-20T18:29:03.673281+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:17:01.882291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082718+0000","last_change":"2024-05-20T18:29:02.686754+0000","last_active":"2024-05-20T18:29:04.082718+0000","last_peered":"2024-05-20T18:29:04.082718+0000","last_clean":"2024-05-20T18:29:04.082718+0000","last_became_active":"2024-05-20T18:29:02.686348+0000","last_became_peered":"2024-05-20T18:29:02.686348+0000","last_unstale":"2024-05-20T18:29:04.082718+0000","last_undegraded":"2024-05-20T18:29:04.082718+0000","last_fullsized":"2024-05-20T18:29:04.082718+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:17:36.464439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"50'3","reported_seq":17,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082492+0000","last_change":"2024-05-20T18:29:01.663167+0000","last_active":"2024-05-20T18:29:04.082492+0000","last_peered":"2024-05-20T18:29:04.082492+0000","last_clean":"2024-05-20T18:29:04.082492+0000","last_became_active":"2024-05-20T18:29:01.662800+0000","last_became_peered":"2024-05-20T18:29:01.662800+0000","last_unstale":"2024-05-20T18:29:04.082492+0000","last_undegraded":"2024-05-20T18:29:04.082492+0000","last_fullsized":"2024-05-20T18:29:04.082492+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:46:50.913690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.674266+0000","last_change":"2024-05-20T18:29:02.672132+0000","last_active":"2024-05-20T18:29:03.674266+0000","last_peered":"2024-05-20T18:29:03.674266+0000","last_clean":"2024-05-20T18:29:03.674266+0000","last_became_active":"2024-05-20T18:29:02.671790+0000","last_became_peered":"2024-05-20T18:29:02.671790+0000","last_unstale":"2024-05-20T18:29:03.674266+0000","last_undegraded":"2024-05-20T18:29:03.674266+0000","last_fullsized":"2024-05-20T18:29:03.674266+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:30:15.509281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"50'6","reported_seq":20,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082901+0000","last_change":"2024-05-20T18:29:01.661776+0000","last_active":"2024-05-20T18:29:04.082901+0000","last_peered":"2024-05-20T18:29:04.082901+0000","last_clean":"2024-05-20T18:29:04.082901+0000","last_became_active":"2024-05-20T18:29:01.661386+0000","last_became_peered":"2024-05-20T18:29:01.661386+0000","last_unstale":"2024-05-20T18:29:04.082901+0000","last_undegraded":"2024-05-20T18:29:04.082901+0000","last_fullsized":"2024-05-20T18:29:04.082901+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:25:09.902538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081856+0000","last_change":"2024-05-20T18:29:02.678560+0000","last_active":"2024-05-20T18:29:04.081856+0000","last_peered":"2024-05-20T18:29:04.081856+0000","last_clean":"2024-05-20T18:29:04.081856+0000","last_became_active":"2024-05-20T18:29:02.678170+0000","last_became_peered":"2024-05-20T18:29:02.678170+0000","last_unstale":"2024-05-20T18:29:04.081856+0000","last_undegraded":"2024-05-20T18:29:04.081856+0000","last_fullsized":"2024-05-20T18:29:04.081856+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:19:37.476208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081607+0000","last_change":"2024-05-20T18:29:01.656783+0000","last_active":"2024-05-20T18:29:04.081607+0000","last_peered":"2024-05-20T18:29:04.081607+0000","last_clean":"2024-05-20T18:29:04.081607+0000","last_became_active":"2024-05-20T18:29:01.656306+0000","last_became_peered":"2024-05-20T18:29:01.656306+0000","last_unstale":"2024-05-20T18:29:04.081607+0000","last_undegraded":"2024-05-20T18:29:04.081607+0000","last_fullsized":"2024-05-20T18:29:04.081607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:46:50.913690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082743+0000","last_change":"2024-05-20T18:29:02.685760+0000","last_active":"2024-05-20T18:29:04.082743+0000","last_peered":"2024-05-20T18:29:04.082743+0000","last_clean":"2024-05-20T18:29:04.082743+0000","last_became_active":"2024-05-20T18:29:02.685135+0000","last_became_peered":"2024-05-20T18:29:02.685135+0000","last_unstale":"2024-05-20T18:29:04.082743+0000","last_undegraded":"2024-05-20T18:29:04.082743+0000","last_fullsized":"2024-05-20T18:29:04.082743+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:46:20.452501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.673891+0000","last_change":"2024-05-20T18:29:01.659912+0000","last_active":"2024-05-20T18:29:03.673891+0000","last_peered":"2024-05-20T18:29:03.673891+0000","last_clean":"2024-05-20T18:29:03.673891+0000","last_became_active":"2024-05-20T18:29:01.658865+0000","last_became_peered":"2024-05-20T18:29:01.658865+0000","last_unstale":"2024-05-20T18:29:03.673891+0000","last_undegraded":"2024-05-20T18:29:03.673891+0000","last_fullsized":"2024-05-20T18:29:03.673891+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:33.142624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082264+0000","last_change":"2024-05-20T18:29:02.679515+0000","last_active":"2024-05-20T18:29:04.082264+0000","last_peered":"2024-05-20T18:29:04.082264+0000","last_clean":"2024-05-20T18:29:04.082264+0000","last_became_active":"2024-05-20T18:29:02.679183+0000","last_became_peered":"2024-05-20T18:29:02.679183+0000","last_unstale":"2024-05-20T18:29:04.082264+0000","last_undegraded":"2024-05-20T18:29:04.082264+0000","last_fullsized":"2024-05-20T18:29:04.082264+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:15:37.388487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082408+0000","last_change":"2024-05-20T18:29:01.659062+0000","last_active":"2024-05-20T18:29:04.082408+0000","last_peered":"2024-05-20T18:29:04.082408+0000","last_clean":"2024-05-20T18:29:04.082408+0000","last_became_active":"2024-05-20T18:29:01.658658+0000","last_became_peered":"2024-05-20T18:29:01.658658+0000","last_unstale":"2024-05-20T18:29:04.082408+0000","last_undegraded":"2024-05-20T18:29:04.082408+0000","last_fullsized":"2024-05-20T18:29:04.082408+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:18:13.553828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.083022+0000","last_change":"2024-05-20T18:29:02.682398+0000","last_active":"2024-05-20T18:29:04.083022+0000","last_peered":"2024-05-20T18:29:04.083022+0000","last_clean":"2024-05-20T18:29:04.083022+0000","last_became_active":"2024-05-20T18:29:02.682022+0000","last_became_peered":"2024-05-20T18:29:02.682022+0000","last_unstale":"2024-05-20T18:29:04.083022+0000","last_undegraded":"2024-05-20T18:29:04.083022+0000","last_fullsized":"2024-05-20T18:29:04.083022+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:35:26.747322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"50'8","reported_seq":22,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082437+0000","last_change":"2024-05-20T18:29:01.663928+0000","last_active":"2024-05-20T18:29:04.082437+0000","last_peered":"2024-05-20T18:29:04.082437+0000","last_clean":"2024-05-20T18:29:04.082437+0000","last_became_active":"2024-05-20T18:29:01.663570+0000","last_became_peered":"2024-05-20T18:29:01.663570+0000","last_unstale":"2024-05-20T18:29:04.082437+0000","last_undegraded":"2024-05-20T18:29:04.082437+0000","last_fullsized":"2024-05-20T18:29:04.082437+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:20:48.010059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082509+0000","last_change":"2024-05-20T18:29:02.678528+0000","last_active":"2024-05-20T18:29:04.082509+0000","last_peered":"2024-05-20T18:29:04.082509+0000","last_clean":"2024-05-20T18:29:04.082509+0000","last_became_active":"2024-05-20T18:29:02.676260+0000","last_became_peered":"2024-05-20T18:29:02.676260+0000","last_unstale":"2024-05-20T18:29:04.082509+0000","last_undegraded":"2024-05-20T18:29:04.082509+0000","last_fullsized":"2024-05-20T18:29:04.082509+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:06:13.229296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.083098+0000","last_change":"2024-05-20T18:29:01.659497+0000","last_active":"2024-05-20T18:29:04.083098+0000","last_peered":"2024-05-20T18:29:04.083098+0000","last_clean":"2024-05-20T18:29:04.083098+0000","last_became_active":"2024-05-20T18:29:01.659189+0000","last_became_peered":"2024-05-20T18:29:01.659189+0000","last_unstale":"2024-05-20T18:29:04.083098+0000","last_undegraded":"2024-05-20T18:29:04.083098+0000","last_fullsized":"2024-05-20T18:29:04.083098+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:31:10.334541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081235+0000","last_change":"2024-05-20T18:29:02.693178+0000","last_active":"2024-05-20T18:29:04.081235+0000","last_peered":"2024-05-20T18:29:04.081235+0000","last_clean":"2024-05-20T18:29:04.081235+0000","last_became_active":"2024-05-20T18:29:02.692824+0000","last_became_peered":"2024-05-20T18:29:02.692824+0000","last_unstale":"2024-05-20T18:29:04.081235+0000","last_undegraded":"2024-05-20T18:29:04.081235+0000","last_fullsized":"2024-05-20T18:29:04.081235+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:00:42.860723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.672481+0000","last_change":"2024-05-20T18:29:01.659031+0000","last_active":"2024-05-20T18:29:03.672481+0000","last_peered":"2024-05-20T18:29:03.672481+0000","last_clean":"2024-05-20T18:29:03.672481+0000","last_became_active":"2024-05-20T18:29:01.658255+0000","last_became_peered":"2024-05-20T18:29:01.658255+0000","last_unstale":"2024-05-20T18:29:03.672481+0000","last_undegraded":"2024-05-20T18:29:03.672481+0000","last_fullsized":"2024-05-20T18:29:03.672481+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:46:43.782231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.673001+0000","last_change":"2024-05-20T18:29:02.683982+0000","last_active":"2024-05-20T18:29:03.673001+0000","last_peered":"2024-05-20T18:29:03.673001+0000","last_clean":"2024-05-20T18:29:03.673001+0000","last_became_active":"2024-05-20T18:29:02.683615+0000","last_became_peered":"2024-05-20T18:29:02.683615+0000","last_unstale":"2024-05-20T18:29:03.673001+0000","last_undegraded":"2024-05-20T18:29:03.673001+0000","last_fullsized":"2024-05-20T18:29:03.673001+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:18:14.558400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.673717+0000","last_change":"2024-05-20T18:29:01.655986+0000","last_active":"2024-05-20T18:29:03.673717+0000","last_peered":"2024-05-20T18:29:03.673717+0000","last_clean":"2024-05-20T18:29:03.673717+0000","last_became_active":"2024-05-20T18:29:01.655641+0000","last_became_peered":"2024-05-20T18:29:01.655641+0000","last_unstale":"2024-05-20T18:29:03.673717+0000","last_undegraded":"2024-05-20T18:29:03.673717+0000","last_fullsized":"2024-05-20T18:29:03.673717+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:26:48.032272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081744+0000","last_change":"2024-05-20T18:29:02.683354+0000","last_active":"2024-05-20T18:29:04.081744+0000","last_peered":"2024-05-20T18:29:04.081744+0000","last_clean":"2024-05-20T18:29:04.081744+0000","last_became_active":"2024-05-20T18:29:02.680578+0000","last_became_peered":"2024-05-20T18:29:02.680578+0000","last_unstale":"2024-05-20T18:29:04.081744+0000","last_undegraded":"2024-05-20T18:29:04.081744+0000","last_fullsized":"2024-05-20T18:29:04.081744+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:56:58.199488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.673340+0000","last_change":"2024-05-20T18:29:01.661629+0000","last_active":"2024-05-20T18:29:03.673340+0000","last_peered":"2024-05-20T18:29:03.673340+0000","last_clean":"2024-05-20T18:29:03.673340+0000","last_became_active":"2024-05-20T18:29:01.661268+0000","last_became_peered":"2024-05-20T18:29:01.661268+0000","last_unstale":"2024-05-20T18:29:03.673340+0000","last_undegraded":"2024-05-20T18:29:03.673340+0000","last_fullsized":"2024-05-20T18:29:03.673340+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:25:22.102705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082072+0000","last_change":"2024-05-20T18:29:02.670089+0000","last_active":"2024-05-20T18:29:04.082072+0000","last_peered":"2024-05-20T18:29:04.082072+0000","last_clean":"2024-05-20T18:29:04.082072+0000","last_became_active":"2024-05-20T18:29:02.669740+0000","last_became_peered":"2024-05-20T18:29:02.669740+0000","last_unstale":"2024-05-20T18:29:04.082072+0000","last_undegraded":"2024-05-20T18:29:04.082072+0000","last_fullsized":"2024-05-20T18:29:04.082072+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:26:40.141288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081811+0000","last_change":"2024-05-20T18:29:01.664131+0000","last_active":"2024-05-20T18:29:04.081811+0000","last_peered":"2024-05-20T18:29:04.081811+0000","last_clean":"2024-05-20T18:29:04.081811+0000","last_became_active":"2024-05-20T18:29:01.663780+0000","last_became_peered":"2024-05-20T18:29:01.663780+0000","last_unstale":"2024-05-20T18:29:04.081811+0000","last_undegraded":"2024-05-20T18:29:04.081811+0000","last_fullsized":"2024-05-20T18:29:04.081811+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:11:29.346230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.673639+0000","last_change":"2024-05-20T18:29:02.661856+0000","last_active":"2024-05-20T18:29:03.673639+0000","last_peered":"2024-05-20T18:29:03.673639+0000","last_clean":"2024-05-20T18:29:03.673639+0000","last_became_active":"2024-05-20T18:29:02.661520+0000","last_became_peered":"2024-05-20T18:29:02.661520+0000","last_unstale":"2024-05-20T18:29:03.673639+0000","last_undegraded":"2024-05-20T18:29:03.673639+0000","last_fullsized":"2024-05-20T18:29:03.673639+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:13:38.743236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081568+0000","last_change":"2024-05-20T18:29:01.665733+0000","last_active":"2024-05-20T18:29:04.081568+0000","last_peered":"2024-05-20T18:29:04.081568+0000","last_clean":"2024-05-20T18:29:04.081568+0000","last_became_active":"2024-05-20T18:29:01.665385+0000","last_became_peered":"2024-05-20T18:29:01.665385+0000","last_unstale":"2024-05-20T18:29:04.081568+0000","last_undegraded":"2024-05-20T18:29:04.081568+0000","last_fullsized":"2024-05-20T18:29:04.081568+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:07:05.206123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082263+0000","last_change":"2024-05-20T18:29:02.688528+0000","last_active":"2024-05-20T18:29:04.082263+0000","last_peered":"2024-05-20T18:29:04.082263+0000","last_clean":"2024-05-20T18:29:04.082263+0000","last_became_active":"2024-05-20T18:29:02.688185+0000","last_became_peered":"2024-05-20T18:29:02.688185+0000","last_unstale":"2024-05-20T18:29:04.082263+0000","last_undegraded":"2024-05-20T18:29:04.082263+0000","last_fullsized":"2024-05-20T18:29:04.082263+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:55:07.462160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081875+0000","last_change":"2024-05-20T18:29:01.655080+0000","last_active":"2024-05-20T18:29:04.081875+0000","last_peered":"2024-05-20T18:29:04.081875+0000","last_clean":"2024-05-20T18:29:04.081875+0000","last_became_active":"2024-05-20T18:29:01.654745+0000","last_became_peered":"2024-05-20T18:29:01.654745+0000","last_unstale":"2024-05-20T18:29:04.081875+0000","last_undegraded":"2024-05-20T18:29:04.081875+0000","last_fullsized":"2024-05-20T18:29:04.081875+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:14:02.291830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082264+0000","last_change":"2024-05-20T18:29:02.689391+0000","last_active":"2024-05-20T18:29:04.082264+0000","last_peered":"2024-05-20T18:29:04.082264+0000","last_clean":"2024-05-20T18:29:04.082264+0000","last_became_active":"2024-05-20T18:29:02.687198+0000","last_became_peered":"2024-05-20T18:29:02.687198+0000","last_unstale":"2024-05-20T18:29:04.082264+0000","last_undegraded":"2024-05-20T18:29:04.082264+0000","last_fullsized":"2024-05-20T18:29:04.082264+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:32.845836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.672053+0000","last_change":"2024-05-20T18:29:01.654572+0000","last_active":"2024-05-20T18:29:03.672053+0000","last_peered":"2024-05-20T18:29:03.672053+0000","last_clean":"2024-05-20T18:29:03.672053+0000","last_became_active":"2024-05-20T18:29:01.654242+0000","last_became_peered":"2024-05-20T18:29:01.654242+0000","last_unstale":"2024-05-20T18:29:03.672053+0000","last_undegraded":"2024-05-20T18:29:03.672053+0000","last_fullsized":"2024-05-20T18:29:03.672053+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:12:23.698473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082361+0000","last_change":"2024-05-20T18:29:02.682283+0000","last_active":"2024-05-20T18:29:04.082361+0000","last_peered":"2024-05-20T18:29:04.082361+0000","last_clean":"2024-05-20T18:29:04.082361+0000","last_became_active":"2024-05-20T18:29:02.681928+0000","last_became_peered":"2024-05-20T18:29:02.681928+0000","last_unstale":"2024-05-20T18:29:04.082361+0000","last_undegraded":"2024-05-20T18:29:04.082361+0000","last_fullsized":"2024-05-20T18:29:04.082361+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:34:33.698879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.673302+0000","last_change":"2024-05-20T18:29:01.659515+0000","last_active":"2024-05-20T18:29:03.673302+0000","last_peered":"2024-05-20T18:29:03.673302+0000","last_clean":"2024-05-20T18:29:03.673302+0000","last_became_active":"2024-05-20T18:29:01.659066+0000","last_became_peered":"2024-05-20T18:29:01.659066+0000","last_unstale":"2024-05-20T18:29:03.673302+0000","last_undegraded":"2024-05-20T18:29:03.673302+0000","last_fullsized":"2024-05-20T18:29:03.673302+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:49:42.304308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081797+0000","last_change":"2024-05-20T18:29:02.677236+0000","last_active":"2024-05-20T18:29:04.081797+0000","last_peered":"2024-05-20T18:29:04.081797+0000","last_clean":"2024-05-20T18:29:04.081797+0000","last_became_active":"2024-05-20T18:29:02.676006+0000","last_became_peered":"2024-05-20T18:29:02.676006+0000","last_unstale":"2024-05-20T18:29:04.081797+0000","last_undegraded":"2024-05-20T18:29:04.081797+0000","last_fullsized":"2024-05-20T18:29:04.081797+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:02:01.847309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082493+0000","last_change":"2024-05-20T18:29:01.658454+0000","last_active":"2024-05-20T18:29:04.082493+0000","last_peered":"2024-05-20T18:29:04.082493+0000","last_clean":"2024-05-20T18:29:04.082493+0000","last_became_active":"2024-05-20T18:29:01.658083+0000","last_became_peered":"2024-05-20T18:29:01.658083+0000","last_unstale":"2024-05-20T18:29:04.082493+0000","last_undegraded":"2024-05-20T18:29:04.082493+0000","last_fullsized":"2024-05-20T18:29:04.082493+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:13:37.738664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081960+0000","last_change":"2024-05-20T18:29:02.675916+0000","last_active":"2024-05-20T18:29:04.081960+0000","last_peered":"2024-05-20T18:29:04.081960+0000","last_clean":"2024-05-20T18:29:04.081960+0000","last_became_active":"2024-05-20T18:29:02.675603+0000","last_became_peered":"2024-05-20T18:29:02.675603+0000","last_unstale":"2024-05-20T18:29:04.081960+0000","last_undegraded":"2024-05-20T18:29:04.081960+0000","last_fullsized":"2024-05-20T18:29:04.081960+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:15:22.834731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081967+0000","last_change":"2024-05-20T18:29:01.647606+0000","last_active":"2024-05-20T18:29:04.081967+0000","last_peered":"2024-05-20T18:29:04.081967+0000","last_clean":"2024-05-20T18:29:04.081967+0000","last_became_active":"2024-05-20T18:29:01.647265+0000","last_became_peered":"2024-05-20T18:29:01.647265+0000","last_unstale":"2024-05-20T18:29:04.081967+0000","last_undegraded":"2024-05-20T18:29:04.081967+0000","last_fullsized":"2024-05-20T18:29:04.081967+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:21:26.188155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081825+0000","last_change":"2024-05-20T18:29:02.691656+0000","last_active":"2024-05-20T18:29:04.081825+0000","last_peered":"2024-05-20T18:29:04.081825+0000","last_clean":"2024-05-20T18:29:04.081825+0000","last_became_active":"2024-05-20T18:29:02.690648+0000","last_became_peered":"2024-05-20T18:29:02.690648+0000","last_unstale":"2024-05-20T18:29:04.081825+0000","last_undegraded":"2024-05-20T18:29:04.081825+0000","last_fullsized":"2024-05-20T18:29:04.081825+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:31:23.977414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082027+0000","last_change":"2024-05-20T18:29:01.663028+0000","last_active":"2024-05-20T18:29:04.082027+0000","last_peered":"2024-05-20T18:29:04.082027+0000","last_clean":"2024-05-20T18:29:04.082027+0000","last_became_active":"2024-05-20T18:29:01.662663+0000","last_became_peered":"2024-05-20T18:29:01.662663+0000","last_unstale":"2024-05-20T18:29:04.082027+0000","last_undegraded":"2024-05-20T18:29:04.082027+0000","last_fullsized":"2024-05-20T18:29:04.082027+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:55:06.457587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081908+0000","last_change":"2024-05-20T18:29:02.678613+0000","last_active":"2024-05-20T18:29:04.081908+0000","last_peered":"2024-05-20T18:29:04.081908+0000","last_clean":"2024-05-20T18:29:04.081908+0000","last_became_active":"2024-05-20T18:29:02.678245+0000","last_became_peered":"2024-05-20T18:29:02.678245+0000","last_unstale":"2024-05-20T18:29:04.081908+0000","last_undegraded":"2024-05-20T18:29:04.081908+0000","last_fullsized":"2024-05-20T18:29:04.081908+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:28:13.588490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.081688+0000","last_change":"2024-05-20T18:29:01.658295+0000","last_active":"2024-05-20T18:29:04.081688+0000","last_peered":"2024-05-20T18:29:04.081688+0000","last_clean":"2024-05-20T18:29:04.081688+0000","last_became_active":"2024-05-20T18:29:01.657954+0000","last_became_peered":"2024-05-20T18:29:01.657954+0000","last_unstale":"2024-05-20T18:29:04.081688+0000","last_undegraded":"2024-05-20T18:29:04.081688+0000","last_fullsized":"2024-05-20T18:29:04.081688+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:31:22.972841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082100+0000","last_change":"2024-05-20T18:29:02.683947+0000","last_active":"2024-05-20T18:29:04.082100+0000","last_peered":"2024-05-20T18:29:04.082100+0000","last_clean":"2024-05-20T18:29:04.082100+0000","last_became_active":"2024-05-20T18:29:02.679378+0000","last_became_peered":"2024-05-20T18:29:02.679378+0000","last_unstale":"2024-05-20T18:29:04.082100+0000","last_undegraded":"2024-05-20T18:29:04.082100+0000","last_fullsized":"2024-05-20T18:29:04.082100+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:20:49.014632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.672829+0000","last_change":"2024-05-20T18:29:01.658476+0000","last_active":"2024-05-20T18:29:03.672829+0000","last_peered":"2024-05-20T18:29:03.672829+0000","last_clean":"2024-05-20T18:29:03.672829+0000","last_became_active":"2024-05-20T18:29:01.658139+0000","last_became_peered":"2024-05-20T18:29:01.658139+0000","last_unstale":"2024-05-20T18:29:03.672829+0000","last_undegraded":"2024-05-20T18:29:03.672829+0000","last_fullsized":"2024-05-20T18:29:03.672829+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:11:29.346230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082349+0000","last_change":"2024-05-20T18:29:02.677001+0000","last_active":"2024-05-20T18:29:04.082349+0000","last_peered":"2024-05-20T18:29:04.082349+0000","last_clean":"2024-05-20T18:29:04.082349+0000","last_became_active":"2024-05-20T18:29:02.676626+0000","last_became_peered":"2024-05-20T18:29:02.676626+0000","last_unstale":"2024-05-20T18:29:04.082349+0000","last_undegraded":"2024-05-20T18:29:04.082349+0000","last_fullsized":"2024-05-20T18:29:04.082349+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:47:29.251792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082042+0000","last_change":"2024-05-20T18:29:01.662968+0000","last_active":"2024-05-20T18:29:04.082042+0000","last_peered":"2024-05-20T18:29:04.082042+0000","last_clean":"2024-05-20T18:29:04.082042+0000","last_became_active":"2024-05-20T18:29:01.662519+0000","last_became_peered":"2024-05-20T18:29:01.662519+0000","last_unstale":"2024-05-20T18:29:04.082042+0000","last_undegraded":"2024-05-20T18:29:04.082042+0000","last_fullsized":"2024-05-20T18:29:04.082042+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:02:27.747348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:03.672999+0000","last_change":"2024-05-20T18:29:02.673465+0000","last_active":"2024-05-20T18:29:03.672999+0000","last_peered":"2024-05-20T18:29:03.672999+0000","last_clean":"2024-05-20T18:29:03.672999+0000","last_became_active":"2024-05-20T18:29:02.673113+0000","last_became_peered":"2024-05-20T18:29:02.673113+0000","last_unstale":"2024-05-20T18:29:03.672999+0000","last_undegraded":"2024-05-20T18:29:03.672999+0000","last_fullsized":"2024-05-20T18:29:03.672999+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:01.635379+0000","last_clean_scrub_stamp":"2024-05-20T18:29:01.635379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:18:14.558400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"51'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082213+0000","last_change":"2024-05-20T18:29:01.656841+0000","last_active":"2024-05-20T18:29:04.082213+0000","last_peered":"2024-05-20T18:29:04.082213+0000","last_clean":"2024-05-20T18:29:04.082213+0000","last_became_active":"2024-05-20T18:29:01.655870+0000","last_became_peered":"2024-05-20T18:29:01.655870+0000","last_unstale":"2024-05-20T18:29:04.082213+0000","last_undegraded":"2024-05-20T18:29:04.082213+0000","last_fullsized":"2024-05-20T18:29:04.082213+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:00.630807+0000","last_clean_scrub_stamp":"2024-05-20T18:29:00.630807+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:38:11.404962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":225,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-05-20T18:29:04.082245+0000","last_change":"2024-05-20T18:27:03.765694+0000","last_active":"2024-05-20T18:29:04.082245+0000","last_peered":"2024-05-20T18:29:04.082245+0000","last_clean":"2024-05-20T18:29:04.082245+0000","last_became_active":"2024-05-20T18:27:03.764695+0000","last_became_peered":"2024-05-20T18:27:03.764695+0000","last_unstale":"2024-05-20T18:29:04.082245+0000","last_undegraded":"2024-05-20T18:29:04.082245+0000","last_fullsized":"2024-05-20T18:29:04.082245+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_clean_scrub_stamp":"2024-05-20T18:27:02.750853+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:30:04.407487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28200,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343640,"statfs":{"total":96636764160,"available":96607887360,"internally_reserved":0,"allocated":1208320,"data_stored":859872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":614400,"data_stored":270032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":606208,"data_stored":269380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":610304,"data_stored":269962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31792,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340048,"statfs":{"total":96636764160,"available":96604209152,"internally_reserved":0,"allocated":622592,"data_stored":271014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":618496,"data_stored":270992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":614400,"data_stored":269540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1208320,"data_stored":859804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T18:29:07.108 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-20T18:29:07.108 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-20T18:29:07.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.267+0000 7f2761764640 1 Processor -- start 2024-05-20T18:29:07.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.267+0000 7f2761764640 1 -- start start 2024-05-20T18:29:07.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.269+0000 7f2761764640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f275c06d3d0 0x7f275c06d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:07.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.269+0000 7f2761764640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f275c0b8190 0x7f275c0b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:07.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.269+0000 7f2761764640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f275c06dd20 con 0x7f275c162820 2024-05-20T18:29:07.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.269+0000 7f2761764640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f275c0d0630 con 0x7f275c0b8190 2024-05-20T18:29:07.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.269+0000 7f2761764640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f275c0d39c0 con 0x7f275c06d3d0 2024-05-20T18:29:07.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.269+0000 7f275affd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f275c162820 0x7f275c0b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36922/0 (socket says 172.21.15.5:36922) 2024-05-20T18:29:07.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.269+0000 7f275affd640 1 -- 172.21.15.5:0/883879770 learned_addr learned my addr 172.21.15.5:0/883879770 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:07.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.269+0000 7f2759ffb640 1 -- 172.21.15.5:0/883879770 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 931583943 0 0) 0x7f2754001490 con 0x7f275c162820 2024-05-20T18:29:07.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.269+0000 7f275a7fc640 1 --2- 172.21.15.5:0/883879770 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f275c06d3d0 0x7f275c06d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:07.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.269+0000 7f275b7fe640 1 --2- 172.21.15.5:0/883879770 >> v2:172.21.15.63:3300/0 conn(0x7f275c0b8190 0x7f275c0b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:07.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.269+0000 7f2759ffb640 1 -- 172.21.15.5:0/883879770 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2748003520 con 0x7f275c162820 2024-05-20T18:29:07.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.269+0000 7f2759ffb640 1 -- 172.21.15.5:0/883879770 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2337536288 0 0) 0x7f2748003520 con 0x7f275c162820 2024-05-20T18:29:07.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.269+0000 7f2759ffb640 1 -- 172.21.15.5:0/883879770 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f275c06d3d0 msgr2=0x7f275c06d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:07.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.269+0000 7f2759ffb640 1 --2- 172.21.15.5:0/883879770 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f275c06d3d0 0x7f275c06d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:07.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.269+0000 7f2759ffb640 1 -- 172.21.15.5:0/883879770 >> v2:172.21.15.63:3300/0 conn(0x7f275c0b8190 msgr2=0x7f275c0b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:07.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.269+0000 7f2759ffb640 1 --2- 172.21.15.5:0/883879770 >> v2:172.21.15.63:3300/0 conn(0x7f275c0b8190 0x7f275c0b85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:07.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.270+0000 7f2759ffb640 1 -- 172.21.15.5:0/883879770 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f275c0d3ca0 con 0x7f275c162820 2024-05-20T18:29:07.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.270+0000 7f2759ffb640 1 -- 172.21.15.5:0/883879770 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f2754001bc0 con 0x7f275c162820 2024-05-20T18:29:07.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.270+0000 7f275a7fc640 1 --2- 172.21.15.5:0/883879770 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f275c06d3d0 0x7f275c06d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:29:07.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.270+0000 7f2759ffb640 1 -- 172.21.15.5:0/883879770 >> v1:172.21.15.5:6789/0 conn(0x7f275c162820 legacy=0x7f275c0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:07.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.270+0000 7f2759ffb640 1 --2- 172.21.15.5:0/883879770 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f275c06d3d0 0x7f27480039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:07.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.270+0000 7f2759ffb640 1 --2- 172.21.15.5:0/883879770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2748003f10 0x7f2748004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:07.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.270+0000 7f2759ffb640 1 --2- 172.21.15.5:0/883879770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2748004d40 0x7f2748005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:07.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.270+0000 7f2759ffb640 1 -- 172.21.15.5:0/883879770 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f275c0d0630 con 0x7f2748004d40 2024-05-20T18:29:07.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.270+0000 7f2759ffb640 1 -- 172.21.15.5:0/883879770 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f275c0d39c0 con 0x7f2748003f10 2024-05-20T18:29:07.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.270+0000 7f2759ffb640 1 -- 172.21.15.5:0/883879770 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f274800bac0 con 0x7f275c06d3d0 2024-05-20T18:29:07.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.270+0000 7f275b7fe640 1 --2- 172.21.15.5:0/883879770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2748004d40 0x7f2748005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:07.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.270+0000 7f275affd640 1 --2- 172.21.15.5:0/883879770 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f275c06d3d0 0x7f27480039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:07.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.270+0000 7f275a7fc640 1 --2- 172.21.15.5:0/883879770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2748003f10 0x7f2748004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:07.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.270+0000 7f275b7fe640 1 -- 172.21.15.5:0/883879770 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f275c06d3d0 msgr2=0x7f27480039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:07.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.270+0000 7f275b7fe640 1 --2- 172.21.15.5:0/883879770 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f275c06d3d0 0x7f27480039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:07.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.270+0000 7f275b7fe640 1 -- 172.21.15.5:0/883879770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2748003f10 msgr2=0x7f2748004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:07.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.270+0000 7f275b7fe640 1 --2- 172.21.15.5:0/883879770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2748003f10 0x7f2748004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:07.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.270+0000 7f275b7fe640 1 -- 172.21.15.5:0/883879770 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2748003520 con 0x7f2748004d40 2024-05-20T18:29:07.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.271+0000 7f275b7fe640 1 --2- 172.21.15.5:0/883879770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2748004d40 0x7f2748005150 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f274c009ac0 tx=0x7f2748005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d6a221af0225eb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:07.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.271+0000 7f2759ffb640 1 -- 172.21.15.5:0/883879770 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274c036070 con 0x7f2748004d40 2024-05-20T18:29:07.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.271+0000 7f2759ffb640 1 -- 172.21.15.5:0/883879770 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f274c031400 con 0x7f2748004d40 2024-05-20T18:29:07.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.271+0000 7f2759ffb640 1 -- 172.21.15.5:0/883879770 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274c009690 con 0x7f2748004d40 2024-05-20T18:29:07.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.272+0000 7f2761764640 1 -- 172.21.15.5:0/883879770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2748004d40 msgr2=0x7f2748005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:07.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.272+0000 7f2761764640 1 --2- 172.21.15.5:0/883879770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2748004d40 0x7f2748005150 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f274c009ac0 tx=0x7f2748005830 comp rx=0 tx=0).stop 2024-05-20T18:29:07.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.272+0000 7f275b7fe640 1 -- 172.21.15.5:0/883879770 reap_dead start 2024-05-20T18:29:07.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.272+0000 7f2761764640 1 -- 172.21.15.5:0/883879770 shutdown_connections 2024-05-20T18:29:07.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.272+0000 7f2761764640 1 -- 172.21.15.5:0/883879770 >> 172.21.15.5:0/883879770 conn(0x7f275c0d4bd0 msgr2=0x7f275c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:07.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.272+0000 7f2761764640 1 -- 172.21.15.5:0/883879770 shutdown_connections 2024-05-20T18:29:07.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.272+0000 7f2761764640 1 -- 172.21.15.5:0/883879770 wait complete. 2024-05-20T18:29:07.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.274+0000 7f2761764640 1 Processor -- start 2024-05-20T18:29:07.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.274+0000 7f2761764640 1 -- start start 2024-05-20T18:29:07.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.275+0000 7f2761764640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2748004d40 0x7f275c150000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:07.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.275+0000 7f2761764640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f275c1530b0 0x7f275c0ab5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:07.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.275+0000 7f2761764640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f275c0abae0 0x7f275c0abef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:07.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.275+0000 7f2761764640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f275c0d0300 con 0x7f275c1530b0 2024-05-20T18:29:07.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.275+0000 7f2761764640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f275c0d4260 con 0x7f275c0abae0 2024-05-20T18:29:07.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.275+0000 7f2761764640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f275c0b4050 con 0x7f2748004d40 2024-05-20T18:29:07.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.275+0000 7f275a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f275c1530b0 0x7f275c0ab5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:07.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.275+0000 7f275a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f275c1530b0 0x7f275c0ab5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42824/0 (socket says 172.21.15.5:42824) 2024-05-20T18:29:07.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.275+0000 7f275a7fc640 1 -- 172.21.15.5:0/2338506846 learned_addr learned my addr 172.21.15.5:0/2338506846 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:07.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.276+0000 7f275affd640 1 --2- 172.21.15.5:0/2338506846 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2748004d40 0x7f275c150000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:07.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.276+0000 7f275b7fe640 1 --2- 172.21.15.5:0/2338506846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f275c0abae0 0x7f275c0abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:07.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.276+0000 7f275a7fc640 1 -- 172.21.15.5:0/2338506846 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2748004d40 msgr2=0x7f275c150000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:07.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.276+0000 7f275a7fc640 1 --2- 172.21.15.5:0/2338506846 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2748004d40 0x7f275c150000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:07.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.276+0000 7f275a7fc640 1 -- 172.21.15.5:0/2338506846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f275c0abae0 msgr2=0x7f275c0abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:07.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.276+0000 7f275a7fc640 1 --2- 172.21.15.5:0/2338506846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f275c0abae0 0x7f275c0abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:07.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.276+0000 7f275a7fc640 1 -- 172.21.15.5:0/2338506846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f275c150540 con 0x7f275c1530b0 2024-05-20T18:29:07.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.276+0000 7f275affd640 1 --2- 172.21.15.5:0/2338506846 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2748004d40 0x7f275c150000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:29:07.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.276+0000 7f275b7fe640 1 --2- 172.21.15.5:0/2338506846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f275c0abae0 0x7f275c0abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:29:07.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.276+0000 7f275a7fc640 1 --2- 172.21.15.5:0/2338506846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f275c1530b0 0x7f275c0ab5a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2744011620 tx=0x7f2744011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:07.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.276+0000 7f27537fe640 1 -- 172.21.15.5:0/2338506846 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744019070 con 0x7f275c1530b0 2024-05-20T18:29:07.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.276+0000 7f27537fe640 1 -- 172.21.15.5:0/2338506846 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2744004220 con 0x7f275c1530b0 2024-05-20T18:29:07.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.276+0000 7f2761764640 1 -- 172.21.15.5:0/2338506846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f275c0cd330 con 0x7f275c1530b0 2024-05-20T18:29:07.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.276+0000 7f27537fe640 1 -- 172.21.15.5:0/2338506846 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274400f040 con 0x7f275c1530b0 2024-05-20T18:29:07.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.276+0000 7f2761764640 1 -- 172.21.15.5:0/2338506846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f275c0cd8f0 con 0x7f275c1530b0 2024-05-20T18:29:07.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.277+0000 7f2761764640 1 -- 172.21.15.5:0/2338506846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f275c075b00 con 0x7f275c1530b0 2024-05-20T18:29:07.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.278+0000 7f27537fe640 1 -- 172.21.15.5:0/2338506846 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f274400f720 con 0x7f275c1530b0 2024-05-20T18:29:07.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.278+0000 7f27537fe640 1 --2- 172.21.15.5:0/2338506846 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f273c06be30 0x7f273c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:07.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.278+0000 7f27537fe640 1 -- 172.21.15.5:0/2338506846 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f274408b350 con 0x7f275c1530b0 2024-05-20T18:29:07.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.278+0000 7f275affd640 1 --2- 172.21.15.5:0/2338506846 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f273c06be30 0x7f273c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:07.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.279+0000 7f275affd640 1 --2- 172.21.15.5:0/2338506846 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f273c06be30 0x7f273c06e340 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f27540013d0 tx=0x7f2754006040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:07.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.283+0000 7f27537fe640 1 -- 172.21.15.5:0/2338506846 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2744059ae0 con 0x7f275c1530b0 2024-05-20T18:29:07.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.545+0000 7f2761764640 1 -- 172.21.15.5:0/2338506846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f275c0cd000 con 0x7f275c1530b0 2024-05-20T18:29:07.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.548+0000 7f27537fe640 1 -- 172.21.15.5:0/2338506846 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7f27440593e0 con 0x7f275c1530b0 2024-05-20T18:29:07.548 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2024-05-20T18:29:07.548 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDFlUtmevaaIhAATIwyXzr0vGT38ZmcMqR2gA== 2024-05-20T18:29:07.548 INFO:teuthology.orchestra.run.smithi005.stdout: caps mds = "allow" 2024-05-20T18:29:07.548 INFO:teuthology.orchestra.run.smithi005.stdout: caps mon = "allow r" 2024-05-20T18:29:07.548 INFO:teuthology.orchestra.run.smithi005.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-20T18:29:07.548 INFO:teuthology.orchestra.run.smithi005.stderr:updated caps for client.0 2024-05-20T18:29:07.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.550+0000 7f2761764640 1 -- 172.21.15.5:0/2338506846 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f273c06be30 msgr2=0x7f273c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:07.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.550+0000 7f2761764640 1 --2- 172.21.15.5:0/2338506846 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f273c06be30 0x7f273c06e340 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f27540013d0 tx=0x7f2754006040 comp rx=0 tx=0).stop 2024-05-20T18:29:07.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.550+0000 7f2761764640 1 -- 172.21.15.5:0/2338506846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f275c1530b0 msgr2=0x7f275c0ab5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:07.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.550+0000 7f2761764640 1 --2- 172.21.15.5:0/2338506846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f275c1530b0 0x7f275c0ab5a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2744011620 tx=0x7f2744011ae0 comp rx=0 tx=0).stop 2024-05-20T18:29:07.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.550+0000 7f2761764640 1 -- 172.21.15.5:0/2338506846 shutdown_connections 2024-05-20T18:29:07.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.550+0000 7f2761764640 1 --2- 172.21.15.5:0/2338506846 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f273c06be30 0x7f273c06e340 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:07.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.550+0000 7f2761764640 1 --2- 172.21.15.5:0/2338506846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f275c0abae0 0x7f275c0abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:07.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.550+0000 7f2761764640 1 --2- 172.21.15.5:0/2338506846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f275c1530b0 0x7f275c0ab5a0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:07.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.550+0000 7f2761764640 1 --2- 172.21.15.5:0/2338506846 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2748004d40 0x7f275c150000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:07.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.550+0000 7f2761764640 1 -- 172.21.15.5:0/2338506846 >> 172.21.15.5:0/2338506846 conn(0x7f275c0d4bd0 msgr2=0x7f275c0b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:07.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.550+0000 7f2761764640 1 -- 172.21.15.5:0/2338506846 shutdown_connections 2024-05-20T18:29:07.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.551+0000 7f2761764640 1 -- 172.21.15.5:0/2338506846 wait complete. 2024-05-20T18:29:07.562 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:29:07.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.677+0000 7f1c3e456640 1 Processor -- start 2024-05-20T18:29:07.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.677+0000 7f1c3e456640 1 -- start start 2024-05-20T18:29:07.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.678+0000 7f1c3e456640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1c380cf250 0x7f1c3815f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:07.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.678+0000 7f1c3e456640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1c380b6680 0x7f1c380b6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:07.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.678+0000 7f1c3e456640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c380b6fd0 con 0x7f1c38163020 2024-05-20T18:29:07.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.678+0000 7f1c3e456640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f1c3815f9a0 con 0x7f1c380cf250 2024-05-20T18:29:07.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.678+0000 7f1c3e456640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1c380d0b10 con 0x7f1c380b6680 2024-05-20T18:29:07.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.678+0000 7f1c37fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1c380cf250 0x7f1c3815f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:07.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.678+0000 7f1c3c9f3640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1c38163020 0x7f1c38163430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36930/0 (socket says 172.21.15.5:36930) 2024-05-20T18:29:07.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.678+0000 7f1c3c9f3640 1 -- 172.21.15.5:0/249811846 learned_addr learned my addr 172.21.15.5:0/249811846 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:07.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.679+0000 7f1c377fe640 1 --2- 172.21.15.5:0/249811846 >> v2:172.21.15.63:3300/0 conn(0x7f1c380b6680 0x7f1c380b6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:07.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.679+0000 7f1c37fff640 1 -- 172.21.15.5:0/249811846 >> v2:172.21.15.63:3300/0 conn(0x7f1c380b6680 msgr2=0x7f1c380b6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:07.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.679+0000 7f1c37fff640 1 --2- 172.21.15.5:0/249811846 >> v2:172.21.15.63:3300/0 conn(0x7f1c380b6680 0x7f1c380b6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:07.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.679+0000 7f1c37fff640 1 -- 172.21.15.5:0/249811846 >> v1:172.21.15.5:6789/0 conn(0x7f1c38163020 legacy=0x7f1c38163430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:07.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.679+0000 7f1c36ffd640 1 -- 172.21.15.5:0/249811846 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2092367646 0 0) 0x7f1c30001490 con 0x7f1c38163020 2024-05-20T18:29:07.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.679+0000 7f1c37fff640 1 -- 172.21.15.5:0/249811846 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c380d0550 con 0x7f1c380cf250 2024-05-20T18:29:07.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.680+0000 7f1c37fff640 1 --2- 172.21.15.5:0/249811846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c380cf250 0x7f1c3815f460 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1c380d4bb0 tx=0x7f1c2802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6993d0d8289a9afa server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:07.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.680+0000 7f1c36ffd640 1 -- 172.21.15.5:0/249811846 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c28037070 con 0x7f1c380cf250 2024-05-20T18:29:07.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.680+0000 7f1c36ffd640 1 -- 172.21.15.5:0/249811846 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c28032440 con 0x7f1c380cf250 2024-05-20T18:29:07.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.680+0000 7f1c36ffd640 1 -- 172.21.15.5:0/249811846 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c28032810 con 0x7f1c380cf250 2024-05-20T18:29:07.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.681+0000 7f1c3e456640 1 -- 172.21.15.5:0/249811846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c380cf250 msgr2=0x7f1c3815f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:07.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.681+0000 7f1c3e456640 1 --2- 172.21.15.5:0/249811846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c380cf250 0x7f1c3815f460 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1c380d4bb0 tx=0x7f1c2802a1a0 comp rx=0 tx=0).stop 2024-05-20T18:29:07.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.681+0000 7f1c3e456640 1 -- 172.21.15.5:0/249811846 shutdown_connections 2024-05-20T18:29:07.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.681+0000 7f1c3e456640 1 --2- 172.21.15.5:0/249811846 >> v2:172.21.15.63:3300/0 conn(0x7f1c380b6680 0x7f1c380b6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:07.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.681+0000 7f1c3e456640 1 --2- 172.21.15.5:0/249811846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c380cf250 0x7f1c3815f460 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:07.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.681+0000 7f1c3e456640 1 -- 172.21.15.5:0/249811846 >> 172.21.15.5:0/249811846 conn(0x7f1c380d1860 msgr2=0x7f1c380d1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:07.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.681+0000 7f1c3e456640 1 -- 172.21.15.5:0/249811846 shutdown_connections 2024-05-20T18:29:07.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.681+0000 7f1c3e456640 1 -- 172.21.15.5:0/249811846 wait complete. 2024-05-20T18:29:07.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.682+0000 7f1c3e456640 1 Processor -- start 2024-05-20T18:29:07.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.683+0000 7f1c3e456640 1 -- start start 2024-05-20T18:29:07.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.684+0000 7f1c3e456640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c380b6680 0x7f1c380b93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:07.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.684+0000 7f1c3e456640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c380cf250 0x7f1c38157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:07.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.684+0000 7f1c3e456640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c38163020 0x7f1c3815fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:07.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.684+0000 7f1c3e456640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c380d2a20 con 0x7f1c38163020 2024-05-20T18:29:07.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.684+0000 7f1c3e456640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1c380d27d0 con 0x7f1c380cf250 2024-05-20T18:29:07.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.684+0000 7f1c3e456640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1c380d2d00 con 0x7f1c380b6680 2024-05-20T18:29:07.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.684+0000 7f1c3c9f3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c38163020 0x7f1c3815fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:07.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.684+0000 7f1c3c9f3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c38163020 0x7f1c3815fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42836/0 (socket says 172.21.15.5:42836) 2024-05-20T18:29:07.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.684+0000 7f1c3c9f3640 1 -- 172.21.15.5:0/3975709689 learned_addr learned my addr 172.21.15.5:0/3975709689 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:07.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.684+0000 7f1c377fe640 1 --2- 172.21.15.5:0/3975709689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c380cf250 0x7f1c38157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:07.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.684+0000 7f1c37fff640 1 --2- 172.21.15.5:0/3975709689 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c380b6680 0x7f1c380b93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:07.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.685+0000 7f1c3c9f3640 1 -- 172.21.15.5:0/3975709689 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c380b6680 msgr2=0x7f1c380b93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:07.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.685+0000 7f1c3c9f3640 1 --2- 172.21.15.5:0/3975709689 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c380b6680 0x7f1c380b93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:07.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.685+0000 7f1c3c9f3640 1 -- 172.21.15.5:0/3975709689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c380cf250 msgr2=0x7f1c38157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:07.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.685+0000 7f1c3c9f3640 1 --2- 172.21.15.5:0/3975709689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c380cf250 0x7f1c38157030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:07.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.685+0000 7f1c3c9f3640 1 -- 172.21.15.5:0/3975709689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c381f6350 con 0x7f1c38163020 2024-05-20T18:29:07.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.685+0000 7f1c3c9f3640 1 --2- 172.21.15.5:0/3975709689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c38163020 0x7f1c3815fb20 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000c730 tx=0x7f1c3000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:07.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.685+0000 7f1c34ff9640 1 -- 172.21.15.5:0/3975709689 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c30018070 con 0x7f1c38163020 2024-05-20T18:29:07.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.685+0000 7f1c34ff9640 1 -- 172.21.15.5:0/3975709689 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c3000ce40 con 0x7f1c38163020 2024-05-20T18:29:07.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.685+0000 7f1c3e456640 1 -- 172.21.15.5:0/3975709689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c380b9c40 con 0x7f1c38163020 2024-05-20T18:29:07.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.685+0000 7f1c34ff9640 1 -- 172.21.15.5:0/3975709689 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3000a860 con 0x7f1c38163020 2024-05-20T18:29:07.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.685+0000 7f1c3e456640 1 -- 172.21.15.5:0/3975709689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c380ba200 con 0x7f1c38163020 2024-05-20T18:29:07.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.686+0000 7f1c3e456640 1 -- 172.21.15.5:0/3975709689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c38075b00 con 0x7f1c38163020 2024-05-20T18:29:07.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.687+0000 7f1c34ff9640 1 -- 172.21.15.5:0/3975709689 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c3001b070 con 0x7f1c38163020 2024-05-20T18:29:07.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.687+0000 7f1c34ff9640 1 --2- 172.21.15.5:0/3975709689 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c2406be60 0x7f1c2406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:07.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.687+0000 7f1c37fff640 1 --2- 172.21.15.5:0/3975709689 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c2406be60 0x7f1c2406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:07.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.687+0000 7f1c34ff9640 1 -- 172.21.15.5:0/3975709689 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f1c3008a530 con 0x7f1c38163020 2024-05-20T18:29:07.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.688+0000 7f1c37fff640 1 --2- 172.21.15.5:0/3975709689 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c2406be60 0x7f1c2406e370 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1c28034000 tx=0x7f1c2802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:07.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.692+0000 7f1c34ff9640 1 -- 172.21.15.5:0/3975709689 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1c380b6fd0 con 0x7f1c38163020 2024-05-20T18:29:07.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.916+0000 7f1c3e456640 1 -- 172.21.15.5:0/3975709689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c380b99d0 con 0x7f1c38163020 2024-05-20T18:29:07.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.917+0000 7f1c34ff9640 1 -- 172.21.15.5:0/3975709689 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5175 (secure 0 0 0) 0x7f1c30058700 con 0x7f1c38163020 2024-05-20T18:29:07.917 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:29:07.917 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5207,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3958852803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3958852803},{"type":"v1","addr":"172.21.15.63:6835","nonce":3958852803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5560,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/3020169757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":3020169757},{"type":"v1","addr":"172.21.15.63:6834","nonce":3020169757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2965018210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2965018210},{"type":"v1","addr":"172.21.15.5:6835","nonce":2965018210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:29:02.038442+0000","modified":"2024-05-20T18:29:05.055248+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5751},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5751":{"gid":5751,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6837/3469824363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3469824363},{"type":"v1","addr":"172.21.15.5:6837","nonce":3469824363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5751,"qdb_cluster":[5751]},"id":3}]} 2024-05-20T18:29:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-05-20T18:29:07.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.919+0000 7f1c3e456640 1 -- 172.21.15.5:0/3975709689 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c2406be60 msgr2=0x7f1c2406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:07.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.919+0000 7f1c3e456640 1 --2- 172.21.15.5:0/3975709689 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c2406be60 0x7f1c2406e370 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1c28034000 tx=0x7f1c2802e000 comp rx=0 tx=0).stop 2024-05-20T18:29:07.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.919+0000 7f1c3e456640 1 -- 172.21.15.5:0/3975709689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c38163020 msgr2=0x7f1c3815fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:07.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.919+0000 7f1c3e456640 1 --2- 172.21.15.5:0/3975709689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c38163020 0x7f1c3815fb20 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000c730 tx=0x7f1c3000cbf0 comp rx=0 tx=0).stop 2024-05-20T18:29:07.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.919+0000 7f1c3e456640 1 -- 172.21.15.5:0/3975709689 shutdown_connections 2024-05-20T18:29:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.919+0000 7f1c3e456640 1 --2- 172.21.15.5:0/3975709689 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c2406be60 0x7f1c2406e370 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.919+0000 7f1c3e456640 1 --2- 172.21.15.5:0/3975709689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c38163020 0x7f1c3815fb20 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.919+0000 7f1c3e456640 1 --2- 172.21.15.5:0/3975709689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c380cf250 0x7f1c38157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.919+0000 7f1c3e456640 1 --2- 172.21.15.5:0/3975709689 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c380b6680 0x7f1c380b93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.919+0000 7f1c3e456640 1 -- 172.21.15.5:0/3975709689 >> 172.21.15.5:0/3975709689 conn(0x7f1c380d1860 msgr2=0x7f1c380b5f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.919+0000 7f1c3e456640 1 -- 172.21.15.5:0/3975709689 shutdown_connections 2024-05-20T18:29:07.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:07.919+0000 7f1c3e456640 1 -- 172.21.15.5:0/3975709689 wait complete. 2024-05-20T18:29:08.333 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T18:29:02.038442+0000', 'modified': '2024-05-20T18:29:05.055248+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5751}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5751': {'gid': 5751, 'name': 'c', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.5:6837/3469824363', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6836', 'nonce': 3469824363}, {'type': 'v1', 'addr': '172.21.15.5:6837', 'nonce': 3469824363}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5751, 'qdb_cluster': [5751]} 2024-05-20T18:29:08.333 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-20T18:29:08.333 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-20T18:29:08.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.453+0000 7f4f85475640 1 Processor -- start 2024-05-20T18:29:08.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.453+0000 7f4f85475640 1 -- start start 2024-05-20T18:29:08.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.454+0000 7f4f85475640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f4f8006d3d0 0x7f4f8006d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:08.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.454+0000 7f4f85475640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f8006dd20 con 0x7f4f80162820 2024-05-20T18:29:08.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.454+0000 7f4f85475640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f800d0630 con 0x7f4f800b8190 2024-05-20T18:29:08.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.455+0000 7f4f85475640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f4f800d39c0 con 0x7f4f8006d3d0 2024-05-20T18:29:08.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.455+0000 7f4f7effd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f4f80162820 0x7f4f800b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36942/0 (socket says 172.21.15.5:36942) 2024-05-20T18:29:08.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.455+0000 7f4f7effd640 1 -- 172.21.15.5:0/106626226 learned_addr learned my addr 172.21.15.5:0/106626226 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:08.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.455+0000 7f4f7e7fc640 1 --2- 172.21.15.5:0/106626226 >> v2:172.21.15.63:3300/0 conn(0x7f4f8006d3d0 0x7f4f8006d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:08.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.455+0000 7f4f7dffb640 1 -- 172.21.15.5:0/106626226 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2388191461 0 0) 0x7f4f70001110 con 0x7f4f80162820 2024-05-20T18:29:08.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.455+0000 7f4f7dffb640 1 -- 172.21.15.5:0/106626226 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f68003520 con 0x7f4f80162820 2024-05-20T18:29:08.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.455+0000 7f4f7dffb640 1 -- 172.21.15.5:0/106626226 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 334112666 0 0) 0x7f4f68003520 con 0x7f4f80162820 2024-05-20T18:29:08.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.455+0000 7f4f7dffb640 1 -- 172.21.15.5:0/106626226 >> v2:172.21.15.63:3300/0 conn(0x7f4f8006d3d0 msgr2=0x7f4f8006d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:08.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.455+0000 7f4f7dffb640 1 --2- 172.21.15.5:0/106626226 >> v2:172.21.15.63:3300/0 conn(0x7f4f8006d3d0 0x7f4f8006d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.455+0000 7f4f7dffb640 1 -- 172.21.15.5:0/106626226 >> v1:172.21.15.63:6789/0 conn(0x7f4f800b8190 legacy=0x7f4f800b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:08.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.455+0000 7f4f7dffb640 1 -- 172.21.15.5:0/106626226 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f800d3ca0 con 0x7f4f80162820 2024-05-20T18:29:08.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.455+0000 7f4f7dffb640 1 -- 172.21.15.5:0/106626226 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f4f70001840 con 0x7f4f80162820 2024-05-20T18:29:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.455+0000 7f4f7dffb640 1 -- 172.21.15.5:0/106626226 >> v1:172.21.15.5:6789/0 conn(0x7f4f80162820 legacy=0x7f4f800b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.455+0000 7f4f7dffb640 1 --2- 172.21.15.5:0/106626226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f680039b0 0x7f4f68003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.455+0000 7f4f7dffb640 1 --2- 172.21.15.5:0/106626226 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f68004800 0x7f4f68004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.455+0000 7f4f7dffb640 1 --2- 172.21.15.5:0/106626226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f680051c0 0x7f4f6800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.455+0000 7f4f7dffb640 1 -- 172.21.15.5:0/106626226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f800d39c0 con 0x7f4f680039b0 2024-05-20T18:29:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.455+0000 7f4f7dffb640 1 -- 172.21.15.5:0/106626226 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4f6800c160 con 0x7f4f680051c0 2024-05-20T18:29:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.455+0000 7f4f7dffb640 1 -- 172.21.15.5:0/106626226 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4f6800c360 con 0x7f4f68004800 2024-05-20T18:29:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.455+0000 7f4f7effd640 1 --2- 172.21.15.5:0/106626226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f680039b0 0x7f4f68003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.456+0000 7f4f7e7fc640 1 --2- 172.21.15.5:0/106626226 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f68004800 0x7f4f68004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.456+0000 7f4f7f7fe640 1 --2- 172.21.15.5:0/106626226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f680051c0 0x7f4f6800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.456+0000 7f4f7effd640 1 -- 172.21.15.5:0/106626226 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f68004800 msgr2=0x7f4f68004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.456+0000 7f4f7effd640 1 --2- 172.21.15.5:0/106626226 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f68004800 0x7f4f68004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.456+0000 7f4f7effd640 1 -- 172.21.15.5:0/106626226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f680051c0 msgr2=0x7f4f6800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.456+0000 7f4f7effd640 1 --2- 172.21.15.5:0/106626226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f680051c0 0x7f4f6800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.456+0000 7f4f7effd640 1 -- 172.21.15.5:0/106626226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f68003520 con 0x7f4f680039b0 2024-05-20T18:29:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.456+0000 7f4f7f7fe640 1 --2- 172.21.15.5:0/106626226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f680051c0 0x7f4f6800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:29:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.456+0000 7f4f7e7fc640 1 --2- 172.21.15.5:0/106626226 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f68004800 0x7f4f68004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:29:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.456+0000 7f4f7f7fe640 1 -- 172.21.15.5:0/106626226 reap_dead start 2024-05-20T18:29:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.457+0000 7f4f7effd640 1 --2- 172.21.15.5:0/106626226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f680039b0 0x7f4f68003e20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4f7000c4c0 tx=0x7f4f68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f3e978e8fc2936 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.457+0000 7f4f7dffb640 1 -- 172.21.15.5:0/106626226 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f70036070 con 0x7f4f680039b0 2024-05-20T18:29:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.457+0000 7f4f7dffb640 1 -- 172.21.15.5:0/106626226 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f70005af0 con 0x7f4f680039b0 2024-05-20T18:29:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.457+0000 7f4f7dffb640 1 -- 172.21.15.5:0/106626226 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7000c0e0 con 0x7f4f680039b0 2024-05-20T18:29:08.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.457+0000 7f4f85475640 1 -- 172.21.15.5:0/106626226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f680039b0 msgr2=0x7f4f68003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:08.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.457+0000 7f4f85475640 1 --2- 172.21.15.5:0/106626226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f680039b0 0x7f4f68003e20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4f7000c4c0 tx=0x7f4f68005830 comp rx=0 tx=0).stop 2024-05-20T18:29:08.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.457+0000 7f4f85475640 1 -- 172.21.15.5:0/106626226 shutdown_connections 2024-05-20T18:29:08.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.457+0000 7f4f85475640 1 --2- 172.21.15.5:0/106626226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f680039b0 0x7f4f68003e20 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.457+0000 7f4f85475640 1 -- 172.21.15.5:0/106626226 >> 172.21.15.5:0/106626226 conn(0x7f4f800d4bd0 msgr2=0x7f4f800d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:08.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.457+0000 7f4f85475640 1 -- 172.21.15.5:0/106626226 shutdown_connections 2024-05-20T18:29:08.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.458+0000 7f4f85475640 1 -- 172.21.15.5:0/106626226 wait complete. 2024-05-20T18:29:08.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.459+0000 7f4f85475640 1 Processor -- start 2024-05-20T18:29:08.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.459+0000 7f4f85475640 1 -- start start 2024-05-20T18:29:08.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.461+0000 7f4f85475640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f680039b0 0x7f4f8015bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:08.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.461+0000 7f4f85475640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f8015af20 0x7f4f800c9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.461+0000 7f4f85475640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f800c9d80 0x7f4f800ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.461+0000 7f4f85475640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f800d0300 con 0x7f4f8015af20 2024-05-20T18:29:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.461+0000 7f4f85475640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4f800d4260 con 0x7f4f680039b0 2024-05-20T18:29:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.461+0000 7f4f85475640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4f800d4780 con 0x7f4f800c9d80 2024-05-20T18:29:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.461+0000 7f4f7e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f8015af20 0x7f4f800c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.461+0000 7f4f7e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f8015af20 0x7f4f800c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42866/0 (socket says 172.21.15.5:42866) 2024-05-20T18:29:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.461+0000 7f4f7e7fc640 1 -- 172.21.15.5:0/2829392384 learned_addr learned my addr 172.21.15.5:0/2829392384 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.461+0000 7f4f7effd640 1 --2- 172.21.15.5:0/2829392384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f680039b0 0x7f4f8015bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.461+0000 7f4f7f7fe640 1 --2- 172.21.15.5:0/2829392384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f800c9d80 0x7f4f800ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.461+0000 7f4f7e7fc640 1 -- 172.21.15.5:0/2829392384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f800c9d80 msgr2=0x7f4f800ca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.461+0000 7f4f7e7fc640 1 --2- 172.21.15.5:0/2829392384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f800c9d80 0x7f4f800ca190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.461+0000 7f4f7e7fc640 1 -- 172.21.15.5:0/2829392384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f680039b0 msgr2=0x7f4f8015bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.461+0000 7f4f7e7fc640 1 --2- 172.21.15.5:0/2829392384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f680039b0 0x7f4f8015bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.461+0000 7f4f7e7fc640 1 -- 172.21.15.5:0/2829392384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f800c87a0 con 0x7f4f8015af20 2024-05-20T18:29:08.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.461+0000 7f4f7effd640 1 --2- 172.21.15.5:0/2829392384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f680039b0 0x7f4f8015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:29:08.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.461+0000 7f4f7e7fc640 1 --2- 172.21.15.5:0/2829392384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f8015af20 0x7f4f800c9840 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4f74011510 tx=0x7f4f74011910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:08.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.462+0000 7f4f677fe640 1 -- 172.21.15.5:0/2829392384 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f74015870 con 0x7f4f8015af20 2024-05-20T18:29:08.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.462+0000 7f4f677fe640 1 -- 172.21.15.5:0/2829392384 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f74015a10 con 0x7f4f8015af20 2024-05-20T18:29:08.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.462+0000 7f4f85475640 1 -- 172.21.15.5:0/2829392384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f800c8a80 con 0x7f4f8015af20 2024-05-20T18:29:08.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.462+0000 7f4f677fe640 1 -- 172.21.15.5:0/2829392384 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f74015e00 con 0x7f4f8015af20 2024-05-20T18:29:08.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.462+0000 7f4f85475640 1 -- 172.21.15.5:0/2829392384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f800c9040 con 0x7f4f8015af20 2024-05-20T18:29:08.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.463+0000 7f4f677fe640 1 -- 172.21.15.5:0/2829392384 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4f7401d400 con 0x7f4f8015af20 2024-05-20T18:29:08.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.464+0000 7f4f677fe640 1 --2- 172.21.15.5:0/2829392384 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4f6c06be60 0x7f4f6c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:08.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.464+0000 7f4f7effd640 1 --2- 172.21.15.5:0/2829392384 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4f6c06be60 0x7f4f6c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:08.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.464+0000 7f4f677fe640 1 -- 172.21.15.5:0/2829392384 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f4f7408b820 con 0x7f4f8015af20 2024-05-20T18:29:08.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.465+0000 7f4f7effd640 1 --2- 172.21.15.5:0/2829392384 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4f6c06be60 0x7f4f6c06e370 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4f70033000 tx=0x7f4f70009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:08.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.465+0000 7f4f8c3f1740 1 Processor -- start 2024-05-20T18:29:08.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.465+0000 7f4f8c3f1740 1 -- start start 2024-05-20T18:29:08.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.467+0000 7f4f8c3f1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560081609270 0x5600819d4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:08.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.467+0000 7f4f8c3f1740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5600815e03b0 0x5600819d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:08.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.467+0000 7f4f8c3f1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600815e8590 0x5600819aa360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:08.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.467+0000 7f4f8c3f1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56008190d4d0 con 0x5600815e8590 2024-05-20T18:29:08.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.467+0000 7f4f8c3f1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5600818b53e0 con 0x560081609270 2024-05-20T18:29:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.467+0000 7f4f8c3f1740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x560081909490 con 0x5600815e03b0 2024-05-20T18:29:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.467+0000 7f4f7f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600815e8590 0x5600819aa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.467+0000 7f4f7f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600815e8590 0x5600819aa360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42872/0 (socket says 172.21.15.5:42872) 2024-05-20T18:29:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.467+0000 7f4f7f7fe640 1 -- 172.21.15.5:0/3487676542 learned_addr learned my addr 172.21.15.5:0/3487676542 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.467+0000 7f4f7effd640 1 --2- 172.21.15.5:0/3487676542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560081609270 0x5600819d4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.467+0000 7f4f7e7fc640 1 --2- 172.21.15.5:0/3487676542 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5600815e03b0 0x5600819d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.467+0000 7f4f7f7fe640 1 -- 172.21.15.5:0/3487676542 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5600815e03b0 msgr2=0x5600819d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.467+0000 7f4f7f7fe640 1 --2- 172.21.15.5:0/3487676542 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5600815e03b0 0x5600819d10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.467+0000 7f4f7f7fe640 1 -- 172.21.15.5:0/3487676542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560081609270 msgr2=0x5600819d4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.467+0000 7f4f7f7fe640 1 --2- 172.21.15.5:0/3487676542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560081609270 0x5600819d4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.467+0000 7f4f7f7fe640 1 -- 172.21.15.5:0/3487676542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600816212d0 con 0x5600815e8590 2024-05-20T18:29:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.467+0000 7f4f7f7fe640 1 --2- 172.21.15.5:0/3487676542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600815e8590 0x5600819aa360 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4f8008d5e0 tx=0x7f4f78001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c06e791c27f1504 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.467+0000 7f4f64ff9640 1 -- 172.21.15.5:0/3487676542 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78006d60 con 0x5600815e8590 2024-05-20T18:29:08.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.467+0000 7f4f64ff9640 1 -- 172.21.15.5:0/3487676542 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f78003970 con 0x5600815e8590 2024-05-20T18:29:08.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.467+0000 7f4f64ff9640 1 -- 172.21.15.5:0/3487676542 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78003b20 con 0x5600815e8590 2024-05-20T18:29:08.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.468+0000 7f4f8c3f1740 1 -- 172.21.15.5:0/3487676542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600815e8590 msgr2=0x5600819aa360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:08.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.468+0000 7f4f8c3f1740 1 --2- 172.21.15.5:0/3487676542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600815e8590 0x5600819aa360 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4f8008d5e0 tx=0x7f4f78001e90 comp rx=0 tx=0).stop 2024-05-20T18:29:08.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.468+0000 7f4f8c3f1740 1 -- 172.21.15.5:0/3487676542 shutdown_connections 2024-05-20T18:29:08.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.468+0000 7f4f8c3f1740 1 --2- 172.21.15.5:0/3487676542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600815e8590 0x5600819aa360 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.468+0000 7f4f8c3f1740 1 --2- 172.21.15.5:0/3487676542 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5600815e03b0 0x5600819d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.468+0000 7f4f8c3f1740 1 --2- 172.21.15.5:0/3487676542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560081609270 0x5600819d4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.468+0000 7f4f8c3f1740 1 -- 172.21.15.5:0/3487676542 >> 172.21.15.5:0/3487676542 conn(0x5600815fd3c0 msgr2=0x5600819a9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:08.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.468+0000 7f4f8c3f1740 1 -- 172.21.15.5:0/3487676542 shutdown_connections 2024-05-20T18:29:08.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.468+0000 7f4f8c3f1740 1 -- 172.21.15.5:0/3487676542 wait complete. 2024-05-20T18:29:08.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.469+0000 7f4f8c3f1740 1 -- start start 2024-05-20T18:29:08.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.469+0000 7f4f8c3f1740 1 Processor -- start 2024-05-20T18:29:08.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.471+0000 7f4f8c3f1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5600815e03b0 0x5600819cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:08.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.471+0000 7f4f8c3f1740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5600815e8590 0x5600819d7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:08.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.471+0000 7f4f8c3f1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081609270 0x5600819d7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:08.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.471+0000 7f4f8c3f1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5600818c88e0 con 0x560081609270 2024-05-20T18:29:08.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.471+0000 7f4f8c3f1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5600819cfc80 con 0x5600815e03b0 2024-05-20T18:29:08.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.471+0000 7f4f8c3f1740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5600819eeee0 con 0x5600815e8590 2024-05-20T18:29:08.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.471+0000 7f4f8c3f1740 10 client.0 _ll_register_callbacks cb 0x5600818959c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:29:08.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.471+0000 7f4f7f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081609270 0x5600819d7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:08.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.471+0000 7f4f7f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081609270 0x5600819d7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42888/0 (socket says 172.21.15.5:42888) 2024-05-20T18:29:08.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.471+0000 7f4f7f7fe640 1 -- 172.21.15.5:0/349407339 learned_addr learned my addr 172.21.15.5:0/349407339 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:08.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.471+0000 7f4f7e7fc640 1 --2- 172.21.15.5:0/349407339 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5600815e8590 0x5600819d7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:08.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.471+0000 7f4f7effd640 1 --2- 172.21.15.5:0/349407339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5600815e03b0 0x5600819cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:08.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.471+0000 7f4f7f7fe640 1 -- 172.21.15.5:0/349407339 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5600815e8590 msgr2=0x5600819d7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:08.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.471+0000 7f4f7f7fe640 1 --2- 172.21.15.5:0/349407339 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5600815e8590 0x5600819d7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.471+0000 7f4f7f7fe640 1 -- 172.21.15.5:0/349407339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5600815e03b0 msgr2=0x5600819cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:08.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.471+0000 7f4f7f7fe640 1 --2- 172.21.15.5:0/349407339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5600815e03b0 0x5600819cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.471+0000 7f4f7f7fe640 1 -- 172.21.15.5:0/349407339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600816eb6b0 con 0x560081609270 2024-05-20T18:29:08.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.471+0000 7f4f7f7fe640 1 --2- 172.21.15.5:0/349407339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081609270 0x5600819d7db0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4f78015000 tx=0x7f4f78001510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83aa89ab53c09f07 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:08.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.471+0000 7f4f5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:29:08.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.471+0000 7f4f5dffb640 1 -- 172.21.15.5:0/349407339 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78011d90 con 0x560081609270 2024-05-20T18:29:08.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.471+0000 7f4f5dffb640 1 -- 172.21.15.5:0/349407339 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f4f7801b070 con 0x560081609270 2024-05-20T18:29:08.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.471+0000 7f4f8c3f1740 1 -- 172.21.15.5:0/349407339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f78003970 con 0x560081609270 2024-05-20T18:29:08.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.472+0000 7f4f5dffb640 1 -- 172.21.15.5:0/349407339 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4f7802d050 con 0x560081609270 2024-05-20T18:29:08.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.472+0000 7f4f8c3f1740 10 client.5880 fetch_fsmap learned FSMap version 26 2024-05-20T18:29:08.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.472+0000 7f4f8c3f1740 1 -- 172.21.15.5:0/349407339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x5600819cc590 con 0x560081609270 2024-05-20T18:29:08.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.472+0000 7f4f5dffb640 1 -- 172.21.15.5:0/349407339 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f78003970 con 0x560081609270 2024-05-20T18:29:08.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.472+0000 7f4f5dffb640 1 -- 172.21.15.5:0/349407339 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f4f78030020 con 0x560081609270 2024-05-20T18:29:08.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.472+0000 7f4f8c3f1740 10 client.5880 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T18:29:08.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.472+0000 7f4f8c3f1740 10 client.5880 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-20T18:29:08.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.472+0000 7f4f8c3f1740 20 client.5880 populate_metadata read hostname 'smithi005' 2024-05-20T18:29:08.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.472+0000 7f4f8c3f1740 1 --2- 172.21.15.5:0/349407339 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x5600815c7e40 0x560081a0c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:08.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.472+0000 7f4f8c3f1740 4 client.5880 mds_command: new command op to 5751 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:29:08.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.472+0000 7f4f8c3f1740 1 -- 172.21.15.5:0/349407339 --> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56008191ac00 con 0x5600815c7e40 2024-05-20T18:29:08.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.472+0000 7f4f7effd640 1 --2- 172.21.15.5:0/349407339 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x5600815c7e40 0x560081a0c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:08.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.472+0000 7f4f5dffb640 10 client.5880 ms_handle_connect on v2:172.21.15.5:6836/3469824363 2024-05-20T18:29:08.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.472+0000 7f4f7effd640 1 --2- 172.21.15.5:0/349407339 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x5600815c7e40 0x560081a0c360 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d0894cae89f952c server_cookie=28ba09a5f8b7690f in_seq=0 out_seq=0 2024-05-20T18:29:08.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.473+0000 7f4f5dffb640 1 -- 172.21.15.5:0/349407339 <== mds.0 v2:172.21.15.5:6836/3469824363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56008191ac00 con 0x5600815c7e40 2024-05-20T18:29:08.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.473+0000 7f4f5dffb640 10 client.5880 handle_command_reply: tid=0 2024-05-20T18:29:08.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.473+0000 7f4f8c3f1740 1 client.5880 shutdown 2024-05-20T18:29:08.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.473+0000 7f4f8c3f1740 1 -- 172.21.15.5:0/349407339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081609270 msgr2=0x5600819d7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:08.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.473+0000 7f4f8c3f1740 1 --2- 172.21.15.5:0/349407339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081609270 0x5600819d7db0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4f78015000 tx=0x7f4f78001510 comp rx=0 tx=0).stop 2024-05-20T18:29:08.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.473+0000 7f4f8c3f1740 1 -- 172.21.15.5:0/349407339 shutdown_connections 2024-05-20T18:29:08.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.473+0000 7f4f8c3f1740 1 --2- 172.21.15.5:0/349407339 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x5600815c7e40 0x560081a0c360 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.473+0000 7f4f8c3f1740 1 --2- 172.21.15.5:0/349407339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560081609270 0x5600819d7db0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.473+0000 7f4f8c3f1740 1 --2- 172.21.15.5:0/349407339 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5600815e8590 0x5600819d7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.473+0000 7f4f8c3f1740 1 --2- 172.21.15.5:0/349407339 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5600815e03b0 0x5600819cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.473+0000 7f4f5dffb640 0 client.5880 ms_handle_reset on v2:172.21.15.5:6836/3469824363 2024-05-20T18:29:08.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.473+0000 7f4f8c3f1740 1 -- 172.21.15.5:0/349407339 >> 172.21.15.5:0/349407339 conn(0x5600815fd3c0 msgr2=0x5600819a9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:08.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.474+0000 7f4f8c3f1740 1 -- 172.21.15.5:0/349407339 shutdown_connections 2024-05-20T18:29:08.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.474+0000 7f4f8c3f1740 1 -- 172.21.15.5:0/349407339 wait complete. 2024-05-20T18:29:08.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.474+0000 7f4f8c3f1740 20 client.5880 trim_cache size 0 max 16384 2024-05-20T18:29:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.493+0000 7f4f8c3f1740 1 Processor -- start 2024-05-20T18:29:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.493+0000 7f4f8c3f1740 1 -- start start 2024-05-20T18:29:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.494+0000 7f4f8c3f1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600815e03b0 0x5600819ee320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.494+0000 7f4f8c3f1740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5600815e8590 0x5600819cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.494+0000 7f4f8c3f1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560081609270 0x5600819cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.494+0000 7f4f8c3f1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f80201be0 con 0x5600815e03b0 2024-05-20T18:29:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.494+0000 7f4f8c3f1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8015a4e0 con 0x560081609270 2024-05-20T18:29:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.494+0000 7f4f8c3f1740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4f802015f0 con 0x5600815e8590 2024-05-20T18:29:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.494+0000 7f4f7effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600815e03b0 0x5600819ee320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.494+0000 7f4f7effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600815e03b0 0x5600819ee320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42894/0 (socket says 172.21.15.5:42894) 2024-05-20T18:29:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.494+0000 7f4f7effd640 1 -- 172.21.15.5:0/2543431504 learned_addr learned my addr 172.21.15.5:0/2543431504 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.494+0000 7f4f7effd640 1 -- 172.21.15.5:0/2543431504 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5600815e8590 msgr2=0x5600819cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.494+0000 7f4f7effd640 1 --2- 172.21.15.5:0/2543431504 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5600815e8590 0x5600819cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.494+0000 7f4f7effd640 1 -- 172.21.15.5:0/2543431504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560081609270 msgr2=0x5600819cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.494+0000 7f4f7effd640 1 --2- 172.21.15.5:0/2543431504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560081609270 0x5600819cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.494+0000 7f4f7effd640 1 -- 172.21.15.5:0/2543431504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600819ce770 con 0x5600815e03b0 2024-05-20T18:29:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.494+0000 7f4f7effd640 1 --2- 172.21.15.5:0/2543431504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600815e03b0 0x5600819ee320 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4f7000c520 tx=0x7f4f70007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eafa0327dbee9b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.495+0000 7f4f5ffff640 1 -- 172.21.15.5:0/2543431504 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f70042040 con 0x5600815e03b0 2024-05-20T18:29:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.495+0000 7f4f5ffff640 1 -- 172.21.15.5:0/2543431504 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f70007800 con 0x5600815e03b0 2024-05-20T18:29:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.495+0000 7f4f5ffff640 1 -- 172.21.15.5:0/2543431504 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f70036040 con 0x5600815e03b0 2024-05-20T18:29:08.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.495+0000 7f4f8c3f1740 1 -- 172.21.15.5:0/2543431504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600815e03b0 msgr2=0x5600819ee320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:08.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.495+0000 7f4f8c3f1740 1 --2- 172.21.15.5:0/2543431504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600815e03b0 0x5600819ee320 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4f7000c520 tx=0x7f4f70007000 comp rx=0 tx=0).stop 2024-05-20T18:29:08.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.495+0000 7f4f8c3f1740 1 -- 172.21.15.5:0/2543431504 shutdown_connections 2024-05-20T18:29:08.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.495+0000 7f4f8c3f1740 1 --2- 172.21.15.5:0/2543431504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560081609270 0x5600819cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.495+0000 7f4f8c3f1740 1 --2- 172.21.15.5:0/2543431504 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5600815e8590 0x5600819cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.495+0000 7f4f8c3f1740 1 --2- 172.21.15.5:0/2543431504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600815e03b0 0x5600819ee320 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.495+0000 7f4f8c3f1740 1 -- 172.21.15.5:0/2543431504 >> 172.21.15.5:0/2543431504 conn(0x5600815fd3c0 msgr2=0x5600819ca2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:08.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.495+0000 7f4f8c3f1740 1 -- 172.21.15.5:0/2543431504 shutdown_connections 2024-05-20T18:29:08.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.495+0000 7f4f8c3f1740 1 -- 172.21.15.5:0/2543431504 wait complete. 2024-05-20T18:29:08.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.496+0000 7f4f8c3f1740 1 -- start start 2024-05-20T18:29:08.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.496+0000 7f4f8c3f1740 1 Processor -- start 2024-05-20T18:29:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.498+0000 7f4f8c3f1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600815e03b0 0x560081a6f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.498+0000 7f4f8c3f1740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5600815e8590 0x560081a70220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.498+0000 7f4f8c3f1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560081609270 0x5600819a9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.498+0000 7f4f8c3f1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5600819ced40 con 0x5600815e03b0 2024-05-20T18:29:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.498+0000 7f4f8c3f1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5600819ce940 con 0x560081609270 2024-05-20T18:29:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.498+0000 7f4f8c3f1740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5600819cf2d0 con 0x5600815e8590 2024-05-20T18:29:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.498+0000 7f4f8c3f1740 10 client.0 _ll_register_callbacks cb 0x7f4f78018020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:29:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.498+0000 7f4f7effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600815e03b0 0x560081a6f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.498+0000 7f4f7effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600815e03b0 0x560081a6f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42900/0 (socket says 172.21.15.5:42900) 2024-05-20T18:29:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.498+0000 7f4f7effd640 1 -- 172.21.15.5:0/3327971092 learned_addr learned my addr 172.21.15.5:0/3327971092 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.498+0000 7f4f7e7fc640 1 --2- 172.21.15.5:0/3327971092 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5600815e8590 0x560081a70220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.498+0000 7f4f7f7fe640 1 --2- 172.21.15.5:0/3327971092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560081609270 0x5600819a9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.498+0000 7f4f7effd640 1 -- 172.21.15.5:0/3327971092 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5600815e8590 msgr2=0x560081a70220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.498+0000 7f4f7effd640 1 --2- 172.21.15.5:0/3327971092 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5600815e8590 0x560081a70220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.498+0000 7f4f7effd640 1 -- 172.21.15.5:0/3327971092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560081609270 msgr2=0x5600819a9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.498+0000 7f4f7effd640 1 --2- 172.21.15.5:0/3327971092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560081609270 0x5600819a9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.498+0000 7f4f7effd640 1 -- 172.21.15.5:0/3327971092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56008199eec0 con 0x5600815e03b0 2024-05-20T18:29:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.498+0000 7f4f7f7fe640 1 --2- 172.21.15.5:0/3327971092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560081609270 0x5600819a9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:29:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.498+0000 7f4f7e7fc640 1 --2- 172.21.15.5:0/3327971092 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5600815e8590 0x560081a70220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:29:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.498+0000 7f4f7effd640 1 --2- 172.21.15.5:0/3327971092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600815e03b0 0x560081a6f2d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f4f7002af00 tx=0x7f4f7002d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4826d4e4ba63f13 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.498+0000 7f4f5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:29:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.498+0000 7f4f5dffb640 1 -- 172.21.15.5:0/3327971092 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7005c060 con 0x5600815e03b0 2024-05-20T18:29:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.498+0000 7f4f5dffb640 1 -- 172.21.15.5:0/3327971092 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f4f7004f4a0 con 0x5600815e03b0 2024-05-20T18:29:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.498+0000 7f4f8c3f1740 1 -- 172.21.15.5:0/3327971092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f70007800 con 0x5600815e03b0 2024-05-20T18:29:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.499+0000 7f4f5dffb640 1 -- 172.21.15.5:0/3327971092 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4f70062050 con 0x5600815e03b0 2024-05-20T18:29:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.499+0000 7f4f8c3f1740 10 client.5886 fetch_fsmap learned FSMap version 26 2024-05-20T18:29:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.499+0000 7f4f8c3f1740 1 -- 172.21.15.5:0/3327971092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x560081a6fd00 con 0x5600815e03b0 2024-05-20T18:29:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.499+0000 7f4f5dffb640 1 -- 172.21.15.5:0/3327971092 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f70007800 con 0x5600815e03b0 2024-05-20T18:29:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.499+0000 7f4f5dffb640 1 -- 172.21.15.5:0/3327971092 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f4f70065020 con 0x5600815e03b0 2024-05-20T18:29:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.499+0000 7f4f8c3f1740 10 client.5886 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T18:29:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.499+0000 7f4f8c3f1740 10 client.5886 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-20T18:29:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.499+0000 7f4f8c3f1740 20 client.5886 populate_metadata read hostname 'smithi005' 2024-05-20T18:29:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.499+0000 7f4f8c3f1740 1 --2- 172.21.15.5:0/3327971092 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x560081a07e50 0x560081a08260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.499+0000 7f4f8c3f1740 4 client.5886 mds_command: new command op to 5751 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T18:29:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.499+0000 7f4f8c3f1740 1 -- 172.21.15.5:0/3327971092 --> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] -- command(tid 0: {"prefix": "status"}) -- 0x5600819a51f0 con 0x560081a07e50 2024-05-20T18:29:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.499+0000 7f4f7e7fc640 1 --2- 172.21.15.5:0/3327971092 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x560081a07e50 0x560081a08260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.499+0000 7f4f5dffb640 10 client.5886 ms_handle_connect on v2:172.21.15.5:6836/3469824363 2024-05-20T18:29:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.499+0000 7f4f7e7fc640 1 --2- 172.21.15.5:0/3327971092 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x560081a07e50 0x560081a08260 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=163bf5811bf652bf server_cookie=29555ac4201f365a in_seq=0 out_seq=0 2024-05-20T18:29:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.500+0000 7f4f5dffb640 1 -- 172.21.15.5:0/3327971092 <== mds.0 v2:172.21.15.5:6836/3469824363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5600819a51f0 con 0x560081a07e50 2024-05-20T18:29:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.500+0000 7f4f5dffb640 10 client.5886 handle_command_reply: tid=0 2024-05-20T18:29:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.500+0000 7f4f8c3f1740 1 client.5886 shutdown 2024-05-20T18:29:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.500+0000 7f4f8c3f1740 1 -- 172.21.15.5:0/3327971092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600815e03b0 msgr2=0x560081a6f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.500+0000 7f4f8c3f1740 1 --2- 172.21.15.5:0/3327971092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600815e03b0 0x560081a6f2d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f4f7002af00 tx=0x7f4f7002d000 comp rx=0 tx=0).stop 2024-05-20T18:29:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.500+0000 7f4f8c3f1740 1 -- 172.21.15.5:0/3327971092 shutdown_connections 2024-05-20T18:29:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.500+0000 7f4f8c3f1740 1 --2- 172.21.15.5:0/3327971092 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x560081a07e50 0x560081a08260 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.500+0000 7f4f8c3f1740 1 --2- 172.21.15.5:0/3327971092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560081609270 0x5600819a9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.500+0000 7f4f8c3f1740 1 --2- 172.21.15.5:0/3327971092 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5600815e8590 0x560081a70220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.500+0000 7f4f5dffb640 0 client.5886 ms_handle_reset on v2:172.21.15.5:6836/3469824363 2024-05-20T18:29:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.500+0000 7f4f8c3f1740 1 --2- 172.21.15.5:0/3327971092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600815e03b0 0x560081a6f2d0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.500+0000 7f4f8c3f1740 1 -- 172.21.15.5:0/3327971092 >> 172.21.15.5:0/3327971092 conn(0x5600815fd3c0 msgr2=0x5600819ca0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.500+0000 7f4f8c3f1740 1 -- 172.21.15.5:0/3327971092 shutdown_connections 2024-05-20T18:29:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.500+0000 7f4f8c3f1740 1 -- 172.21.15.5:0/3327971092 wait complete. 2024-05-20T18:29:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.500+0000 7f4f8c3f1740 20 client.5886 trim_cache size 0 max 16384 2024-05-20T18:29:08.505 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:29:08.505 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "399ab250-fd3c-47fa-a1c8-25e8149c9cac", 2024-05-20T18:29:08.505 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-20T18:29:08.505 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 5751, 2024-05-20T18:29:08.505 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-20T18:29:08.505 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-20T18:29:08.506 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-20T18:29:08.506 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 5.8483107829999996, 2024-05-20T18:29:08.506 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 26, 2024-05-20T18:29:08.506 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 51, 2024-05-20T18:29:08.506 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 51, 2024-05-20T18:29:08.506 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 10.793600334000001 2024-05-20T18:29:08.506 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:29:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.500+0000 7f4f85475640 1 -- 172.21.15.5:0/2829392384 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4f6c06be60 msgr2=0x7f4f6c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.501+0000 7f4f85475640 1 --2- 172.21.15.5:0/2829392384 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4f6c06be60 0x7f4f6c06e370 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4f70033000 tx=0x7f4f70009030 comp rx=0 tx=0).stop 2024-05-20T18:29:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.501+0000 7f4f85475640 1 -- 172.21.15.5:0/2829392384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f8015af20 msgr2=0x7f4f800c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.501+0000 7f4f85475640 1 --2- 172.21.15.5:0/2829392384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f8015af20 0x7f4f800c9840 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4f74011510 tx=0x7f4f74011910 comp rx=0 tx=0).stop 2024-05-20T18:29:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.501+0000 7f4f85475640 1 -- 172.21.15.5:0/2829392384 shutdown_connections 2024-05-20T18:29:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.501+0000 7f4f85475640 1 --2- 172.21.15.5:0/2829392384 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4f6c06be60 0x7f4f6c06e370 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.501+0000 7f4f85475640 1 --2- 172.21.15.5:0/2829392384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f800c9d80 0x7f4f800ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.501+0000 7f4f85475640 1 --2- 172.21.15.5:0/2829392384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f8015af20 0x7f4f800c9840 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.501+0000 7f4f85475640 1 --2- 172.21.15.5:0/2829392384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f680039b0 0x7f4f8015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.501+0000 7f4f85475640 1 -- 172.21.15.5:0/2829392384 >> 172.21.15.5:0/2829392384 conn(0x7f4f800d4bd0 msgr2=0x7f4f800b52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.501+0000 7f4f85475640 1 -- 172.21.15.5:0/2829392384 shutdown_connections 2024-05-20T18:29:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.501+0000 7f4f85475640 1 -- 172.21.15.5:0/2829392384 wait complete. 2024-05-20T18:29:08.512 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-20T18:29:08.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.627+0000 7fbd86ca8640 1 Processor -- start 2024-05-20T18:29:08.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.627+0000 7fbd86ca8640 1 -- start start 2024-05-20T18:29:08.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.628+0000 7fbd86ca8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd800b6620 0x7fbd800ba3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:08.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.628+0000 7fbd86ca8640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbd800d3bf0 0x7fbd800d4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:08.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.628+0000 7fbd86ca8640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd800d4540 con 0x7fbd80162ce0 2024-05-20T18:29:08.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.628+0000 7fbd86ca8640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fbd800d3480 con 0x7fbd800d3bf0 2024-05-20T18:29:08.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.628+0000 7fbd86ca8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbd801634a0 con 0x7fbd800b6620 2024-05-20T18:29:08.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.629+0000 7fbd84a44640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd800b6620 0x7fbd800ba3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:08.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.629+0000 7fbd85245640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbd800d3bf0 0x7fbd800d4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:08.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.629+0000 7fbd7ffff640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fbd80162ce0 0x7fbd800b8070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:47170/0 (socket says 172.21.15.5:47170) 2024-05-20T18:29:08.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.629+0000 7fbd7ffff640 1 -- 172.21.15.5:0/3943540441 learned_addr learned my addr 172.21.15.5:0/3943540441 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:08.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.629+0000 7fbd84a44640 1 -- 172.21.15.5:0/3943540441 >> v2:172.21.15.63:3300/0 conn(0x7fbd800d3bf0 msgr2=0x7fbd800d4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:08.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.629+0000 7fbd84a44640 1 --2- 172.21.15.5:0/3943540441 >> v2:172.21.15.63:3300/0 conn(0x7fbd800d3bf0 0x7fbd800d4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.629+0000 7fbd84a44640 1 -- 172.21.15.5:0/3943540441 >> v1:172.21.15.63:6789/0 conn(0x7fbd80162ce0 legacy=0x7fbd800b8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:08.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.629+0000 7fbd84a44640 1 -- 172.21.15.5:0/3943540441 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd80163780 con 0x7fbd800b6620 2024-05-20T18:29:08.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.630+0000 7fbd84a44640 1 --2- 172.21.15.5:0/3943540441 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd800b6620 0x7fbd800ba3c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbd680097a0 tx=0x7fbd6802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64fad4cc0b6796cb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:08.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.630+0000 7fbd7f7fe640 1 -- 172.21.15.5:0/3943540441 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68037070 con 0x7fbd800b6620 2024-05-20T18:29:08.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.630+0000 7fbd7f7fe640 1 -- 172.21.15.5:0/3943540441 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd68032440 con 0x7fbd800b6620 2024-05-20T18:29:08.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.630+0000 7fbd7f7fe640 1 -- 172.21.15.5:0/3943540441 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68032810 con 0x7fbd800b6620 2024-05-20T18:29:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.631+0000 7fbd86ca8640 1 -- 172.21.15.5:0/3943540441 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd800b6620 msgr2=0x7fbd800ba3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.631+0000 7fbd86ca8640 1 --2- 172.21.15.5:0/3943540441 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd800b6620 0x7fbd800ba3c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbd680097a0 tx=0x7fbd6802a180 comp rx=0 tx=0).stop 2024-05-20T18:29:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.631+0000 7fbd86ca8640 1 -- 172.21.15.5:0/3943540441 shutdown_connections 2024-05-20T18:29:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.631+0000 7fbd86ca8640 1 --2- 172.21.15.5:0/3943540441 >> v2:172.21.15.63:3300/0 conn(0x7fbd800d3bf0 0x7fbd800d4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.631+0000 7fbd86ca8640 1 --2- 172.21.15.5:0/3943540441 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd800b6620 0x7fbd800ba3c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.631+0000 7fbd86ca8640 1 -- 172.21.15.5:0/3943540441 >> 172.21.15.5:0/3943540441 conn(0x7fbd8006d240 msgr2=0x7fbd8006d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.631+0000 7fbd86ca8640 1 -- 172.21.15.5:0/3943540441 shutdown_connections 2024-05-20T18:29:08.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.631+0000 7fbd86ca8640 1 -- 172.21.15.5:0/3943540441 wait complete. 2024-05-20T18:29:08.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.633+0000 7fbd86ca8640 1 Processor -- start 2024-05-20T18:29:08.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.633+0000 7fbd86ca8640 1 -- start start 2024-05-20T18:29:08.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.634+0000 7fbd86ca8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd800b6620 0x7fbd801eb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:08.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.634+0000 7fbd86ca8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd800d3bf0 0x7fbd8014afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:08.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.634+0000 7fbd86ca8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd80162ce0 0x7fbd8014b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:08.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.634+0000 7fbd86ca8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd801630f0 con 0x7fbd800d3bf0 2024-05-20T18:29:08.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.634+0000 7fbd86ca8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbd800d36d0 con 0x7fbd80162ce0 2024-05-20T18:29:08.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.634+0000 7fbd86ca8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbd801eb6f0 con 0x7fbd800b6620 2024-05-20T18:29:08.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.634+0000 7fbd7ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd800d3bf0 0x7fbd8014afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:08.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.634+0000 7fbd7ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd800d3bf0 0x7fbd8014afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42914/0 (socket says 172.21.15.5:42914) 2024-05-20T18:29:08.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.634+0000 7fbd7ffff640 1 -- 172.21.15.5:0/2671754111 learned_addr learned my addr 172.21.15.5:0/2671754111 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:08.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.635+0000 7fbd85245640 1 --2- 172.21.15.5:0/2671754111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd80162ce0 0x7fbd8014b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:08.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.635+0000 7fbd84a44640 1 --2- 172.21.15.5:0/2671754111 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd800b6620 0x7fbd801eb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:08.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.635+0000 7fbd7ffff640 1 -- 172.21.15.5:0/2671754111 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd800b6620 msgr2=0x7fbd801eb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:08.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.635+0000 7fbd7ffff640 1 --2- 172.21.15.5:0/2671754111 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd800b6620 0x7fbd801eb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.635+0000 7fbd7ffff640 1 -- 172.21.15.5:0/2671754111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd80162ce0 msgr2=0x7fbd8014b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:08.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.635+0000 7fbd7ffff640 1 --2- 172.21.15.5:0/2671754111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd80162ce0 0x7fbd8014b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.635+0000 7fbd7ffff640 1 -- 172.21.15.5:0/2671754111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd801ea110 con 0x7fbd800d3bf0 2024-05-20T18:29:08.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.635+0000 7fbd7ffff640 1 --2- 172.21.15.5:0/2671754111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd800d3bf0 0x7fbd8014afa0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000cad0 tx=0x7fbd7000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:08.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.635+0000 7fbd7d7fa640 1 -- 172.21.15.5:0/2671754111 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd70017070 con 0x7fbd800d3bf0 2024-05-20T18:29:08.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.635+0000 7fbd7d7fa640 1 -- 172.21.15.5:0/2671754111 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd7000abd0 con 0x7fbd800d3bf0 2024-05-20T18:29:08.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.635+0000 7fbd86ca8640 1 -- 172.21.15.5:0/2671754111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd801ea3f0 con 0x7fbd800d3bf0 2024-05-20T18:29:08.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.635+0000 7fbd7d7fa640 1 -- 172.21.15.5:0/2671754111 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd70012400 con 0x7fbd800d3bf0 2024-05-20T18:29:08.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.635+0000 7fbd86ca8640 1 -- 172.21.15.5:0/2671754111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd801ea9b0 con 0x7fbd800d3bf0 2024-05-20T18:29:08.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.636+0000 7fbd86ca8640 1 -- 172.21.15.5:0/2671754111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd80075b00 con 0x7fbd800d3bf0 2024-05-20T18:29:08.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.637+0000 7fbd7d7fa640 1 -- 172.21.15.5:0/2671754111 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbd7001a070 con 0x7fbd800d3bf0 2024-05-20T18:29:08.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.637+0000 7fbd7d7fa640 1 --2- 172.21.15.5:0/2671754111 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbd6006be30 0x7fbd6006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:08.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.637+0000 7fbd7d7fa640 1 -- 172.21.15.5:0/2671754111 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7fbd7008aca0 con 0x7fbd800d3bf0 2024-05-20T18:29:08.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.637+0000 7fbd84a44640 1 --2- 172.21.15.5:0/2671754111 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbd6006be30 0x7fbd6006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:08.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.638+0000 7fbd84a44640 1 --2- 172.21.15.5:0/2671754111 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbd6006be30 0x7fbd6006e340 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbd68034000 tx=0x7fbd6802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:08.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.642+0000 7fbd7d7fa640 1 -- 172.21.15.5:0/2671754111 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbd800d4540 con 0x7fbd800d3bf0 2024-05-20T18:29:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.863+0000 7fbd86ca8640 1 -- 172.21.15.5:0/2671754111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbd800cf990 con 0x7fbd800d3bf0 2024-05-20T18:29:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.864+0000 7fbd7d7fa640 1 -- 172.21.15.5:0/2671754111 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7fbd700594c0 con 0x7fbd800d3bf0 2024-05-20T18:29:08.864 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-20T18:29:08.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.866+0000 7fbd86ca8640 1 -- 172.21.15.5:0/2671754111 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbd6006be30 msgr2=0x7fbd6006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:08.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.866+0000 7fbd86ca8640 1 --2- 172.21.15.5:0/2671754111 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbd6006be30 0x7fbd6006e340 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbd68034000 tx=0x7fbd6802e000 comp rx=0 tx=0).stop 2024-05-20T18:29:08.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.866+0000 7fbd86ca8640 1 -- 172.21.15.5:0/2671754111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd800d3bf0 msgr2=0x7fbd8014afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:08.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.866+0000 7fbd86ca8640 1 --2- 172.21.15.5:0/2671754111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd800d3bf0 0x7fbd8014afa0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000cad0 tx=0x7fbd7000cf90 comp rx=0 tx=0).stop 2024-05-20T18:29:08.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.866+0000 7fbd86ca8640 1 -- 172.21.15.5:0/2671754111 shutdown_connections 2024-05-20T18:29:08.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.866+0000 7fbd86ca8640 1 --2- 172.21.15.5:0/2671754111 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbd6006be30 0x7fbd6006e340 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.866+0000 7fbd86ca8640 1 --2- 172.21.15.5:0/2671754111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd80162ce0 0x7fbd8014b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.866+0000 7fbd86ca8640 1 --2- 172.21.15.5:0/2671754111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd800d3bf0 0x7fbd8014afa0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.866+0000 7fbd86ca8640 1 --2- 172.21.15.5:0/2671754111 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd800b6620 0x7fbd801eb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:08.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.866+0000 7fbd86ca8640 1 -- 172.21.15.5:0/2671754111 >> 172.21.15.5:0/2671754111 conn(0x7fbd8006d240 msgr2=0x7fbd800b5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:08.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.866+0000 7fbd86ca8640 1 -- 172.21.15.5:0/2671754111 shutdown_connections 2024-05-20T18:29:08.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:08.866+0000 7fbd86ca8640 1 -- 172.21.15.5:0/2671754111 wait complete. 2024-05-20T18:29:08.877 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-20T18:29:08.877 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-20T18:29:08.877 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-05-20T18:29:08.877 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-20T18:29:08.877 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-20T18:29:08.877 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-20T18:29:08.877 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-20T18:29:08.877 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-20T18:29:08.877 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:29:08.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:29:08.878 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-05-20T18:29:08.894 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T18:29:08.894 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T18:29:08.894 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T18:29:08.895 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:29:08.895 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-20T18:29:08.895 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:29:08.895 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:29:08.895 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-05-20T18:29:08.895 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-05-20T18:29:08.895 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:29:08.895 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-05-20T18:29:08.895 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-05-20T18:29:08.895 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-20T18:29:08.895 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:29:08.895 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-05-20T18:29:08.895 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-20T18:29:08.896 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42549sec preferred_lft 42549sec 2024-05-20T18:29:08.896 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-05-20T18:29:08.896 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:29:08.896 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:29:08.896 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:29:08.896 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-05-20T18:29:08.896 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-20T18:29:08.896 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:29:08.896 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-20T18:29:08.896 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:29:08.896 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::4474:eaff:fe07:4f9a/64 scope link 2024-05-20T18:29:08.896 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:29:08.896 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:29:08.897 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T18:29:08.897 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ce4:f4ff:fe5e:7906/64 scope link 2024-05-20T18:29:08.897 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:29:08.897 INFO:teuthology.orchestra.run.smithi005.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:29:08.897 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether da:26:04:0e:95:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T18:29:08.897 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::d826:4ff:fe0e:950a/64 scope link 2024-05-20T18:29:08.897 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:29:08.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:29:08.897 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-20T18:29:08.958 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-20T18:29:08.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:29:08.961 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-05-20T18:29:09.018 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-05-20T18:29:09.018 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-20T18:29:09.018 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-20T18:29:09.018 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-20T18:29:09.018 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-20T18:29:09.019 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:29:09.019 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-20T18:29:09.019 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-20T18:29:09.019 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-20T18:29:09.019 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-20T18:29:09.020 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-20T18:29:09.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:29:09.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:29:09.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:29:09.151 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-20T18:29:09.167 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-20T18:29:09.168 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-20T18:29:09.168 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:29:09.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:29:09.168 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-05-20T18:29:09.234 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:29:09.234 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-20T18:29:09.291 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:29:09.291 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:29:09.291 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:29:09.346 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-20T18:29:09.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:29:09.347 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:29:09.442 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:29:09.442 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:29:09.442 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:29:09.453 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:29:09.453 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:29:09.453 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:29:09.453 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:29:09.453 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:29:09.461 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:29:09.462 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:29:09.462 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:29:09.462 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:29:09.462 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:29:09.462 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790" 2024-05-20T18:29:09.462 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790:/ 2024-05-20T18:29:09.462 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=399ab250-fd3c-47fa-a1c8-25e8149c9cac" will pass to kernel 2024-05-20T18:29:09.462 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-20T18:29:09.463 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-05-20T18:29:09.463 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-20T18:29:09.463 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-05-20T18:29:09.463 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-20T18:29:09.464 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-20T18:29:09.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:29:09.464 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:29:09.491 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-20T18:29:09.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T18:29:09.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:29:09.597 INFO:teuthology.orchestra.run.smithi005.stdout:client5710 2024-05-20T18:29:09.597 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:29:09.598 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:29:09.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:29:09.614 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-05-20T18:29:09.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:29:09.675 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-05-20T18:29:09.733 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:29:09.733 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:29:09.733 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:29:09.733 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:29:09.733 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-05-20T18:29:09.733 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:29:09.840 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627777 2024-05-20T18:29:09.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:29:09.850 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:29:10.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.005+0000 7fddd983d640 1 Processor -- start 2024-05-20T18:29:10.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.005+0000 7fddd983d640 1 -- start start 2024-05-20T18:29:10.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.007+0000 7fddd983d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fddd40d10e0 0x7fddd40cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.007+0000 7fddd983d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fddd40b7210 0x7fddd40cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.007+0000 7fddd983d640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddd40d1ac0 con 0x7fddd40d14f0 2024-05-20T18:29:10.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.007+0000 7fddd983d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fddd40d1da0 con 0x7fddd40b7210 2024-05-20T18:29:10.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.007+0000 7fddd983d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fddd40d2080 con 0x7fddd40d10e0 2024-05-20T18:29:10.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.007+0000 7fddd37fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fddd40b7210 0x7fddd40cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.007+0000 7fddd37fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fddd40b7210 0x7fddd40cf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42918/0 (socket says 172.21.15.5:42918) 2024-05-20T18:29:10.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.007+0000 7fddd37fe640 1 -- 172.21.15.5:0/844862959 learned_addr learned my addr 172.21.15.5:0/844862959 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:10.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.007+0000 7fddd2ffd640 1 --2- 172.21.15.5:0/844862959 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fddd40d10e0 0x7fddd40cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.007+0000 7fddd37fe640 1 -- 172.21.15.5:0/844862959 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fddd40d10e0 msgr2=0x7fddd40cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:10.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.007+0000 7fddd37fe640 1 --2- 172.21.15.5:0/844862959 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fddd40d10e0 0x7fddd40cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.007+0000 7fddd37fe640 1 -- 172.21.15.5:0/844862959 >> v1:172.21.15.63:6789/0 conn(0x7fddd40d14f0 legacy=0x7fddd40cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:10.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.007+0000 7fddd37fe640 1 -- 172.21.15.5:0/844862959 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd40d2360 con 0x7fddd40b7210 2024-05-20T18:29:10.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.008+0000 7fddd37fe640 1 --2- 172.21.15.5:0/844862959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddd40b7210 0x7fddd40cf680 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fddcc009870 tx=0x7fddcc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d44795b1fe65da1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:10.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.008+0000 7fddd1ffb640 1 -- 172.21.15.5:0/844862959 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddcc037070 con 0x7fddd40b7210 2024-05-20T18:29:10.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.008+0000 7fddd1ffb640 1 -- 172.21.15.5:0/844862959 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddcc032440 con 0x7fddd40b7210 2024-05-20T18:29:10.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.008+0000 7fddd1ffb640 1 -- 172.21.15.5:0/844862959 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddcc032810 con 0x7fddd40b7210 2024-05-20T18:29:10.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.009+0000 7fddd983d640 1 -- 172.21.15.5:0/844862959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddd40b7210 msgr2=0x7fddd40cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.009+0000 7fddd983d640 1 --2- 172.21.15.5:0/844862959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddd40b7210 0x7fddd40cf680 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fddcc009870 tx=0x7fddcc02a210 comp rx=0 tx=0).stop 2024-05-20T18:29:10.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.009+0000 7fddd983d640 1 -- 172.21.15.5:0/844862959 shutdown_connections 2024-05-20T18:29:10.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.009+0000 7fddd983d640 1 --2- 172.21.15.5:0/844862959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddd40b7210 0x7fddd40cf680 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.009+0000 7fddd983d640 1 --2- 172.21.15.5:0/844862959 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fddd40d10e0 0x7fddd40cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.009+0000 7fddd983d640 1 -- 172.21.15.5:0/844862959 >> 172.21.15.5:0/844862959 conn(0x7fddd40b44d0 msgr2=0x7fddd40b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:10.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.009+0000 7fddd983d640 1 -- 172.21.15.5:0/844862959 shutdown_connections 2024-05-20T18:29:10.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.009+0000 7fddd983d640 1 -- 172.21.15.5:0/844862959 wait complete. 2024-05-20T18:29:10.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.011+0000 7fddd983d640 1 Processor -- start 2024-05-20T18:29:10.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.011+0000 7fddd983d640 1 -- start start 2024-05-20T18:29:10.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.012+0000 7fddd983d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fddd40b7210 0x7fddd40cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.012+0000 7fddd983d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddd40d10e0 0x7fddd40cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.012+0000 7fddd983d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fddd40d14f0 0x7fddd41f8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.012+0000 7fddd983d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fddd40b8440 con 0x7fddd40d10e0 2024-05-20T18:29:10.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.012+0000 7fddd983d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fddd40b82c0 con 0x7fddd40d14f0 2024-05-20T18:29:10.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.012+0000 7fddd983d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fddd40709f0 con 0x7fddd40b7210 2024-05-20T18:29:10.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.013+0000 7fddd27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddd40d10e0 0x7fddd40cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.013+0000 7fddd27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddd40d10e0 0x7fddd40cbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42932/0 (socket says 172.21.15.5:42932) 2024-05-20T18:29:10.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.013+0000 7fddd27fc640 1 -- 172.21.15.5:0/1744212455 learned_addr learned my addr 172.21.15.5:0/1744212455 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:10.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.013+0000 7fddd2ffd640 1 --2- 172.21.15.5:0/1744212455 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fddd40b7210 0x7fddd40cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.013+0000 7fddd37fe640 1 --2- 172.21.15.5:0/1744212455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fddd40d14f0 0x7fddd41f8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.013+0000 7fddd27fc640 1 -- 172.21.15.5:0/1744212455 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fddd40b7210 msgr2=0x7fddd40cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.013+0000 7fddd27fc640 1 --2- 172.21.15.5:0/1744212455 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fddd40b7210 0x7fddd40cb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.013+0000 7fddd27fc640 1 -- 172.21.15.5:0/1744212455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fddd40d14f0 msgr2=0x7fddd41f8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.013+0000 7fddd27fc640 1 --2- 172.21.15.5:0/1744212455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fddd40d14f0 0x7fddd41f8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.013+0000 7fddd27fc640 1 -- 172.21.15.5:0/1744212455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd41588c0 con 0x7fddd40d10e0 2024-05-20T18:29:10.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.013+0000 7fddd27fc640 1 --2- 172.21.15.5:0/1744212455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddd40d10e0 0x7fddd40cbcf0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fddc800b3e0 tx=0x7fddc8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:10.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.013+0000 7fddbbfff640 1 -- 172.21.15.5:0/1744212455 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc8015030 con 0x7fddd40d10e0 2024-05-20T18:29:10.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.013+0000 7fddbbfff640 1 -- 172.21.15.5:0/1744212455 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddc8004020 con 0x7fddd40d10e0 2024-05-20T18:29:10.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.013+0000 7fddd983d640 1 -- 172.21.15.5:0/1744212455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd4158ba0 con 0x7fddd40d10e0 2024-05-20T18:29:10.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.013+0000 7fddbbfff640 1 -- 172.21.15.5:0/1744212455 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc8004410 con 0x7fddd40d10e0 2024-05-20T18:29:10.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.013+0000 7fddd983d640 1 -- 172.21.15.5:0/1744212455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd406ded0 con 0x7fddd40d10e0 2024-05-20T18:29:10.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.014+0000 7fddd983d640 1 -- 172.21.15.5:0/1744212455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddd4068770 con 0x7fddd40d10e0 2024-05-20T18:29:10.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.015+0000 7fddbbfff640 1 -- 172.21.15.5:0/1744212455 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fddc8018070 con 0x7fddd40d10e0 2024-05-20T18:29:10.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.016+0000 7fddbbfff640 1 --2- 172.21.15.5:0/1744212455 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fddc006be30 0x7fddc006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.016+0000 7fddbbfff640 1 -- 172.21.15.5:0/1744212455 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7fddc808aa00 con 0x7fddd40d10e0 2024-05-20T18:29:10.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.016+0000 7fddd2ffd640 1 --2- 172.21.15.5:0/1744212455 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fddc006be30 0x7fddc006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.016+0000 7fddd2ffd640 1 --2- 172.21.15.5:0/1744212455 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fddc006be30 0x7fddc006e340 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fddc4003aa0 tx=0x7fddc4009830 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:10.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.020+0000 7fddbbfff640 1 -- 172.21.15.5:0/1744212455 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fddd4068770 con 0x7fddd40d10e0 2024-05-20T18:29:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.245+0000 7fddd983d640 1 -- 172.21.15.5:0/1744212455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fddd40d0f00 con 0x7fddd40d10e0 2024-05-20T18:29:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.246+0000 7fddbbfff640 1 -- 172.21.15.5:0/1744212455 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5175 (secure 0 0 0) 0x7fddd40d0f00 con 0x7fddd40d10e0 2024-05-20T18:29:10.246 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:29:10.246 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5207,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3958852803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3958852803},{"type":"v1","addr":"172.21.15.63:6835","nonce":3958852803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5560,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/3020169757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":3020169757},{"type":"v1","addr":"172.21.15.63:6834","nonce":3020169757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2965018210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2965018210},{"type":"v1","addr":"172.21.15.5:6835","nonce":2965018210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:29:02.038442+0000","modified":"2024-05-20T18:29:05.055248+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5751},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5751":{"gid":5751,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6837/3469824363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3469824363},{"type":"v1","addr":"172.21.15.5:6837","nonce":3469824363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5751,"qdb_cluster":[5751]},"id":3}]} 2024-05-20T18:29:10.246 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-05-20T18:29:10.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.248+0000 7fddd983d640 1 -- 172.21.15.5:0/1744212455 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fddc006be30 msgr2=0x7fddc006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.248+0000 7fddd983d640 1 --2- 172.21.15.5:0/1744212455 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fddc006be30 0x7fddc006e340 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fddc4003aa0 tx=0x7fddc4009830 comp rx=0 tx=0).stop 2024-05-20T18:29:10.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.248+0000 7fddd983d640 1 -- 172.21.15.5:0/1744212455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddd40d10e0 msgr2=0x7fddd40cbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.248+0000 7fddd983d640 1 --2- 172.21.15.5:0/1744212455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddd40d10e0 0x7fddd40cbcf0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fddc800b3e0 tx=0x7fddc8009f90 comp rx=0 tx=0).stop 2024-05-20T18:29:10.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.248+0000 7fddd983d640 1 -- 172.21.15.5:0/1744212455 shutdown_connections 2024-05-20T18:29:10.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.248+0000 7fddd983d640 1 --2- 172.21.15.5:0/1744212455 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fddc006be30 0x7fddc006e340 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.248+0000 7fddd983d640 1 --2- 172.21.15.5:0/1744212455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fddd40d14f0 0x7fddd41f8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.248+0000 7fddd983d640 1 --2- 172.21.15.5:0/1744212455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddd40d10e0 0x7fddd40cbcf0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.248+0000 7fddd983d640 1 --2- 172.21.15.5:0/1744212455 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fddd40b7210 0x7fddd40cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.248+0000 7fddd983d640 1 -- 172.21.15.5:0/1744212455 >> 172.21.15.5:0/1744212455 conn(0x7fddd40b44d0 msgr2=0x7fddd40d29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:10.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.249+0000 7fddd983d640 1 -- 172.21.15.5:0/1744212455 shutdown_connections 2024-05-20T18:29:10.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.249+0000 7fddd983d640 1 -- 172.21.15.5:0/1744212455 wait complete. 2024-05-20T18:29:10.261 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-05-20T18:29:10.397 INFO:teuthology.orchestra.run.smithi005.stdout:{"message":"","return_code":0} 2024-05-20T18:29:10.398 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-20T18:29:10.398 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-05-20T18:29:10.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.455+0000 7f51b5dcbf40 1 Processor -- start 2024-05-20T18:29:10.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.455+0000 7f51b5dcbf40 1 -- start start 2024-05-20T18:29:10.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.456+0000 7f51b5dcbf40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5008ff170 0x55a5008e38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.456+0000 7f51b5dcbf40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5008ff580 0x55a50098f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.456+0000 7f51b5dcbf40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a500988d40 0x55a500a24290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.456+0000 7f51b5dcbf40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a50085fc80 con 0x55a5008ff580 2024-05-20T18:29:10.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.456+0000 7f51b5dcbf40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a500863cc0 con 0x55a5008ff170 2024-05-20T18:29:10.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.456+0000 7f51b5dcbf40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55a5008feb90 con 0x55a500988d40 2024-05-20T18:29:10.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.457+0000 7f51af099640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5008ff580 0x55a50098f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.457+0000 7f51af099640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5008ff580 0x55a50098f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42958/0 (socket says 172.21.15.5:42958) 2024-05-20T18:29:10.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.457+0000 7f51af099640 1 -- 172.21.15.5:0/2308028477 learned_addr learned my addr 172.21.15.5:0/2308028477 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:10.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.457+0000 7f51af89a640 1 --2- 172.21.15.5:0/2308028477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5008ff170 0x55a5008e38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.457+0000 7f51b009b640 1 --2- 172.21.15.5:0/2308028477 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a500988d40 0x55a500a24290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.457+0000 7f51af099640 1 -- 172.21.15.5:0/2308028477 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a500988d40 msgr2=0x55a500a24290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:10.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.457+0000 7f51af099640 1 --2- 172.21.15.5:0/2308028477 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a500988d40 0x55a500a24290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.457+0000 7f51af099640 1 -- 172.21.15.5:0/2308028477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5008ff170 msgr2=0x55a5008e38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:10.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.457+0000 7f51af099640 1 --2- 172.21.15.5:0/2308028477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5008ff170 0x55a5008e38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.457+0000 7f51af099640 1 -- 172.21.15.5:0/2308028477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5008e2770 con 0x55a5008ff580 2024-05-20T18:29:10.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.457+0000 7f51af099640 1 --2- 172.21.15.5:0/2308028477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5008ff580 0x55a50098f5b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f51a4025ec0 tx=0x7f51a4051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6597f2609ef521d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:10.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.457+0000 7f51ae097640 1 -- 172.21.15.5:0/2308028477 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a4058b50 con 0x55a5008ff580 2024-05-20T18:29:10.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.457+0000 7f51ae097640 1 -- 172.21.15.5:0/2308028477 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51a4067070 con 0x55a5008ff580 2024-05-20T18:29:10.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.457+0000 7f51ae097640 1 -- 172.21.15.5:0/2308028477 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a4050400 con 0x55a5008ff580 2024-05-20T18:29:10.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.458+0000 7f51b5dcbf40 1 -- 172.21.15.5:0/2308028477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5008ff580 msgr2=0x55a50098f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.458+0000 7f51b5dcbf40 1 --2- 172.21.15.5:0/2308028477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5008ff580 0x55a50098f5b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f51a4025ec0 tx=0x7f51a4051000 comp rx=0 tx=0).stop 2024-05-20T18:29:10.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.458+0000 7f51b5dcbf40 1 -- 172.21.15.5:0/2308028477 shutdown_connections 2024-05-20T18:29:10.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.458+0000 7f51b5dcbf40 1 --2- 172.21.15.5:0/2308028477 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a500988d40 0x55a500a24290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.458+0000 7f51b5dcbf40 1 --2- 172.21.15.5:0/2308028477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5008ff580 0x55a50098f5b0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.458+0000 7f51b5dcbf40 1 --2- 172.21.15.5:0/2308028477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5008ff170 0x55a5008e38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.458+0000 7f51b5dcbf40 1 -- 172.21.15.5:0/2308028477 >> 172.21.15.5:0/2308028477 conn(0x55a5007ddac0 msgr2=0x55a50098ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:10.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.458+0000 7f51b5dcbf40 1 -- 172.21.15.5:0/2308028477 shutdown_connections 2024-05-20T18:29:10.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.458+0000 7f51b5dcbf40 1 -- 172.21.15.5:0/2308028477 wait complete. 2024-05-20T18:29:10.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.459+0000 7f51b5dcbf40 1 Processor -- start 2024-05-20T18:29:10.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.459+0000 7f51b5dcbf40 1 -- start start 2024-05-20T18:29:10.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.460+0000 7f51b5dcbf40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5008ff170 0x55a5008b7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.460+0000 7f51b5dcbf40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a5008ff580 0x55a500983980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.460+0000 7f51b5dcbf40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a500988d40 0x55a5008fa360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.460+0000 7f51b5dcbf40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a5008e3de0 con 0x55a500988d40 2024-05-20T18:29:10.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.460+0000 7f51b5dcbf40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a5008b8050 con 0x55a5008ff170 2024-05-20T18:29:10.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.460+0000 7f51b5dcbf40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55a5008fb080 con 0x55a5008ff580 2024-05-20T18:29:10.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.460+0000 7f51b009b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a500988d40 0x55a5008fa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.460+0000 7f51b009b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a500988d40 0x55a5008fa360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42974/0 (socket says 172.21.15.5:42974) 2024-05-20T18:29:10.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.460+0000 7f51b009b640 1 -- 172.21.15.5:0/1475833001 learned_addr learned my addr 172.21.15.5:0/1475833001 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:10.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.461+0000 7f51af89a640 1 --2- 172.21.15.5:0/1475833001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5008ff170 0x55a5008b7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.461+0000 7f51af099640 1 --2- 172.21.15.5:0/1475833001 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a5008ff580 0x55a500983980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.461+0000 7f51b009b640 1 -- 172.21.15.5:0/1475833001 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a5008ff580 msgr2=0x55a500983980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.461+0000 7f51b009b640 1 --2- 172.21.15.5:0/1475833001 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a5008ff580 0x55a500983980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.461+0000 7f51b009b640 1 -- 172.21.15.5:0/1475833001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5008ff170 msgr2=0x55a5008b7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.461+0000 7f51b009b640 1 --2- 172.21.15.5:0/1475833001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a5008ff170 0x55a5008b7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.461+0000 7f51b009b640 1 -- 172.21.15.5:0/1475833001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a50098d070 con 0x55a500988d40 2024-05-20T18:29:10.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.461+0000 7f51af099640 1 --2- 172.21.15.5:0/1475833001 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a5008ff580 0x55a500983980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:29:10.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.461+0000 7f51b009b640 1 --2- 172.21.15.5:0/1475833001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a500988d40 0x55a5008fa360 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f51a800de30 tx=0x7f51a800b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:10.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.461+0000 7f51ad095640 1 -- 172.21.15.5:0/1475833001 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a8014020 con 0x55a500988d40 2024-05-20T18:29:10.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.461+0000 7f51ad095640 1 -- 172.21.15.5:0/1475833001 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51a8010410 con 0x55a500988d40 2024-05-20T18:29:10.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.461+0000 7f51b5dcbf40 1 -- 172.21.15.5:0/1475833001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a50098d350 con 0x55a500988d40 2024-05-20T18:29:10.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.461+0000 7f51ad095640 1 -- 172.21.15.5:0/1475833001 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a8008030 con 0x55a500988d40 2024-05-20T18:29:10.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.461+0000 7f51b5dcbf40 1 -- 172.21.15.5:0/1475833001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5008fdf60 con 0x55a500988d40 2024-05-20T18:29:10.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.463+0000 7f51ad095640 1 -- 172.21.15.5:0/1475833001 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f51a8017020 con 0x55a500988d40 2024-05-20T18:29:10.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.463+0000 7f51ad095640 1 --2- 172.21.15.5:0/1475833001 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f519c06a840 0x7f519c06cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.463+0000 7f51ad095640 1 -- 172.21.15.5:0/1475833001 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f51a8089b70 con 0x55a500988d40 2024-05-20T18:29:10.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.463+0000 7f51af89a640 1 --2- 172.21.15.5:0/1475833001 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f519c06a840 0x7f519c06cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.463+0000 7f51b5dcbf40 1 --2- 172.21.15.5:0/1475833001 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55a5008e3f60 0x55a5008e4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.463+0000 7f51b5dcbf40 1 -- 172.21.15.5:0/1475833001 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51) -- 0x55a500983ec0 con 0x55a5008e3f60 2024-05-20T18:29:10.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.463+0000 7f51af099640 1 --2- 172.21.15.5:0/1475833001 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55a5008e3f60 0x55a5008e4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.464+0000 7f51af89a640 1 --2- 172.21.15.5:0/1475833001 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f519c06a840 0x7f519c06cd50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f51a0000f80 tx=0x7f51a0006aa0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:10.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.464+0000 7f51af099640 1 --2- 172.21.15.5:0/1475833001 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55a5008e3f60 0x55a5008e4370 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:10.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.464+0000 7f51af099640 1 -- 172.21.15.5:0/1475833001 <== osd.1 v2:172.21.15.5:6815/550664725 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7f51a4058b70 con 0x55a5008e3f60 2024-05-20T18:29:10.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.464+0000 7f51b5dcbf40 1 -- 172.21.15.5:0/1475833001 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55a5008e3f60 msgr2=0x55a5008e4370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.464+0000 7f51b5dcbf40 1 --2- 172.21.15.5:0/1475833001 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55a5008e3f60 0x55a5008e4370 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.464+0000 7f51b5dcbf40 1 -- 172.21.15.5:0/1475833001 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f519c06a840 msgr2=0x7f519c06cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.464+0000 7f51b5dcbf40 1 --2- 172.21.15.5:0/1475833001 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f519c06a840 0x7f519c06cd50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f51a0000f80 tx=0x7f51a0006aa0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.464+0000 7f51b5dcbf40 1 -- 172.21.15.5:0/1475833001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a500988d40 msgr2=0x55a5008fa360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.464+0000 7f51b5dcbf40 1 --2- 172.21.15.5:0/1475833001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a500988d40 0x55a5008fa360 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f51a800de30 tx=0x7f51a800b5e0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.464+0000 7f51b009b640 1 -- 172.21.15.5:0/1475833001 reap_dead start 2024-05-20T18:29:10.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.464+0000 7f51b5dcbf40 1 -- 172.21.15.5:0/1475833001 shutdown_connections 2024-05-20T18:29:10.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.464+0000 7f51b5dcbf40 1 -- 172.21.15.5:0/1475833001 >> 172.21.15.5:0/1475833001 conn(0x55a5007ddac0 msgr2=0x55a50098f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:10.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.464+0000 7f51b5dcbf40 1 -- 172.21.15.5:0/1475833001 shutdown_connections 2024-05-20T18:29:10.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.464+0000 7f51b5dcbf40 1 -- 172.21.15.5:0/1475833001 wait complete. 2024-05-20T18:29:10.471 INFO:teuthology.orchestra.run.smithi005.stdout:Yalphaparent_a 2024-05-20T18:29:10.471 DEBUG:teuthology.orchestra.run.smithi005:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-20T18:29:10.582 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:29:10.582 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 1099511627777, 2024-05-20T18:29:10.582 INFO:teuthology.orchestra.run.smithi005.stdout: "ancestors": [ 2024-05-20T18:29:10.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:29:10.582 INFO:teuthology.orchestra.run.smithi005.stdout: "dirino": 1099511627776, 2024-05-20T18:29:10.582 INFO:teuthology.orchestra.run.smithi005.stdout: "dname": "alpha", 2024-05-20T18:29:10.582 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 2 2024-05-20T18:29:10.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:29:10.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:29:10.582 INFO:teuthology.orchestra.run.smithi005.stdout: "dirino": 1, 2024-05-20T18:29:10.583 INFO:teuthology.orchestra.run.smithi005.stdout: "dname": "parent_a", 2024-05-20T18:29:10.583 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 4 2024-05-20T18:29:10.583 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:29:10.583 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-20T18:29:10.583 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 8, 2024-05-20T18:29:10.583 INFO:teuthology.orchestra.run.smithi005.stdout: "old_pools": [] 2024-05-20T18:29:10.583 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:29:10.583 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:29:10.586 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2024-05-20T18:29:10.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.645+0000 7faa3d150f40 1 Processor -- start 2024-05-20T18:29:10.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.645+0000 7faa3d150f40 1 -- start start 2024-05-20T18:29:10.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.646+0000 7faa3d150f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa200030f0 0x55780c750e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.646+0000 7faa3d150f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55780c752de0 0x55780c7531f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.646+0000 7faa3d150f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55780c7e21a0 0x55780c7e25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.646+0000 7faa3d150f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55780c625c80 con 0x7faa200030f0 2024-05-20T18:29:10.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.646+0000 7faa3d150f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55780c629cc0 con 0x55780c752de0 2024-05-20T18:29:10.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.646+0000 7faa3d150f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55780c681e80 con 0x55780c7e21a0 2024-05-20T18:29:10.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.646+0000 7faa36c1f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa200030f0 0x55780c750e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.646+0000 7faa36c1f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa200030f0 0x55780c750e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42996/0 (socket says 172.21.15.5:42996) 2024-05-20T18:29:10.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.646+0000 7faa36c1f640 1 -- 172.21.15.5:0/3536291869 learned_addr learned my addr 172.21.15.5:0/3536291869 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:10.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.646+0000 7faa36c1f640 1 -- 172.21.15.5:0/3536291869 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55780c7e21a0 msgr2=0x55780c7e25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:10.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.646+0000 7faa36c1f640 1 --2- 172.21.15.5:0/3536291869 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55780c7e21a0 0x55780c7e25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.646+0000 7faa36c1f640 1 -- 172.21.15.5:0/3536291869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55780c752de0 msgr2=0x55780c7531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:10.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.646+0000 7faa36c1f640 1 --2- 172.21.15.5:0/3536291869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55780c752de0 0x55780c7531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.646+0000 7faa36c1f640 1 -- 172.21.15.5:0/3536291869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55780c753730 con 0x7faa200030f0 2024-05-20T18:29:10.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.646+0000 7faa36c1f640 1 --2- 172.21.15.5:0/3536291869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa200030f0 0x55780c750e00 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7faa28006140 tx=0x7faa28006c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e96fb88825bcb0e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:10.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.647+0000 7faa3541c640 1 -- 172.21.15.5:0/3536291869 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa28003950 con 0x7faa200030f0 2024-05-20T18:29:10.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.647+0000 7faa3541c640 1 -- 172.21.15.5:0/3536291869 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa28017070 con 0x7faa200030f0 2024-05-20T18:29:10.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.647+0000 7faa3541c640 1 -- 172.21.15.5:0/3536291869 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa28003d30 con 0x7faa200030f0 2024-05-20T18:29:10.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.647+0000 7faa3d150f40 1 -- 172.21.15.5:0/3536291869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa200030f0 msgr2=0x55780c750e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.647+0000 7faa3d150f40 1 --2- 172.21.15.5:0/3536291869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa200030f0 0x55780c750e00 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7faa28006140 tx=0x7faa28006c30 comp rx=0 tx=0).stop 2024-05-20T18:29:10.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.647+0000 7faa3d150f40 1 -- 172.21.15.5:0/3536291869 shutdown_connections 2024-05-20T18:29:10.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.647+0000 7faa3d150f40 1 --2- 172.21.15.5:0/3536291869 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55780c7e21a0 0x55780c7e25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.647+0000 7faa3d150f40 1 --2- 172.21.15.5:0/3536291869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55780c752de0 0x55780c7531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.647+0000 7faa3d150f40 1 --2- 172.21.15.5:0/3536291869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa200030f0 0x55780c750e00 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.647+0000 7faa3d150f40 1 -- 172.21.15.5:0/3536291869 >> 172.21.15.5:0/3536291869 conn(0x55780c5a3ac0 msgr2=0x55780c6c9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:10.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.647+0000 7faa3d150f40 1 -- 172.21.15.5:0/3536291869 shutdown_connections 2024-05-20T18:29:10.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.647+0000 7faa3d150f40 1 -- 172.21.15.5:0/3536291869 wait complete. 2024-05-20T18:29:10.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.648+0000 7faa3d150f40 1 Processor -- start 2024-05-20T18:29:10.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.648+0000 7faa3d150f40 1 -- start start 2024-05-20T18:29:10.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.650+0000 7faa3d150f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55780c752de0 0x55780c6c36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.650+0000 7faa3d150f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55780c7e21a0 0x55780c745400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.650+0000 7faa3d150f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa200030f0 0x55780c745940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.650+0000 7faa3d150f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55780c681c30 con 0x7faa200030f0 2024-05-20T18:29:10.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.650+0000 7faa3d150f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55780c752100 con 0x55780c752de0 2024-05-20T18:29:10.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.650+0000 7faa3d150f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55780c751720 con 0x55780c7e21a0 2024-05-20T18:29:10.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.650+0000 7faa37420640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa200030f0 0x55780c745940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.650+0000 7faa37420640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa200030f0 0x55780c745940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43008/0 (socket says 172.21.15.5:43008) 2024-05-20T18:29:10.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.650+0000 7faa37420640 1 -- 172.21.15.5:0/3736111928 learned_addr learned my addr 172.21.15.5:0/3736111928 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:10.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.650+0000 7faa36c1f640 1 --2- 172.21.15.5:0/3736111928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55780c752de0 0x55780c6c36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.650+0000 7faa3641e640 1 --2- 172.21.15.5:0/3736111928 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55780c7e21a0 0x55780c745400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.650+0000 7faa37420640 1 -- 172.21.15.5:0/3736111928 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55780c7e21a0 msgr2=0x55780c745400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.650+0000 7faa37420640 1 --2- 172.21.15.5:0/3736111928 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55780c7e21a0 0x55780c745400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.650+0000 7faa37420640 1 -- 172.21.15.5:0/3736111928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55780c752de0 msgr2=0x55780c6c36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.650+0000 7faa37420640 1 --2- 172.21.15.5:0/3736111928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55780c752de0 0x55780c6c36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.650+0000 7faa37420640 1 -- 172.21.15.5:0/3736111928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55780c6c5420 con 0x7faa200030f0 2024-05-20T18:29:10.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.650+0000 7faa37420640 1 --2- 172.21.15.5:0/3736111928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa200030f0 0x55780c745940 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x55780c66c350 tx=0x7faa3000de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:10.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.650+0000 7faa1ffff640 1 -- 172.21.15.5:0/3736111928 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa300034e0 con 0x7faa200030f0 2024-05-20T18:29:10.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.650+0000 7faa1ffff640 1 -- 172.21.15.5:0/3736111928 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa30016070 con 0x7faa200030f0 2024-05-20T18:29:10.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.650+0000 7faa3d150f40 1 -- 172.21.15.5:0/3736111928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55780c753520 con 0x7faa200030f0 2024-05-20T18:29:10.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.650+0000 7faa1ffff640 1 -- 172.21.15.5:0/3736111928 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3000b970 con 0x7faa200030f0 2024-05-20T18:29:10.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.651+0000 7faa3d150f40 1 -- 172.21.15.5:0/3736111928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55780c6c2eb0 con 0x7faa200030f0 2024-05-20T18:29:10.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.652+0000 7faa1ffff640 1 -- 172.21.15.5:0/3736111928 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faa3000bb10 con 0x7faa200030f0 2024-05-20T18:29:10.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.653+0000 7faa1ffff640 1 --2- 172.21.15.5:0/3736111928 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faa2406a810 0x7faa2406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.653+0000 7faa1ffff640 1 -- 172.21.15.5:0/3736111928 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7faa30089530 con 0x7faa200030f0 2024-05-20T18:29:10.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.653+0000 7faa36c1f640 1 --2- 172.21.15.5:0/3736111928 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faa2406a810 0x7faa2406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.653+0000 7faa3d150f40 1 --2- 172.21.15.5:0/3736111928 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55780c6abbc0 0x55780c754960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.653+0000 7faa3d150f40 1 -- 172.21.15.5:0/3736111928 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51) -- 0x55780c751960 con 0x55780c6abbc0 2024-05-20T18:29:10.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.653+0000 7faa3641e640 1 --2- 172.21.15.5:0/3736111928 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55780c6abbc0 0x55780c754960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.653+0000 7faa36c1f640 1 --2- 172.21.15.5:0/3736111928 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faa2406a810 0x7faa2406cd20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7faa28002250 tx=0x7faa28008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:10.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.653+0000 7faa3641e640 1 --2- 172.21.15.5:0/3736111928 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55780c6abbc0 0x55780c754960 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:10.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.655+0000 7faa3641e640 1 -- 172.21.15.5:0/3736111928 <== osd.1 v2:172.21.15.5:6815/550664725 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v51'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7faa2c04f860 con 0x55780c6abbc0 2024-05-20T18:29:10.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.655+0000 7faa3d150f40 1 -- 172.21.15.5:0/3736111928 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55780c6abbc0 msgr2=0x55780c754960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.655+0000 7faa3d150f40 1 --2- 172.21.15.5:0/3736111928 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55780c6abbc0 0x55780c754960 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.655+0000 7faa3d150f40 1 -- 172.21.15.5:0/3736111928 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faa2406a810 msgr2=0x7faa2406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.655+0000 7faa3d150f40 1 --2- 172.21.15.5:0/3736111928 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faa2406a810 0x7faa2406cd20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7faa28002250 tx=0x7faa28008030 comp rx=0 tx=0).stop 2024-05-20T18:29:10.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.655+0000 7faa3d150f40 1 -- 172.21.15.5:0/3736111928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa200030f0 msgr2=0x55780c745940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.655+0000 7faa3d150f40 1 --2- 172.21.15.5:0/3736111928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa200030f0 0x55780c745940 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x55780c66c350 tx=0x7faa3000de60 comp rx=0 tx=0).stop 2024-05-20T18:29:10.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.655+0000 7faa37420640 1 -- 172.21.15.5:0/3736111928 reap_dead start 2024-05-20T18:29:10.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.655+0000 7faa3d150f40 1 -- 172.21.15.5:0/3736111928 shutdown_connections 2024-05-20T18:29:10.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.655+0000 7faa3d150f40 1 -- 172.21.15.5:0/3736111928 >> 172.21.15.5:0/3736111928 conn(0x55780c5a3ac0 msgr2=0x55780c6c9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:10.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.655+0000 7faa3d150f40 1 -- 172.21.15.5:0/3736111928 shutdown_connections 2024-05-20T18:29:10.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.655+0000 7faa3d150f40 1 -- 172.21.15.5:0/3736111928 wait complete. 2024-05-20T18:29:10.658 DEBUG:teuthology.orchestra.run.smithi005:> sudo daemon-helper kill ceph --cluster ceph -w 2024-05-20T18:29:10.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:29:10.663 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2024-05-20T18:29:10.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.786+0000 7f86d9b57640 1 Processor -- start 2024-05-20T18:29:10.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.786+0000 7f86d9b57640 1 -- start start 2024-05-20T18:29:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.787+0000 7f86d9b57640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f86d40d3510 0x7f86d40b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.787+0000 7f86d9b57640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f86d40cf060 0x7f86d40cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.787+0000 7f86d9b57640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86d40b4300 con 0x7f86d40cf9b0 2024-05-20T18:29:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.787+0000 7f86d9b57640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f86d40cfdc0 con 0x7f86d40d3510 2024-05-20T18:29:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.787+0000 7f86d9b57640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f86d415fac0 con 0x7f86d40cf060 2024-05-20T18:29:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.787+0000 7f86d37fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f86d40d3510 0x7f86d40b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.788+0000 7f86d37fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f86d40d3510 0x7f86d40b3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43018/0 (socket says 172.21.15.5:43018) 2024-05-20T18:29:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.788+0000 7f86d37fe640 1 -- 172.21.15.5:0/2803348170 learned_addr learned my addr 172.21.15.5:0/2803348170 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.788+0000 7f86cbfff640 1 --2- 172.21.15.5:0/2803348170 >> v2:172.21.15.63:3300/0 conn(0x7f86d40cf060 0x7f86d40cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.788+0000 7f86d37fe640 1 -- 172.21.15.5:0/2803348170 >> v2:172.21.15.63:3300/0 conn(0x7f86d40cf060 msgr2=0x7f86d40cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.788+0000 7f86d37fe640 1 --2- 172.21.15.5:0/2803348170 >> v2:172.21.15.63:3300/0 conn(0x7f86d40cf060 0x7f86d40cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.788+0000 7f86d37fe640 1 -- 172.21.15.5:0/2803348170 >> v1:172.21.15.63:6789/0 conn(0x7f86d40cf9b0 legacy=0x7f86d41631b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:10.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.788+0000 7f86d37fe640 1 -- 172.21.15.5:0/2803348170 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d406e090 con 0x7f86d40d3510 2024-05-20T18:29:10.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.789+0000 7f86d37fe640 1 --2- 172.21.15.5:0/2803348170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86d40d3510 0x7f86d40b3dc0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f86d40c64d0 tx=0x7f86c402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36967774c6a0221d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:10.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.789+0000 7f86d2ffd640 1 -- 172.21.15.5:0/2803348170 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c4037070 con 0x7f86d40d3510 2024-05-20T18:29:10.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.789+0000 7f86d2ffd640 1 -- 172.21.15.5:0/2803348170 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86c4032440 con 0x7f86d40d3510 2024-05-20T18:29:10.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.789+0000 7f86d2ffd640 1 -- 172.21.15.5:0/2803348170 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c4032810 con 0x7f86d40d3510 2024-05-20T18:29:10.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.790+0000 7f86d9b57640 1 -- 172.21.15.5:0/2803348170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86d40d3510 msgr2=0x7f86d40b3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.790+0000 7f86d9b57640 1 --2- 172.21.15.5:0/2803348170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86d40d3510 0x7f86d40b3dc0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f86d40c64d0 tx=0x7f86c402a1e0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.790+0000 7f86d9b57640 1 -- 172.21.15.5:0/2803348170 shutdown_connections 2024-05-20T18:29:10.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.790+0000 7f86d9b57640 1 --2- 172.21.15.5:0/2803348170 >> v2:172.21.15.63:3300/0 conn(0x7f86d40cf060 0x7f86d40cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.790+0000 7f86d9b57640 1 --2- 172.21.15.5:0/2803348170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86d40d3510 0x7f86d40b3dc0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.790+0000 7f86d9b57640 1 -- 172.21.15.5:0/2803348170 >> 172.21.15.5:0/2803348170 conn(0x7f86d40d29f0 msgr2=0x7f86d40d2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:10.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.790+0000 7f86d9b57640 1 -- 172.21.15.5:0/2803348170 shutdown_connections 2024-05-20T18:29:10.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.790+0000 7f86d9b57640 1 -- 172.21.15.5:0/2803348170 wait complete. 2024-05-20T18:29:10.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.792+0000 7f86d9b57640 1 Processor -- start 2024-05-20T18:29:10.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.792+0000 7f86d9b57640 1 -- start start 2024-05-20T18:29:10.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.793+0000 7f86d9b57640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86d40cf060 0x7f86d415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.793+0000 7f86d9b57640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86d40cf9b0 0x7f86d41f5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.793+0000 7f86d9b57640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86d40d3510 0x7f86d41f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.793+0000 7f86d9b57640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f86d40d1af0 con 0x7f86d40cf060 2024-05-20T18:29:10.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.793+0000 7f86d9b57640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f86d40d1780 con 0x7f86d40d3510 2024-05-20T18:29:10.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.793+0000 7f86d9b57640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f86d406dcb0 con 0x7f86d40cf9b0 2024-05-20T18:29:10.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.793+0000 7f86d37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86d40cf060 0x7f86d415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.793+0000 7f86d37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86d40cf060 0x7f86d415bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43028/0 (socket says 172.21.15.5:43028) 2024-05-20T18:29:10.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.793+0000 7f86d37fe640 1 -- 172.21.15.5:0/3616523198 learned_addr learned my addr 172.21.15.5:0/3616523198 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:10.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.794+0000 7f86cbfff640 1 --2- 172.21.15.5:0/3616523198 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86d40cf9b0 0x7f86d41f5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.794+0000 7f86d3fff640 1 --2- 172.21.15.5:0/3616523198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86d40d3510 0x7f86d41f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.794+0000 7f86d37fe640 1 -- 172.21.15.5:0/3616523198 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86d40cf9b0 msgr2=0x7f86d41f5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.794+0000 7f86d37fe640 1 --2- 172.21.15.5:0/3616523198 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86d40cf9b0 0x7f86d41f5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.794+0000 7f86d37fe640 1 -- 172.21.15.5:0/3616523198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86d40d3510 msgr2=0x7f86d41f5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.794+0000 7f86d37fe640 1 --2- 172.21.15.5:0/3616523198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86d40d3510 0x7f86d41f5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.794+0000 7f86d37fe640 1 -- 172.21.15.5:0/3616523198 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d41f5fc0 con 0x7f86d40cf060 2024-05-20T18:29:10.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.794+0000 7f86d37fe640 1 --2- 172.21.15.5:0/3616523198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86d40cf060 0x7f86d415bcd0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f86c402a220 tx=0x7f86c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:10.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.794+0000 7f86d0ff9640 1 -- 172.21.15.5:0/3616523198 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c4041020 con 0x7f86d40cf060 2024-05-20T18:29:10.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.794+0000 7f86d0ff9640 1 -- 172.21.15.5:0/3616523198 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86c40306c0 con 0x7f86d40cf060 2024-05-20T18:29:10.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.794+0000 7f86d9b57640 1 -- 172.21.15.5:0/3616523198 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86d409dd10 con 0x7f86d40cf060 2024-05-20T18:29:10.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.794+0000 7f86d0ff9640 1 -- 172.21.15.5:0/3616523198 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c4037040 con 0x7f86d40cf060 2024-05-20T18:29:10.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.794+0000 7f86d9b57640 1 -- 172.21.15.5:0/3616523198 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86d4161be0 con 0x7f86d40cf060 2024-05-20T18:29:10.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.797+0000 7f86d0ff9640 1 -- 172.21.15.5:0/3616523198 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f86c4044020 con 0x7f86d40cf060 2024-05-20T18:29:10.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.797+0000 7f86d0ff9640 1 --2- 172.21.15.5:0/3616523198 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f86b406be30 0x7f86b406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.797+0000 7f86d0ff9640 1 -- 172.21.15.5:0/3616523198 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f86c40ab7a0 con 0x7f86d40cf060 2024-05-20T18:29:10.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.797+0000 7f86cbfff640 1 --2- 172.21.15.5:0/3616523198 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f86b406be30 0x7f86b406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.798+0000 7f86e0a00740 1 Processor -- start 2024-05-20T18:29:10.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.798+0000 7f86e0a00740 1 -- start start 2024-05-20T18:29:10.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.798+0000 7f86cbfff640 1 --2- 172.21.15.5:0/3616523198 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f86b406be30 0x7f86b406e340 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f86d415eb60 tx=0x7f86bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:10.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.799+0000 7f86e0a00740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f2ea675410 0x55f2eaa40430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.799+0000 7f86e0a00740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2ea64c550 0x55f2eaa5b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.799+0000 7f86e0a00740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2ea654730 0x55f2eaa5b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.799+0000 7f86e0a00740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f2ea979990 con 0x55f2ea654730 2024-05-20T18:29:10.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.799+0000 7f86e0a00740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f2ea9215b0 con 0x55f2ea64c550 2024-05-20T18:29:10.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.799+0000 7f86e0a00740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55f2ea975950 con 0x55f2ea675410 2024-05-20T18:29:10.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.799+0000 7f86d3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2ea654730 0x55f2eaa5b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.799+0000 7f86d3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2ea654730 0x55f2eaa5b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43042/0 (socket says 172.21.15.5:43042) 2024-05-20T18:29:10.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.799+0000 7f86d3fff640 1 -- 172.21.15.5:0/1121452751 learned_addr learned my addr 172.21.15.5:0/1121452751 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:10.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.799+0000 7f86cbfff640 1 --2- 172.21.15.5:0/1121452751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2ea64c550 0x55f2eaa5b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.800+0000 7f86d37fe640 1 --2- 172.21.15.5:0/1121452751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f2ea675410 0x55f2eaa40430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.800+0000 7f86d3fff640 1 -- 172.21.15.5:0/1121452751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f2ea675410 msgr2=0x55f2eaa40430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:10.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.800+0000 7f86d3fff640 1 --2- 172.21.15.5:0/1121452751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f2ea675410 0x55f2eaa40430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.800+0000 7f86d3fff640 1 -- 172.21.15.5:0/1121452751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2ea64c550 msgr2=0x55f2eaa5b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.800+0000 7f86d3fff640 1 --2- 172.21.15.5:0/1121452751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2ea64c550 0x55f2eaa5b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.800+0000 7f86d3fff640 1 -- 172.21.15.5:0/1121452751 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2ea7557e0 con 0x55f2ea654730 2024-05-20T18:29:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.800+0000 7f86d3fff640 1 --2- 172.21.15.5:0/1121452751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2ea654730 0x55f2eaa5b6a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f86cc001e90 tx=0x7f86cc0015d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4b24189f34c02c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.800+0000 7f86c97fa640 1 -- 172.21.15.5:0/1121452751 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc0020a0 con 0x55f2ea654730 2024-05-20T18:29:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.800+0000 7f86c97fa640 1 -- 172.21.15.5:0/1121452751 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86cc018070 con 0x55f2ea654730 2024-05-20T18:29:10.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.800+0000 7f86c97fa640 1 -- 172.21.15.5:0/1121452751 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc003b70 con 0x55f2ea654730 2024-05-20T18:29:10.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.801+0000 7f86e0a00740 1 -- 172.21.15.5:0/1121452751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2ea654730 msgr2=0x55f2eaa5b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.801+0000 7f86e0a00740 1 --2- 172.21.15.5:0/1121452751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2ea654730 0x55f2eaa5b6a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f86cc001e90 tx=0x7f86cc0015d0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.801+0000 7f86e0a00740 1 -- 172.21.15.5:0/1121452751 shutdown_connections 2024-05-20T18:29:10.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.801+0000 7f86e0a00740 1 --2- 172.21.15.5:0/1121452751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2ea654730 0x55f2eaa5b6a0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.801+0000 7f86e0a00740 1 --2- 172.21.15.5:0/1121452751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2ea64c550 0x55f2eaa5b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.801+0000 7f86e0a00740 1 --2- 172.21.15.5:0/1121452751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f2ea675410 0x55f2eaa40430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.801+0000 7f86e0a00740 1 -- 172.21.15.5:0/1121452751 >> 172.21.15.5:0/1121452751 conn(0x55f2ea669560 msgr2=0x55f2eaa15a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:10.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.801+0000 7f86e0a00740 1 -- 172.21.15.5:0/1121452751 shutdown_connections 2024-05-20T18:29:10.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.801+0000 7f86e0a00740 1 -- 172.21.15.5:0/1121452751 wait complete. 2024-05-20T18:29:10.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.802+0000 7f86e0a00740 1 -- start start 2024-05-20T18:29:10.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.802+0000 7f86e0a00740 1 Processor -- start 2024-05-20T18:29:10.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.803+0000 7f86e0a00740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2ea64c550 0x55f2eaa31b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.804+0000 7f86e0a00740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2ea654730 0x55f2eaa5a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.804+0000 7f86e0a00740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f2ea675410 0x55f2eaa6d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.804+0000 7f86e0a00740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f2ea934b40 con 0x55f2ea64c550 2024-05-20T18:29:10.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.804+0000 7f86e0a00740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f2eaa32080 con 0x55f2ea654730 2024-05-20T18:29:10.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.804+0000 7f86e0a00740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55f2eaa44100 con 0x55f2ea675410 2024-05-20T18:29:10.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.804+0000 7f86e0a00740 10 client.0 _ll_register_callbacks cb 0x55f2ea9290f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:29:10.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.804+0000 7f86d37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2ea64c550 0x55f2eaa31b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.804+0000 7f86d37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2ea64c550 0x55f2eaa31b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43052/0 (socket says 172.21.15.5:43052) 2024-05-20T18:29:10.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.804+0000 7f86d37fe640 1 -- 172.21.15.5:0/3092903847 learned_addr learned my addr 172.21.15.5:0/3092903847 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:10.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.804+0000 7f86cbfff640 1 --2- 172.21.15.5:0/3092903847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2ea654730 0x55f2eaa5a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.804+0000 7f86d3fff640 1 --2- 172.21.15.5:0/3092903847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f2ea675410 0x55f2eaa6d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.804+0000 7f86d37fe640 1 -- 172.21.15.5:0/3092903847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f2ea675410 msgr2=0x55f2eaa6d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:10.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.804+0000 7f86d37fe640 1 --2- 172.21.15.5:0/3092903847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f2ea675410 0x55f2eaa6d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.804+0000 7f86d37fe640 1 -- 172.21.15.5:0/3092903847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2ea654730 msgr2=0x55f2eaa5a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:10.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.804+0000 7f86d37fe640 1 --2- 172.21.15.5:0/3092903847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2ea654730 0x55f2eaa5a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.804+0000 7f86d37fe640 1 -- 172.21.15.5:0/3092903847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2ea7b5f70 con 0x55f2ea64c550 2024-05-20T18:29:10.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.804+0000 7f86d37fe640 1 --2- 172.21.15.5:0/3092903847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2ea64c550 0x55f2eaa31b40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f86c4079870 tx=0x7f86c40799a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b2889d1f868e9d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:10.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.804+0000 7f86b27fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:29:10.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.804+0000 7f86b27fc640 1 -- 172.21.15.5:0/3092903847 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c40a8060 con 0x55f2ea64c550 2024-05-20T18:29:10.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.804+0000 7f86b27fc640 1 -- 172.21.15.5:0/3092903847 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f86c4082210 con 0x55f2ea64c550 2024-05-20T18:29:10.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.804+0000 7f86e0a00740 1 -- 172.21.15.5:0/3092903847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86cc018070 con 0x55f2ea64c550 2024-05-20T18:29:10.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.805+0000 7f86b27fc640 1 -- 172.21.15.5:0/3092903847 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f86c407ab60 con 0x55f2ea64c550 2024-05-20T18:29:10.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.805+0000 7f86e0a00740 10 client.5928 fetch_fsmap learned FSMap version 26 2024-05-20T18:29:10.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.805+0000 7f86e0a00740 1 -- 172.21.15.5:0/3092903847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55f2ea64d170 con 0x55f2ea64c550 2024-05-20T18:29:10.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.805+0000 7f86b27fc640 1 -- 172.21.15.5:0/3092903847 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86cc018070 con 0x55f2ea64c550 2024-05-20T18:29:10.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.805+0000 7f86b27fc640 1 -- 172.21.15.5:0/3092903847 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f86c403a030 con 0x55f2ea64c550 2024-05-20T18:29:10.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.805+0000 7f86e0a00740 10 client.5928 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T18:29:10.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.805+0000 7f86e0a00740 10 client.5928 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-20T18:29:10.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.805+0000 7f86e0a00740 20 client.5928 populate_metadata read hostname 'smithi005' 2024-05-20T18:29:10.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.805+0000 7f86e0a00740 1 --2- 172.21.15.5:0/3092903847 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x55f2ea6341f0 0x55f2eaa42dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.805+0000 7f86e0a00740 4 client.5928 mds_command: new command op to 5751 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:29:10.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.805+0000 7f86e0a00740 1 -- 172.21.15.5:0/3092903847 --> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2ea9870c0 con 0x55f2ea6341f0 2024-05-20T18:29:10.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.805+0000 7f86cbfff640 1 --2- 172.21.15.5:0/3092903847 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x55f2ea6341f0 0x55f2eaa42dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.806+0000 7f86b27fc640 10 client.5928 ms_handle_connect on v2:172.21.15.5:6836/3469824363 2024-05-20T18:29:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.806+0000 7f86cbfff640 1 --2- 172.21.15.5:0/3092903847 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x55f2ea6341f0 0x55f2eaa42dc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=988d22a5d1895e02 server_cookie=e402ddc4605322ac in_seq=0 out_seq=0 2024-05-20T18:29:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.807+0000 7f86b27fc640 1 -- 172.21.15.5:0/3092903847 <== mds.0 v2:172.21.15.5:6836/3469824363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f2ea9870c0 con 0x55f2ea6341f0 2024-05-20T18:29:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.807+0000 7f86b27fc640 10 client.5928 handle_command_reply: tid=0 2024-05-20T18:29:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.807+0000 7f86e0a00740 1 client.5928 shutdown 2024-05-20T18:29:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.807+0000 7f86e0a00740 1 -- 172.21.15.5:0/3092903847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2ea64c550 msgr2=0x55f2eaa31b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.807+0000 7f86e0a00740 1 --2- 172.21.15.5:0/3092903847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2ea64c550 0x55f2eaa31b40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f86c4079870 tx=0x7f86c40799a0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.807+0000 7f86e0a00740 1 -- 172.21.15.5:0/3092903847 shutdown_connections 2024-05-20T18:29:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.807+0000 7f86e0a00740 1 --2- 172.21.15.5:0/3092903847 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x55f2ea6341f0 0x55f2eaa42dc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.807+0000 7f86b27fc640 0 client.5928 ms_handle_reset on v2:172.21.15.5:6836/3469824363 2024-05-20T18:29:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.807+0000 7f86e0a00740 1 --2- 172.21.15.5:0/3092903847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f2ea675410 0x55f2eaa6d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.807+0000 7f86e0a00740 1 --2- 172.21.15.5:0/3092903847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2ea654730 0x55f2eaa5a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.807+0000 7f86e0a00740 1 --2- 172.21.15.5:0/3092903847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2ea64c550 0x55f2eaa31b40 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.807+0000 7f86e0a00740 1 -- 172.21.15.5:0/3092903847 >> 172.21.15.5:0/3092903847 conn(0x55f2ea669560 msgr2=0x55f2eaa15a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:10.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.807+0000 7f86e0a00740 1 -- 172.21.15.5:0/3092903847 shutdown_connections 2024-05-20T18:29:10.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.807+0000 7f86e0a00740 1 -- 172.21.15.5:0/3092903847 wait complete. 2024-05-20T18:29:10.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.807+0000 7f86e0a00740 20 client.5928 trim_cache size 0 max 16384 2024-05-20T18:29:10.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.808+0000 7f563bc7a640 1 Processor -- start 2024-05-20T18:29:10.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.808+0000 7f563bc7a640 1 -- start start 2024-05-20T18:29:10.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.810+0000 7f563bc7a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5634159b90 0x7f563415c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.810+0000 7f563bc7a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56340cfc60 0x7f56340cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.810+0000 7f563bc7a640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56340d05b0 con 0x7f56340d0070 2024-05-20T18:29:10.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.810+0000 7f563bc7a640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f56340d0890 con 0x7f5634159b90 2024-05-20T18:29:10.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.810+0000 7f563bc7a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f56340d0b70 con 0x7f56340cfc60 2024-05-20T18:29:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.810+0000 7f5639a16640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5634159b90 0x7f563415c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.810+0000 7f5639215640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56340cfc60 0x7f56340cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.810+0000 7f563a217640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f56340d0070 0x7f563406bbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:47232/0 (socket says 172.21.15.5:47232) 2024-05-20T18:29:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.810+0000 7f563a217640 1 -- 172.21.15.5:0/971676502 learned_addr learned my addr 172.21.15.5:0/971676502 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.810+0000 7f5639a16640 1 -- 172.21.15.5:0/971676502 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56340cfc60 msgr2=0x7f56340cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.810+0000 7f5639a16640 1 --2- 172.21.15.5:0/971676502 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56340cfc60 0x7f56340cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.810+0000 7f5639a16640 1 -- 172.21.15.5:0/971676502 >> v1:172.21.15.63:6789/0 conn(0x7f56340d0070 legacy=0x7f563406bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.810+0000 7f5639a16640 1 -- 172.21.15.5:0/971676502 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f563406b6a0 con 0x7f5634159b90 2024-05-20T18:29:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.810+0000 7f5638a14640 1 -- 172.21.15.5:0/971676502 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2795872145 0 0) 0x7f56340d05b0 con 0x7f56340d0070 2024-05-20T18:29:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.810+0000 7f5639215640 1 --2- 172.21.15.5:0/971676502 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56340cfc60 0x7f56340cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:29:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.811+0000 7f5639a16640 1 --2- 172.21.15.5:0/971676502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5634159b90 0x7f563415c460 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f56300097a0 tx=0x7f563002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cac4b2814436fcc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.811+0000 7f5638a14640 1 -- 172.21.15.5:0/971676502 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630037070 con 0x7f5634159b90 2024-05-20T18:29:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.811+0000 7f5638a14640 1 -- 172.21.15.5:0/971676502 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5630032440 con 0x7f5634159b90 2024-05-20T18:29:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.811+0000 7f5638a14640 1 -- 172.21.15.5:0/971676502 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630032810 con 0x7f5634159b90 2024-05-20T18:29:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.812+0000 7f563bc7a640 1 -- 172.21.15.5:0/971676502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5634159b90 msgr2=0x7f563415c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.812+0000 7f563bc7a640 1 --2- 172.21.15.5:0/971676502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5634159b90 0x7f563415c460 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f56300097a0 tx=0x7f563002a180 comp rx=0 tx=0).stop 2024-05-20T18:29:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.812+0000 7f563bc7a640 1 -- 172.21.15.5:0/971676502 shutdown_connections 2024-05-20T18:29:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.812+0000 7f563bc7a640 1 --2- 172.21.15.5:0/971676502 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56340cfc60 0x7f56340cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.812+0000 7f563bc7a640 1 --2- 172.21.15.5:0/971676502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5634159b90 0x7f563415c460 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.812+0000 7f563bc7a640 1 -- 172.21.15.5:0/971676502 >> 172.21.15.5:0/971676502 conn(0x7f56340d1090 msgr2=0x7f56340d14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.812+0000 7f563bc7a640 1 -- 172.21.15.5:0/971676502 shutdown_connections 2024-05-20T18:29:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.812+0000 7f563bc7a640 1 -- 172.21.15.5:0/971676502 wait complete. 2024-05-20T18:29:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.814+0000 7f563bc7a640 1 Processor -- start 2024-05-20T18:29:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.814+0000 7f563bc7a640 1 -- start start 2024-05-20T18:29:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.815+0000 7f563bc7a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56340cfc60 0x7f563414e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.815+0000 7f563bc7a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56340d0070 0x7f563415aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.815+0000 7f563bc7a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5634159b90 0x7f5634151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.815+0000 7f563bc7a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56340ce4b0 con 0x7f56340d0070 2024-05-20T18:29:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.815+0000 7f563bc7a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f56340ce330 con 0x7f5634159b90 2024-05-20T18:29:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.815+0000 7f563bc7a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f56340ce630 con 0x7f56340cfc60 2024-05-20T18:29:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.816+0000 7f5639215640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56340d0070 0x7f563415aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.816+0000 7f5639215640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56340d0070 0x7f563415aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43064/0 (socket says 172.21.15.5:43064) 2024-05-20T18:29:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.816+0000 7f5639215640 1 -- 172.21.15.5:0/1733018515 learned_addr learned my addr 172.21.15.5:0/1733018515 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.816+0000 7f563a217640 1 --2- 172.21.15.5:0/1733018515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5634159b90 0x7f5634151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.816+0000 7f5639a16640 1 --2- 172.21.15.5:0/1733018515 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56340cfc60 0x7f563414e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.816+0000 7f5639215640 1 -- 172.21.15.5:0/1733018515 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56340cfc60 msgr2=0x7f563414e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.816+0000 7f5639215640 1 --2- 172.21.15.5:0/1733018515 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f56340cfc60 0x7f563414e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.816+0000 7f5639215640 1 -- 172.21.15.5:0/1733018515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5634159b90 msgr2=0x7f5634151610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.816+0000 7f5639215640 1 --2- 172.21.15.5:0/1733018515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5634159b90 0x7f5634151610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.816+0000 7f5639215640 1 -- 172.21.15.5:0/1733018515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f563414df90 con 0x7f56340d0070 2024-05-20T18:29:10.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.816+0000 7f5639215640 1 --2- 172.21.15.5:0/1733018515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56340d0070 0x7f563415aa70 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f562400e960 tx=0x7f562400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:10.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.816+0000 7f562a7fc640 1 -- 172.21.15.5:0/1733018515 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5624017070 con 0x7f56340d0070 2024-05-20T18:29:10.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.816+0000 7f562a7fc640 1 -- 172.21.15.5:0/1733018515 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f562400ca00 con 0x7f56340d0070 2024-05-20T18:29:10.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.816+0000 7f563bc7a640 1 -- 172.21.15.5:0/1733018515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f563414e270 con 0x7f56340d0070 2024-05-20T18:29:10.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.816+0000 7f562a7fc640 1 -- 172.21.15.5:0/1733018515 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562400cdd0 con 0x7f56340d0070 2024-05-20T18:29:10.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.817+0000 7f563bc7a640 1 -- 172.21.15.5:0/1733018515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5634159420 con 0x7f56340d0070 2024-05-20T18:29:10.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.817+0000 7f563bc7a640 1 -- 172.21.15.5:0/1733018515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f5634158f30 con 0x7f56340d0070 2024-05-20T18:29:10.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.819+0000 7f562a7fc640 1 -- 172.21.15.5:0/1733018515 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f562401a070 con 0x7f56340d0070 2024-05-20T18:29:10.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.819+0000 7f562a7fc640 1 --2- 172.21.15.5:0/1733018515 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f561806be30 0x7f561806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.819+0000 7f562a7fc640 1 -- 172.21.15.5:0/1733018515 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f562408b780 con 0x7f56340d0070 2024-05-20T18:29:10.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.819+0000 7f5639a16640 1 --2- 172.21.15.5:0/1733018515 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f561806be30 0x7f561806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.820+0000 7f562a7fc640 1 -- 172.21.15.5:0/1733018515 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+501 (secure 0 0 0) 0x7f562408bb50 con 0x7f56340d0070 2024-05-20T18:29:10.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.820+0000 7f563bc7a640 1 -- 172.21.15.5:0/1733018515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f56340690b0 con 0x7f56340d0070 2024-05-20T18:29:10.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.820+0000 7f5639a16640 1 --2- 172.21.15.5:0/1733018515 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f561806be30 0x7f561806e340 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5630034000 tx=0x7f563003c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:10.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.828+0000 7f86e0a00740 1 Processor -- start 2024-05-20T18:29:10.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.828+0000 7f86e0a00740 1 -- start start 2024-05-20T18:29:10.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.829+0000 7f86e0a00740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2ea64c550 0x55f2eaa3b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.829+0000 7f86e0a00740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2ea654730 0x55f2eaa3bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.829+0000 7f86e0a00740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f2ea675410 0x55f2eaa6bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.829+0000 7f86e0a00740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f86d4202470 con 0x55f2ea64c550 2024-05-20T18:29:10.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.829+0000 7f86e0a00740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f86d4201510 con 0x55f2ea654730 2024-05-20T18:29:10.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.829+0000 7f86e0a00740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f86d409f5a0 con 0x55f2ea675410 2024-05-20T18:29:10.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.829+0000 7f86d37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2ea64c550 0x55f2eaa3b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.829+0000 7f86d37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2ea64c550 0x55f2eaa3b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43072/0 (socket says 172.21.15.5:43072) 2024-05-20T18:29:10.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.829+0000 7f86d37fe640 1 -- 172.21.15.5:0/541634559 learned_addr learned my addr 172.21.15.5:0/541634559 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:10.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.829+0000 7f86cbfff640 1 --2- 172.21.15.5:0/541634559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2ea654730 0x55f2eaa3bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.829+0000 7f86d37fe640 1 -- 172.21.15.5:0/541634559 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f2ea675410 msgr2=0x55f2eaa6bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:10.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.829+0000 7f86d3fff640 1 --2- 172.21.15.5:0/541634559 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f2ea675410 0x55f2eaa6bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.829+0000 7f86d37fe640 1 --2- 172.21.15.5:0/541634559 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f2ea675410 0x55f2eaa6bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.829+0000 7f86d37fe640 1 -- 172.21.15.5:0/541634559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2ea654730 msgr2=0x55f2eaa3bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:10.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.829+0000 7f86d37fe640 1 --2- 172.21.15.5:0/541634559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2ea654730 0x55f2eaa3bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.829+0000 7f86d37fe640 1 -- 172.21.15.5:0/541634559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2eaa5aca0 con 0x55f2ea64c550 2024-05-20T18:29:10.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.830+0000 7f86d37fe640 1 --2- 172.21.15.5:0/541634559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2ea64c550 0x55f2eaa3b9a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f86c4075d40 tx=0x7f86c407b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4889ee5f0cdad577 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:10.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.830+0000 7f86c8ff9640 1 -- 172.21.15.5:0/541634559 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c409d070 con 0x55f2ea64c550 2024-05-20T18:29:10.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.830+0000 7f86c8ff9640 1 -- 172.21.15.5:0/541634559 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86c4078fd0 con 0x55f2ea64c550 2024-05-20T18:29:10.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.830+0000 7f86c8ff9640 1 -- 172.21.15.5:0/541634559 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c4082210 con 0x55f2ea64c550 2024-05-20T18:29:10.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.831+0000 7f86e0a00740 1 -- 172.21.15.5:0/541634559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2ea64c550 msgr2=0x55f2eaa3b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.831+0000 7f86e0a00740 1 --2- 172.21.15.5:0/541634559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2ea64c550 0x55f2eaa3b9a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f86c4075d40 tx=0x7f86c407b000 comp rx=0 tx=0).stop 2024-05-20T18:29:10.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.831+0000 7f86e0a00740 1 -- 172.21.15.5:0/541634559 shutdown_connections 2024-05-20T18:29:10.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.831+0000 7f86e0a00740 1 --2- 172.21.15.5:0/541634559 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f2ea675410 0x55f2eaa6bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.831+0000 7f86e0a00740 1 --2- 172.21.15.5:0/541634559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2ea654730 0x55f2eaa3bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.831+0000 7f86e0a00740 1 --2- 172.21.15.5:0/541634559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2ea64c550 0x55f2eaa3b9a0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.831+0000 7f86e0a00740 1 -- 172.21.15.5:0/541634559 >> 172.21.15.5:0/541634559 conn(0x55f2ea669560 msgr2=0x55f2eaa161d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:10.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.831+0000 7f86e0a00740 1 -- 172.21.15.5:0/541634559 shutdown_connections 2024-05-20T18:29:10.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.831+0000 7f86e0a00740 1 -- 172.21.15.5:0/541634559 wait complete. 2024-05-20T18:29:10.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.832+0000 7f86e0a00740 1 -- start start 2024-05-20T18:29:10.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.832+0000 7f86e0a00740 1 Processor -- start 2024-05-20T18:29:10.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.833+0000 7f86e0a00740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2ea64c550 0x55f2eaa3bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.833+0000 7f86e0a00740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2ea654730 0x55f2eaa0f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.833+0000 7f86e0a00740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f2ea675410 0x55f2eaa0f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.833+0000 7f86e0a00740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f86d4200ec0 con 0x55f2ea64c550 2024-05-20T18:29:10.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.833+0000 7f86e0a00740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f2eaa303e0 con 0x55f2ea654730 2024-05-20T18:29:10.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.833+0000 7f86e0a00740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55f2eaa3ef70 con 0x55f2ea675410 2024-05-20T18:29:10.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.833+0000 7f86e0a00740 10 client.0 _ll_register_callbacks cb 0x7f86c4079420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:29:10.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.833+0000 7f86d37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2ea64c550 0x55f2eaa3bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.833+0000 7f86d37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2ea64c550 0x55f2eaa3bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43078/0 (socket says 172.21.15.5:43078) 2024-05-20T18:29:10.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.833+0000 7f86d37fe640 1 -- 172.21.15.5:0/1427457808 learned_addr learned my addr 172.21.15.5:0/1427457808 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:10.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.833+0000 7f86cbfff640 1 --2- 172.21.15.5:0/1427457808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2ea654730 0x55f2eaa0f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.833+0000 7f86d3fff640 1 --2- 172.21.15.5:0/1427457808 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f2ea675410 0x55f2eaa0f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.834+0000 7f86d37fe640 1 -- 172.21.15.5:0/1427457808 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f2ea675410 msgr2=0x55f2eaa0f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:10.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.834+0000 7f86d37fe640 1 --2- 172.21.15.5:0/1427457808 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f2ea675410 0x55f2eaa0f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.834+0000 7f86d37fe640 1 -- 172.21.15.5:0/1427457808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2ea654730 msgr2=0x55f2eaa0f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:10.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.834+0000 7f86d37fe640 1 --2- 172.21.15.5:0/1427457808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2ea654730 0x55f2eaa0f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.834+0000 7f86d37fe640 1 -- 172.21.15.5:0/1427457808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2eaa3ecc0 con 0x55f2ea64c550 2024-05-20T18:29:10.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.834+0000 7f86cbfff640 1 --2- 172.21.15.5:0/1427457808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2ea654730 0x55f2eaa0f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:29:10.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.834+0000 7f86d3fff640 1 --2- 172.21.15.5:0/1427457808 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f2ea675410 0x55f2eaa0f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:29:10.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.834+0000 7f86d37fe640 1 --2- 172.21.15.5:0/1427457808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2ea64c550 0x55f2eaa3bb90 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f86c40a9b40 tx=0x7f86c40ab040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63f65cedf6fc717 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:10.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.834+0000 7f86b27fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:29:10.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.834+0000 7f86b27fc640 1 -- 172.21.15.5:0/1427457808 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c409d070 con 0x55f2ea64c550 2024-05-20T18:29:10.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.834+0000 7f86b27fc640 1 -- 172.21.15.5:0/1427457808 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f86c40a30b0 con 0x55f2ea64c550 2024-05-20T18:29:10.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.834+0000 7f86e0a00740 1 -- 172.21.15.5:0/1427457808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86c4078fd0 con 0x55f2ea64c550 2024-05-20T18:29:10.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.834+0000 7f86b27fc640 1 -- 172.21.15.5:0/1427457808 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f86c4088050 con 0x55f2ea64c550 2024-05-20T18:29:10.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.834+0000 7f86e0a00740 10 client.5937 fetch_fsmap learned FSMap version 26 2024-05-20T18:29:10.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.834+0000 7f86e0a00740 1 -- 172.21.15.5:0/1427457808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55f2eaa3ecc0 con 0x55f2ea64c550 2024-05-20T18:29:10.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.834+0000 7f86b27fc640 1 -- 172.21.15.5:0/1427457808 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86c4078fd0 con 0x55f2ea64c550 2024-05-20T18:29:10.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.834+0000 7f86b27fc640 1 -- 172.21.15.5:0/1427457808 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f86c408b020 con 0x55f2ea64c550 2024-05-20T18:29:10.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.834+0000 7f86e0a00740 10 client.5937 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T18:29:10.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.834+0000 7f86e0a00740 10 client.5937 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-20T18:29:10.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.834+0000 7f86e0a00740 20 client.5937 populate_metadata read hostname 'smithi005' 2024-05-20T18:29:10.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.834+0000 7f86e0a00740 1 --2- 172.21.15.5:0/1427457808 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x55f2eaa157f0 0x55f2eaa0fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.834+0000 7f86e0a00740 4 client.5937 mds_command: new command op to 5751 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-05-20T18:29:10.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.834+0000 7f86e0a00740 1 -- 172.21.15.5:0/1427457808 --> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x55f2ea981410 con 0x55f2eaa157f0 2024-05-20T18:29:10.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.835+0000 7f86cbfff640 1 --2- 172.21.15.5:0/1427457808 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x55f2eaa157f0 0x55f2eaa0fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.835+0000 7f86b27fc640 10 client.5937 ms_handle_connect on v2:172.21.15.5:6836/3469824363 2024-05-20T18:29:10.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.835+0000 7f86cbfff640 1 --2- 172.21.15.5:0/1427457808 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x55f2eaa157f0 0x55f2eaa0fe50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=552a7fb6dda4eddb server_cookie=fac5381de8bd6609 in_seq=0 out_seq=0 2024-05-20T18:29:10.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.836+0000 7f86b27fc640 1 -- 172.21.15.5:0/1427457808 <== mds.0 v2:172.21.15.5:6836/3469824363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f2ea981410 con 0x55f2eaa157f0 2024-05-20T18:29:10.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.836+0000 7f86b27fc640 10 client.5937 handle_command_reply: tid=0 2024-05-20T18:29:10.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.836+0000 7f86e0a00740 1 client.5937 shutdown 2024-05-20T18:29:10.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.836+0000 7f86e0a00740 1 -- 172.21.15.5:0/1427457808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2ea64c550 msgr2=0x55f2eaa3bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.836+0000 7f86e0a00740 1 --2- 172.21.15.5:0/1427457808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2ea64c550 0x55f2eaa3bb90 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f86c40a9b40 tx=0x7f86c40ab040 comp rx=0 tx=0).stop 2024-05-20T18:29:10.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.836+0000 7f86e0a00740 1 -- 172.21.15.5:0/1427457808 shutdown_connections 2024-05-20T18:29:10.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.836+0000 7f86e0a00740 1 --2- 172.21.15.5:0/1427457808 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x55f2eaa157f0 0x55f2eaa0fe50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.836+0000 7f86b27fc640 0 client.5937 ms_handle_reset on v2:172.21.15.5:6836/3469824363 2024-05-20T18:29:10.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.836+0000 7f86e0a00740 1 --2- 172.21.15.5:0/1427457808 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f2ea675410 0x55f2eaa0f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.836+0000 7f86e0a00740 1 --2- 172.21.15.5:0/1427457808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2ea654730 0x55f2eaa0f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.836+0000 7f86e0a00740 1 --2- 172.21.15.5:0/1427457808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2ea64c550 0x55f2eaa3bb90 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.836+0000 7f86e0a00740 1 -- 172.21.15.5:0/1427457808 >> 172.21.15.5:0/1427457808 conn(0x55f2ea669560 msgr2=0x55f2eaa58a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:10.840 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-20T18:29:10.836+0000 7f56f14b2640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5710=0-4194304@1} caps={5710=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5710 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55e449fd4c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T18:29:10.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.836+0000 7f86e0a00740 1 -- 172.21.15.5:0/1427457808 shutdown_connections 2024-05-20T18:29:10.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.836+0000 7f86e0a00740 1 -- 172.21.15.5:0/1427457808 wait complete. 2024-05-20T18:29:10.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.836+0000 7f86e0a00740 20 client.5937 trim_cache size 0 max 16384 2024-05-20T18:29:10.842 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:29:10.842 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-20T18:29:10.842 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "314d484c-aaf6-43ad-8879-8755eff25ead", 2024-05-20T18:29:10.842 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-20T18:29:10.842 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:29:10.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.837+0000 7f86d9b57640 1 -- 172.21.15.5:0/3616523198 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f86b406be30 msgr2=0x7f86b406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.837+0000 7f86d9b57640 1 --2- 172.21.15.5:0/3616523198 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f86b406be30 0x7f86b406e340 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f86d415eb60 tx=0x7f86bc008040 comp rx=0 tx=0).stop 2024-05-20T18:29:10.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.837+0000 7f86d9b57640 1 -- 172.21.15.5:0/3616523198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86d40cf060 msgr2=0x7f86d415bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.837+0000 7f86d9b57640 1 --2- 172.21.15.5:0/3616523198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86d40cf060 0x7f86d415bcd0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f86c402a220 tx=0x7f86c40046c0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.837+0000 7f86d9b57640 1 -- 172.21.15.5:0/3616523198 shutdown_connections 2024-05-20T18:29:10.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.837+0000 7f86d9b57640 1 --2- 172.21.15.5:0/3616523198 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f86b406be30 0x7f86b406e340 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.837+0000 7f86d9b57640 1 --2- 172.21.15.5:0/3616523198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86d40d3510 0x7f86d41f5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.837+0000 7f86d9b57640 1 --2- 172.21.15.5:0/3616523198 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86d40cf9b0 0x7f86d41f5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.837+0000 7f86d9b57640 1 --2- 172.21.15.5:0/3616523198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86d40cf060 0x7f86d415bcd0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.837+0000 7f86d9b57640 1 -- 172.21.15.5:0/3616523198 >> 172.21.15.5:0/3616523198 conn(0x7f86d40d29f0 msgr2=0x7f86d40cd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:10.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.837+0000 7f86d9b57640 1 -- 172.21.15.5:0/3616523198 shutdown_connections 2024-05-20T18:29:10.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.837+0000 7f86d9b57640 1 -- 172.21.15.5:0/3616523198 wait complete. 2024-05-20T18:29:10.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:29:10.848 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-05-20T18:29:10.917 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:29:10.920 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:29:10.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.965+0000 7fde2febc640 1 Processor -- start 2024-05-20T18:29:10.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.965+0000 7fde2febc640 1 -- start start 2024-05-20T18:29:10.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.967+0000 7fde2febc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde28163a10 0x7fde280d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.967+0000 7fde2febc640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde280d0370 con 0x7fde280d14b0 2024-05-20T18:29:10.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.967+0000 7fde2febc640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde280d0650 con 0x7fde280b4010 2024-05-20T18:29:10.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.967+0000 7fde2febc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fde280d3ca0 con 0x7fde28163a10 2024-05-20T18:29:10.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.967+0000 7fde2e459640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fde280d14b0 0x7fde280d18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36966/0 (socket says 172.21.15.5:36966) 2024-05-20T18:29:10.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.967+0000 7fde2e459640 1 -- 172.21.15.5:0/2035588800 learned_addr learned my addr 172.21.15.5:0/2035588800 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:10.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.967+0000 7fde2dc58640 1 --2- 172.21.15.5:0/2035588800 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde28163a10 0x7fde280d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.967+0000 7fde2cc56640 1 -- 172.21.15.5:0/2035588800 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 417687150 0 0) 0x7fde280d0370 con 0x7fde280d14b0 2024-05-20T18:29:10.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.967+0000 7fde2cc56640 1 -- 172.21.15.5:0/2035588800 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde14003520 con 0x7fde280d14b0 2024-05-20T18:29:10.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.967+0000 7fde2cc56640 1 -- 172.21.15.5:0/2035588800 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 278223115 0 0) 0x7fde280d0650 con 0x7fde280b4010 2024-05-20T18:29:10.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.967+0000 7fde2cc56640 1 -- 172.21.15.5:0/2035588800 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde280d0370 con 0x7fde280b4010 2024-05-20T18:29:10.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.968+0000 7fde2dc58640 1 -- 172.21.15.5:0/2035588800 >> v1:172.21.15.63:6789/0 conn(0x7fde280b4010 legacy=0x7fde280cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.968+0000 7fde2dc58640 1 -- 172.21.15.5:0/2035588800 >> v1:172.21.15.5:6789/0 conn(0x7fde280d14b0 legacy=0x7fde280d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.968+0000 7fde2dc58640 1 -- 172.21.15.5:0/2035588800 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde280d3650 con 0x7fde28163a10 2024-05-20T18:29:10.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.968+0000 7fde2dc58640 1 --2- 172.21.15.5:0/2035588800 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde28163a10 0x7fde280d4130 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fde240097d0 tx=0x7fde2402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b1e7ca1b098b35d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:10.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.969+0000 7fde2cc56640 1 -- 172.21.15.5:0/2035588800 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde24037070 con 0x7fde28163a10 2024-05-20T18:29:10.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.969+0000 7fde2cc56640 1 -- 172.21.15.5:0/2035588800 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde24032440 con 0x7fde28163a10 2024-05-20T18:29:10.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.969+0000 7fde2cc56640 1 -- 172.21.15.5:0/2035588800 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde24032810 con 0x7fde28163a10 2024-05-20T18:29:10.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.969+0000 7fde2febc640 1 -- 172.21.15.5:0/2035588800 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde28163a10 msgr2=0x7fde280d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.969+0000 7fde2febc640 1 --2- 172.21.15.5:0/2035588800 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde28163a10 0x7fde280d4130 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fde240097d0 tx=0x7fde2402a210 comp rx=0 tx=0).stop 2024-05-20T18:29:10.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.969+0000 7fde2febc640 1 -- 172.21.15.5:0/2035588800 shutdown_connections 2024-05-20T18:29:10.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.969+0000 7fde2febc640 1 --2- 172.21.15.5:0/2035588800 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde28163a10 0x7fde280d4130 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.969+0000 7fde2febc640 1 -- 172.21.15.5:0/2035588800 >> 172.21.15.5:0/2035588800 conn(0x7fde2806d240 msgr2=0x7fde2806d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:10.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.969+0000 7fde2febc640 1 -- 172.21.15.5:0/2035588800 shutdown_connections 2024-05-20T18:29:10.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.969+0000 7fde2febc640 1 -- 172.21.15.5:0/2035588800 wait complete. 2024-05-20T18:29:10.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.971+0000 7fde2febc640 1 Processor -- start 2024-05-20T18:29:10.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.971+0000 7fde2febc640 1 -- start start 2024-05-20T18:29:10.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.972+0000 7fde2febc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde280b4010 0x7fde281ea8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.972+0000 7fde2febc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde280d14b0 0x7fde28153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.972+0000 7fde2febc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde28163a10 0x7fde280b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.972+0000 7fde2febc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fde280b8390 con 0x7fde28163a10 2024-05-20T18:29:10.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.972+0000 7fde2febc640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fde280b8210 con 0x7fde280b4010 2024-05-20T18:29:10.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.972+0000 7fde2febc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fde280b8510 con 0x7fde280d14b0 2024-05-20T18:29:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.973+0000 7fde2e459640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde28163a10 0x7fde280b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.973+0000 7fde2e459640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde28163a10 0x7fde280b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43092/0 (socket says 172.21.15.5:43092) 2024-05-20T18:29:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.973+0000 7fde2e459640 1 -- 172.21.15.5:0/3370240250 learned_addr learned my addr 172.21.15.5:0/3370240250 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.973+0000 7fde2dc58640 1 --2- 172.21.15.5:0/3370240250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde280b4010 0x7fde281ea8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.973+0000 7fde2d457640 1 --2- 172.21.15.5:0/3370240250 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde280d14b0 0x7fde28153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.973+0000 7fde2e459640 1 -- 172.21.15.5:0/3370240250 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde280d14b0 msgr2=0x7fde28153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.973+0000 7fde2e459640 1 --2- 172.21.15.5:0/3370240250 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde280d14b0 0x7fde28153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.973+0000 7fde2e459640 1 -- 172.21.15.5:0/3370240250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde280b4010 msgr2=0x7fde281ea8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.973+0000 7fde2e459640 1 --2- 172.21.15.5:0/3370240250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde280b4010 0x7fde281ea8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.973+0000 7fde2e459640 1 -- 172.21.15.5:0/3370240250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde281e8c00 con 0x7fde28163a10 2024-05-20T18:29:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.973+0000 7fde2e459640 1 --2- 172.21.15.5:0/3370240250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde28163a10 0x7fde280b6390 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fde2000caf0 tx=0x7fde2000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.973+0000 7fde1e7fc640 1 -- 172.21.15.5:0/3370240250 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde20017070 con 0x7fde28163a10 2024-05-20T18:29:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.973+0000 7fde1e7fc640 1 -- 172.21.15.5:0/3370240250 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde2000aca0 con 0x7fde28163a10 2024-05-20T18:29:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.973+0000 7fde2febc640 1 -- 172.21.15.5:0/3370240250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde281eb240 con 0x7fde28163a10 2024-05-20T18:29:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.973+0000 7fde1e7fc640 1 -- 172.21.15.5:0/3370240250 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde20012620 con 0x7fde28163a10 2024-05-20T18:29:10.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.973+0000 7fde2febc640 1 -- 172.21.15.5:0/3370240250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde281eb800 con 0x7fde28163a10 2024-05-20T18:29:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.975+0000 7fde1e7fc640 1 -- 172.21.15.5:0/3370240250 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fde2001a070 con 0x7fde28163a10 2024-05-20T18:29:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.976+0000 7fde1e7fc640 1 --2- 172.21.15.5:0/3370240250 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fde0c06be30 0x7fde0c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.976+0000 7fde1e7fc640 1 -- 172.21.15.5:0/3370240250 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7fde2008ac50 con 0x7fde28163a10 2024-05-20T18:29:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.976+0000 7fde2dc58640 1 --2- 172.21.15.5:0/3370240250 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fde0c06be30 0x7fde0c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.976+0000 7fde2dc58640 1 --2- 172.21.15.5:0/3370240250 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fde0c06be30 0x7fde0c06e340 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fde24034000 tx=0x7fde2402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:10.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.977+0000 7fde369fd740 1 Processor -- start 2024-05-20T18:29:10.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.977+0000 7fde369fd740 1 -- start start 2024-05-20T18:29:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.978+0000 7fde369fd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9fdc5390 0x55ffa018fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.978+0000 7fde369fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9fd9c4d0 0x55ffa01aa890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.978+0000 7fde369fd740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9fda46b0 0x55ffa01aadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.978+0000 7fde369fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ffa00c9d00 con 0x55ff9fd9c4d0 2024-05-20T18:29:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.978+0000 7fde369fd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ffa00714c0 con 0x55ff9fdc5390 2024-05-20T18:29:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.978+0000 7fde369fd740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55ffa00c5cc0 con 0x55ff9fda46b0 2024-05-20T18:29:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.978+0000 7fde2d457640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9fd9c4d0 0x55ffa01aa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.978+0000 7fde2d457640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9fd9c4d0 0x55ffa01aa890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43100/0 (socket says 172.21.15.5:43100) 2024-05-20T18:29:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.978+0000 7fde2d457640 1 -- 172.21.15.5:0/3658483418 learned_addr learned my addr 172.21.15.5:0/3658483418 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:10.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.978+0000 7fde2dc58640 1 --2- 172.21.15.5:0/3658483418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9fdc5390 0x55ffa018fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.978+0000 7fde2e459640 1 --2- 172.21.15.5:0/3658483418 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9fda46b0 0x55ffa01aadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.979+0000 7fde2d457640 1 -- 172.21.15.5:0/3658483418 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9fda46b0 msgr2=0x55ffa01aadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.979+0000 7fde2d457640 1 --2- 172.21.15.5:0/3658483418 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9fda46b0 0x55ffa01aadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.979+0000 7fde2d457640 1 -- 172.21.15.5:0/3658483418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9fdc5390 msgr2=0x55ffa018fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.979+0000 7fde2d457640 1 --2- 172.21.15.5:0/3658483418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9fdc5390 0x55ffa018fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.979+0000 7fde2d457640 1 -- 172.21.15.5:0/3658483418 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff9fea77d0 con 0x55ff9fd9c4d0 2024-05-20T18:29:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.979+0000 7fde2d457640 1 --2- 172.21.15.5:0/3658483418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9fd9c4d0 0x55ffa01aa890 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fde18001710 tx=0x7fde18007cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f812bd7acc648727 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.979+0000 7fde0b7fe640 1 -- 172.21.15.5:0/3658483418 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde180052e0 con 0x55ff9fd9c4d0 2024-05-20T18:29:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.979+0000 7fde0b7fe640 1 -- 172.21.15.5:0/3658483418 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde18018070 con 0x55ff9fd9c4d0 2024-05-20T18:29:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.979+0000 7fde0b7fe640 1 -- 172.21.15.5:0/3658483418 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1800f9b0 con 0x55ff9fd9c4d0 2024-05-20T18:29:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.980+0000 7fde369fd740 1 -- 172.21.15.5:0/3658483418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9fd9c4d0 msgr2=0x55ffa01aa890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.980+0000 7fde369fd740 1 --2- 172.21.15.5:0/3658483418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9fd9c4d0 0x55ffa01aa890 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fde18001710 tx=0x7fde18007cc0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.980+0000 7fde369fd740 1 -- 172.21.15.5:0/3658483418 shutdown_connections 2024-05-20T18:29:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.980+0000 7fde369fd740 1 --2- 172.21.15.5:0/3658483418 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9fda46b0 0x55ffa01aadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.980+0000 7fde369fd740 1 --2- 172.21.15.5:0/3658483418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9fd9c4d0 0x55ffa01aa890 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.980+0000 7fde369fd740 1 --2- 172.21.15.5:0/3658483418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9fdc5390 0x55ffa018fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.980+0000 7fde369fd740 1 -- 172.21.15.5:0/3658483418 >> 172.21.15.5:0/3658483418 conn(0x55ff9fdb94e0 msgr2=0x55ffa01651c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.980+0000 7fde369fd740 1 -- 172.21.15.5:0/3658483418 shutdown_connections 2024-05-20T18:29:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.980+0000 7fde369fd740 1 -- 172.21.15.5:0/3658483418 wait complete. 2024-05-20T18:29:10.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.981+0000 7fde369fd740 1 -- start start 2024-05-20T18:29:10.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.981+0000 7fde369fd740 1 Processor -- start 2024-05-20T18:29:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.982+0000 7fde369fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9fd9c4d0 0x55ffa018b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.982+0000 7fde369fd740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9fda46b0 0x55ffa01a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.982+0000 7fde369fd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9fdc5390 0x55ffa01a3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.982+0000 7fde369fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ffa0084b90 con 0x55ff9fd9c4d0 2024-05-20T18:29:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.982+0000 7fde369fd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ffa01a57e0 con 0x55ff9fdc5390 2024-05-20T18:29:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.982+0000 7fde369fd740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55ffa018bed0 con 0x55ff9fda46b0 2024-05-20T18:29:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.982+0000 7fde369fd740 10 client.0 _ll_register_callbacks cb 0x55ffa00c0da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:29:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.983+0000 7fde2dc58640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9fd9c4d0 0x55ffa018b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.983+0000 7fde2dc58640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9fd9c4d0 0x55ffa018b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43108/0 (socket says 172.21.15.5:43108) 2024-05-20T18:29:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.983+0000 7fde2dc58640 1 -- 172.21.15.5:0/530478296 learned_addr learned my addr 172.21.15.5:0/530478296 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.983+0000 7fde2e459640 1 --2- 172.21.15.5:0/530478296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9fdc5390 0x55ffa01a3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.983+0000 7fde2d457640 1 --2- 172.21.15.5:0/530478296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9fda46b0 0x55ffa01a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.983+0000 7fde2dc58640 1 -- 172.21.15.5:0/530478296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9fda46b0 msgr2=0x55ffa01a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.983+0000 7fde2dc58640 1 --2- 172.21.15.5:0/530478296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9fda46b0 0x55ffa01a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.983+0000 7fde2dc58640 1 -- 172.21.15.5:0/530478296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9fdc5390 msgr2=0x55ffa01a3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.983+0000 7fde2dc58640 1 --2- 172.21.15.5:0/530478296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9fdc5390 0x55ffa01a3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.983+0000 7fde2dc58640 1 -- 172.21.15.5:0/530478296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff9fddd3f0 con 0x55ff9fd9c4d0 2024-05-20T18:29:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.983+0000 7fde2e459640 1 --2- 172.21.15.5:0/530478296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9fdc5390 0x55ffa01a3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:29:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.983+0000 7fde2d457640 1 --2- 172.21.15.5:0/530478296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9fda46b0 0x55ffa01a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:29:10.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.983+0000 7fde2dc58640 1 --2- 172.21.15.5:0/530478296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9fd9c4d0 0x55ffa018b990 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fde24035040 tx=0x7fde24004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef72871cd255e8d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:10.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.983+0000 7fde08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:29:10.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.983+0000 7fde08ff9640 1 -- 172.21.15.5:0/530478296 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2404e020 con 0x55ff9fd9c4d0 2024-05-20T18:29:10.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.983+0000 7fde08ff9640 1 -- 172.21.15.5:0/530478296 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7fde2404a660 con 0x55ff9fd9c4d0 2024-05-20T18:29:10.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.983+0000 7fde369fd740 1 -- 172.21.15.5:0/530478296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde18018070 con 0x55ff9fd9c4d0 2024-05-20T18:29:10.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.983+0000 7fde08ff9640 1 -- 172.21.15.5:0/530478296 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fde24058050 con 0x55ff9fd9c4d0 2024-05-20T18:29:10.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.983+0000 7fde369fd740 10 client.5949 fetch_fsmap learned FSMap version 26 2024-05-20T18:29:10.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.984+0000 7fde369fd740 1 -- 172.21.15.5:0/530478296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55ff9fd9d0f0 con 0x55ff9fd9c4d0 2024-05-20T18:29:10.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.984+0000 7fde08ff9640 1 -- 172.21.15.5:0/530478296 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde18018070 con 0x55ff9fd9c4d0 2024-05-20T18:29:10.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.984+0000 7fde08ff9640 1 -- 172.21.15.5:0/530478296 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7fde2405b020 con 0x55ff9fd9c4d0 2024-05-20T18:29:10.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.984+0000 7fde369fd740 10 client.5949 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T18:29:10.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.984+0000 7fde369fd740 10 client.5949 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-20T18:29:10.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.984+0000 7fde369fd740 20 client.5949 populate_metadata read hostname 'smithi005' 2024-05-20T18:29:10.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.984+0000 7fde369fd740 1 --2- 172.21.15.5:0/530478296 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x55ff9fd83f60 0x55ffa01bd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:10.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.984+0000 7fde369fd740 4 client.5949 mds_command: new command op to 5751 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:29:10.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.984+0000 7fde369fd740 1 -- 172.21.15.5:0/530478296 --> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffa00d7430 con 0x55ff9fd83f60 2024-05-20T18:29:10.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.984+0000 7fde2d457640 1 --2- 172.21.15.5:0/530478296 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x55ff9fd83f60 0x55ffa01bd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:10.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.984+0000 7fde08ff9640 10 client.5949 ms_handle_connect on v2:172.21.15.5:6836/3469824363 2024-05-20T18:29:10.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.984+0000 7fde2d457640 1 --2- 172.21.15.5:0/530478296 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x55ff9fd83f60 0x55ffa01bd420 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81827fd2b723fabc server_cookie=1c0b2cb9d3214c6 in_seq=0 out_seq=0 2024-05-20T18:29:10.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.985+0000 7fde08ff9640 1 -- 172.21.15.5:0/530478296 <== mds.0 v2:172.21.15.5:6836/3469824363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ffa00d7430 con 0x55ff9fd83f60 2024-05-20T18:29:10.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.985+0000 7fde08ff9640 10 client.5949 handle_command_reply: tid=0 2024-05-20T18:29:10.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.985+0000 7fde369fd740 1 client.5949 shutdown 2024-05-20T18:29:10.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.985+0000 7fde369fd740 1 -- 172.21.15.5:0/530478296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9fd9c4d0 msgr2=0x55ffa018b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:10.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.985+0000 7fde369fd740 1 --2- 172.21.15.5:0/530478296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9fd9c4d0 0x55ffa018b990 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fde24035040 tx=0x7fde24004730 comp rx=0 tx=0).stop 2024-05-20T18:29:10.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.985+0000 7fde369fd740 1 -- 172.21.15.5:0/530478296 shutdown_connections 2024-05-20T18:29:10.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.985+0000 7fde369fd740 1 --2- 172.21.15.5:0/530478296 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x55ff9fd83f60 0x55ffa01bd420 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.985+0000 7fde369fd740 1 --2- 172.21.15.5:0/530478296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9fdc5390 0x55ffa01a3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.985+0000 7fde369fd740 1 --2- 172.21.15.5:0/530478296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9fda46b0 0x55ffa01a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.985+0000 7fde08ff9640 0 client.5949 ms_handle_reset on v2:172.21.15.5:6836/3469824363 2024-05-20T18:29:10.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.985+0000 7fde369fd740 1 --2- 172.21.15.5:0/530478296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9fd9c4d0 0x55ffa018b990 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:10.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.985+0000 7fde369fd740 1 -- 172.21.15.5:0/530478296 >> 172.21.15.5:0/530478296 conn(0x55ff9fdb94e0 msgr2=0x55ffa018c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:10.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.985+0000 7fde369fd740 1 -- 172.21.15.5:0/530478296 shutdown_connections 2024-05-20T18:29:10.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.985+0000 7fde369fd740 1 -- 172.21.15.5:0/530478296 wait complete. 2024-05-20T18:29:10.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:10.986+0000 7fde369fd740 20 client.5949 trim_cache size 0 max 16384 2024-05-20T18:29:11.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.004+0000 7fde369fd740 1 Processor -- start 2024-05-20T18:29:11.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.004+0000 7fde369fd740 1 -- start start 2024-05-20T18:29:11.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.005+0000 7fde369fd740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9fd9c4d0 0x55ffa0163b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:11.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.005+0000 7fde369fd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9fda46b0 0x55ffa018c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:11.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.005+0000 7fde369fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9fdc5390 0x55ffa018c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:11.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.005+0000 7fde369fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ffa01abec0 con 0x55ff9fdc5390 2024-05-20T18:29:11.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.005+0000 7fde369fd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fde28202210 con 0x55ff9fda46b0 2024-05-20T18:29:11.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.005+0000 7fde369fd740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fde28201240 con 0x55ff9fd9c4d0 2024-05-20T18:29:11.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.006+0000 7fde2e459640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9fdc5390 0x55ffa018c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:11.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.006+0000 7fde2e459640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9fdc5390 0x55ffa018c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43122/0 (socket says 172.21.15.5:43122) 2024-05-20T18:29:11.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.006+0000 7fde2e459640 1 -- 172.21.15.5:0/2599526784 learned_addr learned my addr 172.21.15.5:0/2599526784 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:11.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.006+0000 7fde2d457640 1 --2- 172.21.15.5:0/2599526784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9fda46b0 0x55ffa018c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:11.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.006+0000 7fde2dc58640 1 --2- 172.21.15.5:0/2599526784 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9fd9c4d0 0x55ffa0163b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:11.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.006+0000 7fde2e459640 1 -- 172.21.15.5:0/2599526784 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9fd9c4d0 msgr2=0x55ffa0163b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:11.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.006+0000 7fde2e459640 1 --2- 172.21.15.5:0/2599526784 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9fd9c4d0 0x55ffa0163b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:11.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.006+0000 7fde2e459640 1 -- 172.21.15.5:0/2599526784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9fda46b0 msgr2=0x55ffa018c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:11.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.006+0000 7fde2e459640 1 --2- 172.21.15.5:0/2599526784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9fda46b0 0x55ffa018c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:11.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.006+0000 7fde2e459640 1 -- 172.21.15.5:0/2599526784 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffa0188b90 con 0x55ff9fdc5390 2024-05-20T18:29:11.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.006+0000 7fde2e459640 1 --2- 172.21.15.5:0/2599526784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9fdc5390 0x55ffa018c990 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fde20089080 tx=0x7fde2005d2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c758ae8e80f8bda9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:11.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.006+0000 7fde0affd640 1 -- 172.21.15.5:0/2599526784 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde20055520 con 0x55ff9fdc5390 2024-05-20T18:29:11.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.006+0000 7fde0affd640 1 -- 172.21.15.5:0/2599526784 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde20061550 con 0x55ff9fdc5390 2024-05-20T18:29:11.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.006+0000 7fde0affd640 1 -- 172.21.15.5:0/2599526784 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde200556c0 con 0x55ff9fdc5390 2024-05-20T18:29:11.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.007+0000 7fde369fd740 1 -- 172.21.15.5:0/2599526784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9fdc5390 msgr2=0x55ffa018c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:11.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.007+0000 7fde369fd740 1 --2- 172.21.15.5:0/2599526784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9fdc5390 0x55ffa018c990 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fde20089080 tx=0x7fde2005d2f0 comp rx=0 tx=0).stop 2024-05-20T18:29:11.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.007+0000 7fde369fd740 1 -- 172.21.15.5:0/2599526784 shutdown_connections 2024-05-20T18:29:11.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.007+0000 7fde369fd740 1 --2- 172.21.15.5:0/2599526784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9fdc5390 0x55ffa018c990 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:11.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.007+0000 7fde369fd740 1 --2- 172.21.15.5:0/2599526784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9fda46b0 0x55ffa018c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:11.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.007+0000 7fde369fd740 1 --2- 172.21.15.5:0/2599526784 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9fd9c4d0 0x55ffa0163b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:11.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.007+0000 7fde369fd740 1 -- 172.21.15.5:0/2599526784 >> 172.21.15.5:0/2599526784 conn(0x55ff9fdb94e0 msgr2=0x55ffa01a3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:11.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.007+0000 7fde369fd740 1 -- 172.21.15.5:0/2599526784 shutdown_connections 2024-05-20T18:29:11.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.007+0000 7fde369fd740 1 -- 172.21.15.5:0/2599526784 wait complete. 2024-05-20T18:29:11.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.008+0000 7fde369fd740 1 -- start start 2024-05-20T18:29:11.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.008+0000 7fde369fd740 1 Processor -- start 2024-05-20T18:29:11.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.009+0000 7fde369fd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9fd9c4d0 0x55ffa022a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:11.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.009+0000 7fde369fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9fda46b0 0x55ffa015eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:11.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.010+0000 7fde369fd740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9fdc5390 0x55ffa0187840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:11.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.010+0000 7fde369fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fde281e9dc0 con 0x55ff9fda46b0 2024-05-20T18:29:11.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.010+0000 7fde369fd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fde28156810 con 0x55ff9fd9c4d0 2024-05-20T18:29:11.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.010+0000 7fde369fd740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55ffa0192d20 con 0x55ff9fdc5390 2024-05-20T18:29:11.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.010+0000 7fde369fd740 10 client.0 _ll_register_callbacks cb 0x7fde14003880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:29:11.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.010+0000 7fde2d457640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9fda46b0 0x55ffa015eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:11.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.010+0000 7fde2d457640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9fda46b0 0x55ffa015eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43134/0 (socket says 172.21.15.5:43134) 2024-05-20T18:29:11.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.010+0000 7fde2d457640 1 -- 172.21.15.5:0/1862469635 learned_addr learned my addr 172.21.15.5:0/1862469635 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:11.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.010+0000 7fde2dc58640 1 --2- 172.21.15.5:0/1862469635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9fd9c4d0 0x55ffa022a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:11.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.010+0000 7fde2e459640 1 --2- 172.21.15.5:0/1862469635 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9fdc5390 0x55ffa0187840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:11.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.010+0000 7fde2d457640 1 -- 172.21.15.5:0/1862469635 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9fdc5390 msgr2=0x55ffa0187840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:11.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.010+0000 7fde2d457640 1 --2- 172.21.15.5:0/1862469635 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9fdc5390 0x55ffa0187840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:11.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.010+0000 7fde2d457640 1 -- 172.21.15.5:0/1862469635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9fd9c4d0 msgr2=0x55ffa022a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:11.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.010+0000 7fde2d457640 1 --2- 172.21.15.5:0/1862469635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9fd9c4d0 0x55ffa022a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:11.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.010+0000 7fde2d457640 1 -- 172.21.15.5:0/1862469635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffa0160550 con 0x55ff9fda46b0 2024-05-20T18:29:11.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.010+0000 7fde2e459640 1 --2- 172.21.15.5:0/1862469635 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9fdc5390 0x55ffa0187840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:29:11.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.010+0000 7fde2dc58640 1 --2- 172.21.15.5:0/1862469635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9fd9c4d0 0x55ffa022a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:29:11.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.010+0000 7fde2d457640 1 --2- 172.21.15.5:0/1862469635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9fda46b0 0x55ffa015eb90 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x55ffa0161560 tx=0x7fde18000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922f969d1bd641e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:11.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.010+0000 7fde08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:29:11.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.010+0000 7fde08ff9640 1 -- 172.21.15.5:0/1862469635 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1800fbc0 con 0x55ff9fda46b0 2024-05-20T18:29:11.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.010+0000 7fde08ff9640 1 -- 172.21.15.5:0/1862469635 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7fde1800fd60 con 0x55ff9fda46b0 2024-05-20T18:29:11.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.010+0000 7fde369fd740 1 -- 172.21.15.5:0/1862469635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde20061550 con 0x55ff9fda46b0 2024-05-20T18:29:11.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.011+0000 7fde08ff9640 1 -- 172.21.15.5:0/1862469635 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fde18030050 con 0x55ff9fda46b0 2024-05-20T18:29:11.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.011+0000 7fde369fd740 10 client.5955 fetch_fsmap learned FSMap version 26 2024-05-20T18:29:11.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.011+0000 7fde369fd740 1 -- 172.21.15.5:0/1862469635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55ffa0187d80 con 0x55ff9fda46b0 2024-05-20T18:29:11.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.011+0000 7fde08ff9640 1 -- 172.21.15.5:0/1862469635 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde20061550 con 0x55ff9fda46b0 2024-05-20T18:29:11.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.011+0000 7fde08ff9640 1 -- 172.21.15.5:0/1862469635 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7fde18033020 con 0x55ff9fda46b0 2024-05-20T18:29:11.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.011+0000 7fde369fd740 10 client.5955 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T18:29:11.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.011+0000 7fde369fd740 10 client.5955 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-20T18:29:11.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.011+0000 7fde369fd740 20 client.5955 populate_metadata read hostname 'smithi005' 2024-05-20T18:29:11.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.011+0000 7fde369fd740 1 --2- 172.21.15.5:0/1862469635 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x55ffa01a1350 0x55ffa018c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:11.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.011+0000 7fde369fd740 4 client.5955 mds_command: new command op to 5751 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T18:29:11.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.011+0000 7fde369fd740 1 -- 172.21.15.5:0/1862469635 --> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ffa00d1780 con 0x55ffa01a1350 2024-05-20T18:29:11.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.011+0000 7fde2dc58640 1 --2- 172.21.15.5:0/1862469635 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x55ffa01a1350 0x55ffa018c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:11.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.011+0000 7fde08ff9640 10 client.5955 ms_handle_connect on v2:172.21.15.5:6836/3469824363 2024-05-20T18:29:11.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.011+0000 7fde2dc58640 1 --2- 172.21.15.5:0/1862469635 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x55ffa01a1350 0x55ffa018c450 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e545e8a20c43e224 server_cookie=59fa8e25392bebf in_seq=0 out_seq=0 2024-05-20T18:29:11.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.012+0000 7fde08ff9640 1 -- 172.21.15.5:0/1862469635 <== mds.0 v2:172.21.15.5:6836/3469824363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x55ffa00d1780 con 0x55ffa01a1350 2024-05-20T18:29:11.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.012+0000 7fde08ff9640 10 client.5955 handle_command_reply: tid=0 2024-05-20T18:29:11.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.012+0000 7fde369fd740 1 client.5955 shutdown 2024-05-20T18:29:11.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.012+0000 7fde369fd740 1 -- 172.21.15.5:0/1862469635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9fda46b0 msgr2=0x55ffa015eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:11.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.012+0000 7fde369fd740 1 --2- 172.21.15.5:0/1862469635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9fda46b0 0x55ffa015eb90 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x55ffa0161560 tx=0x7fde18000bc0 comp rx=0 tx=0).stop 2024-05-20T18:29:11.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.012+0000 7fde369fd740 1 -- 172.21.15.5:0/1862469635 shutdown_connections 2024-05-20T18:29:11.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.012+0000 7fde369fd740 1 --2- 172.21.15.5:0/1862469635 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x55ffa01a1350 0x55ffa018c450 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:11.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.012+0000 7fde369fd740 1 --2- 172.21.15.5:0/1862469635 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ff9fdc5390 0x55ffa0187840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:11.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.012+0000 7fde369fd740 1 --2- 172.21.15.5:0/1862469635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff9fda46b0 0x55ffa015eb90 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:11.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.012+0000 7fde369fd740 1 --2- 172.21.15.5:0/1862469635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ff9fd9c4d0 0x55ffa022a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:11.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.012+0000 7fde08ff9640 0 client.5955 ms_handle_reset on v2:172.21.15.5:6836/3469824363 2024-05-20T18:29:11.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.012+0000 7fde369fd740 1 -- 172.21.15.5:0/1862469635 >> 172.21.15.5:0/1862469635 conn(0x55ff9fdb94e0 msgr2=0x55ffa0163b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:11.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.012+0000 7fde369fd740 1 -- 172.21.15.5:0/1862469635 shutdown_connections 2024-05-20T18:29:11.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.012+0000 7fde369fd740 1 -- 172.21.15.5:0/1862469635 wait complete. 2024-05-20T18:29:11.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.012+0000 7fde369fd740 20 client.5955 trim_cache size 0 max 16384 2024-05-20T18:29:11.020 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:29:11.020 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-20T18:29:11.020 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-20T18:29:11.020 INFO:teuthology.orchestra.run.smithi005.stdout: "314d484c-aaf6-43ad-8879-8755eff25ead": { 2024-05-20T18:29:11.021 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-05-20T18:29:11.021 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "314d484c-aaf6-43ad-8879-8755eff25ead", 2024-05-20T18:29:11.021 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,repair" 2024-05-20T18:29:11.021 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:29:11.021 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:29:11.021 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:29:11.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.013+0000 7fde2febc640 1 -- 172.21.15.5:0/3370240250 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fde0c06be30 msgr2=0x7fde0c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:11.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.013+0000 7fde2febc640 1 --2- 172.21.15.5:0/3370240250 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fde0c06be30 0x7fde0c06e340 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fde24034000 tx=0x7fde2402e000 comp rx=0 tx=0).stop 2024-05-20T18:29:11.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.013+0000 7fde2febc640 1 -- 172.21.15.5:0/3370240250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde28163a10 msgr2=0x7fde280b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:11.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.013+0000 7fde2febc640 1 --2- 172.21.15.5:0/3370240250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde28163a10 0x7fde280b6390 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fde2000caf0 tx=0x7fde2000a060 comp rx=0 tx=0).stop 2024-05-20T18:29:11.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.013+0000 7fde2febc640 1 -- 172.21.15.5:0/3370240250 shutdown_connections 2024-05-20T18:29:11.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.013+0000 7fde2febc640 1 --2- 172.21.15.5:0/3370240250 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fde0c06be30 0x7fde0c06e340 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:11.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.013+0000 7fde2febc640 1 --2- 172.21.15.5:0/3370240250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde28163a10 0x7fde280b6390 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:11.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.013+0000 7fde2febc640 1 --2- 172.21.15.5:0/3370240250 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde280d14b0 0x7fde28153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:11.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.013+0000 7fde2febc640 1 --2- 172.21.15.5:0/3370240250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde280b4010 0x7fde281ea8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:11.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.013+0000 7fde2febc640 1 -- 172.21.15.5:0/3370240250 >> 172.21.15.5:0/3370240250 conn(0x7fde2806d240 msgr2=0x7fde28162180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:11.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.013+0000 7fde2febc640 1 -- 172.21.15.5:0/3370240250 shutdown_connections 2024-05-20T18:29:11.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:11.013+0000 7fde2febc640 1 -- 172.21.15.5:0/3370240250 wait complete. 2024-05-20T18:29:11.024 INFO:tasks.cephfs.filesystem:scrub status for tag:314d484c-aaf6-43ad-8879-8755eff25ead - {'path': '/', 'tag': '314d484c-aaf6-43ad-8879-8755eff25ead', 'options': 'recursive,repair'} 2024-05-20T18:29:12.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:12.696+0000 7f562a7fc640 1 -- 172.21.15.5:0/1733018515 <== mon.0 v2:172.21.15.5:3300/0 7 ==== log(6 entries from seq 1 at 2024-05-20T18:29:10.836768+0000) ==== 1078+0+0 (secure 0 0 0) 0x7f56240093e0 con 0x7f56340d0070 2024-05-20T18:29:12.696 INFO:teuthology.orchestra.run.smithi005.stdout: cluster: 2024-05-20T18:29:12.696 INFO:teuthology.orchestra.run.smithi005.stdout: id: 399ab250-fd3c-47fa-a1c8-25e8149c9cac 2024-05-20T18:29:12.696 INFO:teuthology.orchestra.run.smithi005.stdout: health: HEALTH_OK 2024-05-20T18:29:12.696 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:29:12.696 INFO:teuthology.orchestra.run.smithi005.stdout: services: 2024-05-20T18:29:12.697 INFO:teuthology.orchestra.run.smithi005.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2024-05-20T18:29:12.697 INFO:teuthology.orchestra.run.smithi005.stdout: mgr: y(active, since 2m), standbys: x 2024-05-20T18:29:12.697 INFO:teuthology.orchestra.run.smithi005.stdout: mds: 1/1 daemons up, 3 standby 2024-05-20T18:29:12.697 INFO:teuthology.orchestra.run.smithi005.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-05-20T18:29:12.697 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:29:12.697 INFO:teuthology.orchestra.run.smithi005.stdout: data: 2024-05-20T18:29:12.697 INFO:teuthology.orchestra.run.smithi005.stdout: volumes: 1/1 healthy 2024-05-20T18:29:12.697 INFO:teuthology.orchestra.run.smithi005.stdout: pools: 3 pools, 97 pgs 2024-05-20T18:29:12.697 INFO:teuthology.orchestra.run.smithi005.stdout: objects: 24 objects, 579 KiB 2024-05-20T18:29:12.697 INFO:teuthology.orchestra.run.smithi005.stdout: usage: 233 MiB used, 720 GiB / 720 GiB avail 2024-05-20T18:29:12.697 INFO:teuthology.orchestra.run.smithi005.stdout: pgs: 97 active+clean 2024-05-20T18:29:12.697 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:29:12.697 INFO:teuthology.orchestra.run.smithi005.stdout: io: 2024-05-20T18:29:12.698 INFO:teuthology.orchestra.run.smithi005.stdout: client: 1.7 KiB/s wr, 0 op/s rd, 5 op/s wr 2024-05-20T18:29:12.698 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:29:12.698 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:29:12.698 INFO:teuthology.orchestra.run.smithi005.stdout:2024-05-20T18:29:10.836768+0000 mds.c [INF] scrub queued for path: / 2024-05-20T18:29:12.698 INFO:teuthology.orchestra.run.smithi005.stdout:2024-05-20T18:29:10.836771+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-05-20T18:29:12.698 INFO:teuthology.orchestra.run.smithi005.stdout:2024-05-20T18:29:10.836775+0000 mds.c [INF] scrub summary: active paths [/] 2024-05-20T18:29:12.698 INFO:teuthology.orchestra.run.smithi005.stdout:2024-05-20T18:29:10.837564+0000 mds.c [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2024-05-20T18:29:12.698 INFO:teuthology.orchestra.run.smithi005.stdout:2024-05-20T18:29:10.837578+0000 mds.c [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2024-05-20T18:29:12.698 INFO:teuthology.orchestra.run.smithi005.stdout:2024-05-20T18:29:10.838248+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-05-20T18:29:14.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:14.703+0000 7f562a7fc640 1 -- 172.21.15.5:0/1733018515 <== mon.0 v2:172.21.15.5:3300/0 8 ==== log(1 entries from seq 7 at 2024-05-20T18:29:12.709540+0000) ==== 189+0+0 (secure 0 0 0) 0x7f5624057810 con 0x7f56340d0070 2024-05-20T18:29:14.703 INFO:teuthology.orchestra.run.smithi005.stdout:2024-05-20T18:29:12.709540+0000 mds.c [INF] scrub summary: idle 2024-05-20T18:29:40.988 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:29:40.992 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:29:41.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:29:41.025 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-05-20T18:29:41.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.162+0000 7fe73b710640 1 Processor -- start 2024-05-20T18:29:41.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.162+0000 7fe73b710640 1 -- start start 2024-05-20T18:29:41.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.164+0000 7fe73b710640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe734162820 0x7fe7340b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.164+0000 7fe73b710640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe73406dae0 0x7fe7340b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.164+0000 7fe73b710640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7340d0630 con 0x7fe7340b3e80 2024-05-20T18:29:41.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.164+0000 7fe73b710640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe7340d39c0 con 0x7fe734162820 2024-05-20T18:29:41.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.164+0000 7fe73b710640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe7340d3ca0 con 0x7fe73406dae0 2024-05-20T18:29:41.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.164+0000 7fe7394ac640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe734162820 0x7fe7340b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.164+0000 7fe739cad640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe73406dae0 0x7fe7340b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.164+0000 7fe739cad640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe73406dae0 0x7fe7340b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:54664/0 (socket says 172.21.15.5:54664) 2024-05-20T18:29:41.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.164+0000 7fe738cab640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fe7340b3e80 0x7fe73406d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:35292/0 (socket says 172.21.15.5:35292) 2024-05-20T18:29:41.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.164+0000 7fe739cad640 1 -- 172.21.15.5:0/3824325386 learned_addr learned my addr 172.21.15.5:0/3824325386 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:41.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.164+0000 7fe7394ac640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe734162820 0x7fe7340b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:35026/0 (socket says 172.21.15.5:35026) 2024-05-20T18:29:41.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.164+0000 7fe72bfff640 1 -- 172.21.15.5:0/3824325386 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2279960232 0 0) 0x7fe7340d0630 con 0x7fe7340b3e80 2024-05-20T18:29:41.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.164+0000 7fe72bfff640 1 -- 172.21.15.5:0/3824325386 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe720003520 con 0x7fe7340b3e80 2024-05-20T18:29:41.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.164+0000 7fe7394ac640 1 -- 172.21.15.5:0/3824325386 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe73406dae0 msgr2=0x7fe7340b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:41.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.164+0000 7fe7394ac640 1 --2- 172.21.15.5:0/3824325386 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe73406dae0 0x7fe7340b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.164+0000 7fe7394ac640 1 -- 172.21.15.5:0/3824325386 >> v1:172.21.15.63:6789/0 conn(0x7fe7340b3e80 legacy=0x7fe73406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:41.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.165+0000 7fe7394ac640 1 -- 172.21.15.5:0/3824325386 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7340d3f80 con 0x7fe734162820 2024-05-20T18:29:41.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.165+0000 7fe739cad640 1 --2- 172.21.15.5:0/3824325386 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe73406dae0 0x7fe7340b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:29:41.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.165+0000 7fe7394ac640 1 --2- 172.21.15.5:0/3824325386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe734162820 0x7fe7340b3940 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe7300097a0 tx=0x7fe73002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fa50bc97abfe3a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:41.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.166+0000 7fe72bfff640 1 -- 172.21.15.5:0/3824325386 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe730037070 con 0x7fe734162820 2024-05-20T18:29:41.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.166+0000 7fe72bfff640 1 -- 172.21.15.5:0/3824325386 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe730032440 con 0x7fe734162820 2024-05-20T18:29:41.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.166+0000 7fe72bfff640 1 -- 172.21.15.5:0/3824325386 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7300327f0 con 0x7fe734162820 2024-05-20T18:29:41.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.166+0000 7fe73b710640 1 -- 172.21.15.5:0/3824325386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe734162820 msgr2=0x7fe7340b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:41.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.166+0000 7fe73b710640 1 --2- 172.21.15.5:0/3824325386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe734162820 0x7fe7340b3940 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe7300097a0 tx=0x7fe73002a180 comp rx=0 tx=0).stop 2024-05-20T18:29:41.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.166+0000 7fe73b710640 1 -- 172.21.15.5:0/3824325386 shutdown_connections 2024-05-20T18:29:41.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.166+0000 7fe73b710640 1 --2- 172.21.15.5:0/3824325386 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe73406dae0 0x7fe7340b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.166+0000 7fe73b710640 1 --2- 172.21.15.5:0/3824325386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe734162820 0x7fe7340b3940 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.166+0000 7fe73b710640 1 -- 172.21.15.5:0/3824325386 >> 172.21.15.5:0/3824325386 conn(0x7fe7340d4bd0 msgr2=0x7fe7340d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:41.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.166+0000 7fe73b710640 1 -- 172.21.15.5:0/3824325386 shutdown_connections 2024-05-20T18:29:41.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.166+0000 7fe73b710640 1 -- 172.21.15.5:0/3824325386 wait complete. 2024-05-20T18:29:41.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.168+0000 7fe73b710640 1 Processor -- start 2024-05-20T18:29:41.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.168+0000 7fe73b710640 1 -- start start 2024-05-20T18:29:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.170+0000 7fe73b710640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7340b3e80 0x7fe73415e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.170+0000 7fe73b710640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe734162820 0x7fe7340c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.170+0000 7fe73b710640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe73415ebe0 0x7fe7340ca230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.170+0000 7fe73b710640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe7340d4780 con 0x7fe7340b3e80 2024-05-20T18:29:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.170+0000 7fe73b710640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe7340d0300 con 0x7fe73415ebe0 2024-05-20T18:29:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.170+0000 7fe73b710640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe7340d4900 con 0x7fe734162820 2024-05-20T18:29:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.170+0000 7fe7394ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7340b3e80 0x7fe73415e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.170+0000 7fe7394ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7340b3e80 0x7fe73415e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35152/0 (socket says 172.21.15.5:35152) 2024-05-20T18:29:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.170+0000 7fe7394ac640 1 -- 172.21.15.5:0/2529601655 learned_addr learned my addr 172.21.15.5:0/2529601655 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.170+0000 7fe738cab640 1 --2- 172.21.15.5:0/2529601655 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe734162820 0x7fe7340c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.170+0000 7fe739cad640 1 --2- 172.21.15.5:0/2529601655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe73415ebe0 0x7fe7340ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.170+0000 7fe7394ac640 1 -- 172.21.15.5:0/2529601655 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe734162820 msgr2=0x7fe7340c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.170+0000 7fe7394ac640 1 --2- 172.21.15.5:0/2529601655 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe734162820 0x7fe7340c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.170+0000 7fe7394ac640 1 -- 172.21.15.5:0/2529601655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe73415ebe0 msgr2=0x7fe7340ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.170+0000 7fe7394ac640 1 --2- 172.21.15.5:0/2529601655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe73415ebe0 0x7fe7340ca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.170+0000 7fe7394ac640 1 -- 172.21.15.5:0/2529601655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73415f160 con 0x7fe7340b3e80 2024-05-20T18:29:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.170+0000 7fe7394ac640 1 --2- 172.21.15.5:0/2529601655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7340b3e80 0x7fe73415e6a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe7300098c0 tx=0x7fe7300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.170+0000 7fe729ffb640 1 -- 172.21.15.5:0/2529601655 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe730037040 con 0x7fe7340b3e80 2024-05-20T18:29:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.171+0000 7fe729ffb640 1 -- 172.21.15.5:0/2529601655 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7300044c0 con 0x7fe7340b3e80 2024-05-20T18:29:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.171+0000 7fe73b710640 1 -- 172.21.15.5:0/2529601655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7340c8f30 con 0x7fe7340b3e80 2024-05-20T18:29:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.171+0000 7fe729ffb640 1 -- 172.21.15.5:0/2529601655 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7300325f0 con 0x7fe7340b3e80 2024-05-20T18:29:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.171+0000 7fe73b710640 1 -- 172.21.15.5:0/2529601655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7340c94f0 con 0x7fe7340b3e80 2024-05-20T18:29:41.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.173+0000 7fe729ffb640 1 -- 172.21.15.5:0/2529601655 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe730044020 con 0x7fe7340b3e80 2024-05-20T18:29:41.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.173+0000 7fe729ffb640 1 --2- 172.21.15.5:0/2529601655 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe71806be60 0x7fe71806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.173+0000 7fe729ffb640 1 -- 172.21.15.5:0/2529601655 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7fe7300ab140 con 0x7fe7340b3e80 2024-05-20T18:29:41.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.173+0000 7fe738cab640 1 --2- 172.21.15.5:0/2529601655 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe71806be60 0x7fe71806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.174+0000 7fe738cab640 1 --2- 172.21.15.5:0/2529601655 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe71806be60 0x7fe71806e370 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe7341619b0 tx=0x7fe724007cf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:41.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.174+0000 7fe7425bc740 1 Processor -- start 2024-05-20T18:29:41.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.175+0000 7fe7425bc740 1 -- start start 2024-05-20T18:29:41.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.176+0000 7fe7425bc740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d64dbc9390 0x55d64df93b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.176+0000 7fe7425bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64dba04d0 0x55d64dfae890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.176+0000 7fe7425bc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d64dba86b0 0x55d64dfaedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.176+0000 7fe7425bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d64decdd00 con 0x55d64dba04d0 2024-05-20T18:29:41.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.176+0000 7fe7425bc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d64de754c0 con 0x55d64dba86b0 2024-05-20T18:29:41.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.176+0000 7fe7425bc740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55d64dec9cc0 con 0x55d64dbc9390 2024-05-20T18:29:41.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.176+0000 7fe738cab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64dba04d0 0x55d64dfae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.176+0000 7fe738cab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64dba04d0 0x55d64dfae890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35168/0 (socket says 172.21.15.5:35168) 2024-05-20T18:29:41.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.176+0000 7fe738cab640 1 -- 172.21.15.5:0/493190021 learned_addr learned my addr 172.21.15.5:0/493190021 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:41.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.176+0000 7fe7394ac640 1 --2- 172.21.15.5:0/493190021 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d64dbc9390 0x55d64df93b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.176+0000 7fe739cad640 1 --2- 172.21.15.5:0/493190021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d64dba86b0 0x55d64dfaedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.176+0000 7fe738cab640 1 -- 172.21.15.5:0/493190021 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d64dbc9390 msgr2=0x55d64df93b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:41.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.176+0000 7fe738cab640 1 --2- 172.21.15.5:0/493190021 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d64dbc9390 0x55d64df93b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.176+0000 7fe738cab640 1 -- 172.21.15.5:0/493190021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d64dba86b0 msgr2=0x55d64dfaedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:41.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.176+0000 7fe738cab640 1 --2- 172.21.15.5:0/493190021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d64dba86b0 0x55d64dfaedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.176+0000 7fe738cab640 1 -- 172.21.15.5:0/493190021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d64dcab7d0 con 0x55d64dba04d0 2024-05-20T18:29:41.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.176+0000 7fe738cab640 1 --2- 172.21.15.5:0/493190021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64dba04d0 0x55d64dfae890 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe724007350 tx=0x7fe724012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3603592dd39b0c00 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:41.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.177+0000 7fe71effd640 1 -- 172.21.15.5:0/493190021 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe724026070 con 0x55d64dba04d0 2024-05-20T18:29:41.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.177+0000 7fe71effd640 1 -- 172.21.15.5:0/493190021 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe724001d10 con 0x55d64dba04d0 2024-05-20T18:29:41.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.177+0000 7fe71effd640 1 -- 172.21.15.5:0/493190021 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe724013d80 con 0x55d64dba04d0 2024-05-20T18:29:41.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.177+0000 7fe7425bc740 1 -- 172.21.15.5:0/493190021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64dba04d0 msgr2=0x55d64dfae890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:41.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.177+0000 7fe7425bc740 1 --2- 172.21.15.5:0/493190021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64dba04d0 0x55d64dfae890 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe724007350 tx=0x7fe724012000 comp rx=0 tx=0).stop 2024-05-20T18:29:41.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.177+0000 7fe7425bc740 1 -- 172.21.15.5:0/493190021 shutdown_connections 2024-05-20T18:29:41.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.177+0000 7fe7425bc740 1 --2- 172.21.15.5:0/493190021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d64dba86b0 0x55d64dfaedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.177+0000 7fe7425bc740 1 --2- 172.21.15.5:0/493190021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64dba04d0 0x55d64dfae890 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.177+0000 7fe7425bc740 1 --2- 172.21.15.5:0/493190021 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d64dbc9390 0x55d64df93b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.177+0000 7fe7425bc740 1 -- 172.21.15.5:0/493190021 >> 172.21.15.5:0/493190021 conn(0x55d64dbbd4e0 msgr2=0x55d64df691c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:41.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.178+0000 7fe7425bc740 1 -- 172.21.15.5:0/493190021 shutdown_connections 2024-05-20T18:29:41.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.178+0000 7fe7425bc740 1 -- 172.21.15.5:0/493190021 wait complete. 2024-05-20T18:29:41.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.178+0000 7fe7425bc740 1 -- start start 2024-05-20T18:29:41.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.179+0000 7fe7425bc740 1 Processor -- start 2024-05-20T18:29:41.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.180+0000 7fe7425bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64dba04d0 0x55d64df96040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.180+0000 7fe7425bc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d64dba86b0 0x55d64dfa9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.180+0000 7fe7425bc740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d64dbc9390 0x55d64dfac620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.180+0000 7fe7425bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d64de88b90 con 0x55d64dba04d0 2024-05-20T18:29:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.180+0000 7fe7425bc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d64dfad340 con 0x55d64dba86b0 2024-05-20T18:29:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.180+0000 7fe7425bc740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55d64df94d50 con 0x55d64dbc9390 2024-05-20T18:29:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.180+0000 7fe7425bc740 10 client.0 _ll_register_callbacks cb 0x55d64dec4da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:29:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.180+0000 7fe7394ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64dba04d0 0x55d64df96040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.180+0000 7fe7394ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64dba04d0 0x55d64df96040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35184/0 (socket says 172.21.15.5:35184) 2024-05-20T18:29:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.180+0000 7fe7394ac640 1 -- 172.21.15.5:0/354392750 learned_addr learned my addr 172.21.15.5:0/354392750 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.180+0000 7fe738cab640 1 --2- 172.21.15.5:0/354392750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d64dba86b0 0x55d64dfa9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.180+0000 7fe739cad640 1 --2- 172.21.15.5:0/354392750 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d64dbc9390 0x55d64dfac620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.181+0000 7fe7394ac640 1 -- 172.21.15.5:0/354392750 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d64dbc9390 msgr2=0x55d64dfac620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.181+0000 7fe7394ac640 1 --2- 172.21.15.5:0/354392750 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d64dbc9390 0x55d64dfac620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.181+0000 7fe7394ac640 1 -- 172.21.15.5:0/354392750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d64dba86b0 msgr2=0x55d64dfa9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.181+0000 7fe7394ac640 1 --2- 172.21.15.5:0/354392750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d64dba86b0 0x55d64dfa9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.181+0000 7fe7394ac640 1 -- 172.21.15.5:0/354392750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d64dbe13f0 con 0x55d64dba04d0 2024-05-20T18:29:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.181+0000 7fe739cad640 1 --2- 172.21.15.5:0/354392750 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d64dbc9390 0x55d64dfac620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:29:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.181+0000 7fe7394ac640 1 --2- 172.21.15.5:0/354392750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64dba04d0 0x55d64df96040 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe7300756e0 tx=0x7fe730079280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c797cbda5f40a41 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.181+0000 7fe713fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:29:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.181+0000 7fe713fff640 1 -- 172.21.15.5:0/354392750 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe730081e40 con 0x55d64dba04d0 2024-05-20T18:29:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.181+0000 7fe713fff640 1 -- 172.21.15.5:0/354392750 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7fe73007d170 con 0x55d64dba04d0 2024-05-20T18:29:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.181+0000 7fe7425bc740 1 -- 172.21.15.5:0/354392750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe724001d10 con 0x55d64dba04d0 2024-05-20T18:29:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.181+0000 7fe713fff640 1 -- 172.21.15.5:0/354392750 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe7300a9050 con 0x55d64dba04d0 2024-05-20T18:29:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.181+0000 7fe7425bc740 10 client.5964 fetch_fsmap learned FSMap version 26 2024-05-20T18:29:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.181+0000 7fe7425bc740 1 -- 172.21.15.5:0/354392750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55d64dba10f0 con 0x55d64dba04d0 2024-05-20T18:29:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.181+0000 7fe713fff640 1 -- 172.21.15.5:0/354392750 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe724001d10 con 0x55d64dba04d0 2024-05-20T18:29:41.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.181+0000 7fe713fff640 1 -- 172.21.15.5:0/354392750 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7fe73003a030 con 0x55d64dba04d0 2024-05-20T18:29:41.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.181+0000 7fe7425bc740 10 client.5964 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T18:29:41.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.181+0000 7fe7425bc740 10 client.5964 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-20T18:29:41.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.181+0000 7fe7425bc740 20 client.5964 populate_metadata read hostname 'smithi005' 2024-05-20T18:29:41.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.182+0000 7fe7425bc740 1 --2- 172.21.15.5:0/354392750 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x55d64db87f60 0x55d64dfaa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.182+0000 7fe7425bc740 4 client.5964 mds_command: new command op to 5751 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:29:41.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.182+0000 7fe7425bc740 1 -- 172.21.15.5:0/354392750 --> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d64dedb430 con 0x55d64db87f60 2024-05-20T18:29:41.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.182+0000 7fe738cab640 1 --2- 172.21.15.5:0/354392750 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x55d64db87f60 0x55d64dfaa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.182+0000 7fe713fff640 10 client.5964 ms_handle_connect on v2:172.21.15.5:6836/3469824363 2024-05-20T18:29:41.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.182+0000 7fe738cab640 1 --2- 172.21.15.5:0/354392750 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x55d64db87f60 0x55d64dfaa6b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b2b09c1806f8603 server_cookie=77f7a68067cba5f2 in_seq=0 out_seq=0 2024-05-20T18:29:41.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.183+0000 7fe713fff640 1 -- 172.21.15.5:0/354392750 <== mds.0 v2:172.21.15.5:6836/3469824363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d64dedb430 con 0x55d64db87f60 2024-05-20T18:29:41.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.183+0000 7fe713fff640 10 client.5964 handle_command_reply: tid=0 2024-05-20T18:29:41.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.183+0000 7fe7425bc740 1 client.5964 shutdown 2024-05-20T18:29:41.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.183+0000 7fe7425bc740 1 -- 172.21.15.5:0/354392750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64dba04d0 msgr2=0x55d64df96040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:41.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.183+0000 7fe7425bc740 1 --2- 172.21.15.5:0/354392750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64dba04d0 0x55d64df96040 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe7300756e0 tx=0x7fe730079280 comp rx=0 tx=0).stop 2024-05-20T18:29:41.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.183+0000 7fe7425bc740 1 -- 172.21.15.5:0/354392750 shutdown_connections 2024-05-20T18:29:41.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.183+0000 7fe7425bc740 1 --2- 172.21.15.5:0/354392750 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x55d64db87f60 0x55d64dfaa6b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.183+0000 7fe7425bc740 1 --2- 172.21.15.5:0/354392750 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d64dbc9390 0x55d64dfac620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.183+0000 7fe7425bc740 1 --2- 172.21.15.5:0/354392750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d64dba86b0 0x55d64dfa9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.183+0000 7fe713fff640 0 client.5964 ms_handle_reset on v2:172.21.15.5:6836/3469824363 2024-05-20T18:29:41.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.183+0000 7fe7425bc740 1 --2- 172.21.15.5:0/354392750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64dba04d0 0x55d64df96040 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.183+0000 7fe7425bc740 1 -- 172.21.15.5:0/354392750 >> 172.21.15.5:0/354392750 conn(0x55d64dbbd4e0 msgr2=0x55d64df67670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:41.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.183+0000 7fe7425bc740 1 -- 172.21.15.5:0/354392750 shutdown_connections 2024-05-20T18:29:41.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.183+0000 7fe7425bc740 1 -- 172.21.15.5:0/354392750 wait complete. 2024-05-20T18:29:41.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.183+0000 7fe7425bc740 20 client.5964 trim_cache size 0 max 16384 2024-05-20T18:29:41.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.201+0000 7fe7425bc740 1 Processor -- start 2024-05-20T18:29:41.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.201+0000 7fe7425bc740 1 -- start start 2024-05-20T18:29:41.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.203+0000 7fe7425bc740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d64dbbd4e0 0x55d64df902e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.203+0000 7fe7425bc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d64dba04d0 0x55d64dfa9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.203+0000 7fe7425bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64dba86b0 0x55d64dfd1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.203+0000 7fe7425bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe7342022e0 con 0x55d64dba86b0 2024-05-20T18:29:41.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.203+0000 7fe7425bc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe734201340 con 0x55d64dba04d0 2024-05-20T18:29:41.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.203+0000 7fe7425bc740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe7342006d0 con 0x55d64dbbd4e0 2024-05-20T18:29:41.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.203+0000 7fe739cad640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64dba86b0 0x55d64dfd1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.203+0000 7fe739cad640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64dba86b0 0x55d64dfd1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35196/0 (socket says 172.21.15.5:35196) 2024-05-20T18:29:41.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.203+0000 7fe739cad640 1 -- 172.21.15.5:0/3250650144 learned_addr learned my addr 172.21.15.5:0/3250650144 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:41.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.203+0000 7fe7394ac640 1 --2- 172.21.15.5:0/3250650144 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d64dbbd4e0 0x55d64df902e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.203+0000 7fe738cab640 1 --2- 172.21.15.5:0/3250650144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d64dba04d0 0x55d64dfa9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.203+0000 7fe739cad640 1 -- 172.21.15.5:0/3250650144 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d64dbbd4e0 msgr2=0x55d64df902e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:41.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.203+0000 7fe739cad640 1 --2- 172.21.15.5:0/3250650144 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d64dbbd4e0 0x55d64df902e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.203+0000 7fe739cad640 1 -- 172.21.15.5:0/3250650144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d64dba04d0 msgr2=0x55d64dfa9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:41.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.203+0000 7fe739cad640 1 --2- 172.21.15.5:0/3250650144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d64dba04d0 0x55d64dfa9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.203+0000 7fe739cad640 1 -- 172.21.15.5:0/3250650144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d64dfaffb0 con 0x55d64dba86b0 2024-05-20T18:29:41.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.203+0000 7fe738cab640 1 --2- 172.21.15.5:0/3250650144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d64dba04d0 0x55d64dfa9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:29:41.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.203+0000 7fe739cad640 1 --2- 172.21.15.5:0/3250650144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64dba86b0 0x55d64dfd1500 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe7340cbc50 tx=0x7fe72c009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d62d9b893fc9be6e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:41.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.204+0000 7fe71e7fc640 1 -- 172.21.15.5:0/3250650144 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c003990 con 0x55d64dba86b0 2024-05-20T18:29:41.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.204+0000 7fe71e7fc640 1 -- 172.21.15.5:0/3250650144 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe72c01b030 con 0x55d64dba86b0 2024-05-20T18:29:41.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.204+0000 7fe71e7fc640 1 -- 172.21.15.5:0/3250650144 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c003d70 con 0x55d64dba86b0 2024-05-20T18:29:41.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.204+0000 7fe7425bc740 1 -- 172.21.15.5:0/3250650144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64dba86b0 msgr2=0x55d64dfd1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:41.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.204+0000 7fe7425bc740 1 --2- 172.21.15.5:0/3250650144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64dba86b0 0x55d64dfd1500 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe7340cbc50 tx=0x7fe72c009810 comp rx=0 tx=0).stop 2024-05-20T18:29:41.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.204+0000 7fe7425bc740 1 -- 172.21.15.5:0/3250650144 shutdown_connections 2024-05-20T18:29:41.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.204+0000 7fe7425bc740 1 --2- 172.21.15.5:0/3250650144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64dba86b0 0x55d64dfd1500 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.204+0000 7fe7425bc740 1 --2- 172.21.15.5:0/3250650144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d64dba04d0 0x55d64dfa9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.204+0000 7fe7425bc740 1 --2- 172.21.15.5:0/3250650144 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d64dbbd4e0 0x55d64df902e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.204+0000 7fe7425bc740 1 -- 172.21.15.5:0/3250650144 >> 172.21.15.5:0/3250650144 conn(0x55d64df969d0 msgr2=0x55d64dfc02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:41.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.204+0000 7fe7425bc740 1 -- 172.21.15.5:0/3250650144 shutdown_connections 2024-05-20T18:29:41.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.204+0000 7fe7425bc740 1 -- 172.21.15.5:0/3250650144 wait complete. 2024-05-20T18:29:41.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.205+0000 7fe7425bc740 1 -- start start 2024-05-20T18:29:41.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.205+0000 7fe7425bc740 1 Processor -- start 2024-05-20T18:29:41.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.207+0000 7fe7425bc740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d64dba04d0 0x55d64dfa9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.207+0000 7fe7425bc740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d64dba86b0 0x55d64dfa9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.207+0000 7fe7425bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64dbbd4e0 0x55d64dfaa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.207+0000 7fe7425bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d64de6b560 con 0x55d64dbbd4e0 2024-05-20T18:29:41.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.207+0000 7fe7425bc740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe734200d20 con 0x55d64dba04d0 2024-05-20T18:29:41.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.207+0000 7fe7425bc740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55d64dfab490 con 0x55d64dba86b0 2024-05-20T18:29:41.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.207+0000 7fe7425bc740 10 client.0 _ll_register_callbacks cb 0x7fe730006ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:29:41.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.207+0000 7fe739cad640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64dbbd4e0 0x55d64dfaa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.207+0000 7fe739cad640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64dbbd4e0 0x55d64dfaa5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35204/0 (socket says 172.21.15.5:35204) 2024-05-20T18:29:41.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.207+0000 7fe739cad640 1 -- 172.21.15.5:0/1861480113 learned_addr learned my addr 172.21.15.5:0/1861480113 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:41.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.207+0000 7fe7394ac640 1 --2- 172.21.15.5:0/1861480113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d64dba04d0 0x55d64dfa9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.207+0000 7fe738cab640 1 --2- 172.21.15.5:0/1861480113 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d64dba86b0 0x55d64dfa9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.207+0000 7fe739cad640 1 -- 172.21.15.5:0/1861480113 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d64dba86b0 msgr2=0x55d64dfa9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:41.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.207+0000 7fe739cad640 1 --2- 172.21.15.5:0/1861480113 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d64dba86b0 0x55d64dfa9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.207+0000 7fe739cad640 1 -- 172.21.15.5:0/1861480113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d64dba04d0 msgr2=0x55d64dfa9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:41.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.207+0000 7fe739cad640 1 --2- 172.21.15.5:0/1861480113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d64dba04d0 0x55d64dfa9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.207+0000 7fe739cad640 1 -- 172.21.15.5:0/1861480113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d64df8dbf0 con 0x55d64dbbd4e0 2024-05-20T18:29:41.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.207+0000 7fe739cad640 1 --2- 172.21.15.5:0/1861480113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64dbbd4e0 0x55d64dfaa5e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe72c009850 tx=0x7fe72c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d455528c89eb2c67 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:41.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.207+0000 7fe713fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:29:41.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.208+0000 7fe713fff640 1 -- 172.21.15.5:0/1861480113 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c004240 con 0x55d64dbbd4e0 2024-05-20T18:29:41.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.208+0000 7fe713fff640 1 -- 172.21.15.5:0/1861480113 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7fe72c02bc80 con 0x55d64dbbd4e0 2024-05-20T18:29:41.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.208+0000 7fe7425bc740 1 -- 172.21.15.5:0/1861480113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe72c01b030 con 0x55d64dbbd4e0 2024-05-20T18:29:41.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.208+0000 7fe713fff640 1 -- 172.21.15.5:0/1861480113 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe72c0043e0 con 0x55d64dbbd4e0 2024-05-20T18:29:41.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.208+0000 7fe7425bc740 10 client.5970 fetch_fsmap learned FSMap version 26 2024-05-20T18:29:41.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.208+0000 7fe7425bc740 1 -- 172.21.15.5:0/1861480113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55d64dfab290 con 0x55d64dbbd4e0 2024-05-20T18:29:41.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.208+0000 7fe713fff640 1 -- 172.21.15.5:0/1861480113 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe72c01b030 con 0x55d64dbbd4e0 2024-05-20T18:29:41.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.208+0000 7fe713fff640 1 -- 172.21.15.5:0/1861480113 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7fe72c028070 con 0x55d64dbbd4e0 2024-05-20T18:29:41.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.208+0000 7fe7425bc740 10 client.5970 fetch_fsmap finished waiting for FSMap version 26 2024-05-20T18:29:41.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.208+0000 7fe7425bc740 10 client.5970 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-20T18:29:41.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.208+0000 7fe7425bc740 20 client.5970 populate_metadata read hostname 'smithi005' 2024-05-20T18:29:41.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.208+0000 7fe7425bc740 1 --2- 172.21.15.5:0/1861480113 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x55d64db5d3a0 0x55d64dfaab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.208+0000 7fe7425bc740 4 client.5970 mds_command: new command op to 5751 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T18:29:41.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.208+0000 7fe7425bc740 1 -- 172.21.15.5:0/1861480113 --> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d64ded5780 con 0x55d64db5d3a0 2024-05-20T18:29:41.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.208+0000 7fe7394ac640 1 --2- 172.21.15.5:0/1861480113 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x55d64db5d3a0 0x55d64dfaab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.209+0000 7fe713fff640 10 client.5970 ms_handle_connect on v2:172.21.15.5:6836/3469824363 2024-05-20T18:29:41.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.209+0000 7fe7394ac640 1 --2- 172.21.15.5:0/1861480113 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x55d64db5d3a0 0x55d64dfaab20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fd55260fcd32dff server_cookie=dec75cc0ea1b07cb in_seq=0 out_seq=0 2024-05-20T18:29:41.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.209+0000 7fe713fff640 1 -- 172.21.15.5:0/1861480113 <== mds.0 v2:172.21.15.5:6836/3469824363 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d64ded5780 con 0x55d64db5d3a0 2024-05-20T18:29:41.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.209+0000 7fe713fff640 10 client.5970 handle_command_reply: tid=0 2024-05-20T18:29:41.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.209+0000 7fe7425bc740 1 client.5970 shutdown 2024-05-20T18:29:41.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.209+0000 7fe7425bc740 1 -- 172.21.15.5:0/1861480113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64dbbd4e0 msgr2=0x55d64dfaa5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:41.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.209+0000 7fe7425bc740 1 --2- 172.21.15.5:0/1861480113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64dbbd4e0 0x55d64dfaa5e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe72c009850 tx=0x7fe72c004ab0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.209+0000 7fe7425bc740 1 -- 172.21.15.5:0/1861480113 shutdown_connections 2024-05-20T18:29:41.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.209+0000 7fe7425bc740 1 --2- 172.21.15.5:0/1861480113 >> [v2:172.21.15.5:6836/3469824363,v1:172.21.15.5:6837/3469824363] conn(0x55d64db5d3a0 0x55d64dfaab20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.209+0000 7fe7425bc740 1 --2- 172.21.15.5:0/1861480113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d64dbbd4e0 0x55d64dfaa5e0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.209+0000 7fe7425bc740 1 --2- 172.21.15.5:0/1861480113 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d64dba86b0 0x55d64dfa9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.209+0000 7fe713fff640 0 client.5970 ms_handle_reset on v2:172.21.15.5:6836/3469824363 2024-05-20T18:29:41.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.209+0000 7fe7425bc740 1 --2- 172.21.15.5:0/1861480113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d64dba04d0 0x55d64dfa9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.209+0000 7fe7425bc740 1 -- 172.21.15.5:0/1861480113 >> 172.21.15.5:0/1861480113 conn(0x55d64df969d0 msgr2=0x55d64df638d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:41.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.209+0000 7fe7425bc740 1 -- 172.21.15.5:0/1861480113 shutdown_connections 2024-05-20T18:29:41.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.209+0000 7fe7425bc740 1 -- 172.21.15.5:0/1861480113 wait complete. 2024-05-20T18:29:41.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.209+0000 7fe7425bc740 20 client.5970 trim_cache size 0 max 16384 2024-05-20T18:29:41.224 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:29:41.225 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-20T18:29:41.225 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-20T18:29:41.225 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:29:41.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.210+0000 7fe73b710640 1 -- 172.21.15.5:0/2529601655 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe71806be60 msgr2=0x7fe71806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:41.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.210+0000 7fe73b710640 1 --2- 172.21.15.5:0/2529601655 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe71806be60 0x7fe71806e370 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe7341619b0 tx=0x7fe724007cf0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.210+0000 7fe73b710640 1 -- 172.21.15.5:0/2529601655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7340b3e80 msgr2=0x7fe73415e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:41.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.210+0000 7fe73b710640 1 --2- 172.21.15.5:0/2529601655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7340b3e80 0x7fe73415e6a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe7300098c0 tx=0x7fe7300046c0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.210+0000 7fe73b710640 1 -- 172.21.15.5:0/2529601655 shutdown_connections 2024-05-20T18:29:41.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.210+0000 7fe73b710640 1 --2- 172.21.15.5:0/2529601655 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe71806be60 0x7fe71806e370 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.210+0000 7fe73b710640 1 --2- 172.21.15.5:0/2529601655 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe73415ebe0 0x7fe7340ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.210+0000 7fe73b710640 1 --2- 172.21.15.5:0/2529601655 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe734162820 0x7fe7340c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.210+0000 7fe73b710640 1 --2- 172.21.15.5:0/2529601655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7340b3e80 0x7fe73415e6a0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.210+0000 7fe73b710640 1 -- 172.21.15.5:0/2529601655 >> 172.21.15.5:0/2529601655 conn(0x7fe7340d4bd0 msgr2=0x7fe734160eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:41.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.210+0000 7fe73b710640 1 -- 172.21.15.5:0/2529601655 shutdown_connections 2024-05-20T18:29:41.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.210+0000 7fe73b710640 1 -- 172.21.15.5:0/2529601655 wait complete. 2024-05-20T18:29:41.228 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T18:29:41.329 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:29:41.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.445+0000 7f2a6e7fe640 1 Processor -- start 2024-05-20T18:29:41.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.445+0000 7f2a6e7fe640 1 -- start start 2024-05-20T18:29:41.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.446+0000 7f2a6e7fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2a680cf3d0 0x7f2a680cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.446+0000 7f2a6e7fe640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a680cfd20 con 0x7f2a680ce7e0 2024-05-20T18:29:41.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.446+0000 7f2a6e7fe640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a680cda20 con 0x7f2a6815f220 2024-05-20T18:29:41.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.446+0000 7f2a6e7fe640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2a680cdd00 con 0x7f2a680cf3d0 2024-05-20T18:29:41.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.446+0000 7f2a6cd9b640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2a680ce7e0 0x7f2a680cebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44666/0 (socket says 172.21.15.5:44666) 2024-05-20T18:29:41.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.446+0000 7f2a6cd9b640 1 -- 172.21.15.5:0/2127117035 learned_addr learned my addr 172.21.15.5:0/2127117035 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:41.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.446+0000 7f2a677fe640 1 --2- 172.21.15.5:0/2127117035 >> v2:172.21.15.63:3300/0 conn(0x7f2a680cf3d0 0x7f2a680cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.446+0000 7f2a66ffd640 1 -- 172.21.15.5:0/2127117035 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 414096658 0 0) 0x7f2a60001490 con 0x7f2a680ce7e0 2024-05-20T18:29:41.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.447+0000 7f2a66ffd640 1 -- 172.21.15.5:0/2127117035 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a50003520 con 0x7f2a680ce7e0 2024-05-20T18:29:41.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.447+0000 7f2a66ffd640 1 -- 172.21.15.5:0/2127117035 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2475370213 0 0) 0x7f2a50003520 con 0x7f2a680ce7e0 2024-05-20T18:29:41.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.447+0000 7f2a677fe640 1 -- 172.21.15.5:0/2127117035 >> v1:172.21.15.63:6789/0 conn(0x7f2a6815f220 legacy=0x7f2a680d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:41.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.447+0000 7f2a677fe640 1 -- 172.21.15.5:0/2127117035 >> v1:172.21.15.5:6789/0 conn(0x7f2a680ce7e0 legacy=0x7f2a680cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:41.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.447+0000 7f2a677fe640 1 -- 172.21.15.5:0/2127117035 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a680cdfe0 con 0x7f2a680cf3d0 2024-05-20T18:29:41.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.448+0000 7f2a677fe640 1 --2- 172.21.15.5:0/2127117035 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a680cf3d0 0x7f2a680cf7e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0097a0 tx=0x7f2a5c02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97ffb3fb490620a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:41.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.448+0000 7f2a66ffd640 1 -- 172.21.15.5:0/2127117035 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c037070 con 0x7f2a680cf3d0 2024-05-20T18:29:41.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.448+0000 7f2a66ffd640 1 -- 172.21.15.5:0/2127117035 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a5c02abc0 con 0x7f2a680cf3d0 2024-05-20T18:29:41.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.448+0000 7f2a66ffd640 1 -- 172.21.15.5:0/2127117035 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a60001bc0 con 0x7f2a680cf3d0 2024-05-20T18:29:41.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.448+0000 7f2a6e7fe640 1 -- 172.21.15.5:0/2127117035 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a680cf3d0 msgr2=0x7f2a680cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:41.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.448+0000 7f2a6e7fe640 1 --2- 172.21.15.5:0/2127117035 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a680cf3d0 0x7f2a680cf7e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0097a0 tx=0x7f2a5c02a130 comp rx=0 tx=0).stop 2024-05-20T18:29:41.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.449+0000 7f2a6e7fe640 1 -- 172.21.15.5:0/2127117035 shutdown_connections 2024-05-20T18:29:41.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.449+0000 7f2a6e7fe640 1 --2- 172.21.15.5:0/2127117035 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a680cf3d0 0x7f2a680cf7e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.449+0000 7f2a6e7fe640 1 -- 172.21.15.5:0/2127117035 >> 172.21.15.5:0/2127117035 conn(0x7f2a680b44d0 msgr2=0x7f2a680b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:41.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.449+0000 7f2a6e7fe640 1 -- 172.21.15.5:0/2127117035 shutdown_connections 2024-05-20T18:29:41.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.449+0000 7f2a6e7fe640 1 -- 172.21.15.5:0/2127117035 wait complete. 2024-05-20T18:29:41.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.450+0000 7f2a6e7fe640 1 Processor -- start 2024-05-20T18:29:41.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.451+0000 7f2a6e7fe640 1 -- start start 2024-05-20T18:29:41.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.452+0000 7f2a6e7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a680ce7e0 0x7f2a680cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.452+0000 7f2a6e7fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a680cf3d0 0x7f2a68158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.452+0000 7f2a6e7fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a6815f220 0x7f2a6815db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.452+0000 7f2a6e7fe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2a680cd680 con 0x7f2a680ce7e0 2024-05-20T18:29:41.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.452+0000 7f2a6e7fe640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2a680d0fd0 con 0x7f2a6815f220 2024-05-20T18:29:41.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.452+0000 7f2a6e7fe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2a680cd800 con 0x7f2a680cf3d0 2024-05-20T18:29:41.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.452+0000 7f2a67fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a680ce7e0 0x7f2a680cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.452+0000 7f2a67fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a680ce7e0 0x7f2a680cb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35218/0 (socket says 172.21.15.5:35218) 2024-05-20T18:29:41.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.452+0000 7f2a67fff640 1 -- 172.21.15.5:0/619509346 learned_addr learned my addr 172.21.15.5:0/619509346 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:41.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.452+0000 7f2a677fe640 1 --2- 172.21.15.5:0/619509346 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a680cf3d0 0x7f2a68158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.452+0000 7f2a6cd9b640 1 --2- 172.21.15.5:0/619509346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a6815f220 0x7f2a6815db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.452+0000 7f2a67fff640 1 -- 172.21.15.5:0/619509346 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a680cf3d0 msgr2=0x7f2a68158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:41.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.452+0000 7f2a67fff640 1 --2- 172.21.15.5:0/619509346 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a680cf3d0 0x7f2a68158300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.452+0000 7f2a67fff640 1 -- 172.21.15.5:0/619509346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a6815f220 msgr2=0x7f2a6815db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:41.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.452+0000 7f2a67fff640 1 --2- 172.21.15.5:0/619509346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a6815f220 0x7f2a6815db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.452+0000 7f2a67fff640 1 -- 172.21.15.5:0/619509346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a6815f6c0 con 0x7f2a680ce7e0 2024-05-20T18:29:41.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.453+0000 7f2a67fff640 1 --2- 172.21.15.5:0/619509346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a680ce7e0 0x7f2a680cb640 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2a5800c3f0 tx=0x7f2a5800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:41.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.453+0000 7f2a64ff9640 1 -- 172.21.15.5:0/619509346 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a58017070 con 0x7f2a680ce7e0 2024-05-20T18:29:41.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.453+0000 7f2a64ff9640 1 -- 172.21.15.5:0/619509346 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a580052f0 con 0x7f2a680ce7e0 2024-05-20T18:29:41.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.453+0000 7f2a6e7fe640 1 -- 172.21.15.5:0/619509346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a68158840 con 0x7f2a680ce7e0 2024-05-20T18:29:41.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.453+0000 7f2a64ff9640 1 -- 172.21.15.5:0/619509346 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a58003830 con 0x7f2a680ce7e0 2024-05-20T18:29:41.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.453+0000 7f2a6e7fe640 1 -- 172.21.15.5:0/619509346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a680c6ed0 con 0x7f2a680ce7e0 2024-05-20T18:29:41.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.454+0000 7f2a6e7fe640 1 -- 172.21.15.5:0/619509346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a68075b00 con 0x7f2a680ce7e0 2024-05-20T18:29:41.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.455+0000 7f2a64ff9640 1 -- 172.21.15.5:0/619509346 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2a5801a070 con 0x7f2a680ce7e0 2024-05-20T18:29:41.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.455+0000 7f2a64ff9640 1 --2- 172.21.15.5:0/619509346 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2a5406be30 0x7f2a5406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.455+0000 7f2a64ff9640 1 -- 172.21.15.5:0/619509346 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f2a5808a3e0 con 0x7f2a680ce7e0 2024-05-20T18:29:41.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.455+0000 7f2a677fe640 1 --2- 172.21.15.5:0/619509346 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2a5406be30 0x7f2a5406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.456+0000 7f2a677fe640 1 --2- 172.21.15.5:0/619509346 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2a5406be30 0x7f2a5406e340 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c034000 tx=0x7f2a5c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:41.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.460+0000 7f2a64ff9640 1 -- 172.21.15.5:0/619509346 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2a6815f6c0 con 0x7f2a680ce7e0 2024-05-20T18:29:41.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.682+0000 7f2a6e7fe640 1 -- 172.21.15.5:0/619509346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a68162c60 con 0x7f2a680ce7e0 2024-05-20T18:29:41.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.683+0000 7f2a64ff9640 1 -- 172.21.15.5:0/619509346 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5175 (secure 0 0 0) 0x7f2a58058b70 con 0x7f2a680ce7e0 2024-05-20T18:29:41.683 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:29:41.683 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5207,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3958852803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3958852803},{"type":"v1","addr":"172.21.15.63:6835","nonce":3958852803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5560,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/3020169757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":3020169757},{"type":"v1","addr":"172.21.15.63:6834","nonce":3020169757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2965018210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2965018210},{"type":"v1","addr":"172.21.15.5:6835","nonce":2965018210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:29:02.038442+0000","modified":"2024-05-20T18:29:05.055248+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5751},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5751":{"gid":5751,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6837/3469824363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3469824363},{"type":"v1","addr":"172.21.15.5:6837","nonce":3469824363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5751,"qdb_cluster":[5751]},"id":3}]} 2024-05-20T18:29:41.683 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-05-20T18:29:41.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.685+0000 7f2a6e7fe640 1 -- 172.21.15.5:0/619509346 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2a5406be30 msgr2=0x7f2a5406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:41.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.685+0000 7f2a6e7fe640 1 --2- 172.21.15.5:0/619509346 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2a5406be30 0x7f2a5406e340 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c034000 tx=0x7f2a5c02e000 comp rx=0 tx=0).stop 2024-05-20T18:29:41.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.685+0000 7f2a6e7fe640 1 -- 172.21.15.5:0/619509346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a680ce7e0 msgr2=0x7f2a680cb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:41.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.685+0000 7f2a6e7fe640 1 --2- 172.21.15.5:0/619509346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a680ce7e0 0x7f2a680cb640 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2a5800c3f0 tx=0x7f2a5800c8b0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.685+0000 7f2a6e7fe640 1 -- 172.21.15.5:0/619509346 shutdown_connections 2024-05-20T18:29:41.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.685+0000 7f2a6e7fe640 1 --2- 172.21.15.5:0/619509346 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2a5406be30 0x7f2a5406e340 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.685+0000 7f2a6e7fe640 1 --2- 172.21.15.5:0/619509346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a6815f220 0x7f2a6815db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.685+0000 7f2a6e7fe640 1 --2- 172.21.15.5:0/619509346 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a680cf3d0 0x7f2a68158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.685+0000 7f2a6e7fe640 1 --2- 172.21.15.5:0/619509346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a680ce7e0 0x7f2a680cb640 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.685+0000 7f2a6e7fe640 1 -- 172.21.15.5:0/619509346 >> 172.21.15.5:0/619509346 conn(0x7f2a680b44d0 msgr2=0x7f2a6815c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:41.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.685+0000 7f2a6e7fe640 1 -- 172.21.15.5:0/619509346 shutdown_connections 2024-05-20T18:29:41.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.685+0000 7f2a6e7fe640 1 -- 172.21.15.5:0/619509346 wait complete. 2024-05-20T18:29:41.696 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-05-20T18:29:41.832 INFO:teuthology.orchestra.run.smithi005.stdout:{"message":"","return_code":0} 2024-05-20T18:29:41.833 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-20T18:29:41.833 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-05-20T18:29:41.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.889+0000 7f3faa9d2f40 1 Processor -- start 2024-05-20T18:29:41.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.889+0000 7f3faa9d2f40 1 -- start start 2024-05-20T18:29:41.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.890+0000 7f3faa9d2f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563aea31f40 0x5563aea302f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.890+0000 7f3faa9d2f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563aea33170 0x5563aeab9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.890+0000 7f3faa9d2f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5563aea33580 0x5563aeac0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.890+0000 7f3faa9d2f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5563aea34b60 con 0x5563aea33170 2024-05-20T18:29:41.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.890+0000 7f3faa9d2f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5563ae993c80 con 0x5563aea31f40 2024-05-20T18:29:41.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.890+0000 7f3faa9d2f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5563aea34e00 con 0x5563aea33580 2024-05-20T18:29:41.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.891+0000 7f3fa3ca0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563aea33170 0x5563aeab9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.891+0000 7f3fa3ca0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563aea33170 0x5563aeab9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35230/0 (socket says 172.21.15.5:35230) 2024-05-20T18:29:41.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.891+0000 7f3fa3ca0640 1 -- 172.21.15.5:0/53517317 learned_addr learned my addr 172.21.15.5:0/53517317 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:41.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.891+0000 7f3fa3ca0640 1 -- 172.21.15.5:0/53517317 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5563aea33580 msgr2=0x5563aeac0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:41.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.891+0000 7f3fa4ca2640 1 --2- 172.21.15.5:0/53517317 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5563aea33580 0x5563aeac0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.891+0000 7f3fa44a1640 1 --2- 172.21.15.5:0/53517317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563aea31f40 0x5563aea302f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.891+0000 7f3fa3ca0640 1 --2- 172.21.15.5:0/53517317 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5563aea33580 0x5563aeac0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.891+0000 7f3fa3ca0640 1 -- 172.21.15.5:0/53517317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563aea31f40 msgr2=0x5563aea302f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:41.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.891+0000 7f3fa3ca0640 1 --2- 172.21.15.5:0/53517317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563aea31f40 0x5563aea302f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.891+0000 7f3fa3ca0640 1 -- 172.21.15.5:0/53517317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563ae918cb0 con 0x5563aea33170 2024-05-20T18:29:41.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.891+0000 7f3fa44a1640 1 --2- 172.21.15.5:0/53517317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563aea31f40 0x5563aea302f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:29:41.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.891+0000 7f3fa3ca0640 1 --2- 172.21.15.5:0/53517317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563aea33170 0x5563aeab9b30 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3f98045df0 tx=0x7f3f9804e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dde3204f5d3b6cb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:41.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.891+0000 7f3fa2c9e640 1 -- 172.21.15.5:0/53517317 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98065070 con 0x5563aea33170 2024-05-20T18:29:41.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.891+0000 7f3fa2c9e640 1 -- 172.21.15.5:0/53517317 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f980549f0 con 0x5563aea33170 2024-05-20T18:29:41.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.891+0000 7f3fa2c9e640 1 -- 172.21.15.5:0/53517317 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98059040 con 0x5563aea33170 2024-05-20T18:29:41.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.892+0000 7f3faa9d2f40 1 -- 172.21.15.5:0/53517317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563aea33170 msgr2=0x5563aeab9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:41.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.892+0000 7f3faa9d2f40 1 --2- 172.21.15.5:0/53517317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563aea33170 0x5563aeab9b30 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3f98045df0 tx=0x7f3f9804e000 comp rx=0 tx=0).stop 2024-05-20T18:29:41.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.892+0000 7f3faa9d2f40 1 -- 172.21.15.5:0/53517317 shutdown_connections 2024-05-20T18:29:41.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.892+0000 7f3faa9d2f40 1 --2- 172.21.15.5:0/53517317 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5563aea33580 0x5563aeac0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.892+0000 7f3faa9d2f40 1 --2- 172.21.15.5:0/53517317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563aea33170 0x5563aeab9b30 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.892+0000 7f3faa9d2f40 1 --2- 172.21.15.5:0/53517317 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563aea31f40 0x5563aea302f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.892+0000 7f3faa9d2f40 1 -- 172.21.15.5:0/53517317 >> 172.21.15.5:0/53517317 conn(0x5563ae911ac0 msgr2=0x5563aeac4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:41.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.892+0000 7f3faa9d2f40 1 -- 172.21.15.5:0/53517317 shutdown_connections 2024-05-20T18:29:41.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.892+0000 7f3faa9d2f40 1 -- 172.21.15.5:0/53517317 wait complete. 2024-05-20T18:29:41.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.893+0000 7f3faa9d2f40 1 Processor -- start 2024-05-20T18:29:41.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.893+0000 7f3faa9d2f40 1 -- start start 2024-05-20T18:29:41.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.894+0000 7f3faa9d2f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563aea31f40 0x5563aeac5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.894+0000 7f3faa9d2f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5563aea33170 0x5563aeabc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.894+0000 7f3faa9d2f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563aea33580 0x5563aea32540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.894+0000 7f3faa9d2f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5563aea188b0 con 0x5563aea31f40 2024-05-20T18:29:41.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.894+0000 7f3faa9d2f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5563aeabec20 con 0x5563aea33580 2024-05-20T18:29:41.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.894+0000 7f3faa9d2f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5563aeabd180 con 0x5563aea33170 2024-05-20T18:29:41.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.894+0000 7f3fa44a1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563aea31f40 0x5563aeac5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.894+0000 7f3fa44a1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563aea31f40 0x5563aeac5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35234/0 (socket says 172.21.15.5:35234) 2024-05-20T18:29:41.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.894+0000 7f3fa44a1640 1 -- 172.21.15.5:0/4292273294 learned_addr learned my addr 172.21.15.5:0/4292273294 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:41.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.895+0000 7f3fa4ca2640 1 --2- 172.21.15.5:0/4292273294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563aea33580 0x5563aea32540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.895+0000 7f3fa3ca0640 1 --2- 172.21.15.5:0/4292273294 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5563aea33170 0x5563aeabc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.895+0000 7f3fa44a1640 1 -- 172.21.15.5:0/4292273294 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5563aea33170 msgr2=0x5563aeabc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:41.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.895+0000 7f3fa44a1640 1 --2- 172.21.15.5:0/4292273294 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5563aea33170 0x5563aeabc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.895+0000 7f3fa44a1640 1 -- 172.21.15.5:0/4292273294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563aea33580 msgr2=0x5563aea32540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:41.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.895+0000 7f3fa44a1640 1 --2- 172.21.15.5:0/4292273294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563aea33580 0x5563aea32540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.895+0000 7f3fa44a1640 1 -- 172.21.15.5:0/4292273294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563aeb552b0 con 0x5563aea31f40 2024-05-20T18:29:41.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.895+0000 7f3fa4ca2640 1 --2- 172.21.15.5:0/4292273294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5563aea33580 0x5563aea32540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:29:41.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.895+0000 7f3fa3ca0640 1 --2- 172.21.15.5:0/4292273294 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5563aea33170 0x5563aeabc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:29:41.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.895+0000 7f3fa44a1640 1 --2- 172.21.15.5:0/4292273294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563aea31f40 0x5563aeac5010 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3f9400bee0 tx=0x7f3f940073f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:41.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.895+0000 7f3fa1c9c640 1 -- 172.21.15.5:0/4292273294 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94007bf0 con 0x5563aea31f40 2024-05-20T18:29:41.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.895+0000 7f3fa1c9c640 1 -- 172.21.15.5:0/4292273294 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f94007d90 con 0x5563aea31f40 2024-05-20T18:29:41.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.895+0000 7f3faa9d2f40 1 -- 172.21.15.5:0/4292273294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5563aeb54cf0 con 0x5563aea31f40 2024-05-20T18:29:41.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.895+0000 7f3fa1c9c640 1 -- 172.21.15.5:0/4292273294 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94015660 con 0x5563aea31f40 2024-05-20T18:29:41.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.895+0000 7f3faa9d2f40 1 -- 172.21.15.5:0/4292273294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563aeabaa80 con 0x5563aea31f40 2024-05-20T18:29:41.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.897+0000 7f3fa1c9c640 1 -- 172.21.15.5:0/4292273294 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3f9401e440 con 0x5563aea31f40 2024-05-20T18:29:41.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.897+0000 7f3fa1c9c640 1 --2- 172.21.15.5:0/4292273294 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3f9006a810 0x7f3f9006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.897+0000 7f3fa3ca0640 1 --2- 172.21.15.5:0/4292273294 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3f9006a810 0x7f3f9006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.897+0000 7f3fa1c9c640 1 -- 172.21.15.5:0/4292273294 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f3f94089410 con 0x5563aea31f40 2024-05-20T18:29:41.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.897+0000 7f3faa9d2f40 1 --2- 172.21.15.5:0/4292273294 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x5563aeb5df90 0x5563aeb5e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:41.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.897+0000 7f3faa9d2f40 1 -- 172.21.15.5:0/4292273294 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51) -- 0x5563aeabcaf0 con 0x5563aeb5df90 2024-05-20T18:29:41.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.898+0000 7f3fa4ca2640 1 --2- 172.21.15.5:0/4292273294 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x5563aeb5df90 0x5563aeb5e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:41.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.898+0000 7f3fa3ca0640 1 --2- 172.21.15.5:0/4292273294 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3f9006a810 0x7f3f9006cd20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3f98045df0 tx=0x7f3f98056040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:41.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.898+0000 7f3fa4ca2640 1 --2- 172.21.15.5:0/4292273294 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x5563aeb5df90 0x5563aeb5e3a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:41.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.898+0000 7f3fa4ca2640 1 -- 172.21.15.5:0/4292273294 <== osd.1 v2:172.21.15.5:6815/550664725 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7f3f9c001750 con 0x5563aeb5df90 2024-05-20T18:29:41.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.898+0000 7f3faa9d2f40 1 -- 172.21.15.5:0/4292273294 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x5563aeb5df90 msgr2=0x5563aeb5e3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:41.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.898+0000 7f3faa9d2f40 1 --2- 172.21.15.5:0/4292273294 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x5563aeb5df90 0x5563aeb5e3a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.898+0000 7f3faa9d2f40 1 -- 172.21.15.5:0/4292273294 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3f9006a810 msgr2=0x7f3f9006cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:41.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.898+0000 7f3faa9d2f40 1 --2- 172.21.15.5:0/4292273294 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3f9006a810 0x7f3f9006cd20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3f98045df0 tx=0x7f3f98056040 comp rx=0 tx=0).stop 2024-05-20T18:29:41.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.898+0000 7f3faa9d2f40 1 -- 172.21.15.5:0/4292273294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563aea31f40 msgr2=0x5563aeac5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:41.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.898+0000 7f3faa9d2f40 1 --2- 172.21.15.5:0/4292273294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563aea31f40 0x5563aeac5010 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3f9400bee0 tx=0x7f3f940073f0 comp rx=0 tx=0).stop 2024-05-20T18:29:41.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.898+0000 7f3fa4ca2640 1 -- 172.21.15.5:0/4292273294 reap_dead start 2024-05-20T18:29:41.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.898+0000 7f3faa9d2f40 1 -- 172.21.15.5:0/4292273294 shutdown_connections 2024-05-20T18:29:41.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.898+0000 7f3faa9d2f40 1 -- 172.21.15.5:0/4292273294 >> 172.21.15.5:0/4292273294 conn(0x5563ae911ac0 msgr2=0x5563aea34340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:41.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.899+0000 7f3faa9d2f40 1 -- 172.21.15.5:0/4292273294 shutdown_connections 2024-05-20T18:29:41.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:41.899+0000 7f3faa9d2f40 1 -- 172.21.15.5:0/4292273294 wait complete. 2024-05-20T18:29:41.902 INFO:teuthology.orchestra.run.smithi005.stdout:Yalphaparent_a 2024-05-20T18:29:41.903 DEBUG:teuthology.orchestra.run.smithi005:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-20T18:29:41.958 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:29:41.959 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 1099511627777, 2024-05-20T18:29:41.959 INFO:teuthology.orchestra.run.smithi005.stdout: "ancestors": [ 2024-05-20T18:29:41.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:29:41.959 INFO:teuthology.orchestra.run.smithi005.stdout: "dirino": 1099511627776, 2024-05-20T18:29:41.959 INFO:teuthology.orchestra.run.smithi005.stdout: "dname": "alpha", 2024-05-20T18:29:41.959 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 2 2024-05-20T18:29:41.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:29:41.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:29:41.959 INFO:teuthology.orchestra.run.smithi005.stdout: "dirino": 1, 2024-05-20T18:29:41.959 INFO:teuthology.orchestra.run.smithi005.stdout: "dname": "parent_a", 2024-05-20T18:29:41.959 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 4 2024-05-20T18:29:41.960 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:29:41.960 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-20T18:29:41.960 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 8, 2024-05-20T18:29:41.960 INFO:teuthology.orchestra.run.smithi005.stdout: "old_pools": [] 2024-05-20T18:29:41.960 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:29:41.960 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:29:41.962 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-20T18:29:41.962 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T18:29:42.039 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T18:29:42.085 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-20T18:29:42.085 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T18:29:42.085 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:29:42.085 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:29:42.101 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:29:42.102 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:29:42.158 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:29:42.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.313+0000 7f8f8cc92640 1 Processor -- start 2024-05-20T18:29:42.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.313+0000 7f8f8cc92640 1 -- start start 2024-05-20T18:29:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.314+0000 7f8f8cc92640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8f88163a10 0x7f8f880d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.314+0000 7f8f8cc92640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8f880b4010 0x7f8f880cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.314+0000 7f8f8cc92640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8f880d14b0 0x7f8f880d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.314+0000 7f8f8cc92640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f8f880d3ca0 con 0x7f8f880d14b0 2024-05-20T18:29:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.314+0000 7f8f8cc92640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8f880d0370 con 0x7f8f880b4010 2024-05-20T18:29:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.314+0000 7f8f8cc92640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8f880d0650 con 0x7f8f88163a10 2024-05-20T18:29:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.314+0000 7f8f86d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8f880d14b0 0x7f8f880d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.314+0000 7f8f86d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8f880d14b0 0x7f8f880d18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35240/0 (socket says 172.21.15.5:35240) 2024-05-20T18:29:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.314+0000 7f8f86d9d640 1 -- 172.21.15.5:0/3164103190 learned_addr learned my addr 172.21.15.5:0/3164103190 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.315+0000 7f8f85d9b640 1 --2- 172.21.15.5:0/3164103190 >> v2:172.21.15.63:3300/0 conn(0x7f8f880b4010 0x7f8f880cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.315+0000 7f8f8659c640 1 --2- 172.21.15.5:0/3164103190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8f88163a10 0x7f8f880d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.315+0000 7f8f86d9d640 1 -- 172.21.15.5:0/3164103190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8f88163a10 msgr2=0x7f8f880d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.315+0000 7f8f86d9d640 1 --2- 172.21.15.5:0/3164103190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8f88163a10 0x7f8f880d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.315+0000 7f8f86d9d640 1 -- 172.21.15.5:0/3164103190 >> v2:172.21.15.63:3300/0 conn(0x7f8f880b4010 msgr2=0x7f8f880cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.315+0000 7f8f86d9d640 1 --2- 172.21.15.5:0/3164103190 >> v2:172.21.15.63:3300/0 conn(0x7f8f880b4010 0x7f8f880cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.315+0000 7f8f86d9d640 1 -- 172.21.15.5:0/3164103190 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f880d3650 con 0x7f8f880d14b0 2024-05-20T18:29:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.315+0000 7f8f85d9b640 1 --2- 172.21.15.5:0/3164103190 >> v2:172.21.15.63:3300/0 conn(0x7f8f880b4010 0x7f8f880cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:29:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.315+0000 7f8f86d9d640 1 --2- 172.21.15.5:0/3164103190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f880d14b0 0x7f8f880d18c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8f880b7710 tx=0x7f8f8002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e393d29eed85b8e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:42.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.316+0000 7f8f8559a640 1 -- 172.21.15.5:0/3164103190 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80037070 con 0x7f8f880d14b0 2024-05-20T18:29:42.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.316+0000 7f8f8559a640 1 -- 172.21.15.5:0/3164103190 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f80032440 con 0x7f8f880d14b0 2024-05-20T18:29:42.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.316+0000 7f8f8559a640 1 -- 172.21.15.5:0/3164103190 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80032810 con 0x7f8f880d14b0 2024-05-20T18:29:42.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.316+0000 7f8f8cc92640 1 -- 172.21.15.5:0/3164103190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f880d14b0 msgr2=0x7f8f880d18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:42.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.316+0000 7f8f8cc92640 1 --2- 172.21.15.5:0/3164103190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f880d14b0 0x7f8f880d18c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8f880b7710 tx=0x7f8f8002a1e0 comp rx=0 tx=0).stop 2024-05-20T18:29:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.316+0000 7f8f8cc92640 1 -- 172.21.15.5:0/3164103190 shutdown_connections 2024-05-20T18:29:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.316+0000 7f8f8cc92640 1 --2- 172.21.15.5:0/3164103190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f880d14b0 0x7f8f880d18c0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.316+0000 7f8f8cc92640 1 --2- 172.21.15.5:0/3164103190 >> v2:172.21.15.63:3300/0 conn(0x7f8f880b4010 0x7f8f880cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.316+0000 7f8f8cc92640 1 --2- 172.21.15.5:0/3164103190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8f88163a10 0x7f8f880d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.316+0000 7f8f8cc92640 1 -- 172.21.15.5:0/3164103190 >> 172.21.15.5:0/3164103190 conn(0x7f8f8806d240 msgr2=0x7f8f8806d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.316+0000 7f8f8cc92640 1 -- 172.21.15.5:0/3164103190 shutdown_connections 2024-05-20T18:29:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.316+0000 7f8f8cc92640 1 -- 172.21.15.5:0/3164103190 wait complete. 2024-05-20T18:29:42.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.318+0000 7f8f8cc92640 1 Processor -- start 2024-05-20T18:29:42.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.318+0000 7f8f8cc92640 1 -- start start 2024-05-20T18:29:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.320+0000 7f8f8cc92640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f880b4010 0x7f8f881e8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.320+0000 7f8f8cc92640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8f880d14b0 0x7f8f88146e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.320+0000 7f8f8cc92640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f88163a10 0x7f8f881eb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.320+0000 7f8f8cc92640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8f880d1e00 con 0x7f8f88163a10 2024-05-20T18:29:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.320+0000 7f8f8cc92640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8f880d0040 con 0x7f8f880b4010 2024-05-20T18:29:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.320+0000 7f8f8cc92640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8f880b8210 con 0x7f8f880d14b0 2024-05-20T18:29:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.320+0000 7f8f86d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f88163a10 0x7f8f881eb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.320+0000 7f8f86d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f88163a10 0x7f8f881eb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35246/0 (socket says 172.21.15.5:35246) 2024-05-20T18:29:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.320+0000 7f8f86d9d640 1 -- 172.21.15.5:0/3779158752 learned_addr learned my addr 172.21.15.5:0/3779158752 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.320+0000 7f8f8659c640 1 --2- 172.21.15.5:0/3779158752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f880b4010 0x7f8f881e8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.320+0000 7f8f85d9b640 1 --2- 172.21.15.5:0/3779158752 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8f880d14b0 0x7f8f88146e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.320+0000 7f8f86d9d640 1 -- 172.21.15.5:0/3779158752 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8f880d14b0 msgr2=0x7f8f88146e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.320+0000 7f8f86d9d640 1 --2- 172.21.15.5:0/3779158752 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8f880d14b0 0x7f8f88146e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.320+0000 7f8f86d9d640 1 -- 172.21.15.5:0/3779158752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f880b4010 msgr2=0x7f8f881e8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.320+0000 7f8f86d9d640 1 --2- 172.21.15.5:0/3779158752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f880b4010 0x7f8f881e8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.320+0000 7f8f86d9d640 1 -- 172.21.15.5:0/3779158752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f881473c0 con 0x7f8f88163a10 2024-05-20T18:29:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.320+0000 7f8f86d9d640 1 --2- 172.21.15.5:0/3779158752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f88163a10 0x7f8f881eb1b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f8f800090b0 tx=0x7f8f800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.321+0000 7f8f7effd640 1 -- 172.21.15.5:0/3779158752 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80041020 con 0x7f8f88163a10 2024-05-20T18:29:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.321+0000 7f8f7effd640 1 -- 172.21.15.5:0/3779158752 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f80030990 con 0x7f8f88163a10 2024-05-20T18:29:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.321+0000 7f8f7effd640 1 -- 172.21.15.5:0/3779158752 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80037040 con 0x7f8f88163a10 2024-05-20T18:29:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.321+0000 7f8f8cc92640 1 -- 172.21.15.5:0/3779158752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f881e9380 con 0x7f8f88163a10 2024-05-20T18:29:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.321+0000 7f8f8cc92640 1 -- 172.21.15.5:0/3779158752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f881e9940 con 0x7f8f88163a10 2024-05-20T18:29:42.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.322+0000 7f8f8cc92640 1 -- 172.21.15.5:0/3779158752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f881e90f0 con 0x7f8f88163a10 2024-05-20T18:29:42.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.322+0000 7f8f7effd640 1 -- 172.21.15.5:0/3779158752 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8f80044020 con 0x7f8f88163a10 2024-05-20T18:29:42.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.323+0000 7f8f7effd640 1 --2- 172.21.15.5:0/3779158752 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8f6806be60 0x7f8f6806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:42.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.323+0000 7f8f7effd640 1 -- 172.21.15.5:0/3779158752 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f8f800ab9d0 con 0x7f8f88163a10 2024-05-20T18:29:42.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.323+0000 7f8f8659c640 1 --2- 172.21.15.5:0/3779158752 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8f6806be60 0x7f8f6806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:42.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.323+0000 7f8f8659c640 1 --2- 172.21.15.5:0/3779158752 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8f6806be60 0x7f8f6806e370 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8f780070b0 tx=0x7f8f78007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:42.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.327+0000 7f8f7effd640 1 -- 172.21.15.5:0/3779158752 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8f8004b030 con 0x7f8f88163a10 2024-05-20T18:29:42.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.511+0000 7f8f8cc92640 1 -- 172.21.15.5:0/3779158752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f880d3050 con 0x7f8f88163a10 2024-05-20T18:29:42.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.512+0000 7f8f7effd640 1 -- 172.21.15.5:0/3779158752 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+16822 (secure 0 0 0) 0x7f8f8007a1f0 con 0x7f8f88163a10 2024-05-20T18:29:42.512 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:29:42.512 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":51,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:29:03.665286+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:29:00.401866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-20T18:29:01.006122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:29:42.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.514+0000 7f8f8cc92640 1 -- 172.21.15.5:0/3779158752 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8f6806be60 msgr2=0x7f8f6806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:42.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.514+0000 7f8f8cc92640 1 --2- 172.21.15.5:0/3779158752 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8f6806be60 0x7f8f6806e370 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8f780070b0 tx=0x7f8f78007040 comp rx=0 tx=0).stop 2024-05-20T18:29:42.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.514+0000 7f8f8cc92640 1 -- 172.21.15.5:0/3779158752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f88163a10 msgr2=0x7f8f881eb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:42.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.514+0000 7f8f8cc92640 1 --2- 172.21.15.5:0/3779158752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f88163a10 0x7f8f881eb1b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f8f800090b0 tx=0x7f8f800046c0 comp rx=0 tx=0).stop 2024-05-20T18:29:42.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.514+0000 7f8f8cc92640 1 -- 172.21.15.5:0/3779158752 shutdown_connections 2024-05-20T18:29:42.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.514+0000 7f8f8cc92640 1 --2- 172.21.15.5:0/3779158752 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8f6806be60 0x7f8f6806e370 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:42.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.514+0000 7f8f8cc92640 1 --2- 172.21.15.5:0/3779158752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f88163a10 0x7f8f881eb1b0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:42.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.514+0000 7f8f8cc92640 1 --2- 172.21.15.5:0/3779158752 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8f880d14b0 0x7f8f88146e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:42.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.514+0000 7f8f8cc92640 1 --2- 172.21.15.5:0/3779158752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f880b4010 0x7f8f881e8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:42.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.514+0000 7f8f8cc92640 1 -- 172.21.15.5:0/3779158752 >> 172.21.15.5:0/3779158752 conn(0x7f8f8806d240 msgr2=0x7f8f880b4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:42.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.515+0000 7f8f8cc92640 1 -- 172.21.15.5:0/3779158752 shutdown_connections 2024-05-20T18:29:42.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.515+0000 7f8f8cc92640 1 -- 172.21.15.5:0/3779158752 wait complete. 2024-05-20T18:29:42.526 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:29:00.401866+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:29:01.006122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:29:42.526 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:29:42.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.645+0000 7f5e3a59d640 1 Processor -- start 2024-05-20T18:29:42.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.645+0000 7f5e3a59d640 1 -- start start 2024-05-20T18:29:42.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.646+0000 7f5e3a59d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5e34074f20 0x7f5e3415ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:42.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.646+0000 7f5e3a59d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e340b2b70 0x7f5e340b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:42.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.646+0000 7f5e3a59d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e34160520 con 0x7f5e3415ccb0 2024-05-20T18:29:42.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.646+0000 7f5e3a59d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f5e3406c2b0 con 0x7f5e34074f20 2024-05-20T18:29:42.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.646+0000 7f5e3a59d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5e340b34c0 con 0x7f5e340b2b70 2024-05-20T18:29:42.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.646+0000 7f5e3959b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5e34074f20 0x7f5e3415ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:42.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.646+0000 7f5e38d9a640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5e3415ccb0 0x7f5e3406c740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44694/0 (socket says 172.21.15.5:44694) 2024-05-20T18:29:42.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.646+0000 7f5e3959b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5e34074f20 0x7f5e3415ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35258/0 (socket says 172.21.15.5:35258) 2024-05-20T18:29:42.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.646+0000 7f5e38d9a640 1 -- 172.21.15.5:0/452179297 learned_addr learned my addr 172.21.15.5:0/452179297 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:42.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.647+0000 7f5e39d9c640 1 --2- 172.21.15.5:0/452179297 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e340b2b70 0x7f5e340b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:42.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.647+0000 7f5e3959b640 1 -- 172.21.15.5:0/452179297 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e340b2b70 msgr2=0x7f5e340b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:42.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.647+0000 7f5e3959b640 1 --2- 172.21.15.5:0/452179297 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e340b2b70 0x7f5e340b2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:42.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.647+0000 7f5e3959b640 1 -- 172.21.15.5:0/452179297 >> v1:172.21.15.5:6789/0 conn(0x7f5e3415ccb0 legacy=0x7f5e3406c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:42.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.647+0000 7f5e3959b640 1 -- 172.21.15.5:0/452179297 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e3406ce50 con 0x7f5e34074f20 2024-05-20T18:29:42.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.647+0000 7f5e33fff640 1 -- 172.21.15.5:0/452179297 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 725934311 0 0) 0x7f5e28001110 con 0x7f5e3415ccb0 2024-05-20T18:29:42.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.648+0000 7f5e3959b640 1 --2- 172.21.15.5:0/452179297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e34074f20 0x7f5e3415ffe0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009d90 tx=0x7f5e2402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff3265d88eb353a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:42.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.648+0000 7f5e33fff640 1 -- 172.21.15.5:0/452179297 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e24007740 con 0x7f5e34074f20 2024-05-20T18:29:42.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.648+0000 7f5e33fff640 1 -- 172.21.15.5:0/452179297 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e240078e0 con 0x7f5e34074f20 2024-05-20T18:29:42.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.648+0000 7f5e33fff640 1 -- 172.21.15.5:0/452179297 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e24007cb0 con 0x7f5e34074f20 2024-05-20T18:29:42.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.648+0000 7f5e3a59d640 1 -- 172.21.15.5:0/452179297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e34074f20 msgr2=0x7f5e3415ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:42.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.648+0000 7f5e3a59d640 1 --2- 172.21.15.5:0/452179297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e34074f20 0x7f5e3415ffe0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009d90 tx=0x7f5e2402a660 comp rx=0 tx=0).stop 2024-05-20T18:29:42.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.648+0000 7f5e3a59d640 1 -- 172.21.15.5:0/452179297 shutdown_connections 2024-05-20T18:29:42.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.648+0000 7f5e3a59d640 1 --2- 172.21.15.5:0/452179297 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e340b2b70 0x7f5e340b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:42.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.648+0000 7f5e3a59d640 1 --2- 172.21.15.5:0/452179297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e34074f20 0x7f5e3415ffe0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:42.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.648+0000 7f5e3a59d640 1 -- 172.21.15.5:0/452179297 >> 172.21.15.5:0/452179297 conn(0x7f5e34072c70 msgr2=0x7f5e340b5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:42.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.649+0000 7f5e3a59d640 1 -- 172.21.15.5:0/452179297 shutdown_connections 2024-05-20T18:29:42.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.649+0000 7f5e3a59d640 1 -- 172.21.15.5:0/452179297 wait complete. 2024-05-20T18:29:42.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.650+0000 7f5e3a59d640 1 Processor -- start 2024-05-20T18:29:42.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.650+0000 7f5e3a59d640 1 -- start start 2024-05-20T18:29:42.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.652+0000 7f5e3a59d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e34074f20 0x7f5e34155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:42.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.652+0000 7f5e3a59d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e340b2b70 0x7f5e340aea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:42.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.652+0000 7f5e3a59d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e3415ccb0 0x7f5e341f5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:42.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.652+0000 7f5e3a59d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3406bc60 con 0x7f5e34074f20 2024-05-20T18:29:42.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.652+0000 7f5e3a59d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3406b980 con 0x7f5e340b2b70 2024-05-20T18:29:42.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.652+0000 7f5e3a59d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5e3406bf40 con 0x7f5e3415ccb0 2024-05-20T18:29:42.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.652+0000 7f5e3959b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e34074f20 0x7f5e34155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:42.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.652+0000 7f5e3959b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e34074f20 0x7f5e34155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35262/0 (socket says 172.21.15.5:35262) 2024-05-20T18:29:42.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.652+0000 7f5e3959b640 1 -- 172.21.15.5:0/1763372085 learned_addr learned my addr 172.21.15.5:0/1763372085 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:42.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.652+0000 7f5e39d9c640 1 --2- 172.21.15.5:0/1763372085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e3415ccb0 0x7f5e341f5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:42.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.652+0000 7f5e38d9a640 1 --2- 172.21.15.5:0/1763372085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e340b2b70 0x7f5e340aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:42.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.652+0000 7f5e3959b640 1 -- 172.21.15.5:0/1763372085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e3415ccb0 msgr2=0x7f5e341f5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:42.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.652+0000 7f5e3959b640 1 --2- 172.21.15.5:0/1763372085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e3415ccb0 0x7f5e341f5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:42.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.652+0000 7f5e3959b640 1 -- 172.21.15.5:0/1763372085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e340b2b70 msgr2=0x7f5e340aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:42.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.652+0000 7f5e3959b640 1 --2- 172.21.15.5:0/1763372085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e340b2b70 0x7f5e340aea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:42.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.652+0000 7f5e3959b640 1 -- 172.21.15.5:0/1763372085 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e340b0b60 con 0x7f5e34074f20 2024-05-20T18:29:42.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.652+0000 7f5e3959b640 1 --2- 172.21.15.5:0/1763372085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e34074f20 0x7f5e34155a00 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009eb0 tx=0x7f5e24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:42.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.652+0000 7f5e31ffb640 1 -- 172.21.15.5:0/1763372085 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2403f070 con 0x7f5e34074f20 2024-05-20T18:29:42.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.653+0000 7f5e31ffb640 1 -- 172.21.15.5:0/1763372085 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e24031c40 con 0x7f5e34074f20 2024-05-20T18:29:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.653+0000 7f5e31ffb640 1 -- 172.21.15.5:0/1763372085 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e24044460 con 0x7f5e34074f20 2024-05-20T18:29:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.653+0000 7f5e3a59d640 1 -- 172.21.15.5:0/1763372085 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e340af5c0 con 0x7f5e34074f20 2024-05-20T18:29:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.653+0000 7f5e3a59d640 1 -- 172.21.15.5:0/1763372085 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e340afb80 con 0x7f5e34074f20 2024-05-20T18:29:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.654+0000 7f5e3a59d640 1 -- 172.21.15.5:0/1763372085 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e34075980 con 0x7f5e34074f20 2024-05-20T18:29:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.654+0000 7f5e31ffb640 1 -- 172.21.15.5:0/1763372085 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5e2403b410 con 0x7f5e34074f20 2024-05-20T18:29:42.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.655+0000 7f5e31ffb640 1 --2- 172.21.15.5:0/1763372085 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5e2006be30 0x7f5e2006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:42.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.655+0000 7f5e31ffb640 1 -- 172.21.15.5:0/1763372085 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f5e240ac8b0 con 0x7f5e34074f20 2024-05-20T18:29:42.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.655+0000 7f5e38d9a640 1 --2- 172.21.15.5:0/1763372085 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5e2006be30 0x7f5e2006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:42.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.655+0000 7f5e38d9a640 1 --2- 172.21.15.5:0/1763372085 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5e2006be30 0x7f5e2006e340 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5e3415a160 tx=0x7f5e28007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:42.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.660+0000 7f5e31ffb640 1 -- 172.21.15.5:0/1763372085 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5e2407b040 con 0x7f5e34074f20 2024-05-20T18:29:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.849+0000 7f5e3a59d640 1 -- 172.21.15.5:0/1763372085 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5e34155f40 con 0x7f5e34074f20 2024-05-20T18:29:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.850+0000 7f5e31ffb640 1 -- 172.21.15.5:0/1763372085 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) ==== 93+0+10 (secure 0 0 0) 0x7f5e2407a940 con 0x7f5e34074f20 2024-05-20T18:29:42.850 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:29:42.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.851+0000 7f5e3a59d640 1 -- 172.21.15.5:0/1763372085 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5e2006be30 msgr2=0x7f5e2006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:42.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.851+0000 7f5e3a59d640 1 --2- 172.21.15.5:0/1763372085 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5e2006be30 0x7f5e2006e340 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5e3415a160 tx=0x7f5e28007c00 comp rx=0 tx=0).stop 2024-05-20T18:29:42.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.851+0000 7f5e3a59d640 1 -- 172.21.15.5:0/1763372085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e34074f20 msgr2=0x7f5e34155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:42.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.851+0000 7f5e3a59d640 1 --2- 172.21.15.5:0/1763372085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e34074f20 0x7f5e34155a00 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009eb0 tx=0x7f5e24004a40 comp rx=0 tx=0).stop 2024-05-20T18:29:42.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.852+0000 7f5e3a59d640 1 -- 172.21.15.5:0/1763372085 shutdown_connections 2024-05-20T18:29:42.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.852+0000 7f5e3a59d640 1 --2- 172.21.15.5:0/1763372085 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5e2006be30 0x7f5e2006e340 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:42.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.852+0000 7f5e3a59d640 1 --2- 172.21.15.5:0/1763372085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e3415ccb0 0x7f5e341f5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:42.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.852+0000 7f5e3a59d640 1 --2- 172.21.15.5:0/1763372085 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e340b2b70 0x7f5e340aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:42.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.852+0000 7f5e3a59d640 1 --2- 172.21.15.5:0/1763372085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e34074f20 0x7f5e34155a00 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:42.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.852+0000 7f5e3a59d640 1 -- 172.21.15.5:0/1763372085 >> 172.21.15.5:0/1763372085 conn(0x7f5e34072c70 msgr2=0x7f5e3415d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:42.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.852+0000 7f5e3a59d640 1 -- 172.21.15.5:0/1763372085 shutdown_connections 2024-05-20T18:29:42.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.852+0000 7f5e3a59d640 1 -- 172.21.15.5:0/1763372085 wait complete. 2024-05-20T18:29:42.863 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:29:42.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.980+0000 7f1808814640 1 Processor -- start 2024-05-20T18:29:42.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.980+0000 7f1808814640 1 -- start start 2024-05-20T18:29:42.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.982+0000 7f1808814640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f180406d3d0 0x7f180406d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:42.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.982+0000 7f1808814640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f180406dd20 con 0x7f1804162820 2024-05-20T18:29:42.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.982+0000 7f1808814640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18040d0630 con 0x7f18040b8190 2024-05-20T18:29:42.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.982+0000 7f1808814640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f18040d39c0 con 0x7f180406d3d0 2024-05-20T18:29:42.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.982+0000 7f180259c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1804162820 0x7f18040b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44696/0 (socket says 172.21.15.5:44696) 2024-05-20T18:29:42.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.982+0000 7f180259c640 1 -- 172.21.15.5:0/1197244153 learned_addr learned my addr 172.21.15.5:0/1197244153 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:42.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.982+0000 7f1801d9b640 1 --2- 172.21.15.5:0/1197244153 >> v2:172.21.15.63:3300/0 conn(0x7f180406d3d0 0x7f180406d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:42.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.982+0000 7f180159a640 1 -- 172.21.15.5:0/1197244153 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2285754383 0 0) 0x7f17f4001490 con 0x7f1804162820 2024-05-20T18:29:42.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.982+0000 7f180159a640 1 -- 172.21.15.5:0/1197244153 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17f0003520 con 0x7f1804162820 2024-05-20T18:29:42.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.983+0000 7f180159a640 1 -- 172.21.15.5:0/1197244153 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1160983628 0 0) 0x7f17f0003520 con 0x7f1804162820 2024-05-20T18:29:42.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.983+0000 7f180159a640 1 -- 172.21.15.5:0/1197244153 >> v2:172.21.15.63:3300/0 conn(0x7f180406d3d0 msgr2=0x7f180406d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:42.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.983+0000 7f180159a640 1 --2- 172.21.15.5:0/1197244153 >> v2:172.21.15.63:3300/0 conn(0x7f180406d3d0 0x7f180406d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:42.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.983+0000 7f180159a640 1 -- 172.21.15.5:0/1197244153 >> v1:172.21.15.63:6789/0 conn(0x7f18040b8190 legacy=0x7f18040b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:42.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.983+0000 7f180159a640 1 -- 172.21.15.5:0/1197244153 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18040d3ca0 con 0x7f1804162820 2024-05-20T18:29:42.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.983+0000 7f180159a640 1 -- 172.21.15.5:0/1197244153 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f17f4001bc0 con 0x7f1804162820 2024-05-20T18:29:42.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.983+0000 7f180159a640 1 -- 172.21.15.5:0/1197244153 >> v1:172.21.15.5:6789/0 conn(0x7f1804162820 legacy=0x7f18040b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:42.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.983+0000 7f180159a640 1 --2- 172.21.15.5:0/1197244153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17f00039b0 0x7f17f0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:42.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.983+0000 7f180159a640 1 --2- 172.21.15.5:0/1197244153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17f0004800 0x7f17f0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:42.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.983+0000 7f180159a640 1 --2- 172.21.15.5:0/1197244153 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17f0005150 0x7f17f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:42.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.983+0000 7f180159a640 1 -- 172.21.15.5:0/1197244153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f18040d39c0 con 0x7f17f00039b0 2024-05-20T18:29:42.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.983+0000 7f180159a640 1 -- 172.21.15.5:0/1197244153 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f17f000c0c0 con 0x7f17f0004800 2024-05-20T18:29:42.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.983+0000 7f180159a640 1 -- 172.21.15.5:0/1197244153 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f17f000c2c0 con 0x7f17f0005150 2024-05-20T18:29:42.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.983+0000 7f180259c640 1 --2- 172.21.15.5:0/1197244153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17f00039b0 0x7f17f0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:42.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.983+0000 7f1801d9b640 1 --2- 172.21.15.5:0/1197244153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17f0004800 0x7f17f0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:42.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.983+0000 7f1802d9d640 1 --2- 172.21.15.5:0/1197244153 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17f0005150 0x7f17f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:42.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.983+0000 7f180259c640 1 -- 172.21.15.5:0/1197244153 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17f0005150 msgr2=0x7f17f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:42.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.983+0000 7f180259c640 1 --2- 172.21.15.5:0/1197244153 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17f0005150 0x7f17f000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:42.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.983+0000 7f180259c640 1 -- 172.21.15.5:0/1197244153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17f0004800 msgr2=0x7f17f0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:42.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.983+0000 7f180259c640 1 --2- 172.21.15.5:0/1197244153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17f0004800 0x7f17f0004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:42.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.983+0000 7f180259c640 1 -- 172.21.15.5:0/1197244153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f0003520 con 0x7f17f00039b0 2024-05-20T18:29:42.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.983+0000 7f1802d9d640 1 -- 172.21.15.5:0/1197244153 reap_dead start 2024-05-20T18:29:42.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.984+0000 7f180259c640 1 --2- 172.21.15.5:0/1197244153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17f00039b0 0x7f17f0003de0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f17f400a9b0 tx=0x7f17f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c92a3f1ec49f191 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:42.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.984+0000 7f180159a640 1 -- 172.21.15.5:0/1197244153 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f400a5d0 con 0x7f17f00039b0 2024-05-20T18:29:42.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.984+0000 7f180159a640 1 -- 172.21.15.5:0/1197244153 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17f402f070 con 0x7f17f00039b0 2024-05-20T18:29:42.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.984+0000 7f180159a640 1 -- 172.21.15.5:0/1197244153 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f402bde0 con 0x7f17f00039b0 2024-05-20T18:29:42.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.985+0000 7f1808814640 1 -- 172.21.15.5:0/1197244153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17f00039b0 msgr2=0x7f17f0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:42.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.985+0000 7f1808814640 1 --2- 172.21.15.5:0/1197244153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17f00039b0 0x7f17f0003de0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f17f400a9b0 tx=0x7f17f0005830 comp rx=0 tx=0).stop 2024-05-20T18:29:42.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.985+0000 7f1808814640 1 -- 172.21.15.5:0/1197244153 shutdown_connections 2024-05-20T18:29:42.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.985+0000 7f1808814640 1 --2- 172.21.15.5:0/1197244153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17f00039b0 0x7f17f0003de0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:42.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.985+0000 7f1808814640 1 -- 172.21.15.5:0/1197244153 >> 172.21.15.5:0/1197244153 conn(0x7f18040d4bd0 msgr2=0x7f18040d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:42.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.985+0000 7f1808814640 1 -- 172.21.15.5:0/1197244153 shutdown_connections 2024-05-20T18:29:42.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.985+0000 7f1808814640 1 -- 172.21.15.5:0/1197244153 wait complete. 2024-05-20T18:29:42.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.987+0000 7f1808814640 1 Processor -- start 2024-05-20T18:29:42.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.987+0000 7f1808814640 1 -- start start 2024-05-20T18:29:42.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.988+0000 7f1808814640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17f00039b0 0x7f18040c8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:42.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.988+0000 7f1808814640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1804155070 0x7f18040b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:42.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.988+0000 7f1808814640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18040b3d70 0x7f18040ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:42.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.988+0000 7f1808814640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f18040d0300 con 0x7f18040b3d70 2024-05-20T18:29:42.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.988+0000 7f1808814640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f18040d4260 con 0x7f17f00039b0 2024-05-20T18:29:42.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.988+0000 7f1808814640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f18040d4780 con 0x7f1804155070 2024-05-20T18:29:42.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.988+0000 7f1802d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18040b3d70 0x7f18040ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:42.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.988+0000 7f1802d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18040b3d70 0x7f18040ca0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35270/0 (socket says 172.21.15.5:35270) 2024-05-20T18:29:42.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.988+0000 7f1802d9d640 1 -- 172.21.15.5:0/468321815 learned_addr learned my addr 172.21.15.5:0/468321815 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:42.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.989+0000 7f1801d9b640 1 --2- 172.21.15.5:0/468321815 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1804155070 0x7f18040b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:42.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.989+0000 7f180259c640 1 --2- 172.21.15.5:0/468321815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17f00039b0 0x7f18040c8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:42.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.989+0000 7f1802d9d640 1 -- 172.21.15.5:0/468321815 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1804155070 msgr2=0x7f18040b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:42.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.989+0000 7f1802d9d640 1 --2- 172.21.15.5:0/468321815 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1804155070 0x7f18040b3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:42.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.989+0000 7f1802d9d640 1 -- 172.21.15.5:0/468321815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17f00039b0 msgr2=0x7f18040c8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:42.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.989+0000 7f1802d9d640 1 --2- 172.21.15.5:0/468321815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17f00039b0 0x7f18040c8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:42.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.989+0000 7f1802d9d640 1 -- 172.21.15.5:0/468321815 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18040b4180 con 0x7f18040b3d70 2024-05-20T18:29:42.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.989+0000 7f1802d9d640 1 --2- 172.21.15.5:0/468321815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18040b3d70 0x7f18040ca0b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f180408d5e0 tx=0x7f17fc00e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:42.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.989+0000 7f17faffd640 1 -- 172.21.15.5:0/468321815 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc019070 con 0x7f18040b3d70 2024-05-20T18:29:42.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.989+0000 7f17faffd640 1 -- 172.21.15.5:0/468321815 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17fc005280 con 0x7f18040b3d70 2024-05-20T18:29:42.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.989+0000 7f17faffd640 1 -- 172.21.15.5:0/468321815 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc007030 con 0x7f18040b3d70 2024-05-20T18:29:42.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.989+0000 7f1808814640 1 -- 172.21.15.5:0/468321815 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18040c91d0 con 0x7f18040b3d70 2024-05-20T18:29:42.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.989+0000 7f1808814640 1 -- 172.21.15.5:0/468321815 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18040c99c0 con 0x7f18040b3d70 2024-05-20T18:29:42.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.990+0000 7f1808814640 1 -- 172.21.15.5:0/468321815 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18040c9450 con 0x7f18040b3d70 2024-05-20T18:29:42.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.991+0000 7f17faffd640 1 -- 172.21.15.5:0/468321815 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f17fc0071d0 con 0x7f18040b3d70 2024-05-20T18:29:42.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.992+0000 7f17faffd640 1 --2- 172.21.15.5:0/468321815 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f17e406be60 0x7f17e406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:42.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.992+0000 7f180259c640 1 --2- 172.21.15.5:0/468321815 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f17e406be60 0x7f17e406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:42.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.992+0000 7f17faffd640 1 -- 172.21.15.5:0/468321815 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f17fc08a470 con 0x7f18040b3d70 2024-05-20T18:29:42.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.992+0000 7f180259c640 1 --2- 172.21.15.5:0/468321815 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f17e406be60 0x7f17e406e370 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f17f4033000 tx=0x7f17f4007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:42.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:42.996+0000 7f17faffd640 1 -- 172.21.15.5:0/468321815 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f17fc058bf0 con 0x7f18040b3d70 2024-05-20T18:29:43.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.181+0000 7f1808814640 1 -- 172.21.15.5:0/468321815 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f18040b5f80 con 0x7f18040b3d70 2024-05-20T18:29:43.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.181+0000 7f17faffd640 1 -- 172.21.15.5:0/468321815 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v51) ==== 104+0+11 (secure 0 0 0) 0x7f17fc0584f0 con 0x7f18040b3d70 2024-05-20T18:29:43.181 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:29:43.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.183+0000 7f1808814640 1 -- 172.21.15.5:0/468321815 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f17e406be60 msgr2=0x7f17e406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:43.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.183+0000 7f1808814640 1 --2- 172.21.15.5:0/468321815 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f17e406be60 0x7f17e406e370 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f17f4033000 tx=0x7f17f4007000 comp rx=0 tx=0).stop 2024-05-20T18:29:43.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.183+0000 7f1808814640 1 -- 172.21.15.5:0/468321815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18040b3d70 msgr2=0x7f18040ca0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:43.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.183+0000 7f1808814640 1 --2- 172.21.15.5:0/468321815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18040b3d70 0x7f18040ca0b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f180408d5e0 tx=0x7f17fc00e820 comp rx=0 tx=0).stop 2024-05-20T18:29:43.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.183+0000 7f1808814640 1 -- 172.21.15.5:0/468321815 shutdown_connections 2024-05-20T18:29:43.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.183+0000 7f1808814640 1 --2- 172.21.15.5:0/468321815 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f17e406be60 0x7f17e406e370 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:43.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.183+0000 7f1808814640 1 --2- 172.21.15.5:0/468321815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18040b3d70 0x7f18040ca0b0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:43.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.183+0000 7f1808814640 1 --2- 172.21.15.5:0/468321815 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1804155070 0x7f18040b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:43.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.184+0000 7f1808814640 1 --2- 172.21.15.5:0/468321815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17f00039b0 0x7f18040c8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:43.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.184+0000 7f1808814640 1 -- 172.21.15.5:0/468321815 >> 172.21.15.5:0/468321815 conn(0x7f18040d4bd0 msgr2=0x7f1804160eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:43.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.184+0000 7f1808814640 1 -- 172.21.15.5:0/468321815 shutdown_connections 2024-05-20T18:29:43.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.184+0000 7f1808814640 1 -- 172.21.15.5:0/468321815 wait complete. 2024-05-20T18:29:43.195 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:29:43.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.314+0000 7fc5aa8fe640 1 Processor -- start 2024-05-20T18:29:43.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.314+0000 7fc5aa8fe640 1 -- start start 2024-05-20T18:29:43.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.316+0000 7fc5aa8fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc5a4162820 0x7fc5a40b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:43.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.316+0000 7fc5aa8fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5a406dae0 0x7fc5a40b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:43.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.316+0000 7fc5aa8fe640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5a40d0630 con 0x7fc5a40b3e80 2024-05-20T18:29:43.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.316+0000 7fc5aa8fe640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fc5a40d39c0 con 0x7fc5a4162820 2024-05-20T18:29:43.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.316+0000 7fc5aa8fe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc5a40d3ca0 con 0x7fc5a406dae0 2024-05-20T18:29:43.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.316+0000 7fc5a3fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc5a4162820 0x7fc5a40b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:43.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.316+0000 7fc5a3fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc5a4162820 0x7fc5a40b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35286/0 (socket says 172.21.15.5:35286) 2024-05-20T18:29:43.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.316+0000 7fc5a3fff640 1 -- 172.21.15.5:0/3524511723 learned_addr learned my addr 172.21.15.5:0/3524511723 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:43.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.316+0000 7fc5a37fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc5a40b3e80 0x7fc5a406d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44706/0 (socket says 172.21.15.5:44706) 2024-05-20T18:29:43.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.316+0000 7fc5a8e9b640 1 --2- 172.21.15.5:0/3524511723 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5a406dae0 0x7fc5a40b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:43.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.316+0000 7fc5a3fff640 1 -- 172.21.15.5:0/3524511723 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5a406dae0 msgr2=0x7fc5a40b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:43.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.316+0000 7fc5a3fff640 1 --2- 172.21.15.5:0/3524511723 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5a406dae0 0x7fc5a40b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:43.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.316+0000 7fc5a3fff640 1 -- 172.21.15.5:0/3524511723 >> v1:172.21.15.5:6789/0 conn(0x7fc5a40b3e80 legacy=0x7fc5a406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:43.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.316+0000 7fc5a3fff640 1 -- 172.21.15.5:0/3524511723 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a40d3f80 con 0x7fc5a4162820 2024-05-20T18:29:43.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.317+0000 7fc5a3fff640 1 --2- 172.21.15.5:0/3524511723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5a4162820 0x7fc5a40b3940 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc594009870 tx=0x7fc59402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434112183d12ce85 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:43.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.317+0000 7fc5a2ffd640 1 -- 172.21.15.5:0/3524511723 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594037070 con 0x7fc5a4162820 2024-05-20T18:29:43.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.317+0000 7fc5a2ffd640 1 -- 172.21.15.5:0/3524511723 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc594032440 con 0x7fc5a4162820 2024-05-20T18:29:43.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.317+0000 7fc5a2ffd640 1 -- 172.21.15.5:0/3524511723 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594032810 con 0x7fc5a4162820 2024-05-20T18:29:43.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.318+0000 7fc5aa8fe640 1 -- 172.21.15.5:0/3524511723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5a4162820 msgr2=0x7fc5a40b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:43.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.318+0000 7fc5aa8fe640 1 --2- 172.21.15.5:0/3524511723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5a4162820 0x7fc5a40b3940 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc594009870 tx=0x7fc59402a270 comp rx=0 tx=0).stop 2024-05-20T18:29:43.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.318+0000 7fc5aa8fe640 1 -- 172.21.15.5:0/3524511723 shutdown_connections 2024-05-20T18:29:43.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.318+0000 7fc5aa8fe640 1 --2- 172.21.15.5:0/3524511723 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5a406dae0 0x7fc5a40b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:43.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.318+0000 7fc5aa8fe640 1 --2- 172.21.15.5:0/3524511723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5a4162820 0x7fc5a40b3940 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:43.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.318+0000 7fc5aa8fe640 1 -- 172.21.15.5:0/3524511723 >> 172.21.15.5:0/3524511723 conn(0x7fc5a40d4bd0 msgr2=0x7fc5a40d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:43.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.318+0000 7fc5aa8fe640 1 -- 172.21.15.5:0/3524511723 shutdown_connections 2024-05-20T18:29:43.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.318+0000 7fc5aa8fe640 1 -- 172.21.15.5:0/3524511723 wait complete. 2024-05-20T18:29:43.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.320+0000 7fc5aa8fe640 1 Processor -- start 2024-05-20T18:29:43.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.320+0000 7fc5aa8fe640 1 -- start start 2024-05-20T18:29:43.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.321+0000 7fc5aa8fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5a40b3e80 0x7fc5a40ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:43.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.321+0000 7fc5aa8fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5a4162820 0x7fc5a415a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:43.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.321+0000 7fc5aa8fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5a415b050 0x7fc5a40cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:43.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.321+0000 7fc5aa8fe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a40d4780 con 0x7fc5a415b050 2024-05-20T18:29:43.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.321+0000 7fc5aa8fe640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a40d0300 con 0x7fc5a40b3e80 2024-05-20T18:29:43.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.321+0000 7fc5aa8fe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc5a40d4900 con 0x7fc5a4162820 2024-05-20T18:29:43.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.321+0000 7fc5a8e9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5a415b050 0x7fc5a40cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:43.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.321+0000 7fc5a8e9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5a415b050 0x7fc5a40cd3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35296/0 (socket says 172.21.15.5:35296) 2024-05-20T18:29:43.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.321+0000 7fc5a8e9b640 1 -- 172.21.15.5:0/925078574 learned_addr learned my addr 172.21.15.5:0/925078574 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:43.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.321+0000 7fc5a3fff640 1 --2- 172.21.15.5:0/925078574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5a40b3e80 0x7fc5a40ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:43.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.321+0000 7fc5a37fe640 1 --2- 172.21.15.5:0/925078574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5a4162820 0x7fc5a415a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:43.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.322+0000 7fc5a8e9b640 1 -- 172.21.15.5:0/925078574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5a4162820 msgr2=0x7fc5a415a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:43.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.322+0000 7fc5a8e9b640 1 --2- 172.21.15.5:0/925078574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5a4162820 0x7fc5a415a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:43.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.322+0000 7fc5a8e9b640 1 -- 172.21.15.5:0/925078574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5a40b3e80 msgr2=0x7fc5a40ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:43.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.322+0000 7fc5a8e9b640 1 --2- 172.21.15.5:0/925078574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5a40b3e80 0x7fc5a40ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:43.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.322+0000 7fc5a8e9b640 1 -- 172.21.15.5:0/925078574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a40ca710 con 0x7fc5a415b050 2024-05-20T18:29:43.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.322+0000 7fc5a3fff640 1 --2- 172.21.15.5:0/925078574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5a40b3e80 0x7fc5a40ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:29:43.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.322+0000 7fc5a8e9b640 1 --2- 172.21.15.5:0/925078574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5a415b050 0x7fc5a40cd3e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00efc0 tx=0x7fc59c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:43.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.322+0000 7fc5a0ff9640 1 -- 172.21.15.5:0/925078574 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c018020 con 0x7fc5a415b050 2024-05-20T18:29:43.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.322+0000 7fc5a0ff9640 1 -- 172.21.15.5:0/925078574 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc59c014480 con 0x7fc5a415b050 2024-05-20T18:29:43.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.322+0000 7fc5aa8fe640 1 -- 172.21.15.5:0/925078574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a40b5a70 con 0x7fc5a415b050 2024-05-20T18:29:43.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.322+0000 7fc5a0ff9640 1 -- 172.21.15.5:0/925078574 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c009e60 con 0x7fc5a415b050 2024-05-20T18:29:43.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.322+0000 7fc5aa8fe640 1 -- 172.21.15.5:0/925078574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a40c96f0 con 0x7fc5a415b050 2024-05-20T18:29:43.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.323+0000 7fc5aa8fe640 1 -- 172.21.15.5:0/925078574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5a40b57e0 con 0x7fc5a415b050 2024-05-20T18:29:43.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.324+0000 7fc5a0ff9640 1 -- 172.21.15.5:0/925078574 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc59c005d10 con 0x7fc5a415b050 2024-05-20T18:29:43.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.324+0000 7fc5a0ff9640 1 --2- 172.21.15.5:0/925078574 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc59006be60 0x7fc59006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:43.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.324+0000 7fc5a0ff9640 1 -- 172.21.15.5:0/925078574 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7fc59c08bd10 con 0x7fc5a415b050 2024-05-20T18:29:43.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.324+0000 7fc5a3fff640 1 --2- 172.21.15.5:0/925078574 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc59006be60 0x7fc59006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:43.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.325+0000 7fc5a3fff640 1 --2- 172.21.15.5:0/925078574 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc59006be60 0x7fc59006e370 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc594034000 tx=0x7fc59402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:43.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.329+0000 7fc5a0ff9640 1 -- 172.21.15.5:0/925078574 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc59c022030 con 0x7fc5a415b050 2024-05-20T18:29:43.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.511+0000 7fc5aa8fe640 1 -- 172.21.15.5:0/925078574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc5a4075b00 con 0x7fc5a415b050 2024-05-20T18:29:43.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.511+0000 7fc5a0ff9640 1 -- 172.21.15.5:0/925078574 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v51) ==== 100+0+11 (secure 0 0 0) 0x7fc59c010040 con 0x7fc5a415b050 2024-05-20T18:29:43.511 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:29:43.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.513+0000 7fc5aa8fe640 1 -- 172.21.15.5:0/925078574 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc59006be60 msgr2=0x7fc59006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:43.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.513+0000 7fc5aa8fe640 1 --2- 172.21.15.5:0/925078574 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc59006be60 0x7fc59006e370 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc594034000 tx=0x7fc59402e000 comp rx=0 tx=0).stop 2024-05-20T18:29:43.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.513+0000 7fc5aa8fe640 1 -- 172.21.15.5:0/925078574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5a415b050 msgr2=0x7fc5a40cd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:43.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.513+0000 7fc5aa8fe640 1 --2- 172.21.15.5:0/925078574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5a415b050 0x7fc5a40cd3e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00efc0 tx=0x7fc59c00c490 comp rx=0 tx=0).stop 2024-05-20T18:29:43.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.513+0000 7fc5aa8fe640 1 -- 172.21.15.5:0/925078574 shutdown_connections 2024-05-20T18:29:43.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.513+0000 7fc5aa8fe640 1 --2- 172.21.15.5:0/925078574 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc59006be60 0x7fc59006e370 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:43.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.513+0000 7fc5aa8fe640 1 --2- 172.21.15.5:0/925078574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5a415b050 0x7fc5a40cd3e0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:43.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.513+0000 7fc5aa8fe640 1 --2- 172.21.15.5:0/925078574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5a4162820 0x7fc5a415a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:43.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.513+0000 7fc5aa8fe640 1 --2- 172.21.15.5:0/925078574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5a40b3e80 0x7fc5a40ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:43.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.513+0000 7fc5aa8fe640 1 -- 172.21.15.5:0/925078574 >> 172.21.15.5:0/925078574 conn(0x7fc5a40d4bd0 msgr2=0x7fc5a4161cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:43.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.513+0000 7fc5aa8fe640 1 -- 172.21.15.5:0/925078574 shutdown_connections 2024-05-20T18:29:43.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.513+0000 7fc5aa8fe640 1 -- 172.21.15.5:0/925078574 wait complete. 2024-05-20T18:29:43.525 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T18:29:43.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.641+0000 7fdd0d98f640 1 Processor -- start 2024-05-20T18:29:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.641+0000 7fdd0d98f640 1 -- start start 2024-05-20T18:29:43.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.642+0000 7fdd0d98f640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fdd0806d9a0 0x7fdd080d1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:43.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.642+0000 7fdd0d98f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd0815d940 0x7fdd0815dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.642+0000 7fdd0d98f640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd0815e6b0 con 0x7fdd080b2f10 2024-05-20T18:29:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.642+0000 7fdd0d98f640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fdd080d2400 con 0x7fdd0806d9a0 2024-05-20T18:29:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.642+0000 7fdd0d98f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdd080d1a30 con 0x7fdd0815d940 2024-05-20T18:29:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.642+0000 7fdd06ffd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fdd080b2f10 0x7fdd0806d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44716/0 (socket says 172.21.15.5:44716) 2024-05-20T18:29:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.642+0000 7fdd06ffd640 1 -- 172.21.15.5:0/3186643328 learned_addr learned my addr 172.21.15.5:0/3186643328 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.643+0000 7fdd05ffb640 1 -- 172.21.15.5:0/3186643328 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1486643527 0 0) 0x7fdd0815e6b0 con 0x7fdd080b2f10 2024-05-20T18:29:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.643+0000 7fdd067fc640 1 --2- 172.21.15.5:0/3186643328 >> v2:172.21.15.63:3300/0 conn(0x7fdd0806d9a0 0x7fdd080d1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.643+0000 7fdd077fe640 1 --2- 172.21.15.5:0/3186643328 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd0815d940 0x7fdd0815dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.643+0000 7fdd05ffb640 1 -- 172.21.15.5:0/3186643328 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdcf4003520 con 0x7fdd080b2f10 2024-05-20T18:29:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.643+0000 7fdd05ffb640 1 -- 172.21.15.5:0/3186643328 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3209312233 0 0) 0x7fdcf4003520 con 0x7fdd080b2f10 2024-05-20T18:29:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.643+0000 7fdd05ffb640 1 -- 172.21.15.5:0/3186643328 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd0815d940 msgr2=0x7fdd0815dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.643+0000 7fdd05ffb640 1 --2- 172.21.15.5:0/3186643328 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd0815d940 0x7fdd0815dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.643+0000 7fdd05ffb640 1 -- 172.21.15.5:0/3186643328 >> v2:172.21.15.63:3300/0 conn(0x7fdd0806d9a0 msgr2=0x7fdd080d1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.643+0000 7fdd05ffb640 1 --2- 172.21.15.5:0/3186643328 >> v2:172.21.15.63:3300/0 conn(0x7fdd0806d9a0 0x7fdd080d1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.643+0000 7fdd05ffb640 1 -- 172.21.15.5:0/3186643328 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd080d1100 con 0x7fdd080b2f10 2024-05-20T18:29:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.643+0000 7fdd05ffb640 1 -- 172.21.15.5:0/3186643328 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7fdcf8001670 con 0x7fdd080b2f10 2024-05-20T18:29:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.643+0000 7fdd05ffb640 1 -- 172.21.15.5:0/3186643328 >> v1:172.21.15.5:6789/0 conn(0x7fdd080b2f10 legacy=0x7fdd0806d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.643+0000 7fdd05ffb640 1 --2- 172.21.15.5:0/3186643328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcf40039b0 0x7fdcf4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.643+0000 7fdd05ffb640 1 --2- 172.21.15.5:0/3186643328 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd0815d940 0x7fdcf4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.643+0000 7fdd05ffb640 1 --2- 172.21.15.5:0/3186643328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf4004d40 0x7fdcf4005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.643+0000 7fdd05ffb640 1 -- 172.21.15.5:0/3186643328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdd080d2400 con 0x7fdcf4004d40 2024-05-20T18:29:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.643+0000 7fdd05ffb640 1 -- 172.21.15.5:0/3186643328 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdd080d1a30 con 0x7fdcf40039b0 2024-05-20T18:29:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.643+0000 7fdd05ffb640 1 -- 172.21.15.5:0/3186643328 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdcf400ba40 con 0x7fdd0815d940 2024-05-20T18:29:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.643+0000 7fdd077fe640 1 --2- 172.21.15.5:0/3186643328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf4004d40 0x7fdcf4005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.643+0000 7fdd06ffd640 1 --2- 172.21.15.5:0/3186643328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcf40039b0 0x7fdcf4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.643+0000 7fdd067fc640 1 --2- 172.21.15.5:0/3186643328 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd0815d940 0x7fdcf4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.643+0000 7fdd077fe640 1 -- 172.21.15.5:0/3186643328 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd0815d940 msgr2=0x7fdcf4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.643+0000 7fdd077fe640 1 --2- 172.21.15.5:0/3186643328 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd0815d940 0x7fdcf4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.643+0000 7fdd077fe640 1 -- 172.21.15.5:0/3186643328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcf40039b0 msgr2=0x7fdcf4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.643+0000 7fdd077fe640 1 --2- 172.21.15.5:0/3186643328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcf40039b0 0x7fdcf4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.643+0000 7fdd077fe640 1 -- 172.21.15.5:0/3186643328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf4003520 con 0x7fdcf4004d40 2024-05-20T18:29:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.644+0000 7fdd077fe640 1 --2- 172.21.15.5:0/3186643328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf4004d40 0x7fdcf4005150 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fdd00009f90 tx=0x7fdcf4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4edad2dbc528df14 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.644+0000 7fdd05ffb640 1 -- 172.21.15.5:0/3186643328 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd00009c10 con 0x7fdcf4004d40 2024-05-20T18:29:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.644+0000 7fdd05ffb640 1 -- 172.21.15.5:0/3186643328 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd00036070 con 0x7fdcf4004d40 2024-05-20T18:29:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.644+0000 7fdd05ffb640 1 -- 172.21.15.5:0/3186643328 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd00031850 con 0x7fdcf4004d40 2024-05-20T18:29:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.645+0000 7fdd0d98f640 1 -- 172.21.15.5:0/3186643328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf4004d40 msgr2=0x7fdcf4005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.645+0000 7fdd0d98f640 1 --2- 172.21.15.5:0/3186643328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf4004d40 0x7fdcf4005150 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fdd00009f90 tx=0x7fdcf4005830 comp rx=0 tx=0).stop 2024-05-20T18:29:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.645+0000 7fdd077fe640 1 -- 172.21.15.5:0/3186643328 reap_dead start 2024-05-20T18:29:43.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.645+0000 7fdd0d98f640 1 -- 172.21.15.5:0/3186643328 shutdown_connections 2024-05-20T18:29:43.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.645+0000 7fdd0d98f640 1 -- 172.21.15.5:0/3186643328 >> 172.21.15.5:0/3186643328 conn(0x7fdd080b6670 msgr2=0x7fdd080b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:43.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.645+0000 7fdd0d98f640 1 -- 172.21.15.5:0/3186643328 shutdown_connections 2024-05-20T18:29:43.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.645+0000 7fdd0d98f640 1 -- 172.21.15.5:0/3186643328 wait complete. 2024-05-20T18:29:43.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.647+0000 7fdd0d98f640 1 Processor -- start 2024-05-20T18:29:43.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.647+0000 7fdd0d98f640 1 -- start start 2024-05-20T18:29:43.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.648+0000 7fdd0d98f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcf4004d40 0x7fdd080af330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:43.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.648+0000 7fdd0d98f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd08156850 0x7fdd081f5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:43.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.648+0000 7fdd0d98f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd081f56b0 0x7fdd0809bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:43.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.648+0000 7fdd0d98f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdd080d16c0 con 0x7fdd08156850 2024-05-20T18:29:43.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.648+0000 7fdd0d98f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdd080b3320 con 0x7fdd081f56b0 2024-05-20T18:29:43.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.648+0000 7fdd0d98f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdd0815dd50 con 0x7fdcf4004d40 2024-05-20T18:29:43.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.648+0000 7fdd067fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd08156850 0x7fdd081f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:43.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.649+0000 7fdd067fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd08156850 0x7fdd081f5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35310/0 (socket says 172.21.15.5:35310) 2024-05-20T18:29:43.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.649+0000 7fdd067fc640 1 -- 172.21.15.5:0/2138074327 learned_addr learned my addr 172.21.15.5:0/2138074327 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:43.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.649+0000 7fdd06ffd640 1 --2- 172.21.15.5:0/2138074327 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcf4004d40 0x7fdd080af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:43.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.649+0000 7fdd077fe640 1 --2- 172.21.15.5:0/2138074327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd081f56b0 0x7fdd0809bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:43.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.649+0000 7fdd067fc640 1 -- 172.21.15.5:0/2138074327 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcf4004d40 msgr2=0x7fdd080af330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:43.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.649+0000 7fdd067fc640 1 --2- 172.21.15.5:0/2138074327 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcf4004d40 0x7fdd080af330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:43.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.649+0000 7fdd067fc640 1 -- 172.21.15.5:0/2138074327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd081f56b0 msgr2=0x7fdd0809bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:43.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.649+0000 7fdd067fc640 1 --2- 172.21.15.5:0/2138074327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd081f56b0 0x7fdd0809bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:43.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.649+0000 7fdd067fc640 1 -- 172.21.15.5:0/2138074327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd080b0610 con 0x7fdd08156850 2024-05-20T18:29:43.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.649+0000 7fdd067fc640 1 --2- 172.21.15.5:0/2138074327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd08156850 0x7fdd081f5170 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0011980 tx=0x7fdcf0011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:43.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.649+0000 7fdcfffff640 1 -- 172.21.15.5:0/2138074327 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0005ba0 con 0x7fdd08156850 2024-05-20T18:29:43.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.649+0000 7fdcfffff640 1 -- 172.21.15.5:0/2138074327 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcf0005d40 con 0x7fdd08156850 2024-05-20T18:29:43.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.649+0000 7fdd0d98f640 1 -- 172.21.15.5:0/2138074327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd081f5df0 con 0x7fdd08156850 2024-05-20T18:29:43.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.649+0000 7fdcfffff640 1 -- 172.21.15.5:0/2138074327 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf000fb20 con 0x7fdd08156850 2024-05-20T18:29:43.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.649+0000 7fdd0d98f640 1 -- 172.21.15.5:0/2138074327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd080ae850 con 0x7fdd08156850 2024-05-20T18:29:43.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.650+0000 7fdd0d98f640 1 -- 172.21.15.5:0/2138074327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd08075b00 con 0x7fdd08156850 2024-05-20T18:29:43.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.651+0000 7fdcfffff640 1 -- 172.21.15.5:0/2138074327 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdcf0012040 con 0x7fdd08156850 2024-05-20T18:29:43.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.651+0000 7fdcfffff640 1 --2- 172.21.15.5:0/2138074327 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdce806be60 0x7fdce806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:43.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.651+0000 7fdcfffff640 1 -- 172.21.15.5:0/2138074327 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7fdcf008b5d0 con 0x7fdd08156850 2024-05-20T18:29:43.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.652+0000 7fdd06ffd640 1 --2- 172.21.15.5:0/2138074327 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdce806be60 0x7fdce806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:43.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.652+0000 7fdd06ffd640 1 --2- 172.21.15.5:0/2138074327 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdce806be60 0x7fdce806e370 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdd080cde90 tx=0x7fdcf8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:43.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.656+0000 7fdcfffff640 1 -- 172.21.15.5:0/2138074327 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fdd080b0610 con 0x7fdd08156850 2024-05-20T18:29:43.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.854+0000 7fdd0d98f640 1 -- 172.21.15.5:0/2138074327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdd081f5b80 con 0x7fdd08156850 2024-05-20T18:29:43.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.854+0000 7fdcfffff640 1 -- 172.21.15.5:0/2138074327 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v51) ==== 80+0+0 (secure 0 0 0) 0x7fdcf0018030 con 0x7fdd08156850 2024-05-20T18:29:43.854 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-05-20T18:29:43.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.856+0000 7fdd0d98f640 1 -- 172.21.15.5:0/2138074327 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdce806be60 msgr2=0x7fdce806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:43.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.856+0000 7fdd0d98f640 1 --2- 172.21.15.5:0/2138074327 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdce806be60 0x7fdce806e370 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdd080cde90 tx=0x7fdcf8009040 comp rx=0 tx=0).stop 2024-05-20T18:29:43.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.856+0000 7fdd0d98f640 1 -- 172.21.15.5:0/2138074327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd08156850 msgr2=0x7fdd081f5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:43.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.856+0000 7fdd0d98f640 1 --2- 172.21.15.5:0/2138074327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd08156850 0x7fdd081f5170 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0011980 tx=0x7fdcf0011e40 comp rx=0 tx=0).stop 2024-05-20T18:29:43.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.856+0000 7fdd0d98f640 1 -- 172.21.15.5:0/2138074327 shutdown_connections 2024-05-20T18:29:43.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.856+0000 7fdd0d98f640 1 --2- 172.21.15.5:0/2138074327 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdce806be60 0x7fdce806e370 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:43.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.856+0000 7fdd0d98f640 1 --2- 172.21.15.5:0/2138074327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd081f56b0 0x7fdd0809bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:43.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.856+0000 7fdd0d98f640 1 --2- 172.21.15.5:0/2138074327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd08156850 0x7fdd081f5170 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:43.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.856+0000 7fdd0d98f640 1 --2- 172.21.15.5:0/2138074327 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcf4004d40 0x7fdd080af330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:43.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.856+0000 7fdd0d98f640 1 -- 172.21.15.5:0/2138074327 >> 172.21.15.5:0/2138074327 conn(0x7fdd080b6670 msgr2=0x7fdd0806b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:43.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.856+0000 7fdd0d98f640 1 -- 172.21.15.5:0/2138074327 shutdown_connections 2024-05-20T18:29:43.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:43.856+0000 7fdd0d98f640 1 -- 172.21.15.5:0/2138074327 wait complete. 2024-05-20T18:29:43.867 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-20T18:29:43.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:29:43.868 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-20T18:29:43.924 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:29:43.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:29:43.924 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-20T18:29:43.951 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T18:29:43.951 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:29:43.951 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:29:43.969 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19732 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T18:29:43.969 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T18:29:43.969 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:29:43.969 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:29:44.008 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15663 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T18:29:44.008 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T18:29:44.008 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:29:44.008 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:29:44.027 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19732 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T18:29:44.027 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:29:44.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.163+0000 7efd51b57640 1 Processor -- start 2024-05-20T18:29:44.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.163+0000 7efd51b57640 1 -- start start 2024-05-20T18:29:44.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.165+0000 7efd51b57640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efd4c15d060 0x7efd4c0b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:44.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.165+0000 7efd51b57640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd4c0b5720 0x7efd4c06f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:44.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.165+0000 7efd51b57640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd4c0b5b30 con 0x7efd4c0b47b0 2024-05-20T18:29:44.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.165+0000 7efd51b57640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7efd4c0b39f0 con 0x7efd4c15d060 2024-05-20T18:29:44.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.165+0000 7efd51b57640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efd4c0b3cd0 con 0x7efd4c0b5720 2024-05-20T18:29:44.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.165+0000 7efd4affd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efd4c15d060 0x7efd4c0b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:44.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.165+0000 7efd4b7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7efd4c0b47b0 0x7efd4c0afad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44722/0 (socket says 172.21.15.5:44722) 2024-05-20T18:29:44.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.165+0000 7efd4b7fe640 1 -- 172.21.15.5:0/2226655284 learned_addr learned my addr 172.21.15.5:0/2226655284 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:44.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.165+0000 7efd4affd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efd4c15d060 0x7efd4c0b51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35318/0 (socket says 172.21.15.5:35318) 2024-05-20T18:29:44.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.165+0000 7efd4bfff640 1 --2- 172.21.15.5:0/2226655284 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd4c0b5720 0x7efd4c06f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:44.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.165+0000 7efd4affd640 1 -- 172.21.15.5:0/2226655284 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd4c0b5720 msgr2=0x7efd4c06f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:44.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.165+0000 7efd4affd640 1 --2- 172.21.15.5:0/2226655284 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd4c0b5720 0x7efd4c06f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:44.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.165+0000 7efd4affd640 1 -- 172.21.15.5:0/2226655284 >> v1:172.21.15.5:6789/0 conn(0x7efd4c0b47b0 legacy=0x7efd4c0afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:44.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.165+0000 7efd4a7fc640 1 -- 172.21.15.5:0/2226655284 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1464809307 0 0) 0x7efd3c001110 con 0x7efd4c0b47b0 2024-05-20T18:29:44.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.165+0000 7efd4affd640 1 -- 172.21.15.5:0/2226655284 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd4c0b3fb0 con 0x7efd4c15d060 2024-05-20T18:29:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.166+0000 7efd4affd640 1 --2- 172.21.15.5:0/2226655284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd4c15d060 0x7efd4c0b51e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7efd4000ba40 tx=0x7efd4002c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b56e9394877e6db8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.166+0000 7efd4a7fc640 1 -- 172.21.15.5:0/2226655284 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd40032750 con 0x7efd4c15d060 2024-05-20T18:29:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.166+0000 7efd4a7fc640 1 -- 172.21.15.5:0/2226655284 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd40030050 con 0x7efd4c15d060 2024-05-20T18:29:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.166+0000 7efd4a7fc640 1 -- 172.21.15.5:0/2226655284 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd40032b10 con 0x7efd4c15d060 2024-05-20T18:29:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.167+0000 7efd51b57640 1 -- 172.21.15.5:0/2226655284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd4c15d060 msgr2=0x7efd4c0b51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.167+0000 7efd51b57640 1 --2- 172.21.15.5:0/2226655284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd4c15d060 0x7efd4c0b51e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7efd4000ba40 tx=0x7efd4002c850 comp rx=0 tx=0).stop 2024-05-20T18:29:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.167+0000 7efd51b57640 1 -- 172.21.15.5:0/2226655284 shutdown_connections 2024-05-20T18:29:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.167+0000 7efd51b57640 1 --2- 172.21.15.5:0/2226655284 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd4c0b5720 0x7efd4c06f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.167+0000 7efd51b57640 1 --2- 172.21.15.5:0/2226655284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd4c15d060 0x7efd4c0b51e0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.167+0000 7efd51b57640 1 -- 172.21.15.5:0/2226655284 >> 172.21.15.5:0/2226655284 conn(0x7efd4c0cd2a0 msgr2=0x7efd4c0cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.167+0000 7efd51b57640 1 -- 172.21.15.5:0/2226655284 shutdown_connections 2024-05-20T18:29:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.167+0000 7efd51b57640 1 -- 172.21.15.5:0/2226655284 wait complete. 2024-05-20T18:29:44.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.169+0000 7efd51b57640 1 Processor -- start 2024-05-20T18:29:44.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.169+0000 7efd51b57640 1 -- start start 2024-05-20T18:29:44.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.170+0000 7efd51b57640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd4c0b5720 0x7efd4c0aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:44.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.170+0000 7efd51b57640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd4c15d060 0x7efd4c147240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:44.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.170+0000 7efd51b57640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd4c147a60 0x7efd4c147e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:44.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.170+0000 7efd51b57640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c159de0 con 0x7efd4c0b5720 2024-05-20T18:29:44.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.170+0000 7efd51b57640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c159c60 con 0x7efd4c15d060 2024-05-20T18:29:44.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.170+0000 7efd51b57640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efd4c06f910 con 0x7efd4c147a60 2024-05-20T18:29:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.170+0000 7efd4b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd4c0b5720 0x7efd4c0aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.170+0000 7efd4b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd4c0b5720 0x7efd4c0aee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35324/0 (socket says 172.21.15.5:35324) 2024-05-20T18:29:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.170+0000 7efd4b7fe640 1 -- 172.21.15.5:0/4187621855 learned_addr learned my addr 172.21.15.5:0/4187621855 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.170+0000 7efd4bfff640 1 --2- 172.21.15.5:0/4187621855 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd4c147a60 0x7efd4c147e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.170+0000 7efd4affd640 1 --2- 172.21.15.5:0/4187621855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd4c15d060 0x7efd4c147240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.171+0000 7efd4b7fe640 1 -- 172.21.15.5:0/4187621855 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd4c147a60 msgr2=0x7efd4c147e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.171+0000 7efd4b7fe640 1 --2- 172.21.15.5:0/4187621855 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd4c147a60 0x7efd4c147e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.171+0000 7efd4b7fe640 1 -- 172.21.15.5:0/4187621855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd4c15d060 msgr2=0x7efd4c147240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.171+0000 7efd4b7fe640 1 --2- 172.21.15.5:0/4187621855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd4c15d060 0x7efd4c147240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.171+0000 7efd4b7fe640 1 -- 172.21.15.5:0/4187621855 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd4c147780 con 0x7efd4c0b5720 2024-05-20T18:29:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.171+0000 7efd4bfff640 1 --2- 172.21.15.5:0/4187621855 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd4c147a60 0x7efd4c147e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:29:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.171+0000 7efd4b7fe640 1 --2- 172.21.15.5:0/4187621855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd4c0b5720 0x7efd4c0aee20 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7efd3c00c440 tx=0x7efd3c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.171+0000 7efd33fff640 1 -- 172.21.15.5:0/4187621855 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c018070 con 0x7efd4c0b5720 2024-05-20T18:29:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.171+0000 7efd33fff640 1 -- 172.21.15.5:0/4187621855 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd3c0074e0 con 0x7efd4c0b5720 2024-05-20T18:29:44.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.171+0000 7efd51b57640 1 -- 172.21.15.5:0/4187621855 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd4c0add80 con 0x7efd4c0b5720 2024-05-20T18:29:44.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.171+0000 7efd33fff640 1 -- 172.21.15.5:0/4187621855 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c004a20 con 0x7efd4c0b5720 2024-05-20T18:29:44.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.171+0000 7efd51b57640 1 -- 172.21.15.5:0/4187621855 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd4c0ae340 con 0x7efd4c0b5720 2024-05-20T18:29:44.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.172+0000 7efd51b57640 1 -- 172.21.15.5:0/4187621855 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd4c075b00 con 0x7efd4c0b5720 2024-05-20T18:29:44.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.173+0000 7efd33fff640 1 -- 172.21.15.5:0/4187621855 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efd3c01b070 con 0x7efd4c0b5720 2024-05-20T18:29:44.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.173+0000 7efd33fff640 1 --2- 172.21.15.5:0/4187621855 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efd3806be30 0x7efd3806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:44.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.173+0000 7efd33fff640 1 -- 172.21.15.5:0/4187621855 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7efd3c08a2d0 con 0x7efd4c0b5720 2024-05-20T18:29:44.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.173+0000 7efd4affd640 1 --2- 172.21.15.5:0/4187621855 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efd3806be30 0x7efd3806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:44.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.174+0000 7efd4affd640 1 --2- 172.21.15.5:0/4187621855 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efd3806be30 0x7efd3806e340 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efd40037000 tx=0x7efd40032000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:44.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.178+0000 7efd33fff640 1 -- 172.21.15.5:0/4187621855 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7efd4c147780 con 0x7efd4c0b5720 2024-05-20T18:29:44.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.400+0000 7efd51b57640 1 -- 172.21.15.5:0/4187621855 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd4c0b4bc0 con 0x7efd4c0b5720 2024-05-20T18:29:44.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.401+0000 7efd33fff640 1 -- 172.21.15.5:0/4187621855 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5175 (secure 0 0 0) 0x7efd4c0b5b30 con 0x7efd4c0b5720 2024-05-20T18:29:44.401 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:29:44.401 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5207,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3958852803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3958852803},{"type":"v1","addr":"172.21.15.63:6835","nonce":3958852803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5560,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/3020169757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":3020169757},{"type":"v1","addr":"172.21.15.63:6834","nonce":3020169757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2965018210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2965018210},{"type":"v1","addr":"172.21.15.5:6835","nonce":2965018210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:29:02.038442+0000","modified":"2024-05-20T18:29:05.055248+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5751},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5751":{"gid":5751,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6837/3469824363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3469824363},{"type":"v1","addr":"172.21.15.5:6837","nonce":3469824363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5751,"qdb_cluster":[5751]},"id":3}]} 2024-05-20T18:29:44.401 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-05-20T18:29:44.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.403+0000 7efd51b57640 1 -- 172.21.15.5:0/4187621855 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efd3806be30 msgr2=0x7efd3806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:44.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.403+0000 7efd51b57640 1 --2- 172.21.15.5:0/4187621855 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efd3806be30 0x7efd3806e340 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efd40037000 tx=0x7efd40032000 comp rx=0 tx=0).stop 2024-05-20T18:29:44.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.403+0000 7efd51b57640 1 -- 172.21.15.5:0/4187621855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd4c0b5720 msgr2=0x7efd4c0aee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:44.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.403+0000 7efd51b57640 1 --2- 172.21.15.5:0/4187621855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd4c0b5720 0x7efd4c0aee20 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7efd3c00c440 tx=0x7efd3c00c900 comp rx=0 tx=0).stop 2024-05-20T18:29:44.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.403+0000 7efd51b57640 1 -- 172.21.15.5:0/4187621855 shutdown_connections 2024-05-20T18:29:44.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.403+0000 7efd51b57640 1 --2- 172.21.15.5:0/4187621855 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efd3806be30 0x7efd3806e340 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:44.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.403+0000 7efd51b57640 1 --2- 172.21.15.5:0/4187621855 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd4c147a60 0x7efd4c147e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:44.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.403+0000 7efd51b57640 1 --2- 172.21.15.5:0/4187621855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd4c15d060 0x7efd4c147240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:44.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.403+0000 7efd51b57640 1 --2- 172.21.15.5:0/4187621855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd4c0b5720 0x7efd4c0aee20 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:44.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.403+0000 7efd51b57640 1 -- 172.21.15.5:0/4187621855 >> 172.21.15.5:0/4187621855 conn(0x7efd4c0cd2a0 msgr2=0x7efd4c15a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:44.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.403+0000 7efd51b57640 1 -- 172.21.15.5:0/4187621855 shutdown_connections 2024-05-20T18:29:44.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.403+0000 7efd51b57640 1 -- 172.21.15.5:0/4187621855 wait complete. 2024-05-20T18:29:44.414 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:29:44.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.528+0000 7f8be64ca640 1 Processor -- start 2024-05-20T18:29:44.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.528+0000 7f8be64ca640 1 -- start start 2024-05-20T18:29:44.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.530+0000 7f8be64ca640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8be015f700 0x7f8be015fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:44.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.530+0000 7f8be64ca640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8be006c180 0x7f8be006c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:44.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.530+0000 7f8be64ca640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8be0160430 con 0x7f8be00af030 2024-05-20T18:29:44.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.530+0000 7f8be64ca640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8be006bcf0 con 0x7f8be006c180 2024-05-20T18:29:44.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.530+0000 7f8be64ca640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8be006b6a0 con 0x7f8be015f700 2024-05-20T18:29:44.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.530+0000 7f8bdf7fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8be015f700 0x7f8be015fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:44.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.530+0000 7f8be4a67640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8be006c180 0x7f8be006c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:44.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.530+0000 7f8be4a67640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8be006c180 0x7f8be006c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:35222/0 (socket says 172.21.15.5:35222) 2024-05-20T18:29:44.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.530+0000 7f8bdf7fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8be015f700 0x7f8be015fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:54860/0 (socket says 172.21.15.5:54860) 2024-05-20T18:29:44.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.530+0000 7f8be4a67640 1 -- 172.21.15.5:0/3382267556 learned_addr learned my addr 172.21.15.5:0/3382267556 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:44.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.530+0000 7f8bdffff640 1 --1- 172.21.15.5:0/3382267556 >> v1:172.21.15.63:6789/0 conn(0x7f8be00af030 0x7f8be00af440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:35308/0 (socket says 172.21.15.5:35308) 2024-05-20T18:29:44.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.530+0000 7f8bdeffd640 1 -- 172.21.15.5:0/3382267556 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2066073966 0 0) 0x7f8be0160430 con 0x7f8be00af030 2024-05-20T18:29:44.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.530+0000 7f8bdeffd640 1 -- 172.21.15.5:0/3382267556 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8bc8003520 con 0x7f8be00af030 2024-05-20T18:29:44.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.531+0000 7f8bdf7fe640 1 -- 172.21.15.5:0/3382267556 >> v2:172.21.15.63:3300/0 conn(0x7f8be006c180 msgr2=0x7f8be006c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:44.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.531+0000 7f8bdf7fe640 1 --2- 172.21.15.5:0/3382267556 >> v2:172.21.15.63:3300/0 conn(0x7f8be006c180 0x7f8be006c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:44.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.531+0000 7f8bdf7fe640 1 -- 172.21.15.5:0/3382267556 >> v1:172.21.15.63:6789/0 conn(0x7f8be00af030 legacy=0x7f8be00af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:44.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.531+0000 7f8bdf7fe640 1 -- 172.21.15.5:0/3382267556 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be006b980 con 0x7f8be015f700 2024-05-20T18:29:44.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.531+0000 7f8be4a67640 1 --2- 172.21.15.5:0/3382267556 >> v2:172.21.15.63:3300/0 conn(0x7f8be006c180 0x7f8be006c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:29:44.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.531+0000 7f8bdf7fe640 1 --2- 172.21.15.5:0/3382267556 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8be015f700 0x7f8be015fb10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40097a0 tx=0x7f8bd402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf4143522e537eb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:44.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.532+0000 7f8bdeffd640 1 -- 172.21.15.5:0/3382267556 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4037070 con 0x7f8be015f700 2024-05-20T18:29:44.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.532+0000 7f8bdeffd640 1 -- 172.21.15.5:0/3382267556 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd4032440 con 0x7f8be015f700 2024-05-20T18:29:44.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.532+0000 7f8bdeffd640 1 -- 172.21.15.5:0/3382267556 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4032810 con 0x7f8be015f700 2024-05-20T18:29:44.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.532+0000 7f8be64ca640 1 -- 172.21.15.5:0/3382267556 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8be015f700 msgr2=0x7f8be015fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:44.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.532+0000 7f8be64ca640 1 --2- 172.21.15.5:0/3382267556 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8be015f700 0x7f8be015fb10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40097a0 tx=0x7f8bd402a180 comp rx=0 tx=0).stop 2024-05-20T18:29:44.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.532+0000 7f8be64ca640 1 -- 172.21.15.5:0/3382267556 shutdown_connections 2024-05-20T18:29:44.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.532+0000 7f8be64ca640 1 --2- 172.21.15.5:0/3382267556 >> v2:172.21.15.63:3300/0 conn(0x7f8be006c180 0x7f8be006c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:44.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.532+0000 7f8be64ca640 1 --2- 172.21.15.5:0/3382267556 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8be015f700 0x7f8be015fb10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:44.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.532+0000 7f8be64ca640 1 -- 172.21.15.5:0/3382267556 >> 172.21.15.5:0/3382267556 conn(0x7f8be00b6620 msgr2=0x7f8be00b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:44.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.532+0000 7f8be64ca640 1 -- 172.21.15.5:0/3382267556 shutdown_connections 2024-05-20T18:29:44.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.532+0000 7f8be64ca640 1 -- 172.21.15.5:0/3382267556 wait complete. 2024-05-20T18:29:44.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.534+0000 7f8be64ca640 1 Processor -- start 2024-05-20T18:29:44.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.534+0000 7f8be64ca640 1 -- start start 2024-05-20T18:29:44.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.535+0000 7f8be64ca640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8be006c180 0x7f8be015a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:44.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.535+0000 7f8be64ca640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8be00af030 0x7f8be0147310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:44.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.535+0000 7f8be64ca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8be015f700 0x7f8be0147ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:44.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.536+0000 7f8be64ca640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8be01601d0 con 0x7f8be015f700 2024-05-20T18:29:44.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.536+0000 7f8be64ca640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8be0160050 con 0x7f8be006c180 2024-05-20T18:29:44.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.536+0000 7f8be64ca640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8be006cad0 con 0x7f8be00af030 2024-05-20T18:29:44.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.536+0000 7f8be4a67640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8be015f700 0x7f8be0147ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:44.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.536+0000 7f8be4a67640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8be015f700 0x7f8be0147ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35332/0 (socket says 172.21.15.5:35332) 2024-05-20T18:29:44.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.536+0000 7f8be4a67640 1 -- 172.21.15.5:0/2387895163 learned_addr learned my addr 172.21.15.5:0/2387895163 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:44.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.536+0000 7f8bdffff640 1 --2- 172.21.15.5:0/2387895163 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8be006c180 0x7f8be015a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:44.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.536+0000 7f8bdf7fe640 1 --2- 172.21.15.5:0/2387895163 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8be00af030 0x7f8be0147310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:44.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.536+0000 7f8be4a67640 1 -- 172.21.15.5:0/2387895163 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8be00af030 msgr2=0x7f8be0147310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:44.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.536+0000 7f8be4a67640 1 --2- 172.21.15.5:0/2387895163 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8be00af030 0x7f8be0147310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:44.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.536+0000 7f8be4a67640 1 -- 172.21.15.5:0/2387895163 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8be006c180 msgr2=0x7f8be015a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:44.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.536+0000 7f8be4a67640 1 --2- 172.21.15.5:0/2387895163 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8be006c180 0x7f8be015a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:44.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.536+0000 7f8be4a67640 1 -- 172.21.15.5:0/2387895163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be015b200 con 0x7f8be015f700 2024-05-20T18:29:44.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.536+0000 7f8be4a67640 1 --2- 172.21.15.5:0/2387895163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8be015f700 0x7f8be0147ef0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800efc0 tx=0x7f8bd800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:44.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.536+0000 7f8bdcff9640 1 -- 172.21.15.5:0/2387895163 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd800cbe0 con 0x7f8be015f700 2024-05-20T18:29:44.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.537+0000 7f8bdcff9640 1 -- 172.21.15.5:0/2387895163 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd800cd80 con 0x7f8be015f700 2024-05-20T18:29:44.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.537+0000 7f8be64ca640 1 -- 172.21.15.5:0/2387895163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be015b4e0 con 0x7f8be015f700 2024-05-20T18:29:44.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.537+0000 7f8bdcff9640 1 -- 172.21.15.5:0/2387895163 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8003bd0 con 0x7f8be015f700 2024-05-20T18:29:44.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.537+0000 7f8be64ca640 1 -- 172.21.15.5:0/2387895163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be015baa0 con 0x7f8be015f700 2024-05-20T18:29:44.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.538+0000 7f8be64ca640 1 -- 172.21.15.5:0/2387895163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8be015af70 con 0x7f8be015f700 2024-05-20T18:29:44.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.538+0000 7f8bdcff9640 1 -- 172.21.15.5:0/2387895163 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8bd801b030 con 0x7f8be015f700 2024-05-20T18:29:44.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.539+0000 7f8bdcff9640 1 --2- 172.21.15.5:0/2387895163 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8bcc06be30 0x7f8bcc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:44.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.539+0000 7f8bdffff640 1 --2- 172.21.15.5:0/2387895163 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8bcc06be30 0x7f8bcc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:44.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.539+0000 7f8bdcff9640 1 -- 172.21.15.5:0/2387895163 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f8bd808b270 con 0x7f8be015f700 2024-05-20T18:29:44.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.539+0000 7f8bdffff640 1 --2- 172.21.15.5:0/2387895163 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8bcc06be30 0x7f8bcc06e340 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0000f60 tx=0x7f8bd0007cf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:44.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.543+0000 7f8bdcff9640 1 -- 172.21.15.5:0/2387895163 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8bd8022020 con 0x7f8be015f700 2024-05-20T18:29:44.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.728+0000 7f8be64ca640 1 -- 172.21.15.5:0/2387895163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8be0075b00 con 0x7f8be015f700 2024-05-20T18:29:44.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.729+0000 7f8bdcff9640 1 -- 172.21.15.5:0/2387895163 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+16822 (secure 0 0 0) 0x7f8bd8010040 con 0x7f8be015f700 2024-05-20T18:29:44.729 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:29:44.729 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":51,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:29:03.665286+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:29:00.401866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-20T18:29:01.006122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:29:44.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.731+0000 7f8be64ca640 1 -- 172.21.15.5:0/2387895163 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8bcc06be30 msgr2=0x7f8bcc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:44.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.731+0000 7f8be64ca640 1 --2- 172.21.15.5:0/2387895163 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8bcc06be30 0x7f8bcc06e340 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0000f60 tx=0x7f8bd0007cf0 comp rx=0 tx=0).stop 2024-05-20T18:29:44.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.731+0000 7f8be64ca640 1 -- 172.21.15.5:0/2387895163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8be015f700 msgr2=0x7f8be0147ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:44.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.731+0000 7f8be64ca640 1 --2- 172.21.15.5:0/2387895163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8be015f700 0x7f8be0147ef0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800efc0 tx=0x7f8bd800c490 comp rx=0 tx=0).stop 2024-05-20T18:29:44.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.731+0000 7f8be64ca640 1 -- 172.21.15.5:0/2387895163 shutdown_connections 2024-05-20T18:29:44.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.731+0000 7f8be64ca640 1 --2- 172.21.15.5:0/2387895163 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8bcc06be30 0x7f8bcc06e340 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:44.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.731+0000 7f8be64ca640 1 --2- 172.21.15.5:0/2387895163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8be015f700 0x7f8be0147ef0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:44.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.731+0000 7f8be64ca640 1 --2- 172.21.15.5:0/2387895163 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8be00af030 0x7f8be0147310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:44.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.731+0000 7f8be64ca640 1 --2- 172.21.15.5:0/2387895163 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8be006c180 0x7f8be015a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:44.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.731+0000 7f8be64ca640 1 -- 172.21.15.5:0/2387895163 >> 172.21.15.5:0/2387895163 conn(0x7f8be00b6620 msgr2=0x7f8be00d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:44.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.731+0000 7f8be64ca640 1 -- 172.21.15.5:0/2387895163 shutdown_connections 2024-05-20T18:29:44.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.731+0000 7f8be64ca640 1 -- 172.21.15.5:0/2387895163 wait complete. 2024-05-20T18:29:44.742 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:29:00.401866+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:29:01.006122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:29:44.743 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:29:44.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.861+0000 7f83cdbe3640 1 Processor -- start 2024-05-20T18:29:44.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.861+0000 7f83cdbe3640 1 -- start start 2024-05-20T18:29:44.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.862+0000 7f83cdbe3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f83c8145cd0 0x7f83c80cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:44.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.862+0000 7f83cdbe3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83c80b21e0 0x7f83c806bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:44.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.862+0000 7f83cdbe3640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83c80b65c0 con 0x7f83c80cdb40 2024-05-20T18:29:44.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.862+0000 7f83cdbe3640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f83c80ce110 con 0x7f83c8145cd0 2024-05-20T18:29:44.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.862+0000 7f83cdbe3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f83c80ce3f0 con 0x7f83c80b21e0 2024-05-20T18:29:44.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.863+0000 7f83c77fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f83c8145cd0 0x7f83c80cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:44.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.863+0000 7f83c77fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f83c8145cd0 0x7f83c80cd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46370/0 (socket says 172.21.15.5:46370) 2024-05-20T18:29:44.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.863+0000 7f83c77fe640 1 -- 172.21.15.5:0/2866771359 learned_addr learned my addr 172.21.15.5:0/2866771359 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:44.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.863+0000 7f83c77fe640 1 -- 172.21.15.5:0/2866771359 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83c80b21e0 msgr2=0x7f83c806bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:44.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.863+0000 7f83c7fff640 1 --2- 172.21.15.5:0/2866771359 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83c80b21e0 0x7f83c806bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:44.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.863+0000 7f83c77fe640 1 --2- 172.21.15.5:0/2866771359 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83c80b21e0 0x7f83c806bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:44.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.863+0000 7f83c77fe640 1 -- 172.21.15.5:0/2866771359 >> v1:172.21.15.63:6789/0 conn(0x7f83c80cdb40 legacy=0x7f83c80b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:44.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.863+0000 7f83c77fe640 1 -- 172.21.15.5:0/2866771359 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c806b730 con 0x7f83c8145cd0 2024-05-20T18:29:44.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.863+0000 7f83c7fff640 1 --2- 172.21.15.5:0/2866771359 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83c80b21e0 0x7f83c806bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:29:44.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.864+0000 7f83c77fe640 1 --2- 172.21.15.5:0/2866771359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83c8145cd0 0x7f83c80cd600 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f83b8009870 tx=0x7f83b802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd45ef6d1fbf5f8e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:44.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.864+0000 7f83c67fc640 1 -- 172.21.15.5:0/2866771359 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8037070 con 0x7f83c8145cd0 2024-05-20T18:29:44.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.864+0000 7f83c67fc640 1 -- 172.21.15.5:0/2866771359 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83b8032440 con 0x7f83c8145cd0 2024-05-20T18:29:44.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.864+0000 7f83c67fc640 1 -- 172.21.15.5:0/2866771359 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8032810 con 0x7f83c8145cd0 2024-05-20T18:29:44.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.865+0000 7f83cdbe3640 1 -- 172.21.15.5:0/2866771359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83c8145cd0 msgr2=0x7f83c80cd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:44.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.865+0000 7f83cdbe3640 1 --2- 172.21.15.5:0/2866771359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83c8145cd0 0x7f83c80cd600 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f83b8009870 tx=0x7f83b802a210 comp rx=0 tx=0).stop 2024-05-20T18:29:44.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.865+0000 7f83cdbe3640 1 -- 172.21.15.5:0/2866771359 shutdown_connections 2024-05-20T18:29:44.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.865+0000 7f83cdbe3640 1 --2- 172.21.15.5:0/2866771359 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83c80b21e0 0x7f83c806bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:44.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.865+0000 7f83cdbe3640 1 --2- 172.21.15.5:0/2866771359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83c8145cd0 0x7f83c80cd600 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:44.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.865+0000 7f83cdbe3640 1 -- 172.21.15.5:0/2866771359 >> 172.21.15.5:0/2866771359 conn(0x7f83c809a4f0 msgr2=0x7f83c80b5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:44.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.865+0000 7f83cdbe3640 1 -- 172.21.15.5:0/2866771359 shutdown_connections 2024-05-20T18:29:44.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.865+0000 7f83cdbe3640 1 -- 172.21.15.5:0/2866771359 wait complete. 2024-05-20T18:29:44.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.866+0000 7f83cdbe3640 1 Processor -- start 2024-05-20T18:29:44.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.867+0000 7f83cdbe3640 1 -- start start 2024-05-20T18:29:44.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.868+0000 7f83cdbe3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83c80b21e0 0x7f83c8134780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:44.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.868+0000 7f83cdbe3640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83c80cdb40 0x7f83c8132d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:44.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.868+0000 7f83cdbe3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83c8145cd0 0x7f83c815e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:44.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.868+0000 7f83cdbe3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f83c806c5f0 con 0x7f83c8145cd0 2024-05-20T18:29:44.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.868+0000 7f83cdbe3640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f83c806c470 con 0x7f83c80cdb40 2024-05-20T18:29:44.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.868+0000 7f83cdbe3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f83c806c770 con 0x7f83c80b21e0 2024-05-20T18:29:44.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.868+0000 7f83c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83c8145cd0 0x7f83c815e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:44.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.868+0000 7f83c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83c8145cd0 0x7f83c815e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46386/0 (socket says 172.21.15.5:46386) 2024-05-20T18:29:44.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.868+0000 7f83c7fff640 1 -- 172.21.15.5:0/4006082701 learned_addr learned my addr 172.21.15.5:0/4006082701 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:44.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.868+0000 7f83c77fe640 1 --2- 172.21.15.5:0/4006082701 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83c80b21e0 0x7f83c8134780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:44.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.868+0000 7f83c6ffd640 1 --2- 172.21.15.5:0/4006082701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83c80cdb40 0x7f83c8132d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:44.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.868+0000 7f83c7fff640 1 -- 172.21.15.5:0/4006082701 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83c80b21e0 msgr2=0x7f83c8134780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:44.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.868+0000 7f83c7fff640 1 --2- 172.21.15.5:0/4006082701 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83c80b21e0 0x7f83c8134780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:44.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.868+0000 7f83c7fff640 1 -- 172.21.15.5:0/4006082701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83c80cdb40 msgr2=0x7f83c8132d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:44.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.868+0000 7f83c7fff640 1 --2- 172.21.15.5:0/4006082701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83c80cdb40 0x7f83c8132d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:44.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.868+0000 7f83c7fff640 1 -- 172.21.15.5:0/4006082701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c8133260 con 0x7f83c8145cd0 2024-05-20T18:29:44.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.868+0000 7f83c77fe640 1 --2- 172.21.15.5:0/4006082701 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83c80b21e0 0x7f83c8134780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:29:44.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.869+0000 7f83c7fff640 1 --2- 172.21.15.5:0/4006082701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83c8145cd0 0x7f83c815e760 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f83c000cd80 tx=0x7f83c000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:44.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.869+0000 7f83affff640 1 -- 172.21.15.5:0/4006082701 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c000f680 con 0x7f83c8145cd0 2024-05-20T18:29:44.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.869+0000 7f83affff640 1 -- 172.21.15.5:0/4006082701 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c000f840 con 0x7f83c8145cd0 2024-05-20T18:29:44.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.869+0000 7f83affff640 1 -- 172.21.15.5:0/4006082701 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c000fc30 con 0x7f83c8145cd0 2024-05-20T18:29:44.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.869+0000 7f83cdbe3640 1 -- 172.21.15.5:0/4006082701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83c81336e0 con 0x7f83c8145cd0 2024-05-20T18:29:44.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.869+0000 7f83cdbe3640 1 -- 172.21.15.5:0/4006082701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83c8133ca0 con 0x7f83c8145cd0 2024-05-20T18:29:44.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.870+0000 7f83cdbe3640 1 -- 172.21.15.5:0/4006082701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83c8075b00 con 0x7f83c8145cd0 2024-05-20T18:29:44.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.870+0000 7f83affff640 1 -- 172.21.15.5:0/4006082701 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f83c00213f0 con 0x7f83c8145cd0 2024-05-20T18:29:44.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.871+0000 7f83affff640 1 --2- 172.21.15.5:0/4006082701 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f83b406be30 0x7f83b406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:44.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.871+0000 7f83affff640 1 -- 172.21.15.5:0/4006082701 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f83c008bdd0 con 0x7f83c8145cd0 2024-05-20T18:29:44.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.871+0000 7f83c77fe640 1 --2- 172.21.15.5:0/4006082701 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f83b406be30 0x7f83b406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:44.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.871+0000 7f83c77fe640 1 --2- 172.21.15.5:0/4006082701 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f83b406be30 0x7f83b406e340 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f83b8034000 tx=0x7f83b802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:44.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:44.875+0000 7f83affff640 1 -- 172.21.15.5:0/4006082701 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f83c0014030 con 0x7f83c8145cd0 2024-05-20T18:29:45.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.057+0000 7f83cdbe3640 1 -- 172.21.15.5:0/4006082701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f83c8144c20 con 0x7f83c8145cd0 2024-05-20T18:29:45.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.058+0000 7f83affff640 1 -- 172.21.15.5:0/4006082701 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) ==== 93+0+10 (secure 0 0 0) 0x7f83c008d020 con 0x7f83c8145cd0 2024-05-20T18:29:45.058 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:29:45.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.060+0000 7f83cdbe3640 1 -- 172.21.15.5:0/4006082701 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f83b406be30 msgr2=0x7f83b406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:45.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.060+0000 7f83cdbe3640 1 --2- 172.21.15.5:0/4006082701 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f83b406be30 0x7f83b406e340 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f83b8034000 tx=0x7f83b802e000 comp rx=0 tx=0).stop 2024-05-20T18:29:45.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.060+0000 7f83cdbe3640 1 -- 172.21.15.5:0/4006082701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83c8145cd0 msgr2=0x7f83c815e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:45.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.060+0000 7f83cdbe3640 1 --2- 172.21.15.5:0/4006082701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83c8145cd0 0x7f83c815e760 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f83c000cd80 tx=0x7f83c000a650 comp rx=0 tx=0).stop 2024-05-20T18:29:45.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.060+0000 7f83cdbe3640 1 -- 172.21.15.5:0/4006082701 shutdown_connections 2024-05-20T18:29:45.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.060+0000 7f83cdbe3640 1 --2- 172.21.15.5:0/4006082701 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f83b406be30 0x7f83b406e340 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:45.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.060+0000 7f83cdbe3640 1 --2- 172.21.15.5:0/4006082701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83c8145cd0 0x7f83c815e760 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:45.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.060+0000 7f83cdbe3640 1 --2- 172.21.15.5:0/4006082701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83c80cdb40 0x7f83c8132d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:45.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.060+0000 7f83cdbe3640 1 --2- 172.21.15.5:0/4006082701 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83c80b21e0 0x7f83c8134780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:45.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.060+0000 7f83cdbe3640 1 -- 172.21.15.5:0/4006082701 >> 172.21.15.5:0/4006082701 conn(0x7f83c809a4f0 msgr2=0x7f83c80b5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:45.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.060+0000 7f83cdbe3640 1 -- 172.21.15.5:0/4006082701 shutdown_connections 2024-05-20T18:29:45.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.060+0000 7f83cdbe3640 1 -- 172.21.15.5:0/4006082701 wait complete. 2024-05-20T18:29:45.071 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:29:45.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.188+0000 7f3c3ec91640 1 Processor -- start 2024-05-20T18:29:45.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.188+0000 7f3c3ec91640 1 -- start start 2024-05-20T18:29:45.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.190+0000 7f3c3ec91640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c38168050 0x7f3c3816a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:45.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.190+0000 7f3c3ec91640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c380d0630 con 0x7f3c3806d3d0 2024-05-20T18:29:45.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.190+0000 7f3c3ec91640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c380d39c0 con 0x7f3c38162820 2024-05-20T18:29:45.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.190+0000 7f3c3ec91640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3c380d3ca0 con 0x7f3c38168050 2024-05-20T18:29:45.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.190+0000 7f3c37fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3c3806d3d0 0x7f3c3806d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37258/0 (socket says 172.21.15.5:37258) 2024-05-20T18:29:45.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.190+0000 7f3c37fff640 1 -- 172.21.15.5:0/4290870644 learned_addr learned my addr 172.21.15.5:0/4290870644 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:45.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.190+0000 7f3c3d22e640 1 --2- 172.21.15.5:0/4290870644 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c38168050 0x7f3c3816a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:45.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.190+0000 7f3c377fe640 1 -- 172.21.15.5:0/4290870644 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3225201020 0 0) 0x7f3c380d0630 con 0x7f3c3806d3d0 2024-05-20T18:29:45.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.190+0000 7f3c377fe640 1 -- 172.21.15.5:0/4290870644 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c20003520 con 0x7f3c3806d3d0 2024-05-20T18:29:45.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.190+0000 7f3c377fe640 1 -- 172.21.15.5:0/4290870644 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 419520576 0 0) 0x7f3c380d39c0 con 0x7f3c38162820 2024-05-20T18:29:45.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.190+0000 7f3c377fe640 1 -- 172.21.15.5:0/4290870644 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c380d0630 con 0x7f3c38162820 2024-05-20T18:29:45.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.190+0000 7f3c377fe640 1 -- 172.21.15.5:0/4290870644 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1447205036 0 0) 0x7f3c20003520 con 0x7f3c3806d3d0 2024-05-20T18:29:45.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.190+0000 7f3c377fe640 1 -- 172.21.15.5:0/4290870644 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c38168050 msgr2=0x7f3c3816a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:45.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.190+0000 7f3c377fe640 1 --2- 172.21.15.5:0/4290870644 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c38168050 0x7f3c3816a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:45.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.190+0000 7f3c377fe640 1 -- 172.21.15.5:0/4290870644 >> v1:172.21.15.63:6789/0 conn(0x7f3c38162820 legacy=0x7f3c380b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:45.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.190+0000 7f3c377fe640 1 -- 172.21.15.5:0/4290870644 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c380d3f80 con 0x7f3c3806d3d0 2024-05-20T18:29:45.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.190+0000 7f3c377fe640 1 -- 172.21.15.5:0/4290870644 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f3c2c001670 con 0x7f3c3806d3d0 2024-05-20T18:29:45.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.190+0000 7f3c377fe640 1 -- 172.21.15.5:0/4290870644 >> v1:172.21.15.5:6789/0 conn(0x7f3c3806d3d0 legacy=0x7f3c3806d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:45.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.191+0000 7f3c377fe640 1 --2- 172.21.15.5:0/4290870644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c20003b70 0x7f3c20003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:45.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.191+0000 7f3c377fe640 1 --2- 172.21.15.5:0/4290870644 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c20004800 0x7f3c20004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:45.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.191+0000 7f3c377fe640 1 --2- 172.21.15.5:0/4290870644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c20005150 0x7f3c2000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:45.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.191+0000 7f3c377fe640 1 -- 172.21.15.5:0/4290870644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3c380d3ca0 con 0x7f3c20003b70 2024-05-20T18:29:45.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.191+0000 7f3c377fe640 1 -- 172.21.15.5:0/4290870644 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2000c090 con 0x7f3c20005150 2024-05-20T18:29:45.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.191+0000 7f3c377fe640 1 -- 172.21.15.5:0/4290870644 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3c2000c290 con 0x7f3c20004800 2024-05-20T18:29:45.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.191+0000 7f3c3d22e640 1 --2- 172.21.15.5:0/4290870644 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c38168050 0x7f3c3816a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:29:45.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.191+0000 7f3c3ca2d640 1 --2- 172.21.15.5:0/4290870644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c20003b70 0x7f3c20003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:45.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.191+0000 7f3c37fff640 1 --2- 172.21.15.5:0/4290870644 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c20004800 0x7f3c20004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:45.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.191+0000 7f3c3d22e640 1 --2- 172.21.15.5:0/4290870644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c20005150 0x7f3c2000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:45.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.191+0000 7f3c3ca2d640 1 -- 172.21.15.5:0/4290870644 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c20004800 msgr2=0x7f3c20004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:45.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.191+0000 7f3c3ca2d640 1 --2- 172.21.15.5:0/4290870644 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c20004800 0x7f3c20004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:45.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.191+0000 7f3c3ca2d640 1 -- 172.21.15.5:0/4290870644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c20005150 msgr2=0x7f3c2000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:45.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.191+0000 7f3c3ca2d640 1 --2- 172.21.15.5:0/4290870644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c20005150 0x7f3c2000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:45.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.191+0000 7f3c3ca2d640 1 -- 172.21.15.5:0/4290870644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c20003520 con 0x7f3c20003b70 2024-05-20T18:29:45.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.191+0000 7f3c3d22e640 1 --2- 172.21.15.5:0/4290870644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c20005150 0x7f3c2000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:29:45.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.192+0000 7f3c3ca2d640 1 --2- 172.21.15.5:0/4290870644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c20003b70 0x7f3c20003fc0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f3c280018c0 tx=0x7f3c20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b406fe7d9e67bdb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:45.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.192+0000 7f3c377fe640 1 -- 172.21.15.5:0/4290870644 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c280014e0 con 0x7f3c20003b70 2024-05-20T18:29:45.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.192+0000 7f3c377fe640 1 -- 172.21.15.5:0/4290870644 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c280052f0 con 0x7f3c20003b70 2024-05-20T18:29:45.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.192+0000 7f3c377fe640 1 -- 172.21.15.5:0/4290870644 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2802ac90 con 0x7f3c20003b70 2024-05-20T18:29:45.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.192+0000 7f3c3ec91640 1 -- 172.21.15.5:0/4290870644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c20003b70 msgr2=0x7f3c20003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:45.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.192+0000 7f3c3ec91640 1 --2- 172.21.15.5:0/4290870644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c20003b70 0x7f3c20003fc0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f3c280018c0 tx=0x7f3c20005830 comp rx=0 tx=0).stop 2024-05-20T18:29:45.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.192+0000 7f3c3d22e640 1 -- 172.21.15.5:0/4290870644 reap_dead start 2024-05-20T18:29:45.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.193+0000 7f3c3ec91640 1 -- 172.21.15.5:0/4290870644 shutdown_connections 2024-05-20T18:29:45.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.193+0000 7f3c3ec91640 1 -- 172.21.15.5:0/4290870644 >> 172.21.15.5:0/4290870644 conn(0x7f3c380d4bd0 msgr2=0x7f3c380d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:45.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.193+0000 7f3c3ec91640 1 -- 172.21.15.5:0/4290870644 shutdown_connections 2024-05-20T18:29:45.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.193+0000 7f3c3ec91640 1 -- 172.21.15.5:0/4290870644 wait complete. 2024-05-20T18:29:45.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.194+0000 7f3c3ec91640 1 Processor -- start 2024-05-20T18:29:45.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.194+0000 7f3c3ec91640 1 -- start start 2024-05-20T18:29:45.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.196+0000 7f3c3ec91640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c20003b70 0x7f3c380cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:45.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.196+0000 7f3c3ec91640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c381555f0 0x7f3c3814ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:45.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.196+0000 7f3c3ec91640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c380c8f40 0x7f3c380c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:45.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.196+0000 7f3c3ec91640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3c380b91a0 con 0x7f3c380c8f40 2024-05-20T18:29:45.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.196+0000 7f3c3ec91640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3c380d0300 con 0x7f3c381555f0 2024-05-20T18:29:45.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.196+0000 7f3c3ec91640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3c380b4050 con 0x7f3c20003b70 2024-05-20T18:29:45.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.196+0000 7f3c3d22e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c380c8f40 0x7f3c380c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:45.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.196+0000 7f3c3d22e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c380c8f40 0x7f3c380c9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46400/0 (socket says 172.21.15.5:46400) 2024-05-20T18:29:45.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.196+0000 7f3c3d22e640 1 -- 172.21.15.5:0/932885003 learned_addr learned my addr 172.21.15.5:0/932885003 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:45.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.196+0000 7f3c37fff640 1 --2- 172.21.15.5:0/932885003 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c381555f0 0x7f3c3814ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:45.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.196+0000 7f3c3ca2d640 1 --2- 172.21.15.5:0/932885003 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c20003b70 0x7f3c380cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:45.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.196+0000 7f3c3d22e640 1 -- 172.21.15.5:0/932885003 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c20003b70 msgr2=0x7f3c380cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:45.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.196+0000 7f3c3d22e640 1 --2- 172.21.15.5:0/932885003 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c20003b70 0x7f3c380cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:45.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.196+0000 7f3c3d22e640 1 -- 172.21.15.5:0/932885003 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c381555f0 msgr2=0x7f3c3814ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:45.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.196+0000 7f3c3d22e640 1 --2- 172.21.15.5:0/932885003 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c381555f0 0x7f3c3814ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:45.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.196+0000 7f3c3d22e640 1 -- 172.21.15.5:0/932885003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c38154b70 con 0x7f3c380c8f40 2024-05-20T18:29:45.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.196+0000 7f3c3ca2d640 1 --2- 172.21.15.5:0/932885003 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c20003b70 0x7f3c380cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:29:45.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.197+0000 7f3c3d22e640 1 --2- 172.21.15.5:0/932885003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c380c8f40 0x7f3c380c9350 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f3c380cbc50 tx=0x7f3c30010880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:45.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.197+0000 7f3c357fa640 1 -- 172.21.15.5:0/932885003 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30003b90 con 0x7f3c380c8f40 2024-05-20T18:29:45.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.197+0000 7f3c357fa640 1 -- 172.21.15.5:0/932885003 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c3001b070 con 0x7f3c380c8f40 2024-05-20T18:29:45.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.197+0000 7f3c357fa640 1 -- 172.21.15.5:0/932885003 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30004030 con 0x7f3c380c8f40 2024-05-20T18:29:45.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.197+0000 7f3c3ec91640 1 -- 172.21.15.5:0/932885003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c380cb540 con 0x7f3c380c8f40 2024-05-20T18:29:45.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.197+0000 7f3c3ec91640 1 -- 172.21.15.5:0/932885003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c380c8b80 con 0x7f3c380c8f40 2024-05-20T18:29:45.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.198+0000 7f3c3ec91640 1 -- 172.21.15.5:0/932885003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c380b7590 con 0x7f3c380c8f40 2024-05-20T18:29:45.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.198+0000 7f3c357fa640 1 -- 172.21.15.5:0/932885003 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3c30005d10 con 0x7f3c380c8f40 2024-05-20T18:29:45.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.199+0000 7f3c357fa640 1 --2- 172.21.15.5:0/932885003 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3c2406be30 0x7f3c2406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:45.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.199+0000 7f3c357fa640 1 -- 172.21.15.5:0/932885003 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f3c3008be10 con 0x7f3c380c8f40 2024-05-20T18:29:45.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.199+0000 7f3c3ca2d640 1 --2- 172.21.15.5:0/932885003 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3c2406be30 0x7f3c2406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:45.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.200+0000 7f3c3ca2d640 1 --2- 172.21.15.5:0/932885003 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3c2406be30 0x7f3c2406e340 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3c280018c0 tx=0x7f3c2802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:45.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.204+0000 7f3c357fa640 1 -- 172.21.15.5:0/932885003 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3c380b7590 con 0x7f3c380c8f40 2024-05-20T18:29:45.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.385+0000 7f3c3ec91640 1 -- 172.21.15.5:0/932885003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3c380ced50 con 0x7f3c380c8f40 2024-05-20T18:29:45.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.386+0000 7f3c357fa640 1 -- 172.21.15.5:0/932885003 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v51) ==== 104+0+11 (secure 0 0 0) 0x7f3c30025020 con 0x7f3c380c8f40 2024-05-20T18:29:45.386 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:29:45.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.388+0000 7f3c3ec91640 1 -- 172.21.15.5:0/932885003 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3c2406be30 msgr2=0x7f3c2406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:45.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.388+0000 7f3c3ec91640 1 --2- 172.21.15.5:0/932885003 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3c2406be30 0x7f3c2406e340 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3c280018c0 tx=0x7f3c2802e000 comp rx=0 tx=0).stop 2024-05-20T18:29:45.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.388+0000 7f3c3ec91640 1 -- 172.21.15.5:0/932885003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c380c8f40 msgr2=0x7f3c380c9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:45.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.388+0000 7f3c3ec91640 1 --2- 172.21.15.5:0/932885003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c380c8f40 0x7f3c380c9350 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f3c380cbc50 tx=0x7f3c30010880 comp rx=0 tx=0).stop 2024-05-20T18:29:45.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.388+0000 7f3c3ec91640 1 -- 172.21.15.5:0/932885003 shutdown_connections 2024-05-20T18:29:45.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.388+0000 7f3c3ec91640 1 --2- 172.21.15.5:0/932885003 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3c2406be30 0x7f3c2406e340 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:45.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.388+0000 7f3c3ec91640 1 --2- 172.21.15.5:0/932885003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c380c8f40 0x7f3c380c9350 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:45.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.388+0000 7f3c3ec91640 1 --2- 172.21.15.5:0/932885003 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c381555f0 0x7f3c3814ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:45.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.388+0000 7f3c3ec91640 1 --2- 172.21.15.5:0/932885003 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c20003b70 0x7f3c380cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:45.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.388+0000 7f3c3ec91640 1 -- 172.21.15.5:0/932885003 >> 172.21.15.5:0/932885003 conn(0x7f3c380d4bd0 msgr2=0x7f3c38160890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:45.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.388+0000 7f3c3ec91640 1 -- 172.21.15.5:0/932885003 shutdown_connections 2024-05-20T18:29:45.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.388+0000 7f3c3ec91640 1 -- 172.21.15.5:0/932885003 wait complete. 2024-05-20T18:29:45.399 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:29:45.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.516+0000 7fc4c8673640 1 Processor -- start 2024-05-20T18:29:45.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.516+0000 7fc4c8673640 1 -- start start 2024-05-20T18:29:45.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.518+0000 7fc4c8673640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc4c0162820 0x7fc4c00b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.518+0000 7fc4c8673640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc4c00b3e80 0x7fc4c006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.518+0000 7fc4c8673640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4c006dd20 con 0x7fc4c006d910 2024-05-20T18:29:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.518+0000 7fc4c8673640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fc4c00d0630 con 0x7fc4c0162820 2024-05-20T18:29:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.518+0000 7fc4c8673640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc4c00d39c0 con 0x7fc4c00b3e80 2024-05-20T18:29:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.518+0000 7fc4c640f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc4c0162820 0x7fc4c00b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.518+0000 7fc4c640f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc4c0162820 0x7fc4c00b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46414/0 (socket says 172.21.15.5:46414) 2024-05-20T18:29:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.518+0000 7fc4c6c10640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc4c006d910 0x7fc4c00d27a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37266/0 (socket says 172.21.15.5:37266) 2024-05-20T18:29:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.518+0000 7fc4c640f640 1 -- 172.21.15.5:0/2830140160 learned_addr learned my addr 172.21.15.5:0/2830140160 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.518+0000 7fc4c5c0e640 1 --2- 172.21.15.5:0/2830140160 >> v2:172.21.15.63:3300/0 conn(0x7fc4c00b3e80 0x7fc4c006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.518+0000 7fc4c640f640 1 -- 172.21.15.5:0/2830140160 >> v2:172.21.15.63:3300/0 conn(0x7fc4c00b3e80 msgr2=0x7fc4c006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.518+0000 7fc4c640f640 1 --2- 172.21.15.5:0/2830140160 >> v2:172.21.15.63:3300/0 conn(0x7fc4c00b3e80 0x7fc4c006d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:45.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.518+0000 7fc4c640f640 1 -- 172.21.15.5:0/2830140160 >> v1:172.21.15.5:6789/0 conn(0x7fc4c006d910 legacy=0x7fc4c00d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:45.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.518+0000 7fc4c540d640 1 -- 172.21.15.5:0/2830140160 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2026359699 0 0) 0x7fc4b8001110 con 0x7fc4c006d910 2024-05-20T18:29:45.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.518+0000 7fc4c640f640 1 -- 172.21.15.5:0/2830140160 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c00d3ca0 con 0x7fc4c0162820 2024-05-20T18:29:45.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.519+0000 7fc4c640f640 1 --2- 172.21.15.5:0/2830140160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4c0162820 0x7fc4c00b3940 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00d1230 tx=0x7fc4b002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3add01d81c09a3c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.519+0000 7fc4c540d640 1 -- 172.21.15.5:0/2830140160 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0036070 con 0x7fc4c0162820 2024-05-20T18:29:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.519+0000 7fc4c540d640 1 -- 172.21.15.5:0/2830140160 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4b0031600 con 0x7fc4c0162820 2024-05-20T18:29:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.519+0000 7fc4c540d640 1 -- 172.21.15.5:0/2830140160 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b00319d0 con 0x7fc4c0162820 2024-05-20T18:29:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.520+0000 7fc4c8673640 1 -- 172.21.15.5:0/2830140160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4c0162820 msgr2=0x7fc4c00b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.520+0000 7fc4c8673640 1 --2- 172.21.15.5:0/2830140160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4c0162820 0x7fc4c00b3940 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00d1230 tx=0x7fc4b002a730 comp rx=0 tx=0).stop 2024-05-20T18:29:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.520+0000 7fc4c8673640 1 -- 172.21.15.5:0/2830140160 shutdown_connections 2024-05-20T18:29:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.520+0000 7fc4c8673640 1 --2- 172.21.15.5:0/2830140160 >> v2:172.21.15.63:3300/0 conn(0x7fc4c00b3e80 0x7fc4c006d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.520+0000 7fc4c8673640 1 --2- 172.21.15.5:0/2830140160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4c0162820 0x7fc4c00b3940 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.520+0000 7fc4c8673640 1 -- 172.21.15.5:0/2830140160 >> 172.21.15.5:0/2830140160 conn(0x7fc4c00d4bd0 msgr2=0x7fc4c00d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.520+0000 7fc4c8673640 1 -- 172.21.15.5:0/2830140160 shutdown_connections 2024-05-20T18:29:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.520+0000 7fc4c8673640 1 -- 172.21.15.5:0/2830140160 wait complete. 2024-05-20T18:29:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.522+0000 7fc4c8673640 1 Processor -- start 2024-05-20T18:29:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.522+0000 7fc4c8673640 1 -- start start 2024-05-20T18:29:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.523+0000 7fc4c8673640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4c006d910 0x7fc4c00c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.523+0000 7fc4c8673640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4c00b3e80 0x7fc4c00c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.523+0000 7fc4c8673640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4c0162820 0x7fc4c00ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.523+0000 7fc4c8673640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c00d0300 con 0x7fc4c00b3e80 2024-05-20T18:29:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.523+0000 7fc4c8673640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c00d4260 con 0x7fc4c0162820 2024-05-20T18:29:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.523+0000 7fc4c8673640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc4c00b91a0 con 0x7fc4c006d910 2024-05-20T18:29:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.523+0000 7fc4c5c0e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4c00b3e80 0x7fc4c00c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.524+0000 7fc4c5c0e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4c00b3e80 0x7fc4c00c9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46418/0 (socket says 172.21.15.5:46418) 2024-05-20T18:29:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.524+0000 7fc4c5c0e640 1 -- 172.21.15.5:0/2549249756 learned_addr learned my addr 172.21.15.5:0/2549249756 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.524+0000 7fc4c6c10640 1 --2- 172.21.15.5:0/2549249756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4c0162820 0x7fc4c00ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.524+0000 7fc4c640f640 1 --2- 172.21.15.5:0/2549249756 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4c006d910 0x7fc4c00c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.524+0000 7fc4c5c0e640 1 -- 172.21.15.5:0/2549249756 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4c006d910 msgr2=0x7fc4c00c98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.524+0000 7fc4c5c0e640 1 --2- 172.21.15.5:0/2549249756 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4c006d910 0x7fc4c00c98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.524+0000 7fc4c5c0e640 1 -- 172.21.15.5:0/2549249756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4c0162820 msgr2=0x7fc4c00ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.524+0000 7fc4c5c0e640 1 --2- 172.21.15.5:0/2549249756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4c0162820 0x7fc4c00ccec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.524+0000 7fc4c5c0e640 1 -- 172.21.15.5:0/2549249756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c00cbe20 con 0x7fc4c00b3e80 2024-05-20T18:29:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.524+0000 7fc4c5c0e640 1 --2- 172.21.15.5:0/2549249756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4c00b3e80 0x7fc4c00c9de0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400e870 tx=0x7fc4b400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.524+0000 7fc4beffd640 1 -- 172.21.15.5:0/2549249756 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4017070 con 0x7fc4c00b3e80 2024-05-20T18:29:45.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.524+0000 7fc4beffd640 1 -- 172.21.15.5:0/2549249756 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4b400c940 con 0x7fc4c00b3e80 2024-05-20T18:29:45.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.524+0000 7fc4c8673640 1 -- 172.21.15.5:0/2549249756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c00cc100 con 0x7fc4c00b3e80 2024-05-20T18:29:45.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.524+0000 7fc4beffd640 1 -- 172.21.15.5:0/2549249756 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b400cd30 con 0x7fc4c00b3e80 2024-05-20T18:29:45.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.524+0000 7fc4c8673640 1 -- 172.21.15.5:0/2549249756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c00cc6c0 con 0x7fc4c00b3e80 2024-05-20T18:29:45.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.526+0000 7fc4c8673640 1 -- 172.21.15.5:0/2549249756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4c0075b00 con 0x7fc4c00b3e80 2024-05-20T18:29:45.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.526+0000 7fc4beffd640 1 -- 172.21.15.5:0/2549249756 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc4b401a070 con 0x7fc4c00b3e80 2024-05-20T18:29:45.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.526+0000 7fc4beffd640 1 --2- 172.21.15.5:0/2549249756 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc4ac06be30 0x7fc4ac06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:45.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.527+0000 7fc4beffd640 1 -- 172.21.15.5:0/2549249756 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7fc4b408b2e0 con 0x7fc4c00b3e80 2024-05-20T18:29:45.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.527+0000 7fc4c640f640 1 --2- 172.21.15.5:0/2549249756 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc4ac06be30 0x7fc4ac06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:45.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.527+0000 7fc4c640f640 1 --2- 172.21.15.5:0/2549249756 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc4ac06be30 0x7fc4ac06e340 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0033000 tx=0x7fc4b0009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:45.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.531+0000 7fc4beffd640 1 -- 172.21.15.5:0/2549249756 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc4b4059380 con 0x7fc4c00b3e80 2024-05-20T18:29:45.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.714+0000 7fc4c8673640 1 -- 172.21.15.5:0/2549249756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc4c00b9620 con 0x7fc4c00b3e80 2024-05-20T18:29:45.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.714+0000 7fc4beffd640 1 -- 172.21.15.5:0/2549249756 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v51) ==== 100+0+11 (secure 0 0 0) 0x7fc4b405d680 con 0x7fc4c00b3e80 2024-05-20T18:29:45.714 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:29:45.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.716+0000 7fc4c8673640 1 -- 172.21.15.5:0/2549249756 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc4ac06be30 msgr2=0x7fc4ac06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:45.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.716+0000 7fc4c8673640 1 --2- 172.21.15.5:0/2549249756 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc4ac06be30 0x7fc4ac06e340 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0033000 tx=0x7fc4b0009000 comp rx=0 tx=0).stop 2024-05-20T18:29:45.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.716+0000 7fc4c8673640 1 -- 172.21.15.5:0/2549249756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4c00b3e80 msgr2=0x7fc4c00c9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:45.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.716+0000 7fc4c8673640 1 --2- 172.21.15.5:0/2549249756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4c00b3e80 0x7fc4c00c9de0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400e870 tx=0x7fc4b400ed30 comp rx=0 tx=0).stop 2024-05-20T18:29:45.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.716+0000 7fc4c8673640 1 -- 172.21.15.5:0/2549249756 shutdown_connections 2024-05-20T18:29:45.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.716+0000 7fc4c8673640 1 --2- 172.21.15.5:0/2549249756 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc4ac06be30 0x7fc4ac06e340 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:45.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.716+0000 7fc4c8673640 1 --2- 172.21.15.5:0/2549249756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4c0162820 0x7fc4c00ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:45.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.716+0000 7fc4c8673640 1 --2- 172.21.15.5:0/2549249756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4c00b3e80 0x7fc4c00c9de0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:45.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.716+0000 7fc4c8673640 1 --2- 172.21.15.5:0/2549249756 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4c006d910 0x7fc4c00c98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:45.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.716+0000 7fc4c8673640 1 -- 172.21.15.5:0/2549249756 >> 172.21.15.5:0/2549249756 conn(0x7fc4c00d4bd0 msgr2=0x7fc4c00b62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:45.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.717+0000 7fc4c8673640 1 -- 172.21.15.5:0/2549249756 shutdown_connections 2024-05-20T18:29:45.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.717+0000 7fc4c8673640 1 -- 172.21.15.5:0/2549249756 wait complete. 2024-05-20T18:29:45.728 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:29:45.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.843+0000 7fe0de8a8640 1 Processor -- start 2024-05-20T18:29:45.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.843+0000 7fe0de8a8640 1 -- start start 2024-05-20T18:29:45.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.844+0000 7fe0de8a8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0d806d3d0 0x7fe0d806d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:45.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.844+0000 7fe0de8a8640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe0d80b8190 0x7fe0d80b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:45.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.844+0000 7fe0de8a8640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0d806dd20 con 0x7fe0d8162820 2024-05-20T18:29:45.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.844+0000 7fe0de8a8640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe0d80d0630 con 0x7fe0d80b8190 2024-05-20T18:29:45.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.844+0000 7fe0de8a8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe0d80d39c0 con 0x7fe0d806d3d0 2024-05-20T18:29:45.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.845+0000 7fe0d7fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe0d8162820 0x7fe0d80b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37280/0 (socket says 172.21.15.5:37280) 2024-05-20T18:29:45.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.845+0000 7fe0d7fff640 1 -- 172.21.15.5:0/248393751 learned_addr learned my addr 172.21.15.5:0/248393751 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:45.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.845+0000 7fe0d77fe640 1 --2- 172.21.15.5:0/248393751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0d806d3d0 0x7fe0d806d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:45.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.845+0000 7fe0dce45640 1 --2- 172.21.15.5:0/248393751 >> v2:172.21.15.63:3300/0 conn(0x7fe0d80b8190 0x7fe0d80b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:45.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.845+0000 7fe0d6ffd640 1 -- 172.21.15.5:0/248393751 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1178474458 0 0) 0x7fe0c8001110 con 0x7fe0d8162820 2024-05-20T18:29:45.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.845+0000 7fe0d6ffd640 1 -- 172.21.15.5:0/248393751 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0c0003520 con 0x7fe0d8162820 2024-05-20T18:29:45.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.845+0000 7fe0d6ffd640 1 -- 172.21.15.5:0/248393751 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2412923857 0 0) 0x7fe0c0003520 con 0x7fe0d8162820 2024-05-20T18:29:45.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.845+0000 7fe0d6ffd640 1 -- 172.21.15.5:0/248393751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0d806d3d0 msgr2=0x7fe0d806d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:45.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.845+0000 7fe0d6ffd640 1 --2- 172.21.15.5:0/248393751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0d806d3d0 0x7fe0d806d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:45.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.845+0000 7fe0d6ffd640 1 -- 172.21.15.5:0/248393751 >> v2:172.21.15.63:3300/0 conn(0x7fe0d80b8190 msgr2=0x7fe0d80b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:45.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.845+0000 7fe0d6ffd640 1 --2- 172.21.15.5:0/248393751 >> v2:172.21.15.63:3300/0 conn(0x7fe0d80b8190 0x7fe0d80b85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:45.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.845+0000 7fe0d6ffd640 1 -- 172.21.15.5:0/248393751 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d80d3ca0 con 0x7fe0d8162820 2024-05-20T18:29:45.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.845+0000 7fe0d6ffd640 1 -- 172.21.15.5:0/248393751 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7fe0c8001840 con 0x7fe0d8162820 2024-05-20T18:29:45.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.845+0000 7fe0d6ffd640 1 -- 172.21.15.5:0/248393751 >> v1:172.21.15.5:6789/0 conn(0x7fe0d8162820 legacy=0x7fe0d80b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:45.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.845+0000 7fe0d6ffd640 1 --2- 172.21.15.5:0/248393751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0d806d3d0 0x7fe0c00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:45.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.845+0000 7fe0d6ffd640 1 --2- 172.21.15.5:0/248393751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0c0003f30 0x7fe0c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:45.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.845+0000 7fe0d6ffd640 1 --2- 172.21.15.5:0/248393751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0c0004d40 0x7fe0c0005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:45.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.845+0000 7fe0d6ffd640 1 -- 172.21.15.5:0/248393751 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d80d0630 con 0x7fe0c0003f30 2024-05-20T18:29:45.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.845+0000 7fe0d6ffd640 1 -- 172.21.15.5:0/248393751 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d80d39c0 con 0x7fe0c0004d40 2024-05-20T18:29:45.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.845+0000 7fe0d6ffd640 1 -- 172.21.15.5:0/248393751 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe0c000ba10 con 0x7fe0d806d3d0 2024-05-20T18:29:45.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.845+0000 7fe0d77fe640 1 --2- 172.21.15.5:0/248393751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0c0003f30 0x7fe0c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:45.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.845+0000 7fe0d7fff640 1 --2- 172.21.15.5:0/248393751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0d806d3d0 0x7fe0c00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:45.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.846+0000 7fe0dce45640 1 --2- 172.21.15.5:0/248393751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0c0004d40 0x7fe0c0005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:45.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.846+0000 7fe0d77fe640 1 -- 172.21.15.5:0/248393751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0d806d3d0 msgr2=0x7fe0c00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:45.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.846+0000 7fe0d77fe640 1 --2- 172.21.15.5:0/248393751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0d806d3d0 0x7fe0c00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:45.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.846+0000 7fe0d77fe640 1 -- 172.21.15.5:0/248393751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0c0004d40 msgr2=0x7fe0c0005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:45.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.846+0000 7fe0d77fe640 1 --2- 172.21.15.5:0/248393751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0c0004d40 0x7fe0c0005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:45.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.846+0000 7fe0d77fe640 1 -- 172.21.15.5:0/248393751 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c0003520 con 0x7fe0c0003f30 2024-05-20T18:29:45.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.846+0000 7fe0dce45640 1 --2- 172.21.15.5:0/248393751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0c0004d40 0x7fe0c0005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:29:45.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.846+0000 7fe0d77fe640 1 --2- 172.21.15.5:0/248393751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0c0003f30 0x7fe0c0004800 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc009a90 tx=0x7fe0c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8846c0bc99168351 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:45.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.847+0000 7fe0d6ffd640 1 -- 172.21.15.5:0/248393751 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc036070 con 0x7fe0c0003f30 2024-05-20T18:29:45.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.847+0000 7fe0d6ffd640 1 -- 172.21.15.5:0/248393751 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0cc031400 con 0x7fe0c0003f30 2024-05-20T18:29:45.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.847+0000 7fe0d6ffd640 1 -- 172.21.15.5:0/248393751 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc009690 con 0x7fe0c0003f30 2024-05-20T18:29:45.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.847+0000 7fe0de8a8640 1 -- 172.21.15.5:0/248393751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0c0003f30 msgr2=0x7fe0c0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:45.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.847+0000 7fe0de8a8640 1 --2- 172.21.15.5:0/248393751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0c0003f30 0x7fe0c0004800 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc009a90 tx=0x7fe0c0005830 comp rx=0 tx=0).stop 2024-05-20T18:29:45.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.847+0000 7fe0dce45640 1 -- 172.21.15.5:0/248393751 reap_dead start 2024-05-20T18:29:45.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.847+0000 7fe0de8a8640 1 -- 172.21.15.5:0/248393751 shutdown_connections 2024-05-20T18:29:45.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.847+0000 7fe0de8a8640 1 -- 172.21.15.5:0/248393751 >> 172.21.15.5:0/248393751 conn(0x7fe0d80d4bd0 msgr2=0x7fe0d80d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:45.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.847+0000 7fe0de8a8640 1 -- 172.21.15.5:0/248393751 shutdown_connections 2024-05-20T18:29:45.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.847+0000 7fe0de8a8640 1 -- 172.21.15.5:0/248393751 wait complete. 2024-05-20T18:29:45.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.849+0000 7fe0de8a8640 1 Processor -- start 2024-05-20T18:29:45.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.849+0000 7fe0de8a8640 1 -- start start 2024-05-20T18:29:45.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.851+0000 7fe0de8a8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0c0003f30 0x7fe0d80cd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:45.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.851+0000 7fe0de8a8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0d80cb6d0 0x7fe0d81555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:45.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.851+0000 7fe0de8a8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0d80c9090 0x7fe0d80cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:45.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.851+0000 7fe0de8a8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d80d0300 con 0x7fe0d80c9090 2024-05-20T18:29:45.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.851+0000 7fe0de8a8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d80d4260 con 0x7fe0c0003f30 2024-05-20T18:29:45.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.851+0000 7fe0de8a8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe0d80b4050 con 0x7fe0d80cb6d0 2024-05-20T18:29:45.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.851+0000 7fe0dce45640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0d80c9090 0x7fe0d80cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:45.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.851+0000 7fe0dce45640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0d80c9090 0x7fe0d80cc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46426/0 (socket says 172.21.15.5:46426) 2024-05-20T18:29:45.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.851+0000 7fe0dce45640 1 -- 172.21.15.5:0/1025554972 learned_addr learned my addr 172.21.15.5:0/1025554972 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:45.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.851+0000 7fe0d7fff640 1 --2- 172.21.15.5:0/1025554972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0c0003f30 0x7fe0d80cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:45.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.851+0000 7fe0d77fe640 1 --2- 172.21.15.5:0/1025554972 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0d80cb6d0 0x7fe0d81555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:45.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.851+0000 7fe0dce45640 1 -- 172.21.15.5:0/1025554972 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0d80cb6d0 msgr2=0x7fe0d81555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:45.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.851+0000 7fe0dce45640 1 --2- 172.21.15.5:0/1025554972 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0d80cb6d0 0x7fe0d81555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:45.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.851+0000 7fe0dce45640 1 -- 172.21.15.5:0/1025554972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0c0003f30 msgr2=0x7fe0d80cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:45.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.851+0000 7fe0dce45640 1 --2- 172.21.15.5:0/1025554972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0c0003f30 0x7fe0d80cd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:45.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.851+0000 7fe0dce45640 1 -- 172.21.15.5:0/1025554972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d8155ae0 con 0x7fe0d80c9090 2024-05-20T18:29:45.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.851+0000 7fe0dce45640 1 --2- 172.21.15.5:0/1025554972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0d80c9090 0x7fe0d80cc850 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe0d808e090 tx=0x7fe0d000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:45.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.851+0000 7fe0d4ff9640 1 -- 172.21.15.5:0/1025554972 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0016420 con 0x7fe0d80c9090 2024-05-20T18:29:45.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.851+0000 7fe0d4ff9640 1 -- 172.21.15.5:0/1025554972 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0d001b070 con 0x7fe0d80c9090 2024-05-20T18:29:45.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.851+0000 7fe0de8a8640 1 -- 172.21.15.5:0/1025554972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0d80c8da0 con 0x7fe0d80c9090 2024-05-20T18:29:45.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.852+0000 7fe0d4ff9640 1 -- 172.21.15.5:0/1025554972 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d00167e0 con 0x7fe0d80c9090 2024-05-20T18:29:45.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.852+0000 7fe0de8a8640 1 -- 172.21.15.5:0/1025554972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0d80c8ac0 con 0x7fe0d80c9090 2024-05-20T18:29:45.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.852+0000 7fe0de8a8640 1 -- 172.21.15.5:0/1025554972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0d8075b00 con 0x7fe0d80c9090 2024-05-20T18:29:45.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.853+0000 7fe0d4ff9640 1 -- 172.21.15.5:0/1025554972 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe0d0005d10 con 0x7fe0d80c9090 2024-05-20T18:29:45.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.854+0000 7fe0d4ff9640 1 --2- 172.21.15.5:0/1025554972 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe0c406be30 0x7fe0c406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:45.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.854+0000 7fe0d4ff9640 1 -- 172.21.15.5:0/1025554972 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7fe0d008bac0 con 0x7fe0d80c9090 2024-05-20T18:29:45.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.854+0000 7fe0d7fff640 1 --2- 172.21.15.5:0/1025554972 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe0c406be30 0x7fe0c406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:45.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.854+0000 7fe0d7fff640 1 --2- 172.21.15.5:0/1025554972 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe0c406be30 0x7fe0c406e340 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe0c80017a0 tx=0x7fe0c8007aa0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:45.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:45.858+0000 7fe0d4ff9640 1 -- 172.21.15.5:0/1025554972 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe0d0025020 con 0x7fe0d80c9090 2024-05-20T18:29:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.081+0000 7fe0de8a8640 1 -- 172.21.15.5:0/1025554972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0d8161190 con 0x7fe0d80c9090 2024-05-20T18:29:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.082+0000 7fe0d4ff9640 1 -- 172.21.15.5:0/1025554972 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5175 (secure 0 0 0) 0x7fe0d8155ae0 con 0x7fe0d80c9090 2024-05-20T18:29:46.082 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:29:46.082 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5207,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3958852803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3958852803},{"type":"v1","addr":"172.21.15.63:6835","nonce":3958852803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5560,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/3020169757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":3020169757},{"type":"v1","addr":"172.21.15.63:6834","nonce":3020169757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2965018210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2965018210},{"type":"v1","addr":"172.21.15.5:6835","nonce":2965018210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:29:02.038442+0000","modified":"2024-05-20T18:29:05.055248+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5751},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5751":{"gid":5751,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6837/3469824363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3469824363},{"type":"v1","addr":"172.21.15.5:6837","nonce":3469824363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5751,"qdb_cluster":[5751]},"id":3}]} 2024-05-20T18:29:46.082 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-05-20T18:29:46.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.084+0000 7fe0de8a8640 1 -- 172.21.15.5:0/1025554972 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe0c406be30 msgr2=0x7fe0c406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:46.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.084+0000 7fe0de8a8640 1 --2- 172.21.15.5:0/1025554972 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe0c406be30 0x7fe0c406e340 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe0c80017a0 tx=0x7fe0c8007aa0 comp rx=0 tx=0).stop 2024-05-20T18:29:46.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.084+0000 7fe0de8a8640 1 -- 172.21.15.5:0/1025554972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0d80c9090 msgr2=0x7fe0d80cc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:46.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.084+0000 7fe0de8a8640 1 --2- 172.21.15.5:0/1025554972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0d80c9090 0x7fe0d80cc850 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe0d808e090 tx=0x7fe0d000f520 comp rx=0 tx=0).stop 2024-05-20T18:29:46.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.084+0000 7fe0de8a8640 1 -- 172.21.15.5:0/1025554972 shutdown_connections 2024-05-20T18:29:46.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.084+0000 7fe0de8a8640 1 --2- 172.21.15.5:0/1025554972 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe0c406be30 0x7fe0c406e340 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:46.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.084+0000 7fe0de8a8640 1 --2- 172.21.15.5:0/1025554972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0d80c9090 0x7fe0d80cc850 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:46.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.084+0000 7fe0de8a8640 1 --2- 172.21.15.5:0/1025554972 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0d80cb6d0 0x7fe0d81555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:46.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.084+0000 7fe0de8a8640 1 --2- 172.21.15.5:0/1025554972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0c0003f30 0x7fe0d80cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:46.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.084+0000 7fe0de8a8640 1 -- 172.21.15.5:0/1025554972 >> 172.21.15.5:0/1025554972 conn(0x7fe0d80d4bd0 msgr2=0x7fe0d80c7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:46.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.084+0000 7fe0de8a8640 1 -- 172.21.15.5:0/1025554972 shutdown_connections 2024-05-20T18:29:46.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.084+0000 7fe0de8a8640 1 -- 172.21.15.5:0/1025554972 wait complete. 2024-05-20T18:29:46.095 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:29:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.210+0000 7f0f4779d640 1 Processor -- start 2024-05-20T18:29:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.210+0000 7f0f4779d640 1 -- start start 2024-05-20T18:29:46.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.211+0000 7f0f4779d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0f40162820 0x7f0f400b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.212+0000 7f0f4779d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f4006dae0 0x7f0f400b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.212+0000 7f0f4779d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f400d0630 con 0x7f0f400b3e80 2024-05-20T18:29:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.212+0000 7f0f4779d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0f400d39c0 con 0x7f0f40162820 2024-05-20T18:29:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.212+0000 7f0f4779d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0f400d3ca0 con 0x7f0f4006dae0 2024-05-20T18:29:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.212+0000 7f0f44d38640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0f400b3e80 0x7f0f4006d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37296/0 (socket says 172.21.15.5:37296) 2024-05-20T18:29:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.212+0000 7f0f44d38640 1 -- 172.21.15.5:0/4022329704 learned_addr learned my addr 172.21.15.5:0/4022329704 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.212+0000 7f0f37fff640 1 -- 172.21.15.5:0/4022329704 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3374091866 0 0) 0x7f0f400d0630 con 0x7f0f400b3e80 2024-05-20T18:29:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.212+0000 7f0f37fff640 1 -- 172.21.15.5:0/4022329704 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f2c003520 con 0x7f0f400b3e80 2024-05-20T18:29:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.212+0000 7f0f45d3a640 1 --2- 172.21.15.5:0/4022329704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f4006dae0 0x7f0f400b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.212+0000 7f0f45539640 1 --2- 172.21.15.5:0/4022329704 >> v2:172.21.15.63:3300/0 conn(0x7f0f40162820 0x7f0f400b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.212+0000 7f0f37fff640 1 -- 172.21.15.5:0/4022329704 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3264977732 0 0) 0x7f0f2c003520 con 0x7f0f400b3e80 2024-05-20T18:29:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.212+0000 7f0f37fff640 1 -- 172.21.15.5:0/4022329704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f4006dae0 msgr2=0x7f0f400b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.212+0000 7f0f37fff640 1 --2- 172.21.15.5:0/4022329704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f4006dae0 0x7f0f400b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.212+0000 7f0f37fff640 1 -- 172.21.15.5:0/4022329704 >> v2:172.21.15.63:3300/0 conn(0x7f0f40162820 msgr2=0x7f0f400b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.212+0000 7f0f37fff640 1 --2- 172.21.15.5:0/4022329704 >> v2:172.21.15.63:3300/0 conn(0x7f0f40162820 0x7f0f400b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.212+0000 7f0f37fff640 1 -- 172.21.15.5:0/4022329704 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f400d3f80 con 0x7f0f400b3e80 2024-05-20T18:29:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.212+0000 7f0f37fff640 1 -- 172.21.15.5:0/4022329704 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f0f30001670 con 0x7f0f400b3e80 2024-05-20T18:29:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.212+0000 7f0f37fff640 1 -- 172.21.15.5:0/4022329704 >> v1:172.21.15.5:6789/0 conn(0x7f0f400b3e80 legacy=0x7f0f4006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.212+0000 7f0f37fff640 1 --2- 172.21.15.5:0/4022329704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f2c0039b0 0x7f0f2c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.212+0000 7f0f37fff640 1 --2- 172.21.15.5:0/4022329704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f2c004340 0x7f0f2c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.212+0000 7f0f37fff640 1 --2- 172.21.15.5:0/4022329704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f2c004ed0 0x7f0f2c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.212+0000 7f0f37fff640 1 -- 172.21.15.5:0/4022329704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0f400d39c0 con 0x7f0f2c004340 2024-05-20T18:29:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.212+0000 7f0f37fff640 1 -- 172.21.15.5:0/4022329704 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0f400d3ca0 con 0x7f0f2c004ed0 2024-05-20T18:29:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.212+0000 7f0f37fff640 1 -- 172.21.15.5:0/4022329704 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0f2c00bf00 con 0x7f0f2c0039b0 2024-05-20T18:29:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.213+0000 7f0f44d38640 1 --2- 172.21.15.5:0/4022329704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f2c004340 0x7f0f2c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.213+0000 7f0f45539640 1 --2- 172.21.15.5:0/4022329704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f2c0039b0 0x7f0f2c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.213+0000 7f0f45d3a640 1 --2- 172.21.15.5:0/4022329704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f2c004ed0 0x7f0f2c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.213+0000 7f0f44d38640 1 -- 172.21.15.5:0/4022329704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f2c0039b0 msgr2=0x7f0f2c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.213+0000 7f0f44d38640 1 --2- 172.21.15.5:0/4022329704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f2c0039b0 0x7f0f2c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.213+0000 7f0f44d38640 1 -- 172.21.15.5:0/4022329704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f2c004ed0 msgr2=0x7f0f2c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.213+0000 7f0f44d38640 1 --2- 172.21.15.5:0/4022329704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f2c004ed0 0x7f0f2c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.213+0000 7f0f44d38640 1 -- 172.21.15.5:0/4022329704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2c003520 con 0x7f0f2c004340 2024-05-20T18:29:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.214+0000 7f0f44d38640 1 --2- 172.21.15.5:0/4022329704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f2c004340 0x7f0f2c004860 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000c240 tx=0x7f0f2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecf560940f992c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.214+0000 7f0f37fff640 1 -- 172.21.15.5:0/4022329704 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3002e070 con 0x7f0f2c004340 2024-05-20T18:29:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.214+0000 7f0f37fff640 1 -- 172.21.15.5:0/4022329704 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f30005af0 con 0x7f0f2c004340 2024-05-20T18:29:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.214+0000 7f0f37fff640 1 -- 172.21.15.5:0/4022329704 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f30007d30 con 0x7f0f2c004340 2024-05-20T18:29:46.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.214+0000 7f0f4779d640 1 -- 172.21.15.5:0/4022329704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f2c004340 msgr2=0x7f0f2c004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:46.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.214+0000 7f0f4779d640 1 --2- 172.21.15.5:0/4022329704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f2c004340 0x7f0f2c004860 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000c240 tx=0x7f0f2c005830 comp rx=0 tx=0).stop 2024-05-20T18:29:46.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.214+0000 7f0f45d3a640 1 -- 172.21.15.5:0/4022329704 reap_dead start 2024-05-20T18:29:46.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.214+0000 7f0f4779d640 1 -- 172.21.15.5:0/4022329704 shutdown_connections 2024-05-20T18:29:46.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.214+0000 7f0f4779d640 1 -- 172.21.15.5:0/4022329704 >> 172.21.15.5:0/4022329704 conn(0x7f0f400d4bd0 msgr2=0x7f0f400d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:46.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.215+0000 7f0f4779d640 1 -- 172.21.15.5:0/4022329704 shutdown_connections 2024-05-20T18:29:46.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.215+0000 7f0f4779d640 1 -- 172.21.15.5:0/4022329704 wait complete. 2024-05-20T18:29:46.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.216+0000 7f0f4779d640 1 Processor -- start 2024-05-20T18:29:46.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.216+0000 7f0f4779d640 1 -- start start 2024-05-20T18:29:46.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.218+0000 7f0f4779d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f2c004340 0x7f0f4015ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:46.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.218+0000 7f0f4779d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f4014dd20 0x7f0f4014a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:46.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.218+0000 7f0f4779d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f4014b040 0x7f0f400b6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:46.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.218+0000 7f0f4779d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0f400d4780 con 0x7f0f4014dd20 2024-05-20T18:29:46.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.218+0000 7f0f4779d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0f400d0300 con 0x7f0f4014b040 2024-05-20T18:29:46.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.218+0000 7f0f4779d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0f400d4900 con 0x7f0f2c004340 2024-05-20T18:29:46.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.218+0000 7f0f44d38640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f4014dd20 0x7f0f4014a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:46.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.218+0000 7f0f44d38640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f4014dd20 0x7f0f4014a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46430/0 (socket says 172.21.15.5:46430) 2024-05-20T18:29:46.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.218+0000 7f0f44d38640 1 -- 172.21.15.5:0/3551753467 learned_addr learned my addr 172.21.15.5:0/3551753467 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:46.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.218+0000 7f0f45539640 1 --2- 172.21.15.5:0/3551753467 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f2c004340 0x7f0f4015ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:46.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.218+0000 7f0f45d3a640 1 --2- 172.21.15.5:0/3551753467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f4014b040 0x7f0f400b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:46.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.218+0000 7f0f44d38640 1 -- 172.21.15.5:0/3551753467 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f2c004340 msgr2=0x7f0f4015ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:46.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.218+0000 7f0f44d38640 1 --2- 172.21.15.5:0/3551753467 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f2c004340 0x7f0f4015ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:46.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.218+0000 7f0f44d38640 1 -- 172.21.15.5:0/3551753467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f4014b040 msgr2=0x7f0f400b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:46.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.218+0000 7f0f44d38640 1 --2- 172.21.15.5:0/3551753467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f4014b040 0x7f0f400b6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:46.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.218+0000 7f0f44d38640 1 -- 172.21.15.5:0/3551753467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f400b5690 con 0x7f0f4014dd20 2024-05-20T18:29:46.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.218+0000 7f0f45539640 1 --2- 172.21.15.5:0/3551753467 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f2c004340 0x7f0f4015ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:29:46.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.218+0000 7f0f44d38640 1 --2- 172.21.15.5:0/3551753467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f4014dd20 0x7f0f4014a8c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000c360 tx=0x7f0f400b7a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:46.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.219+0000 7f0f35ffb640 1 -- 172.21.15.5:0/3551753467 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f30042070 con 0x7f0f4014dd20 2024-05-20T18:29:46.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.219+0000 7f0f35ffb640 1 -- 172.21.15.5:0/3551753467 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f30007480 con 0x7f0f4014dd20 2024-05-20T18:29:46.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.219+0000 7f0f4779d640 1 -- 172.21.15.5:0/3551753467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f400b5970 con 0x7f0f4014dd20 2024-05-20T18:29:46.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.219+0000 7f0f35ffb640 1 -- 172.21.15.5:0/3551753467 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f30001c70 con 0x7f0f4014dd20 2024-05-20T18:29:46.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.219+0000 7f0f4779d640 1 -- 172.21.15.5:0/3551753467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f400b5f30 con 0x7f0f4014dd20 2024-05-20T18:29:46.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.220+0000 7f0f4779d640 1 -- 172.21.15.5:0/3551753467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f4015ef40 con 0x7f0f4014dd20 2024-05-20T18:29:46.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.220+0000 7f0f35ffb640 1 -- 172.21.15.5:0/3551753467 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0f30045070 con 0x7f0f4014dd20 2024-05-20T18:29:46.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.221+0000 7f0f35ffb640 1 --2- 172.21.15.5:0/3551753467 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0f2406be60 0x7f0f2406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:46.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.221+0000 7f0f35ffb640 1 -- 172.21.15.5:0/3551753467 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f0f300aba40 con 0x7f0f4014dd20 2024-05-20T18:29:46.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.221+0000 7f0f45539640 1 --2- 172.21.15.5:0/3551753467 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0f2406be60 0x7f0f2406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:46.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.221+0000 7f0f45539640 1 --2- 172.21.15.5:0/3551753467 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0f2406be60 0x7f0f2406e370 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009bc0 tx=0x7f0f3c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.225+0000 7f0f35ffb640 1 -- 172.21.15.5:0/3551753467 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0f30005ac0 con 0x7f0f4014dd20 2024-05-20T18:29:46.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.409+0000 7f0f4779d640 1 -- 172.21.15.5:0/3551753467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f40075b00 con 0x7f0f4014dd20 2024-05-20T18:29:46.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.410+0000 7f0f35ffb640 1 -- 172.21.15.5:0/3551753467 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+16822 (secure 0 0 0) 0x7f0f3007a390 con 0x7f0f4014dd20 2024-05-20T18:29:46.410 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:29:46.410 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":51,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:29:03.665286+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:29:00.401866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-20T18:29:01.006122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:29:46.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.412+0000 7f0f4779d640 1 -- 172.21.15.5:0/3551753467 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0f2406be60 msgr2=0x7f0f2406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:46.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.412+0000 7f0f4779d640 1 --2- 172.21.15.5:0/3551753467 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0f2406be60 0x7f0f2406e370 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009bc0 tx=0x7f0f3c008040 comp rx=0 tx=0).stop 2024-05-20T18:29:46.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.412+0000 7f0f4779d640 1 -- 172.21.15.5:0/3551753467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f4014dd20 msgr2=0x7f0f4014a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:46.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.412+0000 7f0f4779d640 1 --2- 172.21.15.5:0/3551753467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f4014dd20 0x7f0f4014a8c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000c360 tx=0x7f0f400b7a20 comp rx=0 tx=0).stop 2024-05-20T18:29:46.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.412+0000 7f0f4779d640 1 -- 172.21.15.5:0/3551753467 shutdown_connections 2024-05-20T18:29:46.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.412+0000 7f0f4779d640 1 --2- 172.21.15.5:0/3551753467 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0f2406be60 0x7f0f2406e370 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:46.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.412+0000 7f0f4779d640 1 --2- 172.21.15.5:0/3551753467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f4014b040 0x7f0f400b6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:46.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.412+0000 7f0f4779d640 1 --2- 172.21.15.5:0/3551753467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f4014dd20 0x7f0f4014a8c0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:46.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.412+0000 7f0f4779d640 1 --2- 172.21.15.5:0/3551753467 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0f2c004340 0x7f0f4015ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:46.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.412+0000 7f0f4779d640 1 -- 172.21.15.5:0/3551753467 >> 172.21.15.5:0/3551753467 conn(0x7f0f400d4bd0 msgr2=0x7f0f4015dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:46.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.412+0000 7f0f4779d640 1 -- 172.21.15.5:0/3551753467 shutdown_connections 2024-05-20T18:29:46.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.413+0000 7f0f4779d640 1 -- 172.21.15.5:0/3551753467 wait complete. 2024-05-20T18:29:46.424 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-20T18:29:46.424 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:29:46.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.539+0000 7f4aadf77640 1 Processor -- start 2024-05-20T18:29:46.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.539+0000 7f4aadf77640 1 -- start start 2024-05-20T18:29:46.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.540+0000 7f4aadf77640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f4aa8162820 0x7f4aa80b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:46.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.540+0000 7f4aadf77640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4aa80d0630 con 0x7f4aa80b3e80 2024-05-20T18:29:46.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.540+0000 7f4aadf77640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4aa80d39c0 con 0x7f4aa806dae0 2024-05-20T18:29:46.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.540+0000 7f4aadf77640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f4aa80d3ca0 con 0x7f4aa8162820 2024-05-20T18:29:46.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.540+0000 7f4aa6ffd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f4aa80b3e80 0x7f4aa806d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37308/0 (socket says 172.21.15.5:37308) 2024-05-20T18:29:46.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.540+0000 7f4aa6ffd640 1 -- 172.21.15.5:0/2337048510 learned_addr learned my addr 172.21.15.5:0/2337048510 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:46.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.540+0000 7f4aa77fe640 1 --2- 172.21.15.5:0/2337048510 >> v2:172.21.15.63:3300/0 conn(0x7f4aa8162820 0x7f4aa80b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:46.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.540+0000 7f4aa67fc640 1 -- 172.21.15.5:0/2337048510 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3603238438 0 0) 0x7f4aa80d0630 con 0x7f4aa80b3e80 2024-05-20T18:29:46.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.540+0000 7f4aa67fc640 1 -- 172.21.15.5:0/2337048510 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a90003520 con 0x7f4aa80b3e80 2024-05-20T18:29:46.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.541+0000 7f4aa67fc640 1 -- 172.21.15.5:0/2337048510 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2224440778 0 0) 0x7f4a90003520 con 0x7f4aa80b3e80 2024-05-20T18:29:46.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.541+0000 7f4aa67fc640 1 -- 172.21.15.5:0/2337048510 >> v2:172.21.15.63:3300/0 conn(0x7f4aa8162820 msgr2=0x7f4aa80b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:46.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.541+0000 7f4aa67fc640 1 --2- 172.21.15.5:0/2337048510 >> v2:172.21.15.63:3300/0 conn(0x7f4aa8162820 0x7f4aa80b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:46.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.541+0000 7f4aa67fc640 1 -- 172.21.15.5:0/2337048510 >> v1:172.21.15.63:6789/0 conn(0x7f4aa806dae0 legacy=0x7f4aa80b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:46.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.541+0000 7f4aa67fc640 1 -- 172.21.15.5:0/2337048510 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa80d3f80 con 0x7f4aa80b3e80 2024-05-20T18:29:46.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.541+0000 7f4aa67fc640 1 -- 172.21.15.5:0/2337048510 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f4a9c001670 con 0x7f4aa80b3e80 2024-05-20T18:29:46.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.541+0000 7f4aa67fc640 1 -- 172.21.15.5:0/2337048510 >> v1:172.21.15.5:6789/0 conn(0x7f4aa80b3e80 legacy=0x7f4aa806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:46.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.541+0000 7f4aa67fc640 1 --2- 172.21.15.5:0/2337048510 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a900039b0 0x7f4a90003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:46.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.541+0000 7f4aa67fc640 1 --2- 172.21.15.5:0/2337048510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a90004800 0x7f4a90004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:46.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.541+0000 7f4aa67fc640 1 --2- 172.21.15.5:0/2337048510 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4a900051c0 0x7f4a9000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:46.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.541+0000 7f4aa67fc640 1 -- 172.21.15.5:0/2337048510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa80d3ca0 con 0x7f4a90004800 2024-05-20T18:29:46.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.541+0000 7f4aa67fc640 1 -- 172.21.15.5:0/2337048510 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9000c160 con 0x7f4a900039b0 2024-05-20T18:29:46.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.541+0000 7f4aa67fc640 1 -- 172.21.15.5:0/2337048510 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4a9000c360 con 0x7f4a900051c0 2024-05-20T18:29:46.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.541+0000 7f4aa6ffd640 1 --2- 172.21.15.5:0/2337048510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a90004800 0x7f4a90004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:46.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.541+0000 7f4aa77fe640 1 --2- 172.21.15.5:0/2337048510 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a900039b0 0x7f4a90003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:46.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.541+0000 7f4aa7fff640 1 --2- 172.21.15.5:0/2337048510 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4a900051c0 0x7f4a9000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:46.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.541+0000 7f4aa6ffd640 1 -- 172.21.15.5:0/2337048510 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4a900051c0 msgr2=0x7f4a9000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:46.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.541+0000 7f4aa6ffd640 1 --2- 172.21.15.5:0/2337048510 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4a900051c0 0x7f4a9000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:46.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.541+0000 7f4aa6ffd640 1 -- 172.21.15.5:0/2337048510 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a900039b0 msgr2=0x7f4a90003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:46.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.541+0000 7f4aa6ffd640 1 --2- 172.21.15.5:0/2337048510 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a900039b0 0x7f4a90003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:46.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.541+0000 7f4aa6ffd640 1 -- 172.21.15.5:0/2337048510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a90003520 con 0x7f4a90004800 2024-05-20T18:29:46.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.541+0000 7f4aa77fe640 1 --2- 172.21.15.5:0/2337048510 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4a900039b0 0x7f4a90003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:29:46.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.541+0000 7f4aa7fff640 1 -- 172.21.15.5:0/2337048510 reap_dead start 2024-05-20T18:29:46.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.542+0000 7f4aa6ffd640 1 --2- 172.21.15.5:0/2337048510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a90004800 0x7f4a90004c10 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00c240 tx=0x7f4a90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5b2c0037df5b76 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:46.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.542+0000 7f4aa67fc640 1 -- 172.21.15.5:0/2337048510 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c007cf0 con 0x7f4a90004800 2024-05-20T18:29:46.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.542+0000 7f4aa67fc640 1 -- 172.21.15.5:0/2337048510 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a9c0058d0 con 0x7f4a90004800 2024-05-20T18:29:46.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.542+0000 7f4aa67fc640 1 -- 172.21.15.5:0/2337048510 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c005d20 con 0x7f4a90004800 2024-05-20T18:29:46.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.543+0000 7f4aadf77640 1 -- 172.21.15.5:0/2337048510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a90004800 msgr2=0x7f4a90004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:46.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.543+0000 7f4aadf77640 1 --2- 172.21.15.5:0/2337048510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a90004800 0x7f4a90004c10 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00c240 tx=0x7f4a90005830 comp rx=0 tx=0).stop 2024-05-20T18:29:46.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.543+0000 7f4aadf77640 1 -- 172.21.15.5:0/2337048510 shutdown_connections 2024-05-20T18:29:46.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.543+0000 7f4aadf77640 1 --2- 172.21.15.5:0/2337048510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a90004800 0x7f4a90004c10 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:46.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.543+0000 7f4aadf77640 1 -- 172.21.15.5:0/2337048510 >> 172.21.15.5:0/2337048510 conn(0x7f4aa80d4bd0 msgr2=0x7f4aa80d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:46.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.543+0000 7f4aadf77640 1 -- 172.21.15.5:0/2337048510 shutdown_connections 2024-05-20T18:29:46.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.543+0000 7f4aadf77640 1 -- 172.21.15.5:0/2337048510 wait complete. 2024-05-20T18:29:46.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.545+0000 7f4aadf77640 1 Processor -- start 2024-05-20T18:29:46.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.545+0000 7f4aadf77640 1 -- start start 2024-05-20T18:29:46.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.546+0000 7f4aadf77640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4a90004800 0x7f4aa80b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:46.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.546+0000 7f4aadf77640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4aa80c9cd0 0x7f4aa80ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:46.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.546+0000 7f4aadf77640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4aa80ca620 0x7f4aa814cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:46.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.546+0000 7f4aadf77640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa80d4780 con 0x7f4aa80c9cd0 2024-05-20T18:29:46.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.546+0000 7f4aadf77640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa80d0300 con 0x7f4aa80ca620 2024-05-20T18:29:46.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.546+0000 7f4aadf77640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4aa80d4900 con 0x7f4a90004800 2024-05-20T18:29:46.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.546+0000 7f4aa6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4aa80c9cd0 0x7f4aa80ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:46.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.546+0000 7f4aa6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4aa80c9cd0 0x7f4aa80ca0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46450/0 (socket says 172.21.15.5:46450) 2024-05-20T18:29:46.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.546+0000 7f4aa6ffd640 1 -- 172.21.15.5:0/1826092117 learned_addr learned my addr 172.21.15.5:0/1826092117 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:46.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.547+0000 7f4aa77fe640 1 --2- 172.21.15.5:0/1826092117 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4a90004800 0x7f4aa80b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:46.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.547+0000 7f4aa7fff640 1 --2- 172.21.15.5:0/1826092117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4aa80ca620 0x7f4aa814cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:46.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.547+0000 7f4aa6ffd640 1 -- 172.21.15.5:0/1826092117 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4a90004800 msgr2=0x7f4aa80b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:46.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.547+0000 7f4aa6ffd640 1 --2- 172.21.15.5:0/1826092117 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4a90004800 0x7f4aa80b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:46.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.547+0000 7f4aa6ffd640 1 -- 172.21.15.5:0/1826092117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4aa80ca620 msgr2=0x7f4aa814cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:46.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.547+0000 7f4aa6ffd640 1 --2- 172.21.15.5:0/1826092117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4aa80ca620 0x7f4aa814cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:46.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.547+0000 7f4aa6ffd640 1 -- 172.21.15.5:0/1826092117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa815d7f0 con 0x7f4aa80c9cd0 2024-05-20T18:29:46.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.547+0000 7f4aa6ffd640 1 --2- 172.21.15.5:0/1826092117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4aa80c9cd0 0x7f4aa80ca0e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00c360 tx=0x7f4aa80d0800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:46.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.547+0000 7f4a8ffff640 1 -- 172.21.15.5:0/1826092117 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c001c30 con 0x7f4aa80c9cd0 2024-05-20T18:29:46.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.547+0000 7f4a8ffff640 1 -- 172.21.15.5:0/1826092117 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a9c041070 con 0x7f4aa80c9cd0 2024-05-20T18:29:46.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.547+0000 7f4aadf77640 1 -- 172.21.15.5:0/1826092117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aa815ddb0 con 0x7f4aa80c9cd0 2024-05-20T18:29:46.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.547+0000 7f4a8ffff640 1 -- 172.21.15.5:0/1826092117 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c000ed0 con 0x7f4aa80c9cd0 2024-05-20T18:29:46.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.547+0000 7f4aadf77640 1 -- 172.21.15.5:0/1826092117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aa814bfc0 con 0x7f4aa80c9cd0 2024-05-20T18:29:46.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.548+0000 7f4aadf77640 1 -- 172.21.15.5:0/1826092117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4aa8160b20 con 0x7f4aa80c9cd0 2024-05-20T18:29:46.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.549+0000 7f4a8ffff640 1 -- 172.21.15.5:0/1826092117 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4a9c03b070 con 0x7f4aa80c9cd0 2024-05-20T18:29:46.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.549+0000 7f4a8ffff640 1 --2- 172.21.15.5:0/1826092117 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4a9406bfd0 0x7f4a9406e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:46.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.549+0000 7f4a8ffff640 1 -- 172.21.15.5:0/1826092117 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f4a9c0aab50 con 0x7f4aa80c9cd0 2024-05-20T18:29:46.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.549+0000 7f4aa77fe640 1 --2- 172.21.15.5:0/1826092117 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4a9406bfd0 0x7f4a9406e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:46.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.550+0000 7f4aa77fe640 1 --2- 172.21.15.5:0/1826092117 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4a9406bfd0 0x7f4a9406e4e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4a98009b10 tx=0x7f4a98008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:46.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.554+0000 7f4a8ffff640 1 -- 172.21.15.5:0/1826092117 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4aa815d7f0 con 0x7f4aa80c9cd0 2024-05-20T18:29:46.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.777+0000 7f4aadf77640 1 -- 172.21.15.5:0/1826092117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4aa8075b00 con 0x7f4aa80c9cd0 2024-05-20T18:29:46.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.777+0000 7f4a8ffff640 1 -- 172.21.15.5:0/1826092117 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5175 (secure 0 0 0) 0x7f4a9c0792d0 con 0x7f4aa80c9cd0 2024-05-20T18:29:46.777 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:29:46.778 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5207,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3958852803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3958852803},{"type":"v1","addr":"172.21.15.63:6835","nonce":3958852803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5560,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/3020169757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":3020169757},{"type":"v1","addr":"172.21.15.63:6834","nonce":3020169757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2965018210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2965018210},{"type":"v1","addr":"172.21.15.5:6835","nonce":2965018210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:29:02.038442+0000","modified":"2024-05-20T18:29:05.055248+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5751},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5751":{"gid":5751,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6837/3469824363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3469824363},{"type":"v1","addr":"172.21.15.5:6837","nonce":3469824363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5751,"qdb_cluster":[5751]},"id":3}]} 2024-05-20T18:29:46.778 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-05-20T18:29:46.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.779+0000 7f4aadf77640 1 -- 172.21.15.5:0/1826092117 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4a9406bfd0 msgr2=0x7f4a9406e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:46.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.779+0000 7f4aadf77640 1 --2- 172.21.15.5:0/1826092117 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4a9406bfd0 0x7f4a9406e4e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4a98009b10 tx=0x7f4a98008040 comp rx=0 tx=0).stop 2024-05-20T18:29:46.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.779+0000 7f4aadf77640 1 -- 172.21.15.5:0/1826092117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4aa80c9cd0 msgr2=0x7f4aa80ca0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:46.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.779+0000 7f4aadf77640 1 --2- 172.21.15.5:0/1826092117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4aa80c9cd0 0x7f4aa80ca0e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00c360 tx=0x7f4aa80d0800 comp rx=0 tx=0).stop 2024-05-20T18:29:46.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.779+0000 7f4aadf77640 1 -- 172.21.15.5:0/1826092117 shutdown_connections 2024-05-20T18:29:46.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.779+0000 7f4aadf77640 1 --2- 172.21.15.5:0/1826092117 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4a9406bfd0 0x7f4a9406e4e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:46.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.779+0000 7f4aadf77640 1 --2- 172.21.15.5:0/1826092117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4aa80ca620 0x7f4aa814cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:46.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.779+0000 7f4aadf77640 1 --2- 172.21.15.5:0/1826092117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4aa80c9cd0 0x7f4aa80ca0e0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:46.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.779+0000 7f4aadf77640 1 --2- 172.21.15.5:0/1826092117 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4a90004800 0x7f4aa80b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:46.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.779+0000 7f4aadf77640 1 -- 172.21.15.5:0/1826092117 >> 172.21.15.5:0/1826092117 conn(0x7f4aa80d4bd0 msgr2=0x7f4aa8161500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:46.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.780+0000 7f4aadf77640 1 -- 172.21.15.5:0/1826092117 shutdown_connections 2024-05-20T18:29:46.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.780+0000 7f4aadf77640 1 -- 172.21.15.5:0/1826092117 wait complete. 2024-05-20T18:29:46.791 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:29:46.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.905+0000 7f2fbe8b1640 1 Processor -- start 2024-05-20T18:29:46.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.906+0000 7f2fbe8b1640 1 -- start start 2024-05-20T18:29:46.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.907+0000 7f2fbe8b1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fb8168050 0x7f2fb816a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:46.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.907+0000 7f2fbe8b1640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fb80d0630 con 0x7f2fb806d3d0 2024-05-20T18:29:46.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.907+0000 7f2fbe8b1640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fb80d39c0 con 0x7f2fb8162820 2024-05-20T18:29:46.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.907+0000 7f2fbe8b1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2fb80d3ca0 con 0x7f2fb8168050 2024-05-20T18:29:46.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.907+0000 7f2fb77fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2fb806d3d0 0x7f2fb806d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37316/0 (socket says 172.21.15.5:37316) 2024-05-20T18:29:46.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.907+0000 7f2fb77fe640 1 -- 172.21.15.5:0/2739545024 learned_addr learned my addr 172.21.15.5:0/2739545024 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:46.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.907+0000 7f2fbce4e640 1 --2- 172.21.15.5:0/2739545024 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fb8168050 0x7f2fb816a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:46.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.907+0000 7f2fb6ffd640 1 -- 172.21.15.5:0/2739545024 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2222478530 0 0) 0x7f2fb80d0630 con 0x7f2fb806d3d0 2024-05-20T18:29:46.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.907+0000 7f2fb6ffd640 1 -- 172.21.15.5:0/2739545024 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fa0003520 con 0x7f2fb806d3d0 2024-05-20T18:29:46.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.908+0000 7f2fb6ffd640 1 -- 172.21.15.5:0/2739545024 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3111884571 0 0) 0x7f2fb80d39c0 con 0x7f2fb8162820 2024-05-20T18:29:46.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.908+0000 7f2fb6ffd640 1 -- 172.21.15.5:0/2739545024 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fb80d0630 con 0x7f2fb8162820 2024-05-20T18:29:46.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.908+0000 7f2fb6ffd640 1 -- 172.21.15.5:0/2739545024 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2126325245 0 0) 0x7f2fa0003520 con 0x7f2fb806d3d0 2024-05-20T18:29:46.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.908+0000 7f2fbce4e640 1 -- 172.21.15.5:0/2739545024 >> v1:172.21.15.63:6789/0 conn(0x7f2fb8162820 legacy=0x7f2fb80b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:46.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.908+0000 7f2fbce4e640 1 -- 172.21.15.5:0/2739545024 >> v1:172.21.15.5:6789/0 conn(0x7f2fb806d3d0 legacy=0x7f2fb806d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:46.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.908+0000 7f2fbce4e640 1 -- 172.21.15.5:0/2739545024 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb80d3f80 con 0x7f2fb8168050 2024-05-20T18:29:46.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.908+0000 7f2fbce4e640 1 --2- 172.21.15.5:0/2739545024 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fb8168050 0x7f2fb816a470 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0009dc0 tx=0x7f2fb002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2738b165592aa1b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:46.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.909+0000 7f2fb6ffd640 1 -- 172.21.15.5:0/2739545024 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac001670 con 0x7f2fb8168050 2024-05-20T18:29:46.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.909+0000 7f2fb6ffd640 1 -- 172.21.15.5:0/2739545024 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fb0007540 con 0x7f2fb8168050 2024-05-20T18:29:46.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.909+0000 7f2fb6ffd640 1 -- 172.21.15.5:0/2739545024 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0007910 con 0x7f2fb8168050 2024-05-20T18:29:46.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.909+0000 7f2fbe8b1640 1 -- 172.21.15.5:0/2739545024 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fb8168050 msgr2=0x7f2fb816a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:46.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.909+0000 7f2fbe8b1640 1 --2- 172.21.15.5:0/2739545024 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fb8168050 0x7f2fb816a470 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0009dc0 tx=0x7f2fb002a660 comp rx=0 tx=0).stop 2024-05-20T18:29:46.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.909+0000 7f2fbe8b1640 1 -- 172.21.15.5:0/2739545024 shutdown_connections 2024-05-20T18:29:46.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.909+0000 7f2fbe8b1640 1 --2- 172.21.15.5:0/2739545024 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fb8168050 0x7f2fb816a470 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:46.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.909+0000 7f2fbe8b1640 1 -- 172.21.15.5:0/2739545024 >> 172.21.15.5:0/2739545024 conn(0x7f2fb80d4bd0 msgr2=0x7f2fb80d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:46.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.909+0000 7f2fbe8b1640 1 -- 172.21.15.5:0/2739545024 shutdown_connections 2024-05-20T18:29:46.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.909+0000 7f2fbe8b1640 1 -- 172.21.15.5:0/2739545024 wait complete. 2024-05-20T18:29:46.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.911+0000 7f2fbe8b1640 1 Processor -- start 2024-05-20T18:29:46.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.911+0000 7f2fbe8b1640 1 -- start start 2024-05-20T18:29:46.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.913+0000 7f2fbe8b1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fb806d3d0 0x7f2fb81419a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:46.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.913+0000 7f2fbe8b1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fb8162820 0x7f2fb8144c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:46.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.913+0000 7f2fbe8b1640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2fb8168050 0x7f2fb8145180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:46.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.913+0000 7f2fbe8b1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb80b91a0 con 0x7f2fb806d3d0 2024-05-20T18:29:46.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.913+0000 7f2fbe8b1640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb80d0300 con 0x7f2fb8168050 2024-05-20T18:29:46.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.913+0000 7f2fbe8b1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2fb80b4050 con 0x7f2fb8162820 2024-05-20T18:29:46.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.913+0000 7f2fb7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fb806d3d0 0x7f2fb81419a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:46.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.913+0000 7f2fb7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fb806d3d0 0x7f2fb81419a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46466/0 (socket says 172.21.15.5:46466) 2024-05-20T18:29:46.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.913+0000 7f2fb7fff640 1 -- 172.21.15.5:0/4138357489 learned_addr learned my addr 172.21.15.5:0/4138357489 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:46.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.913+0000 7f2fb77fe640 1 --2- 172.21.15.5:0/4138357489 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fb8162820 0x7f2fb8144c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:46.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.913+0000 7f2fbce4e640 1 --2- 172.21.15.5:0/4138357489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2fb8168050 0x7f2fb8145180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:46.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.913+0000 7f2fb7fff640 1 -- 172.21.15.5:0/4138357489 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fb8162820 msgr2=0x7f2fb8144c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:46.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.913+0000 7f2fb7fff640 1 --2- 172.21.15.5:0/4138357489 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fb8162820 0x7f2fb8144c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:46.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.913+0000 7f2fb7fff640 1 -- 172.21.15.5:0/4138357489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2fb8168050 msgr2=0x7f2fb8145180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:46.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.913+0000 7f2fb7fff640 1 --2- 172.21.15.5:0/4138357489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2fb8168050 0x7f2fb8145180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:46.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.913+0000 7f2fb7fff640 1 -- 172.21.15.5:0/4138357489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb8142490 con 0x7f2fb806d3d0 2024-05-20T18:29:46.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.913+0000 7f2fbce4e640 1 --2- 172.21.15.5:0/4138357489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2fb8168050 0x7f2fb8145180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:29:46.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.913+0000 7f2fb7fff640 1 --2- 172.21.15.5:0/4138357489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fb806d3d0 0x7f2fb81419a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800c460 tx=0x7f2fa800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:46.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.913+0000 7f2fb4ff9640 1 -- 172.21.15.5:0/4138357489 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa80052d0 con 0x7f2fb806d3d0 2024-05-20T18:29:46.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.914+0000 7f2fb4ff9640 1 -- 172.21.15.5:0/4138357489 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fa8005490 con 0x7f2fb806d3d0 2024-05-20T18:29:46.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.914+0000 7f2fb4ff9640 1 -- 172.21.15.5:0/4138357489 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8012870 con 0x7f2fb806d3d0 2024-05-20T18:29:46.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.914+0000 7f2fbe8b1640 1 -- 172.21.15.5:0/4138357489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb8155780 con 0x7f2fb806d3d0 2024-05-20T18:29:46.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.914+0000 7f2fbe8b1640 1 -- 172.21.15.5:0/4138357489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb8154060 con 0x7f2fb806d3d0 2024-05-20T18:29:46.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.915+0000 7f2fbe8b1640 1 -- 172.21.15.5:0/4138357489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fb8075b00 con 0x7f2fb806d3d0 2024-05-20T18:29:46.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.915+0000 7f2fb4ff9640 1 -- 172.21.15.5:0/4138357489 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2fa8012a10 con 0x7f2fb806d3d0 2024-05-20T18:29:46.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.916+0000 7f2fb4ff9640 1 --2- 172.21.15.5:0/4138357489 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2fa406be30 0x7f2fa406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:46.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.916+0000 7f2fb4ff9640 1 -- 172.21.15.5:0/4138357489 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f2fa808abd0 con 0x7f2fb806d3d0 2024-05-20T18:29:46.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.916+0000 7f2fb77fe640 1 --2- 172.21.15.5:0/4138357489 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2fa406be30 0x7f2fa406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:46.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.916+0000 7f2fb77fe640 1 --2- 172.21.15.5:0/4138357489 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2fa406be30 0x7f2fa406e340 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2fb808de20 tx=0x7f2fac001610 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:46.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:46.920+0000 7f2fb4ff9640 1 -- 172.21.15.5:0/4138357489 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2fa8059370 con 0x7f2fb806d3d0 2024-05-20T18:29:47.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.140+0000 7f2fbe8b1640 1 -- 172.21.15.5:0/4138357489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2fb80b9c50 con 0x7f2fb806d3d0 2024-05-20T18:29:47.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.140+0000 7f2fb4ff9640 1 -- 172.21.15.5:0/4138357489 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5175 (secure 0 0 0) 0x7f2fa805f020 con 0x7f2fb806d3d0 2024-05-20T18:29:47.140 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:29:47.140 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5207,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3958852803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3958852803},{"type":"v1","addr":"172.21.15.63:6835","nonce":3958852803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5560,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/3020169757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":3020169757},{"type":"v1","addr":"172.21.15.63:6834","nonce":3020169757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2965018210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2965018210},{"type":"v1","addr":"172.21.15.5:6835","nonce":2965018210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:29:02.038442+0000","modified":"2024-05-20T18:29:05.055248+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5751},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5751":{"gid":5751,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6837/3469824363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3469824363},{"type":"v1","addr":"172.21.15.5:6837","nonce":3469824363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5751,"qdb_cluster":[5751]},"id":3}]} 2024-05-20T18:29:47.141 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-05-20T18:29:47.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.142+0000 7f2fbe8b1640 1 -- 172.21.15.5:0/4138357489 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2fa406be30 msgr2=0x7f2fa406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:47.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.142+0000 7f2fbe8b1640 1 --2- 172.21.15.5:0/4138357489 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2fa406be30 0x7f2fa406e340 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2fb808de20 tx=0x7f2fac001610 comp rx=0 tx=0).stop 2024-05-20T18:29:47.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.142+0000 7f2fbe8b1640 1 -- 172.21.15.5:0/4138357489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fb806d3d0 msgr2=0x7f2fb81419a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:47.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.142+0000 7f2fbe8b1640 1 --2- 172.21.15.5:0/4138357489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fb806d3d0 0x7f2fb81419a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800c460 tx=0x7f2fa800c920 comp rx=0 tx=0).stop 2024-05-20T18:29:47.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.142+0000 7f2fbe8b1640 1 -- 172.21.15.5:0/4138357489 shutdown_connections 2024-05-20T18:29:47.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.142+0000 7f2fbe8b1640 1 --2- 172.21.15.5:0/4138357489 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2fa406be30 0x7f2fa406e340 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:47.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.142+0000 7f2fbe8b1640 1 --2- 172.21.15.5:0/4138357489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2fb8168050 0x7f2fb8145180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:47.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.142+0000 7f2fbe8b1640 1 --2- 172.21.15.5:0/4138357489 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fb8162820 0x7f2fb8144c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:47.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.142+0000 7f2fbe8b1640 1 --2- 172.21.15.5:0/4138357489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fb806d3d0 0x7f2fb81419a0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:47.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.142+0000 7f2fbe8b1640 1 -- 172.21.15.5:0/4138357489 >> 172.21.15.5:0/4138357489 conn(0x7f2fb80d4bd0 msgr2=0x7f2fb8160eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:47.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.142+0000 7f2fbe8b1640 1 -- 172.21.15.5:0/4138357489 shutdown_connections 2024-05-20T18:29:47.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.142+0000 7f2fbe8b1640 1 -- 172.21.15.5:0/4138357489 wait complete. 2024-05-20T18:29:47.154 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:29:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.269+0000 7f65d4c51640 1 Processor -- start 2024-05-20T18:29:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.269+0000 7f65d4c51640 1 -- start start 2024-05-20T18:29:47.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.270+0000 7f65d4c51640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f65d0162e00 0x7f65d006d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.270+0000 7f65d4c51640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65d006c8c0 con 0x7f65d00cfae0 2024-05-20T18:29:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.270+0000 7f65d4c51640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65d01633e0 con 0x7f65d00cd710 2024-05-20T18:29:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.270+0000 7f65d4c51640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f65d00d0ba0 con 0x7f65d0162e00 2024-05-20T18:29:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.270+0000 7f65c7fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f65d00cfae0 0x7f65d00d1030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37324/0 (socket says 172.21.15.5:37324) 2024-05-20T18:29:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.270+0000 7f65c7fff640 1 -- 172.21.15.5:0/456370623 learned_addr learned my addr 172.21.15.5:0/456370623 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.270+0000 7f65ced9d640 1 --2- 172.21.15.5:0/456370623 >> v2:172.21.15.63:3300/0 conn(0x7f65d0162e00 0x7f65d006d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.271+0000 7f65cdd9b640 1 -- 172.21.15.5:0/456370623 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1668551178 0 0) 0x7f65d006c8c0 con 0x7f65d00cfae0 2024-05-20T18:29:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.271+0000 7f65cdd9b640 1 -- 172.21.15.5:0/456370623 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65bc003520 con 0x7f65d00cfae0 2024-05-20T18:29:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.271+0000 7f65cdd9b640 1 -- 172.21.15.5:0/456370623 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3998174812 0 0) 0x7f65bc003520 con 0x7f65d00cfae0 2024-05-20T18:29:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.271+0000 7f65ced9d640 1 -- 172.21.15.5:0/456370623 >> v1:172.21.15.63:6789/0 conn(0x7f65d00cd710 legacy=0x7f65d00cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.271+0000 7f65ced9d640 1 -- 172.21.15.5:0/456370623 >> v1:172.21.15.5:6789/0 conn(0x7f65d00cfae0 legacy=0x7f65d00d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.271+0000 7f65ced9d640 1 -- 172.21.15.5:0/456370623 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d01636c0 con 0x7f65d0162e00 2024-05-20T18:29:47.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.272+0000 7f65ced9d640 1 --2- 172.21.15.5:0/456370623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65d0162e00 0x7f65d006d6b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009d90 tx=0x7f65c802a5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7711b677fb7fe911 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:47.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.272+0000 7f65cdd9b640 1 -- 172.21.15.5:0/456370623 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c8035020 con 0x7f65d0162e00 2024-05-20T18:29:47.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.272+0000 7f65cdd9b640 1 -- 172.21.15.5:0/456370623 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65c8007540 con 0x7f65d0162e00 2024-05-20T18:29:47.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.272+0000 7f65cdd9b640 1 -- 172.21.15.5:0/456370623 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8001690 con 0x7f65d0162e00 2024-05-20T18:29:47.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.273+0000 7f65d4c51640 1 -- 172.21.15.5:0/456370623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65d0162e00 msgr2=0x7f65d006d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:47.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.273+0000 7f65d4c51640 1 --2- 172.21.15.5:0/456370623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65d0162e00 0x7f65d006d6b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009d90 tx=0x7f65c802a5b0 comp rx=0 tx=0).stop 2024-05-20T18:29:47.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.273+0000 7f65d4c51640 1 -- 172.21.15.5:0/456370623 shutdown_connections 2024-05-20T18:29:47.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.273+0000 7f65d4c51640 1 --2- 172.21.15.5:0/456370623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65d0162e00 0x7f65d006d6b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:47.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.273+0000 7f65d4c51640 1 -- 172.21.15.5:0/456370623 >> 172.21.15.5:0/456370623 conn(0x7f65d00b44d0 msgr2=0x7f65d00b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:47.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.273+0000 7f65d4c51640 1 -- 172.21.15.5:0/456370623 shutdown_connections 2024-05-20T18:29:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.273+0000 7f65d4c51640 1 -- 172.21.15.5:0/456370623 wait complete. 2024-05-20T18:29:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.275+0000 7f65d4c51640 1 Processor -- start 2024-05-20T18:29:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.275+0000 7f65d4c51640 1 -- start start 2024-05-20T18:29:47.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.276+0000 7f65d4c51640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d00cd710 0x7f65d01f50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:47.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.276+0000 7f65d4c51640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65d0162e00 0x7f65d01601a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:47.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.276+0000 7f65d4c51640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65d00caee0 0x7f65d00cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:47.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.276+0000 7f65d4c51640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f65d015f2b0 con 0x7f65d00cd710 2024-05-20T18:29:47.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.276+0000 7f65d4c51640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f65d0163c80 con 0x7f65d00caee0 2024-05-20T18:29:47.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.276+0000 7f65d4c51640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f65d015f430 con 0x7f65d0162e00 2024-05-20T18:29:47.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.276+0000 7f65ce59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d00cd710 0x7f65d01f50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:47.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.276+0000 7f65ce59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d00cd710 0x7f65d01f50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46480/0 (socket says 172.21.15.5:46480) 2024-05-20T18:29:47.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.276+0000 7f65ce59c640 1 -- 172.21.15.5:0/973838456 learned_addr learned my addr 172.21.15.5:0/973838456 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:47.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.276+0000 7f65c7fff640 1 --2- 172.21.15.5:0/973838456 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65d0162e00 0x7f65d01601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:47.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.276+0000 7f65ced9d640 1 --2- 172.21.15.5:0/973838456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65d00caee0 0x7f65d00cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:47.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.276+0000 7f65ce59c640 1 -- 172.21.15.5:0/973838456 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65d0162e00 msgr2=0x7f65d01601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:47.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.276+0000 7f65ce59c640 1 --2- 172.21.15.5:0/973838456 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65d0162e00 0x7f65d01601a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:47.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.276+0000 7f65ce59c640 1 -- 172.21.15.5:0/973838456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65d00caee0 msgr2=0x7f65d00cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:47.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.276+0000 7f65ce59c640 1 --2- 172.21.15.5:0/973838456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65d00caee0 0x7f65d00cb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:47.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.276+0000 7f65ce59c640 1 -- 172.21.15.5:0/973838456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d00cb830 con 0x7f65d00cd710 2024-05-20T18:29:47.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.277+0000 7f65ce59c640 1 --2- 172.21.15.5:0/973838456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d00cd710 0x7f65d01f50e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f65c000c4d0 tx=0x7f65c000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:47.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.277+0000 7f65c6ffd640 1 -- 172.21.15.5:0/973838456 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0017070 con 0x7f65d00cd710 2024-05-20T18:29:47.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.277+0000 7f65c6ffd640 1 -- 172.21.15.5:0/973838456 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65c0005480 con 0x7f65d00cd710 2024-05-20T18:29:47.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.277+0000 7f65d4c51640 1 -- 172.21.15.5:0/973838456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65d00c6ca0 con 0x7f65d00cd710 2024-05-20T18:29:47.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.277+0000 7f65c6ffd640 1 -- 172.21.15.5:0/973838456 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0003830 con 0x7f65d00cd710 2024-05-20T18:29:47.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.277+0000 7f65d4c51640 1 -- 172.21.15.5:0/973838456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65d00c7260 con 0x7f65d00cd710 2024-05-20T18:29:47.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.278+0000 7f65d4c51640 1 -- 172.21.15.5:0/973838456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65d0075b00 con 0x7f65d00cd710 2024-05-20T18:29:47.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.278+0000 7f65c6ffd640 1 -- 172.21.15.5:0/973838456 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f65c001a070 con 0x7f65d00cd710 2024-05-20T18:29:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.279+0000 7f65c6ffd640 1 --2- 172.21.15.5:0/973838456 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f65b006be30 0x7f65b006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.279+0000 7f65c7fff640 1 --2- 172.21.15.5:0/973838456 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f65b006be30 0x7f65b006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.279+0000 7f65c6ffd640 1 -- 172.21.15.5:0/973838456 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f65c008a3d0 con 0x7f65d00cd710 2024-05-20T18:29:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.280+0000 7f65c7fff640 1 --2- 172.21.15.5:0/973838456 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f65b006be30 0x7f65b006e340 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f65b8001eb0 tx=0x7f65b8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:47.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.284+0000 7f65c6ffd640 1 -- 172.21.15.5:0/973838456 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f65d00cb830 con 0x7f65d00cd710 2024-05-20T18:29:47.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.468+0000 7f65d4c51640 1 -- 172.21.15.5:0/973838456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f65d01533f0 con 0x7f65d00cd710 2024-05-20T18:29:47.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.469+0000 7f65c6ffd640 1 -- 172.21.15.5:0/973838456 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+16822 (secure 0 0 0) 0x7f65c0058bf0 con 0x7f65d00cd710 2024-05-20T18:29:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:29:47.470 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":51,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:29:03.665286+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:29:00.401866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-20T18:29:01.006122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:29:47.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.471+0000 7f65d4c51640 1 -- 172.21.15.5:0/973838456 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f65b006be30 msgr2=0x7f65b006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:47.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.471+0000 7f65d4c51640 1 --2- 172.21.15.5:0/973838456 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f65b006be30 0x7f65b006e340 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f65b8001eb0 tx=0x7f65b8009040 comp rx=0 tx=0).stop 2024-05-20T18:29:47.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.471+0000 7f65d4c51640 1 -- 172.21.15.5:0/973838456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d00cd710 msgr2=0x7f65d01f50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:47.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.471+0000 7f65d4c51640 1 --2- 172.21.15.5:0/973838456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d00cd710 0x7f65d01f50e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f65c000c4d0 tx=0x7f65c000c990 comp rx=0 tx=0).stop 2024-05-20T18:29:47.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.471+0000 7f65d4c51640 1 -- 172.21.15.5:0/973838456 shutdown_connections 2024-05-20T18:29:47.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.471+0000 7f65d4c51640 1 --2- 172.21.15.5:0/973838456 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f65b006be30 0x7f65b006e340 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:47.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.471+0000 7f65d4c51640 1 --2- 172.21.15.5:0/973838456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65d00caee0 0x7f65d00cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:47.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.471+0000 7f65d4c51640 1 --2- 172.21.15.5:0/973838456 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65d0162e00 0x7f65d01601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:47.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.471+0000 7f65d4c51640 1 --2- 172.21.15.5:0/973838456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d00cd710 0x7f65d01f50e0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:47.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.471+0000 7f65d4c51640 1 -- 172.21.15.5:0/973838456 >> 172.21.15.5:0/973838456 conn(0x7f65d00b44d0 msgr2=0x7f65d00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:47.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.472+0000 7f65d4c51640 1 -- 172.21.15.5:0/973838456 shutdown_connections 2024-05-20T18:29:47.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.472+0000 7f65d4c51640 1 -- 172.21.15.5:0/973838456 wait complete. 2024-05-20T18:29:47.483 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-20T18:29:47.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.599+0000 7ff5e6cca640 1 Processor -- start 2024-05-20T18:29:47.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.599+0000 7ff5e6cca640 1 -- start start 2024-05-20T18:29:47.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.600+0000 7ff5e6cca640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff5e0162820 0x7ff5e00b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:47.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.600+0000 7ff5e6cca640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff5e00b3e80 0x7ff5e006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:47.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.600+0000 7ff5e6cca640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5e006dd20 con 0x7ff5e006d910 2024-05-20T18:29:47.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.600+0000 7ff5e6cca640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7ff5e00d0630 con 0x7ff5e00b3e80 2024-05-20T18:29:47.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.600+0000 7ff5e6cca640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff5e00d39c0 con 0x7ff5e0162820 2024-05-20T18:29:47.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.601+0000 7ff5dffff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff5e00b3e80 0x7ff5e006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:47.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.601+0000 7ff5e5267640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff5e006d910 0x7ff5e00d27a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37336/0 (socket says 172.21.15.5:37336) 2024-05-20T18:29:47.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.601+0000 7ff5e5267640 1 -- 172.21.15.5:0/4234354338 learned_addr learned my addr 172.21.15.5:0/4234354338 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:47.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.601+0000 7ff5e4a66640 1 --2- 172.21.15.5:0/4234354338 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff5e0162820 0x7ff5e00b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:47.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.601+0000 7ff5dffff640 1 -- 172.21.15.5:0/4234354338 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff5e0162820 msgr2=0x7ff5e00b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:47.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.601+0000 7ff5dffff640 1 --2- 172.21.15.5:0/4234354338 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff5e0162820 0x7ff5e00b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:47.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.601+0000 7ff5dffff640 1 -- 172.21.15.5:0/4234354338 >> v1:172.21.15.5:6789/0 conn(0x7ff5e006d910 legacy=0x7ff5e00d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:47.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.601+0000 7ff5dffff640 1 -- 172.21.15.5:0/4234354338 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e00d3ca0 con 0x7ff5e00b3e80 2024-05-20T18:29:47.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.601+0000 7ff5df7fe640 1 -- 172.21.15.5:0/4234354338 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 522157476 0 0) 0x7ff5d8001490 con 0x7ff5e006d910 2024-05-20T18:29:47.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.602+0000 7ff5dffff640 1 --2- 172.21.15.5:0/4234354338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5e00b3e80 0x7ff5e006d3d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc0097a0 tx=0x7ff5cc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=378022c89cd763af server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:47.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.602+0000 7ff5df7fe640 1 -- 172.21.15.5:0/4234354338 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc037070 con 0x7ff5e00b3e80 2024-05-20T18:29:47.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.602+0000 7ff5df7fe640 1 -- 172.21.15.5:0/4234354338 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5cc032440 con 0x7ff5e00b3e80 2024-05-20T18:29:47.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.602+0000 7ff5df7fe640 1 -- 172.21.15.5:0/4234354338 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc032810 con 0x7ff5e00b3e80 2024-05-20T18:29:47.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.602+0000 7ff5e6cca640 1 -- 172.21.15.5:0/4234354338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5e00b3e80 msgr2=0x7ff5e006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:47.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.602+0000 7ff5e6cca640 1 --2- 172.21.15.5:0/4234354338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5e00b3e80 0x7ff5e006d3d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc0097a0 tx=0x7ff5cc02a180 comp rx=0 tx=0).stop 2024-05-20T18:29:47.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.603+0000 7ff5e6cca640 1 -- 172.21.15.5:0/4234354338 shutdown_connections 2024-05-20T18:29:47.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.603+0000 7ff5e6cca640 1 --2- 172.21.15.5:0/4234354338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5e00b3e80 0x7ff5e006d3d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:47.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.603+0000 7ff5e6cca640 1 --2- 172.21.15.5:0/4234354338 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff5e0162820 0x7ff5e00b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:47.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.603+0000 7ff5e6cca640 1 -- 172.21.15.5:0/4234354338 >> 172.21.15.5:0/4234354338 conn(0x7ff5e00d4bd0 msgr2=0x7ff5e00d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:47.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.603+0000 7ff5e6cca640 1 -- 172.21.15.5:0/4234354338 shutdown_connections 2024-05-20T18:29:47.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.603+0000 7ff5e6cca640 1 -- 172.21.15.5:0/4234354338 wait complete. 2024-05-20T18:29:47.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.604+0000 7ff5e6cca640 1 Processor -- start 2024-05-20T18:29:47.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.604+0000 7ff5e6cca640 1 -- start start 2024-05-20T18:29:47.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.606+0000 7ff5e6cca640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5e006d910 0x7ff5e00cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:47.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.606+0000 7ff5e6cca640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff5e00b3e80 0x7ff5e00cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:47.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.606+0000 7ff5e6cca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5e0162820 0x7ff5e0150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:47.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.606+0000 7ff5e6cca640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e00d0300 con 0x7ff5e0162820 2024-05-20T18:29:47.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.606+0000 7ff5e6cca640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e00d4260 con 0x7ff5e006d910 2024-05-20T18:29:47.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.606+0000 7ff5e6cca640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff5e00b91a0 con 0x7ff5e00b3e80 2024-05-20T18:29:47.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.606+0000 7ff5e5267640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5e0162820 0x7ff5e0150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:47.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.606+0000 7ff5e5267640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5e0162820 0x7ff5e0150ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46500/0 (socket says 172.21.15.5:46500) 2024-05-20T18:29:47.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.606+0000 7ff5e5267640 1 -- 172.21.15.5:0/764974124 learned_addr learned my addr 172.21.15.5:0/764974124 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:47.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.606+0000 7ff5e4a66640 1 --2- 172.21.15.5:0/764974124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5e006d910 0x7ff5e00cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:47.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.606+0000 7ff5dffff640 1 --2- 172.21.15.5:0/764974124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff5e00b3e80 0x7ff5e00cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.606+0000 7ff5e5267640 1 -- 172.21.15.5:0/764974124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff5e00b3e80 msgr2=0x7ff5e00cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.606+0000 7ff5e5267640 1 --2- 172.21.15.5:0/764974124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff5e00b3e80 0x7ff5e00cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.606+0000 7ff5e5267640 1 -- 172.21.15.5:0/764974124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5e006d910 msgr2=0x7ff5e00cbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.606+0000 7ff5e5267640 1 --2- 172.21.15.5:0/764974124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5e006d910 0x7ff5e00cbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.606+0000 7ff5e5267640 1 -- 172.21.15.5:0/764974124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e00ccc90 con 0x7ff5e0162820 2024-05-20T18:29:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.606+0000 7ff5e5267640 1 --2- 172.21.15.5:0/764974124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5e0162820 0x7ff5e0150ec0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800c780 tx=0x7ff5d800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.607+0000 7ff5dd7fa640 1 -- 172.21.15.5:0/764974124 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8018070 con 0x7ff5e0162820 2024-05-20T18:29:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.607+0000 7ff5dd7fa640 1 -- 172.21.15.5:0/764974124 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5d800a640 con 0x7ff5e0162820 2024-05-20T18:29:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.607+0000 7ff5e6cca640 1 -- 172.21.15.5:0/764974124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e00ccf70 con 0x7ff5e0162820 2024-05-20T18:29:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.607+0000 7ff5dd7fa640 1 -- 172.21.15.5:0/764974124 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d800aa30 con 0x7ff5e0162820 2024-05-20T18:29:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.607+0000 7ff5e6cca640 1 -- 172.21.15.5:0/764974124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e00cd530 con 0x7ff5e0162820 2024-05-20T18:29:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.608+0000 7ff5e6cca640 1 -- 172.21.15.5:0/764974124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5e0075b00 con 0x7ff5e0162820 2024-05-20T18:29:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.608+0000 7ff5dd7fa640 1 -- 172.21.15.5:0/764974124 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff5d801b070 con 0x7ff5e0162820 2024-05-20T18:29:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.609+0000 7ff5dd7fa640 1 --2- 172.21.15.5:0/764974124 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff5c406be30 0x7ff5c406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.609+0000 7ff5dd7fa640 1 -- 172.21.15.5:0/764974124 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7ff5d808a680 con 0x7ff5e0162820 2024-05-20T18:29:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.609+0000 7ff5e4a66640 1 --2- 172.21.15.5:0/764974124 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff5c406be30 0x7ff5c406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.609+0000 7ff5e4a66640 1 --2- 172.21.15.5:0/764974124 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff5c406be30 0x7ff5c406e340 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4006fd0 tx=0x7ff5d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.614+0000 7ff5dd7fa640 1 -- 172.21.15.5:0/764974124 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff5e006dd20 con 0x7ff5e0162820 2024-05-20T18:29:47.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.839+0000 7ff5e6cca640 1 -- 172.21.15.5:0/764974124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff5e00d5230 con 0x7ff5e0162820 2024-05-20T18:29:47.839 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-20T18:29:47.839+0000 7fd1cf53a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-20T18:29:47.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.841+0000 7ff5dd7fa640 1 -- 172.21.15.5:0/764974124 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7ff5d8058e10 con 0x7ff5e0162820 2024-05-20T18:29:47.841 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-20T18:29:47.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.843+0000 7ff5e6cca640 1 -- 172.21.15.5:0/764974124 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff5c406be30 msgr2=0x7ff5c406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:47.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.844+0000 7ff5e6cca640 1 --2- 172.21.15.5:0/764974124 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff5c406be30 0x7ff5c406e340 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4006fd0 tx=0x7ff5d4008040 comp rx=0 tx=0).stop 2024-05-20T18:29:47.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.844+0000 7ff5e6cca640 1 -- 172.21.15.5:0/764974124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5e0162820 msgr2=0x7ff5e0150ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:47.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.844+0000 7ff5e6cca640 1 --2- 172.21.15.5:0/764974124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5e0162820 0x7ff5e0150ec0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800c780 tx=0x7ff5d800cc40 comp rx=0 tx=0).stop 2024-05-20T18:29:47.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.844+0000 7ff5e6cca640 1 -- 172.21.15.5:0/764974124 shutdown_connections 2024-05-20T18:29:47.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.844+0000 7ff5e6cca640 1 --2- 172.21.15.5:0/764974124 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff5c406be30 0x7ff5c406e340 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:47.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.844+0000 7ff5e6cca640 1 --2- 172.21.15.5:0/764974124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5e0162820 0x7ff5e0150ec0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:47.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.844+0000 7ff5e6cca640 1 --2- 172.21.15.5:0/764974124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff5e00b3e80 0x7ff5e00cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:47.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.844+0000 7ff5e6cca640 1 --2- 172.21.15.5:0/764974124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5e006d910 0x7ff5e00cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:47.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.844+0000 7ff5e6cca640 1 -- 172.21.15.5:0/764974124 >> 172.21.15.5:0/764974124 conn(0x7ff5e00d4bd0 msgr2=0x7ff5e015d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:47.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.844+0000 7ff5e6cca640 1 -- 172.21.15.5:0/764974124 shutdown_connections 2024-05-20T18:29:47.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.844+0000 7ff5e6cca640 1 -- 172.21.15.5:0/764974124 wait complete. 2024-05-20T18:29:47.847 INFO:tasks.ceph.mds.c.smithi005.stderr: -6091> 2024-05-20T18:29:04.632+0000 7f56f54ba640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T18:29:47.850 INFO:tasks.ceph.mds.c.smithi005.stderr: -3149> 2024-05-20T18:29:10.836+0000 7f56f14b2640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5710=0-4194304@1} caps={5710=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5710 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55e449fd4c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T18:29:47.856 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-20T18:29:47.906 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-05-20T18:29:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.975+0000 7fe26245a640 1 Processor -- start 2024-05-20T18:29:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.975+0000 7fe26245a640 1 -- start start 2024-05-20T18:29:47.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.977+0000 7fe26245a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe25c0b0020 0x7fe25c15fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:47.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.977+0000 7fe26245a640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe25c0b51c0 con 0x7fe25c15ca00 2024-05-20T18:29:47.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.977+0000 7fe26245a640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe25c15bf20 con 0x7fe25c06c4c0 2024-05-20T18:29:47.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.977+0000 7fe26245a640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe25c15c200 con 0x7fe25c0b0020 2024-05-20T18:29:47.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.977+0000 7fe25bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe25c0b0020 0x7fe25c15fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:47.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.977+0000 7fe25b7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe25c15ca00 0x7fe25c06bdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37350/0 (socket says 172.21.15.5:37350) 2024-05-20T18:29:47.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.977+0000 7fe25bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe25c0b0020 0x7fe25c15fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46522/0 (socket says 172.21.15.5:46522) 2024-05-20T18:29:47.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.977+0000 7fe25b7fe640 1 -- 172.21.15.5:0/2241521236 learned_addr learned my addr 172.21.15.5:0/2241521236 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:47.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.977+0000 7fe25bfff640 1 -- 172.21.15.5:0/2241521236 >> v1:172.21.15.63:6789/0 conn(0x7fe25c06c4c0 legacy=0x7fe25c0b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:47.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.977+0000 7fe25bfff640 1 -- 172.21.15.5:0/2241521236 >> v1:172.21.15.5:6789/0 conn(0x7fe25c15ca00 legacy=0x7fe25c06bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:47.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.977+0000 7fe25bfff640 1 -- 172.21.15.5:0/2241521236 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe25c15c4e0 con 0x7fe25c0b0020 2024-05-20T18:29:47.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.977+0000 7fe25affd640 1 -- 172.21.15.5:0/2241521236 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 861386510 0 0) 0x7fe25c0b51c0 con 0x7fe25c15ca00 2024-05-20T18:29:47.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.978+0000 7fe25bfff640 1 --2- 172.21.15.5:0/2241521236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe25c0b0020 0x7fe25c15fd80 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe24c0097a0 tx=0x7fe24c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581a24385818c2cd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:47.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.978+0000 7fe25affd640 1 -- 172.21.15.5:0/2241521236 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c037070 con 0x7fe25c0b0020 2024-05-20T18:29:47.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.978+0000 7fe25affd640 1 -- 172.21.15.5:0/2241521236 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe24c032440 con 0x7fe25c0b0020 2024-05-20T18:29:47.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.978+0000 7fe25affd640 1 -- 172.21.15.5:0/2241521236 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c032810 con 0x7fe25c0b0020 2024-05-20T18:29:47.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.979+0000 7fe26245a640 1 -- 172.21.15.5:0/2241521236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe25c0b0020 msgr2=0x7fe25c15fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:47.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.979+0000 7fe26245a640 1 --2- 172.21.15.5:0/2241521236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe25c0b0020 0x7fe25c15fd80 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe24c0097a0 tx=0x7fe24c02a1a0 comp rx=0 tx=0).stop 2024-05-20T18:29:47.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.979+0000 7fe26245a640 1 -- 172.21.15.5:0/2241521236 shutdown_connections 2024-05-20T18:29:47.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.979+0000 7fe26245a640 1 --2- 172.21.15.5:0/2241521236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe25c0b0020 0x7fe25c15fd80 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:47.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.979+0000 7fe26245a640 1 -- 172.21.15.5:0/2241521236 >> 172.21.15.5:0/2241521236 conn(0x7fe25c0cd2a0 msgr2=0x7fe25c0cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:47.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.979+0000 7fe26245a640 1 -- 172.21.15.5:0/2241521236 shutdown_connections 2024-05-20T18:29:47.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.979+0000 7fe26245a640 1 -- 172.21.15.5:0/2241521236 wait complete. 2024-05-20T18:29:47.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.981+0000 7fe26245a640 1 Processor -- start 2024-05-20T18:29:47.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.981+0000 7fe26245a640 1 -- start start 2024-05-20T18:29:47.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.982+0000 7fe26245a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe25c0b0020 0x7fe25c1450d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:47.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.982+0000 7fe26245a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe25c15ca00 0x7fe25c158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:47.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.982+0000 7fe26245a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe25c1591e0 0x7fe25c147160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:47.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.982+0000 7fe26245a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c0b4c00 con 0x7fe25c1591e0 2024-05-20T18:29:47.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.982+0000 7fe26245a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c0b54a0 con 0x7fe25c0b0020 2024-05-20T18:29:47.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.982+0000 7fe26245a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe25c0b4d80 con 0x7fe25c15ca00 2024-05-20T18:29:47.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.982+0000 7fe2609f7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe25c1591e0 0x7fe25c147160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:47.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.983+0000 7fe2609f7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe25c1591e0 0x7fe25c147160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46526/0 (socket says 172.21.15.5:46526) 2024-05-20T18:29:47.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.983+0000 7fe2609f7640 1 -- 172.21.15.5:0/860516630 learned_addr learned my addr 172.21.15.5:0/860516630 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:47.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.983+0000 7fe25bfff640 1 --2- 172.21.15.5:0/860516630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe25c0b0020 0x7fe25c1450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:47.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.983+0000 7fe25b7fe640 1 --2- 172.21.15.5:0/860516630 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe25c15ca00 0x7fe25c158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:47.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.983+0000 7fe2609f7640 1 -- 172.21.15.5:0/860516630 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe25c15ca00 msgr2=0x7fe25c158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:47.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.983+0000 7fe2609f7640 1 --2- 172.21.15.5:0/860516630 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe25c15ca00 0x7fe25c158ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:47.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.983+0000 7fe2609f7640 1 -- 172.21.15.5:0/860516630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe25c0b0020 msgr2=0x7fe25c1450d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:47.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.983+0000 7fe2609f7640 1 --2- 172.21.15.5:0/860516630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe25c0b0020 0x7fe25c1450d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:47.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.983+0000 7fe2609f7640 1 -- 172.21.15.5:0/860516630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe25c147e50 con 0x7fe25c1591e0 2024-05-20T18:29:47.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.983+0000 7fe25bfff640 1 --2- 172.21.15.5:0/860516630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe25c0b0020 0x7fe25c1450d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:29:47.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.983+0000 7fe2609f7640 1 --2- 172.21.15.5:0/860516630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe25c1591e0 0x7fe25c147160 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe25400c680 tx=0x7fe25400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:47.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.983+0000 7fe258ff9640 1 -- 172.21.15.5:0/860516630 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe254003b70 con 0x7fe25c1591e0 2024-05-20T18:29:47.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.983+0000 7fe258ff9640 1 -- 172.21.15.5:0/860516630 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe254003d10 con 0x7fe25c1591e0 2024-05-20T18:29:47.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.983+0000 7fe26245a640 1 -- 172.21.15.5:0/860516630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe25c1581c0 con 0x7fe25c1591e0 2024-05-20T18:29:47.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.983+0000 7fe258ff9640 1 -- 172.21.15.5:0/860516630 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe254005310 con 0x7fe25c1591e0 2024-05-20T18:29:47.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.983+0000 7fe26245a640 1 -- 172.21.15.5:0/860516630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe25c158780 con 0x7fe25c1591e0 2024-05-20T18:29:47.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.985+0000 7fe26245a640 1 -- 172.21.15.5:0/860516630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe25c075b00 con 0x7fe25c1591e0 2024-05-20T18:29:47.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.985+0000 7fe258ff9640 1 -- 172.21.15.5:0/860516630 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe2540054b0 con 0x7fe25c1591e0 2024-05-20T18:29:47.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.986+0000 7fe258ff9640 1 --2- 172.21.15.5:0/860516630 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe24806be30 0x7fe24806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:47.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.986+0000 7fe25bfff640 1 --2- 172.21.15.5:0/860516630 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe24806be30 0x7fe24806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:47.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.986+0000 7fe258ff9640 1 -- 172.21.15.5:0/860516630 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6187+0+0 (secure 0 0 0) 0x7fe25408adf0 con 0x7fe25c1591e0 2024-05-20T18:29:47.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.986+0000 7fe25bfff640 1 --2- 172.21.15.5:0/860516630 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe24806be30 0x7fe24806e340 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe24c034000 tx=0x7fe24c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:47.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:47.990+0000 7fe258ff9640 1 -- 172.21.15.5:0/860516630 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe254059650 con 0x7fe25c1591e0 2024-05-20T18:29:48.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.218+0000 7fe26245a640 1 -- 172.21.15.5:0/860516630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe25c068920 con 0x7fe25c1591e0 2024-05-20T18:29:48.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.846+0000 7fe258ff9640 1 -- 172.21.15.5:0/860516630 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7fe254058f50 con 0x7fe25c1591e0 2024-05-20T18:29:48.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.848+0000 7fe26245a640 1 -- 172.21.15.5:0/860516630 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe24806be30 msgr2=0x7fe24806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:48.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.848+0000 7fe26245a640 1 --2- 172.21.15.5:0/860516630 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe24806be30 0x7fe24806e340 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe24c034000 tx=0x7fe24c02e000 comp rx=0 tx=0).stop 2024-05-20T18:29:48.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.848+0000 7fe26245a640 1 -- 172.21.15.5:0/860516630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe25c1591e0 msgr2=0x7fe25c147160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:48.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.848+0000 7fe26245a640 1 --2- 172.21.15.5:0/860516630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe25c1591e0 0x7fe25c147160 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe25400c680 tx=0x7fe25400cb40 comp rx=0 tx=0).stop 2024-05-20T18:29:48.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.848+0000 7fe26245a640 1 -- 172.21.15.5:0/860516630 shutdown_connections 2024-05-20T18:29:48.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.848+0000 7fe26245a640 1 --2- 172.21.15.5:0/860516630 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe24806be30 0x7fe24806e340 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:48.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.848+0000 7fe26245a640 1 --2- 172.21.15.5:0/860516630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe25c1591e0 0x7fe25c147160 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:48.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.848+0000 7fe26245a640 1 --2- 172.21.15.5:0/860516630 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe25c15ca00 0x7fe25c158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:48.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.848+0000 7fe26245a640 1 --2- 172.21.15.5:0/860516630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe25c0b0020 0x7fe25c1450d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:48.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.848+0000 7fe26245a640 1 -- 172.21.15.5:0/860516630 >> 172.21.15.5:0/860516630 conn(0x7fe25c0cd2a0 msgr2=0x7fe25c06ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:48.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.849+0000 7fe26245a640 1 -- 172.21.15.5:0/860516630 shutdown_connections 2024-05-20T18:29:48.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.849+0000 7fe26245a640 1 -- 172.21.15.5:0/860516630 wait complete. 2024-05-20T18:29:48.863 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-20T18:29:48.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.988+0000 7ff667365640 1 Processor -- start 2024-05-20T18:29:48.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.989+0000 7ff667365640 1 -- start start 2024-05-20T18:29:48.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.990+0000 7ff667365640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff66006d830 0x7ff6600cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:48.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.990+0000 7ff667365640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6600cdf70 0x7ff6600ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:48.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.990+0000 7ff667365640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6600d2740 con 0x7ff6600d2330 2024-05-20T18:29:48.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.990+0000 7ff667365640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff66009c3b0 con 0x7ff66006d830 2024-05-20T18:29:48.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.990+0000 7ff667365640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff66006dda0 con 0x7ff6600cdf70 2024-05-20T18:29:48.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.990+0000 7ff664900640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff66006d830 0x7ff6600cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:48.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.990+0000 7ff665902640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6600cdf70 0x7ff6600ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:48.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.990+0000 7ff665101640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7ff6600d2330 0x7ff66006d120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:58562/0 (socket says 172.21.15.5:58562) 2024-05-20T18:29:48.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.990+0000 7ff665101640 1 -- 172.21.15.5:0/15666638 learned_addr learned my addr 172.21.15.5:0/15666638 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:48.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.991+0000 7ff657fff640 1 -- 172.21.15.5:0/15666638 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1133895449 0 0) 0x7ff6600d2740 con 0x7ff6600d2330 2024-05-20T18:29:48.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.991+0000 7ff657fff640 1 -- 172.21.15.5:0/15666638 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff64c003520 con 0x7ff6600d2330 2024-05-20T18:29:48.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.991+0000 7ff665902640 1 -- 172.21.15.5:0/15666638 >> v2:172.21.15.63:3300/0 conn(0x7ff66006d830 msgr2=0x7ff6600cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:48.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.991+0000 7ff665902640 1 --2- 172.21.15.5:0/15666638 >> v2:172.21.15.63:3300/0 conn(0x7ff66006d830 0x7ff6600cda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:48.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.991+0000 7ff665902640 1 -- 172.21.15.5:0/15666638 >> v1:172.21.15.63:6789/0 conn(0x7ff6600d2330 legacy=0x7ff66006d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:48.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.991+0000 7ff665902640 1 -- 172.21.15.5:0/15666638 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6600b81a0 con 0x7ff6600cdf70 2024-05-20T18:29:48.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.991+0000 7ff664900640 1 --2- 172.21.15.5:0/15666638 >> v2:172.21.15.63:3300/0 conn(0x7ff66006d830 0x7ff6600cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:29:48.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.992+0000 7ff665902640 1 --2- 172.21.15.5:0/15666638 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6600cdf70 0x7ff6600ce380 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff658009d90 tx=0x7ff65802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa67fb4817a88f73 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:48.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.992+0000 7ff657fff640 1 -- 172.21.15.5:0/15666638 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff658007740 con 0x7ff6600cdf70 2024-05-20T18:29:48.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.992+0000 7ff657fff640 1 -- 172.21.15.5:0/15666638 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6580078e0 con 0x7ff6600cdf70 2024-05-20T18:29:48.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.992+0000 7ff657fff640 1 -- 172.21.15.5:0/15666638 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff658007c90 con 0x7ff6600cdf70 2024-05-20T18:29:48.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.993+0000 7ff667365640 1 -- 172.21.15.5:0/15666638 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6600cdf70 msgr2=0x7ff6600ce380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:48.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.993+0000 7ff667365640 1 --2- 172.21.15.5:0/15666638 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6600cdf70 0x7ff6600ce380 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff658009d90 tx=0x7ff65802a660 comp rx=0 tx=0).stop 2024-05-20T18:29:48.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.993+0000 7ff667365640 1 -- 172.21.15.5:0/15666638 shutdown_connections 2024-05-20T18:29:48.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.993+0000 7ff667365640 1 --2- 172.21.15.5:0/15666638 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6600cdf70 0x7ff6600ce380 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:48.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.993+0000 7ff667365640 1 --2- 172.21.15.5:0/15666638 >> v2:172.21.15.63:3300/0 conn(0x7ff66006d830 0x7ff6600cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:48.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.993+0000 7ff667365640 1 -- 172.21.15.5:0/15666638 >> 172.21.15.5:0/15666638 conn(0x7ff6600cf450 msgr2=0x7ff6600cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:48.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.993+0000 7ff667365640 1 -- 172.21.15.5:0/15666638 shutdown_connections 2024-05-20T18:29:48.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.993+0000 7ff667365640 1 -- 172.21.15.5:0/15666638 wait complete. 2024-05-20T18:29:48.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.995+0000 7ff667365640 1 Processor -- start 2024-05-20T18:29:48.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.995+0000 7ff667365640 1 -- start start 2024-05-20T18:29:48.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.996+0000 7ff667365640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff66006d830 0x7ff66015cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:48.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.996+0000 7ff667365640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6600cdf70 0x7ff6601f5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:48.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.996+0000 7ff667365640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6600d2330 0x7ff6601e8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:48.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.996+0000 7ff667365640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff6600d3980 con 0x7ff66006d830 2024-05-20T18:29:48.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.996+0000 7ff667365640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff6600d3800 con 0x7ff6600d2330 2024-05-20T18:29:48.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.996+0000 7ff667365640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff6600b4180 con 0x7ff6600cdf70 2024-05-20T18:29:48.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.996+0000 7ff665101640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff66006d830 0x7ff66015cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:48.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.996+0000 7ff665101640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff66006d830 0x7ff66015cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46532/0 (socket says 172.21.15.5:46532) 2024-05-20T18:29:48.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.996+0000 7ff665101640 1 -- 172.21.15.5:0/592010433 learned_addr learned my addr 172.21.15.5:0/592010433 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:48.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.996+0000 7ff664900640 1 --2- 172.21.15.5:0/592010433 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6600cdf70 0x7ff6601f5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:48.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.996+0000 7ff665902640 1 --2- 172.21.15.5:0/592010433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6600d2330 0x7ff6601e8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:48.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.997+0000 7ff665101640 1 -- 172.21.15.5:0/592010433 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6600cdf70 msgr2=0x7ff6601f5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:48.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.997+0000 7ff665101640 1 --2- 172.21.15.5:0/592010433 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6600cdf70 0x7ff6601f5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:48.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.997+0000 7ff665101640 1 -- 172.21.15.5:0/592010433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6600d2330 msgr2=0x7ff6601e8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:48.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.997+0000 7ff665101640 1 --2- 172.21.15.5:0/592010433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6600d2330 0x7ff6601e8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:48.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.997+0000 7ff665101640 1 -- 172.21.15.5:0/592010433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6600b5dd0 con 0x7ff66006d830 2024-05-20T18:29:48.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.997+0000 7ff665101640 1 --2- 172.21.15.5:0/592010433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff66006d830 0x7ff66015cac0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff65c00c430 tx=0x7ff65c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:48.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.997+0000 7ff655ffb640 1 -- 172.21.15.5:0/592010433 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c017070 con 0x7ff66006d830 2024-05-20T18:29:48.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.997+0000 7ff667365640 1 -- 172.21.15.5:0/592010433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6600b6090 con 0x7ff66006d830 2024-05-20T18:29:48.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.997+0000 7ff655ffb640 1 -- 172.21.15.5:0/592010433 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff65c005d50 con 0x7ff66006d830 2024-05-20T18:29:48.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.997+0000 7ff655ffb640 1 -- 172.21.15.5:0/592010433 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c007310 con 0x7ff66006d830 2024-05-20T18:29:48.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.997+0000 7ff667365640 1 -- 172.21.15.5:0/592010433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6600b6650 con 0x7ff66006d830 2024-05-20T18:29:48.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.998+0000 7ff667365640 1 -- 172.21.15.5:0/592010433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6600690b0 con 0x7ff66006d830 2024-05-20T18:29:48.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.999+0000 7ff655ffb640 1 -- 172.21.15.5:0/592010433 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff65c01a070 con 0x7ff66006d830 2024-05-20T18:29:48.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.999+0000 7ff655ffb640 1 --2- 172.21.15.5:0/592010433 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff64406be30 0x7ff64406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:48.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:48.999+0000 7ff655ffb640 1 -- 172.21.15.5:0/592010433 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6187+0+0 (secure 0 0 0) 0x7ff65c08a4e0 con 0x7ff66006d830 2024-05-20T18:29:48.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:49.000+0000 7ff664900640 1 --2- 172.21.15.5:0/592010433 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff64406be30 0x7ff64406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:49.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:49.000+0000 7ff664900640 1 --2- 172.21.15.5:0/592010433 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff64406be30 0x7ff64406e340 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff6601e8710 tx=0x7ff650009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:49.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:49.004+0000 7ff655ffb640 1 -- 172.21.15.5:0/592010433 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff65c058540 con 0x7ff66006d830 2024-05-20T18:29:49.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:49.189+0000 7ff667365640 1 -- 172.21.15.5:0/592010433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff6600b37c0 con 0x7ff66006d830 2024-05-20T18:29:49.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:49.847+0000 7ff655ffb640 1 -- 172.21.15.5:0/592010433 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v53) ==== 181+0+0 (secure 0 0 0) 0x7ff65c05c840 con 0x7ff66006d830 2024-05-20T18:29:49.847 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-05-20T18:29:49.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:49.851+0000 7ff667365640 1 -- 172.21.15.5:0/592010433 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff64406be30 msgr2=0x7ff64406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:49.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:49.851+0000 7ff667365640 1 --2- 172.21.15.5:0/592010433 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff64406be30 0x7ff64406e340 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff6601e8710 tx=0x7ff650009190 comp rx=0 tx=0).stop 2024-05-20T18:29:49.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:49.852+0000 7ff667365640 1 -- 172.21.15.5:0/592010433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff66006d830 msgr2=0x7ff66015cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:49.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:49.852+0000 7ff667365640 1 --2- 172.21.15.5:0/592010433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff66006d830 0x7ff66015cac0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff65c00c430 tx=0x7ff65c00c8f0 comp rx=0 tx=0).stop 2024-05-20T18:29:49.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:49.852+0000 7ff667365640 1 -- 172.21.15.5:0/592010433 shutdown_connections 2024-05-20T18:29:49.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:49.852+0000 7ff667365640 1 --2- 172.21.15.5:0/592010433 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff64406be30 0x7ff64406e340 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:49.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:49.852+0000 7ff667365640 1 --2- 172.21.15.5:0/592010433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6600d2330 0x7ff6601e8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:49.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:49.852+0000 7ff667365640 1 --2- 172.21.15.5:0/592010433 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6600cdf70 0x7ff6601f5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:49.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:49.852+0000 7ff667365640 1 --2- 172.21.15.5:0/592010433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff66006d830 0x7ff66015cac0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:49.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:49.852+0000 7ff667365640 1 -- 172.21.15.5:0/592010433 >> 172.21.15.5:0/592010433 conn(0x7ff6600cf450 msgr2=0x7ff6600717d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:49.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:49.852+0000 7ff667365640 1 -- 172.21.15.5:0/592010433 shutdown_connections 2024-05-20T18:29:49.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:49.852+0000 7ff667365640 1 -- 172.21.15.5:0/592010433 wait complete. 2024-05-20T18:29:49.873 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-20T18:29:49.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:49.998+0000 7f7b336d3640 1 Processor -- start 2024-05-20T18:29:49.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:49.999+0000 7f7b336d3640 1 -- start start 2024-05-20T18:29:50.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.000+0000 7f7b336d3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b2c0b1df0 0x7f7b2c0d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:50.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.000+0000 7f7b336d3640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7b2c0d14a0 0x7f7b2c0b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:50.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.000+0000 7f7b336d3640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b2c09c350 con 0x7f7b2c0b3280 2024-05-20T18:29:50.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.000+0000 7f7b336d3640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7b2c09bc20 con 0x7f7b2c0d14a0 2024-05-20T18:29:50.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.000+0000 7f7b336d3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7b2c0cedb0 con 0x7f7b2c0b1df0 2024-05-20T18:29:50.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.000+0000 7f7b31c70640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7b2c0b3280 0x7f7b2c0b3690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37356/0 (socket says 172.21.15.5:37356) 2024-05-20T18:29:50.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.000+0000 7f7b31c70640 1 -- 172.21.15.5:0/890937413 learned_addr learned my addr 172.21.15.5:0/890937413 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:50.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.000+0000 7f7b2bfff640 1 -- 172.21.15.5:0/890937413 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 279144264 0 0) 0x7f7b24001110 con 0x7f7b2c0b3280 2024-05-20T18:29:50.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.001+0000 7f7b3146f640 1 --2- 172.21.15.5:0/890937413 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b2c0b1df0 0x7f7b2c0d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:50.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.001+0000 7f7b2bfff640 1 -- 172.21.15.5:0/890937413 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b14003520 con 0x7f7b2c0b3280 2024-05-20T18:29:50.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.001+0000 7f7b30c6e640 1 --2- 172.21.15.5:0/890937413 >> v2:172.21.15.63:3300/0 conn(0x7f7b2c0d14a0 0x7f7b2c0b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:50.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.001+0000 7f7b2bfff640 1 -- 172.21.15.5:0/890937413 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3131848300 0 0) 0x7f7b14003520 con 0x7f7b2c0b3280 2024-05-20T18:29:50.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.001+0000 7f7b2bfff640 1 -- 172.21.15.5:0/890937413 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b2c0b1df0 msgr2=0x7f7b2c0d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:50.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.001+0000 7f7b2bfff640 1 --2- 172.21.15.5:0/890937413 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b2c0b1df0 0x7f7b2c0d0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:50.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.001+0000 7f7b2bfff640 1 -- 172.21.15.5:0/890937413 >> v2:172.21.15.63:3300/0 conn(0x7f7b2c0d14a0 msgr2=0x7f7b2c0b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:50.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.001+0000 7f7b2bfff640 1 --2- 172.21.15.5:0/890937413 >> v2:172.21.15.63:3300/0 conn(0x7f7b2c0d14a0 0x7f7b2c0b2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:50.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.001+0000 7f7b2bfff640 1 -- 172.21.15.5:0/890937413 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b2c0d1a50 con 0x7f7b2c0b3280 2024-05-20T18:29:50.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.001+0000 7f7b2bfff640 1 -- 172.21.15.5:0/890937413 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f7b24001840 con 0x7f7b2c0b3280 2024-05-20T18:29:50.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.001+0000 7f7b2bfff640 1 -- 172.21.15.5:0/890937413 >> v1:172.21.15.5:6789/0 conn(0x7f7b2c0b3280 legacy=0x7f7b2c0b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:50.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.001+0000 7f7b2bfff640 1 --2- 172.21.15.5:0/890937413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b140039b0 0x7f7b14003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:50.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.001+0000 7f7b2bfff640 1 --2- 172.21.15.5:0/890937413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b14004800 0x7f7b14004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:50.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.001+0000 7f7b2bfff640 1 --2- 172.21.15.5:0/890937413 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b2c0b1df0 0x7f7b14005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:50.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.001+0000 7f7b2bfff640 1 -- 172.21.15.5:0/890937413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c09bc20 con 0x7f7b14004800 2024-05-20T18:29:50.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.001+0000 7f7b2bfff640 1 -- 172.21.15.5:0/890937413 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c0cedb0 con 0x7f7b140039b0 2024-05-20T18:29:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.001+0000 7f7b2bfff640 1 -- 172.21.15.5:0/890937413 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7b1400ba70 con 0x7f7b2c0b1df0 2024-05-20T18:29:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.001+0000 7f7b30c6e640 1 --2- 172.21.15.5:0/890937413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b14004800 0x7f7b14004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.001+0000 7f7b3146f640 1 --2- 172.21.15.5:0/890937413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b140039b0 0x7f7b14003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.001+0000 7f7b31c70640 1 --2- 172.21.15.5:0/890937413 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b2c0b1df0 0x7f7b14005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.001+0000 7f7b30c6e640 1 -- 172.21.15.5:0/890937413 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b2c0b1df0 msgr2=0x7f7b14005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.001+0000 7f7b30c6e640 1 --2- 172.21.15.5:0/890937413 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b2c0b1df0 0x7f7b14005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.001+0000 7f7b30c6e640 1 -- 172.21.15.5:0/890937413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b140039b0 msgr2=0x7f7b14003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.001+0000 7f7b30c6e640 1 --2- 172.21.15.5:0/890937413 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b140039b0 0x7f7b14003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.001+0000 7f7b30c6e640 1 -- 172.21.15.5:0/890937413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b14003520 con 0x7f7b14004800 2024-05-20T18:29:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.002+0000 7f7b30c6e640 1 --2- 172.21.15.5:0/890937413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b14004800 0x7f7b14004c10 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009a90 tx=0x7f7b14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d523e87efc807d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:50.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.002+0000 7f7b2bfff640 1 -- 172.21.15.5:0/890937413 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20036070 con 0x7f7b14004800 2024-05-20T18:29:50.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.002+0000 7f7b2bfff640 1 -- 172.21.15.5:0/890937413 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b20031400 con 0x7f7b14004800 2024-05-20T18:29:50.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.002+0000 7f7b2bfff640 1 -- 172.21.15.5:0/890937413 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20009690 con 0x7f7b14004800 2024-05-20T18:29:50.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.003+0000 7f7b336d3640 1 -- 172.21.15.5:0/890937413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b14004800 msgr2=0x7f7b14004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:50.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.003+0000 7f7b336d3640 1 --2- 172.21.15.5:0/890937413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b14004800 0x7f7b14004c10 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009a90 tx=0x7f7b14005830 comp rx=0 tx=0).stop 2024-05-20T18:29:50.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.003+0000 7f7b31c70640 1 -- 172.21.15.5:0/890937413 reap_dead start 2024-05-20T18:29:50.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.003+0000 7f7b336d3640 1 -- 172.21.15.5:0/890937413 shutdown_connections 2024-05-20T18:29:50.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.003+0000 7f7b336d3640 1 -- 172.21.15.5:0/890937413 >> 172.21.15.5:0/890937413 conn(0x7f7b2c0cf450 msgr2=0x7f7b2c0cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:50.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.003+0000 7f7b336d3640 1 -- 172.21.15.5:0/890937413 shutdown_connections 2024-05-20T18:29:50.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.003+0000 7f7b336d3640 1 -- 172.21.15.5:0/890937413 wait complete. 2024-05-20T18:29:50.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.005+0000 7f7b336d3640 1 Processor -- start 2024-05-20T18:29:50.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.005+0000 7f7b336d3640 1 -- start start 2024-05-20T18:29:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.006+0000 7f7b336d3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b14004800 0x7f7b2c1d7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.006+0000 7f7b336d3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b2c0cb6e0 0x7f7b2c0cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.006+0000 7f7b336d3640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b2c1f5b70 0x7f7b2c1f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.006+0000 7f7b336d3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c0ce820 con 0x7f7b14004800 2024-05-20T18:29:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.006+0000 7f7b336d3640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c0b3da0 con 0x7f7b2c1f5b70 2024-05-20T18:29:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.006+0000 7f7b336d3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7b2c0d0650 con 0x7f7b2c0cb6e0 2024-05-20T18:29:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.007+0000 7f7b3146f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b14004800 0x7f7b2c1d7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.007+0000 7f7b3146f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b14004800 0x7f7b2c1d7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46540/0 (socket says 172.21.15.5:46540) 2024-05-20T18:29:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.007+0000 7f7b3146f640 1 -- 172.21.15.5:0/1013273963 learned_addr learned my addr 172.21.15.5:0/1013273963 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.007+0000 7f7b31c70640 1 --2- 172.21.15.5:0/1013273963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b2c1f5b70 0x7f7b2c1f5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.007+0000 7f7b30c6e640 1 --2- 172.21.15.5:0/1013273963 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b2c0cb6e0 0x7f7b2c0cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:50.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.007+0000 7f7b3146f640 1 -- 172.21.15.5:0/1013273963 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b2c0cb6e0 msgr2=0x7f7b2c0cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:50.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.007+0000 7f7b3146f640 1 --2- 172.21.15.5:0/1013273963 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b2c0cb6e0 0x7f7b2c0cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:50.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.007+0000 7f7b3146f640 1 -- 172.21.15.5:0/1013273963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b2c1f5b70 msgr2=0x7f7b2c1f5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:50.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.007+0000 7f7b3146f640 1 --2- 172.21.15.5:0/1013273963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b2c1f5b70 0x7f7b2c1f5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:50.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.007+0000 7f7b3146f640 1 -- 172.21.15.5:0/1013273963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b2c09f630 con 0x7f7b14004800 2024-05-20T18:29:50.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.007+0000 7f7b3146f640 1 --2- 172.21.15.5:0/1013273963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b14004800 0x7f7b2c1d7830 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c011e30 tx=0x7f7b1c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:50.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.007+0000 7f7b29ffb640 1 -- 172.21.15.5:0/1013273963 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c015630 con 0x7f7b14004800 2024-05-20T18:29:50.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.007+0000 7f7b29ffb640 1 -- 172.21.15.5:0/1013273963 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b1c0157d0 con 0x7f7b14004800 2024-05-20T18:29:50.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.007+0000 7f7b336d3640 1 -- 172.21.15.5:0/1013273963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b2c09f910 con 0x7f7b14004800 2024-05-20T18:29:50.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.007+0000 7f7b29ffb640 1 -- 172.21.15.5:0/1013273963 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c015ba0 con 0x7f7b14004800 2024-05-20T18:29:50.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.007+0000 7f7b336d3640 1 -- 172.21.15.5:0/1013273963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b2c09fed0 con 0x7f7b14004800 2024-05-20T18:29:50.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.009+0000 7f7b336d3640 1 -- 172.21.15.5:0/1013273963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b2c0690b0 con 0x7f7b14004800 2024-05-20T18:29:50.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.009+0000 7f7b29ffb640 1 -- 172.21.15.5:0/1013273963 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7b1c01d400 con 0x7f7b14004800 2024-05-20T18:29:50.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.010+0000 7f7b29ffb640 1 --2- 172.21.15.5:0/1013273963 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7b1806be30 0x7f7b1806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:50.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.010+0000 7f7b29ffb640 1 -- 172.21.15.5:0/1013273963 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5749+0+0 (secure 0 0 0) 0x7f7b1c08c190 con 0x7f7b14004800 2024-05-20T18:29:50.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.010+0000 7f7b30c6e640 1 --2- 172.21.15.5:0/1013273963 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7b1806be30 0x7f7b1806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:50.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.010+0000 7f7b30c6e640 1 --2- 172.21.15.5:0/1013273963 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7b1806be30 0x7f7b1806e340 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009a90 tx=0x7f7b20008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:50.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.014+0000 7f7b29ffb640 1 -- 172.21.15.5:0/1013273963 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7b1c019030 con 0x7f7b14004800 2024-05-20T18:29:50.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.197+0000 7f7b336d3640 1 -- 172.21.15.5:0/1013273963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7b2c0ccd60 con 0x7f7b14004800 2024-05-20T18:29:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.849+0000 7f7b29ffb640 1 -- 172.21.15.5:0/1013273963 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v54) ==== 169+0+0 (secure 0 0 0) 0x7f7b1c010040 con 0x7f7b14004800 2024-05-20T18:29:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-05-20T18:29:50.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.852+0000 7f7b336d3640 1 -- 172.21.15.5:0/1013273963 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7b1806be30 msgr2=0x7f7b1806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:50.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.852+0000 7f7b336d3640 1 --2- 172.21.15.5:0/1013273963 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7b1806be30 0x7f7b1806e340 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009a90 tx=0x7f7b20008030 comp rx=0 tx=0).stop 2024-05-20T18:29:50.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.852+0000 7f7b336d3640 1 -- 172.21.15.5:0/1013273963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b14004800 msgr2=0x7f7b2c1d7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:50.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.852+0000 7f7b336d3640 1 --2- 172.21.15.5:0/1013273963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b14004800 0x7f7b2c1d7830 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c011e30 tx=0x7f7b1c00f580 comp rx=0 tx=0).stop 2024-05-20T18:29:50.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.853+0000 7f7b336d3640 1 -- 172.21.15.5:0/1013273963 shutdown_connections 2024-05-20T18:29:50.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.853+0000 7f7b336d3640 1 --2- 172.21.15.5:0/1013273963 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7b1806be30 0x7f7b1806e340 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:50.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.853+0000 7f7b336d3640 1 --2- 172.21.15.5:0/1013273963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b2c1f5b70 0x7f7b2c1f5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:50.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.853+0000 7f7b336d3640 1 --2- 172.21.15.5:0/1013273963 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b2c0cb6e0 0x7f7b2c0cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:50.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.853+0000 7f7b336d3640 1 --2- 172.21.15.5:0/1013273963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b14004800 0x7f7b2c1d7830 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:50.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.853+0000 7f7b336d3640 1 -- 172.21.15.5:0/1013273963 >> 172.21.15.5:0/1013273963 conn(0x7f7b2c0cf450 msgr2=0x7f7b2c0b1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:50.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.853+0000 7f7b336d3640 1 -- 172.21.15.5:0/1013273963 shutdown_connections 2024-05-20T18:29:50.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.854+0000 7f7b336d3640 1 -- 172.21.15.5:0/1013273963 wait complete. 2024-05-20T18:29:50.870 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-20T18:29:50.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.995+0000 7fc3ca826640 1 Processor -- start 2024-05-20T18:29:50.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.995+0000 7fc3ca826640 1 -- start start 2024-05-20T18:29:50.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.997+0000 7fc3ca826640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc3c406d3d0 0x7fc3c406d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:50.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.997+0000 7fc3ca826640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3c406dd20 con 0x7fc3c4162820 2024-05-20T18:29:50.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.997+0000 7fc3ca826640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3c40d0630 con 0x7fc3c40b8190 2024-05-20T18:29:50.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.997+0000 7fc3ca826640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fc3c40d39c0 con 0x7fc3c406d3d0 2024-05-20T18:29:50.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.997+0000 7fc3c3fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc3c4162820 0x7fc3c40b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37370/0 (socket says 172.21.15.5:37370) 2024-05-20T18:29:50.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.997+0000 7fc3c3fff640 1 -- 172.21.15.5:0/3168558120 learned_addr learned my addr 172.21.15.5:0/3168558120 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:50.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.997+0000 7fc3c37fe640 1 --2- 172.21.15.5:0/3168558120 >> v2:172.21.15.5:3300/0 conn(0x7fc3c406d3d0 0x7fc3c406d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:50.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.997+0000 7fc3c2ffd640 1 -- 172.21.15.5:0/3168558120 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1983463925 0 0) 0x7fc3b4001110 con 0x7fc3c4162820 2024-05-20T18:29:50.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.997+0000 7fc3c2ffd640 1 -- 172.21.15.5:0/3168558120 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3ac003520 con 0x7fc3c4162820 2024-05-20T18:29:50.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.997+0000 7fc3c37fe640 1 -- 172.21.15.5:0/3168558120 >> v1:172.21.15.63:6789/0 conn(0x7fc3c40b8190 legacy=0x7fc3c40b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:50.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.997+0000 7fc3c37fe640 1 -- 172.21.15.5:0/3168558120 >> v1:172.21.15.5:6789/0 conn(0x7fc3c4162820 legacy=0x7fc3c40b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:50.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.997+0000 7fc3c37fe640 1 -- 172.21.15.5:0/3168558120 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c40d3ca0 con 0x7fc3c406d3d0 2024-05-20T18:29:50.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.997+0000 7fc3c37fe640 1 --2- 172.21.15.5:0/3168558120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3c406d3d0 0x7fc3c406d7e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc3b80097a0 tx=0x7fc3b802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9012a6b330e6299 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:50.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.997+0000 7fc3c2ffd640 1 -- 172.21.15.5:0/3168558120 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8037070 con 0x7fc3c406d3d0 2024-05-20T18:29:50.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.997+0000 7fc3c2ffd640 1 -- 172.21.15.5:0/3168558120 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3b8032440 con 0x7fc3c406d3d0 2024-05-20T18:29:50.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.997+0000 7fc3c2ffd640 1 -- 172.21.15.5:0/3168558120 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8032810 con 0x7fc3c406d3d0 2024-05-20T18:29:50.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.998+0000 7fc3ca826640 1 -- 172.21.15.5:0/3168558120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3c406d3d0 msgr2=0x7fc3c406d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:50.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.998+0000 7fc3ca826640 1 --2- 172.21.15.5:0/3168558120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3c406d3d0 0x7fc3c406d7e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc3b80097a0 tx=0x7fc3b802a1e0 comp rx=0 tx=0).stop 2024-05-20T18:29:50.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.998+0000 7fc3ca826640 1 -- 172.21.15.5:0/3168558120 shutdown_connections 2024-05-20T18:29:50.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.998+0000 7fc3ca826640 1 --2- 172.21.15.5:0/3168558120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3c406d3d0 0x7fc3c406d7e0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:50.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.998+0000 7fc3ca826640 1 -- 172.21.15.5:0/3168558120 >> 172.21.15.5:0/3168558120 conn(0x7fc3c40d4bd0 msgr2=0x7fc3c40d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:50.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.998+0000 7fc3ca826640 1 -- 172.21.15.5:0/3168558120 shutdown_connections 2024-05-20T18:29:50.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:50.998+0000 7fc3ca826640 1 -- 172.21.15.5:0/3168558120 wait complete. 2024-05-20T18:29:51.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.001+0000 7fc3ca826640 1 Processor -- start 2024-05-20T18:29:51.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.001+0000 7fc3ca826640 1 -- start start 2024-05-20T18:29:51.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.001+0000 7fc3ca826640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3c406d3d0 0x7fc3c41f5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:51.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.001+0000 7fc3ca826640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3c40b8190 0x7fc3c414ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:51.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.001+0000 7fc3ca826640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3c4162820 0x7fc3c41ea760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:51.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.001+0000 7fc3ca826640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c40d0300 con 0x7fc3c4162820 2024-05-20T18:29:51.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.001+0000 7fc3ca826640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c40d42e0 con 0x7fc3c40b8190 2024-05-20T18:29:51.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.001+0000 7fc3ca826640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c40b4050 con 0x7fc3c406d3d0 2024-05-20T18:29:51.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.002+0000 7fc3c8dc3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3c4162820 0x7fc3c41ea760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:51.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.002+0000 7fc3c8dc3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3c4162820 0x7fc3c41ea760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46562/0 (socket says 172.21.15.5:46562) 2024-05-20T18:29:51.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.002+0000 7fc3c8dc3640 1 -- 172.21.15.5:0/4009462572 learned_addr learned my addr 172.21.15.5:0/4009462572 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:51.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.002+0000 7fc3c37fe640 1 --2- 172.21.15.5:0/4009462572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3c40b8190 0x7fc3c414ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:51.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.002+0000 7fc3c3fff640 1 --2- 172.21.15.5:0/4009462572 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3c406d3d0 0x7fc3c41f5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:51.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.002+0000 7fc3c8dc3640 1 -- 172.21.15.5:0/4009462572 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3c406d3d0 msgr2=0x7fc3c41f5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:51.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.002+0000 7fc3c8dc3640 1 --2- 172.21.15.5:0/4009462572 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3c406d3d0 0x7fc3c41f5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:51.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.002+0000 7fc3c8dc3640 1 -- 172.21.15.5:0/4009462572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3c40b8190 msgr2=0x7fc3c414ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:51.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.002+0000 7fc3c8dc3640 1 --2- 172.21.15.5:0/4009462572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3c40b8190 0x7fc3c414ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:51.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.002+0000 7fc3c8dc3640 1 -- 172.21.15.5:0/4009462572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c41f6390 con 0x7fc3c4162820 2024-05-20T18:29:51.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.002+0000 7fc3c8dc3640 1 --2- 172.21.15.5:0/4009462572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3c4162820 0x7fc3c41ea760 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00b970 tx=0x7fc3bc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:51.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.002+0000 7fc3c0ff9640 1 -- 172.21.15.5:0/4009462572 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc019070 con 0x7fc3c4162820 2024-05-20T18:29:51.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.002+0000 7fc3c0ff9640 1 -- 172.21.15.5:0/4009462572 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3bc00cb40 con 0x7fc3c4162820 2024-05-20T18:29:51.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.002+0000 7fc3ca826640 1 -- 172.21.15.5:0/4009462572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c40b6cc0 con 0x7fc3c4162820 2024-05-20T18:29:51.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.002+0000 7fc3c0ff9640 1 -- 172.21.15.5:0/4009462572 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc014440 con 0x7fc3c4162820 2024-05-20T18:29:51.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.002+0000 7fc3ca826640 1 -- 172.21.15.5:0/4009462572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c41e8af0 con 0x7fc3c4162820 2024-05-20T18:29:51.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.003+0000 7fc3ca826640 1 -- 172.21.15.5:0/4009462572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3c4075b00 con 0x7fc3c4162820 2024-05-20T18:29:51.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.004+0000 7fc3c0ff9640 1 -- 172.21.15.5:0/4009462572 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc3bc01c070 con 0x7fc3c4162820 2024-05-20T18:29:51.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.005+0000 7fc3c0ff9640 1 --2- 172.21.15.5:0/4009462572 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc3b006be60 0x7fc3b006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:51.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.005+0000 7fc3c3fff640 1 --2- 172.21.15.5:0/4009462572 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc3b006be60 0x7fc3b006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:51.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.005+0000 7fc3c0ff9640 1 -- 172.21.15.5:0/4009462572 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5335+0+0 (secure 0 0 0) 0x7fc3bc08a900 con 0x7fc3c4162820 2024-05-20T18:29:51.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.005+0000 7fc3c3fff640 1 --2- 172.21.15.5:0/4009462572 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc3b006be60 0x7fc3b006e370 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4005880 tx=0x7fc3b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:51.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.010+0000 7fc3c0ff9640 1 -- 172.21.15.5:0/4009462572 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc3c41f6390 con 0x7fc3c4162820 2024-05-20T18:29:51.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.193+0000 7fc3ca826640 1 -- 172.21.15.5:0/4009462572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fc3c40b6f50 con 0x7fc3c4162820 2024-05-20T18:29:51.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.196+0000 7fc3c0ff9640 1 -- 172.21.15.5:0/4009462572 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7fc3bc058dd0 con 0x7fc3c4162820 2024-05-20T18:29:51.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.198+0000 7fc3ca826640 1 -- 172.21.15.5:0/4009462572 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc3b006be60 msgr2=0x7fc3b006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:51.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.198+0000 7fc3ca826640 1 --2- 172.21.15.5:0/4009462572 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc3b006be60 0x7fc3b006e370 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4005880 tx=0x7fc3b4009040 comp rx=0 tx=0).stop 2024-05-20T18:29:51.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.198+0000 7fc3ca826640 1 -- 172.21.15.5:0/4009462572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3c4162820 msgr2=0x7fc3c41ea760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:51.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.198+0000 7fc3ca826640 1 --2- 172.21.15.5:0/4009462572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3c4162820 0x7fc3c41ea760 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00b970 tx=0x7fc3bc00be30 comp rx=0 tx=0).stop 2024-05-20T18:29:51.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.198+0000 7fc3ca826640 1 -- 172.21.15.5:0/4009462572 shutdown_connections 2024-05-20T18:29:51.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.198+0000 7fc3ca826640 1 --2- 172.21.15.5:0/4009462572 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc3b006be60 0x7fc3b006e370 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:51.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.198+0000 7fc3ca826640 1 --2- 172.21.15.5:0/4009462572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3c4162820 0x7fc3c41ea760 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:51.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.198+0000 7fc3ca826640 1 --2- 172.21.15.5:0/4009462572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3c40b8190 0x7fc3c414ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:51.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.199+0000 7fc3ca826640 1 --2- 172.21.15.5:0/4009462572 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3c406d3d0 0x7fc3c41f5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:51.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.199+0000 7fc3ca826640 1 -- 172.21.15.5:0/4009462572 >> 172.21.15.5:0/4009462572 conn(0x7fc3c40d4bd0 msgr2=0x7fc3c40d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:51.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.199+0000 7fc3ca826640 1 -- 172.21.15.5:0/4009462572 shutdown_connections 2024-05-20T18:29:51.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.199+0000 7fc3ca826640 1 -- 172.21.15.5:0/4009462572 wait complete. 2024-05-20T18:29:51.209 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-05-20T18:29:51.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.326+0000 7f4005832640 1 Processor -- start 2024-05-20T18:29:51.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.326+0000 7f4005832640 1 -- start start 2024-05-20T18:29:51.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.327+0000 7f4005832640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f400006d3d0 0x7f400006d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:51.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.328+0000 7f4005832640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f400006dd20 con 0x7f40000b8190 2024-05-20T18:29:51.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.328+0000 7f4005832640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40000d0630 con 0x7f4000162820 2024-05-20T18:29:51.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.328+0000 7f4005832640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f40000d39c0 con 0x7f400006d3d0 2024-05-20T18:29:51.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.328+0000 7f3fff7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f40000b8190 0x7f40000b85a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37376/0 (socket says 172.21.15.5:37376) 2024-05-20T18:29:51.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.328+0000 7f3fff7fe640 1 -- 172.21.15.5:0/1170301958 learned_addr learned my addr 172.21.15.5:0/1170301958 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:51.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.328+0000 7f3ffe7fc640 1 --2- 172.21.15.5:0/1170301958 >> v2:172.21.15.63:3300/0 conn(0x7f400006d3d0 0x7f400006d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:51.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.328+0000 7f3ffdffb640 1 -- 172.21.15.5:0/1170301958 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3990796871 0 0) 0x7f3ff8001490 con 0x7f40000b8190 2024-05-20T18:29:51.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.328+0000 7f3ffdffb640 1 -- 172.21.15.5:0/1170301958 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3fe8003520 con 0x7f40000b8190 2024-05-20T18:29:51.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.328+0000 7f3ffdffb640 1 -- 172.21.15.5:0/1170301958 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2386885405 0 0) 0x7f3fe8003520 con 0x7f40000b8190 2024-05-20T18:29:51.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.328+0000 7f3ffdffb640 1 -- 172.21.15.5:0/1170301958 >> v2:172.21.15.63:3300/0 conn(0x7f400006d3d0 msgr2=0x7f400006d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:51.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.328+0000 7f3ffdffb640 1 --2- 172.21.15.5:0/1170301958 >> v2:172.21.15.63:3300/0 conn(0x7f400006d3d0 0x7f400006d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:51.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.328+0000 7f3ffdffb640 1 -- 172.21.15.5:0/1170301958 >> v1:172.21.15.63:6789/0 conn(0x7f4000162820 legacy=0x7f40000b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:51.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.328+0000 7f3ffdffb640 1 -- 172.21.15.5:0/1170301958 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40000d3ca0 con 0x7f40000b8190 2024-05-20T18:29:51.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.328+0000 7f3ffdffb640 1 -- 172.21.15.5:0/1170301958 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f3ff8001bc0 con 0x7f40000b8190 2024-05-20T18:29:51.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.328+0000 7f3ffdffb640 1 -- 172.21.15.5:0/1170301958 >> v1:172.21.15.5:6789/0 conn(0x7f40000b8190 legacy=0x7f40000b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:51.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.328+0000 7f3ffdffb640 1 --2- 172.21.15.5:0/1170301958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe80039b0 0x7f3fe8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:51.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.328+0000 7f3ffdffb640 1 --2- 172.21.15.5:0/1170301958 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3fe8004800 0x7f3fe8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:51.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.328+0000 7f3ffdffb640 1 --2- 172.21.15.5:0/1170301958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3fe80051c0 0x7f3fe800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:51.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.328+0000 7f3ffdffb640 1 -- 172.21.15.5:0/1170301958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f40000d39c0 con 0x7f3fe80039b0 2024-05-20T18:29:51.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.328+0000 7f3ffdffb640 1 -- 172.21.15.5:0/1170301958 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe800c160 con 0x7f3fe80051c0 2024-05-20T18:29:51.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.328+0000 7f3ffdffb640 1 -- 172.21.15.5:0/1170301958 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3fe800c360 con 0x7f3fe8004800 2024-05-20T18:29:51.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.329+0000 7f3ffeffd640 1 --2- 172.21.15.5:0/1170301958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe80039b0 0x7f3fe8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:51.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.329+0000 7f3fff7fe640 1 --2- 172.21.15.5:0/1170301958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3fe80051c0 0x7f3fe800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:51.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.329+0000 7f3ffe7fc640 1 --2- 172.21.15.5:0/1170301958 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3fe8004800 0x7f3fe8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:51.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.329+0000 7f3ffeffd640 1 -- 172.21.15.5:0/1170301958 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3fe8004800 msgr2=0x7f3fe8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:51.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.329+0000 7f3ffeffd640 1 --2- 172.21.15.5:0/1170301958 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3fe8004800 0x7f3fe8004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:51.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.329+0000 7f3ffeffd640 1 -- 172.21.15.5:0/1170301958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3fe80051c0 msgr2=0x7f3fe800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:51.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.329+0000 7f3ffeffd640 1 --2- 172.21.15.5:0/1170301958 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3fe80051c0 0x7f3fe800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:51.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.329+0000 7f3fff7fe640 1 -- 172.21.15.5:0/1170301958 reap_dead start 2024-05-20T18:29:51.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.329+0000 7f3ffeffd640 1 -- 172.21.15.5:0/1170301958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe8003520 con 0x7f3fe80039b0 2024-05-20T18:29:51.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.329+0000 7f3ffe7fc640 1 --2- 172.21.15.5:0/1170301958 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3fe8004800 0x7f3fe8004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:29:51.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.330+0000 7f3ffeffd640 1 --2- 172.21.15.5:0/1170301958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe80039b0 0x7f3fe8003e20 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0001750 tx=0x7f3fe8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f021076651ccb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:51.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.330+0000 7f3ffdffb640 1 -- 172.21.15.5:0/1170301958 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0001370 con 0x7f3fe80039b0 2024-05-20T18:29:51.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.330+0000 7f3ffdffb640 1 -- 172.21.15.5:0/1170301958 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ff00052d0 con 0x7f3fe80039b0 2024-05-20T18:29:51.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.330+0000 7f3ffdffb640 1 -- 172.21.15.5:0/1170301958 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff002ab00 con 0x7f3fe80039b0 2024-05-20T18:29:51.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.330+0000 7f4005832640 1 -- 172.21.15.5:0/1170301958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe80039b0 msgr2=0x7f3fe8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:51.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.330+0000 7f4005832640 1 --2- 172.21.15.5:0/1170301958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe80039b0 0x7f3fe8003e20 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0001750 tx=0x7f3fe8005830 comp rx=0 tx=0).stop 2024-05-20T18:29:51.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.331+0000 7f4005832640 1 -- 172.21.15.5:0/1170301958 shutdown_connections 2024-05-20T18:29:51.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.331+0000 7f4005832640 1 --2- 172.21.15.5:0/1170301958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe80039b0 0x7f3fe8003e20 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:51.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.331+0000 7f4005832640 1 -- 172.21.15.5:0/1170301958 >> 172.21.15.5:0/1170301958 conn(0x7f40000d4bd0 msgr2=0x7f40000d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:51.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.331+0000 7f4005832640 1 -- 172.21.15.5:0/1170301958 shutdown_connections 2024-05-20T18:29:51.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.331+0000 7f4005832640 1 -- 172.21.15.5:0/1170301958 wait complete. 2024-05-20T18:29:51.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.332+0000 7f4005832640 1 Processor -- start 2024-05-20T18:29:51.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.333+0000 7f4005832640 1 -- start start 2024-05-20T18:29:51.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.334+0000 7f4005832640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3fe80039b0 0x7f400015c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:51.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.334+0000 7f4005832640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40001f8b30 0x7f400015b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:51.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.334+0000 7f4005832640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4000158270 0x7f4000158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:51.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.334+0000 7f4005832640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f40000d0300 con 0x7f40001f8b30 2024-05-20T18:29:51.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.334+0000 7f4005832640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f40000d4260 con 0x7f4000158270 2024-05-20T18:29:51.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.334+0000 7f4005832640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f40000d4780 con 0x7f3fe80039b0 2024-05-20T18:29:51.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.334+0000 7f3ffe7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40001f8b30 0x7f400015b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:51.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.334+0000 7f3ffe7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40001f8b30 0x7f400015b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46576/0 (socket says 172.21.15.5:46576) 2024-05-20T18:29:51.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.334+0000 7f3ffe7fc640 1 -- 172.21.15.5:0/3663582222 learned_addr learned my addr 172.21.15.5:0/3663582222 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:51.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.334+0000 7f3ffeffd640 1 --2- 172.21.15.5:0/3663582222 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3fe80039b0 0x7f400015c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:51.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.334+0000 7f3fff7fe640 1 --2- 172.21.15.5:0/3663582222 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4000158270 0x7f4000158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:51.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.334+0000 7f3ffe7fc640 1 -- 172.21.15.5:0/3663582222 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3fe80039b0 msgr2=0x7f400015c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:51.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.334+0000 7f3ffe7fc640 1 --2- 172.21.15.5:0/3663582222 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3fe80039b0 0x7f400015c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:51.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.334+0000 7f3ffe7fc640 1 -- 172.21.15.5:0/3663582222 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4000158270 msgr2=0x7f4000158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:51.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.334+0000 7f3ffe7fc640 1 --2- 172.21.15.5:0/3663582222 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4000158270 0x7f4000158680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:51.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.334+0000 7f3ffe7fc640 1 -- 172.21.15.5:0/3663582222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4000158bc0 con 0x7f40001f8b30 2024-05-20T18:29:51.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.334+0000 7f3ffeffd640 1 --2- 172.21.15.5:0/3663582222 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3fe80039b0 0x7f400015c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:29:51.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.335+0000 7f3ffe7fc640 1 --2- 172.21.15.5:0/3663582222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40001f8b30 0x7f400015b840 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4011510 tx=0x7f3ff40119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:51.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.335+0000 7f3fe7fff640 1 -- 172.21.15.5:0/3663582222 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4015870 con 0x7f40001f8b30 2024-05-20T18:29:51.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.335+0000 7f3fe7fff640 1 -- 172.21.15.5:0/3663582222 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ff4015a10 con 0x7f40001f8b30 2024-05-20T18:29:51.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.335+0000 7f4005832640 1 -- 172.21.15.5:0/3663582222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40001f92a0 con 0x7f40001f8b30 2024-05-20T18:29:51.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.335+0000 7f3fe7fff640 1 -- 172.21.15.5:0/3663582222 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4015e00 con 0x7f40001f8b30 2024-05-20T18:29:51.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.335+0000 7f4005832640 1 -- 172.21.15.5:0/3663582222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40001f9860 con 0x7f40001f8b30 2024-05-20T18:29:51.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.336+0000 7f4005832640 1 -- 172.21.15.5:0/3663582222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4000162c30 con 0x7f40001f8b30 2024-05-20T18:29:51.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.336+0000 7f3fe7fff640 1 -- 172.21.15.5:0/3663582222 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3ff401d400 con 0x7f40001f8b30 2024-05-20T18:29:51.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.337+0000 7f3fe7fff640 1 --2- 172.21.15.5:0/3663582222 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3fec06be60 0x7f3fec06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:51.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.337+0000 7f3fe7fff640 1 -- 172.21.15.5:0/3663582222 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5335+0+0 (secure 0 0 0) 0x7f3ff408b500 con 0x7f40001f8b30 2024-05-20T18:29:51.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.337+0000 7f3ffeffd640 1 --2- 172.21.15.5:0/3663582222 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3fec06be60 0x7f3fec06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:51.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.337+0000 7f3ffeffd640 1 --2- 172.21.15.5:0/3663582222 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3fec06be60 0x7f3fec06e370 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0032000 tx=0x7f3ff002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:51.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.342+0000 7f3fe7fff640 1 -- 172.21.15.5:0/3663582222 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4000162c30 con 0x7f40001f8b30 2024-05-20T18:29:51.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.610+0000 7f4005832640 1 -- 172.21.15.5:0/3663582222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7f4000075b00 con 0x7f40001f8b30 2024-05-20T18:29:51.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.858+0000 7f3fe7fff640 1 -- 172.21.15.5:0/3663582222 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v158) ==== 149+0+0 (secure 0 0 0) 0x7f4000158bc0 con 0x7f40001f8b30 2024-05-20T18:29:51.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.861+0000 7f4005832640 1 -- 172.21.15.5:0/3663582222 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3fec06be60 msgr2=0x7f3fec06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:51.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.861+0000 7f4005832640 1 --2- 172.21.15.5:0/3663582222 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3fec06be60 0x7f3fec06e370 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0032000 tx=0x7f3ff002e000 comp rx=0 tx=0).stop 2024-05-20T18:29:51.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.861+0000 7f4005832640 1 -- 172.21.15.5:0/3663582222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40001f8b30 msgr2=0x7f400015b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:51.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.861+0000 7f4005832640 1 --2- 172.21.15.5:0/3663582222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40001f8b30 0x7f400015b840 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4011510 tx=0x7f3ff40119d0 comp rx=0 tx=0).stop 2024-05-20T18:29:51.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.861+0000 7f4005832640 1 -- 172.21.15.5:0/3663582222 shutdown_connections 2024-05-20T18:29:51.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.861+0000 7f4005832640 1 --2- 172.21.15.5:0/3663582222 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3fec06be60 0x7f3fec06e370 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:51.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.861+0000 7f4005832640 1 --2- 172.21.15.5:0/3663582222 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4000158270 0x7f4000158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:51.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.861+0000 7f4005832640 1 --2- 172.21.15.5:0/3663582222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40001f8b30 0x7f400015b840 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:51.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.861+0000 7f4005832640 1 --2- 172.21.15.5:0/3663582222 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3fe80039b0 0x7f400015c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:51.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.861+0000 7f4005832640 1 -- 172.21.15.5:0/3663582222 >> 172.21.15.5:0/3663582222 conn(0x7f40000d4bd0 msgr2=0x7f40000b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:51.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.862+0000 7f4005832640 1 -- 172.21.15.5:0/3663582222 shutdown_connections 2024-05-20T18:29:51.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.862+0000 7f4005832640 1 -- 172.21.15.5:0/3663582222 wait complete. 2024-05-20T18:29:51.873 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2024-05-20T18:29:51.873 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-05-20T18:29:51.873 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-20T18:29:51.874 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:29:51.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.990+0000 7fd38cf09640 1 Processor -- start 2024-05-20T18:29:51.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.990+0000 7fd38cf09640 1 -- start start 2024-05-20T18:29:51.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.991+0000 7fd38cf09640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd388162820 0x7fd3880b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:51.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.991+0000 7fd38cf09640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd38806dae0 0x7fd3880b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:51.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.991+0000 7fd38cf09640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3880d0630 con 0x7fd3880b3e80 2024-05-20T18:29:51.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.991+0000 7fd38cf09640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fd3880d39c0 con 0x7fd38806dae0 2024-05-20T18:29:51.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.991+0000 7fd38cf09640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd3880d3ca0 con 0x7fd388162820 2024-05-20T18:29:51.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.992+0000 7fd386d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd38806dae0 0x7fd3880b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:51.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.992+0000 7fd386d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd38806dae0 0x7fd3880b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46580/0 (socket says 172.21.15.5:46580) 2024-05-20T18:29:51.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.992+0000 7fd386d9d640 1 -- 172.21.15.5:0/2827313616 learned_addr learned my addr 172.21.15.5:0/2827313616 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:51.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.992+0000 7fd38659c640 1 --2- 172.21.15.5:0/2827313616 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd388162820 0x7fd3880b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:51.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.992+0000 7fd386d9d640 1 -- 172.21.15.5:0/2827313616 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd388162820 msgr2=0x7fd3880b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:51.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.992+0000 7fd386d9d640 1 --2- 172.21.15.5:0/2827313616 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd388162820 0x7fd3880b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:51.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.992+0000 7fd386d9d640 1 -- 172.21.15.5:0/2827313616 >> v1:172.21.15.63:6789/0 conn(0x7fd3880b3e80 legacy=0x7fd38806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:51.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.992+0000 7fd386d9d640 1 -- 172.21.15.5:0/2827313616 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3880d3f80 con 0x7fd38806dae0 2024-05-20T18:29:51.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.992+0000 7fd38659c640 1 --2- 172.21.15.5:0/2827313616 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd388162820 0x7fd3880b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:29:51.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.993+0000 7fd386d9d640 1 --2- 172.21.15.5:0/2827313616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd38806dae0 0x7fd3880b8190 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd380009e50 tx=0x7fd38002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62f7aea55a303f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:51.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.993+0000 7fd385d9b640 1 -- 172.21.15.5:0/2827313616 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd380036070 con 0x7fd38806dae0 2024-05-20T18:29:51.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.993+0000 7fd385d9b640 1 -- 172.21.15.5:0/2827313616 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd380031600 con 0x7fd38806dae0 2024-05-20T18:29:51.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.993+0000 7fd385d9b640 1 -- 172.21.15.5:0/2827313616 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3800319d0 con 0x7fd38806dae0 2024-05-20T18:29:51.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.993+0000 7fd38cf09640 1 -- 172.21.15.5:0/2827313616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd38806dae0 msgr2=0x7fd3880b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:51.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.993+0000 7fd38cf09640 1 --2- 172.21.15.5:0/2827313616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd38806dae0 0x7fd3880b8190 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd380009e50 tx=0x7fd38002a710 comp rx=0 tx=0).stop 2024-05-20T18:29:51.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.994+0000 7fd38cf09640 1 -- 172.21.15.5:0/2827313616 shutdown_connections 2024-05-20T18:29:51.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.994+0000 7fd38cf09640 1 --2- 172.21.15.5:0/2827313616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd38806dae0 0x7fd3880b8190 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:51.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.994+0000 7fd38cf09640 1 --2- 172.21.15.5:0/2827313616 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd388162820 0x7fd3880b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:51.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.994+0000 7fd38cf09640 1 -- 172.21.15.5:0/2827313616 >> 172.21.15.5:0/2827313616 conn(0x7fd3880d4bd0 msgr2=0x7fd3880d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:51.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.994+0000 7fd38cf09640 1 -- 172.21.15.5:0/2827313616 shutdown_connections 2024-05-20T18:29:51.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.994+0000 7fd38cf09640 1 -- 172.21.15.5:0/2827313616 wait complete. 2024-05-20T18:29:51.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.995+0000 7fd38cf09640 1 Processor -- start 2024-05-20T18:29:51.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.996+0000 7fd38cf09640 1 -- start start 2024-05-20T18:29:51.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.997+0000 7fd38cf09640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3880b3e80 0x7fd3881f6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:51.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.997+0000 7fd38cf09640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd388162820 0x7fd3881f6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:51.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.997+0000 7fd38cf09640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd388158940 0x7fd3881f91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:51.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.997+0000 7fd38cf09640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd3880d4780 con 0x7fd388162820 2024-05-20T18:29:51.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.997+0000 7fd38cf09640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd3880d0300 con 0x7fd388158940 2024-05-20T18:29:51.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.997+0000 7fd38cf09640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd3880d4900 con 0x7fd3880b3e80 2024-05-20T18:29:51.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.997+0000 7fd37ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd388162820 0x7fd3881f6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:51.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.997+0000 7fd37ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd388162820 0x7fd3881f6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46590/0 (socket says 172.21.15.5:46590) 2024-05-20T18:29:51.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.997+0000 7fd37ffff640 1 -- 172.21.15.5:0/3119060890 learned_addr learned my addr 172.21.15.5:0/3119060890 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:51.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.997+0000 7fd38659c640 1 --2- 172.21.15.5:0/3119060890 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3880b3e80 0x7fd3881f6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:51.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.997+0000 7fd386d9d640 1 --2- 172.21.15.5:0/3119060890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd388158940 0x7fd3881f91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:51.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.997+0000 7fd37ffff640 1 -- 172.21.15.5:0/3119060890 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3880b3e80 msgr2=0x7fd3881f6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:51.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.997+0000 7fd37ffff640 1 --2- 172.21.15.5:0/3119060890 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3880b3e80 0x7fd3881f6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:51.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.997+0000 7fd37ffff640 1 -- 172.21.15.5:0/3119060890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd388158940 msgr2=0x7fd3881f91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:51.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.997+0000 7fd37ffff640 1 --2- 172.21.15.5:0/3119060890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd388158940 0x7fd3881f91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:51.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.997+0000 7fd37ffff640 1 -- 172.21.15.5:0/3119060890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd388159050 con 0x7fd388162820 2024-05-20T18:29:51.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.997+0000 7fd37ffff640 1 --2- 172.21.15.5:0/3119060890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd388162820 0x7fd3881f6790 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fd37000c440 tx=0x7fd37000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:51.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.998+0000 7fd37effd640 1 -- 172.21.15.5:0/3119060890 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3700052d0 con 0x7fd388162820 2024-05-20T18:29:51.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.998+0000 7fd37effd640 1 -- 172.21.15.5:0/3119060890 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd370005470 con 0x7fd388162820 2024-05-20T18:29:51.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.998+0000 7fd38cf09640 1 -- 172.21.15.5:0/3119060890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd388159330 con 0x7fd388162820 2024-05-20T18:29:51.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.998+0000 7fd37effd640 1 -- 172.21.15.5:0/3119060890 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd370007260 con 0x7fd388162820 2024-05-20T18:29:51.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.998+0000 7fd38cf09640 1 -- 172.21.15.5:0/3119060890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3881598f0 con 0x7fd388162820 2024-05-20T18:29:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:51.999+0000 7fd38cf09640 1 -- 172.21.15.5:0/3119060890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd388075b00 con 0x7fd388162820 2024-05-20T18:29:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.000+0000 7fd37effd640 1 -- 172.21.15.5:0/3119060890 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd3700074c0 con 0x7fd388162820 2024-05-20T18:29:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.000+0000 7fd37effd640 1 --2- 172.21.15.5:0/3119060890 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd36806be30 0x7fd36806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.000+0000 7fd37effd640 1 -- 172.21.15.5:0/3119060890 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5335+0+0 (secure 0 0 0) 0x7fd37008a0d0 con 0x7fd388162820 2024-05-20T18:29:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.000+0000 7fd38659c640 1 --2- 172.21.15.5:0/3119060890 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd36806be30 0x7fd36806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.001+0000 7fd38659c640 1 --2- 172.21.15.5:0/3119060890 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd36806be30 0x7fd36806e340 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd378006fd0 tx=0x7fd378008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:52.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.005+0000 7fd37effd640 1 -- 172.21.15.5:0/3119060890 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd3700585a0 con 0x7fd388162820 2024-05-20T18:29:52.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.190+0000 7fd38cf09640 1 -- 172.21.15.5:0/3119060890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd388158de0 con 0x7fd388162820 2024-05-20T18:29:52.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.191+0000 7fd37effd640 1 -- 172.21.15.5:0/3119060890 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+13171 (secure 0 0 0) 0x7fd37005c8a0 con 0x7fd388162820 2024-05-20T18:29:52.190 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:29:52.190 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":54,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:29:50.848700+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6837/3469824363":"2024-05-21T18:29:47.840494+0000","172.21.15.5:6836/3469824363":"2024-05-21T18:29:47.840494+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:29:52.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.193+0000 7fd38cf09640 1 -- 172.21.15.5:0/3119060890 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd36806be30 msgr2=0x7fd36806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:52.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.193+0000 7fd38cf09640 1 --2- 172.21.15.5:0/3119060890 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd36806be30 0x7fd36806e340 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd378006fd0 tx=0x7fd378008040 comp rx=0 tx=0).stop 2024-05-20T18:29:52.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.193+0000 7fd38cf09640 1 -- 172.21.15.5:0/3119060890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd388162820 msgr2=0x7fd3881f6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:52.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.193+0000 7fd38cf09640 1 --2- 172.21.15.5:0/3119060890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd388162820 0x7fd3881f6790 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fd37000c440 tx=0x7fd37000c900 comp rx=0 tx=0).stop 2024-05-20T18:29:52.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.193+0000 7fd38cf09640 1 -- 172.21.15.5:0/3119060890 shutdown_connections 2024-05-20T18:29:52.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.193+0000 7fd38cf09640 1 --2- 172.21.15.5:0/3119060890 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd36806be30 0x7fd36806e340 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:52.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.193+0000 7fd38cf09640 1 --2- 172.21.15.5:0/3119060890 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd388158940 0x7fd3881f91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:52.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.193+0000 7fd38cf09640 1 --2- 172.21.15.5:0/3119060890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd388162820 0x7fd3881f6790 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:52.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.193+0000 7fd38cf09640 1 --2- 172.21.15.5:0/3119060890 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3880b3e80 0x7fd3881f6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:52.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.193+0000 7fd38cf09640 1 -- 172.21.15.5:0/3119060890 >> 172.21.15.5:0/3119060890 conn(0x7fd3880d4bd0 msgr2=0x7fd38815e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:52.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.194+0000 7fd38cf09640 1 -- 172.21.15.5:0/3119060890 shutdown_connections 2024-05-20T18:29:52.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.194+0000 7fd38cf09640 1 -- 172.21.15.5:0/3119060890 wait complete. 2024-05-20T18:29:52.204 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:29:52.204 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:29:52.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.322+0000 7f73b41ea640 1 Processor -- start 2024-05-20T18:29:52.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.322+0000 7f73b41ea640 1 -- start start 2024-05-20T18:29:52.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.324+0000 7f73b41ea640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f73ac162820 0x7f73ac0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:52.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.324+0000 7f73b41ea640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73ac0d0630 con 0x7f73ac0b3e80 2024-05-20T18:29:52.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.324+0000 7f73b41ea640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73ac0d39c0 con 0x7f73ac06dae0 2024-05-20T18:29:52.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.324+0000 7f73b41ea640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f73ac0d3ca0 con 0x7f73ac162820 2024-05-20T18:29:52.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.324+0000 7f73b1785640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f73ac0b3e80 0x7f73ac06d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37378/0 (socket says 172.21.15.5:37378) 2024-05-20T18:29:52.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.324+0000 7f73b1785640 1 -- 172.21.15.5:0/2236452093 learned_addr learned my addr 172.21.15.5:0/2236452093 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:52.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.324+0000 7f73b1f86640 1 --2- 172.21.15.5:0/2236452093 >> v2:172.21.15.63:3300/0 conn(0x7f73ac162820 0x7f73ac0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:52.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.324+0000 7f73b0f84640 1 -- 172.21.15.5:0/2236452093 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2290630222 0 0) 0x7f73ac0d0630 con 0x7f73ac0b3e80 2024-05-20T18:29:52.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.324+0000 7f73b0f84640 1 -- 172.21.15.5:0/2236452093 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7394003520 con 0x7f73ac0b3e80 2024-05-20T18:29:52.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.325+0000 7f73b0f84640 1 -- 172.21.15.5:0/2236452093 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1296685416 0 0) 0x7f7394003520 con 0x7f73ac0b3e80 2024-05-20T18:29:52.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.325+0000 7f73b0f84640 1 -- 172.21.15.5:0/2236452093 >> v2:172.21.15.63:3300/0 conn(0x7f73ac162820 msgr2=0x7f73ac0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:52.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.325+0000 7f73b0f84640 1 --2- 172.21.15.5:0/2236452093 >> v2:172.21.15.63:3300/0 conn(0x7f73ac162820 0x7f73ac0b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:52.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.325+0000 7f73b0f84640 1 -- 172.21.15.5:0/2236452093 >> v1:172.21.15.63:6789/0 conn(0x7f73ac06dae0 legacy=0x7f73ac0b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:52.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.325+0000 7f73b0f84640 1 -- 172.21.15.5:0/2236452093 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ac0d3f80 con 0x7f73ac0b3e80 2024-05-20T18:29:52.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.325+0000 7f73b0f84640 1 -- 172.21.15.5:0/2236452093 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f73a0001670 con 0x7f73ac0b3e80 2024-05-20T18:29:52.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.325+0000 7f73b0f84640 1 -- 172.21.15.5:0/2236452093 >> v1:172.21.15.5:6789/0 conn(0x7f73ac0b3e80 legacy=0x7f73ac06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:52.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.325+0000 7f73b0f84640 1 --2- 172.21.15.5:0/2236452093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73940039b0 0x7f7394003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:52.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.325+0000 7f73b0f84640 1 --2- 172.21.15.5:0/2236452093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7394004800 0x7f7394004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:52.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.325+0000 7f73b0f84640 1 --2- 172.21.15.5:0/2236452093 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73940051c0 0x7f739400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:52.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.325+0000 7f73b0f84640 1 -- 172.21.15.5:0/2236452093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac0d3ca0 con 0x7f73940039b0 2024-05-20T18:29:52.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.325+0000 7f73b0f84640 1 -- 172.21.15.5:0/2236452093 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f739400c160 con 0x7f7394004800 2024-05-20T18:29:52.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.325+0000 7f73b0f84640 1 -- 172.21.15.5:0/2236452093 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f739400c360 con 0x7f73940051c0 2024-05-20T18:29:52.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.325+0000 7f73b1f86640 1 --2- 172.21.15.5:0/2236452093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73940039b0 0x7f7394003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:52.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.325+0000 7f73b1785640 1 --2- 172.21.15.5:0/2236452093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7394004800 0x7f7394004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:52.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.325+0000 7f73b2787640 1 --2- 172.21.15.5:0/2236452093 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73940051c0 0x7f739400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:52.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.325+0000 7f73b1f86640 1 -- 172.21.15.5:0/2236452093 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73940051c0 msgr2=0x7f739400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:52.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.325+0000 7f73b1f86640 1 --2- 172.21.15.5:0/2236452093 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73940051c0 0x7f739400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:52.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.325+0000 7f73b1f86640 1 -- 172.21.15.5:0/2236452093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7394004800 msgr2=0x7f7394004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:52.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.325+0000 7f73b1f86640 1 --2- 172.21.15.5:0/2236452093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7394004800 0x7f7394004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:52.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.325+0000 7f73b1f86640 1 -- 172.21.15.5:0/2236452093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7394003520 con 0x7f73940039b0 2024-05-20T18:29:52.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.325+0000 7f73b2787640 1 --2- 172.21.15.5:0/2236452093 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73940051c0 0x7f739400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:29:52.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.325+0000 7f73b2787640 1 -- 172.21.15.5:0/2236452093 reap_dead start 2024-05-20T18:29:52.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.326+0000 7f73b1f86640 1 --2- 172.21.15.5:0/2236452093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73940039b0 0x7f7394003e20 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f739c009ac0 tx=0x7f7394005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe29c29ed6c6d90 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:52.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.326+0000 7f73b0f84640 1 -- 172.21.15.5:0/2236452093 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c036070 con 0x7f73940039b0 2024-05-20T18:29:52.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.326+0000 7f73b0f84640 1 -- 172.21.15.5:0/2236452093 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f739c031400 con 0x7f73940039b0 2024-05-20T18:29:52.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.326+0000 7f73b0f84640 1 -- 172.21.15.5:0/2236452093 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c009690 con 0x7f73940039b0 2024-05-20T18:29:52.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.327+0000 7f73b41ea640 1 -- 172.21.15.5:0/2236452093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73940039b0 msgr2=0x7f7394003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:52.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.327+0000 7f73b41ea640 1 --2- 172.21.15.5:0/2236452093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73940039b0 0x7f7394003e20 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f739c009ac0 tx=0x7f7394005830 comp rx=0 tx=0).stop 2024-05-20T18:29:52.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.327+0000 7f73b41ea640 1 -- 172.21.15.5:0/2236452093 shutdown_connections 2024-05-20T18:29:52.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.327+0000 7f73b41ea640 1 --2- 172.21.15.5:0/2236452093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73940039b0 0x7f7394003e20 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:52.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.327+0000 7f73b41ea640 1 -- 172.21.15.5:0/2236452093 >> 172.21.15.5:0/2236452093 conn(0x7f73ac0d4bd0 msgr2=0x7f73ac0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:52.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.327+0000 7f73b41ea640 1 -- 172.21.15.5:0/2236452093 shutdown_connections 2024-05-20T18:29:52.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.327+0000 7f73b41ea640 1 -- 172.21.15.5:0/2236452093 wait complete. 2024-05-20T18:29:52.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.329+0000 7f73b41ea640 1 Processor -- start 2024-05-20T18:29:52.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.329+0000 7f73b41ea640 1 -- start start 2024-05-20T18:29:52.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.330+0000 7f73b41ea640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73940039b0 0x7f73ac0ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:52.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.330+0000 7f73b41ea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73ac0b6840 0x7f73ac0b6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:52.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.330+0000 7f73b41ea640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73ac0b45f0 0x7f73ac0b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:52.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.330+0000 7f73b41ea640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac0d4780 con 0x7f73ac0b6840 2024-05-20T18:29:52.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.330+0000 7f73b41ea640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac0d0300 con 0x7f73940039b0 2024-05-20T18:29:52.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.330+0000 7f73b41ea640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f73ac0d4900 con 0x7f73ac0b45f0 2024-05-20T18:29:52.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.330+0000 7f73b1785640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73ac0b6840 0x7f73ac0b6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:52.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.330+0000 7f73b1785640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73ac0b6840 0x7f73ac0b6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46614/0 (socket says 172.21.15.5:46614) 2024-05-20T18:29:52.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.330+0000 7f73b1785640 1 -- 172.21.15.5:0/2229529233 learned_addr learned my addr 172.21.15.5:0/2229529233 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:52.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.330+0000 7f73b2787640 1 --2- 172.21.15.5:0/2229529233 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73ac0b45f0 0x7f73ac0b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:52.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.330+0000 7f73b1785640 1 -- 172.21.15.5:0/2229529233 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73ac0b45f0 msgr2=0x7f73ac0b4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:52.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.330+0000 7f73b1785640 1 --2- 172.21.15.5:0/2229529233 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73ac0b45f0 0x7f73ac0b4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:52.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.330+0000 7f73b1f86640 1 --2- 172.21.15.5:0/2229529233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73940039b0 0x7f73ac0ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:52.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.330+0000 7f73b1785640 1 -- 172.21.15.5:0/2229529233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73940039b0 msgr2=0x7f73ac0ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:52.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.330+0000 7f73b1785640 1 --2- 172.21.15.5:0/2229529233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73940039b0 0x7f73ac0ce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:52.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.330+0000 7f73b1785640 1 -- 172.21.15.5:0/2229529233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ac0b4f40 con 0x7f73ac0b6840 2024-05-20T18:29:52.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.331+0000 7f73b1785640 1 --2- 172.21.15.5:0/2229529233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73ac0b6840 0x7f73ac0b6c50 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f73a00072f0 tx=0x7f73a00077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:52.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.331+0000 7f73aa7fc640 1 -- 172.21.15.5:0/2229529233 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a001a070 con 0x7f73ac0b6840 2024-05-20T18:29:52.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.331+0000 7f73aa7fc640 1 -- 172.21.15.5:0/2229529233 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73a000f9c0 con 0x7f73ac0b6840 2024-05-20T18:29:52.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.331+0000 7f73b41ea640 1 -- 172.21.15.5:0/2229529233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73ac0c9300 con 0x7f73ac0b6840 2024-05-20T18:29:52.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.331+0000 7f73aa7fc640 1 -- 172.21.15.5:0/2229529233 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a000fd70 con 0x7f73ac0b6840 2024-05-20T18:29:52.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.331+0000 7f73b41ea640 1 -- 172.21.15.5:0/2229529233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73ac0c98c0 con 0x7f73ac0b6840 2024-05-20T18:29:52.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.332+0000 7f73b41ea640 1 -- 172.21.15.5:0/2229529233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73ac075b00 con 0x7f73ac0b6840 2024-05-20T18:29:52.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.332+0000 7f73aa7fc640 1 -- 172.21.15.5:0/2229529233 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f73a001d070 con 0x7f73ac0b6840 2024-05-20T18:29:52.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.333+0000 7f73aa7fc640 1 --2- 172.21.15.5:0/2229529233 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f739806be60 0x7f739806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:52.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.333+0000 7f73aa7fc640 1 -- 172.21.15.5:0/2229529233 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5335+0+0 (secure 0 0 0) 0x7f73a008a930 con 0x7f73ac0b6840 2024-05-20T18:29:52.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.333+0000 7f73b1f86640 1 --2- 172.21.15.5:0/2229529233 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f739806be60 0x7f739806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:52.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.334+0000 7f73b1f86640 1 --2- 172.21.15.5:0/2229529233 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f739806be60 0x7f739806e370 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f739c033000 tx=0x7f739c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:52.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.338+0000 7f73aa7fc640 1 -- 172.21.15.5:0/2229529233 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f73ac0b4f40 con 0x7f73ac0b6840 2024-05-20T18:29:52.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.520+0000 7f73b41ea640 1 -- 172.21.15.5:0/2229529233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f73ac0b7390 con 0x7f73ac0b6840 2024-05-20T18:29:52.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.521+0000 7f73aa7fc640 1 -- 172.21.15.5:0/2229529233 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) ==== 93+0+10 (secure 0 0 0) 0x7f73a00593c0 con 0x7f73ac0b6840 2024-05-20T18:29:52.521 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:29:52.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.523+0000 7f73b41ea640 1 -- 172.21.15.5:0/2229529233 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f739806be60 msgr2=0x7f739806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:52.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.523+0000 7f73b41ea640 1 --2- 172.21.15.5:0/2229529233 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f739806be60 0x7f739806e370 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f739c033000 tx=0x7f739c008030 comp rx=0 tx=0).stop 2024-05-20T18:29:52.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.523+0000 7f73b41ea640 1 -- 172.21.15.5:0/2229529233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73ac0b6840 msgr2=0x7f73ac0b6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:52.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.523+0000 7f73b41ea640 1 --2- 172.21.15.5:0/2229529233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73ac0b6840 0x7f73ac0b6c50 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f73a00072f0 tx=0x7f73a00077b0 comp rx=0 tx=0).stop 2024-05-20T18:29:52.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.523+0000 7f73b41ea640 1 -- 172.21.15.5:0/2229529233 shutdown_connections 2024-05-20T18:29:52.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.523+0000 7f73b41ea640 1 --2- 172.21.15.5:0/2229529233 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f739806be60 0x7f739806e370 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:52.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.523+0000 7f73b41ea640 1 --2- 172.21.15.5:0/2229529233 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73ac0b45f0 0x7f73ac0b4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:52.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.523+0000 7f73b41ea640 1 --2- 172.21.15.5:0/2229529233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73ac0b6840 0x7f73ac0b6c50 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:52.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.523+0000 7f73b41ea640 1 --2- 172.21.15.5:0/2229529233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73940039b0 0x7f73ac0ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:52.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.523+0000 7f73b41ea640 1 -- 172.21.15.5:0/2229529233 >> 172.21.15.5:0/2229529233 conn(0x7f73ac0d4bd0 msgr2=0x7f73ac0b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:52.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.523+0000 7f73b41ea640 1 -- 172.21.15.5:0/2229529233 shutdown_connections 2024-05-20T18:29:52.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.523+0000 7f73b41ea640 1 -- 172.21.15.5:0/2229529233 wait complete. 2024-05-20T18:29:52.534 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-05-20T18:29:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.650+0000 7f7eac796640 1 Processor -- start 2024-05-20T18:29:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.650+0000 7f7eac796640 1 -- start start 2024-05-20T18:29:52.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.651+0000 7f7eac796640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ea40b47b0 0x7f7ea40afad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:52.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.651+0000 7f7eac796640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7ea415d060 0x7f7ea40b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:52.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.651+0000 7f7eac796640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ea40b5b30 con 0x7f7ea40b5720 2024-05-20T18:29:52.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.651+0000 7f7eac796640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f7ea40703e0 con 0x7f7ea415d060 2024-05-20T18:29:52.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.651+0000 7f7eac796640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea40b39f0 con 0x7f7ea40b47b0 2024-05-20T18:29:52.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.652+0000 7f7ea9d31640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7ea415d060 0x7f7ea40b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:52.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.652+0000 7f7ea9d31640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7ea415d060 0x7f7ea40b51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46626/0 (socket says 172.21.15.5:46626) 2024-05-20T18:29:52.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.652+0000 7f7eaad33640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7ea40b5720 0x7f7ea415fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37388/0 (socket says 172.21.15.5:37388) 2024-05-20T18:29:52.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.652+0000 7f7ea9d31640 1 -- 172.21.15.5:0/3368684129 learned_addr learned my addr 172.21.15.5:0/3368684129 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:52.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.652+0000 7f7eaa532640 1 --2- 172.21.15.5:0/3368684129 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ea40b47b0 0x7f7ea40afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:52.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.652+0000 7f7ea9d31640 1 -- 172.21.15.5:0/3368684129 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ea40b47b0 msgr2=0x7f7ea40afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:52.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.652+0000 7f7ea9d31640 1 --2- 172.21.15.5:0/3368684129 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ea40b47b0 0x7f7ea40afad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:52.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.652+0000 7f7ea9d31640 1 -- 172.21.15.5:0/3368684129 >> v1:172.21.15.5:6789/0 conn(0x7f7ea40b5720 legacy=0x7f7ea415fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:52.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.652+0000 7f7ea9d31640 1 -- 172.21.15.5:0/3368684129 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea40b3cd0 con 0x7f7ea415d060 2024-05-20T18:29:52.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.652+0000 7f7ea9530640 1 -- 172.21.15.5:0/3368684129 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1946668039 0 0) 0x7f7e9c001490 con 0x7f7ea40b5720 2024-05-20T18:29:52.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.652+0000 7f7eaa532640 1 --2- 172.21.15.5:0/3368684129 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ea40b47b0 0x7f7ea40afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:29:52.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.653+0000 7f7ea9d31640 1 --2- 172.21.15.5:0/3368684129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea415d060 0x7f7ea40b51e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009870 tx=0x7f7e9402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b6a83216de8846 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:52.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.653+0000 7f7ea9530640 1 -- 172.21.15.5:0/3368684129 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94037070 con 0x7f7ea415d060 2024-05-20T18:29:52.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.653+0000 7f7ea9530640 1 -- 172.21.15.5:0/3368684129 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e94032440 con 0x7f7ea415d060 2024-05-20T18:29:52.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.653+0000 7f7ea9530640 1 -- 172.21.15.5:0/3368684129 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94032810 con 0x7f7ea415d060 2024-05-20T18:29:52.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.654+0000 7f7eac796640 1 -- 172.21.15.5:0/3368684129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea415d060 msgr2=0x7f7ea40b51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:52.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.654+0000 7f7eac796640 1 --2- 172.21.15.5:0/3368684129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea415d060 0x7f7ea40b51e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009870 tx=0x7f7e9402a1c0 comp rx=0 tx=0).stop 2024-05-20T18:29:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.654+0000 7f7eac796640 1 -- 172.21.15.5:0/3368684129 shutdown_connections 2024-05-20T18:29:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.654+0000 7f7eac796640 1 --2- 172.21.15.5:0/3368684129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea415d060 0x7f7ea40b51e0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.654+0000 7f7eac796640 1 --2- 172.21.15.5:0/3368684129 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ea40b47b0 0x7f7ea40afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.654+0000 7f7eac796640 1 -- 172.21.15.5:0/3368684129 >> 172.21.15.5:0/3368684129 conn(0x7f7ea40cd2a0 msgr2=0x7f7ea40cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.654+0000 7f7eac796640 1 -- 172.21.15.5:0/3368684129 shutdown_connections 2024-05-20T18:29:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.654+0000 7f7eac796640 1 -- 172.21.15.5:0/3368684129 wait complete. 2024-05-20T18:29:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.656+0000 7f7eac796640 1 Processor -- start 2024-05-20T18:29:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.656+0000 7f7eac796640 1 -- start start 2024-05-20T18:29:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.657+0000 7f7eac796640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ea40b5720 0x7f7ea414a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.657+0000 7f7eac796640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ea415d060 0x7f7ea409b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.657+0000 7f7eac796640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea409c690 0x7f7ea414aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.657+0000 7f7eac796640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea40b0010 con 0x7f7ea409c690 2024-05-20T18:29:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.657+0000 7f7eac796640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea40b4290 con 0x7f7ea415d060 2024-05-20T18:29:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.657+0000 7f7eac796640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea4159c60 con 0x7f7ea40b5720 2024-05-20T18:29:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.657+0000 7f7eaad33640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea409c690 0x7f7ea414aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.657+0000 7f7eaad33640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea409c690 0x7f7ea414aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46630/0 (socket says 172.21.15.5:46630) 2024-05-20T18:29:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.657+0000 7f7eaad33640 1 -- 172.21.15.5:0/236838785 learned_addr learned my addr 172.21.15.5:0/236838785 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.657+0000 7f7ea9d31640 1 --2- 172.21.15.5:0/236838785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ea415d060 0x7f7ea409b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.657+0000 7f7eaa532640 1 --2- 172.21.15.5:0/236838785 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ea40b5720 0x7f7ea414a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.657+0000 7f7eaad33640 1 -- 172.21.15.5:0/236838785 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ea40b5720 msgr2=0x7f7ea414a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.657+0000 7f7eaad33640 1 --2- 172.21.15.5:0/236838785 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ea40b5720 0x7f7ea414a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.657+0000 7f7eaad33640 1 -- 172.21.15.5:0/236838785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ea415d060 msgr2=0x7f7ea409b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.657+0000 7f7eaad33640 1 --2- 172.21.15.5:0/236838785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ea415d060 0x7f7ea409b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.657+0000 7f7eaad33640 1 -- 172.21.15.5:0/236838785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea415ead0 con 0x7f7ea409c690 2024-05-20T18:29:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.658+0000 7f7eaad33640 1 --2- 172.21.15.5:0/236838785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea409c690 0x7f7ea414aa90 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00c6d0 tx=0x7f7e9c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.658+0000 7f7e9affd640 1 -- 172.21.15.5:0/236838785 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c018070 con 0x7f7ea409c690 2024-05-20T18:29:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.658+0000 7f7e9affd640 1 -- 172.21.15.5:0/236838785 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e9c004d60 con 0x7f7ea409c690 2024-05-20T18:29:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.658+0000 7f7eac796640 1 -- 172.21.15.5:0/236838785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea415edb0 con 0x7f7ea409c690 2024-05-20T18:29:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.658+0000 7f7e9affd640 1 -- 172.21.15.5:0/236838785 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c00a880 con 0x7f7ea409c690 2024-05-20T18:29:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.658+0000 7f7eac796640 1 -- 172.21.15.5:0/236838785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea414b560 con 0x7f7ea409c690 2024-05-20T18:29:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.659+0000 7f7eac796640 1 -- 172.21.15.5:0/236838785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ea4075b00 con 0x7f7ea409c690 2024-05-20T18:29:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.659+0000 7f7e9affd640 1 -- 172.21.15.5:0/236838785 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7e9c01b070 con 0x7f7ea409c690 2024-05-20T18:29:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.660+0000 7f7e9affd640 1 --2- 172.21.15.5:0/236838785 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7e8806be60 0x7f7e8806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.660+0000 7f7e9affd640 1 -- 172.21.15.5:0/236838785 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5335+0+0 (secure 0 0 0) 0x7f7e9c08a380 con 0x7f7ea409c690 2024-05-20T18:29:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.660+0000 7f7eaa532640 1 --2- 172.21.15.5:0/236838785 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7e8806be60 0x7f7e8806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.660+0000 7f7eaa532640 1 --2- 172.21.15.5:0/236838785 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7e8806be60 0x7f7e8806e370 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7ea406cf20 tx=0x7f7ea0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.665+0000 7f7e9affd640 1 -- 172.21.15.5:0/236838785 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7ea40b5b30 con 0x7f7ea409c690 2024-05-20T18:29:52.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:52.918+0000 7f7eac796640 1 -- 172.21.15.5:0/236838785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7f7ea415ef80 con 0x7f7ea409c690 2024-05-20T18:29:53.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:53.864+0000 7f7e9affd640 1 -- 172.21.15.5:0/236838785 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v160) ==== 172+0+0 (secure 0 0 0) 0x7f7e9c058850 con 0x7f7ea409c690 2024-05-20T18:29:53.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:53.867+0000 7f7eac796640 1 -- 172.21.15.5:0/236838785 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7e8806be60 msgr2=0x7f7e8806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:53.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:53.867+0000 7f7eac796640 1 --2- 172.21.15.5:0/236838785 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7e8806be60 0x7f7e8806e370 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7ea406cf20 tx=0x7f7ea0008040 comp rx=0 tx=0).stop 2024-05-20T18:29:53.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:53.867+0000 7f7eac796640 1 -- 172.21.15.5:0/236838785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea409c690 msgr2=0x7f7ea414aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:53.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:53.867+0000 7f7eac796640 1 --2- 172.21.15.5:0/236838785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea409c690 0x7f7ea414aa90 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00c6d0 tx=0x7f7e9c00cb90 comp rx=0 tx=0).stop 2024-05-20T18:29:53.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:53.867+0000 7f7eac796640 1 -- 172.21.15.5:0/236838785 shutdown_connections 2024-05-20T18:29:53.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:53.867+0000 7f7eac796640 1 --2- 172.21.15.5:0/236838785 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7e8806be60 0x7f7e8806e370 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:53.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:53.867+0000 7f7eac796640 1 --2- 172.21.15.5:0/236838785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea409c690 0x7f7ea414aa90 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:53.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:53.867+0000 7f7eac796640 1 --2- 172.21.15.5:0/236838785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ea415d060 0x7f7ea409b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:53.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:53.867+0000 7f7eac796640 1 --2- 172.21.15.5:0/236838785 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ea40b5720 0x7f7ea414a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:53.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:53.867+0000 7f7eac796640 1 -- 172.21.15.5:0/236838785 >> 172.21.15.5:0/236838785 conn(0x7f7ea40cd2a0 msgr2=0x7f7ea40af840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:53.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:53.867+0000 7f7eac796640 1 -- 172.21.15.5:0/236838785 shutdown_connections 2024-05-20T18:29:53.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:53.867+0000 7f7eac796640 1 -- 172.21.15.5:0/236838785 wait complete. 2024-05-20T18:29:53.879 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-20T18:29:54.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.001+0000 7ff81c77e640 1 Processor -- start 2024-05-20T18:29:54.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.001+0000 7ff81c77e640 1 -- start start 2024-05-20T18:29:54.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.003+0000 7ff81c77e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff814168050 0x7ff81416a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:54.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.003+0000 7ff81c77e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8140d0630 con 0x7ff81406d3d0 2024-05-20T18:29:54.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.003+0000 7ff81c77e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8140d39c0 con 0x7ff814162820 2024-05-20T18:29:54.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.003+0000 7ff81c77e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff8140d3ca0 con 0x7ff814168050 2024-05-20T18:29:54.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.003+0000 7ff819d19640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff81406d3d0 0x7ff81406d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37400/0 (socket says 172.21.15.5:37400) 2024-05-20T18:29:54.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.003+0000 7ff819d19640 1 -- 172.21.15.5:0/2359384054 learned_addr learned my addr 172.21.15.5:0/2359384054 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:54.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.003+0000 7ff81ad1b640 1 --2- 172.21.15.5:0/2359384054 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff814168050 0x7ff81416a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:54.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.003+0000 7ff819518640 1 -- 172.21.15.5:0/2359384054 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4054380693 0 0) 0x7ff8140d0630 con 0x7ff81406d3d0 2024-05-20T18:29:54.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.003+0000 7ff819518640 1 -- 172.21.15.5:0/2359384054 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff800003520 con 0x7ff81406d3d0 2024-05-20T18:29:54.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.003+0000 7ff819518640 1 -- 172.21.15.5:0/2359384054 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2743515443 0 0) 0x7ff8140d39c0 con 0x7ff814162820 2024-05-20T18:29:54.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.003+0000 7ff819518640 1 -- 172.21.15.5:0/2359384054 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8140d0630 con 0x7ff814162820 2024-05-20T18:29:54.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.003+0000 7ff819518640 1 -- 172.21.15.5:0/2359384054 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 137160332 0 0) 0x7ff800003520 con 0x7ff81406d3d0 2024-05-20T18:29:54.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.003+0000 7ff81ad1b640 1 -- 172.21.15.5:0/2359384054 >> v1:172.21.15.63:6789/0 conn(0x7ff814162820 legacy=0x7ff8140b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:54.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.003+0000 7ff81ad1b640 1 -- 172.21.15.5:0/2359384054 >> v1:172.21.15.5:6789/0 conn(0x7ff81406d3d0 legacy=0x7ff81406d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:54.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.004+0000 7ff81ad1b640 1 -- 172.21.15.5:0/2359384054 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8140d3f80 con 0x7ff814168050 2024-05-20T18:29:54.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.004+0000 7ff81ad1b640 1 --2- 172.21.15.5:0/2359384054 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff814168050 0x7ff81416a470 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff80c0097d0 tx=0x7ff80c02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98fd1f51c0346c2d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:54.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.005+0000 7ff819518640 1 -- 172.21.15.5:0/2359384054 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80c037070 con 0x7ff814168050 2024-05-20T18:29:54.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.005+0000 7ff819518640 1 -- 172.21.15.5:0/2359384054 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff80c02ac30 con 0x7ff814168050 2024-05-20T18:29:54.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.005+0000 7ff819518640 1 -- 172.21.15.5:0/2359384054 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff804001670 con 0x7ff814168050 2024-05-20T18:29:54.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.005+0000 7ff81c77e640 1 -- 172.21.15.5:0/2359384054 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff814168050 msgr2=0x7ff81416a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:54.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.005+0000 7ff81c77e640 1 --2- 172.21.15.5:0/2359384054 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff814168050 0x7ff81416a470 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff80c0097d0 tx=0x7ff80c02a1a0 comp rx=0 tx=0).stop 2024-05-20T18:29:54.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.005+0000 7ff81c77e640 1 -- 172.21.15.5:0/2359384054 shutdown_connections 2024-05-20T18:29:54.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.005+0000 7ff81c77e640 1 --2- 172.21.15.5:0/2359384054 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff814168050 0x7ff81416a470 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:54.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.005+0000 7ff81c77e640 1 -- 172.21.15.5:0/2359384054 >> 172.21.15.5:0/2359384054 conn(0x7ff8140d4bd0 msgr2=0x7ff8140d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:54.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.005+0000 7ff81c77e640 1 -- 172.21.15.5:0/2359384054 shutdown_connections 2024-05-20T18:29:54.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.005+0000 7ff81c77e640 1 -- 172.21.15.5:0/2359384054 wait complete. 2024-05-20T18:29:54.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.007+0000 7ff81c77e640 1 Processor -- start 2024-05-20T18:29:54.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.007+0000 7ff81c77e640 1 -- start start 2024-05-20T18:29:54.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.008+0000 7ff81c77e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff81406d3d0 0x7ff814151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:54.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.008+0000 7ff81c77e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff814162820 0x7ff81415b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:54.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.008+0000 7ff81c77e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff814168050 0x7ff81415be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:54.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.008+0000 7ff81c77e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff8140b91a0 con 0x7ff814162820 2024-05-20T18:29:54.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.008+0000 7ff81c77e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff8140d0300 con 0x7ff81406d3d0 2024-05-20T18:29:54.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.008+0000 7ff81c77e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff8140b4050 con 0x7ff814168050 2024-05-20T18:29:54.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.008+0000 7ff819d19640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff814162820 0x7ff81415b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:54.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.009+0000 7ff819d19640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff814162820 0x7ff81415b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46634/0 (socket says 172.21.15.5:46634) 2024-05-20T18:29:54.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.009+0000 7ff819d19640 1 -- 172.21.15.5:0/4279580865 learned_addr learned my addr 172.21.15.5:0/4279580865 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:54.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.009+0000 7ff81ad1b640 1 --2- 172.21.15.5:0/4279580865 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff814168050 0x7ff81415be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:54.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.009+0000 7ff81a51a640 1 --2- 172.21.15.5:0/4279580865 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff81406d3d0 0x7ff814151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:54.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.009+0000 7ff819d19640 1 -- 172.21.15.5:0/4279580865 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff814168050 msgr2=0x7ff81415be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:54.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.009+0000 7ff819d19640 1 --2- 172.21.15.5:0/4279580865 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff814168050 0x7ff81415be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:54.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.009+0000 7ff819d19640 1 -- 172.21.15.5:0/4279580865 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff81406d3d0 msgr2=0x7ff814151690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:54.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.009+0000 7ff819d19640 1 --2- 172.21.15.5:0/4279580865 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff81406d3d0 0x7ff814151690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:54.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.009+0000 7ff819d19640 1 -- 172.21.15.5:0/4279580865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8140b4550 con 0x7ff814162820 2024-05-20T18:29:54.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.009+0000 7ff819d19640 1 --2- 172.21.15.5:0/4279580865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff814162820 0x7ff81415b8d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff80400c780 tx=0x7ff80400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:54.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.009+0000 7ff80affd640 1 -- 172.21.15.5:0/4279580865 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff804005ad0 con 0x7ff814162820 2024-05-20T18:29:54.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.009+0000 7ff80affd640 1 -- 172.21.15.5:0/4279580865 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff804005c70 con 0x7ff814162820 2024-05-20T18:29:54.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.009+0000 7ff81c77e640 1 -- 172.21.15.5:0/4279580865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff81415f370 con 0x7ff814162820 2024-05-20T18:29:54.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.009+0000 7ff80affd640 1 -- 172.21.15.5:0/4279580865 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8040072b0 con 0x7ff814162820 2024-05-20T18:29:54.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.009+0000 7ff81c77e640 1 -- 172.21.15.5:0/4279580865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff81415f930 con 0x7ff814162820 2024-05-20T18:29:54.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.010+0000 7ff81c77e640 1 -- 172.21.15.5:0/4279580865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8140b7300 con 0x7ff814162820 2024-05-20T18:29:54.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.011+0000 7ff80affd640 1 -- 172.21.15.5:0/4279580865 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff804007450 con 0x7ff814162820 2024-05-20T18:29:54.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.011+0000 7ff80affd640 1 --2- 172.21.15.5:0/4279580865 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff7f806be60 0x7ff7f806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:54.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.012+0000 7ff80affd640 1 -- 172.21.15.5:0/4279580865 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5335+0+0 (secure 0 0 0) 0x7ff80408a9d0 con 0x7ff814162820 2024-05-20T18:29:54.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.012+0000 7ff81a51a640 1 --2- 172.21.15.5:0/4279580865 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff7f806be60 0x7ff7f806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:54.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.012+0000 7ff81a51a640 1 --2- 172.21.15.5:0/4279580865 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff7f806be60 0x7ff7f806e370 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff810001e00 tx=0x7ff810009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:54.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.016+0000 7ff80affd640 1 -- 172.21.15.5:0/4279580865 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff804058ed0 con 0x7ff814162820 2024-05-20T18:29:54.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.197+0000 7ff81c77e640 1 -- 172.21.15.5:0/4279580865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff814075b00 con 0x7ff814162820 2024-05-20T18:29:54.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.201+0000 7ff80affd640 1 -- 172.21.15.5:0/4279580865 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7ff804005e20 con 0x7ff814162820 2024-05-20T18:29:54.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.203+0000 7ff81c77e640 1 -- 172.21.15.5:0/4279580865 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff7f806be60 msgr2=0x7ff7f806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:54.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.203+0000 7ff81c77e640 1 --2- 172.21.15.5:0/4279580865 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff7f806be60 0x7ff7f806e370 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff810001e00 tx=0x7ff810009040 comp rx=0 tx=0).stop 2024-05-20T18:29:54.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.203+0000 7ff81c77e640 1 -- 172.21.15.5:0/4279580865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff814162820 msgr2=0x7ff81415b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:54.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.203+0000 7ff81c77e640 1 --2- 172.21.15.5:0/4279580865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff814162820 0x7ff81415b8d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff80400c780 tx=0x7ff80400cc40 comp rx=0 tx=0).stop 2024-05-20T18:29:54.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.203+0000 7ff81c77e640 1 -- 172.21.15.5:0/4279580865 shutdown_connections 2024-05-20T18:29:54.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.203+0000 7ff81c77e640 1 --2- 172.21.15.5:0/4279580865 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff7f806be60 0x7ff7f806e370 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:54.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.203+0000 7ff81c77e640 1 --2- 172.21.15.5:0/4279580865 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff814168050 0x7ff81415be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:54.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.203+0000 7ff81c77e640 1 --2- 172.21.15.5:0/4279580865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff814162820 0x7ff81415b8d0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:54.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.203+0000 7ff81c77e640 1 --2- 172.21.15.5:0/4279580865 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff81406d3d0 0x7ff814151690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:54.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.203+0000 7ff81c77e640 1 -- 172.21.15.5:0/4279580865 >> 172.21.15.5:0/4279580865 conn(0x7ff8140d4bd0 msgr2=0x7ff814161a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:54.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.203+0000 7ff81c77e640 1 -- 172.21.15.5:0/4279580865 shutdown_connections 2024-05-20T18:29:54.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.203+0000 7ff81c77e640 1 -- 172.21.15.5:0/4279580865 wait complete. 2024-05-20T18:29:54.214 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-20T18:29:54.214 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T18:29:54.252 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T18:29:54.297 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-20T18:29:54.298 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:29:54.298 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:29:54.314 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15663 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T18:29:54.314 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:29:54.314 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:29:54.333 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19732 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T18:29:54.333 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:29:54.333 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:29:54.371 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15663 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T18:29:54.371 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:29:54.371 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:29:54.392 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19732 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T18:29:54.392 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-20T18:29:54.392 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:29:54.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.525+0000 7f50f8606640 1 Processor -- start 2024-05-20T18:29:54.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.525+0000 7f50f8606640 1 -- start start 2024-05-20T18:29:54.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.527+0000 7f50f8606640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f50f0074eb0 0x7f50f00af670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:54.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.527+0000 7f50f8606640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50f00affc0 0x7f50f00b1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:54.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.527+0000 7f50f8606640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50f006c690 con 0x7f50f00afbb0 2024-05-20T18:29:54.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.527+0000 7f50f8606640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f50f006c440 con 0x7f50f0074eb0 2024-05-20T18:29:54.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.527+0000 7f50f8606640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f50f006c970 con 0x7f50f00affc0 2024-05-20T18:29:54.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.527+0000 7f50f63a2640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f50f0074eb0 0x7f50f00af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:54.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.527+0000 7f50f5ba1640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f50f00afbb0 0x7f50f00b0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37412/0 (socket says 172.21.15.5:37412) 2024-05-20T18:29:54.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.527+0000 7f50f5ba1640 1 -- 172.21.15.5:0/3684819415 learned_addr learned my addr 172.21.15.5:0/3684819415 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:54.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.527+0000 7f50f6ba3640 1 --2- 172.21.15.5:0/3684819415 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50f00affc0 0x7f50f00b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:54.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.528+0000 7f50f63a2640 1 -- 172.21.15.5:0/3684819415 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50f00affc0 msgr2=0x7f50f00b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:54.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.528+0000 7f50f63a2640 1 --2- 172.21.15.5:0/3684819415 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50f00affc0 0x7f50f00b1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:54.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.528+0000 7f50f63a2640 1 -- 172.21.15.5:0/3684819415 >> v1:172.21.15.5:6789/0 conn(0x7f50f00afbb0 legacy=0x7f50f00b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:54.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.528+0000 7f50f63a2640 1 -- 172.21.15.5:0/3684819415 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f006cc50 con 0x7f50f0074eb0 2024-05-20T18:29:54.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.528+0000 7f50f63a2640 1 --2- 172.21.15.5:0/3684819415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50f0074eb0 0x7f50f00af670 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f50ec0097a0 tx=0x7f50ec02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b91c49c8ff8b9f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:54.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.528+0000 7f50f53a0640 1 -- 172.21.15.5:0/3684819415 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec037070 con 0x7f50f0074eb0 2024-05-20T18:29:54.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.529+0000 7f50f53a0640 1 -- 172.21.15.5:0/3684819415 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50ec032440 con 0x7f50f0074eb0 2024-05-20T18:29:54.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.529+0000 7f50f53a0640 1 -- 172.21.15.5:0/3684819415 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec032810 con 0x7f50f0074eb0 2024-05-20T18:29:54.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.529+0000 7f50f8606640 1 -- 172.21.15.5:0/3684819415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50f0074eb0 msgr2=0x7f50f00af670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:54.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.529+0000 7f50f8606640 1 --2- 172.21.15.5:0/3684819415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50f0074eb0 0x7f50f00af670 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f50ec0097a0 tx=0x7f50ec02a180 comp rx=0 tx=0).stop 2024-05-20T18:29:54.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.529+0000 7f50f8606640 1 -- 172.21.15.5:0/3684819415 shutdown_connections 2024-05-20T18:29:54.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.529+0000 7f50f8606640 1 --2- 172.21.15.5:0/3684819415 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50f00affc0 0x7f50f00b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:54.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.529+0000 7f50f8606640 1 --2- 172.21.15.5:0/3684819415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50f0074eb0 0x7f50f00af670 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:54.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.529+0000 7f50f8606640 1 -- 172.21.15.5:0/3684819415 >> 172.21.15.5:0/3684819415 conn(0x7f50f0072ed0 msgr2=0x7f50f00b6080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:54.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.529+0000 7f50f8606640 1 -- 172.21.15.5:0/3684819415 shutdown_connections 2024-05-20T18:29:54.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.529+0000 7f50f8606640 1 -- 172.21.15.5:0/3684819415 wait complete. 2024-05-20T18:29:54.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.531+0000 7f50f8606640 1 Processor -- start 2024-05-20T18:29:54.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.531+0000 7f50f8606640 1 -- start start 2024-05-20T18:29:54.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.533+0000 7f50f8606640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50f0074eb0 0x7f50f014c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:54.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.533+0000 7f50f8606640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50f00afbb0 0x7f50f00cd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:54.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.533+0000 7f50f8606640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50f00affc0 0x7f50f00cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:54.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.533+0000 7f50f8606640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50f00b0510 con 0x7f50f00affc0 2024-05-20T18:29:54.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.533+0000 7f50f8606640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f50f006b730 con 0x7f50f00afbb0 2024-05-20T18:29:54.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.533+0000 7f50f8606640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f50f00b0690 con 0x7f50f0074eb0 2024-05-20T18:29:54.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.533+0000 7f50f6ba3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50f00affc0 0x7f50f00cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:54.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.533+0000 7f50f6ba3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50f00affc0 0x7f50f00cdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46654/0 (socket says 172.21.15.5:46654) 2024-05-20T18:29:54.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.533+0000 7f50f6ba3640 1 -- 172.21.15.5:0/2077709803 learned_addr learned my addr 172.21.15.5:0/2077709803 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:54.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.533+0000 7f50f63a2640 1 --2- 172.21.15.5:0/2077709803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50f0074eb0 0x7f50f014c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:54.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.533+0000 7f50f5ba1640 1 --2- 172.21.15.5:0/2077709803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50f00afbb0 0x7f50f00cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:54.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.533+0000 7f50f6ba3640 1 -- 172.21.15.5:0/2077709803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50f0074eb0 msgr2=0x7f50f014c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:54.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.533+0000 7f50f6ba3640 1 --2- 172.21.15.5:0/2077709803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50f0074eb0 0x7f50f014c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:54.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.533+0000 7f50f6ba3640 1 -- 172.21.15.5:0/2077709803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50f00afbb0 msgr2=0x7f50f00cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:54.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.533+0000 7f50f6ba3640 1 --2- 172.21.15.5:0/2077709803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50f00afbb0 0x7f50f00cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:54.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.533+0000 7f50f6ba3640 1 -- 172.21.15.5:0/2077709803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f00ce3f0 con 0x7f50f00affc0 2024-05-20T18:29:54.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.533+0000 7f50f5ba1640 1 --2- 172.21.15.5:0/2077709803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50f00afbb0 0x7f50f00cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:29:54.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.533+0000 7f50f6ba3640 1 --2- 172.21.15.5:0/2077709803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50f00affc0 0x7f50f00cdeb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f50e8007d40 tx=0x7f50e800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:54.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.533+0000 7f50e6ffd640 1 -- 172.21.15.5:0/2077709803 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e8014420 con 0x7f50f00affc0 2024-05-20T18:29:54.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.533+0000 7f50e6ffd640 1 -- 172.21.15.5:0/2077709803 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50e8019070 con 0x7f50f00affc0 2024-05-20T18:29:54.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.534+0000 7f50f8606640 1 -- 172.21.15.5:0/2077709803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f0153950 con 0x7f50f00affc0 2024-05-20T18:29:54.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.534+0000 7f50e6ffd640 1 -- 172.21.15.5:0/2077709803 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e8014820 con 0x7f50f00affc0 2024-05-20T18:29:54.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.534+0000 7f50f8606640 1 -- 172.21.15.5:0/2077709803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f0153f10 con 0x7f50f00affc0 2024-05-20T18:29:54.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.535+0000 7f50f8606640 1 -- 172.21.15.5:0/2077709803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50f0075b00 con 0x7f50f00affc0 2024-05-20T18:29:54.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.535+0000 7f50e6ffd640 1 -- 172.21.15.5:0/2077709803 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f50e80149c0 con 0x7f50f00affc0 2024-05-20T18:29:54.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.536+0000 7f50e6ffd640 1 --2- 172.21.15.5:0/2077709803 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f50d406be30 0x7f50d406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:54.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.536+0000 7f50e6ffd640 1 -- 172.21.15.5:0/2077709803 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5335+0+0 (secure 0 0 0) 0x7f50e808b450 con 0x7f50f00affc0 2024-05-20T18:29:54.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.536+0000 7f50f63a2640 1 --2- 172.21.15.5:0/2077709803 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f50d406be30 0x7f50d406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:54.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.536+0000 7f50f63a2640 1 --2- 172.21.15.5:0/2077709803 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f50d406be30 0x7f50d406e340 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f50ec034000 tx=0x7f50ec02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:54.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.540+0000 7f50e6ffd640 1 -- 172.21.15.5:0/2077709803 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f50e8059f70 con 0x7f50f00affc0 2024-05-20T18:29:54.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.766+0000 7f50f8606640 1 -- 172.21.15.5:0/2077709803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50f009a900 con 0x7f50f00affc0 2024-05-20T18:29:54.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.766+0000 7f50e6ffd640 1 -- 172.21.15.5:0/2077709803 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+3863 (secure 0 0 0) 0x7f50e8059870 con 0x7f50f00affc0 2024-05-20T18:29:54.767 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:29:54.767 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5207,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3958852803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3958852803},{"type":"v1","addr":"172.21.15.63:6835","nonce":3958852803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5560,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/3020169757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":3020169757},{"type":"v1","addr":"172.21.15.63:6834","nonce":3020169757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2965018210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2965018210},{"type":"v1","addr":"172.21.15.5:6835","nonce":2965018210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":6102,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2162786657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2162786657},{"type":"v1","addr":"172.21.15.5:6837","nonce":2162786657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[]} 2024-05-20T18:29:54.767 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-05-20T18:29:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.768+0000 7f50f8606640 1 -- 172.21.15.5:0/2077709803 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f50d406be30 msgr2=0x7f50d406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.768+0000 7f50f8606640 1 --2- 172.21.15.5:0/2077709803 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f50d406be30 0x7f50d406e340 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f50ec034000 tx=0x7f50ec02e000 comp rx=0 tx=0).stop 2024-05-20T18:29:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.768+0000 7f50f8606640 1 -- 172.21.15.5:0/2077709803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50f00affc0 msgr2=0x7f50f00cdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.768+0000 7f50f8606640 1 --2- 172.21.15.5:0/2077709803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50f00affc0 0x7f50f00cdeb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f50e8007d40 tx=0x7f50e800e4b0 comp rx=0 tx=0).stop 2024-05-20T18:29:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.768+0000 7f50f8606640 1 -- 172.21.15.5:0/2077709803 shutdown_connections 2024-05-20T18:29:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.768+0000 7f50f8606640 1 --2- 172.21.15.5:0/2077709803 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f50d406be30 0x7f50d406e340 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.768+0000 7f50f8606640 1 --2- 172.21.15.5:0/2077709803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50f00affc0 0x7f50f00cdeb0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.768+0000 7f50f8606640 1 --2- 172.21.15.5:0/2077709803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50f00afbb0 0x7f50f00cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.768+0000 7f50f8606640 1 --2- 172.21.15.5:0/2077709803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50f0074eb0 0x7f50f014c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.768+0000 7f50f8606640 1 -- 172.21.15.5:0/2077709803 >> 172.21.15.5:0/2077709803 conn(0x7f50f0072ed0 msgr2=0x7f50f006ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.768+0000 7f50f8606640 1 -- 172.21.15.5:0/2077709803 shutdown_connections 2024-05-20T18:29:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.768+0000 7f50f8606640 1 -- 172.21.15.5:0/2077709803 wait complete. 2024-05-20T18:29:54.782 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-20T18:29:54.782 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-20T18:29:54.783 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-20T18:29:54.783 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:29:54.783 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-20T18:29:54.783 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-20T18:29:54.784 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-20T18:29:54.784 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:29:54.784 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-20T18:29:54.784 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-20T18:29:54.785 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-20T18:29:54.785 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:29:54.785 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-20T18:29:54.785 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-20T18:29:54.785 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-20T18:29:54.786 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:29:54.793 INFO:tasks.ceph.mds.a:Stopped 2024-05-20T18:29:54.793 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-20T18:29:54.798 INFO:tasks.ceph.mds.b:Stopped 2024-05-20T18:29:54.798 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-20T18:29:54.799 INFO:tasks.ceph.mds.d:Stopped 2024-05-20T18:29:54.799 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-20T18:29:54.800 INFO:tasks.ceph.mds.c:Stopped 2024-05-20T18:29:54.800 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-20T18:29:54.841 INFO:tasks.ceph.mds.b:Started 2024-05-20T18:29:54.842 INFO:tasks.ceph.mds.d:Started 2024-05-20T18:29:54.842 DEBUG:teuthology.parallel:result is None 2024-05-20T18:29:54.842 DEBUG:teuthology.parallel:result is None 2024-05-20T18:29:54.843 INFO:tasks.ceph.mds.a:Started 2024-05-20T18:29:54.843 DEBUG:teuthology.parallel:result is None 2024-05-20T18:29:54.844 INFO:tasks.ceph.mds.c:Started 2024-05-20T18:29:54.844 DEBUG:teuthology.parallel:result is None 2024-05-20T18:29:54.845 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-20T18:29:54.934 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-05-20T18:29:54.934 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-05-20T18:29:54.940 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-20T18:29:54.940 INFO:tasks.ceph.mds.d.smithi063.stdout:starting mds.d at 2024-05-20T18:29:54.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.966+0000 7f8b3fede640 1 Processor -- start 2024-05-20T18:29:54.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.967+0000 7f8b3fede640 1 -- start start 2024-05-20T18:29:54.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.968+0000 7f8b3fede640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8b380d10e0 0x7f8b380cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:54.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.968+0000 7f8b3fede640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8b380d14f0 0x7f8b380cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:54.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.968+0000 7f8b3fede640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b380cd4d0 0x7f8b380cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:54.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.968+0000 7f8b3fede640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f8b380d1ac0 con 0x7f8b380d10e0 2024-05-20T18:29:54.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.968+0000 7f8b3fede640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8b380d1da0 con 0x7f8b380d14f0 2024-05-20T18:29:54.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.968+0000 7f8b3fede640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8b380d2080 con 0x7f8b380cd4d0 2024-05-20T18:29:54.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.968+0000 7f8b3dc7a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8b380d10e0 0x7f8b380cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:54.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.968+0000 7f8b3dc7a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8b380d10e0 0x7f8b380cf140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33744/0 (socket says 172.21.15.5:33744) 2024-05-20T18:29:54.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.968+0000 7f8b3dc7a640 1 -- 172.21.15.5:0/232171605 learned_addr learned my addr 172.21.15.5:0/232171605 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:54.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.968+0000 7f8b3d479640 1 --2- 172.21.15.5:0/232171605 >> v2:172.21.15.63:3300/0 conn(0x7f8b380d14f0 0x7f8b380cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:54.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.968+0000 7f8b3e47b640 1 --2- 172.21.15.5:0/232171605 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b380cd4d0 0x7f8b380cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:54.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.969+0000 7f8b3e47b640 1 -- 172.21.15.5:0/232171605 >> v2:172.21.15.63:3300/0 conn(0x7f8b380d14f0 msgr2=0x7f8b380cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:54.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.969+0000 7f8b3e47b640 1 --2- 172.21.15.5:0/232171605 >> v2:172.21.15.63:3300/0 conn(0x7f8b380d14f0 0x7f8b380cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:54.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.969+0000 7f8b3e47b640 1 -- 172.21.15.5:0/232171605 >> v2:172.21.15.5:3300/0 conn(0x7f8b380d10e0 msgr2=0x7f8b380cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:54.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.969+0000 7f8b3e47b640 1 --2- 172.21.15.5:0/232171605 >> v2:172.21.15.5:3300/0 conn(0x7f8b380d10e0 0x7f8b380cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:54.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.969+0000 7f8b3e47b640 1 -- 172.21.15.5:0/232171605 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b380d2360 con 0x7f8b380cd4d0 2024-05-20T18:29:54.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.969+0000 7f8b3d479640 1 --2- 172.21.15.5:0/232171605 >> v2:172.21.15.63:3300/0 conn(0x7f8b380d14f0 0x7f8b380cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:29:54.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.969+0000 7f8b3dc7a640 1 --2- 172.21.15.5:0/232171605 >> v2:172.21.15.5:3300/0 conn(0x7f8b380d10e0 0x7f8b380cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:29:54.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.970+0000 7f8b3e47b640 1 --2- 172.21.15.5:0/232171605 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b380cd4d0 0x7f8b380cd8e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8b3808bee0 tx=0x7f8b3002a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f0f277a5b95cc7b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:54.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.970+0000 7f8b3cc78640 1 -- 172.21.15.5:0/232171605 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b30037070 con 0x7f8b380cd4d0 2024-05-20T18:29:54.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.970+0000 7f8b3cc78640 1 -- 172.21.15.5:0/232171605 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b30032440 con 0x7f8b380cd4d0 2024-05-20T18:29:54.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.970+0000 7f8b3cc78640 1 -- 172.21.15.5:0/232171605 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3002ae60 con 0x7f8b380cd4d0 2024-05-20T18:29:54.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.970+0000 7f8b3fede640 1 -- 172.21.15.5:0/232171605 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b380cd4d0 msgr2=0x7f8b380cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:54.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.970+0000 7f8b3fede640 1 --2- 172.21.15.5:0/232171605 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b380cd4d0 0x7f8b380cd8e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8b3808bee0 tx=0x7f8b3002a110 comp rx=0 tx=0).stop 2024-05-20T18:29:54.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.971+0000 7f8b3fede640 1 -- 172.21.15.5:0/232171605 shutdown_connections 2024-05-20T18:29:54.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.971+0000 7f8b3fede640 1 --2- 172.21.15.5:0/232171605 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b380cd4d0 0x7f8b380cd8e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:54.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.971+0000 7f8b3fede640 1 --2- 172.21.15.5:0/232171605 >> v2:172.21.15.63:3300/0 conn(0x7f8b380d14f0 0x7f8b380cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:54.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.971+0000 7f8b3fede640 1 --2- 172.21.15.5:0/232171605 >> v2:172.21.15.5:3300/0 conn(0x7f8b380d10e0 0x7f8b380cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:54.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.971+0000 7f8b3fede640 1 -- 172.21.15.5:0/232171605 >> 172.21.15.5:0/232171605 conn(0x7f8b380b44d0 msgr2=0x7f8b380b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:54.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.971+0000 7f8b3fede640 1 -- 172.21.15.5:0/232171605 shutdown_connections 2024-05-20T18:29:54.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.971+0000 7f8b3fede640 1 -- 172.21.15.5:0/232171605 wait complete. 2024-05-20T18:29:54.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.973+0000 7f8b3fede640 1 Processor -- start 2024-05-20T18:29:54.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.973+0000 7f8b3fede640 1 -- start start 2024-05-20T18:29:54.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.974+0000 7f8b3fede640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b380cd4d0 0x7f8b381f5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:54.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.974+0000 7f8b3fede640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b380d10e0 0x7f8b380cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:54.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.974+0000 7f8b3fede640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b380d14f0 0x7f8b380b6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:54.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.974+0000 7f8b3fede640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8b380b8440 con 0x7f8b380cd4d0 2024-05-20T18:29:54.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.974+0000 7f8b3dc7a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b380cd4d0 0x7f8b381f5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:54.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.974+0000 7f8b3fede640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8b380b82c0 con 0x7f8b380d10e0 2024-05-20T18:29:54.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.974+0000 7f8b3dc7a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b380cd4d0 0x7f8b381f5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33746/0 (socket says 172.21.15.5:33746) 2024-05-20T18:29:54.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.974+0000 7f8b3dc7a640 1 -- 172.21.15.5:0/4009203925 learned_addr learned my addr 172.21.15.5:0/4009203925 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:54.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.974+0000 7f8b3fede640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8b380709f0 con 0x7f8b380d14f0 2024-05-20T18:29:54.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.974+0000 7f8b3d479640 1 --2- 172.21.15.5:0/4009203925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b380d10e0 0x7f8b380cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:54.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.975+0000 7f8b3e47b640 1 --2- 172.21.15.5:0/4009203925 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b380d14f0 0x7f8b380b6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:54.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.975+0000 7f8b3dc7a640 1 -- 172.21.15.5:0/4009203925 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b380d14f0 msgr2=0x7f8b380b6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:54.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.975+0000 7f8b3dc7a640 1 --2- 172.21.15.5:0/4009203925 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b380d14f0 0x7f8b380b6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:54.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.975+0000 7f8b3dc7a640 1 -- 172.21.15.5:0/4009203925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b380d10e0 msgr2=0x7f8b380cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:54.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.975+0000 7f8b3dc7a640 1 --2- 172.21.15.5:0/4009203925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b380d10e0 0x7f8b380cc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:54.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.975+0000 7f8b3dc7a640 1 -- 172.21.15.5:0/4009203925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b380cc560 con 0x7f8b380cd4d0 2024-05-20T18:29:54.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.975+0000 7f8b3e47b640 1 --2- 172.21.15.5:0/4009203925 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b380d14f0 0x7f8b380b6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:29:54.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.975+0000 7f8b3d479640 1 --2- 172.21.15.5:0/4009203925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b380d10e0 0x7f8b380cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:29:54.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.975+0000 7f8b3dc7a640 1 --2- 172.21.15.5:0/4009203925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b380cd4d0 0x7f8b381f5d40 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009e80 tx=0x7f8b3400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:54.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.975+0000 7f8b2e7fc640 1 -- 172.21.15.5:0/4009203925 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3400ce60 con 0x7f8b380cd4d0 2024-05-20T18:29:54.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.975+0000 7f8b2e7fc640 1 -- 172.21.15.5:0/4009203925 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b34019070 con 0x7f8b380cd4d0 2024-05-20T18:29:54.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.975+0000 7f8b3fede640 1 -- 172.21.15.5:0/4009203925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b38158200 con 0x7f8b380cd4d0 2024-05-20T18:29:54.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.975+0000 7f8b2e7fc640 1 -- 172.21.15.5:0/4009203925 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b34014680 con 0x7f8b380cd4d0 2024-05-20T18:29:54.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.975+0000 7f8b3fede640 1 -- 172.21.15.5:0/4009203925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b381587c0 con 0x7f8b380cd4d0 2024-05-20T18:29:54.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.976+0000 7f8b3fede640 1 -- 172.21.15.5:0/4009203925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b38158e80 con 0x7f8b380cd4d0 2024-05-20T18:29:54.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.977+0000 7f8b2e7fc640 1 -- 172.21.15.5:0/4009203925 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8b34005d10 con 0x7f8b380cd4d0 2024-05-20T18:29:54.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.977+0000 7f8b2e7fc640 1 --2- 172.21.15.5:0/4009203925 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8b1c06be30 0x7f8b1c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:54.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.977+0000 7f8b2e7fc640 1 -- 172.21.15.5:0/4009203925 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5335+0+0 (secure 0 0 0) 0x7f8b3408b270 con 0x7f8b380cd4d0 2024-05-20T18:29:54.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.977+0000 7f8b3d479640 1 --2- 172.21.15.5:0/4009203925 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8b1c06be30 0x7f8b1c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:54.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.978+0000 7f8b3d479640 1 --2- 172.21.15.5:0/4009203925 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8b1c06be30 0x7f8b1c06e340 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8b38157720 tx=0x7f8b28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:54.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:54.982+0000 7f8b2e7fc640 1 -- 172.21.15.5:0/4009203925 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8b38158e80 con 0x7f8b380cd4d0 2024-05-20T18:29:55.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:55.177+0000 7f8b3fede640 1 -- 172.21.15.5:0/4009203925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f8b38075b00 con 0x7f8b380cd4d0 2024-05-20T18:29:56.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.215+0000 7f8b2e7fc640 1 -- 172.21.15.5:0/4009203925 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v59) ==== 97+0+0 (secure 0 0 0) 0x7f8b34010040 con 0x7f8b380cd4d0 2024-05-20T18:29:56.215 INFO:teuthology.orchestra.run.smithi005.stderr: removed all blocklist entries 2024-05-20T18:29:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.218+0000 7f8b3fede640 1 -- 172.21.15.5:0/4009203925 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8b1c06be30 msgr2=0x7f8b1c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.218+0000 7f8b3fede640 1 --2- 172.21.15.5:0/4009203925 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8b1c06be30 0x7f8b1c06e340 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8b38157720 tx=0x7f8b28008040 comp rx=0 tx=0).stop 2024-05-20T18:29:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.218+0000 7f8b3fede640 1 -- 172.21.15.5:0/4009203925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b380cd4d0 msgr2=0x7f8b381f5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.218+0000 7f8b3fede640 1 --2- 172.21.15.5:0/4009203925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b380cd4d0 0x7f8b381f5d40 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009e80 tx=0x7f8b3400c490 comp rx=0 tx=0).stop 2024-05-20T18:29:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.218+0000 7f8b3fede640 1 -- 172.21.15.5:0/4009203925 shutdown_connections 2024-05-20T18:29:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.218+0000 7f8b3fede640 1 --2- 172.21.15.5:0/4009203925 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8b1c06be30 0x7f8b1c06e340 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.218+0000 7f8b3fede640 1 --2- 172.21.15.5:0/4009203925 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b380d14f0 0x7f8b380b6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.218+0000 7f8b3fede640 1 --2- 172.21.15.5:0/4009203925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b380d10e0 0x7f8b380cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.218+0000 7f8b3fede640 1 --2- 172.21.15.5:0/4009203925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b380cd4d0 0x7f8b381f5d40 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.218+0000 7f8b3fede640 1 -- 172.21.15.5:0/4009203925 >> 172.21.15.5:0/4009203925 conn(0x7f8b380b44d0 msgr2=0x7f8b3815ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.218+0000 7f8b3fede640 1 -- 172.21.15.5:0/4009203925 shutdown_connections 2024-05-20T18:29:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.218+0000 7f8b3fede640 1 -- 172.21.15.5:0/4009203925 wait complete. 2024-05-20T18:29:56.233 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-20T18:29:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.356+0000 7fd696f5e640 1 Processor -- start 2024-05-20T18:29:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.356+0000 7fd696f5e640 1 -- start start 2024-05-20T18:29:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.357+0000 7fd696f5e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd690163a10 0x7fd6900d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.357+0000 7fd696f5e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6900d14b0 0x7fd6900d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.357+0000 7fd696f5e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6900d0370 con 0x7fd6900b4010 2024-05-20T18:29:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.357+0000 7fd696f5e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fd6900d3ca0 con 0x7fd690163a10 2024-05-20T18:29:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.357+0000 7fd696f5e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd6900d0650 con 0x7fd6900d14b0 2024-05-20T18:29:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.357+0000 7fd694cfa640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd690163a10 0x7fd6900d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.357+0000 7fd687fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd6900b4010 0x7fd6900cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:55146/0 (socket says 172.21.15.5:55146) 2024-05-20T18:29:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.357+0000 7fd687fff640 1 -- 172.21.15.5:0/1277212918 learned_addr learned my addr 172.21.15.5:0/1277212918 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.358+0000 7fd6954fb640 1 --2- 172.21.15.5:0/1277212918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6900d14b0 0x7fd6900d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:56.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.358+0000 7fd694cfa640 1 -- 172.21.15.5:0/1277212918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6900d14b0 msgr2=0x7fd6900d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:56.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.358+0000 7fd694cfa640 1 --2- 172.21.15.5:0/1277212918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6900d14b0 0x7fd6900d18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:56.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.358+0000 7fd694cfa640 1 -- 172.21.15.5:0/1277212918 >> v1:172.21.15.5:6789/0 conn(0x7fd6900b4010 legacy=0x7fd6900cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:56.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.358+0000 7fd6877fe640 1 -- 172.21.15.5:0/1277212918 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3971028990 0 0) 0x7fd6900d0370 con 0x7fd6900b4010 2024-05-20T18:29:56.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.358+0000 7fd694cfa640 1 -- 172.21.15.5:0/1277212918 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6900d3650 con 0x7fd690163a10 2024-05-20T18:29:56.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.358+0000 7fd6954fb640 1 --2- 172.21.15.5:0/1277212918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6900d14b0 0x7fd6900d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:29:56.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.359+0000 7fd694cfa640 1 --2- 172.21.15.5:0/1277212918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd690163a10 0x7fd6900d4130 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd68c0097a0 tx=0x7fd68c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7697be9feea5f8f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:56.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.359+0000 7fd6877fe640 1 -- 172.21.15.5:0/1277212918 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c037070 con 0x7fd690163a10 2024-05-20T18:29:56.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.359+0000 7fd6877fe640 1 -- 172.21.15.5:0/1277212918 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd68c032440 con 0x7fd690163a10 2024-05-20T18:29:56.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.359+0000 7fd6877fe640 1 -- 172.21.15.5:0/1277212918 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c032810 con 0x7fd690163a10 2024-05-20T18:29:56.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.359+0000 7fd696f5e640 1 -- 172.21.15.5:0/1277212918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd690163a10 msgr2=0x7fd6900d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:56.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.359+0000 7fd696f5e640 1 --2- 172.21.15.5:0/1277212918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd690163a10 0x7fd6900d4130 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd68c0097a0 tx=0x7fd68c02a180 comp rx=0 tx=0).stop 2024-05-20T18:29:56.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.360+0000 7fd696f5e640 1 -- 172.21.15.5:0/1277212918 shutdown_connections 2024-05-20T18:29:56.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.360+0000 7fd696f5e640 1 --2- 172.21.15.5:0/1277212918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6900d14b0 0x7fd6900d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:56.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.360+0000 7fd696f5e640 1 --2- 172.21.15.5:0/1277212918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd690163a10 0x7fd6900d4130 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:56.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.360+0000 7fd696f5e640 1 -- 172.21.15.5:0/1277212918 >> 172.21.15.5:0/1277212918 conn(0x7fd69006d240 msgr2=0x7fd69006d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:56.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.360+0000 7fd696f5e640 1 -- 172.21.15.5:0/1277212918 shutdown_connections 2024-05-20T18:29:56.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.360+0000 7fd696f5e640 1 -- 172.21.15.5:0/1277212918 wait complete. 2024-05-20T18:29:56.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.361+0000 7fd696f5e640 1 Processor -- start 2024-05-20T18:29:56.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.361+0000 7fd696f5e640 1 -- start start 2024-05-20T18:29:56.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.363+0000 7fd696f5e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6900b4010 0x7fd69015c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:56.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.363+0000 7fd696f5e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6900d14b0 0x7fd6900ab580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:56.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.363+0000 7fd696f5e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd690163a10 0x7fd6900aed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:56.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.363+0000 7fd696f5e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd6900b8210 con 0x7fd6900d14b0 2024-05-20T18:29:56.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.363+0000 7fd696f5e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd6900d1e00 con 0x7fd690163a10 2024-05-20T18:29:56.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.363+0000 7fd696f5e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd6900b8390 con 0x7fd6900b4010 2024-05-20T18:29:56.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.363+0000 7fd687fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6900d14b0 0x7fd6900ab580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:56.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.363+0000 7fd687fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6900d14b0 0x7fd6900ab580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33776/0 (socket says 172.21.15.5:33776) 2024-05-20T18:29:56.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.363+0000 7fd687fff640 1 -- 172.21.15.5:0/3960949722 learned_addr learned my addr 172.21.15.5:0/3960949722 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:56.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.363+0000 7fd694cfa640 1 --2- 172.21.15.5:0/3960949722 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6900b4010 0x7fd69015c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:56.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.363+0000 7fd6954fb640 1 --2- 172.21.15.5:0/3960949722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd690163a10 0x7fd6900aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:56.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.363+0000 7fd687fff640 1 -- 172.21.15.5:0/3960949722 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6900b4010 msgr2=0x7fd69015c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:56.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.363+0000 7fd687fff640 1 --2- 172.21.15.5:0/3960949722 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6900b4010 0x7fd69015c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:56.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.363+0000 7fd687fff640 1 -- 172.21.15.5:0/3960949722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd690163a10 msgr2=0x7fd6900aed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:56.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.363+0000 7fd687fff640 1 --2- 172.21.15.5:0/3960949722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd690163a10 0x7fd6900aed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:56.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.363+0000 7fd687fff640 1 -- 172.21.15.5:0/3960949722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6900adc70 con 0x7fd6900d14b0 2024-05-20T18:29:56.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.364+0000 7fd687fff640 1 --2- 172.21.15.5:0/3960949722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6900d14b0 0x7fd6900ab580 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd68000c460 tx=0x7fd68000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:56.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.364+0000 7fd6857fa640 1 -- 172.21.15.5:0/3960949722 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd680018070 con 0x7fd6900d14b0 2024-05-20T18:29:56.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.364+0000 7fd696f5e640 1 -- 172.21.15.5:0/3960949722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd69015c9a0 con 0x7fd6900d14b0 2024-05-20T18:29:56.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.364+0000 7fd6857fa640 1 -- 172.21.15.5:0/3960949722 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd680007500 con 0x7fd6900d14b0 2024-05-20T18:29:56.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.364+0000 7fd6857fa640 1 -- 172.21.15.5:0/3960949722 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd680004a20 con 0x7fd6900d14b0 2024-05-20T18:29:56.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.364+0000 7fd696f5e640 1 -- 172.21.15.5:0/3960949722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6900ae230 con 0x7fd6900d14b0 2024-05-20T18:29:56.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.365+0000 7fd696f5e640 1 -- 172.21.15.5:0/3960949722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd690075b00 con 0x7fd6900d14b0 2024-05-20T18:29:56.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.365+0000 7fd6857fa640 1 -- 172.21.15.5:0/3960949722 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd680004bc0 con 0x7fd6900d14b0 2024-05-20T18:29:56.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.366+0000 7fd6857fa640 1 --2- 172.21.15.5:0/3960949722 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd67c06be30 0x7fd67c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:56.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.366+0000 7fd6857fa640 1 -- 172.21.15.5:0/3960949722 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5249+0+0 (secure 0 0 0) 0x7fd68008a0b0 con 0x7fd6900d14b0 2024-05-20T18:29:56.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.366+0000 7fd694cfa640 1 --2- 172.21.15.5:0/3960949722 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd67c06be30 0x7fd67c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:56.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.366+0000 7fd694cfa640 1 --2- 172.21.15.5:0/3960949722 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd67c06be30 0x7fd67c06e340 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd68c034000 tx=0x7fd68c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:56.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.371+0000 7fd6857fa640 1 -- 172.21.15.5:0/3960949722 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd680058420 con 0x7fd6900d14b0 2024-05-20T18:29:56.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.629+0000 7fd696f5e640 1 -- 172.21.15.5:0/3960949722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fd6900d3050 con 0x7fd6900d14b0 2024-05-20T18:29:56.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.629+0000 7fd6857fa640 1 -- 172.21.15.5:0/3960949722 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+5304 (secure 0 0 0) 0x7fd68005c720 con 0x7fd6900d14b0 2024-05-20T18:29:56.629 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:29:56.630 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:29:56.630 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_dump": [ 2024-05-20T18:29:56.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:29:56.630 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.a", 2024-05-20T18:29:56.630 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtm9lSeCBAA/PpiEqojk6ueJo60tU4/iA==", 2024-05-20T18:29:56.630 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:29:56.630 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:29:56.630 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:29:56.630 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:29:56.630 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:29:56.630 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:29:56.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:29:56.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:29:56.631 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.b", 2024-05-20T18:29:56.631 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmdov0FRAACkWAR/0zaCDjgnkHnoHwgw==", 2024-05-20T18:29:56.631 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:29:56.631 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:29:56.631 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:29:56.631 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:29:56.631 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:29:56.631 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:29:56.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:29:56.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:29:56.631 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.c", 2024-05-20T18:29:56.631 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmpEBlEBAAdgxDXj9x38VSOizUgdFpYA==", 2024-05-20T18:29:56.632 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:29:56.632 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:29:56.632 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:29:56.632 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:29:56.632 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:29:56.632 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:29:56.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:29:56.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:29:56.632 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.d", 2024-05-20T18:29:56.632 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtm0Gy9HRAAAkPqcBLwxTwuWmCi7aad1A==", 2024-05-20T18:29:56.632 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:29:56.632 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:29:56.632 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:29:56.633 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:29:56.633 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:29:56.633 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:29:56.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:29:56.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:29:56.633 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.0", 2024-05-20T18:29:56.633 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDHlUtmWFGzBRAA81qN37dlHf6QDZ1FgwbJ9w==", 2024-05-20T18:29:56.633 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:29:56.633 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:29:56.633 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:29:56.633 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:29:56.633 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:29:56.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:29:56.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:29:56.634 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.1", 2024-05-20T18:29:56.634 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDKlUtmBMKsBBAAD4tTTvetME6oxwugTubSKQ==", 2024-05-20T18:29:56.634 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:29:56.634 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:29:56.634 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:29:56.634 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:29:56.634 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:29:56.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:29:56.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:29:56.634 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.2", 2024-05-20T18:29:56.634 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDNlUtm4Rm/AxAAIGWTUxb+LII09Rd+h5cJ8Q==", 2024-05-20T18:29:56.634 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:29:56.635 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:29:56.635 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:29:56.635 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:29:56.635 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:29:56.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:29:56.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:29:56.635 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.3", 2024-05-20T18:29:56.635 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDQlUtmWVxZAxAAB/w6k4WJ58ulEr9zatCRfw==", 2024-05-20T18:29:56.635 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:29:56.635 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:29:56.635 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:29:56.635 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:29:56.635 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:29:56.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:29:56.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:29:56.636 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.4", 2024-05-20T18:29:56.636 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDWlUtmIyokCRAAolOiYVCYaNTQZb1PjtR6fg==", 2024-05-20T18:29:56.636 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:29:56.636 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:29:56.636 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:29:56.636 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:29:56.636 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:29:56.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:29:56.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:29:56.636 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.5", 2024-05-20T18:29:56.636 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZlUtmSf6mCBAAw73nwq/zzt9kIXZymZh93A==", 2024-05-20T18:29:56.636 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:29:56.636 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:29:56.637 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:29:56.637 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:29:56.637 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:29:56.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:29:56.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:29:56.637 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.6", 2024-05-20T18:29:56.637 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDclUtmwwhGCBAA94OBEtj9h7cmCAddYMonaA==", 2024-05-20T18:29:56.637 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:29:56.637 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:29:56.637 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:29:56.637 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:29:56.637 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:29:56.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:29:56.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:29:56.638 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.7", 2024-05-20T18:29:56.638 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDflUtm9wefBxAAvCZTBzRu05koFhc+pH5I8w==", 2024-05-20T18:29:56.638 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:29:56.638 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:29:56.638 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:29:56.638 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:29:56.638 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:29:56.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:29:56.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:29:56.638 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.0", 2024-05-20T18:29:56.638 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmevaaIhAATIwyXzr0vGT38ZmcMqR2gA==", 2024-05-20T18:29:56.638 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:29:56.639 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:29:56.639 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow r", 2024-05-20T18:29:56.639 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-20T18:29:56.639 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:29:56.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:29:56.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:29:56.639 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.1", 2024-05-20T18:29:56.639 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmK7NfKhAADaXl0uVP1Q1gUOlPsstnzg==", 2024-05-20T18:29:56.639 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:29:56.639 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:29:56.639 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-20T18:29:56.639 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-20T18:29:56.639 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-20T18:29:56.639 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:29:56.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:29:56.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:29:56.640 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.2", 2024-05-20T18:29:56.640 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmMJF1JhAAeM+HseqvgvKgG8EnIsZuhg==", 2024-05-20T18:29:56.640 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:29:56.640 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:29:56.640 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-20T18:29:56.640 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-20T18:29:56.640 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-20T18:29:56.640 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:29:56.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:29:56.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:29:56.641 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.3", 2024-05-20T18:29:56.641 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmodA7LhAAUI2v0npB6HHs2n1g3KdTXA==", 2024-05-20T18:29:56.641 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:29:56.641 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:29:56.641 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-20T18:29:56.641 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-20T18:29:56.641 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-20T18:29:56.641 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:29:56.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:29:56.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:29:56.642 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.admin", 2024-05-20T18:29:56.642 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDElUtmZacGJRAAoyr9XNPBr9r0DSU53qCHSQ==", 2024-05-20T18:29:56.642 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:29:56.642 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-20T18:29:56.642 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:29:56.642 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:29:56.642 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:29:56.642 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:29:56.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:29:56.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:29:56.642 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.x", 2024-05-20T18:29:56.642 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmPgq0BBAAAljNeQDsSau0wH2Lh2KdBw==", 2024-05-20T18:29:56.643 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:29:56.643 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-20T18:29:56.643 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-05-20T18:29:56.643 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:29:56.643 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:29:56.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:29:56.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:29:56.643 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.y", 2024-05-20T18:29:56.643 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDElUtm6Xv3OhAA0doOWTC73jy5rOYm7Vf68Q==", 2024-05-20T18:29:56.643 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:29:56.643 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-20T18:29:56.643 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-05-20T18:29:56.643 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:29:56.644 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:29:56.644 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:29:56.644 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-20T18:29:56.644 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:29:56.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.631+0000 7fd696f5e640 1 -- 172.21.15.5:0/3960949722 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd67c06be30 msgr2=0x7fd67c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:56.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.631+0000 7fd696f5e640 1 --2- 172.21.15.5:0/3960949722 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd67c06be30 0x7fd67c06e340 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd68c034000 tx=0x7fd68c02e000 comp rx=0 tx=0).stop 2024-05-20T18:29:56.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.631+0000 7fd696f5e640 1 -- 172.21.15.5:0/3960949722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6900d14b0 msgr2=0x7fd6900ab580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:56.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.631+0000 7fd696f5e640 1 --2- 172.21.15.5:0/3960949722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6900d14b0 0x7fd6900ab580 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd68000c460 tx=0x7fd68000c920 comp rx=0 tx=0).stop 2024-05-20T18:29:56.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.632+0000 7fd696f5e640 1 -- 172.21.15.5:0/3960949722 shutdown_connections 2024-05-20T18:29:56.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.632+0000 7fd696f5e640 1 --2- 172.21.15.5:0/3960949722 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd67c06be30 0x7fd67c06e340 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:56.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.632+0000 7fd696f5e640 1 --2- 172.21.15.5:0/3960949722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd690163a10 0x7fd6900aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:56.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.632+0000 7fd696f5e640 1 --2- 172.21.15.5:0/3960949722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6900d14b0 0x7fd6900ab580 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:56.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.632+0000 7fd696f5e640 1 --2- 172.21.15.5:0/3960949722 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6900b4010 0x7fd69015c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:56.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.632+0000 7fd696f5e640 1 -- 172.21.15.5:0/3960949722 >> 172.21.15.5:0/3960949722 conn(0x7fd69006d240 msgr2=0x7fd69015d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:56.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.632+0000 7fd696f5e640 1 -- 172.21.15.5:0/3960949722 shutdown_connections 2024-05-20T18:29:56.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.632+0000 7fd696f5e640 1 -- 172.21.15.5:0/3960949722 wait complete. 2024-05-20T18:29:56.647 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:29:56.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.761+0000 7f58398e9640 1 Processor -- start 2024-05-20T18:29:56.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.762+0000 7f58398e9640 1 -- start start 2024-05-20T18:29:56.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.763+0000 7f58398e9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5834162820 0x7f58340b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:56.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.763+0000 7f58398e9640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f58340b3e80 0x7f583406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:56.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.763+0000 7f58398e9640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f583406dd20 con 0x7f583406d910 2024-05-20T18:29:56.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.763+0000 7f58398e9640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f58340d0630 con 0x7f5834162820 2024-05-20T18:29:56.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.763+0000 7f58398e9640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f58340d39c0 con 0x7f58340b3e80 2024-05-20T18:29:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.763+0000 7f5832ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5834162820 0x7f58340b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.763+0000 7f58337fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f583406d910 0x7f58340d27a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:55156/0 (socket says 172.21.15.5:55156) 2024-05-20T18:29:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.763+0000 7f58337fe640 1 -- 172.21.15.5:0/2429775648 learned_addr learned my addr 172.21.15.5:0/2429775648 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.763+0000 7f58327fc640 1 --2- 172.21.15.5:0/2429775648 >> v2:172.21.15.63:3300/0 conn(0x7f58340b3e80 0x7f583406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.763+0000 7f5832ffd640 1 -- 172.21.15.5:0/2429775648 >> v2:172.21.15.63:3300/0 conn(0x7f58340b3e80 msgr2=0x7f583406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.763+0000 7f5832ffd640 1 --2- 172.21.15.5:0/2429775648 >> v2:172.21.15.63:3300/0 conn(0x7f58340b3e80 0x7f583406d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.763+0000 7f5832ffd640 1 -- 172.21.15.5:0/2429775648 >> v1:172.21.15.5:6789/0 conn(0x7f583406d910 legacy=0x7f58340d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.763+0000 7f5832ffd640 1 -- 172.21.15.5:0/2429775648 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58340d3ca0 con 0x7f5834162820 2024-05-20T18:29:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.763+0000 7f5831ffb640 1 -- 172.21.15.5:0/2429775648 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 638125783 0 0) 0x7f582c001110 con 0x7f583406d910 2024-05-20T18:29:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.763+0000 7f58327fc640 1 --2- 172.21.15.5:0/2429775648 >> v2:172.21.15.63:3300/0 conn(0x7f58340b3e80 0x7f583406d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:29:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.764+0000 7f5832ffd640 1 --2- 172.21.15.5:0/2429775648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5834162820 0x7f58340b3940 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f58340d1230 tx=0x7f582402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e8d2d3c331d39f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.764+0000 7f5831ffb640 1 -- 172.21.15.5:0/2429775648 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824007740 con 0x7f5834162820 2024-05-20T18:29:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.764+0000 7f5831ffb640 1 -- 172.21.15.5:0/2429775648 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58240078e0 con 0x7f5834162820 2024-05-20T18:29:56.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.764+0000 7f5831ffb640 1 -- 172.21.15.5:0/2429775648 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824007cb0 con 0x7f5834162820 2024-05-20T18:29:56.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.765+0000 7f58398e9640 1 -- 172.21.15.5:0/2429775648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5834162820 msgr2=0x7f58340b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:56.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.765+0000 7f58398e9640 1 --2- 172.21.15.5:0/2429775648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5834162820 0x7f58340b3940 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f58340d1230 tx=0x7f582402a680 comp rx=0 tx=0).stop 2024-05-20T18:29:56.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.765+0000 7f58398e9640 1 -- 172.21.15.5:0/2429775648 shutdown_connections 2024-05-20T18:29:56.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.765+0000 7f58398e9640 1 --2- 172.21.15.5:0/2429775648 >> v2:172.21.15.63:3300/0 conn(0x7f58340b3e80 0x7f583406d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:56.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.765+0000 7f58398e9640 1 --2- 172.21.15.5:0/2429775648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5834162820 0x7f58340b3940 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:56.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.765+0000 7f58398e9640 1 -- 172.21.15.5:0/2429775648 >> 172.21.15.5:0/2429775648 conn(0x7f58340d4bd0 msgr2=0x7f58340d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:56.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.765+0000 7f58398e9640 1 -- 172.21.15.5:0/2429775648 shutdown_connections 2024-05-20T18:29:56.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.765+0000 7f58398e9640 1 -- 172.21.15.5:0/2429775648 wait complete. 2024-05-20T18:29:56.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.767+0000 7f58398e9640 1 Processor -- start 2024-05-20T18:29:56.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.767+0000 7f58398e9640 1 -- start start 2024-05-20T18:29:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.768+0000 7f58398e9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f583406d910 0x7f583414b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.768+0000 7f58398e9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58340b3e80 0x7f58341549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.768+0000 7f58398e9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5834162820 0x7f5834154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.768+0000 7f58398e9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f58340d0300 con 0x7f583406d910 2024-05-20T18:29:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.768+0000 7f58398e9640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f58340d4260 con 0x7f58340b3e80 2024-05-20T18:29:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.768+0000 7f58398e9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f58340b91a0 con 0x7f5834162820 2024-05-20T18:29:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.769+0000 7f5832ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f583406d910 0x7f583414b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.769+0000 7f5832ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f583406d910 0x7f583414b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33782/0 (socket says 172.21.15.5:33782) 2024-05-20T18:29:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.769+0000 7f5832ffd640 1 -- 172.21.15.5:0/2269948422 learned_addr learned my addr 172.21.15.5:0/2269948422 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.769+0000 7f58337fe640 1 --2- 172.21.15.5:0/2269948422 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5834162820 0x7f5834154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.769+0000 7f58327fc640 1 --2- 172.21.15.5:0/2269948422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58340b3e80 0x7f58341549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:56.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.769+0000 7f5832ffd640 1 -- 172.21.15.5:0/2269948422 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5834162820 msgr2=0x7f5834154ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:56.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.769+0000 7f5832ffd640 1 --2- 172.21.15.5:0/2269948422 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5834162820 0x7f5834154ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:56.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.769+0000 7f5832ffd640 1 -- 172.21.15.5:0/2269948422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58340b3e80 msgr2=0x7f58341549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:56.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.769+0000 7f5832ffd640 1 --2- 172.21.15.5:0/2269948422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58340b3e80 0x7f58341549a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:56.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.769+0000 7f5832ffd640 1 -- 172.21.15.5:0/2269948422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f583414b580 con 0x7f583406d910 2024-05-20T18:29:56.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.769+0000 7f5832ffd640 1 --2- 172.21.15.5:0/2269948422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f583406d910 0x7f583414b040 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5824009e80 tx=0x7f5824004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:56.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.769+0000 7f581bfff640 1 -- 172.21.15.5:0/2269948422 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582403f070 con 0x7f583406d910 2024-05-20T18:29:56.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.769+0000 7f581bfff640 1 -- 172.21.15.5:0/2269948422 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5824031c40 con 0x7f583406d910 2024-05-20T18:29:56.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.769+0000 7f581bfff640 1 -- 172.21.15.5:0/2269948422 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824009450 con 0x7f583406d910 2024-05-20T18:29:56.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.769+0000 7f58398e9640 1 -- 172.21.15.5:0/2269948422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5834158170 con 0x7f583406d910 2024-05-20T18:29:56.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.770+0000 7f58398e9640 1 -- 172.21.15.5:0/2269948422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5834158730 con 0x7f583406d910 2024-05-20T18:29:56.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.771+0000 7f58398e9640 1 -- 172.21.15.5:0/2269948422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5834075b00 con 0x7f583406d910 2024-05-20T18:29:56.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.771+0000 7f581bfff640 1 -- 172.21.15.5:0/2269948422 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5824003990 con 0x7f583406d910 2024-05-20T18:29:56.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.772+0000 7f581bfff640 1 --2- 172.21.15.5:0/2269948422 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f582006be30 0x7f582006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:56.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.772+0000 7f581bfff640 1 -- 172.21.15.5:0/2269948422 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5249+0+0 (secure 0 0 0) 0x7f58240abf10 con 0x7f583406d910 2024-05-20T18:29:56.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.772+0000 7f58327fc640 1 --2- 172.21.15.5:0/2269948422 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f582006be30 0x7f582006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:56.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.772+0000 7f58327fc640 1 --2- 172.21.15.5:0/2269948422 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f582006be30 0x7f582006e340 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f583415af70 tx=0x7f5828008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:56.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.777+0000 7f581bfff640 1 -- 172.21.15.5:0/2269948422 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f582407a3a0 con 0x7f583406d910 2024-05-20T18:29:56.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.964+0000 7f58398e9640 1 -- 172.21.15.5:0/2269948422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f583406c530 con 0x7f583406d910 2024-05-20T18:29:56.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.965+0000 7f581bfff640 1 -- 172.21.15.5:0/2269948422 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+13044 (secure 0 0 0) 0x7f582407e6a0 con 0x7f583406d910 2024-05-20T18:29:56.965 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:29:56.965 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":59,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:29:56.214532+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:29:56.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.967+0000 7f58398e9640 1 -- 172.21.15.5:0/2269948422 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f582006be30 msgr2=0x7f582006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:56.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.967+0000 7f58398e9640 1 --2- 172.21.15.5:0/2269948422 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f582006be30 0x7f582006e340 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f583415af70 tx=0x7f5828008040 comp rx=0 tx=0).stop 2024-05-20T18:29:56.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.967+0000 7f58398e9640 1 -- 172.21.15.5:0/2269948422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f583406d910 msgr2=0x7f583414b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:56.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.967+0000 7f58398e9640 1 --2- 172.21.15.5:0/2269948422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f583406d910 0x7f583414b040 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5824009e80 tx=0x7f5824004a40 comp rx=0 tx=0).stop 2024-05-20T18:29:56.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.967+0000 7f58398e9640 1 -- 172.21.15.5:0/2269948422 shutdown_connections 2024-05-20T18:29:56.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.967+0000 7f58398e9640 1 --2- 172.21.15.5:0/2269948422 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f582006be30 0x7f582006e340 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:56.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.967+0000 7f58398e9640 1 --2- 172.21.15.5:0/2269948422 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5834162820 0x7f5834154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:56.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.967+0000 7f58398e9640 1 --2- 172.21.15.5:0/2269948422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58340b3e80 0x7f58341549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:56.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.967+0000 7f58398e9640 1 --2- 172.21.15.5:0/2269948422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f583406d910 0x7f583414b040 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:56.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.967+0000 7f58398e9640 1 -- 172.21.15.5:0/2269948422 >> 172.21.15.5:0/2269948422 conn(0x7f58340d4bd0 msgr2=0x7f58340d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:56.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.967+0000 7f58398e9640 1 -- 172.21.15.5:0/2269948422 shutdown_connections 2024-05-20T18:29:56.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:56.967+0000 7f58398e9640 1 -- 172.21.15.5:0/2269948422 wait complete. 2024-05-20T18:29:56.979 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:29:56.979 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:29:57.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.097+0000 7f7067d9d640 1 Processor -- start 2024-05-20T18:29:57.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.098+0000 7f7067d9d640 1 -- start start 2024-05-20T18:29:57.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.099+0000 7f7067d9d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f706006d3d0 0x7f706006d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:57.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.099+0000 7f7067d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f70600b8190 0x7f70600b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:57.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.099+0000 7f7067d9d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f706006dd20 con 0x7f7060162820 2024-05-20T18:29:57.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.099+0000 7f7067d9d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f70600d0630 con 0x7f70600b8190 2024-05-20T18:29:57.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.099+0000 7f7067d9d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f70600d39c0 con 0x7f706006d3d0 2024-05-20T18:29:57.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.099+0000 7f706633a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f70600b8190 0x7f70600b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:57.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.099+0000 7f7065b39640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7060162820 0x7f70600b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:55162/0 (socket says 172.21.15.5:55162) 2024-05-20T18:29:57.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.099+0000 7f7065b39640 1 -- 172.21.15.5:0/1385587485 learned_addr learned my addr 172.21.15.5:0/1385587485 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:57.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.099+0000 7f7065338640 1 --2- 172.21.15.5:0/1385587485 >> v2:172.21.15.63:3300/0 conn(0x7f706006d3d0 0x7f706006d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:57.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.099+0000 7f706633a640 1 -- 172.21.15.5:0/1385587485 >> v2:172.21.15.63:3300/0 conn(0x7f706006d3d0 msgr2=0x7f706006d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:57.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.099+0000 7f706633a640 1 --2- 172.21.15.5:0/1385587485 >> v2:172.21.15.63:3300/0 conn(0x7f706006d3d0 0x7f706006d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:57.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.099+0000 7f706633a640 1 -- 172.21.15.5:0/1385587485 >> v1:172.21.15.5:6789/0 conn(0x7f7060162820 legacy=0x7f70600b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:57.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.099+0000 7f7064b37640 1 -- 172.21.15.5:0/1385587485 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1837885351 0 0) 0x7f705c001110 con 0x7f7060162820 2024-05-20T18:29:57.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.099+0000 7f706633a640 1 -- 172.21.15.5:0/1385587485 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70600d3ca0 con 0x7f70600b8190 2024-05-20T18:29:57.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.100+0000 7f706633a640 1 --2- 172.21.15.5:0/1385587485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70600b8190 0x7f70600b85a0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f70600d1230 tx=0x7f705802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3840bd51660aa7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:57.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.100+0000 7f7064b37640 1 -- 172.21.15.5:0/1385587485 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7058007740 con 0x7f70600b8190 2024-05-20T18:29:57.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.100+0000 7f7064b37640 1 -- 172.21.15.5:0/1385587485 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70580078e0 con 0x7f70600b8190 2024-05-20T18:29:57.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.100+0000 7f7064b37640 1 -- 172.21.15.5:0/1385587485 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7058007cb0 con 0x7f70600b8190 2024-05-20T18:29:57.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.101+0000 7f7067d9d640 1 -- 172.21.15.5:0/1385587485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70600b8190 msgr2=0x7f70600b85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:57.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.101+0000 7f7067d9d640 1 --2- 172.21.15.5:0/1385587485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70600b8190 0x7f70600b85a0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f70600d1230 tx=0x7f705802a680 comp rx=0 tx=0).stop 2024-05-20T18:29:57.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.101+0000 7f7067d9d640 1 -- 172.21.15.5:0/1385587485 shutdown_connections 2024-05-20T18:29:57.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.101+0000 7f7067d9d640 1 --2- 172.21.15.5:0/1385587485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70600b8190 0x7f70600b85a0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:57.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.101+0000 7f7067d9d640 1 --2- 172.21.15.5:0/1385587485 >> v2:172.21.15.63:3300/0 conn(0x7f706006d3d0 0x7f706006d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:57.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.101+0000 7f7067d9d640 1 -- 172.21.15.5:0/1385587485 >> 172.21.15.5:0/1385587485 conn(0x7f70600d4bd0 msgr2=0x7f70600d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:57.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.101+0000 7f7067d9d640 1 -- 172.21.15.5:0/1385587485 shutdown_connections 2024-05-20T18:29:57.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.101+0000 7f7067d9d640 1 -- 172.21.15.5:0/1385587485 wait complete. 2024-05-20T18:29:57.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.103+0000 7f7067d9d640 1 Processor -- start 2024-05-20T18:29:57.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.103+0000 7f7067d9d640 1 -- start start 2024-05-20T18:29:57.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.104+0000 7f7067d9d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f706006d3d0 0x7f7060146ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:57.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.104+0000 7f7067d9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70600b8190 0x7f7060145570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:57.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.104+0000 7f7067d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7060162820 0x7f7060144b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:57.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.104+0000 7f7067d9d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f70600d0300 con 0x7f7060162820 2024-05-20T18:29:57.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.104+0000 7f7067d9d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f70600d4260 con 0x7f70600b8190 2024-05-20T18:29:57.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.104+0000 7f7067d9d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f70600b4050 con 0x7f706006d3d0 2024-05-20T18:29:57.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.104+0000 7f706633a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7060162820 0x7f7060144b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:57.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.104+0000 7f706633a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7060162820 0x7f7060144b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33804/0 (socket says 172.21.15.5:33804) 2024-05-20T18:29:57.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.104+0000 7f706633a640 1 -- 172.21.15.5:0/3403368383 learned_addr learned my addr 172.21.15.5:0/3403368383 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:57.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.105+0000 7f7065338640 1 --2- 172.21.15.5:0/3403368383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70600b8190 0x7f7060145570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:57.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.105+0000 7f7065b39640 1 --2- 172.21.15.5:0/3403368383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f706006d3d0 0x7f7060146ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:57.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.105+0000 7f706633a640 1 -- 172.21.15.5:0/3403368383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f706006d3d0 msgr2=0x7f7060146ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:57.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.105+0000 7f706633a640 1 --2- 172.21.15.5:0/3403368383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f706006d3d0 0x7f7060146ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:57.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.105+0000 7f706633a640 1 -- 172.21.15.5:0/3403368383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70600b8190 msgr2=0x7f7060145570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:57.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.105+0000 7f706633a640 1 --2- 172.21.15.5:0/3403368383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70600b8190 0x7f7060145570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:57.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.105+0000 7f706633a640 1 -- 172.21.15.5:0/3403368383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7060145f50 con 0x7f7060162820 2024-05-20T18:29:57.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.105+0000 7f706633a640 1 --2- 172.21.15.5:0/3403368383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7060162820 0x7f7060144b20 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f705802ac70 tx=0x7f7058004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:57.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.105+0000 7f70567fc640 1 -- 172.21.15.5:0/3403368383 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705803f070 con 0x7f7060162820 2024-05-20T18:29:57.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.105+0000 7f70567fc640 1 -- 172.21.15.5:0/3403368383 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7058031c90 con 0x7f7060162820 2024-05-20T18:29:57.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.105+0000 7f7067d9d640 1 -- 172.21.15.5:0/3403368383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7060146230 con 0x7f7060162820 2024-05-20T18:29:57.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.105+0000 7f70567fc640 1 -- 172.21.15.5:0/3403368383 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7058044680 con 0x7f7060162820 2024-05-20T18:29:57.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.105+0000 7f7067d9d640 1 -- 172.21.15.5:0/3403368383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70601467f0 con 0x7f7060162820 2024-05-20T18:29:57.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.106+0000 7f7067d9d640 1 -- 172.21.15.5:0/3403368383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7060075b00 con 0x7f7060162820 2024-05-20T18:29:57.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.107+0000 7f70567fc640 1 -- 172.21.15.5:0/3403368383 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f705803b410 con 0x7f7060162820 2024-05-20T18:29:57.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.107+0000 7f70567fc640 1 --2- 172.21.15.5:0/3403368383 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f704406be30 0x7f704406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:57.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.107+0000 7f70567fc640 1 -- 172.21.15.5:0/3403368383 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5249+0+0 (secure 0 0 0) 0x7f70580ac6d0 con 0x7f7060162820 2024-05-20T18:29:57.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.107+0000 7f7065b39640 1 --2- 172.21.15.5:0/3403368383 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f704406be30 0x7f704406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:57.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.108+0000 7f7065b39640 1 --2- 172.21.15.5:0/3403368383 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f704406be30 0x7f704406e340 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f705c000f60 tx=0x7f705c007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:57.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.112+0000 7f70567fc640 1 -- 172.21.15.5:0/3403368383 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f705807acb0 con 0x7f7060162820 2024-05-20T18:29:57.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.297+0000 7f7067d9d640 1 -- 172.21.15.5:0/3403368383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7060145ca0 con 0x7f7060162820 2024-05-20T18:29:57.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.298+0000 7f70567fc640 1 -- 172.21.15.5:0/3403368383 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) ==== 93+0+10 (secure 0 0 0) 0x7f705807efb0 con 0x7f7060162820 2024-05-20T18:29:57.298 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:29:57.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.300+0000 7f7067d9d640 1 -- 172.21.15.5:0/3403368383 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f704406be30 msgr2=0x7f704406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:57.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.300+0000 7f7067d9d640 1 --2- 172.21.15.5:0/3403368383 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f704406be30 0x7f704406e340 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f705c000f60 tx=0x7f705c007c00 comp rx=0 tx=0).stop 2024-05-20T18:29:57.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.300+0000 7f7067d9d640 1 -- 172.21.15.5:0/3403368383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7060162820 msgr2=0x7f7060144b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.300+0000 7f7067d9d640 1 --2- 172.21.15.5:0/3403368383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7060162820 0x7f7060144b20 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f705802ac70 tx=0x7f7058004a40 comp rx=0 tx=0).stop 2024-05-20T18:29:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.300+0000 7f7067d9d640 1 -- 172.21.15.5:0/3403368383 shutdown_connections 2024-05-20T18:29:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.300+0000 7f7067d9d640 1 --2- 172.21.15.5:0/3403368383 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f704406be30 0x7f704406e340 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.300+0000 7f7067d9d640 1 --2- 172.21.15.5:0/3403368383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7060162820 0x7f7060144b20 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.300+0000 7f7067d9d640 1 --2- 172.21.15.5:0/3403368383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70600b8190 0x7f7060145570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.300+0000 7f7067d9d640 1 --2- 172.21.15.5:0/3403368383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f706006d3d0 0x7f7060146ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.300+0000 7f7067d9d640 1 -- 172.21.15.5:0/3403368383 >> 172.21.15.5:0/3403368383 conn(0x7f70600d4bd0 msgr2=0x7f70600d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.300+0000 7f7067d9d640 1 -- 172.21.15.5:0/3403368383 shutdown_connections 2024-05-20T18:29:57.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.300+0000 7f7067d9d640 1 -- 172.21.15.5:0/3403368383 wait complete. 2024-05-20T18:29:57.312 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-20T18:29:57.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.426+0000 7f32944f9640 1 Processor -- start 2024-05-20T18:29:57.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.426+0000 7f32944f9640 1 -- start start 2024-05-20T18:29:57.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.428+0000 7f32944f9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f328c0b47b0 0x7f328c0afad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:57.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.428+0000 7f32944f9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f328c15d060 0x7f328c0b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:57.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.428+0000 7f32944f9640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f328c0b5b30 con 0x7f328c0b5720 2024-05-20T18:29:57.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.428+0000 7f32944f9640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f328c0703e0 con 0x7f328c0b47b0 2024-05-20T18:29:57.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.428+0000 7f32944f9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f328c0b39f0 con 0x7f328c15d060 2024-05-20T18:29:57.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.428+0000 7f3292295640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f328c0b47b0 0x7f328c0afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:57.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.428+0000 7f3292a96640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f328c0b5720 0x7f328c15fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:55178/0 (socket says 172.21.15.5:55178) 2024-05-20T18:29:57.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.428+0000 7f3292a96640 1 -- 172.21.15.5:0/1957145960 learned_addr learned my addr 172.21.15.5:0/1957145960 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:57.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.428+0000 7f3291a94640 1 --2- 172.21.15.5:0/1957145960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f328c15d060 0x7f328c0b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:57.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.428+0000 7f3292295640 1 -- 172.21.15.5:0/1957145960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f328c15d060 msgr2=0x7f328c0b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:57.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.428+0000 7f3292295640 1 --2- 172.21.15.5:0/1957145960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f328c15d060 0x7f328c0b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:57.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.428+0000 7f3292295640 1 -- 172.21.15.5:0/1957145960 >> v1:172.21.15.5:6789/0 conn(0x7f328c0b5720 legacy=0x7f328c15fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:57.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.428+0000 7f3291293640 1 -- 172.21.15.5:0/1957145960 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4004058589 0 0) 0x7f327c001490 con 0x7f328c0b5720 2024-05-20T18:29:57.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.428+0000 7f3292295640 1 -- 172.21.15.5:0/1957145960 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f328c0b3cd0 con 0x7f328c0b47b0 2024-05-20T18:29:57.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.429+0000 7f3292295640 1 --2- 172.21.15.5:0/1957145960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f328c0b47b0 0x7f328c0afad0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f32840097a0 tx=0x7f328402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d076e9661bd20fce server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:57.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.429+0000 7f3291293640 1 -- 172.21.15.5:0/1957145960 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3284037070 con 0x7f328c0b47b0 2024-05-20T18:29:57.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.429+0000 7f3291293640 1 -- 172.21.15.5:0/1957145960 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f328402ae50 con 0x7f328c0b47b0 2024-05-20T18:29:57.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.429+0000 7f3291293640 1 -- 172.21.15.5:0/1957145960 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3284032660 con 0x7f328c0b47b0 2024-05-20T18:29:57.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.430+0000 7f32944f9640 1 -- 172.21.15.5:0/1957145960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f328c0b47b0 msgr2=0x7f328c0afad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:57.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.430+0000 7f32944f9640 1 --2- 172.21.15.5:0/1957145960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f328c0b47b0 0x7f328c0afad0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f32840097a0 tx=0x7f328402a100 comp rx=0 tx=0).stop 2024-05-20T18:29:57.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.430+0000 7f32944f9640 1 -- 172.21.15.5:0/1957145960 shutdown_connections 2024-05-20T18:29:57.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.430+0000 7f32944f9640 1 --2- 172.21.15.5:0/1957145960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f328c15d060 0x7f328c0b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:57.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.430+0000 7f32944f9640 1 --2- 172.21.15.5:0/1957145960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f328c0b47b0 0x7f328c0afad0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:57.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.430+0000 7f32944f9640 1 -- 172.21.15.5:0/1957145960 >> 172.21.15.5:0/1957145960 conn(0x7f328c0cd2a0 msgr2=0x7f328c0cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:57.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.430+0000 7f32944f9640 1 -- 172.21.15.5:0/1957145960 shutdown_connections 2024-05-20T18:29:57.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.430+0000 7f32944f9640 1 -- 172.21.15.5:0/1957145960 wait complete. 2024-05-20T18:29:57.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.432+0000 7f32944f9640 1 Processor -- start 2024-05-20T18:29:57.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.432+0000 7f32944f9640 1 -- start start 2024-05-20T18:29:57.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.433+0000 7f32944f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f328c0b5720 0x7f328c148250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:57.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.433+0000 7f32944f9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f328c15d060 0x7f328c14b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:57.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.433+0000 7f32944f9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f328c15d540 0x7f328c0aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:57.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.433+0000 7f32944f9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f328c0b0010 con 0x7f328c0b5720 2024-05-20T18:29:57.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.433+0000 7f32944f9640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f328c0b4290 con 0x7f328c15d060 2024-05-20T18:29:57.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.433+0000 7f32944f9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f328c159c60 con 0x7f328c15d540 2024-05-20T18:29:57.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.433+0000 7f3292295640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f328c0b5720 0x7f328c148250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:57.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.433+0000 7f3292295640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f328c0b5720 0x7f328c148250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33824/0 (socket says 172.21.15.5:33824) 2024-05-20T18:29:57.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.433+0000 7f3292295640 1 -- 172.21.15.5:0/2956760867 learned_addr learned my addr 172.21.15.5:0/2956760867 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:57.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.433+0000 7f3291a94640 1 --2- 172.21.15.5:0/2956760867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f328c15d060 0x7f328c14b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:57.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.433+0000 7f3292a96640 1 --2- 172.21.15.5:0/2956760867 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f328c15d540 0x7f328c0aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:57.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.433+0000 7f3292295640 1 -- 172.21.15.5:0/2956760867 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f328c15d540 msgr2=0x7f328c0aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:57.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.433+0000 7f3292295640 1 --2- 172.21.15.5:0/2956760867 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f328c15d540 0x7f328c0aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:57.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.433+0000 7f3292295640 1 -- 172.21.15.5:0/2956760867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f328c15d060 msgr2=0x7f328c14b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:57.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.433+0000 7f3292295640 1 --2- 172.21.15.5:0/2956760867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f328c15d060 0x7f328c14b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:57.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.433+0000 7f3292295640 1 -- 172.21.15.5:0/2956760867 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f328c148790 con 0x7f328c0b5720 2024-05-20T18:29:57.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.433+0000 7f3291a94640 1 --2- 172.21.15.5:0/2956760867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f328c15d060 0x7f328c14b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:29:57.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.434+0000 7f3292295640 1 --2- 172.21.15.5:0/2956760867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f328c0b5720 0x7f328c148250 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3284035040 tx=0x7f32840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:57.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.434+0000 7f328affd640 1 -- 172.21.15.5:0/2956760867 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3284037040 con 0x7f328c0b5720 2024-05-20T18:29:57.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.434+0000 7f328affd640 1 -- 172.21.15.5:0/2956760867 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3284004030 con 0x7f328c0b5720 2024-05-20T18:29:57.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.434+0000 7f32944f9640 1 -- 172.21.15.5:0/2956760867 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f328c14ba20 con 0x7f328c0b5720 2024-05-20T18:29:57.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.434+0000 7f328affd640 1 -- 172.21.15.5:0/2956760867 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3284004400 con 0x7f328c0b5720 2024-05-20T18:29:57.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.434+0000 7f32944f9640 1 -- 172.21.15.5:0/2956760867 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f328c1490d0 con 0x7f328c0b5720 2024-05-20T18:29:57.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.435+0000 7f32944f9640 1 -- 172.21.15.5:0/2956760867 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f328c075b00 con 0x7f328c0b5720 2024-05-20T18:29:57.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.435+0000 7f328affd640 1 -- 172.21.15.5:0/2956760867 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3284003610 con 0x7f328c0b5720 2024-05-20T18:29:57.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.436+0000 7f328affd640 1 --2- 172.21.15.5:0/2956760867 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f327806be60 0x7f327806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:57.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.436+0000 7f328affd640 1 -- 172.21.15.5:0/2956760867 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5249+0+0 (secure 0 0 0) 0x7f32840ab820 con 0x7f328c0b5720 2024-05-20T18:29:57.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.436+0000 7f3291a94640 1 --2- 172.21.15.5:0/2956760867 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f327806be60 0x7f327806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:57.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.436+0000 7f3291a94640 1 --2- 172.21.15.5:0/2956760867 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f327806be60 0x7f327806e370 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f328c155ed0 tx=0x7f3280008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:57.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.440+0000 7f328affd640 1 -- 172.21.15.5:0/2956760867 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f328c148790 con 0x7f328c0b5720 2024-05-20T18:29:57.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.630+0000 7f32944f9640 1 -- 172.21.15.5:0/2956760867 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f328c0688a0 con 0x7f328c0b5720 2024-05-20T18:29:57.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.631+0000 7f328affd640 1 -- 172.21.15.5:0/2956760867 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v59) ==== 84+0+65 (secure 0 0 0) 0x7f328407a280 con 0x7f328c0b5720 2024-05-20T18:29:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:29:57.631 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-05-20T18:29:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:29:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-05-20T18:29:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-05-20T18:29:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:29:57.631 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-05-20T18:29:57.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.633+0000 7f32944f9640 1 -- 172.21.15.5:0/2956760867 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f327806be60 msgr2=0x7f327806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:57.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.633+0000 7f32944f9640 1 --2- 172.21.15.5:0/2956760867 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f327806be60 0x7f327806e370 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f328c155ed0 tx=0x7f3280008040 comp rx=0 tx=0).stop 2024-05-20T18:29:57.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.633+0000 7f32944f9640 1 -- 172.21.15.5:0/2956760867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f328c0b5720 msgr2=0x7f328c148250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:57.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.633+0000 7f32944f9640 1 --2- 172.21.15.5:0/2956760867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f328c0b5720 0x7f328c148250 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3284035040 tx=0x7f32840046c0 comp rx=0 tx=0).stop 2024-05-20T18:29:57.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.633+0000 7f32944f9640 1 -- 172.21.15.5:0/2956760867 shutdown_connections 2024-05-20T18:29:57.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.633+0000 7f32944f9640 1 --2- 172.21.15.5:0/2956760867 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f327806be60 0x7f327806e370 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:57.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.633+0000 7f32944f9640 1 --2- 172.21.15.5:0/2956760867 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f328c15d540 0x7f328c0aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:57.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.633+0000 7f32944f9640 1 --2- 172.21.15.5:0/2956760867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f328c15d060 0x7f328c14b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:57.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.633+0000 7f32944f9640 1 --2- 172.21.15.5:0/2956760867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f328c0b5720 0x7f328c148250 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:57.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.633+0000 7f32944f9640 1 -- 172.21.15.5:0/2956760867 >> 172.21.15.5:0/2956760867 conn(0x7f328c0cd2a0 msgr2=0x7f328c159ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:57.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.633+0000 7f32944f9640 1 -- 172.21.15.5:0/2956760867 shutdown_connections 2024-05-20T18:29:57.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.633+0000 7f32944f9640 1 -- 172.21.15.5:0/2956760867 wait complete. 2024-05-20T18:29:57.644 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-20T18:29:57.645 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-20T18:29:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.764+0000 7f90468fe640 1 Processor -- start 2024-05-20T18:29:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.764+0000 7f90468fe640 1 -- start start 2024-05-20T18:29:57.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.765+0000 7f90468fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f90400d1830 0x7f90400ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:57.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.765+0000 7f90468fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90400cea20 0x7f90400b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:57.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.765+0000 7f90468fe640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90400d0160 con 0x7f90400d3510 2024-05-20T18:29:57.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.765+0000 7f90468fe640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f90400b1ff0 con 0x7f90400d1830 2024-05-20T18:29:57.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.765+0000 7f90468fe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f90400cda00 con 0x7f90400cea20 2024-05-20T18:29:57.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.765+0000 7f9044e9b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90400cea20 0x7f90400b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:57.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.766+0000 7f903f7fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f90400d1830 0x7f90400ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.766+0000 7f9044e9b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90400cea20 0x7f90400b21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:35920/0 (socket says 172.21.15.5:35920) 2024-05-20T18:29:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.766+0000 7f9044e9b640 1 -- 172.21.15.5:0/310794858 learned_addr learned my addr 172.21.15.5:0/310794858 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.766+0000 7f903ffff640 1 --1- 172.21.15.5:0/310794858 >> v1:172.21.15.63:6789/0 conn(0x7f90400d3510 0x7f90400b18e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:54394/0 (socket says 172.21.15.5:54394) 2024-05-20T18:29:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.766+0000 7f903f7fe640 1 -- 172.21.15.5:0/310794858 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90400cea20 msgr2=0x7f90400b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:29:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.766+0000 7f903f7fe640 1 --2- 172.21.15.5:0/310794858 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90400cea20 0x7f90400b21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.766+0000 7f903f7fe640 1 -- 172.21.15.5:0/310794858 >> v1:172.21.15.63:6789/0 conn(0x7f90400d3510 legacy=0x7f90400b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.766+0000 7f903f7fe640 1 -- 172.21.15.5:0/310794858 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90400cdce0 con 0x7f90400d1830 2024-05-20T18:29:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.766+0000 7f9044e9b640 1 --2- 172.21.15.5:0/310794858 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90400cea20 0x7f90400b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:29:57.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.767+0000 7f903f7fe640 1 --2- 172.21.15.5:0/310794858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90400d1830 0x7f90400ce4e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f90340097a0 tx=0x7f903402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfc8f850e72ca00a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:57.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.767+0000 7f903effd640 1 -- 172.21.15.5:0/310794858 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034037070 con 0x7f90400d1830 2024-05-20T18:29:57.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.767+0000 7f903effd640 1 -- 172.21.15.5:0/310794858 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f903402ae50 con 0x7f90400d1830 2024-05-20T18:29:57.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.767+0000 7f903effd640 1 -- 172.21.15.5:0/310794858 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034032660 con 0x7f90400d1830 2024-05-20T18:29:57.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.768+0000 7f90468fe640 1 -- 172.21.15.5:0/310794858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90400d1830 msgr2=0x7f90400ce4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:57.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.768+0000 7f90468fe640 1 --2- 172.21.15.5:0/310794858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90400d1830 0x7f90400ce4e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f90340097a0 tx=0x7f903402a100 comp rx=0 tx=0).stop 2024-05-20T18:29:57.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.768+0000 7f90468fe640 1 -- 172.21.15.5:0/310794858 shutdown_connections 2024-05-20T18:29:57.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.768+0000 7f90468fe640 1 --2- 172.21.15.5:0/310794858 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90400cea20 0x7f90400b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:57.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.768+0000 7f90468fe640 1 --2- 172.21.15.5:0/310794858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90400d1830 0x7f90400ce4e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:57.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.768+0000 7f90468fe640 1 -- 172.21.15.5:0/310794858 >> 172.21.15.5:0/310794858 conn(0x7f90400d29f0 msgr2=0x7f90400d2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:57.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.768+0000 7f90468fe640 1 -- 172.21.15.5:0/310794858 shutdown_connections 2024-05-20T18:29:57.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.768+0000 7f90468fe640 1 -- 172.21.15.5:0/310794858 wait complete. 2024-05-20T18:29:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.770+0000 7f90468fe640 1 Processor -- start 2024-05-20T18:29:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.770+0000 7f90468fe640 1 -- start start 2024-05-20T18:29:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.771+0000 7f90468fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90400cea20 0x7f9040151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.771+0000 7f90468fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90400d1830 0x7f90400c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.771+0000 7f90468fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90400d3510 0x7f90400b6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.771+0000 7f90468fe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90400cfd80 con 0x7f90400d3510 2024-05-20T18:29:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.771+0000 7f90468fe640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f90400cd6b0 con 0x7f90400d1830 2024-05-20T18:29:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.771+0000 7f90468fe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f90400cff00 con 0x7f90400cea20 2024-05-20T18:29:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.771+0000 7f9044e9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90400d3510 0x7f90400b6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.771+0000 7f9044e9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90400d3510 0x7f90400b6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33832/0 (socket says 172.21.15.5:33832) 2024-05-20T18:29:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.771+0000 7f9044e9b640 1 -- 172.21.15.5:0/2656813288 learned_addr learned my addr 172.21.15.5:0/2656813288 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.771+0000 7f903f7fe640 1 --2- 172.21.15.5:0/2656813288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90400d1830 0x7f90400c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.772+0000 7f903ffff640 1 --2- 172.21.15.5:0/2656813288 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90400cea20 0x7f9040151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.772+0000 7f9044e9b640 1 -- 172.21.15.5:0/2656813288 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90400cea20 msgr2=0x7f9040151590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.772+0000 7f9044e9b640 1 --2- 172.21.15.5:0/2656813288 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90400cea20 0x7f9040151590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.772+0000 7f9044e9b640 1 -- 172.21.15.5:0/2656813288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90400d1830 msgr2=0x7f90400c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.772+0000 7f9044e9b640 1 --2- 172.21.15.5:0/2656813288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90400d1830 0x7f90400c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.772+0000 7f9044e9b640 1 -- 172.21.15.5:0/2656813288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90400c8fa0 con 0x7f90400d3510 2024-05-20T18:29:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.772+0000 7f9044e9b640 1 --2- 172.21.15.5:0/2656813288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90400d3510 0x7f90400b6e20 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f9038009e70 tx=0x7f903800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.772+0000 7f903cff9640 1 -- 172.21.15.5:0/2656813288 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903800cbe0 con 0x7f90400d3510 2024-05-20T18:29:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.772+0000 7f90468fe640 1 -- 172.21.15.5:0/2656813288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90401f7c20 con 0x7f90400d3510 2024-05-20T18:29:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.772+0000 7f903cff9640 1 -- 172.21.15.5:0/2656813288 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f903800cd80 con 0x7f90400d3510 2024-05-20T18:29:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.772+0000 7f903cff9640 1 -- 172.21.15.5:0/2656813288 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9038003bb0 con 0x7f90400d3510 2024-05-20T18:29:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.772+0000 7f90468fe640 1 -- 172.21.15.5:0/2656813288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90401f7140 con 0x7f90400d3510 2024-05-20T18:29:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.773+0000 7f90468fe640 1 -- 172.21.15.5:0/2656813288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90400690b0 con 0x7f90400d3510 2024-05-20T18:29:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.774+0000 7f903cff9640 1 -- 172.21.15.5:0/2656813288 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f903801b030 con 0x7f90400d3510 2024-05-20T18:29:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.774+0000 7f903cff9640 1 --2- 172.21.15.5:0/2656813288 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f902c06be30 0x7f902c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.774+0000 7f903ffff640 1 --2- 172.21.15.5:0/2656813288 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f902c06be30 0x7f902c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.775+0000 7f903cff9640 1 -- 172.21.15.5:0/2656813288 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5249+0+0 (secure 0 0 0) 0x7f903808aed0 con 0x7f90400d3510 2024-05-20T18:29:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.775+0000 7f903ffff640 1 --2- 172.21.15.5:0/2656813288 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f902c06be30 0x7f902c06e340 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9030007fd0 tx=0x7f9030007e80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:57.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.779+0000 7f903cff9640 1 -- 172.21.15.5:0/2656813288 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9038022020 con 0x7f90400d3510 2024-05-20T18:29:57.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:57.962+0000 7f90468fe640 1 -- 172.21.15.5:0/2656813288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f90400c8d30 con 0x7f90400d3510 2024-05-20T18:29:58.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:58.882+0000 7f903cff9640 1 -- 172.21.15.5:0/2656813288 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v60) ==== 138+0+0 (secure 0 0 0) 0x7f9038010040 con 0x7f90400d3510 2024-05-20T18:29:58.882 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-05-20T18:29:58.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:58.885+0000 7f90468fe640 1 -- 172.21.15.5:0/2656813288 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f902c06be30 msgr2=0x7f902c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:58.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:58.885+0000 7f90468fe640 1 --2- 172.21.15.5:0/2656813288 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f902c06be30 0x7f902c06e340 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9030007fd0 tx=0x7f9030007e80 comp rx=0 tx=0).stop 2024-05-20T18:29:58.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:58.885+0000 7f90468fe640 1 -- 172.21.15.5:0/2656813288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90400d3510 msgr2=0x7f90400b6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:58.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:58.885+0000 7f90468fe640 1 --2- 172.21.15.5:0/2656813288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90400d3510 0x7f90400b6e20 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f9038009e70 tx=0x7f903800c490 comp rx=0 tx=0).stop 2024-05-20T18:29:58.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:58.886+0000 7f90468fe640 1 -- 172.21.15.5:0/2656813288 shutdown_connections 2024-05-20T18:29:58.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:58.886+0000 7f90468fe640 1 --2- 172.21.15.5:0/2656813288 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f902c06be30 0x7f902c06e340 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:58.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:58.886+0000 7f90468fe640 1 --2- 172.21.15.5:0/2656813288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90400d3510 0x7f90400b6e20 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:58.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:58.886+0000 7f90468fe640 1 --2- 172.21.15.5:0/2656813288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90400d1830 0x7f90400c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:58.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:58.886+0000 7f90468fe640 1 --2- 172.21.15.5:0/2656813288 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90400cea20 0x7f9040151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:58.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:58.886+0000 7f90468fe640 1 -- 172.21.15.5:0/2656813288 >> 172.21.15.5:0/2656813288 conn(0x7f90400d29f0 msgr2=0x7f904015cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:58.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:58.886+0000 7f90468fe640 1 -- 172.21.15.5:0/2656813288 shutdown_connections 2024-05-20T18:29:58.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:58.886+0000 7f90468fe640 1 -- 172.21.15.5:0/2656813288 wait complete. 2024-05-20T18:29:58.904 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-20T18:29:59.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.033+0000 7f9878545640 1 Processor -- start 2024-05-20T18:29:59.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.033+0000 7f9878545640 1 -- start start 2024-05-20T18:29:59.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.034+0000 7f9878545640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98700b3930 0x7f98700b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:59.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.034+0000 7f9878545640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98700d05c0 con 0x7f9870163c80 2024-05-20T18:29:59.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.034+0000 7f9878545640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f987006dfe0 con 0x7f987006cf20 2024-05-20T18:29:59.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.034+0000 7f9878545640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f98700d0860 con 0x7f98700b3930 2024-05-20T18:29:59.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.034+0000 7f98762e1640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f9870163c80 0x7f987015f1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:55184/0 (socket says 172.21.15.5:55184) 2024-05-20T18:29:59.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.035+0000 7f98762e1640 1 -- 172.21.15.5:0/1942177209 learned_addr learned my addr 172.21.15.5:0/1942177209 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:29:59.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.035+0000 7f9875ae0640 1 --2- 172.21.15.5:0/1942177209 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98700b3930 0x7f98700b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:59.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.035+0000 7f98752df640 1 -- 172.21.15.5:0/1942177209 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1519686648 0 0) 0x7f98700d05c0 con 0x7f9870163c80 2024-05-20T18:29:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.035+0000 7f98752df640 1 -- 172.21.15.5:0/1942177209 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f985c003520 con 0x7f9870163c80 2024-05-20T18:29:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.035+0000 7f98752df640 1 -- 172.21.15.5:0/1942177209 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3542190988 0 0) 0x7f987006dfe0 con 0x7f987006cf20 2024-05-20T18:29:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.035+0000 7f98752df640 1 -- 172.21.15.5:0/1942177209 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98700d05c0 con 0x7f987006cf20 2024-05-20T18:29:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.035+0000 7f9875ae0640 1 -- 172.21.15.5:0/1942177209 >> v1:172.21.15.63:6789/0 conn(0x7f987006cf20 legacy=0x7f987006d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.035+0000 7f9875ae0640 1 -- 172.21.15.5:0/1942177209 >> v1:172.21.15.5:6789/0 conn(0x7f9870163c80 legacy=0x7f987015f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.035+0000 7f98752df640 1 -- 172.21.15.5:0/1942177209 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3107249661 0 0) 0x7f985c003520 con 0x7f9870163c80 2024-05-20T18:29:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.035+0000 7f9875ae0640 1 -- 172.21.15.5:0/1942177209 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98701631c0 con 0x7f98700b3930 2024-05-20T18:29:59.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.036+0000 7f9875ae0640 1 --2- 172.21.15.5:0/1942177209 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98700b3930 0x7f98700b3d40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f98600097d0 tx=0x7f986002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14209bde012ffe78 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:59.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.036+0000 7f98752df640 1 -- 172.21.15.5:0/1942177209 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860037070 con 0x7f98700b3930 2024-05-20T18:29:59.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.036+0000 7f98752df640 1 -- 172.21.15.5:0/1942177209 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f986002ac30 con 0x7f98700b3930 2024-05-20T18:29:59.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.036+0000 7f98752df640 1 -- 172.21.15.5:0/1942177209 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c001670 con 0x7f98700b3930 2024-05-20T18:29:59.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.037+0000 7f9878545640 1 -- 172.21.15.5:0/1942177209 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98700b3930 msgr2=0x7f98700b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:59.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.037+0000 7f9878545640 1 --2- 172.21.15.5:0/1942177209 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98700b3930 0x7f98700b3d40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f98600097d0 tx=0x7f986002a1a0 comp rx=0 tx=0).stop 2024-05-20T18:29:59.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.037+0000 7f9878545640 1 -- 172.21.15.5:0/1942177209 shutdown_connections 2024-05-20T18:29:59.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.037+0000 7f9878545640 1 --2- 172.21.15.5:0/1942177209 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98700b3930 0x7f98700b3d40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:59.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.037+0000 7f9878545640 1 -- 172.21.15.5:0/1942177209 >> 172.21.15.5:0/1942177209 conn(0x7f98700d4bd0 msgr2=0x7f98700d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:59.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.037+0000 7f9878545640 1 -- 172.21.15.5:0/1942177209 shutdown_connections 2024-05-20T18:29:59.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.037+0000 7f9878545640 1 -- 172.21.15.5:0/1942177209 wait complete. 2024-05-20T18:29:59.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.039+0000 7f9878545640 1 Processor -- start 2024-05-20T18:29:59.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.039+0000 7f9878545640 1 -- start start 2024-05-20T18:29:59.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.040+0000 7f9878545640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f987006cf20 0x7f987014d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:59.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.040+0000 7f9878545640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98700b3930 0x7f98701498b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:59.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.040+0000 7f9878545640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9870163c80 0x7f9870158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:59.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.040+0000 7f9878545640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f987015f8e0 con 0x7f9870163c80 2024-05-20T18:29:59.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.040+0000 7f9878545640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9870163780 con 0x7f987006cf20 2024-05-20T18:29:59.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.040+0000 7f9878545640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f987016a350 con 0x7f98700b3930 2024-05-20T18:29:59.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.040+0000 7f9876ae2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9870163c80 0x7f9870158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:59.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.040+0000 7f9876ae2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9870163c80 0x7f9870158330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33838/0 (socket says 172.21.15.5:33838) 2024-05-20T18:29:59.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.040+0000 7f9876ae2640 1 -- 172.21.15.5:0/2647124623 learned_addr learned my addr 172.21.15.5:0/2647124623 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:29:59.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.040+0000 7f98762e1640 1 --2- 172.21.15.5:0/2647124623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f987006cf20 0x7f987014d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:59.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.040+0000 7f9875ae0640 1 --2- 172.21.15.5:0/2647124623 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98700b3930 0x7f98701498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:59.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.041+0000 7f9876ae2640 1 -- 172.21.15.5:0/2647124623 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98700b3930 msgr2=0x7f98701498b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:59.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.041+0000 7f9876ae2640 1 --2- 172.21.15.5:0/2647124623 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98700b3930 0x7f98701498b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:59.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.041+0000 7f9876ae2640 1 -- 172.21.15.5:0/2647124623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f987006cf20 msgr2=0x7f987014d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:59.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.041+0000 7f9876ae2640 1 --2- 172.21.15.5:0/2647124623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f987006cf20 0x7f987014d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:59.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.041+0000 7f9876ae2640 1 -- 172.21.15.5:0/2647124623 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98701f94f0 con 0x7f9870163c80 2024-05-20T18:29:59.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.041+0000 7f9876ae2640 1 --2- 172.21.15.5:0/2647124623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9870163c80 0x7f9870158330 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f986800c6b0 tx=0x7f986800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:59.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.041+0000 7f9866ffd640 1 -- 172.21.15.5:0/2647124623 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9868003b70 con 0x7f9870163c80 2024-05-20T18:29:59.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.041+0000 7f9866ffd640 1 -- 172.21.15.5:0/2647124623 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9868003d10 con 0x7f9870163c80 2024-05-20T18:29:59.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.041+0000 7f9878545640 1 -- 172.21.15.5:0/2647124623 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98701f97d0 con 0x7f9870163c80 2024-05-20T18:29:59.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.041+0000 7f9866ffd640 1 -- 172.21.15.5:0/2647124623 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9868005310 con 0x7f9870163c80 2024-05-20T18:29:59.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.041+0000 7f9878545640 1 -- 172.21.15.5:0/2647124623 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9870149df0 con 0x7f9870163c80 2024-05-20T18:29:59.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.042+0000 7f9878545640 1 -- 172.21.15.5:0/2647124623 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98701f9240 con 0x7f9870163c80 2024-05-20T18:29:59.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.043+0000 7f9866ffd640 1 -- 172.21.15.5:0/2647124623 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f98680054b0 con 0x7f9870163c80 2024-05-20T18:29:59.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.043+0000 7f9866ffd640 1 --2- 172.21.15.5:0/2647124623 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f985406be60 0x7f985406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:29:59.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.044+0000 7f9866ffd640 1 -- 172.21.15.5:0/2647124623 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5619+0+0 (secure 0 0 0) 0x7f986808ac40 con 0x7f9870163c80 2024-05-20T18:29:59.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.044+0000 7f98762e1640 1 --2- 172.21.15.5:0/2647124623 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f985406be60 0x7f985406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:29:59.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.044+0000 7f98762e1640 1 --2- 172.21.15.5:0/2647124623 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f985406be60 0x7f985406e370 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f986c0017e0 tx=0x7f986c001610 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:29:59.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.049+0000 7f9866ffd640 1 -- 172.21.15.5:0/2647124623 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9868058f70 con 0x7f9870163c80 2024-05-20T18:29:59.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.235+0000 7f9878545640 1 -- 172.21.15.5:0/2647124623 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f98700690b0 con 0x7f9870163c80 2024-05-20T18:29:59.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.890+0000 7f9866ffd640 1 -- 172.21.15.5:0/2647124623 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v61) ==== 170+0+0 (secure 0 0 0) 0x7f986805d270 con 0x7f9870163c80 2024-05-20T18:29:59.891 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-05-20T18:29:59.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.894+0000 7f9878545640 1 -- 172.21.15.5:0/2647124623 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f985406be60 msgr2=0x7f985406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:59.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.894+0000 7f9878545640 1 --2- 172.21.15.5:0/2647124623 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f985406be60 0x7f985406e370 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f986c0017e0 tx=0x7f986c001610 comp rx=0 tx=0).stop 2024-05-20T18:29:59.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.894+0000 7f9878545640 1 -- 172.21.15.5:0/2647124623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9870163c80 msgr2=0x7f9870158330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:29:59.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.894+0000 7f9878545640 1 --2- 172.21.15.5:0/2647124623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9870163c80 0x7f9870158330 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f986800c6b0 tx=0x7f986800cb70 comp rx=0 tx=0).stop 2024-05-20T18:29:59.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.895+0000 7f9878545640 1 -- 172.21.15.5:0/2647124623 shutdown_connections 2024-05-20T18:29:59.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.895+0000 7f9878545640 1 --2- 172.21.15.5:0/2647124623 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f985406be60 0x7f985406e370 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:59.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.895+0000 7f9878545640 1 --2- 172.21.15.5:0/2647124623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9870163c80 0x7f9870158330 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:59.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.895+0000 7f9878545640 1 --2- 172.21.15.5:0/2647124623 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98700b3930 0x7f98701498b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:59.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.895+0000 7f9878545640 1 --2- 172.21.15.5:0/2647124623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f987006cf20 0x7f987014d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:29:59.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.895+0000 7f9878545640 1 -- 172.21.15.5:0/2647124623 >> 172.21.15.5:0/2647124623 conn(0x7f98700d4bd0 msgr2=0x7f9870156b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:29:59.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.895+0000 7f9878545640 1 -- 172.21.15.5:0/2647124623 shutdown_connections 2024-05-20T18:29:59.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:29:59.896+0000 7f9878545640 1 -- 172.21.15.5:0/2647124623 wait complete. 2024-05-20T18:29:59.924 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-20T18:30:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.047+0000 7fd4d785a640 1 Processor -- start 2024-05-20T18:30:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.047+0000 7fd4d785a640 1 -- start start 2024-05-20T18:30:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.048+0000 7fd4d785a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd4d0162820 0x7fd4d00b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.048+0000 7fd4d785a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd4d00b3e80 0x7fd4d006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.048+0000 7fd4d785a640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4d006dd20 con 0x7fd4d006d910 2024-05-20T18:30:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.048+0000 7fd4d785a640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fd4d00d0630 con 0x7fd4d0162820 2024-05-20T18:30:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.048+0000 7fd4d785a640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd4d00d39c0 con 0x7fd4d00b3e80 2024-05-20T18:30:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.049+0000 7fd4d55f6640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd4d0162820 0x7fd4d00b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.049+0000 7fd4d55f6640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd4d0162820 0x7fd4d00b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33848/0 (socket says 172.21.15.5:33848) 2024-05-20T18:30:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.049+0000 7fd4d55f6640 1 -- 172.21.15.5:0/633772611 learned_addr learned my addr 172.21.15.5:0/633772611 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.049+0000 7fd4d4df5640 1 --2- 172.21.15.5:0/633772611 >> v2:172.21.15.63:3300/0 conn(0x7fd4d00b3e80 0x7fd4d006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:00.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.049+0000 7fd4d55f6640 1 -- 172.21.15.5:0/633772611 >> v2:172.21.15.63:3300/0 conn(0x7fd4d00b3e80 msgr2=0x7fd4d006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:00.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.049+0000 7fd4d55f6640 1 --2- 172.21.15.5:0/633772611 >> v2:172.21.15.63:3300/0 conn(0x7fd4d00b3e80 0x7fd4d006d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:00.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.049+0000 7fd4d55f6640 1 -- 172.21.15.5:0/633772611 >> v1:172.21.15.63:6789/0 conn(0x7fd4d006d910 legacy=0x7fd4d00d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:00.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.049+0000 7fd4d55f6640 1 -- 172.21.15.5:0/633772611 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d00d3ca0 con 0x7fd4d0162820 2024-05-20T18:30:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.050+0000 7fd4d55f6640 1 --2- 172.21.15.5:0/633772611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d0162820 0x7fd4d00b3940 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00d1230 tx=0x7fd4cc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f2209d1cadfab11 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.050+0000 7fd4c7fff640 1 -- 172.21.15.5:0/633772611 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc037070 con 0x7fd4d0162820 2024-05-20T18:30:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.050+0000 7fd4c7fff640 1 -- 172.21.15.5:0/633772611 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4cc032440 con 0x7fd4d0162820 2024-05-20T18:30:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.050+0000 7fd4c7fff640 1 -- 172.21.15.5:0/633772611 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc032810 con 0x7fd4d0162820 2024-05-20T18:30:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.051+0000 7fd4d785a640 1 -- 172.21.15.5:0/633772611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d0162820 msgr2=0x7fd4d00b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.051+0000 7fd4d785a640 1 --2- 172.21.15.5:0/633772611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d0162820 0x7fd4d00b3940 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00d1230 tx=0x7fd4cc02a230 comp rx=0 tx=0).stop 2024-05-20T18:30:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.051+0000 7fd4d785a640 1 -- 172.21.15.5:0/633772611 shutdown_connections 2024-05-20T18:30:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.051+0000 7fd4d785a640 1 --2- 172.21.15.5:0/633772611 >> v2:172.21.15.63:3300/0 conn(0x7fd4d00b3e80 0x7fd4d006d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.051+0000 7fd4d785a640 1 --2- 172.21.15.5:0/633772611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d0162820 0x7fd4d00b3940 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.051+0000 7fd4d785a640 1 -- 172.21.15.5:0/633772611 >> 172.21.15.5:0/633772611 conn(0x7fd4d00d4bd0 msgr2=0x7fd4d00d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.051+0000 7fd4d785a640 1 -- 172.21.15.5:0/633772611 shutdown_connections 2024-05-20T18:30:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.051+0000 7fd4d785a640 1 -- 172.21.15.5:0/633772611 wait complete. 2024-05-20T18:30:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.053+0000 7fd4d785a640 1 Processor -- start 2024-05-20T18:30:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.053+0000 7fd4d785a640 1 -- start start 2024-05-20T18:30:00.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.054+0000 7fd4d785a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d006d910 0x7fd4d00c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:00.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.054+0000 7fd4d785a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4d00b3e80 0x7fd4d014b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:00.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.054+0000 7fd4d785a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4d0162820 0x7fd4d01e79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:00.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.054+0000 7fd4d785a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d00d0300 con 0x7fd4d006d910 2024-05-20T18:30:00.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.054+0000 7fd4d785a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d00d4260 con 0x7fd4d00b3e80 2024-05-20T18:30:00.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.054+0000 7fd4d785a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd4d00b91a0 con 0x7fd4d0162820 2024-05-20T18:30:00.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.054+0000 7fd4d55f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d006d910 0x7fd4d00c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:00.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.054+0000 7fd4d55f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d006d910 0x7fd4d00c8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33852/0 (socket says 172.21.15.5:33852) 2024-05-20T18:30:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.054+0000 7fd4d55f6640 1 -- 172.21.15.5:0/2957061873 learned_addr learned my addr 172.21.15.5:0/2957061873 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.054+0000 7fd4d4df5640 1 --2- 172.21.15.5:0/2957061873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4d00b3e80 0x7fd4d014b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.054+0000 7fd4d5df7640 1 --2- 172.21.15.5:0/2957061873 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4d0162820 0x7fd4d01e79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.054+0000 7fd4d55f6640 1 -- 172.21.15.5:0/2957061873 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4d0162820 msgr2=0x7fd4d01e79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.054+0000 7fd4d55f6640 1 --2- 172.21.15.5:0/2957061873 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4d0162820 0x7fd4d01e79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.054+0000 7fd4d55f6640 1 -- 172.21.15.5:0/2957061873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4d00b3e80 msgr2=0x7fd4d014b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.054+0000 7fd4d55f6640 1 --2- 172.21.15.5:0/2957061873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4d00b3e80 0x7fd4d014b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.054+0000 7fd4d55f6640 1 -- 172.21.15.5:0/2957061873 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d014a2b0 con 0x7fd4d006d910 2024-05-20T18:30:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.054+0000 7fd4d5df7640 1 --2- 172.21.15.5:0/2957061873 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4d0162820 0x7fd4d01e79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:30:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.055+0000 7fd4d55f6640 1 --2- 172.21.15.5:0/2957061873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d006d910 0x7fd4d00c8700 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc000c00 tx=0x7fd4cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.055+0000 7fd4c5ffb640 1 -- 172.21.15.5:0/2957061873 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc041020 con 0x7fd4d006d910 2024-05-20T18:30:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.055+0000 7fd4c5ffb640 1 -- 172.21.15.5:0/2957061873 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4cc0306c0 con 0x7fd4d006d910 2024-05-20T18:30:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.055+0000 7fd4d785a640 1 -- 172.21.15.5:0/2957061873 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d014a590 con 0x7fd4d006d910 2024-05-20T18:30:00.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.055+0000 7fd4c5ffb640 1 -- 172.21.15.5:0/2957061873 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc037040 con 0x7fd4d006d910 2024-05-20T18:30:00.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.055+0000 7fd4d785a640 1 -- 172.21.15.5:0/2957061873 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d014ab50 con 0x7fd4d006d910 2024-05-20T18:30:00.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.056+0000 7fd4d785a640 1 -- 172.21.15.5:0/2957061873 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4d0075b00 con 0x7fd4d006d910 2024-05-20T18:30:00.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.057+0000 7fd4c5ffb640 1 -- 172.21.15.5:0/2957061873 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd4cc044020 con 0x7fd4d006d910 2024-05-20T18:30:00.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.057+0000 7fd4c5ffb640 1 --2- 172.21.15.5:0/2957061873 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd4b406be60 0x7fd4b406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:00.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.058+0000 7fd4d4df5640 1 --2- 172.21.15.5:0/2957061873 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd4b406be60 0x7fd4b406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:00.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.058+0000 7fd4c5ffb640 1 -- 172.21.15.5:0/2957061873 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6001+0+0 (secure 0 0 0) 0x7fd4cc0ab870 con 0x7fd4d006d910 2024-05-20T18:30:00.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.058+0000 7fd4d4df5640 1 --2- 172.21.15.5:0/2957061873 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd4b406be60 0x7fd4b406e370 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00cb870 tx=0x7fd4c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:00.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.063+0000 7fd4c5ffb640 1 -- 172.21.15.5:0/2957061873 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd4cc04b030 con 0x7fd4d006d910 2024-05-20T18:30:00.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.284+0000 7fd4d785a640 1 -- 172.21.15.5:0/2957061873 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd4d00cf3a0 con 0x7fd4d006d910 2024-05-20T18:30:00.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.906+0000 7fd4c5ffb640 1 -- 172.21.15.5:0/2957061873 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v30) ==== 173+0+0 (secure 0 0 0) 0x7fd4d006dd20 con 0x7fd4d006d910 2024-05-20T18:30:00.907 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 9 and data pool 10 2024-05-20T18:30:00.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.915+0000 7fd4d785a640 1 -- 172.21.15.5:0/2957061873 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd4b406be60 msgr2=0x7fd4b406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:00.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.915+0000 7fd4d785a640 1 --2- 172.21.15.5:0/2957061873 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd4b406be60 0x7fd4b406e370 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00cb870 tx=0x7fd4c0008040 comp rx=0 tx=0).stop 2024-05-20T18:30:00.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.915+0000 7fd4d785a640 1 -- 172.21.15.5:0/2957061873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d006d910 msgr2=0x7fd4d00c8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:00.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.915+0000 7fd4d785a640 1 --2- 172.21.15.5:0/2957061873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d006d910 0x7fd4d00c8700 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc000c00 tx=0x7fd4cc0046c0 comp rx=0 tx=0).stop 2024-05-20T18:30:00.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.919+0000 7fd4d785a640 1 -- 172.21.15.5:0/2957061873 shutdown_connections 2024-05-20T18:30:00.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.919+0000 7fd4d785a640 1 --2- 172.21.15.5:0/2957061873 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd4b406be60 0x7fd4b406e370 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:00.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.919+0000 7fd4d785a640 1 --2- 172.21.15.5:0/2957061873 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd4d0162820 0x7fd4d01e79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:00.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.919+0000 7fd4d785a640 1 --2- 172.21.15.5:0/2957061873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4d00b3e80 0x7fd4d014b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:00.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.919+0000 7fd4d785a640 1 --2- 172.21.15.5:0/2957061873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d006d910 0x7fd4d00c8700 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:00.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.919+0000 7fd4d785a640 1 -- 172.21.15.5:0/2957061873 >> 172.21.15.5:0/2957061873 conn(0x7fd4d00d4bd0 msgr2=0x7fd4d00b4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:00.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.921+0000 7fd4d785a640 1 -- 172.21.15.5:0/2957061873 shutdown_connections 2024-05-20T18:30:00.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:00.921+0000 7fd4d785a640 1 -- 172.21.15.5:0/2957061873 wait complete. 2024-05-20T18:30:00.940 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:30:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.067+0000 7fa6ef309640 1 Processor -- start 2024-05-20T18:30:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.067+0000 7fa6ef309640 1 -- start start 2024-05-20T18:30:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.068+0000 7fa6ef309640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6e815d530 0x7fa6e815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.068+0000 7fa6ef309640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa6e8070850 0x7fa6e8070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.068+0000 7fa6ef309640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6e80cd580 con 0x7fa6e80b1700 2024-05-20T18:30:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.068+0000 7fa6ef309640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa6e80ce190 con 0x7fa6e8070850 2024-05-20T18:30:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.068+0000 7fa6ef309640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa6e80cd860 con 0x7fa6e815d530 2024-05-20T18:30:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.069+0000 7fa6ec8a4640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa6e80b1700 0x7fa6e80b1b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:55200/0 (socket says 172.21.15.5:55200) 2024-05-20T18:30:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.069+0000 7fa6ec8a4640 1 -- 172.21.15.5:0/2961309174 learned_addr learned my addr 172.21.15.5:0/2961309174 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:30:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.069+0000 7fa6ed0a5640 1 --2- 172.21.15.5:0/2961309174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6e815d530 0x7fa6e815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.069+0000 7fa6ed8a6640 1 --2- 172.21.15.5:0/2961309174 >> v2:172.21.15.63:3300/0 conn(0x7fa6e8070850 0x7fa6e8070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.069+0000 7fa6dffff640 1 -- 172.21.15.5:0/2961309174 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1400995296 0 0) 0x7fa6e80cd580 con 0x7fa6e80b1700 2024-05-20T18:30:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.069+0000 7fa6dffff640 1 -- 172.21.15.5:0/2961309174 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6d4003520 con 0x7fa6e80b1700 2024-05-20T18:30:01.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.069+0000 7fa6ed0a5640 1 -- 172.21.15.5:0/2961309174 >> v2:172.21.15.63:3300/0 conn(0x7fa6e8070850 msgr2=0x7fa6e8070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:01.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.069+0000 7fa6ed0a5640 1 --2- 172.21.15.5:0/2961309174 >> v2:172.21.15.63:3300/0 conn(0x7fa6e8070850 0x7fa6e8070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:01.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.069+0000 7fa6ed0a5640 1 -- 172.21.15.5:0/2961309174 >> v1:172.21.15.5:6789/0 conn(0x7fa6e80b1700 legacy=0x7fa6e80b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:01.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.069+0000 7fa6ed0a5640 1 -- 172.21.15.5:0/2961309174 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e80cdb40 con 0x7fa6e815d530 2024-05-20T18:30:01.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.070+0000 7fa6ed0a5640 1 --2- 172.21.15.5:0/2961309174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6e815d530 0x7fa6e815aaa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40097a0 tx=0x7fa6e402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60b453ec3b13b1a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:01.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.070+0000 7fa6dffff640 1 -- 172.21.15.5:0/2961309174 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4037070 con 0x7fa6e815d530 2024-05-20T18:30:01.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.070+0000 7fa6dffff640 1 -- 172.21.15.5:0/2961309174 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6e4032440 con 0x7fa6e815d530 2024-05-20T18:30:01.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.070+0000 7fa6dffff640 1 -- 172.21.15.5:0/2961309174 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4032810 con 0x7fa6e815d530 2024-05-20T18:30:01.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.071+0000 7fa6ef309640 1 -- 172.21.15.5:0/2961309174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6e815d530 msgr2=0x7fa6e815aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:01.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.071+0000 7fa6ef309640 1 --2- 172.21.15.5:0/2961309174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6e815d530 0x7fa6e815aaa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40097a0 tx=0x7fa6e402a180 comp rx=0 tx=0).stop 2024-05-20T18:30:01.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.071+0000 7fa6ef309640 1 -- 172.21.15.5:0/2961309174 shutdown_connections 2024-05-20T18:30:01.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.071+0000 7fa6ef309640 1 --2- 172.21.15.5:0/2961309174 >> v2:172.21.15.63:3300/0 conn(0x7fa6e8070850 0x7fa6e8070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:01.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.071+0000 7fa6ef309640 1 --2- 172.21.15.5:0/2961309174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6e815d530 0x7fa6e815aaa0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:01.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.071+0000 7fa6ef309640 1 -- 172.21.15.5:0/2961309174 >> 172.21.15.5:0/2961309174 conn(0x7fa6e80d29d0 msgr2=0x7fa6e80d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:01.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.071+0000 7fa6ef309640 1 -- 172.21.15.5:0/2961309174 shutdown_connections 2024-05-20T18:30:01.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.071+0000 7fa6ef309640 1 -- 172.21.15.5:0/2961309174 wait complete. 2024-05-20T18:30:01.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.073+0000 7fa6ef309640 1 Processor -- start 2024-05-20T18:30:01.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.073+0000 7fa6ef309640 1 -- start start 2024-05-20T18:30:01.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.075+0000 7fa6ef309640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6e8070850 0x7fa6e80ca440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:01.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.075+0000 7fa6ef309640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6e80b1700 0x7fa6e80b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:01.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.075+0000 7fa6ef309640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6e815d530 0x7fa6e814e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:01.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.075+0000 7fa6ef309640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e80711a0 con 0x7fa6e815d530 2024-05-20T18:30:01.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.075+0000 7fa6ef309640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e80cd230 con 0x7fa6e80b1700 2024-05-20T18:30:01.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.075+0000 7fa6ef309640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa6e8071320 con 0x7fa6e8070850 2024-05-20T18:30:01.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.075+0000 7fa6ed8a6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6e815d530 0x7fa6e814e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:01.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.075+0000 7fa6ed8a6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6e815d530 0x7fa6e814e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33864/0 (socket says 172.21.15.5:33864) 2024-05-20T18:30:01.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.075+0000 7fa6ed8a6640 1 -- 172.21.15.5:0/2588357493 learned_addr learned my addr 172.21.15.5:0/2588357493 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:01.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.075+0000 7fa6ec8a4640 1 --2- 172.21.15.5:0/2588357493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6e80b1700 0x7fa6e80b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:01.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.075+0000 7fa6ed0a5640 1 --2- 172.21.15.5:0/2588357493 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6e8070850 0x7fa6e80ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:01.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.075+0000 7fa6ed8a6640 1 -- 172.21.15.5:0/2588357493 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6e8070850 msgr2=0x7fa6e80ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.075+0000 7fa6ed8a6640 1 --2- 172.21.15.5:0/2588357493 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6e8070850 0x7fa6e80ca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.075+0000 7fa6ed8a6640 1 -- 172.21.15.5:0/2588357493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6e80b1700 msgr2=0x7fa6e80b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.075+0000 7fa6ed8a6640 1 --2- 172.21.15.5:0/2588357493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6e80b1700 0x7fa6e80b0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.075+0000 7fa6ed8a6640 1 -- 172.21.15.5:0/2588357493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e80cae50 con 0x7fa6e815d530 2024-05-20T18:30:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.075+0000 7fa6ed8a6640 1 --2- 172.21.15.5:0/2588357493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6e815d530 0x7fa6e814e900 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa6e000efc0 tx=0x7fa6e000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.076+0000 7fa6ddffb640 1 -- 172.21.15.5:0/2588357493 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0018020 con 0x7fa6e815d530 2024-05-20T18:30:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.076+0000 7fa6ddffb640 1 -- 172.21.15.5:0/2588357493 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6e0014480 con 0x7fa6e815d530 2024-05-20T18:30:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.076+0000 7fa6ef309640 1 -- 172.21.15.5:0/2588357493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6e80cb130 con 0x7fa6e815d530 2024-05-20T18:30:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.076+0000 7fa6ddffb640 1 -- 172.21.15.5:0/2588357493 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0009e60 con 0x7fa6e815d530 2024-05-20T18:30:01.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.076+0000 7fa6ef309640 1 -- 172.21.15.5:0/2588357493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6e80cb6f0 con 0x7fa6e815d530 2024-05-20T18:30:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.077+0000 7fa6ef309640 1 -- 172.21.15.5:0/2588357493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6e8075b00 con 0x7fa6e815d530 2024-05-20T18:30:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.078+0000 7fa6ddffb640 1 -- 172.21.15.5:0/2588357493 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa6e0005d10 con 0x7fa6e815d530 2024-05-20T18:30:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.078+0000 7fa6ddffb640 1 --2- 172.21.15.5:0/2588357493 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa6cc06be60 0x7fa6cc06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.079+0000 7fa6ddffb640 1 -- 172.21.15.5:0/2588357493 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6101+0+0 (secure 0 0 0) 0x7fa6e008bc20 con 0x7fa6e815d530 2024-05-20T18:30:01.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.079+0000 7fa6ed0a5640 1 --2- 172.21.15.5:0/2588357493 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa6cc06be60 0x7fa6cc06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:01.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.079+0000 7fa6ed0a5640 1 --2- 172.21.15.5:0/2588357493 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa6cc06be60 0x7fa6cc06e370 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4034000 tx=0x7fa6e402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.084+0000 7fa6ddffb640 1 -- 172.21.15.5:0/2588357493 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa6e0022030 con 0x7fa6e815d530 2024-05-20T18:30:01.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.274+0000 7fa6ef309640 1 -- 172.21.15.5:0/2588357493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6e80b2c50 con 0x7fa6e815d530 2024-05-20T18:30:01.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.275+0000 7fa6ddffb640 1 -- 172.21.15.5:0/2588357493 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+16777 (secure 0 0 0) 0x7fa6e0010040 con 0x7fa6e815d530 2024-05-20T18:30:01.275 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:30:01.275 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":62,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:30:00.286233+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:29:57.993861+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-20T18:29:59.267582+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:30:01.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.277+0000 7fa6ef309640 1 -- 172.21.15.5:0/2588357493 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa6cc06be60 msgr2=0x7fa6cc06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:01.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.277+0000 7fa6ef309640 1 --2- 172.21.15.5:0/2588357493 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa6cc06be60 0x7fa6cc06e370 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4034000 tx=0x7fa6e402e000 comp rx=0 tx=0).stop 2024-05-20T18:30:01.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.277+0000 7fa6ef309640 1 -- 172.21.15.5:0/2588357493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6e815d530 msgr2=0x7fa6e814e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:01.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.277+0000 7fa6ef309640 1 --2- 172.21.15.5:0/2588357493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6e815d530 0x7fa6e814e900 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa6e000efc0 tx=0x7fa6e000c490 comp rx=0 tx=0).stop 2024-05-20T18:30:01.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.277+0000 7fa6ef309640 1 -- 172.21.15.5:0/2588357493 shutdown_connections 2024-05-20T18:30:01.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.277+0000 7fa6ef309640 1 --2- 172.21.15.5:0/2588357493 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa6cc06be60 0x7fa6cc06e370 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:01.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.277+0000 7fa6ef309640 1 --2- 172.21.15.5:0/2588357493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6e815d530 0x7fa6e814e900 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:01.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.277+0000 7fa6ef309640 1 --2- 172.21.15.5:0/2588357493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6e80b1700 0x7fa6e80b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:01.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.277+0000 7fa6ef309640 1 --2- 172.21.15.5:0/2588357493 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6e8070850 0x7fa6e80ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:01.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.277+0000 7fa6ef309640 1 -- 172.21.15.5:0/2588357493 >> 172.21.15.5:0/2588357493 conn(0x7fa6e80d29d0 msgr2=0x7fa6e80b7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:01.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.277+0000 7fa6ef309640 1 -- 172.21.15.5:0/2588357493 shutdown_connections 2024-05-20T18:30:01.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.277+0000 7fa6ef309640 1 -- 172.21.15.5:0/2588357493 wait complete. 2024-05-20T18:30:01.289 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:30:01.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.404+0000 7f4f62c75640 1 Processor -- start 2024-05-20T18:30:01.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.404+0000 7f4f62c75640 1 -- start start 2024-05-20T18:30:01.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.406+0000 7f4f62c75640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4f5c0cf3d0 0x7f4f5c0cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:01.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.406+0000 7f4f62c75640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f5c0cfd20 con 0x7f4f5c0ce7e0 2024-05-20T18:30:01.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.406+0000 7f4f62c75640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f5c0cda20 con 0x7f4f5c15f220 2024-05-20T18:30:01.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.406+0000 7f4f62c75640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f4f5c0cdd00 con 0x7f4f5c0cf3d0 2024-05-20T18:30:01.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.406+0000 7f4f61212640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f4f5c0ce7e0 0x7f4f5c0cebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:55206/0 (socket says 172.21.15.5:55206) 2024-05-20T18:30:01.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.406+0000 7f4f61212640 1 -- 172.21.15.5:0/552583179 learned_addr learned my addr 172.21.15.5:0/552583179 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:30:01.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.406+0000 7f4f5bfff640 1 --2- 172.21.15.5:0/552583179 >> v2:172.21.15.5:3300/0 conn(0x7f4f5c0cf3d0 0x7f4f5c0cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:01.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.406+0000 7f4f5bfff640 1 -- 172.21.15.5:0/552583179 >> v1:172.21.15.63:6789/0 conn(0x7f4f5c15f220 legacy=0x7f4f5c0d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:01.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.406+0000 7f4f5b7fe640 1 -- 172.21.15.5:0/552583179 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1734512052 0 0) 0x7f4f5c0cda20 con 0x7f4f5c15f220 2024-05-20T18:30:01.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.406+0000 7f4f5bfff640 1 -- 172.21.15.5:0/552583179 >> v1:172.21.15.5:6789/0 conn(0x7f4f5c0ce7e0 legacy=0x7f4f5c0cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:01.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.406+0000 7f4f5bfff640 1 -- 172.21.15.5:0/552583179 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f5c0cdfe0 con 0x7f4f5c0cf3d0 2024-05-20T18:30:01.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.407+0000 7f4f5bfff640 1 --2- 172.21.15.5:0/552583179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f5c0cf3d0 0x7f4f5c0cf7e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f4f500097a0 tx=0x7f4f5002a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e1af35efe9058c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:01.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.407+0000 7f4f5b7fe640 1 -- 172.21.15.5:0/552583179 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50037070 con 0x7f4f5c0cf3d0 2024-05-20T18:30:01.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.407+0000 7f4f5b7fe640 1 -- 172.21.15.5:0/552583179 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f5002ad30 con 0x7f4f5c0cf3d0 2024-05-20T18:30:01.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.407+0000 7f4f5b7fe640 1 -- 172.21.15.5:0/552583179 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50032660 con 0x7f4f5c0cf3d0 2024-05-20T18:30:01.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.408+0000 7f4f62c75640 1 -- 172.21.15.5:0/552583179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f5c0cf3d0 msgr2=0x7f4f5c0cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:01.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.408+0000 7f4f62c75640 1 --2- 172.21.15.5:0/552583179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f5c0cf3d0 0x7f4f5c0cf7e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f4f500097a0 tx=0x7f4f5002a0a0 comp rx=0 tx=0).stop 2024-05-20T18:30:01.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.408+0000 7f4f62c75640 1 -- 172.21.15.5:0/552583179 shutdown_connections 2024-05-20T18:30:01.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.408+0000 7f4f62c75640 1 --2- 172.21.15.5:0/552583179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f5c0cf3d0 0x7f4f5c0cf7e0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:01.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.408+0000 7f4f62c75640 1 -- 172.21.15.5:0/552583179 >> 172.21.15.5:0/552583179 conn(0x7f4f5c0b44d0 msgr2=0x7f4f5c0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:01.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.408+0000 7f4f62c75640 1 -- 172.21.15.5:0/552583179 shutdown_connections 2024-05-20T18:30:01.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.408+0000 7f4f62c75640 1 -- 172.21.15.5:0/552583179 wait complete. 2024-05-20T18:30:01.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.410+0000 7f4f62c75640 1 Processor -- start 2024-05-20T18:30:01.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.410+0000 7f4f62c75640 1 -- start start 2024-05-20T18:30:01.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.411+0000 7f4f62c75640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f5c0ce7e0 0x7f4f5c1f8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:01.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.411+0000 7f4f62c75640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f5c0cf3d0 0x7f4f5c0ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:01.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.411+0000 7f4f62c75640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f5c15f220 0x7f4f5c15d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:01.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.411+0000 7f4f62c75640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5c0cd680 con 0x7f4f5c0ce7e0 2024-05-20T18:30:01.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.411+0000 7f4f62c75640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5c0d0fd0 con 0x7f4f5c15f220 2024-05-20T18:30:01.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.411+0000 7f4f62c75640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4f5c0cd800 con 0x7f4f5c0cf3d0 2024-05-20T18:30:01.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.411+0000 7f4f60a11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f5c0ce7e0 0x7f4f5c1f8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:01.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.411+0000 7f4f60a11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f5c0ce7e0 0x7f4f5c1f8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33874/0 (socket says 172.21.15.5:33874) 2024-05-20T18:30:01.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.411+0000 7f4f60a11640 1 -- 172.21.15.5:0/3263446677 learned_addr learned my addr 172.21.15.5:0/3263446677 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:01.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.412+0000 7f4f61212640 1 --2- 172.21.15.5:0/3263446677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f5c15f220 0x7f4f5c15d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:01.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.412+0000 7f4f5bfff640 1 --2- 172.21.15.5:0/3263446677 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f5c0cf3d0 0x7f4f5c0ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:01.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.412+0000 7f4f60a11640 1 -- 172.21.15.5:0/3263446677 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f5c0cf3d0 msgr2=0x7f4f5c0ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:01.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.412+0000 7f4f60a11640 1 --2- 172.21.15.5:0/3263446677 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f5c0cf3d0 0x7f4f5c0ca5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:01.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.412+0000 7f4f60a11640 1 -- 172.21.15.5:0/3263446677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f5c15f220 msgr2=0x7f4f5c15d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:01.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.412+0000 7f4f60a11640 1 --2- 172.21.15.5:0/3263446677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f5c15f220 0x7f4f5c15d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:01.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.412+0000 7f4f60a11640 1 -- 172.21.15.5:0/3263446677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f5c0b6800 con 0x7f4f5c0ce7e0 2024-05-20T18:30:01.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.412+0000 7f4f60a11640 1 --2- 172.21.15.5:0/3263446677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f5c0ce7e0 0x7f4f5c1f8710 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00c3e0 tx=0x7f4f4c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:01.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.412+0000 7f4f597fa640 1 -- 172.21.15.5:0/3263446677 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c017070 con 0x7f4f5c0ce7e0 2024-05-20T18:30:01.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.412+0000 7f4f597fa640 1 -- 172.21.15.5:0/3263446677 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f4c0035f0 con 0x7f4f5c0ce7e0 2024-05-20T18:30:01.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.412+0000 7f4f62c75640 1 -- 172.21.15.5:0/3263446677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f5c147df0 con 0x7f4f5c0ce7e0 2024-05-20T18:30:01.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.412+0000 7f4f597fa640 1 -- 172.21.15.5:0/3263446677 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c003a00 con 0x7f4f5c0ce7e0 2024-05-20T18:30:01.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.412+0000 7f4f62c75640 1 -- 172.21.15.5:0/3263446677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f5c1483b0 con 0x7f4f5c0ce7e0 2024-05-20T18:30:01.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.413+0000 7f4f62c75640 1 -- 172.21.15.5:0/3263446677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f5c075b00 con 0x7f4f5c0ce7e0 2024-05-20T18:30:01.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.414+0000 7f4f597fa640 1 -- 172.21.15.5:0/3263446677 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4f4c01a070 con 0x7f4f5c0ce7e0 2024-05-20T18:30:01.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.414+0000 7f4f597fa640 1 --2- 172.21.15.5:0/3263446677 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4f4806be60 0x7f4f4806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:01.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.414+0000 7f4f597fa640 1 -- 172.21.15.5:0/3263446677 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6101+0+0 (secure 0 0 0) 0x7f4f4c08a390 con 0x7f4f5c0ce7e0 2024-05-20T18:30:01.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.414+0000 7f4f5bfff640 1 --2- 172.21.15.5:0/3263446677 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4f4806be60 0x7f4f4806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:01.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.415+0000 7f4f5bfff640 1 --2- 172.21.15.5:0/3263446677 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4f4806be60 0x7f4f4806e370 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4f50034000 tx=0x7f4f5002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:01.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.419+0000 7f4f597fa640 1 -- 172.21.15.5:0/3263446677 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4f4c058bb0 con 0x7f4f5c0ce7e0 2024-05-20T18:30:01.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.604+0000 7f4f62c75640 1 -- 172.21.15.5:0/3263446677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f5c162c60 con 0x7f4f5c0ce7e0 2024-05-20T18:30:01.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.605+0000 7f4f597fa640 1 -- 172.21.15.5:0/3263446677 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+16777 (secure 0 0 0) 0x7f4f4c0584b0 con 0x7f4f5c0ce7e0 2024-05-20T18:30:01.605 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:30:01.605 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":62,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:30:00.286233+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:29:57.993861+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-20T18:29:59.267582+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:30:01.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.607+0000 7f4f62c75640 1 -- 172.21.15.5:0/3263446677 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4f4806be60 msgr2=0x7f4f4806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:01.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.607+0000 7f4f62c75640 1 --2- 172.21.15.5:0/3263446677 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4f4806be60 0x7f4f4806e370 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4f50034000 tx=0x7f4f5002e000 comp rx=0 tx=0).stop 2024-05-20T18:30:01.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.607+0000 7f4f62c75640 1 -- 172.21.15.5:0/3263446677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f5c0ce7e0 msgr2=0x7f4f5c1f8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:01.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.607+0000 7f4f62c75640 1 --2- 172.21.15.5:0/3263446677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f5c0ce7e0 0x7f4f5c1f8710 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00c3e0 tx=0x7f4f4c00c8a0 comp rx=0 tx=0).stop 2024-05-20T18:30:01.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.607+0000 7f4f62c75640 1 -- 172.21.15.5:0/3263446677 shutdown_connections 2024-05-20T18:30:01.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.607+0000 7f4f62c75640 1 --2- 172.21.15.5:0/3263446677 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4f4806be60 0x7f4f4806e370 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:01.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.607+0000 7f4f62c75640 1 --2- 172.21.15.5:0/3263446677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f5c15f220 0x7f4f5c15d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:01.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.607+0000 7f4f62c75640 1 --2- 172.21.15.5:0/3263446677 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f5c0cf3d0 0x7f4f5c0ca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:01.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.607+0000 7f4f62c75640 1 --2- 172.21.15.5:0/3263446677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f5c0ce7e0 0x7f4f5c1f8710 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:01.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.607+0000 7f4f62c75640 1 -- 172.21.15.5:0/3263446677 >> 172.21.15.5:0/3263446677 conn(0x7f4f5c0b44d0 msgr2=0x7f4f5c0d08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:01.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.607+0000 7f4f62c75640 1 -- 172.21.15.5:0/3263446677 shutdown_connections 2024-05-20T18:30:01.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.607+0000 7f4f62c75640 1 -- 172.21.15.5:0/3263446677 wait complete. 2024-05-20T18:30:01.619 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-20T18:30:01.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.737+0000 7ff55391a640 1 Processor -- start 2024-05-20T18:30:01.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.737+0000 7ff55391a640 1 -- start start 2024-05-20T18:30:01.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.738+0000 7ff55391a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff54c168050 0x7ff54c16a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:01.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.738+0000 7ff55391a640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff54c0d0630 con 0x7ff54c06d3d0 2024-05-20T18:30:01.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.738+0000 7ff55391a640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff54c0d39c0 con 0x7ff54c162820 2024-05-20T18:30:01.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.738+0000 7ff55391a640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff54c0d3ca0 con 0x7ff54c168050 2024-05-20T18:30:01.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.739+0000 7ff550eb5640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff54c06d3d0 0x7ff54c06d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:55214/0 (socket says 172.21.15.5:55214) 2024-05-20T18:30:01.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.739+0000 7ff550eb5640 1 -- 172.21.15.5:0/720493783 learned_addr learned my addr 172.21.15.5:0/720493783 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:30:01.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.739+0000 7ff551eb7640 1 --2- 172.21.15.5:0/720493783 >> v2:172.21.15.63:3300/0 conn(0x7ff54c168050 0x7ff54c16a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:01.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.739+0000 7ff543fff640 1 -- 172.21.15.5:0/720493783 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 722049687 0 0) 0x7ff54c0d0630 con 0x7ff54c06d3d0 2024-05-20T18:30:01.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.739+0000 7ff543fff640 1 -- 172.21.15.5:0/720493783 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff538003520 con 0x7ff54c06d3d0 2024-05-20T18:30:01.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.739+0000 7ff551eb7640 1 -- 172.21.15.5:0/720493783 >> v1:172.21.15.63:6789/0 conn(0x7ff54c162820 legacy=0x7ff54c0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:01.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.739+0000 7ff551eb7640 1 -- 172.21.15.5:0/720493783 >> v1:172.21.15.5:6789/0 conn(0x7ff54c06d3d0 legacy=0x7ff54c06d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:01.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.739+0000 7ff551eb7640 1 -- 172.21.15.5:0/720493783 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff54c0d3f80 con 0x7ff54c168050 2024-05-20T18:30:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.740+0000 7ff551eb7640 1 --2- 172.21.15.5:0/720493783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff54c168050 0x7ff54c16a470 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff544009d90 tx=0x7ff54402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=768bbf7bc76454b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.740+0000 7ff543fff640 1 -- 172.21.15.5:0/720493783 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff544007740 con 0x7ff54c168050 2024-05-20T18:30:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.740+0000 7ff543fff640 1 -- 172.21.15.5:0/720493783 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5440078e0 con 0x7ff54c168050 2024-05-20T18:30:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.740+0000 7ff543fff640 1 -- 172.21.15.5:0/720493783 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff544007cb0 con 0x7ff54c168050 2024-05-20T18:30:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.741+0000 7ff55391a640 1 -- 172.21.15.5:0/720493783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff54c168050 msgr2=0x7ff54c16a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.741+0000 7ff55391a640 1 --2- 172.21.15.5:0/720493783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff54c168050 0x7ff54c16a470 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff544009d90 tx=0x7ff54402a6f0 comp rx=0 tx=0).stop 2024-05-20T18:30:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.741+0000 7ff55391a640 1 -- 172.21.15.5:0/720493783 shutdown_connections 2024-05-20T18:30:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.741+0000 7ff55391a640 1 --2- 172.21.15.5:0/720493783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff54c168050 0x7ff54c16a470 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.741+0000 7ff55391a640 1 -- 172.21.15.5:0/720493783 >> 172.21.15.5:0/720493783 conn(0x7ff54c0d4bd0 msgr2=0x7ff54c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.741+0000 7ff55391a640 1 -- 172.21.15.5:0/720493783 shutdown_connections 2024-05-20T18:30:01.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.741+0000 7ff55391a640 1 -- 172.21.15.5:0/720493783 wait complete. 2024-05-20T18:30:01.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.743+0000 7ff55391a640 1 Processor -- start 2024-05-20T18:30:01.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.743+0000 7ff55391a640 1 -- start start 2024-05-20T18:30:01.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.744+0000 7ff55391a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff54c06d3d0 0x7ff54c158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:01.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.744+0000 7ff55391a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff54c162820 0x7ff54c15b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.744+0000 7ff55391a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff54c168050 0x7ff54c0b6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.744+0000 7ff55391a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff54c0b91a0 con 0x7ff54c06d3d0 2024-05-20T18:30:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.744+0000 7ff55391a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff54c0d0300 con 0x7ff54c168050 2024-05-20T18:30:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.744+0000 7ff55391a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff54c0b4050 con 0x7ff54c162820 2024-05-20T18:30:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.744+0000 7ff5516b6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff54c06d3d0 0x7ff54c158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.744+0000 7ff5516b6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff54c06d3d0 0x7ff54c158d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33876/0 (socket says 172.21.15.5:33876) 2024-05-20T18:30:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.744+0000 7ff5516b6640 1 -- 172.21.15.5:0/3814827297 learned_addr learned my addr 172.21.15.5:0/3814827297 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.744+0000 7ff550eb5640 1 --2- 172.21.15.5:0/3814827297 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff54c162820 0x7ff54c15b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.744+0000 7ff551eb7640 1 --2- 172.21.15.5:0/3814827297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff54c168050 0x7ff54c0b6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:01.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.744+0000 7ff5516b6640 1 -- 172.21.15.5:0/3814827297 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff54c162820 msgr2=0x7ff54c15b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:01.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.744+0000 7ff5516b6640 1 --2- 172.21.15.5:0/3814827297 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff54c162820 0x7ff54c15b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:01.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.744+0000 7ff5516b6640 1 -- 172.21.15.5:0/3814827297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff54c168050 msgr2=0x7ff54c0b6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:01.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.744+0000 7ff5516b6640 1 --2- 172.21.15.5:0/3814827297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff54c168050 0x7ff54c0b6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:01.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.744+0000 7ff5516b6640 1 -- 172.21.15.5:0/3814827297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff54c15b700 con 0x7ff54c06d3d0 2024-05-20T18:30:01.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.745+0000 7ff5516b6640 1 --2- 172.21.15.5:0/3814827297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff54c06d3d0 0x7ff54c158d50 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7ff54800c3f0 tx=0x7ff54800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:01.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.745+0000 7ff541ffb640 1 -- 172.21.15.5:0/3814827297 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5480052d0 con 0x7ff54c06d3d0 2024-05-20T18:30:01.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.745+0000 7ff541ffb640 1 -- 172.21.15.5:0/3814827297 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff548005490 con 0x7ff54c06d3d0 2024-05-20T18:30:01.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.745+0000 7ff55391a640 1 -- 172.21.15.5:0/3814827297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff54c157cb0 con 0x7ff54c06d3d0 2024-05-20T18:30:01.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.745+0000 7ff541ffb640 1 -- 172.21.15.5:0/3814827297 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff548007260 con 0x7ff54c06d3d0 2024-05-20T18:30:01.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.745+0000 7ff55391a640 1 -- 172.21.15.5:0/3814827297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff54c158270 con 0x7ff54c06d3d0 2024-05-20T18:30:01.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.746+0000 7ff55391a640 1 -- 172.21.15.5:0/3814827297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff54c075b00 con 0x7ff54c06d3d0 2024-05-20T18:30:01.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.747+0000 7ff541ffb640 1 -- 172.21.15.5:0/3814827297 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff548007400 con 0x7ff54c06d3d0 2024-05-20T18:30:01.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.747+0000 7ff541ffb640 1 --2- 172.21.15.5:0/3814827297 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff53006be60 0x7ff53006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:01.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.747+0000 7ff550eb5640 1 --2- 172.21.15.5:0/3814827297 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff53006be60 0x7ff53006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:01.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.747+0000 7ff541ffb640 1 -- 172.21.15.5:0/3814827297 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6101+0+0 (secure 0 0 0) 0x7ff54808a4e0 con 0x7ff54c06d3d0 2024-05-20T18:30:01.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.748+0000 7ff550eb5640 1 --2- 172.21.15.5:0/3814827297 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff53006be60 0x7ff53006e370 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff54c1f8d00 tx=0x7ff53c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:01.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.752+0000 7ff541ffb640 1 -- 172.21.15.5:0/3814827297 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff548058d00 con 0x7ff54c06d3d0 2024-05-20T18:30:01.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:01.978+0000 7ff55391a640 1 -- 172.21.15.5:0/3814827297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff54c15d200 con 0x7ff54c06d3d0 2024-05-20T18:30:02.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:02.914+0000 7ff541ffb640 1 -- 172.21.15.5:0/3814827297 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v33) ==== 118+0+0 (secure 0 0 0) 0x7ff548058600 con 0x7ff54c06d3d0 2024-05-20T18:30:02.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:02.917+0000 7ff55391a640 1 -- 172.21.15.5:0/3814827297 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff53006be60 msgr2=0x7ff53006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:02.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:02.917+0000 7ff55391a640 1 --2- 172.21.15.5:0/3814827297 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff53006be60 0x7ff53006e370 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff54c1f8d00 tx=0x7ff53c009040 comp rx=0 tx=0).stop 2024-05-20T18:30:02.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:02.917+0000 7ff55391a640 1 -- 172.21.15.5:0/3814827297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff54c06d3d0 msgr2=0x7ff54c158d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:02.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:02.917+0000 7ff55391a640 1 --2- 172.21.15.5:0/3814827297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff54c06d3d0 0x7ff54c158d50 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7ff54800c3f0 tx=0x7ff54800c8b0 comp rx=0 tx=0).stop 2024-05-20T18:30:02.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:02.917+0000 7ff55391a640 1 -- 172.21.15.5:0/3814827297 shutdown_connections 2024-05-20T18:30:02.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:02.917+0000 7ff55391a640 1 --2- 172.21.15.5:0/3814827297 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff53006be60 0x7ff53006e370 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:02.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:02.917+0000 7ff55391a640 1 --2- 172.21.15.5:0/3814827297 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff54c168050 0x7ff54c0b6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:02.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:02.917+0000 7ff55391a640 1 --2- 172.21.15.5:0/3814827297 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff54c162820 0x7ff54c15b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:02.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:02.917+0000 7ff55391a640 1 --2- 172.21.15.5:0/3814827297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff54c06d3d0 0x7ff54c158d50 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:02.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:02.917+0000 7ff55391a640 1 -- 172.21.15.5:0/3814827297 >> 172.21.15.5:0/3814827297 conn(0x7ff54c0d4bd0 msgr2=0x7ff54c160fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:02.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:02.917+0000 7ff55391a640 1 -- 172.21.15.5:0/3814827297 shutdown_connections 2024-05-20T18:30:02.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:02.917+0000 7ff55391a640 1 -- 172.21.15.5:0/3814827297 wait complete. 2024-05-20T18:30:02.931 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-20T18:30:02.931 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-20T18:30:03.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.058+0000 7f1c505c7640 1 Processor -- start 2024-05-20T18:30:03.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.058+0000 7f1c505c7640 1 -- start start 2024-05-20T18:30:03.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.060+0000 7f1c505c7640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1c48162e00 0x7f1c4806d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:03.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.060+0000 7f1c505c7640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c4806c8c0 con 0x7f1c480cfae0 2024-05-20T18:30:03.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.060+0000 7f1c505c7640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c481633e0 con 0x7f1c480cd710 2024-05-20T18:30:03.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.060+0000 7f1c505c7640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f1c480d0ba0 con 0x7f1c48162e00 2024-05-20T18:30:03.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.060+0000 7f1c4db62640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1c480cfae0 0x7f1c480d1030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:55224/0 (socket says 172.21.15.5:55224) 2024-05-20T18:30:03.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.060+0000 7f1c4eb64640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1c48162e00 0x7f1c4806d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:03.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.060+0000 7f1c4db62640 1 -- 172.21.15.5:0/2689659246 learned_addr learned my addr 172.21.15.5:0/2689659246 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:30:03.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.060+0000 7f1c4eb64640 1 -- 172.21.15.5:0/2689659246 >> v1:172.21.15.63:6789/0 conn(0x7f1c480cd710 legacy=0x7f1c480cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:03.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.060+0000 7f1c4eb64640 1 -- 172.21.15.5:0/2689659246 >> v1:172.21.15.5:6789/0 conn(0x7f1c480cfae0 legacy=0x7f1c480d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:03.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.060+0000 7f1c4d361640 1 -- 172.21.15.5:0/2689659246 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4101670532 0 0) 0x7f1c4806c8c0 con 0x7f1c480cfae0 2024-05-20T18:30:03.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.060+0000 7f1c4eb64640 1 -- 172.21.15.5:0/2689659246 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c481636c0 con 0x7f1c48162e00 2024-05-20T18:30:03.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.061+0000 7f1c4eb64640 1 --2- 172.21.15.5:0/2689659246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c48162e00 0x7f1c4806d6b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1c40009d90 tx=0x7f1c4002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339ad5d6262f0a17 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:03.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.061+0000 7f1c4d361640 1 -- 172.21.15.5:0/2689659246 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c40007740 con 0x7f1c48162e00 2024-05-20T18:30:03.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.061+0000 7f1c4d361640 1 -- 172.21.15.5:0/2689659246 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c400078e0 con 0x7f1c48162e00 2024-05-20T18:30:03.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.061+0000 7f1c4d361640 1 -- 172.21.15.5:0/2689659246 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c40007cb0 con 0x7f1c48162e00 2024-05-20T18:30:03.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.062+0000 7f1c505c7640 1 -- 172.21.15.5:0/2689659246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c48162e00 msgr2=0x7f1c4806d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:03.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.062+0000 7f1c505c7640 1 --2- 172.21.15.5:0/2689659246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c48162e00 0x7f1c4806d6b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1c40009d90 tx=0x7f1c4002a6a0 comp rx=0 tx=0).stop 2024-05-20T18:30:03.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.062+0000 7f1c505c7640 1 -- 172.21.15.5:0/2689659246 shutdown_connections 2024-05-20T18:30:03.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.062+0000 7f1c505c7640 1 --2- 172.21.15.5:0/2689659246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c48162e00 0x7f1c4806d6b0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:03.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.062+0000 7f1c505c7640 1 -- 172.21.15.5:0/2689659246 >> 172.21.15.5:0/2689659246 conn(0x7f1c480b44d0 msgr2=0x7f1c480b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:03.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.062+0000 7f1c505c7640 1 -- 172.21.15.5:0/2689659246 shutdown_connections 2024-05-20T18:30:03.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.062+0000 7f1c505c7640 1 -- 172.21.15.5:0/2689659246 wait complete. 2024-05-20T18:30:03.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.064+0000 7f1c505c7640 1 Processor -- start 2024-05-20T18:30:03.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.064+0000 7f1c505c7640 1 -- start start 2024-05-20T18:30:03.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.065+0000 7f1c505c7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c480cd710 0x7f1c480b7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:03.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.065+0000 7f1c505c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c48162e00 0x7f1c480c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:03.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.065+0000 7f1c505c7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c480b5e30 0x7f1c480b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:03.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.065+0000 7f1c505c7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4815f2b0 con 0x7f1c48162e00 2024-05-20T18:30:03.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.065+0000 7f1c505c7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1c48163c80 con 0x7f1c480cd710 2024-05-20T18:30:03.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.065+0000 7f1c505c7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1c4815f430 con 0x7f1c480b5e30 2024-05-20T18:30:03.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.065+0000 7f1c4db62640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c48162e00 0x7f1c480c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:03.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.065+0000 7f1c4db62640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c48162e00 0x7f1c480c5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33888/0 (socket says 172.21.15.5:33888) 2024-05-20T18:30:03.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.065+0000 7f1c4db62640 1 -- 172.21.15.5:0/1792290511 learned_addr learned my addr 172.21.15.5:0/1792290511 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:03.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.065+0000 7f1c4e363640 1 --2- 172.21.15.5:0/1792290511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c480cd710 0x7f1c480b7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:03.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.066+0000 7f1c4db62640 1 -- 172.21.15.5:0/1792290511 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c480b5e30 msgr2=0x7f1c480b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:03.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.066+0000 7f1c4db62640 1 --2- 172.21.15.5:0/1792290511 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c480b5e30 0x7f1c480b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:03.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.066+0000 7f1c4db62640 1 -- 172.21.15.5:0/1792290511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c480cd710 msgr2=0x7f1c480b7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:03.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.066+0000 7f1c4db62640 1 --2- 172.21.15.5:0/1792290511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c480cd710 0x7f1c480b7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:03.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.066+0000 7f1c4db62640 1 -- 172.21.15.5:0/1792290511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c480b6780 con 0x7f1c48162e00 2024-05-20T18:30:03.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.066+0000 7f1c4db62640 1 --2- 172.21.15.5:0/1792290511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c48162e00 0x7f1c480c5ae0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800c460 tx=0x7f1c3800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:03.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.066+0000 7f1c3effd640 1 -- 172.21.15.5:0/1792290511 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c38017070 con 0x7f1c48162e00 2024-05-20T18:30:03.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.066+0000 7f1c3effd640 1 -- 172.21.15.5:0/1792290511 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c38005ce0 con 0x7f1c48162e00 2024-05-20T18:30:03.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.066+0000 7f1c505c7640 1 -- 172.21.15.5:0/1792290511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c480cb750 con 0x7f1c48162e00 2024-05-20T18:30:03.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.066+0000 7f1c3effd640 1 -- 172.21.15.5:0/1792290511 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c38007270 con 0x7f1c48162e00 2024-05-20T18:30:03.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.066+0000 7f1c505c7640 1 -- 172.21.15.5:0/1792290511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c480cbd10 con 0x7f1c48162e00 2024-05-20T18:30:03.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.067+0000 7f1c505c7640 1 -- 172.21.15.5:0/1792290511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c480b8470 con 0x7f1c48162e00 2024-05-20T18:30:03.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.067+0000 7f1c3effd640 1 -- 172.21.15.5:0/1792290511 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c3801a070 con 0x7f1c48162e00 2024-05-20T18:30:03.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.068+0000 7f1c3effd640 1 --2- 172.21.15.5:0/1792290511 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c2c06be60 0x7f1c2c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:03.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.068+0000 7f1c3effd640 1 -- 172.21.15.5:0/1792290511 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7f1c3808a190 con 0x7f1c48162e00 2024-05-20T18:30:03.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.068+0000 7f1c4e363640 1 --2- 172.21.15.5:0/1792290511 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c2c06be60 0x7f1c2c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:03.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.069+0000 7f1c4e363640 1 --2- 172.21.15.5:0/1792290511 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c2c06be60 0x7f1c2c06e370 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1c48162d60 tx=0x7f1c44008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:03.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.073+0000 7f1c3effd640 1 -- 172.21.15.5:0/1792290511 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1c480b8470 con 0x7f1c48162e00 2024-05-20T18:30:03.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.296+0000 7f1c505c7640 1 -- 172.21.15.5:0/1792290511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f1c48075b00 con 0x7f1c48162e00 2024-05-20T18:30:03.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.915+0000 7f1c3effd640 1 -- 172.21.15.5:0/1792290511 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7f1c480b6780 con 0x7f1c48162e00 2024-05-20T18:30:03.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.918+0000 7f1c505c7640 1 -- 172.21.15.5:0/1792290511 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c2c06be60 msgr2=0x7f1c2c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:03.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.918+0000 7f1c505c7640 1 --2- 172.21.15.5:0/1792290511 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c2c06be60 0x7f1c2c06e370 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1c48162d60 tx=0x7f1c44008040 comp rx=0 tx=0).stop 2024-05-20T18:30:03.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.918+0000 7f1c505c7640 1 -- 172.21.15.5:0/1792290511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c48162e00 msgr2=0x7f1c480c5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:03.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.918+0000 7f1c505c7640 1 --2- 172.21.15.5:0/1792290511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c48162e00 0x7f1c480c5ae0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800c460 tx=0x7f1c3800c920 comp rx=0 tx=0).stop 2024-05-20T18:30:03.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.918+0000 7f1c505c7640 1 -- 172.21.15.5:0/1792290511 shutdown_connections 2024-05-20T18:30:03.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.918+0000 7f1c505c7640 1 --2- 172.21.15.5:0/1792290511 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c2c06be60 0x7f1c2c06e370 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:03.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.918+0000 7f1c505c7640 1 --2- 172.21.15.5:0/1792290511 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c480b5e30 0x7f1c480b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:03.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.918+0000 7f1c505c7640 1 --2- 172.21.15.5:0/1792290511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c48162e00 0x7f1c480c5ae0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:03.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.918+0000 7f1c505c7640 1 --2- 172.21.15.5:0/1792290511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c480cd710 0x7f1c480b7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:03.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.918+0000 7f1c505c7640 1 -- 172.21.15.5:0/1792290511 >> 172.21.15.5:0/1792290511 conn(0x7f1c480b44d0 msgr2=0x7f1c480d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:03.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.918+0000 7f1c505c7640 1 -- 172.21.15.5:0/1792290511 shutdown_connections 2024-05-20T18:30:03.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:03.918+0000 7f1c505c7640 1 -- 172.21.15.5:0/1792290511 wait complete. 2024-05-20T18:30:03.933 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-20T18:30:03.933 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:30:04.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.054+0000 7f54f4a67640 1 Processor -- start 2024-05-20T18:30:04.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.054+0000 7f54f4a67640 1 -- start start 2024-05-20T18:30:04.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.056+0000 7f54f4a67640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f54f00d10e0 0x7f54f00cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:04.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.056+0000 7f54f4a67640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f54f00d14f0 0x7f54f00cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:04.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.056+0000 7f54f4a67640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54f00d1ac0 con 0x7f54f00cd4d0 2024-05-20T18:30:04.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.056+0000 7f54f4a67640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f54f00d1da0 con 0x7f54f00d14f0 2024-05-20T18:30:04.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.056+0000 7f54f4a67640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f54f00d2080 con 0x7f54f00d10e0 2024-05-20T18:30:04.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.056+0000 7f54edd9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f54f00d14f0 0x7f54f00cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:04.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.056+0000 7f54edd9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f54f00d14f0 0x7f54f00cf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33904/0 (socket says 172.21.15.5:33904) 2024-05-20T18:30:04.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.056+0000 7f54edd9b640 1 -- 172.21.15.5:0/1976190376 learned_addr learned my addr 172.21.15.5:0/1976190376 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:04.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.056+0000 7f54ee59c640 1 --2- 172.21.15.5:0/1976190376 >> v2:172.21.15.63:3300/0 conn(0x7f54f00d10e0 0x7f54f00cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:04.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.056+0000 7f54edd9b640 1 -- 172.21.15.5:0/1976190376 >> v2:172.21.15.63:3300/0 conn(0x7f54f00d10e0 msgr2=0x7f54f00cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:04.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.056+0000 7f54edd9b640 1 --2- 172.21.15.5:0/1976190376 >> v2:172.21.15.63:3300/0 conn(0x7f54f00d10e0 0x7f54f00cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:04.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.056+0000 7f54edd9b640 1 -- 172.21.15.5:0/1976190376 >> v1:172.21.15.63:6789/0 conn(0x7f54f00cd4d0 legacy=0x7f54f00cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:04.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.057+0000 7f54edd9b640 1 -- 172.21.15.5:0/1976190376 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f00d2360 con 0x7f54f00d14f0 2024-05-20T18:30:04.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.057+0000 7f54ee59c640 1 --2- 172.21.15.5:0/1976190376 >> v2:172.21.15.63:3300/0 conn(0x7f54f00d10e0 0x7f54f00cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:30:04.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.057+0000 7f54edd9b640 1 --2- 172.21.15.5:0/1976190376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f00d14f0 0x7f54f00cf680 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f54f00ccfc0 tx=0x7f54e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9bad5698b5bb4d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:04.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.057+0000 7f54ed59a640 1 -- 172.21.15.5:0/1976190376 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4037070 con 0x7f54f00d14f0 2024-05-20T18:30:04.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.058+0000 7f54ed59a640 1 -- 172.21.15.5:0/1976190376 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e4032440 con 0x7f54f00d14f0 2024-05-20T18:30:04.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.058+0000 7f54ed59a640 1 -- 172.21.15.5:0/1976190376 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4032810 con 0x7f54f00d14f0 2024-05-20T18:30:04.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.058+0000 7f54f4a67640 1 -- 172.21.15.5:0/1976190376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f00d14f0 msgr2=0x7f54f00cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:04.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.058+0000 7f54f4a67640 1 --2- 172.21.15.5:0/1976190376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f00d14f0 0x7f54f00cf680 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f54f00ccfc0 tx=0x7f54e402a270 comp rx=0 tx=0).stop 2024-05-20T18:30:04.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.058+0000 7f54f4a67640 1 -- 172.21.15.5:0/1976190376 shutdown_connections 2024-05-20T18:30:04.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.058+0000 7f54f4a67640 1 --2- 172.21.15.5:0/1976190376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f00d14f0 0x7f54f00cf680 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:04.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.058+0000 7f54f4a67640 1 --2- 172.21.15.5:0/1976190376 >> v2:172.21.15.63:3300/0 conn(0x7f54f00d10e0 0x7f54f00cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:04.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.058+0000 7f54f4a67640 1 -- 172.21.15.5:0/1976190376 >> 172.21.15.5:0/1976190376 conn(0x7f54f00b44d0 msgr2=0x7f54f00b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:04.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.058+0000 7f54f4a67640 1 -- 172.21.15.5:0/1976190376 shutdown_connections 2024-05-20T18:30:04.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.058+0000 7f54f4a67640 1 -- 172.21.15.5:0/1976190376 wait complete. 2024-05-20T18:30:04.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.060+0000 7f54f4a67640 1 Processor -- start 2024-05-20T18:30:04.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.060+0000 7f54f4a67640 1 -- start start 2024-05-20T18:30:04.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.062+0000 7f54f4a67640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f54f00cd4d0 0x7f54f00ca760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:04.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.062+0000 7f54f4a67640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54f00d10e0 0x7f54f0158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:04.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.062+0000 7f54f4a67640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f00d14f0 0x7f54f015ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:04.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.062+0000 7f54f4a67640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f54f00b8440 con 0x7f54f00d14f0 2024-05-20T18:30:04.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.062+0000 7f54f4a67640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f54f00b82c0 con 0x7f54f00d10e0 2024-05-20T18:30:04.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.062+0000 7f54f4a67640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f54f00709f0 con 0x7f54f00cd4d0 2024-05-20T18:30:04.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.062+0000 7f54eed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f00d14f0 0x7f54f015ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:04.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.062+0000 7f54eed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f00d14f0 0x7f54f015ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33910/0 (socket says 172.21.15.5:33910) 2024-05-20T18:30:04.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.062+0000 7f54eed9d640 1 -- 172.21.15.5:0/3145087124 learned_addr learned my addr 172.21.15.5:0/3145087124 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:04.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.062+0000 7f54edd9b640 1 --2- 172.21.15.5:0/3145087124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54f00d10e0 0x7f54f0158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:04.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.062+0000 7f54ee59c640 1 --2- 172.21.15.5:0/3145087124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f54f00cd4d0 0x7f54f00ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:04.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.062+0000 7f54eed9d640 1 -- 172.21.15.5:0/3145087124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f54f00cd4d0 msgr2=0x7f54f00ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:04.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.062+0000 7f54eed9d640 1 --2- 172.21.15.5:0/3145087124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f54f00cd4d0 0x7f54f00ca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:04.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.062+0000 7f54eed9d640 1 -- 172.21.15.5:0/3145087124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54f00d10e0 msgr2=0x7f54f0158140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:04.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.062+0000 7f54eed9d640 1 --2- 172.21.15.5:0/3145087124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54f00d10e0 0x7f54f0158140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:04.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.062+0000 7f54eed9d640 1 -- 172.21.15.5:0/3145087124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f00c99a0 con 0x7f54f00d14f0 2024-05-20T18:30:04.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.062+0000 7f54edd9b640 1 --2- 172.21.15.5:0/3145087124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54f00d10e0 0x7f54f0158140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:30:04.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.062+0000 7f54eed9d640 1 --2- 172.21.15.5:0/3145087124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f00d14f0 0x7f54f015ecd0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f54e800c3f0 tx=0x7f54e800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:04.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.062+0000 7f54deffd640 1 -- 172.21.15.5:0/3145087124 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e80052d0 con 0x7f54f00d14f0 2024-05-20T18:30:04.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.062+0000 7f54deffd640 1 -- 172.21.15.5:0/3145087124 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e8005470 con 0x7f54f00d14f0 2024-05-20T18:30:04.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.062+0000 7f54f4a67640 1 -- 172.21.15.5:0/3145087124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f00c9c80 con 0x7f54f00d14f0 2024-05-20T18:30:04.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.062+0000 7f54deffd640 1 -- 172.21.15.5:0/3145087124 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e8007260 con 0x7f54f00d14f0 2024-05-20T18:30:04.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.063+0000 7f54f4a67640 1 -- 172.21.15.5:0/3145087124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f015ba90 con 0x7f54f00d14f0 2024-05-20T18:30:04.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.064+0000 7f54f4a67640 1 -- 172.21.15.5:0/3145087124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54f00b7b40 con 0x7f54f00d14f0 2024-05-20T18:30:04.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.064+0000 7f54deffd640 1 -- 172.21.15.5:0/3145087124 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f54e8007400 con 0x7f54f00d14f0 2024-05-20T18:30:04.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.065+0000 7f54deffd640 1 --2- 172.21.15.5:0/3145087124 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f54d006be30 0x7f54d006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:04.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.065+0000 7f54deffd640 1 -- 172.21.15.5:0/3145087124 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7f54e808a3c0 con 0x7f54f00d14f0 2024-05-20T18:30:04.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.065+0000 7f54ee59c640 1 --2- 172.21.15.5:0/3145087124 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f54d006be30 0x7f54d006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:04.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.065+0000 7f54ee59c640 1 --2- 172.21.15.5:0/3145087124 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f54d006be30 0x7f54d006e340 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f54e0003fd0 tx=0x7f54e0003ce0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:04.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.070+0000 7f54deffd640 1 -- 172.21.15.5:0/3145087124 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f54f00b7b40 con 0x7f54f00d14f0 2024-05-20T18:30:04.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.294+0000 7f54f4a67640 1 -- 172.21.15.5:0/3145087124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54f0075b00 con 0x7f54f00d14f0 2024-05-20T18:30:04.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.295+0000 7f54deffd640 1 -- 172.21.15.5:0/3145087124 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5176 (secure 0 0 0) 0x7f54e8058be0 con 0x7f54f00d14f0 2024-05-20T18:30:04.295 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:30:04.295 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5480,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/4292435960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":4292435960},{"type":"v1","addr":"172.21.15.63:6835","nonce":4292435960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6186,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2991834087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2991834087},{"type":"v1","addr":"172.21.15.5:6836","nonce":2991834087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6195,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3462206198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":3462206198},{"type":"v1","addr":"172.21.15.5:6837","nonce":3462206198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:30:00.286575+0000","modified":"2024-05-20T18:30:03.297495+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6198":{"gid":6198,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6834/4036379300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":4036379300},{"type":"v1","addr":"172.21.15.63:6834","nonce":4036379300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6198,"qdb_cluster":[6198]},"id":4}]} 2024-05-20T18:30:04.295 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 34 2024-05-20T18:30:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.297+0000 7f54f4a67640 1 -- 172.21.15.5:0/3145087124 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f54d006be30 msgr2=0x7f54d006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.297+0000 7f54f4a67640 1 --2- 172.21.15.5:0/3145087124 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f54d006be30 0x7f54d006e340 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f54e0003fd0 tx=0x7f54e0003ce0 comp rx=0 tx=0).stop 2024-05-20T18:30:04.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.297+0000 7f54f4a67640 1 -- 172.21.15.5:0/3145087124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f00d14f0 msgr2=0x7f54f015ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:04.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.297+0000 7f54f4a67640 1 --2- 172.21.15.5:0/3145087124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f00d14f0 0x7f54f015ecd0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f54e800c3f0 tx=0x7f54e800c8b0 comp rx=0 tx=0).stop 2024-05-20T18:30:04.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.297+0000 7f54f4a67640 1 -- 172.21.15.5:0/3145087124 shutdown_connections 2024-05-20T18:30:04.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.297+0000 7f54f4a67640 1 --2- 172.21.15.5:0/3145087124 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f54d006be30 0x7f54d006e340 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:04.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.297+0000 7f54f4a67640 1 --2- 172.21.15.5:0/3145087124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f00d14f0 0x7f54f015ecd0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:04.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.297+0000 7f54f4a67640 1 --2- 172.21.15.5:0/3145087124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54f00d10e0 0x7f54f0158140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:04.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.297+0000 7f54f4a67640 1 --2- 172.21.15.5:0/3145087124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f54f00cd4d0 0x7f54f00ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:04.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.297+0000 7f54f4a67640 1 -- 172.21.15.5:0/3145087124 >> 172.21.15.5:0/3145087124 conn(0x7f54f00b44d0 msgr2=0x7f54f00b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:04.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.297+0000 7f54f4a67640 1 -- 172.21.15.5:0/3145087124 shutdown_connections 2024-05-20T18:30:04.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.297+0000 7f54f4a67640 1 -- 172.21.15.5:0/3145087124 wait complete. 2024-05-20T18:30:04.309 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:30:04.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.424+0000 7fb09d2d7640 1 Processor -- start 2024-05-20T18:30:04.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.424+0000 7fb09d2d7640 1 -- start start 2024-05-20T18:30:04.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.426+0000 7fb09d2d7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb09806ca80 0x7fb09815f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:04.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.426+0000 7fb09d2d7640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb0980ae730 0x7fb09806c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:04.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.426+0000 7fb09d2d7640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0980b1b80 con 0x7fb0980ae320 2024-05-20T18:30:04.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.426+0000 7fb09d2d7640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fb09806b750 con 0x7fb0980ae730 2024-05-20T18:30:04.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.426+0000 7fb09d2d7640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb0980b12e0 con 0x7fb09806ca80 2024-05-20T18:30:04.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.426+0000 7fb0967fc640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb0980ae320 0x7fb09806bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:55234/0 (socket says 172.21.15.5:55234) 2024-05-20T18:30:04.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.426+0000 7fb0967fc640 1 -- 172.21.15.5:0/3368750372 learned_addr learned my addr 172.21.15.5:0/3368750372 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:30:04.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.426+0000 7fb0977fe640 1 --2- 172.21.15.5:0/3368750372 >> v2:172.21.15.5:3300/0 conn(0x7fb0980ae730 0x7fb09806c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:04.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.426+0000 7fb096ffd640 1 --2- 172.21.15.5:0/3368750372 >> v2:172.21.15.63:3300/0 conn(0x7fb09806ca80 0x7fb09815f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:04.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.426+0000 7fb0977fe640 1 -- 172.21.15.5:0/3368750372 >> v2:172.21.15.63:3300/0 conn(0x7fb09806ca80 msgr2=0x7fb09815f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:04.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.426+0000 7fb0977fe640 1 --2- 172.21.15.5:0/3368750372 >> v2:172.21.15.63:3300/0 conn(0x7fb09806ca80 0x7fb09815f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:04.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.426+0000 7fb0977fe640 1 -- 172.21.15.5:0/3368750372 >> v1:172.21.15.5:6789/0 conn(0x7fb0980ae320 legacy=0x7fb09806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:04.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.426+0000 7fb095ffb640 1 -- 172.21.15.5:0/3368750372 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2710370109 0 0) 0x7fb08c001110 con 0x7fb0980ae320 2024-05-20T18:30:04.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.426+0000 7fb0977fe640 1 -- 172.21.15.5:0/3368750372 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0980b15c0 con 0x7fb0980ae730 2024-05-20T18:30:04.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.427+0000 7fb0977fe640 1 --2- 172.21.15.5:0/3368750372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0980ae730 0x7fb09806c2f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fb098071470 tx=0x7fb09002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f6ad1d9fe5d1cf server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:04.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.427+0000 7fb095ffb640 1 -- 172.21.15.5:0/3368750372 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb090007740 con 0x7fb0980ae730 2024-05-20T18:30:04.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.427+0000 7fb095ffb640 1 -- 172.21.15.5:0/3368750372 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0900078e0 con 0x7fb0980ae730 2024-05-20T18:30:04.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.427+0000 7fb095ffb640 1 -- 172.21.15.5:0/3368750372 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb090007cb0 con 0x7fb0980ae730 2024-05-20T18:30:04.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.428+0000 7fb09d2d7640 1 -- 172.21.15.5:0/3368750372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0980ae730 msgr2=0x7fb09806c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:04.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.428+0000 7fb09d2d7640 1 --2- 172.21.15.5:0/3368750372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0980ae730 0x7fb09806c2f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fb098071470 tx=0x7fb09002a680 comp rx=0 tx=0).stop 2024-05-20T18:30:04.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.428+0000 7fb09d2d7640 1 -- 172.21.15.5:0/3368750372 shutdown_connections 2024-05-20T18:30:04.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.428+0000 7fb09d2d7640 1 --2- 172.21.15.5:0/3368750372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0980ae730 0x7fb09806c2f0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:04.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.428+0000 7fb09d2d7640 1 --2- 172.21.15.5:0/3368750372 >> v2:172.21.15.63:3300/0 conn(0x7fb09806ca80 0x7fb09815f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:04.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.428+0000 7fb09d2d7640 1 -- 172.21.15.5:0/3368750372 >> 172.21.15.5:0/3368750372 conn(0x7fb098072ed0 msgr2=0x7fb0980b6080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:04.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.428+0000 7fb09d2d7640 1 -- 172.21.15.5:0/3368750372 shutdown_connections 2024-05-20T18:30:04.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.428+0000 7fb09d2d7640 1 -- 172.21.15.5:0/3368750372 wait complete. 2024-05-20T18:30:04.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.430+0000 7fb09d2d7640 1 Processor -- start 2024-05-20T18:30:04.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.430+0000 7fb09d2d7640 1 -- start start 2024-05-20T18:30:04.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.431+0000 7fb09d2d7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb09806ca80 0x7fb098160d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:04.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.431+0000 7fb09d2d7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0980ae320 0x7fb0981f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:04.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.431+0000 7fb09d2d7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0980ae730 0x7fb0980cd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:04.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.431+0000 7fb09d2d7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb0980d0c10 con 0x7fb0980ae320 2024-05-20T18:30:04.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.431+0000 7fb09d2d7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb098074e00 con 0x7fb0980ae730 2024-05-20T18:30:04.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.431+0000 7fb09d2d7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb0980d0d90 con 0x7fb09806ca80 2024-05-20T18:30:04.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.431+0000 7fb0967fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0980ae320 0x7fb0981f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:04.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.431+0000 7fb0967fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0980ae320 0x7fb0981f52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33940/0 (socket says 172.21.15.5:33940) 2024-05-20T18:30:04.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.431+0000 7fb0967fc640 1 -- 172.21.15.5:0/4118958461 learned_addr learned my addr 172.21.15.5:0/4118958461 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:04.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.432+0000 7fb0977fe640 1 --2- 172.21.15.5:0/4118958461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0980ae730 0x7fb0980cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:04.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.432+0000 7fb096ffd640 1 --2- 172.21.15.5:0/4118958461 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb09806ca80 0x7fb098160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:04.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.432+0000 7fb0967fc640 1 -- 172.21.15.5:0/4118958461 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb09806ca80 msgr2=0x7fb098160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:04.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.432+0000 7fb0967fc640 1 --2- 172.21.15.5:0/4118958461 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb09806ca80 0x7fb098160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:04.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.432+0000 7fb0967fc640 1 -- 172.21.15.5:0/4118958461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0980ae730 msgr2=0x7fb0980cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:04.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.432+0000 7fb0967fc640 1 --2- 172.21.15.5:0/4118958461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0980ae730 0x7fb0980cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:04.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.432+0000 7fb0967fc640 1 -- 172.21.15.5:0/4118958461 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09815fc60 con 0x7fb0980ae320 2024-05-20T18:30:04.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.432+0000 7fb0967fc640 1 --2- 172.21.15.5:0/4118958461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0980ae320 0x7fb0981f52c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00c460 tx=0x7fb08c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:04.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.432+0000 7fb07f7fe640 1 -- 172.21.15.5:0/4118958461 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c018070 con 0x7fb0980ae320 2024-05-20T18:30:04.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.432+0000 7fb07f7fe640 1 -- 172.21.15.5:0/4118958461 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb08c007500 con 0x7fb0980ae320 2024-05-20T18:30:04.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.432+0000 7fb07f7fe640 1 -- 172.21.15.5:0/4118958461 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c004a20 con 0x7fb0980ae320 2024-05-20T18:30:04.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.432+0000 7fb09d2d7640 1 -- 172.21.15.5:0/4118958461 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb09815ff40 con 0x7fb0980ae320 2024-05-20T18:30:04.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.432+0000 7fb09d2d7640 1 -- 172.21.15.5:0/4118958461 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb098160500 con 0x7fb0980ae320 2024-05-20T18:30:04.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.433+0000 7fb09d2d7640 1 -- 172.21.15.5:0/4118958461 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb098075b00 con 0x7fb0980ae320 2024-05-20T18:30:04.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.434+0000 7fb07f7fe640 1 -- 172.21.15.5:0/4118958461 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb08c01b070 con 0x7fb0980ae320 2024-05-20T18:30:04.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.434+0000 7fb07f7fe640 1 --2- 172.21.15.5:0/4118958461 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb08406be60 0x7fb08406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:04.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.434+0000 7fb07f7fe640 1 -- 172.21.15.5:0/4118958461 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7fb08c08a370 con 0x7fb0980ae320 2024-05-20T18:30:04.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.434+0000 7fb096ffd640 1 --2- 172.21.15.5:0/4118958461 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb08406be60 0x7fb08406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:04.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.435+0000 7fb096ffd640 1 --2- 172.21.15.5:0/4118958461 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb08406be60 0x7fb08406e370 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb088006fd0 tx=0x7fb088008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:04.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.439+0000 7fb07f7fe640 1 -- 172.21.15.5:0/4118958461 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb0980b1b80 con 0x7fb0980ae320 2024-05-20T18:30:04.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.625+0000 7fb09d2d7640 1 -- 172.21.15.5:0/4118958461 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb0980b0eb0 con 0x7fb0980ae320 2024-05-20T18:30:04.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.626+0000 7fb07f7fe640 1 -- 172.21.15.5:0/4118958461 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+16764 (secure 0 0 0) 0x7fb08c058b00 con 0x7fb0980ae320 2024-05-20T18:30:04.626 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:30:04.626 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":63,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:30:01.893348+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:29:57.993861+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-20T18:29:59.267582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:30:04.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.628+0000 7fb09d2d7640 1 -- 172.21.15.5:0/4118958461 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb08406be60 msgr2=0x7fb08406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:04.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.628+0000 7fb09d2d7640 1 --2- 172.21.15.5:0/4118958461 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb08406be60 0x7fb08406e370 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb088006fd0 tx=0x7fb088008040 comp rx=0 tx=0).stop 2024-05-20T18:30:04.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.628+0000 7fb09d2d7640 1 -- 172.21.15.5:0/4118958461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0980ae320 msgr2=0x7fb0981f52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:04.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.628+0000 7fb09d2d7640 1 --2- 172.21.15.5:0/4118958461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0980ae320 0x7fb0981f52c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00c460 tx=0x7fb08c00c920 comp rx=0 tx=0).stop 2024-05-20T18:30:04.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.628+0000 7fb09d2d7640 1 -- 172.21.15.5:0/4118958461 shutdown_connections 2024-05-20T18:30:04.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.628+0000 7fb09d2d7640 1 --2- 172.21.15.5:0/4118958461 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb08406be60 0x7fb08406e370 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:04.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.628+0000 7fb09d2d7640 1 --2- 172.21.15.5:0/4118958461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb0980ae730 0x7fb0980cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:04.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.628+0000 7fb09d2d7640 1 --2- 172.21.15.5:0/4118958461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0980ae320 0x7fb0981f52c0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:04.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.628+0000 7fb09d2d7640 1 --2- 172.21.15.5:0/4118958461 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb09806ca80 0x7fb098160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:04.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.628+0000 7fb09d2d7640 1 -- 172.21.15.5:0/4118958461 >> 172.21.15.5:0/4118958461 conn(0x7fb098072ed0 msgr2=0x7fb0980d1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:04.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.628+0000 7fb09d2d7640 1 -- 172.21.15.5:0/4118958461 shutdown_connections 2024-05-20T18:30:04.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.628+0000 7fb09d2d7640 1 -- 172.21.15.5:0/4118958461 wait complete. 2024-05-20T18:30:04.640 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-20T18:30:04.640 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T18:30:04.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.755+0000 7f12c0ec3640 1 Processor -- start 2024-05-20T18:30:04.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.755+0000 7f12c0ec3640 1 -- start start 2024-05-20T18:30:04.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.756+0000 7f12c0ec3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12bc0b6d90 0x7f12bc15f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:04.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.756+0000 7f12c0ec3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f12bc0d3220 0x7f12bc0d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:04.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.756+0000 7f12c0ec3640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12bc0d0550 con 0x7f12bc0cf250 2024-05-20T18:30:04.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.756+0000 7f12c0ec3640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f12bc15f9a0 con 0x7f12bc0d3220 2024-05-20T18:30:04.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.756+0000 7f12c0ec3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f12bc0d0b10 con 0x7f12bc0b6d90 2024-05-20T18:30:04.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.756+0000 7f12bad9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f12bc0d3220 0x7f12bc0d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:04.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.756+0000 7f12ba59c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f12bc0cf250 0x7f12bc0b6680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43984/0 (socket says 172.21.15.5:43984) 2024-05-20T18:30:04.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.756+0000 7f12ba59c640 1 -- 172.21.15.5:0/2685743579 learned_addr learned my addr 172.21.15.5:0/2685743579 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:30:04.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.757+0000 7f12b3fff640 1 --2- 172.21.15.5:0/2685743579 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12bc0b6d90 0x7f12bc15f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:04.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.757+0000 7f12bad9d640 1 -- 172.21.15.5:0/2685743579 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12bc0b6d90 msgr2=0x7f12bc15f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:04.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.757+0000 7f12bad9d640 1 --2- 172.21.15.5:0/2685743579 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12bc0b6d90 0x7f12bc15f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:04.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.757+0000 7f12bad9d640 1 -- 172.21.15.5:0/2685743579 >> v1:172.21.15.5:6789/0 conn(0x7f12bc0cf250 legacy=0x7f12bc0b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:04.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.757+0000 7f12bad9d640 1 -- 172.21.15.5:0/2685743579 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12bc0d0830 con 0x7f12bc0d3220 2024-05-20T18:30:04.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.757+0000 7f12b3fff640 1 --2- 172.21.15.5:0/2685743579 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12bc0b6d90 0x7f12bc15f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:30:04.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.757+0000 7f12bad9d640 1 --2- 172.21.15.5:0/2685743579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12bc0d3220 0x7f12bc0d3630 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009d90 tx=0x7f12b402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a642e4724a3f16af server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:04.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.758+0000 7f12b9d9b640 1 -- 172.21.15.5:0/2685743579 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b4007740 con 0x7f12bc0d3220 2024-05-20T18:30:04.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.758+0000 7f12b9d9b640 1 -- 172.21.15.5:0/2685743579 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12b40078e0 con 0x7f12bc0d3220 2024-05-20T18:30:04.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.758+0000 7f12b9d9b640 1 -- 172.21.15.5:0/2685743579 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b4007cb0 con 0x7f12bc0d3220 2024-05-20T18:30:04.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.758+0000 7f12c0ec3640 1 -- 172.21.15.5:0/2685743579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12bc0d3220 msgr2=0x7f12bc0d3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:04.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.758+0000 7f12c0ec3640 1 --2- 172.21.15.5:0/2685743579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12bc0d3220 0x7f12bc0d3630 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009d90 tx=0x7f12b402a660 comp rx=0 tx=0).stop 2024-05-20T18:30:04.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.758+0000 7f12c0ec3640 1 -- 172.21.15.5:0/2685743579 shutdown_connections 2024-05-20T18:30:04.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.758+0000 7f12c0ec3640 1 --2- 172.21.15.5:0/2685743579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12bc0d3220 0x7f12bc0d3630 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:04.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.758+0000 7f12c0ec3640 1 --2- 172.21.15.5:0/2685743579 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12bc0b6d90 0x7f12bc15f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:04.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.758+0000 7f12c0ec3640 1 -- 172.21.15.5:0/2685743579 >> 172.21.15.5:0/2685743579 conn(0x7f12bc0d1860 msgr2=0x7f12bc0d1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:04.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.758+0000 7f12c0ec3640 1 -- 172.21.15.5:0/2685743579 shutdown_connections 2024-05-20T18:30:04.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.758+0000 7f12c0ec3640 1 -- 172.21.15.5:0/2685743579 wait complete. 2024-05-20T18:30:04.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.760+0000 7f12c0ec3640 1 Processor -- start 2024-05-20T18:30:04.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.760+0000 7f12c0ec3640 1 -- start start 2024-05-20T18:30:04.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.762+0000 7f12c0ec3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12bc0cf250 0x7f12bc158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:04.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.762+0000 7f12c0ec3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12bc0d3220 0x7f12bc156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:04.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.762+0000 7f12c0ec3640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12bc156650 0x7f12bc155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:04.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.762+0000 7f12c0ec3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12bc0d2d00 con 0x7f12bc0cf250 2024-05-20T18:30:04.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.762+0000 7f12c0ec3640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f12bc0d27d0 con 0x7f12bc156650 2024-05-20T18:30:04.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.762+0000 7f12c0ec3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f12bc0d0140 con 0x7f12bc0d3220 2024-05-20T18:30:04.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.762+0000 7f12ba59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12bc0cf250 0x7f12bc158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:04.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.762+0000 7f12ba59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12bc0cf250 0x7f12bc158940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53046/0 (socket says 172.21.15.5:53046) 2024-05-20T18:30:04.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.762+0000 7f12ba59c640 1 -- 172.21.15.5:0/3790442138 learned_addr learned my addr 172.21.15.5:0/3790442138 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:04.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.762+0000 7f12bad9d640 1 --2- 172.21.15.5:0/3790442138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12bc156650 0x7f12bc155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:04.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.762+0000 7f12b3fff640 1 --2- 172.21.15.5:0/3790442138 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12bc0d3220 0x7f12bc156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:04.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.762+0000 7f12ba59c640 1 -- 172.21.15.5:0/3790442138 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12bc0d3220 msgr2=0x7f12bc156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:04.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.762+0000 7f12ba59c640 1 --2- 172.21.15.5:0/3790442138 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12bc0d3220 0x7f12bc156110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:04.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.762+0000 7f12ba59c640 1 -- 172.21.15.5:0/3790442138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12bc156650 msgr2=0x7f12bc155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:04.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.762+0000 7f12ba59c640 1 --2- 172.21.15.5:0/3790442138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12bc156650 0x7f12bc155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:04.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.762+0000 7f12ba59c640 1 -- 172.21.15.5:0/3790442138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12bc1e9240 con 0x7f12bc0cf250 2024-05-20T18:30:04.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.762+0000 7f12bad9d640 1 --2- 172.21.15.5:0/3790442138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12bc156650 0x7f12bc155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:30:04.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.762+0000 7f12ba59c640 1 --2- 172.21.15.5:0/3790442138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12bc0cf250 0x7f12bc158940 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f12ac00c3f0 tx=0x7f12ac00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:04.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.762+0000 7f12b2ffd640 1 -- 172.21.15.5:0/3790442138 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac017070 con 0x7f12bc0cf250 2024-05-20T18:30:04.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.762+0000 7f12b2ffd640 1 -- 172.21.15.5:0/3790442138 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12ac005ce0 con 0x7f12bc0cf250 2024-05-20T18:30:04.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.762+0000 7f12c0ec3640 1 -- 172.21.15.5:0/3790442138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12bc1e94e0 con 0x7f12bc0cf250 2024-05-20T18:30:04.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.762+0000 7f12b2ffd640 1 -- 172.21.15.5:0/3790442138 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac0072f0 con 0x7f12bc0cf250 2024-05-20T18:30:04.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.763+0000 7f12c0ec3640 1 -- 172.21.15.5:0/3790442138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12bc1eb040 con 0x7f12bc0cf250 2024-05-20T18:30:04.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.763+0000 7f12c0ec3640 1 -- 172.21.15.5:0/3790442138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12bc075b00 con 0x7f12bc0cf250 2024-05-20T18:30:04.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.764+0000 7f12b2ffd640 1 -- 172.21.15.5:0/3790442138 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f12ac01a070 con 0x7f12bc0cf250 2024-05-20T18:30:04.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.765+0000 7f12b2ffd640 1 --2- 172.21.15.5:0/3790442138 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f129c06be60 0x7f129c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:04.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.765+0000 7f12b2ffd640 1 -- 172.21.15.5:0/3790442138 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7f12ac08a3e0 con 0x7f12bc0cf250 2024-05-20T18:30:04.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.765+0000 7f12b3fff640 1 --2- 172.21.15.5:0/3790442138 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f129c06be60 0x7f129c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:04.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.765+0000 7f12b3fff640 1 --2- 172.21.15.5:0/3790442138 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f129c06be60 0x7f129c06e370 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f12bc0d20c0 tx=0x7f12a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:04.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.769+0000 7f12b2ffd640 1 -- 172.21.15.5:0/3790442138 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f12ac058b70 con 0x7f12bc0cf250 2024-05-20T18:30:04.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.952+0000 7f12c0ec3640 1 -- 172.21.15.5:0/3790442138 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f12bc0d45d0 con 0x7f129c06be60 2024-05-20T18:30:04.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.959+0000 7f12b2ffd640 1 -- 172.21.15.5:0/3790442138 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7f12bc0d45d0 con 0x7f129c06be60 2024-05-20T18:30:04.962 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:30:04.963 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-20T18:30:04.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.962+0000 7f12c0ec3640 1 -- 172.21.15.5:0/3790442138 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f129c06be60 msgr2=0x7f129c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:04.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.962+0000 7f12c0ec3640 1 --2- 172.21.15.5:0/3790442138 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f129c06be60 0x7f129c06e370 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f12bc0d20c0 tx=0x7f12a4008040 comp rx=0 tx=0).stop 2024-05-20T18:30:04.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.962+0000 7f12c0ec3640 1 -- 172.21.15.5:0/3790442138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12bc0cf250 msgr2=0x7f12bc158940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:04.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.962+0000 7f12c0ec3640 1 --2- 172.21.15.5:0/3790442138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12bc0cf250 0x7f12bc158940 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f12ac00c3f0 tx=0x7f12ac00c8b0 comp rx=0 tx=0).stop 2024-05-20T18:30:04.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.962+0000 7f12c0ec3640 1 -- 172.21.15.5:0/3790442138 shutdown_connections 2024-05-20T18:30:04.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.962+0000 7f12c0ec3640 1 --2- 172.21.15.5:0/3790442138 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f129c06be60 0x7f129c06e370 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:04.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.962+0000 7f12c0ec3640 1 --2- 172.21.15.5:0/3790442138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f12bc156650 0x7f12bc155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:04.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.962+0000 7f12c0ec3640 1 --2- 172.21.15.5:0/3790442138 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12bc0d3220 0x7f12bc156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:04.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.962+0000 7f12c0ec3640 1 --2- 172.21.15.5:0/3790442138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12bc0cf250 0x7f12bc158940 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:04.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.962+0000 7f12c0ec3640 1 -- 172.21.15.5:0/3790442138 >> 172.21.15.5:0/3790442138 conn(0x7f12bc0d1860 msgr2=0x7f12bc15af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:04.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.962+0000 7f12c0ec3640 1 -- 172.21.15.5:0/3790442138 shutdown_connections 2024-05-20T18:30:04.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:04.962+0000 7f12c0ec3640 1 -- 172.21.15.5:0/3790442138 wait complete. 2024-05-20T18:30:04.973 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-05-20T18:30:04.617630+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243972,"kb_used_data":6988,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730748,"statfs":{"total":773094113280,"available":772844285952,"internally_reserved":0,"allocated":7155712,"data_stored":3834503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.733530"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.907365+0000","last_change":"2024-05-20T18:30:00.903836+0000","last_active":"2024-05-20T18:30:00.907365+0000","last_peered":"2024-05-20T18:30:00.907365+0000","last_clean":"2024-05-20T18:30:00.907365+0000","last_became_active":"2024-05-20T18:30:00.903479+0000","last_became_peered":"2024-05-20T18:30:00.903479+0000","last_unstale":"2024-05-20T18:30:00.907365+0000","last_undegraded":"2024-05-20T18:30:00.907365+0000","last_fullsized":"2024-05-20T18:30:00.907365+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:43:57.279675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.918183+0000","last_change":"2024-05-20T18:30:00.914512+0000","last_active":"2024-05-20T18:30:00.918183+0000","last_peered":"2024-05-20T18:30:00.918183+0000","last_clean":"2024-05-20T18:30:00.918183+0000","last_became_active":"2024-05-20T18:30:00.914163+0000","last_became_peered":"2024-05-20T18:30:00.914163+0000","last_unstale":"2024-05-20T18:30:00.918183+0000","last_undegraded":"2024-05-20T18:30:00.918183+0000","last_fullsized":"2024-05-20T18:30:00.918183+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:16:21.083333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.928469+0000","last_change":"2024-05-20T18:30:00.924738+0000","last_active":"2024-05-20T18:30:00.928469+0000","last_peered":"2024-05-20T18:30:00.928469+0000","last_clean":"2024-05-20T18:30:00.928469+0000","last_became_active":"2024-05-20T18:30:00.924399+0000","last_became_peered":"2024-05-20T18:30:00.924399+0000","last_unstale":"2024-05-20T18:30:00.928469+0000","last_undegraded":"2024-05-20T18:30:00.928469+0000","last_fullsized":"2024-05-20T18:30:00.928469+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:36:17.825672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.925975+0000","last_change":"2024-05-20T18:30:00.922288+0000","last_active":"2024-05-20T18:30:00.925975+0000","last_peered":"2024-05-20T18:30:00.925975+0000","last_clean":"2024-05-20T18:30:00.925975+0000","last_became_active":"2024-05-20T18:30:00.920734+0000","last_became_peered":"2024-05-20T18:30:00.920734+0000","last_unstale":"2024-05-20T18:30:00.925975+0000","last_undegraded":"2024-05-20T18:30:00.925975+0000","last_fullsized":"2024-05-20T18:30:00.925975+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:18:34.713042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.935217+0000","last_change":"2024-05-20T18:30:00.933767+0000","last_active":"2024-05-20T18:30:00.935217+0000","last_peered":"2024-05-20T18:30:00.935217+0000","last_clean":"2024-05-20T18:30:00.935217+0000","last_became_active":"2024-05-20T18:30:00.933409+0000","last_became_peered":"2024-05-20T18:30:00.933409+0000","last_unstale":"2024-05-20T18:30:00.935217+0000","last_undegraded":"2024-05-20T18:30:00.935217+0000","last_fullsized":"2024-05-20T18:30:00.935217+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:28:10.700582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.922804+0000","last_change":"2024-05-20T18:30:00.920884+0000","last_active":"2024-05-20T18:30:00.922804+0000","last_peered":"2024-05-20T18:30:00.922804+0000","last_clean":"2024-05-20T18:30:00.922804+0000","last_became_active":"2024-05-20T18:30:00.919712+0000","last_became_peered":"2024-05-20T18:30:00.919712+0000","last_unstale":"2024-05-20T18:30:00.922804+0000","last_undegraded":"2024-05-20T18:30:00.922804+0000","last_fullsized":"2024-05-20T18:30:00.922804+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:08:43.620630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.924984+0000","last_change":"2024-05-20T18:30:00.922433+0000","last_active":"2024-05-20T18:30:00.924984+0000","last_peered":"2024-05-20T18:30:00.924984+0000","last_clean":"2024-05-20T18:30:00.924984+0000","last_became_active":"2024-05-20T18:30:00.922100+0000","last_became_peered":"2024-05-20T18:30:00.922100+0000","last_unstale":"2024-05-20T18:30:00.924984+0000","last_undegraded":"2024-05-20T18:30:00.924984+0000","last_fullsized":"2024-05-20T18:30:00.924984+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:03:27.000524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.925063+0000","last_change":"2024-05-20T18:30:00.919507+0000","last_active":"2024-05-20T18:30:00.925063+0000","last_peered":"2024-05-20T18:30:00.925063+0000","last_clean":"2024-05-20T18:30:00.925063+0000","last_became_active":"2024-05-20T18:30:00.919096+0000","last_became_peered":"2024-05-20T18:30:00.919096+0000","last_unstale":"2024-05-20T18:30:00.925063+0000","last_undegraded":"2024-05-20T18:30:00.925063+0000","last_fullsized":"2024-05-20T18:30:00.925063+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:06:04.693803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.937649+0000","last_change":"2024-05-20T18:30:00.936123+0000","last_active":"2024-05-20T18:30:00.937649+0000","last_peered":"2024-05-20T18:30:00.937649+0000","last_clean":"2024-05-20T18:30:00.937649+0000","last_became_active":"2024-05-20T18:30:00.935781+0000","last_became_peered":"2024-05-20T18:30:00.935781+0000","last_unstale":"2024-05-20T18:30:00.937649+0000","last_undegraded":"2024-05-20T18:30:00.937649+0000","last_fullsized":"2024-05-20T18:30:00.937649+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:51:26.939060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.934459+0000","last_change":"2024-05-20T18:30:00.931422+0000","last_active":"2024-05-20T18:30:00.934459+0000","last_peered":"2024-05-20T18:30:00.934459+0000","last_clean":"2024-05-20T18:30:00.934459+0000","last_became_active":"2024-05-20T18:30:00.931056+0000","last_became_peered":"2024-05-20T18:30:00.931056+0000","last_unstale":"2024-05-20T18:30:00.934459+0000","last_undegraded":"2024-05-20T18:30:00.934459+0000","last_fullsized":"2024-05-20T18:30:00.934459+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:16:53.943495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.926043+0000","last_change":"2024-05-20T18:30:00.923607+0000","last_active":"2024-05-20T18:30:00.926043+0000","last_peered":"2024-05-20T18:30:00.926043+0000","last_clean":"2024-05-20T18:30:00.926043+0000","last_became_active":"2024-05-20T18:30:00.923270+0000","last_became_peered":"2024-05-20T18:30:00.923270+0000","last_unstale":"2024-05-20T18:30:00.926043+0000","last_undegraded":"2024-05-20T18:30:00.926043+0000","last_fullsized":"2024-05-20T18:30:00.926043+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:30:20.294826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.918062+0000","last_change":"2024-05-20T18:30:00.917196+0000","last_active":"2024-05-20T18:30:00.918062+0000","last_peered":"2024-05-20T18:30:00.918062+0000","last_clean":"2024-05-20T18:30:00.918062+0000","last_became_active":"2024-05-20T18:30:00.916348+0000","last_became_peered":"2024-05-20T18:30:00.916348+0000","last_unstale":"2024-05-20T18:30:00.918062+0000","last_undegraded":"2024-05-20T18:30:00.918062+0000","last_fullsized":"2024-05-20T18:30:00.918062+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:36:17.825672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.924696+0000","last_change":"2024-05-20T18:30:00.922319+0000","last_active":"2024-05-20T18:30:00.924696+0000","last_peered":"2024-05-20T18:30:00.924696+0000","last_clean":"2024-05-20T18:30:00.924696+0000","last_became_active":"2024-05-20T18:30:00.921938+0000","last_became_peered":"2024-05-20T18:30:00.921938+0000","last_unstale":"2024-05-20T18:30:00.924696+0000","last_undegraded":"2024-05-20T18:30:00.924696+0000","last_fullsized":"2024-05-20T18:30:00.924696+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:51:26.939060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.928093+0000","last_change":"2024-05-20T18:30:00.923901+0000","last_active":"2024-05-20T18:30:00.928093+0000","last_peered":"2024-05-20T18:30:00.928093+0000","last_clean":"2024-05-20T18:30:00.928093+0000","last_became_active":"2024-05-20T18:30:00.923159+0000","last_became_peered":"2024-05-20T18:30:00.923159+0000","last_unstale":"2024-05-20T18:30:00.928093+0000","last_undegraded":"2024-05-20T18:30:00.928093+0000","last_fullsized":"2024-05-20T18:30:00.928093+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:09:55.422091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.924110+0000","last_change":"2024-05-20T18:30:00.920906+0000","last_active":"2024-05-20T18:30:00.924110+0000","last_peered":"2024-05-20T18:30:00.924110+0000","last_clean":"2024-05-20T18:30:00.924110+0000","last_became_active":"2024-05-20T18:30:00.920551+0000","last_became_peered":"2024-05-20T18:30:00.920551+0000","last_unstale":"2024-05-20T18:30:00.924110+0000","last_undegraded":"2024-05-20T18:30:00.924110+0000","last_fullsized":"2024-05-20T18:30:00.924110+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:44:30.446400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.915449+0000","last_change":"2024-05-20T18:30:00.911072+0000","last_active":"2024-05-20T18:30:00.915449+0000","last_peered":"2024-05-20T18:30:00.915449+0000","last_clean":"2024-05-20T18:30:00.915449+0000","last_became_active":"2024-05-20T18:30:00.910257+0000","last_became_peered":"2024-05-20T18:30:00.910257+0000","last_unstale":"2024-05-20T18:30:00.915449+0000","last_undegraded":"2024-05-20T18:30:00.915449+0000","last_fullsized":"2024-05-20T18:30:00.915449+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:01:41.109326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.925997+0000","last_change":"2024-05-20T18:30:00.923216+0000","last_active":"2024-05-20T18:30:00.925997+0000","last_peered":"2024-05-20T18:30:00.925997+0000","last_clean":"2024-05-20T18:30:00.925997+0000","last_became_active":"2024-05-20T18:30:00.922894+0000","last_became_peered":"2024-05-20T18:30:00.922894+0000","last_unstale":"2024-05-20T18:30:00.925997+0000","last_undegraded":"2024-05-20T18:30:00.925997+0000","last_fullsized":"2024-05-20T18:30:00.925997+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:16:27.973018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.925930+0000","last_change":"2024-05-20T18:30:00.922933+0000","last_active":"2024-05-20T18:30:00.925930+0000","last_peered":"2024-05-20T18:30:00.925930+0000","last_clean":"2024-05-20T18:30:00.925930+0000","last_became_active":"2024-05-20T18:30:00.922591+0000","last_became_peered":"2024-05-20T18:30:00.922591+0000","last_unstale":"2024-05-20T18:30:00.925930+0000","last_undegraded":"2024-05-20T18:30:00.925930+0000","last_fullsized":"2024-05-20T18:30:00.925930+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:47:18.701103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.913317+0000","last_change":"2024-05-20T18:30:00.908911+0000","last_active":"2024-05-20T18:30:00.913317+0000","last_peered":"2024-05-20T18:30:00.913317+0000","last_clean":"2024-05-20T18:30:00.913317+0000","last_became_active":"2024-05-20T18:30:00.908607+0000","last_became_peered":"2024-05-20T18:30:00.908607+0000","last_unstale":"2024-05-20T18:30:00.913317+0000","last_undegraded":"2024-05-20T18:30:00.913317+0000","last_fullsized":"2024-05-20T18:30:00.913317+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:41:07.371805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.920083+0000","last_change":"2024-05-20T18:30:00.918049+0000","last_active":"2024-05-20T18:30:00.920083+0000","last_peered":"2024-05-20T18:30:00.920083+0000","last_clean":"2024-05-20T18:30:00.920083+0000","last_became_active":"2024-05-20T18:30:00.917717+0000","last_became_peered":"2024-05-20T18:30:00.917717+0000","last_unstale":"2024-05-20T18:30:00.920083+0000","last_undegraded":"2024-05-20T18:30:00.920083+0000","last_fullsized":"2024-05-20T18:30:00.920083+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:47:02.512109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.908268+0000","last_change":"2024-05-20T18:30:00.906062+0000","last_active":"2024-05-20T18:30:00.908268+0000","last_peered":"2024-05-20T18:30:00.908268+0000","last_clean":"2024-05-20T18:30:00.908268+0000","last_became_active":"2024-05-20T18:30:00.905640+0000","last_became_peered":"2024-05-20T18:30:00.905640+0000","last_unstale":"2024-05-20T18:30:00.908268+0000","last_undegraded":"2024-05-20T18:30:00.908268+0000","last_fullsized":"2024-05-20T18:30:00.908268+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:49:04.539026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.925107+0000","last_change":"2024-05-20T18:30:00.923924+0000","last_active":"2024-05-20T18:30:00.925107+0000","last_peered":"2024-05-20T18:30:00.925107+0000","last_clean":"2024-05-20T18:30:00.925107+0000","last_became_active":"2024-05-20T18:30:00.923552+0000","last_became_peered":"2024-05-20T18:30:00.923552+0000","last_unstale":"2024-05-20T18:30:00.925107+0000","last_undegraded":"2024-05-20T18:30:00.925107+0000","last_fullsized":"2024-05-20T18:30:00.925107+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:25:04.796127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.926660+0000","last_change":"2024-05-20T18:30:00.925501+0000","last_active":"2024-05-20T18:30:00.926660+0000","last_peered":"2024-05-20T18:30:00.926660+0000","last_clean":"2024-05-20T18:30:00.926660+0000","last_became_active":"2024-05-20T18:30:00.925165+0000","last_became_peered":"2024-05-20T18:30:00.925165+0000","last_unstale":"2024-05-20T18:30:00.926660+0000","last_undegraded":"2024-05-20T18:30:00.926660+0000","last_fullsized":"2024-05-20T18:30:00.926660+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:26:58.050238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.924458+0000","last_change":"2024-05-20T18:30:00.922044+0000","last_active":"2024-05-20T18:30:00.924458+0000","last_peered":"2024-05-20T18:30:00.924458+0000","last_clean":"2024-05-20T18:30:00.924458+0000","last_became_active":"2024-05-20T18:30:00.921698+0000","last_became_peered":"2024-05-20T18:30:00.921698+0000","last_unstale":"2024-05-20T18:30:00.924458+0000","last_undegraded":"2024-05-20T18:30:00.924458+0000","last_fullsized":"2024-05-20T18:30:00.924458+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:50:48.116694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.911520+0000","last_change":"2024-05-20T18:30:00.908489+0000","last_active":"2024-05-20T18:30:00.911520+0000","last_peered":"2024-05-20T18:30:00.911520+0000","last_clean":"2024-05-20T18:30:00.911520+0000","last_became_active":"2024-05-20T18:30:00.907914+0000","last_became_peered":"2024-05-20T18:30:00.907914+0000","last_unstale":"2024-05-20T18:30:00.911520+0000","last_undegraded":"2024-05-20T18:30:00.911520+0000","last_fullsized":"2024-05-20T18:30:00.911520+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:07:33.080076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.910868+0000","last_change":"2024-05-20T18:30:00.908887+0000","last_active":"2024-05-20T18:30:00.910868+0000","last_peered":"2024-05-20T18:30:00.910868+0000","last_clean":"2024-05-20T18:30:00.910868+0000","last_became_active":"2024-05-20T18:30:00.908282+0000","last_became_peered":"2024-05-20T18:30:00.908282+0000","last_unstale":"2024-05-20T18:30:00.910868+0000","last_undegraded":"2024-05-20T18:30:00.910868+0000","last_fullsized":"2024-05-20T18:30:00.910868+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:44:30.446400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.926007+0000","last_change":"2024-05-20T18:30:00.921763+0000","last_active":"2024-05-20T18:30:00.926007+0000","last_peered":"2024-05-20T18:30:00.926007+0000","last_clean":"2024-05-20T18:30:00.926007+0000","last_became_active":"2024-05-20T18:30:00.921288+0000","last_became_peered":"2024-05-20T18:30:00.921288+0000","last_unstale":"2024-05-20T18:30:00.926007+0000","last_undegraded":"2024-05-20T18:30:00.926007+0000","last_fullsized":"2024-05-20T18:30:00.926007+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:41:07.371805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.920178+0000","last_change":"2024-05-20T18:30:00.917206+0000","last_active":"2024-05-20T18:30:00.920178+0000","last_peered":"2024-05-20T18:30:00.920178+0000","last_clean":"2024-05-20T18:30:00.920178+0000","last_became_active":"2024-05-20T18:30:00.916857+0000","last_became_peered":"2024-05-20T18:30:00.916857+0000","last_unstale":"2024-05-20T18:30:00.920178+0000","last_undegraded":"2024-05-20T18:30:00.920178+0000","last_fullsized":"2024-05-20T18:30:00.920178+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:05:04.426633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.938484+0000","last_change":"2024-05-20T18:30:00.936670+0000","last_active":"2024-05-20T18:30:00.938484+0000","last_peered":"2024-05-20T18:30:00.938484+0000","last_clean":"2024-05-20T18:30:00.938484+0000","last_became_active":"2024-05-20T18:30:00.936318+0000","last_became_peered":"2024-05-20T18:30:00.936318+0000","last_unstale":"2024-05-20T18:30:00.938484+0000","last_undegraded":"2024-05-20T18:30:00.938484+0000","last_fullsized":"2024-05-20T18:30:00.938484+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:28:01.431319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.919818+0000","last_change":"2024-05-20T18:30:00.918890+0000","last_active":"2024-05-20T18:30:00.919818+0000","last_peered":"2024-05-20T18:30:00.919818+0000","last_clean":"2024-05-20T18:30:00.919818+0000","last_became_active":"2024-05-20T18:30:00.918530+0000","last_became_peered":"2024-05-20T18:30:00.918530+0000","last_unstale":"2024-05-20T18:30:00.919818+0000","last_undegraded":"2024-05-20T18:30:00.919818+0000","last_fullsized":"2024-05-20T18:30:00.919818+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:16:53.943495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.924265+0000","last_change":"2024-05-20T18:30:00.915178+0000","last_active":"2024-05-20T18:30:00.924265+0000","last_peered":"2024-05-20T18:30:00.924265+0000","last_clean":"2024-05-20T18:30:00.924265+0000","last_became_active":"2024-05-20T18:30:00.914832+0000","last_became_peered":"2024-05-20T18:30:00.914832+0000","last_unstale":"2024-05-20T18:30:00.924265+0000","last_undegraded":"2024-05-20T18:30:00.924265+0000","last_fullsized":"2024-05-20T18:30:00.924265+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:26:09.155714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.916091+0000","last_change":"2024-05-20T18:30:00.912543+0000","last_active":"2024-05-20T18:30:00.916091+0000","last_peered":"2024-05-20T18:30:00.916091+0000","last_clean":"2024-05-20T18:30:00.916091+0000","last_became_active":"2024-05-20T18:30:00.912187+0000","last_became_peered":"2024-05-20T18:30:00.912187+0000","last_unstale":"2024-05-20T18:30:00.916091+0000","last_undegraded":"2024-05-20T18:30:00.916091+0000","last_fullsized":"2024-05-20T18:30:00.916091+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:35:15.347397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.923922+0000","last_change":"2024-05-20T18:30:00.917405+0000","last_active":"2024-05-20T18:30:00.923922+0000","last_peered":"2024-05-20T18:30:00.923922+0000","last_clean":"2024-05-20T18:30:00.923922+0000","last_became_active":"2024-05-20T18:30:00.916938+0000","last_became_peered":"2024-05-20T18:30:00.916938+0000","last_unstale":"2024-05-20T18:30:00.923922+0000","last_undegraded":"2024-05-20T18:30:00.923922+0000","last_fullsized":"2024-05-20T18:30:00.923922+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:14:57.402663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"62'4","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.935502+0000","last_change":"2024-05-20T18:29:59.912962+0000","last_active":"2024-05-20T18:30:00.935502+0000","last_peered":"2024-05-20T18:30:00.935502+0000","last_clean":"2024-05-20T18:30:00.935502+0000","last_became_active":"2024-05-20T18:29:59.912641+0000","last_became_peered":"2024-05-20T18:29:59.912641+0000","last_unstale":"2024-05-20T18:30:00.935502+0000","last_undegraded":"2024-05-20T18:30:00.935502+0000","last_fullsized":"2024-05-20T18:30:00.935502+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:37:37.860286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.930085+0000","last_change":"2024-05-20T18:30:00.928426+0000","last_active":"2024-05-20T18:30:00.930085+0000","last_peered":"2024-05-20T18:30:00.930085+0000","last_clean":"2024-05-20T18:30:00.930085+0000","last_became_active":"2024-05-20T18:30:00.928075+0000","last_became_peered":"2024-05-20T18:30:00.928075+0000","last_unstale":"2024-05-20T18:30:00.930085+0000","last_undegraded":"2024-05-20T18:30:00.930085+0000","last_fullsized":"2024-05-20T18:30:00.930085+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:56:29.475238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.931890+0000","last_change":"2024-05-20T18:29:59.909960+0000","last_active":"2024-05-20T18:30:00.931890+0000","last_peered":"2024-05-20T18:30:00.931890+0000","last_clean":"2024-05-20T18:30:00.931890+0000","last_became_active":"2024-05-20T18:29:59.909629+0000","last_became_peered":"2024-05-20T18:29:59.909629+0000","last_unstale":"2024-05-20T18:30:00.931890+0000","last_undegraded":"2024-05-20T18:30:00.931890+0000","last_fullsized":"2024-05-20T18:30:00.931890+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:28:20.832622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.916129+0000","last_change":"2024-05-20T18:30:00.913579+0000","last_active":"2024-05-20T18:30:00.916129+0000","last_peered":"2024-05-20T18:30:00.916129+0000","last_clean":"2024-05-20T18:30:00.916129+0000","last_became_active":"2024-05-20T18:30:00.913193+0000","last_became_peered":"2024-05-20T18:30:00.913193+0000","last_unstale":"2024-05-20T18:30:00.916129+0000","last_undegraded":"2024-05-20T18:30:00.916129+0000","last_fullsized":"2024-05-20T18:30:00.916129+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:13:19.360804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.929866+0000","last_change":"2024-05-20T18:29:59.915264+0000","last_active":"2024-05-20T18:30:00.929866+0000","last_peered":"2024-05-20T18:30:00.929866+0000","last_clean":"2024-05-20T18:30:00.929866+0000","last_became_active":"2024-05-20T18:29:59.914809+0000","last_became_peered":"2024-05-20T18:29:59.914809+0000","last_unstale":"2024-05-20T18:30:00.929866+0000","last_undegraded":"2024-05-20T18:30:00.929866+0000","last_fullsized":"2024-05-20T18:30:00.929866+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:51:25.929539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.935305+0000","last_change":"2024-05-20T18:30:00.932564+0000","last_active":"2024-05-20T18:30:00.935305+0000","last_peered":"2024-05-20T18:30:00.935305+0000","last_clean":"2024-05-20T18:30:00.935305+0000","last_became_active":"2024-05-20T18:30:00.931947+0000","last_became_peered":"2024-05-20T18:30:00.931947+0000","last_unstale":"2024-05-20T18:30:00.935305+0000","last_undegraded":"2024-05-20T18:30:00.935305+0000","last_fullsized":"2024-05-20T18:30:00.935305+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:06:04.693803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.929128+0000","last_change":"2024-05-20T18:29:59.908215+0000","last_active":"2024-05-20T18:30:00.929128+0000","last_peered":"2024-05-20T18:30:00.929128+0000","last_clean":"2024-05-20T18:30:00.929128+0000","last_became_active":"2024-05-20T18:29:59.907812+0000","last_became_peered":"2024-05-20T18:29:59.907812+0000","last_unstale":"2024-05-20T18:30:00.929128+0000","last_undegraded":"2024-05-20T18:30:00.929128+0000","last_fullsized":"2024-05-20T18:30:00.929128+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:08:44.746640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.915426+0000","last_change":"2024-05-20T18:30:00.908783+0000","last_active":"2024-05-20T18:30:00.915426+0000","last_peered":"2024-05-20T18:30:00.915426+0000","last_clean":"2024-05-20T18:30:00.915426+0000","last_became_active":"2024-05-20T18:30:00.908345+0000","last_became_peered":"2024-05-20T18:30:00.908345+0000","last_unstale":"2024-05-20T18:30:00.915426+0000","last_undegraded":"2024-05-20T18:30:00.915426+0000","last_fullsized":"2024-05-20T18:30:00.915426+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:39:10.658137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.925846+0000","last_change":"2024-05-20T18:29:59.910015+0000","last_active":"2024-05-20T18:30:00.925846+0000","last_peered":"2024-05-20T18:30:00.925846+0000","last_clean":"2024-05-20T18:30:00.925846+0000","last_became_active":"2024-05-20T18:29:59.909607+0000","last_became_peered":"2024-05-20T18:29:59.909607+0000","last_unstale":"2024-05-20T18:30:00.925846+0000","last_undegraded":"2024-05-20T18:30:00.925846+0000","last_fullsized":"2024-05-20T18:30:00.925846+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:51:48.737011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.925689+0000","last_change":"2024-05-20T18:30:00.924033+0000","last_active":"2024-05-20T18:30:00.925689+0000","last_peered":"2024-05-20T18:30:00.925689+0000","last_clean":"2024-05-20T18:30:00.925689+0000","last_became_active":"2024-05-20T18:30:00.922154+0000","last_became_peered":"2024-05-20T18:30:00.922154+0000","last_unstale":"2024-05-20T18:30:00.925689+0000","last_undegraded":"2024-05-20T18:30:00.925689+0000","last_fullsized":"2024-05-20T18:30:00.925689+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:27:40.715088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.924184+0000","last_change":"2024-05-20T18:29:59.906090+0000","last_active":"2024-05-20T18:30:00.924184+0000","last_peered":"2024-05-20T18:30:00.924184+0000","last_clean":"2024-05-20T18:30:00.924184+0000","last_became_active":"2024-05-20T18:29:59.905762+0000","last_became_peered":"2024-05-20T18:29:59.905762+0000","last_unstale":"2024-05-20T18:30:00.924184+0000","last_undegraded":"2024-05-20T18:30:00.924184+0000","last_fullsized":"2024-05-20T18:30:00.924184+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:37:11.166078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.912188+0000","last_change":"2024-05-20T18:30:00.910857+0000","last_active":"2024-05-20T18:30:00.912188+0000","last_peered":"2024-05-20T18:30:00.912188+0000","last_clean":"2024-05-20T18:30:00.912188+0000","last_became_active":"2024-05-20T18:30:00.909676+0000","last_became_peered":"2024-05-20T18:30:00.909676+0000","last_unstale":"2024-05-20T18:30:00.912188+0000","last_undegraded":"2024-05-20T18:30:00.912188+0000","last_fullsized":"2024-05-20T18:30:00.912188+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:47:18.701103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.900624+0000","last_change":"2024-05-20T18:29:59.914083+0000","last_active":"2024-05-20T18:30:00.900624+0000","last_peered":"2024-05-20T18:30:00.900624+0000","last_clean":"2024-05-20T18:30:00.900624+0000","last_became_active":"2024-05-20T18:29:59.913726+0000","last_became_peered":"2024-05-20T18:29:59.913726+0000","last_unstale":"2024-05-20T18:30:00.900624+0000","last_undegraded":"2024-05-20T18:30:00.900624+0000","last_fullsized":"2024-05-20T18:30:00.900624+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:09:28.306514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.920112+0000","last_change":"2024-05-20T18:30:00.918437+0000","last_active":"2024-05-20T18:30:00.920112+0000","last_peered":"2024-05-20T18:30:00.920112+0000","last_clean":"2024-05-20T18:30:00.920112+0000","last_became_active":"2024-05-20T18:30:00.918110+0000","last_became_peered":"2024-05-20T18:30:00.918110+0000","last_unstale":"2024-05-20T18:30:00.920112+0000","last_undegraded":"2024-05-20T18:30:00.920112+0000","last_fullsized":"2024-05-20T18:30:00.920112+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:10:47.393081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.898571+0000","last_change":"2024-05-20T18:29:59.909196+0000","last_active":"2024-05-20T18:30:00.898571+0000","last_peered":"2024-05-20T18:30:00.898571+0000","last_clean":"2024-05-20T18:30:00.898571+0000","last_became_active":"2024-05-20T18:29:59.908843+0000","last_became_peered":"2024-05-20T18:29:59.908843+0000","last_unstale":"2024-05-20T18:30:00.898571+0000","last_undegraded":"2024-05-20T18:30:00.898571+0000","last_fullsized":"2024-05-20T18:30:00.898571+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:21:46.253714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.911291+0000","last_change":"2024-05-20T18:30:00.908719+0000","last_active":"2024-05-20T18:30:00.911291+0000","last_peered":"2024-05-20T18:30:00.911291+0000","last_clean":"2024-05-20T18:30:00.911291+0000","last_became_active":"2024-05-20T18:30:00.907726+0000","last_became_peered":"2024-05-20T18:30:00.907726+0000","last_unstale":"2024-05-20T18:30:00.911291+0000","last_undegraded":"2024-05-20T18:30:00.911291+0000","last_fullsized":"2024-05-20T18:30:00.911291+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:43:27.761088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.918429+0000","last_change":"2024-05-20T18:29:59.917401+0000","last_active":"2024-05-20T18:30:00.918429+0000","last_peered":"2024-05-20T18:30:00.918429+0000","last_clean":"2024-05-20T18:30:00.918429+0000","last_became_active":"2024-05-20T18:29:59.916950+0000","last_became_peered":"2024-05-20T18:29:59.916950+0000","last_unstale":"2024-05-20T18:30:00.918429+0000","last_undegraded":"2024-05-20T18:30:00.918429+0000","last_fullsized":"2024-05-20T18:30:00.918429+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:07:10.468378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.907376+0000","last_change":"2024-05-20T18:30:00.903962+0000","last_active":"2024-05-20T18:30:00.907376+0000","last_peered":"2024-05-20T18:30:00.907376+0000","last_clean":"2024-05-20T18:30:00.907376+0000","last_became_active":"2024-05-20T18:30:00.903610+0000","last_became_peered":"2024-05-20T18:30:00.903610+0000","last_unstale":"2024-05-20T18:30:00.907376+0000","last_undegraded":"2024-05-20T18:30:00.907376+0000","last_fullsized":"2024-05-20T18:30:00.907376+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:04:16.563394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.908162+0000","last_change":"2024-05-20T18:29:59.908355+0000","last_active":"2024-05-20T18:30:00.908162+0000","last_peered":"2024-05-20T18:30:00.908162+0000","last_clean":"2024-05-20T18:30:00.908162+0000","last_became_active":"2024-05-20T18:29:59.907685+0000","last_became_peered":"2024-05-20T18:29:59.907685+0000","last_unstale":"2024-05-20T18:30:00.908162+0000","last_undegraded":"2024-05-20T18:30:00.908162+0000","last_fullsized":"2024-05-20T18:30:00.908162+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:21:46.253714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.911529+0000","last_change":"2024-05-20T18:30:00.907509+0000","last_active":"2024-05-20T18:30:00.911529+0000","last_peered":"2024-05-20T18:30:00.911529+0000","last_clean":"2024-05-20T18:30:00.911529+0000","last_became_active":"2024-05-20T18:30:00.905850+0000","last_became_peered":"2024-05-20T18:30:00.905850+0000","last_unstale":"2024-05-20T18:30:00.911529+0000","last_undegraded":"2024-05-20T18:30:00.911529+0000","last_fullsized":"2024-05-20T18:30:00.911529+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:56:05.710762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"62'3","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.932562+0000","last_change":"2024-05-20T18:29:59.917391+0000","last_active":"2024-05-20T18:30:00.932562+0000","last_peered":"2024-05-20T18:30:00.932562+0000","last_clean":"2024-05-20T18:30:00.932562+0000","last_became_active":"2024-05-20T18:29:59.916996+0000","last_became_peered":"2024-05-20T18:29:59.916996+0000","last_unstale":"2024-05-20T18:30:00.932562+0000","last_undegraded":"2024-05-20T18:30:00.932562+0000","last_fullsized":"2024-05-20T18:30:00.932562+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:49:03.529505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.924297+0000","last_change":"2024-05-20T18:30:00.918564+0000","last_active":"2024-05-20T18:30:00.924297+0000","last_peered":"2024-05-20T18:30:00.924297+0000","last_clean":"2024-05-20T18:30:00.924297+0000","last_became_active":"2024-05-20T18:30:00.915835+0000","last_became_peered":"2024-05-20T18:30:00.915835+0000","last_unstale":"2024-05-20T18:30:00.924297+0000","last_undegraded":"2024-05-20T18:30:00.924297+0000","last_fullsized":"2024-05-20T18:30:00.924297+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:45:08.997389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"62'6","reported_seq":18,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.938905+0000","last_change":"2024-05-20T18:29:59.912482+0000","last_active":"2024-05-20T18:30:00.938905+0000","last_peered":"2024-05-20T18:30:00.938905+0000","last_clean":"2024-05-20T18:30:00.938905+0000","last_became_active":"2024-05-20T18:29:59.911897+0000","last_became_peered":"2024-05-20T18:29:59.911897+0000","last_unstale":"2024-05-20T18:30:00.938905+0000","last_undegraded":"2024-05-20T18:30:00.938905+0000","last_fullsized":"2024-05-20T18:30:00.938905+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:28:09.691061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.936467+0000","last_change":"2024-05-20T18:30:00.933298+0000","last_active":"2024-05-20T18:30:00.936467+0000","last_peered":"2024-05-20T18:30:00.936467+0000","last_clean":"2024-05-20T18:30:00.936467+0000","last_became_active":"2024-05-20T18:30:00.932894+0000","last_became_peered":"2024-05-20T18:30:00.932894+0000","last_unstale":"2024-05-20T18:30:00.936467+0000","last_undegraded":"2024-05-20T18:30:00.936467+0000","last_fullsized":"2024-05-20T18:30:00.936467+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:37:38.869806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.935648+0000","last_change":"2024-05-20T18:29:59.917535+0000","last_active":"2024-05-20T18:30:00.935648+0000","last_peered":"2024-05-20T18:30:00.935648+0000","last_clean":"2024-05-20T18:30:00.935648+0000","last_became_active":"2024-05-20T18:29:59.917132+0000","last_became_peered":"2024-05-20T18:29:59.917132+0000","last_unstale":"2024-05-20T18:30:00.935648+0000","last_undegraded":"2024-05-20T18:30:00.935648+0000","last_fullsized":"2024-05-20T18:30:00.935648+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:12:42.531877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.917489+0000","last_change":"2024-05-20T18:30:00.915439+0000","last_active":"2024-05-20T18:30:00.917489+0000","last_peered":"2024-05-20T18:30:00.917489+0000","last_clean":"2024-05-20T18:30:00.917489+0000","last_became_active":"2024-05-20T18:30:00.915101+0000","last_became_peered":"2024-05-20T18:30:00.915101+0000","last_unstale":"2024-05-20T18:30:00.917489+0000","last_undegraded":"2024-05-20T18:30:00.917489+0000","last_fullsized":"2024-05-20T18:30:00.917489+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:09:55.422091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.898687+0000","last_change":"2024-05-20T18:29:59.914563+0000","last_active":"2024-05-20T18:30:00.898687+0000","last_peered":"2024-05-20T18:30:00.898687+0000","last_clean":"2024-05-20T18:30:00.898687+0000","last_became_active":"2024-05-20T18:29:59.914241+0000","last_became_peered":"2024-05-20T18:29:59.914241+0000","last_unstale":"2024-05-20T18:30:00.898687+0000","last_undegraded":"2024-05-20T18:30:00.898687+0000","last_fullsized":"2024-05-20T18:30:00.898687+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:48:26.490874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.911805+0000","last_change":"2024-05-20T18:30:00.908147+0000","last_active":"2024-05-20T18:30:00.911805+0000","last_peered":"2024-05-20T18:30:00.911805+0000","last_clean":"2024-05-20T18:30:00.911805+0000","last_became_active":"2024-05-20T18:30:00.907844+0000","last_became_peered":"2024-05-20T18:30:00.907844+0000","last_unstale":"2024-05-20T18:30:00.911805+0000","last_undegraded":"2024-05-20T18:30:00.911805+0000","last_fullsized":"2024-05-20T18:30:00.911805+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:26:09.155714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.910964+0000","last_change":"2024-05-20T18:29:59.916588+0000","last_active":"2024-05-20T18:30:00.910964+0000","last_peered":"2024-05-20T18:30:00.910964+0000","last_clean":"2024-05-20T18:30:00.910964+0000","last_became_active":"2024-05-20T18:29:59.916303+0000","last_became_peered":"2024-05-20T18:29:59.916303+0000","last_unstale":"2024-05-20T18:30:00.910964+0000","last_undegraded":"2024-05-20T18:30:00.910964+0000","last_fullsized":"2024-05-20T18:30:00.910964+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:48:26.490874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.928129+0000","last_change":"2024-05-20T18:30:00.927056+0000","last_active":"2024-05-20T18:30:00.928129+0000","last_peered":"2024-05-20T18:30:00.928129+0000","last_clean":"2024-05-20T18:30:00.928129+0000","last_became_active":"2024-05-20T18:30:00.926692+0000","last_became_peered":"2024-05-20T18:30:00.926692+0000","last_unstale":"2024-05-20T18:30:00.928129+0000","last_undegraded":"2024-05-20T18:30:00.928129+0000","last_fullsized":"2024-05-20T18:30:00.928129+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:48:27.500395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"62'8","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.938975+0000","last_change":"2024-05-20T18:29:59.913397+0000","last_active":"2024-05-20T18:30:00.938975+0000","last_peered":"2024-05-20T18:30:00.938975+0000","last_clean":"2024-05-20T18:30:00.938975+0000","last_became_active":"2024-05-20T18:29:59.913024+0000","last_became_peered":"2024-05-20T18:29:59.913024+0000","last_unstale":"2024-05-20T18:30:00.938975+0000","last_undegraded":"2024-05-20T18:30:00.938975+0000","last_fullsized":"2024-05-20T18:30:00.938975+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:56:04.701242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.938576+0000","last_change":"2024-05-20T18:30:00.936700+0000","last_active":"2024-05-20T18:30:00.938576+0000","last_peered":"2024-05-20T18:30:00.938576+0000","last_clean":"2024-05-20T18:30:00.938576+0000","last_became_active":"2024-05-20T18:30:00.936371+0000","last_became_peered":"2024-05-20T18:30:00.936371+0000","last_unstale":"2024-05-20T18:30:00.938576+0000","last_undegraded":"2024-05-20T18:30:00.938576+0000","last_fullsized":"2024-05-20T18:30:00.938576+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:54:39.522862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.930100+0000","last_change":"2024-05-20T18:29:59.913822+0000","last_active":"2024-05-20T18:30:00.930100+0000","last_peered":"2024-05-20T18:30:00.930100+0000","last_clean":"2024-05-20T18:30:00.930100+0000","last_became_active":"2024-05-20T18:29:59.913482+0000","last_became_peered":"2024-05-20T18:29:59.913482+0000","last_unstale":"2024-05-20T18:30:00.930100+0000","last_undegraded":"2024-05-20T18:30:00.930100+0000","last_fullsized":"2024-05-20T18:30:00.930100+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:51:25.023616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.925912+0000","last_change":"2024-05-20T18:30:00.922460+0000","last_active":"2024-05-20T18:30:00.925912+0000","last_peered":"2024-05-20T18:30:00.925912+0000","last_clean":"2024-05-20T18:30:00.925912+0000","last_became_active":"2024-05-20T18:30:00.922101+0000","last_became_peered":"2024-05-20T18:30:00.922101+0000","last_unstale":"2024-05-20T18:30:00.925912+0000","last_undegraded":"2024-05-20T18:30:00.925912+0000","last_fullsized":"2024-05-20T18:30:00.925912+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:37:38.869806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.938493+0000","last_change":"2024-05-20T18:29:59.917144+0000","last_active":"2024-05-20T18:30:00.938493+0000","last_peered":"2024-05-20T18:30:00.938493+0000","last_clean":"2024-05-20T18:30:00.938493+0000","last_became_active":"2024-05-20T18:29:59.916729+0000","last_became_peered":"2024-05-20T18:29:59.916729+0000","last_unstale":"2024-05-20T18:30:00.938493+0000","last_undegraded":"2024-05-20T18:30:00.938493+0000","last_fullsized":"2024-05-20T18:30:00.938493+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:51:25.023616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.930191+0000","last_change":"2024-05-20T18:30:00.927816+0000","last_active":"2024-05-20T18:30:00.930191+0000","last_peered":"2024-05-20T18:30:00.930191+0000","last_clean":"2024-05-20T18:30:00.930191+0000","last_became_active":"2024-05-20T18:30:00.926661+0000","last_became_peered":"2024-05-20T18:30:00.926661+0000","last_unstale":"2024-05-20T18:30:00.930191+0000","last_undegraded":"2024-05-20T18:30:00.930191+0000","last_fullsized":"2024-05-20T18:30:00.930191+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:36:24.995925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.934337+0000","last_change":"2024-05-20T18:29:59.917688+0000","last_active":"2024-05-20T18:30:00.934337+0000","last_peered":"2024-05-20T18:30:00.934337+0000","last_clean":"2024-05-20T18:30:00.934337+0000","last_became_active":"2024-05-20T18:29:59.916998+0000","last_became_peered":"2024-05-20T18:29:59.916998+0000","last_unstale":"2024-05-20T18:30:00.934337+0000","last_undegraded":"2024-05-20T18:30:00.934337+0000","last_fullsized":"2024-05-20T18:30:00.934337+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:47:38.443964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.918861+0000","last_change":"2024-05-20T18:30:00.915834+0000","last_active":"2024-05-20T18:30:00.918861+0000","last_peered":"2024-05-20T18:30:00.918861+0000","last_clean":"2024-05-20T18:30:00.918861+0000","last_became_active":"2024-05-20T18:30:00.915411+0000","last_became_peered":"2024-05-20T18:30:00.915411+0000","last_unstale":"2024-05-20T18:30:00.918861+0000","last_undegraded":"2024-05-20T18:30:00.918861+0000","last_fullsized":"2024-05-20T18:30:00.918861+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:36:24.995925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.900281+0000","last_change":"2024-05-20T18:29:59.912876+0000","last_active":"2024-05-20T18:30:00.900281+0000","last_peered":"2024-05-20T18:30:00.900281+0000","last_clean":"2024-05-20T18:30:00.900281+0000","last_became_active":"2024-05-20T18:29:59.912423+0000","last_became_peered":"2024-05-20T18:29:59.912423+0000","last_unstale":"2024-05-20T18:30:00.900281+0000","last_undegraded":"2024-05-20T18:30:00.900281+0000","last_fullsized":"2024-05-20T18:30:00.900281+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:44:20.302541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.910090+0000","last_change":"2024-05-20T18:30:00.908373+0000","last_active":"2024-05-20T18:30:00.910090+0000","last_peered":"2024-05-20T18:30:00.910090+0000","last_clean":"2024-05-20T18:30:00.910090+0000","last_became_active":"2024-05-20T18:30:00.907994+0000","last_became_peered":"2024-05-20T18:30:00.907994+0000","last_unstale":"2024-05-20T18:30:00.910090+0000","last_undegraded":"2024-05-20T18:30:00.910090+0000","last_fullsized":"2024-05-20T18:30:00.910090+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:01:04.634825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.927827+0000","last_change":"2024-05-20T18:29:59.913869+0000","last_active":"2024-05-20T18:30:00.927827+0000","last_peered":"2024-05-20T18:30:00.927827+0000","last_clean":"2024-05-20T18:30:00.927827+0000","last_became_active":"2024-05-20T18:29:59.913512+0000","last_became_peered":"2024-05-20T18:29:59.913512+0000","last_unstale":"2024-05-20T18:30:00.927827+0000","last_undegraded":"2024-05-20T18:30:00.927827+0000","last_fullsized":"2024-05-20T18:30:00.927827+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:06:03.684282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.910068+0000","last_change":"2024-05-20T18:30:00.907372+0000","last_active":"2024-05-20T18:30:00.910068+0000","last_peered":"2024-05-20T18:30:00.910068+0000","last_clean":"2024-05-20T18:30:00.910068+0000","last_became_active":"2024-05-20T18:30:00.907051+0000","last_became_peered":"2024-05-20T18:30:00.907051+0000","last_unstale":"2024-05-20T18:30:00.910068+0000","last_undegraded":"2024-05-20T18:30:00.910068+0000","last_fullsized":"2024-05-20T18:30:00.910068+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:12:43.541398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.898593+0000","last_change":"2024-05-20T18:29:59.912694+0000","last_active":"2024-05-20T18:30:00.898593+0000","last_peered":"2024-05-20T18:30:00.898593+0000","last_clean":"2024-05-20T18:30:00.898593+0000","last_became_active":"2024-05-20T18:29:59.912270+0000","last_became_peered":"2024-05-20T18:29:59.912270+0000","last_unstale":"2024-05-20T18:30:00.898593+0000","last_undegraded":"2024-05-20T18:30:00.898593+0000","last_fullsized":"2024-05-20T18:30:00.898593+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:16:34.627569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.914589+0000","last_change":"2024-05-20T18:30:00.911187+0000","last_active":"2024-05-20T18:30:00.914589+0000","last_peered":"2024-05-20T18:30:00.914589+0000","last_clean":"2024-05-20T18:30:00.914589+0000","last_became_active":"2024-05-20T18:30:00.910336+0000","last_became_peered":"2024-05-20T18:30:00.910336+0000","last_unstale":"2024-05-20T18:30:00.914589+0000","last_undegraded":"2024-05-20T18:30:00.914589+0000","last_fullsized":"2024-05-20T18:30:00.914589+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:30:20.294826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.901900+0000","last_change":"2024-05-20T18:29:59.909115+0000","last_active":"2024-05-20T18:30:00.901900+0000","last_peered":"2024-05-20T18:30:00.901900+0000","last_clean":"2024-05-20T18:30:00.901900+0000","last_became_active":"2024-05-20T18:29:59.908764+0000","last_became_peered":"2024-05-20T18:29:59.908764+0000","last_unstale":"2024-05-20T18:30:00.901900+0000","last_undegraded":"2024-05-20T18:30:00.901900+0000","last_fullsized":"2024-05-20T18:30:00.901900+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:54:38.513342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.927281+0000","last_change":"2024-05-20T18:30:00.925040+0000","last_active":"2024-05-20T18:30:00.927281+0000","last_peered":"2024-05-20T18:30:00.927281+0000","last_clean":"2024-05-20T18:30:00.927281+0000","last_became_active":"2024-05-20T18:30:00.924714+0000","last_became_peered":"2024-05-20T18:30:00.924714+0000","last_unstale":"2024-05-20T18:30:00.927281+0000","last_undegraded":"2024-05-20T18:30:00.927281+0000","last_fullsized":"2024-05-20T18:30:00.927281+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:33:26.163997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.934910+0000","last_change":"2024-05-20T18:29:59.913604+0000","last_active":"2024-05-20T18:30:00.934910+0000","last_peered":"2024-05-20T18:30:00.934910+0000","last_clean":"2024-05-20T18:30:00.934910+0000","last_became_active":"2024-05-20T18:29:59.913292+0000","last_became_peered":"2024-05-20T18:29:59.913292+0000","last_unstale":"2024-05-20T18:30:00.934910+0000","last_undegraded":"2024-05-20T18:30:00.934910+0000","last_fullsized":"2024-05-20T18:30:00.934910+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:07:10.468378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.915190+0000","last_change":"2024-05-20T18:30:00.910103+0000","last_active":"2024-05-20T18:30:00.915190+0000","last_peered":"2024-05-20T18:30:00.915190+0000","last_clean":"2024-05-20T18:30:00.915190+0000","last_became_active":"2024-05-20T18:30:00.909685+0000","last_became_peered":"2024-05-20T18:30:00.909685+0000","last_unstale":"2024-05-20T18:30:00.915190+0000","last_undegraded":"2024-05-20T18:30:00.915190+0000","last_fullsized":"2024-05-20T18:30:00.915190+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:14:36.991839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.899625+0000","last_change":"2024-05-20T18:29:59.914038+0000","last_active":"2024-05-20T18:30:00.899625+0000","last_peered":"2024-05-20T18:30:00.899625+0000","last_clean":"2024-05-20T18:30:00.899625+0000","last_became_active":"2024-05-20T18:29:59.913670+0000","last_became_peered":"2024-05-20T18:29:59.913670+0000","last_unstale":"2024-05-20T18:30:00.899625+0000","last_undegraded":"2024-05-20T18:30:00.899625+0000","last_fullsized":"2024-05-20T18:30:00.899625+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:47:38.443964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.916316+0000","last_change":"2024-05-20T18:30:00.914756+0000","last_active":"2024-05-20T18:30:00.916316+0000","last_peered":"2024-05-20T18:30:00.916316+0000","last_clean":"2024-05-20T18:30:00.916316+0000","last_became_active":"2024-05-20T18:30:00.914386+0000","last_became_peered":"2024-05-20T18:30:00.914386+0000","last_unstale":"2024-05-20T18:30:00.916316+0000","last_undegraded":"2024-05-20T18:30:00.916316+0000","last_fullsized":"2024-05-20T18:30:00.916316+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:50:48.116694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.901325+0000","last_change":"2024-05-20T18:29:59.915114+0000","last_active":"2024-05-20T18:30:00.901325+0000","last_peered":"2024-05-20T18:30:00.901325+0000","last_clean":"2024-05-20T18:30:00.901325+0000","last_became_active":"2024-05-20T18:29:59.914755+0000","last_became_peered":"2024-05-20T18:29:59.914755+0000","last_unstale":"2024-05-20T18:30:00.901325+0000","last_undegraded":"2024-05-20T18:30:00.901325+0000","last_fullsized":"2024-05-20T18:30:00.901325+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:28:00.421798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.911584+0000","last_change":"2024-05-20T18:30:00.908378+0000","last_active":"2024-05-20T18:30:00.911584+0000","last_peered":"2024-05-20T18:30:00.911584+0000","last_clean":"2024-05-20T18:30:00.911584+0000","last_became_active":"2024-05-20T18:30:00.908050+0000","last_became_peered":"2024-05-20T18:30:00.908050+0000","last_unstale":"2024-05-20T18:30:00.911584+0000","last_undegraded":"2024-05-20T18:30:00.911584+0000","last_fullsized":"2024-05-20T18:30:00.911584+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:45:08.997389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.896912+0000","last_change":"2024-05-20T18:29:59.907761+0000","last_active":"2024-05-20T18:30:00.896912+0000","last_peered":"2024-05-20T18:30:00.896912+0000","last_clean":"2024-05-20T18:30:00.896912+0000","last_became_active":"2024-05-20T18:29:59.907378+0000","last_became_peered":"2024-05-20T18:29:59.907378+0000","last_unstale":"2024-05-20T18:30:00.896912+0000","last_undegraded":"2024-05-20T18:30:00.896912+0000","last_fullsized":"2024-05-20T18:30:00.896912+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:48:23.509684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.923807+0000","last_change":"2024-05-20T18:30:00.917188+0000","last_active":"2024-05-20T18:30:00.923807+0000","last_peered":"2024-05-20T18:30:00.923807+0000","last_clean":"2024-05-20T18:30:00.923807+0000","last_became_active":"2024-05-20T18:30:00.916715+0000","last_became_peered":"2024-05-20T18:30:00.916715+0000","last_unstale":"2024-05-20T18:30:00.923807+0000","last_undegraded":"2024-05-20T18:30:00.923807+0000","last_fullsized":"2024-05-20T18:30:00.923807+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:56:29.475238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.919432+0000","last_change":"2024-05-20T18:29:59.916303+0000","last_active":"2024-05-20T18:30:00.919432+0000","last_peered":"2024-05-20T18:30:00.919432+0000","last_clean":"2024-05-20T18:30:00.919432+0000","last_became_active":"2024-05-20T18:29:59.915993+0000","last_became_peered":"2024-05-20T18:29:59.915993+0000","last_unstale":"2024-05-20T18:30:00.919432+0000","last_undegraded":"2024-05-20T18:30:00.919432+0000","last_fullsized":"2024-05-20T18:30:00.919432+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:07:32.070556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.936358+0000","last_change":"2024-05-20T18:30:00.932561+0000","last_active":"2024-05-20T18:30:00.936358+0000","last_peered":"2024-05-20T18:30:00.936358+0000","last_clean":"2024-05-20T18:30:00.936358+0000","last_became_active":"2024-05-20T18:30:00.932232+0000","last_became_peered":"2024-05-20T18:30:00.932232+0000","last_unstale":"2024-05-20T18:30:00.936358+0000","last_undegraded":"2024-05-20T18:30:00.936358+0000","last_fullsized":"2024-05-20T18:30:00.936358+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:14:57.402663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.899929+0000","last_change":"2024-05-20T18:29:59.908482+0000","last_active":"2024-05-20T18:30:00.899929+0000","last_peered":"2024-05-20T18:30:00.899929+0000","last_clean":"2024-05-20T18:30:00.899929+0000","last_became_active":"2024-05-20T18:29:59.906921+0000","last_became_peered":"2024-05-20T18:29:59.906921+0000","last_unstale":"2024-05-20T18:30:00.899929+0000","last_undegraded":"2024-05-20T18:30:00.899929+0000","last_fullsized":"2024-05-20T18:30:00.899929+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:15:45.016943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.927423+0000","last_change":"2024-05-20T18:30:00.926328+0000","last_active":"2024-05-20T18:30:00.927423+0000","last_peered":"2024-05-20T18:30:00.927423+0000","last_clean":"2024-05-20T18:30:00.927423+0000","last_became_active":"2024-05-20T18:30:00.923766+0000","last_became_peered":"2024-05-20T18:30:00.923766+0000","last_unstale":"2024-05-20T18:30:00.927423+0000","last_undegraded":"2024-05-20T18:30:00.927423+0000","last_fullsized":"2024-05-20T18:30:00.927423+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:16:35.637090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.900808+0000","last_change":"2024-05-20T18:29:59.914716+0000","last_active":"2024-05-20T18:30:00.900808+0000","last_peered":"2024-05-20T18:30:00.900808+0000","last_clean":"2024-05-20T18:30:00.900808+0000","last_became_active":"2024-05-20T18:29:59.914370+0000","last_became_peered":"2024-05-20T18:29:59.914370+0000","last_unstale":"2024-05-20T18:30:00.900808+0000","last_undegraded":"2024-05-20T18:30:00.900808+0000","last_fullsized":"2024-05-20T18:30:00.900808+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:56:15.821122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.915264+0000","last_change":"2024-05-20T18:30:00.910719+0000","last_active":"2024-05-20T18:30:00.915264+0000","last_peered":"2024-05-20T18:30:00.915264+0000","last_clean":"2024-05-20T18:30:00.915264+0000","last_became_active":"2024-05-20T18:30:00.910404+0000","last_became_peered":"2024-05-20T18:30:00.910404+0000","last_unstale":"2024-05-20T18:30:00.915264+0000","last_undegraded":"2024-05-20T18:30:00.915264+0000","last_fullsized":"2024-05-20T18:30:00.915264+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:31:13.757884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.895574+0000","last_change":"2024-05-20T18:29:59.913801+0000","last_active":"2024-05-20T18:30:00.895574+0000","last_peered":"2024-05-20T18:30:00.895574+0000","last_clean":"2024-05-20T18:30:00.895574+0000","last_became_active":"2024-05-20T18:29:59.913436+0000","last_became_peered":"2024-05-20T18:29:59.913436+0000","last_unstale":"2024-05-20T18:30:00.895574+0000","last_undegraded":"2024-05-20T18:30:00.895574+0000","last_fullsized":"2024-05-20T18:30:00.895574+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:14:56.393143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.918133+0000","last_change":"2024-05-20T18:30:00.909584+0000","last_active":"2024-05-20T18:30:00.918133+0000","last_peered":"2024-05-20T18:30:00.918133+0000","last_clean":"2024-05-20T18:30:00.918133+0000","last_became_active":"2024-05-20T18:30:00.909243+0000","last_became_peered":"2024-05-20T18:30:00.909243+0000","last_unstale":"2024-05-20T18:30:00.918133+0000","last_undegraded":"2024-05-20T18:30:00.918133+0000","last_fullsized":"2024-05-20T18:30:00.918133+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:18:34.713042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.900408+0000","last_change":"2024-05-20T18:29:59.909797+0000","last_active":"2024-05-20T18:30:00.900408+0000","last_peered":"2024-05-20T18:30:00.900408+0000","last_clean":"2024-05-20T18:30:00.900408+0000","last_became_active":"2024-05-20T18:29:59.909506+0000","last_became_peered":"2024-05-20T18:29:59.909506+0000","last_unstale":"2024-05-20T18:30:00.900408+0000","last_undegraded":"2024-05-20T18:30:00.900408+0000","last_fullsized":"2024-05-20T18:30:00.900408+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:21:31.896478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":245,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.904181+0000","last_change":"2024-05-20T18:27:03.765694+0000","last_active":"2024-05-20T18:30:00.904181+0000","last_peered":"2024-05-20T18:30:00.904181+0000","last_clean":"2024-05-20T18:30:00.904181+0000","last_became_active":"2024-05-20T18:27:03.764695+0000","last_became_peered":"2024-05-20T18:27:03.764695+0000","last_unstale":"2024-05-20T18:30:00.904181+0000","last_undegraded":"2024-05-20T18:30:00.904181+0000","last_fullsized":"2024-05-20T18:30:00.904181+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_clean_scrub_stamp":"2024-05-20T18:27:02.750853+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:30:04.407487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1331200,"data_stored":919100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":753664,"data_stored":335933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":737280,"data_stored":328732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27740,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344100,"statfs":{"total":96636764160,"available":96608358400,"internally_reserved":0,"allocated":737280,"data_stored":328732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31916,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339924,"statfs":{"total":96636764160,"available":96604082176,"internally_reserved":0,"allocated":749568,"data_stored":335899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31900,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339940,"statfs":{"total":96636764160,"available":96604098560,"internally_reserved":0,"allocated":737280,"data_stored":328732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":778240,"data_stored":338275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1331200,"data_stored":919100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T18:30:04.978 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T18:30:05.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.129+0000 7f958bfea640 1 Processor -- start 2024-05-20T18:30:05.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.129+0000 7f958bfea640 1 -- start start 2024-05-20T18:30:05.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.131+0000 7f958bfea640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f958406d3d0 0x7f958406d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:05.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.131+0000 7f958bfea640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f95840b8190 0x7f95840b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:05.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.131+0000 7f958bfea640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f958406dd20 con 0x7f9584162820 2024-05-20T18:30:05.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.131+0000 7f958bfea640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f95840d0630 con 0x7f958406d3d0 2024-05-20T18:30:05.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.131+0000 7f958bfea640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f95840d39c0 con 0x7f95840b8190 2024-05-20T18:30:05.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.131+0000 7f9589585640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f958406d3d0 0x7f958406d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:05.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.131+0000 7f9589585640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f958406d3d0 0x7f958406d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53048/0 (socket says 172.21.15.5:53048) 2024-05-20T18:30:05.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.131+0000 7f9589585640 1 -- 172.21.15.5:0/2936747915 learned_addr learned my addr 172.21.15.5:0/2936747915 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:05.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.131+0000 7f958a587640 1 --2- 172.21.15.5:0/2936747915 >> v2:172.21.15.63:3300/0 conn(0x7f95840b8190 0x7f95840b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:05.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.131+0000 7f9589585640 1 -- 172.21.15.5:0/2936747915 >> v2:172.21.15.63:3300/0 conn(0x7f95840b8190 msgr2=0x7f95840b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:05.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.131+0000 7f9589585640 1 --2- 172.21.15.5:0/2936747915 >> v2:172.21.15.63:3300/0 conn(0x7f95840b8190 0x7f95840b85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:05.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.131+0000 7f9589585640 1 -- 172.21.15.5:0/2936747915 >> v1:172.21.15.63:6789/0 conn(0x7f9584162820 legacy=0x7f95840b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:05.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.131+0000 7f9589585640 1 -- 172.21.15.5:0/2936747915 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95840d3ca0 con 0x7f958406d3d0 2024-05-20T18:30:05.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.132+0000 7f9589585640 1 --2- 172.21.15.5:0/2936747915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f958406d3d0 0x7f958406d7e0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f958408e070 tx=0x7f957402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1091f8e57ccd412 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:05.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.132+0000 7f9588d84640 1 -- 172.21.15.5:0/2936747915 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9574037070 con 0x7f958406d3d0 2024-05-20T18:30:05.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.132+0000 7f9588d84640 1 -- 172.21.15.5:0/2936747915 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9574032440 con 0x7f958406d3d0 2024-05-20T18:30:05.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.132+0000 7f9588d84640 1 -- 172.21.15.5:0/2936747915 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9574032810 con 0x7f958406d3d0 2024-05-20T18:30:05.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.133+0000 7f958bfea640 1 -- 172.21.15.5:0/2936747915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f958406d3d0 msgr2=0x7f958406d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:05.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.133+0000 7f958bfea640 1 --2- 172.21.15.5:0/2936747915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f958406d3d0 0x7f958406d7e0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f958408e070 tx=0x7f957402a230 comp rx=0 tx=0).stop 2024-05-20T18:30:05.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.133+0000 7f958bfea640 1 -- 172.21.15.5:0/2936747915 shutdown_connections 2024-05-20T18:30:05.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.133+0000 7f958bfea640 1 --2- 172.21.15.5:0/2936747915 >> v2:172.21.15.63:3300/0 conn(0x7f95840b8190 0x7f95840b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:05.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.133+0000 7f958bfea640 1 --2- 172.21.15.5:0/2936747915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f958406d3d0 0x7f958406d7e0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:05.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.133+0000 7f958bfea640 1 -- 172.21.15.5:0/2936747915 >> 172.21.15.5:0/2936747915 conn(0x7f95840d4bd0 msgr2=0x7f95840d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:05.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.133+0000 7f958bfea640 1 -- 172.21.15.5:0/2936747915 shutdown_connections 2024-05-20T18:30:05.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.133+0000 7f958bfea640 1 -- 172.21.15.5:0/2936747915 wait complete. 2024-05-20T18:30:05.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.135+0000 7f958bfea640 1 Processor -- start 2024-05-20T18:30:05.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.135+0000 7f958bfea640 1 -- start start 2024-05-20T18:30:05.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.136+0000 7f958bfea640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f958406d3d0 0x7f95841f9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:05.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.136+0000 7f958bfea640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f95840b8190 0x7f9584153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:05.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.136+0000 7f958bfea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9584162820 0x7f95840ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:05.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.136+0000 7f958bfea640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f95840d0300 con 0x7f9584162820 2024-05-20T18:30:05.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.136+0000 7f958bfea640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f95840d4260 con 0x7f958406d3d0 2024-05-20T18:30:05.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.136+0000 7f958bfea640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f95840b4050 con 0x7f95840b8190 2024-05-20T18:30:05.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.136+0000 7f958a587640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9584162820 0x7f95840ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:05.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.136+0000 7f958a587640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9584162820 0x7f95840ca8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53056/0 (socket says 172.21.15.5:53056) 2024-05-20T18:30:05.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.136+0000 7f958a587640 1 -- 172.21.15.5:0/397471793 learned_addr learned my addr 172.21.15.5:0/397471793 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:05.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.136+0000 7f9589d86640 1 --2- 172.21.15.5:0/397471793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f958406d3d0 0x7f95841f9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:05.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.136+0000 7f9589585640 1 --2- 172.21.15.5:0/397471793 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f95840b8190 0x7f9584153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:05.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.136+0000 7f958a587640 1 -- 172.21.15.5:0/397471793 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f95840b8190 msgr2=0x7f9584153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:05.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.136+0000 7f958a587640 1 --2- 172.21.15.5:0/397471793 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f95840b8190 0x7f9584153d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:05.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.136+0000 7f958a587640 1 -- 172.21.15.5:0/397471793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f958406d3d0 msgr2=0x7f95841f9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:05.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.136+0000 7f958a587640 1 --2- 172.21.15.5:0/397471793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f958406d3d0 0x7f95841f9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:05.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.136+0000 7f958a587640 1 -- 172.21.15.5:0/397471793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95841595d0 con 0x7f9584162820 2024-05-20T18:30:05.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.136+0000 7f9589d86640 1 --2- 172.21.15.5:0/397471793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f958406d3d0 0x7f95841f9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:30:05.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.137+0000 7f958a587640 1 --2- 172.21.15.5:0/397471793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9584162820 0x7f95840ca8d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f957c00cc60 tx=0x7f957c00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:05.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.137+0000 7f957a7fc640 1 -- 172.21.15.5:0/397471793 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c003990 con 0x7f9584162820 2024-05-20T18:30:05.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.137+0000 7f957a7fc640 1 -- 172.21.15.5:0/397471793 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f957c003b30 con 0x7f9584162820 2024-05-20T18:30:05.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.137+0000 7f957a7fc640 1 -- 172.21.15.5:0/397471793 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c004020 con 0x7f9584162820 2024-05-20T18:30:05.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.137+0000 7f958bfea640 1 -- 172.21.15.5:0/397471793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9584158530 con 0x7f9584162820 2024-05-20T18:30:05.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.137+0000 7f958bfea640 1 -- 172.21.15.5:0/397471793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9584158810 con 0x7f9584162820 2024-05-20T18:30:05.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.138+0000 7f958bfea640 1 -- 172.21.15.5:0/397471793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95840b7560 con 0x7f9584162820 2024-05-20T18:30:05.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.139+0000 7f957a7fc640 1 -- 172.21.15.5:0/397471793 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f957c0041c0 con 0x7f9584162820 2024-05-20T18:30:05.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.139+0000 7f957a7fc640 1 --2- 172.21.15.5:0/397471793 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f956806be60 0x7f956806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:05.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.139+0000 7f9589d86640 1 --2- 172.21.15.5:0/397471793 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f956806be60 0x7f956806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:05.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.139+0000 7f957a7fc640 1 -- 172.21.15.5:0/397471793 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7f957c08b430 con 0x7f9584162820 2024-05-20T18:30:05.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.140+0000 7f9589d86640 1 --2- 172.21.15.5:0/397471793 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f956806be60 0x7f956806e370 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f958000a810 tx=0x7f9580008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:05.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.144+0000 7f957a7fc640 1 -- 172.21.15.5:0/397471793 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f95840b7560 con 0x7f9584162820 2024-05-20T18:30:05.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.326+0000 7f958bfea640 1 -- 172.21.15.5:0/397471793 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f95841582e0 con 0x7f956806be60 2024-05-20T18:30:05.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.333+0000 7f957a7fc640 1 -- 172.21.15.5:0/397471793 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7f95841582e0 con 0x7f956806be60 2024-05-20T18:30:05.335 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:30:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-20T18:30:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.336+0000 7f958bfea640 1 -- 172.21.15.5:0/397471793 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f956806be60 msgr2=0x7f956806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.336+0000 7f958bfea640 1 --2- 172.21.15.5:0/397471793 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f956806be60 0x7f956806e370 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f958000a810 tx=0x7f9580008040 comp rx=0 tx=0).stop 2024-05-20T18:30:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.336+0000 7f958bfea640 1 -- 172.21.15.5:0/397471793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9584162820 msgr2=0x7f95840ca8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.336+0000 7f958bfea640 1 --2- 172.21.15.5:0/397471793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9584162820 0x7f95840ca8d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f957c00cc60 tx=0x7f957c00a5c0 comp rx=0 tx=0).stop 2024-05-20T18:30:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.336+0000 7f958bfea640 1 -- 172.21.15.5:0/397471793 shutdown_connections 2024-05-20T18:30:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.336+0000 7f958bfea640 1 --2- 172.21.15.5:0/397471793 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f956806be60 0x7f956806e370 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.336+0000 7f958bfea640 1 --2- 172.21.15.5:0/397471793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9584162820 0x7f95840ca8d0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:05.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.336+0000 7f958bfea640 1 --2- 172.21.15.5:0/397471793 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f95840b8190 0x7f9584153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:05.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.336+0000 7f958bfea640 1 --2- 172.21.15.5:0/397471793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f958406d3d0 0x7f95841f9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:05.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.336+0000 7f958bfea640 1 -- 172.21.15.5:0/397471793 >> 172.21.15.5:0/397471793 conn(0x7f95840d4bd0 msgr2=0x7f958415d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:05.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.336+0000 7f958bfea640 1 -- 172.21.15.5:0/397471793 shutdown_connections 2024-05-20T18:30:05.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.336+0000 7f958bfea640 1 -- 172.21.15.5:0/397471793 wait complete. 2024-05-20T18:30:05.352 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-05-20T18:30:04.617630+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243972,"kb_used_data":6988,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730748,"statfs":{"total":773094113280,"available":772844285952,"internally_reserved":0,"allocated":7155712,"data_stored":3834503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.733530"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.907365+0000","last_change":"2024-05-20T18:30:00.903836+0000","last_active":"2024-05-20T18:30:00.907365+0000","last_peered":"2024-05-20T18:30:00.907365+0000","last_clean":"2024-05-20T18:30:00.907365+0000","last_became_active":"2024-05-20T18:30:00.903479+0000","last_became_peered":"2024-05-20T18:30:00.903479+0000","last_unstale":"2024-05-20T18:30:00.907365+0000","last_undegraded":"2024-05-20T18:30:00.907365+0000","last_fullsized":"2024-05-20T18:30:00.907365+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:43:57.279675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.918183+0000","last_change":"2024-05-20T18:30:00.914512+0000","last_active":"2024-05-20T18:30:00.918183+0000","last_peered":"2024-05-20T18:30:00.918183+0000","last_clean":"2024-05-20T18:30:00.918183+0000","last_became_active":"2024-05-20T18:30:00.914163+0000","last_became_peered":"2024-05-20T18:30:00.914163+0000","last_unstale":"2024-05-20T18:30:00.918183+0000","last_undegraded":"2024-05-20T18:30:00.918183+0000","last_fullsized":"2024-05-20T18:30:00.918183+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:16:21.083333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.928469+0000","last_change":"2024-05-20T18:30:00.924738+0000","last_active":"2024-05-20T18:30:00.928469+0000","last_peered":"2024-05-20T18:30:00.928469+0000","last_clean":"2024-05-20T18:30:00.928469+0000","last_became_active":"2024-05-20T18:30:00.924399+0000","last_became_peered":"2024-05-20T18:30:00.924399+0000","last_unstale":"2024-05-20T18:30:00.928469+0000","last_undegraded":"2024-05-20T18:30:00.928469+0000","last_fullsized":"2024-05-20T18:30:00.928469+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:36:17.825672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.925975+0000","last_change":"2024-05-20T18:30:00.922288+0000","last_active":"2024-05-20T18:30:00.925975+0000","last_peered":"2024-05-20T18:30:00.925975+0000","last_clean":"2024-05-20T18:30:00.925975+0000","last_became_active":"2024-05-20T18:30:00.920734+0000","last_became_peered":"2024-05-20T18:30:00.920734+0000","last_unstale":"2024-05-20T18:30:00.925975+0000","last_undegraded":"2024-05-20T18:30:00.925975+0000","last_fullsized":"2024-05-20T18:30:00.925975+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:18:34.713042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.935217+0000","last_change":"2024-05-20T18:30:00.933767+0000","last_active":"2024-05-20T18:30:00.935217+0000","last_peered":"2024-05-20T18:30:00.935217+0000","last_clean":"2024-05-20T18:30:00.935217+0000","last_became_active":"2024-05-20T18:30:00.933409+0000","last_became_peered":"2024-05-20T18:30:00.933409+0000","last_unstale":"2024-05-20T18:30:00.935217+0000","last_undegraded":"2024-05-20T18:30:00.935217+0000","last_fullsized":"2024-05-20T18:30:00.935217+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:28:10.700582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.922804+0000","last_change":"2024-05-20T18:30:00.920884+0000","last_active":"2024-05-20T18:30:00.922804+0000","last_peered":"2024-05-20T18:30:00.922804+0000","last_clean":"2024-05-20T18:30:00.922804+0000","last_became_active":"2024-05-20T18:30:00.919712+0000","last_became_peered":"2024-05-20T18:30:00.919712+0000","last_unstale":"2024-05-20T18:30:00.922804+0000","last_undegraded":"2024-05-20T18:30:00.922804+0000","last_fullsized":"2024-05-20T18:30:00.922804+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:08:43.620630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.924984+0000","last_change":"2024-05-20T18:30:00.922433+0000","last_active":"2024-05-20T18:30:00.924984+0000","last_peered":"2024-05-20T18:30:00.924984+0000","last_clean":"2024-05-20T18:30:00.924984+0000","last_became_active":"2024-05-20T18:30:00.922100+0000","last_became_peered":"2024-05-20T18:30:00.922100+0000","last_unstale":"2024-05-20T18:30:00.924984+0000","last_undegraded":"2024-05-20T18:30:00.924984+0000","last_fullsized":"2024-05-20T18:30:00.924984+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:03:27.000524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.925063+0000","last_change":"2024-05-20T18:30:00.919507+0000","last_active":"2024-05-20T18:30:00.925063+0000","last_peered":"2024-05-20T18:30:00.925063+0000","last_clean":"2024-05-20T18:30:00.925063+0000","last_became_active":"2024-05-20T18:30:00.919096+0000","last_became_peered":"2024-05-20T18:30:00.919096+0000","last_unstale":"2024-05-20T18:30:00.925063+0000","last_undegraded":"2024-05-20T18:30:00.925063+0000","last_fullsized":"2024-05-20T18:30:00.925063+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:06:04.693803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.937649+0000","last_change":"2024-05-20T18:30:00.936123+0000","last_active":"2024-05-20T18:30:00.937649+0000","last_peered":"2024-05-20T18:30:00.937649+0000","last_clean":"2024-05-20T18:30:00.937649+0000","last_became_active":"2024-05-20T18:30:00.935781+0000","last_became_peered":"2024-05-20T18:30:00.935781+0000","last_unstale":"2024-05-20T18:30:00.937649+0000","last_undegraded":"2024-05-20T18:30:00.937649+0000","last_fullsized":"2024-05-20T18:30:00.937649+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:51:26.939060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.934459+0000","last_change":"2024-05-20T18:30:00.931422+0000","last_active":"2024-05-20T18:30:00.934459+0000","last_peered":"2024-05-20T18:30:00.934459+0000","last_clean":"2024-05-20T18:30:00.934459+0000","last_became_active":"2024-05-20T18:30:00.931056+0000","last_became_peered":"2024-05-20T18:30:00.931056+0000","last_unstale":"2024-05-20T18:30:00.934459+0000","last_undegraded":"2024-05-20T18:30:00.934459+0000","last_fullsized":"2024-05-20T18:30:00.934459+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:16:53.943495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.926043+0000","last_change":"2024-05-20T18:30:00.923607+0000","last_active":"2024-05-20T18:30:00.926043+0000","last_peered":"2024-05-20T18:30:00.926043+0000","last_clean":"2024-05-20T18:30:00.926043+0000","last_became_active":"2024-05-20T18:30:00.923270+0000","last_became_peered":"2024-05-20T18:30:00.923270+0000","last_unstale":"2024-05-20T18:30:00.926043+0000","last_undegraded":"2024-05-20T18:30:00.926043+0000","last_fullsized":"2024-05-20T18:30:00.926043+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:30:20.294826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.918062+0000","last_change":"2024-05-20T18:30:00.917196+0000","last_active":"2024-05-20T18:30:00.918062+0000","last_peered":"2024-05-20T18:30:00.918062+0000","last_clean":"2024-05-20T18:30:00.918062+0000","last_became_active":"2024-05-20T18:30:00.916348+0000","last_became_peered":"2024-05-20T18:30:00.916348+0000","last_unstale":"2024-05-20T18:30:00.918062+0000","last_undegraded":"2024-05-20T18:30:00.918062+0000","last_fullsized":"2024-05-20T18:30:00.918062+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:36:17.825672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.924696+0000","last_change":"2024-05-20T18:30:00.922319+0000","last_active":"2024-05-20T18:30:00.924696+0000","last_peered":"2024-05-20T18:30:00.924696+0000","last_clean":"2024-05-20T18:30:00.924696+0000","last_became_active":"2024-05-20T18:30:00.921938+0000","last_became_peered":"2024-05-20T18:30:00.921938+0000","last_unstale":"2024-05-20T18:30:00.924696+0000","last_undegraded":"2024-05-20T18:30:00.924696+0000","last_fullsized":"2024-05-20T18:30:00.924696+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:51:26.939060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.928093+0000","last_change":"2024-05-20T18:30:00.923901+0000","last_active":"2024-05-20T18:30:00.928093+0000","last_peered":"2024-05-20T18:30:00.928093+0000","last_clean":"2024-05-20T18:30:00.928093+0000","last_became_active":"2024-05-20T18:30:00.923159+0000","last_became_peered":"2024-05-20T18:30:00.923159+0000","last_unstale":"2024-05-20T18:30:00.928093+0000","last_undegraded":"2024-05-20T18:30:00.928093+0000","last_fullsized":"2024-05-20T18:30:00.928093+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:09:55.422091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.924110+0000","last_change":"2024-05-20T18:30:00.920906+0000","last_active":"2024-05-20T18:30:00.924110+0000","last_peered":"2024-05-20T18:30:00.924110+0000","last_clean":"2024-05-20T18:30:00.924110+0000","last_became_active":"2024-05-20T18:30:00.920551+0000","last_became_peered":"2024-05-20T18:30:00.920551+0000","last_unstale":"2024-05-20T18:30:00.924110+0000","last_undegraded":"2024-05-20T18:30:00.924110+0000","last_fullsized":"2024-05-20T18:30:00.924110+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:44:30.446400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.915449+0000","last_change":"2024-05-20T18:30:00.911072+0000","last_active":"2024-05-20T18:30:00.915449+0000","last_peered":"2024-05-20T18:30:00.915449+0000","last_clean":"2024-05-20T18:30:00.915449+0000","last_became_active":"2024-05-20T18:30:00.910257+0000","last_became_peered":"2024-05-20T18:30:00.910257+0000","last_unstale":"2024-05-20T18:30:00.915449+0000","last_undegraded":"2024-05-20T18:30:00.915449+0000","last_fullsized":"2024-05-20T18:30:00.915449+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:01:41.109326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.925997+0000","last_change":"2024-05-20T18:30:00.923216+0000","last_active":"2024-05-20T18:30:00.925997+0000","last_peered":"2024-05-20T18:30:00.925997+0000","last_clean":"2024-05-20T18:30:00.925997+0000","last_became_active":"2024-05-20T18:30:00.922894+0000","last_became_peered":"2024-05-20T18:30:00.922894+0000","last_unstale":"2024-05-20T18:30:00.925997+0000","last_undegraded":"2024-05-20T18:30:00.925997+0000","last_fullsized":"2024-05-20T18:30:00.925997+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:16:27.973018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.925930+0000","last_change":"2024-05-20T18:30:00.922933+0000","last_active":"2024-05-20T18:30:00.925930+0000","last_peered":"2024-05-20T18:30:00.925930+0000","last_clean":"2024-05-20T18:30:00.925930+0000","last_became_active":"2024-05-20T18:30:00.922591+0000","last_became_peered":"2024-05-20T18:30:00.922591+0000","last_unstale":"2024-05-20T18:30:00.925930+0000","last_undegraded":"2024-05-20T18:30:00.925930+0000","last_fullsized":"2024-05-20T18:30:00.925930+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:47:18.701103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.913317+0000","last_change":"2024-05-20T18:30:00.908911+0000","last_active":"2024-05-20T18:30:00.913317+0000","last_peered":"2024-05-20T18:30:00.913317+0000","last_clean":"2024-05-20T18:30:00.913317+0000","last_became_active":"2024-05-20T18:30:00.908607+0000","last_became_peered":"2024-05-20T18:30:00.908607+0000","last_unstale":"2024-05-20T18:30:00.913317+0000","last_undegraded":"2024-05-20T18:30:00.913317+0000","last_fullsized":"2024-05-20T18:30:00.913317+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:41:07.371805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.920083+0000","last_change":"2024-05-20T18:30:00.918049+0000","last_active":"2024-05-20T18:30:00.920083+0000","last_peered":"2024-05-20T18:30:00.920083+0000","last_clean":"2024-05-20T18:30:00.920083+0000","last_became_active":"2024-05-20T18:30:00.917717+0000","last_became_peered":"2024-05-20T18:30:00.917717+0000","last_unstale":"2024-05-20T18:30:00.920083+0000","last_undegraded":"2024-05-20T18:30:00.920083+0000","last_fullsized":"2024-05-20T18:30:00.920083+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:47:02.512109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.908268+0000","last_change":"2024-05-20T18:30:00.906062+0000","last_active":"2024-05-20T18:30:00.908268+0000","last_peered":"2024-05-20T18:30:00.908268+0000","last_clean":"2024-05-20T18:30:00.908268+0000","last_became_active":"2024-05-20T18:30:00.905640+0000","last_became_peered":"2024-05-20T18:30:00.905640+0000","last_unstale":"2024-05-20T18:30:00.908268+0000","last_undegraded":"2024-05-20T18:30:00.908268+0000","last_fullsized":"2024-05-20T18:30:00.908268+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:49:04.539026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.925107+0000","last_change":"2024-05-20T18:30:00.923924+0000","last_active":"2024-05-20T18:30:00.925107+0000","last_peered":"2024-05-20T18:30:00.925107+0000","last_clean":"2024-05-20T18:30:00.925107+0000","last_became_active":"2024-05-20T18:30:00.923552+0000","last_became_peered":"2024-05-20T18:30:00.923552+0000","last_unstale":"2024-05-20T18:30:00.925107+0000","last_undegraded":"2024-05-20T18:30:00.925107+0000","last_fullsized":"2024-05-20T18:30:00.925107+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:25:04.796127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.926660+0000","last_change":"2024-05-20T18:30:00.925501+0000","last_active":"2024-05-20T18:30:00.926660+0000","last_peered":"2024-05-20T18:30:00.926660+0000","last_clean":"2024-05-20T18:30:00.926660+0000","last_became_active":"2024-05-20T18:30:00.925165+0000","last_became_peered":"2024-05-20T18:30:00.925165+0000","last_unstale":"2024-05-20T18:30:00.926660+0000","last_undegraded":"2024-05-20T18:30:00.926660+0000","last_fullsized":"2024-05-20T18:30:00.926660+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:26:58.050238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.924458+0000","last_change":"2024-05-20T18:30:00.922044+0000","last_active":"2024-05-20T18:30:00.924458+0000","last_peered":"2024-05-20T18:30:00.924458+0000","last_clean":"2024-05-20T18:30:00.924458+0000","last_became_active":"2024-05-20T18:30:00.921698+0000","last_became_peered":"2024-05-20T18:30:00.921698+0000","last_unstale":"2024-05-20T18:30:00.924458+0000","last_undegraded":"2024-05-20T18:30:00.924458+0000","last_fullsized":"2024-05-20T18:30:00.924458+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:50:48.116694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.911520+0000","last_change":"2024-05-20T18:30:00.908489+0000","last_active":"2024-05-20T18:30:00.911520+0000","last_peered":"2024-05-20T18:30:00.911520+0000","last_clean":"2024-05-20T18:30:00.911520+0000","last_became_active":"2024-05-20T18:30:00.907914+0000","last_became_peered":"2024-05-20T18:30:00.907914+0000","last_unstale":"2024-05-20T18:30:00.911520+0000","last_undegraded":"2024-05-20T18:30:00.911520+0000","last_fullsized":"2024-05-20T18:30:00.911520+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:07:33.080076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.910868+0000","last_change":"2024-05-20T18:30:00.908887+0000","last_active":"2024-05-20T18:30:00.910868+0000","last_peered":"2024-05-20T18:30:00.910868+0000","last_clean":"2024-05-20T18:30:00.910868+0000","last_became_active":"2024-05-20T18:30:00.908282+0000","last_became_peered":"2024-05-20T18:30:00.908282+0000","last_unstale":"2024-05-20T18:30:00.910868+0000","last_undegraded":"2024-05-20T18:30:00.910868+0000","last_fullsized":"2024-05-20T18:30:00.910868+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:44:30.446400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.926007+0000","last_change":"2024-05-20T18:30:00.921763+0000","last_active":"2024-05-20T18:30:00.926007+0000","last_peered":"2024-05-20T18:30:00.926007+0000","last_clean":"2024-05-20T18:30:00.926007+0000","last_became_active":"2024-05-20T18:30:00.921288+0000","last_became_peered":"2024-05-20T18:30:00.921288+0000","last_unstale":"2024-05-20T18:30:00.926007+0000","last_undegraded":"2024-05-20T18:30:00.926007+0000","last_fullsized":"2024-05-20T18:30:00.926007+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:41:07.371805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.920178+0000","last_change":"2024-05-20T18:30:00.917206+0000","last_active":"2024-05-20T18:30:00.920178+0000","last_peered":"2024-05-20T18:30:00.920178+0000","last_clean":"2024-05-20T18:30:00.920178+0000","last_became_active":"2024-05-20T18:30:00.916857+0000","last_became_peered":"2024-05-20T18:30:00.916857+0000","last_unstale":"2024-05-20T18:30:00.920178+0000","last_undegraded":"2024-05-20T18:30:00.920178+0000","last_fullsized":"2024-05-20T18:30:00.920178+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:05:04.426633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.938484+0000","last_change":"2024-05-20T18:30:00.936670+0000","last_active":"2024-05-20T18:30:00.938484+0000","last_peered":"2024-05-20T18:30:00.938484+0000","last_clean":"2024-05-20T18:30:00.938484+0000","last_became_active":"2024-05-20T18:30:00.936318+0000","last_became_peered":"2024-05-20T18:30:00.936318+0000","last_unstale":"2024-05-20T18:30:00.938484+0000","last_undegraded":"2024-05-20T18:30:00.938484+0000","last_fullsized":"2024-05-20T18:30:00.938484+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:28:01.431319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.919818+0000","last_change":"2024-05-20T18:30:00.918890+0000","last_active":"2024-05-20T18:30:00.919818+0000","last_peered":"2024-05-20T18:30:00.919818+0000","last_clean":"2024-05-20T18:30:00.919818+0000","last_became_active":"2024-05-20T18:30:00.918530+0000","last_became_peered":"2024-05-20T18:30:00.918530+0000","last_unstale":"2024-05-20T18:30:00.919818+0000","last_undegraded":"2024-05-20T18:30:00.919818+0000","last_fullsized":"2024-05-20T18:30:00.919818+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:16:53.943495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.924265+0000","last_change":"2024-05-20T18:30:00.915178+0000","last_active":"2024-05-20T18:30:00.924265+0000","last_peered":"2024-05-20T18:30:00.924265+0000","last_clean":"2024-05-20T18:30:00.924265+0000","last_became_active":"2024-05-20T18:30:00.914832+0000","last_became_peered":"2024-05-20T18:30:00.914832+0000","last_unstale":"2024-05-20T18:30:00.924265+0000","last_undegraded":"2024-05-20T18:30:00.924265+0000","last_fullsized":"2024-05-20T18:30:00.924265+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:26:09.155714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.916091+0000","last_change":"2024-05-20T18:30:00.912543+0000","last_active":"2024-05-20T18:30:00.916091+0000","last_peered":"2024-05-20T18:30:00.916091+0000","last_clean":"2024-05-20T18:30:00.916091+0000","last_became_active":"2024-05-20T18:30:00.912187+0000","last_became_peered":"2024-05-20T18:30:00.912187+0000","last_unstale":"2024-05-20T18:30:00.916091+0000","last_undegraded":"2024-05-20T18:30:00.916091+0000","last_fullsized":"2024-05-20T18:30:00.916091+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:35:15.347397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.923922+0000","last_change":"2024-05-20T18:30:00.917405+0000","last_active":"2024-05-20T18:30:00.923922+0000","last_peered":"2024-05-20T18:30:00.923922+0000","last_clean":"2024-05-20T18:30:00.923922+0000","last_became_active":"2024-05-20T18:30:00.916938+0000","last_became_peered":"2024-05-20T18:30:00.916938+0000","last_unstale":"2024-05-20T18:30:00.923922+0000","last_undegraded":"2024-05-20T18:30:00.923922+0000","last_fullsized":"2024-05-20T18:30:00.923922+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:14:57.402663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"62'4","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.935502+0000","last_change":"2024-05-20T18:29:59.912962+0000","last_active":"2024-05-20T18:30:00.935502+0000","last_peered":"2024-05-20T18:30:00.935502+0000","last_clean":"2024-05-20T18:30:00.935502+0000","last_became_active":"2024-05-20T18:29:59.912641+0000","last_became_peered":"2024-05-20T18:29:59.912641+0000","last_unstale":"2024-05-20T18:30:00.935502+0000","last_undegraded":"2024-05-20T18:30:00.935502+0000","last_fullsized":"2024-05-20T18:30:00.935502+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:37:37.860286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.930085+0000","last_change":"2024-05-20T18:30:00.928426+0000","last_active":"2024-05-20T18:30:00.930085+0000","last_peered":"2024-05-20T18:30:00.930085+0000","last_clean":"2024-05-20T18:30:00.930085+0000","last_became_active":"2024-05-20T18:30:00.928075+0000","last_became_peered":"2024-05-20T18:30:00.928075+0000","last_unstale":"2024-05-20T18:30:00.930085+0000","last_undegraded":"2024-05-20T18:30:00.930085+0000","last_fullsized":"2024-05-20T18:30:00.930085+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:56:29.475238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.931890+0000","last_change":"2024-05-20T18:29:59.909960+0000","last_active":"2024-05-20T18:30:00.931890+0000","last_peered":"2024-05-20T18:30:00.931890+0000","last_clean":"2024-05-20T18:30:00.931890+0000","last_became_active":"2024-05-20T18:29:59.909629+0000","last_became_peered":"2024-05-20T18:29:59.909629+0000","last_unstale":"2024-05-20T18:30:00.931890+0000","last_undegraded":"2024-05-20T18:30:00.931890+0000","last_fullsized":"2024-05-20T18:30:00.931890+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:28:20.832622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.916129+0000","last_change":"2024-05-20T18:30:00.913579+0000","last_active":"2024-05-20T18:30:00.916129+0000","last_peered":"2024-05-20T18:30:00.916129+0000","last_clean":"2024-05-20T18:30:00.916129+0000","last_became_active":"2024-05-20T18:30:00.913193+0000","last_became_peered":"2024-05-20T18:30:00.913193+0000","last_unstale":"2024-05-20T18:30:00.916129+0000","last_undegraded":"2024-05-20T18:30:00.916129+0000","last_fullsized":"2024-05-20T18:30:00.916129+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:13:19.360804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.929866+0000","last_change":"2024-05-20T18:29:59.915264+0000","last_active":"2024-05-20T18:30:00.929866+0000","last_peered":"2024-05-20T18:30:00.929866+0000","last_clean":"2024-05-20T18:30:00.929866+0000","last_became_active":"2024-05-20T18:29:59.914809+0000","last_became_peered":"2024-05-20T18:29:59.914809+0000","last_unstale":"2024-05-20T18:30:00.929866+0000","last_undegraded":"2024-05-20T18:30:00.929866+0000","last_fullsized":"2024-05-20T18:30:00.929866+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:51:25.929539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.935305+0000","last_change":"2024-05-20T18:30:00.932564+0000","last_active":"2024-05-20T18:30:00.935305+0000","last_peered":"2024-05-20T18:30:00.935305+0000","last_clean":"2024-05-20T18:30:00.935305+0000","last_became_active":"2024-05-20T18:30:00.931947+0000","last_became_peered":"2024-05-20T18:30:00.931947+0000","last_unstale":"2024-05-20T18:30:00.935305+0000","last_undegraded":"2024-05-20T18:30:00.935305+0000","last_fullsized":"2024-05-20T18:30:00.935305+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:06:04.693803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.929128+0000","last_change":"2024-05-20T18:29:59.908215+0000","last_active":"2024-05-20T18:30:00.929128+0000","last_peered":"2024-05-20T18:30:00.929128+0000","last_clean":"2024-05-20T18:30:00.929128+0000","last_became_active":"2024-05-20T18:29:59.907812+0000","last_became_peered":"2024-05-20T18:29:59.907812+0000","last_unstale":"2024-05-20T18:30:00.929128+0000","last_undegraded":"2024-05-20T18:30:00.929128+0000","last_fullsized":"2024-05-20T18:30:00.929128+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:08:44.746640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.915426+0000","last_change":"2024-05-20T18:30:00.908783+0000","last_active":"2024-05-20T18:30:00.915426+0000","last_peered":"2024-05-20T18:30:00.915426+0000","last_clean":"2024-05-20T18:30:00.915426+0000","last_became_active":"2024-05-20T18:30:00.908345+0000","last_became_peered":"2024-05-20T18:30:00.908345+0000","last_unstale":"2024-05-20T18:30:00.915426+0000","last_undegraded":"2024-05-20T18:30:00.915426+0000","last_fullsized":"2024-05-20T18:30:00.915426+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:39:10.658137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.925846+0000","last_change":"2024-05-20T18:29:59.910015+0000","last_active":"2024-05-20T18:30:00.925846+0000","last_peered":"2024-05-20T18:30:00.925846+0000","last_clean":"2024-05-20T18:30:00.925846+0000","last_became_active":"2024-05-20T18:29:59.909607+0000","last_became_peered":"2024-05-20T18:29:59.909607+0000","last_unstale":"2024-05-20T18:30:00.925846+0000","last_undegraded":"2024-05-20T18:30:00.925846+0000","last_fullsized":"2024-05-20T18:30:00.925846+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:51:48.737011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.925689+0000","last_change":"2024-05-20T18:30:00.924033+0000","last_active":"2024-05-20T18:30:00.925689+0000","last_peered":"2024-05-20T18:30:00.925689+0000","last_clean":"2024-05-20T18:30:00.925689+0000","last_became_active":"2024-05-20T18:30:00.922154+0000","last_became_peered":"2024-05-20T18:30:00.922154+0000","last_unstale":"2024-05-20T18:30:00.925689+0000","last_undegraded":"2024-05-20T18:30:00.925689+0000","last_fullsized":"2024-05-20T18:30:00.925689+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:27:40.715088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.924184+0000","last_change":"2024-05-20T18:29:59.906090+0000","last_active":"2024-05-20T18:30:00.924184+0000","last_peered":"2024-05-20T18:30:00.924184+0000","last_clean":"2024-05-20T18:30:00.924184+0000","last_became_active":"2024-05-20T18:29:59.905762+0000","last_became_peered":"2024-05-20T18:29:59.905762+0000","last_unstale":"2024-05-20T18:30:00.924184+0000","last_undegraded":"2024-05-20T18:30:00.924184+0000","last_fullsized":"2024-05-20T18:30:00.924184+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:37:11.166078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.912188+0000","last_change":"2024-05-20T18:30:00.910857+0000","last_active":"2024-05-20T18:30:00.912188+0000","last_peered":"2024-05-20T18:30:00.912188+0000","last_clean":"2024-05-20T18:30:00.912188+0000","last_became_active":"2024-05-20T18:30:00.909676+0000","last_became_peered":"2024-05-20T18:30:00.909676+0000","last_unstale":"2024-05-20T18:30:00.912188+0000","last_undegraded":"2024-05-20T18:30:00.912188+0000","last_fullsized":"2024-05-20T18:30:00.912188+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:47:18.701103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.900624+0000","last_change":"2024-05-20T18:29:59.914083+0000","last_active":"2024-05-20T18:30:00.900624+0000","last_peered":"2024-05-20T18:30:00.900624+0000","last_clean":"2024-05-20T18:30:00.900624+0000","last_became_active":"2024-05-20T18:29:59.913726+0000","last_became_peered":"2024-05-20T18:29:59.913726+0000","last_unstale":"2024-05-20T18:30:00.900624+0000","last_undegraded":"2024-05-20T18:30:00.900624+0000","last_fullsized":"2024-05-20T18:30:00.900624+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:09:28.306514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.920112+0000","last_change":"2024-05-20T18:30:00.918437+0000","last_active":"2024-05-20T18:30:00.920112+0000","last_peered":"2024-05-20T18:30:00.920112+0000","last_clean":"2024-05-20T18:30:00.920112+0000","last_became_active":"2024-05-20T18:30:00.918110+0000","last_became_peered":"2024-05-20T18:30:00.918110+0000","last_unstale":"2024-05-20T18:30:00.920112+0000","last_undegraded":"2024-05-20T18:30:00.920112+0000","last_fullsized":"2024-05-20T18:30:00.920112+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:10:47.393081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.898571+0000","last_change":"2024-05-20T18:29:59.909196+0000","last_active":"2024-05-20T18:30:00.898571+0000","last_peered":"2024-05-20T18:30:00.898571+0000","last_clean":"2024-05-20T18:30:00.898571+0000","last_became_active":"2024-05-20T18:29:59.908843+0000","last_became_peered":"2024-05-20T18:29:59.908843+0000","last_unstale":"2024-05-20T18:30:00.898571+0000","last_undegraded":"2024-05-20T18:30:00.898571+0000","last_fullsized":"2024-05-20T18:30:00.898571+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:21:46.253714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.911291+0000","last_change":"2024-05-20T18:30:00.908719+0000","last_active":"2024-05-20T18:30:00.911291+0000","last_peered":"2024-05-20T18:30:00.911291+0000","last_clean":"2024-05-20T18:30:00.911291+0000","last_became_active":"2024-05-20T18:30:00.907726+0000","last_became_peered":"2024-05-20T18:30:00.907726+0000","last_unstale":"2024-05-20T18:30:00.911291+0000","last_undegraded":"2024-05-20T18:30:00.911291+0000","last_fullsized":"2024-05-20T18:30:00.911291+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:43:27.761088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.918429+0000","last_change":"2024-05-20T18:29:59.917401+0000","last_active":"2024-05-20T18:30:00.918429+0000","last_peered":"2024-05-20T18:30:00.918429+0000","last_clean":"2024-05-20T18:30:00.918429+0000","last_became_active":"2024-05-20T18:29:59.916950+0000","last_became_peered":"2024-05-20T18:29:59.916950+0000","last_unstale":"2024-05-20T18:30:00.918429+0000","last_undegraded":"2024-05-20T18:30:00.918429+0000","last_fullsized":"2024-05-20T18:30:00.918429+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:07:10.468378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.907376+0000","last_change":"2024-05-20T18:30:00.903962+0000","last_active":"2024-05-20T18:30:00.907376+0000","last_peered":"2024-05-20T18:30:00.907376+0000","last_clean":"2024-05-20T18:30:00.907376+0000","last_became_active":"2024-05-20T18:30:00.903610+0000","last_became_peered":"2024-05-20T18:30:00.903610+0000","last_unstale":"2024-05-20T18:30:00.907376+0000","last_undegraded":"2024-05-20T18:30:00.907376+0000","last_fullsized":"2024-05-20T18:30:00.907376+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:04:16.563394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.908162+0000","last_change":"2024-05-20T18:29:59.908355+0000","last_active":"2024-05-20T18:30:00.908162+0000","last_peered":"2024-05-20T18:30:00.908162+0000","last_clean":"2024-05-20T18:30:00.908162+0000","last_became_active":"2024-05-20T18:29:59.907685+0000","last_became_peered":"2024-05-20T18:29:59.907685+0000","last_unstale":"2024-05-20T18:30:00.908162+0000","last_undegraded":"2024-05-20T18:30:00.908162+0000","last_fullsized":"2024-05-20T18:30:00.908162+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:21:46.253714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.911529+0000","last_change":"2024-05-20T18:30:00.907509+0000","last_active":"2024-05-20T18:30:00.911529+0000","last_peered":"2024-05-20T18:30:00.911529+0000","last_clean":"2024-05-20T18:30:00.911529+0000","last_became_active":"2024-05-20T18:30:00.905850+0000","last_became_peered":"2024-05-20T18:30:00.905850+0000","last_unstale":"2024-05-20T18:30:00.911529+0000","last_undegraded":"2024-05-20T18:30:00.911529+0000","last_fullsized":"2024-05-20T18:30:00.911529+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:56:05.710762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"62'3","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.932562+0000","last_change":"2024-05-20T18:29:59.917391+0000","last_active":"2024-05-20T18:30:00.932562+0000","last_peered":"2024-05-20T18:30:00.932562+0000","last_clean":"2024-05-20T18:30:00.932562+0000","last_became_active":"2024-05-20T18:29:59.916996+0000","last_became_peered":"2024-05-20T18:29:59.916996+0000","last_unstale":"2024-05-20T18:30:00.932562+0000","last_undegraded":"2024-05-20T18:30:00.932562+0000","last_fullsized":"2024-05-20T18:30:00.932562+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:49:03.529505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.924297+0000","last_change":"2024-05-20T18:30:00.918564+0000","last_active":"2024-05-20T18:30:00.924297+0000","last_peered":"2024-05-20T18:30:00.924297+0000","last_clean":"2024-05-20T18:30:00.924297+0000","last_became_active":"2024-05-20T18:30:00.915835+0000","last_became_peered":"2024-05-20T18:30:00.915835+0000","last_unstale":"2024-05-20T18:30:00.924297+0000","last_undegraded":"2024-05-20T18:30:00.924297+0000","last_fullsized":"2024-05-20T18:30:00.924297+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:45:08.997389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"62'6","reported_seq":18,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.938905+0000","last_change":"2024-05-20T18:29:59.912482+0000","last_active":"2024-05-20T18:30:00.938905+0000","last_peered":"2024-05-20T18:30:00.938905+0000","last_clean":"2024-05-20T18:30:00.938905+0000","last_became_active":"2024-05-20T18:29:59.911897+0000","last_became_peered":"2024-05-20T18:29:59.911897+0000","last_unstale":"2024-05-20T18:30:00.938905+0000","last_undegraded":"2024-05-20T18:30:00.938905+0000","last_fullsized":"2024-05-20T18:30:00.938905+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:28:09.691061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.936467+0000","last_change":"2024-05-20T18:30:00.933298+0000","last_active":"2024-05-20T18:30:00.936467+0000","last_peered":"2024-05-20T18:30:00.936467+0000","last_clean":"2024-05-20T18:30:00.936467+0000","last_became_active":"2024-05-20T18:30:00.932894+0000","last_became_peered":"2024-05-20T18:30:00.932894+0000","last_unstale":"2024-05-20T18:30:00.936467+0000","last_undegraded":"2024-05-20T18:30:00.936467+0000","last_fullsized":"2024-05-20T18:30:00.936467+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:37:38.869806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.935648+0000","last_change":"2024-05-20T18:29:59.917535+0000","last_active":"2024-05-20T18:30:00.935648+0000","last_peered":"2024-05-20T18:30:00.935648+0000","last_clean":"2024-05-20T18:30:00.935648+0000","last_became_active":"2024-05-20T18:29:59.917132+0000","last_became_peered":"2024-05-20T18:29:59.917132+0000","last_unstale":"2024-05-20T18:30:00.935648+0000","last_undegraded":"2024-05-20T18:30:00.935648+0000","last_fullsized":"2024-05-20T18:30:00.935648+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:12:42.531877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.917489+0000","last_change":"2024-05-20T18:30:00.915439+0000","last_active":"2024-05-20T18:30:00.917489+0000","last_peered":"2024-05-20T18:30:00.917489+0000","last_clean":"2024-05-20T18:30:00.917489+0000","last_became_active":"2024-05-20T18:30:00.915101+0000","last_became_peered":"2024-05-20T18:30:00.915101+0000","last_unstale":"2024-05-20T18:30:00.917489+0000","last_undegraded":"2024-05-20T18:30:00.917489+0000","last_fullsized":"2024-05-20T18:30:00.917489+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:09:55.422091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.898687+0000","last_change":"2024-05-20T18:29:59.914563+0000","last_active":"2024-05-20T18:30:00.898687+0000","last_peered":"2024-05-20T18:30:00.898687+0000","last_clean":"2024-05-20T18:30:00.898687+0000","last_became_active":"2024-05-20T18:29:59.914241+0000","last_became_peered":"2024-05-20T18:29:59.914241+0000","last_unstale":"2024-05-20T18:30:00.898687+0000","last_undegraded":"2024-05-20T18:30:00.898687+0000","last_fullsized":"2024-05-20T18:30:00.898687+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:48:26.490874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.911805+0000","last_change":"2024-05-20T18:30:00.908147+0000","last_active":"2024-05-20T18:30:00.911805+0000","last_peered":"2024-05-20T18:30:00.911805+0000","last_clean":"2024-05-20T18:30:00.911805+0000","last_became_active":"2024-05-20T18:30:00.907844+0000","last_became_peered":"2024-05-20T18:30:00.907844+0000","last_unstale":"2024-05-20T18:30:00.911805+0000","last_undegraded":"2024-05-20T18:30:00.911805+0000","last_fullsized":"2024-05-20T18:30:00.911805+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:26:09.155714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.910964+0000","last_change":"2024-05-20T18:29:59.916588+0000","last_active":"2024-05-20T18:30:00.910964+0000","last_peered":"2024-05-20T18:30:00.910964+0000","last_clean":"2024-05-20T18:30:00.910964+0000","last_became_active":"2024-05-20T18:29:59.916303+0000","last_became_peered":"2024-05-20T18:29:59.916303+0000","last_unstale":"2024-05-20T18:30:00.910964+0000","last_undegraded":"2024-05-20T18:30:00.910964+0000","last_fullsized":"2024-05-20T18:30:00.910964+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:48:26.490874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.928129+0000","last_change":"2024-05-20T18:30:00.927056+0000","last_active":"2024-05-20T18:30:00.928129+0000","last_peered":"2024-05-20T18:30:00.928129+0000","last_clean":"2024-05-20T18:30:00.928129+0000","last_became_active":"2024-05-20T18:30:00.926692+0000","last_became_peered":"2024-05-20T18:30:00.926692+0000","last_unstale":"2024-05-20T18:30:00.928129+0000","last_undegraded":"2024-05-20T18:30:00.928129+0000","last_fullsized":"2024-05-20T18:30:00.928129+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:48:27.500395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"62'8","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.938975+0000","last_change":"2024-05-20T18:29:59.913397+0000","last_active":"2024-05-20T18:30:00.938975+0000","last_peered":"2024-05-20T18:30:00.938975+0000","last_clean":"2024-05-20T18:30:00.938975+0000","last_became_active":"2024-05-20T18:29:59.913024+0000","last_became_peered":"2024-05-20T18:29:59.913024+0000","last_unstale":"2024-05-20T18:30:00.938975+0000","last_undegraded":"2024-05-20T18:30:00.938975+0000","last_fullsized":"2024-05-20T18:30:00.938975+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:56:04.701242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.938576+0000","last_change":"2024-05-20T18:30:00.936700+0000","last_active":"2024-05-20T18:30:00.938576+0000","last_peered":"2024-05-20T18:30:00.938576+0000","last_clean":"2024-05-20T18:30:00.938576+0000","last_became_active":"2024-05-20T18:30:00.936371+0000","last_became_peered":"2024-05-20T18:30:00.936371+0000","last_unstale":"2024-05-20T18:30:00.938576+0000","last_undegraded":"2024-05-20T18:30:00.938576+0000","last_fullsized":"2024-05-20T18:30:00.938576+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:54:39.522862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.930100+0000","last_change":"2024-05-20T18:29:59.913822+0000","last_active":"2024-05-20T18:30:00.930100+0000","last_peered":"2024-05-20T18:30:00.930100+0000","last_clean":"2024-05-20T18:30:00.930100+0000","last_became_active":"2024-05-20T18:29:59.913482+0000","last_became_peered":"2024-05-20T18:29:59.913482+0000","last_unstale":"2024-05-20T18:30:00.930100+0000","last_undegraded":"2024-05-20T18:30:00.930100+0000","last_fullsized":"2024-05-20T18:30:00.930100+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:51:25.023616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.925912+0000","last_change":"2024-05-20T18:30:00.922460+0000","last_active":"2024-05-20T18:30:00.925912+0000","last_peered":"2024-05-20T18:30:00.925912+0000","last_clean":"2024-05-20T18:30:00.925912+0000","last_became_active":"2024-05-20T18:30:00.922101+0000","last_became_peered":"2024-05-20T18:30:00.922101+0000","last_unstale":"2024-05-20T18:30:00.925912+0000","last_undegraded":"2024-05-20T18:30:00.925912+0000","last_fullsized":"2024-05-20T18:30:00.925912+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:37:38.869806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.938493+0000","last_change":"2024-05-20T18:29:59.917144+0000","last_active":"2024-05-20T18:30:00.938493+0000","last_peered":"2024-05-20T18:30:00.938493+0000","last_clean":"2024-05-20T18:30:00.938493+0000","last_became_active":"2024-05-20T18:29:59.916729+0000","last_became_peered":"2024-05-20T18:29:59.916729+0000","last_unstale":"2024-05-20T18:30:00.938493+0000","last_undegraded":"2024-05-20T18:30:00.938493+0000","last_fullsized":"2024-05-20T18:30:00.938493+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:51:25.023616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.930191+0000","last_change":"2024-05-20T18:30:00.927816+0000","last_active":"2024-05-20T18:30:00.930191+0000","last_peered":"2024-05-20T18:30:00.930191+0000","last_clean":"2024-05-20T18:30:00.930191+0000","last_became_active":"2024-05-20T18:30:00.926661+0000","last_became_peered":"2024-05-20T18:30:00.926661+0000","last_unstale":"2024-05-20T18:30:00.930191+0000","last_undegraded":"2024-05-20T18:30:00.930191+0000","last_fullsized":"2024-05-20T18:30:00.930191+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:36:24.995925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.934337+0000","last_change":"2024-05-20T18:29:59.917688+0000","last_active":"2024-05-20T18:30:00.934337+0000","last_peered":"2024-05-20T18:30:00.934337+0000","last_clean":"2024-05-20T18:30:00.934337+0000","last_became_active":"2024-05-20T18:29:59.916998+0000","last_became_peered":"2024-05-20T18:29:59.916998+0000","last_unstale":"2024-05-20T18:30:00.934337+0000","last_undegraded":"2024-05-20T18:30:00.934337+0000","last_fullsized":"2024-05-20T18:30:00.934337+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:47:38.443964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.918861+0000","last_change":"2024-05-20T18:30:00.915834+0000","last_active":"2024-05-20T18:30:00.918861+0000","last_peered":"2024-05-20T18:30:00.918861+0000","last_clean":"2024-05-20T18:30:00.918861+0000","last_became_active":"2024-05-20T18:30:00.915411+0000","last_became_peered":"2024-05-20T18:30:00.915411+0000","last_unstale":"2024-05-20T18:30:00.918861+0000","last_undegraded":"2024-05-20T18:30:00.918861+0000","last_fullsized":"2024-05-20T18:30:00.918861+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:36:24.995925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.900281+0000","last_change":"2024-05-20T18:29:59.912876+0000","last_active":"2024-05-20T18:30:00.900281+0000","last_peered":"2024-05-20T18:30:00.900281+0000","last_clean":"2024-05-20T18:30:00.900281+0000","last_became_active":"2024-05-20T18:29:59.912423+0000","last_became_peered":"2024-05-20T18:29:59.912423+0000","last_unstale":"2024-05-20T18:30:00.900281+0000","last_undegraded":"2024-05-20T18:30:00.900281+0000","last_fullsized":"2024-05-20T18:30:00.900281+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:44:20.302541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.910090+0000","last_change":"2024-05-20T18:30:00.908373+0000","last_active":"2024-05-20T18:30:00.910090+0000","last_peered":"2024-05-20T18:30:00.910090+0000","last_clean":"2024-05-20T18:30:00.910090+0000","last_became_active":"2024-05-20T18:30:00.907994+0000","last_became_peered":"2024-05-20T18:30:00.907994+0000","last_unstale":"2024-05-20T18:30:00.910090+0000","last_undegraded":"2024-05-20T18:30:00.910090+0000","last_fullsized":"2024-05-20T18:30:00.910090+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:01:04.634825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.927827+0000","last_change":"2024-05-20T18:29:59.913869+0000","last_active":"2024-05-20T18:30:00.927827+0000","last_peered":"2024-05-20T18:30:00.927827+0000","last_clean":"2024-05-20T18:30:00.927827+0000","last_became_active":"2024-05-20T18:29:59.913512+0000","last_became_peered":"2024-05-20T18:29:59.913512+0000","last_unstale":"2024-05-20T18:30:00.927827+0000","last_undegraded":"2024-05-20T18:30:00.927827+0000","last_fullsized":"2024-05-20T18:30:00.927827+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:06:03.684282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.910068+0000","last_change":"2024-05-20T18:30:00.907372+0000","last_active":"2024-05-20T18:30:00.910068+0000","last_peered":"2024-05-20T18:30:00.910068+0000","last_clean":"2024-05-20T18:30:00.910068+0000","last_became_active":"2024-05-20T18:30:00.907051+0000","last_became_peered":"2024-05-20T18:30:00.907051+0000","last_unstale":"2024-05-20T18:30:00.910068+0000","last_undegraded":"2024-05-20T18:30:00.910068+0000","last_fullsized":"2024-05-20T18:30:00.910068+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:12:43.541398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.898593+0000","last_change":"2024-05-20T18:29:59.912694+0000","last_active":"2024-05-20T18:30:00.898593+0000","last_peered":"2024-05-20T18:30:00.898593+0000","last_clean":"2024-05-20T18:30:00.898593+0000","last_became_active":"2024-05-20T18:29:59.912270+0000","last_became_peered":"2024-05-20T18:29:59.912270+0000","last_unstale":"2024-05-20T18:30:00.898593+0000","last_undegraded":"2024-05-20T18:30:00.898593+0000","last_fullsized":"2024-05-20T18:30:00.898593+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:16:34.627569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.914589+0000","last_change":"2024-05-20T18:30:00.911187+0000","last_active":"2024-05-20T18:30:00.914589+0000","last_peered":"2024-05-20T18:30:00.914589+0000","last_clean":"2024-05-20T18:30:00.914589+0000","last_became_active":"2024-05-20T18:30:00.910336+0000","last_became_peered":"2024-05-20T18:30:00.910336+0000","last_unstale":"2024-05-20T18:30:00.914589+0000","last_undegraded":"2024-05-20T18:30:00.914589+0000","last_fullsized":"2024-05-20T18:30:00.914589+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:30:20.294826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.901900+0000","last_change":"2024-05-20T18:29:59.909115+0000","last_active":"2024-05-20T18:30:00.901900+0000","last_peered":"2024-05-20T18:30:00.901900+0000","last_clean":"2024-05-20T18:30:00.901900+0000","last_became_active":"2024-05-20T18:29:59.908764+0000","last_became_peered":"2024-05-20T18:29:59.908764+0000","last_unstale":"2024-05-20T18:30:00.901900+0000","last_undegraded":"2024-05-20T18:30:00.901900+0000","last_fullsized":"2024-05-20T18:30:00.901900+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:54:38.513342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.927281+0000","last_change":"2024-05-20T18:30:00.925040+0000","last_active":"2024-05-20T18:30:00.927281+0000","last_peered":"2024-05-20T18:30:00.927281+0000","last_clean":"2024-05-20T18:30:00.927281+0000","last_became_active":"2024-05-20T18:30:00.924714+0000","last_became_peered":"2024-05-20T18:30:00.924714+0000","last_unstale":"2024-05-20T18:30:00.927281+0000","last_undegraded":"2024-05-20T18:30:00.927281+0000","last_fullsized":"2024-05-20T18:30:00.927281+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:33:26.163997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.934910+0000","last_change":"2024-05-20T18:29:59.913604+0000","last_active":"2024-05-20T18:30:00.934910+0000","last_peered":"2024-05-20T18:30:00.934910+0000","last_clean":"2024-05-20T18:30:00.934910+0000","last_became_active":"2024-05-20T18:29:59.913292+0000","last_became_peered":"2024-05-20T18:29:59.913292+0000","last_unstale":"2024-05-20T18:30:00.934910+0000","last_undegraded":"2024-05-20T18:30:00.934910+0000","last_fullsized":"2024-05-20T18:30:00.934910+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:07:10.468378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.915190+0000","last_change":"2024-05-20T18:30:00.910103+0000","last_active":"2024-05-20T18:30:00.915190+0000","last_peered":"2024-05-20T18:30:00.915190+0000","last_clean":"2024-05-20T18:30:00.915190+0000","last_became_active":"2024-05-20T18:30:00.909685+0000","last_became_peered":"2024-05-20T18:30:00.909685+0000","last_unstale":"2024-05-20T18:30:00.915190+0000","last_undegraded":"2024-05-20T18:30:00.915190+0000","last_fullsized":"2024-05-20T18:30:00.915190+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:14:36.991839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.899625+0000","last_change":"2024-05-20T18:29:59.914038+0000","last_active":"2024-05-20T18:30:00.899625+0000","last_peered":"2024-05-20T18:30:00.899625+0000","last_clean":"2024-05-20T18:30:00.899625+0000","last_became_active":"2024-05-20T18:29:59.913670+0000","last_became_peered":"2024-05-20T18:29:59.913670+0000","last_unstale":"2024-05-20T18:30:00.899625+0000","last_undegraded":"2024-05-20T18:30:00.899625+0000","last_fullsized":"2024-05-20T18:30:00.899625+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:47:38.443964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.916316+0000","last_change":"2024-05-20T18:30:00.914756+0000","last_active":"2024-05-20T18:30:00.916316+0000","last_peered":"2024-05-20T18:30:00.916316+0000","last_clean":"2024-05-20T18:30:00.916316+0000","last_became_active":"2024-05-20T18:30:00.914386+0000","last_became_peered":"2024-05-20T18:30:00.914386+0000","last_unstale":"2024-05-20T18:30:00.916316+0000","last_undegraded":"2024-05-20T18:30:00.916316+0000","last_fullsized":"2024-05-20T18:30:00.916316+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:50:48.116694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.901325+0000","last_change":"2024-05-20T18:29:59.915114+0000","last_active":"2024-05-20T18:30:00.901325+0000","last_peered":"2024-05-20T18:30:00.901325+0000","last_clean":"2024-05-20T18:30:00.901325+0000","last_became_active":"2024-05-20T18:29:59.914755+0000","last_became_peered":"2024-05-20T18:29:59.914755+0000","last_unstale":"2024-05-20T18:30:00.901325+0000","last_undegraded":"2024-05-20T18:30:00.901325+0000","last_fullsized":"2024-05-20T18:30:00.901325+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:28:00.421798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.911584+0000","last_change":"2024-05-20T18:30:00.908378+0000","last_active":"2024-05-20T18:30:00.911584+0000","last_peered":"2024-05-20T18:30:00.911584+0000","last_clean":"2024-05-20T18:30:00.911584+0000","last_became_active":"2024-05-20T18:30:00.908050+0000","last_became_peered":"2024-05-20T18:30:00.908050+0000","last_unstale":"2024-05-20T18:30:00.911584+0000","last_undegraded":"2024-05-20T18:30:00.911584+0000","last_fullsized":"2024-05-20T18:30:00.911584+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:45:08.997389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.896912+0000","last_change":"2024-05-20T18:29:59.907761+0000","last_active":"2024-05-20T18:30:00.896912+0000","last_peered":"2024-05-20T18:30:00.896912+0000","last_clean":"2024-05-20T18:30:00.896912+0000","last_became_active":"2024-05-20T18:29:59.907378+0000","last_became_peered":"2024-05-20T18:29:59.907378+0000","last_unstale":"2024-05-20T18:30:00.896912+0000","last_undegraded":"2024-05-20T18:30:00.896912+0000","last_fullsized":"2024-05-20T18:30:00.896912+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:48:23.509684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.923807+0000","last_change":"2024-05-20T18:30:00.917188+0000","last_active":"2024-05-20T18:30:00.923807+0000","last_peered":"2024-05-20T18:30:00.923807+0000","last_clean":"2024-05-20T18:30:00.923807+0000","last_became_active":"2024-05-20T18:30:00.916715+0000","last_became_peered":"2024-05-20T18:30:00.916715+0000","last_unstale":"2024-05-20T18:30:00.923807+0000","last_undegraded":"2024-05-20T18:30:00.923807+0000","last_fullsized":"2024-05-20T18:30:00.923807+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:56:29.475238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.919432+0000","last_change":"2024-05-20T18:29:59.916303+0000","last_active":"2024-05-20T18:30:00.919432+0000","last_peered":"2024-05-20T18:30:00.919432+0000","last_clean":"2024-05-20T18:30:00.919432+0000","last_became_active":"2024-05-20T18:29:59.915993+0000","last_became_peered":"2024-05-20T18:29:59.915993+0000","last_unstale":"2024-05-20T18:30:00.919432+0000","last_undegraded":"2024-05-20T18:30:00.919432+0000","last_fullsized":"2024-05-20T18:30:00.919432+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:07:32.070556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.936358+0000","last_change":"2024-05-20T18:30:00.932561+0000","last_active":"2024-05-20T18:30:00.936358+0000","last_peered":"2024-05-20T18:30:00.936358+0000","last_clean":"2024-05-20T18:30:00.936358+0000","last_became_active":"2024-05-20T18:30:00.932232+0000","last_became_peered":"2024-05-20T18:30:00.932232+0000","last_unstale":"2024-05-20T18:30:00.936358+0000","last_undegraded":"2024-05-20T18:30:00.936358+0000","last_fullsized":"2024-05-20T18:30:00.936358+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:14:57.402663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.899929+0000","last_change":"2024-05-20T18:29:59.908482+0000","last_active":"2024-05-20T18:30:00.899929+0000","last_peered":"2024-05-20T18:30:00.899929+0000","last_clean":"2024-05-20T18:30:00.899929+0000","last_became_active":"2024-05-20T18:29:59.906921+0000","last_became_peered":"2024-05-20T18:29:59.906921+0000","last_unstale":"2024-05-20T18:30:00.899929+0000","last_undegraded":"2024-05-20T18:30:00.899929+0000","last_fullsized":"2024-05-20T18:30:00.899929+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:15:45.016943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.927423+0000","last_change":"2024-05-20T18:30:00.926328+0000","last_active":"2024-05-20T18:30:00.927423+0000","last_peered":"2024-05-20T18:30:00.927423+0000","last_clean":"2024-05-20T18:30:00.927423+0000","last_became_active":"2024-05-20T18:30:00.923766+0000","last_became_peered":"2024-05-20T18:30:00.923766+0000","last_unstale":"2024-05-20T18:30:00.927423+0000","last_undegraded":"2024-05-20T18:30:00.927423+0000","last_fullsized":"2024-05-20T18:30:00.927423+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:16:35.637090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.900808+0000","last_change":"2024-05-20T18:29:59.914716+0000","last_active":"2024-05-20T18:30:00.900808+0000","last_peered":"2024-05-20T18:30:00.900808+0000","last_clean":"2024-05-20T18:30:00.900808+0000","last_became_active":"2024-05-20T18:29:59.914370+0000","last_became_peered":"2024-05-20T18:29:59.914370+0000","last_unstale":"2024-05-20T18:30:00.900808+0000","last_undegraded":"2024-05-20T18:30:00.900808+0000","last_fullsized":"2024-05-20T18:30:00.900808+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:56:15.821122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.915264+0000","last_change":"2024-05-20T18:30:00.910719+0000","last_active":"2024-05-20T18:30:00.915264+0000","last_peered":"2024-05-20T18:30:00.915264+0000","last_clean":"2024-05-20T18:30:00.915264+0000","last_became_active":"2024-05-20T18:30:00.910404+0000","last_became_peered":"2024-05-20T18:30:00.910404+0000","last_unstale":"2024-05-20T18:30:00.915264+0000","last_undegraded":"2024-05-20T18:30:00.915264+0000","last_fullsized":"2024-05-20T18:30:00.915264+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:31:13.757884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.895574+0000","last_change":"2024-05-20T18:29:59.913801+0000","last_active":"2024-05-20T18:30:00.895574+0000","last_peered":"2024-05-20T18:30:00.895574+0000","last_clean":"2024-05-20T18:30:00.895574+0000","last_became_active":"2024-05-20T18:29:59.913436+0000","last_became_peered":"2024-05-20T18:29:59.913436+0000","last_unstale":"2024-05-20T18:30:00.895574+0000","last_undegraded":"2024-05-20T18:30:00.895574+0000","last_fullsized":"2024-05-20T18:30:00.895574+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:14:56.393143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.918133+0000","last_change":"2024-05-20T18:30:00.909584+0000","last_active":"2024-05-20T18:30:00.918133+0000","last_peered":"2024-05-20T18:30:00.918133+0000","last_clean":"2024-05-20T18:30:00.918133+0000","last_became_active":"2024-05-20T18:30:00.909243+0000","last_became_peered":"2024-05-20T18:30:00.909243+0000","last_unstale":"2024-05-20T18:30:00.918133+0000","last_undegraded":"2024-05-20T18:30:00.918133+0000","last_fullsized":"2024-05-20T18:30:00.918133+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:59.883982+0000","last_clean_scrub_stamp":"2024-05-20T18:29:59.883982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:18:34.713042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.900408+0000","last_change":"2024-05-20T18:29:59.909797+0000","last_active":"2024-05-20T18:30:00.900408+0000","last_peered":"2024-05-20T18:30:00.900408+0000","last_clean":"2024-05-20T18:30:00.900408+0000","last_became_active":"2024-05-20T18:29:59.909506+0000","last_became_peered":"2024-05-20T18:29:59.909506+0000","last_unstale":"2024-05-20T18:30:00.900408+0000","last_undegraded":"2024-05-20T18:30:00.900408+0000","last_fullsized":"2024-05-20T18:30:00.900408+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:29:58.874461+0000","last_clean_scrub_stamp":"2024-05-20T18:29:58.874461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:21:31.896478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":245,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-20T18:30:00.904181+0000","last_change":"2024-05-20T18:27:03.765694+0000","last_active":"2024-05-20T18:30:00.904181+0000","last_peered":"2024-05-20T18:30:00.904181+0000","last_clean":"2024-05-20T18:30:00.904181+0000","last_became_active":"2024-05-20T18:27:03.764695+0000","last_became_peered":"2024-05-20T18:27:03.764695+0000","last_unstale":"2024-05-20T18:30:00.904181+0000","last_undegraded":"2024-05-20T18:30:00.904181+0000","last_fullsized":"2024-05-20T18:30:00.904181+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_clean_scrub_stamp":"2024-05-20T18:27:02.750853+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:30:04.407487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1331200,"data_stored":919100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":753664,"data_stored":335933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":737280,"data_stored":328732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27740,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344100,"statfs":{"total":96636764160,"available":96608358400,"internally_reserved":0,"allocated":737280,"data_stored":328732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31916,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339924,"statfs":{"total":96636764160,"available":96604082176,"internally_reserved":0,"allocated":749568,"data_stored":335899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31900,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339940,"statfs":{"total":96636764160,"available":96604098560,"internally_reserved":0,"allocated":737280,"data_stored":328732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":778240,"data_stored":338275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1331200,"data_stored":919100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T18:30:05.356 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-20T18:30:05.357 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-20T18:30:05.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.514+0000 7faf82121640 1 Processor -- start 2024-05-20T18:30:05.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.514+0000 7faf82121640 1 -- start start 2024-05-20T18:30:05.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.515+0000 7faf82121640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7faf7c168050 0x7faf7c16a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:05.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.515+0000 7faf82121640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf7c0d0630 con 0x7faf7c06d3d0 2024-05-20T18:30:05.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.516+0000 7faf82121640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf7c0d39c0 con 0x7faf7c162820 2024-05-20T18:30:05.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.516+0000 7faf82121640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7faf7c0d3ca0 con 0x7faf7c168050 2024-05-20T18:30:05.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.516+0000 7faf7affd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7faf7c06d3d0 0x7faf7c06d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44000/0 (socket says 172.21.15.5:44000) 2024-05-20T18:30:05.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.516+0000 7faf7affd640 1 -- 172.21.15.5:0/496325862 learned_addr learned my addr 172.21.15.5:0/496325862 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:30:05.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.516+0000 7faf7bfff640 1 --2- 172.21.15.5:0/496325862 >> v2:172.21.15.63:3300/0 conn(0x7faf7c168050 0x7faf7c16a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.516+0000 7faf7a7fc640 1 -- 172.21.15.5:0/496325862 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 640476467 0 0) 0x7faf7c0d0630 con 0x7faf7c06d3d0 2024-05-20T18:30:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.516+0000 7faf7a7fc640 1 -- 172.21.15.5:0/496325862 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf64003520 con 0x7faf7c06d3d0 2024-05-20T18:30:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.516+0000 7faf7a7fc640 1 -- 172.21.15.5:0/496325862 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 376713161 0 0) 0x7faf7c0d39c0 con 0x7faf7c162820 2024-05-20T18:30:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.516+0000 7faf7a7fc640 1 -- 172.21.15.5:0/496325862 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf7c0d0630 con 0x7faf7c162820 2024-05-20T18:30:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.516+0000 7faf7a7fc640 1 -- 172.21.15.5:0/496325862 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 127102878 0 0) 0x7faf64003520 con 0x7faf7c06d3d0 2024-05-20T18:30:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.516+0000 7faf7bfff640 1 -- 172.21.15.5:0/496325862 >> v1:172.21.15.63:6789/0 conn(0x7faf7c162820 legacy=0x7faf7c0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.516+0000 7faf7bfff640 1 -- 172.21.15.5:0/496325862 >> v1:172.21.15.5:6789/0 conn(0x7faf7c06d3d0 legacy=0x7faf7c06d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.516+0000 7faf7bfff640 1 -- 172.21.15.5:0/496325862 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf7c0d3f80 con 0x7faf7c168050 2024-05-20T18:30:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.517+0000 7faf7bfff640 1 --2- 172.21.15.5:0/496325862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf7c168050 0x7faf7c16a470 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7faf74009d90 tx=0x7faf7402a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4cc34d50213af56 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.517+0000 7faf7a7fc640 1 -- 172.21.15.5:0/496325862 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf70001670 con 0x7faf7c168050 2024-05-20T18:30:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.517+0000 7faf7a7fc640 1 -- 172.21.15.5:0/496325862 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf74007540 con 0x7faf7c168050 2024-05-20T18:30:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.517+0000 7faf7a7fc640 1 -- 172.21.15.5:0/496325862 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf74007910 con 0x7faf7c168050 2024-05-20T18:30:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.518+0000 7faf82121640 1 -- 172.21.15.5:0/496325862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf7c168050 msgr2=0x7faf7c16a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.518+0000 7faf82121640 1 --2- 172.21.15.5:0/496325862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf7c168050 0x7faf7c16a470 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7faf74009d90 tx=0x7faf7402a680 comp rx=0 tx=0).stop 2024-05-20T18:30:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.518+0000 7faf82121640 1 -- 172.21.15.5:0/496325862 shutdown_connections 2024-05-20T18:30:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.518+0000 7faf82121640 1 --2- 172.21.15.5:0/496325862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf7c168050 0x7faf7c16a470 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.518+0000 7faf82121640 1 -- 172.21.15.5:0/496325862 >> 172.21.15.5:0/496325862 conn(0x7faf7c0d4bd0 msgr2=0x7faf7c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.518+0000 7faf82121640 1 -- 172.21.15.5:0/496325862 shutdown_connections 2024-05-20T18:30:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.518+0000 7faf82121640 1 -- 172.21.15.5:0/496325862 wait complete. 2024-05-20T18:30:05.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.520+0000 7faf82121640 1 Processor -- start 2024-05-20T18:30:05.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.520+0000 7faf82121640 1 -- start start 2024-05-20T18:30:05.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.521+0000 7faf82121640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf7c06d3d0 0x7faf7c1576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:05.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.521+0000 7faf82121640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf7c162820 0x7faf7c0cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:05.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.521+0000 7faf82121640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf7c168050 0x7faf7c0cda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:05.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.521+0000 7faf82121640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c0b91a0 con 0x7faf7c162820 2024-05-20T18:30:05.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.521+0000 7faf82121640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c0d0300 con 0x7faf7c06d3d0 2024-05-20T18:30:05.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.521+0000 7faf82121640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faf7c0b4050 con 0x7faf7c168050 2024-05-20T18:30:05.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.522+0000 7faf7affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf7c162820 0x7faf7c0cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:05.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.522+0000 7faf7affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf7c162820 0x7faf7c0cbc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53070/0 (socket says 172.21.15.5:53070) 2024-05-20T18:30:05.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.522+0000 7faf7affd640 1 -- 172.21.15.5:0/626556469 learned_addr learned my addr 172.21.15.5:0/626556469 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:05.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.522+0000 7faf7bfff640 1 --2- 172.21.15.5:0/626556469 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf7c168050 0x7faf7c0cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:05.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.522+0000 7faf7b7fe640 1 --2- 172.21.15.5:0/626556469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf7c06d3d0 0x7faf7c1576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:05.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.522+0000 7faf7affd640 1 -- 172.21.15.5:0/626556469 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf7c168050 msgr2=0x7faf7c0cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.522+0000 7faf7affd640 1 --2- 172.21.15.5:0/626556469 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf7c168050 0x7faf7c0cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.522+0000 7faf7affd640 1 -- 172.21.15.5:0/626556469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf7c06d3d0 msgr2=0x7faf7c1576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.522+0000 7faf7affd640 1 --2- 172.21.15.5:0/626556469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf7c06d3d0 0x7faf7c1576e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.522+0000 7faf7affd640 1 -- 172.21.15.5:0/626556469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf7c0ceb50 con 0x7faf7c162820 2024-05-20T18:30:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.522+0000 7faf7affd640 1 --2- 172.21.15.5:0/626556469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf7c162820 0x7faf7c0cbc60 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7faf7000c520 tx=0x7faf7000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.522+0000 7faf63fff640 1 -- 172.21.15.5:0/626556469 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf70017070 con 0x7faf7c162820 2024-05-20T18:30:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.522+0000 7faf63fff640 1 -- 172.21.15.5:0/626556469 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf70005ce0 con 0x7faf7c162820 2024-05-20T18:30:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.522+0000 7faf63fff640 1 -- 172.21.15.5:0/626556469 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf700072b0 con 0x7faf7c162820 2024-05-20T18:30:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.522+0000 7faf82121640 1 -- 172.21.15.5:0/626556469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf7c0c9100 con 0x7faf7c162820 2024-05-20T18:30:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.522+0000 7faf82121640 1 -- 172.21.15.5:0/626556469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf7c0c96c0 con 0x7faf7c162820 2024-05-20T18:30:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.523+0000 7faf82121640 1 -- 172.21.15.5:0/626556469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf7c075b00 con 0x7faf7c162820 2024-05-20T18:30:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.524+0000 7faf63fff640 1 -- 172.21.15.5:0/626556469 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faf7001a070 con 0x7faf7c162820 2024-05-20T18:30:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.524+0000 7faf63fff640 1 --2- 172.21.15.5:0/626556469 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faf6806be60 0x7faf6806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.525+0000 7faf63fff640 1 -- 172.21.15.5:0/626556469 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7faf7008a3a0 con 0x7faf7c162820 2024-05-20T18:30:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.525+0000 7faf7b7fe640 1 --2- 172.21.15.5:0/626556469 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faf6806be60 0x7faf6806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:05.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.525+0000 7faf7b7fe640 1 --2- 172.21.15.5:0/626556469 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faf6806be60 0x7faf6806e370 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7faf6c0011e0 tx=0x7faf6c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:05.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.529+0000 7faf63fff640 1 -- 172.21.15.5:0/626556469 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7faf70058570 con 0x7faf7c162820 2024-05-20T18:30:05.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.788+0000 7faf82121640 1 -- 172.21.15.5:0/626556469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7faf7c15d020 con 0x7faf7c162820 2024-05-20T18:30:05.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.790+0000 7faf63fff640 1 -- 172.21.15.5:0/626556469 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7faf7005c870 con 0x7faf7c162820 2024-05-20T18:30:05.790 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2024-05-20T18:30:05.790 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDFlUtmevaaIhAATIwyXzr0vGT38ZmcMqR2gA== 2024-05-20T18:30:05.790 INFO:teuthology.orchestra.run.smithi005.stdout: caps mds = "allow" 2024-05-20T18:30:05.791 INFO:teuthology.orchestra.run.smithi005.stdout: caps mon = "allow r" 2024-05-20T18:30:05.791 INFO:teuthology.orchestra.run.smithi005.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-20T18:30:05.791 INFO:teuthology.orchestra.run.smithi005.stderr:updated caps for client.0 2024-05-20T18:30:05.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.792+0000 7faf82121640 1 -- 172.21.15.5:0/626556469 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faf6806be60 msgr2=0x7faf6806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:05.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.792+0000 7faf82121640 1 --2- 172.21.15.5:0/626556469 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faf6806be60 0x7faf6806e370 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7faf6c0011e0 tx=0x7faf6c009040 comp rx=0 tx=0).stop 2024-05-20T18:30:05.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.792+0000 7faf82121640 1 -- 172.21.15.5:0/626556469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf7c162820 msgr2=0x7faf7c0cbc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:05.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.792+0000 7faf82121640 1 --2- 172.21.15.5:0/626556469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf7c162820 0x7faf7c0cbc60 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7faf7000c520 tx=0x7faf7000c9e0 comp rx=0 tx=0).stop 2024-05-20T18:30:05.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.792+0000 7faf82121640 1 -- 172.21.15.5:0/626556469 shutdown_connections 2024-05-20T18:30:05.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.792+0000 7faf82121640 1 --2- 172.21.15.5:0/626556469 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faf6806be60 0x7faf6806e370 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:05.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.792+0000 7faf82121640 1 --2- 172.21.15.5:0/626556469 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf7c168050 0x7faf7c0cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:05.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.792+0000 7faf82121640 1 --2- 172.21.15.5:0/626556469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf7c162820 0x7faf7c0cbc60 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:05.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.792+0000 7faf82121640 1 --2- 172.21.15.5:0/626556469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf7c06d3d0 0x7faf7c1576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:05.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.792+0000 7faf82121640 1 -- 172.21.15.5:0/626556469 >> 172.21.15.5:0/626556469 conn(0x7faf7c0d4bd0 msgr2=0x7faf7c160eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:05.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.793+0000 7faf82121640 1 -- 172.21.15.5:0/626556469 shutdown_connections 2024-05-20T18:30:05.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.793+0000 7faf82121640 1 -- 172.21.15.5:0/626556469 wait complete. 2024-05-20T18:30:05.804 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:30:05.879 INFO:tasks.ceph.mds.b.smithi063.stderr:2024-05-20T18:30:05.879+0000 7f9764196640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T18:30:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.918+0000 7efdf8af7640 1 Processor -- start 2024-05-20T18:30:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.918+0000 7efdf8af7640 1 -- start start 2024-05-20T18:30:05.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.919+0000 7efdf8af7640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efdf40cfae0 0x7efdf416a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:05.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.919+0000 7efdf8af7640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdf406d0f0 con 0x7efdf40cd680 2024-05-20T18:30:05.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.919+0000 7efdf8af7640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdf406d3d0 con 0x7efdf415f220 2024-05-20T18:30:05.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.919+0000 7efdf8af7640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7efdf40cda90 con 0x7efdf40cfae0 2024-05-20T18:30:05.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.920+0000 7efdf2d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efdf40cfae0 0x7efdf416a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:05.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.920+0000 7efdf259c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7efdf40cd680 0x7efdf406c6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44004/0 (socket says 172.21.15.5:44004) 2024-05-20T18:30:05.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.920+0000 7efdf259c640 1 -- 172.21.15.5:0/1653622881 learned_addr learned my addr 172.21.15.5:0/1653622881 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:30:05.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.920+0000 7efdf2d9d640 1 --2- 172.21.15.5:0/1653622881 >> v2:172.21.15.5:3300/0 conn(0x7efdf40cfae0 0x7efdf416a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53072/0 (socket says 172.21.15.5:53072) 2024-05-20T18:30:05.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.920+0000 7efdf2d9d640 1 -- 172.21.15.5:0/1653622881 >> v1:172.21.15.63:6789/0 conn(0x7efdf415f220 legacy=0x7efdf40cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:05.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.920+0000 7efdf2d9d640 1 -- 172.21.15.5:0/1653622881 >> v1:172.21.15.5:6789/0 conn(0x7efdf40cd680 legacy=0x7efdf406c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:05.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.920+0000 7efdf2d9d640 1 -- 172.21.15.5:0/1653622881 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf406d6b0 con 0x7efdf40cfae0 2024-05-20T18:30:05.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.920+0000 7efdf1d9b640 1 -- 172.21.15.5:0/1653622881 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1337739901 0 0) 0x7efdf406d0f0 con 0x7efdf40cd680 2024-05-20T18:30:05.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.921+0000 7efdf2d9d640 1 --2- 172.21.15.5:0/1653622881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdf40cfae0 0x7efdf416a210 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7efdec009d90 tx=0x7efdec02a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b8bf09af2bbf17 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:05.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.921+0000 7efdf1d9b640 1 -- 172.21.15.5:0/1653622881 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec007740 con 0x7efdf40cfae0 2024-05-20T18:30:05.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.921+0000 7efdf1d9b640 1 -- 172.21.15.5:0/1653622881 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdec0078e0 con 0x7efdf40cfae0 2024-05-20T18:30:05.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.921+0000 7efdf1d9b640 1 -- 172.21.15.5:0/1653622881 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec007cb0 con 0x7efdf40cfae0 2024-05-20T18:30:05.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.922+0000 7efdf8af7640 1 -- 172.21.15.5:0/1653622881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdf40cfae0 msgr2=0x7efdf416a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:05.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.922+0000 7efdf8af7640 1 --2- 172.21.15.5:0/1653622881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdf40cfae0 0x7efdf416a210 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7efdec009d90 tx=0x7efdec02a580 comp rx=0 tx=0).stop 2024-05-20T18:30:05.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.922+0000 7efdf8af7640 1 -- 172.21.15.5:0/1653622881 shutdown_connections 2024-05-20T18:30:05.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.922+0000 7efdf8af7640 1 --2- 172.21.15.5:0/1653622881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdf40cfae0 0x7efdf416a210 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:05.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.922+0000 7efdf8af7640 1 -- 172.21.15.5:0/1653622881 >> 172.21.15.5:0/1653622881 conn(0x7efdf40b44d0 msgr2=0x7efdf40b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:05.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.922+0000 7efdf8af7640 1 -- 172.21.15.5:0/1653622881 shutdown_connections 2024-05-20T18:30:05.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.922+0000 7efdf8af7640 1 -- 172.21.15.5:0/1653622881 wait complete. 2024-05-20T18:30:05.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.924+0000 7efdf8af7640 1 Processor -- start 2024-05-20T18:30:05.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.924+0000 7efdf8af7640 1 -- start start 2024-05-20T18:30:05.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.925+0000 7efdf8af7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efdf40cd680 0x7efdf4150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:05.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.925+0000 7efdf8af7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efdf415f220 0x7efdf40b2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:05.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.925+0000 7efdf8af7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdf40b80a0 0x7efdf40b2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:05.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.925+0000 7efdf8af7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efdf406cdd0 con 0x7efdf40b80a0 2024-05-20T18:30:05.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.925+0000 7efdf8af7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efdf406da10 con 0x7efdf415f220 2024-05-20T18:30:05.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.925+0000 7efdf8af7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efdf415f630 con 0x7efdf40cd680 2024-05-20T18:30:05.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.925+0000 7efdf2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdf40b80a0 0x7efdf40b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:05.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.925+0000 7efdf2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdf40b80a0 0x7efdf40b2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53074/0 (socket says 172.21.15.5:53074) 2024-05-20T18:30:05.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.925+0000 7efdf2d9d640 1 -- 172.21.15.5:0/3216601584 learned_addr learned my addr 172.21.15.5:0/3216601584 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:05.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.925+0000 7efdf259c640 1 --2- 172.21.15.5:0/3216601584 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efdf40cd680 0x7efdf4150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:05.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.925+0000 7efdebfff640 1 --2- 172.21.15.5:0/3216601584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efdf415f220 0x7efdf40b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:05.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.926+0000 7efdf2d9d640 1 -- 172.21.15.5:0/3216601584 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efdf40cd680 msgr2=0x7efdf4150fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:05.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.926+0000 7efdf2d9d640 1 --2- 172.21.15.5:0/3216601584 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efdf40cd680 0x7efdf4150fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:05.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.926+0000 7efdf2d9d640 1 -- 172.21.15.5:0/3216601584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efdf415f220 msgr2=0x7efdf40b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:05.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.926+0000 7efdf2d9d640 1 --2- 172.21.15.5:0/3216601584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efdf415f220 0x7efdf40b2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:05.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.926+0000 7efdf2d9d640 1 -- 172.21.15.5:0/3216601584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf40b6b40 con 0x7efdf40b80a0 2024-05-20T18:30:05.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.926+0000 7efdf2d9d640 1 --2- 172.21.15.5:0/3216601584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdf40b80a0 0x7efdf40b2880 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7efdec02ab70 tx=0x7efdec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:05.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.927+0000 7efdeaffd640 1 -- 172.21.15.5:0/3216601584 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec043dc0 con 0x7efdf40b80a0 2024-05-20T18:30:05.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.927+0000 7efdeaffd640 1 -- 172.21.15.5:0/3216601584 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdec031700 con 0x7efdf40b80a0 2024-05-20T18:30:05.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.928+0000 7efdf8af7640 1 -- 172.21.15.5:0/3216601584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf415f9e0 con 0x7efdf40b80a0 2024-05-20T18:30:05.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.928+0000 7efdeaffd640 1 -- 172.21.15.5:0/3216601584 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec031ab0 con 0x7efdf40b80a0 2024-05-20T18:30:05.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.928+0000 7efdf8af7640 1 -- 172.21.15.5:0/3216601584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf415ffa0 con 0x7efdf40b80a0 2024-05-20T18:30:05.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.929+0000 7efdf8af7640 1 -- 172.21.15.5:0/3216601584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdf4075b00 con 0x7efdf40b80a0 2024-05-20T18:30:05.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.929+0000 7efdeaffd640 1 -- 172.21.15.5:0/3216601584 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efdec039040 con 0x7efdf40b80a0 2024-05-20T18:30:05.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.930+0000 7efdeaffd640 1 --2- 172.21.15.5:0/3216601584 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efdd406be30 0x7efdd406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:05.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.930+0000 7efdeaffd640 1 -- 172.21.15.5:0/3216601584 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7efdec0b4e30 con 0x7efdf40b80a0 2024-05-20T18:30:05.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.930+0000 7efdf259c640 1 --2- 172.21.15.5:0/3216601584 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efdd406be30 0x7efdd406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:05.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.930+0000 7efdf259c640 1 --2- 172.21.15.5:0/3216601584 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efdd406be30 0x7efdd406e340 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efde4007fd0 tx=0x7efde4007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:05.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:05.934+0000 7efdeaffd640 1 -- 172.21.15.5:0/3216601584 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7efdf40b6b40 con 0x7efdf40b80a0 2024-05-20T18:30:06.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.155+0000 7efdf8af7640 1 -- 172.21.15.5:0/3216601584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efdf40b8940 con 0x7efdf40b80a0 2024-05-20T18:30:06.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.155+0000 7efdeaffd640 1 -- 172.21.15.5:0/3216601584 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5176 (secure 0 0 0) 0x7efdec0835c0 con 0x7efdf40b80a0 2024-05-20T18:30:06.155 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:30:06.155 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5480,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/4292435960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":4292435960},{"type":"v1","addr":"172.21.15.63:6835","nonce":4292435960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6186,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2991834087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2991834087},{"type":"v1","addr":"172.21.15.5:6836","nonce":2991834087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6195,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3462206198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":3462206198},{"type":"v1","addr":"172.21.15.5:6837","nonce":3462206198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:30:00.286575+0000","modified":"2024-05-20T18:30:03.297495+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6198":{"gid":6198,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6834/4036379300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":4036379300},{"type":"v1","addr":"172.21.15.63:6834","nonce":4036379300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6198,"qdb_cluster":[6198]},"id":4}]} 2024-05-20T18:30:06.156 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 34 2024-05-20T18:30:06.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.157+0000 7efdf8af7640 1 -- 172.21.15.5:0/3216601584 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efdd406be30 msgr2=0x7efdd406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:06.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.157+0000 7efdf8af7640 1 --2- 172.21.15.5:0/3216601584 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efdd406be30 0x7efdd406e340 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efde4007fd0 tx=0x7efde4007de0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.157+0000 7efdf8af7640 1 -- 172.21.15.5:0/3216601584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdf40b80a0 msgr2=0x7efdf40b2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:06.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.157+0000 7efdf8af7640 1 --2- 172.21.15.5:0/3216601584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdf40b80a0 0x7efdf40b2880 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7efdec02ab70 tx=0x7efdec004a40 comp rx=0 tx=0).stop 2024-05-20T18:30:06.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.157+0000 7efdf8af7640 1 -- 172.21.15.5:0/3216601584 shutdown_connections 2024-05-20T18:30:06.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.157+0000 7efdf8af7640 1 --2- 172.21.15.5:0/3216601584 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efdd406be30 0x7efdd406e340 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.157+0000 7efdf8af7640 1 --2- 172.21.15.5:0/3216601584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdf40b80a0 0x7efdf40b2880 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.157+0000 7efdf8af7640 1 --2- 172.21.15.5:0/3216601584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efdf415f220 0x7efdf40b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.157+0000 7efdf8af7640 1 --2- 172.21.15.5:0/3216601584 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efdf40cd680 0x7efdf4150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.157+0000 7efdf8af7640 1 -- 172.21.15.5:0/3216601584 >> 172.21.15.5:0/3216601584 conn(0x7efdf40b44d0 msgr2=0x7efdf40ce0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:06.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.157+0000 7efdf8af7640 1 -- 172.21.15.5:0/3216601584 shutdown_connections 2024-05-20T18:30:06.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.157+0000 7efdf8af7640 1 -- 172.21.15.5:0/3216601584 wait complete. 2024-05-20T18:30:06.572 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T18:30:00.286575+0000', 'modified': '2024-05-20T18:30:03.297495+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6198}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6198': {'gid': 6198, 'name': 'b', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.63:6834/4036379300', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6832', 'nonce': 4036379300}, {'type': 'v1', 'addr': '172.21.15.63:6834', 'nonce': 4036379300}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6198, 'qdb_cluster': [6198]} 2024-05-20T18:30:06.572 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-20T18:30:06.572 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-20T18:30:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.693+0000 7fae2c8f0640 1 Processor -- start 2024-05-20T18:30:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.693+0000 7fae2c8f0640 1 -- start start 2024-05-20T18:30:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.695+0000 7fae2c8f0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae28162820 0x7fae280b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.695+0000 7fae2c8f0640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fae2806dae0 0x7fae280b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.695+0000 7fae2c8f0640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae280d0630 con 0x7fae280b3e80 2024-05-20T18:30:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.695+0000 7fae2c8f0640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fae280d39c0 con 0x7fae2806dae0 2024-05-20T18:30:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.695+0000 7fae2c8f0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fae280d3ca0 con 0x7fae28162820 2024-05-20T18:30:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.695+0000 7fae26d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fae2806dae0 0x7fae280b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.695+0000 7fae26d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fae2806dae0 0x7fae280b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53080/0 (socket says 172.21.15.5:53080) 2024-05-20T18:30:06.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.695+0000 7fae26d9d640 1 -- 172.21.15.5:0/1000412991 learned_addr learned my addr 172.21.15.5:0/1000412991 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:06.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.695+0000 7fae2659c640 1 --2- 172.21.15.5:0/1000412991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae28162820 0x7fae280b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:06.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.695+0000 7fae26d9d640 1 -- 172.21.15.5:0/1000412991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae28162820 msgr2=0x7fae280b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:06.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.695+0000 7fae26d9d640 1 --2- 172.21.15.5:0/1000412991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae28162820 0x7fae280b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.695+0000 7fae26d9d640 1 -- 172.21.15.5:0/1000412991 >> v1:172.21.15.63:6789/0 conn(0x7fae280b3e80 legacy=0x7fae2806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:06.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.695+0000 7fae26d9d640 1 -- 172.21.15.5:0/1000412991 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae280d3f80 con 0x7fae2806dae0 2024-05-20T18:30:06.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.696+0000 7fae26d9d640 1 --2- 172.21.15.5:0/1000412991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae2806dae0 0x7fae280b8190 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fae20009e50 tx=0x7fae2002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b87ea19c216c079 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:06.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.696+0000 7fae2559a640 1 -- 172.21.15.5:0/1000412991 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20036070 con 0x7fae2806dae0 2024-05-20T18:30:06.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.696+0000 7fae2559a640 1 -- 172.21.15.5:0/1000412991 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae20031600 con 0x7fae2806dae0 2024-05-20T18:30:06.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.696+0000 7fae2559a640 1 -- 172.21.15.5:0/1000412991 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae200319d0 con 0x7fae2806dae0 2024-05-20T18:30:06.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.697+0000 7fae2c8f0640 1 -- 172.21.15.5:0/1000412991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae2806dae0 msgr2=0x7fae280b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.697+0000 7fae2c8f0640 1 --2- 172.21.15.5:0/1000412991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae2806dae0 0x7fae280b8190 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fae20009e50 tx=0x7fae2002a710 comp rx=0 tx=0).stop 2024-05-20T18:30:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.697+0000 7fae2c8f0640 1 -- 172.21.15.5:0/1000412991 shutdown_connections 2024-05-20T18:30:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.697+0000 7fae2c8f0640 1 --2- 172.21.15.5:0/1000412991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae2806dae0 0x7fae280b8190 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.697+0000 7fae2c8f0640 1 --2- 172.21.15.5:0/1000412991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae28162820 0x7fae280b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.697+0000 7fae2c8f0640 1 -- 172.21.15.5:0/1000412991 >> 172.21.15.5:0/1000412991 conn(0x7fae280d4bd0 msgr2=0x7fae280d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.697+0000 7fae2c8f0640 1 -- 172.21.15.5:0/1000412991 shutdown_connections 2024-05-20T18:30:06.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.697+0000 7fae2c8f0640 1 -- 172.21.15.5:0/1000412991 wait complete. 2024-05-20T18:30:06.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.699+0000 7fae2c8f0640 1 Processor -- start 2024-05-20T18:30:06.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.699+0000 7fae2c8f0640 1 -- start start 2024-05-20T18:30:06.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.700+0000 7fae2c8f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae280b3e80 0x7fae280c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:06.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.700+0000 7fae2c8f0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae28162820 0x7fae280caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:06.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.700+0000 7fae2c8f0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae280c9250 0x7fae28155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:06.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.700+0000 7fae2c8f0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fae280d4780 con 0x7fae280b3e80 2024-05-20T18:30:06.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.700+0000 7fae2c8f0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fae280d0300 con 0x7fae280c9250 2024-05-20T18:30:06.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.700+0000 7fae2c8f0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fae280d4900 con 0x7fae28162820 2024-05-20T18:30:06.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.700+0000 7fae2659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae280b3e80 0x7fae280c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:06.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.700+0000 7fae2659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae280b3e80 0x7fae280c8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53090/0 (socket says 172.21.15.5:53090) 2024-05-20T18:30:06.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.701+0000 7fae2659c640 1 -- 172.21.15.5:0/2369564861 learned_addr learned my addr 172.21.15.5:0/2369564861 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:06.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.701+0000 7fae26d9d640 1 --2- 172.21.15.5:0/2369564861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae280c9250 0x7fae28155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.701+0000 7fae25d9b640 1 --2- 172.21.15.5:0/2369564861 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae28162820 0x7fae280caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.701+0000 7fae2659c640 1 -- 172.21.15.5:0/2369564861 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae28162820 msgr2=0x7fae280caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.701+0000 7fae2659c640 1 --2- 172.21.15.5:0/2369564861 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae28162820 0x7fae280caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.701+0000 7fae2659c640 1 -- 172.21.15.5:0/2369564861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae280c9250 msgr2=0x7fae28155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.701+0000 7fae2659c640 1 --2- 172.21.15.5:0/2369564861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae280c9250 0x7fae28155790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.701+0000 7fae2659c640 1 -- 172.21.15.5:0/2369564861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae2815ee30 con 0x7fae280b3e80 2024-05-20T18:30:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.701+0000 7fae26d9d640 1 --2- 172.21.15.5:0/2369564861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae280c9250 0x7fae28155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:30:06.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.701+0000 7fae2659c640 1 --2- 172.21.15.5:0/2369564861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae280b3e80 0x7fae280c8d10 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fae1800e870 tx=0x7fae1800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:06.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.701+0000 7fae16ffd640 1 -- 172.21.15.5:0/2369564861 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18017070 con 0x7fae280b3e80 2024-05-20T18:30:06.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.701+0000 7fae16ffd640 1 -- 172.21.15.5:0/2369564861 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae1800c940 con 0x7fae280b3e80 2024-05-20T18:30:06.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.701+0000 7fae2c8f0640 1 -- 172.21.15.5:0/2369564861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae2815f110 con 0x7fae280b3e80 2024-05-20T18:30:06.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.701+0000 7fae16ffd640 1 -- 172.21.15.5:0/2369564861 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1800cd30 con 0x7fae280b3e80 2024-05-20T18:30:06.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.701+0000 7fae2c8f0640 1 -- 172.21.15.5:0/2369564861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae2815f6d0 con 0x7fae280b3e80 2024-05-20T18:30:06.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.703+0000 7fae16ffd640 1 -- 172.21.15.5:0/2369564861 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fae1801a070 con 0x7fae280b3e80 2024-05-20T18:30:06.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.703+0000 7fae16ffd640 1 --2- 172.21.15.5:0/2369564861 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae0806be60 0x7fae0806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:06.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.704+0000 7fae16ffd640 1 -- 172.21.15.5:0/2369564861 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7fae1808af00 con 0x7fae280b3e80 2024-05-20T18:30:06.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.704+0000 7fae25d9b640 1 --2- 172.21.15.5:0/2369564861 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae0806be60 0x7fae0806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:06.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.704+0000 7fae25d9b640 1 --2- 172.21.15.5:0/2369564861 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae0806be60 0x7fae0806e370 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fae28159d80 tx=0x7fae1c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:06.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.704+0000 7fae33798740 1 Processor -- start 2024-05-20T18:30:06.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.705+0000 7fae33798740 1 -- start start 2024-05-20T18:30:06.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.706+0000 7fae33798740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56492504f270 0x56492541a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:06.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.706+0000 7fae33798740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649250263b0 0x5649254170a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:06.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.706+0000 7fae33798740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56492502e590 0x5649253f0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:06.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.706+0000 7fae33798740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5649253534d0 con 0x56492502e590 2024-05-20T18:30:06.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.706+0000 7fae33798740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5649252fb3e0 con 0x5649250263b0 2024-05-20T18:30:06.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.706+0000 7fae33798740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x56492534f490 con 0x56492504f270 2024-05-20T18:30:06.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.706+0000 7fae26d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56492502e590 0x5649253f0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:06.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.706+0000 7fae26d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56492502e590 0x5649253f0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53098/0 (socket says 172.21.15.5:53098) 2024-05-20T18:30:06.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.706+0000 7fae26d9d640 1 -- 172.21.15.5:0/1922425957 learned_addr learned my addr 172.21.15.5:0/1922425957 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:06.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.706+0000 7fae2659c640 1 --2- 172.21.15.5:0/1922425957 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56492504f270 0x56492541a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:06.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.706+0000 7fae25d9b640 1 --2- 172.21.15.5:0/1922425957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649250263b0 0x5649254170a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:06.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.706+0000 7fae26d9d640 1 -- 172.21.15.5:0/1922425957 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56492504f270 msgr2=0x56492541a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:06.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.706+0000 7fae26d9d640 1 --2- 172.21.15.5:0/1922425957 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56492504f270 0x56492541a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.706+0000 7fae26d9d640 1 -- 172.21.15.5:0/1922425957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649250263b0 msgr2=0x5649254170a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:06.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.706+0000 7fae26d9d640 1 --2- 172.21.15.5:0/1922425957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649250263b0 0x5649254170a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.706+0000 7fae26d9d640 1 -- 172.21.15.5:0/1922425957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649250672d0 con 0x56492502e590 2024-05-20T18:30:06.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.706+0000 7fae25d9b640 1 --2- 172.21.15.5:0/1922425957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649250263b0 0x5649254170a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:30:06.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.707+0000 7fae26d9d640 1 --2- 172.21.15.5:0/1922425957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56492502e590 0x5649253f0360 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fae20009f70 tx=0x7fae2002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354283fc6ec5166b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.707+0000 7fae07fff640 1 -- 172.21.15.5:0/1922425957 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20036040 con 0x56492502e590 2024-05-20T18:30:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.707+0000 7fae07fff640 1 -- 172.21.15.5:0/1922425957 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae20041070 con 0x56492502e590 2024-05-20T18:30:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.707+0000 7fae07fff640 1 -- 172.21.15.5:0/1922425957 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2003c870 con 0x56492502e590 2024-05-20T18:30:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.708+0000 7fae33798740 1 -- 172.21.15.5:0/1922425957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56492502e590 msgr2=0x5649253f0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.708+0000 7fae33798740 1 --2- 172.21.15.5:0/1922425957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56492502e590 0x5649253f0360 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fae20009f70 tx=0x7fae2002e000 comp rx=0 tx=0).stop 2024-05-20T18:30:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.708+0000 7fae33798740 1 -- 172.21.15.5:0/1922425957 shutdown_connections 2024-05-20T18:30:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.708+0000 7fae33798740 1 --2- 172.21.15.5:0/1922425957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56492502e590 0x5649253f0360 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.708+0000 7fae33798740 1 --2- 172.21.15.5:0/1922425957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649250263b0 0x5649254170a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.708+0000 7fae33798740 1 --2- 172.21.15.5:0/1922425957 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56492504f270 0x56492541a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.708+0000 7fae33798740 1 -- 172.21.15.5:0/1922425957 >> 172.21.15.5:0/1922425957 conn(0x5649250433c0 msgr2=0x5649253efaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.708+0000 7fae33798740 1 -- 172.21.15.5:0/1922425957 shutdown_connections 2024-05-20T18:30:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.708+0000 7fae33798740 1 -- 172.21.15.5:0/1922425957 wait complete. 2024-05-20T18:30:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.709+0000 7fae33798740 1 -- start start 2024-05-20T18:30:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.709+0000 7fae33798740 1 Processor -- start 2024-05-20T18:30:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.710+0000 7fae33798740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649250263b0 0x564925435610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.710+0000 7fae33798740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56492502e590 0x56492541b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.710+0000 7fae33798740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56492504f270 0x56492541b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.710+0000 7fae33798740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56492530e8e0 con 0x56492504f270 2024-05-20T18:30:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.710+0000 7fae33798740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564925412e20 con 0x5649250263b0 2024-05-20T18:30:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.710+0000 7fae33798740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x56492541d310 con 0x56492502e590 2024-05-20T18:30:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.710+0000 7fae33798740 10 client.0 _ll_register_callbacks cb 0x5649252db9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:30:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.710+0000 7fae26d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56492504f270 0x56492541b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.710+0000 7fae26d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56492504f270 0x56492541b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53106/0 (socket says 172.21.15.5:53106) 2024-05-20T18:30:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.710+0000 7fae26d9d640 1 -- 172.21.15.5:0/2793876797 learned_addr learned my addr 172.21.15.5:0/2793876797 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.710+0000 7fae2659c640 1 --2- 172.21.15.5:0/2793876797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649250263b0 0x564925435610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.710+0000 7fae25d9b640 1 --2- 172.21.15.5:0/2793876797 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56492502e590 0x56492541b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.711+0000 7fae26d9d640 1 -- 172.21.15.5:0/2793876797 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56492502e590 msgr2=0x56492541b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.711+0000 7fae26d9d640 1 --2- 172.21.15.5:0/2793876797 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56492502e590 0x56492541b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.711+0000 7fae26d9d640 1 -- 172.21.15.5:0/2793876797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649250263b0 msgr2=0x564925435610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.711+0000 7fae26d9d640 1 --2- 172.21.15.5:0/2793876797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649250263b0 0x564925435610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.711+0000 7fae26d9d640 1 -- 172.21.15.5:0/2793876797 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649251316b0 con 0x56492504f270 2024-05-20T18:30:06.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.711+0000 7fae26d9d640 1 --2- 172.21.15.5:0/2793876797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56492504f270 0x56492541b8e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fae200093e0 tx=0x7fae20004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df7640f41b841f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:06.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.711+0000 7fae057fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:30:06.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.711+0000 7fae057fa640 1 -- 172.21.15.5:0/2793876797 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20036050 con 0x56492504f270 2024-05-20T18:30:06.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.711+0000 7fae057fa640 1 -- 172.21.15.5:0/2793876797 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7fae20004450 con 0x56492504f270 2024-05-20T18:30:06.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.711+0000 7fae33798740 1 -- 172.21.15.5:0/2793876797 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae20041070 con 0x56492504f270 2024-05-20T18:30:06.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.711+0000 7fae057fa640 1 -- 172.21.15.5:0/2793876797 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fae20009af0 con 0x56492504f270 2024-05-20T18:30:06.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.711+0000 7fae33798740 10 client.6345 fetch_fsmap learned FSMap version 34 2024-05-20T18:30:06.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.711+0000 7fae33798740 1 -- 172.21.15.5:0/2793876797 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x564925434910 con 0x56492504f270 2024-05-20T18:30:06.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.711+0000 7fae057fa640 1 -- 172.21.15.5:0/2793876797 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae20041070 con 0x56492504f270 2024-05-20T18:30:06.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.711+0000 7fae057fa640 1 -- 172.21.15.5:0/2793876797 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7fae20004650 con 0x56492504f270 2024-05-20T18:30:06.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.711+0000 7fae33798740 10 client.6345 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T18:30:06.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.711+0000 7fae33798740 10 client.6345 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-20T18:30:06.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.712+0000 7fae33798740 20 client.6345 populate_metadata read hostname 'smithi005' 2024-05-20T18:30:06.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.712+0000 7fae33798740 1 --2- 172.21.15.5:0/2793876797 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x56492500de40 0x564925433cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:06.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.712+0000 7fae33798740 4 client.6345 mds_command: new command op to 6198 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:30:06.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.712+0000 7fae33798740 1 -- 172.21.15.5:0/2793876797 --> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564925360c00 con 0x56492500de40 2024-05-20T18:30:06.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.712+0000 7fae2659c640 1 --2- 172.21.15.5:0/2793876797 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x56492500de40 0x564925433cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:06.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.712+0000 7fae057fa640 10 client.6345 ms_handle_connect on v2:172.21.15.63:6832/4036379300 2024-05-20T18:30:06.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.712+0000 7fae2659c640 1 --2- 172.21.15.5:0/2793876797 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x56492500de40 0x564925433cc0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a46c2d56e27b1459 server_cookie=bbb2cd6140cb1068 in_seq=0 out_seq=0 2024-05-20T18:30:06.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.713+0000 7fae057fa640 1 -- 172.21.15.5:0/2793876797 <== mds.0 v2:172.21.15.63:6832/4036379300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564925360c00 con 0x56492500de40 2024-05-20T18:30:06.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.713+0000 7fae057fa640 10 client.6345 handle_command_reply: tid=0 2024-05-20T18:30:06.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.713+0000 7fae33798740 1 client.6345 shutdown 2024-05-20T18:30:06.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.713+0000 7fae33798740 1 -- 172.21.15.5:0/2793876797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56492504f270 msgr2=0x56492541b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:06.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.713+0000 7fae33798740 1 --2- 172.21.15.5:0/2793876797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56492504f270 0x56492541b8e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fae200093e0 tx=0x7fae20004ab0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.713+0000 7fae33798740 1 -- 172.21.15.5:0/2793876797 shutdown_connections 2024-05-20T18:30:06.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.713+0000 7fae33798740 1 --2- 172.21.15.5:0/2793876797 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x56492500de40 0x564925433cc0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.713+0000 7fae33798740 1 --2- 172.21.15.5:0/2793876797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56492504f270 0x56492541b8e0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.713+0000 7fae33798740 1 --2- 172.21.15.5:0/2793876797 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56492502e590 0x56492541b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.713+0000 7fae057fa640 0 client.6345 ms_handle_reset on v2:172.21.15.63:6832/4036379300 2024-05-20T18:30:06.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.713+0000 7fae33798740 1 --2- 172.21.15.5:0/2793876797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649250263b0 0x564925435610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.713+0000 7fae33798740 1 -- 172.21.15.5:0/2793876797 >> 172.21.15.5:0/2793876797 conn(0x5649250433c0 msgr2=0x5649253e9ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:06.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.714+0000 7fae33798740 1 -- 172.21.15.5:0/2793876797 shutdown_connections 2024-05-20T18:30:06.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.714+0000 7fae33798740 1 -- 172.21.15.5:0/2793876797 wait complete. 2024-05-20T18:30:06.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.714+0000 7fae33798740 20 client.6345 trim_cache size 0 max 16384 2024-05-20T18:30:06.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.733+0000 7fae33798740 1 Processor -- start 2024-05-20T18:30:06.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.733+0000 7fae33798740 1 -- start start 2024-05-20T18:30:06.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.734+0000 7fae33798740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56492500de40 0x5649253f1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:06.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.734+0000 7fae33798740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649250263b0 0x564925437ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:06.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.734+0000 7fae33798740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56492502e590 0x564925438220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:06.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.734+0000 7fae33798740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fae282014e0 con 0x56492500de40 2024-05-20T18:30:06.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.734+0000 7fae33798740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fae281599e0 con 0x5649250263b0 2024-05-20T18:30:06.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.734+0000 7fae33798740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fae281f9d00 con 0x56492502e590 2024-05-20T18:30:06.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.734+0000 7fae2659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56492500de40 0x5649253f1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:06.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.734+0000 7fae2659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56492500de40 0x5649253f1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53110/0 (socket says 172.21.15.5:53110) 2024-05-20T18:30:06.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.734+0000 7fae2659c640 1 -- 172.21.15.5:0/2225507346 learned_addr learned my addr 172.21.15.5:0/2225507346 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:06.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.734+0000 7fae2659c640 1 -- 172.21.15.5:0/2225507346 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56492502e590 msgr2=0x564925438220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:06.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.734+0000 7fae2659c640 1 --2- 172.21.15.5:0/2225507346 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56492502e590 0x564925438220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.734+0000 7fae2659c640 1 -- 172.21.15.5:0/2225507346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649250263b0 msgr2=0x564925437ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:06.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.734+0000 7fae2659c640 1 --2- 172.21.15.5:0/2225507346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649250263b0 0x564925437ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.734+0000 7fae2659c640 1 -- 172.21.15.5:0/2225507346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649253e9220 con 0x56492500de40 2024-05-20T18:30:06.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.735+0000 7fae2659c640 1 --2- 172.21.15.5:0/2225507346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56492500de40 0x5649253f1510 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fae18016000 tx=0x7fae18055220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19809322aa75ccee server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:06.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.735+0000 7fae077fe640 1 -- 172.21.15.5:0/2225507346 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18021040 con 0x56492500de40 2024-05-20T18:30:06.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.735+0000 7fae077fe640 1 -- 172.21.15.5:0/2225507346 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae18086070 con 0x56492500de40 2024-05-20T18:30:06.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.735+0000 7fae077fe640 1 -- 172.21.15.5:0/2225507346 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18089310 con 0x56492500de40 2024-05-20T18:30:06.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.735+0000 7fae33798740 1 -- 172.21.15.5:0/2225507346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56492500de40 msgr2=0x5649253f1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:06.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.735+0000 7fae33798740 1 --2- 172.21.15.5:0/2225507346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56492500de40 0x5649253f1510 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fae18016000 tx=0x7fae18055220 comp rx=0 tx=0).stop 2024-05-20T18:30:06.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.736+0000 7fae33798740 1 -- 172.21.15.5:0/2225507346 shutdown_connections 2024-05-20T18:30:06.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.736+0000 7fae33798740 1 --2- 172.21.15.5:0/2225507346 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56492502e590 0x564925438220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.736+0000 7fae33798740 1 --2- 172.21.15.5:0/2225507346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649250263b0 0x564925437ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.736+0000 7fae33798740 1 --2- 172.21.15.5:0/2225507346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56492500de40 0x5649253f1510 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.736+0000 7fae33798740 1 -- 172.21.15.5:0/2225507346 >> 172.21.15.5:0/2225507346 conn(0x5649250433c0 msgr2=0x5649253e5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:06.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.736+0000 7fae33798740 1 -- 172.21.15.5:0/2225507346 shutdown_connections 2024-05-20T18:30:06.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.736+0000 7fae33798740 1 -- 172.21.15.5:0/2225507346 wait complete. 2024-05-20T18:30:06.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.736+0000 7fae33798740 1 -- start start 2024-05-20T18:30:06.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.736+0000 7fae33798740 1 Processor -- start 2024-05-20T18:30:06.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.738+0000 7fae33798740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56492500de40 0x5649254b5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:06.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.738+0000 7fae33798740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649250263b0 0x5649254b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:06.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.738+0000 7fae33798740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56492502e590 0x56492541b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:06.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.738+0000 7fae33798740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564925447a40 con 0x56492500de40 2024-05-20T18:30:06.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.738+0000 7fae33798740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564925435b50 con 0x5649250263b0 2024-05-20T18:30:06.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.738+0000 7fae33798740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564925448090 con 0x56492502e590 2024-05-20T18:30:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.738+0000 7fae33798740 10 client.0 _ll_register_callbacks cb 0x7fae180094f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:30:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.738+0000 7fae2659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56492500de40 0x5649254b5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.738+0000 7fae2659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56492500de40 0x5649254b5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53116/0 (socket says 172.21.15.5:53116) 2024-05-20T18:30:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.738+0000 7fae2659c640 1 -- 172.21.15.5:0/239082762 learned_addr learned my addr 172.21.15.5:0/239082762 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.738+0000 7fae25d9b640 1 --2- 172.21.15.5:0/239082762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649250263b0 0x5649254b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.738+0000 7fae26d9d640 1 --2- 172.21.15.5:0/239082762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56492502e590 0x56492541b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.738+0000 7fae2659c640 1 -- 172.21.15.5:0/239082762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56492502e590 msgr2=0x56492541b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.738+0000 7fae2659c640 1 --2- 172.21.15.5:0/239082762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56492502e590 0x56492541b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.738+0000 7fae2659c640 1 -- 172.21.15.5:0/239082762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649250263b0 msgr2=0x5649254b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.738+0000 7fae2659c640 1 --2- 172.21.15.5:0/239082762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649250263b0 0x5649254b60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.738+0000 7fae2659c640 1 -- 172.21.15.5:0/239082762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649253eeea0 con 0x56492500de40 2024-05-20T18:30:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.738+0000 7fae25d9b640 1 --2- 172.21.15.5:0/239082762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649250263b0 0x5649254b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:30:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.739+0000 7fae2659c640 1 --2- 172.21.15.5:0/239082762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56492500de40 0x5649254b5b60 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fae18016000 tx=0x7fae1806f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=429ac8907564086 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:06.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.739+0000 7fae057fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:30:06.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.739+0000 7fae057fa640 1 -- 172.21.15.5:0/239082762 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1807f060 con 0x56492500de40 2024-05-20T18:30:06.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.739+0000 7fae057fa640 1 -- 172.21.15.5:0/239082762 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7fae180555f0 con 0x56492500de40 2024-05-20T18:30:06.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.739+0000 7fae33798740 1 -- 172.21.15.5:0/239082762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae18086070 con 0x56492500de40 2024-05-20T18:30:06.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.739+0000 7fae057fa640 1 -- 172.21.15.5:0/239082762 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fae18021070 con 0x56492500de40 2024-05-20T18:30:06.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.739+0000 7fae33798740 10 client.6351 fetch_fsmap learned FSMap version 34 2024-05-20T18:30:06.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.739+0000 7fae33798740 1 -- 172.21.15.5:0/239082762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x5649253f1720 con 0x56492500de40 2024-05-20T18:30:06.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.739+0000 7fae057fa640 1 -- 172.21.15.5:0/239082762 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae18086070 con 0x56492500de40 2024-05-20T18:30:06.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.739+0000 7fae057fa640 1 -- 172.21.15.5:0/239082762 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7fae18024060 con 0x56492500de40 2024-05-20T18:30:06.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.739+0000 7fae33798740 10 client.6351 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T18:30:06.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.739+0000 7fae33798740 10 client.6351 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-20T18:30:06.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.739+0000 7fae33798740 20 client.6351 populate_metadata read hostname 'smithi005' 2024-05-20T18:30:06.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.739+0000 7fae33798740 1 --2- 172.21.15.5:0/239082762 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x564924fe3280 0x56492541b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:06.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.739+0000 7fae33798740 4 client.6351 mds_command: new command op to 6198 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T18:30:06.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.739+0000 7fae33798740 1 -- 172.21.15.5:0/239082762 --> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] -- command(tid 0: {"prefix": "status"}) -- 0x564925417390 con 0x564924fe3280 2024-05-20T18:30:06.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.739+0000 7fae25d9b640 1 --2- 172.21.15.5:0/239082762 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x564924fe3280 0x56492541b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:06.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.740+0000 7fae057fa640 10 client.6351 ms_handle_connect on v2:172.21.15.63:6832/4036379300 2024-05-20T18:30:06.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.740+0000 7fae25d9b640 1 --2- 172.21.15.5:0/239082762 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x564924fe3280 0x56492541b950 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9105609bac37c72 server_cookie=6df23e2bbdd7f418 in_seq=0 out_seq=0 2024-05-20T18:30:06.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.740+0000 7fae057fa640 1 -- 172.21.15.5:0/239082762 <== mds.0 v2:172.21.15.63:6832/4036379300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x564925417390 con 0x564924fe3280 2024-05-20T18:30:06.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.740+0000 7fae057fa640 10 client.6351 handle_command_reply: tid=0 2024-05-20T18:30:06.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.740+0000 7fae33798740 1 client.6351 shutdown 2024-05-20T18:30:06.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.740+0000 7fae33798740 1 -- 172.21.15.5:0/239082762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56492500de40 msgr2=0x5649254b5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:06.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.740+0000 7fae33798740 1 --2- 172.21.15.5:0/239082762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56492500de40 0x5649254b5b60 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fae18016000 tx=0x7fae1806f040 comp rx=0 tx=0).stop 2024-05-20T18:30:06.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.740+0000 7fae33798740 1 -- 172.21.15.5:0/239082762 shutdown_connections 2024-05-20T18:30:06.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.740+0000 7fae33798740 1 --2- 172.21.15.5:0/239082762 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x564924fe3280 0x56492541b950 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.740+0000 7fae33798740 1 --2- 172.21.15.5:0/239082762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56492502e590 0x56492541b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.740+0000 7fae33798740 1 --2- 172.21.15.5:0/239082762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5649250263b0 0x5649254b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.740+0000 7fae33798740 1 --2- 172.21.15.5:0/239082762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56492500de40 0x5649254b5b60 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.740+0000 7fae057fa640 0 client.6351 ms_handle_reset on v2:172.21.15.63:6832/4036379300 2024-05-20T18:30:06.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.740+0000 7fae33798740 1 -- 172.21.15.5:0/239082762 >> 172.21.15.5:0/239082762 conn(0x5649250433c0 msgr2=0x5649253e5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:06.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.740+0000 7fae33798740 1 -- 172.21.15.5:0/239082762 shutdown_connections 2024-05-20T18:30:06.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.740+0000 7fae33798740 1 -- 172.21.15.5:0/239082762 wait complete. 2024-05-20T18:30:06.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.741+0000 7fae33798740 20 client.6351 trim_cache size 0 max 16384 2024-05-20T18:30:06.749 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:30:06.749 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "399ab250-fd3c-47fa-a1c8-25e8149c9cac", 2024-05-20T18:30:06.749 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-20T18:30:06.750 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 6198, 2024-05-20T18:30:06.750 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-20T18:30:06.750 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-20T18:30:06.750 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-20T18:30:06.750 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 5.8342363690000001, 2024-05-20T18:30:06.750 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 34, 2024-05-20T18:30:06.750 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 63, 2024-05-20T18:30:06.750 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 63, 2024-05-20T18:30:06.750 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 11.797894701000001 2024-05-20T18:30:06.750 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:30:06.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.741+0000 7fae2c8f0640 1 -- 172.21.15.5:0/2369564861 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae0806be60 msgr2=0x7fae0806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:06.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.741+0000 7fae2c8f0640 1 --2- 172.21.15.5:0/2369564861 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae0806be60 0x7fae0806e370 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fae28159d80 tx=0x7fae1c009040 comp rx=0 tx=0).stop 2024-05-20T18:30:06.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.741+0000 7fae2c8f0640 1 -- 172.21.15.5:0/2369564861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae280b3e80 msgr2=0x7fae280c8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:06.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.741+0000 7fae2c8f0640 1 --2- 172.21.15.5:0/2369564861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae280b3e80 0x7fae280c8d10 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fae1800e870 tx=0x7fae1800ed30 comp rx=0 tx=0).stop 2024-05-20T18:30:06.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.741+0000 7fae2c8f0640 1 -- 172.21.15.5:0/2369564861 shutdown_connections 2024-05-20T18:30:06.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.741+0000 7fae2c8f0640 1 --2- 172.21.15.5:0/2369564861 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae0806be60 0x7fae0806e370 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.741+0000 7fae2c8f0640 1 --2- 172.21.15.5:0/2369564861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae280c9250 0x7fae28155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.741+0000 7fae2c8f0640 1 --2- 172.21.15.5:0/2369564861 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae28162820 0x7fae280caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.741+0000 7fae2c8f0640 1 --2- 172.21.15.5:0/2369564861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae280b3e80 0x7fae280c8d10 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.741+0000 7fae2c8f0640 1 -- 172.21.15.5:0/2369564861 >> 172.21.15.5:0/2369564861 conn(0x7fae280d4bd0 msgr2=0x7fae28161120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:06.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.741+0000 7fae2c8f0640 1 -- 172.21.15.5:0/2369564861 shutdown_connections 2024-05-20T18:30:06.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.741+0000 7fae2c8f0640 1 -- 172.21.15.5:0/2369564861 wait complete. 2024-05-20T18:30:06.753 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-20T18:30:06.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.871+0000 7f347bb38640 1 Processor -- start 2024-05-20T18:30:06.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.872+0000 7f347bb38640 1 -- start start 2024-05-20T18:30:06.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.873+0000 7f347bb38640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3474162820 0x7f34740b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.873+0000 7f347bb38640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f347406dae0 0x7f34740b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.873+0000 7f347bb38640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34740d0630 con 0x7f34740b3e80 2024-05-20T18:30:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.873+0000 7f347bb38640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f34740d39c0 con 0x7f347406dae0 2024-05-20T18:30:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.873+0000 7f347bb38640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f34740d3ca0 con 0x7f3474162820 2024-05-20T18:30:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.873+0000 7f347a0d5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f347406dae0 0x7f34740b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.873+0000 7f347a0d5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f347406dae0 0x7f34740b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53128/0 (socket says 172.21.15.5:53128) 2024-05-20T18:30:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.873+0000 7f347a0d5640 1 -- 172.21.15.5:0/862384876 learned_addr learned my addr 172.21.15.5:0/862384876 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.873+0000 7f34798d4640 1 --2- 172.21.15.5:0/862384876 >> v2:172.21.15.63:3300/0 conn(0x7f3474162820 0x7f34740b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.873+0000 7f347a0d5640 1 -- 172.21.15.5:0/862384876 >> v2:172.21.15.63:3300/0 conn(0x7f3474162820 msgr2=0x7f34740b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.873+0000 7f347a0d5640 1 --2- 172.21.15.5:0/862384876 >> v2:172.21.15.63:3300/0 conn(0x7f3474162820 0x7f34740b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.873+0000 7f347a0d5640 1 -- 172.21.15.5:0/862384876 >> v1:172.21.15.63:6789/0 conn(0x7f34740b3e80 legacy=0x7f347406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.874+0000 7f347a0d5640 1 -- 172.21.15.5:0/862384876 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34740d3f80 con 0x7f347406dae0 2024-05-20T18:30:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.874+0000 7f347a0d5640 1 --2- 172.21.15.5:0/862384876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f347406dae0 0x7f34740b8190 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f34740b7740 tx=0x7f346c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22f5b095d494346 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.874+0000 7f34788d2640 1 -- 172.21.15.5:0/862384876 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c036070 con 0x7f347406dae0 2024-05-20T18:30:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.874+0000 7f34788d2640 1 -- 172.21.15.5:0/862384876 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f346c031600 con 0x7f347406dae0 2024-05-20T18:30:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.874+0000 7f34788d2640 1 -- 172.21.15.5:0/862384876 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c0319d0 con 0x7f347406dae0 2024-05-20T18:30:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.875+0000 7f347bb38640 1 -- 172.21.15.5:0/862384876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f347406dae0 msgr2=0x7f34740b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.875+0000 7f347bb38640 1 --2- 172.21.15.5:0/862384876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f347406dae0 0x7f34740b8190 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f34740b7740 tx=0x7f346c02a730 comp rx=0 tx=0).stop 2024-05-20T18:30:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.875+0000 7f347bb38640 1 -- 172.21.15.5:0/862384876 shutdown_connections 2024-05-20T18:30:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.875+0000 7f347bb38640 1 --2- 172.21.15.5:0/862384876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f347406dae0 0x7f34740b8190 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.875+0000 7f347bb38640 1 --2- 172.21.15.5:0/862384876 >> v2:172.21.15.63:3300/0 conn(0x7f3474162820 0x7f34740b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.875+0000 7f347bb38640 1 -- 172.21.15.5:0/862384876 >> 172.21.15.5:0/862384876 conn(0x7f34740d4bd0 msgr2=0x7f34740d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.875+0000 7f347bb38640 1 -- 172.21.15.5:0/862384876 shutdown_connections 2024-05-20T18:30:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.875+0000 7f347bb38640 1 -- 172.21.15.5:0/862384876 wait complete. 2024-05-20T18:30:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.877+0000 7f347bb38640 1 Processor -- start 2024-05-20T18:30:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.877+0000 7f347bb38640 1 -- start start 2024-05-20T18:30:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.878+0000 7f347bb38640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34740b3e80 0x7f347414ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.878+0000 7f347bb38640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3474162820 0x7f3474150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.878+0000 7f347bb38640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f347414eda0 0x7f347414f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.878+0000 7f347bb38640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f34740d4780 con 0x7f3474162820 2024-05-20T18:30:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.878+0000 7f347bb38640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f34740d0300 con 0x7f34740b3e80 2024-05-20T18:30:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.878+0000 7f347bb38640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f34740d4900 con 0x7f347414eda0 2024-05-20T18:30:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.878+0000 7f34790d3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3474162820 0x7f3474150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.879+0000 7f34790d3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3474162820 0x7f3474150480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53130/0 (socket says 172.21.15.5:53130) 2024-05-20T18:30:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.879+0000 7f34790d3640 1 -- 172.21.15.5:0/1851072026 learned_addr learned my addr 172.21.15.5:0/1851072026 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.879+0000 7f34798d4640 1 --2- 172.21.15.5:0/1851072026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34740b3e80 0x7f347414ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.879+0000 7f347a0d5640 1 --2- 172.21.15.5:0/1851072026 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f347414eda0 0x7f347414f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.879+0000 7f34790d3640 1 -- 172.21.15.5:0/1851072026 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f347414eda0 msgr2=0x7f347414f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.879+0000 7f34790d3640 1 --2- 172.21.15.5:0/1851072026 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f347414eda0 0x7f347414f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.879+0000 7f34790d3640 1 -- 172.21.15.5:0/1851072026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34740b3e80 msgr2=0x7f347414ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.879+0000 7f34790d3640 1 --2- 172.21.15.5:0/1851072026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34740b3e80 0x7f347414ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.879+0000 7f34790d3640 1 -- 172.21.15.5:0/1851072026 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34740cb5d0 con 0x7f3474162820 2024-05-20T18:30:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.879+0000 7f347a0d5640 1 --2- 172.21.15.5:0/1851072026 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f347414eda0 0x7f347414f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:30:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.879+0000 7f34790d3640 1 --2- 172.21.15.5:0/1851072026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3474162820 0x7f3474150480 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f346400c3f0 tx=0x7f346400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.879+0000 7f346a7fc640 1 -- 172.21.15.5:0/1851072026 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34640052d0 con 0x7f3474162820 2024-05-20T18:30:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.879+0000 7f346a7fc640 1 -- 172.21.15.5:0/1851072026 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3464005490 con 0x7f3474162820 2024-05-20T18:30:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.879+0000 7f346a7fc640 1 -- 172.21.15.5:0/1851072026 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3464007260 con 0x7f3474162820 2024-05-20T18:30:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.879+0000 7f347bb38640 1 -- 172.21.15.5:0/1851072026 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f347414dfe0 con 0x7f3474162820 2024-05-20T18:30:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.879+0000 7f347bb38640 1 -- 172.21.15.5:0/1851072026 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f347414e5a0 con 0x7f3474162820 2024-05-20T18:30:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.880+0000 7f347bb38640 1 -- 172.21.15.5:0/1851072026 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34740caf30 con 0x7f3474162820 2024-05-20T18:30:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.881+0000 7f346a7fc640 1 -- 172.21.15.5:0/1851072026 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3464007400 con 0x7f3474162820 2024-05-20T18:30:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.881+0000 7f346a7fc640 1 --2- 172.21.15.5:0/1851072026 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f345806be30 0x7f345806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.881+0000 7f346a7fc640 1 -- 172.21.15.5:0/1851072026 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7f346408a3a0 con 0x7f3474162820 2024-05-20T18:30:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.881+0000 7f34798d4640 1 --2- 172.21.15.5:0/1851072026 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f345806be30 0x7f345806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.882+0000 7f34798d4640 1 --2- 172.21.15.5:0/1851072026 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f345806be30 0x7f345806e340 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3470009aa0 tx=0x7f3470009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:06.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:06.886+0000 7f346a7fc640 1 -- 172.21.15.5:0/1851072026 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3464058600 con 0x7f3474162820 2024-05-20T18:30:07.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:07.111+0000 7f347bb38640 1 -- 172.21.15.5:0/1851072026 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3474075b00 con 0x7f3474162820 2024-05-20T18:30:07.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:07.111+0000 7f346a7fc640 1 -- 172.21.15.5:0/1851072026 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) ==== 53+0+73 (secure 0 0 0) 0x7f346405c900 con 0x7f3474162820 2024-05-20T18:30:07.112 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-20T18:30:07.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:07.114+0000 7f347bb38640 1 -- 172.21.15.5:0/1851072026 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f345806be30 msgr2=0x7f345806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:07.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:07.114+0000 7f347bb38640 1 --2- 172.21.15.5:0/1851072026 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f345806be30 0x7f345806e340 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3470009aa0 tx=0x7f3470009190 comp rx=0 tx=0).stop 2024-05-20T18:30:07.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:07.114+0000 7f347bb38640 1 -- 172.21.15.5:0/1851072026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3474162820 msgr2=0x7f3474150480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:07.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:07.114+0000 7f347bb38640 1 --2- 172.21.15.5:0/1851072026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3474162820 0x7f3474150480 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f346400c3f0 tx=0x7f346400c8b0 comp rx=0 tx=0).stop 2024-05-20T18:30:07.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:07.114+0000 7f347bb38640 1 -- 172.21.15.5:0/1851072026 shutdown_connections 2024-05-20T18:30:07.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:07.114+0000 7f347bb38640 1 --2- 172.21.15.5:0/1851072026 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f345806be30 0x7f345806e340 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:07.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:07.114+0000 7f347bb38640 1 --2- 172.21.15.5:0/1851072026 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f347414eda0 0x7f347414f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:07.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:07.114+0000 7f347bb38640 1 --2- 172.21.15.5:0/1851072026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3474162820 0x7f3474150480 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:07.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:07.114+0000 7f347bb38640 1 --2- 172.21.15.5:0/1851072026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34740b3e80 0x7f347414ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:07.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:07.114+0000 7f347bb38640 1 -- 172.21.15.5:0/1851072026 >> 172.21.15.5:0/1851072026 conn(0x7f34740d4bd0 msgr2=0x7f34740b87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:07.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:07.114+0000 7f347bb38640 1 -- 172.21.15.5:0/1851072026 shutdown_connections 2024-05-20T18:30:07.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:07.114+0000 7f347bb38640 1 -- 172.21.15.5:0/1851072026 wait complete. 2024-05-20T18:30:07.125 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-20T18:30:07.125 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-20T18:30:07.126 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-05-20T18:30:07.126 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-20T18:30:07.126 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-20T18:30:07.126 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-20T18:30:07.126 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-20T18:30:07.126 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-20T18:30:07.127 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:30:07.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:30:07.127 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-05-20T18:30:07.148 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T18:30:07.148 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T18:30:07.149 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T18:30:07.149 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:30:07.149 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-20T18:30:07.149 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:30:07.149 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:30:07.149 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-05-20T18:30:07.149 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-05-20T18:30:07.149 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:30:07.149 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-05-20T18:30:07.149 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-05-20T18:30:07.149 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-20T18:30:07.149 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:30:07.149 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-05-20T18:30:07.150 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-20T18:30:07.150 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42491sec preferred_lft 42491sec 2024-05-20T18:30:07.150 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-05-20T18:30:07.150 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:30:07.150 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:30:07.150 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:30:07.150 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-05-20T18:30:07.150 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-20T18:30:07.150 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:30:07.150 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-20T18:30:07.150 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:30:07.150 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::4474:eaff:fe07:4f9a/64 scope link 2024-05-20T18:30:07.151 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:30:07.151 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:30:07.151 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T18:30:07.151 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ce4:f4ff:fe5e:7906/64 scope link 2024-05-20T18:30:07.151 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:30:07.151 INFO:teuthology.orchestra.run.smithi005.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:30:07.151 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether da:26:04:0e:95:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T18:30:07.151 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::d826:4ff:fe0e:950a/64 scope link 2024-05-20T18:30:07.151 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:30:07.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:30:07.152 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-20T18:30:07.215 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-20T18:30:07.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:30:07.217 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-05-20T18:30:07.274 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-05-20T18:30:07.274 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-20T18:30:07.275 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-20T18:30:07.275 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-20T18:30:07.275 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-20T18:30:07.275 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:30:07.275 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-20T18:30:07.275 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-20T18:30:07.275 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-20T18:30:07.275 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-20T18:30:07.275 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-20T18:30:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:30:07.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:30:07.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:30:07.407 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-20T18:30:07.423 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-20T18:30:07.423 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-20T18:30:07.423 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:30:07.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:30:07.423 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-05-20T18:30:07.490 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:30:07.490 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-20T18:30:07.546 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:30:07.546 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:30:07.546 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:30:07.603 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-20T18:30:07.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:30:07.605 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:30:07.699 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:30:07.699 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:30:07.699 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:30:07.710 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:30:07.710 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:30:07.710 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:30:07.710 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:30:07.710 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:30:07.717 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:30:07.717 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:30:07.717 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:30:07.717 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:30:07.717 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:30:07.717 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790" 2024-05-20T18:30:07.717 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790:/ 2024-05-20T18:30:07.717 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=399ab250-fd3c-47fa-a1c8-25e8149c9cac" will pass to kernel 2024-05-20T18:30:07.718 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-20T18:30:07.718 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-05-20T18:30:07.718 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-20T18:30:07.718 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-05-20T18:30:07.718 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-20T18:30:07.720 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-20T18:30:07.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:30:07.720 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:30:07.751 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-20T18:30:07.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T18:30:07.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:30:07.863 INFO:teuthology.orchestra.run.smithi005.stdout:client5555 2024-05-20T18:30:07.863 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:30:07.863 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:30:07.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:30:07.880 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2024-05-20T18:30:07.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:30:07.944 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2024-05-20T18:30:08.003 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:30:08.003 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:30:08.003 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:30:08.003 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:30:08.003 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2024-05-20T18:30:08.003 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:30:08.111 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627777 2024-05-20T18:30:08.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:30:08.120 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:30:08.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.275+0000 7ff774f57640 1 Processor -- start 2024-05-20T18:30:08.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.275+0000 7ff774f57640 1 -- start start 2024-05-20T18:30:08.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.276+0000 7ff774f57640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7700d1bc0 0x7ff7700d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:08.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.276+0000 7ff774f57640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7700d0370 con 0x7ff770163a10 2024-05-20T18:30:08.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.276+0000 7ff774f57640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7700d0650 con 0x7ff7700b4010 2024-05-20T18:30:08.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.276+0000 7ff774f57640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff7700d3ca0 con 0x7ff7700d1bc0 2024-05-20T18:30:08.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.276+0000 7ff76e59c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff770163a10 0x7ff7700cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44008/0 (socket says 172.21.15.5:44008) 2024-05-20T18:30:08.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.276+0000 7ff76e59c640 1 -- 172.21.15.5:0/501953837 learned_addr learned my addr 172.21.15.5:0/501953837 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:30:08.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.277+0000 7ff76dd9b640 1 -- 172.21.15.5:0/501953837 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3181265682 0 0) 0x7ff7700d0370 con 0x7ff770163a10 2024-05-20T18:30:08.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.277+0000 7ff76dd9b640 1 -- 172.21.15.5:0/501953837 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff75c003520 con 0x7ff770163a10 2024-05-20T18:30:08.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.277+0000 7ff76ed9d640 1 --2- 172.21.15.5:0/501953837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7700d1bc0 0x7ff7700d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:08.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.277+0000 7ff76dd9b640 1 -- 172.21.15.5:0/501953837 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2872710902 0 0) 0x7ff75c003520 con 0x7ff770163a10 2024-05-20T18:30:08.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.277+0000 7ff76dd9b640 1 -- 172.21.15.5:0/501953837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7700d1bc0 msgr2=0x7ff7700d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:08.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.277+0000 7ff76dd9b640 1 --2- 172.21.15.5:0/501953837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7700d1bc0 0x7ff7700d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:08.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.277+0000 7ff76dd9b640 1 -- 172.21.15.5:0/501953837 >> v1:172.21.15.63:6789/0 conn(0x7ff7700b4010 legacy=0x7ff7700d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:08.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.277+0000 7ff76dd9b640 1 -- 172.21.15.5:0/501953837 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7700d3650 con 0x7ff770163a10 2024-05-20T18:30:08.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.277+0000 7ff76dd9b640 1 -- 172.21.15.5:0/501953837 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7ff760001670 con 0x7ff770163a10 2024-05-20T18:30:08.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.277+0000 7ff76dd9b640 1 -- 172.21.15.5:0/501953837 >> v1:172.21.15.5:6789/0 conn(0x7ff770163a10 legacy=0x7ff7700cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:08.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.277+0000 7ff76dd9b640 1 --2- 172.21.15.5:0/501953837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7700d1bc0 0x7ff75c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:08.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.277+0000 7ff76dd9b640 1 --2- 172.21.15.5:0/501953837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff75c003f80 0x7ff75c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:08.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.277+0000 7ff76dd9b640 1 --2- 172.21.15.5:0/501953837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff75c004d40 0x7ff75c005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:08.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.277+0000 7ff76dd9b640 1 -- 172.21.15.5:0/501953837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff7700d3ca0 con 0x7ff75c004d40 2024-05-20T18:30:08.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.277+0000 7ff76dd9b640 1 -- 172.21.15.5:0/501953837 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c00bb80 con 0x7ff75c003f80 2024-05-20T18:30:08.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.277+0000 7ff76dd9b640 1 -- 172.21.15.5:0/501953837 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff75c00bd80 con 0x7ff7700d1bc0 2024-05-20T18:30:08.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.277+0000 7ff76ed9d640 1 --2- 172.21.15.5:0/501953837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff75c004d40 0x7ff75c005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:08.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.277+0000 7ff767fff640 1 --2- 172.21.15.5:0/501953837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff75c003f80 0x7ff75c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:08.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.277+0000 7ff76e59c640 1 --2- 172.21.15.5:0/501953837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7700d1bc0 0x7ff75c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:08.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.278+0000 7ff76ed9d640 1 -- 172.21.15.5:0/501953837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7700d1bc0 msgr2=0x7ff75c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:08.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.278+0000 7ff76ed9d640 1 --2- 172.21.15.5:0/501953837 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7700d1bc0 0x7ff75c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:08.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.278+0000 7ff76ed9d640 1 -- 172.21.15.5:0/501953837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff75c003f80 msgr2=0x7ff75c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:08.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.278+0000 7ff76ed9d640 1 --2- 172.21.15.5:0/501953837 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff75c003f80 0x7ff75c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:08.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.278+0000 7ff76ed9d640 1 -- 172.21.15.5:0/501953837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75c003520 con 0x7ff75c004d40 2024-05-20T18:30:08.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.278+0000 7ff76ed9d640 1 --2- 172.21.15.5:0/501953837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff75c004d40 0x7ff75c005170 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff768009f90 tx=0x7ff75c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8018ff20d1c9afd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:08.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.278+0000 7ff76dd9b640 1 -- 172.21.15.5:0/501953837 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff768009c10 con 0x7ff75c004d40 2024-05-20T18:30:08.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.279+0000 7ff76dd9b640 1 -- 172.21.15.5:0/501953837 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff768036070 con 0x7ff75c004d40 2024-05-20T18:30:08.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.279+0000 7ff76dd9b640 1 -- 172.21.15.5:0/501953837 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff768031850 con 0x7ff75c004d40 2024-05-20T18:30:08.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.279+0000 7ff774f57640 1 -- 172.21.15.5:0/501953837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff75c004d40 msgr2=0x7ff75c005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:08.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.279+0000 7ff774f57640 1 --2- 172.21.15.5:0/501953837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff75c004d40 0x7ff75c005170 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff768009f90 tx=0x7ff75c005830 comp rx=0 tx=0).stop 2024-05-20T18:30:08.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.279+0000 7ff76ed9d640 1 -- 172.21.15.5:0/501953837 reap_dead start 2024-05-20T18:30:08.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.279+0000 7ff774f57640 1 -- 172.21.15.5:0/501953837 shutdown_connections 2024-05-20T18:30:08.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.279+0000 7ff774f57640 1 -- 172.21.15.5:0/501953837 >> 172.21.15.5:0/501953837 conn(0x7ff77006d240 msgr2=0x7ff77006d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:08.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.279+0000 7ff774f57640 1 -- 172.21.15.5:0/501953837 shutdown_connections 2024-05-20T18:30:08.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.279+0000 7ff774f57640 1 -- 172.21.15.5:0/501953837 wait complete. 2024-05-20T18:30:08.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.281+0000 7ff774f57640 1 Processor -- start 2024-05-20T18:30:08.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.281+0000 7ff774f57640 1 -- start start 2024-05-20T18:30:08.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.283+0000 7ff774f57640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff75c004d40 0x7ff7700b6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:08.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.283+0000 7ff774f57640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7700ce040 0x7ff7701553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:08.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.283+0000 7ff774f57640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7701558e0 0x7ff7701f8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:08.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.283+0000 7ff774f57640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff7700d42b0 con 0x7ff75c004d40 2024-05-20T18:30:08.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.283+0000 7ff774f57640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff7700d4130 con 0x7ff7701558e0 2024-05-20T18:30:08.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.283+0000 7ff774f57640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff7700d4430 con 0x7ff7700ce040 2024-05-20T18:30:08.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.283+0000 7ff76e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff75c004d40 0x7ff7700b6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:08.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.283+0000 7ff76e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff75c004d40 0x7ff7700b6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53156/0 (socket says 172.21.15.5:53156) 2024-05-20T18:30:08.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.283+0000 7ff76e59c640 1 -- 172.21.15.5:0/2099837148 learned_addr learned my addr 172.21.15.5:0/2099837148 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:08.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.283+0000 7ff767fff640 1 --2- 172.21.15.5:0/2099837148 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7700ce040 0x7ff7701553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:08.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.283+0000 7ff76ed9d640 1 --2- 172.21.15.5:0/2099837148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7701558e0 0x7ff7701f8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:08.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.283+0000 7ff76e59c640 1 -- 172.21.15.5:0/2099837148 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7700ce040 msgr2=0x7ff7701553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:08.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.283+0000 7ff76e59c640 1 --2- 172.21.15.5:0/2099837148 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7700ce040 0x7ff7701553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:08.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.283+0000 7ff76e59c640 1 -- 172.21.15.5:0/2099837148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7701558e0 msgr2=0x7ff7701f8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:08.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.283+0000 7ff76e59c640 1 --2- 172.21.15.5:0/2099837148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7701558e0 0x7ff7701f8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:08.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.283+0000 7ff76e59c640 1 -- 172.21.15.5:0/2099837148 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff770155cf0 con 0x7ff75c004d40 2024-05-20T18:30:08.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.283+0000 7ff76e59c640 1 --2- 172.21.15.5:0/2099837148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff75c004d40 0x7ff7700b6cb0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ff7600072c0 tx=0x7ff760007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:08.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.284+0000 7ff766ffd640 1 -- 172.21.15.5:0/2099837148 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76000f960 con 0x7ff75c004d40 2024-05-20T18:30:08.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.284+0000 7ff766ffd640 1 -- 172.21.15.5:0/2099837148 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff760018070 con 0x7ff75c004d40 2024-05-20T18:30:08.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.284+0000 7ff774f57640 1 -- 172.21.15.5:0/2099837148 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7700b5c10 con 0x7ff75c004d40 2024-05-20T18:30:08.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.284+0000 7ff766ffd640 1 -- 172.21.15.5:0/2099837148 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76000fd00 con 0x7ff75c004d40 2024-05-20T18:30:08.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.284+0000 7ff774f57640 1 -- 172.21.15.5:0/2099837148 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7700b61d0 con 0x7ff75c004d40 2024-05-20T18:30:08.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.285+0000 7ff774f57640 1 -- 172.21.15.5:0/2099837148 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff77006d650 con 0x7ff75c004d40 2024-05-20T18:30:08.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.285+0000 7ff766ffd640 1 -- 172.21.15.5:0/2099837148 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff76001d460 con 0x7ff75c004d40 2024-05-20T18:30:08.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.286+0000 7ff766ffd640 1 --2- 172.21.15.5:0/2099837148 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff75006be60 0x7ff75006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:08.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.286+0000 7ff767fff640 1 --2- 172.21.15.5:0/2099837148 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff75006be60 0x7ff75006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:08.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.286+0000 7ff766ffd640 1 -- 172.21.15.5:0/2099837148 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7ff76008afb0 con 0x7ff75c004d40 2024-05-20T18:30:08.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.286+0000 7ff767fff640 1 --2- 172.21.15.5:0/2099837148 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff75006be60 0x7ff75006e370 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff77015d640 tx=0x7ff758009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.290+0000 7ff766ffd640 1 -- 172.21.15.5:0/2099837148 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff77006d650 con 0x7ff75c004d40 2024-05-20T18:30:08.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.514+0000 7ff774f57640 1 -- 172.21.15.5:0/2099837148 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff770075b00 con 0x7ff75c004d40 2024-05-20T18:30:08.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.514+0000 7ff766ffd640 1 -- 172.21.15.5:0/2099837148 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5176 (secure 0 0 0) 0x7ff7600590f0 con 0x7ff75c004d40 2024-05-20T18:30:08.514 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:30:08.515 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5480,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/4292435960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":4292435960},{"type":"v1","addr":"172.21.15.63:6835","nonce":4292435960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6186,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2991834087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2991834087},{"type":"v1","addr":"172.21.15.5:6836","nonce":2991834087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6195,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3462206198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":3462206198},{"type":"v1","addr":"172.21.15.5:6837","nonce":3462206198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:30:00.286575+0000","modified":"2024-05-20T18:30:03.297495+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6198":{"gid":6198,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6834/4036379300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":4036379300},{"type":"v1","addr":"172.21.15.63:6834","nonce":4036379300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6198,"qdb_cluster":[6198]},"id":4}]} 2024-05-20T18:30:08.515 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 34 2024-05-20T18:30:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.516+0000 7ff774f57640 1 -- 172.21.15.5:0/2099837148 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff75006be60 msgr2=0x7ff75006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.516+0000 7ff774f57640 1 --2- 172.21.15.5:0/2099837148 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff75006be60 0x7ff75006e370 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff77015d640 tx=0x7ff758009040 comp rx=0 tx=0).stop 2024-05-20T18:30:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.516+0000 7ff774f57640 1 -- 172.21.15.5:0/2099837148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff75c004d40 msgr2=0x7ff7700b6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.516+0000 7ff774f57640 1 --2- 172.21.15.5:0/2099837148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff75c004d40 0x7ff7700b6cb0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ff7600072c0 tx=0x7ff760007780 comp rx=0 tx=0).stop 2024-05-20T18:30:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.516+0000 7ff774f57640 1 -- 172.21.15.5:0/2099837148 shutdown_connections 2024-05-20T18:30:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.516+0000 7ff774f57640 1 --2- 172.21.15.5:0/2099837148 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff75006be60 0x7ff75006e370 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.516+0000 7ff774f57640 1 --2- 172.21.15.5:0/2099837148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7701558e0 0x7ff7701f8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.516+0000 7ff774f57640 1 --2- 172.21.15.5:0/2099837148 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7700ce040 0x7ff7701553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.516+0000 7ff774f57640 1 --2- 172.21.15.5:0/2099837148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff75c004d40 0x7ff7700b6cb0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.516+0000 7ff774f57640 1 -- 172.21.15.5:0/2099837148 >> 172.21.15.5:0/2099837148 conn(0x7ff77006d240 msgr2=0x7ff77015fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.517+0000 7ff774f57640 1 -- 172.21.15.5:0/2099837148 shutdown_connections 2024-05-20T18:30:08.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.517+0000 7ff774f57640 1 -- 172.21.15.5:0/2099837148 wait complete. 2024-05-20T18:30:08.528 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-05-20T18:30:08.690 INFO:teuthology.orchestra.run.smithi063.stdout:{"message":"","return_code":0} 2024-05-20T18:30:08.690 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-20T18:30:08.690 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-05-20T18:30:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.747+0000 7f52f3063f40 1 Processor -- start 2024-05-20T18:30:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.747+0000 7f52f3063f40 1 -- start start 2024-05-20T18:30:08.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.749+0000 7f52f3063f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564cb2758970 0x564cb27e2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:08.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.749+0000 7f52f3063f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cb275a5f0 0x564cb270f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:08.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.749+0000 7f52f3063f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564cb27e9010 0x564cb270f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:08.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.749+0000 7f52f3063f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564cb26b8c80 con 0x564cb275a5f0 2024-05-20T18:30:08.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.749+0000 7f52f3063f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564cb26bccc0 con 0x564cb27e9010 2024-05-20T18:30:08.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.749+0000 7f52f3063f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564cb2740550 con 0x564cb2758970 2024-05-20T18:30:08.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.749+0000 7f52e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cb275a5f0 0x564cb270f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:08.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.749+0000 7f52e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cb275a5f0 0x564cb270f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53164/0 (socket says 172.21.15.5:53164) 2024-05-20T18:30:08.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.749+0000 7f52e7fff640 1 -- 172.21.15.5:0/336403438 learned_addr learned my addr 172.21.15.5:0/336403438 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:08.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.749+0000 7f52ed333640 1 --2- 172.21.15.5:0/336403438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564cb27e9010 0x564cb270f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:08.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.749+0000 7f52ecb32640 1 --2- 172.21.15.5:0/336403438 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564cb2758970 0x564cb27e2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:08.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.749+0000 7f52e7fff640 1 -- 172.21.15.5:0/336403438 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564cb2758970 msgr2=0x564cb27e2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:08.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.749+0000 7f52e7fff640 1 --2- 172.21.15.5:0/336403438 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564cb2758970 0x564cb27e2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:08.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.749+0000 7f52e7fff640 1 -- 172.21.15.5:0/336403438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564cb27e9010 msgr2=0x564cb270f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:08.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.749+0000 7f52e7fff640 1 --2- 172.21.15.5:0/336403438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564cb27e9010 0x564cb270f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:08.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.749+0000 7f52e7fff640 1 -- 172.21.15.5:0/336403438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cb273d4c0 con 0x564cb275a5f0 2024-05-20T18:30:08.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.749+0000 7f52e7fff640 1 --2- 172.21.15.5:0/336403438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cb275a5f0 0x564cb270f0c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f52d800a460 tx=0x7f52d800a920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9d84ac2c6eb772 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:08.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.749+0000 7f52e6ffd640 1 -- 172.21.15.5:0/336403438 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d800b920 con 0x564cb275a5f0 2024-05-20T18:30:08.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.749+0000 7f52e6ffd640 1 -- 172.21.15.5:0/336403438 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52d800f070 con 0x564cb275a5f0 2024-05-20T18:30:08.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.750+0000 7f52e6ffd640 1 -- 172.21.15.5:0/336403438 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d800bd20 con 0x564cb275a5f0 2024-05-20T18:30:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.750+0000 7f52f3063f40 1 -- 172.21.15.5:0/336403438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cb275a5f0 msgr2=0x564cb270f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.750+0000 7f52f3063f40 1 --2- 172.21.15.5:0/336403438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cb275a5f0 0x564cb270f0c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f52d800a460 tx=0x7f52d800a920 comp rx=0 tx=0).stop 2024-05-20T18:30:08.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.750+0000 7f52f3063f40 1 -- 172.21.15.5:0/336403438 shutdown_connections 2024-05-20T18:30:08.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.750+0000 7f52f3063f40 1 --2- 172.21.15.5:0/336403438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564cb27e9010 0x564cb270f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:08.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.750+0000 7f52f3063f40 1 --2- 172.21.15.5:0/336403438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cb275a5f0 0x564cb270f0c0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:08.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.750+0000 7f52f3063f40 1 --2- 172.21.15.5:0/336403438 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564cb2758970 0x564cb27e2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:08.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.750+0000 7f52f3063f40 1 -- 172.21.15.5:0/336403438 >> 172.21.15.5:0/336403438 conn(0x564cb2636ac0 msgr2=0x564cb27e5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:08.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.750+0000 7f52f3063f40 1 -- 172.21.15.5:0/336403438 shutdown_connections 2024-05-20T18:30:08.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.750+0000 7f52f3063f40 1 -- 172.21.15.5:0/336403438 wait complete. 2024-05-20T18:30:08.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.751+0000 7f52f3063f40 1 Processor -- start 2024-05-20T18:30:08.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.751+0000 7f52f3063f40 1 -- start start 2024-05-20T18:30:08.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.752+0000 7f52f3063f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564cb2758970 0x564cb270f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:08.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.752+0000 7f52f3063f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564cb275a5f0 0x564cb27df470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:08.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.752+0000 7f52f3063f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cb27e9010 0x564cb27df9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.752+0000 7f52f3063f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564cb273c8c0 con 0x564cb27e9010 2024-05-20T18:30:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.752+0000 7f52f3063f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564cb2877820 con 0x564cb2758970 2024-05-20T18:30:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.752+0000 7f52f3063f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564cb27e2c40 con 0x564cb275a5f0 2024-05-20T18:30:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.752+0000 7f52ed333640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cb27e9010 0x564cb27df9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.753+0000 7f52ed333640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cb27e9010 0x564cb27df9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53168/0 (socket says 172.21.15.5:53168) 2024-05-20T18:30:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.753+0000 7f52ed333640 1 -- 172.21.15.5:0/2410792053 learned_addr learned my addr 172.21.15.5:0/2410792053 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.753+0000 7f52e7fff640 1 --2- 172.21.15.5:0/2410792053 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564cb275a5f0 0x564cb27df470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.753+0000 7f52ecb32640 1 --2- 172.21.15.5:0/2410792053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564cb2758970 0x564cb270f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.753+0000 7f52ed333640 1 -- 172.21.15.5:0/2410792053 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564cb275a5f0 msgr2=0x564cb27df470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.753+0000 7f52ed333640 1 --2- 172.21.15.5:0/2410792053 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564cb275a5f0 0x564cb27df470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.753+0000 7f52ed333640 1 -- 172.21.15.5:0/2410792053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564cb2758970 msgr2=0x564cb270f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.753+0000 7f52ed333640 1 --2- 172.21.15.5:0/2410792053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564cb2758970 0x564cb270f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.753+0000 7f52ed333640 1 -- 172.21.15.5:0/2410792053 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cb27e50b0 con 0x564cb27e9010 2024-05-20T18:30:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.753+0000 7f52ed333640 1 --2- 172.21.15.5:0/2410792053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cb27e9010 0x564cb27df9b0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f52e8003eb0 tx=0x7f52e8001830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:08.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.753+0000 7f52e5ffb640 1 -- 172.21.15.5:0/2410792053 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e80123f0 con 0x564cb27e9010 2024-05-20T18:30:08.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.753+0000 7f52e5ffb640 1 -- 172.21.15.5:0/2410792053 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52e8017070 con 0x564cb27e9010 2024-05-20T18:30:08.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.753+0000 7f52e5ffb640 1 -- 172.21.15.5:0/2410792053 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e80127b0 con 0x564cb27e9010 2024-05-20T18:30:08.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.753+0000 7f52f3063f40 1 -- 172.21.15.5:0/2410792053 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564cb27e3270 con 0x564cb27e9010 2024-05-20T18:30:08.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.753+0000 7f52f3063f40 1 -- 172.21.15.5:0/2410792053 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cb27e3830 con 0x564cb27e9010 2024-05-20T18:30:08.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.755+0000 7f52e5ffb640 1 -- 172.21.15.5:0/2410792053 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f52e8012950 con 0x564cb27e9010 2024-05-20T18:30:08.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.755+0000 7f52e5ffb640 1 --2- 172.21.15.5:0/2410792053 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f52d006a810 0x7f52d006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:08.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.755+0000 7f52e5ffb640 1 -- 172.21.15.5:0/2410792053 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7f52e8089a10 con 0x564cb27e9010 2024-05-20T18:30:08.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.756+0000 7f52ecb32640 1 --2- 172.21.15.5:0/2410792053 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f52d006a810 0x7f52d006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:08.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.756+0000 7f52f3063f40 1 --2- 172.21.15.5:0/2410792053 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x564cb27e2720 0x564cb27e4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:08.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.756+0000 7f52f3063f40 1 -- 172.21.15.5:0/2410792053 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63) -- 0x564cb270f570 con 0x564cb27e2720 2024-05-20T18:30:08.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.756+0000 7f52e7fff640 1 --2- 172.21.15.5:0/2410792053 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x564cb27e2720 0x564cb27e4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:08.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.756+0000 7f52ecb32640 1 --2- 172.21.15.5:0/2410792053 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f52d006a810 0x7f52d006cd20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f52e0057000 tx=0x7f52e0052000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:08.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.757+0000 7f52e7fff640 1 --2- 172.21.15.5:0/2410792053 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x564cb27e2720 0x564cb27e4030 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:08.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.757+0000 7f52e7fff640 1 -- 172.21.15.5:0/2410792053 <== osd.6 v2:172.21.15.63:6816/3384565140 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) ==== 164+0+94 (crc 0 0 0) 0x7f52d800c8d0 con 0x564cb27e2720 2024-05-20T18:30:08.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.757+0000 7f52f3063f40 1 -- 172.21.15.5:0/2410792053 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x564cb27e2720 msgr2=0x564cb27e4030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:08.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.757+0000 7f52f3063f40 1 --2- 172.21.15.5:0/2410792053 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x564cb27e2720 0x564cb27e4030 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:08.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.757+0000 7f52f3063f40 1 -- 172.21.15.5:0/2410792053 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f52d006a810 msgr2=0x7f52d006cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:08.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.757+0000 7f52f3063f40 1 --2- 172.21.15.5:0/2410792053 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f52d006a810 0x7f52d006cd20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f52e0057000 tx=0x7f52e0052000 comp rx=0 tx=0).stop 2024-05-20T18:30:08.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.757+0000 7f52f3063f40 1 -- 172.21.15.5:0/2410792053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cb27e9010 msgr2=0x564cb27df9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:08.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.757+0000 7f52f3063f40 1 --2- 172.21.15.5:0/2410792053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cb27e9010 0x564cb27df9b0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f52e8003eb0 tx=0x7f52e8001830 comp rx=0 tx=0).stop 2024-05-20T18:30:08.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.757+0000 7f52ed333640 1 -- 172.21.15.5:0/2410792053 reap_dead start 2024-05-20T18:30:08.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.757+0000 7f52f3063f40 1 -- 172.21.15.5:0/2410792053 shutdown_connections 2024-05-20T18:30:08.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.757+0000 7f52f3063f40 1 -- 172.21.15.5:0/2410792053 >> 172.21.15.5:0/2410792053 conn(0x564cb2636ac0 msgr2=0x564cb27e5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:08.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.757+0000 7f52f3063f40 1 -- 172.21.15.5:0/2410792053 shutdown_connections 2024-05-20T18:30:08.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.757+0000 7f52f3063f40 1 -- 172.21.15.5:0/2410792053 wait complete. 2024-05-20T18:30:08.760 INFO:teuthology.orchestra.run.smithi005.stdout:Xfiledir_test 2024-05-20T18:30:08.760 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:30:08.761 DEBUG:teuthology.orchestra.run.smithi005:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-20T18:30:08.818 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:30:08.818 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 1099511627777, 2024-05-20T18:30:08.818 INFO:teuthology.orchestra.run.smithi005.stdout: "ancestors": [ 2024-05-20T18:30:08.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:30:08.818 INFO:teuthology.orchestra.run.smithi005.stdout: "dirino": 1099511627776, 2024-05-20T18:30:08.818 INFO:teuthology.orchestra.run.smithi005.stdout: "dname": "file", 2024-05-20T18:30:08.818 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 2 2024-05-20T18:30:08.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:30:08.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:30:08.818 INFO:teuthology.orchestra.run.smithi005.stdout: "dirino": 1, 2024-05-20T18:30:08.819 INFO:teuthology.orchestra.run.smithi005.stdout: "dname": "dir_test", 2024-05-20T18:30:08.819 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 4 2024-05-20T18:30:08.819 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:30:08.819 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-20T18:30:08.819 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 10, 2024-05-20T18:30:08.819 INFO:teuthology.orchestra.run.smithi005.stdout: "old_pools": [] 2024-05-20T18:30:08.819 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:30:08.819 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:30:08.821 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2024-05-20T18:30:08.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.919+0000 7f344c984f40 1 Processor -- start 2024-05-20T18:30:08.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.919+0000 7f344c984f40 1 -- start start 2024-05-20T18:30:08.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.921+0000 7f344c984f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ccd690460 0x563ccd734870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:08.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.921+0000 7f344c984f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563ccd73e970 0x563ccd6ae640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:08.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.921+0000 7f344c984f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ccd73f020 0x563ccd6aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:08.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.921+0000 7f344c984f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563ccd60dc80 con 0x563ccd73f020 2024-05-20T18:30:08.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.921+0000 7f344c984f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563ccd611cc0 con 0x563ccd690460 2024-05-20T18:30:08.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.921+0000 7f344c984f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x563ccd736860 con 0x563ccd73e970 2024-05-20T18:30:08.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.921+0000 7f3446c54640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ccd73f020 0x563ccd6aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:08.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.921+0000 7f3446c54640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ccd73f020 0x563ccd6aeb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53188/0 (socket says 172.21.15.5:53188) 2024-05-20T18:30:08.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.921+0000 7f343ffff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563ccd73e970 0x563ccd6ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:08.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.921+0000 7f3446c54640 1 -- 172.21.15.5:0/854640645 learned_addr learned my addr 172.21.15.5:0/854640645 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:08.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.921+0000 7f3446453640 1 --2- 172.21.15.5:0/854640645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ccd690460 0x563ccd734870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:08.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.921+0000 7f3446c54640 1 -- 172.21.15.5:0/854640645 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563ccd73e970 msgr2=0x563ccd6ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:08.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.921+0000 7f3446c54640 1 --2- 172.21.15.5:0/854640645 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563ccd73e970 0x563ccd6ae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:08.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.921+0000 7f3446c54640 1 -- 172.21.15.5:0/854640645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ccd690460 msgr2=0x563ccd734870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:08.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.921+0000 7f3446c54640 1 --2- 172.21.15.5:0/854640645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ccd690460 0x563ccd734870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:08.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.921+0000 7f3446c54640 1 -- 172.21.15.5:0/854640645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ccd6ad6e0 con 0x563ccd73f020 2024-05-20T18:30:08.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.921+0000 7f3446c54640 1 --2- 172.21.15.5:0/854640645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ccd73f020 0x563ccd6aeb80 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f3440001590 tx=0x7f3440001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9b76b166b098a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:08.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.922+0000 7f3445451640 1 -- 172.21.15.5:0/854640645 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3440017070 con 0x563ccd73f020 2024-05-20T18:30:08.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.922+0000 7f3445451640 1 -- 172.21.15.5:0/854640645 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34400026b0 con 0x563ccd73f020 2024-05-20T18:30:08.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.922+0000 7f3445451640 1 -- 172.21.15.5:0/854640645 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3440002aa0 con 0x563ccd73f020 2024-05-20T18:30:08.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.922+0000 7f344c984f40 1 -- 172.21.15.5:0/854640645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ccd73f020 msgr2=0x563ccd6aeb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:08.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.922+0000 7f344c984f40 1 --2- 172.21.15.5:0/854640645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ccd73f020 0x563ccd6aeb80 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f3440001590 tx=0x7f3440001a50 comp rx=0 tx=0).stop 2024-05-20T18:30:08.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.922+0000 7f344c984f40 1 -- 172.21.15.5:0/854640645 shutdown_connections 2024-05-20T18:30:08.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.922+0000 7f344c984f40 1 --2- 172.21.15.5:0/854640645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ccd73f020 0x563ccd6aeb80 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:08.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.922+0000 7f344c984f40 1 --2- 172.21.15.5:0/854640645 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563ccd73e970 0x563ccd6ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:08.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.922+0000 7f344c984f40 1 --2- 172.21.15.5:0/854640645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ccd690460 0x563ccd734870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:08.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.922+0000 7f344c984f40 1 -- 172.21.15.5:0/854640645 >> 172.21.15.5:0/854640645 conn(0x563ccd58bac0 msgr2=0x563ccd73d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:08.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.922+0000 7f344c984f40 1 -- 172.21.15.5:0/854640645 shutdown_connections 2024-05-20T18:30:08.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.922+0000 7f344c984f40 1 -- 172.21.15.5:0/854640645 wait complete. 2024-05-20T18:30:08.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.923+0000 7f344c984f40 1 Processor -- start 2024-05-20T18:30:08.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.923+0000 7f344c984f40 1 -- start start 2024-05-20T18:30:08.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.925+0000 7f344c984f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ccd690460 0x563ccd72bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:08.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.925+0000 7f344c984f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563ccd73e970 0x563ccd732630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:08.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.925+0000 7f344c984f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ccd73f020 0x563ccd732b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:08.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.925+0000 7f344c984f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563ccd695bc0 con 0x563ccd690460 2024-05-20T18:30:08.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.925+0000 7f344c984f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563ccd6af7f0 con 0x563ccd73f020 2024-05-20T18:30:08.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.925+0000 7f344c984f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x563ccd734db0 con 0x563ccd73e970 2024-05-20T18:30:08.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.925+0000 7f3446453640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ccd690460 0x563ccd72bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:08.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.925+0000 7f3446453640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ccd690460 0x563ccd72bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53196/0 (socket says 172.21.15.5:53196) 2024-05-20T18:30:08.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.925+0000 7f3446453640 1 -- 172.21.15.5:0/891174294 learned_addr learned my addr 172.21.15.5:0/891174294 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:08.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.925+0000 7f3446c54640 1 --2- 172.21.15.5:0/891174294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ccd73f020 0x563ccd732b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:08.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.925+0000 7f343ffff640 1 --2- 172.21.15.5:0/891174294 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563ccd73e970 0x563ccd732630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:08.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.925+0000 7f3446453640 1 -- 172.21.15.5:0/891174294 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563ccd73e970 msgr2=0x563ccd732630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:08.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.925+0000 7f3446453640 1 --2- 172.21.15.5:0/891174294 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563ccd73e970 0x563ccd732630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:08.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.925+0000 7f3446453640 1 -- 172.21.15.5:0/891174294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ccd73f020 msgr2=0x563ccd732b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:08.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.925+0000 7f3446453640 1 --2- 172.21.15.5:0/891174294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563ccd73f020 0x563ccd732b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:08.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.925+0000 7f3446453640 1 -- 172.21.15.5:0/891174294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ccd6aa520 con 0x563ccd690460 2024-05-20T18:30:08.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.925+0000 7f3446453640 1 --2- 172.21.15.5:0/891174294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ccd690460 0x563ccd72bc50 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f343800efd0 tx=0x7f343800c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:08.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.925+0000 7f343f7fe640 1 -- 172.21.15.5:0/891174294 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343800f650 con 0x563ccd690460 2024-05-20T18:30:08.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.926+0000 7f343f7fe640 1 -- 172.21.15.5:0/891174294 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3438013030 con 0x563ccd690460 2024-05-20T18:30:08.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.926+0000 7f344c984f40 1 -- 172.21.15.5:0/891174294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563ccd736550 con 0x563ccd690460 2024-05-20T18:30:08.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.926+0000 7f343f7fe640 1 -- 172.21.15.5:0/891174294 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343800fa70 con 0x563ccd690460 2024-05-20T18:30:08.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.926+0000 7f344c984f40 1 -- 172.21.15.5:0/891174294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ccd737b50 con 0x563ccd690460 2024-05-20T18:30:08.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.927+0000 7f343f7fe640 1 -- 172.21.15.5:0/891174294 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f343800fc10 con 0x563ccd690460 2024-05-20T18:30:08.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.928+0000 7f343f7fe640 1 --2- 172.21.15.5:0/891174294 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f342806a840 0x7f342806cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:08.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.928+0000 7f343ffff640 1 --2- 172.21.15.5:0/891174294 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f342806a840 0x7f342806cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:08.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.928+0000 7f343f7fe640 1 -- 172.21.15.5:0/891174294 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7f343808a350 con 0x563ccd690460 2024-05-20T18:30:08.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.928+0000 7f344c984f40 1 --2- 172.21.15.5:0/891174294 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x563ccd695760 0x563ccd7e0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:08.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.928+0000 7f344c984f40 1 -- 172.21.15.5:0/891174294 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63) -- 0x563ccd7e14b0 con 0x563ccd695760 2024-05-20T18:30:08.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.928+0000 7f343ffff640 1 --2- 172.21.15.5:0/891174294 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f342806a840 0x7f342806cd50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3430057000 tx=0x7f3430052000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:08.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.928+0000 7f3446c54640 1 --2- 172.21.15.5:0/891174294 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x563ccd695760 0x563ccd7e0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:08.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.929+0000 7f3446c54640 1 --2- 172.21.15.5:0/891174294 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x563ccd695760 0x563ccd7e0f70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:08.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.930+0000 7f3446c54640 1 -- 172.21.15.5:0/891174294 <== osd.6 v2:172.21.15.63:6816/3384565140 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v63'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f344000f070 con 0x563ccd695760 2024-05-20T18:30:08.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.930+0000 7f344c984f40 1 -- 172.21.15.5:0/891174294 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x563ccd695760 msgr2=0x563ccd7e0f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:08.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.930+0000 7f344c984f40 1 --2- 172.21.15.5:0/891174294 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x563ccd695760 0x563ccd7e0f70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:08.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.930+0000 7f344c984f40 1 -- 172.21.15.5:0/891174294 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f342806a840 msgr2=0x7f342806cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:08.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.930+0000 7f344c984f40 1 --2- 172.21.15.5:0/891174294 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f342806a840 0x7f342806cd50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3430057000 tx=0x7f3430052000 comp rx=0 tx=0).stop 2024-05-20T18:30:08.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.930+0000 7f344c984f40 1 -- 172.21.15.5:0/891174294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ccd690460 msgr2=0x563ccd72bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:08.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.930+0000 7f344c984f40 1 --2- 172.21.15.5:0/891174294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ccd690460 0x563ccd72bc50 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f343800efd0 tx=0x7f343800c730 comp rx=0 tx=0).stop 2024-05-20T18:30:08.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.930+0000 7f3446c54640 1 -- 172.21.15.5:0/891174294 reap_dead start 2024-05-20T18:30:08.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.930+0000 7f344c984f40 1 -- 172.21.15.5:0/891174294 shutdown_connections 2024-05-20T18:30:08.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.930+0000 7f344c984f40 1 -- 172.21.15.5:0/891174294 >> 172.21.15.5:0/891174294 conn(0x563ccd58bac0 msgr2=0x563ccd73d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:08.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.930+0000 7f344c984f40 1 -- 172.21.15.5:0/891174294 shutdown_connections 2024-05-20T18:30:08.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:08.930+0000 7f344c984f40 1 -- 172.21.15.5:0/891174294 wait complete. 2024-05-20T18:30:08.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:30:08.933 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2024-05-20T18:30:09.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.053+0000 7f0931b65640 1 Processor -- start 2024-05-20T18:30:09.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.053+0000 7f0931b65640 1 -- start start 2024-05-20T18:30:09.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.054+0000 7f0931b65640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f092c0b3860 0x7f092c0d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.054+0000 7f0931b65640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f092c0d0450 con 0x7f092c06d870 2024-05-20T18:30:09.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.054+0000 7f0931b65640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f092c0d3b10 con 0x7f092c08bd20 2024-05-20T18:30:09.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.054+0000 7f0931b65640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f092c0d3df0 con 0x7f092c0b3860 2024-05-20T18:30:09.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.054+0000 7f092affd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f092c06d870 0x7f092c15f230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44030/0 (socket says 172.21.15.5:44030) 2024-05-20T18:30:09.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.054+0000 7f092affd640 1 -- 172.21.15.5:0/3139556975 learned_addr learned my addr 172.21.15.5:0/3139556975 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:30:09.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.055+0000 7f092b7fe640 1 --2- 172.21.15.5:0/3139556975 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f092c0b3860 0x7f092c0d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.055+0000 7f092a7fc640 1 -- 172.21.15.5:0/3139556975 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3095168490 0 0) 0x7f092c0d0450 con 0x7f092c06d870 2024-05-20T18:30:09.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.055+0000 7f092a7fc640 1 -- 172.21.15.5:0/3139556975 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0914003520 con 0x7f092c06d870 2024-05-20T18:30:09.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.055+0000 7f092a7fc640 1 -- 172.21.15.5:0/3139556975 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1868556103 0 0) 0x7f092c0d3b10 con 0x7f092c08bd20 2024-05-20T18:30:09.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.055+0000 7f092a7fc640 1 -- 172.21.15.5:0/3139556975 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f092c0d0450 con 0x7f092c08bd20 2024-05-20T18:30:09.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.055+0000 7f092a7fc640 1 -- 172.21.15.5:0/3139556975 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3559974179 0 0) 0x7f0914003520 con 0x7f092c06d870 2024-05-20T18:30:09.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.055+0000 7f092b7fe640 1 -- 172.21.15.5:0/3139556975 >> v1:172.21.15.63:6789/0 conn(0x7f092c08bd20 legacy=0x7f092c08c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:09.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.055+0000 7f092b7fe640 1 -- 172.21.15.5:0/3139556975 >> v1:172.21.15.5:6789/0 conn(0x7f092c06d870 legacy=0x7f092c15f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:09.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.055+0000 7f092b7fe640 1 -- 172.21.15.5:0/3139556975 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f092c0d40d0 con 0x7f092c0b3860 2024-05-20T18:30:09.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.056+0000 7f092b7fe640 1 --2- 172.21.15.5:0/3139556975 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f092c0b3860 0x7f092c0d45f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f091c0097d0 tx=0x7f091c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afbc8f0d49b803db server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:09.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.056+0000 7f092a7fc640 1 -- 172.21.15.5:0/3139556975 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c037070 con 0x7f092c0b3860 2024-05-20T18:30:09.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.056+0000 7f092a7fc640 1 -- 172.21.15.5:0/3139556975 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f091c02ab90 con 0x7f092c0b3860 2024-05-20T18:30:09.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.056+0000 7f092a7fc640 1 -- 172.21.15.5:0/3139556975 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0920001670 con 0x7f092c0b3860 2024-05-20T18:30:09.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.057+0000 7f0931b65640 1 -- 172.21.15.5:0/3139556975 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f092c0b3860 msgr2=0x7f092c0d45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:09.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.057+0000 7f0931b65640 1 --2- 172.21.15.5:0/3139556975 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f092c0b3860 0x7f092c0d45f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f091c0097d0 tx=0x7f091c02a100 comp rx=0 tx=0).stop 2024-05-20T18:30:09.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.057+0000 7f0931b65640 1 -- 172.21.15.5:0/3139556975 shutdown_connections 2024-05-20T18:30:09.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.057+0000 7f0931b65640 1 --2- 172.21.15.5:0/3139556975 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f092c0b3860 0x7f092c0d45f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.057+0000 7f0931b65640 1 -- 172.21.15.5:0/3139556975 >> 172.21.15.5:0/3139556975 conn(0x7f092c0b6d10 msgr2=0x7f092c06d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:09.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.057+0000 7f0931b65640 1 -- 172.21.15.5:0/3139556975 shutdown_connections 2024-05-20T18:30:09.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.057+0000 7f0931b65640 1 -- 172.21.15.5:0/3139556975 wait complete. 2024-05-20T18:30:09.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.059+0000 7f0931b65640 1 Processor -- start 2024-05-20T18:30:09.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.059+0000 7f0931b65640 1 -- start start 2024-05-20T18:30:09.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.060+0000 7f0931b65640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f092c06d870 0x7f092c14fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.060+0000 7f0931b65640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f092c08bd20 0x7f092c14e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.060+0000 7f0931b65640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f092c0b3860 0x7f092c1540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.060+0000 7f0931b65640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f092c16a0f0 con 0x7f092c06d870 2024-05-20T18:30:09.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.060+0000 7f0931b65640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f092c169f70 con 0x7f092c08bd20 2024-05-20T18:30:09.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.060+0000 7f0931b65640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f092c16a270 con 0x7f092c0b3860 2024-05-20T18:30:09.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.060+0000 7f092b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f092c06d870 0x7f092c14fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.060+0000 7f092b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f092c06d870 0x7f092c14fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53210/0 (socket says 172.21.15.5:53210) 2024-05-20T18:30:09.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.060+0000 7f092b7fe640 1 -- 172.21.15.5:0/811153783 learned_addr learned my addr 172.21.15.5:0/811153783 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:09.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.060+0000 7f092bfff640 1 --2- 172.21.15.5:0/811153783 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f092c0b3860 0x7f092c1540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.060+0000 7f092affd640 1 --2- 172.21.15.5:0/811153783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f092c08bd20 0x7f092c14e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.061+0000 7f092b7fe640 1 -- 172.21.15.5:0/811153783 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f092c0b3860 msgr2=0x7f092c1540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:09.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.061+0000 7f092b7fe640 1 --2- 172.21.15.5:0/811153783 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f092c0b3860 0x7f092c1540e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.061+0000 7f092b7fe640 1 -- 172.21.15.5:0/811153783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f092c08bd20 msgr2=0x7f092c14e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:09.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.061+0000 7f092b7fe640 1 --2- 172.21.15.5:0/811153783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f092c08bd20 0x7f092c14e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.061+0000 7f092b7fe640 1 -- 172.21.15.5:0/811153783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f092c150480 con 0x7f092c06d870 2024-05-20T18:30:09.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.061+0000 7f092affd640 1 --2- 172.21.15.5:0/811153783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f092c08bd20 0x7f092c14e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:30:09.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.061+0000 7f092b7fe640 1 --2- 172.21.15.5:0/811153783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f092c06d870 0x7f092c14fa00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f091c0097d0 tx=0x7f091c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:09.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.061+0000 7f0913fff640 1 -- 172.21.15.5:0/811153783 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c041070 con 0x7f092c06d870 2024-05-20T18:30:09.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.061+0000 7f0913fff640 1 -- 172.21.15.5:0/811153783 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f091c03c680 con 0x7f092c06d870 2024-05-20T18:30:09.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.061+0000 7f0913fff640 1 -- 172.21.15.5:0/811153783 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c037040 con 0x7f092c06d870 2024-05-20T18:30:09.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.061+0000 7f0931b65640 1 -- 172.21.15.5:0/811153783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f092c150760 con 0x7f092c06d870 2024-05-20T18:30:09.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.061+0000 7f0931b65640 1 -- 172.21.15.5:0/811153783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f092c150d20 con 0x7f092c06d870 2024-05-20T18:30:09.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.063+0000 7f0913fff640 1 -- 172.21.15.5:0/811153783 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f091c044070 con 0x7f092c06d870 2024-05-20T18:30:09.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.063+0000 7f0913fff640 1 --2- 172.21.15.5:0/811153783 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f091806be30 0x7f091806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.064+0000 7f092affd640 1 --2- 172.21.15.5:0/811153783 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f091806be30 0x7f091806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.064+0000 7f0913fff640 1 -- 172.21.15.5:0/811153783 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7f091c0ab870 con 0x7f092c06d870 2024-05-20T18:30:09.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.064+0000 7f092affd640 1 --2- 172.21.15.5:0/811153783 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f091806be30 0x7f091806e340 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f092c14f990 tx=0x7f0920009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:09.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.065+0000 7f0938a0e740 1 Processor -- start 2024-05-20T18:30:09.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.065+0000 7f0938a0e740 1 -- start start 2024-05-20T18:30:09.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.066+0000 7f0938a0e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b35ac34340 0x55b35b000270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.066+0000 7f0938a0e740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b35ac0b480 0x55b35afd58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.066+0000 7f0938a0e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b35ac13660 0x55b35b01c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.066+0000 7f0938a0e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b35af38880 con 0x55b35ac13660 2024-05-20T18:30:09.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.066+0000 7f0938a0e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b35aee0520 con 0x55b35ac34340 2024-05-20T18:30:09.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.066+0000 7f0938a0e740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55b35af34840 con 0x55b35ac0b480 2024-05-20T18:30:09.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.066+0000 7f092bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b35ac13660 0x55b35b01c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.066+0000 7f092bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b35ac13660 0x55b35b01c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53212/0 (socket says 172.21.15.5:53212) 2024-05-20T18:30:09.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.066+0000 7f092bfff640 1 -- 172.21.15.5:0/386097858 learned_addr learned my addr 172.21.15.5:0/386097858 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:09.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.067+0000 7f092b7fe640 1 --2- 172.21.15.5:0/386097858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b35ac34340 0x55b35b000270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.067+0000 7f092affd640 1 --2- 172.21.15.5:0/386097858 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b35ac0b480 0x55b35afd58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.067+0000 7f092bfff640 1 -- 172.21.15.5:0/386097858 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b35ac0b480 msgr2=0x55b35afd58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:09.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.067+0000 7f092bfff640 1 --2- 172.21.15.5:0/386097858 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b35ac0b480 0x55b35afd58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.067+0000 7f092bfff640 1 -- 172.21.15.5:0/386097858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b35ac34340 msgr2=0x55b35b000270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:09.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.067+0000 7f092bfff640 1 --2- 172.21.15.5:0/386097858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b35ac34340 0x55b35b000270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.067+0000 7f092bfff640 1 -- 172.21.15.5:0/386097858 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b35aed4b40 con 0x55b35ac13660 2024-05-20T18:30:09.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.067+0000 7f092affd640 1 --2- 172.21.15.5:0/386097858 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b35ac0b480 0x55b35afd58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:30:09.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.067+0000 7f092b7fe640 1 --2- 172.21.15.5:0/386097858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b35ac34340 0x55b35b000270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:30:09.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.067+0000 7f092bfff640 1 --2- 172.21.15.5:0/386097858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b35ac13660 0x55b35b01c200 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0924001e80 tx=0x7f0924001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e4749d5db8b03f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:09.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.067+0000 7f09117fa640 1 -- 172.21.15.5:0/386097858 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0924002090 con 0x55b35ac13660 2024-05-20T18:30:09.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.067+0000 7f09117fa640 1 -- 172.21.15.5:0/386097858 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0924018070 con 0x55b35ac13660 2024-05-20T18:30:09.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.067+0000 7f09117fa640 1 -- 172.21.15.5:0/386097858 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0924003b70 con 0x55b35ac13660 2024-05-20T18:30:09.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.068+0000 7f0938a0e740 1 -- 172.21.15.5:0/386097858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b35ac13660 msgr2=0x55b35b01c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:09.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.068+0000 7f0938a0e740 1 --2- 172.21.15.5:0/386097858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b35ac13660 0x55b35b01c200 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0924001e80 tx=0x7f0924001630 comp rx=0 tx=0).stop 2024-05-20T18:30:09.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.068+0000 7f0938a0e740 1 -- 172.21.15.5:0/386097858 shutdown_connections 2024-05-20T18:30:09.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.068+0000 7f0938a0e740 1 --2- 172.21.15.5:0/386097858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b35ac13660 0x55b35b01c200 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.068+0000 7f0938a0e740 1 --2- 172.21.15.5:0/386097858 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b35ac0b480 0x55b35afd58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.068+0000 7f0938a0e740 1 --2- 172.21.15.5:0/386097858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b35ac34340 0x55b35b000270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.068+0000 7f0938a0e740 1 -- 172.21.15.5:0/386097858 >> 172.21.15.5:0/386097858 conn(0x55b35ac28490 msgr2=0x55b35afd4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:09.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.068+0000 7f0938a0e740 1 -- 172.21.15.5:0/386097858 shutdown_connections 2024-05-20T18:30:09.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.068+0000 7f0938a0e740 1 -- 172.21.15.5:0/386097858 wait complete. 2024-05-20T18:30:09.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.069+0000 7f0938a0e740 1 -- start start 2024-05-20T18:30:09.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.069+0000 7f0938a0e740 1 Processor -- start 2024-05-20T18:30:09.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.070+0000 7f0938a0e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b35ac0b480 0x55b35affe0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.070+0000 7f0938a0e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b35ac13660 0x55b35b000a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.070+0000 7f0938a0e740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b35ac34340 0x55b35b000f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.070+0000 7f0938a0e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b35aef3ab0 con 0x55b35ac0b480 2024-05-20T18:30:09.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.070+0000 7f0938a0e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b35afffde0 con 0x55b35ac13660 2024-05-20T18:30:09.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.070+0000 7f0938a0e740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55b35b0017e0 con 0x55b35ac34340 2024-05-20T18:30:09.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.070+0000 7f0938a0e740 10 client.0 _ll_register_callbacks cb 0x55b35ac4bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:30:09.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.070+0000 7f092b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b35ac0b480 0x55b35affe0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.071+0000 7f092b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b35ac0b480 0x55b35affe0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53222/0 (socket says 172.21.15.5:53222) 2024-05-20T18:30:09.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.071+0000 7f092b7fe640 1 -- 172.21.15.5:0/3944603118 learned_addr learned my addr 172.21.15.5:0/3944603118 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:09.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.071+0000 7f092bfff640 1 --2- 172.21.15.5:0/3944603118 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b35ac34340 0x55b35b000f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.071+0000 7f092affd640 1 --2- 172.21.15.5:0/3944603118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b35ac13660 0x55b35b000a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.071+0000 7f092b7fe640 1 -- 172.21.15.5:0/3944603118 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b35ac34340 msgr2=0x55b35b000f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:09.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.071+0000 7f092b7fe640 1 --2- 172.21.15.5:0/3944603118 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b35ac34340 0x55b35b000f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.071+0000 7f092b7fe640 1 -- 172.21.15.5:0/3944603118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b35ac13660 msgr2=0x55b35b000a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:09.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.071+0000 7f092b7fe640 1 --2- 172.21.15.5:0/3944603118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b35ac13660 0x55b35b000a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.071+0000 7f092b7fe640 1 -- 172.21.15.5:0/3944603118 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b35ad187b0 con 0x55b35ac0b480 2024-05-20T18:30:09.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.071+0000 7f092bfff640 1 --2- 172.21.15.5:0/3944603118 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b35ac34340 0x55b35b000f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:30:09.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.071+0000 7f092affd640 1 --2- 172.21.15.5:0/3944603118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b35ac13660 0x55b35b000a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:30:09.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.071+0000 7f092b7fe640 1 --2- 172.21.15.5:0/3944603118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b35ac0b480 0x55b35affe0b0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f091c02a650 tx=0x7f091c0090a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=429ab90ded6cb151 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:09.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.071+0000 7f090a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:30:09.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.071+0000 7f090a7fc640 1 -- 172.21.15.5:0/3944603118 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c04b040 con 0x55b35ac0b480 2024-05-20T18:30:09.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.071+0000 7f090a7fc640 1 -- 172.21.15.5:0/3944603118 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7f091c0802b0 con 0x55b35ac0b480 2024-05-20T18:30:09.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.071+0000 7f0938a0e740 1 -- 172.21.15.5:0/3944603118 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0924018070 con 0x55b35ac0b480 2024-05-20T18:30:09.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.071+0000 7f090a7fc640 1 -- 172.21.15.5:0/3944603118 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f091c07dde0 con 0x55b35ac0b480 2024-05-20T18:30:09.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.071+0000 7f0938a0e740 10 client.6399 fetch_fsmap learned FSMap version 34 2024-05-20T18:30:09.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.071+0000 7f0938a0e740 1 -- 172.21.15.5:0/3944603118 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x55b35ad78f40 con 0x55b35ac0b480 2024-05-20T18:30:09.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.072+0000 7f090a7fc640 1 -- 172.21.15.5:0/3944603118 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0924018070 con 0x55b35ac0b480 2024-05-20T18:30:09.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.072+0000 7f090a7fc640 1 -- 172.21.15.5:0/3944603118 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f091c082180 con 0x55b35ac0b480 2024-05-20T18:30:09.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.072+0000 7f0938a0e740 10 client.6399 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T18:30:09.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.072+0000 7f0938a0e740 10 client.6399 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-20T18:30:09.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.072+0000 7f0938a0e740 20 client.6399 populate_metadata read hostname 'smithi005' 2024-05-20T18:30:09.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.072+0000 7f0938a0e740 1 --2- 172.21.15.5:0/3944603118 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55b35abf3120 0x55b35b0020c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.072+0000 7f0938a0e740 4 client.6399 mds_command: new command op to 6198 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:30:09.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.072+0000 7f0938a0e740 1 -- 172.21.15.5:0/3944603118 --> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b35af45fb0 con 0x55b35abf3120 2024-05-20T18:30:09.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.072+0000 7f092affd640 1 --2- 172.21.15.5:0/3944603118 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55b35abf3120 0x55b35b0020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.072+0000 7f092affd640 1 --2- 172.21.15.5:0/3944603118 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55b35abf3120 0x55b35b0020c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aeb48706e164859 server_cookie=725ca86778f761f8 in_seq=0 out_seq=0 2024-05-20T18:30:09.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.072+0000 7f090a7fc640 10 client.6399 ms_handle_connect on v2:172.21.15.63:6832/4036379300 2024-05-20T18:30:09.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.073+0000 7f090a7fc640 1 -- 172.21.15.5:0/3944603118 <== mds.0 v2:172.21.15.63:6832/4036379300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b35af45fb0 con 0x55b35abf3120 2024-05-20T18:30:09.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.073+0000 7f090a7fc640 10 client.6399 handle_command_reply: tid=0 2024-05-20T18:30:09.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.073+0000 7f0938a0e740 1 client.6399 shutdown 2024-05-20T18:30:09.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.073+0000 7f0938a0e740 1 -- 172.21.15.5:0/3944603118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b35ac0b480 msgr2=0x55b35affe0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:09.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.073+0000 7f0938a0e740 1 --2- 172.21.15.5:0/3944603118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b35ac0b480 0x55b35affe0b0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f091c02a650 tx=0x7f091c0090a0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.073+0000 7f0938a0e740 1 -- 172.21.15.5:0/3944603118 shutdown_connections 2024-05-20T18:30:09.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.073+0000 7f0938a0e740 1 --2- 172.21.15.5:0/3944603118 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55b35abf3120 0x55b35b0020c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.073+0000 7f0938a0e740 1 --2- 172.21.15.5:0/3944603118 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b35ac34340 0x55b35b000f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.073+0000 7f0938a0e740 1 --2- 172.21.15.5:0/3944603118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b35ac13660 0x55b35b000a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.073+0000 7f090a7fc640 0 client.6399 ms_handle_reset on v2:172.21.15.63:6832/4036379300 2024-05-20T18:30:09.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.073+0000 7f0938a0e740 1 --2- 172.21.15.5:0/3944603118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b35ac0b480 0x55b35affe0b0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.073+0000 7f0938a0e740 1 -- 172.21.15.5:0/3944603118 >> 172.21.15.5:0/3944603118 conn(0x55b35ac28490 msgr2=0x55b35afd4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:09.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.074+0000 7f0938a0e740 1 -- 172.21.15.5:0/3944603118 shutdown_connections 2024-05-20T18:30:09.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.074+0000 7f0938a0e740 1 -- 172.21.15.5:0/3944603118 wait complete. 2024-05-20T18:30:09.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.074+0000 7f0938a0e740 20 client.6399 trim_cache size 0 max 16384 2024-05-20T18:30:09.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.092+0000 7f0938a0e740 1 Processor -- start 2024-05-20T18:30:09.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.092+0000 7f0938a0e740 1 -- start start 2024-05-20T18:30:09.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.094+0000 7f0938a0e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b35ac0b480 0x55b35affe090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.094+0000 7f0938a0e740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b35ac13660 0x55b35b03dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.094+0000 7f0938a0e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b35ac34340 0x55b35b0105f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.094+0000 7f0938a0e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f092c202390 con 0x55b35ac0b480 2024-05-20T18:30:09.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.094+0000 7f0938a0e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f092c2013f0 con 0x55b35ac34340 2024-05-20T18:30:09.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.094+0000 7f0938a0e740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f092c15b6f0 con 0x55b35ac13660 2024-05-20T18:30:09.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.094+0000 7f092b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b35ac0b480 0x55b35affe090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.094+0000 7f092b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b35ac0b480 0x55b35affe090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53226/0 (socket says 172.21.15.5:53226) 2024-05-20T18:30:09.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.094+0000 7f092b7fe640 1 -- 172.21.15.5:0/1376766169 learned_addr learned my addr 172.21.15.5:0/1376766169 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:09.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.094+0000 7f092bfff640 1 --2- 172.21.15.5:0/1376766169 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b35ac34340 0x55b35b0105f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.094+0000 7f092affd640 1 --2- 172.21.15.5:0/1376766169 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b35ac13660 0x55b35b03dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.094+0000 7f092b7fe640 1 -- 172.21.15.5:0/1376766169 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b35ac13660 msgr2=0x55b35b03dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:09.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.094+0000 7f092b7fe640 1 --2- 172.21.15.5:0/1376766169 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b35ac13660 0x55b35b03dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.094+0000 7f092b7fe640 1 -- 172.21.15.5:0/1376766169 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b35ac34340 msgr2=0x55b35b0105f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:09.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.094+0000 7f092b7fe640 1 --2- 172.21.15.5:0/1376766169 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b35ac34340 0x55b35b0105f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.094+0000 7f092b7fe640 1 -- 172.21.15.5:0/1376766169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b35ac0c0a0 con 0x55b35ac0b480 2024-05-20T18:30:09.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.094+0000 7f092b7fe640 1 --2- 172.21.15.5:0/1376766169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b35ac0b480 0x55b35affe090 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f091c076460 tx=0x7f091c03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3319cb3878735f8c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:09.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.094+0000 7f0910ff9640 1 -- 172.21.15.5:0/1376766169 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c087070 con 0x55b35ac0b480 2024-05-20T18:30:09.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.094+0000 7f0910ff9640 1 -- 172.21.15.5:0/1376766169 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f091c0800b0 con 0x55b35ac0b480 2024-05-20T18:30:09.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.094+0000 7f0910ff9640 1 -- 172.21.15.5:0/1376766169 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c04b040 con 0x55b35ac0b480 2024-05-20T18:30:09.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.095+0000 7f0938a0e740 1 -- 172.21.15.5:0/1376766169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b35ac0b480 msgr2=0x55b35affe090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:09.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.095+0000 7f0938a0e740 1 --2- 172.21.15.5:0/1376766169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b35ac0b480 0x55b35affe090 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f091c076460 tx=0x7f091c03f000 comp rx=0 tx=0).stop 2024-05-20T18:30:09.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.095+0000 7f0938a0e740 1 -- 172.21.15.5:0/1376766169 shutdown_connections 2024-05-20T18:30:09.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.095+0000 7f0938a0e740 1 --2- 172.21.15.5:0/1376766169 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b35ac34340 0x55b35b0105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.095+0000 7f0938a0e740 1 --2- 172.21.15.5:0/1376766169 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b35ac13660 0x55b35b03dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.095+0000 7f0938a0e740 1 --2- 172.21.15.5:0/1376766169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b35ac0b480 0x55b35affe090 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.095+0000 7f0938a0e740 1 -- 172.21.15.5:0/1376766169 >> 172.21.15.5:0/1376766169 conn(0x55b35ac28490 msgr2=0x55b35afd5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:09.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.095+0000 7f0938a0e740 1 -- 172.21.15.5:0/1376766169 shutdown_connections 2024-05-20T18:30:09.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.095+0000 7f0938a0e740 1 -- 172.21.15.5:0/1376766169 wait complete. 2024-05-20T18:30:09.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.096+0000 7f0938a0e740 1 -- start start 2024-05-20T18:30:09.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.096+0000 7f0938a0e740 1 Processor -- start 2024-05-20T18:30:09.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.098+0000 7f0938a0e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b35ac0b480 0x55b35b096eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.098+0000 7f0938a0e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b35ac13660 0x55b35aff09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.098+0000 7f0938a0e740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b35ac34340 0x55b35affc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.098+0000 7f0938a0e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f092c200da0 con 0x55b35ac0b480 2024-05-20T18:30:09.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.098+0000 7f0938a0e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b35aff4020 con 0x55b35ac13660 2024-05-20T18:30:09.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.098+0000 7f0938a0e740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55b35aff1160 con 0x55b35ac34340 2024-05-20T18:30:09.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.098+0000 7f0938a0e740 10 client.0 _ll_register_callbacks cb 0x7f091c009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:30:09.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.098+0000 7f092b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b35ac0b480 0x55b35b096eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.098+0000 7f092b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b35ac0b480 0x55b35b096eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53240/0 (socket says 172.21.15.5:53240) 2024-05-20T18:30:09.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.098+0000 7f092b7fe640 1 -- 172.21.15.5:0/2665569592 learned_addr learned my addr 172.21.15.5:0/2665569592 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:09.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.098+0000 7f092bfff640 1 --2- 172.21.15.5:0/2665569592 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b35ac34340 0x55b35affc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.098+0000 7f092affd640 1 --2- 172.21.15.5:0/2665569592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b35ac13660 0x55b35aff09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.098+0000 7f092b7fe640 1 -- 172.21.15.5:0/2665569592 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b35ac34340 msgr2=0x55b35affc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:09.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.098+0000 7f092b7fe640 1 --2- 172.21.15.5:0/2665569592 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b35ac34340 0x55b35affc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.098+0000 7f092b7fe640 1 -- 172.21.15.5:0/2665569592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b35ac13660 msgr2=0x55b35aff09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:09.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.098+0000 7f092b7fe640 1 --2- 172.21.15.5:0/2665569592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b35ac13660 0x55b35aff09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.098+0000 7f092b7fe640 1 -- 172.21.15.5:0/2665569592 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b35b002650 con 0x55b35ac0b480 2024-05-20T18:30:09.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.098+0000 7f092b7fe640 1 --2- 172.21.15.5:0/2665569592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b35ac0b480 0x55b35b096eb0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f091c077d90 tx=0x7f091c047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19dced3a07419cf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:09.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.098+0000 7f090a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:30:09.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.098+0000 7f090a7fc640 1 -- 172.21.15.5:0/2665569592 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c09c060 con 0x55b35ac0b480 2024-05-20T18:30:09.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.098+0000 7f090a7fc640 1 -- 172.21.15.5:0/2665569592 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7f091c075830 con 0x55b35ac0b480 2024-05-20T18:30:09.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.098+0000 7f0938a0e740 1 -- 172.21.15.5:0/2665569592 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f091c0800b0 con 0x55b35ac0b480 2024-05-20T18:30:09.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.099+0000 7f090a7fc640 1 -- 172.21.15.5:0/2665569592 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f091c087070 con 0x55b35ac0b480 2024-05-20T18:30:09.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.099+0000 7f0938a0e740 10 client.6405 fetch_fsmap learned FSMap version 34 2024-05-20T18:30:09.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.099+0000 7f0938a0e740 1 -- 172.21.15.5:0/2665569592 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x55b35b002650 con 0x55b35ac0b480 2024-05-20T18:30:09.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.099+0000 7f090a7fc640 1 -- 172.21.15.5:0/2665569592 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f091c0800b0 con 0x55b35ac0b480 2024-05-20T18:30:09.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.099+0000 7f090a7fc640 1 -- 172.21.15.5:0/2665569592 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f091c075a30 con 0x55b35ac0b480 2024-05-20T18:30:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.099+0000 7f0938a0e740 10 client.6405 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T18:30:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.099+0000 7f0938a0e740 10 client.6405 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-20T18:30:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.099+0000 7f0938a0e740 20 client.6405 populate_metadata read hostname 'smithi005' 2024-05-20T18:30:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.099+0000 7f0938a0e740 1 --2- 172.21.15.5:0/2665569592 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55b35b02caa0 0x55b35b02bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.099+0000 7f0938a0e740 4 client.6405 mds_command: new command op to 6198 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-05-20T18:30:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.099+0000 7f0938a0e740 1 -- 172.21.15.5:0/2665569592 --> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) -- 0x55b35af40300 con 0x55b35b02caa0 2024-05-20T18:30:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.099+0000 7f092affd640 1 --2- 172.21.15.5:0/2665569592 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55b35b02caa0 0x55b35b02bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.099+0000 7f090a7fc640 10 client.6405 ms_handle_connect on v2:172.21.15.63:6832/4036379300 2024-05-20T18:30:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.099+0000 7f092affd640 1 --2- 172.21.15.5:0/2665569592 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55b35b02caa0 0x55b35b02bc90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9aeed9f702cbe62 server_cookie=abe1930c1d7662f7 in_seq=0 out_seq=0 2024-05-20T18:30:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.100+0000 7f090a7fc640 1 -- 172.21.15.5:0/2665569592 <== mds.0 v2:172.21.15.63:6832/4036379300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b35af40300 con 0x55b35b02caa0 2024-05-20T18:30:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.100+0000 7f090a7fc640 10 client.6405 handle_command_reply: tid=0 2024-05-20T18:30:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.100+0000 7f0938a0e740 1 client.6405 shutdown 2024-05-20T18:30:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.100+0000 7f0938a0e740 1 -- 172.21.15.5:0/2665569592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b35ac0b480 msgr2=0x55b35b096eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.100+0000 7f0938a0e740 1 --2- 172.21.15.5:0/2665569592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b35ac0b480 0x55b35b096eb0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f091c077d90 tx=0x7f091c047000 comp rx=0 tx=0).stop 2024-05-20T18:30:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.100+0000 7f0938a0e740 1 -- 172.21.15.5:0/2665569592 shutdown_connections 2024-05-20T18:30:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.100+0000 7f0938a0e740 1 --2- 172.21.15.5:0/2665569592 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55b35b02caa0 0x55b35b02bc90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.100+0000 7f0938a0e740 1 --2- 172.21.15.5:0/2665569592 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b35ac34340 0x55b35affc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.100+0000 7f0938a0e740 1 --2- 172.21.15.5:0/2665569592 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b35ac13660 0x55b35aff09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.100+0000 7f0938a0e740 1 --2- 172.21.15.5:0/2665569592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b35ac0b480 0x55b35b096eb0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.100+0000 7f0938a0e740 1 -- 172.21.15.5:0/2665569592 >> 172.21.15.5:0/2665569592 conn(0x55b35ac28490 msgr2=0x55b35b03cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.100+0000 7f090a7fc640 0 client.6405 ms_handle_reset on v2:172.21.15.63:6832/4036379300 2024-05-20T18:30:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.101+0000 7f0938a0e740 1 -- 172.21.15.5:0/2665569592 shutdown_connections 2024-05-20T18:30:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.101+0000 7f0938a0e740 1 -- 172.21.15.5:0/2665569592 wait complete. 2024-05-20T18:30:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.101+0000 7f0938a0e740 20 client.6405 trim_cache size 0 max 16384 2024-05-20T18:30:09.105 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:30:09.106 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-20T18:30:09.106 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "20368279-c70a-40f1-a6a8-7973a6f2d67e", 2024-05-20T18:30:09.106 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-20T18:30:09.106 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:30:09.107 INFO:tasks.ceph.mds.b.smithi063.stderr:2024-05-20T18:30:09.101+0000 7f976018e640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5555=0-4194304@1} caps={5555=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5555 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x5611ac41cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T18:30:09.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.101+0000 7f0931b65640 1 -- 172.21.15.5:0/811153783 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f091806be30 msgr2=0x7f091806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:09.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.101+0000 7f0931b65640 1 --2- 172.21.15.5:0/811153783 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f091806be30 0x7f091806e340 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f092c14f990 tx=0x7f0920009040 comp rx=0 tx=0).stop 2024-05-20T18:30:09.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.101+0000 7f0931b65640 1 -- 172.21.15.5:0/811153783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f092c06d870 msgr2=0x7f092c14fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:09.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.101+0000 7f0931b65640 1 --2- 172.21.15.5:0/811153783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f092c06d870 0x7f092c14fa00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f091c0097d0 tx=0x7f091c0046c0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.101+0000 7f0931b65640 1 -- 172.21.15.5:0/811153783 shutdown_connections 2024-05-20T18:30:09.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.101+0000 7f0931b65640 1 --2- 172.21.15.5:0/811153783 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f091806be30 0x7f091806e340 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.101+0000 7f0931b65640 1 --2- 172.21.15.5:0/811153783 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f092c0b3860 0x7f092c1540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.101+0000 7f0931b65640 1 --2- 172.21.15.5:0/811153783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f092c08bd20 0x7f092c14e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.101+0000 7f0931b65640 1 --2- 172.21.15.5:0/811153783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f092c06d870 0x7f092c14fa00 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.101+0000 7f0931b65640 1 -- 172.21.15.5:0/811153783 >> 172.21.15.5:0/811153783 conn(0x7f092c0b6d10 msgr2=0x7f092c158600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:09.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.101+0000 7f0931b65640 1 -- 172.21.15.5:0/811153783 shutdown_connections 2024-05-20T18:30:09.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.101+0000 7f0931b65640 1 -- 172.21.15.5:0/811153783 wait complete. 2024-05-20T18:30:09.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:30:09.113 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-05-20T18:30:09.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.230+0000 7fa021377640 1 Processor -- start 2024-05-20T18:30:09.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.230+0000 7fa021377640 1 -- start start 2024-05-20T18:30:09.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.231+0000 7fa021377640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa01c06d3d0 0x7fa01c06d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.231+0000 7fa021377640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa01c06dd20 con 0x7fa01c162820 2024-05-20T18:30:09.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.231+0000 7fa021377640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa01c0d0630 con 0x7fa01c0b8190 2024-05-20T18:30:09.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.231+0000 7fa021377640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa01c0d39c0 con 0x7fa01c06d3d0 2024-05-20T18:30:09.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.231+0000 7fa01affd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa01c162820 0x7fa01c0b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44046/0 (socket says 172.21.15.5:44046) 2024-05-20T18:30:09.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.231+0000 7fa01affd640 1 -- 172.21.15.5:0/3500848824 learned_addr learned my addr 172.21.15.5:0/3500848824 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:30:09.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.232+0000 7fa01a7fc640 1 --2- 172.21.15.5:0/3500848824 >> v2:172.21.15.63:3300/0 conn(0x7fa01c06d3d0 0x7fa01c06d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.232+0000 7fa019ffb640 1 -- 172.21.15.5:0/3500848824 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2689480987 0 0) 0x7fa00c001110 con 0x7fa01c162820 2024-05-20T18:30:09.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.232+0000 7fa019ffb640 1 -- 172.21.15.5:0/3500848824 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa004003520 con 0x7fa01c162820 2024-05-20T18:30:09.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.232+0000 7fa019ffb640 1 -- 172.21.15.5:0/3500848824 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2609877186 0 0) 0x7fa01c0d0630 con 0x7fa01c0b8190 2024-05-20T18:30:09.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.232+0000 7fa019ffb640 1 -- 172.21.15.5:0/3500848824 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa00c001110 con 0x7fa01c0b8190 2024-05-20T18:30:09.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.232+0000 7fa01a7fc640 1 -- 172.21.15.5:0/3500848824 >> v1:172.21.15.63:6789/0 conn(0x7fa01c0b8190 legacy=0x7fa01c0b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:09.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.232+0000 7fa01a7fc640 1 -- 172.21.15.5:0/3500848824 >> v1:172.21.15.5:6789/0 conn(0x7fa01c162820 legacy=0x7fa01c0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:09.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.232+0000 7fa01a7fc640 1 -- 172.21.15.5:0/3500848824 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa01c0d3ca0 con 0x7fa01c06d3d0 2024-05-20T18:30:09.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.233+0000 7fa01a7fc640 1 --2- 172.21.15.5:0/3500848824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa01c06d3d0 0x7fa01c06d7e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa0100097a0 tx=0x7fa01002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=500452f357b3ed90 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:09.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.233+0000 7fa019ffb640 1 -- 172.21.15.5:0/3500848824 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa010037070 con 0x7fa01c06d3d0 2024-05-20T18:30:09.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.233+0000 7fa019ffb640 1 -- 172.21.15.5:0/3500848824 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa010032440 con 0x7fa01c06d3d0 2024-05-20T18:30:09.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.233+0000 7fa019ffb640 1 -- 172.21.15.5:0/3500848824 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa010032810 con 0x7fa01c06d3d0 2024-05-20T18:30:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.234+0000 7fa021377640 1 -- 172.21.15.5:0/3500848824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa01c06d3d0 msgr2=0x7fa01c06d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.234+0000 7fa021377640 1 --2- 172.21.15.5:0/3500848824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa01c06d3d0 0x7fa01c06d7e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa0100097a0 tx=0x7fa01002a230 comp rx=0 tx=0).stop 2024-05-20T18:30:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.234+0000 7fa021377640 1 -- 172.21.15.5:0/3500848824 shutdown_connections 2024-05-20T18:30:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.234+0000 7fa021377640 1 --2- 172.21.15.5:0/3500848824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa01c06d3d0 0x7fa01c06d7e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.234+0000 7fa021377640 1 -- 172.21.15.5:0/3500848824 >> 172.21.15.5:0/3500848824 conn(0x7fa01c0d4bd0 msgr2=0x7fa01c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.234+0000 7fa021377640 1 -- 172.21.15.5:0/3500848824 shutdown_connections 2024-05-20T18:30:09.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.234+0000 7fa021377640 1 -- 172.21.15.5:0/3500848824 wait complete. 2024-05-20T18:30:09.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.236+0000 7fa021377640 1 Processor -- start 2024-05-20T18:30:09.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.236+0000 7fa021377640 1 -- start start 2024-05-20T18:30:09.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.237+0000 7fa021377640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa01c06d3d0 0x7fa01c0cc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.237+0000 7fa021377640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa01c0b8190 0x7fa01c14df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.237+0000 7fa021377640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa01c162820 0x7fa01c0caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.237+0000 7fa021377640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa01c0d0300 con 0x7fa01c0b8190 2024-05-20T18:30:09.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.237+0000 7fa021377640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa01c0d42e0 con 0x7fa01c162820 2024-05-20T18:30:09.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.237+0000 7fa021377640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa01c0b4050 con 0x7fa01c06d3d0 2024-05-20T18:30:09.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.237+0000 7fa01a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa01c0b8190 0x7fa01c14df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.237+0000 7fa01a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa01c0b8190 0x7fa01c14df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53254/0 (socket says 172.21.15.5:53254) 2024-05-20T18:30:09.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.237+0000 7fa01a7fc640 1 -- 172.21.15.5:0/2292970067 learned_addr learned my addr 172.21.15.5:0/2292970067 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:09.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.237+0000 7fa01b7fe640 1 --2- 172.21.15.5:0/2292970067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa01c162820 0x7fa01c0caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.237+0000 7fa01affd640 1 --2- 172.21.15.5:0/2292970067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa01c06d3d0 0x7fa01c0cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.237+0000 7fa01a7fc640 1 -- 172.21.15.5:0/2292970067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa01c06d3d0 msgr2=0x7fa01c0cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:09.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.237+0000 7fa01a7fc640 1 --2- 172.21.15.5:0/2292970067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa01c06d3d0 0x7fa01c0cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.237+0000 7fa01a7fc640 1 -- 172.21.15.5:0/2292970067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa01c162820 msgr2=0x7fa01c0caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:09.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.237+0000 7fa01a7fc640 1 --2- 172.21.15.5:0/2292970067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa01c162820 0x7fa01c0caae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.237+0000 7fa01a7fc640 1 -- 172.21.15.5:0/2292970067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa01c0ca060 con 0x7fa01c0b8190 2024-05-20T18:30:09.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.238+0000 7fa01a7fc640 1 --2- 172.21.15.5:0/2292970067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa01c0b8190 0x7fa01c14df70 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa0100098c0 tx=0x7fa0100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:09.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.238+0000 7fa0037fe640 1 -- 172.21.15.5:0/2292970067 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa010041030 con 0x7fa01c0b8190 2024-05-20T18:30:09.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.238+0000 7fa0037fe640 1 -- 172.21.15.5:0/2292970067 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa01003d4e0 con 0x7fa01c0b8190 2024-05-20T18:30:09.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.238+0000 7fa021377640 1 -- 172.21.15.5:0/2292970067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa01c14e4b0 con 0x7fa01c0b8190 2024-05-20T18:30:09.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.238+0000 7fa0037fe640 1 -- 172.21.15.5:0/2292970067 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa010037040 con 0x7fa01c0b8190 2024-05-20T18:30:09.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.238+0000 7fa021377640 1 -- 172.21.15.5:0/2292970067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa01c0cc0e0 con 0x7fa01c0b8190 2024-05-20T18:30:09.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.240+0000 7fa0037fe640 1 -- 172.21.15.5:0/2292970067 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa010044070 con 0x7fa01c0b8190 2024-05-20T18:30:09.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.240+0000 7fa0037fe640 1 --2- 172.21.15.5:0/2292970067 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa00806be30 0x7fa00806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.240+0000 7fa0037fe640 1 -- 172.21.15.5:0/2292970067 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7fa0100ab350 con 0x7fa01c0b8190 2024-05-20T18:30:09.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.240+0000 7fa01affd640 1 --2- 172.21.15.5:0/2292970067 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa00806be30 0x7fa00806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.241+0000 7fa01affd640 1 --2- 172.21.15.5:0/2292970067 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa00806be30 0x7fa00806e340 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa00c000f60 tx=0x7fa00c007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:09.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.241+0000 7fa028226740 1 Processor -- start 2024-05-20T18:30:09.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.242+0000 7fa028226740 1 -- start start 2024-05-20T18:30:09.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.243+0000 7fa028226740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cf44c9390 0x560cf4893b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.243+0000 7fa028226740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cf44a04d0 0x560cf48ae890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.243+0000 7fa028226740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cf44a86b0 0x560cf48aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.243+0000 7fa028226740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560cf47cdd00 con 0x560cf44a86b0 2024-05-20T18:30:09.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.243+0000 7fa028226740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560cf47754c0 con 0x560cf44c9390 2024-05-20T18:30:09.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.243+0000 7fa028226740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x560cf47c9cc0 con 0x560cf44a04d0 2024-05-20T18:30:09.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.243+0000 7fa01b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cf44a86b0 0x560cf48aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.243+0000 7fa01b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cf44a86b0 0x560cf48aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53264/0 (socket says 172.21.15.5:53264) 2024-05-20T18:30:09.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.243+0000 7fa01b7fe640 1 -- 172.21.15.5:0/4030911135 learned_addr learned my addr 172.21.15.5:0/4030911135 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:09.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.243+0000 7fa01a7fc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cf44a04d0 0x560cf48ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.243+0000 7fa01affd640 1 --2- 172.21.15.5:0/4030911135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cf44c9390 0x560cf4893b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.243+0000 7fa01b7fe640 1 -- 172.21.15.5:0/4030911135 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cf44a04d0 msgr2=0x560cf48ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:09.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.243+0000 7fa01b7fe640 1 --2- 172.21.15.5:0/4030911135 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cf44a04d0 0x560cf48ae890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.243+0000 7fa01b7fe640 1 -- 172.21.15.5:0/4030911135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cf44c9390 msgr2=0x560cf4893b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:09.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.243+0000 7fa01b7fe640 1 --2- 172.21.15.5:0/4030911135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cf44c9390 0x560cf4893b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.243+0000 7fa01b7fe640 1 -- 172.21.15.5:0/4030911135 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cf45ab7d0 con 0x560cf44a86b0 2024-05-20T18:30:09.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.244+0000 7fa01b7fe640 1 --2- 172.21.15.5:0/4030911135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cf44a86b0 0x560cf48aedd0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa014001ee0 tx=0x7fa014001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3506c4bacaa148bf server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:09.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.244+0000 7fa000ff9640 1 -- 172.21.15.5:0/4030911135 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01400fe50 con 0x560cf44a86b0 2024-05-20T18:30:09.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.244+0000 7fa000ff9640 1 -- 172.21.15.5:0/4030911135 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa014018070 con 0x560cf44a86b0 2024-05-20T18:30:09.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.244+0000 7fa000ff9640 1 -- 172.21.15.5:0/4030911135 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa014003b70 con 0x560cf44a86b0 2024-05-20T18:30:09.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.245+0000 7fa028226740 1 -- 172.21.15.5:0/4030911135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cf44a86b0 msgr2=0x560cf48aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:09.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.245+0000 7fa028226740 1 --2- 172.21.15.5:0/4030911135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cf44a86b0 0x560cf48aedd0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa014001ee0 tx=0x7fa014001630 comp rx=0 tx=0).stop 2024-05-20T18:30:09.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.245+0000 7fa028226740 1 -- 172.21.15.5:0/4030911135 shutdown_connections 2024-05-20T18:30:09.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.245+0000 7fa028226740 1 --2- 172.21.15.5:0/4030911135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cf44a86b0 0x560cf48aedd0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.245+0000 7fa028226740 1 --2- 172.21.15.5:0/4030911135 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cf44a04d0 0x560cf48ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.245+0000 7fa028226740 1 --2- 172.21.15.5:0/4030911135 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cf44c9390 0x560cf4893b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.245+0000 7fa028226740 1 -- 172.21.15.5:0/4030911135 >> 172.21.15.5:0/4030911135 conn(0x560cf44bd4e0 msgr2=0x560cf48691c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:09.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.245+0000 7fa028226740 1 -- 172.21.15.5:0/4030911135 shutdown_connections 2024-05-20T18:30:09.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.245+0000 7fa028226740 1 -- 172.21.15.5:0/4030911135 wait complete. 2024-05-20T18:30:09.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.246+0000 7fa028226740 1 -- start start 2024-05-20T18:30:09.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.246+0000 7fa028226740 1 Processor -- start 2024-05-20T18:30:09.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.247+0000 7fa028226740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cf44a04d0 0x560cf4896110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.247+0000 7fa028226740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cf44a86b0 0x560cf487ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.247+0000 7fa028226740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cf44c9390 0x560cf48953f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.247+0000 7fa028226740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560cf4788b90 con 0x560cf44a86b0 2024-05-20T18:30:09.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.247+0000 7fa028226740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560cf4896960 con 0x560cf44c9390 2024-05-20T18:30:09.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.247+0000 7fa028226740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x560cf487f530 con 0x560cf44a04d0 2024-05-20T18:30:09.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.247+0000 7fa028226740 10 client.0 _ll_register_callbacks cb 0x560cf47c4da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:30:09.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.247+0000 7fa01a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cf44a86b0 0x560cf487ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.247+0000 7fa01a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cf44a86b0 0x560cf487ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53266/0 (socket says 172.21.15.5:53266) 2024-05-20T18:30:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.247+0000 7fa01a7fc640 1 -- 172.21.15.5:0/2985026827 learned_addr learned my addr 172.21.15.5:0/2985026827 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.247+0000 7fa01affd640 1 --2- 172.21.15.5:0/2985026827 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cf44a04d0 0x560cf4896110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.247+0000 7fa01b7fe640 1 --2- 172.21.15.5:0/2985026827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cf44c9390 0x560cf48953f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.247+0000 7fa01a7fc640 1 -- 172.21.15.5:0/2985026827 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cf44a04d0 msgr2=0x560cf4896110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.247+0000 7fa01a7fc640 1 --2- 172.21.15.5:0/2985026827 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cf44a04d0 0x560cf4896110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.247+0000 7fa01a7fc640 1 -- 172.21.15.5:0/2985026827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cf44c9390 msgr2=0x560cf48953f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.247+0000 7fa01a7fc640 1 --2- 172.21.15.5:0/2985026827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cf44c9390 0x560cf48953f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.247+0000 7fa01a7fc640 1 -- 172.21.15.5:0/2985026827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cf44e13f0 con 0x560cf44a86b0 2024-05-20T18:30:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.248+0000 7fa01b7fe640 1 --2- 172.21.15.5:0/2985026827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cf44c9390 0x560cf48953f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:30:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.248+0000 7fa01a7fc640 1 --2- 172.21.15.5:0/2985026827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cf44a86b0 0x560cf487ed30 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fa010079430 tx=0x7fa0100794a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1f70e835e01be2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.248+0000 7f9ff9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:30:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.248+0000 7f9ff9ffb640 1 -- 172.21.15.5:0/2985026827 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa010081b40 con 0x560cf44a86b0 2024-05-20T18:30:09.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.248+0000 7f9ff9ffb640 1 -- 172.21.15.5:0/2985026827 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7fa010081ce0 con 0x560cf44a86b0 2024-05-20T18:30:09.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.248+0000 7fa028226740 1 -- 172.21.15.5:0/2985026827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa014018070 con 0x560cf44a86b0 2024-05-20T18:30:09.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.248+0000 7f9ff9ffb640 1 -- 172.21.15.5:0/2985026827 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa010091180 con 0x560cf44a86b0 2024-05-20T18:30:09.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.248+0000 7fa028226740 10 client.6417 fetch_fsmap learned FSMap version 34 2024-05-20T18:30:09.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.248+0000 7fa028226740 1 -- 172.21.15.5:0/2985026827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x560cf44a10f0 con 0x560cf44a86b0 2024-05-20T18:30:09.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.248+0000 7f9ff9ffb640 1 -- 172.21.15.5:0/2985026827 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa014018070 con 0x560cf44a86b0 2024-05-20T18:30:09.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.248+0000 7f9ff9ffb640 1 -- 172.21.15.5:0/2985026827 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7fa01003a030 con 0x560cf44a86b0 2024-05-20T18:30:09.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.248+0000 7fa028226740 10 client.6417 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T18:30:09.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.248+0000 7fa028226740 10 client.6417 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-20T18:30:09.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.248+0000 7fa028226740 20 client.6417 populate_metadata read hostname 'smithi005' 2024-05-20T18:30:09.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.248+0000 7fa028226740 1 --2- 172.21.15.5:0/2985026827 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x560cf4487f60 0x560cf48b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.248+0000 7fa028226740 4 client.6417 mds_command: new command op to 6198 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:30:09.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.248+0000 7fa028226740 1 -- 172.21.15.5:0/2985026827 --> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cf47db430 con 0x560cf4487f60 2024-05-20T18:30:09.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.249+0000 7fa01affd640 1 --2- 172.21.15.5:0/2985026827 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x560cf4487f60 0x560cf48b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.249+0000 7f9ff9ffb640 10 client.6417 ms_handle_connect on v2:172.21.15.63:6832/4036379300 2024-05-20T18:30:09.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.249+0000 7fa01affd640 1 --2- 172.21.15.5:0/2985026827 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x560cf4487f60 0x560cf48b27b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=182c5523ac6262db server_cookie=e3543b70d0c77b56 in_seq=0 out_seq=0 2024-05-20T18:30:09.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.249+0000 7f9ff9ffb640 1 -- 172.21.15.5:0/2985026827 <== mds.0 v2:172.21.15.63:6832/4036379300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560cf47db430 con 0x560cf4487f60 2024-05-20T18:30:09.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.249+0000 7f9ff9ffb640 10 client.6417 handle_command_reply: tid=0 2024-05-20T18:30:09.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.250+0000 7fa028226740 1 client.6417 shutdown 2024-05-20T18:30:09.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.250+0000 7fa028226740 1 -- 172.21.15.5:0/2985026827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cf44a86b0 msgr2=0x560cf487ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:09.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.250+0000 7fa028226740 1 --2- 172.21.15.5:0/2985026827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cf44a86b0 0x560cf487ed30 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fa010079430 tx=0x7fa0100794a0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.250+0000 7fa028226740 1 -- 172.21.15.5:0/2985026827 shutdown_connections 2024-05-20T18:30:09.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.250+0000 7fa028226740 1 --2- 172.21.15.5:0/2985026827 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x560cf4487f60 0x560cf48b27b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.250+0000 7fa028226740 1 --2- 172.21.15.5:0/2985026827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cf44c9390 0x560cf48953f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.250+0000 7fa028226740 1 --2- 172.21.15.5:0/2985026827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cf44a86b0 0x560cf487ed30 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.250+0000 7fa028226740 1 --2- 172.21.15.5:0/2985026827 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cf44a04d0 0x560cf4896110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.250+0000 7f9ff9ffb640 0 client.6417 ms_handle_reset on v2:172.21.15.63:6832/4036379300 2024-05-20T18:30:09.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.250+0000 7fa028226740 1 -- 172.21.15.5:0/2985026827 >> 172.21.15.5:0/2985026827 conn(0x560cf44bd4e0 msgr2=0x560cf48691c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:09.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.250+0000 7fa028226740 1 -- 172.21.15.5:0/2985026827 shutdown_connections 2024-05-20T18:30:09.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.250+0000 7fa028226740 1 -- 172.21.15.5:0/2985026827 wait complete. 2024-05-20T18:30:09.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.250+0000 7fa028226740 20 client.6417 trim_cache size 0 max 16384 2024-05-20T18:30:09.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.268+0000 7fa028226740 1 Processor -- start 2024-05-20T18:30:09.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.268+0000 7fa028226740 1 -- start start 2024-05-20T18:30:09.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.269+0000 7fa028226740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cf44a04d0 0x560cf48b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.269+0000 7fa028226740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cf44a86b0 0x560cf487ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.269+0000 7fa028226740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cf44c9390 0x560cf4893b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.269+0000 7fa028226740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa01c202410 con 0x560cf44c9390 2024-05-20T18:30:09.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.269+0000 7fa028226740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa01c2013f0 con 0x560cf44a86b0 2024-05-20T18:30:09.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.269+0000 7fa028226740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa01c0cec20 con 0x560cf44a04d0 2024-05-20T18:30:09.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.270+0000 7fa01b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cf44c9390 0x560cf4893b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.270+0000 7fa01affd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cf44a04d0 0x560cf48b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.270+0000 7fa01b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cf44c9390 0x560cf4893b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53272/0 (socket says 172.21.15.5:53272) 2024-05-20T18:30:09.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.270+0000 7fa01b7fe640 1 -- 172.21.15.5:0/1784963259 learned_addr learned my addr 172.21.15.5:0/1784963259 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:09.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.270+0000 7fa01a7fc640 1 --2- 172.21.15.5:0/1784963259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cf44a86b0 0x560cf487ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.270+0000 7fa01b7fe640 1 -- 172.21.15.5:0/1784963259 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cf44a04d0 msgr2=0x560cf48b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:09.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.270+0000 7fa01b7fe640 1 --2- 172.21.15.5:0/1784963259 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cf44a04d0 0x560cf48b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.270+0000 7fa01b7fe640 1 -- 172.21.15.5:0/1784963259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cf44a86b0 msgr2=0x560cf487ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:09.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.270+0000 7fa01b7fe640 1 --2- 172.21.15.5:0/1784963259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cf44a86b0 0x560cf487ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.270+0000 7fa01b7fe640 1 -- 172.21.15.5:0/1784963259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cf4867210 con 0x560cf44c9390 2024-05-20T18:30:09.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.270+0000 7fa01b7fe640 1 --2- 172.21.15.5:0/1784963259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cf44c9390 0x560cf4893b60 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa014015040 tx=0x7fa01400a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cd3cf54b85a3ca server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:09.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.270+0000 7f9ffbfff640 1 -- 172.21.15.5:0/1784963259 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa014006b50 con 0x560cf44c9390 2024-05-20T18:30:09.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.270+0000 7f9ffbfff640 1 -- 172.21.15.5:0/1784963259 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa014021070 con 0x560cf44c9390 2024-05-20T18:30:09.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.270+0000 7f9ffbfff640 1 -- 172.21.15.5:0/1784963259 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa014003970 con 0x560cf44c9390 2024-05-20T18:30:09.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.271+0000 7fa028226740 1 -- 172.21.15.5:0/1784963259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cf44c9390 msgr2=0x560cf4893b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:09.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.271+0000 7fa028226740 1 --2- 172.21.15.5:0/1784963259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cf44c9390 0x560cf4893b60 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa014015040 tx=0x7fa01400a000 comp rx=0 tx=0).stop 2024-05-20T18:30:09.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.271+0000 7fa028226740 1 -- 172.21.15.5:0/1784963259 shutdown_connections 2024-05-20T18:30:09.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.271+0000 7fa028226740 1 --2- 172.21.15.5:0/1784963259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cf44c9390 0x560cf4893b60 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.271+0000 7fa028226740 1 --2- 172.21.15.5:0/1784963259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cf44a86b0 0x560cf487ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.271+0000 7fa028226740 1 --2- 172.21.15.5:0/1784963259 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cf44a04d0 0x560cf48b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.271+0000 7fa028226740 1 -- 172.21.15.5:0/1784963259 >> 172.21.15.5:0/1784963259 conn(0x560cf44bd4e0 msgr2=0x560cf48aa6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:09.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.271+0000 7fa028226740 1 -- 172.21.15.5:0/1784963259 shutdown_connections 2024-05-20T18:30:09.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.271+0000 7fa028226740 1 -- 172.21.15.5:0/1784963259 wait complete. 2024-05-20T18:30:09.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.272+0000 7fa028226740 1 -- start start 2024-05-20T18:30:09.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.272+0000 7fa028226740 1 Processor -- start 2024-05-20T18:30:09.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.273+0000 7fa028226740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cf44a04d0 0x560cf4893b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.273+0000 7fa028226740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cf44a86b0 0x560cf48940a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.273+0000 7fa028226740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cf44c9390 0x560cf487d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.273+0000 7fa028226740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560cf4866fd0 con 0x560cf44a04d0 2024-05-20T18:30:09.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.273+0000 7fa028226740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa01c200dd0 con 0x560cf44a86b0 2024-05-20T18:30:09.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.273+0000 7fa028226740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x560cf48948b0 con 0x560cf44c9390 2024-05-20T18:30:09.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.273+0000 7fa028226740 10 client.0 _ll_register_callbacks cb 0x7fa010079360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:30:09.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.273+0000 7fa01affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cf44a04d0 0x560cf4893b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.273+0000 7fa01affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cf44a04d0 0x560cf4893b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53286/0 (socket says 172.21.15.5:53286) 2024-05-20T18:30:09.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.273+0000 7fa01affd640 1 -- 172.21.15.5:0/2540578117 learned_addr learned my addr 172.21.15.5:0/2540578117 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:09.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.273+0000 7fa01a7fc640 1 --2- 172.21.15.5:0/2540578117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cf44a86b0 0x560cf48940a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.273+0000 7fa01b7fe640 1 --2- 172.21.15.5:0/2540578117 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cf44c9390 0x560cf487d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.274+0000 7fa01affd640 1 -- 172.21.15.5:0/2540578117 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cf44c9390 msgr2=0x560cf487d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:09.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.274+0000 7fa01affd640 1 --2- 172.21.15.5:0/2540578117 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cf44c9390 0x560cf487d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.274+0000 7fa01affd640 1 -- 172.21.15.5:0/2540578117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cf44a86b0 msgr2=0x560cf48940a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:09.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.274+0000 7fa01affd640 1 --2- 172.21.15.5:0/2540578117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cf44a86b0 0x560cf48940a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.274+0000 7fa01affd640 1 -- 172.21.15.5:0/2540578117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cf48632f0 con 0x560cf44a04d0 2024-05-20T18:30:09.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.274+0000 7fa01b7fe640 1 --2- 172.21.15.5:0/2540578117 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cf44c9390 0x560cf487d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:30:09.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.274+0000 7fa01affd640 1 --2- 172.21.15.5:0/2540578117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cf44a04d0 0x560cf4893b60 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x560cf48978c0 tx=0x7fa00c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4d3257af13f41f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:09.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.274+0000 7f9ff9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:30:09.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.274+0000 7f9ff9ffb640 1 -- 172.21.15.5:0/2540578117 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00c005850 con 0x560cf44a04d0 2024-05-20T18:30:09.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.274+0000 7f9ff9ffb640 1 -- 172.21.15.5:0/2540578117 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7fa00c0059f0 con 0x560cf44a04d0 2024-05-20T18:30:09.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.274+0000 7fa028226740 1 -- 172.21.15.5:0/2540578117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa014021070 con 0x560cf44a04d0 2024-05-20T18:30:09.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.274+0000 7f9ff9ffb640 1 -- 172.21.15.5:0/2540578117 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa00c038050 con 0x560cf44a04d0 2024-05-20T18:30:09.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.274+0000 7fa028226740 10 client.6423 fetch_fsmap learned FSMap version 34 2024-05-20T18:30:09.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.274+0000 7fa028226740 1 -- 172.21.15.5:0/2540578117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x560cf487e340 con 0x560cf44a04d0 2024-05-20T18:30:09.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.274+0000 7f9ff9ffb640 1 -- 172.21.15.5:0/2540578117 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa014021070 con 0x560cf44a04d0 2024-05-20T18:30:09.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.274+0000 7f9ff9ffb640 1 -- 172.21.15.5:0/2540578117 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7fa00c03b020 con 0x560cf44a04d0 2024-05-20T18:30:09.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.274+0000 7fa028226740 10 client.6423 fetch_fsmap finished waiting for FSMap version 34 2024-05-20T18:30:09.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.274+0000 7fa028226740 10 client.6423 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-20T18:30:09.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.274+0000 7fa028226740 20 client.6423 populate_metadata read hostname 'smithi005' 2024-05-20T18:30:09.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.274+0000 7fa028226740 1 --2- 172.21.15.5:0/2540578117 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x560cf48c06a0 0x560cf487db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:09.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.274+0000 7fa028226740 4 client.6423 mds_command: new command op to 6198 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T18:30:09.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.274+0000 7fa028226740 1 -- 172.21.15.5:0/2540578117 --> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560cf48ab2c0 con 0x560cf48c06a0 2024-05-20T18:30:09.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.275+0000 7fa01a7fc640 1 --2- 172.21.15.5:0/2540578117 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x560cf48c06a0 0x560cf487db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:09.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.275+0000 7fa01a7fc640 1 --2- 172.21.15.5:0/2540578117 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x560cf48c06a0 0x560cf487db10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca63b8c608641b53 server_cookie=37791105fcb5b9ef in_seq=0 out_seq=0 2024-05-20T18:30:09.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.275+0000 7f9ff9ffb640 10 client.6423 ms_handle_connect on v2:172.21.15.63:6832/4036379300 2024-05-20T18:30:09.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.275+0000 7f9ff9ffb640 1 -- 172.21.15.5:0/2540578117 <== mds.0 v2:172.21.15.63:6832/4036379300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+253 (crc 0 0 0) 0x560cf48ab2c0 con 0x560cf48c06a0 2024-05-20T18:30:09.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.275+0000 7f9ff9ffb640 10 client.6423 handle_command_reply: tid=0 2024-05-20T18:30:09.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.275+0000 7fa028226740 1 client.6423 shutdown 2024-05-20T18:30:09.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.275+0000 7fa028226740 1 -- 172.21.15.5:0/2540578117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cf44a04d0 msgr2=0x560cf4893b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:09.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.275+0000 7fa028226740 1 --2- 172.21.15.5:0/2540578117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cf44a04d0 0x560cf4893b60 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x560cf48978c0 tx=0x7fa00c012000 comp rx=0 tx=0).stop 2024-05-20T18:30:09.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.275+0000 7fa028226740 1 -- 172.21.15.5:0/2540578117 shutdown_connections 2024-05-20T18:30:09.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.275+0000 7fa028226740 1 --2- 172.21.15.5:0/2540578117 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x560cf48c06a0 0x560cf487db10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.275+0000 7fa028226740 1 --2- 172.21.15.5:0/2540578117 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cf44c9390 0x560cf487d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.275+0000 7f9ff9ffb640 0 client.6423 ms_handle_reset on v2:172.21.15.63:6832/4036379300 2024-05-20T18:30:09.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.275+0000 7fa028226740 1 --2- 172.21.15.5:0/2540578117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cf44a86b0 0x560cf48940a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.275+0000 7fa028226740 1 --2- 172.21.15.5:0/2540578117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cf44a04d0 0x560cf4893b60 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.275+0000 7fa028226740 1 -- 172.21.15.5:0/2540578117 >> 172.21.15.5:0/2540578117 conn(0x560cf44bd4e0 msgr2=0x560cf48aa6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:09.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.275+0000 7fa028226740 1 -- 172.21.15.5:0/2540578117 shutdown_connections 2024-05-20T18:30:09.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.275+0000 7fa028226740 1 -- 172.21.15.5:0/2540578117 wait complete. 2024-05-20T18:30:09.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.276+0000 7fa028226740 20 client.6423 trim_cache size 0 max 16384 2024-05-20T18:30:09.282 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:30:09.283 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-20T18:30:09.283 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-20T18:30:09.283 INFO:teuthology.orchestra.run.smithi005.stdout: "20368279-c70a-40f1-a6a8-7973a6f2d67e": { 2024-05-20T18:30:09.283 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-05-20T18:30:09.283 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "20368279-c70a-40f1-a6a8-7973a6f2d67e", 2024-05-20T18:30:09.283 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive" 2024-05-20T18:30:09.283 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:30:09.283 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:30:09.283 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:30:09.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.276+0000 7fa021377640 1 -- 172.21.15.5:0/2292970067 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa00806be30 msgr2=0x7fa00806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:09.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.276+0000 7fa021377640 1 --2- 172.21.15.5:0/2292970067 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa00806be30 0x7fa00806e340 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa00c000f60 tx=0x7fa00c007d50 comp rx=0 tx=0).stop 2024-05-20T18:30:09.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.276+0000 7fa021377640 1 -- 172.21.15.5:0/2292970067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa01c0b8190 msgr2=0x7fa01c14df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:09.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.276+0000 7fa021377640 1 --2- 172.21.15.5:0/2292970067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa01c0b8190 0x7fa01c14df70 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa0100098c0 tx=0x7fa0100046c0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.276+0000 7fa021377640 1 -- 172.21.15.5:0/2292970067 shutdown_connections 2024-05-20T18:30:09.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.276+0000 7fa021377640 1 --2- 172.21.15.5:0/2292970067 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa00806be30 0x7fa00806e340 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.276+0000 7fa021377640 1 --2- 172.21.15.5:0/2292970067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa01c162820 0x7fa01c0caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.276+0000 7fa021377640 1 --2- 172.21.15.5:0/2292970067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa01c0b8190 0x7fa01c14df70 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.276+0000 7fa021377640 1 --2- 172.21.15.5:0/2292970067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa01c06d3d0 0x7fa01c0cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:09.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.276+0000 7fa021377640 1 -- 172.21.15.5:0/2292970067 >> 172.21.15.5:0/2292970067 conn(0x7fa01c0d4bd0 msgr2=0x7fa01c15eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:09.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.276+0000 7fa021377640 1 -- 172.21.15.5:0/2292970067 shutdown_connections 2024-05-20T18:30:09.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:09.276+0000 7fa021377640 1 -- 172.21.15.5:0/2292970067 wait complete. 2024-05-20T18:30:09.287 INFO:tasks.cephfs.filesystem:scrub status for tag:20368279-c70a-40f1-a6a8-7973a6f2d67e - {'path': '/', 'tag': '20368279-c70a-40f1-a6a8-7973a6f2d67e', 'options': 'recursive'} 2024-05-20T18:30:11.063 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:30:11.066 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:30:12.946 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-20T18:30:12.946+0000 7fd1d1d3f640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-20T18:30:39.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:30:39.289 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-05-20T18:30:39.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.421+0000 7f78bf5ea640 1 Processor -- start 2024-05-20T18:30:39.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.421+0000 7f78bf5ea640 1 -- start start 2024-05-20T18:30:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.422+0000 7f78bf5ea640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f78b80b1e10 0x7f78b815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.422+0000 7f78bf5ea640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f78b8070850 0x7f78b8070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.422+0000 7f78bf5ea640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78b80cd580 con 0x7f78b815d530 2024-05-20T18:30:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.422+0000 7f78bf5ea640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f78b80ce190 con 0x7f78b80b1e10 2024-05-20T18:30:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.422+0000 7f78bf5ea640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f78b80cd860 con 0x7f78b8070850 2024-05-20T18:30:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.423+0000 7f78bcb85640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f78b80b1e10 0x7f78b815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.423+0000 7f78bdb87640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f78b8070850 0x7f78b8070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.423+0000 7f78bdb87640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f78b8070850 0x7f78b8070c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:51688/0 (socket says 172.21.15.5:51688) 2024-05-20T18:30:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.423+0000 7f78bcb85640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f78b80b1e10 0x7f78b815aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:44724/0 (socket says 172.21.15.5:44724) 2024-05-20T18:30:39.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.423+0000 7f78bdb87640 1 -- 172.21.15.5:0/240845994 learned_addr learned my addr 172.21.15.5:0/240845994 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:39.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.423+0000 7f78bd386640 1 --1- 172.21.15.5:0/240845994 >> v1:172.21.15.63:6789/0 conn(0x7f78b815d530 0x7f78b80b1700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:48166/0 (socket says 172.21.15.5:48166) 2024-05-20T18:30:39.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.423+0000 7f78bcb85640 1 -- 172.21.15.5:0/240845994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f78b8070850 msgr2=0x7f78b8070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:39.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.423+0000 7f78bcb85640 1 --2- 172.21.15.5:0/240845994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f78b8070850 0x7f78b8070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.423+0000 7f78bcb85640 1 -- 172.21.15.5:0/240845994 >> v1:172.21.15.63:6789/0 conn(0x7f78b815d530 legacy=0x7f78b80b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:39.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.423+0000 7f78bcb85640 1 -- 172.21.15.5:0/240845994 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b80cdb40 con 0x7f78b80b1e10 2024-05-20T18:30:39.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.423+0000 7f78bdb87640 1 --2- 172.21.15.5:0/240845994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f78b8070850 0x7f78b8070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:30:39.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.424+0000 7f78bcb85640 1 --2- 172.21.15.5:0/240845994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78b80b1e10 0x7f78b815aaa0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f78a80097a0 tx=0x7f78a802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd083ae9da8a9b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:39.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.424+0000 7f78affff640 1 -- 172.21.15.5:0/240845994 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a8037070 con 0x7f78b80b1e10 2024-05-20T18:30:39.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.424+0000 7f78affff640 1 -- 172.21.15.5:0/240845994 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78a8032440 con 0x7f78b80b1e10 2024-05-20T18:30:39.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.424+0000 7f78affff640 1 -- 172.21.15.5:0/240845994 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a8032810 con 0x7f78b80b1e10 2024-05-20T18:30:39.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.425+0000 7f78bf5ea640 1 -- 172.21.15.5:0/240845994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78b80b1e10 msgr2=0x7f78b815aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:39.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.425+0000 7f78bf5ea640 1 --2- 172.21.15.5:0/240845994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78b80b1e10 0x7f78b815aaa0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f78a80097a0 tx=0x7f78a802a180 comp rx=0 tx=0).stop 2024-05-20T18:30:39.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.425+0000 7f78bf5ea640 1 -- 172.21.15.5:0/240845994 shutdown_connections 2024-05-20T18:30:39.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.425+0000 7f78bf5ea640 1 --2- 172.21.15.5:0/240845994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f78b8070850 0x7f78b8070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.425+0000 7f78bf5ea640 1 --2- 172.21.15.5:0/240845994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78b80b1e10 0x7f78b815aaa0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.425+0000 7f78bf5ea640 1 -- 172.21.15.5:0/240845994 >> 172.21.15.5:0/240845994 conn(0x7f78b80d29d0 msgr2=0x7f78b80d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:39.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.425+0000 7f78bf5ea640 1 -- 172.21.15.5:0/240845994 shutdown_connections 2024-05-20T18:30:39.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.425+0000 7f78bf5ea640 1 -- 172.21.15.5:0/240845994 wait complete. 2024-05-20T18:30:39.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.427+0000 7f78bf5ea640 1 Processor -- start 2024-05-20T18:30:39.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.427+0000 7f78bf5ea640 1 -- start start 2024-05-20T18:30:39.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.428+0000 7f78bf5ea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78b8070850 0x7f78b81570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.428+0000 7f78bf5ea640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f78b80b1e10 0x7f78b815c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.428+0000 7f78bf5ea640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78b815d530 0x7f78b806c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.428+0000 7f78bf5ea640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78b80711a0 con 0x7f78b8070850 2024-05-20T18:30:39.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.428+0000 7f78bf5ea640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f78b80cd230 con 0x7f78b815d530 2024-05-20T18:30:39.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.428+0000 7f78bf5ea640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f78b8071320 con 0x7f78b80b1e10 2024-05-20T18:30:39.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.428+0000 7f78bd386640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78b8070850 0x7f78b81570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.428+0000 7f78bd386640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78b8070850 0x7f78b81570b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52652/0 (socket says 172.21.15.5:52652) 2024-05-20T18:30:39.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.428+0000 7f78bd386640 1 -- 172.21.15.5:0/3449966378 learned_addr learned my addr 172.21.15.5:0/3449966378 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:39.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.429+0000 7f78bdb87640 1 --2- 172.21.15.5:0/3449966378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78b815d530 0x7f78b806c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.429+0000 7f78bcb85640 1 --2- 172.21.15.5:0/3449966378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f78b80b1e10 0x7f78b815c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.429+0000 7f78bd386640 1 -- 172.21.15.5:0/3449966378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f78b80b1e10 msgr2=0x7f78b815c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:39.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.429+0000 7f78bd386640 1 --2- 172.21.15.5:0/3449966378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f78b80b1e10 0x7f78b815c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.429+0000 7f78bd386640 1 -- 172.21.15.5:0/3449966378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78b815d530 msgr2=0x7f78b806c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:39.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.429+0000 7f78bd386640 1 --2- 172.21.15.5:0/3449966378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78b815d530 0x7f78b806c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.429+0000 7f78bd386640 1 -- 172.21.15.5:0/3449966378 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b815caf0 con 0x7f78b8070850 2024-05-20T18:30:39.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.429+0000 7f78bcb85640 1 --2- 172.21.15.5:0/3449966378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f78b80b1e10 0x7f78b815c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:30:39.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.429+0000 7f78bd386640 1 --2- 172.21.15.5:0/3449966378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78b8070850 0x7f78b81570b0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f78b400c470 tx=0x7f78b400c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:39.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.429+0000 7f78adffb640 1 -- 172.21.15.5:0/3449966378 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b40074e0 con 0x7f78b8070850 2024-05-20T18:30:39.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.429+0000 7f78adffb640 1 -- 172.21.15.5:0/3449966378 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b4007680 con 0x7f78b8070850 2024-05-20T18:30:39.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.429+0000 7f78adffb640 1 -- 172.21.15.5:0/3449966378 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4004a00 con 0x7f78b8070850 2024-05-20T18:30:39.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.429+0000 7f78bf5ea640 1 -- 172.21.15.5:0/3449966378 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78b815baf0 con 0x7f78b8070850 2024-05-20T18:30:39.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.429+0000 7f78bf5ea640 1 -- 172.21.15.5:0/3449966378 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78b81f6670 con 0x7f78b8070850 2024-05-20T18:30:39.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.431+0000 7f78adffb640 1 -- 172.21.15.5:0/3449966378 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f78b401a020 con 0x7f78b8070850 2024-05-20T18:30:39.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.432+0000 7f78adffb640 1 --2- 172.21.15.5:0/3449966378 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f789c06be30 0x7f789c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.432+0000 7f78bcb85640 1 --2- 172.21.15.5:0/3449966378 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f789c06be30 0x7f789c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.432+0000 7f78adffb640 1 -- 172.21.15.5:0/3449966378 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7f78b408a680 con 0x7f78b8070850 2024-05-20T18:30:39.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.432+0000 7f78bcb85640 1 --2- 172.21.15.5:0/3449966378 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f789c06be30 0x7f789c06e340 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f78a8034000 tx=0x7f78a802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:39.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.433+0000 7f78c656a740 1 Processor -- start 2024-05-20T18:30:39.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.433+0000 7f78c656a740 1 -- start start 2024-05-20T18:30:39.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.435+0000 7f78c656a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdccef4390 0x55fdcd2beb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.435+0000 7f78c656a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fdccecb4d0 0x55fdcd2d9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.435+0000 7f78c656a740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fdcced36b0 0x55fdcd2d9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.435+0000 7f78c656a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fdcd1f8d00 con 0x55fdccef4390 2024-05-20T18:30:39.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.435+0000 7f78c656a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fdcd1a04c0 con 0x55fdccecb4d0 2024-05-20T18:30:39.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.435+0000 7f78c656a740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55fdcd1f4cc0 con 0x55fdcced36b0 2024-05-20T18:30:39.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.435+0000 7f78bd386640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdccef4390 0x55fdcd2beb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.435+0000 7f78bd386640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdccef4390 0x55fdcd2beb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52662/0 (socket says 172.21.15.5:52662) 2024-05-20T18:30:39.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.435+0000 7f78bd386640 1 -- 172.21.15.5:0/2678090067 learned_addr learned my addr 172.21.15.5:0/2678090067 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:39.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.435+0000 7f78bdb87640 1 --2- 172.21.15.5:0/2678090067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fdcced36b0 0x55fdcd2d9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.435+0000 7f78bcb85640 1 --2- 172.21.15.5:0/2678090067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fdccecb4d0 0x55fdcd2d9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.435+0000 7f78bd386640 1 -- 172.21.15.5:0/2678090067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fdcced36b0 msgr2=0x55fdcd2d9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:39.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.435+0000 7f78bd386640 1 --2- 172.21.15.5:0/2678090067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fdcced36b0 0x55fdcd2d9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.435+0000 7f78bd386640 1 -- 172.21.15.5:0/2678090067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fdccecb4d0 msgr2=0x55fdcd2d9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:39.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.435+0000 7f78bd386640 1 --2- 172.21.15.5:0/2678090067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fdccecb4d0 0x55fdcd2d9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.435+0000 7f78bd386640 1 -- 172.21.15.5:0/2678090067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdccfd67d0 con 0x55fdccef4390 2024-05-20T18:30:39.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.435+0000 7f78bd386640 1 --2- 172.21.15.5:0/2678090067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdccef4390 0x55fdcd2beb60 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f78b40070a0 tx=0x7f78b4058860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70250a3a5ef29fb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:39.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.435+0000 7f789affd640 1 -- 172.21.15.5:0/2678090067 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b405f070 con 0x55fdccef4390 2024-05-20T18:30:39.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.435+0000 7f789affd640 1 -- 172.21.15.5:0/2678090067 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b4056aa0 con 0x55fdccef4390 2024-05-20T18:30:39.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.435+0000 7f789affd640 1 -- 172.21.15.5:0/2678090067 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4058a00 con 0x55fdccef4390 2024-05-20T18:30:39.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.436+0000 7f78c656a740 1 -- 172.21.15.5:0/2678090067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdccef4390 msgr2=0x55fdcd2beb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:39.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.436+0000 7f78c656a740 1 --2- 172.21.15.5:0/2678090067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdccef4390 0x55fdcd2beb60 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f78b40070a0 tx=0x7f78b4058860 comp rx=0 tx=0).stop 2024-05-20T18:30:39.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.436+0000 7f78c656a740 1 -- 172.21.15.5:0/2678090067 shutdown_connections 2024-05-20T18:30:39.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.436+0000 7f78c656a740 1 --2- 172.21.15.5:0/2678090067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fdcced36b0 0x55fdcd2d9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.436+0000 7f78c656a740 1 --2- 172.21.15.5:0/2678090067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fdccecb4d0 0x55fdcd2d9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.436+0000 7f78c656a740 1 --2- 172.21.15.5:0/2678090067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdccef4390 0x55fdcd2beb60 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.436+0000 7f78c656a740 1 -- 172.21.15.5:0/2678090067 >> 172.21.15.5:0/2678090067 conn(0x55fdccee84e0 msgr2=0x55fdcd2941c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:39.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.436+0000 7f78c656a740 1 -- 172.21.15.5:0/2678090067 shutdown_connections 2024-05-20T18:30:39.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.436+0000 7f78c656a740 1 -- 172.21.15.5:0/2678090067 wait complete. 2024-05-20T18:30:39.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.437+0000 7f78c656a740 1 -- start start 2024-05-20T18:30:39.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.437+0000 7f78c656a740 1 Processor -- start 2024-05-20T18:30:39.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.439+0000 7f78c656a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdccecb4d0 0x55fdcd2d3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.439+0000 7f78c656a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fdcced36b0 0x55fdcd2d1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.439+0000 7f78c656a740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fdccef4390 0x55fdcd2b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.439+0000 7f78c656a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fdcd1b3b90 con 0x55fdccecb4d0 2024-05-20T18:30:39.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.439+0000 7f78c656a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fdcd2b68c0 con 0x55fdcced36b0 2024-05-20T18:30:39.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.439+0000 7f78c656a740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55fdcd2d22a0 con 0x55fdccef4390 2024-05-20T18:30:39.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.439+0000 7f78c656a740 10 client.0 _ll_register_callbacks cb 0x55fdcd1efda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:30:39.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.439+0000 7f78bd386640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdccecb4d0 0x55fdcd2d3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.439+0000 7f78bd386640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdccecb4d0 0x55fdcd2d3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52676/0 (socket says 172.21.15.5:52676) 2024-05-20T18:30:39.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.439+0000 7f78bd386640 1 -- 172.21.15.5:0/3018271573 learned_addr learned my addr 172.21.15.5:0/3018271573 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:39.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.439+0000 7f78bcb85640 1 --2- 172.21.15.5:0/3018271573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fdcced36b0 0x55fdcd2d1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.439+0000 7f78bdb87640 1 --2- 172.21.15.5:0/3018271573 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fdccef4390 0x55fdcd2b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.439+0000 7f78bd386640 1 -- 172.21.15.5:0/3018271573 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fdccef4390 msgr2=0x55fdcd2b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:39.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.439+0000 7f78bd386640 1 --2- 172.21.15.5:0/3018271573 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fdccef4390 0x55fdcd2b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.439+0000 7f78bd386640 1 -- 172.21.15.5:0/3018271573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fdcced36b0 msgr2=0x55fdcd2d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:39.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.439+0000 7f78bd386640 1 --2- 172.21.15.5:0/3018271573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fdcced36b0 0x55fdcd2d1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.439+0000 7f78bd386640 1 -- 172.21.15.5:0/3018271573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdccf0c3f0 con 0x55fdccecb4d0 2024-05-20T18:30:39.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.439+0000 7f78bcb85640 1 --2- 172.21.15.5:0/3018271573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fdcced36b0 0x55fdcd2d1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:30:39.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.439+0000 7f78bd386640 1 --2- 172.21.15.5:0/3018271573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdccecb4d0 0x55fdcd2d3430 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f78b40586e0 tx=0x7f78b4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a40d4cfc3bf3dd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:39.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.439+0000 7f7893fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:30:39.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.440+0000 7f7893fff640 1 -- 172.21.15.5:0/3018271573 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4060ef0 con 0x55fdccecb4d0 2024-05-20T18:30:39.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.440+0000 7f7893fff640 1 -- 172.21.15.5:0/3018271573 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7f78b4061090 con 0x55fdccecb4d0 2024-05-20T18:30:39.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.440+0000 7f78c656a740 1 -- 172.21.15.5:0/3018271573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78b4056aa0 con 0x55fdccecb4d0 2024-05-20T18:30:39.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.440+0000 7f7893fff640 1 -- 172.21.15.5:0/3018271573 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f78b4004ba0 con 0x55fdccecb4d0 2024-05-20T18:30:39.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.440+0000 7f78c656a740 10 client.6432 fetch_fsmap learned FSMap version 35 2024-05-20T18:30:39.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.440+0000 7f78c656a740 1 -- 172.21.15.5:0/3018271573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55fdccecc0f0 con 0x55fdccecb4d0 2024-05-20T18:30:39.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.440+0000 7f7893fff640 1 -- 172.21.15.5:0/3018271573 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b4056aa0 con 0x55fdccecb4d0 2024-05-20T18:30:39.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.440+0000 7f7893fff640 1 -- 172.21.15.5:0/3018271573 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f78b405dce0 con 0x55fdccecb4d0 2024-05-20T18:30:39.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.440+0000 7f78c656a740 10 client.6432 fetch_fsmap finished waiting for FSMap version 35 2024-05-20T18:30:39.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.440+0000 7f78c656a740 10 client.6432 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-20T18:30:39.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.440+0000 7f78c656a740 20 client.6432 populate_metadata read hostname 'smithi005' 2024-05-20T18:30:39.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.440+0000 7f78c656a740 1 --2- 172.21.15.5:0/3018271573 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55fdcceb2f60 0x55fdcd28e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.440+0000 7f78c656a740 4 client.6432 mds_command: new command op to 6198 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:30:39.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.440+0000 7f78c656a740 1 -- 172.21.15.5:0/3018271573 --> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdcd206430 con 0x55fdcceb2f60 2024-05-20T18:30:39.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.440+0000 7f78bcb85640 1 --2- 172.21.15.5:0/3018271573 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55fdcceb2f60 0x55fdcd28e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.441+0000 7f78bcb85640 1 --2- 172.21.15.5:0/3018271573 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55fdcceb2f60 0x55fdcd28e6d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a73f8a25dc11d815 server_cookie=92fb89aeb42cbbe6 in_seq=0 out_seq=0 2024-05-20T18:30:39.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.441+0000 7f7893fff640 10 client.6432 ms_handle_connect on v2:172.21.15.63:6832/4036379300 2024-05-20T18:30:39.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.442+0000 7f7893fff640 1 -- 172.21.15.5:0/3018271573 <== mds.0 v2:172.21.15.63:6832/4036379300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fdcd206430 con 0x55fdcceb2f60 2024-05-20T18:30:39.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.442+0000 7f7893fff640 10 client.6432 handle_command_reply: tid=0 2024-05-20T18:30:39.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.442+0000 7f78c656a740 1 client.6432 shutdown 2024-05-20T18:30:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.442+0000 7f78c656a740 1 -- 172.21.15.5:0/3018271573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdccecb4d0 msgr2=0x55fdcd2d3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.442+0000 7f78c656a740 1 --2- 172.21.15.5:0/3018271573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdccecb4d0 0x55fdcd2d3430 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f78b40586e0 tx=0x7f78b4000bc0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.442+0000 7f78c656a740 1 -- 172.21.15.5:0/3018271573 shutdown_connections 2024-05-20T18:30:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.442+0000 7f78c656a740 1 --2- 172.21.15.5:0/3018271573 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55fdcceb2f60 0x55fdcd28e6d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.442+0000 7f78c656a740 1 --2- 172.21.15.5:0/3018271573 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fdccef4390 0x55fdcd2b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.442+0000 7f78c656a740 1 --2- 172.21.15.5:0/3018271573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fdcced36b0 0x55fdcd2d1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.442+0000 7f7893fff640 0 client.6432 ms_handle_reset on v2:172.21.15.63:6832/4036379300 2024-05-20T18:30:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.442+0000 7f78c656a740 1 --2- 172.21.15.5:0/3018271573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdccecb4d0 0x55fdcd2d3430 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.442+0000 7f78c656a740 1 -- 172.21.15.5:0/3018271573 >> 172.21.15.5:0/3018271573 conn(0x55fdccee84e0 msgr2=0x55fdcd2941c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.442+0000 7f78c656a740 1 -- 172.21.15.5:0/3018271573 shutdown_connections 2024-05-20T18:30:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.442+0000 7f78c656a740 1 -- 172.21.15.5:0/3018271573 wait complete. 2024-05-20T18:30:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.442+0000 7f78c656a740 20 client.6432 trim_cache size 0 max 16384 2024-05-20T18:30:39.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.460+0000 7f78c656a740 1 Processor -- start 2024-05-20T18:30:39.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.461+0000 7f78c656a740 1 -- start start 2024-05-20T18:30:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.462+0000 7f78c656a740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fdccecb4d0 0x55fdcd294900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.462+0000 7f78c656a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fdcced36b0 0x55fdcd292670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.462+0000 7f78c656a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdccef4390 0x55fdcd292bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.462+0000 7f78c656a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fdcd2c2e10 con 0x55fdccef4390 2024-05-20T18:30:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.462+0000 7f78c656a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f78b8202590 con 0x55fdcced36b0 2024-05-20T18:30:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.462+0000 7f78c656a740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f78b82015b0 con 0x55fdccecb4d0 2024-05-20T18:30:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.462+0000 7f78bdb87640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdccef4390 0x55fdcd292bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.462+0000 7f78bdb87640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdccef4390 0x55fdcd292bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52692/0 (socket says 172.21.15.5:52692) 2024-05-20T18:30:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.462+0000 7f78bdb87640 1 -- 172.21.15.5:0/2934221724 learned_addr learned my addr 172.21.15.5:0/2934221724 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.462+0000 7f78bd386640 1 --2- 172.21.15.5:0/2934221724 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fdccecb4d0 0x55fdcd294900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.462+0000 7f78bcb85640 1 --2- 172.21.15.5:0/2934221724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fdcced36b0 0x55fdcd292670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.462+0000 7f78bdb87640 1 -- 172.21.15.5:0/2934221724 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fdccecb4d0 msgr2=0x55fdcd294900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.462+0000 7f78bdb87640 1 --2- 172.21.15.5:0/2934221724 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fdccecb4d0 0x55fdcd294900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.462+0000 7f78bdb87640 1 -- 172.21.15.5:0/2934221724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fdcced36b0 msgr2=0x55fdcd292670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.462+0000 7f78bdb87640 1 --2- 172.21.15.5:0/2934221724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fdcced36b0 0x55fdcd292670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.462+0000 7f78bdb87640 1 -- 172.21.15.5:0/2934221724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdcd2c1280 con 0x55fdccef4390 2024-05-20T18:30:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.462+0000 7f78bd386640 1 --2- 172.21.15.5:0/2934221724 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fdccecb4d0 0x55fdcd294900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:30:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.462+0000 7f78bdb87640 1 --2- 172.21.15.5:0/2934221724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdccef4390 0x55fdcd292bb0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f78b0009400 tx=0x7f78b000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa71bf9479ed1a73 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.463+0000 7f789a7fc640 1 -- 172.21.15.5:0/2934221724 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b0017820 con 0x55fdccef4390 2024-05-20T18:30:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.463+0000 7f789a7fc640 1 -- 172.21.15.5:0/2934221724 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b00179c0 con 0x55fdccef4390 2024-05-20T18:30:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.463+0000 7f789a7fc640 1 -- 172.21.15.5:0/2934221724 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b0017d90 con 0x55fdccef4390 2024-05-20T18:30:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.463+0000 7f78c656a740 1 -- 172.21.15.5:0/2934221724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdccef4390 msgr2=0x55fdcd292bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.463+0000 7f78c656a740 1 --2- 172.21.15.5:0/2934221724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdccef4390 0x55fdcd292bb0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f78b0009400 tx=0x7f78b000b040 comp rx=0 tx=0).stop 2024-05-20T18:30:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.463+0000 7f78c656a740 1 -- 172.21.15.5:0/2934221724 shutdown_connections 2024-05-20T18:30:39.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.463+0000 7f78c656a740 1 --2- 172.21.15.5:0/2934221724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdccef4390 0x55fdcd292bb0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.463+0000 7f78c656a740 1 --2- 172.21.15.5:0/2934221724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fdcced36b0 0x55fdcd292670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.463+0000 7f78c656a740 1 --2- 172.21.15.5:0/2934221724 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fdccecb4d0 0x55fdcd294900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.463+0000 7f78c656a740 1 -- 172.21.15.5:0/2934221724 >> 172.21.15.5:0/2934221724 conn(0x55fdccee84e0 msgr2=0x55fdcd2940f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:39.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.463+0000 7f78c656a740 1 -- 172.21.15.5:0/2934221724 shutdown_connections 2024-05-20T18:30:39.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.463+0000 7f78c656a740 1 -- 172.21.15.5:0/2934221724 wait complete. 2024-05-20T18:30:39.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.464+0000 7f78c656a740 1 -- start start 2024-05-20T18:30:39.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.464+0000 7f78c656a740 1 Processor -- start 2024-05-20T18:30:39.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.466+0000 7f78c656a740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fdccecb4d0 0x55fdcd2d9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.466+0000 7f78c656a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fdcced36b0 0x55fdcd2f27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.466+0000 7f78c656a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdccef4390 0x55fdcd292670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.466+0000 7f78c656a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78b80b7450 con 0x55fdccef4390 2024-05-20T18:30:39.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.466+0000 7f78c656a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f78b81f9e60 con 0x55fdcced36b0 2024-05-20T18:30:39.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.466+0000 7f78c656a740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55fdcd2b6360 con 0x55fdccecb4d0 2024-05-20T18:30:39.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.466+0000 7f78c656a740 10 client.0 _ll_register_callbacks cb 0x7f78b4058ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:30:39.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.466+0000 7f78bdb87640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdccef4390 0x55fdcd292670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.466+0000 7f78bdb87640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdccef4390 0x55fdcd292670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52706/0 (socket says 172.21.15.5:52706) 2024-05-20T18:30:39.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.466+0000 7f78bdb87640 1 -- 172.21.15.5:0/1858536729 learned_addr learned my addr 172.21.15.5:0/1858536729 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:39.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.466+0000 7f78bd386640 1 --2- 172.21.15.5:0/1858536729 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fdccecb4d0 0x55fdcd2d9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.466+0000 7f78bcb85640 1 --2- 172.21.15.5:0/1858536729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fdcced36b0 0x55fdcd2f27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.466+0000 7f78bdb87640 1 -- 172.21.15.5:0/1858536729 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fdccecb4d0 msgr2=0x55fdcd2d9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.466+0000 7f78bdb87640 1 --2- 172.21.15.5:0/1858536729 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fdccecb4d0 0x55fdcd2d9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.466+0000 7f78bdb87640 1 -- 172.21.15.5:0/1858536729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fdcced36b0 msgr2=0x55fdcd2f27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.466+0000 7f78bdb87640 1 --2- 172.21.15.5:0/1858536729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fdcced36b0 0x55fdcd2f27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.466+0000 7f78bdb87640 1 -- 172.21.15.5:0/1858536729 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdcd294b50 con 0x55fdccef4390 2024-05-20T18:30:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.466+0000 7f78bdb87640 1 --2- 172.21.15.5:0/1858536729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdccef4390 0x55fdcd292670 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f78b0009e00 tx=0x7f78b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ea8eafa6d4bbd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.466+0000 7f7893fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:30:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.467+0000 7f7893fff640 1 -- 172.21.15.5:0/1858536729 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b002d060 con 0x55fdccef4390 2024-05-20T18:30:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.467+0000 7f7893fff640 1 -- 172.21.15.5:0/1858536729 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7f78b001f800 con 0x55fdccef4390 2024-05-20T18:30:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.467+0000 7f78c656a740 1 -- 172.21.15.5:0/1858536729 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78b00179c0 con 0x55fdccef4390 2024-05-20T18:30:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.467+0000 7f7893fff640 1 -- 172.21.15.5:0/1858536729 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f78b0033050 con 0x55fdccef4390 2024-05-20T18:30:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.467+0000 7f78c656a740 10 client.6438 fetch_fsmap learned FSMap version 35 2024-05-20T18:30:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.467+0000 7f78c656a740 1 -- 172.21.15.5:0/1858536729 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55fdcd2f2d10 con 0x55fdccef4390 2024-05-20T18:30:39.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.467+0000 7f7893fff640 1 -- 172.21.15.5:0/1858536729 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b00179c0 con 0x55fdccef4390 2024-05-20T18:30:39.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.467+0000 7f7893fff640 1 -- 172.21.15.5:0/1858536729 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f78b0036020 con 0x55fdccef4390 2024-05-20T18:30:39.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.467+0000 7f78c656a740 10 client.6438 fetch_fsmap finished waiting for FSMap version 35 2024-05-20T18:30:39.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.467+0000 7f78c656a740 10 client.6438 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-20T18:30:39.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.467+0000 7f78c656a740 20 client.6438 populate_metadata read hostname 'smithi005' 2024-05-20T18:30:39.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.467+0000 7f78c656a740 1 --2- 172.21.15.5:0/1858536729 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55fdcd355f30 0x55fdcd292bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.467+0000 7f78c656a740 4 client.6438 mds_command: new command op to 6198 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T18:30:39.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.467+0000 7f78c656a740 1 -- 172.21.15.5:0/1858536729 --> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fdcd2dbd80 con 0x55fdcd355f30 2024-05-20T18:30:39.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.467+0000 7f78bd386640 1 --2- 172.21.15.5:0/1858536729 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55fdcd355f30 0x55fdcd292bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.468+0000 7f78bd386640 1 --2- 172.21.15.5:0/1858536729 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55fdcd355f30 0x55fdcd292bb0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=547ea443baaba4ed server_cookie=e564b7a8591bcbff in_seq=0 out_seq=0 2024-05-20T18:30:39.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.468+0000 7f7893fff640 10 client.6438 ms_handle_connect on v2:172.21.15.63:6832/4036379300 2024-05-20T18:30:39.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.468+0000 7f7893fff640 1 -- 172.21.15.5:0/1858536729 <== mds.0 v2:172.21.15.63:6832/4036379300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fdcd2dbd80 con 0x55fdcd355f30 2024-05-20T18:30:39.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.468+0000 7f7893fff640 10 client.6438 handle_command_reply: tid=0 2024-05-20T18:30:39.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.468+0000 7f78c656a740 1 client.6438 shutdown 2024-05-20T18:30:39.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.468+0000 7f78c656a740 1 -- 172.21.15.5:0/1858536729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdccef4390 msgr2=0x55fdcd292670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:39.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.468+0000 7f78c656a740 1 --2- 172.21.15.5:0/1858536729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdccef4390 0x55fdcd292670 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f78b0009e00 tx=0x7f78b0004a40 comp rx=0 tx=0).stop 2024-05-20T18:30:39.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.468+0000 7f78c656a740 1 -- 172.21.15.5:0/1858536729 shutdown_connections 2024-05-20T18:30:39.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.468+0000 7f78c656a740 1 --2- 172.21.15.5:0/1858536729 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55fdcd355f30 0x55fdcd292bb0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.468+0000 7f78c656a740 1 --2- 172.21.15.5:0/1858536729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdccef4390 0x55fdcd292670 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.468+0000 7f78c656a740 1 --2- 172.21.15.5:0/1858536729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fdcced36b0 0x55fdcd2f27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.468+0000 7f7893fff640 0 client.6438 ms_handle_reset on v2:172.21.15.63:6832/4036379300 2024-05-20T18:30:39.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.468+0000 7f78c656a740 1 --2- 172.21.15.5:0/1858536729 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fdccecb4d0 0x55fdcd2d9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.468+0000 7f78c656a740 1 -- 172.21.15.5:0/1858536729 >> 172.21.15.5:0/1858536729 conn(0x55fdccee84e0 msgr2=0x55fdcd2940f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:39.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.468+0000 7f78c656a740 1 -- 172.21.15.5:0/1858536729 shutdown_connections 2024-05-20T18:30:39.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.468+0000 7f78c656a740 1 -- 172.21.15.5:0/1858536729 wait complete. 2024-05-20T18:30:39.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.468+0000 7f78c656a740 20 client.6438 trim_cache size 0 max 16384 2024-05-20T18:30:39.475 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:30:39.475 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-20T18:30:39.475 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-20T18:30:39.475 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:30:39.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.469+0000 7f78bf5ea640 1 -- 172.21.15.5:0/3449966378 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f789c06be30 msgr2=0x7f789c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:39.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.469+0000 7f78bf5ea640 1 --2- 172.21.15.5:0/3449966378 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f789c06be30 0x7f789c06e340 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f78a8034000 tx=0x7f78a802e000 comp rx=0 tx=0).stop 2024-05-20T18:30:39.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.469+0000 7f78bf5ea640 1 -- 172.21.15.5:0/3449966378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78b8070850 msgr2=0x7f78b81570b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:39.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.469+0000 7f78bf5ea640 1 --2- 172.21.15.5:0/3449966378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78b8070850 0x7f78b81570b0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f78b400c470 tx=0x7f78b400c930 comp rx=0 tx=0).stop 2024-05-20T18:30:39.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.469+0000 7f78bf5ea640 1 -- 172.21.15.5:0/3449966378 shutdown_connections 2024-05-20T18:30:39.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.469+0000 7f78bf5ea640 1 --2- 172.21.15.5:0/3449966378 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f789c06be30 0x7f789c06e340 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.469+0000 7f78bf5ea640 1 --2- 172.21.15.5:0/3449966378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78b815d530 0x7f78b806c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.469+0000 7f78bf5ea640 1 --2- 172.21.15.5:0/3449966378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f78b80b1e10 0x7f78b815c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.469+0000 7f78bf5ea640 1 --2- 172.21.15.5:0/3449966378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78b8070850 0x7f78b81570b0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.469+0000 7f78bf5ea640 1 -- 172.21.15.5:0/3449966378 >> 172.21.15.5:0/3449966378 conn(0x7f78b80d29d0 msgr2=0x7f78b80d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:39.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.469+0000 7f78bf5ea640 1 -- 172.21.15.5:0/3449966378 shutdown_connections 2024-05-20T18:30:39.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.469+0000 7f78bf5ea640 1 -- 172.21.15.5:0/3449966378 wait complete. 2024-05-20T18:30:39.480 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T18:30:39.481 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:30:39.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.595+0000 7f953e9ce640 1 Processor -- start 2024-05-20T18:30:39.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.595+0000 7f953e9ce640 1 -- start start 2024-05-20T18:30:39.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.597+0000 7f953e9ce640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9538168050 0x7f953816a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.597+0000 7f953e9ce640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95380d0630 con 0x7f953806d3d0 2024-05-20T18:30:39.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.597+0000 7f953e9ce640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95380d39c0 con 0x7f9538162820 2024-05-20T18:30:39.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.597+0000 7f953e9ce640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f95380d3ca0 con 0x7f9538168050 2024-05-20T18:30:39.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.597+0000 7f953cf6b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9538168050 0x7f953816a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.597+0000 7f95377fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f953806d3d0 0x7f953806d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56920/0 (socket says 172.21.15.5:56920) 2024-05-20T18:30:39.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.597+0000 7f95377fe640 1 -- 172.21.15.5:0/3270484368 learned_addr learned my addr 172.21.15.5:0/3270484368 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:30:39.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.597+0000 7f953cf6b640 1 -- 172.21.15.5:0/3270484368 >> v1:172.21.15.63:6789/0 conn(0x7f9538162820 legacy=0x7f95380b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:39.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.597+0000 7f953cf6b640 1 -- 172.21.15.5:0/3270484368 >> v1:172.21.15.5:6789/0 conn(0x7f953806d3d0 legacy=0x7f953806d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:39.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.597+0000 7f953cf6b640 1 -- 172.21.15.5:0/3270484368 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95380d3f80 con 0x7f9538168050 2024-05-20T18:30:39.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.598+0000 7f953cf6b640 1 --2- 172.21.15.5:0/3270484368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9538168050 0x7f953816a470 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9530009d90 tx=0x7f953002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78e280a2d98708b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:39.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.598+0000 7f9536ffd640 1 -- 172.21.15.5:0/3270484368 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530007740 con 0x7f9538168050 2024-05-20T18:30:39.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.598+0000 7f9536ffd640 1 -- 172.21.15.5:0/3270484368 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95300078e0 con 0x7f9538168050 2024-05-20T18:30:39.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.598+0000 7f9536ffd640 1 -- 172.21.15.5:0/3270484368 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530007cb0 con 0x7f9538168050 2024-05-20T18:30:39.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.599+0000 7f953e9ce640 1 -- 172.21.15.5:0/3270484368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9538168050 msgr2=0x7f953816a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:39.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.599+0000 7f953e9ce640 1 --2- 172.21.15.5:0/3270484368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9538168050 0x7f953816a470 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9530009d90 tx=0x7f953002a6a0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.599+0000 7f953e9ce640 1 -- 172.21.15.5:0/3270484368 shutdown_connections 2024-05-20T18:30:39.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.599+0000 7f953e9ce640 1 --2- 172.21.15.5:0/3270484368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9538168050 0x7f953816a470 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.599+0000 7f953e9ce640 1 -- 172.21.15.5:0/3270484368 >> 172.21.15.5:0/3270484368 conn(0x7f95380d4bd0 msgr2=0x7f95380d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:39.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.599+0000 7f953e9ce640 1 -- 172.21.15.5:0/3270484368 shutdown_connections 2024-05-20T18:30:39.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.599+0000 7f953e9ce640 1 -- 172.21.15.5:0/3270484368 wait complete. 2024-05-20T18:30:39.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.601+0000 7f953e9ce640 1 Processor -- start 2024-05-20T18:30:39.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.601+0000 7f953e9ce640 1 -- start start 2024-05-20T18:30:39.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.602+0000 7f953e9ce640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f953806d3d0 0x7f95380cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.602+0000 7f953e9ce640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9538162820 0x7f95380b63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.602+0000 7f953e9ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9538168050 0x7f9538155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.602+0000 7f953e9ce640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f95380b91a0 con 0x7f9538168050 2024-05-20T18:30:39.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.602+0000 7f953e9ce640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f95380d0300 con 0x7f953806d3d0 2024-05-20T18:30:39.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.602+0000 7f953e9ce640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f95380b4050 con 0x7f9538162820 2024-05-20T18:30:39.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.602+0000 7f953cf6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9538168050 0x7f9538155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.602+0000 7f953cf6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9538168050 0x7f9538155050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52726/0 (socket says 172.21.15.5:52726) 2024-05-20T18:30:39.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.602+0000 7f953cf6b640 1 -- 172.21.15.5:0/302242224 learned_addr learned my addr 172.21.15.5:0/302242224 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:39.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.603+0000 7f95377fe640 1 --2- 172.21.15.5:0/302242224 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9538162820 0x7f95380b63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.603+0000 7f9537fff640 1 --2- 172.21.15.5:0/302242224 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f953806d3d0 0x7f95380cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.603+0000 7f953cf6b640 1 -- 172.21.15.5:0/302242224 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9538162820 msgr2=0x7f95380b63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:39.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.603+0000 7f953cf6b640 1 --2- 172.21.15.5:0/302242224 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9538162820 0x7f95380b63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.603+0000 7f953cf6b640 1 -- 172.21.15.5:0/302242224 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f953806d3d0 msgr2=0x7f95380cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:39.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.603+0000 7f953cf6b640 1 --2- 172.21.15.5:0/302242224 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f953806d3d0 0x7f95380cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.603+0000 7f953cf6b640 1 -- 172.21.15.5:0/302242224 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95380cc8c0 con 0x7f9538168050 2024-05-20T18:30:39.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.603+0000 7f953cf6b640 1 --2- 172.21.15.5:0/302242224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9538168050 0x7f9538155050 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f9530009eb0 tx=0x7f9530004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:39.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.603+0000 7f9534ff9640 1 -- 172.21.15.5:0/302242224 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953003f070 con 0x7f9538168050 2024-05-20T18:30:39.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.603+0000 7f9534ff9640 1 -- 172.21.15.5:0/302242224 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9530031c90 con 0x7f9538168050 2024-05-20T18:30:39.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.603+0000 7f953e9ce640 1 -- 172.21.15.5:0/302242224 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95380ccba0 con 0x7f9538168050 2024-05-20T18:30:39.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.603+0000 7f9534ff9640 1 -- 172.21.15.5:0/302242224 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530043630 con 0x7f9538168050 2024-05-20T18:30:39.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.603+0000 7f953e9ce640 1 -- 172.21.15.5:0/302242224 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95380cd160 con 0x7f9538168050 2024-05-20T18:30:39.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.604+0000 7f953e9ce640 1 -- 172.21.15.5:0/302242224 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f953808de70 con 0x7f9538168050 2024-05-20T18:30:39.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.605+0000 7f9534ff9640 1 -- 172.21.15.5:0/302242224 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f953003b410 con 0x7f9538168050 2024-05-20T18:30:39.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.605+0000 7f9534ff9640 1 --2- 172.21.15.5:0/302242224 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f952406be60 0x7f952406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.605+0000 7f9534ff9640 1 -- 172.21.15.5:0/302242224 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7f95300ac5d0 con 0x7f9538168050 2024-05-20T18:30:39.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.605+0000 7f9537fff640 1 --2- 172.21.15.5:0/302242224 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f952406be60 0x7f952406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.606+0000 7f9537fff640 1 --2- 172.21.15.5:0/302242224 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f952406be60 0x7f952406e370 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9528006fd0 tx=0x7f9528008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:39.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.610+0000 7f9534ff9640 1 -- 172.21.15.5:0/302242224 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f953808de70 con 0x7f9538168050 2024-05-20T18:30:39.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.831+0000 7f953e9ce640 1 -- 172.21.15.5:0/302242224 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9538075b00 con 0x7f9538168050 2024-05-20T18:30:39.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.832+0000 7f9534ff9640 1 -- 172.21.15.5:0/302242224 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5176 (secure 0 0 0) 0x7f953007a7c0 con 0x7f9538168050 2024-05-20T18:30:39.832 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:30:39.832 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5480,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/4292435960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":4292435960},{"type":"v1","addr":"172.21.15.63:6835","nonce":4292435960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6186,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2991834087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2991834087},{"type":"v1","addr":"172.21.15.5:6836","nonce":2991834087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6195,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3462206198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":3462206198},{"type":"v1","addr":"172.21.15.5:6837","nonce":3462206198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:30:00.286575+0000","modified":"2024-05-20T18:30:12.940337+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6198":{"gid":6198,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":6,"addr":"172.21.15.63:6834/4036379300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":4036379300},{"type":"v1","addr":"172.21.15.63:6834","nonce":4036379300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6198,"qdb_cluster":[6198]},"id":4}]} 2024-05-20T18:30:39.832 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 35 2024-05-20T18:30:39.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.834+0000 7f953e9ce640 1 -- 172.21.15.5:0/302242224 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f952406be60 msgr2=0x7f952406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:39.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.834+0000 7f953e9ce640 1 --2- 172.21.15.5:0/302242224 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f952406be60 0x7f952406e370 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9528006fd0 tx=0x7f9528008040 comp rx=0 tx=0).stop 2024-05-20T18:30:39.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.834+0000 7f953e9ce640 1 -- 172.21.15.5:0/302242224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9538168050 msgr2=0x7f9538155050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:39.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.834+0000 7f953e9ce640 1 --2- 172.21.15.5:0/302242224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9538168050 0x7f9538155050 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f9530009eb0 tx=0x7f9530004a40 comp rx=0 tx=0).stop 2024-05-20T18:30:39.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.834+0000 7f953e9ce640 1 -- 172.21.15.5:0/302242224 shutdown_connections 2024-05-20T18:30:39.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.834+0000 7f953e9ce640 1 --2- 172.21.15.5:0/302242224 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f952406be60 0x7f952406e370 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.834+0000 7f953e9ce640 1 --2- 172.21.15.5:0/302242224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9538168050 0x7f9538155050 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.834+0000 7f953e9ce640 1 --2- 172.21.15.5:0/302242224 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9538162820 0x7f95380b63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.834+0000 7f953e9ce640 1 --2- 172.21.15.5:0/302242224 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f953806d3d0 0x7f95380cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.834+0000 7f953e9ce640 1 -- 172.21.15.5:0/302242224 >> 172.21.15.5:0/302242224 conn(0x7f95380d4bd0 msgr2=0x7f953815f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:39.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.834+0000 7f953e9ce640 1 -- 172.21.15.5:0/302242224 shutdown_connections 2024-05-20T18:30:39.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.834+0000 7f953e9ce640 1 -- 172.21.15.5:0/302242224 wait complete. 2024-05-20T18:30:39.845 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b damage ls --format=json-pretty 2024-05-20T18:30:39.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.964+0000 7faf7b844640 1 Processor -- start 2024-05-20T18:30:39.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.964+0000 7faf7b844640 1 -- start start 2024-05-20T18:30:39.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.966+0000 7faf7b844640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf741576d0 0x7faf7406c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.966+0000 7faf7b844640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7faf740cedd0 0x7faf7406e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.966+0000 7faf7b844640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf74157ae0 con 0x7faf7415d5f0 2024-05-20T18:30:39.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.966+0000 7faf7b844640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7faf7406cee0 con 0x7faf740cedd0 2024-05-20T18:30:39.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.966+0000 7faf7b844640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faf7406b900 con 0x7faf741576d0 2024-05-20T18:30:39.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.966+0000 7faf79de1640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7faf740cedd0 0x7faf7406e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.966+0000 7faf79de1640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7faf740cedd0 0x7faf7406e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52730/0 (socket says 172.21.15.5:52730) 2024-05-20T18:30:39.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.966+0000 7faf79de1640 1 -- 172.21.15.5:0/465667645 learned_addr learned my addr 172.21.15.5:0/465667645 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:39.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.966+0000 7faf78ddf640 1 --2- 172.21.15.5:0/465667645 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf741576d0 0x7faf7406c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.966+0000 7faf79de1640 1 -- 172.21.15.5:0/465667645 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf741576d0 msgr2=0x7faf7406c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:39.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.966+0000 7faf79de1640 1 --2- 172.21.15.5:0/465667645 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf741576d0 0x7faf7406c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.966+0000 7faf79de1640 1 -- 172.21.15.5:0/465667645 >> v1:172.21.15.63:6789/0 conn(0x7faf7415d5f0 legacy=0x7faf740ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:39.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.966+0000 7faf79de1640 1 -- 172.21.15.5:0/465667645 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf7406bbe0 con 0x7faf740cedd0 2024-05-20T18:30:39.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.967+0000 7faf79de1640 1 --2- 172.21.15.5:0/465667645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf740cedd0 0x7faf7406e3a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7faf6c009e50 tx=0x7faf6c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410bccaf572800e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:39.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.967+0000 7faf6bfff640 1 -- 172.21.15.5:0/465667645 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c036070 con 0x7faf740cedd0 2024-05-20T18:30:39.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.967+0000 7faf6bfff640 1 -- 172.21.15.5:0/465667645 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf6c031600 con 0x7faf740cedd0 2024-05-20T18:30:39.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.967+0000 7faf6bfff640 1 -- 172.21.15.5:0/465667645 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c0319d0 con 0x7faf740cedd0 2024-05-20T18:30:39.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.968+0000 7faf7b844640 1 -- 172.21.15.5:0/465667645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf740cedd0 msgr2=0x7faf7406e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:39.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.968+0000 7faf7b844640 1 --2- 172.21.15.5:0/465667645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf740cedd0 0x7faf7406e3a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7faf6c009e50 tx=0x7faf6c02a680 comp rx=0 tx=0).stop 2024-05-20T18:30:39.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.968+0000 7faf7b844640 1 -- 172.21.15.5:0/465667645 shutdown_connections 2024-05-20T18:30:39.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.968+0000 7faf7b844640 1 --2- 172.21.15.5:0/465667645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf740cedd0 0x7faf7406e3a0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.968+0000 7faf7b844640 1 --2- 172.21.15.5:0/465667645 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf741576d0 0x7faf7406c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.968+0000 7faf7b844640 1 -- 172.21.15.5:0/465667645 >> 172.21.15.5:0/465667645 conn(0x7faf740cd2a0 msgr2=0x7faf740cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:39.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.968+0000 7faf7b844640 1 -- 172.21.15.5:0/465667645 shutdown_connections 2024-05-20T18:30:39.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.968+0000 7faf7b844640 1 -- 172.21.15.5:0/465667645 wait complete. 2024-05-20T18:30:39.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.970+0000 7faf7b844640 1 Processor -- start 2024-05-20T18:30:39.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.970+0000 7faf7b844640 1 -- start start 2024-05-20T18:30:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.971+0000 7faf7b844640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf740cedd0 0x7faf7413d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.971+0000 7faf7b844640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf741576d0 0x7faf74160840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.971+0000 7faf7b844640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf7415d5f0 0x7faf741523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.971+0000 7faf7b844640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faf7406c480 con 0x7faf740cedd0 2024-05-20T18:30:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.971+0000 7faf7b844640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faf7406c1a0 con 0x7faf7415d5f0 2024-05-20T18:30:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.971+0000 7faf7b844640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faf740cf1e0 con 0x7faf741576d0 2024-05-20T18:30:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.971+0000 7faf795e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf740cedd0 0x7faf7413d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.972+0000 7faf795e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf740cedd0 0x7faf7413d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52738/0 (socket says 172.21.15.5:52738) 2024-05-20T18:30:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.972+0000 7faf795e0640 1 -- 172.21.15.5:0/2438638321 learned_addr learned my addr 172.21.15.5:0/2438638321 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.972+0000 7faf79de1640 1 --2- 172.21.15.5:0/2438638321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf7415d5f0 0x7faf741523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.972+0000 7faf78ddf640 1 --2- 172.21.15.5:0/2438638321 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf741576d0 0x7faf74160840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.972+0000 7faf795e0640 1 -- 172.21.15.5:0/2438638321 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf741576d0 msgr2=0x7faf74160840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:39.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.972+0000 7faf795e0640 1 --2- 172.21.15.5:0/2438638321 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf741576d0 0x7faf74160840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.972+0000 7faf795e0640 1 -- 172.21.15.5:0/2438638321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf7415d5f0 msgr2=0x7faf741523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:39.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.972+0000 7faf795e0640 1 --2- 172.21.15.5:0/2438638321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf7415d5f0 0x7faf741523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.972+0000 7faf795e0640 1 -- 172.21.15.5:0/2438638321 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf74160d80 con 0x7faf740cedd0 2024-05-20T18:30:39.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.972+0000 7faf795e0640 1 --2- 172.21.15.5:0/2438638321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf740cedd0 0x7faf7413d210 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7faf7000c440 tx=0x7faf7000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:39.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.972+0000 7faf69ffb640 1 -- 172.21.15.5:0/2438638321 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf700052d0 con 0x7faf740cedd0 2024-05-20T18:30:39.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.972+0000 7faf69ffb640 1 -- 172.21.15.5:0/2438638321 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf70005470 con 0x7faf740cedd0 2024-05-20T18:30:39.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.972+0000 7faf7b844640 1 -- 172.21.15.5:0/2438638321 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf740afef0 con 0x7faf740cedd0 2024-05-20T18:30:39.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.972+0000 7faf69ffb640 1 -- 172.21.15.5:0/2438638321 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf70007260 con 0x7faf740cedd0 2024-05-20T18:30:39.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.972+0000 7faf7b844640 1 -- 172.21.15.5:0/2438638321 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf74156cf0 con 0x7faf740cedd0 2024-05-20T18:30:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.974+0000 7faf69ffb640 1 -- 172.21.15.5:0/2438638321 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faf70007400 con 0x7faf740cedd0 2024-05-20T18:30:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.975+0000 7faf69ffb640 1 --2- 172.21.15.5:0/2438638321 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faf5806be60 0x7faf5806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.975+0000 7faf69ffb640 1 -- 172.21.15.5:0/2438638321 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7faf7008a400 con 0x7faf740cedd0 2024-05-20T18:30:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.975+0000 7faf78ddf640 1 --2- 172.21.15.5:0/2438638321 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faf5806be60 0x7faf5806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.975+0000 7faf78ddf640 1 --2- 172.21.15.5:0/2438638321 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faf5806be60 0x7faf5806e370 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7faf74156750 tx=0x7faf64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:39.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.976+0000 7faf827c9740 1 Processor -- start 2024-05-20T18:30:39.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.976+0000 7faf827c9740 1 -- start start 2024-05-20T18:30:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.977+0000 7faf827c9740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55855ac13350 0x55855afde670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.977+0000 7faf827c9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55855abea490 0x55855aff93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.977+0000 7faf827c9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55855abf2670 0x55855aff98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.977+0000 7faf827c9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55855af17a30 con 0x55855abf2670 2024-05-20T18:30:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.977+0000 7faf827c9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55855aeb5590 con 0x55855abea490 2024-05-20T18:30:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.977+0000 7faf827c9740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55855af139f0 con 0x55855ac13350 2024-05-20T18:30:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.977+0000 7faf79de1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55855abf2670 0x55855aff98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.977+0000 7faf79de1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55855abf2670 0x55855aff98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52740/0 (socket says 172.21.15.5:52740) 2024-05-20T18:30:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.977+0000 7faf79de1640 1 -- 172.21.15.5:0/859880878 learned_addr learned my addr 172.21.15.5:0/859880878 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.978+0000 7faf78ddf640 1 --2- 172.21.15.5:0/859880878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55855abea490 0x55855aff93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.978+0000 7faf795e0640 1 --2- 172.21.15.5:0/859880878 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55855ac13350 0x55855afde670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.978+0000 7faf79de1640 1 -- 172.21.15.5:0/859880878 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55855ac13350 msgr2=0x55855afde670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.978+0000 7faf79de1640 1 --2- 172.21.15.5:0/859880878 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55855ac13350 0x55855afde670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.978+0000 7faf79de1640 1 -- 172.21.15.5:0/859880878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55855abea490 msgr2=0x55855aff93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.978+0000 7faf79de1640 1 --2- 172.21.15.5:0/859880878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55855abea490 0x55855aff93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.978+0000 7faf79de1640 1 -- 172.21.15.5:0/859880878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55855acf77c0 con 0x55855abf2670 2024-05-20T18:30:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.978+0000 7faf79de1640 1 --2- 172.21.15.5:0/859880878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55855abf2670 0x55855aff98e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7faf6c009f70 tx=0x7faf6c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dedd933e4e6900a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.978+0000 7faf5effd640 1 -- 172.21.15.5:0/859880878 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c036040 con 0x55855abf2670 2024-05-20T18:30:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.978+0000 7faf5effd640 1 -- 172.21.15.5:0/859880878 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf6c037040 con 0x55855abf2670 2024-05-20T18:30:39.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.978+0000 7faf5effd640 1 -- 172.21.15.5:0/859880878 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c031920 con 0x55855abf2670 2024-05-20T18:30:39.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.979+0000 7faf827c9740 1 -- 172.21.15.5:0/859880878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55855abf2670 msgr2=0x55855aff98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:39.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.979+0000 7faf827c9740 1 --2- 172.21.15.5:0/859880878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55855abf2670 0x55855aff98e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7faf6c009f70 tx=0x7faf6c02e000 comp rx=0 tx=0).stop 2024-05-20T18:30:39.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.979+0000 7faf827c9740 1 -- 172.21.15.5:0/859880878 shutdown_connections 2024-05-20T18:30:39.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.979+0000 7faf827c9740 1 --2- 172.21.15.5:0/859880878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55855abf2670 0x55855aff98e0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.979+0000 7faf827c9740 1 --2- 172.21.15.5:0/859880878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55855abea490 0x55855aff93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.979+0000 7faf827c9740 1 --2- 172.21.15.5:0/859880878 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55855ac13350 0x55855afde670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.979+0000 7faf827c9740 1 -- 172.21.15.5:0/859880878 >> 172.21.15.5:0/859880878 conn(0x55855ac074a0 msgr2=0x55855afb3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:39.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.979+0000 7faf827c9740 1 -- 172.21.15.5:0/859880878 shutdown_connections 2024-05-20T18:30:39.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.979+0000 7faf827c9740 1 -- 172.21.15.5:0/859880878 wait complete. 2024-05-20T18:30:39.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.980+0000 7faf827c9740 1 -- start start 2024-05-20T18:30:39.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.980+0000 7faf827c9740 1 Processor -- start 2024-05-20T18:30:39.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.981+0000 7faf827c9740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55855abea490 0x55855afdde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.981+0000 7faf827c9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55855abf2670 0x55855afb4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.981+0000 7faf827c9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55855ac13350 0x55855afb4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.981+0000 7faf827c9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55855aeb3060 con 0x55855ac13350 2024-05-20T18:30:39.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.981+0000 7faf827c9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55855b009b80 con 0x55855abf2670 2024-05-20T18:30:39.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.981+0000 7faf827c9740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55855afd8020 con 0x55855abea490 2024-05-20T18:30:39.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.982+0000 7faf827c9740 10 client.0 _ll_register_callbacks cb 0x55855aed7dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:30:39.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.982+0000 7faf79de1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55855ac13350 0x55855afb4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.982+0000 7faf79de1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55855ac13350 0x55855afb4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52754/0 (socket says 172.21.15.5:52754) 2024-05-20T18:30:39.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.982+0000 7faf79de1640 1 -- 172.21.15.5:0/246816182 learned_addr learned my addr 172.21.15.5:0/246816182 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:39.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.982+0000 7faf795e0640 1 --2- 172.21.15.5:0/246816182 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55855abea490 0x55855afdde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.982+0000 7faf78ddf640 1 --2- 172.21.15.5:0/246816182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55855abf2670 0x55855afb4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.982+0000 7faf79de1640 1 -- 172.21.15.5:0/246816182 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55855abea490 msgr2=0x55855afdde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:39.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.982+0000 7faf79de1640 1 --2- 172.21.15.5:0/246816182 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55855abea490 0x55855afdde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.982+0000 7faf79de1640 1 -- 172.21.15.5:0/246816182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55855abf2670 msgr2=0x55855afb4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:39.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.982+0000 7faf79de1640 1 --2- 172.21.15.5:0/246816182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55855abf2670 0x55855afb4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.982+0000 7faf79de1640 1 -- 172.21.15.5:0/246816182 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55855ad57f50 con 0x55855ac13350 2024-05-20T18:30:39.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.982+0000 7faf78ddf640 1 --2- 172.21.15.5:0/246816182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55855abf2670 0x55855afb4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:30:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.982+0000 7faf79de1640 1 --2- 172.21.15.5:0/246816182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55855ac13350 0x55855afb4770 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7faf6c009f40 tx=0x7faf6c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e3ef3f1952c76d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.982+0000 7faf53fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:30:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.982+0000 7faf53fff640 1 -- 172.21.15.5:0/246816182 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c036040 con 0x55855ac13350 2024-05-20T18:30:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.982+0000 7faf53fff640 1 -- 172.21.15.5:0/246816182 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7faf6c0318d0 con 0x55855ac13350 2024-05-20T18:30:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.982+0000 7faf827c9740 1 -- 172.21.15.5:0/246816182 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf6c037040 con 0x55855ac13350 2024-05-20T18:30:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.983+0000 7faf53fff640 1 -- 172.21.15.5:0/246816182 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7faf6c031ad0 con 0x55855ac13350 2024-05-20T18:30:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.983+0000 7faf827c9740 10 client.6459 fetch_fsmap learned FSMap version 35 2024-05-20T18:30:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.983+0000 7faf827c9740 1 -- 172.21.15.5:0/246816182 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55855abeb0b0 con 0x55855ac13350 2024-05-20T18:30:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.983+0000 7faf53fff640 1 -- 172.21.15.5:0/246816182 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf6c037040 con 0x55855ac13350 2024-05-20T18:30:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.983+0000 7faf53fff640 1 -- 172.21.15.5:0/246816182 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7faf6c031c70 con 0x55855ac13350 2024-05-20T18:30:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.983+0000 7faf827c9740 10 client.6459 fetch_fsmap finished waiting for FSMap version 35 2024-05-20T18:30:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.983+0000 7faf827c9740 10 client.6459 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-20T18:30:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.983+0000 7faf827c9740 20 client.6459 populate_metadata read hostname 'smithi005' 2024-05-20T18:30:39.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.983+0000 7faf827c9740 1 --2- 172.21.15.5:0/246816182 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55855abd2130 0x55855afdc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:39.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.983+0000 7faf827c9740 4 client.6459 mds_command: new command op to 6198 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:30:39.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.983+0000 7faf827c9740 1 -- 172.21.15.5:0/246816182 --> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55855af25160 con 0x55855abd2130 2024-05-20T18:30:39.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.983+0000 7faf795e0640 1 --2- 172.21.15.5:0/246816182 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55855abd2130 0x55855afdc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:39.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.984+0000 7faf53fff640 10 client.6459 ms_handle_connect on v2:172.21.15.63:6832/4036379300 2024-05-20T18:30:39.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.984+0000 7faf795e0640 1 --2- 172.21.15.5:0/246816182 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55855abd2130 0x55855afdc470 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d99abbd614dc3d8e server_cookie=28237996c179d579 in_seq=0 out_seq=0 2024-05-20T18:30:39.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.984+0000 7faf53fff640 1 -- 172.21.15.5:0/246816182 <== mds.0 v2:172.21.15.63:6832/4036379300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55855af25160 con 0x55855abd2130 2024-05-20T18:30:39.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.984+0000 7faf53fff640 10 client.6459 handle_command_reply: tid=0 2024-05-20T18:30:39.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.984+0000 7faf827c9740 1 client.6459 shutdown 2024-05-20T18:30:39.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.984+0000 7faf827c9740 1 -- 172.21.15.5:0/246816182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55855ac13350 msgr2=0x55855afb4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:39.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.984+0000 7faf827c9740 1 --2- 172.21.15.5:0/246816182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55855ac13350 0x55855afb4770 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7faf6c009f40 tx=0x7faf6c004ab0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.984+0000 7faf827c9740 1 -- 172.21.15.5:0/246816182 shutdown_connections 2024-05-20T18:30:39.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.984+0000 7faf827c9740 1 --2- 172.21.15.5:0/246816182 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55855abd2130 0x55855afdc470 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.985+0000 7faf827c9740 1 --2- 172.21.15.5:0/246816182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55855ac13350 0x55855afb4770 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.985+0000 7faf827c9740 1 --2- 172.21.15.5:0/246816182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55855abf2670 0x55855afb4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.985+0000 7faf53fff640 0 client.6459 ms_handle_reset on v2:172.21.15.63:6832/4036379300 2024-05-20T18:30:39.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.985+0000 7faf827c9740 1 --2- 172.21.15.5:0/246816182 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55855abea490 0x55855afdde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:39.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.985+0000 7faf827c9740 1 -- 172.21.15.5:0/246816182 >> 172.21.15.5:0/246816182 conn(0x55855ac074a0 msgr2=0x55855afb2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:39.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.985+0000 7faf827c9740 1 -- 172.21.15.5:0/246816182 shutdown_connections 2024-05-20T18:30:39.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.985+0000 7faf827c9740 1 -- 172.21.15.5:0/246816182 wait complete. 2024-05-20T18:30:39.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.985+0000 7faf827c9740 20 client.6459 trim_cache size 0 max 16384 2024-05-20T18:30:39.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.999+0000 7faf827c9740 1 Processor -- start 2024-05-20T18:30:39.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:39.999+0000 7faf827c9740 1 -- start start 2024-05-20T18:30:40.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.000+0000 7faf827c9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55855abea490 0x55855aff9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.000+0000 7faf827c9740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55855abf2670 0x55855afdcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.000+0000 7faf827c9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55855ac13350 0x55855afdd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.000+0000 7faf827c9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faf741ff1b0 con 0x55855abea490 2024-05-20T18:30:40.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.000+0000 7faf827c9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faf7415fa00 con 0x55855ac13350 2024-05-20T18:30:40.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.000+0000 7faf827c9740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faf741feb90 con 0x55855abf2670 2024-05-20T18:30:40.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.000+0000 7faf795e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55855abea490 0x55855aff9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.000+0000 7faf795e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55855abea490 0x55855aff9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52764/0 (socket says 172.21.15.5:52764) 2024-05-20T18:30:40.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.000+0000 7faf795e0640 1 -- 172.21.15.5:0/2506123595 learned_addr learned my addr 172.21.15.5:0/2506123595 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:40.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.000+0000 7faf795e0640 1 -- 172.21.15.5:0/2506123595 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55855abf2670 msgr2=0x55855afdcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:40.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.000+0000 7faf795e0640 1 --2- 172.21.15.5:0/2506123595 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55855abf2670 0x55855afdcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.000+0000 7faf795e0640 1 -- 172.21.15.5:0/2506123595 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55855ac13350 msgr2=0x55855afdd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:40.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.000+0000 7faf795e0640 1 --2- 172.21.15.5:0/2506123595 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55855ac13350 0x55855afdd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.000+0000 7faf795e0640 1 -- 172.21.15.5:0/2506123595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55855affaac0 con 0x55855abea490 2024-05-20T18:30:40.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.000+0000 7faf795e0640 1 --2- 172.21.15.5:0/2506123595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55855abea490 0x55855aff9e20 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7faf70058230 tx=0x7faf70000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c16c528257d1fb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:40.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.001+0000 7faf5e7fc640 1 -- 172.21.15.5:0/2506123595 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf70056830 con 0x55855abea490 2024-05-20T18:30:40.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.001+0000 7faf5e7fc640 1 -- 172.21.15.5:0/2506123595 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf7008a0b0 con 0x55855abea490 2024-05-20T18:30:40.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.001+0000 7faf5e7fc640 1 -- 172.21.15.5:0/2506123595 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf700701d0 con 0x55855abea490 2024-05-20T18:30:40.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.001+0000 7faf827c9740 1 -- 172.21.15.5:0/2506123595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55855abea490 msgr2=0x55855aff9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:40.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.001+0000 7faf827c9740 1 --2- 172.21.15.5:0/2506123595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55855abea490 0x55855aff9e20 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7faf70058230 tx=0x7faf70000ed0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.001+0000 7faf827c9740 1 -- 172.21.15.5:0/2506123595 shutdown_connections 2024-05-20T18:30:40.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.001+0000 7faf827c9740 1 --2- 172.21.15.5:0/2506123595 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55855ac13350 0x55855afdd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.001+0000 7faf827c9740 1 --2- 172.21.15.5:0/2506123595 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55855abf2670 0x55855afdcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.001+0000 7faf827c9740 1 --2- 172.21.15.5:0/2506123595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55855abea490 0x55855aff9e20 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.001+0000 7faf827c9740 1 -- 172.21.15.5:0/2506123595 >> 172.21.15.5:0/2506123595 conn(0x55855ac074a0 msgr2=0x55855afa9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:40.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.001+0000 7faf827c9740 1 -- 172.21.15.5:0/2506123595 shutdown_connections 2024-05-20T18:30:40.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.001+0000 7faf827c9740 1 -- 172.21.15.5:0/2506123595 wait complete. 2024-05-20T18:30:40.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.002+0000 7faf827c9740 1 -- start start 2024-05-20T18:30:40.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.002+0000 7faf827c9740 1 Processor -- start 2024-05-20T18:30:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.004+0000 7faf827c9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55855abea490 0x55855afcacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.004+0000 7faf827c9740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55855abf2670 0x55855afad570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.004+0000 7faf827c9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55855ac13350 0x55855afb2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.004+0000 7faf827c9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55855b008e40 con 0x55855abea490 2024-05-20T18:30:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.004+0000 7faf827c9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55855afcc0f0 con 0x55855ac13350 2024-05-20T18:30:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.004+0000 7faf827c9740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55855b009460 con 0x55855abf2670 2024-05-20T18:30:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.004+0000 7faf827c9740 10 client.0 _ll_register_callbacks cb 0x7faf64007650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:30:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.004+0000 7faf795e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55855abea490 0x55855afcacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.004+0000 7faf795e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55855abea490 0x55855afcacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52772/0 (socket says 172.21.15.5:52772) 2024-05-20T18:30:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.004+0000 7faf795e0640 1 -- 172.21.15.5:0/3700763535 learned_addr learned my addr 172.21.15.5:0/3700763535 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.004+0000 7faf78ddf640 1 --2- 172.21.15.5:0/3700763535 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55855abf2670 0x55855afad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.004+0000 7faf79de1640 1 --2- 172.21.15.5:0/3700763535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55855ac13350 0x55855afb2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.004+0000 7faf795e0640 1 -- 172.21.15.5:0/3700763535 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55855abf2670 msgr2=0x55855afad570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.004+0000 7faf795e0640 1 --2- 172.21.15.5:0/3700763535 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55855abf2670 0x55855afad570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.004+0000 7faf795e0640 1 -- 172.21.15.5:0/3700763535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55855ac13350 msgr2=0x55855afb2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.004+0000 7faf795e0640 1 --2- 172.21.15.5:0/3700763535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55855ac13350 0x55855afb2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.004+0000 7faf795e0640 1 -- 172.21.15.5:0/3700763535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55855b0099b0 con 0x55855abea490 2024-05-20T18:30:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.004+0000 7faf78ddf640 1 --2- 172.21.15.5:0/3700763535 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55855abf2670 0x55855afad570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:30:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.004+0000 7faf79de1640 1 --2- 172.21.15.5:0/3700763535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55855ac13350 0x55855afb2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:30:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.004+0000 7faf795e0640 1 --2- 172.21.15.5:0/3700763535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55855abea490 0x55855afcacd0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7faf700886d0 tx=0x7faf7000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d38169145c36a98 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.004+0000 7faf53fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:30:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.004+0000 7faf53fff640 1 -- 172.21.15.5:0/3700763535 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf70054ba0 con 0x55855abea490 2024-05-20T18:30:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.004+0000 7faf53fff640 1 -- 172.21.15.5:0/3700763535 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7faf700701d0 con 0x55855abea490 2024-05-20T18:30:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.004+0000 7faf827c9740 1 -- 172.21.15.5:0/3700763535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf7008a0b0 con 0x55855abea490 2024-05-20T18:30:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.005+0000 7faf53fff640 1 -- 172.21.15.5:0/3700763535 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7faf70054d40 con 0x55855abea490 2024-05-20T18:30:40.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.005+0000 7faf827c9740 10 client.6465 fetch_fsmap learned FSMap version 35 2024-05-20T18:30:40.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.005+0000 7faf827c9740 1 -- 172.21.15.5:0/3700763535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55855affb010 con 0x55855abea490 2024-05-20T18:30:40.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.005+0000 7faf53fff640 1 -- 172.21.15.5:0/3700763535 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf7008a0b0 con 0x55855abea490 2024-05-20T18:30:40.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.005+0000 7faf53fff640 1 -- 172.21.15.5:0/3700763535 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7faf70057980 con 0x55855abea490 2024-05-20T18:30:40.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.005+0000 7faf827c9740 10 client.6465 fetch_fsmap finished waiting for FSMap version 35 2024-05-20T18:30:40.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.005+0000 7faf827c9740 10 client.6465 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-20T18:30:40.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.005+0000 7faf827c9740 20 client.6465 populate_metadata read hostname 'smithi005' 2024-05-20T18:30:40.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.005+0000 7faf827c9740 1 --2- 172.21.15.5:0/3700763535 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55855afaeb40 0x55855b011e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.005+0000 7faf827c9740 4 client.6465 mds_command: new command op to 6198 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-20T18:30:40.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.005+0000 7faf827c9740 1 -- 172.21.15.5:0/3700763535 --> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55855b0085b0 con 0x55855afaeb40 2024-05-20T18:30:40.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.005+0000 7faf78ddf640 1 --2- 172.21.15.5:0/3700763535 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55855afaeb40 0x55855b011e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.006+0000 7faf78ddf640 1 --2- 172.21.15.5:0/3700763535 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55855afaeb40 0x55855b011e60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d433dd31e13dbc7 server_cookie=9750fb22adede7e0 in_seq=0 out_seq=0 2024-05-20T18:30:40.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.006+0000 7faf53fff640 10 client.6465 ms_handle_connect on v2:172.21.15.63:6832/4036379300 2024-05-20T18:30:40.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.006+0000 7faf53fff640 1 -- 172.21.15.5:0/3700763535 <== mds.0 v2:172.21.15.63:6832/4036379300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+141 (crc 0 0 0) 0x55855b0085b0 con 0x55855afaeb40 2024-05-20T18:30:40.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.006+0000 7faf53fff640 10 client.6465 handle_command_reply: tid=0 2024-05-20T18:30:40.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.006+0000 7faf827c9740 1 client.6465 shutdown 2024-05-20T18:30:40.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.006+0000 7faf827c9740 1 -- 172.21.15.5:0/3700763535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55855abea490 msgr2=0x55855afcacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:40.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.006+0000 7faf827c9740 1 --2- 172.21.15.5:0/3700763535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55855abea490 0x55855afcacd0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7faf700886d0 tx=0x7faf7000a000 comp rx=0 tx=0).stop 2024-05-20T18:30:40.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.006+0000 7faf827c9740 1 -- 172.21.15.5:0/3700763535 shutdown_connections 2024-05-20T18:30:40.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.006+0000 7faf827c9740 1 --2- 172.21.15.5:0/3700763535 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55855afaeb40 0x55855b011e60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.006+0000 7faf827c9740 1 --2- 172.21.15.5:0/3700763535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55855ac13350 0x55855afb2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.006+0000 7faf827c9740 1 --2- 172.21.15.5:0/3700763535 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55855abf2670 0x55855afad570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.006+0000 7faf53fff640 0 client.6465 ms_handle_reset on v2:172.21.15.63:6832/4036379300 2024-05-20T18:30:40.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.006+0000 7faf827c9740 1 --2- 172.21.15.5:0/3700763535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55855abea490 0x55855afcacd0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.006+0000 7faf827c9740 1 -- 172.21.15.5:0/3700763535 >> 172.21.15.5:0/3700763535 conn(0x55855ac074a0 msgr2=0x55855afdcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:40.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.006+0000 7faf827c9740 1 -- 172.21.15.5:0/3700763535 shutdown_connections 2024-05-20T18:30:40.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.006+0000 7faf827c9740 1 -- 172.21.15.5:0/3700763535 wait complete. 2024-05-20T18:30:40.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.006+0000 7faf827c9740 20 client.6465 trim_cache size 0 max 16384 2024-05-20T18:30:40.012 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:30:40.012 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-05-20T18:30:40.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:30:40.012 INFO:teuthology.orchestra.run.smithi005.stdout: "damage_type": "backtrace", 2024-05-20T18:30:40.012 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 2585882889, 2024-05-20T18:30:40.012 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 1099511627777, 2024-05-20T18:30:40.012 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/dir_test/file" 2024-05-20T18:30:40.012 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:30:40.012 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-05-20T18:30:40.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.007+0000 7faf7b844640 1 -- 172.21.15.5:0/2438638321 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faf5806be60 msgr2=0x7faf5806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:40.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.007+0000 7faf7b844640 1 --2- 172.21.15.5:0/2438638321 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faf5806be60 0x7faf5806e370 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7faf74156750 tx=0x7faf64008040 comp rx=0 tx=0).stop 2024-05-20T18:30:40.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.007+0000 7faf7b844640 1 -- 172.21.15.5:0/2438638321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf740cedd0 msgr2=0x7faf7413d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:40.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.007+0000 7faf7b844640 1 --2- 172.21.15.5:0/2438638321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf740cedd0 0x7faf7413d210 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7faf7000c440 tx=0x7faf7000c900 comp rx=0 tx=0).stop 2024-05-20T18:30:40.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.007+0000 7faf7b844640 1 -- 172.21.15.5:0/2438638321 shutdown_connections 2024-05-20T18:30:40.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.007+0000 7faf7b844640 1 --2- 172.21.15.5:0/2438638321 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faf5806be60 0x7faf5806e370 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.007+0000 7faf7b844640 1 --2- 172.21.15.5:0/2438638321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf7415d5f0 0x7faf741523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.007+0000 7faf7b844640 1 --2- 172.21.15.5:0/2438638321 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf741576d0 0x7faf74160840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.007+0000 7faf7b844640 1 --2- 172.21.15.5:0/2438638321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf740cedd0 0x7faf7413d210 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.007+0000 7faf7b844640 1 -- 172.21.15.5:0/2438638321 >> 172.21.15.5:0/2438638321 conn(0x7faf740cd2a0 msgr2=0x7faf740b04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:40.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.007+0000 7faf7b844640 1 -- 172.21.15.5:0/2438638321 shutdown_connections 2024-05-20T18:30:40.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.007+0000 7faf7b844640 1 -- 172.21.15.5:0/2438638321 wait complete. 2024-05-20T18:30:40.018 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-20T18:30:40.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.133+0000 7efc0f196640 1 Processor -- start 2024-05-20T18:30:40.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.133+0000 7efc0f196640 1 -- start start 2024-05-20T18:30:40.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.134+0000 7efc0f196640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efc080b4070 0x7efc080ba5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.134+0000 7efc0f196640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7efc080d0560 0x7efc0806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.134+0000 7efc0f196640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc0806dd20 con 0x7efc0806d910 2024-05-20T18:30:40.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.134+0000 7efc0f196640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7efc080b8490 con 0x7efc080b4070 2024-05-20T18:30:40.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.134+0000 7efc0f196640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7efc0815fa90 con 0x7efc080d0560 2024-05-20T18:30:40.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.134+0000 7efc0cf32640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efc080b4070 0x7efc080ba5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.134+0000 7efc0cf32640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efc080b4070 0x7efc080ba5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52776/0 (socket says 172.21.15.5:52776) 2024-05-20T18:30:40.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.134+0000 7efc0cf32640 1 -- 172.21.15.5:0/1682985299 learned_addr learned my addr 172.21.15.5:0/1682985299 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:40.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.135+0000 7efc07fff640 1 --2- 172.21.15.5:0/1682985299 >> v2:172.21.15.63:3300/0 conn(0x7efc080d0560 0x7efc0806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.135+0000 7efc0cf32640 1 -- 172.21.15.5:0/1682985299 >> v2:172.21.15.63:3300/0 conn(0x7efc080d0560 msgr2=0x7efc0806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:40.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.135+0000 7efc0cf32640 1 --2- 172.21.15.5:0/1682985299 >> v2:172.21.15.63:3300/0 conn(0x7efc080d0560 0x7efc0806d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.135+0000 7efc0cf32640 1 -- 172.21.15.5:0/1682985299 >> v1:172.21.15.63:6789/0 conn(0x7efc0806d910 legacy=0x7efc080d27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:40.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.135+0000 7efc0cf32640 1 -- 172.21.15.5:0/1682985299 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc080d3f00 con 0x7efc080b4070 2024-05-20T18:30:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.135+0000 7efc0cf32640 1 --2- 172.21.15.5:0/1682985299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc080b4070 0x7efc080ba5c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7efc080d5d10 tx=0x7efc0002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6bd2b5e7fe29d0a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.136+0000 7efc077fe640 1 -- 172.21.15.5:0/1682985299 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc00037070 con 0x7efc080b4070 2024-05-20T18:30:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.136+0000 7efc077fe640 1 -- 172.21.15.5:0/1682985299 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc00032440 con 0x7efc080b4070 2024-05-20T18:30:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.136+0000 7efc077fe640 1 -- 172.21.15.5:0/1682985299 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc00032810 con 0x7efc080b4070 2024-05-20T18:30:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.136+0000 7efc0f196640 1 -- 172.21.15.5:0/1682985299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc080b4070 msgr2=0x7efc080ba5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.136+0000 7efc0f196640 1 --2- 172.21.15.5:0/1682985299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc080b4070 0x7efc080ba5c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7efc080d5d10 tx=0x7efc0002a1e0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.136+0000 7efc0f196640 1 -- 172.21.15.5:0/1682985299 shutdown_connections 2024-05-20T18:30:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.136+0000 7efc0f196640 1 --2- 172.21.15.5:0/1682985299 >> v2:172.21.15.63:3300/0 conn(0x7efc080d0560 0x7efc0806d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.136+0000 7efc0f196640 1 --2- 172.21.15.5:0/1682985299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc080b4070 0x7efc080ba5c0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.136+0000 7efc0f196640 1 -- 172.21.15.5:0/1682985299 >> 172.21.15.5:0/1682985299 conn(0x7efc080d4c10 msgr2=0x7efc080d5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.136+0000 7efc0f196640 1 -- 172.21.15.5:0/1682985299 shutdown_connections 2024-05-20T18:30:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.136+0000 7efc0f196640 1 -- 172.21.15.5:0/1682985299 wait complete. 2024-05-20T18:30:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.138+0000 7efc0f196640 1 Processor -- start 2024-05-20T18:30:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.138+0000 7efc0f196640 1 -- start start 2024-05-20T18:30:40.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.140+0000 7efc0f196640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc0806d910 0x7efc080cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.140+0000 7efc0f196640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efc080b4070 0x7efc08160450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.140+0000 7efc0f196640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc080d0560 0x7efc0814d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.140+0000 7efc0f196640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efc080d47a0 con 0x7efc0806d910 2024-05-20T18:30:40.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.140+0000 7efc0f196640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efc080d44c0 con 0x7efc080d0560 2024-05-20T18:30:40.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.140+0000 7efc0f196640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efc080bab00 con 0x7efc080b4070 2024-05-20T18:30:40.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.140+0000 7efc0cf32640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc0806d910 0x7efc080cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.140+0000 7efc0cf32640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc0806d910 0x7efc080cb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52782/0 (socket says 172.21.15.5:52782) 2024-05-20T18:30:40.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.140+0000 7efc0cf32640 1 -- 172.21.15.5:0/618451956 learned_addr learned my addr 172.21.15.5:0/618451956 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:40.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.140+0000 7efc07fff640 1 --2- 172.21.15.5:0/618451956 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efc080b4070 0x7efc08160450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.140+0000 7efc0d733640 1 --2- 172.21.15.5:0/618451956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc080d0560 0x7efc0814d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.140+0000 7efc0cf32640 1 -- 172.21.15.5:0/618451956 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efc080b4070 msgr2=0x7efc08160450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:40.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.140+0000 7efc0cf32640 1 --2- 172.21.15.5:0/618451956 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efc080b4070 0x7efc08160450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.140+0000 7efc0cf32640 1 -- 172.21.15.5:0/618451956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc080d0560 msgr2=0x7efc0814d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:40.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.140+0000 7efc0cf32640 1 --2- 172.21.15.5:0/618451956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc080d0560 0x7efc0814d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.140+0000 7efc0cf32640 1 -- 172.21.15.5:0/618451956 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0815f610 con 0x7efc0806d910 2024-05-20T18:30:40.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.140+0000 7efc0cf32640 1 --2- 172.21.15.5:0/618451956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc0806d910 0x7efc080cb120 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7efc0002a220 tx=0x7efc000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:40.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.140+0000 7efc057fa640 1 -- 172.21.15.5:0/618451956 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc00041020 con 0x7efc0806d910 2024-05-20T18:30:40.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.140+0000 7efc057fa640 1 -- 172.21.15.5:0/618451956 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc000306c0 con 0x7efc0806d910 2024-05-20T18:30:40.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.140+0000 7efc0f196640 1 -- 172.21.15.5:0/618451956 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc080ca680 con 0x7efc0806d910 2024-05-20T18:30:40.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.140+0000 7efc057fa640 1 -- 172.21.15.5:0/618451956 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc00037040 con 0x7efc0806d910 2024-05-20T18:30:40.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.141+0000 7efc0f196640 1 -- 172.21.15.5:0/618451956 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc080ca920 con 0x7efc0806d910 2024-05-20T18:30:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.142+0000 7efc0f196640 1 -- 172.21.15.5:0/618451956 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc080690b0 con 0x7efc0806d910 2024-05-20T18:30:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.142+0000 7efc057fa640 1 -- 172.21.15.5:0/618451956 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efc00044020 con 0x7efc0806d910 2024-05-20T18:30:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.143+0000 7efc057fa640 1 --2- 172.21.15.5:0/618451956 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efbe806be60 0x7efbe806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.143+0000 7efc057fa640 1 -- 172.21.15.5:0/618451956 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7efc000ab800 con 0x7efc0806d910 2024-05-20T18:30:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.143+0000 7efc07fff640 1 --2- 172.21.15.5:0/618451956 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efbe806be60 0x7efbe806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.143+0000 7efc07fff640 1 --2- 172.21.15.5:0/618451956 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efbe806be60 0x7efbe806e370 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7efc081f9840 tx=0x7efbf80039b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:40.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.148+0000 7efc057fa640 1 -- 172.21.15.5:0/618451956 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7efc0004b030 con 0x7efc0806d910 2024-05-20T18:30:40.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.371+0000 7efc0f196640 1 -- 172.21.15.5:0/618451956 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7efc080ba800 con 0x7efc0806d910 2024-05-20T18:30:40.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.371+0000 7efc057fa640 1 -- 172.21.15.5:0/618451956 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7efc0007a020 con 0x7efc0806d910 2024-05-20T18:30:40.372 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:30:40.372 INFO:teuthology.orchestra.run.smithi005.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-05-20T18:30:40.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.373+0000 7efc0f196640 1 -- 172.21.15.5:0/618451956 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efbe806be60 msgr2=0x7efbe806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:40.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.373+0000 7efc0f196640 1 --2- 172.21.15.5:0/618451956 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efbe806be60 0x7efbe806e370 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7efc081f9840 tx=0x7efbf80039b0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.374+0000 7efc0f196640 1 -- 172.21.15.5:0/618451956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc0806d910 msgr2=0x7efc080cb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:40.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.374+0000 7efc0f196640 1 --2- 172.21.15.5:0/618451956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc0806d910 0x7efc080cb120 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7efc0002a220 tx=0x7efc000046c0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.374+0000 7efc0f196640 1 -- 172.21.15.5:0/618451956 shutdown_connections 2024-05-20T18:30:40.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.374+0000 7efc0f196640 1 --2- 172.21.15.5:0/618451956 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efbe806be60 0x7efbe806e370 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.374+0000 7efc0f196640 1 --2- 172.21.15.5:0/618451956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc080d0560 0x7efc0814d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.374+0000 7efc0f196640 1 --2- 172.21.15.5:0/618451956 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efc080b4070 0x7efc08160450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.374+0000 7efc0f196640 1 --2- 172.21.15.5:0/618451956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc0806d910 0x7efc080cb120 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.374+0000 7efc0f196640 1 -- 172.21.15.5:0/618451956 >> 172.21.15.5:0/618451956 conn(0x7efc080d4c10 msgr2=0x7efc080cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:40.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.374+0000 7efc0f196640 1 -- 172.21.15.5:0/618451956 shutdown_connections 2024-05-20T18:30:40.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.374+0000 7efc0f196640 1 -- 172.21.15.5:0/618451956 wait complete. 2024-05-20T18:30:40.385 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-20T18:30:40.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:30:40.385 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive,force 2024-05-20T18:30:40.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.504+0000 7f90e541b640 1 Processor -- start 2024-05-20T18:30:40.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.504+0000 7f90e541b640 1 -- start start 2024-05-20T18:30:40.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.506+0000 7f90e541b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f90e00b1df0 0x7f90e00d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.506+0000 7f90e541b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f90e00b3450 0x7f90e00b3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.506+0000 7f90e541b640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90e009c350 con 0x7f90e00d14a0 2024-05-20T18:30:40.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.506+0000 7f90e541b640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f90e009bc20 con 0x7f90e00b1df0 2024-05-20T18:30:40.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.506+0000 7f90e541b640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f90e00cedb0 con 0x7f90e00b3450 2024-05-20T18:30:40.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.506+0000 7f90deffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f90e00b1df0 0x7f90e00d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.506+0000 7f90deffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f90e00b1df0 0x7f90e00d0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52792/0 (socket says 172.21.15.5:52792) 2024-05-20T18:30:40.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.506+0000 7f90deffd640 1 -- 172.21.15.5:0/3275419259 learned_addr learned my addr 172.21.15.5:0/3275419259 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:40.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.506+0000 7f90deffd640 1 -- 172.21.15.5:0/3275419259 >> v2:172.21.15.63:3300/0 conn(0x7f90e00b3450 msgr2=0x7f90e00b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:40.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.506+0000 7f90deffd640 1 --2- 172.21.15.5:0/3275419259 >> v2:172.21.15.63:3300/0 conn(0x7f90e00b3450 0x7f90e00b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.506+0000 7f90deffd640 1 -- 172.21.15.5:0/3275419259 >> v1:172.21.15.63:6789/0 conn(0x7f90e00d14a0 legacy=0x7f90e00b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:40.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.506+0000 7f90deffd640 1 -- 172.21.15.5:0/3275419259 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e00d1a50 con 0x7f90e00b1df0 2024-05-20T18:30:40.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.507+0000 7f90deffd640 1 --2- 172.21.15.5:0/3275419259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90e00b1df0 0x7f90e00d0f60 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f90e008bb60 tx=0x7f90d002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9bafa7cbe3cd8c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:40.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.507+0000 7f90ddffb640 1 -- 172.21.15.5:0/3275419259 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0004030 con 0x7f90e00b1df0 2024-05-20T18:30:40.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.507+0000 7f90ddffb640 1 -- 172.21.15.5:0/3275419259 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90d00041d0 con 0x7f90e00b1df0 2024-05-20T18:30:40.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.507+0000 7f90ddffb640 1 -- 172.21.15.5:0/3275419259 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0003610 con 0x7f90e00b1df0 2024-05-20T18:30:40.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.508+0000 7f90e541b640 1 -- 172.21.15.5:0/3275419259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90e00b1df0 msgr2=0x7f90e00d0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:40.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.508+0000 7f90e541b640 1 --2- 172.21.15.5:0/3275419259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90e00b1df0 0x7f90e00d0f60 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f90e008bb60 tx=0x7f90d002a180 comp rx=0 tx=0).stop 2024-05-20T18:30:40.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.508+0000 7f90e541b640 1 -- 172.21.15.5:0/3275419259 shutdown_connections 2024-05-20T18:30:40.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.508+0000 7f90e541b640 1 --2- 172.21.15.5:0/3275419259 >> v2:172.21.15.63:3300/0 conn(0x7f90e00b3450 0x7f90e00b3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.508+0000 7f90e541b640 1 --2- 172.21.15.5:0/3275419259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90e00b1df0 0x7f90e00d0f60 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.508+0000 7f90e541b640 1 -- 172.21.15.5:0/3275419259 >> 172.21.15.5:0/3275419259 conn(0x7f90e00cf450 msgr2=0x7f90e00cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:40.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.508+0000 7f90e541b640 1 -- 172.21.15.5:0/3275419259 shutdown_connections 2024-05-20T18:30:40.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.508+0000 7f90e541b640 1 -- 172.21.15.5:0/3275419259 wait complete. 2024-05-20T18:30:40.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.510+0000 7f90e541b640 1 Processor -- start 2024-05-20T18:30:40.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.510+0000 7f90e541b640 1 -- start start 2024-05-20T18:30:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.511+0000 7f90e541b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90e00b1df0 0x7f90e009fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.511+0000 7f90e541b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90e00b3450 0x7f90e01f3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.511+0000 7f90e541b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90e00d14a0 0x7f90e00c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.511+0000 7f90e541b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90e00ce820 con 0x7f90e00b1df0 2024-05-20T18:30:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.511+0000 7f90e541b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f90e00b3da0 con 0x7f90e00d14a0 2024-05-20T18:30:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.511+0000 7f90e541b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f90e00d0650 con 0x7f90e00b3450 2024-05-20T18:30:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.511+0000 7f90deffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90e00b1df0 0x7f90e009fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.511+0000 7f90deffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90e00b1df0 0x7f90e009fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52806/0 (socket says 172.21.15.5:52806) 2024-05-20T18:30:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.511+0000 7f90deffd640 1 -- 172.21.15.5:0/2167458137 learned_addr learned my addr 172.21.15.5:0/2167458137 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.512+0000 7f90deffd640 1 -- 172.21.15.5:0/2167458137 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90e00b3450 msgr2=0x7f90e01f3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.512+0000 7f90deffd640 1 --2- 172.21.15.5:0/2167458137 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90e00b3450 0x7f90e01f3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.512+0000 7f90deffd640 1 -- 172.21.15.5:0/2167458137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90e00d14a0 msgr2=0x7f90e00c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.512+0000 7f90df7fe640 1 --2- 172.21.15.5:0/2167458137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90e00d14a0 0x7f90e00c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.512+0000 7f90deffd640 1 --2- 172.21.15.5:0/2167458137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90e00d14a0 0x7f90e00c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.512+0000 7f90deffd640 1 -- 172.21.15.5:0/2167458137 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e00c6f10 con 0x7f90e00b1df0 2024-05-20T18:30:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.512+0000 7f90df7fe640 1 --2- 172.21.15.5:0/2167458137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90e00d14a0 0x7f90e00c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:30:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.512+0000 7f90deffd640 1 --2- 172.21.15.5:0/2167458137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90e00b1df0 0x7f90e009fc00 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f90d002a1c0 tx=0x7f90d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.512+0000 7f90c77fe640 1 -- 172.21.15.5:0/2167458137 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0041020 con 0x7f90e00b1df0 2024-05-20T18:30:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.512+0000 7f90c77fe640 1 -- 172.21.15.5:0/2167458137 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90d0003c50 con 0x7f90e00b1df0 2024-05-20T18:30:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.512+0000 7f90e541b640 1 -- 172.21.15.5:0/2167458137 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90e00c71f0 con 0x7f90e00b1df0 2024-05-20T18:30:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.512+0000 7f90c77fe640 1 -- 172.21.15.5:0/2167458137 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0004030 con 0x7f90e00b1df0 2024-05-20T18:30:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.512+0000 7f90e541b640 1 -- 172.21.15.5:0/2167458137 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90e00c77b0 con 0x7f90e00b1df0 2024-05-20T18:30:40.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.514+0000 7f90c77fe640 1 -- 172.21.15.5:0/2167458137 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f90d0044020 con 0x7f90e00b1df0 2024-05-20T18:30:40.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.514+0000 7f90c77fe640 1 --2- 172.21.15.5:0/2167458137 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f90cc06be60 0x7f90cc06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.515+0000 7f90c77fe640 1 -- 172.21.15.5:0/2167458137 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7f90d00abe10 con 0x7f90e00b1df0 2024-05-20T18:30:40.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.515+0000 7f90de7fc640 1 --2- 172.21.15.5:0/2167458137 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f90cc06be60 0x7f90cc06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.515+0000 7f90de7fc640 1 --2- 172.21.15.5:0/2167458137 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f90cc06be60 0x7f90cc06e370 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f90e00b7950 tx=0x7f90d4006ba0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:40.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.515+0000 7f90ec3a4740 1 Processor -- start 2024-05-20T18:30:40.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.516+0000 7f90ec3a4740 1 -- start start 2024-05-20T18:30:40.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.517+0000 7f90ec3a4740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e117a394d0 0x55e117e05830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.517+0000 7f90ec3a4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e117a10610 0x55e117e20560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.517+0000 7f90ec3a4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e117a187f0 0x55e117e20aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.517+0000 7f90ec3a4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e117d3d960 con 0x55e117a187f0 2024-05-20T18:30:40.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.517+0000 7f90ec3a4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e117ce55f0 con 0x55e117a10610 2024-05-20T18:30:40.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.517+0000 7f90ec3a4740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55e117d39920 con 0x55e117a394d0 2024-05-20T18:30:40.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.517+0000 7f90df7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e117a187f0 0x55e117e20aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.517+0000 7f90df7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e117a187f0 0x55e117e20aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52814/0 (socket says 172.21.15.5:52814) 2024-05-20T18:30:40.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.517+0000 7f90df7fe640 1 -- 172.21.15.5:0/26064232 learned_addr learned my addr 172.21.15.5:0/26064232 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:40.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.517+0000 7f90deffd640 1 --2- 172.21.15.5:0/26064232 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e117a394d0 0x55e117e05830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.517+0000 7f90de7fc640 1 --2- 172.21.15.5:0/26064232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e117a10610 0x55e117e20560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.517+0000 7f90df7fe640 1 -- 172.21.15.5:0/26064232 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e117a394d0 msgr2=0x55e117e05830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:40.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.517+0000 7f90df7fe640 1 --2- 172.21.15.5:0/26064232 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e117a394d0 0x55e117e05830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.517+0000 7f90df7fe640 1 -- 172.21.15.5:0/26064232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e117a10610 msgr2=0x55e117e20560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:40.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.517+0000 7f90df7fe640 1 --2- 172.21.15.5:0/26064232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e117a10610 0x55e117e20560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.517+0000 7f90df7fe640 1 -- 172.21.15.5:0/26064232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e117b1d940 con 0x55e117a187f0 2024-05-20T18:30:40.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.517+0000 7f90de7fc640 1 --2- 172.21.15.5:0/26064232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e117a10610 0x55e117e20560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:30:40.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.517+0000 7f90df7fe640 1 --2- 172.21.15.5:0/26064232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e117a187f0 0x55e117e20aa0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f90d8007f30 tx=0x7f90d800c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc2b253db8a411d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:40.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.518+0000 7f90c4ff9640 1 -- 172.21.15.5:0/26064232 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d8014070 con 0x55e117a187f0 2024-05-20T18:30:40.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.518+0000 7f90c4ff9640 1 -- 172.21.15.5:0/26064232 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90d800f850 con 0x55e117a187f0 2024-05-20T18:30:40.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.518+0000 7f90c4ff9640 1 -- 172.21.15.5:0/26064232 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d800fc20 con 0x55e117a187f0 2024-05-20T18:30:40.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.518+0000 7f90ec3a4740 1 -- 172.21.15.5:0/26064232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e117a187f0 msgr2=0x55e117e20aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:40.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.518+0000 7f90ec3a4740 1 --2- 172.21.15.5:0/26064232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e117a187f0 0x55e117e20aa0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f90d8007f30 tx=0x7f90d800c750 comp rx=0 tx=0).stop 2024-05-20T18:30:40.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.519+0000 7f90ec3a4740 1 -- 172.21.15.5:0/26064232 shutdown_connections 2024-05-20T18:30:40.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.519+0000 7f90ec3a4740 1 --2- 172.21.15.5:0/26064232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e117a187f0 0x55e117e20aa0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.519+0000 7f90ec3a4740 1 --2- 172.21.15.5:0/26064232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e117a10610 0x55e117e20560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.519+0000 7f90ec3a4740 1 --2- 172.21.15.5:0/26064232 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e117a394d0 0x55e117e05830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.519+0000 7f90ec3a4740 1 -- 172.21.15.5:0/26064232 >> 172.21.15.5:0/26064232 conn(0x55e117a2d620 msgr2=0x55e117ddae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:40.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.519+0000 7f90ec3a4740 1 -- 172.21.15.5:0/26064232 shutdown_connections 2024-05-20T18:30:40.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.519+0000 7f90ec3a4740 1 -- 172.21.15.5:0/26064232 wait complete. 2024-05-20T18:30:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.519+0000 7f90ec3a4740 1 -- start start 2024-05-20T18:30:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.520+0000 7f90ec3a4740 1 Processor -- start 2024-05-20T18:30:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.521+0000 7f90ec3a4740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e117a10610 0x55e117e083e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.521+0000 7f90ec3a4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e117a187f0 0x55e117e04060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.521+0000 7f90ec3a4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e117a394d0 0x55e117e045a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.521+0000 7f90ec3a4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e117cf8b80 con 0x55e117a394d0 2024-05-20T18:30:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.521+0000 7f90ec3a4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e117e018c0 con 0x55e117a187f0 2024-05-20T18:30:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.521+0000 7f90ec3a4740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55e117e30820 con 0x55e117a10610 2024-05-20T18:30:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.521+0000 7f90ec3a4740 10 client.0 _ll_register_callbacks cb 0x55e117cdbb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:30:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.521+0000 7f90df7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e117a394d0 0x55e117e045a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.521+0000 7f90df7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e117a394d0 0x55e117e045a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52822/0 (socket says 172.21.15.5:52822) 2024-05-20T18:30:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.521+0000 7f90df7fe640 1 -- 172.21.15.5:0/4095069763 learned_addr learned my addr 172.21.15.5:0/4095069763 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.521+0000 7f90deffd640 1 --2- 172.21.15.5:0/4095069763 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e117a10610 0x55e117e083e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.521+0000 7f90de7fc640 1 --2- 172.21.15.5:0/4095069763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e117a187f0 0x55e117e04060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.521+0000 7f90df7fe640 1 -- 172.21.15.5:0/4095069763 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e117a10610 msgr2=0x55e117e083e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.521+0000 7f90df7fe640 1 --2- 172.21.15.5:0/4095069763 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e117a10610 0x55e117e083e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.521+0000 7f90df7fe640 1 -- 172.21.15.5:0/4095069763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e117a187f0 msgr2=0x55e117e04060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.521+0000 7f90df7fe640 1 --2- 172.21.15.5:0/4095069763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e117a187f0 0x55e117e04060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.521+0000 7f90df7fe640 1 -- 172.21.15.5:0/4095069763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e117b7e0d0 con 0x55e117a394d0 2024-05-20T18:30:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.521+0000 7f90de7fc640 1 --2- 172.21.15.5:0/4095069763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e117a187f0 0x55e117e04060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:30:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.522+0000 7f90df7fe640 1 --2- 172.21.15.5:0/4095069763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e117a394d0 0x55e117e045a0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f90d800c790 tx=0x7f90d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a055b30533640019 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.522+0000 7f90bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:30:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.522+0000 7f90bdffb640 1 -- 172.21.15.5:0/4095069763 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d8028060 con 0x55e117a394d0 2024-05-20T18:30:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.522+0000 7f90bdffb640 1 -- 172.21.15.5:0/4095069763 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7f90d800f650 con 0x55e117a394d0 2024-05-20T18:30:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.522+0000 7f90ec3a4740 1 -- 172.21.15.5:0/4095069763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90d800f850 con 0x55e117a394d0 2024-05-20T18:30:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.522+0000 7f90bdffb640 1 -- 172.21.15.5:0/4095069763 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f90d802e050 con 0x55e117a394d0 2024-05-20T18:30:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.522+0000 7f90ec3a4740 10 client.6483 fetch_fsmap learned FSMap version 35 2024-05-20T18:30:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.522+0000 7f90ec3a4740 1 -- 172.21.15.5:0/4095069763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55e117a11230 con 0x55e117a394d0 2024-05-20T18:30:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.522+0000 7f90bdffb640 1 -- 172.21.15.5:0/4095069763 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90d800f850 con 0x55e117a394d0 2024-05-20T18:30:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.522+0000 7f90bdffb640 1 -- 172.21.15.5:0/4095069763 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f90d8031020 con 0x55e117a394d0 2024-05-20T18:30:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.522+0000 7f90ec3a4740 10 client.6483 fetch_fsmap finished waiting for FSMap version 35 2024-05-20T18:30:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.522+0000 7f90ec3a4740 10 client.6483 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-20T18:30:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.522+0000 7f90ec3a4740 20 client.6483 populate_metadata read hostname 'smithi005' 2024-05-20T18:30:40.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.522+0000 7f90ec3a4740 1 --2- 172.21.15.5:0/4095069763 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55e1179f82b0 0x55e117e06c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.522+0000 7f90ec3a4740 4 client.6483 mds_command: new command op to 6198 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:30:40.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.522+0000 7f90ec3a4740 1 -- 172.21.15.5:0/4095069763 --> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e117d4b090 con 0x55e1179f82b0 2024-05-20T18:30:40.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.523+0000 7f90deffd640 1 --2- 172.21.15.5:0/4095069763 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55e1179f82b0 0x55e117e06c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.523+0000 7f90bdffb640 10 client.6483 ms_handle_connect on v2:172.21.15.63:6832/4036379300 2024-05-20T18:30:40.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.523+0000 7f90deffd640 1 --2- 172.21.15.5:0/4095069763 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55e1179f82b0 0x55e117e06c10 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25921f0ea2220f87 server_cookie=d7a45a62efc6374d in_seq=0 out_seq=0 2024-05-20T18:30:40.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.524+0000 7f90bdffb640 1 -- 172.21.15.5:0/4095069763 <== mds.0 v2:172.21.15.63:6832/4036379300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e117d4b090 con 0x55e1179f82b0 2024-05-20T18:30:40.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.524+0000 7f90bdffb640 10 client.6483 handle_command_reply: tid=0 2024-05-20T18:30:40.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.524+0000 7f90ec3a4740 1 client.6483 shutdown 2024-05-20T18:30:40.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.524+0000 7f90ec3a4740 1 -- 172.21.15.5:0/4095069763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e117a394d0 msgr2=0x55e117e045a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:40.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.524+0000 7f90ec3a4740 1 --2- 172.21.15.5:0/4095069763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e117a394d0 0x55e117e045a0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f90d800c790 tx=0x7f90d8004a40 comp rx=0 tx=0).stop 2024-05-20T18:30:40.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.524+0000 7f90ec3a4740 1 -- 172.21.15.5:0/4095069763 shutdown_connections 2024-05-20T18:30:40.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.524+0000 7f90ec3a4740 1 --2- 172.21.15.5:0/4095069763 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55e1179f82b0 0x55e117e06c10 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.524+0000 7f90ec3a4740 1 --2- 172.21.15.5:0/4095069763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e117a394d0 0x55e117e045a0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.524+0000 7f90ec3a4740 1 --2- 172.21.15.5:0/4095069763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e117a187f0 0x55e117e04060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.524+0000 7f90bdffb640 0 client.6483 ms_handle_reset on v2:172.21.15.63:6832/4036379300 2024-05-20T18:30:40.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.524+0000 7f90ec3a4740 1 --2- 172.21.15.5:0/4095069763 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e117a10610 0x55e117e083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.524+0000 7f90ec3a4740 1 -- 172.21.15.5:0/4095069763 >> 172.21.15.5:0/4095069763 conn(0x55e117a2d620 msgr2=0x55e117ddae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:40.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.524+0000 7f90ec3a4740 1 -- 172.21.15.5:0/4095069763 shutdown_connections 2024-05-20T18:30:40.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.524+0000 7f90ec3a4740 1 -- 172.21.15.5:0/4095069763 wait complete. 2024-05-20T18:30:40.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.524+0000 7f90ec3a4740 20 client.6483 trim_cache size 0 max 16384 2024-05-20T18:30:40.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.542+0000 7f90ec3a4740 1 Processor -- start 2024-05-20T18:30:40.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.543+0000 7f90ec3a4740 1 -- start start 2024-05-20T18:30:40.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.544+0000 7f90ec3a4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e117a10610 0x55e117e20920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.544+0000 7f90ec3a4740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e117a187f0 0x55e117e07fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.544+0000 7f90ec3a4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e117a394d0 0x55e117e22120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.544+0000 7f90ec3a4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e117dd61d0 con 0x55e117a394d0 2024-05-20T18:30:40.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.544+0000 7f90ec3a4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f90e0200320 con 0x55e117a10610 2024-05-20T18:30:40.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.544+0000 7f90ec3a4740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f90e01ff370 con 0x55e117a187f0 2024-05-20T18:30:40.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.544+0000 7f90df7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e117a394d0 0x55e117e22120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.544+0000 7f90df7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e117a394d0 0x55e117e22120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52824/0 (socket says 172.21.15.5:52824) 2024-05-20T18:30:40.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.544+0000 7f90df7fe640 1 -- 172.21.15.5:0/1273604269 learned_addr learned my addr 172.21.15.5:0/1273604269 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:40.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.544+0000 7f90deffd640 1 --2- 172.21.15.5:0/1273604269 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e117a10610 0x55e117e20920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.544+0000 7f90de7fc640 1 --2- 172.21.15.5:0/1273604269 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e117a187f0 0x55e117e07fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.544+0000 7f90df7fe640 1 -- 172.21.15.5:0/1273604269 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e117a187f0 msgr2=0x55e117e07fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:40.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.544+0000 7f90df7fe640 1 --2- 172.21.15.5:0/1273604269 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e117a187f0 0x55e117e07fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.544+0000 7f90df7fe640 1 -- 172.21.15.5:0/1273604269 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e117a10610 msgr2=0x55e117e20920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:40.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.544+0000 7f90df7fe640 1 --2- 172.21.15.5:0/1273604269 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e117a10610 0x55e117e20920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.544+0000 7f90df7fe640 1 -- 172.21.15.5:0/1273604269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e117e1c2e0 con 0x55e117a394d0 2024-05-20T18:30:40.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.544+0000 7f90df7fe640 1 --2- 172.21.15.5:0/1273604269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e117a394d0 0x55e117e22120 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f90d8010fd0 tx=0x7f90d801c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebade3c61c09788 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:40.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.545+0000 7f90bffff640 1 -- 172.21.15.5:0/1273604269 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d8036070 con 0x55e117a394d0 2024-05-20T18:30:40.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.545+0000 7f90bffff640 1 -- 172.21.15.5:0/1273604269 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90d8019960 con 0x55e117a394d0 2024-05-20T18:30:40.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.545+0000 7f90bffff640 1 -- 172.21.15.5:0/1273604269 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d8014040 con 0x55e117a394d0 2024-05-20T18:30:40.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.545+0000 7f90ec3a4740 1 -- 172.21.15.5:0/1273604269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e117a394d0 msgr2=0x55e117e22120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:40.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.545+0000 7f90ec3a4740 1 --2- 172.21.15.5:0/1273604269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e117a394d0 0x55e117e22120 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f90d8010fd0 tx=0x7f90d801c070 comp rx=0 tx=0).stop 2024-05-20T18:30:40.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.545+0000 7f90ec3a4740 1 -- 172.21.15.5:0/1273604269 shutdown_connections 2024-05-20T18:30:40.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.545+0000 7f90ec3a4740 1 --2- 172.21.15.5:0/1273604269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e117a394d0 0x55e117e22120 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.545+0000 7f90ec3a4740 1 --2- 172.21.15.5:0/1273604269 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e117a187f0 0x55e117e07fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.545+0000 7f90ec3a4740 1 --2- 172.21.15.5:0/1273604269 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e117a10610 0x55e117e20920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.545+0000 7f90ec3a4740 1 -- 172.21.15.5:0/1273604269 >> 172.21.15.5:0/1273604269 conn(0x55e117a2d620 msgr2=0x55e117e2ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:40.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.545+0000 7f90ec3a4740 1 -- 172.21.15.5:0/1273604269 shutdown_connections 2024-05-20T18:30:40.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.545+0000 7f90ec3a4740 1 -- 172.21.15.5:0/1273604269 wait complete. 2024-05-20T18:30:40.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.546+0000 7f90ec3a4740 1 -- start start 2024-05-20T18:30:40.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.546+0000 7f90ec3a4740 1 Processor -- start 2024-05-20T18:30:40.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.548+0000 7f90ec3a4740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e117a10610 0x55e117e07fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.548+0000 7f90ec3a4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e117a187f0 0x55e117e21ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.548+0000 7f90ec3a4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e117a394d0 0x55e117e22430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.548+0000 7f90ec3a4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90e009edb0 con 0x55e117a394d0 2024-05-20T18:30:40.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.548+0000 7f90ec3a4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f90e009e760 con 0x55e117a187f0 2024-05-20T18:30:40.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.548+0000 7f90ec3a4740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55e117ddad90 con 0x55e117a10610 2024-05-20T18:30:40.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.548+0000 7f90ec3a4740 10 client.0 _ll_register_callbacks cb 0x7f90d800fde0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:30:40.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.548+0000 7f90df7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e117a394d0 0x55e117e22430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.548+0000 7f90df7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e117a394d0 0x55e117e22430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52828/0 (socket says 172.21.15.5:52828) 2024-05-20T18:30:40.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.548+0000 7f90df7fe640 1 -- 172.21.15.5:0/3779353913 learned_addr learned my addr 172.21.15.5:0/3779353913 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:40.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.548+0000 7f90deffd640 1 --2- 172.21.15.5:0/3779353913 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e117a10610 0x55e117e07fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.548+0000 7f90de7fc640 1 --2- 172.21.15.5:0/3779353913 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e117a187f0 0x55e117e21ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.548+0000 7f90df7fe640 1 -- 172.21.15.5:0/3779353913 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e117a10610 msgr2=0x55e117e07fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:40.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.548+0000 7f90df7fe640 1 --2- 172.21.15.5:0/3779353913 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e117a10610 0x55e117e07fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.548+0000 7f90df7fe640 1 -- 172.21.15.5:0/3779353913 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e117a187f0 msgr2=0x55e117e21ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:40.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.548+0000 7f90df7fe640 1 --2- 172.21.15.5:0/3779353913 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e117a187f0 0x55e117e21ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.548+0000 7f90df7fe640 1 -- 172.21.15.5:0/3779353913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e117e21c80 con 0x55e117a394d0 2024-05-20T18:30:40.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.548+0000 7f90df7fe640 1 --2- 172.21.15.5:0/3779353913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e117a394d0 0x55e117e22430 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f90d8000f80 tx=0x7f90d80077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00e5cdf3db0bdd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:40.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.548+0000 7f90bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:30:40.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.548+0000 7f90bdffb640 1 -- 172.21.15.5:0/3779353913 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d8036040 con 0x55e117a394d0 2024-05-20T18:30:40.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.548+0000 7f90bdffb640 1 -- 172.21.15.5:0/3779353913 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7f90d803c890 con 0x55e117a394d0 2024-05-20T18:30:40.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.548+0000 7f90ec3a4740 1 -- 172.21.15.5:0/3779353913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90d8019960 con 0x55e117a394d0 2024-05-20T18:30:40.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.549+0000 7f90bdffb640 1 -- 172.21.15.5:0/3779353913 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f90d8049050 con 0x55e117a394d0 2024-05-20T18:30:40.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.549+0000 7f90ec3a4740 10 client.6489 fetch_fsmap learned FSMap version 35 2024-05-20T18:30:40.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.549+0000 7f90ec3a4740 1 -- 172.21.15.5:0/3779353913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55e117ea0810 con 0x55e117a394d0 2024-05-20T18:30:40.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.549+0000 7f90bdffb640 1 -- 172.21.15.5:0/3779353913 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90d8019960 con 0x55e117a394d0 2024-05-20T18:30:40.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.549+0000 7f90bdffb640 1 -- 172.21.15.5:0/3779353913 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f90d804c020 con 0x55e117a394d0 2024-05-20T18:30:40.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.549+0000 7f90ec3a4740 10 client.6489 fetch_fsmap finished waiting for FSMap version 35 2024-05-20T18:30:40.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.549+0000 7f90ec3a4740 10 client.6489 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-20T18:30:40.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.549+0000 7f90ec3a4740 20 client.6489 populate_metadata read hostname 'smithi005' 2024-05-20T18:30:40.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.549+0000 7f90ec3a4740 1 --2- 172.21.15.5:0/3779353913 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55e117e2d780 0x55e117e2db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.549+0000 7f90ec3a4740 4 client.6489 mds_command: new command op to 6198 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}] 2024-05-20T18:30:40.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.549+0000 7f90ec3a4740 1 -- 172.21.15.5:0/3779353913 --> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}) -- 0x55e117d453e0 con 0x55e117e2d780 2024-05-20T18:30:40.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.549+0000 7f90deffd640 1 --2- 172.21.15.5:0/3779353913 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55e117e2d780 0x55e117e2db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.550+0000 7f90bdffb640 10 client.6489 ms_handle_connect on v2:172.21.15.63:6832/4036379300 2024-05-20T18:30:40.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.550+0000 7f90deffd640 1 --2- 172.21.15.5:0/3779353913 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55e117e2d780 0x55e117e2db90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd3270f10063ce84 server_cookie=fa57361b7de9895f in_seq=0 out_seq=0 2024-05-20T18:30:40.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.551+0000 7f90bdffb640 1 -- 172.21.15.5:0/3779353913 <== mds.0 v2:172.21.15.63:6832/4036379300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e117d453e0 con 0x55e117e2d780 2024-05-20T18:30:40.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.551+0000 7f90bdffb640 10 client.6489 handle_command_reply: tid=0 2024-05-20T18:30:40.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.551+0000 7f90ec3a4740 1 client.6489 shutdown 2024-05-20T18:30:40.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.551+0000 7f90ec3a4740 1 -- 172.21.15.5:0/3779353913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e117a394d0 msgr2=0x55e117e22430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:40.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.551+0000 7f90ec3a4740 1 --2- 172.21.15.5:0/3779353913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e117a394d0 0x55e117e22430 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f90d8000f80 tx=0x7f90d80077d0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.551+0000 7f90ec3a4740 1 -- 172.21.15.5:0/3779353913 shutdown_connections 2024-05-20T18:30:40.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.551+0000 7f90ec3a4740 1 --2- 172.21.15.5:0/3779353913 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55e117e2d780 0x55e117e2db90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.551+0000 7f90ec3a4740 1 --2- 172.21.15.5:0/3779353913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e117a394d0 0x55e117e22430 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.551+0000 7f90ec3a4740 1 --2- 172.21.15.5:0/3779353913 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e117a187f0 0x55e117e21ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.551+0000 7f90ec3a4740 1 --2- 172.21.15.5:0/3779353913 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e117a10610 0x55e117e07fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.551+0000 7f90bdffb640 0 client.6489 ms_handle_reset on v2:172.21.15.63:6832/4036379300 2024-05-20T18:30:40.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.551+0000 7f90ec3a4740 1 -- 172.21.15.5:0/3779353913 >> 172.21.15.5:0/3779353913 conn(0x55e117a2d620 msgr2=0x55e117e24ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:40.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.551+0000 7f90ec3a4740 1 -- 172.21.15.5:0/3779353913 shutdown_connections 2024-05-20T18:30:40.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.551+0000 7f90ec3a4740 1 -- 172.21.15.5:0/3779353913 wait complete. 2024-05-20T18:30:40.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.551+0000 7f90ec3a4740 20 client.6489 trim_cache size 0 max 16384 2024-05-20T18:30:40.556 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:30:40.556 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-20T18:30:40.556 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "c91f971a-767d-4d2c-ad04-68bbc09b23e2", 2024-05-20T18:30:40.556 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-20T18:30:40.556 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:30:40.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.551+0000 7f90e541b640 1 -- 172.21.15.5:0/2167458137 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f90cc06be60 msgr2=0x7f90cc06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:40.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.551+0000 7f90e541b640 1 --2- 172.21.15.5:0/2167458137 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f90cc06be60 0x7f90cc06e370 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f90e00b7950 tx=0x7f90d4006ba0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.551+0000 7f90e541b640 1 -- 172.21.15.5:0/2167458137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90e00b1df0 msgr2=0x7f90e009fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:40.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.551+0000 7f90e541b640 1 --2- 172.21.15.5:0/2167458137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90e00b1df0 0x7f90e009fc00 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f90d002a1c0 tx=0x7f90d00046c0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.552+0000 7f90e541b640 1 -- 172.21.15.5:0/2167458137 shutdown_connections 2024-05-20T18:30:40.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.552+0000 7f90e541b640 1 --2- 172.21.15.5:0/2167458137 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f90cc06be60 0x7f90cc06e370 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.552+0000 7f90e541b640 1 --2- 172.21.15.5:0/2167458137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90e00d14a0 0x7f90e00c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.552+0000 7f90e541b640 1 --2- 172.21.15.5:0/2167458137 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90e00b3450 0x7f90e01f3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.552+0000 7f90e541b640 1 --2- 172.21.15.5:0/2167458137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90e00b1df0 0x7f90e009fc00 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.552+0000 7f90e541b640 1 -- 172.21.15.5:0/2167458137 >> 172.21.15.5:0/2167458137 conn(0x7f90e00cf450 msgr2=0x7f90e0070680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:40.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.552+0000 7f90e541b640 1 -- 172.21.15.5:0/2167458137 shutdown_connections 2024-05-20T18:30:40.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.552+0000 7f90e541b640 1 -- 172.21.15.5:0/2167458137 wait complete. 2024-05-20T18:30:40.559 INFO:tasks.ceph.mds.b.smithi063.stderr:2024-05-20T18:30:40.552+0000 7f976018e640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5555=0-4194304@1} caps={5555=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5555 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x5611ac41cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T18:30:40.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:30:40.563 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-05-20T18:30:40.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.680+0000 7f1f03a44640 1 Processor -- start 2024-05-20T18:30:40.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.680+0000 7f1f03a44640 1 -- start start 2024-05-20T18:30:40.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.681+0000 7f1f03a44640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1efc06d3d0 0x7f1efc06d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.681+0000 7f1f03a44640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1efc0b8190 0x7f1efc0b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.681+0000 7f1f03a44640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1efc06dd20 con 0x7f1efc162820 2024-05-20T18:30:40.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.681+0000 7f1f03a44640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f1efc0d0630 con 0x7f1efc06d3d0 2024-05-20T18:30:40.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.681+0000 7f1f03a44640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1efc0d39c0 con 0x7f1efc0b8190 2024-05-20T18:30:40.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.682+0000 7f1f00fdf640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1efc06d3d0 0x7f1efc06d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.682+0000 7f1f00fdf640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1efc06d3d0 0x7f1efc06d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52830/0 (socket says 172.21.15.5:52830) 2024-05-20T18:30:40.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.682+0000 7f1f00fdf640 1 -- 172.21.15.5:0/1391342010 learned_addr learned my addr 172.21.15.5:0/1391342010 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:40.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.682+0000 7f1f01fe1640 1 --2- 172.21.15.5:0/1391342010 >> v2:172.21.15.63:3300/0 conn(0x7f1efc0b8190 0x7f1efc0b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.682+0000 7f1f00fdf640 1 -- 172.21.15.5:0/1391342010 >> v2:172.21.15.63:3300/0 conn(0x7f1efc0b8190 msgr2=0x7f1efc0b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:40.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.682+0000 7f1f00fdf640 1 --2- 172.21.15.5:0/1391342010 >> v2:172.21.15.63:3300/0 conn(0x7f1efc0b8190 0x7f1efc0b85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.682+0000 7f1f00fdf640 1 -- 172.21.15.5:0/1391342010 >> v1:172.21.15.63:6789/0 conn(0x7f1efc162820 legacy=0x7f1efc0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:40.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.682+0000 7f1f00fdf640 1 -- 172.21.15.5:0/1391342010 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1efc0d3ca0 con 0x7f1efc06d3d0 2024-05-20T18:30:40.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.683+0000 7f1f00fdf640 1 --2- 172.21.15.5:0/1391342010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1efc06d3d0 0x7f1efc06d7e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f1efc08e070 tx=0x7f1eec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18add311154110e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:40.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.683+0000 7f1ef3fff640 1 -- 172.21.15.5:0/1391342010 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eec037070 con 0x7f1efc06d3d0 2024-05-20T18:30:40.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.683+0000 7f1ef3fff640 1 -- 172.21.15.5:0/1391342010 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eec032440 con 0x7f1efc06d3d0 2024-05-20T18:30:40.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.683+0000 7f1ef3fff640 1 -- 172.21.15.5:0/1391342010 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eec032810 con 0x7f1efc06d3d0 2024-05-20T18:30:40.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.684+0000 7f1f03a44640 1 -- 172.21.15.5:0/1391342010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1efc06d3d0 msgr2=0x7f1efc06d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:40.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.684+0000 7f1f03a44640 1 --2- 172.21.15.5:0/1391342010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1efc06d3d0 0x7f1efc06d7e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f1efc08e070 tx=0x7f1eec02a270 comp rx=0 tx=0).stop 2024-05-20T18:30:40.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.684+0000 7f1f03a44640 1 -- 172.21.15.5:0/1391342010 shutdown_connections 2024-05-20T18:30:40.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.684+0000 7f1f03a44640 1 --2- 172.21.15.5:0/1391342010 >> v2:172.21.15.63:3300/0 conn(0x7f1efc0b8190 0x7f1efc0b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.684+0000 7f1f03a44640 1 --2- 172.21.15.5:0/1391342010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1efc06d3d0 0x7f1efc06d7e0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.684+0000 7f1f03a44640 1 -- 172.21.15.5:0/1391342010 >> 172.21.15.5:0/1391342010 conn(0x7f1efc0d4bd0 msgr2=0x7f1efc0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:40.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.684+0000 7f1f03a44640 1 -- 172.21.15.5:0/1391342010 shutdown_connections 2024-05-20T18:30:40.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.684+0000 7f1f03a44640 1 -- 172.21.15.5:0/1391342010 wait complete. 2024-05-20T18:30:40.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.686+0000 7f1f03a44640 1 Processor -- start 2024-05-20T18:30:40.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.686+0000 7f1f03a44640 1 -- start start 2024-05-20T18:30:40.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.687+0000 7f1f03a44640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1efc06d3d0 0x7f1efc0cec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.687+0000 7f1f03a44640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1efc0b8190 0x7f1efc1fd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.687+0000 7f1f03a44640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1efc162820 0x7f1efc1fdab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.687+0000 7f1f03a44640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc0d0300 con 0x7f1efc06d3d0 2024-05-20T18:30:40.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.687+0000 7f1f03a44640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc0d4260 con 0x7f1efc0b8190 2024-05-20T18:30:40.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.687+0000 7f1f03a44640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1efc0b4050 con 0x7f1efc162820 2024-05-20T18:30:40.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.687+0000 7f1f017e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1efc06d3d0 0x7f1efc0cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.687+0000 7f1f017e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1efc06d3d0 0x7f1efc0cec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52844/0 (socket says 172.21.15.5:52844) 2024-05-20T18:30:40.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.687+0000 7f1f017e0640 1 -- 172.21.15.5:0/511797971 learned_addr learned my addr 172.21.15.5:0/511797971 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:40.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.687+0000 7f1f01fe1640 1 --2- 172.21.15.5:0/511797971 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1efc162820 0x7f1efc1fdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.687+0000 7f1f00fdf640 1 --2- 172.21.15.5:0/511797971 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1efc0b8190 0x7f1efc1fd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.688+0000 7f1f017e0640 1 -- 172.21.15.5:0/511797971 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1efc162820 msgr2=0x7f1efc1fdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:40.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.688+0000 7f1f017e0640 1 --2- 172.21.15.5:0/511797971 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1efc162820 0x7f1efc1fdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.688+0000 7f1f017e0640 1 -- 172.21.15.5:0/511797971 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1efc0b8190 msgr2=0x7f1efc1fd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:40.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.688+0000 7f1f017e0640 1 --2- 172.21.15.5:0/511797971 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1efc0b8190 0x7f1efc1fd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.688+0000 7f1f017e0640 1 -- 172.21.15.5:0/511797971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1efc0b6600 con 0x7f1efc06d3d0 2024-05-20T18:30:40.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.688+0000 7f1f017e0640 1 --2- 172.21.15.5:0/511797971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1efc06d3d0 0x7f1efc0cec40 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800c3f0 tx=0x7f1ef800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:40.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.688+0000 7f1ef1ffb640 1 -- 172.21.15.5:0/511797971 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef80052d0 con 0x7f1efc06d3d0 2024-05-20T18:30:40.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.688+0000 7f1ef1ffb640 1 -- 172.21.15.5:0/511797971 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ef8005490 con 0x7f1efc06d3d0 2024-05-20T18:30:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.688+0000 7f1ef1ffb640 1 -- 172.21.15.5:0/511797971 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8007260 con 0x7f1efc06d3d0 2024-05-20T18:30:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.688+0000 7f1f03a44640 1 -- 172.21.15.5:0/511797971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1efc0b68e0 con 0x7f1efc06d3d0 2024-05-20T18:30:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.688+0000 7f1f03a44640 1 -- 172.21.15.5:0/511797971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1efc1513e0 con 0x7f1efc06d3d0 2024-05-20T18:30:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.690+0000 7f1ef1ffb640 1 -- 172.21.15.5:0/511797971 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1ef8007400 con 0x7f1efc06d3d0 2024-05-20T18:30:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.690+0000 7f1ef1ffb640 1 --2- 172.21.15.5:0/511797971 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1ee006be30 0x7f1ee006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.691+0000 7f1f00fdf640 1 --2- 172.21.15.5:0/511797971 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1ee006be30 0x7f1ee006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.691+0000 7f1ef1ffb640 1 -- 172.21.15.5:0/511797971 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7f1ef808a4e0 con 0x7f1efc06d3d0 2024-05-20T18:30:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.691+0000 7f1f00fdf640 1 --2- 172.21.15.5:0/511797971 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1ee006be30 0x7f1ee006e340 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1eec034000 tx=0x7f1eec02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.692+0000 7f1f0a9c9740 1 Processor -- start 2024-05-20T18:30:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.692+0000 7f1f0a9c9740 1 -- start start 2024-05-20T18:30:40.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.693+0000 7f1f0a9c9740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562cb975e390 0x562cb9b28b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.693+0000 7f1f0a9c9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cb97354d0 0x562cb9b43890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.693+0000 7f1f0a9c9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cb973d6b0 0x562cb9b43dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.693+0000 7f1f0a9c9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562cb9a62d00 con 0x562cb97354d0 2024-05-20T18:30:40.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.693+0000 7f1f0a9c9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562cb9a0a4c0 con 0x562cb973d6b0 2024-05-20T18:30:40.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.693+0000 7f1f0a9c9740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x562cb9a5ecc0 con 0x562cb975e390 2024-05-20T18:30:40.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.693+0000 7f1f00fdf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cb97354d0 0x562cb9b43890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.693+0000 7f1f00fdf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cb97354d0 0x562cb9b43890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52858/0 (socket says 172.21.15.5:52858) 2024-05-20T18:30:40.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.693+0000 7f1f00fdf640 1 -- 172.21.15.5:0/2498675481 learned_addr learned my addr 172.21.15.5:0/2498675481 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:40.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.693+0000 7f1f017e0640 1 --2- 172.21.15.5:0/2498675481 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562cb975e390 0x562cb9b28b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.693+0000 7f1f01fe1640 1 --2- 172.21.15.5:0/2498675481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cb973d6b0 0x562cb9b43dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.693+0000 7f1f00fdf640 1 -- 172.21.15.5:0/2498675481 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562cb975e390 msgr2=0x562cb9b28b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:40.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.693+0000 7f1f00fdf640 1 --2- 172.21.15.5:0/2498675481 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562cb975e390 0x562cb9b28b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.693+0000 7f1f00fdf640 1 -- 172.21.15.5:0/2498675481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cb973d6b0 msgr2=0x562cb9b43dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:40.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.693+0000 7f1f00fdf640 1 --2- 172.21.15.5:0/2498675481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cb973d6b0 0x562cb9b43dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.693+0000 7f1f00fdf640 1 -- 172.21.15.5:0/2498675481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cb98407d0 con 0x562cb97354d0 2024-05-20T18:30:40.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.694+0000 7f1f00fdf640 1 --2- 172.21.15.5:0/2498675481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cb97354d0 0x562cb9b43890 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1eec02a860 tx=0x7f1eec02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5a2526d1147355 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:40.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.694+0000 7f1edeffd640 1 -- 172.21.15.5:0/2498675481 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eec037040 con 0x562cb97354d0 2024-05-20T18:30:40.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.694+0000 7f1edeffd640 1 -- 172.21.15.5:0/2498675481 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eec04c070 con 0x562cb97354d0 2024-05-20T18:30:40.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.694+0000 7f1edeffd640 1 -- 172.21.15.5:0/2498675481 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eec032e60 con 0x562cb97354d0 2024-05-20T18:30:40.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.695+0000 7f1f0a9c9740 1 -- 172.21.15.5:0/2498675481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cb97354d0 msgr2=0x562cb9b43890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:40.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.695+0000 7f1f0a9c9740 1 --2- 172.21.15.5:0/2498675481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cb97354d0 0x562cb9b43890 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1eec02a860 tx=0x7f1eec02c000 comp rx=0 tx=0).stop 2024-05-20T18:30:40.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.695+0000 7f1f0a9c9740 1 -- 172.21.15.5:0/2498675481 shutdown_connections 2024-05-20T18:30:40.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.695+0000 7f1f0a9c9740 1 --2- 172.21.15.5:0/2498675481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cb973d6b0 0x562cb9b43dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.695+0000 7f1f0a9c9740 1 --2- 172.21.15.5:0/2498675481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cb97354d0 0x562cb9b43890 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.695+0000 7f1f0a9c9740 1 --2- 172.21.15.5:0/2498675481 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562cb975e390 0x562cb9b28b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.695+0000 7f1f0a9c9740 1 -- 172.21.15.5:0/2498675481 >> 172.21.15.5:0/2498675481 conn(0x562cb97524e0 msgr2=0x562cb9afe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:40.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.695+0000 7f1f0a9c9740 1 -- 172.21.15.5:0/2498675481 shutdown_connections 2024-05-20T18:30:40.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.695+0000 7f1f0a9c9740 1 -- 172.21.15.5:0/2498675481 wait complete. 2024-05-20T18:30:40.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.696+0000 7f1f0a9c9740 1 -- start start 2024-05-20T18:30:40.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.696+0000 7f1f0a9c9740 1 Processor -- start 2024-05-20T18:30:40.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.697+0000 7f1f0a9c9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cb97354d0 0x562cb9b29b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.697+0000 7f1f0a9c9740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562cb973d6b0 0x562cb9b2a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.697+0000 7f1f0a9c9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cb975e390 0x562cb9b453d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.697+0000 7f1f0a9c9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562cb9a1db90 con 0x562cb975e390 2024-05-20T18:30:40.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.697+0000 7f1f0a9c9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562cb9b3aa20 con 0x562cb97354d0 2024-05-20T18:30:40.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.697+0000 7f1f0a9c9740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x562cb9b3ae80 con 0x562cb973d6b0 2024-05-20T18:30:40.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.697+0000 7f1f0a9c9740 10 client.0 _ll_register_callbacks cb 0x562cb9a59da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:30:40.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.697+0000 7f1f01fe1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cb975e390 0x562cb9b453d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.697+0000 7f1f01fe1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cb975e390 0x562cb9b453d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52860/0 (socket says 172.21.15.5:52860) 2024-05-20T18:30:40.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.697+0000 7f1f01fe1640 1 -- 172.21.15.5:0/2895243248 learned_addr learned my addr 172.21.15.5:0/2895243248 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:40.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.697+0000 7f1f00fdf640 1 --2- 172.21.15.5:0/2895243248 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562cb973d6b0 0x562cb9b2a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.697+0000 7f1f017e0640 1 --2- 172.21.15.5:0/2895243248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cb97354d0 0x562cb9b29b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.698+0000 7f1f01fe1640 1 -- 172.21.15.5:0/2895243248 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562cb973d6b0 msgr2=0x562cb9b2a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:40.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.698+0000 7f1f01fe1640 1 --2- 172.21.15.5:0/2895243248 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562cb973d6b0 0x562cb9b2a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.698+0000 7f1f01fe1640 1 -- 172.21.15.5:0/2895243248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cb97354d0 msgr2=0x562cb9b29b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:40.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.698+0000 7f1f01fe1640 1 --2- 172.21.15.5:0/2895243248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cb97354d0 0x562cb9b29b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.698+0000 7f1f01fe1640 1 -- 172.21.15.5:0/2895243248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cb97763f0 con 0x562cb975e390 2024-05-20T18:30:40.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.698+0000 7f1f00fdf640 1 --2- 172.21.15.5:0/2895243248 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562cb973d6b0 0x562cb9b2a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:30:40.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.698+0000 7f1f01fe1640 1 --2- 172.21.15.5:0/2895243248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cb975e390 0x562cb9b453d0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4007dd0 tx=0x7f1ef4012610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2ff414cfc3fdb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:40.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.698+0000 7f1ed7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:30:40.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.698+0000 7f1ed7fff640 1 -- 172.21.15.5:0/2895243248 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef401c020 con 0x562cb975e390 2024-05-20T18:30:40.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.698+0000 7f1f0a9c9740 1 -- 172.21.15.5:0/2895243248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1eec04c070 con 0x562cb975e390 2024-05-20T18:30:40.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.698+0000 7f1ed7fff640 1 -- 172.21.15.5:0/2895243248 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7f1ef4018470 con 0x562cb975e390 2024-05-20T18:30:40.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.698+0000 7f1ed7fff640 1 -- 172.21.15.5:0/2895243248 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1ef4026050 con 0x562cb975e390 2024-05-20T18:30:40.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.698+0000 7f1f0a9c9740 10 client.6501 fetch_fsmap learned FSMap version 35 2024-05-20T18:30:40.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.698+0000 7f1f0a9c9740 1 -- 172.21.15.5:0/2895243248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x562cb97360f0 con 0x562cb975e390 2024-05-20T18:30:40.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.698+0000 7f1ed7fff640 1 -- 172.21.15.5:0/2895243248 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eec04c070 con 0x562cb975e390 2024-05-20T18:30:40.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.698+0000 7f1ed7fff640 1 -- 172.21.15.5:0/2895243248 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f1ef4029020 con 0x562cb975e390 2024-05-20T18:30:40.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.698+0000 7f1f0a9c9740 10 client.6501 fetch_fsmap finished waiting for FSMap version 35 2024-05-20T18:30:40.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.698+0000 7f1f0a9c9740 10 client.6501 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-20T18:30:40.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.698+0000 7f1f0a9c9740 20 client.6501 populate_metadata read hostname 'smithi005' 2024-05-20T18:30:40.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.698+0000 7f1f0a9c9740 1 --2- 172.21.15.5:0/2895243248 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x562cb971cf60 0x562cb9b2c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.698+0000 7f1f0a9c9740 4 client.6501 mds_command: new command op to 6198 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:30:40.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.698+0000 7f1f0a9c9740 1 -- 172.21.15.5:0/2895243248 --> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cb9a70430 con 0x562cb971cf60 2024-05-20T18:30:40.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.699+0000 7f1f017e0640 1 --2- 172.21.15.5:0/2895243248 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x562cb971cf60 0x562cb9b2c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.699+0000 7f1ed7fff640 10 client.6501 ms_handle_connect on v2:172.21.15.63:6832/4036379300 2024-05-20T18:30:40.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.699+0000 7f1f017e0640 1 --2- 172.21.15.5:0/2895243248 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x562cb971cf60 0x562cb9b2c2a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b77c4ca7a534255 server_cookie=9b0ab14b49d34ce7 in_seq=0 out_seq=0 2024-05-20T18:30:40.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.700+0000 7f1ed7fff640 1 -- 172.21.15.5:0/2895243248 <== mds.0 v2:172.21.15.63:6832/4036379300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562cb9a70430 con 0x562cb971cf60 2024-05-20T18:30:40.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.700+0000 7f1ed7fff640 10 client.6501 handle_command_reply: tid=0 2024-05-20T18:30:40.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.700+0000 7f1f0a9c9740 1 client.6501 shutdown 2024-05-20T18:30:40.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.700+0000 7f1f0a9c9740 1 -- 172.21.15.5:0/2895243248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cb975e390 msgr2=0x562cb9b453d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:40.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.700+0000 7f1f0a9c9740 1 --2- 172.21.15.5:0/2895243248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cb975e390 0x562cb9b453d0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4007dd0 tx=0x7f1ef4012610 comp rx=0 tx=0).stop 2024-05-20T18:30:40.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.700+0000 7f1f0a9c9740 1 -- 172.21.15.5:0/2895243248 shutdown_connections 2024-05-20T18:30:40.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.700+0000 7f1f0a9c9740 1 --2- 172.21.15.5:0/2895243248 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x562cb971cf60 0x562cb9b2c2a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.700+0000 7f1f0a9c9740 1 --2- 172.21.15.5:0/2895243248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cb975e390 0x562cb9b453d0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.700+0000 7f1f0a9c9740 1 --2- 172.21.15.5:0/2895243248 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562cb973d6b0 0x562cb9b2a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.700+0000 7f1f0a9c9740 1 --2- 172.21.15.5:0/2895243248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cb97354d0 0x562cb9b29b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.700+0000 7f1ed7fff640 0 client.6501 ms_handle_reset on v2:172.21.15.63:6832/4036379300 2024-05-20T18:30:40.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.700+0000 7f1f0a9c9740 1 -- 172.21.15.5:0/2895243248 >> 172.21.15.5:0/2895243248 conn(0x562cb97524e0 msgr2=0x562cb9afe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:40.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.700+0000 7f1f0a9c9740 1 -- 172.21.15.5:0/2895243248 shutdown_connections 2024-05-20T18:30:40.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.700+0000 7f1f0a9c9740 1 -- 172.21.15.5:0/2895243248 wait complete. 2024-05-20T18:30:40.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.700+0000 7f1f0a9c9740 20 client.6501 trim_cache size 0 max 16384 2024-05-20T18:30:40.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.719+0000 7f1f0a9c9740 1 Processor -- start 2024-05-20T18:30:40.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.719+0000 7f1f0a9c9740 1 -- start start 2024-05-20T18:30:40.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.721+0000 7f1f0a9c9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cb97354d0 0x562cb9b40240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.721+0000 7f1f0a9c9740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562cb973d6b0 0x562cb9b39300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.721+0000 7f1f0a9c9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cb975e390 0x562cb9b39840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.721+0000 7f1f0a9c9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562cb9b465d0 con 0x562cb975e390 2024-05-20T18:30:40.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.721+0000 7f1f0a9c9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc2066c0 con 0x562cb97354d0 2024-05-20T18:30:40.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.721+0000 7f1f0a9c9740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1efc2058e0 con 0x562cb973d6b0 2024-05-20T18:30:40.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.721+0000 7f1f01fe1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cb975e390 0x562cb9b39840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.721+0000 7f1f01fe1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cb975e390 0x562cb9b39840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52876/0 (socket says 172.21.15.5:52876) 2024-05-20T18:30:40.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.721+0000 7f1f01fe1640 1 -- 172.21.15.5:0/1490261813 learned_addr learned my addr 172.21.15.5:0/1490261813 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:40.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.721+0000 7f1f00fdf640 1 --2- 172.21.15.5:0/1490261813 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562cb973d6b0 0x562cb9b39300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.721+0000 7f1f017e0640 1 --2- 172.21.15.5:0/1490261813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cb97354d0 0x562cb9b40240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.721+0000 7f1f01fe1640 1 -- 172.21.15.5:0/1490261813 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562cb973d6b0 msgr2=0x562cb9b39300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.721+0000 7f1f01fe1640 1 --2- 172.21.15.5:0/1490261813 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562cb973d6b0 0x562cb9b39300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.721+0000 7f1f01fe1640 1 -- 172.21.15.5:0/1490261813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cb97354d0 msgr2=0x562cb9b40240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.721+0000 7f1f01fe1640 1 --2- 172.21.15.5:0/1490261813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cb97354d0 0x562cb9b40240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.721+0000 7f1f01fe1640 1 -- 172.21.15.5:0/1490261813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cb9aff620 con 0x562cb975e390 2024-05-20T18:30:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.721+0000 7f1f01fe1640 1 --2- 172.21.15.5:0/1490261813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cb975e390 0x562cb9b39840 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4012b90 tx=0x7f1ef4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85824856c2af80fe server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.721+0000 7f1ede7fc640 1 -- 172.21.15.5:0/1490261813 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef402f070 con 0x562cb975e390 2024-05-20T18:30:40.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.721+0000 7f1ede7fc640 1 -- 172.21.15.5:0/1490261813 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ef4018690 con 0x562cb975e390 2024-05-20T18:30:40.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.721+0000 7f1ede7fc640 1 -- 172.21.15.5:0/1490261813 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef401c070 con 0x562cb975e390 2024-05-20T18:30:40.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.722+0000 7f1f0a9c9740 1 -- 172.21.15.5:0/1490261813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cb975e390 msgr2=0x562cb9b39840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:40.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.722+0000 7f1f0a9c9740 1 --2- 172.21.15.5:0/1490261813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cb975e390 0x562cb9b39840 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4012b90 tx=0x7f1ef4018000 comp rx=0 tx=0).stop 2024-05-20T18:30:40.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.722+0000 7f1f0a9c9740 1 -- 172.21.15.5:0/1490261813 shutdown_connections 2024-05-20T18:30:40.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.722+0000 7f1f0a9c9740 1 --2- 172.21.15.5:0/1490261813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cb975e390 0x562cb9b39840 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.722+0000 7f1f0a9c9740 1 --2- 172.21.15.5:0/1490261813 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562cb973d6b0 0x562cb9b39300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.722+0000 7f1f0a9c9740 1 --2- 172.21.15.5:0/1490261813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cb97354d0 0x562cb9b40240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.722+0000 7f1f0a9c9740 1 -- 172.21.15.5:0/1490261813 >> 172.21.15.5:0/1490261813 conn(0x562cb97524e0 msgr2=0x562cb9b28b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:40.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.722+0000 7f1f0a9c9740 1 -- 172.21.15.5:0/1490261813 shutdown_connections 2024-05-20T18:30:40.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.722+0000 7f1f0a9c9740 1 -- 172.21.15.5:0/1490261813 wait complete. 2024-05-20T18:30:40.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.723+0000 7f1f0a9c9740 1 -- start start 2024-05-20T18:30:40.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.723+0000 7f1f0a9c9740 1 Processor -- start 2024-05-20T18:30:40.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.725+0000 7f1f0a9c9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cb97354d0 0x562cb9bbfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.725+0000 7f1f0a9c9740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562cb973d6b0 0x562cb9afc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.725+0000 7f1f0a9c9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cb975e390 0x562cb9b24bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.725+0000 7f1f0a9c9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc150c40 con 0x562cb97354d0 2024-05-20T18:30:40.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.725+0000 7f1f0a9c9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc1505f0 con 0x562cb975e390 2024-05-20T18:30:40.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.725+0000 7f1f0a9c9740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x562cb9afcff0 con 0x562cb973d6b0 2024-05-20T18:30:40.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.725+0000 7f1f0a9c9740 10 client.0 _ll_register_callbacks cb 0x7f1ef4009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:30:40.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.725+0000 7f1f017e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cb97354d0 0x562cb9bbfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.725+0000 7f1f017e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cb97354d0 0x562cb9bbfd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52882/0 (socket says 172.21.15.5:52882) 2024-05-20T18:30:40.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.725+0000 7f1f017e0640 1 -- 172.21.15.5:0/2265940604 learned_addr learned my addr 172.21.15.5:0/2265940604 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:30:40.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.725+0000 7f1f00fdf640 1 --2- 172.21.15.5:0/2265940604 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562cb973d6b0 0x562cb9afc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.725+0000 7f1f017e0640 1 -- 172.21.15.5:0/2265940604 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562cb973d6b0 msgr2=0x562cb9afc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:40.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.725+0000 7f1f017e0640 1 --2- 172.21.15.5:0/2265940604 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562cb973d6b0 0x562cb9afc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.725+0000 7f1f01fe1640 1 --2- 172.21.15.5:0/2265940604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cb975e390 0x562cb9b24bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.725+0000 7f1f017e0640 1 -- 172.21.15.5:0/2265940604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cb975e390 msgr2=0x562cb9b24bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:30:40.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.725+0000 7f1f017e0640 1 --2- 172.21.15.5:0/2265940604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cb975e390 0x562cb9b24bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.725+0000 7f1f017e0640 1 -- 172.21.15.5:0/2265940604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cb9b44cd0 con 0x562cb97354d0 2024-05-20T18:30:40.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.725+0000 7f1f00fdf640 1 --2- 172.21.15.5:0/2265940604 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562cb973d6b0 0x562cb9afc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:30:40.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.725+0000 7f1f017e0640 1 --2- 172.21.15.5:0/2265940604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cb97354d0 0x562cb9bbfd60 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8060cb0 tx=0x7f1ef8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2009d14f48f291ff server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:30:40.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.725+0000 7f1ed7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:30:40.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.725+0000 7f1ed7fff640 1 -- 172.21.15.5:0/2265940604 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef80611e0 con 0x562cb97354d0 2024-05-20T18:30:40.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.725+0000 7f1ed7fff640 1 -- 172.21.15.5:0/2265940604 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7f1ef8056900 con 0x562cb97354d0 2024-05-20T18:30:40.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.726+0000 7f1f0a9c9740 1 -- 172.21.15.5:0/2265940604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ef4018690 con 0x562cb97354d0 2024-05-20T18:30:40.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.726+0000 7f1ed7fff640 1 -- 172.21.15.5:0/2265940604 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1ef8054c70 con 0x562cb97354d0 2024-05-20T18:30:40.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.726+0000 7f1f0a9c9740 10 client.6507 fetch_fsmap learned FSMap version 35 2024-05-20T18:30:40.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.726+0000 7f1f0a9c9740 1 -- 172.21.15.5:0/2265940604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x562cb9afcd90 con 0x562cb97354d0 2024-05-20T18:30:40.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.726+0000 7f1ed7fff640 1 -- 172.21.15.5:0/2265940604 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ef4018690 con 0x562cb97354d0 2024-05-20T18:30:40.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.726+0000 7f1ed7fff640 1 -- 172.21.15.5:0/2265940604 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f1ef8060d50 con 0x562cb97354d0 2024-05-20T18:30:40.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.726+0000 7f1f0a9c9740 10 client.6507 fetch_fsmap finished waiting for FSMap version 35 2024-05-20T18:30:40.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.726+0000 7f1f0a9c9740 10 client.6507 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-20T18:30:40.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.726+0000 7f1f0a9c9740 20 client.6507 populate_metadata read hostname 'smithi005' 2024-05-20T18:30:40.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.726+0000 7f1f0a9c9740 1 --2- 172.21.15.5:0/2265940604 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x562cb9b3a160 0x562cb9af7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:30:40.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.726+0000 7f1f0a9c9740 4 client.6507 mds_command: new command op to 6198 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T18:30:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.726+0000 7f1f0a9c9740 1 -- 172.21.15.5:0/2265940604 --> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562cb9a6a780 con 0x562cb9b3a160 2024-05-20T18:30:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.726+0000 7f1f00fdf640 1 --2- 172.21.15.5:0/2265940604 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x562cb9b3a160 0x562cb9af7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:30:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.727+0000 7f1ed7fff640 10 client.6507 ms_handle_connect on v2:172.21.15.63:6832/4036379300 2024-05-20T18:30:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.727+0000 7f1f00fdf640 1 --2- 172.21.15.5:0/2265940604 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x562cb9b3a160 0x562cb9af7ea0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb7ac3a5839d0bd6 server_cookie=afd0df94b4cc5ff5 in_seq=0 out_seq=0 2024-05-20T18:30:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.727+0000 7f1ed7fff640 1 -- 172.21.15.5:0/2265940604 <== mds.0 v2:172.21.15.63:6832/4036379300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x562cb9a6a780 con 0x562cb9b3a160 2024-05-20T18:30:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.727+0000 7f1ed7fff640 10 client.6507 handle_command_reply: tid=0 2024-05-20T18:30:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.727+0000 7f1f0a9c9740 1 client.6507 shutdown 2024-05-20T18:30:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.727+0000 7f1f0a9c9740 1 -- 172.21.15.5:0/2265940604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cb97354d0 msgr2=0x562cb9bbfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.727+0000 7f1f0a9c9740 1 --2- 172.21.15.5:0/2265940604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cb97354d0 0x562cb9bbfd60 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8060cb0 tx=0x7f1ef8000ed0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.727+0000 7f1f0a9c9740 1 -- 172.21.15.5:0/2265940604 shutdown_connections 2024-05-20T18:30:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.727+0000 7f1f0a9c9740 1 --2- 172.21.15.5:0/2265940604 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x562cb9b3a160 0x562cb9af7ea0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.727+0000 7f1f0a9c9740 1 --2- 172.21.15.5:0/2265940604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562cb975e390 0x562cb9b24bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.727+0000 7f1ed7fff640 0 client.6507 ms_handle_reset on v2:172.21.15.63:6832/4036379300 2024-05-20T18:30:40.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.727+0000 7f1f0a9c9740 1 --2- 172.21.15.5:0/2265940604 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562cb973d6b0 0x562cb9afc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.727+0000 7f1f0a9c9740 1 --2- 172.21.15.5:0/2265940604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cb97354d0 0x562cb9bbfd60 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.727+0000 7f1f0a9c9740 1 -- 172.21.15.5:0/2265940604 >> 172.21.15.5:0/2265940604 conn(0x562cb97524e0 msgr2=0x562cb9b28b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:40.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.727+0000 7f1f0a9c9740 1 -- 172.21.15.5:0/2265940604 shutdown_connections 2024-05-20T18:30:40.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.727+0000 7f1f0a9c9740 1 -- 172.21.15.5:0/2265940604 wait complete. 2024-05-20T18:30:40.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.727+0000 7f1f0a9c9740 20 client.6507 trim_cache size 0 max 16384 2024-05-20T18:30:40.735 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:30:40.735 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-20T18:30:40.735 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-20T18:30:40.735 INFO:teuthology.orchestra.run.smithi005.stdout: "c91f971a-767d-4d2c-ad04-68bbc09b23e2": { 2024-05-20T18:30:40.735 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-05-20T18:30:40.735 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "c91f971a-767d-4d2c-ad04-68bbc09b23e2", 2024-05-20T18:30:40.735 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,repair,force" 2024-05-20T18:30:40.735 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:30:40.736 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:30:40.736 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:30:40.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.728+0000 7f1f03a44640 1 -- 172.21.15.5:0/511797971 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1ee006be30 msgr2=0x7f1ee006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:40.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.728+0000 7f1f03a44640 1 --2- 172.21.15.5:0/511797971 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1ee006be30 0x7f1ee006e340 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1eec034000 tx=0x7f1eec02e000 comp rx=0 tx=0).stop 2024-05-20T18:30:40.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.728+0000 7f1f03a44640 1 -- 172.21.15.5:0/511797971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1efc06d3d0 msgr2=0x7f1efc0cec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:30:40.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.728+0000 7f1f03a44640 1 --2- 172.21.15.5:0/511797971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1efc06d3d0 0x7f1efc0cec40 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800c3f0 tx=0x7f1ef800c8b0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.728+0000 7f1f03a44640 1 -- 172.21.15.5:0/511797971 shutdown_connections 2024-05-20T18:30:40.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.728+0000 7f1f03a44640 1 --2- 172.21.15.5:0/511797971 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1ee006be30 0x7f1ee006e340 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.728+0000 7f1f03a44640 1 --2- 172.21.15.5:0/511797971 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1efc162820 0x7f1efc1fdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.728+0000 7f1f03a44640 1 --2- 172.21.15.5:0/511797971 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1efc0b8190 0x7f1efc1fd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.728+0000 7f1f03a44640 1 --2- 172.21.15.5:0/511797971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1efc06d3d0 0x7f1efc0cec40 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:30:40.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.728+0000 7f1f03a44640 1 -- 172.21.15.5:0/511797971 >> 172.21.15.5:0/511797971 conn(0x7f1efc0d4bd0 msgr2=0x7f1efc15e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:30:40.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.728+0000 7f1f03a44640 1 -- 172.21.15.5:0/511797971 shutdown_connections 2024-05-20T18:30:40.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:30:40.728+0000 7f1f03a44640 1 -- 172.21.15.5:0/511797971 wait complete. 2024-05-20T18:30:40.740 INFO:tasks.cephfs.filesystem:scrub status for tag:c91f971a-767d-4d2c-ad04-68bbc09b23e2 - {'path': '/', 'tag': 'c91f971a-767d-4d2c-ad04-68bbc09b23e2', 'options': 'recursive,repair,force'} 2024-05-20T18:30:41.129 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:30:41.132 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:31:10.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:31:10.741 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-05-20T18:31:10.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.873+0000 7f3f7a41b640 1 Processor -- start 2024-05-20T18:31:10.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.873+0000 7f3f7a41b640 1 -- start start 2024-05-20T18:31:10.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.874+0000 7f3f7a41b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3f7415f220 0x7f3f740d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:10.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.874+0000 7f3f7a41b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3f740d0e00 0x7f3f740cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:10.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.874+0000 7f3f7a41b640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f740cfd20 con 0x7f3f740cf910 2024-05-20T18:31:10.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.874+0000 7f3f7a41b640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f3f740cda20 con 0x7f3f740d0e00 2024-05-20T18:31:10.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.874+0000 7f3f7a41b640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f3f740cdd00 con 0x7f3f7415f220 2024-05-20T18:31:10.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.875+0000 7f3f737fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3f740d0e00 0x7f3f740cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:10.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.875+0000 7f3f789b8640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3f740cf910 0x7f3f740ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37364/0 (socket says 172.21.15.5:37364) 2024-05-20T18:31:10.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.875+0000 7f3f789b8640 1 -- 172.21.15.5:0/2119141675 learned_addr learned my addr 172.21.15.5:0/2119141675 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:10.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.875+0000 7f3f73fff640 1 --2- 172.21.15.5:0/2119141675 >> v2:172.21.15.63:3300/0 conn(0x7f3f7415f220 0x7f3f740d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:10.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.875+0000 7f3f737fe640 1 -- 172.21.15.5:0/2119141675 >> v2:172.21.15.63:3300/0 conn(0x7f3f7415f220 msgr2=0x7f3f740d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:10.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.875+0000 7f3f737fe640 1 --2- 172.21.15.5:0/2119141675 >> v2:172.21.15.63:3300/0 conn(0x7f3f7415f220 0x7f3f740d08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.875+0000 7f3f737fe640 1 -- 172.21.15.5:0/2119141675 >> v1:172.21.15.5:6789/0 conn(0x7f3f740cf910 legacy=0x7f3f740ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:10.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.875+0000 7f3f72ffd640 1 -- 172.21.15.5:0/2119141675 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1900657176 0 0) 0x7f3f6c001490 con 0x7f3f740cf910 2024-05-20T18:31:10.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.875+0000 7f3f737fe640 1 -- 172.21.15.5:0/2119141675 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f740cdfe0 con 0x7f3f740d0e00 2024-05-20T18:31:10.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.876+0000 7f3f737fe640 1 --2- 172.21.15.5:0/2119141675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f740d0e00 0x7f3f740cf3d0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f3f7406e810 tx=0x7f3f6802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97f6b89b2e033e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:10.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.876+0000 7f3f72ffd640 1 -- 172.21.15.5:0/2119141675 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f68037070 con 0x7f3f740d0e00 2024-05-20T18:31:10.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.876+0000 7f3f72ffd640 1 -- 172.21.15.5:0/2119141675 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f68032440 con 0x7f3f740d0e00 2024-05-20T18:31:10.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.876+0000 7f3f72ffd640 1 -- 172.21.15.5:0/2119141675 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6802ae60 con 0x7f3f740d0e00 2024-05-20T18:31:10.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.877+0000 7f3f7a41b640 1 -- 172.21.15.5:0/2119141675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f740d0e00 msgr2=0x7f3f740cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:10.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.877+0000 7f3f7a41b640 1 --2- 172.21.15.5:0/2119141675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f740d0e00 0x7f3f740cf3d0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f3f7406e810 tx=0x7f3f6802a110 comp rx=0 tx=0).stop 2024-05-20T18:31:10.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.877+0000 7f3f7a41b640 1 -- 172.21.15.5:0/2119141675 shutdown_connections 2024-05-20T18:31:10.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.877+0000 7f3f7a41b640 1 --2- 172.21.15.5:0/2119141675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f740d0e00 0x7f3f740cf3d0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.877+0000 7f3f7a41b640 1 --2- 172.21.15.5:0/2119141675 >> v2:172.21.15.63:3300/0 conn(0x7f3f7415f220 0x7f3f740d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.877+0000 7f3f7a41b640 1 -- 172.21.15.5:0/2119141675 >> 172.21.15.5:0/2119141675 conn(0x7f3f740b44d0 msgr2=0x7f3f740b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:10.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.877+0000 7f3f7a41b640 1 -- 172.21.15.5:0/2119141675 shutdown_connections 2024-05-20T18:31:10.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.877+0000 7f3f7a41b640 1 -- 172.21.15.5:0/2119141675 wait complete. 2024-05-20T18:31:10.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.878+0000 7f3f7a41b640 1 Processor -- start 2024-05-20T18:31:10.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.879+0000 7f3f7a41b640 1 -- start start 2024-05-20T18:31:10.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.880+0000 7f3f7a41b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f740cf910 0x7f3f740c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:10.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.880+0000 7f3f7a41b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f740d0e00 0x7f3f740c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:10.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.880+0000 7f3f7a41b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f7415f220 0x7f3f7415f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:10.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.880+0000 7f3f7a41b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3f740cd800 con 0x7f3f740d0e00 2024-05-20T18:31:10.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.880+0000 7f3f7a41b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3f740cd680 con 0x7f3f7415f220 2024-05-20T18:31:10.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.880+0000 7f3f7a41b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3f7415f630 con 0x7f3f740cf910 2024-05-20T18:31:10.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.880+0000 7f3f737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f740d0e00 0x7f3f740c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:10.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.880+0000 7f3f737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f740d0e00 0x7f3f740c7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40050/0 (socket says 172.21.15.5:40050) 2024-05-20T18:31:10.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.880+0000 7f3f737fe640 1 -- 172.21.15.5:0/3873284942 learned_addr learned my addr 172.21.15.5:0/3873284942 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:10.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.880+0000 7f3f73fff640 1 --2- 172.21.15.5:0/3873284942 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f740cf910 0x7f3f740c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:10.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.880+0000 7f3f789b8640 1 --2- 172.21.15.5:0/3873284942 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f7415f220 0x7f3f7415f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:10.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.880+0000 7f3f737fe640 1 -- 172.21.15.5:0/3873284942 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f740cf910 msgr2=0x7f3f740c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:10.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.880+0000 7f3f737fe640 1 --2- 172.21.15.5:0/3873284942 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f740cf910 0x7f3f740c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.880+0000 7f3f737fe640 1 -- 172.21.15.5:0/3873284942 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f7415f220 msgr2=0x7f3f7415f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:10.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.880+0000 7f3f737fe640 1 --2- 172.21.15.5:0/3873284942 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f7415f220 0x7f3f7415f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.881+0000 7f3f737fe640 1 -- 172.21.15.5:0/3873284942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f74161840 con 0x7f3f740d0e00 2024-05-20T18:31:10.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.881+0000 7f3f737fe640 1 --2- 172.21.15.5:0/3873284942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f740d0e00 0x7f3f740c7a20 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f3f68000c00 tx=0x7f3f680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:10.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.881+0000 7f3f70ff9640 1 -- 172.21.15.5:0/3873284942 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f68037040 con 0x7f3f740d0e00 2024-05-20T18:31:10.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.881+0000 7f3f70ff9640 1 -- 172.21.15.5:0/3873284942 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f68042070 con 0x7f3f740d0e00 2024-05-20T18:31:10.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.881+0000 7f3f7a41b640 1 -- 172.21.15.5:0/3873284942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f741f9670 con 0x7f3f740d0e00 2024-05-20T18:31:10.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.881+0000 7f3f70ff9640 1 -- 172.21.15.5:0/3873284942 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f68030960 con 0x7f3f740d0e00 2024-05-20T18:31:10.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.881+0000 7f3f7a41b640 1 -- 172.21.15.5:0/3873284942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f741f9c30 con 0x7f3f740d0e00 2024-05-20T18:31:10.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.883+0000 7f3f70ff9640 1 -- 172.21.15.5:0/3873284942 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3f68030b00 con 0x7f3f740d0e00 2024-05-20T18:31:10.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.883+0000 7f3f70ff9640 1 --2- 172.21.15.5:0/3873284942 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3f6006be30 0x7f3f6006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:10.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.883+0000 7f3f70ff9640 1 -- 172.21.15.5:0/3873284942 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6246+0+0 (secure 0 0 0) 0x7f3f680ab920 con 0x7f3f740d0e00 2024-05-20T18:31:10.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.883+0000 7f3f73fff640 1 --2- 172.21.15.5:0/3873284942 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3f6006be30 0x7f3f6006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:10.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.884+0000 7f3f73fff640 1 --2- 172.21.15.5:0/3873284942 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3f6006be30 0x7f3f6006e340 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3f740c64c0 tx=0x7f3f64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:10.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.885+0000 7f3f8139f740 1 Processor -- start 2024-05-20T18:31:10.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.885+0000 7f3f8139f740 1 -- start start 2024-05-20T18:31:10.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.886+0000 7f3f8139f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601660dc390 0x5601664a6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:10.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.886+0000 7f3f8139f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601660b34d0 0x5601664c1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:10.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.886+0000 7f3f8139f740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5601660bb6b0 0x5601664c1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:10.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.886+0000 7f3f8139f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5601663e0d00 con 0x5601660dc390 2024-05-20T18:31:10.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.886+0000 7f3f8139f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5601663884c0 con 0x5601660b34d0 2024-05-20T18:31:10.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.886+0000 7f3f8139f740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5601663dccc0 con 0x5601660bb6b0 2024-05-20T18:31:10.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.886+0000 7f3f73fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601660dc390 0x5601664a6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:10.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.886+0000 7f3f73fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601660dc390 0x5601664a6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40058/0 (socket says 172.21.15.5:40058) 2024-05-20T18:31:10.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.886+0000 7f3f73fff640 1 -- 172.21.15.5:0/1269572753 learned_addr learned my addr 172.21.15.5:0/1269572753 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:10.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.886+0000 7f3f737fe640 1 --2- 172.21.15.5:0/1269572753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601660b34d0 0x5601664c1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:10.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.886+0000 7f3f789b8640 1 --2- 172.21.15.5:0/1269572753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5601660bb6b0 0x5601664c1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:10.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.886+0000 7f3f73fff640 1 -- 172.21.15.5:0/1269572753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5601660bb6b0 msgr2=0x5601664c1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:10.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.886+0000 7f3f73fff640 1 --2- 172.21.15.5:0/1269572753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5601660bb6b0 0x5601664c1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.886+0000 7f3f73fff640 1 -- 172.21.15.5:0/1269572753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601660b34d0 msgr2=0x5601664c1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:10.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.886+0000 7f3f73fff640 1 --2- 172.21.15.5:0/1269572753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601660b34d0 0x5601664c1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.886+0000 7f3f73fff640 1 -- 172.21.15.5:0/1269572753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601661be7d0 con 0x5601660dc390 2024-05-20T18:31:10.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.886+0000 7f3f737fe640 1 --2- 172.21.15.5:0/1269572753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601660b34d0 0x5601664c1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:10.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.887+0000 7f3f73fff640 1 --2- 172.21.15.5:0/1269572753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601660dc390 0x5601664a6b60 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f3f64011d00 tx=0x7f3f64014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e39e4569c954947 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:10.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.887+0000 7f3f59ffb640 1 -- 172.21.15.5:0/1269572753 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f64011d30 con 0x5601660dc390 2024-05-20T18:31:10.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.887+0000 7f3f59ffb640 1 -- 172.21.15.5:0/1269572753 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f64026070 con 0x5601660dc390 2024-05-20T18:31:10.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.887+0000 7f3f59ffb640 1 -- 172.21.15.5:0/1269572753 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f64021640 con 0x5601660dc390 2024-05-20T18:31:10.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.888+0000 7f3f8139f740 1 -- 172.21.15.5:0/1269572753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601660dc390 msgr2=0x5601664a6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:10.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.888+0000 7f3f8139f740 1 --2- 172.21.15.5:0/1269572753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601660dc390 0x5601664a6b60 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f3f64011d00 tx=0x7f3f64014000 comp rx=0 tx=0).stop 2024-05-20T18:31:10.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.888+0000 7f3f8139f740 1 -- 172.21.15.5:0/1269572753 shutdown_connections 2024-05-20T18:31:10.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.888+0000 7f3f8139f740 1 --2- 172.21.15.5:0/1269572753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5601660bb6b0 0x5601664c1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.888+0000 7f3f8139f740 1 --2- 172.21.15.5:0/1269572753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601660b34d0 0x5601664c1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.888+0000 7f3f8139f740 1 --2- 172.21.15.5:0/1269572753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601660dc390 0x5601664a6b60 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.888+0000 7f3f8139f740 1 -- 172.21.15.5:0/1269572753 >> 172.21.15.5:0/1269572753 conn(0x5601660d04e0 msgr2=0x56016647c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:10.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.888+0000 7f3f8139f740 1 -- 172.21.15.5:0/1269572753 shutdown_connections 2024-05-20T18:31:10.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.888+0000 7f3f8139f740 1 -- 172.21.15.5:0/1269572753 wait complete. 2024-05-20T18:31:10.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.889+0000 7f3f8139f740 1 -- start start 2024-05-20T18:31:10.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.889+0000 7f3f8139f740 1 Processor -- start 2024-05-20T18:31:10.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.890+0000 7f3f8139f740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5601660b34d0 0x5601664bb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:10.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.890+0000 7f3f8139f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601660bb6b0 0x5601664a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:10.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.890+0000 7f3f8139f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601660dc390 0x5601664bc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:10.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.890+0000 7f3f8139f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56016639bb90 con 0x5601660bb6b0 2024-05-20T18:31:10.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.890+0000 7f3f8139f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5601664d1db0 con 0x5601660dc390 2024-05-20T18:31:10.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.890+0000 7f3f8139f740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5601664ba1d0 con 0x5601660b34d0 2024-05-20T18:31:10.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.890+0000 7f3f8139f740 10 client.0 _ll_register_callbacks cb 0x5601663d7da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:31:10.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.890+0000 7f3f737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601660bb6b0 0x5601664a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.890+0000 7f3f737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601660bb6b0 0x5601664a23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40066/0 (socket says 172.21.15.5:40066) 2024-05-20T18:31:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.890+0000 7f3f737fe640 1 -- 172.21.15.5:0/553473433 learned_addr learned my addr 172.21.15.5:0/553473433 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.891+0000 7f3f73fff640 1 --2- 172.21.15.5:0/553473433 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5601660b34d0 0x5601664bb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.891+0000 7f3f789b8640 1 --2- 172.21.15.5:0/553473433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601660dc390 0x5601664bc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.891+0000 7f3f737fe640 1 -- 172.21.15.5:0/553473433 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5601660b34d0 msgr2=0x5601664bb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.891+0000 7f3f737fe640 1 --2- 172.21.15.5:0/553473433 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5601660b34d0 0x5601664bb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.891+0000 7f3f737fe640 1 -- 172.21.15.5:0/553473433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601660dc390 msgr2=0x5601664bc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.891+0000 7f3f737fe640 1 --2- 172.21.15.5:0/553473433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601660dc390 0x5601664bc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.891+0000 7f3f737fe640 1 -- 172.21.15.5:0/553473433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601660f43f0 con 0x5601660bb6b0 2024-05-20T18:31:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.891+0000 7f3f789b8640 1 --2- 172.21.15.5:0/553473433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601660dc390 0x5601664bc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.891+0000 7f3f737fe640 1 --2- 172.21.15.5:0/553473433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601660bb6b0 0x5601664a23f0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f3f680a9c30 tx=0x7f3f68032440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72392dadc6e8fbb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.891+0000 7f3f52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:31:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.891+0000 7f3f52ffd640 1 -- 172.21.15.5:0/553473433 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f680a7020 con 0x5601660bb6b0 2024-05-20T18:31:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.891+0000 7f3f52ffd640 1 -- 172.21.15.5:0/553473433 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 6246+0+0 (secure 0 0 0) 0x7f3f68082100 con 0x5601660bb6b0 2024-05-20T18:31:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.891+0000 7f3f8139f740 1 -- 172.21.15.5:0/553473433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f64026070 con 0x5601660bb6b0 2024-05-20T18:31:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.891+0000 7f3f52ffd640 1 -- 172.21.15.5:0/553473433 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3f6807a020 con 0x5601660bb6b0 2024-05-20T18:31:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.891+0000 7f3f8139f740 10 client.6522 fetch_fsmap learned FSMap version 36 2024-05-20T18:31:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.891+0000 7f3f8139f740 1 -- 172.21.15.5:0/553473433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x5601660b40f0 con 0x5601660bb6b0 2024-05-20T18:31:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.891+0000 7f3f52ffd640 1 -- 172.21.15.5:0/553473433 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f64026070 con 0x5601660bb6b0 2024-05-20T18:31:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.892+0000 7f3f52ffd640 1 -- 172.21.15.5:0/553473433 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f3f6803a070 con 0x5601660bb6b0 2024-05-20T18:31:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.892+0000 7f3f8139f740 10 client.6522 fetch_fsmap finished waiting for FSMap version 36 2024-05-20T18:31:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.892+0000 7f3f8139f740 10 client.6522 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-20T18:31:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.892+0000 7f3f8139f740 20 client.6522 populate_metadata read hostname 'smithi005' 2024-05-20T18:31:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.892+0000 7f3f8139f740 1 --2- 172.21.15.5:0/553473433 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x56016609af60 0x5601664a92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.892+0000 7f3f8139f740 4 client.6522 mds_command: new command op to 6198 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:31:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.892+0000 7f3f8139f740 1 -- 172.21.15.5:0/553473433 --> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601663ee430 con 0x56016609af60 2024-05-20T18:31:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.892+0000 7f3f73fff640 1 --2- 172.21.15.5:0/553473433 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x56016609af60 0x5601664a92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.892+0000 7f3f52ffd640 10 client.6522 ms_handle_connect on v2:172.21.15.63:6832/4036379300 2024-05-20T18:31:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.892+0000 7f3f73fff640 1 --2- 172.21.15.5:0/553473433 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x56016609af60 0x5601664a92d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e6a64a19c4218cf server_cookie=a9246ffd09d990d2 in_seq=0 out_seq=0 2024-05-20T18:31:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.893+0000 7f3f52ffd640 1 -- 172.21.15.5:0/553473433 <== mds.0 v2:172.21.15.63:6832/4036379300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5601663ee430 con 0x56016609af60 2024-05-20T18:31:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.893+0000 7f3f52ffd640 10 client.6522 handle_command_reply: tid=0 2024-05-20T18:31:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.893+0000 7f3f8139f740 1 client.6522 shutdown 2024-05-20T18:31:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.893+0000 7f3f8139f740 1 -- 172.21.15.5:0/553473433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601660bb6b0 msgr2=0x5601664a23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.893+0000 7f3f8139f740 1 --2- 172.21.15.5:0/553473433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601660bb6b0 0x5601664a23f0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f3f680a9c30 tx=0x7f3f68032440 comp rx=0 tx=0).stop 2024-05-20T18:31:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.893+0000 7f3f8139f740 1 -- 172.21.15.5:0/553473433 shutdown_connections 2024-05-20T18:31:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.893+0000 7f3f8139f740 1 --2- 172.21.15.5:0/553473433 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x56016609af60 0x5601664a92d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.893+0000 7f3f8139f740 1 --2- 172.21.15.5:0/553473433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5601660dc390 0x5601664bc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.893+0000 7f3f8139f740 1 --2- 172.21.15.5:0/553473433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601660bb6b0 0x5601664a23f0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.893+0000 7f3f52ffd640 0 client.6522 ms_handle_reset on v2:172.21.15.63:6832/4036379300 2024-05-20T18:31:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.893+0000 7f3f8139f740 1 --2- 172.21.15.5:0/553473433 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5601660b34d0 0x5601664bb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.893+0000 7f3f8139f740 1 -- 172.21.15.5:0/553473433 >> 172.21.15.5:0/553473433 conn(0x5601660d04e0 msgr2=0x56016647c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.893+0000 7f3f8139f740 1 -- 172.21.15.5:0/553473433 shutdown_connections 2024-05-20T18:31:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.893+0000 7f3f8139f740 1 -- 172.21.15.5:0/553473433 wait complete. 2024-05-20T18:31:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.894+0000 7f3f8139f740 20 client.6522 trim_cache size 0 max 16384 2024-05-20T18:31:10.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.912+0000 7f3f8139f740 1 Processor -- start 2024-05-20T18:31:10.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.912+0000 7f3f8139f740 1 -- start start 2024-05-20T18:31:10.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.913+0000 7f3f8139f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56016609af60 0x5601664a6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:10.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.913+0000 7f3f8139f740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5601660b34d0 0x5601664a1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:10.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.913+0000 7f3f8139f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601660bb6b0 0x5601664a1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:10.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.913+0000 7f3f8139f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3f74202800 con 0x5601660bb6b0 2024-05-20T18:31:10.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.913+0000 7f3f8139f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3f742019c0 con 0x56016609af60 2024-05-20T18:31:10.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.913+0000 7f3f8139f740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3f741f60b0 con 0x5601660b34d0 2024-05-20T18:31:10.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.913+0000 7f3f789b8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601660bb6b0 0x5601664a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:10.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.914+0000 7f3f789b8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601660bb6b0 0x5601664a1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40080/0 (socket says 172.21.15.5:40080) 2024-05-20T18:31:10.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.914+0000 7f3f789b8640 1 -- 172.21.15.5:0/289318032 learned_addr learned my addr 172.21.15.5:0/289318032 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:10.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.914+0000 7f3f737fe640 1 --2- 172.21.15.5:0/289318032 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5601660b34d0 0x5601664a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:10.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.914+0000 7f3f73fff640 1 --2- 172.21.15.5:0/289318032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56016609af60 0x5601664a6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:10.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.914+0000 7f3f789b8640 1 -- 172.21.15.5:0/289318032 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5601660b34d0 msgr2=0x5601664a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.914+0000 7f3f789b8640 1 --2- 172.21.15.5:0/289318032 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5601660b34d0 0x5601664a1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.914+0000 7f3f789b8640 1 -- 172.21.15.5:0/289318032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56016609af60 msgr2=0x5601664a6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.914+0000 7f3f789b8640 1 --2- 172.21.15.5:0/289318032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56016609af60 0x5601664a6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.914+0000 7f3f789b8640 1 -- 172.21.15.5:0/289318032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560166471830 con 0x5601660bb6b0 2024-05-20T18:31:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.914+0000 7f3f737fe640 1 --2- 172.21.15.5:0/289318032 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5601660b34d0 0x5601664a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.914+0000 7f3f789b8640 1 --2- 172.21.15.5:0/289318032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601660bb6b0 0x5601664a1980 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c0012e0 tx=0x7f3f6c0078b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a9ee41e00a9a7a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.914+0000 7f3f597fa640 1 -- 172.21.15.5:0/289318032 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c01a020 con 0x5601660bb6b0 2024-05-20T18:31:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.914+0000 7f3f597fa640 1 -- 172.21.15.5:0/289318032 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f6c012810 con 0x5601660bb6b0 2024-05-20T18:31:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.914+0000 7f3f597fa640 1 -- 172.21.15.5:0/289318032 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c012bc0 con 0x5601660bb6b0 2024-05-20T18:31:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.915+0000 7f3f8139f740 1 -- 172.21.15.5:0/289318032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601660bb6b0 msgr2=0x5601664a1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.915+0000 7f3f8139f740 1 --2- 172.21.15.5:0/289318032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601660bb6b0 0x5601664a1980 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c0012e0 tx=0x7f3f6c0078b0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.915+0000 7f3f8139f740 1 -- 172.21.15.5:0/289318032 shutdown_connections 2024-05-20T18:31:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.915+0000 7f3f8139f740 1 --2- 172.21.15.5:0/289318032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601660bb6b0 0x5601664a1980 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.915+0000 7f3f8139f740 1 --2- 172.21.15.5:0/289318032 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5601660b34d0 0x5601664a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.915+0000 7f3f8139f740 1 --2- 172.21.15.5:0/289318032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56016609af60 0x5601664a6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.915+0000 7f3f8139f740 1 -- 172.21.15.5:0/289318032 >> 172.21.15.5:0/289318032 conn(0x5601660d04e0 msgr2=0x56016647c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:10.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.915+0000 7f3f8139f740 1 -- 172.21.15.5:0/289318032 shutdown_connections 2024-05-20T18:31:10.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.915+0000 7f3f8139f740 1 -- 172.21.15.5:0/289318032 wait complete. 2024-05-20T18:31:10.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.916+0000 7f3f8139f740 1 -- start start 2024-05-20T18:31:10.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.916+0000 7f3f8139f740 1 Processor -- start 2024-05-20T18:31:10.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.917+0000 7f3f8139f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56016609af60 0x56016653d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:10.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.917+0000 7f3f8139f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601660b34d0 0x56016647c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:10.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.917+0000 7f3f8139f740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5601660bb6b0 0x5601664e3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:10.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.917+0000 7f3f8139f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3f741f6700 con 0x5601660b34d0 2024-05-20T18:31:10.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.917+0000 7f3f8139f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5601664729c0 con 0x56016609af60 2024-05-20T18:31:10.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.917+0000 7f3f8139f740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5601664aa750 con 0x5601660bb6b0 2024-05-20T18:31:10.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.917+0000 7f3f8139f740 10 client.0 _ll_register_callbacks cb 0x7f3f6802a610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:31:10.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.918+0000 7f3f737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601660b34d0 0x56016647c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:10.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.918+0000 7f3f737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601660b34d0 0x56016647c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40082/0 (socket says 172.21.15.5:40082) 2024-05-20T18:31:10.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.918+0000 7f3f737fe640 1 -- 172.21.15.5:0/2641704928 learned_addr learned my addr 172.21.15.5:0/2641704928 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:10.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.918+0000 7f3f789b8640 1 --2- 172.21.15.5:0/2641704928 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5601660bb6b0 0x5601664e3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:10.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.918+0000 7f3f73fff640 1 --2- 172.21.15.5:0/2641704928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56016609af60 0x56016653d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:10.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.918+0000 7f3f737fe640 1 -- 172.21.15.5:0/2641704928 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5601660bb6b0 msgr2=0x5601664e3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:10.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.918+0000 7f3f737fe640 1 --2- 172.21.15.5:0/2641704928 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5601660bb6b0 0x5601664e3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.918+0000 7f3f737fe640 1 -- 172.21.15.5:0/2641704928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56016609af60 msgr2=0x56016653d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:10.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.918+0000 7f3f737fe640 1 --2- 172.21.15.5:0/2641704928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56016609af60 0x56016653d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.918+0000 7f3f737fe640 1 -- 172.21.15.5:0/2641704928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601664aa9a0 con 0x5601660b34d0 2024-05-20T18:31:10.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.918+0000 7f3f737fe640 1 --2- 172.21.15.5:0/2641704928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601660b34d0 0x56016647c230 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f3f680091d0 tx=0x7f3f6807c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec3d783fbe73624 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:10.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.918+0000 7f3f52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:31:10.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.918+0000 7f3f52ffd640 1 -- 172.21.15.5:0/2641704928 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f680a7070 con 0x5601660b34d0 2024-05-20T18:31:10.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.918+0000 7f3f52ffd640 1 -- 172.21.15.5:0/2641704928 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 6246+0+0 (secure 0 0 0) 0x7f3f68078f80 con 0x5601660b34d0 2024-05-20T18:31:10.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.918+0000 7f3f8139f740 1 -- 172.21.15.5:0/2641704928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f6c012810 con 0x5601660b34d0 2024-05-20T18:31:10.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.918+0000 7f3f52ffd640 1 -- 172.21.15.5:0/2641704928 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3f68079460 con 0x5601660b34d0 2024-05-20T18:31:10.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.918+0000 7f3f8139f740 10 client.6528 fetch_fsmap learned FSMap version 36 2024-05-20T18:31:10.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.918+0000 7f3f8139f740 1 -- 172.21.15.5:0/2641704928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x5601664aa9a0 con 0x5601660b34d0 2024-05-20T18:31:10.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.919+0000 7f3f52ffd640 1 -- 172.21.15.5:0/2641704928 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f6c012810 con 0x5601660b34d0 2024-05-20T18:31:10.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.919+0000 7f3f52ffd640 1 -- 172.21.15.5:0/2641704928 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f3f6809f020 con 0x5601660b34d0 2024-05-20T18:31:10.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.919+0000 7f3f8139f740 10 client.6528 fetch_fsmap finished waiting for FSMap version 36 2024-05-20T18:31:10.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.919+0000 7f3f8139f740 10 client.6528 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-05-20T18:31:10.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.919+0000 7f3f8139f740 20 client.6528 populate_metadata read hostname 'smithi005' 2024-05-20T18:31:10.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.919+0000 7f3f8139f740 1 --2- 172.21.15.5:0/2641704928 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x5601660703a0 0x5601664e4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.919+0000 7f3f8139f740 4 client.6528 mds_command: new command op to 6198 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T18:31:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.919+0000 7f3f8139f740 1 -- 172.21.15.5:0/2641704928 --> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5601663e8780 con 0x5601660703a0 2024-05-20T18:31:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.919+0000 7f3f73fff640 1 --2- 172.21.15.5:0/2641704928 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x5601660703a0 0x5601664e4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.919+0000 7f3f52ffd640 10 client.6528 ms_handle_connect on v2:172.21.15.63:6832/4036379300 2024-05-20T18:31:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.919+0000 7f3f73fff640 1 --2- 172.21.15.5:0/2641704928 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x5601660703a0 0x5601664e4450 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23e00953838914bc server_cookie=ba8a75a2c63bba0 in_seq=0 out_seq=0 2024-05-20T18:31:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.920+0000 7f3f52ffd640 1 -- 172.21.15.5:0/2641704928 <== mds.0 v2:172.21.15.63:6832/4036379300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5601663e8780 con 0x5601660703a0 2024-05-20T18:31:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.920+0000 7f3f52ffd640 10 client.6528 handle_command_reply: tid=0 2024-05-20T18:31:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.920+0000 7f3f8139f740 1 client.6528 shutdown 2024-05-20T18:31:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.920+0000 7f3f8139f740 1 -- 172.21.15.5:0/2641704928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601660b34d0 msgr2=0x56016647c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.920+0000 7f3f8139f740 1 --2- 172.21.15.5:0/2641704928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601660b34d0 0x56016647c230 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f3f680091d0 tx=0x7f3f6807c040 comp rx=0 tx=0).stop 2024-05-20T18:31:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.920+0000 7f3f8139f740 1 -- 172.21.15.5:0/2641704928 shutdown_connections 2024-05-20T18:31:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.920+0000 7f3f8139f740 1 --2- 172.21.15.5:0/2641704928 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x5601660703a0 0x5601664e4450 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.920+0000 7f3f8139f740 1 --2- 172.21.15.5:0/2641704928 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5601660bb6b0 0x5601664e3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.920+0000 7f3f8139f740 1 --2- 172.21.15.5:0/2641704928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601660b34d0 0x56016647c230 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.920+0000 7f3f8139f740 1 --2- 172.21.15.5:0/2641704928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56016609af60 0x56016653d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.920+0000 7f3f52ffd640 0 client.6528 ms_handle_reset on v2:172.21.15.63:6832/4036379300 2024-05-20T18:31:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.920+0000 7f3f8139f740 1 -- 172.21.15.5:0/2641704928 >> 172.21.15.5:0/2641704928 conn(0x5601660d04e0 msgr2=0x5601664a3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.920+0000 7f3f8139f740 1 -- 172.21.15.5:0/2641704928 shutdown_connections 2024-05-20T18:31:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.920+0000 7f3f8139f740 1 -- 172.21.15.5:0/2641704928 wait complete. 2024-05-20T18:31:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.920+0000 7f3f8139f740 20 client.6528 trim_cache size 0 max 16384 2024-05-20T18:31:10.926 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:31:10.926 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-20T18:31:10.927 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-20T18:31:10.927 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:31:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.920+0000 7f3f7a41b640 1 -- 172.21.15.5:0/3873284942 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3f6006be30 msgr2=0x7f3f6006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.920+0000 7f3f7a41b640 1 --2- 172.21.15.5:0/3873284942 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3f6006be30 0x7f3f6006e340 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3f740c64c0 tx=0x7f3f64008040 comp rx=0 tx=0).stop 2024-05-20T18:31:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.920+0000 7f3f7a41b640 1 -- 172.21.15.5:0/3873284942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f740d0e00 msgr2=0x7f3f740c7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.920+0000 7f3f7a41b640 1 --2- 172.21.15.5:0/3873284942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f740d0e00 0x7f3f740c7a20 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f3f68000c00 tx=0x7f3f680046c0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.921+0000 7f3f7a41b640 1 -- 172.21.15.5:0/3873284942 shutdown_connections 2024-05-20T18:31:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.921+0000 7f3f7a41b640 1 --2- 172.21.15.5:0/3873284942 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3f6006be30 0x7f3f6006e340 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.921+0000 7f3f7a41b640 1 --2- 172.21.15.5:0/3873284942 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f7415f220 0x7f3f7415f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.921+0000 7f3f7a41b640 1 --2- 172.21.15.5:0/3873284942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f740d0e00 0x7f3f740c7a20 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.921+0000 7f3f7a41b640 1 --2- 172.21.15.5:0/3873284942 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f740cf910 0x7f3f740c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.921+0000 7f3f7a41b640 1 -- 172.21.15.5:0/3873284942 >> 172.21.15.5:0/3873284942 conn(0x7f3f740b44d0 msgr2=0x7f3f740b3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.921+0000 7f3f7a41b640 1 -- 172.21.15.5:0/3873284942 shutdown_connections 2024-05-20T18:31:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:10.921+0000 7f3f7a41b640 1 -- 172.21.15.5:0/3873284942 wait complete. 2024-05-20T18:31:10.932 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T18:31:10.932 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:31:11.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.047+0000 7fdc9e1e3640 1 Processor -- start 2024-05-20T18:31:11.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.047+0000 7fdc9e1e3640 1 -- start start 2024-05-20T18:31:11.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.048+0000 7fdc9e1e3640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fdc980cd710 0x7fdc98162e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.048+0000 7fdc9e1e3640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc9806c8c0 con 0x7fdc980cfae0 2024-05-20T18:31:11.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.048+0000 7fdc9e1e3640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc9806dac0 con 0x7fdc9806d6b0 2024-05-20T18:31:11.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.048+0000 7fdc9e1e3640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fdc980d0ba0 con 0x7fdc980cd710 2024-05-20T18:31:11.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.049+0000 7fdc97fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fdc980cfae0 0x7fdc980d1030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37380/0 (socket says 172.21.15.5:37380) 2024-05-20T18:31:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.049+0000 7fdc97fff640 1 -- 172.21.15.5:0/2771551752 learned_addr learned my addr 172.21.15.5:0/2771551752 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.049+0000 7fdc977fe640 1 --2- 172.21.15.5:0/2771551752 >> v2:172.21.15.63:3300/0 conn(0x7fdc980cd710 0x7fdc98162e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.049+0000 7fdc96ffd640 1 -- 172.21.15.5:0/2771551752 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3258511294 0 0) 0x7fdc9806c8c0 con 0x7fdc980cfae0 2024-05-20T18:31:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.049+0000 7fdc96ffd640 1 -- 172.21.15.5:0/2771551752 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc84003520 con 0x7fdc980cfae0 2024-05-20T18:31:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.049+0000 7fdc96ffd640 1 -- 172.21.15.5:0/2771551752 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1259552270 0 0) 0x7fdc84003520 con 0x7fdc980cfae0 2024-05-20T18:31:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.049+0000 7fdc96ffd640 1 -- 172.21.15.5:0/2771551752 >> v2:172.21.15.63:3300/0 conn(0x7fdc980cd710 msgr2=0x7fdc98162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.049+0000 7fdc96ffd640 1 --2- 172.21.15.5:0/2771551752 >> v2:172.21.15.63:3300/0 conn(0x7fdc980cd710 0x7fdc98162e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.049+0000 7fdc96ffd640 1 -- 172.21.15.5:0/2771551752 >> v1:172.21.15.63:6789/0 conn(0x7fdc9806d6b0 legacy=0x7fdc980cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.049+0000 7fdc96ffd640 1 -- 172.21.15.5:0/2771551752 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc981633e0 con 0x7fdc980cfae0 2024-05-20T18:31:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.049+0000 7fdc96ffd640 1 -- 172.21.15.5:0/2771551752 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7fdc90001a10 con 0x7fdc980cfae0 2024-05-20T18:31:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.049+0000 7fdc96ffd640 1 -- 172.21.15.5:0/2771551752 >> v1:172.21.15.5:6789/0 conn(0x7fdc980cfae0 legacy=0x7fdc980d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.049+0000 7fdc96ffd640 1 --2- 172.21.15.5:0/2771551752 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdc840039b0 0x7fdc84003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.049+0000 7fdc96ffd640 1 --2- 172.21.15.5:0/2771551752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc84004800 0x7fdc84004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.049+0000 7fdc96ffd640 1 --2- 172.21.15.5:0/2771551752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc84005270 0x7fdc8400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.049+0000 7fdc96ffd640 1 -- 172.21.15.5:0/2771551752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdc980d0ba0 con 0x7fdc84004800 2024-05-20T18:31:11.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.049+0000 7fdc96ffd640 1 -- 172.21.15.5:0/2771551752 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8400c190 con 0x7fdc84005270 2024-05-20T18:31:11.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.049+0000 7fdc96ffd640 1 -- 172.21.15.5:0/2771551752 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdc8400c390 con 0x7fdc840039b0 2024-05-20T18:31:11.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.049+0000 7fdc8ffff640 1 --2- 172.21.15.5:0/2771551752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc84004800 0x7fdc84004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.050+0000 7fdc977fe640 1 --2- 172.21.15.5:0/2771551752 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdc840039b0 0x7fdc84003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.050+0000 7fdc97fff640 1 --2- 172.21.15.5:0/2771551752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc84005270 0x7fdc8400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.050+0000 7fdc8ffff640 1 -- 172.21.15.5:0/2771551752 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdc840039b0 msgr2=0x7fdc84003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:11.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.050+0000 7fdc8ffff640 1 --2- 172.21.15.5:0/2771551752 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdc840039b0 0x7fdc84003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.050+0000 7fdc8ffff640 1 -- 172.21.15.5:0/2771551752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc84005270 msgr2=0x7fdc8400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:11.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.050+0000 7fdc8ffff640 1 --2- 172.21.15.5:0/2771551752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc84005270 0x7fdc8400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.050+0000 7fdc8ffff640 1 -- 172.21.15.5:0/2771551752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc84003520 con 0x7fdc84004800 2024-05-20T18:31:11.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.050+0000 7fdc97fff640 1 -- 172.21.15.5:0/2771551752 reap_dead start 2024-05-20T18:31:11.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.051+0000 7fdc8ffff640 1 --2- 172.21.15.5:0/2771551752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc84004800 0x7fdc84004c10 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fdc80001750 tx=0x7fdc84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531b99a7259c3b1e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:11.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.051+0000 7fdc96ffd640 1 -- 172.21.15.5:0/2771551752 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc80001370 con 0x7fdc84004800 2024-05-20T18:31:11.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.051+0000 7fdc96ffd640 1 -- 172.21.15.5:0/2771551752 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc800052d0 con 0x7fdc84004800 2024-05-20T18:31:11.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.051+0000 7fdc96ffd640 1 -- 172.21.15.5:0/2771551752 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8002abc0 con 0x7fdc84004800 2024-05-20T18:31:11.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.051+0000 7fdc9e1e3640 1 -- 172.21.15.5:0/2771551752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc84004800 msgr2=0x7fdc84004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:11.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.051+0000 7fdc9e1e3640 1 --2- 172.21.15.5:0/2771551752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc84004800 0x7fdc84004c10 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fdc80001750 tx=0x7fdc84005830 comp rx=0 tx=0).stop 2024-05-20T18:31:11.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.051+0000 7fdc9e1e3640 1 -- 172.21.15.5:0/2771551752 shutdown_connections 2024-05-20T18:31:11.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.052+0000 7fdc9e1e3640 1 --2- 172.21.15.5:0/2771551752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc84004800 0x7fdc84004c10 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.052+0000 7fdc9e1e3640 1 -- 172.21.15.5:0/2771551752 >> 172.21.15.5:0/2771551752 conn(0x7fdc980b44d0 msgr2=0x7fdc980b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:11.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.052+0000 7fdc9e1e3640 1 -- 172.21.15.5:0/2771551752 shutdown_connections 2024-05-20T18:31:11.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.052+0000 7fdc9e1e3640 1 -- 172.21.15.5:0/2771551752 wait complete. 2024-05-20T18:31:11.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.053+0000 7fdc9e1e3640 1 Processor -- start 2024-05-20T18:31:11.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.053+0000 7fdc9e1e3640 1 -- start start 2024-05-20T18:31:11.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.055+0000 7fdc9e1e3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdc84004800 0x7fdc981f96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.055+0000 7fdc9e1e3640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc981595e0 0x7fdc98160680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.055+0000 7fdc9e1e3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc98160bc0 0x7fdc980cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.055+0000 7fdc9e1e3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdc98163c80 con 0x7fdc98160bc0 2024-05-20T18:31:11.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.055+0000 7fdc9e1e3640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdc981639a0 con 0x7fdc981595e0 2024-05-20T18:31:11.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.055+0000 7fdc9e1e3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdc98162e00 con 0x7fdc84004800 2024-05-20T18:31:11.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.055+0000 7fdc97fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc98160bc0 0x7fdc980cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.055+0000 7fdc97fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc98160bc0 0x7fdc980cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40102/0 (socket says 172.21.15.5:40102) 2024-05-20T18:31:11.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.055+0000 7fdc97fff640 1 -- 172.21.15.5:0/2134847006 learned_addr learned my addr 172.21.15.5:0/2134847006 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:11.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.055+0000 7fdc977fe640 1 --2- 172.21.15.5:0/2134847006 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdc84004800 0x7fdc981f96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.055+0000 7fdc8ffff640 1 --2- 172.21.15.5:0/2134847006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc981595e0 0x7fdc98160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.055+0000 7fdc97fff640 1 -- 172.21.15.5:0/2134847006 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdc84004800 msgr2=0x7fdc981f96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:11.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.055+0000 7fdc97fff640 1 --2- 172.21.15.5:0/2134847006 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdc84004800 0x7fdc981f96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.055+0000 7fdc97fff640 1 -- 172.21.15.5:0/2134847006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc981595e0 msgr2=0x7fdc98160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:11.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.055+0000 7fdc97fff640 1 --2- 172.21.15.5:0/2134847006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc981595e0 0x7fdc98160680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.055+0000 7fdc97fff640 1 -- 172.21.15.5:0/2134847006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc9815f5e0 con 0x7fdc98160bc0 2024-05-20T18:31:11.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.055+0000 7fdc97fff640 1 --2- 172.21.15.5:0/2134847006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc98160bc0 0x7fdc980cb5e0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fdc980b2370 tx=0x7fdc90007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:11.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.055+0000 7fdc94ff9640 1 -- 172.21.15.5:0/2134847006 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9000ab60 con 0x7fdc98160bc0 2024-05-20T18:31:11.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.056+0000 7fdc94ff9640 1 -- 172.21.15.5:0/2134847006 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc90018070 con 0x7fdc98160bc0 2024-05-20T18:31:11.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.056+0000 7fdc9e1e3640 1 -- 172.21.15.5:0/2134847006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc9815f8c0 con 0x7fdc98160bc0 2024-05-20T18:31:11.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.056+0000 7fdc94ff9640 1 -- 172.21.15.5:0/2134847006 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc900133f0 con 0x7fdc98160bc0 2024-05-20T18:31:11.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.056+0000 7fdc9e1e3640 1 -- 172.21.15.5:0/2134847006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc9815fe80 con 0x7fdc98160bc0 2024-05-20T18:31:11.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.056+0000 7fdc9e1e3640 1 -- 172.21.15.5:0/2134847006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc98075b00 con 0x7fdc98160bc0 2024-05-20T18:31:11.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.057+0000 7fdc94ff9640 1 -- 172.21.15.5:0/2134847006 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdc90013590 con 0x7fdc98160bc0 2024-05-20T18:31:11.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.058+0000 7fdc94ff9640 1 --2- 172.21.15.5:0/2134847006 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdc7806be30 0x7fdc7806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.058+0000 7fdc94ff9640 1 -- 172.21.15.5:0/2134847006 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6246+0+0 (secure 0 0 0) 0x7fdc9008b540 con 0x7fdc98160bc0 2024-05-20T18:31:11.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.058+0000 7fdc977fe640 1 --2- 172.21.15.5:0/2134847006 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdc7806be30 0x7fdc7806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.058+0000 7fdc977fe640 1 --2- 172.21.15.5:0/2134847006 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdc7806be30 0x7fdc7806e340 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdc88006b10 tx=0x7fdc88009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:11.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.063+0000 7fdc94ff9640 1 -- 172.21.15.5:0/2134847006 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fdc900595f0 con 0x7fdc98160bc0 2024-05-20T18:31:11.204 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:31:11.206 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:31:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.283+0000 7fdc9e1e3640 1 -- 172.21.15.5:0/2134847006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc980b8940 con 0x7fdc98160bc0 2024-05-20T18:31:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.283+0000 7fdc94ff9640 1 -- 172.21.15.5:0/2134847006 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5177 (secure 0 0 0) 0x7fdc9005d8f0 con 0x7fdc98160bc0 2024-05-20T18:31:11.284 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:31:11.284 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5480,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/4292435960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":4292435960},{"type":"v1","addr":"172.21.15.63:6835","nonce":4292435960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6186,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2991834087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2991834087},{"type":"v1","addr":"172.21.15.5:6836","nonce":2991834087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6195,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3462206198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":3462206198},{"type":"v1","addr":"172.21.15.5:6837","nonce":3462206198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:30:00.286575+0000","modified":"2024-05-20T18:30:48.943731+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6198":{"gid":6198,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":15,"addr":"172.21.15.63:6834/4036379300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":4036379300},{"type":"v1","addr":"172.21.15.63:6834","nonce":4036379300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6198,"qdb_cluster":[6198]},"id":4}]} 2024-05-20T18:31:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 36 2024-05-20T18:31:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.286+0000 7fdc9e1e3640 1 -- 172.21.15.5:0/2134847006 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdc7806be30 msgr2=0x7fdc7806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.286+0000 7fdc9e1e3640 1 --2- 172.21.15.5:0/2134847006 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdc7806be30 0x7fdc7806e340 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdc88006b10 tx=0x7fdc88009190 comp rx=0 tx=0).stop 2024-05-20T18:31:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.286+0000 7fdc9e1e3640 1 -- 172.21.15.5:0/2134847006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc98160bc0 msgr2=0x7fdc980cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.286+0000 7fdc9e1e3640 1 --2- 172.21.15.5:0/2134847006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc98160bc0 0x7fdc980cb5e0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fdc980b2370 tx=0x7fdc90007b00 comp rx=0 tx=0).stop 2024-05-20T18:31:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.286+0000 7fdc9e1e3640 1 -- 172.21.15.5:0/2134847006 shutdown_connections 2024-05-20T18:31:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.286+0000 7fdc9e1e3640 1 --2- 172.21.15.5:0/2134847006 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdc7806be30 0x7fdc7806e340 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.286+0000 7fdc9e1e3640 1 --2- 172.21.15.5:0/2134847006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc98160bc0 0x7fdc980cb5e0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.286+0000 7fdc9e1e3640 1 --2- 172.21.15.5:0/2134847006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc981595e0 0x7fdc98160680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.286+0000 7fdc9e1e3640 1 --2- 172.21.15.5:0/2134847006 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdc84004800 0x7fdc981f96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.286+0000 7fdc9e1e3640 1 -- 172.21.15.5:0/2134847006 >> 172.21.15.5:0/2134847006 conn(0x7fdc980b44d0 msgr2=0x7fdc980b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:11.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.286+0000 7fdc9e1e3640 1 -- 172.21.15.5:0/2134847006 shutdown_connections 2024-05-20T18:31:11.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.286+0000 7fdc9e1e3640 1 -- 172.21.15.5:0/2134847006 wait complete. 2024-05-20T18:31:11.298 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b damage ls --format=json-pretty 2024-05-20T18:31:11.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.456+0000 7fe1d363d640 1 Processor -- start 2024-05-20T18:31:11.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.457+0000 7fe1d363d640 1 -- start start 2024-05-20T18:31:11.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.458+0000 7fe1d363d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe1cc162820 0x7fe1cc0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.458+0000 7fe1d363d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1cc06dae0 0x7fe1cc0b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.458+0000 7fe1d363d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1cc0d0630 con 0x7fe1cc0b3e80 2024-05-20T18:31:11.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.458+0000 7fe1d363d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe1cc0d39c0 con 0x7fe1cc162820 2024-05-20T18:31:11.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.458+0000 7fe1d363d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe1cc0d3ca0 con 0x7fe1cc06dae0 2024-05-20T18:31:11.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.458+0000 7fe1d13d9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe1cc162820 0x7fe1cc0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.458+0000 7fe1d0bd8640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe1cc0b3e80 0x7fe1cc06d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37388/0 (socket says 172.21.15.5:37388) 2024-05-20T18:31:11.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.458+0000 7fe1d0bd8640 1 -- 172.21.15.5:0/2860739646 learned_addr learned my addr 172.21.15.5:0/2860739646 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:11.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.458+0000 7fe1d1bda640 1 --2- 172.21.15.5:0/2860739646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1cc06dae0 0x7fe1cc0b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.458+0000 7fe1c3fff640 1 -- 172.21.15.5:0/2860739646 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1979223132 0 0) 0x7fe1cc0d0630 con 0x7fe1cc0b3e80 2024-05-20T18:31:11.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.459+0000 7fe1c3fff640 1 -- 172.21.15.5:0/2860739646 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1b8003520 con 0x7fe1cc0b3e80 2024-05-20T18:31:11.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.459+0000 7fe1d13d9640 1 -- 172.21.15.5:0/2860739646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1cc06dae0 msgr2=0x7fe1cc0b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:11.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.459+0000 7fe1d13d9640 1 --2- 172.21.15.5:0/2860739646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1cc06dae0 0x7fe1cc0b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.459+0000 7fe1d13d9640 1 -- 172.21.15.5:0/2860739646 >> v1:172.21.15.5:6789/0 conn(0x7fe1cc0b3e80 legacy=0x7fe1cc06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.459+0000 7fe1d13d9640 1 -- 172.21.15.5:0/2860739646 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1cc0d3f80 con 0x7fe1cc162820 2024-05-20T18:31:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.459+0000 7fe1d13d9640 1 --2- 172.21.15.5:0/2860739646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1cc162820 0x7fe1cc0b3940 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe1c80097a0 tx=0x7fe1c802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c49f850a773a97f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.459+0000 7fe1c3fff640 1 -- 172.21.15.5:0/2860739646 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8037070 con 0x7fe1cc162820 2024-05-20T18:31:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.460+0000 7fe1c3fff640 1 -- 172.21.15.5:0/2860739646 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1c8032440 con 0x7fe1cc162820 2024-05-20T18:31:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.460+0000 7fe1c3fff640 1 -- 172.21.15.5:0/2860739646 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8032810 con 0x7fe1cc162820 2024-05-20T18:31:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.460+0000 7fe1d363d640 1 -- 172.21.15.5:0/2860739646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1cc162820 msgr2=0x7fe1cc0b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.460+0000 7fe1d363d640 1 --2- 172.21.15.5:0/2860739646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1cc162820 0x7fe1cc0b3940 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe1c80097a0 tx=0x7fe1c802a180 comp rx=0 tx=0).stop 2024-05-20T18:31:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.460+0000 7fe1d363d640 1 -- 172.21.15.5:0/2860739646 shutdown_connections 2024-05-20T18:31:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.460+0000 7fe1d363d640 1 --2- 172.21.15.5:0/2860739646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1cc06dae0 0x7fe1cc0b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.460+0000 7fe1d363d640 1 --2- 172.21.15.5:0/2860739646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1cc162820 0x7fe1cc0b3940 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.460+0000 7fe1d363d640 1 -- 172.21.15.5:0/2860739646 >> 172.21.15.5:0/2860739646 conn(0x7fe1cc0d4bd0 msgr2=0x7fe1cc0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.460+0000 7fe1d363d640 1 -- 172.21.15.5:0/2860739646 shutdown_connections 2024-05-20T18:31:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.460+0000 7fe1d363d640 1 -- 172.21.15.5:0/2860739646 wait complete. 2024-05-20T18:31:11.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.462+0000 7fe1d363d640 1 Processor -- start 2024-05-20T18:31:11.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.462+0000 7fe1d363d640 1 -- start start 2024-05-20T18:31:11.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.464+0000 7fe1d363d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1cc0b3e80 0x7fe1cc1f8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.464+0000 7fe1d363d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1cc162820 0x7fe1cc0b6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.464+0000 7fe1d363d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1cc0b9440 0x7fe1cc0b9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.464+0000 7fe1d363d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc0d4780 con 0x7fe1cc0b9440 2024-05-20T18:31:11.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.464+0000 7fe1d363d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc0d0300 con 0x7fe1cc162820 2024-05-20T18:31:11.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.464+0000 7fe1d363d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe1cc0d4900 con 0x7fe1cc0b3e80 2024-05-20T18:31:11.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.464+0000 7fe1d1bda640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1cc0b9440 0x7fe1cc0b9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.464+0000 7fe1d1bda640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1cc0b9440 0x7fe1cc0b9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40114/0 (socket says 172.21.15.5:40114) 2024-05-20T18:31:11.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.464+0000 7fe1d1bda640 1 -- 172.21.15.5:0/4052500767 learned_addr learned my addr 172.21.15.5:0/4052500767 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:11.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.464+0000 7fe1d0bd8640 1 --2- 172.21.15.5:0/4052500767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1cc162820 0x7fe1cc0b6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.464+0000 7fe1d13d9640 1 --2- 172.21.15.5:0/4052500767 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1cc0b3e80 0x7fe1cc1f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.464+0000 7fe1d1bda640 1 -- 172.21.15.5:0/4052500767 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1cc0b3e80 msgr2=0x7fe1cc1f8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:11.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.464+0000 7fe1d1bda640 1 --2- 172.21.15.5:0/4052500767 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1cc0b3e80 0x7fe1cc1f8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.464+0000 7fe1d1bda640 1 -- 172.21.15.5:0/4052500767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1cc162820 msgr2=0x7fe1cc0b6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:11.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.464+0000 7fe1d1bda640 1 --2- 172.21.15.5:0/4052500767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1cc162820 0x7fe1cc0b6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.464+0000 7fe1d1bda640 1 -- 172.21.15.5:0/4052500767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1cc0b9d90 con 0x7fe1cc0b9440 2024-05-20T18:31:11.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.464+0000 7fe1d13d9640 1 --2- 172.21.15.5:0/4052500767 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1cc0b3e80 0x7fe1cc1f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:11.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.464+0000 7fe1d1bda640 1 --2- 172.21.15.5:0/4052500767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1cc0b9440 0x7fe1cc0b9850 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fe1c400ef90 tx=0x7fe1c400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:11.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.464+0000 7fe1c1ffb640 1 -- 172.21.15.5:0/4052500767 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4018020 con 0x7fe1cc0b9440 2024-05-20T18:31:11.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.464+0000 7fe1c1ffb640 1 -- 172.21.15.5:0/4052500767 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1c4014480 con 0x7fe1cc0b9440 2024-05-20T18:31:11.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.464+0000 7fe1d363d640 1 -- 172.21.15.5:0/4052500767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1cc0c96d0 con 0x7fe1cc0b9440 2024-05-20T18:31:11.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.464+0000 7fe1c1ffb640 1 -- 172.21.15.5:0/4052500767 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4009e60 con 0x7fe1cc0b9440 2024-05-20T18:31:11.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.465+0000 7fe1d363d640 1 -- 172.21.15.5:0/4052500767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1cc0c9c50 con 0x7fe1cc0b9440 2024-05-20T18:31:11.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.466+0000 7fe1c1ffb640 1 -- 172.21.15.5:0/4052500767 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe1c4005d10 con 0x7fe1cc0b9440 2024-05-20T18:31:11.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.467+0000 7fe1c1ffb640 1 --2- 172.21.15.5:0/4052500767 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe1b006be30 0x7fe1b006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.467+0000 7fe1c1ffb640 1 -- 172.21.15.5:0/4052500767 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6246+0+0 (secure 0 0 0) 0x7fe1c408be70 con 0x7fe1cc0b9440 2024-05-20T18:31:11.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.467+0000 7fe1d13d9640 1 --2- 172.21.15.5:0/4052500767 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe1b006be30 0x7fe1b006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.467+0000 7fe1d13d9640 1 --2- 172.21.15.5:0/4052500767 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe1b006be30 0x7fe1b006e340 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8034000 tx=0x7fe1c802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:11.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.468+0000 7fe1da5c4740 1 Processor -- start 2024-05-20T18:31:11.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.468+0000 7fe1da5c4740 1 -- start start 2024-05-20T18:31:11.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.469+0000 7fe1da5c4740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d976546350 0x55d976911670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.469+0000 7fe1da5c4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d97651d490 0x55d97692c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.469+0000 7fe1da5c4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d976525670 0x55d97692c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.469+0000 7fe1da5c4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d97684aa30 con 0x55d976525670 2024-05-20T18:31:11.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.469+0000 7fe1da5c4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d9767e8590 con 0x55d97651d490 2024-05-20T18:31:11.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.469+0000 7fe1da5c4740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55d9768469f0 con 0x55d976546350 2024-05-20T18:31:11.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.469+0000 7fe1d1bda640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d976525670 0x55d97692c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.469+0000 7fe1d1bda640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d976525670 0x55d97692c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40128/0 (socket says 172.21.15.5:40128) 2024-05-20T18:31:11.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.470+0000 7fe1d1bda640 1 -- 172.21.15.5:0/2640898302 learned_addr learned my addr 172.21.15.5:0/2640898302 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:11.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.470+0000 7fe1d0bd8640 1 --2- 172.21.15.5:0/2640898302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d97651d490 0x55d97692c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.470+0000 7fe1d13d9640 1 --2- 172.21.15.5:0/2640898302 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d976546350 0x55d976911670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.470+0000 7fe1d1bda640 1 -- 172.21.15.5:0/2640898302 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d976546350 msgr2=0x55d976911670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:11.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.470+0000 7fe1d1bda640 1 --2- 172.21.15.5:0/2640898302 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d976546350 0x55d976911670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.470+0000 7fe1d1bda640 1 -- 172.21.15.5:0/2640898302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d97651d490 msgr2=0x55d97692c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:11.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.470+0000 7fe1d1bda640 1 --2- 172.21.15.5:0/2640898302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d97651d490 0x55d97692c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.470+0000 7fe1d1bda640 1 -- 172.21.15.5:0/2640898302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d97662a7c0 con 0x55d976525670 2024-05-20T18:31:11.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.470+0000 7fe1d1bda640 1 --2- 172.21.15.5:0/2640898302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d976525670 0x55d97692c8e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4009c50 tx=0x7fe1c4059dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15a0b5997d97b9a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:11.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.470+0000 7fe1b6ffd640 1 -- 172.21.15.5:0/2640898302 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c405d040 con 0x55d976525670 2024-05-20T18:31:11.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.470+0000 7fe1b6ffd640 1 -- 172.21.15.5:0/2640898302 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1c4062a40 con 0x55d976525670 2024-05-20T18:31:11.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.470+0000 7fe1b6ffd640 1 -- 172.21.15.5:0/2640898302 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c40580d0 con 0x55d976525670 2024-05-20T18:31:11.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.471+0000 7fe1da5c4740 1 -- 172.21.15.5:0/2640898302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d976525670 msgr2=0x55d97692c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:11.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.471+0000 7fe1da5c4740 1 --2- 172.21.15.5:0/2640898302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d976525670 0x55d97692c8e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4009c50 tx=0x7fe1c4059dd0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.471+0000 7fe1da5c4740 1 -- 172.21.15.5:0/2640898302 shutdown_connections 2024-05-20T18:31:11.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.471+0000 7fe1da5c4740 1 --2- 172.21.15.5:0/2640898302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d976525670 0x55d97692c8e0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.471+0000 7fe1da5c4740 1 --2- 172.21.15.5:0/2640898302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d97651d490 0x55d97692c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.471+0000 7fe1da5c4740 1 --2- 172.21.15.5:0/2640898302 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d976546350 0x55d976911670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.471+0000 7fe1da5c4740 1 -- 172.21.15.5:0/2640898302 >> 172.21.15.5:0/2640898302 conn(0x55d97653a4a0 msgr2=0x55d9768e6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:11.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.471+0000 7fe1da5c4740 1 -- 172.21.15.5:0/2640898302 shutdown_connections 2024-05-20T18:31:11.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.471+0000 7fe1da5c4740 1 -- 172.21.15.5:0/2640898302 wait complete. 2024-05-20T18:31:11.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.472+0000 7fe1da5c4740 1 -- start start 2024-05-20T18:31:11.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.472+0000 7fe1da5c4740 1 Processor -- start 2024-05-20T18:31:11.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.473+0000 7fe1da5c4740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d97651d490 0x55d976925fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.473+0000 7fe1da5c4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d976525670 0x55d976913820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.473+0000 7fe1da5c4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d976546350 0x55d97693b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.473+0000 7fe1da5c4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d9767e6060 con 0x55d976525670 2024-05-20T18:31:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.473+0000 7fe1da5c4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d97693c2d0 con 0x55d976546350 2024-05-20T18:31:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.473+0000 7fe1da5c4740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55d976913d60 con 0x55d97651d490 2024-05-20T18:31:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.473+0000 7fe1da5c4740 10 client.0 _ll_register_callbacks cb 0x55d97680add0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:31:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.474+0000 7fe1d0bd8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d976525670 0x55d976913820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.474+0000 7fe1d0bd8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d976525670 0x55d976913820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40138/0 (socket says 172.21.15.5:40138) 2024-05-20T18:31:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.474+0000 7fe1d0bd8640 1 -- 172.21.15.5:0/255019646 learned_addr learned my addr 172.21.15.5:0/255019646 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.474+0000 7fe1d13d9640 1 --2- 172.21.15.5:0/255019646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d97651d490 0x55d976925fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.474+0000 7fe1d1bda640 1 --2- 172.21.15.5:0/255019646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d976546350 0x55d97693b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.474+0000 7fe1d0bd8640 1 -- 172.21.15.5:0/255019646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d97651d490 msgr2=0x55d976925fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.474+0000 7fe1d0bd8640 1 --2- 172.21.15.5:0/255019646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d97651d490 0x55d976925fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.474+0000 7fe1d0bd8640 1 -- 172.21.15.5:0/255019646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d976546350 msgr2=0x55d97693b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.474+0000 7fe1d0bd8640 1 --2- 172.21.15.5:0/255019646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d976546350 0x55d97693b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.474+0000 7fe1d0bd8640 1 -- 172.21.15.5:0/255019646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d97668af50 con 0x55d976525670 2024-05-20T18:31:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.474+0000 7fe1d1bda640 1 --2- 172.21.15.5:0/255019646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d976546350 0x55d97693b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.474+0000 7fe1d13d9640 1 --2- 172.21.15.5:0/255019646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d97651d490 0x55d976925fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.474+0000 7fe1d0bd8640 1 --2- 172.21.15.5:0/255019646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d976525670 0x55d976913820 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc000f60 tx=0x7fe1bc001580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc444adbde100c7e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.474+0000 7fe1abfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:31:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.474+0000 7fe1abfff640 1 -- 172.21.15.5:0/255019646 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc015470 con 0x55d976525670 2024-05-20T18:31:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.474+0000 7fe1abfff640 1 -- 172.21.15.5:0/255019646 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 6246+0+0 (secure 0 0 0) 0x7fe1bc015610 con 0x55d976525670 2024-05-20T18:31:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.474+0000 7fe1da5c4740 1 -- 172.21.15.5:0/255019646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1c4062a40 con 0x55d976525670 2024-05-20T18:31:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.474+0000 7fe1abfff640 1 -- 172.21.15.5:0/255019646 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe1bc022050 con 0x55d976525670 2024-05-20T18:31:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.474+0000 7fe1da5c4740 10 client.6549 fetch_fsmap learned FSMap version 36 2024-05-20T18:31:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.474+0000 7fe1da5c4740 1 -- 172.21.15.5:0/255019646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55d97651e0b0 con 0x55d976525670 2024-05-20T18:31:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.475+0000 7fe1abfff640 1 -- 172.21.15.5:0/255019646 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1c4062a40 con 0x55d976525670 2024-05-20T18:31:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.475+0000 7fe1abfff640 1 -- 172.21.15.5:0/255019646 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7fe1bc025020 con 0x55d976525670 2024-05-20T18:31:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.475+0000 7fe1da5c4740 10 client.6549 fetch_fsmap finished waiting for FSMap version 36 2024-05-20T18:31:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.475+0000 7fe1da5c4740 10 client.6549 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-20T18:31:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.475+0000 7fe1da5c4740 20 client.6549 populate_metadata read hostname 'smithi005' 2024-05-20T18:31:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.475+0000 7fe1da5c4740 1 --2- 172.21.15.5:0/255019646 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55d976505130 0x55d9769495a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.475+0000 7fe1da5c4740 4 client.6549 mds_command: new command op to 6198 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:31:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.475+0000 7fe1da5c4740 1 -- 172.21.15.5:0/255019646 --> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d976858160 con 0x55d976505130 2024-05-20T18:31:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.475+0000 7fe1d13d9640 1 --2- 172.21.15.5:0/255019646 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55d976505130 0x55d9769495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.476+0000 7fe1abfff640 10 client.6549 ms_handle_connect on v2:172.21.15.63:6832/4036379300 2024-05-20T18:31:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.476+0000 7fe1d13d9640 1 --2- 172.21.15.5:0/255019646 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55d976505130 0x55d9769495a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b143c6dcec5d55e7 server_cookie=d3cf4d9d4ea771c8 in_seq=0 out_seq=0 2024-05-20T18:31:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.476+0000 7fe1abfff640 1 -- 172.21.15.5:0/255019646 <== mds.0 v2:172.21.15.63:6832/4036379300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d976858160 con 0x55d976505130 2024-05-20T18:31:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.476+0000 7fe1abfff640 10 client.6549 handle_command_reply: tid=0 2024-05-20T18:31:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.476+0000 7fe1da5c4740 1 client.6549 shutdown 2024-05-20T18:31:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.476+0000 7fe1da5c4740 1 -- 172.21.15.5:0/255019646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d976525670 msgr2=0x55d976913820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.476+0000 7fe1da5c4740 1 --2- 172.21.15.5:0/255019646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d976525670 0x55d976913820 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc000f60 tx=0x7fe1bc001580 comp rx=0 tx=0).stop 2024-05-20T18:31:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.477+0000 7fe1da5c4740 1 -- 172.21.15.5:0/255019646 shutdown_connections 2024-05-20T18:31:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.477+0000 7fe1da5c4740 1 --2- 172.21.15.5:0/255019646 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55d976505130 0x55d9769495a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.477+0000 7fe1da5c4740 1 --2- 172.21.15.5:0/255019646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d976546350 0x55d97693b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.477+0000 7fe1da5c4740 1 --2- 172.21.15.5:0/255019646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d976525670 0x55d976913820 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.477+0000 7fe1abfff640 0 client.6549 ms_handle_reset on v2:172.21.15.63:6832/4036379300 2024-05-20T18:31:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.477+0000 7fe1da5c4740 1 --2- 172.21.15.5:0/255019646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d97651d490 0x55d976925fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.477+0000 7fe1da5c4740 1 -- 172.21.15.5:0/255019646 >> 172.21.15.5:0/255019646 conn(0x55d97653a4a0 msgr2=0x55d9768e0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.477+0000 7fe1da5c4740 1 -- 172.21.15.5:0/255019646 shutdown_connections 2024-05-20T18:31:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.477+0000 7fe1da5c4740 1 -- 172.21.15.5:0/255019646 wait complete. 2024-05-20T18:31:11.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.477+0000 7fe1da5c4740 20 client.6549 trim_cache size 0 max 16384 2024-05-20T18:31:11.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.491+0000 7fe1da5c4740 1 Processor -- start 2024-05-20T18:31:11.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.491+0000 7fe1da5c4740 1 -- start start 2024-05-20T18:31:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.492+0000 7fe1da5c4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d976505130 0x55d9769a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.492+0000 7fe1da5c4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d97651d490 0x55d97690c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.492+0000 7fe1da5c4740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d976525670 0x55d97690cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.492+0000 7fe1da5c4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc2013a0 con 0x55d97651d490 2024-05-20T18:31:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.492+0000 7fe1da5c4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc15d220 con 0x55d976505130 2024-05-20T18:31:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.492+0000 7fe1da5c4740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe1cc200cd0 con 0x55d976525670 2024-05-20T18:31:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.492+0000 7fe1d0bd8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d97651d490 0x55d97690c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.492+0000 7fe1d0bd8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d97651d490 0x55d97690c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40140/0 (socket says 172.21.15.5:40140) 2024-05-20T18:31:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.492+0000 7fe1d0bd8640 1 -- 172.21.15.5:0/436838565 learned_addr learned my addr 172.21.15.5:0/436838565 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.492+0000 7fe1d1bda640 1 --2- 172.21.15.5:0/436838565 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d976525670 0x55d97690cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.492+0000 7fe1d13d9640 1 --2- 172.21.15.5:0/436838565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d976505130 0x55d9769a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.492+0000 7fe1d0bd8640 1 -- 172.21.15.5:0/436838565 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d976525670 msgr2=0x55d97690cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.492+0000 7fe1d0bd8640 1 --2- 172.21.15.5:0/436838565 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d976525670 0x55d97690cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.492+0000 7fe1d0bd8640 1 -- 172.21.15.5:0/436838565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d976505130 msgr2=0x55d9769a2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.492+0000 7fe1d0bd8640 1 --2- 172.21.15.5:0/436838565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d976505130 0x55d9769a2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.492+0000 7fe1d0bd8640 1 -- 172.21.15.5:0/436838565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d976931190 con 0x55d97651d490 2024-05-20T18:31:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.493+0000 7fe1d0bd8640 1 --2- 172.21.15.5:0/436838565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d97651d490 0x55d97690c7d0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc007b80 tx=0x7fe1bc00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa3f9e88e4bee3e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.493+0000 7fe1b67fc640 1 -- 172.21.15.5:0/436838565 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc015470 con 0x55d97651d490 2024-05-20T18:31:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.493+0000 7fe1b67fc640 1 -- 172.21.15.5:0/436838565 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1bc02a020 con 0x55d97651d490 2024-05-20T18:31:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.493+0000 7fe1b67fc640 1 -- 172.21.15.5:0/436838565 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc015b80 con 0x55d97651d490 2024-05-20T18:31:11.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.493+0000 7fe1da5c4740 1 -- 172.21.15.5:0/436838565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d97651d490 msgr2=0x55d97690c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:11.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.493+0000 7fe1da5c4740 1 --2- 172.21.15.5:0/436838565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d97651d490 0x55d97690c7d0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc007b80 tx=0x7fe1bc00f000 comp rx=0 tx=0).stop 2024-05-20T18:31:11.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.493+0000 7fe1da5c4740 1 -- 172.21.15.5:0/436838565 shutdown_connections 2024-05-20T18:31:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.493+0000 7fe1da5c4740 1 --2- 172.21.15.5:0/436838565 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d976525670 0x55d97690cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.493+0000 7fe1da5c4740 1 --2- 172.21.15.5:0/436838565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d97651d490 0x55d97690c7d0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.493+0000 7fe1da5c4740 1 --2- 172.21.15.5:0/436838565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d976505130 0x55d9769a2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.493+0000 7fe1da5c4740 1 -- 172.21.15.5:0/436838565 >> 172.21.15.5:0/436838565 conn(0x55d97653a4a0 msgr2=0x55d97693b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.493+0000 7fe1da5c4740 1 -- 172.21.15.5:0/436838565 shutdown_connections 2024-05-20T18:31:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.494+0000 7fe1da5c4740 1 -- 172.21.15.5:0/436838565 wait complete. 2024-05-20T18:31:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.494+0000 7fe1da5c4740 1 -- start start 2024-05-20T18:31:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.494+0000 7fe1da5c4740 1 Processor -- start 2024-05-20T18:31:11.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.496+0000 7fe1da5c4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d976505130 0x55d97690c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.496+0000 7fe1da5c4740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d97651d490 0x55d97690cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.496+0000 7fe1da5c4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d976525670 0x55d976944970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.496+0000 7fe1da5c4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d976912760 con 0x55d976525670 2024-05-20T18:31:11.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.496+0000 7fe1da5c4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d97693c010 con 0x55d976505130 2024-05-20T18:31:11.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.496+0000 7fe1da5c4740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55d976912d80 con 0x55d97651d490 2024-05-20T18:31:11.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.496+0000 7fe1da5c4740 10 client.0 _ll_register_callbacks cb 0x7fe1bc005990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:31:11.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.496+0000 7fe1d1bda640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d976525670 0x55d976944970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.496+0000 7fe1d1bda640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d976525670 0x55d976944970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40152/0 (socket says 172.21.15.5:40152) 2024-05-20T18:31:11.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.496+0000 7fe1d1bda640 1 -- 172.21.15.5:0/2734912537 learned_addr learned my addr 172.21.15.5:0/2734912537 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:11.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.496+0000 7fe1d13d9640 1 --2- 172.21.15.5:0/2734912537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d976505130 0x55d97690c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.496+0000 7fe1d1bda640 1 -- 172.21.15.5:0/2734912537 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d97651d490 msgr2=0x55d97690cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:11.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.496+0000 7fe1d1bda640 1 --2- 172.21.15.5:0/2734912537 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d97651d490 0x55d97690cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.496+0000 7fe1d1bda640 1 -- 172.21.15.5:0/2734912537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d976505130 msgr2=0x55d97690c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:11.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.496+0000 7fe1d1bda640 1 --2- 172.21.15.5:0/2734912537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d976505130 0x55d97690c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.496+0000 7fe1d1bda640 1 -- 172.21.15.5:0/2734912537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d976922b50 con 0x55d976525670 2024-05-20T18:31:11.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.496+0000 7fe1d1bda640 1 --2- 172.21.15.5:0/2734912537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d976525670 0x55d976944970 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe1c405e340 tx=0x7fe1c4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841543598f9e9766 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:11.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.496+0000 7fe1abfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:31:11.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.497+0000 7fe1abfff640 1 -- 172.21.15.5:0/2734912537 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c405d040 con 0x55d976525670 2024-05-20T18:31:11.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.497+0000 7fe1abfff640 1 -- 172.21.15.5:0/2734912537 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 6246+0+0 (secure 0 0 0) 0x7fe1c407ecc0 con 0x55d976525670 2024-05-20T18:31:11.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.497+0000 7fe1da5c4740 1 -- 172.21.15.5:0/2734912537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1bc02a020 con 0x55d976525670 2024-05-20T18:31:11.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.497+0000 7fe1abfff640 1 -- 172.21.15.5:0/2734912537 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe1c4059220 con 0x55d976525670 2024-05-20T18:31:11.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.497+0000 7fe1da5c4740 10 client.6555 fetch_fsmap learned FSMap version 36 2024-05-20T18:31:11.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.497+0000 7fe1da5c4740 1 -- 172.21.15.5:0/2734912537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55d97692c860 con 0x55d976525670 2024-05-20T18:31:11.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.497+0000 7fe1abfff640 1 -- 172.21.15.5:0/2734912537 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1bc02a020 con 0x55d976525670 2024-05-20T18:31:11.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.497+0000 7fe1abfff640 1 -- 172.21.15.5:0/2734912537 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7fe1c407c030 con 0x55d976525670 2024-05-20T18:31:11.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.497+0000 7fe1da5c4740 10 client.6555 fetch_fsmap finished waiting for FSMap version 36 2024-05-20T18:31:11.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.497+0000 7fe1da5c4740 10 client.6555 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-20T18:31:11.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.497+0000 7fe1da5c4740 20 client.6555 populate_metadata read hostname 'smithi005' 2024-05-20T18:31:11.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.497+0000 7fe1da5c4740 1 --2- 172.21.15.5:0/2734912537 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55d9764da570 0x55d976945300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.497+0000 7fe1da5c4740 4 client.6555 mds_command: new command op to 6198 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-20T18:31:11.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.497+0000 7fe1da5c4740 1 -- 172.21.15.5:0/2734912537 --> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55d9768524b0 con 0x55d9764da570 2024-05-20T18:31:11.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.497+0000 7fe1d13d9640 1 --2- 172.21.15.5:0/2734912537 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55d9764da570 0x55d976945300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.498+0000 7fe1d13d9640 1 --2- 172.21.15.5:0/2734912537 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55d9764da570 0x55d976945300 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cdaf423e23b6ad8 server_cookie=2216e75f130524ba in_seq=0 out_seq=0 2024-05-20T18:31:11.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.498+0000 7fe1abfff640 10 client.6555 ms_handle_connect on v2:172.21.15.63:6832/4036379300 2024-05-20T18:31:11.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.498+0000 7fe1abfff640 1 -- 172.21.15.5:0/2734912537 <== mds.0 v2:172.21.15.63:6832/4036379300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d9768524b0 con 0x55d9764da570 2024-05-20T18:31:11.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.498+0000 7fe1abfff640 10 client.6555 handle_command_reply: tid=0 2024-05-20T18:31:11.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.498+0000 7fe1da5c4740 1 client.6555 shutdown 2024-05-20T18:31:11.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.498+0000 7fe1da5c4740 1 -- 172.21.15.5:0/2734912537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d976525670 msgr2=0x55d976944970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:11.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.498+0000 7fe1da5c4740 1 --2- 172.21.15.5:0/2734912537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d976525670 0x55d976944970 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe1c405e340 tx=0x7fe1c4013000 comp rx=0 tx=0).stop 2024-05-20T18:31:11.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.498+0000 7fe1da5c4740 1 -- 172.21.15.5:0/2734912537 shutdown_connections 2024-05-20T18:31:11.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.498+0000 7fe1da5c4740 1 --2- 172.21.15.5:0/2734912537 >> [v2:172.21.15.63:6832/4036379300,v1:172.21.15.63:6834/4036379300] conn(0x55d9764da570 0x55d976945300 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.498+0000 7fe1da5c4740 1 --2- 172.21.15.5:0/2734912537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d976525670 0x55d976944970 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.498+0000 7fe1abfff640 0 client.6555 ms_handle_reset on v2:172.21.15.63:6832/4036379300 2024-05-20T18:31:11.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.498+0000 7fe1da5c4740 1 --2- 172.21.15.5:0/2734912537 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d97651d490 0x55d97690cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.498+0000 7fe1da5c4740 1 --2- 172.21.15.5:0/2734912537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d976505130 0x55d97690c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.498+0000 7fe1da5c4740 1 -- 172.21.15.5:0/2734912537 >> 172.21.15.5:0/2734912537 conn(0x55d97653a4a0 msgr2=0x55d97693b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:11.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.498+0000 7fe1da5c4740 1 -- 172.21.15.5:0/2734912537 shutdown_connections 2024-05-20T18:31:11.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.498+0000 7fe1da5c4740 1 -- 172.21.15.5:0/2734912537 wait complete. 2024-05-20T18:31:11.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.498+0000 7fe1da5c4740 20 client.6555 trim_cache size 0 max 16384 2024-05-20T18:31:11.504 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:31:11.504 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-20T18:31:11.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.499+0000 7fe1d363d640 1 -- 172.21.15.5:0/4052500767 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe1b006be30 msgr2=0x7fe1b006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:11.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.499+0000 7fe1d363d640 1 --2- 172.21.15.5:0/4052500767 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe1b006be30 0x7fe1b006e340 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8034000 tx=0x7fe1c802e000 comp rx=0 tx=0).stop 2024-05-20T18:31:11.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.499+0000 7fe1d363d640 1 -- 172.21.15.5:0/4052500767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1cc0b9440 msgr2=0x7fe1cc0b9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:11.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.499+0000 7fe1d363d640 1 --2- 172.21.15.5:0/4052500767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1cc0b9440 0x7fe1cc0b9850 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fe1c400ef90 tx=0x7fe1c400c490 comp rx=0 tx=0).stop 2024-05-20T18:31:11.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.499+0000 7fe1d363d640 1 -- 172.21.15.5:0/4052500767 shutdown_connections 2024-05-20T18:31:11.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.499+0000 7fe1d363d640 1 --2- 172.21.15.5:0/4052500767 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe1b006be30 0x7fe1b006e340 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.499+0000 7fe1d363d640 1 --2- 172.21.15.5:0/4052500767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1cc0b9440 0x7fe1cc0b9850 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.499+0000 7fe1d363d640 1 --2- 172.21.15.5:0/4052500767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1cc162820 0x7fe1cc0b6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.499+0000 7fe1d363d640 1 --2- 172.21.15.5:0/4052500767 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1cc0b3e80 0x7fe1cc1f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.499+0000 7fe1d363d640 1 -- 172.21.15.5:0/4052500767 >> 172.21.15.5:0/4052500767 conn(0x7fe1cc0d4bd0 msgr2=0x7fe1cc0ba560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:11.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.499+0000 7fe1d363d640 1 -- 172.21.15.5:0/4052500767 shutdown_connections 2024-05-20T18:31:11.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.499+0000 7fe1d363d640 1 -- 172.21.15.5:0/4052500767 wait complete. 2024-05-20T18:31:11.510 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-20T18:31:11.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.624+0000 7f590dd77640 1 Processor -- start 2024-05-20T18:31:11.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.624+0000 7f590dd77640 1 -- start start 2024-05-20T18:31:11.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.625+0000 7f590dd77640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59080b4070 0x7f59080ba5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.625+0000 7f590dd77640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f590806dae0 0x7f59080b8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.625+0000 7f590dd77640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f590815fa90 con 0x7f59080d0560 2024-05-20T18:31:11.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.625+0000 7f590dd77640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f59080b8490 con 0x7f590806dae0 2024-05-20T18:31:11.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.625+0000 7f590dd77640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f59080d3f00 con 0x7f59080b4070 2024-05-20T18:31:11.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.626+0000 7f5907fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f590806dae0 0x7f59080b8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.626+0000 7f5907fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f590806dae0 0x7f59080b8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40158/0 (socket says 172.21.15.5:40158) 2024-05-20T18:31:11.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.626+0000 7f5907fff640 1 -- 172.21.15.5:0/2050685056 learned_addr learned my addr 172.21.15.5:0/2050685056 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:11.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.626+0000 7f59077fe640 1 --2- 172.21.15.5:0/2050685056 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59080b4070 0x7f59080ba5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.626+0000 7f5907fff640 1 -- 172.21.15.5:0/2050685056 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59080b4070 msgr2=0x7f59080ba5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:11.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.626+0000 7f5907fff640 1 --2- 172.21.15.5:0/2050685056 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59080b4070 0x7f59080ba5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.626+0000 7f5907fff640 1 -- 172.21.15.5:0/2050685056 >> v1:172.21.15.63:6789/0 conn(0x7f59080d0560 legacy=0x7f590806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.626+0000 7f5907fff640 1 -- 172.21.15.5:0/2050685056 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59080d41e0 con 0x7f590806dae0 2024-05-20T18:31:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.627+0000 7f5907fff640 1 --2- 172.21.15.5:0/2050685056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f590806dae0 0x7f59080b8920 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5900009e50 tx=0x7f590002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba14b31e61a72e61 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.627+0000 7f59067fc640 1 -- 172.21.15.5:0/2050685056 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900036070 con 0x7f590806dae0 2024-05-20T18:31:11.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.627+0000 7f59067fc640 1 -- 172.21.15.5:0/2050685056 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5900031600 con 0x7f590806dae0 2024-05-20T18:31:11.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.627+0000 7f59067fc640 1 -- 172.21.15.5:0/2050685056 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59000319d0 con 0x7f590806dae0 2024-05-20T18:31:11.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.628+0000 7f590dd77640 1 -- 172.21.15.5:0/2050685056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f590806dae0 msgr2=0x7f59080b8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:11.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.628+0000 7f590dd77640 1 --2- 172.21.15.5:0/2050685056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f590806dae0 0x7f59080b8920 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5900009e50 tx=0x7f590002a710 comp rx=0 tx=0).stop 2024-05-20T18:31:11.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.628+0000 7f590dd77640 1 -- 172.21.15.5:0/2050685056 shutdown_connections 2024-05-20T18:31:11.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.628+0000 7f590dd77640 1 --2- 172.21.15.5:0/2050685056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f590806dae0 0x7f59080b8920 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.628+0000 7f590dd77640 1 --2- 172.21.15.5:0/2050685056 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59080b4070 0x7f59080ba5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.628+0000 7f590dd77640 1 -- 172.21.15.5:0/2050685056 >> 172.21.15.5:0/2050685056 conn(0x7f59080d4c10 msgr2=0x7f59080d5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:11.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.628+0000 7f590dd77640 1 -- 172.21.15.5:0/2050685056 shutdown_connections 2024-05-20T18:31:11.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.628+0000 7f590dd77640 1 -- 172.21.15.5:0/2050685056 wait complete. 2024-05-20T18:31:11.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.630+0000 7f590dd77640 1 Processor -- start 2024-05-20T18:31:11.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.630+0000 7f590dd77640 1 -- start start 2024-05-20T18:31:11.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.631+0000 7f590dd77640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59080b4070 0x7f59080cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.631+0000 7f590dd77640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59080d0560 0x7f59081f8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.631+0000 7f590dd77640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f590814cb60 0x7f59081f8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.631+0000 7f590dd77640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f59080bab00 con 0x7f590814cb60 2024-05-20T18:31:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.631+0000 7f590dd77640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f59080d47a0 con 0x7f59080b4070 2024-05-20T18:31:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.631+0000 7f590dd77640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f59080d3b20 con 0x7f59080d0560 2024-05-20T18:31:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.631+0000 7f5907fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f590814cb60 0x7f59081f8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.631+0000 7f5907fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f590814cb60 0x7f59081f8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40164/0 (socket says 172.21.15.5:40164) 2024-05-20T18:31:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.631+0000 7f5907fff640 1 -- 172.21.15.5:0/2036263564 learned_addr learned my addr 172.21.15.5:0/2036263564 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.631+0000 7f5906ffd640 1 --2- 172.21.15.5:0/2036263564 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59080d0560 0x7f59081f8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.632+0000 7f5907fff640 1 -- 172.21.15.5:0/2036263564 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59080d0560 msgr2=0x7f59081f8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:11.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.632+0000 7f5907fff640 1 --2- 172.21.15.5:0/2036263564 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59080d0560 0x7f59081f8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.632+0000 7f59077fe640 1 --2- 172.21.15.5:0/2036263564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59080b4070 0x7f59080cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.632+0000 7f5907fff640 1 -- 172.21.15.5:0/2036263564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59080b4070 msgr2=0x7f59080cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:11.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.632+0000 7f5907fff640 1 --2- 172.21.15.5:0/2036263564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59080b4070 0x7f59080cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.632+0000 7f5907fff640 1 -- 172.21.15.5:0/2036263564 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59080cd0e0 con 0x7f590814cb60 2024-05-20T18:31:11.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.632+0000 7f5907fff640 1 --2- 172.21.15.5:0/2036263564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f590814cb60 0x7f59081f8bb0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f5900009f70 tx=0x7f5900004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:11.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.632+0000 7f58effff640 1 -- 172.21.15.5:0/2036263564 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900036050 con 0x7f590814cb60 2024-05-20T18:31:11.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.632+0000 7f58effff640 1 -- 172.21.15.5:0/2036263564 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5900042070 con 0x7f590814cb60 2024-05-20T18:31:11.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.632+0000 7f590dd77640 1 -- 172.21.15.5:0/2036263564 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59080cd3c0 con 0x7f590814cb60 2024-05-20T18:31:11.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.632+0000 7f58effff640 1 -- 172.21.15.5:0/2036263564 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900009510 con 0x7f590814cb60 2024-05-20T18:31:11.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.632+0000 7f590dd77640 1 -- 172.21.15.5:0/2036263564 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59081474b0 con 0x7f590814cb60 2024-05-20T18:31:11.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.633+0000 7f590dd77640 1 -- 172.21.15.5:0/2036263564 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59080690b0 con 0x7f590814cb60 2024-05-20T18:31:11.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.634+0000 7f58effff640 1 -- 172.21.15.5:0/2036263564 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5900007bf0 con 0x7f590814cb60 2024-05-20T18:31:11.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.634+0000 7f58effff640 1 --2- 172.21.15.5:0/2036263564 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f58f406be60 0x7f58f406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:11.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.634+0000 7f58effff640 1 -- 172.21.15.5:0/2036263564 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6246+0+0 (secure 0 0 0) 0x7f59000ab7e0 con 0x7f590814cb60 2024-05-20T18:31:11.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.634+0000 7f59077fe640 1 --2- 172.21.15.5:0/2036263564 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f58f406be60 0x7f58f406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:11.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.635+0000 7f59077fe640 1 --2- 172.21.15.5:0/2036263564 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f58f406be60 0x7f58f406e370 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f58f8009970 tx=0x7f58f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:11.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.639+0000 7f58effff640 1 -- 172.21.15.5:0/2036263564 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f590004b020 con 0x7f590814cb60 2024-05-20T18:31:11.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.862+0000 7f590dd77640 1 -- 172.21.15.5:0/2036263564 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f590808d9f0 con 0x7f590814cb60 2024-05-20T18:31:11.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.863+0000 7f58effff640 1 -- 172.21.15.5:0/2036263564 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f5900039300 con 0x7f590814cb60 2024-05-20T18:31:11.863 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:31:11.863 INFO:teuthology.orchestra.run.smithi005.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-20T18:31:11.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.865+0000 7f590dd77640 1 -- 172.21.15.5:0/2036263564 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f58f406be60 msgr2=0x7f58f406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:11.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.865+0000 7f590dd77640 1 --2- 172.21.15.5:0/2036263564 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f58f406be60 0x7f58f406e370 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f58f8009970 tx=0x7f58f8008040 comp rx=0 tx=0).stop 2024-05-20T18:31:11.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.865+0000 7f590dd77640 1 -- 172.21.15.5:0/2036263564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f590814cb60 msgr2=0x7f59081f8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:11.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.865+0000 7f590dd77640 1 --2- 172.21.15.5:0/2036263564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f590814cb60 0x7f59081f8bb0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f5900009f70 tx=0x7f5900004a40 comp rx=0 tx=0).stop 2024-05-20T18:31:11.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.865+0000 7f590dd77640 1 -- 172.21.15.5:0/2036263564 shutdown_connections 2024-05-20T18:31:11.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.865+0000 7f590dd77640 1 --2- 172.21.15.5:0/2036263564 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f58f406be60 0x7f58f406e370 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.865+0000 7f590dd77640 1 --2- 172.21.15.5:0/2036263564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f590814cb60 0x7f59081f8bb0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.865+0000 7f590dd77640 1 --2- 172.21.15.5:0/2036263564 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59080d0560 0x7f59081f8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.865+0000 7f590dd77640 1 --2- 172.21.15.5:0/2036263564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f59080b4070 0x7f59080cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:11.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.865+0000 7f590dd77640 1 -- 172.21.15.5:0/2036263564 >> 172.21.15.5:0/2036263564 conn(0x7f59080d4c10 msgr2=0x7f590815f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:11.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.865+0000 7f590dd77640 1 -- 172.21.15.5:0/2036263564 shutdown_connections 2024-05-20T18:31:11.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:11.865+0000 7f590dd77640 1 -- 172.21.15.5:0/2036263564 wait complete. 2024-05-20T18:31:11.876 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-20T18:31:11.877 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-20T18:31:11.877 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T18:31:11.915 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T18:31:11.970 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-20T18:31:11.970 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T18:31:11.970 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:31:11.970 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:31:11.987 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:31:11.987 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:31:12.044 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:31:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.200+0000 7efd8995e640 1 Processor -- start 2024-05-20T18:31:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.200+0000 7efd8995e640 1 -- start start 2024-05-20T18:31:12.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.201+0000 7efd8995e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd8415d530 0x7efd8415aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:12.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.201+0000 7efd8995e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7efd84070850 0x7efd84070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:12.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.201+0000 7efd8995e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd840cd580 con 0x7efd840b1700 2024-05-20T18:31:12.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.201+0000 7efd8995e640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7efd840ce190 con 0x7efd84070850 2024-05-20T18:31:12.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.201+0000 7efd8995e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efd840cd860 con 0x7efd8415d530 2024-05-20T18:31:12.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.201+0000 7efd7bfff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7efd840b1700 0x7efd840b1b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37404/0 (socket says 172.21.15.5:37404) 2024-05-20T18:31:12.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.201+0000 7efd7bfff640 1 -- 172.21.15.5:0/2120408423 learned_addr learned my addr 172.21.15.5:0/2120408423 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:12.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.202+0000 7efd82ffd640 1 --2- 172.21.15.5:0/2120408423 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd8415d530 0x7efd8415aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:12.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.202+0000 7efd837fe640 1 --2- 172.21.15.5:0/2120408423 >> v2:172.21.15.63:3300/0 conn(0x7efd84070850 0x7efd84070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:12.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.202+0000 7efd827fc640 1 -- 172.21.15.5:0/2120408423 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3618471490 0 0) 0x7efd840cd580 con 0x7efd840b1700 2024-05-20T18:31:12.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.202+0000 7efd827fc640 1 -- 172.21.15.5:0/2120408423 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd70003520 con 0x7efd840b1700 2024-05-20T18:31:12.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.202+0000 7efd827fc640 1 -- 172.21.15.5:0/2120408423 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1793065886 0 0) 0x7efd70003520 con 0x7efd840b1700 2024-05-20T18:31:12.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.202+0000 7efd827fc640 1 -- 172.21.15.5:0/2120408423 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd8415d530 msgr2=0x7efd8415aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:12.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.202+0000 7efd827fc640 1 --2- 172.21.15.5:0/2120408423 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd8415d530 0x7efd8415aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:12.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.202+0000 7efd827fc640 1 -- 172.21.15.5:0/2120408423 >> v2:172.21.15.63:3300/0 conn(0x7efd84070850 msgr2=0x7efd84070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:12.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.202+0000 7efd827fc640 1 --2- 172.21.15.5:0/2120408423 >> v2:172.21.15.63:3300/0 conn(0x7efd84070850 0x7efd84070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:12.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.202+0000 7efd827fc640 1 -- 172.21.15.5:0/2120408423 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd840cdb40 con 0x7efd840b1700 2024-05-20T18:31:12.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.202+0000 7efd827fc640 1 -- 172.21.15.5:0/2120408423 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7efd6c001670 con 0x7efd840b1700 2024-05-20T18:31:12.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.202+0000 7efd827fc640 1 -- 172.21.15.5:0/2120408423 >> v1:172.21.15.5:6789/0 conn(0x7efd840b1700 legacy=0x7efd840b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:12.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.202+0000 7efd827fc640 1 --2- 172.21.15.5:0/2120408423 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd8415d530 0x7efd700039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:12.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.202+0000 7efd827fc640 1 --2- 172.21.15.5:0/2120408423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd70003f30 0x7efd70004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:12.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.202+0000 7efd827fc640 1 --2- 172.21.15.5:0/2120408423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd70004d40 0x7efd70005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:12.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.202+0000 7efd827fc640 1 -- 172.21.15.5:0/2120408423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efd840ce190 con 0x7efd70004d40 2024-05-20T18:31:12.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.202+0000 7efd827fc640 1 -- 172.21.15.5:0/2120408423 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efd840cd860 con 0x7efd70003f30 2024-05-20T18:31:12.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.202+0000 7efd827fc640 1 -- 172.21.15.5:0/2120408423 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efd7000ba10 con 0x7efd8415d530 2024-05-20T18:31:12.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.202+0000 7efd837fe640 1 --2- 172.21.15.5:0/2120408423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd70004d40 0x7efd70005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:12.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.202+0000 7efd82ffd640 1 --2- 172.21.15.5:0/2120408423 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd8415d530 0x7efd700039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:12.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.202+0000 7efd7bfff640 1 --2- 172.21.15.5:0/2120408423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd70003f30 0x7efd70004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:12.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.203+0000 7efd837fe640 1 -- 172.21.15.5:0/2120408423 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd8415d530 msgr2=0x7efd700039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:12.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.203+0000 7efd837fe640 1 --2- 172.21.15.5:0/2120408423 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd8415d530 0x7efd700039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:12.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.203+0000 7efd837fe640 1 -- 172.21.15.5:0/2120408423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd70003f30 msgr2=0x7efd70004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:12.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.203+0000 7efd837fe640 1 --2- 172.21.15.5:0/2120408423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd70003f30 0x7efd70004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:12.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.203+0000 7efd837fe640 1 -- 172.21.15.5:0/2120408423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd70003520 con 0x7efd70004d40 2024-05-20T18:31:12.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.203+0000 7efd837fe640 1 --2- 172.21.15.5:0/2120408423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd70004d40 0x7efd70005150 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7efd840d3610 tx=0x7efd70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e755648011204c6d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:12.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.204+0000 7efd827fc640 1 -- 172.21.15.5:0/2120408423 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c037070 con 0x7efd70004d40 2024-05-20T18:31:12.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.204+0000 7efd827fc640 1 -- 172.21.15.5:0/2120408423 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd7c02e050 con 0x7efd70004d40 2024-05-20T18:31:12.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.204+0000 7efd827fc640 1 -- 172.21.15.5:0/2120408423 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c009c10 con 0x7efd70004d40 2024-05-20T18:31:12.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.204+0000 7efd8995e640 1 -- 172.21.15.5:0/2120408423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd70004d40 msgr2=0x7efd70005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:12.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.204+0000 7efd8995e640 1 --2- 172.21.15.5:0/2120408423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd70004d40 0x7efd70005150 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7efd840d3610 tx=0x7efd70005830 comp rx=0 tx=0).stop 2024-05-20T18:31:12.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.204+0000 7efd837fe640 1 -- 172.21.15.5:0/2120408423 reap_dead start 2024-05-20T18:31:12.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.204+0000 7efd8995e640 1 -- 172.21.15.5:0/2120408423 shutdown_connections 2024-05-20T18:31:12.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.204+0000 7efd8995e640 1 -- 172.21.15.5:0/2120408423 >> 172.21.15.5:0/2120408423 conn(0x7efd840d29d0 msgr2=0x7efd840d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:12.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.204+0000 7efd8995e640 1 -- 172.21.15.5:0/2120408423 shutdown_connections 2024-05-20T18:31:12.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.204+0000 7efd8995e640 1 -- 172.21.15.5:0/2120408423 wait complete. 2024-05-20T18:31:12.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.206+0000 7efd8995e640 1 Processor -- start 2024-05-20T18:31:12.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.206+0000 7efd8995e640 1 -- start start 2024-05-20T18:31:12.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.208+0000 7efd8995e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd70004d40 0x7efd840cc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:12.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.208+0000 7efd8995e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd8415cd10 0x7efd840c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:12.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.208+0000 7efd8995e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd8415b490 0x7efd840c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:12.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.208+0000 7efd8995e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efd840711a0 con 0x7efd70004d40 2024-05-20T18:31:12.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.208+0000 7efd8995e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efd840cd230 con 0x7efd8415cd10 2024-05-20T18:31:12.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.208+0000 7efd8995e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efd84071320 con 0x7efd8415b490 2024-05-20T18:31:12.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.208+0000 7efd82ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd70004d40 0x7efd840cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:12.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.208+0000 7efd82ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd70004d40 0x7efd840cc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40180/0 (socket says 172.21.15.5:40180) 2024-05-20T18:31:12.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.208+0000 7efd82ffd640 1 -- 172.21.15.5:0/815038688 learned_addr learned my addr 172.21.15.5:0/815038688 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:12.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.208+0000 7efd7bfff640 1 --2- 172.21.15.5:0/815038688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd8415cd10 0x7efd840c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:12.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.208+0000 7efd837fe640 1 --2- 172.21.15.5:0/815038688 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd8415b490 0x7efd840c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:12.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.208+0000 7efd82ffd640 1 -- 172.21.15.5:0/815038688 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd8415b490 msgr2=0x7efd840c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:12.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.208+0000 7efd82ffd640 1 --2- 172.21.15.5:0/815038688 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd8415b490 0x7efd840c9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:12.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.208+0000 7efd82ffd640 1 -- 172.21.15.5:0/815038688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd8415cd10 msgr2=0x7efd840c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:12.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.208+0000 7efd82ffd640 1 --2- 172.21.15.5:0/815038688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd8415cd10 0x7efd840c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:12.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.208+0000 7efd82ffd640 1 -- 172.21.15.5:0/815038688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8415bc70 con 0x7efd70004d40 2024-05-20T18:31:12.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.208+0000 7efd7bfff640 1 --2- 172.21.15.5:0/815038688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd8415cd10 0x7efd840c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:12.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.208+0000 7efd82ffd640 1 --2- 172.21.15.5:0/815038688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd70004d40 0x7efd840cc380 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7efd74011910 tx=0x7efd74011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:12.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.208+0000 7efd80ff9640 1 -- 172.21.15.5:0/815038688 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7400faf0 con 0x7efd70004d40 2024-05-20T18:31:12.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.208+0000 7efd80ff9640 1 -- 172.21.15.5:0/815038688 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd7400fc90 con 0x7efd70004d40 2024-05-20T18:31:12.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.209+0000 7efd8995e640 1 -- 172.21.15.5:0/815038688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd8415bf50 con 0x7efd70004d40 2024-05-20T18:31:12.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.209+0000 7efd80ff9640 1 -- 172.21.15.5:0/815038688 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74014440 con 0x7efd70004d40 2024-05-20T18:31:12.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.209+0000 7efd8995e640 1 -- 172.21.15.5:0/815038688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd8415c510 con 0x7efd70004d40 2024-05-20T18:31:12.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.210+0000 7efd8995e640 1 -- 172.21.15.5:0/815038688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd84075b00 con 0x7efd70004d40 2024-05-20T18:31:12.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.210+0000 7efd80ff9640 1 -- 172.21.15.5:0/815038688 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efd74012040 con 0x7efd70004d40 2024-05-20T18:31:12.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.211+0000 7efd80ff9640 1 --2- 172.21.15.5:0/815038688 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efd6406be30 0x7efd6406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:12.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.211+0000 7efd7bfff640 1 --2- 172.21.15.5:0/815038688 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efd6406be30 0x7efd6406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:12.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.211+0000 7efd80ff9640 1 -- 172.21.15.5:0/815038688 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6246+0+0 (secure 0 0 0) 0x7efd7408b820 con 0x7efd70004d40 2024-05-20T18:31:12.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.211+0000 7efd7bfff640 1 --2- 172.21.15.5:0/815038688 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efd6406be30 0x7efd6406e340 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efd6c007fd0 tx=0x7efd6c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:12.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.216+0000 7efd80ff9640 1 -- 172.21.15.5:0/815038688 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7efd74019030 con 0x7efd70004d40 2024-05-20T18:31:12.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.403+0000 7efd8995e640 1 -- 172.21.15.5:0/815038688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd840c6080 con 0x7efd70004d40 2024-05-20T18:31:12.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.404+0000 7efd80ff9640 1 -- 172.21.15.5:0/815038688 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+17004 (secure 0 0 0) 0x7efd74059f20 con 0x7efd70004d40 2024-05-20T18:31:12.404 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:31:12.405 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":65,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:30:57.074551+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:29:57.993861+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-20T18:29:59.267582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.18","mappings":[{"from":7,"to":2}]},{"pgid":"10.c","mappings":[{"from":4,"to":5}]},{"pgid":"10.1d","mappings":[{"from":4,"to":3}]},{"pgid":"10.33","mappings":[{"from":4,"to":3}]},{"pgid":"10.37","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:31:12.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.406+0000 7efd8995e640 1 -- 172.21.15.5:0/815038688 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efd6406be30 msgr2=0x7efd6406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:12.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.406+0000 7efd8995e640 1 --2- 172.21.15.5:0/815038688 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efd6406be30 0x7efd6406e340 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efd6c007fd0 tx=0x7efd6c009040 comp rx=0 tx=0).stop 2024-05-20T18:31:12.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.406+0000 7efd8995e640 1 -- 172.21.15.5:0/815038688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd70004d40 msgr2=0x7efd840cc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:12.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.406+0000 7efd8995e640 1 --2- 172.21.15.5:0/815038688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd70004d40 0x7efd840cc380 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7efd74011910 tx=0x7efd74011dd0 comp rx=0 tx=0).stop 2024-05-20T18:31:12.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.406+0000 7efd8995e640 1 -- 172.21.15.5:0/815038688 shutdown_connections 2024-05-20T18:31:12.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.406+0000 7efd8995e640 1 --2- 172.21.15.5:0/815038688 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efd6406be30 0x7efd6406e340 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:12.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.406+0000 7efd8995e640 1 --2- 172.21.15.5:0/815038688 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd8415b490 0x7efd840c9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:12.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.406+0000 7efd8995e640 1 --2- 172.21.15.5:0/815038688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd8415cd10 0x7efd840c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:12.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.406+0000 7efd8995e640 1 --2- 172.21.15.5:0/815038688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd70004d40 0x7efd840cc380 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:12.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.406+0000 7efd8995e640 1 -- 172.21.15.5:0/815038688 >> 172.21.15.5:0/815038688 conn(0x7efd840d29d0 msgr2=0x7efd840b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:12.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.406+0000 7efd8995e640 1 -- 172.21.15.5:0/815038688 shutdown_connections 2024-05-20T18:31:12.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.407+0000 7efd8995e640 1 -- 172.21.15.5:0/815038688 wait complete. 2024-05-20T18:31:12.418 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:29:57.993861+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:29:59.267582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:31:12.419 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:31:12.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.536+0000 7fe55b7ce640 1 Processor -- start 2024-05-20T18:31:12.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.536+0000 7fe55b7ce640 1 -- start start 2024-05-20T18:31:12.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.537+0000 7fe55b7ce640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe5540d1bc0 0x7fe5540d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:12.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.537+0000 7fe55b7ce640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5540d0370 con 0x7fe5540b4010 2024-05-20T18:31:12.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.537+0000 7fe55b7ce640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5540d0650 con 0x7fe554163a10 2024-05-20T18:31:12.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.537+0000 7fe55b7ce640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe5540d3ca0 con 0x7fe5540d1bc0 2024-05-20T18:31:12.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.537+0000 7fe558d69640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe5540b4010 0x7fe5540d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37418/0 (socket says 172.21.15.5:37418) 2024-05-20T18:31:12.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.537+0000 7fe558d69640 1 -- 172.21.15.5:0/2206409492 learned_addr learned my addr 172.21.15.5:0/2206409492 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:12.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.538+0000 7fe559d6b640 1 --2- 172.21.15.5:0/2206409492 >> v2:172.21.15.63:3300/0 conn(0x7fe5540d1bc0 0x7fe5540d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:12.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.538+0000 7fe54bfff640 1 -- 172.21.15.5:0/2206409492 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 961399972 0 0) 0x7fe5540d0370 con 0x7fe5540b4010 2024-05-20T18:31:12.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.538+0000 7fe54bfff640 1 -- 172.21.15.5:0/2206409492 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe540003520 con 0x7fe5540b4010 2024-05-20T18:31:12.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.538+0000 7fe54bfff640 1 -- 172.21.15.5:0/2206409492 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 534664564 0 0) 0x7fe540003520 con 0x7fe5540b4010 2024-05-20T18:31:12.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.538+0000 7fe54bfff640 1 -- 172.21.15.5:0/2206409492 >> v2:172.21.15.63:3300/0 conn(0x7fe5540d1bc0 msgr2=0x7fe5540d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:12.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.538+0000 7fe54bfff640 1 --2- 172.21.15.5:0/2206409492 >> v2:172.21.15.63:3300/0 conn(0x7fe5540d1bc0 0x7fe5540d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:12.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.538+0000 7fe54bfff640 1 -- 172.21.15.5:0/2206409492 >> v1:172.21.15.63:6789/0 conn(0x7fe554163a10 legacy=0x7fe5540cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:12.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.538+0000 7fe54bfff640 1 -- 172.21.15.5:0/2206409492 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5540d3650 con 0x7fe5540b4010 2024-05-20T18:31:12.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.538+0000 7fe54bfff640 1 -- 172.21.15.5:0/2206409492 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7fe544001670 con 0x7fe5540b4010 2024-05-20T18:31:12.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.538+0000 7fe54bfff640 1 -- 172.21.15.5:0/2206409492 >> v1:172.21.15.5:6789/0 conn(0x7fe5540b4010 legacy=0x7fe5540d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:12.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.538+0000 7fe54bfff640 1 --2- 172.21.15.5:0/2206409492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5400039b0 0x7fe540003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:12.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.538+0000 7fe54bfff640 1 --2- 172.21.15.5:0/2206409492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe540004800 0x7fe540004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:12.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.538+0000 7fe54bfff640 1 --2- 172.21.15.5:0/2206409492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5400051c0 0x7fe54000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:12.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.538+0000 7fe54bfff640 1 -- 172.21.15.5:0/2206409492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe5540d3ca0 con 0x7fe5400051c0 2024-05-20T18:31:12.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.538+0000 7fe54bfff640 1 -- 172.21.15.5:0/2206409492 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe54000c160 con 0x7fe5400039b0 2024-05-20T18:31:12.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.538+0000 7fe54bfff640 1 -- 172.21.15.5:0/2206409492 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe54000c360 con 0x7fe540004800 2024-05-20T18:31:12.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.538+0000 7fe559d6b640 1 --2- 172.21.15.5:0/2206409492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5400051c0 0x7fe54000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:12.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.538+0000 7fe55956a640 1 --2- 172.21.15.5:0/2206409492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5400039b0 0x7fe540003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:12.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.538+0000 7fe558d69640 1 --2- 172.21.15.5:0/2206409492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe540004800 0x7fe540004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:12.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.539+0000 7fe559d6b640 1 -- 172.21.15.5:0/2206409492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe540004800 msgr2=0x7fe540004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:12.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.539+0000 7fe559d6b640 1 --2- 172.21.15.5:0/2206409492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe540004800 0x7fe540004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:12.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.539+0000 7fe559d6b640 1 -- 172.21.15.5:0/2206409492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5400039b0 msgr2=0x7fe540003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:12.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.539+0000 7fe559d6b640 1 --2- 172.21.15.5:0/2206409492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5400039b0 0x7fe540003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:12.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.539+0000 7fe559d6b640 1 -- 172.21.15.5:0/2206409492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe540003520 con 0x7fe5400051c0 2024-05-20T18:31:12.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.539+0000 7fe559d6b640 1 -- 172.21.15.5:0/2206409492 reap_dead start 2024-05-20T18:31:12.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.539+0000 7fe559d6b640 1 --2- 172.21.15.5:0/2206409492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5400051c0 0x7fe54000b9c0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fe54c009fc0 tx=0x7fe540005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f837a5d23c1d5a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:12.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.540+0000 7fe54bfff640 1 -- 172.21.15.5:0/2206409492 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c009c10 con 0x7fe5400051c0 2024-05-20T18:31:12.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.540+0000 7fe54bfff640 1 -- 172.21.15.5:0/2206409492 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe54c003bd0 con 0x7fe5400051c0 2024-05-20T18:31:12.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.540+0000 7fe54bfff640 1 -- 172.21.15.5:0/2206409492 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c004020 con 0x7fe5400051c0 2024-05-20T18:31:12.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.540+0000 7fe55b7ce640 1 -- 172.21.15.5:0/2206409492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5400051c0 msgr2=0x7fe54000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:12.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.540+0000 7fe55b7ce640 1 --2- 172.21.15.5:0/2206409492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5400051c0 0x7fe54000b9c0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fe54c009fc0 tx=0x7fe540005830 comp rx=0 tx=0).stop 2024-05-20T18:31:12.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.540+0000 7fe55b7ce640 1 -- 172.21.15.5:0/2206409492 shutdown_connections 2024-05-20T18:31:12.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.540+0000 7fe55b7ce640 1 --2- 172.21.15.5:0/2206409492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5400051c0 0x7fe54000b9c0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:12.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.540+0000 7fe55b7ce640 1 -- 172.21.15.5:0/2206409492 >> 172.21.15.5:0/2206409492 conn(0x7fe55406d240 msgr2=0x7fe55406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:12.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.540+0000 7fe55b7ce640 1 -- 172.21.15.5:0/2206409492 shutdown_connections 2024-05-20T18:31:12.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.540+0000 7fe55b7ce640 1 -- 172.21.15.5:0/2206409492 wait complete. 2024-05-20T18:31:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.542+0000 7fe55b7ce640 1 Processor -- start 2024-05-20T18:31:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.542+0000 7fe55b7ce640 1 -- start start 2024-05-20T18:31:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.543+0000 7fe55b7ce640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5400051c0 0x7fe5541f54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.543+0000 7fe55b7ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe554157110 0x7fe554153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.544+0000 7fe55b7ce640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5540b2b10 0x7fe5540b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.544+0000 7fe55b7ce640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe5540d42b0 con 0x7fe554157110 2024-05-20T18:31:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.544+0000 7fe55b7ce640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe5540d4130 con 0x7fe5540b2b10 2024-05-20T18:31:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.544+0000 7fe55b7ce640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe5540d4430 con 0x7fe5400051c0 2024-05-20T18:31:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.544+0000 7fe558d69640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe554157110 0x7fe554153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.544+0000 7fe558d69640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe554157110 0x7fe554153b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40196/0 (socket says 172.21.15.5:40196) 2024-05-20T18:31:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.544+0000 7fe558d69640 1 -- 172.21.15.5:0/3881796453 learned_addr learned my addr 172.21.15.5:0/3881796453 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.544+0000 7fe55956a640 1 --2- 172.21.15.5:0/3881796453 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5400051c0 0x7fe5541f54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.544+0000 7fe559d6b640 1 --2- 172.21.15.5:0/3881796453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5540b2b10 0x7fe5540b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.544+0000 7fe558d69640 1 -- 172.21.15.5:0/3881796453 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5400051c0 msgr2=0x7fe5541f54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.544+0000 7fe558d69640 1 --2- 172.21.15.5:0/3881796453 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5400051c0 0x7fe5541f54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.544+0000 7fe558d69640 1 -- 172.21.15.5:0/3881796453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5540b2b10 msgr2=0x7fe5540b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.544+0000 7fe558d69640 1 --2- 172.21.15.5:0/3881796453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5540b2b10 0x7fe5540b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.544+0000 7fe558d69640 1 -- 172.21.15.5:0/3881796453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe554154b30 con 0x7fe554157110 2024-05-20T18:31:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.544+0000 7fe558d69640 1 --2- 172.21.15.5:0/3881796453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe554157110 0x7fe554153b70 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fe5440072c0 tx=0x7fe544007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.544+0000 7fe549ffb640 1 -- 172.21.15.5:0/3881796453 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe544001660 con 0x7fe554157110 2024-05-20T18:31:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.544+0000 7fe549ffb640 1 -- 172.21.15.5:0/3881796453 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe544018070 con 0x7fe554157110 2024-05-20T18:31:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.544+0000 7fe549ffb640 1 -- 172.21.15.5:0/3881796453 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54400fb80 con 0x7fe554157110 2024-05-20T18:31:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.544+0000 7fe55b7ce640 1 -- 172.21.15.5:0/3881796453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5540b3460 con 0x7fe554157110 2024-05-20T18:31:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.545+0000 7fe55b7ce640 1 -- 172.21.15.5:0/3881796453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe554155080 con 0x7fe554157110 2024-05-20T18:31:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.546+0000 7fe55b7ce640 1 -- 172.21.15.5:0/3881796453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe554075b00 con 0x7fe554157110 2024-05-20T18:31:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.546+0000 7fe549ffb640 1 -- 172.21.15.5:0/3881796453 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe54400fd20 con 0x7fe554157110 2024-05-20T18:31:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.546+0000 7fe549ffb640 1 --2- 172.21.15.5:0/3881796453 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe53806bfd0 0x7fe53806e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.547+0000 7fe549ffb640 1 -- 172.21.15.5:0/3881796453 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6246+0+0 (secure 0 0 0) 0x7fe54408afe0 con 0x7fe554157110 2024-05-20T18:31:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.547+0000 7fe55956a640 1 --2- 172.21.15.5:0/3881796453 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe53806bfd0 0x7fe53806e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.547+0000 7fe55956a640 1 --2- 172.21.15.5:0/3881796453 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe53806bfd0 0x7fe53806e4e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe550000c00 tx=0x7fe550009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:12.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.551+0000 7fe549ffb640 1 -- 172.21.15.5:0/3881796453 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe5540b3460 con 0x7fe554157110 2024-05-20T18:31:12.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.735+0000 7fe55b7ce640 1 -- 172.21.15.5:0/3881796453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe5540d3050 con 0x7fe554157110 2024-05-20T18:31:12.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.736+0000 7fe549ffb640 1 -- 172.21.15.5:0/3881796453 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) ==== 93+0+10 (secure 0 0 0) 0x7fe544059090 con 0x7fe554157110 2024-05-20T18:31:12.736 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:31:12.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.738+0000 7fe55b7ce640 1 -- 172.21.15.5:0/3881796453 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe53806bfd0 msgr2=0x7fe53806e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:12.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.738+0000 7fe55b7ce640 1 --2- 172.21.15.5:0/3881796453 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe53806bfd0 0x7fe53806e4e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe550000c00 tx=0x7fe550009040 comp rx=0 tx=0).stop 2024-05-20T18:31:12.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.738+0000 7fe55b7ce640 1 -- 172.21.15.5:0/3881796453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe554157110 msgr2=0x7fe554153b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:12.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.738+0000 7fe55b7ce640 1 --2- 172.21.15.5:0/3881796453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe554157110 0x7fe554153b70 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fe5440072c0 tx=0x7fe544007780 comp rx=0 tx=0).stop 2024-05-20T18:31:12.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.738+0000 7fe55b7ce640 1 -- 172.21.15.5:0/3881796453 shutdown_connections 2024-05-20T18:31:12.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.738+0000 7fe55b7ce640 1 --2- 172.21.15.5:0/3881796453 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe53806bfd0 0x7fe53806e4e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:12.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.738+0000 7fe55b7ce640 1 --2- 172.21.15.5:0/3881796453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5540b2b10 0x7fe5540b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:12.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.738+0000 7fe55b7ce640 1 --2- 172.21.15.5:0/3881796453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe554157110 0x7fe554153b70 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:12.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.738+0000 7fe55b7ce640 1 --2- 172.21.15.5:0/3881796453 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5400051c0 0x7fe5541f54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:12.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.738+0000 7fe55b7ce640 1 -- 172.21.15.5:0/3881796453 >> 172.21.15.5:0/3881796453 conn(0x7fe55406d240 msgr2=0x7fe55415f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:12.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.738+0000 7fe55b7ce640 1 -- 172.21.15.5:0/3881796453 shutdown_connections 2024-05-20T18:31:12.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.738+0000 7fe55b7ce640 1 -- 172.21.15.5:0/3881796453 wait complete. 2024-05-20T18:31:12.749 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:31:12.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.867+0000 7f225f59e640 1 Processor -- start 2024-05-20T18:31:12.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.867+0000 7f225f59e640 1 -- start start 2024-05-20T18:31:12.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.869+0000 7f225f59e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22600b3d70 0x7f22600d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:12.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.869+0000 7f225f59e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f22600d17e0 0x7f22600d3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:12.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.869+0000 7f225f59e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22601605a0 con 0x7f22600cf860 2024-05-20T18:31:12.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.869+0000 7f225f59e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f22600b4180 con 0x7f22600d17e0 2024-05-20T18:31:12.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.869+0000 7f225f59e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f226015ffe0 con 0x7f22600b3d70 2024-05-20T18:31:12.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.869+0000 7f225ed9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f22600d17e0 0x7f22600d3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:12.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.869+0000 7f225e59c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f22600cf860 0x7f2260163520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37430/0 (socket says 172.21.15.5:37430) 2024-05-20T18:31:12.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.869+0000 7f225e59c640 1 -- 172.21.15.5:0/2563759743 learned_addr learned my addr 172.21.15.5:0/2563759743 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:12.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.869+0000 7f225dd9b640 1 --2- 172.21.15.5:0/2563759743 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22600b3d70 0x7f22600d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:12.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.869+0000 7f225ed9d640 1 -- 172.21.15.5:0/2563759743 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22600b3d70 msgr2=0x7f22600d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:12.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.869+0000 7f225ed9d640 1 --2- 172.21.15.5:0/2563759743 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22600b3d70 0x7f22600d12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:12.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.869+0000 7f225ed9d640 1 -- 172.21.15.5:0/2563759743 >> v1:172.21.15.5:6789/0 conn(0x7f22600cf860 legacy=0x7f2260163520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:12.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.869+0000 7f225ed9d640 1 -- 172.21.15.5:0/2563759743 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22601602c0 con 0x7f22600d17e0 2024-05-20T18:31:12.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.869+0000 7f225d59a640 1 -- 172.21.15.5:0/2563759743 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2522145268 0 0) 0x7f22601605a0 con 0x7f22600cf860 2024-05-20T18:31:12.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.870+0000 7f225dd9b640 1 --2- 172.21.15.5:0/2563759743 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22600b3d70 0x7f22600d12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:12.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.870+0000 7f225ed9d640 1 --2- 172.21.15.5:0/2563759743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22600d17e0 0x7f22600d3140 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f2258009d90 tx=0x7f225802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c64640734363673 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:12.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.870+0000 7f225d59a640 1 -- 172.21.15.5:0/2563759743 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2258031600 con 0x7f22600d17e0 2024-05-20T18:31:12.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.870+0000 7f225d59a640 1 -- 172.21.15.5:0/2563759743 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22580317a0 con 0x7f22600d17e0 2024-05-20T18:31:12.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.870+0000 7f225d59a640 1 -- 172.21.15.5:0/2563759743 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2258031b70 con 0x7f22600d17e0 2024-05-20T18:31:12.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.871+0000 7f225f59e640 1 -- 172.21.15.5:0/2563759743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22600d17e0 msgr2=0x7f22600d3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:12.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.871+0000 7f225f59e640 1 --2- 172.21.15.5:0/2563759743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22600d17e0 0x7f22600d3140 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f2258009d90 tx=0x7f225802a5d0 comp rx=0 tx=0).stop 2024-05-20T18:31:12.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.871+0000 7f225f59e640 1 -- 172.21.15.5:0/2563759743 shutdown_connections 2024-05-20T18:31:12.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.871+0000 7f225f59e640 1 --2- 172.21.15.5:0/2563759743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22600d17e0 0x7f22600d3140 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:12.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.871+0000 7f225f59e640 1 --2- 172.21.15.5:0/2563759743 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22600b3d70 0x7f22600d12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:12.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.871+0000 7f225f59e640 1 -- 172.21.15.5:0/2563759743 >> 172.21.15.5:0/2563759743 conn(0x7f226006dbb0 msgr2=0x7f226006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:12.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.871+0000 7f225f59e640 1 -- 172.21.15.5:0/2563759743 shutdown_connections 2024-05-20T18:31:12.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.871+0000 7f225f59e640 1 -- 172.21.15.5:0/2563759743 wait complete. 2024-05-20T18:31:12.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.873+0000 7f225f59e640 1 Processor -- start 2024-05-20T18:31:12.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.873+0000 7f225f59e640 1 -- start start 2024-05-20T18:31:12.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.874+0000 7f225f59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22600b3d70 0x7f226015e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:12.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.874+0000 7f225f59e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22600cf860 0x7f22600ac410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:12.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.874+0000 7f225f59e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22600d17e0 0x7f22600b6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:12.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.874+0000 7f225f59e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f22600d1bf0 con 0x7f22600b3d70 2024-05-20T18:31:12.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.875+0000 7f225f59e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2260163000 con 0x7f22600cf860 2024-05-20T18:31:12.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.875+0000 7f225f59e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f22600d3680 con 0x7f22600d17e0 2024-05-20T18:31:12.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.875+0000 7f225e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22600b3d70 0x7f226015e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:12.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.875+0000 7f225e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22600b3d70 0x7f226015e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40208/0 (socket says 172.21.15.5:40208) 2024-05-20T18:31:12.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.875+0000 7f225e59c640 1 -- 172.21.15.5:0/1278457158 learned_addr learned my addr 172.21.15.5:0/1278457158 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:12.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.875+0000 7f225ed9d640 1 --2- 172.21.15.5:0/1278457158 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22600d17e0 0x7f22600b6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:12.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.875+0000 7f225dd9b640 1 --2- 172.21.15.5:0/1278457158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22600cf860 0x7f22600ac410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:12.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.875+0000 7f225e59c640 1 -- 172.21.15.5:0/1278457158 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22600d17e0 msgr2=0x7f22600b6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:12.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.875+0000 7f225e59c640 1 --2- 172.21.15.5:0/1278457158 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22600d17e0 0x7f22600b6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:12.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.875+0000 7f225e59c640 1 -- 172.21.15.5:0/1278457158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22600cf860 msgr2=0x7f22600ac410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:12.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.875+0000 7f225e59c640 1 --2- 172.21.15.5:0/1278457158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22600cf860 0x7f22600ac410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:12.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.875+0000 7f225e59c640 1 -- 172.21.15.5:0/1278457158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22600c8a00 con 0x7f22600b3d70 2024-05-20T18:31:12.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.875+0000 7f225dd9b640 1 --2- 172.21.15.5:0/1278457158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22600cf860 0x7f22600ac410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:12.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.875+0000 7f225e59c640 1 --2- 172.21.15.5:0/1278457158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22600b3d70 0x7f226015e150 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f225000c410 tx=0x7f225000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:12.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.875+0000 7f224effd640 1 -- 172.21.15.5:0/1278457158 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22500074e0 con 0x7f22600b3d70 2024-05-20T18:31:12.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.875+0000 7f224effd640 1 -- 172.21.15.5:0/1278457158 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22500047e0 con 0x7f22600b3d70 2024-05-20T18:31:12.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.875+0000 7f225f59e640 1 -- 172.21.15.5:0/1278457158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22600c8ce0 con 0x7f22600b3d70 2024-05-20T18:31:12.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.875+0000 7f224effd640 1 -- 172.21.15.5:0/1278457158 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2250004bd0 con 0x7f22600b3d70 2024-05-20T18:31:12.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.875+0000 7f225f59e640 1 -- 172.21.15.5:0/1278457158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22600c92a0 con 0x7f22600b3d70 2024-05-20T18:31:12.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.876+0000 7f225f59e640 1 -- 172.21.15.5:0/1278457158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22600cefc0 con 0x7f22600b3d70 2024-05-20T18:31:12.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.877+0000 7f224effd640 1 -- 172.21.15.5:0/1278457158 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2250011070 con 0x7f22600b3d70 2024-05-20T18:31:12.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.878+0000 7f224effd640 1 --2- 172.21.15.5:0/1278457158 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f224006be60 0x7f224006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:12.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.878+0000 7f224effd640 1 -- 172.21.15.5:0/1278457158 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6246+0+0 (secure 0 0 0) 0x7f225008ad20 con 0x7f22600b3d70 2024-05-20T18:31:12.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.878+0000 7f225dd9b640 1 --2- 172.21.15.5:0/1278457158 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f224006be60 0x7f224006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:12.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.878+0000 7f225dd9b640 1 --2- 172.21.15.5:0/1278457158 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f224006be60 0x7f224006e370 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f22600cde30 tx=0x7f2254008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:12.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:12.882+0000 7f224effd640 1 -- 172.21.15.5:0/1278457158 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2250059420 con 0x7f22600b3d70 2024-05-20T18:31:13.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.068+0000 7f225f59e640 1 -- 172.21.15.5:0/1278457158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f22600d1e00 con 0x7f22600b3d70 2024-05-20T18:31:13.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.068+0000 7f224effd640 1 -- 172.21.15.5:0/1278457158 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) ==== 104+0+11 (secure 0 0 0) 0x7f2250058d20 con 0x7f22600b3d70 2024-05-20T18:31:13.068 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:31:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.070+0000 7f225f59e640 1 -- 172.21.15.5:0/1278457158 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f224006be60 msgr2=0x7f224006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.070+0000 7f225f59e640 1 --2- 172.21.15.5:0/1278457158 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f224006be60 0x7f224006e370 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f22600cde30 tx=0x7f2254008040 comp rx=0 tx=0).stop 2024-05-20T18:31:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.070+0000 7f225f59e640 1 -- 172.21.15.5:0/1278457158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22600b3d70 msgr2=0x7f226015e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.070+0000 7f225f59e640 1 --2- 172.21.15.5:0/1278457158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22600b3d70 0x7f226015e150 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f225000c410 tx=0x7f225000c8d0 comp rx=0 tx=0).stop 2024-05-20T18:31:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.070+0000 7f225f59e640 1 -- 172.21.15.5:0/1278457158 shutdown_connections 2024-05-20T18:31:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.070+0000 7f225f59e640 1 --2- 172.21.15.5:0/1278457158 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f224006be60 0x7f224006e370 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.070+0000 7f225f59e640 1 --2- 172.21.15.5:0/1278457158 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f22600d17e0 0x7f22600b6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.070+0000 7f225f59e640 1 --2- 172.21.15.5:0/1278457158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22600cf860 0x7f22600ac410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.070+0000 7f225f59e640 1 --2- 172.21.15.5:0/1278457158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22600b3d70 0x7f226015e150 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.070+0000 7f225f59e640 1 -- 172.21.15.5:0/1278457158 >> 172.21.15.5:0/1278457158 conn(0x7f226006dbb0 msgr2=0x7f226006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.070+0000 7f225f59e640 1 -- 172.21.15.5:0/1278457158 shutdown_connections 2024-05-20T18:31:13.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.070+0000 7f225f59e640 1 -- 172.21.15.5:0/1278457158 wait complete. 2024-05-20T18:31:13.082 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:31:13.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.199+0000 7fd26f237640 1 Processor -- start 2024-05-20T18:31:13.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.200+0000 7fd26f237640 1 -- start start 2024-05-20T18:31:13.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.201+0000 7fd26f237640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2680b6620 0x7fd2680ba3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:13.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.201+0000 7fd26f237640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd2680d3bf0 0x7fd2680d4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:13.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.201+0000 7fd26f237640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2680d4540 con 0x7fd268162ce0 2024-05-20T18:31:13.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.201+0000 7fd26f237640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fd2680d3480 con 0x7fd2680d3bf0 2024-05-20T18:31:13.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.201+0000 7fd26f237640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd2681634a0 con 0x7fd2680b6620 2024-05-20T18:31:13.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.201+0000 7fd26d7d4640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd2680d3bf0 0x7fd2680d4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:13.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.201+0000 7fd267fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd268162ce0 0x7fd2680b8070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37440/0 (socket says 172.21.15.5:37440) 2024-05-20T18:31:13.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.201+0000 7fd267fff640 1 -- 172.21.15.5:0/2786892083 learned_addr learned my addr 172.21.15.5:0/2786892083 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:13.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.201+0000 7fd26cfd3640 1 --2- 172.21.15.5:0/2786892083 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2680b6620 0x7fd2680ba3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:13.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.201+0000 7fd26d7d4640 1 -- 172.21.15.5:0/2786892083 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2680b6620 msgr2=0x7fd2680ba3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:13.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.201+0000 7fd26d7d4640 1 --2- 172.21.15.5:0/2786892083 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2680b6620 0x7fd2680ba3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:13.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.201+0000 7fd26d7d4640 1 -- 172.21.15.5:0/2786892083 >> v1:172.21.15.5:6789/0 conn(0x7fd268162ce0 legacy=0x7fd2680b8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:13.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.201+0000 7fd26d7d4640 1 -- 172.21.15.5:0/2786892083 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd268163780 con 0x7fd2680d3bf0 2024-05-20T18:31:13.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.201+0000 7fd2677fe640 1 -- 172.21.15.5:0/2786892083 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2005817559 0 0) 0x7fd25c001110 con 0x7fd268162ce0 2024-05-20T18:31:13.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.202+0000 7fd26d7d4640 1 --2- 172.21.15.5:0/2786892083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2680d3bf0 0x7fd2680d4000 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd260009d90 tx=0x7fd26002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ce2f71ede3d7b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:13.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.202+0000 7fd2677fe640 1 -- 172.21.15.5:0/2786892083 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd260007740 con 0x7fd2680d3bf0 2024-05-20T18:31:13.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.202+0000 7fd2677fe640 1 -- 172.21.15.5:0/2786892083 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2600078e0 con 0x7fd2680d3bf0 2024-05-20T18:31:13.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.202+0000 7fd2677fe640 1 -- 172.21.15.5:0/2786892083 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd260007cb0 con 0x7fd2680d3bf0 2024-05-20T18:31:13.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.203+0000 7fd26f237640 1 -- 172.21.15.5:0/2786892083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2680d3bf0 msgr2=0x7fd2680d4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:13.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.203+0000 7fd26f237640 1 --2- 172.21.15.5:0/2786892083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2680d3bf0 0x7fd2680d4000 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd260009d90 tx=0x7fd26002a660 comp rx=0 tx=0).stop 2024-05-20T18:31:13.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.203+0000 7fd26f237640 1 -- 172.21.15.5:0/2786892083 shutdown_connections 2024-05-20T18:31:13.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.203+0000 7fd26f237640 1 --2- 172.21.15.5:0/2786892083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2680d3bf0 0x7fd2680d4000 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:13.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.203+0000 7fd26f237640 1 --2- 172.21.15.5:0/2786892083 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2680b6620 0x7fd2680ba3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:13.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.203+0000 7fd26f237640 1 -- 172.21.15.5:0/2786892083 >> 172.21.15.5:0/2786892083 conn(0x7fd26806d240 msgr2=0x7fd26806d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:13.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.203+0000 7fd26f237640 1 -- 172.21.15.5:0/2786892083 shutdown_connections 2024-05-20T18:31:13.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.203+0000 7fd26f237640 1 -- 172.21.15.5:0/2786892083 wait complete. 2024-05-20T18:31:13.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.205+0000 7fd26f237640 1 Processor -- start 2024-05-20T18:31:13.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.205+0000 7fd26f237640 1 -- start start 2024-05-20T18:31:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.206+0000 7fd26f237640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2680b6620 0x7fd2680b2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.207+0000 7fd26f237640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2680d3bf0 0x7fd268152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.207+0000 7fd26f237640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd268162ce0 0x7fd2680cfc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.207+0000 7fd26f237640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd2681630f0 con 0x7fd2680b6620 2024-05-20T18:31:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.207+0000 7fd26f237640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd2680d36d0 con 0x7fd2680d3bf0 2024-05-20T18:31:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.207+0000 7fd26f237640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd2680aec80 con 0x7fd268162ce0 2024-05-20T18:31:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.207+0000 7fd26cfd3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2680b6620 0x7fd2680b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.207+0000 7fd26cfd3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2680b6620 0x7fd2680b2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40212/0 (socket says 172.21.15.5:40212) 2024-05-20T18:31:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.207+0000 7fd26cfd3640 1 -- 172.21.15.5:0/2062059623 learned_addr learned my addr 172.21.15.5:0/2062059623 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.207+0000 7fd267fff640 1 --2- 172.21.15.5:0/2062059623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2680d3bf0 0x7fd268152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.207+0000 7fd26d7d4640 1 --2- 172.21.15.5:0/2062059623 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd268162ce0 0x7fd2680cfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.207+0000 7fd26cfd3640 1 -- 172.21.15.5:0/2062059623 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd268162ce0 msgr2=0x7fd2680cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.207+0000 7fd26cfd3640 1 --2- 172.21.15.5:0/2062059623 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd268162ce0 0x7fd2680cfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.207+0000 7fd26cfd3640 1 -- 172.21.15.5:0/2062059623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2680d3bf0 msgr2=0x7fd268152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.207+0000 7fd26cfd3640 1 --2- 172.21.15.5:0/2062059623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2680d3bf0 0x7fd268152f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.207+0000 7fd26cfd3640 1 -- 172.21.15.5:0/2062059623 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2680b29c0 con 0x7fd2680b6620 2024-05-20T18:31:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.207+0000 7fd26cfd3640 1 --2- 172.21.15.5:0/2062059623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2680b6620 0x7fd2680b2480 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fd25800d5f0 tx=0x7fd25800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:13.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.208+0000 7fd2657fa640 1 -- 172.21.15.5:0/2062059623 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2580125f0 con 0x7fd2680b6620 2024-05-20T18:31:13.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.208+0000 7fd2657fa640 1 -- 172.21.15.5:0/2062059623 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd258019070 con 0x7fd2680b6620 2024-05-20T18:31:13.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.208+0000 7fd26f237640 1 -- 172.21.15.5:0/2062059623 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2680b2e20 con 0x7fd2680b6620 2024-05-20T18:31:13.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.208+0000 7fd2657fa640 1 -- 172.21.15.5:0/2062059623 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2580129d0 con 0x7fd2680b6620 2024-05-20T18:31:13.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.208+0000 7fd26f237640 1 -- 172.21.15.5:0/2062059623 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2680b33e0 con 0x7fd2680b6620 2024-05-20T18:31:13.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.209+0000 7fd26f237640 1 -- 172.21.15.5:0/2062059623 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2680b35e0 con 0x7fd2680b6620 2024-05-20T18:31:13.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.210+0000 7fd2657fa640 1 -- 172.21.15.5:0/2062059623 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd258012b70 con 0x7fd2680b6620 2024-05-20T18:31:13.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.210+0000 7fd2657fa640 1 --2- 172.21.15.5:0/2062059623 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd25406be30 0x7fd25406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:13.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.211+0000 7fd2657fa640 1 -- 172.21.15.5:0/2062059623 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6246+0+0 (secure 0 0 0) 0x7fd25808b420 con 0x7fd2680b6620 2024-05-20T18:31:13.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.211+0000 7fd267fff640 1 --2- 172.21.15.5:0/2062059623 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd25406be30 0x7fd25406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:13.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.211+0000 7fd267fff640 1 --2- 172.21.15.5:0/2062059623 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd25406be30 0x7fd25406e340 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd268160590 tx=0x7fd25c007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:13.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.215+0000 7fd2657fa640 1 -- 172.21.15.5:0/2062059623 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd2680b35e0 con 0x7fd2680b6620 2024-05-20T18:31:13.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.399+0000 7fd26f237640 1 -- 172.21.15.5:0/2062059623 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd268075b00 con 0x7fd2680b6620 2024-05-20T18:31:13.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.400+0000 7fd2657fa640 1 -- 172.21.15.5:0/2062059623 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) ==== 100+0+11 (secure 0 0 0) 0x7fd2580594c0 con 0x7fd2680b6620 2024-05-20T18:31:13.400 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:31:13.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.402+0000 7fd26f237640 1 -- 172.21.15.5:0/2062059623 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd25406be30 msgr2=0x7fd25406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:13.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.402+0000 7fd26f237640 1 --2- 172.21.15.5:0/2062059623 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd25406be30 0x7fd25406e340 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd268160590 tx=0x7fd25c007de0 comp rx=0 tx=0).stop 2024-05-20T18:31:13.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.402+0000 7fd26f237640 1 -- 172.21.15.5:0/2062059623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2680b6620 msgr2=0x7fd2680b2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:13.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.402+0000 7fd26f237640 1 --2- 172.21.15.5:0/2062059623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2680b6620 0x7fd2680b2480 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fd25800d5f0 tx=0x7fd25800dab0 comp rx=0 tx=0).stop 2024-05-20T18:31:13.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.402+0000 7fd26f237640 1 -- 172.21.15.5:0/2062059623 shutdown_connections 2024-05-20T18:31:13.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.402+0000 7fd26f237640 1 --2- 172.21.15.5:0/2062059623 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd25406be30 0x7fd25406e340 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:13.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.402+0000 7fd26f237640 1 --2- 172.21.15.5:0/2062059623 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd268162ce0 0x7fd2680cfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:13.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.402+0000 7fd26f237640 1 --2- 172.21.15.5:0/2062059623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2680d3bf0 0x7fd268152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:13.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.402+0000 7fd26f237640 1 --2- 172.21.15.5:0/2062059623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2680b6620 0x7fd2680b2480 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:13.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.402+0000 7fd26f237640 1 -- 172.21.15.5:0/2062059623 >> 172.21.15.5:0/2062059623 conn(0x7fd26806d240 msgr2=0x7fd2680b8030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:13.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.402+0000 7fd26f237640 1 -- 172.21.15.5:0/2062059623 shutdown_connections 2024-05-20T18:31:13.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.402+0000 7fd26f237640 1 -- 172.21.15.5:0/2062059623 wait complete. 2024-05-20T18:31:13.415 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T18:31:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.531+0000 7f9053938640 1 Processor -- start 2024-05-20T18:31:13.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.531+0000 7f9053938640 1 -- start start 2024-05-20T18:31:13.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.533+0000 7f9053938640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f904c0cd680 0x7f904c15f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:13.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.533+0000 7f9053938640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f904c06c6c0 0x7f904c06cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:13.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.533+0000 7f9053938640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f904c06d0f0 con 0x7f904c0cf3d0 2024-05-20T18:31:13.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.533+0000 7f9053938640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f904c0cda90 con 0x7f904c0cd680 2024-05-20T18:31:13.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.533+0000 7f9053938640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f904c15f760 con 0x7f904c06c6c0 2024-05-20T18:31:13.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.533+0000 7f90516d4640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f904c0cd680 0x7f904c15f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:13.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.533+0000 7f90516d4640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f904c0cd680 0x7f904c15f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40216/0 (socket says 172.21.15.5:40216) 2024-05-20T18:31:13.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.533+0000 7f90516d4640 1 -- 172.21.15.5:0/2780890730 learned_addr learned my addr 172.21.15.5:0/2780890730 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:13.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.533+0000 7f9050ed3640 1 --2- 172.21.15.5:0/2780890730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f904c06c6c0 0x7f904c06cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:13.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.533+0000 7f90516d4640 1 -- 172.21.15.5:0/2780890730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f904c06c6c0 msgr2=0x7f904c06cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:13.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.533+0000 7f90516d4640 1 --2- 172.21.15.5:0/2780890730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f904c06c6c0 0x7f904c06cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:13.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.533+0000 7f90516d4640 1 -- 172.21.15.5:0/2780890730 >> v1:172.21.15.63:6789/0 conn(0x7f904c0cf3d0 legacy=0x7f904c0cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:13.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.533+0000 7f90516d4640 1 -- 172.21.15.5:0/2780890730 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904c06d3d0 con 0x7f904c0cd680 2024-05-20T18:31:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.533+0000 7f9050ed3640 1 --2- 172.21.15.5:0/2780890730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f904c06c6c0 0x7f904c06cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:13.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.534+0000 7f90516d4640 1 --2- 172.21.15.5:0/2780890730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f904c0cd680 0x7f904c15f220 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9048009870 tx=0x7f904802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e84ebb1573ce8c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:13.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.534+0000 7f9043fff640 1 -- 172.21.15.5:0/2780890730 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9048037070 con 0x7f904c0cd680 2024-05-20T18:31:13.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.534+0000 7f9043fff640 1 -- 172.21.15.5:0/2780890730 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9048032440 con 0x7f904c0cd680 2024-05-20T18:31:13.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.534+0000 7f9043fff640 1 -- 172.21.15.5:0/2780890730 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9048032810 con 0x7f904c0cd680 2024-05-20T18:31:13.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.535+0000 7f9053938640 1 -- 172.21.15.5:0/2780890730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f904c0cd680 msgr2=0x7f904c15f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:13.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.535+0000 7f9053938640 1 --2- 172.21.15.5:0/2780890730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f904c0cd680 0x7f904c15f220 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9048009870 tx=0x7f904802a1c0 comp rx=0 tx=0).stop 2024-05-20T18:31:13.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.535+0000 7f9053938640 1 -- 172.21.15.5:0/2780890730 shutdown_connections 2024-05-20T18:31:13.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.535+0000 7f9053938640 1 --2- 172.21.15.5:0/2780890730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f904c06c6c0 0x7f904c06cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:13.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.535+0000 7f9053938640 1 --2- 172.21.15.5:0/2780890730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f904c0cd680 0x7f904c15f220 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:13.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.535+0000 7f9053938640 1 -- 172.21.15.5:0/2780890730 >> 172.21.15.5:0/2780890730 conn(0x7f904c0b44d0 msgr2=0x7f904c0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:13.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.535+0000 7f9053938640 1 -- 172.21.15.5:0/2780890730 shutdown_connections 2024-05-20T18:31:13.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.535+0000 7f9053938640 1 -- 172.21.15.5:0/2780890730 wait complete. 2024-05-20T18:31:13.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.537+0000 7f9053938640 1 Processor -- start 2024-05-20T18:31:13.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.537+0000 7f9053938640 1 -- start start 2024-05-20T18:31:13.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.538+0000 7f9053938640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f904c06c6c0 0x7f904c15aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:13.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.538+0000 7f9053938640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f904c0cd680 0x7f904c151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:13.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.538+0000 7f9053938640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f904c0cf3d0 0x7f904c15de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:13.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.538+0000 7f9053938640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f904c06d990 con 0x7f904c0cf3d0 2024-05-20T18:31:13.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.538+0000 7f9053938640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f904c0d0b40 con 0x7f904c06c6c0 2024-05-20T18:31:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.538+0000 7f9053938640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f904c0d2460 con 0x7f904c0cd680 2024-05-20T18:31:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.538+0000 7f9051ed5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f904c0cf3d0 0x7f904c15de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.538+0000 7f9051ed5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f904c0cf3d0 0x7f904c15de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40226/0 (socket says 172.21.15.5:40226) 2024-05-20T18:31:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.538+0000 7f9051ed5640 1 -- 172.21.15.5:0/1665807399 learned_addr learned my addr 172.21.15.5:0/1665807399 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.538+0000 7f9050ed3640 1 --2- 172.21.15.5:0/1665807399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f904c0cd680 0x7f904c151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.538+0000 7f90516d4640 1 --2- 172.21.15.5:0/1665807399 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f904c06c6c0 0x7f904c15aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.539+0000 7f9051ed5640 1 -- 172.21.15.5:0/1665807399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f904c0cd680 msgr2=0x7f904c151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.539+0000 7f9051ed5640 1 --2- 172.21.15.5:0/1665807399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f904c0cd680 0x7f904c151c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.539+0000 7f9051ed5640 1 -- 172.21.15.5:0/1665807399 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f904c06c6c0 msgr2=0x7f904c15aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.539+0000 7f9051ed5640 1 --2- 172.21.15.5:0/1665807399 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f904c06c6c0 0x7f904c15aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.539+0000 7f9051ed5640 1 -- 172.21.15.5:0/1665807399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904c0b6580 con 0x7f904c0cf3d0 2024-05-20T18:31:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.539+0000 7f9051ed5640 1 --2- 172.21.15.5:0/1665807399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f904c0cf3d0 0x7f904c15de00 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f904400c6e0 tx=0x7f904400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.539+0000 7f9041ffb640 1 -- 172.21.15.5:0/1665807399 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044003b70 con 0x7f904c0cf3d0 2024-05-20T18:31:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.539+0000 7f9041ffb640 1 -- 172.21.15.5:0/1665807399 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9044003d30 con 0x7f904c0cf3d0 2024-05-20T18:31:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.539+0000 7f9041ffb640 1 -- 172.21.15.5:0/1665807399 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044005310 con 0x7f904c0cf3d0 2024-05-20T18:31:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.539+0000 7f9053938640 1 -- 172.21.15.5:0/1665807399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f904c0b6860 con 0x7f904c0cf3d0 2024-05-20T18:31:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.539+0000 7f9053938640 1 -- 172.21.15.5:0/1665807399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f904c0b6e20 con 0x7f904c0cf3d0 2024-05-20T18:31:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.540+0000 7f9053938640 1 -- 172.21.15.5:0/1665807399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f904c0b62f0 con 0x7f904c0cf3d0 2024-05-20T18:31:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.541+0000 7f9041ffb640 1 -- 172.21.15.5:0/1665807399 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f90440054b0 con 0x7f904c0cf3d0 2024-05-20T18:31:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.541+0000 7f9041ffb640 1 --2- 172.21.15.5:0/1665807399 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f903006be60 0x7f903006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.541+0000 7f9041ffb640 1 -- 172.21.15.5:0/1665807399 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6246+0+0 (secure 0 0 0) 0x7f904408ae10 con 0x7f904c0cf3d0 2024-05-20T18:31:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.541+0000 7f90516d4640 1 --2- 172.21.15.5:0/1665807399 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f903006be60 0x7f903006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.542+0000 7f90516d4640 1 --2- 172.21.15.5:0/1665807399 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f903006be60 0x7f903006e370 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9048034000 tx=0x7f904802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:13.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.546+0000 7f9041ffb640 1 -- 172.21.15.5:0/1665807399 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9044058ed0 con 0x7f904c0cf3d0 2024-05-20T18:31:13.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.742+0000 7f9053938640 1 -- 172.21.15.5:0/1665807399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f904c075b00 con 0x7f904c0cf3d0 2024-05-20T18:31:13.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.743+0000 7f9041ffb640 1 -- 172.21.15.5:0/1665807399 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v65) ==== 80+0+0 (secure 0 0 0) 0x7f904405d1d0 con 0x7f904c0cf3d0 2024-05-20T18:31:13.743 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-05-20T18:31:13.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.745+0000 7f9053938640 1 -- 172.21.15.5:0/1665807399 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f903006be60 msgr2=0x7f903006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:13.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.745+0000 7f9053938640 1 --2- 172.21.15.5:0/1665807399 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f903006be60 0x7f903006e370 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9048034000 tx=0x7f904802e000 comp rx=0 tx=0).stop 2024-05-20T18:31:13.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.745+0000 7f9053938640 1 -- 172.21.15.5:0/1665807399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f904c0cf3d0 msgr2=0x7f904c15de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:13.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.745+0000 7f9053938640 1 --2- 172.21.15.5:0/1665807399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f904c0cf3d0 0x7f904c15de00 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f904400c6e0 tx=0x7f904400cba0 comp rx=0 tx=0).stop 2024-05-20T18:31:13.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.745+0000 7f9053938640 1 -- 172.21.15.5:0/1665807399 shutdown_connections 2024-05-20T18:31:13.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.745+0000 7f9053938640 1 --2- 172.21.15.5:0/1665807399 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f903006be60 0x7f903006e370 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:13.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.745+0000 7f9053938640 1 --2- 172.21.15.5:0/1665807399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f904c0cf3d0 0x7f904c15de00 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:13.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.745+0000 7f9053938640 1 --2- 172.21.15.5:0/1665807399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f904c0cd680 0x7f904c151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:13.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.745+0000 7f9053938640 1 --2- 172.21.15.5:0/1665807399 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f904c06c6c0 0x7f904c15aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:13.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.745+0000 7f9053938640 1 -- 172.21.15.5:0/1665807399 >> 172.21.15.5:0/1665807399 conn(0x7f904c0b44d0 msgr2=0x7f904c0cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:13.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.745+0000 7f9053938640 1 -- 172.21.15.5:0/1665807399 shutdown_connections 2024-05-20T18:31:13.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:13.745+0000 7f9053938640 1 -- 172.21.15.5:0/1665807399 wait complete. 2024-05-20T18:31:13.757 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-20T18:31:13.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:31:13.757 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-20T18:31:13.808 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:31:13.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:31:13.809 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-20T18:31:13.832 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T18:31:13.832 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:31:13.833 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:31:13.858 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19732 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T18:31:13.858 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T18:31:13.859 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:31:13.859 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:31:13.890 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15663 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T18:31:13.890 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T18:31:13.890 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:31:13.890 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:31:13.918 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19732 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T18:31:13.918 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:31:14.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.044+0000 7efe2ceca640 1 Processor -- start 2024-05-20T18:31:14.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.044+0000 7efe2ceca640 1 -- start start 2024-05-20T18:31:14.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.045+0000 7efe2ceca640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7efe28162820 0x7efe280b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:14.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.045+0000 7efe2ceca640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efe2806dae0 0x7efe280b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:14.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.045+0000 7efe2ceca640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe280d0630 con 0x7efe280b3e80 2024-05-20T18:31:14.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.045+0000 7efe2ceca640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7efe280d39c0 con 0x7efe2806dae0 2024-05-20T18:31:14.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.045+0000 7efe2ceca640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7efe280d3ca0 con 0x7efe28162820 2024-05-20T18:31:14.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.045+0000 7efe26d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efe2806dae0 0x7efe280b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:14.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.046+0000 7efe26d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efe2806dae0 0x7efe280b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40236/0 (socket says 172.21.15.5:40236) 2024-05-20T18:31:14.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.046+0000 7efe26d9d640 1 -- 172.21.15.5:0/1774182108 learned_addr learned my addr 172.21.15.5:0/1774182108 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:14.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.046+0000 7efe2659c640 1 --2- 172.21.15.5:0/1774182108 >> v2:172.21.15.63:3300/0 conn(0x7efe28162820 0x7efe280b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:14.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.046+0000 7efe26d9d640 1 -- 172.21.15.5:0/1774182108 >> v2:172.21.15.63:3300/0 conn(0x7efe28162820 msgr2=0x7efe280b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:14.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.046+0000 7efe26d9d640 1 --2- 172.21.15.5:0/1774182108 >> v2:172.21.15.63:3300/0 conn(0x7efe28162820 0x7efe280b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:14.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.046+0000 7efe26d9d640 1 -- 172.21.15.5:0/1774182108 >> v1:172.21.15.63:6789/0 conn(0x7efe280b3e80 legacy=0x7efe2806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:14.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.046+0000 7efe26d9d640 1 -- 172.21.15.5:0/1774182108 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe280d3f80 con 0x7efe2806dae0 2024-05-20T18:31:14.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.047+0000 7efe26d9d640 1 --2- 172.21.15.5:0/1774182108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe2806dae0 0x7efe280b8190 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7efe280b7740 tx=0x7efe2002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1969373eafaf90ee server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:14.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.047+0000 7efe25d9b640 1 -- 172.21.15.5:0/1774182108 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe20036070 con 0x7efe2806dae0 2024-05-20T18:31:14.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.047+0000 7efe25d9b640 1 -- 172.21.15.5:0/1774182108 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe20031600 con 0x7efe2806dae0 2024-05-20T18:31:14.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.047+0000 7efe25d9b640 1 -- 172.21.15.5:0/1774182108 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe200319d0 con 0x7efe2806dae0 2024-05-20T18:31:14.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.047+0000 7efe2ceca640 1 -- 172.21.15.5:0/1774182108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe2806dae0 msgr2=0x7efe280b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:14.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.047+0000 7efe2ceca640 1 --2- 172.21.15.5:0/1774182108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe2806dae0 0x7efe280b8190 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7efe280b7740 tx=0x7efe2002a730 comp rx=0 tx=0).stop 2024-05-20T18:31:14.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.048+0000 7efe2ceca640 1 -- 172.21.15.5:0/1774182108 shutdown_connections 2024-05-20T18:31:14.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.048+0000 7efe2ceca640 1 --2- 172.21.15.5:0/1774182108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe2806dae0 0x7efe280b8190 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:14.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.048+0000 7efe2ceca640 1 --2- 172.21.15.5:0/1774182108 >> v2:172.21.15.63:3300/0 conn(0x7efe28162820 0x7efe280b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:14.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.048+0000 7efe2ceca640 1 -- 172.21.15.5:0/1774182108 >> 172.21.15.5:0/1774182108 conn(0x7efe280d4bd0 msgr2=0x7efe280d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:14.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.048+0000 7efe2ceca640 1 -- 172.21.15.5:0/1774182108 shutdown_connections 2024-05-20T18:31:14.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.048+0000 7efe2ceca640 1 -- 172.21.15.5:0/1774182108 wait complete. 2024-05-20T18:31:14.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.050+0000 7efe2ceca640 1 Processor -- start 2024-05-20T18:31:14.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.050+0000 7efe2ceca640 1 -- start start 2024-05-20T18:31:14.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.051+0000 7efe2ceca640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe280b3e80 0x7efe2814ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:14.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.051+0000 7efe2ceca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe28162820 0x7efe28150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:14.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.051+0000 7efe2ceca640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe2814eda0 0x7efe2814f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:14.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.051+0000 7efe2ceca640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efe280d4780 con 0x7efe28162820 2024-05-20T18:31:14.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.051+0000 7efe2ceca640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efe280d0300 con 0x7efe2814eda0 2024-05-20T18:31:14.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.051+0000 7efe2ceca640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efe280d4900 con 0x7efe280b3e80 2024-05-20T18:31:14.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.051+0000 7efe1dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe28162820 0x7efe28150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:14.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.051+0000 7efe1dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe28162820 0x7efe28150480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40248/0 (socket says 172.21.15.5:40248) 2024-05-20T18:31:14.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.051+0000 7efe1dd9b640 1 -- 172.21.15.5:0/3848183363 learned_addr learned my addr 172.21.15.5:0/3848183363 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:14.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.051+0000 7efe26d9d640 1 --2- 172.21.15.5:0/3848183363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe2814eda0 0x7efe2814f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:14.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.051+0000 7efe2659c640 1 --2- 172.21.15.5:0/3848183363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe280b3e80 0x7efe2814ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:14.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.052+0000 7efe1dd9b640 1 -- 172.21.15.5:0/3848183363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe280b3e80 msgr2=0x7efe2814ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:14.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.052+0000 7efe1dd9b640 1 --2- 172.21.15.5:0/3848183363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe280b3e80 0x7efe2814ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:14.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.052+0000 7efe1dd9b640 1 -- 172.21.15.5:0/3848183363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe2814eda0 msgr2=0x7efe2814f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:14.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.052+0000 7efe1dd9b640 1 --2- 172.21.15.5:0/3848183363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe2814eda0 0x7efe2814f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:14.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.052+0000 7efe1dd9b640 1 -- 172.21.15.5:0/3848183363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe280cb5d0 con 0x7efe28162820 2024-05-20T18:31:14.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.052+0000 7efe1dd9b640 1 --2- 172.21.15.5:0/3848183363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe28162820 0x7efe28150480 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7efe1000c3f0 tx=0x7efe1000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:14.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.052+0000 7efe1f7fe640 1 -- 172.21.15.5:0/3848183363 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe100052d0 con 0x7efe28162820 2024-05-20T18:31:14.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.052+0000 7efe1f7fe640 1 -- 172.21.15.5:0/3848183363 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe10005490 con 0x7efe28162820 2024-05-20T18:31:14.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.052+0000 7efe2ceca640 1 -- 172.21.15.5:0/3848183363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe2814dfe0 con 0x7efe28162820 2024-05-20T18:31:14.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.052+0000 7efe1f7fe640 1 -- 172.21.15.5:0/3848183363 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe10007260 con 0x7efe28162820 2024-05-20T18:31:14.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.052+0000 7efe2ceca640 1 -- 172.21.15.5:0/3848183363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe2814e5a0 con 0x7efe28162820 2024-05-20T18:31:14.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.053+0000 7efe2ceca640 1 -- 172.21.15.5:0/3848183363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe280caf30 con 0x7efe28162820 2024-05-20T18:31:14.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.054+0000 7efe1f7fe640 1 -- 172.21.15.5:0/3848183363 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efe10007400 con 0x7efe28162820 2024-05-20T18:31:14.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.054+0000 7efe1f7fe640 1 --2- 172.21.15.5:0/3848183363 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efe0806be30 0x7efe0806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:14.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.054+0000 7efe1f7fe640 1 -- 172.21.15.5:0/3848183363 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6246+0+0 (secure 0 0 0) 0x7efe1008a430 con 0x7efe28162820 2024-05-20T18:31:14.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.054+0000 7efe2659c640 1 --2- 172.21.15.5:0/3848183363 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efe0806be30 0x7efe0806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:14.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.055+0000 7efe2659c640 1 --2- 172.21.15.5:0/3848183363 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efe0806be30 0x7efe0806e340 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efe18009aa0 tx=0x7efe18009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:14.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.059+0000 7efe1f7fe640 1 -- 172.21.15.5:0/3848183363 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7efe10058600 con 0x7efe28162820 2024-05-20T18:31:14.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.278+0000 7efe2ceca640 1 -- 172.21.15.5:0/3848183363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe28075b00 con 0x7efe28162820 2024-05-20T18:31:14.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.279+0000 7efe1f7fe640 1 -- 172.21.15.5:0/3848183363 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5177 (secure 0 0 0) 0x7efe1005c900 con 0x7efe28162820 2024-05-20T18:31:14.282 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:31:14.282 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5480,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/4292435960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":4292435960},{"type":"v1","addr":"172.21.15.63:6835","nonce":4292435960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6186,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2991834087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2991834087},{"type":"v1","addr":"172.21.15.5:6836","nonce":2991834087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6195,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3462206198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":3462206198},{"type":"v1","addr":"172.21.15.5:6837","nonce":3462206198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:30:00.286575+0000","modified":"2024-05-20T18:30:48.943731+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6198":{"gid":6198,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":15,"addr":"172.21.15.63:6834/4036379300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":4036379300},{"type":"v1","addr":"172.21.15.63:6834","nonce":4036379300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6198,"qdb_cluster":[6198]},"id":4}]} 2024-05-20T18:31:14.282 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 36 2024-05-20T18:31:14.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.281+0000 7efe2ceca640 1 -- 172.21.15.5:0/3848183363 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efe0806be30 msgr2=0x7efe0806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:14.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.281+0000 7efe2ceca640 1 --2- 172.21.15.5:0/3848183363 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efe0806be30 0x7efe0806e340 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efe18009aa0 tx=0x7efe18009190 comp rx=0 tx=0).stop 2024-05-20T18:31:14.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.281+0000 7efe2ceca640 1 -- 172.21.15.5:0/3848183363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe28162820 msgr2=0x7efe28150480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:14.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.281+0000 7efe2ceca640 1 --2- 172.21.15.5:0/3848183363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe28162820 0x7efe28150480 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7efe1000c3f0 tx=0x7efe1000c8b0 comp rx=0 tx=0).stop 2024-05-20T18:31:14.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.281+0000 7efe2ceca640 1 -- 172.21.15.5:0/3848183363 shutdown_connections 2024-05-20T18:31:14.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.281+0000 7efe2ceca640 1 --2- 172.21.15.5:0/3848183363 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efe0806be30 0x7efe0806e340 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:14.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.281+0000 7efe2ceca640 1 --2- 172.21.15.5:0/3848183363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe2814eda0 0x7efe2814f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:14.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.281+0000 7efe2ceca640 1 --2- 172.21.15.5:0/3848183363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe28162820 0x7efe28150480 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:14.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.281+0000 7efe2ceca640 1 --2- 172.21.15.5:0/3848183363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe280b3e80 0x7efe2814ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:14.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.281+0000 7efe2ceca640 1 -- 172.21.15.5:0/3848183363 >> 172.21.15.5:0/3848183363 conn(0x7efe280d4bd0 msgr2=0x7efe280b87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:14.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.281+0000 7efe2ceca640 1 -- 172.21.15.5:0/3848183363 shutdown_connections 2024-05-20T18:31:14.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.281+0000 7efe2ceca640 1 -- 172.21.15.5:0/3848183363 wait complete. 2024-05-20T18:31:14.293 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:31:14.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.409+0000 7f7d5eb9d640 1 Processor -- start 2024-05-20T18:31:14.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.409+0000 7f7d5eb9d640 1 -- start start 2024-05-20T18:31:14.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.410+0000 7f7d5eb9d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d580b6c70 0x7f7d580d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:14.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.410+0000 7f7d5eb9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7d580d2060 0x7f7d580cd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:14.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.410+0000 7f7d5eb9d640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d5815f690 con 0x7f7d580ae650 2024-05-20T18:31:14.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.410+0000 7f7d5eb9d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f7d580ccb70 con 0x7f7d580d2060 2024-05-20T18:31:14.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.410+0000 7f7d5eb9d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7d580cbf60 con 0x7f7d580b6c70 2024-05-20T18:31:14.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.411+0000 7f7d5e39c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7d580d2060 0x7f7d580cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:14.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.411+0000 7f7d5e39c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7d580d2060 0x7f7d580cd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40258/0 (socket says 172.21.15.5:40258) 2024-05-20T18:31:14.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.411+0000 7f7d5e39c640 1 -- 172.21.15.5:0/1957413563 learned_addr learned my addr 172.21.15.5:0/1957413563 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:14.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.411+0000 7f7d5d39a640 1 --2- 172.21.15.5:0/1957413563 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d580b6c70 0x7f7d580d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:14.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.411+0000 7f7d5e39c640 1 -- 172.21.15.5:0/1957413563 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d580b6c70 msgr2=0x7f7d580d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:14.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.411+0000 7f7d5e39c640 1 --2- 172.21.15.5:0/1957413563 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d580b6c70 0x7f7d580d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:14.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.411+0000 7f7d5e39c640 1 -- 172.21.15.5:0/1957413563 >> v1:172.21.15.63:6789/0 conn(0x7f7d580ae650 legacy=0x7f7d580b6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:14.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.411+0000 7f7d5e39c640 1 -- 172.21.15.5:0/1957413563 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d580cc240 con 0x7f7d580d2060 2024-05-20T18:31:14.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.412+0000 7f7d5e39c640 1 --2- 172.21.15.5:0/1957413563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d580d2060 0x7f7d580cd000 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009e50 tx=0x7f7d5002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba3473ceaa6f3307 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:14.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.412+0000 7f7d5cb99640 1 -- 172.21.15.5:0/1957413563 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d50036070 con 0x7f7d580d2060 2024-05-20T18:31:14.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.412+0000 7f7d5cb99640 1 -- 172.21.15.5:0/1957413563 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d50031600 con 0x7f7d580d2060 2024-05-20T18:31:14.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.412+0000 7f7d5cb99640 1 -- 172.21.15.5:0/1957413563 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d500319d0 con 0x7f7d580d2060 2024-05-20T18:31:14.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.413+0000 7f7d5eb9d640 1 -- 172.21.15.5:0/1957413563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d580d2060 msgr2=0x7f7d580cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:14.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.413+0000 7f7d5eb9d640 1 --2- 172.21.15.5:0/1957413563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d580d2060 0x7f7d580cd000 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009e50 tx=0x7f7d5002a680 comp rx=0 tx=0).stop 2024-05-20T18:31:14.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.413+0000 7f7d5eb9d640 1 -- 172.21.15.5:0/1957413563 shutdown_connections 2024-05-20T18:31:14.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.413+0000 7f7d5eb9d640 1 --2- 172.21.15.5:0/1957413563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d580d2060 0x7f7d580cd000 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:14.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.413+0000 7f7d5eb9d640 1 --2- 172.21.15.5:0/1957413563 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d580b6c70 0x7f7d580d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:14.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.413+0000 7f7d5eb9d640 1 -- 172.21.15.5:0/1957413563 >> 172.21.15.5:0/1957413563 conn(0x7f7d5809a4f0 msgr2=0x7f7d5809a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:14.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.413+0000 7f7d5eb9d640 1 -- 172.21.15.5:0/1957413563 shutdown_connections 2024-05-20T18:31:14.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.413+0000 7f7d5eb9d640 1 -- 172.21.15.5:0/1957413563 wait complete. 2024-05-20T18:31:14.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.414+0000 7f7d5eb9d640 1 Processor -- start 2024-05-20T18:31:14.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.415+0000 7f7d5eb9d640 1 -- start start 2024-05-20T18:31:14.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.416+0000 7f7d5eb9d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d580ae650 0x7f7d580caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:14.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.416+0000 7f7d5eb9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d580d2060 0x7f7d58153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:14.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.416+0000 7f7d5eb9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d5806c570 0x7f7d5806c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:14.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.416+0000 7f7d5eb9d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7d580d2470 con 0x7f7d5806c570 2024-05-20T18:31:14.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.416+0000 7f7d5eb9d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7d580cc800 con 0x7f7d580d2060 2024-05-20T18:31:14.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.416+0000 7f7d5eb9d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7d581f75d0 con 0x7f7d580ae650 2024-05-20T18:31:14.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.416+0000 7f7d5e39c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d5806c570 0x7f7d5806c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:14.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.416+0000 7f7d5e39c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d5806c570 0x7f7d5806c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40272/0 (socket says 172.21.15.5:40272) 2024-05-20T18:31:14.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.416+0000 7f7d5e39c640 1 -- 172.21.15.5:0/1444670241 learned_addr learned my addr 172.21.15.5:0/1444670241 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:14.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.416+0000 7f7d5db9b640 1 --2- 172.21.15.5:0/1444670241 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d580ae650 0x7f7d580caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:14.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.416+0000 7f7d5d39a640 1 --2- 172.21.15.5:0/1444670241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d580d2060 0x7f7d58153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:14.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.416+0000 7f7d5e39c640 1 -- 172.21.15.5:0/1444670241 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d580ae650 msgr2=0x7f7d580caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:14.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.416+0000 7f7d5e39c640 1 --2- 172.21.15.5:0/1444670241 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d580ae650 0x7f7d580caf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:14.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.416+0000 7f7d5e39c640 1 -- 172.21.15.5:0/1444670241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d580d2060 msgr2=0x7f7d58153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:14.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.416+0000 7f7d5e39c640 1 --2- 172.21.15.5:0/1444670241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d580d2060 0x7f7d58153d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:14.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.416+0000 7f7d5e39c640 1 -- 172.21.15.5:0/1444670241 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d58154250 con 0x7f7d5806c570 2024-05-20T18:31:14.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.416+0000 7f7d5e39c640 1 --2- 172.21.15.5:0/1444670241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d5806c570 0x7f7d5806c980 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009f70 tx=0x7f7d50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:14.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.417+0000 7f7d4e7fc640 1 -- 172.21.15.5:0/1444670241 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5003f020 con 0x7f7d5806c570 2024-05-20T18:31:14.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.417+0000 7f7d4e7fc640 1 -- 172.21.15.5:0/1444670241 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d50037040 con 0x7f7d5806c570 2024-05-20T18:31:14.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.417+0000 7f7d5eb9d640 1 -- 172.21.15.5:0/1444670241 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d5806cec0 con 0x7f7d5806c570 2024-05-20T18:31:14.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.417+0000 7f7d4e7fc640 1 -- 172.21.15.5:0/1444670241 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d50036070 con 0x7f7d5806c570 2024-05-20T18:31:14.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.417+0000 7f7d5eb9d640 1 -- 172.21.15.5:0/1444670241 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d581f8a40 con 0x7f7d5806c570 2024-05-20T18:31:14.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.418+0000 7f7d5eb9d640 1 -- 172.21.15.5:0/1444670241 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d58075980 con 0x7f7d5806c570 2024-05-20T18:31:14.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.418+0000 7f7d4e7fc640 1 -- 172.21.15.5:0/1444670241 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7d50042020 con 0x7f7d5806c570 2024-05-20T18:31:14.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.419+0000 7f7d4e7fc640 1 --2- 172.21.15.5:0/1444670241 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7d3c06be60 0x7f7d3c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:14.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.419+0000 7f7d4e7fc640 1 -- 172.21.15.5:0/1444670241 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6246+0+0 (secure 0 0 0) 0x7f7d500ab5f0 con 0x7f7d5806c570 2024-05-20T18:31:14.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.419+0000 7f7d5db9b640 1 --2- 172.21.15.5:0/1444670241 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7d3c06be60 0x7f7d3c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:14.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.419+0000 7f7d5db9b640 1 --2- 172.21.15.5:0/1444670241 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7d3c06be60 0x7f7d3c06e370 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7d54007fd0 tx=0x7f7d54009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:14.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.423+0000 7f7d4e7fc640 1 -- 172.21.15.5:0/1444670241 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7d50049030 con 0x7f7d5806c570 2024-05-20T18:31:14.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.609+0000 7f7d5eb9d640 1 -- 172.21.15.5:0/1444670241 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d5809ab30 con 0x7f7d5806c570 2024-05-20T18:31:14.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.610+0000 7f7d4e7fc640 1 -- 172.21.15.5:0/1444670241 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+17004 (secure 0 0 0) 0x7f7d5806cec0 con 0x7f7d5806c570 2024-05-20T18:31:14.611 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:31:14.611 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":65,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:30:57.074551+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:29:57.993861+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-20T18:29:59.267582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.18","mappings":[{"from":7,"to":2}]},{"pgid":"10.c","mappings":[{"from":4,"to":5}]},{"pgid":"10.1d","mappings":[{"from":4,"to":3}]},{"pgid":"10.33","mappings":[{"from":4,"to":3}]},{"pgid":"10.37","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:31:14.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.612+0000 7f7d5eb9d640 1 -- 172.21.15.5:0/1444670241 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7d3c06be60 msgr2=0x7f7d3c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.612+0000 7f7d5eb9d640 1 --2- 172.21.15.5:0/1444670241 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7d3c06be60 0x7f7d3c06e370 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7d54007fd0 tx=0x7f7d54009040 comp rx=0 tx=0).stop 2024-05-20T18:31:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.612+0000 7f7d5eb9d640 1 -- 172.21.15.5:0/1444670241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d5806c570 msgr2=0x7f7d5806c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.612+0000 7f7d5eb9d640 1 --2- 172.21.15.5:0/1444670241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d5806c570 0x7f7d5806c980 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009f70 tx=0x7f7d50004a40 comp rx=0 tx=0).stop 2024-05-20T18:31:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.613+0000 7f7d5eb9d640 1 -- 172.21.15.5:0/1444670241 shutdown_connections 2024-05-20T18:31:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.613+0000 7f7d5eb9d640 1 --2- 172.21.15.5:0/1444670241 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7d3c06be60 0x7f7d3c06e370 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.613+0000 7f7d5eb9d640 1 --2- 172.21.15.5:0/1444670241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d5806c570 0x7f7d5806c980 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.613+0000 7f7d5eb9d640 1 --2- 172.21.15.5:0/1444670241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d580d2060 0x7f7d58153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.613+0000 7f7d5eb9d640 1 --2- 172.21.15.5:0/1444670241 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d580ae650 0x7f7d580caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.613+0000 7f7d5eb9d640 1 -- 172.21.15.5:0/1444670241 >> 172.21.15.5:0/1444670241 conn(0x7f7d5809a4f0 msgr2=0x7f7d5815ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.613+0000 7f7d5eb9d640 1 -- 172.21.15.5:0/1444670241 shutdown_connections 2024-05-20T18:31:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.613+0000 7f7d5eb9d640 1 -- 172.21.15.5:0/1444670241 wait complete. 2024-05-20T18:31:14.624 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:29:57.993861+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:29:59.267582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:31:14.625 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:31:14.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.743+0000 7f9c6878b640 1 Processor -- start 2024-05-20T18:31:14.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.743+0000 7f9c6878b640 1 -- start start 2024-05-20T18:31:14.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.744+0000 7f9c6878b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c600b1e10 0x7f9c6015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:14.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.744+0000 7f9c6878b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9c60070850 0x7f9c60070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:14.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.744+0000 7f9c6878b640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c600cd580 con 0x7f9c6015d530 2024-05-20T18:31:14.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.744+0000 7f9c6878b640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f9c600ce190 con 0x7f9c60070850 2024-05-20T18:31:14.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.744+0000 7f9c6878b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9c600cd860 con 0x7f9c600b1e10 2024-05-20T18:31:14.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.744+0000 7f9c66d28640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9c60070850 0x7f9c60070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:14.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.744+0000 7f9c66d28640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9c60070850 0x7f9c60070c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37424/0 (socket says 172.21.15.5:37424) 2024-05-20T18:31:14.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.744+0000 7f9c66d28640 1 -- 172.21.15.5:0/1159119040 learned_addr learned my addr 172.21.15.5:0/1159119040 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:14.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.744+0000 7f9c65d26640 1 --2- 172.21.15.5:0/1159119040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c600b1e10 0x7f9c6015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:14.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.745+0000 7f9c66d28640 1 -- 172.21.15.5:0/1159119040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c600b1e10 msgr2=0x7f9c6015aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:14.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.745+0000 7f9c66d28640 1 --2- 172.21.15.5:0/1159119040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c600b1e10 0x7f9c6015aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:14.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.745+0000 7f9c66d28640 1 -- 172.21.15.5:0/1159119040 >> v1:172.21.15.63:6789/0 conn(0x7f9c6015d530 legacy=0x7f9c600b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:14.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.745+0000 7f9c66d28640 1 -- 172.21.15.5:0/1159119040 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c600cdb40 con 0x7f9c60070850 2024-05-20T18:31:14.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.745+0000 7f9c66d28640 1 --2- 172.21.15.5:0/1159119040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c60070850 0x7f9c60070c60 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f9c58009870 tx=0x7f9c5802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5326d206e6b3a6ea server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:14.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.746+0000 7f9c65525640 1 -- 172.21.15.5:0/1159119040 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c58037070 con 0x7f9c60070850 2024-05-20T18:31:14.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.746+0000 7f9c65525640 1 -- 172.21.15.5:0/1159119040 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c58032440 con 0x7f9c60070850 2024-05-20T18:31:14.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.746+0000 7f9c65525640 1 -- 172.21.15.5:0/1159119040 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c58032810 con 0x7f9c60070850 2024-05-20T18:31:14.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.746+0000 7f9c6878b640 1 -- 172.21.15.5:0/1159119040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c60070850 msgr2=0x7f9c60070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:14.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.746+0000 7f9c6878b640 1 --2- 172.21.15.5:0/1159119040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c60070850 0x7f9c60070c60 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f9c58009870 tx=0x7f9c5802a270 comp rx=0 tx=0).stop 2024-05-20T18:31:14.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.746+0000 7f9c6878b640 1 -- 172.21.15.5:0/1159119040 shutdown_connections 2024-05-20T18:31:14.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.746+0000 7f9c6878b640 1 --2- 172.21.15.5:0/1159119040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c60070850 0x7f9c60070c60 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:14.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.746+0000 7f9c6878b640 1 --2- 172.21.15.5:0/1159119040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c600b1e10 0x7f9c6015aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:14.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.746+0000 7f9c6878b640 1 -- 172.21.15.5:0/1159119040 >> 172.21.15.5:0/1159119040 conn(0x7f9c600d29d0 msgr2=0x7f9c600d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:14.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.746+0000 7f9c6878b640 1 -- 172.21.15.5:0/1159119040 shutdown_connections 2024-05-20T18:31:14.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.746+0000 7f9c6878b640 1 -- 172.21.15.5:0/1159119040 wait complete. 2024-05-20T18:31:14.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.748+0000 7f9c6878b640 1 Processor -- start 2024-05-20T18:31:14.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.748+0000 7f9c6878b640 1 -- start start 2024-05-20T18:31:14.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.749+0000 7f9c6878b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c60070850 0x7f9c600c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:14.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.749+0000 7f9c6878b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c600b1e10 0x7f9c600c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:14.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.749+0000 7f9c6878b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c6015d530 0x7f9c6015bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:14.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.750+0000 7f9c6878b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9c600711a0 con 0x7f9c60070850 2024-05-20T18:31:14.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.750+0000 7f9c6878b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9c600cd230 con 0x7f9c600b1e10 2024-05-20T18:31:14.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.750+0000 7f9c6878b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9c60071320 con 0x7f9c6015d530 2024-05-20T18:31:14.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.750+0000 7f9c66527640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c60070850 0x7f9c600c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:14.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.750+0000 7f9c66527640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c60070850 0x7f9c600c8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37438/0 (socket says 172.21.15.5:37438) 2024-05-20T18:31:14.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.750+0000 7f9c66527640 1 -- 172.21.15.5:0/365903221 learned_addr learned my addr 172.21.15.5:0/365903221 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:14.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.750+0000 7f9c65d26640 1 --2- 172.21.15.5:0/365903221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c600b1e10 0x7f9c600c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:14.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.750+0000 7f9c66d28640 1 --2- 172.21.15.5:0/365903221 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c6015d530 0x7f9c6015bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:14.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.750+0000 7f9c66527640 1 -- 172.21.15.5:0/365903221 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c6015d530 msgr2=0x7f9c6015bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:14.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.750+0000 7f9c66527640 1 --2- 172.21.15.5:0/365903221 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c6015d530 0x7f9c6015bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:14.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.750+0000 7f9c66527640 1 -- 172.21.15.5:0/365903221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c600b1e10 msgr2=0x7f9c600c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:14.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.750+0000 7f9c66527640 1 --2- 172.21.15.5:0/365903221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c600b1e10 0x7f9c600c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:14.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.750+0000 7f9c66527640 1 -- 172.21.15.5:0/365903221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c600ca720 con 0x7f9c60070850 2024-05-20T18:31:14.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.750+0000 7f9c66527640 1 --2- 172.21.15.5:0/365903221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c60070850 0x7f9c600c8b70 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f9c5000c6b0 tx=0x7f9c5000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:14.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.750+0000 7f9c5effd640 1 -- 172.21.15.5:0/365903221 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c50016020 con 0x7f9c60070850 2024-05-20T18:31:14.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.750+0000 7f9c5effd640 1 -- 172.21.15.5:0/365903221 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c50006d50 con 0x7f9c60070850 2024-05-20T18:31:14.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.750+0000 7f9c6878b640 1 -- 172.21.15.5:0/365903221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c600caa00 con 0x7f9c60070850 2024-05-20T18:31:14.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.750+0000 7f9c5effd640 1 -- 172.21.15.5:0/365903221 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c50003bb0 con 0x7f9c60070850 2024-05-20T18:31:14.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.750+0000 7f9c6878b640 1 -- 172.21.15.5:0/365903221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c600cafc0 con 0x7f9c60070850 2024-05-20T18:31:14.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.752+0000 7f9c6878b640 1 -- 172.21.15.5:0/365903221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c60075b00 con 0x7f9c60070850 2024-05-20T18:31:14.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.752+0000 7f9c5effd640 1 -- 172.21.15.5:0/365903221 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9c500050d0 con 0x7f9c60070850 2024-05-20T18:31:14.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.752+0000 7f9c5effd640 1 --2- 172.21.15.5:0/365903221 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9c4c06be30 0x7f9c4c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:14.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.753+0000 7f9c5effd640 1 -- 172.21.15.5:0/365903221 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6246+0+0 (secure 0 0 0) 0x7f9c5008ab70 con 0x7f9c60070850 2024-05-20T18:31:14.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.753+0000 7f9c65d26640 1 --2- 172.21.15.5:0/365903221 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9c4c06be30 0x7f9c4c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:14.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.753+0000 7f9c65d26640 1 --2- 172.21.15.5:0/365903221 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9c4c06be30 0x7f9c4c06e340 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9c600b1de0 tx=0x7f9c54009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:14.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.757+0000 7f9c5effd640 1 -- 172.21.15.5:0/365903221 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9c50020020 con 0x7f9c60070850 2024-05-20T18:31:14.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.938+0000 7f9c6878b640 1 -- 172.21.15.5:0/365903221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9c6015e7e0 con 0x7f9c60070850 2024-05-20T18:31:14.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.938+0000 7f9c5effd640 1 -- 172.21.15.5:0/365903221 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) ==== 93+0+10 (secure 0 0 0) 0x7f9c50058ca0 con 0x7f9c60070850 2024-05-20T18:31:14.939 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:31:14.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.940+0000 7f9c6878b640 1 -- 172.21.15.5:0/365903221 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9c4c06be30 msgr2=0x7f9c4c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:14.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.940+0000 7f9c6878b640 1 --2- 172.21.15.5:0/365903221 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9c4c06be30 0x7f9c4c06e340 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9c600b1de0 tx=0x7f9c54009190 comp rx=0 tx=0).stop 2024-05-20T18:31:14.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.940+0000 7f9c6878b640 1 -- 172.21.15.5:0/365903221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c60070850 msgr2=0x7f9c600c8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:14.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.940+0000 7f9c6878b640 1 --2- 172.21.15.5:0/365903221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c60070850 0x7f9c600c8b70 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f9c5000c6b0 tx=0x7f9c5000cb70 comp rx=0 tx=0).stop 2024-05-20T18:31:14.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.940+0000 7f9c6878b640 1 -- 172.21.15.5:0/365903221 shutdown_connections 2024-05-20T18:31:14.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.940+0000 7f9c6878b640 1 --2- 172.21.15.5:0/365903221 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9c4c06be30 0x7f9c4c06e340 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:14.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.940+0000 7f9c6878b640 1 --2- 172.21.15.5:0/365903221 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c6015d530 0x7f9c6015bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:14.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.940+0000 7f9c6878b640 1 --2- 172.21.15.5:0/365903221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c600b1e10 0x7f9c600c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:14.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.940+0000 7f9c6878b640 1 --2- 172.21.15.5:0/365903221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c60070850 0x7f9c600c8b70 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:14.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.940+0000 7f9c6878b640 1 -- 172.21.15.5:0/365903221 >> 172.21.15.5:0/365903221 conn(0x7f9c600d29d0 msgr2=0x7f9c600b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:14.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.940+0000 7f9c6878b640 1 -- 172.21.15.5:0/365903221 shutdown_connections 2024-05-20T18:31:14.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:14.940+0000 7f9c6878b640 1 -- 172.21.15.5:0/365903221 wait complete. 2024-05-20T18:31:14.955 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:31:15.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.073+0000 7fb93de0d640 1 Processor -- start 2024-05-20T18:31:15.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.073+0000 7fb93de0d640 1 -- start start 2024-05-20T18:31:15.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.074+0000 7fb93de0d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb9380d10e0 0x7fb9380cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:15.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.074+0000 7fb93de0d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb9380d14f0 0x7fb9380cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:15.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.074+0000 7fb93de0d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9380d1ac0 con 0x7fb9380cd4d0 2024-05-20T18:31:15.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.074+0000 7fb93de0d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fb9380d1da0 con 0x7fb9380d14f0 2024-05-20T18:31:15.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.074+0000 7fb93de0d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb9380d2080 con 0x7fb9380d10e0 2024-05-20T18:31:15.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.075+0000 7fb936ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb9380d14f0 0x7fb9380cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:15.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.075+0000 7fb937fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb9380cd4d0 0x7fb9380cd8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45370/0 (socket says 172.21.15.5:45370) 2024-05-20T18:31:15.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.075+0000 7fb936ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb9380d14f0 0x7fb9380cf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37444/0 (socket says 172.21.15.5:37444) 2024-05-20T18:31:15.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.075+0000 7fb937fff640 1 -- 172.21.15.5:0/986212722 learned_addr learned my addr 172.21.15.5:0/986212722 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:15.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.075+0000 7fb9377fe640 1 --2- 172.21.15.5:0/986212722 >> v2:172.21.15.63:3300/0 conn(0x7fb9380d10e0 0x7fb9380cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:15.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.075+0000 7fb936ffd640 1 -- 172.21.15.5:0/986212722 >> v2:172.21.15.63:3300/0 conn(0x7fb9380d10e0 msgr2=0x7fb9380cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:15.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.075+0000 7fb936ffd640 1 --2- 172.21.15.5:0/986212722 >> v2:172.21.15.63:3300/0 conn(0x7fb9380d10e0 0x7fb9380cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:15.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.075+0000 7fb936ffd640 1 -- 172.21.15.5:0/986212722 >> v1:172.21.15.5:6789/0 conn(0x7fb9380cd4d0 legacy=0x7fb9380cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:15.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.075+0000 7fb936ffd640 1 -- 172.21.15.5:0/986212722 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9380d2360 con 0x7fb9380d14f0 2024-05-20T18:31:15.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.075+0000 7fb9367fc640 1 -- 172.21.15.5:0/986212722 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3305902828 0 0) 0x7fb9380d1ac0 con 0x7fb9380cd4d0 2024-05-20T18:31:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.076+0000 7fb936ffd640 1 --2- 172.21.15.5:0/986212722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9380d14f0 0x7fb9380cf680 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fb9380ccfc0 tx=0x7fb92c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2760a9165877ce0c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.076+0000 7fb9367fc640 1 -- 172.21.15.5:0/986212722 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c037070 con 0x7fb9380d14f0 2024-05-20T18:31:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.076+0000 7fb9367fc640 1 -- 172.21.15.5:0/986212722 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb92c032440 con 0x7fb9380d14f0 2024-05-20T18:31:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.076+0000 7fb9367fc640 1 -- 172.21.15.5:0/986212722 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c032810 con 0x7fb9380d14f0 2024-05-20T18:31:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.077+0000 7fb93de0d640 1 -- 172.21.15.5:0/986212722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9380d14f0 msgr2=0x7fb9380cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.077+0000 7fb93de0d640 1 --2- 172.21.15.5:0/986212722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9380d14f0 0x7fb9380cf680 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fb9380ccfc0 tx=0x7fb92c02a1a0 comp rx=0 tx=0).stop 2024-05-20T18:31:15.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.077+0000 7fb93de0d640 1 -- 172.21.15.5:0/986212722 shutdown_connections 2024-05-20T18:31:15.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.077+0000 7fb93de0d640 1 --2- 172.21.15.5:0/986212722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9380d14f0 0x7fb9380cf680 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:15.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.077+0000 7fb93de0d640 1 --2- 172.21.15.5:0/986212722 >> v2:172.21.15.63:3300/0 conn(0x7fb9380d10e0 0x7fb9380cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:15.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.077+0000 7fb93de0d640 1 -- 172.21.15.5:0/986212722 >> 172.21.15.5:0/986212722 conn(0x7fb9380b44d0 msgr2=0x7fb9380b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:15.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.077+0000 7fb93de0d640 1 -- 172.21.15.5:0/986212722 shutdown_connections 2024-05-20T18:31:15.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.077+0000 7fb93de0d640 1 -- 172.21.15.5:0/986212722 wait complete. 2024-05-20T18:31:15.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.078+0000 7fb93de0d640 1 Processor -- start 2024-05-20T18:31:15.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.079+0000 7fb93de0d640 1 -- start start 2024-05-20T18:31:15.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.080+0000 7fb93de0d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9380cd4d0 0x7fb93815de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:15.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.080+0000 7fb93de0d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9380d10e0 0x7fb938157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:15.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.080+0000 7fb93de0d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9380d14f0 0x7fb9380c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:15.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.080+0000 7fb93de0d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb9380b8440 con 0x7fb9380cd4d0 2024-05-20T18:31:15.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.080+0000 7fb93de0d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb9380b82c0 con 0x7fb9380d10e0 2024-05-20T18:31:15.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.080+0000 7fb93de0d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb9380709f0 con 0x7fb9380d14f0 2024-05-20T18:31:15.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.080+0000 7fb9377fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9380cd4d0 0x7fb93815de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:15.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.080+0000 7fb9377fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9380cd4d0 0x7fb93815de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37458/0 (socket says 172.21.15.5:37458) 2024-05-20T18:31:15.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.080+0000 7fb9377fe640 1 -- 172.21.15.5:0/1630375674 learned_addr learned my addr 172.21.15.5:0/1630375674 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:15.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.080+0000 7fb936ffd640 1 --2- 172.21.15.5:0/1630375674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9380d10e0 0x7fb938157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.080+0000 7fb937fff640 1 --2- 172.21.15.5:0/1630375674 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9380d14f0 0x7fb9380c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.080+0000 7fb9377fe640 1 -- 172.21.15.5:0/1630375674 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9380d14f0 msgr2=0x7fb9380c64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.080+0000 7fb9377fe640 1 --2- 172.21.15.5:0/1630375674 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9380d14f0 0x7fb9380c64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.080+0000 7fb9377fe640 1 -- 172.21.15.5:0/1630375674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9380d10e0 msgr2=0x7fb938157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.080+0000 7fb9377fe640 1 --2- 172.21.15.5:0/1630375674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9380d10e0 0x7fb938157b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.081+0000 7fb9377fe640 1 -- 172.21.15.5:0/1630375674 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93815e690 con 0x7fb9380cd4d0 2024-05-20T18:31:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.081+0000 7fb936ffd640 1 --2- 172.21.15.5:0/1630375674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9380d10e0 0x7fb938157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.081+0000 7fb937fff640 1 --2- 172.21.15.5:0/1630375674 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9380d14f0 0x7fb9380c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.081+0000 7fb9377fe640 1 --2- 172.21.15.5:0/1630375674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9380cd4d0 0x7fb93815de60 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb92800e8a0 tx=0x7fb92800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.081+0000 7fb91ffff640 1 -- 172.21.15.5:0/1630375674 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb928017070 con 0x7fb9380cd4d0 2024-05-20T18:31:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.081+0000 7fb91ffff640 1 -- 172.21.15.5:0/1630375674 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb92800c940 con 0x7fb9380cd4d0 2024-05-20T18:31:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.081+0000 7fb93de0d640 1 -- 172.21.15.5:0/1630375674 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb93815e970 con 0x7fb9380cd4d0 2024-05-20T18:31:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.081+0000 7fb91ffff640 1 -- 172.21.15.5:0/1630375674 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92800cd30 con 0x7fb9380cd4d0 2024-05-20T18:31:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.081+0000 7fb93de0d640 1 -- 172.21.15.5:0/1630375674 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb93815ef30 con 0x7fb9380cd4d0 2024-05-20T18:31:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.082+0000 7fb93de0d640 1 -- 172.21.15.5:0/1630375674 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb93806c2d0 con 0x7fb9380cd4d0 2024-05-20T18:31:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.083+0000 7fb91ffff640 1 -- 172.21.15.5:0/1630375674 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb92801a070 con 0x7fb9380cd4d0 2024-05-20T18:31:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.083+0000 7fb91ffff640 1 --2- 172.21.15.5:0/1630375674 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb92406be60 0x7fb92406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.083+0000 7fb91ffff640 1 -- 172.21.15.5:0/1630375674 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6246+0+0 (secure 0 0 0) 0x7fb92808b290 con 0x7fb9380cd4d0 2024-05-20T18:31:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.083+0000 7fb936ffd640 1 --2- 172.21.15.5:0/1630375674 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb92406be60 0x7fb92406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.084+0000 7fb936ffd640 1 --2- 172.21.15.5:0/1630375674 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb92406be60 0x7fb92406e370 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb92c034000 tx=0x7fb92c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:15.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.088+0000 7fb91ffff640 1 -- 172.21.15.5:0/1630375674 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb93806c2d0 con 0x7fb9380cd4d0 2024-05-20T18:31:15.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.272+0000 7fb93de0d640 1 -- 172.21.15.5:0/1630375674 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb9380c62a0 con 0x7fb9380cd4d0 2024-05-20T18:31:15.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.273+0000 7fb91ffff640 1 -- 172.21.15.5:0/1630375674 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) ==== 104+0+11 (secure 0 0 0) 0x7fb9380c62a0 con 0x7fb9380cd4d0 2024-05-20T18:31:15.273 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:31:15.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.275+0000 7fb93de0d640 1 -- 172.21.15.5:0/1630375674 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb92406be60 msgr2=0x7fb92406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:15.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.275+0000 7fb93de0d640 1 --2- 172.21.15.5:0/1630375674 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb92406be60 0x7fb92406e370 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb92c034000 tx=0x7fb92c02e000 comp rx=0 tx=0).stop 2024-05-20T18:31:15.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.275+0000 7fb93de0d640 1 -- 172.21.15.5:0/1630375674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9380cd4d0 msgr2=0x7fb93815de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:15.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.275+0000 7fb93de0d640 1 --2- 172.21.15.5:0/1630375674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9380cd4d0 0x7fb93815de60 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb92800e8a0 tx=0x7fb92800ed60 comp rx=0 tx=0).stop 2024-05-20T18:31:15.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.275+0000 7fb93de0d640 1 -- 172.21.15.5:0/1630375674 shutdown_connections 2024-05-20T18:31:15.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.275+0000 7fb93de0d640 1 --2- 172.21.15.5:0/1630375674 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb92406be60 0x7fb92406e370 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:15.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.275+0000 7fb93de0d640 1 --2- 172.21.15.5:0/1630375674 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9380d14f0 0x7fb9380c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:15.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.275+0000 7fb93de0d640 1 --2- 172.21.15.5:0/1630375674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9380d10e0 0x7fb938157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:15.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.275+0000 7fb93de0d640 1 --2- 172.21.15.5:0/1630375674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9380cd4d0 0x7fb93815de60 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:15.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.275+0000 7fb93de0d640 1 -- 172.21.15.5:0/1630375674 >> 172.21.15.5:0/1630375674 conn(0x7fb9380b44d0 msgr2=0x7fb9380b2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:15.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.275+0000 7fb93de0d640 1 -- 172.21.15.5:0/1630375674 shutdown_connections 2024-05-20T18:31:15.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.275+0000 7fb93de0d640 1 -- 172.21.15.5:0/1630375674 wait complete. 2024-05-20T18:31:15.287 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:31:15.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.405+0000 7fd53f93e640 1 Processor -- start 2024-05-20T18:31:15.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.405+0000 7fd53f93e640 1 -- start start 2024-05-20T18:31:15.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.406+0000 7fd53f93e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5380b1e10 0x7fd53815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:15.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.406+0000 7fd53f93e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd538070850 0x7fd538070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:15.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.406+0000 7fd53f93e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5380cd580 con 0x7fd53815d530 2024-05-20T18:31:15.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.406+0000 7fd53f93e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fd5380ce190 con 0x7fd538070850 2024-05-20T18:31:15.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.406+0000 7fd53f93e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd5380cd860 con 0x7fd5380b1e10 2024-05-20T18:31:15.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.406+0000 7fd53dedb640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd538070850 0x7fd538070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:15.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.406+0000 7fd53d6da640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd53815d530 0x7fd5380b1700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45378/0 (socket says 172.21.15.5:45378) 2024-05-20T18:31:15.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.406+0000 7fd53d6da640 1 -- 172.21.15.5:0/1287895672 learned_addr learned my addr 172.21.15.5:0/1287895672 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:15.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.407+0000 7fd53ced9640 1 --2- 172.21.15.5:0/1287895672 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5380b1e10 0x7fd53815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:15.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.407+0000 7fd53dedb640 1 -- 172.21.15.5:0/1287895672 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5380b1e10 msgr2=0x7fd53815aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:15.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.407+0000 7fd53dedb640 1 --2- 172.21.15.5:0/1287895672 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5380b1e10 0x7fd53815aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:15.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.407+0000 7fd53dedb640 1 -- 172.21.15.5:0/1287895672 >> v1:172.21.15.5:6789/0 conn(0x7fd53815d530 legacy=0x7fd5380b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:15.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.407+0000 7fd53dedb640 1 -- 172.21.15.5:0/1287895672 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5380cdb40 con 0x7fd538070850 2024-05-20T18:31:15.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.407+0000 7fd52ffff640 1 -- 172.21.15.5:0/1287895672 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3192605621 0 0) 0x7fd5380cd580 con 0x7fd53815d530 2024-05-20T18:31:15.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.407+0000 7fd53ced9640 1 --2- 172.21.15.5:0/1287895672 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5380b1e10 0x7fd53815aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:15.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.408+0000 7fd53dedb640 1 --2- 172.21.15.5:0/1287895672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd538070850 0x7fd538070c60 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fd530009d90 tx=0x7fd53002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d595dbfbece6f66b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:15.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.408+0000 7fd52ffff640 1 -- 172.21.15.5:0/1287895672 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd530007740 con 0x7fd538070850 2024-05-20T18:31:15.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.408+0000 7fd52ffff640 1 -- 172.21.15.5:0/1287895672 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5300078e0 con 0x7fd538070850 2024-05-20T18:31:15.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.408+0000 7fd52ffff640 1 -- 172.21.15.5:0/1287895672 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd530007cb0 con 0x7fd538070850 2024-05-20T18:31:15.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.408+0000 7fd53f93e640 1 -- 172.21.15.5:0/1287895672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd538070850 msgr2=0x7fd538070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:15.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.408+0000 7fd53f93e640 1 --2- 172.21.15.5:0/1287895672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd538070850 0x7fd538070c60 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fd530009d90 tx=0x7fd53002a660 comp rx=0 tx=0).stop 2024-05-20T18:31:15.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.408+0000 7fd53f93e640 1 -- 172.21.15.5:0/1287895672 shutdown_connections 2024-05-20T18:31:15.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.408+0000 7fd53f93e640 1 --2- 172.21.15.5:0/1287895672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd538070850 0x7fd538070c60 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:15.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.408+0000 7fd53f93e640 1 --2- 172.21.15.5:0/1287895672 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5380b1e10 0x7fd53815aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:15.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.408+0000 7fd53f93e640 1 -- 172.21.15.5:0/1287895672 >> 172.21.15.5:0/1287895672 conn(0x7fd5380d29d0 msgr2=0x7fd5380d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:15.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.409+0000 7fd53f93e640 1 -- 172.21.15.5:0/1287895672 shutdown_connections 2024-05-20T18:31:15.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.409+0000 7fd53f93e640 1 -- 172.21.15.5:0/1287895672 wait complete. 2024-05-20T18:31:15.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.410+0000 7fd53f93e640 1 Processor -- start 2024-05-20T18:31:15.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.410+0000 7fd53f93e640 1 -- start start 2024-05-20T18:31:15.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.412+0000 7fd53f93e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd538070850 0x7fd5380c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:15.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.412+0000 7fd53f93e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5380b1e10 0x7fd5380c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:15.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.412+0000 7fd53f93e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd53815d530 0x7fd5380c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:15.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.412+0000 7fd53f93e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd5380711a0 con 0x7fd53815d530 2024-05-20T18:31:15.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.412+0000 7fd53f93e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd5380cd230 con 0x7fd5380b1e10 2024-05-20T18:31:15.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.412+0000 7fd53f93e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd538071320 con 0x7fd538070850 2024-05-20T18:31:15.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.412+0000 7fd53dedb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd53815d530 0x7fd5380c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:15.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.412+0000 7fd53dedb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd53815d530 0x7fd5380c88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37472/0 (socket says 172.21.15.5:37472) 2024-05-20T18:31:15.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.412+0000 7fd53dedb640 1 -- 172.21.15.5:0/1372447375 learned_addr learned my addr 172.21.15.5:0/1372447375 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:15.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.412+0000 7fd53d6da640 1 --2- 172.21.15.5:0/1372447375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd538070850 0x7fd5380c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:15.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.412+0000 7fd53ced9640 1 --2- 172.21.15.5:0/1372447375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5380b1e10 0x7fd5380c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:15.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.412+0000 7fd53dedb640 1 -- 172.21.15.5:0/1372447375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd538070850 msgr2=0x7fd5380c7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:15.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.412+0000 7fd53dedb640 1 --2- 172.21.15.5:0/1372447375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd538070850 0x7fd5380c7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:15.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.412+0000 7fd53dedb640 1 -- 172.21.15.5:0/1372447375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5380b1e10 msgr2=0x7fd5380c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:15.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.412+0000 7fd53dedb640 1 --2- 172.21.15.5:0/1372447375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5380b1e10 0x7fd5380c8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:15.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.412+0000 7fd53dedb640 1 -- 172.21.15.5:0/1372447375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5380cbd30 con 0x7fd53815d530 2024-05-20T18:31:15.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.412+0000 7fd53dedb640 1 --2- 172.21.15.5:0/1372447375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd53815d530 0x7fd5380c88d0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fd53002ac50 tx=0x7fd530004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:15.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.412+0000 7fd52dffb640 1 -- 172.21.15.5:0/1372447375 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53003f070 con 0x7fd53815d530 2024-05-20T18:31:15.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.413+0000 7fd52dffb640 1 -- 172.21.15.5:0/1372447375 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd530031c90 con 0x7fd53815d530 2024-05-20T18:31:15.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.413+0000 7fd53f93e640 1 -- 172.21.15.5:0/1372447375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5380cc010 con 0x7fd53815d530 2024-05-20T18:31:15.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.413+0000 7fd52dffb640 1 -- 172.21.15.5:0/1372447375 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd530043630 con 0x7fd53815d530 2024-05-20T18:31:15.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.413+0000 7fd53f93e640 1 -- 172.21.15.5:0/1372447375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5380cc5d0 con 0x7fd53815d530 2024-05-20T18:31:15.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.414+0000 7fd53f93e640 1 -- 172.21.15.5:0/1372447375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5380c59b0 con 0x7fd53815d530 2024-05-20T18:31:15.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.414+0000 7fd52dffb640 1 -- 172.21.15.5:0/1372447375 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd53003b410 con 0x7fd53815d530 2024-05-20T18:31:15.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.415+0000 7fd52dffb640 1 --2- 172.21.15.5:0/1372447375 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd51c06be60 0x7fd51c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:15.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.415+0000 7fd52dffb640 1 -- 172.21.15.5:0/1372447375 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6246+0+0 (secure 0 0 0) 0x7fd5300ac4f0 con 0x7fd53815d530 2024-05-20T18:31:15.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.415+0000 7fd53d6da640 1 --2- 172.21.15.5:0/1372447375 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd51c06be60 0x7fd51c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:15.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.416+0000 7fd53d6da640 1 --2- 172.21.15.5:0/1372447375 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd51c06be60 0x7fd51c06e370 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd534007fd0 tx=0x7fd534007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:15.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.420+0000 7fd52dffb640 1 -- 172.21.15.5:0/1372447375 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd5380c59b0 con 0x7fd53815d530 2024-05-20T18:31:15.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.603+0000 7fd53f93e640 1 -- 172.21.15.5:0/1372447375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd538075b00 con 0x7fd53815d530 2024-05-20T18:31:15.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.603+0000 7fd52dffb640 1 -- 172.21.15.5:0/1372447375 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) ==== 100+0+11 (secure 0 0 0) 0x7fd53007a660 con 0x7fd53815d530 2024-05-20T18:31:15.603 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:31:15.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.605+0000 7fd53f93e640 1 -- 172.21.15.5:0/1372447375 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd51c06be60 msgr2=0x7fd51c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:15.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.605+0000 7fd53f93e640 1 --2- 172.21.15.5:0/1372447375 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd51c06be60 0x7fd51c06e370 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd534007fd0 tx=0x7fd534007de0 comp rx=0 tx=0).stop 2024-05-20T18:31:15.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.605+0000 7fd53f93e640 1 -- 172.21.15.5:0/1372447375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd53815d530 msgr2=0x7fd5380c88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:15.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.605+0000 7fd53f93e640 1 --2- 172.21.15.5:0/1372447375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd53815d530 0x7fd5380c88d0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fd53002ac50 tx=0x7fd530004a40 comp rx=0 tx=0).stop 2024-05-20T18:31:15.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.605+0000 7fd53f93e640 1 -- 172.21.15.5:0/1372447375 shutdown_connections 2024-05-20T18:31:15.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.605+0000 7fd53f93e640 1 --2- 172.21.15.5:0/1372447375 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd51c06be60 0x7fd51c06e370 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:15.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.605+0000 7fd53f93e640 1 --2- 172.21.15.5:0/1372447375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd53815d530 0x7fd5380c88d0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:15.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.605+0000 7fd53f93e640 1 --2- 172.21.15.5:0/1372447375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5380b1e10 0x7fd5380c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:15.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.605+0000 7fd53f93e640 1 --2- 172.21.15.5:0/1372447375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd538070850 0x7fd5380c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:15.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.605+0000 7fd53f93e640 1 -- 172.21.15.5:0/1372447375 >> 172.21.15.5:0/1372447375 conn(0x7fd5380d29d0 msgr2=0x7fd5380b2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:15.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.605+0000 7fd53f93e640 1 -- 172.21.15.5:0/1372447375 shutdown_connections 2024-05-20T18:31:15.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.605+0000 7fd53f93e640 1 -- 172.21.15.5:0/1372447375 wait complete. 2024-05-20T18:31:15.617 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:31:15.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.732+0000 7fd0a8538640 1 Processor -- start 2024-05-20T18:31:15.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.732+0000 7fd0a8538640 1 -- start start 2024-05-20T18:31:15.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.733+0000 7fd0a8538640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd0a0168050 0x7fd0a016a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:15.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.733+0000 7fd0a8538640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0a00d0630 con 0x7fd0a006d3d0 2024-05-20T18:31:15.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.733+0000 7fd0a8538640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0a00d39c0 con 0x7fd0a0162820 2024-05-20T18:31:15.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.733+0000 7fd0a8538640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd0a00d3ca0 con 0x7fd0a0168050 2024-05-20T18:31:15.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.734+0000 7fd0a5ad3640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd0a006d3d0 0x7fd0a006d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45380/0 (socket says 172.21.15.5:45380) 2024-05-20T18:31:15.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.734+0000 7fd0a5ad3640 1 -- 172.21.15.5:0/2812836868 learned_addr learned my addr 172.21.15.5:0/2812836868 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:15.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.734+0000 7fd0a6ad5640 1 --2- 172.21.15.5:0/2812836868 >> v2:172.21.15.63:3300/0 conn(0x7fd0a0168050 0x7fd0a016a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:15.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.734+0000 7fd0a52d2640 1 -- 172.21.15.5:0/2812836868 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2012388508 0 0) 0x7fd0a00d0630 con 0x7fd0a006d3d0 2024-05-20T18:31:15.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.734+0000 7fd0a52d2640 1 -- 172.21.15.5:0/2812836868 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd08c003520 con 0x7fd0a006d3d0 2024-05-20T18:31:15.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.734+0000 7fd0a6ad5640 1 -- 172.21.15.5:0/2812836868 >> v1:172.21.15.63:6789/0 conn(0x7fd0a0162820 legacy=0x7fd0a00b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:15.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.734+0000 7fd0a52d2640 1 -- 172.21.15.5:0/2812836868 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3922077056 0 0) 0x7fd0a00d39c0 con 0x7fd0a0162820 2024-05-20T18:31:15.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.734+0000 7fd0a6ad5640 1 -- 172.21.15.5:0/2812836868 >> v1:172.21.15.5:6789/0 conn(0x7fd0a006d3d0 legacy=0x7fd0a006d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:15.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.734+0000 7fd0a6ad5640 1 -- 172.21.15.5:0/2812836868 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a00d3f80 con 0x7fd0a0168050 2024-05-20T18:31:15.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.735+0000 7fd0a6ad5640 1 --2- 172.21.15.5:0/2812836868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0a0168050 0x7fd0a016a470 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd098009d90 tx=0x7fd09802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a37b4173d421bac4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:15.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.735+0000 7fd0a52d2640 1 -- 172.21.15.5:0/2812836868 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098007740 con 0x7fd0a0168050 2024-05-20T18:31:15.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.735+0000 7fd0a52d2640 1 -- 172.21.15.5:0/2812836868 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0980078e0 con 0x7fd0a0168050 2024-05-20T18:31:15.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.735+0000 7fd0a52d2640 1 -- 172.21.15.5:0/2812836868 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098007cb0 con 0x7fd0a0168050 2024-05-20T18:31:15.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.736+0000 7fd0a8538640 1 -- 172.21.15.5:0/2812836868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0a0168050 msgr2=0x7fd0a016a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:15.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.736+0000 7fd0a8538640 1 --2- 172.21.15.5:0/2812836868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0a0168050 0x7fd0a016a470 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd098009d90 tx=0x7fd09802a6f0 comp rx=0 tx=0).stop 2024-05-20T18:31:15.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.736+0000 7fd0a8538640 1 -- 172.21.15.5:0/2812836868 shutdown_connections 2024-05-20T18:31:15.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.736+0000 7fd0a8538640 1 --2- 172.21.15.5:0/2812836868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0a0168050 0x7fd0a016a470 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:15.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.736+0000 7fd0a8538640 1 -- 172.21.15.5:0/2812836868 >> 172.21.15.5:0/2812836868 conn(0x7fd0a00d4bd0 msgr2=0x7fd0a00d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:15.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.736+0000 7fd0a8538640 1 -- 172.21.15.5:0/2812836868 shutdown_connections 2024-05-20T18:31:15.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.736+0000 7fd0a8538640 1 -- 172.21.15.5:0/2812836868 wait complete. 2024-05-20T18:31:15.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.738+0000 7fd0a8538640 1 Processor -- start 2024-05-20T18:31:15.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.738+0000 7fd0a8538640 1 -- start start 2024-05-20T18:31:15.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.739+0000 7fd0a8538640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a006d3d0 0x7fd0a015ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:15.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.739+0000 7fd0a8538640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0a0162820 0x7fd0a00c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:15.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.739+0000 7fd0a8538640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0a0168050 0x7fd0a00ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:15.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.739+0000 7fd0a8538640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a00b91a0 con 0x7fd0a006d3d0 2024-05-20T18:31:15.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.739+0000 7fd0a8538640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a00d0300 con 0x7fd0a0168050 2024-05-20T18:31:15.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.739+0000 7fd0a8538640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd0a00b4050 con 0x7fd0a0162820 2024-05-20T18:31:15.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.739+0000 7fd0a62d4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a006d3d0 0x7fd0a015ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:15.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.739+0000 7fd0a62d4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a006d3d0 0x7fd0a015ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37474/0 (socket says 172.21.15.5:37474) 2024-05-20T18:31:15.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.739+0000 7fd0a62d4640 1 -- 172.21.15.5:0/2562636473 learned_addr learned my addr 172.21.15.5:0/2562636473 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:15.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.740+0000 7fd0a6ad5640 1 --2- 172.21.15.5:0/2562636473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0a0168050 0x7fd0a00ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:15.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.740+0000 7fd0a5ad3640 1 --2- 172.21.15.5:0/2562636473 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0a0162820 0x7fd0a00c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:15.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.740+0000 7fd0a62d4640 1 -- 172.21.15.5:0/2562636473 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0a0162820 msgr2=0x7fd0a00c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:15.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.740+0000 7fd0a62d4640 1 --2- 172.21.15.5:0/2562636473 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0a0162820 0x7fd0a00c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:15.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.740+0000 7fd0a62d4640 1 -- 172.21.15.5:0/2562636473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0a0168050 msgr2=0x7fd0a00ca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:15.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.740+0000 7fd0a62d4640 1 --2- 172.21.15.5:0/2562636473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0a0168050 0x7fd0a00ca190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:15.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.740+0000 7fd0a62d4640 1 -- 172.21.15.5:0/2562636473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a00c91b0 con 0x7fd0a006d3d0 2024-05-20T18:31:15.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.740+0000 7fd0a5ad3640 1 --2- 172.21.15.5:0/2562636473 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0a0162820 0x7fd0a00c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:15.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.740+0000 7fd0a62d4640 1 --2- 172.21.15.5:0/2562636473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a006d3d0 0x7fd0a015ca00 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00c410 tx=0x7fd09c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:15.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.740+0000 7fd096ffd640 1 -- 172.21.15.5:0/2562636473 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c017070 con 0x7fd0a006d3d0 2024-05-20T18:31:15.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.740+0000 7fd096ffd640 1 -- 172.21.15.5:0/2562636473 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd09c0035f0 con 0x7fd0a006d3d0 2024-05-20T18:31:15.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.740+0000 7fd0a8538640 1 -- 172.21.15.5:0/2562636473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a00b64e0 con 0x7fd0a006d3d0 2024-05-20T18:31:15.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.740+0000 7fd096ffd640 1 -- 172.21.15.5:0/2562636473 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c0039e0 con 0x7fd0a006d3d0 2024-05-20T18:31:15.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.740+0000 7fd0a8538640 1 -- 172.21.15.5:0/2562636473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a00b67c0 con 0x7fd0a006d3d0 2024-05-20T18:31:15.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.741+0000 7fd0a8538640 1 -- 172.21.15.5:0/2562636473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a0075b00 con 0x7fd0a006d3d0 2024-05-20T18:31:15.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.742+0000 7fd096ffd640 1 -- 172.21.15.5:0/2562636473 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd09c01a070 con 0x7fd0a006d3d0 2024-05-20T18:31:15.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.742+0000 7fd096ffd640 1 --2- 172.21.15.5:0/2562636473 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd08406be30 0x7fd08406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:15.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.742+0000 7fd096ffd640 1 -- 172.21.15.5:0/2562636473 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6246+0+0 (secure 0 0 0) 0x7fd09c08a500 con 0x7fd0a006d3d0 2024-05-20T18:31:15.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.742+0000 7fd0a5ad3640 1 --2- 172.21.15.5:0/2562636473 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd08406be30 0x7fd08406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:15.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.743+0000 7fd0a5ad3640 1 --2- 172.21.15.5:0/2562636473 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd08406be30 0x7fd08406e340 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd0a00ceab0 tx=0x7fd090009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:15.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.747+0000 7fd096ffd640 1 -- 172.21.15.5:0/2562636473 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd09c058c90 con 0x7fd0a006d3d0 2024-05-20T18:31:15.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.969+0000 7fd0a8538640 1 -- 172.21.15.5:0/2562636473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0a0067b60 con 0x7fd0a006d3d0 2024-05-20T18:31:15.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.970+0000 7fd096ffd640 1 -- 172.21.15.5:0/2562636473 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5177 (secure 0 0 0) 0x7fd09c058590 con 0x7fd0a006d3d0 2024-05-20T18:31:15.970 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:31:15.970 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5480,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/4292435960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":4292435960},{"type":"v1","addr":"172.21.15.63:6835","nonce":4292435960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6186,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2991834087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2991834087},{"type":"v1","addr":"172.21.15.5:6836","nonce":2991834087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6195,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3462206198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":3462206198},{"type":"v1","addr":"172.21.15.5:6837","nonce":3462206198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:30:00.286575+0000","modified":"2024-05-20T18:30:48.943731+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6198":{"gid":6198,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":15,"addr":"172.21.15.63:6834/4036379300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":4036379300},{"type":"v1","addr":"172.21.15.63:6834","nonce":4036379300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6198,"qdb_cluster":[6198]},"id":4}]} 2024-05-20T18:31:15.970 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 36 2024-05-20T18:31:15.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.971+0000 7fd0a8538640 1 -- 172.21.15.5:0/2562636473 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd08406be30 msgr2=0x7fd08406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:15.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.971+0000 7fd0a8538640 1 --2- 172.21.15.5:0/2562636473 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd08406be30 0x7fd08406e340 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd0a00ceab0 tx=0x7fd090009040 comp rx=0 tx=0).stop 2024-05-20T18:31:15.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.971+0000 7fd0a8538640 1 -- 172.21.15.5:0/2562636473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a006d3d0 msgr2=0x7fd0a015ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:15.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.971+0000 7fd0a8538640 1 --2- 172.21.15.5:0/2562636473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a006d3d0 0x7fd0a015ca00 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00c410 tx=0x7fd09c00c8d0 comp rx=0 tx=0).stop 2024-05-20T18:31:15.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.971+0000 7fd0a8538640 1 -- 172.21.15.5:0/2562636473 shutdown_connections 2024-05-20T18:31:15.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.972+0000 7fd0a8538640 1 --2- 172.21.15.5:0/2562636473 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd08406be30 0x7fd08406e340 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:15.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.972+0000 7fd0a8538640 1 --2- 172.21.15.5:0/2562636473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0a0168050 0x7fd0a00ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:15.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.972+0000 7fd0a8538640 1 --2- 172.21.15.5:0/2562636473 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0a0162820 0x7fd0a00c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:15.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.972+0000 7fd0a8538640 1 --2- 172.21.15.5:0/2562636473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a006d3d0 0x7fd0a015ca00 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:15.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.972+0000 7fd0a8538640 1 -- 172.21.15.5:0/2562636473 >> 172.21.15.5:0/2562636473 conn(0x7fd0a00d4bd0 msgr2=0x7fd0a015ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:15.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.972+0000 7fd0a8538640 1 -- 172.21.15.5:0/2562636473 shutdown_connections 2024-05-20T18:31:15.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:15.972+0000 7fd0a8538640 1 -- 172.21.15.5:0/2562636473 wait complete. 2024-05-20T18:31:15.983 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:31:16.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.098+0000 7fbf44a3d640 1 Processor -- start 2024-05-20T18:31:16.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.098+0000 7fbf44a3d640 1 -- start start 2024-05-20T18:31:16.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.099+0000 7fbf44a3d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbf4006d9a0 0x7fbf400d1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:16.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.099+0000 7fbf44a3d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf4015e6b0 con 0x7fbf400b2f10 2024-05-20T18:31:16.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.099+0000 7fbf44a3d640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf400d1100 con 0x7fbf4015d940 2024-05-20T18:31:16.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.099+0000 7fbf44a3d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fbf400d2400 con 0x7fbf4006d9a0 2024-05-20T18:31:16.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.099+0000 7fbf3dd9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbf4006d9a0 0x7fbf400d1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:16.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.099+0000 7fbf3dd9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbf4006d9a0 0x7fbf400d1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37478/0 (socket says 172.21.15.5:37478) 2024-05-20T18:31:16.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.099+0000 7fbf3e59c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbf400b2f10 0x7fbf4006d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45388/0 (socket says 172.21.15.5:45388) 2024-05-20T18:31:16.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.099+0000 7fbf3dd9b640 1 -- 172.21.15.5:0/599841296 learned_addr learned my addr 172.21.15.5:0/599841296 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:16.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.100+0000 7fbf3dd9b640 1 -- 172.21.15.5:0/599841296 >> v1:172.21.15.63:6789/0 conn(0x7fbf4015d940 legacy=0x7fbf4015dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:16.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.100+0000 7fbf3dd9b640 1 -- 172.21.15.5:0/599841296 >> v1:172.21.15.5:6789/0 conn(0x7fbf400b2f10 legacy=0x7fbf4006d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:16.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.100+0000 7fbf3dd9b640 1 -- 172.21.15.5:0/599841296 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf400d13e0 con 0x7fbf4006d9a0 2024-05-20T18:31:16.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.100+0000 7fbf3d59a640 1 -- 172.21.15.5:0/599841296 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1957596685 0 0) 0x7fbf4015e6b0 con 0x7fbf400b2f10 2024-05-20T18:31:16.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.100+0000 7fbf3dd9b640 1 --2- 172.21.15.5:0/599841296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf4006d9a0 0x7fbf400d1ec0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fbf28009850 tx=0x7fbf2802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee0e77323747a38 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:16.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.101+0000 7fbf3d59a640 1 -- 172.21.15.5:0/599841296 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf28004030 con 0x7fbf4006d9a0 2024-05-20T18:31:16.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.101+0000 7fbf3d59a640 1 -- 172.21.15.5:0/599841296 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf280041d0 con 0x7fbf4006d9a0 2024-05-20T18:31:16.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.101+0000 7fbf3d59a640 1 -- 172.21.15.5:0/599841296 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf28003610 con 0x7fbf4006d9a0 2024-05-20T18:31:16.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.101+0000 7fbf44a3d640 1 -- 172.21.15.5:0/599841296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf4006d9a0 msgr2=0x7fbf400d1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:16.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.101+0000 7fbf44a3d640 1 --2- 172.21.15.5:0/599841296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf4006d9a0 0x7fbf400d1ec0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fbf28009850 tx=0x7fbf2802a250 comp rx=0 tx=0).stop 2024-05-20T18:31:16.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.101+0000 7fbf44a3d640 1 -- 172.21.15.5:0/599841296 shutdown_connections 2024-05-20T18:31:16.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.101+0000 7fbf44a3d640 1 --2- 172.21.15.5:0/599841296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf4006d9a0 0x7fbf400d1ec0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:16.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.101+0000 7fbf44a3d640 1 -- 172.21.15.5:0/599841296 >> 172.21.15.5:0/599841296 conn(0x7fbf400b6670 msgr2=0x7fbf400b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:16.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.101+0000 7fbf44a3d640 1 -- 172.21.15.5:0/599841296 shutdown_connections 2024-05-20T18:31:16.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.101+0000 7fbf44a3d640 1 -- 172.21.15.5:0/599841296 wait complete. 2024-05-20T18:31:16.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.103+0000 7fbf44a3d640 1 Processor -- start 2024-05-20T18:31:16.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.103+0000 7fbf44a3d640 1 -- start start 2024-05-20T18:31:16.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.105+0000 7fbf44a3d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf4006d9a0 0x7fbf401460e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:16.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.105+0000 7fbf44a3d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf400b2f10 0x7fbf4015c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:16.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.105+0000 7fbf44a3d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf4015d940 0x7fbf4015c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:16.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.105+0000 7fbf44a3d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbf400b3320 con 0x7fbf4015d940 2024-05-20T18:31:16.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.105+0000 7fbf44a3d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbf400d1a30 con 0x7fbf400b2f10 2024-05-20T18:31:16.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.105+0000 7fbf44a3d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbf4015e460 con 0x7fbf4006d9a0 2024-05-20T18:31:16.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.105+0000 7fbf3ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf4015d940 0x7fbf4015c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:16.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.105+0000 7fbf3ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf4015d940 0x7fbf4015c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37492/0 (socket says 172.21.15.5:37492) 2024-05-20T18:31:16.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.105+0000 7fbf3ed9d640 1 -- 172.21.15.5:0/2339627192 learned_addr learned my addr 172.21.15.5:0/2339627192 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:16.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.105+0000 7fbf3e59c640 1 --2- 172.21.15.5:0/2339627192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf4006d9a0 0x7fbf401460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:16.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.105+0000 7fbf3dd9b640 1 --2- 172.21.15.5:0/2339627192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf400b2f10 0x7fbf4015c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:16.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.105+0000 7fbf3ed9d640 1 -- 172.21.15.5:0/2339627192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf4006d9a0 msgr2=0x7fbf401460e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:16.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.105+0000 7fbf3ed9d640 1 --2- 172.21.15.5:0/2339627192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf4006d9a0 0x7fbf401460e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:16.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.105+0000 7fbf3ed9d640 1 -- 172.21.15.5:0/2339627192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf400b2f10 msgr2=0x7fbf4015c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:16.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.105+0000 7fbf3ed9d640 1 --2- 172.21.15.5:0/2339627192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf400b2f10 0x7fbf4015c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:16.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.105+0000 7fbf3ed9d640 1 -- 172.21.15.5:0/2339627192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf400af730 con 0x7fbf4015d940 2024-05-20T18:31:16.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.105+0000 7fbf3ed9d640 1 --2- 172.21.15.5:0/2339627192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf4015d940 0x7fbf4015c720 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000c3c0 tx=0x7fbf3000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:16.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.106+0000 7fbf36ffd640 1 -- 172.21.15.5:0/2339627192 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf30017070 con 0x7fbf4015d940 2024-05-20T18:31:16.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.106+0000 7fbf36ffd640 1 -- 172.21.15.5:0/2339627192 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf300052d0 con 0x7fbf4015d940 2024-05-20T18:31:16.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.106+0000 7fbf36ffd640 1 -- 172.21.15.5:0/2339627192 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf30003830 con 0x7fbf4015d940 2024-05-20T18:31:16.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.106+0000 7fbf44a3d640 1 -- 172.21.15.5:0/2339627192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf400afa10 con 0x7fbf4015d940 2024-05-20T18:31:16.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.106+0000 7fbf44a3d640 1 -- 172.21.15.5:0/2339627192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf400affd0 con 0x7fbf4015d940 2024-05-20T18:31:16.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.107+0000 7fbf44a3d640 1 -- 172.21.15.5:0/2339627192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf40075b00 con 0x7fbf4015d940 2024-05-20T18:31:16.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.107+0000 7fbf36ffd640 1 -- 172.21.15.5:0/2339627192 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbf3001a070 con 0x7fbf4015d940 2024-05-20T18:31:16.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.108+0000 7fbf36ffd640 1 --2- 172.21.15.5:0/2339627192 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf2006be60 0x7fbf2006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:16.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.108+0000 7fbf36ffd640 1 -- 172.21.15.5:0/2339627192 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6246+0+0 (secure 0 0 0) 0x7fbf3008a2a0 con 0x7fbf4015d940 2024-05-20T18:31:16.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.108+0000 7fbf3e59c640 1 --2- 172.21.15.5:0/2339627192 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf2006be60 0x7fbf2006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:16.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.108+0000 7fbf3e59c640 1 --2- 172.21.15.5:0/2339627192 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf2006be60 0x7fbf2006e370 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbf38007f50 tx=0x7fbf38007ea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:16.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.112+0000 7fbf36ffd640 1 -- 172.21.15.5:0/2339627192 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbf30058410 con 0x7fbf4015d940 2024-05-20T18:31:16.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.297+0000 7fbf44a3d640 1 -- 172.21.15.5:0/2339627192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf40157210 con 0x7fbf4015d940 2024-05-20T18:31:16.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.299+0000 7fbf36ffd640 1 -- 172.21.15.5:0/2339627192 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+17004 (secure 0 0 0) 0x7fbf3005c710 con 0x7fbf4015d940 2024-05-20T18:31:16.299 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:31:16.299 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":65,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:30:57.074551+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:29:57.993861+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-20T18:29:59.267582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.18","mappings":[{"from":7,"to":2}]},{"pgid":"10.c","mappings":[{"from":4,"to":5}]},{"pgid":"10.1d","mappings":[{"from":4,"to":3}]},{"pgid":"10.33","mappings":[{"from":4,"to":3}]},{"pgid":"10.37","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:31:16.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.301+0000 7fbf44a3d640 1 -- 172.21.15.5:0/2339627192 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf2006be60 msgr2=0x7fbf2006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:16.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.301+0000 7fbf44a3d640 1 --2- 172.21.15.5:0/2339627192 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf2006be60 0x7fbf2006e370 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbf38007f50 tx=0x7fbf38007ea0 comp rx=0 tx=0).stop 2024-05-20T18:31:16.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.301+0000 7fbf44a3d640 1 -- 172.21.15.5:0/2339627192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf4015d940 msgr2=0x7fbf4015c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:16.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.301+0000 7fbf44a3d640 1 --2- 172.21.15.5:0/2339627192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf4015d940 0x7fbf4015c720 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000c3c0 tx=0x7fbf3000c7c0 comp rx=0 tx=0).stop 2024-05-20T18:31:16.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.301+0000 7fbf44a3d640 1 -- 172.21.15.5:0/2339627192 shutdown_connections 2024-05-20T18:31:16.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.301+0000 7fbf44a3d640 1 --2- 172.21.15.5:0/2339627192 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf2006be60 0x7fbf2006e370 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:16.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.301+0000 7fbf44a3d640 1 --2- 172.21.15.5:0/2339627192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf4015d940 0x7fbf4015c720 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:16.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.301+0000 7fbf44a3d640 1 --2- 172.21.15.5:0/2339627192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf400b2f10 0x7fbf4015c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:16.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.301+0000 7fbf44a3d640 1 --2- 172.21.15.5:0/2339627192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf4006d9a0 0x7fbf401460e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:16.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.301+0000 7fbf44a3d640 1 -- 172.21.15.5:0/2339627192 >> 172.21.15.5:0/2339627192 conn(0x7fbf400b6670 msgr2=0x7fbf401e8650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:16.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.301+0000 7fbf44a3d640 1 -- 172.21.15.5:0/2339627192 shutdown_connections 2024-05-20T18:31:16.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.301+0000 7fbf44a3d640 1 -- 172.21.15.5:0/2339627192 wait complete. 2024-05-20T18:31:16.313 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-20T18:31:16.313 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:31:16.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.427+0000 7feadc58f640 1 Processor -- start 2024-05-20T18:31:16.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.427+0000 7feadc58f640 1 -- start start 2024-05-20T18:31:16.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.429+0000 7feadc58f640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fead40d4400 0x7fead415f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:16.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.429+0000 7feadc58f640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fead408d2a0 con 0x7fead406cf00 2024-05-20T18:31:16.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.429+0000 7feadc58f640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fead40d4810 con 0x7fead40d2780 2024-05-20T18:31:16.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.429+0000 7feadc58f640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fead40d3f70 con 0x7fead40d4400 2024-05-20T18:31:16.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.429+0000 7fead9b2a640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fead406cf00 0x7fead406d310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45396/0 (socket says 172.21.15.5:45396) 2024-05-20T18:31:16.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.429+0000 7fead9b2a640 1 -- 172.21.15.5:0/2735586791 learned_addr learned my addr 172.21.15.5:0/2735586791 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:16.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.429+0000 7fead9329640 1 -- 172.21.15.5:0/2735586791 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3965314860 0 0) 0x7feac4001110 con 0x7fead406cf00 2024-05-20T18:31:16.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.429+0000 7fead9329640 1 -- 172.21.15.5:0/2735586791 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feac0003520 con 0x7fead406cf00 2024-05-20T18:31:16.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.429+0000 7feada32b640 1 --2- 172.21.15.5:0/2735586791 >> v2:172.21.15.63:3300/0 conn(0x7fead40d4400 0x7fead415f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:16.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.430+0000 7fead9329640 1 -- 172.21.15.5:0/2735586791 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 75615804 0 0) 0x7feac0003520 con 0x7fead406cf00 2024-05-20T18:31:16.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.430+0000 7fead9329640 1 -- 172.21.15.5:0/2735586791 >> v2:172.21.15.63:3300/0 conn(0x7fead40d4400 msgr2=0x7fead415f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:16.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.430+0000 7fead9329640 1 --2- 172.21.15.5:0/2735586791 >> v2:172.21.15.63:3300/0 conn(0x7fead40d4400 0x7fead415f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:16.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.430+0000 7fead9329640 1 -- 172.21.15.5:0/2735586791 >> v1:172.21.15.63:6789/0 conn(0x7fead40d2780 legacy=0x7fead40d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:16.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.430+0000 7fead9329640 1 -- 172.21.15.5:0/2735586791 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead40d3360 con 0x7fead406cf00 2024-05-20T18:31:16.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.430+0000 7fead9329640 1 -- 172.21.15.5:0/2735586791 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7feac4001840 con 0x7fead406cf00 2024-05-20T18:31:16.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.430+0000 7fead9329640 1 -- 172.21.15.5:0/2735586791 >> v1:172.21.15.5:6789/0 conn(0x7fead406cf00 legacy=0x7fead406d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:16.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.430+0000 7fead9329640 1 --2- 172.21.15.5:0/2735586791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feac00039b0 0x7feac0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:16.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.430+0000 7fead9329640 1 --2- 172.21.15.5:0/2735586791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feac0004800 0x7feac0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:16.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.430+0000 7fead9329640 1 --2- 172.21.15.5:0/2735586791 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feac0005270 0x7feac000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:16.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.430+0000 7fead9329640 1 -- 172.21.15.5:0/2735586791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fead40d3f70 con 0x7feac00039b0 2024-05-20T18:31:16.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.430+0000 7fead9329640 1 -- 172.21.15.5:0/2735586791 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feac000c180 con 0x7feac0004800 2024-05-20T18:31:16.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.430+0000 7fead9329640 1 -- 172.21.15.5:0/2735586791 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7feac000c380 con 0x7feac0005270 2024-05-20T18:31:16.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.430+0000 7feada32b640 1 --2- 172.21.15.5:0/2735586791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feac00039b0 0x7feac0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:16.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.430+0000 7fead9b2a640 1 --2- 172.21.15.5:0/2735586791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feac0004800 0x7feac0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:16.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.430+0000 7feadab2c640 1 --2- 172.21.15.5:0/2735586791 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feac0005270 0x7feac000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:16.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.430+0000 7feada32b640 1 -- 172.21.15.5:0/2735586791 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feac0005270 msgr2=0x7feac000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:16.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.430+0000 7feada32b640 1 --2- 172.21.15.5:0/2735586791 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feac0005270 0x7feac000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:16.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.430+0000 7feada32b640 1 -- 172.21.15.5:0/2735586791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feac0004800 msgr2=0x7feac0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:16.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.430+0000 7feada32b640 1 --2- 172.21.15.5:0/2735586791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feac0004800 0x7feac0004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:16.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.430+0000 7feada32b640 1 -- 172.21.15.5:0/2735586791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac0003520 con 0x7feac00039b0 2024-05-20T18:31:16.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.430+0000 7fead9b2a640 1 --2- 172.21.15.5:0/2735586791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feac0004800 0x7feac0004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:16.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.430+0000 7feadab2c640 1 -- 172.21.15.5:0/2735586791 reap_dead start 2024-05-20T18:31:16.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.431+0000 7feada32b640 1 --2- 172.21.15.5:0/2735586791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feac00039b0 0x7feac0003e40 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fead0009ac0 tx=0x7feac0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94a90bc17a0c52f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:16.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.431+0000 7fead9329640 1 -- 172.21.15.5:0/2735586791 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead0036070 con 0x7feac00039b0 2024-05-20T18:31:16.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.431+0000 7fead9329640 1 -- 172.21.15.5:0/2735586791 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fead0031400 con 0x7feac00039b0 2024-05-20T18:31:16.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.431+0000 7fead9329640 1 -- 172.21.15.5:0/2735586791 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead0009690 con 0x7feac00039b0 2024-05-20T18:31:16.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.432+0000 7feadc58f640 1 -- 172.21.15.5:0/2735586791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feac00039b0 msgr2=0x7feac0003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:16.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.432+0000 7feadc58f640 1 --2- 172.21.15.5:0/2735586791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feac00039b0 0x7feac0003e40 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fead0009ac0 tx=0x7feac0005830 comp rx=0 tx=0).stop 2024-05-20T18:31:16.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.432+0000 7feadc58f640 1 -- 172.21.15.5:0/2735586791 shutdown_connections 2024-05-20T18:31:16.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.432+0000 7feadc58f640 1 --2- 172.21.15.5:0/2735586791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feac00039b0 0x7feac0003e40 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:16.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.432+0000 7feadc58f640 1 -- 172.21.15.5:0/2735586791 >> 172.21.15.5:0/2735586791 conn(0x7fead40d4bb0 msgr2=0x7fead40d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:16.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.432+0000 7feadc58f640 1 -- 172.21.15.5:0/2735586791 shutdown_connections 2024-05-20T18:31:16.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.432+0000 7feadc58f640 1 -- 172.21.15.5:0/2735586791 wait complete. 2024-05-20T18:31:16.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.434+0000 7feadc58f640 1 Processor -- start 2024-05-20T18:31:16.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.434+0000 7feadc58f640 1 -- start start 2024-05-20T18:31:16.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.435+0000 7feadc58f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feac00039b0 0x7fead41f95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:16.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.435+0000 7feadc58f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fead4150bc0 0x7fead40b9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:16.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.435+0000 7feadc58f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fead4150fd0 0x7fead40b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:16.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.435+0000 7feadc58f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fead40d3c00 con 0x7fead4150bc0 2024-05-20T18:31:16.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.435+0000 7feadc58f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fead40d3920 con 0x7fead4150fd0 2024-05-20T18:31:16.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.435+0000 7feadc58f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fead40d2b90 con 0x7feac00039b0 2024-05-20T18:31:16.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.435+0000 7fead9b2a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fead4150bc0 0x7fead40b9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:16.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.435+0000 7fead9b2a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fead4150bc0 0x7fead40b9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37504/0 (socket says 172.21.15.5:37504) 2024-05-20T18:31:16.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.435+0000 7fead9b2a640 1 -- 172.21.15.5:0/155478013 learned_addr learned my addr 172.21.15.5:0/155478013 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:16.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.435+0000 7feadab2c640 1 --2- 172.21.15.5:0/155478013 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fead4150fd0 0x7fead40b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:16.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.435+0000 7feada32b640 1 --2- 172.21.15.5:0/155478013 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feac00039b0 0x7fead41f95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:16.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.436+0000 7fead9b2a640 1 -- 172.21.15.5:0/155478013 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feac00039b0 msgr2=0x7fead41f95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:16.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.436+0000 7fead9b2a640 1 --2- 172.21.15.5:0/155478013 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feac00039b0 0x7fead41f95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:16.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.436+0000 7fead9b2a640 1 -- 172.21.15.5:0/155478013 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fead4150fd0 msgr2=0x7fead40b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:16.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.436+0000 7fead9b2a640 1 --2- 172.21.15.5:0/155478013 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fead4150fd0 0x7fead40b9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:16.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.436+0000 7fead9b2a640 1 -- 172.21.15.5:0/155478013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead415b7b0 con 0x7fead4150bc0 2024-05-20T18:31:16.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.436+0000 7fead9b2a640 1 --2- 172.21.15.5:0/155478013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fead4150bc0 0x7fead40b9ab0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7feac4007330 tx=0x7feac40077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:16.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.436+0000 7feacaffd640 1 -- 172.21.15.5:0/155478013 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac400e950 con 0x7fead4150bc0 2024-05-20T18:31:16.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.436+0000 7feacaffd640 1 -- 172.21.15.5:0/155478013 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feac4018070 con 0x7fead4150bc0 2024-05-20T18:31:16.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.436+0000 7feadc58f640 1 -- 172.21.15.5:0/155478013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fead415ba90 con 0x7fead4150bc0 2024-05-20T18:31:16.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.436+0000 7feacaffd640 1 -- 172.21.15.5:0/155478013 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac400ed10 con 0x7fead4150bc0 2024-05-20T18:31:16.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.436+0000 7feadc58f640 1 -- 172.21.15.5:0/155478013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fead415c050 con 0x7fead4150bc0 2024-05-20T18:31:16.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.437+0000 7feadc58f640 1 -- 172.21.15.5:0/155478013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fead40735a0 con 0x7fead4150bc0 2024-05-20T18:31:16.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.438+0000 7feacaffd640 1 -- 172.21.15.5:0/155478013 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feac401d460 con 0x7fead4150bc0 2024-05-20T18:31:16.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.438+0000 7feacaffd640 1 --2- 172.21.15.5:0/155478013 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feab806be60 0x7feab806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:16.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.439+0000 7feacaffd640 1 -- 172.21.15.5:0/155478013 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6246+0+0 (secure 0 0 0) 0x7feac408af10 con 0x7fead4150bc0 2024-05-20T18:31:16.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.439+0000 7feada32b640 1 --2- 172.21.15.5:0/155478013 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feab806be60 0x7feab806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:16.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.439+0000 7feada32b640 1 --2- 172.21.15.5:0/155478013 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feab806be60 0x7feab806e370 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fead0033000 tx=0x7fead0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:16.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.443+0000 7feacaffd640 1 -- 172.21.15.5:0/155478013 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fead40735a0 con 0x7fead4150bc0 2024-05-20T18:31:16.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.667+0000 7feadc58f640 1 -- 172.21.15.5:0/155478013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fead4075b00 con 0x7fead4150bc0 2024-05-20T18:31:16.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.668+0000 7feacaffd640 1 -- 172.21.15.5:0/155478013 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5177 (secure 0 0 0) 0x7fead408d2a0 con 0x7fead4150bc0 2024-05-20T18:31:16.668 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:31:16.668 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5480,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/4292435960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":4292435960},{"type":"v1","addr":"172.21.15.63:6835","nonce":4292435960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6186,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2991834087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2991834087},{"type":"v1","addr":"172.21.15.5:6836","nonce":2991834087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6195,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3462206198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":3462206198},{"type":"v1","addr":"172.21.15.5:6837","nonce":3462206198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:30:00.286575+0000","modified":"2024-05-20T18:30:48.943731+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6198":{"gid":6198,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":15,"addr":"172.21.15.63:6834/4036379300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":4036379300},{"type":"v1","addr":"172.21.15.63:6834","nonce":4036379300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6198,"qdb_cluster":[6198]},"id":4}]} 2024-05-20T18:31:16.668 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 36 2024-05-20T18:31:16.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.670+0000 7feadc58f640 1 -- 172.21.15.5:0/155478013 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feab806be60 msgr2=0x7feab806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:16.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.670+0000 7feadc58f640 1 --2- 172.21.15.5:0/155478013 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feab806be60 0x7feab806e370 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fead0033000 tx=0x7fead0008030 comp rx=0 tx=0).stop 2024-05-20T18:31:16.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.670+0000 7feadc58f640 1 -- 172.21.15.5:0/155478013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fead4150bc0 msgr2=0x7fead40b9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:16.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.670+0000 7feadc58f640 1 --2- 172.21.15.5:0/155478013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fead4150bc0 0x7fead40b9ab0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7feac4007330 tx=0x7feac40077f0 comp rx=0 tx=0).stop 2024-05-20T18:31:16.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.670+0000 7feadc58f640 1 -- 172.21.15.5:0/155478013 shutdown_connections 2024-05-20T18:31:16.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.670+0000 7feadc58f640 1 --2- 172.21.15.5:0/155478013 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feab806be60 0x7feab806e370 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:16.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.670+0000 7feadc58f640 1 --2- 172.21.15.5:0/155478013 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fead4150fd0 0x7fead40b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:16.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.670+0000 7feadc58f640 1 --2- 172.21.15.5:0/155478013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fead4150bc0 0x7fead40b9ab0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:16.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.670+0000 7feadc58f640 1 --2- 172.21.15.5:0/155478013 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feac00039b0 0x7fead41f95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:16.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.670+0000 7feadc58f640 1 -- 172.21.15.5:0/155478013 >> 172.21.15.5:0/155478013 conn(0x7fead40d4bb0 msgr2=0x7fead40cdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:16.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.670+0000 7feadc58f640 1 -- 172.21.15.5:0/155478013 shutdown_connections 2024-05-20T18:31:16.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.670+0000 7feadc58f640 1 -- 172.21.15.5:0/155478013 wait complete. 2024-05-20T18:31:16.682 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:31:16.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.796+0000 7fe46e63d640 1 Processor -- start 2024-05-20T18:31:16.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.796+0000 7fe46e63d640 1 -- start start 2024-05-20T18:31:16.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.798+0000 7fe46e63d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe468163a10 0x7fe4680d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:16.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.798+0000 7fe46e63d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4680b4010 0x7fe4680cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:16.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.798+0000 7fe46e63d640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4680d0370 con 0x7fe4680d14b0 2024-05-20T18:31:16.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.798+0000 7fe46e63d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe4680d3ca0 con 0x7fe468163a10 2024-05-20T18:31:16.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.798+0000 7fe46e63d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe4680d0650 con 0x7fe4680b4010 2024-05-20T18:31:16.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.798+0000 7fe467fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe468163a10 0x7fe4680d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:16.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.798+0000 7fe467fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe468163a10 0x7fe4680d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37506/0 (socket says 172.21.15.5:37506) 2024-05-20T18:31:16.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.798+0000 7fe467fff640 1 -- 172.21.15.5:0/1874854350 learned_addr learned my addr 172.21.15.5:0/1874854350 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:16.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.798+0000 7fe4677fe640 1 --2- 172.21.15.5:0/1874854350 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4680b4010 0x7fe4680cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:16.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.798+0000 7fe467fff640 1 -- 172.21.15.5:0/1874854350 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4680b4010 msgr2=0x7fe4680cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:16.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.798+0000 7fe467fff640 1 --2- 172.21.15.5:0/1874854350 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4680b4010 0x7fe4680cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:16.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.798+0000 7fe467fff640 1 -- 172.21.15.5:0/1874854350 >> v1:172.21.15.63:6789/0 conn(0x7fe4680d14b0 legacy=0x7fe4680d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:16.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.798+0000 7fe467fff640 1 -- 172.21.15.5:0/1874854350 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4680d3650 con 0x7fe468163a10 2024-05-20T18:31:16.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.798+0000 7fe4677fe640 1 --2- 172.21.15.5:0/1874854350 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4680b4010 0x7fe4680cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:16.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.799+0000 7fe467fff640 1 --2- 172.21.15.5:0/1874854350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe468163a10 0x7fe4680d4130 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe458009870 tx=0x7fe45802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f32ad479b51fa4d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:16.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.799+0000 7fe466ffd640 1 -- 172.21.15.5:0/1874854350 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe458037070 con 0x7fe468163a10 2024-05-20T18:31:16.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.799+0000 7fe466ffd640 1 -- 172.21.15.5:0/1874854350 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe458032440 con 0x7fe468163a10 2024-05-20T18:31:16.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.799+0000 7fe466ffd640 1 -- 172.21.15.5:0/1874854350 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe458032810 con 0x7fe468163a10 2024-05-20T18:31:16.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.800+0000 7fe46e63d640 1 -- 172.21.15.5:0/1874854350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe468163a10 msgr2=0x7fe4680d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:16.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.800+0000 7fe46e63d640 1 --2- 172.21.15.5:0/1874854350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe468163a10 0x7fe4680d4130 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe458009870 tx=0x7fe45802a250 comp rx=0 tx=0).stop 2024-05-20T18:31:16.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.800+0000 7fe46e63d640 1 -- 172.21.15.5:0/1874854350 shutdown_connections 2024-05-20T18:31:16.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.800+0000 7fe46e63d640 1 --2- 172.21.15.5:0/1874854350 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4680b4010 0x7fe4680cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:16.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.800+0000 7fe46e63d640 1 --2- 172.21.15.5:0/1874854350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe468163a10 0x7fe4680d4130 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:16.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.800+0000 7fe46e63d640 1 -- 172.21.15.5:0/1874854350 >> 172.21.15.5:0/1874854350 conn(0x7fe46806d240 msgr2=0x7fe46806d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:16.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.800+0000 7fe46e63d640 1 -- 172.21.15.5:0/1874854350 shutdown_connections 2024-05-20T18:31:16.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.800+0000 7fe46e63d640 1 -- 172.21.15.5:0/1874854350 wait complete. 2024-05-20T18:31:16.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.802+0000 7fe46e63d640 1 Processor -- start 2024-05-20T18:31:16.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.802+0000 7fe46e63d640 1 -- start start 2024-05-20T18:31:16.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.803+0000 7fe46e63d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4680b4010 0x7fe4680b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:16.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.803+0000 7fe46e63d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4680d14b0 0x7fe46815d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:16.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.804+0000 7fe46e63d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe468163a10 0x7fe4681f8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:16.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.804+0000 7fe46e63d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe4680b8210 con 0x7fe4680d14b0 2024-05-20T18:31:16.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.804+0000 7fe46e63d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe4680d0040 con 0x7fe4680b4010 2024-05-20T18:31:16.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.804+0000 7fe46e63d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe4680b8390 con 0x7fe468163a10 2024-05-20T18:31:16.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.804+0000 7fe4677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4680d14b0 0x7fe46815d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:16.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.804+0000 7fe4677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4680d14b0 0x7fe46815d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37512/0 (socket says 172.21.15.5:37512) 2024-05-20T18:31:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.804+0000 7fe4677fe640 1 -- 172.21.15.5:0/2005091811 learned_addr learned my addr 172.21.15.5:0/2005091811 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.804+0000 7fe467fff640 1 --2- 172.21.15.5:0/2005091811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4680b4010 0x7fe4680b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.804+0000 7fe46cbda640 1 --2- 172.21.15.5:0/2005091811 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe468163a10 0x7fe4681f8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.804+0000 7fe4677fe640 1 -- 172.21.15.5:0/2005091811 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe468163a10 msgr2=0x7fe4681f8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.804+0000 7fe4677fe640 1 --2- 172.21.15.5:0/2005091811 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe468163a10 0x7fe4681f8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.804+0000 7fe4677fe640 1 -- 172.21.15.5:0/2005091811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4680b4010 msgr2=0x7fe4680b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.804+0000 7fe4677fe640 1 --2- 172.21.15.5:0/2005091811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4680b4010 0x7fe4680b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.804+0000 7fe4677fe640 1 -- 172.21.15.5:0/2005091811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4681f9e60 con 0x7fe4680d14b0 2024-05-20T18:31:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.804+0000 7fe467fff640 1 --2- 172.21.15.5:0/2005091811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4680b4010 0x7fe4680b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.804+0000 7fe4677fe640 1 --2- 172.21.15.5:0/2005091811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4680d14b0 0x7fe46815d050 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fe45000d800 tx=0x7fe45000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.804+0000 7fe464ff9640 1 -- 172.21.15.5:0/2005091811 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe450004030 con 0x7fe4680d14b0 2024-05-20T18:31:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.804+0000 7fe464ff9640 1 -- 172.21.15.5:0/2005091811 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4500041d0 con 0x7fe4680d14b0 2024-05-20T18:31:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.804+0000 7fe46e63d640 1 -- 172.21.15.5:0/2005091811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4681ea060 con 0x7fe4680d14b0 2024-05-20T18:31:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.804+0000 7fe464ff9640 1 -- 172.21.15.5:0/2005091811 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4500096a0 con 0x7fe4680d14b0 2024-05-20T18:31:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.805+0000 7fe46e63d640 1 -- 172.21.15.5:0/2005091811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4680ab720 con 0x7fe4680d14b0 2024-05-20T18:31:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.805+0000 7fe46e63d640 1 -- 172.21.15.5:0/2005091811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4681ea580 con 0x7fe4680d14b0 2024-05-20T18:31:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.806+0000 7fe464ff9640 1 -- 172.21.15.5:0/2005091811 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe450009840 con 0x7fe4680d14b0 2024-05-20T18:31:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.807+0000 7fe464ff9640 1 --2- 172.21.15.5:0/2005091811 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe44806be60 0x7fe44806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.807+0000 7fe467fff640 1 --2- 172.21.15.5:0/2005091811 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe44806be60 0x7fe44806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.807+0000 7fe464ff9640 1 -- 172.21.15.5:0/2005091811 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6246+0+0 (secure 0 0 0) 0x7fe45008b360 con 0x7fe4680d14b0 2024-05-20T18:31:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.807+0000 7fe467fff640 1 --2- 172.21.15.5:0/2005091811 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe44806be60 0x7fe44806e370 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe458034000 tx=0x7fe45802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:16.811+0000 7fe464ff9640 1 -- 172.21.15.5:0/2005091811 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe450059a60 con 0x7fe4680d14b0 2024-05-20T18:31:17.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.031+0000 7fe46e63d640 1 -- 172.21.15.5:0/2005091811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe468075b00 con 0x7fe4680d14b0 2024-05-20T18:31:17.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.031+0000 7fe464ff9640 1 -- 172.21.15.5:0/2005091811 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5177 (secure 0 0 0) 0x7fe450059360 con 0x7fe4680d14b0 2024-05-20T18:31:17.032 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:31:17.032 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5480,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/4292435960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":4292435960},{"type":"v1","addr":"172.21.15.63:6835","nonce":4292435960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6186,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2991834087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2991834087},{"type":"v1","addr":"172.21.15.5:6836","nonce":2991834087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6195,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3462206198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":3462206198},{"type":"v1","addr":"172.21.15.5:6837","nonce":3462206198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:30:00.286575+0000","modified":"2024-05-20T18:30:48.943731+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6198":{"gid":6198,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":15,"addr":"172.21.15.63:6834/4036379300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":4036379300},{"type":"v1","addr":"172.21.15.63:6834","nonce":4036379300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6198,"qdb_cluster":[6198]},"id":4}]} 2024-05-20T18:31:17.032 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 36 2024-05-20T18:31:17.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.033+0000 7fe46e63d640 1 -- 172.21.15.5:0/2005091811 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe44806be60 msgr2=0x7fe44806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:17.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.033+0000 7fe46e63d640 1 --2- 172.21.15.5:0/2005091811 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe44806be60 0x7fe44806e370 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe458034000 tx=0x7fe45802e000 comp rx=0 tx=0).stop 2024-05-20T18:31:17.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.033+0000 7fe46e63d640 1 -- 172.21.15.5:0/2005091811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4680d14b0 msgr2=0x7fe46815d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:17.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.033+0000 7fe46e63d640 1 --2- 172.21.15.5:0/2005091811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4680d14b0 0x7fe46815d050 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fe45000d800 tx=0x7fe45000dcc0 comp rx=0 tx=0).stop 2024-05-20T18:31:17.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.033+0000 7fe46e63d640 1 -- 172.21.15.5:0/2005091811 shutdown_connections 2024-05-20T18:31:17.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.033+0000 7fe46e63d640 1 --2- 172.21.15.5:0/2005091811 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe44806be60 0x7fe44806e370 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:17.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.033+0000 7fe46e63d640 1 --2- 172.21.15.5:0/2005091811 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe468163a10 0x7fe4681f8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:17.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.033+0000 7fe46e63d640 1 --2- 172.21.15.5:0/2005091811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4680d14b0 0x7fe46815d050 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:17.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.033+0000 7fe46e63d640 1 --2- 172.21.15.5:0/2005091811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4680b4010 0x7fe4680b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:17.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.033+0000 7fe46e63d640 1 -- 172.21.15.5:0/2005091811 >> 172.21.15.5:0/2005091811 conn(0x7fe46806d240 msgr2=0x7fe46815e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:17.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.034+0000 7fe46e63d640 1 -- 172.21.15.5:0/2005091811 shutdown_connections 2024-05-20T18:31:17.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.034+0000 7fe46e63d640 1 -- 172.21.15.5:0/2005091811 wait complete. 2024-05-20T18:31:17.045 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:31:17.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.159+0000 7f61db444640 1 Processor -- start 2024-05-20T18:31:17.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.160+0000 7f61db444640 1 -- start start 2024-05-20T18:31:17.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.161+0000 7f61db444640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f61d40cd680 0x7f61d415f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:17.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.161+0000 7f61db444640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61d406d0f0 con 0x7f61d40cfae0 2024-05-20T18:31:17.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.161+0000 7f61db444640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61d406d3d0 con 0x7f61d406c6c0 2024-05-20T18:31:17.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.161+0000 7f61db444640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f61d40cda90 con 0x7f61d40cd680 2024-05-20T18:31:17.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.161+0000 7f61d91e0640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f61d40cd680 0x7f61d415f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:17.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.161+0000 7f61d99e1640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f61d40cfae0 0x7f61d40d1060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45398/0 (socket says 172.21.15.5:45398) 2024-05-20T18:31:17.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.161+0000 7f61d99e1640 1 -- 172.21.15.5:0/2220631992 learned_addr learned my addr 172.21.15.5:0/2220631992 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:17.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.162+0000 7f61d91e0640 1 -- 172.21.15.5:0/2220631992 >> v1:172.21.15.63:6789/0 conn(0x7f61d406c6c0 legacy=0x7f61d40cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:17.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.162+0000 7f61d91e0640 1 -- 172.21.15.5:0/2220631992 >> v1:172.21.15.5:6789/0 conn(0x7f61d40cfae0 legacy=0x7f61d40d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:17.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.162+0000 7f61d91e0640 1 -- 172.21.15.5:0/2220631992 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d406d6b0 con 0x7f61d40cd680 2024-05-20T18:31:17.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.162+0000 7f61d91e0640 1 --2- 172.21.15.5:0/2220631992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61d40cd680 0x7f61d415f220 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f61d00097a0 tx=0x7f61d002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75416f85e0fd8f41 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.162+0000 7f61cbfff640 1 -- 172.21.15.5:0/2220631992 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0037070 con 0x7f61d40cd680 2024-05-20T18:31:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.163+0000 7f61cbfff640 1 -- 172.21.15.5:0/2220631992 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d0032440 con 0x7f61d40cd680 2024-05-20T18:31:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.163+0000 7f61cbfff640 1 -- 172.21.15.5:0/2220631992 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d002ae60 con 0x7f61d40cd680 2024-05-20T18:31:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.163+0000 7f61db444640 1 -- 172.21.15.5:0/2220631992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61d40cd680 msgr2=0x7f61d415f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.163+0000 7f61db444640 1 --2- 172.21.15.5:0/2220631992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61d40cd680 0x7f61d415f220 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f61d00097a0 tx=0x7f61d002a110 comp rx=0 tx=0).stop 2024-05-20T18:31:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.163+0000 7f61db444640 1 -- 172.21.15.5:0/2220631992 shutdown_connections 2024-05-20T18:31:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.163+0000 7f61db444640 1 --2- 172.21.15.5:0/2220631992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61d40cd680 0x7f61d415f220 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.163+0000 7f61db444640 1 -- 172.21.15.5:0/2220631992 >> 172.21.15.5:0/2220631992 conn(0x7f61d40b44d0 msgr2=0x7f61d40b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.163+0000 7f61db444640 1 -- 172.21.15.5:0/2220631992 shutdown_connections 2024-05-20T18:31:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.163+0000 7f61db444640 1 -- 172.21.15.5:0/2220631992 wait complete. 2024-05-20T18:31:17.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.165+0000 7f61db444640 1 Processor -- start 2024-05-20T18:31:17.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.165+0000 7f61db444640 1 -- start start 2024-05-20T18:31:17.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.166+0000 7f61db444640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61d406c6c0 0x7f61d4152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:17.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.166+0000 7f61db444640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61d40cd680 0x7f61d40b6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:17.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.166+0000 7f61db444640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61d415fe20 0x7f61d4160230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:17.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.166+0000 7f61db444640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61d40d2460 con 0x7f61d406c6c0 2024-05-20T18:31:17.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.166+0000 7f61db444640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f61d406d990 con 0x7f61d415fe20 2024-05-20T18:31:17.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.167+0000 7f61db444640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f61d40d25e0 con 0x7f61d40cd680 2024-05-20T18:31:17.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.167+0000 7f61d91e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61d406c6c0 0x7f61d4152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:17.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.167+0000 7f61d91e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61d406c6c0 0x7f61d4152150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37518/0 (socket says 172.21.15.5:37518) 2024-05-20T18:31:17.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.167+0000 7f61d91e0640 1 -- 172.21.15.5:0/3130340994 learned_addr learned my addr 172.21.15.5:0/3130340994 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:17.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.167+0000 7f61d99e1640 1 --2- 172.21.15.5:0/3130340994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61d415fe20 0x7f61d4160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:17.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.167+0000 7f61d89df640 1 --2- 172.21.15.5:0/3130340994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61d40cd680 0x7f61d40b6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:17.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.167+0000 7f61d91e0640 1 -- 172.21.15.5:0/3130340994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61d40cd680 msgr2=0x7f61d40b6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:17.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.167+0000 7f61d91e0640 1 --2- 172.21.15.5:0/3130340994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61d40cd680 0x7f61d40b6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:17.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.167+0000 7f61d91e0640 1 -- 172.21.15.5:0/3130340994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61d415fe20 msgr2=0x7f61d4160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:17.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.167+0000 7f61d91e0640 1 --2- 172.21.15.5:0/3130340994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61d415fe20 0x7f61d4160230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:17.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.167+0000 7f61d91e0640 1 -- 172.21.15.5:0/3130340994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d415f940 con 0x7f61d406c6c0 2024-05-20T18:31:17.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.167+0000 7f61d99e1640 1 --2- 172.21.15.5:0/3130340994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61d415fe20 0x7f61d4160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:17.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.167+0000 7f61d91e0640 1 --2- 172.21.15.5:0/3130340994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61d406c6c0 0x7f61d4152150 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f61d0035040 tx=0x7f61d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:17.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.167+0000 7f61c9ffb640 1 -- 172.21.15.5:0/3130340994 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0037040 con 0x7f61d406c6c0 2024-05-20T18:31:17.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.167+0000 7f61c9ffb640 1 -- 172.21.15.5:0/3130340994 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d0030770 con 0x7f61d406c6c0 2024-05-20T18:31:17.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.167+0000 7f61db444640 1 -- 172.21.15.5:0/3130340994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61d40b7ae0 con 0x7f61d406c6c0 2024-05-20T18:31:17.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.167+0000 7f61c9ffb640 1 -- 172.21.15.5:0/3130340994 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0030b40 con 0x7f61d406c6c0 2024-05-20T18:31:17.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.168+0000 7f61db444640 1 -- 172.21.15.5:0/3130340994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61d40b80a0 con 0x7f61d406c6c0 2024-05-20T18:31:17.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.169+0000 7f61db444640 1 -- 172.21.15.5:0/3130340994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61d4075b00 con 0x7f61d406c6c0 2024-05-20T18:31:17.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.169+0000 7f61c9ffb640 1 -- 172.21.15.5:0/3130340994 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f61d0044020 con 0x7f61d406c6c0 2024-05-20T18:31:17.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.170+0000 7f61c9ffb640 1 --2- 172.21.15.5:0/3130340994 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f61b806be60 0x7f61b806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:17.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.170+0000 7f61d89df640 1 --2- 172.21.15.5:0/3130340994 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f61b806be60 0x7f61b806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:17.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.170+0000 7f61c9ffb640 1 -- 172.21.15.5:0/3130340994 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6246+0+0 (secure 0 0 0) 0x7f61d00ab630 con 0x7f61d406c6c0 2024-05-20T18:31:17.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.171+0000 7f61d89df640 1 --2- 172.21.15.5:0/3130340994 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f61b806be60 0x7f61b806e370 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f61c4007fd0 tx=0x7f61c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:17.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.175+0000 7f61c9ffb640 1 -- 172.21.15.5:0/3130340994 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f61d0032e20 con 0x7f61d406c6c0 2024-05-20T18:31:17.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.359+0000 7f61db444640 1 -- 172.21.15.5:0/3130340994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f61d40b8940 con 0x7f61d406c6c0 2024-05-20T18:31:17.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.360+0000 7f61c9ffb640 1 -- 172.21.15.5:0/3130340994 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+17004 (secure 0 0 0) 0x7f61d0079dc0 con 0x7f61d406c6c0 2024-05-20T18:31:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:31:17.361 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":65,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:30:57.074551+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:29:57.993861+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-20T18:29:59.267582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.18","mappings":[{"from":7,"to":2}]},{"pgid":"10.c","mappings":[{"from":4,"to":5}]},{"pgid":"10.1d","mappings":[{"from":4,"to":3}]},{"pgid":"10.33","mappings":[{"from":4,"to":3}]},{"pgid":"10.37","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:31:17.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.362+0000 7f61db444640 1 -- 172.21.15.5:0/3130340994 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f61b806be60 msgr2=0x7f61b806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:17.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.362+0000 7f61db444640 1 --2- 172.21.15.5:0/3130340994 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f61b806be60 0x7f61b806e370 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f61c4007fd0 tx=0x7f61c4009040 comp rx=0 tx=0).stop 2024-05-20T18:31:17.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.362+0000 7f61db444640 1 -- 172.21.15.5:0/3130340994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61d406c6c0 msgr2=0x7f61d4152150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:17.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.362+0000 7f61db444640 1 --2- 172.21.15.5:0/3130340994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61d406c6c0 0x7f61d4152150 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f61d0035040 tx=0x7f61d00046c0 comp rx=0 tx=0).stop 2024-05-20T18:31:17.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.362+0000 7f61db444640 1 -- 172.21.15.5:0/3130340994 shutdown_connections 2024-05-20T18:31:17.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.362+0000 7f61db444640 1 --2- 172.21.15.5:0/3130340994 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f61b806be60 0x7f61b806e370 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:17.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.362+0000 7f61db444640 1 --2- 172.21.15.5:0/3130340994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61d415fe20 0x7f61d4160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:17.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.362+0000 7f61db444640 1 --2- 172.21.15.5:0/3130340994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f61d40cd680 0x7f61d40b6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:17.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.362+0000 7f61db444640 1 --2- 172.21.15.5:0/3130340994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61d406c6c0 0x7f61d4152150 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:17.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.362+0000 7f61db444640 1 -- 172.21.15.5:0/3130340994 >> 172.21.15.5:0/3130340994 conn(0x7f61d40b44d0 msgr2=0x7f61d40d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:17.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.362+0000 7f61db444640 1 -- 172.21.15.5:0/3130340994 shutdown_connections 2024-05-20T18:31:17.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.362+0000 7f61db444640 1 -- 172.21.15.5:0/3130340994 wait complete. 2024-05-20T18:31:17.374 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-20T18:31:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.489+0000 7f723a68d640 1 Processor -- start 2024-05-20T18:31:17.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.489+0000 7f723a68d640 1 -- start start 2024-05-20T18:31:17.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.491+0000 7f723a68d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7234162820 0x7f72340b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:17.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.491+0000 7f723a68d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72340d0630 con 0x7f72340b3e80 2024-05-20T18:31:17.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.491+0000 7f723a68d640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72340d39c0 con 0x7f723406dae0 2024-05-20T18:31:17.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.491+0000 7f723a68d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f72340d3ca0 con 0x7f7234162820 2024-05-20T18:31:17.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.491+0000 7f72337fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f72340b3e80 0x7f723406d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45404/0 (socket says 172.21.15.5:45404) 2024-05-20T18:31:17.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.491+0000 7f72337fe640 1 -- 172.21.15.5:0/1952132669 learned_addr learned my addr 172.21.15.5:0/1952132669 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:17.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.491+0000 7f7233fff640 1 --2- 172.21.15.5:0/1952132669 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7234162820 0x7f72340b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:17.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.491+0000 7f7232ffd640 1 -- 172.21.15.5:0/1952132669 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 676554575 0 0) 0x7f72340d0630 con 0x7f72340b3e80 2024-05-20T18:31:17.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.491+0000 7f7232ffd640 1 -- 172.21.15.5:0/1952132669 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f721c003520 con 0x7f72340b3e80 2024-05-20T18:31:17.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.491+0000 7f7232ffd640 1 -- 172.21.15.5:0/1952132669 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 9784936 0 0) 0x7f72340d39c0 con 0x7f723406dae0 2024-05-20T18:31:17.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.491+0000 7f7232ffd640 1 -- 172.21.15.5:0/1952132669 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f72340d0630 con 0x7f723406dae0 2024-05-20T18:31:17.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.491+0000 7f7232ffd640 1 -- 172.21.15.5:0/1952132669 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2404019451 0 0) 0x7f721c003520 con 0x7f72340b3e80 2024-05-20T18:31:17.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.491+0000 7f7233fff640 1 -- 172.21.15.5:0/1952132669 >> v1:172.21.15.63:6789/0 conn(0x7f723406dae0 legacy=0x7f72340b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:17.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.491+0000 7f7233fff640 1 -- 172.21.15.5:0/1952132669 >> v1:172.21.15.5:6789/0 conn(0x7f72340b3e80 legacy=0x7f723406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:17.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.491+0000 7f7233fff640 1 -- 172.21.15.5:0/1952132669 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72340d3f80 con 0x7f7234162820 2024-05-20T18:31:17.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.492+0000 7f7233fff640 1 --2- 172.21.15.5:0/1952132669 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7234162820 0x7f72340b3940 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f72240097d0 tx=0x7f722402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b96ccb9ac5a5f732 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:17.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.492+0000 7f7232ffd640 1 -- 172.21.15.5:0/1952132669 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7224037070 con 0x7f7234162820 2024-05-20T18:31:17.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.492+0000 7f7232ffd640 1 -- 172.21.15.5:0/1952132669 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f722402ac30 con 0x7f7234162820 2024-05-20T18:31:17.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.493+0000 7f7232ffd640 1 -- 172.21.15.5:0/1952132669 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7228001670 con 0x7f7234162820 2024-05-20T18:31:17.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.493+0000 7f723a68d640 1 -- 172.21.15.5:0/1952132669 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7234162820 msgr2=0x7f72340b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:17.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.493+0000 7f723a68d640 1 --2- 172.21.15.5:0/1952132669 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7234162820 0x7f72340b3940 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f72240097d0 tx=0x7f722402a1a0 comp rx=0 tx=0).stop 2024-05-20T18:31:17.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.493+0000 7f723a68d640 1 -- 172.21.15.5:0/1952132669 shutdown_connections 2024-05-20T18:31:17.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.493+0000 7f723a68d640 1 --2- 172.21.15.5:0/1952132669 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7234162820 0x7f72340b3940 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:17.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.493+0000 7f723a68d640 1 -- 172.21.15.5:0/1952132669 >> 172.21.15.5:0/1952132669 conn(0x7f72340d4bd0 msgr2=0x7f72340d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:17.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.493+0000 7f723a68d640 1 -- 172.21.15.5:0/1952132669 shutdown_connections 2024-05-20T18:31:17.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.493+0000 7f723a68d640 1 -- 172.21.15.5:0/1952132669 wait complete. 2024-05-20T18:31:17.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.495+0000 7f723a68d640 1 Processor -- start 2024-05-20T18:31:17.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.495+0000 7f723a68d640 1 -- start start 2024-05-20T18:31:17.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.496+0000 7f723a68d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72340b3e80 0x7f72341f7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:17.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.496+0000 7f723a68d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7234162820 0x7f72340b9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:17.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.496+0000 7f723a68d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7234144eb0 0x7f72341452c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:17.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.497+0000 7f723a68d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f72340d4780 con 0x7f7234144eb0 2024-05-20T18:31:17.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.497+0000 7f723a68d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f72340d0300 con 0x7f7234162820 2024-05-20T18:31:17.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.497+0000 7f723a68d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f72340d4900 con 0x7f72340b3e80 2024-05-20T18:31:17.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.497+0000 7f7238c2a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7234144eb0 0x7f72341452c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:17.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.497+0000 7f7238c2a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7234144eb0 0x7f72341452c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37534/0 (socket says 172.21.15.5:37534) 2024-05-20T18:31:17.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.497+0000 7f7238c2a640 1 -- 172.21.15.5:0/3755348988 learned_addr learned my addr 172.21.15.5:0/3755348988 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:17.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.497+0000 7f7233fff640 1 --2- 172.21.15.5:0/3755348988 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72340b3e80 0x7f72341f7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:17.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.497+0000 7f72337fe640 1 --2- 172.21.15.5:0/3755348988 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7234162820 0x7f72340b9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:17.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.497+0000 7f7238c2a640 1 -- 172.21.15.5:0/3755348988 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72340b3e80 msgr2=0x7f72341f7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:17.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.497+0000 7f7238c2a640 1 --2- 172.21.15.5:0/3755348988 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72340b3e80 0x7f72341f7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:17.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.497+0000 7f7238c2a640 1 -- 172.21.15.5:0/3755348988 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7234162820 msgr2=0x7f72340b9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:17.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.497+0000 7f7238c2a640 1 --2- 172.21.15.5:0/3755348988 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7234162820 0x7f72340b9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:17.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.497+0000 7f7238c2a640 1 -- 172.21.15.5:0/3755348988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72340b46d0 con 0x7f7234144eb0 2024-05-20T18:31:17.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.497+0000 7f7238c2a640 1 --2- 172.21.15.5:0/3755348988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7234144eb0 0x7f72341452c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f722c00c710 tx=0x7f722c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:17.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.497+0000 7f7230ff9640 1 -- 172.21.15.5:0/3755348988 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722c003b70 con 0x7f7234144eb0 2024-05-20T18:31:17.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.497+0000 7f7230ff9640 1 -- 172.21.15.5:0/3755348988 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f722c003d10 con 0x7f7234144eb0 2024-05-20T18:31:17.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.497+0000 7f723a68d640 1 -- 172.21.15.5:0/3755348988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72340b49b0 con 0x7f7234144eb0 2024-05-20T18:31:17.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.497+0000 7f7230ff9640 1 -- 172.21.15.5:0/3755348988 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722c00aa50 con 0x7f7234144eb0 2024-05-20T18:31:17.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.497+0000 7f723a68d640 1 -- 172.21.15.5:0/3755348988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72340b9160 con 0x7f7234144eb0 2024-05-20T18:31:17.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.498+0000 7f723a68d640 1 -- 172.21.15.5:0/3755348988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f723415bfc0 con 0x7f7234144eb0 2024-05-20T18:31:17.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.499+0000 7f7230ff9640 1 -- 172.21.15.5:0/3755348988 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f722c00abf0 con 0x7f7234144eb0 2024-05-20T18:31:17.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.499+0000 7f7230ff9640 1 --2- 172.21.15.5:0/3755348988 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f722006be30 0x7f722006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:17.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.499+0000 7f7230ff9640 1 -- 172.21.15.5:0/3755348988 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6246+0+0 (secure 0 0 0) 0x7f722c08b020 con 0x7f7234144eb0 2024-05-20T18:31:17.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.500+0000 7f7233fff640 1 --2- 172.21.15.5:0/3755348988 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f722006be30 0x7f722006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:17.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.500+0000 7f7233fff640 1 --2- 172.21.15.5:0/3755348988 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f722006be30 0x7f722006e340 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7224034000 tx=0x7f722402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:17.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.504+0000 7f7230ff9640 1 -- 172.21.15.5:0/3755348988 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f723415bfc0 con 0x7f7234144eb0 2024-05-20T18:31:17.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.729+0000 7f723a68d640 1 -- 172.21.15.5:0/3755348988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7234075b00 con 0x7f7234144eb0 2024-05-20T18:31:17.730 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-20T18:31:17.730+0000 7fd1cf53a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-20T18:31:17.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.731+0000 7f7230ff9640 1 -- 172.21.15.5:0/3755348988 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) ==== 161+0+0 (secure 0 0 0) 0x7f722c00ce20 con 0x7f7234144eb0 2024-05-20T18:31:17.731 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-20T18:31:17.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.734+0000 7f723a68d640 1 -- 172.21.15.5:0/3755348988 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f722006be30 msgr2=0x7f722006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:17.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.734+0000 7f723a68d640 1 --2- 172.21.15.5:0/3755348988 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f722006be30 0x7f722006e340 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7224034000 tx=0x7f722402e000 comp rx=0 tx=0).stop 2024-05-20T18:31:17.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.734+0000 7f723a68d640 1 -- 172.21.15.5:0/3755348988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7234144eb0 msgr2=0x7f72341452c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:17.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.734+0000 7f723a68d640 1 --2- 172.21.15.5:0/3755348988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7234144eb0 0x7f72341452c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f722c00c710 tx=0x7f722c00cbd0 comp rx=0 tx=0).stop 2024-05-20T18:31:17.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.734+0000 7f723a68d640 1 -- 172.21.15.5:0/3755348988 shutdown_connections 2024-05-20T18:31:17.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.734+0000 7f723a68d640 1 --2- 172.21.15.5:0/3755348988 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f722006be30 0x7f722006e340 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:17.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.734+0000 7f723a68d640 1 --2- 172.21.15.5:0/3755348988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7234144eb0 0x7f72341452c0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:17.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.734+0000 7f723a68d640 1 --2- 172.21.15.5:0/3755348988 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7234162820 0x7f72340b9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:17.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.734+0000 7f723a68d640 1 --2- 172.21.15.5:0/3755348988 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72340b3e80 0x7f72341f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:17.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.734+0000 7f723a68d640 1 -- 172.21.15.5:0/3755348988 >> 172.21.15.5:0/3755348988 conn(0x7f72340d4bd0 msgr2=0x7f723415dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:17.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.734+0000 7f723a68d640 1 -- 172.21.15.5:0/3755348988 shutdown_connections 2024-05-20T18:31:17.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.734+0000 7f723a68d640 1 -- 172.21.15.5:0/3755348988 wait complete. 2024-05-20T18:31:17.740 INFO:tasks.ceph.mds.b.smithi063.stderr: -7659> 2024-05-20T18:30:05.879+0000 7f9764196640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T18:31:17.743 INFO:tasks.ceph.mds.b.smithi063.stderr: -4855> 2024-05-20T18:30:09.101+0000 7f976018e640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5555=0-4194304@1} caps={5555=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5555 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x5611ac41cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T18:31:17.745 INFO:tasks.ceph.mds.b.smithi063.stderr: -3095> 2024-05-20T18:30:40.552+0000 7f976018e640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5555=0-4194304@1} caps={5555=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5555 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x5611ac41cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T18:31:17.746 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-20T18:31:17.805 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-20T18:31:17.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.863+0000 7f4e88e5a640 1 Processor -- start 2024-05-20T18:31:17.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.863+0000 7f4e88e5a640 1 -- start start 2024-05-20T18:31:17.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.864+0000 7f4e88e5a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e840cf250 0x7f4e8415f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:17.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.864+0000 7f4e88e5a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4e840d3220 0x7f4e840d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:17.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.864+0000 7f4e88e5a640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e840d0550 con 0x7f4e840b6680 2024-05-20T18:31:17.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.864+0000 7f4e88e5a640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f4e8415f9a0 con 0x7f4e840d3220 2024-05-20T18:31:17.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.864+0000 7f4e88e5a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4e840d0b10 con 0x7f4e840cf250 2024-05-20T18:31:17.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.864+0000 7f4e82d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4e840d3220 0x7f4e840d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:17.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.864+0000 7f4e82d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4e840d3220 0x7f4e840d3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37540/0 (socket says 172.21.15.5:37540) 2024-05-20T18:31:17.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.865+0000 7f4e82d9d640 1 -- 172.21.15.5:0/3879103001 learned_addr learned my addr 172.21.15.5:0/3879103001 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:17.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.865+0000 7f4e8259c640 1 --2- 172.21.15.5:0/3879103001 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e840cf250 0x7f4e8415f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:17.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.865+0000 7f4e8259c640 1 -- 172.21.15.5:0/3879103001 >> v2:172.21.15.5:3300/0 conn(0x7f4e840d3220 msgr2=0x7f4e840d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:17.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.865+0000 7f4e8259c640 1 --2- 172.21.15.5:0/3879103001 >> v2:172.21.15.5:3300/0 conn(0x7f4e840d3220 0x7f4e840d3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:17.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.865+0000 7f4e8259c640 1 -- 172.21.15.5:0/3879103001 >> v1:172.21.15.63:6789/0 conn(0x7f4e840b6680 legacy=0x7f4e840b6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:17.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.865+0000 7f4e8259c640 1 -- 172.21.15.5:0/3879103001 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e840d0830 con 0x7f4e840cf250 2024-05-20T18:31:17.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.865+0000 7f4e8159a640 1 -- 172.21.15.5:0/3879103001 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 228723375 0 0) 0x7f4e840d0550 con 0x7f4e840b6680 2024-05-20T18:31:17.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.866+0000 7f4e8259c640 1 --2- 172.21.15.5:0/3879103001 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e840cf250 0x7f4e8415f460 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4e740097a0 tx=0x7f4e7402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc680b73d3f30db1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:17.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.866+0000 7f4e8159a640 1 -- 172.21.15.5:0/3879103001 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e74037070 con 0x7f4e840cf250 2024-05-20T18:31:17.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.866+0000 7f4e8159a640 1 -- 172.21.15.5:0/3879103001 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e7402ae50 con 0x7f4e840cf250 2024-05-20T18:31:17.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.866+0000 7f4e8159a640 1 -- 172.21.15.5:0/3879103001 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e74032660 con 0x7f4e840cf250 2024-05-20T18:31:17.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.866+0000 7f4e88e5a640 1 -- 172.21.15.5:0/3879103001 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e840cf250 msgr2=0x7f4e8415f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:17.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.866+0000 7f4e88e5a640 1 --2- 172.21.15.5:0/3879103001 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e840cf250 0x7f4e8415f460 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4e740097a0 tx=0x7f4e7402a100 comp rx=0 tx=0).stop 2024-05-20T18:31:17.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.867+0000 7f4e88e5a640 1 -- 172.21.15.5:0/3879103001 shutdown_connections 2024-05-20T18:31:17.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.867+0000 7f4e88e5a640 1 --2- 172.21.15.5:0/3879103001 >> v2:172.21.15.5:3300/0 conn(0x7f4e840d3220 0x7f4e840d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:17.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.867+0000 7f4e88e5a640 1 --2- 172.21.15.5:0/3879103001 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e840cf250 0x7f4e8415f460 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:17.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.867+0000 7f4e88e5a640 1 -- 172.21.15.5:0/3879103001 >> 172.21.15.5:0/3879103001 conn(0x7f4e840d1860 msgr2=0x7f4e840d1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:17.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.867+0000 7f4e88e5a640 1 -- 172.21.15.5:0/3879103001 shutdown_connections 2024-05-20T18:31:17.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.867+0000 7f4e88e5a640 1 -- 172.21.15.5:0/3879103001 wait complete. 2024-05-20T18:31:17.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.869+0000 7f4e88e5a640 1 Processor -- start 2024-05-20T18:31:17.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.869+0000 7f4e88e5a640 1 -- start start 2024-05-20T18:31:17.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.870+0000 7f4e88e5a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e840b6680 0x7f4e84161c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:17.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.870+0000 7f4e88e5a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e840cf250 0x7f4e841f83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:17.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.870+0000 7f4e88e5a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e840d3220 0x7f4e8415dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:17.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.870+0000 7f4e88e5a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e840d2d00 con 0x7f4e840d3220 2024-05-20T18:31:17.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.870+0000 7f4e88e5a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4e840d27d0 con 0x7f4e840cf250 2024-05-20T18:31:17.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.870+0000 7f4e88e5a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4e840d0140 con 0x7f4e840b6680 2024-05-20T18:31:17.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.870+0000 7f4e82d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e840d3220 0x7f4e8415dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:17.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.870+0000 7f4e82d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e840d3220 0x7f4e8415dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37554/0 (socket says 172.21.15.5:37554) 2024-05-20T18:31:17.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.870+0000 7f4e82d9d640 1 -- 172.21.15.5:0/3455490262 learned_addr learned my addr 172.21.15.5:0/3455490262 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:17.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.870+0000 7f4e81d9b640 1 --2- 172.21.15.5:0/3455490262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e840cf250 0x7f4e841f83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:17.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.870+0000 7f4e8259c640 1 --2- 172.21.15.5:0/3455490262 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e840b6680 0x7f4e84161c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:17.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.870+0000 7f4e82d9d640 1 -- 172.21.15.5:0/3455490262 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e840b6680 msgr2=0x7f4e84161c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:17.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.870+0000 7f4e82d9d640 1 --2- 172.21.15.5:0/3455490262 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e840b6680 0x7f4e84161c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:17.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.870+0000 7f4e82d9d640 1 -- 172.21.15.5:0/3455490262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e840cf250 msgr2=0x7f4e841f83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:17.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.870+0000 7f4e82d9d640 1 --2- 172.21.15.5:0/3455490262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e840cf250 0x7f4e841f83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:17.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.870+0000 7f4e82d9d640 1 -- 172.21.15.5:0/3455490262 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e841e9da0 con 0x7f4e840d3220 2024-05-20T18:31:17.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.870+0000 7f4e81d9b640 1 --2- 172.21.15.5:0/3455490262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e840cf250 0x7f4e841f83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:17.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.871+0000 7f4e82d9d640 1 --2- 172.21.15.5:0/3455490262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e840d3220 0x7f4e8415dd90 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009cb0 tx=0x7f4e7c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:17.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.871+0000 7f4e72ffd640 1 -- 172.21.15.5:0/3455490262 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c00cd20 con 0x7f4e840d3220 2024-05-20T18:31:17.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.871+0000 7f4e72ffd640 1 -- 172.21.15.5:0/3455490262 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e7c003990 con 0x7f4e840d3220 2024-05-20T18:31:17.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.871+0000 7f4e72ffd640 1 -- 172.21.15.5:0/3455490262 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c003d80 con 0x7f4e840d3220 2024-05-20T18:31:17.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.871+0000 7f4e88e5a640 1 -- 172.21.15.5:0/3455490262 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e841ea080 con 0x7f4e840d3220 2024-05-20T18:31:17.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.871+0000 7f4e88e5a640 1 -- 172.21.15.5:0/3455490262 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e841ea640 con 0x7f4e840d3220 2024-05-20T18:31:17.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.872+0000 7f4e88e5a640 1 -- 172.21.15.5:0/3455490262 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e84075b00 con 0x7f4e840d3220 2024-05-20T18:31:17.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.873+0000 7f4e72ffd640 1 -- 172.21.15.5:0/3455490262 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4e7c012050 con 0x7f4e840d3220 2024-05-20T18:31:17.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.873+0000 7f4e72ffd640 1 --2- 172.21.15.5:0/3455490262 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4e6406be30 0x7f4e6406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:17.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.873+0000 7f4e72ffd640 1 -- 172.21.15.5:0/3455490262 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6332+0+0 (secure 0 0 0) 0x7f4e7c08b610 con 0x7f4e840d3220 2024-05-20T18:31:17.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.873+0000 7f4e8259c640 1 --2- 172.21.15.5:0/3455490262 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4e6406be30 0x7f4e6406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:17.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.874+0000 7f4e8259c640 1 --2- 172.21.15.5:0/3455490262 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4e6406be30 0x7f4e6406e340 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4e74034000 tx=0x7f4e7402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:17.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:17.878+0000 7f4e72ffd640 1 -- 172.21.15.5:0/3455490262 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4e7c010040 con 0x7f4e840d3220 2024-05-20T18:31:18.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.099+0000 7f4e88e5a640 1 -- 172.21.15.5:0/3455490262 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4e841e9b30 con 0x7f4e840d3220 2024-05-20T18:31:18.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.733+0000 7f4e72ffd640 1 -- 172.21.15.5:0/3455490262 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) ==== 104+0+0 (secure 0 0 0) 0x7f4e7c059cc0 con 0x7f4e840d3220 2024-05-20T18:31:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.736+0000 7f4e88e5a640 1 -- 172.21.15.5:0/3455490262 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4e6406be30 msgr2=0x7f4e6406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.736+0000 7f4e88e5a640 1 --2- 172.21.15.5:0/3455490262 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4e6406be30 0x7f4e6406e340 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4e74034000 tx=0x7f4e7402e000 comp rx=0 tx=0).stop 2024-05-20T18:31:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.736+0000 7f4e88e5a640 1 -- 172.21.15.5:0/3455490262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e840d3220 msgr2=0x7f4e8415dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.736+0000 7f4e88e5a640 1 --2- 172.21.15.5:0/3455490262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e840d3220 0x7f4e8415dd90 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009cb0 tx=0x7f4e7c00c490 comp rx=0 tx=0).stop 2024-05-20T18:31:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.736+0000 7f4e88e5a640 1 -- 172.21.15.5:0/3455490262 shutdown_connections 2024-05-20T18:31:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.736+0000 7f4e88e5a640 1 --2- 172.21.15.5:0/3455490262 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4e6406be30 0x7f4e6406e340 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.736+0000 7f4e88e5a640 1 --2- 172.21.15.5:0/3455490262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e840d3220 0x7f4e8415dd90 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.736+0000 7f4e88e5a640 1 --2- 172.21.15.5:0/3455490262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e840cf250 0x7f4e841f83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.736+0000 7f4e88e5a640 1 --2- 172.21.15.5:0/3455490262 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e840b6680 0x7f4e84161c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:18.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.736+0000 7f4e88e5a640 1 -- 172.21.15.5:0/3455490262 >> 172.21.15.5:0/3455490262 conn(0x7f4e840d1860 msgr2=0x7f4e84162fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:18.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.736+0000 7f4e88e5a640 1 -- 172.21.15.5:0/3455490262 shutdown_connections 2024-05-20T18:31:18.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.736+0000 7f4e88e5a640 1 -- 172.21.15.5:0/3455490262 wait complete. 2024-05-20T18:31:18.751 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-20T18:31:18.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.875+0000 7fd0e51fe640 1 Processor -- start 2024-05-20T18:31:18.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.875+0000 7fd0e51fe640 1 -- start start 2024-05-20T18:31:18.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.876+0000 7fd0e51fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0e00d3510 0x7fd0e00b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:18.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.876+0000 7fd0e51fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd0e00cfb80 0x7fd0e006cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:18.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.876+0000 7fd0e51fe640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0e00b4300 con 0x7fd0e00cf060 2024-05-20T18:31:18.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.876+0000 7fd0e51fe640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fd0e015fac0 con 0x7fd0e00cfb80 2024-05-20T18:31:18.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.876+0000 7fd0e51fe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd0e006e090 con 0x7fd0e00d3510 2024-05-20T18:31:18.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.876+0000 7fd0df59e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd0e00cfb80 0x7fd0e006cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:18.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.876+0000 7fd0d659c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd0e00cf060 0x7fd0e00cf470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45410/0 (socket says 172.21.15.5:45410) 2024-05-20T18:31:18.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.876+0000 7fd0d659c640 1 -- 172.21.15.5:0/252914516 learned_addr learned my addr 172.21.15.5:0/252914516 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:18.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.877+0000 7fd0ded9d640 1 --2- 172.21.15.5:0/252914516 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0e00d3510 0x7fd0e00b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:18.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.877+0000 7fd0df59e640 1 -- 172.21.15.5:0/252914516 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0e00d3510 msgr2=0x7fd0e00b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:18.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.877+0000 7fd0df59e640 1 --2- 172.21.15.5:0/252914516 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0e00d3510 0x7fd0e00b3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:18.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.877+0000 7fd0df59e640 1 -- 172.21.15.5:0/252914516 >> v1:172.21.15.5:6789/0 conn(0x7fd0e00cf060 legacy=0x7fd0e00cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:18.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.877+0000 7fd0df59e640 1 -- 172.21.15.5:0/252914516 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e00d14a0 con 0x7fd0e00cfb80 2024-05-20T18:31:18.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.877+0000 7fd0de59c640 1 -- 172.21.15.5:0/252914516 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3299354206 0 0) 0x7fd0c8001110 con 0x7fd0e00cf060 2024-05-20T18:31:18.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.877+0000 7fd0ded9d640 1 --2- 172.21.15.5:0/252914516 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0e00d3510 0x7fd0e00b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:18.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.877+0000 7fd0df59e640 1 --2- 172.21.15.5:0/252914516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0e00cfb80 0x7fd0e006cca0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8009d90 tx=0x7fd0d802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3e2c79eebda6b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:18.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.878+0000 7fd0de59c640 1 -- 172.21.15.5:0/252914516 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8031600 con 0x7fd0e00cfb80 2024-05-20T18:31:18.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.878+0000 7fd0de59c640 1 -- 172.21.15.5:0/252914516 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0d80317a0 con 0x7fd0e00cfb80 2024-05-20T18:31:18.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.878+0000 7fd0de59c640 1 -- 172.21.15.5:0/252914516 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8031b70 con 0x7fd0e00cfb80 2024-05-20T18:31:18.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.878+0000 7fd0e51fe640 1 -- 172.21.15.5:0/252914516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0e00cfb80 msgr2=0x7fd0e006cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:18.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.878+0000 7fd0e51fe640 1 --2- 172.21.15.5:0/252914516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0e00cfb80 0x7fd0e006cca0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8009d90 tx=0x7fd0d802a5d0 comp rx=0 tx=0).stop 2024-05-20T18:31:18.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.878+0000 7fd0e51fe640 1 -- 172.21.15.5:0/252914516 shutdown_connections 2024-05-20T18:31:18.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.878+0000 7fd0e51fe640 1 --2- 172.21.15.5:0/252914516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0e00cfb80 0x7fd0e006cca0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:18.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.878+0000 7fd0e51fe640 1 --2- 172.21.15.5:0/252914516 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0e00d3510 0x7fd0e00b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:18.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.878+0000 7fd0e51fe640 1 -- 172.21.15.5:0/252914516 >> 172.21.15.5:0/252914516 conn(0x7fd0e00d29f0 msgr2=0x7fd0e00d2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:18.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.878+0000 7fd0e51fe640 1 -- 172.21.15.5:0/252914516 shutdown_connections 2024-05-20T18:31:18.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.878+0000 7fd0e51fe640 1 -- 172.21.15.5:0/252914516 wait complete. 2024-05-20T18:31:18.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.880+0000 7fd0e51fe640 1 Processor -- start 2024-05-20T18:31:18.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.881+0000 7fd0e51fe640 1 -- start start 2024-05-20T18:31:18.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.882+0000 7fd0e51fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0e00cf060 0x7fd0e01e9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:18.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.882+0000 7fd0e51fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0e00d3510 0x7fd0e015cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:18.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.882+0000 7fd0e51fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0e015d300 0x7fd0e00af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:18.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.882+0000 7fd0e51fe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e006d1e0 con 0x7fd0e015d300 2024-05-20T18:31:18.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.882+0000 7fd0e51fe640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e00d1af0 con 0x7fd0e00d3510 2024-05-20T18:31:18.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.882+0000 7fd0e51fe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd0e006d360 con 0x7fd0e00cf060 2024-05-20T18:31:18.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.882+0000 7fd0df59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0e015d300 0x7fd0e00af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:18.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.882+0000 7fd0df59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0e015d300 0x7fd0e00af8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37574/0 (socket says 172.21.15.5:37574) 2024-05-20T18:31:18.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.882+0000 7fd0df59e640 1 -- 172.21.15.5:0/3852430330 learned_addr learned my addr 172.21.15.5:0/3852430330 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:18.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.882+0000 7fd0d659c640 1 --2- 172.21.15.5:0/3852430330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0e00d3510 0x7fd0e015cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:18.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.882+0000 7fd0ded9d640 1 --2- 172.21.15.5:0/3852430330 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0e00cf060 0x7fd0e01e9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:18.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.882+0000 7fd0df59e640 1 -- 172.21.15.5:0/3852430330 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0e00cf060 msgr2=0x7fd0e01e9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:18.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.882+0000 7fd0df59e640 1 --2- 172.21.15.5:0/3852430330 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0e00cf060 0x7fd0e01e9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:18.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.882+0000 7fd0df59e640 1 -- 172.21.15.5:0/3852430330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0e00d3510 msgr2=0x7fd0e015cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:18.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.882+0000 7fd0df59e640 1 --2- 172.21.15.5:0/3852430330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0e00d3510 0x7fd0e015cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:18.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.882+0000 7fd0df59e640 1 -- 172.21.15.5:0/3852430330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e01e80d0 con 0x7fd0e015d300 2024-05-20T18:31:18.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.882+0000 7fd0d659c640 1 --2- 172.21.15.5:0/3852430330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0e00d3510 0x7fd0e015cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:18.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.882+0000 7fd0ded9d640 1 --2- 172.21.15.5:0/3852430330 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0e00cf060 0x7fd0e01e9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:18.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.882+0000 7fd0df59e640 1 --2- 172.21.15.5:0/3852430330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0e015d300 0x7fd0e00af8c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8009eb0 tx=0x7fd0d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:18.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.883+0000 7fd0d7fff640 1 -- 172.21.15.5:0/3852430330 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d803f070 con 0x7fd0e015d300 2024-05-20T18:31:18.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.883+0000 7fd0d7fff640 1 -- 172.21.15.5:0/3852430330 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0d8030b60 con 0x7fd0e015d300 2024-05-20T18:31:18.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.883+0000 7fd0e51fe640 1 -- 172.21.15.5:0/3852430330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0e01e83b0 con 0x7fd0e015d300 2024-05-20T18:31:18.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.883+0000 7fd0d7fff640 1 -- 172.21.15.5:0/3852430330 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8009450 con 0x7fd0e015d300 2024-05-20T18:31:18.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.883+0000 7fd0e51fe640 1 -- 172.21.15.5:0/3852430330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0e01e8970 con 0x7fd0e015d300 2024-05-20T18:31:18.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.884+0000 7fd0e51fe640 1 -- 172.21.15.5:0/3852430330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0e01e7e20 con 0x7fd0e015d300 2024-05-20T18:31:18.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.885+0000 7fd0d7fff640 1 -- 172.21.15.5:0/3852430330 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd0d803b410 con 0x7fd0e015d300 2024-05-20T18:31:18.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.885+0000 7fd0d7fff640 1 --2- 172.21.15.5:0/3852430330 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd0c006be30 0x7fd0c006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:18.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.885+0000 7fd0ded9d640 1 --2- 172.21.15.5:0/3852430330 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd0c006be30 0x7fd0c006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:18.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.885+0000 7fd0d7fff640 1 -- 172.21.15.5:0/3852430330 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6332+0+0 (secure 0 0 0) 0x7fd0d80ac320 con 0x7fd0e015d300 2024-05-20T18:31:18.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.886+0000 7fd0ded9d640 1 --2- 172.21.15.5:0/3852430330 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd0c006be30 0x7fd0c006e340 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd0e008bff0 tx=0x7fd0d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:18.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:18.890+0000 7fd0d7fff640 1 -- 172.21.15.5:0/3852430330 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd0d807aaf0 con 0x7fd0e015d300 2024-05-20T18:31:19.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.075+0000 7fd0e51fe640 1 -- 172.21.15.5:0/3852430330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd0e00690b0 con 0x7fd0e015d300 2024-05-20T18:31:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.735+0000 7fd0d7fff640 1 -- 172.21.15.5:0/3852430330 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v67) ==== 181+0+0 (secure 0 0 0) 0x7fd0d807a3f0 con 0x7fd0e015d300 2024-05-20T18:31:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-05-20T18:31:19.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.739+0000 7fd0e51fe640 1 -- 172.21.15.5:0/3852430330 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd0c006be30 msgr2=0x7fd0c006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:19.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.739+0000 7fd0e51fe640 1 --2- 172.21.15.5:0/3852430330 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd0c006be30 0x7fd0c006e340 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd0e008bff0 tx=0x7fd0d0008040 comp rx=0 tx=0).stop 2024-05-20T18:31:19.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.739+0000 7fd0e51fe640 1 -- 172.21.15.5:0/3852430330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0e015d300 msgr2=0x7fd0e00af8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:19.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.739+0000 7fd0e51fe640 1 --2- 172.21.15.5:0/3852430330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0e015d300 0x7fd0e00af8c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8009eb0 tx=0x7fd0d8004a40 comp rx=0 tx=0).stop 2024-05-20T18:31:19.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.739+0000 7fd0e51fe640 1 -- 172.21.15.5:0/3852430330 shutdown_connections 2024-05-20T18:31:19.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.739+0000 7fd0e51fe640 1 --2- 172.21.15.5:0/3852430330 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd0c006be30 0x7fd0c006e340 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:19.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.739+0000 7fd0e51fe640 1 --2- 172.21.15.5:0/3852430330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0e015d300 0x7fd0e00af8c0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.739+0000 7fd0e51fe640 1 --2- 172.21.15.5:0/3852430330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0e00d3510 0x7fd0e015cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.739+0000 7fd0e51fe640 1 --2- 172.21.15.5:0/3852430330 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0e00cf060 0x7fd0e01e9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.739+0000 7fd0e51fe640 1 -- 172.21.15.5:0/3852430330 >> 172.21.15.5:0/3852430330 conn(0x7fd0e00d29f0 msgr2=0x7fd0e00b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.739+0000 7fd0e51fe640 1 -- 172.21.15.5:0/3852430330 shutdown_connections 2024-05-20T18:31:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.740+0000 7fd0e51fe640 1 -- 172.21.15.5:0/3852430330 wait complete. 2024-05-20T18:31:19.756 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-20T18:31:19.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.882+0000 7f3c15267640 1 Processor -- start 2024-05-20T18:31:19.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.882+0000 7f3c15267640 1 -- start start 2024-05-20T18:31:19.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.883+0000 7f3c15267640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c100d45f0 0x7f3c1006d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:19.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.883+0000 7f3c15267640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3c1008bd20 0x7f3c1008c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:19.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.883+0000 7f3c15267640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c1008c670 con 0x7f3c100b3860 2024-05-20T18:31:19.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.883+0000 7f3c15267640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f3c100d0450 con 0x7f3c1008bd20 2024-05-20T18:31:19.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.883+0000 7f3c15267640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3c100d3b10 con 0x7f3c100d45f0 2024-05-20T18:31:19.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.883+0000 7f3c0effd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3c100b3860 0x7f3c1015f230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45414/0 (socket says 172.21.15.5:45414) 2024-05-20T18:31:19.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.884+0000 7f3c0effd640 1 -- 172.21.15.5:0/621969996 learned_addr learned my addr 172.21.15.5:0/621969996 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:19.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.884+0000 7f3c0f7fe640 1 --2- 172.21.15.5:0/621969996 >> v2:172.21.15.63:3300/0 conn(0x7f3c1008bd20 0x7f3c1008c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:19.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.884+0000 7f3c0e7fc640 1 --2- 172.21.15.5:0/621969996 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c100d45f0 0x7f3c1006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:19.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.884+0000 7f3c0dffb640 1 -- 172.21.15.5:0/621969996 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1659596483 0 0) 0x7f3c00001110 con 0x7f3c100b3860 2024-05-20T18:31:19.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.884+0000 7f3c0dffb640 1 -- 172.21.15.5:0/621969996 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3bf8003520 con 0x7f3c100b3860 2024-05-20T18:31:19.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.884+0000 7f3c0dffb640 1 -- 172.21.15.5:0/621969996 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2211473742 0 0) 0x7f3bf8003520 con 0x7f3c100b3860 2024-05-20T18:31:19.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.884+0000 7f3c0dffb640 1 -- 172.21.15.5:0/621969996 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c100d45f0 msgr2=0x7f3c1006d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:19.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.884+0000 7f3c0dffb640 1 --2- 172.21.15.5:0/621969996 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c100d45f0 0x7f3c1006d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:19.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.884+0000 7f3c0dffb640 1 -- 172.21.15.5:0/621969996 >> v2:172.21.15.63:3300/0 conn(0x7f3c1008bd20 msgr2=0x7f3c1008c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:19.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.884+0000 7f3c0dffb640 1 --2- 172.21.15.5:0/621969996 >> v2:172.21.15.63:3300/0 conn(0x7f3c1008bd20 0x7f3c1008c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:19.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.884+0000 7f3c0dffb640 1 -- 172.21.15.5:0/621969996 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c100d3df0 con 0x7f3c100b3860 2024-05-20T18:31:19.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.884+0000 7f3c0dffb640 1 -- 172.21.15.5:0/621969996 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f3c00001840 con 0x7f3c100b3860 2024-05-20T18:31:19.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.884+0000 7f3c0dffb640 1 -- 172.21.15.5:0/621969996 >> v1:172.21.15.5:6789/0 conn(0x7f3c100b3860 legacy=0x7f3c1015f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:19.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.884+0000 7f3c0dffb640 1 --2- 172.21.15.5:0/621969996 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c100d45f0 0x7f3bf80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:19.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.884+0000 7f3c0dffb640 1 --2- 172.21.15.5:0/621969996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bf8003f30 0x7f3bf8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:19.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.884+0000 7f3c0dffb640 1 --2- 172.21.15.5:0/621969996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bf8004d40 0x7f3bf8005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:19.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.884+0000 7f3c0dffb640 1 -- 172.21.15.5:0/621969996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3c100d0450 con 0x7f3bf8003f30 2024-05-20T18:31:19.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.884+0000 7f3c0dffb640 1 -- 172.21.15.5:0/621969996 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3c100d3b10 con 0x7f3bf8004d40 2024-05-20T18:31:19.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.884+0000 7f3c0dffb640 1 -- 172.21.15.5:0/621969996 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3bf800bb50 con 0x7f3c100d45f0 2024-05-20T18:31:19.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.884+0000 7f3c0e7fc640 1 --2- 172.21.15.5:0/621969996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bf8003f30 0x7f3bf8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:19.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.884+0000 7f3c0effd640 1 --2- 172.21.15.5:0/621969996 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c100d45f0 0x7f3bf80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:19.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.884+0000 7f3c0f7fe640 1 --2- 172.21.15.5:0/621969996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bf8004d40 0x7f3bf8005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:19.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.885+0000 7f3c0e7fc640 1 -- 172.21.15.5:0/621969996 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c100d45f0 msgr2=0x7f3bf80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:19.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.885+0000 7f3c0e7fc640 1 --2- 172.21.15.5:0/621969996 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c100d45f0 0x7f3bf80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:19.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.885+0000 7f3c0e7fc640 1 -- 172.21.15.5:0/621969996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bf8004d40 msgr2=0x7f3bf8005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:19.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.885+0000 7f3c0e7fc640 1 --2- 172.21.15.5:0/621969996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bf8004d40 0x7f3bf8005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:19.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.885+0000 7f3c0e7fc640 1 -- 172.21.15.5:0/621969996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf8003520 con 0x7f3bf8003f30 2024-05-20T18:31:19.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.885+0000 7f3c0e7fc640 1 --2- 172.21.15.5:0/621969996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bf8003f30 0x7f3bf8004800 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f3c10068200 tx=0x7f3bf8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5379ac7d64ae37b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:19.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.885+0000 7f3c0dffb640 1 -- 172.21.15.5:0/621969996 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04036070 con 0x7f3bf8003f30 2024-05-20T18:31:19.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.886+0000 7f3c0dffb640 1 -- 172.21.15.5:0/621969996 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c04031400 con 0x7f3bf8003f30 2024-05-20T18:31:19.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.886+0000 7f3c0dffb640 1 -- 172.21.15.5:0/621969996 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04009690 con 0x7f3bf8003f30 2024-05-20T18:31:19.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.886+0000 7f3c15267640 1 -- 172.21.15.5:0/621969996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bf8003f30 msgr2=0x7f3bf8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:19.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.886+0000 7f3c15267640 1 --2- 172.21.15.5:0/621969996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bf8003f30 0x7f3bf8004800 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f3c10068200 tx=0x7f3bf8005830 comp rx=0 tx=0).stop 2024-05-20T18:31:19.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.886+0000 7f3c0f7fe640 1 -- 172.21.15.5:0/621969996 reap_dead start 2024-05-20T18:31:19.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.886+0000 7f3c15267640 1 -- 172.21.15.5:0/621969996 shutdown_connections 2024-05-20T18:31:19.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.886+0000 7f3c15267640 1 -- 172.21.15.5:0/621969996 >> 172.21.15.5:0/621969996 conn(0x7f3c100b6d10 msgr2=0x7f3c1006d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:19.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.886+0000 7f3c15267640 1 -- 172.21.15.5:0/621969996 shutdown_connections 2024-05-20T18:31:19.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.886+0000 7f3c15267640 1 -- 172.21.15.5:0/621969996 wait complete. 2024-05-20T18:31:19.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.888+0000 7f3c15267640 1 Processor -- start 2024-05-20T18:31:19.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.888+0000 7f3c15267640 1 -- start start 2024-05-20T18:31:19.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.890+0000 7f3c15267640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bf8003f30 0x7f3c101f54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:19.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.890+0000 7f3c15267640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c100ce190 0x7f3c100ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:19.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.890+0000 7f3c15267640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c100c98b0 0x7f3c100c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:19.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.890+0000 7f3c15267640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1006d870 con 0x7f3c100ce190 2024-05-20T18:31:19.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.890+0000 7f3c15267640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3c100d0730 con 0x7f3bf8003f30 2024-05-20T18:31:19.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.890+0000 7f3c15267640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3c1006d9f0 con 0x7f3c100c98b0 2024-05-20T18:31:19.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.890+0000 7f3c0e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c100ce190 0x7f3c100ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:19.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.890+0000 7f3c0e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c100ce190 0x7f3c100ce5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37598/0 (socket says 172.21.15.5:37598) 2024-05-20T18:31:19.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.890+0000 7f3c0e7fc640 1 -- 172.21.15.5:0/3628370326 learned_addr learned my addr 172.21.15.5:0/3628370326 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:19.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.890+0000 7f3c0effd640 1 --2- 172.21.15.5:0/3628370326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bf8003f30 0x7f3c101f54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:19.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.890+0000 7f3c0f7fe640 1 --2- 172.21.15.5:0/3628370326 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c100c98b0 0x7f3c100c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:19.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.890+0000 7f3c0e7fc640 1 -- 172.21.15.5:0/3628370326 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c100c98b0 msgr2=0x7f3c100c9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:19.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.890+0000 7f3c0e7fc640 1 --2- 172.21.15.5:0/3628370326 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c100c98b0 0x7f3c100c9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:19.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.890+0000 7f3c0e7fc640 1 -- 172.21.15.5:0/3628370326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bf8003f30 msgr2=0x7f3c101f54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:19.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.890+0000 7f3c0e7fc640 1 --2- 172.21.15.5:0/3628370326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bf8003f30 0x7f3c101f54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:19.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.890+0000 7f3c0e7fc640 1 -- 172.21.15.5:0/3628370326 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c100cede0 con 0x7f3c100ce190 2024-05-20T18:31:19.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.890+0000 7f3c0effd640 1 --2- 172.21.15.5:0/3628370326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bf8003f30 0x7f3c101f54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:19.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.890+0000 7f3c0e7fc640 1 --2- 172.21.15.5:0/3628370326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c100ce190 0x7f3c100ce5a0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f3c04034040 tx=0x7f3c04004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:19.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.890+0000 7f3bf77fe640 1 -- 172.21.15.5:0/3628370326 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04040020 con 0x7f3c100ce190 2024-05-20T18:31:19.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.890+0000 7f3bf77fe640 1 -- 172.21.15.5:0/3628370326 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c0403b6a0 con 0x7f3c100ce190 2024-05-20T18:31:19.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.891+0000 7f3c15267640 1 -- 172.21.15.5:0/3628370326 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c100cf0c0 con 0x7f3c100ce190 2024-05-20T18:31:19.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.891+0000 7f3bf77fe640 1 -- 172.21.15.5:0/3628370326 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04036040 con 0x7f3c100ce190 2024-05-20T18:31:19.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.891+0000 7f3c15267640 1 -- 172.21.15.5:0/3628370326 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c100c9390 con 0x7f3c100ce190 2024-05-20T18:31:19.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.891+0000 7f3c15267640 1 -- 172.21.15.5:0/3628370326 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c1006c2c0 con 0x7f3c100ce190 2024-05-20T18:31:19.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.892+0000 7f3bf77fe640 1 -- 172.21.15.5:0/3628370326 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3c04043020 con 0x7f3c100ce190 2024-05-20T18:31:19.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.893+0000 7f3bf77fe640 1 --2- 172.21.15.5:0/3628370326 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3bfc06be30 0x7f3bfc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:19.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.893+0000 7f3bf77fe640 1 -- 172.21.15.5:0/3628370326 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5865+0+0 (secure 0 0 0) 0x7f3c040ab700 con 0x7f3c100ce190 2024-05-20T18:31:19.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.893+0000 7f3c0effd640 1 --2- 172.21.15.5:0/3628370326 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3bfc06be30 0x7f3bfc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:19.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.893+0000 7f3c0effd640 1 --2- 172.21.15.5:0/3628370326 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3bfc06be30 0x7f3bfc06e340 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3c100d15d0 tx=0x7f3c00009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:19.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:19.897+0000 7f3bf77fe640 1 -- 172.21.15.5:0/3628370326 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3c1006c2c0 con 0x7f3c100ce190 2024-05-20T18:31:20.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.087+0000 7f3c15267640 1 -- 172.21.15.5:0/3628370326 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3c100690b0 con 0x7f3c100ce190 2024-05-20T18:31:20.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.736+0000 7f3bf77fe640 1 -- 172.21.15.5:0/3628370326 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v68) ==== 169+0+0 (secure 0 0 0) 0x7f3c0404a030 con 0x7f3c100ce190 2024-05-20T18:31:20.737 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-05-20T18:31:20.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.741+0000 7f3c15267640 1 -- 172.21.15.5:0/3628370326 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3bfc06be30 msgr2=0x7f3bfc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:20.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.741+0000 7f3c15267640 1 --2- 172.21.15.5:0/3628370326 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3bfc06be30 0x7f3bfc06e340 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3c100d15d0 tx=0x7f3c00009040 comp rx=0 tx=0).stop 2024-05-20T18:31:20.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.741+0000 7f3c15267640 1 -- 172.21.15.5:0/3628370326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c100ce190 msgr2=0x7f3c100ce5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:20.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.741+0000 7f3c15267640 1 --2- 172.21.15.5:0/3628370326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c100ce190 0x7f3c100ce5a0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f3c04034040 tx=0x7f3c04004730 comp rx=0 tx=0).stop 2024-05-20T18:31:20.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.741+0000 7f3c15267640 1 -- 172.21.15.5:0/3628370326 shutdown_connections 2024-05-20T18:31:20.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.741+0000 7f3c15267640 1 --2- 172.21.15.5:0/3628370326 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3bfc06be30 0x7f3bfc06e340 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:20.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.741+0000 7f3c15267640 1 --2- 172.21.15.5:0/3628370326 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c100c98b0 0x7f3c100c9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:20.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.741+0000 7f3c15267640 1 --2- 172.21.15.5:0/3628370326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c100ce190 0x7f3c100ce5a0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:20.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.741+0000 7f3c15267640 1 --2- 172.21.15.5:0/3628370326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3bf8003f30 0x7f3c101f54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:20.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.741+0000 7f3c15267640 1 -- 172.21.15.5:0/3628370326 >> 172.21.15.5:0/3628370326 conn(0x7f3c100b6d10 msgr2=0x7f3c1015cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:20.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.741+0000 7f3c15267640 1 -- 172.21.15.5:0/3628370326 shutdown_connections 2024-05-20T18:31:20.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.741+0000 7f3c15267640 1 -- 172.21.15.5:0/3628370326 wait complete. 2024-05-20T18:31:20.758 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-20T18:31:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.882+0000 7f499a00d640 1 Processor -- start 2024-05-20T18:31:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.882+0000 7f499a00d640 1 -- start start 2024-05-20T18:31:20.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.883+0000 7f499a00d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f499415d5f0 0x7f49941576d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:20.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.883+0000 7f499a00d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f49940cedd0 0x7f499406e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:20.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.883+0000 7f499a00d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f499406b900 con 0x7f499406c9a0 2024-05-20T18:31:20.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.883+0000 7f499a00d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f499406bbe0 con 0x7f49940cedd0 2024-05-20T18:31:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.883+0000 7f499a00d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f499406bec0 con 0x7f499415d5f0 2024-05-20T18:31:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.883+0000 7f4993fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f49940cedd0 0x7f499406e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.883+0000 7f4992ffd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f499406c9a0 0x7f49940ce6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45430/0 (socket says 172.21.15.5:45430) 2024-05-20T18:31:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.884+0000 7f4992ffd640 1 -- 172.21.15.5:0/508128751 learned_addr learned my addr 172.21.15.5:0/508128751 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.884+0000 7f49937fe640 1 --2- 172.21.15.5:0/508128751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f499415d5f0 0x7f49941576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.884+0000 7f4993fff640 1 -- 172.21.15.5:0/508128751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f499415d5f0 msgr2=0x7f49941576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.884+0000 7f4993fff640 1 --2- 172.21.15.5:0/508128751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f499415d5f0 0x7f49941576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.884+0000 7f4993fff640 1 -- 172.21.15.5:0/508128751 >> v1:172.21.15.5:6789/0 conn(0x7f499406c9a0 legacy=0x7f49940ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.884+0000 7f49927fc640 1 -- 172.21.15.5:0/508128751 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4181673635 0 0) 0x7f499406b900 con 0x7f499406c9a0 2024-05-20T18:31:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.884+0000 7f4993fff640 1 -- 172.21.15.5:0/508128751 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f499406c1a0 con 0x7f49940cedd0 2024-05-20T18:31:20.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.885+0000 7f4993fff640 1 --2- 172.21.15.5:0/508128751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49940cedd0 0x7f499406e3a0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f498c009d90 tx=0x7f498c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=679ab7939eab3da7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:20.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.885+0000 7f49927fc640 1 -- 172.21.15.5:0/508128751 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c031600 con 0x7f49940cedd0 2024-05-20T18:31:20.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.885+0000 7f49927fc640 1 -- 172.21.15.5:0/508128751 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f498c0317a0 con 0x7f49940cedd0 2024-05-20T18:31:20.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.885+0000 7f49927fc640 1 -- 172.21.15.5:0/508128751 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c031b70 con 0x7f49940cedd0 2024-05-20T18:31:20.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.885+0000 7f499a00d640 1 -- 172.21.15.5:0/508128751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49940cedd0 msgr2=0x7f499406e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:20.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.885+0000 7f499a00d640 1 --2- 172.21.15.5:0/508128751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49940cedd0 0x7f499406e3a0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f498c009d90 tx=0x7f498c02a5d0 comp rx=0 tx=0).stop 2024-05-20T18:31:20.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.886+0000 7f499a00d640 1 -- 172.21.15.5:0/508128751 shutdown_connections 2024-05-20T18:31:20.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.886+0000 7f499a00d640 1 --2- 172.21.15.5:0/508128751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49940cedd0 0x7f499406e3a0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:20.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.886+0000 7f499a00d640 1 --2- 172.21.15.5:0/508128751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f499415d5f0 0x7f49941576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:20.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.886+0000 7f499a00d640 1 -- 172.21.15.5:0/508128751 >> 172.21.15.5:0/508128751 conn(0x7f49940cd2a0 msgr2=0x7f49940cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:20.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.886+0000 7f499a00d640 1 -- 172.21.15.5:0/508128751 shutdown_connections 2024-05-20T18:31:20.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.886+0000 7f499a00d640 1 -- 172.21.15.5:0/508128751 wait complete. 2024-05-20T18:31:20.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.887+0000 7f499a00d640 1 Processor -- start 2024-05-20T18:31:20.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.888+0000 7f499a00d640 1 -- start start 2024-05-20T18:31:20.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.889+0000 7f499a00d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f499406c9a0 0x7f499415c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:20.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.889+0000 7f499a00d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49940cedd0 0x7f49940ae750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:20.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.889+0000 7f499a00d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f499415d5f0 0x7f49940b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:20.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.889+0000 7f499a00d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f49940cf1e0 con 0x7f499406c9a0 2024-05-20T18:31:20.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.889+0000 7f499a00d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f499406cdb0 con 0x7f499415d5f0 2024-05-20T18:31:20.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.889+0000 7f499a00d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f499406b520 con 0x7f49940cedd0 2024-05-20T18:31:20.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.889+0000 7f49937fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f499406c9a0 0x7f499415c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:20.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.889+0000 7f49937fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f499406c9a0 0x7f499415c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37608/0 (socket says 172.21.15.5:37608) 2024-05-20T18:31:20.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.889+0000 7f49937fe640 1 -- 172.21.15.5:0/2276035436 learned_addr learned my addr 172.21.15.5:0/2276035436 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:20.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.889+0000 7f49937fe640 1 -- 172.21.15.5:0/2276035436 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49940cedd0 msgr2=0x7f49940ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:20.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.889+0000 7f49937fe640 1 --2- 172.21.15.5:0/2276035436 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49940cedd0 0x7f49940ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:20.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.889+0000 7f49937fe640 1 -- 172.21.15.5:0/2276035436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f499415d5f0 msgr2=0x7f49940b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:20.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.889+0000 7f49937fe640 1 --2- 172.21.15.5:0/2276035436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f499415d5f0 0x7f49940b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:20.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.889+0000 7f49937fe640 1 -- 172.21.15.5:0/2276035436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49940af440 con 0x7f499406c9a0 2024-05-20T18:31:20.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.890+0000 7f49937fe640 1 --2- 172.21.15.5:0/2276035436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f499406c9a0 0x7f499415c280 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f498400e870 tx=0x7f498400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:20.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.890+0000 7f497bfff640 1 -- 172.21.15.5:0/2276035436 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4984017020 con 0x7f499406c9a0 2024-05-20T18:31:20.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.890+0000 7f497bfff640 1 -- 172.21.15.5:0/2276035436 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4984004240 con 0x7f499406c9a0 2024-05-20T18:31:20.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.890+0000 7f497bfff640 1 -- 172.21.15.5:0/2276035436 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498400c730 con 0x7f499406c9a0 2024-05-20T18:31:20.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.890+0000 7f499a00d640 1 -- 172.21.15.5:0/2276035436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49940b3a60 con 0x7f499406c9a0 2024-05-20T18:31:20.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.890+0000 7f499a00d640 1 -- 172.21.15.5:0/2276035436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49940af720 con 0x7f499406c9a0 2024-05-20T18:31:20.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.891+0000 7f499a00d640 1 -- 172.21.15.5:0/2276035436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4994075b00 con 0x7f499406c9a0 2024-05-20T18:31:20.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.891+0000 7f497bfff640 1 -- 172.21.15.5:0/2276035436 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f498401a020 con 0x7f499406c9a0 2024-05-20T18:31:20.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.892+0000 7f497bfff640 1 --2- 172.21.15.5:0/2276035436 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f498006be60 0x7f498006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:20.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.892+0000 7f497bfff640 1 -- 172.21.15.5:0/2276035436 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5335+0+0 (secure 0 0 0) 0x7f498408ae70 con 0x7f499406c9a0 2024-05-20T18:31:20.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.892+0000 7f4992ffd640 1 --2- 172.21.15.5:0/2276035436 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f498006be60 0x7f498006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:20.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.892+0000 7f4992ffd640 1 --2- 172.21.15.5:0/2276035436 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f498006be60 0x7f498006e370 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4988007fd0 tx=0x7f4988007960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:20.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:20.897+0000 7f497bfff640 1 -- 172.21.15.5:0/2276035436 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4984021030 con 0x7f499406c9a0 2024-05-20T18:31:21.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.084+0000 7f499a00d640 1 -- 172.21.15.5:0/2276035436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f49941607b0 con 0x7f499406c9a0 2024-05-20T18:31:21.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.088+0000 7f497bfff640 1 -- 172.21.15.5:0/2276035436 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7f4984059900 con 0x7f499406c9a0 2024-05-20T18:31:21.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.090+0000 7f499a00d640 1 -- 172.21.15.5:0/2276035436 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f498006be60 msgr2=0x7f498006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:21.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.090+0000 7f499a00d640 1 --2- 172.21.15.5:0/2276035436 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f498006be60 0x7f498006e370 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4988007fd0 tx=0x7f4988007960 comp rx=0 tx=0).stop 2024-05-20T18:31:21.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.090+0000 7f499a00d640 1 -- 172.21.15.5:0/2276035436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f499406c9a0 msgr2=0x7f499415c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:21.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.090+0000 7f499a00d640 1 --2- 172.21.15.5:0/2276035436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f499406c9a0 0x7f499415c280 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f498400e870 tx=0x7f498400ed30 comp rx=0 tx=0).stop 2024-05-20T18:31:21.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.090+0000 7f499a00d640 1 -- 172.21.15.5:0/2276035436 shutdown_connections 2024-05-20T18:31:21.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.090+0000 7f499a00d640 1 --2- 172.21.15.5:0/2276035436 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f498006be60 0x7f498006e370 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:21.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.090+0000 7f499a00d640 1 --2- 172.21.15.5:0/2276035436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f499415d5f0 0x7f49940b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:21.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.090+0000 7f499a00d640 1 --2- 172.21.15.5:0/2276035436 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49940cedd0 0x7f49940ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:21.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.090+0000 7f499a00d640 1 --2- 172.21.15.5:0/2276035436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f499406c9a0 0x7f499415c280 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:21.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.090+0000 7f499a00d640 1 -- 172.21.15.5:0/2276035436 >> 172.21.15.5:0/2276035436 conn(0x7f49940cd2a0 msgr2=0x7f499415da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:21.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.090+0000 7f499a00d640 1 -- 172.21.15.5:0/2276035436 shutdown_connections 2024-05-20T18:31:21.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.090+0000 7f499a00d640 1 -- 172.21.15.5:0/2276035436 wait complete. 2024-05-20T18:31:21.102 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-05-20T18:31:21.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.219+0000 7f8b7b50f640 1 Processor -- start 2024-05-20T18:31:21.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.220+0000 7f8b7b50f640 1 -- start start 2024-05-20T18:31:21.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.221+0000 7f8b7b50f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8b7409ac00 0x7f8b740d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:21.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.221+0000 7f8b7b50f640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b74160700 con 0x7f8b741602f0 2024-05-20T18:31:21.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.221+0000 7f8b7b50f640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b740af9d0 con 0x7f8b7406da80 2024-05-20T18:31:21.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.221+0000 7f8b7b50f640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f8b74159da0 con 0x7f8b7409ac00 2024-05-20T18:31:21.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.221+0000 7f8b79aac640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8b7409ac00 0x7f8b740d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:21.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.221+0000 7f8b78aaa640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8b741602f0 0x7f8b7409a4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45436/0 (socket says 172.21.15.5:45436) 2024-05-20T18:31:21.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.221+0000 7f8b78aaa640 1 -- 172.21.15.5:0/1091468878 learned_addr learned my addr 172.21.15.5:0/1091468878 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:21.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.222+0000 7f8b79aac640 1 -- 172.21.15.5:0/1091468878 >> v1:172.21.15.63:6789/0 conn(0x7f8b7406da80 legacy=0x7f8b740b1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:21.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.222+0000 7f8b79aac640 1 -- 172.21.15.5:0/1091468878 >> v1:172.21.15.5:6789/0 conn(0x7f8b741602f0 legacy=0x7f8b7409a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:21.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.222+0000 7f8b79aac640 1 -- 172.21.15.5:0/1091468878 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b740af410 con 0x7f8b7409ac00 2024-05-20T18:31:21.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.223+0000 7f8b79aac640 1 --2- 172.21.15.5:0/1091468878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b7409ac00 0x7f8b740d1c40 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c0097a0 tx=0x7f8b6c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89ead81f0ab6e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:21.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.223+0000 7f8b6bfff640 1 -- 172.21.15.5:0/1091468878 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c037070 con 0x7f8b7409ac00 2024-05-20T18:31:21.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.223+0000 7f8b6bfff640 1 -- 172.21.15.5:0/1091468878 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b6c02ae50 con 0x7f8b7409ac00 2024-05-20T18:31:21.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.223+0000 7f8b6bfff640 1 -- 172.21.15.5:0/1091468878 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c032660 con 0x7f8b7409ac00 2024-05-20T18:31:21.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.223+0000 7f8b7b50f640 1 -- 172.21.15.5:0/1091468878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b7409ac00 msgr2=0x7f8b740d1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:21.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.223+0000 7f8b7b50f640 1 --2- 172.21.15.5:0/1091468878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b7409ac00 0x7f8b740d1c40 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c0097a0 tx=0x7f8b6c02a140 comp rx=0 tx=0).stop 2024-05-20T18:31:21.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.223+0000 7f8b7b50f640 1 -- 172.21.15.5:0/1091468878 shutdown_connections 2024-05-20T18:31:21.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.223+0000 7f8b7b50f640 1 --2- 172.21.15.5:0/1091468878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b7409ac00 0x7f8b740d1c40 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:21.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.223+0000 7f8b7b50f640 1 -- 172.21.15.5:0/1091468878 >> 172.21.15.5:0/1091468878 conn(0x7f8b740b6670 msgr2=0x7f8b740b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:21.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.224+0000 7f8b7b50f640 1 -- 172.21.15.5:0/1091468878 shutdown_connections 2024-05-20T18:31:21.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.224+0000 7f8b7b50f640 1 -- 172.21.15.5:0/1091468878 wait complete. 2024-05-20T18:31:21.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.225+0000 7f8b7b50f640 1 Processor -- start 2024-05-20T18:31:21.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.225+0000 7f8b7b50f640 1 -- start start 2024-05-20T18:31:21.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.227+0000 7f8b7b50f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b7406da80 0x7f8b7409f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:21.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.227+0000 7f8b7b50f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b741602f0 0x7f8b741e9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:21.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.227+0000 7f8b7b50f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b741ea260 0x7f8b7415f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:21.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.227+0000 7f8b7b50f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8b740d17b0 con 0x7f8b741602f0 2024-05-20T18:31:21.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.227+0000 7f8b7b50f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8b740af1c0 con 0x7f8b7406da80 2024-05-20T18:31:21.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.227+0000 7f8b7b50f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8b740b2390 con 0x7f8b741ea260 2024-05-20T18:31:21.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.227+0000 7f8b78aaa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b741602f0 0x7f8b741e9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:21.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.227+0000 7f8b78aaa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b741602f0 0x7f8b741e9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37630/0 (socket says 172.21.15.5:37630) 2024-05-20T18:31:21.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.227+0000 7f8b78aaa640 1 -- 172.21.15.5:0/867953258 learned_addr learned my addr 172.21.15.5:0/867953258 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:21.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.227+0000 7f8b792ab640 1 --2- 172.21.15.5:0/867953258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b7406da80 0x7f8b7409f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:21.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.227+0000 7f8b79aac640 1 --2- 172.21.15.5:0/867953258 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b741ea260 0x7f8b7415f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:21.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.227+0000 7f8b78aaa640 1 -- 172.21.15.5:0/867953258 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b741ea260 msgr2=0x7f8b7415f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:21.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.227+0000 7f8b78aaa640 1 --2- 172.21.15.5:0/867953258 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b741ea260 0x7f8b7415f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:21.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.227+0000 7f8b78aaa640 1 -- 172.21.15.5:0/867953258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b7406da80 msgr2=0x7f8b7409f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:21.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.227+0000 7f8b78aaa640 1 --2- 172.21.15.5:0/867953258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b7406da80 0x7f8b7409f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:21.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.227+0000 7f8b78aaa640 1 -- 172.21.15.5:0/867953258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7409cff0 con 0x7f8b741602f0 2024-05-20T18:31:21.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.227+0000 7f8b78aaa640 1 --2- 172.21.15.5:0/867953258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b741602f0 0x7f8b741e9d20 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400c3c0 tx=0x7f8b6400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:21.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.227+0000 7f8b69ffb640 1 -- 172.21.15.5:0/867953258 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b64017070 con 0x7f8b741602f0 2024-05-20T18:31:21.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.228+0000 7f8b69ffb640 1 -- 172.21.15.5:0/867953258 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b640074e0 con 0x7f8b741602f0 2024-05-20T18:31:21.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.228+0000 7f8b7b50f640 1 -- 172.21.15.5:0/867953258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b7409d2d0 con 0x7f8b741602f0 2024-05-20T18:31:21.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.228+0000 7f8b69ffb640 1 -- 172.21.15.5:0/867953258 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b640059a0 con 0x7f8b741602f0 2024-05-20T18:31:21.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.228+0000 7f8b7b50f640 1 -- 172.21.15.5:0/867953258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b7409d890 con 0x7f8b741602f0 2024-05-20T18:31:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.229+0000 7f8b7b50f640 1 -- 172.21.15.5:0/867953258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b74075b00 con 0x7f8b741602f0 2024-05-20T18:31:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.229+0000 7f8b69ffb640 1 -- 172.21.15.5:0/867953258 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8b6401a070 con 0x7f8b741602f0 2024-05-20T18:31:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.230+0000 7f8b69ffb640 1 --2- 172.21.15.5:0/867953258 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8b5806be30 0x7f8b5806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.230+0000 7f8b69ffb640 1 -- 172.21.15.5:0/867953258 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5335+0+0 (secure 0 0 0) 0x7f8b64089f70 con 0x7f8b741602f0 2024-05-20T18:31:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.230+0000 7f8b792ab640 1 --2- 172.21.15.5:0/867953258 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8b5806be30 0x7f8b5806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:21.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.231+0000 7f8b792ab640 1 --2- 172.21.15.5:0/867953258 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8b5806be30 0x7f8b5806e340 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8b740b6ca0 tx=0x7f8b7000d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:21.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.235+0000 7f8b69ffb640 1 -- 172.21.15.5:0/867953258 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8b74160700 con 0x7f8b741602f0 2024-05-20T18:31:21.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.493+0000 7f8b7b50f640 1 -- 172.21.15.5:0/867953258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7f8b740aec70 con 0x7f8b741602f0 2024-05-20T18:31:21.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.738+0000 7f8b69ffb640 1 -- 172.21.15.5:0/867953258 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v224) ==== 169+0+0 (secure 0 0 0) 0x7f8b64058440 con 0x7f8b741602f0 2024-05-20T18:31:21.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.740+0000 7f8b7b50f640 1 -- 172.21.15.5:0/867953258 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8b5806be30 msgr2=0x7f8b5806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:21.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.740+0000 7f8b7b50f640 1 --2- 172.21.15.5:0/867953258 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8b5806be30 0x7f8b5806e340 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8b740b6ca0 tx=0x7f8b7000d040 comp rx=0 tx=0).stop 2024-05-20T18:31:21.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.740+0000 7f8b7b50f640 1 -- 172.21.15.5:0/867953258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b741602f0 msgr2=0x7f8b741e9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:21.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.740+0000 7f8b7b50f640 1 --2- 172.21.15.5:0/867953258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b741602f0 0x7f8b741e9d20 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400c3c0 tx=0x7f8b6400c7c0 comp rx=0 tx=0).stop 2024-05-20T18:31:21.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.741+0000 7f8b7b50f640 1 -- 172.21.15.5:0/867953258 shutdown_connections 2024-05-20T18:31:21.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.741+0000 7f8b7b50f640 1 --2- 172.21.15.5:0/867953258 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8b5806be30 0x7f8b5806e340 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:21.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.741+0000 7f8b7b50f640 1 --2- 172.21.15.5:0/867953258 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b741ea260 0x7f8b7415f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:21.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.741+0000 7f8b7b50f640 1 --2- 172.21.15.5:0/867953258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b741602f0 0x7f8b741e9d20 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:21.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.741+0000 7f8b7b50f640 1 --2- 172.21.15.5:0/867953258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b7406da80 0x7f8b7409f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:21.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.741+0000 7f8b7b50f640 1 -- 172.21.15.5:0/867953258 >> 172.21.15.5:0/867953258 conn(0x7f8b740b6670 msgr2=0x7f8b7406ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:21.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.741+0000 7f8b7b50f640 1 -- 172.21.15.5:0/867953258 shutdown_connections 2024-05-20T18:31:21.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.741+0000 7f8b7b50f640 1 -- 172.21.15.5:0/867953258 wait complete. 2024-05-20T18:31:21.753 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-05-20T18:31:21.753 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-05-20T18:31:21.754 INFO:tasks.cephfs_test_runner:test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-20T18:31:21.754 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:31:21.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.869+0000 7fedee1ea640 1 Processor -- start 2024-05-20T18:31:21.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.869+0000 7fedee1ea640 1 -- start start 2024-05-20T18:31:21.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.871+0000 7fedee1ea640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fede806ca80 0x7fede815f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:21.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.871+0000 7fedee1ea640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fede80ae730 0x7fede806c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:21.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.871+0000 7fedee1ea640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fede80b1b80 con 0x7fede80ae320 2024-05-20T18:31:21.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.871+0000 7fedee1ea640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fede806b750 con 0x7fede80ae730 2024-05-20T18:31:21.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.871+0000 7fedee1ea640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fede80b12e0 con 0x7fede806ca80 2024-05-20T18:31:21.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.871+0000 7fede6ffd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fede80ae320 0x7fede806bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45450/0 (socket says 172.21.15.5:45450) 2024-05-20T18:31:21.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.871+0000 7fede6ffd640 1 -- 172.21.15.5:0/340492960 learned_addr learned my addr 172.21.15.5:0/340492960 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:21.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.871+0000 7fede77fe640 1 --2- 172.21.15.5:0/340492960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fede806ca80 0x7fede815f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:21.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.871+0000 7fede7fff640 1 --2- 172.21.15.5:0/340492960 >> v2:172.21.15.63:3300/0 conn(0x7fede80ae730 0x7fede806c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:21.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.871+0000 7fede67fc640 1 -- 172.21.15.5:0/340492960 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2029725931 0 0) 0x7feddc001110 con 0x7fede80ae320 2024-05-20T18:31:21.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.871+0000 7fede67fc640 1 -- 172.21.15.5:0/340492960 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fedd0003520 con 0x7fede80ae320 2024-05-20T18:31:21.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.871+0000 7fede67fc640 1 -- 172.21.15.5:0/340492960 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3877165077 0 0) 0x7fedd0003520 con 0x7fede80ae320 2024-05-20T18:31:21.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.871+0000 7fede67fc640 1 -- 172.21.15.5:0/340492960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fede806ca80 msgr2=0x7fede815f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:21.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.871+0000 7fede67fc640 1 --2- 172.21.15.5:0/340492960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fede806ca80 0x7fede815f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:21.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.871+0000 7fede67fc640 1 -- 172.21.15.5:0/340492960 >> v2:172.21.15.63:3300/0 conn(0x7fede80ae730 msgr2=0x7fede806c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:21.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.871+0000 7fede67fc640 1 --2- 172.21.15.5:0/340492960 >> v2:172.21.15.63:3300/0 conn(0x7fede80ae730 0x7fede806c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:21.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.871+0000 7fede67fc640 1 -- 172.21.15.5:0/340492960 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede80b15c0 con 0x7fede80ae320 2024-05-20T18:31:21.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.871+0000 7fede67fc640 1 -- 172.21.15.5:0/340492960 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7feddc001840 con 0x7fede80ae320 2024-05-20T18:31:21.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.871+0000 7fede67fc640 1 -- 172.21.15.5:0/340492960 >> v1:172.21.15.5:6789/0 conn(0x7fede80ae320 legacy=0x7fede806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:21.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.871+0000 7fede67fc640 1 --2- 172.21.15.5:0/340492960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fede806ca80 0x7fedd00039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:21.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.872+0000 7fede67fc640 1 --2- 172.21.15.5:0/340492960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fedd0003f50 0x7fedd0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:21.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.872+0000 7fede67fc640 1 --2- 172.21.15.5:0/340492960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedd0004d40 0x7fedd0005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:21.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.872+0000 7fede67fc640 1 -- 172.21.15.5:0/340492960 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fede806b750 con 0x7fedd0004d40 2024-05-20T18:31:21.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.872+0000 7fede67fc640 1 -- 172.21.15.5:0/340492960 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fede80b12e0 con 0x7fedd0003f50 2024-05-20T18:31:21.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.872+0000 7fede67fc640 1 -- 172.21.15.5:0/340492960 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fedd000ba40 con 0x7fede806ca80 2024-05-20T18:31:21.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.872+0000 7fede7fff640 1 --2- 172.21.15.5:0/340492960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedd0004d40 0x7fedd0005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:21.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.872+0000 7fede77fe640 1 --2- 172.21.15.5:0/340492960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fede806ca80 0x7fedd00039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:21.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.872+0000 7fede6ffd640 1 --2- 172.21.15.5:0/340492960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fedd0003f50 0x7fedd0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:21.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.872+0000 7fede7fff640 1 -- 172.21.15.5:0/340492960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fede806ca80 msgr2=0x7fedd00039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:21.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.872+0000 7fede7fff640 1 --2- 172.21.15.5:0/340492960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fede806ca80 0x7fedd00039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:21.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.872+0000 7fede7fff640 1 -- 172.21.15.5:0/340492960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fedd0003f50 msgr2=0x7fedd0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:21.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.872+0000 7fede7fff640 1 --2- 172.21.15.5:0/340492960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fedd0003f50 0x7fedd0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:21.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.872+0000 7fede7fff640 1 -- 172.21.15.5:0/340492960 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd0003520 con 0x7fedd0004d40 2024-05-20T18:31:21.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.872+0000 7fede6ffd640 1 --2- 172.21.15.5:0/340492960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fedd0003f50 0x7fedd0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.873+0000 7fede7fff640 1 --2- 172.21.15.5:0/340492960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedd0004d40 0x7fedd0005150 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fede80753b0 tx=0x7fedd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba66ab436b53d867 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.873+0000 7fede67fc640 1 -- 172.21.15.5:0/340492960 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0009c10 con 0x7fedd0004d40 2024-05-20T18:31:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.873+0000 7fede67fc640 1 -- 172.21.15.5:0/340492960 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fede0036070 con 0x7fedd0004d40 2024-05-20T18:31:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.873+0000 7fede67fc640 1 -- 172.21.15.5:0/340492960 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0031810 con 0x7fedd0004d40 2024-05-20T18:31:21.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.874+0000 7fedee1ea640 1 -- 172.21.15.5:0/340492960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedd0004d40 msgr2=0x7fedd0005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:21.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.874+0000 7fedee1ea640 1 --2- 172.21.15.5:0/340492960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedd0004d40 0x7fedd0005150 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fede80753b0 tx=0x7fedd0005830 comp rx=0 tx=0).stop 2024-05-20T18:31:21.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.874+0000 7fede7fff640 1 -- 172.21.15.5:0/340492960 reap_dead start 2024-05-20T18:31:21.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.874+0000 7fedee1ea640 1 -- 172.21.15.5:0/340492960 shutdown_connections 2024-05-20T18:31:21.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.874+0000 7fedee1ea640 1 -- 172.21.15.5:0/340492960 >> 172.21.15.5:0/340492960 conn(0x7fede8072ed0 msgr2=0x7fede80b6080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:21.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.874+0000 7fedee1ea640 1 -- 172.21.15.5:0/340492960 shutdown_connections 2024-05-20T18:31:21.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.874+0000 7fedee1ea640 1 -- 172.21.15.5:0/340492960 wait complete. 2024-05-20T18:31:21.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.875+0000 7fedee1ea640 1 Processor -- start 2024-05-20T18:31:21.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.876+0000 7fedee1ea640 1 -- start start 2024-05-20T18:31:21.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.877+0000 7fedee1ea640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fede81474e0 0x7fede806fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:21.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.877+0000 7fedee1ea640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fede81478f0 0x7fede80702a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:21.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.877+0000 7fedee1ea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fede8147e40 0x7fede8148250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:21.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.877+0000 7fedee1ea640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fede815f280 con 0x7fede8147e40 2024-05-20T18:31:21.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.877+0000 7fedee1ea640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fede8074e00 con 0x7fede81474e0 2024-05-20T18:31:21.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.877+0000 7fedee1ea640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fede815f400 con 0x7fede81478f0 2024-05-20T18:31:21.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.877+0000 7fede7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fede8147e40 0x7fede8148250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:21.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.877+0000 7fede7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fede8147e40 0x7fede8148250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37654/0 (socket says 172.21.15.5:37654) 2024-05-20T18:31:21.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.877+0000 7fede7fff640 1 -- 172.21.15.5:0/2555263608 learned_addr learned my addr 172.21.15.5:0/2555263608 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:21.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.877+0000 7fede77fe640 1 --2- 172.21.15.5:0/2555263608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fede81474e0 0x7fede806fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:21.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.877+0000 7fede6ffd640 1 --2- 172.21.15.5:0/2555263608 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fede81478f0 0x7fede80702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:21.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.877+0000 7fede7fff640 1 -- 172.21.15.5:0/2555263608 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fede81478f0 msgr2=0x7fede80702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:21.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.878+0000 7fede7fff640 1 --2- 172.21.15.5:0/2555263608 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fede81478f0 0x7fede80702a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:21.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.878+0000 7fede7fff640 1 -- 172.21.15.5:0/2555263608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fede81474e0 msgr2=0x7fede806fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:21.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.878+0000 7fede7fff640 1 --2- 172.21.15.5:0/2555263608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fede81474e0 0x7fede806fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:21.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.878+0000 7fede7fff640 1 -- 172.21.15.5:0/2555263608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede8148b00 con 0x7fede8147e40 2024-05-20T18:31:21.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.878+0000 7fede77fe640 1 --2- 172.21.15.5:0/2555263608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fede81474e0 0x7fede806fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:21.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.878+0000 7fede7fff640 1 --2- 172.21.15.5:0/2555263608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fede8147e40 0x7fede8148250 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fede0033040 tx=0x7fede0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:21.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.878+0000 7fedcffff640 1 -- 172.21.15.5:0/2555263608 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0009c10 con 0x7fede8147e40 2024-05-20T18:31:21.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.878+0000 7fedcffff640 1 -- 172.21.15.5:0/2555263608 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fede0045430 con 0x7fede8147e40 2024-05-20T18:31:21.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.878+0000 7fedee1ea640 1 -- 172.21.15.5:0/2555263608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede8148de0 con 0x7fede8147e40 2024-05-20T18:31:21.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.878+0000 7fedcffff640 1 -- 172.21.15.5:0/2555263608 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0045820 con 0x7fede8147e40 2024-05-20T18:31:21.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.878+0000 7fedee1ea640 1 -- 172.21.15.5:0/2555263608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede81493a0 con 0x7fede8147e40 2024-05-20T18:31:21.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.879+0000 7fedee1ea640 1 -- 172.21.15.5:0/2555263608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede8075b00 con 0x7fede8147e40 2024-05-20T18:31:21.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.880+0000 7fedcffff640 1 -- 172.21.15.5:0/2555263608 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fede0005d10 con 0x7fede8147e40 2024-05-20T18:31:21.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.880+0000 7fedcffff640 1 --2- 172.21.15.5:0/2555263608 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fedd406be30 0x7fedd406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:21.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.880+0000 7fedcffff640 1 -- 172.21.15.5:0/2555263608 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5335+0+0 (secure 0 0 0) 0x7fede00abf50 con 0x7fede8147e40 2024-05-20T18:31:21.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.880+0000 7fede77fe640 1 --2- 172.21.15.5:0/2555263608 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fedd406be30 0x7fedd406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:21.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.881+0000 7fede77fe640 1 --2- 172.21.15.5:0/2555263608 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fedd406be30 0x7fedd406e340 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fede809b960 tx=0x7fedd8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:21.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:21.885+0000 7fedcffff640 1 -- 172.21.15.5:0/2555263608 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fede002e050 con 0x7fede8147e40 2024-05-20T18:31:22.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.069+0000 7fedee1ea640 1 -- 172.21.15.5:0/2555263608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fede80b0eb0 con 0x7fede8147e40 2024-05-20T18:31:22.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.070+0000 7fedcffff640 1 -- 172.21.15.5:0/2555263608 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+13174 (secure 0 0 0) 0x7fede007aa70 con 0x7fede8147e40 2024-05-20T18:31:22.070 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:31:22.070 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":68,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:31:20.736034+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6834/4036379300":"2024-05-21T18:31:17.730951+0000","172.21.15.63:6832/4036379300":"2024-05-21T18:31:17.730951+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:31:22.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.072+0000 7fedee1ea640 1 -- 172.21.15.5:0/2555263608 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fedd406be30 msgr2=0x7fedd406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:22.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.072+0000 7fedee1ea640 1 --2- 172.21.15.5:0/2555263608 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fedd406be30 0x7fedd406e340 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fede809b960 tx=0x7fedd8008040 comp rx=0 tx=0).stop 2024-05-20T18:31:22.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.072+0000 7fedee1ea640 1 -- 172.21.15.5:0/2555263608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fede8147e40 msgr2=0x7fede8148250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:22.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.072+0000 7fedee1ea640 1 --2- 172.21.15.5:0/2555263608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fede8147e40 0x7fede8148250 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fede0033040 tx=0x7fede0009750 comp rx=0 tx=0).stop 2024-05-20T18:31:22.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.072+0000 7fedee1ea640 1 -- 172.21.15.5:0/2555263608 shutdown_connections 2024-05-20T18:31:22.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.072+0000 7fedee1ea640 1 --2- 172.21.15.5:0/2555263608 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fedd406be30 0x7fedd406e340 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:22.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.072+0000 7fedee1ea640 1 --2- 172.21.15.5:0/2555263608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fede8147e40 0x7fede8148250 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:22.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.072+0000 7fedee1ea640 1 --2- 172.21.15.5:0/2555263608 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fede81478f0 0x7fede80702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:22.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.072+0000 7fedee1ea640 1 --2- 172.21.15.5:0/2555263608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fede81474e0 0x7fede806fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:22.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.072+0000 7fedee1ea640 1 -- 172.21.15.5:0/2555263608 >> 172.21.15.5:0/2555263608 conn(0x7fede8072ed0 msgr2=0x7fede815d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:22.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.072+0000 7fedee1ea640 1 -- 172.21.15.5:0/2555263608 shutdown_connections 2024-05-20T18:31:22.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.072+0000 7fedee1ea640 1 -- 172.21.15.5:0/2555263608 wait complete. 2024-05-20T18:31:22.084 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:31:22.084 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:31:22.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.201+0000 7f31f7c73640 1 Processor -- start 2024-05-20T18:31:22.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.201+0000 7f31f7c73640 1 -- start start 2024-05-20T18:31:22.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.203+0000 7f31f7c73640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f31f0162820 0x7f31f00b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:22.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.203+0000 7f31f7c73640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f31f006dae0 0x7f31f00b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:22.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.203+0000 7f31f7c73640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31f00d0630 con 0x7f31f00b3e80 2024-05-20T18:31:22.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.203+0000 7f31f7c73640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f31f00d39c0 con 0x7f31f0162820 2024-05-20T18:31:22.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.203+0000 7f31f7c73640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f31f00d3ca0 con 0x7f31f006dae0 2024-05-20T18:31:22.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.203+0000 7f31f5a0f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f31f0162820 0x7f31f00b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:22.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.203+0000 7f31f520e640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f31f00b3e80 0x7f31f006d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45466/0 (socket says 172.21.15.5:45466) 2024-05-20T18:31:22.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.203+0000 7f31f520e640 1 -- 172.21.15.5:0/2465387580 learned_addr learned my addr 172.21.15.5:0/2465387580 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:22.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.203+0000 7f31f6210640 1 --2- 172.21.15.5:0/2465387580 >> v2:172.21.15.63:3300/0 conn(0x7f31f006dae0 0x7f31f00b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:22.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.203+0000 7f31f4a0d640 1 -- 172.21.15.5:0/2465387580 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1986709157 0 0) 0x7f31f00d0630 con 0x7f31f00b3e80 2024-05-20T18:31:22.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.203+0000 7f31f4a0d640 1 -- 172.21.15.5:0/2465387580 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31dc003520 con 0x7f31f00b3e80 2024-05-20T18:31:22.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.203+0000 7f31f5a0f640 1 -- 172.21.15.5:0/2465387580 >> v2:172.21.15.63:3300/0 conn(0x7f31f006dae0 msgr2=0x7f31f00b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:22.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.203+0000 7f31f5a0f640 1 --2- 172.21.15.5:0/2465387580 >> v2:172.21.15.63:3300/0 conn(0x7f31f006dae0 0x7f31f00b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:22.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.203+0000 7f31f5a0f640 1 -- 172.21.15.5:0/2465387580 >> v1:172.21.15.5:6789/0 conn(0x7f31f00b3e80 legacy=0x7f31f006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:22.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.203+0000 7f31f5a0f640 1 -- 172.21.15.5:0/2465387580 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f00d3f80 con 0x7f31f0162820 2024-05-20T18:31:22.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.204+0000 7f31f5a0f640 1 --2- 172.21.15.5:0/2465387580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31f0162820 0x7f31f00b3940 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f31f008e070 tx=0x7f31ec02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b199078ba51b542f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:22.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.204+0000 7f31f4a0d640 1 -- 172.21.15.5:0/2465387580 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec007740 con 0x7f31f0162820 2024-05-20T18:31:22.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.204+0000 7f31f4a0d640 1 -- 172.21.15.5:0/2465387580 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31ec0078e0 con 0x7f31f0162820 2024-05-20T18:31:22.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.204+0000 7f31f4a0d640 1 -- 172.21.15.5:0/2465387580 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec007cb0 con 0x7f31f0162820 2024-05-20T18:31:22.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.205+0000 7f31f7c73640 1 -- 172.21.15.5:0/2465387580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31f0162820 msgr2=0x7f31f00b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:22.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.205+0000 7f31f7c73640 1 --2- 172.21.15.5:0/2465387580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31f0162820 0x7f31f00b3940 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f31f008e070 tx=0x7f31ec02a680 comp rx=0 tx=0).stop 2024-05-20T18:31:22.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.205+0000 7f31f7c73640 1 -- 172.21.15.5:0/2465387580 shutdown_connections 2024-05-20T18:31:22.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.205+0000 7f31f7c73640 1 --2- 172.21.15.5:0/2465387580 >> v2:172.21.15.63:3300/0 conn(0x7f31f006dae0 0x7f31f00b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:22.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.205+0000 7f31f7c73640 1 --2- 172.21.15.5:0/2465387580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31f0162820 0x7f31f00b3940 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:22.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.205+0000 7f31f7c73640 1 -- 172.21.15.5:0/2465387580 >> 172.21.15.5:0/2465387580 conn(0x7f31f00d4bd0 msgr2=0x7f31f00d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:22.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.205+0000 7f31f7c73640 1 -- 172.21.15.5:0/2465387580 shutdown_connections 2024-05-20T18:31:22.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.205+0000 7f31f7c73640 1 -- 172.21.15.5:0/2465387580 wait complete. 2024-05-20T18:31:22.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.207+0000 7f31f7c73640 1 Processor -- start 2024-05-20T18:31:22.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.207+0000 7f31f7c73640 1 -- start start 2024-05-20T18:31:22.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.208+0000 7f31f7c73640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31f00b3e80 0x7f31f00cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:22.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.208+0000 7f31f7c73640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31f0162820 0x7f31f014d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:22.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.208+0000 7f31f7c73640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31f0155a10 0x7f31f00b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:22.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.208+0000 7f31f7c73640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f31f00d4780 con 0x7f31f0155a10 2024-05-20T18:31:22.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.208+0000 7f31f7c73640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f31f00d0300 con 0x7f31f0162820 2024-05-20T18:31:22.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.208+0000 7f31f7c73640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f31f00d4900 con 0x7f31f00b3e80 2024-05-20T18:31:22.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.208+0000 7f31f6210640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31f0155a10 0x7f31f00b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:22.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.208+0000 7f31f6210640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31f0155a10 0x7f31f00b6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37684/0 (socket says 172.21.15.5:37684) 2024-05-20T18:31:22.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.208+0000 7f31f6210640 1 -- 172.21.15.5:0/928348602 learned_addr learned my addr 172.21.15.5:0/928348602 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:22.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.208+0000 7f31f520e640 1 --2- 172.21.15.5:0/928348602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31f0162820 0x7f31f014d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:22.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.208+0000 7f31f5a0f640 1 --2- 172.21.15.5:0/928348602 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31f00b3e80 0x7f31f00cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:22.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.209+0000 7f31f6210640 1 -- 172.21.15.5:0/928348602 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31f00b3e80 msgr2=0x7f31f00cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:22.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.209+0000 7f31f6210640 1 --2- 172.21.15.5:0/928348602 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31f00b3e80 0x7f31f00cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:22.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.209+0000 7f31f6210640 1 -- 172.21.15.5:0/928348602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31f0162820 msgr2=0x7f31f014d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:22.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.209+0000 7f31f6210640 1 --2- 172.21.15.5:0/928348602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31f0162820 0x7f31f014d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:22.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.209+0000 7f31f6210640 1 -- 172.21.15.5:0/928348602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f00b4f80 con 0x7f31f0155a10 2024-05-20T18:31:22.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.209+0000 7f31f6210640 1 --2- 172.21.15.5:0/928348602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31f0155a10 0x7f31f00b6170 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f31e800e820 tx=0x7f31e800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:22.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.209+0000 7f31e67fc640 1 -- 172.21.15.5:0/928348602 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e8017070 con 0x7f31f0155a10 2024-05-20T18:31:22.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.209+0000 7f31e67fc640 1 -- 172.21.15.5:0/928348602 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31e800ee70 con 0x7f31f0155a10 2024-05-20T18:31:22.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.209+0000 7f31e67fc640 1 -- 172.21.15.5:0/928348602 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e800cb60 con 0x7f31f0155a10 2024-05-20T18:31:22.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.209+0000 7f31f7c73640 1 -- 172.21.15.5:0/928348602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31f00b5260 con 0x7f31f0155a10 2024-05-20T18:31:22.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.209+0000 7f31f7c73640 1 -- 172.21.15.5:0/928348602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31f00c9930 con 0x7f31f0155a10 2024-05-20T18:31:22.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.210+0000 7f31f7c73640 1 -- 172.21.15.5:0/928348602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31f015f0e0 con 0x7f31f0155a10 2024-05-20T18:31:22.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.211+0000 7f31e67fc640 1 -- 172.21.15.5:0/928348602 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f31e801a070 con 0x7f31f0155a10 2024-05-20T18:31:22.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.211+0000 7f31e67fc640 1 --2- 172.21.15.5:0/928348602 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f31d406be30 0x7f31d406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:22.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.211+0000 7f31e67fc640 1 -- 172.21.15.5:0/928348602 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5335+0+0 (secure 0 0 0) 0x7f31e808aeb0 con 0x7f31f0155a10 2024-05-20T18:31:22.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.211+0000 7f31f5a0f640 1 --2- 172.21.15.5:0/928348602 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f31d406be30 0x7f31d406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:22.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.212+0000 7f31f5a0f640 1 --2- 172.21.15.5:0/928348602 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f31d406be30 0x7f31d406e340 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f31ec034000 tx=0x7f31ec02f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:22.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.216+0000 7f31e67fc640 1 -- 172.21.15.5:0/928348602 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f31f015f0e0 con 0x7f31f0155a10 2024-05-20T18:31:22.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.402+0000 7f31f7c73640 1 -- 172.21.15.5:0/928348602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f31f00cb630 con 0x7f31f0155a10 2024-05-20T18:31:22.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.402+0000 7f31e67fc640 1 -- 172.21.15.5:0/928348602 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) ==== 93+0+10 (secure 0 0 0) 0x7f31e8059370 con 0x7f31f0155a10 2024-05-20T18:31:22.403 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:31:22.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.404+0000 7f31f7c73640 1 -- 172.21.15.5:0/928348602 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f31d406be30 msgr2=0x7f31d406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:22.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.404+0000 7f31f7c73640 1 --2- 172.21.15.5:0/928348602 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f31d406be30 0x7f31d406e340 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f31ec034000 tx=0x7f31ec02f000 comp rx=0 tx=0).stop 2024-05-20T18:31:22.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.404+0000 7f31f7c73640 1 -- 172.21.15.5:0/928348602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31f0155a10 msgr2=0x7f31f00b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:22.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.404+0000 7f31f7c73640 1 --2- 172.21.15.5:0/928348602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31f0155a10 0x7f31f00b6170 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f31e800e820 tx=0x7f31e800ec20 comp rx=0 tx=0).stop 2024-05-20T18:31:22.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.404+0000 7f31f7c73640 1 -- 172.21.15.5:0/928348602 shutdown_connections 2024-05-20T18:31:22.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.404+0000 7f31f7c73640 1 --2- 172.21.15.5:0/928348602 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f31d406be30 0x7f31d406e340 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:22.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.404+0000 7f31f7c73640 1 --2- 172.21.15.5:0/928348602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31f0155a10 0x7f31f00b6170 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:22.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.404+0000 7f31f7c73640 1 --2- 172.21.15.5:0/928348602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f31f0162820 0x7f31f014d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:22.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.404+0000 7f31f7c73640 1 --2- 172.21.15.5:0/928348602 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31f00b3e80 0x7f31f00cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:22.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.404+0000 7f31f7c73640 1 -- 172.21.15.5:0/928348602 >> 172.21.15.5:0/928348602 conn(0x7f31f00d4bd0 msgr2=0x7f31f0161900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:22.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.404+0000 7f31f7c73640 1 -- 172.21.15.5:0/928348602 shutdown_connections 2024-05-20T18:31:22.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.404+0000 7f31f7c73640 1 -- 172.21.15.5:0/928348602 wait complete. 2024-05-20T18:31:22.416 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-05-20T18:31:22.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.531+0000 7f711c157640 1 Processor -- start 2024-05-20T18:31:22.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.531+0000 7f711c157640 1 -- start start 2024-05-20T18:31:22.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.533+0000 7f711c157640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f71140af030 0x7f71140af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:22.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.533+0000 7f711c157640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f711415f700 0x7f711415fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:22.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.533+0000 7f711c157640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71140af980 con 0x7f711409a4f0 2024-05-20T18:31:22.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.533+0000 7f711c157640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f711406bcf0 con 0x7f71140af030 2024-05-20T18:31:22.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.533+0000 7f711c157640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7114160430 con 0x7f711415f700 2024-05-20T18:31:22.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.533+0000 7f7119ef3640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f71140af030 0x7f71140af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:22.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.533+0000 7f71196f2640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f711415f700 0x7f711415fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:22.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.533+0000 7f7119ef3640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f71140af030 0x7f71140af440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:53834/0 (socket says 172.21.15.5:53834) 2024-05-20T18:31:22.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.533+0000 7f711a6f4640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f711409a4f0 0x7f711409a900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:41304/0 (socket says 172.21.15.5:41304) 2024-05-20T18:31:22.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.533+0000 7f7119ef3640 1 -- 172.21.15.5:0/2358855497 learned_addr learned my addr 172.21.15.5:0/2358855497 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:22.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.533+0000 7f7119ef3640 1 -- 172.21.15.5:0/2358855497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f711415f700 msgr2=0x7f711415fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:22.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.533+0000 7f7119ef3640 1 --2- 172.21.15.5:0/2358855497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f711415f700 0x7f711415fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:22.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.533+0000 7f7119ef3640 1 -- 172.21.15.5:0/2358855497 >> v1:172.21.15.63:6789/0 conn(0x7f711409a4f0 legacy=0x7f711409a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:22.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.533+0000 7f7118ef1640 1 -- 172.21.15.5:0/2358855497 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2488883889 0 0) 0x7f710c001490 con 0x7f711409a4f0 2024-05-20T18:31:22.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.533+0000 7f7119ef3640 1 -- 172.21.15.5:0/2358855497 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711406b6a0 con 0x7f71140af030 2024-05-20T18:31:22.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.533+0000 7f71196f2640 1 --2- 172.21.15.5:0/2358855497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f711415f700 0x7f711415fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:31:22.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.534+0000 7f7119ef3640 1 --2- 172.21.15.5:0/2358855497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71140af030 0x7f71140af440 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7110009870 tx=0x7f711002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9affecb7c41effb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:22.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.534+0000 7f7118ef1640 1 -- 172.21.15.5:0/2358855497 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110037070 con 0x7f71140af030 2024-05-20T18:31:22.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.534+0000 7f7118ef1640 1 -- 172.21.15.5:0/2358855497 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7110032440 con 0x7f71140af030 2024-05-20T18:31:22.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.534+0000 7f7118ef1640 1 -- 172.21.15.5:0/2358855497 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110032810 con 0x7f71140af030 2024-05-20T18:31:22.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.535+0000 7f711c157640 1 -- 172.21.15.5:0/2358855497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71140af030 msgr2=0x7f71140af440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:22.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.535+0000 7f711c157640 1 --2- 172.21.15.5:0/2358855497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71140af030 0x7f71140af440 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7110009870 tx=0x7f711002a250 comp rx=0 tx=0).stop 2024-05-20T18:31:22.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.535+0000 7f711c157640 1 -- 172.21.15.5:0/2358855497 shutdown_connections 2024-05-20T18:31:22.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.535+0000 7f711c157640 1 --2- 172.21.15.5:0/2358855497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f711415f700 0x7f711415fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:22.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.535+0000 7f711c157640 1 --2- 172.21.15.5:0/2358855497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71140af030 0x7f71140af440 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:22.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.535+0000 7f711c157640 1 -- 172.21.15.5:0/2358855497 >> 172.21.15.5:0/2358855497 conn(0x7f71140b6620 msgr2=0x7f71140b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:22.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.535+0000 7f711c157640 1 -- 172.21.15.5:0/2358855497 shutdown_connections 2024-05-20T18:31:22.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.535+0000 7f711c157640 1 -- 172.21.15.5:0/2358855497 wait complete. 2024-05-20T18:31:22.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.537+0000 7f711c157640 1 Processor -- start 2024-05-20T18:31:22.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.537+0000 7f711c157640 1 -- start start 2024-05-20T18:31:22.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.538+0000 7f711c157640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f711409a4f0 0x7f7114156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:22.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.538+0000 7f711c157640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71140af030 0x7f71141f9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:22.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.538+0000 7f711c157640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f711415f700 0x7f71141f29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:22.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.538+0000 7f711c157640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7114160050 con 0x7f711415f700 2024-05-20T18:31:22.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.538+0000 7f711c157640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7114159aa0 con 0x7f71140af030 2024-05-20T18:31:22.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.538+0000 7f711c157640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f71141601d0 con 0x7f711409a4f0 2024-05-20T18:31:22.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.539+0000 7f711a6f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f711415f700 0x7f71141f29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:22.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.539+0000 7f711a6f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f711415f700 0x7f71141f29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37698/0 (socket says 172.21.15.5:37698) 2024-05-20T18:31:22.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.539+0000 7f711a6f4640 1 -- 172.21.15.5:0/284451606 learned_addr learned my addr 172.21.15.5:0/284451606 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:22.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.539+0000 7f71196f2640 1 --2- 172.21.15.5:0/284451606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71140af030 0x7f71141f9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:22.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.539+0000 7f7119ef3640 1 --2- 172.21.15.5:0/284451606 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f711409a4f0 0x7f7114156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:22.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.539+0000 7f711a6f4640 1 -- 172.21.15.5:0/284451606 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f711409a4f0 msgr2=0x7f7114156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:22.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.539+0000 7f711a6f4640 1 --2- 172.21.15.5:0/284451606 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f711409a4f0 0x7f7114156d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:22.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.539+0000 7f711a6f4640 1 -- 172.21.15.5:0/284451606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71140af030 msgr2=0x7f71141f9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:22.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.539+0000 7f711a6f4640 1 --2- 172.21.15.5:0/284451606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71140af030 0x7f71141f9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:22.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.539+0000 7f711a6f4640 1 -- 172.21.15.5:0/284451606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71141f9dd0 con 0x7f711415f700 2024-05-20T18:31:22.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.539+0000 7f711a6f4640 1 --2- 172.21.15.5:0/284451606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f711415f700 0x7f71141f29b0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f710c00c700 tx=0x7f710c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:22.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.539+0000 7f710a7fc640 1 -- 172.21.15.5:0/284451606 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c018070 con 0x7f711415f700 2024-05-20T18:31:22.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.539+0000 7f710a7fc640 1 -- 172.21.15.5:0/284451606 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f710c00ce10 con 0x7f711415f700 2024-05-20T18:31:22.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.539+0000 7f710a7fc640 1 -- 172.21.15.5:0/284451606 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c00a880 con 0x7f711415f700 2024-05-20T18:31:22.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.539+0000 7f711c157640 1 -- 172.21.15.5:0/284451606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7114157950 con 0x7f711415f700 2024-05-20T18:31:22.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.540+0000 7f711c157640 1 -- 172.21.15.5:0/284451606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7114157f10 con 0x7f711415f700 2024-05-20T18:31:22.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.540+0000 7f711c157640 1 -- 172.21.15.5:0/284451606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7114158710 con 0x7f711415f700 2024-05-20T18:31:22.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.541+0000 7f710a7fc640 1 -- 172.21.15.5:0/284451606 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f710c00aa20 con 0x7f711415f700 2024-05-20T18:31:22.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.541+0000 7f710a7fc640 1 --2- 172.21.15.5:0/284451606 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f70f806be30 0x7f70f806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:22.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.542+0000 7f710a7fc640 1 -- 172.21.15.5:0/284451606 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5335+0+0 (secure 0 0 0) 0x7f710c08a150 con 0x7f711415f700 2024-05-20T18:31:22.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.542+0000 7f7119ef3640 1 --2- 172.21.15.5:0/284451606 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f70f806be30 0x7f70f806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:22.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.542+0000 7f7119ef3640 1 --2- 172.21.15.5:0/284451606 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f70f806be30 0x7f70f806e340 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7110034000 tx=0x7f711002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:22.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.546+0000 7f710a7fc640 1 -- 172.21.15.5:0/284451606 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f71141f9dd0 con 0x7f711415f700 2024-05-20T18:31:22.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:22.804+0000 7f711c157640 1 -- 172.21.15.5:0/284451606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7f7114075b00 con 0x7f711415f700 2024-05-20T18:31:23.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.746+0000 7f710a7fc640 1 -- 172.21.15.5:0/284451606 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v226) ==== 169+0+0 (secure 0 0 0) 0x7f71140af980 con 0x7f711415f700 2024-05-20T18:31:23.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.749+0000 7f711c157640 1 -- 172.21.15.5:0/284451606 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f70f806be30 msgr2=0x7f70f806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:23.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.749+0000 7f711c157640 1 --2- 172.21.15.5:0/284451606 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f70f806be30 0x7f70f806e340 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7110034000 tx=0x7f711002e000 comp rx=0 tx=0).stop 2024-05-20T18:31:23.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.749+0000 7f711c157640 1 -- 172.21.15.5:0/284451606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f711415f700 msgr2=0x7f71141f29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:23.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.749+0000 7f711c157640 1 --2- 172.21.15.5:0/284451606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f711415f700 0x7f71141f29b0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f710c00c700 tx=0x7f710c00cbc0 comp rx=0 tx=0).stop 2024-05-20T18:31:23.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.749+0000 7f711c157640 1 -- 172.21.15.5:0/284451606 shutdown_connections 2024-05-20T18:31:23.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.749+0000 7f711c157640 1 --2- 172.21.15.5:0/284451606 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f70f806be30 0x7f70f806e340 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:23.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.749+0000 7f711c157640 1 --2- 172.21.15.5:0/284451606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f711415f700 0x7f71141f29b0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:23.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.750+0000 7f711c157640 1 --2- 172.21.15.5:0/284451606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71140af030 0x7f71141f9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:23.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.750+0000 7f711c157640 1 --2- 172.21.15.5:0/284451606 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f711409a4f0 0x7f7114156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:23.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.750+0000 7f711c157640 1 -- 172.21.15.5:0/284451606 >> 172.21.15.5:0/284451606 conn(0x7f71140b6620 msgr2=0x7f7114070300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:23.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.750+0000 7f711c157640 1 -- 172.21.15.5:0/284451606 shutdown_connections 2024-05-20T18:31:23.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.750+0000 7f711c157640 1 -- 172.21.15.5:0/284451606 wait complete. 2024-05-20T18:31:23.764 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-20T18:31:23.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.889+0000 7f69b8996640 1 Processor -- start 2024-05-20T18:31:23.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.889+0000 7f69b8996640 1 -- start start 2024-05-20T18:31:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.890+0000 7f69b8996640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f69b4162820 0x7f69b40b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.890+0000 7f69b8996640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69b40b3e80 0x7f69b406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.890+0000 7f69b8996640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f69b406d910 0x7f69b40d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.890+0000 7f69b8996640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f69b406dd20 con 0x7f69b406d910 2024-05-20T18:31:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.890+0000 7f69b8996640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f69b40d0630 con 0x7f69b4162820 2024-05-20T18:31:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.890+0000 7f69b8996640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f69b40d39c0 con 0x7f69b40b3e80 2024-05-20T18:31:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.890+0000 7f69b2d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f69b406d910 0x7f69b40d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.890+0000 7f69b2d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f69b406d910 0x7f69b40d27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37700/0 (socket says 172.21.15.5:37700) 2024-05-20T18:31:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.890+0000 7f69b2d9d640 1 -- 172.21.15.5:0/1254169211 learned_addr learned my addr 172.21.15.5:0/1254169211 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.891+0000 7f69b259c640 1 --2- 172.21.15.5:0/1254169211 >> v2:172.21.15.63:3300/0 conn(0x7f69b4162820 0x7f69b40b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.891+0000 7f69b1d9b640 1 --2- 172.21.15.5:0/1254169211 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69b40b3e80 0x7f69b406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.891+0000 7f69b2d9d640 1 -- 172.21.15.5:0/1254169211 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69b40b3e80 msgr2=0x7f69b406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.891+0000 7f69b2d9d640 1 --2- 172.21.15.5:0/1254169211 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69b40b3e80 0x7f69b406d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.891+0000 7f69b2d9d640 1 -- 172.21.15.5:0/1254169211 >> v2:172.21.15.63:3300/0 conn(0x7f69b4162820 msgr2=0x7f69b40b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.891+0000 7f69b2d9d640 1 --2- 172.21.15.5:0/1254169211 >> v2:172.21.15.63:3300/0 conn(0x7f69b4162820 0x7f69b40b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.891+0000 7f69b2d9d640 1 -- 172.21.15.5:0/1254169211 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b40d3ca0 con 0x7f69b406d910 2024-05-20T18:31:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.891+0000 7f69b259c640 1 --2- 172.21.15.5:0/1254169211 >> v2:172.21.15.63:3300/0 conn(0x7f69b4162820 0x7f69b40b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.892+0000 7f69b2d9d640 1 --2- 172.21.15.5:0/1254169211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69b406d910 0x7f69b40d27a0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f69b40b7740 tx=0x7f69ac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38e36a44d15107c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.892+0000 7f69b159a640 1 -- 172.21.15.5:0/1254169211 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac037070 con 0x7f69b406d910 2024-05-20T18:31:23.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.892+0000 7f69b159a640 1 -- 172.21.15.5:0/1254169211 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69ac032440 con 0x7f69b406d910 2024-05-20T18:31:23.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.892+0000 7f69b159a640 1 -- 172.21.15.5:0/1254169211 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac032810 con 0x7f69b406d910 2024-05-20T18:31:23.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.892+0000 7f69b8996640 1 -- 172.21.15.5:0/1254169211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69b406d910 msgr2=0x7f69b40d27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:23.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.892+0000 7f69b8996640 1 --2- 172.21.15.5:0/1254169211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69b406d910 0x7f69b40d27a0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f69b40b7740 tx=0x7f69ac02a1e0 comp rx=0 tx=0).stop 2024-05-20T18:31:23.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.893+0000 7f69b8996640 1 -- 172.21.15.5:0/1254169211 shutdown_connections 2024-05-20T18:31:23.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.893+0000 7f69b8996640 1 --2- 172.21.15.5:0/1254169211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69b406d910 0x7f69b40d27a0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:23.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.893+0000 7f69b8996640 1 --2- 172.21.15.5:0/1254169211 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69b40b3e80 0x7f69b406d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:23.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.893+0000 7f69b8996640 1 --2- 172.21.15.5:0/1254169211 >> v2:172.21.15.63:3300/0 conn(0x7f69b4162820 0x7f69b40b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:23.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.893+0000 7f69b8996640 1 -- 172.21.15.5:0/1254169211 >> 172.21.15.5:0/1254169211 conn(0x7f69b40d4bd0 msgr2=0x7f69b40d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:23.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.893+0000 7f69b8996640 1 -- 172.21.15.5:0/1254169211 shutdown_connections 2024-05-20T18:31:23.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.893+0000 7f69b8996640 1 -- 172.21.15.5:0/1254169211 wait complete. 2024-05-20T18:31:23.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.895+0000 7f69b8996640 1 Processor -- start 2024-05-20T18:31:23.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.895+0000 7f69b8996640 1 -- start start 2024-05-20T18:31:23.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.896+0000 7f69b8996640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69b406d910 0x7f69b40b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:23.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.896+0000 7f69b8996640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69b40b3e80 0x7f69b41e89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:23.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.896+0000 7f69b8996640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69b4162820 0x7f69b40b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:23.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.896+0000 7f69b8996640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f69b40d0300 con 0x7f69b406d910 2024-05-20T18:31:23.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.896+0000 7f69b8996640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f69b40d4260 con 0x7f69b4162820 2024-05-20T18:31:23.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.896+0000 7f69b8996640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f69b40d4780 con 0x7f69b40b3e80 2024-05-20T18:31:23.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.896+0000 7f69b259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69b406d910 0x7f69b40b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:23.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.896+0000 7f69b259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69b406d910 0x7f69b40b4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37716/0 (socket says 172.21.15.5:37716) 2024-05-20T18:31:23.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.896+0000 7f69b259c640 1 -- 172.21.15.5:0/3168630067 learned_addr learned my addr 172.21.15.5:0/3168630067 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:23.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.896+0000 7f69b1d9b640 1 --2- 172.21.15.5:0/3168630067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69b40b3e80 0x7f69b41e89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:23.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.896+0000 7f69b2d9d640 1 --2- 172.21.15.5:0/3168630067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69b4162820 0x7f69b40b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:23.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.896+0000 7f69b259c640 1 -- 172.21.15.5:0/3168630067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69b40b3e80 msgr2=0x7f69b41e89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:23.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.896+0000 7f69b259c640 1 --2- 172.21.15.5:0/3168630067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69b40b3e80 0x7f69b41e89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:23.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.896+0000 7f69b259c640 1 -- 172.21.15.5:0/3168630067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69b4162820 msgr2=0x7f69b40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:23.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.896+0000 7f69b259c640 1 --2- 172.21.15.5:0/3168630067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69b4162820 0x7f69b40b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:23.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.896+0000 7f69b259c640 1 -- 172.21.15.5:0/3168630067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b40b5580 con 0x7f69b406d910 2024-05-20T18:31:23.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.896+0000 7f69b2d9d640 1 --2- 172.21.15.5:0/3168630067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69b4162820 0x7f69b40b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:23.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.897+0000 7f69b259c640 1 --2- 172.21.15.5:0/3168630067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69b406d910 0x7f69b40b4d30 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f69a400ce00 tx=0x7f69a400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:23.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.897+0000 7f69a2ffd640 1 -- 172.21.15.5:0/3168630067 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a400f890 con 0x7f69b406d910 2024-05-20T18:31:23.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.897+0000 7f69a2ffd640 1 -- 172.21.15.5:0/3168630067 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69a400fa30 con 0x7f69b406d910 2024-05-20T18:31:23.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.897+0000 7f69a2ffd640 1 -- 172.21.15.5:0/3168630067 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a400fe00 con 0x7f69b406d910 2024-05-20T18:31:23.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.897+0000 7f69b8996640 1 -- 172.21.15.5:0/3168630067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69b41e7bf0 con 0x7f69b406d910 2024-05-20T18:31:23.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.897+0000 7f69b8996640 1 -- 172.21.15.5:0/3168630067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69b41e81b0 con 0x7f69b406d910 2024-05-20T18:31:23.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.898+0000 7f69b8996640 1 -- 172.21.15.5:0/3168630067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69b406e400 con 0x7f69b406d910 2024-05-20T18:31:23.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.898+0000 7f69a2ffd640 1 -- 172.21.15.5:0/3168630067 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f69a4018400 con 0x7f69b406d910 2024-05-20T18:31:23.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.899+0000 7f69a2ffd640 1 --2- 172.21.15.5:0/3168630067 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f699406be30 0x7f699406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:23.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.899+0000 7f69b1d9b640 1 --2- 172.21.15.5:0/3168630067 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f699406be30 0x7f699406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:23.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.899+0000 7f69a2ffd640 1 -- 172.21.15.5:0/3168630067 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5335+0+0 (secure 0 0 0) 0x7f69a408bb10 con 0x7f69b406d910 2024-05-20T18:31:23.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.900+0000 7f69b1d9b640 1 --2- 172.21.15.5:0/3168630067 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f699406be30 0x7f699406e340 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f69a8006fd0 tx=0x7f69a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:23.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:23.904+0000 7f69a2ffd640 1 -- 172.21.15.5:0/3168630067 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f69b406e400 con 0x7f69b406d910 2024-05-20T18:31:24.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.089+0000 7f69b8996640 1 -- 172.21.15.5:0/3168630067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f69b4075b00 con 0x7f69b406d910 2024-05-20T18:31:24.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.092+0000 7f69a2ffd640 1 -- 172.21.15.5:0/3168630067 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f69a4014030 con 0x7f69b406d910 2024-05-20T18:31:24.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.095+0000 7f69b8996640 1 -- 172.21.15.5:0/3168630067 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f699406be30 msgr2=0x7f699406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:24.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.095+0000 7f69b8996640 1 --2- 172.21.15.5:0/3168630067 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f699406be30 0x7f699406e340 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f69a8006fd0 tx=0x7f69a8008040 comp rx=0 tx=0).stop 2024-05-20T18:31:24.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.095+0000 7f69b8996640 1 -- 172.21.15.5:0/3168630067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69b406d910 msgr2=0x7f69b40b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:24.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.095+0000 7f69b8996640 1 --2- 172.21.15.5:0/3168630067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69b406d910 0x7f69b40b4d30 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f69a400ce00 tx=0x7f69a400a650 comp rx=0 tx=0).stop 2024-05-20T18:31:24.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.095+0000 7f69b8996640 1 -- 172.21.15.5:0/3168630067 shutdown_connections 2024-05-20T18:31:24.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.095+0000 7f69b8996640 1 --2- 172.21.15.5:0/3168630067 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f699406be30 0x7f699406e340 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:24.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.095+0000 7f69b8996640 1 --2- 172.21.15.5:0/3168630067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69b4162820 0x7f69b40b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:24.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.095+0000 7f69b8996640 1 --2- 172.21.15.5:0/3168630067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69b40b3e80 0x7f69b41e89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:24.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.095+0000 7f69b8996640 1 --2- 172.21.15.5:0/3168630067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69b406d910 0x7f69b40b4d30 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:24.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.095+0000 7f69b8996640 1 -- 172.21.15.5:0/3168630067 >> 172.21.15.5:0/3168630067 conn(0x7f69b40d4bd0 msgr2=0x7f69b40d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:24.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.095+0000 7f69b8996640 1 -- 172.21.15.5:0/3168630067 shutdown_connections 2024-05-20T18:31:24.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.095+0000 7f69b8996640 1 -- 172.21.15.5:0/3168630067 wait complete. 2024-05-20T18:31:24.106 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-20T18:31:24.107 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T18:31:24.145 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T18:31:24.200 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-20T18:31:24.200 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:31:24.200 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:31:24.216 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15663 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T18:31:24.217 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:31:24.217 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:31:24.236 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19732 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T18:31:24.236 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:31:24.236 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:31:24.273 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15663 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T18:31:24.273 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:31:24.274 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:31:24.294 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19732 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T18:31:24.294 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-20T18:31:24.295 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:31:24.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.428+0000 7fe32ff2a640 1 Processor -- start 2024-05-20T18:31:24.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.428+0000 7fe32ff2a640 1 -- start start 2024-05-20T18:31:24.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.429+0000 7fe32ff2a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe328163a10 0x7fe3280d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:24.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.429+0000 7fe32ff2a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe3280b4010 0x7fe3280cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:24.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.429+0000 7fe32ff2a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe3280d14b0 0x7fe3280d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:24.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.429+0000 7fe32ff2a640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe3280d3ca0 con 0x7fe3280b4010 2024-05-20T18:31:24.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.429+0000 7fe32ff2a640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe3280d0370 con 0x7fe3280d14b0 2024-05-20T18:31:24.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.429+0000 7fe32ff2a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe3280d0650 con 0x7fe328163a10 2024-05-20T18:31:24.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.429+0000 7fe32d4c5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe3280b4010 0x7fe3280cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:24.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.429+0000 7fe32d4c5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe3280b4010 0x7fe3280cfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37718/0 (socket says 172.21.15.5:37718) 2024-05-20T18:31:24.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.429+0000 7fe32d4c5640 1 -- 172.21.15.5:0/148996381 learned_addr learned my addr 172.21.15.5:0/148996381 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:24.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.430+0000 7fe32e4c7640 1 --2- 172.21.15.5:0/148996381 >> v2:172.21.15.63:3300/0 conn(0x7fe3280d14b0 0x7fe3280d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:24.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.430+0000 7fe32dcc6640 1 --2- 172.21.15.5:0/148996381 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe328163a10 0x7fe3280d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:24.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.430+0000 7fe32d4c5640 1 -- 172.21.15.5:0/148996381 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe328163a10 msgr2=0x7fe3280d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:24.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.430+0000 7fe32d4c5640 1 --2- 172.21.15.5:0/148996381 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe328163a10 0x7fe3280d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:24.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.430+0000 7fe32d4c5640 1 -- 172.21.15.5:0/148996381 >> v2:172.21.15.63:3300/0 conn(0x7fe3280d14b0 msgr2=0x7fe3280d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:24.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.430+0000 7fe32d4c5640 1 --2- 172.21.15.5:0/148996381 >> v2:172.21.15.63:3300/0 conn(0x7fe3280d14b0 0x7fe3280d18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:24.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.430+0000 7fe32d4c5640 1 -- 172.21.15.5:0/148996381 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3280d3650 con 0x7fe3280b4010 2024-05-20T18:31:24.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.430+0000 7fe32e4c7640 1 --2- 172.21.15.5:0/148996381 >> v2:172.21.15.63:3300/0 conn(0x7fe3280d14b0 0x7fe3280d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:24.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.431+0000 7fe32d4c5640 1 --2- 172.21.15.5:0/148996381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3280b4010 0x7fe3280cfb00 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fe3280b4590 tx=0x7fe31802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b989adc7172fb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:24.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.431+0000 7fe32ccc4640 1 -- 172.21.15.5:0/148996381 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe318037070 con 0x7fe3280b4010 2024-05-20T18:31:24.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.431+0000 7fe32ccc4640 1 -- 172.21.15.5:0/148996381 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe318032440 con 0x7fe3280b4010 2024-05-20T18:31:24.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.431+0000 7fe32ccc4640 1 -- 172.21.15.5:0/148996381 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe318032810 con 0x7fe3280b4010 2024-05-20T18:31:24.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.431+0000 7fe32ff2a640 1 -- 172.21.15.5:0/148996381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3280b4010 msgr2=0x7fe3280cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:24.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.431+0000 7fe32ff2a640 1 --2- 172.21.15.5:0/148996381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3280b4010 0x7fe3280cfb00 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fe3280b4590 tx=0x7fe31802a1e0 comp rx=0 tx=0).stop 2024-05-20T18:31:24.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.431+0000 7fe32ff2a640 1 -- 172.21.15.5:0/148996381 shutdown_connections 2024-05-20T18:31:24.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.431+0000 7fe32ff2a640 1 --2- 172.21.15.5:0/148996381 >> v2:172.21.15.63:3300/0 conn(0x7fe3280d14b0 0x7fe3280d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:24.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.431+0000 7fe32ff2a640 1 --2- 172.21.15.5:0/148996381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3280b4010 0x7fe3280cfb00 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:24.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.431+0000 7fe32ff2a640 1 --2- 172.21.15.5:0/148996381 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe328163a10 0x7fe3280d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:24.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.431+0000 7fe32ff2a640 1 -- 172.21.15.5:0/148996381 >> 172.21.15.5:0/148996381 conn(0x7fe32806d240 msgr2=0x7fe32806d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:24.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.432+0000 7fe32ff2a640 1 -- 172.21.15.5:0/148996381 shutdown_connections 2024-05-20T18:31:24.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.432+0000 7fe32ff2a640 1 -- 172.21.15.5:0/148996381 wait complete. 2024-05-20T18:31:24.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.433+0000 7fe32ff2a640 1 Processor -- start 2024-05-20T18:31:24.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.433+0000 7fe32ff2a640 1 -- start start 2024-05-20T18:31:24.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.435+0000 7fe32ff2a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3280b4010 0x7fe3280ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:24.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.435+0000 7fe32ff2a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3280d14b0 0x7fe3281f60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:24.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.435+0000 7fe32ff2a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe328163a10 0x7fe3281f7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:24.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.435+0000 7fe32ff2a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe3280d1e00 con 0x7fe3280b4010 2024-05-20T18:31:24.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.435+0000 7fe32ff2a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe3280d0040 con 0x7fe3280d14b0 2024-05-20T18:31:24.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.435+0000 7fe32ff2a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe3280b8210 con 0x7fe328163a10 2024-05-20T18:31:24.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.435+0000 7fe32dcc6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3280b4010 0x7fe3280ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:24.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.435+0000 7fe32dcc6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3280b4010 0x7fe3280ce0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37734/0 (socket says 172.21.15.5:37734) 2024-05-20T18:31:24.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.435+0000 7fe32dcc6640 1 -- 172.21.15.5:0/1537155076 learned_addr learned my addr 172.21.15.5:0/1537155076 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:24.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.435+0000 7fe32d4c5640 1 --2- 172.21.15.5:0/1537155076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3280d14b0 0x7fe3281f60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:24.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.435+0000 7fe32e4c7640 1 --2- 172.21.15.5:0/1537155076 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe328163a10 0x7fe3281f7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:24.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.435+0000 7fe32dcc6640 1 -- 172.21.15.5:0/1537155076 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe328163a10 msgr2=0x7fe3281f7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:24.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.435+0000 7fe32dcc6640 1 --2- 172.21.15.5:0/1537155076 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe328163a10 0x7fe3281f7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:24.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.435+0000 7fe32dcc6640 1 -- 172.21.15.5:0/1537155076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3280d14b0 msgr2=0x7fe3281f60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:24.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.435+0000 7fe32dcc6640 1 --2- 172.21.15.5:0/1537155076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3280d14b0 0x7fe3281f60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:24.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.435+0000 7fe32dcc6640 1 -- 172.21.15.5:0/1537155076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3281f6630 con 0x7fe3280b4010 2024-05-20T18:31:24.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.435+0000 7fe32dcc6640 1 --2- 172.21.15.5:0/1537155076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3280b4010 0x7fe3280ce0f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fe32400e840 tx=0x7fe32400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:24.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.436+0000 7fe31e7fc640 1 -- 172.21.15.5:0/1537155076 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe324017070 con 0x7fe3280b4010 2024-05-20T18:31:24.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.436+0000 7fe31e7fc640 1 -- 172.21.15.5:0/1537155076 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe32400c940 con 0x7fe3280b4010 2024-05-20T18:31:24.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.436+0000 7fe32ff2a640 1 -- 172.21.15.5:0/1537155076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3281f6b40 con 0x7fe3280b4010 2024-05-20T18:31:24.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.436+0000 7fe31e7fc640 1 -- 172.21.15.5:0/1537155076 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32400cd10 con 0x7fe3280b4010 2024-05-20T18:31:24.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.436+0000 7fe32ff2a640 1 -- 172.21.15.5:0/1537155076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3281f7100 con 0x7fe3280b4010 2024-05-20T18:31:24.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.436+0000 7fe32ff2a640 1 -- 172.21.15.5:0/1537155076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe328075b00 con 0x7fe3280b4010 2024-05-20T18:31:24.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.437+0000 7fe31e7fc640 1 -- 172.21.15.5:0/1537155076 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe32401a070 con 0x7fe3280b4010 2024-05-20T18:31:24.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.438+0000 7fe31e7fc640 1 --2- 172.21.15.5:0/1537155076 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe30c06be60 0x7fe30c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:24.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.438+0000 7fe31e7fc640 1 -- 172.21.15.5:0/1537155076 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5335+0+0 (secure 0 0 0) 0x7fe32408aa40 con 0x7fe3280b4010 2024-05-20T18:31:24.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.438+0000 7fe32d4c5640 1 --2- 172.21.15.5:0/1537155076 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe30c06be60 0x7fe30c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:24.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.438+0000 7fe32d4c5640 1 --2- 172.21.15.5:0/1537155076 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe30c06be60 0x7fe30c06e370 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe318034000 tx=0x7fe31802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:24.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.442+0000 7fe31e7fc640 1 -- 172.21.15.5:0/1537155076 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe324058f10 con 0x7fe3280b4010 2024-05-20T18:31:24.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.665+0000 7fe32ff2a640 1 -- 172.21.15.5:0/1537155076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3280d3050 con 0x7fe3280b4010 2024-05-20T18:31:24.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.665+0000 7fe31e7fc640 1 -- 172.21.15.5:0/1537155076 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+3860 (secure 0 0 0) 0x7fe32408ae20 con 0x7fe3280b4010 2024-05-20T18:31:24.666 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:31:24.666 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":38,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5480,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/4292435960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":4292435960},{"type":"v1","addr":"172.21.15.63:6835","nonce":4292435960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6186,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2991834087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2991834087},{"type":"v1","addr":"172.21.15.5:6836","nonce":2991834087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6195,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3462206198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":3462206198},{"type":"v1","addr":"172.21.15.5:6837","nonce":3462206198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6292,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/248804278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":248804278},{"type":"v1","addr":"172.21.15.63:6834","nonce":248804278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[]} 2024-05-20T18:31:24.666 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 38 2024-05-20T18:31:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.668+0000 7fe32ff2a640 1 -- 172.21.15.5:0/1537155076 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe30c06be60 msgr2=0x7fe30c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.668+0000 7fe32ff2a640 1 --2- 172.21.15.5:0/1537155076 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe30c06be60 0x7fe30c06e370 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe318034000 tx=0x7fe31802e000 comp rx=0 tx=0).stop 2024-05-20T18:31:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.668+0000 7fe32ff2a640 1 -- 172.21.15.5:0/1537155076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3280b4010 msgr2=0x7fe3280ce0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.668+0000 7fe32ff2a640 1 --2- 172.21.15.5:0/1537155076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3280b4010 0x7fe3280ce0f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fe32400e840 tx=0x7fe32400ed00 comp rx=0 tx=0).stop 2024-05-20T18:31:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.668+0000 7fe32ff2a640 1 -- 172.21.15.5:0/1537155076 shutdown_connections 2024-05-20T18:31:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.668+0000 7fe32ff2a640 1 --2- 172.21.15.5:0/1537155076 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe30c06be60 0x7fe30c06e370 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.668+0000 7fe32ff2a640 1 --2- 172.21.15.5:0/1537155076 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe328163a10 0x7fe3281f7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.668+0000 7fe32ff2a640 1 --2- 172.21.15.5:0/1537155076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3280d14b0 0x7fe3281f60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.668+0000 7fe32ff2a640 1 --2- 172.21.15.5:0/1537155076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3280b4010 0x7fe3280ce0f0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.668+0000 7fe32ff2a640 1 -- 172.21.15.5:0/1537155076 >> 172.21.15.5:0/1537155076 conn(0x7fe32806d240 msgr2=0x7fe328160e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.668+0000 7fe32ff2a640 1 -- 172.21.15.5:0/1537155076 shutdown_connections 2024-05-20T18:31:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.668+0000 7fe32ff2a640 1 -- 172.21.15.5:0/1537155076 wait complete. 2024-05-20T18:31:24.679 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-20T18:31:24.680 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-20T18:31:24.680 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-20T18:31:24.680 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:31:24.680 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-20T18:31:24.680 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-20T18:31:24.680 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-20T18:31:24.680 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:31:24.681 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-20T18:31:24.681 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-20T18:31:24.681 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-20T18:31:24.681 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:31:24.681 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-20T18:31:24.681 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-20T18:31:24.681 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-20T18:31:24.682 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:31:24.690 INFO:tasks.ceph.mds.a:Stopped 2024-05-20T18:31:24.690 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-20T18:31:24.692 INFO:tasks.ceph.mds.c:Stopped 2024-05-20T18:31:24.692 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-20T18:31:24.697 INFO:tasks.ceph.mds.d:Stopped 2024-05-20T18:31:24.697 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-20T18:31:24.698 INFO:tasks.ceph.mds.b:Stopped 2024-05-20T18:31:24.698 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-20T18:31:24.736 INFO:tasks.ceph.mds.a:Started 2024-05-20T18:31:24.737 DEBUG:teuthology.parallel:result is None 2024-05-20T18:31:24.738 INFO:tasks.ceph.mds.c:Started 2024-05-20T18:31:24.738 DEBUG:teuthology.parallel:result is None 2024-05-20T18:31:24.740 INFO:tasks.ceph.mds.d:Started 2024-05-20T18:31:24.741 DEBUG:teuthology.parallel:result is None 2024-05-20T18:31:24.742 INFO:tasks.ceph.mds.b:Started 2024-05-20T18:31:24.742 DEBUG:teuthology.parallel:result is None 2024-05-20T18:31:24.743 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-20T18:31:24.827 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-05-20T18:31:24.830 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-05-20T18:31:24.842 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-20T18:31:24.843 INFO:tasks.ceph.mds.d.smithi063.stdout:starting mds.d at 2024-05-20T18:31:24.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.866+0000 7fac728b6640 1 Processor -- start 2024-05-20T18:31:24.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.866+0000 7fac728b6640 1 -- start start 2024-05-20T18:31:24.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.867+0000 7fac728b6640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac6c163a10 0x7fac6c0d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:24.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.867+0000 7fac728b6640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fac6c0b4010 0x7fac6c0cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.867+0000 7fac728b6640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fac6c0d14b0 0x7fac6c0d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.868+0000 7fac728b6640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fac6c0d3ca0 con 0x7fac6c0d14b0 2024-05-20T18:31:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.868+0000 7fac728b6640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fac6c0d0370 con 0x7fac6c0b4010 2024-05-20T18:31:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.868+0000 7fac728b6640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fac6c0d0650 con 0x7fac6c163a10 2024-05-20T18:31:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.868+0000 7fac6bfff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac6c163a10 0x7fac6c0d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.868+0000 7fac6b7fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fac6c0b4010 0x7fac6c0cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.868+0000 7fac70e53640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fac6c0d14b0 0x7fac6c0d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.868+0000 7fac6bfff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac6c163a10 0x7fac6c0d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:37548/0 (socket says 172.21.15.5:37548) 2024-05-20T18:31:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.868+0000 7fac6bfff640 1 -- 172.21.15.5:0/3394571205 learned_addr learned my addr 172.21.15.5:0/3394571205 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.868+0000 7fac6b7fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fac6c0b4010 0x7fac6c0cfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:33092/0 (socket says 172.21.15.5:33092) 2024-05-20T18:31:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.868+0000 7fac70e53640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fac6c0d14b0 0x7fac6c0d18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35260/0 (socket says 172.21.15.5:35260) 2024-05-20T18:31:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.868+0000 7fac6bfff640 1 -- 172.21.15.5:0/3394571205 >> v2:172.21.15.63:3300/0 conn(0x7fac6c0b4010 msgr2=0x7fac6c0cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.868+0000 7fac6bfff640 1 --2- 172.21.15.5:0/3394571205 >> v2:172.21.15.63:3300/0 conn(0x7fac6c0b4010 0x7fac6c0cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:24.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.868+0000 7fac6bfff640 1 -- 172.21.15.5:0/3394571205 >> v2:172.21.15.5:3300/0 conn(0x7fac6c0d14b0 msgr2=0x7fac6c0d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:24.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.868+0000 7fac6bfff640 1 --2- 172.21.15.5:0/3394571205 >> v2:172.21.15.5:3300/0 conn(0x7fac6c0d14b0 0x7fac6c0d18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:24.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.868+0000 7fac6bfff640 1 -- 172.21.15.5:0/3394571205 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6c0d3650 con 0x7fac6c163a10 2024-05-20T18:31:24.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.868+0000 7fac70e53640 1 --2- 172.21.15.5:0/3394571205 >> v2:172.21.15.5:3300/0 conn(0x7fac6c0d14b0 0x7fac6c0d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:31:24.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.868+0000 7fac6b7fe640 1 --2- 172.21.15.5:0/3394571205 >> v2:172.21.15.63:3300/0 conn(0x7fac6c0b4010 0x7fac6c0cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:31:24.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.869+0000 7fac6bfff640 1 --2- 172.21.15.5:0/3394571205 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac6c163a10 0x7fac6c0d4130 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0b7710 tx=0x7fac5c02a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a7f691c1b9eb70b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:24.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.869+0000 7fac6affd640 1 -- 172.21.15.5:0/3394571205 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c037070 con 0x7fac6c163a10 2024-05-20T18:31:24.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.869+0000 7fac6affd640 1 -- 172.21.15.5:0/3394571205 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac5c032440 con 0x7fac6c163a10 2024-05-20T18:31:24.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.869+0000 7fac6affd640 1 -- 172.21.15.5:0/3394571205 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c032810 con 0x7fac6c163a10 2024-05-20T18:31:24.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.870+0000 7fac728b6640 1 -- 172.21.15.5:0/3394571205 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac6c163a10 msgr2=0x7fac6c0d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:24.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.870+0000 7fac728b6640 1 --2- 172.21.15.5:0/3394571205 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac6c163a10 0x7fac6c0d4130 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0b7710 tx=0x7fac5c02a1e0 comp rx=0 tx=0).stop 2024-05-20T18:31:24.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.870+0000 7fac728b6640 1 -- 172.21.15.5:0/3394571205 shutdown_connections 2024-05-20T18:31:24.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.870+0000 7fac728b6640 1 --2- 172.21.15.5:0/3394571205 >> v2:172.21.15.5:3300/0 conn(0x7fac6c0d14b0 0x7fac6c0d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:24.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.870+0000 7fac728b6640 1 --2- 172.21.15.5:0/3394571205 >> v2:172.21.15.63:3300/0 conn(0x7fac6c0b4010 0x7fac6c0cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:24.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.870+0000 7fac728b6640 1 --2- 172.21.15.5:0/3394571205 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac6c163a10 0x7fac6c0d4130 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:24.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.870+0000 7fac728b6640 1 -- 172.21.15.5:0/3394571205 >> 172.21.15.5:0/3394571205 conn(0x7fac6c06d240 msgr2=0x7fac6c06d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:24.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.870+0000 7fac728b6640 1 -- 172.21.15.5:0/3394571205 shutdown_connections 2024-05-20T18:31:24.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.870+0000 7fac728b6640 1 -- 172.21.15.5:0/3394571205 wait complete. 2024-05-20T18:31:24.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.872+0000 7fac728b6640 1 Processor -- start 2024-05-20T18:31:24.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.872+0000 7fac728b6640 1 -- start start 2024-05-20T18:31:24.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.873+0000 7fac728b6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac6c0b4010 0x7fac6c157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:24.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.873+0000 7fac728b6640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac6c0d14b0 0x7fac6c147bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:24.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.874+0000 7fac728b6640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac6c163a10 0x7fac6c149f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:24.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.874+0000 7fac728b6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c0d1e00 con 0x7fac6c0b4010 2024-05-20T18:31:24.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.874+0000 7fac728b6640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c0d0040 con 0x7fac6c0d14b0 2024-05-20T18:31:24.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.874+0000 7fac6bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac6c0b4010 0x7fac6c157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:24.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.874+0000 7fac6bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac6c0b4010 0x7fac6c157e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35270/0 (socket says 172.21.15.5:35270) 2024-05-20T18:31:24.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.874+0000 7fac6bfff640 1 -- 172.21.15.5:0/1624246347 learned_addr learned my addr 172.21.15.5:0/1624246347 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:24.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.874+0000 7fac728b6640 1 -- 172.21.15.5:0/1624246347 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fac6c0b8210 con 0x7fac6c163a10 2024-05-20T18:31:24.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.874+0000 7fac70e53640 1 --2- 172.21.15.5:0/1624246347 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac6c163a10 0x7fac6c149f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:24.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.874+0000 7fac6b7fe640 1 --2- 172.21.15.5:0/1624246347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac6c0d14b0 0x7fac6c147bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:24.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.874+0000 7fac6bfff640 1 -- 172.21.15.5:0/1624246347 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac6c163a10 msgr2=0x7fac6c149f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:24.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.874+0000 7fac6bfff640 1 --2- 172.21.15.5:0/1624246347 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac6c163a10 0x7fac6c149f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:24.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.874+0000 7fac6bfff640 1 -- 172.21.15.5:0/1624246347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac6c0d14b0 msgr2=0x7fac6c147bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:24.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.874+0000 7fac6bfff640 1 --2- 172.21.15.5:0/1624246347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac6c0d14b0 0x7fac6c147bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:24.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.874+0000 7fac6bfff640 1 -- 172.21.15.5:0/1624246347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6c158b60 con 0x7fac6c0b4010 2024-05-20T18:31:24.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.874+0000 7fac6bfff640 1 --2- 172.21.15.5:0/1624246347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac6c0b4010 0x7fac6c157e00 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fac5c02a220 tx=0x7fac5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:24.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.874+0000 7fac68ff9640 1 -- 172.21.15.5:0/1624246347 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c041020 con 0x7fac6c0b4010 2024-05-20T18:31:24.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.874+0000 7fac68ff9640 1 -- 172.21.15.5:0/1624246347 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac5c03d620 con 0x7fac6c0b4010 2024-05-20T18:31:24.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.874+0000 7fac68ff9640 1 -- 172.21.15.5:0/1624246347 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c037040 con 0x7fac6c0b4010 2024-05-20T18:31:24.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.874+0000 7fac728b6640 1 -- 172.21.15.5:0/1624246347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac6c158e40 con 0x7fac6c0b4010 2024-05-20T18:31:24.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.875+0000 7fac728b6640 1 -- 172.21.15.5:0/1624246347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac6c159400 con 0x7fac6c0b4010 2024-05-20T18:31:24.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.875+0000 7fac728b6640 1 -- 172.21.15.5:0/1624246347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac6c075b00 con 0x7fac6c0b4010 2024-05-20T18:31:24.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.876+0000 7fac68ff9640 1 -- 172.21.15.5:0/1624246347 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fac5c044020 con 0x7fac6c0b4010 2024-05-20T18:31:24.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.876+0000 7fac68ff9640 1 --2- 172.21.15.5:0/1624246347 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fac5806be30 0x7fac5806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:24.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.877+0000 7fac68ff9640 1 -- 172.21.15.5:0/1624246347 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5335+0+0 (secure 0 0 0) 0x7fac5c0ab360 con 0x7fac6c0b4010 2024-05-20T18:31:24.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.877+0000 7fac6b7fe640 1 --2- 172.21.15.5:0/1624246347 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fac5806be30 0x7fac5806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:24.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.877+0000 7fac6b7fe640 1 --2- 172.21.15.5:0/1624246347 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fac5806be30 0x7fac5806e340 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fac6c15d1e0 tx=0x7fac60009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:24.881+0000 7fac68ff9640 1 -- 172.21.15.5:0/1624246347 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fac5c04b030 con 0x7fac6c0b4010 2024-05-20T18:31:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:25.077+0000 7fac728b6640 1 -- 172.21.15.5:0/1624246347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fac6c0d3050 con 0x7fac6c0b4010 2024-05-20T18:31:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.105+0000 7fac68ff9640 1 -- 172.21.15.5:0/1624246347 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v73) ==== 97+0+0 (secure 0 0 0) 0x7fac5c0a9e20 con 0x7fac6c0b4010 2024-05-20T18:31:26.105 INFO:teuthology.orchestra.run.smithi005.stderr: removed all blocklist entries 2024-05-20T18:31:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.108+0000 7fac728b6640 1 -- 172.21.15.5:0/1624246347 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fac5806be30 msgr2=0x7fac5806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.108+0000 7fac728b6640 1 --2- 172.21.15.5:0/1624246347 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fac5806be30 0x7fac5806e340 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fac6c15d1e0 tx=0x7fac60009190 comp rx=0 tx=0).stop 2024-05-20T18:31:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.108+0000 7fac728b6640 1 -- 172.21.15.5:0/1624246347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac6c0b4010 msgr2=0x7fac6c157e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.108+0000 7fac728b6640 1 --2- 172.21.15.5:0/1624246347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac6c0b4010 0x7fac6c157e00 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fac5c02a220 tx=0x7fac5c0046c0 comp rx=0 tx=0).stop 2024-05-20T18:31:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.108+0000 7fac728b6640 1 -- 172.21.15.5:0/1624246347 shutdown_connections 2024-05-20T18:31:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.108+0000 7fac728b6640 1 --2- 172.21.15.5:0/1624246347 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fac5806be30 0x7fac5806e340 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.108+0000 7fac728b6640 1 --2- 172.21.15.5:0/1624246347 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac6c163a10 0x7fac6c149f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:26.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.108+0000 7fac728b6640 1 --2- 172.21.15.5:0/1624246347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac6c0d14b0 0x7fac6c147bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:26.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.108+0000 7fac728b6640 1 --2- 172.21.15.5:0/1624246347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac6c0b4010 0x7fac6c157e00 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:26.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.108+0000 7fac728b6640 1 -- 172.21.15.5:0/1624246347 >> 172.21.15.5:0/1624246347 conn(0x7fac6c06d240 msgr2=0x7fac6c15ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:26.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.108+0000 7fac728b6640 1 -- 172.21.15.5:0/1624246347 shutdown_connections 2024-05-20T18:31:26.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.108+0000 7fac728b6640 1 -- 172.21.15.5:0/1624246347 wait complete. 2024-05-20T18:31:26.122 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-20T18:31:26.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.244+0000 7f2b8bb1a640 1 Processor -- start 2024-05-20T18:31:26.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.244+0000 7f2b8bb1a640 1 -- start start 2024-05-20T18:31:26.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.245+0000 7f2b8bb1a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2b84163a10 0x7f2b840d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:26.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.245+0000 7f2b8bb1a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b840b4010 0x7f2b840cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:26.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.245+0000 7f2b8bb1a640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b840d0370 con 0x7f2b840d14b0 2024-05-20T18:31:26.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.245+0000 7f2b8bb1a640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2b840d3ca0 con 0x7f2b84163a10 2024-05-20T18:31:26.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.245+0000 7f2b8bb1a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2b840d0650 con 0x7f2b840b4010 2024-05-20T18:31:26.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.246+0000 7f2b898b6640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2b84163a10 0x7f2b840d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:26.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.246+0000 7f2b890b5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b840b4010 0x7f2b840cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:26.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.246+0000 7f2b898b6640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2b84163a10 0x7f2b840d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:33104/0 (socket says 172.21.15.5:33104) 2024-05-20T18:31:26.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.246+0000 7f2b898b6640 1 -- 172.21.15.5:0/4082011360 learned_addr learned my addr 172.21.15.5:0/4082011360 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:26.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.246+0000 7f2b890b5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b840b4010 0x7f2b840cfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:37568/0 (socket says 172.21.15.5:37568) 2024-05-20T18:31:26.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.246+0000 7f2b890b5640 1 -- 172.21.15.5:0/4082011360 >> v2:172.21.15.63:3300/0 conn(0x7f2b84163a10 msgr2=0x7f2b840d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:26.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.246+0000 7f2b890b5640 1 --2- 172.21.15.5:0/4082011360 >> v2:172.21.15.63:3300/0 conn(0x7f2b84163a10 0x7f2b840d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:26.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.246+0000 7f2b890b5640 1 -- 172.21.15.5:0/4082011360 >> v1:172.21.15.63:6789/0 conn(0x7f2b840d14b0 legacy=0x7f2b840d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:26.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.246+0000 7f2b890b5640 1 -- 172.21.15.5:0/4082011360 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b840d3650 con 0x7f2b840b4010 2024-05-20T18:31:26.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.246+0000 7f2b898b6640 1 --2- 172.21.15.5:0/4082011360 >> v2:172.21.15.63:3300/0 conn(0x7f2b84163a10 0x7f2b840d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:31:26.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.247+0000 7f2b890b5640 1 --2- 172.21.15.5:0/4082011360 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b840b4010 0x7f2b840cfb00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2b740097a0 tx=0x7f2b7402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87c120e511ab07fd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:26.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.247+0000 7f2b888b4640 1 -- 172.21.15.5:0/4082011360 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b74037070 con 0x7f2b840b4010 2024-05-20T18:31:26.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.247+0000 7f2b888b4640 1 -- 172.21.15.5:0/4082011360 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b74032440 con 0x7f2b840b4010 2024-05-20T18:31:26.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.247+0000 7f2b888b4640 1 -- 172.21.15.5:0/4082011360 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b74032810 con 0x7f2b840b4010 2024-05-20T18:31:26.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.248+0000 7f2b8bb1a640 1 -- 172.21.15.5:0/4082011360 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b840b4010 msgr2=0x7f2b840cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:26.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.248+0000 7f2b8bb1a640 1 --2- 172.21.15.5:0/4082011360 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b840b4010 0x7f2b840cfb00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2b740097a0 tx=0x7f2b7402a180 comp rx=0 tx=0).stop 2024-05-20T18:31:26.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.248+0000 7f2b8bb1a640 1 -- 172.21.15.5:0/4082011360 shutdown_connections 2024-05-20T18:31:26.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.248+0000 7f2b8bb1a640 1 --2- 172.21.15.5:0/4082011360 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b840b4010 0x7f2b840cfb00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:26.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.248+0000 7f2b8bb1a640 1 --2- 172.21.15.5:0/4082011360 >> v2:172.21.15.63:3300/0 conn(0x7f2b84163a10 0x7f2b840d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:26.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.248+0000 7f2b8bb1a640 1 -- 172.21.15.5:0/4082011360 >> 172.21.15.5:0/4082011360 conn(0x7f2b8406d240 msgr2=0x7f2b8406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:26.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.248+0000 7f2b8bb1a640 1 -- 172.21.15.5:0/4082011360 shutdown_connections 2024-05-20T18:31:26.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.248+0000 7f2b8bb1a640 1 -- 172.21.15.5:0/4082011360 wait complete. 2024-05-20T18:31:26.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.250+0000 7f2b8bb1a640 1 Processor -- start 2024-05-20T18:31:26.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.250+0000 7f2b8bb1a640 1 -- start start 2024-05-20T18:31:26.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.251+0000 7f2b8bb1a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b840b4010 0x7f2b840ac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:26.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.251+0000 7f2b8bb1a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b840d14b0 0x7f2b8415cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:26.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.251+0000 7f2b8bb1a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b84163a10 0x7f2b840ab580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:26.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.251+0000 7f2b8bb1a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2b840b8210 con 0x7f2b840d14b0 2024-05-20T18:31:26.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.251+0000 7f2b8bb1a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2b840d0040 con 0x7f2b840b4010 2024-05-20T18:31:26.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.251+0000 7f2b8bb1a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2b840b8390 con 0x7f2b84163a10 2024-05-20T18:31:26.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.251+0000 7f2b890b5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b840d14b0 0x7f2b8415cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:26.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.251+0000 7f2b890b5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b840d14b0 0x7f2b8415cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35286/0 (socket says 172.21.15.5:35286) 2024-05-20T18:31:26.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.251+0000 7f2b890b5640 1 -- 172.21.15.5:0/3360655631 learned_addr learned my addr 172.21.15.5:0/3360655631 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:26.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.251+0000 7f2b898b6640 1 --2- 172.21.15.5:0/3360655631 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b840b4010 0x7f2b840ac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:26.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.251+0000 7f2b8a0b7640 1 --2- 172.21.15.5:0/3360655631 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b84163a10 0x7f2b840ab580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:26.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.252+0000 7f2b890b5640 1 -- 172.21.15.5:0/3360655631 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b84163a10 msgr2=0x7f2b840ab580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:26.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.252+0000 7f2b890b5640 1 --2- 172.21.15.5:0/3360655631 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b84163a10 0x7f2b840ab580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:26.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.252+0000 7f2b890b5640 1 -- 172.21.15.5:0/3360655631 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b840b4010 msgr2=0x7f2b840ac3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:26.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.252+0000 7f2b890b5640 1 --2- 172.21.15.5:0/3360655631 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b840b4010 0x7f2b840ac3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:26.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.252+0000 7f2b890b5640 1 -- 172.21.15.5:0/3360655631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b840ae2a0 con 0x7f2b840d14b0 2024-05-20T18:31:26.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.252+0000 7f2b898b6640 1 --2- 172.21.15.5:0/3360655631 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b840b4010 0x7f2b840ac3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:26.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.252+0000 7f2b8a0b7640 1 --2- 172.21.15.5:0/3360655631 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b84163a10 0x7f2b840ab580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:26.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.252+0000 7f2b890b5640 1 --2- 172.21.15.5:0/3360655631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b840d14b0 0x7f2b8415cc00 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f2b74000c00 tx=0x7f2b740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:26.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.252+0000 7f2b7a7fc640 1 -- 172.21.15.5:0/3360655631 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b74037040 con 0x7f2b840d14b0 2024-05-20T18:31:26.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.252+0000 7f2b7a7fc640 1 -- 172.21.15.5:0/3360655631 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b74042070 con 0x7f2b840d14b0 2024-05-20T18:31:26.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.252+0000 7f2b8bb1a640 1 -- 172.21.15.5:0/3360655631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b840adfc0 con 0x7f2b840d14b0 2024-05-20T18:31:26.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.252+0000 7f2b7a7fc640 1 -- 172.21.15.5:0/3360655631 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b74030940 con 0x7f2b840d14b0 2024-05-20T18:31:26.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.252+0000 7f2b8bb1a640 1 -- 172.21.15.5:0/3360655631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b841ea070 con 0x7f2b840d14b0 2024-05-20T18:31:26.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.253+0000 7f2b8bb1a640 1 -- 172.21.15.5:0/3360655631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b840add30 con 0x7f2b840d14b0 2024-05-20T18:31:26.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.254+0000 7f2b7a7fc640 1 -- 172.21.15.5:0/3360655631 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2b74030ae0 con 0x7f2b840d14b0 2024-05-20T18:31:26.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.254+0000 7f2b7a7fc640 1 --2- 172.21.15.5:0/3360655631 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2b6806be60 0x7f2b6806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:26.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.254+0000 7f2b7a7fc640 1 -- 172.21.15.5:0/3360655631 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5249+0+0 (secure 0 0 0) 0x7f2b740ab310 con 0x7f2b840d14b0 2024-05-20T18:31:26.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.254+0000 7f2b898b6640 1 --2- 172.21.15.5:0/3360655631 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2b6806be60 0x7f2b6806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:26.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.255+0000 7f2b898b6640 1 --2- 172.21.15.5:0/3360655631 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2b6806be60 0x7f2b6806e370 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2b840d3020 tx=0x7f2b80008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.259+0000 7f2b7a7fc640 1 -- 172.21.15.5:0/3360655631 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2b840ae2a0 con 0x7f2b840d14b0 2024-05-20T18:31:26.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.512+0000 7f2b8bb1a640 1 -- 172.21.15.5:0/3360655631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f2b84075b00 con 0x7f2b840d14b0 2024-05-20T18:31:26.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.512+0000 7f2b7a7fc640 1 -- 172.21.15.5:0/3360655631 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7f2b74079700 con 0x7f2b840d14b0 2024-05-20T18:31:26.513 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:31:26.514 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:31:26.514 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_dump": [ 2024-05-20T18:31:26.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:31:26.514 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.a", 2024-05-20T18:31:26.514 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtm9lSeCBAA/PpiEqojk6ueJo60tU4/iA==", 2024-05-20T18:31:26.515 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:31:26.515 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:31:26.515 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:31:26.515 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:31:26.515 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:31:26.515 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:31:26.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:31:26.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:31:26.516 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.b", 2024-05-20T18:31:26.516 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmdov0FRAACkWAR/0zaCDjgnkHnoHwgw==", 2024-05-20T18:31:26.516 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:31:26.517 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:31:26.517 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:31:26.517 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:31:26.517 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:31:26.517 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:31:26.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:31:26.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:31:26.518 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.c", 2024-05-20T18:31:26.518 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmpEBlEBAAdgxDXj9x38VSOizUgdFpYA==", 2024-05-20T18:31:26.518 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:31:26.519 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:31:26.519 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:31:26.519 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:31:26.519 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:31:26.519 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:31:26.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:31:26.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:31:26.520 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.d", 2024-05-20T18:31:26.520 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtm0Gy9HRAAAkPqcBLwxTwuWmCi7aad1A==", 2024-05-20T18:31:26.520 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:31:26.521 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:31:26.521 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:31:26.521 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:31:26.521 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:31:26.521 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:31:26.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:31:26.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:31:26.522 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.0", 2024-05-20T18:31:26.522 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDHlUtmWFGzBRAA81qN37dlHf6QDZ1FgwbJ9w==", 2024-05-20T18:31:26.522 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:31:26.523 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:31:26.523 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:31:26.523 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:31:26.523 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:31:26.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:31:26.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:31:26.524 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.1", 2024-05-20T18:31:26.524 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDKlUtmBMKsBBAAD4tTTvetME6oxwugTubSKQ==", 2024-05-20T18:31:26.524 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:31:26.524 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:31:26.524 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:31:26.525 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:31:26.525 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:31:26.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:31:26.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:31:26.525 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.2", 2024-05-20T18:31:26.526 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDNlUtm4Rm/AxAAIGWTUxb+LII09Rd+h5cJ8Q==", 2024-05-20T18:31:26.526 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:31:26.526 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:31:26.527 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:31:26.527 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:31:26.527 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:31:26.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:31:26.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:31:26.528 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.3", 2024-05-20T18:31:26.528 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDQlUtmWVxZAxAAB/w6k4WJ58ulEr9zatCRfw==", 2024-05-20T18:31:26.528 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:31:26.528 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:31:26.528 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:31:26.529 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:31:26.529 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:31:26.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:31:26.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:31:26.529 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.4", 2024-05-20T18:31:26.530 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDWlUtmIyokCRAAolOiYVCYaNTQZb1PjtR6fg==", 2024-05-20T18:31:26.530 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:31:26.530 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:31:26.530 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:31:26.530 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:31:26.531 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:31:26.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:31:26.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:31:26.531 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.5", 2024-05-20T18:31:26.531 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZlUtmSf6mCBAAw73nwq/zzt9kIXZymZh93A==", 2024-05-20T18:31:26.531 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:31:26.532 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:31:26.532 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:31:26.532 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:31:26.532 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:31:26.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:31:26.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:31:26.533 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.6", 2024-05-20T18:31:26.533 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDclUtmwwhGCBAA94OBEtj9h7cmCAddYMonaA==", 2024-05-20T18:31:26.533 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:31:26.533 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:31:26.533 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:31:26.534 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:31:26.534 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:31:26.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:31:26.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:31:26.534 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.7", 2024-05-20T18:31:26.534 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDflUtm9wefBxAAvCZTBzRu05koFhc+pH5I8w==", 2024-05-20T18:31:26.534 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:31:26.534 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:31:26.534 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:31:26.535 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:31:26.535 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:31:26.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:31:26.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:31:26.536 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.0", 2024-05-20T18:31:26.536 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmevaaIhAATIwyXzr0vGT38ZmcMqR2gA==", 2024-05-20T18:31:26.536 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:31:26.536 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:31:26.536 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow r", 2024-05-20T18:31:26.536 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-20T18:31:26.536 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:31:26.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:31:26.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:31:26.537 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.1", 2024-05-20T18:31:26.537 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmK7NfKhAADaXl0uVP1Q1gUOlPsstnzg==", 2024-05-20T18:31:26.537 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:31:26.537 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:31:26.537 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-20T18:31:26.537 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-20T18:31:26.537 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-20T18:31:26.537 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:31:26.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:31:26.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:31:26.538 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.2", 2024-05-20T18:31:26.538 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmMJF1JhAAeM+HseqvgvKgG8EnIsZuhg==", 2024-05-20T18:31:26.538 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:31:26.539 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:31:26.539 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-20T18:31:26.539 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-20T18:31:26.539 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-20T18:31:26.539 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:31:26.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:31:26.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:31:26.540 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.3", 2024-05-20T18:31:26.540 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmodA7LhAAUI2v0npB6HHs2n1g3KdTXA==", 2024-05-20T18:31:26.540 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:31:26.540 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:31:26.540 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-20T18:31:26.541 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-20T18:31:26.541 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-20T18:31:26.541 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:31:26.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:31:26.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:31:26.542 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.admin", 2024-05-20T18:31:26.542 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDElUtmZacGJRAAoyr9XNPBr9r0DSU53qCHSQ==", 2024-05-20T18:31:26.542 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:31:26.542 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-20T18:31:26.542 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:31:26.543 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:31:26.543 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:31:26.543 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:31:26.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:31:26.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:31:26.543 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.x", 2024-05-20T18:31:26.544 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmPgq0BBAAAljNeQDsSau0wH2Lh2KdBw==", 2024-05-20T18:31:26.544 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:31:26.544 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-20T18:31:26.544 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-05-20T18:31:26.544 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:31:26.544 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:31:26.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:31:26.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:31:26.545 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.y", 2024-05-20T18:31:26.545 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDElUtm6Xv3OhAA0doOWTC73jy5rOYm7Vf68Q==", 2024-05-20T18:31:26.545 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:31:26.546 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-20T18:31:26.546 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-05-20T18:31:26.546 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:31:26.546 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:31:26.546 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:31:26.546 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-20T18:31:26.547 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:31:26.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.514+0000 7f2b8bb1a640 1 -- 172.21.15.5:0/3360655631 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2b6806be60 msgr2=0x7f2b6806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:26.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.514+0000 7f2b8bb1a640 1 --2- 172.21.15.5:0/3360655631 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2b6806be60 0x7f2b6806e370 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2b840d3020 tx=0x7f2b80008040 comp rx=0 tx=0).stop 2024-05-20T18:31:26.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.514+0000 7f2b8bb1a640 1 -- 172.21.15.5:0/3360655631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b840d14b0 msgr2=0x7f2b8415cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:26.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.514+0000 7f2b8bb1a640 1 --2- 172.21.15.5:0/3360655631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b840d14b0 0x7f2b8415cc00 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f2b74000c00 tx=0x7f2b740046c0 comp rx=0 tx=0).stop 2024-05-20T18:31:26.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.514+0000 7f2b8bb1a640 1 -- 172.21.15.5:0/3360655631 shutdown_connections 2024-05-20T18:31:26.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.514+0000 7f2b8bb1a640 1 --2- 172.21.15.5:0/3360655631 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2b6806be60 0x7f2b6806e370 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:26.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.514+0000 7f2b8bb1a640 1 --2- 172.21.15.5:0/3360655631 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b84163a10 0x7f2b840ab580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:26.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.514+0000 7f2b8bb1a640 1 --2- 172.21.15.5:0/3360655631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b840d14b0 0x7f2b8415cc00 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:26.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.514+0000 7f2b8bb1a640 1 --2- 172.21.15.5:0/3360655631 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b840b4010 0x7f2b840ac3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:26.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.514+0000 7f2b8bb1a640 1 -- 172.21.15.5:0/3360655631 >> 172.21.15.5:0/3360655631 conn(0x7f2b8406d240 msgr2=0x7f2b8415f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:26.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.515+0000 7f2b8bb1a640 1 -- 172.21.15.5:0/3360655631 shutdown_connections 2024-05-20T18:31:26.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.515+0000 7f2b8bb1a640 1 -- 172.21.15.5:0/3360655631 wait complete. 2024-05-20T18:31:26.552 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:31:26.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.667+0000 7fe925ade640 1 Processor -- start 2024-05-20T18:31:26.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.667+0000 7fe925ade640 1 -- start start 2024-05-20T18:31:26.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.668+0000 7fe925ade640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe9200cd680 0x7fe92015f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.668+0000 7fe925ade640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9200cfae0 0x7fe92006cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.668+0000 7fe925ade640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe92006d0f0 con 0x7fe92006c6c0 2024-05-20T18:31:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.668+0000 7fe925ade640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe9200cda90 con 0x7fe9200cd680 2024-05-20T18:31:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.668+0000 7fe925ade640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe92015f760 con 0x7fe9200cfae0 2024-05-20T18:31:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.668+0000 7fe91f7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe9200cd680 0x7fe92015f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.668+0000 7fe91f7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe9200cd680 0x7fe92015f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35290/0 (socket says 172.21.15.5:35290) 2024-05-20T18:31:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.668+0000 7fe91f7fe640 1 -- 172.21.15.5:0/245973358 learned_addr learned my addr 172.21.15.5:0/245973358 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.668+0000 7fe91ffff640 1 --2- 172.21.15.5:0/245973358 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9200cfae0 0x7fe92006cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.668+0000 7fe91f7fe640 1 -- 172.21.15.5:0/245973358 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9200cfae0 msgr2=0x7fe92006cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.668+0000 7fe91f7fe640 1 --2- 172.21.15.5:0/245973358 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9200cfae0 0x7fe92006cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.668+0000 7fe91f7fe640 1 -- 172.21.15.5:0/245973358 >> v1:172.21.15.63:6789/0 conn(0x7fe92006c6c0 legacy=0x7fe9200cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.668+0000 7fe91f7fe640 1 -- 172.21.15.5:0/245973358 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92006d3d0 con 0x7fe9200cd680 2024-05-20T18:31:26.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.669+0000 7fe91f7fe640 1 --2- 172.21.15.5:0/245973358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9200cd680 0x7fe92015f220 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fe910009e50 tx=0x7fe91002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b6929e69cdd437 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:26.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.669+0000 7fe91e7fc640 1 -- 172.21.15.5:0/245973358 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe910036070 con 0x7fe9200cd680 2024-05-20T18:31:26.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.669+0000 7fe91e7fc640 1 -- 172.21.15.5:0/245973358 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe910031600 con 0x7fe9200cd680 2024-05-20T18:31:26.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.669+0000 7fe91e7fc640 1 -- 172.21.15.5:0/245973358 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9100319d0 con 0x7fe9200cd680 2024-05-20T18:31:26.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.670+0000 7fe925ade640 1 -- 172.21.15.5:0/245973358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9200cd680 msgr2=0x7fe92015f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:26.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.670+0000 7fe925ade640 1 --2- 172.21.15.5:0/245973358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9200cd680 0x7fe92015f220 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fe910009e50 tx=0x7fe91002a680 comp rx=0 tx=0).stop 2024-05-20T18:31:26.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.670+0000 7fe925ade640 1 -- 172.21.15.5:0/245973358 shutdown_connections 2024-05-20T18:31:26.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.670+0000 7fe925ade640 1 --2- 172.21.15.5:0/245973358 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9200cfae0 0x7fe92006cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:26.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.670+0000 7fe925ade640 1 --2- 172.21.15.5:0/245973358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9200cd680 0x7fe92015f220 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:26.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.670+0000 7fe925ade640 1 -- 172.21.15.5:0/245973358 >> 172.21.15.5:0/245973358 conn(0x7fe9200b44d0 msgr2=0x7fe9200b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:26.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.670+0000 7fe925ade640 1 -- 172.21.15.5:0/245973358 shutdown_connections 2024-05-20T18:31:26.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.670+0000 7fe925ade640 1 -- 172.21.15.5:0/245973358 wait complete. 2024-05-20T18:31:26.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.672+0000 7fe925ade640 1 Processor -- start 2024-05-20T18:31:26.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.672+0000 7fe925ade640 1 -- start start 2024-05-20T18:31:26.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.673+0000 7fe925ade640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe92006c6c0 0x7fe9200c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:26.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.673+0000 7fe925ade640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9200cd680 0x7fe920151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:26.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.673+0000 7fe925ade640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9201f5670 0x7fe9200c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:26.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.673+0000 7fe925ade640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe92006d990 con 0x7fe9201f5670 2024-05-20T18:31:26.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.673+0000 7fe925ade640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe9200d0b40 con 0x7fe9200cd680 2024-05-20T18:31:26.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.673+0000 7fe925ade640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe9200d2460 con 0x7fe92006c6c0 2024-05-20T18:31:26.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.673+0000 7fe91ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9201f5670 0x7fe9200c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:26.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.673+0000 7fe91ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9201f5670 0x7fe9200c76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35292/0 (socket says 172.21.15.5:35292) 2024-05-20T18:31:26.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.673+0000 7fe91ffff640 1 -- 172.21.15.5:0/2342866275 learned_addr learned my addr 172.21.15.5:0/2342866275 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:26.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.674+0000 7fe91f7fe640 1 --2- 172.21.15.5:0/2342866275 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe92006c6c0 0x7fe9200c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:26.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.674+0000 7fe91effd640 1 --2- 172.21.15.5:0/2342866275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9200cd680 0x7fe920151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:26.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.674+0000 7fe91ffff640 1 -- 172.21.15.5:0/2342866275 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe92006c6c0 msgr2=0x7fe9200c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:26.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.674+0000 7fe91ffff640 1 --2- 172.21.15.5:0/2342866275 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe92006c6c0 0x7fe9200c8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:26.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.674+0000 7fe91ffff640 1 -- 172.21.15.5:0/2342866275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9200cd680 msgr2=0x7fe920151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:26.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.674+0000 7fe91ffff640 1 --2- 172.21.15.5:0/2342866275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9200cd680 0x7fe920151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:26.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.674+0000 7fe91ffff640 1 -- 172.21.15.5:0/2342866275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9200c8c40 con 0x7fe9201f5670 2024-05-20T18:31:26.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.674+0000 7fe91effd640 1 --2- 172.21.15.5:0/2342866275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9200cd680 0x7fe920151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:26.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.674+0000 7fe91ffff640 1 --2- 172.21.15.5:0/2342866275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9201f5670 0x7fe9200c76f0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fe91800d600 tx=0x7fe91800dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:26.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.674+0000 7fe917fff640 1 -- 172.21.15.5:0/2342866275 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9180125f0 con 0x7fe9201f5670 2024-05-20T18:31:26.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.674+0000 7fe917fff640 1 -- 172.21.15.5:0/2342866275 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe918019070 con 0x7fe9201f5670 2024-05-20T18:31:26.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.674+0000 7fe925ade640 1 -- 172.21.15.5:0/2342866275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe920152290 con 0x7fe9201f5670 2024-05-20T18:31:26.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.674+0000 7fe917fff640 1 -- 172.21.15.5:0/2342866275 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9180129f0 con 0x7fe9201f5670 2024-05-20T18:31:26.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.674+0000 7fe925ade640 1 -- 172.21.15.5:0/2342866275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9201f6130 con 0x7fe9201f5670 2024-05-20T18:31:26.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.675+0000 7fe925ade640 1 -- 172.21.15.5:0/2342866275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe920075b00 con 0x7fe9201f5670 2024-05-20T18:31:26.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.676+0000 7fe917fff640 1 -- 172.21.15.5:0/2342866275 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe918012b90 con 0x7fe9201f5670 2024-05-20T18:31:26.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.677+0000 7fe917fff640 1 --2- 172.21.15.5:0/2342866275 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe90006be60 0x7fe90006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:26.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.677+0000 7fe917fff640 1 -- 172.21.15.5:0/2342866275 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5249+0+0 (secure 0 0 0) 0x7fe91808b090 con 0x7fe9201f5670 2024-05-20T18:31:26.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.677+0000 7fe91f7fe640 1 --2- 172.21.15.5:0/2342866275 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe90006be60 0x7fe90006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:26.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.677+0000 7fe91f7fe640 1 --2- 172.21.15.5:0/2342866275 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe90006be60 0x7fe90006e370 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe910033000 tx=0x7fe910009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:26.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.681+0000 7fe917fff640 1 -- 172.21.15.5:0/2342866275 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe918059b70 con 0x7fe9201f5670 2024-05-20T18:31:26.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.869+0000 7fe925ade640 1 -- 172.21.15.5:0/2342866275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9200b8940 con 0x7fe9201f5670 2024-05-20T18:31:26.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.870+0000 7fe917fff640 1 -- 172.21.15.5:0/2342866275 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+13045 (secure 0 0 0) 0x7fe918059470 con 0x7fe9201f5670 2024-05-20T18:31:26.870 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:31:26.870 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":73,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:31:26.104134+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:31:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.872+0000 7fe925ade640 1 -- 172.21.15.5:0/2342866275 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe90006be60 msgr2=0x7fe90006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.872+0000 7fe925ade640 1 --2- 172.21.15.5:0/2342866275 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe90006be60 0x7fe90006e370 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe910033000 tx=0x7fe910009000 comp rx=0 tx=0).stop 2024-05-20T18:31:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.872+0000 7fe925ade640 1 -- 172.21.15.5:0/2342866275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9201f5670 msgr2=0x7fe9200c76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.872+0000 7fe925ade640 1 --2- 172.21.15.5:0/2342866275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9201f5670 0x7fe9200c76f0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fe91800d600 tx=0x7fe91800dac0 comp rx=0 tx=0).stop 2024-05-20T18:31:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.872+0000 7fe925ade640 1 -- 172.21.15.5:0/2342866275 shutdown_connections 2024-05-20T18:31:26.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.872+0000 7fe925ade640 1 --2- 172.21.15.5:0/2342866275 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe90006be60 0x7fe90006e370 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:26.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.872+0000 7fe925ade640 1 --2- 172.21.15.5:0/2342866275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9201f5670 0x7fe9200c76f0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:26.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.872+0000 7fe925ade640 1 --2- 172.21.15.5:0/2342866275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9200cd680 0x7fe920151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:26.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.872+0000 7fe925ade640 1 --2- 172.21.15.5:0/2342866275 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe92006c6c0 0x7fe9200c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:26.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.872+0000 7fe925ade640 1 -- 172.21.15.5:0/2342866275 >> 172.21.15.5:0/2342866275 conn(0x7fe9200b44d0 msgr2=0x7fe9200708d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:26.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.872+0000 7fe925ade640 1 -- 172.21.15.5:0/2342866275 shutdown_connections 2024-05-20T18:31:26.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:26.872+0000 7fe925ade640 1 -- 172.21.15.5:0/2342866275 wait complete. 2024-05-20T18:31:26.884 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:31:26.884 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:31:27.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.002+0000 7f93a1b65640 1 Processor -- start 2024-05-20T18:31:27.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.002+0000 7f93a1b65640 1 -- start start 2024-05-20T18:31:27.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.003+0000 7f93a1b65640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f939c071630 0x7f939c15f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:27.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.003+0000 7f93a1b65640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f939c070850 0x7f939c070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:27.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.003+0000 7f93a1b65640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f939c0711a0 con 0x7f939c0cd1b0 2024-05-20T18:31:27.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.003+0000 7f93a1b65640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f939c15f790 con 0x7f939c070850 2024-05-20T18:31:27.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.003+0000 7f93a1b65640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f939c06e590 con 0x7f939c071630 2024-05-20T18:31:27.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.003+0000 7f939bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f939c070850 0x7f939c070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:27.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.003+0000 7f939affd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f939c0cd1b0 0x7f939c06d7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43406/0 (socket says 172.21.15.5:43406) 2024-05-20T18:31:27.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.003+0000 7f939affd640 1 -- 172.21.15.5:0/3179398670 learned_addr learned my addr 172.21.15.5:0/3179398670 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:27.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.004+0000 7f939b7fe640 1 --2- 172.21.15.5:0/3179398670 >> v2:172.21.15.63:3300/0 conn(0x7f939c071630 0x7f939c15f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:27.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.004+0000 7f939bfff640 1 -- 172.21.15.5:0/3179398670 >> v2:172.21.15.63:3300/0 conn(0x7f939c071630 msgr2=0x7f939c15f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:27.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.004+0000 7f939bfff640 1 --2- 172.21.15.5:0/3179398670 >> v2:172.21.15.63:3300/0 conn(0x7f939c071630 0x7f939c15f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:27.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.004+0000 7f939bfff640 1 -- 172.21.15.5:0/3179398670 >> v1:172.21.15.5:6789/0 conn(0x7f939c0cd1b0 legacy=0x7f939c06d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:27.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.004+0000 7f939a7fc640 1 -- 172.21.15.5:0/3179398670 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3423284121 0 0) 0x7f9384001110 con 0x7f939c0cd1b0 2024-05-20T18:31:27.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.004+0000 7f939bfff640 1 -- 172.21.15.5:0/3179398670 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939c0ce2b0 con 0x7f939c070850 2024-05-20T18:31:27.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.005+0000 7f939bfff640 1 --2- 172.21.15.5:0/3179398670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f939c070850 0x7f939c070c60 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f939c068510 tx=0x7f939402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5556f3682a6b9c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:27.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.005+0000 7f939a7fc640 1 -- 172.21.15.5:0/3179398670 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9394037070 con 0x7f939c070850 2024-05-20T18:31:27.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.005+0000 7f939a7fc640 1 -- 172.21.15.5:0/3179398670 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9394032440 con 0x7f939c070850 2024-05-20T18:31:27.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.005+0000 7f939a7fc640 1 -- 172.21.15.5:0/3179398670 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939402ae60 con 0x7f939c070850 2024-05-20T18:31:27.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.005+0000 7f93a1b65640 1 -- 172.21.15.5:0/3179398670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f939c070850 msgr2=0x7f939c070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:27.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.005+0000 7f93a1b65640 1 --2- 172.21.15.5:0/3179398670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f939c070850 0x7f939c070c60 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f939c068510 tx=0x7f939402a110 comp rx=0 tx=0).stop 2024-05-20T18:31:27.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.006+0000 7f93a1b65640 1 -- 172.21.15.5:0/3179398670 shutdown_connections 2024-05-20T18:31:27.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.006+0000 7f93a1b65640 1 --2- 172.21.15.5:0/3179398670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f939c070850 0x7f939c070c60 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:27.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.006+0000 7f93a1b65640 1 --2- 172.21.15.5:0/3179398670 >> v2:172.21.15.63:3300/0 conn(0x7f939c071630 0x7f939c15f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:27.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.006+0000 7f93a1b65640 1 -- 172.21.15.5:0/3179398670 >> 172.21.15.5:0/3179398670 conn(0x7f939c0d29d0 msgr2=0x7f939c0d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:27.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.006+0000 7f93a1b65640 1 -- 172.21.15.5:0/3179398670 shutdown_connections 2024-05-20T18:31:27.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.006+0000 7f93a1b65640 1 -- 172.21.15.5:0/3179398670 wait complete. 2024-05-20T18:31:27.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.007+0000 7f93a1b65640 1 Processor -- start 2024-05-20T18:31:27.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.008+0000 7f93a1b65640 1 -- start start 2024-05-20T18:31:27.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.009+0000 7f93a1b65640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f939c070850 0x7f939c154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:27.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.009+0000 7f93a1b65640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f939c071630 0x7f939c150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:27.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.009+0000 7f93a1b65640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f939c0cd1b0 0x7f939c1f5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:27.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.009+0000 7f93a1b65640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f939c06ded0 con 0x7f939c0cd1b0 2024-05-20T18:31:27.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.009+0000 7f93a1b65640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f939c06cfc0 con 0x7f939c070850 2024-05-20T18:31:27.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.009+0000 7f93a1b65640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f939c06d2a0 con 0x7f939c071630 2024-05-20T18:31:27.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.009+0000 7f939bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f939c0cd1b0 0x7f939c1f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:27.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.009+0000 7f939bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f939c0cd1b0 0x7f939c1f5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35314/0 (socket says 172.21.15.5:35314) 2024-05-20T18:31:27.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.009+0000 7f939bfff640 1 -- 172.21.15.5:0/2405507206 learned_addr learned my addr 172.21.15.5:0/2405507206 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:27.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.009+0000 7f939b7fe640 1 --2- 172.21.15.5:0/2405507206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f939c070850 0x7f939c154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:27.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.009+0000 7f939affd640 1 --2- 172.21.15.5:0/2405507206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f939c071630 0x7f939c150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:27.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.009+0000 7f939bfff640 1 -- 172.21.15.5:0/2405507206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f939c071630 msgr2=0x7f939c150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:27.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.009+0000 7f939bfff640 1 --2- 172.21.15.5:0/2405507206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f939c071630 0x7f939c150b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:27.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.009+0000 7f939bfff640 1 -- 172.21.15.5:0/2405507206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f939c070850 msgr2=0x7f939c154810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:27.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.009+0000 7f939bfff640 1 --2- 172.21.15.5:0/2405507206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f939c070850 0x7f939c154810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:27.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.009+0000 7f939bfff640 1 -- 172.21.15.5:0/2405507206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939c151560 con 0x7f939c0cd1b0 2024-05-20T18:31:27.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.009+0000 7f939b7fe640 1 --2- 172.21.15.5:0/2405507206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f939c070850 0x7f939c154810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:27.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.010+0000 7f939bfff640 1 --2- 172.21.15.5:0/2405507206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f939c0cd1b0 0x7f939c1f5690 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f9394035040 tx=0x7f93940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.010+0000 7f9393fff640 1 -- 172.21.15.5:0/2405507206 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9394037040 con 0x7f939c0cd1b0 2024-05-20T18:31:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.010+0000 7f9393fff640 1 -- 172.21.15.5:0/2405507206 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9394042070 con 0x7f939c0cd1b0 2024-05-20T18:31:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.010+0000 7f9393fff640 1 -- 172.21.15.5:0/2405507206 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9394030b30 con 0x7f939c0cd1b0 2024-05-20T18:31:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.010+0000 7f93a1b65640 1 -- 172.21.15.5:0/2405507206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f939c0b8cd0 con 0x7f939c0cd1b0 2024-05-20T18:31:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.010+0000 7f93a1b65640 1 -- 172.21.15.5:0/2405507206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f939c0b0120 con 0x7f939c0cd1b0 2024-05-20T18:31:27.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.011+0000 7f93a1b65640 1 -- 172.21.15.5:0/2405507206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f939c0b0380 con 0x7f939c0cd1b0 2024-05-20T18:31:27.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.011+0000 7f9393fff640 1 -- 172.21.15.5:0/2405507206 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9394030cd0 con 0x7f939c0cd1b0 2024-05-20T18:31:27.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.012+0000 7f9393fff640 1 --2- 172.21.15.5:0/2405507206 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f937c06be30 0x7f937c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:27.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.012+0000 7f939b7fe640 1 --2- 172.21.15.5:0/2405507206 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f937c06be30 0x7f937c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:27.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.012+0000 7f9393fff640 1 -- 172.21.15.5:0/2405507206 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5249+0+0 (secure 0 0 0) 0x7f93940ab840 con 0x7f939c0cd1b0 2024-05-20T18:31:27.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.013+0000 7f939b7fe640 1 --2- 172.21.15.5:0/2405507206 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f937c06be30 0x7f937c06e340 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f938c009f50 tx=0x7f938c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:27.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.017+0000 7f9393fff640 1 -- 172.21.15.5:0/2405507206 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9394032e20 con 0x7f939c0cd1b0 2024-05-20T18:31:27.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.200+0000 7f93a1b65640 1 -- 172.21.15.5:0/2405507206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f939c075b00 con 0x7f939c0cd1b0 2024-05-20T18:31:27.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.200+0000 7f9393fff640 1 -- 172.21.15.5:0/2405507206 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7f9394045360 con 0x7f939c0cd1b0 2024-05-20T18:31:27.201 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:31:27.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.202+0000 7f93a1b65640 1 -- 172.21.15.5:0/2405507206 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f937c06be30 msgr2=0x7f937c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:27.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.202+0000 7f93a1b65640 1 --2- 172.21.15.5:0/2405507206 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f937c06be30 0x7f937c06e340 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f938c009f50 tx=0x7f938c00b040 comp rx=0 tx=0).stop 2024-05-20T18:31:27.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.202+0000 7f93a1b65640 1 -- 172.21.15.5:0/2405507206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f939c0cd1b0 msgr2=0x7f939c1f5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:27.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.202+0000 7f93a1b65640 1 --2- 172.21.15.5:0/2405507206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f939c0cd1b0 0x7f939c1f5690 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f9394035040 tx=0x7f93940046c0 comp rx=0 tx=0).stop 2024-05-20T18:31:27.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.202+0000 7f93a1b65640 1 -- 172.21.15.5:0/2405507206 shutdown_connections 2024-05-20T18:31:27.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.202+0000 7f93a1b65640 1 --2- 172.21.15.5:0/2405507206 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f937c06be30 0x7f937c06e340 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:27.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.202+0000 7f93a1b65640 1 --2- 172.21.15.5:0/2405507206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f939c0cd1b0 0x7f939c1f5690 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:27.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.202+0000 7f93a1b65640 1 --2- 172.21.15.5:0/2405507206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f939c071630 0x7f939c150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:27.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.202+0000 7f93a1b65640 1 --2- 172.21.15.5:0/2405507206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f939c070850 0x7f939c154810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:27.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.202+0000 7f93a1b65640 1 -- 172.21.15.5:0/2405507206 >> 172.21.15.5:0/2405507206 conn(0x7f939c0d29d0 msgr2=0x7f939c15e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:27.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.202+0000 7f93a1b65640 1 -- 172.21.15.5:0/2405507206 shutdown_connections 2024-05-20T18:31:27.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.202+0000 7f93a1b65640 1 -- 172.21.15.5:0/2405507206 wait complete. 2024-05-20T18:31:27.214 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-20T18:31:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.329+0000 7f7757ee9640 1 Processor -- start 2024-05-20T18:31:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.329+0000 7f7757ee9640 1 -- start start 2024-05-20T18:31:27.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.330+0000 7f7757ee9640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7750162820 0x7f77500b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:27.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.330+0000 7f7757ee9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f775006dae0 0x7f77500b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:27.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.330+0000 7f7757ee9640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77500d0630 con 0x7f77500b3e80 2024-05-20T18:31:27.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.330+0000 7f7757ee9640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f77500d39c0 con 0x7f775006dae0 2024-05-20T18:31:27.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.330+0000 7f7757ee9640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f77500d3ca0 con 0x7f7750162820 2024-05-20T18:31:27.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.330+0000 7f7756486640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f775006dae0 0x7f77500b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:27.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.330+0000 7f7756486640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f775006dae0 0x7f77500b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35320/0 (socket says 172.21.15.5:35320) 2024-05-20T18:31:27.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.330+0000 7f7756486640 1 -- 172.21.15.5:0/405302827 learned_addr learned my addr 172.21.15.5:0/405302827 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:27.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.331+0000 7f7755c85640 1 --2- 172.21.15.5:0/405302827 >> v2:172.21.15.63:3300/0 conn(0x7f7750162820 0x7f77500b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:27.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.331+0000 7f7756486640 1 -- 172.21.15.5:0/405302827 >> v2:172.21.15.63:3300/0 conn(0x7f7750162820 msgr2=0x7f77500b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:27.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.331+0000 7f7756486640 1 --2- 172.21.15.5:0/405302827 >> v2:172.21.15.63:3300/0 conn(0x7f7750162820 0x7f77500b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:27.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.331+0000 7f7756486640 1 -- 172.21.15.5:0/405302827 >> v1:172.21.15.63:6789/0 conn(0x7f77500b3e80 legacy=0x7f775006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:27.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.331+0000 7f7756486640 1 -- 172.21.15.5:0/405302827 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77500d3f80 con 0x7f775006dae0 2024-05-20T18:31:27.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.331+0000 7f7756486640 1 --2- 172.21.15.5:0/405302827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f775006dae0 0x7f77500b8190 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f77500b7740 tx=0x7f774802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0c275653a70e1d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:27.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.332+0000 7f7754c83640 1 -- 172.21.15.5:0/405302827 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7748036070 con 0x7f775006dae0 2024-05-20T18:31:27.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.332+0000 7f7754c83640 1 -- 172.21.15.5:0/405302827 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7748031600 con 0x7f775006dae0 2024-05-20T18:31:27.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.332+0000 7f7754c83640 1 -- 172.21.15.5:0/405302827 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77480319d0 con 0x7f775006dae0 2024-05-20T18:31:27.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.332+0000 7f7757ee9640 1 -- 172.21.15.5:0/405302827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f775006dae0 msgr2=0x7f77500b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:27.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.332+0000 7f7757ee9640 1 --2- 172.21.15.5:0/405302827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f775006dae0 0x7f77500b8190 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f77500b7740 tx=0x7f774802a6f0 comp rx=0 tx=0).stop 2024-05-20T18:31:27.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.332+0000 7f7757ee9640 1 -- 172.21.15.5:0/405302827 shutdown_connections 2024-05-20T18:31:27.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.332+0000 7f7757ee9640 1 --2- 172.21.15.5:0/405302827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f775006dae0 0x7f77500b8190 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:27.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.332+0000 7f7757ee9640 1 --2- 172.21.15.5:0/405302827 >> v2:172.21.15.63:3300/0 conn(0x7f7750162820 0x7f77500b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:27.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.332+0000 7f7757ee9640 1 -- 172.21.15.5:0/405302827 >> 172.21.15.5:0/405302827 conn(0x7f77500d4bd0 msgr2=0x7f77500d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:27.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.333+0000 7f7757ee9640 1 -- 172.21.15.5:0/405302827 shutdown_connections 2024-05-20T18:31:27.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.333+0000 7f7757ee9640 1 -- 172.21.15.5:0/405302827 wait complete. 2024-05-20T18:31:27.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.334+0000 7f7757ee9640 1 Processor -- start 2024-05-20T18:31:27.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.334+0000 7f7757ee9640 1 -- start start 2024-05-20T18:31:27.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.336+0000 7f7757ee9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77500b3e80 0x7f7750156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:27.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.336+0000 7f7757ee9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7750162820 0x7f77500b95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:27.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.336+0000 7f7757ee9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7750144f00 0x7f7750159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:27.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.336+0000 7f7757ee9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f77500d4780 con 0x7f7750144f00 2024-05-20T18:31:27.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.336+0000 7f7757ee9640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f77500d0300 con 0x7f77500b3e80 2024-05-20T18:31:27.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.336+0000 7f7757ee9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f77500d4900 con 0x7f7750162820 2024-05-20T18:31:27.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.336+0000 7f7756486640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7750144f00 0x7f7750159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:27.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.336+0000 7f7756486640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7750144f00 0x7f7750159650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35326/0 (socket says 172.21.15.5:35326) 2024-05-20T18:31:27.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.336+0000 7f7756486640 1 -- 172.21.15.5:0/1295830940 learned_addr learned my addr 172.21.15.5:0/1295830940 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:27.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.336+0000 7f7755c85640 1 --2- 172.21.15.5:0/1295830940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77500b3e80 0x7f7750156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:27.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.336+0000 7f7755484640 1 --2- 172.21.15.5:0/1295830940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7750162820 0x7f77500b95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:27.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.336+0000 7f7756486640 1 -- 172.21.15.5:0/1295830940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7750162820 msgr2=0x7f77500b95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:27.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.336+0000 7f7756486640 1 --2- 172.21.15.5:0/1295830940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7750162820 0x7f77500b95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:27.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.336+0000 7f7756486640 1 -- 172.21.15.5:0/1295830940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77500b3e80 msgr2=0x7f7750156c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:27.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.336+0000 7f7756486640 1 --2- 172.21.15.5:0/1295830940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77500b3e80 0x7f7750156c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:27.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.336+0000 7f7756486640 1 -- 172.21.15.5:0/1295830940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77500b9af0 con 0x7f7750144f00 2024-05-20T18:31:27.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.336+0000 7f7756486640 1 --2- 172.21.15.5:0/1295830940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7750144f00 0x7f7750159650 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f7748009f40 tx=0x7f7748004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:27.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.336+0000 7f77467fc640 1 -- 172.21.15.5:0/1295830940 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7748036050 con 0x7f7750144f00 2024-05-20T18:31:27.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.337+0000 7f77467fc640 1 -- 172.21.15.5:0/1295830940 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7748042070 con 0x7f7750144f00 2024-05-20T18:31:27.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.337+0000 7f7757ee9640 1 -- 172.21.15.5:0/1295830940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77501585f0 con 0x7f7750144f00 2024-05-20T18:31:27.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.337+0000 7f77467fc640 1 -- 172.21.15.5:0/1295830940 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7748009510 con 0x7f7750144f00 2024-05-20T18:31:27.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.337+0000 7f7757ee9640 1 -- 172.21.15.5:0/1295830940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7750158b70 con 0x7f7750144f00 2024-05-20T18:31:27.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.337+0000 7f7757ee9640 1 -- 172.21.15.5:0/1295830940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7750072fc0 con 0x7f7750144f00 2024-05-20T18:31:27.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.338+0000 7f77467fc640 1 -- 172.21.15.5:0/1295830940 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7748007b90 con 0x7f7750144f00 2024-05-20T18:31:27.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.339+0000 7f77467fc640 1 --2- 172.21.15.5:0/1295830940 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f773406be30 0x7f773406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:27.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.339+0000 7f77467fc640 1 -- 172.21.15.5:0/1295830940 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5249+0+0 (secure 0 0 0) 0x7f77480abaf0 con 0x7f7750144f00 2024-05-20T18:31:27.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.339+0000 7f7755c85640 1 --2- 172.21.15.5:0/1295830940 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f773406be30 0x7f773406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:27.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.339+0000 7f7755c85640 1 --2- 172.21.15.5:0/1295830940 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f773406be30 0x7f773406e340 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f774c0079a0 tx=0x7f774c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:27.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.343+0000 7f77467fc640 1 -- 172.21.15.5:0/1295830940 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7750072fc0 con 0x7f7750144f00 2024-05-20T18:31:27.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.532+0000 7f7757ee9640 1 -- 172.21.15.5:0/1295830940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7750075b00 con 0x7f7750144f00 2024-05-20T18:31:27.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.532+0000 7f77467fc640 1 -- 172.21.15.5:0/1295830940 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v73) ==== 84+0+65 (secure 0 0 0) 0x7f77500b9af0 con 0x7f7750144f00 2024-05-20T18:31:27.533 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:31:27.533 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-05-20T18:31:27.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:31:27.533 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-05-20T18:31:27.533 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-05-20T18:31:27.533 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:31:27.533 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-05-20T18:31:27.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.534+0000 7f7757ee9640 1 -- 172.21.15.5:0/1295830940 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f773406be30 msgr2=0x7f773406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:27.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.534+0000 7f7757ee9640 1 --2- 172.21.15.5:0/1295830940 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f773406be30 0x7f773406e340 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f774c0079a0 tx=0x7f774c008040 comp rx=0 tx=0).stop 2024-05-20T18:31:27.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.534+0000 7f7757ee9640 1 -- 172.21.15.5:0/1295830940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7750144f00 msgr2=0x7f7750159650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:27.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.534+0000 7f7757ee9640 1 --2- 172.21.15.5:0/1295830940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7750144f00 0x7f7750159650 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f7748009f40 tx=0x7f7748004a40 comp rx=0 tx=0).stop 2024-05-20T18:31:27.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.534+0000 7f7757ee9640 1 -- 172.21.15.5:0/1295830940 shutdown_connections 2024-05-20T18:31:27.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.534+0000 7f7757ee9640 1 --2- 172.21.15.5:0/1295830940 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f773406be30 0x7f773406e340 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:27.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.534+0000 7f7757ee9640 1 --2- 172.21.15.5:0/1295830940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7750144f00 0x7f7750159650 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:27.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.534+0000 7f7757ee9640 1 --2- 172.21.15.5:0/1295830940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7750162820 0x7f77500b95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:27.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.534+0000 7f7757ee9640 1 --2- 172.21.15.5:0/1295830940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77500b3e80 0x7f7750156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:27.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.534+0000 7f7757ee9640 1 -- 172.21.15.5:0/1295830940 >> 172.21.15.5:0/1295830940 conn(0x7f77500d4bd0 msgr2=0x7f77501e7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:27.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.534+0000 7f7757ee9640 1 -- 172.21.15.5:0/1295830940 shutdown_connections 2024-05-20T18:31:27.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.534+0000 7f7757ee9640 1 -- 172.21.15.5:0/1295830940 wait complete. 2024-05-20T18:31:27.546 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-20T18:31:27.546 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-20T18:31:27.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.665+0000 7f883d20d640 1 Processor -- start 2024-05-20T18:31:27.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.665+0000 7f883d20d640 1 -- start start 2024-05-20T18:31:27.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.666+0000 7f883d20d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f88380b3860 0x7f88380d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:27.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.666+0000 7f883d20d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f883808bd20 0x7f883808c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:27.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.666+0000 7f883d20d640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f883808c670 con 0x7f883806d870 2024-05-20T18:31:27.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.666+0000 7f883d20d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f88380d0450 con 0x7f88380b3860 2024-05-20T18:31:27.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.666+0000 7f883d20d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f88380d3b10 con 0x7f883808bd20 2024-05-20T18:31:27.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.667+0000 7f8836d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f88380b3860 0x7f88380d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:27.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.667+0000 7f8836d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f88380b3860 0x7f88380d45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35334/0 (socket says 172.21.15.5:35334) 2024-05-20T18:31:27.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.667+0000 7f8836d9d640 1 -- 172.21.15.5:0/722992365 learned_addr learned my addr 172.21.15.5:0/722992365 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:27.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.667+0000 7f8836d9d640 1 -- 172.21.15.5:0/722992365 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f883808bd20 msgr2=0x7f883808c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:27.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.667+0000 7f8836d9d640 1 --2- 172.21.15.5:0/722992365 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f883808bd20 0x7f883808c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:27.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.667+0000 7f8836d9d640 1 -- 172.21.15.5:0/722992365 >> v1:172.21.15.63:6789/0 conn(0x7f883806d870 legacy=0x7f883815f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:27.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.667+0000 7f8836d9d640 1 -- 172.21.15.5:0/722992365 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88380d3df0 con 0x7f88380b3860 2024-05-20T18:31:27.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.668+0000 7f8836d9d640 1 --2- 172.21.15.5:0/722992365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88380b3860 0x7f88380d45f0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f8828009870 tx=0x7f882802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4adb22fbc6402294 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:27.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.668+0000 7f8835d9b640 1 -- 172.21.15.5:0/722992365 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828037070 con 0x7f88380b3860 2024-05-20T18:31:27.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.668+0000 7f8835d9b640 1 -- 172.21.15.5:0/722992365 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f882802ae10 con 0x7f88380b3860 2024-05-20T18:31:27.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.668+0000 7f8835d9b640 1 -- 172.21.15.5:0/722992365 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828032660 con 0x7f88380b3860 2024-05-20T18:31:27.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.668+0000 7f883d20d640 1 -- 172.21.15.5:0/722992365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88380b3860 msgr2=0x7f88380d45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:27.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.668+0000 7f883d20d640 1 --2- 172.21.15.5:0/722992365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88380b3860 0x7f88380d45f0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f8828009870 tx=0x7f882802a100 comp rx=0 tx=0).stop 2024-05-20T18:31:27.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.669+0000 7f883d20d640 1 -- 172.21.15.5:0/722992365 shutdown_connections 2024-05-20T18:31:27.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.669+0000 7f883d20d640 1 --2- 172.21.15.5:0/722992365 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f883808bd20 0x7f883808c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:27.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.669+0000 7f883d20d640 1 --2- 172.21.15.5:0/722992365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88380b3860 0x7f88380d45f0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:27.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.669+0000 7f883d20d640 1 -- 172.21.15.5:0/722992365 >> 172.21.15.5:0/722992365 conn(0x7f88380b6d10 msgr2=0x7f883806d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:27.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.669+0000 7f883d20d640 1 -- 172.21.15.5:0/722992365 shutdown_connections 2024-05-20T18:31:27.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.669+0000 7f883d20d640 1 -- 172.21.15.5:0/722992365 wait complete. 2024-05-20T18:31:27.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.671+0000 7f883d20d640 1 Processor -- start 2024-05-20T18:31:27.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.671+0000 7f883d20d640 1 -- start start 2024-05-20T18:31:27.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.672+0000 7f883d20d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f883806d870 0x7f883815c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:27.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.672+0000 7f883d20d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883808bd20 0x7f8838153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:27.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.672+0000 7f883d20d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88380b3860 0x7f883815db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:27.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.672+0000 7f883d20d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f88380b81b0 con 0x7f883808bd20 2024-05-20T18:31:27.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.672+0000 7f883d20d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f88380d0730 con 0x7f88380b3860 2024-05-20T18:31:27.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.672+0000 7f883d20d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f88380b8330 con 0x7f883806d870 2024-05-20T18:31:27.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.672+0000 7f883659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883808bd20 0x7f8838153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:27.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.672+0000 7f883659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883808bd20 0x7f8838153970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35344/0 (socket says 172.21.15.5:35344) 2024-05-20T18:31:27.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.672+0000 7f883659c640 1 -- 172.21.15.5:0/2935056383 learned_addr learned my addr 172.21.15.5:0/2935056383 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:27.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.672+0000 7f8836d9d640 1 --2- 172.21.15.5:0/2935056383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f883806d870 0x7f883815c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:27.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.672+0000 7f883759e640 1 --2- 172.21.15.5:0/2935056383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88380b3860 0x7f883815db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:27.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.673+0000 7f883659c640 1 -- 172.21.15.5:0/2935056383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f883806d870 msgr2=0x7f883815c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:27.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.673+0000 7f883659c640 1 --2- 172.21.15.5:0/2935056383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f883806d870 0x7f883815c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:27.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.673+0000 7f883659c640 1 -- 172.21.15.5:0/2935056383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88380b3860 msgr2=0x7f883815db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:27.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.673+0000 7f883659c640 1 --2- 172.21.15.5:0/2935056383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88380b3860 0x7f883815db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:27.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.673+0000 7f883659c640 1 -- 172.21.15.5:0/2935056383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8838153eb0 con 0x7f883808bd20 2024-05-20T18:31:27.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.673+0000 7f883659c640 1 --2- 172.21.15.5:0/2935056383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883808bd20 0x7f8838153970 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f882c00b3f0 tx=0x7f882c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:27.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.673+0000 7f881f7fe640 1 -- 172.21.15.5:0/2935056383 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c015020 con 0x7f883808bd20 2024-05-20T18:31:27.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.673+0000 7f881f7fe640 1 -- 172.21.15.5:0/2935056383 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f882c011830 con 0x7f883808bd20 2024-05-20T18:31:27.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.673+0000 7f881f7fe640 1 -- 172.21.15.5:0/2935056383 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c011c00 con 0x7f883808bd20 2024-05-20T18:31:27.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.673+0000 7f883d20d640 1 -- 172.21.15.5:0/2935056383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88381e5b30 con 0x7f883808bd20 2024-05-20T18:31:27.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.673+0000 7f883d20d640 1 -- 172.21.15.5:0/2935056383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88381e60f0 con 0x7f883808bd20 2024-05-20T18:31:27.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.674+0000 7f883d20d640 1 -- 172.21.15.5:0/2935056383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f883815eb10 con 0x7f883808bd20 2024-05-20T18:31:27.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.675+0000 7f881f7fe640 1 -- 172.21.15.5:0/2935056383 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f882c018020 con 0x7f883808bd20 2024-05-20T18:31:27.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.675+0000 7f881f7fe640 1 --2- 172.21.15.5:0/2935056383 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f882406be60 0x7f882406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:27.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.675+0000 7f881f7fe640 1 -- 172.21.15.5:0/2935056383 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5249+0+0 (secure 0 0 0) 0x7f882c08a640 con 0x7f883808bd20 2024-05-20T18:31:27.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.675+0000 7f8836d9d640 1 --2- 172.21.15.5:0/2935056383 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f882406be60 0x7f882406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:27.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.676+0000 7f8836d9d640 1 --2- 172.21.15.5:0/2935056383 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f882406be60 0x7f882406e370 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8828034000 tx=0x7f882802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:27.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.680+0000 7f881f7fe640 1 -- 172.21.15.5:0/2935056383 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f883815eb10 con 0x7f883808bd20 2024-05-20T18:31:27.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:27.864+0000 7f883d20d640 1 -- 172.21.15.5:0/2935056383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f88380690b0 con 0x7f883808bd20 2024-05-20T18:31:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.763+0000 7f881f7fe640 1 -- 172.21.15.5:0/2935056383 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v74) ==== 138+0+0 (secure 0 0 0) 0x7f882c01f070 con 0x7f883808bd20 2024-05-20T18:31:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-05-20T18:31:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.767+0000 7f883d20d640 1 -- 172.21.15.5:0/2935056383 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f882406be60 msgr2=0x7f882406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.767+0000 7f883d20d640 1 --2- 172.21.15.5:0/2935056383 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f882406be60 0x7f882406e370 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8828034000 tx=0x7f882802e000 comp rx=0 tx=0).stop 2024-05-20T18:31:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.767+0000 7f883d20d640 1 -- 172.21.15.5:0/2935056383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883808bd20 msgr2=0x7f8838153970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.767+0000 7f883d20d640 1 --2- 172.21.15.5:0/2935056383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883808bd20 0x7f8838153970 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f882c00b3f0 tx=0x7f882c00b8b0 comp rx=0 tx=0).stop 2024-05-20T18:31:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.767+0000 7f883d20d640 1 -- 172.21.15.5:0/2935056383 shutdown_connections 2024-05-20T18:31:28.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.767+0000 7f883d20d640 1 --2- 172.21.15.5:0/2935056383 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f882406be60 0x7f882406e370 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:28.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.767+0000 7f883d20d640 1 --2- 172.21.15.5:0/2935056383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88380b3860 0x7f883815db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:28.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.767+0000 7f883d20d640 1 --2- 172.21.15.5:0/2935056383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883808bd20 0x7f8838153970 secure :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f882c00b3f0 tx=0x7f882c00b8b0 comp rx=0 tx=0).stop 2024-05-20T18:31:28.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.767+0000 7f883d20d640 1 --2- 172.21.15.5:0/2935056383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f883806d870 0x7f883815c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:28.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.767+0000 7f883d20d640 1 -- 172.21.15.5:0/2935056383 >> 172.21.15.5:0/2935056383 conn(0x7f88380b6d10 msgr2=0x7f88380b5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:28.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.767+0000 7f883d20d640 1 -- 172.21.15.5:0/2935056383 shutdown_connections 2024-05-20T18:31:28.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.768+0000 7f883d20d640 1 -- 172.21.15.5:0/2935056383 wait complete. 2024-05-20T18:31:28.786 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-20T18:31:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.914+0000 7fdcf661b640 1 Processor -- start 2024-05-20T18:31:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.914+0000 7fdcf661b640 1 -- start start 2024-05-20T18:31:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.915+0000 7fdcf661b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fdcf009b280 0x7fdcf009be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.915+0000 7fdcf661b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcf00b3ff0 0x7fdcf00b6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.915+0000 7fdcf661b640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcf00ce950 con 0x7fdcf00d0ac0 2024-05-20T18:31:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.915+0000 7fdcf661b640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fdcf00ce720 con 0x7fdcf009b280 2024-05-20T18:31:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.915+0000 7fdcf661b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdcf00d0fc0 con 0x7fdcf00b3ff0 2024-05-20T18:31:28.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.916+0000 7fdcf4bb8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcf00b3ff0 0x7fdcf00b6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:28.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.916+0000 7fdcef7fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fdcf009b280 0x7fdcf009be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:28.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.916+0000 7fdcf4bb8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcf00b3ff0 0x7fdcf00b6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:37630/0 (socket says 172.21.15.5:37630) 2024-05-20T18:31:28.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.916+0000 7fdcf4bb8640 1 -- 172.21.15.5:0/4257783527 learned_addr learned my addr 172.21.15.5:0/4257783527 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:28.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.916+0000 7fdcef7fe640 1 -- 172.21.15.5:0/4257783527 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcf00b3ff0 msgr2=0x7fdcf00b6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:28.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.916+0000 7fdcef7fe640 1 --2- 172.21.15.5:0/4257783527 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcf00b3ff0 0x7fdcf00b6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:28.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.916+0000 7fdcef7fe640 1 -- 172.21.15.5:0/4257783527 >> v1:172.21.15.63:6789/0 conn(0x7fdcf00d0ac0 legacy=0x7fdcf009ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:28.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.916+0000 7fdcef7fe640 1 -- 172.21.15.5:0/4257783527 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf00cec30 con 0x7fdcf009b280 2024-05-20T18:31:28.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.916+0000 7fdcf4bb8640 1 --2- 172.21.15.5:0/4257783527 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcf00b3ff0 0x7fdcf00b6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:31:28.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.917+0000 7fdcef7fe640 1 --2- 172.21.15.5:0/4257783527 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcf009b280 0x7fdcf009be70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdce40097a0 tx=0x7fdce402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2af4bb97a6f284f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:28.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.917+0000 7fdceeffd640 1 -- 172.21.15.5:0/4257783527 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce4037070 con 0x7fdcf009b280 2024-05-20T18:31:28.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.917+0000 7fdceeffd640 1 -- 172.21.15.5:0/4257783527 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdce4032440 con 0x7fdcf009b280 2024-05-20T18:31:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.917+0000 7fdceeffd640 1 -- 172.21.15.5:0/4257783527 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce4032810 con 0x7fdcf009b280 2024-05-20T18:31:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.918+0000 7fdcf661b640 1 -- 172.21.15.5:0/4257783527 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcf009b280 msgr2=0x7fdcf009be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.918+0000 7fdcf661b640 1 --2- 172.21.15.5:0/4257783527 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcf009b280 0x7fdcf009be70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdce40097a0 tx=0x7fdce402a180 comp rx=0 tx=0).stop 2024-05-20T18:31:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.918+0000 7fdcf661b640 1 -- 172.21.15.5:0/4257783527 shutdown_connections 2024-05-20T18:31:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.918+0000 7fdcf661b640 1 --2- 172.21.15.5:0/4257783527 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcf00b3ff0 0x7fdcf00b6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.918+0000 7fdcf661b640 1 --2- 172.21.15.5:0/4257783527 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcf009b280 0x7fdcf009be70 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.918+0000 7fdcf661b640 1 -- 172.21.15.5:0/4257783527 >> 172.21.15.5:0/4257783527 conn(0x7fdcf00b44b0 msgr2=0x7fdcf00b48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.918+0000 7fdcf661b640 1 -- 172.21.15.5:0/4257783527 shutdown_connections 2024-05-20T18:31:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.918+0000 7fdcf661b640 1 -- 172.21.15.5:0/4257783527 wait complete. 2024-05-20T18:31:28.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.920+0000 7fdcf661b640 1 Processor -- start 2024-05-20T18:31:28.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.920+0000 7fdcf661b640 1 -- start start 2024-05-20T18:31:28.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.921+0000 7fdcf661b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf00b3ff0 0x7fdcf00c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.921+0000 7fdcf661b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcf00d0ac0 0x7fdcf00ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.921+0000 7fdcf661b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcf014f6a0 0x7fdcf00cb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.921+0000 7fdcf661b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf00b8230 con 0x7fdcf00b3ff0 2024-05-20T18:31:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.921+0000 7fdcf661b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf00b5b80 con 0x7fdcf00d0ac0 2024-05-20T18:31:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.921+0000 7fdcf661b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdcf00c7410 con 0x7fdcf014f6a0 2024-05-20T18:31:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.921+0000 7fdceffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf00b3ff0 0x7fdcf00c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.921+0000 7fdceffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf00b3ff0 0x7fdcf00c9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35354/0 (socket says 172.21.15.5:35354) 2024-05-20T18:31:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.921+0000 7fdceffff640 1 -- 172.21.15.5:0/3100711735 learned_addr learned my addr 172.21.15.5:0/3100711735 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.921+0000 7fdcf4bb8640 1 --2- 172.21.15.5:0/3100711735 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcf014f6a0 0x7fdcf00cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.921+0000 7fdcef7fe640 1 --2- 172.21.15.5:0/3100711735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcf00d0ac0 0x7fdcf00ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.922+0000 7fdceffff640 1 -- 172.21.15.5:0/3100711735 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcf014f6a0 msgr2=0x7fdcf00cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.922+0000 7fdceffff640 1 --2- 172.21.15.5:0/3100711735 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcf014f6a0 0x7fdcf00cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.922+0000 7fdceffff640 1 -- 172.21.15.5:0/3100711735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcf00d0ac0 msgr2=0x7fdcf00ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.922+0000 7fdceffff640 1 --2- 172.21.15.5:0/3100711735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcf00d0ac0 0x7fdcf00ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.922+0000 7fdceffff640 1 -- 172.21.15.5:0/3100711735 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf014e8e0 con 0x7fdcf00b3ff0 2024-05-20T18:31:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.922+0000 7fdcf4bb8640 1 --2- 172.21.15.5:0/3100711735 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcf014f6a0 0x7fdcf00cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.922+0000 7fdcef7fe640 1 --2- 172.21.15.5:0/3100711735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcf00d0ac0 0x7fdcf00ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.922+0000 7fdceffff640 1 --2- 172.21.15.5:0/3100711735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf00b3ff0 0x7fdcf00c9da0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fdce000c3c0 tx=0x7fdce000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.922+0000 7fdcecff9640 1 -- 172.21.15.5:0/3100711735 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce0017070 con 0x7fdcf00b3ff0 2024-05-20T18:31:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.922+0000 7fdcecff9640 1 -- 172.21.15.5:0/3100711735 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdce00052f0 con 0x7fdcf00b3ff0 2024-05-20T18:31:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.922+0000 7fdcf661b640 1 -- 172.21.15.5:0/3100711735 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf014ebc0 con 0x7fdcf00b3ff0 2024-05-20T18:31:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.922+0000 7fdcecff9640 1 -- 172.21.15.5:0/3100711735 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce0003850 con 0x7fdcf00b3ff0 2024-05-20T18:31:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.922+0000 7fdcf661b640 1 -- 172.21.15.5:0/3100711735 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcf014f180 con 0x7fdcf00b3ff0 2024-05-20T18:31:28.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.923+0000 7fdcf661b640 1 -- 172.21.15.5:0/3100711735 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcf00690b0 con 0x7fdcf00b3ff0 2024-05-20T18:31:28.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.924+0000 7fdcecff9640 1 -- 172.21.15.5:0/3100711735 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdce001a070 con 0x7fdcf00b3ff0 2024-05-20T18:31:28.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.925+0000 7fdcecff9640 1 --2- 172.21.15.5:0/3100711735 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdcdc06be60 0x7fdcdc06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:28.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.925+0000 7fdcecff9640 1 -- 172.21.15.5:0/3100711735 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5619+0+0 (secure 0 0 0) 0x7fdce008a080 con 0x7fdcf00b3ff0 2024-05-20T18:31:28.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.925+0000 7fdcef7fe640 1 --2- 172.21.15.5:0/3100711735 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdcdc06be60 0x7fdcdc06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:28.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.925+0000 7fdcef7fe640 1 --2- 172.21.15.5:0/3100711735 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdcdc06be60 0x7fdcdc06e370 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdce4034000 tx=0x7fdce402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:28.930+0000 7fdcecff9640 1 -- 172.21.15.5:0/3100711735 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fdce0058a80 con 0x7fdcf00b3ff0 2024-05-20T18:31:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.114+0000 7fdcf661b640 1 -- 172.21.15.5:0/3100711735 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fdcf0075180 con 0x7fdcf00b3ff0 2024-05-20T18:31:29.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.765+0000 7fdcecff9640 1 -- 172.21.15.5:0/3100711735 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v75) ==== 170+0+0 (secure 0 0 0) 0x7fdce0058380 con 0x7fdcf00b3ff0 2024-05-20T18:31:29.766 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-05-20T18:31:29.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.769+0000 7fdcf661b640 1 -- 172.21.15.5:0/3100711735 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdcdc06be60 msgr2=0x7fdcdc06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:29.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.769+0000 7fdcf661b640 1 --2- 172.21.15.5:0/3100711735 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdcdc06be60 0x7fdcdc06e370 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdce4034000 tx=0x7fdce402e000 comp rx=0 tx=0).stop 2024-05-20T18:31:29.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.769+0000 7fdcf661b640 1 -- 172.21.15.5:0/3100711735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf00b3ff0 msgr2=0x7fdcf00c9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:29.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.769+0000 7fdcf661b640 1 --2- 172.21.15.5:0/3100711735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf00b3ff0 0x7fdcf00c9da0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fdce000c3c0 tx=0x7fdce000c880 comp rx=0 tx=0).stop 2024-05-20T18:31:29.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.769+0000 7fdcf661b640 1 -- 172.21.15.5:0/3100711735 shutdown_connections 2024-05-20T18:31:29.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.769+0000 7fdcf661b640 1 --2- 172.21.15.5:0/3100711735 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdcdc06be60 0x7fdcdc06e370 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:29.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.769+0000 7fdcf661b640 1 --2- 172.21.15.5:0/3100711735 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcf014f6a0 0x7fdcf00cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:29.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.769+0000 7fdcf661b640 1 --2- 172.21.15.5:0/3100711735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcf00d0ac0 0x7fdcf00ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:29.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.769+0000 7fdcf661b640 1 --2- 172.21.15.5:0/3100711735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf00b3ff0 0x7fdcf00c9da0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:29.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.769+0000 7fdcf661b640 1 -- 172.21.15.5:0/3100711735 >> 172.21.15.5:0/3100711735 conn(0x7fdcf00b44b0 msgr2=0x7fdcf015f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:29.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.769+0000 7fdcf661b640 1 -- 172.21.15.5:0/3100711735 shutdown_connections 2024-05-20T18:31:29.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.769+0000 7fdcf661b640 1 -- 172.21.15.5:0/3100711735 wait complete. 2024-05-20T18:31:29.801 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-20T18:31:29.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.924+0000 7fe5fe784640 1 Processor -- start 2024-05-20T18:31:29.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.924+0000 7fe5fe784640 1 -- start start 2024-05-20T18:31:29.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.925+0000 7fe5fe784640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5f8162820 0x7fe5f80b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:29.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.925+0000 7fe5fe784640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe5f806dae0 0x7fe5f80b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:29.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.925+0000 7fe5fe784640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5f80d0630 con 0x7fe5f80b3e80 2024-05-20T18:31:29.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.925+0000 7fe5fe784640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe5f80d39c0 con 0x7fe5f806dae0 2024-05-20T18:31:29.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.925+0000 7fe5fe784640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe5f80d3ca0 con 0x7fe5f8162820 2024-05-20T18:31:29.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.926+0000 7fe5fcd21640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe5f806dae0 0x7fe5f80b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:29.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.926+0000 7fe5fcd21640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe5f806dae0 0x7fe5f80b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35360/0 (socket says 172.21.15.5:35360) 2024-05-20T18:31:29.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.926+0000 7fe5fcd21640 1 -- 172.21.15.5:0/3441502552 learned_addr learned my addr 172.21.15.5:0/3441502552 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:29.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.926+0000 7fe5f7fff640 1 --2- 172.21.15.5:0/3441502552 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5f8162820 0x7fe5f80b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:29.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.926+0000 7fe5fcd21640 1 -- 172.21.15.5:0/3441502552 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5f8162820 msgr2=0x7fe5f80b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:29.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.926+0000 7fe5fcd21640 1 --2- 172.21.15.5:0/3441502552 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5f8162820 0x7fe5f80b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:29.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.926+0000 7fe5fcd21640 1 -- 172.21.15.5:0/3441502552 >> v1:172.21.15.63:6789/0 conn(0x7fe5f80b3e80 legacy=0x7fe5f806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:29.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.926+0000 7fe5fcd21640 1 -- 172.21.15.5:0/3441502552 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f80d3f80 con 0x7fe5f806dae0 2024-05-20T18:31:29.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.927+0000 7fe5fcd21640 1 --2- 172.21.15.5:0/3441502552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5f806dae0 0x7fe5f80b8190 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0009e50 tx=0x7fe5f002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ede19d1d3a4a51 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:29.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.927+0000 7fe5f6ffd640 1 -- 172.21.15.5:0/3441502552 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0036070 con 0x7fe5f806dae0 2024-05-20T18:31:29.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.927+0000 7fe5f6ffd640 1 -- 172.21.15.5:0/3441502552 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5f0031600 con 0x7fe5f806dae0 2024-05-20T18:31:29.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.927+0000 7fe5f6ffd640 1 -- 172.21.15.5:0/3441502552 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f00319d0 con 0x7fe5f806dae0 2024-05-20T18:31:29.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.928+0000 7fe5fe784640 1 -- 172.21.15.5:0/3441502552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5f806dae0 msgr2=0x7fe5f80b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:29.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.928+0000 7fe5fe784640 1 --2- 172.21.15.5:0/3441502552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5f806dae0 0x7fe5f80b8190 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0009e50 tx=0x7fe5f002a710 comp rx=0 tx=0).stop 2024-05-20T18:31:29.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.928+0000 7fe5fe784640 1 -- 172.21.15.5:0/3441502552 shutdown_connections 2024-05-20T18:31:29.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.928+0000 7fe5fe784640 1 --2- 172.21.15.5:0/3441502552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5f806dae0 0x7fe5f80b8190 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:29.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.928+0000 7fe5fe784640 1 --2- 172.21.15.5:0/3441502552 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5f8162820 0x7fe5f80b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:29.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.928+0000 7fe5fe784640 1 -- 172.21.15.5:0/3441502552 >> 172.21.15.5:0/3441502552 conn(0x7fe5f80d4bd0 msgr2=0x7fe5f80d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:29.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.928+0000 7fe5fe784640 1 -- 172.21.15.5:0/3441502552 shutdown_connections 2024-05-20T18:31:29.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.928+0000 7fe5fe784640 1 -- 172.21.15.5:0/3441502552 wait complete. 2024-05-20T18:31:29.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.929+0000 7fe5fe784640 1 Processor -- start 2024-05-20T18:31:29.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.930+0000 7fe5fe784640 1 -- start start 2024-05-20T18:31:29.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.931+0000 7fe5fe784640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5f80b3e80 0x7fe5f80c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:29.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.931+0000 7fe5fe784640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5f8162820 0x7fe5f80caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:29.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.931+0000 7fe5fe784640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5f80c9250 0x7fe5f8155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.931+0000 7fe5fe784640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f80d4780 con 0x7fe5f80c9250 2024-05-20T18:31:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.931+0000 7fe5fe784640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f80d0300 con 0x7fe5f80b3e80 2024-05-20T18:31:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.931+0000 7fe5fe784640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe5f80d4900 con 0x7fe5f8162820 2024-05-20T18:31:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.931+0000 7fe5fcd21640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5f80c9250 0x7fe5f8155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.931+0000 7fe5fcd21640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5f80c9250 0x7fe5f8155790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35368/0 (socket says 172.21.15.5:35368) 2024-05-20T18:31:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.931+0000 7fe5fcd21640 1 -- 172.21.15.5:0/3375254237 learned_addr learned my addr 172.21.15.5:0/3375254237 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.931+0000 7fe5f77fe640 1 --2- 172.21.15.5:0/3375254237 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5f8162820 0x7fe5f80caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.931+0000 7fe5f7fff640 1 --2- 172.21.15.5:0/3375254237 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5f80b3e80 0x7fe5f80c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.931+0000 7fe5fcd21640 1 -- 172.21.15.5:0/3375254237 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5f8162820 msgr2=0x7fe5f80caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.931+0000 7fe5fcd21640 1 --2- 172.21.15.5:0/3375254237 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5f8162820 0x7fe5f80caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.931+0000 7fe5fcd21640 1 -- 172.21.15.5:0/3375254237 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5f80b3e80 msgr2=0x7fe5f80c8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.931+0000 7fe5fcd21640 1 --2- 172.21.15.5:0/3375254237 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5f80b3e80 0x7fe5f80c8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.931+0000 7fe5fcd21640 1 -- 172.21.15.5:0/3375254237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f815ee30 con 0x7fe5f80c9250 2024-05-20T18:31:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.931+0000 7fe5f77fe640 1 --2- 172.21.15.5:0/3375254237 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5f8162820 0x7fe5f80caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.931+0000 7fe5f7fff640 1 --2- 172.21.15.5:0/3375254237 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5f80b3e80 0x7fe5f80c8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.932+0000 7fe5fcd21640 1 --2- 172.21.15.5:0/3375254237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5f80c9250 0x7fe5f8155790 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0034040 tx=0x7fe5f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.932+0000 7fe5f4ff9640 1 -- 172.21.15.5:0/3375254237 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0036050 con 0x7fe5f80c9250 2024-05-20T18:31:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.932+0000 7fe5f4ff9640 1 -- 172.21.15.5:0/3375254237 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5f0042070 con 0x7fe5f80c9250 2024-05-20T18:31:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.932+0000 7fe5fe784640 1 -- 172.21.15.5:0/3375254237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5f815f110 con 0x7fe5f80c9250 2024-05-20T18:31:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.932+0000 7fe5f4ff9640 1 -- 172.21.15.5:0/3375254237 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0009510 con 0x7fe5f80c9250 2024-05-20T18:31:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.932+0000 7fe5fe784640 1 -- 172.21.15.5:0/3375254237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5f815f6d0 con 0x7fe5f80c9250 2024-05-20T18:31:29.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.933+0000 7fe5fe784640 1 -- 172.21.15.5:0/3375254237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5f815eba0 con 0x7fe5f80c9250 2024-05-20T18:31:29.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.933+0000 7fe5f4ff9640 1 -- 172.21.15.5:0/3375254237 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe5f0007bf0 con 0x7fe5f80c9250 2024-05-20T18:31:29.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.934+0000 7fe5f4ff9640 1 --2- 172.21.15.5:0/3375254237 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe5e406be60 0x7fe5e406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:29.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.934+0000 7fe5f4ff9640 1 -- 172.21.15.5:0/3375254237 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6001+0+0 (secure 0 0 0) 0x7fe5f00abde0 con 0x7fe5f80c9250 2024-05-20T18:31:29.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.934+0000 7fe5f7fff640 1 --2- 172.21.15.5:0/3375254237 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe5e406be60 0x7fe5e406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:29.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.934+0000 7fe5f7fff640 1 --2- 172.21.15.5:0/3375254237 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe5e406be60 0x7fe5e406e370 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8009a20 tx=0x7fe5e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:29.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:29.939+0000 7fe5f4ff9640 1 -- 172.21.15.5:0/3375254237 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe5f007a660 con 0x7fe5f80c9250 2024-05-20T18:31:30.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.162+0000 7fe5fe784640 1 -- 172.21.15.5:0/3375254237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe5f80d5510 con 0x7fe5f80c9250 2024-05-20T18:31:30.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.791+0000 7fe5f4ff9640 1 -- 172.21.15.5:0/3375254237 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v40) ==== 174+0+0 (secure 0 0 0) 0x7fe5f80d5510 con 0x7fe5f80c9250 2024-05-20T18:31:30.792 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 11 and data pool 12 2024-05-20T18:31:30.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.805+0000 7fe5fe784640 1 -- 172.21.15.5:0/3375254237 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe5e406be60 msgr2=0x7fe5e406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:30.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.805+0000 7fe5fe784640 1 --2- 172.21.15.5:0/3375254237 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe5e406be60 0x7fe5e406e370 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8009a20 tx=0x7fe5e8008040 comp rx=0 tx=0).stop 2024-05-20T18:31:30.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.805+0000 7fe5fe784640 1 -- 172.21.15.5:0/3375254237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5f80c9250 msgr2=0x7fe5f8155790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:30.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.805+0000 7fe5fe784640 1 --2- 172.21.15.5:0/3375254237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5f80c9250 0x7fe5f8155790 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0034040 tx=0x7fe5f0004a40 comp rx=0 tx=0).stop 2024-05-20T18:31:30.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.806+0000 7fe5fe784640 1 -- 172.21.15.5:0/3375254237 shutdown_connections 2024-05-20T18:31:30.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.807+0000 7fe5fe784640 1 --2- 172.21.15.5:0/3375254237 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe5e406be60 0x7fe5e406e370 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:30.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.807+0000 7fe5fe784640 1 --2- 172.21.15.5:0/3375254237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5f80c9250 0x7fe5f8155790 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:30.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.807+0000 7fe5fe784640 1 --2- 172.21.15.5:0/3375254237 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5f8162820 0x7fe5f80caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:30.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.807+0000 7fe5fe784640 1 --2- 172.21.15.5:0/3375254237 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5f80b3e80 0x7fe5f80c8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:30.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.807+0000 7fe5fe784640 1 -- 172.21.15.5:0/3375254237 >> 172.21.15.5:0/3375254237 conn(0x7fe5f80d4bd0 msgr2=0x7fe5f8161120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:30.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.807+0000 7fe5fe784640 1 -- 172.21.15.5:0/3375254237 shutdown_connections 2024-05-20T18:31:30.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.807+0000 7fe5fe784640 1 -- 172.21.15.5:0/3375254237 wait complete. 2024-05-20T18:31:30.828 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:31:30.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.946+0000 7fea00fc2640 1 Processor -- start 2024-05-20T18:31:30.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.946+0000 7fea00fc2640 1 -- start start 2024-05-20T18:31:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.947+0000 7fea00fc2640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe9fc071630 0x7fe9fc15f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.947+0000 7fea00fc2640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe9fc0cd1b0 0x7fe9fc06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.947+0000 7fea00fc2640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9fc070850 0x7fe9fc070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.947+0000 7fea00fc2640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe9fc15f790 con 0x7fe9fc0cd1b0 2024-05-20T18:31:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.947+0000 7fea00fc2640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe9fc06e590 con 0x7fe9fc071630 2024-05-20T18:31:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.947+0000 7fea00fc2640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe9fc0ce590 con 0x7fe9fc070850 2024-05-20T18:31:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.947+0000 7fe9f9d9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe9fc0cd1b0 0x7fe9fc06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.947+0000 7fe9f9d9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe9fc0cd1b0 0x7fe9fc06d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35376/0 (socket says 172.21.15.5:35376) 2024-05-20T18:31:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.947+0000 7fe9f9d9b640 1 -- 172.21.15.5:0/3035045206 learned_addr learned my addr 172.21.15.5:0/3035045206 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:30.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.947+0000 7fe9fad9d640 1 --2- 172.21.15.5:0/3035045206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9fc070850 0x7fe9fc070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:30.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.947+0000 7fe9fa59c640 1 --2- 172.21.15.5:0/3035045206 >> v2:172.21.15.63:3300/0 conn(0x7fe9fc071630 0x7fe9fc15f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:30.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.948+0000 7fe9f9d9b640 1 -- 172.21.15.5:0/3035045206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9fc070850 msgr2=0x7fe9fc070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:30.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.948+0000 7fe9f9d9b640 1 --2- 172.21.15.5:0/3035045206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9fc070850 0x7fe9fc070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:30.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.948+0000 7fe9f9d9b640 1 -- 172.21.15.5:0/3035045206 >> v2:172.21.15.63:3300/0 conn(0x7fe9fc071630 msgr2=0x7fe9fc15f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:30.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.948+0000 7fe9f9d9b640 1 --2- 172.21.15.5:0/3035045206 >> v2:172.21.15.63:3300/0 conn(0x7fe9fc071630 0x7fe9fc15f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:30.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.948+0000 7fe9f9d9b640 1 -- 172.21.15.5:0/3035045206 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9fc0711a0 con 0x7fe9fc0cd1b0 2024-05-20T18:31:30.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.948+0000 7fe9fa59c640 1 --2- 172.21.15.5:0/3035045206 >> v2:172.21.15.63:3300/0 conn(0x7fe9fc071630 0x7fe9fc15f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:30.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.948+0000 7fe9f9d9b640 1 --2- 172.21.15.5:0/3035045206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9fc0cd1b0 0x7fe9fc06d7c0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc068510 tx=0x7fe9f002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a41d59337523d8eb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:30.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.948+0000 7fe9f959a640 1 -- 172.21.15.5:0/3035045206 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0037070 con 0x7fe9fc0cd1b0 2024-05-20T18:31:30.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.949+0000 7fe9f959a640 1 -- 172.21.15.5:0/3035045206 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9f0032440 con 0x7fe9fc0cd1b0 2024-05-20T18:31:30.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.949+0000 7fe9f959a640 1 -- 172.21.15.5:0/3035045206 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0032810 con 0x7fe9fc0cd1b0 2024-05-20T18:31:30.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.949+0000 7fea00fc2640 1 -- 172.21.15.5:0/3035045206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9fc0cd1b0 msgr2=0x7fe9fc06d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.949+0000 7fea00fc2640 1 --2- 172.21.15.5:0/3035045206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9fc0cd1b0 0x7fe9fc06d7c0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc068510 tx=0x7fe9f002a140 comp rx=0 tx=0).stop 2024-05-20T18:31:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.949+0000 7fea00fc2640 1 -- 172.21.15.5:0/3035045206 shutdown_connections 2024-05-20T18:31:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.949+0000 7fea00fc2640 1 --2- 172.21.15.5:0/3035045206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9fc070850 0x7fe9fc070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.949+0000 7fea00fc2640 1 --2- 172.21.15.5:0/3035045206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9fc0cd1b0 0x7fe9fc06d7c0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.949+0000 7fea00fc2640 1 --2- 172.21.15.5:0/3035045206 >> v2:172.21.15.63:3300/0 conn(0x7fe9fc071630 0x7fe9fc15f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.949+0000 7fea00fc2640 1 -- 172.21.15.5:0/3035045206 >> 172.21.15.5:0/3035045206 conn(0x7fe9fc0d29d0 msgr2=0x7fe9fc0d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.949+0000 7fea00fc2640 1 -- 172.21.15.5:0/3035045206 shutdown_connections 2024-05-20T18:31:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.949+0000 7fea00fc2640 1 -- 172.21.15.5:0/3035045206 wait complete. 2024-05-20T18:31:30.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.951+0000 7fea00fc2640 1 Processor -- start 2024-05-20T18:31:30.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.951+0000 7fea00fc2640 1 -- start start 2024-05-20T18:31:30.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.953+0000 7fea00fc2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9fc070850 0x7fe9fc158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:30.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.953+0000 7fea00fc2640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9fc071630 0x7fe9fc0b6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:30.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.953+0000 7fea00fc2640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9fc0cd1b0 0x7fe9fc0b7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:30.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.953+0000 7fea00fc2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc06cfc0 con 0x7fe9fc070850 2024-05-20T18:31:30.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.953+0000 7fea00fc2640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc06cce0 con 0x7fe9fc071630 2024-05-20T18:31:30.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.953+0000 7fea00fc2640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe9fc06d2a0 con 0x7fe9fc0cd1b0 2024-05-20T18:31:30.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.953+0000 7fe9fa59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9fc070850 0x7fe9fc158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:30.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.953+0000 7fe9fa59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9fc070850 0x7fe9fc158fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35390/0 (socket says 172.21.15.5:35390) 2024-05-20T18:31:30.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.953+0000 7fe9fa59c640 1 -- 172.21.15.5:0/525940970 learned_addr learned my addr 172.21.15.5:0/525940970 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:30.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.953+0000 7fe9fad9d640 1 --2- 172.21.15.5:0/525940970 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9fc0cd1b0 0x7fe9fc0b7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:30.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.953+0000 7fe9f9d9b640 1 --2- 172.21.15.5:0/525940970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9fc071630 0x7fe9fc0b6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:30.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.953+0000 7fe9fa59c640 1 -- 172.21.15.5:0/525940970 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9fc0cd1b0 msgr2=0x7fe9fc0b7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:30.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.953+0000 7fe9fa59c640 1 --2- 172.21.15.5:0/525940970 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9fc0cd1b0 0x7fe9fc0b7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:30.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.953+0000 7fe9fa59c640 1 -- 172.21.15.5:0/525940970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9fc071630 msgr2=0x7fe9fc0b6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:30.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.953+0000 7fe9fa59c640 1 --2- 172.21.15.5:0/525940970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9fc071630 0x7fe9fc0b6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:30.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.953+0000 7fe9fa59c640 1 -- 172.21.15.5:0/525940970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9fc0b6570 con 0x7fe9fc070850 2024-05-20T18:31:30.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.953+0000 7fe9fa59c640 1 --2- 172.21.15.5:0/525940970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9fc070850 0x7fe9fc158fd0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec009fd0 tx=0x7fe9ec00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:30.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.953+0000 7fe9eaffd640 1 -- 172.21.15.5:0/525940970 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec01a070 con 0x7fe9fc070850 2024-05-20T18:31:30.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.954+0000 7fe9eaffd640 1 -- 172.21.15.5:0/525940970 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9ec003bb0 con 0x7fe9fc070850 2024-05-20T18:31:30.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.954+0000 7fea00fc2640 1 -- 172.21.15.5:0/525940970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9fc160b70 con 0x7fe9fc070850 2024-05-20T18:31:30.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.954+0000 7fe9eaffd640 1 -- 172.21.15.5:0/525940970 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec004030 con 0x7fe9fc070850 2024-05-20T18:31:30.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.954+0000 7fea00fc2640 1 -- 172.21.15.5:0/525940970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9fc161130 con 0x7fe9fc070850 2024-05-20T18:31:30.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.955+0000 7fea00fc2640 1 -- 172.21.15.5:0/525940970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9fc075b00 con 0x7fe9fc070850 2024-05-20T18:31:30.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.955+0000 7fe9eaffd640 1 -- 172.21.15.5:0/525940970 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe9ec01d070 con 0x7fe9fc070850 2024-05-20T18:31:30.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.956+0000 7fe9eaffd640 1 --2- 172.21.15.5:0/525940970 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe9dc06be60 0x7fe9dc06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:30.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.956+0000 7fe9eaffd640 1 -- 172.21.15.5:0/525940970 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6101+0+0 (secure 0 0 0) 0x7fe9ec08b290 con 0x7fe9fc070850 2024-05-20T18:31:30.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.956+0000 7fe9f9d9b640 1 --2- 172.21.15.5:0/525940970 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe9dc06be60 0x7fe9dc06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:30.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.956+0000 7fe9f9d9b640 1 --2- 172.21.15.5:0/525940970 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe9dc06be60 0x7fe9dc06e370 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0034000 tx=0x7fe9f002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:30.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:30.961+0000 7fe9eaffd640 1 -- 172.21.15.5:0/525940970 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe9ec010070 con 0x7fe9fc070850 2024-05-20T18:31:31.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.145+0000 7fea00fc2640 1 -- 172.21.15.5:0/525940970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9fc071410 con 0x7fe9fc070850 2024-05-20T18:31:31.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.146+0000 7fe9eaffd640 1 -- 172.21.15.5:0/525940970 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+16838 (secure 0 0 0) 0x7fe9ec0592b0 con 0x7fe9fc070850 2024-05-20T18:31:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:31:31.147 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":76,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:31:30.163830+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:31:27.896386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-20T18:31:29.146912+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:31:31.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.148+0000 7fea00fc2640 1 -- 172.21.15.5:0/525940970 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe9dc06be60 msgr2=0x7fe9dc06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:31.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.148+0000 7fea00fc2640 1 --2- 172.21.15.5:0/525940970 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe9dc06be60 0x7fe9dc06e370 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0034000 tx=0x7fe9f002e000 comp rx=0 tx=0).stop 2024-05-20T18:31:31.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.148+0000 7fea00fc2640 1 -- 172.21.15.5:0/525940970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9fc070850 msgr2=0x7fe9fc158fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:31.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.148+0000 7fea00fc2640 1 --2- 172.21.15.5:0/525940970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9fc070850 0x7fe9fc158fd0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec009fd0 tx=0x7fe9ec00d560 comp rx=0 tx=0).stop 2024-05-20T18:31:31.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.148+0000 7fea00fc2640 1 -- 172.21.15.5:0/525940970 shutdown_connections 2024-05-20T18:31:31.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.148+0000 7fea00fc2640 1 --2- 172.21.15.5:0/525940970 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe9dc06be60 0x7fe9dc06e370 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:31.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.148+0000 7fea00fc2640 1 --2- 172.21.15.5:0/525940970 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9fc0cd1b0 0x7fe9fc0b7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:31.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.148+0000 7fea00fc2640 1 --2- 172.21.15.5:0/525940970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9fc071630 0x7fe9fc0b6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:31.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.148+0000 7fea00fc2640 1 --2- 172.21.15.5:0/525940970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9fc070850 0x7fe9fc158fd0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:31.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.148+0000 7fea00fc2640 1 -- 172.21.15.5:0/525940970 >> 172.21.15.5:0/525940970 conn(0x7fe9fc0d29d0 msgr2=0x7fe9fc0d1770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:31.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.148+0000 7fea00fc2640 1 -- 172.21.15.5:0/525940970 shutdown_connections 2024-05-20T18:31:31.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.148+0000 7fea00fc2640 1 -- 172.21.15.5:0/525940970 wait complete. 2024-05-20T18:31:31.160 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:31:31.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.275+0000 7f032cb65640 1 Processor -- start 2024-05-20T18:31:31.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.275+0000 7f032cb65640 1 -- start start 2024-05-20T18:31:31.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.276+0000 7f032cb65640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0328162820 0x7f03280b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:31.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.277+0000 7f032cb65640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f032806dae0 0x7f03280b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:31.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.277+0000 7f032cb65640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03280d0630 con 0x7f03280b3e80 2024-05-20T18:31:31.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.277+0000 7f032cb65640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f03280d39c0 con 0x7f0328162820 2024-05-20T18:31:31.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.277+0000 7f032cb65640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f03280d3ca0 con 0x7f032806dae0 2024-05-20T18:31:31.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.277+0000 7f032659c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0328162820 0x7f03280b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:31.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.277+0000 7f0325d9b640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f03280b3e80 0x7f032806d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43408/0 (socket says 172.21.15.5:43408) 2024-05-20T18:31:31.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.277+0000 7f0325d9b640 1 -- 172.21.15.5:0/1051216950 learned_addr learned my addr 172.21.15.5:0/1051216950 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:31.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.277+0000 7f0326d9d640 1 --2- 172.21.15.5:0/1051216950 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f032806dae0 0x7f03280b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:31.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.277+0000 7f032659c640 1 -- 172.21.15.5:0/1051216950 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f032806dae0 msgr2=0x7f03280b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:31.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.277+0000 7f032659c640 1 --2- 172.21.15.5:0/1051216950 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f032806dae0 0x7f03280b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:31.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.277+0000 7f032659c640 1 -- 172.21.15.5:0/1051216950 >> v1:172.21.15.5:6789/0 conn(0x7f03280b3e80 legacy=0x7f032806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:31.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.277+0000 7f032659c640 1 -- 172.21.15.5:0/1051216950 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03280d3f80 con 0x7f0328162820 2024-05-20T18:31:31.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.278+0000 7f032659c640 1 --2- 172.21.15.5:0/1051216950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0328162820 0x7f03280b3940 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f0320009d90 tx=0x7f032002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0465f4e37299b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:31.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.278+0000 7f032559a640 1 -- 172.21.15.5:0/1051216950 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0320007740 con 0x7f0328162820 2024-05-20T18:31:31.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.278+0000 7f032559a640 1 -- 172.21.15.5:0/1051216950 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03200078e0 con 0x7f0328162820 2024-05-20T18:31:31.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.278+0000 7f032559a640 1 -- 172.21.15.5:0/1051216950 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0320007cb0 con 0x7f0328162820 2024-05-20T18:31:31.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.279+0000 7f032cb65640 1 -- 172.21.15.5:0/1051216950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0328162820 msgr2=0x7f03280b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:31.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.279+0000 7f032cb65640 1 --2- 172.21.15.5:0/1051216950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0328162820 0x7f03280b3940 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f0320009d90 tx=0x7f032002a660 comp rx=0 tx=0).stop 2024-05-20T18:31:31.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.279+0000 7f032cb65640 1 -- 172.21.15.5:0/1051216950 shutdown_connections 2024-05-20T18:31:31.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.279+0000 7f032cb65640 1 --2- 172.21.15.5:0/1051216950 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f032806dae0 0x7f03280b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:31.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.279+0000 7f032cb65640 1 --2- 172.21.15.5:0/1051216950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0328162820 0x7f03280b3940 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:31.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.279+0000 7f032cb65640 1 -- 172.21.15.5:0/1051216950 >> 172.21.15.5:0/1051216950 conn(0x7f03280d4bd0 msgr2=0x7f03280d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:31.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.279+0000 7f032cb65640 1 -- 172.21.15.5:0/1051216950 shutdown_connections 2024-05-20T18:31:31.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.279+0000 7f032cb65640 1 -- 172.21.15.5:0/1051216950 wait complete. 2024-05-20T18:31:31.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.281+0000 7f032cb65640 1 Processor -- start 2024-05-20T18:31:31.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.281+0000 7f032cb65640 1 -- start start 2024-05-20T18:31:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.282+0000 7f032cb65640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03280b3e80 0x7f03280cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.282+0000 7f032cb65640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0328162820 0x7f03281f49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.282+0000 7f032cb65640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03281f4f00 0x7f03280cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.282+0000 7f032cb65640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f03280d4780 con 0x7f03280b3e80 2024-05-20T18:31:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.282+0000 7f032cb65640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f03280d0300 con 0x7f0328162820 2024-05-20T18:31:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.282+0000 7f032cb65640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f03280d4900 con 0x7f03281f4f00 2024-05-20T18:31:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.282+0000 7f032659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03280b3e80 0x7f03280cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.282+0000 7f032659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03280b3e80 0x7f03280cbf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35418/0 (socket says 172.21.15.5:35418) 2024-05-20T18:31:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.282+0000 7f032659c640 1 -- 172.21.15.5:0/4106832438 learned_addr learned my addr 172.21.15.5:0/4106832438 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.282+0000 7f0326d9d640 1 --2- 172.21.15.5:0/4106832438 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03281f4f00 0x7f03280cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.282+0000 7f0325d9b640 1 --2- 172.21.15.5:0/4106832438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0328162820 0x7f03281f49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.283+0000 7f032659c640 1 -- 172.21.15.5:0/4106832438 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03281f4f00 msgr2=0x7f03280cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.283+0000 7f032659c640 1 --2- 172.21.15.5:0/4106832438 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03281f4f00 0x7f03280cd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.283+0000 7f032659c640 1 -- 172.21.15.5:0/4106832438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0328162820 msgr2=0x7f03281f49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.283+0000 7f032659c640 1 --2- 172.21.15.5:0/4106832438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0328162820 0x7f03281f49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.283+0000 7f032659c640 1 -- 172.21.15.5:0/4106832438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03280c8e70 con 0x7f03280b3e80 2024-05-20T18:31:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.283+0000 7f032659c640 1 --2- 172.21.15.5:0/4106832438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03280b3e80 0x7f03280cbf00 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f0320009d90 tx=0x7f0320004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.283+0000 7f031effd640 1 -- 172.21.15.5:0/4106832438 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032003f070 con 0x7f03280b3e80 2024-05-20T18:31:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.283+0000 7f031effd640 1 -- 172.21.15.5:0/4106832438 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0320031c40 con 0x7f03280b3e80 2024-05-20T18:31:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.283+0000 7f032cb65640 1 -- 172.21.15.5:0/4106832438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03280cec70 con 0x7f03280b3e80 2024-05-20T18:31:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.283+0000 7f031effd640 1 -- 172.21.15.5:0/4106832438 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0320044460 con 0x7f03280b3e80 2024-05-20T18:31:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.283+0000 7f032cb65640 1 -- 172.21.15.5:0/4106832438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03280c9110 con 0x7f03280b3e80 2024-05-20T18:31:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.284+0000 7f032cb65640 1 -- 172.21.15.5:0/4106832438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0328075b00 con 0x7f03280b3e80 2024-05-20T18:31:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.285+0000 7f031effd640 1 -- 172.21.15.5:0/4106832438 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0320044600 con 0x7f03280b3e80 2024-05-20T18:31:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.285+0000 7f031effd640 1 --2- 172.21.15.5:0/4106832438 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f030806be60 0x7f030806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.286+0000 7f031effd640 1 -- 172.21.15.5:0/4106832438 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6101+0+0 (secure 0 0 0) 0x7f03200ac680 con 0x7f03280b3e80 2024-05-20T18:31:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.286+0000 7f0325d9b640 1 --2- 172.21.15.5:0/4106832438 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f030806be60 0x7f030806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.286+0000 7f0325d9b640 1 --2- 172.21.15.5:0/4106832438 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f030806be60 0x7f030806e370 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f03281e8f60 tx=0x7f0310007e80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:31.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.290+0000 7f031effd640 1 -- 172.21.15.5:0/4106832438 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f032007a850 con 0x7f03280b3e80 2024-05-20T18:31:31.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.482+0000 7f032cb65640 1 -- 172.21.15.5:0/4106832438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f03280bab10 con 0x7f03280b3e80 2024-05-20T18:31:31.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.483+0000 7f031effd640 1 -- 172.21.15.5:0/4106832438 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+16838 (secure 0 0 0) 0x7f032007eb50 con 0x7f03280b3e80 2024-05-20T18:31:31.483 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:31:31.483 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":76,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:31:30.163830+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:31:27.896386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-20T18:31:29.146912+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:31:31.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.485+0000 7f032cb65640 1 -- 172.21.15.5:0/4106832438 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f030806be60 msgr2=0x7f030806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:31.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.485+0000 7f032cb65640 1 --2- 172.21.15.5:0/4106832438 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f030806be60 0x7f030806e370 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f03281e8f60 tx=0x7f0310007e80 comp rx=0 tx=0).stop 2024-05-20T18:31:31.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.485+0000 7f032cb65640 1 -- 172.21.15.5:0/4106832438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03280b3e80 msgr2=0x7f03280cbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:31.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.485+0000 7f032cb65640 1 --2- 172.21.15.5:0/4106832438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03280b3e80 0x7f03280cbf00 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f0320009d90 tx=0x7f0320004a40 comp rx=0 tx=0).stop 2024-05-20T18:31:31.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.485+0000 7f032cb65640 1 -- 172.21.15.5:0/4106832438 shutdown_connections 2024-05-20T18:31:31.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.485+0000 7f032cb65640 1 --2- 172.21.15.5:0/4106832438 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f030806be60 0x7f030806e370 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:31.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.485+0000 7f032cb65640 1 --2- 172.21.15.5:0/4106832438 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03281f4f00 0x7f03280cd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:31.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.485+0000 7f032cb65640 1 --2- 172.21.15.5:0/4106832438 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0328162820 0x7f03281f49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:31.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.485+0000 7f032cb65640 1 --2- 172.21.15.5:0/4106832438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03280b3e80 0x7f03280cbf00 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:31.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.485+0000 7f032cb65640 1 -- 172.21.15.5:0/4106832438 >> 172.21.15.5:0/4106832438 conn(0x7f03280d4bd0 msgr2=0x7f032815e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:31.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.485+0000 7f032cb65640 1 -- 172.21.15.5:0/4106832438 shutdown_connections 2024-05-20T18:31:31.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.485+0000 7f032cb65640 1 -- 172.21.15.5:0/4106832438 wait complete. 2024-05-20T18:31:31.497 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-20T18:31:31.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.615+0000 7fa97b565640 1 Processor -- start 2024-05-20T18:31:31.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.615+0000 7fa97b565640 1 -- start start 2024-05-20T18:31:31.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.617+0000 7fa97b565640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9740afad0 0x7fa9740b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:31.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.617+0000 7fa97b565640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa97415d160 0x7fa9740b5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:31.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.617+0000 7fa97b565640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9740b3aa0 con 0x7fa9740afee0 2024-05-20T18:31:31.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.617+0000 7fa97b565640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa9740b3850 con 0x7fa97415d160 2024-05-20T18:31:31.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.617+0000 7fa97b565640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa9740703e0 con 0x7fa9740afad0 2024-05-20T18:31:31.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.617+0000 7fa979301640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9740afad0 0x7fa9740b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:31.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.617+0000 7fa979b02640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa97415d160 0x7fa9740b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:31.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.617+0000 7fa979301640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9740afad0 0x7fa9740b51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:37686/0 (socket says 172.21.15.5:37686) 2024-05-20T18:31:31.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.617+0000 7fa979b02640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa97415d160 0x7fa9740b5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:33244/0 (socket says 172.21.15.5:33244) 2024-05-20T18:31:31.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.617+0000 7fa979301640 1 -- 172.21.15.5:0/3240637883 learned_addr learned my addr 172.21.15.5:0/3240637883 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:31.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.617+0000 7fa979301640 1 -- 172.21.15.5:0/3240637883 >> v2:172.21.15.63:3300/0 conn(0x7fa97415d160 msgr2=0x7fa9740b5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:31.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.617+0000 7fa979301640 1 --2- 172.21.15.5:0/3240637883 >> v2:172.21.15.63:3300/0 conn(0x7fa97415d160 0x7fa9740b5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:31.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.617+0000 7fa979301640 1 -- 172.21.15.5:0/3240637883 >> v1:172.21.15.63:6789/0 conn(0x7fa9740afee0 legacy=0x7fa97415ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:31.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.618+0000 7fa979301640 1 -- 172.21.15.5:0/3240637883 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9740b3d80 con 0x7fa9740afad0 2024-05-20T18:31:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.618+0000 7fa979301640 1 --2- 172.21.15.5:0/3240637883 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9740afad0 0x7fa9740b51e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa96c009870 tx=0x7fa96c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8f20ec9538d343d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.619+0000 7fa973fff640 1 -- 172.21.15.5:0/3240637883 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c037070 con 0x7fa9740afad0 2024-05-20T18:31:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.619+0000 7fa973fff640 1 -- 172.21.15.5:0/3240637883 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa96c032440 con 0x7fa9740afad0 2024-05-20T18:31:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.619+0000 7fa973fff640 1 -- 172.21.15.5:0/3240637883 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c032810 con 0x7fa9740afad0 2024-05-20T18:31:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.619+0000 7fa97b565640 1 -- 172.21.15.5:0/3240637883 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9740afad0 msgr2=0x7fa9740b51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.619+0000 7fa97b565640 1 --2- 172.21.15.5:0/3240637883 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9740afad0 0x7fa9740b51e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa96c009870 tx=0x7fa96c02a270 comp rx=0 tx=0).stop 2024-05-20T18:31:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.619+0000 7fa97b565640 1 -- 172.21.15.5:0/3240637883 shutdown_connections 2024-05-20T18:31:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.619+0000 7fa97b565640 1 --2- 172.21.15.5:0/3240637883 >> v2:172.21.15.63:3300/0 conn(0x7fa97415d160 0x7fa9740b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.619+0000 7fa97b565640 1 --2- 172.21.15.5:0/3240637883 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9740afad0 0x7fa9740b51e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.619+0000 7fa97b565640 1 -- 172.21.15.5:0/3240637883 >> 172.21.15.5:0/3240637883 conn(0x7fa9740cd2a0 msgr2=0x7fa9740cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.619+0000 7fa97b565640 1 -- 172.21.15.5:0/3240637883 shutdown_connections 2024-05-20T18:31:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.619+0000 7fa97b565640 1 -- 172.21.15.5:0/3240637883 wait complete. 2024-05-20T18:31:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.621+0000 7fa97b565640 1 Processor -- start 2024-05-20T18:31:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.621+0000 7fa97b565640 1 -- start start 2024-05-20T18:31:31.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.622+0000 7fa97b565640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9740afad0 0x7fa9741492f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:31.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.622+0000 7fa97b565640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9740afee0 0x7fa974149af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:31.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.622+0000 7fa97b565640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa97415d160 0x7fa97414e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:31.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.622+0000 7fa97b565640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa9740b4860 con 0x7fa9740afad0 2024-05-20T18:31:31.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.622+0000 7fa97b565640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa9740b4340 con 0x7fa97415d160 2024-05-20T18:31:31.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.622+0000 7fa97b565640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa97414a360 con 0x7fa9740afee0 2024-05-20T18:31:31.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.622+0000 7fa979301640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9740afad0 0x7fa9741492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:31.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.623+0000 7fa979301640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9740afad0 0x7fa9741492f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35426/0 (socket says 172.21.15.5:35426) 2024-05-20T18:31:31.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.623+0000 7fa979301640 1 -- 172.21.15.5:0/2508289802 learned_addr learned my addr 172.21.15.5:0/2508289802 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:31.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.623+0000 7fa978b00640 1 --2- 172.21.15.5:0/2508289802 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9740afee0 0x7fa974149af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:31.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.623+0000 7fa979b02640 1 --2- 172.21.15.5:0/2508289802 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa97415d160 0x7fa97414e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:31.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.623+0000 7fa979301640 1 -- 172.21.15.5:0/2508289802 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9740afee0 msgr2=0x7fa974149af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:31.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.623+0000 7fa979301640 1 --2- 172.21.15.5:0/2508289802 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9740afee0 0x7fa974149af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:31.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.623+0000 7fa979301640 1 -- 172.21.15.5:0/2508289802 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa97415d160 msgr2=0x7fa97414e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:31.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.623+0000 7fa979301640 1 --2- 172.21.15.5:0/2508289802 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa97415d160 0x7fa97414e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:31.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.623+0000 7fa979301640 1 -- 172.21.15.5:0/2508289802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97415ef90 con 0x7fa9740afad0 2024-05-20T18:31:31.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.623+0000 7fa979301640 1 --2- 172.21.15.5:0/2508289802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9740afad0 0x7fa9741492f0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fa96c000c00 tx=0x7fa96c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:31.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.623+0000 7fa971ffb640 1 -- 172.21.15.5:0/2508289802 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c037040 con 0x7fa9740afad0 2024-05-20T18:31:31.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.623+0000 7fa971ffb640 1 -- 172.21.15.5:0/2508289802 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa96c042070 con 0x7fa9740afad0 2024-05-20T18:31:31.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.623+0000 7fa971ffb640 1 -- 172.21.15.5:0/2508289802 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c030880 con 0x7fa9740afad0 2024-05-20T18:31:31.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.623+0000 7fa97b565640 1 -- 172.21.15.5:0/2508289802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa97415f270 con 0x7fa9740afad0 2024-05-20T18:31:31.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.624+0000 7fa97b565640 1 -- 172.21.15.5:0/2508289802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa974149830 con 0x7fa9740afad0 2024-05-20T18:31:31.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.625+0000 7fa97b565640 1 -- 172.21.15.5:0/2508289802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa974075b00 con 0x7fa9740afad0 2024-05-20T18:31:31.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.625+0000 7fa971ffb640 1 -- 172.21.15.5:0/2508289802 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa96c030a20 con 0x7fa9740afad0 2024-05-20T18:31:31.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.626+0000 7fa971ffb640 1 --2- 172.21.15.5:0/2508289802 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa96006be60 0x7fa96006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:31.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.626+0000 7fa971ffb640 1 -- 172.21.15.5:0/2508289802 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6101+0+0 (secure 0 0 0) 0x7fa96c0ab910 con 0x7fa9740afad0 2024-05-20T18:31:31.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.626+0000 7fa978b00640 1 --2- 172.21.15.5:0/2508289802 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa96006be60 0x7fa96006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:31.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.626+0000 7fa978b00640 1 --2- 172.21.15.5:0/2508289802 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa96006be60 0x7fa96006e370 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa968007fd0 tx=0x7fa968009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:31.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.630+0000 7fa971ffb640 1 -- 172.21.15.5:0/2508289802 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa96c07a000 con 0x7fa9740afad0 2024-05-20T18:31:31.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:31.855+0000 7fa97b565640 1 -- 172.21.15.5:0/2508289802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa974166bf0 con 0x7fa9740afad0 2024-05-20T18:31:32.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.799+0000 7fa971ffb640 1 -- 172.21.15.5:0/2508289802 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v43) ==== 118+0+0 (secure 0 0 0) 0x7fa96c079900 con 0x7fa9740afad0 2024-05-20T18:31:32.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.802+0000 7fa97b565640 1 -- 172.21.15.5:0/2508289802 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa96006be60 msgr2=0x7fa96006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:32.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.802+0000 7fa97b565640 1 --2- 172.21.15.5:0/2508289802 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa96006be60 0x7fa96006e370 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa968007fd0 tx=0x7fa968009040 comp rx=0 tx=0).stop 2024-05-20T18:31:32.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.802+0000 7fa97b565640 1 -- 172.21.15.5:0/2508289802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9740afad0 msgr2=0x7fa9741492f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:32.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.802+0000 7fa97b565640 1 --2- 172.21.15.5:0/2508289802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9740afad0 0x7fa9741492f0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fa96c000c00 tx=0x7fa96c0046c0 comp rx=0 tx=0).stop 2024-05-20T18:31:32.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.802+0000 7fa97b565640 1 -- 172.21.15.5:0/2508289802 shutdown_connections 2024-05-20T18:31:32.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.802+0000 7fa97b565640 1 --2- 172.21.15.5:0/2508289802 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa96006be60 0x7fa96006e370 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:32.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.802+0000 7fa97b565640 1 --2- 172.21.15.5:0/2508289802 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa97415d160 0x7fa97414e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:32.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.802+0000 7fa97b565640 1 --2- 172.21.15.5:0/2508289802 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9740afee0 0x7fa974149af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:32.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.802+0000 7fa97b565640 1 --2- 172.21.15.5:0/2508289802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9740afad0 0x7fa9741492f0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:32.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.802+0000 7fa97b565640 1 -- 172.21.15.5:0/2508289802 >> 172.21.15.5:0/2508289802 conn(0x7fa9740cd2a0 msgr2=0x7fa9740b04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:32.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.802+0000 7fa97b565640 1 -- 172.21.15.5:0/2508289802 shutdown_connections 2024-05-20T18:31:32.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.803+0000 7fa97b565640 1 -- 172.21.15.5:0/2508289802 wait complete. 2024-05-20T18:31:32.818 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-20T18:31:32.818 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-20T18:31:32.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.945+0000 7f425965a640 1 Processor -- start 2024-05-20T18:31:32.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.945+0000 7f425965a640 1 -- start start 2024-05-20T18:31:32.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.947+0000 7f425965a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4254162820 0x7f42540b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:32.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.947+0000 7f425965a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f425406dae0 0x7f42540b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:32.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.947+0000 7f425965a640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42540d0630 con 0x7f42540b3e80 2024-05-20T18:31:32.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.947+0000 7f425965a640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f42540d39c0 con 0x7f4254162820 2024-05-20T18:31:32.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.947+0000 7f425965a640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f42540d3ca0 con 0x7f425406dae0 2024-05-20T18:31:32.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.947+0000 7f4252ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4254162820 0x7f42540b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:32.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.947+0000 7f4252ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4254162820 0x7f42540b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35442/0 (socket says 172.21.15.5:35442) 2024-05-20T18:31:32.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.947+0000 7f4252ffd640 1 -- 172.21.15.5:0/2543516338 learned_addr learned my addr 172.21.15.5:0/2543516338 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:32.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.947+0000 7f42537fe640 1 --2- 172.21.15.5:0/2543516338 >> v2:172.21.15.63:3300/0 conn(0x7f425406dae0 0x7f42540b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:32.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.947+0000 7f4252ffd640 1 -- 172.21.15.5:0/2543516338 >> v2:172.21.15.63:3300/0 conn(0x7f425406dae0 msgr2=0x7f42540b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:32.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.947+0000 7f4252ffd640 1 --2- 172.21.15.5:0/2543516338 >> v2:172.21.15.63:3300/0 conn(0x7f425406dae0 0x7f42540b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:32.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.947+0000 7f4252ffd640 1 -- 172.21.15.5:0/2543516338 >> v1:172.21.15.63:6789/0 conn(0x7f42540b3e80 legacy=0x7f425406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:32.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.947+0000 7f4252ffd640 1 -- 172.21.15.5:0/2543516338 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42540d3f80 con 0x7f4254162820 2024-05-20T18:31:32.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.947+0000 7f4252ffd640 1 --2- 172.21.15.5:0/2543516338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4254162820 0x7f42540b3940 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f425408e070 tx=0x7f424402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4273d3da7b0f242 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:32.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.947+0000 7f4251ffb640 1 -- 172.21.15.5:0/2543516338 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4244037070 con 0x7f4254162820 2024-05-20T18:31:32.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.947+0000 7f4251ffb640 1 -- 172.21.15.5:0/2543516338 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4244032440 con 0x7f4254162820 2024-05-20T18:31:32.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.947+0000 7f4251ffb640 1 -- 172.21.15.5:0/2543516338 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4244032810 con 0x7f4254162820 2024-05-20T18:31:32.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.948+0000 7f425965a640 1 -- 172.21.15.5:0/2543516338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4254162820 msgr2=0x7f42540b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:32.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.948+0000 7f425965a640 1 --2- 172.21.15.5:0/2543516338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4254162820 0x7f42540b3940 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f425408e070 tx=0x7f424402a230 comp rx=0 tx=0).stop 2024-05-20T18:31:32.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.948+0000 7f425965a640 1 -- 172.21.15.5:0/2543516338 shutdown_connections 2024-05-20T18:31:32.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.948+0000 7f425965a640 1 --2- 172.21.15.5:0/2543516338 >> v2:172.21.15.63:3300/0 conn(0x7f425406dae0 0x7f42540b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:32.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.948+0000 7f425965a640 1 --2- 172.21.15.5:0/2543516338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4254162820 0x7f42540b3940 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:32.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.948+0000 7f425965a640 1 -- 172.21.15.5:0/2543516338 >> 172.21.15.5:0/2543516338 conn(0x7f42540d4bd0 msgr2=0x7f42540d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:32.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.948+0000 7f425965a640 1 -- 172.21.15.5:0/2543516338 shutdown_connections 2024-05-20T18:31:32.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.948+0000 7f425965a640 1 -- 172.21.15.5:0/2543516338 wait complete. 2024-05-20T18:31:32.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.951+0000 7f425965a640 1 Processor -- start 2024-05-20T18:31:32.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.951+0000 7f425965a640 1 -- start start 2024-05-20T18:31:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.952+0000 7f425965a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42540b3e80 0x7f4254151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.952+0000 7f425965a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4254162820 0x7f42540c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.952+0000 7f425965a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42540c9720 0x7f42540c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.952+0000 7f425965a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f42540d4780 con 0x7f42540c9720 2024-05-20T18:31:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.952+0000 7f425965a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f42540d0300 con 0x7f42540b3e80 2024-05-20T18:31:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.952+0000 7f425965a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f42540d4900 con 0x7f4254162820 2024-05-20T18:31:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.952+0000 7f42537fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42540c9720 0x7f42540c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.952+0000 7f42537fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42540c9720 0x7f42540c9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35444/0 (socket says 172.21.15.5:35444) 2024-05-20T18:31:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.952+0000 7f42537fe640 1 -- 172.21.15.5:0/3278708925 learned_addr learned my addr 172.21.15.5:0/3278708925 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.952+0000 7f4252ffd640 1 --2- 172.21.15.5:0/3278708925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42540b3e80 0x7f4254151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.952+0000 7f42527fc640 1 --2- 172.21.15.5:0/3278708925 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4254162820 0x7f42540c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.952+0000 7f42537fe640 1 -- 172.21.15.5:0/3278708925 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4254162820 msgr2=0x7f42540c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.952+0000 7f42537fe640 1 --2- 172.21.15.5:0/3278708925 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4254162820 0x7f42540c91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.952+0000 7f42537fe640 1 -- 172.21.15.5:0/3278708925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42540b3e80 msgr2=0x7f4254151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.952+0000 7f42537fe640 1 --2- 172.21.15.5:0/3278708925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42540b3e80 0x7f4254151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.952+0000 7f42537fe640 1 -- 172.21.15.5:0/3278708925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42540ca070 con 0x7f42540c9720 2024-05-20T18:31:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.952+0000 7f42537fe640 1 --2- 172.21.15.5:0/3278708925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42540c9720 0x7f42540c9b30 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f424c007e90 tx=0x7f424c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.952+0000 7f423b7fe640 1 -- 172.21.15.5:0/3278708925 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424c014420 con 0x7f42540c9720 2024-05-20T18:31:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.952+0000 7f423b7fe640 1 -- 172.21.15.5:0/3278708925 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f424c019070 con 0x7f42540c9720 2024-05-20T18:31:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.952+0000 7f425965a640 1 -- 172.21.15.5:0/3278708925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42540caa80 con 0x7f42540c9720 2024-05-20T18:31:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.952+0000 7f423b7fe640 1 -- 172.21.15.5:0/3278708925 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424c014800 con 0x7f42540c9720 2024-05-20T18:31:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.952+0000 7f425965a640 1 -- 172.21.15.5:0/3278708925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42540c8700 con 0x7f42540c9720 2024-05-20T18:31:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.953+0000 7f425965a640 1 -- 172.21.15.5:0/3278708925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42540ca7f0 con 0x7f42540c9720 2024-05-20T18:31:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.954+0000 7f423b7fe640 1 -- 172.21.15.5:0/3278708925 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f424c0149a0 con 0x7f42540c9720 2024-05-20T18:31:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.954+0000 7f423b7fe640 1 --2- 172.21.15.5:0/3278708925 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f424006be30 0x7f424006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.954+0000 7f423b7fe640 1 -- 172.21.15.5:0/3278708925 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6101+0+0 (secure 0 0 0) 0x7f424c08b800 con 0x7f42540c9720 2024-05-20T18:31:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.954+0000 7f4252ffd640 1 --2- 172.21.15.5:0/3278708925 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f424006be30 0x7f424006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.955+0000 7f4252ffd640 1 --2- 172.21.15.5:0/3278708925 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f424006be30 0x7f424006e340 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4244034000 tx=0x7f424402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:32.959+0000 7f423b7fe640 1 -- 172.21.15.5:0/3278708925 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f424c05a020 con 0x7f42540c9720 2024-05-20T18:31:33.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.184+0000 7f425965a640 1 -- 172.21.15.5:0/3278708925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f425415da60 con 0x7f42540c9720 2024-05-20T18:31:33.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.802+0000 7f423b7fe640 1 -- 172.21.15.5:0/3278708925 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v44) ==== 122+0+0 (secure 0 0 0) 0x7f424c059920 con 0x7f42540c9720 2024-05-20T18:31:33.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.804+0000 7f425965a640 1 -- 172.21.15.5:0/3278708925 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f424006be30 msgr2=0x7f424006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:33.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.804+0000 7f425965a640 1 --2- 172.21.15.5:0/3278708925 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f424006be30 0x7f424006e340 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4244034000 tx=0x7f424402e000 comp rx=0 tx=0).stop 2024-05-20T18:31:33.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.805+0000 7f425965a640 1 -- 172.21.15.5:0/3278708925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42540c9720 msgr2=0x7f42540c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:33.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.805+0000 7f425965a640 1 --2- 172.21.15.5:0/3278708925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42540c9720 0x7f42540c9b30 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f424c007e90 tx=0x7f424c00e4b0 comp rx=0 tx=0).stop 2024-05-20T18:31:33.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.805+0000 7f425965a640 1 -- 172.21.15.5:0/3278708925 shutdown_connections 2024-05-20T18:31:33.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.805+0000 7f425965a640 1 --2- 172.21.15.5:0/3278708925 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f424006be30 0x7f424006e340 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:33.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.805+0000 7f425965a640 1 --2- 172.21.15.5:0/3278708925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42540c9720 0x7f42540c9b30 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:33.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.805+0000 7f425965a640 1 --2- 172.21.15.5:0/3278708925 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4254162820 0x7f42540c91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:33.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.805+0000 7f425965a640 1 --2- 172.21.15.5:0/3278708925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42540b3e80 0x7f4254151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:33.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.805+0000 7f425965a640 1 -- 172.21.15.5:0/3278708925 >> 172.21.15.5:0/3278708925 conn(0x7f42540d4bd0 msgr2=0x7f425415e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:33.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.805+0000 7f425965a640 1 -- 172.21.15.5:0/3278708925 shutdown_connections 2024-05-20T18:31:33.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.805+0000 7f425965a640 1 -- 172.21.15.5:0/3278708925 wait complete. 2024-05-20T18:31:33.819 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-20T18:31:33.819 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:31:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.941+0000 7f402d25a640 1 Processor -- start 2024-05-20T18:31:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.941+0000 7f402d25a640 1 -- start start 2024-05-20T18:31:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.943+0000 7f402d25a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4028162820 0x7f40280b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.943+0000 7f402d25a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f402806dae0 0x7f40280b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.943+0000 7f402d25a640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40280d0630 con 0x7f40280b3e80 2024-05-20T18:31:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.943+0000 7f402d25a640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f40280d39c0 con 0x7f402806dae0 2024-05-20T18:31:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.943+0000 7f402d25a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f40280d3ca0 con 0x7f4028162820 2024-05-20T18:31:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.943+0000 7f402759e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f402806dae0 0x7f40280b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.943+0000 7f402659c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f40280b3e80 0x7f402806d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43418/0 (socket says 172.21.15.5:43418) 2024-05-20T18:31:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.943+0000 7f402659c640 1 -- 172.21.15.5:0/4064606542 learned_addr learned my addr 172.21.15.5:0/4064606542 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.943+0000 7f4026d9d640 1 --2- 172.21.15.5:0/4064606542 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4028162820 0x7f40280b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:33.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.943+0000 7f402759e640 1 -- 172.21.15.5:0/4064606542 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4028162820 msgr2=0x7f40280b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:33.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.943+0000 7f402759e640 1 --2- 172.21.15.5:0/4064606542 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4028162820 0x7f40280b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:33.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.943+0000 7f402759e640 1 -- 172.21.15.5:0/4064606542 >> v1:172.21.15.5:6789/0 conn(0x7f40280b3e80 legacy=0x7f402806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:33.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.943+0000 7f402759e640 1 -- 172.21.15.5:0/4064606542 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40280d3f80 con 0x7f402806dae0 2024-05-20T18:31:33.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.943+0000 7f4025d9b640 1 -- 172.21.15.5:0/4064606542 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3254876849 0 0) 0x7f40280d0630 con 0x7f40280b3e80 2024-05-20T18:31:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.944+0000 7f402759e640 1 --2- 172.21.15.5:0/4064606542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f402806dae0 0x7f40280b8190 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f4020009d90 tx=0x7f402002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac452dc4326d448d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.944+0000 7f4025d9b640 1 -- 172.21.15.5:0/4064606542 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4020007740 con 0x7f402806dae0 2024-05-20T18:31:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.944+0000 7f4025d9b640 1 -- 172.21.15.5:0/4064606542 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40200078e0 con 0x7f402806dae0 2024-05-20T18:31:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.944+0000 7f4025d9b640 1 -- 172.21.15.5:0/4064606542 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4020007cb0 con 0x7f402806dae0 2024-05-20T18:31:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.945+0000 7f402d25a640 1 -- 172.21.15.5:0/4064606542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f402806dae0 msgr2=0x7f40280b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:33.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.945+0000 7f402d25a640 1 --2- 172.21.15.5:0/4064606542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f402806dae0 0x7f40280b8190 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f4020009d90 tx=0x7f402002a660 comp rx=0 tx=0).stop 2024-05-20T18:31:33.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.945+0000 7f402d25a640 1 -- 172.21.15.5:0/4064606542 shutdown_connections 2024-05-20T18:31:33.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.945+0000 7f402d25a640 1 --2- 172.21.15.5:0/4064606542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f402806dae0 0x7f40280b8190 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:33.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.945+0000 7f402d25a640 1 --2- 172.21.15.5:0/4064606542 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4028162820 0x7f40280b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:33.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.945+0000 7f402d25a640 1 -- 172.21.15.5:0/4064606542 >> 172.21.15.5:0/4064606542 conn(0x7f40280d4bd0 msgr2=0x7f40280d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:33.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.945+0000 7f402d25a640 1 -- 172.21.15.5:0/4064606542 shutdown_connections 2024-05-20T18:31:33.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.945+0000 7f402d25a640 1 -- 172.21.15.5:0/4064606542 wait complete. 2024-05-20T18:31:33.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.947+0000 7f402d25a640 1 Processor -- start 2024-05-20T18:31:33.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.947+0000 7f402d25a640 1 -- start start 2024-05-20T18:31:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.948+0000 7f402d25a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40280b3e80 0x7f40280b6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.948+0000 7f402d25a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4028162820 0x7f402815a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.949+0000 7f402d25a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f40280b5040 0x7f402815aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.949+0000 7f402d25a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f40280d4780 con 0x7f4028162820 2024-05-20T18:31:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.949+0000 7f402d25a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f40280d0300 con 0x7f40280b3e80 2024-05-20T18:31:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.949+0000 7f402d25a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f40280d4900 con 0x7f40280b5040 2024-05-20T18:31:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.949+0000 7f402659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4028162820 0x7f402815a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.949+0000 7f402659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4028162820 0x7f402815a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35468/0 (socket says 172.21.15.5:35468) 2024-05-20T18:31:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.949+0000 7f402659c640 1 -- 172.21.15.5:0/1544271129 learned_addr learned my addr 172.21.15.5:0/1544271129 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.949+0000 7f402759e640 1 --2- 172.21.15.5:0/1544271129 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f40280b5040 0x7f402815aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.949+0000 7f4026d9d640 1 --2- 172.21.15.5:0/1544271129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40280b3e80 0x7f40280b6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.949+0000 7f402659c640 1 -- 172.21.15.5:0/1544271129 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f40280b5040 msgr2=0x7f402815aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.949+0000 7f402659c640 1 --2- 172.21.15.5:0/1544271129 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f40280b5040 0x7f402815aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.949+0000 7f402659c640 1 -- 172.21.15.5:0/1544271129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40280b3e80 msgr2=0x7f40280b6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.949+0000 7f402659c640 1 --2- 172.21.15.5:0/1544271129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40280b3e80 0x7f40280b6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.949+0000 7f402659c640 1 -- 172.21.15.5:0/1544271129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40280ceae0 con 0x7f4028162820 2024-05-20T18:31:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.949+0000 7f402659c640 1 --2- 172.21.15.5:0/1544271129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4028162820 0x7f402815a5a0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f401000c440 tx=0x7f401000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.949+0000 7f401f7fe640 1 -- 172.21.15.5:0/1544271129 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4010018070 con 0x7f4028162820 2024-05-20T18:31:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.949+0000 7f401f7fe640 1 -- 172.21.15.5:0/1544271129 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40100074e0 con 0x7f4028162820 2024-05-20T18:31:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.949+0000 7f402d25a640 1 -- 172.21.15.5:0/1544271129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40280ccca0 con 0x7f4028162820 2024-05-20T18:31:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.949+0000 7f401f7fe640 1 -- 172.21.15.5:0/1544271129 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4010004a20 con 0x7f4028162820 2024-05-20T18:31:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.949+0000 7f402d25a640 1 -- 172.21.15.5:0/1544271129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40280cd260 con 0x7f4028162820 2024-05-20T18:31:33.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.951+0000 7f402d25a640 1 -- 172.21.15.5:0/1544271129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4028075b00 con 0x7f4028162820 2024-05-20T18:31:33.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.951+0000 7f401f7fe640 1 -- 172.21.15.5:0/1544271129 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f401001b070 con 0x7f4028162820 2024-05-20T18:31:33.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.951+0000 7f401f7fe640 1 --2- 172.21.15.5:0/1544271129 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f400806be60 0x7f400806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:33.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.952+0000 7f401f7fe640 1 -- 172.21.15.5:0/1544271129 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6101+0+0 (secure 0 0 0) 0x7f401008a2e0 con 0x7f4028162820 2024-05-20T18:31:33.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.952+0000 7f4026d9d640 1 --2- 172.21.15.5:0/1544271129 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f400806be60 0x7f400806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:33.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.952+0000 7f4026d9d640 1 --2- 172.21.15.5:0/1544271129 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f400806be60 0x7f400806e370 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4018009aa0 tx=0x7f4018009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:33.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:33.956+0000 7f401f7fe640 1 -- 172.21.15.5:0/1544271129 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4010058380 con 0x7f4028162820 2024-05-20T18:31:34.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.177+0000 7f402d25a640 1 -- 172.21.15.5:0/1544271129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40280b7290 con 0x7f4028162820 2024-05-20T18:31:34.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.177+0000 7f401f7fe640 1 -- 172.21.15.5:0/1544271129 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5171 (secure 0 0 0) 0x7f401005c680 con 0x7f4028162820 2024-05-20T18:31:34.177 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:31:34.177 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1890759603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1890759603},{"type":"v1","addr":"172.21.15.63:6834","nonce":1890759603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5825,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/700447403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":700447403},{"type":"v1","addr":"172.21.15.63:6835","nonce":700447403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3382943275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3382943275},{"type":"v1","addr":"172.21.15.5:6837","nonce":3382943275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:31:30.164179+0000","modified":"2024-05-20T18:31:33.185897+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6762},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6762":{"gid":6762,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6836/791176355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":791176355},{"type":"v1","addr":"172.21.15.5:6836","nonce":791176355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6762,"qdb_cluster":[6762]},"id":5}]} 2024-05-20T18:31:34.177 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 44 2024-05-20T18:31:34.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.179+0000 7f402d25a640 1 -- 172.21.15.5:0/1544271129 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f400806be60 msgr2=0x7f400806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:34.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.179+0000 7f402d25a640 1 --2- 172.21.15.5:0/1544271129 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f400806be60 0x7f400806e370 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4018009aa0 tx=0x7f4018009190 comp rx=0 tx=0).stop 2024-05-20T18:31:34.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.179+0000 7f402d25a640 1 -- 172.21.15.5:0/1544271129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4028162820 msgr2=0x7f402815a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:34.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.179+0000 7f402d25a640 1 --2- 172.21.15.5:0/1544271129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4028162820 0x7f402815a5a0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f401000c440 tx=0x7f401000c900 comp rx=0 tx=0).stop 2024-05-20T18:31:34.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.179+0000 7f402d25a640 1 -- 172.21.15.5:0/1544271129 shutdown_connections 2024-05-20T18:31:34.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.179+0000 7f402d25a640 1 --2- 172.21.15.5:0/1544271129 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f400806be60 0x7f400806e370 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:34.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.179+0000 7f402d25a640 1 --2- 172.21.15.5:0/1544271129 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f40280b5040 0x7f402815aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:34.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.179+0000 7f402d25a640 1 --2- 172.21.15.5:0/1544271129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4028162820 0x7f402815a5a0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:34.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.179+0000 7f402d25a640 1 --2- 172.21.15.5:0/1544271129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40280b3e80 0x7f40280b6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:34.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.179+0000 7f402d25a640 1 -- 172.21.15.5:0/1544271129 >> 172.21.15.5:0/1544271129 conn(0x7f40280d4bd0 msgr2=0x7f4028161bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:34.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.179+0000 7f402d25a640 1 -- 172.21.15.5:0/1544271129 shutdown_connections 2024-05-20T18:31:34.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.179+0000 7f402d25a640 1 -- 172.21.15.5:0/1544271129 wait complete. 2024-05-20T18:31:34.190 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:31:34.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.305+0000 7fcf5b510640 1 Processor -- start 2024-05-20T18:31:34.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.306+0000 7fcf5b510640 1 -- start start 2024-05-20T18:31:34.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.307+0000 7fcf5b510640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcf54162820 0x7fcf540b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:34.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.307+0000 7fcf5b510640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf5406dae0 0x7fcf540b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:34.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.307+0000 7fcf5b510640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf540d0630 con 0x7fcf540b3e80 2024-05-20T18:31:34.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.307+0000 7fcf5b510640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fcf540d39c0 con 0x7fcf54162820 2024-05-20T18:31:34.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.307+0000 7fcf5b510640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcf540d3ca0 con 0x7fcf5406dae0 2024-05-20T18:31:34.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.307+0000 7fcf592ac640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcf54162820 0x7fcf540b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:34.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.307+0000 7fcf592ac640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcf54162820 0x7fcf540b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35474/0 (socket says 172.21.15.5:35474) 2024-05-20T18:31:34.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.307+0000 7fcf592ac640 1 -- 172.21.15.5:0/881119910 learned_addr learned my addr 172.21.15.5:0/881119910 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:34.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.308+0000 7fcf59aad640 1 --2- 172.21.15.5:0/881119910 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf5406dae0 0x7fcf540b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:34.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.308+0000 7fcf592ac640 1 -- 172.21.15.5:0/881119910 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf5406dae0 msgr2=0x7fcf540b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:34.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.308+0000 7fcf592ac640 1 --2- 172.21.15.5:0/881119910 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf5406dae0 0x7fcf540b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:34.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.308+0000 7fcf592ac640 1 -- 172.21.15.5:0/881119910 >> v1:172.21.15.63:6789/0 conn(0x7fcf540b3e80 legacy=0x7fcf5406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:34.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.308+0000 7fcf592ac640 1 -- 172.21.15.5:0/881119910 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf540d3f80 con 0x7fcf54162820 2024-05-20T18:31:34.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.308+0000 7fcf59aad640 1 --2- 172.21.15.5:0/881119910 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf5406dae0 0x7fcf540b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:34.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.308+0000 7fcf592ac640 1 --2- 172.21.15.5:0/881119910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf54162820 0x7fcf540b3940 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009870 tx=0x7fcf5002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df297557390b2731 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:34.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.309+0000 7fcf4bfff640 1 -- 172.21.15.5:0/881119910 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50037070 con 0x7fcf54162820 2024-05-20T18:31:34.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.309+0000 7fcf4bfff640 1 -- 172.21.15.5:0/881119910 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf50032440 con 0x7fcf54162820 2024-05-20T18:31:34.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.309+0000 7fcf4bfff640 1 -- 172.21.15.5:0/881119910 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50032810 con 0x7fcf54162820 2024-05-20T18:31:34.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.309+0000 7fcf5b510640 1 -- 172.21.15.5:0/881119910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf54162820 msgr2=0x7fcf540b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:34.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.309+0000 7fcf5b510640 1 --2- 172.21.15.5:0/881119910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf54162820 0x7fcf540b3940 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009870 tx=0x7fcf5002a210 comp rx=0 tx=0).stop 2024-05-20T18:31:34.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.309+0000 7fcf5b510640 1 -- 172.21.15.5:0/881119910 shutdown_connections 2024-05-20T18:31:34.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.309+0000 7fcf5b510640 1 --2- 172.21.15.5:0/881119910 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf5406dae0 0x7fcf540b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:34.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.309+0000 7fcf5b510640 1 --2- 172.21.15.5:0/881119910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf54162820 0x7fcf540b3940 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:34.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.309+0000 7fcf5b510640 1 -- 172.21.15.5:0/881119910 >> 172.21.15.5:0/881119910 conn(0x7fcf540d4bd0 msgr2=0x7fcf540d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:34.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.309+0000 7fcf5b510640 1 -- 172.21.15.5:0/881119910 shutdown_connections 2024-05-20T18:31:34.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.309+0000 7fcf5b510640 1 -- 172.21.15.5:0/881119910 wait complete. 2024-05-20T18:31:34.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.311+0000 7fcf5b510640 1 Processor -- start 2024-05-20T18:31:34.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.311+0000 7fcf5b510640 1 -- start start 2024-05-20T18:31:34.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.312+0000 7fcf5b510640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf540b3e80 0x7fcf540cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:34.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.313+0000 7fcf5b510640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf54162820 0x7fcf5414cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:34.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.313+0000 7fcf5b510640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf540c8b00 0x7fcf540c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:34.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.313+0000 7fcf5b510640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcf540d4780 con 0x7fcf540b3e80 2024-05-20T18:31:34.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.313+0000 7fcf5b510640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcf540d0300 con 0x7fcf54162820 2024-05-20T18:31:34.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.313+0000 7fcf5b510640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcf540d4900 con 0x7fcf540c8b00 2024-05-20T18:31:34.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.313+0000 7fcf592ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf540b3e80 0x7fcf540cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:34.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.313+0000 7fcf592ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf540b3e80 0x7fcf540cb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35478/0 (socket says 172.21.15.5:35478) 2024-05-20T18:31:34.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.313+0000 7fcf592ac640 1 -- 172.21.15.5:0/1476525533 learned_addr learned my addr 172.21.15.5:0/1476525533 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:34.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.313+0000 7fcf59aad640 1 --2- 172.21.15.5:0/1476525533 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf540c8b00 0x7fcf540c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:34.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.313+0000 7fcf58aab640 1 --2- 172.21.15.5:0/1476525533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf54162820 0x7fcf5414cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:34.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.313+0000 7fcf592ac640 1 -- 172.21.15.5:0/1476525533 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf540c8b00 msgr2=0x7fcf540c8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:34.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.313+0000 7fcf592ac640 1 --2- 172.21.15.5:0/1476525533 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf540c8b00 0x7fcf540c8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:34.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.313+0000 7fcf592ac640 1 -- 172.21.15.5:0/1476525533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf54162820 msgr2=0x7fcf5414cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:34.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.313+0000 7fcf592ac640 1 --2- 172.21.15.5:0/1476525533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf54162820 0x7fcf5414cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:34.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.313+0000 7fcf592ac640 1 -- 172.21.15.5:0/1476525533 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf540ce200 con 0x7fcf540b3e80 2024-05-20T18:31:34.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.313+0000 7fcf592ac640 1 --2- 172.21.15.5:0/1476525533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf540b3e80 0x7fcf540cb550 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009870 tx=0x7fcf500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:34.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.313+0000 7fcf49ffb640 1 -- 172.21.15.5:0/1476525533 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50041020 con 0x7fcf540b3e80 2024-05-20T18:31:34.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.313+0000 7fcf49ffb640 1 -- 172.21.15.5:0/1476525533 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf50030970 con 0x7fcf540b3e80 2024-05-20T18:31:34.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.313+0000 7fcf5b510640 1 -- 172.21.15.5:0/1476525533 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf540ce4e0 con 0x7fcf540b3e80 2024-05-20T18:31:34.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.313+0000 7fcf49ffb640 1 -- 172.21.15.5:0/1476525533 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50037040 con 0x7fcf540b3e80 2024-05-20T18:31:34.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.314+0000 7fcf5b510640 1 -- 172.21.15.5:0/1476525533 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf540ceaa0 con 0x7fcf540b3e80 2024-05-20T18:31:34.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.315+0000 7fcf5b510640 1 -- 172.21.15.5:0/1476525533 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf54075b00 con 0x7fcf540b3e80 2024-05-20T18:31:34.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.315+0000 7fcf49ffb640 1 -- 172.21.15.5:0/1476525533 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcf50044020 con 0x7fcf540b3e80 2024-05-20T18:31:34.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.316+0000 7fcf49ffb640 1 --2- 172.21.15.5:0/1476525533 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcf3806be60 0x7fcf3806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:34.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.316+0000 7fcf49ffb640 1 -- 172.21.15.5:0/1476525533 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6101+0+0 (secure 0 0 0) 0x7fcf500abb40 con 0x7fcf540b3e80 2024-05-20T18:31:34.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.316+0000 7fcf58aab640 1 --2- 172.21.15.5:0/1476525533 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcf3806be60 0x7fcf3806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:34.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.316+0000 7fcf58aab640 1 --2- 172.21.15.5:0/1476525533 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcf3806be60 0x7fcf3806e370 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fcf44006fd0 tx=0x7fcf44008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:34.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.321+0000 7fcf49ffb640 1 -- 172.21.15.5:0/1476525533 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fcf5004b030 con 0x7fcf540b3e80 2024-05-20T18:31:34.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.504+0000 7fcf5b510640 1 -- 172.21.15.5:0/1476525533 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf540cee10 con 0x7fcf540b3e80 2024-05-20T18:31:34.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.505+0000 7fcf49ffb640 1 -- 172.21.15.5:0/1476525533 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+16825 (secure 0 0 0) 0x7fcf5007a230 con 0x7fcf540b3e80 2024-05-20T18:31:34.504 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:31:34.505 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":77,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:31:31.787691+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:31:27.896386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-20T18:31:29.146912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:31:34.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.507+0000 7fcf5b510640 1 -- 172.21.15.5:0/1476525533 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcf3806be60 msgr2=0x7fcf3806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:34.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.507+0000 7fcf5b510640 1 --2- 172.21.15.5:0/1476525533 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcf3806be60 0x7fcf3806e370 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fcf44006fd0 tx=0x7fcf44008040 comp rx=0 tx=0).stop 2024-05-20T18:31:34.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.507+0000 7fcf5b510640 1 -- 172.21.15.5:0/1476525533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf540b3e80 msgr2=0x7fcf540cb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:34.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.507+0000 7fcf5b510640 1 --2- 172.21.15.5:0/1476525533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf540b3e80 0x7fcf540cb550 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009870 tx=0x7fcf500046c0 comp rx=0 tx=0).stop 2024-05-20T18:31:34.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.507+0000 7fcf5b510640 1 -- 172.21.15.5:0/1476525533 shutdown_connections 2024-05-20T18:31:34.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.507+0000 7fcf5b510640 1 --2- 172.21.15.5:0/1476525533 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcf3806be60 0x7fcf3806e370 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:34.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.507+0000 7fcf5b510640 1 --2- 172.21.15.5:0/1476525533 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf540c8b00 0x7fcf540c8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:34.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.507+0000 7fcf5b510640 1 --2- 172.21.15.5:0/1476525533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf54162820 0x7fcf5414cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:34.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.507+0000 7fcf5b510640 1 --2- 172.21.15.5:0/1476525533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf540b3e80 0x7fcf540cb550 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:34.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.507+0000 7fcf5b510640 1 -- 172.21.15.5:0/1476525533 >> 172.21.15.5:0/1476525533 conn(0x7fcf540d4bd0 msgr2=0x7fcf541e97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:34.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.507+0000 7fcf5b510640 1 -- 172.21.15.5:0/1476525533 shutdown_connections 2024-05-20T18:31:34.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.507+0000 7fcf5b510640 1 -- 172.21.15.5:0/1476525533 wait complete. 2024-05-20T18:31:34.518 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-20T18:31:34.518 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T18:31:34.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.634+0000 7fe81ece9640 1 Processor -- start 2024-05-20T18:31:34.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.634+0000 7fe81ece9640 1 -- start start 2024-05-20T18:31:34.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.635+0000 7fe81ece9640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe818163a10 0x7fe8180d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:34.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.635+0000 7fe81ece9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8180d14b0 0x7fe8180d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:34.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.635+0000 7fe81ece9640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8180d0370 con 0x7fe8180b4010 2024-05-20T18:31:34.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.635+0000 7fe81ece9640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe8180d3ca0 con 0x7fe818163a10 2024-05-20T18:31:34.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.635+0000 7fe81ece9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe8180d0650 con 0x7fe8180d14b0 2024-05-20T18:31:34.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.636+0000 7fe81ca85640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe818163a10 0x7fe8180d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.636+0000 7fe81d286640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8180d14b0 0x7fe8180d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.636+0000 7fe81ca85640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe818163a10 0x7fe8180d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:33292/0 (socket says 172.21.15.5:33292) 2024-05-20T18:31:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.636+0000 7fe81d286640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8180d14b0 0x7fe8180d18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:37738/0 (socket says 172.21.15.5:37738) 2024-05-20T18:31:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.636+0000 7fe817fff640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fe8180b4010 0x7fe8180cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:35420/0 (socket says 172.21.15.5:35420) 2024-05-20T18:31:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.636+0000 7fe81ca85640 1 -- 172.21.15.5:0/71113071 learned_addr learned my addr 172.21.15.5:0/71113071 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.636+0000 7fe81ca85640 1 -- 172.21.15.5:0/71113071 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8180d14b0 msgr2=0x7fe8180d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.636+0000 7fe81ca85640 1 --2- 172.21.15.5:0/71113071 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8180d14b0 0x7fe8180d18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.636+0000 7fe81ca85640 1 -- 172.21.15.5:0/71113071 >> v1:172.21.15.63:6789/0 conn(0x7fe8180b4010 legacy=0x7fe8180cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.636+0000 7fe81ca85640 1 -- 172.21.15.5:0/71113071 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8180d3650 con 0x7fe818163a10 2024-05-20T18:31:34.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.637+0000 7fe81ca85640 1 --2- 172.21.15.5:0/71113071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe818163a10 0x7fe8180d4130 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009870 tx=0x7fe80c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1f85237c3ddd40e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:34.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.637+0000 7fe8177fe640 1 -- 172.21.15.5:0/71113071 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80c037070 con 0x7fe818163a10 2024-05-20T18:31:34.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.637+0000 7fe8177fe640 1 -- 172.21.15.5:0/71113071 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe80c032440 con 0x7fe818163a10 2024-05-20T18:31:34.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.637+0000 7fe8177fe640 1 -- 172.21.15.5:0/71113071 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80c032810 con 0x7fe818163a10 2024-05-20T18:31:34.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.638+0000 7fe81ece9640 1 -- 172.21.15.5:0/71113071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe818163a10 msgr2=0x7fe8180d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:34.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.638+0000 7fe81ece9640 1 --2- 172.21.15.5:0/71113071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe818163a10 0x7fe8180d4130 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009870 tx=0x7fe80c02a270 comp rx=0 tx=0).stop 2024-05-20T18:31:34.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.638+0000 7fe81ece9640 1 -- 172.21.15.5:0/71113071 shutdown_connections 2024-05-20T18:31:34.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.638+0000 7fe81ece9640 1 --2- 172.21.15.5:0/71113071 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8180d14b0 0x7fe8180d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:34.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.638+0000 7fe81ece9640 1 --2- 172.21.15.5:0/71113071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe818163a10 0x7fe8180d4130 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:34.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.638+0000 7fe81ece9640 1 -- 172.21.15.5:0/71113071 >> 172.21.15.5:0/71113071 conn(0x7fe81806d240 msgr2=0x7fe81806d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:34.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.638+0000 7fe81ece9640 1 -- 172.21.15.5:0/71113071 shutdown_connections 2024-05-20T18:31:34.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.638+0000 7fe81ece9640 1 -- 172.21.15.5:0/71113071 wait complete. 2024-05-20T18:31:34.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.640+0000 7fe81ece9640 1 Processor -- start 2024-05-20T18:31:34.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.640+0000 7fe81ece9640 1 -- start start 2024-05-20T18:31:34.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.641+0000 7fe81ece9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8180b4010 0x7fe81815b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:34.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.641+0000 7fe81ece9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8180d14b0 0x7fe818154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:34.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.641+0000 7fe81ece9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe818163a10 0x7fe81815a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:34.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.641+0000 7fe81ece9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe8180b8210 con 0x7fe8180b4010 2024-05-20T18:31:34.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.641+0000 7fe81ece9640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe8180d1e00 con 0x7fe818163a10 2024-05-20T18:31:34.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.641+0000 7fe81ece9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe8180b8390 con 0x7fe8180d14b0 2024-05-20T18:31:34.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.641+0000 7fe81ca85640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8180b4010 0x7fe81815b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:34.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.641+0000 7fe81ca85640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8180b4010 0x7fe81815b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35494/0 (socket says 172.21.15.5:35494) 2024-05-20T18:31:34.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.641+0000 7fe81ca85640 1 -- 172.21.15.5:0/1979602239 learned_addr learned my addr 172.21.15.5:0/1979602239 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:34.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.641+0000 7fe81d286640 1 --2- 172.21.15.5:0/1979602239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe818163a10 0x7fe81815a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:34.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.641+0000 7fe817fff640 1 --2- 172.21.15.5:0/1979602239 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8180d14b0 0x7fe818154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:34.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.641+0000 7fe81ca85640 1 -- 172.21.15.5:0/1979602239 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8180d14b0 msgr2=0x7fe818154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:34.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.642+0000 7fe81ca85640 1 --2- 172.21.15.5:0/1979602239 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8180d14b0 0x7fe818154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:34.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.642+0000 7fe81ca85640 1 -- 172.21.15.5:0/1979602239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe818163a10 msgr2=0x7fe81815a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:34.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.642+0000 7fe81ca85640 1 --2- 172.21.15.5:0/1979602239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe818163a10 0x7fe81815a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:34.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.642+0000 7fe81ca85640 1 -- 172.21.15.5:0/1979602239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe818160100 con 0x7fe8180b4010 2024-05-20T18:31:34.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.642+0000 7fe817fff640 1 --2- 172.21.15.5:0/1979602239 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8180d14b0 0x7fe818154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:34.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.642+0000 7fe81ca85640 1 --2- 172.21.15.5:0/1979602239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8180b4010 0x7fe81815b100 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fe80c000c00 tx=0x7fe80c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:34.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.642+0000 7fe8157fa640 1 -- 172.21.15.5:0/1979602239 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80c037040 con 0x7fe8180b4010 2024-05-20T18:31:34.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.642+0000 7fe8157fa640 1 -- 172.21.15.5:0/1979602239 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe80c042070 con 0x7fe8180b4010 2024-05-20T18:31:34.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.642+0000 7fe81ece9640 1 -- 172.21.15.5:0/1979602239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe818157420 con 0x7fe8180b4010 2024-05-20T18:31:34.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.642+0000 7fe8157fa640 1 -- 172.21.15.5:0/1979602239 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80c030880 con 0x7fe8180b4010 2024-05-20T18:31:34.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.642+0000 7fe81ece9640 1 -- 172.21.15.5:0/1979602239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8181603c0 con 0x7fe8180b4010 2024-05-20T18:31:34.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.643+0000 7fe81ece9640 1 -- 172.21.15.5:0/1979602239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe818075b00 con 0x7fe8180b4010 2024-05-20T18:31:34.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.644+0000 7fe8157fa640 1 -- 172.21.15.5:0/1979602239 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe80c030a20 con 0x7fe8180b4010 2024-05-20T18:31:34.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.644+0000 7fe8157fa640 1 --2- 172.21.15.5:0/1979602239 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe7fc06be60 0x7fe7fc06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:34.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.644+0000 7fe8157fa640 1 -- 172.21.15.5:0/1979602239 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6101+0+0 (secure 0 0 0) 0x7fe80c0ab780 con 0x7fe8180b4010 2024-05-20T18:31:34.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.644+0000 7fe817fff640 1 --2- 172.21.15.5:0/1979602239 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe7fc06be60 0x7fe7fc06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:34.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.645+0000 7fe817fff640 1 --2- 172.21.15.5:0/1979602239 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe7fc06be60 0x7fe7fc06e370 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe804007fd0 tx=0x7fe804007d80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:34.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.649+0000 7fe8157fa640 1 -- 172.21.15.5:0/1979602239 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe80c079fa0 con 0x7fe8180b4010 2024-05-20T18:31:34.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.829+0000 7fe81ece9640 1 -- 172.21.15.5:0/1979602239 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe8180d3050 con 0x7fe7fc06be60 2024-05-20T18:31:34.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.835+0000 7fe8157fa640 1 -- 172.21.15.5:0/1979602239 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252005 (secure 0 0 0) 0x7fe8180d3050 con 0x7fe7fc06be60 2024-05-20T18:31:34.838 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:31:34.839 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-20T18:31:34.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.838+0000 7fe81ece9640 1 -- 172.21.15.5:0/1979602239 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe7fc06be60 msgr2=0x7fe7fc06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:34.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.838+0000 7fe81ece9640 1 --2- 172.21.15.5:0/1979602239 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe7fc06be60 0x7fe7fc06e370 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe804007fd0 tx=0x7fe804007d80 comp rx=0 tx=0).stop 2024-05-20T18:31:34.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.838+0000 7fe81ece9640 1 -- 172.21.15.5:0/1979602239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8180b4010 msgr2=0x7fe81815b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:34.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.838+0000 7fe81ece9640 1 --2- 172.21.15.5:0/1979602239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8180b4010 0x7fe81815b100 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fe80c000c00 tx=0x7fe80c0046c0 comp rx=0 tx=0).stop 2024-05-20T18:31:34.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.838+0000 7fe81ece9640 1 -- 172.21.15.5:0/1979602239 shutdown_connections 2024-05-20T18:31:34.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.838+0000 7fe81ece9640 1 --2- 172.21.15.5:0/1979602239 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe7fc06be60 0x7fe7fc06e370 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:34.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.838+0000 7fe81ece9640 1 --2- 172.21.15.5:0/1979602239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe818163a10 0x7fe81815a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:34.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.838+0000 7fe81ece9640 1 --2- 172.21.15.5:0/1979602239 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8180d14b0 0x7fe818154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:34.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.838+0000 7fe81ece9640 1 --2- 172.21.15.5:0/1979602239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8180b4010 0x7fe81815b100 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:34.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.838+0000 7fe81ece9640 1 -- 172.21.15.5:0/1979602239 >> 172.21.15.5:0/1979602239 conn(0x7fe81806d240 msgr2=0x7fe81815ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:34.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.838+0000 7fe81ece9640 1 -- 172.21.15.5:0/1979602239 shutdown_connections 2024-05-20T18:31:34.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:34.838+0000 7fe81ece9640 1 -- 172.21.15.5:0/1979602239 wait complete. 2024-05-20T18:31:34.849 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2024-05-20T18:31:34.642249+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257852,"kb_used_data":8396,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716868,"statfs":{"total":773094113280,"available":772830072832,"internally_reserved":0,"allocated":8597504,"data_stored":4534560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.877671"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.806930+0000","last_change":"2024-05-20T18:31:30.803994+0000","last_active":"2024-05-20T18:31:30.806930+0000","last_peered":"2024-05-20T18:31:30.806930+0000","last_clean":"2024-05-20T18:31:30.806930+0000","last_became_active":"2024-05-20T18:31:30.803685+0000","last_became_peered":"2024-05-20T18:31:30.803685+0000","last_unstale":"2024-05-20T18:31:30.806930+0000","last_undegraded":"2024-05-20T18:31:30.806930+0000","last_fullsized":"2024-05-20T18:31:30.806930+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:36:09.353546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.801307+0000","last_change":"2024-05-20T18:31:30.799754+0000","last_active":"2024-05-20T18:31:30.801307+0000","last_peered":"2024-05-20T18:31:30.801307+0000","last_clean":"2024-05-20T18:31:30.801307+0000","last_became_active":"2024-05-20T18:31:30.799447+0000","last_became_peered":"2024-05-20T18:31:30.799447+0000","last_unstale":"2024-05-20T18:31:30.801307+0000","last_undegraded":"2024-05-20T18:31:30.801307+0000","last_fullsized":"2024-05-20T18:31:30.801307+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:03:06.377489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.810898+0000","last_change":"2024-05-20T18:31:30.801665+0000","last_active":"2024-05-20T18:31:30.810898+0000","last_peered":"2024-05-20T18:31:30.810898+0000","last_clean":"2024-05-20T18:31:30.810898+0000","last_became_active":"2024-05-20T18:31:30.801373+0000","last_became_peered":"2024-05-20T18:31:30.801373+0000","last_unstale":"2024-05-20T18:31:30.810898+0000","last_undegraded":"2024-05-20T18:31:30.810898+0000","last_fullsized":"2024-05-20T18:31:30.810898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:13:28.735527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.803104+0000","last_change":"2024-05-20T18:31:30.797241+0000","last_active":"2024-05-20T18:31:30.803104+0000","last_peered":"2024-05-20T18:31:30.803104+0000","last_clean":"2024-05-20T18:31:30.803104+0000","last_became_active":"2024-05-20T18:31:30.796907+0000","last_became_peered":"2024-05-20T18:31:30.796907+0000","last_unstale":"2024-05-20T18:31:30.803104+0000","last_undegraded":"2024-05-20T18:31:30.803104+0000","last_fullsized":"2024-05-20T18:31:30.803104+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:02:16.521927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.808729+0000","last_change":"2024-05-20T18:31:30.804803+0000","last_active":"2024-05-20T18:31:30.808729+0000","last_peered":"2024-05-20T18:31:30.808729+0000","last_clean":"2024-05-20T18:31:30.808729+0000","last_became_active":"2024-05-20T18:31:30.804470+0000","last_became_peered":"2024-05-20T18:31:30.804470+0000","last_unstale":"2024-05-20T18:31:30.808729+0000","last_undegraded":"2024-05-20T18:31:30.808729+0000","last_fullsized":"2024-05-20T18:31:30.808729+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:37:47.705579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.813978+0000","last_change":"2024-05-20T18:31:30.808081+0000","last_active":"2024-05-20T18:31:30.813978+0000","last_peered":"2024-05-20T18:31:30.813978+0000","last_clean":"2024-05-20T18:31:30.813978+0000","last_became_active":"2024-05-20T18:31:30.804981+0000","last_became_peered":"2024-05-20T18:31:30.804981+0000","last_unstale":"2024-05-20T18:31:30.813978+0000","last_undegraded":"2024-05-20T18:31:30.813978+0000","last_fullsized":"2024-05-20T18:31:30.813978+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:43:45.131596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.803164+0000","last_change":"2024-05-20T18:31:30.801157+0000","last_active":"2024-05-20T18:31:30.803164+0000","last_peered":"2024-05-20T18:31:30.803164+0000","last_clean":"2024-05-20T18:31:30.803164+0000","last_became_active":"2024-05-20T18:31:30.800796+0000","last_became_peered":"2024-05-20T18:31:30.800796+0000","last_unstale":"2024-05-20T18:31:30.803164+0000","last_undegraded":"2024-05-20T18:31:30.803164+0000","last_fullsized":"2024-05-20T18:31:30.803164+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:28:27.930145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.806951+0000","last_change":"2024-05-20T18:31:30.802273+0000","last_active":"2024-05-20T18:31:30.806951+0000","last_peered":"2024-05-20T18:31:30.806951+0000","last_clean":"2024-05-20T18:31:30.806951+0000","last_became_active":"2024-05-20T18:31:30.801960+0000","last_became_peered":"2024-05-20T18:31:30.801960+0000","last_unstale":"2024-05-20T18:31:30.806951+0000","last_undegraded":"2024-05-20T18:31:30.806951+0000","last_fullsized":"2024-05-20T18:31:30.806951+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:11:25.301998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.809043+0000","last_change":"2024-05-20T18:31:30.807811+0000","last_active":"2024-05-20T18:31:30.809043+0000","last_peered":"2024-05-20T18:31:30.809043+0000","last_clean":"2024-05-20T18:31:30.809043+0000","last_became_active":"2024-05-20T18:31:30.807469+0000","last_became_peered":"2024-05-20T18:31:30.807469+0000","last_unstale":"2024-05-20T18:31:30.809043+0000","last_undegraded":"2024-05-20T18:31:30.809043+0000","last_fullsized":"2024-05-20T18:31:30.809043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:10:59.195942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.806179+0000","last_change":"2024-05-20T18:31:30.799274+0000","last_active":"2024-05-20T18:31:30.806179+0000","last_peered":"2024-05-20T18:31:30.806179+0000","last_clean":"2024-05-20T18:31:30.806179+0000","last_became_active":"2024-05-20T18:31:30.798746+0000","last_became_peered":"2024-05-20T18:31:30.798746+0000","last_unstale":"2024-05-20T18:31:30.806179+0000","last_undegraded":"2024-05-20T18:31:30.806179+0000","last_fullsized":"2024-05-20T18:31:30.806179+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:39:38.803158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.814497+0000","last_change":"2024-05-20T18:31:30.804852+0000","last_active":"2024-05-20T18:31:30.814497+0000","last_peered":"2024-05-20T18:31:30.814497+0000","last_clean":"2024-05-20T18:31:30.814497+0000","last_became_active":"2024-05-20T18:31:30.804486+0000","last_became_peered":"2024-05-20T18:31:30.804486+0000","last_unstale":"2024-05-20T18:31:30.814497+0000","last_undegraded":"2024-05-20T18:31:30.814497+0000","last_fullsized":"2024-05-20T18:31:30.814497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:38:41.987602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.803073+0000","last_change":"2024-05-20T18:31:30.795408+0000","last_active":"2024-05-20T18:31:30.803073+0000","last_peered":"2024-05-20T18:31:30.803073+0000","last_clean":"2024-05-20T18:31:30.803073+0000","last_became_active":"2024-05-20T18:31:30.795104+0000","last_became_peered":"2024-05-20T18:31:30.795104+0000","last_unstale":"2024-05-20T18:31:30.803073+0000","last_undegraded":"2024-05-20T18:31:30.803073+0000","last_fullsized":"2024-05-20T18:31:30.803073+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:35:59.575287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.815767+0000","last_change":"2024-05-20T18:31:30.813213+0000","last_active":"2024-05-20T18:31:30.815767+0000","last_peered":"2024-05-20T18:31:30.815767+0000","last_clean":"2024-05-20T18:31:30.815767+0000","last_became_active":"2024-05-20T18:31:30.812894+0000","last_became_peered":"2024-05-20T18:31:30.812894+0000","last_unstale":"2024-05-20T18:31:30.815767+0000","last_undegraded":"2024-05-20T18:31:30.815767+0000","last_fullsized":"2024-05-20T18:31:30.815767+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:18:23.823402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.807928+0000","last_change":"2024-05-20T18:31:30.804426+0000","last_active":"2024-05-20T18:31:30.807928+0000","last_peered":"2024-05-20T18:31:30.807928+0000","last_clean":"2024-05-20T18:31:30.807928+0000","last_became_active":"2024-05-20T18:31:30.804020+0000","last_became_peered":"2024-05-20T18:31:30.804020+0000","last_unstale":"2024-05-20T18:31:30.807928+0000","last_undegraded":"2024-05-20T18:31:30.807928+0000","last_fullsized":"2024-05-20T18:31:30.807928+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:31:45.080109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.814228+0000","last_change":"2024-05-20T18:31:30.800832+0000","last_active":"2024-05-20T18:31:30.814228+0000","last_peered":"2024-05-20T18:31:30.814228+0000","last_clean":"2024-05-20T18:31:30.814228+0000","last_became_active":"2024-05-20T18:31:30.800468+0000","last_became_peered":"2024-05-20T18:31:30.800468+0000","last_unstale":"2024-05-20T18:31:30.814228+0000","last_undegraded":"2024-05-20T18:31:30.814228+0000","last_fullsized":"2024-05-20T18:31:30.814228+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:30:09.481775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.804348+0000","last_change":"2024-05-20T18:31:30.800500+0000","last_active":"2024-05-20T18:31:30.804348+0000","last_peered":"2024-05-20T18:31:30.804348+0000","last_clean":"2024-05-20T18:31:30.804348+0000","last_became_active":"2024-05-20T18:31:30.799974+0000","last_became_peered":"2024-05-20T18:31:30.799974+0000","last_unstale":"2024-05-20T18:31:30.804348+0000","last_undegraded":"2024-05-20T18:31:30.804348+0000","last_fullsized":"2024-05-20T18:31:30.804348+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:34:43.715283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.806990+0000","last_change":"2024-05-20T18:31:30.799031+0000","last_active":"2024-05-20T18:31:30.806990+0000","last_peered":"2024-05-20T18:31:30.806990+0000","last_clean":"2024-05-20T18:31:30.806990+0000","last_became_active":"2024-05-20T18:31:30.798559+0000","last_became_peered":"2024-05-20T18:31:30.798559+0000","last_unstale":"2024-05-20T18:31:30.806990+0000","last_undegraded":"2024-05-20T18:31:30.806990+0000","last_fullsized":"2024-05-20T18:31:30.806990+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:46:00.326307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.810408+0000","last_change":"2024-05-20T18:31:30.799918+0000","last_active":"2024-05-20T18:31:30.810408+0000","last_peered":"2024-05-20T18:31:30.810408+0000","last_clean":"2024-05-20T18:31:30.810408+0000","last_became_active":"2024-05-20T18:31:30.799596+0000","last_became_peered":"2024-05-20T18:31:30.799596+0000","last_unstale":"2024-05-20T18:31:30.810408+0000","last_undegraded":"2024-05-20T18:31:30.810408+0000","last_fullsized":"2024-05-20T18:31:30.810408+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:18:35.214658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.805643+0000","last_change":"2024-05-20T18:31:30.803385+0000","last_active":"2024-05-20T18:31:30.805643+0000","last_peered":"2024-05-20T18:31:30.805643+0000","last_clean":"2024-05-20T18:31:30.805643+0000","last_became_active":"2024-05-20T18:31:30.802737+0000","last_became_peered":"2024-05-20T18:31:30.802737+0000","last_unstale":"2024-05-20T18:31:30.805643+0000","last_undegraded":"2024-05-20T18:31:30.805643+0000","last_fullsized":"2024-05-20T18:31:30.805643+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:10:22.671462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.819226+0000","last_change":"2024-05-20T18:31:30.816537+0000","last_active":"2024-05-20T18:31:30.819226+0000","last_peered":"2024-05-20T18:31:30.819226+0000","last_clean":"2024-05-20T18:31:30.819226+0000","last_became_active":"2024-05-20T18:31:30.816226+0000","last_became_peered":"2024-05-20T18:31:30.816226+0000","last_unstale":"2024-05-20T18:31:30.819226+0000","last_undegraded":"2024-05-20T18:31:30.819226+0000","last_fullsized":"2024-05-20T18:31:30.819226+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:52:56.818967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.808815+0000","last_change":"2024-05-20T18:31:30.805340+0000","last_active":"2024-05-20T18:31:30.808815+0000","last_peered":"2024-05-20T18:31:30.808815+0000","last_clean":"2024-05-20T18:31:30.808815+0000","last_became_active":"2024-05-20T18:31:30.804968+0000","last_became_peered":"2024-05-20T18:31:30.804968+0000","last_unstale":"2024-05-20T18:31:30.808815+0000","last_undegraded":"2024-05-20T18:31:30.808815+0000","last_fullsized":"2024-05-20T18:31:30.808815+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:23:02.785905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.818892+0000","last_change":"2024-05-20T18:31:30.815412+0000","last_active":"2024-05-20T18:31:30.818892+0000","last_peered":"2024-05-20T18:31:30.818892+0000","last_clean":"2024-05-20T18:31:30.818892+0000","last_became_active":"2024-05-20T18:31:30.815118+0000","last_became_peered":"2024-05-20T18:31:30.815118+0000","last_unstale":"2024-05-20T18:31:30.818892+0000","last_undegraded":"2024-05-20T18:31:30.818892+0000","last_fullsized":"2024-05-20T18:31:30.818892+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:07:34.573709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.799244+0000","last_change":"2024-05-20T18:31:30.793306+0000","last_active":"2024-05-20T18:31:30.799244+0000","last_peered":"2024-05-20T18:31:30.799244+0000","last_clean":"2024-05-20T18:31:30.799244+0000","last_became_active":"2024-05-20T18:31:30.792967+0000","last_became_peered":"2024-05-20T18:31:30.792967+0000","last_unstale":"2024-05-20T18:31:30.799244+0000","last_undegraded":"2024-05-20T18:31:30.799244+0000","last_fullsized":"2024-05-20T18:31:30.799244+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:38:42.055506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.814218+0000","last_change":"2024-05-20T18:31:30.808953+0000","last_active":"2024-05-20T18:31:30.814218+0000","last_peered":"2024-05-20T18:31:30.814218+0000","last_clean":"2024-05-20T18:31:30.814218+0000","last_became_active":"2024-05-20T18:31:30.808638+0000","last_became_peered":"2024-05-20T18:31:30.808638+0000","last_unstale":"2024-05-20T18:31:30.814218+0000","last_undegraded":"2024-05-20T18:31:30.814218+0000","last_fullsized":"2024-05-20T18:31:30.814218+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:56:43.205875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.814263+0000","last_change":"2024-05-20T18:31:30.809078+0000","last_active":"2024-05-20T18:31:30.814263+0000","last_peered":"2024-05-20T18:31:30.814263+0000","last_clean":"2024-05-20T18:31:30.814263+0000","last_became_active":"2024-05-20T18:31:30.808679+0000","last_became_peered":"2024-05-20T18:31:30.808679+0000","last_unstale":"2024-05-20T18:31:30.814263+0000","last_undegraded":"2024-05-20T18:31:30.814263+0000","last_fullsized":"2024-05-20T18:31:30.814263+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:33:51.968504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.803106+0000","last_change":"2024-05-20T18:31:30.796531+0000","last_active":"2024-05-20T18:31:30.803106+0000","last_peered":"2024-05-20T18:31:30.803106+0000","last_clean":"2024-05-20T18:31:30.803106+0000","last_became_active":"2024-05-20T18:31:30.796199+0000","last_became_peered":"2024-05-20T18:31:30.796199+0000","last_unstale":"2024-05-20T18:31:30.803106+0000","last_undegraded":"2024-05-20T18:31:30.803106+0000","last_fullsized":"2024-05-20T18:31:30.803106+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:05:22.123725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.804282+0000","last_change":"2024-05-20T18:31:30.801409+0000","last_active":"2024-05-20T18:31:30.804282+0000","last_peered":"2024-05-20T18:31:30.804282+0000","last_clean":"2024-05-20T18:31:30.804282+0000","last_became_active":"2024-05-20T18:31:30.800968+0000","last_became_peered":"2024-05-20T18:31:30.800968+0000","last_unstale":"2024-05-20T18:31:30.804282+0000","last_undegraded":"2024-05-20T18:31:30.804282+0000","last_fullsized":"2024-05-20T18:31:30.804282+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:30:09.481775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.798564+0000","last_change":"2024-05-20T18:31:30.796685+0000","last_active":"2024-05-20T18:31:30.798564+0000","last_peered":"2024-05-20T18:31:30.798564+0000","last_clean":"2024-05-20T18:31:30.798564+0000","last_became_active":"2024-05-20T18:31:30.796247+0000","last_became_peered":"2024-05-20T18:31:30.796247+0000","last_unstale":"2024-05-20T18:31:30.798564+0000","last_undegraded":"2024-05-20T18:31:30.798564+0000","last_fullsized":"2024-05-20T18:31:30.798564+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:14:13.421305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.792726+0000","last_change":"2024-05-20T18:31:30.788824+0000","last_active":"2024-05-20T18:31:30.792726+0000","last_peered":"2024-05-20T18:31:30.792726+0000","last_clean":"2024-05-20T18:31:30.792726+0000","last_became_active":"2024-05-20T18:31:30.788481+0000","last_became_peered":"2024-05-20T18:31:30.788481+0000","last_unstale":"2024-05-20T18:31:30.792726+0000","last_undegraded":"2024-05-20T18:31:30.792726+0000","last_fullsized":"2024-05-20T18:31:30.792726+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:49:05.583126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.806899+0000","last_change":"2024-05-20T18:31:30.803916+0000","last_active":"2024-05-20T18:31:30.806899+0000","last_peered":"2024-05-20T18:31:30.806899+0000","last_clean":"2024-05-20T18:31:30.806899+0000","last_became_active":"2024-05-20T18:31:30.802543+0000","last_became_peered":"2024-05-20T18:31:30.802543+0000","last_unstale":"2024-05-20T18:31:30.806899+0000","last_undegraded":"2024-05-20T18:31:30.806899+0000","last_fullsized":"2024-05-20T18:31:30.806899+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:16:42.686921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.810740+0000","last_change":"2024-05-20T18:31:30.804864+0000","last_active":"2024-05-20T18:31:30.810740+0000","last_peered":"2024-05-20T18:31:30.810740+0000","last_clean":"2024-05-20T18:31:30.810740+0000","last_became_active":"2024-05-20T18:31:30.804513+0000","last_became_peered":"2024-05-20T18:31:30.804513+0000","last_unstale":"2024-05-20T18:31:30.810740+0000","last_undegraded":"2024-05-20T18:31:30.810740+0000","last_fullsized":"2024-05-20T18:31:30.810740+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:40:02.269716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.804370+0000","last_change":"2024-05-20T18:31:30.800518+0000","last_active":"2024-05-20T18:31:30.804370+0000","last_peered":"2024-05-20T18:31:30.804370+0000","last_clean":"2024-05-20T18:31:30.804370+0000","last_became_active":"2024-05-20T18:31:30.799574+0000","last_became_peered":"2024-05-20T18:31:30.799574+0000","last_unstale":"2024-05-20T18:31:30.804370+0000","last_undegraded":"2024-05-20T18:31:30.804370+0000","last_fullsized":"2024-05-20T18:31:30.804370+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:03:06.377489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.808291+0000","last_change":"2024-05-20T18:31:30.797496+0000","last_active":"2024-05-20T18:31:30.808291+0000","last_peered":"2024-05-20T18:31:30.808291+0000","last_clean":"2024-05-20T18:31:30.808291+0000","last_became_active":"2024-05-20T18:31:30.797187+0000","last_became_peered":"2024-05-20T18:31:30.797187+0000","last_unstale":"2024-05-20T18:31:30.808291+0000","last_undegraded":"2024-05-20T18:31:30.808291+0000","last_fullsized":"2024-05-20T18:31:30.808291+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:26:34.676034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"76'4","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.819554+0000","last_change":"2024-05-20T18:31:29.789722+0000","last_active":"2024-05-20T18:31:30.819554+0000","last_peered":"2024-05-20T18:31:30.819554+0000","last_clean":"2024-05-20T18:31:30.819554+0000","last_became_active":"2024-05-20T18:31:29.789393+0000","last_became_peered":"2024-05-20T18:31:29.789393+0000","last_unstale":"2024-05-20T18:31:30.819554+0000","last_undegraded":"2024-05-20T18:31:30.819554+0000","last_fullsized":"2024-05-20T18:31:30.819554+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:52:54.906881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.797778+0000","last_change":"2024-05-20T18:31:30.794764+0000","last_active":"2024-05-20T18:31:30.797778+0000","last_peered":"2024-05-20T18:31:30.797778+0000","last_clean":"2024-05-20T18:31:30.797778+0000","last_became_active":"2024-05-20T18:31:30.794455+0000","last_became_peered":"2024-05-20T18:31:30.794455+0000","last_unstale":"2024-05-20T18:31:30.797778+0000","last_undegraded":"2024-05-20T18:31:30.797778+0000","last_fullsized":"2024-05-20T18:31:30.797778+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:06:08.211253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"76'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.810984+0000","last_change":"2024-05-20T18:31:29.785992+0000","last_active":"2024-05-20T18:31:30.810984+0000","last_peered":"2024-05-20T18:31:30.810984+0000","last_clean":"2024-05-20T18:31:30.810984+0000","last_became_active":"2024-05-20T18:31:29.785657+0000","last_became_peered":"2024-05-20T18:31:29.785657+0000","last_unstale":"2024-05-20T18:31:30.810984+0000","last_undegraded":"2024-05-20T18:31:30.810984+0000","last_fullsized":"2024-05-20T18:31:30.810984+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:39:37.796995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.810990+0000","last_change":"2024-05-20T18:31:30.808624+0000","last_active":"2024-05-20T18:31:30.810990+0000","last_peered":"2024-05-20T18:31:30.810990+0000","last_clean":"2024-05-20T18:31:30.810990+0000","last_became_active":"2024-05-20T18:31:30.808299+0000","last_became_peered":"2024-05-20T18:31:30.808299+0000","last_unstale":"2024-05-20T18:31:30.810990+0000","last_undegraded":"2024-05-20T18:31:30.810990+0000","last_fullsized":"2024-05-20T18:31:30.810990+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:05:46.443301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"76'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.814892+0000","last_change":"2024-05-20T18:31:29.789603+0000","last_active":"2024-05-20T18:31:30.814892+0000","last_peered":"2024-05-20T18:31:30.814892+0000","last_clean":"2024-05-20T18:31:30.814892+0000","last_became_active":"2024-05-20T18:31:29.789259+0000","last_became_peered":"2024-05-20T18:31:29.789259+0000","last_unstale":"2024-05-20T18:31:30.814892+0000","last_undegraded":"2024-05-20T18:31:30.814892+0000","last_fullsized":"2024-05-20T18:31:30.814892+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:05:21.117562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.813288+0000","last_change":"2024-05-20T18:31:30.810923+0000","last_active":"2024-05-20T18:31:30.813288+0000","last_peered":"2024-05-20T18:31:30.813288+0000","last_clean":"2024-05-20T18:31:30.813288+0000","last_became_active":"2024-05-20T18:31:30.810553+0000","last_became_peered":"2024-05-20T18:31:30.810553+0000","last_unstale":"2024-05-20T18:31:30.813288+0000","last_undegraded":"2024-05-20T18:31:30.813288+0000","last_fullsized":"2024-05-20T18:31:30.813288+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:12:17.272988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"76'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.813821+0000","last_change":"2024-05-20T18:31:29.788526+0000","last_active":"2024-05-20T18:31:30.813821+0000","last_peered":"2024-05-20T18:31:30.813821+0000","last_clean":"2024-05-20T18:31:30.813821+0000","last_became_active":"2024-05-20T18:31:29.788171+0000","last_became_peered":"2024-05-20T18:31:29.788171+0000","last_unstale":"2024-05-20T18:31:30.813821+0000","last_undegraded":"2024-05-20T18:31:30.813821+0000","last_fullsized":"2024-05-20T18:31:30.813821+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:48:31.385853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.799934+0000","last_change":"2024-05-20T18:31:30.798539+0000","last_active":"2024-05-20T18:31:30.799934+0000","last_peered":"2024-05-20T18:31:30.799934+0000","last_clean":"2024-05-20T18:31:30.799934+0000","last_became_active":"2024-05-20T18:31:30.798211+0000","last_became_peered":"2024-05-20T18:31:30.798211+0000","last_unstale":"2024-05-20T18:31:30.799934+0000","last_undegraded":"2024-05-20T18:31:30.799934+0000","last_fullsized":"2024-05-20T18:31:30.799934+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:34:43.715283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"76'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.812985+0000","last_change":"2024-05-20T18:31:29.784698+0000","last_active":"2024-05-20T18:31:30.812985+0000","last_peered":"2024-05-20T18:31:30.812985+0000","last_clean":"2024-05-20T18:31:30.812985+0000","last_became_active":"2024-05-20T18:31:29.784363+0000","last_became_peered":"2024-05-20T18:31:29.784363+0000","last_unstale":"2024-05-20T18:31:30.812985+0000","last_undegraded":"2024-05-20T18:31:30.812985+0000","last_fullsized":"2024-05-20T18:31:30.812985+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:45:50.185805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.814470+0000","last_change":"2024-05-20T18:31:30.803690+0000","last_active":"2024-05-20T18:31:30.814470+0000","last_peered":"2024-05-20T18:31:30.814470+0000","last_clean":"2024-05-20T18:31:30.814470+0000","last_became_active":"2024-05-20T18:31:30.803385+0000","last_became_peered":"2024-05-20T18:31:30.803385+0000","last_unstale":"2024-05-20T18:31:30.814470+0000","last_undegraded":"2024-05-20T18:31:30.814470+0000","last_fullsized":"2024-05-20T18:31:30.814470+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:36:09.353546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"76'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.814290+0000","last_change":"2024-05-20T18:31:29.789808+0000","last_active":"2024-05-20T18:31:30.814290+0000","last_peered":"2024-05-20T18:31:30.814290+0000","last_clean":"2024-05-20T18:31:30.814290+0000","last_became_active":"2024-05-20T18:31:29.789454+0000","last_became_peered":"2024-05-20T18:31:29.789454+0000","last_unstale":"2024-05-20T18:31:30.814290+0000","last_undegraded":"2024-05-20T18:31:30.814290+0000","last_fullsized":"2024-05-20T18:31:30.814290+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:28:46.277507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.811172+0000","last_change":"2024-05-20T18:31:30.808285+0000","last_active":"2024-05-20T18:31:30.811172+0000","last_peered":"2024-05-20T18:31:30.811172+0000","last_clean":"2024-05-20T18:31:30.811172+0000","last_became_active":"2024-05-20T18:31:30.807793+0000","last_became_peered":"2024-05-20T18:31:30.807793+0000","last_unstale":"2024-05-20T18:31:30.811172+0000","last_undegraded":"2024-05-20T18:31:30.811172+0000","last_fullsized":"2024-05-20T18:31:30.811172+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:57:46.710550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.781343+0000","last_change":"2024-05-20T18:31:29.795340+0000","last_active":"2024-05-20T18:31:30.781343+0000","last_peered":"2024-05-20T18:31:30.781343+0000","last_clean":"2024-05-20T18:31:30.781343+0000","last_became_active":"2024-05-20T18:31:29.795020+0000","last_became_peered":"2024-05-20T18:31:29.795020+0000","last_unstale":"2024-05-20T18:31:30.781343+0000","last_undegraded":"2024-05-20T18:31:30.781343+0000","last_fullsized":"2024-05-20T18:31:30.781343+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:09:35.144930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.803496+0000","last_change":"2024-05-20T18:31:30.801956+0000","last_active":"2024-05-20T18:31:30.803496+0000","last_peered":"2024-05-20T18:31:30.803496+0000","last_clean":"2024-05-20T18:31:30.803496+0000","last_became_active":"2024-05-20T18:31:30.801649+0000","last_became_peered":"2024-05-20T18:31:30.801649+0000","last_unstale":"2024-05-20T18:31:30.803496+0000","last_undegraded":"2024-05-20T18:31:30.803496+0000","last_fullsized":"2024-05-20T18:31:30.803496+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:18:35.214658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.787618+0000","last_change":"2024-05-20T18:31:29.791538+0000","last_active":"2024-05-20T18:31:30.787618+0000","last_peered":"2024-05-20T18:31:30.787618+0000","last_clean":"2024-05-20T18:31:30.787618+0000","last_became_active":"2024-05-20T18:31:29.791228+0000","last_became_peered":"2024-05-20T18:31:29.791228+0000","last_unstale":"2024-05-20T18:31:30.787618+0000","last_undegraded":"2024-05-20T18:31:30.787618+0000","last_fullsized":"2024-05-20T18:31:30.787618+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:18:04.510834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.796899+0000","last_change":"2024-05-20T18:31:30.794339+0000","last_active":"2024-05-20T18:31:30.796899+0000","last_peered":"2024-05-20T18:31:30.796899+0000","last_clean":"2024-05-20T18:31:30.796899+0000","last_became_active":"2024-05-20T18:31:30.794018+0000","last_became_peered":"2024-05-20T18:31:30.794018+0000","last_unstale":"2024-05-20T18:31:30.796899+0000","last_undegraded":"2024-05-20T18:31:30.796899+0000","last_fullsized":"2024-05-20T18:31:30.796899+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:39:38.803158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.795526+0000","last_change":"2024-05-20T18:31:29.795246+0000","last_active":"2024-05-20T18:31:30.795526+0000","last_peered":"2024-05-20T18:31:30.795526+0000","last_clean":"2024-05-20T18:31:30.795526+0000","last_became_active":"2024-05-20T18:31:29.794822+0000","last_became_peered":"2024-05-20T18:31:29.794822+0000","last_unstale":"2024-05-20T18:31:30.795526+0000","last_undegraded":"2024-05-20T18:31:30.795526+0000","last_fullsized":"2024-05-20T18:31:30.795526+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:06:07.205090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.803145+0000","last_change":"2024-05-20T18:31:30.796802+0000","last_active":"2024-05-20T18:31:30.803145+0000","last_peered":"2024-05-20T18:31:30.803145+0000","last_clean":"2024-05-20T18:31:30.803145+0000","last_became_active":"2024-05-20T18:31:30.796451+0000","last_became_peered":"2024-05-20T18:31:30.796451+0000","last_unstale":"2024-05-20T18:31:30.803145+0000","last_undegraded":"2024-05-20T18:31:30.803145+0000","last_fullsized":"2024-05-20T18:31:30.803145+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:49:05.583126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.790571+0000","last_change":"2024-05-20T18:31:29.792224+0000","last_active":"2024-05-20T18:31:30.790571+0000","last_peered":"2024-05-20T18:31:30.790571+0000","last_clean":"2024-05-20T18:31:30.790571+0000","last_became_active":"2024-05-20T18:31:29.791891+0000","last_became_peered":"2024-05-20T18:31:29.791891+0000","last_unstale":"2024-05-20T18:31:30.790571+0000","last_undegraded":"2024-05-20T18:31:30.790571+0000","last_fullsized":"2024-05-20T18:31:30.790571+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:09:35.144930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.809300+0000","last_change":"2024-05-20T18:31:30.807512+0000","last_active":"2024-05-20T18:31:30.809300+0000","last_peered":"2024-05-20T18:31:30.809300+0000","last_clean":"2024-05-20T18:31:30.809300+0000","last_became_active":"2024-05-20T18:31:30.807194+0000","last_became_peered":"2024-05-20T18:31:30.807194+0000","last_unstale":"2024-05-20T18:31:30.809300+0000","last_undegraded":"2024-05-20T18:31:30.809300+0000","last_fullsized":"2024-05-20T18:31:30.809300+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:38:20.581222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"76'3","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.819919+0000","last_change":"2024-05-20T18:31:29.785846+0000","last_active":"2024-05-20T18:31:30.819919+0000","last_peered":"2024-05-20T18:31:30.819919+0000","last_clean":"2024-05-20T18:31:30.819919+0000","last_became_active":"2024-05-20T18:31:29.785526+0000","last_became_peered":"2024-05-20T18:31:29.785526+0000","last_unstale":"2024-05-20T18:31:30.819919+0000","last_undegraded":"2024-05-20T18:31:30.819919+0000","last_fullsized":"2024-05-20T18:31:30.819919+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:49:08.327229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.810691+0000","last_change":"2024-05-20T18:31:30.802361+0000","last_active":"2024-05-20T18:31:30.810691+0000","last_peered":"2024-05-20T18:31:30.810691+0000","last_clean":"2024-05-20T18:31:30.810691+0000","last_became_active":"2024-05-20T18:31:30.802022+0000","last_became_peered":"2024-05-20T18:31:30.802022+0000","last_unstale":"2024-05-20T18:31:30.810691+0000","last_undegraded":"2024-05-20T18:31:30.810691+0000","last_fullsized":"2024-05-20T18:31:30.810691+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:04:32.739946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"76'6","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.821504+0000","last_change":"2024-05-20T18:31:29.785804+0000","last_active":"2024-05-20T18:31:30.821504+0000","last_peered":"2024-05-20T18:31:30.821504+0000","last_clean":"2024-05-20T18:31:30.821504+0000","last_became_active":"2024-05-20T18:31:29.785452+0000","last_became_peered":"2024-05-20T18:31:29.785452+0000","last_unstale":"2024-05-20T18:31:30.821504+0000","last_undegraded":"2024-05-20T18:31:30.821504+0000","last_fullsized":"2024-05-20T18:31:30.821504+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:05:45.437138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.808856+0000","last_change":"2024-05-20T18:31:30.805529+0000","last_active":"2024-05-20T18:31:30.808856+0000","last_peered":"2024-05-20T18:31:30.808856+0000","last_clean":"2024-05-20T18:31:30.808856+0000","last_became_active":"2024-05-20T18:31:30.805197+0000","last_became_peered":"2024-05-20T18:31:30.805197+0000","last_unstale":"2024-05-20T18:31:30.808856+0000","last_undegraded":"2024-05-20T18:31:30.808856+0000","last_fullsized":"2024-05-20T18:31:30.808856+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:06:08.211253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"76'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.813615+0000","last_change":"2024-05-20T18:31:29.785826+0000","last_active":"2024-05-20T18:31:30.813615+0000","last_peered":"2024-05-20T18:31:30.813615+0000","last_clean":"2024-05-20T18:31:30.813615+0000","last_became_active":"2024-05-20T18:31:29.785475+0000","last_became_peered":"2024-05-20T18:31:29.785475+0000","last_unstale":"2024-05-20T18:31:30.813615+0000","last_undegraded":"2024-05-20T18:31:30.813615+0000","last_fullsized":"2024-05-20T18:31:30.813615+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:47:45.282380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.812154+0000","last_change":"2024-05-20T18:31:30.807788+0000","last_active":"2024-05-20T18:31:30.812154+0000","last_peered":"2024-05-20T18:31:30.812154+0000","last_clean":"2024-05-20T18:31:30.812154+0000","last_became_active":"2024-05-20T18:31:30.807478+0000","last_became_peered":"2024-05-20T18:31:30.807478+0000","last_unstale":"2024-05-20T18:31:30.812154+0000","last_undegraded":"2024-05-20T18:31:30.812154+0000","last_fullsized":"2024-05-20T18:31:30.812154+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:29:12.378846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.793841+0000","last_change":"2024-05-20T18:31:29.787484+0000","last_active":"2024-05-20T18:31:30.793841+0000","last_peered":"2024-05-20T18:31:30.793841+0000","last_clean":"2024-05-20T18:31:30.793841+0000","last_became_active":"2024-05-20T18:31:29.787019+0000","last_became_peered":"2024-05-20T18:31:29.787019+0000","last_unstale":"2024-05-20T18:31:30.793841+0000","last_undegraded":"2024-05-20T18:31:30.793841+0000","last_fullsized":"2024-05-20T18:31:30.793841+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:49:04.576963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.802373+0000","last_change":"2024-05-20T18:31:30.798213+0000","last_active":"2024-05-20T18:31:30.802373+0000","last_peered":"2024-05-20T18:31:30.802373+0000","last_clean":"2024-05-20T18:31:30.802373+0000","last_became_active":"2024-05-20T18:31:30.797842+0000","last_became_peered":"2024-05-20T18:31:30.797842+0000","last_unstale":"2024-05-20T18:31:30.802373+0000","last_undegraded":"2024-05-20T18:31:30.802373+0000","last_fullsized":"2024-05-20T18:31:30.802373+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:26:34.676034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.788467+0000","last_change":"2024-05-20T18:31:29.789763+0000","last_active":"2024-05-20T18:31:30.788467+0000","last_peered":"2024-05-20T18:31:30.788467+0000","last_clean":"2024-05-20T18:31:30.788467+0000","last_became_active":"2024-05-20T18:31:29.789425+0000","last_became_peered":"2024-05-20T18:31:29.789425+0000","last_unstale":"2024-05-20T18:31:30.788467+0000","last_undegraded":"2024-05-20T18:31:30.788467+0000","last_fullsized":"2024-05-20T18:31:30.788467+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:08:40.351642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.809880+0000","last_change":"2024-05-20T18:31:30.805626+0000","last_active":"2024-05-20T18:31:30.809880+0000","last_peered":"2024-05-20T18:31:30.809880+0000","last_clean":"2024-05-20T18:31:30.809880+0000","last_became_active":"2024-05-20T18:31:30.805260+0000","last_became_peered":"2024-05-20T18:31:30.805260+0000","last_unstale":"2024-05-20T18:31:30.809880+0000","last_undegraded":"2024-05-20T18:31:30.809880+0000","last_fullsized":"2024-05-20T18:31:30.809880+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:04:32.739946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"76'8","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.817404+0000","last_change":"2024-05-20T18:31:29.797682+0000","last_active":"2024-05-20T18:31:30.817404+0000","last_peered":"2024-05-20T18:31:30.817404+0000","last_clean":"2024-05-20T18:31:30.817404+0000","last_became_active":"2024-05-20T18:31:29.797386+0000","last_became_peered":"2024-05-20T18:31:29.797386+0000","last_unstale":"2024-05-20T18:31:30.817404+0000","last_undegraded":"2024-05-20T18:31:30.817404+0000","last_fullsized":"2024-05-20T18:31:30.817404+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:05:21.117562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.805607+0000","last_change":"2024-05-20T18:31:30.802154+0000","last_active":"2024-05-20T18:31:30.805607+0000","last_peered":"2024-05-20T18:31:30.805607+0000","last_clean":"2024-05-20T18:31:30.805607+0000","last_became_active":"2024-05-20T18:31:30.801704+0000","last_became_peered":"2024-05-20T18:31:30.801704+0000","last_unstale":"2024-05-20T18:31:30.805607+0000","last_undegraded":"2024-05-20T18:31:30.805607+0000","last_fullsized":"2024-05-20T18:31:30.805607+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:20:13.702810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"76'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.813089+0000","last_change":"2024-05-20T18:31:29.787007+0000","last_active":"2024-05-20T18:31:30.813089+0000","last_peered":"2024-05-20T18:31:30.813089+0000","last_clean":"2024-05-20T18:31:30.813089+0000","last_became_active":"2024-05-20T18:31:29.786624+0000","last_became_peered":"2024-05-20T18:31:29.786624+0000","last_unstale":"2024-05-20T18:31:30.813089+0000","last_undegraded":"2024-05-20T18:31:30.813089+0000","last_fullsized":"2024-05-20T18:31:30.813089+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:43:03.024863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.806889+0000","last_change":"2024-05-20T18:31:30.799848+0000","last_active":"2024-05-20T18:31:30.806889+0000","last_peered":"2024-05-20T18:31:30.806889+0000","last_clean":"2024-05-20T18:31:30.806889+0000","last_became_active":"2024-05-20T18:31:30.799548+0000","last_became_peered":"2024-05-20T18:31:30.799548+0000","last_unstale":"2024-05-20T18:31:30.806889+0000","last_undegraded":"2024-05-20T18:31:30.806889+0000","last_fullsized":"2024-05-20T18:31:30.806889+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:52:17.996601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"76'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.817886+0000","last_change":"2024-05-20T18:31:29.782077+0000","last_active":"2024-05-20T18:31:30.817886+0000","last_peered":"2024-05-20T18:31:30.817886+0000","last_clean":"2024-05-20T18:31:30.817886+0000","last_became_active":"2024-05-20T18:31:29.781723+0000","last_became_peered":"2024-05-20T18:31:29.781723+0000","last_unstale":"2024-05-20T18:31:30.817886+0000","last_undegraded":"2024-05-20T18:31:30.817886+0000","last_fullsized":"2024-05-20T18:31:30.817886+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:17:14.900208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.804795+0000","last_change":"2024-05-20T18:31:30.798880+0000","last_active":"2024-05-20T18:31:30.804795+0000","last_peered":"2024-05-20T18:31:30.804795+0000","last_clean":"2024-05-20T18:31:30.804795+0000","last_became_active":"2024-05-20T18:31:30.798532+0000","last_became_peered":"2024-05-20T18:31:30.798532+0000","last_unstale":"2024-05-20T18:31:30.804795+0000","last_undegraded":"2024-05-20T18:31:30.804795+0000","last_fullsized":"2024-05-20T18:31:30.804795+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:09:02.959983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"76'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.818585+0000","last_change":"2024-05-20T18:31:29.792065+0000","last_active":"2024-05-20T18:31:30.818585+0000","last_peered":"2024-05-20T18:31:30.818585+0000","last_clean":"2024-05-20T18:31:30.818585+0000","last_became_active":"2024-05-20T18:31:29.791740+0000","last_became_peered":"2024-05-20T18:31:29.791740+0000","last_unstale":"2024-05-20T18:31:30.818585+0000","last_undegraded":"2024-05-20T18:31:30.818585+0000","last_fullsized":"2024-05-20T18:31:30.818585+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:50:33.412770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.805679+0000","last_change":"2024-05-20T18:31:30.803445+0000","last_active":"2024-05-20T18:31:30.805679+0000","last_peered":"2024-05-20T18:31:30.805679+0000","last_clean":"2024-05-20T18:31:30.805679+0000","last_became_active":"2024-05-20T18:31:30.803037+0000","last_became_peered":"2024-05-20T18:31:30.803037+0000","last_unstale":"2024-05-20T18:31:30.805679+0000","last_undegraded":"2024-05-20T18:31:30.805679+0000","last_fullsized":"2024-05-20T18:31:30.805679+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:27:28.606942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.785548+0000","last_change":"2024-05-20T18:31:29.789338+0000","last_active":"2024-05-20T18:31:30.785548+0000","last_peered":"2024-05-20T18:31:30.785548+0000","last_clean":"2024-05-20T18:31:30.785548+0000","last_became_active":"2024-05-20T18:31:29.788897+0000","last_became_peered":"2024-05-20T18:31:29.788897+0000","last_unstale":"2024-05-20T18:31:30.785548+0000","last_undegraded":"2024-05-20T18:31:30.785548+0000","last_fullsized":"2024-05-20T18:31:30.785548+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:12:16.266825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.796178+0000","last_change":"2024-05-20T18:31:30.793864+0000","last_active":"2024-05-20T18:31:30.796178+0000","last_peered":"2024-05-20T18:31:30.796178+0000","last_clean":"2024-05-20T18:31:30.796178+0000","last_became_active":"2024-05-20T18:31:30.793532+0000","last_became_peered":"2024-05-20T18:31:30.793532+0000","last_unstale":"2024-05-20T18:31:30.796178+0000","last_undegraded":"2024-05-20T18:31:30.796178+0000","last_fullsized":"2024-05-20T18:31:30.796178+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:02:16.521927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"76'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.818508+0000","last_change":"2024-05-20T18:31:29.788731+0000","last_active":"2024-05-20T18:31:30.818508+0000","last_peered":"2024-05-20T18:31:30.818508+0000","last_clean":"2024-05-20T18:31:30.818508+0000","last_became_active":"2024-05-20T18:31:29.788375+0000","last_became_peered":"2024-05-20T18:31:29.788375+0000","last_unstale":"2024-05-20T18:31:30.818508+0000","last_undegraded":"2024-05-20T18:31:30.818508+0000","last_fullsized":"2024-05-20T18:31:30.818508+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:18:33.200826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.814287+0000","last_change":"2024-05-20T18:31:30.801556+0000","last_active":"2024-05-20T18:31:30.814287+0000","last_peered":"2024-05-20T18:31:30.814287+0000","last_clean":"2024-05-20T18:31:30.814287+0000","last_became_active":"2024-05-20T18:31:30.800294+0000","last_became_peered":"2024-05-20T18:31:30.800294+0000","last_unstale":"2024-05-20T18:31:30.814287+0000","last_undegraded":"2024-05-20T18:31:30.814287+0000","last_fullsized":"2024-05-20T18:31:30.814287+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:16:42.686921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.790112+0000","last_change":"2024-05-20T18:31:29.787690+0000","last_active":"2024-05-20T18:31:30.790112+0000","last_peered":"2024-05-20T18:31:30.790112+0000","last_clean":"2024-05-20T18:31:30.790112+0000","last_became_active":"2024-05-20T18:31:29.787018+0000","last_became_peered":"2024-05-20T18:31:29.787018+0000","last_unstale":"2024-05-20T18:31:30.790112+0000","last_undegraded":"2024-05-20T18:31:30.790112+0000","last_fullsized":"2024-05-20T18:31:30.790112+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:56:46.630704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.799407+0000","last_change":"2024-05-20T18:31:30.792475+0000","last_active":"2024-05-20T18:31:30.799407+0000","last_peered":"2024-05-20T18:31:30.799407+0000","last_clean":"2024-05-20T18:31:30.799407+0000","last_became_active":"2024-05-20T18:31:30.792120+0000","last_became_peered":"2024-05-20T18:31:30.792120+0000","last_unstale":"2024-05-20T18:31:30.799407+0000","last_undegraded":"2024-05-20T18:31:30.799407+0000","last_fullsized":"2024-05-20T18:31:30.799407+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:12:17.272988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.790910+0000","last_change":"2024-05-20T18:31:29.786166+0000","last_active":"2024-05-20T18:31:30.790910+0000","last_peered":"2024-05-20T18:31:30.790910+0000","last_clean":"2024-05-20T18:31:30.790910+0000","last_became_active":"2024-05-20T18:31:29.785819+0000","last_became_peered":"2024-05-20T18:31:29.785819+0000","last_unstale":"2024-05-20T18:31:30.790910+0000","last_undegraded":"2024-05-20T18:31:30.790910+0000","last_fullsized":"2024-05-20T18:31:30.790910+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:10:09.430550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.799106+0000","last_change":"2024-05-20T18:31:30.792824+0000","last_active":"2024-05-20T18:31:30.799106+0000","last_peered":"2024-05-20T18:31:30.799106+0000","last_clean":"2024-05-20T18:31:30.799106+0000","last_became_active":"2024-05-20T18:31:30.792457+0000","last_became_peered":"2024-05-20T18:31:30.792457+0000","last_unstale":"2024-05-20T18:31:30.799106+0000","last_undegraded":"2024-05-20T18:31:30.799106+0000","last_fullsized":"2024-05-20T18:31:30.799106+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:53:19.626439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"76'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.817886+0000","last_change":"2024-05-20T18:31:29.786504+0000","last_active":"2024-05-20T18:31:30.817886+0000","last_peered":"2024-05-20T18:31:30.817886+0000","last_clean":"2024-05-20T18:31:30.817886+0000","last_became_active":"2024-05-20T18:31:29.786144+0000","last_became_peered":"2024-05-20T18:31:29.786144+0000","last_unstale":"2024-05-20T18:31:30.817886+0000","last_undegraded":"2024-05-20T18:31:30.817886+0000","last_fullsized":"2024-05-20T18:31:30.817886+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:13:48.095566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.809700+0000","last_change":"2024-05-20T18:31:30.805782+0000","last_active":"2024-05-20T18:31:30.809700+0000","last_peered":"2024-05-20T18:31:30.809700+0000","last_clean":"2024-05-20T18:31:30.809700+0000","last_became_active":"2024-05-20T18:31:30.805381+0000","last_became_peered":"2024-05-20T18:31:30.805381+0000","last_unstale":"2024-05-20T18:31:30.809700+0000","last_undegraded":"2024-05-20T18:31:30.809700+0000","last_fullsized":"2024-05-20T18:31:30.809700+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:40:02.269716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.781462+0000","last_change":"2024-05-20T18:31:29.795801+0000","last_active":"2024-05-20T18:31:30.781462+0000","last_peered":"2024-05-20T18:31:30.781462+0000","last_clean":"2024-05-20T18:31:30.781462+0000","last_became_active":"2024-05-20T18:31:29.795105+0000","last_became_peered":"2024-05-20T18:31:29.795105+0000","last_unstale":"2024-05-20T18:31:30.781462+0000","last_undegraded":"2024-05-20T18:31:30.781462+0000","last_fullsized":"2024-05-20T18:31:30.781462+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:35:58.569124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.812459+0000","last_change":"2024-05-20T18:31:30.809542+0000","last_active":"2024-05-20T18:31:30.812459+0000","last_peered":"2024-05-20T18:31:30.812459+0000","last_clean":"2024-05-20T18:31:30.812459+0000","last_became_active":"2024-05-20T18:31:30.807732+0000","last_became_peered":"2024-05-20T18:31:30.807732+0000","last_unstale":"2024-05-20T18:31:30.812459+0000","last_undegraded":"2024-05-20T18:31:30.812459+0000","last_fullsized":"2024-05-20T18:31:30.812459+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:48:32.392016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.794249+0000","last_change":"2024-05-20T18:31:29.784997+0000","last_active":"2024-05-20T18:31:30.794249+0000","last_peered":"2024-05-20T18:31:30.794249+0000","last_clean":"2024-05-20T18:31:30.794249+0000","last_became_active":"2024-05-20T18:31:29.784681+0000","last_became_peered":"2024-05-20T18:31:29.784681+0000","last_unstale":"2024-05-20T18:31:30.794249+0000","last_undegraded":"2024-05-20T18:31:30.794249+0000","last_fullsized":"2024-05-20T18:31:30.794249+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:02:15.515764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.796752+0000","last_change":"2024-05-20T18:31:30.795220+0000","last_active":"2024-05-20T18:31:30.796752+0000","last_peered":"2024-05-20T18:31:30.796752+0000","last_clean":"2024-05-20T18:31:30.796752+0000","last_became_active":"2024-05-20T18:31:30.794915+0000","last_became_peered":"2024-05-20T18:31:30.794915+0000","last_unstale":"2024-05-20T18:31:30.796752+0000","last_undegraded":"2024-05-20T18:31:30.796752+0000","last_fullsized":"2024-05-20T18:31:30.796752+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:43:04.031026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.793178+0000","last_change":"2024-05-20T18:31:29.791935+0000","last_active":"2024-05-20T18:31:30.793178+0000","last_peered":"2024-05-20T18:31:30.793178+0000","last_clean":"2024-05-20T18:31:30.793178+0000","last_became_active":"2024-05-20T18:31:29.791219+0000","last_became_peered":"2024-05-20T18:31:29.791219+0000","last_unstale":"2024-05-20T18:31:30.793178+0000","last_undegraded":"2024-05-20T18:31:30.793178+0000","last_fullsized":"2024-05-20T18:31:30.793178+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:35:58.569124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.811970+0000","last_change":"2024-05-20T18:31:30.807004+0000","last_active":"2024-05-20T18:31:30.811970+0000","last_peered":"2024-05-20T18:31:30.811970+0000","last_clean":"2024-05-20T18:31:30.811970+0000","last_became_active":"2024-05-20T18:31:30.806468+0000","last_became_peered":"2024-05-20T18:31:30.806468+0000","last_unstale":"2024-05-20T18:31:30.811970+0000","last_undegraded":"2024-05-20T18:31:30.811970+0000","last_fullsized":"2024-05-20T18:31:30.811970+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:11:30.282843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.784333+0000","last_change":"2024-05-20T18:31:29.790063+0000","last_active":"2024-05-20T18:31:30.784333+0000","last_peered":"2024-05-20T18:31:30.784333+0000","last_clean":"2024-05-20T18:31:30.784333+0000","last_became_active":"2024-05-20T18:31:29.788253+0000","last_became_peered":"2024-05-20T18:31:29.788253+0000","last_unstale":"2024-05-20T18:31:30.784333+0000","last_undegraded":"2024-05-20T18:31:30.784333+0000","last_fullsized":"2024-05-20T18:31:30.784333+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:57:45.704387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.805961+0000","last_change":"2024-05-20T18:31:30.802951+0000","last_active":"2024-05-20T18:31:30.805961+0000","last_peered":"2024-05-20T18:31:30.805961+0000","last_clean":"2024-05-20T18:31:30.805961+0000","last_became_active":"2024-05-20T18:31:30.802589+0000","last_became_peered":"2024-05-20T18:31:30.802589+0000","last_unstale":"2024-05-20T18:31:30.805961+0000","last_undegraded":"2024-05-20T18:31:30.805961+0000","last_fullsized":"2024-05-20T18:31:30.805961+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:13:28.735527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.787880+0000","last_change":"2024-05-20T18:31:29.790390+0000","last_active":"2024-05-20T18:31:30.787880+0000","last_peered":"2024-05-20T18:31:30.787880+0000","last_clean":"2024-05-20T18:31:30.787880+0000","last_became_active":"2024-05-20T18:31:29.790092+0000","last_became_peered":"2024-05-20T18:31:29.790092+0000","last_unstale":"2024-05-20T18:31:30.787880+0000","last_undegraded":"2024-05-20T18:31:30.787880+0000","last_fullsized":"2024-05-20T18:31:30.787880+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:49:56.374138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.818078+0000","last_change":"2024-05-20T18:31:30.815761+0000","last_active":"2024-05-20T18:31:30.818078+0000","last_peered":"2024-05-20T18:31:30.818078+0000","last_clean":"2024-05-20T18:31:30.818078+0000","last_became_active":"2024-05-20T18:31:30.815454+0000","last_became_peered":"2024-05-20T18:31:30.815454+0000","last_unstale":"2024-05-20T18:31:30.818078+0000","last_undegraded":"2024-05-20T18:31:30.818078+0000","last_fullsized":"2024-05-20T18:31:30.818078+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:11:30.282843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.783760+0000","last_change":"2024-05-20T18:31:29.788645+0000","last_active":"2024-05-20T18:31:30.783760+0000","last_peered":"2024-05-20T18:31:30.783760+0000","last_clean":"2024-05-20T18:31:30.783760+0000","last_became_active":"2024-05-20T18:31:29.788309+0000","last_became_peered":"2024-05-20T18:31:29.788309+0000","last_unstale":"2024-05-20T18:31:30.783760+0000","last_undegraded":"2024-05-20T18:31:30.783760+0000","last_fullsized":"2024-05-20T18:31:30.783760+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:10:58.189779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.808313+0000","last_change":"2024-05-20T18:31:30.799115+0000","last_active":"2024-05-20T18:31:30.808313+0000","last_peered":"2024-05-20T18:31:30.808313+0000","last_clean":"2024-05-20T18:31:30.808313+0000","last_became_active":"2024-05-20T18:31:30.798804+0000","last_became_peered":"2024-05-20T18:31:30.798804+0000","last_unstale":"2024-05-20T18:31:30.808313+0000","last_undegraded":"2024-05-20T18:31:30.808313+0000","last_fullsized":"2024-05-20T18:31:30.808313+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:28:27.930145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.785221+0000","last_change":"2024-05-20T18:31:29.779068+0000","last_active":"2024-05-20T18:31:30.785221+0000","last_peered":"2024-05-20T18:31:30.785221+0000","last_clean":"2024-05-20T18:31:30.785221+0000","last_became_active":"2024-05-20T18:31:29.778707+0000","last_became_peered":"2024-05-20T18:31:29.778707+0000","last_unstale":"2024-05-20T18:31:30.785221+0000","last_undegraded":"2024-05-20T18:31:30.785221+0000","last_fullsized":"2024-05-20T18:31:30.785221+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:48:31.385853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.812321+0000","last_change":"2024-05-20T18:31:30.809305+0000","last_active":"2024-05-20T18:31:30.812321+0000","last_peered":"2024-05-20T18:31:30.812321+0000","last_clean":"2024-05-20T18:31:30.812321+0000","last_became_active":"2024-05-20T18:31:30.808975+0000","last_became_peered":"2024-05-20T18:31:30.808975+0000","last_unstale":"2024-05-20T18:31:30.812321+0000","last_undegraded":"2024-05-20T18:31:30.812321+0000","last_fullsized":"2024-05-20T18:31:30.812321+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:38:41.987602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.794249+0000","last_change":"2024-05-20T18:31:29.786718+0000","last_active":"2024-05-20T18:31:30.794249+0000","last_peered":"2024-05-20T18:31:30.794249+0000","last_clean":"2024-05-20T18:31:30.794249+0000","last_became_active":"2024-05-20T18:31:29.786367+0000","last_became_peered":"2024-05-20T18:31:29.786367+0000","last_unstale":"2024-05-20T18:31:30.794249+0000","last_undegraded":"2024-05-20T18:31:30.794249+0000","last_fullsized":"2024-05-20T18:31:30.794249+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:14:33.322262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":271,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.791666+0000","last_change":"2024-05-20T18:27:03.765694+0000","last_active":"2024-05-20T18:31:30.791666+0000","last_peered":"2024-05-20T18:31:30.791666+0000","last_clean":"2024-05-20T18:31:30.791666+0000","last_became_active":"2024-05-20T18:27:03.764695+0000","last_became_peered":"2024-05-20T18:27:03.764695+0000","last_unstale":"2024-05-20T18:31:30.791666+0000","last_undegraded":"2024-05-20T18:31:30.791666+0000","last_fullsized":"2024-05-20T18:31:30.791666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_clean_scrub_stamp":"2024-05-20T18:27:02.750853+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:30:04.407487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574906,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1511424,"data_stored":1005734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574904,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":942080,"data_stored":424145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574904,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":933888,"data_stored":422567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574904,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":917504,"data_stored":415366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574906,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339744,"statfs":{"total":96636764160,"available":96603897856,"internally_reserved":0,"allocated":933888,"data_stored":423115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574905,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":917504,"data_stored":415366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":917504,"data_stored":415366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574905,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1523712,"data_stored":1012901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T18:31:34.853 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T18:31:35.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.004+0000 7f3078f4c640 1 Processor -- start 2024-05-20T18:31:35.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.004+0000 7f3078f4c640 1 -- start start 2024-05-20T18:31:35.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.006+0000 7f3078f4c640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f307406d3d0 0x7f307406d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:35.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.006+0000 7f3078f4c640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f307406dd20 con 0x7f30740b8190 2024-05-20T18:31:35.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.006+0000 7f3078f4c640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30740d0630 con 0x7f3074162820 2024-05-20T18:31:35.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.006+0000 7f3078f4c640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f30740d39c0 con 0x7f307406d3d0 2024-05-20T18:31:35.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.006+0000 7f3072d9d640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f30740b8190 0x7f30740b85a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56850/0 (socket says 172.21.15.5:56850) 2024-05-20T18:31:35.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.006+0000 7f3072d9d640 1 -- 172.21.15.5:0/2920192927 learned_addr learned my addr 172.21.15.5:0/2920192927 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:35.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.006+0000 7f3071d9b640 1 --2- 172.21.15.5:0/2920192927 >> v2:172.21.15.63:3300/0 conn(0x7f307406d3d0 0x7f307406d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:35.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.006+0000 7f307159a640 1 -- 172.21.15.5:0/2920192927 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 674181585 0 0) 0x7f3064001110 con 0x7f30740b8190 2024-05-20T18:31:35.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.006+0000 7f307159a640 1 -- 172.21.15.5:0/2920192927 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3060003520 con 0x7f30740b8190 2024-05-20T18:31:35.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.007+0000 7f307159a640 1 -- 172.21.15.5:0/2920192927 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 772401869 0 0) 0x7f3060003520 con 0x7f30740b8190 2024-05-20T18:31:35.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.007+0000 7f307159a640 1 -- 172.21.15.5:0/2920192927 >> v2:172.21.15.63:3300/0 conn(0x7f307406d3d0 msgr2=0x7f307406d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:35.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.007+0000 7f307159a640 1 --2- 172.21.15.5:0/2920192927 >> v2:172.21.15.63:3300/0 conn(0x7f307406d3d0 0x7f307406d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:35.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.007+0000 7f307159a640 1 -- 172.21.15.5:0/2920192927 >> v1:172.21.15.63:6789/0 conn(0x7f3074162820 legacy=0x7f30740b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:35.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.007+0000 7f307159a640 1 -- 172.21.15.5:0/2920192927 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30740d3ca0 con 0x7f30740b8190 2024-05-20T18:31:35.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.007+0000 7f307159a640 1 -- 172.21.15.5:0/2920192927 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f3064001840 con 0x7f30740b8190 2024-05-20T18:31:35.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.007+0000 7f307159a640 1 -- 172.21.15.5:0/2920192927 >> v1:172.21.15.5:6789/0 conn(0x7f30740b8190 legacy=0x7f30740b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:35.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.007+0000 7f307159a640 1 --2- 172.21.15.5:0/2920192927 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f30600039b0 0x7f3060003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:35.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.007+0000 7f307159a640 1 --2- 172.21.15.5:0/2920192927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3060004800 0x7f3060004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:35.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.007+0000 7f307159a640 1 --2- 172.21.15.5:0/2920192927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30600051c0 0x7f306000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:35.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.007+0000 7f307159a640 1 -- 172.21.15.5:0/2920192927 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f30740d39c0 con 0x7f30600051c0 2024-05-20T18:31:35.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.007+0000 7f307159a640 1 -- 172.21.15.5:0/2920192927 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f306000c160 con 0x7f3060004800 2024-05-20T18:31:35.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.007+0000 7f307159a640 1 -- 172.21.15.5:0/2920192927 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f306000c360 con 0x7f30600039b0 2024-05-20T18:31:35.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.007+0000 7f3072d9d640 1 --2- 172.21.15.5:0/2920192927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30600051c0 0x7f306000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:35.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.007+0000 7f3071d9b640 1 --2- 172.21.15.5:0/2920192927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3060004800 0x7f3060004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:35.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.007+0000 7f307259c640 1 --2- 172.21.15.5:0/2920192927 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f30600039b0 0x7f3060003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:35.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.007+0000 7f3072d9d640 1 -- 172.21.15.5:0/2920192927 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f30600039b0 msgr2=0x7f3060003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:35.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.007+0000 7f3072d9d640 1 --2- 172.21.15.5:0/2920192927 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f30600039b0 0x7f3060003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:35.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.007+0000 7f3072d9d640 1 -- 172.21.15.5:0/2920192927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3060004800 msgr2=0x7f3060004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:35.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.007+0000 7f3072d9d640 1 --2- 172.21.15.5:0/2920192927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3060004800 0x7f3060004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:35.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.007+0000 7f3072d9d640 1 -- 172.21.15.5:0/2920192927 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3060003520 con 0x7f30600051c0 2024-05-20T18:31:35.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.008+0000 7f3072d9d640 1 -- 172.21.15.5:0/2920192927 reap_dead start 2024-05-20T18:31:35.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.008+0000 7f3072d9d640 1 --2- 172.21.15.5:0/2920192927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30600051c0 0x7f306000b9c0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f306400c4c0 tx=0x7f3060005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f4dce983d5c06f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:35.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.008+0000 7f307159a640 1 -- 172.21.15.5:0/2920192927 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064036070 con 0x7f30600051c0 2024-05-20T18:31:35.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.008+0000 7f307159a640 1 -- 172.21.15.5:0/2920192927 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3064005af0 con 0x7f30600051c0 2024-05-20T18:31:35.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.008+0000 7f307159a640 1 -- 172.21.15.5:0/2920192927 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306400c0e0 con 0x7f30600051c0 2024-05-20T18:31:35.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.009+0000 7f3078f4c640 1 -- 172.21.15.5:0/2920192927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30600051c0 msgr2=0x7f306000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:35.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.009+0000 7f3078f4c640 1 --2- 172.21.15.5:0/2920192927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30600051c0 0x7f306000b9c0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f306400c4c0 tx=0x7f3060005830 comp rx=0 tx=0).stop 2024-05-20T18:31:35.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.009+0000 7f3078f4c640 1 -- 172.21.15.5:0/2920192927 shutdown_connections 2024-05-20T18:31:35.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.009+0000 7f3078f4c640 1 --2- 172.21.15.5:0/2920192927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30600051c0 0x7f306000b9c0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:35.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.009+0000 7f3078f4c640 1 -- 172.21.15.5:0/2920192927 >> 172.21.15.5:0/2920192927 conn(0x7f30740d4bd0 msgr2=0x7f30740d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:35.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.009+0000 7f3078f4c640 1 -- 172.21.15.5:0/2920192927 shutdown_connections 2024-05-20T18:31:35.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.009+0000 7f3078f4c640 1 -- 172.21.15.5:0/2920192927 wait complete. 2024-05-20T18:31:35.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.011+0000 7f3078f4c640 1 Processor -- start 2024-05-20T18:31:35.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.011+0000 7f3078f4c640 1 -- start start 2024-05-20T18:31:35.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.012+0000 7f3078f4c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30600051c0 0x7f307415cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:35.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.012+0000 7f3078f4c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30740cbfb0 0x7f30740cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:35.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.012+0000 7f3078f4c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f307414d710 0x7f307414db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:35.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.012+0000 7f3078f4c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f30740d0300 con 0x7f30740cbfb0 2024-05-20T18:31:35.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.012+0000 7f3078f4c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f30740d4260 con 0x7f30600051c0 2024-05-20T18:31:35.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.012+0000 7f3078f4c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f30740d4780 con 0x7f307414d710 2024-05-20T18:31:35.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.013+0000 7f3071d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30740cbfb0 0x7f30740cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:35.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.013+0000 7f3071d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30740cbfb0 0x7f30740cc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57976/0 (socket says 172.21.15.5:57976) 2024-05-20T18:31:35.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.013+0000 7f3071d9b640 1 -- 172.21.15.5:0/4126258673 learned_addr learned my addr 172.21.15.5:0/4126258673 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:35.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.013+0000 7f307259c640 1 --2- 172.21.15.5:0/4126258673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30600051c0 0x7f307415cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:35.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.013+0000 7f3072d9d640 1 --2- 172.21.15.5:0/4126258673 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f307414d710 0x7f307414db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:35.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.013+0000 7f3071d9b640 1 -- 172.21.15.5:0/4126258673 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f307414d710 msgr2=0x7f307414db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:35.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.013+0000 7f3071d9b640 1 --2- 172.21.15.5:0/4126258673 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f307414d710 0x7f307414db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:35.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.013+0000 7f3071d9b640 1 -- 172.21.15.5:0/4126258673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30600051c0 msgr2=0x7f307415cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:35.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.013+0000 7f3071d9b640 1 --2- 172.21.15.5:0/4126258673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30600051c0 0x7f307415cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:35.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.013+0000 7f3071d9b640 1 -- 172.21.15.5:0/4126258673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30740ccd70 con 0x7f30740cbfb0 2024-05-20T18:31:35.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.013+0000 7f307259c640 1 --2- 172.21.15.5:0/4126258673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30600051c0 0x7f307415cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:35.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.013+0000 7f3071d9b640 1 --2- 172.21.15.5:0/4126258673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30740cbfb0 0x7f30740cc3c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f305c011910 tx=0x7f305c011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:35.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.013+0000 7f306affd640 1 -- 172.21.15.5:0/4126258673 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c00faf0 con 0x7f30740cbfb0 2024-05-20T18:31:35.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.013+0000 7f306affd640 1 -- 172.21.15.5:0/4126258673 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f305c00fc90 con 0x7f30740cbfb0 2024-05-20T18:31:35.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.013+0000 7f3078f4c640 1 -- 172.21.15.5:0/4126258673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30740cd050 con 0x7f30740cbfb0 2024-05-20T18:31:35.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.013+0000 7f306affd640 1 -- 172.21.15.5:0/4126258673 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c014440 con 0x7f30740cbfb0 2024-05-20T18:31:35.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.013+0000 7f3078f4c640 1 -- 172.21.15.5:0/4126258673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30740cd610 con 0x7f30740cbfb0 2024-05-20T18:31:35.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.015+0000 7f3078f4c640 1 -- 172.21.15.5:0/4126258673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3074072f60 con 0x7f30740cbfb0 2024-05-20T18:31:35.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.015+0000 7f306affd640 1 -- 172.21.15.5:0/4126258673 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f305c012040 con 0x7f30740cbfb0 2024-05-20T18:31:35.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.015+0000 7f306affd640 1 --2- 172.21.15.5:0/4126258673 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f305406be30 0x7f305406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:35.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.015+0000 7f306affd640 1 -- 172.21.15.5:0/4126258673 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6101+0+0 (secure 0 0 0) 0x7f305c08b6c0 con 0x7f30740cbfb0 2024-05-20T18:31:35.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.015+0000 7f307259c640 1 --2- 172.21.15.5:0/4126258673 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f305406be30 0x7f305406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:35.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.016+0000 7f307259c640 1 --2- 172.21.15.5:0/4126258673 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f305406be30 0x7f305406e340 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f306c000f80 tx=0x7f306c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:35.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.020+0000 7f306affd640 1 -- 172.21.15.5:0/4126258673 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3074072f60 con 0x7f30740cbfb0 2024-05-20T18:31:35.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.198+0000 7f3078f4c640 1 -- 172.21.15.5:0/4126258673 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3074075b00 con 0x7f305406be30 2024-05-20T18:31:35.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.204+0000 7f306affd640 1 -- 172.21.15.5:0/4126258673 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252005 (secure 0 0 0) 0x7f3074075b00 con 0x7f305406be30 2024-05-20T18:31:35.206 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:31:35.207 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-20T18:31:35.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.207+0000 7f3078f4c640 1 -- 172.21.15.5:0/4126258673 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f305406be30 msgr2=0x7f305406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:35.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.207+0000 7f3078f4c640 1 --2- 172.21.15.5:0/4126258673 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f305406be30 0x7f305406e340 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f306c000f80 tx=0x7f306c009040 comp rx=0 tx=0).stop 2024-05-20T18:31:35.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.207+0000 7f3078f4c640 1 -- 172.21.15.5:0/4126258673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30740cbfb0 msgr2=0x7f30740cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:35.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.207+0000 7f3078f4c640 1 --2- 172.21.15.5:0/4126258673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30740cbfb0 0x7f30740cc3c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f305c011910 tx=0x7f305c011dd0 comp rx=0 tx=0).stop 2024-05-20T18:31:35.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.207+0000 7f3078f4c640 1 -- 172.21.15.5:0/4126258673 shutdown_connections 2024-05-20T18:31:35.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.207+0000 7f3078f4c640 1 --2- 172.21.15.5:0/4126258673 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f305406be30 0x7f305406e340 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:35.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.207+0000 7f3078f4c640 1 --2- 172.21.15.5:0/4126258673 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f307414d710 0x7f307414db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:35.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.207+0000 7f3078f4c640 1 --2- 172.21.15.5:0/4126258673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30740cbfb0 0x7f30740cc3c0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:35.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.207+0000 7f3078f4c640 1 --2- 172.21.15.5:0/4126258673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30600051c0 0x7f307415cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:35.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.207+0000 7f3078f4c640 1 -- 172.21.15.5:0/4126258673 >> 172.21.15.5:0/4126258673 conn(0x7f30740d4bd0 msgr2=0x7f3074160eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:35.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.207+0000 7f3078f4c640 1 -- 172.21.15.5:0/4126258673 shutdown_connections 2024-05-20T18:31:35.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.207+0000 7f3078f4c640 1 -- 172.21.15.5:0/4126258673 wait complete. 2024-05-20T18:31:35.218 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2024-05-20T18:31:34.642249+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257852,"kb_used_data":8396,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716868,"statfs":{"total":773094113280,"available":772830072832,"internally_reserved":0,"allocated":8597504,"data_stored":4534560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.877671"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.806930+0000","last_change":"2024-05-20T18:31:30.803994+0000","last_active":"2024-05-20T18:31:30.806930+0000","last_peered":"2024-05-20T18:31:30.806930+0000","last_clean":"2024-05-20T18:31:30.806930+0000","last_became_active":"2024-05-20T18:31:30.803685+0000","last_became_peered":"2024-05-20T18:31:30.803685+0000","last_unstale":"2024-05-20T18:31:30.806930+0000","last_undegraded":"2024-05-20T18:31:30.806930+0000","last_fullsized":"2024-05-20T18:31:30.806930+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:36:09.353546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.801307+0000","last_change":"2024-05-20T18:31:30.799754+0000","last_active":"2024-05-20T18:31:30.801307+0000","last_peered":"2024-05-20T18:31:30.801307+0000","last_clean":"2024-05-20T18:31:30.801307+0000","last_became_active":"2024-05-20T18:31:30.799447+0000","last_became_peered":"2024-05-20T18:31:30.799447+0000","last_unstale":"2024-05-20T18:31:30.801307+0000","last_undegraded":"2024-05-20T18:31:30.801307+0000","last_fullsized":"2024-05-20T18:31:30.801307+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:03:06.377489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.810898+0000","last_change":"2024-05-20T18:31:30.801665+0000","last_active":"2024-05-20T18:31:30.810898+0000","last_peered":"2024-05-20T18:31:30.810898+0000","last_clean":"2024-05-20T18:31:30.810898+0000","last_became_active":"2024-05-20T18:31:30.801373+0000","last_became_peered":"2024-05-20T18:31:30.801373+0000","last_unstale":"2024-05-20T18:31:30.810898+0000","last_undegraded":"2024-05-20T18:31:30.810898+0000","last_fullsized":"2024-05-20T18:31:30.810898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:13:28.735527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.803104+0000","last_change":"2024-05-20T18:31:30.797241+0000","last_active":"2024-05-20T18:31:30.803104+0000","last_peered":"2024-05-20T18:31:30.803104+0000","last_clean":"2024-05-20T18:31:30.803104+0000","last_became_active":"2024-05-20T18:31:30.796907+0000","last_became_peered":"2024-05-20T18:31:30.796907+0000","last_unstale":"2024-05-20T18:31:30.803104+0000","last_undegraded":"2024-05-20T18:31:30.803104+0000","last_fullsized":"2024-05-20T18:31:30.803104+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:02:16.521927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.808729+0000","last_change":"2024-05-20T18:31:30.804803+0000","last_active":"2024-05-20T18:31:30.808729+0000","last_peered":"2024-05-20T18:31:30.808729+0000","last_clean":"2024-05-20T18:31:30.808729+0000","last_became_active":"2024-05-20T18:31:30.804470+0000","last_became_peered":"2024-05-20T18:31:30.804470+0000","last_unstale":"2024-05-20T18:31:30.808729+0000","last_undegraded":"2024-05-20T18:31:30.808729+0000","last_fullsized":"2024-05-20T18:31:30.808729+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:37:47.705579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.813978+0000","last_change":"2024-05-20T18:31:30.808081+0000","last_active":"2024-05-20T18:31:30.813978+0000","last_peered":"2024-05-20T18:31:30.813978+0000","last_clean":"2024-05-20T18:31:30.813978+0000","last_became_active":"2024-05-20T18:31:30.804981+0000","last_became_peered":"2024-05-20T18:31:30.804981+0000","last_unstale":"2024-05-20T18:31:30.813978+0000","last_undegraded":"2024-05-20T18:31:30.813978+0000","last_fullsized":"2024-05-20T18:31:30.813978+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:43:45.131596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.803164+0000","last_change":"2024-05-20T18:31:30.801157+0000","last_active":"2024-05-20T18:31:30.803164+0000","last_peered":"2024-05-20T18:31:30.803164+0000","last_clean":"2024-05-20T18:31:30.803164+0000","last_became_active":"2024-05-20T18:31:30.800796+0000","last_became_peered":"2024-05-20T18:31:30.800796+0000","last_unstale":"2024-05-20T18:31:30.803164+0000","last_undegraded":"2024-05-20T18:31:30.803164+0000","last_fullsized":"2024-05-20T18:31:30.803164+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:28:27.930145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.806951+0000","last_change":"2024-05-20T18:31:30.802273+0000","last_active":"2024-05-20T18:31:30.806951+0000","last_peered":"2024-05-20T18:31:30.806951+0000","last_clean":"2024-05-20T18:31:30.806951+0000","last_became_active":"2024-05-20T18:31:30.801960+0000","last_became_peered":"2024-05-20T18:31:30.801960+0000","last_unstale":"2024-05-20T18:31:30.806951+0000","last_undegraded":"2024-05-20T18:31:30.806951+0000","last_fullsized":"2024-05-20T18:31:30.806951+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:11:25.301998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.809043+0000","last_change":"2024-05-20T18:31:30.807811+0000","last_active":"2024-05-20T18:31:30.809043+0000","last_peered":"2024-05-20T18:31:30.809043+0000","last_clean":"2024-05-20T18:31:30.809043+0000","last_became_active":"2024-05-20T18:31:30.807469+0000","last_became_peered":"2024-05-20T18:31:30.807469+0000","last_unstale":"2024-05-20T18:31:30.809043+0000","last_undegraded":"2024-05-20T18:31:30.809043+0000","last_fullsized":"2024-05-20T18:31:30.809043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:10:59.195942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.806179+0000","last_change":"2024-05-20T18:31:30.799274+0000","last_active":"2024-05-20T18:31:30.806179+0000","last_peered":"2024-05-20T18:31:30.806179+0000","last_clean":"2024-05-20T18:31:30.806179+0000","last_became_active":"2024-05-20T18:31:30.798746+0000","last_became_peered":"2024-05-20T18:31:30.798746+0000","last_unstale":"2024-05-20T18:31:30.806179+0000","last_undegraded":"2024-05-20T18:31:30.806179+0000","last_fullsized":"2024-05-20T18:31:30.806179+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:39:38.803158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.814497+0000","last_change":"2024-05-20T18:31:30.804852+0000","last_active":"2024-05-20T18:31:30.814497+0000","last_peered":"2024-05-20T18:31:30.814497+0000","last_clean":"2024-05-20T18:31:30.814497+0000","last_became_active":"2024-05-20T18:31:30.804486+0000","last_became_peered":"2024-05-20T18:31:30.804486+0000","last_unstale":"2024-05-20T18:31:30.814497+0000","last_undegraded":"2024-05-20T18:31:30.814497+0000","last_fullsized":"2024-05-20T18:31:30.814497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:38:41.987602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.803073+0000","last_change":"2024-05-20T18:31:30.795408+0000","last_active":"2024-05-20T18:31:30.803073+0000","last_peered":"2024-05-20T18:31:30.803073+0000","last_clean":"2024-05-20T18:31:30.803073+0000","last_became_active":"2024-05-20T18:31:30.795104+0000","last_became_peered":"2024-05-20T18:31:30.795104+0000","last_unstale":"2024-05-20T18:31:30.803073+0000","last_undegraded":"2024-05-20T18:31:30.803073+0000","last_fullsized":"2024-05-20T18:31:30.803073+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:35:59.575287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.815767+0000","last_change":"2024-05-20T18:31:30.813213+0000","last_active":"2024-05-20T18:31:30.815767+0000","last_peered":"2024-05-20T18:31:30.815767+0000","last_clean":"2024-05-20T18:31:30.815767+0000","last_became_active":"2024-05-20T18:31:30.812894+0000","last_became_peered":"2024-05-20T18:31:30.812894+0000","last_unstale":"2024-05-20T18:31:30.815767+0000","last_undegraded":"2024-05-20T18:31:30.815767+0000","last_fullsized":"2024-05-20T18:31:30.815767+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:18:23.823402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.807928+0000","last_change":"2024-05-20T18:31:30.804426+0000","last_active":"2024-05-20T18:31:30.807928+0000","last_peered":"2024-05-20T18:31:30.807928+0000","last_clean":"2024-05-20T18:31:30.807928+0000","last_became_active":"2024-05-20T18:31:30.804020+0000","last_became_peered":"2024-05-20T18:31:30.804020+0000","last_unstale":"2024-05-20T18:31:30.807928+0000","last_undegraded":"2024-05-20T18:31:30.807928+0000","last_fullsized":"2024-05-20T18:31:30.807928+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:31:45.080109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.814228+0000","last_change":"2024-05-20T18:31:30.800832+0000","last_active":"2024-05-20T18:31:30.814228+0000","last_peered":"2024-05-20T18:31:30.814228+0000","last_clean":"2024-05-20T18:31:30.814228+0000","last_became_active":"2024-05-20T18:31:30.800468+0000","last_became_peered":"2024-05-20T18:31:30.800468+0000","last_unstale":"2024-05-20T18:31:30.814228+0000","last_undegraded":"2024-05-20T18:31:30.814228+0000","last_fullsized":"2024-05-20T18:31:30.814228+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:30:09.481775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.804348+0000","last_change":"2024-05-20T18:31:30.800500+0000","last_active":"2024-05-20T18:31:30.804348+0000","last_peered":"2024-05-20T18:31:30.804348+0000","last_clean":"2024-05-20T18:31:30.804348+0000","last_became_active":"2024-05-20T18:31:30.799974+0000","last_became_peered":"2024-05-20T18:31:30.799974+0000","last_unstale":"2024-05-20T18:31:30.804348+0000","last_undegraded":"2024-05-20T18:31:30.804348+0000","last_fullsized":"2024-05-20T18:31:30.804348+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:34:43.715283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.806990+0000","last_change":"2024-05-20T18:31:30.799031+0000","last_active":"2024-05-20T18:31:30.806990+0000","last_peered":"2024-05-20T18:31:30.806990+0000","last_clean":"2024-05-20T18:31:30.806990+0000","last_became_active":"2024-05-20T18:31:30.798559+0000","last_became_peered":"2024-05-20T18:31:30.798559+0000","last_unstale":"2024-05-20T18:31:30.806990+0000","last_undegraded":"2024-05-20T18:31:30.806990+0000","last_fullsized":"2024-05-20T18:31:30.806990+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:46:00.326307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.810408+0000","last_change":"2024-05-20T18:31:30.799918+0000","last_active":"2024-05-20T18:31:30.810408+0000","last_peered":"2024-05-20T18:31:30.810408+0000","last_clean":"2024-05-20T18:31:30.810408+0000","last_became_active":"2024-05-20T18:31:30.799596+0000","last_became_peered":"2024-05-20T18:31:30.799596+0000","last_unstale":"2024-05-20T18:31:30.810408+0000","last_undegraded":"2024-05-20T18:31:30.810408+0000","last_fullsized":"2024-05-20T18:31:30.810408+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:18:35.214658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.805643+0000","last_change":"2024-05-20T18:31:30.803385+0000","last_active":"2024-05-20T18:31:30.805643+0000","last_peered":"2024-05-20T18:31:30.805643+0000","last_clean":"2024-05-20T18:31:30.805643+0000","last_became_active":"2024-05-20T18:31:30.802737+0000","last_became_peered":"2024-05-20T18:31:30.802737+0000","last_unstale":"2024-05-20T18:31:30.805643+0000","last_undegraded":"2024-05-20T18:31:30.805643+0000","last_fullsized":"2024-05-20T18:31:30.805643+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:10:22.671462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.819226+0000","last_change":"2024-05-20T18:31:30.816537+0000","last_active":"2024-05-20T18:31:30.819226+0000","last_peered":"2024-05-20T18:31:30.819226+0000","last_clean":"2024-05-20T18:31:30.819226+0000","last_became_active":"2024-05-20T18:31:30.816226+0000","last_became_peered":"2024-05-20T18:31:30.816226+0000","last_unstale":"2024-05-20T18:31:30.819226+0000","last_undegraded":"2024-05-20T18:31:30.819226+0000","last_fullsized":"2024-05-20T18:31:30.819226+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:52:56.818967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.808815+0000","last_change":"2024-05-20T18:31:30.805340+0000","last_active":"2024-05-20T18:31:30.808815+0000","last_peered":"2024-05-20T18:31:30.808815+0000","last_clean":"2024-05-20T18:31:30.808815+0000","last_became_active":"2024-05-20T18:31:30.804968+0000","last_became_peered":"2024-05-20T18:31:30.804968+0000","last_unstale":"2024-05-20T18:31:30.808815+0000","last_undegraded":"2024-05-20T18:31:30.808815+0000","last_fullsized":"2024-05-20T18:31:30.808815+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:23:02.785905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.818892+0000","last_change":"2024-05-20T18:31:30.815412+0000","last_active":"2024-05-20T18:31:30.818892+0000","last_peered":"2024-05-20T18:31:30.818892+0000","last_clean":"2024-05-20T18:31:30.818892+0000","last_became_active":"2024-05-20T18:31:30.815118+0000","last_became_peered":"2024-05-20T18:31:30.815118+0000","last_unstale":"2024-05-20T18:31:30.818892+0000","last_undegraded":"2024-05-20T18:31:30.818892+0000","last_fullsized":"2024-05-20T18:31:30.818892+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:07:34.573709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.799244+0000","last_change":"2024-05-20T18:31:30.793306+0000","last_active":"2024-05-20T18:31:30.799244+0000","last_peered":"2024-05-20T18:31:30.799244+0000","last_clean":"2024-05-20T18:31:30.799244+0000","last_became_active":"2024-05-20T18:31:30.792967+0000","last_became_peered":"2024-05-20T18:31:30.792967+0000","last_unstale":"2024-05-20T18:31:30.799244+0000","last_undegraded":"2024-05-20T18:31:30.799244+0000","last_fullsized":"2024-05-20T18:31:30.799244+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:38:42.055506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.814218+0000","last_change":"2024-05-20T18:31:30.808953+0000","last_active":"2024-05-20T18:31:30.814218+0000","last_peered":"2024-05-20T18:31:30.814218+0000","last_clean":"2024-05-20T18:31:30.814218+0000","last_became_active":"2024-05-20T18:31:30.808638+0000","last_became_peered":"2024-05-20T18:31:30.808638+0000","last_unstale":"2024-05-20T18:31:30.814218+0000","last_undegraded":"2024-05-20T18:31:30.814218+0000","last_fullsized":"2024-05-20T18:31:30.814218+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:56:43.205875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.814263+0000","last_change":"2024-05-20T18:31:30.809078+0000","last_active":"2024-05-20T18:31:30.814263+0000","last_peered":"2024-05-20T18:31:30.814263+0000","last_clean":"2024-05-20T18:31:30.814263+0000","last_became_active":"2024-05-20T18:31:30.808679+0000","last_became_peered":"2024-05-20T18:31:30.808679+0000","last_unstale":"2024-05-20T18:31:30.814263+0000","last_undegraded":"2024-05-20T18:31:30.814263+0000","last_fullsized":"2024-05-20T18:31:30.814263+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:33:51.968504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.803106+0000","last_change":"2024-05-20T18:31:30.796531+0000","last_active":"2024-05-20T18:31:30.803106+0000","last_peered":"2024-05-20T18:31:30.803106+0000","last_clean":"2024-05-20T18:31:30.803106+0000","last_became_active":"2024-05-20T18:31:30.796199+0000","last_became_peered":"2024-05-20T18:31:30.796199+0000","last_unstale":"2024-05-20T18:31:30.803106+0000","last_undegraded":"2024-05-20T18:31:30.803106+0000","last_fullsized":"2024-05-20T18:31:30.803106+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:05:22.123725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.804282+0000","last_change":"2024-05-20T18:31:30.801409+0000","last_active":"2024-05-20T18:31:30.804282+0000","last_peered":"2024-05-20T18:31:30.804282+0000","last_clean":"2024-05-20T18:31:30.804282+0000","last_became_active":"2024-05-20T18:31:30.800968+0000","last_became_peered":"2024-05-20T18:31:30.800968+0000","last_unstale":"2024-05-20T18:31:30.804282+0000","last_undegraded":"2024-05-20T18:31:30.804282+0000","last_fullsized":"2024-05-20T18:31:30.804282+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:30:09.481775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.798564+0000","last_change":"2024-05-20T18:31:30.796685+0000","last_active":"2024-05-20T18:31:30.798564+0000","last_peered":"2024-05-20T18:31:30.798564+0000","last_clean":"2024-05-20T18:31:30.798564+0000","last_became_active":"2024-05-20T18:31:30.796247+0000","last_became_peered":"2024-05-20T18:31:30.796247+0000","last_unstale":"2024-05-20T18:31:30.798564+0000","last_undegraded":"2024-05-20T18:31:30.798564+0000","last_fullsized":"2024-05-20T18:31:30.798564+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:14:13.421305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.792726+0000","last_change":"2024-05-20T18:31:30.788824+0000","last_active":"2024-05-20T18:31:30.792726+0000","last_peered":"2024-05-20T18:31:30.792726+0000","last_clean":"2024-05-20T18:31:30.792726+0000","last_became_active":"2024-05-20T18:31:30.788481+0000","last_became_peered":"2024-05-20T18:31:30.788481+0000","last_unstale":"2024-05-20T18:31:30.792726+0000","last_undegraded":"2024-05-20T18:31:30.792726+0000","last_fullsized":"2024-05-20T18:31:30.792726+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:49:05.583126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.806899+0000","last_change":"2024-05-20T18:31:30.803916+0000","last_active":"2024-05-20T18:31:30.806899+0000","last_peered":"2024-05-20T18:31:30.806899+0000","last_clean":"2024-05-20T18:31:30.806899+0000","last_became_active":"2024-05-20T18:31:30.802543+0000","last_became_peered":"2024-05-20T18:31:30.802543+0000","last_unstale":"2024-05-20T18:31:30.806899+0000","last_undegraded":"2024-05-20T18:31:30.806899+0000","last_fullsized":"2024-05-20T18:31:30.806899+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:16:42.686921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.810740+0000","last_change":"2024-05-20T18:31:30.804864+0000","last_active":"2024-05-20T18:31:30.810740+0000","last_peered":"2024-05-20T18:31:30.810740+0000","last_clean":"2024-05-20T18:31:30.810740+0000","last_became_active":"2024-05-20T18:31:30.804513+0000","last_became_peered":"2024-05-20T18:31:30.804513+0000","last_unstale":"2024-05-20T18:31:30.810740+0000","last_undegraded":"2024-05-20T18:31:30.810740+0000","last_fullsized":"2024-05-20T18:31:30.810740+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:40:02.269716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.804370+0000","last_change":"2024-05-20T18:31:30.800518+0000","last_active":"2024-05-20T18:31:30.804370+0000","last_peered":"2024-05-20T18:31:30.804370+0000","last_clean":"2024-05-20T18:31:30.804370+0000","last_became_active":"2024-05-20T18:31:30.799574+0000","last_became_peered":"2024-05-20T18:31:30.799574+0000","last_unstale":"2024-05-20T18:31:30.804370+0000","last_undegraded":"2024-05-20T18:31:30.804370+0000","last_fullsized":"2024-05-20T18:31:30.804370+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:03:06.377489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.808291+0000","last_change":"2024-05-20T18:31:30.797496+0000","last_active":"2024-05-20T18:31:30.808291+0000","last_peered":"2024-05-20T18:31:30.808291+0000","last_clean":"2024-05-20T18:31:30.808291+0000","last_became_active":"2024-05-20T18:31:30.797187+0000","last_became_peered":"2024-05-20T18:31:30.797187+0000","last_unstale":"2024-05-20T18:31:30.808291+0000","last_undegraded":"2024-05-20T18:31:30.808291+0000","last_fullsized":"2024-05-20T18:31:30.808291+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:26:34.676034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"76'4","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.819554+0000","last_change":"2024-05-20T18:31:29.789722+0000","last_active":"2024-05-20T18:31:30.819554+0000","last_peered":"2024-05-20T18:31:30.819554+0000","last_clean":"2024-05-20T18:31:30.819554+0000","last_became_active":"2024-05-20T18:31:29.789393+0000","last_became_peered":"2024-05-20T18:31:29.789393+0000","last_unstale":"2024-05-20T18:31:30.819554+0000","last_undegraded":"2024-05-20T18:31:30.819554+0000","last_fullsized":"2024-05-20T18:31:30.819554+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:52:54.906881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.797778+0000","last_change":"2024-05-20T18:31:30.794764+0000","last_active":"2024-05-20T18:31:30.797778+0000","last_peered":"2024-05-20T18:31:30.797778+0000","last_clean":"2024-05-20T18:31:30.797778+0000","last_became_active":"2024-05-20T18:31:30.794455+0000","last_became_peered":"2024-05-20T18:31:30.794455+0000","last_unstale":"2024-05-20T18:31:30.797778+0000","last_undegraded":"2024-05-20T18:31:30.797778+0000","last_fullsized":"2024-05-20T18:31:30.797778+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:06:08.211253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"76'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.810984+0000","last_change":"2024-05-20T18:31:29.785992+0000","last_active":"2024-05-20T18:31:30.810984+0000","last_peered":"2024-05-20T18:31:30.810984+0000","last_clean":"2024-05-20T18:31:30.810984+0000","last_became_active":"2024-05-20T18:31:29.785657+0000","last_became_peered":"2024-05-20T18:31:29.785657+0000","last_unstale":"2024-05-20T18:31:30.810984+0000","last_undegraded":"2024-05-20T18:31:30.810984+0000","last_fullsized":"2024-05-20T18:31:30.810984+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:39:37.796995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.810990+0000","last_change":"2024-05-20T18:31:30.808624+0000","last_active":"2024-05-20T18:31:30.810990+0000","last_peered":"2024-05-20T18:31:30.810990+0000","last_clean":"2024-05-20T18:31:30.810990+0000","last_became_active":"2024-05-20T18:31:30.808299+0000","last_became_peered":"2024-05-20T18:31:30.808299+0000","last_unstale":"2024-05-20T18:31:30.810990+0000","last_undegraded":"2024-05-20T18:31:30.810990+0000","last_fullsized":"2024-05-20T18:31:30.810990+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:05:46.443301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"76'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.814892+0000","last_change":"2024-05-20T18:31:29.789603+0000","last_active":"2024-05-20T18:31:30.814892+0000","last_peered":"2024-05-20T18:31:30.814892+0000","last_clean":"2024-05-20T18:31:30.814892+0000","last_became_active":"2024-05-20T18:31:29.789259+0000","last_became_peered":"2024-05-20T18:31:29.789259+0000","last_unstale":"2024-05-20T18:31:30.814892+0000","last_undegraded":"2024-05-20T18:31:30.814892+0000","last_fullsized":"2024-05-20T18:31:30.814892+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:05:21.117562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.813288+0000","last_change":"2024-05-20T18:31:30.810923+0000","last_active":"2024-05-20T18:31:30.813288+0000","last_peered":"2024-05-20T18:31:30.813288+0000","last_clean":"2024-05-20T18:31:30.813288+0000","last_became_active":"2024-05-20T18:31:30.810553+0000","last_became_peered":"2024-05-20T18:31:30.810553+0000","last_unstale":"2024-05-20T18:31:30.813288+0000","last_undegraded":"2024-05-20T18:31:30.813288+0000","last_fullsized":"2024-05-20T18:31:30.813288+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:12:17.272988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"76'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.813821+0000","last_change":"2024-05-20T18:31:29.788526+0000","last_active":"2024-05-20T18:31:30.813821+0000","last_peered":"2024-05-20T18:31:30.813821+0000","last_clean":"2024-05-20T18:31:30.813821+0000","last_became_active":"2024-05-20T18:31:29.788171+0000","last_became_peered":"2024-05-20T18:31:29.788171+0000","last_unstale":"2024-05-20T18:31:30.813821+0000","last_undegraded":"2024-05-20T18:31:30.813821+0000","last_fullsized":"2024-05-20T18:31:30.813821+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:48:31.385853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.799934+0000","last_change":"2024-05-20T18:31:30.798539+0000","last_active":"2024-05-20T18:31:30.799934+0000","last_peered":"2024-05-20T18:31:30.799934+0000","last_clean":"2024-05-20T18:31:30.799934+0000","last_became_active":"2024-05-20T18:31:30.798211+0000","last_became_peered":"2024-05-20T18:31:30.798211+0000","last_unstale":"2024-05-20T18:31:30.799934+0000","last_undegraded":"2024-05-20T18:31:30.799934+0000","last_fullsized":"2024-05-20T18:31:30.799934+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:34:43.715283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"76'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.812985+0000","last_change":"2024-05-20T18:31:29.784698+0000","last_active":"2024-05-20T18:31:30.812985+0000","last_peered":"2024-05-20T18:31:30.812985+0000","last_clean":"2024-05-20T18:31:30.812985+0000","last_became_active":"2024-05-20T18:31:29.784363+0000","last_became_peered":"2024-05-20T18:31:29.784363+0000","last_unstale":"2024-05-20T18:31:30.812985+0000","last_undegraded":"2024-05-20T18:31:30.812985+0000","last_fullsized":"2024-05-20T18:31:30.812985+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:45:50.185805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.814470+0000","last_change":"2024-05-20T18:31:30.803690+0000","last_active":"2024-05-20T18:31:30.814470+0000","last_peered":"2024-05-20T18:31:30.814470+0000","last_clean":"2024-05-20T18:31:30.814470+0000","last_became_active":"2024-05-20T18:31:30.803385+0000","last_became_peered":"2024-05-20T18:31:30.803385+0000","last_unstale":"2024-05-20T18:31:30.814470+0000","last_undegraded":"2024-05-20T18:31:30.814470+0000","last_fullsized":"2024-05-20T18:31:30.814470+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:36:09.353546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"76'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.814290+0000","last_change":"2024-05-20T18:31:29.789808+0000","last_active":"2024-05-20T18:31:30.814290+0000","last_peered":"2024-05-20T18:31:30.814290+0000","last_clean":"2024-05-20T18:31:30.814290+0000","last_became_active":"2024-05-20T18:31:29.789454+0000","last_became_peered":"2024-05-20T18:31:29.789454+0000","last_unstale":"2024-05-20T18:31:30.814290+0000","last_undegraded":"2024-05-20T18:31:30.814290+0000","last_fullsized":"2024-05-20T18:31:30.814290+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:28:46.277507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.811172+0000","last_change":"2024-05-20T18:31:30.808285+0000","last_active":"2024-05-20T18:31:30.811172+0000","last_peered":"2024-05-20T18:31:30.811172+0000","last_clean":"2024-05-20T18:31:30.811172+0000","last_became_active":"2024-05-20T18:31:30.807793+0000","last_became_peered":"2024-05-20T18:31:30.807793+0000","last_unstale":"2024-05-20T18:31:30.811172+0000","last_undegraded":"2024-05-20T18:31:30.811172+0000","last_fullsized":"2024-05-20T18:31:30.811172+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:57:46.710550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.781343+0000","last_change":"2024-05-20T18:31:29.795340+0000","last_active":"2024-05-20T18:31:30.781343+0000","last_peered":"2024-05-20T18:31:30.781343+0000","last_clean":"2024-05-20T18:31:30.781343+0000","last_became_active":"2024-05-20T18:31:29.795020+0000","last_became_peered":"2024-05-20T18:31:29.795020+0000","last_unstale":"2024-05-20T18:31:30.781343+0000","last_undegraded":"2024-05-20T18:31:30.781343+0000","last_fullsized":"2024-05-20T18:31:30.781343+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:09:35.144930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.803496+0000","last_change":"2024-05-20T18:31:30.801956+0000","last_active":"2024-05-20T18:31:30.803496+0000","last_peered":"2024-05-20T18:31:30.803496+0000","last_clean":"2024-05-20T18:31:30.803496+0000","last_became_active":"2024-05-20T18:31:30.801649+0000","last_became_peered":"2024-05-20T18:31:30.801649+0000","last_unstale":"2024-05-20T18:31:30.803496+0000","last_undegraded":"2024-05-20T18:31:30.803496+0000","last_fullsized":"2024-05-20T18:31:30.803496+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:18:35.214658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.787618+0000","last_change":"2024-05-20T18:31:29.791538+0000","last_active":"2024-05-20T18:31:30.787618+0000","last_peered":"2024-05-20T18:31:30.787618+0000","last_clean":"2024-05-20T18:31:30.787618+0000","last_became_active":"2024-05-20T18:31:29.791228+0000","last_became_peered":"2024-05-20T18:31:29.791228+0000","last_unstale":"2024-05-20T18:31:30.787618+0000","last_undegraded":"2024-05-20T18:31:30.787618+0000","last_fullsized":"2024-05-20T18:31:30.787618+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:18:04.510834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.796899+0000","last_change":"2024-05-20T18:31:30.794339+0000","last_active":"2024-05-20T18:31:30.796899+0000","last_peered":"2024-05-20T18:31:30.796899+0000","last_clean":"2024-05-20T18:31:30.796899+0000","last_became_active":"2024-05-20T18:31:30.794018+0000","last_became_peered":"2024-05-20T18:31:30.794018+0000","last_unstale":"2024-05-20T18:31:30.796899+0000","last_undegraded":"2024-05-20T18:31:30.796899+0000","last_fullsized":"2024-05-20T18:31:30.796899+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:39:38.803158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.795526+0000","last_change":"2024-05-20T18:31:29.795246+0000","last_active":"2024-05-20T18:31:30.795526+0000","last_peered":"2024-05-20T18:31:30.795526+0000","last_clean":"2024-05-20T18:31:30.795526+0000","last_became_active":"2024-05-20T18:31:29.794822+0000","last_became_peered":"2024-05-20T18:31:29.794822+0000","last_unstale":"2024-05-20T18:31:30.795526+0000","last_undegraded":"2024-05-20T18:31:30.795526+0000","last_fullsized":"2024-05-20T18:31:30.795526+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:06:07.205090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.803145+0000","last_change":"2024-05-20T18:31:30.796802+0000","last_active":"2024-05-20T18:31:30.803145+0000","last_peered":"2024-05-20T18:31:30.803145+0000","last_clean":"2024-05-20T18:31:30.803145+0000","last_became_active":"2024-05-20T18:31:30.796451+0000","last_became_peered":"2024-05-20T18:31:30.796451+0000","last_unstale":"2024-05-20T18:31:30.803145+0000","last_undegraded":"2024-05-20T18:31:30.803145+0000","last_fullsized":"2024-05-20T18:31:30.803145+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:49:05.583126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.790571+0000","last_change":"2024-05-20T18:31:29.792224+0000","last_active":"2024-05-20T18:31:30.790571+0000","last_peered":"2024-05-20T18:31:30.790571+0000","last_clean":"2024-05-20T18:31:30.790571+0000","last_became_active":"2024-05-20T18:31:29.791891+0000","last_became_peered":"2024-05-20T18:31:29.791891+0000","last_unstale":"2024-05-20T18:31:30.790571+0000","last_undegraded":"2024-05-20T18:31:30.790571+0000","last_fullsized":"2024-05-20T18:31:30.790571+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:09:35.144930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.809300+0000","last_change":"2024-05-20T18:31:30.807512+0000","last_active":"2024-05-20T18:31:30.809300+0000","last_peered":"2024-05-20T18:31:30.809300+0000","last_clean":"2024-05-20T18:31:30.809300+0000","last_became_active":"2024-05-20T18:31:30.807194+0000","last_became_peered":"2024-05-20T18:31:30.807194+0000","last_unstale":"2024-05-20T18:31:30.809300+0000","last_undegraded":"2024-05-20T18:31:30.809300+0000","last_fullsized":"2024-05-20T18:31:30.809300+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:38:20.581222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"76'3","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.819919+0000","last_change":"2024-05-20T18:31:29.785846+0000","last_active":"2024-05-20T18:31:30.819919+0000","last_peered":"2024-05-20T18:31:30.819919+0000","last_clean":"2024-05-20T18:31:30.819919+0000","last_became_active":"2024-05-20T18:31:29.785526+0000","last_became_peered":"2024-05-20T18:31:29.785526+0000","last_unstale":"2024-05-20T18:31:30.819919+0000","last_undegraded":"2024-05-20T18:31:30.819919+0000","last_fullsized":"2024-05-20T18:31:30.819919+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:49:08.327229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.810691+0000","last_change":"2024-05-20T18:31:30.802361+0000","last_active":"2024-05-20T18:31:30.810691+0000","last_peered":"2024-05-20T18:31:30.810691+0000","last_clean":"2024-05-20T18:31:30.810691+0000","last_became_active":"2024-05-20T18:31:30.802022+0000","last_became_peered":"2024-05-20T18:31:30.802022+0000","last_unstale":"2024-05-20T18:31:30.810691+0000","last_undegraded":"2024-05-20T18:31:30.810691+0000","last_fullsized":"2024-05-20T18:31:30.810691+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:04:32.739946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"76'6","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.821504+0000","last_change":"2024-05-20T18:31:29.785804+0000","last_active":"2024-05-20T18:31:30.821504+0000","last_peered":"2024-05-20T18:31:30.821504+0000","last_clean":"2024-05-20T18:31:30.821504+0000","last_became_active":"2024-05-20T18:31:29.785452+0000","last_became_peered":"2024-05-20T18:31:29.785452+0000","last_unstale":"2024-05-20T18:31:30.821504+0000","last_undegraded":"2024-05-20T18:31:30.821504+0000","last_fullsized":"2024-05-20T18:31:30.821504+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:05:45.437138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.808856+0000","last_change":"2024-05-20T18:31:30.805529+0000","last_active":"2024-05-20T18:31:30.808856+0000","last_peered":"2024-05-20T18:31:30.808856+0000","last_clean":"2024-05-20T18:31:30.808856+0000","last_became_active":"2024-05-20T18:31:30.805197+0000","last_became_peered":"2024-05-20T18:31:30.805197+0000","last_unstale":"2024-05-20T18:31:30.808856+0000","last_undegraded":"2024-05-20T18:31:30.808856+0000","last_fullsized":"2024-05-20T18:31:30.808856+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:06:08.211253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"76'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.813615+0000","last_change":"2024-05-20T18:31:29.785826+0000","last_active":"2024-05-20T18:31:30.813615+0000","last_peered":"2024-05-20T18:31:30.813615+0000","last_clean":"2024-05-20T18:31:30.813615+0000","last_became_active":"2024-05-20T18:31:29.785475+0000","last_became_peered":"2024-05-20T18:31:29.785475+0000","last_unstale":"2024-05-20T18:31:30.813615+0000","last_undegraded":"2024-05-20T18:31:30.813615+0000","last_fullsized":"2024-05-20T18:31:30.813615+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:47:45.282380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.812154+0000","last_change":"2024-05-20T18:31:30.807788+0000","last_active":"2024-05-20T18:31:30.812154+0000","last_peered":"2024-05-20T18:31:30.812154+0000","last_clean":"2024-05-20T18:31:30.812154+0000","last_became_active":"2024-05-20T18:31:30.807478+0000","last_became_peered":"2024-05-20T18:31:30.807478+0000","last_unstale":"2024-05-20T18:31:30.812154+0000","last_undegraded":"2024-05-20T18:31:30.812154+0000","last_fullsized":"2024-05-20T18:31:30.812154+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:29:12.378846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.793841+0000","last_change":"2024-05-20T18:31:29.787484+0000","last_active":"2024-05-20T18:31:30.793841+0000","last_peered":"2024-05-20T18:31:30.793841+0000","last_clean":"2024-05-20T18:31:30.793841+0000","last_became_active":"2024-05-20T18:31:29.787019+0000","last_became_peered":"2024-05-20T18:31:29.787019+0000","last_unstale":"2024-05-20T18:31:30.793841+0000","last_undegraded":"2024-05-20T18:31:30.793841+0000","last_fullsized":"2024-05-20T18:31:30.793841+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:49:04.576963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.802373+0000","last_change":"2024-05-20T18:31:30.798213+0000","last_active":"2024-05-20T18:31:30.802373+0000","last_peered":"2024-05-20T18:31:30.802373+0000","last_clean":"2024-05-20T18:31:30.802373+0000","last_became_active":"2024-05-20T18:31:30.797842+0000","last_became_peered":"2024-05-20T18:31:30.797842+0000","last_unstale":"2024-05-20T18:31:30.802373+0000","last_undegraded":"2024-05-20T18:31:30.802373+0000","last_fullsized":"2024-05-20T18:31:30.802373+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:26:34.676034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.788467+0000","last_change":"2024-05-20T18:31:29.789763+0000","last_active":"2024-05-20T18:31:30.788467+0000","last_peered":"2024-05-20T18:31:30.788467+0000","last_clean":"2024-05-20T18:31:30.788467+0000","last_became_active":"2024-05-20T18:31:29.789425+0000","last_became_peered":"2024-05-20T18:31:29.789425+0000","last_unstale":"2024-05-20T18:31:30.788467+0000","last_undegraded":"2024-05-20T18:31:30.788467+0000","last_fullsized":"2024-05-20T18:31:30.788467+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:08:40.351642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.809880+0000","last_change":"2024-05-20T18:31:30.805626+0000","last_active":"2024-05-20T18:31:30.809880+0000","last_peered":"2024-05-20T18:31:30.809880+0000","last_clean":"2024-05-20T18:31:30.809880+0000","last_became_active":"2024-05-20T18:31:30.805260+0000","last_became_peered":"2024-05-20T18:31:30.805260+0000","last_unstale":"2024-05-20T18:31:30.809880+0000","last_undegraded":"2024-05-20T18:31:30.809880+0000","last_fullsized":"2024-05-20T18:31:30.809880+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:04:32.739946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"76'8","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.817404+0000","last_change":"2024-05-20T18:31:29.797682+0000","last_active":"2024-05-20T18:31:30.817404+0000","last_peered":"2024-05-20T18:31:30.817404+0000","last_clean":"2024-05-20T18:31:30.817404+0000","last_became_active":"2024-05-20T18:31:29.797386+0000","last_became_peered":"2024-05-20T18:31:29.797386+0000","last_unstale":"2024-05-20T18:31:30.817404+0000","last_undegraded":"2024-05-20T18:31:30.817404+0000","last_fullsized":"2024-05-20T18:31:30.817404+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:05:21.117562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.805607+0000","last_change":"2024-05-20T18:31:30.802154+0000","last_active":"2024-05-20T18:31:30.805607+0000","last_peered":"2024-05-20T18:31:30.805607+0000","last_clean":"2024-05-20T18:31:30.805607+0000","last_became_active":"2024-05-20T18:31:30.801704+0000","last_became_peered":"2024-05-20T18:31:30.801704+0000","last_unstale":"2024-05-20T18:31:30.805607+0000","last_undegraded":"2024-05-20T18:31:30.805607+0000","last_fullsized":"2024-05-20T18:31:30.805607+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:20:13.702810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"76'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.813089+0000","last_change":"2024-05-20T18:31:29.787007+0000","last_active":"2024-05-20T18:31:30.813089+0000","last_peered":"2024-05-20T18:31:30.813089+0000","last_clean":"2024-05-20T18:31:30.813089+0000","last_became_active":"2024-05-20T18:31:29.786624+0000","last_became_peered":"2024-05-20T18:31:29.786624+0000","last_unstale":"2024-05-20T18:31:30.813089+0000","last_undegraded":"2024-05-20T18:31:30.813089+0000","last_fullsized":"2024-05-20T18:31:30.813089+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:43:03.024863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.806889+0000","last_change":"2024-05-20T18:31:30.799848+0000","last_active":"2024-05-20T18:31:30.806889+0000","last_peered":"2024-05-20T18:31:30.806889+0000","last_clean":"2024-05-20T18:31:30.806889+0000","last_became_active":"2024-05-20T18:31:30.799548+0000","last_became_peered":"2024-05-20T18:31:30.799548+0000","last_unstale":"2024-05-20T18:31:30.806889+0000","last_undegraded":"2024-05-20T18:31:30.806889+0000","last_fullsized":"2024-05-20T18:31:30.806889+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:52:17.996601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"76'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.817886+0000","last_change":"2024-05-20T18:31:29.782077+0000","last_active":"2024-05-20T18:31:30.817886+0000","last_peered":"2024-05-20T18:31:30.817886+0000","last_clean":"2024-05-20T18:31:30.817886+0000","last_became_active":"2024-05-20T18:31:29.781723+0000","last_became_peered":"2024-05-20T18:31:29.781723+0000","last_unstale":"2024-05-20T18:31:30.817886+0000","last_undegraded":"2024-05-20T18:31:30.817886+0000","last_fullsized":"2024-05-20T18:31:30.817886+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:17:14.900208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.804795+0000","last_change":"2024-05-20T18:31:30.798880+0000","last_active":"2024-05-20T18:31:30.804795+0000","last_peered":"2024-05-20T18:31:30.804795+0000","last_clean":"2024-05-20T18:31:30.804795+0000","last_became_active":"2024-05-20T18:31:30.798532+0000","last_became_peered":"2024-05-20T18:31:30.798532+0000","last_unstale":"2024-05-20T18:31:30.804795+0000","last_undegraded":"2024-05-20T18:31:30.804795+0000","last_fullsized":"2024-05-20T18:31:30.804795+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:09:02.959983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"76'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.818585+0000","last_change":"2024-05-20T18:31:29.792065+0000","last_active":"2024-05-20T18:31:30.818585+0000","last_peered":"2024-05-20T18:31:30.818585+0000","last_clean":"2024-05-20T18:31:30.818585+0000","last_became_active":"2024-05-20T18:31:29.791740+0000","last_became_peered":"2024-05-20T18:31:29.791740+0000","last_unstale":"2024-05-20T18:31:30.818585+0000","last_undegraded":"2024-05-20T18:31:30.818585+0000","last_fullsized":"2024-05-20T18:31:30.818585+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:50:33.412770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.805679+0000","last_change":"2024-05-20T18:31:30.803445+0000","last_active":"2024-05-20T18:31:30.805679+0000","last_peered":"2024-05-20T18:31:30.805679+0000","last_clean":"2024-05-20T18:31:30.805679+0000","last_became_active":"2024-05-20T18:31:30.803037+0000","last_became_peered":"2024-05-20T18:31:30.803037+0000","last_unstale":"2024-05-20T18:31:30.805679+0000","last_undegraded":"2024-05-20T18:31:30.805679+0000","last_fullsized":"2024-05-20T18:31:30.805679+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:27:28.606942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.785548+0000","last_change":"2024-05-20T18:31:29.789338+0000","last_active":"2024-05-20T18:31:30.785548+0000","last_peered":"2024-05-20T18:31:30.785548+0000","last_clean":"2024-05-20T18:31:30.785548+0000","last_became_active":"2024-05-20T18:31:29.788897+0000","last_became_peered":"2024-05-20T18:31:29.788897+0000","last_unstale":"2024-05-20T18:31:30.785548+0000","last_undegraded":"2024-05-20T18:31:30.785548+0000","last_fullsized":"2024-05-20T18:31:30.785548+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:12:16.266825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.796178+0000","last_change":"2024-05-20T18:31:30.793864+0000","last_active":"2024-05-20T18:31:30.796178+0000","last_peered":"2024-05-20T18:31:30.796178+0000","last_clean":"2024-05-20T18:31:30.796178+0000","last_became_active":"2024-05-20T18:31:30.793532+0000","last_became_peered":"2024-05-20T18:31:30.793532+0000","last_unstale":"2024-05-20T18:31:30.796178+0000","last_undegraded":"2024-05-20T18:31:30.796178+0000","last_fullsized":"2024-05-20T18:31:30.796178+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:02:16.521927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"76'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.818508+0000","last_change":"2024-05-20T18:31:29.788731+0000","last_active":"2024-05-20T18:31:30.818508+0000","last_peered":"2024-05-20T18:31:30.818508+0000","last_clean":"2024-05-20T18:31:30.818508+0000","last_became_active":"2024-05-20T18:31:29.788375+0000","last_became_peered":"2024-05-20T18:31:29.788375+0000","last_unstale":"2024-05-20T18:31:30.818508+0000","last_undegraded":"2024-05-20T18:31:30.818508+0000","last_fullsized":"2024-05-20T18:31:30.818508+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:18:33.200826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.814287+0000","last_change":"2024-05-20T18:31:30.801556+0000","last_active":"2024-05-20T18:31:30.814287+0000","last_peered":"2024-05-20T18:31:30.814287+0000","last_clean":"2024-05-20T18:31:30.814287+0000","last_became_active":"2024-05-20T18:31:30.800294+0000","last_became_peered":"2024-05-20T18:31:30.800294+0000","last_unstale":"2024-05-20T18:31:30.814287+0000","last_undegraded":"2024-05-20T18:31:30.814287+0000","last_fullsized":"2024-05-20T18:31:30.814287+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:16:42.686921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.790112+0000","last_change":"2024-05-20T18:31:29.787690+0000","last_active":"2024-05-20T18:31:30.790112+0000","last_peered":"2024-05-20T18:31:30.790112+0000","last_clean":"2024-05-20T18:31:30.790112+0000","last_became_active":"2024-05-20T18:31:29.787018+0000","last_became_peered":"2024-05-20T18:31:29.787018+0000","last_unstale":"2024-05-20T18:31:30.790112+0000","last_undegraded":"2024-05-20T18:31:30.790112+0000","last_fullsized":"2024-05-20T18:31:30.790112+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:56:46.630704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.799407+0000","last_change":"2024-05-20T18:31:30.792475+0000","last_active":"2024-05-20T18:31:30.799407+0000","last_peered":"2024-05-20T18:31:30.799407+0000","last_clean":"2024-05-20T18:31:30.799407+0000","last_became_active":"2024-05-20T18:31:30.792120+0000","last_became_peered":"2024-05-20T18:31:30.792120+0000","last_unstale":"2024-05-20T18:31:30.799407+0000","last_undegraded":"2024-05-20T18:31:30.799407+0000","last_fullsized":"2024-05-20T18:31:30.799407+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:12:17.272988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.790910+0000","last_change":"2024-05-20T18:31:29.786166+0000","last_active":"2024-05-20T18:31:30.790910+0000","last_peered":"2024-05-20T18:31:30.790910+0000","last_clean":"2024-05-20T18:31:30.790910+0000","last_became_active":"2024-05-20T18:31:29.785819+0000","last_became_peered":"2024-05-20T18:31:29.785819+0000","last_unstale":"2024-05-20T18:31:30.790910+0000","last_undegraded":"2024-05-20T18:31:30.790910+0000","last_fullsized":"2024-05-20T18:31:30.790910+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:10:09.430550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.799106+0000","last_change":"2024-05-20T18:31:30.792824+0000","last_active":"2024-05-20T18:31:30.799106+0000","last_peered":"2024-05-20T18:31:30.799106+0000","last_clean":"2024-05-20T18:31:30.799106+0000","last_became_active":"2024-05-20T18:31:30.792457+0000","last_became_peered":"2024-05-20T18:31:30.792457+0000","last_unstale":"2024-05-20T18:31:30.799106+0000","last_undegraded":"2024-05-20T18:31:30.799106+0000","last_fullsized":"2024-05-20T18:31:30.799106+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:53:19.626439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"76'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.817886+0000","last_change":"2024-05-20T18:31:29.786504+0000","last_active":"2024-05-20T18:31:30.817886+0000","last_peered":"2024-05-20T18:31:30.817886+0000","last_clean":"2024-05-20T18:31:30.817886+0000","last_became_active":"2024-05-20T18:31:29.786144+0000","last_became_peered":"2024-05-20T18:31:29.786144+0000","last_unstale":"2024-05-20T18:31:30.817886+0000","last_undegraded":"2024-05-20T18:31:30.817886+0000","last_fullsized":"2024-05-20T18:31:30.817886+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:13:48.095566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.809700+0000","last_change":"2024-05-20T18:31:30.805782+0000","last_active":"2024-05-20T18:31:30.809700+0000","last_peered":"2024-05-20T18:31:30.809700+0000","last_clean":"2024-05-20T18:31:30.809700+0000","last_became_active":"2024-05-20T18:31:30.805381+0000","last_became_peered":"2024-05-20T18:31:30.805381+0000","last_unstale":"2024-05-20T18:31:30.809700+0000","last_undegraded":"2024-05-20T18:31:30.809700+0000","last_fullsized":"2024-05-20T18:31:30.809700+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:40:02.269716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.781462+0000","last_change":"2024-05-20T18:31:29.795801+0000","last_active":"2024-05-20T18:31:30.781462+0000","last_peered":"2024-05-20T18:31:30.781462+0000","last_clean":"2024-05-20T18:31:30.781462+0000","last_became_active":"2024-05-20T18:31:29.795105+0000","last_became_peered":"2024-05-20T18:31:29.795105+0000","last_unstale":"2024-05-20T18:31:30.781462+0000","last_undegraded":"2024-05-20T18:31:30.781462+0000","last_fullsized":"2024-05-20T18:31:30.781462+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:35:58.569124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.812459+0000","last_change":"2024-05-20T18:31:30.809542+0000","last_active":"2024-05-20T18:31:30.812459+0000","last_peered":"2024-05-20T18:31:30.812459+0000","last_clean":"2024-05-20T18:31:30.812459+0000","last_became_active":"2024-05-20T18:31:30.807732+0000","last_became_peered":"2024-05-20T18:31:30.807732+0000","last_unstale":"2024-05-20T18:31:30.812459+0000","last_undegraded":"2024-05-20T18:31:30.812459+0000","last_fullsized":"2024-05-20T18:31:30.812459+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:48:32.392016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.794249+0000","last_change":"2024-05-20T18:31:29.784997+0000","last_active":"2024-05-20T18:31:30.794249+0000","last_peered":"2024-05-20T18:31:30.794249+0000","last_clean":"2024-05-20T18:31:30.794249+0000","last_became_active":"2024-05-20T18:31:29.784681+0000","last_became_peered":"2024-05-20T18:31:29.784681+0000","last_unstale":"2024-05-20T18:31:30.794249+0000","last_undegraded":"2024-05-20T18:31:30.794249+0000","last_fullsized":"2024-05-20T18:31:30.794249+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:02:15.515764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.796752+0000","last_change":"2024-05-20T18:31:30.795220+0000","last_active":"2024-05-20T18:31:30.796752+0000","last_peered":"2024-05-20T18:31:30.796752+0000","last_clean":"2024-05-20T18:31:30.796752+0000","last_became_active":"2024-05-20T18:31:30.794915+0000","last_became_peered":"2024-05-20T18:31:30.794915+0000","last_unstale":"2024-05-20T18:31:30.796752+0000","last_undegraded":"2024-05-20T18:31:30.796752+0000","last_fullsized":"2024-05-20T18:31:30.796752+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:43:04.031026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.793178+0000","last_change":"2024-05-20T18:31:29.791935+0000","last_active":"2024-05-20T18:31:30.793178+0000","last_peered":"2024-05-20T18:31:30.793178+0000","last_clean":"2024-05-20T18:31:30.793178+0000","last_became_active":"2024-05-20T18:31:29.791219+0000","last_became_peered":"2024-05-20T18:31:29.791219+0000","last_unstale":"2024-05-20T18:31:30.793178+0000","last_undegraded":"2024-05-20T18:31:30.793178+0000","last_fullsized":"2024-05-20T18:31:30.793178+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:35:58.569124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.811970+0000","last_change":"2024-05-20T18:31:30.807004+0000","last_active":"2024-05-20T18:31:30.811970+0000","last_peered":"2024-05-20T18:31:30.811970+0000","last_clean":"2024-05-20T18:31:30.811970+0000","last_became_active":"2024-05-20T18:31:30.806468+0000","last_became_peered":"2024-05-20T18:31:30.806468+0000","last_unstale":"2024-05-20T18:31:30.811970+0000","last_undegraded":"2024-05-20T18:31:30.811970+0000","last_fullsized":"2024-05-20T18:31:30.811970+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:11:30.282843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.784333+0000","last_change":"2024-05-20T18:31:29.790063+0000","last_active":"2024-05-20T18:31:30.784333+0000","last_peered":"2024-05-20T18:31:30.784333+0000","last_clean":"2024-05-20T18:31:30.784333+0000","last_became_active":"2024-05-20T18:31:29.788253+0000","last_became_peered":"2024-05-20T18:31:29.788253+0000","last_unstale":"2024-05-20T18:31:30.784333+0000","last_undegraded":"2024-05-20T18:31:30.784333+0000","last_fullsized":"2024-05-20T18:31:30.784333+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:57:45.704387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.805961+0000","last_change":"2024-05-20T18:31:30.802951+0000","last_active":"2024-05-20T18:31:30.805961+0000","last_peered":"2024-05-20T18:31:30.805961+0000","last_clean":"2024-05-20T18:31:30.805961+0000","last_became_active":"2024-05-20T18:31:30.802589+0000","last_became_peered":"2024-05-20T18:31:30.802589+0000","last_unstale":"2024-05-20T18:31:30.805961+0000","last_undegraded":"2024-05-20T18:31:30.805961+0000","last_fullsized":"2024-05-20T18:31:30.805961+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:13:28.735527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.787880+0000","last_change":"2024-05-20T18:31:29.790390+0000","last_active":"2024-05-20T18:31:30.787880+0000","last_peered":"2024-05-20T18:31:30.787880+0000","last_clean":"2024-05-20T18:31:30.787880+0000","last_became_active":"2024-05-20T18:31:29.790092+0000","last_became_peered":"2024-05-20T18:31:29.790092+0000","last_unstale":"2024-05-20T18:31:30.787880+0000","last_undegraded":"2024-05-20T18:31:30.787880+0000","last_fullsized":"2024-05-20T18:31:30.787880+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:49:56.374138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.818078+0000","last_change":"2024-05-20T18:31:30.815761+0000","last_active":"2024-05-20T18:31:30.818078+0000","last_peered":"2024-05-20T18:31:30.818078+0000","last_clean":"2024-05-20T18:31:30.818078+0000","last_became_active":"2024-05-20T18:31:30.815454+0000","last_became_peered":"2024-05-20T18:31:30.815454+0000","last_unstale":"2024-05-20T18:31:30.818078+0000","last_undegraded":"2024-05-20T18:31:30.818078+0000","last_fullsized":"2024-05-20T18:31:30.818078+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:11:30.282843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.783760+0000","last_change":"2024-05-20T18:31:29.788645+0000","last_active":"2024-05-20T18:31:30.783760+0000","last_peered":"2024-05-20T18:31:30.783760+0000","last_clean":"2024-05-20T18:31:30.783760+0000","last_became_active":"2024-05-20T18:31:29.788309+0000","last_became_peered":"2024-05-20T18:31:29.788309+0000","last_unstale":"2024-05-20T18:31:30.783760+0000","last_undegraded":"2024-05-20T18:31:30.783760+0000","last_fullsized":"2024-05-20T18:31:30.783760+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:10:58.189779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.808313+0000","last_change":"2024-05-20T18:31:30.799115+0000","last_active":"2024-05-20T18:31:30.808313+0000","last_peered":"2024-05-20T18:31:30.808313+0000","last_clean":"2024-05-20T18:31:30.808313+0000","last_became_active":"2024-05-20T18:31:30.798804+0000","last_became_peered":"2024-05-20T18:31:30.798804+0000","last_unstale":"2024-05-20T18:31:30.808313+0000","last_undegraded":"2024-05-20T18:31:30.808313+0000","last_fullsized":"2024-05-20T18:31:30.808313+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:28:27.930145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.785221+0000","last_change":"2024-05-20T18:31:29.779068+0000","last_active":"2024-05-20T18:31:30.785221+0000","last_peered":"2024-05-20T18:31:30.785221+0000","last_clean":"2024-05-20T18:31:30.785221+0000","last_became_active":"2024-05-20T18:31:29.778707+0000","last_became_peered":"2024-05-20T18:31:29.778707+0000","last_unstale":"2024-05-20T18:31:30.785221+0000","last_undegraded":"2024-05-20T18:31:30.785221+0000","last_fullsized":"2024-05-20T18:31:30.785221+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:48:31.385853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.812321+0000","last_change":"2024-05-20T18:31:30.809305+0000","last_active":"2024-05-20T18:31:30.812321+0000","last_peered":"2024-05-20T18:31:30.812321+0000","last_clean":"2024-05-20T18:31:30.812321+0000","last_became_active":"2024-05-20T18:31:30.808975+0000","last_became_peered":"2024-05-20T18:31:30.808975+0000","last_unstale":"2024-05-20T18:31:30.812321+0000","last_undegraded":"2024-05-20T18:31:30.812321+0000","last_fullsized":"2024-05-20T18:31:30.812321+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:29.763889+0000","last_clean_scrub_stamp":"2024-05-20T18:31:29.763889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:38:41.987602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.794249+0000","last_change":"2024-05-20T18:31:29.786718+0000","last_active":"2024-05-20T18:31:30.794249+0000","last_peered":"2024-05-20T18:31:30.794249+0000","last_clean":"2024-05-20T18:31:30.794249+0000","last_became_active":"2024-05-20T18:31:29.786367+0000","last_became_peered":"2024-05-20T18:31:29.786367+0000","last_unstale":"2024-05-20T18:31:30.794249+0000","last_undegraded":"2024-05-20T18:31:30.794249+0000","last_fullsized":"2024-05-20T18:31:30.794249+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:31:28.757726+0000","last_clean_scrub_stamp":"2024-05-20T18:31:28.757726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:14:33.322262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":271,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-20T18:31:30.791666+0000","last_change":"2024-05-20T18:27:03.765694+0000","last_active":"2024-05-20T18:31:30.791666+0000","last_peered":"2024-05-20T18:31:30.791666+0000","last_clean":"2024-05-20T18:31:30.791666+0000","last_became_active":"2024-05-20T18:27:03.764695+0000","last_became_peered":"2024-05-20T18:27:03.764695+0000","last_unstale":"2024-05-20T18:31:30.791666+0000","last_undegraded":"2024-05-20T18:31:30.791666+0000","last_fullsized":"2024-05-20T18:31:30.791666+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_clean_scrub_stamp":"2024-05-20T18:27:02.750853+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:30:04.407487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574906,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1511424,"data_stored":1005734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574904,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":942080,"data_stored":424145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574904,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":933888,"data_stored":422567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574904,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":917504,"data_stored":415366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574906,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339744,"statfs":{"total":96636764160,"available":96603897856,"internally_reserved":0,"allocated":933888,"data_stored":423115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574905,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":917504,"data_stored":415366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":917504,"data_stored":415366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574905,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1523712,"data_stored":1012901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T18:31:35.222 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-20T18:31:35.223 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-20T18:31:35.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.385+0000 7f859c6d7640 1 Processor -- start 2024-05-20T18:31:35.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.385+0000 7f859c6d7640 1 -- start start 2024-05-20T18:31:35.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.387+0000 7f859c6d7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85940cd710 0x7f8594162e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:35.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.387+0000 7f859c6d7640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f859406d6b0 0x7f859415f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:35.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.387+0000 7f859c6d7640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f859406c8c0 con 0x7f85940cf3d0 2024-05-20T18:31:35.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.387+0000 7f859c6d7640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f85940d0ba0 con 0x7f859406d6b0 2024-05-20T18:31:35.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.387+0000 7f859c6d7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f859406dac0 con 0x7f85940cd710 2024-05-20T18:31:35.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.387+0000 7f8599c72640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f859406d6b0 0x7f859415f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:35.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.387+0000 7f8599c72640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f859406d6b0 0x7f859415f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57980/0 (socket says 172.21.15.5:57980) 2024-05-20T18:31:35.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.387+0000 7f8599c72640 1 -- 172.21.15.5:0/1157714632 learned_addr learned my addr 172.21.15.5:0/1157714632 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:35.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.387+0000 7f859a473640 1 --2- 172.21.15.5:0/1157714632 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85940cd710 0x7f8594162e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:35.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.387+0000 7f8599c72640 1 -- 172.21.15.5:0/1157714632 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85940cd710 msgr2=0x7f8594162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:35.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.387+0000 7f8599c72640 1 --2- 172.21.15.5:0/1157714632 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85940cd710 0x7f8594162e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:35.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.387+0000 7f8599c72640 1 -- 172.21.15.5:0/1157714632 >> v1:172.21.15.63:6789/0 conn(0x7f85940cf3d0 legacy=0x7f85940cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:35.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.387+0000 7f8599c72640 1 -- 172.21.15.5:0/1157714632 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85941633e0 con 0x7f859406d6b0 2024-05-20T18:31:35.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.388+0000 7f859a473640 1 --2- 172.21.15.5:0/1157714632 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85940cd710 0x7f8594162e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:35.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.388+0000 7f8599c72640 1 --2- 172.21.15.5:0/1157714632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f859406d6b0 0x7f859415f2b0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f85940d0d20 tx=0x7f858802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1457a223394d5f48 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.388+0000 7f8599471640 1 -- 172.21.15.5:0/1157714632 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588037070 con 0x7f859406d6b0 2024-05-20T18:31:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.388+0000 7f8599471640 1 -- 172.21.15.5:0/1157714632 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8588032440 con 0x7f859406d6b0 2024-05-20T18:31:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.388+0000 7f8599471640 1 -- 172.21.15.5:0/1157714632 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588032810 con 0x7f859406d6b0 2024-05-20T18:31:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.389+0000 7f859c6d7640 1 -- 172.21.15.5:0/1157714632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f859406d6b0 msgr2=0x7f859415f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.389+0000 7f859c6d7640 1 --2- 172.21.15.5:0/1157714632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f859406d6b0 0x7f859415f2b0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f85940d0d20 tx=0x7f858802a250 comp rx=0 tx=0).stop 2024-05-20T18:31:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.389+0000 7f859c6d7640 1 -- 172.21.15.5:0/1157714632 shutdown_connections 2024-05-20T18:31:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.389+0000 7f859c6d7640 1 --2- 172.21.15.5:0/1157714632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f859406d6b0 0x7f859415f2b0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.389+0000 7f859c6d7640 1 --2- 172.21.15.5:0/1157714632 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85940cd710 0x7f8594162e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.389+0000 7f859c6d7640 1 -- 172.21.15.5:0/1157714632 >> 172.21.15.5:0/1157714632 conn(0x7f85940b44d0 msgr2=0x7f85940b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.389+0000 7f859c6d7640 1 -- 172.21.15.5:0/1157714632 shutdown_connections 2024-05-20T18:31:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.389+0000 7f859c6d7640 1 -- 172.21.15.5:0/1157714632 wait complete. 2024-05-20T18:31:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.391+0000 7f859c6d7640 1 Processor -- start 2024-05-20T18:31:35.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.391+0000 7f859c6d7640 1 -- start start 2024-05-20T18:31:35.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.392+0000 7f859c6d7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f859406d6b0 0x7f85940b7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:35.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.392+0000 7f859c6d7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85940cd710 0x7f85941f5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:35.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.392+0000 7f859c6d7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85940cf3d0 0x7f85941f5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:35.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.392+0000 7f859c6d7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8594163c80 con 0x7f859406d6b0 2024-05-20T18:31:35.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.392+0000 7f859c6d7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f85941639a0 con 0x7f85940cd710 2024-05-20T18:31:35.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.392+0000 7f859c6d7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f85940d1030 con 0x7f85940cf3d0 2024-05-20T18:31:35.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.392+0000 7f859a473640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f859406d6b0 0x7f85940b7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:35.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.392+0000 7f859a473640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f859406d6b0 0x7f85940b7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57988/0 (socket says 172.21.15.5:57988) 2024-05-20T18:31:35.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.392+0000 7f859a473640 1 -- 172.21.15.5:0/3021941553 learned_addr learned my addr 172.21.15.5:0/3021941553 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:35.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.393+0000 7f8599c72640 1 --2- 172.21.15.5:0/3021941553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85940cd710 0x7f85941f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:35.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.393+0000 7f859ac74640 1 --2- 172.21.15.5:0/3021941553 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85940cf3d0 0x7f85941f5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.393+0000 7f859a473640 1 -- 172.21.15.5:0/3021941553 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85940cf3d0 msgr2=0x7f85941f5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.393+0000 7f859a473640 1 --2- 172.21.15.5:0/3021941553 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85940cf3d0 0x7f85941f5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.393+0000 7f859a473640 1 -- 172.21.15.5:0/3021941553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85940cd710 msgr2=0x7f85941f5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.393+0000 7f859a473640 1 --2- 172.21.15.5:0/3021941553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85940cd710 0x7f85941f5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.393+0000 7f859a473640 1 -- 172.21.15.5:0/3021941553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85940b68b0 con 0x7f859406d6b0 2024-05-20T18:31:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.393+0000 7f859ac74640 1 --2- 172.21.15.5:0/3021941553 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85940cf3d0 0x7f85941f5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.393+0000 7f859a473640 1 --2- 172.21.15.5:0/3021941553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f859406d6b0 0x7f85940b7490 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f8584007e40 tx=0x7f858400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.393+0000 7f8592ffd640 1 -- 172.21.15.5:0/3021941553 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8584014620 con 0x7f859406d6b0 2024-05-20T18:31:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.393+0000 7f8592ffd640 1 -- 172.21.15.5:0/3021941553 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8584019070 con 0x7f859406d6b0 2024-05-20T18:31:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.393+0000 7f859c6d7640 1 -- 172.21.15.5:0/3021941553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85940ca3f0 con 0x7f859406d6b0 2024-05-20T18:31:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.393+0000 7f8592ffd640 1 -- 172.21.15.5:0/3021941553 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8584014a00 con 0x7f859406d6b0 2024-05-20T18:31:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.393+0000 7f859c6d7640 1 -- 172.21.15.5:0/3021941553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85940ca970 con 0x7f859406d6b0 2024-05-20T18:31:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.394+0000 7f859c6d7640 1 -- 172.21.15.5:0/3021941553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8594075b00 con 0x7f859406d6b0 2024-05-20T18:31:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.395+0000 7f8592ffd640 1 -- 172.21.15.5:0/3021941553 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8584014ba0 con 0x7f859406d6b0 2024-05-20T18:31:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.396+0000 7f8592ffd640 1 --2- 172.21.15.5:0/3021941553 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f858006be60 0x7f858006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.396+0000 7f8599c72640 1 --2- 172.21.15.5:0/3021941553 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f858006be60 0x7f858006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.396+0000 7f8592ffd640 1 -- 172.21.15.5:0/3021941553 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6101+0+0 (secure 0 0 0) 0x7f858408b720 con 0x7f859406d6b0 2024-05-20T18:31:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.396+0000 7f8599c72640 1 --2- 172.21.15.5:0/3021941553 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f858006be60 0x7f858006e370 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8588034000 tx=0x7f858802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:35.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.401+0000 7f8592ffd640 1 -- 172.21.15.5:0/3021941553 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8584059f40 con 0x7f859406d6b0 2024-05-20T18:31:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.653+0000 7f859c6d7640 1 -- 172.21.15.5:0/3021941553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f85940b8940 con 0x7f859406d6b0 2024-05-20T18:31:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.656+0000 7f8592ffd640 1 -- 172.21.15.5:0/3021941553 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7f8584059840 con 0x7f859406d6b0 2024-05-20T18:31:35.656 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2024-05-20T18:31:35.657 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDFlUtmevaaIhAATIwyXzr0vGT38ZmcMqR2gA== 2024-05-20T18:31:35.657 INFO:teuthology.orchestra.run.smithi005.stdout: caps mds = "allow" 2024-05-20T18:31:35.657 INFO:teuthology.orchestra.run.smithi005.stdout: caps mon = "allow r" 2024-05-20T18:31:35.657 INFO:teuthology.orchestra.run.smithi005.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-20T18:31:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:updated caps for client.0 2024-05-20T18:31:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.658+0000 7f859c6d7640 1 -- 172.21.15.5:0/3021941553 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f858006be60 msgr2=0x7f858006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.658+0000 7f859c6d7640 1 --2- 172.21.15.5:0/3021941553 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f858006be60 0x7f858006e370 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8588034000 tx=0x7f858802e000 comp rx=0 tx=0).stop 2024-05-20T18:31:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.658+0000 7f859c6d7640 1 -- 172.21.15.5:0/3021941553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f859406d6b0 msgr2=0x7f85940b7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.658+0000 7f859c6d7640 1 --2- 172.21.15.5:0/3021941553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f859406d6b0 0x7f85940b7490 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f8584007e40 tx=0x7f858400e4b0 comp rx=0 tx=0).stop 2024-05-20T18:31:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.658+0000 7f859c6d7640 1 -- 172.21.15.5:0/3021941553 shutdown_connections 2024-05-20T18:31:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.658+0000 7f859c6d7640 1 --2- 172.21.15.5:0/3021941553 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f858006be60 0x7f858006e370 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.658+0000 7f859c6d7640 1 --2- 172.21.15.5:0/3021941553 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85940cf3d0 0x7f85941f5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.658+0000 7f859c6d7640 1 --2- 172.21.15.5:0/3021941553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85940cd710 0x7f85941f5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.658+0000 7f859c6d7640 1 --2- 172.21.15.5:0/3021941553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f859406d6b0 0x7f85940b7490 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.658+0000 7f859c6d7640 1 -- 172.21.15.5:0/3021941553 >> 172.21.15.5:0/3021941553 conn(0x7f85940b44d0 msgr2=0x7f85940b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.658+0000 7f859c6d7640 1 -- 172.21.15.5:0/3021941553 shutdown_connections 2024-05-20T18:31:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.658+0000 7f859c6d7640 1 -- 172.21.15.5:0/3021941553 wait complete. 2024-05-20T18:31:35.670 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:31:35.761 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-20T18:31:35.761+0000 7f1d343e7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T18:31:35.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.789+0000 7fa58f651640 1 Processor -- start 2024-05-20T18:31:35.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.789+0000 7fa58f651640 1 -- start start 2024-05-20T18:31:35.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.790+0000 7fa58f651640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa58806d3d0 0x7fa58806d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:35.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.790+0000 7fa58f651640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa58806dd20 con 0x7fa588162820 2024-05-20T18:31:35.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.790+0000 7fa58f651640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5880d0630 con 0x7fa5880b8190 2024-05-20T18:31:35.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.790+0000 7fa58f651640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa5880d39c0 con 0x7fa58806d3d0 2024-05-20T18:31:35.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.790+0000 7fa58d3ed640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa588162820 0x7fa5880b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56864/0 (socket says 172.21.15.5:56864) 2024-05-20T18:31:35.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.790+0000 7fa58d3ed640 1 -- 172.21.15.5:0/1308105539 learned_addr learned my addr 172.21.15.5:0/1308105539 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:35.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.791+0000 7fa58cbec640 1 --2- 172.21.15.5:0/1308105539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa58806d3d0 0x7fa58806d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:35.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.791+0000 7fa587fff640 1 -- 172.21.15.5:0/1308105539 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3944757857 0 0) 0x7fa578001490 con 0x7fa588162820 2024-05-20T18:31:35.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.791+0000 7fa587fff640 1 -- 172.21.15.5:0/1308105539 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa570003520 con 0x7fa588162820 2024-05-20T18:31:35.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.791+0000 7fa587fff640 1 -- 172.21.15.5:0/1308105539 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3388232339 0 0) 0x7fa5880d0630 con 0x7fa5880b8190 2024-05-20T18:31:35.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.791+0000 7fa587fff640 1 -- 172.21.15.5:0/1308105539 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa578001490 con 0x7fa5880b8190 2024-05-20T18:31:35.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.791+0000 7fa587fff640 1 -- 172.21.15.5:0/1308105539 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1090935017 0 0) 0x7fa570003520 con 0x7fa588162820 2024-05-20T18:31:35.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.791+0000 7fa587fff640 1 -- 172.21.15.5:0/1308105539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa58806d3d0 msgr2=0x7fa58806d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:35.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.791+0000 7fa587fff640 1 --2- 172.21.15.5:0/1308105539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa58806d3d0 0x7fa58806d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:35.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.791+0000 7fa587fff640 1 -- 172.21.15.5:0/1308105539 >> v1:172.21.15.63:6789/0 conn(0x7fa5880b8190 legacy=0x7fa5880b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:35.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.791+0000 7fa587fff640 1 -- 172.21.15.5:0/1308105539 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5880d3ca0 con 0x7fa588162820 2024-05-20T18:31:35.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.791+0000 7fa587fff640 1 -- 172.21.15.5:0/1308105539 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7fa578001bc0 con 0x7fa588162820 2024-05-20T18:31:35.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.791+0000 7fa58cbec640 1 --2- 172.21.15.5:0/1308105539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa58806d3d0 0x7fa58806d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:31:35.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.791+0000 7fa587fff640 1 -- 172.21.15.5:0/1308105539 >> v1:172.21.15.5:6789/0 conn(0x7fa588162820 legacy=0x7fa5880b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:35.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.791+0000 7fa587fff640 1 --2- 172.21.15.5:0/1308105539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa570003b70 0x7fa570003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:35.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.791+0000 7fa587fff640 1 --2- 172.21.15.5:0/1308105539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa58806d3d0 0x7fa570004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:35.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.791+0000 7fa587fff640 1 --2- 172.21.15.5:0/1308105539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa570004d40 0x7fa570005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:35.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.791+0000 7fa587fff640 1 -- 172.21.15.5:0/1308105539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa5880d39c0 con 0x7fa570004d40 2024-05-20T18:31:35.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.791+0000 7fa587fff640 1 -- 172.21.15.5:0/1308105539 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa57000baa0 con 0x7fa570003b70 2024-05-20T18:31:35.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.791+0000 7fa587fff640 1 -- 172.21.15.5:0/1308105539 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa57000bca0 con 0x7fa58806d3d0 2024-05-20T18:31:35.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.791+0000 7fa58d3ed640 1 --2- 172.21.15.5:0/1308105539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa570003b70 0x7fa570003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:35.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.791+0000 7fa58dbee640 1 --2- 172.21.15.5:0/1308105539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa570004d40 0x7fa570005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:35.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.791+0000 7fa58cbec640 1 --2- 172.21.15.5:0/1308105539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa58806d3d0 0x7fa570004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:35.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.791+0000 7fa58dbee640 1 -- 172.21.15.5:0/1308105539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa58806d3d0 msgr2=0x7fa570004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:35.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.792+0000 7fa58dbee640 1 --2- 172.21.15.5:0/1308105539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa58806d3d0 0x7fa570004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:35.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.792+0000 7fa58dbee640 1 -- 172.21.15.5:0/1308105539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa570003b70 msgr2=0x7fa570003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:35.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.792+0000 7fa58dbee640 1 --2- 172.21.15.5:0/1308105539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa570003b70 0x7fa570003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:35.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.792+0000 7fa58dbee640 1 -- 172.21.15.5:0/1308105539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa570003520 con 0x7fa570004d40 2024-05-20T18:31:35.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.792+0000 7fa58dbee640 1 --2- 172.21.15.5:0/1308105539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa570004d40 0x7fa570005150 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fa5800018a0 tx=0x7fa570005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc1e93fe8694d138 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:35.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.792+0000 7fa587fff640 1 -- 172.21.15.5:0/1308105539 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5800014f0 con 0x7fa570004d40 2024-05-20T18:31:35.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.792+0000 7fa587fff640 1 -- 172.21.15.5:0/1308105539 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5800052d0 con 0x7fa570004d40 2024-05-20T18:31:35.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.792+0000 7fa587fff640 1 -- 172.21.15.5:0/1308105539 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58002ac60 con 0x7fa570004d40 2024-05-20T18:31:35.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.793+0000 7fa58f651640 1 -- 172.21.15.5:0/1308105539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa570004d40 msgr2=0x7fa570005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:35.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.793+0000 7fa58f651640 1 --2- 172.21.15.5:0/1308105539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa570004d40 0x7fa570005150 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fa5800018a0 tx=0x7fa570005830 comp rx=0 tx=0).stop 2024-05-20T18:31:35.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.793+0000 7fa58dbee640 1 -- 172.21.15.5:0/1308105539 reap_dead start 2024-05-20T18:31:35.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.793+0000 7fa58f651640 1 -- 172.21.15.5:0/1308105539 shutdown_connections 2024-05-20T18:31:35.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.793+0000 7fa58f651640 1 -- 172.21.15.5:0/1308105539 >> 172.21.15.5:0/1308105539 conn(0x7fa5880d4bd0 msgr2=0x7fa5880d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:35.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.793+0000 7fa58f651640 1 -- 172.21.15.5:0/1308105539 shutdown_connections 2024-05-20T18:31:35.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.793+0000 7fa58f651640 1 -- 172.21.15.5:0/1308105539 wait complete. 2024-05-20T18:31:35.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.795+0000 7fa58f651640 1 Processor -- start 2024-05-20T18:31:35.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.795+0000 7fa58f651640 1 -- start start 2024-05-20T18:31:35.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.796+0000 7fa58f651640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5880b2f10 0x7fa5880b2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:35.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.796+0000 7fa58f651640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa58815c530 0x7fa5880ac020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:35.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.796+0000 7fa58f651640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5880ac560 0x7fa5881f6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:35.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.796+0000 7fa58f651640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa5880d0300 con 0x7fa5880b2f10 2024-05-20T18:31:35.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.796+0000 7fa58f651640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa5880d4260 con 0x7fa5880ac560 2024-05-20T18:31:35.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.796+0000 7fa58f651640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa5880b4050 con 0x7fa58815c530 2024-05-20T18:31:35.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.797+0000 7fa58d3ed640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5880b2f10 0x7fa5880b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:35.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.797+0000 7fa58d3ed640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5880b2f10 0x7fa5880b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58004/0 (socket says 172.21.15.5:58004) 2024-05-20T18:31:35.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.797+0000 7fa58d3ed640 1 -- 172.21.15.5:0/2039130344 learned_addr learned my addr 172.21.15.5:0/2039130344 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:35.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.797+0000 7fa58dbee640 1 --2- 172.21.15.5:0/2039130344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5880ac560 0x7fa5881f6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:35.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.797+0000 7fa58cbec640 1 --2- 172.21.15.5:0/2039130344 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa58815c530 0x7fa5880ac020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:35.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.797+0000 7fa58d3ed640 1 -- 172.21.15.5:0/2039130344 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa58815c530 msgr2=0x7fa5880ac020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:35.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.797+0000 7fa58d3ed640 1 --2- 172.21.15.5:0/2039130344 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa58815c530 0x7fa5880ac020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:35.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.797+0000 7fa58d3ed640 1 -- 172.21.15.5:0/2039130344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5880ac560 msgr2=0x7fa5881f6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:35.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.797+0000 7fa58d3ed640 1 --2- 172.21.15.5:0/2039130344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5880ac560 0x7fa5881f6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:35.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.797+0000 7fa58d3ed640 1 -- 172.21.15.5:0/2039130344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5881f5700 con 0x7fa5880b2f10 2024-05-20T18:31:35.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.797+0000 7fa58dbee640 1 --2- 172.21.15.5:0/2039130344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5880ac560 0x7fa5881f6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:35.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.797+0000 7fa58cbec640 1 --2- 172.21.15.5:0/2039130344 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa58815c530 0x7fa5880ac020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:35.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.797+0000 7fa58d3ed640 1 --2- 172.21.15.5:0/2039130344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5880b2f10 0x7fa5880b2930 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fa578007140 tx=0x7fa578007600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:35.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.797+0000 7fa585ffb640 1 -- 172.21.15.5:0/2039130344 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa578013600 con 0x7fa5880b2f10 2024-05-20T18:31:35.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.797+0000 7fa585ffb640 1 -- 172.21.15.5:0/2039130344 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa578018070 con 0x7fa5880b2f10 2024-05-20T18:31:35.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.797+0000 7fa58f651640 1 -- 172.21.15.5:0/2039130344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5880ac970 con 0x7fa5880b2f10 2024-05-20T18:31:35.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.797+0000 7fa585ffb640 1 -- 172.21.15.5:0/2039130344 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa578013a00 con 0x7fa5880b2f10 2024-05-20T18:31:35.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.797+0000 7fa58f651640 1 -- 172.21.15.5:0/2039130344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5881f5c60 con 0x7fa5880b2f10 2024-05-20T18:31:35.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.798+0000 7fa58f651640 1 -- 172.21.15.5:0/2039130344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa588075b00 con 0x7fa5880b2f10 2024-05-20T18:31:35.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.799+0000 7fa585ffb640 1 -- 172.21.15.5:0/2039130344 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa578013ba0 con 0x7fa5880b2f10 2024-05-20T18:31:35.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.799+0000 7fa585ffb640 1 --2- 172.21.15.5:0/2039130344 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa57406be30 0x7fa57406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:35.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.800+0000 7fa585ffb640 1 -- 172.21.15.5:0/2039130344 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6101+0+0 (secure 0 0 0) 0x7fa57808bbe0 con 0x7fa5880b2f10 2024-05-20T18:31:35.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.800+0000 7fa58cbec640 1 --2- 172.21.15.5:0/2039130344 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa57406be30 0x7fa57406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:35.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.800+0000 7fa58cbec640 1 --2- 172.21.15.5:0/2039130344 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa57406be30 0x7fa57406e340 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009ab0 tx=0x7fa57c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:35.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:35.804+0000 7fa585ffb640 1 -- 172.21.15.5:0/2039130344 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa5880ac970 con 0x7fa5880b2f10 2024-05-20T18:31:36.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.029+0000 7fa58f651640 1 -- 172.21.15.5:0/2039130344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa58806c530 con 0x7fa5880b2f10 2024-05-20T18:31:36.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.029+0000 7fa585ffb640 1 -- 172.21.15.5:0/2039130344 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5171 (secure 0 0 0) 0x7fa5881f5700 con 0x7fa5880b2f10 2024-05-20T18:31:36.030 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:31:36.030 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1890759603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1890759603},{"type":"v1","addr":"172.21.15.63:6834","nonce":1890759603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5825,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/700447403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":700447403},{"type":"v1","addr":"172.21.15.63:6835","nonce":700447403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3382943275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3382943275},{"type":"v1","addr":"172.21.15.5:6837","nonce":3382943275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:31:30.164179+0000","modified":"2024-05-20T18:31:33.185897+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6762},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6762":{"gid":6762,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6836/791176355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":791176355},{"type":"v1","addr":"172.21.15.5:6836","nonce":791176355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6762,"qdb_cluster":[6762]},"id":5}]} 2024-05-20T18:31:36.030 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 44 2024-05-20T18:31:36.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.031+0000 7fa58f651640 1 -- 172.21.15.5:0/2039130344 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa57406be30 msgr2=0x7fa57406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:36.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.031+0000 7fa58f651640 1 --2- 172.21.15.5:0/2039130344 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa57406be30 0x7fa57406e340 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009ab0 tx=0x7fa57c008040 comp rx=0 tx=0).stop 2024-05-20T18:31:36.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.031+0000 7fa58f651640 1 -- 172.21.15.5:0/2039130344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5880b2f10 msgr2=0x7fa5880b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:36.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.031+0000 7fa58f651640 1 --2- 172.21.15.5:0/2039130344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5880b2f10 0x7fa5880b2930 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fa578007140 tx=0x7fa578007600 comp rx=0 tx=0).stop 2024-05-20T18:31:36.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.031+0000 7fa58f651640 1 -- 172.21.15.5:0/2039130344 shutdown_connections 2024-05-20T18:31:36.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.031+0000 7fa58f651640 1 --2- 172.21.15.5:0/2039130344 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa57406be30 0x7fa57406e340 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.031+0000 7fa58f651640 1 --2- 172.21.15.5:0/2039130344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5880ac560 0x7fa5881f6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.031+0000 7fa58f651640 1 --2- 172.21.15.5:0/2039130344 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa58815c530 0x7fa5880ac020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.031+0000 7fa58f651640 1 --2- 172.21.15.5:0/2039130344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5880b2f10 0x7fa5880b2930 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.031+0000 7fa58f651640 1 -- 172.21.15.5:0/2039130344 >> 172.21.15.5:0/2039130344 conn(0x7fa5880d4bd0 msgr2=0x7fa58815fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:36.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.031+0000 7fa58f651640 1 -- 172.21.15.5:0/2039130344 shutdown_connections 2024-05-20T18:31:36.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.031+0000 7fa58f651640 1 -- 172.21.15.5:0/2039130344 wait complete. 2024-05-20T18:31:36.445 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T18:31:30.164179+0000', 'modified': '2024-05-20T18:31:33.185897+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6762}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6762': {'gid': 6762, 'name': 'c', 'rank': 0, 'incarnation': 40, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.5:6836/791176355', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6835', 'nonce': 791176355}, {'type': 'v1', 'addr': '172.21.15.5:6836', 'nonce': 791176355}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6762, 'qdb_cluster': [6762]} 2024-05-20T18:31:36.446 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-20T18:31:36.446 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-20T18:31:36.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.565+0000 7f7100309640 1 Processor -- start 2024-05-20T18:31:36.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.565+0000 7f7100309640 1 -- start start 2024-05-20T18:31:36.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.566+0000 7f7100309640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f70f815d530 0x7f70f815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:36.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.566+0000 7f7100309640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70f8070850 0x7f70f8070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:36.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.566+0000 7f7100309640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70f80cd580 con 0x7f70f80b1700 2024-05-20T18:31:36.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.566+0000 7f7100309640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f70f80ce190 con 0x7f70f815d530 2024-05-20T18:31:36.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.566+0000 7f7100309640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f70f80cd860 con 0x7f70f8070850 2024-05-20T18:31:36.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.566+0000 7f70fe0a5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f70f815d530 0x7f70f815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:36.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.566+0000 7f70fd8a4640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f70f80b1700 0x7f70f80b1b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56880/0 (socket says 172.21.15.5:56880) 2024-05-20T18:31:36.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.567+0000 7f70fd8a4640 1 -- 172.21.15.5:0/1938687135 learned_addr learned my addr 172.21.15.5:0/1938687135 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:36.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.567+0000 7f70fe8a6640 1 --2- 172.21.15.5:0/1938687135 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70f8070850 0x7f70f8070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:36.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.567+0000 7f70fe0a5640 1 -- 172.21.15.5:0/1938687135 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70f8070850 msgr2=0x7f70f8070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:36.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.567+0000 7f70fe0a5640 1 --2- 172.21.15.5:0/1938687135 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70f8070850 0x7f70f8070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.567+0000 7f70fe0a5640 1 -- 172.21.15.5:0/1938687135 >> v1:172.21.15.5:6789/0 conn(0x7f70f80b1700 legacy=0x7f70f80b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:36.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.567+0000 7f70fe0a5640 1 -- 172.21.15.5:0/1938687135 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f80cdb40 con 0x7f70f815d530 2024-05-20T18:31:36.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.567+0000 7f70fd0a3640 1 -- 172.21.15.5:0/1938687135 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1545809179 0 0) 0x7f70f80cd580 con 0x7f70f80b1700 2024-05-20T18:31:36.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.567+0000 7f70fe8a6640 1 --2- 172.21.15.5:0/1938687135 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70f8070850 0x7f70f8070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:36.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.568+0000 7f70fe0a5640 1 --2- 172.21.15.5:0/1938687135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70f815d530 0x7f70f815aaa0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f70f40097a0 tx=0x7f70f402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca0ca224ded678 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.568+0000 7f70fd0a3640 1 -- 172.21.15.5:0/1938687135 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f4037070 con 0x7f70f815d530 2024-05-20T18:31:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.568+0000 7f70fd0a3640 1 -- 172.21.15.5:0/1938687135 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70f4032440 con 0x7f70f815d530 2024-05-20T18:31:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.568+0000 7f70fd0a3640 1 -- 172.21.15.5:0/1938687135 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f4032810 con 0x7f70f815d530 2024-05-20T18:31:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.568+0000 7f7100309640 1 -- 172.21.15.5:0/1938687135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70f815d530 msgr2=0x7f70f815aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.568+0000 7f7100309640 1 --2- 172.21.15.5:0/1938687135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70f815d530 0x7f70f815aaa0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f70f40097a0 tx=0x7f70f402a180 comp rx=0 tx=0).stop 2024-05-20T18:31:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.569+0000 7f7100309640 1 -- 172.21.15.5:0/1938687135 shutdown_connections 2024-05-20T18:31:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.569+0000 7f7100309640 1 --2- 172.21.15.5:0/1938687135 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70f8070850 0x7f70f8070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.569+0000 7f7100309640 1 --2- 172.21.15.5:0/1938687135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70f815d530 0x7f70f815aaa0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.569+0000 7f7100309640 1 -- 172.21.15.5:0/1938687135 >> 172.21.15.5:0/1938687135 conn(0x7f70f80d29d0 msgr2=0x7f70f80d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.569+0000 7f7100309640 1 -- 172.21.15.5:0/1938687135 shutdown_connections 2024-05-20T18:31:36.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.569+0000 7f7100309640 1 -- 172.21.15.5:0/1938687135 wait complete. 2024-05-20T18:31:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.570+0000 7f7100309640 1 Processor -- start 2024-05-20T18:31:36.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.571+0000 7f7100309640 1 -- start start 2024-05-20T18:31:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.572+0000 7f7100309640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70f8070850 0x7f70f80c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.572+0000 7f7100309640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70f80b1700 0x7f70f81f87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.572+0000 7f7100309640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70f815d530 0x7f70f81f8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.572+0000 7f7100309640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f70f80711a0 con 0x7f70f8070850 2024-05-20T18:31:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.572+0000 7f7100309640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f70f80cd230 con 0x7f70f815d530 2024-05-20T18:31:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.572+0000 7f7100309640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f70f8071320 con 0x7f70f80b1700 2024-05-20T18:31:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.572+0000 7f70fe0a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70f8070850 0x7f70f80c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.572+0000 7f70fe0a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70f8070850 0x7f70f80c9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58028/0 (socket says 172.21.15.5:58028) 2024-05-20T18:31:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.572+0000 7f70fe0a5640 1 -- 172.21.15.5:0/3695792852 learned_addr learned my addr 172.21.15.5:0/3695792852 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.572+0000 7f70fd8a4640 1 --2- 172.21.15.5:0/3695792852 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70f80b1700 0x7f70f81f87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:36.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.572+0000 7f70fe8a6640 1 --2- 172.21.15.5:0/3695792852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70f815d530 0x7f70f81f8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:36.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.572+0000 7f70fe0a5640 1 -- 172.21.15.5:0/3695792852 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70f80b1700 msgr2=0x7f70f81f87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:36.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.572+0000 7f70fe0a5640 1 --2- 172.21.15.5:0/3695792852 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70f80b1700 0x7f70f81f87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.572+0000 7f70fe0a5640 1 -- 172.21.15.5:0/3695792852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70f815d530 msgr2=0x7f70f81f8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:36.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.572+0000 7f70fe0a5640 1 --2- 172.21.15.5:0/3695792852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70f815d530 0x7f70f81f8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.572+0000 7f70fe0a5640 1 -- 172.21.15.5:0/3695792852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f806cff0 con 0x7f70f8070850 2024-05-20T18:31:36.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.572+0000 7f70fd8a4640 1 --2- 172.21.15.5:0/3695792852 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70f80b1700 0x7f70f81f87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:36.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.572+0000 7f70fe8a6640 1 --2- 172.21.15.5:0/3695792852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70f815d530 0x7f70f81f8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:36.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.573+0000 7f70fe0a5640 1 --2- 172.21.15.5:0/3695792852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70f8070850 0x7f70f80c9a80 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f70f40098c0 tx=0x7f70f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:36.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.573+0000 7f70eeffd640 1 -- 172.21.15.5:0/3695792852 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f4037040 con 0x7f70f8070850 2024-05-20T18:31:36.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.573+0000 7f70eeffd640 1 -- 172.21.15.5:0/3695792852 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70f4042070 con 0x7f70f8070850 2024-05-20T18:31:36.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.573+0000 7f7100309640 1 -- 172.21.15.5:0/3695792852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70f80c9fc0 con 0x7f70f8070850 2024-05-20T18:31:36.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.573+0000 7f70eeffd640 1 -- 172.21.15.5:0/3695792852 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f4030bb0 con 0x7f70f8070850 2024-05-20T18:31:36.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.573+0000 7f7100309640 1 -- 172.21.15.5:0/3695792852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70f806d570 con 0x7f70f8070850 2024-05-20T18:31:36.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.575+0000 7f70eeffd640 1 -- 172.21.15.5:0/3695792852 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f70f4047460 con 0x7f70f8070850 2024-05-20T18:31:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.575+0000 7f70eeffd640 1 --2- 172.21.15.5:0/3695792852 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f70dc06be30 0x7f70dc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.575+0000 7f70eeffd640 1 -- 172.21.15.5:0/3695792852 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6101+0+0 (secure 0 0 0) 0x7f70f40aba90 con 0x7f70f8070850 2024-05-20T18:31:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.575+0000 7f70fd8a4640 1 --2- 172.21.15.5:0/3695792852 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f70dc06be30 0x7f70dc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.576+0000 7f70fd8a4640 1 --2- 172.21.15.5:0/3695792852 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f70dc06be30 0x7f70dc06e340 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f70e8000f60 tx=0x7f70e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.576+0000 7f71071b5740 1 Processor -- start 2024-05-20T18:31:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.577+0000 7f71071b5740 1 -- start start 2024-05-20T18:31:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.578+0000 7f71071b5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd7c91d270 0x55bd7cce8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.578+0000 7f71071b5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd7c8f43b0 0x55bd7cce50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.578+0000 7f71071b5740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bd7c8fc590 0x55bd7ccbe360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.578+0000 7f71071b5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bd7cc214d0 con 0x55bd7c91d270 2024-05-20T18:31:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.578+0000 7f71071b5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bd7cbc93e0 con 0x55bd7c8f43b0 2024-05-20T18:31:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.578+0000 7f71071b5740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55bd7cc1d490 con 0x55bd7c8fc590 2024-05-20T18:31:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.578+0000 7f70fe0a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd7c91d270 0x55bd7cce8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.578+0000 7f70fe0a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd7c91d270 0x55bd7cce8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58030/0 (socket says 172.21.15.5:58030) 2024-05-20T18:31:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.578+0000 7f70fe0a5640 1 -- 172.21.15.5:0/1288956762 learned_addr learned my addr 172.21.15.5:0/1288956762 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.578+0000 7f70fd8a4640 1 --2- 172.21.15.5:0/1288956762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd7c8f43b0 0x55bd7cce50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.578+0000 7f70fe8a6640 1 --2- 172.21.15.5:0/1288956762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bd7c8fc590 0x55bd7ccbe360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.578+0000 7f70fe0a5640 1 -- 172.21.15.5:0/1288956762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bd7c8fc590 msgr2=0x55bd7ccbe360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.578+0000 7f70fe0a5640 1 --2- 172.21.15.5:0/1288956762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bd7c8fc590 0x55bd7ccbe360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.578+0000 7f70fe0a5640 1 -- 172.21.15.5:0/1288956762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd7c8f43b0 msgr2=0x55bd7cce50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.578+0000 7f70fe0a5640 1 --2- 172.21.15.5:0/1288956762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd7c8f43b0 0x55bd7cce50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.578+0000 7f70fe0a5640 1 -- 172.21.15.5:0/1288956762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd7c9352d0 con 0x55bd7c91d270 2024-05-20T18:31:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.578+0000 7f70fd8a4640 1 --2- 172.21.15.5:0/1288956762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd7c8f43b0 0x55bd7cce50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.579+0000 7f70fe0a5640 1 --2- 172.21.15.5:0/1288956762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd7c91d270 0x55bd7cce8700 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f70f4006f80 tx=0x7f70f4079540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a18286b47c4a27 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.579+0000 7f70dbfff640 1 -- 172.21.15.5:0/1288956762 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f40a8070 con 0x55bd7c91d270 2024-05-20T18:31:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.579+0000 7f70dbfff640 1 -- 172.21.15.5:0/1288956762 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70f407ae50 con 0x55bd7c91d270 2024-05-20T18:31:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.579+0000 7f70dbfff640 1 -- 172.21.15.5:0/1288956762 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f4082790 con 0x55bd7c91d270 2024-05-20T18:31:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.580+0000 7f71071b5740 1 -- 172.21.15.5:0/1288956762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd7c91d270 msgr2=0x55bd7cce8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:36.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.580+0000 7f71071b5740 1 --2- 172.21.15.5:0/1288956762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd7c91d270 0x55bd7cce8700 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f70f4006f80 tx=0x7f70f4079540 comp rx=0 tx=0).stop 2024-05-20T18:31:36.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.580+0000 7f71071b5740 1 -- 172.21.15.5:0/1288956762 shutdown_connections 2024-05-20T18:31:36.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.580+0000 7f71071b5740 1 --2- 172.21.15.5:0/1288956762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bd7c8fc590 0x55bd7ccbe360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.580+0000 7f71071b5740 1 --2- 172.21.15.5:0/1288956762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd7c8f43b0 0x55bd7cce50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.580+0000 7f71071b5740 1 --2- 172.21.15.5:0/1288956762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd7c91d270 0x55bd7cce8700 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.580+0000 7f71071b5740 1 -- 172.21.15.5:0/1288956762 >> 172.21.15.5:0/1288956762 conn(0x55bd7c9113c0 msgr2=0x55bd7ccbdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:36.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.580+0000 7f71071b5740 1 -- 172.21.15.5:0/1288956762 shutdown_connections 2024-05-20T18:31:36.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.580+0000 7f71071b5740 1 -- 172.21.15.5:0/1288956762 wait complete. 2024-05-20T18:31:36.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.581+0000 7f71071b5740 1 -- start start 2024-05-20T18:31:36.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.581+0000 7f71071b5740 1 Processor -- start 2024-05-20T18:31:36.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.582+0000 7f71071b5740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bd7c8f43b0 0x55bd7cd01e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:36.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.582+0000 7f71071b5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd7c8fc590 0x55bd7cd034f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:36.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.582+0000 7f71071b5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd7c91d270 0x55bd7ccea7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.582+0000 7f71071b5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bd7cc214d0 con 0x55bd7c91d270 2024-05-20T18:31:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.582+0000 7f71071b5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bd7cbdc8e0 con 0x55bd7c8fc590 2024-05-20T18:31:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.582+0000 7f71071b5740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55bd7cd03a30 con 0x55bd7c8f43b0 2024-05-20T18:31:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.582+0000 7f71071b5740 10 client.0 _ll_register_callbacks cb 0x55bd7cba99c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:31:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.582+0000 7f70fe8a6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd7c91d270 0x55bd7ccea7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.582+0000 7f70fe8a6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd7c91d270 0x55bd7ccea7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58040/0 (socket says 172.21.15.5:58040) 2024-05-20T18:31:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.582+0000 7f70fe8a6640 1 -- 172.21.15.5:0/385818624 learned_addr learned my addr 172.21.15.5:0/385818624 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.582+0000 7f70fd8a4640 1 --2- 172.21.15.5:0/385818624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd7c8fc590 0x55bd7cd034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.582+0000 7f70fe0a5640 1 --2- 172.21.15.5:0/385818624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bd7c8f43b0 0x55bd7cd01e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.583+0000 7f70fe8a6640 1 -- 172.21.15.5:0/385818624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bd7c8f43b0 msgr2=0x55bd7cd01e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.583+0000 7f70fe8a6640 1 --2- 172.21.15.5:0/385818624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bd7c8f43b0 0x55bd7cd01e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.583+0000 7f70fe8a6640 1 -- 172.21.15.5:0/385818624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd7c8fc590 msgr2=0x55bd7cd034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.583+0000 7f70fe8a6640 1 --2- 172.21.15.5:0/385818624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd7c8fc590 0x55bd7cd034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.583+0000 7f70fe8a6640 1 -- 172.21.15.5:0/385818624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd7c9ff6b0 con 0x55bd7c91d270 2024-05-20T18:31:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.583+0000 7f70fe8a6640 1 --2- 172.21.15.5:0/385818624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd7c91d270 0x55bd7ccea7e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f70f0007a30 tx=0x7f70f0007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4f8f357147f9927 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.583+0000 7f70d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:31:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.583+0000 7f70d97fa640 1 -- 172.21.15.5:0/385818624 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f000fc00 con 0x55bd7c91d270 2024-05-20T18:31:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.583+0000 7f70d97fa640 1 -- 172.21.15.5:0/385818624 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 6101+0+0 (secure 0 0 0) 0x7f70f000fda0 con 0x55bd7c91d270 2024-05-20T18:31:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.583+0000 7f71071b5740 1 -- 172.21.15.5:0/385818624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70f407ae50 con 0x55bd7c91d270 2024-05-20T18:31:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.583+0000 7f70d97fa640 1 -- 172.21.15.5:0/385818624 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f70f0023050 con 0x55bd7c91d270 2024-05-20T18:31:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.583+0000 7f71071b5740 10 client.6900 fetch_fsmap learned FSMap version 44 2024-05-20T18:31:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.583+0000 7f71071b5740 1 -- 172.21.15.5:0/385818624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x55bd7cd128d0 con 0x55bd7c91d270 2024-05-20T18:31:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.584+0000 7f70d97fa640 1 -- 172.21.15.5:0/385818624 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70f407ae50 con 0x55bd7c91d270 2024-05-20T18:31:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.584+0000 7f70d97fa640 1 -- 172.21.15.5:0/385818624 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 3494+0+0 (secure 0 0 0) 0x7f70f0026020 con 0x55bd7c91d270 2024-05-20T18:31:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.584+0000 7f71071b5740 10 client.6900 fetch_fsmap finished waiting for FSMap version 44 2024-05-20T18:31:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.584+0000 7f71071b5740 10 client.6900 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-20T18:31:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.584+0000 7f71071b5740 20 client.6900 populate_metadata read hostname 'smithi005' 2024-05-20T18:31:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.584+0000 7f71071b5740 1 --2- 172.21.15.5:0/385818624 >> [v2:172.21.15.5:6835/791176355,v1:172.21.15.5:6836/791176355] conn(0x55bd7c8dbe40 0x55bd7cd1fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.584+0000 7f71071b5740 4 client.6900 mds_command: new command op to 6762 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:31:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.584+0000 7f71071b5740 1 -- 172.21.15.5:0/385818624 --> [v2:172.21.15.5:6835/791176355,v1:172.21.15.5:6836/791176355] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd7cc2ec00 con 0x55bd7c8dbe40 2024-05-20T18:31:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.584+0000 7f70fe0a5640 1 --2- 172.21.15.5:0/385818624 >> [v2:172.21.15.5:6835/791176355,v1:172.21.15.5:6836/791176355] conn(0x55bd7c8dbe40 0x55bd7cd1fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.584+0000 7f70d97fa640 10 client.6900 ms_handle_connect on v2:172.21.15.5:6835/791176355 2024-05-20T18:31:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.584+0000 7f70fe0a5640 1 --2- 172.21.15.5:0/385818624 >> [v2:172.21.15.5:6835/791176355,v1:172.21.15.5:6836/791176355] conn(0x55bd7c8dbe40 0x55bd7cd1fe10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b3601aa207728c3 server_cookie=cee12ab485f41a5f in_seq=0 out_seq=0 2024-05-20T18:31:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.585+0000 7f70d97fa640 1 -- 172.21.15.5:0/385818624 <== mds.0 v2:172.21.15.5:6835/791176355 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bd7cc2ec00 con 0x55bd7c8dbe40 2024-05-20T18:31:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.585+0000 7f70d97fa640 10 client.6900 handle_command_reply: tid=0 2024-05-20T18:31:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.585+0000 7f71071b5740 1 client.6900 shutdown 2024-05-20T18:31:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.585+0000 7f71071b5740 1 -- 172.21.15.5:0/385818624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd7c91d270 msgr2=0x55bd7ccea7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.585+0000 7f71071b5740 1 --2- 172.21.15.5:0/385818624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd7c91d270 0x55bd7ccea7e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f70f0007a30 tx=0x7f70f0007ef0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.585+0000 7f71071b5740 1 -- 172.21.15.5:0/385818624 shutdown_connections 2024-05-20T18:31:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.585+0000 7f71071b5740 1 --2- 172.21.15.5:0/385818624 >> [v2:172.21.15.5:6835/791176355,v1:172.21.15.5:6836/791176355] conn(0x55bd7c8dbe40 0x55bd7cd1fe10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.585+0000 7f71071b5740 1 --2- 172.21.15.5:0/385818624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd7c91d270 0x55bd7ccea7e0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.585+0000 7f71071b5740 1 --2- 172.21.15.5:0/385818624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd7c8fc590 0x55bd7cd034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.585+0000 7f71071b5740 1 --2- 172.21.15.5:0/385818624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bd7c8f43b0 0x55bd7cd01e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.585+0000 7f70d97fa640 0 client.6900 ms_handle_reset on v2:172.21.15.5:6835/791176355 2024-05-20T18:31:36.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.585+0000 7f71071b5740 1 -- 172.21.15.5:0/385818624 >> 172.21.15.5:0/385818624 conn(0x55bd7c9113c0 msgr2=0x55bd7ccbdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:36.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.585+0000 7f71071b5740 1 -- 172.21.15.5:0/385818624 shutdown_connections 2024-05-20T18:31:36.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.585+0000 7f71071b5740 1 -- 172.21.15.5:0/385818624 wait complete. 2024-05-20T18:31:36.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.586+0000 7f71071b5740 20 client.6900 trim_cache size 0 max 16384 2024-05-20T18:31:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.605+0000 7f71071b5740 1 Processor -- start 2024-05-20T18:31:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.605+0000 7f71071b5740 1 -- start start 2024-05-20T18:31:36.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.606+0000 7f71071b5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd7c8f43b0 0x55bd7cd11870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:36.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.606+0000 7f71071b5740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bd7c8fc590 0x55bd7cce8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:36.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.606+0000 7f71071b5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd7c91d270 0x55bd7cce8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:36.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.606+0000 7f71071b5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f70f8201340 con 0x55bd7c8f43b0 2024-05-20T18:31:36.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.606+0000 7f71071b5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f70f82006d0 con 0x55bd7c91d270 2024-05-20T18:31:36.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.606+0000 7f71071b5740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f70f8200d20 con 0x55bd7c8fc590 2024-05-20T18:31:36.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.606+0000 7f70fe0a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd7c8f43b0 0x55bd7cd11870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:36.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.606+0000 7f70fe0a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd7c8f43b0 0x55bd7cd11870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58054/0 (socket says 172.21.15.5:58054) 2024-05-20T18:31:36.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.606+0000 7f70fe0a5640 1 -- 172.21.15.5:0/545548957 learned_addr learned my addr 172.21.15.5:0/545548957 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:36.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.607+0000 7f70fe0a5640 1 -- 172.21.15.5:0/545548957 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bd7c8fc590 msgr2=0x55bd7cce8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:36.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.607+0000 7f70fe0a5640 1 --2- 172.21.15.5:0/545548957 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bd7c8fc590 0x55bd7cce8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.607+0000 7f70fe0a5640 1 -- 172.21.15.5:0/545548957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd7c91d270 msgr2=0x55bd7cce8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:36.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.607+0000 7f70fe0a5640 1 --2- 172.21.15.5:0/545548957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd7c91d270 0x55bd7cce8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.607+0000 7f70fe0a5640 1 -- 172.21.15.5:0/545548957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd7ccbfd10 con 0x55bd7c8f43b0 2024-05-20T18:31:36.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.607+0000 7f70fe0a5640 1 --2- 172.21.15.5:0/545548957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd7c8f43b0 0x55bd7cd11870 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f70f40a9e80 tx=0x7f70f407b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87150c54d1672407 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:36.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.607+0000 7f70db7fe640 1 -- 172.21.15.5:0/545548957 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f40a8040 con 0x55bd7c8f43b0 2024-05-20T18:31:36.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.607+0000 7f70db7fe640 1 -- 172.21.15.5:0/545548957 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70f407ae50 con 0x55bd7c8f43b0 2024-05-20T18:31:36.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.607+0000 7f70db7fe640 1 -- 172.21.15.5:0/545548957 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f4077c40 con 0x55bd7c8f43b0 2024-05-20T18:31:36.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.608+0000 7f71071b5740 1 -- 172.21.15.5:0/545548957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd7c8f43b0 msgr2=0x55bd7cd11870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:36.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.608+0000 7f71071b5740 1 --2- 172.21.15.5:0/545548957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd7c8f43b0 0x55bd7cd11870 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f70f40a9e80 tx=0x7f70f407b000 comp rx=0 tx=0).stop 2024-05-20T18:31:36.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.608+0000 7f71071b5740 1 -- 172.21.15.5:0/545548957 shutdown_connections 2024-05-20T18:31:36.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.608+0000 7f71071b5740 1 --2- 172.21.15.5:0/545548957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd7c91d270 0x55bd7cce8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.608+0000 7f71071b5740 1 --2- 172.21.15.5:0/545548957 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bd7c8fc590 0x55bd7cce8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.608+0000 7f71071b5740 1 --2- 172.21.15.5:0/545548957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd7c8f43b0 0x55bd7cd11870 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.608+0000 7f71071b5740 1 -- 172.21.15.5:0/545548957 >> 172.21.15.5:0/545548957 conn(0x55bd7c9113c0 msgr2=0x55bd7ccbc420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:36.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.608+0000 7f71071b5740 1 -- 172.21.15.5:0/545548957 shutdown_connections 2024-05-20T18:31:36.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.608+0000 7f71071b5740 1 -- 172.21.15.5:0/545548957 wait complete. 2024-05-20T18:31:36.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.609+0000 7f71071b5740 1 -- start start 2024-05-20T18:31:36.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.609+0000 7f71071b5740 1 Processor -- start 2024-05-20T18:31:36.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.610+0000 7f71071b5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd7c8f43b0 0x55bd7cd09b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:36.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.610+0000 7f71071b5740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bd7c8fc590 0x55bd7cd0a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:36.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.610+0000 7f71071b5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd7c91d270 0x55bd7cd18540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:36.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.610+0000 7f71071b5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bd7cce9740 con 0x55bd7c91d270 2024-05-20T18:31:36.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.610+0000 7f71071b5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bd7cd12c10 con 0x55bd7c8f43b0 2024-05-20T18:31:36.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.610+0000 7f71071b5740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55bd7cce9d90 con 0x55bd7c8fc590 2024-05-20T18:31:36.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.610+0000 7f71071b5740 10 client.0 _ll_register_callbacks cb 0x7f70f0009670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:31:36.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.610+0000 7f70fe8a6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd7c91d270 0x55bd7cd18540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:36.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.610+0000 7f70fe8a6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd7c91d270 0x55bd7cd18540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58066/0 (socket says 172.21.15.5:58066) 2024-05-20T18:31:36.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.610+0000 7f70fe8a6640 1 -- 172.21.15.5:0/4018148299 learned_addr learned my addr 172.21.15.5:0/4018148299 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:36.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.610+0000 7f70fd8a4640 1 --2- 172.21.15.5:0/4018148299 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bd7c8fc590 0x55bd7cd0a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:36.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.610+0000 7f70fe0a5640 1 --2- 172.21.15.5:0/4018148299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd7c8f43b0 0x55bd7cd09b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:36.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.611+0000 7f70fe8a6640 1 -- 172.21.15.5:0/4018148299 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bd7c8fc590 msgr2=0x55bd7cd0a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:36.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.611+0000 7f70fe8a6640 1 --2- 172.21.15.5:0/4018148299 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bd7c8fc590 0x55bd7cd0a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.611+0000 7f70fe8a6640 1 -- 172.21.15.5:0/4018148299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd7c8f43b0 msgr2=0x55bd7cd09b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:36.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.611+0000 7f70fe8a6640 1 --2- 172.21.15.5:0/4018148299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd7c8f43b0 0x55bd7cd09b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.611+0000 7f70fe8a6640 1 -- 172.21.15.5:0/4018148299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd7ccbd3f0 con 0x55bd7c91d270 2024-05-20T18:31:36.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.611+0000 7f70fe0a5640 1 --2- 172.21.15.5:0/4018148299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd7c8f43b0 0x55bd7cd09b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:36.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.611+0000 7f70fe8a6640 1 --2- 172.21.15.5:0/4018148299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd7c91d270 0x55bd7cd18540 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f70f0007f30 tx=0x7f70f0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6089d84e71b0d4c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:36.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.611+0000 7f70d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:31:36.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.611+0000 7f70d97fa640 1 -- 172.21.15.5:0/4018148299 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0016b20 con 0x55bd7c91d270 2024-05-20T18:31:36.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.611+0000 7f70d97fa640 1 -- 172.21.15.5:0/4018148299 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 6101+0+0 (secure 0 0 0) 0x7f70f0016cc0 con 0x55bd7c91d270 2024-05-20T18:31:36.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.611+0000 7f71071b5740 1 -- 172.21.15.5:0/4018148299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70f407ae50 con 0x55bd7c91d270 2024-05-20T18:31:36.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.611+0000 7f70d97fa640 1 -- 172.21.15.5:0/4018148299 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f70f000f580 con 0x55bd7c91d270 2024-05-20T18:31:36.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.611+0000 7f71071b5740 10 client.6906 fetch_fsmap learned FSMap version 44 2024-05-20T18:31:36.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.611+0000 7f71071b5740 1 -- 172.21.15.5:0/4018148299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x55bd7cd006c0 con 0x55bd7c91d270 2024-05-20T18:31:36.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.611+0000 7f70d97fa640 1 -- 172.21.15.5:0/4018148299 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70f407ae50 con 0x55bd7c91d270 2024-05-20T18:31:36.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.612+0000 7f70d97fa640 1 -- 172.21.15.5:0/4018148299 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 3494+0+0 (secure 0 0 0) 0x7f70f0039050 con 0x55bd7c91d270 2024-05-20T18:31:36.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.612+0000 7f71071b5740 10 client.6906 fetch_fsmap finished waiting for FSMap version 44 2024-05-20T18:31:36.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.612+0000 7f71071b5740 10 client.6906 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-20T18:31:36.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.612+0000 7f71071b5740 20 client.6906 populate_metadata read hostname 'smithi005' 2024-05-20T18:31:36.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.612+0000 7f71071b5740 1 --2- 172.21.15.5:0/4018148299 >> [v2:172.21.15.5:6835/791176355,v1:172.21.15.5:6836/791176355] conn(0x55bd7cd046f0 0x55bd7ccfebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:36.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.612+0000 7f71071b5740 4 client.6906 mds_command: new command op to 6762 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T18:31:36.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.612+0000 7f71071b5740 1 -- 172.21.15.5:0/4018148299 --> [v2:172.21.15.5:6835/791176355,v1:172.21.15.5:6836/791176355] -- command(tid 0: {"prefix": "status"}) -- 0x55bd7ccb98b0 con 0x55bd7cd046f0 2024-05-20T18:31:36.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.612+0000 7f70fe0a5640 1 --2- 172.21.15.5:0/4018148299 >> [v2:172.21.15.5:6835/791176355,v1:172.21.15.5:6836/791176355] conn(0x55bd7cd046f0 0x55bd7ccfebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:36.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.612+0000 7f70d97fa640 10 client.6906 ms_handle_connect on v2:172.21.15.5:6835/791176355 2024-05-20T18:31:36.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.612+0000 7f70fe0a5640 1 --2- 172.21.15.5:0/4018148299 >> [v2:172.21.15.5:6835/791176355,v1:172.21.15.5:6836/791176355] conn(0x55bd7cd046f0 0x55bd7ccfebc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e1c3658a3f8a7dc server_cookie=dc7e168b41ff597c in_seq=0 out_seq=0 2024-05-20T18:31:36.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.613+0000 7f70d97fa640 1 -- 172.21.15.5:0/4018148299 <== mds.0 v2:172.21.15.5:6835/791176355 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55bd7ccb98b0 con 0x55bd7cd046f0 2024-05-20T18:31:36.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.613+0000 7f70d97fa640 10 client.6906 handle_command_reply: tid=0 2024-05-20T18:31:36.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.613+0000 7f71071b5740 1 client.6906 shutdown 2024-05-20T18:31:36.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.613+0000 7f71071b5740 1 -- 172.21.15.5:0/4018148299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd7c91d270 msgr2=0x55bd7cd18540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:36.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.613+0000 7f71071b5740 1 --2- 172.21.15.5:0/4018148299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd7c91d270 0x55bd7cd18540 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f70f0007f30 tx=0x7f70f0004ab0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.613+0000 7f71071b5740 1 -- 172.21.15.5:0/4018148299 shutdown_connections 2024-05-20T18:31:36.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.613+0000 7f71071b5740 1 --2- 172.21.15.5:0/4018148299 >> [v2:172.21.15.5:6835/791176355,v1:172.21.15.5:6836/791176355] conn(0x55bd7cd046f0 0x55bd7ccfebc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.613+0000 7f71071b5740 1 --2- 172.21.15.5:0/4018148299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd7c91d270 0x55bd7cd18540 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.613+0000 7f71071b5740 1 --2- 172.21.15.5:0/4018148299 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bd7c8fc590 0x55bd7cd0a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.613+0000 7f71071b5740 1 --2- 172.21.15.5:0/4018148299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bd7c8f43b0 0x55bd7cd09b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.613+0000 7f70d97fa640 0 client.6906 ms_handle_reset on v2:172.21.15.5:6835/791176355 2024-05-20T18:31:36.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.613+0000 7f71071b5740 1 -- 172.21.15.5:0/4018148299 >> 172.21.15.5:0/4018148299 conn(0x55bd7c9113c0 msgr2=0x55bd7ccbc420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:36.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.613+0000 7f71071b5740 1 -- 172.21.15.5:0/4018148299 shutdown_connections 2024-05-20T18:31:36.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.613+0000 7f71071b5740 1 -- 172.21.15.5:0/4018148299 wait complete. 2024-05-20T18:31:36.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.613+0000 7f71071b5740 20 client.6906 trim_cache size 0 max 16384 2024-05-20T18:31:36.619 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:31:36.619 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "399ab250-fd3c-47fa-a1c8-25e8149c9cac", 2024-05-20T18:31:36.619 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-20T18:31:36.619 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 6762, 2024-05-20T18:31:36.619 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-20T18:31:36.619 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-20T18:31:36.620 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-20T18:31:36.620 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 5.8248662710000003, 2024-05-20T18:31:36.620 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 44, 2024-05-20T18:31:36.620 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 77, 2024-05-20T18:31:36.620 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 77, 2024-05-20T18:31:36.620 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 11.782307088 2024-05-20T18:31:36.620 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:31:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.613+0000 7f7100309640 1 -- 172.21.15.5:0/3695792852 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f70dc06be30 msgr2=0x7f70dc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.613+0000 7f7100309640 1 --2- 172.21.15.5:0/3695792852 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f70dc06be30 0x7f70dc06e340 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f70e8000f60 tx=0x7f70e8009040 comp rx=0 tx=0).stop 2024-05-20T18:31:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.613+0000 7f7100309640 1 -- 172.21.15.5:0/3695792852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70f8070850 msgr2=0x7f70f80c9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.614+0000 7f7100309640 1 --2- 172.21.15.5:0/3695792852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70f8070850 0x7f70f80c9a80 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f70f40098c0 tx=0x7f70f40046c0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.614+0000 7f7100309640 1 -- 172.21.15.5:0/3695792852 shutdown_connections 2024-05-20T18:31:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.614+0000 7f7100309640 1 --2- 172.21.15.5:0/3695792852 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f70dc06be30 0x7f70dc06e340 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.614+0000 7f7100309640 1 --2- 172.21.15.5:0/3695792852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70f815d530 0x7f70f81f8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.614+0000 7f7100309640 1 --2- 172.21.15.5:0/3695792852 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70f80b1700 0x7f70f81f87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.614+0000 7f7100309640 1 --2- 172.21.15.5:0/3695792852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70f8070850 0x7f70f80c9a80 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.614+0000 7f7100309640 1 -- 172.21.15.5:0/3695792852 >> 172.21.15.5:0/3695792852 conn(0x7f70f80d29d0 msgr2=0x7f70f815ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.614+0000 7f7100309640 1 -- 172.21.15.5:0/3695792852 shutdown_connections 2024-05-20T18:31:36.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.614+0000 7f7100309640 1 -- 172.21.15.5:0/3695792852 wait complete. 2024-05-20T18:31:36.625 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-20T18:31:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.740+0000 7f7209f77640 1 Processor -- start 2024-05-20T18:31:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.740+0000 7f7209f77640 1 -- start start 2024-05-20T18:31:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.741+0000 7f7209f77640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7204162820 0x7f72040b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.741+0000 7f7209f77640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f720406dae0 0x7f72040b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.741+0000 7f7209f77640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72040d0630 con 0x7f72040b3e80 2024-05-20T18:31:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.741+0000 7f7209f77640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f72040d39c0 con 0x7f720406dae0 2024-05-20T18:31:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.741+0000 7f7209f77640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f72040d3ca0 con 0x7f7204162820 2024-05-20T18:31:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.742+0000 7f7203fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f720406dae0 0x7f72040b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.742+0000 7f7203fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f720406dae0 0x7f72040b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58080/0 (socket says 172.21.15.5:58080) 2024-05-20T18:31:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.742+0000 7f7203fff640 1 -- 172.21.15.5:0/3065016958 learned_addr learned my addr 172.21.15.5:0/3065016958 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.742+0000 7f72037fe640 1 --2- 172.21.15.5:0/3065016958 >> v2:172.21.15.63:3300/0 conn(0x7f7204162820 0x7f72040b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.742+0000 7f7203fff640 1 -- 172.21.15.5:0/3065016958 >> v2:172.21.15.63:3300/0 conn(0x7f7204162820 msgr2=0x7f72040b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.742+0000 7f7203fff640 1 --2- 172.21.15.5:0/3065016958 >> v2:172.21.15.63:3300/0 conn(0x7f7204162820 0x7f72040b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.742+0000 7f7203fff640 1 -- 172.21.15.5:0/3065016958 >> v1:172.21.15.63:6789/0 conn(0x7f72040b3e80 legacy=0x7f720406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.742+0000 7f7203fff640 1 -- 172.21.15.5:0/3065016958 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72040d3f80 con 0x7f720406dae0 2024-05-20T18:31:36.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.743+0000 7f7203fff640 1 --2- 172.21.15.5:0/3065016958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f720406dae0 0x7f72040b8190 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f72040b7740 tx=0x7f71fc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1a61c775ad1d76 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:36.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.743+0000 7f7202ffd640 1 -- 172.21.15.5:0/3065016958 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc036070 con 0x7f720406dae0 2024-05-20T18:31:36.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.743+0000 7f7202ffd640 1 -- 172.21.15.5:0/3065016958 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71fc031600 con 0x7f720406dae0 2024-05-20T18:31:36.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.743+0000 7f7202ffd640 1 -- 172.21.15.5:0/3065016958 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc0319d0 con 0x7f720406dae0 2024-05-20T18:31:36.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.744+0000 7f7209f77640 1 -- 172.21.15.5:0/3065016958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f720406dae0 msgr2=0x7f72040b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:36.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.744+0000 7f7209f77640 1 --2- 172.21.15.5:0/3065016958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f720406dae0 0x7f72040b8190 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f72040b7740 tx=0x7f71fc02a6f0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.744+0000 7f7209f77640 1 -- 172.21.15.5:0/3065016958 shutdown_connections 2024-05-20T18:31:36.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.744+0000 7f7209f77640 1 --2- 172.21.15.5:0/3065016958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f720406dae0 0x7f72040b8190 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.744+0000 7f7209f77640 1 --2- 172.21.15.5:0/3065016958 >> v2:172.21.15.63:3300/0 conn(0x7f7204162820 0x7f72040b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.744+0000 7f7209f77640 1 -- 172.21.15.5:0/3065016958 >> 172.21.15.5:0/3065016958 conn(0x7f72040d4bd0 msgr2=0x7f72040d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:36.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.744+0000 7f7209f77640 1 -- 172.21.15.5:0/3065016958 shutdown_connections 2024-05-20T18:31:36.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.744+0000 7f7209f77640 1 -- 172.21.15.5:0/3065016958 wait complete. 2024-05-20T18:31:36.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.745+0000 7f7209f77640 1 Processor -- start 2024-05-20T18:31:36.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.746+0000 7f7209f77640 1 -- start start 2024-05-20T18:31:36.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.747+0000 7f7209f77640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72040b3e80 0x7f72041f5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:36.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.747+0000 7f7209f77640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7204162820 0x7f72040c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:36.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.747+0000 7f7209f77640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72040c98c0 0x7f7204151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:36.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.747+0000 7f7209f77640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f72040d4780 con 0x7f72040b3e80 2024-05-20T18:31:36.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.747+0000 7f7209f77640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f72040d0300 con 0x7f7204162820 2024-05-20T18:31:36.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.747+0000 7f7209f77640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f72040d4900 con 0x7f72040c98c0 2024-05-20T18:31:36.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.747+0000 7f72037fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72040b3e80 0x7f72041f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:36.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.747+0000 7f72037fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72040b3e80 0x7f72041f5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58096/0 (socket says 172.21.15.5:58096) 2024-05-20T18:31:36.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.747+0000 7f72037fe640 1 -- 172.21.15.5:0/2277211719 learned_addr learned my addr 172.21.15.5:0/2277211719 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:36.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.747+0000 7f71fbfff640 1 --2- 172.21.15.5:0/2277211719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7204162820 0x7f72040c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:36.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.747+0000 7f7203fff640 1 --2- 172.21.15.5:0/2277211719 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72040c98c0 0x7f7204151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:36.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.747+0000 7f72037fe640 1 -- 172.21.15.5:0/2277211719 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72040c98c0 msgr2=0x7f7204151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:36.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.747+0000 7f72037fe640 1 --2- 172.21.15.5:0/2277211719 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72040c98c0 0x7f7204151e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.747+0000 7f72037fe640 1 -- 172.21.15.5:0/2277211719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7204162820 msgr2=0x7f72040c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:36.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.747+0000 7f72037fe640 1 --2- 172.21.15.5:0/2277211719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7204162820 0x7f72040c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.747+0000 7f72037fe640 1 -- 172.21.15.5:0/2277211719 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72041f5d20 con 0x7f72040b3e80 2024-05-20T18:31:36.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.747+0000 7f71fbfff640 1 --2- 172.21.15.5:0/2277211719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7204162820 0x7f72040c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:36.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.748+0000 7f72037fe640 1 --2- 172.21.15.5:0/2277211719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72040b3e80 0x7f72041f5230 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f71f400d5f0 tx=0x7f71f400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:36.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.748+0000 7f7200ff9640 1 -- 172.21.15.5:0/2277211719 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f40125f0 con 0x7f72040b3e80 2024-05-20T18:31:36.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.748+0000 7f7200ff9640 1 -- 172.21.15.5:0/2277211719 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71f4019070 con 0x7f72040b3e80 2024-05-20T18:31:36.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.748+0000 7f7209f77640 1 -- 172.21.15.5:0/2277211719 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72041f6000 con 0x7f72040b3e80 2024-05-20T18:31:36.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.748+0000 7f7200ff9640 1 -- 172.21.15.5:0/2277211719 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f40129d0 con 0x7f72040b3e80 2024-05-20T18:31:36.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.748+0000 7f7209f77640 1 -- 172.21.15.5:0/2277211719 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72040c8b40 con 0x7f72040b3e80 2024-05-20T18:31:36.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.749+0000 7f7209f77640 1 -- 172.21.15.5:0/2277211719 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72040ba7e0 con 0x7f72040b3e80 2024-05-20T18:31:36.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.749+0000 7f7200ff9640 1 -- 172.21.15.5:0/2277211719 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f71f4012b70 con 0x7f72040b3e80 2024-05-20T18:31:36.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.750+0000 7f7200ff9640 1 --2- 172.21.15.5:0/2277211719 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f71e406be30 0x7f71e406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:36.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.750+0000 7f7200ff9640 1 -- 172.21.15.5:0/2277211719 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6101+0+0 (secure 0 0 0) 0x7f71f408b4f0 con 0x7f72040b3e80 2024-05-20T18:31:36.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.750+0000 7f71fbfff640 1 --2- 172.21.15.5:0/2277211719 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f71e406be30 0x7f71e406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:36.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.751+0000 7f71fbfff640 1 --2- 172.21.15.5:0/2277211719 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f71e406be30 0x7f71e406e340 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f720415e7a0 tx=0x7f71ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:36.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.754+0000 7f7200ff9640 1 -- 172.21.15.5:0/2277211719 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f72040ba7e0 con 0x7f72040b3e80 2024-05-20T18:31:36.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.977+0000 7f7209f77640 1 -- 172.21.15.5:0/2277211719 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7204162c30 con 0x7f72040b3e80 2024-05-20T18:31:36.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.977+0000 7f7200ff9640 1 -- 172.21.15.5:0/2277211719 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) ==== 53+0+73 (secure 0 0 0) 0x7f7204162c30 con 0x7f72040b3e80 2024-05-20T18:31:36.978 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-20T18:31:36.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.979+0000 7f7209f77640 1 -- 172.21.15.5:0/2277211719 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f71e406be30 msgr2=0x7f71e406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:36.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.979+0000 7f7209f77640 1 --2- 172.21.15.5:0/2277211719 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f71e406be30 0x7f71e406e340 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f720415e7a0 tx=0x7f71ec008040 comp rx=0 tx=0).stop 2024-05-20T18:31:36.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.979+0000 7f7209f77640 1 -- 172.21.15.5:0/2277211719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72040b3e80 msgr2=0x7f72041f5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:36.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.979+0000 7f7209f77640 1 --2- 172.21.15.5:0/2277211719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72040b3e80 0x7f72041f5230 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f71f400d5f0 tx=0x7f71f400dab0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.979+0000 7f7209f77640 1 -- 172.21.15.5:0/2277211719 shutdown_connections 2024-05-20T18:31:36.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.979+0000 7f7209f77640 1 --2- 172.21.15.5:0/2277211719 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f71e406be30 0x7f71e406e340 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.979+0000 7f7209f77640 1 --2- 172.21.15.5:0/2277211719 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72040c98c0 0x7f7204151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.979+0000 7f7209f77640 1 --2- 172.21.15.5:0/2277211719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7204162820 0x7f72040c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.979+0000 7f7209f77640 1 --2- 172.21.15.5:0/2277211719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72040b3e80 0x7f72041f5230 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:36.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.979+0000 7f7209f77640 1 -- 172.21.15.5:0/2277211719 >> 172.21.15.5:0/2277211719 conn(0x7f72040d4bd0 msgr2=0x7f720415ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:36.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.979+0000 7f7209f77640 1 -- 172.21.15.5:0/2277211719 shutdown_connections 2024-05-20T18:31:36.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:36.979+0000 7f7209f77640 1 -- 172.21.15.5:0/2277211719 wait complete. 2024-05-20T18:31:36.991 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-20T18:31:36.991 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-20T18:31:36.991 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-05-20T18:31:36.991 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-20T18:31:36.991 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-20T18:31:36.991 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-20T18:31:36.991 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-20T18:31:36.991 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-20T18:31:36.991 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:31:36.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:31:36.992 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-05-20T18:31:37.008 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T18:31:37.009 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T18:31:37.009 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T18:31:37.009 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:31:37.009 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-20T18:31:37.009 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:31:37.009 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:31:37.009 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-05-20T18:31:37.009 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-05-20T18:31:37.009 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:31:37.009 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-05-20T18:31:37.009 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-05-20T18:31:37.009 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-20T18:31:37.009 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:31:37.010 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-05-20T18:31:37.010 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-20T18:31:37.010 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42401sec preferred_lft 42401sec 2024-05-20T18:31:37.010 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-05-20T18:31:37.010 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:31:37.010 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:31:37.010 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:31:37.010 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-05-20T18:31:37.010 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-20T18:31:37.010 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:31:37.010 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-20T18:31:37.011 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:31:37.011 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::4474:eaff:fe07:4f9a/64 scope link 2024-05-20T18:31:37.011 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:31:37.011 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:31:37.011 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T18:31:37.011 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ce4:f4ff:fe5e:7906/64 scope link 2024-05-20T18:31:37.011 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:31:37.011 INFO:teuthology.orchestra.run.smithi005.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:31:37.011 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether da:26:04:0e:95:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T18:31:37.011 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::d826:4ff:fe0e:950a/64 scope link 2024-05-20T18:31:37.011 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:31:37.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:31:37.012 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-20T18:31:37.074 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-20T18:31:37.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:31:37.075 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-05-20T18:31:37.132 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-05-20T18:31:37.132 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-20T18:31:37.132 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-20T18:31:37.132 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-20T18:31:37.132 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-20T18:31:37.132 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:31:37.132 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-20T18:31:37.133 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-20T18:31:37.133 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-20T18:31:37.133 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-20T18:31:37.133 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-20T18:31:37.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:31:37.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:31:37.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:31:37.262 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-20T18:31:37.278 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-20T18:31:37.278 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-20T18:31:37.278 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:31:37.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:31:37.279 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-05-20T18:31:37.344 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:31:37.345 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-20T18:31:37.401 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:31:37.401 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:31:37.402 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:31:37.457 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-20T18:31:37.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:31:37.457 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:31:37.553 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:31:37.553 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:31:37.553 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:31:37.564 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:31:37.564 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:31:37.564 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:31:37.564 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:31:37.564 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:31:37.570 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:31:37.570 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:31:37.570 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:31:37.570 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:31:37.570 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:31:37.570 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790" 2024-05-20T18:31:37.570 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790:/ 2024-05-20T18:31:37.570 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=399ab250-fd3c-47fa-a1c8-25e8149c9cac" will pass to kernel 2024-05-20T18:31:37.572 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-20T18:31:37.572 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-05-20T18:31:37.572 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-20T18:31:37.572 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-05-20T18:31:37.572 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-20T18:31:37.573 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-20T18:31:37.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:31:37.573 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:31:37.604 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-20T18:31:37.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T18:31:37.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:31:37.709 INFO:teuthology.orchestra.run.smithi005.stdout:client5915 2024-05-20T18:31:37.709 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:31:37.710 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:31:37.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:31:37.726 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-05-20T18:31:37.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:31:37.789 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-05-20T18:31:37.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:31:37.847 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-05-20T18:31:37.904 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:31:37.905 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:31:37.905 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:31:37.905 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:31:37.905 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-05-20T18:31:37.905 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:31:38.013 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627776 2024-05-20T18:31:38.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:31:38.021 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:31:38.022 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:31:38.079 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:31:38.079 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:31:38.136 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:31:38.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.290+0000 7fec35256640 1 Processor -- start 2024-05-20T18:31:38.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.290+0000 7fec35256640 1 -- start start 2024-05-20T18:31:38.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.292+0000 7fec35256640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec3006d3d0 0x7fec3006d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:38.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.292+0000 7fec35256640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fec300b8190 0x7fec300b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:38.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.292+0000 7fec35256640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec3006dd20 con 0x7fec30162820 2024-05-20T18:31:38.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.292+0000 7fec35256640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fec300d0630 con 0x7fec300b8190 2024-05-20T18:31:38.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.292+0000 7fec35256640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fec300d39c0 con 0x7fec3006d3d0 2024-05-20T18:31:38.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.292+0000 7fec2f59e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fec300b8190 0x7fec300b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:38.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.292+0000 7fec2f59e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fec300b8190 0x7fec300b85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58112/0 (socket says 172.21.15.5:58112) 2024-05-20T18:31:38.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.292+0000 7fec2f59e640 1 -- 172.21.15.5:0/3205398906 learned_addr learned my addr 172.21.15.5:0/3205398906 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:38.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.292+0000 7fec2e59c640 1 --2- 172.21.15.5:0/3205398906 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec3006d3d0 0x7fec3006d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:38.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.292+0000 7fec2f59e640 1 -- 172.21.15.5:0/3205398906 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec3006d3d0 msgr2=0x7fec3006d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:38.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.292+0000 7fec2f59e640 1 --2- 172.21.15.5:0/3205398906 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec3006d3d0 0x7fec3006d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:38.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.292+0000 7fec2f59e640 1 -- 172.21.15.5:0/3205398906 >> v1:172.21.15.63:6789/0 conn(0x7fec30162820 legacy=0x7fec300b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:38.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.292+0000 7fec2f59e640 1 -- 172.21.15.5:0/3205398906 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec300d3ca0 con 0x7fec300b8190 2024-05-20T18:31:38.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.293+0000 7fec2f59e640 1 --2- 172.21.15.5:0/3205398906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec300b8190 0x7fec300b85a0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fec28009870 tx=0x7fec2802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4c0b5f37cfa462 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:38.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.293+0000 7fec2dd9b640 1 -- 172.21.15.5:0/3205398906 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec28037070 con 0x7fec300b8190 2024-05-20T18:31:38.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.293+0000 7fec2dd9b640 1 -- 172.21.15.5:0/3205398906 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec28032440 con 0x7fec300b8190 2024-05-20T18:31:38.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.293+0000 7fec2dd9b640 1 -- 172.21.15.5:0/3205398906 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec28032810 con 0x7fec300b8190 2024-05-20T18:31:38.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.294+0000 7fec35256640 1 -- 172.21.15.5:0/3205398906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec300b8190 msgr2=0x7fec300b85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:38.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.294+0000 7fec35256640 1 --2- 172.21.15.5:0/3205398906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec300b8190 0x7fec300b85a0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fec28009870 tx=0x7fec2802a270 comp rx=0 tx=0).stop 2024-05-20T18:31:38.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.294+0000 7fec35256640 1 -- 172.21.15.5:0/3205398906 shutdown_connections 2024-05-20T18:31:38.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.294+0000 7fec35256640 1 --2- 172.21.15.5:0/3205398906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec300b8190 0x7fec300b85a0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:38.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.294+0000 7fec35256640 1 --2- 172.21.15.5:0/3205398906 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec3006d3d0 0x7fec3006d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:38.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.294+0000 7fec35256640 1 -- 172.21.15.5:0/3205398906 >> 172.21.15.5:0/3205398906 conn(0x7fec300d4bd0 msgr2=0x7fec300d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:38.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.294+0000 7fec35256640 1 -- 172.21.15.5:0/3205398906 shutdown_connections 2024-05-20T18:31:38.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.294+0000 7fec35256640 1 -- 172.21.15.5:0/3205398906 wait complete. 2024-05-20T18:31:38.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.296+0000 7fec35256640 1 Processor -- start 2024-05-20T18:31:38.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.296+0000 7fec35256640 1 -- start start 2024-05-20T18:31:38.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.297+0000 7fec35256640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec3006d3d0 0x7fec3014ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:38.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.297+0000 7fec35256640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec300b8190 0x7fec300cea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:38.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.297+0000 7fec35256640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec30162820 0x7fec300b6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:38.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.297+0000 7fec35256640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fec300d0300 con 0x7fec3006d3d0 2024-05-20T18:31:38.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.297+0000 7fec35256640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fec300d4260 con 0x7fec30162820 2024-05-20T18:31:38.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.297+0000 7fec35256640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fec300b4050 con 0x7fec300b8190 2024-05-20T18:31:38.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.297+0000 7fec2ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec3006d3d0 0x7fec3014ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:38.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.297+0000 7fec2ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec3006d3d0 0x7fec3014ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58116/0 (socket says 172.21.15.5:58116) 2024-05-20T18:31:38.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.297+0000 7fec2ed9d640 1 -- 172.21.15.5:0/3335605860 learned_addr learned my addr 172.21.15.5:0/3335605860 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:38.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.298+0000 7fec2e59c640 1 --2- 172.21.15.5:0/3335605860 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec300b8190 0x7fec300cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:38.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.298+0000 7fec2f59e640 1 --2- 172.21.15.5:0/3335605860 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec30162820 0x7fec300b6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:38.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.298+0000 7fec2ed9d640 1 -- 172.21.15.5:0/3335605860 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec300b8190 msgr2=0x7fec300cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:38.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.298+0000 7fec2ed9d640 1 --2- 172.21.15.5:0/3335605860 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec300b8190 0x7fec300cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:38.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.298+0000 7fec2ed9d640 1 -- 172.21.15.5:0/3335605860 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec30162820 msgr2=0x7fec300b6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:38.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.298+0000 7fec2ed9d640 1 --2- 172.21.15.5:0/3335605860 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec30162820 0x7fec300b6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:38.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.298+0000 7fec2ed9d640 1 -- 172.21.15.5:0/3335605860 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec300cbdb0 con 0x7fec3006d3d0 2024-05-20T18:31:38.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.298+0000 7fec2ed9d640 1 --2- 172.21.15.5:0/3335605860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec3006d3d0 0x7fec3014ae10 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fec2000c760 tx=0x7fec2000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:38.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.298+0000 7fec277fe640 1 -- 172.21.15.5:0/3335605860 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec20016020 con 0x7fec3006d3d0 2024-05-20T18:31:38.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.298+0000 7fec277fe640 1 -- 172.21.15.5:0/3335605860 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec20006d50 con 0x7fec3006d3d0 2024-05-20T18:31:38.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.298+0000 7fec35256640 1 -- 172.21.15.5:0/3335605860 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec300cc090 con 0x7fec3006d3d0 2024-05-20T18:31:38.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.298+0000 7fec277fe640 1 -- 172.21.15.5:0/3335605860 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec20003bb0 con 0x7fec3006d3d0 2024-05-20T18:31:38.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.298+0000 7fec35256640 1 -- 172.21.15.5:0/3335605860 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec300cc650 con 0x7fec3006d3d0 2024-05-20T18:31:38.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.299+0000 7fec35256640 1 -- 172.21.15.5:0/3335605860 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec30075b00 con 0x7fec3006d3d0 2024-05-20T18:31:38.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.300+0000 7fec277fe640 1 -- 172.21.15.5:0/3335605860 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fec2000a850 con 0x7fec3006d3d0 2024-05-20T18:31:38.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.300+0000 7fec277fe640 1 --2- 172.21.15.5:0/3335605860 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fec1006be30 0x7fec1006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:38.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.300+0000 7fec277fe640 1 -- 172.21.15.5:0/3335605860 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6101+0+0 (secure 0 0 0) 0x7fec2008ac90 con 0x7fec3006d3d0 2024-05-20T18:31:38.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.300+0000 7fec2e59c640 1 --2- 172.21.15.5:0/3335605860 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fec1006be30 0x7fec1006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:38.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.301+0000 7fec2e59c640 1 --2- 172.21.15.5:0/3335605860 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fec1006be30 0x7fec1006e340 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fec18006fd0 tx=0x7fec18008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:38.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.305+0000 7fec277fe640 1 -- 172.21.15.5:0/3335605860 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fec20020020 con 0x7fec3006d3d0 2024-05-20T18:31:38.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.488+0000 7fec35256640 1 -- 172.21.15.5:0/3335605860 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec3006e150 con 0x7fec3006d3d0 2024-05-20T18:31:38.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.490+0000 7fec277fe640 1 -- 172.21.15.5:0/3335605860 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+16825 (secure 0 0 0) 0x7fec3006dd20 con 0x7fec3006d3d0 2024-05-20T18:31:38.490 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:31:38.490 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":77,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:31:31.787691+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:31:27.896386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-20T18:31:29.146912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:31:38.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.492+0000 7fec35256640 1 -- 172.21.15.5:0/3335605860 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fec1006be30 msgr2=0x7fec1006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:38.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.492+0000 7fec35256640 1 --2- 172.21.15.5:0/3335605860 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fec1006be30 0x7fec1006e340 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fec18006fd0 tx=0x7fec18008040 comp rx=0 tx=0).stop 2024-05-20T18:31:38.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.492+0000 7fec35256640 1 -- 172.21.15.5:0/3335605860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec3006d3d0 msgr2=0x7fec3014ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:38.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.492+0000 7fec35256640 1 --2- 172.21.15.5:0/3335605860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec3006d3d0 0x7fec3014ae10 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fec2000c760 tx=0x7fec2000cc20 comp rx=0 tx=0).stop 2024-05-20T18:31:38.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.492+0000 7fec35256640 1 -- 172.21.15.5:0/3335605860 shutdown_connections 2024-05-20T18:31:38.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.492+0000 7fec35256640 1 --2- 172.21.15.5:0/3335605860 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fec1006be30 0x7fec1006e340 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:38.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.492+0000 7fec35256640 1 --2- 172.21.15.5:0/3335605860 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec30162820 0x7fec300b6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:38.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.492+0000 7fec35256640 1 --2- 172.21.15.5:0/3335605860 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec300b8190 0x7fec300cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:38.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.492+0000 7fec35256640 1 --2- 172.21.15.5:0/3335605860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec3006d3d0 0x7fec3014ae10 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:38.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.492+0000 7fec35256640 1 -- 172.21.15.5:0/3335605860 >> 172.21.15.5:0/3335605860 conn(0x7fec300d4bd0 msgr2=0x7fec3015d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:38.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.492+0000 7fec35256640 1 -- 172.21.15.5:0/3335605860 shutdown_connections 2024-05-20T18:31:38.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.492+0000 7fec35256640 1 -- 172.21.15.5:0/3335605860 wait complete. 2024-05-20T18:31:38.504 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:31:27.896386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:31:29.146912+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:31:38.504 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:31:38.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.622+0000 7f4c64310640 1 Processor -- start 2024-05-20T18:31:38.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.622+0000 7f4c64310640 1 -- start start 2024-05-20T18:31:38.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.623+0000 7f4c64310640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c5c163a10 0x7f4c5c0d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:38.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.623+0000 7f4c64310640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4c5c0b4010 0x7f4c5c0cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:38.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.623+0000 7f4c64310640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c5c0d0370 con 0x7f4c5c0d14b0 2024-05-20T18:31:38.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.623+0000 7f4c64310640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f4c5c0d3ca0 con 0x7f4c5c0b4010 2024-05-20T18:31:38.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.623+0000 7f4c64310640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4c5c0d0650 con 0x7f4c5c163a10 2024-05-20T18:31:38.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.623+0000 7f4c618ab640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4c5c0b4010 0x7f4c5c0cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:38.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.623+0000 7f4c618ab640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4c5c0b4010 0x7f4c5c0cfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58122/0 (socket says 172.21.15.5:58122) 2024-05-20T18:31:38.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.623+0000 7f4c618ab640 1 -- 172.21.15.5:0/1083990237 learned_addr learned my addr 172.21.15.5:0/1083990237 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:38.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.624+0000 7f4c620ac640 1 --2- 172.21.15.5:0/1083990237 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c5c163a10 0x7f4c5c0d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:38.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.624+0000 7f4c618ab640 1 -- 172.21.15.5:0/1083990237 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c5c163a10 msgr2=0x7f4c5c0d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:38.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.624+0000 7f4c618ab640 1 --2- 172.21.15.5:0/1083990237 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c5c163a10 0x7f4c5c0d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:38.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.624+0000 7f4c618ab640 1 -- 172.21.15.5:0/1083990237 >> v1:172.21.15.63:6789/0 conn(0x7f4c5c0d14b0 legacy=0x7f4c5c0d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:38.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.624+0000 7f4c618ab640 1 -- 172.21.15.5:0/1083990237 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c5c0d3650 con 0x7f4c5c0b4010 2024-05-20T18:31:38.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.625+0000 7f4c618ab640 1 --2- 172.21.15.5:0/1083990237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c5c0b4010 0x7f4c5c0cfb00 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f4c50009870 tx=0x7f4c5002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5956f1bbd0d67695 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:38.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.625+0000 7f4c610aa640 1 -- 172.21.15.5:0/1083990237 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c50037070 con 0x7f4c5c0b4010 2024-05-20T18:31:38.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.625+0000 7f4c610aa640 1 -- 172.21.15.5:0/1083990237 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c50032440 con 0x7f4c5c0b4010 2024-05-20T18:31:38.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.625+0000 7f4c610aa640 1 -- 172.21.15.5:0/1083990237 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c50032810 con 0x7f4c5c0b4010 2024-05-20T18:31:38.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.625+0000 7f4c64310640 1 -- 172.21.15.5:0/1083990237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c5c0b4010 msgr2=0x7f4c5c0cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:38.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.625+0000 7f4c64310640 1 --2- 172.21.15.5:0/1083990237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c5c0b4010 0x7f4c5c0cfb00 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f4c50009870 tx=0x7f4c5002a270 comp rx=0 tx=0).stop 2024-05-20T18:31:38.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.626+0000 7f4c64310640 1 -- 172.21.15.5:0/1083990237 shutdown_connections 2024-05-20T18:31:38.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.626+0000 7f4c64310640 1 --2- 172.21.15.5:0/1083990237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c5c0b4010 0x7f4c5c0cfb00 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:38.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.626+0000 7f4c64310640 1 --2- 172.21.15.5:0/1083990237 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c5c163a10 0x7f4c5c0d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:38.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.626+0000 7f4c64310640 1 -- 172.21.15.5:0/1083990237 >> 172.21.15.5:0/1083990237 conn(0x7f4c5c06d240 msgr2=0x7f4c5c06d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:38.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.626+0000 7f4c64310640 1 -- 172.21.15.5:0/1083990237 shutdown_connections 2024-05-20T18:31:38.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.626+0000 7f4c64310640 1 -- 172.21.15.5:0/1083990237 wait complete. 2024-05-20T18:31:38.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.627+0000 7f4c64310640 1 Processor -- start 2024-05-20T18:31:38.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.627+0000 7f4c64310640 1 -- start start 2024-05-20T18:31:38.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.629+0000 7f4c64310640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c5c0b4010 0x7f4c5c0afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:38.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.629+0000 7f4c64310640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c5c0d14b0 0x7f4c5c1eafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:38.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.629+0000 7f4c64310640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c5c163a10 0x7f4c5c14f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:38.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.629+0000 7f4c64310640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c0b8210 con 0x7f4c5c0d14b0 2024-05-20T18:31:38.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.629+0000 7f4c64310640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c0d0040 con 0x7f4c5c0b4010 2024-05-20T18:31:38.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.629+0000 7f4c64310640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4c5c0b8390 con 0x7f4c5c163a10 2024-05-20T18:31:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.629+0000 7f4c618ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c5c0d14b0 0x7f4c5c1eafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.629+0000 7f4c618ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c5c0d14b0 0x7f4c5c1eafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58132/0 (socket says 172.21.15.5:58132) 2024-05-20T18:31:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.629+0000 7f4c618ab640 1 -- 172.21.15.5:0/1564661329 learned_addr learned my addr 172.21.15.5:0/1564661329 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.629+0000 7f4c620ac640 1 --2- 172.21.15.5:0/1564661329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c5c0b4010 0x7f4c5c0afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.629+0000 7f4c628ad640 1 --2- 172.21.15.5:0/1564661329 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c5c163a10 0x7f4c5c14f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.629+0000 7f4c618ab640 1 -- 172.21.15.5:0/1564661329 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c5c163a10 msgr2=0x7f4c5c14f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.629+0000 7f4c618ab640 1 --2- 172.21.15.5:0/1564661329 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c5c163a10 0x7f4c5c14f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.629+0000 7f4c618ab640 1 -- 172.21.15.5:0/1564661329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c5c0b4010 msgr2=0x7f4c5c0afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.629+0000 7f4c618ab640 1 --2- 172.21.15.5:0/1564661329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c5c0b4010 0x7f4c5c0afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.629+0000 7f4c618ab640 1 -- 172.21.15.5:0/1564661329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c5c1f55b0 con 0x7f4c5c0d14b0 2024-05-20T18:31:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.629+0000 7f4c628ad640 1 --2- 172.21.15.5:0/1564661329 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c5c163a10 0x7f4c5c14f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.629+0000 7f4c618ab640 1 --2- 172.21.15.5:0/1564661329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c5c0d14b0 0x7f4c5c1eafb0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f4c50000c00 tx=0x7f4c500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.630+0000 7f4c5affd640 1 -- 172.21.15.5:0/1564661329 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c50037040 con 0x7f4c5c0d14b0 2024-05-20T18:31:38.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.630+0000 7f4c5affd640 1 -- 172.21.15.5:0/1564661329 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c50042070 con 0x7f4c5c0d14b0 2024-05-20T18:31:38.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.630+0000 7f4c64310640 1 -- 172.21.15.5:0/1564661329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c5c0b1b70 con 0x7f4c5c0d14b0 2024-05-20T18:31:38.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.630+0000 7f4c5affd640 1 -- 172.21.15.5:0/1564661329 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c50030880 con 0x7f4c5c0d14b0 2024-05-20T18:31:38.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.630+0000 7f4c64310640 1 -- 172.21.15.5:0/1564661329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c5c0b2130 con 0x7f4c5c0d14b0 2024-05-20T18:31:38.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.631+0000 7f4c64310640 1 -- 172.21.15.5:0/1564661329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c5c0b01d0 con 0x7f4c5c0d14b0 2024-05-20T18:31:38.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.631+0000 7f4c5affd640 1 -- 172.21.15.5:0/1564661329 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4c50030a20 con 0x7f4c5c0d14b0 2024-05-20T18:31:38.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.632+0000 7f4c5affd640 1 --2- 172.21.15.5:0/1564661329 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4c4806be30 0x7f4c4806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:38.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.632+0000 7f4c620ac640 1 --2- 172.21.15.5:0/1564661329 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4c4806be30 0x7f4c4806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:38.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.632+0000 7f4c5affd640 1 -- 172.21.15.5:0/1564661329 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6101+0+0 (secure 0 0 0) 0x7f4c500ab910 con 0x7f4c5c0d14b0 2024-05-20T18:31:38.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.632+0000 7f4c620ac640 1 --2- 172.21.15.5:0/1564661329 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4c4806be30 0x7f4c4806e340 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c0d3020 tx=0x7f4c4c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:38.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.636+0000 7f4c5affd640 1 -- 172.21.15.5:0/1564661329 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4c50045360 con 0x7f4c5c0d14b0 2024-05-20T18:31:38.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.824+0000 7f4c64310640 1 -- 172.21.15.5:0/1564661329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4c5c075b00 con 0x7f4c5c0d14b0 2024-05-20T18:31:38.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.825+0000 7f4c5affd640 1 -- 172.21.15.5:0/1564661329 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7f4c5007a0b0 con 0x7f4c5c0d14b0 2024-05-20T18:31:38.825 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:31:38.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.827+0000 7f4c64310640 1 -- 172.21.15.5:0/1564661329 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4c4806be30 msgr2=0x7f4c4806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:38.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.827+0000 7f4c64310640 1 --2- 172.21.15.5:0/1564661329 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4c4806be30 0x7f4c4806e340 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c0d3020 tx=0x7f4c4c00b040 comp rx=0 tx=0).stop 2024-05-20T18:31:38.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.827+0000 7f4c64310640 1 -- 172.21.15.5:0/1564661329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c5c0d14b0 msgr2=0x7f4c5c1eafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:38.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.827+0000 7f4c64310640 1 --2- 172.21.15.5:0/1564661329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c5c0d14b0 0x7f4c5c1eafb0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f4c50000c00 tx=0x7f4c500046c0 comp rx=0 tx=0).stop 2024-05-20T18:31:38.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.827+0000 7f4c64310640 1 -- 172.21.15.5:0/1564661329 shutdown_connections 2024-05-20T18:31:38.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.827+0000 7f4c64310640 1 --2- 172.21.15.5:0/1564661329 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4c4806be30 0x7f4c4806e340 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:38.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.827+0000 7f4c64310640 1 --2- 172.21.15.5:0/1564661329 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c5c163a10 0x7f4c5c14f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:38.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.827+0000 7f4c64310640 1 --2- 172.21.15.5:0/1564661329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c5c0d14b0 0x7f4c5c1eafb0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:38.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.827+0000 7f4c64310640 1 --2- 172.21.15.5:0/1564661329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c5c0b4010 0x7f4c5c0afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:38.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.827+0000 7f4c64310640 1 -- 172.21.15.5:0/1564661329 >> 172.21.15.5:0/1564661329 conn(0x7f4c5c06d240 msgr2=0x7f4c5c161f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:38.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.827+0000 7f4c64310640 1 -- 172.21.15.5:0/1564661329 shutdown_connections 2024-05-20T18:31:38.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.827+0000 7f4c64310640 1 -- 172.21.15.5:0/1564661329 wait complete. 2024-05-20T18:31:38.838 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:31:38.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.956+0000 7f538074c640 1 Processor -- start 2024-05-20T18:31:38.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.956+0000 7f538074c640 1 -- start start 2024-05-20T18:31:38.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.957+0000 7f538074c640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5378163a10 0x7f53780d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:38.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.957+0000 7f538074c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f53780b4010 0x7f53780cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:38.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.958+0000 7f538074c640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53780d0370 con 0x7f53780d14b0 2024-05-20T18:31:38.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.958+0000 7f538074c640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f53780d3ca0 con 0x7f53780b4010 2024-05-20T18:31:38.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.958+0000 7f538074c640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f53780d0650 con 0x7f5378163a10 2024-05-20T18:31:38.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.958+0000 7f537dce7640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f53780b4010 0x7f53780cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:38.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.958+0000 7f537ece9640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f53780d14b0 0x7f53780d18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56882/0 (socket says 172.21.15.5:56882) 2024-05-20T18:31:38.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.958+0000 7f537ece9640 1 -- 172.21.15.5:0/423562258 learned_addr learned my addr 172.21.15.5:0/423562258 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:38.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.958+0000 7f537e4e8640 1 --2- 172.21.15.5:0/423562258 >> v2:172.21.15.63:3300/0 conn(0x7f5378163a10 0x7f53780d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:38.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.958+0000 7f537dce7640 1 -- 172.21.15.5:0/423562258 >> v2:172.21.15.63:3300/0 conn(0x7f5378163a10 msgr2=0x7f53780d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:38.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.958+0000 7f537dce7640 1 --2- 172.21.15.5:0/423562258 >> v2:172.21.15.63:3300/0 conn(0x7f5378163a10 0x7f53780d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:38.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.958+0000 7f537dce7640 1 -- 172.21.15.5:0/423562258 >> v1:172.21.15.5:6789/0 conn(0x7f53780d14b0 legacy=0x7f53780d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:38.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.958+0000 7f537dce7640 1 -- 172.21.15.5:0/423562258 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53780d3650 con 0x7f53780b4010 2024-05-20T18:31:38.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.958+0000 7f537d4e6640 1 -- 172.21.15.5:0/423562258 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1792481963 0 0) 0x7f53780d0370 con 0x7f53780d14b0 2024-05-20T18:31:38.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.958+0000 7f537e4e8640 1 --2- 172.21.15.5:0/423562258 >> v2:172.21.15.63:3300/0 conn(0x7f5378163a10 0x7f53780d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:38.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.959+0000 7f537dce7640 1 --2- 172.21.15.5:0/423562258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53780b4010 0x7f53780cfb00 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f53780cf1c0 tx=0x7f536802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd05beda948cec3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:38.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.959+0000 7f537d4e6640 1 -- 172.21.15.5:0/423562258 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368037070 con 0x7f53780b4010 2024-05-20T18:31:38.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.959+0000 7f537d4e6640 1 -- 172.21.15.5:0/423562258 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5368032440 con 0x7f53780b4010 2024-05-20T18:31:38.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.959+0000 7f537d4e6640 1 -- 172.21.15.5:0/423562258 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368032810 con 0x7f53780b4010 2024-05-20T18:31:38.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.960+0000 7f538074c640 1 -- 172.21.15.5:0/423562258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53780b4010 msgr2=0x7f53780cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:38.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.960+0000 7f538074c640 1 --2- 172.21.15.5:0/423562258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53780b4010 0x7f53780cfb00 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f53780cf1c0 tx=0x7f536802a1a0 comp rx=0 tx=0).stop 2024-05-20T18:31:38.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.960+0000 7f538074c640 1 -- 172.21.15.5:0/423562258 shutdown_connections 2024-05-20T18:31:38.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.960+0000 7f538074c640 1 --2- 172.21.15.5:0/423562258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53780b4010 0x7f53780cfb00 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:38.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.960+0000 7f538074c640 1 --2- 172.21.15.5:0/423562258 >> v2:172.21.15.63:3300/0 conn(0x7f5378163a10 0x7f53780d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:38.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.960+0000 7f538074c640 1 -- 172.21.15.5:0/423562258 >> 172.21.15.5:0/423562258 conn(0x7f537806d240 msgr2=0x7f537806d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:38.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.960+0000 7f538074c640 1 -- 172.21.15.5:0/423562258 shutdown_connections 2024-05-20T18:31:38.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.960+0000 7f538074c640 1 -- 172.21.15.5:0/423562258 wait complete. 2024-05-20T18:31:38.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.962+0000 7f538074c640 1 Processor -- start 2024-05-20T18:31:38.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.962+0000 7f538074c640 1 -- start start 2024-05-20T18:31:38.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.963+0000 7f538074c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f53780b4010 0x7f53780b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:38.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.963+0000 7f538074c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53780d14b0 0x7f53780b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:38.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.963+0000 7f538074c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5378163a10 0x7f53780b6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:38.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.963+0000 7f538074c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53780b8210 con 0x7f53780d14b0 2024-05-20T18:31:38.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.963+0000 7f538074c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f53780d0040 con 0x7f5378163a10 2024-05-20T18:31:38.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.963+0000 7f538074c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f53780b8390 con 0x7f53780b4010 2024-05-20T18:31:38.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.963+0000 7f537dce7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53780d14b0 0x7f53780b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:38.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.963+0000 7f537dce7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53780d14b0 0x7f53780b6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58148/0 (socket says 172.21.15.5:58148) 2024-05-20T18:31:38.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.963+0000 7f537dce7640 1 -- 172.21.15.5:0/3410812601 learned_addr learned my addr 172.21.15.5:0/3410812601 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:38.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.963+0000 7f537e4e8640 1 --2- 172.21.15.5:0/3410812601 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f53780b4010 0x7f53780b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:38.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.963+0000 7f537ece9640 1 --2- 172.21.15.5:0/3410812601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5378163a10 0x7f53780b6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:38.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.963+0000 7f537dce7640 1 -- 172.21.15.5:0/3410812601 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f53780b4010 msgr2=0x7f53780b7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:38.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.963+0000 7f537dce7640 1 --2- 172.21.15.5:0/3410812601 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f53780b4010 0x7f53780b7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:38.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.964+0000 7f537dce7640 1 -- 172.21.15.5:0/3410812601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5378163a10 msgr2=0x7f53780b6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:38.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.964+0000 7f537dce7640 1 --2- 172.21.15.5:0/3410812601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5378163a10 0x7f53780b6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:38.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.964+0000 7f537dce7640 1 -- 172.21.15.5:0/3410812601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53781f5cc0 con 0x7f53780d14b0 2024-05-20T18:31:38.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.964+0000 7f537ece9640 1 --2- 172.21.15.5:0/3410812601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5378163a10 0x7f53780b6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:38.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.964+0000 7f537dce7640 1 --2- 172.21.15.5:0/3410812601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53780d14b0 0x7f53780b6240 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f5368000c00 tx=0x7f53680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:38.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.964+0000 7f536effd640 1 -- 172.21.15.5:0/3410812601 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368037050 con 0x7f53780d14b0 2024-05-20T18:31:38.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.964+0000 7f536effd640 1 -- 172.21.15.5:0/3410812601 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5368038040 con 0x7f53780d14b0 2024-05-20T18:31:38.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.964+0000 7f538074c640 1 -- 172.21.15.5:0/3410812601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53781f5fa0 con 0x7f53780d14b0 2024-05-20T18:31:38.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.964+0000 7f536effd640 1 -- 172.21.15.5:0/3410812601 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368004470 con 0x7f53780d14b0 2024-05-20T18:31:38.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.964+0000 7f538074c640 1 -- 172.21.15.5:0/3410812601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53781f6560 con 0x7f53780d14b0 2024-05-20T18:31:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.965+0000 7f538074c640 1 -- 172.21.15.5:0/3410812601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5378075b00 con 0x7f53780d14b0 2024-05-20T18:31:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.966+0000 7f536effd640 1 -- 172.21.15.5:0/3410812601 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5368030900 con 0x7f53780d14b0 2024-05-20T18:31:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.966+0000 7f536effd640 1 --2- 172.21.15.5:0/3410812601 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f535c06be30 0x7f535c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.966+0000 7f536effd640 1 -- 172.21.15.5:0/3410812601 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6101+0+0 (secure 0 0 0) 0x7f53680aba60 con 0x7f53780d14b0 2024-05-20T18:31:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.967+0000 7f537e4e8640 1 --2- 172.21.15.5:0/3410812601 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f535c06be30 0x7f535c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:38.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.967+0000 7f537e4e8640 1 --2- 172.21.15.5:0/3410812601 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f535c06be30 0x7f535c06e340 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5374006fd0 tx=0x7f5374008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:38.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:38.971+0000 7f536effd640 1 -- 172.21.15.5:0/3410812601 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f536807a1f0 con 0x7f53780d14b0 2024-05-20T18:31:39.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.154+0000 7f538074c640 1 -- 172.21.15.5:0/3410812601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f53781f5a50 con 0x7f53780d14b0 2024-05-20T18:31:39.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.154+0000 7f536effd640 1 -- 172.21.15.5:0/3410812601 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) ==== 104+0+11 (secure 0 0 0) 0x7f5368079af0 con 0x7f53780d14b0 2024-05-20T18:31:39.155 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:31:39.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.156+0000 7f538074c640 1 -- 172.21.15.5:0/3410812601 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f535c06be30 msgr2=0x7f535c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:39.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.156+0000 7f538074c640 1 --2- 172.21.15.5:0/3410812601 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f535c06be30 0x7f535c06e340 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5374006fd0 tx=0x7f5374008040 comp rx=0 tx=0).stop 2024-05-20T18:31:39.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.156+0000 7f538074c640 1 -- 172.21.15.5:0/3410812601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53780d14b0 msgr2=0x7f53780b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:39.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.156+0000 7f538074c640 1 --2- 172.21.15.5:0/3410812601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53780d14b0 0x7f53780b6240 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f5368000c00 tx=0x7f53680046c0 comp rx=0 tx=0).stop 2024-05-20T18:31:39.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.156+0000 7f538074c640 1 -- 172.21.15.5:0/3410812601 shutdown_connections 2024-05-20T18:31:39.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.156+0000 7f538074c640 1 --2- 172.21.15.5:0/3410812601 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f535c06be30 0x7f535c06e340 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:39.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.156+0000 7f538074c640 1 --2- 172.21.15.5:0/3410812601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5378163a10 0x7f53780b6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:39.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.156+0000 7f538074c640 1 --2- 172.21.15.5:0/3410812601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53780d14b0 0x7f53780b6240 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:39.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.156+0000 7f538074c640 1 --2- 172.21.15.5:0/3410812601 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f53780b4010 0x7f53780b7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:39.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.156+0000 7f538074c640 1 -- 172.21.15.5:0/3410812601 >> 172.21.15.5:0/3410812601 conn(0x7f537806d240 msgr2=0x7f537815ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:39.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.156+0000 7f538074c640 1 -- 172.21.15.5:0/3410812601 shutdown_connections 2024-05-20T18:31:39.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.156+0000 7f538074c640 1 -- 172.21.15.5:0/3410812601 wait complete. 2024-05-20T18:31:39.168 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:31:39.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.286+0000 7f391bc61640 1 Processor -- start 2024-05-20T18:31:39.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.286+0000 7f391bc61640 1 -- start start 2024-05-20T18:31:39.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.287+0000 7f391bc61640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f391415d2a0 0x7f39140b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:39.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.287+0000 7f391bc61640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f391409b440 con 0x7f39140d1fc0 2024-05-20T18:31:39.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.287+0000 7f391bc61640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39140b30e0 con 0x7f391409a4f0 2024-05-20T18:31:39.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.287+0000 7f391bc61640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f391406b6a0 con 0x7f391415d2a0 2024-05-20T18:31:39.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.287+0000 7f39191fc640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f39140d1fc0 0x7f391406c180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56888/0 (socket says 172.21.15.5:56888) 2024-05-20T18:31:39.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.288+0000 7f39191fc640 1 -- 172.21.15.5:0/769210892 learned_addr learned my addr 172.21.15.5:0/769210892 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:39.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.288+0000 7f391a1fe640 1 --2- 172.21.15.5:0/769210892 >> v2:172.21.15.63:3300/0 conn(0x7f391415d2a0 0x7f39140b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:39.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.288+0000 7f39189fb640 1 -- 172.21.15.5:0/769210892 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2566597348 0 0) 0x7f391409b440 con 0x7f39140d1fc0 2024-05-20T18:31:39.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.288+0000 7f39189fb640 1 -- 172.21.15.5:0/769210892 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3900003520 con 0x7f39140d1fc0 2024-05-20T18:31:39.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.288+0000 7f39189fb640 1 -- 172.21.15.5:0/769210892 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 391095510 0 0) 0x7f3900003520 con 0x7f39140d1fc0 2024-05-20T18:31:39.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.288+0000 7f39189fb640 1 -- 172.21.15.5:0/769210892 >> v2:172.21.15.63:3300/0 conn(0x7f391415d2a0 msgr2=0x7f39140b2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:39.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.288+0000 7f39189fb640 1 --2- 172.21.15.5:0/769210892 >> v2:172.21.15.63:3300/0 conn(0x7f391415d2a0 0x7f39140b2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:39.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.288+0000 7f39189fb640 1 -- 172.21.15.5:0/769210892 >> v1:172.21.15.63:6789/0 conn(0x7f391409a4f0 legacy=0x7f39140d18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:39.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.288+0000 7f39189fb640 1 -- 172.21.15.5:0/769210892 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391406b980 con 0x7f39140d1fc0 2024-05-20T18:31:39.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.288+0000 7f39189fb640 1 -- 172.21.15.5:0/769210892 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f3904001670 con 0x7f39140d1fc0 2024-05-20T18:31:39.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.288+0000 7f39189fb640 1 -- 172.21.15.5:0/769210892 >> v1:172.21.15.5:6789/0 conn(0x7f39140d1fc0 legacy=0x7f391406c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:39.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.288+0000 7f39189fb640 1 --2- 172.21.15.5:0/769210892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39000039b0 0x7f3900003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:39.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.288+0000 7f39189fb640 1 --2- 172.21.15.5:0/769210892 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3900004800 0x7f3900004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:39.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.288+0000 7f39189fb640 1 --2- 172.21.15.5:0/769210892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39000051c0 0x7f390000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:39.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.288+0000 7f39189fb640 1 -- 172.21.15.5:0/769210892 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f391406b6a0 con 0x7f39000039b0 2024-05-20T18:31:39.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.288+0000 7f39189fb640 1 -- 172.21.15.5:0/769210892 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f390000c160 con 0x7f39000051c0 2024-05-20T18:31:39.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.288+0000 7f39189fb640 1 -- 172.21.15.5:0/769210892 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f390000c360 con 0x7f3900004800 2024-05-20T18:31:39.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.288+0000 7f39199fd640 1 --2- 172.21.15.5:0/769210892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39000039b0 0x7f3900003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:39.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.288+0000 7f391a1fe640 1 --2- 172.21.15.5:0/769210892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39000051c0 0x7f390000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:39.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.289+0000 7f39191fc640 1 --2- 172.21.15.5:0/769210892 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3900004800 0x7f3900004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:39.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.289+0000 7f39199fd640 1 -- 172.21.15.5:0/769210892 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3900004800 msgr2=0x7f3900004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:39.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.289+0000 7f39199fd640 1 --2- 172.21.15.5:0/769210892 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3900004800 0x7f3900004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:39.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.289+0000 7f39199fd640 1 -- 172.21.15.5:0/769210892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39000051c0 msgr2=0x7f390000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:39.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.289+0000 7f39199fd640 1 --2- 172.21.15.5:0/769210892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39000051c0 0x7f390000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:39.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.289+0000 7f391a1fe640 1 -- 172.21.15.5:0/769210892 reap_dead start 2024-05-20T18:31:39.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.289+0000 7f39199fd640 1 -- 172.21.15.5:0/769210892 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3900003520 con 0x7f39000039b0 2024-05-20T18:31:39.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.289+0000 7f39191fc640 1 --2- 172.21.15.5:0/769210892 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3900004800 0x7f3900004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:39.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.289+0000 7f39199fd640 1 --2- 172.21.15.5:0/769210892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39000039b0 0x7f3900003e20 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f3910001750 tx=0x7f3900005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb40f53cfe4b8f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:39.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.290+0000 7f39189fb640 1 -- 172.21.15.5:0/769210892 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3910001370 con 0x7f39000039b0 2024-05-20T18:31:39.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.290+0000 7f39189fb640 1 -- 172.21.15.5:0/769210892 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39100052d0 con 0x7f39000039b0 2024-05-20T18:31:39.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.290+0000 7f39189fb640 1 -- 172.21.15.5:0/769210892 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391002ab00 con 0x7f39000039b0 2024-05-20T18:31:39.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.290+0000 7f391bc61640 1 -- 172.21.15.5:0/769210892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39000039b0 msgr2=0x7f3900003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:39.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.290+0000 7f391bc61640 1 --2- 172.21.15.5:0/769210892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39000039b0 0x7f3900003e20 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f3910001750 tx=0x7f3900005830 comp rx=0 tx=0).stop 2024-05-20T18:31:39.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.290+0000 7f391bc61640 1 -- 172.21.15.5:0/769210892 shutdown_connections 2024-05-20T18:31:39.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.290+0000 7f391bc61640 1 --2- 172.21.15.5:0/769210892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39000039b0 0x7f3900003e20 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:39.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.290+0000 7f391bc61640 1 -- 172.21.15.5:0/769210892 >> 172.21.15.5:0/769210892 conn(0x7f39140b6620 msgr2=0x7f39140b6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:39.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.291+0000 7f391bc61640 1 -- 172.21.15.5:0/769210892 shutdown_connections 2024-05-20T18:31:39.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.291+0000 7f391bc61640 1 -- 172.21.15.5:0/769210892 wait complete. 2024-05-20T18:31:39.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.292+0000 7f391bc61640 1 Processor -- start 2024-05-20T18:31:39.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.292+0000 7f391bc61640 1 -- start start 2024-05-20T18:31:39.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.294+0000 7f391bc61640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39000039b0 0x7f391415e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:39.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.294+0000 7f391bc61640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f391415d6b0 0x7f3914156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:39.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.294+0000 7f391bc61640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f391406c080 0x7f391406c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:39.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.294+0000 7f391bc61640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39140b2bc0 con 0x7f39000039b0 2024-05-20T18:31:39.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.294+0000 7f391bc61640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f39140b2a40 con 0x7f391406c080 2024-05-20T18:31:39.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.294+0000 7f391bc61640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f39140b2d40 con 0x7f391415d6b0 2024-05-20T18:31:39.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.294+0000 7f39199fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39000039b0 0x7f391415e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:39.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.294+0000 7f39199fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39000039b0 0x7f391415e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58160/0 (socket says 172.21.15.5:58160) 2024-05-20T18:31:39.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.294+0000 7f39199fd640 1 -- 172.21.15.5:0/3428139082 learned_addr learned my addr 172.21.15.5:0/3428139082 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:39.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.294+0000 7f391a1fe640 1 --2- 172.21.15.5:0/3428139082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f391406c080 0x7f391406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:39.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.294+0000 7f39191fc640 1 --2- 172.21.15.5:0/3428139082 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f391415d6b0 0x7f3914156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:39.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.294+0000 7f39199fd640 1 -- 172.21.15.5:0/3428139082 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f391415d6b0 msgr2=0x7f3914156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:39.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.294+0000 7f39199fd640 1 --2- 172.21.15.5:0/3428139082 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f391415d6b0 0x7f3914156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:39.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.294+0000 7f39199fd640 1 -- 172.21.15.5:0/3428139082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f391406c080 msgr2=0x7f391406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:39.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.294+0000 7f39199fd640 1 --2- 172.21.15.5:0/3428139082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f391406c080 0x7f391406c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:39.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.294+0000 7f39199fd640 1 -- 172.21.15.5:0/3428139082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391406c9d0 con 0x7f39000039b0 2024-05-20T18:31:39.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.294+0000 7f391a1fe640 1 --2- 172.21.15.5:0/3428139082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f391406c080 0x7f391406c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:39.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.294+0000 7f39199fd640 1 --2- 172.21.15.5:0/3428139082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39000039b0 0x7f391415e250 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f39100017b0 tx=0x7f3910001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:39.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.294+0000 7f390a7fc640 1 -- 172.21.15.5:0/3428139082 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3910037040 con 0x7f39000039b0 2024-05-20T18:31:39.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.294+0000 7f390a7fc640 1 -- 172.21.15.5:0/3428139082 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3910040070 con 0x7f39000039b0 2024-05-20T18:31:39.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.295+0000 7f391bc61640 1 -- 172.21.15.5:0/3428139082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f391415eaa0 con 0x7f39000039b0 2024-05-20T18:31:39.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.295+0000 7f390a7fc640 1 -- 172.21.15.5:0/3428139082 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3910007690 con 0x7f39000039b0 2024-05-20T18:31:39.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.295+0000 7f391bc61640 1 -- 172.21.15.5:0/3428139082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39140ca250 con 0x7f39000039b0 2024-05-20T18:31:39.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.296+0000 7f391bc61640 1 -- 172.21.15.5:0/3428139082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3914075b00 con 0x7f39000039b0 2024-05-20T18:31:39.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.296+0000 7f390a7fc640 1 -- 172.21.15.5:0/3428139082 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f39100052d0 con 0x7f39000039b0 2024-05-20T18:31:39.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.297+0000 7f390a7fc640 1 --2- 172.21.15.5:0/3428139082 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f38f806bfd0 0x7f38f806e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:39.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.297+0000 7f390a7fc640 1 -- 172.21.15.5:0/3428139082 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6101+0+0 (secure 0 0 0) 0x7f39100aadf0 con 0x7f39000039b0 2024-05-20T18:31:39.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.297+0000 7f39191fc640 1 --2- 172.21.15.5:0/3428139082 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f38f806bfd0 0x7f38f806e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:39.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.297+0000 7f39191fc640 1 --2- 172.21.15.5:0/3428139082 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f38f806bfd0 0x7f38f806e4e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f391415deb0 tx=0x7f3904009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:39.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.301+0000 7f390a7fc640 1 -- 172.21.15.5:0/3428139082 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f391406c9d0 con 0x7f39000039b0 2024-05-20T18:31:39.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.484+0000 7f391bc61640 1 -- 172.21.15.5:0/3428139082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f391406fcd0 con 0x7f39000039b0 2024-05-20T18:31:39.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.484+0000 7f390a7fc640 1 -- 172.21.15.5:0/3428139082 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) ==== 100+0+11 (secure 0 0 0) 0x7f3910078f30 con 0x7f39000039b0 2024-05-20T18:31:39.485 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:31:39.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.486+0000 7f391bc61640 1 -- 172.21.15.5:0/3428139082 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f38f806bfd0 msgr2=0x7f38f806e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:39.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.486+0000 7f391bc61640 1 --2- 172.21.15.5:0/3428139082 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f38f806bfd0 0x7f38f806e4e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f391415deb0 tx=0x7f3904009040 comp rx=0 tx=0).stop 2024-05-20T18:31:39.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.486+0000 7f391bc61640 1 -- 172.21.15.5:0/3428139082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39000039b0 msgr2=0x7f391415e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:39.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.486+0000 7f391bc61640 1 --2- 172.21.15.5:0/3428139082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39000039b0 0x7f391415e250 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f39100017b0 tx=0x7f3910001120 comp rx=0 tx=0).stop 2024-05-20T18:31:39.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.486+0000 7f391bc61640 1 -- 172.21.15.5:0/3428139082 shutdown_connections 2024-05-20T18:31:39.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.486+0000 7f391bc61640 1 --2- 172.21.15.5:0/3428139082 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f38f806bfd0 0x7f38f806e4e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:39.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.486+0000 7f391bc61640 1 --2- 172.21.15.5:0/3428139082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f391406c080 0x7f391406c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:39.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.486+0000 7f391bc61640 1 --2- 172.21.15.5:0/3428139082 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f391415d6b0 0x7f3914156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:39.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.486+0000 7f391bc61640 1 --2- 172.21.15.5:0/3428139082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39000039b0 0x7f391415e250 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:39.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.486+0000 7f391bc61640 1 -- 172.21.15.5:0/3428139082 >> 172.21.15.5:0/3428139082 conn(0x7f39140b6620 msgr2=0x7f39140b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:39.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.486+0000 7f391bc61640 1 -- 172.21.15.5:0/3428139082 shutdown_connections 2024-05-20T18:31:39.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.486+0000 7f391bc61640 1 -- 172.21.15.5:0/3428139082 wait complete. 2024-05-20T18:31:39.498 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T18:31:39.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.614+0000 7f1c34ad7640 1 Processor -- start 2024-05-20T18:31:39.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.614+0000 7f1c34ad7640 1 -- start start 2024-05-20T18:31:39.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.615+0000 7f1c34ad7640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1c300d4400 0x7f1c3015f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:39.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.615+0000 7f1c34ad7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c300d2780 0x7f1c300d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:39.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.615+0000 7f1c34ad7640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c3008d2a0 con 0x7f1c3006cf00 2024-05-20T18:31:39.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.615+0000 7f1c34ad7640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f1c300d3f70 con 0x7f1c300d4400 2024-05-20T18:31:39.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.615+0000 7f1c34ad7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1c300d30d0 con 0x7f1c300d2780 2024-05-20T18:31:39.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.615+0000 7f1c2e59c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1c300d4400 0x7f1c3015f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:39.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.615+0000 7f1c2e59c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1c300d4400 0x7f1c3015f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58174/0 (socket says 172.21.15.5:58174) 2024-05-20T18:31:39.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.615+0000 7f1c2e59c640 1 -- 172.21.15.5:0/2865083209 learned_addr learned my addr 172.21.15.5:0/2865083209 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:39.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.616+0000 7f1c2ed9d640 1 --2- 172.21.15.5:0/2865083209 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c300d2780 0x7f1c300d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:39.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.616+0000 7f1c2e59c640 1 -- 172.21.15.5:0/2865083209 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c300d2780 msgr2=0x7f1c300d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:39.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.616+0000 7f1c2e59c640 1 --2- 172.21.15.5:0/2865083209 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c300d2780 0x7f1c300d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:39.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.616+0000 7f1c2e59c640 1 -- 172.21.15.5:0/2865083209 >> v1:172.21.15.63:6789/0 conn(0x7f1c3006cf00 legacy=0x7f1c3006d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:39.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.616+0000 7f1c2e59c640 1 -- 172.21.15.5:0/2865083209 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c300d4810 con 0x7f1c300d4400 2024-05-20T18:31:39.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.616+0000 7f1c2e59c640 1 --2- 172.21.15.5:0/2865083209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c300d4400 0x7f1c3015f440 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009870 tx=0x7f1c2002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea671678024fa943 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:39.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.617+0000 7f1c2dd9b640 1 -- 172.21.15.5:0/2865083209 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c20037070 con 0x7f1c300d4400 2024-05-20T18:31:39.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.617+0000 7f1c2dd9b640 1 -- 172.21.15.5:0/2865083209 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c20032440 con 0x7f1c300d4400 2024-05-20T18:31:39.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.617+0000 7f1c2dd9b640 1 -- 172.21.15.5:0/2865083209 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c20032810 con 0x7f1c300d4400 2024-05-20T18:31:39.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.617+0000 7f1c34ad7640 1 -- 172.21.15.5:0/2865083209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c300d4400 msgr2=0x7f1c3015f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:39.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.617+0000 7f1c34ad7640 1 --2- 172.21.15.5:0/2865083209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c300d4400 0x7f1c3015f440 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009870 tx=0x7f1c2002a2a0 comp rx=0 tx=0).stop 2024-05-20T18:31:39.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.617+0000 7f1c34ad7640 1 -- 172.21.15.5:0/2865083209 shutdown_connections 2024-05-20T18:31:39.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.617+0000 7f1c34ad7640 1 --2- 172.21.15.5:0/2865083209 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c300d2780 0x7f1c300d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:39.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.617+0000 7f1c34ad7640 1 --2- 172.21.15.5:0/2865083209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c300d4400 0x7f1c3015f440 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:39.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.617+0000 7f1c34ad7640 1 -- 172.21.15.5:0/2865083209 >> 172.21.15.5:0/2865083209 conn(0x7f1c300d4bb0 msgr2=0x7f1c300d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:39.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.617+0000 7f1c34ad7640 1 -- 172.21.15.5:0/2865083209 shutdown_connections 2024-05-20T18:31:39.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.617+0000 7f1c34ad7640 1 -- 172.21.15.5:0/2865083209 wait complete. 2024-05-20T18:31:39.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.619+0000 7f1c34ad7640 1 Processor -- start 2024-05-20T18:31:39.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.619+0000 7f1c34ad7640 1 -- start start 2024-05-20T18:31:39.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.621+0000 7f1c34ad7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c3006cf00 0x7f1c300b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:39.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.621+0000 7f1c34ad7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c300d2780 0x7f1c301f65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:39.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.621+0000 7f1c34ad7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c300d4400 0x7f1c30161590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:39.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.621+0000 7f1c34ad7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c300d3920 con 0x7f1c300d4400 2024-05-20T18:31:39.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.621+0000 7f1c34ad7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1c300d3640 con 0x7f1c300d2780 2024-05-20T18:31:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.621+0000 7f1c34ad7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1c300d3c00 con 0x7f1c3006cf00 2024-05-20T18:31:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.621+0000 7f1c2ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c300d4400 0x7f1c30161590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.621+0000 7f1c2ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c300d4400 0x7f1c30161590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58182/0 (socket says 172.21.15.5:58182) 2024-05-20T18:31:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.621+0000 7f1c2ed9d640 1 -- 172.21.15.5:0/1905355460 learned_addr learned my addr 172.21.15.5:0/1905355460 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.621+0000 7f1c2e59c640 1 --2- 172.21.15.5:0/1905355460 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c3006cf00 0x7f1c300b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.621+0000 7f1c27fff640 1 --2- 172.21.15.5:0/1905355460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c300d2780 0x7f1c301f65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.621+0000 7f1c2ed9d640 1 -- 172.21.15.5:0/1905355460 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c3006cf00 msgr2=0x7f1c300b5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.621+0000 7f1c2ed9d640 1 --2- 172.21.15.5:0/1905355460 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c3006cf00 0x7f1c300b5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.621+0000 7f1c2ed9d640 1 -- 172.21.15.5:0/1905355460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c300d2780 msgr2=0x7f1c301f65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.621+0000 7f1c2ed9d640 1 --2- 172.21.15.5:0/1905355460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c300d2780 0x7f1c301f65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.621+0000 7f1c2ed9d640 1 -- 172.21.15.5:0/1905355460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c301f6ae0 con 0x7f1c300d4400 2024-05-20T18:31:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.621+0000 7f1c2ed9d640 1 --2- 172.21.15.5:0/1905355460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c300d4400 0x7f1c30161590 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f1c2800cc00 tx=0x7f1c2800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.622+0000 7f1c26ffd640 1 -- 172.21.15.5:0/1905355460 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2800ce10 con 0x7f1c300d4400 2024-05-20T18:31:39.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.622+0000 7f1c26ffd640 1 -- 172.21.15.5:0/1905355460 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c28003990 con 0x7f1c300d4400 2024-05-20T18:31:39.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.622+0000 7f1c26ffd640 1 -- 172.21.15.5:0/1905355460 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c28003d60 con 0x7f1c300d4400 2024-05-20T18:31:39.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.622+0000 7f1c34ad7640 1 -- 172.21.15.5:0/1905355460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c3015c620 con 0x7f1c300d4400 2024-05-20T18:31:39.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.622+0000 7f1c34ad7640 1 -- 172.21.15.5:0/1905355460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c301f7130 con 0x7f1c300d4400 2024-05-20T18:31:39.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.623+0000 7f1c34ad7640 1 -- 172.21.15.5:0/1905355460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c300d2d70 con 0x7f1c300d4400 2024-05-20T18:31:39.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.623+0000 7f1c26ffd640 1 -- 172.21.15.5:0/1905355460 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c28003f00 con 0x7f1c300d4400 2024-05-20T18:31:39.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.624+0000 7f1c26ffd640 1 --2- 172.21.15.5:0/1905355460 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c1006be30 0x7f1c1006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:39.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.624+0000 7f1c26ffd640 1 -- 172.21.15.5:0/1905355460 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6101+0+0 (secure 0 0 0) 0x7f1c2808b400 con 0x7f1c300d4400 2024-05-20T18:31:39.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.624+0000 7f1c2e59c640 1 --2- 172.21.15.5:0/1905355460 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c1006be30 0x7f1c1006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:39.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.624+0000 7f1c2e59c640 1 --2- 172.21.15.5:0/1905355460 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c1006be30 0x7f1c1006e340 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1c20034000 tx=0x7f1c2002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:39.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.628+0000 7f1c26ffd640 1 -- 172.21.15.5:0/1905355460 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1c300d2d70 con 0x7f1c300d4400 2024-05-20T18:31:39.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.824+0000 7f1c34ad7640 1 -- 172.21.15.5:0/1905355460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1c301f6ec0 con 0x7f1c300d4400 2024-05-20T18:31:39.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.826+0000 7f1c26ffd640 1 -- 172.21.15.5:0/1905355460 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v77) ==== 80+0+0 (secure 0 0 0) 0x7f1c301f6ae0 con 0x7f1c300d4400 2024-05-20T18:31:39.826 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-05-20T18:31:39.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.828+0000 7f1c34ad7640 1 -- 172.21.15.5:0/1905355460 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c1006be30 msgr2=0x7f1c1006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:39.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.828+0000 7f1c34ad7640 1 --2- 172.21.15.5:0/1905355460 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c1006be30 0x7f1c1006e340 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1c20034000 tx=0x7f1c2002e000 comp rx=0 tx=0).stop 2024-05-20T18:31:39.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.828+0000 7f1c34ad7640 1 -- 172.21.15.5:0/1905355460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c300d4400 msgr2=0x7f1c30161590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:39.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.828+0000 7f1c34ad7640 1 --2- 172.21.15.5:0/1905355460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c300d4400 0x7f1c30161590 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f1c2800cc00 tx=0x7f1c2800a5c0 comp rx=0 tx=0).stop 2024-05-20T18:31:39.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.828+0000 7f1c34ad7640 1 -- 172.21.15.5:0/1905355460 shutdown_connections 2024-05-20T18:31:39.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.828+0000 7f1c34ad7640 1 --2- 172.21.15.5:0/1905355460 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c1006be30 0x7f1c1006e340 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:39.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.828+0000 7f1c34ad7640 1 --2- 172.21.15.5:0/1905355460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c300d4400 0x7f1c30161590 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:39.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.828+0000 7f1c34ad7640 1 --2- 172.21.15.5:0/1905355460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c300d2780 0x7f1c301f65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:39.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.828+0000 7f1c34ad7640 1 --2- 172.21.15.5:0/1905355460 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c3006cf00 0x7f1c300b5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:39.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.828+0000 7f1c34ad7640 1 -- 172.21.15.5:0/1905355460 >> 172.21.15.5:0/1905355460 conn(0x7f1c300d4bb0 msgr2=0x7f1c300d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:39.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.828+0000 7f1c34ad7640 1 -- 172.21.15.5:0/1905355460 shutdown_connections 2024-05-20T18:31:39.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:39.828+0000 7f1c34ad7640 1 -- 172.21.15.5:0/1905355460 wait complete. 2024-05-20T18:31:39.840 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-20T18:31:39.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:31:39.841 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-20T18:31:39.884 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:31:39.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:31:39.884 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-20T18:31:39.940 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:31:40.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.094+0000 7f9c09abc640 1 Processor -- start 2024-05-20T18:31:40.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.094+0000 7f9c09abc640 1 -- start start 2024-05-20T18:31:40.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.096+0000 7f9c09abc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9c04162820 0x7f9c040b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:40.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.096+0000 7f9c09abc640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9c0406dae0 0x7f9c040b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:40.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.096+0000 7f9c09abc640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c040d0630 con 0x7f9c040b3e80 2024-05-20T18:31:40.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.096+0000 7f9c09abc640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f9c040d39c0 con 0x7f9c04162820 2024-05-20T18:31:40.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.096+0000 7f9c09abc640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f9c040d3ca0 con 0x7f9c0406dae0 2024-05-20T18:31:40.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.096+0000 7f9c037fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9c04162820 0x7f9c040b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:40.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.096+0000 7f9c02ffd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f9c040b3e80 0x7f9c0406d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56892/0 (socket says 172.21.15.5:56892) 2024-05-20T18:31:40.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.096+0000 7f9c02ffd640 1 -- 172.21.15.5:0/1288312443 learned_addr learned my addr 172.21.15.5:0/1288312443 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:40.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.096+0000 7f9c03fff640 1 --2- 172.21.15.5:0/1288312443 >> v2:172.21.15.63:3300/0 conn(0x7f9c0406dae0 0x7f9c040b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:40.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.096+0000 7f9c037fe640 1 -- 172.21.15.5:0/1288312443 >> v2:172.21.15.63:3300/0 conn(0x7f9c0406dae0 msgr2=0x7f9c040b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:40.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.096+0000 7f9c037fe640 1 --2- 172.21.15.5:0/1288312443 >> v2:172.21.15.63:3300/0 conn(0x7f9c0406dae0 0x7f9c040b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:40.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.096+0000 7f9c037fe640 1 -- 172.21.15.5:0/1288312443 >> v1:172.21.15.5:6789/0 conn(0x7f9c040b3e80 legacy=0x7f9c0406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:40.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.096+0000 7f9c037fe640 1 -- 172.21.15.5:0/1288312443 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c040d3f80 con 0x7f9c04162820 2024-05-20T18:31:40.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.097+0000 7f9c037fe640 1 --2- 172.21.15.5:0/1288312443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c04162820 0x7f9c040b3940 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f9c0408e070 tx=0x7f9bf402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3285a65c7ec4235c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:40.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.097+0000 7f9c027fc640 1 -- 172.21.15.5:0/1288312443 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4037070 con 0x7f9c04162820 2024-05-20T18:31:40.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.097+0000 7f9c027fc640 1 -- 172.21.15.5:0/1288312443 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bf4032440 con 0x7f9c04162820 2024-05-20T18:31:40.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.097+0000 7f9c027fc640 1 -- 172.21.15.5:0/1288312443 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4032810 con 0x7f9c04162820 2024-05-20T18:31:40.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.098+0000 7f9c09abc640 1 -- 172.21.15.5:0/1288312443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c04162820 msgr2=0x7f9c040b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:40.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.098+0000 7f9c09abc640 1 --2- 172.21.15.5:0/1288312443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c04162820 0x7f9c040b3940 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f9c0408e070 tx=0x7f9bf402a1a0 comp rx=0 tx=0).stop 2024-05-20T18:31:40.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.098+0000 7f9c09abc640 1 -- 172.21.15.5:0/1288312443 shutdown_connections 2024-05-20T18:31:40.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.098+0000 7f9c09abc640 1 --2- 172.21.15.5:0/1288312443 >> v2:172.21.15.63:3300/0 conn(0x7f9c0406dae0 0x7f9c040b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:40.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.098+0000 7f9c09abc640 1 --2- 172.21.15.5:0/1288312443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c04162820 0x7f9c040b3940 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:40.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.098+0000 7f9c09abc640 1 -- 172.21.15.5:0/1288312443 >> 172.21.15.5:0/1288312443 conn(0x7f9c040d4bd0 msgr2=0x7f9c040d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:40.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.098+0000 7f9c09abc640 1 -- 172.21.15.5:0/1288312443 shutdown_connections 2024-05-20T18:31:40.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.098+0000 7f9c09abc640 1 -- 172.21.15.5:0/1288312443 wait complete. 2024-05-20T18:31:40.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.100+0000 7f9c09abc640 1 Processor -- start 2024-05-20T18:31:40.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.100+0000 7f9c09abc640 1 -- start start 2024-05-20T18:31:40.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.101+0000 7f9c09abc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c040b3e80 0x7f9c040cc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:40.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.101+0000 7f9c09abc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c04162820 0x7f9c041498b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:40.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.101+0000 7f9c09abc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c0414cb10 0x7f9c0415fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:40.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.101+0000 7f9c09abc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9c040d4780 con 0x7f9c04162820 2024-05-20T18:31:40.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.101+0000 7f9c09abc640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9c040d0300 con 0x7f9c040b3e80 2024-05-20T18:31:40.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.101+0000 7f9c09abc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9c040d4900 con 0x7f9c0414cb10 2024-05-20T18:31:40.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.102+0000 7f9c02ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c04162820 0x7f9c041498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:40.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.102+0000 7f9c02ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c04162820 0x7f9c041498b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58200/0 (socket says 172.21.15.5:58200) 2024-05-20T18:31:40.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.102+0000 7f9c02ffd640 1 -- 172.21.15.5:0/149625364 learned_addr learned my addr 172.21.15.5:0/149625364 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:40.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.102+0000 7f9c037fe640 1 --2- 172.21.15.5:0/149625364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c040b3e80 0x7f9c040cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:40.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.102+0000 7f9c03fff640 1 --2- 172.21.15.5:0/149625364 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c0414cb10 0x7f9c0415fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:40.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.102+0000 7f9c02ffd640 1 -- 172.21.15.5:0/149625364 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c0414cb10 msgr2=0x7f9c0415fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:40.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.102+0000 7f9c02ffd640 1 --2- 172.21.15.5:0/149625364 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c0414cb10 0x7f9c0415fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:40.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.102+0000 7f9c02ffd640 1 -- 172.21.15.5:0/149625364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c040b3e80 msgr2=0x7f9c040cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:40.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.102+0000 7f9c02ffd640 1 --2- 172.21.15.5:0/149625364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c040b3e80 0x7f9c040cc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:40.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.102+0000 7f9c02ffd640 1 -- 172.21.15.5:0/149625364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0414a410 con 0x7f9c04162820 2024-05-20T18:31:40.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.102+0000 7f9c03fff640 1 --2- 172.21.15.5:0/149625364 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c0414cb10 0x7f9c0415fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:40.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.102+0000 7f9c037fe640 1 --2- 172.21.15.5:0/149625364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c040b3e80 0x7f9c040cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:40.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.102+0000 7f9c02ffd640 1 --2- 172.21.15.5:0/149625364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c04162820 0x7f9c041498b0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f9bf800c3c0 tx=0x7f9bf800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:40.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.102+0000 7f9bebfff640 1 -- 172.21.15.5:0/149625364 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8017070 con 0x7f9c04162820 2024-05-20T18:31:40.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.102+0000 7f9bebfff640 1 -- 172.21.15.5:0/149625364 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bf8005ce0 con 0x7f9c04162820 2024-05-20T18:31:40.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.102+0000 7f9c09abc640 1 -- 172.21.15.5:0/149625364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c0414a6f0 con 0x7f9c04162820 2024-05-20T18:31:40.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.102+0000 7f9bebfff640 1 -- 172.21.15.5:0/149625364 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf80072f0 con 0x7f9c04162820 2024-05-20T18:31:40.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.102+0000 7f9c09abc640 1 -- 172.21.15.5:0/149625364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c0414acb0 con 0x7f9c04162820 2024-05-20T18:31:40.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.103+0000 7f9c09abc640 1 -- 172.21.15.5:0/149625364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c04075b00 con 0x7f9c04162820 2024-05-20T18:31:40.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.104+0000 7f9bebfff640 1 -- 172.21.15.5:0/149625364 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9bf801a070 con 0x7f9c04162820 2024-05-20T18:31:40.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.104+0000 7f9bebfff640 1 --2- 172.21.15.5:0/149625364 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9bf006be30 0x7f9bf006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:40.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.105+0000 7f9bebfff640 1 -- 172.21.15.5:0/149625364 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6101+0+0 (secure 0 0 0) 0x7f9bf808a350 con 0x7f9c04162820 2024-05-20T18:31:40.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.105+0000 7f9c037fe640 1 --2- 172.21.15.5:0/149625364 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9bf006be30 0x7f9bf006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:40.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.105+0000 7f9c037fe640 1 --2- 172.21.15.5:0/149625364 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9bf006be30 0x7f9bf006e340 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4034000 tx=0x7f9bf402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:40.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.109+0000 7f9bebfff640 1 -- 172.21.15.5:0/149625364 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9bf8058b70 con 0x7f9c04162820 2024-05-20T18:31:40.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.331+0000 7f9c09abc640 1 -- 172.21.15.5:0/149625364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c0414a170 con 0x7f9c04162820 2024-05-20T18:31:40.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.331+0000 7f9bebfff640 1 -- 172.21.15.5:0/149625364 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5171 (secure 0 0 0) 0x7f9bf8058470 con 0x7f9c04162820 2024-05-20T18:31:40.332 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:31:40.332 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1890759603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1890759603},{"type":"v1","addr":"172.21.15.63:6834","nonce":1890759603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5825,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/700447403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":700447403},{"type":"v1","addr":"172.21.15.63:6835","nonce":700447403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3382943275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3382943275},{"type":"v1","addr":"172.21.15.5:6837","nonce":3382943275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:31:30.164179+0000","modified":"2024-05-20T18:31:33.185897+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6762},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6762":{"gid":6762,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6836/791176355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":791176355},{"type":"v1","addr":"172.21.15.5:6836","nonce":791176355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6762,"qdb_cluster":[6762]},"id":5}]} 2024-05-20T18:31:40.332 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 44 2024-05-20T18:31:40.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.333+0000 7f9c09abc640 1 -- 172.21.15.5:0/149625364 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9bf006be30 msgr2=0x7f9bf006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:40.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.333+0000 7f9c09abc640 1 --2- 172.21.15.5:0/149625364 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9bf006be30 0x7f9bf006e340 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4034000 tx=0x7f9bf402e000 comp rx=0 tx=0).stop 2024-05-20T18:31:40.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.333+0000 7f9c09abc640 1 -- 172.21.15.5:0/149625364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c04162820 msgr2=0x7f9c041498b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:40.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.333+0000 7f9c09abc640 1 --2- 172.21.15.5:0/149625364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c04162820 0x7f9c041498b0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f9bf800c3c0 tx=0x7f9bf800c880 comp rx=0 tx=0).stop 2024-05-20T18:31:40.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.334+0000 7f9c09abc640 1 -- 172.21.15.5:0/149625364 shutdown_connections 2024-05-20T18:31:40.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.334+0000 7f9c09abc640 1 --2- 172.21.15.5:0/149625364 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9bf006be30 0x7f9bf006e340 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:40.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.334+0000 7f9c09abc640 1 --2- 172.21.15.5:0/149625364 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c0414cb10 0x7f9c0415fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:40.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.334+0000 7f9c09abc640 1 --2- 172.21.15.5:0/149625364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c04162820 0x7f9c041498b0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:40.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.334+0000 7f9c09abc640 1 --2- 172.21.15.5:0/149625364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c040b3e80 0x7f9c040cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:40.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.334+0000 7f9c09abc640 1 -- 172.21.15.5:0/149625364 >> 172.21.15.5:0/149625364 conn(0x7f9c040d4bd0 msgr2=0x7f9c0415da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:40.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.334+0000 7f9c09abc640 1 -- 172.21.15.5:0/149625364 shutdown_connections 2024-05-20T18:31:40.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.334+0000 7f9c09abc640 1 -- 172.21.15.5:0/149625364 wait complete. 2024-05-20T18:31:40.345 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-05-20T18:31:40.484 INFO:teuthology.orchestra.run.smithi005.stdout:{"message":"","return_code":0} 2024-05-20T18:31:40.484 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-20T18:31:40.485 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-20T18:31:40.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.600+0000 7fc6ae4f0640 1 Processor -- start 2024-05-20T18:31:40.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.600+0000 7fc6ae4f0640 1 -- start start 2024-05-20T18:31:40.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.601+0000 7fc6ae4f0640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc6a815d530 0x7fc6a815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:40.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.601+0000 7fc6ae4f0640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6a80cd580 con 0x7fc6a8070850 2024-05-20T18:31:40.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.601+0000 7fc6ae4f0640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6a80cd860 con 0x7fc6a80b1700 2024-05-20T18:31:40.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.601+0000 7fc6ae4f0640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fc6a80ce190 con 0x7fc6a815d530 2024-05-20T18:31:40.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.601+0000 7fc6a7fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc6a815d530 0x7fc6a815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:40.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.601+0000 7fc6aca8d640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc6a8070850 0x7fc6a8070c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56900/0 (socket says 172.21.15.5:56900) 2024-05-20T18:31:40.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.601+0000 7fc6aca8d640 1 -- 172.21.15.5:0/1487323585 learned_addr learned my addr 172.21.15.5:0/1487323585 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:40.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.602+0000 7fc6a6ffd640 1 -- 172.21.15.5:0/1487323585 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2962204897 0 0) 0x7fc6a80cd580 con 0x7fc6a8070850 2024-05-20T18:31:40.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.602+0000 7fc6a6ffd640 1 -- 172.21.15.5:0/1487323585 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc690003520 con 0x7fc6a8070850 2024-05-20T18:31:40.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.602+0000 7fc6a7fff640 1 -- 172.21.15.5:0/1487323585 >> v1:172.21.15.63:6789/0 conn(0x7fc6a80b1700 legacy=0x7fc6a80b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:40.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.602+0000 7fc6a7fff640 1 -- 172.21.15.5:0/1487323585 >> v1:172.21.15.5:6789/0 conn(0x7fc6a8070850 legacy=0x7fc6a8070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:40.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.602+0000 7fc6a7fff640 1 -- 172.21.15.5:0/1487323585 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a80cdb40 con 0x7fc6a815d530 2024-05-20T18:31:40.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.603+0000 7fc6a7fff640 1 --2- 172.21.15.5:0/1487323585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6a815d530 0x7fc6a815aaa0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fc6980097a0 tx=0x7fc69802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ccaa9e5a7863e15 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:40.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.603+0000 7fc6a6ffd640 1 -- 172.21.15.5:0/1487323585 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc698037070 con 0x7fc6a815d530 2024-05-20T18:31:40.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.603+0000 7fc6a6ffd640 1 -- 172.21.15.5:0/1487323585 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc698032440 con 0x7fc6a815d530 2024-05-20T18:31:40.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.603+0000 7fc6a6ffd640 1 -- 172.21.15.5:0/1487323585 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc698032810 con 0x7fc6a815d530 2024-05-20T18:31:40.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.603+0000 7fc6ae4f0640 1 -- 172.21.15.5:0/1487323585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6a815d530 msgr2=0x7fc6a815aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:40.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.603+0000 7fc6ae4f0640 1 --2- 172.21.15.5:0/1487323585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6a815d530 0x7fc6a815aaa0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fc6980097a0 tx=0x7fc69802a230 comp rx=0 tx=0).stop 2024-05-20T18:31:40.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.603+0000 7fc6ae4f0640 1 -- 172.21.15.5:0/1487323585 shutdown_connections 2024-05-20T18:31:40.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.603+0000 7fc6ae4f0640 1 --2- 172.21.15.5:0/1487323585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6a815d530 0x7fc6a815aaa0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:40.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.603+0000 7fc6ae4f0640 1 -- 172.21.15.5:0/1487323585 >> 172.21.15.5:0/1487323585 conn(0x7fc6a80d29d0 msgr2=0x7fc6a80d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:40.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.603+0000 7fc6ae4f0640 1 -- 172.21.15.5:0/1487323585 shutdown_connections 2024-05-20T18:31:40.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.603+0000 7fc6ae4f0640 1 -- 172.21.15.5:0/1487323585 wait complete. 2024-05-20T18:31:40.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.605+0000 7fc6ae4f0640 1 Processor -- start 2024-05-20T18:31:40.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.605+0000 7fc6ae4f0640 1 -- start start 2024-05-20T18:31:40.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.607+0000 7fc6ae4f0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc6a8070850 0x7fc6a81f4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:40.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.607+0000 7fc6ae4f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6a80b1700 0x7fc6a80ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:40.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.607+0000 7fc6ae4f0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6a815d530 0x7fc6a80cad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:40.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.607+0000 7fc6ae4f0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a8071370 con 0x7fc6a80b1700 2024-05-20T18:31:40.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.607+0000 7fc6ae4f0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a80cd230 con 0x7fc6a815d530 2024-05-20T18:31:40.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.607+0000 7fc6ae4f0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc6a80714f0 con 0x7fc6a8070850 2024-05-20T18:31:40.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.607+0000 7fc6a77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6a80b1700 0x7fc6a80ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:40.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.607+0000 7fc6a77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6a80b1700 0x7fc6a80ca850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58220/0 (socket says 172.21.15.5:58220) 2024-05-20T18:31:40.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.607+0000 7fc6a77fe640 1 -- 172.21.15.5:0/2402099957 learned_addr learned my addr 172.21.15.5:0/2402099957 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:40.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.607+0000 7fc6a7fff640 1 --2- 172.21.15.5:0/2402099957 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc6a8070850 0x7fc6a81f4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:40.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.607+0000 7fc6aca8d640 1 --2- 172.21.15.5:0/2402099957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6a815d530 0x7fc6a80cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:40.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.607+0000 7fc6a77fe640 1 -- 172.21.15.5:0/2402099957 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc6a8070850 msgr2=0x7fc6a81f4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:40.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.607+0000 7fc6a77fe640 1 --2- 172.21.15.5:0/2402099957 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc6a8070850 0x7fc6a81f4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:40.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.607+0000 7fc6a77fe640 1 -- 172.21.15.5:0/2402099957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6a815d530 msgr2=0x7fc6a80cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:40.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.607+0000 7fc6a77fe640 1 --2- 172.21.15.5:0/2402099957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6a815d530 0x7fc6a80cad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:40.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.607+0000 7fc6a77fe640 1 -- 172.21.15.5:0/2402099957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a806d320 con 0x7fc6a80b1700 2024-05-20T18:31:40.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.607+0000 7fc6a7fff640 1 --2- 172.21.15.5:0/2402099957 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc6a8070850 0x7fc6a81f4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:40.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.607+0000 7fc6a77fe640 1 --2- 172.21.15.5:0/2402099957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6a80b1700 0x7fc6a80ca850 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc69c00c3c0 tx=0x7fc69c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:40.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.607+0000 7fc6a4ff9640 1 -- 172.21.15.5:0/2402099957 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c0052d0 con 0x7fc6a80b1700 2024-05-20T18:31:40.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.607+0000 7fc6a4ff9640 1 -- 172.21.15.5:0/2402099957 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc69c005490 con 0x7fc6a80b1700 2024-05-20T18:31:40.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.607+0000 7fc6ae4f0640 1 -- 172.21.15.5:0/2402099957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6a806d600 con 0x7fc6a80b1700 2024-05-20T18:31:40.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.607+0000 7fc6a4ff9640 1 -- 172.21.15.5:0/2402099957 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c007260 con 0x7fc6a80b1700 2024-05-20T18:31:40.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.608+0000 7fc6ae4f0640 1 -- 172.21.15.5:0/2402099957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6a806dbc0 con 0x7fc6a80b1700 2024-05-20T18:31:40.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.609+0000 7fc6ae4f0640 1 -- 172.21.15.5:0/2402099957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6a8075b00 con 0x7fc6a80b1700 2024-05-20T18:31:40.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.609+0000 7fc6a4ff9640 1 -- 172.21.15.5:0/2402099957 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc69c007400 con 0x7fc6a80b1700 2024-05-20T18:31:40.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.610+0000 7fc6a4ff9640 1 --2- 172.21.15.5:0/2402099957 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc69406be30 0x7fc69406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:40.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.610+0000 7fc6a7fff640 1 --2- 172.21.15.5:0/2402099957 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc69406be30 0x7fc69406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:40.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.610+0000 7fc6a4ff9640 1 -- 172.21.15.5:0/2402099957 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6101+0+0 (secure 0 0 0) 0x7fc69c08a580 con 0x7fc6a80b1700 2024-05-20T18:31:40.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.611+0000 7fc6a7fff640 1 --2- 172.21.15.5:0/2402099957 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc69406be30 0x7fc69406e340 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc698034000 tx=0x7fc69802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:40.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.615+0000 7fc6a4ff9640 1 -- 172.21.15.5:0/2402099957 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc69c058cf0 con 0x7fc6a80b1700 2024-05-20T18:31:40.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.838+0000 7fc6ae4f0640 1 -- 172.21.15.5:0/2402099957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc6a80b5570 con 0x7fc6a80b1700 2024-05-20T18:31:40.838 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-20T18:31:40.838+0000 7fd1cf53a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-20T18:31:40.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.839+0000 7fc6a4ff9640 1 -- 172.21.15.5:0/2402099957 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v45) ==== 161+0+0 (secure 0 0 0) 0x7fc69c0585f0 con 0x7fc6a80b1700 2024-05-20T18:31:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-20T18:31:40.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.843+0000 7fc6ae4f0640 1 -- 172.21.15.5:0/2402099957 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc69406be30 msgr2=0x7fc69406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:40.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.843+0000 7fc6ae4f0640 1 --2- 172.21.15.5:0/2402099957 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc69406be30 0x7fc69406e340 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc698034000 tx=0x7fc69802e000 comp rx=0 tx=0).stop 2024-05-20T18:31:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.843+0000 7fc6ae4f0640 1 -- 172.21.15.5:0/2402099957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6a80b1700 msgr2=0x7fc6a80ca850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.843+0000 7fc6ae4f0640 1 --2- 172.21.15.5:0/2402099957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6a80b1700 0x7fc6a80ca850 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc69c00c3c0 tx=0x7fc69c00c880 comp rx=0 tx=0).stop 2024-05-20T18:31:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.843+0000 7fc6ae4f0640 1 -- 172.21.15.5:0/2402099957 shutdown_connections 2024-05-20T18:31:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.843+0000 7fc6ae4f0640 1 --2- 172.21.15.5:0/2402099957 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc69406be30 0x7fc69406e340 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.843+0000 7fc6ae4f0640 1 --2- 172.21.15.5:0/2402099957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6a815d530 0x7fc6a80cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.843+0000 7fc6ae4f0640 1 --2- 172.21.15.5:0/2402099957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6a80b1700 0x7fc6a80ca850 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.843+0000 7fc6ae4f0640 1 --2- 172.21.15.5:0/2402099957 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc6a8070850 0x7fc6a81f4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.843+0000 7fc6ae4f0640 1 -- 172.21.15.5:0/2402099957 >> 172.21.15.5:0/2402099957 conn(0x7fc6a80d29d0 msgr2=0x7fc6a815d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.843+0000 7fc6ae4f0640 1 -- 172.21.15.5:0/2402099957 shutdown_connections 2024-05-20T18:31:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.843+0000 7fc6ae4f0640 1 -- 172.21.15.5:0/2402099957 wait complete. 2024-05-20T18:31:40.846 INFO:tasks.ceph.mds.c.smithi005.stderr: -4597> 2024-05-20T18:31:35.761+0000 7f1d343e7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T18:31:40.858 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-05-20T18:31:40.908 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-05-20T18:31:40.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.918+0000 7f8ec6e49f40 1 Processor -- start 2024-05-20T18:31:40.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.918+0000 7f8ec6e49f40 1 -- start start 2024-05-20T18:31:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.919+0000 7f8ec6e49f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e9bffd760 0x558e9c0cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.919+0000 7f8ec6e49f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558e9c02de70 0x558e9c042110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.919+0000 7f8ec091a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e9bffd760 0x558e9c0cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.919+0000 7f8ec091a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e9bffd760 0x558e9c0cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58266/0 (socket says 172.21.15.5:58266) 2024-05-20T18:31:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.919+0000 7f8ec091a640 1 -- 172.21.15.5:0/1952286874 learned_addr learned my addr 172.21.15.5:0/1952286874 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.919+0000 7f8ec6e49f40 1 --2- 172.21.15.5:0/1952286874 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558e9c048110 0x558e9c0d4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.919+0000 7f8ec6e49f40 1 -- 172.21.15.5:0/1952286874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558e9bfa5c80 con 0x558e9bffd760 2024-05-20T18:31:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.919+0000 7f8ec6e49f40 1 -- 172.21.15.5:0/1952286874 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558e9c02e280 con 0x558e9c048110 2024-05-20T18:31:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.919+0000 7f8ec6e49f40 1 -- 172.21.15.5:0/1952286874 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x558e9c0d0e30 con 0x558e9c02de70 2024-05-20T18:31:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.919+0000 7f8ec091a640 1 -- 172.21.15.5:0/1952286874 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558e9c02de70 msgr2=0x558e9c042110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.919+0000 7f8ec091a640 1 --2- 172.21.15.5:0/1952286874 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558e9c02de70 0x558e9c042110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.919+0000 7f8ec091a640 1 -- 172.21.15.5:0/1952286874 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558e9c048110 msgr2=0x558e9c0d4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.919+0000 7f8ec091a640 1 --2- 172.21.15.5:0/1952286874 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558e9c048110 0x558e9c0d4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.919+0000 7f8ec091a640 1 -- 172.21.15.5:0/1952286874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e9bf2acb0 con 0x558e9bffd760 2024-05-20T18:31:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.919+0000 7f8ec091a640 1 --2- 172.21.15.5:0/1952286874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e9bffd760 0x558e9c0cf510 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f8eb000b240 tx=0x7f8eb000b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae199c5d6e87b1d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.920+0000 7f8ebf117640 1 -- 172.21.15.5:0/1952286874 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0008070 con 0x558e9bffd760 2024-05-20T18:31:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.920+0000 7f8ebf117640 1 -- 172.21.15.5:0/1952286874 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eb000d500 con 0x558e9bffd760 2024-05-20T18:31:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.920+0000 7f8ebf117640 1 -- 172.21.15.5:0/1952286874 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb000d8b0 con 0x558e9bffd760 2024-05-20T18:31:40.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.920+0000 7f8ec6e49f40 1 -- 172.21.15.5:0/1952286874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e9bffd760 msgr2=0x558e9c0cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:40.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.920+0000 7f8ec6e49f40 1 --2- 172.21.15.5:0/1952286874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e9bffd760 0x558e9c0cf510 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f8eb000b240 tx=0x7f8eb000b700 comp rx=0 tx=0).stop 2024-05-20T18:31:40.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.920+0000 7f8ec6e49f40 1 -- 172.21.15.5:0/1952286874 shutdown_connections 2024-05-20T18:31:40.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.920+0000 7f8ec6e49f40 1 --2- 172.21.15.5:0/1952286874 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558e9c048110 0x558e9c0d4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:40.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.920+0000 7f8ec6e49f40 1 --2- 172.21.15.5:0/1952286874 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558e9c02de70 0x558e9c042110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:40.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.920+0000 7f8ec6e49f40 1 --2- 172.21.15.5:0/1952286874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e9bffd760 0x558e9c0cf510 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:40.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.920+0000 7f8ec6e49f40 1 -- 172.21.15.5:0/1952286874 >> 172.21.15.5:0/1952286874 conn(0x558e9bf23ac0 msgr2=0x558e9c0290b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:40.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.920+0000 7f8ec6e49f40 1 -- 172.21.15.5:0/1952286874 shutdown_connections 2024-05-20T18:31:40.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.920+0000 7f8ec6e49f40 1 -- 172.21.15.5:0/1952286874 wait complete. 2024-05-20T18:31:40.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.921+0000 7f8ec6e49f40 1 Processor -- start 2024-05-20T18:31:40.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.921+0000 7f8ec6e49f40 1 -- start start 2024-05-20T18:31:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.922+0000 7f8ec6e49f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e9bffd760 0x558e9c0cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.922+0000 7f8ec6e49f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558e9c02de70 0x558e9c0403e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.923+0000 7f8ec6e49f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558e9c048110 0x558e9c040920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.923+0000 7f8ec6e49f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558e9c0d5b20 con 0x558e9bffd760 2024-05-20T18:31:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.923+0000 7f8ec6e49f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558e9c028280 con 0x558e9c02de70 2024-05-20T18:31:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.923+0000 7f8ec6e49f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x558e9c041610 con 0x558e9c048110 2024-05-20T18:31:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.923+0000 7f8ec091a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e9bffd760 0x558e9c0cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.923+0000 7f8ec091a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e9bffd760 0x558e9c0cbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58272/0 (socket says 172.21.15.5:58272) 2024-05-20T18:31:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.923+0000 7f8ec091a640 1 -- 172.21.15.5:0/4058424159 learned_addr learned my addr 172.21.15.5:0/4058424159 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.923+0000 7f8ec111b640 1 --2- 172.21.15.5:0/4058424159 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558e9c048110 0x558e9c040920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.923+0000 7f8ec0119640 1 --2- 172.21.15.5:0/4058424159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558e9c02de70 0x558e9c0403e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.923+0000 7f8ec091a640 1 -- 172.21.15.5:0/4058424159 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558e9c048110 msgr2=0x558e9c040920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.923+0000 7f8ec091a640 1 --2- 172.21.15.5:0/4058424159 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558e9c048110 0x558e9c040920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.923+0000 7f8ec091a640 1 -- 172.21.15.5:0/4058424159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558e9c02de70 msgr2=0x558e9c0403e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.923+0000 7f8ec091a640 1 --2- 172.21.15.5:0/4058424159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558e9c02de70 0x558e9c0403e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.923+0000 7f8ec091a640 1 -- 172.21.15.5:0/4058424159 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e9c045ba0 con 0x558e9bffd760 2024-05-20T18:31:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.923+0000 7f8ec091a640 1 --2- 172.21.15.5:0/4058424159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e9bffd760 0x558e9c0cbdd0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f8eb000bc00 tx=0x7f8eb0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.923+0000 7f8ebe115640 1 -- 172.21.15.5:0/4058424159 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0020020 con 0x558e9bffd760 2024-05-20T18:31:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.923+0000 7f8ebe115640 1 -- 172.21.15.5:0/4058424159 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eb001c870 con 0x558e9bffd760 2024-05-20T18:31:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.923+0000 7f8ebe115640 1 -- 172.21.15.5:0/4058424159 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb001cca0 con 0x558e9bffd760 2024-05-20T18:31:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.924+0000 7f8ec6e49f40 1 -- 172.21.15.5:0/4058424159 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558e9c047ee0 con 0x558e9bffd760 2024-05-20T18:31:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.924+0000 7f8ec6e49f40 1 -- 172.21.15.5:0/4058424159 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e9c160d50 con 0x558e9bffd760 2024-05-20T18:31:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.925+0000 7f8ebe115640 1 -- 172.21.15.5:0/4058424159 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8eb0023020 con 0x558e9bffd760 2024-05-20T18:31:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.926+0000 7f8ebe115640 1 --2- 172.21.15.5:0/4058424159 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8eac06a810 0x7f8eac06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:40.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.926+0000 7f8ebe115640 1 -- 172.21.15.5:0/4058424159 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f8eb008bb40 con 0x558e9bffd760 2024-05-20T18:31:40.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.926+0000 7f8ec0119640 1 --2- 172.21.15.5:0/4058424159 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8eac06a810 0x7f8eac06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:40.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.926+0000 7f8ec6e49f40 1 --2- 172.21.15.5:0/4058424159 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x558e9c0d14a0 0x558e9c160600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:40.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.926+0000 7f8ec6e49f40 1 -- 172.21.15.5:0/4058424159 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78) -- 0x558e9c178a10 con 0x558e9c0d14a0 2024-05-20T18:31:40.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.926+0000 7f8ec111b640 1 --2- 172.21.15.5:0/4058424159 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x558e9c0d14a0 0x558e9c160600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:40.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.926+0000 7f8ec0119640 1 --2- 172.21.15.5:0/4058424159 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8eac06a810 0x7f8eac06cd20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4057000 tx=0x7f8eb4051000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:40.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.927+0000 7f8ec111b640 1 --2- 172.21.15.5:0/4058424159 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x558e9c0d14a0 0x558e9c160600 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:40.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.928+0000 7f8ec111b640 1 -- 172.21.15.5:0/4058424159 <== osd.0 v2:172.21.15.5:6808/688093005 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v78'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8eb800c5d0 con 0x558e9c0d14a0 2024-05-20T18:31:40.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.928+0000 7f8ec6e49f40 1 -- 172.21.15.5:0/4058424159 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x558e9c0d14a0 msgr2=0x558e9c160600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:40.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.928+0000 7f8ec6e49f40 1 --2- 172.21.15.5:0/4058424159 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x558e9c0d14a0 0x558e9c160600 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:40.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.928+0000 7f8ec6e49f40 1 -- 172.21.15.5:0/4058424159 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8eac06a810 msgr2=0x7f8eac06cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:40.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.928+0000 7f8ec6e49f40 1 --2- 172.21.15.5:0/4058424159 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8eac06a810 0x7f8eac06cd20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4057000 tx=0x7f8eb4051000 comp rx=0 tx=0).stop 2024-05-20T18:31:40.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.928+0000 7f8ec6e49f40 1 -- 172.21.15.5:0/4058424159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e9bffd760 msgr2=0x558e9c0cbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:40.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.928+0000 7f8ec6e49f40 1 --2- 172.21.15.5:0/4058424159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e9bffd760 0x558e9c0cbdd0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f8eb000bc00 tx=0x7f8eb0000ed0 comp rx=0 tx=0).stop 2024-05-20T18:31:40.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.928+0000 7f8ec111b640 1 -- 172.21.15.5:0/4058424159 reap_dead start 2024-05-20T18:31:40.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.928+0000 7f8ec6e49f40 1 -- 172.21.15.5:0/4058424159 shutdown_connections 2024-05-20T18:31:40.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.928+0000 7f8ec6e49f40 1 -- 172.21.15.5:0/4058424159 >> 172.21.15.5:0/4058424159 conn(0x558e9bf23ac0 msgr2=0x558e9c029a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:40.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.928+0000 7f8ec6e49f40 1 -- 172.21.15.5:0/4058424159 shutdown_connections 2024-05-20T18:31:40.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:40.928+0000 7f8ec6e49f40 1 -- 172.21.15.5:0/4058424159 wait complete. 2024-05-20T18:31:40.931 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-20T18:31:41.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.048+0000 7efc88008640 1 Processor -- start 2024-05-20T18:31:41.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.049+0000 7efc88008640 1 -- start start 2024-05-20T18:31:41.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.050+0000 7efc88008640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efc8015f220 0x7efc800d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:41.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.050+0000 7efc88008640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc800cda20 con 0x7efc800cfae0 2024-05-20T18:31:41.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.050+0000 7efc88008640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc800cdd00 con 0x7efc800d0e00 2024-05-20T18:31:41.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.050+0000 7efc88008640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7efc800cdfe0 con 0x7efc8015f220 2024-05-20T18:31:41.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.050+0000 7efc85da4640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efc8015f220 0x7efc800d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:41.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.050+0000 7efc865a5640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7efc800cfae0 0x7efc800ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56924/0 (socket says 172.21.15.5:56924) 2024-05-20T18:31:41.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.050+0000 7efc865a5640 1 -- 172.21.15.5:0/162192716 learned_addr learned my addr 172.21.15.5:0/162192716 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:41.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.050+0000 7efc85da4640 1 -- 172.21.15.5:0/162192716 >> v1:172.21.15.63:6789/0 conn(0x7efc800d0e00 legacy=0x7efc800cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:41.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.051+0000 7efc85da4640 1 -- 172.21.15.5:0/162192716 >> v1:172.21.15.5:6789/0 conn(0x7efc800cfae0 legacy=0x7efc800ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:41.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.051+0000 7efc85da4640 1 -- 172.21.15.5:0/162192716 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc800ce2c0 con 0x7efc8015f220 2024-05-20T18:31:41.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.051+0000 7efc85da4640 1 --2- 172.21.15.5:0/162192716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc8015f220 0x7efc800d08c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7efc7c0097a0 tx=0x7efc7c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=233b1fc9705bb289 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:41.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.051+0000 7efc84da2640 1 -- 172.21.15.5:0/162192716 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c037070 con 0x7efc8015f220 2024-05-20T18:31:41.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.051+0000 7efc84da2640 1 -- 172.21.15.5:0/162192716 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc7c032440 con 0x7efc8015f220 2024-05-20T18:31:41.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.051+0000 7efc84da2640 1 -- 172.21.15.5:0/162192716 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c032810 con 0x7efc8015f220 2024-05-20T18:31:41.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.052+0000 7efc88008640 1 -- 172.21.15.5:0/162192716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc8015f220 msgr2=0x7efc800d08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:41.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.052+0000 7efc88008640 1 --2- 172.21.15.5:0/162192716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc8015f220 0x7efc800d08c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7efc7c0097a0 tx=0x7efc7c02a1a0 comp rx=0 tx=0).stop 2024-05-20T18:31:41.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.052+0000 7efc88008640 1 -- 172.21.15.5:0/162192716 shutdown_connections 2024-05-20T18:31:41.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.052+0000 7efc88008640 1 --2- 172.21.15.5:0/162192716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc8015f220 0x7efc800d08c0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:41.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.052+0000 7efc88008640 1 -- 172.21.15.5:0/162192716 >> 172.21.15.5:0/162192716 conn(0x7efc800b44d0 msgr2=0x7efc800b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:41.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.052+0000 7efc88008640 1 -- 172.21.15.5:0/162192716 shutdown_connections 2024-05-20T18:31:41.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.052+0000 7efc88008640 1 -- 172.21.15.5:0/162192716 wait complete. 2024-05-20T18:31:41.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.054+0000 7efc88008640 1 Processor -- start 2024-05-20T18:31:41.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.054+0000 7efc88008640 1 -- start start 2024-05-20T18:31:41.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.055+0000 7efc88008640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc800d0e00 0x7efc800b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:41.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.055+0000 7efc88008640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efc8015f220 0x7efc80160220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:41.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.055+0000 7efc88008640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc8014d0d0 0x7efc8014c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:41.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.055+0000 7efc88008640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efc8015f7b0 con 0x7efc8014d0d0 2024-05-20T18:31:41.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.055+0000 7efc88008640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efc8015f630 con 0x7efc800d0e00 2024-05-20T18:31:41.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.055+0000 7efc88008640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efc800ceef0 con 0x7efc8015f220 2024-05-20T18:31:41.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.055+0000 7efc865a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc8014d0d0 0x7efc8014c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:41.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.055+0000 7efc865a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc8014d0d0 0x7efc8014c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58300/0 (socket says 172.21.15.5:58300) 2024-05-20T18:31:41.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.055+0000 7efc865a5640 1 -- 172.21.15.5:0/1878649407 learned_addr learned my addr 172.21.15.5:0/1878649407 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:41.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.056+0000 7efc85da4640 1 --2- 172.21.15.5:0/1878649407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc800d0e00 0x7efc800b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:41.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.056+0000 7efc855a3640 1 --2- 172.21.15.5:0/1878649407 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efc8015f220 0x7efc80160220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:41.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.056+0000 7efc865a5640 1 -- 172.21.15.5:0/1878649407 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efc8015f220 msgr2=0x7efc80160220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:41.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.056+0000 7efc865a5640 1 --2- 172.21.15.5:0/1878649407 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efc8015f220 0x7efc80160220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:41.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.056+0000 7efc865a5640 1 -- 172.21.15.5:0/1878649407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc800d0e00 msgr2=0x7efc800b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:41.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.056+0000 7efc865a5640 1 --2- 172.21.15.5:0/1878649407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc800d0e00 0x7efc800b55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:41.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.056+0000 7efc865a5640 1 -- 172.21.15.5:0/1878649407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc800cabd0 con 0x7efc8014d0d0 2024-05-20T18:31:41.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.056+0000 7efc85da4640 1 --2- 172.21.15.5:0/1878649407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc800d0e00 0x7efc800b55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:41.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.056+0000 7efc855a3640 1 --2- 172.21.15.5:0/1878649407 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efc8015f220 0x7efc80160220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:41.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.056+0000 7efc865a5640 1 --2- 172.21.15.5:0/1878649407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc8014d0d0 0x7efc8014c9b0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7efc7800ca80 tx=0x7efc7800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:41.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.056+0000 7efc767fc640 1 -- 172.21.15.5:0/1878649407 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc78017070 con 0x7efc8014d0d0 2024-05-20T18:31:41.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.056+0000 7efc767fc640 1 -- 172.21.15.5:0/1878649407 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc7800ac30 con 0x7efc8014d0d0 2024-05-20T18:31:41.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.056+0000 7efc88008640 1 -- 172.21.15.5:0/1878649407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc800caeb0 con 0x7efc8014d0d0 2024-05-20T18:31:41.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.056+0000 7efc767fc640 1 -- 172.21.15.5:0/1878649407 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc78012600 con 0x7efc8014d0d0 2024-05-20T18:31:41.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.056+0000 7efc88008640 1 -- 172.21.15.5:0/1878649407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc800cb470 con 0x7efc8014d0d0 2024-05-20T18:31:41.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.057+0000 7efc88008640 1 -- 172.21.15.5:0/1878649407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc80075b00 con 0x7efc8014d0d0 2024-05-20T18:31:41.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.058+0000 7efc767fc640 1 -- 172.21.15.5:0/1878649407 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efc7801a070 con 0x7efc8014d0d0 2024-05-20T18:31:41.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.059+0000 7efc767fc640 1 --2- 172.21.15.5:0/1878649407 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efc6406be30 0x7efc6406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:41.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.059+0000 7efc767fc640 1 -- 172.21.15.5:0/1878649407 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7efc7808ac60 con 0x7efc8014d0d0 2024-05-20T18:31:41.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.059+0000 7efc85da4640 1 --2- 172.21.15.5:0/1878649407 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efc6406be30 0x7efc6406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:41.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.059+0000 7efc85da4640 1 --2- 172.21.15.5:0/1878649407 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efc6406be30 0x7efc6406e340 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7efc7c034000 tx=0x7efc7c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:41.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.063+0000 7efc767fc640 1 -- 172.21.15.5:0/1878649407 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7efc7800ae20 con 0x7efc8014d0d0 2024-05-20T18:31:41.269 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:31:41.271 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:31:41.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.284+0000 7efc88008640 1 -- 172.21.15.5:0/1878649407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7efc80162c60 con 0x7efc8014d0d0 2024-05-20T18:31:41.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.841+0000 7efc767fc640 1 -- 172.21.15.5:0/1878649407 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v46) ==== 162+0+0 (secure 0 0 0) 0x7efc78058de0 con 0x7efc8014d0d0 2024-05-20T18:31:41.841 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-20T18:31:41.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.844+0000 7efc88008640 1 -- 172.21.15.5:0/1878649407 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efc6406be30 msgr2=0x7efc6406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:41.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.844+0000 7efc88008640 1 --2- 172.21.15.5:0/1878649407 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efc6406be30 0x7efc6406e340 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7efc7c034000 tx=0x7efc7c02e000 comp rx=0 tx=0).stop 2024-05-20T18:31:41.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.844+0000 7efc88008640 1 -- 172.21.15.5:0/1878649407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc8014d0d0 msgr2=0x7efc8014c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:41.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.844+0000 7efc88008640 1 --2- 172.21.15.5:0/1878649407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc8014d0d0 0x7efc8014c9b0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7efc7800ca80 tx=0x7efc7800cf40 comp rx=0 tx=0).stop 2024-05-20T18:31:41.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.844+0000 7efc88008640 1 -- 172.21.15.5:0/1878649407 shutdown_connections 2024-05-20T18:31:41.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.844+0000 7efc88008640 1 --2- 172.21.15.5:0/1878649407 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efc6406be30 0x7efc6406e340 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:41.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.844+0000 7efc88008640 1 --2- 172.21.15.5:0/1878649407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc8014d0d0 0x7efc8014c9b0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:41.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.844+0000 7efc88008640 1 --2- 172.21.15.5:0/1878649407 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efc8015f220 0x7efc80160220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:41.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.845+0000 7efc88008640 1 --2- 172.21.15.5:0/1878649407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc800d0e00 0x7efc800b55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:41.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.845+0000 7efc88008640 1 -- 172.21.15.5:0/1878649407 >> 172.21.15.5:0/1878649407 conn(0x7efc800b44d0 msgr2=0x7efc800cf3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:41.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.845+0000 7efc88008640 1 -- 172.21.15.5:0/1878649407 shutdown_connections 2024-05-20T18:31:41.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.845+0000 7efc88008640 1 -- 172.21.15.5:0/1878649407 wait complete. 2024-05-20T18:31:41.862 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:31:41.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.981+0000 7f7d85b16640 1 Processor -- start 2024-05-20T18:31:41.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.981+0000 7f7d85b16640 1 -- start start 2024-05-20T18:31:41.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.982+0000 7f7d85b16640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d80163a10 0x7f7d800d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:41.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.982+0000 7f7d85b16640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7d800b4010 0x7f7d800cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:41.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.982+0000 7f7d85b16640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d800d0370 con 0x7f7d800d14b0 2024-05-20T18:31:41.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.982+0000 7f7d85b16640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f7d800d3ca0 con 0x7f7d800b4010 2024-05-20T18:31:41.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.982+0000 7f7d85b16640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7d800d0650 con 0x7f7d80163a10 2024-05-20T18:31:41.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.982+0000 7f7d7effd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7d800b4010 0x7f7d800cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:41.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.982+0000 7f7d7effd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7d800b4010 0x7f7d800cfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58302/0 (socket says 172.21.15.5:58302) 2024-05-20T18:31:41.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.982+0000 7f7d7effd640 1 -- 172.21.15.5:0/2733098574 learned_addr learned my addr 172.21.15.5:0/2733098574 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:41.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.983+0000 7f7d7f7fe640 1 --2- 172.21.15.5:0/2733098574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d80163a10 0x7f7d800d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:41.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.983+0000 7f7d7effd640 1 -- 172.21.15.5:0/2733098574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d80163a10 msgr2=0x7f7d800d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:41.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.983+0000 7f7d7effd640 1 --2- 172.21.15.5:0/2733098574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d80163a10 0x7f7d800d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:41.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.983+0000 7f7d7effd640 1 -- 172.21.15.5:0/2733098574 >> v1:172.21.15.63:6789/0 conn(0x7f7d800d14b0 legacy=0x7f7d800d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:41.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.983+0000 7f7d7effd640 1 -- 172.21.15.5:0/2733098574 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d800d3650 con 0x7f7d800b4010 2024-05-20T18:31:41.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.983+0000 7f7d7f7fe640 1 --2- 172.21.15.5:0/2733098574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d80163a10 0x7f7d800d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:41.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.984+0000 7f7d7effd640 1 --2- 172.21.15.5:0/2733098574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d800b4010 0x7f7d800cfb00 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f7d74009e50 tx=0x7f7d7402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce90bcc1c6713a74 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:41.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.984+0000 7f7d7e7fc640 1 -- 172.21.15.5:0/2733098574 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d74036070 con 0x7f7d800b4010 2024-05-20T18:31:41.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.984+0000 7f7d7e7fc640 1 -- 172.21.15.5:0/2733098574 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d74031600 con 0x7f7d800b4010 2024-05-20T18:31:41.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.984+0000 7f7d7e7fc640 1 -- 172.21.15.5:0/2733098574 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d740319d0 con 0x7f7d800b4010 2024-05-20T18:31:41.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.984+0000 7f7d85b16640 1 -- 172.21.15.5:0/2733098574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d800b4010 msgr2=0x7f7d800cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:41.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.984+0000 7f7d85b16640 1 --2- 172.21.15.5:0/2733098574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d800b4010 0x7f7d800cfb00 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f7d74009e50 tx=0x7f7d7402a710 comp rx=0 tx=0).stop 2024-05-20T18:31:41.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.984+0000 7f7d85b16640 1 -- 172.21.15.5:0/2733098574 shutdown_connections 2024-05-20T18:31:41.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.984+0000 7f7d85b16640 1 --2- 172.21.15.5:0/2733098574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d800b4010 0x7f7d800cfb00 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:41.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.984+0000 7f7d85b16640 1 --2- 172.21.15.5:0/2733098574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d80163a10 0x7f7d800d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:41.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.984+0000 7f7d85b16640 1 -- 172.21.15.5:0/2733098574 >> 172.21.15.5:0/2733098574 conn(0x7f7d8006d240 msgr2=0x7f7d8006d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:41.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.985+0000 7f7d85b16640 1 -- 172.21.15.5:0/2733098574 shutdown_connections 2024-05-20T18:31:41.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.985+0000 7f7d85b16640 1 -- 172.21.15.5:0/2733098574 wait complete. 2024-05-20T18:31:41.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.986+0000 7f7d85b16640 1 Processor -- start 2024-05-20T18:31:41.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.986+0000 7f7d85b16640 1 -- start start 2024-05-20T18:31:41.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.988+0000 7f7d85b16640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d800b4010 0x7f7d800ce480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:41.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.988+0000 7f7d85b16640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d800d14b0 0x7f7d80162240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:41.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.988+0000 7f7d85b16640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d80163a10 0x7f7d80159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:41.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.988+0000 7f7d85b16640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7d800b8210 con 0x7f7d800d14b0 2024-05-20T18:31:41.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.988+0000 7f7d85b16640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7d800d0040 con 0x7f7d80163a10 2024-05-20T18:31:41.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.988+0000 7f7d85b16640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7d800b8390 con 0x7f7d800b4010 2024-05-20T18:31:41.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.988+0000 7f7d7effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d800d14b0 0x7f7d80162240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:41.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.988+0000 7f7d7effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d800d14b0 0x7f7d80162240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58318/0 (socket says 172.21.15.5:58318) 2024-05-20T18:31:41.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.988+0000 7f7d7effd640 1 -- 172.21.15.5:0/1236165850 learned_addr learned my addr 172.21.15.5:0/1236165850 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:41.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.988+0000 7f7d7f7fe640 1 --2- 172.21.15.5:0/1236165850 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d800b4010 0x7f7d800ce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:41.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.988+0000 7f7d7ffff640 1 --2- 172.21.15.5:0/1236165850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d80163a10 0x7f7d80159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:41.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.988+0000 7f7d7effd640 1 -- 172.21.15.5:0/1236165850 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d800b4010 msgr2=0x7f7d800ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:41.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.988+0000 7f7d7effd640 1 --2- 172.21.15.5:0/1236165850 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d800b4010 0x7f7d800ce480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:41.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.988+0000 7f7d7effd640 1 -- 172.21.15.5:0/1236165850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d80163a10 msgr2=0x7f7d80159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:41.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.988+0000 7f7d7effd640 1 --2- 172.21.15.5:0/1236165850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d80163a10 0x7f7d80159bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:41.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.988+0000 7f7d7effd640 1 -- 172.21.15.5:0/1236165850 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d801607f0 con 0x7f7d800d14b0 2024-05-20T18:31:41.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.988+0000 7f7d7ffff640 1 --2- 172.21.15.5:0/1236165850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d80163a10 0x7f7d80159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:41.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.988+0000 7f7d7effd640 1 --2- 172.21.15.5:0/1236165850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d800d14b0 0x7f7d80162240 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f7d74034040 tx=0x7f7d74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:41.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.989+0000 7f7d67fff640 1 -- 172.21.15.5:0/1236165850 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7403f020 con 0x7f7d800d14b0 2024-05-20T18:31:41.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.989+0000 7f7d67fff640 1 -- 172.21.15.5:0/1236165850 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d74037040 con 0x7f7d800d14b0 2024-05-20T18:31:41.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.989+0000 7f7d67fff640 1 -- 172.21.15.5:0/1236165850 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d74036070 con 0x7f7d800d14b0 2024-05-20T18:31:41.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.989+0000 7f7d85b16640 1 -- 172.21.15.5:0/1236165850 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d80160ad0 con 0x7f7d800d14b0 2024-05-20T18:31:41.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.989+0000 7f7d85b16640 1 -- 172.21.15.5:0/1236165850 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d80161090 con 0x7f7d800d14b0 2024-05-20T18:31:41.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.990+0000 7f7d85b16640 1 -- 172.21.15.5:0/1236165850 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d80075b00 con 0x7f7d800d14b0 2024-05-20T18:31:41.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.990+0000 7f7d67fff640 1 -- 172.21.15.5:0/1236165850 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7d74042020 con 0x7f7d800d14b0 2024-05-20T18:31:41.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.991+0000 7f7d67fff640 1 --2- 172.21.15.5:0/1236165850 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7d6c06be30 0x7f7d6c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:41.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.991+0000 7f7d67fff640 1 -- 172.21.15.5:0/1236165850 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f7d740ab6e0 con 0x7f7d800d14b0 2024-05-20T18:31:41.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.991+0000 7f7d7f7fe640 1 --2- 172.21.15.5:0/1236165850 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7d6c06be30 0x7f7d6c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:41.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.991+0000 7f7d7f7fe640 1 --2- 172.21.15.5:0/1236165850 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7d6c06be30 0x7f7d6c06e340 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7d70006fd0 tx=0x7f7d70008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:41.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:41.995+0000 7f7d67fff640 1 -- 172.21.15.5:0/1236165850 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7d74049030 con 0x7f7d800d14b0 2024-05-20T18:31:42.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:42.218+0000 7f7d85b16640 1 -- 172.21.15.5:0/1236165850 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d800d3050 con 0x7f7d800d14b0 2024-05-20T18:31:42.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:42.219+0000 7f7d67fff640 1 -- 172.21.15.5:0/1236165850 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5168 (secure 0 0 0) 0x7f7d74079d80 con 0x7f7d800d14b0 2024-05-20T18:31:42.219 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:31:42.220 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1890759603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1890759603},{"type":"v1","addr":"172.21.15.63:6834","nonce":1890759603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5825,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/700447403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":700447403},{"type":"v1","addr":"172.21.15.63:6835","nonce":700447403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3382943275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3382943275},{"type":"v1","addr":"172.21.15.5:6837","nonce":3382943275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:31:30.164179+0000","modified":"2024-05-20T18:31:41.845620+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6972":{"gid":6972,"name":"c","rank":0,"incarnation":47,"state":"up:replay","state_seq":1,"addr":"172.21.15.5:6836/3866822752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":3866822752},{"type":"v1","addr":"172.21.15.5:6836","nonce":3866822752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-05-20T18:31:42.220 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 47 2024-05-20T18:31:42.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:42.221+0000 7f7d85b16640 1 -- 172.21.15.5:0/1236165850 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7d6c06be30 msgr2=0x7f7d6c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:42.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:42.221+0000 7f7d85b16640 1 --2- 172.21.15.5:0/1236165850 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7d6c06be30 0x7f7d6c06e340 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7d70006fd0 tx=0x7f7d70008040 comp rx=0 tx=0).stop 2024-05-20T18:31:42.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:42.221+0000 7f7d85b16640 1 -- 172.21.15.5:0/1236165850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d800d14b0 msgr2=0x7f7d80162240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:42.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:42.221+0000 7f7d85b16640 1 --2- 172.21.15.5:0/1236165850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d800d14b0 0x7f7d80162240 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f7d74034040 tx=0x7f7d74004a40 comp rx=0 tx=0).stop 2024-05-20T18:31:42.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:42.221+0000 7f7d85b16640 1 -- 172.21.15.5:0/1236165850 shutdown_connections 2024-05-20T18:31:42.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:42.221+0000 7f7d85b16640 1 --2- 172.21.15.5:0/1236165850 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7d6c06be30 0x7f7d6c06e340 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:42.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:42.221+0000 7f7d85b16640 1 --2- 172.21.15.5:0/1236165850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d80163a10 0x7f7d80159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:42.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:42.221+0000 7f7d85b16640 1 --2- 172.21.15.5:0/1236165850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d800d14b0 0x7f7d80162240 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:42.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:42.221+0000 7f7d85b16640 1 --2- 172.21.15.5:0/1236165850 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d800b4010 0x7f7d800ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:42.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:42.221+0000 7f7d85b16640 1 -- 172.21.15.5:0/1236165850 >> 172.21.15.5:0/1236165850 conn(0x7f7d8006d240 msgr2=0x7f7d800b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:42.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:42.221+0000 7f7d85b16640 1 -- 172.21.15.5:0/1236165850 shutdown_connections 2024-05-20T18:31:42.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:42.221+0000 7f7d85b16640 1 -- 172.21.15.5:0/1236165850 wait complete. 2024-05-20T18:31:42.635 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T18:31:30.164179+0000', 'modified': '2024-05-20T18:31:41.845620+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 78, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6972}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6972': {'gid': 6972, 'name': 'c', 'rank': 0, 'incarnation': 47, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.5:6836/3866822752', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6835', 'nonce': 3866822752}, {'type': 'v1', 'addr': '172.21.15.5:6836', 'nonce': 3866822752}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-20T18:31:42.635 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6972:up:replay 2024-05-20T18:31:43.636 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:31:43.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.766+0000 7f5beff4c640 1 Processor -- start 2024-05-20T18:31:43.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.766+0000 7f5beff4c640 1 -- start start 2024-05-20T18:31:43.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.767+0000 7f5beff4c640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5be8162820 0x7f5be80b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:43.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.767+0000 7f5beff4c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80b3e80 0x7f5be806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.767+0000 7f5beff4c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5be806d910 0x7f5be80d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.767+0000 7f5beff4c640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f5be806dd20 con 0x7f5be806d910 2024-05-20T18:31:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.767+0000 7f5beff4c640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f5be80d0630 con 0x7f5be8162820 2024-05-20T18:31:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.767+0000 7f5beff4c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5be80d39c0 con 0x7f5be80b3e80 2024-05-20T18:31:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.767+0000 7f5bee4e9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5be806d910 0x7f5be80d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.767+0000 7f5bee4e9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5be806d910 0x7f5be80d27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58334/0 (socket says 172.21.15.5:58334) 2024-05-20T18:31:43.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.767+0000 7f5bee4e9640 1 -- 172.21.15.5:0/961741101 learned_addr learned my addr 172.21.15.5:0/961741101 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:43.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.768+0000 7f5bed4e7640 1 --2- 172.21.15.5:0/961741101 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80b3e80 0x7f5be806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:43.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.768+0000 7f5bedce8640 1 --2- 172.21.15.5:0/961741101 >> v2:172.21.15.63:3300/0 conn(0x7f5be8162820 0x7f5be80b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:43.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.768+0000 7f5bee4e9640 1 -- 172.21.15.5:0/961741101 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80b3e80 msgr2=0x7f5be806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:43.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.768+0000 7f5bee4e9640 1 --2- 172.21.15.5:0/961741101 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80b3e80 0x7f5be806d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:43.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.768+0000 7f5bee4e9640 1 -- 172.21.15.5:0/961741101 >> v2:172.21.15.63:3300/0 conn(0x7f5be8162820 msgr2=0x7f5be80b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:43.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.768+0000 7f5bee4e9640 1 --2- 172.21.15.5:0/961741101 >> v2:172.21.15.63:3300/0 conn(0x7f5be8162820 0x7f5be80b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:43.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.768+0000 7f5bee4e9640 1 -- 172.21.15.5:0/961741101 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be80d3ca0 con 0x7f5be806d910 2024-05-20T18:31:43.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.769+0000 7f5bee4e9640 1 --2- 172.21.15.5:0/961741101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5be806d910 0x7f5be80d27a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f5be80b7740 tx=0x7f5be002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=113da38e86838607 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:43.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.769+0000 7f5becce6640 1 -- 172.21.15.5:0/961741101 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be0036070 con 0x7f5be806d910 2024-05-20T18:31:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.769+0000 7f5becce6640 1 -- 172.21.15.5:0/961741101 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5be0031600 con 0x7f5be806d910 2024-05-20T18:31:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.769+0000 7f5becce6640 1 -- 172.21.15.5:0/961741101 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be00319d0 con 0x7f5be806d910 2024-05-20T18:31:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.769+0000 7f5beff4c640 1 -- 172.21.15.5:0/961741101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5be806d910 msgr2=0x7f5be80d27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.769+0000 7f5beff4c640 1 --2- 172.21.15.5:0/961741101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5be806d910 0x7f5be80d27a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f5be80b7740 tx=0x7f5be002a6a0 comp rx=0 tx=0).stop 2024-05-20T18:31:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.769+0000 7f5beff4c640 1 -- 172.21.15.5:0/961741101 shutdown_connections 2024-05-20T18:31:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.769+0000 7f5beff4c640 1 --2- 172.21.15.5:0/961741101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5be806d910 0x7f5be80d27a0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.769+0000 7f5beff4c640 1 --2- 172.21.15.5:0/961741101 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80b3e80 0x7f5be806d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.769+0000 7f5beff4c640 1 --2- 172.21.15.5:0/961741101 >> v2:172.21.15.63:3300/0 conn(0x7f5be8162820 0x7f5be80b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.769+0000 7f5beff4c640 1 -- 172.21.15.5:0/961741101 >> 172.21.15.5:0/961741101 conn(0x7f5be80d4bd0 msgr2=0x7f5be80d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.770+0000 7f5beff4c640 1 -- 172.21.15.5:0/961741101 shutdown_connections 2024-05-20T18:31:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.770+0000 7f5beff4c640 1 -- 172.21.15.5:0/961741101 wait complete. 2024-05-20T18:31:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.771+0000 7f5beff4c640 1 Processor -- start 2024-05-20T18:31:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.771+0000 7f5beff4c640 1 -- start start 2024-05-20T18:31:43.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.773+0000 7f5beff4c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be806d910 0x7f5be814e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:43.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.773+0000 7f5beff4c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5be80b3e80 0x7f5be814eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:43.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.773+0000 7f5beff4c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be8162820 0x7f5be80c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:43.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.773+0000 7f5beff4c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5be80d0300 con 0x7f5be80b3e80 2024-05-20T18:31:43.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.773+0000 7f5beff4c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5be80d4260 con 0x7f5be8162820 2024-05-20T18:31:43.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.773+0000 7f5beff4c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5be80d4780 con 0x7f5be806d910 2024-05-20T18:31:43.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.773+0000 7f5bed4e7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5be80b3e80 0x7f5be814eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:43.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.773+0000 7f5bed4e7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5be80b3e80 0x7f5be814eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58348/0 (socket says 172.21.15.5:58348) 2024-05-20T18:31:43.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.773+0000 7f5bed4e7640 1 -- 172.21.15.5:0/4199788815 learned_addr learned my addr 172.21.15.5:0/4199788815 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:43.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.773+0000 7f5bedce8640 1 --2- 172.21.15.5:0/4199788815 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be806d910 0x7f5be814e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:43.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.773+0000 7f5bee4e9640 1 --2- 172.21.15.5:0/4199788815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be8162820 0x7f5be80c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:43.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.773+0000 7f5bed4e7640 1 -- 172.21.15.5:0/4199788815 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be806d910 msgr2=0x7f5be814e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:43.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.773+0000 7f5bed4e7640 1 --2- 172.21.15.5:0/4199788815 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be806d910 0x7f5be814e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:43.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.773+0000 7f5bed4e7640 1 -- 172.21.15.5:0/4199788815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be8162820 msgr2=0x7f5be80c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:43.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.773+0000 7f5bed4e7640 1 --2- 172.21.15.5:0/4199788815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be8162820 0x7f5be80c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:43.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.773+0000 7f5bed4e7640 1 -- 172.21.15.5:0/4199788815 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be814f2e0 con 0x7f5be80b3e80 2024-05-20T18:31:43.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.773+0000 7f5bed4e7640 1 --2- 172.21.15.5:0/4199788815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5be80b3e80 0x7f5be814eda0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800d860 tx=0x7f5bd800dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.774+0000 7f5bde7fc640 1 -- 172.21.15.5:0/4199788815 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8018030 con 0x7f5be80b3e80 2024-05-20T18:31:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.774+0000 7f5bde7fc640 1 -- 172.21.15.5:0/4199788815 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bd80096a0 con 0x7f5be80b3e80 2024-05-20T18:31:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.774+0000 7f5bde7fc640 1 -- 172.21.15.5:0/4199788815 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8009890 con 0x7f5be80b3e80 2024-05-20T18:31:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.774+0000 7f5beff4c640 1 -- 172.21.15.5:0/4199788815 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be80c89a0 con 0x7f5be80b3e80 2024-05-20T18:31:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.774+0000 7f5beff4c640 1 -- 172.21.15.5:0/4199788815 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be80c8f60 con 0x7f5be80b3e80 2024-05-20T18:31:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.775+0000 7f5beff4c640 1 -- 172.21.15.5:0/4199788815 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5be80c8700 con 0x7f5be80b3e80 2024-05-20T18:31:43.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.775+0000 7f5bde7fc640 1 -- 172.21.15.5:0/4199788815 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5bd801b070 con 0x7f5be80b3e80 2024-05-20T18:31:43.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.776+0000 7f5bde7fc640 1 --2- 172.21.15.5:0/4199788815 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5bcc06be60 0x7f5bcc06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:43.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.776+0000 7f5bedce8640 1 --2- 172.21.15.5:0/4199788815 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5bcc06be60 0x7f5bcc06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:43.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.776+0000 7f5bde7fc640 1 -- 172.21.15.5:0/4199788815 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f5bd808b1b0 con 0x7f5be80b3e80 2024-05-20T18:31:43.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.777+0000 7f5bedce8640 1 --2- 172.21.15.5:0/4199788815 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5bcc06be60 0x7f5bcc06e370 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5be40079a0 tx=0x7f5be4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:43.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:43.781+0000 7f5bde7fc640 1 -- 172.21.15.5:0/4199788815 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5be80c8700 con 0x7f5be80b3e80 2024-05-20T18:31:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:44.003+0000 7f5beff4c640 1 -- 172.21.15.5:0/4199788815 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5be8075b00 con 0x7f5be80b3e80 2024-05-20T18:31:44.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:44.003+0000 7f5bde7fc640 1 -- 172.21.15.5:0/4199788815 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5168 (secure 0 0 0) 0x7f5bd80592a0 con 0x7f5be80b3e80 2024-05-20T18:31:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:31:44.004 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1890759603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1890759603},{"type":"v1","addr":"172.21.15.63:6834","nonce":1890759603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5825,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/700447403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":700447403},{"type":"v1","addr":"172.21.15.63:6835","nonce":700447403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3382943275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3382943275},{"type":"v1","addr":"172.21.15.5:6837","nonce":3382943275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:31:30.164179+0000","modified":"2024-05-20T18:31:42.853119+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6972":{"gid":6972,"name":"c","rank":0,"incarnation":47,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.5:6836/3866822752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":3866822752},{"type":"v1","addr":"172.21.15.5:6836","nonce":3866822752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-05-20T18:31:44.004 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 49 2024-05-20T18:31:44.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:44.005+0000 7f5beff4c640 1 -- 172.21.15.5:0/4199788815 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5bcc06be60 msgr2=0x7f5bcc06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:44.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:44.005+0000 7f5beff4c640 1 --2- 172.21.15.5:0/4199788815 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5bcc06be60 0x7f5bcc06e370 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5be40079a0 tx=0x7f5be4008040 comp rx=0 tx=0).stop 2024-05-20T18:31:44.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:44.005+0000 7f5beff4c640 1 -- 172.21.15.5:0/4199788815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5be80b3e80 msgr2=0x7f5be814eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:44.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:44.005+0000 7f5beff4c640 1 --2- 172.21.15.5:0/4199788815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5be80b3e80 0x7f5be814eda0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800d860 tx=0x7f5bd800dd20 comp rx=0 tx=0).stop 2024-05-20T18:31:44.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:44.005+0000 7f5beff4c640 1 -- 172.21.15.5:0/4199788815 shutdown_connections 2024-05-20T18:31:44.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:44.005+0000 7f5beff4c640 1 --2- 172.21.15.5:0/4199788815 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5bcc06be60 0x7f5bcc06e370 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:44.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:44.005+0000 7f5beff4c640 1 --2- 172.21.15.5:0/4199788815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be8162820 0x7f5be80c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:44.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:44.005+0000 7f5beff4c640 1 --2- 172.21.15.5:0/4199788815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5be80b3e80 0x7f5be814eda0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:44.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:44.005+0000 7f5beff4c640 1 --2- 172.21.15.5:0/4199788815 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be806d910 0x7f5be814e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:44.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:44.005+0000 7f5beff4c640 1 -- 172.21.15.5:0/4199788815 >> 172.21.15.5:0/4199788815 conn(0x7f5be80d4bd0 msgr2=0x7f5be80b8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:44.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:44.005+0000 7f5beff4c640 1 -- 172.21.15.5:0/4199788815 shutdown_connections 2024-05-20T18:31:44.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:44.005+0000 7f5beff4c640 1 -- 172.21.15.5:0/4199788815 wait complete. 2024-05-20T18:31:44.419 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T18:31:30.164179+0000', 'modified': '2024-05-20T18:31:42.853119+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 78, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6972}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6972': {'gid': 6972, 'name': 'c', 'rank': 0, 'incarnation': 47, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.5:6836/3866822752', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6835', 'nonce': 3866822752}, {'type': 'v1', 'addr': '172.21.15.5:6836', 'nonce': 3866822752}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-20T18:31:44.420 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6972:up:rejoin 2024-05-20T18:31:45.421 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:31:45.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.551+0000 7fa11e073640 1 Processor -- start 2024-05-20T18:31:45.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.551+0000 7fa11e073640 1 -- start start 2024-05-20T18:31:45.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.552+0000 7fa11e073640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa1180b47b0 0x7fa1180afad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:45.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.552+0000 7fa11e073640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa11815d060 0x7fa1180b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:45.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.552+0000 7fa11e073640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1180b5720 0x7fa11815fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:45.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.553+0000 7fa11e073640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fa1180b5b30 con 0x7fa1180b47b0 2024-05-20T18:31:45.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.553+0000 7fa11e073640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa1180b39f0 con 0x7fa11815d060 2024-05-20T18:31:45.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.553+0000 7fa11e073640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa1180b3cd0 con 0x7fa1180b5720 2024-05-20T18:31:45.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.553+0000 7fa1177fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa1180b47b0 0x7fa1180afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:45.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.553+0000 7fa1177fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa1180b47b0 0x7fa1180afad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51664/0 (socket says 172.21.15.5:51664) 2024-05-20T18:31:45.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.553+0000 7fa1177fe640 1 -- 172.21.15.5:0/2230668308 learned_addr learned my addr 172.21.15.5:0/2230668308 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:45.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.553+0000 7fa116ffd640 1 --2- 172.21.15.5:0/2230668308 >> v2:172.21.15.63:3300/0 conn(0x7fa11815d060 0x7fa1180b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:45.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.553+0000 7fa117fff640 1 --2- 172.21.15.5:0/2230668308 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1180b5720 0x7fa11815fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:45.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.553+0000 7fa1177fe640 1 -- 172.21.15.5:0/2230668308 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1180b5720 msgr2=0x7fa11815fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:45.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.553+0000 7fa1177fe640 1 --2- 172.21.15.5:0/2230668308 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1180b5720 0x7fa11815fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:45.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.553+0000 7fa1177fe640 1 -- 172.21.15.5:0/2230668308 >> v2:172.21.15.63:3300/0 conn(0x7fa11815d060 msgr2=0x7fa1180b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:45.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.553+0000 7fa1177fe640 1 --2- 172.21.15.5:0/2230668308 >> v2:172.21.15.63:3300/0 conn(0x7fa11815d060 0x7fa1180b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:45.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.553+0000 7fa1177fe640 1 -- 172.21.15.5:0/2230668308 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1180b3fb0 con 0x7fa1180b47b0 2024-05-20T18:31:45.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.554+0000 7fa1177fe640 1 --2- 172.21.15.5:0/2230668308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1180b47b0 0x7fa1180afad0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa11806b980 tx=0x7fa10802c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc037ff863829980 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:45.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.554+0000 7fa1167fc640 1 -- 172.21.15.5:0/2230668308 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa108032700 con 0x7fa1180b47b0 2024-05-20T18:31:45.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.554+0000 7fa1167fc640 1 -- 172.21.15.5:0/2230668308 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1080328a0 con 0x7fa1180b47b0 2024-05-20T18:31:45.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.554+0000 7fa1167fc640 1 -- 172.21.15.5:0/2230668308 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa108032c70 con 0x7fa1180b47b0 2024-05-20T18:31:45.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.555+0000 7fa11e073640 1 -- 172.21.15.5:0/2230668308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1180b47b0 msgr2=0x7fa1180afad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:45.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.555+0000 7fa11e073640 1 --2- 172.21.15.5:0/2230668308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1180b47b0 0x7fa1180afad0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa11806b980 tx=0x7fa10802c800 comp rx=0 tx=0).stop 2024-05-20T18:31:45.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.555+0000 7fa11e073640 1 -- 172.21.15.5:0/2230668308 shutdown_connections 2024-05-20T18:31:45.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.555+0000 7fa11e073640 1 --2- 172.21.15.5:0/2230668308 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1180b5720 0x7fa11815fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:45.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.555+0000 7fa11e073640 1 --2- 172.21.15.5:0/2230668308 >> v2:172.21.15.63:3300/0 conn(0x7fa11815d060 0x7fa1180b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:45.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.555+0000 7fa11e073640 1 --2- 172.21.15.5:0/2230668308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1180b47b0 0x7fa1180afad0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:45.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.555+0000 7fa11e073640 1 -- 172.21.15.5:0/2230668308 >> 172.21.15.5:0/2230668308 conn(0x7fa1180cd2a0 msgr2=0x7fa1180cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:45.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.555+0000 7fa11e073640 1 -- 172.21.15.5:0/2230668308 shutdown_connections 2024-05-20T18:31:45.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.555+0000 7fa11e073640 1 -- 172.21.15.5:0/2230668308 wait complete. 2024-05-20T18:31:45.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.557+0000 7fa11e073640 1 Processor -- start 2024-05-20T18:31:45.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.557+0000 7fa11e073640 1 -- start start 2024-05-20T18:31:45.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.558+0000 7fa11e073640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1180b5720 0x7fa118146110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:45.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.558+0000 7fa11e073640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa11815d060 0x7fa1181526b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:45.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.558+0000 7fa11e073640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa118151050 0x7fa118152bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:45.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.558+0000 7fa11e073640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa118159c60 con 0x7fa118151050 2024-05-20T18:31:45.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.558+0000 7fa11e073640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa1180b0010 con 0x7fa1180b5720 2024-05-20T18:31:45.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.558+0000 7fa11e073640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa118159de0 con 0x7fa11815d060 2024-05-20T18:31:45.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.558+0000 7fa117fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa118151050 0x7fa118152bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:45.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.558+0000 7fa117fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa118151050 0x7fa118152bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51676/0 (socket says 172.21.15.5:51676) 2024-05-20T18:31:45.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.558+0000 7fa117fff640 1 -- 172.21.15.5:0/2668269453 learned_addr learned my addr 172.21.15.5:0/2668269453 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:45.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.558+0000 7fa116ffd640 1 --2- 172.21.15.5:0/2668269453 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa11815d060 0x7fa1181526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:45.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.558+0000 7fa1177fe640 1 --2- 172.21.15.5:0/2668269453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1180b5720 0x7fa118146110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:45.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.559+0000 7fa117fff640 1 -- 172.21.15.5:0/2668269453 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa11815d060 msgr2=0x7fa1181526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:45.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.559+0000 7fa117fff640 1 --2- 172.21.15.5:0/2668269453 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa11815d060 0x7fa1181526b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:45.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.559+0000 7fa117fff640 1 -- 172.21.15.5:0/2668269453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1180b5720 msgr2=0x7fa118146110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:45.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.559+0000 7fa117fff640 1 --2- 172.21.15.5:0/2668269453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1180b5720 0x7fa118146110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:45.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.559+0000 7fa117fff640 1 -- 172.21.15.5:0/2668269453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa118155c50 con 0x7fa118151050 2024-05-20T18:31:45.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.559+0000 7fa117fff640 1 --2- 172.21.15.5:0/2668269453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa118151050 0x7fa118152bf0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fa110007d40 tx=0x7fa11000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:45.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.559+0000 7fa0fffff640 1 -- 172.21.15.5:0/2668269453 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110014420 con 0x7fa118151050 2024-05-20T18:31:45.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.559+0000 7fa0fffff640 1 -- 172.21.15.5:0/2668269453 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa110019070 con 0x7fa118151050 2024-05-20T18:31:45.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.559+0000 7fa11e073640 1 -- 172.21.15.5:0/2668269453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa118154620 con 0x7fa118151050 2024-05-20T18:31:45.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.559+0000 7fa0fffff640 1 -- 172.21.15.5:0/2668269453 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110014800 con 0x7fa118151050 2024-05-20T18:31:45.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.559+0000 7fa11e073640 1 -- 172.21.15.5:0/2668269453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa118154be0 con 0x7fa118151050 2024-05-20T18:31:45.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.560+0000 7fa11e073640 1 -- 172.21.15.5:0/2668269453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa118075b00 con 0x7fa118151050 2024-05-20T18:31:45.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.561+0000 7fa0fffff640 1 -- 172.21.15.5:0/2668269453 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa1100149a0 con 0x7fa118151050 2024-05-20T18:31:45.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.561+0000 7fa0fffff640 1 --2- 172.21.15.5:0/2668269453 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa10406be30 0x7fa10406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:45.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.561+0000 7fa1177fe640 1 --2- 172.21.15.5:0/2668269453 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa10406be30 0x7fa10406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:45.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.561+0000 7fa0fffff640 1 -- 172.21.15.5:0/2668269453 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7fa11008b840 con 0x7fa118151050 2024-05-20T18:31:45.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.562+0000 7fa1177fe640 1 --2- 172.21.15.5:0/2668269453 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa10406be30 0x7fa10406e340 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa108037000 tx=0x7fa108032000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:45.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.566+0000 7fa0fffff640 1 -- 172.21.15.5:0/2668269453 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa110059f80 con 0x7fa118151050 2024-05-20T18:31:45.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.787+0000 7fa11e073640 1 -- 172.21.15.5:0/2668269453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1181559e0 con 0x7fa118151050 2024-05-20T18:31:45.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.788+0000 7fa0fffff640 1 -- 172.21.15.5:0/2668269453 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5175 (secure 0 0 0) 0x7fa110059880 con 0x7fa118151050 2024-05-20T18:31:45.789 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:31:45.789 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1890759603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1890759603},{"type":"v1","addr":"172.21.15.63:6834","nonce":1890759603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5825,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/700447403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":700447403},{"type":"v1","addr":"172.21.15.63:6835","nonce":700447403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3382943275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3382943275},{"type":"v1","addr":"172.21.15.5:6837","nonce":3382943275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:31:30.164179+0000","modified":"2024-05-20T18:31:44.852381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6972":{"gid":6972,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6836/3866822752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":3866822752},{"type":"v1","addr":"172.21.15.5:6836","nonce":3866822752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6972,"qdb_cluster":[6972]},"id":5}]} 2024-05-20T18:31:45.789 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 50 2024-05-20T18:31:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.790+0000 7fa11e073640 1 -- 172.21.15.5:0/2668269453 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa10406be30 msgr2=0x7fa10406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.790+0000 7fa11e073640 1 --2- 172.21.15.5:0/2668269453 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa10406be30 0x7fa10406e340 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa108037000 tx=0x7fa108032000 comp rx=0 tx=0).stop 2024-05-20T18:31:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.790+0000 7fa11e073640 1 -- 172.21.15.5:0/2668269453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa118151050 msgr2=0x7fa118152bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.790+0000 7fa11e073640 1 --2- 172.21.15.5:0/2668269453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa118151050 0x7fa118152bf0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fa110007d40 tx=0x7fa11000e4b0 comp rx=0 tx=0).stop 2024-05-20T18:31:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.790+0000 7fa11e073640 1 -- 172.21.15.5:0/2668269453 shutdown_connections 2024-05-20T18:31:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.790+0000 7fa11e073640 1 --2- 172.21.15.5:0/2668269453 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa10406be30 0x7fa10406e340 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.790+0000 7fa11e073640 1 --2- 172.21.15.5:0/2668269453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa118151050 0x7fa118152bf0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.790+0000 7fa11e073640 1 --2- 172.21.15.5:0/2668269453 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa11815d060 0x7fa1181526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.790+0000 7fa11e073640 1 --2- 172.21.15.5:0/2668269453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1180b5720 0x7fa118146110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.790+0000 7fa11e073640 1 -- 172.21.15.5:0/2668269453 >> 172.21.15.5:0/2668269453 conn(0x7fa1180cd2a0 msgr2=0x7fa1180cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.790+0000 7fa11e073640 1 -- 172.21.15.5:0/2668269453 shutdown_connections 2024-05-20T18:31:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:45.790+0000 7fa11e073640 1 -- 172.21.15.5:0/2668269453 wait complete. 2024-05-20T18:31:46.205 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T18:31:30.164179+0000', 'modified': '2024-05-20T18:31:44.852381+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 78, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6972}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6972': {'gid': 6972, 'name': 'c', 'rank': 0, 'incarnation': 47, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.5:6836/3866822752', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6835', 'nonce': 3866822752}, {'type': 'v1', 'addr': '172.21.15.5:6836', 'nonce': 3866822752}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6972, 'qdb_cluster': [6972]} 2024-05-20T18:31:46.205 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-20T18:31:46.206 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-20T18:31:46.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.325+0000 7f3fa4a2b640 1 Processor -- start 2024-05-20T18:31:46.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.325+0000 7f3fa4a2b640 1 -- start start 2024-05-20T18:31:46.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.327+0000 7f3fa4a2b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3fa006d3d0 0x7f3fa006d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:46.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.327+0000 7f3fa4a2b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3fa00b8190 0x7f3fa00b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:46.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.327+0000 7f3fa4a2b640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fa006dd20 con 0x7f3fa0162820 2024-05-20T18:31:46.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.327+0000 7f3fa4a2b640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f3fa00d0630 con 0x7f3fa006d3d0 2024-05-20T18:31:46.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.327+0000 7f3fa4a2b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3fa00d39c0 con 0x7f3fa00b8190 2024-05-20T18:31:46.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.327+0000 7f3f9ed9d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3fa00b8190 0x7f3fa00b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:46.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.327+0000 7f3f9dd9b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3fa006d3d0 0x7f3fa006d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:46.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.327+0000 7f3f9dd9b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3fa006d3d0 0x7f3fa006d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:34332/0 (socket says 172.21.15.5:34332) 2024-05-20T18:31:46.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.327+0000 7f3f9ed9d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3fa00b8190 0x7f3fa00b85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:56760/0 (socket says 172.21.15.5:56760) 2024-05-20T18:31:46.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.327+0000 7f3f9dd9b640 1 -- 172.21.15.5:0/3029127907 learned_addr learned my addr 172.21.15.5:0/3029127907 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:46.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.328+0000 7f3f9dd9b640 1 -- 172.21.15.5:0/3029127907 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3fa00b8190 msgr2=0x7f3fa00b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:46.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.328+0000 7f3f9dd9b640 1 --2- 172.21.15.5:0/3029127907 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3fa00b8190 0x7f3fa00b85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.328+0000 7f3f9dd9b640 1 -- 172.21.15.5:0/3029127907 >> v1:172.21.15.63:6789/0 conn(0x7f3fa0162820 legacy=0x7f3fa00b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:46.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.328+0000 7f3f9dd9b640 1 -- 172.21.15.5:0/3029127907 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa00d3ca0 con 0x7f3fa006d3d0 2024-05-20T18:31:46.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.328+0000 7f3f9ed9d640 1 --2- 172.21.15.5:0/3029127907 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3fa00b8190 0x7f3fa00b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:31:46.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.328+0000 7f3f9dd9b640 1 --2- 172.21.15.5:0/3029127907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3fa006d3d0 0x7f3fa006d7e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009870 tx=0x7f3f9402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4782441d5483d754 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:46.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.329+0000 7f3f9d59a640 1 -- 172.21.15.5:0/3029127907 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94037070 con 0x7f3fa006d3d0 2024-05-20T18:31:46.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.329+0000 7f3f9d59a640 1 -- 172.21.15.5:0/3029127907 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f94032440 con 0x7f3fa006d3d0 2024-05-20T18:31:46.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.329+0000 7f3f9d59a640 1 -- 172.21.15.5:0/3029127907 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94032810 con 0x7f3fa006d3d0 2024-05-20T18:31:46.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.329+0000 7f3fa4a2b640 1 -- 172.21.15.5:0/3029127907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3fa006d3d0 msgr2=0x7f3fa006d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:46.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.329+0000 7f3fa4a2b640 1 --2- 172.21.15.5:0/3029127907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3fa006d3d0 0x7f3fa006d7e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009870 tx=0x7f3f9402a270 comp rx=0 tx=0).stop 2024-05-20T18:31:46.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.329+0000 7f3fa4a2b640 1 -- 172.21.15.5:0/3029127907 shutdown_connections 2024-05-20T18:31:46.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.329+0000 7f3fa4a2b640 1 --2- 172.21.15.5:0/3029127907 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3fa00b8190 0x7f3fa00b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.329+0000 7f3fa4a2b640 1 --2- 172.21.15.5:0/3029127907 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3fa006d3d0 0x7f3fa006d7e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.329+0000 7f3fa4a2b640 1 -- 172.21.15.5:0/3029127907 >> 172.21.15.5:0/3029127907 conn(0x7f3fa00d4bd0 msgr2=0x7f3fa00d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:46.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.329+0000 7f3fa4a2b640 1 -- 172.21.15.5:0/3029127907 shutdown_connections 2024-05-20T18:31:46.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.329+0000 7f3fa4a2b640 1 -- 172.21.15.5:0/3029127907 wait complete. 2024-05-20T18:31:46.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.331+0000 7f3fa4a2b640 1 Processor -- start 2024-05-20T18:31:46.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.331+0000 7f3fa4a2b640 1 -- start start 2024-05-20T18:31:46.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.333+0000 7f3fa4a2b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3fa006d3d0 0x7f3fa01f95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:46.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.333+0000 7f3fa4a2b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fa00b8190 0x7f3fa014eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:46.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.333+0000 7f3fa4a2b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3fa0162820 0x7f3fa01e9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:46.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.333+0000 7f3fa4a2b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa00d0300 con 0x7f3fa00b8190 2024-05-20T18:31:46.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.333+0000 7f3fa4a2b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa00d4260 con 0x7f3fa0162820 2024-05-20T18:31:46.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.333+0000 7f3fa4a2b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3fa00b4050 con 0x7f3fa006d3d0 2024-05-20T18:31:46.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.333+0000 7f3f9dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fa00b8190 0x7f3fa014eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:46.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.333+0000 7f3f9dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fa00b8190 0x7f3fa014eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51684/0 (socket says 172.21.15.5:51684) 2024-05-20T18:31:46.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.333+0000 7f3f9dd9b640 1 -- 172.21.15.5:0/1253900299 learned_addr learned my addr 172.21.15.5:0/1253900299 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:46.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.333+0000 7f3f9e59c640 1 --2- 172.21.15.5:0/1253900299 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3fa006d3d0 0x7f3fa01f95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:46.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.333+0000 7f3f9ed9d640 1 --2- 172.21.15.5:0/1253900299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3fa0162820 0x7f3fa01e9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:46.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.333+0000 7f3f9dd9b640 1 -- 172.21.15.5:0/1253900299 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3fa006d3d0 msgr2=0x7f3fa01f95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:46.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.333+0000 7f3f9dd9b640 1 --2- 172.21.15.5:0/1253900299 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3fa006d3d0 0x7f3fa01f95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.333+0000 7f3f9dd9b640 1 -- 172.21.15.5:0/1253900299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3fa0162820 msgr2=0x7f3fa01e9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:46.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.333+0000 7f3f9dd9b640 1 --2- 172.21.15.5:0/1253900299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3fa0162820 0x7f3fa01e9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.333+0000 7f3f9dd9b640 1 -- 172.21.15.5:0/1253900299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa014f530 con 0x7f3fa00b8190 2024-05-20T18:31:46.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.333+0000 7f3f9ed9d640 1 --2- 172.21.15.5:0/1253900299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3fa0162820 0x7f3fa01e9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:46.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.333+0000 7f3f9dd9b640 1 --2- 172.21.15.5:0/1253900299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fa00b8190 0x7f3fa014eff0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f3f94000c00 tx=0x7f3f940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:46.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.334+0000 7f3f8effd640 1 -- 172.21.15.5:0/1253900299 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94037040 con 0x7f3fa00b8190 2024-05-20T18:31:46.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.334+0000 7f3f8effd640 1 -- 172.21.15.5:0/1253900299 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f94042070 con 0x7f3fa00b8190 2024-05-20T18:31:46.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.334+0000 7f3f8effd640 1 -- 172.21.15.5:0/1253900299 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94030880 con 0x7f3fa00b8190 2024-05-20T18:31:46.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.334+0000 7f3fa4a2b640 1 -- 172.21.15.5:0/1253900299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fa01e80b0 con 0x7f3fa00b8190 2024-05-20T18:31:46.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.334+0000 7f3fa4a2b640 1 -- 172.21.15.5:0/1253900299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fa01e8670 con 0x7f3fa00b8190 2024-05-20T18:31:46.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.335+0000 7f3f8effd640 1 -- 172.21.15.5:0/1253900299 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3f94030a20 con 0x7f3fa00b8190 2024-05-20T18:31:46.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.336+0000 7f3f8effd640 1 --2- 172.21.15.5:0/1253900299 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3f8006be60 0x7f3f8006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:46.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.336+0000 7f3f8effd640 1 -- 172.21.15.5:0/1253900299 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f3f940aba10 con 0x7f3fa00b8190 2024-05-20T18:31:46.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.336+0000 7f3f9e59c640 1 --2- 172.21.15.5:0/1253900299 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3f8006be60 0x7f3f8006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:46.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.336+0000 7f3f9e59c640 1 --2- 172.21.15.5:0/1253900299 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3f8006be60 0x7f3f8006e370 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3f90007fd0 tx=0x7f3f90009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:46.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.337+0000 7f3fab9ae740 1 Processor -- start 2024-05-20T18:31:46.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.337+0000 7f3fab9ae740 1 -- start start 2024-05-20T18:31:46.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.338+0000 7f3fab9ae740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56520463c270 0x565204a07700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:46.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.338+0000 7f3fab9ae740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652046133b0 0x565204a040a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:46.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.338+0000 7f3fab9ae740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56520461b590 0x5652049dd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:46.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.338+0000 7f3fab9ae740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5652049404d0 con 0x5652046133b0 2024-05-20T18:31:46.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.338+0000 7f3fab9ae740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5652048e83e0 con 0x56520461b590 2024-05-20T18:31:46.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.338+0000 7f3fab9ae740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x56520493c490 con 0x56520463c270 2024-05-20T18:31:46.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.339+0000 7f3f9dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652046133b0 0x565204a040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:46.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.339+0000 7f3f9dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652046133b0 0x565204a040a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51692/0 (socket says 172.21.15.5:51692) 2024-05-20T18:31:46.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.339+0000 7f3f9dd9b640 1 -- 172.21.15.5:0/3658412650 learned_addr learned my addr 172.21.15.5:0/3658412650 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:46.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.339+0000 7f3f9e59c640 1 --2- 172.21.15.5:0/3658412650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56520463c270 0x565204a07700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:46.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.339+0000 7f3f9ed9d640 1 --2- 172.21.15.5:0/3658412650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56520461b590 0x5652049dd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:46.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.339+0000 7f3f9dd9b640 1 -- 172.21.15.5:0/3658412650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56520463c270 msgr2=0x565204a07700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:46.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.339+0000 7f3f9dd9b640 1 --2- 172.21.15.5:0/3658412650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56520463c270 0x565204a07700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.339+0000 7f3f9dd9b640 1 -- 172.21.15.5:0/3658412650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56520461b590 msgr2=0x5652049dd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:46.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.339+0000 7f3f9dd9b640 1 --2- 172.21.15.5:0/3658412650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56520461b590 0x5652049dd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.339+0000 7f3f9dd9b640 1 -- 172.21.15.5:0/3658412650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652046542d0 con 0x5652046133b0 2024-05-20T18:31:46.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.339+0000 7f3f9e59c640 1 --2- 172.21.15.5:0/3658412650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56520463c270 0x565204a07700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:46.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.339+0000 7f3f9dd9b640 1 --2- 172.21.15.5:0/3658412650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652046133b0 0x565204a040a0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f3f94035040 tx=0x7f3f94079350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6599e6939f7bd199 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:46.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.339+0000 7f3f7ffff640 1 -- 172.21.15.5:0/3658412650 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9407dce0 con 0x5652046133b0 2024-05-20T18:31:46.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.339+0000 7f3f7ffff640 1 -- 172.21.15.5:0/3658412650 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f9408de50 con 0x5652046133b0 2024-05-20T18:31:46.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.339+0000 7f3f7ffff640 1 -- 172.21.15.5:0/3658412650 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f940825a0 con 0x5652046133b0 2024-05-20T18:31:46.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.340+0000 7f3fab9ae740 1 -- 172.21.15.5:0/3658412650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652046133b0 msgr2=0x565204a040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:46.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.340+0000 7f3fab9ae740 1 --2- 172.21.15.5:0/3658412650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652046133b0 0x565204a040a0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f3f94035040 tx=0x7f3f94079350 comp rx=0 tx=0).stop 2024-05-20T18:31:46.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.340+0000 7f3fab9ae740 1 -- 172.21.15.5:0/3658412650 shutdown_connections 2024-05-20T18:31:46.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.340+0000 7f3fab9ae740 1 --2- 172.21.15.5:0/3658412650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56520461b590 0x5652049dd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.340+0000 7f3fab9ae740 1 --2- 172.21.15.5:0/3658412650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652046133b0 0x565204a040a0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.340+0000 7f3fab9ae740 1 --2- 172.21.15.5:0/3658412650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56520463c270 0x565204a07700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.340+0000 7f3fab9ae740 1 -- 172.21.15.5:0/3658412650 >> 172.21.15.5:0/3658412650 conn(0x5652046303c0 msgr2=0x5652049dcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:46.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.340+0000 7f3fab9ae740 1 -- 172.21.15.5:0/3658412650 shutdown_connections 2024-05-20T18:31:46.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.340+0000 7f3fab9ae740 1 -- 172.21.15.5:0/3658412650 wait complete. 2024-05-20T18:31:46.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.341+0000 7f3fab9ae740 1 -- start start 2024-05-20T18:31:46.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.341+0000 7f3fab9ae740 1 Processor -- start 2024-05-20T18:31:46.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.343+0000 7f3fab9ae740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5652046133b0 0x565204a222c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:46.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.343+0000 7f3fab9ae740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56520461b590 0x565204a09d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:46.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.343+0000 7f3fab9ae740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56520463c270 0x565204a0a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:46.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.343+0000 7f3fab9ae740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5652049404d0 con 0x56520463c270 2024-05-20T18:31:46.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.343+0000 7f3fab9ae740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5652048fb8e0 con 0x5652046133b0 2024-05-20T18:31:46.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.343+0000 7f3fab9ae740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x565204a0b280 con 0x56520461b590 2024-05-20T18:31:46.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.343+0000 7f3fab9ae740 10 client.0 _ll_register_callbacks cb 0x5652048c89c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:31:46.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.343+0000 7f3f9ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56520463c270 0x565204a0a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:46.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.343+0000 7f3f9ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56520463c270 0x565204a0a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51700/0 (socket says 172.21.15.5:51700) 2024-05-20T18:31:46.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.343+0000 7f3f9ed9d640 1 -- 172.21.15.5:0/3895485486 learned_addr learned my addr 172.21.15.5:0/3895485486 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:46.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.343+0000 7f3f9e59c640 1 --2- 172.21.15.5:0/3895485486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5652046133b0 0x565204a222c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:46.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.343+0000 7f3f9dd9b640 1 --2- 172.21.15.5:0/3895485486 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56520461b590 0x565204a09d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:46.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.343+0000 7f3f9ed9d640 1 -- 172.21.15.5:0/3895485486 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56520461b590 msgr2=0x565204a09d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:46.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.343+0000 7f3f9ed9d640 1 --2- 172.21.15.5:0/3895485486 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56520461b590 0x565204a09d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.343+0000 7f3f9ed9d640 1 -- 172.21.15.5:0/3895485486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5652046133b0 msgr2=0x565204a222c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:46.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.343+0000 7f3f9ed9d640 1 --2- 172.21.15.5:0/3895485486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5652046133b0 0x565204a222c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.343+0000 7f3f9ed9d640 1 -- 172.21.15.5:0/3895485486 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56520471e6b0 con 0x56520463c270 2024-05-20T18:31:46.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.343+0000 7f3f9ed9d640 1 --2- 172.21.15.5:0/3895485486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56520463c270 0x565204a0a290 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f3f98007dd0 tx=0x7f3f98014690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb4571430b0bc9d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.343+0000 7f3f7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:31:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.344+0000 7f3f7d7fa640 1 -- 172.21.15.5:0/3895485486 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9801c020 con 0x56520463c270 2024-05-20T18:31:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.344+0000 7f3f7d7fa640 1 -- 172.21.15.5:0/3895485486 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f3f98012040 con 0x56520463c270 2024-05-20T18:31:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.344+0000 7f3fab9ae740 1 -- 172.21.15.5:0/3895485486 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f9408de50 con 0x56520463c270 2024-05-20T18:31:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.344+0000 7f3f7d7fa640 1 -- 172.21.15.5:0/3895485486 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f3f98026050 con 0x56520463c270 2024-05-20T18:31:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.344+0000 7f3fab9ae740 10 client.7020 fetch_fsmap learned FSMap version 50 2024-05-20T18:31:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.344+0000 7f3fab9ae740 1 -- 172.21.15.5:0/3895485486 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x565204a02a10 con 0x56520463c270 2024-05-20T18:31:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.344+0000 7f3f7d7fa640 1 -- 172.21.15.5:0/3895485486 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f9408de50 con 0x56520463c270 2024-05-20T18:31:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.344+0000 7f3f7d7fa640 1 -- 172.21.15.5:0/3895485486 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f3f98029020 con 0x56520463c270 2024-05-20T18:31:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.344+0000 7f3fab9ae740 10 client.7020 fetch_fsmap finished waiting for FSMap version 50 2024-05-20T18:31:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.344+0000 7f3fab9ae740 10 client.7020 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-20T18:31:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.344+0000 7f3fab9ae740 20 client.7020 populate_metadata read hostname 'smithi005' 2024-05-20T18:31:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.344+0000 7f3fab9ae740 1 --2- 172.21.15.5:0/3895485486 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5652045fae40 0x565204a3eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.344+0000 7f3fab9ae740 4 client.7020 mds_command: new command op to 6972 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:31:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.344+0000 7f3fab9ae740 1 -- 172.21.15.5:0/3895485486 --> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56520494dc00 con 0x5652045fae40 2024-05-20T18:31:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.344+0000 7f3f9e59c640 1 --2- 172.21.15.5:0/3895485486 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5652045fae40 0x565204a3eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.345+0000 7f3f7d7fa640 10 client.7020 ms_handle_connect on v2:172.21.15.5:6835/3866822752 2024-05-20T18:31:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.345+0000 7f3f9e59c640 1 --2- 172.21.15.5:0/3895485486 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5652045fae40 0x565204a3eed0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4ace315b230ace1 server_cookie=87faf35f39164d7c in_seq=0 out_seq=0 2024-05-20T18:31:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.345+0000 7f3f7d7fa640 1 -- 172.21.15.5:0/3895485486 <== mds.0 v2:172.21.15.5:6835/3866822752 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56520494dc00 con 0x5652045fae40 2024-05-20T18:31:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.345+0000 7f3f7d7fa640 10 client.7020 handle_command_reply: tid=0 2024-05-20T18:31:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.345+0000 7f3fab9ae740 1 client.7020 shutdown 2024-05-20T18:31:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.346+0000 7f3fab9ae740 1 -- 172.21.15.5:0/3895485486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56520463c270 msgr2=0x565204a0a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.346+0000 7f3fab9ae740 1 --2- 172.21.15.5:0/3895485486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56520463c270 0x565204a0a290 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f3f98007dd0 tx=0x7f3f98014690 comp rx=0 tx=0).stop 2024-05-20T18:31:46.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.346+0000 7f3fab9ae740 1 -- 172.21.15.5:0/3895485486 shutdown_connections 2024-05-20T18:31:46.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.346+0000 7f3fab9ae740 1 --2- 172.21.15.5:0/3895485486 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5652045fae40 0x565204a3eed0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.346+0000 7f3fab9ae740 1 --2- 172.21.15.5:0/3895485486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56520463c270 0x565204a0a290 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.346+0000 7f3fab9ae740 1 --2- 172.21.15.5:0/3895485486 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56520461b590 0x565204a09d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.346+0000 7f3f7d7fa640 0 client.7020 ms_handle_reset on v2:172.21.15.5:6835/3866822752 2024-05-20T18:31:46.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.346+0000 7f3fab9ae740 1 --2- 172.21.15.5:0/3895485486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5652046133b0 0x565204a222c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.346+0000 7f3fab9ae740 1 -- 172.21.15.5:0/3895485486 >> 172.21.15.5:0/3895485486 conn(0x5652046303c0 msgr2=0x5652049dcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:46.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.346+0000 7f3fab9ae740 1 -- 172.21.15.5:0/3895485486 shutdown_connections 2024-05-20T18:31:46.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.346+0000 7f3fab9ae740 1 -- 172.21.15.5:0/3895485486 wait complete. 2024-05-20T18:31:46.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.346+0000 7f3fab9ae740 20 client.7020 trim_cache size 0 max 16384 2024-05-20T18:31:46.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.365+0000 7f3fab9ae740 1 Processor -- start 2024-05-20T18:31:46.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.365+0000 7f3fab9ae740 1 -- start start 2024-05-20T18:31:46.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.366+0000 7f3fab9ae740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652046133b0 0x565204a041e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:46.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.366+0000 7f3fab9ae740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56520461b590 0x565204a17b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:46.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.366+0000 7f3fab9ae740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56520463c270 0x565204a180d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:46.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.366+0000 7f3fab9ae740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa02016d0 con 0x5652046133b0 2024-05-20T18:31:46.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.366+0000 7f3fab9ae740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa00ccda0 con 0x56520463c270 2024-05-20T18:31:46.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.366+0000 7f3fab9ae740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3fa0201080 con 0x56520461b590 2024-05-20T18:31:46.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.366+0000 7f3f9e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652046133b0 0x565204a041e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:46.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.366+0000 7f3f9e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652046133b0 0x565204a041e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51706/0 (socket says 172.21.15.5:51706) 2024-05-20T18:31:46.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.366+0000 7f3f9e59c640 1 -- 172.21.15.5:0/1477128877 learned_addr learned my addr 172.21.15.5:0/1477128877 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:46.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.367+0000 7f3f9dd9b640 1 --2- 172.21.15.5:0/1477128877 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56520461b590 0x565204a17b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:46.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.367+0000 7f3f9ed9d640 1 --2- 172.21.15.5:0/1477128877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56520463c270 0x565204a180d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:46.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.367+0000 7f3f9e59c640 1 -- 172.21.15.5:0/1477128877 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56520461b590 msgr2=0x565204a17b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:46.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.367+0000 7f3f9e59c640 1 --2- 172.21.15.5:0/1477128877 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56520461b590 0x565204a17b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.367+0000 7f3f9e59c640 1 -- 172.21.15.5:0/1477128877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56520463c270 msgr2=0x565204a180d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:46.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.367+0000 7f3f9e59c640 1 --2- 172.21.15.5:0/1477128877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56520463c270 0x565204a180d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.367+0000 7f3f9e59c640 1 -- 172.21.15.5:0/1477128877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652049d95b0 con 0x5652046133b0 2024-05-20T18:31:46.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.367+0000 7f3f9dd9b640 1 --2- 172.21.15.5:0/1477128877 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56520461b590 0x565204a17b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:46.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.367+0000 7f3f9ed9d640 1 --2- 172.21.15.5:0/1477128877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56520463c270 0x565204a180d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:46.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.367+0000 7f3f9e59c640 1 --2- 172.21.15.5:0/1477128877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652046133b0 0x565204a041e0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f3f90005620 tx=0x7f3f90017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a10ee60d2fbe4f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:46.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.367+0000 7f3f7f7fe640 1 -- 172.21.15.5:0/1477128877 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9001e040 con 0x5652046133b0 2024-05-20T18:31:46.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.367+0000 7f3f7f7fe640 1 -- 172.21.15.5:0/1477128877 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f9002a070 con 0x5652046133b0 2024-05-20T18:31:46.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.367+0000 7f3f7f7fe640 1 -- 172.21.15.5:0/1477128877 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f90025660 con 0x5652046133b0 2024-05-20T18:31:46.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.368+0000 7f3fab9ae740 1 -- 172.21.15.5:0/1477128877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652046133b0 msgr2=0x565204a041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:46.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.368+0000 7f3fab9ae740 1 --2- 172.21.15.5:0/1477128877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652046133b0 0x565204a041e0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f3f90005620 tx=0x7f3f90017040 comp rx=0 tx=0).stop 2024-05-20T18:31:46.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.368+0000 7f3fab9ae740 1 -- 172.21.15.5:0/1477128877 shutdown_connections 2024-05-20T18:31:46.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.368+0000 7f3fab9ae740 1 --2- 172.21.15.5:0/1477128877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56520463c270 0x565204a180d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.368+0000 7f3fab9ae740 1 --2- 172.21.15.5:0/1477128877 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56520461b590 0x565204a17b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.368+0000 7f3fab9ae740 1 --2- 172.21.15.5:0/1477128877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652046133b0 0x565204a041e0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.368+0000 7f3fab9ae740 1 -- 172.21.15.5:0/1477128877 >> 172.21.15.5:0/1477128877 conn(0x5652046303c0 msgr2=0x5652049d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:46.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.368+0000 7f3fab9ae740 1 -- 172.21.15.5:0/1477128877 shutdown_connections 2024-05-20T18:31:46.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.368+0000 7f3fab9ae740 1 -- 172.21.15.5:0/1477128877 wait complete. 2024-05-20T18:31:46.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.369+0000 7f3fab9ae740 1 -- start start 2024-05-20T18:31:46.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.369+0000 7f3fab9ae740 1 Processor -- start 2024-05-20T18:31:46.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.370+0000 7f3fab9ae740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5652046133b0 0x565204a00610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:46.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.370+0000 7f3fab9ae740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56520461b590 0x565204a21f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:46.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.370+0000 7f3fab9ae740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56520463c270 0x565204a22440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:46.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.370+0000 7f3fab9ae740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x565204a3a9e0 con 0x56520463c270 2024-05-20T18:31:46.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.370+0000 7f3fab9ae740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x565204a0af50 con 0x56520461b590 2024-05-20T18:31:46.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.370+0000 7f3fab9ae740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x565204a3b030 con 0x5652046133b0 2024-05-20T18:31:46.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.370+0000 7f3fab9ae740 10 client.0 _ll_register_callbacks cb 0x7f3f980079b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:31:46.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.370+0000 7f3f9ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56520463c270 0x565204a22440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:46.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.370+0000 7f3f9ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56520463c270 0x565204a22440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51718/0 (socket says 172.21.15.5:51718) 2024-05-20T18:31:46.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.370+0000 7f3f9ed9d640 1 -- 172.21.15.5:0/3736482727 learned_addr learned my addr 172.21.15.5:0/3736482727 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:46.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.370+0000 7f3f9e59c640 1 --2- 172.21.15.5:0/3736482727 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5652046133b0 0x565204a00610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:46.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.370+0000 7f3f9dd9b640 1 --2- 172.21.15.5:0/3736482727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56520461b590 0x565204a21f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:46.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.371+0000 7f3f9ed9d640 1 -- 172.21.15.5:0/3736482727 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5652046133b0 msgr2=0x565204a00610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:46.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.371+0000 7f3f9ed9d640 1 --2- 172.21.15.5:0/3736482727 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5652046133b0 0x565204a00610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.371+0000 7f3f9ed9d640 1 -- 172.21.15.5:0/3736482727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56520461b590 msgr2=0x565204a21f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:46.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.371+0000 7f3f9ed9d640 1 --2- 172.21.15.5:0/3736482727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56520461b590 0x565204a21f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.371+0000 7f3f9ed9d640 1 -- 172.21.15.5:0/3736482727 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565204a185a0 con 0x56520463c270 2024-05-20T18:31:46.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.371+0000 7f3f9ed9d640 1 --2- 172.21.15.5:0/3736482727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56520463c270 0x565204a22440 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800b010 tx=0x7f3f98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ef07ce10176a45 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:46.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.371+0000 7f3f7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:31:46.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.371+0000 7f3f7d7fa640 1 -- 172.21.15.5:0/3736482727 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9801c030 con 0x56520463c270 2024-05-20T18:31:46.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.371+0000 7f3f7d7fa640 1 -- 172.21.15.5:0/3736482727 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f3f98017710 con 0x56520463c270 2024-05-20T18:31:46.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.371+0000 7f3fab9ae740 1 -- 172.21.15.5:0/3736482727 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f9002a070 con 0x56520463c270 2024-05-20T18:31:46.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.371+0000 7f3f7d7fa640 1 -- 172.21.15.5:0/3736482727 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f3f98003990 con 0x56520463c270 2024-05-20T18:31:46.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.371+0000 7f3fab9ae740 10 client.7026 fetch_fsmap learned FSMap version 50 2024-05-20T18:31:46.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.371+0000 7f3fab9ae740 1 -- 172.21.15.5:0/3736482727 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x5652049d6880 con 0x56520463c270 2024-05-20T18:31:46.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.371+0000 7f3f7d7fa640 1 -- 172.21.15.5:0/3736482727 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f9002a070 con 0x56520463c270 2024-05-20T18:31:46.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.372+0000 7f3f7d7fa640 1 -- 172.21.15.5:0/3736482727 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f3f98029040 con 0x56520463c270 2024-05-20T18:31:46.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.372+0000 7f3fab9ae740 10 client.7026 fetch_fsmap finished waiting for FSMap version 50 2024-05-20T18:31:46.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.372+0000 7f3fab9ae740 10 client.7026 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-20T18:31:46.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.372+0000 7f3fab9ae740 20 client.7026 populate_metadata read hostname 'smithi005' 2024-05-20T18:31:46.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.372+0000 7f3fab9ae740 1 --2- 172.21.15.5:0/3736482727 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5652049dd5c0 0x565204a35ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:46.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.372+0000 7f3fab9ae740 4 client.7026 mds_command: new command op to 6972 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T18:31:46.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.372+0000 7f3fab9ae740 1 -- 172.21.15.5:0/3736482727 --> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] -- command(tid 0: {"prefix": "status"}) -- 0x565204947f50 con 0x5652049dd5c0 2024-05-20T18:31:46.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.372+0000 7f3f9e59c640 1 --2- 172.21.15.5:0/3736482727 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5652049dd5c0 0x565204a35ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:46.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.372+0000 7f3f7d7fa640 10 client.7026 ms_handle_connect on v2:172.21.15.5:6835/3866822752 2024-05-20T18:31:46.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.372+0000 7f3f9e59c640 1 --2- 172.21.15.5:0/3736482727 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5652049dd5c0 0x565204a35ee0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=641eca48326affc1 server_cookie=3167536fa790e2b4 in_seq=0 out_seq=0 2024-05-20T18:31:46.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.372+0000 7f3f7d7fa640 1 -- 172.21.15.5:0/3736482727 <== mds.0 v2:172.21.15.5:6835/3866822752 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x565204947f50 con 0x5652049dd5c0 2024-05-20T18:31:46.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.373+0000 7f3f7d7fa640 10 client.7026 handle_command_reply: tid=0 2024-05-20T18:31:46.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.373+0000 7f3fab9ae740 1 client.7026 shutdown 2024-05-20T18:31:46.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.373+0000 7f3fab9ae740 1 -- 172.21.15.5:0/3736482727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56520463c270 msgr2=0x565204a22440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:46.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.373+0000 7f3fab9ae740 1 --2- 172.21.15.5:0/3736482727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56520463c270 0x565204a22440 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800b010 tx=0x7f3f98004a40 comp rx=0 tx=0).stop 2024-05-20T18:31:46.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.373+0000 7f3fab9ae740 1 -- 172.21.15.5:0/3736482727 shutdown_connections 2024-05-20T18:31:46.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.373+0000 7f3fab9ae740 1 --2- 172.21.15.5:0/3736482727 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5652049dd5c0 0x565204a35ee0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.373+0000 7f3fab9ae740 1 --2- 172.21.15.5:0/3736482727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56520463c270 0x565204a22440 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.373+0000 7f3fab9ae740 1 --2- 172.21.15.5:0/3736482727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56520461b590 0x565204a21f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.373+0000 7f3fab9ae740 1 --2- 172.21.15.5:0/3736482727 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5652046133b0 0x565204a00610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.373+0000 7f3f7d7fa640 0 client.7026 ms_handle_reset on v2:172.21.15.5:6835/3866822752 2024-05-20T18:31:46.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.373+0000 7f3fab9ae740 1 -- 172.21.15.5:0/3736482727 >> 172.21.15.5:0/3736482727 conn(0x5652046303c0 msgr2=0x5652049d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:46.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.373+0000 7f3fab9ae740 1 -- 172.21.15.5:0/3736482727 shutdown_connections 2024-05-20T18:31:46.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.373+0000 7f3fab9ae740 1 -- 172.21.15.5:0/3736482727 wait complete. 2024-05-20T18:31:46.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.373+0000 7f3fab9ae740 20 client.7026 trim_cache size 0 max 16384 2024-05-20T18:31:46.380 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:31:46.380 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "399ab250-fd3c-47fa-a1c8-25e8149c9cac", 2024-05-20T18:31:46.380 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-20T18:31:46.380 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 6972, 2024-05-20T18:31:46.380 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-20T18:31:46.380 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-20T18:31:46.380 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-20T18:31:46.380 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 4.5226723949999998, 2024-05-20T18:31:46.380 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 50, 2024-05-20T18:31:46.380 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 78, 2024-05-20T18:31:46.380 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 78, 2024-05-20T18:31:46.381 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 5.4630172760000004 2024-05-20T18:31:46.381 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:31:46.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.373+0000 7f3fa4a2b640 1 -- 172.21.15.5:0/1253900299 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3f8006be60 msgr2=0x7f3f8006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:46.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.373+0000 7f3fa4a2b640 1 --2- 172.21.15.5:0/1253900299 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3f8006be60 0x7f3f8006e370 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3f90007fd0 tx=0x7f3f90009040 comp rx=0 tx=0).stop 2024-05-20T18:31:46.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.373+0000 7f3fa4a2b640 1 -- 172.21.15.5:0/1253900299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fa00b8190 msgr2=0x7f3fa014eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:46.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.373+0000 7f3fa4a2b640 1 --2- 172.21.15.5:0/1253900299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fa00b8190 0x7f3fa014eff0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f3f94000c00 tx=0x7f3f940046c0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.374+0000 7f3fa4a2b640 1 -- 172.21.15.5:0/1253900299 shutdown_connections 2024-05-20T18:31:46.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.374+0000 7f3fa4a2b640 1 --2- 172.21.15.5:0/1253900299 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3f8006be60 0x7f3f8006e370 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.374+0000 7f3fa4a2b640 1 --2- 172.21.15.5:0/1253900299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3fa0162820 0x7f3fa01e9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.374+0000 7f3fa4a2b640 1 --2- 172.21.15.5:0/1253900299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fa00b8190 0x7f3fa014eff0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.374+0000 7f3fa4a2b640 1 --2- 172.21.15.5:0/1253900299 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3fa006d3d0 0x7f3fa01f95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.374+0000 7f3fa4a2b640 1 -- 172.21.15.5:0/1253900299 >> 172.21.15.5:0/1253900299 conn(0x7f3fa00d4bd0 msgr2=0x7f3fa00b6fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:46.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.374+0000 7f3fa4a2b640 1 -- 172.21.15.5:0/1253900299 shutdown_connections 2024-05-20T18:31:46.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.374+0000 7f3fa4a2b640 1 -- 172.21.15.5:0/1253900299 wait complete. 2024-05-20T18:31:46.385 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-20T18:31:46.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.499+0000 7f9319157640 1 Processor -- start 2024-05-20T18:31:46.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.499+0000 7f9319157640 1 -- start start 2024-05-20T18:31:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.501+0000 7f9319157640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9314168050 0x7f931416a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.501+0000 7f9319157640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93140d0630 con 0x7f931406d3d0 2024-05-20T18:31:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.501+0000 7f9319157640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93140d39c0 con 0x7f9314162820 2024-05-20T18:31:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.501+0000 7f9319157640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f93140d3ca0 con 0x7f9314168050 2024-05-20T18:31:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.501+0000 7f931259c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f931406d3d0 0x7f931406d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:48842/0 (socket says 172.21.15.5:48842) 2024-05-20T18:31:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.501+0000 7f931259c640 1 -- 172.21.15.5:0/2916844976 learned_addr learned my addr 172.21.15.5:0/2916844976 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.501+0000 7f931359e640 1 --2- 172.21.15.5:0/2916844976 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9314168050 0x7f931416a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.501+0000 7f9311d9b640 1 -- 172.21.15.5:0/2916844976 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 688108630 0 0) 0x7f93140d0630 con 0x7f931406d3d0 2024-05-20T18:31:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.501+0000 7f9311d9b640 1 -- 172.21.15.5:0/2916844976 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92fc003520 con 0x7f931406d3d0 2024-05-20T18:31:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.502+0000 7f9311d9b640 1 -- 172.21.15.5:0/2916844976 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 967972204 0 0) 0x7f92fc003520 con 0x7f931406d3d0 2024-05-20T18:31:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.502+0000 7f9311d9b640 1 -- 172.21.15.5:0/2916844976 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9314168050 msgr2=0x7f931416a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.502+0000 7f9311d9b640 1 --2- 172.21.15.5:0/2916844976 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9314168050 0x7f931416a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.502+0000 7f9311d9b640 1 -- 172.21.15.5:0/2916844976 >> v1:172.21.15.63:6789/0 conn(0x7f9314162820 legacy=0x7f93140b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.502+0000 7f9311d9b640 1 -- 172.21.15.5:0/2916844976 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93140d3f80 con 0x7f931406d3d0 2024-05-20T18:31:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.502+0000 7f9311d9b640 1 -- 172.21.15.5:0/2916844976 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f9308001670 con 0x7f931406d3d0 2024-05-20T18:31:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.502+0000 7f9311d9b640 1 -- 172.21.15.5:0/2916844976 >> v1:172.21.15.5:6789/0 conn(0x7f931406d3d0 legacy=0x7f931406d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.502+0000 7f9311d9b640 1 --2- 172.21.15.5:0/2916844976 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9314168050 0x7f92fc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.502+0000 7f9311d9b640 1 --2- 172.21.15.5:0/2916844976 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92fc003f60 0x7f92fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.502+0000 7f9311d9b640 1 --2- 172.21.15.5:0/2916844976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92fc004d40 0x7f92fc005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.502+0000 7f9311d9b640 1 -- 172.21.15.5:0/2916844976 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f93140d3ca0 con 0x7f92fc004d40 2024-05-20T18:31:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.502+0000 7f9311d9b640 1 -- 172.21.15.5:0/2916844976 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc00bb50 con 0x7f92fc003f60 2024-05-20T18:31:46.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.502+0000 7f9311d9b640 1 -- 172.21.15.5:0/2916844976 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f92fc00bd50 con 0x7f9314168050 2024-05-20T18:31:46.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.502+0000 7f9312d9d640 1 --2- 172.21.15.5:0/2916844976 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9314168050 0x7f92fc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:46.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.502+0000 7f931359e640 1 --2- 172.21.15.5:0/2916844976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92fc004d40 0x7f92fc005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:46.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.502+0000 7f931259c640 1 --2- 172.21.15.5:0/2916844976 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92fc003f60 0x7f92fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:46.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.502+0000 7f931359e640 1 -- 172.21.15.5:0/2916844976 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9314168050 msgr2=0x7f92fc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:46.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.502+0000 7f931359e640 1 --2- 172.21.15.5:0/2916844976 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9314168050 0x7f92fc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.502+0000 7f931359e640 1 -- 172.21.15.5:0/2916844976 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92fc003f60 msgr2=0x7f92fc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:46.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.502+0000 7f931359e640 1 --2- 172.21.15.5:0/2916844976 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92fc003f60 0x7f92fc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.502+0000 7f931359e640 1 -- 172.21.15.5:0/2916844976 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92fc003520 con 0x7f92fc004d40 2024-05-20T18:31:46.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.503+0000 7f931359e640 1 --2- 172.21.15.5:0/2916844976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92fc004d40 0x7f92fc005150 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f93140cbc50 tx=0x7f92fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d02c1432bb27bf5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:46.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.503+0000 7f9311d9b640 1 -- 172.21.15.5:0/2916844976 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c009c10 con 0x7f92fc004d40 2024-05-20T18:31:46.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.503+0000 7f9311d9b640 1 -- 172.21.15.5:0/2916844976 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f930c036070 con 0x7f92fc004d40 2024-05-20T18:31:46.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.503+0000 7f9311d9b640 1 -- 172.21.15.5:0/2916844976 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c0317f0 con 0x7f92fc004d40 2024-05-20T18:31:46.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.504+0000 7f9319157640 1 -- 172.21.15.5:0/2916844976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92fc004d40 msgr2=0x7f92fc005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.504+0000 7f9319157640 1 --2- 172.21.15.5:0/2916844976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92fc004d40 0x7f92fc005150 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f93140cbc50 tx=0x7f92fc005830 comp rx=0 tx=0).stop 2024-05-20T18:31:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.504+0000 7f931359e640 1 -- 172.21.15.5:0/2916844976 reap_dead start 2024-05-20T18:31:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.504+0000 7f9319157640 1 -- 172.21.15.5:0/2916844976 shutdown_connections 2024-05-20T18:31:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.504+0000 7f9319157640 1 -- 172.21.15.5:0/2916844976 >> 172.21.15.5:0/2916844976 conn(0x7f93140d4bd0 msgr2=0x7f93140d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.504+0000 7f9319157640 1 -- 172.21.15.5:0/2916844976 shutdown_connections 2024-05-20T18:31:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.504+0000 7f9319157640 1 -- 172.21.15.5:0/2916844976 wait complete. 2024-05-20T18:31:46.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.506+0000 7f9319157640 1 Processor -- start 2024-05-20T18:31:46.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.506+0000 7f9319157640 1 -- start start 2024-05-20T18:31:46.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.507+0000 7f9319157640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92fc004d40 0x7f931414ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:46.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.507+0000 7f9319157640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f931414aeb0 0x7f93140c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:46.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.507+0000 7f9319157640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93140c9e50 0x7f93140cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:46.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.507+0000 7f9319157640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f93140b91a0 con 0x7f93140c9e50 2024-05-20T18:31:46.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.507+0000 7f9319157640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f93140d0300 con 0x7f92fc004d40 2024-05-20T18:31:46.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.507+0000 7f9319157640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f93140b4050 con 0x7f931414aeb0 2024-05-20T18:31:46.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.507+0000 7f931359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93140c9e50 0x7f93140cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:46.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.507+0000 7f931359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93140c9e50 0x7f93140cac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51740/0 (socket says 172.21.15.5:51740) 2024-05-20T18:31:46.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.507+0000 7f931359e640 1 -- 172.21.15.5:0/201250084 learned_addr learned my addr 172.21.15.5:0/201250084 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:46.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.507+0000 7f931259c640 1 --2- 172.21.15.5:0/201250084 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f931414aeb0 0x7f93140c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:46.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.507+0000 7f9312d9d640 1 --2- 172.21.15.5:0/201250084 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92fc004d40 0x7f931414ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:46.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.508+0000 7f931359e640 1 -- 172.21.15.5:0/201250084 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f931414aeb0 msgr2=0x7f93140c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:46.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.508+0000 7f931359e640 1 --2- 172.21.15.5:0/201250084 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f931414aeb0 0x7f93140c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.508+0000 7f931359e640 1 -- 172.21.15.5:0/201250084 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92fc004d40 msgr2=0x7f931414ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:46.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.508+0000 7f931359e640 1 --2- 172.21.15.5:0/201250084 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92fc004d40 0x7f931414ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.508+0000 7f931359e640 1 -- 172.21.15.5:0/201250084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93140c8870 con 0x7f93140c9e50 2024-05-20T18:31:46.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.508+0000 7f931359e640 1 --2- 172.21.15.5:0/201250084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93140c9e50 0x7f93140cac00 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f930c033040 tx=0x7f930c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:46.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.508+0000 7f92fb7fe640 1 -- 172.21.15.5:0/201250084 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c009c10 con 0x7f93140c9e50 2024-05-20T18:31:46.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.508+0000 7f92fb7fe640 1 -- 172.21.15.5:0/201250084 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f930c045430 con 0x7f93140c9e50 2024-05-20T18:31:46.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.508+0000 7f9319157640 1 -- 172.21.15.5:0/201250084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93140c8b50 con 0x7f93140c9e50 2024-05-20T18:31:46.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.508+0000 7f92fb7fe640 1 -- 172.21.15.5:0/201250084 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c045800 con 0x7f93140c9e50 2024-05-20T18:31:46.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.508+0000 7f9319157640 1 -- 172.21.15.5:0/201250084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93140c9110 con 0x7f93140c9e50 2024-05-20T18:31:46.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.509+0000 7f9319157640 1 -- 172.21.15.5:0/201250084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9314075b00 con 0x7f93140c9e50 2024-05-20T18:31:46.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.510+0000 7f92fb7fe640 1 -- 172.21.15.5:0/201250084 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f930c005d10 con 0x7f93140c9e50 2024-05-20T18:31:46.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.510+0000 7f92fb7fe640 1 --2- 172.21.15.5:0/201250084 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f930006be30 0x7f930006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:46.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.510+0000 7f92fb7fe640 1 -- 172.21.15.5:0/201250084 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f930c0abcc0 con 0x7f93140c9e50 2024-05-20T18:31:46.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.510+0000 7f9312d9d640 1 --2- 172.21.15.5:0/201250084 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f930006be30 0x7f930006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:46.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.511+0000 7f9312d9d640 1 --2- 172.21.15.5:0/201250084 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f930006be30 0x7f930006e340 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f93040017e0 tx=0x7f9304009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:46.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.515+0000 7f92fb7fe640 1 -- 172.21.15.5:0/201250084 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f930c079e20 con 0x7f93140c9e50 2024-05-20T18:31:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.739+0000 7f9319157640 1 -- 172.21.15.5:0/201250084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f93140cef40 con 0x7f93140c9e50 2024-05-20T18:31:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.740+0000 7f92fb7fe640 1 -- 172.21.15.5:0/201250084 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v50) ==== 53+0+73 (secure 0 0 0) 0x7f930c07e120 con 0x7f93140c9e50 2024-05-20T18:31:46.740 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-20T18:31:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.742+0000 7f9319157640 1 -- 172.21.15.5:0/201250084 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f930006be30 msgr2=0x7f930006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.742+0000 7f9319157640 1 --2- 172.21.15.5:0/201250084 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f930006be30 0x7f930006e340 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f93040017e0 tx=0x7f9304009040 comp rx=0 tx=0).stop 2024-05-20T18:31:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.742+0000 7f9319157640 1 -- 172.21.15.5:0/201250084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93140c9e50 msgr2=0x7f93140cac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.742+0000 7f9319157640 1 --2- 172.21.15.5:0/201250084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93140c9e50 0x7f93140cac00 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f930c033040 tx=0x7f930c009750 comp rx=0 tx=0).stop 2024-05-20T18:31:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.742+0000 7f9319157640 1 -- 172.21.15.5:0/201250084 shutdown_connections 2024-05-20T18:31:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.742+0000 7f9319157640 1 --2- 172.21.15.5:0/201250084 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f930006be30 0x7f930006e340 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.742+0000 7f9319157640 1 --2- 172.21.15.5:0/201250084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93140c9e50 0x7f93140cac00 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.742+0000 7f9319157640 1 --2- 172.21.15.5:0/201250084 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f931414aeb0 0x7f93140c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.742+0000 7f9319157640 1 --2- 172.21.15.5:0/201250084 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92fc004d40 0x7f931414ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.742+0000 7f9319157640 1 -- 172.21.15.5:0/201250084 >> 172.21.15.5:0/201250084 conn(0x7f93140d4bd0 msgr2=0x7f93140b6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.742+0000 7f9319157640 1 -- 172.21.15.5:0/201250084 shutdown_connections 2024-05-20T18:31:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46.742+0000 7f9319157640 1 -- 172.21.15.5:0/201250084 wait complete. 2024-05-20T18:31:46.753 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-20T18:31:46.753 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-20T18:31:46.754 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-05-20T18:31:46.754 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-20T18:31:46.754 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-20T18:31:46.754 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-20T18:31:46.754 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-20T18:31:46.754 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-20T18:31:46.754 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:31:46.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:31:46.754 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-05-20T18:31:46.771 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T18:31:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T18:31:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T18:31:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:31:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-20T18:31:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:31:46.771 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:31:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-05-20T18:31:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-05-20T18:31:46.771 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:31:46.772 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-05-20T18:31:46.772 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-05-20T18:31:46.772 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-20T18:31:46.772 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:31:46.772 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-05-20T18:31:46.772 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-20T18:31:46.772 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42391sec preferred_lft 42391sec 2024-05-20T18:31:46.772 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-05-20T18:31:46.772 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:31:46.772 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:31:46.772 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:31:46.772 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-05-20T18:31:46.772 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-20T18:31:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:31:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-20T18:31:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:31:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::4474:eaff:fe07:4f9a/64 scope link 2024-05-20T18:31:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:31:46.773 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:31:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T18:31:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ce4:f4ff:fe5e:7906/64 scope link 2024-05-20T18:31:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:31:46.773 INFO:teuthology.orchestra.run.smithi005.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:31:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether da:26:04:0e:95:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T18:31:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::d826:4ff:fe0e:950a/64 scope link 2024-05-20T18:31:46.773 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:31:46.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:31:46.774 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-20T18:31:46.836 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-20T18:31:46.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:31:46.838 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-05-20T18:31:46.852 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-20T18:31:46.852+0000 7f712b412640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T18:31:46.868 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-05-20T18:31:46.868 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-20T18:31:46.868 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-20T18:31:46.868 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-20T18:31:46.868 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-20T18:31:46.869 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:31:46.869 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-20T18:31:46.869 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-20T18:31:46.869 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-20T18:31:46.869 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-20T18:31:46.869 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-20T18:31:46.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:31:46.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:31:46.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:31:46.999 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-20T18:31:47.015 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-20T18:31:47.015 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-20T18:31:47.016 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:31:47.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:31:47.016 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-05-20T18:31:47.082 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:31:47.082 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-20T18:31:47.138 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:31:47.142 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:31:47.142 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:31:47.195 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-20T18:31:47.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:31:47.195 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:31:47.291 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:31:47.291 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:31:47.291 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:31:47.302 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:31:47.302 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:31:47.302 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:31:47.302 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:31:47.302 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:31:47.308 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:31:47.309 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:31:47.309 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:31:47.309 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:31:47.309 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:31:47.309 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790" 2024-05-20T18:31:47.309 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790:/ 2024-05-20T18:31:47.309 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=399ab250-fd3c-47fa-a1c8-25e8149c9cac" will pass to kernel 2024-05-20T18:31:47.310 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-20T18:31:47.310 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-05-20T18:31:47.310 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-20T18:31:47.310 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-05-20T18:31:47.310 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-20T18:31:47.311 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-20T18:31:47.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:31:47.312 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:31:47.342 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-20T18:31:47.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T18:31:47.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:31:47.440 INFO:teuthology.orchestra.run.smithi005.stdout:client7035 2024-05-20T18:31:47.440 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:31:47.440 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:31:47.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:31:47.457 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-05-20T18:31:47.515 INFO:teuthology.orchestra.run.smithi005.stdout:file_undamaged 2024-05-20T18:31:47.516 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:31:47.516 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:31:47.517 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:31:47.517 DEBUG:teuthology.orchestra.run.smithi005:> import json 2024-05-20T18:31:47.517 DEBUG:teuthology.orchestra.run.smithi005:> import sys 2024-05-20T18:31:47.517 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:31:47.517 DEBUG:teuthology.orchestra.run.smithi005:> try: 2024-05-20T18:31:47.517 DEBUG:teuthology.orchestra.run.smithi005:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-05-20T18:31:47.517 DEBUG:teuthology.orchestra.run.smithi005:> except OSError as e: 2024-05-20T18:31:47.517 DEBUG:teuthology.orchestra.run.smithi005:> sys.exit(e.errno) 2024-05-20T18:31:47.517 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:31:47.517 DEBUG:teuthology.orchestra.run.smithi005:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-05-20T18:31:47.517 DEBUG:teuthology.orchestra.run.smithi005:> print(json.dumps( 2024-05-20T18:31:47.518 DEBUG:teuthology.orchestra.run.smithi005:> dict([(a, getattr(s, a)) for a in attrs]), 2024-05-20T18:31:47.518 DEBUG:teuthology.orchestra.run.smithi005:> indent=2)) 2024-05-20T18:31:47.518 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:31:47.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-05-20T18:31:47.634 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-05-20T18:31:47.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:31:47.635 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-05-20T18:31:47.692 INFO:teuthology.orchestra.run.smithi005.stdout:2 2024-05-20T18:31:47.692 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:31:47.692 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:31:47.750 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:31:47.750 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:31:47.807 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:31:47.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.961+0000 7fec4c6a8640 1 Processor -- start 2024-05-20T18:31:47.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.961+0000 7fec4c6a8640 1 -- start start 2024-05-20T18:31:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.963+0000 7fec4c6a8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec440b47b0 0x7fec440afad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.963+0000 7fec4c6a8640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fec440b58f0 0x7fec4406f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.963+0000 7fec4c6a8640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec440b39f0 con 0x7fec4415d060 2024-05-20T18:31:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.963+0000 7fec4c6a8640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fec440b3cd0 con 0x7fec440b58f0 2024-05-20T18:31:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.963+0000 7fec4c6a8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fec440b3fb0 con 0x7fec440b47b0 2024-05-20T18:31:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.963+0000 7fec4ac45640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fec440b58f0 0x7fec4406f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.963+0000 7fec4ac45640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fec440b58f0 0x7fec4406f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51756/0 (socket says 172.21.15.5:51756) 2024-05-20T18:31:47.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.963+0000 7fec4ac45640 1 -- 172.21.15.5:0/1822458753 learned_addr learned my addr 172.21.15.5:0/1822458753 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.963+0000 7fec4a444640 1 --2- 172.21.15.5:0/1822458753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec440b47b0 0x7fec440afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.963+0000 7fec4ac45640 1 -- 172.21.15.5:0/1822458753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec440b47b0 msgr2=0x7fec440afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.963+0000 7fec4ac45640 1 --2- 172.21.15.5:0/1822458753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec440b47b0 0x7fec440afad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.963+0000 7fec4ac45640 1 -- 172.21.15.5:0/1822458753 >> v1:172.21.15.63:6789/0 conn(0x7fec4415d060 legacy=0x7fec440b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.963+0000 7fec4ac45640 1 -- 172.21.15.5:0/1822458753 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec440b4290 con 0x7fec440b58f0 2024-05-20T18:31:47.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.964+0000 7fec4a444640 1 --2- 172.21.15.5:0/1822458753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec440b47b0 0x7fec440afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:47.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.964+0000 7fec4ac45640 1 --2- 172.21.15.5:0/1822458753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec440b58f0 0x7fec4406f3d0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fec3400d150 tx=0x7fec3402c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcda7a6363cb1a2b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:47.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.964+0000 7fec49442640 1 -- 172.21.15.5:0/1822458753 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34039070 con 0x7fec440b58f0 2024-05-20T18:31:47.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.964+0000 7fec49442640 1 -- 172.21.15.5:0/1822458753 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec34009600 con 0x7fec440b58f0 2024-05-20T18:31:47.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.964+0000 7fec49442640 1 -- 172.21.15.5:0/1822458753 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec340099d0 con 0x7fec440b58f0 2024-05-20T18:31:47.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.965+0000 7fec4c6a8640 1 -- 172.21.15.5:0/1822458753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec440b58f0 msgr2=0x7fec4406f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:47.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.965+0000 7fec4c6a8640 1 --2- 172.21.15.5:0/1822458753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec440b58f0 0x7fec4406f3d0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fec3400d150 tx=0x7fec3402c890 comp rx=0 tx=0).stop 2024-05-20T18:31:47.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.965+0000 7fec4c6a8640 1 -- 172.21.15.5:0/1822458753 shutdown_connections 2024-05-20T18:31:47.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.965+0000 7fec4c6a8640 1 --2- 172.21.15.5:0/1822458753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec440b58f0 0x7fec4406f3d0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:47.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.965+0000 7fec4c6a8640 1 --2- 172.21.15.5:0/1822458753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec440b47b0 0x7fec440afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:47.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.965+0000 7fec4c6a8640 1 -- 172.21.15.5:0/1822458753 >> 172.21.15.5:0/1822458753 conn(0x7fec440cd2a0 msgr2=0x7fec440cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:47.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.965+0000 7fec4c6a8640 1 -- 172.21.15.5:0/1822458753 shutdown_connections 2024-05-20T18:31:47.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.965+0000 7fec4c6a8640 1 -- 172.21.15.5:0/1822458753 wait complete. 2024-05-20T18:31:47.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.967+0000 7fec4c6a8640 1 Processor -- start 2024-05-20T18:31:47.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.967+0000 7fec4c6a8640 1 -- start start 2024-05-20T18:31:47.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.968+0000 7fec4c6a8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec4415d060 0x7fec44155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:47.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.968+0000 7fec4c6a8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec44152d10 0x7fec441efe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:47.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.968+0000 7fec4c6a8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec441f0390 0x7fec441577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:47.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.968+0000 7fec4c6a8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fec44159de0 con 0x7fec441f0390 2024-05-20T18:31:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.968+0000 7fec4c6a8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fec44159c60 con 0x7fec44152d10 2024-05-20T18:31:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.968+0000 7fec4c6a8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fec4406f910 con 0x7fec4415d060 2024-05-20T18:31:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.969+0000 7fec4ac45640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec441f0390 0x7fec441577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.969+0000 7fec4ac45640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec441f0390 0x7fec441577a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51762/0 (socket says 172.21.15.5:51762) 2024-05-20T18:31:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.969+0000 7fec4ac45640 1 -- 172.21.15.5:0/1533499925 learned_addr learned my addr 172.21.15.5:0/1533499925 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.969+0000 7fec4a444640 1 --2- 172.21.15.5:0/1533499925 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec4415d060 0x7fec44155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:47.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.969+0000 7fec49c43640 1 --2- 172.21.15.5:0/1533499925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec44152d10 0x7fec441efe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:47.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.969+0000 7fec4ac45640 1 -- 172.21.15.5:0/1533499925 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec4415d060 msgr2=0x7fec44155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:47.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.969+0000 7fec4ac45640 1 --2- 172.21.15.5:0/1533499925 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec4415d060 0x7fec44155ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:47.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.969+0000 7fec4ac45640 1 -- 172.21.15.5:0/1533499925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec44152d10 msgr2=0x7fec441efe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:47.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.969+0000 7fec4ac45640 1 --2- 172.21.15.5:0/1533499925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec44152d10 0x7fec441efe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:47.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.969+0000 7fec4ac45640 1 -- 172.21.15.5:0/1533499925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec441f4c60 con 0x7fec441f0390 2024-05-20T18:31:47.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.969+0000 7fec4ac45640 1 --2- 172.21.15.5:0/1533499925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec441f0390 0x7fec441577a0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fec34037040 tx=0x7fec34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:47.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.969+0000 7fec42ffd640 1 -- 172.21.15.5:0/1533499925 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34039050 con 0x7fec441f0390 2024-05-20T18:31:47.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.969+0000 7fec42ffd640 1 -- 172.21.15.5:0/1533499925 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec34045070 con 0x7fec441f0390 2024-05-20T18:31:47.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.969+0000 7fec4c6a8640 1 -- 172.21.15.5:0/1533499925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec44156020 con 0x7fec441f0390 2024-05-20T18:31:47.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.969+0000 7fec42ffd640 1 -- 172.21.15.5:0/1533499925 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec340346c0 con 0x7fec441f0390 2024-05-20T18:31:47.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.969+0000 7fec4c6a8640 1 -- 172.21.15.5:0/1533499925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec441f43c0 con 0x7fec441f0390 2024-05-20T18:31:47.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.971+0000 7fec4c6a8640 1 -- 172.21.15.5:0/1533499925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec441f3e40 con 0x7fec441f0390 2024-05-20T18:31:47.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.971+0000 7fec42ffd640 1 -- 172.21.15.5:0/1533499925 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fec34034860 con 0x7fec441f0390 2024-05-20T18:31:47.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.972+0000 7fec42ffd640 1 --2- 172.21.15.5:0/1533499925 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fec3006be30 0x7fec3006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:47.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.972+0000 7fec42ffd640 1 -- 172.21.15.5:0/1533499925 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7fec340ae060 con 0x7fec441f0390 2024-05-20T18:31:47.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.972+0000 7fec4a444640 1 --2- 172.21.15.5:0/1533499925 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fec3006be30 0x7fec3006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:47.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.972+0000 7fec4a444640 1 --2- 172.21.15.5:0/1533499925 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fec3006be30 0x7fec3006e340 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fec3c006fd0 tx=0x7fec3c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:47.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:47.976+0000 7fec42ffd640 1 -- 172.21.15.5:0/1533499925 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fec441f3e40 con 0x7fec441f0390 2024-05-20T18:31:48.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.160+0000 7fec4c6a8640 1 -- 172.21.15.5:0/1533499925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec4415c6b0 con 0x7fec441f0390 2024-05-20T18:31:48.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.161+0000 7fec42ffd640 1 -- 172.21.15.5:0/1533499925 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+16950 (secure 0 0 0) 0x7fec3407c300 con 0x7fec441f0390 2024-05-20T18:31:48.162 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:31:48.162 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":78,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:31:40.839398+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:31:27.896386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-20T18:31:29.146912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6836/791176355":"2024-05-21T18:31:40.839373+0000","172.21.15.5:6835/791176355":"2024-05-21T18:31:40.839373+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:31:48.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.163+0000 7fec4c6a8640 1 -- 172.21.15.5:0/1533499925 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fec3006be30 msgr2=0x7fec3006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:48.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.163+0000 7fec4c6a8640 1 --2- 172.21.15.5:0/1533499925 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fec3006be30 0x7fec3006e340 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fec3c006fd0 tx=0x7fec3c008040 comp rx=0 tx=0).stop 2024-05-20T18:31:48.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.163+0000 7fec4c6a8640 1 -- 172.21.15.5:0/1533499925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec441f0390 msgr2=0x7fec441577a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:48.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.163+0000 7fec4c6a8640 1 --2- 172.21.15.5:0/1533499925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec441f0390 0x7fec441577a0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fec34037040 tx=0x7fec34004a40 comp rx=0 tx=0).stop 2024-05-20T18:31:48.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.163+0000 7fec4c6a8640 1 -- 172.21.15.5:0/1533499925 shutdown_connections 2024-05-20T18:31:48.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.163+0000 7fec4c6a8640 1 --2- 172.21.15.5:0/1533499925 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fec3006be30 0x7fec3006e340 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:48.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.163+0000 7fec4c6a8640 1 --2- 172.21.15.5:0/1533499925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec441f0390 0x7fec441577a0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:48.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.163+0000 7fec4c6a8640 1 --2- 172.21.15.5:0/1533499925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec44152d10 0x7fec441efe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:48.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.163+0000 7fec4c6a8640 1 --2- 172.21.15.5:0/1533499925 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec4415d060 0x7fec44155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:48.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.163+0000 7fec4c6a8640 1 -- 172.21.15.5:0/1533499925 >> 172.21.15.5:0/1533499925 conn(0x7fec440cd2a0 msgr2=0x7fec44156b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:48.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.163+0000 7fec4c6a8640 1 -- 172.21.15.5:0/1533499925 shutdown_connections 2024-05-20T18:31:48.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.163+0000 7fec4c6a8640 1 -- 172.21.15.5:0/1533499925 wait complete. 2024-05-20T18:31:48.175 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:31:27.896386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:31:29.146912+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:31:48.176 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:31:48.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.293+0000 7f6c42bc3640 1 Processor -- start 2024-05-20T18:31:48.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.293+0000 7f6c42bc3640 1 -- start start 2024-05-20T18:31:48.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.295+0000 7f6c42bc3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c3c0d10e0 0x7f6c3c0cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:48.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.295+0000 7f6c42bc3640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c3c0ce2f0 con 0x7f6c3c0b7210 2024-05-20T18:31:48.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.295+0000 7f6c42bc3640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c3c0d1ac0 con 0x7f6c3c0d14f0 2024-05-20T18:31:48.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.295+0000 7f6c42bc3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6c3c0d1da0 con 0x7f6c3c0d10e0 2024-05-20T18:31:48.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.295+0000 7f6c41160640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f6c3c0b7210 0x7f6c3c0cdbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:48846/0 (socket says 172.21.15.5:48846) 2024-05-20T18:31:48.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.295+0000 7f6c41160640 1 -- 172.21.15.5:0/1643984779 learned_addr learned my addr 172.21.15.5:0/1643984779 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:48.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.295+0000 7f6c4095f640 1 --2- 172.21.15.5:0/1643984779 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c3c0d10e0 0x7f6c3c0cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:48.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.295+0000 7f6c3b7fe640 1 -- 172.21.15.5:0/1643984779 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4112616485 0 0) 0x7f6c3c0ce2f0 con 0x7f6c3c0b7210 2024-05-20T18:31:48.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.295+0000 7f6c3b7fe640 1 -- 172.21.15.5:0/1643984779 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c24003520 con 0x7f6c3c0b7210 2024-05-20T18:31:48.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.295+0000 7f6c3b7fe640 1 -- 172.21.15.5:0/1643984779 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 734701821 0 0) 0x7f6c24003520 con 0x7f6c3c0b7210 2024-05-20T18:31:48.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.296+0000 7f6c3b7fe640 1 -- 172.21.15.5:0/1643984779 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c3c0d10e0 msgr2=0x7f6c3c0cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:48.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.296+0000 7f6c3b7fe640 1 --2- 172.21.15.5:0/1643984779 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c3c0d10e0 0x7f6c3c0cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:48.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.296+0000 7f6c3b7fe640 1 -- 172.21.15.5:0/1643984779 >> v1:172.21.15.63:6789/0 conn(0x7f6c3c0d14f0 legacy=0x7f6c3c0cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:48.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.296+0000 7f6c3b7fe640 1 -- 172.21.15.5:0/1643984779 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c3c0d2080 con 0x7f6c3c0b7210 2024-05-20T18:31:48.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.296+0000 7f6c3b7fe640 1 -- 172.21.15.5:0/1643984779 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f6c340019f0 con 0x7f6c3c0b7210 2024-05-20T18:31:48.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.296+0000 7f6c3b7fe640 1 -- 172.21.15.5:0/1643984779 >> v1:172.21.15.5:6789/0 conn(0x7f6c3c0b7210 legacy=0x7f6c3c0cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:48.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.296+0000 7f6c3b7fe640 1 --2- 172.21.15.5:0/1643984779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c240039e0 0x7f6c24003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:48.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.296+0000 7f6c3b7fe640 1 --2- 172.21.15.5:0/1643984779 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c3c0d10e0 0x7f6c24004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:48.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.296+0000 7f6c3b7fe640 1 --2- 172.21.15.5:0/1643984779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c24004d40 0x7f6c24005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:48.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.296+0000 7f6c3b7fe640 1 -- 172.21.15.5:0/1643984779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c0d1da0 con 0x7f6c24004d40 2024-05-20T18:31:48.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.296+0000 7f6c3b7fe640 1 -- 172.21.15.5:0/1643984779 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2400baf0 con 0x7f6c240039e0 2024-05-20T18:31:48.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.296+0000 7f6c3b7fe640 1 -- 172.21.15.5:0/1643984779 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6c2400bcf0 con 0x7f6c3c0d10e0 2024-05-20T18:31:48.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.296+0000 7f6c41160640 1 --2- 172.21.15.5:0/1643984779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c24004d40 0x7f6c24005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:48.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.296+0000 7f6c4095f640 1 --2- 172.21.15.5:0/1643984779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c240039e0 0x7f6c24003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:48.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.296+0000 7f6c3bfff640 1 --2- 172.21.15.5:0/1643984779 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c3c0d10e0 0x7f6c24004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:48.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.296+0000 7f6c41160640 1 -- 172.21.15.5:0/1643984779 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c3c0d10e0 msgr2=0x7f6c24004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:48.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.296+0000 7f6c41160640 1 --2- 172.21.15.5:0/1643984779 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c3c0d10e0 0x7f6c24004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:48.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.296+0000 7f6c41160640 1 -- 172.21.15.5:0/1643984779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c240039e0 msgr2=0x7f6c24003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:48.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.296+0000 7f6c41160640 1 --2- 172.21.15.5:0/1643984779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c240039e0 0x7f6c24003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:48.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.296+0000 7f6c41160640 1 -- 172.21.15.5:0/1643984779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c24003520 con 0x7f6c24004d40 2024-05-20T18:31:48.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.297+0000 7f6c41160640 1 --2- 172.21.15.5:0/1643984779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c24004d40 0x7f6c24005150 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0d02d0 tx=0x7f6c24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923b946a422dc7d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:48.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.297+0000 7f6c3b7fe640 1 -- 172.21.15.5:0/1643984779 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3400a460 con 0x7f6c24004d40 2024-05-20T18:31:48.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.297+0000 7f6c3b7fe640 1 -- 172.21.15.5:0/1643984779 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c3402f070 con 0x7f6c24004d40 2024-05-20T18:31:48.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.297+0000 7f6c3b7fe640 1 -- 172.21.15.5:0/1643984779 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3402bc50 con 0x7f6c24004d40 2024-05-20T18:31:48.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.298+0000 7f6c42bc3640 1 -- 172.21.15.5:0/1643984779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c24004d40 msgr2=0x7f6c24005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:48.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.298+0000 7f6c42bc3640 1 --2- 172.21.15.5:0/1643984779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c24004d40 0x7f6c24005150 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0d02d0 tx=0x7f6c24005830 comp rx=0 tx=0).stop 2024-05-20T18:31:48.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.298+0000 7f6c41160640 1 -- 172.21.15.5:0/1643984779 reap_dead start 2024-05-20T18:31:48.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.298+0000 7f6c42bc3640 1 -- 172.21.15.5:0/1643984779 shutdown_connections 2024-05-20T18:31:48.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.298+0000 7f6c42bc3640 1 -- 172.21.15.5:0/1643984779 >> 172.21.15.5:0/1643984779 conn(0x7f6c3c0b44d0 msgr2=0x7f6c3c0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:48.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.298+0000 7f6c42bc3640 1 -- 172.21.15.5:0/1643984779 shutdown_connections 2024-05-20T18:31:48.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.298+0000 7f6c42bc3640 1 -- 172.21.15.5:0/1643984779 wait complete. 2024-05-20T18:31:48.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.300+0000 7f6c42bc3640 1 Processor -- start 2024-05-20T18:31:48.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.300+0000 7f6c42bc3640 1 -- start start 2024-05-20T18:31:48.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.301+0000 7f6c42bc3640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c3c1f7860 0x7f6c3c0c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:48.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.301+0000 7f6c42bc3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c3c1e62a0 0x7f6c3c1f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:48.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.301+0000 7f6c42bc3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c3c1e66b0 0x7f6c3c1f70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:48.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.301+0000 7f6c42bc3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c0b82c0 con 0x7f6c3c1e66b0 2024-05-20T18:31:48.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.301+0000 7f6c42bc3640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c0d2640 con 0x7f6c3c1f7860 2024-05-20T18:31:48.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.301+0000 7f6c42bc3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6c3c0b8440 con 0x7f6c3c1e62a0 2024-05-20T18:31:48.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.301+0000 7f6c41160640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c3c1e66b0 0x7f6c3c1f70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:48.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.301+0000 7f6c41160640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c3c1e66b0 0x7f6c3c1f70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51770/0 (socket says 172.21.15.5:51770) 2024-05-20T18:31:48.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.301+0000 7f6c41160640 1 -- 172.21.15.5:0/2125903748 learned_addr learned my addr 172.21.15.5:0/2125903748 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:48.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.301+0000 7f6c3bfff640 1 --2- 172.21.15.5:0/2125903748 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c3c1e62a0 0x7f6c3c1f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:48.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.301+0000 7f6c4095f640 1 --2- 172.21.15.5:0/2125903748 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c3c1f7860 0x7f6c3c0c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:48.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.302+0000 7f6c41160640 1 -- 172.21.15.5:0/2125903748 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c3c1e62a0 msgr2=0x7f6c3c1f6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:48.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.302+0000 7f6c41160640 1 --2- 172.21.15.5:0/2125903748 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c3c1e62a0 0x7f6c3c1f6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:48.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.302+0000 7f6c41160640 1 -- 172.21.15.5:0/2125903748 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c3c1f7860 msgr2=0x7f6c3c0c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:48.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.302+0000 7f6c41160640 1 --2- 172.21.15.5:0/2125903748 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c3c1f7860 0x7f6c3c0c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:48.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.302+0000 7f6c41160640 1 -- 172.21.15.5:0/2125903748 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c3c1e5a40 con 0x7f6c3c1e66b0 2024-05-20T18:31:48.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.302+0000 7f6c41160640 1 --2- 172.21.15.5:0/2125903748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c3c1e66b0 0x7f6c3c1f70b0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f6c3402b4a0 tx=0x7f6c3c0d1c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:48.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.302+0000 7f6c397fa640 1 -- 172.21.15.5:0/2125903748 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3400a460 con 0x7f6c3c1e66b0 2024-05-20T18:31:48.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.302+0000 7f6c397fa640 1 -- 172.21.15.5:0/2125903748 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c34034470 con 0x7f6c3c1e66b0 2024-05-20T18:31:48.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.302+0000 7f6c42bc3640 1 -- 172.21.15.5:0/2125903748 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c3c1e5ce0 con 0x7f6c3c1e66b0 2024-05-20T18:31:48.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.302+0000 7f6c397fa640 1 -- 172.21.15.5:0/2125903748 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34001fb0 con 0x7f6c3c1e66b0 2024-05-20T18:31:48.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.302+0000 7f6c42bc3640 1 -- 172.21.15.5:0/2125903748 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c3c0c87c0 con 0x7f6c3c1e66b0 2024-05-20T18:31:48.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.303+0000 7f6c42bc3640 1 -- 172.21.15.5:0/2125903748 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c3c075b00 con 0x7f6c3c1e66b0 2024-05-20T18:31:48.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.304+0000 7f6c397fa640 1 -- 172.21.15.5:0/2125903748 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6c34034820 con 0x7f6c3c1e66b0 2024-05-20T18:31:48.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.304+0000 7f6c397fa640 1 --2- 172.21.15.5:0/2125903748 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6c2806be30 0x7f6c2806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:48.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.304+0000 7f6c397fa640 1 -- 172.21.15.5:0/2125903748 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f6c340ab120 con 0x7f6c3c1e66b0 2024-05-20T18:31:48.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.304+0000 7f6c4095f640 1 --2- 172.21.15.5:0/2125903748 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6c2806be30 0x7f6c2806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:48.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.305+0000 7f6c4095f640 1 --2- 172.21.15.5:0/2125903748 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6c2806be30 0x7f6c2806e340 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c14ccf0 tx=0x7f6c2c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:48.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.309+0000 7f6c397fa640 1 -- 172.21.15.5:0/2125903748 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6c340798e0 con 0x7f6c3c1e66b0 2024-05-20T18:31:48.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.491+0000 7f6c42bc3640 1 -- 172.21.15.5:0/2125903748 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6c3c0d0220 con 0x7f6c3c1e66b0 2024-05-20T18:31:48.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.492+0000 7f6c397fa640 1 -- 172.21.15.5:0/2125903748 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7f6c340791e0 con 0x7f6c3c1e66b0 2024-05-20T18:31:48.492 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:31:48.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.494+0000 7f6c42bc3640 1 -- 172.21.15.5:0/2125903748 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6c2806be30 msgr2=0x7f6c2806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:48.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.494+0000 7f6c42bc3640 1 --2- 172.21.15.5:0/2125903748 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6c2806be30 0x7f6c2806e340 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c14ccf0 tx=0x7f6c2c008040 comp rx=0 tx=0).stop 2024-05-20T18:31:48.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.494+0000 7f6c42bc3640 1 -- 172.21.15.5:0/2125903748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c3c1e66b0 msgr2=0x7f6c3c1f70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:48.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.494+0000 7f6c42bc3640 1 --2- 172.21.15.5:0/2125903748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c3c1e66b0 0x7f6c3c1f70b0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f6c3402b4a0 tx=0x7f6c3c0d1c90 comp rx=0 tx=0).stop 2024-05-20T18:31:48.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.494+0000 7f6c42bc3640 1 -- 172.21.15.5:0/2125903748 shutdown_connections 2024-05-20T18:31:48.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.494+0000 7f6c42bc3640 1 --2- 172.21.15.5:0/2125903748 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6c2806be30 0x7f6c2806e340 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:48.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.494+0000 7f6c42bc3640 1 --2- 172.21.15.5:0/2125903748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c3c1e66b0 0x7f6c3c1f70b0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:48.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.494+0000 7f6c42bc3640 1 --2- 172.21.15.5:0/2125903748 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c3c1e62a0 0x7f6c3c1f6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:48.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.494+0000 7f6c42bc3640 1 --2- 172.21.15.5:0/2125903748 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c3c1f7860 0x7f6c3c0c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:48.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.494+0000 7f6c42bc3640 1 -- 172.21.15.5:0/2125903748 >> 172.21.15.5:0/2125903748 conn(0x7f6c3c0b44d0 msgr2=0x7f6c3c0b39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:48.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.494+0000 7f6c42bc3640 1 -- 172.21.15.5:0/2125903748 shutdown_connections 2024-05-20T18:31:48.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.494+0000 7f6c42bc3640 1 -- 172.21.15.5:0/2125903748 wait complete. 2024-05-20T18:31:48.506 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:31:48.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.623+0000 7f1c94b09640 1 Processor -- start 2024-05-20T18:31:48.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.623+0000 7f1c94b09640 1 -- start start 2024-05-20T18:31:48.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.624+0000 7f1c94b09640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c90162820 0x7f1c900b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:48.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.624+0000 7f1c94b09640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1c9006dae0 0x7f1c900b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:48.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.624+0000 7f1c94b09640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c900d0630 con 0x7f1c900b3e80 2024-05-20T18:31:48.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.624+0000 7f1c94b09640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1c900d39c0 con 0x7f1c9006dae0 2024-05-20T18:31:48.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.624+0000 7f1c94b09640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1c900d3ca0 con 0x7f1c90162820 2024-05-20T18:31:48.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.625+0000 7f1c8e59c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c90162820 0x7f1c900b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:48.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.625+0000 7f1c8ed9d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1c9006dae0 0x7f1c900b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:48.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.625+0000 7f1c8ed9d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1c9006dae0 0x7f1c900b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:34456/0 (socket says 172.21.15.5:34456) 2024-05-20T18:31:48.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.625+0000 7f1c8e59c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c90162820 0x7f1c900b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:56886/0 (socket says 172.21.15.5:56886) 2024-05-20T18:31:48.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.625+0000 7f1c8ed9d640 1 -- 172.21.15.5:0/2000497027 learned_addr learned my addr 172.21.15.5:0/2000497027 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:48.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.625+0000 7f1c8dd9b640 1 --1- 172.21.15.5:0/2000497027 >> v1:172.21.15.63:6789/0 conn(0x7f1c900b3e80 0x7f1c9006d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:37846/0 (socket says 172.21.15.5:37846) 2024-05-20T18:31:48.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.625+0000 7f1c8ed9d640 1 -- 172.21.15.5:0/2000497027 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c90162820 msgr2=0x7f1c900b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:48.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.625+0000 7f1c8ed9d640 1 --2- 172.21.15.5:0/2000497027 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c90162820 0x7f1c900b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:48.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.625+0000 7f1c8ed9d640 1 -- 172.21.15.5:0/2000497027 >> v1:172.21.15.63:6789/0 conn(0x7f1c900b3e80 legacy=0x7f1c9006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:48.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.625+0000 7f1c8d59a640 1 -- 172.21.15.5:0/2000497027 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3630001992 0 0) 0x7f1c900d0630 con 0x7f1c900b3e80 2024-05-20T18:31:48.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.625+0000 7f1c8ed9d640 1 -- 172.21.15.5:0/2000497027 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c900d3f80 con 0x7f1c9006dae0 2024-05-20T18:31:48.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.626+0000 7f1c8ed9d640 1 --2- 172.21.15.5:0/2000497027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c9006dae0 0x7f1c900b8190 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009e50 tx=0x7f1c8802a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17f8f67fd41b1b3e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:48.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.626+0000 7f1c8d59a640 1 -- 172.21.15.5:0/2000497027 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c88036070 con 0x7f1c9006dae0 2024-05-20T18:31:48.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.626+0000 7f1c8d59a640 1 -- 172.21.15.5:0/2000497027 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c88031600 con 0x7f1c9006dae0 2024-05-20T18:31:48.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.626+0000 7f1c8d59a640 1 -- 172.21.15.5:0/2000497027 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c880319d0 con 0x7f1c9006dae0 2024-05-20T18:31:48.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.627+0000 7f1c94b09640 1 -- 172.21.15.5:0/2000497027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c9006dae0 msgr2=0x7f1c900b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:48.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.627+0000 7f1c94b09640 1 --2- 172.21.15.5:0/2000497027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c9006dae0 0x7f1c900b8190 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009e50 tx=0x7f1c8802a710 comp rx=0 tx=0).stop 2024-05-20T18:31:48.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.627+0000 7f1c94b09640 1 -- 172.21.15.5:0/2000497027 shutdown_connections 2024-05-20T18:31:48.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.627+0000 7f1c94b09640 1 --2- 172.21.15.5:0/2000497027 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c9006dae0 0x7f1c900b8190 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:48.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.627+0000 7f1c94b09640 1 --2- 172.21.15.5:0/2000497027 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c90162820 0x7f1c900b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:48.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.627+0000 7f1c94b09640 1 -- 172.21.15.5:0/2000497027 >> 172.21.15.5:0/2000497027 conn(0x7f1c900d4bd0 msgr2=0x7f1c900d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:48.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.627+0000 7f1c94b09640 1 -- 172.21.15.5:0/2000497027 shutdown_connections 2024-05-20T18:31:48.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.627+0000 7f1c94b09640 1 -- 172.21.15.5:0/2000497027 wait complete. 2024-05-20T18:31:48.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.629+0000 7f1c94b09640 1 Processor -- start 2024-05-20T18:31:48.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.629+0000 7f1c94b09640 1 -- start start 2024-05-20T18:31:48.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.630+0000 7f1c94b09640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c900b3e80 0x7f1c901f4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.630+0000 7f1c94b09640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c90162820 0x7f1c90146810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.630+0000 7f1c94b09640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c9014a940 0x7f1c90146d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.630+0000 7f1c94b09640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c900d4780 con 0x7f1c90162820 2024-05-20T18:31:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.630+0000 7f1c94b09640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1c900d0300 con 0x7f1c9014a940 2024-05-20T18:31:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.630+0000 7f1c94b09640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1c900d4900 con 0x7f1c900b3e80 2024-05-20T18:31:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.631+0000 7f1c8dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c90162820 0x7f1c90146810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.631+0000 7f1c8dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c90162820 0x7f1c90146810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51784/0 (socket says 172.21.15.5:51784) 2024-05-20T18:31:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.631+0000 7f1c8dd9b640 1 -- 172.21.15.5:0/220995730 learned_addr learned my addr 172.21.15.5:0/220995730 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.631+0000 7f1c8e59c640 1 --2- 172.21.15.5:0/220995730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c900b3e80 0x7f1c901f4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.631+0000 7f1c8ed9d640 1 --2- 172.21.15.5:0/220995730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c9014a940 0x7f1c90146d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.631+0000 7f1c8dd9b640 1 -- 172.21.15.5:0/220995730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c900b3e80 msgr2=0x7f1c901f4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.631+0000 7f1c8dd9b640 1 --2- 172.21.15.5:0/220995730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c900b3e80 0x7f1c901f4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.631+0000 7f1c8dd9b640 1 -- 172.21.15.5:0/220995730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c9014a940 msgr2=0x7f1c90146d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.631+0000 7f1c8dd9b640 1 --2- 172.21.15.5:0/220995730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c9014a940 0x7f1c90146d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.631+0000 7f1c8dd9b640 1 -- 172.21.15.5:0/220995730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c900b52a0 con 0x7f1c90162820 2024-05-20T18:31:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.631+0000 7f1c8dd9b640 1 --2- 172.21.15.5:0/220995730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c90162820 0x7f1c90146810 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f1c7800c440 tx=0x7f1c7800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.631+0000 7f1c86ffd640 1 -- 172.21.15.5:0/220995730 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c78018070 con 0x7f1c90162820 2024-05-20T18:31:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.631+0000 7f1c86ffd640 1 -- 172.21.15.5:0/220995730 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c780074e0 con 0x7f1c90162820 2024-05-20T18:31:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.631+0000 7f1c94b09640 1 -- 172.21.15.5:0/220995730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c901f4840 con 0x7f1c90162820 2024-05-20T18:31:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.631+0000 7f1c86ffd640 1 -- 172.21.15.5:0/220995730 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c78004a20 con 0x7f1c90162820 2024-05-20T18:31:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.631+0000 7f1c94b09640 1 -- 172.21.15.5:0/220995730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c900b59f0 con 0x7f1c90162820 2024-05-20T18:31:48.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.632+0000 7f1c94b09640 1 -- 172.21.15.5:0/220995730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c90075b00 con 0x7f1c90162820 2024-05-20T18:31:48.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.633+0000 7f1c86ffd640 1 -- 172.21.15.5:0/220995730 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c7801b070 con 0x7f1c90162820 2024-05-20T18:31:48.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.634+0000 7f1c86ffd640 1 --2- 172.21.15.5:0/220995730 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c7006be60 0x7f1c7006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:48.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.634+0000 7f1c86ffd640 1 -- 172.21.15.5:0/220995730 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f1c7808a240 con 0x7f1c90162820 2024-05-20T18:31:48.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.634+0000 7f1c8e59c640 1 --2- 172.21.15.5:0/220995730 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c7006be60 0x7f1c7006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:48.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.634+0000 7f1c8e59c640 1 --2- 172.21.15.5:0/220995730 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c7006be60 0x7f1c7006e370 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1c80009a30 tx=0x7f1c80008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:48.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.638+0000 7f1c86ffd640 1 -- 172.21.15.5:0/220995730 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1c78058aa0 con 0x7f1c90162820 2024-05-20T18:31:48.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.824+0000 7f1c94b09640 1 -- 172.21.15.5:0/220995730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1c900b54a0 con 0x7f1c90162820 2024-05-20T18:31:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.824+0000 7f1c86ffd640 1 -- 172.21.15.5:0/220995730 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) ==== 104+0+11 (secure 0 0 0) 0x7f1c780583a0 con 0x7f1c90162820 2024-05-20T18:31:48.825 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:31:48.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.827+0000 7f1c94b09640 1 -- 172.21.15.5:0/220995730 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c7006be60 msgr2=0x7f1c7006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:48.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.827+0000 7f1c94b09640 1 --2- 172.21.15.5:0/220995730 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c7006be60 0x7f1c7006e370 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1c80009a30 tx=0x7f1c80008040 comp rx=0 tx=0).stop 2024-05-20T18:31:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.827+0000 7f1c94b09640 1 -- 172.21.15.5:0/220995730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c90162820 msgr2=0x7f1c90146810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.827+0000 7f1c94b09640 1 --2- 172.21.15.5:0/220995730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c90162820 0x7f1c90146810 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f1c7800c440 tx=0x7f1c7800c900 comp rx=0 tx=0).stop 2024-05-20T18:31:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.827+0000 7f1c94b09640 1 -- 172.21.15.5:0/220995730 shutdown_connections 2024-05-20T18:31:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.827+0000 7f1c94b09640 1 --2- 172.21.15.5:0/220995730 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c7006be60 0x7f1c7006e370 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.827+0000 7f1c94b09640 1 --2- 172.21.15.5:0/220995730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c9014a940 0x7f1c90146d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.827+0000 7f1c94b09640 1 --2- 172.21.15.5:0/220995730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c90162820 0x7f1c90146810 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.827+0000 7f1c94b09640 1 --2- 172.21.15.5:0/220995730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c900b3e80 0x7f1c901f4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.827+0000 7f1c94b09640 1 -- 172.21.15.5:0/220995730 >> 172.21.15.5:0/220995730 conn(0x7f1c900d4bd0 msgr2=0x7f1c90160cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.827+0000 7f1c94b09640 1 -- 172.21.15.5:0/220995730 shutdown_connections 2024-05-20T18:31:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.827+0000 7f1c94b09640 1 -- 172.21.15.5:0/220995730 wait complete. 2024-05-20T18:31:48.839 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:31:48.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.957+0000 7f7c1a31a640 1 Processor -- start 2024-05-20T18:31:48.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.957+0000 7f7c1a31a640 1 -- start start 2024-05-20T18:31:48.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.958+0000 7f7c1a31a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7c140af030 0x7f7c140af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:48.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.958+0000 7f7c1a31a640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c140af980 con 0x7f7c1406c180 2024-05-20T18:31:48.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.958+0000 7f7c1a31a640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c14160430 con 0x7f7c1415f700 2024-05-20T18:31:48.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.958+0000 7f7c1a31a640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7c1406bcf0 con 0x7f7c140af030 2024-05-20T18:31:48.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.958+0000 7f7c188b7640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7c1406c180 0x7f7c1406c590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:48856/0 (socket says 172.21.15.5:48856) 2024-05-20T18:31:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.958+0000 7f7c188b7640 1 -- 172.21.15.5:0/1670491251 learned_addr learned my addr 172.21.15.5:0/1670491251 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.958+0000 7f7c13fff640 1 --2- 172.21.15.5:0/1670491251 >> v2:172.21.15.63:3300/0 conn(0x7f7c140af030 0x7f7c140af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.959+0000 7f7c12ffd640 1 -- 172.21.15.5:0/1670491251 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3316121576 0 0) 0x7f7c0c001490 con 0x7f7c1406c180 2024-05-20T18:31:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.959+0000 7f7c12ffd640 1 -- 172.21.15.5:0/1670491251 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7bfc003520 con 0x7f7c1406c180 2024-05-20T18:31:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.959+0000 7f7c12ffd640 1 -- 172.21.15.5:0/1670491251 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2236488853 0 0) 0x7f7bfc003520 con 0x7f7c1406c180 2024-05-20T18:31:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.959+0000 7f7c13fff640 1 -- 172.21.15.5:0/1670491251 >> v1:172.21.15.63:6789/0 conn(0x7f7c1415f700 legacy=0x7f7c1415fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.959+0000 7f7c13fff640 1 -- 172.21.15.5:0/1670491251 >> v1:172.21.15.5:6789/0 conn(0x7f7c1406c180 legacy=0x7f7c1406c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:48.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.959+0000 7f7c13fff640 1 -- 172.21.15.5:0/1670491251 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c1406b6a0 con 0x7f7c140af030 2024-05-20T18:31:48.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.960+0000 7f7c13fff640 1 --2- 172.21.15.5:0/1670491251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c140af030 0x7f7c140af440 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7c040097a0 tx=0x7f7c0402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=545c542421c01b89 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:48.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.960+0000 7f7c12ffd640 1 -- 172.21.15.5:0/1670491251 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04037070 con 0x7f7c140af030 2024-05-20T18:31:48.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.960+0000 7f7c12ffd640 1 -- 172.21.15.5:0/1670491251 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c0402ac50 con 0x7f7c140af030 2024-05-20T18:31:48.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.960+0000 7f7c12ffd640 1 -- 172.21.15.5:0/1670491251 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c001be0 con 0x7f7c140af030 2024-05-20T18:31:48.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.960+0000 7f7c1a31a640 1 -- 172.21.15.5:0/1670491251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c140af030 msgr2=0x7f7c140af440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:48.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.960+0000 7f7c1a31a640 1 --2- 172.21.15.5:0/1670491251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c140af030 0x7f7c140af440 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7c040097a0 tx=0x7f7c0402a1c0 comp rx=0 tx=0).stop 2024-05-20T18:31:48.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.961+0000 7f7c1a31a640 1 -- 172.21.15.5:0/1670491251 shutdown_connections 2024-05-20T18:31:48.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.961+0000 7f7c1a31a640 1 --2- 172.21.15.5:0/1670491251 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c140af030 0x7f7c140af440 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:48.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.961+0000 7f7c1a31a640 1 -- 172.21.15.5:0/1670491251 >> 172.21.15.5:0/1670491251 conn(0x7f7c140b6620 msgr2=0x7f7c140b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:48.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.961+0000 7f7c1a31a640 1 -- 172.21.15.5:0/1670491251 shutdown_connections 2024-05-20T18:31:48.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.961+0000 7f7c1a31a640 1 -- 172.21.15.5:0/1670491251 wait complete. 2024-05-20T18:31:48.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.962+0000 7f7c1a31a640 1 Processor -- start 2024-05-20T18:31:48.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.963+0000 7f7c1a31a640 1 -- start start 2024-05-20T18:31:48.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.964+0000 7f7c1a31a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c1406c180 0x7f7c1409bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:48.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.964+0000 7f7c1a31a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c140af030 0x7f7c1415a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:48.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.964+0000 7f7c1a31a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c1415f700 0x7f7c1415adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:48.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.964+0000 7f7c1a31a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1406cca0 con 0x7f7c140af030 2024-05-20T18:31:48.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.964+0000 7f7c1a31a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7c14159aa0 con 0x7f7c1406c180 2024-05-20T18:31:48.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.964+0000 7f7c1a31a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7c1406ce20 con 0x7f7c1415f700 2024-05-20T18:31:48.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.964+0000 7f7c137fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c140af030 0x7f7c1415a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:48.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.964+0000 7f7c137fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c140af030 0x7f7c1415a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51790/0 (socket says 172.21.15.5:51790) 2024-05-20T18:31:48.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.964+0000 7f7c137fe640 1 -- 172.21.15.5:0/767434668 learned_addr learned my addr 172.21.15.5:0/767434668 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:48.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.964+0000 7f7c188b7640 1 --2- 172.21.15.5:0/767434668 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c1415f700 0x7f7c1415adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:48.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.964+0000 7f7c13fff640 1 --2- 172.21.15.5:0/767434668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c1406c180 0x7f7c1409bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:48.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.964+0000 7f7c137fe640 1 -- 172.21.15.5:0/767434668 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c1415f700 msgr2=0x7f7c1415adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:48.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.964+0000 7f7c137fe640 1 --2- 172.21.15.5:0/767434668 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c1415f700 0x7f7c1415adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:48.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.964+0000 7f7c137fe640 1 -- 172.21.15.5:0/767434668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c1406c180 msgr2=0x7f7c1409bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:48.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.964+0000 7f7c137fe640 1 --2- 172.21.15.5:0/767434668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c1406c180 0x7f7c1409bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:48.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.964+0000 7f7c137fe640 1 -- 172.21.15.5:0/767434668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c1415a3a0 con 0x7f7c140af030 2024-05-20T18:31:48.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.964+0000 7f7c13fff640 1 --2- 172.21.15.5:0/767434668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c1406c180 0x7f7c1409bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:48.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.965+0000 7f7c137fe640 1 --2- 172.21.15.5:0/767434668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c140af030 0x7f7c1415a880 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800c430 tx=0x7f7c0800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:48.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.965+0000 7f7c10ff9640 1 -- 172.21.15.5:0/767434668 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c08017070 con 0x7f7c140af030 2024-05-20T18:31:48.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.965+0000 7f7c10ff9640 1 -- 172.21.15.5:0/767434668 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c080052d0 con 0x7f7c140af030 2024-05-20T18:31:48.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.965+0000 7f7c10ff9640 1 -- 172.21.15.5:0/767434668 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c08003830 con 0x7f7c140af030 2024-05-20T18:31:48.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.965+0000 7f7c1a31a640 1 -- 172.21.15.5:0/767434668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c1415e230 con 0x7f7c140af030 2024-05-20T18:31:48.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.965+0000 7f7c1a31a640 1 -- 172.21.15.5:0/767434668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c1415e7f0 con 0x7f7c140af030 2024-05-20T18:31:48.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.966+0000 7f7c1a31a640 1 -- 172.21.15.5:0/767434668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c14075b00 con 0x7f7c140af030 2024-05-20T18:31:48.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.967+0000 7f7c10ff9640 1 -- 172.21.15.5:0/767434668 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7c0801a070 con 0x7f7c140af030 2024-05-20T18:31:48.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.967+0000 7f7c10ff9640 1 --2- 172.21.15.5:0/767434668 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7c0006be60 0x7f7c0006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:48.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.967+0000 7f7c10ff9640 1 -- 172.21.15.5:0/767434668 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f7c0808a420 con 0x7f7c140af030 2024-05-20T18:31:48.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.967+0000 7f7c13fff640 1 --2- 172.21.15.5:0/767434668 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7c0006be60 0x7f7c0006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:48.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.968+0000 7f7c13fff640 1 --2- 172.21.15.5:0/767434668 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7c0006be60 0x7f7c0006e370 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7c04034000 tx=0x7f7c0402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:48.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:48.972+0000 7f7c10ff9640 1 -- 172.21.15.5:0/767434668 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7c08058b60 con 0x7f7c140af030 2024-05-20T18:31:49.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.155+0000 7f7c1a31a640 1 -- 172.21.15.5:0/767434668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7c1415dfc0 con 0x7f7c140af030 2024-05-20T18:31:49.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.155+0000 7f7c10ff9640 1 -- 172.21.15.5:0/767434668 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) ==== 100+0+11 (secure 0 0 0) 0x7f7c08058460 con 0x7f7c140af030 2024-05-20T18:31:49.156 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:31:49.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.157+0000 7f7c1a31a640 1 -- 172.21.15.5:0/767434668 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7c0006be60 msgr2=0x7f7c0006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:49.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.157+0000 7f7c1a31a640 1 --2- 172.21.15.5:0/767434668 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7c0006be60 0x7f7c0006e370 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7c04034000 tx=0x7f7c0402e000 comp rx=0 tx=0).stop 2024-05-20T18:31:49.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.157+0000 7f7c1a31a640 1 -- 172.21.15.5:0/767434668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c140af030 msgr2=0x7f7c1415a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:49.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.157+0000 7f7c1a31a640 1 --2- 172.21.15.5:0/767434668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c140af030 0x7f7c1415a880 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800c430 tx=0x7f7c0800c8f0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.157+0000 7f7c1a31a640 1 -- 172.21.15.5:0/767434668 shutdown_connections 2024-05-20T18:31:49.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.157+0000 7f7c1a31a640 1 --2- 172.21.15.5:0/767434668 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7c0006be60 0x7f7c0006e370 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.157+0000 7f7c1a31a640 1 --2- 172.21.15.5:0/767434668 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c1415f700 0x7f7c1415adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.157+0000 7f7c1a31a640 1 --2- 172.21.15.5:0/767434668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c140af030 0x7f7c1415a880 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.157+0000 7f7c1a31a640 1 --2- 172.21.15.5:0/767434668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c1406c180 0x7f7c1409bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.157+0000 7f7c1a31a640 1 -- 172.21.15.5:0/767434668 >> 172.21.15.5:0/767434668 conn(0x7f7c140b6620 msgr2=0x7f7c1415fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:49.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.157+0000 7f7c1a31a640 1 -- 172.21.15.5:0/767434668 shutdown_connections 2024-05-20T18:31:49.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.157+0000 7f7c1a31a640 1 -- 172.21.15.5:0/767434668 wait complete. 2024-05-20T18:31:49.169 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T18:31:49.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.285+0000 7f7956026640 1 Processor -- start 2024-05-20T18:31:49.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.285+0000 7f7956026640 1 -- start start 2024-05-20T18:31:49.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.286+0000 7f7956026640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f79500b47b0 0x7f79500afad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.286+0000 7f7956026640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79500b39f0 con 0x7f79500b58f0 2024-05-20T18:31:49.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.286+0000 7f7956026640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79500b3cd0 con 0x7f795015d060 2024-05-20T18:31:49.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.286+0000 7f7956026640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f79500b3fb0 con 0x7f79500b47b0 2024-05-20T18:31:49.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.286+0000 7f794ffff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f79500b58f0 0x7f795006f3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:48864/0 (socket says 172.21.15.5:48864) 2024-05-20T18:31:49.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.286+0000 7f794ffff640 1 -- 172.21.15.5:0/3945854525 learned_addr learned my addr 172.21.15.5:0/3945854525 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:49.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.286+0000 7f794f7fe640 1 --2- 172.21.15.5:0/3945854525 >> v2:172.21.15.63:3300/0 conn(0x7f79500b47b0 0x7f79500afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.286+0000 7f794effd640 1 -- 172.21.15.5:0/3945854525 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1918980620 0 0) 0x7f79500b39f0 con 0x7f79500b58f0 2024-05-20T18:31:49.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.286+0000 7f794effd640 1 -- 172.21.15.5:0/3945854525 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f793c003520 con 0x7f79500b58f0 2024-05-20T18:31:49.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.287+0000 7f794effd640 1 -- 172.21.15.5:0/3945854525 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 375557679 0 0) 0x7f793c003520 con 0x7f79500b58f0 2024-05-20T18:31:49.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.287+0000 7f794effd640 1 -- 172.21.15.5:0/3945854525 >> v2:172.21.15.63:3300/0 conn(0x7f79500b47b0 msgr2=0x7f79500afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:49.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.287+0000 7f794effd640 1 --2- 172.21.15.5:0/3945854525 >> v2:172.21.15.63:3300/0 conn(0x7f79500b47b0 0x7f79500afad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.287+0000 7f794effd640 1 -- 172.21.15.5:0/3945854525 >> v1:172.21.15.63:6789/0 conn(0x7f795015d060 legacy=0x7f79500b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:49.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.287+0000 7f794effd640 1 -- 172.21.15.5:0/3945854525 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79500b4290 con 0x7f79500b58f0 2024-05-20T18:31:49.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.287+0000 7f794effd640 1 -- 172.21.15.5:0/3945854525 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f79480019f0 con 0x7f79500b58f0 2024-05-20T18:31:49.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.287+0000 7f794effd640 1 -- 172.21.15.5:0/3945854525 >> v1:172.21.15.5:6789/0 conn(0x7f79500b58f0 legacy=0x7f795006f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:49.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.287+0000 7f794effd640 1 --2- 172.21.15.5:0/3945854525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f793c0039b0 0x7f793c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.287+0000 7f794effd640 1 --2- 172.21.15.5:0/3945854525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f793c004800 0x7f793c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.287+0000 7f794effd640 1 --2- 172.21.15.5:0/3945854525 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f793c005250 0x7f793c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.287+0000 7f794effd640 1 -- 172.21.15.5:0/3945854525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f79500b3fb0 con 0x7f793c004800 2024-05-20T18:31:49.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.287+0000 7f794effd640 1 -- 172.21.15.5:0/3945854525 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f793c00c1e0 con 0x7f793c0039b0 2024-05-20T18:31:49.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.287+0000 7f794effd640 1 -- 172.21.15.5:0/3945854525 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f793c00c3e0 con 0x7f793c005250 2024-05-20T18:31:49.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.287+0000 7f7947fff640 1 --2- 172.21.15.5:0/3945854525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f793c004800 0x7f793c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.287+0000 7f794f7fe640 1 --2- 172.21.15.5:0/3945854525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f793c0039b0 0x7f793c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.287+0000 7f794ffff640 1 --2- 172.21.15.5:0/3945854525 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f793c005250 0x7f793c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.287+0000 7f7947fff640 1 -- 172.21.15.5:0/3945854525 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f793c005250 msgr2=0x7f793c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.287+0000 7f7947fff640 1 --2- 172.21.15.5:0/3945854525 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f793c005250 0x7f793c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.287+0000 7f7947fff640 1 -- 172.21.15.5:0/3945854525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f793c0039b0 msgr2=0x7f793c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.287+0000 7f7947fff640 1 --2- 172.21.15.5:0/3945854525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f793c0039b0 0x7f793c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.287+0000 7f7947fff640 1 -- 172.21.15.5:0/3945854525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f793c003520 con 0x7f793c004800 2024-05-20T18:31:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.287+0000 7f794ffff640 1 -- 172.21.15.5:0/3945854525 reap_dead start 2024-05-20T18:31:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.288+0000 7f7947fff640 1 --2- 172.21.15.5:0/3945854525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f793c004800 0x7f793c004c10 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f7950071bc0 tx=0x7f793c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bfed39dbdf3c683 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.288+0000 7f794effd640 1 -- 172.21.15.5:0/3945854525 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7938001370 con 0x7f793c004800 2024-05-20T18:31:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.288+0000 7f794effd640 1 -- 172.21.15.5:0/3945854525 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79380052d0 con 0x7f793c004800 2024-05-20T18:31:49.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.288+0000 7f794effd640 1 -- 172.21.15.5:0/3945854525 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793802ab90 con 0x7f793c004800 2024-05-20T18:31:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.289+0000 7f7956026640 1 -- 172.21.15.5:0/3945854525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f793c004800 msgr2=0x7f793c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.289+0000 7f7956026640 1 --2- 172.21.15.5:0/3945854525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f793c004800 0x7f793c004c10 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f7950071bc0 tx=0x7f793c005830 comp rx=0 tx=0).stop 2024-05-20T18:31:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.289+0000 7f7956026640 1 -- 172.21.15.5:0/3945854525 shutdown_connections 2024-05-20T18:31:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.289+0000 7f7956026640 1 --2- 172.21.15.5:0/3945854525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f793c004800 0x7f793c004c10 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.289+0000 7f7956026640 1 -- 172.21.15.5:0/3945854525 >> 172.21.15.5:0/3945854525 conn(0x7f79500cd2a0 msgr2=0x7f79500cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.289+0000 7f7956026640 1 -- 172.21.15.5:0/3945854525 shutdown_connections 2024-05-20T18:31:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.289+0000 7f7956026640 1 -- 172.21.15.5:0/3945854525 wait complete. 2024-05-20T18:31:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.291+0000 7f7956026640 1 Processor -- start 2024-05-20T18:31:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.291+0000 7f7956026640 1 -- start start 2024-05-20T18:31:49.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.292+0000 7f7956026640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f793c004800 0x7f79500ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.292+0000 7f7956026640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7950157550 0x7f79501501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.292+0000 7f7956026640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79500cc300 0x7f79500cc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.292+0000 7f7956026640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f79500b47b0 con 0x7f7950157550 2024-05-20T18:31:49.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.292+0000 7f7956026640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7950159e70 con 0x7f793c004800 2024-05-20T18:31:49.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.292+0000 7f7956026640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f79500b4930 con 0x7f79500cc300 2024-05-20T18:31:49.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.292+0000 7f7947fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7950157550 0x7f79501501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.292+0000 7f7947fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7950157550 0x7f79501501b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51812/0 (socket says 172.21.15.5:51812) 2024-05-20T18:31:49.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.292+0000 7f7947fff640 1 -- 172.21.15.5:0/1686175280 learned_addr learned my addr 172.21.15.5:0/1686175280 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:49.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.293+0000 7f794ffff640 1 --2- 172.21.15.5:0/1686175280 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79500cc300 0x7f79500cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.293+0000 7f794f7fe640 1 --2- 172.21.15.5:0/1686175280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f793c004800 0x7f79500ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.293+0000 7f7947fff640 1 -- 172.21.15.5:0/1686175280 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79500cc300 msgr2=0x7f79500cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:49.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.293+0000 7f7947fff640 1 --2- 172.21.15.5:0/1686175280 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79500cc300 0x7f79500cc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.293+0000 7f7947fff640 1 -- 172.21.15.5:0/1686175280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f793c004800 msgr2=0x7f79500ca5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:49.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.293+0000 7f7947fff640 1 --2- 172.21.15.5:0/1686175280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f793c004800 0x7f79500ca5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.293+0000 7f7947fff640 1 -- 172.21.15.5:0/1686175280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79500ccc50 con 0x7f7950157550 2024-05-20T18:31:49.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.293+0000 7f7947fff640 1 --2- 172.21.15.5:0/1686175280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7950157550 0x7f79501501b0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f79380017e0 tx=0x7f7938001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:49.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.293+0000 7f794cff9640 1 -- 172.21.15.5:0/1686175280 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7938037040 con 0x7f7950157550 2024-05-20T18:31:49.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.293+0000 7f794cff9640 1 -- 172.21.15.5:0/1686175280 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7938041070 con 0x7f7950157550 2024-05-20T18:31:49.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.293+0000 7f794cff9640 1 -- 172.21.15.5:0/1686175280 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7938007a30 con 0x7f7950157550 2024-05-20T18:31:49.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.293+0000 7f7956026640 1 -- 172.21.15.5:0/1686175280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f795015e750 con 0x7f7950157550 2024-05-20T18:31:49.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.293+0000 7f7956026640 1 -- 172.21.15.5:0/1686175280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f795015ed10 con 0x7f7950157550 2024-05-20T18:31:49.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.294+0000 7f7956026640 1 -- 172.21.15.5:0/1686175280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79500cac20 con 0x7f7950157550 2024-05-20T18:31:49.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.295+0000 7f794cff9640 1 -- 172.21.15.5:0/1686175280 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f79380052d0 con 0x7f7950157550 2024-05-20T18:31:49.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.295+0000 7f794cff9640 1 --2- 172.21.15.5:0/1686175280 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f793006be30 0x7f793006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.295+0000 7f794cff9640 1 -- 172.21.15.5:0/1686175280 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f79380aae30 con 0x7f7950157550 2024-05-20T18:31:49.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.295+0000 7f794f7fe640 1 --2- 172.21.15.5:0/1686175280 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f793006be30 0x7f793006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.296+0000 7f794f7fe640 1 --2- 172.21.15.5:0/1686175280 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f793006be30 0x7f793006e340 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7950071620 tx=0x7f7940008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:49.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.300+0000 7f794cff9640 1 -- 172.21.15.5:0/1686175280 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f79500ccc50 con 0x7f7950157550 2024-05-20T18:31:49.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.495+0000 7f7956026640 1 -- 172.21.15.5:0/1686175280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7950158ef0 con 0x7f7950157550 2024-05-20T18:31:49.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.496+0000 7f794cff9640 1 -- 172.21.15.5:0/1686175280 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v78) ==== 80+0+118 (secure 0 0 0) 0x7f7950158ef0 con 0x7f7950157550 2024-05-20T18:31:49.496 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6836/791176355 2024-05-21T18:31:40.839373+0000 2024-05-20T18:31:49.496 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6835/791176355 2024-05-21T18:31:40.839373+0000 2024-05-20T18:31:49.497 INFO:teuthology.orchestra.run.smithi005.stderr:listed 2 entries 2024-05-20T18:31:49.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.498+0000 7f7956026640 1 -- 172.21.15.5:0/1686175280 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f793006be30 msgr2=0x7f793006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:49.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.498+0000 7f7956026640 1 --2- 172.21.15.5:0/1686175280 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f793006be30 0x7f793006e340 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7950071620 tx=0x7f7940008040 comp rx=0 tx=0).stop 2024-05-20T18:31:49.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.498+0000 7f7956026640 1 -- 172.21.15.5:0/1686175280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7950157550 msgr2=0x7f79501501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:49.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.498+0000 7f7956026640 1 --2- 172.21.15.5:0/1686175280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7950157550 0x7f79501501b0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f79380017e0 tx=0x7f7938001120 comp rx=0 tx=0).stop 2024-05-20T18:31:49.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.498+0000 7f7956026640 1 -- 172.21.15.5:0/1686175280 shutdown_connections 2024-05-20T18:31:49.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.498+0000 7f7956026640 1 --2- 172.21.15.5:0/1686175280 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f793006be30 0x7f793006e340 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.498+0000 7f7956026640 1 --2- 172.21.15.5:0/1686175280 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79500cc300 0x7f79500cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.498+0000 7f7956026640 1 --2- 172.21.15.5:0/1686175280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7950157550 0x7f79501501b0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.498+0000 7f7956026640 1 --2- 172.21.15.5:0/1686175280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f793c004800 0x7f79500ca5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.498+0000 7f7956026640 1 -- 172.21.15.5:0/1686175280 >> 172.21.15.5:0/1686175280 conn(0x7f79500cd2a0 msgr2=0x7f795006fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:49.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.498+0000 7f7956026640 1 -- 172.21.15.5:0/1686175280 shutdown_connections 2024-05-20T18:31:49.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.498+0000 7f7956026640 1 -- 172.21.15.5:0/1686175280 wait complete. 2024-05-20T18:31:49.509 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-20T18:31:49.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:31:49.510 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-20T18:31:49.539 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:31:49.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:31:49.540 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-20T18:31:49.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:31:49.596 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir recursive 2024-05-20T18:31:49.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.754+0000 7f20570f9640 1 Processor -- start 2024-05-20T18:31:49.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.754+0000 7f20570f9640 1 -- start start 2024-05-20T18:31:49.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.755+0000 7f20570f9640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f20500b3860 0x7f20500d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.755+0000 7f20570f9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f205008bd20 0x7f205008c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.755+0000 7f20570f9640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f205008c670 con 0x7f205006d870 2024-05-20T18:31:49.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.755+0000 7f20570f9640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f20500d0450 con 0x7f205008bd20 2024-05-20T18:31:49.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.755+0000 7f20570f9640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f20500d3b10 con 0x7f20500b3860 2024-05-20T18:31:49.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.756+0000 7f2055696640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f205008bd20 0x7f205008c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.756+0000 7f204ffff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f205006d870 0x7f205015f230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:48880/0 (socket says 172.21.15.5:48880) 2024-05-20T18:31:49.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.756+0000 7f204ffff640 1 -- 172.21.15.5:0/298862922 learned_addr learned my addr 172.21.15.5:0/298862922 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:49.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.756+0000 7f2054e95640 1 --2- 172.21.15.5:0/298862922 >> v2:172.21.15.63:3300/0 conn(0x7f20500b3860 0x7f20500d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.756+0000 7f2055696640 1 -- 172.21.15.5:0/298862922 >> v2:172.21.15.63:3300/0 conn(0x7f20500b3860 msgr2=0x7f20500d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:49.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.756+0000 7f2055696640 1 --2- 172.21.15.5:0/298862922 >> v2:172.21.15.63:3300/0 conn(0x7f20500b3860 0x7f20500d45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.756+0000 7f2055696640 1 -- 172.21.15.5:0/298862922 >> v1:172.21.15.5:6789/0 conn(0x7f205006d870 legacy=0x7f205015f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:49.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.756+0000 7f204f7fe640 1 -- 172.21.15.5:0/298862922 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1500669312 0 0) 0x7f2044001110 con 0x7f205006d870 2024-05-20T18:31:49.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.756+0000 7f2055696640 1 -- 172.21.15.5:0/298862922 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20500d3df0 con 0x7f205008bd20 2024-05-20T18:31:49.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.757+0000 7f2055696640 1 --2- 172.21.15.5:0/298862922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f205008bd20 0x7f205008c130 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f205008dcf0 tx=0x7f204802a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e2032378f1c697 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:49.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.757+0000 7f204f7fe640 1 -- 172.21.15.5:0/298862922 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2048007740 con 0x7f205008bd20 2024-05-20T18:31:49.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.757+0000 7f204f7fe640 1 -- 172.21.15.5:0/298862922 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20480078e0 con 0x7f205008bd20 2024-05-20T18:31:49.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.757+0000 7f204f7fe640 1 -- 172.21.15.5:0/298862922 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2048007cb0 con 0x7f205008bd20 2024-05-20T18:31:49.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.758+0000 7f20570f9640 1 -- 172.21.15.5:0/298862922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f205008bd20 msgr2=0x7f205008c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:49.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.758+0000 7f20570f9640 1 --2- 172.21.15.5:0/298862922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f205008bd20 0x7f205008c130 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f205008dcf0 tx=0x7f204802a560 comp rx=0 tx=0).stop 2024-05-20T18:31:49.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.758+0000 7f20570f9640 1 -- 172.21.15.5:0/298862922 shutdown_connections 2024-05-20T18:31:49.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.758+0000 7f20570f9640 1 --2- 172.21.15.5:0/298862922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f205008bd20 0x7f205008c130 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.758+0000 7f20570f9640 1 --2- 172.21.15.5:0/298862922 >> v2:172.21.15.63:3300/0 conn(0x7f20500b3860 0x7f20500d45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.758+0000 7f20570f9640 1 -- 172.21.15.5:0/298862922 >> 172.21.15.5:0/298862922 conn(0x7f20500b6d10 msgr2=0x7f205006d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:49.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.758+0000 7f20570f9640 1 -- 172.21.15.5:0/298862922 shutdown_connections 2024-05-20T18:31:49.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.758+0000 7f20570f9640 1 -- 172.21.15.5:0/298862922 wait complete. 2024-05-20T18:31:49.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.759+0000 7f20570f9640 1 Processor -- start 2024-05-20T18:31:49.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.760+0000 7f20570f9640 1 -- start start 2024-05-20T18:31:49.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.761+0000 7f20570f9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f205006d870 0x7f2050155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.761+0000 7f20570f9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f205008bd20 0x7f20500ce150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.761+0000 7f20570f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20500b3860 0x7f2050159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.761+0000 7f20570f9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f20500b81b0 con 0x7f20500b3860 2024-05-20T18:31:49.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.761+0000 7f20570f9640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f20500d0730 con 0x7f205006d870 2024-05-20T18:31:49.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.761+0000 7f20570f9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f20500b8330 con 0x7f205008bd20 2024-05-20T18:31:49.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.761+0000 7f2055696640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20500b3860 0x7f2050159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.761+0000 7f2055696640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20500b3860 0x7f2050159360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51836/0 (socket says 172.21.15.5:51836) 2024-05-20T18:31:49.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.761+0000 7f2055696640 1 -- 172.21.15.5:0/4158230895 learned_addr learned my addr 172.21.15.5:0/4158230895 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:49.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.761+0000 7f204ffff640 1 --2- 172.21.15.5:0/4158230895 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f205008bd20 0x7f20500ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.761+0000 7f2054e95640 1 --2- 172.21.15.5:0/4158230895 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f205006d870 0x7f2050155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.761+0000 7f2055696640 1 -- 172.21.15.5:0/4158230895 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f205008bd20 msgr2=0x7f20500ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:49.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.761+0000 7f2055696640 1 --2- 172.21.15.5:0/4158230895 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f205008bd20 0x7f20500ce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.761+0000 7f2055696640 1 -- 172.21.15.5:0/4158230895 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f205006d870 msgr2=0x7f2050155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:49.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.761+0000 7f2055696640 1 --2- 172.21.15.5:0/4158230895 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f205006d870 0x7f2050155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.761+0000 7f2055696640 1 -- 172.21.15.5:0/4158230895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20500ceec0 con 0x7f20500b3860 2024-05-20T18:31:49.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.761+0000 7f2054e95640 1 --2- 172.21.15.5:0/4158230895 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f205006d870 0x7f2050155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:49.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.762+0000 7f2055696640 1 --2- 172.21.15.5:0/4158230895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20500b3860 0x7f2050159360 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f2048009e50 tx=0x7f2048004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:49.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.762+0000 7f204d7fa640 1 -- 172.21.15.5:0/4158230895 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204803f070 con 0x7f20500b3860 2024-05-20T18:31:49.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.762+0000 7f204d7fa640 1 -- 172.21.15.5:0/4158230895 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2048031bf0 con 0x7f20500b3860 2024-05-20T18:31:49.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.762+0000 7f20570f9640 1 -- 172.21.15.5:0/4158230895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20500cf1a0 con 0x7f20500b3860 2024-05-20T18:31:49.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.762+0000 7f204d7fa640 1 -- 172.21.15.5:0/4158230895 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2048044460 con 0x7f20500b3860 2024-05-20T18:31:49.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.762+0000 7f20570f9640 1 -- 172.21.15.5:0/4158230895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20500c9770 con 0x7f20500b3860 2024-05-20T18:31:49.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.764+0000 7f204d7fa640 1 -- 172.21.15.5:0/4158230895 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f204803b410 con 0x7f20500b3860 2024-05-20T18:31:49.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.764+0000 7f204d7fa640 1 --2- 172.21.15.5:0/4158230895 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f203c06be60 0x7f203c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.765+0000 7f2054e95640 1 --2- 172.21.15.5:0/4158230895 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f203c06be60 0x7f203c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.765+0000 7f204d7fa640 1 -- 172.21.15.5:0/4158230895 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f20480ac970 con 0x7f20500b3860 2024-05-20T18:31:49.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.765+0000 7f2054e95640 1 --2- 172.21.15.5:0/4158230895 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f203c06be60 0x7f203c06e370 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2040006fd0 tx=0x7f2040008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:49.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.766+0000 7f205df9c740 1 Processor -- start 2024-05-20T18:31:49.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.766+0000 7f205df9c740 1 -- start start 2024-05-20T18:31:49.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.767+0000 7f205df9c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74da7a5b0 0x55e74de45970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.767+0000 7f205df9c740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e74da516f0 0x55e74de606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.767+0000 7f205df9c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e74da598d0 0x55e74de60be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.767+0000 7f205df9c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e74dd7ef50 con 0x55e74da7a5b0 2024-05-20T18:31:49.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.767+0000 7f205df9c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e74dd26510 con 0x55e74da598d0 2024-05-20T18:31:49.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.767+0000 7f205df9c740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55e74dd7af10 con 0x55e74da516f0 2024-05-20T18:31:49.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.767+0000 7f2054e95640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74da7a5b0 0x55e74de45970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.767+0000 7f2054e95640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74da7a5b0 0x55e74de45970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51846/0 (socket says 172.21.15.5:51846) 2024-05-20T18:31:49.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.767+0000 7f2054e95640 1 -- 172.21.15.5:0/3776860511 learned_addr learned my addr 172.21.15.5:0/3776860511 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:49.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.767+0000 7f2054e95640 1 -- 172.21.15.5:0/3776860511 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e74da516f0 msgr2=0x55e74de606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:49.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.767+0000 7f2054e95640 1 --2- 172.21.15.5:0/3776860511 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e74da516f0 0x55e74de606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.767+0000 7f2054e95640 1 -- 172.21.15.5:0/3776860511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e74da598d0 msgr2=0x55e74de60be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:49.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.767+0000 7f2054e95640 1 --2- 172.21.15.5:0/3776860511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e74da598d0 0x55e74de60be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.767+0000 7f2054e95640 1 -- 172.21.15.5:0/3776860511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e74db5ea20 con 0x55e74da7a5b0 2024-05-20T18:31:49.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.767+0000 7f2054e95640 1 --2- 172.21.15.5:0/3776860511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74da7a5b0 0x55e74de45970 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f204000fa80 tx=0x7f2040012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e695a344b9d0fdf5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:49.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.767+0000 7f20367fc640 1 -- 172.21.15.5:0/3776860511 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204001b050 con 0x55e74da7a5b0 2024-05-20T18:31:49.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.768+0000 7f20367fc640 1 -- 172.21.15.5:0/3776860511 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2040025070 con 0x55e74da7a5b0 2024-05-20T18:31:49.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.768+0000 7f20367fc640 1 -- 172.21.15.5:0/3776860511 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040014de0 con 0x55e74da7a5b0 2024-05-20T18:31:49.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.768+0000 7f205df9c740 1 -- 172.21.15.5:0/3776860511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74da7a5b0 msgr2=0x55e74de45970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:49.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.768+0000 7f205df9c740 1 --2- 172.21.15.5:0/3776860511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74da7a5b0 0x55e74de45970 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f204000fa80 tx=0x7f2040012000 comp rx=0 tx=0).stop 2024-05-20T18:31:49.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.768+0000 7f205df9c740 1 -- 172.21.15.5:0/3776860511 shutdown_connections 2024-05-20T18:31:49.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.768+0000 7f205df9c740 1 --2- 172.21.15.5:0/3776860511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e74da598d0 0x55e74de60be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.768+0000 7f205df9c740 1 --2- 172.21.15.5:0/3776860511 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e74da516f0 0x55e74de606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.768+0000 7f205df9c740 1 --2- 172.21.15.5:0/3776860511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74da7a5b0 0x55e74de45970 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.768+0000 7f205df9c740 1 -- 172.21.15.5:0/3776860511 >> 172.21.15.5:0/3776860511 conn(0x55e74da6e700 msgr2=0x55e74de1afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:49.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.768+0000 7f205df9c740 1 -- 172.21.15.5:0/3776860511 shutdown_connections 2024-05-20T18:31:49.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.768+0000 7f205df9c740 1 -- 172.21.15.5:0/3776860511 wait complete. 2024-05-20T18:31:49.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.769+0000 7f205df9c740 1 -- start start 2024-05-20T18:31:49.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.769+0000 7f205df9c740 1 Processor -- start 2024-05-20T18:31:49.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.771+0000 7f205df9c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e74da516f0 0x55e74de5d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.771+0000 7f205df9c740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e74da598d0 0x55e74de5e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.771+0000 7f205df9c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74da7a5b0 0x55e74de59ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.771+0000 7f205df9c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e74dd39c20 con 0x55e74da7a5b0 2024-05-20T18:31:49.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.771+0000 7f205df9c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e74de5eb40 con 0x55e74da516f0 2024-05-20T18:31:49.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.771+0000 7f205df9c740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55e74de58be0 con 0x55e74da598d0 2024-05-20T18:31:49.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.771+0000 7f205df9c740 10 client.0 _ll_register_callbacks cb 0x55e74dd768a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:31:49.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.771+0000 7f2055696640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74da7a5b0 0x55e74de59ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.771+0000 7f2055696640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74da7a5b0 0x55e74de59ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51858/0 (socket says 172.21.15.5:51858) 2024-05-20T18:31:49.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.771+0000 7f2055696640 1 -- 172.21.15.5:0/3159054295 learned_addr learned my addr 172.21.15.5:0/3159054295 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:49.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.771+0000 7f204ffff640 1 --2- 172.21.15.5:0/3159054295 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e74da598d0 0x55e74de5e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.771+0000 7f2054e95640 1 --2- 172.21.15.5:0/3159054295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e74da516f0 0x55e74de5d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.771+0000 7f2055696640 1 -- 172.21.15.5:0/3159054295 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e74da598d0 msgr2=0x55e74de5e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:49.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.771+0000 7f2055696640 1 --2- 172.21.15.5:0/3159054295 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e74da598d0 0x55e74de5e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.771+0000 7f2055696640 1 -- 172.21.15.5:0/3159054295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e74da516f0 msgr2=0x55e74de5d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:49.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.771+0000 7f2055696640 1 --2- 172.21.15.5:0/3159054295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e74da516f0 0x55e74de5d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.771+0000 7f2055696640 1 -- 172.21.15.5:0/3159054295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e74da52310 con 0x55e74da7a5b0 2024-05-20T18:31:49.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.771+0000 7f2054e95640 1 --2- 172.21.15.5:0/3159054295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e74da516f0 0x55e74de5d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:49.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.771+0000 7f2055696640 1 --2- 172.21.15.5:0/3159054295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74da7a5b0 0x55e74de59ed0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f2048083050 tx=0x7f2048007840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8d77a0fb1eb4b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:49.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.771+0000 7f202f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:31:49.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.772+0000 7f202f7fe640 1 -- 172.21.15.5:0/3159054295 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2048083580 con 0x55e74da7a5b0 2024-05-20T18:31:49.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.772+0000 7f202f7fe640 1 -- 172.21.15.5:0/3159054295 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f204807acb0 con 0x55e74da7a5b0 2024-05-20T18:31:49.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.772+0000 7f205df9c740 1 -- 172.21.15.5:0/3159054295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2040025070 con 0x55e74da7a5b0 2024-05-20T18:31:49.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.772+0000 7f202f7fe640 1 -- 172.21.15.5:0/3159054295 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f204807aed0 con 0x55e74da7a5b0 2024-05-20T18:31:49.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.772+0000 7f205df9c740 10 client.7077 fetch_fsmap learned FSMap version 50 2024-05-20T18:31:49.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.772+0000 7f205df9c740 1 -- 172.21.15.5:0/3159054295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x55e74dbbf1b0 con 0x55e74da7a5b0 2024-05-20T18:31:49.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.772+0000 7f202f7fe640 1 -- 172.21.15.5:0/3159054295 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2040025070 con 0x55e74da7a5b0 2024-05-20T18:31:49.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.772+0000 7f202f7fe640 1 -- 172.21.15.5:0/3159054295 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f20480830f0 con 0x55e74da7a5b0 2024-05-20T18:31:49.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.772+0000 7f205df9c740 10 client.7077 fetch_fsmap finished waiting for FSMap version 50 2024-05-20T18:31:49.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.772+0000 7f205df9c740 10 client.7077 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-05-20T18:31:49.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.772+0000 7f205df9c740 20 client.7077 populate_metadata read hostname 'smithi005' 2024-05-20T18:31:49.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.772+0000 7f205df9c740 1 --2- 172.21.15.5:0/3159054295 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x55e74da39180 0x55e74de48110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.772+0000 7f205df9c740 4 client.7077 mds_command: new command op to 6972 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:31:49.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.772+0000 7f205df9c740 1 -- 172.21.15.5:0/3159054295 --> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e74dd8c680 con 0x55e74da39180 2024-05-20T18:31:49.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.772+0000 7f2054e95640 1 --2- 172.21.15.5:0/3159054295 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x55e74da39180 0x55e74de48110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.773+0000 7f202f7fe640 10 client.7077 ms_handle_connect on v2:172.21.15.5:6835/3866822752 2024-05-20T18:31:49.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.773+0000 7f2054e95640 1 --2- 172.21.15.5:0/3159054295 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x55e74da39180 0x55e74de48110 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c61fd93c91621cb4 server_cookie=27f85f55a694fb83 in_seq=0 out_seq=0 2024-05-20T18:31:49.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.773+0000 7f202f7fe640 1 -- 172.21.15.5:0/3159054295 <== mds.0 v2:172.21.15.5:6835/3866822752 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e74dd8c680 con 0x55e74da39180 2024-05-20T18:31:49.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.773+0000 7f202f7fe640 10 client.7077 handle_command_reply: tid=0 2024-05-20T18:31:49.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.773+0000 7f205df9c740 1 client.7077 shutdown 2024-05-20T18:31:49.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.773+0000 7f205df9c740 1 -- 172.21.15.5:0/3159054295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74da7a5b0 msgr2=0x55e74de59ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:49.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.773+0000 7f205df9c740 1 --2- 172.21.15.5:0/3159054295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74da7a5b0 0x55e74de59ed0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f2048083050 tx=0x7f2048007840 comp rx=0 tx=0).stop 2024-05-20T18:31:49.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.774+0000 7f205df9c740 1 -- 172.21.15.5:0/3159054295 shutdown_connections 2024-05-20T18:31:49.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.774+0000 7f205df9c740 1 --2- 172.21.15.5:0/3159054295 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x55e74da39180 0x55e74de48110 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.774+0000 7f205df9c740 1 --2- 172.21.15.5:0/3159054295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74da7a5b0 0x55e74de59ed0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.774+0000 7f205df9c740 1 --2- 172.21.15.5:0/3159054295 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e74da598d0 0x55e74de5e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.774+0000 7f205df9c740 1 --2- 172.21.15.5:0/3159054295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e74da516f0 0x55e74de5d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.774+0000 7f205df9c740 1 -- 172.21.15.5:0/3159054295 >> 172.21.15.5:0/3159054295 conn(0x55e74da6e700 msgr2=0x55e74de1afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.774+0000 7f202f7fe640 0 client.7077 ms_handle_reset on v2:172.21.15.5:6835/3866822752 2024-05-20T18:31:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.774+0000 7f205df9c740 1 -- 172.21.15.5:0/3159054295 shutdown_connections 2024-05-20T18:31:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.774+0000 7f205df9c740 1 -- 172.21.15.5:0/3159054295 wait complete. 2024-05-20T18:31:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.774+0000 7f205df9c740 20 client.7077 trim_cache size 0 max 16384 2024-05-20T18:31:49.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.792+0000 7f205df9c740 1 Processor -- start 2024-05-20T18:31:49.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.792+0000 7f205df9c740 1 -- start start 2024-05-20T18:31:49.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.793+0000 7f205df9c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e74da516f0 0x55e74de57ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.794+0000 7f205df9c740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e74da598d0 0x55e74de5cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.794+0000 7f205df9c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74da7a5b0 0x55e74de40f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.794+0000 7f205df9c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2050202470 con 0x55e74da7a5b0 2024-05-20T18:31:49.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.794+0000 7f205df9c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2050201470 con 0x55e74da516f0 2024-05-20T18:31:49.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.794+0000 7f205df9c740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f20500cb090 con 0x55e74da598d0 2024-05-20T18:31:49.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.794+0000 7f2055696640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74da7a5b0 0x55e74de40f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.794+0000 7f2055696640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74da7a5b0 0x55e74de40f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51872/0 (socket says 172.21.15.5:51872) 2024-05-20T18:31:49.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.794+0000 7f2055696640 1 -- 172.21.15.5:0/2348101576 learned_addr learned my addr 172.21.15.5:0/2348101576 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:49.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.794+0000 7f2055696640 1 -- 172.21.15.5:0/2348101576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e74da598d0 msgr2=0x55e74de5cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:49.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.794+0000 7f2055696640 1 --2- 172.21.15.5:0/2348101576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e74da598d0 0x55e74de5cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.794+0000 7f2055696640 1 -- 172.21.15.5:0/2348101576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e74da516f0 msgr2=0x55e74de57ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:49.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.794+0000 7f2055696640 1 --2- 172.21.15.5:0/2348101576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e74da516f0 0x55e74de57ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.794+0000 7f2055696640 1 -- 172.21.15.5:0/2348101576 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e74de5e4a0 con 0x55e74da7a5b0 2024-05-20T18:31:49.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.794+0000 7f2055696640 1 --2- 172.21.15.5:0/2348101576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74da7a5b0 0x55e74de40f20 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f2048076eb0 tx=0x7f2048046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4878e8e0521f41af server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:49.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.794+0000 7f2035ffb640 1 -- 172.21.15.5:0/2348101576 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20480830f0 con 0x55e74da7a5b0 2024-05-20T18:31:49.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.794+0000 7f2035ffb640 1 -- 172.21.15.5:0/2348101576 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2048081070 con 0x55e74da7a5b0 2024-05-20T18:31:49.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.794+0000 7f2035ffb640 1 -- 172.21.15.5:0/2348101576 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204807a110 con 0x55e74da7a5b0 2024-05-20T18:31:49.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.795+0000 7f205df9c740 1 -- 172.21.15.5:0/2348101576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74da7a5b0 msgr2=0x55e74de40f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:49.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.795+0000 7f205df9c740 1 --2- 172.21.15.5:0/2348101576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74da7a5b0 0x55e74de40f20 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f2048076eb0 tx=0x7f2048046000 comp rx=0 tx=0).stop 2024-05-20T18:31:49.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.795+0000 7f205df9c740 1 -- 172.21.15.5:0/2348101576 shutdown_connections 2024-05-20T18:31:49.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.795+0000 7f205df9c740 1 --2- 172.21.15.5:0/2348101576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74da7a5b0 0x55e74de40f20 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.795+0000 7f205df9c740 1 --2- 172.21.15.5:0/2348101576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e74da598d0 0x55e74de5cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.795+0000 7f205df9c740 1 --2- 172.21.15.5:0/2348101576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e74da516f0 0x55e74de57ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.795+0000 7f205df9c740 1 -- 172.21.15.5:0/2348101576 >> 172.21.15.5:0/2348101576 conn(0x55e74da6e700 msgr2=0x55e74de11440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:49.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.795+0000 7f205df9c740 1 -- 172.21.15.5:0/2348101576 shutdown_connections 2024-05-20T18:31:49.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.795+0000 7f205df9c740 1 -- 172.21.15.5:0/2348101576 wait complete. 2024-05-20T18:31:49.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.796+0000 7f205df9c740 1 -- start start 2024-05-20T18:31:49.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.796+0000 7f205df9c740 1 Processor -- start 2024-05-20T18:31:49.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.797+0000 7f205df9c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74da516f0 0x55e74de59eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.797+0000 7f205df9c740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e74da598d0 0x55e74de40f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.797+0000 7f205df9c740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e74da7a5b0 0x55e74de41460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.797+0000 7f205df9c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2050200e50 con 0x55e74da516f0 2024-05-20T18:31:49.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.797+0000 7f205df9c740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e74dede0e0 con 0x55e74da598d0 2024-05-20T18:31:49.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.797+0000 7f205df9c740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55e74de3b690 con 0x55e74da7a5b0 2024-05-20T18:31:49.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.797+0000 7f205df9c740 10 client.0 _ll_register_callbacks cb 0x7f20480ac040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:31:49.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.797+0000 7f2054e95640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74da516f0 0x55e74de59eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.798+0000 7f2054e95640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74da516f0 0x55e74de59eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51882/0 (socket says 172.21.15.5:51882) 2024-05-20T18:31:49.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.798+0000 7f2054e95640 1 -- 172.21.15.5:0/1268965369 learned_addr learned my addr 172.21.15.5:0/1268965369 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:49.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.798+0000 7f2055696640 1 --2- 172.21.15.5:0/1268965369 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e74da7a5b0 0x55e74de41460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.798+0000 7f204ffff640 1 --2- 172.21.15.5:0/1268965369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e74da598d0 0x55e74de40f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.798+0000 7f2054e95640 1 -- 172.21.15.5:0/1268965369 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e74da7a5b0 msgr2=0x55e74de41460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:49.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.798+0000 7f2054e95640 1 --2- 172.21.15.5:0/1268965369 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e74da7a5b0 0x55e74de41460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.798+0000 7f2054e95640 1 -- 172.21.15.5:0/1268965369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e74da598d0 msgr2=0x55e74de40f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:49.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.798+0000 7f2054e95640 1 --2- 172.21.15.5:0/1268965369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e74da598d0 0x55e74de40f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.798+0000 7f2054e95640 1 -- 172.21.15.5:0/1268965369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e74da2b490 con 0x55e74da516f0 2024-05-20T18:31:49.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.798+0000 7f2055696640 1 --2- 172.21.15.5:0/1268965369 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e74da7a5b0 0x55e74de41460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:49.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.798+0000 7f2054e95640 1 --2- 172.21.15.5:0/1268965369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74da516f0 0x55e74de59eb0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f2040022040 tx=0x7f20400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ce15d3f8be2246 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:49.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.798+0000 7f202f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:31:49.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.798+0000 7f202f7fe640 1 -- 172.21.15.5:0/1268965369 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204003b060 con 0x55e74da516f0 2024-05-20T18:31:49.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.798+0000 7f202f7fe640 1 -- 172.21.15.5:0/1268965369 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f2040020850 con 0x55e74da516f0 2024-05-20T18:31:49.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.798+0000 7f205df9c740 1 -- 172.21.15.5:0/1268965369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2048081070 con 0x55e74da516f0 2024-05-20T18:31:49.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.798+0000 7f202f7fe640 1 -- 172.21.15.5:0/1268965369 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f2040041050 con 0x55e74da516f0 2024-05-20T18:31:49.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.798+0000 7f205df9c740 10 client.7083 fetch_fsmap learned FSMap version 50 2024-05-20T18:31:49.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.798+0000 7f205df9c740 1 -- 172.21.15.5:0/1268965369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x55e74dec6160 con 0x55e74da516f0 2024-05-20T18:31:49.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.799+0000 7f202f7fe640 1 -- 172.21.15.5:0/1268965369 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2048081070 con 0x55e74da516f0 2024-05-20T18:31:49.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.799+0000 7f202f7fe640 1 -- 172.21.15.5:0/1268965369 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f2040044020 con 0x55e74da516f0 2024-05-20T18:31:49.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.799+0000 7f205df9c740 10 client.7083 fetch_fsmap finished waiting for FSMap version 50 2024-05-20T18:31:49.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.799+0000 7f205df9c740 10 client.7083 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-05-20T18:31:49.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.799+0000 7f205df9c740 20 client.7083 populate_metadata read hostname 'smithi005' 2024-05-20T18:31:49.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.799+0000 7f205df9c740 1 --2- 172.21.15.5:0/1268965369 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x55e74de1acc0 0x55e74de1b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.799+0000 7f205df9c740 4 client.7083 mds_command: new command op to 6972 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}] 2024-05-20T18:31:49.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.799+0000 7f205df9c740 1 -- 172.21.15.5:0/1268965369 --> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}) -- 0x55e74dd869d0 con 0x55e74de1acc0 2024-05-20T18:31:49.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.799+0000 7f204ffff640 1 --2- 172.21.15.5:0/1268965369 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x55e74de1acc0 0x55e74de1b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.799+0000 7f202f7fe640 10 client.7083 ms_handle_connect on v2:172.21.15.5:6835/3866822752 2024-05-20T18:31:49.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.799+0000 7f204ffff640 1 --2- 172.21.15.5:0/1268965369 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x55e74de1acc0 0x55e74de1b0d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff9e4388e05278f0 server_cookie=36a010e3b2ea493d in_seq=0 out_seq=0 2024-05-20T18:31:49.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.800+0000 7f202f7fe640 1 -- 172.21.15.5:0/1268965369 <== mds.0 v2:172.21.15.5:6835/3866822752 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e74dd869d0 con 0x55e74de1acc0 2024-05-20T18:31:49.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.800+0000 7f202f7fe640 10 client.7083 handle_command_reply: tid=0 2024-05-20T18:31:49.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.800+0000 7f205df9c740 1 client.7083 shutdown 2024-05-20T18:31:49.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.800+0000 7f205df9c740 1 -- 172.21.15.5:0/1268965369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74da516f0 msgr2=0x55e74de59eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:49.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.800+0000 7f205df9c740 1 --2- 172.21.15.5:0/1268965369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74da516f0 0x55e74de59eb0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f2040022040 tx=0x7f20400046c0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.800+0000 7f205df9c740 1 -- 172.21.15.5:0/1268965369 shutdown_connections 2024-05-20T18:31:49.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.800+0000 7f205df9c740 1 --2- 172.21.15.5:0/1268965369 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e74da7a5b0 0x55e74de41460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.800+0000 7f205df9c740 1 --2- 172.21.15.5:0/1268965369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e74da598d0 0x55e74de40f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.800+0000 7f205df9c740 1 --2- 172.21.15.5:0/1268965369 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x55e74de1acc0 0x55e74de1b0d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.800+0000 7f205df9c740 1 --2- 172.21.15.5:0/1268965369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e74da516f0 0x55e74de59eb0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.800+0000 7f202f7fe640 0 client.7083 ms_handle_reset on v2:172.21.15.5:6835/3866822752 2024-05-20T18:31:49.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.800+0000 7f205df9c740 1 -- 172.21.15.5:0/1268965369 >> 172.21.15.5:0/1268965369 conn(0x55e74da6e700 msgr2=0x55e74de19480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:49.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.801+0000 7f205df9c740 1 -- 172.21.15.5:0/1268965369 shutdown_connections 2024-05-20T18:31:49.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.801+0000 7f205df9c740 1 -- 172.21.15.5:0/1268965369 wait complete. 2024-05-20T18:31:49.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.801+0000 7f205df9c740 20 client.7083 trim_cache size 0 max 16384 2024-05-20T18:31:49.820 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:31:49.820 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-20T18:31:49.820 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "87ec622d-94ba-4fe5-b06c-04e8ae0e6e5e", 2024-05-20T18:31:49.820 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-20T18:31:49.821 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:31:49.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.801+0000 7f20570f9640 1 -- 172.21.15.5:0/4158230895 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f203c06be60 msgr2=0x7f203c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:49.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.801+0000 7f20570f9640 1 --2- 172.21.15.5:0/4158230895 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f203c06be60 0x7f203c06e370 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2040006fd0 tx=0x7f2040008040 comp rx=0 tx=0).stop 2024-05-20T18:31:49.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.801+0000 7f20570f9640 1 -- 172.21.15.5:0/4158230895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20500b3860 msgr2=0x7f2050159360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:49.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.801+0000 7f20570f9640 1 --2- 172.21.15.5:0/4158230895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20500b3860 0x7f2050159360 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f2048009e50 tx=0x7f2048004a40 comp rx=0 tx=0).stop 2024-05-20T18:31:49.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.801+0000 7f20570f9640 1 -- 172.21.15.5:0/4158230895 shutdown_connections 2024-05-20T18:31:49.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.801+0000 7f20570f9640 1 --2- 172.21.15.5:0/4158230895 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f203c06be60 0x7f203c06e370 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.801+0000 7f20570f9640 1 --2- 172.21.15.5:0/4158230895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20500b3860 0x7f2050159360 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.801+0000 7f20570f9640 1 --2- 172.21.15.5:0/4158230895 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f205008bd20 0x7f20500ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.801+0000 7f20570f9640 1 --2- 172.21.15.5:0/4158230895 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f205006d870 0x7f2050155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.801+0000 7f20570f9640 1 -- 172.21.15.5:0/4158230895 >> 172.21.15.5:0/4158230895 conn(0x7f20500b6d10 msgr2=0x7f20500b5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:49.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.801+0000 7f20570f9640 1 -- 172.21.15.5:0/4158230895 shutdown_connections 2024-05-20T18:31:49.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.801+0000 7f20570f9640 1 -- 172.21.15.5:0/4158230895 wait complete. 2024-05-20T18:31:49.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:31:49.825 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-05-20T18:31:49.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.942+0000 7f913ee8b640 1 Processor -- start 2024-05-20T18:31:49.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.942+0000 7f913ee8b640 1 -- start start 2024-05-20T18:31:49.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.944+0000 7f913ee8b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f91380d10e0 0x7f91380cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.944+0000 7f913ee8b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f91380b7210 0x7f91380cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.944+0000 7f913ee8b640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91380d1ac0 con 0x7f91380d14f0 2024-05-20T18:31:49.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.944+0000 7f913ee8b640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f91380d1da0 con 0x7f91380b7210 2024-05-20T18:31:49.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.944+0000 7f913ee8b640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f91380d2080 con 0x7f91380d10e0 2024-05-20T18:31:49.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.944+0000 7f913d428640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f91380b7210 0x7f91380cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.944+0000 7f9137fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f91380d14f0 0x7f91380cd4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:48886/0 (socket says 172.21.15.5:48886) 2024-05-20T18:31:49.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.944+0000 7f9137fff640 1 -- 172.21.15.5:0/2741356934 learned_addr learned my addr 172.21.15.5:0/2741356934 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:31:49.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.944+0000 7f913cc27640 1 --2- 172.21.15.5:0/2741356934 >> v2:172.21.15.63:3300/0 conn(0x7f91380d10e0 0x7f91380cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.944+0000 7f91377fe640 1 -- 172.21.15.5:0/2741356934 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 69182872 0 0) 0x7f91380d1ac0 con 0x7f91380d14f0 2024-05-20T18:31:49.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.944+0000 7f91377fe640 1 -- 172.21.15.5:0/2741356934 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9120003520 con 0x7f91380d14f0 2024-05-20T18:31:49.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.944+0000 7f913d428640 1 -- 172.21.15.5:0/2741356934 >> v2:172.21.15.63:3300/0 conn(0x7f91380d10e0 msgr2=0x7f91380cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:49.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.944+0000 7f913d428640 1 --2- 172.21.15.5:0/2741356934 >> v2:172.21.15.63:3300/0 conn(0x7f91380d10e0 0x7f91380cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.944+0000 7f913d428640 1 -- 172.21.15.5:0/2741356934 >> v1:172.21.15.5:6789/0 conn(0x7f91380d14f0 legacy=0x7f91380cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:49.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.944+0000 7f913d428640 1 -- 172.21.15.5:0/2741356934 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91380d2360 con 0x7f91380b7210 2024-05-20T18:31:49.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.945+0000 7f913d428640 1 --2- 172.21.15.5:0/2741356934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91380b7210 0x7f91380cf680 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f91380ccfc0 tx=0x7f913002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d85e1aaab2e2cc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:49.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.945+0000 7f91377fe640 1 -- 172.21.15.5:0/2741356934 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130007740 con 0x7f91380b7210 2024-05-20T18:31:49.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.945+0000 7f91377fe640 1 -- 172.21.15.5:0/2741356934 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91300078e0 con 0x7f91380b7210 2024-05-20T18:31:49.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.945+0000 7f91377fe640 1 -- 172.21.15.5:0/2741356934 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130007cb0 con 0x7f91380b7210 2024-05-20T18:31:49.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.946+0000 7f913ee8b640 1 -- 172.21.15.5:0/2741356934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91380b7210 msgr2=0x7f91380cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:49.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.946+0000 7f913ee8b640 1 --2- 172.21.15.5:0/2741356934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91380b7210 0x7f91380cf680 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f91380ccfc0 tx=0x7f913002a680 comp rx=0 tx=0).stop 2024-05-20T18:31:49.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.946+0000 7f913ee8b640 1 -- 172.21.15.5:0/2741356934 shutdown_connections 2024-05-20T18:31:49.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.946+0000 7f913ee8b640 1 --2- 172.21.15.5:0/2741356934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91380b7210 0x7f91380cf680 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.946+0000 7f913ee8b640 1 --2- 172.21.15.5:0/2741356934 >> v2:172.21.15.63:3300/0 conn(0x7f91380d10e0 0x7f91380cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.946+0000 7f913ee8b640 1 -- 172.21.15.5:0/2741356934 >> 172.21.15.5:0/2741356934 conn(0x7f91380b44d0 msgr2=0x7f91380b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:49.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.946+0000 7f913ee8b640 1 -- 172.21.15.5:0/2741356934 shutdown_connections 2024-05-20T18:31:49.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.946+0000 7f913ee8b640 1 -- 172.21.15.5:0/2741356934 wait complete. 2024-05-20T18:31:49.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.948+0000 7f913ee8b640 1 Processor -- start 2024-05-20T18:31:49.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.948+0000 7f913ee8b640 1 -- start start 2024-05-20T18:31:49.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.949+0000 7f913ee8b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91380b7210 0x7f91380c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.949+0000 7f913ee8b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91380d10e0 0x7f91380c7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.949+0000 7f913ee8b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91380d14f0 0x7f91380c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.949+0000 7f913ee8b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f91380b8440 con 0x7f91380b7210 2024-05-20T18:31:49.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.949+0000 7f913ee8b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f91380b82c0 con 0x7f91380d14f0 2024-05-20T18:31:49.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.949+0000 7f913ee8b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f91380709f0 con 0x7f91380d10e0 2024-05-20T18:31:49.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.949+0000 7f913cc27640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91380b7210 0x7f91380c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.949+0000 7f913cc27640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91380b7210 0x7f91380c6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51896/0 (socket says 172.21.15.5:51896) 2024-05-20T18:31:49.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.949+0000 7f913cc27640 1 -- 172.21.15.5:0/3888245103 learned_addr learned my addr 172.21.15.5:0/3888245103 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:49.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.950+0000 7f913d428640 1 --2- 172.21.15.5:0/3888245103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91380d14f0 0x7f91380c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.950+0000 7f9137fff640 1 --2- 172.21.15.5:0/3888245103 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91380d10e0 0x7f91380c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.950+0000 7f913cc27640 1 -- 172.21.15.5:0/3888245103 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91380d10e0 msgr2=0x7f91380c7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:49.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.950+0000 7f913cc27640 1 --2- 172.21.15.5:0/3888245103 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91380d10e0 0x7f91380c7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.950+0000 7f913cc27640 1 -- 172.21.15.5:0/3888245103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91380d14f0 msgr2=0x7f91380c79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:49.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.950+0000 7f913cc27640 1 --2- 172.21.15.5:0/3888245103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91380d14f0 0x7f91380c79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.950+0000 7f913cc27640 1 -- 172.21.15.5:0/3888245103 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f913806cb80 con 0x7f91380b7210 2024-05-20T18:31:49.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.950+0000 7f913d428640 1 --2- 172.21.15.5:0/3888245103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91380d14f0 0x7f91380c79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:49.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.950+0000 7f913cc27640 1 --2- 172.21.15.5:0/3888245103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91380b7210 0x7f91380c6ba0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f912800d620 tx=0x7f912800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:49.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.950+0000 7f91357fa640 1 -- 172.21.15.5:0/3888245103 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912800b620 con 0x7f91380b7210 2024-05-20T18:31:49.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.950+0000 7f91357fa640 1 -- 172.21.15.5:0/3888245103 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9128019070 con 0x7f91380b7210 2024-05-20T18:31:49.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.950+0000 7f913ee8b640 1 -- 172.21.15.5:0/3888245103 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f913806ce60 con 0x7f91380b7210 2024-05-20T18:31:49.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.950+0000 7f91357fa640 1 -- 172.21.15.5:0/3888245103 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912800ba00 con 0x7f91380b7210 2024-05-20T18:31:49.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.950+0000 7f913ee8b640 1 -- 172.21.15.5:0/3888245103 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f913806d420 con 0x7f91380b7210 2024-05-20T18:31:49.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.952+0000 7f91357fa640 1 -- 172.21.15.5:0/3888245103 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f912800bba0 con 0x7f91380b7210 2024-05-20T18:31:49.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.952+0000 7f91357fa640 1 --2- 172.21.15.5:0/3888245103 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f912406be30 0x7f912406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.952+0000 7f91357fa640 1 -- 172.21.15.5:0/3888245103 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f912808b4c0 con 0x7f91380b7210 2024-05-20T18:31:49.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.952+0000 7f9137fff640 1 --2- 172.21.15.5:0/3888245103 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f912406be30 0x7f912406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.953+0000 7f9137fff640 1 --2- 172.21.15.5:0/3888245103 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f912406be30 0x7f912406e340 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f913806e140 tx=0x7f912c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:49.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.953+0000 7f9145e13740 1 Processor -- start 2024-05-20T18:31:49.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.954+0000 7f9145e13740 1 -- start start 2024-05-20T18:31:49.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.955+0000 7f9145e13740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f2f88d390 0x557f2fc57b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.955+0000 7f9145e13740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f2f8644d0 0x557f2fc72890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.955+0000 7f9145e13740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557f2f86c6b0 0x557f2fc72dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.955+0000 7f9145e13740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557f2fb91d00 con 0x557f2f8644d0 2024-05-20T18:31:49.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.955+0000 7f9145e13740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557f2fb394c0 con 0x557f2f88d390 2024-05-20T18:31:49.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.955+0000 7f9145e13740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x557f2fb8dcc0 con 0x557f2f86c6b0 2024-05-20T18:31:49.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.955+0000 7f9137fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f2f8644d0 0x557f2fc72890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.955+0000 7f9137fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f2f8644d0 0x557f2fc72890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51908/0 (socket says 172.21.15.5:51908) 2024-05-20T18:31:49.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.955+0000 7f9137fff640 1 -- 172.21.15.5:0/605957632 learned_addr learned my addr 172.21.15.5:0/605957632 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:49.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.955+0000 7f913d428640 1 --2- 172.21.15.5:0/605957632 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557f2f86c6b0 0x557f2fc72dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.955+0000 7f913cc27640 1 --2- 172.21.15.5:0/605957632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f2f88d390 0x557f2fc57b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.955+0000 7f9137fff640 1 -- 172.21.15.5:0/605957632 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557f2f86c6b0 msgr2=0x557f2fc72dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:49.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.955+0000 7f9137fff640 1 --2- 172.21.15.5:0/605957632 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557f2f86c6b0 0x557f2fc72dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.955+0000 7f9137fff640 1 -- 172.21.15.5:0/605957632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f2f88d390 msgr2=0x557f2fc57b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:49.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.955+0000 7f9137fff640 1 --2- 172.21.15.5:0/605957632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f2f88d390 0x557f2fc57b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.955+0000 7f9137fff640 1 -- 172.21.15.5:0/605957632 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f2f96f7d0 con 0x557f2f8644d0 2024-05-20T18:31:49.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.955+0000 7f913cc27640 1 --2- 172.21.15.5:0/605957632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f2f88d390 0x557f2fc57b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:49.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.955+0000 7f913d428640 1 --2- 172.21.15.5:0/605957632 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557f2f86c6b0 0x557f2fc72dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:31:49.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.955+0000 7f9137fff640 1 --2- 172.21.15.5:0/605957632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f2f8644d0 0x557f2fc72890 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f912c005c10 tx=0x7f912c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7c56567a210384 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:49.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.956+0000 7f911e7fc640 1 -- 172.21.15.5:0/605957632 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c005c40 con 0x557f2f8644d0 2024-05-20T18:31:49.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.956+0000 7f911e7fc640 1 -- 172.21.15.5:0/605957632 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f912c005de0 con 0x557f2f8644d0 2024-05-20T18:31:49.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.956+0000 7f911e7fc640 1 -- 172.21.15.5:0/605957632 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c014bb0 con 0x557f2f8644d0 2024-05-20T18:31:49.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.956+0000 7f9145e13740 1 -- 172.21.15.5:0/605957632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f2f8644d0 msgr2=0x557f2fc72890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:49.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.956+0000 7f9145e13740 1 --2- 172.21.15.5:0/605957632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f2f8644d0 0x557f2fc72890 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f912c005c10 tx=0x7f912c010000 comp rx=0 tx=0).stop 2024-05-20T18:31:49.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.956+0000 7f9145e13740 1 -- 172.21.15.5:0/605957632 shutdown_connections 2024-05-20T18:31:49.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.956+0000 7f9145e13740 1 --2- 172.21.15.5:0/605957632 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557f2f86c6b0 0x557f2fc72dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.956+0000 7f9145e13740 1 --2- 172.21.15.5:0/605957632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f2f8644d0 0x557f2fc72890 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.956+0000 7f9145e13740 1 --2- 172.21.15.5:0/605957632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f2f88d390 0x557f2fc57b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.956+0000 7f9145e13740 1 -- 172.21.15.5:0/605957632 >> 172.21.15.5:0/605957632 conn(0x557f2f8814e0 msgr2=0x557f2fc2d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:49.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.956+0000 7f9145e13740 1 -- 172.21.15.5:0/605957632 shutdown_connections 2024-05-20T18:31:49.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.956+0000 7f9145e13740 1 -- 172.21.15.5:0/605957632 wait complete. 2024-05-20T18:31:49.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.957+0000 7f9145e13740 1 -- start start 2024-05-20T18:31:49.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.958+0000 7f9145e13740 1 Processor -- start 2024-05-20T18:31:49.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.959+0000 7f9145e13740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f2f8644d0 0x557f2fc6bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.959+0000 7f9145e13740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557f2f86c6b0 0x557f2fc6c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.959+0000 7f9145e13740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f2f88d390 0x557f2fc72910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.959+0000 7f9145e13740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557f2fb4cb90 con 0x557f2f88d390 2024-05-20T18:31:49.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.959+0000 7f9145e13740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557f2fc76560 con 0x557f2f8644d0 2024-05-20T18:31:49.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.959+0000 7f9145e13740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x557f2fc597f0 con 0x557f2f86c6b0 2024-05-20T18:31:49.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.959+0000 7f9145e13740 10 client.0 _ll_register_callbacks cb 0x557f2fb88da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:31:49.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.959+0000 7f913d428640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f2f88d390 0x557f2fc72910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.959+0000 7f913d428640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f2f88d390 0x557f2fc72910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51920/0 (socket says 172.21.15.5:51920) 2024-05-20T18:31:49.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.959+0000 7f913d428640 1 -- 172.21.15.5:0/1651914550 learned_addr learned my addr 172.21.15.5:0/1651914550 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:49.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.959+0000 7f9137fff640 1 --2- 172.21.15.5:0/1651914550 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557f2f86c6b0 0x557f2fc6c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.959+0000 7f913cc27640 1 --2- 172.21.15.5:0/1651914550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f2f8644d0 0x557f2fc6bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.959+0000 7f913d428640 1 -- 172.21.15.5:0/1651914550 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557f2f86c6b0 msgr2=0x557f2fc6c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:49.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.959+0000 7f913d428640 1 --2- 172.21.15.5:0/1651914550 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557f2f86c6b0 0x557f2fc6c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.959+0000 7f913d428640 1 -- 172.21.15.5:0/1651914550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f2f8644d0 msgr2=0x557f2fc6bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:49.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.959+0000 7f913d428640 1 --2- 172.21.15.5:0/1651914550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f2f8644d0 0x557f2fc6bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.959+0000 7f913d428640 1 -- 172.21.15.5:0/1651914550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f2f8a53f0 con 0x557f2f88d390 2024-05-20T18:31:49.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.959+0000 7f913cc27640 1 --2- 172.21.15.5:0/1651914550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f2f8644d0 0x557f2fc6bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:31:49.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.960+0000 7f913d428640 1 --2- 172.21.15.5:0/1651914550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f2f88d390 0x557f2fc72910 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f913002a6c0 tx=0x7f9130004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c11268143e4af2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:49.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.960+0000 7f91177fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:31:49.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.960+0000 7f91177fe640 1 -- 172.21.15.5:0/1651914550 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130007da0 con 0x557f2f88d390 2024-05-20T18:31:49.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.960+0000 7f91177fe640 1 -- 172.21.15.5:0/1651914550 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f9130047d10 con 0x557f2f88d390 2024-05-20T18:31:49.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.960+0000 7f9145e13740 1 -- 172.21.15.5:0/1651914550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f912c005de0 con 0x557f2f88d390 2024-05-20T18:31:49.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.960+0000 7f91177fe640 1 -- 172.21.15.5:0/1651914550 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f913003d430 con 0x557f2f88d390 2024-05-20T18:31:49.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.960+0000 7f9145e13740 10 client.7098 fetch_fsmap learned FSMap version 50 2024-05-20T18:31:49.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.960+0000 7f9145e13740 1 -- 172.21.15.5:0/1651914550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x557f2f8650f0 con 0x557f2f88d390 2024-05-20T18:31:49.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.960+0000 7f91177fe640 1 -- 172.21.15.5:0/1651914550 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f912c005de0 con 0x557f2f88d390 2024-05-20T18:31:49.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.960+0000 7f91177fe640 1 -- 172.21.15.5:0/1651914550 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f913003d5d0 con 0x557f2f88d390 2024-05-20T18:31:49.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.960+0000 7f9145e13740 10 client.7098 fetch_fsmap finished waiting for FSMap version 50 2024-05-20T18:31:49.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.960+0000 7f9145e13740 10 client.7098 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-05-20T18:31:49.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.960+0000 7f9145e13740 20 client.7098 populate_metadata read hostname 'smithi005' 2024-05-20T18:31:49.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.960+0000 7f9145e13740 1 --2- 172.21.15.5:0/1651914550 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x557f2f84bf60 0x557f2fc75d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.960+0000 7f9145e13740 4 client.7098 mds_command: new command op to 6972 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:31:49.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.960+0000 7f9145e13740 1 -- 172.21.15.5:0/1651914550 --> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f2fb9f430 con 0x557f2f84bf60 2024-05-20T18:31:49.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.961+0000 7f913cc27640 1 --2- 172.21.15.5:0/1651914550 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x557f2f84bf60 0x557f2fc75d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.961+0000 7f91177fe640 10 client.7098 ms_handle_connect on v2:172.21.15.5:6835/3866822752 2024-05-20T18:31:49.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.961+0000 7f913cc27640 1 --2- 172.21.15.5:0/1651914550 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x557f2f84bf60 0x557f2fc75d80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ead86919a05a30e6 server_cookie=f6019d5d70022401 in_seq=0 out_seq=0 2024-05-20T18:31:49.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.961+0000 7f91177fe640 1 -- 172.21.15.5:0/1651914550 <== mds.0 v2:172.21.15.5:6835/3866822752 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557f2fb9f430 con 0x557f2f84bf60 2024-05-20T18:31:49.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.961+0000 7f91177fe640 10 client.7098 handle_command_reply: tid=0 2024-05-20T18:31:49.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.962+0000 7f9145e13740 1 client.7098 shutdown 2024-05-20T18:31:49.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.962+0000 7f9145e13740 1 -- 172.21.15.5:0/1651914550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f2f88d390 msgr2=0x557f2fc72910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:49.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.962+0000 7f9145e13740 1 --2- 172.21.15.5:0/1651914550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f2f88d390 0x557f2fc72910 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f913002a6c0 tx=0x7f9130004ab0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.962+0000 7f9145e13740 1 -- 172.21.15.5:0/1651914550 shutdown_connections 2024-05-20T18:31:49.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.962+0000 7f9145e13740 1 --2- 172.21.15.5:0/1651914550 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x557f2f84bf60 0x557f2fc75d80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.962+0000 7f9145e13740 1 --2- 172.21.15.5:0/1651914550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f2f88d390 0x557f2fc72910 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.962+0000 7f9145e13740 1 --2- 172.21.15.5:0/1651914550 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557f2f86c6b0 0x557f2fc6c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.962+0000 7f91177fe640 0 client.7098 ms_handle_reset on v2:172.21.15.5:6835/3866822752 2024-05-20T18:31:49.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.962+0000 7f9145e13740 1 --2- 172.21.15.5:0/1651914550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f2f8644d0 0x557f2fc6bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.962+0000 7f9145e13740 1 -- 172.21.15.5:0/1651914550 >> 172.21.15.5:0/1651914550 conn(0x557f2f8814e0 msgr2=0x557f2fc26b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:49.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.962+0000 7f9145e13740 1 -- 172.21.15.5:0/1651914550 shutdown_connections 2024-05-20T18:31:49.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.962+0000 7f9145e13740 1 -- 172.21.15.5:0/1651914550 wait complete. 2024-05-20T18:31:49.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.962+0000 7f9145e13740 20 client.7098 trim_cache size 0 max 16384 2024-05-20T18:31:49.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.980+0000 7f9145e13740 1 Processor -- start 2024-05-20T18:31:49.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.981+0000 7f9145e13740 1 -- start start 2024-05-20T18:31:49.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.982+0000 7f9145e13740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557f2f8644d0 0x557f2fc27ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.982+0000 7f9145e13740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f2f86c6b0 0x557f2fc95c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.982+0000 7f9145e13740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f2f88d390 0x557f2fc961c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.982+0000 7f9145e13740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9138202170 con 0x557f2f88d390 2024-05-20T18:31:49.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.982+0000 7f9145e13740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9138201210 con 0x557f2f86c6b0 2024-05-20T18:31:49.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.982+0000 7f9145e13740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9138200570 con 0x557f2f8644d0 2024-05-20T18:31:49.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.982+0000 7f913d428640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f2f88d390 0x557f2fc961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.982+0000 7f913d428640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f2f88d390 0x557f2fc961c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51930/0 (socket says 172.21.15.5:51930) 2024-05-20T18:31:49.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.982+0000 7f913d428640 1 -- 172.21.15.5:0/1387628721 learned_addr learned my addr 172.21.15.5:0/1387628721 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:49.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.982+0000 7f913d428640 1 -- 172.21.15.5:0/1387628721 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557f2f8644d0 msgr2=0x557f2fc27ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:49.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.982+0000 7f9137fff640 1 --2- 172.21.15.5:0/1387628721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f2f86c6b0 0x557f2fc95c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.982+0000 7f913cc27640 1 --2- 172.21.15.5:0/1387628721 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557f2f8644d0 0x557f2fc27ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.982+0000 7f913d428640 1 --2- 172.21.15.5:0/1387628721 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557f2f8644d0 0x557f2fc27ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.982+0000 7f913d428640 1 -- 172.21.15.5:0/1387628721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f2f86c6b0 msgr2=0x557f2fc95c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:49.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.982+0000 7f913d428640 1 --2- 172.21.15.5:0/1387628721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f2f86c6b0 0x557f2fc95c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.982+0000 7f913d428640 1 -- 172.21.15.5:0/1387628721 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f2fc199a0 con 0x557f2f88d390 2024-05-20T18:31:49.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.982+0000 7f913d428640 1 --2- 172.21.15.5:0/1387628721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f2f88d390 0x557f2fc961c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f913002ac30 tx=0x7f913003f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148fff44213fec66 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:49.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.983+0000 7f911dffb640 1 -- 172.21.15.5:0/1387628721 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130055070 con 0x557f2f88d390 2024-05-20T18:31:49.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.983+0000 7f911dffb640 1 -- 172.21.15.5:0/1387628721 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91300094d0 con 0x557f2f88d390 2024-05-20T18:31:49.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.983+0000 7f911dffb640 1 -- 172.21.15.5:0/1387628721 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130007e00 con 0x557f2f88d390 2024-05-20T18:31:49.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.983+0000 7f9145e13740 1 -- 172.21.15.5:0/1387628721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f2f88d390 msgr2=0x557f2fc961c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:49.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.983+0000 7f9145e13740 1 --2- 172.21.15.5:0/1387628721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f2f88d390 0x557f2fc961c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f913002ac30 tx=0x7f913003f070 comp rx=0 tx=0).stop 2024-05-20T18:31:49.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.983+0000 7f9145e13740 1 -- 172.21.15.5:0/1387628721 shutdown_connections 2024-05-20T18:31:49.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.983+0000 7f9145e13740 1 --2- 172.21.15.5:0/1387628721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f2f88d390 0x557f2fc961c0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.983+0000 7f9145e13740 1 --2- 172.21.15.5:0/1387628721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f2f86c6b0 0x557f2fc95c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.983+0000 7f9145e13740 1 --2- 172.21.15.5:0/1387628721 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557f2f8644d0 0x557f2fc27ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.983+0000 7f9145e13740 1 -- 172.21.15.5:0/1387628721 >> 172.21.15.5:0/1387628721 conn(0x557f2f8814e0 msgr2=0x557f2fc754e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:49.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.983+0000 7f9145e13740 1 -- 172.21.15.5:0/1387628721 shutdown_connections 2024-05-20T18:31:49.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.983+0000 7f9145e13740 1 -- 172.21.15.5:0/1387628721 wait complete. 2024-05-20T18:31:49.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.984+0000 7f9145e13740 1 -- start start 2024-05-20T18:31:49.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.984+0000 7f9145e13740 1 Processor -- start 2024-05-20T18:31:49.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.986+0000 7f9145e13740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557f2f8644d0 0x557f2fc8b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.986+0000 7f9145e13740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f2f86c6b0 0x557f2fc73f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.986+0000 7f9145e13740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f2f88d390 0x557f2fc76c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.986+0000 7f9145e13740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9138200bc0 con 0x557f2f86c6b0 2024-05-20T18:31:49.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.986+0000 7f9145e13740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557f2fc26e10 con 0x557f2f88d390 2024-05-20T18:31:49.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.986+0000 7f9145e13740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x557f2fc85400 con 0x557f2f8644d0 2024-05-20T18:31:49.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.986+0000 7f9145e13740 10 client.0 _ll_register_callbacks cb 0x7f91200036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:31:49.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.986+0000 7f9137fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f2f86c6b0 0x557f2fc73f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.986+0000 7f9137fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f2f86c6b0 0x557f2fc73f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51936/0 (socket says 172.21.15.5:51936) 2024-05-20T18:31:49.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.986+0000 7f9137fff640 1 -- 172.21.15.5:0/2662476781 learned_addr learned my addr 172.21.15.5:0/2662476781 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:31:49.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.986+0000 7f913cc27640 1 --2- 172.21.15.5:0/2662476781 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557f2f8644d0 0x557f2fc8b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.986+0000 7f913d428640 1 --2- 172.21.15.5:0/2662476781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f2f88d390 0x557f2fc76c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.986+0000 7f9137fff640 1 -- 172.21.15.5:0/2662476781 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557f2f8644d0 msgr2=0x557f2fc8b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:49.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.986+0000 7f9137fff640 1 --2- 172.21.15.5:0/2662476781 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557f2f8644d0 0x557f2fc8b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.986+0000 7f9137fff640 1 -- 172.21.15.5:0/2662476781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f2f88d390 msgr2=0x557f2fc76c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:31:49.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.986+0000 7f9137fff640 1 --2- 172.21.15.5:0/2662476781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f2f88d390 0x557f2fc76c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.986+0000 7f9137fff640 1 -- 172.21.15.5:0/2662476781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f2fc85150 con 0x557f2f86c6b0 2024-05-20T18:31:49.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.986+0000 7f9137fff640 1 --2- 172.21.15.5:0/2662476781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f2f86c6b0 0x557f2fc73f70 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f912c001eb0 tx=0x7f912c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568f75b5eeb587ff server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:31:49.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.986+0000 7f91177fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:31:49.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.987+0000 7f91177fe640 1 -- 172.21.15.5:0/2662476781 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c014dc0 con 0x557f2f86c6b0 2024-05-20T18:31:49.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.987+0000 7f91177fe640 1 -- 172.21.15.5:0/2662476781 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f912c02eca0 con 0x557f2f86c6b0 2024-05-20T18:31:49.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.987+0000 7f9145e13740 1 -- 172.21.15.5:0/2662476781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91300094d0 con 0x557f2f86c6b0 2024-05-20T18:31:49.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.987+0000 7f91177fe640 1 -- 172.21.15.5:0/2662476781 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f912c020450 con 0x557f2f86c6b0 2024-05-20T18:31:49.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.987+0000 7f9145e13740 10 client.7104 fetch_fsmap learned FSMap version 50 2024-05-20T18:31:49.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.987+0000 7f9145e13740 1 -- 172.21.15.5:0/2662476781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x557f2fc8bd50 con 0x557f2f86c6b0 2024-05-20T18:31:49.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.987+0000 7f91177fe640 1 -- 172.21.15.5:0/2662476781 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91300094d0 con 0x557f2f86c6b0 2024-05-20T18:31:49.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.987+0000 7f91177fe640 1 -- 172.21.15.5:0/2662476781 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f912c035050 con 0x557f2f86c6b0 2024-05-20T18:31:49.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.987+0000 7f9145e13740 10 client.7104 fetch_fsmap finished waiting for FSMap version 50 2024-05-20T18:31:49.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.987+0000 7f9145e13740 10 client.7104 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-05-20T18:31:49.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.987+0000 7f9145e13740 20 client.7104 populate_metadata read hostname 'smithi005' 2024-05-20T18:31:49.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.987+0000 7f9145e13740 1 --2- 172.21.15.5:0/2662476781 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x557f2fc59d70 0x557f2fc77150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:31:49.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.987+0000 7f9145e13740 4 client.7104 mds_command: new command op to 6972 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T18:31:49.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.987+0000 7f9145e13740 1 -- 172.21.15.5:0/2662476781 --> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557f2fb99780 con 0x557f2fc59d70 2024-05-20T18:31:49.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.987+0000 7f913cc27640 1 --2- 172.21.15.5:0/2662476781 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x557f2fc59d70 0x557f2fc77150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:31:49.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.988+0000 7f91177fe640 10 client.7104 ms_handle_connect on v2:172.21.15.5:6835/3866822752 2024-05-20T18:31:49.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.988+0000 7f913cc27640 1 --2- 172.21.15.5:0/2662476781 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x557f2fc59d70 0x557f2fc77150 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50bc70c0c42cb434 server_cookie=78c4257547dfb896 in_seq=0 out_seq=0 2024-05-20T18:31:49.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.988+0000 7f91177fe640 1 -- 172.21.15.5:0/2662476781 <== mds.0 v2:172.21.15.5:6835/3866822752 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557f2fb99780 con 0x557f2fc59d70 2024-05-20T18:31:49.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.988+0000 7f91177fe640 10 client.7104 handle_command_reply: tid=0 2024-05-20T18:31:49.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.988+0000 7f9145e13740 1 client.7104 shutdown 2024-05-20T18:31:49.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.988+0000 7f9145e13740 1 -- 172.21.15.5:0/2662476781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f2f86c6b0 msgr2=0x557f2fc73f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:49.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.988+0000 7f9145e13740 1 --2- 172.21.15.5:0/2662476781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f2f86c6b0 0x557f2fc73f70 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f912c001eb0 tx=0x7f912c000bc0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.988+0000 7f9145e13740 1 -- 172.21.15.5:0/2662476781 shutdown_connections 2024-05-20T18:31:49.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.988+0000 7f9145e13740 1 --2- 172.21.15.5:0/2662476781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557f2f88d390 0x557f2fc76c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.988+0000 7f9145e13740 1 --2- 172.21.15.5:0/2662476781 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x557f2fc59d70 0x557f2fc77150 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.988+0000 7f9145e13740 1 --2- 172.21.15.5:0/2662476781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f2f86c6b0 0x557f2fc73f70 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.988+0000 7f9145e13740 1 --2- 172.21.15.5:0/2662476781 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557f2f8644d0 0x557f2fc8b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.988+0000 7f91177fe640 0 client.7104 ms_handle_reset on v2:172.21.15.5:6835/3866822752 2024-05-20T18:31:49.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.988+0000 7f9145e13740 1 -- 172.21.15.5:0/2662476781 >> 172.21.15.5:0/2662476781 conn(0x557f2f8814e0 msgr2=0x557f2fc95d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:49.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.988+0000 7f9145e13740 1 -- 172.21.15.5:0/2662476781 shutdown_connections 2024-05-20T18:31:49.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.988+0000 7f9145e13740 1 -- 172.21.15.5:0/2662476781 wait complete. 2024-05-20T18:31:49.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.988+0000 7f9145e13740 20 client.7104 trim_cache size 0 max 16384 2024-05-20T18:31:49.994 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:31:49.994 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-20T18:31:49.995 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-20T18:31:49.995 INFO:teuthology.orchestra.run.smithi005.stdout: "87ec622d-94ba-4fe5-b06c-04e8ae0e6e5e": { 2024-05-20T18:31:49.995 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/subdir", 2024-05-20T18:31:49.995 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "87ec622d-94ba-4fe5-b06c-04e8ae0e6e5e", 2024-05-20T18:31:49.995 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive" 2024-05-20T18:31:49.995 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:31:49.995 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:31:49.995 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:31:49.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.989+0000 7f913ee8b640 1 -- 172.21.15.5:0/3888245103 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f912406be30 msgr2=0x7f912406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:49.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.989+0000 7f913ee8b640 1 --2- 172.21.15.5:0/3888245103 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f912406be30 0x7f912406e340 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f913806e140 tx=0x7f912c009040 comp rx=0 tx=0).stop 2024-05-20T18:31:49.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.989+0000 7f913ee8b640 1 -- 172.21.15.5:0/3888245103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91380b7210 msgr2=0x7f91380c6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:31:49.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.989+0000 7f913ee8b640 1 --2- 172.21.15.5:0/3888245103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91380b7210 0x7f91380c6ba0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f912800d620 tx=0x7f912800dae0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.989+0000 7f913ee8b640 1 -- 172.21.15.5:0/3888245103 shutdown_connections 2024-05-20T18:31:49.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.989+0000 7f913ee8b640 1 --2- 172.21.15.5:0/3888245103 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f912406be30 0x7f912406e340 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.989+0000 7f913ee8b640 1 --2- 172.21.15.5:0/3888245103 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91380d14f0 0x7f91380c79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.989+0000 7f913ee8b640 1 --2- 172.21.15.5:0/3888245103 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91380d10e0 0x7f91380c7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.989+0000 7f913ee8b640 1 --2- 172.21.15.5:0/3888245103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91380b7210 0x7f91380c6ba0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:31:49.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.989+0000 7f913ee8b640 1 -- 172.21.15.5:0/3888245103 >> 172.21.15.5:0/3888245103 conn(0x7f91380b44d0 msgr2=0x7f913815ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:31:49.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.989+0000 7f913ee8b640 1 -- 172.21.15.5:0/3888245103 shutdown_connections 2024-05-20T18:31:49.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:31:49.989+0000 7f913ee8b640 1 -- 172.21.15.5:0/3888245103 wait complete. 2024-05-20T18:31:50.001 INFO:tasks.cephfs.filesystem:scrub status for tag:87ec622d-94ba-4fe5-b06c-04e8ae0e6e5e - {'path': '/subdir', 'tag': '87ec622d-94ba-4fe5-b06c-04e8ae0e6e5e', 'options': 'recursive'} 2024-05-20T18:31:51.879 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-20T18:31:51.878+0000 7fd1d1d3f640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-20T18:32:11.334 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:32:11.338 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:32:20.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:32:20.002 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-05-20T18:32:20.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.133+0000 7fcb9e273640 1 Processor -- start 2024-05-20T18:32:20.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.133+0000 7fcb9e273640 1 -- start start 2024-05-20T18:32:20.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.135+0000 7fcb9e273640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcb9815d460 0x7fcb9806b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.135+0000 7fcb9e273640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcb980d1780 0x7fcb980d1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.135+0000 7fcb9e273640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb9815d870 con 0x7fcb9809a4f0 2024-05-20T18:32:20.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.135+0000 7fcb9e273640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fcb980d20d0 con 0x7fcb980d1780 2024-05-20T18:32:20.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.135+0000 7fcb9e273640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcb9815dae0 con 0x7fcb9815d460 2024-05-20T18:32:20.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.135+0000 7fcb977fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcb980d1780 0x7fcb980d1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.135+0000 7fcb977fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcb980d1780 0x7fcb980d1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60586/0 (socket says 172.21.15.5:60586) 2024-05-20T18:32:20.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.135+0000 7fcb97fff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcb9815d460 0x7fcb9806b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.135+0000 7fcb977fe640 1 -- 172.21.15.5:0/2827317341 learned_addr learned my addr 172.21.15.5:0/2827317341 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:20.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.135+0000 7fcb977fe640 1 -- 172.21.15.5:0/2827317341 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcb9815d460 msgr2=0x7fcb9806b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:20.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.135+0000 7fcb977fe640 1 --2- 172.21.15.5:0/2827317341 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcb9815d460 0x7fcb9806b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.135+0000 7fcb977fe640 1 -- 172.21.15.5:0/2827317341 >> v1:172.21.15.63:6789/0 conn(0x7fcb9809a4f0 legacy=0x7fcb9809a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:20.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.135+0000 7fcb977fe640 1 -- 172.21.15.5:0/2827317341 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb9815ddc0 con 0x7fcb980d1780 2024-05-20T18:32:20.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.136+0000 7fcb97fff640 1 --2- 172.21.15.5:0/2827317341 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcb9815d460 0x7fcb9806b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:32:20.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.135+0000 7fcb96ffd640 1 -- 172.21.15.5:0/2827317341 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 383814923 0 0) 0x7fcb9815d870 con 0x7fcb9809a4f0 2024-05-20T18:32:20.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.136+0000 7fcb977fe640 1 --2- 172.21.15.5:0/2827317341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb980d1780 0x7fcb980d1b90 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c009870 tx=0x7fcb8c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c233b39c17de16c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:20.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.136+0000 7fcb96ffd640 1 -- 172.21.15.5:0/2827317341 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c037070 con 0x7fcb980d1780 2024-05-20T18:32:20.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.136+0000 7fcb96ffd640 1 -- 172.21.15.5:0/2827317341 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb8c032440 con 0x7fcb980d1780 2024-05-20T18:32:20.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.136+0000 7fcb96ffd640 1 -- 172.21.15.5:0/2827317341 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c032810 con 0x7fcb980d1780 2024-05-20T18:32:20.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.137+0000 7fcb9e273640 1 -- 172.21.15.5:0/2827317341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb980d1780 msgr2=0x7fcb980d1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:20.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.137+0000 7fcb9e273640 1 --2- 172.21.15.5:0/2827317341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb980d1780 0x7fcb980d1b90 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c009870 tx=0x7fcb8c02a270 comp rx=0 tx=0).stop 2024-05-20T18:32:20.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.137+0000 7fcb9e273640 1 -- 172.21.15.5:0/2827317341 shutdown_connections 2024-05-20T18:32:20.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.137+0000 7fcb9e273640 1 --2- 172.21.15.5:0/2827317341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb980d1780 0x7fcb980d1b90 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.137+0000 7fcb9e273640 1 --2- 172.21.15.5:0/2827317341 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcb9815d460 0x7fcb9806b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.137+0000 7fcb9e273640 1 -- 172.21.15.5:0/2827317341 >> 172.21.15.5:0/2827317341 conn(0x7fcb980b6620 msgr2=0x7fcb980b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:20.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.137+0000 7fcb9e273640 1 -- 172.21.15.5:0/2827317341 shutdown_connections 2024-05-20T18:32:20.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.137+0000 7fcb9e273640 1 -- 172.21.15.5:0/2827317341 wait complete. 2024-05-20T18:32:20.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.139+0000 7fcb9e273640 1 Processor -- start 2024-05-20T18:32:20.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.139+0000 7fcb9e273640 1 -- start start 2024-05-20T18:32:20.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.141+0000 7fcb9e273640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb9809a4f0 0x7fcb981f29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.141+0000 7fcb9e273640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcb980d1780 0x7fcb980cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.141+0000 7fcb9e273640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb9815d460 0x7fcb98140ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.141+0000 7fcb9e273640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9806bdb0 con 0x7fcb9815d460 2024-05-20T18:32:20.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.141+0000 7fcb9e273640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcb980b12d0 con 0x7fcb9809a4f0 2024-05-20T18:32:20.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.141+0000 7fcb9e273640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcb9815e380 con 0x7fcb980d1780 2024-05-20T18:32:20.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.141+0000 7fcb9c810640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb9815d460 0x7fcb98140ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.141+0000 7fcb9c810640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb9815d460 0x7fcb98140ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60592/0 (socket says 172.21.15.5:60592) 2024-05-20T18:32:20.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.141+0000 7fcb9c810640 1 -- 172.21.15.5:0/2420415215 learned_addr learned my addr 172.21.15.5:0/2420415215 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:20.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.141+0000 7fcb97fff640 1 --2- 172.21.15.5:0/2420415215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb9809a4f0 0x7fcb981f29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.141+0000 7fcb977fe640 1 --2- 172.21.15.5:0/2420415215 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcb980d1780 0x7fcb980cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.141+0000 7fcb9c810640 1 -- 172.21.15.5:0/2420415215 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcb980d1780 msgr2=0x7fcb980cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:20.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.141+0000 7fcb9c810640 1 --2- 172.21.15.5:0/2420415215 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcb980d1780 0x7fcb980cf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.141+0000 7fcb9c810640 1 -- 172.21.15.5:0/2420415215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb9809a4f0 msgr2=0x7fcb981f29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:20.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.141+0000 7fcb9c810640 1 --2- 172.21.15.5:0/2420415215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb9809a4f0 0x7fcb981f29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.141+0000 7fcb9c810640 1 -- 172.21.15.5:0/2420415215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb980d0350 con 0x7fcb9815d460 2024-05-20T18:32:20.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.141+0000 7fcb9c810640 1 --2- 172.21.15.5:0/2420415215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb9815d460 0x7fcb98140ca0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000c710 tx=0x7fcb9000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:20.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.141+0000 7fcb94ff9640 1 -- 172.21.15.5:0/2420415215 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb90017070 con 0x7fcb9815d460 2024-05-20T18:32:20.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.142+0000 7fcb94ff9640 1 -- 172.21.15.5:0/2420415215 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb90003da0 con 0x7fcb9815d460 2024-05-20T18:32:20.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.142+0000 7fcb94ff9640 1 -- 172.21.15.5:0/2420415215 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb90005290 con 0x7fcb9815d460 2024-05-20T18:32:20.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.142+0000 7fcb9e273640 1 -- 172.21.15.5:0/2420415215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb980d0630 con 0x7fcb9815d460 2024-05-20T18:32:20.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.142+0000 7fcb9e273640 1 -- 172.21.15.5:0/2420415215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb980d0bf0 con 0x7fcb9815d460 2024-05-20T18:32:20.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.143+0000 7fcb94ff9640 1 -- 172.21.15.5:0/2420415215 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcb9001a070 con 0x7fcb9815d460 2024-05-20T18:32:20.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.144+0000 7fcb94ff9640 1 --2- 172.21.15.5:0/2420415215 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcb8406be30 0x7fcb8406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.144+0000 7fcb94ff9640 1 -- 172.21.15.5:0/2420415215 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7fcb9008a650 con 0x7fcb9815d460 2024-05-20T18:32:20.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.144+0000 7fcb97fff640 1 --2- 172.21.15.5:0/2420415215 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcb8406be30 0x7fcb8406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.145+0000 7fcb97fff640 1 --2- 172.21.15.5:0/2420415215 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcb8406be30 0x7fcb8406e340 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcb88009560 tx=0x7fcb88009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:20.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.145+0000 7fcba51fa740 1 Processor -- start 2024-05-20T18:32:20.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.145+0000 7fcba51fa740 1 -- start start 2024-05-20T18:32:20.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.146+0000 7fcba51fa740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646c6bf7390 0x5646c6fbe330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.146+0000 7fcba51fa740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5646c6bce4d0 0x5646c6fddac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.146+0000 7fcba51fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646c6bd66b0 0x5646c6fde000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.146+0000 7fcba51fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5646c6efbd00 con 0x5646c6bd66b0 2024-05-20T18:32:20.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.146+0000 7fcba51fa740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5646c6ea34c0 con 0x5646c6bf7390 2024-05-20T18:32:20.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.146+0000 7fcba51fa740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5646c6ef7cc0 con 0x5646c6bce4d0 2024-05-20T18:32:20.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.147+0000 7fcb9c810640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646c6bd66b0 0x5646c6fde000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.147+0000 7fcb9c810640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646c6bd66b0 0x5646c6fde000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60608/0 (socket says 172.21.15.5:60608) 2024-05-20T18:32:20.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.147+0000 7fcb9c810640 1 -- 172.21.15.5:0/2474275089 learned_addr learned my addr 172.21.15.5:0/2474275089 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:20.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.147+0000 7fcb97fff640 1 --2- 172.21.15.5:0/2474275089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646c6bf7390 0x5646c6fbe330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.147+0000 7fcb977fe640 1 --2- 172.21.15.5:0/2474275089 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5646c6bce4d0 0x5646c6fddac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.147+0000 7fcb9c810640 1 -- 172.21.15.5:0/2474275089 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5646c6bce4d0 msgr2=0x5646c6fddac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:20.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.147+0000 7fcb9c810640 1 --2- 172.21.15.5:0/2474275089 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5646c6bce4d0 0x5646c6fddac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.147+0000 7fcb9c810640 1 -- 172.21.15.5:0/2474275089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646c6bf7390 msgr2=0x5646c6fbe330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:20.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.147+0000 7fcb9c810640 1 --2- 172.21.15.5:0/2474275089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646c6bf7390 0x5646c6fbe330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.147+0000 7fcb9c810640 1 -- 172.21.15.5:0/2474275089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646c6cd97d0 con 0x5646c6bd66b0 2024-05-20T18:32:20.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.147+0000 7fcb9c810640 1 --2- 172.21.15.5:0/2474275089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646c6bd66b0 0x5646c6fde000 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fcb90060dd0 tx=0x7fcb90001400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6511ef6d64e72aa server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:20.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.147+0000 7fcb7dffb640 1 -- 172.21.15.5:0/2474275089 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb90070070 con 0x5646c6bd66b0 2024-05-20T18:32:20.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.147+0000 7fcb7dffb640 1 -- 172.21.15.5:0/2474275089 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb90056a40 con 0x5646c6bd66b0 2024-05-20T18:32:20.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.147+0000 7fcb7dffb640 1 -- 172.21.15.5:0/2474275089 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb90060e70 con 0x5646c6bd66b0 2024-05-20T18:32:20.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.148+0000 7fcba51fa740 1 -- 172.21.15.5:0/2474275089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646c6bd66b0 msgr2=0x5646c6fde000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:20.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.148+0000 7fcba51fa740 1 --2- 172.21.15.5:0/2474275089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646c6bd66b0 0x5646c6fde000 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fcb90060dd0 tx=0x7fcb90001400 comp rx=0 tx=0).stop 2024-05-20T18:32:20.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.148+0000 7fcba51fa740 1 -- 172.21.15.5:0/2474275089 shutdown_connections 2024-05-20T18:32:20.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.148+0000 7fcba51fa740 1 --2- 172.21.15.5:0/2474275089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646c6bd66b0 0x5646c6fde000 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.148+0000 7fcba51fa740 1 --2- 172.21.15.5:0/2474275089 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5646c6bce4d0 0x5646c6fddac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.148+0000 7fcba51fa740 1 --2- 172.21.15.5:0/2474275089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646c6bf7390 0x5646c6fbe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.148+0000 7fcba51fa740 1 -- 172.21.15.5:0/2474275089 >> 172.21.15.5:0/2474275089 conn(0x5646c6beb4e0 msgr2=0x5646c6f971c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:20.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.148+0000 7fcba51fa740 1 -- 172.21.15.5:0/2474275089 shutdown_connections 2024-05-20T18:32:20.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.148+0000 7fcba51fa740 1 -- 172.21.15.5:0/2474275089 wait complete. 2024-05-20T18:32:20.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.149+0000 7fcba51fa740 1 -- start start 2024-05-20T18:32:20.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.149+0000 7fcba51fa740 1 Processor -- start 2024-05-20T18:32:20.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.151+0000 7fcba51fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646c6bce4d0 0x5646c6fbcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.151+0000 7fcba51fa740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5646c6bd66b0 0x5646c6fc1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.151+0000 7fcba51fa740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646c6bf7390 0x5646c6fc2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.151+0000 7fcba51fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5646c6eb6b90 con 0x5646c6bce4d0 2024-05-20T18:32:20.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.151+0000 7fcba51fa740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5646c6ff3c60 con 0x5646c6bf7390 2024-05-20T18:32:20.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.151+0000 7fcba51fa740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5646c6fbd0c0 con 0x5646c6bd66b0 2024-05-20T18:32:20.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.151+0000 7fcba51fa740 10 client.0 _ll_register_callbacks cb 0x5646c6ef2da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:32:20.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.151+0000 7fcb97fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646c6bce4d0 0x5646c6fbcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.151+0000 7fcb97fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646c6bce4d0 0x5646c6fbcb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60616/0 (socket says 172.21.15.5:60616) 2024-05-20T18:32:20.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.151+0000 7fcb97fff640 1 -- 172.21.15.5:0/2803621101 learned_addr learned my addr 172.21.15.5:0/2803621101 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:20.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.151+0000 7fcb977fe640 1 --2- 172.21.15.5:0/2803621101 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5646c6bd66b0 0x5646c6fc1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.151+0000 7fcb9c810640 1 --2- 172.21.15.5:0/2803621101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646c6bf7390 0x5646c6fc2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.151+0000 7fcb97fff640 1 -- 172.21.15.5:0/2803621101 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5646c6bd66b0 msgr2=0x5646c6fc1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:20.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.151+0000 7fcb97fff640 1 --2- 172.21.15.5:0/2803621101 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5646c6bd66b0 0x5646c6fc1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.151+0000 7fcb97fff640 1 -- 172.21.15.5:0/2803621101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646c6bf7390 msgr2=0x5646c6fc2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:20.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.151+0000 7fcb97fff640 1 --2- 172.21.15.5:0/2803621101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646c6bf7390 0x5646c6fc2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.151+0000 7fcb97fff640 1 -- 172.21.15.5:0/2803621101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646c6c0f3f0 con 0x5646c6bce4d0 2024-05-20T18:32:20.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.151+0000 7fcb97fff640 1 --2- 172.21.15.5:0/2803621101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646c6bce4d0 0x5646c6fbcb80 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fcb88012000 tx=0x7fcb880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8cf50d7624e5f24 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:20.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.151+0000 7fcb76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:32:20.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.151+0000 7fcb76ffd640 1 -- 172.21.15.5:0/2803621101 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb88027070 con 0x5646c6bce4d0 2024-05-20T18:32:20.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.151+0000 7fcb76ffd640 1 -- 172.21.15.5:0/2803621101 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7fcb88014440 con 0x5646c6bce4d0 2024-05-20T18:32:20.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.151+0000 7fcba51fa740 1 -- 172.21.15.5:0/2803621101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb90056a40 con 0x5646c6bce4d0 2024-05-20T18:32:20.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.152+0000 7fcb76ffd640 1 -- 172.21.15.5:0/2803621101 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fcb8801d040 con 0x5646c6bce4d0 2024-05-20T18:32:20.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.152+0000 7fcba51fa740 10 client.7116 fetch_fsmap learned FSMap version 51 2024-05-20T18:32:20.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.152+0000 7fcba51fa740 1 -- 172.21.15.5:0/2803621101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x5646c6bcf0f0 con 0x5646c6bce4d0 2024-05-20T18:32:20.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.152+0000 7fcb76ffd640 1 -- 172.21.15.5:0/2803621101 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb90056a40 con 0x5646c6bce4d0 2024-05-20T18:32:20.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.152+0000 7fcb76ffd640 1 -- 172.21.15.5:0/2803621101 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7fcb88005990 con 0x5646c6bce4d0 2024-05-20T18:32:20.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.152+0000 7fcba51fa740 10 client.7116 fetch_fsmap finished waiting for FSMap version 51 2024-05-20T18:32:20.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.152+0000 7fcba51fa740 10 client.7116 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-05-20T18:32:20.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.152+0000 7fcba51fa740 20 client.7116 populate_metadata read hostname 'smithi005' 2024-05-20T18:32:20.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.152+0000 7fcba51fa740 1 --2- 172.21.15.5:0/2803621101 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5646c6bb5f60 0x5646c6fc2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.152+0000 7fcba51fa740 4 client.7116 mds_command: new command op to 6972 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:32:20.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.152+0000 7fcba51fa740 1 -- 172.21.15.5:0/2803621101 --> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646c6f09430 con 0x5646c6bb5f60 2024-05-20T18:32:20.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.152+0000 7fcb977fe640 1 --2- 172.21.15.5:0/2803621101 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5646c6bb5f60 0x5646c6fc2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.152+0000 7fcb76ffd640 10 client.7116 ms_handle_connect on v2:172.21.15.5:6835/3866822752 2024-05-20T18:32:20.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.152+0000 7fcb977fe640 1 --2- 172.21.15.5:0/2803621101 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5646c6bb5f60 0x5646c6fc2dc0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21f86b3bf0b0e37 server_cookie=b78ca542073c5b3 in_seq=0 out_seq=0 2024-05-20T18:32:20.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.153+0000 7fcb76ffd640 1 -- 172.21.15.5:0/2803621101 <== mds.0 v2:172.21.15.5:6835/3866822752 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5646c6f09430 con 0x5646c6bb5f60 2024-05-20T18:32:20.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.153+0000 7fcb76ffd640 10 client.7116 handle_command_reply: tid=0 2024-05-20T18:32:20.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.153+0000 7fcba51fa740 1 client.7116 shutdown 2024-05-20T18:32:20.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.153+0000 7fcba51fa740 1 -- 172.21.15.5:0/2803621101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646c6bce4d0 msgr2=0x5646c6fbcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:20.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.153+0000 7fcba51fa740 1 --2- 172.21.15.5:0/2803621101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646c6bce4d0 0x5646c6fbcb80 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fcb88012000 tx=0x7fcb880046c0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.153+0000 7fcba51fa740 1 -- 172.21.15.5:0/2803621101 shutdown_connections 2024-05-20T18:32:20.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.153+0000 7fcba51fa740 1 --2- 172.21.15.5:0/2803621101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646c6bf7390 0x5646c6fc2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.153+0000 7fcba51fa740 1 --2- 172.21.15.5:0/2803621101 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5646c6bd66b0 0x5646c6fc1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.153+0000 7fcba51fa740 1 --2- 172.21.15.5:0/2803621101 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5646c6bb5f60 0x5646c6fc2dc0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.153+0000 7fcba51fa740 1 --2- 172.21.15.5:0/2803621101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646c6bce4d0 0x5646c6fbcb80 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.153+0000 7fcb76ffd640 0 client.7116 ms_handle_reset on v2:172.21.15.5:6835/3866822752 2024-05-20T18:32:20.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.153+0000 7fcba51fa740 1 -- 172.21.15.5:0/2803621101 >> 172.21.15.5:0/2803621101 conn(0x5646c6beb4e0 msgr2=0x5646c6f98850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:20.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.153+0000 7fcba51fa740 1 -- 172.21.15.5:0/2803621101 shutdown_connections 2024-05-20T18:32:20.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.154+0000 7fcba51fa740 1 -- 172.21.15.5:0/2803621101 wait complete. 2024-05-20T18:32:20.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.154+0000 7fcba51fa740 20 client.7116 trim_cache size 0 max 16384 2024-05-20T18:32:20.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.172+0000 7fcba51fa740 1 Processor -- start 2024-05-20T18:32:20.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.172+0000 7fcba51fa740 1 -- start start 2024-05-20T18:32:20.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.173+0000 7fcba51fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646c6bce4d0 0x5646c6fc2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.173+0000 7fcba51fa740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5646c6bd66b0 0x5646c6fc5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.173+0000 7fcba51fa740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646c6bf7390 0x5646c70541c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.173+0000 7fcba51fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcb981fdf10 con 0x5646c6bce4d0 2024-05-20T18:32:20.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.173+0000 7fcba51fa740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcb980b01b0 con 0x5646c6bf7390 2024-05-20T18:32:20.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.173+0000 7fcba51fa740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcb9814abf0 con 0x5646c6bd66b0 2024-05-20T18:32:20.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.173+0000 7fcb97fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646c6bce4d0 0x5646c6fc2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.173+0000 7fcb97fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646c6bce4d0 0x5646c6fc2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60618/0 (socket says 172.21.15.5:60618) 2024-05-20T18:32:20.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.174+0000 7fcb97fff640 1 -- 172.21.15.5:0/524444808 learned_addr learned my addr 172.21.15.5:0/524444808 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:20.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.174+0000 7fcb9c810640 1 --2- 172.21.15.5:0/524444808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646c6bf7390 0x5646c70541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.174+0000 7fcb977fe640 1 --2- 172.21.15.5:0/524444808 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5646c6bd66b0 0x5646c6fc5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.174+0000 7fcb97fff640 1 -- 172.21.15.5:0/524444808 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5646c6bd66b0 msgr2=0x5646c6fc5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:20.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.174+0000 7fcb97fff640 1 --2- 172.21.15.5:0/524444808 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5646c6bd66b0 0x5646c6fc5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.174+0000 7fcb97fff640 1 -- 172.21.15.5:0/524444808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646c6bf7390 msgr2=0x5646c70541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:20.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.174+0000 7fcb97fff640 1 --2- 172.21.15.5:0/524444808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646c6bf7390 0x5646c70541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.174+0000 7fcb97fff640 1 -- 172.21.15.5:0/524444808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646c6fbcef0 con 0x5646c6bce4d0 2024-05-20T18:32:20.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.174+0000 7fcb977fe640 1 --2- 172.21.15.5:0/524444808 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5646c6bd66b0 0x5646c6fc5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:32:20.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.174+0000 7fcb97fff640 1 --2- 172.21.15.5:0/524444808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646c6bce4d0 0x5646c6fc2d40 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fcb88003f20 tx=0x7fcb88024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ed7a697a4b2348 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:20.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.174+0000 7fcb7d7fa640 1 -- 172.21.15.5:0/524444808 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb88027050 con 0x5646c6bce4d0 2024-05-20T18:32:20.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.174+0000 7fcb7d7fa640 1 -- 172.21.15.5:0/524444808 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb8801e040 con 0x5646c6bce4d0 2024-05-20T18:32:20.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.174+0000 7fcb7d7fa640 1 -- 172.21.15.5:0/524444808 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb88006130 con 0x5646c6bce4d0 2024-05-20T18:32:20.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.175+0000 7fcba51fa740 1 -- 172.21.15.5:0/524444808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646c6bce4d0 msgr2=0x5646c6fc2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:20.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.175+0000 7fcba51fa740 1 --2- 172.21.15.5:0/524444808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646c6bce4d0 0x5646c6fc2d40 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fcb88003f20 tx=0x7fcb88024000 comp rx=0 tx=0).stop 2024-05-20T18:32:20.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.175+0000 7fcba51fa740 1 -- 172.21.15.5:0/524444808 shutdown_connections 2024-05-20T18:32:20.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.175+0000 7fcba51fa740 1 --2- 172.21.15.5:0/524444808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646c6bf7390 0x5646c70541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.175+0000 7fcba51fa740 1 --2- 172.21.15.5:0/524444808 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5646c6bd66b0 0x5646c6fc5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.175+0000 7fcba51fa740 1 --2- 172.21.15.5:0/524444808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646c6bce4d0 0x5646c6fc2d40 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.175+0000 7fcba51fa740 1 -- 172.21.15.5:0/524444808 >> 172.21.15.5:0/524444808 conn(0x5646c6beb4e0 msgr2=0x5646c6fbb850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:20.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.175+0000 7fcba51fa740 1 -- 172.21.15.5:0/524444808 shutdown_connections 2024-05-20T18:32:20.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.175+0000 7fcba51fa740 1 -- 172.21.15.5:0/524444808 wait complete. 2024-05-20T18:32:20.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.176+0000 7fcba51fa740 1 -- start start 2024-05-20T18:32:20.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.176+0000 7fcba51fa740 1 Processor -- start 2024-05-20T18:32:20.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.177+0000 7fcba51fa740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646c6bce4d0 0x5646c6fc1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.177+0000 7fcba51fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646c6bd66b0 0x5646c6f97910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.178+0000 7fcba51fa740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5646c6bf7390 0x5646c6fbb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.178+0000 7fcba51fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcb980afb60 con 0x5646c6bd66b0 2024-05-20T18:32:20.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.178+0000 7fcba51fa740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5646c7003ff0 con 0x5646c6bce4d0 2024-05-20T18:32:20.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.178+0000 7fcba51fa740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5646c6fc21c0 con 0x5646c6bf7390 2024-05-20T18:32:20.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.178+0000 7fcba51fa740 10 client.0 _ll_register_callbacks cb 0x7fcb88004110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:32:20.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.178+0000 7fcb977fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646c6bd66b0 0x5646c6f97910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.178+0000 7fcb977fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646c6bd66b0 0x5646c6f97910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60630/0 (socket says 172.21.15.5:60630) 2024-05-20T18:32:20.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.178+0000 7fcb977fe640 1 -- 172.21.15.5:0/1167968745 learned_addr learned my addr 172.21.15.5:0/1167968745 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:20.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.178+0000 7fcb97fff640 1 --2- 172.21.15.5:0/1167968745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646c6bce4d0 0x5646c6fc1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.178+0000 7fcb9c810640 1 --2- 172.21.15.5:0/1167968745 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5646c6bf7390 0x5646c6fbb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.178+0000 7fcb977fe640 1 -- 172.21.15.5:0/1167968745 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5646c6bf7390 msgr2=0x5646c6fbb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:20.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.178+0000 7fcb977fe640 1 --2- 172.21.15.5:0/1167968745 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5646c6bf7390 0x5646c6fbb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.178+0000 7fcb977fe640 1 -- 172.21.15.5:0/1167968745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646c6bce4d0 msgr2=0x5646c6fc1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:20.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.178+0000 7fcb977fe640 1 --2- 172.21.15.5:0/1167968745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646c6bce4d0 0x5646c6fc1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.178+0000 7fcb977fe640 1 -- 172.21.15.5:0/1167968745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646c6f91e10 con 0x5646c6bd66b0 2024-05-20T18:32:20.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.178+0000 7fcb977fe640 1 --2- 172.21.15.5:0/1167968745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646c6bd66b0 0x5646c6f97910 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c02a2b0 tx=0x7fcb8c02c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af78f5a51ead31e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:20.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.178+0000 7fcb76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:32:20.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.178+0000 7fcb76ffd640 1 -- 172.21.15.5:0/1167968745 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c04a070 con 0x5646c6bd66b0 2024-05-20T18:32:20.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.178+0000 7fcb76ffd640 1 -- 172.21.15.5:0/1167968745 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7fcb8c032960 con 0x5646c6bd66b0 2024-05-20T18:32:20.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.178+0000 7fcba51fa740 1 -- 172.21.15.5:0/1167968745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb8801e040 con 0x5646c6bd66b0 2024-05-20T18:32:20.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.179+0000 7fcb76ffd640 1 -- 172.21.15.5:0/1167968745 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fcb8c055050 con 0x5646c6bd66b0 2024-05-20T18:32:20.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.179+0000 7fcba51fa740 10 client.7122 fetch_fsmap learned FSMap version 51 2024-05-20T18:32:20.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.179+0000 7fcba51fa740 1 -- 172.21.15.5:0/1167968745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x5646c6ff2a90 con 0x5646c6bd66b0 2024-05-20T18:32:20.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.179+0000 7fcb76ffd640 1 -- 172.21.15.5:0/1167968745 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb8801e040 con 0x5646c6bd66b0 2024-05-20T18:32:20.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.179+0000 7fcb76ffd640 1 -- 172.21.15.5:0/1167968745 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7fcb8c058020 con 0x5646c6bd66b0 2024-05-20T18:32:20.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.179+0000 7fcba51fa740 10 client.7122 fetch_fsmap finished waiting for FSMap version 51 2024-05-20T18:32:20.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.179+0000 7fcba51fa740 10 client.7122 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-05-20T18:32:20.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.179+0000 7fcba51fa740 20 client.7122 populate_metadata read hostname 'smithi005' 2024-05-20T18:32:20.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.179+0000 7fcba51fa740 1 --2- 172.21.15.5:0/1167968745 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5646c6fc2380 0x5646c6fbbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.179+0000 7fcba51fa740 4 client.7122 mds_command: new command op to 6972 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T18:32:20.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.179+0000 7fcba51fa740 1 -- 172.21.15.5:0/1167968745 --> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5646c6f03780 con 0x5646c6fc2380 2024-05-20T18:32:20.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.179+0000 7fcb97fff640 1 --2- 172.21.15.5:0/1167968745 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5646c6fc2380 0x5646c6fbbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.179+0000 7fcb76ffd640 10 client.7122 ms_handle_connect on v2:172.21.15.5:6835/3866822752 2024-05-20T18:32:20.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.179+0000 7fcb97fff640 1 --2- 172.21.15.5:0/1167968745 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5646c6fc2380 0x5646c6fbbd90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cca08f974ff0646 server_cookie=7c66d58b1501e4ad in_seq=0 out_seq=0 2024-05-20T18:32:20.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.180+0000 7fcb76ffd640 1 -- 172.21.15.5:0/1167968745 <== mds.0 v2:172.21.15.5:6835/3866822752 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5646c6f03780 con 0x5646c6fc2380 2024-05-20T18:32:20.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.180+0000 7fcb76ffd640 10 client.7122 handle_command_reply: tid=0 2024-05-20T18:32:20.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.180+0000 7fcba51fa740 1 client.7122 shutdown 2024-05-20T18:32:20.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.180+0000 7fcba51fa740 1 -- 172.21.15.5:0/1167968745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646c6bd66b0 msgr2=0x5646c6f97910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:20.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.180+0000 7fcba51fa740 1 --2- 172.21.15.5:0/1167968745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646c6bd66b0 0x5646c6f97910 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c02a2b0 tx=0x7fcb8c02c070 comp rx=0 tx=0).stop 2024-05-20T18:32:20.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.180+0000 7fcba51fa740 1 -- 172.21.15.5:0/1167968745 shutdown_connections 2024-05-20T18:32:20.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.180+0000 7fcba51fa740 1 --2- 172.21.15.5:0/1167968745 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5646c6bf7390 0x5646c6fbb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.180+0000 7fcba51fa740 1 --2- 172.21.15.5:0/1167968745 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5646c6fc2380 0x5646c6fbbd90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.180+0000 7fcba51fa740 1 --2- 172.21.15.5:0/1167968745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646c6bd66b0 0x5646c6f97910 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.180+0000 7fcba51fa740 1 --2- 172.21.15.5:0/1167968745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5646c6bce4d0 0x5646c6fc1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.180+0000 7fcb76ffd640 0 client.7122 ms_handle_reset on v2:172.21.15.5:6835/3866822752 2024-05-20T18:32:20.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.180+0000 7fcba51fa740 1 -- 172.21.15.5:0/1167968745 >> 172.21.15.5:0/1167968745 conn(0x5646c6beb4e0 msgr2=0x5646c6fdf770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:20.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.180+0000 7fcba51fa740 1 -- 172.21.15.5:0/1167968745 shutdown_connections 2024-05-20T18:32:20.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.180+0000 7fcba51fa740 1 -- 172.21.15.5:0/1167968745 wait complete. 2024-05-20T18:32:20.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.180+0000 7fcba51fa740 20 client.7122 trim_cache size 0 max 16384 2024-05-20T18:32:20.186 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:32:20.186 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-20T18:32:20.186 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-20T18:32:20.186 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:32:20.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.181+0000 7fcb9e273640 1 -- 172.21.15.5:0/2420415215 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcb8406be30 msgr2=0x7fcb8406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:20.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.181+0000 7fcb9e273640 1 --2- 172.21.15.5:0/2420415215 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcb8406be30 0x7fcb8406e340 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcb88009560 tx=0x7fcb88009190 comp rx=0 tx=0).stop 2024-05-20T18:32:20.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.181+0000 7fcb9e273640 1 -- 172.21.15.5:0/2420415215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb9815d460 msgr2=0x7fcb98140ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.181+0000 7fcb9e273640 1 --2- 172.21.15.5:0/2420415215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb9815d460 0x7fcb98140ca0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000c710 tx=0x7fcb9000cbd0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.181+0000 7fcb9e273640 1 -- 172.21.15.5:0/2420415215 shutdown_connections 2024-05-20T18:32:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.181+0000 7fcb9e273640 1 --2- 172.21.15.5:0/2420415215 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcb8406be30 0x7fcb8406e340 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.181+0000 7fcb9e273640 1 --2- 172.21.15.5:0/2420415215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb9815d460 0x7fcb98140ca0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.181+0000 7fcb9e273640 1 --2- 172.21.15.5:0/2420415215 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcb980d1780 0x7fcb980cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.181+0000 7fcb9e273640 1 --2- 172.21.15.5:0/2420415215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcb9809a4f0 0x7fcb981f29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.181+0000 7fcb9e273640 1 -- 172.21.15.5:0/2420415215 >> 172.21.15.5:0/2420415215 conn(0x7fcb980b6620 msgr2=0x7fcb9815ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.181+0000 7fcb9e273640 1 -- 172.21.15.5:0/2420415215 shutdown_connections 2024-05-20T18:32:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.181+0000 7fcb9e273640 1 -- 172.21.15.5:0/2420415215 wait complete. 2024-05-20T18:32:20.192 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T18:32:20.193 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:32:20.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.306+0000 7f0e909b5640 1 Processor -- start 2024-05-20T18:32:20.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.306+0000 7f0e909b5640 1 -- start start 2024-05-20T18:32:20.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.308+0000 7f0e909b5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0e8c0b47b0 0x7f0e8c0afad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.308+0000 7f0e909b5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e8c0b58f0 0x7f0e8c06f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.308+0000 7f0e909b5640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e8c0b39f0 con 0x7f0e8c15d060 2024-05-20T18:32:20.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.308+0000 7f0e909b5640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f0e8c0b3cd0 con 0x7f0e8c0b47b0 2024-05-20T18:32:20.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.308+0000 7f0e909b5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0e8c0b3fb0 con 0x7f0e8c0b58f0 2024-05-20T18:32:20.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.308+0000 7f0e8a59c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0e8c0b47b0 0x7f0e8c0afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.308+0000 7f0e8a59c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0e8c0b47b0 0x7f0e8c0afad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60636/0 (socket says 172.21.15.5:60636) 2024-05-20T18:32:20.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.308+0000 7f0e8a59c640 1 -- 172.21.15.5:0/3983720463 learned_addr learned my addr 172.21.15.5:0/3983720463 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:20.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.308+0000 7f0e8ad9d640 1 --2- 172.21.15.5:0/3983720463 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e8c0b58f0 0x7f0e8c06f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.308+0000 7f0e8a59c640 1 -- 172.21.15.5:0/3983720463 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e8c0b58f0 msgr2=0x7f0e8c06f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:20.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.308+0000 7f0e8a59c640 1 --2- 172.21.15.5:0/3983720463 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e8c0b58f0 0x7f0e8c06f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.308+0000 7f0e8a59c640 1 -- 172.21.15.5:0/3983720463 >> v1:172.21.15.63:6789/0 conn(0x7f0e8c15d060 legacy=0x7f0e8c0b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:20.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.308+0000 7f0e8a59c640 1 -- 172.21.15.5:0/3983720463 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e8c0b4290 con 0x7f0e8c0b47b0 2024-05-20T18:32:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.309+0000 7f0e8a59c640 1 --2- 172.21.15.5:0/3983720463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e8c0b47b0 0x7f0e8c0afad0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009870 tx=0x7f0e7c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18e0036dd2bacc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.309+0000 7f0e8959a640 1 -- 172.21.15.5:0/3983720463 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c037070 con 0x7f0e8c0b47b0 2024-05-20T18:32:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.309+0000 7f0e8959a640 1 -- 172.21.15.5:0/3983720463 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e7c032440 con 0x7f0e8c0b47b0 2024-05-20T18:32:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.309+0000 7f0e8959a640 1 -- 172.21.15.5:0/3983720463 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c032810 con 0x7f0e8c0b47b0 2024-05-20T18:32:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.310+0000 7f0e909b5640 1 -- 172.21.15.5:0/3983720463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e8c0b47b0 msgr2=0x7f0e8c0afad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.310+0000 7f0e909b5640 1 --2- 172.21.15.5:0/3983720463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e8c0b47b0 0x7f0e8c0afad0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009870 tx=0x7f0e7c02a1c0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.310+0000 7f0e909b5640 1 -- 172.21.15.5:0/3983720463 shutdown_connections 2024-05-20T18:32:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.310+0000 7f0e909b5640 1 --2- 172.21.15.5:0/3983720463 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e8c0b58f0 0x7f0e8c06f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.310+0000 7f0e909b5640 1 --2- 172.21.15.5:0/3983720463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e8c0b47b0 0x7f0e8c0afad0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.310+0000 7f0e909b5640 1 -- 172.21.15.5:0/3983720463 >> 172.21.15.5:0/3983720463 conn(0x7f0e8c0cd2a0 msgr2=0x7f0e8c0cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:20.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.310+0000 7f0e909b5640 1 -- 172.21.15.5:0/3983720463 shutdown_connections 2024-05-20T18:32:20.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.310+0000 7f0e909b5640 1 -- 172.21.15.5:0/3983720463 wait complete. 2024-05-20T18:32:20.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.312+0000 7f0e909b5640 1 Processor -- start 2024-05-20T18:32:20.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.312+0000 7f0e909b5640 1 -- start start 2024-05-20T18:32:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.313+0000 7f0e909b5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e8c15d060 0x7f0e8c1f6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.313+0000 7f0e909b5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e8c1f0130 0x7f0e8c14ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.313+0000 7f0e909b5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e8c150500 0x7f0e8c15d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.313+0000 7f0e909b5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c159de0 con 0x7f0e8c1f0130 2024-05-20T18:32:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.313+0000 7f0e909b5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c159c60 con 0x7f0e8c150500 2024-05-20T18:32:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.313+0000 7f0e909b5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0e8c06f910 con 0x7f0e8c15d060 2024-05-20T18:32:20.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.314+0000 7f0e89d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e8c1f0130 0x7f0e8c14ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.314+0000 7f0e89d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e8c1f0130 0x7f0e8c14ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60648/0 (socket says 172.21.15.5:60648) 2024-05-20T18:32:20.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.314+0000 7f0e89d9b640 1 -- 172.21.15.5:0/1531684079 learned_addr learned my addr 172.21.15.5:0/1531684079 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:20.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.314+0000 7f0e8a59c640 1 --2- 172.21.15.5:0/1531684079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e8c15d060 0x7f0e8c1f6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.314+0000 7f0e8ad9d640 1 --2- 172.21.15.5:0/1531684079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e8c150500 0x7f0e8c15d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.314+0000 7f0e89d9b640 1 -- 172.21.15.5:0/1531684079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e8c15d060 msgr2=0x7f0e8c1f6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:20.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.314+0000 7f0e89d9b640 1 --2- 172.21.15.5:0/1531684079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e8c15d060 0x7f0e8c1f6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.314+0000 7f0e89d9b640 1 -- 172.21.15.5:0/1531684079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e8c150500 msgr2=0x7f0e8c15d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:20.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.314+0000 7f0e89d9b640 1 --2- 172.21.15.5:0/1531684079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e8c150500 0x7f0e8c15d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.314+0000 7f0e89d9b640 1 -- 172.21.15.5:0/1531684079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e8c150910 con 0x7f0e8c1f0130 2024-05-20T18:32:20.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.314+0000 7f0e8ad9d640 1 --2- 172.21.15.5:0/1531684079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e8c150500 0x7f0e8c15d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:32:20.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.314+0000 7f0e89d9b640 1 --2- 172.21.15.5:0/1531684079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e8c1f0130 0x7f0e8c14ffc0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400c3c0 tx=0x7f0e7400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:20.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.314+0000 7f0e82ffd640 1 -- 172.21.15.5:0/1531684079 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e740052d0 con 0x7f0e8c1f0130 2024-05-20T18:32:20.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.314+0000 7f0e82ffd640 1 -- 172.21.15.5:0/1531684079 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e74005490 con 0x7f0e8c1f0130 2024-05-20T18:32:20.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.314+0000 7f0e909b5640 1 -- 172.21.15.5:0/1531684079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e8c15f060 con 0x7f0e8c1f0130 2024-05-20T18:32:20.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.314+0000 7f0e82ffd640 1 -- 172.21.15.5:0/1531684079 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e74007260 con 0x7f0e8c1f0130 2024-05-20T18:32:20.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.314+0000 7f0e909b5640 1 -- 172.21.15.5:0/1531684079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e8c1f34c0 con 0x7f0e8c1f0130 2024-05-20T18:32:20.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.315+0000 7f0e909b5640 1 -- 172.21.15.5:0/1531684079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e8c075b00 con 0x7f0e8c1f0130 2024-05-20T18:32:20.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.316+0000 7f0e82ffd640 1 -- 172.21.15.5:0/1531684079 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0e74007400 con 0x7f0e8c1f0130 2024-05-20T18:32:20.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.316+0000 7f0e82ffd640 1 --2- 172.21.15.5:0/1531684079 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0e6c06be30 0x7f0e6c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.317+0000 7f0e8a59c640 1 --2- 172.21.15.5:0/1531684079 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0e6c06be30 0x7f0e6c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.317+0000 7f0e82ffd640 1 -- 172.21.15.5:0/1531684079 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f0e7408a670 con 0x7f0e8c1f0130 2024-05-20T18:32:20.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.317+0000 7f0e8a59c640 1 --2- 172.21.15.5:0/1531684079 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0e6c06be30 0x7f0e6c06e340 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c034000 tx=0x7f0e7c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:20.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.321+0000 7f0e82ffd640 1 -- 172.21.15.5:0/1531684079 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0e8c150910 con 0x7f0e8c1f0130 2024-05-20T18:32:20.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.540+0000 7f0e909b5640 1 -- 172.21.15.5:0/1531684079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e8c15eb90 con 0x7f0e8c1f0130 2024-05-20T18:32:20.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.541+0000 7f0e82ffd640 1 -- 172.21.15.5:0/1531684079 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5175 (secure 0 0 0) 0x7f0e74058d10 con 0x7f0e8c1f0130 2024-05-20T18:32:20.542 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:32:20.542 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1890759603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1890759603},{"type":"v1","addr":"172.21.15.63:6834","nonce":1890759603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5825,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/700447403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":700447403},{"type":"v1","addr":"172.21.15.63:6835","nonce":700447403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3382943275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3382943275},{"type":"v1","addr":"172.21.15.5:6837","nonce":3382943275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:31:30.164179+0000","modified":"2024-05-20T18:31:51.853233+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6972":{"gid":6972,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6836/3866822752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":3866822752},{"type":"v1","addr":"172.21.15.5:6836","nonce":3866822752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6972,"qdb_cluster":[6972]},"id":5}]} 2024-05-20T18:32:20.542 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 51 2024-05-20T18:32:20.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.543+0000 7f0e909b5640 1 -- 172.21.15.5:0/1531684079 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0e6c06be30 msgr2=0x7f0e6c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:20.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.543+0000 7f0e909b5640 1 --2- 172.21.15.5:0/1531684079 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0e6c06be30 0x7f0e6c06e340 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c034000 tx=0x7f0e7c02e000 comp rx=0 tx=0).stop 2024-05-20T18:32:20.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.543+0000 7f0e909b5640 1 -- 172.21.15.5:0/1531684079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e8c1f0130 msgr2=0x7f0e8c14ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:20.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.543+0000 7f0e909b5640 1 --2- 172.21.15.5:0/1531684079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e8c1f0130 0x7f0e8c14ffc0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400c3c0 tx=0x7f0e7400c7c0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.543+0000 7f0e909b5640 1 -- 172.21.15.5:0/1531684079 shutdown_connections 2024-05-20T18:32:20.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.543+0000 7f0e909b5640 1 --2- 172.21.15.5:0/1531684079 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0e6c06be30 0x7f0e6c06e340 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.543+0000 7f0e909b5640 1 --2- 172.21.15.5:0/1531684079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e8c150500 0x7f0e8c15d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.543+0000 7f0e909b5640 1 --2- 172.21.15.5:0/1531684079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e8c1f0130 0x7f0e8c14ffc0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.543+0000 7f0e909b5640 1 --2- 172.21.15.5:0/1531684079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e8c15d060 0x7f0e8c1f6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.543+0000 7f0e909b5640 1 -- 172.21.15.5:0/1531684079 >> 172.21.15.5:0/1531684079 conn(0x7f0e8c0cd2a0 msgr2=0x7f0e8c158e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:20.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.543+0000 7f0e909b5640 1 -- 172.21.15.5:0/1531684079 shutdown_connections 2024-05-20T18:32:20.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.543+0000 7f0e909b5640 1 -- 172.21.15.5:0/1531684079 wait complete. 2024-05-20T18:32:20.556 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-05-20T18:32:20.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.672+0000 7f607d721640 1 Processor -- start 2024-05-20T18:32:20.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.672+0000 7f607d721640 1 -- start start 2024-05-20T18:32:20.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.674+0000 7f607d721640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f607815d530 0x7f607815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.674+0000 7f607d721640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6078070850 0x7f6078070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.674+0000 7f607d721640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60780cd580 con 0x7f60780b1700 2024-05-20T18:32:20.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.674+0000 7f607d721640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f60780ce190 con 0x7f6078070850 2024-05-20T18:32:20.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.674+0000 7f607d721640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f60780cd860 con 0x7f607815d530 2024-05-20T18:32:20.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.674+0000 7f60777fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6078070850 0x7f6078070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.674+0000 7f60777fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6078070850 0x7f6078070c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60650/0 (socket says 172.21.15.5:60650) 2024-05-20T18:32:20.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.674+0000 7f60777fe640 1 -- 172.21.15.5:0/191649344 learned_addr learned my addr 172.21.15.5:0/191649344 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:20.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.674+0000 7f6076ffd640 1 --2- 172.21.15.5:0/191649344 >> v2:172.21.15.63:3300/0 conn(0x7f607815d530 0x7f607815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.674+0000 7f60777fe640 1 -- 172.21.15.5:0/191649344 >> v2:172.21.15.63:3300/0 conn(0x7f607815d530 msgr2=0x7f607815aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:20.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.674+0000 7f60777fe640 1 --2- 172.21.15.5:0/191649344 >> v2:172.21.15.63:3300/0 conn(0x7f607815d530 0x7f607815aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.674+0000 7f60777fe640 1 -- 172.21.15.5:0/191649344 >> v1:172.21.15.63:6789/0 conn(0x7f60780b1700 legacy=0x7f60780b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:20.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.674+0000 7f60777fe640 1 -- 172.21.15.5:0/191649344 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60780cdb40 con 0x7f6078070850 2024-05-20T18:32:20.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.675+0000 7f60777fe640 1 --2- 172.21.15.5:0/191649344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6078070850 0x7f6078070c60 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f607804f790 tx=0x7f607002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c745e1925f5a34 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:20.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.675+0000 7f6075ffb640 1 -- 172.21.15.5:0/191649344 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6070036070 con 0x7f6078070850 2024-05-20T18:32:20.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.675+0000 7f6075ffb640 1 -- 172.21.15.5:0/191649344 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6070031600 con 0x7f6078070850 2024-05-20T18:32:20.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.675+0000 7f6075ffb640 1 -- 172.21.15.5:0/191649344 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60700319d0 con 0x7f6078070850 2024-05-20T18:32:20.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.676+0000 7f607d721640 1 -- 172.21.15.5:0/191649344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6078070850 msgr2=0x7f6078070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:20.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.676+0000 7f607d721640 1 --2- 172.21.15.5:0/191649344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6078070850 0x7f6078070c60 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f607804f790 tx=0x7f607002a6f0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.676+0000 7f607d721640 1 -- 172.21.15.5:0/191649344 shutdown_connections 2024-05-20T18:32:20.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.676+0000 7f607d721640 1 --2- 172.21.15.5:0/191649344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6078070850 0x7f6078070c60 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.676+0000 7f607d721640 1 --2- 172.21.15.5:0/191649344 >> v2:172.21.15.63:3300/0 conn(0x7f607815d530 0x7f607815aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.676+0000 7f607d721640 1 -- 172.21.15.5:0/191649344 >> 172.21.15.5:0/191649344 conn(0x7f60780d29d0 msgr2=0x7f60780d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:20.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.676+0000 7f607d721640 1 -- 172.21.15.5:0/191649344 shutdown_connections 2024-05-20T18:32:20.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.676+0000 7f607d721640 1 -- 172.21.15.5:0/191649344 wait complete. 2024-05-20T18:32:20.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.678+0000 7f607d721640 1 Processor -- start 2024-05-20T18:32:20.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.678+0000 7f607d721640 1 -- start start 2024-05-20T18:32:20.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.679+0000 7f607d721640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6078070850 0x7f607806d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.679+0000 7f607d721640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60780b1700 0x7f60781f55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.679+0000 7f607d721640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f607815d530 0x7f60780cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.679+0000 7f607d721640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f60780711a0 con 0x7f60780b1700 2024-05-20T18:32:20.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.679+0000 7f607d721640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f60780cd230 con 0x7f607815d530 2024-05-20T18:32:20.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.679+0000 7f607d721640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6078071320 con 0x7f6078070850 2024-05-20T18:32:20.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.679+0000 7f60767fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60780b1700 0x7f60781f55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.679+0000 7f60767fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60780b1700 0x7f60781f55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60662/0 (socket says 172.21.15.5:60662) 2024-05-20T18:32:20.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.679+0000 7f60767fc640 1 -- 172.21.15.5:0/3266620834 learned_addr learned my addr 172.21.15.5:0/3266620834 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:20.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.680+0000 7f6076ffd640 1 --2- 172.21.15.5:0/3266620834 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6078070850 0x7f607806d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.680+0000 7f60777fe640 1 --2- 172.21.15.5:0/3266620834 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f607815d530 0x7f60780cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.680+0000 7f60767fc640 1 -- 172.21.15.5:0/3266620834 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6078070850 msgr2=0x7f607806d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:20.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.680+0000 7f60767fc640 1 --2- 172.21.15.5:0/3266620834 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6078070850 0x7f607806d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.680+0000 7f60767fc640 1 -- 172.21.15.5:0/3266620834 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f607815d530 msgr2=0x7f60780cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:20.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.680+0000 7f60767fc640 1 --2- 172.21.15.5:0/3266620834 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f607815d530 0x7f60780cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.680+0000 7f60767fc640 1 -- 172.21.15.5:0/3266620834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60780ca9b0 con 0x7f60780b1700 2024-05-20T18:32:20.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.680+0000 7f60767fc640 1 --2- 172.21.15.5:0/3266620834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60780b1700 0x7f60781f55a0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f606c00b420 tx=0x7f606c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:20.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.680+0000 7f605f7fe640 1 -- 172.21.15.5:0/3266620834 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c011830 con 0x7f60780b1700 2024-05-20T18:32:20.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.680+0000 7f605f7fe640 1 -- 172.21.15.5:0/3266620834 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f606c016070 con 0x7f60780b1700 2024-05-20T18:32:20.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.680+0000 7f607d721640 1 -- 172.21.15.5:0/3266620834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60780cac20 con 0x7f60780b1700 2024-05-20T18:32:20.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.680+0000 7f605f7fe640 1 -- 172.21.15.5:0/3266620834 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c011c10 con 0x7f60780b1700 2024-05-20T18:32:20.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.680+0000 7f607d721640 1 -- 172.21.15.5:0/3266620834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60780cb1e0 con 0x7f60780b1700 2024-05-20T18:32:20.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.682+0000 7f605f7fe640 1 -- 172.21.15.5:0/3266620834 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f606c01b460 con 0x7f60780b1700 2024-05-20T18:32:20.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.683+0000 7f605f7fe640 1 --2- 172.21.15.5:0/3266620834 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f606406be30 0x7f606406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.683+0000 7f605f7fe640 1 -- 172.21.15.5:0/3266620834 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f606c08abe0 con 0x7f60780b1700 2024-05-20T18:32:20.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.683+0000 7f6076ffd640 1 --2- 172.21.15.5:0/3266620834 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f606406be30 0x7f606406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.683+0000 7f6076ffd640 1 --2- 172.21.15.5:0/3266620834 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f606406be30 0x7f606406e340 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6068006fd0 tx=0x7f6068008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:20.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.684+0000 7f60845c8740 1 Processor -- start 2024-05-20T18:32:20.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.684+0000 7f60845c8740 1 -- start start 2024-05-20T18:32:20.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.685+0000 7f60845c8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626c88d3350 0x5626c8c9e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.685+0000 7f60845c8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626c88aa490 0x5626c8cb93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.685+0000 7f60845c8740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5626c88b2670 0x5626c8cb98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.685+0000 7f60845c8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5626c8bd7a30 con 0x5626c88aa490 2024-05-20T18:32:20.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.685+0000 7f60845c8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5626c8b75590 con 0x5626c88d3350 2024-05-20T18:32:20.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.685+0000 7f60845c8740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5626c8bd39f0 con 0x5626c88b2670 2024-05-20T18:32:20.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.685+0000 7f60767fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626c88aa490 0x5626c8cb93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.685+0000 7f60767fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626c88aa490 0x5626c8cb93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60670/0 (socket says 172.21.15.5:60670) 2024-05-20T18:32:20.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.685+0000 7f60767fc640 1 -- 172.21.15.5:0/4227936617 learned_addr learned my addr 172.21.15.5:0/4227936617 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:20.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.685+0000 7f6076ffd640 1 --2- 172.21.15.5:0/4227936617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626c88d3350 0x5626c8c9e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.685+0000 7f60777fe640 1 --2- 172.21.15.5:0/4227936617 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5626c88b2670 0x5626c8cb98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.686+0000 7f60767fc640 1 -- 172.21.15.5:0/4227936617 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5626c88b2670 msgr2=0x5626c8cb98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:20.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.686+0000 7f60767fc640 1 --2- 172.21.15.5:0/4227936617 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5626c88b2670 0x5626c8cb98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.686+0000 7f60767fc640 1 -- 172.21.15.5:0/4227936617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626c88d3350 msgr2=0x5626c8c9e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:20.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.686+0000 7f60767fc640 1 --2- 172.21.15.5:0/4227936617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626c88d3350 0x5626c8c9e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.686+0000 7f60767fc640 1 -- 172.21.15.5:0/4227936617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626c89b77c0 con 0x5626c88aa490 2024-05-20T18:32:20.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.686+0000 7f6076ffd640 1 --2- 172.21.15.5:0/4227936617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626c88d3350 0x5626c8c9e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:32:20.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.686+0000 7f60767fc640 1 --2- 172.21.15.5:0/4227936617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626c88aa490 0x5626c8cb93a0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f606c088e70 tx=0x7f606c058ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383172298bb6f96f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:20.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.686+0000 7f605cff9640 1 -- 172.21.15.5:0/4227936617 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c087070 con 0x5626c88aa490 2024-05-20T18:32:20.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.686+0000 7f605cff9640 1 -- 172.21.15.5:0/4227936617 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f606c056f20 con 0x5626c88aa490 2024-05-20T18:32:20.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.686+0000 7f605cff9640 1 -- 172.21.15.5:0/4227936617 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c058070 con 0x5626c88aa490 2024-05-20T18:32:20.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.687+0000 7f60845c8740 1 -- 172.21.15.5:0/4227936617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626c88aa490 msgr2=0x5626c8cb93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:20.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.687+0000 7f60845c8740 1 --2- 172.21.15.5:0/4227936617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626c88aa490 0x5626c8cb93a0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f606c088e70 tx=0x7f606c058ce0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.687+0000 7f60845c8740 1 -- 172.21.15.5:0/4227936617 shutdown_connections 2024-05-20T18:32:20.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.687+0000 7f60845c8740 1 --2- 172.21.15.5:0/4227936617 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5626c88b2670 0x5626c8cb98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.687+0000 7f60845c8740 1 --2- 172.21.15.5:0/4227936617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626c88aa490 0x5626c8cb93a0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.687+0000 7f60845c8740 1 --2- 172.21.15.5:0/4227936617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626c88d3350 0x5626c8c9e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.687+0000 7f60845c8740 1 -- 172.21.15.5:0/4227936617 >> 172.21.15.5:0/4227936617 conn(0x5626c88c74a0 msgr2=0x5626c8c73cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:20.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.687+0000 7f60845c8740 1 -- 172.21.15.5:0/4227936617 shutdown_connections 2024-05-20T18:32:20.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.687+0000 7f60845c8740 1 -- 172.21.15.5:0/4227936617 wait complete. 2024-05-20T18:32:20.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.688+0000 7f60845c8740 1 -- start start 2024-05-20T18:32:20.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.688+0000 7f60845c8740 1 Processor -- start 2024-05-20T18:32:20.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.689+0000 7f60845c8740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5626c88aa490 0x5626c8cc83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.689+0000 7f60845c8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626c88b2670 0x5626c8c9fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.689+0000 7f60845c8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626c88d3350 0x5626c8ca0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.689+0000 7f60845c8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5626c8b73060 con 0x5626c88d3350 2024-05-20T18:32:20.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.689+0000 7f60845c8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5626c8cc8c60 con 0x5626c88b2670 2024-05-20T18:32:20.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.689+0000 7f60845c8740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5626c8ccbde0 con 0x5626c88aa490 2024-05-20T18:32:20.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.689+0000 7f60845c8740 10 client.0 _ll_register_callbacks cb 0x5626c8b97dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:32:20.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.689+0000 7f60777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626c88d3350 0x5626c8ca0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.689+0000 7f60777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626c88d3350 0x5626c8ca0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60680/0 (socket says 172.21.15.5:60680) 2024-05-20T18:32:20.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.689+0000 7f60777fe640 1 -- 172.21.15.5:0/852132794 learned_addr learned my addr 172.21.15.5:0/852132794 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:20.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.689+0000 7f6076ffd640 1 --2- 172.21.15.5:0/852132794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5626c88aa490 0x5626c8cc83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.689+0000 7f60767fc640 1 --2- 172.21.15.5:0/852132794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626c88b2670 0x5626c8c9fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.690+0000 7f60777fe640 1 -- 172.21.15.5:0/852132794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5626c88aa490 msgr2=0x5626c8cc83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:20.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.690+0000 7f60777fe640 1 --2- 172.21.15.5:0/852132794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5626c88aa490 0x5626c8cc83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.690+0000 7f60777fe640 1 -- 172.21.15.5:0/852132794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626c88b2670 msgr2=0x5626c8c9fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.690+0000 7f60777fe640 1 --2- 172.21.15.5:0/852132794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626c88b2670 0x5626c8c9fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.690+0000 7f60777fe640 1 -- 172.21.15.5:0/852132794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626c8a17f50 con 0x5626c88d3350 2024-05-20T18:32:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.690+0000 7f60767fc640 1 --2- 172.21.15.5:0/852132794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626c88b2670 0x5626c8c9fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:32:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.690+0000 7f60777fe640 1 --2- 172.21.15.5:0/852132794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626c88d3350 0x5626c8ca0110 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f607002a730 tx=0x7f6070004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14a190e2a4131af server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.690+0000 7f6055ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:32:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.690+0000 7f6055ffb640 1 -- 172.21.15.5:0/852132794 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6070036040 con 0x5626c88d3350 2024-05-20T18:32:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.690+0000 7f6055ffb640 1 -- 172.21.15.5:0/852132794 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f6070031bb0 con 0x5626c88d3350 2024-05-20T18:32:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.690+0000 7f60845c8740 1 -- 172.21.15.5:0/852132794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f606c056f20 con 0x5626c88d3350 2024-05-20T18:32:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.690+0000 7f6055ffb640 1 -- 172.21.15.5:0/852132794 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f607004d050 con 0x5626c88d3350 2024-05-20T18:32:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.690+0000 7f60845c8740 10 client.7140 fetch_fsmap learned FSMap version 51 2024-05-20T18:32:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.690+0000 7f60845c8740 1 -- 172.21.15.5:0/852132794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x5626c88ab0b0 con 0x5626c88d3350 2024-05-20T18:32:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.690+0000 7f6055ffb640 1 -- 172.21.15.5:0/852132794 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f606c056f20 con 0x5626c88d3350 2024-05-20T18:32:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.690+0000 7f6055ffb640 1 -- 172.21.15.5:0/852132794 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f6070050020 con 0x5626c88d3350 2024-05-20T18:32:20.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.690+0000 7f60845c8740 10 client.7140 fetch_fsmap finished waiting for FSMap version 51 2024-05-20T18:32:20.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.690+0000 7f60845c8740 10 client.7140 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-20T18:32:20.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.690+0000 7f60845c8740 20 client.7140 populate_metadata read hostname 'smithi005' 2024-05-20T18:32:20.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.690+0000 7f60845c8740 1 --2- 172.21.15.5:0/852132794 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5626c8892130 0x5626c8cb4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.690+0000 7f60845c8740 4 client.7140 mds_command: new command op to 6972 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:32:20.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.690+0000 7f60845c8740 1 -- 172.21.15.5:0/852132794 --> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626c8be5160 con 0x5626c8892130 2024-05-20T18:32:20.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.691+0000 7f6076ffd640 1 --2- 172.21.15.5:0/852132794 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5626c8892130 0x5626c8cb4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.691+0000 7f6055ffb640 10 client.7140 ms_handle_connect on v2:172.21.15.5:6835/3866822752 2024-05-20T18:32:20.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.691+0000 7f6076ffd640 1 --2- 172.21.15.5:0/852132794 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5626c8892130 0x5626c8cb4000 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=303d29c13e0c8991 server_cookie=91281086f60c3c76 in_seq=0 out_seq=0 2024-05-20T18:32:20.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.692+0000 7f6055ffb640 1 -- 172.21.15.5:0/852132794 <== mds.0 v2:172.21.15.5:6835/3866822752 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5626c8be5160 con 0x5626c8892130 2024-05-20T18:32:20.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.692+0000 7f6055ffb640 10 client.7140 handle_command_reply: tid=0 2024-05-20T18:32:20.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.692+0000 7f60845c8740 1 client.7140 shutdown 2024-05-20T18:32:20.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.692+0000 7f60845c8740 1 -- 172.21.15.5:0/852132794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626c88d3350 msgr2=0x5626c8ca0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:20.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.692+0000 7f60845c8740 1 --2- 172.21.15.5:0/852132794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626c88d3350 0x5626c8ca0110 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f607002a730 tx=0x7f6070004ab0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.692+0000 7f60845c8740 1 -- 172.21.15.5:0/852132794 shutdown_connections 2024-05-20T18:32:20.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.692+0000 7f60845c8740 1 --2- 172.21.15.5:0/852132794 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5626c8892130 0x5626c8cb4000 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.692+0000 7f60845c8740 1 --2- 172.21.15.5:0/852132794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626c88d3350 0x5626c8ca0110 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.692+0000 7f60845c8740 1 --2- 172.21.15.5:0/852132794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626c88b2670 0x5626c8c9fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.692+0000 7f6055ffb640 0 client.7140 ms_handle_reset on v2:172.21.15.5:6835/3866822752 2024-05-20T18:32:20.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.692+0000 7f60845c8740 1 --2- 172.21.15.5:0/852132794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5626c88aa490 0x5626c8cc83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.692+0000 7f60845c8740 1 -- 172.21.15.5:0/852132794 >> 172.21.15.5:0/852132794 conn(0x5626c88c74a0 msgr2=0x5626c8c9e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:20.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.692+0000 7f60845c8740 1 -- 172.21.15.5:0/852132794 shutdown_connections 2024-05-20T18:32:20.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.692+0000 7f60845c8740 1 -- 172.21.15.5:0/852132794 wait complete. 2024-05-20T18:32:20.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.692+0000 7f60845c8740 20 client.7140 trim_cache size 0 max 16384 2024-05-20T18:32:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.706+0000 7f60845c8740 1 Processor -- start 2024-05-20T18:32:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.706+0000 7f60845c8740 1 -- start start 2024-05-20T18:32:20.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.707+0000 7f60845c8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626c88aa490 0x5626c8cdc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.707+0000 7f60845c8740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5626c88b2670 0x5626c8c75420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.707+0000 7f60845c8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626c88d3350 0x5626c8c9a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.707+0000 7f60845c8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6078201340 con 0x5626c88d3350 2024-05-20T18:32:20.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.707+0000 7f60845c8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f60782006b0 con 0x5626c88aa490 2024-05-20T18:32:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.707+0000 7f60845c8740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6078200d00 con 0x5626c88b2670 2024-05-20T18:32:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.707+0000 7f60777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626c88d3350 0x5626c8c9a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.707+0000 7f60777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626c88d3350 0x5626c8c9a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60692/0 (socket says 172.21.15.5:60692) 2024-05-20T18:32:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.707+0000 7f60777fe640 1 -- 172.21.15.5:0/2354732822 learned_addr learned my addr 172.21.15.5:0/2354732822 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.708+0000 7f60767fc640 1 --2- 172.21.15.5:0/2354732822 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5626c88b2670 0x5626c8c75420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.708+0000 7f6076ffd640 1 --2- 172.21.15.5:0/2354732822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626c88aa490 0x5626c8cdc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.708+0000 7f60777fe640 1 -- 172.21.15.5:0/2354732822 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5626c88b2670 msgr2=0x5626c8c75420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.708+0000 7f60777fe640 1 --2- 172.21.15.5:0/2354732822 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5626c88b2670 0x5626c8c75420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.708+0000 7f60777fe640 1 -- 172.21.15.5:0/2354732822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626c88aa490 msgr2=0x5626c8cdc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.708+0000 7f60777fe640 1 --2- 172.21.15.5:0/2354732822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626c88aa490 0x5626c8cdc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.708+0000 7f60777fe640 1 -- 172.21.15.5:0/2354732822 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626c8cbddf0 con 0x5626c88d3350 2024-05-20T18:32:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.708+0000 7f60767fc640 1 --2- 172.21.15.5:0/2354732822 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5626c88b2670 0x5626c8c75420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:32:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.708+0000 7f6076ffd640 1 --2- 172.21.15.5:0/2354732822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626c88aa490 0x5626c8cdc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:32:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.708+0000 7f60777fe640 1 --2- 172.21.15.5:0/2354732822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626c88d3350 0x5626c8c9a3f0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f6070009db0 tx=0x7f607003d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a59b13783d4615 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.708+0000 7f6057fff640 1 -- 172.21.15.5:0/2354732822 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6070036040 con 0x5626c88d3350 2024-05-20T18:32:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.708+0000 7f6057fff640 1 -- 172.21.15.5:0/2354732822 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6070003b90 con 0x5626c88d3350 2024-05-20T18:32:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.708+0000 7f6057fff640 1 -- 172.21.15.5:0/2354732822 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6070031820 con 0x5626c88d3350 2024-05-20T18:32:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.709+0000 7f60845c8740 1 -- 172.21.15.5:0/2354732822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626c88d3350 msgr2=0x5626c8c9a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.709+0000 7f60845c8740 1 --2- 172.21.15.5:0/2354732822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626c88d3350 0x5626c8c9a3f0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f6070009db0 tx=0x7f607003d000 comp rx=0 tx=0).stop 2024-05-20T18:32:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.709+0000 7f60845c8740 1 -- 172.21.15.5:0/2354732822 shutdown_connections 2024-05-20T18:32:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.709+0000 7f60845c8740 1 --2- 172.21.15.5:0/2354732822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626c88d3350 0x5626c8c9a3f0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.709+0000 7f60845c8740 1 --2- 172.21.15.5:0/2354732822 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5626c88b2670 0x5626c8c75420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.709+0000 7f60845c8740 1 --2- 172.21.15.5:0/2354732822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626c88aa490 0x5626c8cdc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.709+0000 7f60845c8740 1 -- 172.21.15.5:0/2354732822 >> 172.21.15.5:0/2354732822 conn(0x5626c88c74a0 msgr2=0x5626c8c9fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.709+0000 7f60845c8740 1 -- 172.21.15.5:0/2354732822 shutdown_connections 2024-05-20T18:32:20.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.709+0000 7f60845c8740 1 -- 172.21.15.5:0/2354732822 wait complete. 2024-05-20T18:32:20.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.710+0000 7f60845c8740 1 -- start start 2024-05-20T18:32:20.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.710+0000 7f60845c8740 1 Processor -- start 2024-05-20T18:32:20.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.711+0000 7f60845c8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626c88aa490 0x5626c8d35e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.711+0000 7f60845c8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626c88b2670 0x5626c8c6d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.711+0000 7f60845c8740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5626c88d3350 0x5626c8c6dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.711+0000 7f60845c8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5626c8cbd300 con 0x5626c88b2670 2024-05-20T18:32:20.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.711+0000 7f60845c8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5626c8cc8ac0 con 0x5626c88aa490 2024-05-20T18:32:20.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.711+0000 7f60845c8740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5626c8cbd8c0 con 0x5626c88d3350 2024-05-20T18:32:20.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.711+0000 7f60845c8740 10 client.0 _ll_register_callbacks cb 0x7f6070034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:32:20.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.711+0000 7f60767fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626c88b2670 0x5626c8c6d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.711+0000 7f60767fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626c88b2670 0x5626c8c6d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60698/0 (socket says 172.21.15.5:60698) 2024-05-20T18:32:20.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.711+0000 7f60767fc640 1 -- 172.21.15.5:0/3351888408 learned_addr learned my addr 172.21.15.5:0/3351888408 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:20.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.711+0000 7f6076ffd640 1 --2- 172.21.15.5:0/3351888408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626c88aa490 0x5626c8d35e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.711+0000 7f60777fe640 1 --2- 172.21.15.5:0/3351888408 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5626c88d3350 0x5626c8c6dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.711+0000 7f60767fc640 1 -- 172.21.15.5:0/3351888408 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5626c88d3350 msgr2=0x5626c8c6dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:20.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.711+0000 7f60767fc640 1 --2- 172.21.15.5:0/3351888408 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5626c88d3350 0x5626c8c6dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.711+0000 7f60767fc640 1 -- 172.21.15.5:0/3351888408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626c88aa490 msgr2=0x5626c8d35e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:20.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.711+0000 7f60767fc640 1 --2- 172.21.15.5:0/3351888408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626c88aa490 0x5626c8d35e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.711+0000 7f60767fc640 1 -- 172.21.15.5:0/3351888408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626c8c9ed00 con 0x5626c88b2670 2024-05-20T18:32:20.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.711+0000 7f6076ffd640 1 --2- 172.21.15.5:0/3351888408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626c88aa490 0x5626c8d35e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:32:20.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.712+0000 7f60767fc640 1 --2- 172.21.15.5:0/3351888408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626c88b2670 0x5626c8c6d570 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f606c055080 tx=0x7f606c01d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e7fa276a11d439 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:20.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.712+0000 7f6055ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:32:20.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.712+0000 7f6055ffb640 1 -- 172.21.15.5:0/3351888408 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c087040 con 0x5626c88b2670 2024-05-20T18:32:20.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.712+0000 7f6055ffb640 1 -- 172.21.15.5:0/3351888408 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f606c056f20 con 0x5626c88b2670 2024-05-20T18:32:20.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.712+0000 7f60845c8740 1 -- 172.21.15.5:0/3351888408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6070003b90 con 0x5626c88b2670 2024-05-20T18:32:20.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.712+0000 7f6055ffb640 1 -- 172.21.15.5:0/3351888408 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f606c055290 con 0x5626c88b2670 2024-05-20T18:32:20.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.712+0000 7f60845c8740 10 client.7146 fetch_fsmap learned FSMap version 51 2024-05-20T18:32:20.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.712+0000 7f60845c8740 1 -- 172.21.15.5:0/3351888408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x5626c8c75c50 con 0x5626c88b2670 2024-05-20T18:32:20.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.712+0000 7f6055ffb640 1 -- 172.21.15.5:0/3351888408 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6070003b90 con 0x5626c88b2670 2024-05-20T18:32:20.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.712+0000 7f6055ffb640 1 -- 172.21.15.5:0/3351888408 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f606c058070 con 0x5626c88b2670 2024-05-20T18:32:20.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.712+0000 7f60845c8740 10 client.7146 fetch_fsmap finished waiting for FSMap version 51 2024-05-20T18:32:20.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.712+0000 7f60845c8740 10 client.7146 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-20T18:32:20.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.712+0000 7f60845c8740 20 client.7146 populate_metadata read hostname 'smithi005' 2024-05-20T18:32:20.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.712+0000 7f60845c8740 1 --2- 172.21.15.5:0/3351888408 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5626c8c74060 0x5626c8cb3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.712+0000 7f60845c8740 4 client.7146 mds_command: new command op to 6972 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-20T18:32:20.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.712+0000 7f60845c8740 1 -- 172.21.15.5:0/3351888408 --> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x5626c8bdf4b0 con 0x5626c8c74060 2024-05-20T18:32:20.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.712+0000 7f6076ffd640 1 --2- 172.21.15.5:0/3351888408 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5626c8c74060 0x5626c8cb3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.713+0000 7f6055ffb640 10 client.7146 ms_handle_connect on v2:172.21.15.5:6835/3866822752 2024-05-20T18:32:20.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.713+0000 7f6076ffd640 1 --2- 172.21.15.5:0/3351888408 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5626c8c74060 0x5626c8cb3f60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a07d555c8707247 server_cookie=26c2eb1972a6acee in_seq=0 out_seq=0 2024-05-20T18:32:20.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.713+0000 7f6055ffb640 1 -- 172.21.15.5:0/3351888408 <== mds.0 v2:172.21.15.5:6835/3866822752 1 ==== command_reply(tid 0: 0 ) ==== 8+0+237 (crc 0 0 0) 0x5626c8bdf4b0 con 0x5626c8c74060 2024-05-20T18:32:20.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.713+0000 7f6055ffb640 10 client.7146 handle_command_reply: tid=0 2024-05-20T18:32:20.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.713+0000 7f60845c8740 1 client.7146 shutdown 2024-05-20T18:32:20.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.713+0000 7f60845c8740 1 -- 172.21.15.5:0/3351888408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626c88b2670 msgr2=0x5626c8c6d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:20.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.713+0000 7f60845c8740 1 --2- 172.21.15.5:0/3351888408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626c88b2670 0x5626c8c6d570 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f606c055080 tx=0x7f606c01d040 comp rx=0 tx=0).stop 2024-05-20T18:32:20.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.713+0000 7f60845c8740 1 -- 172.21.15.5:0/3351888408 shutdown_connections 2024-05-20T18:32:20.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.713+0000 7f60845c8740 1 --2- 172.21.15.5:0/3351888408 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5626c88d3350 0x5626c8c6dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.713+0000 7f60845c8740 1 --2- 172.21.15.5:0/3351888408 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5626c8c74060 0x5626c8cb3f60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.713+0000 7f60845c8740 1 --2- 172.21.15.5:0/3351888408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626c88b2670 0x5626c8c6d570 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.713+0000 7f60845c8740 1 --2- 172.21.15.5:0/3351888408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626c88aa490 0x5626c8d35e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.713+0000 7f60845c8740 1 -- 172.21.15.5:0/3351888408 >> 172.21.15.5:0/3351888408 conn(0x5626c88c74a0 msgr2=0x5626c8ca1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:20.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.713+0000 7f6055ffb640 0 client.7146 ms_handle_reset on v2:172.21.15.5:6835/3866822752 2024-05-20T18:32:20.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.713+0000 7f60845c8740 1 -- 172.21.15.5:0/3351888408 shutdown_connections 2024-05-20T18:32:20.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.713+0000 7f60845c8740 1 -- 172.21.15.5:0/3351888408 wait complete. 2024-05-20T18:32:20.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.713+0000 7f60845c8740 20 client.7146 trim_cache size 0 max 16384 2024-05-20T18:32:20.721 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:32:20.722 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-05-20T18:32:20.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:32:20.722 INFO:teuthology.orchestra.run.smithi005.stdout: "damage_type": "dentry", 2024-05-20T18:32:20.722 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 4068266878, 2024-05-20T18:32:20.722 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 1099511627776, 2024-05-20T18:32:20.722 INFO:teuthology.orchestra.run.smithi005.stdout: "frag": "*", 2024-05-20T18:32:20.722 INFO:teuthology.orchestra.run.smithi005.stdout: "dname": "file_to_be_damaged", 2024-05-20T18:32:20.722 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_id": "head", 2024-05-20T18:32:20.722 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/subdir/file_to_be_damaged" 2024-05-20T18:32:20.722 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:32:20.722 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-05-20T18:32:20.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.714+0000 7f607d721640 1 -- 172.21.15.5:0/3266620834 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f606406be30 msgr2=0x7f606406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:20.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.714+0000 7f607d721640 1 --2- 172.21.15.5:0/3266620834 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f606406be30 0x7f606406e340 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6068006fd0 tx=0x7f6068008040 comp rx=0 tx=0).stop 2024-05-20T18:32:20.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.714+0000 7f607d721640 1 -- 172.21.15.5:0/3266620834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60780b1700 msgr2=0x7f60781f55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:20.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.714+0000 7f607d721640 1 --2- 172.21.15.5:0/3266620834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60780b1700 0x7f60781f55a0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f606c00b420 tx=0x7f606c00b8e0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.714+0000 7f607d721640 1 -- 172.21.15.5:0/3266620834 shutdown_connections 2024-05-20T18:32:20.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.714+0000 7f607d721640 1 --2- 172.21.15.5:0/3266620834 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f606406be30 0x7f606406e340 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.714+0000 7f607d721640 1 --2- 172.21.15.5:0/3266620834 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f607815d530 0x7f60780cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.714+0000 7f607d721640 1 --2- 172.21.15.5:0/3266620834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60780b1700 0x7f60781f55a0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.714+0000 7f607d721640 1 --2- 172.21.15.5:0/3266620834 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6078070850 0x7f607806d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.714+0000 7f607d721640 1 -- 172.21.15.5:0/3266620834 >> 172.21.15.5:0/3266620834 conn(0x7f60780d29d0 msgr2=0x7f60780b77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:20.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.714+0000 7f607d721640 1 -- 172.21.15.5:0/3266620834 shutdown_connections 2024-05-20T18:32:20.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.714+0000 7f607d721640 1 -- 172.21.15.5:0/3266620834 wait complete. 2024-05-20T18:32:20.726 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-20T18:32:20.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.839+0000 7f8054b3e640 1 Processor -- start 2024-05-20T18:32:20.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.839+0000 7f8054b3e640 1 -- start start 2024-05-20T18:32:20.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.841+0000 7f8054b3e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f80500d4120 0x7f80500d4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.841+0000 7f8054b3e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f80500d5030 0x7f805015f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.841+0000 7f8054b3e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80500d5440 con 0x7f805015fc60 2024-05-20T18:32:20.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.841+0000 7f8054b3e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f8050160380 con 0x7f80500d5030 2024-05-20T18:32:20.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.841+0000 7f8054b3e640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8050160660 con 0x7f80500d4120 2024-05-20T18:32:20.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.841+0000 7f804dd9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f80500d5030 0x7f805015f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.841+0000 7f804dd9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f80500d5030 0x7f805015f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60714/0 (socket says 172.21.15.5:60714) 2024-05-20T18:32:20.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.841+0000 7f804dd9b640 1 -- 172.21.15.5:0/265150485 learned_addr learned my addr 172.21.15.5:0/265150485 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:20.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.841+0000 7f804e59c640 1 --2- 172.21.15.5:0/265150485 >> v2:172.21.15.63:3300/0 conn(0x7f80500d4120 0x7f80500d4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.841+0000 7f804dd9b640 1 -- 172.21.15.5:0/265150485 >> v2:172.21.15.63:3300/0 conn(0x7f80500d4120 msgr2=0x7f80500d4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:20.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.841+0000 7f804dd9b640 1 --2- 172.21.15.5:0/265150485 >> v2:172.21.15.63:3300/0 conn(0x7f80500d4120 0x7f80500d4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.841+0000 7f804dd9b640 1 -- 172.21.15.5:0/265150485 >> v1:172.21.15.63:6789/0 conn(0x7f805015fc60 legacy=0x7f8050160960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:20.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.841+0000 7f804dd9b640 1 -- 172.21.15.5:0/265150485 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80500d3640 con 0x7f80500d5030 2024-05-20T18:32:20.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.842+0000 7f804dd9b640 1 --2- 172.21.15.5:0/265150485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80500d5030 0x7f805015f720 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f80500d27a0 tx=0x7f803802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2423bf74bd4e9d8b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:20.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.842+0000 7f804d59a640 1 -- 172.21.15.5:0/265150485 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038037070 con 0x7f80500d5030 2024-05-20T18:32:20.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.842+0000 7f804d59a640 1 -- 172.21.15.5:0/265150485 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8038032440 con 0x7f80500d5030 2024-05-20T18:32:20.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.842+0000 7f804d59a640 1 -- 172.21.15.5:0/265150485 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038032810 con 0x7f80500d5030 2024-05-20T18:32:20.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.843+0000 7f8054b3e640 1 -- 172.21.15.5:0/265150485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80500d5030 msgr2=0x7f805015f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:20.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.843+0000 7f8054b3e640 1 --2- 172.21.15.5:0/265150485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80500d5030 0x7f805015f720 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f80500d27a0 tx=0x7f803802a270 comp rx=0 tx=0).stop 2024-05-20T18:32:20.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.843+0000 7f8054b3e640 1 -- 172.21.15.5:0/265150485 shutdown_connections 2024-05-20T18:32:20.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.843+0000 7f8054b3e640 1 --2- 172.21.15.5:0/265150485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80500d5030 0x7f805015f720 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.843+0000 7f8054b3e640 1 --2- 172.21.15.5:0/265150485 >> v2:172.21.15.63:3300/0 conn(0x7f80500d4120 0x7f80500d4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.843+0000 7f8054b3e640 1 -- 172.21.15.5:0/265150485 >> 172.21.15.5:0/265150485 conn(0x7f805006b520 msgr2=0x7f805006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:20.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.843+0000 7f8054b3e640 1 -- 172.21.15.5:0/265150485 shutdown_connections 2024-05-20T18:32:20.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.843+0000 7f8054b3e640 1 -- 172.21.15.5:0/265150485 wait complete. 2024-05-20T18:32:20.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.845+0000 7f8054b3e640 1 Processor -- start 2024-05-20T18:32:20.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.845+0000 7f8054b3e640 1 -- start start 2024-05-20T18:32:20.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.846+0000 7f8054b3e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80500d4120 0x7f80501490a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.846+0000 7f8054b3e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80500d5030 0x7f805014bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.846+0000 7f8054b3e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f805015fc60 0x7f805014a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.846+0000 7f8054b3e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8050160070 con 0x7f80500d4120 2024-05-20T18:32:20.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.846+0000 7f8054b3e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f80500d31a0 con 0x7f805015fc60 2024-05-20T18:32:20.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.846+0000 7f8054b3e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f80500d4530 con 0x7f80500d5030 2024-05-20T18:32:20.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.846+0000 7f804e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80500d4120 0x7f80501490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.846+0000 7f804e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80500d4120 0x7f80501490a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60730/0 (socket says 172.21.15.5:60730) 2024-05-20T18:32:20.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.846+0000 7f804e59c640 1 -- 172.21.15.5:0/2298064811 learned_addr learned my addr 172.21.15.5:0/2298064811 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:20.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.847+0000 7f804ed9d640 1 --2- 172.21.15.5:0/2298064811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f805015fc60 0x7f805014a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.847+0000 7f804dd9b640 1 --2- 172.21.15.5:0/2298064811 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80500d5030 0x7f805014bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.847+0000 7f804e59c640 1 -- 172.21.15.5:0/2298064811 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80500d5030 msgr2=0x7f805014bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:20.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.847+0000 7f804e59c640 1 --2- 172.21.15.5:0/2298064811 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80500d5030 0x7f805014bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.847+0000 7f804e59c640 1 -- 172.21.15.5:0/2298064811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f805015fc60 msgr2=0x7f805014a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:20.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.847+0000 7f804e59c640 1 --2- 172.21.15.5:0/2298064811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f805015fc60 0x7f805014a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:20.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.847+0000 7f804e59c640 1 -- 172.21.15.5:0/2298064811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f805014aed0 con 0x7f80500d4120 2024-05-20T18:32:20.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.847+0000 7f804dd9b640 1 --2- 172.21.15.5:0/2298064811 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80500d5030 0x7f805014bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:32:20.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.847+0000 7f804e59c640 1 --2- 172.21.15.5:0/2298064811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80500d4120 0x7f80501490a0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f804000efc0 tx=0x7f804000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:20.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.847+0000 7f8046ffd640 1 -- 172.21.15.5:0/2298064811 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8040018020 con 0x7f80500d4120 2024-05-20T18:32:20.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.847+0000 7f8046ffd640 1 -- 172.21.15.5:0/2298064811 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8040014480 con 0x7f80500d4120 2024-05-20T18:32:20.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.847+0000 7f8054b3e640 1 -- 172.21.15.5:0/2298064811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80501498f0 con 0x7f80500d4120 2024-05-20T18:32:20.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.847+0000 7f8046ffd640 1 -- 172.21.15.5:0/2298064811 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8040014850 con 0x7f80500d4120 2024-05-20T18:32:20.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.847+0000 7f8054b3e640 1 -- 172.21.15.5:0/2298064811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8050149eb0 con 0x7f80500d4120 2024-05-20T18:32:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.848+0000 7f8054b3e640 1 -- 172.21.15.5:0/2298064811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80500690b0 con 0x7f80500d4120 2024-05-20T18:32:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.849+0000 7f8046ffd640 1 -- 172.21.15.5:0/2298064811 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8040005d10 con 0x7f80500d4120 2024-05-20T18:32:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.850+0000 7f8046ffd640 1 --2- 172.21.15.5:0/2298064811 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f803006be60 0x7f803006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.850+0000 7f804dd9b640 1 --2- 172.21.15.5:0/2298064811 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f803006be60 0x7f803006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.850+0000 7f8046ffd640 1 -- 172.21.15.5:0/2298064811 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f804008baa0 con 0x7f80500d4120 2024-05-20T18:32:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.850+0000 7f804dd9b640 1 --2- 172.21.15.5:0/2298064811 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f803006be60 0x7f803006e370 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8038034000 tx=0x7f803802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:20.854+0000 7f8046ffd640 1 -- 172.21.15.5:0/2298064811 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8040022070 con 0x7f80500d4120 2024-05-20T18:32:21.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.082+0000 7f8054b3e640 1 -- 172.21.15.5:0/2298064811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f80500d2320 con 0x7f80500d4120 2024-05-20T18:32:21.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.082+0000 7f8046ffd640 1 -- 172.21.15.5:0/2298064811 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f8040010040 con 0x7f80500d4120 2024-05-20T18:32:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:32:21.083 INFO:teuthology.orchestra.run.smithi005.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-05-20T18:32:21.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.084+0000 7f8054b3e640 1 -- 172.21.15.5:0/2298064811 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f803006be60 msgr2=0x7f803006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:21.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.084+0000 7f8054b3e640 1 --2- 172.21.15.5:0/2298064811 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f803006be60 0x7f803006e370 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8038034000 tx=0x7f803802e000 comp rx=0 tx=0).stop 2024-05-20T18:32:21.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.084+0000 7f8054b3e640 1 -- 172.21.15.5:0/2298064811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80500d4120 msgr2=0x7f80501490a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:21.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.084+0000 7f8054b3e640 1 --2- 172.21.15.5:0/2298064811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80500d4120 0x7f80501490a0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f804000efc0 tx=0x7f804000c490 comp rx=0 tx=0).stop 2024-05-20T18:32:21.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.084+0000 7f8054b3e640 1 -- 172.21.15.5:0/2298064811 shutdown_connections 2024-05-20T18:32:21.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.084+0000 7f8054b3e640 1 --2- 172.21.15.5:0/2298064811 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f803006be60 0x7f803006e370 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.084+0000 7f8054b3e640 1 --2- 172.21.15.5:0/2298064811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f805015fc60 0x7f805014a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.084+0000 7f8054b3e640 1 --2- 172.21.15.5:0/2298064811 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80500d5030 0x7f805014bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.084+0000 7f8054b3e640 1 --2- 172.21.15.5:0/2298064811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80500d4120 0x7f80501490a0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.084+0000 7f8054b3e640 1 -- 172.21.15.5:0/2298064811 >> 172.21.15.5:0/2298064811 conn(0x7f805006b520 msgr2=0x7f80500b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:21.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.084+0000 7f8054b3e640 1 -- 172.21.15.5:0/2298064811 shutdown_connections 2024-05-20T18:32:21.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.084+0000 7f8054b3e640 1 -- 172.21.15.5:0/2298064811 wait complete. 2024-05-20T18:32:21.096 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-20T18:32:21.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:32:21.096 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir repair,recursive 2024-05-20T18:32:21.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.214+0000 7fbf7386c640 1 Processor -- start 2024-05-20T18:32:21.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.214+0000 7fbf7386c640 1 -- start start 2024-05-20T18:32:21.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.216+0000 7fbf7386c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf6c0d2330 0x7fbf6c06d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.216+0000 7fbf7386c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbf6c0ce140 0x7fbf6c0d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.216+0000 7fbf7386c640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf6c0d2740 con 0x7fbf6c06d660 2024-05-20T18:32:21.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.216+0000 7fbf7386c640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fbf6c09c3b0 con 0x7fbf6c0ce140 2024-05-20T18:32:21.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.216+0000 7fbf7386c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbf6c06dda0 con 0x7fbf6c0d2330 2024-05-20T18:32:21.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.216+0000 7fbf71e09640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbf6c0ce140 0x7fbf6c0d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.216+0000 7fbf71e09640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbf6c0ce140 0x7fbf6c0d0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60742/0 (socket says 172.21.15.5:60742) 2024-05-20T18:32:21.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.216+0000 7fbf71e09640 1 -- 172.21.15.5:0/2397795892 learned_addr learned my addr 172.21.15.5:0/2397795892 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:21.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.216+0000 7fbf71608640 1 --2- 172.21.15.5:0/2397795892 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf6c0d2330 0x7fbf6c06d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.216+0000 7fbf71e09640 1 -- 172.21.15.5:0/2397795892 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf6c0d2330 msgr2=0x7fbf6c06d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:21.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.216+0000 7fbf71e09640 1 --2- 172.21.15.5:0/2397795892 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf6c0d2330 0x7fbf6c06d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.216+0000 7fbf71e09640 1 -- 172.21.15.5:0/2397795892 >> v1:172.21.15.63:6789/0 conn(0x7fbf6c06d660 legacy=0x7fbf6c0cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:21.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.216+0000 7fbf71e09640 1 -- 172.21.15.5:0/2397795892 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6c0b81a0 con 0x7fbf6c0ce140 2024-05-20T18:32:21.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.217+0000 7fbf71e09640 1 --2- 172.21.15.5:0/2397795892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf6c0ce140 0x7fbf6c0d0f40 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fbf64009e50 tx=0x7fbf6402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b9eef199087272 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:21.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.217+0000 7fbf63fff640 1 -- 172.21.15.5:0/2397795892 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64036070 con 0x7fbf6c0ce140 2024-05-20T18:32:21.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.217+0000 7fbf63fff640 1 -- 172.21.15.5:0/2397795892 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf64031600 con 0x7fbf6c0ce140 2024-05-20T18:32:21.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.217+0000 7fbf63fff640 1 -- 172.21.15.5:0/2397795892 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf640319d0 con 0x7fbf6c0ce140 2024-05-20T18:32:21.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.218+0000 7fbf7386c640 1 -- 172.21.15.5:0/2397795892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf6c0ce140 msgr2=0x7fbf6c0d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:21.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.218+0000 7fbf7386c640 1 --2- 172.21.15.5:0/2397795892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf6c0ce140 0x7fbf6c0d0f40 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fbf64009e50 tx=0x7fbf6402a710 comp rx=0 tx=0).stop 2024-05-20T18:32:21.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.218+0000 7fbf7386c640 1 -- 172.21.15.5:0/2397795892 shutdown_connections 2024-05-20T18:32:21.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.218+0000 7fbf7386c640 1 --2- 172.21.15.5:0/2397795892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf6c0ce140 0x7fbf6c0d0f40 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.218+0000 7fbf7386c640 1 --2- 172.21.15.5:0/2397795892 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf6c0d2330 0x7fbf6c06d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.218+0000 7fbf7386c640 1 -- 172.21.15.5:0/2397795892 >> 172.21.15.5:0/2397795892 conn(0x7fbf6c0cf450 msgr2=0x7fbf6c0cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:21.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.218+0000 7fbf7386c640 1 -- 172.21.15.5:0/2397795892 shutdown_connections 2024-05-20T18:32:21.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.218+0000 7fbf7386c640 1 -- 172.21.15.5:0/2397795892 wait complete. 2024-05-20T18:32:21.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.220+0000 7fbf7386c640 1 Processor -- start 2024-05-20T18:32:21.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.220+0000 7fbf7386c640 1 -- start start 2024-05-20T18:32:21.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.221+0000 7fbf7386c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf6c06d660 0x7fbf6c15ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.221+0000 7fbf7386c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf6c0ce140 0x7fbf6c150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.221+0000 7fbf7386c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf6c0d2330 0x7fbf6c0b6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.221+0000 7fbf7386c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c0d3800 con 0x7fbf6c0ce140 2024-05-20T18:32:21.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.221+0000 7fbf7386c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c06da70 con 0x7fbf6c06d660 2024-05-20T18:32:21.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.221+0000 7fbf7386c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbf6c0d3980 con 0x7fbf6c0d2330 2024-05-20T18:32:21.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.221+0000 7fbf70e07640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf6c0ce140 0x7fbf6c150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.221+0000 7fbf70e07640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf6c0ce140 0x7fbf6c150ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60758/0 (socket says 172.21.15.5:60758) 2024-05-20T18:32:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.221+0000 7fbf70e07640 1 -- 172.21.15.5:0/2573774753 learned_addr learned my addr 172.21.15.5:0/2573774753 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.221+0000 7fbf71608640 1 --2- 172.21.15.5:0/2573774753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf6c06d660 0x7fbf6c15ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.222+0000 7fbf71e09640 1 --2- 172.21.15.5:0/2573774753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf6c0d2330 0x7fbf6c0b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.222+0000 7fbf70e07640 1 -- 172.21.15.5:0/2573774753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf6c0d2330 msgr2=0x7fbf6c0b6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.222+0000 7fbf70e07640 1 --2- 172.21.15.5:0/2573774753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf6c0d2330 0x7fbf6c0b6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.222+0000 7fbf70e07640 1 -- 172.21.15.5:0/2573774753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf6c06d660 msgr2=0x7fbf6c15ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.222+0000 7fbf70e07640 1 --2- 172.21.15.5:0/2573774753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf6c06d660 0x7fbf6c15ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.222+0000 7fbf70e07640 1 -- 172.21.15.5:0/2573774753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6c15f050 con 0x7fbf6c0ce140 2024-05-20T18:32:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.222+0000 7fbf71e09640 1 --2- 172.21.15.5:0/2573774753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf6c0d2330 0x7fbf6c0b6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:32:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.222+0000 7fbf70e07640 1 --2- 172.21.15.5:0/2573774753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf6c0ce140 0x7fbf6c150ea0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00c3c0 tx=0x7fbf5c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.222+0000 7fbf61ffb640 1 -- 172.21.15.5:0/2573774753 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c017070 con 0x7fbf6c0ce140 2024-05-20T18:32:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.222+0000 7fbf61ffb640 1 -- 172.21.15.5:0/2573774753 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf5c0052f0 con 0x7fbf6c0ce140 2024-05-20T18:32:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.222+0000 7fbf7386c640 1 -- 172.21.15.5:0/2573774753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf6c15e7b0 con 0x7fbf6c0ce140 2024-05-20T18:32:21.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.222+0000 7fbf61ffb640 1 -- 172.21.15.5:0/2573774753 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c003830 con 0x7fbf6c0ce140 2024-05-20T18:32:21.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.222+0000 7fbf7386c640 1 -- 172.21.15.5:0/2573774753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf6c15ed70 con 0x7fbf6c0ce140 2024-05-20T18:32:21.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.224+0000 7fbf61ffb640 1 -- 172.21.15.5:0/2573774753 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbf5c01a070 con 0x7fbf6c0ce140 2024-05-20T18:32:21.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.224+0000 7fbf61ffb640 1 --2- 172.21.15.5:0/2573774753 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf5006be60 0x7fbf5006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.225+0000 7fbf71608640 1 --2- 172.21.15.5:0/2573774753 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf5006be60 0x7fbf5006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.225+0000 7fbf61ffb640 1 -- 172.21.15.5:0/2573774753 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7fbf5c08a3d0 con 0x7fbf6c0ce140 2024-05-20T18:32:21.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.225+0000 7fbf71608640 1 --2- 172.21.15.5:0/2573774753 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf5006be60 0x7fbf5006e370 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbf6800b9f0 tx=0x7fbf68008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:21.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.226+0000 7fbf7a7ee740 1 Processor -- start 2024-05-20T18:32:21.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.226+0000 7fbf7a7ee740 1 -- start start 2024-05-20T18:32:21.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.227+0000 7fbf7a7ee740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5588963b2680 0x55889677d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.227+0000 7fbf7a7ee740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5588963897c0 0x558896797f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.227+0000 7fbf7a7ee740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588963919a0 0x558896798470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.227+0000 7fbf7a7ee740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5588966b6b60 con 0x5588963919a0 2024-05-20T18:32:21.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.227+0000 7fbf7a7ee740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55889665e780 con 0x5588963897c0 2024-05-20T18:32:21.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.227+0000 7fbf7a7ee740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5588966b2b20 con 0x5588963b2680 2024-05-20T18:32:21.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.227+0000 7fbf71e09640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588963919a0 0x558896798470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.227+0000 7fbf71e09640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588963919a0 0x558896798470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60774/0 (socket says 172.21.15.5:60774) 2024-05-20T18:32:21.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.227+0000 7fbf71e09640 1 -- 172.21.15.5:0/1294341489 learned_addr learned my addr 172.21.15.5:0/1294341489 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:21.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.227+0000 7fbf71608640 1 --2- 172.21.15.5:0/1294341489 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5588963b2680 0x55889677d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.227+0000 7fbf70e07640 1 --2- 172.21.15.5:0/1294341489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5588963897c0 0x558896797f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.227+0000 7fbf71e09640 1 -- 172.21.15.5:0/1294341489 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5588963b2680 msgr2=0x55889677d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:21.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.227+0000 7fbf71e09640 1 --2- 172.21.15.5:0/1294341489 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5588963b2680 0x55889677d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.227+0000 7fbf71e09640 1 -- 172.21.15.5:0/1294341489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5588963897c0 msgr2=0x558896797f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:21.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.227+0000 7fbf71e09640 1 --2- 172.21.15.5:0/1294341489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5588963897c0 0x558896797f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.227+0000 7fbf71e09640 1 -- 172.21.15.5:0/1294341489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558896492a50 con 0x5588963919a0 2024-05-20T18:32:21.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.228+0000 7fbf71e09640 1 --2- 172.21.15.5:0/1294341489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588963919a0 0x558896798470 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fbf64009f70 tx=0x7fbf6402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e8c08a47cedb8f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:21.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.228+0000 7fbf56ffd640 1 -- 172.21.15.5:0/1294341489 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64036040 con 0x5588963919a0 2024-05-20T18:32:21.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.228+0000 7fbf56ffd640 1 -- 172.21.15.5:0/1294341489 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf64041070 con 0x5588963919a0 2024-05-20T18:32:21.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.228+0000 7fbf56ffd640 1 -- 172.21.15.5:0/1294341489 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6403c890 con 0x5588963919a0 2024-05-20T18:32:21.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.228+0000 7fbf7a7ee740 1 -- 172.21.15.5:0/1294341489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588963919a0 msgr2=0x558896798470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:21.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.228+0000 7fbf7a7ee740 1 --2- 172.21.15.5:0/1294341489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588963919a0 0x558896798470 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fbf64009f70 tx=0x7fbf6402e000 comp rx=0 tx=0).stop 2024-05-20T18:32:21.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.229+0000 7fbf7a7ee740 1 -- 172.21.15.5:0/1294341489 shutdown_connections 2024-05-20T18:32:21.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.229+0000 7fbf7a7ee740 1 --2- 172.21.15.5:0/1294341489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588963919a0 0x558896798470 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.229+0000 7fbf7a7ee740 1 --2- 172.21.15.5:0/1294341489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5588963897c0 0x558896797f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.229+0000 7fbf7a7ee740 1 --2- 172.21.15.5:0/1294341489 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5588963b2680 0x55889677d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.229+0000 7fbf7a7ee740 1 -- 172.21.15.5:0/1294341489 >> 172.21.15.5:0/1294341489 conn(0x5588963a67d0 msgr2=0x558896752860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:21.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.229+0000 7fbf7a7ee740 1 -- 172.21.15.5:0/1294341489 shutdown_connections 2024-05-20T18:32:21.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.229+0000 7fbf7a7ee740 1 -- 172.21.15.5:0/1294341489 wait complete. 2024-05-20T18:32:21.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.229+0000 7fbf7a7ee740 1 -- start start 2024-05-20T18:32:21.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.230+0000 7fbf7a7ee740 1 Processor -- start 2024-05-20T18:32:21.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.231+0000 7fbf7a7ee740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5588963897c0 0x558896778620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.231+0000 7fbf7a7ee740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5588963919a0 0x5588967941a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.231+0000 7fbf7a7ee740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588963b2680 0x55889674ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.231+0000 7fbf7a7ee740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558896671d10 con 0x5588963b2680 2024-05-20T18:32:21.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.231+0000 7fbf7a7ee740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5588967a84e0 con 0x5588963897c0 2024-05-20T18:32:21.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.231+0000 7fbf7a7ee740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55889677ec40 con 0x5588963919a0 2024-05-20T18:32:21.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.231+0000 7fbf7a7ee740 10 client.0 _ll_register_callbacks cb 0x5588966662c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:32:21.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.231+0000 7fbf71e09640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588963b2680 0x55889674ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.231+0000 7fbf71e09640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588963b2680 0x55889674ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60788/0 (socket says 172.21.15.5:60788) 2024-05-20T18:32:21.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.231+0000 7fbf71e09640 1 -- 172.21.15.5:0/1201362682 learned_addr learned my addr 172.21.15.5:0/1201362682 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:21.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.231+0000 7fbf70e07640 1 --2- 172.21.15.5:0/1201362682 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5588963919a0 0x5588967941a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.231+0000 7fbf71608640 1 --2- 172.21.15.5:0/1201362682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5588963897c0 0x558896778620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.232+0000 7fbf71e09640 1 -- 172.21.15.5:0/1201362682 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5588963919a0 msgr2=0x5588967941a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:21.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.232+0000 7fbf71e09640 1 --2- 172.21.15.5:0/1201362682 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5588963919a0 0x5588967941a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.232+0000 7fbf71e09640 1 -- 172.21.15.5:0/1201362682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5588963897c0 msgr2=0x558896778620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:21.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.232+0000 7fbf71e09640 1 --2- 172.21.15.5:0/1201362682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5588963897c0 0x558896778620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.232+0000 7fbf71e09640 1 -- 172.21.15.5:0/1201362682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55889638a3e0 con 0x5588963b2680 2024-05-20T18:32:21.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.232+0000 7fbf70e07640 1 --2- 172.21.15.5:0/1201362682 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5588963919a0 0x5588967941a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:32:21.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.232+0000 7fbf71608640 1 --2- 172.21.15.5:0/1201362682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5588963897c0 0x558896778620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:32:21.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.232+0000 7fbf71e09640 1 --2- 172.21.15.5:0/1201362682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588963b2680 0x55889674ca10 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fbf6402a750 tx=0x7fbf64004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cccd6b9a51748cba server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:21.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.232+0000 7fbf4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:32:21.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.232+0000 7fbf4bfff640 1 -- 172.21.15.5:0/1201362682 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64036050 con 0x5588963b2680 2024-05-20T18:32:21.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.232+0000 7fbf7a7ee740 1 -- 172.21.15.5:0/1201362682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf64041070 con 0x5588963b2680 2024-05-20T18:32:21.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.232+0000 7fbf4bfff640 1 -- 172.21.15.5:0/1201362682 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7fbf64004340 con 0x5588963b2680 2024-05-20T18:32:21.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.232+0000 7fbf4bfff640 1 -- 172.21.15.5:0/1201362682 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fbf64004540 con 0x5588963b2680 2024-05-20T18:32:21.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.232+0000 7fbf7a7ee740 10 client.7164 fetch_fsmap learned FSMap version 51 2024-05-20T18:32:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.232+0000 7fbf7a7ee740 1 -- 172.21.15.5:0/1201362682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x5588964f31e0 con 0x5588963b2680 2024-05-20T18:32:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.232+0000 7fbf4bfff640 1 -- 172.21.15.5:0/1201362682 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf64041070 con 0x5588963b2680 2024-05-20T18:32:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.232+0000 7fbf4bfff640 1 -- 172.21.15.5:0/1201362682 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7fbf64058080 con 0x5588963b2680 2024-05-20T18:32:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.232+0000 7fbf7a7ee740 10 client.7164 fetch_fsmap finished waiting for FSMap version 51 2024-05-20T18:32:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.232+0000 7fbf7a7ee740 10 client.7164 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-05-20T18:32:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.233+0000 7fbf7a7ee740 20 client.7164 populate_metadata read hostname 'smithi005' 2024-05-20T18:32:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.233+0000 7fbf7a7ee740 1 --2- 172.21.15.5:0/1201362682 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x558896371250 0x5588967a72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.233+0000 7fbf7a7ee740 4 client.7164 mds_command: new command op to 6972 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:32:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.233+0000 7fbf7a7ee740 1 -- 172.21.15.5:0/1201362682 --> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588966c4290 con 0x558896371250 2024-05-20T18:32:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.233+0000 7fbf71608640 1 --2- 172.21.15.5:0/1201362682 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x558896371250 0x5588967a72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.233+0000 7fbf4bfff640 10 client.7164 ms_handle_connect on v2:172.21.15.5:6835/3866822752 2024-05-20T18:32:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.233+0000 7fbf71608640 1 --2- 172.21.15.5:0/1201362682 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x558896371250 0x5588967a72a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b552a971b6b4afd server_cookie=ae6a66c8dd5d5133 in_seq=0 out_seq=0 2024-05-20T18:32:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.234+0000 7fbf4bfff640 1 -- 172.21.15.5:0/1201362682 <== mds.0 v2:172.21.15.5:6835/3866822752 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5588966c4290 con 0x558896371250 2024-05-20T18:32:21.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.234+0000 7fbf4bfff640 10 client.7164 handle_command_reply: tid=0 2024-05-20T18:32:21.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.234+0000 7fbf7a7ee740 1 client.7164 shutdown 2024-05-20T18:32:21.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.234+0000 7fbf7a7ee740 1 -- 172.21.15.5:0/1201362682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588963b2680 msgr2=0x55889674ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:21.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.234+0000 7fbf7a7ee740 1 --2- 172.21.15.5:0/1201362682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588963b2680 0x55889674ca10 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fbf6402a750 tx=0x7fbf64004ab0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.234+0000 7fbf7a7ee740 1 -- 172.21.15.5:0/1201362682 shutdown_connections 2024-05-20T18:32:21.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.234+0000 7fbf7a7ee740 1 --2- 172.21.15.5:0/1201362682 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x558896371250 0x5588967a72a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.234+0000 7fbf7a7ee740 1 --2- 172.21.15.5:0/1201362682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588963b2680 0x55889674ca10 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.234+0000 7fbf7a7ee740 1 --2- 172.21.15.5:0/1201362682 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5588963919a0 0x5588967941a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.234+0000 7fbf7a7ee740 1 --2- 172.21.15.5:0/1201362682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5588963897c0 0x558896778620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.234+0000 7fbf4bfff640 0 client.7164 ms_handle_reset on v2:172.21.15.5:6835/3866822752 2024-05-20T18:32:21.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.234+0000 7fbf7a7ee740 1 -- 172.21.15.5:0/1201362682 >> 172.21.15.5:0/1201362682 conn(0x5588963a67d0 msgr2=0x55889677d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:21.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.234+0000 7fbf7a7ee740 1 -- 172.21.15.5:0/1201362682 shutdown_connections 2024-05-20T18:32:21.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.234+0000 7fbf7a7ee740 1 -- 172.21.15.5:0/1201362682 wait complete. 2024-05-20T18:32:21.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.234+0000 7fbf7a7ee740 20 client.7164 trim_cache size 0 max 16384 2024-05-20T18:32:21.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.253+0000 7fbf7a7ee740 1 Processor -- start 2024-05-20T18:32:21.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.253+0000 7fbf7a7ee740 1 -- start start 2024-05-20T18:32:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.254+0000 7fbf7a7ee740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5588963897c0 0x5588967baec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.254+0000 7fbf7a7ee740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5588963919a0 0x55889679a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.254+0000 7fbf7a7ee740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588963b2680 0x55889677d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.254+0000 7fbf7a7ee740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c2023c0 con 0x5588963b2680 2024-05-20T18:32:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.254+0000 7fbf7a7ee740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c201400 con 0x5588963897c0 2024-05-20T18:32:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.254+0000 7fbf7a7ee740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbf6c200810 con 0x5588963919a0 2024-05-20T18:32:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.254+0000 7fbf71e09640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588963b2680 0x55889677d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.254+0000 7fbf71e09640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588963b2680 0x55889677d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60792/0 (socket says 172.21.15.5:60792) 2024-05-20T18:32:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.254+0000 7fbf71e09640 1 -- 172.21.15.5:0/1299767379 learned_addr learned my addr 172.21.15.5:0/1299767379 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:21.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.254+0000 7fbf70e07640 1 --2- 172.21.15.5:0/1299767379 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5588963919a0 0x55889679a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.254+0000 7fbf71608640 1 --2- 172.21.15.5:0/1299767379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5588963897c0 0x5588967baec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.254+0000 7fbf71e09640 1 -- 172.21.15.5:0/1299767379 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5588963919a0 msgr2=0x55889679a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:21.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.254+0000 7fbf71e09640 1 --2- 172.21.15.5:0/1299767379 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5588963919a0 0x55889679a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.254+0000 7fbf71e09640 1 -- 172.21.15.5:0/1299767379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5588963897c0 msgr2=0x5588967baec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:21.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.254+0000 7fbf71e09640 1 --2- 172.21.15.5:0/1299767379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5588963897c0 0x5588967baec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.254+0000 7fbf71e09640 1 -- 172.21.15.5:0/1299767379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55889677e400 con 0x5588963b2680 2024-05-20T18:32:21.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.254+0000 7fbf71608640 1 --2- 172.21.15.5:0/1299767379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5588963897c0 0x5588967baec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:32:21.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.254+0000 7fbf71e09640 1 --2- 172.21.15.5:0/1299767379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588963b2680 0x55889677d3f0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fbf6402ad00 tx=0x7fbf64046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=924f5f8be8d8a417 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:21.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.255+0000 7fbf567fc640 1 -- 172.21.15.5:0/1299767379 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64036020 con 0x5588963b2680 2024-05-20T18:32:21.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.255+0000 7fbf567fc640 1 -- 172.21.15.5:0/1299767379 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf64041040 con 0x5588963b2680 2024-05-20T18:32:21.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.255+0000 7fbf567fc640 1 -- 172.21.15.5:0/1299767379 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64004140 con 0x5588963b2680 2024-05-20T18:32:21.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.255+0000 7fbf7a7ee740 1 -- 172.21.15.5:0/1299767379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588963b2680 msgr2=0x55889677d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:21.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.255+0000 7fbf7a7ee740 1 --2- 172.21.15.5:0/1299767379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588963b2680 0x55889677d3f0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fbf6402ad00 tx=0x7fbf64046000 comp rx=0 tx=0).stop 2024-05-20T18:32:21.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.255+0000 7fbf7a7ee740 1 -- 172.21.15.5:0/1299767379 shutdown_connections 2024-05-20T18:32:21.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.255+0000 7fbf7a7ee740 1 --2- 172.21.15.5:0/1299767379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588963b2680 0x55889677d3f0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.255+0000 7fbf7a7ee740 1 --2- 172.21.15.5:0/1299767379 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5588963919a0 0x55889679a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.255+0000 7fbf7a7ee740 1 --2- 172.21.15.5:0/1299767379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5588963897c0 0x5588967baec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.255+0000 7fbf7a7ee740 1 -- 172.21.15.5:0/1299767379 >> 172.21.15.5:0/1299767379 conn(0x5588963a67d0 msgr2=0x558896778430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:21.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.255+0000 7fbf7a7ee740 1 -- 172.21.15.5:0/1299767379 shutdown_connections 2024-05-20T18:32:21.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.255+0000 7fbf7a7ee740 1 -- 172.21.15.5:0/1299767379 wait complete. 2024-05-20T18:32:21.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.256+0000 7fbf7a7ee740 1 -- start start 2024-05-20T18:32:21.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.256+0000 7fbf7a7ee740 1 Processor -- start 2024-05-20T18:32:21.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.258+0000 7fbf7a7ee740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5588963897c0 0x5588967a7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.258+0000 7fbf7a7ee740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588963919a0 0x5588967a7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.258+0000 7fbf7a7ee740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5588963b2680 0x558896778430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.258+0000 7fbf7a7ee740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c200e30 con 0x5588963919a0 2024-05-20T18:32:21.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.258+0000 7fbf7a7ee740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558896798fa0 con 0x5588963897c0 2024-05-20T18:32:21.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.258+0000 7fbf7a7ee740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55889674c8a0 con 0x5588963b2680 2024-05-20T18:32:21.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.258+0000 7fbf7a7ee740 10 client.0 _ll_register_callbacks cb 0x7fbf64009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:32:21.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.258+0000 7fbf70e07640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588963919a0 0x5588967a7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.258+0000 7fbf70e07640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588963919a0 0x5588967a7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60796/0 (socket says 172.21.15.5:60796) 2024-05-20T18:32:21.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.258+0000 7fbf70e07640 1 -- 172.21.15.5:0/2931357440 learned_addr learned my addr 172.21.15.5:0/2931357440 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:21.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.258+0000 7fbf71608640 1 --2- 172.21.15.5:0/2931357440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5588963897c0 0x5588967a7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.258+0000 7fbf71e09640 1 --2- 172.21.15.5:0/2931357440 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5588963b2680 0x558896778430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.258+0000 7fbf70e07640 1 -- 172.21.15.5:0/2931357440 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5588963b2680 msgr2=0x558896778430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:21.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.258+0000 7fbf70e07640 1 --2- 172.21.15.5:0/2931357440 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5588963b2680 0x558896778430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.258+0000 7fbf70e07640 1 -- 172.21.15.5:0/2931357440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5588963897c0 msgr2=0x5588967a7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:21.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.258+0000 7fbf70e07640 1 --2- 172.21.15.5:0/2931357440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5588963897c0 0x5588967a7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.258+0000 7fbf70e07640 1 -- 172.21.15.5:0/2931357440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588963634a0 con 0x5588963919a0 2024-05-20T18:32:21.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.258+0000 7fbf71e09640 1 --2- 172.21.15.5:0/2931357440 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5588963b2680 0x558896778430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:32:21.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.258+0000 7fbf70e07640 1 --2- 172.21.15.5:0/2931357440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588963919a0 0x5588967a7b40 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c054870 tx=0x7fbf5c058490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f599f992ccea96 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:21.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.258+0000 7fbf4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:32:21.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.259+0000 7fbf4bfff640 1 -- 172.21.15.5:0/2931357440 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c08a0b0 con 0x5588963919a0 2024-05-20T18:32:21.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.259+0000 7fbf4bfff640 1 -- 172.21.15.5:0/2931357440 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7fbf5c054a80 con 0x5588963919a0 2024-05-20T18:32:21.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.259+0000 7fbf7a7ee740 1 -- 172.21.15.5:0/2931357440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf64041040 con 0x5588963919a0 2024-05-20T18:32:21.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.259+0000 7fbf4bfff640 1 -- 172.21.15.5:0/2931357440 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fbf5c075050 con 0x5588963919a0 2024-05-20T18:32:21.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.259+0000 7fbf7a7ee740 10 client.7170 fetch_fsmap learned FSMap version 51 2024-05-20T18:32:21.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.259+0000 7fbf7a7ee740 1 -- 172.21.15.5:0/2931357440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x558896750db0 con 0x5588963919a0 2024-05-20T18:32:21.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.259+0000 7fbf4bfff640 1 -- 172.21.15.5:0/2931357440 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf64041040 con 0x5588963919a0 2024-05-20T18:32:21.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.259+0000 7fbf4bfff640 1 -- 172.21.15.5:0/2931357440 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7fbf5c0772e0 con 0x5588963919a0 2024-05-20T18:32:21.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.259+0000 7fbf7a7ee740 10 client.7170 fetch_fsmap finished waiting for FSMap version 51 2024-05-20T18:32:21.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.259+0000 7fbf7a7ee740 10 client.7170 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-05-20T18:32:21.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.259+0000 7fbf7a7ee740 20 client.7170 populate_metadata read hostname 'smithi005' 2024-05-20T18:32:21.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.259+0000 7fbf7a7ee740 1 --2- 172.21.15.5:0/2931357440 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x55889677cc00 0x558896778970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.259+0000 7fbf7a7ee740 4 client.7170 mds_command: new command op to 6972 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2024-05-20T18:32:21.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.259+0000 7fbf7a7ee740 1 -- 172.21.15.5:0/2931357440 --> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) -- 0x5588966be5e0 con 0x55889677cc00 2024-05-20T18:32:21.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.259+0000 7fbf71608640 1 --2- 172.21.15.5:0/2931357440 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x55889677cc00 0x558896778970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.260+0000 7fbf4bfff640 10 client.7170 ms_handle_connect on v2:172.21.15.5:6835/3866822752 2024-05-20T18:32:21.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.260+0000 7fbf71608640 1 --2- 172.21.15.5:0/2931357440 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x55889677cc00 0x558896778970 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e95faff96cfc7d server_cookie=b597f47fc00cf476 in_seq=0 out_seq=0 2024-05-20T18:32:21.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.261+0000 7fbf4bfff640 1 -- 172.21.15.5:0/2931357440 <== mds.0 v2:172.21.15.5:6835/3866822752 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5588966be5e0 con 0x55889677cc00 2024-05-20T18:32:21.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.261+0000 7fbf4bfff640 10 client.7170 handle_command_reply: tid=0 2024-05-20T18:32:21.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.261+0000 7fbf7a7ee740 1 client.7170 shutdown 2024-05-20T18:32:21.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.261+0000 7fbf7a7ee740 1 -- 172.21.15.5:0/2931357440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588963919a0 msgr2=0x5588967a7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:21.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.261+0000 7fbf7a7ee740 1 --2- 172.21.15.5:0/2931357440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588963919a0 0x5588967a7b40 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c054870 tx=0x7fbf5c058490 comp rx=0 tx=0).stop 2024-05-20T18:32:21.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.261+0000 7fbf7a7ee740 1 -- 172.21.15.5:0/2931357440 shutdown_connections 2024-05-20T18:32:21.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.261+0000 7fbf7a7ee740 1 --2- 172.21.15.5:0/2931357440 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5588963b2680 0x558896778430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.261+0000 7fbf7a7ee740 1 --2- 172.21.15.5:0/2931357440 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x55889677cc00 0x558896778970 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.261+0000 7fbf7a7ee740 1 --2- 172.21.15.5:0/2931357440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588963919a0 0x5588967a7b40 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.261+0000 7fbf4bfff640 0 client.7170 ms_handle_reset on v2:172.21.15.5:6835/3866822752 2024-05-20T18:32:21.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.261+0000 7fbf7a7ee740 1 --2- 172.21.15.5:0/2931357440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5588963897c0 0x5588967a7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.261+0000 7fbf7a7ee740 1 -- 172.21.15.5:0/2931357440 >> 172.21.15.5:0/2931357440 conn(0x5588963a67d0 msgr2=0x55889674eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:21.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.261+0000 7fbf7a7ee740 1 -- 172.21.15.5:0/2931357440 shutdown_connections 2024-05-20T18:32:21.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.261+0000 7fbf7a7ee740 1 -- 172.21.15.5:0/2931357440 wait complete. 2024-05-20T18:32:21.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.261+0000 7fbf7a7ee740 20 client.7170 trim_cache size 0 max 16384 2024-05-20T18:32:21.267 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:32:21.267 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-20T18:32:21.267 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "519300df-413d-44e0-86b3-c6d2904a3f72", 2024-05-20T18:32:21.267 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-20T18:32:21.267 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:32:21.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.262+0000 7fbf7386c640 1 -- 172.21.15.5:0/2573774753 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf5006be60 msgr2=0x7fbf5006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:21.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.262+0000 7fbf7386c640 1 --2- 172.21.15.5:0/2573774753 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf5006be60 0x7fbf5006e370 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbf6800b9f0 tx=0x7fbf68008040 comp rx=0 tx=0).stop 2024-05-20T18:32:21.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.262+0000 7fbf7386c640 1 -- 172.21.15.5:0/2573774753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf6c0ce140 msgr2=0x7fbf6c150ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:21.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.262+0000 7fbf7386c640 1 --2- 172.21.15.5:0/2573774753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf6c0ce140 0x7fbf6c150ea0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00c3c0 tx=0x7fbf5c00c880 comp rx=0 tx=0).stop 2024-05-20T18:32:21.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.262+0000 7fbf7386c640 1 -- 172.21.15.5:0/2573774753 shutdown_connections 2024-05-20T18:32:21.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.262+0000 7fbf7386c640 1 --2- 172.21.15.5:0/2573774753 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf5006be60 0x7fbf5006e370 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.262+0000 7fbf7386c640 1 --2- 172.21.15.5:0/2573774753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf6c0d2330 0x7fbf6c0b6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.262+0000 7fbf7386c640 1 --2- 172.21.15.5:0/2573774753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf6c0ce140 0x7fbf6c150ea0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.262+0000 7fbf7386c640 1 --2- 172.21.15.5:0/2573774753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf6c06d660 0x7fbf6c15ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.262+0000 7fbf7386c640 1 -- 172.21.15.5:0/2573774753 >> 172.21.15.5:0/2573774753 conn(0x7fbf6c0cf450 msgr2=0x7fbf6c071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:21.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.262+0000 7fbf7386c640 1 -- 172.21.15.5:0/2573774753 shutdown_connections 2024-05-20T18:32:21.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.262+0000 7fbf7386c640 1 -- 172.21.15.5:0/2573774753 wait complete. 2024-05-20T18:32:21.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:32:21.273 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-05-20T18:32:21.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.389+0000 7fd0f408b640 1 Processor -- start 2024-05-20T18:32:21.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.390+0000 7fd0f408b640 1 -- start start 2024-05-20T18:32:21.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.391+0000 7fd0f408b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0ec0b6d90 0x7fd0ec15f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.391+0000 7fd0f408b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd0ec0d3220 0x7fd0ec0d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.391+0000 7fd0f408b640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0ec0d0550 con 0x7fd0ec0cf250 2024-05-20T18:32:21.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.391+0000 7fd0f408b640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd0ec15f9a0 con 0x7fd0ec0d3220 2024-05-20T18:32:21.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.391+0000 7fd0f408b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd0ec0d0b10 con 0x7fd0ec0b6d90 2024-05-20T18:32:21.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.391+0000 7fd0f1e27640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd0ec0cf250 0x7fd0ec0b6680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53750/0 (socket says 172.21.15.5:53750) 2024-05-20T18:32:21.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.391+0000 7fd0f1e27640 1 -- 172.21.15.5:0/812036360 learned_addr learned my addr 172.21.15.5:0/812036360 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:32:21.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.391+0000 7fd0f1626640 1 --2- 172.21.15.5:0/812036360 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0ec0b6d90 0x7fd0ec15f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.391+0000 7fd0f2628640 1 --2- 172.21.15.5:0/812036360 >> v2:172.21.15.63:3300/0 conn(0x7fd0ec0d3220 0x7fd0ec0d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.391+0000 7fd0f0e25640 1 -- 172.21.15.5:0/812036360 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 771576826 0 0) 0x7fd0ec0d0550 con 0x7fd0ec0cf250 2024-05-20T18:32:21.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.391+0000 7fd0f0e25640 1 -- 172.21.15.5:0/812036360 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0d8003520 con 0x7fd0ec0cf250 2024-05-20T18:32:21.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.392+0000 7fd0f1626640 1 -- 172.21.15.5:0/812036360 >> v2:172.21.15.63:3300/0 conn(0x7fd0ec0d3220 msgr2=0x7fd0ec0d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:21.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.392+0000 7fd0f1626640 1 --2- 172.21.15.5:0/812036360 >> v2:172.21.15.63:3300/0 conn(0x7fd0ec0d3220 0x7fd0ec0d3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.392+0000 7fd0f1626640 1 -- 172.21.15.5:0/812036360 >> v1:172.21.15.5:6789/0 conn(0x7fd0ec0cf250 legacy=0x7fd0ec0b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:21.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.392+0000 7fd0f1626640 1 -- 172.21.15.5:0/812036360 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0ec0d0830 con 0x7fd0ec0b6d90 2024-05-20T18:32:21.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.392+0000 7fd0f2628640 1 --2- 172.21.15.5:0/812036360 >> v2:172.21.15.63:3300/0 conn(0x7fd0ec0d3220 0x7fd0ec0d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:32:21.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.392+0000 7fd0f1626640 1 --2- 172.21.15.5:0/812036360 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0ec0b6d90 0x7fd0ec15f460 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc0097a0 tx=0x7fd0dc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2208fd00704bc36 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:21.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.393+0000 7fd0f0e25640 1 -- 172.21.15.5:0/812036360 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc037070 con 0x7fd0ec0b6d90 2024-05-20T18:32:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.393+0000 7fd0f0e25640 1 -- 172.21.15.5:0/812036360 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0dc032440 con 0x7fd0ec0b6d90 2024-05-20T18:32:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.393+0000 7fd0f0e25640 1 -- 172.21.15.5:0/812036360 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc032810 con 0x7fd0ec0b6d90 2024-05-20T18:32:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.393+0000 7fd0f408b640 1 -- 172.21.15.5:0/812036360 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0ec0b6d90 msgr2=0x7fd0ec15f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.393+0000 7fd0f408b640 1 --2- 172.21.15.5:0/812036360 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0ec0b6d90 0x7fd0ec15f460 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc0097a0 tx=0x7fd0dc02a180 comp rx=0 tx=0).stop 2024-05-20T18:32:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.393+0000 7fd0f408b640 1 -- 172.21.15.5:0/812036360 shutdown_connections 2024-05-20T18:32:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.393+0000 7fd0f408b640 1 --2- 172.21.15.5:0/812036360 >> v2:172.21.15.63:3300/0 conn(0x7fd0ec0d3220 0x7fd0ec0d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.393+0000 7fd0f408b640 1 --2- 172.21.15.5:0/812036360 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0ec0b6d90 0x7fd0ec15f460 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.393+0000 7fd0f408b640 1 -- 172.21.15.5:0/812036360 >> 172.21.15.5:0/812036360 conn(0x7fd0ec0d1860 msgr2=0x7fd0ec0d1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.393+0000 7fd0f408b640 1 -- 172.21.15.5:0/812036360 shutdown_connections 2024-05-20T18:32:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.393+0000 7fd0f408b640 1 -- 172.21.15.5:0/812036360 wait complete. 2024-05-20T18:32:21.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.395+0000 7fd0f408b640 1 Processor -- start 2024-05-20T18:32:21.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.395+0000 7fd0f408b640 1 -- start start 2024-05-20T18:32:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.397+0000 7fd0f408b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0ec0cf250 0x7fd0ec156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.397+0000 7fd0f408b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0ec0d3220 0x7fd0ec1571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.397+0000 7fd0f408b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0ec161730 0x7fd0ec161b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.397+0000 7fd0f408b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec0d2d00 con 0x7fd0ec0d3220 2024-05-20T18:32:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.397+0000 7fd0f408b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec0d27d0 con 0x7fd0ec0cf250 2024-05-20T18:32:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.397+0000 7fd0f408b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd0ec0d0140 con 0x7fd0ec161730 2024-05-20T18:32:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.397+0000 7fd0f1626640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0ec0d3220 0x7fd0ec1571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.397+0000 7fd0f1626640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0ec0d3220 0x7fd0ec1571d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60798/0 (socket says 172.21.15.5:60798) 2024-05-20T18:32:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.397+0000 7fd0f1626640 1 -- 172.21.15.5:0/1977126668 learned_addr learned my addr 172.21.15.5:0/1977126668 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.397+0000 7fd0f1e27640 1 --2- 172.21.15.5:0/1977126668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0ec0cf250 0x7fd0ec156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.397+0000 7fd0f2628640 1 --2- 172.21.15.5:0/1977126668 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0ec161730 0x7fd0ec161b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.397+0000 7fd0f1626640 1 -- 172.21.15.5:0/1977126668 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0ec161730 msgr2=0x7fd0ec161b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.397+0000 7fd0f1626640 1 --2- 172.21.15.5:0/1977126668 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0ec161730 0x7fd0ec161b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.397+0000 7fd0f1626640 1 -- 172.21.15.5:0/1977126668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0ec0cf250 msgr2=0x7fd0ec156c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.397+0000 7fd0f1626640 1 --2- 172.21.15.5:0/1977126668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0ec0cf250 0x7fd0ec156c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.397+0000 7fd0f1626640 1 -- 172.21.15.5:0/1977126668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0ec160700 con 0x7fd0ec0d3220 2024-05-20T18:32:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.397+0000 7fd0f1626640 1 --2- 172.21.15.5:0/1977126668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0ec0d3220 0x7fd0ec1571d0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc0098c0 tx=0x7fd0dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.397+0000 7fd0e27fc640 1 -- 172.21.15.5:0/1977126668 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc041030 con 0x7fd0ec0d3220 2024-05-20T18:32:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.397+0000 7fd0e27fc640 1 -- 172.21.15.5:0/1977126668 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0dc03d4e0 con 0x7fd0ec0d3220 2024-05-20T18:32:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.397+0000 7fd0f408b640 1 -- 172.21.15.5:0/1977126668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0ec160970 con 0x7fd0ec0d3220 2024-05-20T18:32:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.397+0000 7fd0e27fc640 1 -- 172.21.15.5:0/1977126668 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc037040 con 0x7fd0ec0d3220 2024-05-20T18:32:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.398+0000 7fd0f408b640 1 -- 172.21.15.5:0/1977126668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0ec160f30 con 0x7fd0ec0d3220 2024-05-20T18:32:21.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.399+0000 7fd0e27fc640 1 -- 172.21.15.5:0/1977126668 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd0dc044070 con 0x7fd0ec0d3220 2024-05-20T18:32:21.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.400+0000 7fd0e27fc640 1 --2- 172.21.15.5:0/1977126668 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd0d006be30 0x7fd0d006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.400+0000 7fd0e27fc640 1 -- 172.21.15.5:0/1977126668 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7fd0dc0ab3d0 con 0x7fd0ec0d3220 2024-05-20T18:32:21.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.400+0000 7fd0f1e27640 1 --2- 172.21.15.5:0/1977126668 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd0d006be30 0x7fd0d006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.401+0000 7fd0f1e27640 1 --2- 172.21.15.5:0/1977126668 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd0d006be30 0x7fd0d006e340 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8005880 tx=0x7fd0e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.401+0000 7fd0fb011740 1 Processor -- start 2024-05-20T18:32:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.401+0000 7fd0fb011740 1 -- start start 2024-05-20T18:32:21.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.402+0000 7fd0fb011740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651076e3390 0x565107aadb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.402+0000 7fd0fb011740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651076ba4d0 0x565107ac8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.402+0000 7fd0fb011740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651076c26b0 0x565107ac8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.402+0000 7fd0fb011740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5651079e7d00 con 0x5651076c26b0 2024-05-20T18:32:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.402+0000 7fd0fb011740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56510798f4c0 con 0x5651076ba4d0 2024-05-20T18:32:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.402+0000 7fd0fb011740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5651079e3cc0 con 0x5651076e3390 2024-05-20T18:32:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.402+0000 7fd0f2628640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651076c26b0 0x565107ac8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.402+0000 7fd0f2628640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651076c26b0 0x565107ac8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60812/0 (socket says 172.21.15.5:60812) 2024-05-20T18:32:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.403+0000 7fd0f2628640 1 -- 172.21.15.5:0/872125775 learned_addr learned my addr 172.21.15.5:0/872125775 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.403+0000 7fd0f1e27640 1 --2- 172.21.15.5:0/872125775 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651076e3390 0x565107aadb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.403+0000 7fd0f1626640 1 --2- 172.21.15.5:0/872125775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651076ba4d0 0x565107ac8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.403+0000 7fd0f2628640 1 -- 172.21.15.5:0/872125775 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651076e3390 msgr2=0x565107aadb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.403+0000 7fd0f2628640 1 --2- 172.21.15.5:0/872125775 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651076e3390 0x565107aadb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.403+0000 7fd0f2628640 1 -- 172.21.15.5:0/872125775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651076ba4d0 msgr2=0x565107ac8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.403+0000 7fd0f2628640 1 --2- 172.21.15.5:0/872125775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651076ba4d0 0x565107ac8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.403+0000 7fd0f2628640 1 -- 172.21.15.5:0/872125775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651077c57d0 con 0x5651076c26b0 2024-05-20T18:32:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.403+0000 7fd0f2628640 1 --2- 172.21.15.5:0/872125775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651076c26b0 0x565107ac8dd0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40113f0 tx=0x7fd0e4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b724688e1209ac18 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.403+0000 7fd0cf7fe640 1 -- 172.21.15.5:0/872125775 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4015900 con 0x5651076c26b0 2024-05-20T18:32:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.403+0000 7fd0cf7fe640 1 -- 172.21.15.5:0/872125775 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0e4015aa0 con 0x5651076c26b0 2024-05-20T18:32:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.403+0000 7fd0cf7fe640 1 -- 172.21.15.5:0/872125775 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e401d400 con 0x5651076c26b0 2024-05-20T18:32:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.404+0000 7fd0fb011740 1 -- 172.21.15.5:0/872125775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651076c26b0 msgr2=0x565107ac8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.404+0000 7fd0fb011740 1 --2- 172.21.15.5:0/872125775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651076c26b0 0x565107ac8dd0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40113f0 tx=0x7fd0e4009750 comp rx=0 tx=0).stop 2024-05-20T18:32:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.404+0000 7fd0fb011740 1 -- 172.21.15.5:0/872125775 shutdown_connections 2024-05-20T18:32:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.404+0000 7fd0fb011740 1 --2- 172.21.15.5:0/872125775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651076c26b0 0x565107ac8dd0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.404+0000 7fd0fb011740 1 --2- 172.21.15.5:0/872125775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651076ba4d0 0x565107ac8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.404+0000 7fd0fb011740 1 --2- 172.21.15.5:0/872125775 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651076e3390 0x565107aadb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.404+0000 7fd0fb011740 1 -- 172.21.15.5:0/872125775 >> 172.21.15.5:0/872125775 conn(0x5651076d74e0 msgr2=0x565107a831c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.404+0000 7fd0fb011740 1 -- 172.21.15.5:0/872125775 shutdown_connections 2024-05-20T18:32:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.404+0000 7fd0fb011740 1 -- 172.21.15.5:0/872125775 wait complete. 2024-05-20T18:32:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.405+0000 7fd0fb011740 1 -- start start 2024-05-20T18:32:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.405+0000 7fd0fb011740 1 Processor -- start 2024-05-20T18:32:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.406+0000 7fd0fb011740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651076ba4d0 0x565107ac6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.406+0000 7fd0fb011740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651076c26b0 0x565107aa9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.406+0000 7fd0fb011740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651076e3390 0x565107ab09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.406+0000 7fd0fb011740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5651079a2b90 con 0x5651076c26b0 2024-05-20T18:32:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.406+0000 7fd0fb011740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x565107ac6d20 con 0x5651076e3390 2024-05-20T18:32:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.406+0000 7fd0fb011740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x565107aa9a50 con 0x5651076ba4d0 2024-05-20T18:32:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.406+0000 7fd0fb011740 10 client.0 _ll_register_callbacks cb 0x5651079deda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:32:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.406+0000 7fd0f1626640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651076c26b0 0x565107aa9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.407+0000 7fd0f1626640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651076c26b0 0x565107aa9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60818/0 (socket says 172.21.15.5:60818) 2024-05-20T18:32:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.407+0000 7fd0f1626640 1 -- 172.21.15.5:0/886954030 learned_addr learned my addr 172.21.15.5:0/886954030 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.407+0000 7fd0f1e27640 1 --2- 172.21.15.5:0/886954030 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651076ba4d0 0x565107ac6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.407+0000 7fd0f2628640 1 --2- 172.21.15.5:0/886954030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651076e3390 0x565107ab09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.407+0000 7fd0f1626640 1 -- 172.21.15.5:0/886954030 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651076ba4d0 msgr2=0x565107ac6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.407+0000 7fd0f1626640 1 --2- 172.21.15.5:0/886954030 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651076ba4d0 0x565107ac6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.407+0000 7fd0f1626640 1 -- 172.21.15.5:0/886954030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651076e3390 msgr2=0x565107ab09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.407+0000 7fd0f1626640 1 --2- 172.21.15.5:0/886954030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651076e3390 0x565107ab09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.407+0000 7fd0f1626640 1 -- 172.21.15.5:0/886954030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651076fb3f0 con 0x5651076c26b0 2024-05-20T18:32:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.407+0000 7fd0f1626640 1 --2- 172.21.15.5:0/886954030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651076c26b0 0x565107aa9220 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc006e60 tx=0x7fd0dc0794b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97628f47b1d1a9a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.407+0000 7fd0ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:32:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.407+0000 7fd0ccff9640 1 -- 172.21.15.5:0/886954030 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc081ad0 con 0x5651076c26b0 2024-05-20T18:32:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.407+0000 7fd0ccff9640 1 -- 172.21.15.5:0/886954030 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7fd0dc081c70 con 0x5651076c26b0 2024-05-20T18:32:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.407+0000 7fd0fb011740 1 -- 172.21.15.5:0/886954030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0e4015aa0 con 0x5651076c26b0 2024-05-20T18:32:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.407+0000 7fd0ccff9640 1 -- 172.21.15.5:0/886954030 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fd0dc0910c0 con 0x5651076c26b0 2024-05-20T18:32:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.407+0000 7fd0fb011740 10 client.7182 fetch_fsmap learned FSMap version 51 2024-05-20T18:32:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.407+0000 7fd0fb011740 1 -- 172.21.15.5:0/886954030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x5651076bb0f0 con 0x5651076c26b0 2024-05-20T18:32:21.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.407+0000 7fd0ccff9640 1 -- 172.21.15.5:0/886954030 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0e4015aa0 con 0x5651076c26b0 2024-05-20T18:32:21.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.408+0000 7fd0ccff9640 1 -- 172.21.15.5:0/886954030 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7fd0dc03a030 con 0x5651076c26b0 2024-05-20T18:32:21.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.408+0000 7fd0fb011740 10 client.7182 fetch_fsmap finished waiting for FSMap version 51 2024-05-20T18:32:21.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.408+0000 7fd0fb011740 10 client.7182 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-05-20T18:32:21.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.408+0000 7fd0fb011740 20 client.7182 populate_metadata read hostname 'smithi005' 2024-05-20T18:32:21.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.408+0000 7fd0fb011740 1 --2- 172.21.15.5:0/886954030 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5651076a1f60 0x565107ac9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.408+0000 7fd0fb011740 4 client.7182 mds_command: new command op to 6972 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:32:21.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.408+0000 7fd0fb011740 1 -- 172.21.15.5:0/886954030 --> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651079f5430 con 0x5651076a1f60 2024-05-20T18:32:21.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.408+0000 7fd0f1e27640 1 --2- 172.21.15.5:0/886954030 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5651076a1f60 0x565107ac9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.408+0000 7fd0ccff9640 10 client.7182 ms_handle_connect on v2:172.21.15.5:6835/3866822752 2024-05-20T18:32:21.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.408+0000 7fd0f1e27640 1 --2- 172.21.15.5:0/886954030 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5651076a1f60 0x565107ac9070 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fd277d44629b967 server_cookie=99a97fbaa75e0638 in_seq=0 out_seq=0 2024-05-20T18:32:21.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.409+0000 7fd0ccff9640 1 -- 172.21.15.5:0/886954030 <== mds.0 v2:172.21.15.5:6835/3866822752 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5651079f5430 con 0x5651076a1f60 2024-05-20T18:32:21.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.409+0000 7fd0ccff9640 10 client.7182 handle_command_reply: tid=0 2024-05-20T18:32:21.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.409+0000 7fd0fb011740 1 client.7182 shutdown 2024-05-20T18:32:21.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.409+0000 7fd0fb011740 1 -- 172.21.15.5:0/886954030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651076c26b0 msgr2=0x565107aa9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:21.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.409+0000 7fd0fb011740 1 --2- 172.21.15.5:0/886954030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651076c26b0 0x565107aa9220 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc006e60 tx=0x7fd0dc0794b0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.409+0000 7fd0fb011740 1 -- 172.21.15.5:0/886954030 shutdown_connections 2024-05-20T18:32:21.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.409+0000 7fd0fb011740 1 --2- 172.21.15.5:0/886954030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651076e3390 0x565107ab09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.409+0000 7fd0fb011740 1 --2- 172.21.15.5:0/886954030 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5651076a1f60 0x565107ac9070 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.409+0000 7fd0fb011740 1 --2- 172.21.15.5:0/886954030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651076c26b0 0x565107aa9220 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.409+0000 7fd0fb011740 1 --2- 172.21.15.5:0/886954030 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651076ba4d0 0x565107ac6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.409+0000 7fd0ccff9640 0 client.7182 ms_handle_reset on v2:172.21.15.5:6835/3866822752 2024-05-20T18:32:21.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.409+0000 7fd0fb011740 1 -- 172.21.15.5:0/886954030 >> 172.21.15.5:0/886954030 conn(0x5651076d74e0 msgr2=0x565107a831c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:21.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.409+0000 7fd0fb011740 1 -- 172.21.15.5:0/886954030 shutdown_connections 2024-05-20T18:32:21.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.409+0000 7fd0fb011740 1 -- 172.21.15.5:0/886954030 wait complete. 2024-05-20T18:32:21.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.409+0000 7fd0fb011740 20 client.7182 trim_cache size 0 max 16384 2024-05-20T18:32:21.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.428+0000 7fd0fb011740 1 Processor -- start 2024-05-20T18:32:21.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.428+0000 7fd0fb011740 1 -- start start 2024-05-20T18:32:21.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.429+0000 7fd0fb011740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651076ba4d0 0x565107aaee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.429+0000 7fd0fb011740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651076c26b0 0x565107a7cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.429+0000 7fd0fb011740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651076e3390 0x565107a7d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.429+0000 7fd0fb011740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec202420 con 0x5651076e3390 2024-05-20T18:32:21.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.429+0000 7fd0fb011740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec201420 con 0x5651076c26b0 2024-05-20T18:32:21.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.429+0000 7fd0fb011740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd0ec0ba190 con 0x5651076ba4d0 2024-05-20T18:32:21.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.429+0000 7fd0f2628640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651076e3390 0x565107a7d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.429+0000 7fd0f2628640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651076e3390 0x565107a7d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60820/0 (socket says 172.21.15.5:60820) 2024-05-20T18:32:21.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.429+0000 7fd0f2628640 1 -- 172.21.15.5:0/2360347657 learned_addr learned my addr 172.21.15.5:0/2360347657 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:21.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.429+0000 7fd0f1e27640 1 --2- 172.21.15.5:0/2360347657 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651076ba4d0 0x565107aaee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.429+0000 7fd0f1626640 1 --2- 172.21.15.5:0/2360347657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651076c26b0 0x565107a7cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.429+0000 7fd0f2628640 1 -- 172.21.15.5:0/2360347657 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651076ba4d0 msgr2=0x565107aaee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:21.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.429+0000 7fd0f2628640 1 --2- 172.21.15.5:0/2360347657 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651076ba4d0 0x565107aaee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.429+0000 7fd0f2628640 1 -- 172.21.15.5:0/2360347657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651076c26b0 msgr2=0x565107a7cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:21.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.429+0000 7fd0f2628640 1 --2- 172.21.15.5:0/2360347657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651076c26b0 0x565107a7cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.429+0000 7fd0f2628640 1 -- 172.21.15.5:0/2360347657 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565107ac93e0 con 0x5651076e3390 2024-05-20T18:32:21.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.429+0000 7fd0f1e27640 1 --2- 172.21.15.5:0/2360347657 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651076ba4d0 0x565107aaee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:32:21.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.430+0000 7fd0f2628640 1 --2- 172.21.15.5:0/2360347657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651076e3390 0x565107a7d0d0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4019000 tx=0x7fd0e4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc56f7d19213c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:21.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.430+0000 7fd0ceffd640 1 -- 172.21.15.5:0/2360347657 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4015700 con 0x5651076e3390 2024-05-20T18:32:21.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.430+0000 7fd0ceffd640 1 -- 172.21.15.5:0/2360347657 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0e402c070 con 0x5651076e3390 2024-05-20T18:32:21.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.430+0000 7fd0ceffd640 1 -- 172.21.15.5:0/2360347657 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4015b00 con 0x5651076e3390 2024-05-20T18:32:21.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.431+0000 7fd0fb011740 1 -- 172.21.15.5:0/2360347657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651076e3390 msgr2=0x565107a7d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:21.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.431+0000 7fd0fb011740 1 --2- 172.21.15.5:0/2360347657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651076e3390 0x565107a7d0d0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4019000 tx=0x7fd0e4017000 comp rx=0 tx=0).stop 2024-05-20T18:32:21.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.431+0000 7fd0fb011740 1 -- 172.21.15.5:0/2360347657 shutdown_connections 2024-05-20T18:32:21.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.431+0000 7fd0fb011740 1 --2- 172.21.15.5:0/2360347657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651076e3390 0x565107a7d0d0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.431+0000 7fd0fb011740 1 --2- 172.21.15.5:0/2360347657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651076c26b0 0x565107a7cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.431+0000 7fd0fb011740 1 --2- 172.21.15.5:0/2360347657 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651076ba4d0 0x565107aaee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.431+0000 7fd0fb011740 1 -- 172.21.15.5:0/2360347657 >> 172.21.15.5:0/2360347657 conn(0x5651076d74e0 msgr2=0x565107ab07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:21.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.431+0000 7fd0fb011740 1 -- 172.21.15.5:0/2360347657 shutdown_connections 2024-05-20T18:32:21.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.431+0000 7fd0fb011740 1 -- 172.21.15.5:0/2360347657 wait complete. 2024-05-20T18:32:21.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.431+0000 7fd0fb011740 1 -- start start 2024-05-20T18:32:21.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.432+0000 7fd0fb011740 1 Processor -- start 2024-05-20T18:32:21.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.433+0000 7fd0fb011740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651076ba4d0 0x565107a7cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.433+0000 7fd0fb011740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651076c26b0 0x565107a7d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.433+0000 7fd0fb011740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651076e3390 0x565107b45a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.433+0000 7fd0fb011740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x565107adb060 con 0x5651076e3390 2024-05-20T18:32:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.433+0000 7fd0fb011740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec200e00 con 0x5651076c26b0 2024-05-20T18:32:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.433+0000 7fd0fb011740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x565107aa5f40 con 0x5651076ba4d0 2024-05-20T18:32:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.433+0000 7fd0fb011740 10 client.0 _ll_register_callbacks cb 0x7fd0dc009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:32:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.433+0000 7fd0f2628640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651076e3390 0x565107b45a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.433+0000 7fd0f2628640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651076e3390 0x565107b45a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60834/0 (socket says 172.21.15.5:60834) 2024-05-20T18:32:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.433+0000 7fd0f2628640 1 -- 172.21.15.5:0/604092026 learned_addr learned my addr 172.21.15.5:0/604092026 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.433+0000 7fd0f1e27640 1 --2- 172.21.15.5:0/604092026 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651076ba4d0 0x565107a7cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.433+0000 7fd0f1626640 1 --2- 172.21.15.5:0/604092026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651076c26b0 0x565107a7d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.434+0000 7fd0f2628640 1 -- 172.21.15.5:0/604092026 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651076ba4d0 msgr2=0x565107a7cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:21.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.434+0000 7fd0f2628640 1 --2- 172.21.15.5:0/604092026 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651076ba4d0 0x565107a7cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.434+0000 7fd0f2628640 1 -- 172.21.15.5:0/604092026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651076c26b0 msgr2=0x565107a7d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:21.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.434+0000 7fd0f2628640 1 --2- 172.21.15.5:0/604092026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651076c26b0 0x565107a7d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.434+0000 7fd0f2628640 1 -- 172.21.15.5:0/604092026 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565107acaa80 con 0x5651076e3390 2024-05-20T18:32:21.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.434+0000 7fd0f2628640 1 --2- 172.21.15.5:0/604092026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651076e3390 0x565107b45a10 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400b010 tx=0x7fd0e4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70cf04078ae8bca0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:21.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.434+0000 7fd0ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:32:21.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.434+0000 7fd0ccff9640 1 -- 172.21.15.5:0/604092026 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e401d810 con 0x5651076e3390 2024-05-20T18:32:21.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.434+0000 7fd0ccff9640 1 -- 172.21.15.5:0/604092026 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7fd0e401d9b0 con 0x5651076e3390 2024-05-20T18:32:21.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.434+0000 7fd0fb011740 1 -- 172.21.15.5:0/604092026 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0e402c070 con 0x5651076e3390 2024-05-20T18:32:21.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.434+0000 7fd0ccff9640 1 -- 172.21.15.5:0/604092026 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fd0e4041050 con 0x5651076e3390 2024-05-20T18:32:21.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.434+0000 7fd0fb011740 10 client.7188 fetch_fsmap learned FSMap version 51 2024-05-20T18:32:21.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.434+0000 7fd0fb011740 1 -- 172.21.15.5:0/604092026 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x565107a83380 con 0x5651076e3390 2024-05-20T18:32:21.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.434+0000 7fd0ccff9640 1 -- 172.21.15.5:0/604092026 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0e402c070 con 0x5651076e3390 2024-05-20T18:32:21.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.434+0000 7fd0ccff9640 1 -- 172.21.15.5:0/604092026 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7fd0e4044020 con 0x5651076e3390 2024-05-20T18:32:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.434+0000 7fd0fb011740 10 client.7188 fetch_fsmap finished waiting for FSMap version 51 2024-05-20T18:32:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.434+0000 7fd0fb011740 10 client.7188 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-05-20T18:32:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.434+0000 7fd0fb011740 20 client.7188 populate_metadata read hostname 'smithi005' 2024-05-20T18:32:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.434+0000 7fd0fb011740 1 --2- 172.21.15.5:0/604092026 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x565107ac7aa0 0x565107b45f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.434+0000 7fd0fb011740 4 client.7188 mds_command: new command op to 6972 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T18:32:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.434+0000 7fd0fb011740 1 -- 172.21.15.5:0/604092026 --> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5651079ef780 con 0x565107ac7aa0 2024-05-20T18:32:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.435+0000 7fd0f1e27640 1 --2- 172.21.15.5:0/604092026 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x565107ac7aa0 0x565107b45f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.435+0000 7fd0ccff9640 10 client.7188 ms_handle_connect on v2:172.21.15.5:6835/3866822752 2024-05-20T18:32:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.435+0000 7fd0f1e27640 1 --2- 172.21.15.5:0/604092026 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x565107ac7aa0 0x565107b45f50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51b35943e486f40f server_cookie=57ca661c72c37a63 in_seq=0 out_seq=0 2024-05-20T18:32:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.435+0000 7fd0ccff9640 1 -- 172.21.15.5:0/604092026 <== mds.0 v2:172.21.15.5:6835/3866822752 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x5651079ef780 con 0x565107ac7aa0 2024-05-20T18:32:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.435+0000 7fd0ccff9640 10 client.7188 handle_command_reply: tid=0 2024-05-20T18:32:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.435+0000 7fd0fb011740 1 client.7188 shutdown 2024-05-20T18:32:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.435+0000 7fd0fb011740 1 -- 172.21.15.5:0/604092026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651076e3390 msgr2=0x565107b45a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.435+0000 7fd0fb011740 1 --2- 172.21.15.5:0/604092026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651076e3390 0x565107b45a10 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400b010 tx=0x7fd0e4004ab0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.435+0000 7fd0fb011740 1 -- 172.21.15.5:0/604092026 shutdown_connections 2024-05-20T18:32:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.435+0000 7fd0fb011740 1 --2- 172.21.15.5:0/604092026 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x565107ac7aa0 0x565107b45f50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.435+0000 7fd0fb011740 1 --2- 172.21.15.5:0/604092026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651076e3390 0x565107b45a10 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.435+0000 7fd0fb011740 1 --2- 172.21.15.5:0/604092026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651076c26b0 0x565107a7d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.435+0000 7fd0ccff9640 0 client.7188 ms_handle_reset on v2:172.21.15.5:6835/3866822752 2024-05-20T18:32:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.435+0000 7fd0fb011740 1 --2- 172.21.15.5:0/604092026 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651076ba4d0 0x565107a7cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.435+0000 7fd0fb011740 1 -- 172.21.15.5:0/604092026 >> 172.21.15.5:0/604092026 conn(0x5651076d74e0 msgr2=0x565107aadb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.436+0000 7fd0fb011740 1 -- 172.21.15.5:0/604092026 shutdown_connections 2024-05-20T18:32:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.436+0000 7fd0fb011740 1 -- 172.21.15.5:0/604092026 wait complete. 2024-05-20T18:32:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.436+0000 7fd0fb011740 20 client.7188 trim_cache size 0 max 16384 2024-05-20T18:32:21.441 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:32:21.442 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-20T18:32:21.442 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-20T18:32:21.442 INFO:teuthology.orchestra.run.smithi005.stdout: "519300df-413d-44e0-86b3-c6d2904a3f72": { 2024-05-20T18:32:21.442 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/subdir", 2024-05-20T18:32:21.442 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "519300df-413d-44e0-86b3-c6d2904a3f72", 2024-05-20T18:32:21.442 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,repair" 2024-05-20T18:32:21.442 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:32:21.442 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:32:21.442 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:32:21.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.436+0000 7fd0f408b640 1 -- 172.21.15.5:0/1977126668 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd0d006be30 msgr2=0x7fd0d006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:21.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.436+0000 7fd0f408b640 1 --2- 172.21.15.5:0/1977126668 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd0d006be30 0x7fd0d006e340 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8005880 tx=0x7fd0e8009040 comp rx=0 tx=0).stop 2024-05-20T18:32:21.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.436+0000 7fd0f408b640 1 -- 172.21.15.5:0/1977126668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0ec0d3220 msgr2=0x7fd0ec1571d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:21.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.436+0000 7fd0f408b640 1 --2- 172.21.15.5:0/1977126668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0ec0d3220 0x7fd0ec1571d0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc0098c0 tx=0x7fd0dc0046c0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.436+0000 7fd0f408b640 1 -- 172.21.15.5:0/1977126668 shutdown_connections 2024-05-20T18:32:21.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.436+0000 7fd0f408b640 1 --2- 172.21.15.5:0/1977126668 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd0d006be30 0x7fd0d006e340 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.436+0000 7fd0f408b640 1 --2- 172.21.15.5:0/1977126668 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0ec161730 0x7fd0ec161b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.436+0000 7fd0f408b640 1 --2- 172.21.15.5:0/1977126668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0ec0d3220 0x7fd0ec1571d0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.436+0000 7fd0f408b640 1 --2- 172.21.15.5:0/1977126668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0ec0cf250 0x7fd0ec156c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:21.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.436+0000 7fd0f408b640 1 -- 172.21.15.5:0/1977126668 >> 172.21.15.5:0/1977126668 conn(0x7fd0ec0d1860 msgr2=0x7fd0ec15abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:21.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.436+0000 7fd0f408b640 1 -- 172.21.15.5:0/1977126668 shutdown_connections 2024-05-20T18:32:21.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:21.436+0000 7fd0f408b640 1 -- 172.21.15.5:0/1977126668 wait complete. 2024-05-20T18:32:21.448 INFO:tasks.cephfs.filesystem:scrub status for tag:519300df-413d-44e0-86b3-c6d2904a3f72 - {'path': '/subdir', 'tag': '519300df-413d-44e0-86b3-c6d2904a3f72', 'options': 'recursive,repair'} 2024-05-20T18:32:41.398 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:32:41.402 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:32:51.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:32:51.449 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-05-20T18:32:51.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.580+0000 7f397eebc640 1 Processor -- start 2024-05-20T18:32:51.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.580+0000 7f397eebc640 1 -- start start 2024-05-20T18:32:51.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.581+0000 7f397eebc640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f397806d3d0 0x7f397806d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:51.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.581+0000 7f397eebc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f39780b8190 0x7f39780b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:51.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.581+0000 7f397eebc640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f397806dd20 con 0x7f3978162820 2024-05-20T18:32:51.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.581+0000 7f397eebc640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f39780d0630 con 0x7f39780b8190 2024-05-20T18:32:51.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.581+0000 7f397eebc640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f39780d39c0 con 0x7f397806d3d0 2024-05-20T18:32:51.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.581+0000 7f397d459640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f39780b8190 0x7f39780b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:51.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.581+0000 7f397d459640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f39780b8190 0x7f39780b85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37382/0 (socket says 172.21.15.5:37382) 2024-05-20T18:32:51.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.581+0000 7f397d459640 1 -- 172.21.15.5:0/1630814655 learned_addr learned my addr 172.21.15.5:0/1630814655 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:51.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.582+0000 7f3977fff640 1 --2- 172.21.15.5:0/1630814655 >> v2:172.21.15.63:3300/0 conn(0x7f397806d3d0 0x7f397806d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:51.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.582+0000 7f397d459640 1 -- 172.21.15.5:0/1630814655 >> v2:172.21.15.63:3300/0 conn(0x7f397806d3d0 msgr2=0x7f397806d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:51.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.582+0000 7f397d459640 1 --2- 172.21.15.5:0/1630814655 >> v2:172.21.15.63:3300/0 conn(0x7f397806d3d0 0x7f397806d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.582+0000 7f397d459640 1 -- 172.21.15.5:0/1630814655 >> v1:172.21.15.63:6789/0 conn(0x7f3978162820 legacy=0x7f39780b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:51.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.582+0000 7f397d459640 1 -- 172.21.15.5:0/1630814655 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39780d3ca0 con 0x7f39780b8190 2024-05-20T18:32:51.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.582+0000 7f3977fff640 1 --2- 172.21.15.5:0/1630814655 >> v2:172.21.15.63:3300/0 conn(0x7f397806d3d0 0x7f397806d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:32:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.583+0000 7f397d459640 1 --2- 172.21.15.5:0/1630814655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39780b8190 0x7f39780b85a0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f39780d1230 tx=0x7f397002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3938ed2fabd4dddf server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.583+0000 7f39777fe640 1 -- 172.21.15.5:0/1630814655 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970036070 con 0x7f39780b8190 2024-05-20T18:32:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.583+0000 7f39777fe640 1 -- 172.21.15.5:0/1630814655 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3970031600 con 0x7f39780b8190 2024-05-20T18:32:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.583+0000 7f39777fe640 1 -- 172.21.15.5:0/1630814655 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39700319d0 con 0x7f39780b8190 2024-05-20T18:32:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.583+0000 7f397eebc640 1 -- 172.21.15.5:0/1630814655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39780b8190 msgr2=0x7f39780b85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.583+0000 7f397eebc640 1 --2- 172.21.15.5:0/1630814655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39780b8190 0x7f39780b85a0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f39780d1230 tx=0x7f397002a6f0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.583+0000 7f397eebc640 1 -- 172.21.15.5:0/1630814655 shutdown_connections 2024-05-20T18:32:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.583+0000 7f397eebc640 1 --2- 172.21.15.5:0/1630814655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39780b8190 0x7f39780b85a0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.583+0000 7f397eebc640 1 --2- 172.21.15.5:0/1630814655 >> v2:172.21.15.63:3300/0 conn(0x7f397806d3d0 0x7f397806d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.583+0000 7f397eebc640 1 -- 172.21.15.5:0/1630814655 >> 172.21.15.5:0/1630814655 conn(0x7f39780d4bd0 msgr2=0x7f39780d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.584+0000 7f397eebc640 1 -- 172.21.15.5:0/1630814655 shutdown_connections 2024-05-20T18:32:51.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.584+0000 7f397eebc640 1 -- 172.21.15.5:0/1630814655 wait complete. 2024-05-20T18:32:51.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.585+0000 7f397eebc640 1 Processor -- start 2024-05-20T18:32:51.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.585+0000 7f397eebc640 1 -- start start 2024-05-20T18:32:51.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.587+0000 7f397eebc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f397806d3d0 0x7f397814a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:51.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.587+0000 7f397eebc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f39780b8190 0x7f397814bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:51.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.587+0000 7f397eebc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3978162820 0x7f397814c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:51.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.587+0000 7f397eebc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39780d0300 con 0x7f397806d3d0 2024-05-20T18:32:51.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.587+0000 7f397eebc640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f39780d4260 con 0x7f3978162820 2024-05-20T18:32:51.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.587+0000 7f397eebc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f39780b4050 con 0x7f39780b8190 2024-05-20T18:32:51.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.587+0000 7f397cc58640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f397806d3d0 0x7f397814a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:51.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.587+0000 7f397cc58640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f397806d3d0 0x7f397814a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37384/0 (socket says 172.21.15.5:37384) 2024-05-20T18:32:51.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.587+0000 7f397cc58640 1 -- 172.21.15.5:0/591854512 learned_addr learned my addr 172.21.15.5:0/591854512 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:51.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.587+0000 7f397d459640 1 --2- 172.21.15.5:0/591854512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3978162820 0x7f397814c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:51.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.587+0000 7f3977fff640 1 --2- 172.21.15.5:0/591854512 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f39780b8190 0x7f397814bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:51.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.587+0000 7f397cc58640 1 -- 172.21.15.5:0/591854512 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f39780b8190 msgr2=0x7f397814bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:51.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.587+0000 7f397cc58640 1 --2- 172.21.15.5:0/591854512 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f39780b8190 0x7f397814bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.587+0000 7f397cc58640 1 -- 172.21.15.5:0/591854512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3978162820 msgr2=0x7f397814c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:51.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.587+0000 7f397cc58640 1 --2- 172.21.15.5:0/591854512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3978162820 0x7f397814c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.587+0000 7f397cc58640 1 -- 172.21.15.5:0/591854512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f397814c9c0 con 0x7f397806d3d0 2024-05-20T18:32:51.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.588+0000 7f397cc58640 1 --2- 172.21.15.5:0/591854512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f397806d3d0 0x7f397814a2e0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f396800b3f0 tx=0x7f396800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:51.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.588+0000 7f39757fa640 1 -- 172.21.15.5:0/591854512 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968015020 con 0x7f397806d3d0 2024-05-20T18:32:51.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.588+0000 7f39757fa640 1 -- 172.21.15.5:0/591854512 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3968011830 con 0x7f397806d3d0 2024-05-20T18:32:51.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.588+0000 7f397eebc640 1 -- 172.21.15.5:0/591854512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f397814e170 con 0x7f397806d3d0 2024-05-20T18:32:51.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.588+0000 7f39757fa640 1 -- 172.21.15.5:0/591854512 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968011c20 con 0x7f397806d3d0 2024-05-20T18:32:51.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.588+0000 7f397eebc640 1 -- 172.21.15.5:0/591854512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f397814e730 con 0x7f397806d3d0 2024-05-20T18:32:51.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.590+0000 7f39757fa640 1 -- 172.21.15.5:0/591854512 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3968018020 con 0x7f397806d3d0 2024-05-20T18:32:51.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.590+0000 7f39757fa640 1 --2- 172.21.15.5:0/591854512 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f396406be30 0x7f396406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:51.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.590+0000 7f39757fa640 1 -- 172.21.15.5:0/591854512 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f396808ace0 con 0x7f397806d3d0 2024-05-20T18:32:51.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.590+0000 7f3977fff640 1 --2- 172.21.15.5:0/591854512 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f396406be30 0x7f396406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:51.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.591+0000 7f3977fff640 1 --2- 172.21.15.5:0/591854512 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f396406be30 0x7f396406e340 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f39780ba690 tx=0x7f396c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:51.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.591+0000 7f39859fd740 1 Processor -- start 2024-05-20T18:32:51.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.591+0000 7f39859fd740 1 -- start start 2024-05-20T18:32:51.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.592+0000 7f39859fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559195ca0390 0x55919606ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:51.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.592+0000 7f39859fd740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559195c774d0 0x559196085890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:51.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.592+0000 7f39859fd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559195c7f6b0 0x559196085dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:51.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.592+0000 7f39859fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559195fa4d00 con 0x559195ca0390 2024-05-20T18:32:51.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.592+0000 7f39859fd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559195f4c4c0 con 0x559195c7f6b0 2024-05-20T18:32:51.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.592+0000 7f39859fd740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x559195fa0cc0 con 0x559195c774d0 2024-05-20T18:32:51.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.593+0000 7f397cc58640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559195ca0390 0x55919606ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:51.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.593+0000 7f397cc58640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559195ca0390 0x55919606ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37396/0 (socket says 172.21.15.5:37396) 2024-05-20T18:32:51.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.593+0000 7f397cc58640 1 -- 172.21.15.5:0/3103927570 learned_addr learned my addr 172.21.15.5:0/3103927570 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.593+0000 7f3977fff640 1 --2- 172.21.15.5:0/3103927570 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559195c774d0 0x559196085890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.593+0000 7f397d459640 1 --2- 172.21.15.5:0/3103927570 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559195c7f6b0 0x559196085dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.593+0000 7f397cc58640 1 -- 172.21.15.5:0/3103927570 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559195c774d0 msgr2=0x559196085890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.593+0000 7f397cc58640 1 --2- 172.21.15.5:0/3103927570 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559195c774d0 0x559196085890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.593+0000 7f397cc58640 1 -- 172.21.15.5:0/3103927570 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559195c7f6b0 msgr2=0x559196085dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.593+0000 7f397cc58640 1 --2- 172.21.15.5:0/3103927570 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559195c7f6b0 0x559196085dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.593+0000 7f397cc58640 1 -- 172.21.15.5:0/3103927570 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559195d827d0 con 0x559195ca0390 2024-05-20T18:32:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.593+0000 7f397cc58640 1 --2- 172.21.15.5:0/3103927570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559195ca0390 0x55919606ab60 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f3968009370 tx=0x7f39680619b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4833a2774b112c0e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.593+0000 7f395e7fc640 1 -- 172.21.15.5:0/3103927570 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968057030 con 0x559195ca0390 2024-05-20T18:32:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.593+0000 7f395e7fc640 1 -- 172.21.15.5:0/3103927570 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39680553a0 con 0x559195ca0390 2024-05-20T18:32:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.593+0000 7f395e7fc640 1 -- 172.21.15.5:0/3103927570 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39680583c0 con 0x559195ca0390 2024-05-20T18:32:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.594+0000 7f39859fd740 1 -- 172.21.15.5:0/3103927570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559195ca0390 msgr2=0x55919606ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.594+0000 7f39859fd740 1 --2- 172.21.15.5:0/3103927570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559195ca0390 0x55919606ab60 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f3968009370 tx=0x7f39680619b0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.594+0000 7f39859fd740 1 -- 172.21.15.5:0/3103927570 shutdown_connections 2024-05-20T18:32:51.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.594+0000 7f39859fd740 1 --2- 172.21.15.5:0/3103927570 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559195c7f6b0 0x559196085dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.594+0000 7f39859fd740 1 --2- 172.21.15.5:0/3103927570 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559195c774d0 0x559196085890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.594+0000 7f39859fd740 1 --2- 172.21.15.5:0/3103927570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559195ca0390 0x55919606ab60 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.594+0000 7f39859fd740 1 -- 172.21.15.5:0/3103927570 >> 172.21.15.5:0/3103927570 conn(0x559195c944e0 msgr2=0x5591960401c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:51.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.594+0000 7f39859fd740 1 -- 172.21.15.5:0/3103927570 shutdown_connections 2024-05-20T18:32:51.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.594+0000 7f39859fd740 1 -- 172.21.15.5:0/3103927570 wait complete. 2024-05-20T18:32:51.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.595+0000 7f39859fd740 1 -- start start 2024-05-20T18:32:51.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.595+0000 7f39859fd740 1 Processor -- start 2024-05-20T18:32:51.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.597+0000 7f39859fd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559195c774d0 0x55919606d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:51.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.597+0000 7f39859fd740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559195c7f6b0 0x559196039f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:51.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.597+0000 7f39859fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559195ca0390 0x55919603a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:51.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.597+0000 7f39859fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559195f5fb90 con 0x559195ca0390 2024-05-20T18:32:51.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.597+0000 7f39859fd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55919607e040 con 0x559195c774d0 2024-05-20T18:32:51.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.597+0000 7f39859fd740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55919606beb0 con 0x559195c7f6b0 2024-05-20T18:32:51.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.597+0000 7f39859fd740 10 client.0 _ll_register_callbacks cb 0x559195f9bda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:32:51.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.597+0000 7f397d459640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559195ca0390 0x55919603a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:51.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.597+0000 7f397d459640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559195ca0390 0x55919603a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37406/0 (socket says 172.21.15.5:37406) 2024-05-20T18:32:51.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.597+0000 7f397d459640 1 -- 172.21.15.5:0/1143819983 learned_addr learned my addr 172.21.15.5:0/1143819983 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:51.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.597+0000 7f397cc58640 1 --2- 172.21.15.5:0/1143819983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559195c774d0 0x55919606d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:51.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.597+0000 7f3977fff640 1 --2- 172.21.15.5:0/1143819983 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559195c7f6b0 0x559196039f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:51.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.597+0000 7f397d459640 1 -- 172.21.15.5:0/1143819983 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559195c7f6b0 msgr2=0x559196039f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:51.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.597+0000 7f397d459640 1 --2- 172.21.15.5:0/1143819983 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559195c7f6b0 0x559196039f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.597+0000 7f397d459640 1 -- 172.21.15.5:0/1143819983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559195c774d0 msgr2=0x55919606d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:51.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.597+0000 7f397d459640 1 --2- 172.21.15.5:0/1143819983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559195c774d0 0x55919606d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.597+0000 7f397d459640 1 -- 172.21.15.5:0/1143819983 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559195cb83f0 con 0x559195ca0390 2024-05-20T18:32:51.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.597+0000 7f397d459640 1 --2- 172.21.15.5:0/1143819983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559195ca0390 0x55919603a440 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f3970034040 tx=0x7f3970004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd4095fd41890b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:51.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.597+0000 7f39577fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:32:51.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.597+0000 7f39577fe640 1 -- 172.21.15.5:0/1143819983 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970036040 con 0x559195ca0390 2024-05-20T18:32:51.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.598+0000 7f39859fd740 1 -- 172.21.15.5:0/1143819983 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39680553a0 con 0x559195ca0390 2024-05-20T18:32:51.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.598+0000 7f39577fe640 1 -- 172.21.15.5:0/1143819983 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f397003d430 con 0x559195ca0390 2024-05-20T18:32:51.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.598+0000 7f39577fe640 1 -- 172.21.15.5:0/1143819983 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f397004f050 con 0x559195ca0390 2024-05-20T18:32:51.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.598+0000 7f39859fd740 10 client.7200 fetch_fsmap learned FSMap version 52 2024-05-20T18:32:51.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.598+0000 7f39859fd740 1 -- 172.21.15.5:0/1143819983 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x559195c780f0 con 0x559195ca0390 2024-05-20T18:32:51.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.598+0000 7f39577fe640 1 -- 172.21.15.5:0/1143819983 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39680553a0 con 0x559195ca0390 2024-05-20T18:32:51.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.598+0000 7f39577fe640 1 -- 172.21.15.5:0/1143819983 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f3970052020 con 0x559195ca0390 2024-05-20T18:32:51.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.598+0000 7f39859fd740 10 client.7200 fetch_fsmap finished waiting for FSMap version 52 2024-05-20T18:32:51.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.598+0000 7f39859fd740 10 client.7200 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-05-20T18:32:51.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.598+0000 7f39859fd740 20 client.7200 populate_metadata read hostname 'smithi005' 2024-05-20T18:32:51.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.598+0000 7f39859fd740 1 --2- 172.21.15.5:0/1143819983 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x559195c5ef60 0x559196098290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:51.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.598+0000 7f39859fd740 4 client.7200 mds_command: new command op to 6972 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:32:51.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.598+0000 7f39859fd740 1 -- 172.21.15.5:0/1143819983 --> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559195fb2430 con 0x559195c5ef60 2024-05-20T18:32:51.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.598+0000 7f397cc58640 1 --2- 172.21.15.5:0/1143819983 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x559195c5ef60 0x559196098290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:51.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.599+0000 7f39577fe640 10 client.7200 ms_handle_connect on v2:172.21.15.5:6835/3866822752 2024-05-20T18:32:51.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.599+0000 7f397cc58640 1 --2- 172.21.15.5:0/1143819983 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x559195c5ef60 0x559196098290 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=613258385bb3d421 server_cookie=59505459ec63d23a in_seq=0 out_seq=0 2024-05-20T18:32:51.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.599+0000 7f39577fe640 1 -- 172.21.15.5:0/1143819983 <== mds.0 v2:172.21.15.5:6835/3866822752 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559195fb2430 con 0x559195c5ef60 2024-05-20T18:32:51.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.599+0000 7f39577fe640 10 client.7200 handle_command_reply: tid=0 2024-05-20T18:32:51.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.599+0000 7f39859fd740 1 client.7200 shutdown 2024-05-20T18:32:51.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.600+0000 7f39859fd740 1 -- 172.21.15.5:0/1143819983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559195ca0390 msgr2=0x55919603a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:51.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.600+0000 7f39859fd740 1 --2- 172.21.15.5:0/1143819983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559195ca0390 0x55919603a440 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f3970034040 tx=0x7f3970004ab0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.600+0000 7f39859fd740 1 -- 172.21.15.5:0/1143819983 shutdown_connections 2024-05-20T18:32:51.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.600+0000 7f39859fd740 1 --2- 172.21.15.5:0/1143819983 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x559195c5ef60 0x559196098290 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.600+0000 7f39859fd740 1 --2- 172.21.15.5:0/1143819983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559195ca0390 0x55919603a440 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.600+0000 7f39859fd740 1 --2- 172.21.15.5:0/1143819983 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559195c7f6b0 0x559196039f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.600+0000 7f39859fd740 1 --2- 172.21.15.5:0/1143819983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559195c774d0 0x55919606d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.600+0000 7f39577fe640 0 client.7200 ms_handle_reset on v2:172.21.15.5:6835/3866822752 2024-05-20T18:32:51.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.600+0000 7f39859fd740 1 -- 172.21.15.5:0/1143819983 >> 172.21.15.5:0/1143819983 conn(0x559195c944e0 msgr2=0x5591960401c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:51.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.600+0000 7f39859fd740 1 -- 172.21.15.5:0/1143819983 shutdown_connections 2024-05-20T18:32:51.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.600+0000 7f39859fd740 1 -- 172.21.15.5:0/1143819983 wait complete. 2024-05-20T18:32:51.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.600+0000 7f39859fd740 20 client.7200 trim_cache size 0 max 16384 2024-05-20T18:32:51.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.619+0000 7f39859fd740 1 Processor -- start 2024-05-20T18:32:51.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.619+0000 7f39859fd740 1 -- start start 2024-05-20T18:32:51.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.620+0000 7f39859fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559195c774d0 0x55919607b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:51.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.620+0000 7f39859fd740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559195c7f6b0 0x55919606d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:51.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.620+0000 7f39859fd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559195ca0390 0x55919606d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:51.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.620+0000 7f39859fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39782023f0 con 0x559195c774d0 2024-05-20T18:32:51.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.620+0000 7f39859fd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3978201410 con 0x559195ca0390 2024-05-20T18:32:51.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.620+0000 7f39859fd740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f397814fad0 con 0x559195c7f6b0 2024-05-20T18:32:51.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.620+0000 7f397cc58640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559195c774d0 0x55919607b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:51.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.620+0000 7f397cc58640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559195c774d0 0x55919607b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37422/0 (socket says 172.21.15.5:37422) 2024-05-20T18:32:51.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.620+0000 7f397cc58640 1 -- 172.21.15.5:0/3266357994 learned_addr learned my addr 172.21.15.5:0/3266357994 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:51.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.620+0000 7f3977fff640 1 --2- 172.21.15.5:0/3266357994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559195c7f6b0 0x55919606d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:51.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.620+0000 7f397cc58640 1 -- 172.21.15.5:0/3266357994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559195c7f6b0 msgr2=0x55919606d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:51.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.620+0000 7f397cc58640 1 --2- 172.21.15.5:0/3266357994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559195c7f6b0 0x55919606d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.620+0000 7f397cc58640 1 -- 172.21.15.5:0/3266357994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559195ca0390 msgr2=0x55919606d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:51.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.620+0000 7f397cc58640 1 --2- 172.21.15.5:0/3266357994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559195ca0390 0x55919606d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.620+0000 7f397cc58640 1 -- 172.21.15.5:0/3266357994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55919603e210 con 0x559195c774d0 2024-05-20T18:32:51.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.620+0000 7f397cc58640 1 --2- 172.21.15.5:0/3266357994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559195c774d0 0x55919607b300 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f3968055190 tx=0x7f396805b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5dd77a84644532 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:51.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.621+0000 7f395dffb640 1 -- 172.21.15.5:0/3266357994 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968057030 con 0x559195c774d0 2024-05-20T18:32:51.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.621+0000 7f395dffb640 1 -- 172.21.15.5:0/3266357994 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39680553a0 con 0x559195c774d0 2024-05-20T18:32:51.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.621+0000 7f395dffb640 1 -- 172.21.15.5:0/3266357994 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968058380 con 0x559195c774d0 2024-05-20T18:32:51.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.621+0000 7f39859fd740 1 -- 172.21.15.5:0/3266357994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559195c774d0 msgr2=0x55919607b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:51.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.621+0000 7f39859fd740 1 --2- 172.21.15.5:0/3266357994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559195c774d0 0x55919607b300 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f3968055190 tx=0x7f396805b000 comp rx=0 tx=0).stop 2024-05-20T18:32:51.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.621+0000 7f39859fd740 1 -- 172.21.15.5:0/3266357994 shutdown_connections 2024-05-20T18:32:51.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.621+0000 7f39859fd740 1 --2- 172.21.15.5:0/3266357994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559195ca0390 0x55919606d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.621+0000 7f39859fd740 1 --2- 172.21.15.5:0/3266357994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559195c7f6b0 0x55919606d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.621+0000 7f39859fd740 1 --2- 172.21.15.5:0/3266357994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559195c774d0 0x55919607b300 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.621+0000 7f39859fd740 1 -- 172.21.15.5:0/3266357994 >> 172.21.15.5:0/3266357994 conn(0x559195c944e0 msgr2=0x55919606b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:51.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.621+0000 7f39859fd740 1 -- 172.21.15.5:0/3266357994 shutdown_connections 2024-05-20T18:32:51.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.621+0000 7f39859fd740 1 -- 172.21.15.5:0/3266357994 wait complete. 2024-05-20T18:32:51.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.622+0000 7f39859fd740 1 -- start start 2024-05-20T18:32:51.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.623+0000 7f39859fd740 1 Processor -- start 2024-05-20T18:32:51.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.624+0000 7f39859fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559195c774d0 0x55919606e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:51.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.624+0000 7f39859fd740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559195c7f6b0 0x55919603e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:51.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.624+0000 7f39859fd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559195ca0390 0x559196067720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:51.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.624+0000 7f39859fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3978150120 con 0x559195c774d0 2024-05-20T18:32:51.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.624+0000 7f39859fd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55919603f0b0 con 0x559195ca0390 2024-05-20T18:32:51.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.624+0000 7f39859fd740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5591960408c0 con 0x559195c7f6b0 2024-05-20T18:32:51.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.624+0000 7f39859fd740 10 client.0 _ll_register_callbacks cb 0x7f3970009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:32:51.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.624+0000 7f397cc58640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559195c774d0 0x55919606e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:51.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.624+0000 7f397cc58640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559195c774d0 0x55919606e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37436/0 (socket says 172.21.15.5:37436) 2024-05-20T18:32:51.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.624+0000 7f397cc58640 1 -- 172.21.15.5:0/2419382094 learned_addr learned my addr 172.21.15.5:0/2419382094 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:51.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.624+0000 7f3977fff640 1 --2- 172.21.15.5:0/2419382094 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559195c7f6b0 0x55919603e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:51.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.624+0000 7f397d459640 1 --2- 172.21.15.5:0/2419382094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559195ca0390 0x559196067720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:51.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.624+0000 7f397cc58640 1 -- 172.21.15.5:0/2419382094 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559195c7f6b0 msgr2=0x55919603e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:51.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.624+0000 7f397cc58640 1 --2- 172.21.15.5:0/2419382094 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559195c7f6b0 0x55919603e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.624+0000 7f397cc58640 1 -- 172.21.15.5:0/2419382094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559195ca0390 msgr2=0x559196067720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:51.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.624+0000 7f397cc58640 1 --2- 172.21.15.5:0/2419382094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559195ca0390 0x559196067720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.624+0000 7f397cc58640 1 -- 172.21.15.5:0/2419382094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55919603ebb0 con 0x559195c774d0 2024-05-20T18:32:51.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.624+0000 7f397d459640 1 --2- 172.21.15.5:0/2419382094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559195ca0390 0x559196067720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:32:51.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.625+0000 7f397cc58640 1 --2- 172.21.15.5:0/2419382094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559195c774d0 0x55919606e1d0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f3968004730 tx=0x7f3968009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f62b58b15718d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:51.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.625+0000 7f39577fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:32:51.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.625+0000 7f39577fe640 1 -- 172.21.15.5:0/2419382094 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39680570f0 con 0x559195c774d0 2024-05-20T18:32:51.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.625+0000 7f39577fe640 1 -- 172.21.15.5:0/2419382094 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f3968058390 con 0x559195c774d0 2024-05-20T18:32:51.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.625+0000 7f39859fd740 1 -- 172.21.15.5:0/2419382094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39680553a0 con 0x559195c774d0 2024-05-20T18:32:51.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.625+0000 7f39577fe640 1 -- 172.21.15.5:0/2419382094 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f3968061b40 con 0x559195c774d0 2024-05-20T18:32:51.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.625+0000 7f39859fd740 10 client.7206 fetch_fsmap learned FSMap version 52 2024-05-20T18:32:51.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.625+0000 7f39859fd740 1 -- 172.21.15.5:0/2419382094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55919603ebb0 con 0x559195c774d0 2024-05-20T18:32:51.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.625+0000 7f39577fe640 1 -- 172.21.15.5:0/2419382094 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39680553a0 con 0x559195c774d0 2024-05-20T18:32:51.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.625+0000 7f39577fe640 1 -- 172.21.15.5:0/2419382094 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f3968061ce0 con 0x559195c774d0 2024-05-20T18:32:51.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.625+0000 7f39859fd740 10 client.7206 fetch_fsmap finished waiting for FSMap version 52 2024-05-20T18:32:51.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.625+0000 7f39859fd740 10 client.7206 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-05-20T18:32:51.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.625+0000 7f39859fd740 20 client.7206 populate_metadata read hostname 'smithi005' 2024-05-20T18:32:51.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.625+0000 7f39859fd740 1 --2- 172.21.15.5:0/2419382094 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5591961020c0 0x55919607cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:51.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.625+0000 7f39859fd740 4 client.7206 mds_command: new command op to 6972 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T18:32:51.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.625+0000 7f39859fd740 1 -- 172.21.15.5:0/2419382094 --> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559195fac780 con 0x5591961020c0 2024-05-20T18:32:51.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.625+0000 7f3977fff640 1 --2- 172.21.15.5:0/2419382094 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5591961020c0 0x55919607cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:51.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.626+0000 7f39577fe640 10 client.7206 ms_handle_connect on v2:172.21.15.5:6835/3866822752 2024-05-20T18:32:51.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.626+0000 7f3977fff640 1 --2- 172.21.15.5:0/2419382094 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5591961020c0 0x55919607cac0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8105d0c5458577d5 server_cookie=7f05bab1a632d7fe in_seq=0 out_seq=0 2024-05-20T18:32:51.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.626+0000 7f39577fe640 1 -- 172.21.15.5:0/2419382094 <== mds.0 v2:172.21.15.5:6835/3866822752 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559195fac780 con 0x5591961020c0 2024-05-20T18:32:51.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.626+0000 7f39577fe640 10 client.7206 handle_command_reply: tid=0 2024-05-20T18:32:51.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.626+0000 7f39859fd740 1 client.7206 shutdown 2024-05-20T18:32:51.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.626+0000 7f39859fd740 1 -- 172.21.15.5:0/2419382094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559195c774d0 msgr2=0x55919606e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:51.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.626+0000 7f39859fd740 1 --2- 172.21.15.5:0/2419382094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559195c774d0 0x55919606e1d0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f3968004730 tx=0x7f3968009300 comp rx=0 tx=0).stop 2024-05-20T18:32:51.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.626+0000 7f39859fd740 1 -- 172.21.15.5:0/2419382094 shutdown_connections 2024-05-20T18:32:51.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.626+0000 7f39859fd740 1 --2- 172.21.15.5:0/2419382094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559195ca0390 0x559196067720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.626+0000 7f39859fd740 1 --2- 172.21.15.5:0/2419382094 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559195c7f6b0 0x55919603e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.626+0000 7f39859fd740 1 --2- 172.21.15.5:0/2419382094 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x5591961020c0 0x55919607cac0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.626+0000 7f39859fd740 1 --2- 172.21.15.5:0/2419382094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559195c774d0 0x55919606e1d0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.626+0000 7f39577fe640 0 client.7206 ms_handle_reset on v2:172.21.15.5:6835/3866822752 2024-05-20T18:32:51.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.626+0000 7f39859fd740 1 -- 172.21.15.5:0/2419382094 >> 172.21.15.5:0/2419382094 conn(0x559195c944e0 msgr2=0x55919606b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:51.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.626+0000 7f39859fd740 1 -- 172.21.15.5:0/2419382094 shutdown_connections 2024-05-20T18:32:51.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.626+0000 7f39859fd740 1 -- 172.21.15.5:0/2419382094 wait complete. 2024-05-20T18:32:51.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.627+0000 7f39859fd740 20 client.7206 trim_cache size 0 max 16384 2024-05-20T18:32:51.635 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:32:51.636 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-20T18:32:51.636 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-20T18:32:51.636 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:32:51.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.627+0000 7f397eebc640 1 -- 172.21.15.5:0/591854512 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f396406be30 msgr2=0x7f396406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:51.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.627+0000 7f397eebc640 1 --2- 172.21.15.5:0/591854512 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f396406be30 0x7f396406e340 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f39780ba690 tx=0x7f396c008040 comp rx=0 tx=0).stop 2024-05-20T18:32:51.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.627+0000 7f397eebc640 1 -- 172.21.15.5:0/591854512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f397806d3d0 msgr2=0x7f397814a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:51.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.627+0000 7f397eebc640 1 --2- 172.21.15.5:0/591854512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f397806d3d0 0x7f397814a2e0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f396800b3f0 tx=0x7f396800b8b0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.627+0000 7f397eebc640 1 -- 172.21.15.5:0/591854512 shutdown_connections 2024-05-20T18:32:51.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.627+0000 7f397eebc640 1 --2- 172.21.15.5:0/591854512 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f396406be30 0x7f396406e340 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.627+0000 7f397eebc640 1 --2- 172.21.15.5:0/591854512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3978162820 0x7f397814c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.627+0000 7f397eebc640 1 --2- 172.21.15.5:0/591854512 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f39780b8190 0x7f397814bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.627+0000 7f397eebc640 1 --2- 172.21.15.5:0/591854512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f397806d3d0 0x7f397814a2e0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.627+0000 7f397eebc640 1 -- 172.21.15.5:0/591854512 >> 172.21.15.5:0/591854512 conn(0x7f39780d4bd0 msgr2=0x7f397815db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:51.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.627+0000 7f397eebc640 1 -- 172.21.15.5:0/591854512 shutdown_connections 2024-05-20T18:32:51.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.627+0000 7f397eebc640 1 -- 172.21.15.5:0/591854512 wait complete. 2024-05-20T18:32:51.639 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T18:32:51.639 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:32:51.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.753+0000 7fcd06731640 1 Processor -- start 2024-05-20T18:32:51.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.753+0000 7fcd06731640 1 -- start start 2024-05-20T18:32:51.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.755+0000 7fcd06731640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fcd00162820 0x7fcd000b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:51.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.755+0000 7fcd06731640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd0006dae0 0x7fcd000b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:51.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.755+0000 7fcd06731640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd000d0630 con 0x7fcd000b3e80 2024-05-20T18:32:51.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.755+0000 7fcd06731640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fcd000d39c0 con 0x7fcd00162820 2024-05-20T18:32:51.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.755+0000 7fcd06731640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcd000d3ca0 con 0x7fcd0006dae0 2024-05-20T18:32:51.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.755+0000 7fccfffff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fcd00162820 0x7fcd000b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:51.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.755+0000 7fcd04cce640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd0006dae0 0x7fcd000b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:51.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.755+0000 7fcd04cce640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd0006dae0 0x7fcd000b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:46966/0 (socket says 172.21.15.5:46966) 2024-05-20T18:32:51.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.755+0000 7fcd04cce640 1 -- 172.21.15.5:0/96696790 learned_addr learned my addr 172.21.15.5:0/96696790 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:51.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.755+0000 7fccfffff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fcd00162820 0x7fcd000b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:49616/0 (socket says 172.21.15.5:49616) 2024-05-20T18:32:51.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.755+0000 7fcd04cce640 1 -- 172.21.15.5:0/96696790 >> v2:172.21.15.63:3300/0 conn(0x7fcd00162820 msgr2=0x7fcd000b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:51.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.755+0000 7fcd04cce640 1 --2- 172.21.15.5:0/96696790 >> v2:172.21.15.63:3300/0 conn(0x7fcd00162820 0x7fcd000b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.755+0000 7fcd04cce640 1 -- 172.21.15.5:0/96696790 >> v1:172.21.15.63:6789/0 conn(0x7fcd000b3e80 legacy=0x7fcd0006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:51.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.755+0000 7fcd04cce640 1 -- 172.21.15.5:0/96696790 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd000d3f80 con 0x7fcd0006dae0 2024-05-20T18:32:51.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.756+0000 7fcd04cce640 1 --2- 172.21.15.5:0/96696790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd0006dae0 0x7fcd000b8190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fccf8009e50 tx=0x7fccf802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ea6447e111cf992 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:51.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.756+0000 7fccff7fe640 1 -- 172.21.15.5:0/96696790 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf8036070 con 0x7fcd0006dae0 2024-05-20T18:32:51.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.756+0000 7fccff7fe640 1 -- 172.21.15.5:0/96696790 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccf8031600 con 0x7fcd0006dae0 2024-05-20T18:32:51.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.756+0000 7fccff7fe640 1 -- 172.21.15.5:0/96696790 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf80319d0 con 0x7fcd0006dae0 2024-05-20T18:32:51.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.757+0000 7fcd06731640 1 -- 172.21.15.5:0/96696790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd0006dae0 msgr2=0x7fcd000b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:51.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.757+0000 7fcd06731640 1 --2- 172.21.15.5:0/96696790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd0006dae0 0x7fcd000b8190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fccf8009e50 tx=0x7fccf802a710 comp rx=0 tx=0).stop 2024-05-20T18:32:51.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.757+0000 7fcd06731640 1 -- 172.21.15.5:0/96696790 shutdown_connections 2024-05-20T18:32:51.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.757+0000 7fcd06731640 1 --2- 172.21.15.5:0/96696790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd0006dae0 0x7fcd000b8190 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.757+0000 7fcd06731640 1 --2- 172.21.15.5:0/96696790 >> v2:172.21.15.63:3300/0 conn(0x7fcd00162820 0x7fcd000b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.757+0000 7fcd06731640 1 -- 172.21.15.5:0/96696790 >> 172.21.15.5:0/96696790 conn(0x7fcd000d4bd0 msgr2=0x7fcd000d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:51.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.757+0000 7fcd06731640 1 -- 172.21.15.5:0/96696790 shutdown_connections 2024-05-20T18:32:51.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.757+0000 7fcd06731640 1 -- 172.21.15.5:0/96696790 wait complete. 2024-05-20T18:32:51.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.759+0000 7fcd06731640 1 Processor -- start 2024-05-20T18:32:51.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.759+0000 7fcd06731640 1 -- start start 2024-05-20T18:32:51.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.760+0000 7fcd06731640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd000b3e80 0x7fcd0014c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:51.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.760+0000 7fcd06731640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd00162820 0x7fcd0014cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:51.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.760+0000 7fcd06731640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd000c8700 0x7fcd00152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:51.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.760+0000 7fcd06731640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcd000d4780 con 0x7fcd000b3e80 2024-05-20T18:32:51.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.760+0000 7fcd06731640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcd000d0300 con 0x7fcd00162820 2024-05-20T18:32:51.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.760+0000 7fcd06731640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcd000d4900 con 0x7fcd000c8700 2024-05-20T18:32:51.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.760+0000 7fccfffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd000b3e80 0x7fcd0014c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:51.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.761+0000 7fccfffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd000b3e80 0x7fcd0014c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37438/0 (socket says 172.21.15.5:37438) 2024-05-20T18:32:51.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.761+0000 7fccfffff640 1 -- 172.21.15.5:0/875250232 learned_addr learned my addr 172.21.15.5:0/875250232 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:51.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.761+0000 7fcd04cce640 1 --2- 172.21.15.5:0/875250232 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd000c8700 0x7fcd00152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:51.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.761+0000 7fccf7fff640 1 --2- 172.21.15.5:0/875250232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd00162820 0x7fcd0014cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:51.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.761+0000 7fccfffff640 1 -- 172.21.15.5:0/875250232 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd000c8700 msgr2=0x7fcd00152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:51.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.761+0000 7fccfffff640 1 --2- 172.21.15.5:0/875250232 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd000c8700 0x7fcd00152860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.761+0000 7fccfffff640 1 -- 172.21.15.5:0/875250232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd00162820 msgr2=0x7fcd0014cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:51.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.761+0000 7fccfffff640 1 --2- 172.21.15.5:0/875250232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd00162820 0x7fcd0014cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.761+0000 7fccfffff640 1 -- 172.21.15.5:0/875250232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd001502f0 con 0x7fcd000b3e80 2024-05-20T18:32:51.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.761+0000 7fccfffff640 1 --2- 172.21.15.5:0/875250232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd000b3e80 0x7fcd0014c700 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fccf000e8a0 tx=0x7fccf000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:51.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.761+0000 7fccfd7fa640 1 -- 172.21.15.5:0/875250232 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf0017070 con 0x7fcd000b3e80 2024-05-20T18:32:51.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.761+0000 7fccfd7fa640 1 -- 172.21.15.5:0/875250232 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccf000c940 con 0x7fcd000b3e80 2024-05-20T18:32:51.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.761+0000 7fcd06731640 1 -- 172.21.15.5:0/875250232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd000c8e30 con 0x7fcd000b3e80 2024-05-20T18:32:51.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.761+0000 7fccfd7fa640 1 -- 172.21.15.5:0/875250232 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf000cd30 con 0x7fcd000b3e80 2024-05-20T18:32:51.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.761+0000 7fcd06731640 1 -- 172.21.15.5:0/875250232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd000c93f0 con 0x7fcd000b3e80 2024-05-20T18:32:51.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.762+0000 7fcd06731640 1 -- 172.21.15.5:0/875250232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd00075b00 con 0x7fcd000b3e80 2024-05-20T18:32:51.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.763+0000 7fccfd7fa640 1 -- 172.21.15.5:0/875250232 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fccf001a070 con 0x7fcd000b3e80 2024-05-20T18:32:51.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.763+0000 7fccfd7fa640 1 --2- 172.21.15.5:0/875250232 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcce006be60 0x7fcce006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:51.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.764+0000 7fccfd7fa640 1 -- 172.21.15.5:0/875250232 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7fccf008b370 con 0x7fcd000b3e80 2024-05-20T18:32:51.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.764+0000 7fccf7fff640 1 --2- 172.21.15.5:0/875250232 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcce006be60 0x7fcce006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:51.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.764+0000 7fccf7fff640 1 --2- 172.21.15.5:0/875250232 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcce006be60 0x7fcce006e370 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fcce8007fd0 tx=0x7fcce8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:51.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.768+0000 7fccfd7fa640 1 -- 172.21.15.5:0/875250232 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fccf0059a30 con 0x7fcd000b3e80 2024-05-20T18:32:51.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.991+0000 7fcd06731640 1 -- 172.21.15.5:0/875250232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd0006df90 con 0x7fcd000b3e80 2024-05-20T18:32:51.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.991+0000 7fccfd7fa640 1 -- 172.21.15.5:0/875250232 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5176 (secure 0 0 0) 0x7fccf0059330 con 0x7fcd000b3e80 2024-05-20T18:32:51.992 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:32:51.992 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1890759603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1890759603},{"type":"v1","addr":"172.21.15.63:6834","nonce":1890759603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5825,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/700447403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":700447403},{"type":"v1","addr":"172.21.15.63:6835","nonce":700447403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3382943275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3382943275},{"type":"v1","addr":"172.21.15.5:6837","nonce":3382943275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:31:30.164179+0000","modified":"2024-05-20T18:32:27.855796+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6972":{"gid":6972,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":15,"addr":"172.21.15.5:6836/3866822752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":3866822752},{"type":"v1","addr":"172.21.15.5:6836","nonce":3866822752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6972,"qdb_cluster":[6972]},"id":5}]} 2024-05-20T18:32:51.992 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 52 2024-05-20T18:32:51.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.993+0000 7fcd06731640 1 -- 172.21.15.5:0/875250232 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcce006be60 msgr2=0x7fcce006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:51.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.993+0000 7fcd06731640 1 --2- 172.21.15.5:0/875250232 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcce006be60 0x7fcce006e370 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fcce8007fd0 tx=0x7fcce8009040 comp rx=0 tx=0).stop 2024-05-20T18:32:51.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.993+0000 7fcd06731640 1 -- 172.21.15.5:0/875250232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd000b3e80 msgr2=0x7fcd0014c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:51.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.993+0000 7fcd06731640 1 --2- 172.21.15.5:0/875250232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd000b3e80 0x7fcd0014c700 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fccf000e8a0 tx=0x7fccf000ed60 comp rx=0 tx=0).stop 2024-05-20T18:32:51.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.993+0000 7fcd06731640 1 -- 172.21.15.5:0/875250232 shutdown_connections 2024-05-20T18:32:51.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.993+0000 7fcd06731640 1 --2- 172.21.15.5:0/875250232 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcce006be60 0x7fcce006e370 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.993+0000 7fcd06731640 1 --2- 172.21.15.5:0/875250232 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd000c8700 0x7fcd00152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.993+0000 7fcd06731640 1 --2- 172.21.15.5:0/875250232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd00162820 0x7fcd0014cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.993+0000 7fcd06731640 1 --2- 172.21.15.5:0/875250232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd000b3e80 0x7fcd0014c700 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:51.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.993+0000 7fcd06731640 1 -- 172.21.15.5:0/875250232 >> 172.21.15.5:0/875250232 conn(0x7fcd000d4bd0 msgr2=0x7fcd000d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:51.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.993+0000 7fcd06731640 1 -- 172.21.15.5:0/875250232 shutdown_connections 2024-05-20T18:32:51.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:51.993+0000 7fcd06731640 1 -- 172.21.15.5:0/875250232 wait complete. 2024-05-20T18:32:52.006 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-05-20T18:32:52.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.122+0000 7f9ed5456640 1 Processor -- start 2024-05-20T18:32:52.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.122+0000 7f9ed5456640 1 -- start start 2024-05-20T18:32:52.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.124+0000 7f9ed5456640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9ed0162820 0x7f9ed00b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.124+0000 7f9ed5456640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ed006dae0 0x7f9ed00b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.124+0000 7f9ed5456640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ed00d0630 con 0x7f9ed00b3e80 2024-05-20T18:32:52.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.124+0000 7f9ed5456640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f9ed00d39c0 con 0x7f9ed0162820 2024-05-20T18:32:52.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.124+0000 7f9ed5456640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9ed00d3ca0 con 0x7f9ed006dae0 2024-05-20T18:32:52.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.124+0000 7f9ecf7fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ed006dae0 0x7f9ed00b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.124+0000 7f9eceffd640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9ed0162820 0x7f9ed00b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.124+0000 7f9eceffd640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9ed0162820 0x7f9ed00b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:49626/0 (socket says 172.21.15.5:49626) 2024-05-20T18:32:52.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.124+0000 7f9ecf7fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ed006dae0 0x7f9ed00b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:46980/0 (socket says 172.21.15.5:46980) 2024-05-20T18:32:52.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.124+0000 7f9eceffd640 1 -- 172.21.15.5:0/3117145039 learned_addr learned my addr 172.21.15.5:0/3117145039 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:52.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.125+0000 7f9eceffd640 1 -- 172.21.15.5:0/3117145039 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ed006dae0 msgr2=0x7f9ed00b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:52.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.125+0000 7f9eceffd640 1 --2- 172.21.15.5:0/3117145039 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ed006dae0 0x7f9ed00b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.125+0000 7f9eceffd640 1 -- 172.21.15.5:0/3117145039 >> v1:172.21.15.63:6789/0 conn(0x7f9ed00b3e80 legacy=0x7f9ed006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:52.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.125+0000 7f9eceffd640 1 -- 172.21.15.5:0/3117145039 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed00d3f80 con 0x7f9ed0162820 2024-05-20T18:32:52.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.125+0000 7f9ecf7fe640 1 --2- 172.21.15.5:0/3117145039 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ed006dae0 0x7f9ed00b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:32:52.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.125+0000 7f9eceffd640 1 --2- 172.21.15.5:0/3117145039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ed0162820 0x7f9ed00b3940 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8009e50 tx=0x7f9eb802a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=560cdcdc74be0c1a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:52.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.126+0000 7f9ecdffb640 1 -- 172.21.15.5:0/3117145039 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8036070 con 0x7f9ed0162820 2024-05-20T18:32:52.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.126+0000 7f9ecdffb640 1 -- 172.21.15.5:0/3117145039 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eb8031600 con 0x7f9ed0162820 2024-05-20T18:32:52.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.126+0000 7f9ecdffb640 1 -- 172.21.15.5:0/3117145039 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb80319d0 con 0x7f9ed0162820 2024-05-20T18:32:52.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.126+0000 7f9ed5456640 1 -- 172.21.15.5:0/3117145039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ed0162820 msgr2=0x7f9ed00b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:52.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.126+0000 7f9ed5456640 1 --2- 172.21.15.5:0/3117145039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ed0162820 0x7f9ed00b3940 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8009e50 tx=0x7f9eb802a710 comp rx=0 tx=0).stop 2024-05-20T18:32:52.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.126+0000 7f9ed5456640 1 -- 172.21.15.5:0/3117145039 shutdown_connections 2024-05-20T18:32:52.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.126+0000 7f9ed5456640 1 --2- 172.21.15.5:0/3117145039 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ed006dae0 0x7f9ed00b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.126+0000 7f9ed5456640 1 --2- 172.21.15.5:0/3117145039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ed0162820 0x7f9ed00b3940 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.126+0000 7f9ed5456640 1 -- 172.21.15.5:0/3117145039 >> 172.21.15.5:0/3117145039 conn(0x7f9ed00d4bd0 msgr2=0x7f9ed00d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:52.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.126+0000 7f9ed5456640 1 -- 172.21.15.5:0/3117145039 shutdown_connections 2024-05-20T18:32:52.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.126+0000 7f9ed5456640 1 -- 172.21.15.5:0/3117145039 wait complete. 2024-05-20T18:32:52.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.128+0000 7f9ed5456640 1 Processor -- start 2024-05-20T18:32:52.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.128+0000 7f9ed5456640 1 -- start start 2024-05-20T18:32:52.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.130+0000 7f9ed5456640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ed00b3e80 0x7f9ed01f91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.130+0000 7f9ed5456640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ed0162820 0x7f9ed00ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.130+0000 7f9ed5456640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ed00cd4a0 0x7f9ed00cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.130+0000 7f9ed5456640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed00d4780 con 0x7f9ed00cd4a0 2024-05-20T18:32:52.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.130+0000 7f9ed5456640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed00d0300 con 0x7f9ed0162820 2024-05-20T18:32:52.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.130+0000 7f9ed5456640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9ed00d4900 con 0x7f9ed00b3e80 2024-05-20T18:32:52.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.130+0000 7f9ecf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ed00cd4a0 0x7f9ed00cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.130+0000 7f9ecf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ed00cd4a0 0x7f9ed00cd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37440/0 (socket says 172.21.15.5:37440) 2024-05-20T18:32:52.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.130+0000 7f9ecf7fe640 1 -- 172.21.15.5:0/2253934261 learned_addr learned my addr 172.21.15.5:0/2253934261 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:52.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.130+0000 7f9eceffd640 1 --2- 172.21.15.5:0/2253934261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ed00b3e80 0x7f9ed01f91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.130+0000 7f9ece7fc640 1 --2- 172.21.15.5:0/2253934261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ed0162820 0x7f9ed00ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.130+0000 7f9ecf7fe640 1 -- 172.21.15.5:0/2253934261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ed00b3e80 msgr2=0x7f9ed01f91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:52.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.130+0000 7f9ecf7fe640 1 --2- 172.21.15.5:0/2253934261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ed00b3e80 0x7f9ed01f91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.130+0000 7f9ecf7fe640 1 -- 172.21.15.5:0/2253934261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ed0162820 msgr2=0x7f9ed00ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:52.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.130+0000 7f9ecf7fe640 1 --2- 172.21.15.5:0/2253934261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ed0162820 0x7f9ed00ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.130+0000 7f9ecf7fe640 1 -- 172.21.15.5:0/2253934261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed01f9710 con 0x7f9ed00cd4a0 2024-05-20T18:32:52.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.131+0000 7f9ecf7fe640 1 --2- 172.21.15.5:0/2253934261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ed00cd4a0 0x7f9ed00cd8b0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f9ec800e890 tx=0x7f9ec800ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:52.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.131+0000 7f9ec77fe640 1 -- 172.21.15.5:0/2253934261 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8017070 con 0x7f9ed00cd4a0 2024-05-20T18:32:52.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.131+0000 7f9ec77fe640 1 -- 172.21.15.5:0/2253934261 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ec800c940 con 0x7f9ed00cd4a0 2024-05-20T18:32:52.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.131+0000 7f9ed5456640 1 -- 172.21.15.5:0/2253934261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ed00ca7b0 con 0x7f9ed00cd4a0 2024-05-20T18:32:52.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.131+0000 7f9ec77fe640 1 -- 172.21.15.5:0/2253934261 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec800cd10 con 0x7f9ed00cd4a0 2024-05-20T18:32:52.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.131+0000 7f9ed5456640 1 -- 172.21.15.5:0/2253934261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ed00cc290 con 0x7f9ed00cd4a0 2024-05-20T18:32:52.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.133+0000 7f9ec77fe640 1 -- 172.21.15.5:0/2253934261 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9ec801a070 con 0x7f9ed00cd4a0 2024-05-20T18:32:52.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.133+0000 7f9ec77fe640 1 --2- 172.21.15.5:0/2253934261 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9eb006be30 0x7f9eb006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.133+0000 7f9ec77fe640 1 -- 172.21.15.5:0/2253934261 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f9ec808aea0 con 0x7f9ed00cd4a0 2024-05-20T18:32:52.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.133+0000 7f9eceffd640 1 --2- 172.21.15.5:0/2253934261 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9eb006be30 0x7f9eb006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.134+0000 7f9eceffd640 1 --2- 172.21.15.5:0/2253934261 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9eb006be30 0x7f9eb006e340 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8033000 tx=0x7f9eb8009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:52.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.134+0000 7f9edc3d1740 1 Processor -- start 2024-05-20T18:32:52.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.135+0000 7f9edc3d1740 1 -- start start 2024-05-20T18:32:52.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.136+0000 7f9edc3d1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bc05ed350 0x564bc09b8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.136+0000 7f9edc3d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bc05c4490 0x564bc09d33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.136+0000 7f9edc3d1740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564bc05cc670 0x564bc09d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.136+0000 7f9edc3d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564bc08f1a30 con 0x564bc05c4490 2024-05-20T18:32:52.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.136+0000 7f9edc3d1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564bc088f590 con 0x564bc05ed350 2024-05-20T18:32:52.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.136+0000 7f9edc3d1740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564bc08ed9f0 con 0x564bc05cc670 2024-05-20T18:32:52.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.136+0000 7f9ece7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bc05c4490 0x564bc09d33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.136+0000 7f9ece7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bc05c4490 0x564bc09d33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37456/0 (socket says 172.21.15.5:37456) 2024-05-20T18:32:52.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.136+0000 7f9ece7fc640 1 -- 172.21.15.5:0/1665818401 learned_addr learned my addr 172.21.15.5:0/1665818401 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:52.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.136+0000 7f9ecf7fe640 1 --2- 172.21.15.5:0/1665818401 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564bc05cc670 0x564bc09d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.136+0000 7f9eceffd640 1 --2- 172.21.15.5:0/1665818401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bc05ed350 0x564bc09b8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.136+0000 7f9ece7fc640 1 -- 172.21.15.5:0/1665818401 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564bc05cc670 msgr2=0x564bc09d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:52.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.136+0000 7f9ece7fc640 1 --2- 172.21.15.5:0/1665818401 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564bc05cc670 0x564bc09d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.136+0000 7f9ece7fc640 1 -- 172.21.15.5:0/1665818401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bc05ed350 msgr2=0x564bc09b8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:52.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.136+0000 7f9ece7fc640 1 --2- 172.21.15.5:0/1665818401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bc05ed350 0x564bc09b8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.136+0000 7f9ece7fc640 1 -- 172.21.15.5:0/1665818401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bc06d17c0 con 0x564bc05c4490 2024-05-20T18:32:52.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.136+0000 7f9ece7fc640 1 --2- 172.21.15.5:0/1665818401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bc05c4490 0x564bc09d33a0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0001780 tx=0x7f9ec0007e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415f878747f88bad server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:52.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.137+0000 7f9ec4ff9640 1 -- 172.21.15.5:0/1665818401 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec00052e0 con 0x564bc05c4490 2024-05-20T18:32:52.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.137+0000 7f9ec4ff9640 1 -- 172.21.15.5:0/1665818401 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ec0018070 con 0x564bc05c4490 2024-05-20T18:32:52.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.137+0000 7f9ec4ff9640 1 -- 172.21.15.5:0/1665818401 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec000f9b0 con 0x564bc05c4490 2024-05-20T18:32:52.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.137+0000 7f9edc3d1740 1 -- 172.21.15.5:0/1665818401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bc05c4490 msgr2=0x564bc09d33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:52.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.137+0000 7f9edc3d1740 1 --2- 172.21.15.5:0/1665818401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bc05c4490 0x564bc09d33a0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0001780 tx=0x7f9ec0007e50 comp rx=0 tx=0).stop 2024-05-20T18:32:52.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.137+0000 7f9edc3d1740 1 -- 172.21.15.5:0/1665818401 shutdown_connections 2024-05-20T18:32:52.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.137+0000 7f9edc3d1740 1 --2- 172.21.15.5:0/1665818401 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564bc05cc670 0x564bc09d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.137+0000 7f9edc3d1740 1 --2- 172.21.15.5:0/1665818401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bc05c4490 0x564bc09d33a0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.137+0000 7f9edc3d1740 1 --2- 172.21.15.5:0/1665818401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bc05ed350 0x564bc09b8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.137+0000 7f9edc3d1740 1 -- 172.21.15.5:0/1665818401 >> 172.21.15.5:0/1665818401 conn(0x564bc05e14a0 msgr2=0x564bc098dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:52.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.138+0000 7f9edc3d1740 1 -- 172.21.15.5:0/1665818401 shutdown_connections 2024-05-20T18:32:52.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.138+0000 7f9edc3d1740 1 -- 172.21.15.5:0/1665818401 wait complete. 2024-05-20T18:32:52.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.138+0000 7f9edc3d1740 1 -- start start 2024-05-20T18:32:52.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.139+0000 7f9edc3d1740 1 Processor -- start 2024-05-20T18:32:52.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.140+0000 7f9edc3d1740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564bc05c4490 0x564bc09aa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.140+0000 7f9edc3d1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bc05cc670 0x564bc09aac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.140+0000 7f9edc3d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bc05ed350 0x564bc09bb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.140+0000 7f9edc3d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564bc088d060 con 0x564bc05ed350 2024-05-20T18:32:52.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.140+0000 7f9edc3d1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564bc09a9a30 con 0x564bc05cc670 2024-05-20T18:32:52.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.140+0000 7f9edc3d1740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564bc09ab750 con 0x564bc05c4490 2024-05-20T18:32:52.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.140+0000 7f9edc3d1740 10 client.0 _ll_register_callbacks cb 0x564bc08b1dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:32:52.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.140+0000 7f9ecf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bc05ed350 0x564bc09bb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.140+0000 7f9ecf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bc05ed350 0x564bc09bb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37458/0 (socket says 172.21.15.5:37458) 2024-05-20T18:32:52.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.140+0000 7f9ecf7fe640 1 -- 172.21.15.5:0/2351085329 learned_addr learned my addr 172.21.15.5:0/2351085329 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:52.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.140+0000 7f9eceffd640 1 --2- 172.21.15.5:0/2351085329 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564bc05c4490 0x564bc09aa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.140+0000 7f9ece7fc640 1 --2- 172.21.15.5:0/2351085329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bc05cc670 0x564bc09aac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.140+0000 7f9ecf7fe640 1 -- 172.21.15.5:0/2351085329 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564bc05c4490 msgr2=0x564bc09aa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:52.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.140+0000 7f9ecf7fe640 1 --2- 172.21.15.5:0/2351085329 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564bc05c4490 0x564bc09aa640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.140+0000 7f9ecf7fe640 1 -- 172.21.15.5:0/2351085329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bc05cc670 msgr2=0x564bc09aac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:52.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.140+0000 7f9ecf7fe640 1 --2- 172.21.15.5:0/2351085329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bc05cc670 0x564bc09aac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.140+0000 7f9ecf7fe640 1 -- 172.21.15.5:0/2351085329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bc0731f50 con 0x564bc05ed350 2024-05-20T18:32:52.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.140+0000 7f9ecf7fe640 1 --2- 172.21.15.5:0/2351085329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bc05ed350 0x564bc09bb730 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f9ec800e860 tx=0x7f9ec8058960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc904ef59a55f4e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:52.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.140+0000 7f9eadffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:32:52.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.141+0000 7f9eadffb640 1 -- 172.21.15.5:0/2351085329 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8021040 con 0x564bc05ed350 2024-05-20T18:32:52.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.141+0000 7f9eadffb640 1 -- 172.21.15.5:0/2351085329 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f9ec8076d90 con 0x564bc05ed350 2024-05-20T18:32:52.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.141+0000 7f9edc3d1740 1 -- 172.21.15.5:0/2351085329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ec0018070 con 0x564bc05ed350 2024-05-20T18:32:52.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.141+0000 7f9eadffb640 1 -- 172.21.15.5:0/2351085329 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f9ec8061ad0 con 0x564bc05ed350 2024-05-20T18:32:52.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.141+0000 7f9edc3d1740 10 client.7218 fetch_fsmap learned FSMap version 52 2024-05-20T18:32:52.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.141+0000 7f9edc3d1740 1 -- 172.21.15.5:0/2351085329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x564bc05c50b0 con 0x564bc05ed350 2024-05-20T18:32:52.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.141+0000 7f9eadffb640 1 -- 172.21.15.5:0/2351085329 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ec0018070 con 0x564bc05ed350 2024-05-20T18:32:52.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.141+0000 7f9eadffb640 1 -- 172.21.15.5:0/2351085329 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f9ec8076060 con 0x564bc05ed350 2024-05-20T18:32:52.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.141+0000 7f9edc3d1740 10 client.7218 fetch_fsmap finished waiting for FSMap version 52 2024-05-20T18:32:52.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.141+0000 7f9edc3d1740 10 client.7218 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-20T18:32:52.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.141+0000 7f9edc3d1740 20 client.7218 populate_metadata read hostname 'smithi005' 2024-05-20T18:32:52.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.141+0000 7f9edc3d1740 1 --2- 172.21.15.5:0/2351085329 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x564bc05ac130 0x564bc09cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.141+0000 7f9edc3d1740 4 client.7218 mds_command: new command op to 6972 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:32:52.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.141+0000 7f9edc3d1740 1 -- 172.21.15.5:0/2351085329 --> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bc08ff160 con 0x564bc05ac130 2024-05-20T18:32:52.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.141+0000 7f9eceffd640 1 --2- 172.21.15.5:0/2351085329 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x564bc05ac130 0x564bc09cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.142+0000 7f9eadffb640 10 client.7218 ms_handle_connect on v2:172.21.15.5:6835/3866822752 2024-05-20T18:32:52.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.142+0000 7f9eceffd640 1 --2- 172.21.15.5:0/2351085329 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x564bc05ac130 0x564bc09cdfb0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c43072cd00c36ffd server_cookie=c4e9ac047e949c6 in_seq=0 out_seq=0 2024-05-20T18:32:52.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.142+0000 7f9eadffb640 1 -- 172.21.15.5:0/2351085329 <== mds.0 v2:172.21.15.5:6835/3866822752 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564bc08ff160 con 0x564bc05ac130 2024-05-20T18:32:52.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.142+0000 7f9eadffb640 10 client.7218 handle_command_reply: tid=0 2024-05-20T18:32:52.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.142+0000 7f9edc3d1740 1 client.7218 shutdown 2024-05-20T18:32:52.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.142+0000 7f9edc3d1740 1 -- 172.21.15.5:0/2351085329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bc05ed350 msgr2=0x564bc09bb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:52.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.142+0000 7f9edc3d1740 1 --2- 172.21.15.5:0/2351085329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bc05ed350 0x564bc09bb730 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f9ec800e860 tx=0x7f9ec8058960 comp rx=0 tx=0).stop 2024-05-20T18:32:52.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.142+0000 7f9edc3d1740 1 -- 172.21.15.5:0/2351085329 shutdown_connections 2024-05-20T18:32:52.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.142+0000 7f9edc3d1740 1 --2- 172.21.15.5:0/2351085329 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x564bc05ac130 0x564bc09cdfb0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.142+0000 7f9edc3d1740 1 --2- 172.21.15.5:0/2351085329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bc05ed350 0x564bc09bb730 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.142+0000 7f9edc3d1740 1 --2- 172.21.15.5:0/2351085329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bc05cc670 0x564bc09aac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.142+0000 7f9eadffb640 0 client.7218 ms_handle_reset on v2:172.21.15.5:6835/3866822752 2024-05-20T18:32:52.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.142+0000 7f9edc3d1740 1 --2- 172.21.15.5:0/2351085329 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564bc05c4490 0x564bc09aa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.142+0000 7f9edc3d1740 1 -- 172.21.15.5:0/2351085329 >> 172.21.15.5:0/2351085329 conn(0x564bc05e14a0 msgr2=0x564bc09b5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:52.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.143+0000 7f9edc3d1740 1 -- 172.21.15.5:0/2351085329 shutdown_connections 2024-05-20T18:32:52.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.143+0000 7f9edc3d1740 1 -- 172.21.15.5:0/2351085329 wait complete. 2024-05-20T18:32:52.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.143+0000 7f9edc3d1740 20 client.7218 trim_cache size 0 max 16384 2024-05-20T18:32:52.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.157+0000 7f9edc3d1740 1 Processor -- start 2024-05-20T18:32:52.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.157+0000 7f9edc3d1740 1 -- start start 2024-05-20T18:32:52.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.158+0000 7f9edc3d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bc05c4490 0x564bc09b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.158+0000 7f9edc3d1740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564bc05cc670 0x564bc09aa450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.158+0000 7f9edc3d1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bc05ed350 0x564bc09aa990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.158+0000 7f9edc3d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed0201510 con 0x564bc05c4490 2024-05-20T18:32:52.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.158+0000 7f9edc3d1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed00c9990 con 0x564bc05ed350 2024-05-20T18:32:52.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.158+0000 7f9edc3d1740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9ed0200ed0 con 0x564bc05cc670 2024-05-20T18:32:52.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.158+0000 7f9eceffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bc05c4490 0x564bc09b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.158+0000 7f9eceffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bc05c4490 0x564bc09b4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37466/0 (socket says 172.21.15.5:37466) 2024-05-20T18:32:52.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.158+0000 7f9eceffd640 1 -- 172.21.15.5:0/462572616 learned_addr learned my addr 172.21.15.5:0/462572616 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:52.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.158+0000 7f9ece7fc640 1 --2- 172.21.15.5:0/462572616 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564bc05cc670 0x564bc09aa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.158+0000 7f9ecf7fe640 1 --2- 172.21.15.5:0/462572616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bc05ed350 0x564bc09aa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.158+0000 7f9eceffd640 1 -- 172.21.15.5:0/462572616 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564bc05cc670 msgr2=0x564bc09aa450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:52.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.158+0000 7f9eceffd640 1 --2- 172.21.15.5:0/462572616 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564bc05cc670 0x564bc09aa450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.158+0000 7f9eceffd640 1 -- 172.21.15.5:0/462572616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bc05ed350 msgr2=0x564bc09aa990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:52.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.158+0000 7f9eceffd640 1 --2- 172.21.15.5:0/462572616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bc05ed350 0x564bc09aa990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.158+0000 7f9eceffd640 1 -- 172.21.15.5:0/462572616 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bc0984300 con 0x564bc05c4490 2024-05-20T18:32:52.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.158+0000 7f9ece7fc640 1 --2- 172.21.15.5:0/462572616 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564bc05cc670 0x564bc09aa450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:32:52.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.158+0000 7f9ecf7fe640 1 --2- 172.21.15.5:0/462572616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bc05ed350 0x564bc09aa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:32:52.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.158+0000 7f9eceffd640 1 --2- 172.21.15.5:0/462572616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bc05c4490 0x564bc09b4d60 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8009f40 tx=0x7f9eb802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db81a2c1c9c5c3ba server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:52.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.159+0000 7f9eaffff640 1 -- 172.21.15.5:0/462572616 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8036040 con 0x564bc05c4490 2024-05-20T18:32:52.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.159+0000 7f9eaffff640 1 -- 172.21.15.5:0/462572616 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eb8037040 con 0x564bc05c4490 2024-05-20T18:32:52.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.159+0000 7f9eaffff640 1 -- 172.21.15.5:0/462572616 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8048da0 con 0x564bc05c4490 2024-05-20T18:32:52.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.159+0000 7f9edc3d1740 1 -- 172.21.15.5:0/462572616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bc05c4490 msgr2=0x564bc09b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:52.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.159+0000 7f9edc3d1740 1 --2- 172.21.15.5:0/462572616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bc05c4490 0x564bc09b4d60 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8009f40 tx=0x7f9eb802e000 comp rx=0 tx=0).stop 2024-05-20T18:32:52.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.159+0000 7f9edc3d1740 1 -- 172.21.15.5:0/462572616 shutdown_connections 2024-05-20T18:32:52.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.159+0000 7f9edc3d1740 1 --2- 172.21.15.5:0/462572616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bc05ed350 0x564bc09aa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.159+0000 7f9edc3d1740 1 --2- 172.21.15.5:0/462572616 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564bc05cc670 0x564bc09aa450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.159+0000 7f9edc3d1740 1 --2- 172.21.15.5:0/462572616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bc05c4490 0x564bc09b4d60 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.159+0000 7f9edc3d1740 1 -- 172.21.15.5:0/462572616 >> 172.21.15.5:0/462572616 conn(0x564bc05e14a0 msgr2=0x564bc09bb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:52.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.159+0000 7f9edc3d1740 1 -- 172.21.15.5:0/462572616 shutdown_connections 2024-05-20T18:32:52.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.159+0000 7f9edc3d1740 1 -- 172.21.15.5:0/462572616 wait complete. 2024-05-20T18:32:52.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.160+0000 7f9edc3d1740 1 -- start start 2024-05-20T18:32:52.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.160+0000 7f9edc3d1740 1 Processor -- start 2024-05-20T18:32:52.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.161+0000 7f9edc3d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bc05c4490 0x564bc09b5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.162+0000 7f9edc3d1740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564bc05cc670 0x564bc09ea430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.162+0000 7f9edc3d1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bc05ed350 0x564bc09e4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.162+0000 7f9edc3d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564bc09a8aa0 con 0x564bc05c4490 2024-05-20T18:32:52.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.162+0000 7f9edc3d1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564bc09bbd20 con 0x564bc05ed350 2024-05-20T18:32:52.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.162+0000 7f9edc3d1740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564bc09a9030 con 0x564bc05cc670 2024-05-20T18:32:52.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.162+0000 7f9edc3d1740 10 client.0 _ll_register_callbacks cb 0x7f9eb8034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:32:52.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.162+0000 7f9eceffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bc05c4490 0x564bc09b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.162+0000 7f9eceffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bc05c4490 0x564bc09b5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37482/0 (socket says 172.21.15.5:37482) 2024-05-20T18:32:52.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.162+0000 7f9eceffd640 1 -- 172.21.15.5:0/466432493 learned_addr learned my addr 172.21.15.5:0/466432493 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:52.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.162+0000 7f9ece7fc640 1 --2- 172.21.15.5:0/466432493 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564bc05cc670 0x564bc09ea430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.162+0000 7f9ecf7fe640 1 --2- 172.21.15.5:0/466432493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bc05ed350 0x564bc09e4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.162+0000 7f9eceffd640 1 -- 172.21.15.5:0/466432493 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564bc05cc670 msgr2=0x564bc09ea430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:52.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.162+0000 7f9eceffd640 1 --2- 172.21.15.5:0/466432493 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564bc05cc670 0x564bc09ea430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.162+0000 7f9eceffd640 1 -- 172.21.15.5:0/466432493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bc05ed350 msgr2=0x564bc09e4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:52.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.162+0000 7f9eceffd640 1 --2- 172.21.15.5:0/466432493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bc05ed350 0x564bc09e4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.162+0000 7f9eceffd640 1 -- 172.21.15.5:0/466432493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bc09a9f10 con 0x564bc05c4490 2024-05-20T18:32:52.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.162+0000 7f9ece7fc640 1 --2- 172.21.15.5:0/466432493 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564bc05cc670 0x564bc09ea430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:32:52.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.162+0000 7f9ecf7fe640 1 --2- 172.21.15.5:0/466432493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bc05ed350 0x564bc09e4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:32:52.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.162+0000 7f9eceffd640 1 --2- 172.21.15.5:0/466432493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bc05c4490 0x564bc09b5300 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8031730 tx=0x7f9eb8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69ebe526549b1cd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:52.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.162+0000 7f9eadffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:32:52.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.162+0000 7f9eadffb640 1 -- 172.21.15.5:0/466432493 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8036040 con 0x564bc05c4490 2024-05-20T18:32:52.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.162+0000 7f9eadffb640 1 -- 172.21.15.5:0/466432493 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f9eb8003c10 con 0x564bc05c4490 2024-05-20T18:32:52.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.162+0000 7f9edc3d1740 1 -- 172.21.15.5:0/466432493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9eb8037040 con 0x564bc05c4490 2024-05-20T18:32:52.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.163+0000 7f9eadffb640 1 -- 172.21.15.5:0/466432493 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f9eb8003e10 con 0x564bc05c4490 2024-05-20T18:32:52.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.163+0000 7f9edc3d1740 10 client.7224 fetch_fsmap learned FSMap version 52 2024-05-20T18:32:52.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.163+0000 7f9edc3d1740 1 -- 172.21.15.5:0/466432493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x564bc09aa7e0 con 0x564bc05c4490 2024-05-20T18:32:52.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.163+0000 7f9eadffb640 1 -- 172.21.15.5:0/466432493 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eb8037040 con 0x564bc05c4490 2024-05-20T18:32:52.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.163+0000 7f9eadffb640 1 -- 172.21.15.5:0/466432493 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f9eb8048980 con 0x564bc05c4490 2024-05-20T18:32:52.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.163+0000 7f9edc3d1740 10 client.7224 fetch_fsmap finished waiting for FSMap version 52 2024-05-20T18:32:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.163+0000 7f9edc3d1740 10 client.7224 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-20T18:32:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.163+0000 7f9edc3d1740 20 client.7224 populate_metadata read hostname 'smithi005' 2024-05-20T18:32:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.163+0000 7f9edc3d1740 1 --2- 172.21.15.5:0/466432493 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x564bc09e52f0 0x564bc098da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.163+0000 7f9edc3d1740 4 client.7224 mds_command: new command op to 6972 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-20T18:32:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.163+0000 7f9edc3d1740 1 -- 172.21.15.5:0/466432493 --> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x564bc08f94b0 con 0x564bc09e52f0 2024-05-20T18:32:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.163+0000 7f9ece7fc640 1 --2- 172.21.15.5:0/466432493 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x564bc09e52f0 0x564bc098da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.163+0000 7f9eadffb640 10 client.7224 ms_handle_connect on v2:172.21.15.5:6835/3866822752 2024-05-20T18:32:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.163+0000 7f9ece7fc640 1 --2- 172.21.15.5:0/466432493 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x564bc09e52f0 0x564bc098da10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d5e63a0aa4543da server_cookie=1782a03ba3c15f1d in_seq=0 out_seq=0 2024-05-20T18:32:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.164+0000 7f9eadffb640 1 -- 172.21.15.5:0/466432493 <== mds.0 v2:172.21.15.5:6835/3866822752 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564bc08f94b0 con 0x564bc09e52f0 2024-05-20T18:32:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.164+0000 7f9eadffb640 10 client.7224 handle_command_reply: tid=0 2024-05-20T18:32:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.164+0000 7f9edc3d1740 1 client.7224 shutdown 2024-05-20T18:32:52.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.164+0000 7f9edc3d1740 1 -- 172.21.15.5:0/466432493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bc05c4490 msgr2=0x564bc09b5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:52.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.164+0000 7f9edc3d1740 1 --2- 172.21.15.5:0/466432493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bc05c4490 0x564bc09b5300 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8031730 tx=0x7f9eb8004ab0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.164+0000 7f9edc3d1740 1 -- 172.21.15.5:0/466432493 shutdown_connections 2024-05-20T18:32:52.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.164+0000 7f9edc3d1740 1 --2- 172.21.15.5:0/466432493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564bc05ed350 0x564bc09e4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.164+0000 7f9edc3d1740 1 --2- 172.21.15.5:0/466432493 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564bc05cc670 0x564bc09ea430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.164+0000 7f9edc3d1740 1 --2- 172.21.15.5:0/466432493 >> [v2:172.21.15.5:6835/3866822752,v1:172.21.15.5:6836/3866822752] conn(0x564bc09e52f0 0x564bc098da10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.164+0000 7f9edc3d1740 1 --2- 172.21.15.5:0/466432493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564bc05c4490 0x564bc09b5300 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.164+0000 7f9eadffb640 0 client.7224 ms_handle_reset on v2:172.21.15.5:6835/3866822752 2024-05-20T18:32:52.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.164+0000 7f9edc3d1740 1 -- 172.21.15.5:0/466432493 >> 172.21.15.5:0/466432493 conn(0x564bc05e14a0 msgr2=0x564bc09b81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:52.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.164+0000 7f9edc3d1740 1 -- 172.21.15.5:0/466432493 shutdown_connections 2024-05-20T18:32:52.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.164+0000 7f9edc3d1740 1 -- 172.21.15.5:0/466432493 wait complete. 2024-05-20T18:32:52.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.164+0000 7f9edc3d1740 20 client.7224 trim_cache size 0 max 16384 2024-05-20T18:32:52.172 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:32:52.172 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-20T18:32:52.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.164+0000 7f9ed5456640 1 -- 172.21.15.5:0/2253934261 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9eb006be30 msgr2=0x7f9eb006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:52.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.164+0000 7f9ed5456640 1 --2- 172.21.15.5:0/2253934261 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9eb006be30 0x7f9eb006e340 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8033000 tx=0x7f9eb8009000 comp rx=0 tx=0).stop 2024-05-20T18:32:52.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.164+0000 7f9ed5456640 1 -- 172.21.15.5:0/2253934261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ed00cd4a0 msgr2=0x7f9ed00cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:52.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.164+0000 7f9ed5456640 1 --2- 172.21.15.5:0/2253934261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ed00cd4a0 0x7f9ed00cd8b0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f9ec800e890 tx=0x7f9ec800ed50 comp rx=0 tx=0).stop 2024-05-20T18:32:52.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.165+0000 7f9ed5456640 1 -- 172.21.15.5:0/2253934261 shutdown_connections 2024-05-20T18:32:52.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.165+0000 7f9ed5456640 1 --2- 172.21.15.5:0/2253934261 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9eb006be30 0x7f9eb006e340 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.165+0000 7f9ed5456640 1 --2- 172.21.15.5:0/2253934261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ed00cd4a0 0x7f9ed00cd8b0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.165+0000 7f9ed5456640 1 --2- 172.21.15.5:0/2253934261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ed0162820 0x7f9ed00ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.165+0000 7f9ed5456640 1 --2- 172.21.15.5:0/2253934261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ed00b3e80 0x7f9ed01f91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.165+0000 7f9ed5456640 1 -- 172.21.15.5:0/2253934261 >> 172.21.15.5:0/2253934261 conn(0x7f9ed00d4bd0 msgr2=0x7f9ed00b9ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:52.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.165+0000 7f9ed5456640 1 -- 172.21.15.5:0/2253934261 shutdown_connections 2024-05-20T18:32:52.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.165+0000 7f9ed5456640 1 -- 172.21.15.5:0/2253934261 wait complete. 2024-05-20T18:32:52.176 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-20T18:32:52.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.290+0000 7f06e527a640 1 Processor -- start 2024-05-20T18:32:52.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.290+0000 7f06e527a640 1 -- start start 2024-05-20T18:32:52.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.292+0000 7f06e527a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06e00b4070 0x7f06e00ba5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.292+0000 7f06e527a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f06e006dae0 0x7f06e00b8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.292+0000 7f06e527a640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06e015fa90 con 0x7f06e00d0560 2024-05-20T18:32:52.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.292+0000 7f06e527a640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f06e00b8490 con 0x7f06e006dae0 2024-05-20T18:32:52.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.292+0000 7f06e527a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f06e00d3f00 con 0x7f06e00b4070 2024-05-20T18:32:52.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.292+0000 7f06df7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f06e006dae0 0x7f06e00b8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.292+0000 7f06df7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f06e006dae0 0x7f06e00b8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37488/0 (socket says 172.21.15.5:37488) 2024-05-20T18:32:52.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.292+0000 7f06df7fe640 1 -- 172.21.15.5:0/1011960823 learned_addr learned my addr 172.21.15.5:0/1011960823 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:52.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.292+0000 7f06deffd640 1 --2- 172.21.15.5:0/1011960823 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06e00b4070 0x7f06e00ba5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.292+0000 7f06df7fe640 1 -- 172.21.15.5:0/1011960823 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06e00b4070 msgr2=0x7f06e00ba5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:52.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.292+0000 7f06df7fe640 1 --2- 172.21.15.5:0/1011960823 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06e00b4070 0x7f06e00ba5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.292+0000 7f06df7fe640 1 -- 172.21.15.5:0/1011960823 >> v1:172.21.15.63:6789/0 conn(0x7f06e00d0560 legacy=0x7f06e006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:52.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.292+0000 7f06df7fe640 1 -- 172.21.15.5:0/1011960823 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e00d41e0 con 0x7f06e006dae0 2024-05-20T18:32:52.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.292+0000 7f06deffd640 1 --2- 172.21.15.5:0/1011960823 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06e00b4070 0x7f06e00ba5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:32:52.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.293+0000 7f06df7fe640 1 --2- 172.21.15.5:0/1011960823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06e006dae0 0x7f06e00b8920 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009e50 tx=0x7f06d802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0da29f622af9c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:52.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.293+0000 7f06de7fc640 1 -- 172.21.15.5:0/1011960823 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8036070 con 0x7f06e006dae0 2024-05-20T18:32:52.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.293+0000 7f06de7fc640 1 -- 172.21.15.5:0/1011960823 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06d8031600 con 0x7f06e006dae0 2024-05-20T18:32:52.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.293+0000 7f06de7fc640 1 -- 172.21.15.5:0/1011960823 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d80319d0 con 0x7f06e006dae0 2024-05-20T18:32:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.294+0000 7f06e527a640 1 -- 172.21.15.5:0/1011960823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06e006dae0 msgr2=0x7f06e00b8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.294+0000 7f06e527a640 1 --2- 172.21.15.5:0/1011960823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06e006dae0 0x7f06e00b8920 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009e50 tx=0x7f06d802a6c0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.294+0000 7f06e527a640 1 -- 172.21.15.5:0/1011960823 shutdown_connections 2024-05-20T18:32:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.294+0000 7f06e527a640 1 --2- 172.21.15.5:0/1011960823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06e006dae0 0x7f06e00b8920 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.294+0000 7f06e527a640 1 --2- 172.21.15.5:0/1011960823 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06e00b4070 0x7f06e00ba5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.294+0000 7f06e527a640 1 -- 172.21.15.5:0/1011960823 >> 172.21.15.5:0/1011960823 conn(0x7f06e00d4c10 msgr2=0x7f06e00d5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.294+0000 7f06e527a640 1 -- 172.21.15.5:0/1011960823 shutdown_connections 2024-05-20T18:32:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.294+0000 7f06e527a640 1 -- 172.21.15.5:0/1011960823 wait complete. 2024-05-20T18:32:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.296+0000 7f06e527a640 1 Processor -- start 2024-05-20T18:32:52.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.296+0000 7f06e527a640 1 -- start start 2024-05-20T18:32:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.297+0000 7f06e527a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06e00b4070 0x7f06e0161f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.297+0000 7f06e527a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06e00d0560 0x7f06e0160470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.297+0000 7f06e527a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06e00cc820 0x7f06e01609b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.297+0000 7f06e527a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f06e00bab00 con 0x7f06e00b4070 2024-05-20T18:32:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.297+0000 7f06e527a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f06e00d47a0 con 0x7f06e00d0560 2024-05-20T18:32:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.297+0000 7f06e527a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f06e00d3b20 con 0x7f06e00cc820 2024-05-20T18:32:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.297+0000 7f06deffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06e00b4070 0x7f06e0161f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.297+0000 7f06deffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06e00b4070 0x7f06e0161f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37492/0 (socket says 172.21.15.5:37492) 2024-05-20T18:32:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.297+0000 7f06deffd640 1 -- 172.21.15.5:0/2004975307 learned_addr learned my addr 172.21.15.5:0/2004975307 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.297+0000 7f06df7fe640 1 --2- 172.21.15.5:0/2004975307 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06e00cc820 0x7f06e01609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.298+0000 7f06d7fff640 1 --2- 172.21.15.5:0/2004975307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06e00d0560 0x7f06e0160470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.298+0000 7f06deffd640 1 -- 172.21.15.5:0/2004975307 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06e00cc820 msgr2=0x7f06e01609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.298+0000 7f06deffd640 1 --2- 172.21.15.5:0/2004975307 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06e00cc820 0x7f06e01609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.298+0000 7f06deffd640 1 -- 172.21.15.5:0/2004975307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06e00d0560 msgr2=0x7f06e0160470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.298+0000 7f06deffd640 1 --2- 172.21.15.5:0/2004975307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06e00d0560 0x7f06e0160470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.298+0000 7f06deffd640 1 -- 172.21.15.5:0/2004975307 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e00ccf30 con 0x7f06e00b4070 2024-05-20T18:32:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.298+0000 7f06df7fe640 1 --2- 172.21.15.5:0/2004975307 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06e00cc820 0x7f06e01609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:32:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.298+0000 7f06d7fff640 1 --2- 172.21.15.5:0/2004975307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06e00d0560 0x7f06e0160470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:32:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.298+0000 7f06deffd640 1 --2- 172.21.15.5:0/2004975307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06e00b4070 0x7f06e0161f00 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f06d000c820 tx=0x7f06d000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.298+0000 7f06d77fe640 1 -- 172.21.15.5:0/2004975307 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0015070 con 0x7f06e00b4070 2024-05-20T18:32:52.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.298+0000 7f06d77fe640 1 -- 172.21.15.5:0/2004975307 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06d0007a30 con 0x7f06e00b4070 2024-05-20T18:32:52.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.298+0000 7f06e527a640 1 -- 172.21.15.5:0/2004975307 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06e00cd210 con 0x7f06e00b4070 2024-05-20T18:32:52.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.298+0000 7f06d77fe640 1 -- 172.21.15.5:0/2004975307 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0007de0 con 0x7f06e00b4070 2024-05-20T18:32:52.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.298+0000 7f06e527a640 1 -- 172.21.15.5:0/2004975307 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06e00cd7d0 con 0x7f06e00b4070 2024-05-20T18:32:52.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.299+0000 7f06e527a640 1 -- 172.21.15.5:0/2004975307 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06e00cefb0 con 0x7f06e00b4070 2024-05-20T18:32:52.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.300+0000 7f06d77fe640 1 -- 172.21.15.5:0/2004975307 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f06d0018070 con 0x7f06e00b4070 2024-05-20T18:32:52.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.300+0000 7f06d77fe640 1 --2- 172.21.15.5:0/2004975307 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f06c006be30 0x7f06c006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.300+0000 7f06d77fe640 1 -- 172.21.15.5:0/2004975307 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f06d008afd0 con 0x7f06e00b4070 2024-05-20T18:32:52.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.300+0000 7f06d7fff640 1 --2- 172.21.15.5:0/2004975307 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f06c006be30 0x7f06c006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.301+0000 7f06d7fff640 1 --2- 172.21.15.5:0/2004975307 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f06c006be30 0x7f06c006e340 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f06e0154e70 tx=0x7f06c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:52.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.305+0000 7f06d77fe640 1 -- 172.21.15.5:0/2004975307 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f06e00cefb0 con 0x7f06e00b4070 2024-05-20T18:32:52.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.530+0000 7f06e527a640 1 -- 172.21.15.5:0/2004975307 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f06e00690b0 con 0x7f06e00b4070 2024-05-20T18:32:52.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.531+0000 7f06d77fe640 1 -- 172.21.15.5:0/2004975307 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f06d0059710 con 0x7f06e00b4070 2024-05-20T18:32:52.532 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:32:52.532 INFO:teuthology.orchestra.run.smithi005.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-20T18:32:52.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.533+0000 7f06e527a640 1 -- 172.21.15.5:0/2004975307 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f06c006be30 msgr2=0x7f06c006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:52.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.533+0000 7f06e527a640 1 --2- 172.21.15.5:0/2004975307 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f06c006be30 0x7f06c006e340 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f06e0154e70 tx=0x7f06c8008040 comp rx=0 tx=0).stop 2024-05-20T18:32:52.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.533+0000 7f06e527a640 1 -- 172.21.15.5:0/2004975307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06e00b4070 msgr2=0x7f06e0161f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:52.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.533+0000 7f06e527a640 1 --2- 172.21.15.5:0/2004975307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06e00b4070 0x7f06e0161f00 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f06d000c820 tx=0x7f06d000cce0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.533+0000 7f06e527a640 1 -- 172.21.15.5:0/2004975307 shutdown_connections 2024-05-20T18:32:52.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.533+0000 7f06e527a640 1 --2- 172.21.15.5:0/2004975307 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f06c006be30 0x7f06c006e340 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.533+0000 7f06e527a640 1 --2- 172.21.15.5:0/2004975307 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06e00cc820 0x7f06e01609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.533+0000 7f06e527a640 1 --2- 172.21.15.5:0/2004975307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06e00d0560 0x7f06e0160470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.533+0000 7f06e527a640 1 --2- 172.21.15.5:0/2004975307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06e00b4070 0x7f06e0161f00 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.533+0000 7f06e527a640 1 -- 172.21.15.5:0/2004975307 >> 172.21.15.5:0/2004975307 conn(0x7f06e00d4c10 msgr2=0x7f06e00b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:52.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.533+0000 7f06e527a640 1 -- 172.21.15.5:0/2004975307 shutdown_connections 2024-05-20T18:32:52.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.533+0000 7f06e527a640 1 -- 172.21.15.5:0/2004975307 wait complete. 2024-05-20T18:32:52.546 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-20T18:32:52.546 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-20T18:32:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.661+0000 7f2f4161b640 1 Processor -- start 2024-05-20T18:32:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.661+0000 7f2f4161b640 1 -- start start 2024-05-20T18:32:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.662+0000 7f2f4161b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f3c15f220 0x7f2f3c0cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.662+0000 7f2f4161b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2f3c0cf910 0x7f2f3c0d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.662+0000 7f2f4161b640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f3c0cfd20 con 0x7f2f3c0cd680 2024-05-20T18:32:52.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.662+0000 7f2f4161b640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f2f3c0cda90 con 0x7f2f3c0cf910 2024-05-20T18:32:52.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.662+0000 7f2f4161b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2f3c06d0f0 con 0x7f2f3c15f220 2024-05-20T18:32:52.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.662+0000 7f2f3b7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2f3c0cf910 0x7f2f3c0d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.663+0000 7f2f3b7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2f3c0cf910 0x7f2f3c0d1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37504/0 (socket says 172.21.15.5:37504) 2024-05-20T18:32:52.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.663+0000 7f2f3b7fe640 1 -- 172.21.15.5:0/2045587140 learned_addr learned my addr 172.21.15.5:0/2045587140 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:52.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.663+0000 7f2f3a7fc640 1 --2- 172.21.15.5:0/2045587140 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f3c15f220 0x7f2f3c0cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.663+0000 7f2f3b7fe640 1 -- 172.21.15.5:0/2045587140 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f3c15f220 msgr2=0x7f2f3c0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:52.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.663+0000 7f2f3b7fe640 1 --2- 172.21.15.5:0/2045587140 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f3c15f220 0x7f2f3c0cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.663+0000 7f2f3b7fe640 1 -- 172.21.15.5:0/2045587140 >> v1:172.21.15.63:6789/0 conn(0x7f2f3c0cd680 legacy=0x7f2f3c06c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:52.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.663+0000 7f2f3b7fe640 1 -- 172.21.15.5:0/2045587140 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f3c06d3d0 con 0x7f2f3c0cf910 2024-05-20T18:32:52.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.664+0000 7f2f3b7fe640 1 --2- 172.21.15.5:0/2045587140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f3c0cf910 0x7f2f3c0d1060 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009e50 tx=0x7f2f3402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9fa920edb2fa77a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:52.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.664+0000 7f2f39ffb640 1 -- 172.21.15.5:0/2045587140 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f34036070 con 0x7f2f3c0cf910 2024-05-20T18:32:52.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.664+0000 7f2f39ffb640 1 -- 172.21.15.5:0/2045587140 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f34031600 con 0x7f2f3c0cf910 2024-05-20T18:32:52.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.664+0000 7f2f39ffb640 1 -- 172.21.15.5:0/2045587140 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f340319d0 con 0x7f2f3c0cf910 2024-05-20T18:32:52.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.664+0000 7f2f4161b640 1 -- 172.21.15.5:0/2045587140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f3c0cf910 msgr2=0x7f2f3c0d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:52.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.664+0000 7f2f4161b640 1 --2- 172.21.15.5:0/2045587140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f3c0cf910 0x7f2f3c0d1060 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009e50 tx=0x7f2f3402a680 comp rx=0 tx=0).stop 2024-05-20T18:32:52.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.664+0000 7f2f4161b640 1 -- 172.21.15.5:0/2045587140 shutdown_connections 2024-05-20T18:32:52.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.664+0000 7f2f4161b640 1 --2- 172.21.15.5:0/2045587140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f3c0cf910 0x7f2f3c0d1060 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.664+0000 7f2f4161b640 1 --2- 172.21.15.5:0/2045587140 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f3c15f220 0x7f2f3c0cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.664+0000 7f2f4161b640 1 -- 172.21.15.5:0/2045587140 >> 172.21.15.5:0/2045587140 conn(0x7f2f3c0b44d0 msgr2=0x7f2f3c0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:52.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.665+0000 7f2f4161b640 1 -- 172.21.15.5:0/2045587140 shutdown_connections 2024-05-20T18:32:52.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.665+0000 7f2f4161b640 1 -- 172.21.15.5:0/2045587140 wait complete. 2024-05-20T18:32:52.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.666+0000 7f2f4161b640 1 Processor -- start 2024-05-20T18:32:52.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.667+0000 7f2f4161b640 1 -- start start 2024-05-20T18:32:52.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.668+0000 7f2f4161b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f3c0cd680 0x7f2f3c0b7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.668+0000 7f2f4161b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f3c0cf910 0x7f2f3c157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.668+0000 7f2f4161b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f3c15f220 0x7f2f3c0b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.668+0000 7f2f4161b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c06d990 con 0x7f2f3c15f220 2024-05-20T18:32:52.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.668+0000 7f2f4161b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c0d0b40 con 0x7f2f3c0cf910 2024-05-20T18:32:52.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.668+0000 7f2f4161b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2f3c06cdd0 con 0x7f2f3c0cd680 2024-05-20T18:32:52.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.668+0000 7f2f3b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f3c15f220 0x7f2f3c0b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.668+0000 7f2f3b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f3c15f220 0x7f2f3c0b57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37514/0 (socket says 172.21.15.5:37514) 2024-05-20T18:32:52.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.668+0000 7f2f3b7fe640 1 -- 172.21.15.5:0/3572632640 learned_addr learned my addr 172.21.15.5:0/3572632640 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:52.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.668+0000 7f2f3a7fc640 1 --2- 172.21.15.5:0/3572632640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f3c0cf910 0x7f2f3c157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.668+0000 7f2f3affd640 1 --2- 172.21.15.5:0/3572632640 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f3c0cd680 0x7f2f3c0b7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.668+0000 7f2f3b7fe640 1 -- 172.21.15.5:0/3572632640 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f3c0cd680 msgr2=0x7f2f3c0b7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:52.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.668+0000 7f2f3b7fe640 1 --2- 172.21.15.5:0/3572632640 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f3c0cd680 0x7f2f3c0b7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.668+0000 7f2f3b7fe640 1 -- 172.21.15.5:0/3572632640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f3c0cf910 msgr2=0x7f2f3c157db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:52.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.668+0000 7f2f3b7fe640 1 --2- 172.21.15.5:0/3572632640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f3c0cf910 0x7f2f3c157db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:52.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.668+0000 7f2f3b7fe640 1 -- 172.21.15.5:0/3572632640 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f3c0b7bc0 con 0x7f2f3c15f220 2024-05-20T18:32:52.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.668+0000 7f2f3b7fe640 1 --2- 172.21.15.5:0/3572632640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f3c15f220 0x7f2f3c0b57c0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f2f34034040 tx=0x7f2f34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:52.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.669+0000 7f2f237fe640 1 -- 172.21.15.5:0/3572632640 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f34036050 con 0x7f2f3c15f220 2024-05-20T18:32:52.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.669+0000 7f2f237fe640 1 -- 172.21.15.5:0/3572632640 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f34042070 con 0x7f2f3c15f220 2024-05-20T18:32:52.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.669+0000 7f2f237fe640 1 -- 172.21.15.5:0/3572632640 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f34009510 con 0x7f2f3c15f220 2024-05-20T18:32:52.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.669+0000 7f2f4161b640 1 -- 172.21.15.5:0/3572632640 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f3c0b80d0 con 0x7f2f3c15f220 2024-05-20T18:32:52.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.669+0000 7f2f4161b640 1 -- 172.21.15.5:0/3572632640 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f3c1f8fd0 con 0x7f2f3c15f220 2024-05-20T18:32:52.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.670+0000 7f2f4161b640 1 -- 172.21.15.5:0/3572632640 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f3c0b7e40 con 0x7f2f3c15f220 2024-05-20T18:32:52.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.671+0000 7f2f237fe640 1 -- 172.21.15.5:0/3572632640 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2f34007bf0 con 0x7f2f3c15f220 2024-05-20T18:32:52.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.671+0000 7f2f237fe640 1 --2- 172.21.15.5:0/3572632640 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2f2806be30 0x7f2f2806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:52.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.671+0000 7f2f3affd640 1 --2- 172.21.15.5:0/3572632640 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2f2806be30 0x7f2f2806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:52.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.671+0000 7f2f237fe640 1 -- 172.21.15.5:0/3572632640 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f2f340abdf0 con 0x7f2f3c15f220 2024-05-20T18:32:52.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.671+0000 7f2f3affd640 1 --2- 172.21.15.5:0/3572632640 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2f2806be30 0x7f2f2806e340 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0b1cd0 tx=0x7f2f2c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:52.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.676+0000 7f2f237fe640 1 -- 172.21.15.5:0/3572632640 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2f34079ee0 con 0x7f2f3c15f220 2024-05-20T18:32:52.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.902+0000 7f2f4161b640 1 -- 172.21.15.5:0/3572632640 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2f3c075b00 con 0x7f2f3c15f220 2024-05-20T18:32:53.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.902+0000 7f2f237fe640 1 -- 172.21.15.5:0/3572632640 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v52) ==== 53+0+73 (secure 0 0 0) 0x7f2f3407e1e0 con 0x7f2f3c15f220 2024-05-20T18:32:53.046 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-20T18:32:53.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.904+0000 7f2f4161b640 1 -- 172.21.15.5:0/3572632640 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2f2806be30 msgr2=0x7f2f2806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:53.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.904+0000 7f2f4161b640 1 --2- 172.21.15.5:0/3572632640 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2f2806be30 0x7f2f2806e340 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0b1cd0 tx=0x7f2f2c009040 comp rx=0 tx=0).stop 2024-05-20T18:32:53.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.904+0000 7f2f4161b640 1 -- 172.21.15.5:0/3572632640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f3c15f220 msgr2=0x7f2f3c0b57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:53.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.904+0000 7f2f4161b640 1 --2- 172.21.15.5:0/3572632640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f3c15f220 0x7f2f3c0b57c0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f2f34034040 tx=0x7f2f34004a40 comp rx=0 tx=0).stop 2024-05-20T18:32:53.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.904+0000 7f2f4161b640 1 -- 172.21.15.5:0/3572632640 shutdown_connections 2024-05-20T18:32:53.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.904+0000 7f2f4161b640 1 --2- 172.21.15.5:0/3572632640 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2f2806be30 0x7f2f2806e340 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:53.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.904+0000 7f2f4161b640 1 --2- 172.21.15.5:0/3572632640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f3c15f220 0x7f2f3c0b57c0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:53.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.904+0000 7f2f4161b640 1 --2- 172.21.15.5:0/3572632640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f3c0cf910 0x7f2f3c157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:53.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.905+0000 7f2f4161b640 1 --2- 172.21.15.5:0/3572632640 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2f3c0cd680 0x7f2f3c0b7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:53.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.905+0000 7f2f4161b640 1 -- 172.21.15.5:0/3572632640 >> 172.21.15.5:0/3572632640 conn(0x7f2f3c0b44d0 msgr2=0x7f2f3c0d0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:53.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.905+0000 7f2f4161b640 1 -- 172.21.15.5:0/3572632640 shutdown_connections 2024-05-20T18:32:53.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:52.905+0000 7f2f4161b640 1 -- 172.21.15.5:0/3572632640 wait complete. 2024-05-20T18:32:53.048 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-20T18:32:53.048 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-20T18:32:53.048 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-05-20T18:32:53.049 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-20T18:32:53.049 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-20T18:32:53.049 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-20T18:32:53.049 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-20T18:32:53.049 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-20T18:32:53.049 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:32:53.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:32:53.049 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-05-20T18:32:53.067 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T18:32:53.067 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T18:32:53.067 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T18:32:53.068 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:32:53.068 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-20T18:32:53.068 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:32:53.068 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:32:53.068 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-05-20T18:32:53.068 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-05-20T18:32:53.068 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:32:53.068 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-05-20T18:32:53.068 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-05-20T18:32:53.068 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-20T18:32:53.068 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:32:53.068 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-05-20T18:32:53.068 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-20T18:32:53.069 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42325sec preferred_lft 42325sec 2024-05-20T18:32:53.069 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-05-20T18:32:53.069 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:32:53.069 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:32:53.069 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:32:53.069 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-05-20T18:32:53.069 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-20T18:32:53.069 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:32:53.069 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-20T18:32:53.069 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:32:53.069 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::4474:eaff:fe07:4f9a/64 scope link 2024-05-20T18:32:53.069 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:32:53.069 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:32:53.069 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T18:32:53.070 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ce4:f4ff:fe5e:7906/64 scope link 2024-05-20T18:32:53.070 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:32:53.070 INFO:teuthology.orchestra.run.smithi005.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:32:53.070 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether da:26:04:0e:95:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T18:32:53.070 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::d826:4ff:fe0e:950a/64 scope link 2024-05-20T18:32:53.070 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:32:53.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:32:53.070 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-20T18:32:53.132 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-20T18:32:53.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:32:53.134 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-05-20T18:32:53.191 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-05-20T18:32:53.191 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-20T18:32:53.191 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-20T18:32:53.191 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-20T18:32:53.191 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-20T18:32:53.192 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:32:53.192 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-20T18:32:53.192 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-20T18:32:53.192 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-20T18:32:53.192 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-20T18:32:53.192 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-20T18:32:53.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:32:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:32:53.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:32:53.322 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-20T18:32:53.338 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-20T18:32:53.338 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-20T18:32:53.338 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:32:53.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:32:53.339 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-05-20T18:32:53.404 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:32:53.405 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-20T18:32:53.460 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:32:53.461 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:32:53.461 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:32:53.516 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-20T18:32:53.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:32:53.516 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:32:53.612 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:32:53.612 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:32:53.612 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:32:53.626 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:32:53.626 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:32:53.626 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:32:53.626 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:32:53.627 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:32:53.632 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:32:53.633 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:32:53.633 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:32:53.633 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:32:53.633 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:32:53.633 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790" 2024-05-20T18:32:53.633 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790:/ 2024-05-20T18:32:53.633 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=399ab250-fd3c-47fa-a1c8-25e8149c9cac" will pass to kernel 2024-05-20T18:32:53.639 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-20T18:32:53.639 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-05-20T18:32:53.639 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-20T18:32:53.639 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-05-20T18:32:53.639 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-20T18:32:53.639 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-20T18:32:53.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:32:53.639 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:32:53.666 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-20T18:32:53.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T18:32:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:32:53.764 INFO:teuthology.orchestra.run.smithi005.stdout:client6113 2024-05-20T18:32:53.765 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:32:53.765 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:32:53.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:32:53.782 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-05-20T18:32:53.839 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-20T18:32:53.840 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-05-20T18:32:53.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.937+0000 7febed70ff40 1 Processor -- start 2024-05-20T18:32:53.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.937+0000 7febed70ff40 1 -- start start 2024-05-20T18:32:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.938+0000 7febed70ff40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febd0003ab0 0x556374ebee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.938+0000 7febed70ff40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556374eb9bb0 0x556374ec46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.938+0000 7febed70ff40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556374ec4c00 0x556374ec33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.938+0000 7febed70ff40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556374d95c80 con 0x7febd0003ab0 2024-05-20T18:32:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.938+0000 7febed70ff40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556374d99cc0 con 0x556374ec4c00 2024-05-20T18:32:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.938+0000 7febed70ff40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x556374ebf710 con 0x556374eb9bb0 2024-05-20T18:32:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.938+0000 7febe71e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febd0003ab0 0x556374ebee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.938+0000 7febe71e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febd0003ab0 0x556374ebee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37532/0 (socket says 172.21.15.5:37532) 2024-05-20T18:32:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.938+0000 7febe71e0640 1 -- 172.21.15.5:0/1224835180 learned_addr learned my addr 172.21.15.5:0/1224835180 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.938+0000 7febe69df640 1 --2- 172.21.15.5:0/1224835180 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556374eb9bb0 0x556374ec46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.938+0000 7febe79e1640 1 --2- 172.21.15.5:0/1224835180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556374ec4c00 0x556374ec33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.938+0000 7febe71e0640 1 -- 172.21.15.5:0/1224835180 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556374eb9bb0 msgr2=0x556374ec46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:53.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.938+0000 7febe71e0640 1 --2- 172.21.15.5:0/1224835180 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556374eb9bb0 0x556374ec46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:53.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.938+0000 7febe71e0640 1 -- 172.21.15.5:0/1224835180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556374ec4c00 msgr2=0x556374ec33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:53.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.938+0000 7febe71e0640 1 --2- 172.21.15.5:0/1224835180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556374ec4c00 0x556374ec33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:53.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.938+0000 7febe71e0640 1 -- 172.21.15.5:0/1224835180 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556374e393b0 con 0x7febd0003ab0 2024-05-20T18:32:53.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.938+0000 7febe79e1640 1 --2- 172.21.15.5:0/1224835180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556374ec4c00 0x556374ec33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:32:53.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.938+0000 7febe71e0640 1 --2- 172.21.15.5:0/1224835180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febd0003ab0 0x556374ebee80 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7febd8002f30 tx=0x7febd800e760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00ae49356210370 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:53.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.939+0000 7febe59dd640 1 -- 172.21.15.5:0/1224835180 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd8001ac0 con 0x7febd0003ab0 2024-05-20T18:32:53.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.939+0000 7febe59dd640 1 -- 172.21.15.5:0/1224835180 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febd8001c60 con 0x7febd0003ab0 2024-05-20T18:32:53.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.939+0000 7febe59dd640 1 -- 172.21.15.5:0/1224835180 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd8002230 con 0x7febd0003ab0 2024-05-20T18:32:53.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.939+0000 7febed70ff40 1 -- 172.21.15.5:0/1224835180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febd0003ab0 msgr2=0x556374ebee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:53.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.939+0000 7febed70ff40 1 --2- 172.21.15.5:0/1224835180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febd0003ab0 0x556374ebee80 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7febd8002f30 tx=0x7febd800e760 comp rx=0 tx=0).stop 2024-05-20T18:32:53.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.939+0000 7febed70ff40 1 -- 172.21.15.5:0/1224835180 shutdown_connections 2024-05-20T18:32:53.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.939+0000 7febed70ff40 1 --2- 172.21.15.5:0/1224835180 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556374ec4c00 0x556374ec33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:53.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.939+0000 7febed70ff40 1 --2- 172.21.15.5:0/1224835180 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556374eb9bb0 0x556374ec46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:53.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.939+0000 7febed70ff40 1 --2- 172.21.15.5:0/1224835180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febd0003ab0 0x556374ebee80 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:53.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.939+0000 7febed70ff40 1 -- 172.21.15.5:0/1224835180 >> 172.21.15.5:0/1224835180 conn(0x556374d13ac0 msgr2=0x556374ec1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:53.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.939+0000 7febed70ff40 1 -- 172.21.15.5:0/1224835180 shutdown_connections 2024-05-20T18:32:53.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.939+0000 7febed70ff40 1 -- 172.21.15.5:0/1224835180 wait complete. 2024-05-20T18:32:53.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.940+0000 7febed70ff40 1 Processor -- start 2024-05-20T18:32:53.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.940+0000 7febed70ff40 1 -- start start 2024-05-20T18:32:53.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.941+0000 7febed70ff40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556374eb9bb0 0x556374e1a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:53.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.941+0000 7febed70ff40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556374ec4c00 0x556374eaf780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:53.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.941+0000 7febed70ff40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febd0003ab0 0x556374eb7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:53.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.941+0000 7febed70ff40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556374e39160 con 0x556374eb9bb0 2024-05-20T18:32:53.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.941+0000 7febed70ff40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556374ec4090 con 0x7febd0003ab0 2024-05-20T18:32:53.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.942+0000 7febed70ff40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x556374eb0320 con 0x556374ec4c00 2024-05-20T18:32:53.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.942+0000 7febe71e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556374eb9bb0 0x556374e1a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:53.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.942+0000 7febe71e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556374eb9bb0 0x556374e1a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37546/0 (socket says 172.21.15.5:37546) 2024-05-20T18:32:53.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.942+0000 7febe71e0640 1 -- 172.21.15.5:0/2665536186 learned_addr learned my addr 172.21.15.5:0/2665536186 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:53.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.942+0000 7febe69df640 1 --2- 172.21.15.5:0/2665536186 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556374ec4c00 0x556374eaf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:53.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.942+0000 7febe79e1640 1 --2- 172.21.15.5:0/2665536186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febd0003ab0 0x556374eb7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:53.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.942+0000 7febe71e0640 1 -- 172.21.15.5:0/2665536186 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556374ec4c00 msgr2=0x556374eaf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:53.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.942+0000 7febe71e0640 1 --2- 172.21.15.5:0/2665536186 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556374ec4c00 0x556374eaf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:53.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.942+0000 7febe71e0640 1 -- 172.21.15.5:0/2665536186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febd0003ab0 msgr2=0x556374eb7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:53.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.942+0000 7febe71e0640 1 --2- 172.21.15.5:0/2665536186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febd0003ab0 0x556374eb7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:53.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.942+0000 7febe71e0640 1 -- 172.21.15.5:0/2665536186 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556374e192e0 con 0x556374eb9bb0 2024-05-20T18:32:53.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.942+0000 7febe69df640 1 --2- 172.21.15.5:0/2665536186 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556374ec4c00 0x556374eaf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:32:53.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.942+0000 7febe79e1640 1 --2- 172.21.15.5:0/2665536186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7febd0003ab0 0x556374eb7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:32:53.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.942+0000 7febe71e0640 1 --2- 172.21.15.5:0/2665536186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556374eb9bb0 0x556374e1a0a0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7febd800e7a0 tx=0x7febd8001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:53.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.942+0000 7febe49db640 1 -- 172.21.15.5:0/2665536186 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd8001ac0 con 0x556374eb9bb0 2024-05-20T18:32:53.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.942+0000 7febe49db640 1 -- 172.21.15.5:0/2665536186 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febd8022030 con 0x556374eb9bb0 2024-05-20T18:32:53.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.942+0000 7febe49db640 1 -- 172.21.15.5:0/2665536186 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd8011a80 con 0x556374eb9bb0 2024-05-20T18:32:53.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.942+0000 7febed70ff40 1 -- 172.21.15.5:0/2665536186 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556374e195c0 con 0x556374eb9bb0 2024-05-20T18:32:53.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.942+0000 7febed70ff40 1 -- 172.21.15.5:0/2665536186 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556374e19b80 con 0x556374eb9bb0 2024-05-20T18:32:53.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.944+0000 7febe49db640 1 -- 172.21.15.5:0/2665536186 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7febd8011c20 con 0x556374eb9bb0 2024-05-20T18:32:53.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.944+0000 7febe49db640 1 --2- 172.21.15.5:0/2665536186 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7febd406a810 0x7febd406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:53.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.944+0000 7febe49db640 1 -- 172.21.15.5:0/2665536186 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7febd808c1b0 con 0x556374eb9bb0 2024-05-20T18:32:53.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.944+0000 7febe69df640 1 --2- 172.21.15.5:0/2665536186 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7febd406a810 0x7febd406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:53.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.944+0000 7febed70ff40 1 --2- 172.21.15.5:0/2665536186 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x556374ec4630 0x556374f685b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:53.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.944+0000 7febed70ff40 1 -- 172.21.15.5:0/2665536186 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78) -- 0x556374eabe20 con 0x556374ec4630 2024-05-20T18:32:53.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.945+0000 7febe79e1640 1 --2- 172.21.15.5:0/2665536186 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x556374ec4630 0x556374f685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:53.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.945+0000 7febe69df640 1 --2- 172.21.15.5:0/2665536186 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7febd406a810 0x7febd406cd20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x556374ebe860 tx=0x7febdc001c40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:53.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.945+0000 7febe79e1640 1 --2- 172.21.15.5:0/2665536186 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x556374ec4630 0x556374f685b0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:53.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.947+0000 7febe79e1640 1 -- 172.21.15.5:0/2665536186 <== osd.0 v2:172.21.15.5:6808/688093005 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v78'12 uv12 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7febe0005040 con 0x556374ec4630 2024-05-20T18:32:53.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.947+0000 7febed70ff40 1 -- 172.21.15.5:0/2665536186 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x556374ec4630 msgr2=0x556374f685b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:53.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.947+0000 7febed70ff40 1 --2- 172.21.15.5:0/2665536186 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x556374ec4630 0x556374f685b0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:53.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.947+0000 7febed70ff40 1 -- 172.21.15.5:0/2665536186 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7febd406a810 msgr2=0x7febd406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:53.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.947+0000 7febed70ff40 1 --2- 172.21.15.5:0/2665536186 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7febd406a810 0x7febd406cd20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x556374ebe860 tx=0x7febdc001c40 comp rx=0 tx=0).stop 2024-05-20T18:32:53.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.947+0000 7febed70ff40 1 -- 172.21.15.5:0/2665536186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556374eb9bb0 msgr2=0x556374e1a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:53.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.947+0000 7febed70ff40 1 --2- 172.21.15.5:0/2665536186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556374eb9bb0 0x556374e1a0a0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7febd800e7a0 tx=0x7febd8001190 comp rx=0 tx=0).stop 2024-05-20T18:32:53.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.947+0000 7febe79e1640 1 -- 172.21.15.5:0/2665536186 reap_dead start 2024-05-20T18:32:53.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.947+0000 7febed70ff40 1 -- 172.21.15.5:0/2665536186 shutdown_connections 2024-05-20T18:32:53.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.947+0000 7febed70ff40 1 -- 172.21.15.5:0/2665536186 >> 172.21.15.5:0/2665536186 conn(0x556374d13ac0 msgr2=0x556374ebf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:53.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.947+0000 7febed70ff40 1 -- 172.21.15.5:0/2665536186 shutdown_connections 2024-05-20T18:32:53.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:53.947+0000 7febed70ff40 1 -- 172.21.15.5:0/2665536186 wait complete. 2024-05-20T18:32:53.952 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-20T18:32:53.952 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T18:32:54.004 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T18:32:54.054 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-20T18:32:54.054 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T18:32:54.054 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:32:54.054 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:32:54.071 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:32:54.071 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:32:54.128 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:32:54.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.282+0000 7fe03e32c640 1 Processor -- start 2024-05-20T18:32:54.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.282+0000 7fe03e32c640 1 -- start start 2024-05-20T18:32:54.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.283+0000 7fe03e32c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe038168050 0x7fe03816a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:54.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.283+0000 7fe03e32c640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0380d0630 con 0x7fe03806d3d0 2024-05-20T18:32:54.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.283+0000 7fe03e32c640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0380d39c0 con 0x7fe038162820 2024-05-20T18:32:54.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.283+0000 7fe03e32c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe0380d3ca0 con 0x7fe038168050 2024-05-20T18:32:54.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.283+0000 7fe0377fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe03806d3d0 0x7fe03806d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50130/0 (socket says 172.21.15.5:50130) 2024-05-20T18:32:54.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.284+0000 7fe0377fe640 1 -- 172.21.15.5:0/3021491124 learned_addr learned my addr 172.21.15.5:0/3021491124 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:32:54.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.284+0000 7fe03c8c9640 1 --2- 172.21.15.5:0/3021491124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe038168050 0x7fe03816a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:54.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.284+0000 7fe036ffd640 1 -- 172.21.15.5:0/3021491124 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2630520460 0 0) 0x7fe0380d0630 con 0x7fe03806d3d0 2024-05-20T18:32:54.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.284+0000 7fe036ffd640 1 -- 172.21.15.5:0/3021491124 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe020003520 con 0x7fe03806d3d0 2024-05-20T18:32:54.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.284+0000 7fe036ffd640 1 -- 172.21.15.5:0/3021491124 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2662438653 0 0) 0x7fe020003520 con 0x7fe03806d3d0 2024-05-20T18:32:54.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.284+0000 7fe036ffd640 1 -- 172.21.15.5:0/3021491124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe038168050 msgr2=0x7fe03816a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:54.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.284+0000 7fe036ffd640 1 --2- 172.21.15.5:0/3021491124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe038168050 0x7fe03816a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:54.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.284+0000 7fe036ffd640 1 -- 172.21.15.5:0/3021491124 >> v1:172.21.15.63:6789/0 conn(0x7fe038162820 legacy=0x7fe0380b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:54.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.284+0000 7fe036ffd640 1 -- 172.21.15.5:0/3021491124 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0380d3f80 con 0x7fe03806d3d0 2024-05-20T18:32:54.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.284+0000 7fe036ffd640 1 -- 172.21.15.5:0/3021491124 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7fe02c001670 con 0x7fe03806d3d0 2024-05-20T18:32:54.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.284+0000 7fe036ffd640 1 -- 172.21.15.5:0/3021491124 >> v1:172.21.15.5:6789/0 conn(0x7fe03806d3d0 legacy=0x7fe03806d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:54.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.284+0000 7fe036ffd640 1 --2- 172.21.15.5:0/3021491124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0200039e0 0x7fe020003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:54.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.284+0000 7fe036ffd640 1 --2- 172.21.15.5:0/3021491124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe038168050 0x7fe020004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:54.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.284+0000 7fe036ffd640 1 --2- 172.21.15.5:0/3021491124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe020004d40 0x7fe020005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:54.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.284+0000 7fe036ffd640 1 -- 172.21.15.5:0/3021491124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe0380d3ca0 con 0x7fe020004d40 2024-05-20T18:32:54.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.284+0000 7fe036ffd640 1 -- 172.21.15.5:0/3021491124 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe02000ba70 con 0x7fe0200039e0 2024-05-20T18:32:54.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.284+0000 7fe036ffd640 1 -- 172.21.15.5:0/3021491124 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe02000bc70 con 0x7fe038168050 2024-05-20T18:32:54.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.284+0000 7fe03c8c9640 1 --2- 172.21.15.5:0/3021491124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe020004d40 0x7fe020005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:54.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.284+0000 7fe0377fe640 1 --2- 172.21.15.5:0/3021491124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe038168050 0x7fe020004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:54.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.284+0000 7fe037fff640 1 --2- 172.21.15.5:0/3021491124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0200039e0 0x7fe020003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:54.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.285+0000 7fe03c8c9640 1 -- 172.21.15.5:0/3021491124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe038168050 msgr2=0x7fe020004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:54.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.285+0000 7fe03c8c9640 1 --2- 172.21.15.5:0/3021491124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe038168050 0x7fe020004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:54.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.285+0000 7fe03c8c9640 1 -- 172.21.15.5:0/3021491124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0200039e0 msgr2=0x7fe020003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:54.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.285+0000 7fe03c8c9640 1 --2- 172.21.15.5:0/3021491124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0200039e0 0x7fe020003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:54.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.285+0000 7fe03c8c9640 1 -- 172.21.15.5:0/3021491124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe020003520 con 0x7fe020004d40 2024-05-20T18:32:54.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.285+0000 7fe037fff640 1 --2- 172.21.15.5:0/3021491124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0200039e0 0x7fe020003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:32:54.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.285+0000 7fe03c8c9640 1 --2- 172.21.15.5:0/3021491124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe020004d40 0x7fe020005150 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fe0380cbc50 tx=0x7fe020005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21352c8acf2a64e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:54.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.286+0000 7fe036ffd640 1 -- 172.21.15.5:0/3021491124 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe030009c10 con 0x7fe020004d40 2024-05-20T18:32:54.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.286+0000 7fe036ffd640 1 -- 172.21.15.5:0/3021491124 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe030036070 con 0x7fe020004d40 2024-05-20T18:32:54.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.286+0000 7fe036ffd640 1 -- 172.21.15.5:0/3021491124 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0300317f0 con 0x7fe020004d40 2024-05-20T18:32:54.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.286+0000 7fe03e32c640 1 -- 172.21.15.5:0/3021491124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe020004d40 msgr2=0x7fe020005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:54.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.286+0000 7fe03e32c640 1 --2- 172.21.15.5:0/3021491124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe020004d40 0x7fe020005150 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fe0380cbc50 tx=0x7fe020005830 comp rx=0 tx=0).stop 2024-05-20T18:32:54.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.286+0000 7fe03c8c9640 1 -- 172.21.15.5:0/3021491124 reap_dead start 2024-05-20T18:32:54.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.286+0000 7fe03e32c640 1 -- 172.21.15.5:0/3021491124 shutdown_connections 2024-05-20T18:32:54.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.286+0000 7fe03e32c640 1 -- 172.21.15.5:0/3021491124 >> 172.21.15.5:0/3021491124 conn(0x7fe0380d4bd0 msgr2=0x7fe0380d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:54.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.286+0000 7fe03e32c640 1 -- 172.21.15.5:0/3021491124 shutdown_connections 2024-05-20T18:32:54.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.286+0000 7fe03e32c640 1 -- 172.21.15.5:0/3021491124 wait complete. 2024-05-20T18:32:54.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.288+0000 7fe03e32c640 1 Processor -- start 2024-05-20T18:32:54.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.288+0000 7fe03e32c640 1 -- start start 2024-05-20T18:32:54.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.290+0000 7fe03e32c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe020004d40 0x7fe03813fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:54.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.290+0000 7fe03e32c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe038153940 0x7fe038141ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:54.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.290+0000 7fe03e32c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0381420e0 0x7fe038157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:54.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.290+0000 7fe03e32c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe0380b91a0 con 0x7fe020004d40 2024-05-20T18:32:54.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.290+0000 7fe03e32c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe0380d0300 con 0x7fe038153940 2024-05-20T18:32:54.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.290+0000 7fe03e32c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe0380b4050 con 0x7fe0381420e0 2024-05-20T18:32:54.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.290+0000 7fe037fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe020004d40 0x7fe03813fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:54.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.290+0000 7fe037fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe020004d40 0x7fe03813fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37556/0 (socket says 172.21.15.5:37556) 2024-05-20T18:32:54.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.290+0000 7fe037fff640 1 -- 172.21.15.5:0/4231006304 learned_addr learned my addr 172.21.15.5:0/4231006304 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:54.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.290+0000 7fe03c8c9640 1 --2- 172.21.15.5:0/4231006304 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0381420e0 0x7fe038157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:54.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.290+0000 7fe0377fe640 1 --2- 172.21.15.5:0/4231006304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe038153940 0x7fe038141ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:54.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.290+0000 7fe037fff640 1 -- 172.21.15.5:0/4231006304 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0381420e0 msgr2=0x7fe038157740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:54.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.290+0000 7fe037fff640 1 --2- 172.21.15.5:0/4231006304 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0381420e0 0x7fe038157740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:54.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.290+0000 7fe037fff640 1 -- 172.21.15.5:0/4231006304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe038153940 msgr2=0x7fe038141ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:54.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.290+0000 7fe037fff640 1 --2- 172.21.15.5:0/4231006304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe038153940 0x7fe038141ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:54.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.290+0000 7fe037fff640 1 -- 172.21.15.5:0/4231006304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe038142890 con 0x7fe020004d40 2024-05-20T18:32:54.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.290+0000 7fe037fff640 1 --2- 172.21.15.5:0/4231006304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe020004d40 0x7fe03813fb10 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fe02800e800 tx=0x7fe02800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:54.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.290+0000 7fe034ff9640 1 -- 172.21.15.5:0/4231006304 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0280052e0 con 0x7fe020004d40 2024-05-20T18:32:54.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.290+0000 7fe034ff9640 1 -- 172.21.15.5:0/4231006304 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe028007020 con 0x7fe020004d40 2024-05-20T18:32:54.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.290+0000 7fe03e32c640 1 -- 172.21.15.5:0/4231006304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe038156c60 con 0x7fe020004d40 2024-05-20T18:32:54.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.290+0000 7fe034ff9640 1 -- 172.21.15.5:0/4231006304 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0280071f0 con 0x7fe020004d40 2024-05-20T18:32:54.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.291+0000 7fe03e32c640 1 -- 172.21.15.5:0/4231006304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe038157220 con 0x7fe020004d40 2024-05-20T18:32:54.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.291+0000 7fe03e32c640 1 -- 172.21.15.5:0/4231006304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe038075b00 con 0x7fe020004d40 2024-05-20T18:32:54.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.292+0000 7fe034ff9640 1 -- 172.21.15.5:0/4231006304 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe02801a020 con 0x7fe020004d40 2024-05-20T18:32:54.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.293+0000 7fe034ff9640 1 --2- 172.21.15.5:0/4231006304 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe02406be60 0x7fe02406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:54.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.293+0000 7fe034ff9640 1 -- 172.21.15.5:0/4231006304 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7fe02808a8e0 con 0x7fe020004d40 2024-05-20T18:32:54.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.293+0000 7fe0377fe640 1 --2- 172.21.15.5:0/4231006304 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe02406be60 0x7fe02406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:54.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.293+0000 7fe0377fe640 1 --2- 172.21.15.5:0/4231006304 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe02406be60 0x7fe02406e370 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe03815f450 tx=0x7fe02c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:54.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.297+0000 7fe034ff9640 1 -- 172.21.15.5:0/4231006304 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe028059020 con 0x7fe020004d40 2024-05-20T18:32:54.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.479+0000 7fe03e32c640 1 -- 172.21.15.5:0/4231006304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe038156a40 con 0x7fe020004d40 2024-05-20T18:32:54.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.481+0000 7fe034ff9640 1 -- 172.21.15.5:0/4231006304 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+16950 (secure 0 0 0) 0x7fe028058920 con 0x7fe020004d40 2024-05-20T18:32:54.482 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:32:54.482 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":78,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:31:40.839398+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:31:27.896386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-20T18:31:29.146912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6836/791176355":"2024-05-21T18:31:40.839373+0000","172.21.15.5:6835/791176355":"2024-05-21T18:31:40.839373+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:32:54.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.483+0000 7fe03e32c640 1 -- 172.21.15.5:0/4231006304 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe02406be60 msgr2=0x7fe02406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:54.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.483+0000 7fe03e32c640 1 --2- 172.21.15.5:0/4231006304 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe02406be60 0x7fe02406e370 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe03815f450 tx=0x7fe02c009040 comp rx=0 tx=0).stop 2024-05-20T18:32:54.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.483+0000 7fe03e32c640 1 -- 172.21.15.5:0/4231006304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe020004d40 msgr2=0x7fe03813fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:54.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.483+0000 7fe03e32c640 1 --2- 172.21.15.5:0/4231006304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe020004d40 0x7fe03813fb10 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fe02800e800 tx=0x7fe02800ecc0 comp rx=0 tx=0).stop 2024-05-20T18:32:54.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.483+0000 7fe03e32c640 1 -- 172.21.15.5:0/4231006304 shutdown_connections 2024-05-20T18:32:54.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.483+0000 7fe03e32c640 1 --2- 172.21.15.5:0/4231006304 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe02406be60 0x7fe02406e370 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:54.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.483+0000 7fe03e32c640 1 --2- 172.21.15.5:0/4231006304 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0381420e0 0x7fe038157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:54.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.483+0000 7fe03e32c640 1 --2- 172.21.15.5:0/4231006304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe038153940 0x7fe038141ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:54.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.483+0000 7fe03e32c640 1 --2- 172.21.15.5:0/4231006304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe020004d40 0x7fe03813fb10 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:54.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.483+0000 7fe03e32c640 1 -- 172.21.15.5:0/4231006304 >> 172.21.15.5:0/4231006304 conn(0x7fe0380d4bd0 msgr2=0x7fe03815afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:54.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.483+0000 7fe03e32c640 1 -- 172.21.15.5:0/4231006304 shutdown_connections 2024-05-20T18:32:54.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.483+0000 7fe03e32c640 1 -- 172.21.15.5:0/4231006304 wait complete. 2024-05-20T18:32:54.495 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:31:27.896386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:31:29.146912+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:32:54.495 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:32:54.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.613+0000 7fcd0e099640 1 Processor -- start 2024-05-20T18:32:54.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.613+0000 7fcd0e099640 1 -- start start 2024-05-20T18:32:54.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.615+0000 7fcd0e099640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fcd08162820 0x7fcd080b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:54.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.615+0000 7fcd0e099640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcd080b3e80 0x7fcd0806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:54.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.615+0000 7fcd0e099640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd0806dd20 con 0x7fcd0806d910 2024-05-20T18:32:54.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.615+0000 7fcd0e099640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fcd080d0630 con 0x7fcd080b3e80 2024-05-20T18:32:54.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.615+0000 7fcd0e099640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fcd080d39c0 con 0x7fcd08162820 2024-05-20T18:32:54.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.615+0000 7fcd06ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcd080b3e80 0x7fcd0806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:54.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.615+0000 7fcd06ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcd080b3e80 0x7fcd0806d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37568/0 (socket says 172.21.15.5:37568) 2024-05-20T18:32:54.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.615+0000 7fcd06ffd640 1 -- 172.21.15.5:0/2409319059 learned_addr learned my addr 172.21.15.5:0/2409319059 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:54.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.615+0000 7fcd077fe640 1 --2- 172.21.15.5:0/2409319059 >> v2:172.21.15.63:3300/0 conn(0x7fcd08162820 0x7fcd080b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:54.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.615+0000 7fcd06ffd640 1 -- 172.21.15.5:0/2409319059 >> v2:172.21.15.63:3300/0 conn(0x7fcd08162820 msgr2=0x7fcd080b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:54.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.615+0000 7fcd06ffd640 1 --2- 172.21.15.5:0/2409319059 >> v2:172.21.15.63:3300/0 conn(0x7fcd08162820 0x7fcd080b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:54.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.615+0000 7fcd06ffd640 1 -- 172.21.15.5:0/2409319059 >> v1:172.21.15.63:6789/0 conn(0x7fcd0806d910 legacy=0x7fcd080d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:54.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.615+0000 7fcd06ffd640 1 -- 172.21.15.5:0/2409319059 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd080d3ca0 con 0x7fcd080b3e80 2024-05-20T18:32:54.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.616+0000 7fcd06ffd640 1 --2- 172.21.15.5:0/2409319059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd080b3e80 0x7fcd0806d3d0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fcd080b7740 tx=0x7fccf002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d149ee2f54d6fd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:54.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.616+0000 7fcd067fc640 1 -- 172.21.15.5:0/2409319059 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf0037070 con 0x7fcd080b3e80 2024-05-20T18:32:54.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.616+0000 7fcd067fc640 1 -- 172.21.15.5:0/2409319059 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccf0032440 con 0x7fcd080b3e80 2024-05-20T18:32:54.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.616+0000 7fcd067fc640 1 -- 172.21.15.5:0/2409319059 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf0032810 con 0x7fcd080b3e80 2024-05-20T18:32:54.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.617+0000 7fcd0e099640 1 -- 172.21.15.5:0/2409319059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd080b3e80 msgr2=0x7fcd0806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:54.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.617+0000 7fcd0e099640 1 --2- 172.21.15.5:0/2409319059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd080b3e80 0x7fcd0806d3d0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fcd080b7740 tx=0x7fccf002a270 comp rx=0 tx=0).stop 2024-05-20T18:32:54.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.617+0000 7fcd0e099640 1 -- 172.21.15.5:0/2409319059 shutdown_connections 2024-05-20T18:32:54.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.617+0000 7fcd0e099640 1 --2- 172.21.15.5:0/2409319059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd080b3e80 0x7fcd0806d3d0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:54.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.617+0000 7fcd0e099640 1 --2- 172.21.15.5:0/2409319059 >> v2:172.21.15.63:3300/0 conn(0x7fcd08162820 0x7fcd080b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:54.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.617+0000 7fcd0e099640 1 -- 172.21.15.5:0/2409319059 >> 172.21.15.5:0/2409319059 conn(0x7fcd080d4bd0 msgr2=0x7fcd080d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:54.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.617+0000 7fcd0e099640 1 -- 172.21.15.5:0/2409319059 shutdown_connections 2024-05-20T18:32:54.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.617+0000 7fcd0e099640 1 -- 172.21.15.5:0/2409319059 wait complete. 2024-05-20T18:32:54.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.619+0000 7fcd0e099640 1 Processor -- start 2024-05-20T18:32:54.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.619+0000 7fcd0e099640 1 -- start start 2024-05-20T18:32:54.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.620+0000 7fcd0e099640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd0806d910 0x7fcd081539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:54.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.620+0000 7fcd0e099640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd080b3e80 0x7fcd080c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:54.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.621+0000 7fcd0e099640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd08162820 0x7fcd080b6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:54.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.621+0000 7fcd0e099640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcd080d0300 con 0x7fcd08162820 2024-05-20T18:32:54.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.621+0000 7fcd0e099640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcd080d4260 con 0x7fcd0806d910 2024-05-20T18:32:54.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.621+0000 7fcd0e099640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcd080b91a0 con 0x7fcd080b3e80 2024-05-20T18:32:54.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.621+0000 7fcd07fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd08162820 0x7fcd080b6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:54.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.621+0000 7fcd077fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd0806d910 0x7fcd081539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:54.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.621+0000 7fcd07fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd08162820 0x7fcd080b6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37578/0 (socket says 172.21.15.5:37578) 2024-05-20T18:32:54.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.621+0000 7fcd07fff640 1 -- 172.21.15.5:0/3295680300 learned_addr learned my addr 172.21.15.5:0/3295680300 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:54.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.621+0000 7fcd06ffd640 1 --2- 172.21.15.5:0/3295680300 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd080b3e80 0x7fcd080c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:54.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.621+0000 7fcd07fff640 1 -- 172.21.15.5:0/3295680300 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd080b3e80 msgr2=0x7fcd080c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:54.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.621+0000 7fcd07fff640 1 --2- 172.21.15.5:0/3295680300 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd080b3e80 0x7fcd080c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:54.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.621+0000 7fcd07fff640 1 -- 172.21.15.5:0/3295680300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd0806d910 msgr2=0x7fcd081539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:54.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.621+0000 7fcd07fff640 1 --2- 172.21.15.5:0/3295680300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd0806d910 0x7fcd081539c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:54.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.621+0000 7fcd07fff640 1 -- 172.21.15.5:0/3295680300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd0815ac30 con 0x7fcd08162820 2024-05-20T18:32:54.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.621+0000 7fcd07fff640 1 --2- 172.21.15.5:0/3295680300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd08162820 0x7fcd080b6810 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000c390 tx=0x7fcd0000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:54.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.621+0000 7fccfffff640 1 -- 172.21.15.5:0/3295680300 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd00017070 con 0x7fcd08162820 2024-05-20T18:32:54.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.621+0000 7fccfffff640 1 -- 172.21.15.5:0/3295680300 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd000052d0 con 0x7fcd08162820 2024-05-20T18:32:54.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.621+0000 7fccfffff640 1 -- 172.21.15.5:0/3295680300 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd00003810 con 0x7fcd08162820 2024-05-20T18:32:54.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.621+0000 7fcd0e099640 1 -- 172.21.15.5:0/3295680300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd0815af10 con 0x7fcd08162820 2024-05-20T18:32:54.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.621+0000 7fcd0e099640 1 -- 172.21.15.5:0/3295680300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd080b5e20 con 0x7fcd08162820 2024-05-20T18:32:54.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.623+0000 7fcd0e099640 1 -- 172.21.15.5:0/3295680300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd080d12c0 con 0x7fcd08162820 2024-05-20T18:32:54.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.623+0000 7fccfffff640 1 -- 172.21.15.5:0/3295680300 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcd0001a070 con 0x7fcd08162820 2024-05-20T18:32:54.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.624+0000 7fccfffff640 1 --2- 172.21.15.5:0/3295680300 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcce806be30 0x7fcce806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:54.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.624+0000 7fcd077fe640 1 --2- 172.21.15.5:0/3295680300 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcce806be30 0x7fcce806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:54.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.624+0000 7fccfffff640 1 -- 172.21.15.5:0/3295680300 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7fcd0008a2d0 con 0x7fcd08162820 2024-05-20T18:32:54.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.624+0000 7fcd077fe640 1 --2- 172.21.15.5:0/3295680300 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcce806be30 0x7fcce806e340 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fccf8011340 tx=0x7fccf800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:54.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.629+0000 7fccfffff640 1 -- 172.21.15.5:0/3295680300 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fcd080d12c0 con 0x7fcd08162820 2024-05-20T18:32:54.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.812+0000 7fcd0e099640 1 -- 172.21.15.5:0/3295680300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcd08075b00 con 0x7fcd08162820 2024-05-20T18:32:54.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.812+0000 7fccfffff640 1 -- 172.21.15.5:0/3295680300 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7fcd0806dd20 con 0x7fcd08162820 2024-05-20T18:32:54.813 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:32:54.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.814+0000 7fcd0e099640 1 -- 172.21.15.5:0/3295680300 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcce806be30 msgr2=0x7fcce806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:54.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.814+0000 7fcd0e099640 1 --2- 172.21.15.5:0/3295680300 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcce806be30 0x7fcce806e340 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fccf8011340 tx=0x7fccf800b040 comp rx=0 tx=0).stop 2024-05-20T18:32:54.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.814+0000 7fcd0e099640 1 -- 172.21.15.5:0/3295680300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd08162820 msgr2=0x7fcd080b6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:54.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.814+0000 7fcd0e099640 1 --2- 172.21.15.5:0/3295680300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd08162820 0x7fcd080b6810 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000c390 tx=0x7fcd0000c850 comp rx=0 tx=0).stop 2024-05-20T18:32:54.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.814+0000 7fcd0e099640 1 -- 172.21.15.5:0/3295680300 shutdown_connections 2024-05-20T18:32:54.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.814+0000 7fcd0e099640 1 --2- 172.21.15.5:0/3295680300 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcce806be30 0x7fcce806e340 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:54.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.814+0000 7fcd0e099640 1 --2- 172.21.15.5:0/3295680300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd08162820 0x7fcd080b6810 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:54.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.814+0000 7fcd0e099640 1 --2- 172.21.15.5:0/3295680300 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd080b3e80 0x7fcd080c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:54.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.814+0000 7fcd0e099640 1 --2- 172.21.15.5:0/3295680300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd0806d910 0x7fcd081539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:54.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.814+0000 7fcd0e099640 1 -- 172.21.15.5:0/3295680300 >> 172.21.15.5:0/3295680300 conn(0x7fcd080d4bd0 msgr2=0x7fcd080ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:54.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.814+0000 7fcd0e099640 1 -- 172.21.15.5:0/3295680300 shutdown_connections 2024-05-20T18:32:54.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.814+0000 7fcd0e099640 1 -- 172.21.15.5:0/3295680300 wait complete. 2024-05-20T18:32:54.826 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:32:54.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.943+0000 7f16c3dcc640 1 Processor -- start 2024-05-20T18:32:54.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.943+0000 7f16c3dcc640 1 -- start start 2024-05-20T18:32:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.944+0000 7f16c3dcc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f16bc0d1bc0 0x7f16bc0d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.944+0000 7f16c3dcc640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16bc0d0370 con 0x7f16bc0b4010 2024-05-20T18:32:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.944+0000 7f16c3dcc640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16bc0d0650 con 0x7f16bc163a10 2024-05-20T18:32:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.944+0000 7f16c3dcc640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f16bc0d3ca0 con 0x7f16bc0d1bc0 2024-05-20T18:32:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.945+0000 7f16c1367640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f16bc0b4010 0x7f16bc0d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46802/0 (socket says 172.21.15.5:46802) 2024-05-20T18:32:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.945+0000 7f16c2369640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f16bc0d1bc0 0x7f16bc0d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.945+0000 7f16c1367640 1 -- 172.21.15.5:0/793406851 learned_addr learned my addr 172.21.15.5:0/793406851 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:32:54.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.945+0000 7f16c2369640 1 -- 172.21.15.5:0/793406851 >> v1:172.21.15.63:6789/0 conn(0x7f16bc163a10 legacy=0x7f16bc0cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:54.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.945+0000 7f16c2369640 1 -- 172.21.15.5:0/793406851 >> v1:172.21.15.5:6789/0 conn(0x7f16bc0b4010 legacy=0x7f16bc0d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:54.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.945+0000 7f16c2369640 1 -- 172.21.15.5:0/793406851 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16bc0d3650 con 0x7f16bc0d1bc0 2024-05-20T18:32:54.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.946+0000 7f16c2369640 1 --2- 172.21.15.5:0/793406851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16bc0d1bc0 0x7f16bc0d4130 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f16b4009d90 tx=0x7f16b402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a784837cbb997082 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:54.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.946+0000 7f16c0b66640 1 -- 172.21.15.5:0/793406851 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b4007740 con 0x7f16bc0d1bc0 2024-05-20T18:32:54.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.946+0000 7f16c0b66640 1 -- 172.21.15.5:0/793406851 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16b40078e0 con 0x7f16bc0d1bc0 2024-05-20T18:32:54.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.946+0000 7f16c0b66640 1 -- 172.21.15.5:0/793406851 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b4007cb0 con 0x7f16bc0d1bc0 2024-05-20T18:32:54.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.946+0000 7f16c3dcc640 1 -- 172.21.15.5:0/793406851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16bc0d1bc0 msgr2=0x7f16bc0d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:54.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.946+0000 7f16c3dcc640 1 --2- 172.21.15.5:0/793406851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16bc0d1bc0 0x7f16bc0d4130 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f16b4009d90 tx=0x7f16b402a6f0 comp rx=0 tx=0).stop 2024-05-20T18:32:54.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.946+0000 7f16c3dcc640 1 -- 172.21.15.5:0/793406851 shutdown_connections 2024-05-20T18:32:54.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.946+0000 7f16c3dcc640 1 --2- 172.21.15.5:0/793406851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16bc0d1bc0 0x7f16bc0d4130 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:54.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.946+0000 7f16c3dcc640 1 -- 172.21.15.5:0/793406851 >> 172.21.15.5:0/793406851 conn(0x7f16bc06d240 msgr2=0x7f16bc06d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:54.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.947+0000 7f16c3dcc640 1 -- 172.21.15.5:0/793406851 shutdown_connections 2024-05-20T18:32:54.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.947+0000 7f16c3dcc640 1 -- 172.21.15.5:0/793406851 wait complete. 2024-05-20T18:32:54.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.948+0000 7f16c3dcc640 1 Processor -- start 2024-05-20T18:32:54.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.948+0000 7f16c3dcc640 1 -- start start 2024-05-20T18:32:54.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.950+0000 7f16c3dcc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16bc0b4010 0x7f16bc0b4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:54.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.950+0000 7f16c3dcc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16bc0d1bc0 0x7f16bc153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:54.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.950+0000 7f16c3dcc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16bc163a10 0x7f16bc160120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:54.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.950+0000 7f16c3dcc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc0b8390 con 0x7f16bc0b4010 2024-05-20T18:32:54.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.950+0000 7f16c3dcc640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc0b8210 con 0x7f16bc0d1bc0 2024-05-20T18:32:54.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.950+0000 7f16c3dcc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f16bc0b8510 con 0x7f16bc163a10 2024-05-20T18:32:54.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.950+0000 7f16c1b68640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16bc0b4010 0x7f16bc0b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:54.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.950+0000 7f16c1b68640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16bc0b4010 0x7f16bc0b4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44540/0 (socket says 172.21.15.5:44540) 2024-05-20T18:32:54.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.950+0000 7f16c1b68640 1 -- 172.21.15.5:0/1606252889 learned_addr learned my addr 172.21.15.5:0/1606252889 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:54.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.950+0000 7f16c2369640 1 --2- 172.21.15.5:0/1606252889 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16bc163a10 0x7f16bc160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:54.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.950+0000 7f16c1367640 1 --2- 172.21.15.5:0/1606252889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16bc0d1bc0 0x7f16bc153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:54.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.950+0000 7f16c1b68640 1 -- 172.21.15.5:0/1606252889 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16bc163a10 msgr2=0x7f16bc160120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:54.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.950+0000 7f16c1b68640 1 --2- 172.21.15.5:0/1606252889 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16bc163a10 0x7f16bc160120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:54.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.950+0000 7f16c1b68640 1 -- 172.21.15.5:0/1606252889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16bc0d1bc0 msgr2=0x7f16bc153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:54.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.950+0000 7f16c1b68640 1 --2- 172.21.15.5:0/1606252889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16bc0d1bc0 0x7f16bc153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:54.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.950+0000 7f16c1b68640 1 -- 172.21.15.5:0/1606252889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16bc1619c0 con 0x7f16bc0b4010 2024-05-20T18:32:54.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.950+0000 7f16c2369640 1 --2- 172.21.15.5:0/1606252889 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16bc163a10 0x7f16bc160120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:32:54.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.950+0000 7f16c1367640 1 --2- 172.21.15.5:0/1606252889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16bc0d1bc0 0x7f16bc153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:32:54.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.950+0000 7f16c1b68640 1 --2- 172.21.15.5:0/1606252889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16bc0b4010 0x7f16bc0b4620 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f16b800c440 tx=0x7f16b800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:54.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.950+0000 7f16b27fc640 1 -- 172.21.15.5:0/1606252889 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b80052d0 con 0x7f16bc0b4010 2024-05-20T18:32:54.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.950+0000 7f16b27fc640 1 -- 172.21.15.5:0/1606252889 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16b8005470 con 0x7f16bc0b4010 2024-05-20T18:32:54.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.950+0000 7f16c3dcc640 1 -- 172.21.15.5:0/1606252889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16bc161ca0 con 0x7f16bc0b4010 2024-05-20T18:32:54.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.950+0000 7f16b27fc640 1 -- 172.21.15.5:0/1606252889 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b8007260 con 0x7f16bc0b4010 2024-05-20T18:32:54.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.950+0000 7f16c3dcc640 1 -- 172.21.15.5:0/1606252889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16bc155000 con 0x7f16bc0b4010 2024-05-20T18:32:54.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.951+0000 7f16c3dcc640 1 -- 172.21.15.5:0/1606252889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16bc075b00 con 0x7f16bc0b4010 2024-05-20T18:32:54.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.952+0000 7f16b27fc640 1 -- 172.21.15.5:0/1606252889 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f16b8007400 con 0x7f16bc0b4010 2024-05-20T18:32:54.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.953+0000 7f16b27fc640 1 --2- 172.21.15.5:0/1606252889 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f16a006be30 0x7f16a006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:54.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.953+0000 7f16b27fc640 1 -- 172.21.15.5:0/1606252889 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f16b808a4b0 con 0x7f16bc0b4010 2024-05-20T18:32:54.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.953+0000 7f16c1367640 1 --2- 172.21.15.5:0/1606252889 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f16a006be30 0x7f16a006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:54.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.953+0000 7f16c1367640 1 --2- 172.21.15.5:0/1606252889 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f16a006be30 0x7f16a006e340 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f16ac007fd0 tx=0x7f16ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:54.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:54.957+0000 7f16b27fc640 1 -- 172.21.15.5:0/1606252889 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f16b8058c80 con 0x7f16bc0b4010 2024-05-20T18:32:55.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.138+0000 7f16c3dcc640 1 -- 172.21.15.5:0/1606252889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f16bc161e70 con 0x7f16bc0b4010 2024-05-20T18:32:55.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.138+0000 7f16b27fc640 1 -- 172.21.15.5:0/1606252889 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) ==== 104+0+11 (secure 0 0 0) 0x7f16b8058580 con 0x7f16bc0b4010 2024-05-20T18:32:55.139 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:32:55.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.140+0000 7f16c3dcc640 1 -- 172.21.15.5:0/1606252889 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f16a006be30 msgr2=0x7f16a006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:55.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.140+0000 7f16c3dcc640 1 --2- 172.21.15.5:0/1606252889 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f16a006be30 0x7f16a006e340 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f16ac007fd0 tx=0x7f16ac009040 comp rx=0 tx=0).stop 2024-05-20T18:32:55.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.140+0000 7f16c3dcc640 1 -- 172.21.15.5:0/1606252889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16bc0b4010 msgr2=0x7f16bc0b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:55.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.140+0000 7f16c3dcc640 1 --2- 172.21.15.5:0/1606252889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16bc0b4010 0x7f16bc0b4620 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f16b800c440 tx=0x7f16b800c900 comp rx=0 tx=0).stop 2024-05-20T18:32:55.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.140+0000 7f16c3dcc640 1 -- 172.21.15.5:0/1606252889 shutdown_connections 2024-05-20T18:32:55.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.140+0000 7f16c3dcc640 1 --2- 172.21.15.5:0/1606252889 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f16a006be30 0x7f16a006e340 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:55.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.140+0000 7f16c3dcc640 1 --2- 172.21.15.5:0/1606252889 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16bc163a10 0x7f16bc160120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:55.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.140+0000 7f16c3dcc640 1 --2- 172.21.15.5:0/1606252889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16bc0d1bc0 0x7f16bc153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:55.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.140+0000 7f16c3dcc640 1 --2- 172.21.15.5:0/1606252889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16bc0b4010 0x7f16bc0b4620 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:55.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.140+0000 7f16c3dcc640 1 -- 172.21.15.5:0/1606252889 >> 172.21.15.5:0/1606252889 conn(0x7f16bc06d240 msgr2=0x7f16bc15f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:55.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.140+0000 7f16c3dcc640 1 -- 172.21.15.5:0/1606252889 shutdown_connections 2024-05-20T18:32:55.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.140+0000 7f16c3dcc640 1 -- 172.21.15.5:0/1606252889 wait complete. 2024-05-20T18:32:55.152 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:32:55.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.269+0000 7f022ae1b640 1 Processor -- start 2024-05-20T18:32:55.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.269+0000 7f022ae1b640 1 -- start start 2024-05-20T18:32:55.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.270+0000 7f022ae1b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0224163a10 0x7f02240d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:55.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.270+0000 7f022ae1b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f02240b4010 0x7f02240cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:55.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.270+0000 7f022ae1b640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02240d0370 con 0x7f02240d14b0 2024-05-20T18:32:55.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.270+0000 7f022ae1b640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f02240d3ca0 con 0x7f02240b4010 2024-05-20T18:32:55.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.270+0000 7f022ae1b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f02240d0650 con 0x7f0224163a10 2024-05-20T18:32:55.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.270+0000 7f02293b8640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f02240d14b0 0x7f02240d18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46808/0 (socket says 172.21.15.5:46808) 2024-05-20T18:32:55.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.270+0000 7f02293b8640 1 -- 172.21.15.5:0/3364294175 learned_addr learned my addr 172.21.15.5:0/3364294175 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:32:55.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.270+0000 7f0228bb7640 1 --2- 172.21.15.5:0/3364294175 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0224163a10 0x7f02240d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:55.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.270+0000 7f0223fff640 1 --2- 172.21.15.5:0/3364294175 >> v2:172.21.15.63:3300/0 conn(0x7f02240b4010 0x7f02240cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:55.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.271+0000 7f02237fe640 1 -- 172.21.15.5:0/3364294175 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1940058929 0 0) 0x7f02240d0370 con 0x7f02240d14b0 2024-05-20T18:32:55.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.271+0000 7f02237fe640 1 -- 172.21.15.5:0/3364294175 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0210003520 con 0x7f02240d14b0 2024-05-20T18:32:55.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.271+0000 7f02237fe640 1 -- 172.21.15.5:0/3364294175 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1460094805 0 0) 0x7f0210003520 con 0x7f02240d14b0 2024-05-20T18:32:55.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.271+0000 7f02237fe640 1 -- 172.21.15.5:0/3364294175 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0224163a10 msgr2=0x7f02240d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:55.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.271+0000 7f02237fe640 1 --2- 172.21.15.5:0/3364294175 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0224163a10 0x7f02240d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:55.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.271+0000 7f02237fe640 1 -- 172.21.15.5:0/3364294175 >> v2:172.21.15.63:3300/0 conn(0x7f02240b4010 msgr2=0x7f02240cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:55.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.271+0000 7f02237fe640 1 --2- 172.21.15.5:0/3364294175 >> v2:172.21.15.63:3300/0 conn(0x7f02240b4010 0x7f02240cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:55.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.271+0000 7f02237fe640 1 -- 172.21.15.5:0/3364294175 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02240d3650 con 0x7f02240d14b0 2024-05-20T18:32:55.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.271+0000 7f02237fe640 1 -- 172.21.15.5:0/3364294175 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f021c001670 con 0x7f02240d14b0 2024-05-20T18:32:55.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.271+0000 7f0223fff640 1 --2- 172.21.15.5:0/3364294175 >> v2:172.21.15.63:3300/0 conn(0x7f02240b4010 0x7f02240cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:32:55.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.271+0000 7f02237fe640 1 -- 172.21.15.5:0/3364294175 >> v1:172.21.15.5:6789/0 conn(0x7f02240d14b0 legacy=0x7f02240d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:55.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.271+0000 7f02237fe640 1 --2- 172.21.15.5:0/3364294175 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02100039b0 0x7f0210003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:55.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.271+0000 7f02237fe640 1 --2- 172.21.15.5:0/3364294175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0210004800 0x7f0210004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:55.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.271+0000 7f02237fe640 1 --2- 172.21.15.5:0/3364294175 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0210005150 0x7f021000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:55.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.271+0000 7f02237fe640 1 -- 172.21.15.5:0/3364294175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f02240d3ca0 con 0x7f0210004800 2024-05-20T18:32:55.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.271+0000 7f02237fe640 1 -- 172.21.15.5:0/3364294175 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f02240d0650 con 0x7f0210005150 2024-05-20T18:32:55.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.271+0000 7f02237fe640 1 -- 172.21.15.5:0/3364294175 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f021000bf80 con 0x7f02100039b0 2024-05-20T18:32:55.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.271+0000 7f0228bb7640 1 --2- 172.21.15.5:0/3364294175 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0224163a10 0x7f02240d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:32:55.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.271+0000 7f0223fff640 1 --2- 172.21.15.5:0/3364294175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0210004800 0x7f0210004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:55.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.271+0000 7f02293b8640 1 --2- 172.21.15.5:0/3364294175 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0210005150 0x7f021000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:55.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.271+0000 7f0228bb7640 1 --2- 172.21.15.5:0/3364294175 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02100039b0 0x7f0210003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:55.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.271+0000 7f0223fff640 1 -- 172.21.15.5:0/3364294175 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02100039b0 msgr2=0x7f0210003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:55.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.271+0000 7f0223fff640 1 --2- 172.21.15.5:0/3364294175 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02100039b0 0x7f0210003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:55.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.271+0000 7f0223fff640 1 -- 172.21.15.5:0/3364294175 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0210005150 msgr2=0x7f021000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:55.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.271+0000 7f0223fff640 1 --2- 172.21.15.5:0/3364294175 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0210005150 0x7f021000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:55.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.271+0000 7f0223fff640 1 -- 172.21.15.5:0/3364294175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0210003520 con 0x7f0210004800 2024-05-20T18:32:55.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.272+0000 7f0223fff640 1 --2- 172.21.15.5:0/3364294175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0210004800 0x7f0210004c10 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f02240b7f80 tx=0x7f0210005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ef2cf64526e4ff server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:55.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.272+0000 7f02237fe640 1 -- 172.21.15.5:0/3364294175 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0214033070 con 0x7f0210004800 2024-05-20T18:32:55.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.272+0000 7f02237fe640 1 -- 172.21.15.5:0/3364294175 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f021402f630 con 0x7f0210004800 2024-05-20T18:32:55.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.272+0000 7f02237fe640 1 -- 172.21.15.5:0/3364294175 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0214009de0 con 0x7f0210004800 2024-05-20T18:32:55.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.273+0000 7f022ae1b640 1 -- 172.21.15.5:0/3364294175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0210004800 msgr2=0x7f0210004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:55.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.273+0000 7f022ae1b640 1 --2- 172.21.15.5:0/3364294175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0210004800 0x7f0210004c10 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f02240b7f80 tx=0x7f0210005830 comp rx=0 tx=0).stop 2024-05-20T18:32:55.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.273+0000 7f02293b8640 1 -- 172.21.15.5:0/3364294175 reap_dead start 2024-05-20T18:32:55.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.273+0000 7f022ae1b640 1 -- 172.21.15.5:0/3364294175 shutdown_connections 2024-05-20T18:32:55.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.273+0000 7f022ae1b640 1 -- 172.21.15.5:0/3364294175 >> 172.21.15.5:0/3364294175 conn(0x7f022406d240 msgr2=0x7f022406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:55.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.273+0000 7f022ae1b640 1 -- 172.21.15.5:0/3364294175 shutdown_connections 2024-05-20T18:32:55.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.273+0000 7f022ae1b640 1 -- 172.21.15.5:0/3364294175 wait complete. 2024-05-20T18:32:55.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.275+0000 7f022ae1b640 1 Processor -- start 2024-05-20T18:32:55.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.275+0000 7f022ae1b640 1 -- start start 2024-05-20T18:32:55.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.276+0000 7f022ae1b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0210004800 0x7f02240b5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:55.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.276+0000 7f022ae1b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02240ce2a0 0x7f02240b5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:55.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.276+0000 7f022ae1b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02240b60f0 0x7f0224155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:55.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.276+0000 7f022ae1b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f02240d4130 con 0x7f02240ce2a0 2024-05-20T18:32:55.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.276+0000 7f022ae1b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f02240d0040 con 0x7f0210004800 2024-05-20T18:32:55.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.276+0000 7f022ae1b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f02240d42b0 con 0x7f02240b60f0 2024-05-20T18:32:55.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.276+0000 7f0223fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02240ce2a0 0x7f02240b5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:55.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.276+0000 7f0223fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02240ce2a0 0x7f02240b5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44572/0 (socket says 172.21.15.5:44572) 2024-05-20T18:32:55.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.276+0000 7f0223fff640 1 -- 172.21.15.5:0/3427734753 learned_addr learned my addr 172.21.15.5:0/3427734753 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:55.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.277+0000 7f0228bb7640 1 --2- 172.21.15.5:0/3427734753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0210004800 0x7f02240b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:55.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.277+0000 7f02293b8640 1 --2- 172.21.15.5:0/3427734753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02240b60f0 0x7f0224155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:55.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.277+0000 7f0223fff640 1 -- 172.21.15.5:0/3427734753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02240b60f0 msgr2=0x7f0224155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:55.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.277+0000 7f0223fff640 1 --2- 172.21.15.5:0/3427734753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02240b60f0 0x7f0224155ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:55.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.277+0000 7f0223fff640 1 -- 172.21.15.5:0/3427734753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0210004800 msgr2=0x7f02240b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:55.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.277+0000 7f0223fff640 1 --2- 172.21.15.5:0/3427734753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0210004800 0x7f02240b5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:55.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.277+0000 7f0223fff640 1 -- 172.21.15.5:0/3427734753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f022415b1f0 con 0x7f02240ce2a0 2024-05-20T18:32:55.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.277+0000 7f0223fff640 1 --2- 172.21.15.5:0/3427734753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02240ce2a0 0x7f02240b5bb0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f0224068160 tx=0x7f0214009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:55.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.277+0000 7f02217fa640 1 -- 172.21.15.5:0/3427734753 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0214046070 con 0x7f02240ce2a0 2024-05-20T18:32:55.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.277+0000 7f02217fa640 1 -- 172.21.15.5:0/3427734753 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f021402f430 con 0x7f02240ce2a0 2024-05-20T18:32:55.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.277+0000 7f02217fa640 1 -- 172.21.15.5:0/3427734753 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0214033040 con 0x7f02240ce2a0 2024-05-20T18:32:55.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.277+0000 7f022ae1b640 1 -- 172.21.15.5:0/3427734753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02241551c0 con 0x7f02240ce2a0 2024-05-20T18:32:55.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.277+0000 7f022ae1b640 1 -- 172.21.15.5:0/3427734753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0224155780 con 0x7f02240ce2a0 2024-05-20T18:32:55.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.278+0000 7f022ae1b640 1 -- 172.21.15.5:0/3427734753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02240d0930 con 0x7f02240ce2a0 2024-05-20T18:32:55.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.279+0000 7f02217fa640 1 -- 172.21.15.5:0/3427734753 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0214049070 con 0x7f02240ce2a0 2024-05-20T18:32:55.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.279+0000 7f02217fa640 1 --2- 172.21.15.5:0/3427734753 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f020406be30 0x7f020406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:55.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.279+0000 7f02217fa640 1 -- 172.21.15.5:0/3427734753 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f02140abf10 con 0x7f02240ce2a0 2024-05-20T18:32:55.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.279+0000 7f0228bb7640 1 --2- 172.21.15.5:0/3427734753 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f020406be30 0x7f020406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:55.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.280+0000 7f0228bb7640 1 --2- 172.21.15.5:0/3427734753 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f020406be30 0x7f020406e340 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f02240cf1c0 tx=0x7f020c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:55.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.284+0000 7f02217fa640 1 -- 172.21.15.5:0/3427734753 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f02240d0930 con 0x7f02240ce2a0 2024-05-20T18:32:55.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.467+0000 7f022ae1b640 1 -- 172.21.15.5:0/3427734753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0224154fb0 con 0x7f02240ce2a0 2024-05-20T18:32:55.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.467+0000 7f02217fa640 1 -- 172.21.15.5:0/3427734753 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) ==== 100+0+11 (secure 0 0 0) 0x7f021402f600 con 0x7f02240ce2a0 2024-05-20T18:32:55.468 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:32:55.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.469+0000 7f022ae1b640 1 -- 172.21.15.5:0/3427734753 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f020406be30 msgr2=0x7f020406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:55.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.469+0000 7f022ae1b640 1 --2- 172.21.15.5:0/3427734753 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f020406be30 0x7f020406e340 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f02240cf1c0 tx=0x7f020c008040 comp rx=0 tx=0).stop 2024-05-20T18:32:55.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.469+0000 7f022ae1b640 1 -- 172.21.15.5:0/3427734753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02240ce2a0 msgr2=0x7f02240b5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:55.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.469+0000 7f022ae1b640 1 --2- 172.21.15.5:0/3427734753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02240ce2a0 0x7f02240b5bb0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f0224068160 tx=0x7f0214009750 comp rx=0 tx=0).stop 2024-05-20T18:32:55.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.469+0000 7f022ae1b640 1 -- 172.21.15.5:0/3427734753 shutdown_connections 2024-05-20T18:32:55.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.469+0000 7f022ae1b640 1 --2- 172.21.15.5:0/3427734753 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f020406be30 0x7f020406e340 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:55.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.469+0000 7f022ae1b640 1 --2- 172.21.15.5:0/3427734753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02240b60f0 0x7f0224155ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:55.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.469+0000 7f022ae1b640 1 --2- 172.21.15.5:0/3427734753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02240ce2a0 0x7f02240b5bb0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:55.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.469+0000 7f022ae1b640 1 --2- 172.21.15.5:0/3427734753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0210004800 0x7f02240b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:55.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.469+0000 7f022ae1b640 1 -- 172.21.15.5:0/3427734753 >> 172.21.15.5:0/3427734753 conn(0x7f022406d240 msgr2=0x7f02241622f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:55.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.469+0000 7f022ae1b640 1 -- 172.21.15.5:0/3427734753 shutdown_connections 2024-05-20T18:32:55.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.469+0000 7f022ae1b640 1 -- 172.21.15.5:0/3427734753 wait complete. 2024-05-20T18:32:55.481 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T18:32:55.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.596+0000 7f69d6b45640 1 Processor -- start 2024-05-20T18:32:55.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.596+0000 7f69d6b45640 1 -- start start 2024-05-20T18:32:55.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.598+0000 7f69d6b45640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69d0162820 0x7f69d00b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:55.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.598+0000 7f69d6b45640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f69d00b3e80 0x7f69d006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:55.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.598+0000 7f69d6b45640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f69d006d910 0x7f69d00d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:55.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.598+0000 7f69d6b45640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f69d006dd20 con 0x7f69d00b3e80 2024-05-20T18:32:55.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.598+0000 7f69d6b45640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f69d00d0630 con 0x7f69d006d910 2024-05-20T18:32:55.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.598+0000 7f69d6b45640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f69d00d39c0 con 0x7f69d0162820 2024-05-20T18:32:55.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.598+0000 7f69cffff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f69d00b3e80 0x7f69d006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:55.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.598+0000 7f69cffff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f69d00b3e80 0x7f69d006d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44582/0 (socket says 172.21.15.5:44582) 2024-05-20T18:32:55.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.598+0000 7f69cffff640 1 -- 172.21.15.5:0/2955420903 learned_addr learned my addr 172.21.15.5:0/2955420903 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:55.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.598+0000 7f69d48e1640 1 --2- 172.21.15.5:0/2955420903 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69d0162820 0x7f69d00b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:55.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.598+0000 7f69d50e2640 1 --2- 172.21.15.5:0/2955420903 >> v2:172.21.15.63:3300/0 conn(0x7f69d006d910 0x7f69d00d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:55.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.598+0000 7f69cffff640 1 -- 172.21.15.5:0/2955420903 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69d0162820 msgr2=0x7f69d00b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:55.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.598+0000 7f69cffff640 1 --2- 172.21.15.5:0/2955420903 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69d0162820 0x7f69d00b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:55.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.598+0000 7f69cffff640 1 -- 172.21.15.5:0/2955420903 >> v2:172.21.15.63:3300/0 conn(0x7f69d006d910 msgr2=0x7f69d00d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:55.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.598+0000 7f69cffff640 1 --2- 172.21.15.5:0/2955420903 >> v2:172.21.15.63:3300/0 conn(0x7f69d006d910 0x7f69d00d27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:55.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.598+0000 7f69cffff640 1 -- 172.21.15.5:0/2955420903 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69d00d3ca0 con 0x7f69d00b3e80 2024-05-20T18:32:55.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.599+0000 7f69cffff640 1 --2- 172.21.15.5:0/2955420903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69d00b3e80 0x7f69d006d3d0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f69d006ce40 tx=0x7f69c402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e3e02ddd975ecf server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:55.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.599+0000 7f69cf7fe640 1 -- 172.21.15.5:0/2955420903 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c4037070 con 0x7f69d00b3e80 2024-05-20T18:32:55.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.599+0000 7f69cf7fe640 1 -- 172.21.15.5:0/2955420903 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69c4032440 con 0x7f69d00b3e80 2024-05-20T18:32:55.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.599+0000 7f69cf7fe640 1 -- 172.21.15.5:0/2955420903 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c4032810 con 0x7f69d00b3e80 2024-05-20T18:32:55.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.600+0000 7f69d6b45640 1 -- 172.21.15.5:0/2955420903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69d00b3e80 msgr2=0x7f69d006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:55.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.600+0000 7f69d6b45640 1 --2- 172.21.15.5:0/2955420903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69d00b3e80 0x7f69d006d3d0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f69d006ce40 tx=0x7f69c402a1e0 comp rx=0 tx=0).stop 2024-05-20T18:32:55.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.600+0000 7f69d6b45640 1 -- 172.21.15.5:0/2955420903 shutdown_connections 2024-05-20T18:32:55.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.600+0000 7f69d6b45640 1 --2- 172.21.15.5:0/2955420903 >> v2:172.21.15.63:3300/0 conn(0x7f69d006d910 0x7f69d00d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:55.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.600+0000 7f69d6b45640 1 --2- 172.21.15.5:0/2955420903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69d00b3e80 0x7f69d006d3d0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:55.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.600+0000 7f69d6b45640 1 --2- 172.21.15.5:0/2955420903 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69d0162820 0x7f69d00b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:55.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.600+0000 7f69d6b45640 1 -- 172.21.15.5:0/2955420903 >> 172.21.15.5:0/2955420903 conn(0x7f69d00d4bd0 msgr2=0x7f69d00d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:55.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.600+0000 7f69d6b45640 1 -- 172.21.15.5:0/2955420903 shutdown_connections 2024-05-20T18:32:55.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.600+0000 7f69d6b45640 1 -- 172.21.15.5:0/2955420903 wait complete. 2024-05-20T18:32:55.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.602+0000 7f69d6b45640 1 Processor -- start 2024-05-20T18:32:55.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.602+0000 7f69d6b45640 1 -- start start 2024-05-20T18:32:55.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.603+0000 7f69d6b45640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69d006d910 0x7f69d0157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:55.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.603+0000 7f69d6b45640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69d00b3e80 0x7f69d01f55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:55.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.603+0000 7f69d6b45640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69d0162820 0x7f69d01f5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:55.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.603+0000 7f69d6b45640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f69d00d0300 con 0x7f69d006d910 2024-05-20T18:32:55.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.603+0000 7f69d6b45640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f69d00d4260 con 0x7f69d00b3e80 2024-05-20T18:32:55.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.603+0000 7f69d6b45640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f69d00d4780 con 0x7f69d0162820 2024-05-20T18:32:55.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.603+0000 7f69d48e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69d006d910 0x7f69d0157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:55.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.603+0000 7f69d48e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69d006d910 0x7f69d0157190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44590/0 (socket says 172.21.15.5:44590) 2024-05-20T18:32:55.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.603+0000 7f69d48e1640 1 -- 172.21.15.5:0/45202242 learned_addr learned my addr 172.21.15.5:0/45202242 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:55.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.604+0000 7f69d50e2640 1 --2- 172.21.15.5:0/45202242 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69d0162820 0x7f69d01f5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:55.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.604+0000 7f69cffff640 1 --2- 172.21.15.5:0/45202242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69d00b3e80 0x7f69d01f55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:55.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.604+0000 7f69d48e1640 1 -- 172.21.15.5:0/45202242 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69d0162820 msgr2=0x7f69d01f5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:55.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.604+0000 7f69d48e1640 1 --2- 172.21.15.5:0/45202242 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69d0162820 0x7f69d01f5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:55.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.604+0000 7f69d48e1640 1 -- 172.21.15.5:0/45202242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69d00b3e80 msgr2=0x7f69d01f55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:55.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.604+0000 7f69d48e1640 1 --2- 172.21.15.5:0/45202242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69d00b3e80 0x7f69d01f55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:55.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.604+0000 7f69d48e1640 1 -- 172.21.15.5:0/45202242 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69d01f66a0 con 0x7f69d006d910 2024-05-20T18:32:55.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.604+0000 7f69d48e1640 1 --2- 172.21.15.5:0/45202242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69d006d910 0x7f69d0157190 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f69c000e870 tx=0x7f69c000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:55.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.604+0000 7f69cd7fa640 1 -- 172.21.15.5:0/45202242 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c0017070 con 0x7f69d006d910 2024-05-20T18:32:55.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.604+0000 7f69cd7fa640 1 -- 172.21.15.5:0/45202242 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69c000c940 con 0x7f69d006d910 2024-05-20T18:32:55.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.604+0000 7f69d6b45640 1 -- 172.21.15.5:0/45202242 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69d01f6980 con 0x7f69d006d910 2024-05-20T18:32:55.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.604+0000 7f69cd7fa640 1 -- 172.21.15.5:0/45202242 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c000cd30 con 0x7f69d006d910 2024-05-20T18:32:55.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.604+0000 7f69d6b45640 1 -- 172.21.15.5:0/45202242 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69d01f6f40 con 0x7f69d006d910 2024-05-20T18:32:55.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.605+0000 7f69d6b45640 1 -- 172.21.15.5:0/45202242 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69d0075b00 con 0x7f69d006d910 2024-05-20T18:32:55.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.606+0000 7f69cd7fa640 1 -- 172.21.15.5:0/45202242 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f69c001a070 con 0x7f69d006d910 2024-05-20T18:32:55.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.606+0000 7f69cd7fa640 1 --2- 172.21.15.5:0/45202242 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f69bc06be30 0x7f69bc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:55.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.606+0000 7f69cffff640 1 --2- 172.21.15.5:0/45202242 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f69bc06be30 0x7f69bc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:55.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.606+0000 7f69cd7fa640 1 -- 172.21.15.5:0/45202242 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f69c008af80 con 0x7f69d006d910 2024-05-20T18:32:55.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.607+0000 7f69cffff640 1 --2- 172.21.15.5:0/45202242 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f69bc06be30 0x7f69bc06e340 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f69c4034000 tx=0x7f69c402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:55.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.611+0000 7f69cd7fa640 1 -- 172.21.15.5:0/45202242 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f69c00596c0 con 0x7f69d006d910 2024-05-20T18:32:55.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.808+0000 7f69d6b45640 1 -- 172.21.15.5:0/45202242 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f69d0068a90 con 0x7f69d006d910 2024-05-20T18:32:55.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.809+0000 7f69cd7fa640 1 -- 172.21.15.5:0/45202242 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v78) ==== 80+0+118 (secure 0 0 0) 0x7f69c0058fc0 con 0x7f69d006d910 2024-05-20T18:32:55.809 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6836/791176355 2024-05-21T18:31:40.839373+0000 2024-05-20T18:32:55.810 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6835/791176355 2024-05-21T18:31:40.839373+0000 2024-05-20T18:32:55.810 INFO:teuthology.orchestra.run.smithi005.stderr:listed 2 entries 2024-05-20T18:32:55.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.810+0000 7f69d6b45640 1 -- 172.21.15.5:0/45202242 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f69bc06be30 msgr2=0x7f69bc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:55.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.810+0000 7f69d6b45640 1 --2- 172.21.15.5:0/45202242 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f69bc06be30 0x7f69bc06e340 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f69c4034000 tx=0x7f69c402e000 comp rx=0 tx=0).stop 2024-05-20T18:32:55.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.810+0000 7f69d6b45640 1 -- 172.21.15.5:0/45202242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69d006d910 msgr2=0x7f69d0157190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:55.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.810+0000 7f69d6b45640 1 --2- 172.21.15.5:0/45202242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69d006d910 0x7f69d0157190 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f69c000e870 tx=0x7f69c000ed30 comp rx=0 tx=0).stop 2024-05-20T18:32:55.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.811+0000 7f69d6b45640 1 -- 172.21.15.5:0/45202242 shutdown_connections 2024-05-20T18:32:55.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.811+0000 7f69d6b45640 1 --2- 172.21.15.5:0/45202242 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f69bc06be30 0x7f69bc06e340 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:55.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.811+0000 7f69d6b45640 1 --2- 172.21.15.5:0/45202242 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f69d0162820 0x7f69d01f5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:55.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.811+0000 7f69d6b45640 1 --2- 172.21.15.5:0/45202242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69d00b3e80 0x7f69d01f55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:55.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.811+0000 7f69d6b45640 1 --2- 172.21.15.5:0/45202242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69d006d910 0x7f69d0157190 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:55.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.811+0000 7f69d6b45640 1 -- 172.21.15.5:0/45202242 >> 172.21.15.5:0/45202242 conn(0x7f69d00d4bd0 msgr2=0x7f69d00b8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:55.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.811+0000 7f69d6b45640 1 -- 172.21.15.5:0/45202242 shutdown_connections 2024-05-20T18:32:55.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:55.811+0000 7f69d6b45640 1 -- 172.21.15.5:0/45202242 wait complete. 2024-05-20T18:32:55.822 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-20T18:32:55.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:32:55.823 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-20T18:32:55.851 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:32:55.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:32:55.852 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-20T18:32:55.907 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T18:32:55.908 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:32:55.908 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:32:55.929 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19732 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T18:32:55.929 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T18:32:55.929 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:32:55.929 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:32:55.963 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15663 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T18:32:55.963 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T18:32:55.964 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:32:55.964 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:32:55.989 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19732 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T18:32:55.989 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:32:56.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.117+0000 7f7f81d9c640 1 Processor -- start 2024-05-20T18:32:56.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.117+0000 7f7f81d9c640 1 -- start start 2024-05-20T18:32:56.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.119+0000 7f7f81d9c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f7c0d10e0 0x7f7f7c0cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:56.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.119+0000 7f7f81d9c640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f7c0ce2f0 con 0x7f7f7c0d14f0 2024-05-20T18:32:56.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.119+0000 7f7f81d9c640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f7c0d1ac0 con 0x7f7f7c0b7210 2024-05-20T18:32:56.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.119+0000 7f7f81d9c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7f7c0d1da0 con 0x7f7f7c0d10e0 2024-05-20T18:32:56.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.119+0000 7f7f7affd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7f7c0d14f0 0x7f7f7c0cd4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46816/0 (socket says 172.21.15.5:46816) 2024-05-20T18:32:56.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.119+0000 7f7f7affd640 1 -- 172.21.15.5:0/3078602879 learned_addr learned my addr 172.21.15.5:0/3078602879 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:32:56.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.119+0000 7f7f7b7fe640 1 --2- 172.21.15.5:0/3078602879 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f7c0d10e0 0x7f7f7c0cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:56.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.119+0000 7f7f7a7fc640 1 -- 172.21.15.5:0/3078602879 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 890507414 0 0) 0x7f7f7c0ce2f0 con 0x7f7f7c0d14f0 2024-05-20T18:32:56.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.119+0000 7f7f7a7fc640 1 -- 172.21.15.5:0/3078602879 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f64003520 con 0x7f7f7c0d14f0 2024-05-20T18:32:56.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.119+0000 7f7f7a7fc640 1 -- 172.21.15.5:0/3078602879 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1507755553 0 0) 0x7f7f7c0d1ac0 con 0x7f7f7c0b7210 2024-05-20T18:32:56.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.119+0000 7f7f7a7fc640 1 -- 172.21.15.5:0/3078602879 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f7c0ce2f0 con 0x7f7f7c0b7210 2024-05-20T18:32:56.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.119+0000 7f7f7a7fc640 1 -- 172.21.15.5:0/3078602879 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3327509962 0 0) 0x7f7f64003520 con 0x7f7f7c0d14f0 2024-05-20T18:32:56.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.119+0000 7f7f7a7fc640 1 -- 172.21.15.5:0/3078602879 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f7c0d10e0 msgr2=0x7f7f7c0cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:56.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.119+0000 7f7f7a7fc640 1 --2- 172.21.15.5:0/3078602879 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f7c0d10e0 0x7f7f7c0cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:56.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.119+0000 7f7f7a7fc640 1 -- 172.21.15.5:0/3078602879 >> v1:172.21.15.63:6789/0 conn(0x7f7f7c0b7210 legacy=0x7f7f7c0cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:56.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.119+0000 7f7f7a7fc640 1 -- 172.21.15.5:0/3078602879 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f7c0d2080 con 0x7f7f7c0d14f0 2024-05-20T18:32:56.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.119+0000 7f7f7a7fc640 1 -- 172.21.15.5:0/3078602879 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f7f70001670 con 0x7f7f7c0d14f0 2024-05-20T18:32:56.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.119+0000 7f7f7a7fc640 1 -- 172.21.15.5:0/3078602879 >> v1:172.21.15.5:6789/0 conn(0x7f7f7c0d14f0 legacy=0x7f7f7c0cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:56.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.120+0000 7f7f7a7fc640 1 --2- 172.21.15.5:0/3078602879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f64003b70 0x7f7f64003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:56.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.120+0000 7f7f7a7fc640 1 --2- 172.21.15.5:0/3078602879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f64004800 0x7f7f64004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:56.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.120+0000 7f7f7a7fc640 1 --2- 172.21.15.5:0/3078602879 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f640051a0 0x7f7f6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:56.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.120+0000 7f7f7a7fc640 1 -- 172.21.15.5:0/3078602879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7c0d1da0 con 0x7f7f64003b70 2024-05-20T18:32:56.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.120+0000 7f7f7a7fc640 1 -- 172.21.15.5:0/3078602879 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6400c090 con 0x7f7f64004800 2024-05-20T18:32:56.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.120+0000 7f7f7a7fc640 1 -- 172.21.15.5:0/3078602879 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7f6400c290 con 0x7f7f640051a0 2024-05-20T18:32:56.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.120+0000 7f7f7b7fe640 1 --2- 172.21.15.5:0/3078602879 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f7c0d10e0 0x7f7f7c0cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:32:56.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.120+0000 7f7f7b7fe640 1 --2- 172.21.15.5:0/3078602879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f64003b70 0x7f7f64003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:56.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.120+0000 7f7f7affd640 1 --2- 172.21.15.5:0/3078602879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f64004800 0x7f7f64004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:56.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.120+0000 7f7f7bfff640 1 --2- 172.21.15.5:0/3078602879 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f640051a0 0x7f7f6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:56.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.120+0000 7f7f7b7fe640 1 -- 172.21.15.5:0/3078602879 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f640051a0 msgr2=0x7f7f6400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:56.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.120+0000 7f7f7b7fe640 1 --2- 172.21.15.5:0/3078602879 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f640051a0 0x7f7f6400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:56.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.120+0000 7f7f7b7fe640 1 -- 172.21.15.5:0/3078602879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f64004800 msgr2=0x7f7f64004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:56.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.120+0000 7f7f7b7fe640 1 --2- 172.21.15.5:0/3078602879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f64004800 0x7f7f64004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:56.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.120+0000 7f7f7b7fe640 1 -- 172.21.15.5:0/3078602879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f64003520 con 0x7f7f64003b70 2024-05-20T18:32:56.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.121+0000 7f7f7b7fe640 1 --2- 172.21.15.5:0/3078602879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f64003b70 0x7f7f64003fc0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0b5280 tx=0x7f7f64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bde55a7d41e74e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:56.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.121+0000 7f7f7a7fc640 1 -- 172.21.15.5:0/3078602879 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c036070 con 0x7f7f64003b70 2024-05-20T18:32:56.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.121+0000 7f7f7a7fc640 1 -- 172.21.15.5:0/3078602879 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f6c004030 con 0x7f7f64003b70 2024-05-20T18:32:56.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.121+0000 7f7f7a7fc640 1 -- 172.21.15.5:0/3078602879 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c0097d0 con 0x7f7f64003b70 2024-05-20T18:32:56.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.122+0000 7f7f81d9c640 1 -- 172.21.15.5:0/3078602879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f64003b70 msgr2=0x7f7f64003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:56.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.122+0000 7f7f81d9c640 1 --2- 172.21.15.5:0/3078602879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f64003b70 0x7f7f64003fc0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0b5280 tx=0x7f7f64005830 comp rx=0 tx=0).stop 2024-05-20T18:32:56.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.122+0000 7f7f7bfff640 1 -- 172.21.15.5:0/3078602879 reap_dead start 2024-05-20T18:32:56.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.122+0000 7f7f81d9c640 1 -- 172.21.15.5:0/3078602879 shutdown_connections 2024-05-20T18:32:56.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.122+0000 7f7f81d9c640 1 -- 172.21.15.5:0/3078602879 >> 172.21.15.5:0/3078602879 conn(0x7f7f7c0b44d0 msgr2=0x7f7f7c0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:56.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.122+0000 7f7f81d9c640 1 -- 172.21.15.5:0/3078602879 shutdown_connections 2024-05-20T18:32:56.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.122+0000 7f7f81d9c640 1 -- 172.21.15.5:0/3078602879 wait complete. 2024-05-20T18:32:56.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.124+0000 7f7f81d9c640 1 Processor -- start 2024-05-20T18:32:56.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.124+0000 7f7f81d9c640 1 -- start start 2024-05-20T18:32:56.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.125+0000 7f7f81d9c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f64003b70 0x7f7f7c15e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:56.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.125+0000 7f7f81d9c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f7c15e780 0x7f7f7c15eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:56.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.125+0000 7f7f81d9c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f7c0c9960 0x7f7f7c0c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:56.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.125+0000 7f7f81d9c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7c0b82c0 con 0x7f7f7c0c9960 2024-05-20T18:32:56.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.125+0000 7f7f81d9c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7c0d2640 con 0x7f7f64003b70 2024-05-20T18:32:56.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.125+0000 7f7f81d9c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7f7c0b8440 con 0x7f7f7c15e780 2024-05-20T18:32:56.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.125+0000 7f7f7bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f7c0c9960 0x7f7f7c0c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:56.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.125+0000 7f7f7bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f7c0c9960 0x7f7f7c0c9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44602/0 (socket says 172.21.15.5:44602) 2024-05-20T18:32:56.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.125+0000 7f7f7bfff640 1 -- 172.21.15.5:0/1591926787 learned_addr learned my addr 172.21.15.5:0/1591926787 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:56.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.125+0000 7f7f7affd640 1 --2- 172.21.15.5:0/1591926787 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f7c15e780 0x7f7f7c15eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:56.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.125+0000 7f7f7b7fe640 1 --2- 172.21.15.5:0/1591926787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f64003b70 0x7f7f7c15e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:56.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.125+0000 7f7f7bfff640 1 -- 172.21.15.5:0/1591926787 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f7c15e780 msgr2=0x7f7f7c15eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:56.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.125+0000 7f7f7bfff640 1 --2- 172.21.15.5:0/1591926787 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f7c15e780 0x7f7f7c15eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:56.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.125+0000 7f7f7bfff640 1 -- 172.21.15.5:0/1591926787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f64003b70 msgr2=0x7f7f7c15e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:56.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.125+0000 7f7f7bfff640 1 --2- 172.21.15.5:0/1591926787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f64003b70 0x7f7f7c15e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:56.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.126+0000 7f7f7bfff640 1 -- 172.21.15.5:0/1591926787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f7c15f0d0 con 0x7f7f7c0c9960 2024-05-20T18:32:56.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.126+0000 7f7f7bfff640 1 --2- 172.21.15.5:0/1591926787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f7c0c9960 0x7f7f7c0c9d70 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0d02d0 tx=0x7f7f7400feb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:56.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.126+0000 7f7f63fff640 1 -- 172.21.15.5:0/1591926787 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f74019070 con 0x7f7f7c0c9960 2024-05-20T18:32:56.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.126+0000 7f7f63fff640 1 -- 172.21.15.5:0/1591926787 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f74006e20 con 0x7f7f7c0c9960 2024-05-20T18:32:56.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.126+0000 7f7f81d9c640 1 -- 172.21.15.5:0/1591926787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f7c06d3b0 con 0x7f7f7c0c9960 2024-05-20T18:32:56.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.126+0000 7f7f63fff640 1 -- 172.21.15.5:0/1591926787 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f74003bb0 con 0x7f7f7c0c9960 2024-05-20T18:32:56.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.126+0000 7f7f81d9c640 1 -- 172.21.15.5:0/1591926787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f7c06d970 con 0x7f7f7c0c9960 2024-05-20T18:32:56.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.127+0000 7f7f81d9c640 1 -- 172.21.15.5:0/1591926787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f7c06d120 con 0x7f7f7c0c9960 2024-05-20T18:32:56.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.128+0000 7f7f63fff640 1 -- 172.21.15.5:0/1591926787 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7f7401c070 con 0x7f7f7c0c9960 2024-05-20T18:32:56.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.128+0000 7f7f63fff640 1 --2- 172.21.15.5:0/1591926787 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7f6806be30 0x7f7f6806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:56.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.128+0000 7f7f63fff640 1 -- 172.21.15.5:0/1591926787 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f7f7408a260 con 0x7f7f7c0c9960 2024-05-20T18:32:56.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.128+0000 7f7f7b7fe640 1 --2- 172.21.15.5:0/1591926787 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7f6806be30 0x7f7f6806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:56.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.129+0000 7f7f7b7fe640 1 --2- 172.21.15.5:0/1591926787 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7f6806be30 0x7f7f6806e340 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0b5280 tx=0x7f7f6c008070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:56.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.133+0000 7f7f63fff640 1 -- 172.21.15.5:0/1591926787 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7f74058350 con 0x7f7f7c0c9960 2024-05-20T18:32:56.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.354+0000 7f7f81d9c640 1 -- 172.21.15.5:0/1591926787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f7c15d150 con 0x7f7f7c0c9960 2024-05-20T18:32:56.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.354+0000 7f7f63fff640 1 -- 172.21.15.5:0/1591926787 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5176 (secure 0 0 0) 0x7f7f7405c650 con 0x7f7f7c0c9960 2024-05-20T18:32:56.355 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:32:56.355 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1890759603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1890759603},{"type":"v1","addr":"172.21.15.63:6834","nonce":1890759603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5825,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/700447403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":700447403},{"type":"v1","addr":"172.21.15.63:6835","nonce":700447403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3382943275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3382943275},{"type":"v1","addr":"172.21.15.5:6837","nonce":3382943275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:31:30.164179+0000","modified":"2024-05-20T18:32:27.855796+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6972":{"gid":6972,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":15,"addr":"172.21.15.5:6836/3866822752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":3866822752},{"type":"v1","addr":"172.21.15.5:6836","nonce":3866822752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6972,"qdb_cluster":[6972]},"id":5}]} 2024-05-20T18:32:56.355 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 52 2024-05-20T18:32:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.356+0000 7f7f81d9c640 1 -- 172.21.15.5:0/1591926787 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7f6806be30 msgr2=0x7f7f6806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.356+0000 7f7f81d9c640 1 --2- 172.21.15.5:0/1591926787 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7f6806be30 0x7f7f6806e340 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0b5280 tx=0x7f7f6c008070 comp rx=0 tx=0).stop 2024-05-20T18:32:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.356+0000 7f7f81d9c640 1 -- 172.21.15.5:0/1591926787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f7c0c9960 msgr2=0x7f7f7c0c9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.356+0000 7f7f81d9c640 1 --2- 172.21.15.5:0/1591926787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f7c0c9960 0x7f7f7c0c9d70 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0d02d0 tx=0x7f7f7400feb0 comp rx=0 tx=0).stop 2024-05-20T18:32:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.356+0000 7f7f81d9c640 1 -- 172.21.15.5:0/1591926787 shutdown_connections 2024-05-20T18:32:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.356+0000 7f7f81d9c640 1 --2- 172.21.15.5:0/1591926787 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7f6806be30 0x7f7f6806e340 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.356+0000 7f7f81d9c640 1 --2- 172.21.15.5:0/1591926787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f7c0c9960 0x7f7f7c0c9d70 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.356+0000 7f7f81d9c640 1 --2- 172.21.15.5:0/1591926787 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f7c15e780 0x7f7f7c15eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.356+0000 7f7f81d9c640 1 --2- 172.21.15.5:0/1591926787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f64003b70 0x7f7f7c15e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.356+0000 7f7f81d9c640 1 -- 172.21.15.5:0/1591926787 >> 172.21.15.5:0/1591926787 conn(0x7f7f7c0b44d0 msgr2=0x7f7f7c0b1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.356+0000 7f7f81d9c640 1 -- 172.21.15.5:0/1591926787 shutdown_connections 2024-05-20T18:32:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.356+0000 7f7f81d9c640 1 -- 172.21.15.5:0/1591926787 wait complete. 2024-05-20T18:32:56.369 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:32:56.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.483+0000 7fc20a137640 1 Processor -- start 2024-05-20T18:32:56.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.483+0000 7fc20a137640 1 -- start start 2024-05-20T18:32:56.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.484+0000 7fc20a137640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc20415d530 0x7fc20415aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:56.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.484+0000 7fc20a137640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc2040b1700 0x7fc2040b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:56.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.484+0000 7fc20a137640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc204070850 0x7fc204070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:56.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.484+0000 7fc20a137640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fc2040ce190 con 0x7fc204070850 2024-05-20T18:32:56.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.484+0000 7fc20a137640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc2040cd580 con 0x7fc2040b1700 2024-05-20T18:32:56.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.484+0000 7fc20a137640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc2040cd860 con 0x7fc20415d530 2024-05-20T18:32:56.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.485+0000 7fc203fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc204070850 0x7fc204070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:56.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.485+0000 7fc203fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc204070850 0x7fc204070c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44614/0 (socket says 172.21.15.5:44614) 2024-05-20T18:32:56.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.485+0000 7fc203fff640 1 -- 172.21.15.5:0/2807992311 learned_addr learned my addr 172.21.15.5:0/2807992311 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:56.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.485+0000 7fc2037fe640 1 --2- 172.21.15.5:0/2807992311 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc20415d530 0x7fc20415aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:56.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.485+0000 7fc1fbfff640 1 --2- 172.21.15.5:0/2807992311 >> v2:172.21.15.63:3300/0 conn(0x7fc2040b1700 0x7fc2040b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:56.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.485+0000 7fc203fff640 1 -- 172.21.15.5:0/2807992311 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc20415d530 msgr2=0x7fc20415aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:56.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.485+0000 7fc203fff640 1 --2- 172.21.15.5:0/2807992311 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc20415d530 0x7fc20415aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:56.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.485+0000 7fc203fff640 1 -- 172.21.15.5:0/2807992311 >> v2:172.21.15.63:3300/0 conn(0x7fc2040b1700 msgr2=0x7fc2040b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:56.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.485+0000 7fc203fff640 1 --2- 172.21.15.5:0/2807992311 >> v2:172.21.15.63:3300/0 conn(0x7fc2040b1700 0x7fc2040b1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:56.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.485+0000 7fc203fff640 1 -- 172.21.15.5:0/2807992311 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2040cdb40 con 0x7fc204070850 2024-05-20T18:32:56.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.486+0000 7fc203fff640 1 --2- 172.21.15.5:0/2807992311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc204070850 0x7fc204070c60 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fc20406c2f0 tx=0x7fc1fc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f51585badbd0d8f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:56.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.486+0000 7fc202ffd640 1 -- 172.21.15.5:0/2807992311 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc036070 con 0x7fc204070850 2024-05-20T18:32:56.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.486+0000 7fc202ffd640 1 -- 172.21.15.5:0/2807992311 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1fc031600 con 0x7fc204070850 2024-05-20T18:32:56.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.486+0000 7fc202ffd640 1 -- 172.21.15.5:0/2807992311 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc0319d0 con 0x7fc204070850 2024-05-20T18:32:56.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.487+0000 7fc20a137640 1 -- 172.21.15.5:0/2807992311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc204070850 msgr2=0x7fc204070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:56.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.487+0000 7fc20a137640 1 --2- 172.21.15.5:0/2807992311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc204070850 0x7fc204070c60 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fc20406c2f0 tx=0x7fc1fc02a6a0 comp rx=0 tx=0).stop 2024-05-20T18:32:56.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.487+0000 7fc20a137640 1 -- 172.21.15.5:0/2807992311 shutdown_connections 2024-05-20T18:32:56.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.487+0000 7fc20a137640 1 --2- 172.21.15.5:0/2807992311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc204070850 0x7fc204070c60 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:56.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.487+0000 7fc20a137640 1 --2- 172.21.15.5:0/2807992311 >> v2:172.21.15.63:3300/0 conn(0x7fc2040b1700 0x7fc2040b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:56.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.487+0000 7fc20a137640 1 --2- 172.21.15.5:0/2807992311 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc20415d530 0x7fc20415aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:56.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.487+0000 7fc20a137640 1 -- 172.21.15.5:0/2807992311 >> 172.21.15.5:0/2807992311 conn(0x7fc2040d29d0 msgr2=0x7fc2040d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:56.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.487+0000 7fc20a137640 1 -- 172.21.15.5:0/2807992311 shutdown_connections 2024-05-20T18:32:56.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.487+0000 7fc20a137640 1 -- 172.21.15.5:0/2807992311 wait complete. 2024-05-20T18:32:56.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.489+0000 7fc20a137640 1 Processor -- start 2024-05-20T18:32:56.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.489+0000 7fc20a137640 1 -- start start 2024-05-20T18:32:56.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.490+0000 7fc20a137640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc204070850 0x7fc204155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:56.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.490+0000 7fc20a137640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2040b1700 0x7fc2040c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:56.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.490+0000 7fc20a137640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc20415d530 0x7fc2040c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:56.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.490+0000 7fc20a137640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc2040b2050 con 0x7fc2040b1700 2024-05-20T18:32:56.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.490+0000 7fc20a137640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc2040cd230 con 0x7fc20415d530 2024-05-20T18:32:56.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.490+0000 7fc20a137640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc2040711a0 con 0x7fc204070850 2024-05-20T18:32:56.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.490+0000 7fc1fbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2040b1700 0x7fc2040c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:56.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.490+0000 7fc1fbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2040b1700 0x7fc2040c8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44624/0 (socket says 172.21.15.5:44624) 2024-05-20T18:32:56.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.490+0000 7fc1fbfff640 1 -- 172.21.15.5:0/2879129204 learned_addr learned my addr 172.21.15.5:0/2879129204 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:56.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.490+0000 7fc2037fe640 1 --2- 172.21.15.5:0/2879129204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc204070850 0x7fc204155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:56.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.490+0000 7fc1fbfff640 1 -- 172.21.15.5:0/2879129204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc204070850 msgr2=0x7fc204155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:56.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.490+0000 7fc1fbfff640 1 --2- 172.21.15.5:0/2879129204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc204070850 0x7fc204155220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:56.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.490+0000 7fc1fbfff640 1 -- 172.21.15.5:0/2879129204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc20415d530 msgr2=0x7fc2040c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:56.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.490+0000 7fc203fff640 1 --2- 172.21.15.5:0/2879129204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc20415d530 0x7fc2040c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:56.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.490+0000 7fc1fbfff640 1 --2- 172.21.15.5:0/2879129204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc20415d530 0x7fc2040c91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:56.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.490+0000 7fc1fbfff640 1 -- 172.21.15.5:0/2879129204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2040c7520 con 0x7fc2040b1700 2024-05-20T18:32:56.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.490+0000 7fc203fff640 1 --2- 172.21.15.5:0/2879129204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc20415d530 0x7fc2040c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:32:56.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.491+0000 7fc1fbfff640 1 --2- 172.21.15.5:0/2879129204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2040b1700 0x7fc2040c8c60 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec00d5a0 tx=0x7fc1ec00da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:56.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.491+0000 7fc200ff9640 1 -- 172.21.15.5:0/2879129204 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec018020 con 0x7fc2040b1700 2024-05-20T18:32:56.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.491+0000 7fc200ff9640 1 -- 172.21.15.5:0/2879129204 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1ec014620 con 0x7fc2040b1700 2024-05-20T18:32:56.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.491+0000 7fc200ff9640 1 -- 172.21.15.5:0/2879129204 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec0149f0 con 0x7fc2040b1700 2024-05-20T18:32:56.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.491+0000 7fc20a137640 1 -- 172.21.15.5:0/2879129204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2041f3970 con 0x7fc2040b1700 2024-05-20T18:32:56.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.491+0000 7fc20a137640 1 -- 172.21.15.5:0/2879129204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2041f3f30 con 0x7fc2040b1700 2024-05-20T18:32:56.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.492+0000 7fc20a137640 1 -- 172.21.15.5:0/2879129204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc204075b00 con 0x7fc2040b1700 2024-05-20T18:32:56.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.493+0000 7fc200ff9640 1 -- 172.21.15.5:0/2879129204 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc1ec01b020 con 0x7fc2040b1700 2024-05-20T18:32:56.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.493+0000 7fc200ff9640 1 --2- 172.21.15.5:0/2879129204 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc1e406be60 0x7fc1e406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:56.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.493+0000 7fc200ff9640 1 -- 172.21.15.5:0/2879129204 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7fc1ec08b4b0 con 0x7fc2040b1700 2024-05-20T18:32:56.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.493+0000 7fc2037fe640 1 --2- 172.21.15.5:0/2879129204 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc1e406be60 0x7fc1e406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:56.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.494+0000 7fc2037fe640 1 --2- 172.21.15.5:0/2879129204 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc1e406be60 0x7fc1e406e370 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4006fd0 tx=0x7fc1f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:56.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.498+0000 7fc200ff9640 1 -- 172.21.15.5:0/2879129204 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc1ec022020 con 0x7fc2040b1700 2024-05-20T18:32:56.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.682+0000 7fc20a137640 1 -- 172.21.15.5:0/2879129204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc2040c67d0 con 0x7fc2040b1700 2024-05-20T18:32:56.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.683+0000 7fc200ff9640 1 -- 172.21.15.5:0/2879129204 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+16950 (secure 0 0 0) 0x7fc1ec059bf0 con 0x7fc2040b1700 2024-05-20T18:32:56.684 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:32:56.685 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":78,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:31:40.839398+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:31:27.896386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-20T18:31:29.146912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6836/791176355":"2024-05-21T18:31:40.839373+0000","172.21.15.5:6835/791176355":"2024-05-21T18:31:40.839373+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:32:56.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.685+0000 7fc20a137640 1 -- 172.21.15.5:0/2879129204 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc1e406be60 msgr2=0x7fc1e406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:56.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.685+0000 7fc20a137640 1 --2- 172.21.15.5:0/2879129204 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc1e406be60 0x7fc1e406e370 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4006fd0 tx=0x7fc1f4008040 comp rx=0 tx=0).stop 2024-05-20T18:32:56.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.685+0000 7fc20a137640 1 -- 172.21.15.5:0/2879129204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2040b1700 msgr2=0x7fc2040c8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:56.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.685+0000 7fc20a137640 1 --2- 172.21.15.5:0/2879129204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2040b1700 0x7fc2040c8c60 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec00d5a0 tx=0x7fc1ec00da60 comp rx=0 tx=0).stop 2024-05-20T18:32:56.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.685+0000 7fc20a137640 1 -- 172.21.15.5:0/2879129204 shutdown_connections 2024-05-20T18:32:56.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.685+0000 7fc20a137640 1 --2- 172.21.15.5:0/2879129204 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc1e406be60 0x7fc1e406e370 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:56.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.686+0000 7fc20a137640 1 --2- 172.21.15.5:0/2879129204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc20415d530 0x7fc2040c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:56.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.686+0000 7fc20a137640 1 --2- 172.21.15.5:0/2879129204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2040b1700 0x7fc2040c8c60 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:56.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.686+0000 7fc20a137640 1 --2- 172.21.15.5:0/2879129204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc204070850 0x7fc204155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:56.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.686+0000 7fc20a137640 1 -- 172.21.15.5:0/2879129204 >> 172.21.15.5:0/2879129204 conn(0x7fc2040d29d0 msgr2=0x7fc20415f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:56.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.686+0000 7fc20a137640 1 -- 172.21.15.5:0/2879129204 shutdown_connections 2024-05-20T18:32:56.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.686+0000 7fc20a137640 1 -- 172.21.15.5:0/2879129204 wait complete. 2024-05-20T18:32:56.698 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:31:27.896386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:31:29.146912+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:32:56.699 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:32:56.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.816+0000 7f150ed4c640 1 Processor -- start 2024-05-20T18:32:56.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.816+0000 7f150ed4c640 1 -- start start 2024-05-20T18:32:56.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.818+0000 7f150ed4c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f150806eab0 0x7f150806cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:56.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.818+0000 7f150ed4c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1508157770 0x7f150815d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:56.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.818+0000 7f150ed4c640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f150806ba20 con 0x7f150806b520 2024-05-20T18:32:56.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.818+0000 7f150ed4c640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f150806c630 con 0x7f1508157770 2024-05-20T18:32:56.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.818+0000 7f150ed4c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f150806bd00 con 0x7f150806eab0 2024-05-20T18:32:56.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.818+0000 7f150d2e9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1508157770 0x7f150815d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:56.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.818+0000 7f150d2e9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1508157770 0x7f150815d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44632/0 (socket says 172.21.15.5:44632) 2024-05-20T18:32:56.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.818+0000 7f150d2e9640 1 -- 172.21.15.5:0/3232738620 learned_addr learned my addr 172.21.15.5:0/3232738620 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:56.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.818+0000 7f1507fff640 1 --2- 172.21.15.5:0/3232738620 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f150806eab0 0x7f150806cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:56.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.818+0000 7f150d2e9640 1 -- 172.21.15.5:0/3232738620 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f150806eab0 msgr2=0x7f150806cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:56.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.818+0000 7f150d2e9640 1 --2- 172.21.15.5:0/3232738620 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f150806eab0 0x7f150806cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:56.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.818+0000 7f150d2e9640 1 -- 172.21.15.5:0/3232738620 >> v1:172.21.15.63:6789/0 conn(0x7f150806b520 legacy=0x7f150806e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:56.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.818+0000 7f150d2e9640 1 -- 172.21.15.5:0/3232738620 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150806bfe0 con 0x7f1508157770 2024-05-20T18:32:56.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.819+0000 7f150d2e9640 1 --2- 172.21.15.5:0/3232738620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1508157770 0x7f150815d5f0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f1500009e50 tx=0x7f150002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7879d0752c5c75 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:56.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.819+0000 7f15077fe640 1 -- 172.21.15.5:0/3232738620 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500036070 con 0x7f1508157770 2024-05-20T18:32:56.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.819+0000 7f15077fe640 1 -- 172.21.15.5:0/3232738620 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1500031600 con 0x7f1508157770 2024-05-20T18:32:56.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.819+0000 7f15077fe640 1 -- 172.21.15.5:0/3232738620 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15000319d0 con 0x7f1508157770 2024-05-20T18:32:56.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.820+0000 7f150ed4c640 1 -- 172.21.15.5:0/3232738620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1508157770 msgr2=0x7f150815d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:56.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.820+0000 7f150ed4c640 1 --2- 172.21.15.5:0/3232738620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1508157770 0x7f150815d5f0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f1500009e50 tx=0x7f150002a6c0 comp rx=0 tx=0).stop 2024-05-20T18:32:56.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.820+0000 7f150ed4c640 1 -- 172.21.15.5:0/3232738620 shutdown_connections 2024-05-20T18:32:56.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.820+0000 7f150ed4c640 1 --2- 172.21.15.5:0/3232738620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1508157770 0x7f150815d5f0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:56.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.820+0000 7f150ed4c640 1 --2- 172.21.15.5:0/3232738620 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f150806eab0 0x7f150806cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:56.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.820+0000 7f150ed4c640 1 -- 172.21.15.5:0/3232738620 >> 172.21.15.5:0/3232738620 conn(0x7f15080cd2a0 msgr2=0x7f15080cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:56.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.820+0000 7f150ed4c640 1 -- 172.21.15.5:0/3232738620 shutdown_connections 2024-05-20T18:32:56.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.820+0000 7f150ed4c640 1 -- 172.21.15.5:0/3232738620 wait complete. 2024-05-20T18:32:56.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.822+0000 7f150ed4c640 1 Processor -- start 2024-05-20T18:32:56.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.822+0000 7f150ed4c640 1 -- start start 2024-05-20T18:32:56.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.823+0000 7f150ed4c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f150806b520 0x7f150814a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:56.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.823+0000 7f150ed4c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1508157770 0x7f15080ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:56.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.823+0000 7f150ed4c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15080aedc0 0x7f150815d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:56.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.823+0000 7f150ed4c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f150806f0c0 con 0x7f15080aedc0 2024-05-20T18:32:56.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.823+0000 7f150ed4c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f150806ef40 con 0x7f1508157770 2024-05-20T18:32:56.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.823+0000 7f150ed4c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f15080ce6c0 con 0x7f150806b520 2024-05-20T18:32:56.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.823+0000 7f150d2e9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15080aedc0 0x7f150815d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:56.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.823+0000 7f150d2e9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15080aedc0 0x7f150815d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44648/0 (socket says 172.21.15.5:44648) 2024-05-20T18:32:56.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.823+0000 7f150d2e9640 1 -- 172.21.15.5:0/2155633689 learned_addr learned my addr 172.21.15.5:0/2155633689 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:56.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.823+0000 7f150cae8640 1 --2- 172.21.15.5:0/2155633689 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f150806b520 0x7f150814a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:56.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.823+0000 7f1507fff640 1 --2- 172.21.15.5:0/2155633689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1508157770 0x7f15080ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:56.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.824+0000 7f150d2e9640 1 -- 172.21.15.5:0/2155633689 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f150806b520 msgr2=0x7f150814a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:56.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.824+0000 7f150d2e9640 1 --2- 172.21.15.5:0/2155633689 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f150806b520 0x7f150814a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:56.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.824+0000 7f150d2e9640 1 -- 172.21.15.5:0/2155633689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1508157770 msgr2=0x7f15080ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:56.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.824+0000 7f150d2e9640 1 --2- 172.21.15.5:0/2155633689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1508157770 0x7f15080ae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:56.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.824+0000 7f150d2e9640 1 -- 172.21.15.5:0/2155633689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150814af00 con 0x7f15080aedc0 2024-05-20T18:32:56.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.824+0000 7f150d2e9640 1 --2- 172.21.15.5:0/2155633689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15080aedc0 0x7f150815d460 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f1500009e50 tx=0x7f1500004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:56.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.824+0000 7f15057fa640 1 -- 172.21.15.5:0/2155633689 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500041070 con 0x7f15080aedc0 2024-05-20T18:32:56.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.824+0000 7f15057fa640 1 -- 172.21.15.5:0/2155633689 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f150003c650 con 0x7f15080aedc0 2024-05-20T18:32:56.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.824+0000 7f150ed4c640 1 -- 172.21.15.5:0/2155633689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f150815e9a0 con 0x7f15080aedc0 2024-05-20T18:32:56.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.824+0000 7f15057fa640 1 -- 172.21.15.5:0/2155633689 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500036040 con 0x7f15080aedc0 2024-05-20T18:32:56.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.824+0000 7f150ed4c640 1 -- 172.21.15.5:0/2155633689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f150815ef60 con 0x7f15080aedc0 2024-05-20T18:32:56.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.825+0000 7f150ed4c640 1 -- 172.21.15.5:0/2155633689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1508075b00 con 0x7f15080aedc0 2024-05-20T18:32:56.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.826+0000 7f15057fa640 1 -- 172.21.15.5:0/2155633689 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1500044070 con 0x7f15080aedc0 2024-05-20T18:32:56.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.826+0000 7f15057fa640 1 --2- 172.21.15.5:0/2155633689 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f14f406be60 0x7f14f406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:56.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.827+0000 7f15057fa640 1 -- 172.21.15.5:0/2155633689 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f15000abb20 con 0x7f15080aedc0 2024-05-20T18:32:56.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.827+0000 7f150cae8640 1 --2- 172.21.15.5:0/2155633689 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f14f406be60 0x7f14f406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:56.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.827+0000 7f150cae8640 1 --2- 172.21.15.5:0/2155633689 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f14f406be60 0x7f14f406e370 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f14f8006fd0 tx=0x7f14f8009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:56.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:56.831+0000 7f15057fa640 1 -- 172.21.15.5:0/2155633689 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1500079c10 con 0x7f15080aedc0 2024-05-20T18:32:57.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.016+0000 7f150ed4c640 1 -- 172.21.15.5:0/2155633689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f15080b1590 con 0x7f15080aedc0 2024-05-20T18:32:57.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.018+0000 7f15057fa640 1 -- 172.21.15.5:0/2155633689 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7f150007df10 con 0x7f15080aedc0 2024-05-20T18:32:57.018 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:32:57.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.020+0000 7f150ed4c640 1 -- 172.21.15.5:0/2155633689 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f14f406be60 msgr2=0x7f14f406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:57.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.020+0000 7f150ed4c640 1 --2- 172.21.15.5:0/2155633689 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f14f406be60 0x7f14f406e370 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f14f8006fd0 tx=0x7f14f8009060 comp rx=0 tx=0).stop 2024-05-20T18:32:57.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.020+0000 7f150ed4c640 1 -- 172.21.15.5:0/2155633689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15080aedc0 msgr2=0x7f150815d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:57.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.020+0000 7f150ed4c640 1 --2- 172.21.15.5:0/2155633689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15080aedc0 0x7f150815d460 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f1500009e50 tx=0x7f1500004a40 comp rx=0 tx=0).stop 2024-05-20T18:32:57.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.020+0000 7f150ed4c640 1 -- 172.21.15.5:0/2155633689 shutdown_connections 2024-05-20T18:32:57.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.020+0000 7f150ed4c640 1 --2- 172.21.15.5:0/2155633689 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f14f406be60 0x7f14f406e370 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:57.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.020+0000 7f150ed4c640 1 --2- 172.21.15.5:0/2155633689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15080aedc0 0x7f150815d460 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:57.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.020+0000 7f150ed4c640 1 --2- 172.21.15.5:0/2155633689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1508157770 0x7f15080ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:57.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.020+0000 7f150ed4c640 1 --2- 172.21.15.5:0/2155633689 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f150806b520 0x7f150814a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:57.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.020+0000 7f150ed4c640 1 -- 172.21.15.5:0/2155633689 >> 172.21.15.5:0/2155633689 conn(0x7f15080cd2a0 msgr2=0x7f15080b1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:57.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.020+0000 7f150ed4c640 1 -- 172.21.15.5:0/2155633689 shutdown_connections 2024-05-20T18:32:57.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.020+0000 7f150ed4c640 1 -- 172.21.15.5:0/2155633689 wait complete. 2024-05-20T18:32:57.032 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:32:57.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.149+0000 7f03adc8b640 1 Processor -- start 2024-05-20T18:32:57.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.149+0000 7f03adc8b640 1 -- start start 2024-05-20T18:32:57.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.151+0000 7f03adc8b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f03a8162820 0x7f03a80b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:57.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.151+0000 7f03adc8b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03a80b3e80 0x7f03a806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:57.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.151+0000 7f03adc8b640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03a806dd20 con 0x7f03a806d910 2024-05-20T18:32:57.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.151+0000 7f03adc8b640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f03a80d0630 con 0x7f03a8162820 2024-05-20T18:32:57.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.151+0000 7f03adc8b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f03a80d39c0 con 0x7f03a80b3e80 2024-05-20T18:32:57.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.151+0000 7f03a7fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f03a806d910 0x7f03a80d27a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46828/0 (socket says 172.21.15.5:46828) 2024-05-20T18:32:57.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.151+0000 7f03a7fff640 1 -- 172.21.15.5:0/3378890032 learned_addr learned my addr 172.21.15.5:0/3378890032 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:32:57.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.151+0000 7f03a6ffd640 1 --2- 172.21.15.5:0/3378890032 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03a80b3e80 0x7f03a806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:57.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.151+0000 7f03a77fe640 1 --2- 172.21.15.5:0/3378890032 >> v2:172.21.15.63:3300/0 conn(0x7f03a8162820 0x7f03a80b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:57.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.151+0000 7f03a67fc640 1 -- 172.21.15.5:0/3378890032 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2896523789 0 0) 0x7f03a0001490 con 0x7f03a806d910 2024-05-20T18:32:57.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.151+0000 7f03a67fc640 1 -- 172.21.15.5:0/3378890032 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0390003520 con 0x7f03a806d910 2024-05-20T18:32:57.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.151+0000 7f03a6ffd640 1 -- 172.21.15.5:0/3378890032 >> v2:172.21.15.63:3300/0 conn(0x7f03a8162820 msgr2=0x7f03a80b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:57.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.151+0000 7f03a6ffd640 1 --2- 172.21.15.5:0/3378890032 >> v2:172.21.15.63:3300/0 conn(0x7f03a8162820 0x7f03a80b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:57.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.151+0000 7f03a6ffd640 1 -- 172.21.15.5:0/3378890032 >> v1:172.21.15.5:6789/0 conn(0x7f03a806d910 legacy=0x7f03a80d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:57.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.151+0000 7f03a6ffd640 1 -- 172.21.15.5:0/3378890032 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a80d3ca0 con 0x7f03a80b3e80 2024-05-20T18:32:57.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.151+0000 7f03a77fe640 1 --2- 172.21.15.5:0/3378890032 >> v2:172.21.15.63:3300/0 conn(0x7f03a8162820 0x7f03a80b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:32:57.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.152+0000 7f03a6ffd640 1 --2- 172.21.15.5:0/3378890032 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03a80b3e80 0x7f03a806d3d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f039c0097a0 tx=0x7f039c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6b423fc5e80199f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:57.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.152+0000 7f03a67fc640 1 -- 172.21.15.5:0/3378890032 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c037070 con 0x7f03a80b3e80 2024-05-20T18:32:57.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.152+0000 7f03a67fc640 1 -- 172.21.15.5:0/3378890032 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f039c032440 con 0x7f03a80b3e80 2024-05-20T18:32:57.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.152+0000 7f03a67fc640 1 -- 172.21.15.5:0/3378890032 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c0327f0 con 0x7f03a80b3e80 2024-05-20T18:32:57.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.153+0000 7f03adc8b640 1 -- 172.21.15.5:0/3378890032 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03a80b3e80 msgr2=0x7f03a806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:57.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.153+0000 7f03adc8b640 1 --2- 172.21.15.5:0/3378890032 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03a80b3e80 0x7f03a806d3d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f039c0097a0 tx=0x7f039c02a180 comp rx=0 tx=0).stop 2024-05-20T18:32:57.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.153+0000 7f03adc8b640 1 -- 172.21.15.5:0/3378890032 shutdown_connections 2024-05-20T18:32:57.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.153+0000 7f03adc8b640 1 --2- 172.21.15.5:0/3378890032 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03a80b3e80 0x7f03a806d3d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:57.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.153+0000 7f03adc8b640 1 --2- 172.21.15.5:0/3378890032 >> v2:172.21.15.63:3300/0 conn(0x7f03a8162820 0x7f03a80b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:57.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.153+0000 7f03adc8b640 1 -- 172.21.15.5:0/3378890032 >> 172.21.15.5:0/3378890032 conn(0x7f03a80d4bd0 msgr2=0x7f03a80d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:57.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.153+0000 7f03adc8b640 1 -- 172.21.15.5:0/3378890032 shutdown_connections 2024-05-20T18:32:57.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.153+0000 7f03adc8b640 1 -- 172.21.15.5:0/3378890032 wait complete. 2024-05-20T18:32:57.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.155+0000 7f03adc8b640 1 Processor -- start 2024-05-20T18:32:57.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.155+0000 7f03adc8b640 1 -- start start 2024-05-20T18:32:57.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.156+0000 7f03adc8b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03a806d910 0x7f03a81e9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:57.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.156+0000 7f03adc8b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03a80b3e80 0x7f03a81f85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:57.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.156+0000 7f03adc8b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03a8162820 0x7f03a80ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:57.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.156+0000 7f03adc8b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f03a80d0300 con 0x7f03a806d910 2024-05-20T18:32:57.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.156+0000 7f03adc8b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f03a80d4260 con 0x7f03a8162820 2024-05-20T18:32:57.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.156+0000 7f03adc8b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f03a80b91a0 con 0x7f03a80b3e80 2024-05-20T18:32:57.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.157+0000 7f03a77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03a806d910 0x7f03a81e9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:57.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.157+0000 7f03a77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03a806d910 0x7f03a81e9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44650/0 (socket says 172.21.15.5:44650) 2024-05-20T18:32:57.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.157+0000 7f03a77fe640 1 -- 172.21.15.5:0/2579688309 learned_addr learned my addr 172.21.15.5:0/2579688309 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:57.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.157+0000 7f03a7fff640 1 --2- 172.21.15.5:0/2579688309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03a8162820 0x7f03a80ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:57.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.157+0000 7f03a6ffd640 1 --2- 172.21.15.5:0/2579688309 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03a80b3e80 0x7f03a81f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:57.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.157+0000 7f03a77fe640 1 -- 172.21.15.5:0/2579688309 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03a80b3e80 msgr2=0x7f03a81f85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:57.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.157+0000 7f03a77fe640 1 --2- 172.21.15.5:0/2579688309 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03a80b3e80 0x7f03a81f85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:57.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.157+0000 7f03a77fe640 1 -- 172.21.15.5:0/2579688309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03a8162820 msgr2=0x7f03a80ca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:57.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.157+0000 7f03a77fe640 1 --2- 172.21.15.5:0/2579688309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03a8162820 0x7f03a80ca1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:57.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.157+0000 7f03a77fe640 1 -- 172.21.15.5:0/2579688309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a81e98a0 con 0x7f03a806d910 2024-05-20T18:32:57.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.157+0000 7f03a77fe640 1 --2- 172.21.15.5:0/2579688309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03a806d910 0x7f03a81e9360 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f039800e990 tx=0x7f039800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:57.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.157+0000 7f038ffff640 1 -- 172.21.15.5:0/2579688309 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0398017070 con 0x7f03a806d910 2024-05-20T18:32:57.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.157+0000 7f038ffff640 1 -- 172.21.15.5:0/2579688309 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f039800c9e0 con 0x7f03a806d910 2024-05-20T18:32:57.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.157+0000 7f03adc8b640 1 -- 172.21.15.5:0/2579688309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03a815d2f0 con 0x7f03a806d910 2024-05-20T18:32:57.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.157+0000 7f038ffff640 1 -- 172.21.15.5:0/2579688309 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039800cdb0 con 0x7f03a806d910 2024-05-20T18:32:57.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.157+0000 7f03adc8b640 1 -- 172.21.15.5:0/2579688309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03a80cc420 con 0x7f03a806d910 2024-05-20T18:32:57.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.158+0000 7f03adc8b640 1 -- 172.21.15.5:0/2579688309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03a8073a00 con 0x7f03a806d910 2024-05-20T18:32:57.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.159+0000 7f038ffff640 1 -- 172.21.15.5:0/2579688309 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f039801a070 con 0x7f03a806d910 2024-05-20T18:32:57.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.160+0000 7f038ffff640 1 --2- 172.21.15.5:0/2579688309 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f039406be30 0x7f039406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:57.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.160+0000 7f03a6ffd640 1 --2- 172.21.15.5:0/2579688309 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f039406be30 0x7f039406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:57.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.160+0000 7f038ffff640 1 -- 172.21.15.5:0/2579688309 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f039808afd0 con 0x7f03a806d910 2024-05-20T18:32:57.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.160+0000 7f03a6ffd640 1 --2- 172.21.15.5:0/2579688309 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f039406be30 0x7f039406e340 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f039c034000 tx=0x7f039c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:57.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.165+0000 7f038ffff640 1 -- 172.21.15.5:0/2579688309 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f03a8073a00 con 0x7f03a806d910 2024-05-20T18:32:57.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.346+0000 7f03adc8b640 1 -- 172.21.15.5:0/2579688309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f03a806df00 con 0x7f03a806d910 2024-05-20T18:32:57.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.346+0000 7f038ffff640 1 -- 172.21.15.5:0/2579688309 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) ==== 104+0+11 (secure 0 0 0) 0x7f03a806df00 con 0x7f03a806d910 2024-05-20T18:32:57.347 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:32:57.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.348+0000 7f03adc8b640 1 -- 172.21.15.5:0/2579688309 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f039406be30 msgr2=0x7f039406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:57.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.348+0000 7f03adc8b640 1 --2- 172.21.15.5:0/2579688309 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f039406be30 0x7f039406e340 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f039c034000 tx=0x7f039c02e000 comp rx=0 tx=0).stop 2024-05-20T18:32:57.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.348+0000 7f03adc8b640 1 -- 172.21.15.5:0/2579688309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03a806d910 msgr2=0x7f03a81e9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:57.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.348+0000 7f03adc8b640 1 --2- 172.21.15.5:0/2579688309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03a806d910 0x7f03a81e9360 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f039800e990 tx=0x7f039800ee50 comp rx=0 tx=0).stop 2024-05-20T18:32:57.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.348+0000 7f03adc8b640 1 -- 172.21.15.5:0/2579688309 shutdown_connections 2024-05-20T18:32:57.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.348+0000 7f03adc8b640 1 --2- 172.21.15.5:0/2579688309 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f039406be30 0x7f039406e340 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:57.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.348+0000 7f03adc8b640 1 --2- 172.21.15.5:0/2579688309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03a8162820 0x7f03a80ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:57.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.348+0000 7f03adc8b640 1 --2- 172.21.15.5:0/2579688309 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03a80b3e80 0x7f03a81f85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:57.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.348+0000 7f03adc8b640 1 --2- 172.21.15.5:0/2579688309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03a806d910 0x7f03a81e9360 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:57.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.348+0000 7f03adc8b640 1 -- 172.21.15.5:0/2579688309 >> 172.21.15.5:0/2579688309 conn(0x7f03a80d4bd0 msgr2=0x7f03a80b8190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:57.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.348+0000 7f03adc8b640 1 -- 172.21.15.5:0/2579688309 shutdown_connections 2024-05-20T18:32:57.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.348+0000 7f03adc8b640 1 -- 172.21.15.5:0/2579688309 wait complete. 2024-05-20T18:32:57.360 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:32:57.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.477+0000 7ff7a573e640 1 Processor -- start 2024-05-20T18:32:57.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.477+0000 7ff7a573e640 1 -- start start 2024-05-20T18:32:57.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.478+0000 7ff7a573e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff7a00afad0 0x7ff7a00b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:57.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.478+0000 7ff7a573e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7a00afee0 0x7ff7a00b5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:57.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.478+0000 7ff7a573e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7a00b3aa0 con 0x7ff7a015ca50 2024-05-20T18:32:57.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.478+0000 7ff7a573e640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff7a00b3850 con 0x7ff7a00afad0 2024-05-20T18:32:57.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.478+0000 7ff7a573e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff7a00703e0 con 0x7ff7a00afee0 2024-05-20T18:32:57.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.478+0000 7ff79f7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff7a015ca50 0x7ff7a015ce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46838/0 (socket says 172.21.15.5:46838) 2024-05-20T18:32:57.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.479+0000 7ff79f7fe640 1 -- 172.21.15.5:0/1618889019 learned_addr learned my addr 172.21.15.5:0/1618889019 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:32:57.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.479+0000 7ff79e7fc640 1 --2- 172.21.15.5:0/1618889019 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7a00afee0 0x7ff7a00b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:57.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.479+0000 7ff79effd640 1 --2- 172.21.15.5:0/1618889019 >> v2:172.21.15.63:3300/0 conn(0x7ff7a00afad0 0x7ff7a00b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:57.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.479+0000 7ff79dffb640 1 -- 172.21.15.5:0/1618889019 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1525336231 0 0) 0x7ff7a00b3aa0 con 0x7ff7a015ca50 2024-05-20T18:32:57.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.479+0000 7ff79dffb640 1 -- 172.21.15.5:0/1618889019 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff788003520 con 0x7ff7a015ca50 2024-05-20T18:32:57.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.479+0000 7ff79e7fc640 1 -- 172.21.15.5:0/1618889019 >> v2:172.21.15.63:3300/0 conn(0x7ff7a00afad0 msgr2=0x7ff7a00b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:57.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.479+0000 7ff79e7fc640 1 --2- 172.21.15.5:0/1618889019 >> v2:172.21.15.63:3300/0 conn(0x7ff7a00afad0 0x7ff7a00b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:57.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.479+0000 7ff79e7fc640 1 -- 172.21.15.5:0/1618889019 >> v1:172.21.15.5:6789/0 conn(0x7ff7a015ca50 legacy=0x7ff7a015ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:57.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.479+0000 7ff79e7fc640 1 -- 172.21.15.5:0/1618889019 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a00b3d80 con 0x7ff7a00afee0 2024-05-20T18:32:57.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.480+0000 7ff79e7fc640 1 --2- 172.21.15.5:0/1618889019 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7a00afee0 0x7ff7a00b5720 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff7940097a0 tx=0x7ff79402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=352b4c6a069f0e7a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:57.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.480+0000 7ff79dffb640 1 -- 172.21.15.5:0/1618889019 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff794037070 con 0x7ff7a00afee0 2024-05-20T18:32:57.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.480+0000 7ff79dffb640 1 -- 172.21.15.5:0/1618889019 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff79402ae50 con 0x7ff7a00afee0 2024-05-20T18:32:57.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.480+0000 7ff79dffb640 1 -- 172.21.15.5:0/1618889019 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff794032640 con 0x7ff7a00afee0 2024-05-20T18:32:57.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.481+0000 7ff7a573e640 1 -- 172.21.15.5:0/1618889019 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7a00afee0 msgr2=0x7ff7a00b5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:57.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.481+0000 7ff7a573e640 1 --2- 172.21.15.5:0/1618889019 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7a00afee0 0x7ff7a00b5720 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff7940097a0 tx=0x7ff79402a100 comp rx=0 tx=0).stop 2024-05-20T18:32:57.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.481+0000 7ff7a573e640 1 -- 172.21.15.5:0/1618889019 shutdown_connections 2024-05-20T18:32:57.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.481+0000 7ff7a573e640 1 --2- 172.21.15.5:0/1618889019 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7a00afee0 0x7ff7a00b5720 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:57.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.481+0000 7ff7a573e640 1 --2- 172.21.15.5:0/1618889019 >> v2:172.21.15.63:3300/0 conn(0x7ff7a00afad0 0x7ff7a00b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:57.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.481+0000 7ff7a573e640 1 -- 172.21.15.5:0/1618889019 >> 172.21.15.5:0/1618889019 conn(0x7ff7a00cd2a0 msgr2=0x7ff7a00cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:57.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.481+0000 7ff7a573e640 1 -- 172.21.15.5:0/1618889019 shutdown_connections 2024-05-20T18:32:57.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.481+0000 7ff7a573e640 1 -- 172.21.15.5:0/1618889019 wait complete. 2024-05-20T18:32:57.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.483+0000 7ff7a573e640 1 Processor -- start 2024-05-20T18:32:57.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.483+0000 7ff7a573e640 1 -- start start 2024-05-20T18:32:57.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.484+0000 7ff7a573e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7a00afad0 0x7ff7a01513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:57.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.484+0000 7ff7a573e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7a00afee0 0x7ff7a0154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:57.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.484+0000 7ff7a573e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7a015ca50 0x7ff7a014f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:57.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.484+0000 7ff7a573e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a00b4860 con 0x7ff7a00afad0 2024-05-20T18:32:57.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.484+0000 7ff7a573e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a00b4340 con 0x7ff7a015ca50 2024-05-20T18:32:57.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.484+0000 7ff7a573e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff7a0150c60 con 0x7ff7a00afee0 2024-05-20T18:32:57.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.484+0000 7ff79effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7a00afad0 0x7ff7a01513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:57.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.484+0000 7ff79effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7a00afad0 0x7ff7a01513b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44656/0 (socket says 172.21.15.5:44656) 2024-05-20T18:32:57.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.484+0000 7ff79effd640 1 -- 172.21.15.5:0/2432934275 learned_addr learned my addr 172.21.15.5:0/2432934275 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:57.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.484+0000 7ff79e7fc640 1 --2- 172.21.15.5:0/2432934275 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7a00afee0 0x7ff7a0154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:57.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.485+0000 7ff79effd640 1 -- 172.21.15.5:0/2432934275 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7a00afee0 msgr2=0x7ff7a0154220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:57.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.485+0000 7ff79f7fe640 1 --2- 172.21.15.5:0/2432934275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7a015ca50 0x7ff7a014f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:57.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.485+0000 7ff79effd640 1 --2- 172.21.15.5:0/2432934275 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7a00afee0 0x7ff7a0154220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:57.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.485+0000 7ff79effd640 1 -- 172.21.15.5:0/2432934275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7a015ca50 msgr2=0x7ff7a014f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:57.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.485+0000 7ff79effd640 1 --2- 172.21.15.5:0/2432934275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7a015ca50 0x7ff7a014f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:57.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.485+0000 7ff79effd640 1 -- 172.21.15.5:0/2432934275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a0154ea0 con 0x7ff7a00afad0 2024-05-20T18:32:57.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.485+0000 7ff79effd640 1 --2- 172.21.15.5:0/2432934275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7a00afad0 0x7ff7a01513b0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7ff790010a80 tx=0x7ff790010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:57.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.485+0000 7ff7877fe640 1 -- 172.21.15.5:0/2432934275 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79001e070 con 0x7ff7a00afad0 2024-05-20T18:32:57.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.485+0000 7ff7877fe640 1 -- 172.21.15.5:0/2432934275 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff790011450 con 0x7ff7a00afad0 2024-05-20T18:32:57.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.485+0000 7ff7877fe640 1 -- 172.21.15.5:0/2432934275 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff790011820 con 0x7ff7a00afad0 2024-05-20T18:32:57.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.485+0000 7ff7a573e640 1 -- 172.21.15.5:0/2432934275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a0155180 con 0x7ff7a00afad0 2024-05-20T18:32:57.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.485+0000 7ff7a573e640 1 -- 172.21.15.5:0/2432934275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a014f400 con 0x7ff7a00afad0 2024-05-20T18:32:57.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.486+0000 7ff7a573e640 1 -- 172.21.15.5:0/2432934275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7a0075b00 con 0x7ff7a00afad0 2024-05-20T18:32:57.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.487+0000 7ff7877fe640 1 -- 172.21.15.5:0/2432934275 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff790021070 con 0x7ff7a00afad0 2024-05-20T18:32:57.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.487+0000 7ff7877fe640 1 --2- 172.21.15.5:0/2432934275 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff78c06be30 0x7ff78c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:57.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.487+0000 7ff7877fe640 1 -- 172.21.15.5:0/2432934275 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7ff79008d0c0 con 0x7ff7a00afad0 2024-05-20T18:32:57.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.487+0000 7ff79e7fc640 1 --2- 172.21.15.5:0/2432934275 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff78c06be30 0x7ff78c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:57.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.488+0000 7ff79e7fc640 1 --2- 172.21.15.5:0/2432934275 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff78c06be30 0x7ff78c06e340 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff794034000 tx=0x7ff79402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:57.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.492+0000 7ff7877fe640 1 -- 172.21.15.5:0/2432934275 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff79005b240 con 0x7ff7a00afad0 2024-05-20T18:32:57.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.678+0000 7ff7a573e640 1 -- 172.21.15.5:0/2432934275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff7a015c270 con 0x7ff7a00afad0 2024-05-20T18:32:57.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.678+0000 7ff7877fe640 1 -- 172.21.15.5:0/2432934275 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) ==== 100+0+11 (secure 0 0 0) 0x7ff79005f540 con 0x7ff7a00afad0 2024-05-20T18:32:57.679 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:32:57.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.680+0000 7ff7a573e640 1 -- 172.21.15.5:0/2432934275 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff78c06be30 msgr2=0x7ff78c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:57.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.680+0000 7ff7a573e640 1 --2- 172.21.15.5:0/2432934275 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff78c06be30 0x7ff78c06e340 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff794034000 tx=0x7ff79402e000 comp rx=0 tx=0).stop 2024-05-20T18:32:57.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.680+0000 7ff7a573e640 1 -- 172.21.15.5:0/2432934275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7a00afad0 msgr2=0x7ff7a01513b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:57.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.680+0000 7ff7a573e640 1 --2- 172.21.15.5:0/2432934275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7a00afad0 0x7ff7a01513b0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7ff790010a80 tx=0x7ff790010f40 comp rx=0 tx=0).stop 2024-05-20T18:32:57.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.680+0000 7ff7a573e640 1 -- 172.21.15.5:0/2432934275 shutdown_connections 2024-05-20T18:32:57.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.680+0000 7ff7a573e640 1 --2- 172.21.15.5:0/2432934275 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff78c06be30 0x7ff78c06e340 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:57.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.680+0000 7ff7a573e640 1 --2- 172.21.15.5:0/2432934275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7a015ca50 0x7ff7a014f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:57.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.680+0000 7ff7a573e640 1 --2- 172.21.15.5:0/2432934275 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7a00afee0 0x7ff7a0154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:57.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.680+0000 7ff7a573e640 1 --2- 172.21.15.5:0/2432934275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7a00afad0 0x7ff7a01513b0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:57.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.680+0000 7ff7a573e640 1 -- 172.21.15.5:0/2432934275 >> 172.21.15.5:0/2432934275 conn(0x7ff7a00cd2a0 msgr2=0x7ff7a0158680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:57.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.680+0000 7ff7a573e640 1 -- 172.21.15.5:0/2432934275 shutdown_connections 2024-05-20T18:32:57.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.680+0000 7ff7a573e640 1 -- 172.21.15.5:0/2432934275 wait complete. 2024-05-20T18:32:57.693 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:32:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.808+0000 7f3c1af5e640 1 Processor -- start 2024-05-20T18:32:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.808+0000 7f3c1af5e640 1 -- start start 2024-05-20T18:32:57.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.810+0000 7f3c1af5e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c1406d3d0 0x7f3c1406d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:57.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.810+0000 7f3c1af5e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c1406dd20 con 0x7f3c140b8190 2024-05-20T18:32:57.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.810+0000 7f3c1af5e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c140d0630 con 0x7f3c14162820 2024-05-20T18:32:57.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.810+0000 7f3c1af5e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3c140d39c0 con 0x7f3c1406d3d0 2024-05-20T18:32:57.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.810+0000 7f3c194fb640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3c140b8190 0x7f3c140b85a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46844/0 (socket says 172.21.15.5:46844) 2024-05-20T18:32:57.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.810+0000 7f3c194fb640 1 -- 172.21.15.5:0/3971114728 learned_addr learned my addr 172.21.15.5:0/3971114728 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:32:57.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.810+0000 7f3c13fff640 1 --2- 172.21.15.5:0/3971114728 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c1406d3d0 0x7f3c1406d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:57.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.810+0000 7f3c137fe640 1 -- 172.21.15.5:0/3971114728 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 771516474 0 0) 0x7f3c0c001490 con 0x7f3c140b8190 2024-05-20T18:32:57.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.810+0000 7f3c137fe640 1 -- 172.21.15.5:0/3971114728 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3bfc003520 con 0x7f3c140b8190 2024-05-20T18:32:57.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.810+0000 7f3c137fe640 1 -- 172.21.15.5:0/3971114728 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 965766296 0 0) 0x7f3c140d0630 con 0x7f3c14162820 2024-05-20T18:32:57.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.810+0000 7f3c137fe640 1 -- 172.21.15.5:0/3971114728 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c0c001490 con 0x7f3c14162820 2024-05-20T18:32:57.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.810+0000 7f3c13fff640 1 -- 172.21.15.5:0/3971114728 >> v1:172.21.15.63:6789/0 conn(0x7f3c14162820 legacy=0x7f3c140b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:57.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.810+0000 7f3c13fff640 1 -- 172.21.15.5:0/3971114728 >> v1:172.21.15.5:6789/0 conn(0x7f3c140b8190 legacy=0x7f3c140b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:57.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.810+0000 7f3c13fff640 1 -- 172.21.15.5:0/3971114728 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c140d3ca0 con 0x7f3c1406d3d0 2024-05-20T18:32:57.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.811+0000 7f3c13fff640 1 --2- 172.21.15.5:0/3971114728 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c1406d3d0 0x7f3c1406d7e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3c080097d0 tx=0x7f3c0802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=126c3d16368c808e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:57.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.811+0000 7f3c137fe640 1 -- 172.21.15.5:0/3971114728 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08037070 con 0x7f3c1406d3d0 2024-05-20T18:32:57.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.811+0000 7f3c137fe640 1 -- 172.21.15.5:0/3971114728 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c08032440 con 0x7f3c1406d3d0 2024-05-20T18:32:57.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.811+0000 7f3c137fe640 1 -- 172.21.15.5:0/3971114728 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c080327f0 con 0x7f3c1406d3d0 2024-05-20T18:32:57.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.812+0000 7f3c1af5e640 1 -- 172.21.15.5:0/3971114728 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c1406d3d0 msgr2=0x7f3c1406d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:57.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.812+0000 7f3c1af5e640 1 --2- 172.21.15.5:0/3971114728 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c1406d3d0 0x7f3c1406d7e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3c080097d0 tx=0x7f3c0802a210 comp rx=0 tx=0).stop 2024-05-20T18:32:57.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.812+0000 7f3c1af5e640 1 -- 172.21.15.5:0/3971114728 shutdown_connections 2024-05-20T18:32:57.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.812+0000 7f3c1af5e640 1 --2- 172.21.15.5:0/3971114728 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c1406d3d0 0x7f3c1406d7e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:57.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.812+0000 7f3c1af5e640 1 -- 172.21.15.5:0/3971114728 >> 172.21.15.5:0/3971114728 conn(0x7f3c140d4bd0 msgr2=0x7f3c140d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:57.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.812+0000 7f3c1af5e640 1 -- 172.21.15.5:0/3971114728 shutdown_connections 2024-05-20T18:32:57.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.812+0000 7f3c1af5e640 1 -- 172.21.15.5:0/3971114728 wait complete. 2024-05-20T18:32:57.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.814+0000 7f3c1af5e640 1 Processor -- start 2024-05-20T18:32:57.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.814+0000 7f3c1af5e640 1 -- start start 2024-05-20T18:32:57.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.815+0000 7f3c1af5e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c1406d3d0 0x7f3c140cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:57.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.815+0000 7f3c1af5e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c140b8190 0x7f3c141591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:57.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.815+0000 7f3c1af5e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c14162820 0x7f3c14159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:57.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.815+0000 7f3c1af5e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3c140d0300 con 0x7f3c1406d3d0 2024-05-20T18:32:57.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.815+0000 7f3c1af5e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3c140d4260 con 0x7f3c14162820 2024-05-20T18:32:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.815+0000 7f3c1af5e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3c140b4050 con 0x7f3c140b8190 2024-05-20T18:32:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.816+0000 7f3c18cfa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c1406d3d0 0x7f3c140cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.816+0000 7f3c18cfa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c1406d3d0 0x7f3c140cacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44660/0 (socket says 172.21.15.5:44660) 2024-05-20T18:32:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.816+0000 7f3c18cfa640 1 -- 172.21.15.5:0/2555080146 learned_addr learned my addr 172.21.15.5:0/2555080146 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.816+0000 7f3c194fb640 1 --2- 172.21.15.5:0/2555080146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c14162820 0x7f3c14159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.816+0000 7f3c13fff640 1 --2- 172.21.15.5:0/2555080146 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c140b8190 0x7f3c141591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.816+0000 7f3c18cfa640 1 -- 172.21.15.5:0/2555080146 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c140b8190 msgr2=0x7f3c141591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.816+0000 7f3c18cfa640 1 --2- 172.21.15.5:0/2555080146 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c140b8190 0x7f3c141591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.816+0000 7f3c18cfa640 1 -- 172.21.15.5:0/2555080146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c14162820 msgr2=0x7f3c14159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.816+0000 7f3c18cfa640 1 --2- 172.21.15.5:0/2555080146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c14162820 0x7f3c14159730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.816+0000 7f3c18cfa640 1 -- 172.21.15.5:0/2555080146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c140cb4d0 con 0x7f3c1406d3d0 2024-05-20T18:32:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.816+0000 7f3c18cfa640 1 --2- 172.21.15.5:0/2555080146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c1406d3d0 0x7f3c140cacb0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400c440 tx=0x7f3c0400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.816+0000 7f3c117fa640 1 -- 172.21.15.5:0/2555080146 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c040052d0 con 0x7f3c1406d3d0 2024-05-20T18:32:57.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.816+0000 7f3c117fa640 1 -- 172.21.15.5:0/2555080146 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c04005490 con 0x7f3c1406d3d0 2024-05-20T18:32:57.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.816+0000 7f3c1af5e640 1 -- 172.21.15.5:0/2555080146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c140cb7b0 con 0x7f3c1406d3d0 2024-05-20T18:32:57.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.816+0000 7f3c117fa640 1 -- 172.21.15.5:0/2555080146 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04012870 con 0x7f3c1406d3d0 2024-05-20T18:32:57.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.816+0000 7f3c1af5e640 1 -- 172.21.15.5:0/2555080146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c140ca120 con 0x7f3c1406d3d0 2024-05-20T18:32:57.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.817+0000 7f3c1af5e640 1 -- 172.21.15.5:0/2555080146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c14075b00 con 0x7f3c1406d3d0 2024-05-20T18:32:57.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.818+0000 7f3c117fa640 1 -- 172.21.15.5:0/2555080146 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3c04012a10 con 0x7f3c1406d3d0 2024-05-20T18:32:57.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.819+0000 7f3c117fa640 1 --2- 172.21.15.5:0/2555080146 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3c0006be60 0x7f3c0006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:57.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.819+0000 7f3c117fa640 1 -- 172.21.15.5:0/2555080146 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f3c0408ac80 con 0x7f3c1406d3d0 2024-05-20T18:32:57.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.819+0000 7f3c13fff640 1 --2- 172.21.15.5:0/2555080146 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3c0006be60 0x7f3c0006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:57.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.819+0000 7f3c13fff640 1 --2- 172.21.15.5:0/2555080146 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3c0006be60 0x7f3c0006e370 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3c08034000 tx=0x7f3c0802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:57.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:57.823+0000 7f3c117fa640 1 -- 172.21.15.5:0/2555080146 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3c04059480 con 0x7f3c1406d3d0 2024-05-20T18:32:58.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.043+0000 7f3c1af5e640 1 -- 172.21.15.5:0/2555080146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c14160530 con 0x7f3c1406d3d0 2024-05-20T18:32:58.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.044+0000 7f3c117fa640 1 -- 172.21.15.5:0/2555080146 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5176 (secure 0 0 0) 0x7f3c04058d80 con 0x7f3c1406d3d0 2024-05-20T18:32:58.045 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:32:58.045 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1890759603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1890759603},{"type":"v1","addr":"172.21.15.63:6834","nonce":1890759603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5825,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/700447403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":700447403},{"type":"v1","addr":"172.21.15.63:6835","nonce":700447403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3382943275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3382943275},{"type":"v1","addr":"172.21.15.5:6837","nonce":3382943275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:31:30.164179+0000","modified":"2024-05-20T18:32:27.855796+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6972":{"gid":6972,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":15,"addr":"172.21.15.5:6836/3866822752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":3866822752},{"type":"v1","addr":"172.21.15.5:6836","nonce":3866822752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6972,"qdb_cluster":[6972]},"id":5}]} 2024-05-20T18:32:58.045 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 52 2024-05-20T18:32:58.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.046+0000 7f3c1af5e640 1 -- 172.21.15.5:0/2555080146 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3c0006be60 msgr2=0x7f3c0006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:58.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.046+0000 7f3c1af5e640 1 --2- 172.21.15.5:0/2555080146 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3c0006be60 0x7f3c0006e370 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3c08034000 tx=0x7f3c0802e000 comp rx=0 tx=0).stop 2024-05-20T18:32:58.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.046+0000 7f3c1af5e640 1 -- 172.21.15.5:0/2555080146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c1406d3d0 msgr2=0x7f3c140cacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:58.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.046+0000 7f3c1af5e640 1 --2- 172.21.15.5:0/2555080146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c1406d3d0 0x7f3c140cacb0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400c440 tx=0x7f3c0400c900 comp rx=0 tx=0).stop 2024-05-20T18:32:58.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.046+0000 7f3c1af5e640 1 -- 172.21.15.5:0/2555080146 shutdown_connections 2024-05-20T18:32:58.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.046+0000 7f3c1af5e640 1 --2- 172.21.15.5:0/2555080146 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3c0006be60 0x7f3c0006e370 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:58.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.046+0000 7f3c1af5e640 1 --2- 172.21.15.5:0/2555080146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c14162820 0x7f3c14159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:58.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.046+0000 7f3c1af5e640 1 --2- 172.21.15.5:0/2555080146 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c140b8190 0x7f3c141591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:58.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.046+0000 7f3c1af5e640 1 --2- 172.21.15.5:0/2555080146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c1406d3d0 0x7f3c140cacb0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:58.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.046+0000 7f3c1af5e640 1 -- 172.21.15.5:0/2555080146 >> 172.21.15.5:0/2555080146 conn(0x7f3c140d4bd0 msgr2=0x7f3c1415f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:58.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.046+0000 7f3c1af5e640 1 -- 172.21.15.5:0/2555080146 shutdown_connections 2024-05-20T18:32:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.046+0000 7f3c1af5e640 1 -- 172.21.15.5:0/2555080146 wait complete. 2024-05-20T18:32:58.058 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:32:58.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.172+0000 7f75dff45640 1 Processor -- start 2024-05-20T18:32:58.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.172+0000 7f75dff45640 1 -- start start 2024-05-20T18:32:58.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.174+0000 7f75dff45640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f75d80d10e0 0x7f75d80cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:58.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.174+0000 7f75dff45640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f75d80b7210 0x7f75d80cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:58.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.174+0000 7f75dff45640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75d80d1ac0 con 0x7f75d80d14f0 2024-05-20T18:32:58.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.174+0000 7f75dff45640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f75d80d1da0 con 0x7f75d80d10e0 2024-05-20T18:32:58.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.174+0000 7f75dff45640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f75d80d2080 con 0x7f75d80b7210 2024-05-20T18:32:58.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.174+0000 7f75ddce1640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f75d80d10e0 0x7f75d80cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:58.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.174+0000 7f75ddce1640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f75d80d10e0 0x7f75d80cf140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44674/0 (socket says 172.21.15.5:44674) 2024-05-20T18:32:58.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.174+0000 7f75ddce1640 1 -- 172.21.15.5:0/4120261062 learned_addr learned my addr 172.21.15.5:0/4120261062 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:58.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.174+0000 7f75de4e2640 1 --2- 172.21.15.5:0/4120261062 >> v2:172.21.15.63:3300/0 conn(0x7f75d80b7210 0x7f75d80cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:58.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.174+0000 7f75ddce1640 1 -- 172.21.15.5:0/4120261062 >> v2:172.21.15.63:3300/0 conn(0x7f75d80b7210 msgr2=0x7f75d80cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:58.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.174+0000 7f75ddce1640 1 --2- 172.21.15.5:0/4120261062 >> v2:172.21.15.63:3300/0 conn(0x7f75d80b7210 0x7f75d80cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:58.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.174+0000 7f75ddce1640 1 -- 172.21.15.5:0/4120261062 >> v1:172.21.15.63:6789/0 conn(0x7f75d80d14f0 legacy=0x7f75d80cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:58.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.174+0000 7f75ddce1640 1 -- 172.21.15.5:0/4120261062 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d80d2360 con 0x7f75d80d10e0 2024-05-20T18:32:58.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.175+0000 7f75ddce1640 1 --2- 172.21.15.5:0/4120261062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75d80d10e0 0x7f75d80cf140 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f75d80b58a0 tx=0x7f75c802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c6d350139bc09c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:58.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.175+0000 7f75dccdf640 1 -- 172.21.15.5:0/4120261062 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8036070 con 0x7f75d80d10e0 2024-05-20T18:32:58.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.175+0000 7f75dccdf640 1 -- 172.21.15.5:0/4120261062 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c8031600 con 0x7f75d80d10e0 2024-05-20T18:32:58.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.175+0000 7f75dccdf640 1 -- 172.21.15.5:0/4120261062 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c80319d0 con 0x7f75d80d10e0 2024-05-20T18:32:58.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.176+0000 7f75dff45640 1 -- 172.21.15.5:0/4120261062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75d80d10e0 msgr2=0x7f75d80cf140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:58.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.176+0000 7f75dff45640 1 --2- 172.21.15.5:0/4120261062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75d80d10e0 0x7f75d80cf140 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f75d80b58a0 tx=0x7f75c802a6f0 comp rx=0 tx=0).stop 2024-05-20T18:32:58.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.176+0000 7f75dff45640 1 -- 172.21.15.5:0/4120261062 shutdown_connections 2024-05-20T18:32:58.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.176+0000 7f75dff45640 1 --2- 172.21.15.5:0/4120261062 >> v2:172.21.15.63:3300/0 conn(0x7f75d80b7210 0x7f75d80cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:58.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.176+0000 7f75dff45640 1 --2- 172.21.15.5:0/4120261062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75d80d10e0 0x7f75d80cf140 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:58.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.176+0000 7f75dff45640 1 -- 172.21.15.5:0/4120261062 >> 172.21.15.5:0/4120261062 conn(0x7f75d80b44d0 msgr2=0x7f75d80b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:58.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.176+0000 7f75dff45640 1 -- 172.21.15.5:0/4120261062 shutdown_connections 2024-05-20T18:32:58.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.176+0000 7f75dff45640 1 -- 172.21.15.5:0/4120261062 wait complete. 2024-05-20T18:32:58.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.178+0000 7f75dff45640 1 Processor -- start 2024-05-20T18:32:58.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.178+0000 7f75dff45640 1 -- start start 2024-05-20T18:32:58.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.179+0000 7f75dff45640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75d80b7210 0x7f75d81e58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:58.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.179+0000 7f75dff45640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75d80d10e0 0x7f75d815f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:58.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.179+0000 7f75dff45640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f75d80d14f0 0x7f75d814eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:58.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.179+0000 7f75dff45640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f75d80b8440 con 0x7f75d80b7210 2024-05-20T18:32:58.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.179+0000 7f75dff45640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f75d80b82c0 con 0x7f75d80d10e0 2024-05-20T18:32:58.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.179+0000 7f75dff45640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f75d80709f0 con 0x7f75d80d14f0 2024-05-20T18:32:58.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.179+0000 7f75ddce1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75d80b7210 0x7f75d81e58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:58.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.179+0000 7f75ddce1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75d80b7210 0x7f75d81e58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44676/0 (socket says 172.21.15.5:44676) 2024-05-20T18:32:58.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.179+0000 7f75ddce1640 1 -- 172.21.15.5:0/2079840572 learned_addr learned my addr 172.21.15.5:0/2079840572 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.180+0000 7f75dd4e0640 1 --2- 172.21.15.5:0/2079840572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75d80d10e0 0x7f75d815f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.180+0000 7f75de4e2640 1 --2- 172.21.15.5:0/2079840572 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f75d80d14f0 0x7f75d814eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.180+0000 7f75ddce1640 1 -- 172.21.15.5:0/2079840572 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f75d80d14f0 msgr2=0x7f75d814eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.180+0000 7f75ddce1640 1 --2- 172.21.15.5:0/2079840572 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f75d80d14f0 0x7f75d814eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.180+0000 7f75ddce1640 1 -- 172.21.15.5:0/2079840572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75d80d10e0 msgr2=0x7f75d815f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.180+0000 7f75ddce1640 1 --2- 172.21.15.5:0/2079840572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75d80d10e0 0x7f75d815f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.180+0000 7f75ddce1640 1 -- 172.21.15.5:0/2079840572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d81e6130 con 0x7f75d80b7210 2024-05-20T18:32:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.180+0000 7f75ddce1640 1 --2- 172.21.15.5:0/2079840572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75d80b7210 0x7f75d81e58e0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009f40 tx=0x7f75c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.180+0000 7f75d67fc640 1 -- 172.21.15.5:0/2079840572 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8036050 con 0x7f75d80b7210 2024-05-20T18:32:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.180+0000 7f75d67fc640 1 -- 172.21.15.5:0/2079840572 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c8042070 con 0x7f75d80b7210 2024-05-20T18:32:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.180+0000 7f75d67fc640 1 -- 172.21.15.5:0/2079840572 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8009510 con 0x7f75d80b7210 2024-05-20T18:32:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.180+0000 7f75dff45640 1 -- 172.21.15.5:0/2079840572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d81e6410 con 0x7f75d80b7210 2024-05-20T18:32:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.180+0000 7f75dff45640 1 -- 172.21.15.5:0/2079840572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d81e69d0 con 0x7f75d80b7210 2024-05-20T18:32:58.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.181+0000 7f75dff45640 1 -- 172.21.15.5:0/2079840572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75d8075b00 con 0x7f75d80b7210 2024-05-20T18:32:58.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.182+0000 7f75d67fc640 1 -- 172.21.15.5:0/2079840572 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f75c8007c50 con 0x7f75d80b7210 2024-05-20T18:32:58.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.182+0000 7f75d67fc640 1 --2- 172.21.15.5:0/2079840572 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f75c406be60 0x7f75c406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:58.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.182+0000 7f75d67fc640 1 -- 172.21.15.5:0/2079840572 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f75c80ac030 con 0x7f75d80b7210 2024-05-20T18:32:58.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.182+0000 7f75dd4e0640 1 --2- 172.21.15.5:0/2079840572 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f75c406be60 0x7f75c406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:58.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.183+0000 7f75dd4e0640 1 --2- 172.21.15.5:0/2079840572 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f75c406be60 0x7f75c406e370 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f75cc006fd0 tx=0x7f75cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:58.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.187+0000 7f75d67fc640 1 -- 172.21.15.5:0/2079840572 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f75c807a770 con 0x7f75d80b7210 2024-05-20T18:32:58.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.372+0000 7f75dff45640 1 -- 172.21.15.5:0/2079840572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f75d815d090 con 0x7f75d80b7210 2024-05-20T18:32:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.374+0000 7f75d67fc640 1 -- 172.21.15.5:0/2079840572 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+16950 (secure 0 0 0) 0x7f75c807a070 con 0x7f75d80b7210 2024-05-20T18:32:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:32:58.375 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":78,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:31:40.839398+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:31:27.896386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-20T18:31:29.146912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6836/791176355":"2024-05-21T18:31:40.839373+0000","172.21.15.5:6835/791176355":"2024-05-21T18:31:40.839373+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:32:58.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.376+0000 7f75dff45640 1 -- 172.21.15.5:0/2079840572 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f75c406be60 msgr2=0x7f75c406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:58.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.376+0000 7f75dff45640 1 --2- 172.21.15.5:0/2079840572 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f75c406be60 0x7f75c406e370 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f75cc006fd0 tx=0x7f75cc008040 comp rx=0 tx=0).stop 2024-05-20T18:32:58.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.376+0000 7f75dff45640 1 -- 172.21.15.5:0/2079840572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75d80b7210 msgr2=0x7f75d81e58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:58.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.376+0000 7f75dff45640 1 --2- 172.21.15.5:0/2079840572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75d80b7210 0x7f75d81e58e0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009f40 tx=0x7f75c8004a40 comp rx=0 tx=0).stop 2024-05-20T18:32:58.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.376+0000 7f75dff45640 1 -- 172.21.15.5:0/2079840572 shutdown_connections 2024-05-20T18:32:58.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.376+0000 7f75dff45640 1 --2- 172.21.15.5:0/2079840572 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f75c406be60 0x7f75c406e370 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:58.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.376+0000 7f75dff45640 1 --2- 172.21.15.5:0/2079840572 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f75d80d14f0 0x7f75d814eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:58.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.376+0000 7f75dff45640 1 --2- 172.21.15.5:0/2079840572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75d80d10e0 0x7f75d815f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:58.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.376+0000 7f75dff45640 1 --2- 172.21.15.5:0/2079840572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75d80b7210 0x7f75d81e58e0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:58.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.376+0000 7f75dff45640 1 -- 172.21.15.5:0/2079840572 >> 172.21.15.5:0/2079840572 conn(0x7f75d80b44d0 msgr2=0x7f75d80cdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:58.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.376+0000 7f75dff45640 1 -- 172.21.15.5:0/2079840572 shutdown_connections 2024-05-20T18:32:58.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.376+0000 7f75dff45640 1 -- 172.21.15.5:0/2079840572 wait complete. 2024-05-20T18:32:58.388 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-20T18:32:58.388 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:32:58.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.502+0000 7fbbd326c640 1 Processor -- start 2024-05-20T18:32:58.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.502+0000 7fbbd326c640 1 -- start start 2024-05-20T18:32:58.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.503+0000 7fbbd326c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbbcc15d530 0x7fbbcc15aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:58.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.504+0000 7fbbd326c640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbcc0cd580 con 0x7fbbcc070850 2024-05-20T18:32:58.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.504+0000 7fbbd326c640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbcc0cd860 con 0x7fbbcc0b1700 2024-05-20T18:32:58.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.504+0000 7fbbd326c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbbcc0ce190 con 0x7fbbcc15d530 2024-05-20T18:32:58.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.504+0000 7fbbd1809640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbbcc070850 0x7fbbcc070c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46846/0 (socket says 172.21.15.5:46846) 2024-05-20T18:32:58.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.504+0000 7fbbd1809640 1 -- 172.21.15.5:0/362773875 learned_addr learned my addr 172.21.15.5:0/362773875 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:32:58.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.504+0000 7fbbd1008640 1 --2- 172.21.15.5:0/362773875 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbbcc15d530 0x7fbbcc15aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:58.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.504+0000 7fbbc3fff640 1 -- 172.21.15.5:0/362773875 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2010924011 0 0) 0x7fbbcc0cd580 con 0x7fbbcc070850 2024-05-20T18:32:58.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.504+0000 7fbbc3fff640 1 -- 172.21.15.5:0/362773875 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbbb8003520 con 0x7fbbcc070850 2024-05-20T18:32:58.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.504+0000 7fbbc3fff640 1 -- 172.21.15.5:0/362773875 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2448931282 0 0) 0x7fbbcc0cd860 con 0x7fbbcc0b1700 2024-05-20T18:32:58.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.504+0000 7fbbc3fff640 1 -- 172.21.15.5:0/362773875 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbbcc0cd580 con 0x7fbbcc0b1700 2024-05-20T18:32:58.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.504+0000 7fbbd1008640 1 -- 172.21.15.5:0/362773875 >> v1:172.21.15.63:6789/0 conn(0x7fbbcc0b1700 legacy=0x7fbbcc0b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:58.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.504+0000 7fbbd1008640 1 -- 172.21.15.5:0/362773875 >> v1:172.21.15.5:6789/0 conn(0x7fbbcc070850 legacy=0x7fbbcc070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:58.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.504+0000 7fbbd1008640 1 -- 172.21.15.5:0/362773875 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbcc0cdb40 con 0x7fbbcc15d530 2024-05-20T18:32:58.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.505+0000 7fbbd1008640 1 --2- 172.21.15.5:0/362773875 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbbcc15d530 0x7fbbcc15aaa0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbbc80097d0 tx=0x7fbbc802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed5507f21da8e788 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:58.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.505+0000 7fbbc3fff640 1 -- 172.21.15.5:0/362773875 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc8037070 con 0x7fbbcc15d530 2024-05-20T18:32:58.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.505+0000 7fbbc3fff640 1 -- 172.21.15.5:0/362773875 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbc8032440 con 0x7fbbcc15d530 2024-05-20T18:32:58.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.505+0000 7fbbc3fff640 1 -- 172.21.15.5:0/362773875 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc8032810 con 0x7fbbcc15d530 2024-05-20T18:32:58.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.506+0000 7fbbd326c640 1 -- 172.21.15.5:0/362773875 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbbcc15d530 msgr2=0x7fbbcc15aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:58.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.506+0000 7fbbd326c640 1 --2- 172.21.15.5:0/362773875 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbbcc15d530 0x7fbbcc15aaa0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbbc80097d0 tx=0x7fbbc802a210 comp rx=0 tx=0).stop 2024-05-20T18:32:58.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.506+0000 7fbbd326c640 1 -- 172.21.15.5:0/362773875 shutdown_connections 2024-05-20T18:32:58.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.506+0000 7fbbd326c640 1 --2- 172.21.15.5:0/362773875 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbbcc15d530 0x7fbbcc15aaa0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:58.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.506+0000 7fbbd326c640 1 -- 172.21.15.5:0/362773875 >> 172.21.15.5:0/362773875 conn(0x7fbbcc0d29d0 msgr2=0x7fbbcc0d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:58.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.506+0000 7fbbd326c640 1 -- 172.21.15.5:0/362773875 shutdown_connections 2024-05-20T18:32:58.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.506+0000 7fbbd326c640 1 -- 172.21.15.5:0/362773875 wait complete. 2024-05-20T18:32:58.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.508+0000 7fbbd326c640 1 Processor -- start 2024-05-20T18:32:58.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.508+0000 7fbbd326c640 1 -- start start 2024-05-20T18:32:58.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.509+0000 7fbbd326c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbcc070850 0x7fbbcc0c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:58.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.509+0000 7fbbd326c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbcc0b1700 0x7fbbcc154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:58.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.509+0000 7fbbd326c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbbcc15d530 0x7fbbcc15c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:58.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.509+0000 7fbbd326c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc071370 con 0x7fbbcc0b1700 2024-05-20T18:32:58.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.509+0000 7fbbd326c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc0cd230 con 0x7fbbcc070850 2024-05-20T18:32:58.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.509+0000 7fbbd326c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbbcc0714f0 con 0x7fbbcc15d530 2024-05-20T18:32:58.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.509+0000 7fbbd0807640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbcc0b1700 0x7fbbcc154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:58.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.509+0000 7fbbd0807640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbcc0b1700 0x7fbbcc154f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44684/0 (socket says 172.21.15.5:44684) 2024-05-20T18:32:58.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.509+0000 7fbbd0807640 1 -- 172.21.15.5:0/2035192573 learned_addr learned my addr 172.21.15.5:0/2035192573 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:58.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.510+0000 7fbbd1809640 1 --2- 172.21.15.5:0/2035192573 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbbcc15d530 0x7fbbcc15c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:58.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.510+0000 7fbbd1008640 1 --2- 172.21.15.5:0/2035192573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbcc070850 0x7fbbcc0c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:58.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.510+0000 7fbbd0807640 1 -- 172.21.15.5:0/2035192573 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbbcc15d530 msgr2=0x7fbbcc15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:58.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.510+0000 7fbbd0807640 1 --2- 172.21.15.5:0/2035192573 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbbcc15d530 0x7fbbcc15c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:58.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.510+0000 7fbbd0807640 1 -- 172.21.15.5:0/2035192573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbcc070850 msgr2=0x7fbbcc0c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:58.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.510+0000 7fbbd0807640 1 --2- 172.21.15.5:0/2035192573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbcc070850 0x7fbbcc0c6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:58.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.510+0000 7fbbd0807640 1 -- 172.21.15.5:0/2035192573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbcc0c6a40 con 0x7fbbcc0b1700 2024-05-20T18:32:58.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.510+0000 7fbbd0807640 1 --2- 172.21.15.5:0/2035192573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbcc0b1700 0x7fbbcc154f90 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc00c3f0 tx=0x7fbbbc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:58.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.510+0000 7fbbc1ffb640 1 -- 172.21.15.5:0/2035192573 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc017070 con 0x7fbbcc0b1700 2024-05-20T18:32:58.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.510+0000 7fbbc1ffb640 1 -- 172.21.15.5:0/2035192573 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbbc0052f0 con 0x7fbbcc0b1700 2024-05-20T18:32:58.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.510+0000 7fbbc1ffb640 1 -- 172.21.15.5:0/2035192573 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc003830 con 0x7fbbcc0b1700 2024-05-20T18:32:58.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.510+0000 7fbbd326c640 1 -- 172.21.15.5:0/2035192573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbcc15b110 con 0x7fbbcc0b1700 2024-05-20T18:32:58.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.511+0000 7fbbd326c640 1 -- 172.21.15.5:0/2035192573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbcc15b6d0 con 0x7fbbcc0b1700 2024-05-20T18:32:58.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.511+0000 7fbbd326c640 1 -- 172.21.15.5:0/2035192573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbcc075b00 con 0x7fbbcc0b1700 2024-05-20T18:32:58.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.512+0000 7fbbc1ffb640 1 -- 172.21.15.5:0/2035192573 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbbbc01a070 con 0x7fbbcc0b1700 2024-05-20T18:32:58.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.513+0000 7fbbc1ffb640 1 --2- 172.21.15.5:0/2035192573 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbbb006be30 0x7fbbb006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:58.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.513+0000 7fbbd1008640 1 --2- 172.21.15.5:0/2035192573 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbbb006be30 0x7fbbb006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:58.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.513+0000 7fbbc1ffb640 1 -- 172.21.15.5:0/2035192573 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7fbbbc08a3d0 con 0x7fbbcc0b1700 2024-05-20T18:32:58.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.513+0000 7fbbd1008640 1 --2- 172.21.15.5:0/2035192573 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbbb006be30 0x7fbbb006e340 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8034000 tx=0x7fbbc802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:58.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.517+0000 7fbbc1ffb640 1 -- 172.21.15.5:0/2035192573 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbbbc0584d0 con 0x7fbbcc0b1700 2024-05-20T18:32:58.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.744+0000 7fbbd326c640 1 -- 172.21.15.5:0/2035192573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbbcc0d2260 con 0x7fbbcc0b1700 2024-05-20T18:32:58.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.744+0000 7fbbc1ffb640 1 -- 172.21.15.5:0/2035192573 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5176 (secure 0 0 0) 0x7fbbbc05c7d0 con 0x7fbbcc0b1700 2024-05-20T18:32:58.745 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:32:58.745 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1890759603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1890759603},{"type":"v1","addr":"172.21.15.63:6834","nonce":1890759603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5825,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/700447403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":700447403},{"type":"v1","addr":"172.21.15.63:6835","nonce":700447403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3382943275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3382943275},{"type":"v1","addr":"172.21.15.5:6837","nonce":3382943275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:31:30.164179+0000","modified":"2024-05-20T18:32:27.855796+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6972":{"gid":6972,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":15,"addr":"172.21.15.5:6836/3866822752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":3866822752},{"type":"v1","addr":"172.21.15.5:6836","nonce":3866822752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6972,"qdb_cluster":[6972]},"id":5}]} 2024-05-20T18:32:58.746 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 52 2024-05-20T18:32:58.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.747+0000 7fbbd326c640 1 -- 172.21.15.5:0/2035192573 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbbb006be30 msgr2=0x7fbbb006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:58.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.747+0000 7fbbd326c640 1 --2- 172.21.15.5:0/2035192573 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbbb006be30 0x7fbbb006e340 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8034000 tx=0x7fbbc802e000 comp rx=0 tx=0).stop 2024-05-20T18:32:58.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.747+0000 7fbbd326c640 1 -- 172.21.15.5:0/2035192573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbcc0b1700 msgr2=0x7fbbcc154f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:58.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.747+0000 7fbbd326c640 1 --2- 172.21.15.5:0/2035192573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbcc0b1700 0x7fbbcc154f90 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc00c3f0 tx=0x7fbbbc00c8b0 comp rx=0 tx=0).stop 2024-05-20T18:32:58.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.747+0000 7fbbd326c640 1 -- 172.21.15.5:0/2035192573 shutdown_connections 2024-05-20T18:32:58.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.747+0000 7fbbd326c640 1 --2- 172.21.15.5:0/2035192573 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbbb006be30 0x7fbbb006e340 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:58.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.747+0000 7fbbd326c640 1 --2- 172.21.15.5:0/2035192573 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbbcc15d530 0x7fbbcc15c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:58.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.747+0000 7fbbd326c640 1 --2- 172.21.15.5:0/2035192573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbcc0b1700 0x7fbbcc154f90 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:58.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.747+0000 7fbbd326c640 1 --2- 172.21.15.5:0/2035192573 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbcc070850 0x7fbbcc0c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:58.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.747+0000 7fbbd326c640 1 -- 172.21.15.5:0/2035192573 >> 172.21.15.5:0/2035192573 conn(0x7fbbcc0d29d0 msgr2=0x7fbbcc070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:58.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.747+0000 7fbbd326c640 1 -- 172.21.15.5:0/2035192573 shutdown_connections 2024-05-20T18:32:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.747+0000 7fbbd326c640 1 -- 172.21.15.5:0/2035192573 wait complete. 2024-05-20T18:32:58.759 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:32:58.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.876+0000 7f334e484640 1 Processor -- start 2024-05-20T18:32:58.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.876+0000 7f334e484640 1 -- start start 2024-05-20T18:32:58.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.877+0000 7f334e484640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33480d17f0 0x7f334815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:58.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.877+0000 7f334e484640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33480cd580 con 0x7f33480b1e10 2024-05-20T18:32:58.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.877+0000 7f334e484640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33480cd860 con 0x7f334815d530 2024-05-20T18:32:58.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.877+0000 7f334e484640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f33480ce190 con 0x7f33480d17f0 2024-05-20T18:32:58.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.877+0000 7f33477fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f33480b1e10 0x7f3348070850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46858/0 (socket says 172.21.15.5:46858) 2024-05-20T18:32:58.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.877+0000 7f33477fe640 1 -- 172.21.15.5:0/2048867222 learned_addr learned my addr 172.21.15.5:0/2048867222 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:32:58.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.878+0000 7f334ca21640 1 --2- 172.21.15.5:0/2048867222 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33480d17f0 0x7f334815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:58.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.878+0000 7f3346ffd640 1 -- 172.21.15.5:0/2048867222 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2432191455 0 0) 0x7f33480cd580 con 0x7f33480b1e10 2024-05-20T18:32:58.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.878+0000 7f3346ffd640 1 -- 172.21.15.5:0/2048867222 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3330003520 con 0x7f33480b1e10 2024-05-20T18:32:58.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.878+0000 7f3346ffd640 1 -- 172.21.15.5:0/2048867222 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3596144427 0 0) 0x7f3330003520 con 0x7f33480b1e10 2024-05-20T18:32:58.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.878+0000 7f3346ffd640 1 -- 172.21.15.5:0/2048867222 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33480d17f0 msgr2=0x7f334815aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:58.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.878+0000 7f3346ffd640 1 --2- 172.21.15.5:0/2048867222 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33480d17f0 0x7f334815aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:58.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.878+0000 7f3346ffd640 1 -- 172.21.15.5:0/2048867222 >> v1:172.21.15.63:6789/0 conn(0x7f334815d530 legacy=0x7f33480b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:58.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.878+0000 7f3346ffd640 1 -- 172.21.15.5:0/2048867222 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33480cdb40 con 0x7f33480b1e10 2024-05-20T18:32:58.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.878+0000 7f3346ffd640 1 -- 172.21.15.5:0/2048867222 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f333c001670 con 0x7f33480b1e10 2024-05-20T18:32:58.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.878+0000 7f3346ffd640 1 -- 172.21.15.5:0/2048867222 >> v1:172.21.15.5:6789/0 conn(0x7f33480b1e10 legacy=0x7f3348070850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:58.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.878+0000 7f3346ffd640 1 --2- 172.21.15.5:0/2048867222 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33300039e0 0x7f3330003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:58.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.878+0000 7f3346ffd640 1 --2- 172.21.15.5:0/2048867222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3330004800 0x7f3330004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:58.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.878+0000 7f3346ffd640 1 --2- 172.21.15.5:0/2048867222 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33480d17f0 0x7f3330005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:58.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.878+0000 7f3346ffd640 1 -- 172.21.15.5:0/2048867222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f33480ce190 con 0x7f3330004800 2024-05-20T18:32:58.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.878+0000 7f3346ffd640 1 -- 172.21.15.5:0/2048867222 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f333000baf0 con 0x7f33300039e0 2024-05-20T18:32:58.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.878+0000 7f3346ffd640 1 -- 172.21.15.5:0/2048867222 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f333000bcf0 con 0x7f33480d17f0 2024-05-20T18:32:58.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.878+0000 7f33477fe640 1 --2- 172.21.15.5:0/2048867222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3330004800 0x7f3330004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:58.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.878+0000 7f334ca21640 1 --2- 172.21.15.5:0/2048867222 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33480d17f0 0x7f3330005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:58.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.878+0000 7f3347fff640 1 --2- 172.21.15.5:0/2048867222 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33300039e0 0x7f3330003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:58.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.879+0000 7f33477fe640 1 -- 172.21.15.5:0/2048867222 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33480d17f0 msgr2=0x7f3330005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:58.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.879+0000 7f33477fe640 1 --2- 172.21.15.5:0/2048867222 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33480d17f0 0x7f3330005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:58.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.879+0000 7f33477fe640 1 -- 172.21.15.5:0/2048867222 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33300039e0 msgr2=0x7f3330003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:58.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.879+0000 7f33477fe640 1 --2- 172.21.15.5:0/2048867222 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33300039e0 0x7f3330003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:58.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.879+0000 7f33477fe640 1 -- 172.21.15.5:0/2048867222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3330003520 con 0x7f3330004800 2024-05-20T18:32:58.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.879+0000 7f33477fe640 1 --2- 172.21.15.5:0/2048867222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3330004800 0x7f3330004c10 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f333c00c210 tx=0x7f3330005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d4c5a0baa27a22e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:58.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.879+0000 7f3346ffd640 1 -- 172.21.15.5:0/2048867222 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c007d10 con 0x7f3330004800 2024-05-20T18:32:58.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.880+0000 7f3346ffd640 1 -- 172.21.15.5:0/2048867222 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f333c0058d0 con 0x7f3330004800 2024-05-20T18:32:58.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.880+0000 7f3346ffd640 1 -- 172.21.15.5:0/2048867222 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c005ce0 con 0x7f3330004800 2024-05-20T18:32:58.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.880+0000 7f334e484640 1 -- 172.21.15.5:0/2048867222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3330004800 msgr2=0x7f3330004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:58.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.880+0000 7f334e484640 1 --2- 172.21.15.5:0/2048867222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3330004800 0x7f3330004c10 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f333c00c210 tx=0x7f3330005830 comp rx=0 tx=0).stop 2024-05-20T18:32:58.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.880+0000 7f334ca21640 1 -- 172.21.15.5:0/2048867222 reap_dead start 2024-05-20T18:32:58.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.880+0000 7f334e484640 1 -- 172.21.15.5:0/2048867222 shutdown_connections 2024-05-20T18:32:58.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.880+0000 7f334e484640 1 -- 172.21.15.5:0/2048867222 >> 172.21.15.5:0/2048867222 conn(0x7f33480d29d0 msgr2=0x7f33480d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:58.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.880+0000 7f334e484640 1 -- 172.21.15.5:0/2048867222 shutdown_connections 2024-05-20T18:32:58.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.880+0000 7f334e484640 1 -- 172.21.15.5:0/2048867222 wait complete. 2024-05-20T18:32:58.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.882+0000 7f334e484640 1 Processor -- start 2024-05-20T18:32:58.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.882+0000 7f334e484640 1 -- start start 2024-05-20T18:32:58.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.883+0000 7f334e484640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3330004800 0x7f33480c8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:58.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.884+0000 7f334e484640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3348158fd0 0x7f334815c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:58.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.884+0000 7f334e484640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33481593e0 0x7f334815c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:58.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.884+0000 7f334e484640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f334815aaa0 con 0x7f3330004800 2024-05-20T18:32:58.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.884+0000 7f334e484640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f33480cd230 con 0x7f3348158fd0 2024-05-20T18:32:58.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.884+0000 7f334e484640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f334815ac20 con 0x7f33481593e0 2024-05-20T18:32:58.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.884+0000 7f3347fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3330004800 0x7f33480c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:58.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.884+0000 7f3347fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3330004800 0x7f33480c8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44706/0 (socket says 172.21.15.5:44706) 2024-05-20T18:32:58.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.884+0000 7f3347fff640 1 -- 172.21.15.5:0/3927416900 learned_addr learned my addr 172.21.15.5:0/3927416900 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:58.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.884+0000 7f334ca21640 1 --2- 172.21.15.5:0/3927416900 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33481593e0 0x7f334815c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:58.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.884+0000 7f33477fe640 1 --2- 172.21.15.5:0/3927416900 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3348158fd0 0x7f334815c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:58.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.884+0000 7f3347fff640 1 -- 172.21.15.5:0/3927416900 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33481593e0 msgr2=0x7f334815c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:58.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.884+0000 7f3347fff640 1 --2- 172.21.15.5:0/3927416900 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33481593e0 0x7f334815c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:58.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.884+0000 7f3347fff640 1 -- 172.21.15.5:0/3927416900 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3348158fd0 msgr2=0x7f334815c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:58.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.884+0000 7f3347fff640 1 --2- 172.21.15.5:0/3927416900 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3348158fd0 0x7f334815c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:58.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.884+0000 7f3347fff640 1 -- 172.21.15.5:0/3927416900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33480c9c90 con 0x7f3330004800 2024-05-20T18:32:58.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.884+0000 7f33477fe640 1 --2- 172.21.15.5:0/3927416900 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3348158fd0 0x7f334815c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:32:58.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.884+0000 7f3347fff640 1 --2- 172.21.15.5:0/3927416900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3330004800 0x7f33480c8110 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f333800f5c0 tx=0x7f333800fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:58.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.884+0000 7f3344ff9640 1 -- 172.21.15.5:0/3927416900 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33380052e0 con 0x7f3330004800 2024-05-20T18:32:58.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.884+0000 7f3344ff9640 1 -- 172.21.15.5:0/3927416900 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3338018070 con 0x7f3330004800 2024-05-20T18:32:58.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.884+0000 7f334e484640 1 -- 172.21.15.5:0/3927416900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33480c9f70 con 0x7f3330004800 2024-05-20T18:32:58.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.884+0000 7f3344ff9640 1 -- 172.21.15.5:0/3927416900 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3338003810 con 0x7f3330004800 2024-05-20T18:32:58.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.884+0000 7f334e484640 1 -- 172.21.15.5:0/3927416900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33480ca530 con 0x7f3330004800 2024-05-20T18:32:58.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.885+0000 7f334e484640 1 -- 172.21.15.5:0/3927416900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3348075b00 con 0x7f3330004800 2024-05-20T18:32:58.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.886+0000 7f3344ff9640 1 -- 172.21.15.5:0/3927416900 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f33380039b0 con 0x7f3330004800 2024-05-20T18:32:58.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.887+0000 7f3344ff9640 1 --2- 172.21.15.5:0/3927416900 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f333406be30 0x7f333406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:58.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.887+0000 7f33477fe640 1 --2- 172.21.15.5:0/3927416900 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f333406be30 0x7f333406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:58.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.887+0000 7f3344ff9640 1 -- 172.21.15.5:0/3927416900 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f333808ae30 con 0x7f3330004800 2024-05-20T18:32:58.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.887+0000 7f33477fe640 1 --2- 172.21.15.5:0/3927416900 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f333406be30 0x7f333406e340 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f333c033000 tx=0x7f333c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:58.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:58.892+0000 7f3344ff9640 1 -- 172.21.15.5:0/3927416900 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3338059580 con 0x7f3330004800 2024-05-20T18:32:59.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.114+0000 7f334e484640 1 -- 172.21.15.5:0/3927416900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33480b1bf0 con 0x7f3330004800 2024-05-20T18:32:59.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.114+0000 7f3344ff9640 1 -- 172.21.15.5:0/3927416900 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5176 (secure 0 0 0) 0x7f333805c450 con 0x7f3330004800 2024-05-20T18:32:59.115 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:32:59.115 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1890759603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1890759603},{"type":"v1","addr":"172.21.15.63:6834","nonce":1890759603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5825,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/700447403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":700447403},{"type":"v1","addr":"172.21.15.63:6835","nonce":700447403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3382943275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3382943275},{"type":"v1","addr":"172.21.15.5:6837","nonce":3382943275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:31:30.164179+0000","modified":"2024-05-20T18:32:27.855796+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6972":{"gid":6972,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":15,"addr":"172.21.15.5:6836/3866822752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":3866822752},{"type":"v1","addr":"172.21.15.5:6836","nonce":3866822752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6972,"qdb_cluster":[6972]},"id":5}]} 2024-05-20T18:32:59.115 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 52 2024-05-20T18:32:59.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.116+0000 7f334e484640 1 -- 172.21.15.5:0/3927416900 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f333406be30 msgr2=0x7f333406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:59.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.116+0000 7f334e484640 1 --2- 172.21.15.5:0/3927416900 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f333406be30 0x7f333406e340 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f333c033000 tx=0x7f333c02e000 comp rx=0 tx=0).stop 2024-05-20T18:32:59.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.116+0000 7f334e484640 1 -- 172.21.15.5:0/3927416900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3330004800 msgr2=0x7f33480c8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:59.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.116+0000 7f334e484640 1 --2- 172.21.15.5:0/3927416900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3330004800 0x7f33480c8110 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f333800f5c0 tx=0x7f333800fa80 comp rx=0 tx=0).stop 2024-05-20T18:32:59.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.116+0000 7f334e484640 1 -- 172.21.15.5:0/3927416900 shutdown_connections 2024-05-20T18:32:59.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.116+0000 7f334e484640 1 --2- 172.21.15.5:0/3927416900 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f333406be30 0x7f333406e340 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.116+0000 7f334e484640 1 --2- 172.21.15.5:0/3927416900 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33481593e0 0x7f334815c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.116+0000 7f334e484640 1 --2- 172.21.15.5:0/3927416900 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3348158fd0 0x7f334815c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.116+0000 7f334e484640 1 --2- 172.21.15.5:0/3927416900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3330004800 0x7f33480c8110 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.116+0000 7f334e484640 1 -- 172.21.15.5:0/3927416900 >> 172.21.15.5:0/3927416900 conn(0x7f33480d29d0 msgr2=0x7f334815f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:59.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.116+0000 7f334e484640 1 -- 172.21.15.5:0/3927416900 shutdown_connections 2024-05-20T18:32:59.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.116+0000 7f334e484640 1 -- 172.21.15.5:0/3927416900 wait complete. 2024-05-20T18:32:59.128 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:32:59.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.243+0000 7f75818a6640 1 Processor -- start 2024-05-20T18:32:59.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.243+0000 7f75818a6640 1 -- start start 2024-05-20T18:32:59.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.244+0000 7f75818a6640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f757c071630 0x7f757c15f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:59.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.244+0000 7f75818a6640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f757c0cd1b0 0x7f757c06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:59.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.244+0000 7f75818a6640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f757c0ce2b0 con 0x7f757c070850 2024-05-20T18:32:59.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.244+0000 7f75818a6640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f757c15f790 con 0x7f757c0cd1b0 2024-05-20T18:32:59.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.244+0000 7f75818a6640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f757c06e590 con 0x7f757c071630 2024-05-20T18:32:59.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.244+0000 7f757a7fc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f757c0cd1b0 0x7f757c06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:59.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.244+0000 7f757a7fc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f757c0cd1b0 0x7f757c06d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44718/0 (socket says 172.21.15.5:44718) 2024-05-20T18:32:59.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.244+0000 7f757a7fc640 1 -- 172.21.15.5:0/578929919 learned_addr learned my addr 172.21.15.5:0/578929919 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:59.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.244+0000 7f757affd640 1 --2- 172.21.15.5:0/578929919 >> v2:172.21.15.63:3300/0 conn(0x7f757c071630 0x7f757c15f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:59.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.244+0000 7f757a7fc640 1 -- 172.21.15.5:0/578929919 >> v2:172.21.15.63:3300/0 conn(0x7f757c071630 msgr2=0x7f757c15f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:59.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.244+0000 7f757a7fc640 1 --2- 172.21.15.5:0/578929919 >> v2:172.21.15.63:3300/0 conn(0x7f757c071630 0x7f757c15f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.244+0000 7f757a7fc640 1 -- 172.21.15.5:0/578929919 >> v1:172.21.15.63:6789/0 conn(0x7f757c070850 legacy=0x7f757c070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:59.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.244+0000 7f757a7fc640 1 -- 172.21.15.5:0/578929919 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757c06cce0 con 0x7f757c0cd1b0 2024-05-20T18:32:59.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.244+0000 7f757affd640 1 --2- 172.21.15.5:0/578929919 >> v2:172.21.15.63:3300/0 conn(0x7f757c071630 0x7f757c15f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:32:59.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.245+0000 7f757a7fc640 1 --2- 172.21.15.5:0/578929919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f757c0cd1b0 0x7f757c06d7c0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f757c068510 tx=0x7f757002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1972fff0343668c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:59.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.245+0000 7f7579ffb640 1 -- 172.21.15.5:0/578929919 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7570037070 con 0x7f757c0cd1b0 2024-05-20T18:32:59.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.245+0000 7f7579ffb640 1 -- 172.21.15.5:0/578929919 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7570032440 con 0x7f757c0cd1b0 2024-05-20T18:32:59.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.245+0000 7f7579ffb640 1 -- 172.21.15.5:0/578929919 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7570032810 con 0x7f757c0cd1b0 2024-05-20T18:32:59.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.246+0000 7f75818a6640 1 -- 172.21.15.5:0/578929919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f757c0cd1b0 msgr2=0x7f757c06d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:59.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.246+0000 7f75818a6640 1 --2- 172.21.15.5:0/578929919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f757c0cd1b0 0x7f757c06d7c0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f757c068510 tx=0x7f757002a1a0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.246+0000 7f75818a6640 1 -- 172.21.15.5:0/578929919 shutdown_connections 2024-05-20T18:32:59.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.246+0000 7f75818a6640 1 --2- 172.21.15.5:0/578929919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f757c0cd1b0 0x7f757c06d7c0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.246+0000 7f75818a6640 1 --2- 172.21.15.5:0/578929919 >> v2:172.21.15.63:3300/0 conn(0x7f757c071630 0x7f757c15f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.246+0000 7f75818a6640 1 -- 172.21.15.5:0/578929919 >> 172.21.15.5:0/578929919 conn(0x7f757c0d29d0 msgr2=0x7f757c0d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:59.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.246+0000 7f75818a6640 1 -- 172.21.15.5:0/578929919 shutdown_connections 2024-05-20T18:32:59.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.246+0000 7f75818a6640 1 -- 172.21.15.5:0/578929919 wait complete. 2024-05-20T18:32:59.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.248+0000 7f75818a6640 1 Processor -- start 2024-05-20T18:32:59.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.248+0000 7f75818a6640 1 -- start start 2024-05-20T18:32:59.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.249+0000 7f75818a6640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f757c070850 0x7f757c1f5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:59.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.249+0000 7f75818a6640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f757c071630 0x7f757c159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:59.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.249+0000 7f75818a6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f757c0cd1b0 0x7f757c160f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:59.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.249+0000 7f75818a6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f757c06d2a0 con 0x7f757c0cd1b0 2024-05-20T18:32:59.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.249+0000 7f75818a6640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f757c06c990 con 0x7f757c071630 2024-05-20T18:32:59.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.249+0000 7f75818a6640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f757c06dd00 con 0x7f757c070850 2024-05-20T18:32:59.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.250+0000 7f757b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f757c0cd1b0 0x7f757c160f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:59.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.250+0000 7f757b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f757c0cd1b0 0x7f757c160f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44728/0 (socket says 172.21.15.5:44728) 2024-05-20T18:32:59.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.250+0000 7f757b7fe640 1 -- 172.21.15.5:0/1449351344 learned_addr learned my addr 172.21.15.5:0/1449351344 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:59.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.250+0000 7f757affd640 1 --2- 172.21.15.5:0/1449351344 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f757c070850 0x7f757c1f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:59.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.250+0000 7f757a7fc640 1 --2- 172.21.15.5:0/1449351344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f757c071630 0x7f757c159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:59.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.250+0000 7f757b7fe640 1 -- 172.21.15.5:0/1449351344 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f757c070850 msgr2=0x7f757c1f5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:59.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.250+0000 7f757b7fe640 1 --2- 172.21.15.5:0/1449351344 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f757c070850 0x7f757c1f5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.250+0000 7f757b7fe640 1 -- 172.21.15.5:0/1449351344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f757c071630 msgr2=0x7f757c159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:59.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.250+0000 7f757b7fe640 1 --2- 172.21.15.5:0/1449351344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f757c071630 0x7f757c159390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.250+0000 7f757b7fe640 1 -- 172.21.15.5:0/1449351344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757c15fee0 con 0x7f757c0cd1b0 2024-05-20T18:32:59.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.250+0000 7f757b7fe640 1 --2- 172.21.15.5:0/1449351344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f757c0cd1b0 0x7f757c160f80 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f757400b410 tx=0x7f757400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:59.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.250+0000 7f7563fff640 1 -- 172.21.15.5:0/1449351344 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7574015030 con 0x7f757c0cd1b0 2024-05-20T18:32:59.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.250+0000 7f7563fff640 1 -- 172.21.15.5:0/1449351344 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7574004020 con 0x7f757c0cd1b0 2024-05-20T18:32:59.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.250+0000 7f7563fff640 1 -- 172.21.15.5:0/1449351344 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7574004410 con 0x7f757c0cd1b0 2024-05-20T18:32:59.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.250+0000 7f75818a6640 1 -- 172.21.15.5:0/1449351344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f757c1601c0 con 0x7f757c0cd1b0 2024-05-20T18:32:59.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.250+0000 7f75818a6640 1 -- 172.21.15.5:0/1449351344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f757c160780 con 0x7f757c0cd1b0 2024-05-20T18:32:59.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.252+0000 7f75818a6640 1 -- 172.21.15.5:0/1449351344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f757c162af0 con 0x7f757c0cd1b0 2024-05-20T18:32:59.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.252+0000 7f7563fff640 1 -- 172.21.15.5:0/1449351344 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7574018070 con 0x7f757c0cd1b0 2024-05-20T18:32:59.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.253+0000 7f7563fff640 1 --2- 172.21.15.5:0/1449351344 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f756806be60 0x7f756806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:59.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.253+0000 7f7563fff640 1 -- 172.21.15.5:0/1449351344 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f757408aa00 con 0x7f757c0cd1b0 2024-05-20T18:32:59.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.253+0000 7f757affd640 1 --2- 172.21.15.5:0/1449351344 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f756806be60 0x7f756806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:59.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.253+0000 7f757affd640 1 --2- 172.21.15.5:0/1449351344 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f756806be60 0x7f756806e370 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f756c009fd0 tx=0x7f756c009870 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:59.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.257+0000 7f7563fff640 1 -- 172.21.15.5:0/1449351344 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7574058af0 con 0x7f757c0cd1b0 2024-05-20T18:32:59.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.441+0000 7f75818a6640 1 -- 172.21.15.5:0/1449351344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f757c075b00 con 0x7f757c0cd1b0 2024-05-20T18:32:59.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.442+0000 7f7563fff640 1 -- 172.21.15.5:0/1449351344 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+16950 (secure 0 0 0) 0x7f757405cdf0 con 0x7f757c0cd1b0 2024-05-20T18:32:59.443 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:32:59.443 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":78,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:31:40.839398+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:31:27.896386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-20T18:31:29.146912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6836/791176355":"2024-05-21T18:31:40.839373+0000","172.21.15.5:6835/791176355":"2024-05-21T18:31:40.839373+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:32:59.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.444+0000 7f75818a6640 1 -- 172.21.15.5:0/1449351344 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f756806be60 msgr2=0x7f756806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:59.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.444+0000 7f75818a6640 1 --2- 172.21.15.5:0/1449351344 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f756806be60 0x7f756806e370 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f756c009fd0 tx=0x7f756c009870 comp rx=0 tx=0).stop 2024-05-20T18:32:59.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.444+0000 7f75818a6640 1 -- 172.21.15.5:0/1449351344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f757c0cd1b0 msgr2=0x7f757c160f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:59.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.444+0000 7f75818a6640 1 --2- 172.21.15.5:0/1449351344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f757c0cd1b0 0x7f757c160f80 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f757400b410 tx=0x7f757400b8d0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.444+0000 7f75818a6640 1 -- 172.21.15.5:0/1449351344 shutdown_connections 2024-05-20T18:32:59.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.444+0000 7f75818a6640 1 --2- 172.21.15.5:0/1449351344 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f756806be60 0x7f756806e370 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.444+0000 7f75818a6640 1 --2- 172.21.15.5:0/1449351344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f757c0cd1b0 0x7f757c160f80 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.444+0000 7f75818a6640 1 --2- 172.21.15.5:0/1449351344 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f757c071630 0x7f757c159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.444+0000 7f75818a6640 1 --2- 172.21.15.5:0/1449351344 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f757c070850 0x7f757c1f5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.444+0000 7f75818a6640 1 -- 172.21.15.5:0/1449351344 >> 172.21.15.5:0/1449351344 conn(0x7f757c0d29d0 msgr2=0x7f757c159bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:59.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.444+0000 7f75818a6640 1 -- 172.21.15.5:0/1449351344 shutdown_connections 2024-05-20T18:32:59.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.444+0000 7f75818a6640 1 -- 172.21.15.5:0/1449351344 wait complete. 2024-05-20T18:32:59.457 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-20T18:32:59.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.572+0000 7f1b7f4a0640 1 Processor -- start 2024-05-20T18:32:59.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.572+0000 7f1b7f4a0640 1 -- start start 2024-05-20T18:32:59.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.573+0000 7f1b7f4a0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b78162820 0x7f1b780b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:59.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.573+0000 7f1b7f4a0640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1b780b3e80 0x7f1b7806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:59.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.573+0000 7f1b7f4a0640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b7806dd20 con 0x7f1b7806d910 2024-05-20T18:32:59.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.573+0000 7f1b7f4a0640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f1b780d0630 con 0x7f1b780b3e80 2024-05-20T18:32:59.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.573+0000 7f1b7f4a0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1b780d39c0 con 0x7f1b78162820 2024-05-20T18:32:59.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.573+0000 7f1b7ca3b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1b780b3e80 0x7f1b7806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:59.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.573+0000 7f1b7ca3b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1b780b3e80 0x7f1b7806d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44740/0 (socket says 172.21.15.5:44740) 2024-05-20T18:32:59.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.573+0000 7f1b7ca3b640 1 -- 172.21.15.5:0/3147445678 learned_addr learned my addr 172.21.15.5:0/3147445678 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:59.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.574+0000 7f1b7d23c640 1 --2- 172.21.15.5:0/3147445678 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b78162820 0x7f1b780b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:59.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.574+0000 7f1b7ca3b640 1 -- 172.21.15.5:0/3147445678 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b78162820 msgr2=0x7f1b780b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:59.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.574+0000 7f1b7ca3b640 1 --2- 172.21.15.5:0/3147445678 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b78162820 0x7f1b780b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.574+0000 7f1b7ca3b640 1 -- 172.21.15.5:0/3147445678 >> v1:172.21.15.63:6789/0 conn(0x7f1b7806d910 legacy=0x7f1b780d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:59.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.574+0000 7f1b7ca3b640 1 -- 172.21.15.5:0/3147445678 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b780d3ca0 con 0x7f1b780b3e80 2024-05-20T18:32:59.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.575+0000 7f1b7ca3b640 1 --2- 172.21.15.5:0/3147445678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b780b3e80 0x7f1b7806d3d0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f1b68009870 tx=0x7f1b6802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc57759f527c00b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:59.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.575+0000 7f1b6ffff640 1 -- 172.21.15.5:0/3147445678 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68037070 con 0x7f1b780b3e80 2024-05-20T18:32:59.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.575+0000 7f1b6ffff640 1 -- 172.21.15.5:0/3147445678 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b68032440 con 0x7f1b780b3e80 2024-05-20T18:32:59.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.575+0000 7f1b6ffff640 1 -- 172.21.15.5:0/3147445678 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68032810 con 0x7f1b780b3e80 2024-05-20T18:32:59.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.575+0000 7f1b7f4a0640 1 -- 172.21.15.5:0/3147445678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b780b3e80 msgr2=0x7f1b7806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:59.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.575+0000 7f1b7f4a0640 1 --2- 172.21.15.5:0/3147445678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b780b3e80 0x7f1b7806d3d0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f1b68009870 tx=0x7f1b6802a270 comp rx=0 tx=0).stop 2024-05-20T18:32:59.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.575+0000 7f1b7f4a0640 1 -- 172.21.15.5:0/3147445678 shutdown_connections 2024-05-20T18:32:59.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.575+0000 7f1b7f4a0640 1 --2- 172.21.15.5:0/3147445678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b780b3e80 0x7f1b7806d3d0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.575+0000 7f1b7f4a0640 1 --2- 172.21.15.5:0/3147445678 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b78162820 0x7f1b780b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.575+0000 7f1b7f4a0640 1 -- 172.21.15.5:0/3147445678 >> 172.21.15.5:0/3147445678 conn(0x7f1b780d4bd0 msgr2=0x7f1b780d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:59.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.576+0000 7f1b7f4a0640 1 -- 172.21.15.5:0/3147445678 shutdown_connections 2024-05-20T18:32:59.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.576+0000 7f1b7f4a0640 1 -- 172.21.15.5:0/3147445678 wait complete. 2024-05-20T18:32:59.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.577+0000 7f1b7f4a0640 1 Processor -- start 2024-05-20T18:32:59.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.577+0000 7f1b7f4a0640 1 -- start start 2024-05-20T18:32:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.579+0000 7f1b7f4a0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b7806d910 0x7f1b780c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.579+0000 7f1b7f4a0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b780b3e80 0x7f1b781e92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.579+0000 7f1b7f4a0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b78162820 0x7f1b7814b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.579+0000 7f1b7f4a0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1b780d0300 con 0x7f1b7806d910 2024-05-20T18:32:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.579+0000 7f1b7f4a0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1b780d4260 con 0x7f1b780b3e80 2024-05-20T18:32:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.579+0000 7f1b7f4a0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1b780b91a0 con 0x7f1b78162820 2024-05-20T18:32:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.579+0000 7f1b7d23c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b7806d910 0x7f1b780c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.579+0000 7f1b7d23c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b7806d910 0x7f1b780c8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44746/0 (socket says 172.21.15.5:44746) 2024-05-20T18:32:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.579+0000 7f1b7d23c640 1 -- 172.21.15.5:0/2802243390 learned_addr learned my addr 172.21.15.5:0/2802243390 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.579+0000 7f1b7da3d640 1 --2- 172.21.15.5:0/2802243390 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b78162820 0x7f1b7814b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.579+0000 7f1b7ca3b640 1 --2- 172.21.15.5:0/2802243390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b780b3e80 0x7f1b781e92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.579+0000 7f1b7d23c640 1 -- 172.21.15.5:0/2802243390 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b78162820 msgr2=0x7f1b7814b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.579+0000 7f1b7d23c640 1 --2- 172.21.15.5:0/2802243390 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b78162820 0x7f1b7814b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.579+0000 7f1b7d23c640 1 -- 172.21.15.5:0/2802243390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b780b3e80 msgr2=0x7f1b781e92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.579+0000 7f1b7d23c640 1 --2- 172.21.15.5:0/2802243390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b780b3e80 0x7f1b781e92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.579+0000 7f1b7d23c640 1 -- 172.21.15.5:0/2802243390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7814a2d0 con 0x7f1b7806d910 2024-05-20T18:32:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.579+0000 7f1b7ca3b640 1 --2- 172.21.15.5:0/2802243390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b780b3e80 0x7f1b781e92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:32:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.579+0000 7f1b7d23c640 1 --2- 172.21.15.5:0/2802243390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b7806d910 0x7f1b780c8700 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400e8a0 tx=0x7f1b7400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.579+0000 7f1b6dffb640 1 -- 172.21.15.5:0/2802243390 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b74017070 con 0x7f1b7806d910 2024-05-20T18:32:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.579+0000 7f1b6dffb640 1 -- 172.21.15.5:0/2802243390 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b7400c940 con 0x7f1b7806d910 2024-05-20T18:32:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.579+0000 7f1b7f4a0640 1 -- 172.21.15.5:0/2802243390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b7814a5b0 con 0x7f1b7806d910 2024-05-20T18:32:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.579+0000 7f1b6dffb640 1 -- 172.21.15.5:0/2802243390 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7400cd30 con 0x7f1b7806d910 2024-05-20T18:32:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.580+0000 7f1b7f4a0640 1 -- 172.21.15.5:0/2802243390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b7814ab70 con 0x7f1b7806d910 2024-05-20T18:32:59.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.581+0000 7f1b7f4a0640 1 -- 172.21.15.5:0/2802243390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b780d4780 con 0x7f1b7806d910 2024-05-20T18:32:59.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.581+0000 7f1b6dffb640 1 -- 172.21.15.5:0/2802243390 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b7401a070 con 0x7f1b7806d910 2024-05-20T18:32:59.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.582+0000 7f1b6dffb640 1 --2- 172.21.15.5:0/2802243390 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1b5c06be30 0x7f1b5c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:59.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.582+0000 7f1b6dffb640 1 -- 172.21.15.5:0/2802243390 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6187+0+0 (secure 0 0 0) 0x7f1b7408b240 con 0x7f1b7806d910 2024-05-20T18:32:59.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.582+0000 7f1b7ca3b640 1 --2- 172.21.15.5:0/2802243390 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1b5c06be30 0x7f1b5c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:59.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.582+0000 7f1b7ca3b640 1 --2- 172.21.15.5:0/2802243390 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1b5c06be30 0x7f1b5c06e340 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1b68034000 tx=0x7f1b6802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:59.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.586+0000 7f1b6dffb640 1 -- 172.21.15.5:0/2802243390 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1b780d4780 con 0x7f1b7806d910 2024-05-20T18:32:59.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.808+0000 7f1b7f4a0640 1 -- 172.21.15.5:0/2802243390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1b78075b00 con 0x7f1b7806d910 2024-05-20T18:32:59.809 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-20T18:32:59.809+0000 7fd1cf53a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-20T18:32:59.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.810+0000 7f1b6dffb640 1 -- 172.21.15.5:0/2802243390 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v53) ==== 161+0+0 (secure 0 0 0) 0x7f1b74059320 con 0x7f1b7806d910 2024-05-20T18:32:59.811 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-20T18:32:59.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.813+0000 7f1b7f4a0640 1 -- 172.21.15.5:0/2802243390 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1b5c06be30 msgr2=0x7f1b5c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:59.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.813+0000 7f1b7f4a0640 1 --2- 172.21.15.5:0/2802243390 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1b5c06be30 0x7f1b5c06e340 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1b68034000 tx=0x7f1b6802e000 comp rx=0 tx=0).stop 2024-05-20T18:32:59.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.813+0000 7f1b7f4a0640 1 -- 172.21.15.5:0/2802243390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b7806d910 msgr2=0x7f1b780c8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:59.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.813+0000 7f1b7f4a0640 1 --2- 172.21.15.5:0/2802243390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b7806d910 0x7f1b780c8700 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400e8a0 tx=0x7f1b7400ed60 comp rx=0 tx=0).stop 2024-05-20T18:32:59.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.813+0000 7f1b7f4a0640 1 -- 172.21.15.5:0/2802243390 shutdown_connections 2024-05-20T18:32:59.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.813+0000 7f1b7f4a0640 1 --2- 172.21.15.5:0/2802243390 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1b5c06be30 0x7f1b5c06e340 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.813+0000 7f1b7f4a0640 1 --2- 172.21.15.5:0/2802243390 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b78162820 0x7f1b7814b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.813+0000 7f1b7f4a0640 1 --2- 172.21.15.5:0/2802243390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b780b3e80 0x7f1b781e92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.813+0000 7f1b7f4a0640 1 --2- 172.21.15.5:0/2802243390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b7806d910 0x7f1b780c8700 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.813+0000 7f1b7f4a0640 1 -- 172.21.15.5:0/2802243390 >> 172.21.15.5:0/2802243390 conn(0x7f1b780d4bd0 msgr2=0x7f1b780b9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:59.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.814+0000 7f1b7f4a0640 1 -- 172.21.15.5:0/2802243390 shutdown_connections 2024-05-20T18:32:59.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.814+0000 7f1b7f4a0640 1 -- 172.21.15.5:0/2802243390 wait complete. 2024-05-20T18:32:59.818 INFO:tasks.ceph.mds.c.smithi005.stderr: -4193> 2024-05-20T18:31:46.852+0000 7f712b412640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T18:32:59.829 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-20T18:32:59.882 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-05-20T18:32:59.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.950+0000 7f6027acc640 1 Processor -- start 2024-05-20T18:32:59.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.950+0000 7f6027acc640 1 -- start start 2024-05-20T18:32:59.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.951+0000 7f6027acc640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f602015d530 0x7f602015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:59.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.951+0000 7f6027acc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60200b1700 0x7f60200b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:59.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.951+0000 7f6027acc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6020070850 0x7f6020070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:59.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.951+0000 7f6027acc640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f60200ce190 con 0x7f6020070850 2024-05-20T18:32:59.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.951+0000 7f6027acc640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f60200cd580 con 0x7f602015d530 2024-05-20T18:32:59.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.951+0000 7f6027acc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f60200cd860 con 0x7f60200b1700 2024-05-20T18:32:59.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.951+0000 7f6026069640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6020070850 0x7f6020070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:59.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.951+0000 7f6026069640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6020070850 0x7f6020070c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44764/0 (socket says 172.21.15.5:44764) 2024-05-20T18:32:59.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.951+0000 7f6026069640 1 -- 172.21.15.5:0/894409736 learned_addr learned my addr 172.21.15.5:0/894409736 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:59.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.951+0000 7f6025067640 1 --2- 172.21.15.5:0/894409736 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60200b1700 0x7f60200b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:59.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.951+0000 7f6025868640 1 --2- 172.21.15.5:0/894409736 >> v2:172.21.15.63:3300/0 conn(0x7f602015d530 0x7f602015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:59.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.952+0000 7f6026069640 1 -- 172.21.15.5:0/894409736 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60200b1700 msgr2=0x7f60200b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:59.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.952+0000 7f6026069640 1 --2- 172.21.15.5:0/894409736 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60200b1700 0x7f60200b1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.952+0000 7f6026069640 1 -- 172.21.15.5:0/894409736 >> v2:172.21.15.63:3300/0 conn(0x7f602015d530 msgr2=0x7f602015aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:32:59.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.952+0000 7f6026069640 1 --2- 172.21.15.5:0/894409736 >> v2:172.21.15.63:3300/0 conn(0x7f602015d530 0x7f602015aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.952+0000 7f6026069640 1 -- 172.21.15.5:0/894409736 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60200cdb40 con 0x7f6020070850 2024-05-20T18:32:59.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.953+0000 7f6026069640 1 --2- 172.21.15.5:0/894409736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6020070850 0x7f6020070c60 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f602004f790 tx=0x7f601802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c6618ced44174b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:59.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.953+0000 7f6024866640 1 -- 172.21.15.5:0/894409736 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6018037070 con 0x7f6020070850 2024-05-20T18:32:59.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.953+0000 7f6024866640 1 -- 172.21.15.5:0/894409736 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6018032440 con 0x7f6020070850 2024-05-20T18:32:59.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.953+0000 7f6024866640 1 -- 172.21.15.5:0/894409736 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6018032810 con 0x7f6020070850 2024-05-20T18:32:59.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.953+0000 7f6027acc640 1 -- 172.21.15.5:0/894409736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6020070850 msgr2=0x7f6020070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:59.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.953+0000 7f6027acc640 1 --2- 172.21.15.5:0/894409736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6020070850 0x7f6020070c60 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f602004f790 tx=0x7f601802a1e0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.953+0000 7f6027acc640 1 -- 172.21.15.5:0/894409736 shutdown_connections 2024-05-20T18:32:59.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.953+0000 7f6027acc640 1 --2- 172.21.15.5:0/894409736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6020070850 0x7f6020070c60 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.953+0000 7f6027acc640 1 --2- 172.21.15.5:0/894409736 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60200b1700 0x7f60200b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.953+0000 7f6027acc640 1 --2- 172.21.15.5:0/894409736 >> v2:172.21.15.63:3300/0 conn(0x7f602015d530 0x7f602015aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.953+0000 7f6027acc640 1 -- 172.21.15.5:0/894409736 >> 172.21.15.5:0/894409736 conn(0x7f60200d29d0 msgr2=0x7f60200d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:32:59.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.954+0000 7f6027acc640 1 -- 172.21.15.5:0/894409736 shutdown_connections 2024-05-20T18:32:59.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.954+0000 7f6027acc640 1 -- 172.21.15.5:0/894409736 wait complete. 2024-05-20T18:32:59.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.955+0000 7f6027acc640 1 Processor -- start 2024-05-20T18:32:59.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.955+0000 7f6027acc640 1 -- start start 2024-05-20T18:32:59.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.957+0000 7f6027acc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6020070850 0x7f60201551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:59.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.957+0000 7f6027acc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60200b1700 0x7f602006d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:59.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.957+0000 7f6027acc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f602015d530 0x7f602015baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:59.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.957+0000 7f6027acc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f60200b2050 con 0x7f6020070850 2024-05-20T18:32:59.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.957+0000 7f6027acc640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f60200cd230 con 0x7f602015d530 2024-05-20T18:32:59.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.957+0000 7f6027acc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f60200711a0 con 0x7f60200b1700 2024-05-20T18:32:59.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.957+0000 7f6025868640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6020070850 0x7f60201551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:59.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.957+0000 7f6025868640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6020070850 0x7f60201551b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44772/0 (socket says 172.21.15.5:44772) 2024-05-20T18:32:59.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.957+0000 7f6025868640 1 -- 172.21.15.5:0/3077335628 learned_addr learned my addr 172.21.15.5:0/3077335628 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:32:59.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.957+0000 7f6025067640 1 --2- 172.21.15.5:0/3077335628 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60200b1700 0x7f602006d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:59.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.957+0000 7f6026069640 1 --2- 172.21.15.5:0/3077335628 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f602015d530 0x7f602015baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:59.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.957+0000 7f6025868640 1 -- 172.21.15.5:0/3077335628 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60200b1700 msgr2=0x7f602006d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.957+0000 7f6025868640 1 --2- 172.21.15.5:0/3077335628 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60200b1700 0x7f602006d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.957+0000 7f6025868640 1 -- 172.21.15.5:0/3077335628 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f602015d530 msgr2=0x7f602015baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:32:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.957+0000 7f6025868640 1 --2- 172.21.15.5:0/3077335628 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f602015d530 0x7f602015baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:32:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.957+0000 7f6025868640 1 -- 172.21.15.5:0/3077335628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f602006db00 con 0x7f6020070850 2024-05-20T18:32:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.957+0000 7f6026069640 1 --2- 172.21.15.5:0/3077335628 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f602015d530 0x7f602015baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:32:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.957+0000 7f6025868640 1 --2- 172.21.15.5:0/3077335628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6020070850 0x7f60201551b0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f601c009cd0 tx=0x7f601c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.957+0000 7f60167fc640 1 -- 172.21.15.5:0/3077335628 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c018020 con 0x7f6020070850 2024-05-20T18:32:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.957+0000 7f60167fc640 1 -- 172.21.15.5:0/3077335628 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f601c014480 con 0x7f6020070850 2024-05-20T18:32:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.957+0000 7f6027acc640 1 -- 172.21.15.5:0/3077335628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f602006dde0 con 0x7f6020070850 2024-05-20T18:32:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.957+0000 7f60167fc640 1 -- 172.21.15.5:0/3077335628 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c014850 con 0x7f6020070850 2024-05-20T18:32:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.958+0000 7f6027acc640 1 -- 172.21.15.5:0/3077335628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60201556f0 con 0x7f6020070850 2024-05-20T18:32:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.958+0000 7f6027acc640 1 -- 172.21.15.5:0/3077335628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6020075b00 con 0x7f6020070850 2024-05-20T18:32:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.959+0000 7f60167fc640 1 -- 172.21.15.5:0/3077335628 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f601c005d10 con 0x7f6020070850 2024-05-20T18:32:59.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.960+0000 7f60167fc640 1 --2- 172.21.15.5:0/3077335628 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f600406be60 0x7f600406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:32:59.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.960+0000 7f6025067640 1 --2- 172.21.15.5:0/3077335628 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f600406be60 0x7f600406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:32:59.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.960+0000 7f60167fc640 1 -- 172.21.15.5:0/3077335628 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6273+0+0 (secure 0 0 0) 0x7f601c08bdd0 con 0x7f6020070850 2024-05-20T18:32:59.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.960+0000 7f6025067640 1 --2- 172.21.15.5:0/3077335628 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f600406be60 0x7f600406e370 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f60201f8640 tx=0x7f6010009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:32:59.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:32:59.965+0000 7f60167fc640 1 -- 172.21.15.5:0/3077335628 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f601c022030 con 0x7f6020070850 2024-05-20T18:33:00.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.188+0000 7f6027acc640 1 -- 172.21.15.5:0/3077335628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f60200c60a0 con 0x7f6020070850 2024-05-20T18:33:00.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.812+0000 7f60167fc640 1 -- 172.21.15.5:0/3077335628 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v54) ==== 104+0+0 (secure 0 0 0) 0x7f601c010040 con 0x7f6020070850 2024-05-20T18:33:00.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.815+0000 7f6027acc640 1 -- 172.21.15.5:0/3077335628 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f600406be60 msgr2=0x7f600406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:00.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.815+0000 7f6027acc640 1 --2- 172.21.15.5:0/3077335628 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f600406be60 0x7f600406e370 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f60201f8640 tx=0x7f6010009190 comp rx=0 tx=0).stop 2024-05-20T18:33:00.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.815+0000 7f6027acc640 1 -- 172.21.15.5:0/3077335628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6020070850 msgr2=0x7f60201551b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:00.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.815+0000 7f6027acc640 1 --2- 172.21.15.5:0/3077335628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6020070850 0x7f60201551b0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f601c009cd0 tx=0x7f601c00c490 comp rx=0 tx=0).stop 2024-05-20T18:33:00.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.815+0000 7f6027acc640 1 -- 172.21.15.5:0/3077335628 shutdown_connections 2024-05-20T18:33:00.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.815+0000 7f6027acc640 1 --2- 172.21.15.5:0/3077335628 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f600406be60 0x7f600406e370 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:00.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.815+0000 7f6027acc640 1 --2- 172.21.15.5:0/3077335628 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f602015d530 0x7f602015baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:00.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.815+0000 7f6027acc640 1 --2- 172.21.15.5:0/3077335628 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60200b1700 0x7f602006d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:00.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.815+0000 7f6027acc640 1 --2- 172.21.15.5:0/3077335628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6020070850 0x7f60201551b0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:00.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.815+0000 7f6027acc640 1 -- 172.21.15.5:0/3077335628 >> 172.21.15.5:0/3077335628 conn(0x7f60200d29d0 msgr2=0x7f6020158ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:00.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.815+0000 7f6027acc640 1 -- 172.21.15.5:0/3077335628 shutdown_connections 2024-05-20T18:33:00.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.815+0000 7f6027acc640 1 -- 172.21.15.5:0/3077335628 wait complete. 2024-05-20T18:33:00.830 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-20T18:33:00.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.956+0000 7f775353e640 1 Processor -- start 2024-05-20T18:33:00.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.956+0000 7f775353e640 1 -- start start 2024-05-20T18:33:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.958+0000 7f775353e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f774c09e740 0x7f774c15d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.958+0000 7f775353e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f774c0b00f0 con 0x7f774c0af730 2024-05-20T18:33:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.958+0000 7f775353e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f774c0ad8d0 con 0x7f774c070010 2024-05-20T18:33:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.958+0000 7f775353e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f774c0adbb0 con 0x7f774c09e740 2024-05-20T18:33:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.958+0000 7f77512da640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f774c09e740 0x7f774c15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.958+0000 7f7750ad9640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f774c0af730 0x7f774c06f900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46868/0 (socket says 172.21.15.5:46868) 2024-05-20T18:33:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.958+0000 7f7750ad9640 1 -- 172.21.15.5:0/2908917662 learned_addr learned my addr 172.21.15.5:0/2908917662 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:33:00.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.958+0000 7f77512da640 1 -- 172.21.15.5:0/2908917662 >> v1:172.21.15.63:6789/0 conn(0x7f774c070010 legacy=0x7f774c15c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:00.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.958+0000 7f77512da640 1 -- 172.21.15.5:0/2908917662 >> v1:172.21.15.5:6789/0 conn(0x7f774c0af730 legacy=0x7f774c06f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:00.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.958+0000 7f77512da640 1 -- 172.21.15.5:0/2908917662 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774c0ade90 con 0x7f774c09e740 2024-05-20T18:33:00.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.958+0000 7f774bfff640 1 -- 172.21.15.5:0/2908917662 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2343525143 0 0) 0x7f774c0b00f0 con 0x7f774c0af730 2024-05-20T18:33:00.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.959+0000 7f77512da640 1 --2- 172.21.15.5:0/2908917662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f774c09e740 0x7f774c15d860 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f773c009dc0 tx=0x7f773c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b141e3d074a24ec8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:00.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.959+0000 7f774bfff640 1 -- 172.21.15.5:0/2908917662 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c007740 con 0x7f774c09e740 2024-05-20T18:33:00.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.959+0000 7f774bfff640 1 -- 172.21.15.5:0/2908917662 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f773c0078e0 con 0x7f774c09e740 2024-05-20T18:33:00.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.959+0000 7f774bfff640 1 -- 172.21.15.5:0/2908917662 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c007cb0 con 0x7f774c09e740 2024-05-20T18:33:00.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.960+0000 7f775353e640 1 -- 172.21.15.5:0/2908917662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f774c09e740 msgr2=0x7f774c15d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:00.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.960+0000 7f775353e640 1 --2- 172.21.15.5:0/2908917662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f774c09e740 0x7f774c15d860 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f773c009dc0 tx=0x7f773c02a6a0 comp rx=0 tx=0).stop 2024-05-20T18:33:00.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.960+0000 7f775353e640 1 -- 172.21.15.5:0/2908917662 shutdown_connections 2024-05-20T18:33:00.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.960+0000 7f775353e640 1 --2- 172.21.15.5:0/2908917662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f774c09e740 0x7f774c15d860 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:00.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.960+0000 7f775353e640 1 -- 172.21.15.5:0/2908917662 >> 172.21.15.5:0/2908917662 conn(0x7f774c0d0e10 msgr2=0x7f774c0723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:00.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.960+0000 7f775353e640 1 -- 172.21.15.5:0/2908917662 shutdown_connections 2024-05-20T18:33:00.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.960+0000 7f775353e640 1 -- 172.21.15.5:0/2908917662 wait complete. 2024-05-20T18:33:00.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.962+0000 7f775353e640 1 Processor -- start 2024-05-20T18:33:00.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.962+0000 7f775353e640 1 -- start start 2024-05-20T18:33:00.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.963+0000 7f775353e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f774c09e740 0x7f774c15fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:00.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.963+0000 7f775353e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f774c0af730 0x7f774c154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:00.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.963+0000 7f775353e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f774c160310 0x7f774c159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:00.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.963+0000 7f775353e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f774c0cff50 con 0x7f774c0af730 2024-05-20T18:33:00.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.963+0000 7f775353e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f774c0cfdd0 con 0x7f774c09e740 2024-05-20T18:33:00.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.963+0000 7f775353e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f774c0afb40 con 0x7f774c160310 2024-05-20T18:33:00.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.963+0000 7f7750ad9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f774c0af730 0x7f774c154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:00.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.963+0000 7f7750ad9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f774c0af730 0x7f774c154a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44788/0 (socket says 172.21.15.5:44788) 2024-05-20T18:33:00.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.963+0000 7f7750ad9640 1 -- 172.21.15.5:0/1456594226 learned_addr learned my addr 172.21.15.5:0/1456594226 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:00.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.964+0000 7f77512da640 1 --2- 172.21.15.5:0/1456594226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f774c09e740 0x7f774c15fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:00.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.964+0000 7f7751adb640 1 --2- 172.21.15.5:0/1456594226 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f774c160310 0x7f774c159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:00.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.964+0000 7f7750ad9640 1 -- 172.21.15.5:0/1456594226 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f774c160310 msgr2=0x7f774c159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:00.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.964+0000 7f7750ad9640 1 --2- 172.21.15.5:0/1456594226 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f774c160310 0x7f774c159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:00.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.964+0000 7f7750ad9640 1 -- 172.21.15.5:0/1456594226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f774c09e740 msgr2=0x7f774c15fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:00.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.964+0000 7f7750ad9640 1 --2- 172.21.15.5:0/1456594226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f774c09e740 0x7f774c15fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:00.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.964+0000 7f7750ad9640 1 -- 172.21.15.5:0/1456594226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774c151370 con 0x7f774c0af730 2024-05-20T18:33:00.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.964+0000 7f7750ad9640 1 --2- 172.21.15.5:0/1456594226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f774c0af730 0x7f774c154a20 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f774000c420 tx=0x7f774000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:00.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.964+0000 7f7749ffb640 1 -- 172.21.15.5:0/1456594226 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77400074e0 con 0x7f774c0af730 2024-05-20T18:33:00.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.964+0000 7f7749ffb640 1 -- 172.21.15.5:0/1456594226 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77400047e0 con 0x7f774c0af730 2024-05-20T18:33:00.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.964+0000 7f775353e640 1 -- 172.21.15.5:0/1456594226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f774c151610 con 0x7f774c0af730 2024-05-20T18:33:00.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.964+0000 7f7749ffb640 1 -- 172.21.15.5:0/1456594226 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7740004bd0 con 0x7f774c0af730 2024-05-20T18:33:00.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.964+0000 7f775353e640 1 -- 172.21.15.5:0/1456594226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f774c151bd0 con 0x7f774c0af730 2024-05-20T18:33:00.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.965+0000 7f775353e640 1 -- 172.21.15.5:0/1456594226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f774c0690b0 con 0x7f774c0af730 2024-05-20T18:33:00.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.966+0000 7f7749ffb640 1 -- 172.21.15.5:0/1456594226 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7740011070 con 0x7f774c0af730 2024-05-20T18:33:00.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.967+0000 7f7749ffb640 1 --2- 172.21.15.5:0/1456594226 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f773806be30 0x7f773806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:00.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.967+0000 7f7749ffb640 1 -- 172.21.15.5:0/1456594226 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6273+0+0 (secure 0 0 0) 0x7f774008ad90 con 0x7f774c0af730 2024-05-20T18:33:00.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.967+0000 7f77512da640 1 --2- 172.21.15.5:0/1456594226 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f773806be30 0x7f773806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:00.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.967+0000 7f77512da640 1 --2- 172.21.15.5:0/1456594226 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f773806be30 0x7f773806e340 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f773c034000 tx=0x7f773c033040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:00.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:00.971+0000 7f7749ffb640 1 -- 172.21.15.5:0/1456594226 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f774000e280 con 0x7f774c0af730 2024-05-20T18:33:01.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:01.154+0000 7f775353e640 1 -- 172.21.15.5:0/1456594226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f774c159660 con 0x7f774c0af730 2024-05-20T18:33:02.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.034+0000 7f7749ffb640 1 -- 172.21.15.5:0/1456594226 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v80) ==== 181+0+0 (secure 0 0 0) 0x7f7740059460 con 0x7f774c0af730 2024-05-20T18:33:02.035 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-05-20T18:33:02.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.038+0000 7f775353e640 1 -- 172.21.15.5:0/1456594226 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f773806be30 msgr2=0x7f773806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:02.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.038+0000 7f775353e640 1 --2- 172.21.15.5:0/1456594226 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f773806be30 0x7f773806e340 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f773c034000 tx=0x7f773c033040 comp rx=0 tx=0).stop 2024-05-20T18:33:02.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.038+0000 7f775353e640 1 -- 172.21.15.5:0/1456594226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f774c0af730 msgr2=0x7f774c154a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:02.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.038+0000 7f775353e640 1 --2- 172.21.15.5:0/1456594226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f774c0af730 0x7f774c154a20 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f774000c420 tx=0x7f774000c8e0 comp rx=0 tx=0).stop 2024-05-20T18:33:02.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.038+0000 7f775353e640 1 -- 172.21.15.5:0/1456594226 shutdown_connections 2024-05-20T18:33:02.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.038+0000 7f775353e640 1 --2- 172.21.15.5:0/1456594226 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f773806be30 0x7f773806e340 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:02.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.038+0000 7f775353e640 1 --2- 172.21.15.5:0/1456594226 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f774c160310 0x7f774c159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:02.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.038+0000 7f775353e640 1 --2- 172.21.15.5:0/1456594226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f774c0af730 0x7f774c154a20 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:02.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.038+0000 7f775353e640 1 --2- 172.21.15.5:0/1456594226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f774c09e740 0x7f774c15fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:02.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.038+0000 7f775353e640 1 -- 172.21.15.5:0/1456594226 >> 172.21.15.5:0/1456594226 conn(0x7f774c0d0e10 msgr2=0x7f774c0b16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:02.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.038+0000 7f775353e640 1 -- 172.21.15.5:0/1456594226 shutdown_connections 2024-05-20T18:33:02.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.038+0000 7f775353e640 1 -- 172.21.15.5:0/1456594226 wait complete. 2024-05-20T18:33:02.053 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-20T18:33:02.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.187+0000 7f24703e3640 1 Processor -- start 2024-05-20T18:33:02.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.187+0000 7f24703e3640 1 -- start start 2024-05-20T18:33:02.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.189+0000 7f24703e3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f246806ff10 0x7f2468160b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:02.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.189+0000 7f24703e3640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f246815ddc0 0x7f246815e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:02.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.189+0000 7f24703e3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f246806c530 0x7f246806c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:02.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.189+0000 7f24703e3640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f246806ce80 con 0x7f246806c530 2024-05-20T18:33:02.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.189+0000 7f24703e3640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f24681613d0 con 0x7f246815ddc0 2024-05-20T18:33:02.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.189+0000 7f24703e3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f24681616b0 con 0x7f246806ff10 2024-05-20T18:33:02.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.189+0000 7f246e980640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f246806c530 0x7f246806c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:02.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.189+0000 7f246e980640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f246806c530 0x7f246806c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44796/0 (socket says 172.21.15.5:44796) 2024-05-20T18:33:02.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.189+0000 7f246e980640 1 -- 172.21.15.5:0/740769002 learned_addr learned my addr 172.21.15.5:0/740769002 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:02.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.189+0000 7f246d97e640 1 --2- 172.21.15.5:0/740769002 >> v2:172.21.15.63:3300/0 conn(0x7f246815ddc0 0x7f246815e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:02.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.189+0000 7f246e980640 1 -- 172.21.15.5:0/740769002 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f246806ff10 msgr2=0x7f2468160b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:02.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.189+0000 7f246e17f640 1 --2- 172.21.15.5:0/740769002 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f246806ff10 0x7f2468160b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:02.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.189+0000 7f246e980640 1 --2- 172.21.15.5:0/740769002 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f246806ff10 0x7f2468160b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:02.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.189+0000 7f246e980640 1 -- 172.21.15.5:0/740769002 >> v2:172.21.15.63:3300/0 conn(0x7f246815ddc0 msgr2=0x7f246815e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:02.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.189+0000 7f246e980640 1 --2- 172.21.15.5:0/740769002 >> v2:172.21.15.63:3300/0 conn(0x7f246815ddc0 0x7f246815e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:02.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.189+0000 7f246e980640 1 -- 172.21.15.5:0/740769002 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2468161990 con 0x7f246806c530 2024-05-20T18:33:02.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.189+0000 7f246d97e640 1 --2- 172.21.15.5:0/740769002 >> v2:172.21.15.63:3300/0 conn(0x7f246815ddc0 0x7f246815e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:33:02.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.189+0000 7f246e17f640 1 --2- 172.21.15.5:0/740769002 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f246806ff10 0x7f2468160b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:33:02.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.190+0000 7f246e980640 1 --2- 172.21.15.5:0/740769002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246806c530 0x7f246806c940 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f24680b5b80 tx=0x7f246002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6886144e53277c01 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:02.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.190+0000 7f246d17d640 1 -- 172.21.15.5:0/740769002 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2460037070 con 0x7f246806c530 2024-05-20T18:33:02.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.190+0000 7f246d17d640 1 -- 172.21.15.5:0/740769002 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2460032440 con 0x7f246806c530 2024-05-20T18:33:02.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.190+0000 7f246d17d640 1 -- 172.21.15.5:0/740769002 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2460032810 con 0x7f246806c530 2024-05-20T18:33:02.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.191+0000 7f24703e3640 1 -- 172.21.15.5:0/740769002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246806c530 msgr2=0x7f246806c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:02.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.191+0000 7f24703e3640 1 --2- 172.21.15.5:0/740769002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246806c530 0x7f246806c940 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f24680b5b80 tx=0x7f246002a140 comp rx=0 tx=0).stop 2024-05-20T18:33:02.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.191+0000 7f24703e3640 1 -- 172.21.15.5:0/740769002 shutdown_connections 2024-05-20T18:33:02.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.191+0000 7f24703e3640 1 --2- 172.21.15.5:0/740769002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246806c530 0x7f246806c940 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:02.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.191+0000 7f24703e3640 1 --2- 172.21.15.5:0/740769002 >> v2:172.21.15.63:3300/0 conn(0x7f246815ddc0 0x7f246815e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:02.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.191+0000 7f24703e3640 1 --2- 172.21.15.5:0/740769002 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f246806ff10 0x7f2468160b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:02.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.191+0000 7f24703e3640 1 -- 172.21.15.5:0/740769002 >> 172.21.15.5:0/740769002 conn(0x7f24680d0e10 msgr2=0x7f24680723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:02.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.191+0000 7f24703e3640 1 -- 172.21.15.5:0/740769002 shutdown_connections 2024-05-20T18:33:02.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.191+0000 7f24703e3640 1 -- 172.21.15.5:0/740769002 wait complete. 2024-05-20T18:33:02.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.193+0000 7f24703e3640 1 Processor -- start 2024-05-20T18:33:02.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.193+0000 7f24703e3640 1 -- start start 2024-05-20T18:33:02.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.194+0000 7f24703e3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f246806c530 0x7f246814efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:02.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.194+0000 7f24703e3640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f246806ff10 0x7f246815f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:02.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.194+0000 7f24703e3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246815ddc0 0x7f246815fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:02.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.194+0000 7f24703e3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f246815aec0 con 0x7f246815ddc0 2024-05-20T18:33:02.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.194+0000 7f24703e3640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f24681610c0 con 0x7f246806ff10 2024-05-20T18:33:02.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.194+0000 7f24703e3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f24680ad830 con 0x7f246806c530 2024-05-20T18:33:02.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.194+0000 7f246e980640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246815ddc0 0x7f246815fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:02.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.195+0000 7f246e980640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246815ddc0 0x7f246815fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44800/0 (socket says 172.21.15.5:44800) 2024-05-20T18:33:02.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.195+0000 7f246e980640 1 -- 172.21.15.5:0/1482998915 learned_addr learned my addr 172.21.15.5:0/1482998915 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:02.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.195+0000 7f246d97e640 1 --2- 172.21.15.5:0/1482998915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f246806ff10 0x7f246815f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:02.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.195+0000 7f246e17f640 1 --2- 172.21.15.5:0/1482998915 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f246806c530 0x7f246814efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:02.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.195+0000 7f246e980640 1 -- 172.21.15.5:0/1482998915 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f246806c530 msgr2=0x7f246814efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:02.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.195+0000 7f246e980640 1 --2- 172.21.15.5:0/1482998915 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f246806c530 0x7f246814efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:02.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.195+0000 7f246e980640 1 -- 172.21.15.5:0/1482998915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f246806ff10 msgr2=0x7f246815f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:02.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.195+0000 7f246e980640 1 --2- 172.21.15.5:0/1482998915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f246806ff10 0x7f246815f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:02.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.195+0000 7f246e980640 1 -- 172.21.15.5:0/1482998915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f246814f4f0 con 0x7f246815ddc0 2024-05-20T18:33:02.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.195+0000 7f246e980640 1 --2- 172.21.15.5:0/1482998915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246815ddc0 0x7f246815fd00 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f246002a180 tx=0x7f24600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:02.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.195+0000 7f245effd640 1 -- 172.21.15.5:0/1482998915 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2460037040 con 0x7f246815ddc0 2024-05-20T18:33:02.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.195+0000 7f245effd640 1 -- 172.21.15.5:0/1482998915 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2460042070 con 0x7f246815ddc0 2024-05-20T18:33:02.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.195+0000 7f24703e3640 1 -- 172.21.15.5:0/1482998915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24680b1b10 con 0x7f246815ddc0 2024-05-20T18:33:02.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.195+0000 7f245effd640 1 -- 172.21.15.5:0/1482998915 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2460030bd0 con 0x7f246815ddc0 2024-05-20T18:33:02.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.195+0000 7f24703e3640 1 -- 172.21.15.5:0/1482998915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24680b20d0 con 0x7f246815ddc0 2024-05-20T18:33:02.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.196+0000 7f24703e3640 1 -- 172.21.15.5:0/1482998915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24680b22d0 con 0x7f246815ddc0 2024-05-20T18:33:02.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.197+0000 7f245effd640 1 -- 172.21.15.5:0/1482998915 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2460047460 con 0x7f246815ddc0 2024-05-20T18:33:02.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.198+0000 7f245effd640 1 --2- 172.21.15.5:0/1482998915 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f244c06be60 0x7f244c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:02.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.198+0000 7f246e17f640 1 --2- 172.21.15.5:0/1482998915 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f244c06be60 0x7f244c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:02.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.198+0000 7f245effd640 1 -- 172.21.15.5:0/1482998915 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5835+0+0 (secure 0 0 0) 0x7f24600abb50 con 0x7f246815ddc0 2024-05-20T18:33:02.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.198+0000 7f246e17f640 1 --2- 172.21.15.5:0/1482998915 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f244c06be60 0x7f244c06e370 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2464007fc0 tx=0x7f2464007550 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:02.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.202+0000 7f245effd640 1 -- 172.21.15.5:0/1482998915 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f24680b22d0 con 0x7f246815ddc0 2024-05-20T18:33:02.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:02.385+0000 7f24703e3640 1 -- 172.21.15.5:0/1482998915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f24680690b0 con 0x7f246815ddc0 2024-05-20T18:33:03.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.039+0000 7f245effd640 1 -- 172.21.15.5:0/1482998915 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v81) ==== 169+0+0 (secure 0 0 0) 0x7f2460079da0 con 0x7f246815ddc0 2024-05-20T18:33:03.040 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-05-20T18:33:03.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.043+0000 7f24703e3640 1 -- 172.21.15.5:0/1482998915 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f244c06be60 msgr2=0x7f244c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:03.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.043+0000 7f24703e3640 1 --2- 172.21.15.5:0/1482998915 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f244c06be60 0x7f244c06e370 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2464007fc0 tx=0x7f2464007550 comp rx=0 tx=0).stop 2024-05-20T18:33:03.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.043+0000 7f24703e3640 1 -- 172.21.15.5:0/1482998915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246815ddc0 msgr2=0x7f246815fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:03.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.043+0000 7f24703e3640 1 --2- 172.21.15.5:0/1482998915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246815ddc0 0x7f246815fd00 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f246002a180 tx=0x7f24600046c0 comp rx=0 tx=0).stop 2024-05-20T18:33:03.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.044+0000 7f24703e3640 1 -- 172.21.15.5:0/1482998915 shutdown_connections 2024-05-20T18:33:03.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.044+0000 7f24703e3640 1 --2- 172.21.15.5:0/1482998915 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f244c06be60 0x7f244c06e370 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:03.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.044+0000 7f24703e3640 1 --2- 172.21.15.5:0/1482998915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246815ddc0 0x7f246815fd00 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:03.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.044+0000 7f24703e3640 1 --2- 172.21.15.5:0/1482998915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f246806ff10 0x7f246815f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:03.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.044+0000 7f24703e3640 1 --2- 172.21.15.5:0/1482998915 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f246806c530 0x7f246814efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:03.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.044+0000 7f24703e3640 1 -- 172.21.15.5:0/1482998915 >> 172.21.15.5:0/1482998915 conn(0x7f24680d0e10 msgr2=0x7f246815bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:03.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.044+0000 7f24703e3640 1 -- 172.21.15.5:0/1482998915 shutdown_connections 2024-05-20T18:33:03.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.044+0000 7f24703e3640 1 -- 172.21.15.5:0/1482998915 wait complete. 2024-05-20T18:33:03.060 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-20T18:33:03.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.183+0000 7f21f30d3640 1 Processor -- start 2024-05-20T18:33:03.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.184+0000 7f21f30d3640 1 -- start start 2024-05-20T18:33:03.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.185+0000 7f21f30d3640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f21ec0cd710 0x7f21ec162e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:03.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.185+0000 7f21f30d3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f21ec06d6b0 0x7f21ec15f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:03.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.185+0000 7f21f30d3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21ec0cf3d0 0x7f21ec0cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:03.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.185+0000 7f21f30d3640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f21ec0d0ba0 con 0x7f21ec06d6b0 2024-05-20T18:33:03.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.185+0000 7f21f30d3640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f21ec0cfd20 con 0x7f21ec0cd710 2024-05-20T18:33:03.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.185+0000 7f21f30d3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f21ec06c8c0 con 0x7f21ec0cf3d0 2024-05-20T18:33:03.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.185+0000 7f21ebfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f21ec06d6b0 0x7f21ec15f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:03.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.185+0000 7f21ebfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f21ec06d6b0 0x7f21ec15f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44802/0 (socket says 172.21.15.5:44802) 2024-05-20T18:33:03.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.185+0000 7f21ebfff640 1 -- 172.21.15.5:0/3232872665 learned_addr learned my addr 172.21.15.5:0/3232872665 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:03.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.185+0000 7f21ebfff640 1 -- 172.21.15.5:0/3232872665 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21ec0cf3d0 msgr2=0x7f21ec0cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:03.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.185+0000 7f21f1670640 1 --2- 172.21.15.5:0/3232872665 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21ec0cf3d0 0x7f21ec0cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:03.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.185+0000 7f21f0e6f640 1 --2- 172.21.15.5:0/3232872665 >> v2:172.21.15.63:3300/0 conn(0x7f21ec0cd710 0x7f21ec162e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:03.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.185+0000 7f21ebfff640 1 --2- 172.21.15.5:0/3232872665 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21ec0cf3d0 0x7f21ec0cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:03.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.185+0000 7f21ebfff640 1 -- 172.21.15.5:0/3232872665 >> v2:172.21.15.63:3300/0 conn(0x7f21ec0cd710 msgr2=0x7f21ec162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:03.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.185+0000 7f21ebfff640 1 --2- 172.21.15.5:0/3232872665 >> v2:172.21.15.63:3300/0 conn(0x7f21ec0cd710 0x7f21ec162e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:03.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.185+0000 7f21ebfff640 1 -- 172.21.15.5:0/3232872665 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21ec06dac0 con 0x7f21ec06d6b0 2024-05-20T18:33:03.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.186+0000 7f21f1670640 1 --2- 172.21.15.5:0/3232872665 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21ec0cf3d0 0x7f21ec0cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:33:03.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.186+0000 7f21ebfff640 1 --2- 172.21.15.5:0/3232872665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21ec06d6b0 0x7f21ec15f2b0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f21ec06dc90 tx=0x7f21e002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0fbd0540467f703 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:03.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.186+0000 7f21eb7fe640 1 -- 172.21.15.5:0/3232872665 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0037070 con 0x7f21ec06d6b0 2024-05-20T18:33:03.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.186+0000 7f21eb7fe640 1 -- 172.21.15.5:0/3232872665 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e0032440 con 0x7f21ec06d6b0 2024-05-20T18:33:03.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.186+0000 7f21eb7fe640 1 -- 172.21.15.5:0/3232872665 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0032810 con 0x7f21ec06d6b0 2024-05-20T18:33:03.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.187+0000 7f21f30d3640 1 -- 172.21.15.5:0/3232872665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21ec06d6b0 msgr2=0x7f21ec15f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:03.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.187+0000 7f21f30d3640 1 --2- 172.21.15.5:0/3232872665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21ec06d6b0 0x7f21ec15f2b0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f21ec06dc90 tx=0x7f21e002a1c0 comp rx=0 tx=0).stop 2024-05-20T18:33:03.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.187+0000 7f21f30d3640 1 -- 172.21.15.5:0/3232872665 shutdown_connections 2024-05-20T18:33:03.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.187+0000 7f21f30d3640 1 --2- 172.21.15.5:0/3232872665 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21ec0cf3d0 0x7f21ec0cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:03.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.187+0000 7f21f30d3640 1 --2- 172.21.15.5:0/3232872665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21ec06d6b0 0x7f21ec15f2b0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:03.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.187+0000 7f21f30d3640 1 --2- 172.21.15.5:0/3232872665 >> v2:172.21.15.63:3300/0 conn(0x7f21ec0cd710 0x7f21ec162e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:03.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.187+0000 7f21f30d3640 1 -- 172.21.15.5:0/3232872665 >> 172.21.15.5:0/3232872665 conn(0x7f21ec0b44d0 msgr2=0x7f21ec0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:03.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.187+0000 7f21f30d3640 1 -- 172.21.15.5:0/3232872665 shutdown_connections 2024-05-20T18:33:03.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.187+0000 7f21f30d3640 1 -- 172.21.15.5:0/3232872665 wait complete. 2024-05-20T18:33:03.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.189+0000 7f21f30d3640 1 Processor -- start 2024-05-20T18:33:03.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.189+0000 7f21f30d3640 1 -- start start 2024-05-20T18:33:03.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.190+0000 7f21f30d3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21ec06d6b0 0x7f21ec1f95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:03.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.190+0000 7f21f30d3640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21ec0cd710 0x7f21ec0c9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:03.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.190+0000 7f21f30d3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21ec0cf3d0 0x7f21ec0cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:03.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.190+0000 7f21f30d3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec1639a0 con 0x7f21ec0cf3d0 2024-05-20T18:33:03.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.190+0000 7f21f30d3640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec1636c0 con 0x7f21ec0cd710 2024-05-20T18:33:03.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.190+0000 7f21f30d3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f21ec163c80 con 0x7f21ec06d6b0 2024-05-20T18:33:03.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.191+0000 7f21f1670640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21ec0cf3d0 0x7f21ec0cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:03.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.191+0000 7f21f0e6f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21ec06d6b0 0x7f21ec1f95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:03.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.191+0000 7f21f1670640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21ec0cf3d0 0x7f21ec0cb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44812/0 (socket says 172.21.15.5:44812) 2024-05-20T18:33:03.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.191+0000 7f21f1670640 1 -- 172.21.15.5:0/2832017443 learned_addr learned my addr 172.21.15.5:0/2832017443 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:03.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.191+0000 7f21ebfff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21ec0cd710 0x7f21ec0c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:03.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.191+0000 7f21f1670640 1 -- 172.21.15.5:0/2832017443 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21ec06d6b0 msgr2=0x7f21ec1f95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:03.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.191+0000 7f21f1670640 1 --2- 172.21.15.5:0/2832017443 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21ec06d6b0 0x7f21ec1f95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:03.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.191+0000 7f21f1670640 1 -- 172.21.15.5:0/2832017443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21ec0cd710 msgr2=0x7f21ec0c9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:03.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.191+0000 7f21f1670640 1 --2- 172.21.15.5:0/2832017443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21ec0cd710 0x7f21ec0c9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:03.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.191+0000 7f21f1670640 1 -- 172.21.15.5:0/2832017443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21ec0b6050 con 0x7f21ec0cf3d0 2024-05-20T18:33:03.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.191+0000 7f21f1670640 1 --2- 172.21.15.5:0/2832017443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21ec0cf3d0 0x7f21ec0cb2b0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f21e400cd40 tx=0x7f21e400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:03.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.191+0000 7f21e97fa640 1 -- 172.21.15.5:0/2832017443 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4003b90 con 0x7f21ec0cf3d0 2024-05-20T18:33:03.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.191+0000 7f21e97fa640 1 -- 172.21.15.5:0/2832017443 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e4003d50 con 0x7f21ec0cf3d0 2024-05-20T18:33:03.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.191+0000 7f21e97fa640 1 -- 172.21.15.5:0/2832017443 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4004260 con 0x7f21ec0cf3d0 2024-05-20T18:33:03.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.191+0000 7f21f30d3640 1 -- 172.21.15.5:0/2832017443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21ec0b5a90 con 0x7f21ec0cf3d0 2024-05-20T18:33:03.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.191+0000 7f21f30d3640 1 -- 172.21.15.5:0/2832017443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21ec1f56e0 con 0x7f21ec0cf3d0 2024-05-20T18:33:03.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.193+0000 7f21f30d3640 1 -- 172.21.15.5:0/2832017443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21ec075b00 con 0x7f21ec0cf3d0 2024-05-20T18:33:03.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.193+0000 7f21e97fa640 1 -- 172.21.15.5:0/2832017443 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f21e4004400 con 0x7f21ec0cf3d0 2024-05-20T18:33:03.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.194+0000 7f21e97fa640 1 --2- 172.21.15.5:0/2832017443 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f21d806be60 0x7f21d806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:03.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.194+0000 7f21e97fa640 1 -- 172.21.15.5:0/2832017443 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5421+0+0 (secure 0 0 0) 0x7f21e408b6e0 con 0x7f21ec0cf3d0 2024-05-20T18:33:03.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.194+0000 7f21f0e6f640 1 --2- 172.21.15.5:0/2832017443 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f21d806be60 0x7f21d806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:03.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.194+0000 7f21f0e6f640 1 --2- 172.21.15.5:0/2832017443 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f21d806be60 0x7f21d806e370 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f21ec162d60 tx=0x7f21dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:03.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.198+0000 7f21e97fa640 1 -- 172.21.15.5:0/2832017443 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f21e4014030 con 0x7f21ec0cf3d0 2024-05-20T18:33:03.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.390+0000 7f21f30d3640 1 -- 172.21.15.5:0/2832017443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f21ec1f9b20 con 0x7f21ec0cf3d0 2024-05-20T18:33:03.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.393+0000 7f21e97fa640 1 -- 172.21.15.5:0/2832017443 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f21e4059ad0 con 0x7f21ec0cf3d0 2024-05-20T18:33:03.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.395+0000 7f21f30d3640 1 -- 172.21.15.5:0/2832017443 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f21d806be60 msgr2=0x7f21d806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:03.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.395+0000 7f21f30d3640 1 --2- 172.21.15.5:0/2832017443 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f21d806be60 0x7f21d806e370 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f21ec162d60 tx=0x7f21dc008040 comp rx=0 tx=0).stop 2024-05-20T18:33:03.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.395+0000 7f21f30d3640 1 -- 172.21.15.5:0/2832017443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21ec0cf3d0 msgr2=0x7f21ec0cb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:03.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.395+0000 7f21f30d3640 1 --2- 172.21.15.5:0/2832017443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21ec0cf3d0 0x7f21ec0cb2b0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f21e400cd40 tx=0x7f21e400a650 comp rx=0 tx=0).stop 2024-05-20T18:33:03.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.396+0000 7f21f30d3640 1 -- 172.21.15.5:0/2832017443 shutdown_connections 2024-05-20T18:33:03.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.396+0000 7f21f30d3640 1 --2- 172.21.15.5:0/2832017443 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f21d806be60 0x7f21d806e370 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:03.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.396+0000 7f21f30d3640 1 --2- 172.21.15.5:0/2832017443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f21ec0cf3d0 0x7f21ec0cb2b0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:03.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.396+0000 7f21f30d3640 1 --2- 172.21.15.5:0/2832017443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21ec0cd710 0x7f21ec0c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:03.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.396+0000 7f21f30d3640 1 --2- 172.21.15.5:0/2832017443 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f21ec06d6b0 0x7f21ec1f95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:03.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.396+0000 7f21f30d3640 1 -- 172.21.15.5:0/2832017443 >> 172.21.15.5:0/2832017443 conn(0x7f21ec0b44d0 msgr2=0x7f21ec159ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:03.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.396+0000 7f21f30d3640 1 -- 172.21.15.5:0/2832017443 shutdown_connections 2024-05-20T18:33:03.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.396+0000 7f21f30d3640 1 -- 172.21.15.5:0/2832017443 wait complete. 2024-05-20T18:33:03.409 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-05-20T18:33:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.523+0000 7f23f13e0640 1 Processor -- start 2024-05-20T18:33:03.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.523+0000 7f23f13e0640 1 -- start start 2024-05-20T18:33:03.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.524+0000 7f23f13e0640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f23ec15d530 0x7f23ec15aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:03.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.524+0000 7f23f13e0640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f23ec0b1700 0x7f23ec0b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:03.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.524+0000 7f23f13e0640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23ec0cd580 con 0x7f23ec070850 2024-05-20T18:33:03.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.525+0000 7f23f13e0640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f23ec0ce190 con 0x7f23ec0b1700 2024-05-20T18:33:03.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.525+0000 7f23f13e0640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f23ec0cd860 con 0x7f23ec15d530 2024-05-20T18:33:03.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.525+0000 7f23eb7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f23ec070850 0x7f23ec070c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46870/0 (socket says 172.21.15.5:46870) 2024-05-20T18:33:03.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.525+0000 7f23eb7fe640 1 -- 172.21.15.5:0/4141540594 learned_addr learned my addr 172.21.15.5:0/4141540594 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:33:03.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.525+0000 7f23ea7fc640 1 --2- 172.21.15.5:0/4141540594 >> v2:172.21.15.5:3300/0 conn(0x7f23ec0b1700 0x7f23ec0b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:03.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.525+0000 7f23eaffd640 1 --2- 172.21.15.5:0/4141540594 >> v2:172.21.15.63:3300/0 conn(0x7f23ec15d530 0x7f23ec15aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:03.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.525+0000 7f23e9ffb640 1 -- 172.21.15.5:0/4141540594 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1656452971 0 0) 0x7f23ec0cd580 con 0x7f23ec070850 2024-05-20T18:33:03.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.525+0000 7f23e9ffb640 1 -- 172.21.15.5:0/4141540594 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23d4003520 con 0x7f23ec070850 2024-05-20T18:33:03.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.525+0000 7f23ea7fc640 1 -- 172.21.15.5:0/4141540594 >> v2:172.21.15.63:3300/0 conn(0x7f23ec15d530 msgr2=0x7f23ec15aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:03.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.525+0000 7f23ea7fc640 1 --2- 172.21.15.5:0/4141540594 >> v2:172.21.15.63:3300/0 conn(0x7f23ec15d530 0x7f23ec15aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:03.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.525+0000 7f23ea7fc640 1 -- 172.21.15.5:0/4141540594 >> v1:172.21.15.5:6789/0 conn(0x7f23ec070850 legacy=0x7f23ec070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:03.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.525+0000 7f23ea7fc640 1 -- 172.21.15.5:0/4141540594 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23ec0cdb40 con 0x7f23ec0b1700 2024-05-20T18:33:03.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.526+0000 7f23ea7fc640 1 --2- 172.21.15.5:0/4141540594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23ec0b1700 0x7f23ec0b1b10 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f23ec04f790 tx=0x7f23e002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8bcfb657b48561d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:03.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.526+0000 7f23e9ffb640 1 -- 172.21.15.5:0/4141540594 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0037070 con 0x7f23ec0b1700 2024-05-20T18:33:03.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.526+0000 7f23e9ffb640 1 -- 172.21.15.5:0/4141540594 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23e0032440 con 0x7f23ec0b1700 2024-05-20T18:33:03.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.526+0000 7f23e9ffb640 1 -- 172.21.15.5:0/4141540594 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0032810 con 0x7f23ec0b1700 2024-05-20T18:33:03.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.527+0000 7f23f13e0640 1 -- 172.21.15.5:0/4141540594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23ec0b1700 msgr2=0x7f23ec0b1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:03.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.527+0000 7f23f13e0640 1 --2- 172.21.15.5:0/4141540594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23ec0b1700 0x7f23ec0b1b10 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f23ec04f790 tx=0x7f23e002a1a0 comp rx=0 tx=0).stop 2024-05-20T18:33:03.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.527+0000 7f23f13e0640 1 -- 172.21.15.5:0/4141540594 shutdown_connections 2024-05-20T18:33:03.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.527+0000 7f23f13e0640 1 --2- 172.21.15.5:0/4141540594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23ec0b1700 0x7f23ec0b1b10 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:03.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.527+0000 7f23f13e0640 1 --2- 172.21.15.5:0/4141540594 >> v2:172.21.15.63:3300/0 conn(0x7f23ec15d530 0x7f23ec15aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:03.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.527+0000 7f23f13e0640 1 -- 172.21.15.5:0/4141540594 >> 172.21.15.5:0/4141540594 conn(0x7f23ec0d29d0 msgr2=0x7f23ec0d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:03.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.527+0000 7f23f13e0640 1 -- 172.21.15.5:0/4141540594 shutdown_connections 2024-05-20T18:33:03.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.527+0000 7f23f13e0640 1 -- 172.21.15.5:0/4141540594 wait complete. 2024-05-20T18:33:03.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.529+0000 7f23f13e0640 1 Processor -- start 2024-05-20T18:33:03.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.529+0000 7f23f13e0640 1 -- start start 2024-05-20T18:33:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.530+0000 7f23f13e0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f23ec070850 0x7f23ec14c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.530+0000 7f23f13e0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23ec0b1700 0x7f23ec0c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.530+0000 7f23f13e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23ec15d530 0x7f23ec0c6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.530+0000 7f23f13e0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec0b2050 con 0x7f23ec15d530 2024-05-20T18:33:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.530+0000 7f23f13e0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec0cd230 con 0x7f23ec0b1700 2024-05-20T18:33:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.530+0000 7f23f13e0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f23ec071370 con 0x7f23ec070850 2024-05-20T18:33:03.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.530+0000 7f23eb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23ec15d530 0x7f23ec0c6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:03.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.530+0000 7f23eb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23ec15d530 0x7f23ec0c6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44834/0 (socket says 172.21.15.5:44834) 2024-05-20T18:33:03.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.530+0000 7f23eb7fe640 1 -- 172.21.15.5:0/1912888264 learned_addr learned my addr 172.21.15.5:0/1912888264 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:03.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.530+0000 7f23eaffd640 1 --2- 172.21.15.5:0/1912888264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f23ec070850 0x7f23ec14c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:03.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.530+0000 7f23ea7fc640 1 --2- 172.21.15.5:0/1912888264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23ec0b1700 0x7f23ec0c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:03.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.531+0000 7f23eb7fe640 1 -- 172.21.15.5:0/1912888264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f23ec070850 msgr2=0x7f23ec14c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:03.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.531+0000 7f23eb7fe640 1 --2- 172.21.15.5:0/1912888264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f23ec070850 0x7f23ec14c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:03.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.531+0000 7f23eb7fe640 1 -- 172.21.15.5:0/1912888264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23ec0b1700 msgr2=0x7f23ec0c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:03.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.531+0000 7f23eb7fe640 1 --2- 172.21.15.5:0/1912888264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23ec0b1700 0x7f23ec0c6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:03.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.531+0000 7f23eb7fe640 1 -- 172.21.15.5:0/1912888264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23ec0b6c80 con 0x7f23ec15d530 2024-05-20T18:33:03.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.531+0000 7f23ea7fc640 1 --2- 172.21.15.5:0/1912888264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23ec0b1700 0x7f23ec0c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:03.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.531+0000 7f23eb7fe640 1 --2- 172.21.15.5:0/1912888264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23ec15d530 0x7f23ec0c6a40 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f23e4001eb0 tx=0x7f23e400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:03.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.531+0000 7f23d37fe640 1 -- 172.21.15.5:0/1912888264 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4017070 con 0x7f23ec15d530 2024-05-20T18:33:03.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.531+0000 7f23d37fe640 1 -- 172.21.15.5:0/1912888264 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23e4005ce0 con 0x7f23ec15d530 2024-05-20T18:33:03.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.531+0000 7f23f13e0640 1 -- 172.21.15.5:0/1912888264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23ec0b6f60 con 0x7f23ec15d530 2024-05-20T18:33:03.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.531+0000 7f23d37fe640 1 -- 172.21.15.5:0/1912888264 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e40072d0 con 0x7f23ec15d530 2024-05-20T18:33:03.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.531+0000 7f23f13e0640 1 -- 172.21.15.5:0/1912888264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23ec0b7520 con 0x7f23ec15d530 2024-05-20T18:33:03.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.532+0000 7f23f13e0640 1 -- 172.21.15.5:0/1912888264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23ec075b00 con 0x7f23ec15d530 2024-05-20T18:33:03.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.533+0000 7f23d37fe640 1 -- 172.21.15.5:0/1912888264 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f23e401a070 con 0x7f23ec15d530 2024-05-20T18:33:03.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.533+0000 7f23d37fe640 1 --2- 172.21.15.5:0/1912888264 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f23d806be60 0x7f23d806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:03.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.534+0000 7f23eaffd640 1 --2- 172.21.15.5:0/1912888264 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f23d806be60 0x7f23d806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:03.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.534+0000 7f23d37fe640 1 -- 172.21.15.5:0/1912888264 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5421+0+0 (secure 0 0 0) 0x7f23e4089ff0 con 0x7f23ec15d530 2024-05-20T18:33:03.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.534+0000 7f23eaffd640 1 --2- 172.21.15.5:0/1912888264 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f23d806be60 0x7f23d806e370 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0d29a0 tx=0x7f23dc00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:03.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.538+0000 7f23d37fe640 1 -- 172.21.15.5:0/1912888264 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f23e4058470 con 0x7f23ec15d530 2024-05-20T18:33:03.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:03.795+0000 7f23f13e0640 1 -- 172.21.15.5:0/1912888264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7f23ec06f850 con 0x7f23ec15d530 2024-05-20T18:33:04.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.040+0000 7f23d37fe640 1 -- 172.21.15.5:0/1912888264 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v300) ==== 166+0+0 (secure 0 0 0) 0x7f23e405c770 con 0x7f23ec15d530 2024-05-20T18:33:04.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.043+0000 7f23f13e0640 1 -- 172.21.15.5:0/1912888264 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f23d806be60 msgr2=0x7f23d806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:04.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.043+0000 7f23f13e0640 1 --2- 172.21.15.5:0/1912888264 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f23d806be60 0x7f23d806e370 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0d29a0 tx=0x7f23dc00b040 comp rx=0 tx=0).stop 2024-05-20T18:33:04.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.043+0000 7f23f13e0640 1 -- 172.21.15.5:0/1912888264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23ec15d530 msgr2=0x7f23ec0c6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:04.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.043+0000 7f23f13e0640 1 --2- 172.21.15.5:0/1912888264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23ec15d530 0x7f23ec0c6a40 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f23e4001eb0 tx=0x7f23e400c820 comp rx=0 tx=0).stop 2024-05-20T18:33:04.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.043+0000 7f23f13e0640 1 -- 172.21.15.5:0/1912888264 shutdown_connections 2024-05-20T18:33:04.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.043+0000 7f23f13e0640 1 --2- 172.21.15.5:0/1912888264 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f23d806be60 0x7f23d806e370 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:04.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.043+0000 7f23f13e0640 1 --2- 172.21.15.5:0/1912888264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23ec15d530 0x7f23ec0c6a40 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:04.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.043+0000 7f23f13e0640 1 --2- 172.21.15.5:0/1912888264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f23ec0b1700 0x7f23ec0c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:04.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.043+0000 7f23f13e0640 1 --2- 172.21.15.5:0/1912888264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f23ec070850 0x7f23ec14c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:04.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.043+0000 7f23f13e0640 1 -- 172.21.15.5:0/1912888264 >> 172.21.15.5:0/1912888264 conn(0x7f23ec0d29d0 msgr2=0x7f23ec0c5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:04.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.043+0000 7f23f13e0640 1 -- 172.21.15.5:0/1912888264 shutdown_connections 2024-05-20T18:33:04.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.043+0000 7f23f13e0640 1 -- 172.21.15.5:0/1912888264 wait complete. 2024-05-20T18:33:04.055 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-05-20T18:33:04.056 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-05-20T18:33:04.056 INFO:tasks.cephfs_test_runner:test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-20T18:33:04.056 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:33:04.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.172+0000 7fd3d5e06640 1 Processor -- start 2024-05-20T18:33:04.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.172+0000 7fd3d5e06640 1 -- start start 2024-05-20T18:33:04.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.173+0000 7fd3d5e06640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd3d006d3d0 0x7fd3d006d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:04.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.173+0000 7fd3d5e06640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3d006dd20 con 0x7fd3d00b8190 2024-05-20T18:33:04.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.173+0000 7fd3d5e06640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3d00d0630 con 0x7fd3d0162820 2024-05-20T18:33:04.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.173+0000 7fd3d5e06640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fd3d00d39c0 con 0x7fd3d006d3d0 2024-05-20T18:33:04.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.174+0000 7fd3c7fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd3d006d3d0 0x7fd3d006d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:04.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.174+0000 7fd3cffff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd3d00b8190 0x7fd3d00b85a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46886/0 (socket says 172.21.15.5:46886) 2024-05-20T18:33:04.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.174+0000 7fd3cffff640 1 -- 172.21.15.5:0/2266987308 learned_addr learned my addr 172.21.15.5:0/2266987308 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:33:04.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.174+0000 7fd3c7fff640 1 -- 172.21.15.5:0/2266987308 >> v1:172.21.15.63:6789/0 conn(0x7fd3d0162820 legacy=0x7fd3d00b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:04.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.174+0000 7fd3ceffd640 1 -- 172.21.15.5:0/2266987308 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 782980484 0 0) 0x7fd3c8001490 con 0x7fd3d00b8190 2024-05-20T18:33:04.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.174+0000 7fd3c7fff640 1 -- 172.21.15.5:0/2266987308 >> v1:172.21.15.5:6789/0 conn(0x7fd3d00b8190 legacy=0x7fd3d00b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.174+0000 7fd3c7fff640 1 -- 172.21.15.5:0/2266987308 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d00d3ca0 con 0x7fd3d006d3d0 2024-05-20T18:33:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.175+0000 7fd3c7fff640 1 --2- 172.21.15.5:0/2266987308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3d006d3d0 0x7fd3d006d7e0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fd3b80097a0 tx=0x7fd3b802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3250e2eff15348 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.175+0000 7fd3ceffd640 1 -- 172.21.15.5:0/2266987308 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8037070 con 0x7fd3d006d3d0 2024-05-20T18:33:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.175+0000 7fd3ceffd640 1 -- 172.21.15.5:0/2266987308 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b8032440 con 0x7fd3d006d3d0 2024-05-20T18:33:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.175+0000 7fd3ceffd640 1 -- 172.21.15.5:0/2266987308 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8032810 con 0x7fd3d006d3d0 2024-05-20T18:33:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.176+0000 7fd3d5e06640 1 -- 172.21.15.5:0/2266987308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3d006d3d0 msgr2=0x7fd3d006d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.176+0000 7fd3d5e06640 1 --2- 172.21.15.5:0/2266987308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3d006d3d0 0x7fd3d006d7e0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fd3b80097a0 tx=0x7fd3b802a230 comp rx=0 tx=0).stop 2024-05-20T18:33:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.176+0000 7fd3d5e06640 1 -- 172.21.15.5:0/2266987308 shutdown_connections 2024-05-20T18:33:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.176+0000 7fd3d5e06640 1 --2- 172.21.15.5:0/2266987308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3d006d3d0 0x7fd3d006d7e0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:04.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.176+0000 7fd3d5e06640 1 -- 172.21.15.5:0/2266987308 >> 172.21.15.5:0/2266987308 conn(0x7fd3d00d4bd0 msgr2=0x7fd3d00d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:04.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.176+0000 7fd3d5e06640 1 -- 172.21.15.5:0/2266987308 shutdown_connections 2024-05-20T18:33:04.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.176+0000 7fd3d5e06640 1 -- 172.21.15.5:0/2266987308 wait complete. 2024-05-20T18:33:04.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.177+0000 7fd3d5e06640 1 Processor -- start 2024-05-20T18:33:04.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.178+0000 7fd3d5e06640 1 -- start start 2024-05-20T18:33:04.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.179+0000 7fd3d5e06640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3d006d3d0 0x7fd3d01f4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:04.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.179+0000 7fd3d5e06640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3d00b8190 0x7fd3d01e7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:04.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.179+0000 7fd3d5e06640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3d0162820 0x7fd3d00ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:04.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.179+0000 7fd3d5e06640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d00d0300 con 0x7fd3d00b8190 2024-05-20T18:33:04.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.179+0000 7fd3d5e06640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d00d42e0 con 0x7fd3d006d3d0 2024-05-20T18:33:04.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.179+0000 7fd3d5e06640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd3d00b4050 con 0x7fd3d0162820 2024-05-20T18:33:04.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.179+0000 7fd3c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3d00b8190 0x7fd3d01e7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:04.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.179+0000 7fd3c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3d00b8190 0x7fd3d01e7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44850/0 (socket says 172.21.15.5:44850) 2024-05-20T18:33:04.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.179+0000 7fd3c7fff640 1 -- 172.21.15.5:0/1588620559 learned_addr learned my addr 172.21.15.5:0/1588620559 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:04.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.179+0000 7fd3cffff640 1 --2- 172.21.15.5:0/1588620559 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3d0162820 0x7fd3d00ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:04.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.179+0000 7fd3cf7fe640 1 --2- 172.21.15.5:0/1588620559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3d006d3d0 0x7fd3d01f4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:04.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.179+0000 7fd3c7fff640 1 -- 172.21.15.5:0/1588620559 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3d0162820 msgr2=0x7fd3d00ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:04.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.179+0000 7fd3c7fff640 1 --2- 172.21.15.5:0/1588620559 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3d0162820 0x7fd3d00ca8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:04.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.179+0000 7fd3c7fff640 1 -- 172.21.15.5:0/1588620559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3d006d3d0 msgr2=0x7fd3d01f4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:04.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.179+0000 7fd3c7fff640 1 --2- 172.21.15.5:0/1588620559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3d006d3d0 0x7fd3d01f4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:04.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.179+0000 7fd3c7fff640 1 -- 172.21.15.5:0/1588620559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d00cbe00 con 0x7fd3d00b8190 2024-05-20T18:33:04.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.179+0000 7fd3cffff640 1 --2- 172.21.15.5:0/1588620559 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3d0162820 0x7fd3d00ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:04.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.179+0000 7fd3c7fff640 1 --2- 172.21.15.5:0/1588620559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3d00b8190 0x7fd3d01e7be0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8000c00 tx=0x7fd3b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:04.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.180+0000 7fd3ccff9640 1 -- 172.21.15.5:0/1588620559 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8037040 con 0x7fd3d00b8190 2024-05-20T18:33:04.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.180+0000 7fd3ccff9640 1 -- 172.21.15.5:0/1588620559 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b8042070 con 0x7fd3d00b8190 2024-05-20T18:33:04.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.180+0000 7fd3d5e06640 1 -- 172.21.15.5:0/1588620559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d00cc0e0 con 0x7fd3d00b8190 2024-05-20T18:33:04.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.180+0000 7fd3ccff9640 1 -- 172.21.15.5:0/1588620559 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8030a70 con 0x7fd3d00b8190 2024-05-20T18:33:04.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.180+0000 7fd3d5e06640 1 -- 172.21.15.5:0/1588620559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d00cc6a0 con 0x7fd3d00b8190 2024-05-20T18:33:04.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.181+0000 7fd3d5e06640 1 -- 172.21.15.5:0/1588620559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3d0075b00 con 0x7fd3d00b8190 2024-05-20T18:33:04.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.181+0000 7fd3ccff9640 1 -- 172.21.15.5:0/1588620559 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd3b8030c10 con 0x7fd3d00b8190 2024-05-20T18:33:04.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.182+0000 7fd3ccff9640 1 --2- 172.21.15.5:0/1588620559 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd3b006be60 0x7fd3b006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:04.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.182+0000 7fd3cf7fe640 1 --2- 172.21.15.5:0/1588620559 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd3b006be60 0x7fd3b006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:04.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.182+0000 7fd3ccff9640 1 -- 172.21.15.5:0/1588620559 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5421+0+0 (secure 0 0 0) 0x7fd3b80ab6f0 con 0x7fd3d00b8190 2024-05-20T18:33:04.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.183+0000 7fd3cf7fe640 1 --2- 172.21.15.5:0/1588620559 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd3b006be60 0x7fd3b006e370 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0000f60 tx=0x7fd3c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:04.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.187+0000 7fd3ccff9640 1 -- 172.21.15.5:0/1588620559 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd3b804c020 con 0x7fd3d00b8190 2024-05-20T18:33:04.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.371+0000 7fd3d5e06640 1 -- 172.21.15.5:0/1588620559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd3d0160530 con 0x7fd3d00b8190 2024-05-20T18:33:04.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.372+0000 7fd3ccff9640 1 -- 172.21.15.5:0/1588620559 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+13298 (secure 0 0 0) 0x7fd3b8045360 con 0x7fd3d00b8190 2024-05-20T18:33:04.373 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:33:04.373 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":81,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:33:03.035695+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6836/3866822752":"2024-05-21T18:32:59.810152+0000","172.21.15.5:6836/791176355":"2024-05-21T18:31:40.839373+0000","172.21.15.5:6835/3866822752":"2024-05-21T18:32:59.810152+0000","172.21.15.5:6835/791176355":"2024-05-21T18:31:40.839373+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:33:04.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.374+0000 7fd3d5e06640 1 -- 172.21.15.5:0/1588620559 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd3b006be60 msgr2=0x7fd3b006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:04.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.374+0000 7fd3d5e06640 1 --2- 172.21.15.5:0/1588620559 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd3b006be60 0x7fd3b006e370 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0000f60 tx=0x7fd3c0009040 comp rx=0 tx=0).stop 2024-05-20T18:33:04.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.374+0000 7fd3d5e06640 1 -- 172.21.15.5:0/1588620559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3d00b8190 msgr2=0x7fd3d01e7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:04.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.374+0000 7fd3d5e06640 1 --2- 172.21.15.5:0/1588620559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3d00b8190 0x7fd3d01e7be0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8000c00 tx=0x7fd3b80046c0 comp rx=0 tx=0).stop 2024-05-20T18:33:04.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.374+0000 7fd3d5e06640 1 -- 172.21.15.5:0/1588620559 shutdown_connections 2024-05-20T18:33:04.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.374+0000 7fd3d5e06640 1 --2- 172.21.15.5:0/1588620559 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd3b006be60 0x7fd3b006e370 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:04.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.374+0000 7fd3d5e06640 1 --2- 172.21.15.5:0/1588620559 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3d0162820 0x7fd3d00ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:04.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.374+0000 7fd3d5e06640 1 --2- 172.21.15.5:0/1588620559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3d00b8190 0x7fd3d01e7be0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:04.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.374+0000 7fd3d5e06640 1 --2- 172.21.15.5:0/1588620559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3d006d3d0 0x7fd3d01f4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:04.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.374+0000 7fd3d5e06640 1 -- 172.21.15.5:0/1588620559 >> 172.21.15.5:0/1588620559 conn(0x7fd3d00d4bd0 msgr2=0x7fd3d00ba610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:04.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.374+0000 7fd3d5e06640 1 -- 172.21.15.5:0/1588620559 shutdown_connections 2024-05-20T18:33:04.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.374+0000 7fd3d5e06640 1 -- 172.21.15.5:0/1588620559 wait complete. 2024-05-20T18:33:04.387 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:33:04.387 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:33:04.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.506+0000 7fa09b56b640 1 Processor -- start 2024-05-20T18:33:04.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.506+0000 7fa09b56b640 1 -- start start 2024-05-20T18:33:04.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.507+0000 7fa09b56b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa0940b4010 0x7fa0940d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:04.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.507+0000 7fa09b56b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa0940d14b0 0x7fa0940d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:04.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.507+0000 7fa09b56b640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0940d0370 con 0x7fa094163a10 2024-05-20T18:33:04.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.507+0000 7fa09b56b640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fa0940d3ca0 con 0x7fa0940b4010 2024-05-20T18:33:04.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.507+0000 7fa09b56b640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa0940d0650 con 0x7fa0940d14b0 2024-05-20T18:33:04.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.507+0000 7fa099307640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa094163a10 0x7fa0940cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46888/0 (socket says 172.21.15.5:46888) 2024-05-20T18:33:04.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.507+0000 7fa099307640 1 -- 172.21.15.5:0/3474686474 learned_addr learned my addr 172.21.15.5:0/3474686474 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:33:04.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.507+0000 7fa098b06640 1 --2- 172.21.15.5:0/3474686474 >> v2:172.21.15.5:3300/0 conn(0x7fa0940b4010 0x7fa0940d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:04.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.507+0000 7fa099b08640 1 --2- 172.21.15.5:0/3474686474 >> v2:172.21.15.63:3300/0 conn(0x7fa0940d14b0 0x7fa0940d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:04.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.508+0000 7fa093fff640 1 -- 172.21.15.5:0/3474686474 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2755191958 0 0) 0x7fa0940d0370 con 0x7fa094163a10 2024-05-20T18:33:04.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.508+0000 7fa093fff640 1 -- 172.21.15.5:0/3474686474 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa07c003520 con 0x7fa094163a10 2024-05-20T18:33:04.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.508+0000 7fa098b06640 1 -- 172.21.15.5:0/3474686474 >> v2:172.21.15.63:3300/0 conn(0x7fa0940d14b0 msgr2=0x7fa0940d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:04.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.508+0000 7fa098b06640 1 --2- 172.21.15.5:0/3474686474 >> v2:172.21.15.63:3300/0 conn(0x7fa0940d14b0 0x7fa0940d18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:04.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.508+0000 7fa098b06640 1 -- 172.21.15.5:0/3474686474 >> v1:172.21.15.5:6789/0 conn(0x7fa094163a10 legacy=0x7fa0940cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:04.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.508+0000 7fa098b06640 1 -- 172.21.15.5:0/3474686474 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0940d3650 con 0x7fa0940b4010 2024-05-20T18:33:04.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.508+0000 7fa098b06640 1 --2- 172.21.15.5:0/3474686474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0940b4010 0x7fa0940d4130 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fa0940b4590 tx=0x7fa08802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca2fbe4ae0464d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:04.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.509+0000 7fa093fff640 1 -- 172.21.15.5:0/3474686474 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa088037070 con 0x7fa0940b4010 2024-05-20T18:33:04.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.509+0000 7fa093fff640 1 -- 172.21.15.5:0/3474686474 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa088032440 con 0x7fa0940b4010 2024-05-20T18:33:04.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.509+0000 7fa093fff640 1 -- 172.21.15.5:0/3474686474 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa088032810 con 0x7fa0940b4010 2024-05-20T18:33:04.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.509+0000 7fa09b56b640 1 -- 172.21.15.5:0/3474686474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0940b4010 msgr2=0x7fa0940d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:04.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.509+0000 7fa09b56b640 1 --2- 172.21.15.5:0/3474686474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0940b4010 0x7fa0940d4130 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fa0940b4590 tx=0x7fa08802a1a0 comp rx=0 tx=0).stop 2024-05-20T18:33:04.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.509+0000 7fa09b56b640 1 -- 172.21.15.5:0/3474686474 shutdown_connections 2024-05-20T18:33:04.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.509+0000 7fa09b56b640 1 --2- 172.21.15.5:0/3474686474 >> v2:172.21.15.63:3300/0 conn(0x7fa0940d14b0 0x7fa0940d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:04.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.509+0000 7fa09b56b640 1 --2- 172.21.15.5:0/3474686474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0940b4010 0x7fa0940d4130 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:04.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.509+0000 7fa09b56b640 1 -- 172.21.15.5:0/3474686474 >> 172.21.15.5:0/3474686474 conn(0x7fa09406d240 msgr2=0x7fa09406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:04.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.509+0000 7fa09b56b640 1 -- 172.21.15.5:0/3474686474 shutdown_connections 2024-05-20T18:33:04.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.510+0000 7fa09b56b640 1 -- 172.21.15.5:0/3474686474 wait complete. 2024-05-20T18:33:04.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.511+0000 7fa09b56b640 1 Processor -- start 2024-05-20T18:33:04.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.511+0000 7fa09b56b640 1 -- start start 2024-05-20T18:33:04.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.513+0000 7fa09b56b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0940b4010 0x7fa0941f8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:04.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.513+0000 7fa09b56b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0940d14b0 0x7fa0941f9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:04.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.513+0000 7fa09b56b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa094163a10 0x7fa0941f4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:04.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.513+0000 7fa09b56b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa0940b8210 con 0x7fa0940d14b0 2024-05-20T18:33:04.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.513+0000 7fa09b56b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa0940d1e00 con 0x7fa0940b4010 2024-05-20T18:33:04.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.513+0000 7fa09b56b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa0940b8390 con 0x7fa094163a10 2024-05-20T18:33:04.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.513+0000 7fa098b06640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0940d14b0 0x7fa0941f9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:04.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.513+0000 7fa098b06640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0940d14b0 0x7fa0941f9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44866/0 (socket says 172.21.15.5:44866) 2024-05-20T18:33:04.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.513+0000 7fa098b06640 1 -- 172.21.15.5:0/2207848215 learned_addr learned my addr 172.21.15.5:0/2207848215 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:04.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.513+0000 7fa099b08640 1 --2- 172.21.15.5:0/2207848215 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa094163a10 0x7fa0941f4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:04.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.513+0000 7fa099307640 1 --2- 172.21.15.5:0/2207848215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0940b4010 0x7fa0941f8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:04.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.513+0000 7fa098b06640 1 -- 172.21.15.5:0/2207848215 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa094163a10 msgr2=0x7fa0941f4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:04.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.513+0000 7fa098b06640 1 --2- 172.21.15.5:0/2207848215 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa094163a10 0x7fa0941f4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:04.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.513+0000 7fa098b06640 1 -- 172.21.15.5:0/2207848215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0940b4010 msgr2=0x7fa0941f8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:04.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.513+0000 7fa098b06640 1 --2- 172.21.15.5:0/2207848215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0940b4010 0x7fa0941f8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:04.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.513+0000 7fa098b06640 1 -- 172.21.15.5:0/2207848215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa094147b50 con 0x7fa0940d14b0 2024-05-20T18:33:04.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.513+0000 7fa098b06640 1 --2- 172.21.15.5:0/2207848215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0940d14b0 0x7fa0941f9130 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fa088009890 tx=0x7fa0880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:04.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.513+0000 7fa091ffb640 1 -- 172.21.15.5:0/2207848215 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa088041020 con 0x7fa0940d14b0 2024-05-20T18:33:04.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.513+0000 7fa091ffb640 1 -- 172.21.15.5:0/2207848215 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa08803d700 con 0x7fa0940d14b0 2024-05-20T18:33:04.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.513+0000 7fa09b56b640 1 -- 172.21.15.5:0/2207848215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa094159160 con 0x7fa0940d14b0 2024-05-20T18:33:04.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.513+0000 7fa091ffb640 1 -- 172.21.15.5:0/2207848215 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa088037040 con 0x7fa0940d14b0 2024-05-20T18:33:04.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.514+0000 7fa09b56b640 1 -- 172.21.15.5:0/2207848215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa094159720 con 0x7fa0940d14b0 2024-05-20T18:33:04.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.515+0000 7fa09b56b640 1 -- 172.21.15.5:0/2207848215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa094073910 con 0x7fa0940d14b0 2024-05-20T18:33:04.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.515+0000 7fa091ffb640 1 -- 172.21.15.5:0/2207848215 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa088044020 con 0x7fa0940d14b0 2024-05-20T18:33:04.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.516+0000 7fa091ffb640 1 --2- 172.21.15.5:0/2207848215 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa08006be60 0x7fa08006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:04.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.516+0000 7fa091ffb640 1 -- 172.21.15.5:0/2207848215 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5421+0+0 (secure 0 0 0) 0x7fa0880ab230 con 0x7fa0940d14b0 2024-05-20T18:33:04.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.516+0000 7fa099307640 1 --2- 172.21.15.5:0/2207848215 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa08006be60 0x7fa08006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:04.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.516+0000 7fa099307640 1 --2- 172.21.15.5:0/2207848215 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa08006be60 0x7fa08006e370 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa08c005880 tx=0x7fa08c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:04.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.520+0000 7fa091ffb640 1 -- 172.21.15.5:0/2207848215 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa094073910 con 0x7fa0940d14b0 2024-05-20T18:33:04.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.703+0000 7fa09b56b640 1 -- 172.21.15.5:0/2207848215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa094075b00 con 0x7fa0940d14b0 2024-05-20T18:33:04.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.704+0000 7fa091ffb640 1 -- 172.21.15.5:0/2207848215 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7fa08804b070 con 0x7fa0940d14b0 2024-05-20T18:33:04.704 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:33:04.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.706+0000 7fa09b56b640 1 -- 172.21.15.5:0/2207848215 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa08006be60 msgr2=0x7fa08006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:04.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.706+0000 7fa09b56b640 1 --2- 172.21.15.5:0/2207848215 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa08006be60 0x7fa08006e370 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa08c005880 tx=0x7fa08c009040 comp rx=0 tx=0).stop 2024-05-20T18:33:04.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.706+0000 7fa09b56b640 1 -- 172.21.15.5:0/2207848215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0940d14b0 msgr2=0x7fa0941f9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:04.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.706+0000 7fa09b56b640 1 --2- 172.21.15.5:0/2207848215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0940d14b0 0x7fa0941f9130 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fa088009890 tx=0x7fa0880046c0 comp rx=0 tx=0).stop 2024-05-20T18:33:04.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.706+0000 7fa09b56b640 1 -- 172.21.15.5:0/2207848215 shutdown_connections 2024-05-20T18:33:04.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.706+0000 7fa09b56b640 1 --2- 172.21.15.5:0/2207848215 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa08006be60 0x7fa08006e370 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:04.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.706+0000 7fa09b56b640 1 --2- 172.21.15.5:0/2207848215 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa094163a10 0x7fa0941f4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:04.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.706+0000 7fa09b56b640 1 --2- 172.21.15.5:0/2207848215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0940d14b0 0x7fa0941f9130 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:04.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.706+0000 7fa09b56b640 1 --2- 172.21.15.5:0/2207848215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0940b4010 0x7fa0941f8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:04.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.706+0000 7fa09b56b640 1 -- 172.21.15.5:0/2207848215 >> 172.21.15.5:0/2207848215 conn(0x7fa09406d240 msgr2=0x7fa0940b4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:04.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.706+0000 7fa09b56b640 1 -- 172.21.15.5:0/2207848215 shutdown_connections 2024-05-20T18:33:04.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.706+0000 7fa09b56b640 1 -- 172.21.15.5:0/2207848215 wait complete. 2024-05-20T18:33:04.719 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-05-20T18:33:04.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.835+0000 7f91ac040640 1 Processor -- start 2024-05-20T18:33:04.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.835+0000 7f91ac040640 1 -- start start 2024-05-20T18:33:04.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.836+0000 7f91ac040640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91a4071630 0x7f91a415f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:04.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.836+0000 7f91ac040640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f91a4070850 0x7f91a4070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:04.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.836+0000 7f91ac040640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91a40711a0 con 0x7f91a40cd1b0 2024-05-20T18:33:04.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.836+0000 7f91ac040640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f91a415f790 con 0x7f91a4070850 2024-05-20T18:33:04.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.836+0000 7f91ac040640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f91a406e590 con 0x7f91a4071630 2024-05-20T18:33:04.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.836+0000 7f91a95db640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f91a40cd1b0 0x7f91a406d7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38152/0 (socket says 172.21.15.5:38152) 2024-05-20T18:33:04.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.836+0000 7f91a95db640 1 -- 172.21.15.5:0/2558592920 learned_addr learned my addr 172.21.15.5:0/2558592920 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:33:04.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.837+0000 7f91aa5dd640 1 --2- 172.21.15.5:0/2558592920 >> v2:172.21.15.63:3300/0 conn(0x7f91a4070850 0x7f91a4070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:04.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.837+0000 7f91a9ddc640 1 --2- 172.21.15.5:0/2558592920 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91a4071630 0x7f91a415f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:04.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.837+0000 7f91a8dda640 1 -- 172.21.15.5:0/2558592920 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2087672210 0 0) 0x7f9198001110 con 0x7f91a40cd1b0 2024-05-20T18:33:04.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.837+0000 7f91a8dda640 1 -- 172.21.15.5:0/2558592920 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f918c003520 con 0x7f91a40cd1b0 2024-05-20T18:33:04.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.837+0000 7f91a9ddc640 1 -- 172.21.15.5:0/2558592920 >> v2:172.21.15.63:3300/0 conn(0x7f91a4070850 msgr2=0x7f91a4070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:04.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.837+0000 7f91a8dda640 1 -- 172.21.15.5:0/2558592920 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1391265116 0 0) 0x7f918c003520 con 0x7f91a40cd1b0 2024-05-20T18:33:04.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.837+0000 7f91a9ddc640 1 --2- 172.21.15.5:0/2558592920 >> v2:172.21.15.63:3300/0 conn(0x7f91a4070850 0x7f91a4070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:04.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.837+0000 7f91a9ddc640 1 -- 172.21.15.5:0/2558592920 >> v1:172.21.15.5:6789/0 conn(0x7f91a40cd1b0 legacy=0x7f91a406d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:04.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.837+0000 7f91a9ddc640 1 -- 172.21.15.5:0/2558592920 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a40ce2b0 con 0x7f91a4071630 2024-05-20T18:33:04.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.837+0000 7f91aa5dd640 1 --2- 172.21.15.5:0/2558592920 >> v2:172.21.15.63:3300/0 conn(0x7f91a4070850 0x7f91a4070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:33:04.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.838+0000 7f91a9ddc640 1 --2- 172.21.15.5:0/2558592920 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91a4071630 0x7f91a415f250 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f919c0097a0 tx=0x7f919c02a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ef663c48849148c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:04.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.838+0000 7f91a8dda640 1 -- 172.21.15.5:0/2558592920 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c037070 con 0x7f91a4071630 2024-05-20T18:33:04.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.838+0000 7f91a8dda640 1 -- 172.21.15.5:0/2558592920 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f919c02ab10 con 0x7f91a4071630 2024-05-20T18:33:04.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.838+0000 7f91a8dda640 1 -- 172.21.15.5:0/2558592920 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9198001840 con 0x7f91a4071630 2024-05-20T18:33:04.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.839+0000 7f91ac040640 1 -- 172.21.15.5:0/2558592920 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91a4071630 msgr2=0x7f91a415f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:04.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.839+0000 7f91ac040640 1 --2- 172.21.15.5:0/2558592920 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91a4071630 0x7f91a415f250 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f919c0097a0 tx=0x7f919c02a080 comp rx=0 tx=0).stop 2024-05-20T18:33:04.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.839+0000 7f91ac040640 1 -- 172.21.15.5:0/2558592920 shutdown_connections 2024-05-20T18:33:04.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.839+0000 7f91ac040640 1 --2- 172.21.15.5:0/2558592920 >> v2:172.21.15.63:3300/0 conn(0x7f91a4070850 0x7f91a4070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:04.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.839+0000 7f91ac040640 1 --2- 172.21.15.5:0/2558592920 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91a4071630 0x7f91a415f250 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:04.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.839+0000 7f91ac040640 1 -- 172.21.15.5:0/2558592920 >> 172.21.15.5:0/2558592920 conn(0x7f91a40d29d0 msgr2=0x7f91a40d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:04.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.839+0000 7f91ac040640 1 -- 172.21.15.5:0/2558592920 shutdown_connections 2024-05-20T18:33:04.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.839+0000 7f91ac040640 1 -- 172.21.15.5:0/2558592920 wait complete. 2024-05-20T18:33:04.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.841+0000 7f91ac040640 1 Processor -- start 2024-05-20T18:33:04.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.841+0000 7f91ac040640 1 -- start start 2024-05-20T18:33:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.842+0000 7f91ac040640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91a4070850 0x7f91a40cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.842+0000 7f91ac040640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91a4071630 0x7f91a4148610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.842+0000 7f91ac040640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91a40cd1b0 0x7f91a4148b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.842+0000 7f91ac040640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f91a406ded0 con 0x7f91a4070850 2024-05-20T18:33:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.842+0000 7f91ac040640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f91a406cfc0 con 0x7f91a40cd1b0 2024-05-20T18:33:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.842+0000 7f91ac040640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f91a406d2a0 con 0x7f91a4071630 2024-05-20T18:33:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.842+0000 7f91a9ddc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91a4070850 0x7f91a40cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.842+0000 7f91a9ddc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91a4070850 0x7f91a40cbf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56494/0 (socket says 172.21.15.5:56494) 2024-05-20T18:33:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.842+0000 7f91a9ddc640 1 -- 172.21.15.5:0/3019289019 learned_addr learned my addr 172.21.15.5:0/3019289019 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.842+0000 7f91a95db640 1 --2- 172.21.15.5:0/3019289019 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91a4071630 0x7f91a4148610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.842+0000 7f91aa5dd640 1 --2- 172.21.15.5:0/3019289019 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91a40cd1b0 0x7f91a4148b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.843+0000 7f91a9ddc640 1 -- 172.21.15.5:0/3019289019 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91a4071630 msgr2=0x7f91a4148610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.843+0000 7f91a9ddc640 1 --2- 172.21.15.5:0/3019289019 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91a4071630 0x7f91a4148610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.843+0000 7f91a9ddc640 1 -- 172.21.15.5:0/3019289019 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91a40cd1b0 msgr2=0x7f91a4148b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.843+0000 7f91a9ddc640 1 --2- 172.21.15.5:0/3019289019 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91a40cd1b0 0x7f91a4148b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.843+0000 7f91a9ddc640 1 -- 172.21.15.5:0/3019289019 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a415f910 con 0x7f91a4070850 2024-05-20T18:33:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.843+0000 7f91a95db640 1 --2- 172.21.15.5:0/3019289019 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91a4071630 0x7f91a4148610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.843+0000 7f91aa5dd640 1 --2- 172.21.15.5:0/3019289019 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91a40cd1b0 0x7f91a4148b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.843+0000 7f91a9ddc640 1 --2- 172.21.15.5:0/3019289019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91a4070850 0x7f91a40cbf90 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f919c035040 tx=0x7f919c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.843+0000 7f91a27fc640 1 -- 172.21.15.5:0/3019289019 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c042020 con 0x7f91a4070850 2024-05-20T18:33:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.843+0000 7f91a27fc640 1 -- 172.21.15.5:0/3019289019 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f919c03d670 con 0x7f91a4070850 2024-05-20T18:33:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.843+0000 7f91a27fc640 1 -- 172.21.15.5:0/3019289019 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c037040 con 0x7f91a4070850 2024-05-20T18:33:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.843+0000 7f91ac040640 1 -- 172.21.15.5:0/3019289019 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a41466f0 con 0x7f91a4070850 2024-05-20T18:33:04.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.843+0000 7f91ac040640 1 -- 172.21.15.5:0/3019289019 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a4146c70 con 0x7f91a4070850 2024-05-20T18:33:04.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.844+0000 7f91ac040640 1 -- 172.21.15.5:0/3019289019 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91a4075b00 con 0x7f91a4070850 2024-05-20T18:33:04.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.845+0000 7f91a27fc640 1 -- 172.21.15.5:0/3019289019 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f919c045020 con 0x7f91a4070850 2024-05-20T18:33:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.845+0000 7f91a27fc640 1 --2- 172.21.15.5:0/3019289019 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f919006be30 0x7f919006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.845+0000 7f91a27fc640 1 -- 172.21.15.5:0/3019289019 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5421+0+0 (secure 0 0 0) 0x7f919c0ab730 con 0x7f91a4070850 2024-05-20T18:33:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.845+0000 7f91a95db640 1 --2- 172.21.15.5:0/3019289019 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f919006be30 0x7f919006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.846+0000 7f91a95db640 1 --2- 172.21.15.5:0/3019289019 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f919006be30 0x7f919006e340 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f91a41f8dc0 tx=0x7f9198009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:04.850+0000 7f91a27fc640 1 -- 172.21.15.5:0/3019289019 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f919c04c030 con 0x7f91a4070850 2024-05-20T18:33:05.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:05.105+0000 7f91ac040640 1 -- 172.21.15.5:0/3019289019 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7f91a4071410 con 0x7f91a4070850 2024-05-20T18:33:06.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.049+0000 7f91a27fc640 1 -- 172.21.15.5:0/3019289019 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v302) ==== 170+0+0 (secure 0 0 0) 0x7f919c07a0f0 con 0x7f91a4070850 2024-05-20T18:33:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.052+0000 7f91ac040640 1 -- 172.21.15.5:0/3019289019 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f919006be30 msgr2=0x7f919006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.052+0000 7f91ac040640 1 --2- 172.21.15.5:0/3019289019 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f919006be30 0x7f919006e340 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f91a41f8dc0 tx=0x7f9198009040 comp rx=0 tx=0).stop 2024-05-20T18:33:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.052+0000 7f91ac040640 1 -- 172.21.15.5:0/3019289019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91a4070850 msgr2=0x7f91a40cbf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:06.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.052+0000 7f91ac040640 1 --2- 172.21.15.5:0/3019289019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91a4070850 0x7f91a40cbf90 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f919c035040 tx=0x7f919c0046c0 comp rx=0 tx=0).stop 2024-05-20T18:33:06.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.053+0000 7f91ac040640 1 -- 172.21.15.5:0/3019289019 shutdown_connections 2024-05-20T18:33:06.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.053+0000 7f91ac040640 1 --2- 172.21.15.5:0/3019289019 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f919006be30 0x7f919006e340 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:06.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.053+0000 7f91ac040640 1 --2- 172.21.15.5:0/3019289019 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91a40cd1b0 0x7f91a4148b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:06.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.053+0000 7f91ac040640 1 --2- 172.21.15.5:0/3019289019 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91a4071630 0x7f91a4148610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:06.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.053+0000 7f91ac040640 1 --2- 172.21.15.5:0/3019289019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91a4070850 0x7f91a40cbf90 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:06.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.053+0000 7f91ac040640 1 -- 172.21.15.5:0/3019289019 >> 172.21.15.5:0/3019289019 conn(0x7f91a40d29d0 msgr2=0x7f91a40b7370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:06.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.053+0000 7f91ac040640 1 -- 172.21.15.5:0/3019289019 shutdown_connections 2024-05-20T18:33:06.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.053+0000 7f91ac040640 1 -- 172.21.15.5:0/3019289019 wait complete. 2024-05-20T18:33:06.067 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-20T18:33:06.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.192+0000 7f6567680640 1 Processor -- start 2024-05-20T18:33:06.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.192+0000 7f6567680640 1 -- start start 2024-05-20T18:33:06.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.194+0000 7f6567680640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6560162820 0x7f65600b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:06.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.194+0000 7f6567680640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f656006dae0 0x7f65600b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:06.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.194+0000 7f6567680640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65600d0630 con 0x7f65600b3e80 2024-05-20T18:33:06.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.194+0000 7f6567680640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f65600d39c0 con 0x7f6560162820 2024-05-20T18:33:06.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.194+0000 7f6567680640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f65600d3ca0 con 0x7f656006dae0 2024-05-20T18:33:06.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.194+0000 7f656541c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6560162820 0x7f65600b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:06.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.194+0000 7f656541c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6560162820 0x7f65600b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56498/0 (socket says 172.21.15.5:56498) 2024-05-20T18:33:06.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.194+0000 7f656541c640 1 -- 172.21.15.5:0/91732896 learned_addr learned my addr 172.21.15.5:0/91732896 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:06.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.194+0000 7f6565c1d640 1 --2- 172.21.15.5:0/91732896 >> v2:172.21.15.63:3300/0 conn(0x7f656006dae0 0x7f65600b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:06.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.194+0000 7f656541c640 1 -- 172.21.15.5:0/91732896 >> v2:172.21.15.63:3300/0 conn(0x7f656006dae0 msgr2=0x7f65600b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:06.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.194+0000 7f656541c640 1 --2- 172.21.15.5:0/91732896 >> v2:172.21.15.63:3300/0 conn(0x7f656006dae0 0x7f65600b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:06.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.194+0000 7f656541c640 1 -- 172.21.15.5:0/91732896 >> v1:172.21.15.63:6789/0 conn(0x7f65600b3e80 legacy=0x7f656006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:06.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.194+0000 7f656541c640 1 -- 172.21.15.5:0/91732896 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65600d3f80 con 0x7f6560162820 2024-05-20T18:33:06.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.195+0000 7f656541c640 1 --2- 172.21.15.5:0/91732896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6560162820 0x7f65600b3940 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f656008e070 tx=0x7f655c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ba97974b82458d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:06.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.195+0000 7f6557fff640 1 -- 172.21.15.5:0/91732896 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c037070 con 0x7f6560162820 2024-05-20T18:33:06.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.195+0000 7f6557fff640 1 -- 172.21.15.5:0/91732896 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f655c032440 con 0x7f6560162820 2024-05-20T18:33:06.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.195+0000 7f6557fff640 1 -- 172.21.15.5:0/91732896 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c032810 con 0x7f6560162820 2024-05-20T18:33:06.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.196+0000 7f6567680640 1 -- 172.21.15.5:0/91732896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6560162820 msgr2=0x7f65600b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:06.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.196+0000 7f6567680640 1 --2- 172.21.15.5:0/91732896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6560162820 0x7f65600b3940 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f656008e070 tx=0x7f655c02a270 comp rx=0 tx=0).stop 2024-05-20T18:33:06.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.196+0000 7f6567680640 1 -- 172.21.15.5:0/91732896 shutdown_connections 2024-05-20T18:33:06.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.196+0000 7f6567680640 1 --2- 172.21.15.5:0/91732896 >> v2:172.21.15.63:3300/0 conn(0x7f656006dae0 0x7f65600b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:06.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.196+0000 7f6567680640 1 --2- 172.21.15.5:0/91732896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6560162820 0x7f65600b3940 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:06.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.196+0000 7f6567680640 1 -- 172.21.15.5:0/91732896 >> 172.21.15.5:0/91732896 conn(0x7f65600d4bd0 msgr2=0x7f65600d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:06.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.196+0000 7f6567680640 1 -- 172.21.15.5:0/91732896 shutdown_connections 2024-05-20T18:33:06.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.196+0000 7f6567680640 1 -- 172.21.15.5:0/91732896 wait complete. 2024-05-20T18:33:06.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.198+0000 7f6567680640 1 Processor -- start 2024-05-20T18:33:06.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.198+0000 7f6567680640 1 -- start start 2024-05-20T18:33:06.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.199+0000 7f6567680640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65600b3e80 0x7f656014d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:06.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.199+0000 7f6567680640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6560162820 0x7f65601f8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:06.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.199+0000 7f6567680640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65601f87b0 0x7f65600c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:06.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.199+0000 7f6567680640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f65600d4780 con 0x7f65601f87b0 2024-05-20T18:33:06.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.199+0000 7f6567680640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f65600d0300 con 0x7f6560162820 2024-05-20T18:33:06.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.199+0000 7f6567680640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f65600d4900 con 0x7f65600b3e80 2024-05-20T18:33:06.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.200+0000 7f6565c1d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65601f87b0 0x7f65600c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:06.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.200+0000 7f6565c1d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65601f87b0 0x7f65600c9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56500/0 (socket says 172.21.15.5:56500) 2024-05-20T18:33:06.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.200+0000 7f6565c1d640 1 -- 172.21.15.5:0/3564687281 learned_addr learned my addr 172.21.15.5:0/3564687281 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:06.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.200+0000 7f656541c640 1 --2- 172.21.15.5:0/3564687281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65600b3e80 0x7f656014d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:06.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.200+0000 7f6564c1b640 1 --2- 172.21.15.5:0/3564687281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6560162820 0x7f65601f8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:06.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.200+0000 7f6565c1d640 1 -- 172.21.15.5:0/3564687281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65600b3e80 msgr2=0x7f656014d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:06.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.200+0000 7f6565c1d640 1 --2- 172.21.15.5:0/3564687281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65600b3e80 0x7f656014d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:06.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.200+0000 7f6565c1d640 1 -- 172.21.15.5:0/3564687281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6560162820 msgr2=0x7f65601f8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:06.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.200+0000 7f6565c1d640 1 --2- 172.21.15.5:0/3564687281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6560162820 0x7f65601f8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:06.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.200+0000 7f6565c1d640 1 -- 172.21.15.5:0/3564687281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65601f8bc0 con 0x7f65601f87b0 2024-05-20T18:33:06.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.200+0000 7f6565c1d640 1 --2- 172.21.15.5:0/3564687281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65601f87b0 0x7f65600c9660 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f655800efc0 tx=0x7f655800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:06.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.200+0000 7f6555ffb640 1 -- 172.21.15.5:0/3564687281 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6558018020 con 0x7f65601f87b0 2024-05-20T18:33:06.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.200+0000 7f6555ffb640 1 -- 172.21.15.5:0/3564687281 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6558014480 con 0x7f65601f87b0 2024-05-20T18:33:06.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.200+0000 7f6567680640 1 -- 172.21.15.5:0/3564687281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65600c88a0 con 0x7f65601f87b0 2024-05-20T18:33:06.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.200+0000 7f6555ffb640 1 -- 172.21.15.5:0/3564687281 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6558014870 con 0x7f65601f87b0 2024-05-20T18:33:06.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.200+0000 7f6567680640 1 -- 172.21.15.5:0/3564687281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65600c8e60 con 0x7f65601f87b0 2024-05-20T18:33:06.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.201+0000 7f6567680640 1 -- 172.21.15.5:0/3564687281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6560075b00 con 0x7f65601f87b0 2024-05-20T18:33:06.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.202+0000 7f6555ffb640 1 -- 172.21.15.5:0/3564687281 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6558005d10 con 0x7f65601f87b0 2024-05-20T18:33:06.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.202+0000 7f6555ffb640 1 --2- 172.21.15.5:0/3564687281 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f654406be30 0x7f654406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:06.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.203+0000 7f6555ffb640 1 -- 172.21.15.5:0/3564687281 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5421+0+0 (secure 0 0 0) 0x7f655808ba00 con 0x7f65601f87b0 2024-05-20T18:33:06.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.203+0000 7f656541c640 1 --2- 172.21.15.5:0/3564687281 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f654406be30 0x7f654406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:06.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.203+0000 7f656541c640 1 --2- 172.21.15.5:0/3564687281 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f654406be30 0x7f654406e340 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f655c034000 tx=0x7f655c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:06.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.207+0000 7f6555ffb640 1 -- 172.21.15.5:0/3564687281 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6558022030 con 0x7f65601f87b0 2024-05-20T18:33:06.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.390+0000 7f6567680640 1 -- 172.21.15.5:0/3564687281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f65600b6b60 con 0x7f65601f87b0 2024-05-20T18:33:06.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.393+0000 7f6555ffb640 1 -- 172.21.15.5:0/3564687281 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f65601f8bc0 con 0x7f65601f87b0 2024-05-20T18:33:06.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.396+0000 7f6567680640 1 -- 172.21.15.5:0/3564687281 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f654406be30 msgr2=0x7f654406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:06.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.396+0000 7f6567680640 1 --2- 172.21.15.5:0/3564687281 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f654406be30 0x7f654406e340 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f655c034000 tx=0x7f655c02e000 comp rx=0 tx=0).stop 2024-05-20T18:33:06.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.396+0000 7f6567680640 1 -- 172.21.15.5:0/3564687281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65601f87b0 msgr2=0x7f65600c9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:06.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.396+0000 7f6567680640 1 --2- 172.21.15.5:0/3564687281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65601f87b0 0x7f65600c9660 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f655800efc0 tx=0x7f655800c490 comp rx=0 tx=0).stop 2024-05-20T18:33:06.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.396+0000 7f6567680640 1 -- 172.21.15.5:0/3564687281 shutdown_connections 2024-05-20T18:33:06.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.396+0000 7f6567680640 1 --2- 172.21.15.5:0/3564687281 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f654406be30 0x7f654406e340 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:06.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.396+0000 7f6567680640 1 --2- 172.21.15.5:0/3564687281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65601f87b0 0x7f65600c9660 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:06.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.396+0000 7f6567680640 1 --2- 172.21.15.5:0/3564687281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6560162820 0x7f65601f8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:06.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.396+0000 7f6567680640 1 --2- 172.21.15.5:0/3564687281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65600b3e80 0x7f656014d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:06.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.396+0000 7f6567680640 1 -- 172.21.15.5:0/3564687281 >> 172.21.15.5:0/3564687281 conn(0x7f65600d4bd0 msgr2=0x7f656015e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:06.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.396+0000 7f6567680640 1 -- 172.21.15.5:0/3564687281 shutdown_connections 2024-05-20T18:33:06.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.396+0000 7f6567680640 1 -- 172.21.15.5:0/3564687281 wait complete. 2024-05-20T18:33:06.408 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-20T18:33:06.408 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T18:33:06.455 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T18:33:06.513 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-20T18:33:06.513 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:33:06.513 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:33:06.529 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15663 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T18:33:06.529 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:33:06.529 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:33:06.548 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19732 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T18:33:06.548 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:33:06.548 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:33:06.586 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15663 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T18:33:06.586 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:33:06.586 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:33:06.607 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19732 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T18:33:06.608 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-20T18:33:06.608 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:33:06.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.741+0000 7f051183f640 1 Processor -- start 2024-05-20T18:33:06.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.741+0000 7f051183f640 1 -- start start 2024-05-20T18:33:06.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.742+0000 7f051183f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f050c0b1e10 0x7f050c15aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:06.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.742+0000 7f051183f640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f050c070850 0x7f050c070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:06.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.742+0000 7f051183f640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f050c0cd580 con 0x7f050c15d530 2024-05-20T18:33:06.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.742+0000 7f051183f640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f050c0ce190 con 0x7f050c0b1e10 2024-05-20T18:33:06.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.742+0000 7f051183f640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f050c0cd860 con 0x7f050c070850 2024-05-20T18:33:06.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.742+0000 7f0503fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f050c0b1e10 0x7f050c15aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:06.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.742+0000 7f050affd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f050c15d530 0x7f050c0b1700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38164/0 (socket says 172.21.15.5:38164) 2024-05-20T18:33:06.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.742+0000 7f050affd640 1 -- 172.21.15.5:0/3343607328 learned_addr learned my addr 172.21.15.5:0/3343607328 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:33:06.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.743+0000 7f050b7fe640 1 --2- 172.21.15.5:0/3343607328 >> v2:172.21.15.63:3300/0 conn(0x7f050c070850 0x7f050c070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:06.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.743+0000 7f0503fff640 1 -- 172.21.15.5:0/3343607328 >> v2:172.21.15.63:3300/0 conn(0x7f050c070850 msgr2=0x7f050c070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.743+0000 7f0503fff640 1 --2- 172.21.15.5:0/3343607328 >> v2:172.21.15.63:3300/0 conn(0x7f050c070850 0x7f050c070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.743+0000 7f0503fff640 1 -- 172.21.15.5:0/3343607328 >> v1:172.21.15.5:6789/0 conn(0x7f050c15d530 legacy=0x7f050c0b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.743+0000 7f0503fff640 1 -- 172.21.15.5:0/3343607328 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f050c0cdb40 con 0x7f050c0b1e10 2024-05-20T18:33:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.743+0000 7f050a7fc640 1 -- 172.21.15.5:0/3343607328 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 660898243 0 0) 0x7f050c0cd580 con 0x7f050c15d530 2024-05-20T18:33:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.743+0000 7f050b7fe640 1 --2- 172.21.15.5:0/3343607328 >> v2:172.21.15.63:3300/0 conn(0x7f050c070850 0x7f050c070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:33:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.744+0000 7f0503fff640 1 --2- 172.21.15.5:0/3343607328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f050c0b1e10 0x7f050c15aaa0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f050c0b8400 tx=0x7f04f402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df1977b485c69d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.744+0000 7f050a7fc640 1 -- 172.21.15.5:0/3343607328 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f4037070 con 0x7f050c0b1e10 2024-05-20T18:33:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.744+0000 7f050a7fc640 1 -- 172.21.15.5:0/3343607328 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f4032440 con 0x7f050c0b1e10 2024-05-20T18:33:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.744+0000 7f050a7fc640 1 -- 172.21.15.5:0/3343607328 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f4032810 con 0x7f050c0b1e10 2024-05-20T18:33:06.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.744+0000 7f051183f640 1 -- 172.21.15.5:0/3343607328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f050c0b1e10 msgr2=0x7f050c15aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:06.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.744+0000 7f051183f640 1 --2- 172.21.15.5:0/3343607328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f050c0b1e10 0x7f050c15aaa0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f050c0b8400 tx=0x7f04f402a1a0 comp rx=0 tx=0).stop 2024-05-20T18:33:06.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.745+0000 7f051183f640 1 -- 172.21.15.5:0/3343607328 shutdown_connections 2024-05-20T18:33:06.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.745+0000 7f051183f640 1 --2- 172.21.15.5:0/3343607328 >> v2:172.21.15.63:3300/0 conn(0x7f050c070850 0x7f050c070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:06.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.745+0000 7f051183f640 1 --2- 172.21.15.5:0/3343607328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f050c0b1e10 0x7f050c15aaa0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:06.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.745+0000 7f051183f640 1 -- 172.21.15.5:0/3343607328 >> 172.21.15.5:0/3343607328 conn(0x7f050c0d29d0 msgr2=0x7f050c0d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:06.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.745+0000 7f051183f640 1 -- 172.21.15.5:0/3343607328 shutdown_connections 2024-05-20T18:33:06.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.745+0000 7f051183f640 1 -- 172.21.15.5:0/3343607328 wait complete. 2024-05-20T18:33:06.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.746+0000 7f051183f640 1 Processor -- start 2024-05-20T18:33:06.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.747+0000 7f051183f640 1 -- start start 2024-05-20T18:33:06.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.748+0000 7f051183f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f050c070850 0x7f050c1f3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:06.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.748+0000 7f051183f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f050c0b1e10 0x7f050c0c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:06.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.748+0000 7f051183f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f050c15d530 0x7f050c0d1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:06.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.748+0000 7f051183f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f050c0711a0 con 0x7f050c0b1e10 2024-05-20T18:33:06.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.748+0000 7f051183f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f050c0cd230 con 0x7f050c15d530 2024-05-20T18:33:06.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.748+0000 7f051183f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f050c071320 con 0x7f050c070850 2024-05-20T18:33:06.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.748+0000 7f0503fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f050c0b1e10 0x7f050c0c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:06.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.748+0000 7f0503fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f050c0b1e10 0x7f050c0c8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56510/0 (socket says 172.21.15.5:56510) 2024-05-20T18:33:06.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.748+0000 7f0503fff640 1 -- 172.21.15.5:0/2887705918 learned_addr learned my addr 172.21.15.5:0/2887705918 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:06.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.748+0000 7f0503fff640 1 -- 172.21.15.5:0/2887705918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f050c070850 msgr2=0x7f050c1f3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:06.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.748+0000 7f0503fff640 1 --2- 172.21.15.5:0/2887705918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f050c070850 0x7f050c1f3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:06.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.748+0000 7f0503fff640 1 -- 172.21.15.5:0/2887705918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f050c15d530 msgr2=0x7f050c0d1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:06.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.748+0000 7f0503fff640 1 --2- 172.21.15.5:0/2887705918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f050c15d530 0x7f050c0d1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:06.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.748+0000 7f0503fff640 1 -- 172.21.15.5:0/2887705918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f050c1f4470 con 0x7f050c0b1e10 2024-05-20T18:33:06.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.748+0000 7f0503fff640 1 --2- 172.21.15.5:0/2887705918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f050c0b1e10 0x7f050c0c8990 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f04f4000c00 tx=0x7f04f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:06.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.749+0000 7f0508ff9640 1 -- 172.21.15.5:0/2887705918 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f4037040 con 0x7f050c0b1e10 2024-05-20T18:33:06.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.749+0000 7f0508ff9640 1 -- 172.21.15.5:0/2887705918 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f4042070 con 0x7f050c0b1e10 2024-05-20T18:33:06.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.749+0000 7f0508ff9640 1 -- 172.21.15.5:0/2887705918 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f4030bd0 con 0x7f050c0b1e10 2024-05-20T18:33:06.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.749+0000 7f051183f640 1 -- 172.21.15.5:0/2887705918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f050c0c66a0 con 0x7f050c0b1e10 2024-05-20T18:33:06.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.749+0000 7f051183f640 1 -- 172.21.15.5:0/2887705918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f050c0c6c60 con 0x7f050c0b1e10 2024-05-20T18:33:06.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.750+0000 7f051183f640 1 -- 172.21.15.5:0/2887705918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f050c075b00 con 0x7f050c0b1e10 2024-05-20T18:33:06.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.751+0000 7f0508ff9640 1 -- 172.21.15.5:0/2887705918 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f04f4047460 con 0x7f050c0b1e10 2024-05-20T18:33:06.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.751+0000 7f0508ff9640 1 --2- 172.21.15.5:0/2887705918 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f04ec06be60 0x7f04ec06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:06.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.751+0000 7f050affd640 1 --2- 172.21.15.5:0/2887705918 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f04ec06be60 0x7f04ec06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:06.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.751+0000 7f0508ff9640 1 -- 172.21.15.5:0/2887705918 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5421+0+0 (secure 0 0 0) 0x7f04f40ab9a0 con 0x7f050c0b1e10 2024-05-20T18:33:06.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.752+0000 7f050affd640 1 --2- 172.21.15.5:0/2887705918 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f04ec06be60 0x7f04ec06e370 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f04fc007fd0 tx=0x7f04fc007960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:06.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.756+0000 7f0508ff9640 1 -- 172.21.15.5:0/2887705918 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f04f404c020 con 0x7f050c0b1e10 2024-05-20T18:33:06.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.977+0000 7f051183f640 1 -- 172.21.15.5:0/2887705918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f050c15fcd0 con 0x7f050c0b1e10 2024-05-20T18:33:06.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.977+0000 7f0508ff9640 1 -- 172.21.15.5:0/2887705918 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+3860 (secure 0 0 0) 0x7f04f4032e20 con 0x7f050c0b1e10 2024-05-20T18:33:06.978 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:33:06.978 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":54,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/1890759603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":1890759603},{"type":"v1","addr":"172.21.15.63:6834","nonce":1890759603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5825,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/700447403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":700447403},{"type":"v1","addr":"172.21.15.63:6835","nonce":700447403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3382943275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3382943275},{"type":"v1","addr":"172.21.15.5:6837","nonce":3382943275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":7350,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/1702305188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":1702305188},{"type":"v1","addr":"172.21.15.5:6836","nonce":1702305188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[]} 2024-05-20T18:33:06.978 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 54 2024-05-20T18:33:06.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.979+0000 7f051183f640 1 -- 172.21.15.5:0/2887705918 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f04ec06be60 msgr2=0x7f04ec06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:06.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.979+0000 7f051183f640 1 --2- 172.21.15.5:0/2887705918 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f04ec06be60 0x7f04ec06e370 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f04fc007fd0 tx=0x7f04fc007960 comp rx=0 tx=0).stop 2024-05-20T18:33:06.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.979+0000 7f051183f640 1 -- 172.21.15.5:0/2887705918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f050c0b1e10 msgr2=0x7f050c0c8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:06.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.979+0000 7f051183f640 1 --2- 172.21.15.5:0/2887705918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f050c0b1e10 0x7f050c0c8990 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f04f4000c00 tx=0x7f04f40046c0 comp rx=0 tx=0).stop 2024-05-20T18:33:06.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.979+0000 7f051183f640 1 -- 172.21.15.5:0/2887705918 shutdown_connections 2024-05-20T18:33:06.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.979+0000 7f051183f640 1 --2- 172.21.15.5:0/2887705918 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f04ec06be60 0x7f04ec06e370 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:06.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.980+0000 7f051183f640 1 --2- 172.21.15.5:0/2887705918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f050c15d530 0x7f050c0d1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:06.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.980+0000 7f051183f640 1 --2- 172.21.15.5:0/2887705918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f050c0b1e10 0x7f050c0c8990 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:06.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.980+0000 7f051183f640 1 --2- 172.21.15.5:0/2887705918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f050c070850 0x7f050c1f3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:06.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.980+0000 7f051183f640 1 -- 172.21.15.5:0/2887705918 >> 172.21.15.5:0/2887705918 conn(0x7f050c0d29d0 msgr2=0x7f050c15ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:06.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.980+0000 7f051183f640 1 -- 172.21.15.5:0/2887705918 shutdown_connections 2024-05-20T18:33:06.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:06.980+0000 7f051183f640 1 -- 172.21.15.5:0/2887705918 wait complete. 2024-05-20T18:33:06.992 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-20T18:33:06.992 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-20T18:33:06.992 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-20T18:33:06.992 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:33:06.992 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-20T18:33:06.992 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-20T18:33:06.993 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-20T18:33:06.993 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:33:06.993 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-20T18:33:06.993 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-20T18:33:06.993 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-20T18:33:06.993 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:33:06.993 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-20T18:33:06.994 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-20T18:33:06.994 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-20T18:33:06.994 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:33:07.002 INFO:tasks.ceph.mds.a:Stopped 2024-05-20T18:33:07.002 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-20T18:33:07.005 INFO:tasks.ceph.mds.c:Stopped 2024-05-20T18:33:07.005 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-20T18:33:07.006 INFO:tasks.ceph.mds.b:Stopped 2024-05-20T18:33:07.006 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-20T18:33:07.007 INFO:tasks.ceph.mds.d:Stopped 2024-05-20T18:33:07.007 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-20T18:33:07.046 INFO:tasks.ceph.mds.a:Started 2024-05-20T18:33:07.047 DEBUG:teuthology.parallel:result is None 2024-05-20T18:33:07.048 INFO:tasks.ceph.mds.c:Started 2024-05-20T18:33:07.048 DEBUG:teuthology.parallel:result is None 2024-05-20T18:33:07.049 INFO:tasks.ceph.mds.b:Started 2024-05-20T18:33:07.049 DEBUG:teuthology.parallel:result is None 2024-05-20T18:33:07.050 INFO:tasks.ceph.mds.d:Started 2024-05-20T18:33:07.050 DEBUG:teuthology.parallel:result is None 2024-05-20T18:33:07.051 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-20T18:33:07.139 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-05-20T18:33:07.139 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-05-20T18:33:07.151 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-20T18:33:07.152 INFO:tasks.ceph.mds.d.smithi063.stdout:starting mds.d at 2024-05-20T18:33:07.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.172+0000 7f2e4defe640 1 Processor -- start 2024-05-20T18:33:07.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.172+0000 7f2e4defe640 1 -- start start 2024-05-20T18:33:07.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.174+0000 7f2e4defe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2e4815d530 0x7f2e4815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:07.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.174+0000 7f2e4defe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2e48070850 0x7f2e48070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:07.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.174+0000 7f2e4defe640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e480cd580 con 0x7f2e480b1700 2024-05-20T18:33:07.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.174+0000 7f2e4defe640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f2e480ce190 con 0x7f2e4815d530 2024-05-20T18:33:07.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.174+0000 7f2e4defe640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2e480cd860 con 0x7f2e48070850 2024-05-20T18:33:07.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.174+0000 7f2e477fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2e4815d530 0x7f2e4815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:07.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.174+0000 7f2e477fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2e4815d530 0x7f2e4815aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56570/0 (socket says 172.21.15.5:56570) 2024-05-20T18:33:07.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.174+0000 7f2e477fe640 1 -- 172.21.15.5:0/2319977383 learned_addr learned my addr 172.21.15.5:0/2319977383 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:07.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.174+0000 7f2e47fff640 1 --2- 172.21.15.5:0/2319977383 >> v2:172.21.15.63:3300/0 conn(0x7f2e48070850 0x7f2e48070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:07.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.174+0000 7f2e477fe640 1 -- 172.21.15.5:0/2319977383 >> v2:172.21.15.63:3300/0 conn(0x7f2e48070850 msgr2=0x7f2e48070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:07.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.174+0000 7f2e477fe640 1 --2- 172.21.15.5:0/2319977383 >> v2:172.21.15.63:3300/0 conn(0x7f2e48070850 0x7f2e48070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:07.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.174+0000 7f2e477fe640 1 -- 172.21.15.5:0/2319977383 >> v1:172.21.15.63:6789/0 conn(0x7f2e480b1700 legacy=0x7f2e480b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:07.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.174+0000 7f2e477fe640 1 -- 172.21.15.5:0/2319977383 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e480cdb40 con 0x7f2e4815d530 2024-05-20T18:33:07.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.175+0000 7f2e477fe640 1 --2- 172.21.15.5:0/2319977383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e4815d530 0x7f2e4815aaa0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f2e480b8400 tx=0x7f2e3802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62af5e3cc3ebe075 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:07.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.175+0000 7f2e46ffd640 1 -- 172.21.15.5:0/2319977383 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38037070 con 0x7f2e4815d530 2024-05-20T18:33:07.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.175+0000 7f2e46ffd640 1 -- 172.21.15.5:0/2319977383 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e38032440 con 0x7f2e4815d530 2024-05-20T18:33:07.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.175+0000 7f2e46ffd640 1 -- 172.21.15.5:0/2319977383 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38032810 con 0x7f2e4815d530 2024-05-20T18:33:07.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.176+0000 7f2e4defe640 1 -- 172.21.15.5:0/2319977383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e4815d530 msgr2=0x7f2e4815aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:07.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.176+0000 7f2e4defe640 1 --2- 172.21.15.5:0/2319977383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e4815d530 0x7f2e4815aaa0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f2e480b8400 tx=0x7f2e3802a270 comp rx=0 tx=0).stop 2024-05-20T18:33:07.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.176+0000 7f2e4defe640 1 -- 172.21.15.5:0/2319977383 shutdown_connections 2024-05-20T18:33:07.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.176+0000 7f2e4defe640 1 --2- 172.21.15.5:0/2319977383 >> v2:172.21.15.63:3300/0 conn(0x7f2e48070850 0x7f2e48070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:07.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.176+0000 7f2e4defe640 1 --2- 172.21.15.5:0/2319977383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e4815d530 0x7f2e4815aaa0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:07.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.176+0000 7f2e4defe640 1 -- 172.21.15.5:0/2319977383 >> 172.21.15.5:0/2319977383 conn(0x7f2e480d29d0 msgr2=0x7f2e480d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:07.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.176+0000 7f2e4defe640 1 -- 172.21.15.5:0/2319977383 shutdown_connections 2024-05-20T18:33:07.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.176+0000 7f2e4defe640 1 -- 172.21.15.5:0/2319977383 wait complete. 2024-05-20T18:33:07.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.178+0000 7f2e4defe640 1 Processor -- start 2024-05-20T18:33:07.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.178+0000 7f2e4defe640 1 -- start start 2024-05-20T18:33:07.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.179+0000 7f2e4defe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e48070850 0x7f2e480ca390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:07.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.179+0000 7f2e4defe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e480b1700 0x7f2e480cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:07.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.179+0000 7f2e4defe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e4815d530 0x7f2e480cc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.179+0000 7f2e4defe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2e480711a0 con 0x7f2e480b1700 2024-05-20T18:33:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.179+0000 7f2e4defe640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2e480cd230 con 0x7f2e48070850 2024-05-20T18:33:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.180+0000 7f2e4defe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2e48071320 con 0x7f2e4815d530 2024-05-20T18:33:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.180+0000 7f2e3effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e480b1700 0x7f2e480cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.180+0000 7f2e3effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e480b1700 0x7f2e480cbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56572/0 (socket says 172.21.15.5:56572) 2024-05-20T18:33:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.180+0000 7f2e3effd640 1 -- 172.21.15.5:0/882297486 learned_addr learned my addr 172.21.15.5:0/882297486 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.180+0000 7f2e477fe640 1 --2- 172.21.15.5:0/882297486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e48070850 0x7f2e480ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.180+0000 7f2e47fff640 1 --2- 172.21.15.5:0/882297486 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e4815d530 0x7f2e480cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.180+0000 7f2e3effd640 1 -- 172.21.15.5:0/882297486 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e4815d530 msgr2=0x7f2e480cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.180+0000 7f2e3effd640 1 --2- 172.21.15.5:0/882297486 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e4815d530 0x7f2e480cc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.180+0000 7f2e3effd640 1 -- 172.21.15.5:0/882297486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e48070850 msgr2=0x7f2e480ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.180+0000 7f2e3effd640 1 --2- 172.21.15.5:0/882297486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e48070850 0x7f2e480ca390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.180+0000 7f2e3effd640 1 -- 172.21.15.5:0/882297486 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4806c630 con 0x7f2e480b1700 2024-05-20T18:33:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.180+0000 7f2e3effd640 1 --2- 172.21.15.5:0/882297486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e480b1700 0x7f2e480cbe40 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000c3c0 tx=0x7f2e3000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.180+0000 7f2e44ff9640 1 -- 172.21.15.5:0/882297486 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e300052d0 con 0x7f2e480b1700 2024-05-20T18:33:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.180+0000 7f2e44ff9640 1 -- 172.21.15.5:0/882297486 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e30005490 con 0x7f2e480b1700 2024-05-20T18:33:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.180+0000 7f2e4defe640 1 -- 172.21.15.5:0/882297486 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e480c9c20 con 0x7f2e480b1700 2024-05-20T18:33:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.180+0000 7f2e44ff9640 1 -- 172.21.15.5:0/882297486 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e30007260 con 0x7f2e480b1700 2024-05-20T18:33:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.180+0000 7f2e4defe640 1 -- 172.21.15.5:0/882297486 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e4806c8d0 con 0x7f2e480b1700 2024-05-20T18:33:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.181+0000 7f2e4defe640 1 -- 172.21.15.5:0/882297486 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e48075b00 con 0x7f2e480b1700 2024-05-20T18:33:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.182+0000 7f2e44ff9640 1 -- 172.21.15.5:0/882297486 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2e30007400 con 0x7f2e480b1700 2024-05-20T18:33:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.183+0000 7f2e44ff9640 1 --2- 172.21.15.5:0/882297486 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2e2806be30 0x7f2e2806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.183+0000 7f2e477fe640 1 --2- 172.21.15.5:0/882297486 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2e2806be30 0x7f2e2806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.183+0000 7f2e44ff9640 1 -- 172.21.15.5:0/882297486 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5421+0+0 (secure 0 0 0) 0x7f2e3008a240 con 0x7f2e480b1700 2024-05-20T18:33:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.183+0000 7f2e477fe640 1 --2- 172.21.15.5:0/882297486 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2e2806be30 0x7f2e2806e340 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2e38034000 tx=0x7f2e3802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:07.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.187+0000 7f2e44ff9640 1 -- 172.21.15.5:0/882297486 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2e30058630 con 0x7f2e480b1700 2024-05-20T18:33:07.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:07.383+0000 7f2e4defe640 1 -- 172.21.15.5:0/882297486 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f2e4815cab0 con 0x7f2e480b1700 2024-05-20T18:33:08.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.410+0000 7f2e44ff9640 1 -- 172.21.15.5:0/882297486 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v86) ==== 97+0+0 (secure 0 0 0) 0x7f2e3005c930 con 0x7f2e480b1700 2024-05-20T18:33:08.414 INFO:teuthology.orchestra.run.smithi005.stderr: removed all blocklist entries 2024-05-20T18:33:08.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.413+0000 7f2e4defe640 1 -- 172.21.15.5:0/882297486 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2e2806be30 msgr2=0x7f2e2806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:08.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.413+0000 7f2e4defe640 1 --2- 172.21.15.5:0/882297486 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2e2806be30 0x7f2e2806e340 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2e38034000 tx=0x7f2e3802e000 comp rx=0 tx=0).stop 2024-05-20T18:33:08.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.413+0000 7f2e4defe640 1 -- 172.21.15.5:0/882297486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e480b1700 msgr2=0x7f2e480cbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:08.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.413+0000 7f2e4defe640 1 --2- 172.21.15.5:0/882297486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e480b1700 0x7f2e480cbe40 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000c3c0 tx=0x7f2e3000c880 comp rx=0 tx=0).stop 2024-05-20T18:33:08.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.413+0000 7f2e4defe640 1 -- 172.21.15.5:0/882297486 shutdown_connections 2024-05-20T18:33:08.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.413+0000 7f2e4defe640 1 --2- 172.21.15.5:0/882297486 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2e2806be30 0x7f2e2806e340 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:08.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.413+0000 7f2e4defe640 1 --2- 172.21.15.5:0/882297486 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e4815d530 0x7f2e480cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:08.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.413+0000 7f2e4defe640 1 --2- 172.21.15.5:0/882297486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e480b1700 0x7f2e480cbe40 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:08.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.413+0000 7f2e4defe640 1 --2- 172.21.15.5:0/882297486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e48070850 0x7f2e480ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:08.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.413+0000 7f2e4defe640 1 -- 172.21.15.5:0/882297486 >> 172.21.15.5:0/882297486 conn(0x7f2e480d29d0 msgr2=0x7f2e480b3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:08.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.414+0000 7f2e4defe640 1 -- 172.21.15.5:0/882297486 shutdown_connections 2024-05-20T18:33:08.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.414+0000 7f2e4defe640 1 -- 172.21.15.5:0/882297486 wait complete. 2024-05-20T18:33:08.430 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-20T18:33:08.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.554+0000 7faad5de3640 1 Processor -- start 2024-05-20T18:33:08.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.555+0000 7faad5de3640 1 -- start start 2024-05-20T18:33:08.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.556+0000 7faad5de3640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7faad006d3d0 0x7faad006d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.556+0000 7faad5de3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7faad00b8190 0x7faad00b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.556+0000 7faad5de3640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faad006dd20 con 0x7faad0162820 2024-05-20T18:33:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.556+0000 7faad5de3640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7faad00d0630 con 0x7faad00b8190 2024-05-20T18:33:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.556+0000 7faad5de3640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7faad00d39c0 con 0x7faad006d3d0 2024-05-20T18:33:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.556+0000 7faacffff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7faad00b8190 0x7faad00b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.556+0000 7faacf7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7faad0162820 0x7faad00b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38186/0 (socket says 172.21.15.5:38186) 2024-05-20T18:33:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.556+0000 7faacf7fe640 1 -- 172.21.15.5:0/1804693545 learned_addr learned my addr 172.21.15.5:0/1804693545 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:33:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.556+0000 7faaceffd640 1 --2- 172.21.15.5:0/1804693545 >> v2:172.21.15.63:3300/0 conn(0x7faad006d3d0 0x7faad006d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.556+0000 7faacffff640 1 -- 172.21.15.5:0/1804693545 >> v2:172.21.15.63:3300/0 conn(0x7faad006d3d0 msgr2=0x7faad006d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.556+0000 7faacffff640 1 --2- 172.21.15.5:0/1804693545 >> v2:172.21.15.63:3300/0 conn(0x7faad006d3d0 0x7faad006d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.557+0000 7faacffff640 1 -- 172.21.15.5:0/1804693545 >> v1:172.21.15.5:6789/0 conn(0x7faad0162820 legacy=0x7faad00b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.557+0000 7faace7fc640 1 -- 172.21.15.5:0/1804693545 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3182724643 0 0) 0x7faac0001110 con 0x7faad0162820 2024-05-20T18:33:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.557+0000 7faacffff640 1 -- 172.21.15.5:0/1804693545 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad00d3ca0 con 0x7faad00b8190 2024-05-20T18:33:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.557+0000 7faacffff640 1 --2- 172.21.15.5:0/1804693545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad00b8190 0x7faad00b85a0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7faad00d1230 tx=0x7faac802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44ae52bf0ee88d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.557+0000 7faace7fc640 1 -- 172.21.15.5:0/1804693545 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac8007740 con 0x7faad00b8190 2024-05-20T18:33:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.557+0000 7faace7fc640 1 -- 172.21.15.5:0/1804693545 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faac80078e0 con 0x7faad00b8190 2024-05-20T18:33:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.557+0000 7faace7fc640 1 -- 172.21.15.5:0/1804693545 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac8007cb0 con 0x7faad00b8190 2024-05-20T18:33:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.558+0000 7faad5de3640 1 -- 172.21.15.5:0/1804693545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad00b8190 msgr2=0x7faad00b85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.558+0000 7faad5de3640 1 --2- 172.21.15.5:0/1804693545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad00b8190 0x7faad00b85a0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7faad00d1230 tx=0x7faac802a680 comp rx=0 tx=0).stop 2024-05-20T18:33:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.558+0000 7faad5de3640 1 -- 172.21.15.5:0/1804693545 shutdown_connections 2024-05-20T18:33:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.558+0000 7faad5de3640 1 --2- 172.21.15.5:0/1804693545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad00b8190 0x7faad00b85a0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.558+0000 7faad5de3640 1 --2- 172.21.15.5:0/1804693545 >> v2:172.21.15.63:3300/0 conn(0x7faad006d3d0 0x7faad006d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.558+0000 7faad5de3640 1 -- 172.21.15.5:0/1804693545 >> 172.21.15.5:0/1804693545 conn(0x7faad00d4bd0 msgr2=0x7faad00d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.558+0000 7faad5de3640 1 -- 172.21.15.5:0/1804693545 shutdown_connections 2024-05-20T18:33:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.558+0000 7faad5de3640 1 -- 172.21.15.5:0/1804693545 wait complete. 2024-05-20T18:33:08.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.560+0000 7faad5de3640 1 Processor -- start 2024-05-20T18:33:08.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.560+0000 7faad5de3640 1 -- start start 2024-05-20T18:33:08.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.561+0000 7faad5de3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad006d3d0 0x7faad015a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:08.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.561+0000 7faad5de3640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faad00b8190 0x7faad0157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:08.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.562+0000 7faad5de3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faad0162820 0x7faad0149aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:08.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.562+0000 7faad5de3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faad00d0300 con 0x7faad006d3d0 2024-05-20T18:33:08.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.562+0000 7faad5de3640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faad00d4260 con 0x7faad00b8190 2024-05-20T18:33:08.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.562+0000 7faad5de3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faad00b4050 con 0x7faad0162820 2024-05-20T18:33:08.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.562+0000 7faacf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad006d3d0 0x7faad015a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:08.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.562+0000 7faacf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad006d3d0 0x7faad015a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56586/0 (socket says 172.21.15.5:56586) 2024-05-20T18:33:08.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.562+0000 7faacf7fe640 1 -- 172.21.15.5:0/3816972600 learned_addr learned my addr 172.21.15.5:0/3816972600 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:08.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.562+0000 7faaceffd640 1 --2- 172.21.15.5:0/3816972600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faad00b8190 0x7faad0157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:08.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.562+0000 7faacffff640 1 --2- 172.21.15.5:0/3816972600 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faad0162820 0x7faad0149aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:08.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.562+0000 7faacf7fe640 1 -- 172.21.15.5:0/3816972600 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faad0162820 msgr2=0x7faad0149aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:08.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.562+0000 7faacf7fe640 1 --2- 172.21.15.5:0/3816972600 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faad0162820 0x7faad0149aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:08.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.562+0000 7faacf7fe640 1 -- 172.21.15.5:0/3816972600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faad00b8190 msgr2=0x7faad0157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:08.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.562+0000 7faacf7fe640 1 --2- 172.21.15.5:0/3816972600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faad00b8190 0x7faad0157df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:08.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.562+0000 7faacf7fe640 1 -- 172.21.15.5:0/3816972600 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad015ae30 con 0x7faad006d3d0 2024-05-20T18:33:08.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.562+0000 7faacf7fe640 1 --2- 172.21.15.5:0/3816972600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad006d3d0 0x7faad015a8f0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7faac000c460 tx=0x7faac000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:08.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.562+0000 7faab7fff640 1 -- 172.21.15.5:0/3816972600 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac0018070 con 0x7faad006d3d0 2024-05-20T18:33:08.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.562+0000 7faab7fff640 1 -- 172.21.15.5:0/3816972600 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faac0007500 con 0x7faad006d3d0 2024-05-20T18:33:08.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.562+0000 7faad5de3640 1 -- 172.21.15.5:0/3816972600 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faad0148a00 con 0x7faad006d3d0 2024-05-20T18:33:08.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.562+0000 7faab7fff640 1 -- 172.21.15.5:0/3816972600 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac0004a20 con 0x7faad006d3d0 2024-05-20T18:33:08.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.563+0000 7faad5de3640 1 -- 172.21.15.5:0/3816972600 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faad0148fc0 con 0x7faad006d3d0 2024-05-20T18:33:08.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.563+0000 7faad5de3640 1 -- 172.21.15.5:0/3816972600 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faad0075b00 con 0x7faad006d3d0 2024-05-20T18:33:08.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.564+0000 7faab7fff640 1 -- 172.21.15.5:0/3816972600 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faac0004bc0 con 0x7faad006d3d0 2024-05-20T18:33:08.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.565+0000 7faab7fff640 1 --2- 172.21.15.5:0/3816972600 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faabc06be30 0x7faabc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:08.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.565+0000 7faab7fff640 1 -- 172.21.15.5:0/3816972600 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5249+0+0 (secure 0 0 0) 0x7faac0089fa0 con 0x7faad006d3d0 2024-05-20T18:33:08.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.565+0000 7faaceffd640 1 --2- 172.21.15.5:0/3816972600 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faabc06be30 0x7faabc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:08.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.565+0000 7faaceffd640 1 --2- 172.21.15.5:0/3816972600 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faabc06be30 0x7faabc06e340 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7faad01f8630 tx=0x7faac4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:08.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.569+0000 7faab7fff640 1 -- 172.21.15.5:0/3816972600 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7faad006dd20 con 0x7faad006d3d0 2024-05-20T18:33:08.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.826+0000 7faad5de3640 1 -- 172.21.15.5:0/3816972600 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7faad0067b60 con 0x7faad006d3d0 2024-05-20T18:33:08.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.826+0000 7faab7fff640 1 -- 172.21.15.5:0/3816972600 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7faac0058440 con 0x7faad006d3d0 2024-05-20T18:33:08.827 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:33:08.827 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:33:08.827 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_dump": [ 2024-05-20T18:33:08.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:33:08.828 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.a", 2024-05-20T18:33:08.828 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtm9lSeCBAA/PpiEqojk6ueJo60tU4/iA==", 2024-05-20T18:33:08.828 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:33:08.828 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:33:08.828 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:33:08.828 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:33:08.829 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:33:08.829 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:33:08.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:33:08.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:33:08.829 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.b", 2024-05-20T18:33:08.829 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmdov0FRAACkWAR/0zaCDjgnkHnoHwgw==", 2024-05-20T18:33:08.830 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:33:08.830 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:33:08.830 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:33:08.830 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:33:08.830 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:33:08.830 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:33:08.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:33:08.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:33:08.830 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.c", 2024-05-20T18:33:08.830 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmpEBlEBAAdgxDXj9x38VSOizUgdFpYA==", 2024-05-20T18:33:08.830 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:33:08.830 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:33:08.830 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:33:08.831 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:33:08.831 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:33:08.831 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:33:08.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:33:08.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:33:08.831 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.d", 2024-05-20T18:33:08.831 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtm0Gy9HRAAAkPqcBLwxTwuWmCi7aad1A==", 2024-05-20T18:33:08.831 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:33:08.831 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:33:08.831 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:33:08.831 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:33:08.831 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:33:08.831 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:33:08.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:33:08.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:33:08.832 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.0", 2024-05-20T18:33:08.832 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDHlUtmWFGzBRAA81qN37dlHf6QDZ1FgwbJ9w==", 2024-05-20T18:33:08.832 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:33:08.832 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:33:08.832 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:33:08.832 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:33:08.832 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:33:08.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:33:08.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:33:08.832 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.1", 2024-05-20T18:33:08.832 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDKlUtmBMKsBBAAD4tTTvetME6oxwugTubSKQ==", 2024-05-20T18:33:08.833 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:33:08.833 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:33:08.833 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:33:08.833 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:33:08.833 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:33:08.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:33:08.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:33:08.833 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.2", 2024-05-20T18:33:08.833 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDNlUtm4Rm/AxAAIGWTUxb+LII09Rd+h5cJ8Q==", 2024-05-20T18:33:08.833 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:33:08.833 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:33:08.833 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:33:08.833 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:33:08.834 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:33:08.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:33:08.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:33:08.834 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.3", 2024-05-20T18:33:08.834 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDQlUtmWVxZAxAAB/w6k4WJ58ulEr9zatCRfw==", 2024-05-20T18:33:08.834 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:33:08.834 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:33:08.834 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:33:08.834 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:33:08.834 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:33:08.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:33:08.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:33:08.834 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.4", 2024-05-20T18:33:08.835 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDWlUtmIyokCRAAolOiYVCYaNTQZb1PjtR6fg==", 2024-05-20T18:33:08.835 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:33:08.835 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:33:08.835 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:33:08.835 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:33:08.835 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:33:08.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:33:08.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:33:08.835 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.5", 2024-05-20T18:33:08.835 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZlUtmSf6mCBAAw73nwq/zzt9kIXZymZh93A==", 2024-05-20T18:33:08.835 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:33:08.835 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:33:08.835 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:33:08.836 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:33:08.836 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:33:08.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:33:08.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:33:08.836 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.6", 2024-05-20T18:33:08.836 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDclUtmwwhGCBAA94OBEtj9h7cmCAddYMonaA==", 2024-05-20T18:33:08.836 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:33:08.836 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:33:08.836 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:33:08.836 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:33:08.836 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:33:08.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:33:08.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:33:08.836 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.7", 2024-05-20T18:33:08.837 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDflUtm9wefBxAAvCZTBzRu05koFhc+pH5I8w==", 2024-05-20T18:33:08.837 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:33:08.837 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:33:08.837 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:33:08.837 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:33:08.837 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:33:08.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:33:08.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:33:08.837 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.0", 2024-05-20T18:33:08.837 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmevaaIhAATIwyXzr0vGT38ZmcMqR2gA==", 2024-05-20T18:33:08.837 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:33:08.837 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:33:08.837 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow r", 2024-05-20T18:33:08.838 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-20T18:33:08.838 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:33:08.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:33:08.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:33:08.838 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.1", 2024-05-20T18:33:08.838 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmK7NfKhAADaXl0uVP1Q1gUOlPsstnzg==", 2024-05-20T18:33:08.838 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:33:08.838 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:33:08.838 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-20T18:33:08.838 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-20T18:33:08.838 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-20T18:33:08.838 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:33:08.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:33:08.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:33:08.839 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.2", 2024-05-20T18:33:08.839 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmMJF1JhAAeM+HseqvgvKgG8EnIsZuhg==", 2024-05-20T18:33:08.839 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:33:08.839 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:33:08.839 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-20T18:33:08.839 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-20T18:33:08.839 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-20T18:33:08.839 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:33:08.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:33:08.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:33:08.839 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.3", 2024-05-20T18:33:08.839 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmodA7LhAAUI2v0npB6HHs2n1g3KdTXA==", 2024-05-20T18:33:08.839 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:33:08.840 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:33:08.840 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-20T18:33:08.840 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-20T18:33:08.840 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-20T18:33:08.840 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:33:08.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:33:08.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:33:08.840 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.admin", 2024-05-20T18:33:08.840 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDElUtmZacGJRAAoyr9XNPBr9r0DSU53qCHSQ==", 2024-05-20T18:33:08.840 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:33:08.840 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-20T18:33:08.840 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:33:08.841 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:33:08.841 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:33:08.841 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:33:08.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:33:08.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:33:08.841 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.x", 2024-05-20T18:33:08.841 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmPgq0BBAAAljNeQDsSau0wH2Lh2KdBw==", 2024-05-20T18:33:08.841 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:33:08.841 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-20T18:33:08.841 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-05-20T18:33:08.841 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:33:08.841 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:33:08.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:33:08.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:33:08.842 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.y", 2024-05-20T18:33:08.842 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDElUtm6Xv3OhAA0doOWTC73jy5rOYm7Vf68Q==", 2024-05-20T18:33:08.842 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:33:08.842 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-20T18:33:08.842 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-05-20T18:33:08.842 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:33:08.842 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:33:08.842 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:33:08.842 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-20T18:33:08.842 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:33:08.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.828+0000 7faad5de3640 1 -- 172.21.15.5:0/3816972600 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faabc06be30 msgr2=0x7faabc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:08.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.828+0000 7faad5de3640 1 --2- 172.21.15.5:0/3816972600 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faabc06be30 0x7faabc06e340 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7faad01f8630 tx=0x7faac4008040 comp rx=0 tx=0).stop 2024-05-20T18:33:08.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.828+0000 7faad5de3640 1 -- 172.21.15.5:0/3816972600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad006d3d0 msgr2=0x7faad015a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:08.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.828+0000 7faad5de3640 1 --2- 172.21.15.5:0/3816972600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad006d3d0 0x7faad015a8f0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7faac000c460 tx=0x7faac000c920 comp rx=0 tx=0).stop 2024-05-20T18:33:08.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.828+0000 7faad5de3640 1 -- 172.21.15.5:0/3816972600 shutdown_connections 2024-05-20T18:33:08.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.828+0000 7faad5de3640 1 --2- 172.21.15.5:0/3816972600 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faabc06be30 0x7faabc06e340 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:08.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.828+0000 7faad5de3640 1 --2- 172.21.15.5:0/3816972600 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faad0162820 0x7faad0149aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:08.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.828+0000 7faad5de3640 1 --2- 172.21.15.5:0/3816972600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faad00b8190 0x7faad0157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:08.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.828+0000 7faad5de3640 1 --2- 172.21.15.5:0/3816972600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad006d3d0 0x7faad015a8f0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:08.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.828+0000 7faad5de3640 1 -- 172.21.15.5:0/3816972600 >> 172.21.15.5:0/3816972600 conn(0x7faad00d4bd0 msgr2=0x7faad015d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:08.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.829+0000 7faad5de3640 1 -- 172.21.15.5:0/3816972600 shutdown_connections 2024-05-20T18:33:08.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.829+0000 7faad5de3640 1 -- 172.21.15.5:0/3816972600 wait complete. 2024-05-20T18:33:08.846 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:33:08.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.959+0000 7f1df532f640 1 Processor -- start 2024-05-20T18:33:08.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.959+0000 7f1df532f640 1 -- start start 2024-05-20T18:33:08.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.961+0000 7f1df532f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1df0162820 0x7f1df00b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:08.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.961+0000 7f1df532f640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1df00b3e80 0x7f1df006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:08.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.961+0000 7f1df532f640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1df006dd20 con 0x7f1df006d910 2024-05-20T18:33:08.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.961+0000 7f1df532f640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1df00d0630 con 0x7f1df00b3e80 2024-05-20T18:33:08.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.961+0000 7f1df532f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1df00d39c0 con 0x7f1df0162820 2024-05-20T18:33:08.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.961+0000 7f1deeffd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1df0162820 0x7f1df00b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:08.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.961+0000 7f1dee7fc640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1df00b3e80 0x7f1df006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:08.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.961+0000 7f1deeffd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1df0162820 0x7f1df00b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:57624/0 (socket says 172.21.15.5:57624) 2024-05-20T18:33:08.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.961+0000 7f1deeffd640 1 -- 172.21.15.5:0/1463204206 learned_addr learned my addr 172.21.15.5:0/1463204206 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:08.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.962+0000 7f1deeffd640 1 -- 172.21.15.5:0/1463204206 >> v2:172.21.15.63:3300/0 conn(0x7f1df00b3e80 msgr2=0x7f1df006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:08.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.962+0000 7f1deeffd640 1 --2- 172.21.15.5:0/1463204206 >> v2:172.21.15.63:3300/0 conn(0x7f1df00b3e80 0x7f1df006d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:08.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.962+0000 7f1deeffd640 1 -- 172.21.15.5:0/1463204206 >> v1:172.21.15.63:6789/0 conn(0x7f1df006d910 legacy=0x7f1df00d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:08.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.962+0000 7f1deeffd640 1 -- 172.21.15.5:0/1463204206 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df00d3ca0 con 0x7f1df0162820 2024-05-20T18:33:08.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.962+0000 7f1deeffd640 1 --2- 172.21.15.5:0/1463204206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1df0162820 0x7f1df00b3940 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1de0009e50 tx=0x7f1de002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3419c619bfd34a42 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:08.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.963+0000 7f1dedffb640 1 -- 172.21.15.5:0/1463204206 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de0036070 con 0x7f1df0162820 2024-05-20T18:33:08.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.963+0000 7f1dedffb640 1 -- 172.21.15.5:0/1463204206 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1de0031600 con 0x7f1df0162820 2024-05-20T18:33:08.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.963+0000 7f1dedffb640 1 -- 172.21.15.5:0/1463204206 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de00319d0 con 0x7f1df0162820 2024-05-20T18:33:08.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.963+0000 7f1df532f640 1 -- 172.21.15.5:0/1463204206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1df0162820 msgr2=0x7f1df00b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:08.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.963+0000 7f1df532f640 1 --2- 172.21.15.5:0/1463204206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1df0162820 0x7f1df00b3940 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1de0009e50 tx=0x7f1de002a710 comp rx=0 tx=0).stop 2024-05-20T18:33:08.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.963+0000 7f1df532f640 1 -- 172.21.15.5:0/1463204206 shutdown_connections 2024-05-20T18:33:08.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.963+0000 7f1df532f640 1 --2- 172.21.15.5:0/1463204206 >> v2:172.21.15.63:3300/0 conn(0x7f1df00b3e80 0x7f1df006d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:08.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.963+0000 7f1df532f640 1 --2- 172.21.15.5:0/1463204206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1df0162820 0x7f1df00b3940 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:08.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.963+0000 7f1df532f640 1 -- 172.21.15.5:0/1463204206 >> 172.21.15.5:0/1463204206 conn(0x7f1df00d4bd0 msgr2=0x7f1df00d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:08.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.963+0000 7f1df532f640 1 -- 172.21.15.5:0/1463204206 shutdown_connections 2024-05-20T18:33:08.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.963+0000 7f1df532f640 1 -- 172.21.15.5:0/1463204206 wait complete. 2024-05-20T18:33:08.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.965+0000 7f1df532f640 1 Processor -- start 2024-05-20T18:33:08.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.965+0000 7f1df532f640 1 -- start start 2024-05-20T18:33:08.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.967+0000 7f1df532f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1df006d910 0x7f1df014ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:08.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.967+0000 7f1df532f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1df00b3e80 0x7f1df01e8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:08.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.967+0000 7f1df532f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1df0162820 0x7f1df01e88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:08.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.967+0000 7f1df532f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1df00d0300 con 0x7f1df0162820 2024-05-20T18:33:08.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.967+0000 7f1df532f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1df00d4260 con 0x7f1df00b3e80 2024-05-20T18:33:08.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.967+0000 7f1df532f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1df00b91a0 con 0x7f1df006d910 2024-05-20T18:33:08.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.967+0000 7f1def7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1df0162820 0x7f1df01e88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:08.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.967+0000 7f1deeffd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1df006d910 0x7f1df014ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:08.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.967+0000 7f1dee7fc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1df00b3e80 0x7f1df01e8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:08.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.967+0000 7f1deeffd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1df006d910 0x7f1df014ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:57638/0 (socket says 172.21.15.5:57638) 2024-05-20T18:33:08.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.967+0000 7f1def7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1df0162820 0x7f1df01e88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56588/0 (socket says 172.21.15.5:56588) 2024-05-20T18:33:08.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.967+0000 7f1deeffd640 1 -- 172.21.15.5:0/2004210549 learned_addr learned my addr 172.21.15.5:0/2004210549 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:08.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.967+0000 7f1def7fe640 1 -- 172.21.15.5:0/2004210549 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1df006d910 msgr2=0x7f1df014ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:08.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.967+0000 7f1def7fe640 1 --2- 172.21.15.5:0/2004210549 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1df006d910 0x7f1df014ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:08.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.967+0000 7f1def7fe640 1 -- 172.21.15.5:0/2004210549 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1df00b3e80 msgr2=0x7f1df01e8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:08.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.967+0000 7f1def7fe640 1 --2- 172.21.15.5:0/2004210549 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1df00b3e80 0x7f1df01e8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:08.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.967+0000 7f1def7fe640 1 -- 172.21.15.5:0/2004210549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df01e90f0 con 0x7f1df0162820 2024-05-20T18:33:08.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.967+0000 7f1deeffd640 1 --2- 172.21.15.5:0/2004210549 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1df006d910 0x7f1df014ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:33:08.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.967+0000 7f1dee7fc640 1 --2- 172.21.15.5:0/2004210549 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1df00b3e80 0x7f1df01e8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:33:08.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.967+0000 7f1def7fe640 1 --2- 172.21.15.5:0/2004210549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1df0162820 0x7f1df01e88b0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f1de800c3c0 tx=0x7f1de800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:08.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.968+0000 7f1de77fe640 1 -- 172.21.15.5:0/2004210549 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de8017070 con 0x7f1df0162820 2024-05-20T18:33:08.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.968+0000 7f1de77fe640 1 -- 172.21.15.5:0/2004210549 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1de80052d0 con 0x7f1df0162820 2024-05-20T18:33:08.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.968+0000 7f1df532f640 1 -- 172.21.15.5:0/2004210549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1df01e93d0 con 0x7f1df0162820 2024-05-20T18:33:08.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.968+0000 7f1de77fe640 1 -- 172.21.15.5:0/2004210549 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de8003830 con 0x7f1df0162820 2024-05-20T18:33:08.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.968+0000 7f1df532f640 1 -- 172.21.15.5:0/2004210549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1df014dd20 con 0x7f1df0162820 2024-05-20T18:33:08.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.969+0000 7f1df532f640 1 -- 172.21.15.5:0/2004210549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1df01e8e60 con 0x7f1df0162820 2024-05-20T18:33:08.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.969+0000 7f1de77fe640 1 -- 172.21.15.5:0/2004210549 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1de801a070 con 0x7f1df0162820 2024-05-20T18:33:08.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.970+0000 7f1de77fe640 1 --2- 172.21.15.5:0/2004210549 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1dd006be60 0x7f1dd006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:08.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.970+0000 7f1de77fe640 1 -- 172.21.15.5:0/2004210549 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5249+0+0 (secure 0 0 0) 0x7f1de8089eb0 con 0x7f1df0162820 2024-05-20T18:33:08.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.970+0000 7f1deeffd640 1 --2- 172.21.15.5:0/2004210549 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1dd006be60 0x7f1dd006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:08.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.970+0000 7f1deeffd640 1 --2- 172.21.15.5:0/2004210549 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1dd006be60 0x7f1dd006e370 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1de0000f80 tx=0x7f1de0009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:08.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:08.974+0000 7f1de77fe640 1 -- 172.21.15.5:0/2004210549 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1df006dd20 con 0x7f1df0162820 2024-05-20T18:33:09.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.158+0000 7f1df532f640 1 -- 172.21.15.5:0/2004210549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1df0075b00 con 0x7f1df0162820 2024-05-20T18:33:09.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.158+0000 7f1de77fe640 1 -- 172.21.15.5:0/2004210549 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+13045 (secure 0 0 0) 0x7f1de8058400 con 0x7f1df0162820 2024-05-20T18:33:09.160 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:33:09.160 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":86,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:33:08.409913+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:33:09.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.160+0000 7f1df532f640 1 -- 172.21.15.5:0/2004210549 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1dd006be60 msgr2=0x7f1dd006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:09.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.160+0000 7f1df532f640 1 --2- 172.21.15.5:0/2004210549 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1dd006be60 0x7f1dd006e370 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1de0000f80 tx=0x7f1de0009000 comp rx=0 tx=0).stop 2024-05-20T18:33:09.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.160+0000 7f1df532f640 1 -- 172.21.15.5:0/2004210549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1df0162820 msgr2=0x7f1df01e88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:09.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.160+0000 7f1df532f640 1 --2- 172.21.15.5:0/2004210549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1df0162820 0x7f1df01e88b0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f1de800c3c0 tx=0x7f1de800c880 comp rx=0 tx=0).stop 2024-05-20T18:33:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.160+0000 7f1df532f640 1 -- 172.21.15.5:0/2004210549 shutdown_connections 2024-05-20T18:33:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.160+0000 7f1df532f640 1 --2- 172.21.15.5:0/2004210549 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1dd006be60 0x7f1dd006e370 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.160+0000 7f1df532f640 1 --2- 172.21.15.5:0/2004210549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1df0162820 0x7f1df01e88b0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.160+0000 7f1df532f640 1 --2- 172.21.15.5:0/2004210549 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1df00b3e80 0x7f1df01e8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.160+0000 7f1df532f640 1 --2- 172.21.15.5:0/2004210549 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1df006d910 0x7f1df014ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.161+0000 7f1df532f640 1 -- 172.21.15.5:0/2004210549 >> 172.21.15.5:0/2004210549 conn(0x7f1df00d4bd0 msgr2=0x7f1df00d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.161+0000 7f1df532f640 1 -- 172.21.15.5:0/2004210549 shutdown_connections 2024-05-20T18:33:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.161+0000 7f1df532f640 1 -- 172.21.15.5:0/2004210549 wait complete. 2024-05-20T18:33:09.173 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:33:09.174 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:33:09.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.293+0000 7fb2fc044640 1 Processor -- start 2024-05-20T18:33:09.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.293+0000 7fb2fc044640 1 -- start start 2024-05-20T18:33:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.294+0000 7fb2fc044640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb2f40cd680 0x7fb2f415f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.294+0000 7fb2fc044640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb2f406c6c0 0x7fb2f406cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.294+0000 7fb2fc044640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2f40cf3d0 0x7fb2f40cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.294+0000 7fb2fc044640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fb2f40cda90 con 0x7fb2f406c6c0 2024-05-20T18:33:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.294+0000 7fb2fc044640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb2f415f760 con 0x7fb2f40cd680 2024-05-20T18:33:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.294+0000 7fb2fc044640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb2f40cfd20 con 0x7fb2f40cf3d0 2024-05-20T18:33:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.295+0000 7fb2f95df640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb2f406c6c0 0x7fb2f406cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.295+0000 7fb2f95df640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb2f406c6c0 0x7fb2f406cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56598/0 (socket says 172.21.15.5:56598) 2024-05-20T18:33:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.295+0000 7fb2f95df640 1 -- 172.21.15.5:0/19571196 learned_addr learned my addr 172.21.15.5:0/19571196 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.295+0000 7fb2f9de0640 1 --2- 172.21.15.5:0/19571196 >> v2:172.21.15.63:3300/0 conn(0x7fb2f40cd680 0x7fb2f415f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.295+0000 7fb2fa5e1640 1 --2- 172.21.15.5:0/19571196 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2f40cf3d0 0x7fb2f40cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.295+0000 7fb2f95df640 1 -- 172.21.15.5:0/19571196 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2f40cf3d0 msgr2=0x7fb2f40cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.295+0000 7fb2f95df640 1 --2- 172.21.15.5:0/19571196 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2f40cf3d0 0x7fb2f40cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.295+0000 7fb2f95df640 1 -- 172.21.15.5:0/19571196 >> v2:172.21.15.63:3300/0 conn(0x7fb2f40cd680 msgr2=0x7fb2f415f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.295+0000 7fb2f95df640 1 --2- 172.21.15.5:0/19571196 >> v2:172.21.15.63:3300/0 conn(0x7fb2f40cd680 0x7fb2f415f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.295+0000 7fb2f95df640 1 -- 172.21.15.5:0/19571196 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f406d0f0 con 0x7fb2f406c6c0 2024-05-20T18:33:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.295+0000 7fb2fa5e1640 1 --2- 172.21.15.5:0/19571196 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2f40cf3d0 0x7fb2f40cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.296+0000 7fb2f95df640 1 --2- 172.21.15.5:0/19571196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2f406c6c0 0x7fb2f406cad0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fb2f408baf0 tx=0x7fb2e402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b224d1b057868a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:09.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.296+0000 7fb2f8dde640 1 -- 172.21.15.5:0/19571196 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4037070 con 0x7fb2f406c6c0 2024-05-20T18:33:09.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.296+0000 7fb2f8dde640 1 -- 172.21.15.5:0/19571196 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2e4032440 con 0x7fb2f406c6c0 2024-05-20T18:33:09.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.296+0000 7fb2f8dde640 1 -- 172.21.15.5:0/19571196 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4032810 con 0x7fb2f406c6c0 2024-05-20T18:33:09.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.297+0000 7fb2fc044640 1 -- 172.21.15.5:0/19571196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2f406c6c0 msgr2=0x7fb2f406cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:09.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.297+0000 7fb2fc044640 1 --2- 172.21.15.5:0/19571196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2f406c6c0 0x7fb2f406cad0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fb2f408baf0 tx=0x7fb2e402a140 comp rx=0 tx=0).stop 2024-05-20T18:33:09.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.297+0000 7fb2fc044640 1 -- 172.21.15.5:0/19571196 shutdown_connections 2024-05-20T18:33:09.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.297+0000 7fb2fc044640 1 --2- 172.21.15.5:0/19571196 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2f40cf3d0 0x7fb2f40cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:09.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.297+0000 7fb2fc044640 1 --2- 172.21.15.5:0/19571196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2f406c6c0 0x7fb2f406cad0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:09.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.297+0000 7fb2fc044640 1 --2- 172.21.15.5:0/19571196 >> v2:172.21.15.63:3300/0 conn(0x7fb2f40cd680 0x7fb2f415f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:09.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.297+0000 7fb2fc044640 1 -- 172.21.15.5:0/19571196 >> 172.21.15.5:0/19571196 conn(0x7fb2f40b44d0 msgr2=0x7fb2f40b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:09.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.297+0000 7fb2fc044640 1 -- 172.21.15.5:0/19571196 shutdown_connections 2024-05-20T18:33:09.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.297+0000 7fb2fc044640 1 -- 172.21.15.5:0/19571196 wait complete. 2024-05-20T18:33:09.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.299+0000 7fb2fc044640 1 Processor -- start 2024-05-20T18:33:09.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.299+0000 7fb2fc044640 1 -- start start 2024-05-20T18:33:09.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.300+0000 7fb2fc044640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2f406c6c0 0x7fb2f40b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:09.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.300+0000 7fb2fc044640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2f40cd680 0x7fb2f415dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:09.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.300+0000 7fb2fc044640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2f40cf3d0 0x7fb2f4160a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:09.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.300+0000 7fb2fc044640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f40d0b40 con 0x7fb2f406c6c0 2024-05-20T18:33:09.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.300+0000 7fb2fc044640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f406d6b0 con 0x7fb2f40cd680 2024-05-20T18:33:09.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.300+0000 7fb2fc044640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb2f406d990 con 0x7fb2f40cf3d0 2024-05-20T18:33:09.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.300+0000 7fb2f9de0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2f406c6c0 0x7fb2f40b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:09.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.300+0000 7fb2f9de0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2f406c6c0 0x7fb2f40b7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56614/0 (socket says 172.21.15.5:56614) 2024-05-20T18:33:09.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.300+0000 7fb2f9de0640 1 -- 172.21.15.5:0/603115709 learned_addr learned my addr 172.21.15.5:0/603115709 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:09.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.300+0000 7fb2f95df640 1 --2- 172.21.15.5:0/603115709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2f40cd680 0x7fb2f415dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:09.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.300+0000 7fb2fa5e1640 1 --2- 172.21.15.5:0/603115709 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2f40cf3d0 0x7fb2f4160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:09.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.301+0000 7fb2f9de0640 1 -- 172.21.15.5:0/603115709 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2f40cf3d0 msgr2=0x7fb2f4160a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:09.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.301+0000 7fb2f9de0640 1 --2- 172.21.15.5:0/603115709 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2f40cf3d0 0x7fb2f4160a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:09.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.301+0000 7fb2f9de0640 1 -- 172.21.15.5:0/603115709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2f40cd680 msgr2=0x7fb2f415dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:09.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.301+0000 7fb2f9de0640 1 --2- 172.21.15.5:0/603115709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2f40cd680 0x7fb2f415dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:09.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.301+0000 7fb2f9de0640 1 -- 172.21.15.5:0/603115709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f40c9a50 con 0x7fb2f406c6c0 2024-05-20T18:33:09.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.301+0000 7fb2f9de0640 1 --2- 172.21.15.5:0/603115709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2f406c6c0 0x7fb2f40b7d20 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000ef00 tx=0x7fb2f000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:09.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.301+0000 7fb2ea7fc640 1 -- 172.21.15.5:0/603115709 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f0018020 con 0x7fb2f406c6c0 2024-05-20T18:33:09.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.301+0000 7fb2ea7fc640 1 -- 172.21.15.5:0/603115709 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2f000bd90 con 0x7fb2f406c6c0 2024-05-20T18:33:09.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.301+0000 7fb2fc044640 1 -- 172.21.15.5:0/603115709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f40c9d30 con 0x7fb2f406c6c0 2024-05-20T18:33:09.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.301+0000 7fb2ea7fc640 1 -- 172.21.15.5:0/603115709 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f0014620 con 0x7fb2f406c6c0 2024-05-20T18:33:09.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.301+0000 7fb2fc044640 1 -- 172.21.15.5:0/603115709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f40ca2f0 con 0x7fb2f406c6c0 2024-05-20T18:33:09.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.302+0000 7fb2fc044640 1 -- 172.21.15.5:0/603115709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2f4075b00 con 0x7fb2f406c6c0 2024-05-20T18:33:09.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.303+0000 7fb2ea7fc640 1 -- 172.21.15.5:0/603115709 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb2f0005d10 con 0x7fb2f406c6c0 2024-05-20T18:33:09.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.303+0000 7fb2ea7fc640 1 --2- 172.21.15.5:0/603115709 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb2d806be60 0x7fb2d806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:09.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.303+0000 7fb2ea7fc640 1 -- 172.21.15.5:0/603115709 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5249+0+0 (secure 0 0 0) 0x7fb2f008b490 con 0x7fb2f406c6c0 2024-05-20T18:33:09.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.303+0000 7fb2f95df640 1 --2- 172.21.15.5:0/603115709 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb2d806be60 0x7fb2d806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:09.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.304+0000 7fb2f95df640 1 --2- 172.21.15.5:0/603115709 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb2d806be60 0x7fb2d806e370 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4034000 tx=0x7fb2e402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:09.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.308+0000 7fb2ea7fc640 1 -- 172.21.15.5:0/603115709 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb2f0022030 con 0x7fb2f406c6c0 2024-05-20T18:33:09.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.493+0000 7fb2fc044640 1 -- 172.21.15.5:0/603115709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb2f40b8940 con 0x7fb2f406c6c0 2024-05-20T18:33:09.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.493+0000 7fb2ea7fc640 1 -- 172.21.15.5:0/603115709 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) ==== 93+0+10 (secure 0 0 0) 0x7fb2f0059920 con 0x7fb2f406c6c0 2024-05-20T18:33:09.494 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:33:09.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.495+0000 7fb2fc044640 1 -- 172.21.15.5:0/603115709 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb2d806be60 msgr2=0x7fb2d806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:09.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.495+0000 7fb2fc044640 1 --2- 172.21.15.5:0/603115709 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb2d806be60 0x7fb2d806e370 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4034000 tx=0x7fb2e402e000 comp rx=0 tx=0).stop 2024-05-20T18:33:09.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.495+0000 7fb2fc044640 1 -- 172.21.15.5:0/603115709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2f406c6c0 msgr2=0x7fb2f40b7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:09.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.495+0000 7fb2fc044640 1 --2- 172.21.15.5:0/603115709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2f406c6c0 0x7fb2f40b7d20 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000ef00 tx=0x7fb2f000b480 comp rx=0 tx=0).stop 2024-05-20T18:33:09.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.495+0000 7fb2fc044640 1 -- 172.21.15.5:0/603115709 shutdown_connections 2024-05-20T18:33:09.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.495+0000 7fb2fc044640 1 --2- 172.21.15.5:0/603115709 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb2d806be60 0x7fb2d806e370 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:09.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.495+0000 7fb2fc044640 1 --2- 172.21.15.5:0/603115709 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2f40cf3d0 0x7fb2f4160a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:09.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.495+0000 7fb2fc044640 1 --2- 172.21.15.5:0/603115709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2f40cd680 0x7fb2f415dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:09.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.495+0000 7fb2fc044640 1 --2- 172.21.15.5:0/603115709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2f406c6c0 0x7fb2f40b7d20 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:09.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.495+0000 7fb2fc044640 1 -- 172.21.15.5:0/603115709 >> 172.21.15.5:0/603115709 conn(0x7fb2f40b44d0 msgr2=0x7fb2f40b34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:09.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.496+0000 7fb2fc044640 1 -- 172.21.15.5:0/603115709 shutdown_connections 2024-05-20T18:33:09.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.496+0000 7fb2fc044640 1 -- 172.21.15.5:0/603115709 wait complete. 2024-05-20T18:33:09.508 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-20T18:33:09.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.622+0000 7f109bdea640 1 Processor -- start 2024-05-20T18:33:09.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.623+0000 7f109bdea640 1 -- start start 2024-05-20T18:33:09.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.624+0000 7f109bdea640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f109415f220 0x7f10940d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:09.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.624+0000 7f109bdea640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f10940cfae0 0x7f10940ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:09.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.624+0000 7f109bdea640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10940cda20 con 0x7f10940d0e00 2024-05-20T18:33:09.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.624+0000 7f109bdea640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f10940cdd00 con 0x7f109415f220 2024-05-20T18:33:09.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.624+0000 7f109bdea640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f10940cdfe0 con 0x7f10940cfae0 2024-05-20T18:33:09.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.624+0000 7f1099b86640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f109415f220 0x7f10940d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:09.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.624+0000 7f1099385640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f10940d0e00 0x7f10940cf3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38190/0 (socket says 172.21.15.5:38190) 2024-05-20T18:33:09.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.624+0000 7f1099385640 1 -- 172.21.15.5:0/302937804 learned_addr learned my addr 172.21.15.5:0/302937804 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:33:09.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.625+0000 7f1099b86640 1 -- 172.21.15.5:0/302937804 >> v2:172.21.15.63:3300/0 conn(0x7f10940cfae0 msgr2=0x7f10940ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:09.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.625+0000 7f109a387640 1 --2- 172.21.15.5:0/302937804 >> v2:172.21.15.63:3300/0 conn(0x7f10940cfae0 0x7f10940ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:09.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.625+0000 7f1099b86640 1 --2- 172.21.15.5:0/302937804 >> v2:172.21.15.63:3300/0 conn(0x7f10940cfae0 0x7f10940ce7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:09.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.625+0000 7f1099b86640 1 -- 172.21.15.5:0/302937804 >> v1:172.21.15.5:6789/0 conn(0x7f10940d0e00 legacy=0x7f10940cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:09.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.625+0000 7f1098b84640 1 -- 172.21.15.5:0/302937804 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 44838813 0 0) 0x7f10940cda20 con 0x7f10940d0e00 2024-05-20T18:33:09.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.625+0000 7f1099b86640 1 -- 172.21.15.5:0/302937804 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10940ce2c0 con 0x7f109415f220 2024-05-20T18:33:09.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.625+0000 7f1099b86640 1 --2- 172.21.15.5:0/302937804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f109415f220 0x7f10940d08c0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f109406e3e0 tx=0x7f109002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36bd82f54ccc36ea server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:09.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.626+0000 7f1098b84640 1 -- 172.21.15.5:0/302937804 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1090037070 con 0x7f109415f220 2024-05-20T18:33:09.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.626+0000 7f1098b84640 1 -- 172.21.15.5:0/302937804 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1090032440 con 0x7f109415f220 2024-05-20T18:33:09.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.626+0000 7f1098b84640 1 -- 172.21.15.5:0/302937804 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109002ae60 con 0x7f109415f220 2024-05-20T18:33:09.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.626+0000 7f109bdea640 1 -- 172.21.15.5:0/302937804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f109415f220 msgr2=0x7f10940d08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:09.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.626+0000 7f109bdea640 1 --2- 172.21.15.5:0/302937804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f109415f220 0x7f10940d08c0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f109406e3e0 tx=0x7f109002a110 comp rx=0 tx=0).stop 2024-05-20T18:33:09.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.626+0000 7f109bdea640 1 -- 172.21.15.5:0/302937804 shutdown_connections 2024-05-20T18:33:09.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.626+0000 7f109bdea640 1 --2- 172.21.15.5:0/302937804 >> v2:172.21.15.63:3300/0 conn(0x7f10940cfae0 0x7f10940ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:09.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.626+0000 7f109bdea640 1 --2- 172.21.15.5:0/302937804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f109415f220 0x7f10940d08c0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:09.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.626+0000 7f109bdea640 1 -- 172.21.15.5:0/302937804 >> 172.21.15.5:0/302937804 conn(0x7f10940b44d0 msgr2=0x7f10940b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:09.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.626+0000 7f109bdea640 1 -- 172.21.15.5:0/302937804 shutdown_connections 2024-05-20T18:33:09.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.626+0000 7f109bdea640 1 -- 172.21.15.5:0/302937804 wait complete. 2024-05-20T18:33:09.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.628+0000 7f109bdea640 1 Processor -- start 2024-05-20T18:33:09.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.628+0000 7f109bdea640 1 -- start start 2024-05-20T18:33:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.629+0000 7f109bdea640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f10940d0e00 0x7f109414fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.629+0000 7f109bdea640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f109415f220 0x7f109415c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.629+0000 7f109bdea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10940b61f0 0x7f109415cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.629+0000 7f109bdea640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f109415f7b0 con 0x7f10940b61f0 2024-05-20T18:33:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.629+0000 7f109bdea640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f109415f630 con 0x7f10940d0e00 2024-05-20T18:33:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.629+0000 7f109bdea640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f10940ced20 con 0x7f109415f220 2024-05-20T18:33:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.630+0000 7f109a387640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10940b61f0 0x7f109415cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.630+0000 7f109a387640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10940b61f0 0x7f109415cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56640/0 (socket says 172.21.15.5:56640) 2024-05-20T18:33:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.630+0000 7f109a387640 1 -- 172.21.15.5:0/993949700 learned_addr learned my addr 172.21.15.5:0/993949700 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.630+0000 7f1099b86640 1 --2- 172.21.15.5:0/993949700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f10940d0e00 0x7f109414fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.630+0000 7f1099385640 1 --2- 172.21.15.5:0/993949700 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f109415f220 0x7f109415c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.630+0000 7f109a387640 1 -- 172.21.15.5:0/993949700 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f109415f220 msgr2=0x7f109415c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.630+0000 7f109a387640 1 --2- 172.21.15.5:0/993949700 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f109415f220 0x7f109415c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.630+0000 7f109a387640 1 -- 172.21.15.5:0/993949700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f10940d0e00 msgr2=0x7f109414fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.630+0000 7f109a387640 1 --2- 172.21.15.5:0/993949700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f10940d0e00 0x7f109414fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.630+0000 7f109a387640 1 -- 172.21.15.5:0/993949700 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f109415c360 con 0x7f10940b61f0 2024-05-20T18:33:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.630+0000 7f1099385640 1 --2- 172.21.15.5:0/993949700 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f109415f220 0x7f109415c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:09.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.630+0000 7f109a387640 1 --2- 172.21.15.5:0/993949700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10940b61f0 0x7f109415cde0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f108c00cc30 tx=0x7f108c00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:09.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.630+0000 7f108a7fc640 1 -- 172.21.15.5:0/993949700 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c00ce40 con 0x7f10940b61f0 2024-05-20T18:33:09.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.630+0000 7f108a7fc640 1 -- 172.21.15.5:0/993949700 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f108c003990 con 0x7f10940b61f0 2024-05-20T18:33:09.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.630+0000 7f109bdea640 1 -- 172.21.15.5:0/993949700 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10940b5710 con 0x7f10940b61f0 2024-05-20T18:33:09.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.630+0000 7f108a7fc640 1 -- 172.21.15.5:0/993949700 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c003d40 con 0x7f10940b61f0 2024-05-20T18:33:09.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.630+0000 7f109bdea640 1 -- 172.21.15.5:0/993949700 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10940b5cd0 con 0x7f10940b61f0 2024-05-20T18:33:09.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.631+0000 7f109bdea640 1 -- 172.21.15.5:0/993949700 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1094075b00 con 0x7f10940b61f0 2024-05-20T18:33:09.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.632+0000 7f108a7fc640 1 -- 172.21.15.5:0/993949700 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f108c003fa0 con 0x7f10940b61f0 2024-05-20T18:33:09.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.633+0000 7f108a7fc640 1 --2- 172.21.15.5:0/993949700 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f107806be30 0x7f107806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:09.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.633+0000 7f1099b86640 1 --2- 172.21.15.5:0/993949700 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f107806be30 0x7f107806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:09.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.633+0000 7f108a7fc640 1 -- 172.21.15.5:0/993949700 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5249+0+0 (secure 0 0 0) 0x7f108c08b090 con 0x7f10940b61f0 2024-05-20T18:33:09.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.633+0000 7f1099b86640 1 --2- 172.21.15.5:0/993949700 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f107806be30 0x7f107806e340 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1090034000 tx=0x7f109002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:09.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.637+0000 7f108a7fc640 1 -- 172.21.15.5:0/993949700 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f108c014030 con 0x7f10940b61f0 2024-05-20T18:33:09.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.822+0000 7f109bdea640 1 -- 172.21.15.5:0/993949700 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f1094158db0 con 0x7f10940b61f0 2024-05-20T18:33:09.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.822+0000 7f108a7fc640 1 -- 172.21.15.5:0/993949700 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v86) ==== 84+0+65 (secure 0 0 0) 0x7f108c0595b0 con 0x7f10940b61f0 2024-05-20T18:33:09.823 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:33:09.824 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-05-20T18:33:09.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:33:09.824 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-05-20T18:33:09.824 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-05-20T18:33:09.824 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:33:09.824 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-05-20T18:33:09.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.824+0000 7f109bdea640 1 -- 172.21.15.5:0/993949700 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f107806be30 msgr2=0x7f107806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:09.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.824+0000 7f109bdea640 1 --2- 172.21.15.5:0/993949700 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f107806be30 0x7f107806e340 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1090034000 tx=0x7f109002e000 comp rx=0 tx=0).stop 2024-05-20T18:33:09.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.824+0000 7f109bdea640 1 -- 172.21.15.5:0/993949700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10940b61f0 msgr2=0x7f109415cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:09.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.824+0000 7f109bdea640 1 --2- 172.21.15.5:0/993949700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10940b61f0 0x7f109415cde0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f108c00cc30 tx=0x7f108c00a5c0 comp rx=0 tx=0).stop 2024-05-20T18:33:09.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.824+0000 7f109bdea640 1 -- 172.21.15.5:0/993949700 shutdown_connections 2024-05-20T18:33:09.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.824+0000 7f109bdea640 1 --2- 172.21.15.5:0/993949700 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f107806be30 0x7f107806e340 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:09.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.824+0000 7f109bdea640 1 --2- 172.21.15.5:0/993949700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10940b61f0 0x7f109415cde0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:09.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.824+0000 7f109bdea640 1 --2- 172.21.15.5:0/993949700 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f109415f220 0x7f109415c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:09.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.824+0000 7f109bdea640 1 --2- 172.21.15.5:0/993949700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f10940d0e00 0x7f109414fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:09.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.825+0000 7f109bdea640 1 -- 172.21.15.5:0/993949700 >> 172.21.15.5:0/993949700 conn(0x7f10940b44d0 msgr2=0x7f10940b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:09.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.825+0000 7f109bdea640 1 -- 172.21.15.5:0/993949700 shutdown_connections 2024-05-20T18:33:09.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.825+0000 7f109bdea640 1 -- 172.21.15.5:0/993949700 wait complete. 2024-05-20T18:33:09.836 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-20T18:33:09.837 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-20T18:33:09.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.954+0000 7fe904110640 1 Processor -- start 2024-05-20T18:33:09.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.955+0000 7fe904110640 1 -- start start 2024-05-20T18:33:09.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.956+0000 7fe904110640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe8fc0d2330 0x7fe8fc06d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:09.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.956+0000 7fe904110640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe8fc06d660 0x7fe8fc0cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:09.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.956+0000 7fe904110640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8fc0d2740 con 0x7fe8fc0cdf70 2024-05-20T18:33:09.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.956+0000 7fe904110640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe8fc09c3b0 con 0x7fe8fc06d660 2024-05-20T18:33:09.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.956+0000 7fe904110640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe8fc06dda0 con 0x7fe8fc0d2330 2024-05-20T18:33:09.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.956+0000 7fe9016ab640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe8fc06d660 0x7fe8fc0cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:09.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.956+0000 7fe9016ab640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe8fc06d660 0x7fe8fc0cda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56650/0 (socket says 172.21.15.5:56650) 2024-05-20T18:33:09.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.956+0000 7fe9016ab640 1 -- 172.21.15.5:0/4022027008 learned_addr learned my addr 172.21.15.5:0/4022027008 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:09.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.956+0000 7fe901eac640 1 --2- 172.21.15.5:0/4022027008 >> v2:172.21.15.63:3300/0 conn(0x7fe8fc0d2330 0x7fe8fc06d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:09.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.956+0000 7fe9016ab640 1 -- 172.21.15.5:0/4022027008 >> v2:172.21.15.63:3300/0 conn(0x7fe8fc0d2330 msgr2=0x7fe8fc06d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:09.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.956+0000 7fe9016ab640 1 --2- 172.21.15.5:0/4022027008 >> v2:172.21.15.63:3300/0 conn(0x7fe8fc0d2330 0x7fe8fc06d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:09.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.956+0000 7fe9016ab640 1 -- 172.21.15.5:0/4022027008 >> v1:172.21.15.63:6789/0 conn(0x7fe8fc0cdf70 legacy=0x7fe8fc163770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:09.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.956+0000 7fe9016ab640 1 -- 172.21.15.5:0/4022027008 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8fc0b81a0 con 0x7fe8fc06d660 2024-05-20T18:33:09.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.957+0000 7fe9016ab640 1 --2- 172.21.15.5:0/4022027008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8fc06d660 0x7fe8fc0cda30 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc06c520 tx=0x7fe8ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd94ca98f0b2b179 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:09.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.957+0000 7fe900eaa640 1 -- 172.21.15.5:0/4022027008 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec037070 con 0x7fe8fc06d660 2024-05-20T18:33:09.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.957+0000 7fe900eaa640 1 -- 172.21.15.5:0/4022027008 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8ec032440 con 0x7fe8fc06d660 2024-05-20T18:33:09.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.957+0000 7fe900eaa640 1 -- 172.21.15.5:0/4022027008 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec032810 con 0x7fe8fc06d660 2024-05-20T18:33:09.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.958+0000 7fe904110640 1 -- 172.21.15.5:0/4022027008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8fc06d660 msgr2=0x7fe8fc0cda30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:09.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.958+0000 7fe904110640 1 --2- 172.21.15.5:0/4022027008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8fc06d660 0x7fe8fc0cda30 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc06c520 tx=0x7fe8ec02a270 comp rx=0 tx=0).stop 2024-05-20T18:33:09.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.958+0000 7fe904110640 1 -- 172.21.15.5:0/4022027008 shutdown_connections 2024-05-20T18:33:09.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.958+0000 7fe904110640 1 --2- 172.21.15.5:0/4022027008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8fc06d660 0x7fe8fc0cda30 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:09.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.958+0000 7fe904110640 1 --2- 172.21.15.5:0/4022027008 >> v2:172.21.15.63:3300/0 conn(0x7fe8fc0d2330 0x7fe8fc06d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:09.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.958+0000 7fe904110640 1 -- 172.21.15.5:0/4022027008 >> 172.21.15.5:0/4022027008 conn(0x7fe8fc0cf450 msgr2=0x7fe8fc0cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:09.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.958+0000 7fe904110640 1 -- 172.21.15.5:0/4022027008 shutdown_connections 2024-05-20T18:33:09.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.958+0000 7fe904110640 1 -- 172.21.15.5:0/4022027008 wait complete. 2024-05-20T18:33:09.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.960+0000 7fe904110640 1 Processor -- start 2024-05-20T18:33:09.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.960+0000 7fe904110640 1 -- start start 2024-05-20T18:33:09.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.961+0000 7fe904110640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8fc06d660 0x7fe8fc15be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:09.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.961+0000 7fe904110640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8fc0cdf70 0x7fe8fc14ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:09.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.961+0000 7fe904110640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8fc0d2330 0x7fe8fc1595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:09.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.961+0000 7fe904110640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc0d3800 con 0x7fe8fc0d2330 2024-05-20T18:33:09.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.961+0000 7fe904110640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc06da70 con 0x7fe8fc0cdf70 2024-05-20T18:33:09.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.961+0000 7fe904110640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe8fc0d3980 con 0x7fe8fc06d660 2024-05-20T18:33:09.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.961+0000 7fe9026ad640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8fc0d2330 0x7fe8fc1595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:09.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.962+0000 7fe9026ad640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8fc0d2330 0x7fe8fc1595c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56664/0 (socket says 172.21.15.5:56664) 2024-05-20T18:33:09.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.962+0000 7fe9026ad640 1 -- 172.21.15.5:0/2506715431 learned_addr learned my addr 172.21.15.5:0/2506715431 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:09.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.962+0000 7fe9016ab640 1 --2- 172.21.15.5:0/2506715431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8fc0cdf70 0x7fe8fc14ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:09.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.962+0000 7fe901eac640 1 --2- 172.21.15.5:0/2506715431 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8fc06d660 0x7fe8fc15be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:09.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.962+0000 7fe9026ad640 1 -- 172.21.15.5:0/2506715431 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8fc06d660 msgr2=0x7fe8fc15be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:09.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.962+0000 7fe9026ad640 1 --2- 172.21.15.5:0/2506715431 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8fc06d660 0x7fe8fc15be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:09.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.962+0000 7fe9026ad640 1 -- 172.21.15.5:0/2506715431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8fc0cdf70 msgr2=0x7fe8fc14ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:09.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.962+0000 7fe9026ad640 1 --2- 172.21.15.5:0/2506715431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8fc0cdf70 0x7fe8fc14ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:09.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.962+0000 7fe9026ad640 1 -- 172.21.15.5:0/2506715431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8fc14f370 con 0x7fe8fc0d2330 2024-05-20T18:33:09.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.962+0000 7fe9016ab640 1 --2- 172.21.15.5:0/2506715431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8fc0cdf70 0x7fe8fc14ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:33:09.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.962+0000 7fe9026ad640 1 --2- 172.21.15.5:0/2506715431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8fc0d2330 0x7fe8fc1595c0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400c760 tx=0x7fe8f400cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:09.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.962+0000 7fe8f27fc640 1 -- 172.21.15.5:0/2506715431 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4016020 con 0x7fe8fc0d2330 2024-05-20T18:33:09.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.962+0000 7fe8f27fc640 1 -- 172.21.15.5:0/2506715431 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8f4006d50 con 0x7fe8fc0d2330 2024-05-20T18:33:09.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.962+0000 7fe904110640 1 -- 172.21.15.5:0/2506715431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8fc15c6d0 con 0x7fe8fc0d2330 2024-05-20T18:33:09.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.962+0000 7fe8f27fc640 1 -- 172.21.15.5:0/2506715431 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4003bb0 con 0x7fe8fc0d2330 2024-05-20T18:33:09.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.962+0000 7fe904110640 1 -- 172.21.15.5:0/2506715431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8fc15cc90 con 0x7fe8fc0d2330 2024-05-20T18:33:09.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.963+0000 7fe904110640 1 -- 172.21.15.5:0/2506715431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8fc0690b0 con 0x7fe8fc0d2330 2024-05-20T18:33:09.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.964+0000 7fe8f27fc640 1 -- 172.21.15.5:0/2506715431 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe8f40050d0 con 0x7fe8fc0d2330 2024-05-20T18:33:09.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.964+0000 7fe8f27fc640 1 --2- 172.21.15.5:0/2506715431 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe8e006be60 0x7fe8e006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:09.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.965+0000 7fe8f27fc640 1 -- 172.21.15.5:0/2506715431 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5249+0+0 (secure 0 0 0) 0x7fe8f408a940 con 0x7fe8fc0d2330 2024-05-20T18:33:09.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.965+0000 7fe901eac640 1 --2- 172.21.15.5:0/2506715431 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe8e006be60 0x7fe8e006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:09.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.965+0000 7fe901eac640 1 --2- 172.21.15.5:0/2506715431 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe8e006be60 0x7fe8e006e370 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8009960 tx=0x7fe8f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:09.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:09.969+0000 7fe8f27fc640 1 -- 172.21.15.5:0/2506715431 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe8f4020020 con 0x7fe8fc0d2330 2024-05-20T18:33:10.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:10.153+0000 7fe904110640 1 -- 172.21.15.5:0/2506715431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe8fc0d0220 con 0x7fe8fc0d2330 2024-05-20T18:33:11.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.065+0000 7fe8f27fc640 1 -- 172.21.15.5:0/2506715431 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v87) ==== 138+0+0 (secure 0 0 0) 0x7fe8f4059420 con 0x7fe8fc0d2330 2024-05-20T18:33:11.065 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-05-20T18:33:11.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.068+0000 7fe904110640 1 -- 172.21.15.5:0/2506715431 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe8e006be60 msgr2=0x7fe8e006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:11.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.068+0000 7fe904110640 1 --2- 172.21.15.5:0/2506715431 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe8e006be60 0x7fe8e006e370 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8009960 tx=0x7fe8f8008040 comp rx=0 tx=0).stop 2024-05-20T18:33:11.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.068+0000 7fe904110640 1 -- 172.21.15.5:0/2506715431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8fc0d2330 msgr2=0x7fe8fc1595c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:11.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.068+0000 7fe904110640 1 --2- 172.21.15.5:0/2506715431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8fc0d2330 0x7fe8fc1595c0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400c760 tx=0x7fe8f400cb60 comp rx=0 tx=0).stop 2024-05-20T18:33:11.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.068+0000 7fe904110640 1 -- 172.21.15.5:0/2506715431 shutdown_connections 2024-05-20T18:33:11.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.068+0000 7fe904110640 1 --2- 172.21.15.5:0/2506715431 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe8e006be60 0x7fe8e006e370 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:11.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.068+0000 7fe904110640 1 --2- 172.21.15.5:0/2506715431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8fc0d2330 0x7fe8fc1595c0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:11.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.068+0000 7fe904110640 1 --2- 172.21.15.5:0/2506715431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8fc0cdf70 0x7fe8fc14ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:11.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.068+0000 7fe904110640 1 --2- 172.21.15.5:0/2506715431 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8fc06d660 0x7fe8fc15be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:11.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.068+0000 7fe904110640 1 -- 172.21.15.5:0/2506715431 >> 172.21.15.5:0/2506715431 conn(0x7fe8fc0cf450 msgr2=0x7fe8fc162e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:11.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.068+0000 7fe904110640 1 -- 172.21.15.5:0/2506715431 shutdown_connections 2024-05-20T18:33:11.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.068+0000 7fe904110640 1 -- 172.21.15.5:0/2506715431 wait complete. 2024-05-20T18:33:11.086 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-20T18:33:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.214+0000 7f935623d640 1 Processor -- start 2024-05-20T18:33:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.214+0000 7f935623d640 1 -- start start 2024-05-20T18:33:11.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.215+0000 7f935623d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f93500d46e0 0x7f93500b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:11.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.215+0000 7f935623d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f935008c4c0 0x7f93500b8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:11.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.216+0000 7f935623d640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93500d2890 con 0x7f935015f320 2024-05-20T18:33:11.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.216+0000 7f935623d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f93500d2b70 con 0x7f93500d46e0 2024-05-20T18:33:11.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.216+0000 7f935623d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f93500d2e50 con 0x7f935008c4c0 2024-05-20T18:33:11.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.216+0000 7f934f7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f93500d46e0 0x7f93500b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:11.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.216+0000 7f934f7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f93500d46e0 0x7f93500b3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56676/0 (socket says 172.21.15.5:56676) 2024-05-20T18:33:11.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.216+0000 7f934f7fe640 1 -- 172.21.15.5:0/743018962 learned_addr learned my addr 172.21.15.5:0/743018962 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:11.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.216+0000 7f934ffff640 1 --2- 172.21.15.5:0/743018962 >> v2:172.21.15.63:3300/0 conn(0x7f935008c4c0 0x7f93500b8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:11.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.216+0000 7f934f7fe640 1 -- 172.21.15.5:0/743018962 >> v2:172.21.15.63:3300/0 conn(0x7f935008c4c0 msgr2=0x7f93500b8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:11.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.216+0000 7f934f7fe640 1 --2- 172.21.15.5:0/743018962 >> v2:172.21.15.63:3300/0 conn(0x7f935008c4c0 0x7f93500b8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:11.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.216+0000 7f934f7fe640 1 -- 172.21.15.5:0/743018962 >> v1:172.21.15.63:6789/0 conn(0x7f935015f320 legacy=0x7f935008bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:11.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.216+0000 7f934f7fe640 1 -- 172.21.15.5:0/743018962 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93500d3130 con 0x7f93500d46e0 2024-05-20T18:33:11.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.217+0000 7f934f7fe640 1 --2- 172.21.15.5:0/743018962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93500d46e0 0x7f93500b3cd0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f93500bad10 tx=0x7f934002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff716f82e2de1d06 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:11.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.217+0000 7f934effd640 1 -- 172.21.15.5:0/743018962 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340037070 con 0x7f93500d46e0 2024-05-20T18:33:11.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.217+0000 7f934effd640 1 -- 172.21.15.5:0/743018962 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9340032440 con 0x7f93500d46e0 2024-05-20T18:33:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.217+0000 7f934effd640 1 -- 172.21.15.5:0/743018962 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340032810 con 0x7f93500d46e0 2024-05-20T18:33:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.218+0000 7f935623d640 1 -- 172.21.15.5:0/743018962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93500d46e0 msgr2=0x7f93500b3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.218+0000 7f935623d640 1 --2- 172.21.15.5:0/743018962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93500d46e0 0x7f93500b3cd0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f93500bad10 tx=0x7f934002a230 comp rx=0 tx=0).stop 2024-05-20T18:33:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.218+0000 7f935623d640 1 -- 172.21.15.5:0/743018962 shutdown_connections 2024-05-20T18:33:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.218+0000 7f935623d640 1 --2- 172.21.15.5:0/743018962 >> v2:172.21.15.63:3300/0 conn(0x7f935008c4c0 0x7f93500b8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.218+0000 7f935623d640 1 --2- 172.21.15.5:0/743018962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93500d46e0 0x7f93500b3cd0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.218+0000 7f935623d640 1 -- 172.21.15.5:0/743018962 >> 172.21.15.5:0/743018962 conn(0x7f93500d1cd0 msgr2=0x7f93500d4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.218+0000 7f935623d640 1 -- 172.21.15.5:0/743018962 shutdown_connections 2024-05-20T18:33:11.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.218+0000 7f935623d640 1 -- 172.21.15.5:0/743018962 wait complete. 2024-05-20T18:33:11.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.220+0000 7f935623d640 1 Processor -- start 2024-05-20T18:33:11.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.220+0000 7f935623d640 1 -- start start 2024-05-20T18:33:11.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.221+0000 7f935623d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93500d46e0 0x7f93501596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:11.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.221+0000 7f935623d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f935015f320 0x7f93500cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:11.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.221+0000 7f935623d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f935015f8b0 0x7f9350155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:11.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.221+0000 7f935623d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f935015f730 con 0x7f93500d46e0 2024-05-20T18:33:11.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.221+0000 7f935623d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f935008ddb0 con 0x7f935015f8b0 2024-05-20T18:33:11.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.221+0000 7f935623d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f93500d3930 con 0x7f935015f320 2024-05-20T18:33:11.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.221+0000 7f934f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93500d46e0 0x7f93501596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:11.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.221+0000 7f934f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93500d46e0 0x7f93501596b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56688/0 (socket says 172.21.15.5:56688) 2024-05-20T18:33:11.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.221+0000 7f934f7fe640 1 -- 172.21.15.5:0/863127724 learned_addr learned my addr 172.21.15.5:0/863127724 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:11.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.221+0000 7f934ffff640 1 --2- 172.21.15.5:0/863127724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f935015f8b0 0x7f9350155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:11.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.221+0000 7f9347fff640 1 --2- 172.21.15.5:0/863127724 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f935015f320 0x7f93500cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:11.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.222+0000 7f934f7fe640 1 -- 172.21.15.5:0/863127724 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f935015f320 msgr2=0x7f93500cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:11.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.222+0000 7f934f7fe640 1 --2- 172.21.15.5:0/863127724 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f935015f320 0x7f93500cf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:11.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.222+0000 7f934f7fe640 1 -- 172.21.15.5:0/863127724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f935015f8b0 msgr2=0x7f9350155bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:11.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.222+0000 7f934f7fe640 1 --2- 172.21.15.5:0/863127724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f935015f8b0 0x7f9350155bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:11.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.222+0000 7f934f7fe640 1 -- 172.21.15.5:0/863127724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9350157150 con 0x7f93500d46e0 2024-05-20T18:33:11.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.222+0000 7f934ffff640 1 --2- 172.21.15.5:0/863127724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f935015f8b0 0x7f9350155bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:11.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.222+0000 7f934f7fe640 1 --2- 172.21.15.5:0/863127724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93500d46e0 0x7f93501596b0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f9340000c00 tx=0x7f93400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:11.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.222+0000 7f934cff9640 1 -- 172.21.15.5:0/863127724 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340041020 con 0x7f93500d46e0 2024-05-20T18:33:11.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.222+0000 7f934cff9640 1 -- 172.21.15.5:0/863127724 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93400306c0 con 0x7f93500d46e0 2024-05-20T18:33:11.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.222+0000 7f935623d640 1 -- 172.21.15.5:0/863127724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9350157430 con 0x7f93500d46e0 2024-05-20T18:33:11.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.222+0000 7f934cff9640 1 -- 172.21.15.5:0/863127724 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340037040 con 0x7f93500d46e0 2024-05-20T18:33:11.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.222+0000 7f935623d640 1 -- 172.21.15.5:0/863127724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93501579f0 con 0x7f93500d46e0 2024-05-20T18:33:11.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.223+0000 7f935623d640 1 -- 172.21.15.5:0/863127724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93500690b0 con 0x7f93500d46e0 2024-05-20T18:33:11.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.224+0000 7f934cff9640 1 -- 172.21.15.5:0/863127724 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9340044020 con 0x7f93500d46e0 2024-05-20T18:33:11.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.224+0000 7f934cff9640 1 --2- 172.21.15.5:0/863127724 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f933006be30 0x7f933006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:11.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.225+0000 7f9347fff640 1 --2- 172.21.15.5:0/863127724 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f933006be30 0x7f933006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:11.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.225+0000 7f934cff9640 1 -- 172.21.15.5:0/863127724 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5619+0+0 (secure 0 0 0) 0x7f93400ab720 con 0x7f93500d46e0 2024-05-20T18:33:11.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.225+0000 7f9347fff640 1 --2- 172.21.15.5:0/863127724 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f933006be30 0x7f933006e340 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9338006fd0 tx=0x7f9338008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.229+0000 7f934cff9640 1 -- 172.21.15.5:0/863127724 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f934004b030 con 0x7f93500d46e0 2024-05-20T18:33:11.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:11.420+0000 7f935623d640 1 -- 172.21.15.5:0/863127724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f935015ce30 con 0x7f93500d46e0 2024-05-20T18:33:11.462 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:33:11.465 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:33:12.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.067+0000 7f934cff9640 1 -- 172.21.15.5:0/863127724 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v88) ==== 170+0+0 (secure 0 0 0) 0x7f934007a090 con 0x7f93500d46e0 2024-05-20T18:33:12.068 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-05-20T18:33:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.069+0000 7f935623d640 1 -- 172.21.15.5:0/863127724 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f933006be30 msgr2=0x7f933006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.069+0000 7f935623d640 1 --2- 172.21.15.5:0/863127724 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f933006be30 0x7f933006e340 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9338006fd0 tx=0x7f9338008040 comp rx=0 tx=0).stop 2024-05-20T18:33:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.069+0000 7f935623d640 1 -- 172.21.15.5:0/863127724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93500d46e0 msgr2=0x7f93501596b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.069+0000 7f935623d640 1 --2- 172.21.15.5:0/863127724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93500d46e0 0x7f93501596b0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f9340000c00 tx=0x7f93400046c0 comp rx=0 tx=0).stop 2024-05-20T18:33:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.069+0000 7f935623d640 1 -- 172.21.15.5:0/863127724 shutdown_connections 2024-05-20T18:33:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.069+0000 7f935623d640 1 --2- 172.21.15.5:0/863127724 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f933006be30 0x7f933006e340 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.069+0000 7f935623d640 1 --2- 172.21.15.5:0/863127724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f935015f8b0 0x7f9350155bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.069+0000 7f935623d640 1 --2- 172.21.15.5:0/863127724 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f935015f320 0x7f93500cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.069+0000 7f935623d640 1 --2- 172.21.15.5:0/863127724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93500d46e0 0x7f93501596b0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.069+0000 7f935623d640 1 -- 172.21.15.5:0/863127724 >> 172.21.15.5:0/863127724 conn(0x7f93500d1cd0 msgr2=0x7f93500b5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.070+0000 7f935623d640 1 -- 172.21.15.5:0/863127724 shutdown_connections 2024-05-20T18:33:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.070+0000 7f935623d640 1 -- 172.21.15.5:0/863127724 wait complete. 2024-05-20T18:33:12.096 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-20T18:33:12.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.215+0000 7f3daf85a640 1 Processor -- start 2024-05-20T18:33:12.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.215+0000 7f3daf85a640 1 -- start start 2024-05-20T18:33:12.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.217+0000 7f3daf85a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3da80b00b0 0x7f3da80b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:12.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.217+0000 7f3daf85a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3da815fd80 0x7f3da806bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:12.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.217+0000 7f3daf85a640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3da806c700 con 0x7f3da806c2f0 2024-05-20T18:33:12.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.217+0000 7f3daf85a640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f3da815c690 con 0x7f3da80b00b0 2024-05-20T18:33:12.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.217+0000 7f3daf85a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3da80b4860 con 0x7f3da815fd80 2024-05-20T18:33:12.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.217+0000 7f3dad5f6640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3da80b00b0 0x7f3da80b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:12.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.217+0000 7f3daddf7640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3da806c2f0 0x7f3da80ce6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38204/0 (socket says 172.21.15.5:38204) 2024-05-20T18:33:12.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.217+0000 7f3dad5f6640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3da80b00b0 0x7f3da80b4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56700/0 (socket says 172.21.15.5:56700) 2024-05-20T18:33:12.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.217+0000 7f3daddf7640 1 -- 172.21.15.5:0/40679296 learned_addr learned my addr 172.21.15.5:0/40679296 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:33:12.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.217+0000 7f3dacdf5640 1 --2- 172.21.15.5:0/40679296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3da815fd80 0x7f3da806bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:12.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.218+0000 7f3dad5f6640 1 -- 172.21.15.5:0/40679296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3da815fd80 msgr2=0x7f3da806bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:12.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.218+0000 7f3dad5f6640 1 --2- 172.21.15.5:0/40679296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3da815fd80 0x7f3da806bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:12.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.218+0000 7f3dad5f6640 1 -- 172.21.15.5:0/40679296 >> v1:172.21.15.5:6789/0 conn(0x7f3da806c2f0 legacy=0x7f3da80ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:12.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.218+0000 7f3d9bfff640 1 -- 172.21.15.5:0/40679296 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2635353098 0 0) 0x7f3da0001490 con 0x7f3da806c2f0 2024-05-20T18:33:12.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.218+0000 7f3dad5f6640 1 -- 172.21.15.5:0/40679296 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da80b52e0 con 0x7f3da80b00b0 2024-05-20T18:33:12.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.218+0000 7f3dad5f6640 1 --2- 172.21.15.5:0/40679296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da80b00b0 0x7f3da80b4c00 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f3da815c810 tx=0x7f3da402b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b22654cf1ca19ee server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:12.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.218+0000 7f3d9bfff640 1 -- 172.21.15.5:0/40679296 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da400b040 con 0x7f3da80b00b0 2024-05-20T18:33:12.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.219+0000 7f3d9bfff640 1 -- 172.21.15.5:0/40679296 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3da4030710 con 0x7f3da80b00b0 2024-05-20T18:33:12.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.219+0000 7f3d9bfff640 1 -- 172.21.15.5:0/40679296 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da4030ae0 con 0x7f3da80b00b0 2024-05-20T18:33:12.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.219+0000 7f3daf85a640 1 -- 172.21.15.5:0/40679296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da80b00b0 msgr2=0x7f3da80b4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:12.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.219+0000 7f3daf85a640 1 --2- 172.21.15.5:0/40679296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da80b00b0 0x7f3da80b4c00 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f3da815c810 tx=0x7f3da402b4d0 comp rx=0 tx=0).stop 2024-05-20T18:33:12.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.219+0000 7f3daf85a640 1 -- 172.21.15.5:0/40679296 shutdown_connections 2024-05-20T18:33:12.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.219+0000 7f3daf85a640 1 --2- 172.21.15.5:0/40679296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3da815fd80 0x7f3da806bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:12.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.219+0000 7f3daf85a640 1 --2- 172.21.15.5:0/40679296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da80b00b0 0x7f3da80b4c00 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:12.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.219+0000 7f3daf85a640 1 -- 172.21.15.5:0/40679296 >> 172.21.15.5:0/40679296 conn(0x7f3da80cd2a0 msgr2=0x7f3da80cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:12.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.220+0000 7f3daf85a640 1 -- 172.21.15.5:0/40679296 shutdown_connections 2024-05-20T18:33:12.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.220+0000 7f3daf85a640 1 -- 172.21.15.5:0/40679296 wait complete. 2024-05-20T18:33:12.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.221+0000 7f3daf85a640 1 Processor -- start 2024-05-20T18:33:12.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.221+0000 7f3daf85a640 1 -- start start 2024-05-20T18:33:12.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.223+0000 7f3daf85a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3da806c2f0 0x7f3da81f3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:12.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.223+0000 7f3daf85a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da80b00b0 0x7f3da81533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:12.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.223+0000 7f3daf85a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3da815fd80 0x7f3da8155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:12.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.223+0000 7f3daf85a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3da8160190 con 0x7f3da80b00b0 2024-05-20T18:33:12.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.223+0000 7f3daf85a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3da815c320 con 0x7f3da815fd80 2024-05-20T18:33:12.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.223+0000 7f3daf85a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3da815bc30 con 0x7f3da806c2f0 2024-05-20T18:33:12.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.223+0000 7f3dacdf5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da80b00b0 0x7f3da81533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:12.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.223+0000 7f3dacdf5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da80b00b0 0x7f3da81533e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56716/0 (socket says 172.21.15.5:56716) 2024-05-20T18:33:12.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.223+0000 7f3dacdf5640 1 -- 172.21.15.5:0/4256898033 learned_addr learned my addr 172.21.15.5:0/4256898033 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:12.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.223+0000 7f3daddf7640 1 --2- 172.21.15.5:0/4256898033 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3da815fd80 0x7f3da8155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:12.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.223+0000 7f3dad5f6640 1 --2- 172.21.15.5:0/4256898033 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3da806c2f0 0x7f3da81f3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:12.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.223+0000 7f3dacdf5640 1 -- 172.21.15.5:0/4256898033 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3da806c2f0 msgr2=0x7f3da81f3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:12.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.223+0000 7f3dacdf5640 1 --2- 172.21.15.5:0/4256898033 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3da806c2f0 0x7f3da81f3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:12.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.223+0000 7f3dacdf5640 1 -- 172.21.15.5:0/4256898033 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3da815fd80 msgr2=0x7f3da8155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:12.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.223+0000 7f3dacdf5640 1 --2- 172.21.15.5:0/4256898033 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3da815fd80 0x7f3da8155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:12.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.223+0000 7f3dacdf5640 1 -- 172.21.15.5:0/4256898033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da815d290 con 0x7f3da80b00b0 2024-05-20T18:33:12.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.223+0000 7f3dacdf5640 1 --2- 172.21.15.5:0/4256898033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da80b00b0 0x7f3da81533e0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00e8d0 tx=0x7f3d9c00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:12.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.223+0000 7f3d99ffb640 1 -- 172.21.15.5:0/4256898033 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c017070 con 0x7f3da80b00b0 2024-05-20T18:33:12.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.224+0000 7f3d99ffb640 1 -- 172.21.15.5:0/4256898033 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d9c00c940 con 0x7f3da80b00b0 2024-05-20T18:33:12.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.224+0000 7f3d99ffb640 1 -- 172.21.15.5:0/4256898033 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c00cd30 con 0x7f3da80b00b0 2024-05-20T18:33:12.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.224+0000 7f3daf85a640 1 -- 172.21.15.5:0/4256898033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da815d570 con 0x7f3da80b00b0 2024-05-20T18:33:12.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.224+0000 7f3daf85a640 1 -- 172.21.15.5:0/4256898033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da815db30 con 0x7f3da80b00b0 2024-05-20T18:33:12.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.225+0000 7f3daf85a640 1 -- 172.21.15.5:0/4256898033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3da8075b00 con 0x7f3da80b00b0 2024-05-20T18:33:12.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.226+0000 7f3d99ffb640 1 -- 172.21.15.5:0/4256898033 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3d9c01a070 con 0x7f3da80b00b0 2024-05-20T18:33:12.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.227+0000 7f3d99ffb640 1 --2- 172.21.15.5:0/4256898033 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3d8c06be30 0x7f3d8c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:12.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.227+0000 7f3d99ffb640 1 -- 172.21.15.5:0/4256898033 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6001+0+0 (secure 0 0 0) 0x7f3d9c08b540 con 0x7f3da80b00b0 2024-05-20T18:33:12.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.227+0000 7f3dad5f6640 1 --2- 172.21.15.5:0/4256898033 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3d8c06be30 0x7f3d8c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:12.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.227+0000 7f3dad5f6640 1 --2- 172.21.15.5:0/4256898033 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3d8c06be30 0x7f3d8c06e340 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3da4035000 tx=0x7f3da4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:12.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.231+0000 7f3d99ffb640 1 -- 172.21.15.5:0/4256898033 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3d9c059ff0 con 0x7f3da80b00b0 2024-05-20T18:33:12.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:12.455+0000 7f3daf85a640 1 -- 172.21.15.5:0/4256898033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f3da8159a90 con 0x7f3da80b00b0 2024-05-20T18:33:13.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.090+0000 7f3d99ffb640 1 -- 172.21.15.5:0/4256898033 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v56) ==== 174+0+0 (secure 0 0 0) 0x7f3d9c0598f0 con 0x7f3da80b00b0 2024-05-20T18:33:13.094 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 13 and data pool 14 2024-05-20T18:33:13.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.101+0000 7f3daf85a640 1 -- 172.21.15.5:0/4256898033 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3d8c06be30 msgr2=0x7f3d8c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:13.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.101+0000 7f3daf85a640 1 --2- 172.21.15.5:0/4256898033 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3d8c06be30 0x7f3d8c06e340 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3da4035000 tx=0x7f3da4008040 comp rx=0 tx=0).stop 2024-05-20T18:33:13.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.101+0000 7f3daf85a640 1 -- 172.21.15.5:0/4256898033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da80b00b0 msgr2=0x7f3da81533e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:13.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.101+0000 7f3daf85a640 1 --2- 172.21.15.5:0/4256898033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da80b00b0 0x7f3da81533e0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00e8d0 tx=0x7f3d9c00ed90 comp rx=0 tx=0).stop 2024-05-20T18:33:13.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.105+0000 7f3daf85a640 1 -- 172.21.15.5:0/4256898033 shutdown_connections 2024-05-20T18:33:13.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.105+0000 7f3daf85a640 1 --2- 172.21.15.5:0/4256898033 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3d8c06be30 0x7f3d8c06e340 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:13.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.105+0000 7f3daf85a640 1 --2- 172.21.15.5:0/4256898033 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3da815fd80 0x7f3da8155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:13.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.105+0000 7f3daf85a640 1 --2- 172.21.15.5:0/4256898033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3da80b00b0 0x7f3da81533e0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:13.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.105+0000 7f3daf85a640 1 --2- 172.21.15.5:0/4256898033 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3da806c2f0 0x7f3da81f3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:13.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.105+0000 7f3daf85a640 1 -- 172.21.15.5:0/4256898033 >> 172.21.15.5:0/4256898033 conn(0x7f3da80cd2a0 msgr2=0x7f3da8159fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:13.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.107+0000 7f3daf85a640 1 -- 172.21.15.5:0/4256898033 shutdown_connections 2024-05-20T18:33:13.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.107+0000 7f3daf85a640 1 -- 172.21.15.5:0/4256898033 wait complete. 2024-05-20T18:33:13.129 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:33:13.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.247+0000 7f366bbc3640 1 Processor -- start 2024-05-20T18:33:13.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.247+0000 7f366bbc3640 1 -- start start 2024-05-20T18:33:13.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.249+0000 7f366bbc3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f36641576d0 0x7f366406c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:13.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.249+0000 7f366bbc3640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3664157ae0 con 0x7f36640cedd0 2024-05-20T18:33:13.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.249+0000 7f366bbc3640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f366406b900 con 0x7f366415d5f0 2024-05-20T18:33:13.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.249+0000 7f366bbc3640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f366406cee0 con 0x7f36641576d0 2024-05-20T18:33:13.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.249+0000 7f366915e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f36641576d0 0x7f366406c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:13.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.249+0000 7f366a160640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f36640cedd0 0x7f366406e3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38210/0 (socket says 172.21.15.5:38210) 2024-05-20T18:33:13.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.249+0000 7f366915e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f36641576d0 0x7f366406c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56732/0 (socket says 172.21.15.5:56732) 2024-05-20T18:33:13.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.249+0000 7f366a160640 1 -- 172.21.15.5:0/3306376765 learned_addr learned my addr 172.21.15.5:0/3306376765 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:33:13.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.249+0000 7f366915e640 1 -- 172.21.15.5:0/3306376765 >> v1:172.21.15.63:6789/0 conn(0x7f366415d5f0 legacy=0x7f36640ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:13.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.249+0000 7f366915e640 1 -- 172.21.15.5:0/3306376765 >> v1:172.21.15.5:6789/0 conn(0x7f36640cedd0 legacy=0x7f366406e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:13.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.249+0000 7f366915e640 1 -- 172.21.15.5:0/3306376765 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366406bbe0 con 0x7f36641576d0 2024-05-20T18:33:13.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.249+0000 7f366895d640 1 -- 172.21.15.5:0/3306376765 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2646166475 0 0) 0x7f3664157ae0 con 0x7f36640cedd0 2024-05-20T18:33:13.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.250+0000 7f366915e640 1 --2- 172.21.15.5:0/3306376765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36641576d0 0x7f366406c9a0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f36580097a0 tx=0x7f365802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f93feb13c2600d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:13.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.250+0000 7f366895d640 1 -- 172.21.15.5:0/3306376765 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658037070 con 0x7f36641576d0 2024-05-20T18:33:13.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.250+0000 7f366895d640 1 -- 172.21.15.5:0/3306376765 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f365802ae50 con 0x7f36641576d0 2024-05-20T18:33:13.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.250+0000 7f366895d640 1 -- 172.21.15.5:0/3306376765 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658032660 con 0x7f36641576d0 2024-05-20T18:33:13.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.251+0000 7f366bbc3640 1 -- 172.21.15.5:0/3306376765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36641576d0 msgr2=0x7f366406c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:13.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.251+0000 7f366bbc3640 1 --2- 172.21.15.5:0/3306376765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36641576d0 0x7f366406c9a0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f36580097a0 tx=0x7f365802a140 comp rx=0 tx=0).stop 2024-05-20T18:33:13.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.251+0000 7f366bbc3640 1 -- 172.21.15.5:0/3306376765 shutdown_connections 2024-05-20T18:33:13.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.251+0000 7f366bbc3640 1 --2- 172.21.15.5:0/3306376765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36641576d0 0x7f366406c9a0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:13.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.251+0000 7f366bbc3640 1 -- 172.21.15.5:0/3306376765 >> 172.21.15.5:0/3306376765 conn(0x7f36640cd2a0 msgr2=0x7f36640cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:13.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.251+0000 7f366bbc3640 1 -- 172.21.15.5:0/3306376765 shutdown_connections 2024-05-20T18:33:13.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.251+0000 7f366bbc3640 1 -- 172.21.15.5:0/3306376765 wait complete. 2024-05-20T18:33:13.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.253+0000 7f366bbc3640 1 Processor -- start 2024-05-20T18:33:13.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.253+0000 7f366bbc3640 1 -- start start 2024-05-20T18:33:13.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.254+0000 7f366bbc3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36640cedd0 0x7f36641f5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:13.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.254+0000 7f366bbc3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36641576d0 0x7f36640ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:13.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.254+0000 7f366bbc3640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f366415d5f0 0x7f36640b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:13.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.254+0000 7f366bbc3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f366406c480 con 0x7f36641576d0 2024-05-20T18:33:13.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.254+0000 7f366bbc3640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f366406c1a0 con 0x7f366415d5f0 2024-05-20T18:33:13.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.254+0000 7f366bbc3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f36640cf1e0 con 0x7f36640cedd0 2024-05-20T18:33:13.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.254+0000 7f366915e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36641576d0 0x7f36640ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:13.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.254+0000 7f366915e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36641576d0 0x7f36640ae150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56744/0 (socket says 172.21.15.5:56744) 2024-05-20T18:33:13.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.254+0000 7f366915e640 1 -- 172.21.15.5:0/1034908509 learned_addr learned my addr 172.21.15.5:0/1034908509 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:13.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.254+0000 7f366915e640 1 -- 172.21.15.5:0/1034908509 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36640cedd0 msgr2=0x7f36641f5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:13.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.254+0000 7f366915e640 1 --2- 172.21.15.5:0/1034908509 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36640cedd0 0x7f36641f5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:13.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.254+0000 7f366915e640 1 -- 172.21.15.5:0/1034908509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f366415d5f0 msgr2=0x7f36640b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:13.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.254+0000 7f366a160640 1 --2- 172.21.15.5:0/1034908509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f366415d5f0 0x7f36640b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:13.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.255+0000 7f366915e640 1 --2- 172.21.15.5:0/1034908509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f366415d5f0 0x7f36640b0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:13.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.255+0000 7f366915e640 1 -- 172.21.15.5:0/1034908509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36640adf50 con 0x7f36641576d0 2024-05-20T18:33:13.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.255+0000 7f366915e640 1 --2- 172.21.15.5:0/1034908509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36641576d0 0x7f36640ae150 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f36580098c0 tx=0x7f36580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:13.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.255+0000 7f36627fc640 1 -- 172.21.15.5:0/1034908509 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658042070 con 0x7f36641576d0 2024-05-20T18:33:13.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.255+0000 7f36627fc640 1 -- 172.21.15.5:0/1034908509 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f365803e670 con 0x7f36641576d0 2024-05-20T18:33:13.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.255+0000 7f36627fc640 1 -- 172.21.15.5:0/1034908509 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658037040 con 0x7f36641576d0 2024-05-20T18:33:13.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.255+0000 7f366bbc3640 1 -- 172.21.15.5:0/1034908509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36640aed30 con 0x7f36641576d0 2024-05-20T18:33:13.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.255+0000 7f366bbc3640 1 -- 172.21.15.5:0/1034908509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36640af2f0 con 0x7f36641576d0 2024-05-20T18:33:13.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.256+0000 7f366bbc3640 1 -- 172.21.15.5:0/1034908509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3664075b00 con 0x7f36641576d0 2024-05-20T18:33:13.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.257+0000 7f36627fc640 1 -- 172.21.15.5:0/1034908509 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3658045070 con 0x7f36641576d0 2024-05-20T18:33:13.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.257+0000 7f36627fc640 1 --2- 172.21.15.5:0/1034908509 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f365006be30 0x7f365006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:13.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.258+0000 7f366995f640 1 --2- 172.21.15.5:0/1034908509 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f365006be30 0x7f365006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:13.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.258+0000 7f36627fc640 1 -- 172.21.15.5:0/1034908509 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6101+0+0 (secure 0 0 0) 0x7f36580ab900 con 0x7f36641576d0 2024-05-20T18:33:13.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.258+0000 7f366995f640 1 --2- 172.21.15.5:0/1034908509 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f365006be30 0x7f365006e340 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f36640b0490 tx=0x7f3654009230 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:13.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.262+0000 7f36627fc640 1 -- 172.21.15.5:0/1034908509 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f36640adf50 con 0x7f36641576d0 2024-05-20T18:33:13.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.451+0000 7f366bbc3640 1 -- 172.21.15.5:0/1034908509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36640aea50 con 0x7f36641576d0 2024-05-20T18:33:13.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.452+0000 7f36627fc640 1 -- 172.21.15.5:0/1034908509 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+16778 (secure 0 0 0) 0x7f365807a010 con 0x7f36641576d0 2024-05-20T18:33:13.453 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:33:13.453 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":89,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:33:12.457375+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:33:10.184948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-20T18:33:11.453356+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:33:13.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.454+0000 7f366bbc3640 1 -- 172.21.15.5:0/1034908509 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f365006be30 msgr2=0x7f365006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:13.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.454+0000 7f366bbc3640 1 --2- 172.21.15.5:0/1034908509 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f365006be30 0x7f365006e340 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f36640b0490 tx=0x7f3654009230 comp rx=0 tx=0).stop 2024-05-20T18:33:13.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.454+0000 7f366bbc3640 1 -- 172.21.15.5:0/1034908509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36641576d0 msgr2=0x7f36640ae150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:13.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.454+0000 7f366bbc3640 1 --2- 172.21.15.5:0/1034908509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36641576d0 0x7f36640ae150 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f36580098c0 tx=0x7f36580046c0 comp rx=0 tx=0).stop 2024-05-20T18:33:13.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.454+0000 7f366bbc3640 1 -- 172.21.15.5:0/1034908509 shutdown_connections 2024-05-20T18:33:13.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.454+0000 7f366bbc3640 1 --2- 172.21.15.5:0/1034908509 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f365006be30 0x7f365006e340 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:13.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.454+0000 7f366bbc3640 1 --2- 172.21.15.5:0/1034908509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f366415d5f0 0x7f36640b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:13.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.454+0000 7f366bbc3640 1 --2- 172.21.15.5:0/1034908509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36641576d0 0x7f36640ae150 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:13.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.454+0000 7f366bbc3640 1 --2- 172.21.15.5:0/1034908509 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36640cedd0 0x7f36641f5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:13.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.454+0000 7f366bbc3640 1 -- 172.21.15.5:0/1034908509 >> 172.21.15.5:0/1034908509 conn(0x7f36640cd2a0 msgr2=0x7f3664152d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:13.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.454+0000 7f366bbc3640 1 -- 172.21.15.5:0/1034908509 shutdown_connections 2024-05-20T18:33:13.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.454+0000 7f366bbc3640 1 -- 172.21.15.5:0/1034908509 wait complete. 2024-05-20T18:33:13.467 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:33:13.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.581+0000 7f1c8a88b640 1 Processor -- start 2024-05-20T18:33:13.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.581+0000 7f1c8a88b640 1 -- start start 2024-05-20T18:33:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.583+0000 7f1c8a88b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1c840b47b0 0x7f1c840afad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.583+0000 7f1c8a88b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c840b58f0 0x7f1c8406f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.583+0000 7f1c8a88b640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c840b39f0 con 0x7f1c8415d060 2024-05-20T18:33:13.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.583+0000 7f1c8a88b640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1c840b3cd0 con 0x7f1c840b47b0 2024-05-20T18:33:13.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.583+0000 7f1c8a88b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1c840b3fb0 con 0x7f1c840b58f0 2024-05-20T18:33:13.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.583+0000 7f1c83fff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1c840b47b0 0x7f1c840afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:13.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.583+0000 7f1c88e28640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c840b58f0 0x7f1c8406f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:13.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.583+0000 7f1c83fff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1c840b47b0 0x7f1c840afad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:57972/0 (socket says 172.21.15.5:57972) 2024-05-20T18:33:13.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.583+0000 7f1c83fff640 1 -- 172.21.15.5:0/304998721 learned_addr learned my addr 172.21.15.5:0/304998721 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:13.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.584+0000 7f1c83fff640 1 -- 172.21.15.5:0/304998721 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c840b58f0 msgr2=0x7f1c8406f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:13.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.584+0000 7f1c83fff640 1 --2- 172.21.15.5:0/304998721 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c840b58f0 0x7f1c8406f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:13.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.584+0000 7f1c83fff640 1 -- 172.21.15.5:0/304998721 >> v1:172.21.15.63:6789/0 conn(0x7f1c8415d060 legacy=0x7f1c840b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:13.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.584+0000 7f1c83fff640 1 -- 172.21.15.5:0/304998721 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c840b4290 con 0x7f1c840b47b0 2024-05-20T18:33:13.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.584+0000 7f1c88e28640 1 --2- 172.21.15.5:0/304998721 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c840b58f0 0x7f1c8406f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:33:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.584+0000 7f1c83fff640 1 --2- 172.21.15.5:0/304998721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c840b47b0 0x7f1c840afad0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1c7400bb10 tx=0x7f1c7402c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eff2124bbb59b495 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.585+0000 7f1c837fe640 1 -- 172.21.15.5:0/304998721 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c74032790 con 0x7f1c840b47b0 2024-05-20T18:33:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.585+0000 7f1c837fe640 1 -- 172.21.15.5:0/304998721 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c74038070 con 0x7f1c840b47b0 2024-05-20T18:33:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.585+0000 7f1c837fe640 1 -- 172.21.15.5:0/304998721 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c74032b50 con 0x7f1c840b47b0 2024-05-20T18:33:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.585+0000 7f1c8a88b640 1 -- 172.21.15.5:0/304998721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c840b47b0 msgr2=0x7f1c840afad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.585+0000 7f1c8a88b640 1 --2- 172.21.15.5:0/304998721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c840b47b0 0x7f1c840afad0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1c7400bb10 tx=0x7f1c7402c890 comp rx=0 tx=0).stop 2024-05-20T18:33:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.586+0000 7f1c8a88b640 1 -- 172.21.15.5:0/304998721 shutdown_connections 2024-05-20T18:33:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.586+0000 7f1c8a88b640 1 --2- 172.21.15.5:0/304998721 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c840b58f0 0x7f1c8406f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:13.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.586+0000 7f1c8a88b640 1 --2- 172.21.15.5:0/304998721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c840b47b0 0x7f1c840afad0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:13.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.586+0000 7f1c8a88b640 1 -- 172.21.15.5:0/304998721 >> 172.21.15.5:0/304998721 conn(0x7f1c840cd2a0 msgr2=0x7f1c840cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:13.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.586+0000 7f1c8a88b640 1 -- 172.21.15.5:0/304998721 shutdown_connections 2024-05-20T18:33:13.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.586+0000 7f1c8a88b640 1 -- 172.21.15.5:0/304998721 wait complete. 2024-05-20T18:33:13.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.587+0000 7f1c8a88b640 1 Processor -- start 2024-05-20T18:33:13.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.587+0000 7f1c8a88b640 1 -- start start 2024-05-20T18:33:13.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.589+0000 7f1c8a88b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c8415d060 0x7f1c840af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:13.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.589+0000 7f1c8a88b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c840adff0 0x7f1c840ae400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:13.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.589+0000 7f1c8a88b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c840ae940 0x7f1c84153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:13.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.589+0000 7f1c8a88b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c84159de0 con 0x7f1c8415d060 2024-05-20T18:33:13.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.589+0000 7f1c8a88b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1c84159c60 con 0x7f1c840ae940 2024-05-20T18:33:13.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.589+0000 7f1c8a88b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1c8406f910 con 0x7f1c840adff0 2024-05-20T18:33:13.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.589+0000 7f1c83fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c8415d060 0x7f1c840af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:13.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.589+0000 7f1c83fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c8415d060 0x7f1c840af8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56760/0 (socket says 172.21.15.5:56760) 2024-05-20T18:33:13.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.589+0000 7f1c83fff640 1 -- 172.21.15.5:0/1156676580 learned_addr learned my addr 172.21.15.5:0/1156676580 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:13.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.589+0000 7f1c88e28640 1 --2- 172.21.15.5:0/1156676580 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c840ae940 0x7f1c84153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:13.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.589+0000 7f1c7bfff640 1 --2- 172.21.15.5:0/1156676580 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c840adff0 0x7f1c840ae400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:13.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.589+0000 7f1c83fff640 1 -- 172.21.15.5:0/1156676580 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c840adff0 msgr2=0x7f1c840ae400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:13.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.589+0000 7f1c83fff640 1 --2- 172.21.15.5:0/1156676580 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c840adff0 0x7f1c840ae400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:13.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.589+0000 7f1c83fff640 1 -- 172.21.15.5:0/1156676580 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c840ae940 msgr2=0x7f1c84153f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:13.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.589+0000 7f1c83fff640 1 --2- 172.21.15.5:0/1156676580 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c840ae940 0x7f1c84153f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:13.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.589+0000 7f1c83fff640 1 -- 172.21.15.5:0/1156676580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c840ad830 con 0x7f1c8415d060 2024-05-20T18:33:13.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.589+0000 7f1c83fff640 1 --2- 172.21.15.5:0/1156676580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c8415d060 0x7f1c840af8d0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f1c74000c00 tx=0x7f1c740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:13.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.590+0000 7f1c817fa640 1 -- 172.21.15.5:0/1156676580 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c74033620 con 0x7f1c8415d060 2024-05-20T18:33:13.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.590+0000 7f1c817fa640 1 -- 172.21.15.5:0/1156676580 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c740337c0 con 0x7f1c8415d060 2024-05-20T18:33:13.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.590+0000 7f1c8a88b640 1 -- 172.21.15.5:0/1156676580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c840afe10 con 0x7f1c8415d060 2024-05-20T18:33:13.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.590+0000 7f1c817fa640 1 -- 172.21.15.5:0/1156676580 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c74033bb0 con 0x7f1c8415d060 2024-05-20T18:33:13.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.590+0000 7f1c8a88b640 1 -- 172.21.15.5:0/1156676580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c84155490 con 0x7f1c8415d060 2024-05-20T18:33:13.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.591+0000 7f1c8a88b640 1 -- 172.21.15.5:0/1156676580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c84075b00 con 0x7f1c8415d060 2024-05-20T18:33:13.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.591+0000 7f1c817fa640 1 -- 172.21.15.5:0/1156676580 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c7404d060 con 0x7f1c8415d060 2024-05-20T18:33:13.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.592+0000 7f1c817fa640 1 --2- 172.21.15.5:0/1156676580 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c6406be60 0x7f1c6406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:13.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.592+0000 7f1c817fa640 1 -- 172.21.15.5:0/1156676580 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6101+0+0 (secure 0 0 0) 0x7f1c740adc80 con 0x7f1c8415d060 2024-05-20T18:33:13.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.592+0000 7f1c7bfff640 1 --2- 172.21.15.5:0/1156676580 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c6406be60 0x7f1c6406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:13.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.592+0000 7f1c7bfff640 1 --2- 172.21.15.5:0/1156676580 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c6406be60 0x7f1c6406e370 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1c8415cce0 tx=0x7f1c6c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:13.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.596+0000 7f1c817fa640 1 -- 172.21.15.5:0/1156676580 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1c74041070 con 0x7f1c8415d060 2024-05-20T18:33:13.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.780+0000 7f1c8a88b640 1 -- 172.21.15.5:0/1156676580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1c840b4db0 con 0x7f1c8415d060 2024-05-20T18:33:13.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.781+0000 7f1c817fa640 1 -- 172.21.15.5:0/1156676580 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+16778 (secure 0 0 0) 0x7f1c7407c420 con 0x7f1c8415d060 2024-05-20T18:33:13.782 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:33:13.783 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":89,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:33:12.457375+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:33:10.184948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-20T18:33:11.453356+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:33:13.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.783+0000 7f1c8a88b640 1 -- 172.21.15.5:0/1156676580 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c6406be60 msgr2=0x7f1c6406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:13.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.783+0000 7f1c8a88b640 1 --2- 172.21.15.5:0/1156676580 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c6406be60 0x7f1c6406e370 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1c8415cce0 tx=0x7f1c6c009040 comp rx=0 tx=0).stop 2024-05-20T18:33:13.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.783+0000 7f1c8a88b640 1 -- 172.21.15.5:0/1156676580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c8415d060 msgr2=0x7f1c840af8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:13.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.783+0000 7f1c8a88b640 1 --2- 172.21.15.5:0/1156676580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c8415d060 0x7f1c840af8d0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f1c74000c00 tx=0x7f1c740046c0 comp rx=0 tx=0).stop 2024-05-20T18:33:13.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.783+0000 7f1c8a88b640 1 -- 172.21.15.5:0/1156676580 shutdown_connections 2024-05-20T18:33:13.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.783+0000 7f1c8a88b640 1 --2- 172.21.15.5:0/1156676580 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c6406be60 0x7f1c6406e370 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:13.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.783+0000 7f1c8a88b640 1 --2- 172.21.15.5:0/1156676580 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c840ae940 0x7f1c84153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:13.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.783+0000 7f1c8a88b640 1 --2- 172.21.15.5:0/1156676580 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c840adff0 0x7f1c840ae400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:13.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.783+0000 7f1c8a88b640 1 --2- 172.21.15.5:0/1156676580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c8415d060 0x7f1c840af8d0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:13.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.783+0000 7f1c8a88b640 1 -- 172.21.15.5:0/1156676580 >> 172.21.15.5:0/1156676580 conn(0x7f1c840cd2a0 msgr2=0x7f1c840cfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:13.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.783+0000 7f1c8a88b640 1 -- 172.21.15.5:0/1156676580 shutdown_connections 2024-05-20T18:33:13.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.783+0000 7f1c8a88b640 1 -- 172.21.15.5:0/1156676580 wait complete. 2024-05-20T18:33:13.796 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-20T18:33:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.912+0000 7f1b74f65640 1 Processor -- start 2024-05-20T18:33:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.912+0000 7f1b74f65640 1 -- start start 2024-05-20T18:33:13.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.914+0000 7f1b74f65640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1b7006d3d0 0x7f1b7006d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:13.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.914+0000 7f1b74f65640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b700b8190 0x7f1b700b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:13.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.914+0000 7f1b74f65640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b7006dd20 con 0x7f1b70162820 2024-05-20T18:33:13.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.914+0000 7f1b74f65640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1b700d0630 con 0x7f1b7006d3d0 2024-05-20T18:33:13.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.914+0000 7f1b74f65640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1b700d39c0 con 0x7f1b700b8190 2024-05-20T18:33:13.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.914+0000 7f1b6e59c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1b70162820 0x7f1b700b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38216/0 (socket says 172.21.15.5:38216) 2024-05-20T18:33:13.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.914+0000 7f1b6e59c640 1 -- 172.21.15.5:0/4244611458 learned_addr learned my addr 172.21.15.5:0/4244611458 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:33:13.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.914+0000 7f1b6ed9d640 1 --2- 172.21.15.5:0/4244611458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b700b8190 0x7f1b700b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:13.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.914+0000 7f1b6d59a640 1 -- 172.21.15.5:0/4244611458 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1655800682 0 0) 0x7f1b68001490 con 0x7f1b70162820 2024-05-20T18:33:13.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.914+0000 7f1b6dd9b640 1 --2- 172.21.15.5:0/4244611458 >> v2:172.21.15.63:3300/0 conn(0x7f1b7006d3d0 0x7f1b7006d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:13.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.914+0000 7f1b6d59a640 1 -- 172.21.15.5:0/4244611458 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b5c003520 con 0x7f1b70162820 2024-05-20T18:33:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.915+0000 7f1b6d59a640 1 -- 172.21.15.5:0/4244611458 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4034691594 0 0) 0x7f1b5c003520 con 0x7f1b70162820 2024-05-20T18:33:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.915+0000 7f1b6d59a640 1 -- 172.21.15.5:0/4244611458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b700b8190 msgr2=0x7f1b700b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.915+0000 7f1b6d59a640 1 --2- 172.21.15.5:0/4244611458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b700b8190 0x7f1b700b85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.915+0000 7f1b6d59a640 1 -- 172.21.15.5:0/4244611458 >> v2:172.21.15.63:3300/0 conn(0x7f1b7006d3d0 msgr2=0x7f1b7006d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.915+0000 7f1b6d59a640 1 --2- 172.21.15.5:0/4244611458 >> v2:172.21.15.63:3300/0 conn(0x7f1b7006d3d0 0x7f1b7006d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.915+0000 7f1b6d59a640 1 -- 172.21.15.5:0/4244611458 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b700d3ca0 con 0x7f1b70162820 2024-05-20T18:33:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.915+0000 7f1b6d59a640 1 -- 172.21.15.5:0/4244611458 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f1b68001bc0 con 0x7f1b70162820 2024-05-20T18:33:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.915+0000 7f1b6d59a640 1 -- 172.21.15.5:0/4244611458 >> v1:172.21.15.5:6789/0 conn(0x7f1b70162820 legacy=0x7f1b700b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:13.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.915+0000 7f1b6d59a640 1 --2- 172.21.15.5:0/4244611458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b5c0039b0 0x7f1b5c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.915+0000 7f1b6d59a640 1 --2- 172.21.15.5:0/4244611458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b5c004800 0x7f1b5c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.915+0000 7f1b6d59a640 1 --2- 172.21.15.5:0/4244611458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b700b8190 0x7f1b5c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.915+0000 7f1b6d59a640 1 -- 172.21.15.5:0/4244611458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1b700d0630 con 0x7f1b5c0039b0 2024-05-20T18:33:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.915+0000 7f1b6d59a640 1 -- 172.21.15.5:0/4244611458 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1b700d39c0 con 0x7f1b5c004800 2024-05-20T18:33:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.915+0000 7f1b6d59a640 1 -- 172.21.15.5:0/4244611458 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1b5c00b9c0 con 0x7f1b700b8190 2024-05-20T18:33:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.915+0000 7f1b6e59c640 1 --2- 172.21.15.5:0/4244611458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b5c0039b0 0x7f1b5c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.915+0000 7f1b6dd9b640 1 --2- 172.21.15.5:0/4244611458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b5c004800 0x7f1b5c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.915+0000 7f1b6ed9d640 1 --2- 172.21.15.5:0/4244611458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b700b8190 0x7f1b5c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.915+0000 7f1b6e59c640 1 -- 172.21.15.5:0/4244611458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b700b8190 msgr2=0x7f1b5c005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.915+0000 7f1b6e59c640 1 --2- 172.21.15.5:0/4244611458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b700b8190 0x7f1b5c005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.915+0000 7f1b6e59c640 1 -- 172.21.15.5:0/4244611458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b5c004800 msgr2=0x7f1b5c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.915+0000 7f1b6e59c640 1 --2- 172.21.15.5:0/4244611458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b5c004800 0x7f1b5c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:13.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.915+0000 7f1b6e59c640 1 -- 172.21.15.5:0/4244611458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5c003520 con 0x7f1b5c0039b0 2024-05-20T18:33:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.915+0000 7f1b6dd9b640 1 --2- 172.21.15.5:0/4244611458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b5c004800 0x7f1b5c004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.916+0000 7f1b6e59c640 1 --2- 172.21.15.5:0/4244611458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b5c0039b0 0x7f1b5c003e20 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800aa50 tx=0x7f1b5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922e1ad3d9109ec5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.916+0000 7f1b6d59a640 1 -- 172.21.15.5:0/4244611458 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6800a670 con 0x7f1b5c0039b0 2024-05-20T18:33:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.916+0000 7f1b6d59a640 1 -- 172.21.15.5:0/4244611458 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b6802f070 con 0x7f1b5c0039b0 2024-05-20T18:33:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.916+0000 7f1b6d59a640 1 -- 172.21.15.5:0/4244611458 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68034470 con 0x7f1b5c0039b0 2024-05-20T18:33:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.917+0000 7f1b74f65640 1 -- 172.21.15.5:0/4244611458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b5c0039b0 msgr2=0x7f1b5c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.917+0000 7f1b74f65640 1 --2- 172.21.15.5:0/4244611458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b5c0039b0 0x7f1b5c003e20 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800aa50 tx=0x7f1b5c005830 comp rx=0 tx=0).stop 2024-05-20T18:33:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.917+0000 7f1b6ed9d640 1 -- 172.21.15.5:0/4244611458 reap_dead start 2024-05-20T18:33:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.917+0000 7f1b74f65640 1 -- 172.21.15.5:0/4244611458 shutdown_connections 2024-05-20T18:33:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.917+0000 7f1b74f65640 1 -- 172.21.15.5:0/4244611458 >> 172.21.15.5:0/4244611458 conn(0x7f1b700d4bd0 msgr2=0x7f1b700d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.917+0000 7f1b74f65640 1 -- 172.21.15.5:0/4244611458 shutdown_connections 2024-05-20T18:33:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.917+0000 7f1b74f65640 1 -- 172.21.15.5:0/4244611458 wait complete. 2024-05-20T18:33:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.919+0000 7f1b74f65640 1 Processor -- start 2024-05-20T18:33:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.919+0000 7f1b74f65640 1 -- start start 2024-05-20T18:33:13.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.920+0000 7f1b74f65640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b5c0039b0 0x7f1b701f5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:13.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.920+0000 7f1b74f65640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b700cde60 0x7f1b700ce270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:13.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.920+0000 7f1b74f65640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b7014b880 0x7f1b7014bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:13.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.920+0000 7f1b74f65640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1b700d0300 con 0x7f1b7014b880 2024-05-20T18:33:13.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.920+0000 7f1b74f65640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1b700d4260 con 0x7f1b5c0039b0 2024-05-20T18:33:13.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.920+0000 7f1b74f65640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1b700b4050 con 0x7f1b700cde60 2024-05-20T18:33:13.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.920+0000 7f1b6ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b7014b880 0x7f1b7014bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:13.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.920+0000 7f1b6ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b7014b880 0x7f1b7014bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56772/0 (socket says 172.21.15.5:56772) 2024-05-20T18:33:13.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.921+0000 7f1b6ed9d640 1 -- 172.21.15.5:0/103087347 learned_addr learned my addr 172.21.15.5:0/103087347 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:13.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.921+0000 7f1b6dd9b640 1 --2- 172.21.15.5:0/103087347 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b700cde60 0x7f1b700ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:13.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.921+0000 7f1b6e59c640 1 --2- 172.21.15.5:0/103087347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b5c0039b0 0x7f1b701f5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:13.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.921+0000 7f1b6ed9d640 1 -- 172.21.15.5:0/103087347 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b700cde60 msgr2=0x7f1b700ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:13.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.921+0000 7f1b6ed9d640 1 --2- 172.21.15.5:0/103087347 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b700cde60 0x7f1b700ce270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:13.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.921+0000 7f1b6ed9d640 1 -- 172.21.15.5:0/103087347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b5c0039b0 msgr2=0x7f1b701f5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:13.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.921+0000 7f1b6ed9d640 1 --2- 172.21.15.5:0/103087347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b5c0039b0 0x7f1b701f5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:13.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.921+0000 7f1b6ed9d640 1 -- 172.21.15.5:0/103087347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b700cd0a0 con 0x7f1b7014b880 2024-05-20T18:33:13.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.921+0000 7f1b6ed9d640 1 --2- 172.21.15.5:0/103087347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b7014b880 0x7f1b7014bc90 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f1b7008e090 tx=0x7f1b60011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:13.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.921+0000 7f1b66ffd640 1 -- 172.21.15.5:0/103087347 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60019020 con 0x7f1b7014b880 2024-05-20T18:33:13.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.921+0000 7f1b66ffd640 1 -- 172.21.15.5:0/103087347 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b60003610 con 0x7f1b7014b880 2024-05-20T18:33:13.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.921+0000 7f1b66ffd640 1 -- 172.21.15.5:0/103087347 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60003a00 con 0x7f1b7014b880 2024-05-20T18:33:13.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.921+0000 7f1b74f65640 1 -- 172.21.15.5:0/103087347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b700cd380 con 0x7f1b7014b880 2024-05-20T18:33:13.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.921+0000 7f1b74f65640 1 -- 172.21.15.5:0/103087347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b700cd940 con 0x7f1b7014b880 2024-05-20T18:33:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.922+0000 7f1b74f65640 1 -- 172.21.15.5:0/103087347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b701e7db0 con 0x7f1b7014b880 2024-05-20T18:33:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.923+0000 7f1b66ffd640 1 -- 172.21.15.5:0/103087347 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b6001c020 con 0x7f1b7014b880 2024-05-20T18:33:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.923+0000 7f1b66ffd640 1 --2- 172.21.15.5:0/103087347 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1b5006be60 0x7f1b5006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.924+0000 7f1b66ffd640 1 -- 172.21.15.5:0/103087347 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6101+0+0 (secure 0 0 0) 0x7f1b6008b070 con 0x7f1b7014b880 2024-05-20T18:33:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.924+0000 7f1b6e59c640 1 --2- 172.21.15.5:0/103087347 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1b5006be60 0x7f1b5006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.924+0000 7f1b6e59c640 1 --2- 172.21.15.5:0/103087347 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1b5006be60 0x7f1b5006e370 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1b68033000 tx=0x7f1b68007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:13.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:13.928+0000 7f1b66ffd640 1 -- 172.21.15.5:0/103087347 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1b60023030 con 0x7f1b7014b880 2024-05-20T18:33:14.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:14.154+0000 7f1b74f65640 1 -- 172.21.15.5:0/103087347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1b70075b00 con 0x7f1b7014b880 2024-05-20T18:33:15.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.105+0000 7f1b66ffd640 1 -- 172.21.15.5:0/103087347 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v59) ==== 118+0+0 (secure 0 0 0) 0x7f1b600592d0 con 0x7f1b7014b880 2024-05-20T18:33:15.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.108+0000 7f1b74f65640 1 -- 172.21.15.5:0/103087347 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1b5006be60 msgr2=0x7f1b5006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:15.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.108+0000 7f1b74f65640 1 --2- 172.21.15.5:0/103087347 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1b5006be60 0x7f1b5006e370 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1b68033000 tx=0x7f1b68007000 comp rx=0 tx=0).stop 2024-05-20T18:33:15.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.108+0000 7f1b74f65640 1 -- 172.21.15.5:0/103087347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b7014b880 msgr2=0x7f1b7014bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:15.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.108+0000 7f1b74f65640 1 --2- 172.21.15.5:0/103087347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b7014b880 0x7f1b7014bc90 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f1b7008e090 tx=0x7f1b60011920 comp rx=0 tx=0).stop 2024-05-20T18:33:15.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.108+0000 7f1b74f65640 1 -- 172.21.15.5:0/103087347 shutdown_connections 2024-05-20T18:33:15.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.108+0000 7f1b74f65640 1 --2- 172.21.15.5:0/103087347 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1b5006be60 0x7f1b5006e370 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:15.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.108+0000 7f1b74f65640 1 --2- 172.21.15.5:0/103087347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b7014b880 0x7f1b7014bc90 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:15.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.108+0000 7f1b74f65640 1 --2- 172.21.15.5:0/103087347 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b700cde60 0x7f1b700ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:15.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.108+0000 7f1b74f65640 1 --2- 172.21.15.5:0/103087347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b5c0039b0 0x7f1b701f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:15.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.108+0000 7f1b74f65640 1 -- 172.21.15.5:0/103087347 >> 172.21.15.5:0/103087347 conn(0x7f1b700d4bd0 msgr2=0x7f1b70161360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:15.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.108+0000 7f1b74f65640 1 -- 172.21.15.5:0/103087347 shutdown_connections 2024-05-20T18:33:15.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.108+0000 7f1b74f65640 1 -- 172.21.15.5:0/103087347 wait complete. 2024-05-20T18:33:15.122 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-20T18:33:15.122 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-20T18:33:15.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.247+0000 7fb04fdfe640 1 Processor -- start 2024-05-20T18:33:15.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.248+0000 7fb04fdfe640 1 -- start start 2024-05-20T18:33:15.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.249+0000 7fb04fdfe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb048162820 0x7fb0480b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:15.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.249+0000 7fb04fdfe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0480b3e80 0x7fb04806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:15.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.249+0000 7fb04fdfe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb04806d910 0x7fb0480d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:15.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.249+0000 7fb04fdfe640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fb04806dd20 con 0x7fb048162820 2024-05-20T18:33:15.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.249+0000 7fb04fdfe640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb0480d0630 con 0x7fb04806d910 2024-05-20T18:33:15.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.249+0000 7fb04fdfe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb0480d39c0 con 0x7fb0480b3e80 2024-05-20T18:33:15.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.249+0000 7fb04db9a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb048162820 0x7fb0480b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:15.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.249+0000 7fb04db9a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb048162820 0x7fb0480b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58912/0 (socket says 172.21.15.5:58912) 2024-05-20T18:33:15.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.249+0000 7fb04db9a640 1 -- 172.21.15.5:0/497287607 learned_addr learned my addr 172.21.15.5:0/497287607 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:15.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.249+0000 7fb04e39b640 1 --2- 172.21.15.5:0/497287607 >> v2:172.21.15.63:3300/0 conn(0x7fb04806d910 0x7fb0480d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:15.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.249+0000 7fb04d399640 1 --2- 172.21.15.5:0/497287607 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0480b3e80 0x7fb04806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:15.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.250+0000 7fb04db9a640 1 -- 172.21.15.5:0/497287607 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0480b3e80 msgr2=0x7fb04806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:15.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.250+0000 7fb04db9a640 1 --2- 172.21.15.5:0/497287607 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0480b3e80 0x7fb04806d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:15.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.250+0000 7fb04db9a640 1 -- 172.21.15.5:0/497287607 >> v2:172.21.15.63:3300/0 conn(0x7fb04806d910 msgr2=0x7fb0480d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:15.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.250+0000 7fb04db9a640 1 --2- 172.21.15.5:0/497287607 >> v2:172.21.15.63:3300/0 conn(0x7fb04806d910 0x7fb0480d27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:15.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.250+0000 7fb04db9a640 1 -- 172.21.15.5:0/497287607 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0480d3ca0 con 0x7fb048162820 2024-05-20T18:33:15.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.250+0000 7fb04db9a640 1 --2- 172.21.15.5:0/497287607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb048162820 0x7fb0480b3940 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fb04808d5e0 tx=0x7fb04002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294d0aa8dcba3d84 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:15.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.250+0000 7fb04cb98640 1 -- 172.21.15.5:0/497287607 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb040037070 con 0x7fb048162820 2024-05-20T18:33:15.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.250+0000 7fb04cb98640 1 -- 172.21.15.5:0/497287607 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb040032440 con 0x7fb048162820 2024-05-20T18:33:15.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.250+0000 7fb04cb98640 1 -- 172.21.15.5:0/497287607 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb040032810 con 0x7fb048162820 2024-05-20T18:33:15.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.251+0000 7fb04fdfe640 1 -- 172.21.15.5:0/497287607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb048162820 msgr2=0x7fb0480b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:15.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.251+0000 7fb04fdfe640 1 --2- 172.21.15.5:0/497287607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb048162820 0x7fb0480b3940 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fb04808d5e0 tx=0x7fb04002a1e0 comp rx=0 tx=0).stop 2024-05-20T18:33:15.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.251+0000 7fb04fdfe640 1 -- 172.21.15.5:0/497287607 shutdown_connections 2024-05-20T18:33:15.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.251+0000 7fb04fdfe640 1 --2- 172.21.15.5:0/497287607 >> v2:172.21.15.63:3300/0 conn(0x7fb04806d910 0x7fb0480d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:15.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.251+0000 7fb04fdfe640 1 --2- 172.21.15.5:0/497287607 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb0480b3e80 0x7fb04806d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:15.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.251+0000 7fb04fdfe640 1 --2- 172.21.15.5:0/497287607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb048162820 0x7fb0480b3940 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:15.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.251+0000 7fb04fdfe640 1 -- 172.21.15.5:0/497287607 >> 172.21.15.5:0/497287607 conn(0x7fb0480d4bd0 msgr2=0x7fb0480d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:15.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.251+0000 7fb04fdfe640 1 -- 172.21.15.5:0/497287607 shutdown_connections 2024-05-20T18:33:15.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.251+0000 7fb04fdfe640 1 -- 172.21.15.5:0/497287607 wait complete. 2024-05-20T18:33:15.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.253+0000 7fb04fdfe640 1 Processor -- start 2024-05-20T18:33:15.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.253+0000 7fb04fdfe640 1 -- start start 2024-05-20T18:33:15.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.254+0000 7fb04fdfe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb04806d910 0x7fb0481f81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:15.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.254+0000 7fb04fdfe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0480b3e80 0x7fb0480b6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:15.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.254+0000 7fb04fdfe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb048162820 0x7fb0480b6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:15.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.254+0000 7fb04fdfe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb0480d0300 con 0x7fb0480b3e80 2024-05-20T18:33:15.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.254+0000 7fb04fdfe640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb0480d4260 con 0x7fb048162820 2024-05-20T18:33:15.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.254+0000 7fb04fdfe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb0480d4780 con 0x7fb04806d910 2024-05-20T18:33:15.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.254+0000 7fb04d399640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0480b3e80 0x7fb0480b6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:15.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.254+0000 7fb04d399640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0480b3e80 0x7fb0480b6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58926/0 (socket says 172.21.15.5:58926) 2024-05-20T18:33:15.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.255+0000 7fb04d399640 1 -- 172.21.15.5:0/2002222599 learned_addr learned my addr 172.21.15.5:0/2002222599 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:15.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.255+0000 7fb04db9a640 1 --2- 172.21.15.5:0/2002222599 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb04806d910 0x7fb0481f81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:15.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.255+0000 7fb04e39b640 1 --2- 172.21.15.5:0/2002222599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb048162820 0x7fb0480b6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:15.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.255+0000 7fb04d399640 1 -- 172.21.15.5:0/2002222599 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb04806d910 msgr2=0x7fb0481f81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:15.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.255+0000 7fb04d399640 1 --2- 172.21.15.5:0/2002222599 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb04806d910 0x7fb0481f81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:15.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.255+0000 7fb04d399640 1 -- 172.21.15.5:0/2002222599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb048162820 msgr2=0x7fb0480b6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:15.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.255+0000 7fb04d399640 1 --2- 172.21.15.5:0/2002222599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb048162820 0x7fb0480b6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:15.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.255+0000 7fb04d399640 1 -- 172.21.15.5:0/2002222599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0481f86e0 con 0x7fb0480b3e80 2024-05-20T18:33:15.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.255+0000 7fb04d399640 1 --2- 172.21.15.5:0/2002222599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0480b3e80 0x7fb0480b6770 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00d620 tx=0x7fb03c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:15.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.255+0000 7fb0467fc640 1 -- 172.21.15.5:0/2002222599 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c00b620 con 0x7fb0480b3e80 2024-05-20T18:33:15.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.255+0000 7fb0467fc640 1 -- 172.21.15.5:0/2002222599 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb03c019070 con 0x7fb0480b3e80 2024-05-20T18:33:15.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.255+0000 7fb0467fc640 1 -- 172.21.15.5:0/2002222599 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c00ba00 con 0x7fb0480b3e80 2024-05-20T18:33:15.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.255+0000 7fb04fdfe640 1 -- 172.21.15.5:0/2002222599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0480ccee0 con 0x7fb0480b3e80 2024-05-20T18:33:15.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.255+0000 7fb04fdfe640 1 -- 172.21.15.5:0/2002222599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0480cd4a0 con 0x7fb0480b3e80 2024-05-20T18:33:15.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.256+0000 7fb04fdfe640 1 -- 172.21.15.5:0/2002222599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb048075b00 con 0x7fb0480b3e80 2024-05-20T18:33:15.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.257+0000 7fb0467fc640 1 -- 172.21.15.5:0/2002222599 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb03c00bba0 con 0x7fb0480b3e80 2024-05-20T18:33:15.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.257+0000 7fb0467fc640 1 --2- 172.21.15.5:0/2002222599 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb03406be30 0x7fb03406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:15.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.258+0000 7fb0467fc640 1 -- 172.21.15.5:0/2002222599 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6101+0+0 (secure 0 0 0) 0x7fb03c08b330 con 0x7fb0480b3e80 2024-05-20T18:33:15.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.258+0000 7fb04db9a640 1 --2- 172.21.15.5:0/2002222599 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb03406be30 0x7fb03406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:15.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.258+0000 7fb04db9a640 1 --2- 172.21.15.5:0/2002222599 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb03406be30 0x7fb03406e340 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb040034000 tx=0x7fb04002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:15.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.262+0000 7fb0467fc640 1 -- 172.21.15.5:0/2002222599 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb03c059470 con 0x7fb0480b3e80 2024-05-20T18:33:15.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:15.488+0000 7fb04fdfe640 1 -- 172.21.15.5:0/2002222599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb04808d930 con 0x7fb0480b3e80 2024-05-20T18:33:16.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.107+0000 7fb0467fc640 1 -- 172.21.15.5:0/2002222599 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v60) ==== 122+0+0 (secure 0 0 0) 0x7fb03c05d770 con 0x7fb0480b3e80 2024-05-20T18:33:16.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.110+0000 7fb04fdfe640 1 -- 172.21.15.5:0/2002222599 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb03406be30 msgr2=0x7fb03406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:16.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.110+0000 7fb04fdfe640 1 --2- 172.21.15.5:0/2002222599 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb03406be30 0x7fb03406e340 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb040034000 tx=0x7fb04002e000 comp rx=0 tx=0).stop 2024-05-20T18:33:16.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.110+0000 7fb04fdfe640 1 -- 172.21.15.5:0/2002222599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0480b3e80 msgr2=0x7fb0480b6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:16.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.110+0000 7fb04fdfe640 1 --2- 172.21.15.5:0/2002222599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0480b3e80 0x7fb0480b6770 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00d620 tx=0x7fb03c00dae0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.110+0000 7fb04fdfe640 1 -- 172.21.15.5:0/2002222599 shutdown_connections 2024-05-20T18:33:16.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.110+0000 7fb04fdfe640 1 --2- 172.21.15.5:0/2002222599 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb03406be30 0x7fb03406e340 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.110+0000 7fb04fdfe640 1 --2- 172.21.15.5:0/2002222599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb048162820 0x7fb0480b6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.110+0000 7fb04fdfe640 1 --2- 172.21.15.5:0/2002222599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0480b3e80 0x7fb0480b6770 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.110+0000 7fb04fdfe640 1 --2- 172.21.15.5:0/2002222599 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb04806d910 0x7fb0481f81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.110+0000 7fb04fdfe640 1 -- 172.21.15.5:0/2002222599 >> 172.21.15.5:0/2002222599 conn(0x7fb0480d4bd0 msgr2=0x7fb04815eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:16.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.110+0000 7fb04fdfe640 1 -- 172.21.15.5:0/2002222599 shutdown_connections 2024-05-20T18:33:16.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.110+0000 7fb04fdfe640 1 -- 172.21.15.5:0/2002222599 wait complete. 2024-05-20T18:33:16.123 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-20T18:33:16.124 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:33:16.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.247+0000 7f1021879640 1 Processor -- start 2024-05-20T18:33:16.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.247+0000 7f1021879640 1 -- start start 2024-05-20T18:33:16.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.248+0000 7f1021879640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f101c162820 0x7f101c0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:16.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.248+0000 7f1021879640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f101c0b3e80 0x7f101c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:16.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.248+0000 7f1021879640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f101c06d910 0x7f101c0d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:16.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.249+0000 7f1021879640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f101c06dd20 con 0x7f101c06d910 2024-05-20T18:33:16.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.249+0000 7f1021879640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f101c0d0630 con 0x7f101c0b3e80 2024-05-20T18:33:16.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.249+0000 7f1021879640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f101c0d39c0 con 0x7f101c162820 2024-05-20T18:33:16.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.249+0000 7f101b7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f101c06d910 0x7f101c0d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:16.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.249+0000 7f101b7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f101c06d910 0x7f101c0d27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58928/0 (socket says 172.21.15.5:58928) 2024-05-20T18:33:16.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.249+0000 7f101b7fe640 1 -- 172.21.15.5:0/2573205151 learned_addr learned my addr 172.21.15.5:0/2573205151 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:16.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.249+0000 7f1013fff640 1 --2- 172.21.15.5:0/2573205151 >> v2:172.21.15.63:3300/0 conn(0x7f101c0b3e80 0x7f101c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:16.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.249+0000 7f101affd640 1 --2- 172.21.15.5:0/2573205151 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f101c162820 0x7f101c0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:16.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.249+0000 7f101b7fe640 1 -- 172.21.15.5:0/2573205151 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f101c162820 msgr2=0x7f101c0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:16.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.249+0000 7f101b7fe640 1 --2- 172.21.15.5:0/2573205151 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f101c162820 0x7f101c0b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.249+0000 7f101b7fe640 1 -- 172.21.15.5:0/2573205151 >> v2:172.21.15.63:3300/0 conn(0x7f101c0b3e80 msgr2=0x7f101c06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:16.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.249+0000 7f101b7fe640 1 --2- 172.21.15.5:0/2573205151 >> v2:172.21.15.63:3300/0 conn(0x7f101c0b3e80 0x7f101c06d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.249+0000 7f101b7fe640 1 -- 172.21.15.5:0/2573205151 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f101c0d3ca0 con 0x7f101c06d910 2024-05-20T18:33:16.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.250+0000 7f101b7fe640 1 --2- 172.21.15.5:0/2573205151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f101c06d910 0x7f101c0d27a0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f101c08d710 tx=0x7f101402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86de1b79244531dc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:16.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.250+0000 7f101a7fc640 1 -- 172.21.15.5:0/2573205151 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1014036070 con 0x7f101c06d910 2024-05-20T18:33:16.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.250+0000 7f101a7fc640 1 -- 172.21.15.5:0/2573205151 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1014031600 con 0x7f101c06d910 2024-05-20T18:33:16.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.250+0000 7f101a7fc640 1 -- 172.21.15.5:0/2573205151 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10140319d0 con 0x7f101c06d910 2024-05-20T18:33:16.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.251+0000 7f1021879640 1 -- 172.21.15.5:0/2573205151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f101c06d910 msgr2=0x7f101c0d27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:16.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.251+0000 7f1021879640 1 --2- 172.21.15.5:0/2573205151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f101c06d910 0x7f101c0d27a0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f101c08d710 tx=0x7f101402a6a0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.251+0000 7f1021879640 1 -- 172.21.15.5:0/2573205151 shutdown_connections 2024-05-20T18:33:16.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.251+0000 7f1021879640 1 --2- 172.21.15.5:0/2573205151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f101c06d910 0x7f101c0d27a0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.251+0000 7f1021879640 1 --2- 172.21.15.5:0/2573205151 >> v2:172.21.15.63:3300/0 conn(0x7f101c0b3e80 0x7f101c06d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.251+0000 7f1021879640 1 --2- 172.21.15.5:0/2573205151 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f101c162820 0x7f101c0b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.251+0000 7f1021879640 1 -- 172.21.15.5:0/2573205151 >> 172.21.15.5:0/2573205151 conn(0x7f101c0d4bd0 msgr2=0x7f101c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:16.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.251+0000 7f1021879640 1 -- 172.21.15.5:0/2573205151 shutdown_connections 2024-05-20T18:33:16.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.251+0000 7f1021879640 1 -- 172.21.15.5:0/2573205151 wait complete. 2024-05-20T18:33:16.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.253+0000 7f1021879640 1 Processor -- start 2024-05-20T18:33:16.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.253+0000 7f1021879640 1 -- start start 2024-05-20T18:33:16.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.254+0000 7f1021879640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f101c06d910 0x7f101c1f8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:16.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.254+0000 7f1021879640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f101c0b3e80 0x7f101c0ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:16.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.254+0000 7f1021879640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f101c162820 0x7f101c0caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:16.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.254+0000 7f1021879640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f101c0d0300 con 0x7f101c06d910 2024-05-20T18:33:16.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.254+0000 7f1021879640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f101c0d4260 con 0x7f101c0b3e80 2024-05-20T18:33:16.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.254+0000 7f1021879640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f101c0d4780 con 0x7f101c162820 2024-05-20T18:33:16.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.254+0000 7f101affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f101c06d910 0x7f101c1f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:16.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.254+0000 7f101affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f101c06d910 0x7f101c1f8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58934/0 (socket says 172.21.15.5:58934) 2024-05-20T18:33:16.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.254+0000 7f101affd640 1 -- 172.21.15.5:0/1132796075 learned_addr learned my addr 172.21.15.5:0/1132796075 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:16.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.254+0000 7f1013fff640 1 --2- 172.21.15.5:0/1132796075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f101c0b3e80 0x7f101c0ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:16.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.254+0000 7f101b7fe640 1 --2- 172.21.15.5:0/1132796075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f101c162820 0x7f101c0caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:16.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.254+0000 7f101affd640 1 -- 172.21.15.5:0/1132796075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f101c162820 msgr2=0x7f101c0caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:16.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.254+0000 7f101affd640 1 --2- 172.21.15.5:0/1132796075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f101c162820 0x7f101c0caa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.254+0000 7f101affd640 1 -- 172.21.15.5:0/1132796075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f101c0b3e80 msgr2=0x7f101c0ca4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:16.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.254+0000 7f101affd640 1 --2- 172.21.15.5:0/1132796075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f101c0b3e80 0x7f101c0ca4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.254+0000 7f101affd640 1 -- 172.21.15.5:0/1132796075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f101c0b5220 con 0x7f101c06d910 2024-05-20T18:33:16.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.254+0000 7f1013fff640 1 --2- 172.21.15.5:0/1132796075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f101c0b3e80 0x7f101c0ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:16.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.255+0000 7f101affd640 1 --2- 172.21.15.5:0/1132796075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f101c06d910 0x7f101c1f8580 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f100c00e870 tx=0x7f100c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:16.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.255+0000 7f1018ff9640 1 -- 172.21.15.5:0/1132796075 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c017070 con 0x7f101c06d910 2024-05-20T18:33:16.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.255+0000 7f1018ff9640 1 -- 172.21.15.5:0/1132796075 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f100c00c940 con 0x7f101c06d910 2024-05-20T18:33:16.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.255+0000 7f1018ff9640 1 -- 172.21.15.5:0/1132796075 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c00cd30 con 0x7f101c06d910 2024-05-20T18:33:16.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.255+0000 7f1021879640 1 -- 172.21.15.5:0/1132796075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f101c0b5500 con 0x7f101c06d910 2024-05-20T18:33:16.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.255+0000 7f1021879640 1 -- 172.21.15.5:0/1132796075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f101c0b5ac0 con 0x7f101c06d910 2024-05-20T18:33:16.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.256+0000 7f1021879640 1 -- 172.21.15.5:0/1132796075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f101c075b00 con 0x7f101c06d910 2024-05-20T18:33:16.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.257+0000 7f1018ff9640 1 -- 172.21.15.5:0/1132796075 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f100c01a070 con 0x7f101c06d910 2024-05-20T18:33:16.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.257+0000 7f1018ff9640 1 --2- 172.21.15.5:0/1132796075 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0ffc06be60 0x7f0ffc06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:16.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.257+0000 7f1018ff9640 1 -- 172.21.15.5:0/1132796075 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6101+0+0 (secure 0 0 0) 0x7f100c08ae60 con 0x7f101c06d910 2024-05-20T18:33:16.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.257+0000 7f1013fff640 1 --2- 172.21.15.5:0/1132796075 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0ffc06be60 0x7f0ffc06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:16.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.258+0000 7f1013fff640 1 --2- 172.21.15.5:0/1132796075 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0ffc06be60 0x7f0ffc06e370 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f101c0b6730 tx=0x7f1004008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:16.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.262+0000 7f1018ff9640 1 -- 172.21.15.5:0/1132796075 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f100c059680 con 0x7f101c06d910 2024-05-20T18:33:16.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.483+0000 7f1021879640 1 -- 172.21.15.5:0/1132796075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f101c06d3d0 con 0x7f101c06d910 2024-05-20T18:33:16.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.484+0000 7f1018ff9640 1 -- 172.21.15.5:0/1132796075 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+5174 (secure 0 0 0) 0x7f100c058f80 con 0x7f101c06d910 2024-05-20T18:33:16.483 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:33:16.484 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3705772553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3705772553},{"type":"v1","addr":"172.21.15.63:6835","nonce":3705772553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":7434,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/776188138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":776188138},{"type":"v1","addr":"172.21.15.5:6835","nonce":776188138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":7437,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3556492262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3556492262},{"type":"v1","addr":"172.21.15.5:6837","nonce":3556492262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:33:12.457716+0000","modified":"2024-05-20T18:33:15.489024+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"d","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6834/2759396130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":2759396130},{"type":"v1","addr":"172.21.15.63:6834","nonce":2759396130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7449,"qdb_cluster":[7449]},"id":6}]} 2024-05-20T18:33:16.484 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 60 2024-05-20T18:33:16.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.485+0000 7f1021879640 1 -- 172.21.15.5:0/1132796075 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0ffc06be60 msgr2=0x7f0ffc06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:16.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.485+0000 7f1021879640 1 --2- 172.21.15.5:0/1132796075 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0ffc06be60 0x7f0ffc06e370 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f101c0b6730 tx=0x7f1004008040 comp rx=0 tx=0).stop 2024-05-20T18:33:16.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.485+0000 7f1021879640 1 -- 172.21.15.5:0/1132796075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f101c06d910 msgr2=0x7f101c1f8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:16.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.485+0000 7f1021879640 1 --2- 172.21.15.5:0/1132796075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f101c06d910 0x7f101c1f8580 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f100c00e870 tx=0x7f100c00ed30 comp rx=0 tx=0).stop 2024-05-20T18:33:16.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.486+0000 7f1021879640 1 -- 172.21.15.5:0/1132796075 shutdown_connections 2024-05-20T18:33:16.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.486+0000 7f1021879640 1 --2- 172.21.15.5:0/1132796075 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0ffc06be60 0x7f0ffc06e370 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.486+0000 7f1021879640 1 --2- 172.21.15.5:0/1132796075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f101c162820 0x7f101c0caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.486+0000 7f1021879640 1 --2- 172.21.15.5:0/1132796075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f101c0b3e80 0x7f101c0ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.486+0000 7f1021879640 1 --2- 172.21.15.5:0/1132796075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f101c06d910 0x7f101c1f8580 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.486+0000 7f1021879640 1 -- 172.21.15.5:0/1132796075 >> 172.21.15.5:0/1132796075 conn(0x7f101c0d4bd0 msgr2=0x7f101c0b9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:16.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.486+0000 7f1021879640 1 -- 172.21.15.5:0/1132796075 shutdown_connections 2024-05-20T18:33:16.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.486+0000 7f1021879640 1 -- 172.21.15.5:0/1132796075 wait complete. 2024-05-20T18:33:16.497 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:33:16.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.612+0000 7f3e18b37640 1 Processor -- start 2024-05-20T18:33:16.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.612+0000 7f3e18b37640 1 -- start start 2024-05-20T18:33:16.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.614+0000 7f3e18b37640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3e14162820 0x7f3e140b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:16.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.614+0000 7f3e18b37640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3e1406dae0 0x7f3e140b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:16.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.614+0000 7f3e18b37640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e140d0630 con 0x7f3e140b3e80 2024-05-20T18:33:16.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.614+0000 7f3e18b37640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f3e140d39c0 con 0x7f3e1406dae0 2024-05-20T18:33:16.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.614+0000 7f3e18b37640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f3e140d3ca0 con 0x7f3e14162820 2024-05-20T18:33:16.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.614+0000 7f3e12d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3e1406dae0 0x7f3e140b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:16.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.614+0000 7f3e12d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3e1406dae0 0x7f3e140b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58940/0 (socket says 172.21.15.5:58940) 2024-05-20T18:33:16.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.614+0000 7f3e12d9d640 1 -- 172.21.15.5:0/2613417034 learned_addr learned my addr 172.21.15.5:0/2613417034 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:16.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.614+0000 7f3e1259c640 1 --2- 172.21.15.5:0/2613417034 >> v2:172.21.15.63:3300/0 conn(0x7f3e14162820 0x7f3e140b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:16.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.614+0000 7f3e12d9d640 1 -- 172.21.15.5:0/2613417034 >> v2:172.21.15.63:3300/0 conn(0x7f3e14162820 msgr2=0x7f3e140b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:16.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.614+0000 7f3e12d9d640 1 --2- 172.21.15.5:0/2613417034 >> v2:172.21.15.63:3300/0 conn(0x7f3e14162820 0x7f3e140b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.614+0000 7f3e12d9d640 1 -- 172.21.15.5:0/2613417034 >> v1:172.21.15.63:6789/0 conn(0x7f3e140b3e80 legacy=0x7f3e1406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:16.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.614+0000 7f3e12d9d640 1 -- 172.21.15.5:0/2613417034 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e140d3f80 con 0x7f3e1406dae0 2024-05-20T18:33:16.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.614+0000 7f3e1259c640 1 --2- 172.21.15.5:0/2613417034 >> v2:172.21.15.63:3300/0 conn(0x7f3e14162820 0x7f3e140b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:33:16.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.615+0000 7f3e12d9d640 1 --2- 172.21.15.5:0/2613417034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1406dae0 0x7f3e140b8190 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f3e140b7740 tx=0x7f3e0c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1db14cbc4fbceb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:16.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.615+0000 7f3e1159a640 1 -- 172.21.15.5:0/2613417034 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c036070 con 0x7f3e1406dae0 2024-05-20T18:33:16.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.615+0000 7f3e1159a640 1 -- 172.21.15.5:0/2613417034 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e0c031600 con 0x7f3e1406dae0 2024-05-20T18:33:16.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.615+0000 7f3e1159a640 1 -- 172.21.15.5:0/2613417034 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c0319d0 con 0x7f3e1406dae0 2024-05-20T18:33:16.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.616+0000 7f3e18b37640 1 -- 172.21.15.5:0/2613417034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1406dae0 msgr2=0x7f3e140b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:16.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.616+0000 7f3e18b37640 1 --2- 172.21.15.5:0/2613417034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1406dae0 0x7f3e140b8190 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f3e140b7740 tx=0x7f3e0c02a6f0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.616+0000 7f3e18b37640 1 -- 172.21.15.5:0/2613417034 shutdown_connections 2024-05-20T18:33:16.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.616+0000 7f3e18b37640 1 --2- 172.21.15.5:0/2613417034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1406dae0 0x7f3e140b8190 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.616+0000 7f3e18b37640 1 --2- 172.21.15.5:0/2613417034 >> v2:172.21.15.63:3300/0 conn(0x7f3e14162820 0x7f3e140b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.616+0000 7f3e18b37640 1 -- 172.21.15.5:0/2613417034 >> 172.21.15.5:0/2613417034 conn(0x7f3e140d4bd0 msgr2=0x7f3e140d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:16.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.616+0000 7f3e18b37640 1 -- 172.21.15.5:0/2613417034 shutdown_connections 2024-05-20T18:33:16.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.616+0000 7f3e18b37640 1 -- 172.21.15.5:0/2613417034 wait complete. 2024-05-20T18:33:16.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.618+0000 7f3e18b37640 1 Processor -- start 2024-05-20T18:33:16.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.618+0000 7f3e18b37640 1 -- start start 2024-05-20T18:33:16.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.619+0000 7f3e18b37640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e140b3e80 0x7f3e141577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:16.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.619+0000 7f3e18b37640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e14162820 0x7f3e14155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:16.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.619+0000 7f3e18b37640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e140b85d0 0x7f3e140b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:16.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.619+0000 7f3e18b37640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3e140d4780 con 0x7f3e14162820 2024-05-20T18:33:16.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.619+0000 7f3e18b37640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3e140d0300 con 0x7f3e140b85d0 2024-05-20T18:33:16.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.619+0000 7f3e18b37640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3e140d4900 con 0x7f3e140b3e80 2024-05-20T18:33:16.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.619+0000 7f3e11d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e14162820 0x7f3e14155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:16.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.619+0000 7f3e11d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e14162820 0x7f3e14155ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58946/0 (socket says 172.21.15.5:58946) 2024-05-20T18:33:16.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.619+0000 7f3e11d9b640 1 -- 172.21.15.5:0/917155042 learned_addr learned my addr 172.21.15.5:0/917155042 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:16.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.619+0000 7f3e12d9d640 1 --2- 172.21.15.5:0/917155042 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e140b85d0 0x7f3e140b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:16.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.619+0000 7f3e1259c640 1 --2- 172.21.15.5:0/917155042 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e140b3e80 0x7f3e141577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:16.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.620+0000 7f3e11d9b640 1 -- 172.21.15.5:0/917155042 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e140b3e80 msgr2=0x7f3e141577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:16.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.620+0000 7f3e11d9b640 1 --2- 172.21.15.5:0/917155042 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e140b3e80 0x7f3e141577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.620+0000 7f3e11d9b640 1 -- 172.21.15.5:0/917155042 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e140b85d0 msgr2=0x7f3e140b89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:16.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.620+0000 7f3e11d9b640 1 --2- 172.21.15.5:0/917155042 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e140b85d0 0x7f3e140b89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.620+0000 7f3e11d9b640 1 -- 172.21.15.5:0/917155042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e14156f40 con 0x7f3e14162820 2024-05-20T18:33:16.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.620+0000 7f3e12d9d640 1 --2- 172.21.15.5:0/917155042 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e140b85d0 0x7f3e140b89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:16.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.620+0000 7f3e11d9b640 1 --2- 172.21.15.5:0/917155042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e14162820 0x7f3e14155ff0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f3e0800b410 tx=0x7f3e0800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:16.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.620+0000 7f3e02ffd640 1 -- 172.21.15.5:0/917155042 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e08015020 con 0x7f3e14162820 2024-05-20T18:33:16.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.620+0000 7f3e02ffd640 1 -- 172.21.15.5:0/917155042 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e08011830 con 0x7f3e14162820 2024-05-20T18:33:16.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.620+0000 7f3e18b37640 1 -- 172.21.15.5:0/917155042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e140ce260 con 0x7f3e14162820 2024-05-20T18:33:16.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.620+0000 7f3e02ffd640 1 -- 172.21.15.5:0/917155042 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e08011c20 con 0x7f3e14162820 2024-05-20T18:33:16.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.620+0000 7f3e18b37640 1 -- 172.21.15.5:0/917155042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e14157220 con 0x7f3e14162820 2024-05-20T18:33:16.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.621+0000 7f3e18b37640 1 -- 172.21.15.5:0/917155042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e14075b00 con 0x7f3e14162820 2024-05-20T18:33:16.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.622+0000 7f3e02ffd640 1 -- 172.21.15.5:0/917155042 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3e08018020 con 0x7f3e14162820 2024-05-20T18:33:16.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.622+0000 7f3e02ffd640 1 --2- 172.21.15.5:0/917155042 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3df406be60 0x7f3df406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:16.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.622+0000 7f3e02ffd640 1 -- 172.21.15.5:0/917155042 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6101+0+0 (secure 0 0 0) 0x7f3e0808af50 con 0x7f3e14162820 2024-05-20T18:33:16.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.622+0000 7f3e1259c640 1 --2- 172.21.15.5:0/917155042 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3df406be60 0x7f3df406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:16.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.623+0000 7f3e1259c640 1 --2- 172.21.15.5:0/917155042 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3df406be60 0x7f3df406e370 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400a950 tx=0x7f3e04008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:16.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.627+0000 7f3e02ffd640 1 -- 172.21.15.5:0/917155042 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3e0801f020 con 0x7f3e14162820 2024-05-20T18:33:16.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.810+0000 7f3e18b37640 1 -- 172.21.15.5:0/917155042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e141e94c0 con 0x7f3e14162820 2024-05-20T18:33:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.812+0000 7f3e02ffd640 1 -- 172.21.15.5:0/917155042 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+16765 (secure 0 0 0) 0x7f3e080172c0 con 0x7f3e14162820 2024-05-20T18:33:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:33:16.812 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":90,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:33:14.088906+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:33:10.184948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-20T18:33:11.453356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:33:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.814+0000 7f3e18b37640 1 -- 172.21.15.5:0/917155042 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3df406be60 msgr2=0x7f3df406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.814+0000 7f3e18b37640 1 --2- 172.21.15.5:0/917155042 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3df406be60 0x7f3df406e370 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400a950 tx=0x7f3e04008040 comp rx=0 tx=0).stop 2024-05-20T18:33:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.814+0000 7f3e18b37640 1 -- 172.21.15.5:0/917155042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e14162820 msgr2=0x7f3e14155ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.814+0000 7f3e18b37640 1 --2- 172.21.15.5:0/917155042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e14162820 0x7f3e14155ff0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f3e0800b410 tx=0x7f3e0800b8d0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.814+0000 7f3e18b37640 1 -- 172.21.15.5:0/917155042 shutdown_connections 2024-05-20T18:33:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.814+0000 7f3e18b37640 1 --2- 172.21.15.5:0/917155042 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3df406be60 0x7f3df406e370 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.814+0000 7f3e18b37640 1 --2- 172.21.15.5:0/917155042 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e140b85d0 0x7f3e140b89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.814+0000 7f3e18b37640 1 --2- 172.21.15.5:0/917155042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e14162820 0x7f3e14155ff0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.814+0000 7f3e18b37640 1 --2- 172.21.15.5:0/917155042 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e140b3e80 0x7f3e141577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.814+0000 7f3e18b37640 1 -- 172.21.15.5:0/917155042 >> 172.21.15.5:0/917155042 conn(0x7f3e140d4bd0 msgr2=0x7f3e140cee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.814+0000 7f3e18b37640 1 -- 172.21.15.5:0/917155042 shutdown_connections 2024-05-20T18:33:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.814+0000 7f3e18b37640 1 -- 172.21.15.5:0/917155042 wait complete. 2024-05-20T18:33:16.825 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-20T18:33:16.826 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T18:33:16.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.941+0000 7ffbbb7f5640 1 Processor -- start 2024-05-20T18:33:16.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.941+0000 7ffbbb7f5640 1 -- start start 2024-05-20T18:33:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.942+0000 7ffbbb7f5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ffbb40d1bc0 0x7ffbb40d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.942+0000 7ffbbb7f5640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbb40d0370 con 0x7ffbb4163a10 2024-05-20T18:33:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.942+0000 7ffbbb7f5640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbb40d0650 con 0x7ffbb40b4010 2024-05-20T18:33:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.942+0000 7ffbbb7f5640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7ffbb40d3ca0 con 0x7ffbb40d1bc0 2024-05-20T18:33:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.942+0000 7ffbb9591640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ffbb4163a10 0x7ffbb40cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35450/0 (socket says 172.21.15.5:35450) 2024-05-20T18:33:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.942+0000 7ffbb9d92640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ffbb40d1bc0 0x7ffbb40d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.942+0000 7ffbb9591640 1 -- 172.21.15.5:0/2224095432 learned_addr learned my addr 172.21.15.5:0/2224095432 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:33:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.943+0000 7ffbb9d92640 1 -- 172.21.15.5:0/2224095432 >> v1:172.21.15.63:6789/0 conn(0x7ffbb40b4010 legacy=0x7ffbb40d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.943+0000 7ffbb9d92640 1 -- 172.21.15.5:0/2224095432 >> v1:172.21.15.5:6789/0 conn(0x7ffbb4163a10 legacy=0x7ffbb40cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.943+0000 7ffbb9d92640 1 -- 172.21.15.5:0/2224095432 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb40d3650 con 0x7ffbb40d1bc0 2024-05-20T18:33:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.943+0000 7ffbabfff640 1 -- 172.21.15.5:0/2224095432 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2973401093 0 0) 0x7ffbb40d0370 con 0x7ffbb4163a10 2024-05-20T18:33:16.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.943+0000 7ffbb9d92640 1 --2- 172.21.15.5:0/2224095432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb40d1bc0 0x7ffbb40d4130 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7ffbac009d90 tx=0x7ffbac02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a1a9dc1ca90f8b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:16.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.944+0000 7ffbabfff640 1 -- 172.21.15.5:0/2224095432 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac007740 con 0x7ffbb40d1bc0 2024-05-20T18:33:16.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.944+0000 7ffbabfff640 1 -- 172.21.15.5:0/2224095432 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbac0078e0 con 0x7ffbb40d1bc0 2024-05-20T18:33:16.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.944+0000 7ffbabfff640 1 -- 172.21.15.5:0/2224095432 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac007cb0 con 0x7ffbb40d1bc0 2024-05-20T18:33:16.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.944+0000 7ffbbb7f5640 1 -- 172.21.15.5:0/2224095432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb40d1bc0 msgr2=0x7ffbb40d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:16.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.944+0000 7ffbbb7f5640 1 --2- 172.21.15.5:0/2224095432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb40d1bc0 0x7ffbb40d4130 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7ffbac009d90 tx=0x7ffbac02a6f0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.944+0000 7ffbbb7f5640 1 -- 172.21.15.5:0/2224095432 shutdown_connections 2024-05-20T18:33:16.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.944+0000 7ffbbb7f5640 1 --2- 172.21.15.5:0/2224095432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb40d1bc0 0x7ffbb40d4130 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.944+0000 7ffbbb7f5640 1 -- 172.21.15.5:0/2224095432 >> 172.21.15.5:0/2224095432 conn(0x7ffbb406d240 msgr2=0x7ffbb406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:16.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.944+0000 7ffbbb7f5640 1 -- 172.21.15.5:0/2224095432 shutdown_connections 2024-05-20T18:33:16.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.944+0000 7ffbbb7f5640 1 -- 172.21.15.5:0/2224095432 wait complete. 2024-05-20T18:33:16.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.946+0000 7ffbbb7f5640 1 Processor -- start 2024-05-20T18:33:16.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.946+0000 7ffbbb7f5640 1 -- start start 2024-05-20T18:33:16.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.947+0000 7ffbbb7f5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb40b4010 0x7ffbb41f5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:16.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.947+0000 7ffbbb7f5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbb40d1bc0 0x7ffbb4155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:16.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.947+0000 7ffbbb7f5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffbb4163a10 0x7ffbb41542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:16.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.947+0000 7ffbbb7f5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb40b8390 con 0x7ffbb40b4010 2024-05-20T18:33:16.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.947+0000 7ffbbb7f5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb40b8210 con 0x7ffbb40d1bc0 2024-05-20T18:33:16.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.947+0000 7ffbbb7f5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffbb40b8510 con 0x7ffbb4163a10 2024-05-20T18:33:16.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.947+0000 7ffbb9591640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb40b4010 0x7ffbb41f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:16.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.948+0000 7ffbb9591640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb40b4010 0x7ffbb41f5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58966/0 (socket says 172.21.15.5:58966) 2024-05-20T18:33:16.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.948+0000 7ffbb9591640 1 -- 172.21.15.5:0/2624323045 learned_addr learned my addr 172.21.15.5:0/2624323045 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:16.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.948+0000 7ffbb8d90640 1 --2- 172.21.15.5:0/2624323045 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbb40d1bc0 0x7ffbb4155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:16.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.948+0000 7ffbb9d92640 1 --2- 172.21.15.5:0/2624323045 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffbb4163a10 0x7ffbb41542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:16.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.948+0000 7ffbb9591640 1 -- 172.21.15.5:0/2624323045 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffbb4163a10 msgr2=0x7ffbb41542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:16.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.948+0000 7ffbb9591640 1 --2- 172.21.15.5:0/2624323045 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffbb4163a10 0x7ffbb41542e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.948+0000 7ffbb9591640 1 -- 172.21.15.5:0/2624323045 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbb40d1bc0 msgr2=0x7ffbb4155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:16.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.948+0000 7ffbb9591640 1 --2- 172.21.15.5:0/2624323045 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbb40d1bc0 0x7ffbb4155a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:16.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.948+0000 7ffbb9591640 1 -- 172.21.15.5:0/2624323045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb4154820 con 0x7ffbb40b4010 2024-05-20T18:33:16.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.948+0000 7ffbb9591640 1 --2- 172.21.15.5:0/2624323045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb40b4010 0x7ffbb41f5170 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000c3c0 tx=0x7ffbb000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:16.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.948+0000 7ffba9ffb640 1 -- 172.21.15.5:0/2624323045 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb00074e0 con 0x7ffbb40b4010 2024-05-20T18:33:16.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.948+0000 7ffba9ffb640 1 -- 172.21.15.5:0/2624323045 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbb00047e0 con 0x7ffbb40b4010 2024-05-20T18:33:16.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.948+0000 7ffbbb7f5640 1 -- 172.21.15.5:0/2624323045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb41563c0 con 0x7ffbb40b4010 2024-05-20T18:33:16.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.948+0000 7ffba9ffb640 1 -- 172.21.15.5:0/2624323045 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0004bd0 con 0x7ffbb40b4010 2024-05-20T18:33:16.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.948+0000 7ffbbb7f5640 1 -- 172.21.15.5:0/2624323045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb40b4b60 con 0x7ffbb40b4010 2024-05-20T18:33:16.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.949+0000 7ffbbb7f5640 1 -- 172.21.15.5:0/2624323045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbb4075b00 con 0x7ffbb40b4010 2024-05-20T18:33:16.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.950+0000 7ffba9ffb640 1 -- 172.21.15.5:0/2624323045 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffbb0011070 con 0x7ffbb40b4010 2024-05-20T18:33:16.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.951+0000 7ffba9ffb640 1 --2- 172.21.15.5:0/2624323045 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffb9806be60 0x7ffb9806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:16.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.951+0000 7ffba9ffb640 1 -- 172.21.15.5:0/2624323045 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6101+0+0 (secure 0 0 0) 0x7ffbb008ac10 con 0x7ffbb40b4010 2024-05-20T18:33:16.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.951+0000 7ffbb8d90640 1 --2- 172.21.15.5:0/2624323045 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffb9806be60 0x7ffb9806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:16.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.951+0000 7ffbb8d90640 1 --2- 172.21.15.5:0/2624323045 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffb9806be60 0x7ffb9806e370 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ffbb415e4c0 tx=0x7ffba4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:16.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:16.955+0000 7ffba9ffb640 1 -- 172.21.15.5:0/2624323045 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ffbb4154820 con 0x7ffbb40b4010 2024-05-20T18:33:17.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.135+0000 7ffbbb7f5640 1 -- 172.21.15.5:0/2624323045 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffbb40d3050 con 0x7ffb9806be60 2024-05-20T18:33:17.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.141+0000 7ffba9ffb640 1 -- 172.21.15.5:0/2624323045 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252074 (secure 0 0 0) 0x7ffba40558c0 con 0x7ffb9806be60 2024-05-20T18:33:17.145 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:33:17.146 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-20T18:33:17.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.145+0000 7ffbbb7f5640 1 -- 172.21.15.5:0/2624323045 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffb9806be60 msgr2=0x7ffb9806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:17.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.145+0000 7ffbbb7f5640 1 --2- 172.21.15.5:0/2624323045 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffb9806be60 0x7ffb9806e370 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ffbb415e4c0 tx=0x7ffba4009040 comp rx=0 tx=0).stop 2024-05-20T18:33:17.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.145+0000 7ffbbb7f5640 1 -- 172.21.15.5:0/2624323045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb40b4010 msgr2=0x7ffbb41f5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:17.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.145+0000 7ffbbb7f5640 1 --2- 172.21.15.5:0/2624323045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb40b4010 0x7ffbb41f5170 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000c3c0 tx=0x7ffbb000c880 comp rx=0 tx=0).stop 2024-05-20T18:33:17.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.145+0000 7ffbbb7f5640 1 -- 172.21.15.5:0/2624323045 shutdown_connections 2024-05-20T18:33:17.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.145+0000 7ffbbb7f5640 1 --2- 172.21.15.5:0/2624323045 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffb9806be60 0x7ffb9806e370 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:17.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.145+0000 7ffbbb7f5640 1 --2- 172.21.15.5:0/2624323045 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffbb4163a10 0x7ffbb41542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:17.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.145+0000 7ffbbb7f5640 1 --2- 172.21.15.5:0/2624323045 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbb40d1bc0 0x7ffbb4155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:17.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.145+0000 7ffbbb7f5640 1 --2- 172.21.15.5:0/2624323045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb40b4010 0x7ffbb41f5170 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:17.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.145+0000 7ffbbb7f5640 1 -- 172.21.15.5:0/2624323045 >> 172.21.15.5:0/2624323045 conn(0x7ffbb406d240 msgr2=0x7ffbb40d4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:17.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.145+0000 7ffbbb7f5640 1 -- 172.21.15.5:0/2624323045 shutdown_connections 2024-05-20T18:33:17.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.145+0000 7ffbbb7f5640 1 -- 172.21.15.5:0/2624323045 wait complete. 2024-05-20T18:33:17.158 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":271,"stamp":"2024-05-20T18:33:16.668213+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":177,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259388,"kb_used_data":9896,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715332,"statfs":{"total":773094113280,"available":772828499968,"internally_reserved":0,"allocated":10133504,"data_stored":5262268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.601547"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.096297+0000","last_change":"2024-05-20T18:33:13.113701+0000","last_active":"2024-05-20T18:33:14.096297+0000","last_peered":"2024-05-20T18:33:14.096297+0000","last_clean":"2024-05-20T18:33:14.096297+0000","last_became_active":"2024-05-20T18:33:13.113395+0000","last_became_peered":"2024-05-20T18:33:13.113395+0000","last_unstale":"2024-05-20T18:33:14.096297+0000","last_undegraded":"2024-05-20T18:33:14.096297+0000","last_fullsized":"2024-05-20T18:33:14.096297+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:14:33.945373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.104111+0000","last_change":"2024-05-20T18:33:13.100252+0000","last_active":"2024-05-20T18:33:14.104111+0000","last_peered":"2024-05-20T18:33:14.104111+0000","last_clean":"2024-05-20T18:33:14.104111+0000","last_became_active":"2024-05-20T18:33:13.099916+0000","last_became_peered":"2024-05-20T18:33:13.099916+0000","last_unstale":"2024-05-20T18:33:14.104111+0000","last_undegraded":"2024-05-20T18:33:14.104111+0000","last_fullsized":"2024-05-20T18:33:14.104111+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:33:29.819795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.096118+0000","last_change":"2024-05-20T18:33:13.115725+0000","last_active":"2024-05-20T18:33:14.096118+0000","last_peered":"2024-05-20T18:33:14.096118+0000","last_clean":"2024-05-20T18:33:14.096118+0000","last_became_active":"2024-05-20T18:33:13.115383+0000","last_became_peered":"2024-05-20T18:33:13.115383+0000","last_unstale":"2024-05-20T18:33:14.096118+0000","last_undegraded":"2024-05-20T18:33:14.096118+0000","last_fullsized":"2024-05-20T18:33:14.096118+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:24:00.139714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243983+0000","last_change":"2024-05-20T18:33:13.108686+0000","last_active":"2024-05-20T18:33:14.243983+0000","last_peered":"2024-05-20T18:33:14.243983+0000","last_clean":"2024-05-20T18:33:14.243983+0000","last_became_active":"2024-05-20T18:33:13.108291+0000","last_became_peered":"2024-05-20T18:33:13.108291+0000","last_unstale":"2024-05-20T18:33:14.243983+0000","last_undegraded":"2024-05-20T18:33:14.243983+0000","last_fullsized":"2024-05-20T18:33:14.243983+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:54:26.495327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.098858+0000","last_change":"2024-05-20T18:33:13.091713+0000","last_active":"2024-05-20T18:33:14.098858+0000","last_peered":"2024-05-20T18:33:14.098858+0000","last_clean":"2024-05-20T18:33:14.098858+0000","last_became_active":"2024-05-20T18:33:13.091374+0000","last_became_peered":"2024-05-20T18:33:13.091374+0000","last_unstale":"2024-05-20T18:33:14.098858+0000","last_undegraded":"2024-05-20T18:33:14.098858+0000","last_fullsized":"2024-05-20T18:33:14.098858+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:17:58.281816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.242675+0000","last_change":"2024-05-20T18:33:13.109151+0000","last_active":"2024-05-20T18:33:14.242675+0000","last_peered":"2024-05-20T18:33:14.242675+0000","last_clean":"2024-05-20T18:33:14.242675+0000","last_became_active":"2024-05-20T18:33:13.108479+0000","last_became_peered":"2024-05-20T18:33:13.108479+0000","last_unstale":"2024-05-20T18:33:14.242675+0000","last_undegraded":"2024-05-20T18:33:14.242675+0000","last_fullsized":"2024-05-20T18:33:14.242675+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:57:36.648081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243084+0000","last_change":"2024-05-20T18:33:13.099632+0000","last_active":"2024-05-20T18:33:14.243084+0000","last_peered":"2024-05-20T18:33:14.243084+0000","last_clean":"2024-05-20T18:33:14.243084+0000","last_became_active":"2024-05-20T18:33:13.099306+0000","last_became_peered":"2024-05-20T18:33:13.099306+0000","last_unstale":"2024-05-20T18:33:14.243084+0000","last_undegraded":"2024-05-20T18:33:14.243084+0000","last_fullsized":"2024-05-20T18:33:14.243084+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:33:24.687918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243732+0000","last_change":"2024-05-20T18:33:13.108305+0000","last_active":"2024-05-20T18:33:14.243732+0000","last_peered":"2024-05-20T18:33:14.243732+0000","last_clean":"2024-05-20T18:33:14.243732+0000","last_became_active":"2024-05-20T18:33:13.107989+0000","last_became_peered":"2024-05-20T18:33:13.107989+0000","last_unstale":"2024-05-20T18:33:14.243732+0000","last_undegraded":"2024-05-20T18:33:14.243732+0000","last_fullsized":"2024-05-20T18:33:14.243732+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:52:04.290712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.098654+0000","last_change":"2024-05-20T18:33:13.103879+0000","last_active":"2024-05-20T18:33:14.098654+0000","last_peered":"2024-05-20T18:33:14.098654+0000","last_clean":"2024-05-20T18:33:14.098654+0000","last_became_active":"2024-05-20T18:33:13.103550+0000","last_became_peered":"2024-05-20T18:33:13.103550+0000","last_unstale":"2024-05-20T18:33:14.098654+0000","last_undegraded":"2024-05-20T18:33:14.098654+0000","last_fullsized":"2024-05-20T18:33:14.098654+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:52:04.290712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.242865+0000","last_change":"2024-05-20T18:33:13.111617+0000","last_active":"2024-05-20T18:33:14.242865+0000","last_peered":"2024-05-20T18:33:14.242865+0000","last_clean":"2024-05-20T18:33:14.242865+0000","last_became_active":"2024-05-20T18:33:13.111178+0000","last_became_peered":"2024-05-20T18:33:13.111178+0000","last_unstale":"2024-05-20T18:33:14.242865+0000","last_undegraded":"2024-05-20T18:33:14.242865+0000","last_fullsized":"2024-05-20T18:33:14.242865+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:04:48.679542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.098765+0000","last_change":"2024-05-20T18:33:13.104475+0000","last_active":"2024-05-20T18:33:14.098765+0000","last_peered":"2024-05-20T18:33:14.098765+0000","last_clean":"2024-05-20T18:33:14.098765+0000","last_became_active":"2024-05-20T18:33:13.104143+0000","last_became_peered":"2024-05-20T18:33:13.104143+0000","last_unstale":"2024-05-20T18:33:14.098765+0000","last_undegraded":"2024-05-20T18:33:14.098765+0000","last_fullsized":"2024-05-20T18:33:14.098765+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:54:26.495327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.242521+0000","last_change":"2024-05-20T18:33:13.098050+0000","last_active":"2024-05-20T18:33:14.242521+0000","last_peered":"2024-05-20T18:33:14.242521+0000","last_clean":"2024-05-20T18:33:14.242521+0000","last_became_active":"2024-05-20T18:33:13.097707+0000","last_became_peered":"2024-05-20T18:33:13.097707+0000","last_unstale":"2024-05-20T18:33:14.242521+0000","last_undegraded":"2024-05-20T18:33:14.242521+0000","last_fullsized":"2024-05-20T18:33:14.242521+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:44:46.333079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243204+0000","last_change":"2024-05-20T18:33:13.104575+0000","last_active":"2024-05-20T18:33:14.243204+0000","last_peered":"2024-05-20T18:33:14.243204+0000","last_clean":"2024-05-20T18:33:14.243204+0000","last_became_active":"2024-05-20T18:33:13.104194+0000","last_became_peered":"2024-05-20T18:33:13.104194+0000","last_unstale":"2024-05-20T18:33:14.243204+0000","last_undegraded":"2024-05-20T18:33:14.243204+0000","last_fullsized":"2024-05-20T18:33:14.243204+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:47:24.701055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099465+0000","last_change":"2024-05-20T18:33:13.110314+0000","last_active":"2024-05-20T18:33:14.099465+0000","last_peered":"2024-05-20T18:33:14.099465+0000","last_clean":"2024-05-20T18:33:14.099465+0000","last_became_active":"2024-05-20T18:33:13.109911+0000","last_became_peered":"2024-05-20T18:33:13.109911+0000","last_unstale":"2024-05-20T18:33:14.099465+0000","last_undegraded":"2024-05-20T18:33:14.099465+0000","last_fullsized":"2024-05-20T18:33:14.099465+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:04:16.738506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243989+0000","last_change":"2024-05-20T18:33:13.102506+0000","last_active":"2024-05-20T18:33:14.243989+0000","last_peered":"2024-05-20T18:33:14.243989+0000","last_clean":"2024-05-20T18:33:14.243989+0000","last_became_active":"2024-05-20T18:33:13.102172+0000","last_became_peered":"2024-05-20T18:33:13.102172+0000","last_unstale":"2024-05-20T18:33:14.243989+0000","last_undegraded":"2024-05-20T18:33:14.243989+0000","last_fullsized":"2024-05-20T18:33:14.243989+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:17:58.281816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.242675+0000","last_change":"2024-05-20T18:33:13.101347+0000","last_active":"2024-05-20T18:33:14.242675+0000","last_peered":"2024-05-20T18:33:14.242675+0000","last_clean":"2024-05-20T18:33:14.242675+0000","last_became_active":"2024-05-20T18:33:13.100728+0000","last_became_peered":"2024-05-20T18:33:13.100728+0000","last_unstale":"2024-05-20T18:33:14.242675+0000","last_undegraded":"2024-05-20T18:33:14.242675+0000","last_fullsized":"2024-05-20T18:33:14.242675+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:55:04.812299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243523+0000","last_change":"2024-05-20T18:33:13.102752+0000","last_active":"2024-05-20T18:33:14.243523+0000","last_peered":"2024-05-20T18:33:14.243523+0000","last_clean":"2024-05-20T18:33:14.243523+0000","last_became_active":"2024-05-20T18:33:13.102458+0000","last_became_peered":"2024-05-20T18:33:13.102458+0000","last_unstale":"2024-05-20T18:33:14.243523+0000","last_undegraded":"2024-05-20T18:33:14.243523+0000","last_fullsized":"2024-05-20T18:33:14.243523+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:27:49.342593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.096390+0000","last_change":"2024-05-20T18:33:13.107657+0000","last_active":"2024-05-20T18:33:14.096390+0000","last_peered":"2024-05-20T18:33:14.096390+0000","last_clean":"2024-05-20T18:33:14.096390+0000","last_became_active":"2024-05-20T18:33:13.107328+0000","last_became_peered":"2024-05-20T18:33:13.107328+0000","last_unstale":"2024-05-20T18:33:14.096390+0000","last_undegraded":"2024-05-20T18:33:14.096390+0000","last_fullsized":"2024-05-20T18:33:14.096390+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:55:04.812299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.098967+0000","last_change":"2024-05-20T18:33:13.112305+0000","last_active":"2024-05-20T18:33:14.098967+0000","last_peered":"2024-05-20T18:33:14.098967+0000","last_clean":"2024-05-20T18:33:14.098967+0000","last_became_active":"2024-05-20T18:33:13.107659+0000","last_became_peered":"2024-05-20T18:33:13.107659+0000","last_unstale":"2024-05-20T18:33:14.098967+0000","last_undegraded":"2024-05-20T18:33:14.098967+0000","last_fullsized":"2024-05-20T18:33:14.098967+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:25:36.065702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.244119+0000","last_change":"2024-05-20T18:33:13.103127+0000","last_active":"2024-05-20T18:33:14.244119+0000","last_peered":"2024-05-20T18:33:14.244119+0000","last_clean":"2024-05-20T18:33:14.244119+0000","last_became_active":"2024-05-20T18:33:13.102756+0000","last_became_peered":"2024-05-20T18:33:13.102756+0000","last_unstale":"2024-05-20T18:33:14.244119+0000","last_undegraded":"2024-05-20T18:33:14.244119+0000","last_fullsized":"2024-05-20T18:33:14.244119+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:21:39.855675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.098563+0000","last_change":"2024-05-20T18:33:13.101453+0000","last_active":"2024-05-20T18:33:14.098563+0000","last_peered":"2024-05-20T18:33:14.098563+0000","last_clean":"2024-05-20T18:33:14.098563+0000","last_became_active":"2024-05-20T18:33:13.100767+0000","last_became_peered":"2024-05-20T18:33:13.100767+0000","last_unstale":"2024-05-20T18:33:14.098563+0000","last_undegraded":"2024-05-20T18:33:14.098563+0000","last_fullsized":"2024-05-20T18:33:14.098563+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:52:50.494627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099036+0000","last_change":"2024-05-20T18:33:13.109376+0000","last_active":"2024-05-20T18:33:14.099036+0000","last_peered":"2024-05-20T18:33:14.099036+0000","last_clean":"2024-05-20T18:33:14.099036+0000","last_became_active":"2024-05-20T18:33:13.108345+0000","last_became_peered":"2024-05-20T18:33:13.108345+0000","last_unstale":"2024-05-20T18:33:14.099036+0000","last_undegraded":"2024-05-20T18:33:14.099036+0000","last_fullsized":"2024-05-20T18:33:14.099036+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:05:02.724916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099547+0000","last_change":"2024-05-20T18:33:13.103036+0000","last_active":"2024-05-20T18:33:14.099547+0000","last_peered":"2024-05-20T18:33:14.099547+0000","last_clean":"2024-05-20T18:33:14.099547+0000","last_became_active":"2024-05-20T18:33:13.102553+0000","last_became_peered":"2024-05-20T18:33:13.102553+0000","last_unstale":"2024-05-20T18:33:14.099547+0000","last_undegraded":"2024-05-20T18:33:14.099547+0000","last_fullsized":"2024-05-20T18:33:14.099547+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:23:27.680097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.096158+0000","last_change":"2024-05-20T18:33:13.106263+0000","last_active":"2024-05-20T18:33:14.096158+0000","last_peered":"2024-05-20T18:33:14.096158+0000","last_clean":"2024-05-20T18:33:14.096158+0000","last_became_active":"2024-05-20T18:33:13.105927+0000","last_became_peered":"2024-05-20T18:33:13.105927+0000","last_unstale":"2024-05-20T18:33:14.096158+0000","last_undegraded":"2024-05-20T18:33:14.096158+0000","last_fullsized":"2024-05-20T18:33:14.096158+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:42:41.424610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.104211+0000","last_change":"2024-05-20T18:33:13.100803+0000","last_active":"2024-05-20T18:33:14.104211+0000","last_peered":"2024-05-20T18:33:14.104211+0000","last_clean":"2024-05-20T18:33:14.104211+0000","last_became_active":"2024-05-20T18:33:13.097534+0000","last_became_peered":"2024-05-20T18:33:13.097534+0000","last_unstale":"2024-05-20T18:33:14.104211+0000","last_undegraded":"2024-05-20T18:33:14.104211+0000","last_fullsized":"2024-05-20T18:33:14.104211+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:39:10.618607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.098714+0000","last_change":"2024-05-20T18:33:13.107870+0000","last_active":"2024-05-20T18:33:14.098714+0000","last_peered":"2024-05-20T18:33:14.098714+0000","last_clean":"2024-05-20T18:33:14.098714+0000","last_became_active":"2024-05-20T18:33:13.107530+0000","last_became_peered":"2024-05-20T18:33:13.107530+0000","last_unstale":"2024-05-20T18:33:14.098714+0000","last_undegraded":"2024-05-20T18:33:14.098714+0000","last_fullsized":"2024-05-20T18:33:14.098714+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:18:03.936593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.242887+0000","last_change":"2024-05-20T18:33:13.097688+0000","last_active":"2024-05-20T18:33:14.242887+0000","last_peered":"2024-05-20T18:33:14.242887+0000","last_clean":"2024-05-20T18:33:14.242887+0000","last_became_active":"2024-05-20T18:33:13.097322+0000","last_became_peered":"2024-05-20T18:33:13.097322+0000","last_unstale":"2024-05-20T18:33:14.242887+0000","last_undegraded":"2024-05-20T18:33:14.242887+0000","last_fullsized":"2024-05-20T18:33:14.242887+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:31:51.783828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.244171+0000","last_change":"2024-05-20T18:33:13.107890+0000","last_active":"2024-05-20T18:33:14.244171+0000","last_peered":"2024-05-20T18:33:14.244171+0000","last_clean":"2024-05-20T18:33:14.244171+0000","last_became_active":"2024-05-20T18:33:13.107533+0000","last_became_peered":"2024-05-20T18:33:13.107533+0000","last_unstale":"2024-05-20T18:33:14.244171+0000","last_undegraded":"2024-05-20T18:33:14.244171+0000","last_fullsized":"2024-05-20T18:33:14.244171+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:25:36.065702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243276+0000","last_change":"2024-05-20T18:33:13.102602+0000","last_active":"2024-05-20T18:33:14.243276+0000","last_peered":"2024-05-20T18:33:14.243276+0000","last_clean":"2024-05-20T18:33:14.243276+0000","last_became_active":"2024-05-20T18:33:13.102178+0000","last_became_peered":"2024-05-20T18:33:13.102178+0000","last_unstale":"2024-05-20T18:33:14.243276+0000","last_undegraded":"2024-05-20T18:33:14.243276+0000","last_fullsized":"2024-05-20T18:33:14.243276+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:37:16.850745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099337+0000","last_change":"2024-05-20T18:33:13.096634+0000","last_active":"2024-05-20T18:33:14.099337+0000","last_peered":"2024-05-20T18:33:14.099337+0000","last_clean":"2024-05-20T18:33:14.099337+0000","last_became_active":"2024-05-20T18:33:13.095730+0000","last_became_peered":"2024-05-20T18:33:13.095730+0000","last_unstale":"2024-05-20T18:33:14.099337+0000","last_undegraded":"2024-05-20T18:33:14.099337+0000","last_fullsized":"2024-05-20T18:33:14.099337+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:46:57.728633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.103747+0000","last_change":"2024-05-20T18:33:13.107961+0000","last_active":"2024-05-20T18:33:14.103747+0000","last_peered":"2024-05-20T18:33:14.103747+0000","last_clean":"2024-05-20T18:33:14.103747+0000","last_became_active":"2024-05-20T18:33:13.107609+0000","last_became_peered":"2024-05-20T18:33:13.107609+0000","last_unstale":"2024-05-20T18:33:14.103747+0000","last_undegraded":"2024-05-20T18:33:14.103747+0000","last_fullsized":"2024-05-20T18:33:14.103747+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:35:56.983794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.242847+0000","last_change":"2024-05-20T18:33:13.101086+0000","last_active":"2024-05-20T18:33:14.242847+0000","last_peered":"2024-05-20T18:33:14.242847+0000","last_clean":"2024-05-20T18:33:14.242847+0000","last_became_active":"2024-05-20T18:33:13.100289+0000","last_became_peered":"2024-05-20T18:33:13.100289+0000","last_unstale":"2024-05-20T18:33:14.242847+0000","last_undegraded":"2024-05-20T18:33:14.242847+0000","last_fullsized":"2024-05-20T18:33:14.242847+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:17:03.547006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243945+0000","last_change":"2024-05-20T18:33:13.100725+0000","last_active":"2024-05-20T18:33:14.243945+0000","last_peered":"2024-05-20T18:33:14.243945+0000","last_clean":"2024-05-20T18:33:14.243945+0000","last_became_active":"2024-05-20T18:33:13.099258+0000","last_became_peered":"2024-05-20T18:33:13.099258+0000","last_unstale":"2024-05-20T18:33:14.243945+0000","last_undegraded":"2024-05-20T18:33:14.243945+0000","last_fullsized":"2024-05-20T18:33:14.243945+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:56:50.410289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"89'4","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099900+0000","last_change":"2024-05-20T18:33:12.094249+0000","last_active":"2024-05-20T18:33:14.099900+0000","last_peered":"2024-05-20T18:33:14.099900+0000","last_clean":"2024-05-20T18:33:14.099900+0000","last_became_active":"2024-05-20T18:33:12.093887+0000","last_became_peered":"2024-05-20T18:33:12.093887+0000","last_unstale":"2024-05-20T18:33:14.099900+0000","last_undegraded":"2024-05-20T18:33:14.099900+0000","last_fullsized":"2024-05-20T18:33:14.099900+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:35:13.707739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243190+0000","last_change":"2024-05-20T18:33:13.100911+0000","last_active":"2024-05-20T18:33:14.243190+0000","last_peered":"2024-05-20T18:33:14.243190+0000","last_clean":"2024-05-20T18:33:14.243190+0000","last_became_active":"2024-05-20T18:33:13.099691+0000","last_became_peered":"2024-05-20T18:33:13.099691+0000","last_unstale":"2024-05-20T18:33:14.243190+0000","last_undegraded":"2024-05-20T18:33:14.243190+0000","last_fullsized":"2024-05-20T18:33:14.243190+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:31:40.148877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.098997+0000","last_change":"2024-05-20T18:33:12.083543+0000","last_active":"2024-05-20T18:33:14.098997+0000","last_peered":"2024-05-20T18:33:14.098997+0000","last_clean":"2024-05-20T18:33:14.098997+0000","last_became_active":"2024-05-20T18:33:12.083187+0000","last_became_peered":"2024-05-20T18:33:12.083187+0000","last_unstale":"2024-05-20T18:33:14.098997+0000","last_undegraded":"2024-05-20T18:33:14.098997+0000","last_fullsized":"2024-05-20T18:33:14.098997+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:29:09.903414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243415+0000","last_change":"2024-05-20T18:33:13.110254+0000","last_active":"2024-05-20T18:33:14.243415+0000","last_peered":"2024-05-20T18:33:14.243415+0000","last_clean":"2024-05-20T18:33:14.243415+0000","last_became_active":"2024-05-20T18:33:13.109930+0000","last_became_peered":"2024-05-20T18:33:13.109930+0000","last_unstale":"2024-05-20T18:33:14.243415+0000","last_undegraded":"2024-05-20T18:33:14.243415+0000","last_fullsized":"2024-05-20T18:33:14.243415+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:51:29.485838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243109+0000","last_change":"2024-05-20T18:33:12.090717+0000","last_active":"2024-05-20T18:33:14.243109+0000","last_peered":"2024-05-20T18:33:14.243109+0000","last_clean":"2024-05-20T18:33:14.243109+0000","last_became_active":"2024-05-20T18:33:12.090282+0000","last_became_peered":"2024-05-20T18:33:12.090282+0000","last_unstale":"2024-05-20T18:33:14.243109+0000","last_undegraded":"2024-05-20T18:33:14.243109+0000","last_fullsized":"2024-05-20T18:33:14.243109+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:43:08.640518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243680+0000","last_change":"2024-05-20T18:33:13.092762+0000","last_active":"2024-05-20T18:33:14.243680+0000","last_peered":"2024-05-20T18:33:14.243680+0000","last_clean":"2024-05-20T18:33:14.243680+0000","last_became_active":"2024-05-20T18:33:13.092433+0000","last_became_peered":"2024-05-20T18:33:13.092433+0000","last_unstale":"2024-05-20T18:33:14.243680+0000","last_undegraded":"2024-05-20T18:33:14.243680+0000","last_fullsized":"2024-05-20T18:33:14.243680+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:45:11.338040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099225+0000","last_change":"2024-05-20T18:33:12.093122+0000","last_active":"2024-05-20T18:33:14.099225+0000","last_peered":"2024-05-20T18:33:14.099225+0000","last_clean":"2024-05-20T18:33:14.099225+0000","last_became_active":"2024-05-20T18:33:12.092754+0000","last_became_peered":"2024-05-20T18:33:12.092754+0000","last_unstale":"2024-05-20T18:33:14.099225+0000","last_undegraded":"2024-05-20T18:33:14.099225+0000","last_fullsized":"2024-05-20T18:33:14.099225+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:56:13.575392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243939+0000","last_change":"2024-05-20T18:33:13.101534+0000","last_active":"2024-05-20T18:33:14.243939+0000","last_peered":"2024-05-20T18:33:14.243939+0000","last_clean":"2024-05-20T18:33:14.243939+0000","last_became_active":"2024-05-20T18:33:13.101146+0000","last_became_peered":"2024-05-20T18:33:13.101146+0000","last_unstale":"2024-05-20T18:33:14.243939+0000","last_undegraded":"2024-05-20T18:33:14.243939+0000","last_fullsized":"2024-05-20T18:33:14.243939+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:18:03.936593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.098935+0000","last_change":"2024-05-20T18:33:12.092546+0000","last_active":"2024-05-20T18:33:14.098935+0000","last_peered":"2024-05-20T18:33:14.098935+0000","last_clean":"2024-05-20T18:33:14.098935+0000","last_became_active":"2024-05-20T18:33:12.092003+0000","last_became_peered":"2024-05-20T18:33:12.092003+0000","last_unstale":"2024-05-20T18:33:14.098935+0000","last_undegraded":"2024-05-20T18:33:14.098935+0000","last_fullsized":"2024-05-20T18:33:14.098935+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:21:54.999282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099599+0000","last_change":"2024-05-20T18:33:13.089420+0000","last_active":"2024-05-20T18:33:14.099599+0000","last_peered":"2024-05-20T18:33:14.099599+0000","last_clean":"2024-05-20T18:33:14.099599+0000","last_became_active":"2024-05-20T18:33:13.088884+0000","last_became_peered":"2024-05-20T18:33:13.088884+0000","last_unstale":"2024-05-20T18:33:14.099599+0000","last_undegraded":"2024-05-20T18:33:14.099599+0000","last_fullsized":"2024-05-20T18:33:14.099599+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:18:03.936593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.244023+0000","last_change":"2024-05-20T18:33:12.091475+0000","last_active":"2024-05-20T18:33:14.244023+0000","last_peered":"2024-05-20T18:33:14.244023+0000","last_clean":"2024-05-20T18:33:14.244023+0000","last_became_active":"2024-05-20T18:33:12.091194+0000","last_became_peered":"2024-05-20T18:33:12.091194+0000","last_unstale":"2024-05-20T18:33:14.244023+0000","last_undegraded":"2024-05-20T18:33:14.244023+0000","last_fullsized":"2024-05-20T18:33:14.244023+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:33:26.376581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243064+0000","last_change":"2024-05-20T18:33:13.108254+0000","last_active":"2024-05-20T18:33:14.243064+0000","last_peered":"2024-05-20T18:33:14.243064+0000","last_clean":"2024-05-20T18:33:14.243064+0000","last_became_active":"2024-05-20T18:33:13.107875+0000","last_became_peered":"2024-05-20T18:33:13.107875+0000","last_unstale":"2024-05-20T18:33:14.243064+0000","last_undegraded":"2024-05-20T18:33:14.243064+0000","last_fullsized":"2024-05-20T18:33:14.243064+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:08:30.714625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.096691+0000","last_change":"2024-05-20T18:33:12.085691+0000","last_active":"2024-05-20T18:33:14.096691+0000","last_peered":"2024-05-20T18:33:14.096691+0000","last_clean":"2024-05-20T18:33:14.096691+0000","last_became_active":"2024-05-20T18:33:12.085396+0000","last_became_peered":"2024-05-20T18:33:12.085396+0000","last_unstale":"2024-05-20T18:33:14.096691+0000","last_undegraded":"2024-05-20T18:33:14.096691+0000","last_fullsized":"2024-05-20T18:33:14.096691+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:27:27.616696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243374+0000","last_change":"2024-05-20T18:33:13.102297+0000","last_active":"2024-05-20T18:33:14.243374+0000","last_peered":"2024-05-20T18:33:14.243374+0000","last_clean":"2024-05-20T18:33:14.243374+0000","last_became_active":"2024-05-20T18:33:13.101934+0000","last_became_peered":"2024-05-20T18:33:13.101934+0000","last_unstale":"2024-05-20T18:33:14.243374+0000","last_undegraded":"2024-05-20T18:33:14.243374+0000","last_fullsized":"2024-05-20T18:33:14.243374+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:16:15.456403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.242904+0000","last_change":"2024-05-20T18:33:12.094291+0000","last_active":"2024-05-20T18:33:14.242904+0000","last_peered":"2024-05-20T18:33:14.242904+0000","last_clean":"2024-05-20T18:33:14.242904+0000","last_became_active":"2024-05-20T18:33:12.093942+0000","last_became_peered":"2024-05-20T18:33:12.093942+0000","last_unstale":"2024-05-20T18:33:14.242904+0000","last_undegraded":"2024-05-20T18:33:14.242904+0000","last_fullsized":"2024-05-20T18:33:14.242904+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:44:05.126905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.098667+0000","last_change":"2024-05-20T18:33:13.099069+0000","last_active":"2024-05-20T18:33:14.098667+0000","last_peered":"2024-05-20T18:33:14.098667+0000","last_clean":"2024-05-20T18:33:14.098667+0000","last_became_active":"2024-05-20T18:33:13.098717+0000","last_became_peered":"2024-05-20T18:33:13.098717+0000","last_unstale":"2024-05-20T18:33:14.098667+0000","last_undegraded":"2024-05-20T18:33:14.098667+0000","last_fullsized":"2024-05-20T18:33:14.098667+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:23:27.680097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.104255+0000","last_change":"2024-05-20T18:33:12.092274+0000","last_active":"2024-05-20T18:33:14.104255+0000","last_peered":"2024-05-20T18:33:14.104255+0000","last_clean":"2024-05-20T18:33:14.104255+0000","last_became_active":"2024-05-20T18:33:12.091927+0000","last_became_peered":"2024-05-20T18:33:12.091927+0000","last_unstale":"2024-05-20T18:33:14.104255+0000","last_undegraded":"2024-05-20T18:33:14.104255+0000","last_fullsized":"2024-05-20T18:33:14.104255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:25:35.060122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243377+0000","last_change":"2024-05-20T18:33:13.107302+0000","last_active":"2024-05-20T18:33:14.243377+0000","last_peered":"2024-05-20T18:33:14.243377+0000","last_clean":"2024-05-20T18:33:14.243377+0000","last_became_active":"2024-05-20T18:33:13.106926+0000","last_became_peered":"2024-05-20T18:33:13.106926+0000","last_unstale":"2024-05-20T18:33:14.243377+0000","last_undegraded":"2024-05-20T18:33:14.243377+0000","last_fullsized":"2024-05-20T18:33:14.243377+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:07:50.193612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.096444+0000","last_change":"2024-05-20T18:33:12.092923+0000","last_active":"2024-05-20T18:33:14.096444+0000","last_peered":"2024-05-20T18:33:14.096444+0000","last_clean":"2024-05-20T18:33:14.096444+0000","last_became_active":"2024-05-20T18:33:12.092632+0000","last_became_peered":"2024-05-20T18:33:12.092632+0000","last_unstale":"2024-05-20T18:33:14.096444+0000","last_undegraded":"2024-05-20T18:33:14.096444+0000","last_fullsized":"2024-05-20T18:33:14.096444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:35:47.930046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.096530+0000","last_change":"2024-05-20T18:33:13.114561+0000","last_active":"2024-05-20T18:33:14.096530+0000","last_peered":"2024-05-20T18:33:14.096530+0000","last_clean":"2024-05-20T18:33:14.096530+0000","last_became_active":"2024-05-20T18:33:13.114253+0000","last_became_peered":"2024-05-20T18:33:13.114253+0000","last_unstale":"2024-05-20T18:33:14.096530+0000","last_undegraded":"2024-05-20T18:33:14.096530+0000","last_fullsized":"2024-05-20T18:33:14.096530+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:57:36.648081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"89'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099058+0000","last_change":"2024-05-20T18:33:12.085648+0000","last_active":"2024-05-20T18:33:14.099058+0000","last_peered":"2024-05-20T18:33:14.099058+0000","last_clean":"2024-05-20T18:33:14.099058+0000","last_became_active":"2024-05-20T18:33:12.085224+0000","last_became_peered":"2024-05-20T18:33:12.085224+0000","last_unstale":"2024-05-20T18:33:14.099058+0000","last_undegraded":"2024-05-20T18:33:14.099058+0000","last_fullsized":"2024-05-20T18:33:14.099058+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:12:03.967934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243446+0000","last_change":"2024-05-20T18:33:13.118642+0000","last_active":"2024-05-20T18:33:14.243446+0000","last_peered":"2024-05-20T18:33:14.243446+0000","last_clean":"2024-05-20T18:33:14.243446+0000","last_became_active":"2024-05-20T18:33:13.118314+0000","last_became_peered":"2024-05-20T18:33:13.118314+0000","last_unstale":"2024-05-20T18:33:14.243446+0000","last_undegraded":"2024-05-20T18:33:14.243446+0000","last_fullsized":"2024-05-20T18:33:14.243446+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:52:50.494627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"89'6","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.244004+0000","last_change":"2024-05-20T18:33:12.097189+0000","last_active":"2024-05-20T18:33:14.244004+0000","last_peered":"2024-05-20T18:33:14.244004+0000","last_clean":"2024-05-20T18:33:14.244004+0000","last_became_active":"2024-05-20T18:33:12.096897+0000","last_became_peered":"2024-05-20T18:33:12.096897+0000","last_unstale":"2024-05-20T18:33:14.244004+0000","last_undegraded":"2024-05-20T18:33:14.244004+0000","last_fullsized":"2024-05-20T18:33:14.244004+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:08:20.867030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243924+0000","last_change":"2024-05-20T18:33:13.106684+0000","last_active":"2024-05-20T18:33:14.243924+0000","last_peered":"2024-05-20T18:33:14.243924+0000","last_clean":"2024-05-20T18:33:14.243924+0000","last_became_active":"2024-05-20T18:33:13.106354+0000","last_became_peered":"2024-05-20T18:33:13.106354+0000","last_unstale":"2024-05-20T18:33:14.243924+0000","last_undegraded":"2024-05-20T18:33:14.243924+0000","last_fullsized":"2024-05-20T18:33:14.243924+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:23:27.680097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243199+0000","last_change":"2024-05-20T18:33:12.094515+0000","last_active":"2024-05-20T18:33:14.243199+0000","last_peered":"2024-05-20T18:33:14.243199+0000","last_clean":"2024-05-20T18:33:14.243199+0000","last_became_active":"2024-05-20T18:33:12.093783+0000","last_became_peered":"2024-05-20T18:33:12.093783+0000","last_unstale":"2024-05-20T18:33:14.243199+0000","last_undegraded":"2024-05-20T18:33:14.243199+0000","last_fullsized":"2024-05-20T18:33:14.243199+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:49:59.457852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.098291+0000","last_change":"2024-05-20T18:33:13.097112+0000","last_active":"2024-05-20T18:33:14.098291+0000","last_peered":"2024-05-20T18:33:14.098291+0000","last_clean":"2024-05-20T18:33:14.098291+0000","last_became_active":"2024-05-20T18:33:13.096761+0000","last_became_peered":"2024-05-20T18:33:13.096761+0000","last_unstale":"2024-05-20T18:33:14.098291+0000","last_undegraded":"2024-05-20T18:33:14.098291+0000","last_fullsized":"2024-05-20T18:33:14.098291+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:16:15.456403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243123+0000","last_change":"2024-05-20T18:33:12.089463+0000","last_active":"2024-05-20T18:33:14.243123+0000","last_peered":"2024-05-20T18:33:14.243123+0000","last_clean":"2024-05-20T18:33:14.243123+0000","last_became_active":"2024-05-20T18:33:12.089162+0000","last_became_peered":"2024-05-20T18:33:12.089162+0000","last_unstale":"2024-05-20T18:33:14.243123+0000","last_undegraded":"2024-05-20T18:33:14.243123+0000","last_fullsized":"2024-05-20T18:33:14.243123+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:37:15.845164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.242891+0000","last_change":"2024-05-20T18:33:13.112356+0000","last_active":"2024-05-20T18:33:14.242891+0000","last_peered":"2024-05-20T18:33:14.242891+0000","last_clean":"2024-05-20T18:33:14.242891+0000","last_became_active":"2024-05-20T18:33:13.112058+0000","last_became_peered":"2024-05-20T18:33:13.112058+0000","last_unstale":"2024-05-20T18:33:14.242891+0000","last_undegraded":"2024-05-20T18:33:14.242891+0000","last_fullsized":"2024-05-20T18:33:14.242891+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:18:24.988974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243581+0000","last_change":"2024-05-20T18:33:12.091927+0000","last_active":"2024-05-20T18:33:14.243581+0000","last_peered":"2024-05-20T18:33:14.243581+0000","last_clean":"2024-05-20T18:33:14.243581+0000","last_became_active":"2024-05-20T18:33:12.091635+0000","last_became_peered":"2024-05-20T18:33:12.091635+0000","last_unstale":"2024-05-20T18:33:14.243581+0000","last_undegraded":"2024-05-20T18:33:14.243581+0000","last_fullsized":"2024-05-20T18:33:14.243581+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:40:01.877694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.242788+0000","last_change":"2024-05-20T18:33:13.110476+0000","last_active":"2024-05-20T18:33:14.242788+0000","last_peered":"2024-05-20T18:33:14.242788+0000","last_clean":"2024-05-20T18:33:14.242788+0000","last_became_active":"2024-05-20T18:33:13.110145+0000","last_became_peered":"2024-05-20T18:33:13.110145+0000","last_unstale":"2024-05-20T18:33:14.242788+0000","last_undegraded":"2024-05-20T18:33:14.242788+0000","last_fullsized":"2024-05-20T18:33:14.242788+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:36:26.017336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"89'8","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.244007+0000","last_change":"2024-05-20T18:33:12.091517+0000","last_active":"2024-05-20T18:33:14.244007+0000","last_peered":"2024-05-20T18:33:14.244007+0000","last_clean":"2024-05-20T18:33:14.244007+0000","last_became_active":"2024-05-20T18:33:12.091249+0000","last_became_peered":"2024-05-20T18:33:12.091249+0000","last_unstale":"2024-05-20T18:33:14.244007+0000","last_undegraded":"2024-05-20T18:33:14.244007+0000","last_fullsized":"2024-05-20T18:33:14.244007+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:30:43.896361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.244016+0000","last_change":"2024-05-20T18:33:13.099468+0000","last_active":"2024-05-20T18:33:14.244016+0000","last_peered":"2024-05-20T18:33:14.244016+0000","last_clean":"2024-05-20T18:33:14.244016+0000","last_became_active":"2024-05-20T18:33:13.098799+0000","last_became_peered":"2024-05-20T18:33:13.098799+0000","last_unstale":"2024-05-20T18:33:14.244016+0000","last_undegraded":"2024-05-20T18:33:14.244016+0000","last_fullsized":"2024-05-20T18:33:14.244016+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:06:40.101331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099952+0000","last_change":"2024-05-20T18:33:12.090460+0000","last_active":"2024-05-20T18:33:14.099952+0000","last_peered":"2024-05-20T18:33:14.099952+0000","last_clean":"2024-05-20T18:33:14.099952+0000","last_became_active":"2024-05-20T18:33:12.090114+0000","last_became_peered":"2024-05-20T18:33:12.090114+0000","last_unstale":"2024-05-20T18:33:14.099952+0000","last_undegraded":"2024-05-20T18:33:14.099952+0000","last_fullsized":"2024-05-20T18:33:14.099952+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:41:08.634509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099189+0000","last_change":"2024-05-20T18:33:13.095643+0000","last_active":"2024-05-20T18:33:14.099189+0000","last_peered":"2024-05-20T18:33:14.099189+0000","last_clean":"2024-05-20T18:33:14.099189+0000","last_became_active":"2024-05-20T18:33:13.095269+0000","last_became_peered":"2024-05-20T18:33:13.095269+0000","last_unstale":"2024-05-20T18:33:14.099189+0000","last_undegraded":"2024-05-20T18:33:14.099189+0000","last_fullsized":"2024-05-20T18:33:14.099189+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:52:04.290712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243391+0000","last_change":"2024-05-20T18:33:12.092847+0000","last_active":"2024-05-20T18:33:14.243391+0000","last_peered":"2024-05-20T18:33:14.243391+0000","last_clean":"2024-05-20T18:33:14.243391+0000","last_became_active":"2024-05-20T18:33:12.092515+0000","last_became_peered":"2024-05-20T18:33:12.092515+0000","last_unstale":"2024-05-20T18:33:14.243391+0000","last_undegraded":"2024-05-20T18:33:14.243391+0000","last_fullsized":"2024-05-20T18:33:14.243391+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:56:20.494612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.098914+0000","last_change":"2024-05-20T18:33:13.093718+0000","last_active":"2024-05-20T18:33:14.098914+0000","last_peered":"2024-05-20T18:33:14.098914+0000","last_clean":"2024-05-20T18:33:14.098914+0000","last_became_active":"2024-05-20T18:33:13.093356+0000","last_became_peered":"2024-05-20T18:33:13.093356+0000","last_unstale":"2024-05-20T18:33:14.098914+0000","last_undegraded":"2024-05-20T18:33:14.098914+0000","last_fullsized":"2024-05-20T18:33:14.098914+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:21:39.855675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099354+0000","last_change":"2024-05-20T18:33:12.094666+0000","last_active":"2024-05-20T18:33:14.099354+0000","last_peered":"2024-05-20T18:33:14.099354+0000","last_clean":"2024-05-20T18:33:14.099354+0000","last_became_active":"2024-05-20T18:33:12.094351+0000","last_became_peered":"2024-05-20T18:33:12.094351+0000","last_unstale":"2024-05-20T18:33:14.099354+0000","last_undegraded":"2024-05-20T18:33:14.099354+0000","last_fullsized":"2024-05-20T18:33:14.099354+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:33:26.376581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243839+0000","last_change":"2024-05-20T18:33:13.109890+0000","last_active":"2024-05-20T18:33:14.243839+0000","last_peered":"2024-05-20T18:33:14.243839+0000","last_clean":"2024-05-20T18:33:14.243839+0000","last_became_active":"2024-05-20T18:33:13.109179+0000","last_became_peered":"2024-05-20T18:33:13.109179+0000","last_unstale":"2024-05-20T18:33:14.243839+0000","last_undegraded":"2024-05-20T18:33:14.243839+0000","last_fullsized":"2024-05-20T18:33:14.243839+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:22:26.619923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.103894+0000","last_change":"2024-05-20T18:33:12.091317+0000","last_active":"2024-05-20T18:33:14.103894+0000","last_peered":"2024-05-20T18:33:14.103894+0000","last_clean":"2024-05-20T18:33:14.103894+0000","last_became_active":"2024-05-20T18:33:12.089309+0000","last_became_peered":"2024-05-20T18:33:12.089309+0000","last_unstale":"2024-05-20T18:33:14.103894+0000","last_undegraded":"2024-05-20T18:33:14.103894+0000","last_fullsized":"2024-05-20T18:33:14.103894+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:54:25.489746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.103967+0000","last_change":"2024-05-20T18:33:13.102528+0000","last_active":"2024-05-20T18:33:14.103967+0000","last_peered":"2024-05-20T18:33:14.103967+0000","last_clean":"2024-05-20T18:33:14.103967+0000","last_became_active":"2024-05-20T18:33:13.102223+0000","last_became_peered":"2024-05-20T18:33:13.102223+0000","last_unstale":"2024-05-20T18:33:14.103967+0000","last_undegraded":"2024-05-20T18:33:14.103967+0000","last_fullsized":"2024-05-20T18:33:14.103967+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:56:21.500193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.242829+0000","last_change":"2024-05-20T18:33:12.091634+0000","last_active":"2024-05-20T18:33:14.242829+0000","last_peered":"2024-05-20T18:33:14.242829+0000","last_clean":"2024-05-20T18:33:14.242829+0000","last_became_active":"2024-05-20T18:33:12.091365+0000","last_became_peered":"2024-05-20T18:33:12.091365+0000","last_unstale":"2024-05-20T18:33:14.242829+0000","last_undegraded":"2024-05-20T18:33:14.242829+0000","last_fullsized":"2024-05-20T18:33:14.242829+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:15:30.398201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.103825+0000","last_change":"2024-05-20T18:33:13.108416+0000","last_active":"2024-05-20T18:33:14.103825+0000","last_peered":"2024-05-20T18:33:14.103825+0000","last_clean":"2024-05-20T18:33:14.103825+0000","last_became_active":"2024-05-20T18:33:13.107616+0000","last_became_peered":"2024-05-20T18:33:13.107616+0000","last_unstale":"2024-05-20T18:33:14.103825+0000","last_undegraded":"2024-05-20T18:33:14.103825+0000","last_fullsized":"2024-05-20T18:33:14.103825+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:50:00.463433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.103829+0000","last_change":"2024-05-20T18:33:12.087884+0000","last_active":"2024-05-20T18:33:14.103829+0000","last_peered":"2024-05-20T18:33:14.103829+0000","last_clean":"2024-05-20T18:33:14.103829+0000","last_became_active":"2024-05-20T18:33:12.087526+0000","last_became_peered":"2024-05-20T18:33:12.087526+0000","last_unstale":"2024-05-20T18:33:14.103829+0000","last_undegraded":"2024-05-20T18:33:14.103829+0000","last_fullsized":"2024-05-20T18:33:14.103829+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:21:38.850094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.242904+0000","last_change":"2024-05-20T18:33:13.099965+0000","last_active":"2024-05-20T18:33:14.242904+0000","last_peered":"2024-05-20T18:33:14.242904+0000","last_clean":"2024-05-20T18:33:14.242904+0000","last_became_active":"2024-05-20T18:33:13.099575+0000","last_became_peered":"2024-05-20T18:33:13.099575+0000","last_unstale":"2024-05-20T18:33:14.242904+0000","last_undegraded":"2024-05-20T18:33:14.242904+0000","last_fullsized":"2024-05-20T18:33:14.242904+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:05:02.392436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.242293+0000","last_change":"2024-05-20T18:33:12.083347+0000","last_active":"2024-05-20T18:33:14.242293+0000","last_peered":"2024-05-20T18:33:14.242293+0000","last_clean":"2024-05-20T18:33:14.242293+0000","last_became_active":"2024-05-20T18:33:12.083031+0000","last_became_peered":"2024-05-20T18:33:12.083031+0000","last_unstale":"2024-05-20T18:33:14.242293+0000","last_undegraded":"2024-05-20T18:33:14.242293+0000","last_fullsized":"2024-05-20T18:33:14.242293+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:30:28.580142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099145+0000","last_change":"2024-05-20T18:33:13.102762+0000","last_active":"2024-05-20T18:33:14.099145+0000","last_peered":"2024-05-20T18:33:14.099145+0000","last_clean":"2024-05-20T18:33:14.099145+0000","last_became_active":"2024-05-20T18:33:13.102382+0000","last_became_peered":"2024-05-20T18:33:13.102382+0000","last_unstale":"2024-05-20T18:33:14.099145+0000","last_undegraded":"2024-05-20T18:33:14.099145+0000","last_fullsized":"2024-05-20T18:33:14.099145+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:52:50.494627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243245+0000","last_change":"2024-05-20T18:33:12.089397+0000","last_active":"2024-05-20T18:33:14.243245+0000","last_peered":"2024-05-20T18:33:14.243245+0000","last_clean":"2024-05-20T18:33:14.243245+0000","last_became_active":"2024-05-20T18:33:12.089013+0000","last_became_peered":"2024-05-20T18:33:12.089013+0000","last_unstale":"2024-05-20T18:33:14.243245+0000","last_undegraded":"2024-05-20T18:33:14.243245+0000","last_fullsized":"2024-05-20T18:33:14.243245+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:33:28.814214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243488+0000","last_change":"2024-05-20T18:33:13.099641+0000","last_active":"2024-05-20T18:33:14.243488+0000","last_peered":"2024-05-20T18:33:14.243488+0000","last_clean":"2024-05-20T18:33:14.243488+0000","last_became_active":"2024-05-20T18:33:13.099123+0000","last_became_peered":"2024-05-20T18:33:13.099123+0000","last_unstale":"2024-05-20T18:33:14.243488+0000","last_undegraded":"2024-05-20T18:33:14.243488+0000","last_fullsized":"2024-05-20T18:33:14.243488+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:20:06.218692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"90'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099232+0000","last_change":"2024-05-20T18:33:12.093624+0000","last_active":"2024-05-20T18:33:14.099232+0000","last_peered":"2024-05-20T18:33:14.099232+0000","last_clean":"2024-05-20T18:33:14.099232+0000","last_became_active":"2024-05-20T18:33:12.092398+0000","last_became_peered":"2024-05-20T18:33:12.092398+0000","last_unstale":"2024-05-20T18:33:14.099232+0000","last_undegraded":"2024-05-20T18:33:14.099232+0000","last_fullsized":"2024-05-20T18:33:14.099232+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:40:01.877694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243184+0000","last_change":"2024-05-20T18:33:13.094860+0000","last_active":"2024-05-20T18:33:14.243184+0000","last_peered":"2024-05-20T18:33:14.243184+0000","last_clean":"2024-05-20T18:33:14.243184+0000","last_became_active":"2024-05-20T18:33:13.094548+0000","last_became_peered":"2024-05-20T18:33:13.094548+0000","last_unstale":"2024-05-20T18:33:14.243184+0000","last_undegraded":"2024-05-20T18:33:14.243184+0000","last_fullsized":"2024-05-20T18:33:14.243184+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:21:50.789908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099280+0000","last_change":"2024-05-20T18:33:12.085651+0000","last_active":"2024-05-20T18:33:14.099280+0000","last_peered":"2024-05-20T18:33:14.099280+0000","last_clean":"2024-05-20T18:33:14.099280+0000","last_became_active":"2024-05-20T18:33:12.085202+0000","last_became_peered":"2024-05-20T18:33:12.085202+0000","last_unstale":"2024-05-20T18:33:14.099280+0000","last_undegraded":"2024-05-20T18:33:14.099280+0000","last_fullsized":"2024-05-20T18:33:14.099280+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:45:26.428068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.104278+0000","last_change":"2024-05-20T18:33:13.101659+0000","last_active":"2024-05-20T18:33:14.104278+0000","last_peered":"2024-05-20T18:33:14.104278+0000","last_clean":"2024-05-20T18:33:14.104278+0000","last_became_active":"2024-05-20T18:33:13.101287+0000","last_became_peered":"2024-05-20T18:33:13.101287+0000","last_unstale":"2024-05-20T18:33:14.104278+0000","last_undegraded":"2024-05-20T18:33:14.104278+0000","last_fullsized":"2024-05-20T18:33:14.104278+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:46:43.255085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099366+0000","last_change":"2024-05-20T18:33:12.089871+0000","last_active":"2024-05-20T18:33:14.099366+0000","last_peered":"2024-05-20T18:33:14.099366+0000","last_clean":"2024-05-20T18:33:14.099366+0000","last_became_active":"2024-05-20T18:33:12.089475+0000","last_became_peered":"2024-05-20T18:33:12.089475+0000","last_unstale":"2024-05-20T18:33:14.099366+0000","last_undegraded":"2024-05-20T18:33:14.099366+0000","last_fullsized":"2024-05-20T18:33:14.099366+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:06:00.465798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.242372+0000","last_change":"2024-05-20T18:33:13.099974+0000","last_active":"2024-05-20T18:33:14.242372+0000","last_peered":"2024-05-20T18:33:14.242372+0000","last_clean":"2024-05-20T18:33:14.242372+0000","last_became_active":"2024-05-20T18:33:13.099435+0000","last_became_peered":"2024-05-20T18:33:13.099435+0000","last_unstale":"2024-05-20T18:33:14.242372+0000","last_undegraded":"2024-05-20T18:33:14.242372+0000","last_fullsized":"2024-05-20T18:33:14.242372+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:07:50.513306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243170+0000","last_change":"2024-05-20T18:33:12.090507+0000","last_active":"2024-05-20T18:33:14.243170+0000","last_peered":"2024-05-20T18:33:14.243170+0000","last_clean":"2024-05-20T18:33:14.243170+0000","last_became_active":"2024-05-20T18:33:12.090162+0000","last_became_peered":"2024-05-20T18:33:12.090162+0000","last_unstale":"2024-05-20T18:33:14.243170+0000","last_undegraded":"2024-05-20T18:33:14.243170+0000","last_fullsized":"2024-05-20T18:33:14.243170+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:33:23.682337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.096664+0000","last_change":"2024-05-20T18:33:13.105646+0000","last_active":"2024-05-20T18:33:14.096664+0000","last_peered":"2024-05-20T18:33:14.096664+0000","last_clean":"2024-05-20T18:33:14.096664+0000","last_became_active":"2024-05-20T18:33:13.105276+0000","last_became_peered":"2024-05-20T18:33:13.105276+0000","last_unstale":"2024-05-20T18:33:14.096664+0000","last_undegraded":"2024-05-20T18:33:14.096664+0000","last_fullsized":"2024-05-20T18:33:14.096664+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:17:03.547006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.103717+0000","last_change":"2024-05-20T18:33:12.090897+0000","last_active":"2024-05-20T18:33:14.103717+0000","last_peered":"2024-05-20T18:33:14.103717+0000","last_clean":"2024-05-20T18:33:14.103717+0000","last_became_active":"2024-05-20T18:33:12.090564+0000","last_became_peered":"2024-05-20T18:33:12.090564+0000","last_unstale":"2024-05-20T18:33:14.103717+0000","last_undegraded":"2024-05-20T18:33:14.103717+0000","last_fullsized":"2024-05-20T18:33:14.103717+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:17:57.276235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.098672+0000","last_change":"2024-05-20T18:33:13.105892+0000","last_active":"2024-05-20T18:33:14.098672+0000","last_peered":"2024-05-20T18:33:14.098672+0000","last_clean":"2024-05-20T18:33:14.098672+0000","last_became_active":"2024-05-20T18:33:13.105576+0000","last_became_peered":"2024-05-20T18:33:13.105576+0000","last_unstale":"2024-05-20T18:33:14.098672+0000","last_undegraded":"2024-05-20T18:33:14.098672+0000","last_fullsized":"2024-05-20T18:33:14.098672+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:16:15.456403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099637+0000","last_change":"2024-05-20T18:33:12.091271+0000","last_active":"2024-05-20T18:33:14.099637+0000","last_peered":"2024-05-20T18:33:14.099637+0000","last_clean":"2024-05-20T18:33:14.099637+0000","last_became_active":"2024-05-20T18:33:12.090943+0000","last_became_peered":"2024-05-20T18:33:12.090943+0000","last_unstale":"2024-05-20T18:33:14.099637+0000","last_undegraded":"2024-05-20T18:33:14.099637+0000","last_fullsized":"2024-05-20T18:33:14.099637+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:23:32.815272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243106+0000","last_change":"2024-05-20T18:33:13.103661+0000","last_active":"2024-05-20T18:33:14.243106+0000","last_peered":"2024-05-20T18:33:14.243106+0000","last_clean":"2024-05-20T18:33:14.243106+0000","last_became_active":"2024-05-20T18:33:13.103295+0000","last_became_peered":"2024-05-20T18:33:13.103295+0000","last_unstale":"2024-05-20T18:33:14.243106+0000","last_undegraded":"2024-05-20T18:33:14.243106+0000","last_fullsized":"2024-05-20T18:33:14.243106+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:19:48.464838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243132+0000","last_change":"2024-05-20T18:33:12.095449+0000","last_active":"2024-05-20T18:33:14.243132+0000","last_peered":"2024-05-20T18:33:14.243132+0000","last_clean":"2024-05-20T18:33:14.243132+0000","last_became_active":"2024-05-20T18:33:12.095139+0000","last_became_peered":"2024-05-20T18:33:12.095139+0000","last_unstale":"2024-05-20T18:33:14.243132+0000","last_undegraded":"2024-05-20T18:33:14.243132+0000","last_fullsized":"2024-05-20T18:33:14.243132+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:35:55.978213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243365+0000","last_change":"2024-05-20T18:33:13.100731+0000","last_active":"2024-05-20T18:33:14.243365+0000","last_peered":"2024-05-20T18:33:14.243365+0000","last_clean":"2024-05-20T18:33:14.243365+0000","last_became_active":"2024-05-20T18:33:13.100375+0000","last_became_peered":"2024-05-20T18:33:13.100375+0000","last_unstale":"2024-05-20T18:33:14.243365+0000","last_undegraded":"2024-05-20T18:33:14.243365+0000","last_fullsized":"2024-05-20T18:33:14.243365+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:52:45.545185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.244197+0000","last_change":"2024-05-20T18:33:12.099344+0000","last_active":"2024-05-20T18:33:14.244197+0000","last_peered":"2024-05-20T18:33:14.244197+0000","last_clean":"2024-05-20T18:33:14.244197+0000","last_became_active":"2024-05-20T18:33:12.099023+0000","last_became_peered":"2024-05-20T18:33:12.099023+0000","last_unstale":"2024-05-20T18:33:14.244197+0000","last_undegraded":"2024-05-20T18:33:14.244197+0000","last_fullsized":"2024-05-20T18:33:14.244197+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:45:26.428068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":298,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243232+0000","last_change":"2024-05-20T18:27:03.765694+0000","last_active":"2024-05-20T18:33:14.243232+0000","last_peered":"2024-05-20T18:33:14.243232+0000","last_clean":"2024-05-20T18:33:14.243232+0000","last_became_active":"2024-05-20T18:27:03.764695+0000","last_became_peered":"2024-05-20T18:27:03.764695+0000","last_unstale":"2024-05-20T18:33:14.243232+0000","last_undegraded":"2024-05-20T18:33:14.243232+0000","last_fullsized":"2024-05-20T18:33:14.243232+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_clean_scrub_stamp":"2024-05-20T18:27:02.750853+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:30:04.407487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574927,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1712128,"data_stored":1100013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1130496,"data_stored":511805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574925,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1110016,"data_stored":509589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574925,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1118208,"data_stored":509749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574927,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1110016,"data_stored":509589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574926,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1110016,"data_stored":509589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1114112,"data_stored":509659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574926,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32872,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338968,"statfs":{"total":96636764160,"available":96603103232,"internally_reserved":0,"allocated":1728512,"data_stored":1102275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T18:33:17.163 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T18:33:17.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.313+0000 7fcae4ee2640 1 Processor -- start 2024-05-20T18:33:17.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.313+0000 7fcae4ee2640 1 -- start start 2024-05-20T18:33:17.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.314+0000 7fcae4ee2640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcae0162820 0x7fcae00b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:17.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.314+0000 7fcae4ee2640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcae00b3e80 0x7fcae006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:17.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.314+0000 7fcae4ee2640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcae006dd20 con 0x7fcae006d910 2024-05-20T18:33:17.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.314+0000 7fcae4ee2640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fcae00d0630 con 0x7fcae0162820 2024-05-20T18:33:17.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.314+0000 7fcae4ee2640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcae00d39c0 con 0x7fcae00b3e80 2024-05-20T18:33:17.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.315+0000 7fcade59c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcae0162820 0x7fcae00b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:17.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.315+0000 7fcade59c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcae0162820 0x7fcae00b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58978/0 (socket says 172.21.15.5:58978) 2024-05-20T18:33:17.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.315+0000 7fcade59c640 1 -- 172.21.15.5:0/2685774340 learned_addr learned my addr 172.21.15.5:0/2685774340 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:17.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.315+0000 7fcaddd9b640 1 --2- 172.21.15.5:0/2685774340 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcae00b3e80 0x7fcae006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:17.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.315+0000 7fcade59c640 1 -- 172.21.15.5:0/2685774340 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcae00b3e80 msgr2=0x7fcae006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:17.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.315+0000 7fcade59c640 1 --2- 172.21.15.5:0/2685774340 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcae00b3e80 0x7fcae006d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:17.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.315+0000 7fcade59c640 1 -- 172.21.15.5:0/2685774340 >> v1:172.21.15.63:6789/0 conn(0x7fcae006d910 legacy=0x7fcae00d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:17.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.315+0000 7fcade59c640 1 -- 172.21.15.5:0/2685774340 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae00d3ca0 con 0x7fcae0162820 2024-05-20T18:33:17.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.315+0000 7fcaddd9b640 1 --2- 172.21.15.5:0/2685774340 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcae00b3e80 0x7fcae006d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:17.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.316+0000 7fcade59c640 1 --2- 172.21.15.5:0/2685774340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcae0162820 0x7fcae00b3940 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fcad0009870 tx=0x7fcad002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b088ed3f18fe937b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:17.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.316+0000 7fcadd59a640 1 -- 172.21.15.5:0/2685774340 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad0037070 con 0x7fcae0162820 2024-05-20T18:33:17.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.316+0000 7fcadd59a640 1 -- 172.21.15.5:0/2685774340 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcad0032440 con 0x7fcae0162820 2024-05-20T18:33:17.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.316+0000 7fcadd59a640 1 -- 172.21.15.5:0/2685774340 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad0032810 con 0x7fcae0162820 2024-05-20T18:33:17.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.316+0000 7fcae4ee2640 1 -- 172.21.15.5:0/2685774340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcae0162820 msgr2=0x7fcae00b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:17.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.316+0000 7fcae4ee2640 1 --2- 172.21.15.5:0/2685774340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcae0162820 0x7fcae00b3940 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fcad0009870 tx=0x7fcad002a270 comp rx=0 tx=0).stop 2024-05-20T18:33:17.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.317+0000 7fcae4ee2640 1 -- 172.21.15.5:0/2685774340 shutdown_connections 2024-05-20T18:33:17.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.317+0000 7fcae4ee2640 1 --2- 172.21.15.5:0/2685774340 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcae00b3e80 0x7fcae006d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:17.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.317+0000 7fcae4ee2640 1 --2- 172.21.15.5:0/2685774340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcae0162820 0x7fcae00b3940 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:17.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.317+0000 7fcae4ee2640 1 -- 172.21.15.5:0/2685774340 >> 172.21.15.5:0/2685774340 conn(0x7fcae00d4bd0 msgr2=0x7fcae00d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:17.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.317+0000 7fcae4ee2640 1 -- 172.21.15.5:0/2685774340 shutdown_connections 2024-05-20T18:33:17.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.317+0000 7fcae4ee2640 1 -- 172.21.15.5:0/2685774340 wait complete. 2024-05-20T18:33:17.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.319+0000 7fcae4ee2640 1 Processor -- start 2024-05-20T18:33:17.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.319+0000 7fcae4ee2640 1 -- start start 2024-05-20T18:33:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.320+0000 7fcae4ee2640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcae006d910 0x7fcae00c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.320+0000 7fcae4ee2640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcae00b3e80 0x7fcae01f51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.320+0000 7fcae4ee2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcae0162820 0x7fcae01f8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.320+0000 7fcae4ee2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcae00d0300 con 0x7fcae0162820 2024-05-20T18:33:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.320+0000 7fcae4ee2640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcae00d4260 con 0x7fcae006d910 2024-05-20T18:33:17.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.320+0000 7fcae4ee2640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcae00b91a0 con 0x7fcae00b3e80 2024-05-20T18:33:17.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.320+0000 7fcaded9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcae0162820 0x7fcae01f8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:17.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.320+0000 7fcaded9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcae0162820 0x7fcae01f8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58982/0 (socket says 172.21.15.5:58982) 2024-05-20T18:33:17.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.320+0000 7fcaded9d640 1 -- 172.21.15.5:0/2115745532 learned_addr learned my addr 172.21.15.5:0/2115745532 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:17.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.320+0000 7fcade59c640 1 --2- 172.21.15.5:0/2115745532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcae006d910 0x7fcae00c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:17.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.320+0000 7fcaddd9b640 1 --2- 172.21.15.5:0/2115745532 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcae00b3e80 0x7fcae01f51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:17.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.320+0000 7fcaded9d640 1 -- 172.21.15.5:0/2115745532 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcae00b3e80 msgr2=0x7fcae01f51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:17.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.320+0000 7fcaded9d640 1 --2- 172.21.15.5:0/2115745532 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcae00b3e80 0x7fcae01f51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:17.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.320+0000 7fcaded9d640 1 -- 172.21.15.5:0/2115745532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcae006d910 msgr2=0x7fcae00c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:17.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.320+0000 7fcaded9d640 1 --2- 172.21.15.5:0/2115745532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcae006d910 0x7fcae00c8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:17.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.320+0000 7fcaded9d640 1 -- 172.21.15.5:0/2115745532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae00c96d0 con 0x7fcae0162820 2024-05-20T18:33:17.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.321+0000 7fcaded9d640 1 --2- 172.21.15.5:0/2115745532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcae0162820 0x7fcae01f8ce0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fcad800c6b0 tx=0x7fcad800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:17.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.321+0000 7fcaceffd640 1 -- 172.21.15.5:0/2115745532 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad8003b70 con 0x7fcae0162820 2024-05-20T18:33:17.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.321+0000 7fcaceffd640 1 -- 172.21.15.5:0/2115745532 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcad8003d10 con 0x7fcae0162820 2024-05-20T18:33:17.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.321+0000 7fcae4ee2640 1 -- 172.21.15.5:0/2115745532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcae00cae60 con 0x7fcae0162820 2024-05-20T18:33:17.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.321+0000 7fcaceffd640 1 -- 172.21.15.5:0/2115745532 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad8005310 con 0x7fcae0162820 2024-05-20T18:33:17.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.321+0000 7fcae4ee2640 1 -- 172.21.15.5:0/2115745532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcae00cb420 con 0x7fcae0162820 2024-05-20T18:33:17.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.322+0000 7fcae4ee2640 1 -- 172.21.15.5:0/2115745532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcae0075b00 con 0x7fcae0162820 2024-05-20T18:33:17.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.322+0000 7fcaceffd640 1 -- 172.21.15.5:0/2115745532 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcad80054b0 con 0x7fcae0162820 2024-05-20T18:33:17.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.323+0000 7fcaceffd640 1 --2- 172.21.15.5:0/2115745532 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcac006be30 0x7fcac006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:17.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.323+0000 7fcade59c640 1 --2- 172.21.15.5:0/2115745532 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcac006be30 0x7fcac006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:17.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.323+0000 7fcaceffd640 1 -- 172.21.15.5:0/2115745532 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6101+0+0 (secure 0 0 0) 0x7fcad808aca0 con 0x7fcae0162820 2024-05-20T18:33:17.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.324+0000 7fcade59c640 1 --2- 172.21.15.5:0/2115745532 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcac006be30 0x7fcac006e340 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fcad0034000 tx=0x7fcad002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:17.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.328+0000 7fcaceffd640 1 -- 172.21.15.5:0/2115745532 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fcae006dd20 con 0x7fcae0162820 2024-05-20T18:33:17.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.507+0000 7fcae4ee2640 1 -- 172.21.15.5:0/2115745532 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcae0160a90 con 0x7fcac006be30 2024-05-20T18:33:17.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.514+0000 7fcaceffd640 1 -- 172.21.15.5:0/2115745532 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252074 (secure 0 0 0) 0x7fcae0160a90 con 0x7fcac006be30 2024-05-20T18:33:17.517 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:33:17.518 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-20T18:33:17.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.517+0000 7fcae4ee2640 1 -- 172.21.15.5:0/2115745532 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcac006be30 msgr2=0x7fcac006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:17.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.517+0000 7fcae4ee2640 1 --2- 172.21.15.5:0/2115745532 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcac006be30 0x7fcac006e340 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fcad0034000 tx=0x7fcad002e000 comp rx=0 tx=0).stop 2024-05-20T18:33:17.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.517+0000 7fcae4ee2640 1 -- 172.21.15.5:0/2115745532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcae0162820 msgr2=0x7fcae01f8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:17.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.517+0000 7fcae4ee2640 1 --2- 172.21.15.5:0/2115745532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcae0162820 0x7fcae01f8ce0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fcad800c6b0 tx=0x7fcad800cb70 comp rx=0 tx=0).stop 2024-05-20T18:33:17.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.517+0000 7fcae4ee2640 1 -- 172.21.15.5:0/2115745532 shutdown_connections 2024-05-20T18:33:17.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.517+0000 7fcae4ee2640 1 --2- 172.21.15.5:0/2115745532 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcac006be30 0x7fcac006e340 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:17.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.517+0000 7fcae4ee2640 1 --2- 172.21.15.5:0/2115745532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcae0162820 0x7fcae01f8ce0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:17.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.517+0000 7fcae4ee2640 1 --2- 172.21.15.5:0/2115745532 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcae00b3e80 0x7fcae01f51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:17.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.517+0000 7fcae4ee2640 1 --2- 172.21.15.5:0/2115745532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcae006d910 0x7fcae00c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:17.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.517+0000 7fcae4ee2640 1 -- 172.21.15.5:0/2115745532 >> 172.21.15.5:0/2115745532 conn(0x7fcae00d4bd0 msgr2=0x7fcae00d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:17.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.517+0000 7fcae4ee2640 1 -- 172.21.15.5:0/2115745532 shutdown_connections 2024-05-20T18:33:17.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.517+0000 7fcae4ee2640 1 -- 172.21.15.5:0/2115745532 wait complete. 2024-05-20T18:33:17.529 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":271,"stamp":"2024-05-20T18:33:16.668213+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":177,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259388,"kb_used_data":9896,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715332,"statfs":{"total":773094113280,"available":772828499968,"internally_reserved":0,"allocated":10133504,"data_stored":5262268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.601547"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.096297+0000","last_change":"2024-05-20T18:33:13.113701+0000","last_active":"2024-05-20T18:33:14.096297+0000","last_peered":"2024-05-20T18:33:14.096297+0000","last_clean":"2024-05-20T18:33:14.096297+0000","last_became_active":"2024-05-20T18:33:13.113395+0000","last_became_peered":"2024-05-20T18:33:13.113395+0000","last_unstale":"2024-05-20T18:33:14.096297+0000","last_undegraded":"2024-05-20T18:33:14.096297+0000","last_fullsized":"2024-05-20T18:33:14.096297+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:14:33.945373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.104111+0000","last_change":"2024-05-20T18:33:13.100252+0000","last_active":"2024-05-20T18:33:14.104111+0000","last_peered":"2024-05-20T18:33:14.104111+0000","last_clean":"2024-05-20T18:33:14.104111+0000","last_became_active":"2024-05-20T18:33:13.099916+0000","last_became_peered":"2024-05-20T18:33:13.099916+0000","last_unstale":"2024-05-20T18:33:14.104111+0000","last_undegraded":"2024-05-20T18:33:14.104111+0000","last_fullsized":"2024-05-20T18:33:14.104111+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:33:29.819795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.096118+0000","last_change":"2024-05-20T18:33:13.115725+0000","last_active":"2024-05-20T18:33:14.096118+0000","last_peered":"2024-05-20T18:33:14.096118+0000","last_clean":"2024-05-20T18:33:14.096118+0000","last_became_active":"2024-05-20T18:33:13.115383+0000","last_became_peered":"2024-05-20T18:33:13.115383+0000","last_unstale":"2024-05-20T18:33:14.096118+0000","last_undegraded":"2024-05-20T18:33:14.096118+0000","last_fullsized":"2024-05-20T18:33:14.096118+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:24:00.139714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243983+0000","last_change":"2024-05-20T18:33:13.108686+0000","last_active":"2024-05-20T18:33:14.243983+0000","last_peered":"2024-05-20T18:33:14.243983+0000","last_clean":"2024-05-20T18:33:14.243983+0000","last_became_active":"2024-05-20T18:33:13.108291+0000","last_became_peered":"2024-05-20T18:33:13.108291+0000","last_unstale":"2024-05-20T18:33:14.243983+0000","last_undegraded":"2024-05-20T18:33:14.243983+0000","last_fullsized":"2024-05-20T18:33:14.243983+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:54:26.495327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.098858+0000","last_change":"2024-05-20T18:33:13.091713+0000","last_active":"2024-05-20T18:33:14.098858+0000","last_peered":"2024-05-20T18:33:14.098858+0000","last_clean":"2024-05-20T18:33:14.098858+0000","last_became_active":"2024-05-20T18:33:13.091374+0000","last_became_peered":"2024-05-20T18:33:13.091374+0000","last_unstale":"2024-05-20T18:33:14.098858+0000","last_undegraded":"2024-05-20T18:33:14.098858+0000","last_fullsized":"2024-05-20T18:33:14.098858+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:17:58.281816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.242675+0000","last_change":"2024-05-20T18:33:13.109151+0000","last_active":"2024-05-20T18:33:14.242675+0000","last_peered":"2024-05-20T18:33:14.242675+0000","last_clean":"2024-05-20T18:33:14.242675+0000","last_became_active":"2024-05-20T18:33:13.108479+0000","last_became_peered":"2024-05-20T18:33:13.108479+0000","last_unstale":"2024-05-20T18:33:14.242675+0000","last_undegraded":"2024-05-20T18:33:14.242675+0000","last_fullsized":"2024-05-20T18:33:14.242675+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:57:36.648081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243084+0000","last_change":"2024-05-20T18:33:13.099632+0000","last_active":"2024-05-20T18:33:14.243084+0000","last_peered":"2024-05-20T18:33:14.243084+0000","last_clean":"2024-05-20T18:33:14.243084+0000","last_became_active":"2024-05-20T18:33:13.099306+0000","last_became_peered":"2024-05-20T18:33:13.099306+0000","last_unstale":"2024-05-20T18:33:14.243084+0000","last_undegraded":"2024-05-20T18:33:14.243084+0000","last_fullsized":"2024-05-20T18:33:14.243084+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:33:24.687918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243732+0000","last_change":"2024-05-20T18:33:13.108305+0000","last_active":"2024-05-20T18:33:14.243732+0000","last_peered":"2024-05-20T18:33:14.243732+0000","last_clean":"2024-05-20T18:33:14.243732+0000","last_became_active":"2024-05-20T18:33:13.107989+0000","last_became_peered":"2024-05-20T18:33:13.107989+0000","last_unstale":"2024-05-20T18:33:14.243732+0000","last_undegraded":"2024-05-20T18:33:14.243732+0000","last_fullsized":"2024-05-20T18:33:14.243732+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:52:04.290712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.098654+0000","last_change":"2024-05-20T18:33:13.103879+0000","last_active":"2024-05-20T18:33:14.098654+0000","last_peered":"2024-05-20T18:33:14.098654+0000","last_clean":"2024-05-20T18:33:14.098654+0000","last_became_active":"2024-05-20T18:33:13.103550+0000","last_became_peered":"2024-05-20T18:33:13.103550+0000","last_unstale":"2024-05-20T18:33:14.098654+0000","last_undegraded":"2024-05-20T18:33:14.098654+0000","last_fullsized":"2024-05-20T18:33:14.098654+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:52:04.290712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.242865+0000","last_change":"2024-05-20T18:33:13.111617+0000","last_active":"2024-05-20T18:33:14.242865+0000","last_peered":"2024-05-20T18:33:14.242865+0000","last_clean":"2024-05-20T18:33:14.242865+0000","last_became_active":"2024-05-20T18:33:13.111178+0000","last_became_peered":"2024-05-20T18:33:13.111178+0000","last_unstale":"2024-05-20T18:33:14.242865+0000","last_undegraded":"2024-05-20T18:33:14.242865+0000","last_fullsized":"2024-05-20T18:33:14.242865+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:04:48.679542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.098765+0000","last_change":"2024-05-20T18:33:13.104475+0000","last_active":"2024-05-20T18:33:14.098765+0000","last_peered":"2024-05-20T18:33:14.098765+0000","last_clean":"2024-05-20T18:33:14.098765+0000","last_became_active":"2024-05-20T18:33:13.104143+0000","last_became_peered":"2024-05-20T18:33:13.104143+0000","last_unstale":"2024-05-20T18:33:14.098765+0000","last_undegraded":"2024-05-20T18:33:14.098765+0000","last_fullsized":"2024-05-20T18:33:14.098765+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:54:26.495327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.242521+0000","last_change":"2024-05-20T18:33:13.098050+0000","last_active":"2024-05-20T18:33:14.242521+0000","last_peered":"2024-05-20T18:33:14.242521+0000","last_clean":"2024-05-20T18:33:14.242521+0000","last_became_active":"2024-05-20T18:33:13.097707+0000","last_became_peered":"2024-05-20T18:33:13.097707+0000","last_unstale":"2024-05-20T18:33:14.242521+0000","last_undegraded":"2024-05-20T18:33:14.242521+0000","last_fullsized":"2024-05-20T18:33:14.242521+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:44:46.333079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243204+0000","last_change":"2024-05-20T18:33:13.104575+0000","last_active":"2024-05-20T18:33:14.243204+0000","last_peered":"2024-05-20T18:33:14.243204+0000","last_clean":"2024-05-20T18:33:14.243204+0000","last_became_active":"2024-05-20T18:33:13.104194+0000","last_became_peered":"2024-05-20T18:33:13.104194+0000","last_unstale":"2024-05-20T18:33:14.243204+0000","last_undegraded":"2024-05-20T18:33:14.243204+0000","last_fullsized":"2024-05-20T18:33:14.243204+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:47:24.701055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099465+0000","last_change":"2024-05-20T18:33:13.110314+0000","last_active":"2024-05-20T18:33:14.099465+0000","last_peered":"2024-05-20T18:33:14.099465+0000","last_clean":"2024-05-20T18:33:14.099465+0000","last_became_active":"2024-05-20T18:33:13.109911+0000","last_became_peered":"2024-05-20T18:33:13.109911+0000","last_unstale":"2024-05-20T18:33:14.099465+0000","last_undegraded":"2024-05-20T18:33:14.099465+0000","last_fullsized":"2024-05-20T18:33:14.099465+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:04:16.738506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243989+0000","last_change":"2024-05-20T18:33:13.102506+0000","last_active":"2024-05-20T18:33:14.243989+0000","last_peered":"2024-05-20T18:33:14.243989+0000","last_clean":"2024-05-20T18:33:14.243989+0000","last_became_active":"2024-05-20T18:33:13.102172+0000","last_became_peered":"2024-05-20T18:33:13.102172+0000","last_unstale":"2024-05-20T18:33:14.243989+0000","last_undegraded":"2024-05-20T18:33:14.243989+0000","last_fullsized":"2024-05-20T18:33:14.243989+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:17:58.281816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.242675+0000","last_change":"2024-05-20T18:33:13.101347+0000","last_active":"2024-05-20T18:33:14.242675+0000","last_peered":"2024-05-20T18:33:14.242675+0000","last_clean":"2024-05-20T18:33:14.242675+0000","last_became_active":"2024-05-20T18:33:13.100728+0000","last_became_peered":"2024-05-20T18:33:13.100728+0000","last_unstale":"2024-05-20T18:33:14.242675+0000","last_undegraded":"2024-05-20T18:33:14.242675+0000","last_fullsized":"2024-05-20T18:33:14.242675+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:55:04.812299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243523+0000","last_change":"2024-05-20T18:33:13.102752+0000","last_active":"2024-05-20T18:33:14.243523+0000","last_peered":"2024-05-20T18:33:14.243523+0000","last_clean":"2024-05-20T18:33:14.243523+0000","last_became_active":"2024-05-20T18:33:13.102458+0000","last_became_peered":"2024-05-20T18:33:13.102458+0000","last_unstale":"2024-05-20T18:33:14.243523+0000","last_undegraded":"2024-05-20T18:33:14.243523+0000","last_fullsized":"2024-05-20T18:33:14.243523+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:27:49.342593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.096390+0000","last_change":"2024-05-20T18:33:13.107657+0000","last_active":"2024-05-20T18:33:14.096390+0000","last_peered":"2024-05-20T18:33:14.096390+0000","last_clean":"2024-05-20T18:33:14.096390+0000","last_became_active":"2024-05-20T18:33:13.107328+0000","last_became_peered":"2024-05-20T18:33:13.107328+0000","last_unstale":"2024-05-20T18:33:14.096390+0000","last_undegraded":"2024-05-20T18:33:14.096390+0000","last_fullsized":"2024-05-20T18:33:14.096390+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:55:04.812299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.098967+0000","last_change":"2024-05-20T18:33:13.112305+0000","last_active":"2024-05-20T18:33:14.098967+0000","last_peered":"2024-05-20T18:33:14.098967+0000","last_clean":"2024-05-20T18:33:14.098967+0000","last_became_active":"2024-05-20T18:33:13.107659+0000","last_became_peered":"2024-05-20T18:33:13.107659+0000","last_unstale":"2024-05-20T18:33:14.098967+0000","last_undegraded":"2024-05-20T18:33:14.098967+0000","last_fullsized":"2024-05-20T18:33:14.098967+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:25:36.065702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.244119+0000","last_change":"2024-05-20T18:33:13.103127+0000","last_active":"2024-05-20T18:33:14.244119+0000","last_peered":"2024-05-20T18:33:14.244119+0000","last_clean":"2024-05-20T18:33:14.244119+0000","last_became_active":"2024-05-20T18:33:13.102756+0000","last_became_peered":"2024-05-20T18:33:13.102756+0000","last_unstale":"2024-05-20T18:33:14.244119+0000","last_undegraded":"2024-05-20T18:33:14.244119+0000","last_fullsized":"2024-05-20T18:33:14.244119+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:21:39.855675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.098563+0000","last_change":"2024-05-20T18:33:13.101453+0000","last_active":"2024-05-20T18:33:14.098563+0000","last_peered":"2024-05-20T18:33:14.098563+0000","last_clean":"2024-05-20T18:33:14.098563+0000","last_became_active":"2024-05-20T18:33:13.100767+0000","last_became_peered":"2024-05-20T18:33:13.100767+0000","last_unstale":"2024-05-20T18:33:14.098563+0000","last_undegraded":"2024-05-20T18:33:14.098563+0000","last_fullsized":"2024-05-20T18:33:14.098563+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:52:50.494627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099036+0000","last_change":"2024-05-20T18:33:13.109376+0000","last_active":"2024-05-20T18:33:14.099036+0000","last_peered":"2024-05-20T18:33:14.099036+0000","last_clean":"2024-05-20T18:33:14.099036+0000","last_became_active":"2024-05-20T18:33:13.108345+0000","last_became_peered":"2024-05-20T18:33:13.108345+0000","last_unstale":"2024-05-20T18:33:14.099036+0000","last_undegraded":"2024-05-20T18:33:14.099036+0000","last_fullsized":"2024-05-20T18:33:14.099036+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:05:02.724916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099547+0000","last_change":"2024-05-20T18:33:13.103036+0000","last_active":"2024-05-20T18:33:14.099547+0000","last_peered":"2024-05-20T18:33:14.099547+0000","last_clean":"2024-05-20T18:33:14.099547+0000","last_became_active":"2024-05-20T18:33:13.102553+0000","last_became_peered":"2024-05-20T18:33:13.102553+0000","last_unstale":"2024-05-20T18:33:14.099547+0000","last_undegraded":"2024-05-20T18:33:14.099547+0000","last_fullsized":"2024-05-20T18:33:14.099547+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:23:27.680097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.096158+0000","last_change":"2024-05-20T18:33:13.106263+0000","last_active":"2024-05-20T18:33:14.096158+0000","last_peered":"2024-05-20T18:33:14.096158+0000","last_clean":"2024-05-20T18:33:14.096158+0000","last_became_active":"2024-05-20T18:33:13.105927+0000","last_became_peered":"2024-05-20T18:33:13.105927+0000","last_unstale":"2024-05-20T18:33:14.096158+0000","last_undegraded":"2024-05-20T18:33:14.096158+0000","last_fullsized":"2024-05-20T18:33:14.096158+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:42:41.424610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.104211+0000","last_change":"2024-05-20T18:33:13.100803+0000","last_active":"2024-05-20T18:33:14.104211+0000","last_peered":"2024-05-20T18:33:14.104211+0000","last_clean":"2024-05-20T18:33:14.104211+0000","last_became_active":"2024-05-20T18:33:13.097534+0000","last_became_peered":"2024-05-20T18:33:13.097534+0000","last_unstale":"2024-05-20T18:33:14.104211+0000","last_undegraded":"2024-05-20T18:33:14.104211+0000","last_fullsized":"2024-05-20T18:33:14.104211+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:39:10.618607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.098714+0000","last_change":"2024-05-20T18:33:13.107870+0000","last_active":"2024-05-20T18:33:14.098714+0000","last_peered":"2024-05-20T18:33:14.098714+0000","last_clean":"2024-05-20T18:33:14.098714+0000","last_became_active":"2024-05-20T18:33:13.107530+0000","last_became_peered":"2024-05-20T18:33:13.107530+0000","last_unstale":"2024-05-20T18:33:14.098714+0000","last_undegraded":"2024-05-20T18:33:14.098714+0000","last_fullsized":"2024-05-20T18:33:14.098714+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:18:03.936593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.242887+0000","last_change":"2024-05-20T18:33:13.097688+0000","last_active":"2024-05-20T18:33:14.242887+0000","last_peered":"2024-05-20T18:33:14.242887+0000","last_clean":"2024-05-20T18:33:14.242887+0000","last_became_active":"2024-05-20T18:33:13.097322+0000","last_became_peered":"2024-05-20T18:33:13.097322+0000","last_unstale":"2024-05-20T18:33:14.242887+0000","last_undegraded":"2024-05-20T18:33:14.242887+0000","last_fullsized":"2024-05-20T18:33:14.242887+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:31:51.783828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.244171+0000","last_change":"2024-05-20T18:33:13.107890+0000","last_active":"2024-05-20T18:33:14.244171+0000","last_peered":"2024-05-20T18:33:14.244171+0000","last_clean":"2024-05-20T18:33:14.244171+0000","last_became_active":"2024-05-20T18:33:13.107533+0000","last_became_peered":"2024-05-20T18:33:13.107533+0000","last_unstale":"2024-05-20T18:33:14.244171+0000","last_undegraded":"2024-05-20T18:33:14.244171+0000","last_fullsized":"2024-05-20T18:33:14.244171+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:25:36.065702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243276+0000","last_change":"2024-05-20T18:33:13.102602+0000","last_active":"2024-05-20T18:33:14.243276+0000","last_peered":"2024-05-20T18:33:14.243276+0000","last_clean":"2024-05-20T18:33:14.243276+0000","last_became_active":"2024-05-20T18:33:13.102178+0000","last_became_peered":"2024-05-20T18:33:13.102178+0000","last_unstale":"2024-05-20T18:33:14.243276+0000","last_undegraded":"2024-05-20T18:33:14.243276+0000","last_fullsized":"2024-05-20T18:33:14.243276+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:37:16.850745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099337+0000","last_change":"2024-05-20T18:33:13.096634+0000","last_active":"2024-05-20T18:33:14.099337+0000","last_peered":"2024-05-20T18:33:14.099337+0000","last_clean":"2024-05-20T18:33:14.099337+0000","last_became_active":"2024-05-20T18:33:13.095730+0000","last_became_peered":"2024-05-20T18:33:13.095730+0000","last_unstale":"2024-05-20T18:33:14.099337+0000","last_undegraded":"2024-05-20T18:33:14.099337+0000","last_fullsized":"2024-05-20T18:33:14.099337+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:46:57.728633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.103747+0000","last_change":"2024-05-20T18:33:13.107961+0000","last_active":"2024-05-20T18:33:14.103747+0000","last_peered":"2024-05-20T18:33:14.103747+0000","last_clean":"2024-05-20T18:33:14.103747+0000","last_became_active":"2024-05-20T18:33:13.107609+0000","last_became_peered":"2024-05-20T18:33:13.107609+0000","last_unstale":"2024-05-20T18:33:14.103747+0000","last_undegraded":"2024-05-20T18:33:14.103747+0000","last_fullsized":"2024-05-20T18:33:14.103747+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:35:56.983794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.242847+0000","last_change":"2024-05-20T18:33:13.101086+0000","last_active":"2024-05-20T18:33:14.242847+0000","last_peered":"2024-05-20T18:33:14.242847+0000","last_clean":"2024-05-20T18:33:14.242847+0000","last_became_active":"2024-05-20T18:33:13.100289+0000","last_became_peered":"2024-05-20T18:33:13.100289+0000","last_unstale":"2024-05-20T18:33:14.242847+0000","last_undegraded":"2024-05-20T18:33:14.242847+0000","last_fullsized":"2024-05-20T18:33:14.242847+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:17:03.547006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243945+0000","last_change":"2024-05-20T18:33:13.100725+0000","last_active":"2024-05-20T18:33:14.243945+0000","last_peered":"2024-05-20T18:33:14.243945+0000","last_clean":"2024-05-20T18:33:14.243945+0000","last_became_active":"2024-05-20T18:33:13.099258+0000","last_became_peered":"2024-05-20T18:33:13.099258+0000","last_unstale":"2024-05-20T18:33:14.243945+0000","last_undegraded":"2024-05-20T18:33:14.243945+0000","last_fullsized":"2024-05-20T18:33:14.243945+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:56:50.410289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"89'4","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099900+0000","last_change":"2024-05-20T18:33:12.094249+0000","last_active":"2024-05-20T18:33:14.099900+0000","last_peered":"2024-05-20T18:33:14.099900+0000","last_clean":"2024-05-20T18:33:14.099900+0000","last_became_active":"2024-05-20T18:33:12.093887+0000","last_became_peered":"2024-05-20T18:33:12.093887+0000","last_unstale":"2024-05-20T18:33:14.099900+0000","last_undegraded":"2024-05-20T18:33:14.099900+0000","last_fullsized":"2024-05-20T18:33:14.099900+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:35:13.707739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243190+0000","last_change":"2024-05-20T18:33:13.100911+0000","last_active":"2024-05-20T18:33:14.243190+0000","last_peered":"2024-05-20T18:33:14.243190+0000","last_clean":"2024-05-20T18:33:14.243190+0000","last_became_active":"2024-05-20T18:33:13.099691+0000","last_became_peered":"2024-05-20T18:33:13.099691+0000","last_unstale":"2024-05-20T18:33:14.243190+0000","last_undegraded":"2024-05-20T18:33:14.243190+0000","last_fullsized":"2024-05-20T18:33:14.243190+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:31:40.148877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.098997+0000","last_change":"2024-05-20T18:33:12.083543+0000","last_active":"2024-05-20T18:33:14.098997+0000","last_peered":"2024-05-20T18:33:14.098997+0000","last_clean":"2024-05-20T18:33:14.098997+0000","last_became_active":"2024-05-20T18:33:12.083187+0000","last_became_peered":"2024-05-20T18:33:12.083187+0000","last_unstale":"2024-05-20T18:33:14.098997+0000","last_undegraded":"2024-05-20T18:33:14.098997+0000","last_fullsized":"2024-05-20T18:33:14.098997+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:29:09.903414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243415+0000","last_change":"2024-05-20T18:33:13.110254+0000","last_active":"2024-05-20T18:33:14.243415+0000","last_peered":"2024-05-20T18:33:14.243415+0000","last_clean":"2024-05-20T18:33:14.243415+0000","last_became_active":"2024-05-20T18:33:13.109930+0000","last_became_peered":"2024-05-20T18:33:13.109930+0000","last_unstale":"2024-05-20T18:33:14.243415+0000","last_undegraded":"2024-05-20T18:33:14.243415+0000","last_fullsized":"2024-05-20T18:33:14.243415+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:51:29.485838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243109+0000","last_change":"2024-05-20T18:33:12.090717+0000","last_active":"2024-05-20T18:33:14.243109+0000","last_peered":"2024-05-20T18:33:14.243109+0000","last_clean":"2024-05-20T18:33:14.243109+0000","last_became_active":"2024-05-20T18:33:12.090282+0000","last_became_peered":"2024-05-20T18:33:12.090282+0000","last_unstale":"2024-05-20T18:33:14.243109+0000","last_undegraded":"2024-05-20T18:33:14.243109+0000","last_fullsized":"2024-05-20T18:33:14.243109+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:43:08.640518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243680+0000","last_change":"2024-05-20T18:33:13.092762+0000","last_active":"2024-05-20T18:33:14.243680+0000","last_peered":"2024-05-20T18:33:14.243680+0000","last_clean":"2024-05-20T18:33:14.243680+0000","last_became_active":"2024-05-20T18:33:13.092433+0000","last_became_peered":"2024-05-20T18:33:13.092433+0000","last_unstale":"2024-05-20T18:33:14.243680+0000","last_undegraded":"2024-05-20T18:33:14.243680+0000","last_fullsized":"2024-05-20T18:33:14.243680+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:45:11.338040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099225+0000","last_change":"2024-05-20T18:33:12.093122+0000","last_active":"2024-05-20T18:33:14.099225+0000","last_peered":"2024-05-20T18:33:14.099225+0000","last_clean":"2024-05-20T18:33:14.099225+0000","last_became_active":"2024-05-20T18:33:12.092754+0000","last_became_peered":"2024-05-20T18:33:12.092754+0000","last_unstale":"2024-05-20T18:33:14.099225+0000","last_undegraded":"2024-05-20T18:33:14.099225+0000","last_fullsized":"2024-05-20T18:33:14.099225+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:56:13.575392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243939+0000","last_change":"2024-05-20T18:33:13.101534+0000","last_active":"2024-05-20T18:33:14.243939+0000","last_peered":"2024-05-20T18:33:14.243939+0000","last_clean":"2024-05-20T18:33:14.243939+0000","last_became_active":"2024-05-20T18:33:13.101146+0000","last_became_peered":"2024-05-20T18:33:13.101146+0000","last_unstale":"2024-05-20T18:33:14.243939+0000","last_undegraded":"2024-05-20T18:33:14.243939+0000","last_fullsized":"2024-05-20T18:33:14.243939+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:18:03.936593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.098935+0000","last_change":"2024-05-20T18:33:12.092546+0000","last_active":"2024-05-20T18:33:14.098935+0000","last_peered":"2024-05-20T18:33:14.098935+0000","last_clean":"2024-05-20T18:33:14.098935+0000","last_became_active":"2024-05-20T18:33:12.092003+0000","last_became_peered":"2024-05-20T18:33:12.092003+0000","last_unstale":"2024-05-20T18:33:14.098935+0000","last_undegraded":"2024-05-20T18:33:14.098935+0000","last_fullsized":"2024-05-20T18:33:14.098935+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:21:54.999282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099599+0000","last_change":"2024-05-20T18:33:13.089420+0000","last_active":"2024-05-20T18:33:14.099599+0000","last_peered":"2024-05-20T18:33:14.099599+0000","last_clean":"2024-05-20T18:33:14.099599+0000","last_became_active":"2024-05-20T18:33:13.088884+0000","last_became_peered":"2024-05-20T18:33:13.088884+0000","last_unstale":"2024-05-20T18:33:14.099599+0000","last_undegraded":"2024-05-20T18:33:14.099599+0000","last_fullsized":"2024-05-20T18:33:14.099599+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:18:03.936593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.244023+0000","last_change":"2024-05-20T18:33:12.091475+0000","last_active":"2024-05-20T18:33:14.244023+0000","last_peered":"2024-05-20T18:33:14.244023+0000","last_clean":"2024-05-20T18:33:14.244023+0000","last_became_active":"2024-05-20T18:33:12.091194+0000","last_became_peered":"2024-05-20T18:33:12.091194+0000","last_unstale":"2024-05-20T18:33:14.244023+0000","last_undegraded":"2024-05-20T18:33:14.244023+0000","last_fullsized":"2024-05-20T18:33:14.244023+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:33:26.376581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243064+0000","last_change":"2024-05-20T18:33:13.108254+0000","last_active":"2024-05-20T18:33:14.243064+0000","last_peered":"2024-05-20T18:33:14.243064+0000","last_clean":"2024-05-20T18:33:14.243064+0000","last_became_active":"2024-05-20T18:33:13.107875+0000","last_became_peered":"2024-05-20T18:33:13.107875+0000","last_unstale":"2024-05-20T18:33:14.243064+0000","last_undegraded":"2024-05-20T18:33:14.243064+0000","last_fullsized":"2024-05-20T18:33:14.243064+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:08:30.714625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.096691+0000","last_change":"2024-05-20T18:33:12.085691+0000","last_active":"2024-05-20T18:33:14.096691+0000","last_peered":"2024-05-20T18:33:14.096691+0000","last_clean":"2024-05-20T18:33:14.096691+0000","last_became_active":"2024-05-20T18:33:12.085396+0000","last_became_peered":"2024-05-20T18:33:12.085396+0000","last_unstale":"2024-05-20T18:33:14.096691+0000","last_undegraded":"2024-05-20T18:33:14.096691+0000","last_fullsized":"2024-05-20T18:33:14.096691+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:27:27.616696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243374+0000","last_change":"2024-05-20T18:33:13.102297+0000","last_active":"2024-05-20T18:33:14.243374+0000","last_peered":"2024-05-20T18:33:14.243374+0000","last_clean":"2024-05-20T18:33:14.243374+0000","last_became_active":"2024-05-20T18:33:13.101934+0000","last_became_peered":"2024-05-20T18:33:13.101934+0000","last_unstale":"2024-05-20T18:33:14.243374+0000","last_undegraded":"2024-05-20T18:33:14.243374+0000","last_fullsized":"2024-05-20T18:33:14.243374+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:16:15.456403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.242904+0000","last_change":"2024-05-20T18:33:12.094291+0000","last_active":"2024-05-20T18:33:14.242904+0000","last_peered":"2024-05-20T18:33:14.242904+0000","last_clean":"2024-05-20T18:33:14.242904+0000","last_became_active":"2024-05-20T18:33:12.093942+0000","last_became_peered":"2024-05-20T18:33:12.093942+0000","last_unstale":"2024-05-20T18:33:14.242904+0000","last_undegraded":"2024-05-20T18:33:14.242904+0000","last_fullsized":"2024-05-20T18:33:14.242904+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:44:05.126905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.098667+0000","last_change":"2024-05-20T18:33:13.099069+0000","last_active":"2024-05-20T18:33:14.098667+0000","last_peered":"2024-05-20T18:33:14.098667+0000","last_clean":"2024-05-20T18:33:14.098667+0000","last_became_active":"2024-05-20T18:33:13.098717+0000","last_became_peered":"2024-05-20T18:33:13.098717+0000","last_unstale":"2024-05-20T18:33:14.098667+0000","last_undegraded":"2024-05-20T18:33:14.098667+0000","last_fullsized":"2024-05-20T18:33:14.098667+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:23:27.680097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.104255+0000","last_change":"2024-05-20T18:33:12.092274+0000","last_active":"2024-05-20T18:33:14.104255+0000","last_peered":"2024-05-20T18:33:14.104255+0000","last_clean":"2024-05-20T18:33:14.104255+0000","last_became_active":"2024-05-20T18:33:12.091927+0000","last_became_peered":"2024-05-20T18:33:12.091927+0000","last_unstale":"2024-05-20T18:33:14.104255+0000","last_undegraded":"2024-05-20T18:33:14.104255+0000","last_fullsized":"2024-05-20T18:33:14.104255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:25:35.060122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243377+0000","last_change":"2024-05-20T18:33:13.107302+0000","last_active":"2024-05-20T18:33:14.243377+0000","last_peered":"2024-05-20T18:33:14.243377+0000","last_clean":"2024-05-20T18:33:14.243377+0000","last_became_active":"2024-05-20T18:33:13.106926+0000","last_became_peered":"2024-05-20T18:33:13.106926+0000","last_unstale":"2024-05-20T18:33:14.243377+0000","last_undegraded":"2024-05-20T18:33:14.243377+0000","last_fullsized":"2024-05-20T18:33:14.243377+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:07:50.193612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.096444+0000","last_change":"2024-05-20T18:33:12.092923+0000","last_active":"2024-05-20T18:33:14.096444+0000","last_peered":"2024-05-20T18:33:14.096444+0000","last_clean":"2024-05-20T18:33:14.096444+0000","last_became_active":"2024-05-20T18:33:12.092632+0000","last_became_peered":"2024-05-20T18:33:12.092632+0000","last_unstale":"2024-05-20T18:33:14.096444+0000","last_undegraded":"2024-05-20T18:33:14.096444+0000","last_fullsized":"2024-05-20T18:33:14.096444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:35:47.930046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.096530+0000","last_change":"2024-05-20T18:33:13.114561+0000","last_active":"2024-05-20T18:33:14.096530+0000","last_peered":"2024-05-20T18:33:14.096530+0000","last_clean":"2024-05-20T18:33:14.096530+0000","last_became_active":"2024-05-20T18:33:13.114253+0000","last_became_peered":"2024-05-20T18:33:13.114253+0000","last_unstale":"2024-05-20T18:33:14.096530+0000","last_undegraded":"2024-05-20T18:33:14.096530+0000","last_fullsized":"2024-05-20T18:33:14.096530+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:57:36.648081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"89'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099058+0000","last_change":"2024-05-20T18:33:12.085648+0000","last_active":"2024-05-20T18:33:14.099058+0000","last_peered":"2024-05-20T18:33:14.099058+0000","last_clean":"2024-05-20T18:33:14.099058+0000","last_became_active":"2024-05-20T18:33:12.085224+0000","last_became_peered":"2024-05-20T18:33:12.085224+0000","last_unstale":"2024-05-20T18:33:14.099058+0000","last_undegraded":"2024-05-20T18:33:14.099058+0000","last_fullsized":"2024-05-20T18:33:14.099058+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:12:03.967934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243446+0000","last_change":"2024-05-20T18:33:13.118642+0000","last_active":"2024-05-20T18:33:14.243446+0000","last_peered":"2024-05-20T18:33:14.243446+0000","last_clean":"2024-05-20T18:33:14.243446+0000","last_became_active":"2024-05-20T18:33:13.118314+0000","last_became_peered":"2024-05-20T18:33:13.118314+0000","last_unstale":"2024-05-20T18:33:14.243446+0000","last_undegraded":"2024-05-20T18:33:14.243446+0000","last_fullsized":"2024-05-20T18:33:14.243446+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:52:50.494627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"89'6","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.244004+0000","last_change":"2024-05-20T18:33:12.097189+0000","last_active":"2024-05-20T18:33:14.244004+0000","last_peered":"2024-05-20T18:33:14.244004+0000","last_clean":"2024-05-20T18:33:14.244004+0000","last_became_active":"2024-05-20T18:33:12.096897+0000","last_became_peered":"2024-05-20T18:33:12.096897+0000","last_unstale":"2024-05-20T18:33:14.244004+0000","last_undegraded":"2024-05-20T18:33:14.244004+0000","last_fullsized":"2024-05-20T18:33:14.244004+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:08:20.867030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243924+0000","last_change":"2024-05-20T18:33:13.106684+0000","last_active":"2024-05-20T18:33:14.243924+0000","last_peered":"2024-05-20T18:33:14.243924+0000","last_clean":"2024-05-20T18:33:14.243924+0000","last_became_active":"2024-05-20T18:33:13.106354+0000","last_became_peered":"2024-05-20T18:33:13.106354+0000","last_unstale":"2024-05-20T18:33:14.243924+0000","last_undegraded":"2024-05-20T18:33:14.243924+0000","last_fullsized":"2024-05-20T18:33:14.243924+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:23:27.680097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243199+0000","last_change":"2024-05-20T18:33:12.094515+0000","last_active":"2024-05-20T18:33:14.243199+0000","last_peered":"2024-05-20T18:33:14.243199+0000","last_clean":"2024-05-20T18:33:14.243199+0000","last_became_active":"2024-05-20T18:33:12.093783+0000","last_became_peered":"2024-05-20T18:33:12.093783+0000","last_unstale":"2024-05-20T18:33:14.243199+0000","last_undegraded":"2024-05-20T18:33:14.243199+0000","last_fullsized":"2024-05-20T18:33:14.243199+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:49:59.457852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.098291+0000","last_change":"2024-05-20T18:33:13.097112+0000","last_active":"2024-05-20T18:33:14.098291+0000","last_peered":"2024-05-20T18:33:14.098291+0000","last_clean":"2024-05-20T18:33:14.098291+0000","last_became_active":"2024-05-20T18:33:13.096761+0000","last_became_peered":"2024-05-20T18:33:13.096761+0000","last_unstale":"2024-05-20T18:33:14.098291+0000","last_undegraded":"2024-05-20T18:33:14.098291+0000","last_fullsized":"2024-05-20T18:33:14.098291+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:16:15.456403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243123+0000","last_change":"2024-05-20T18:33:12.089463+0000","last_active":"2024-05-20T18:33:14.243123+0000","last_peered":"2024-05-20T18:33:14.243123+0000","last_clean":"2024-05-20T18:33:14.243123+0000","last_became_active":"2024-05-20T18:33:12.089162+0000","last_became_peered":"2024-05-20T18:33:12.089162+0000","last_unstale":"2024-05-20T18:33:14.243123+0000","last_undegraded":"2024-05-20T18:33:14.243123+0000","last_fullsized":"2024-05-20T18:33:14.243123+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:37:15.845164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.242891+0000","last_change":"2024-05-20T18:33:13.112356+0000","last_active":"2024-05-20T18:33:14.242891+0000","last_peered":"2024-05-20T18:33:14.242891+0000","last_clean":"2024-05-20T18:33:14.242891+0000","last_became_active":"2024-05-20T18:33:13.112058+0000","last_became_peered":"2024-05-20T18:33:13.112058+0000","last_unstale":"2024-05-20T18:33:14.242891+0000","last_undegraded":"2024-05-20T18:33:14.242891+0000","last_fullsized":"2024-05-20T18:33:14.242891+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:18:24.988974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243581+0000","last_change":"2024-05-20T18:33:12.091927+0000","last_active":"2024-05-20T18:33:14.243581+0000","last_peered":"2024-05-20T18:33:14.243581+0000","last_clean":"2024-05-20T18:33:14.243581+0000","last_became_active":"2024-05-20T18:33:12.091635+0000","last_became_peered":"2024-05-20T18:33:12.091635+0000","last_unstale":"2024-05-20T18:33:14.243581+0000","last_undegraded":"2024-05-20T18:33:14.243581+0000","last_fullsized":"2024-05-20T18:33:14.243581+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:40:01.877694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.242788+0000","last_change":"2024-05-20T18:33:13.110476+0000","last_active":"2024-05-20T18:33:14.242788+0000","last_peered":"2024-05-20T18:33:14.242788+0000","last_clean":"2024-05-20T18:33:14.242788+0000","last_became_active":"2024-05-20T18:33:13.110145+0000","last_became_peered":"2024-05-20T18:33:13.110145+0000","last_unstale":"2024-05-20T18:33:14.242788+0000","last_undegraded":"2024-05-20T18:33:14.242788+0000","last_fullsized":"2024-05-20T18:33:14.242788+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:36:26.017336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"89'8","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.244007+0000","last_change":"2024-05-20T18:33:12.091517+0000","last_active":"2024-05-20T18:33:14.244007+0000","last_peered":"2024-05-20T18:33:14.244007+0000","last_clean":"2024-05-20T18:33:14.244007+0000","last_became_active":"2024-05-20T18:33:12.091249+0000","last_became_peered":"2024-05-20T18:33:12.091249+0000","last_unstale":"2024-05-20T18:33:14.244007+0000","last_undegraded":"2024-05-20T18:33:14.244007+0000","last_fullsized":"2024-05-20T18:33:14.244007+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:30:43.896361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.244016+0000","last_change":"2024-05-20T18:33:13.099468+0000","last_active":"2024-05-20T18:33:14.244016+0000","last_peered":"2024-05-20T18:33:14.244016+0000","last_clean":"2024-05-20T18:33:14.244016+0000","last_became_active":"2024-05-20T18:33:13.098799+0000","last_became_peered":"2024-05-20T18:33:13.098799+0000","last_unstale":"2024-05-20T18:33:14.244016+0000","last_undegraded":"2024-05-20T18:33:14.244016+0000","last_fullsized":"2024-05-20T18:33:14.244016+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:06:40.101331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099952+0000","last_change":"2024-05-20T18:33:12.090460+0000","last_active":"2024-05-20T18:33:14.099952+0000","last_peered":"2024-05-20T18:33:14.099952+0000","last_clean":"2024-05-20T18:33:14.099952+0000","last_became_active":"2024-05-20T18:33:12.090114+0000","last_became_peered":"2024-05-20T18:33:12.090114+0000","last_unstale":"2024-05-20T18:33:14.099952+0000","last_undegraded":"2024-05-20T18:33:14.099952+0000","last_fullsized":"2024-05-20T18:33:14.099952+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:41:08.634509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099189+0000","last_change":"2024-05-20T18:33:13.095643+0000","last_active":"2024-05-20T18:33:14.099189+0000","last_peered":"2024-05-20T18:33:14.099189+0000","last_clean":"2024-05-20T18:33:14.099189+0000","last_became_active":"2024-05-20T18:33:13.095269+0000","last_became_peered":"2024-05-20T18:33:13.095269+0000","last_unstale":"2024-05-20T18:33:14.099189+0000","last_undegraded":"2024-05-20T18:33:14.099189+0000","last_fullsized":"2024-05-20T18:33:14.099189+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:52:04.290712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243391+0000","last_change":"2024-05-20T18:33:12.092847+0000","last_active":"2024-05-20T18:33:14.243391+0000","last_peered":"2024-05-20T18:33:14.243391+0000","last_clean":"2024-05-20T18:33:14.243391+0000","last_became_active":"2024-05-20T18:33:12.092515+0000","last_became_peered":"2024-05-20T18:33:12.092515+0000","last_unstale":"2024-05-20T18:33:14.243391+0000","last_undegraded":"2024-05-20T18:33:14.243391+0000","last_fullsized":"2024-05-20T18:33:14.243391+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:56:20.494612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.098914+0000","last_change":"2024-05-20T18:33:13.093718+0000","last_active":"2024-05-20T18:33:14.098914+0000","last_peered":"2024-05-20T18:33:14.098914+0000","last_clean":"2024-05-20T18:33:14.098914+0000","last_became_active":"2024-05-20T18:33:13.093356+0000","last_became_peered":"2024-05-20T18:33:13.093356+0000","last_unstale":"2024-05-20T18:33:14.098914+0000","last_undegraded":"2024-05-20T18:33:14.098914+0000","last_fullsized":"2024-05-20T18:33:14.098914+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:21:39.855675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099354+0000","last_change":"2024-05-20T18:33:12.094666+0000","last_active":"2024-05-20T18:33:14.099354+0000","last_peered":"2024-05-20T18:33:14.099354+0000","last_clean":"2024-05-20T18:33:14.099354+0000","last_became_active":"2024-05-20T18:33:12.094351+0000","last_became_peered":"2024-05-20T18:33:12.094351+0000","last_unstale":"2024-05-20T18:33:14.099354+0000","last_undegraded":"2024-05-20T18:33:14.099354+0000","last_fullsized":"2024-05-20T18:33:14.099354+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:33:26.376581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243839+0000","last_change":"2024-05-20T18:33:13.109890+0000","last_active":"2024-05-20T18:33:14.243839+0000","last_peered":"2024-05-20T18:33:14.243839+0000","last_clean":"2024-05-20T18:33:14.243839+0000","last_became_active":"2024-05-20T18:33:13.109179+0000","last_became_peered":"2024-05-20T18:33:13.109179+0000","last_unstale":"2024-05-20T18:33:14.243839+0000","last_undegraded":"2024-05-20T18:33:14.243839+0000","last_fullsized":"2024-05-20T18:33:14.243839+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:22:26.619923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.103894+0000","last_change":"2024-05-20T18:33:12.091317+0000","last_active":"2024-05-20T18:33:14.103894+0000","last_peered":"2024-05-20T18:33:14.103894+0000","last_clean":"2024-05-20T18:33:14.103894+0000","last_became_active":"2024-05-20T18:33:12.089309+0000","last_became_peered":"2024-05-20T18:33:12.089309+0000","last_unstale":"2024-05-20T18:33:14.103894+0000","last_undegraded":"2024-05-20T18:33:14.103894+0000","last_fullsized":"2024-05-20T18:33:14.103894+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:54:25.489746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.103967+0000","last_change":"2024-05-20T18:33:13.102528+0000","last_active":"2024-05-20T18:33:14.103967+0000","last_peered":"2024-05-20T18:33:14.103967+0000","last_clean":"2024-05-20T18:33:14.103967+0000","last_became_active":"2024-05-20T18:33:13.102223+0000","last_became_peered":"2024-05-20T18:33:13.102223+0000","last_unstale":"2024-05-20T18:33:14.103967+0000","last_undegraded":"2024-05-20T18:33:14.103967+0000","last_fullsized":"2024-05-20T18:33:14.103967+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:56:21.500193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.242829+0000","last_change":"2024-05-20T18:33:12.091634+0000","last_active":"2024-05-20T18:33:14.242829+0000","last_peered":"2024-05-20T18:33:14.242829+0000","last_clean":"2024-05-20T18:33:14.242829+0000","last_became_active":"2024-05-20T18:33:12.091365+0000","last_became_peered":"2024-05-20T18:33:12.091365+0000","last_unstale":"2024-05-20T18:33:14.242829+0000","last_undegraded":"2024-05-20T18:33:14.242829+0000","last_fullsized":"2024-05-20T18:33:14.242829+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:15:30.398201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.103825+0000","last_change":"2024-05-20T18:33:13.108416+0000","last_active":"2024-05-20T18:33:14.103825+0000","last_peered":"2024-05-20T18:33:14.103825+0000","last_clean":"2024-05-20T18:33:14.103825+0000","last_became_active":"2024-05-20T18:33:13.107616+0000","last_became_peered":"2024-05-20T18:33:13.107616+0000","last_unstale":"2024-05-20T18:33:14.103825+0000","last_undegraded":"2024-05-20T18:33:14.103825+0000","last_fullsized":"2024-05-20T18:33:14.103825+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:50:00.463433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.103829+0000","last_change":"2024-05-20T18:33:12.087884+0000","last_active":"2024-05-20T18:33:14.103829+0000","last_peered":"2024-05-20T18:33:14.103829+0000","last_clean":"2024-05-20T18:33:14.103829+0000","last_became_active":"2024-05-20T18:33:12.087526+0000","last_became_peered":"2024-05-20T18:33:12.087526+0000","last_unstale":"2024-05-20T18:33:14.103829+0000","last_undegraded":"2024-05-20T18:33:14.103829+0000","last_fullsized":"2024-05-20T18:33:14.103829+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:21:38.850094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.242904+0000","last_change":"2024-05-20T18:33:13.099965+0000","last_active":"2024-05-20T18:33:14.242904+0000","last_peered":"2024-05-20T18:33:14.242904+0000","last_clean":"2024-05-20T18:33:14.242904+0000","last_became_active":"2024-05-20T18:33:13.099575+0000","last_became_peered":"2024-05-20T18:33:13.099575+0000","last_unstale":"2024-05-20T18:33:14.242904+0000","last_undegraded":"2024-05-20T18:33:14.242904+0000","last_fullsized":"2024-05-20T18:33:14.242904+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:05:02.392436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.242293+0000","last_change":"2024-05-20T18:33:12.083347+0000","last_active":"2024-05-20T18:33:14.242293+0000","last_peered":"2024-05-20T18:33:14.242293+0000","last_clean":"2024-05-20T18:33:14.242293+0000","last_became_active":"2024-05-20T18:33:12.083031+0000","last_became_peered":"2024-05-20T18:33:12.083031+0000","last_unstale":"2024-05-20T18:33:14.242293+0000","last_undegraded":"2024-05-20T18:33:14.242293+0000","last_fullsized":"2024-05-20T18:33:14.242293+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:30:28.580142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099145+0000","last_change":"2024-05-20T18:33:13.102762+0000","last_active":"2024-05-20T18:33:14.099145+0000","last_peered":"2024-05-20T18:33:14.099145+0000","last_clean":"2024-05-20T18:33:14.099145+0000","last_became_active":"2024-05-20T18:33:13.102382+0000","last_became_peered":"2024-05-20T18:33:13.102382+0000","last_unstale":"2024-05-20T18:33:14.099145+0000","last_undegraded":"2024-05-20T18:33:14.099145+0000","last_fullsized":"2024-05-20T18:33:14.099145+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:52:50.494627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243245+0000","last_change":"2024-05-20T18:33:12.089397+0000","last_active":"2024-05-20T18:33:14.243245+0000","last_peered":"2024-05-20T18:33:14.243245+0000","last_clean":"2024-05-20T18:33:14.243245+0000","last_became_active":"2024-05-20T18:33:12.089013+0000","last_became_peered":"2024-05-20T18:33:12.089013+0000","last_unstale":"2024-05-20T18:33:14.243245+0000","last_undegraded":"2024-05-20T18:33:14.243245+0000","last_fullsized":"2024-05-20T18:33:14.243245+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:33:28.814214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243488+0000","last_change":"2024-05-20T18:33:13.099641+0000","last_active":"2024-05-20T18:33:14.243488+0000","last_peered":"2024-05-20T18:33:14.243488+0000","last_clean":"2024-05-20T18:33:14.243488+0000","last_became_active":"2024-05-20T18:33:13.099123+0000","last_became_peered":"2024-05-20T18:33:13.099123+0000","last_unstale":"2024-05-20T18:33:14.243488+0000","last_undegraded":"2024-05-20T18:33:14.243488+0000","last_fullsized":"2024-05-20T18:33:14.243488+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:20:06.218692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"90'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099232+0000","last_change":"2024-05-20T18:33:12.093624+0000","last_active":"2024-05-20T18:33:14.099232+0000","last_peered":"2024-05-20T18:33:14.099232+0000","last_clean":"2024-05-20T18:33:14.099232+0000","last_became_active":"2024-05-20T18:33:12.092398+0000","last_became_peered":"2024-05-20T18:33:12.092398+0000","last_unstale":"2024-05-20T18:33:14.099232+0000","last_undegraded":"2024-05-20T18:33:14.099232+0000","last_fullsized":"2024-05-20T18:33:14.099232+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:40:01.877694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243184+0000","last_change":"2024-05-20T18:33:13.094860+0000","last_active":"2024-05-20T18:33:14.243184+0000","last_peered":"2024-05-20T18:33:14.243184+0000","last_clean":"2024-05-20T18:33:14.243184+0000","last_became_active":"2024-05-20T18:33:13.094548+0000","last_became_peered":"2024-05-20T18:33:13.094548+0000","last_unstale":"2024-05-20T18:33:14.243184+0000","last_undegraded":"2024-05-20T18:33:14.243184+0000","last_fullsized":"2024-05-20T18:33:14.243184+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:21:50.789908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099280+0000","last_change":"2024-05-20T18:33:12.085651+0000","last_active":"2024-05-20T18:33:14.099280+0000","last_peered":"2024-05-20T18:33:14.099280+0000","last_clean":"2024-05-20T18:33:14.099280+0000","last_became_active":"2024-05-20T18:33:12.085202+0000","last_became_peered":"2024-05-20T18:33:12.085202+0000","last_unstale":"2024-05-20T18:33:14.099280+0000","last_undegraded":"2024-05-20T18:33:14.099280+0000","last_fullsized":"2024-05-20T18:33:14.099280+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:45:26.428068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.104278+0000","last_change":"2024-05-20T18:33:13.101659+0000","last_active":"2024-05-20T18:33:14.104278+0000","last_peered":"2024-05-20T18:33:14.104278+0000","last_clean":"2024-05-20T18:33:14.104278+0000","last_became_active":"2024-05-20T18:33:13.101287+0000","last_became_peered":"2024-05-20T18:33:13.101287+0000","last_unstale":"2024-05-20T18:33:14.104278+0000","last_undegraded":"2024-05-20T18:33:14.104278+0000","last_fullsized":"2024-05-20T18:33:14.104278+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:46:43.255085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099366+0000","last_change":"2024-05-20T18:33:12.089871+0000","last_active":"2024-05-20T18:33:14.099366+0000","last_peered":"2024-05-20T18:33:14.099366+0000","last_clean":"2024-05-20T18:33:14.099366+0000","last_became_active":"2024-05-20T18:33:12.089475+0000","last_became_peered":"2024-05-20T18:33:12.089475+0000","last_unstale":"2024-05-20T18:33:14.099366+0000","last_undegraded":"2024-05-20T18:33:14.099366+0000","last_fullsized":"2024-05-20T18:33:14.099366+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:06:00.465798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.242372+0000","last_change":"2024-05-20T18:33:13.099974+0000","last_active":"2024-05-20T18:33:14.242372+0000","last_peered":"2024-05-20T18:33:14.242372+0000","last_clean":"2024-05-20T18:33:14.242372+0000","last_became_active":"2024-05-20T18:33:13.099435+0000","last_became_peered":"2024-05-20T18:33:13.099435+0000","last_unstale":"2024-05-20T18:33:14.242372+0000","last_undegraded":"2024-05-20T18:33:14.242372+0000","last_fullsized":"2024-05-20T18:33:14.242372+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:07:50.513306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243170+0000","last_change":"2024-05-20T18:33:12.090507+0000","last_active":"2024-05-20T18:33:14.243170+0000","last_peered":"2024-05-20T18:33:14.243170+0000","last_clean":"2024-05-20T18:33:14.243170+0000","last_became_active":"2024-05-20T18:33:12.090162+0000","last_became_peered":"2024-05-20T18:33:12.090162+0000","last_unstale":"2024-05-20T18:33:14.243170+0000","last_undegraded":"2024-05-20T18:33:14.243170+0000","last_fullsized":"2024-05-20T18:33:14.243170+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:33:23.682337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.096664+0000","last_change":"2024-05-20T18:33:13.105646+0000","last_active":"2024-05-20T18:33:14.096664+0000","last_peered":"2024-05-20T18:33:14.096664+0000","last_clean":"2024-05-20T18:33:14.096664+0000","last_became_active":"2024-05-20T18:33:13.105276+0000","last_became_peered":"2024-05-20T18:33:13.105276+0000","last_unstale":"2024-05-20T18:33:14.096664+0000","last_undegraded":"2024-05-20T18:33:14.096664+0000","last_fullsized":"2024-05-20T18:33:14.096664+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:17:03.547006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.103717+0000","last_change":"2024-05-20T18:33:12.090897+0000","last_active":"2024-05-20T18:33:14.103717+0000","last_peered":"2024-05-20T18:33:14.103717+0000","last_clean":"2024-05-20T18:33:14.103717+0000","last_became_active":"2024-05-20T18:33:12.090564+0000","last_became_peered":"2024-05-20T18:33:12.090564+0000","last_unstale":"2024-05-20T18:33:14.103717+0000","last_undegraded":"2024-05-20T18:33:14.103717+0000","last_fullsized":"2024-05-20T18:33:14.103717+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:17:57.276235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.098672+0000","last_change":"2024-05-20T18:33:13.105892+0000","last_active":"2024-05-20T18:33:14.098672+0000","last_peered":"2024-05-20T18:33:14.098672+0000","last_clean":"2024-05-20T18:33:14.098672+0000","last_became_active":"2024-05-20T18:33:13.105576+0000","last_became_peered":"2024-05-20T18:33:13.105576+0000","last_unstale":"2024-05-20T18:33:14.098672+0000","last_undegraded":"2024-05-20T18:33:14.098672+0000","last_fullsized":"2024-05-20T18:33:14.098672+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:16:15.456403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.099637+0000","last_change":"2024-05-20T18:33:12.091271+0000","last_active":"2024-05-20T18:33:14.099637+0000","last_peered":"2024-05-20T18:33:14.099637+0000","last_clean":"2024-05-20T18:33:14.099637+0000","last_became_active":"2024-05-20T18:33:12.090943+0000","last_became_peered":"2024-05-20T18:33:12.090943+0000","last_unstale":"2024-05-20T18:33:14.099637+0000","last_undegraded":"2024-05-20T18:33:14.099637+0000","last_fullsized":"2024-05-20T18:33:14.099637+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:23:32.815272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243106+0000","last_change":"2024-05-20T18:33:13.103661+0000","last_active":"2024-05-20T18:33:14.243106+0000","last_peered":"2024-05-20T18:33:14.243106+0000","last_clean":"2024-05-20T18:33:14.243106+0000","last_became_active":"2024-05-20T18:33:13.103295+0000","last_became_peered":"2024-05-20T18:33:13.103295+0000","last_unstale":"2024-05-20T18:33:14.243106+0000","last_undegraded":"2024-05-20T18:33:14.243106+0000","last_fullsized":"2024-05-20T18:33:14.243106+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:19:48.464838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243132+0000","last_change":"2024-05-20T18:33:12.095449+0000","last_active":"2024-05-20T18:33:14.243132+0000","last_peered":"2024-05-20T18:33:14.243132+0000","last_clean":"2024-05-20T18:33:14.243132+0000","last_became_active":"2024-05-20T18:33:12.095139+0000","last_became_peered":"2024-05-20T18:33:12.095139+0000","last_unstale":"2024-05-20T18:33:14.243132+0000","last_undegraded":"2024-05-20T18:33:14.243132+0000","last_fullsized":"2024-05-20T18:33:14.243132+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:35:55.978213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243365+0000","last_change":"2024-05-20T18:33:13.100731+0000","last_active":"2024-05-20T18:33:14.243365+0000","last_peered":"2024-05-20T18:33:14.243365+0000","last_clean":"2024-05-20T18:33:14.243365+0000","last_became_active":"2024-05-20T18:33:13.100375+0000","last_became_peered":"2024-05-20T18:33:13.100375+0000","last_unstale":"2024-05-20T18:33:14.243365+0000","last_undegraded":"2024-05-20T18:33:14.243365+0000","last_fullsized":"2024-05-20T18:33:14.243365+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:12.065942+0000","last_clean_scrub_stamp":"2024-05-20T18:33:12.065942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:52:45.545185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.244197+0000","last_change":"2024-05-20T18:33:12.099344+0000","last_active":"2024-05-20T18:33:14.244197+0000","last_peered":"2024-05-20T18:33:14.244197+0000","last_clean":"2024-05-20T18:33:14.244197+0000","last_became_active":"2024-05-20T18:33:12.099023+0000","last_became_peered":"2024-05-20T18:33:12.099023+0000","last_unstale":"2024-05-20T18:33:14.244197+0000","last_undegraded":"2024-05-20T18:33:14.244197+0000","last_fullsized":"2024-05-20T18:33:14.244197+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:33:11.060361+0000","last_clean_scrub_stamp":"2024-05-20T18:33:11.060361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:45:26.428068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":298,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-20T18:33:14.243232+0000","last_change":"2024-05-20T18:27:03.765694+0000","last_active":"2024-05-20T18:33:14.243232+0000","last_peered":"2024-05-20T18:33:14.243232+0000","last_clean":"2024-05-20T18:33:14.243232+0000","last_became_active":"2024-05-20T18:27:03.764695+0000","last_became_peered":"2024-05-20T18:27:03.764695+0000","last_unstale":"2024-05-20T18:33:14.243232+0000","last_undegraded":"2024-05-20T18:33:14.243232+0000","last_fullsized":"2024-05-20T18:33:14.243232+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_clean_scrub_stamp":"2024-05-20T18:27:02.750853+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:30:04.407487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574927,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1712128,"data_stored":1100013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1130496,"data_stored":511805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574925,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1110016,"data_stored":509589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574925,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1118208,"data_stored":509749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574927,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1110016,"data_stored":509589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574926,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1110016,"data_stored":509589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1114112,"data_stored":509659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574926,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32872,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338968,"statfs":{"total":96636764160,"available":96603103232,"internally_reserved":0,"allocated":1728512,"data_stored":1102275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T18:33:17.535 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-20T18:33:17.535 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-20T18:33:17.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.691+0000 7f9df437e640 1 Processor -- start 2024-05-20T18:33:17.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.691+0000 7f9df437e640 1 -- start start 2024-05-20T18:33:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.692+0000 7f9df437e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9dec163a10 0x7f9dec0d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.692+0000 7f9df437e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9dec0d14b0 0x7f9dec0d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.692+0000 7f9df437e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dec0d0370 con 0x7f9dec0b4010 2024-05-20T18:33:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.692+0000 7f9df437e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f9dec0d3ca0 con 0x7f9dec0d14b0 2024-05-20T18:33:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.692+0000 7f9df437e640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f9dec0d0650 con 0x7f9dec163a10 2024-05-20T18:33:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.692+0000 7f9df291b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9dec0d14b0 0x7f9dec0d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.692+0000 7f9df291b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9dec0d14b0 0x7f9dec0d18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58988/0 (socket says 172.21.15.5:58988) 2024-05-20T18:33:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.692+0000 7f9df291b640 1 -- 172.21.15.5:0/3324033929 learned_addr learned my addr 172.21.15.5:0/3324033929 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.693+0000 7f9df211a640 1 --2- 172.21.15.5:0/3324033929 >> v2:172.21.15.63:3300/0 conn(0x7f9dec163a10 0x7f9dec0d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.693+0000 7f9df291b640 1 -- 172.21.15.5:0/3324033929 >> v2:172.21.15.63:3300/0 conn(0x7f9dec163a10 msgr2=0x7f9dec0d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.693+0000 7f9df291b640 1 --2- 172.21.15.5:0/3324033929 >> v2:172.21.15.63:3300/0 conn(0x7f9dec163a10 0x7f9dec0d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.693+0000 7f9df291b640 1 -- 172.21.15.5:0/3324033929 >> v1:172.21.15.63:6789/0 conn(0x7f9dec0b4010 legacy=0x7f9dec0cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.693+0000 7f9df291b640 1 -- 172.21.15.5:0/3324033929 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dec0d3650 con 0x7f9dec0d14b0 2024-05-20T18:33:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.693+0000 7f9df291b640 1 --2- 172.21.15.5:0/3324033929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dec0d14b0 0x7f9dec0d18c0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f9dec0cf1c0 tx=0x7f9ddc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1303f62c881252 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.694+0000 7f9df1118640 1 -- 172.21.15.5:0/3324033929 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc036070 con 0x7f9dec0d14b0 2024-05-20T18:33:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.694+0000 7f9df1118640 1 -- 172.21.15.5:0/3324033929 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ddc031600 con 0x7f9dec0d14b0 2024-05-20T18:33:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.694+0000 7f9df1118640 1 -- 172.21.15.5:0/3324033929 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc0319d0 con 0x7f9dec0d14b0 2024-05-20T18:33:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.694+0000 7f9df437e640 1 -- 172.21.15.5:0/3324033929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dec0d14b0 msgr2=0x7f9dec0d18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.694+0000 7f9df437e640 1 --2- 172.21.15.5:0/3324033929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dec0d14b0 0x7f9dec0d18c0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f9dec0cf1c0 tx=0x7f9ddc02a6f0 comp rx=0 tx=0).stop 2024-05-20T18:33:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.694+0000 7f9df437e640 1 -- 172.21.15.5:0/3324033929 shutdown_connections 2024-05-20T18:33:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.694+0000 7f9df437e640 1 --2- 172.21.15.5:0/3324033929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dec0d14b0 0x7f9dec0d18c0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.694+0000 7f9df437e640 1 --2- 172.21.15.5:0/3324033929 >> v2:172.21.15.63:3300/0 conn(0x7f9dec163a10 0x7f9dec0d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.694+0000 7f9df437e640 1 -- 172.21.15.5:0/3324033929 >> 172.21.15.5:0/3324033929 conn(0x7f9dec06d240 msgr2=0x7f9dec06d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.694+0000 7f9df437e640 1 -- 172.21.15.5:0/3324033929 shutdown_connections 2024-05-20T18:33:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.694+0000 7f9df437e640 1 -- 172.21.15.5:0/3324033929 wait complete. 2024-05-20T18:33:17.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.696+0000 7f9df437e640 1 Processor -- start 2024-05-20T18:33:17.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.696+0000 7f9df437e640 1 -- start start 2024-05-20T18:33:17.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.698+0000 7f9df437e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9dec0b4010 0x7f9dec15d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:17.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.698+0000 7f9df437e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9dec0d14b0 0x7f9dec1615d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:17.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.698+0000 7f9df437e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dec163a10 0x7f9dec161b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:17.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.698+0000 7f9df437e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec0b8210 con 0x7f9dec163a10 2024-05-20T18:33:17.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.698+0000 7f9df437e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec0d1e00 con 0x7f9dec0b4010 2024-05-20T18:33:17.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.698+0000 7f9df437e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9dec0b8390 con 0x7f9dec0d14b0 2024-05-20T18:33:17.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.698+0000 7f9df291b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dec163a10 0x7f9dec161b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:17.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.698+0000 7f9df291b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dec163a10 0x7f9dec161b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58992/0 (socket says 172.21.15.5:58992) 2024-05-20T18:33:17.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.698+0000 7f9df291b640 1 -- 172.21.15.5:0/3527137181 learned_addr learned my addr 172.21.15.5:0/3527137181 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:17.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.698+0000 7f9df211a640 1 --2- 172.21.15.5:0/3527137181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9dec0b4010 0x7f9dec15d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:17.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.698+0000 7f9df1919640 1 --2- 172.21.15.5:0/3527137181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9dec0d14b0 0x7f9dec1615d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:17.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.698+0000 7f9df291b640 1 -- 172.21.15.5:0/3527137181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9dec0d14b0 msgr2=0x7f9dec1615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:17.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.698+0000 7f9df291b640 1 --2- 172.21.15.5:0/3527137181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9dec0d14b0 0x7f9dec1615d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:17.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.698+0000 7f9df291b640 1 -- 172.21.15.5:0/3527137181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9dec0b4010 msgr2=0x7f9dec15d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:17.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.698+0000 7f9df291b640 1 --2- 172.21.15.5:0/3527137181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9dec0b4010 0x7f9dec15d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:17.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.698+0000 7f9df291b640 1 -- 172.21.15.5:0/3527137181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dec0ab620 con 0x7f9dec163a10 2024-05-20T18:33:17.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.698+0000 7f9df291b640 1 --2- 172.21.15.5:0/3527137181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dec163a10 0x7f9dec161b10 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc034040 tx=0x7f9ddc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:17.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.699+0000 7f9deaffd640 1 -- 172.21.15.5:0/3527137181 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc036050 con 0x7f9dec163a10 2024-05-20T18:33:17.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.699+0000 7f9deaffd640 1 -- 172.21.15.5:0/3527137181 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ddc042070 con 0x7f9dec163a10 2024-05-20T18:33:17.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.699+0000 7f9deaffd640 1 -- 172.21.15.5:0/3527137181 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc009510 con 0x7f9dec163a10 2024-05-20T18:33:17.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.699+0000 7f9df437e640 1 -- 172.21.15.5:0/3527137181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dec0ab900 con 0x7f9dec163a10 2024-05-20T18:33:17.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.699+0000 7f9df437e640 1 -- 172.21.15.5:0/3527137181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dec0abec0 con 0x7f9dec163a10 2024-05-20T18:33:17.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.700+0000 7f9df437e640 1 -- 172.21.15.5:0/3527137181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dec075b00 con 0x7f9dec163a10 2024-05-20T18:33:17.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.700+0000 7f9deaffd640 1 -- 172.21.15.5:0/3527137181 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9ddc007bf0 con 0x7f9dec163a10 2024-05-20T18:33:17.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.701+0000 7f9deaffd640 1 --2- 172.21.15.5:0/3527137181 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9dd806be30 0x7f9dd806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:17.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.701+0000 7f9deaffd640 1 -- 172.21.15.5:0/3527137181 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6101+0+0 (secure 0 0 0) 0x7f9ddc0abfc0 con 0x7f9dec163a10 2024-05-20T18:33:17.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.701+0000 7f9df211a640 1 --2- 172.21.15.5:0/3527137181 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9dd806be30 0x7f9dd806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:17.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.701+0000 7f9df211a640 1 --2- 172.21.15.5:0/3527137181 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9dd806be30 0x7f9dd806e340 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9dec0d3020 tx=0x7f9de4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:17.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.705+0000 7f9deaffd640 1 -- 172.21.15.5:0/3527137181 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9ddc07a180 con 0x7f9dec163a10 2024-05-20T18:33:17.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.960+0000 7f9df437e640 1 -- 172.21.15.5:0/3527137181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f9dec0ceaa0 con 0x7f9dec163a10 2024-05-20T18:33:17.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.963+0000 7f9deaffd640 1 -- 172.21.15.5:0/3527137181 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7f9ddc07e480 con 0x7f9dec163a10 2024-05-20T18:33:17.963 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2024-05-20T18:33:17.964 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDFlUtmevaaIhAATIwyXzr0vGT38ZmcMqR2gA== 2024-05-20T18:33:17.964 INFO:teuthology.orchestra.run.smithi005.stdout: caps mds = "allow" 2024-05-20T18:33:17.964 INFO:teuthology.orchestra.run.smithi005.stdout: caps mon = "allow r" 2024-05-20T18:33:17.964 INFO:teuthology.orchestra.run.smithi005.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-20T18:33:17.964 INFO:teuthology.orchestra.run.smithi005.stderr:updated caps for client.0 2024-05-20T18:33:17.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.965+0000 7f9df437e640 1 -- 172.21.15.5:0/3527137181 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9dd806be30 msgr2=0x7f9dd806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:17.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.965+0000 7f9df437e640 1 --2- 172.21.15.5:0/3527137181 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9dd806be30 0x7f9dd806e340 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9dec0d3020 tx=0x7f9de4008040 comp rx=0 tx=0).stop 2024-05-20T18:33:17.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.965+0000 7f9df437e640 1 -- 172.21.15.5:0/3527137181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dec163a10 msgr2=0x7f9dec161b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:17.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.965+0000 7f9df437e640 1 --2- 172.21.15.5:0/3527137181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dec163a10 0x7f9dec161b10 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc034040 tx=0x7f9ddc004a40 comp rx=0 tx=0).stop 2024-05-20T18:33:17.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.965+0000 7f9df437e640 1 -- 172.21.15.5:0/3527137181 shutdown_connections 2024-05-20T18:33:17.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.965+0000 7f9df437e640 1 --2- 172.21.15.5:0/3527137181 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9dd806be30 0x7f9dd806e340 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:17.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.965+0000 7f9df437e640 1 --2- 172.21.15.5:0/3527137181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dec163a10 0x7f9dec161b10 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:17.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.965+0000 7f9df437e640 1 --2- 172.21.15.5:0/3527137181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9dec0d14b0 0x7f9dec1615d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:17.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.965+0000 7f9df437e640 1 --2- 172.21.15.5:0/3527137181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9dec0b4010 0x7f9dec15d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:17.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.965+0000 7f9df437e640 1 -- 172.21.15.5:0/3527137181 >> 172.21.15.5:0/3527137181 conn(0x7f9dec06d240 msgr2=0x7f9dec0ba460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:17.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.965+0000 7f9df437e640 1 -- 172.21.15.5:0/3527137181 shutdown_connections 2024-05-20T18:33:17.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:17.965+0000 7f9df437e640 1 -- 172.21.15.5:0/3527137181 wait complete. 2024-05-20T18:33:17.977 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:33:18.064 INFO:tasks.ceph.mds.d.smithi063.stderr:2024-05-20T18:33:18.064+0000 7f4d9b212640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T18:33:18.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.090+0000 7fec2be98640 1 Processor -- start 2024-05-20T18:33:18.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.091+0000 7fec2be98640 1 -- start start 2024-05-20T18:33:18.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.092+0000 7fec2be98640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fec240cf3d0 0x7fec240cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:18.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.092+0000 7fec2be98640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec240cfd20 con 0x7fec2415f220 2024-05-20T18:33:18.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.092+0000 7fec2be98640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec240cda20 con 0x7fec240ce7e0 2024-05-20T18:33:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.092+0000 7fec2be98640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fec240cdd00 con 0x7fec240cf3d0 2024-05-20T18:33:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.092+0000 7fec29433640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fec240cf3d0 0x7fec240cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.092+0000 7fec29c34640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fec2415f220 0x7fec240d08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35458/0 (socket says 172.21.15.5:35458) 2024-05-20T18:33:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.092+0000 7fec29c34640 1 -- 172.21.15.5:0/4166354536 learned_addr learned my addr 172.21.15.5:0/4166354536 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:33:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.093+0000 7fec29433640 1 -- 172.21.15.5:0/4166354536 >> v1:172.21.15.63:6789/0 conn(0x7fec240ce7e0 legacy=0x7fec240cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.093+0000 7fec29433640 1 -- 172.21.15.5:0/4166354536 >> v1:172.21.15.5:6789/0 conn(0x7fec2415f220 legacy=0x7fec240d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.093+0000 7fec29433640 1 -- 172.21.15.5:0/4166354536 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec240cdfe0 con 0x7fec240cf3d0 2024-05-20T18:33:18.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.093+0000 7fec28c32640 1 -- 172.21.15.5:0/4166354536 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 137539866 0 0) 0x7fec20001490 con 0x7fec2415f220 2024-05-20T18:33:18.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.093+0000 7fec29433640 1 --2- 172.21.15.5:0/4166354536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec240cf3d0 0x7fec240cf7e0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fec140097a0 tx=0x7fec1402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443ef7c8f300c7b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:18.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.093+0000 7fec28c32640 1 -- 172.21.15.5:0/4166354536 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec14037070 con 0x7fec240cf3d0 2024-05-20T18:33:18.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.093+0000 7fec28c32640 1 -- 172.21.15.5:0/4166354536 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec1402ae50 con 0x7fec240cf3d0 2024-05-20T18:33:18.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.093+0000 7fec28c32640 1 -- 172.21.15.5:0/4166354536 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec14032660 con 0x7fec240cf3d0 2024-05-20T18:33:18.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.094+0000 7fec2be98640 1 -- 172.21.15.5:0/4166354536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec240cf3d0 msgr2=0x7fec240cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:18.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.094+0000 7fec2be98640 1 --2- 172.21.15.5:0/4166354536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec240cf3d0 0x7fec240cf7e0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fec140097a0 tx=0x7fec1402a140 comp rx=0 tx=0).stop 2024-05-20T18:33:18.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.094+0000 7fec2be98640 1 -- 172.21.15.5:0/4166354536 shutdown_connections 2024-05-20T18:33:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.094+0000 7fec2be98640 1 --2- 172.21.15.5:0/4166354536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec240cf3d0 0x7fec240cf7e0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.094+0000 7fec2be98640 1 -- 172.21.15.5:0/4166354536 >> 172.21.15.5:0/4166354536 conn(0x7fec240b44d0 msgr2=0x7fec240b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.094+0000 7fec2be98640 1 -- 172.21.15.5:0/4166354536 shutdown_connections 2024-05-20T18:33:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.094+0000 7fec2be98640 1 -- 172.21.15.5:0/4166354536 wait complete. 2024-05-20T18:33:18.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.096+0000 7fec2be98640 1 Processor -- start 2024-05-20T18:33:18.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.096+0000 7fec2be98640 1 -- start start 2024-05-20T18:33:18.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.097+0000 7fec2be98640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec240ce7e0 0x7fec24151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:18.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.097+0000 7fec2be98640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec240cf3d0 0x7fec240b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:18.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.097+0000 7fec2be98640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec2415f220 0x7fec240cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:18.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.097+0000 7fec2be98640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fec240cd680 con 0x7fec240ce7e0 2024-05-20T18:33:18.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.097+0000 7fec2be98640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fec240d0fd0 con 0x7fec2415f220 2024-05-20T18:33:18.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.097+0000 7fec2be98640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fec240cd800 con 0x7fec240cf3d0 2024-05-20T18:33:18.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.097+0000 7fec29c34640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec240ce7e0 0x7fec24151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:18.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.098+0000 7fec29c34640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec240ce7e0 0x7fec24151f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59008/0 (socket says 172.21.15.5:59008) 2024-05-20T18:33:18.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.098+0000 7fec29c34640 1 -- 172.21.15.5:0/566465194 learned_addr learned my addr 172.21.15.5:0/566465194 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:18.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.098+0000 7fec2a435640 1 --2- 172.21.15.5:0/566465194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec2415f220 0x7fec240cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:18.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.098+0000 7fec29433640 1 --2- 172.21.15.5:0/566465194 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec240cf3d0 0x7fec240b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:18.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.098+0000 7fec29c34640 1 -- 172.21.15.5:0/566465194 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec240cf3d0 msgr2=0x7fec240b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:18.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.098+0000 7fec29c34640 1 --2- 172.21.15.5:0/566465194 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec240cf3d0 0x7fec240b55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.098+0000 7fec29c34640 1 -- 172.21.15.5:0/566465194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec2415f220 msgr2=0x7fec240cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:18.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.098+0000 7fec29c34640 1 --2- 172.21.15.5:0/566465194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec2415f220 0x7fec240cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.098+0000 7fec29c34640 1 -- 172.21.15.5:0/566465194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec240b7340 con 0x7fec240ce7e0 2024-05-20T18:33:18.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.098+0000 7fec2a435640 1 --2- 172.21.15.5:0/566465194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec2415f220 0x7fec240cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:33:18.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.098+0000 7fec29433640 1 --2- 172.21.15.5:0/566465194 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec240cf3d0 0x7fec240b55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:33:18.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.098+0000 7fec29c34640 1 --2- 172.21.15.5:0/566465194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec240ce7e0 0x7fec24151f50 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fec2000c7b0 tx=0x7fec2000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:18.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.098+0000 7fec1a7fc640 1 -- 172.21.15.5:0/566465194 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec20018070 con 0x7fec240ce7e0 2024-05-20T18:33:18.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.098+0000 7fec1a7fc640 1 -- 172.21.15.5:0/566465194 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec2000a640 con 0x7fec240ce7e0 2024-05-20T18:33:18.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.098+0000 7fec1a7fc640 1 -- 172.21.15.5:0/566465194 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2000aa30 con 0x7fec240ce7e0 2024-05-20T18:33:18.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.098+0000 7fec2be98640 1 -- 172.21.15.5:0/566465194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec240b62a0 con 0x7fec240ce7e0 2024-05-20T18:33:18.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.098+0000 7fec2be98640 1 -- 172.21.15.5:0/566465194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec240b6860 con 0x7fec240ce7e0 2024-05-20T18:33:18.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.099+0000 7fec2be98640 1 -- 172.21.15.5:0/566465194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec24075b00 con 0x7fec240ce7e0 2024-05-20T18:33:18.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.100+0000 7fec1a7fc640 1 -- 172.21.15.5:0/566465194 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fec2001b070 con 0x7fec240ce7e0 2024-05-20T18:33:18.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.100+0000 7fec1a7fc640 1 --2- 172.21.15.5:0/566465194 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fec0806be30 0x7fec0806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:18.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.100+0000 7fec1a7fc640 1 -- 172.21.15.5:0/566465194 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6101+0+0 (secure 0 0 0) 0x7fec2008a760 con 0x7fec240ce7e0 2024-05-20T18:33:18.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.100+0000 7fec29433640 1 --2- 172.21.15.5:0/566465194 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fec0806be30 0x7fec0806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:18.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.101+0000 7fec29433640 1 --2- 172.21.15.5:0/566465194 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fec0806be30 0x7fec0806e340 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fec14034000 tx=0x7fec1402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:18.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.105+0000 7fec1a7fc640 1 -- 172.21.15.5:0/566465194 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fec240cfd20 con 0x7fec240ce7e0 2024-05-20T18:33:18.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.329+0000 7fec2be98640 1 -- 172.21.15.5:0/566465194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec24162810 con 0x7fec240ce7e0 2024-05-20T18:33:18.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.329+0000 7fec1a7fc640 1 -- 172.21.15.5:0/566465194 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+5174 (secure 0 0 0) 0x7fec20058e50 con 0x7fec240ce7e0 2024-05-20T18:33:18.330 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:33:18.330 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3705772553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3705772553},{"type":"v1","addr":"172.21.15.63:6835","nonce":3705772553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":7434,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/776188138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":776188138},{"type":"v1","addr":"172.21.15.5:6835","nonce":776188138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":7437,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3556492262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3556492262},{"type":"v1","addr":"172.21.15.5:6837","nonce":3556492262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:33:12.457716+0000","modified":"2024-05-20T18:33:15.489024+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"d","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6834/2759396130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":2759396130},{"type":"v1","addr":"172.21.15.63:6834","nonce":2759396130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7449,"qdb_cluster":[7449]},"id":6}]} 2024-05-20T18:33:18.331 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 60 2024-05-20T18:33:18.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.331+0000 7fec2be98640 1 -- 172.21.15.5:0/566465194 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fec0806be30 msgr2=0x7fec0806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:18.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.331+0000 7fec2be98640 1 --2- 172.21.15.5:0/566465194 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fec0806be30 0x7fec0806e340 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fec14034000 tx=0x7fec1402e000 comp rx=0 tx=0).stop 2024-05-20T18:33:18.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.331+0000 7fec2be98640 1 -- 172.21.15.5:0/566465194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec240ce7e0 msgr2=0x7fec24151f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:18.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.331+0000 7fec2be98640 1 --2- 172.21.15.5:0/566465194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec240ce7e0 0x7fec24151f50 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fec2000c7b0 tx=0x7fec2000cc70 comp rx=0 tx=0).stop 2024-05-20T18:33:18.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.331+0000 7fec2be98640 1 -- 172.21.15.5:0/566465194 shutdown_connections 2024-05-20T18:33:18.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.331+0000 7fec2be98640 1 --2- 172.21.15.5:0/566465194 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fec0806be30 0x7fec0806e340 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.331+0000 7fec2be98640 1 --2- 172.21.15.5:0/566465194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec2415f220 0x7fec240cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.331+0000 7fec2be98640 1 --2- 172.21.15.5:0/566465194 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec240cf3d0 0x7fec240b55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.331+0000 7fec2be98640 1 --2- 172.21.15.5:0/566465194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec240ce7e0 0x7fec24151f50 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.331+0000 7fec2be98640 1 -- 172.21.15.5:0/566465194 >> 172.21.15.5:0/566465194 conn(0x7fec240b44d0 msgr2=0x7fec24162e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:18.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.331+0000 7fec2be98640 1 -- 172.21.15.5:0/566465194 shutdown_connections 2024-05-20T18:33:18.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.331+0000 7fec2be98640 1 -- 172.21.15.5:0/566465194 wait complete. 2024-05-20T18:33:18.746 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 60, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T18:33:12.457716+0000', 'modified': '2024-05-20T18:33:15.489024+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7449}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7449': {'gid': 7449, 'name': 'd', 'rank': 0, 'incarnation': 56, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.63:6834/2759396130', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6832', 'nonce': 2759396130}, {'type': 'v1', 'addr': '172.21.15.63:6834', 'nonce': 2759396130}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7449, 'qdb_cluster': [7449]} 2024-05-20T18:33:18.746 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-20T18:33:18.746 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-20T18:33:18.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.869+0000 7fae5466c640 1 Processor -- start 2024-05-20T18:33:18.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.869+0000 7fae5466c640 1 -- start start 2024-05-20T18:33:18.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.871+0000 7fae5466c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fae4c162820 0x7fae4c0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:18.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.871+0000 7fae5466c640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fae4c06dae0 0x7fae4c0b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:18.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.871+0000 7fae5466c640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae4c0d0630 con 0x7fae4c0b3e80 2024-05-20T18:33:18.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.871+0000 7fae5466c640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fae4c0d39c0 con 0x7fae4c162820 2024-05-20T18:33:18.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.871+0000 7fae5466c640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fae4c0d3ca0 con 0x7fae4c06dae0 2024-05-20T18:33:18.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.871+0000 7fae52408640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fae4c162820 0x7fae4c0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:18.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.871+0000 7fae51c07640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fae4c0b3e80 0x7fae4c06d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35474/0 (socket says 172.21.15.5:35474) 2024-05-20T18:33:18.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.871+0000 7fae51c07640 1 -- 172.21.15.5:0/3209994265 learned_addr learned my addr 172.21.15.5:0/3209994265 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:33:18.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.871+0000 7fae52c09640 1 --2- 172.21.15.5:0/3209994265 >> v2:172.21.15.63:3300/0 conn(0x7fae4c06dae0 0x7fae4c0b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:18.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.871+0000 7fae52408640 1 -- 172.21.15.5:0/3209994265 >> v2:172.21.15.63:3300/0 conn(0x7fae4c06dae0 msgr2=0x7fae4c0b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:18.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.871+0000 7fae52408640 1 --2- 172.21.15.5:0/3209994265 >> v2:172.21.15.63:3300/0 conn(0x7fae4c06dae0 0x7fae4c0b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.871+0000 7fae52408640 1 -- 172.21.15.5:0/3209994265 >> v1:172.21.15.5:6789/0 conn(0x7fae4c0b3e80 legacy=0x7fae4c06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:18.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.871+0000 7fae52408640 1 -- 172.21.15.5:0/3209994265 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4c0d3f80 con 0x7fae4c162820 2024-05-20T18:33:18.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.871+0000 7fae51406640 1 -- 172.21.15.5:0/3209994265 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3571796606 0 0) 0x7fae4c0d0630 con 0x7fae4c0b3e80 2024-05-20T18:33:18.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.872+0000 7fae52408640 1 --2- 172.21.15.5:0/3209994265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae4c162820 0x7fae4c0b3940 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fae4c08e070 tx=0x7fae4802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b8fb3ce60a969b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:18.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.872+0000 7fae51406640 1 -- 172.21.15.5:0/3209994265 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae48037070 con 0x7fae4c162820 2024-05-20T18:33:18.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.872+0000 7fae51406640 1 -- 172.21.15.5:0/3209994265 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae48032440 con 0x7fae4c162820 2024-05-20T18:33:18.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.872+0000 7fae51406640 1 -- 172.21.15.5:0/3209994265 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae48032810 con 0x7fae4c162820 2024-05-20T18:33:18.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.873+0000 7fae5466c640 1 -- 172.21.15.5:0/3209994265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae4c162820 msgr2=0x7fae4c0b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:18.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.873+0000 7fae5466c640 1 --2- 172.21.15.5:0/3209994265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae4c162820 0x7fae4c0b3940 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fae4c08e070 tx=0x7fae4802a1a0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.873+0000 7fae5466c640 1 -- 172.21.15.5:0/3209994265 shutdown_connections 2024-05-20T18:33:18.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.873+0000 7fae5466c640 1 --2- 172.21.15.5:0/3209994265 >> v2:172.21.15.63:3300/0 conn(0x7fae4c06dae0 0x7fae4c0b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.873+0000 7fae5466c640 1 --2- 172.21.15.5:0/3209994265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae4c162820 0x7fae4c0b3940 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.873+0000 7fae5466c640 1 -- 172.21.15.5:0/3209994265 >> 172.21.15.5:0/3209994265 conn(0x7fae4c0d4bd0 msgr2=0x7fae4c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:18.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.873+0000 7fae5466c640 1 -- 172.21.15.5:0/3209994265 shutdown_connections 2024-05-20T18:33:18.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.873+0000 7fae5466c640 1 -- 172.21.15.5:0/3209994265 wait complete. 2024-05-20T18:33:18.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.875+0000 7fae5466c640 1 Processor -- start 2024-05-20T18:33:18.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.875+0000 7fae5466c640 1 -- start start 2024-05-20T18:33:18.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.876+0000 7fae5466c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae4c0b3e80 0x7fae4c0ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:18.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.876+0000 7fae5466c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae4c162820 0x7fae4c14cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:18.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.876+0000 7fae5466c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae4c0cd9f0 0x7fae4c0cde00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:18.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.876+0000 7fae5466c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c0d4780 con 0x7fae4c0b3e80 2024-05-20T18:33:18.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.876+0000 7fae5466c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c0d0300 con 0x7fae4c0cd9f0 2024-05-20T18:33:18.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.876+0000 7fae5466c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fae4c0d4900 con 0x7fae4c162820 2024-05-20T18:33:18.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.876+0000 7fae52408640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae4c0b3e80 0x7fae4c0ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:18.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.876+0000 7fae52408640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae4c0b3e80 0x7fae4c0ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59026/0 (socket says 172.21.15.5:59026) 2024-05-20T18:33:18.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.876+0000 7fae52408640 1 -- 172.21.15.5:0/3398860820 learned_addr learned my addr 172.21.15.5:0/3398860820 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:18.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.876+0000 7fae52c09640 1 --2- 172.21.15.5:0/3398860820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae4c0cd9f0 0x7fae4c0cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:18.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.876+0000 7fae51c07640 1 --2- 172.21.15.5:0/3398860820 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae4c162820 0x7fae4c14cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:18.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.877+0000 7fae52408640 1 -- 172.21.15.5:0/3398860820 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae4c162820 msgr2=0x7fae4c14cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:18.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.877+0000 7fae52408640 1 --2- 172.21.15.5:0/3398860820 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae4c162820 0x7fae4c14cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.877+0000 7fae52408640 1 -- 172.21.15.5:0/3398860820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae4c0cd9f0 msgr2=0x7fae4c0cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:18.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.877+0000 7fae52408640 1 --2- 172.21.15.5:0/3398860820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae4c0cd9f0 0x7fae4c0cde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.877+0000 7fae52408640 1 -- 172.21.15.5:0/3398860820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4c0cbd20 con 0x7fae4c0b3e80 2024-05-20T18:33:18.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.877+0000 7fae52c09640 1 --2- 172.21.15.5:0/3398860820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae4c0cd9f0 0x7fae4c0cde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:33:18.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.877+0000 7fae52408640 1 --2- 172.21.15.5:0/3398860820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae4c0b3e80 0x7fae4c0ccdc0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fae48000c00 tx=0x7fae480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:18.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.877+0000 7fae42ffd640 1 -- 172.21.15.5:0/3398860820 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae48037040 con 0x7fae4c0b3e80 2024-05-20T18:33:18.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.877+0000 7fae42ffd640 1 -- 172.21.15.5:0/3398860820 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae48042070 con 0x7fae4c0b3e80 2024-05-20T18:33:18.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.877+0000 7fae5466c640 1 -- 172.21.15.5:0/3398860820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae4c0cc000 con 0x7fae4c0b3e80 2024-05-20T18:33:18.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.877+0000 7fae42ffd640 1 -- 172.21.15.5:0/3398860820 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae48030bd0 con 0x7fae4c0b3e80 2024-05-20T18:33:18.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.877+0000 7fae5466c640 1 -- 172.21.15.5:0/3398860820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae4c0cc5c0 con 0x7fae4c0b3e80 2024-05-20T18:33:18.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.879+0000 7fae42ffd640 1 -- 172.21.15.5:0/3398860820 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fae48047460 con 0x7fae4c0b3e80 2024-05-20T18:33:18.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.880+0000 7fae42ffd640 1 --2- 172.21.15.5:0/3398860820 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae3006be30 0x7fae3006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:18.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.880+0000 7fae42ffd640 1 -- 172.21.15.5:0/3398860820 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6101+0+0 (secure 0 0 0) 0x7fae480abcf0 con 0x7fae4c0b3e80 2024-05-20T18:33:18.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.880+0000 7fae51c07640 1 --2- 172.21.15.5:0/3398860820 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae3006be30 0x7fae3006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:18.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.880+0000 7fae51c07640 1 --2- 172.21.15.5:0/3398860820 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae3006be30 0x7fae3006e340 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fae4c14d770 tx=0x7fae3c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:18.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.881+0000 7fae5b5ef740 1 Processor -- start 2024-05-20T18:33:18.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.881+0000 7fae5b5ef740 1 -- start start 2024-05-20T18:33:18.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.882+0000 7fae5b5ef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603d11aa270 0x5603d1575700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:18.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.882+0000 7fae5b5ef740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5603d11813b0 0x5603d15720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:18.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.882+0000 7fae5b5ef740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603d1189590 0x5603d154b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:18.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.882+0000 7fae5b5ef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5603d14ae4d0 con 0x5603d11aa270 2024-05-20T18:33:18.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.882+0000 7fae5b5ef740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5603d14563e0 con 0x5603d1189590 2024-05-20T18:33:18.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.882+0000 7fae5b5ef740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5603d14aa490 con 0x5603d11813b0 2024-05-20T18:33:18.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.882+0000 7fae52408640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603d11aa270 0x5603d1575700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:18.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.882+0000 7fae52408640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603d11aa270 0x5603d1575700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59036/0 (socket says 172.21.15.5:59036) 2024-05-20T18:33:18.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.882+0000 7fae52408640 1 -- 172.21.15.5:0/1544452264 learned_addr learned my addr 172.21.15.5:0/1544452264 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:18.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.882+0000 7fae51c07640 1 --2- 172.21.15.5:0/1544452264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5603d11813b0 0x5603d15720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:18.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.882+0000 7fae52c09640 1 --2- 172.21.15.5:0/1544452264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603d1189590 0x5603d154b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:18.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.883+0000 7fae52408640 1 -- 172.21.15.5:0/1544452264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5603d11813b0 msgr2=0x5603d15720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:18.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.883+0000 7fae52408640 1 --2- 172.21.15.5:0/1544452264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5603d11813b0 0x5603d15720a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.883+0000 7fae52408640 1 -- 172.21.15.5:0/1544452264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603d1189590 msgr2=0x5603d154b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:18.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.883+0000 7fae52408640 1 --2- 172.21.15.5:0/1544452264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603d1189590 0x5603d154b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.883+0000 7fae52408640 1 -- 172.21.15.5:0/1544452264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603d11c22d0 con 0x5603d11aa270 2024-05-20T18:33:18.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.883+0000 7fae52408640 1 --2- 172.21.15.5:0/1544452264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603d11aa270 0x5603d1575700 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fae480761e0 tx=0x7fae480796e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebb4ccf9a74c06e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:18.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.883+0000 7fae2ffff640 1 -- 172.21.15.5:0/1544452264 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4807e130 con 0x5603d11aa270 2024-05-20T18:33:18.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.883+0000 7fae2ffff640 1 -- 172.21.15.5:0/1544452264 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae480829f0 con 0x5603d11aa270 2024-05-20T18:33:18.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.883+0000 7fae2ffff640 1 -- 172.21.15.5:0/1544452264 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae48030d70 con 0x5603d11aa270 2024-05-20T18:33:18.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.884+0000 7fae5b5ef740 1 -- 172.21.15.5:0/1544452264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603d11aa270 msgr2=0x5603d1575700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:18.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.884+0000 7fae5b5ef740 1 --2- 172.21.15.5:0/1544452264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603d11aa270 0x5603d1575700 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fae480761e0 tx=0x7fae480796e0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.884+0000 7fae5b5ef740 1 -- 172.21.15.5:0/1544452264 shutdown_connections 2024-05-20T18:33:18.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.884+0000 7fae5b5ef740 1 --2- 172.21.15.5:0/1544452264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603d1189590 0x5603d154b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.884+0000 7fae5b5ef740 1 --2- 172.21.15.5:0/1544452264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5603d11813b0 0x5603d15720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.884+0000 7fae5b5ef740 1 --2- 172.21.15.5:0/1544452264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603d11aa270 0x5603d1575700 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.884+0000 7fae5b5ef740 1 -- 172.21.15.5:0/1544452264 >> 172.21.15.5:0/1544452264 conn(0x5603d119e3c0 msgr2=0x5603d154aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:18.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.884+0000 7fae5b5ef740 1 -- 172.21.15.5:0/1544452264 shutdown_connections 2024-05-20T18:33:18.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.884+0000 7fae5b5ef740 1 -- 172.21.15.5:0/1544452264 wait complete. 2024-05-20T18:33:18.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.885+0000 7fae5b5ef740 1 -- start start 2024-05-20T18:33:18.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.885+0000 7fae5b5ef740 1 Processor -- start 2024-05-20T18:33:18.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.886+0000 7fae5b5ef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603d11813b0 0x5603d15a27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:18.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.886+0000 7fae5b5ef740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5603d1189590 0x5603d15a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:18.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.886+0000 7fae5b5ef740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603d11aa270 0x5603d158ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:18.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.886+0000 7fae5b5ef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5603d14ae4d0 con 0x5603d11813b0 2024-05-20T18:33:18.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.886+0000 7fae5b5ef740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5603d14698e0 con 0x5603d11aa270 2024-05-20T18:33:18.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.886+0000 7fae5b5ef740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5603d1570f10 con 0x5603d1189590 2024-05-20T18:33:18.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.886+0000 7fae5b5ef740 10 client.0 _ll_register_callbacks cb 0x5603d14369c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:33:18.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.886+0000 7fae52408640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603d11813b0 0x5603d15a27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:18.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.886+0000 7fae52408640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603d11813b0 0x5603d15a27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59044/0 (socket says 172.21.15.5:59044) 2024-05-20T18:33:18.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.886+0000 7fae52408640 1 -- 172.21.15.5:0/44960978 learned_addr learned my addr 172.21.15.5:0/44960978 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:18.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.887+0000 7fae52c09640 1 --2- 172.21.15.5:0/44960978 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603d11aa270 0x5603d158ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:18.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.887+0000 7fae51c07640 1 --2- 172.21.15.5:0/44960978 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5603d1189590 0x5603d15a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:18.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.887+0000 7fae52408640 1 -- 172.21.15.5:0/44960978 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5603d1189590 msgr2=0x5603d15a2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:18.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.887+0000 7fae52408640 1 --2- 172.21.15.5:0/44960978 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5603d1189590 0x5603d15a2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.887+0000 7fae52408640 1 -- 172.21.15.5:0/44960978 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603d11aa270 msgr2=0x5603d158ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:18.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.887+0000 7fae52408640 1 --2- 172.21.15.5:0/44960978 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603d11aa270 0x5603d158ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.887+0000 7fae52408640 1 -- 172.21.15.5:0/44960978 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603d128c6b0 con 0x5603d11813b0 2024-05-20T18:33:18.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.887+0000 7fae51c07640 1 --2- 172.21.15.5:0/44960978 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5603d1189590 0x5603d15a2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:18.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.887+0000 7fae52408640 1 --2- 172.21.15.5:0/44960978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603d11813b0 0x5603d15a27f0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fae480761e0 tx=0x7fae48049070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f2505d8f2e6af5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:18.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.887+0000 7fae2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:33:18.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.887+0000 7fae2d7fa640 1 -- 172.21.15.5:0/44960978 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4807e130 con 0x5603d11813b0 2024-05-20T18:33:18.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.887+0000 7fae2d7fa640 1 -- 172.21.15.5:0/44960978 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6101+0+0 (secure 0 0 0) 0x7fae480763f0 con 0x5603d11813b0 2024-05-20T18:33:18.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.887+0000 7fae5b5ef740 1 -- 172.21.15.5:0/44960978 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae480829f0 con 0x5603d11813b0 2024-05-20T18:33:18.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.887+0000 7fae2d7fa640 1 -- 172.21.15.5:0/44960978 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fae480a0050 con 0x5603d11813b0 2024-05-20T18:33:18.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.887+0000 7fae5b5ef740 10 client.7587 fetch_fsmap learned FSMap version 60 2024-05-20T18:33:18.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.887+0000 7fae5b5ef740 1 -- 172.21.15.5:0/44960978 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) -- 0x5603d1570c50 con 0x5603d11813b0 2024-05-20T18:33:18.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.887+0000 7fae2d7fa640 1 -- 172.21.15.5:0/44960978 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae480829f0 con 0x5603d11813b0 2024-05-20T18:33:18.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.887+0000 7fae2d7fa640 1 -- 172.21.15.5:0/44960978 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 60) ==== 3494+0+0 (secure 0 0 0) 0x7fae480a3020 con 0x5603d11813b0 2024-05-20T18:33:18.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.888+0000 7fae5b5ef740 10 client.7587 fetch_fsmap finished waiting for FSMap version 60 2024-05-20T18:33:18.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.888+0000 7fae5b5ef740 10 client.7587 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-20T18:33:18.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.888+0000 7fae5b5ef740 20 client.7587 populate_metadata read hostname 'smithi005' 2024-05-20T18:33:18.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.888+0000 7fae5b5ef740 1 --2- 172.21.15.5:0/44960978 >> [v2:172.21.15.63:6832/2759396130,v1:172.21.15.63:6834/2759396130] conn(0x5603d1168e40 0x5603d15accf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:18.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.888+0000 7fae5b5ef740 4 client.7587 mds_command: new command op to 7449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:33:18.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.888+0000 7fae5b5ef740 1 -- 172.21.15.5:0/44960978 --> [v2:172.21.15.63:6832/2759396130,v1:172.21.15.63:6834/2759396130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603d14bbc00 con 0x5603d1168e40 2024-05-20T18:33:18.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.888+0000 7fae51c07640 1 --2- 172.21.15.5:0/44960978 >> [v2:172.21.15.63:6832/2759396130,v1:172.21.15.63:6834/2759396130] conn(0x5603d1168e40 0x5603d15accf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:18.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.888+0000 7fae2d7fa640 10 client.7587 ms_handle_connect on v2:172.21.15.63:6832/2759396130 2024-05-20T18:33:18.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.888+0000 7fae51c07640 1 --2- 172.21.15.5:0/44960978 >> [v2:172.21.15.63:6832/2759396130,v1:172.21.15.63:6834/2759396130] conn(0x5603d1168e40 0x5603d15accf0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f42408a802dc4520 server_cookie=6b0dcdfc8267768b in_seq=0 out_seq=0 2024-05-20T18:33:18.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.889+0000 7fae2d7fa640 1 -- 172.21.15.5:0/44960978 <== mds.0 v2:172.21.15.63:6832/2759396130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5603d14bbc00 con 0x5603d1168e40 2024-05-20T18:33:18.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.889+0000 7fae2d7fa640 10 client.7587 handle_command_reply: tid=0 2024-05-20T18:33:18.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.889+0000 7fae5b5ef740 1 client.7587 shutdown 2024-05-20T18:33:18.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.889+0000 7fae5b5ef740 1 -- 172.21.15.5:0/44960978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603d11813b0 msgr2=0x5603d15a27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:18.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.889+0000 7fae5b5ef740 1 --2- 172.21.15.5:0/44960978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603d11813b0 0x5603d15a27f0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fae480761e0 tx=0x7fae48049070 comp rx=0 tx=0).stop 2024-05-20T18:33:18.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.889+0000 7fae5b5ef740 1 -- 172.21.15.5:0/44960978 shutdown_connections 2024-05-20T18:33:18.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.889+0000 7fae5b5ef740 1 --2- 172.21.15.5:0/44960978 >> [v2:172.21.15.63:6832/2759396130,v1:172.21.15.63:6834/2759396130] conn(0x5603d1168e40 0x5603d15accf0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.889+0000 7fae5b5ef740 1 --2- 172.21.15.5:0/44960978 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603d11aa270 0x5603d158ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.889+0000 7fae5b5ef740 1 --2- 172.21.15.5:0/44960978 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5603d1189590 0x5603d15a2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.889+0000 7fae5b5ef740 1 --2- 172.21.15.5:0/44960978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603d11813b0 0x5603d15a27f0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.889+0000 7fae2d7fa640 0 client.7587 ms_handle_reset on v2:172.21.15.63:6832/2759396130 2024-05-20T18:33:18.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.889+0000 7fae5b5ef740 1 -- 172.21.15.5:0/44960978 >> 172.21.15.5:0/44960978 conn(0x5603d119e3c0 msgr2=0x5603d154aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:18.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.890+0000 7fae5b5ef740 1 -- 172.21.15.5:0/44960978 shutdown_connections 2024-05-20T18:33:18.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.890+0000 7fae5b5ef740 1 -- 172.21.15.5:0/44960978 wait complete. 2024-05-20T18:33:18.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.890+0000 7fae5b5ef740 20 client.7587 trim_cache size 0 max 16384 2024-05-20T18:33:18.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.909+0000 7fae5b5ef740 1 Processor -- start 2024-05-20T18:33:18.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.909+0000 7fae5b5ef740 1 -- start start 2024-05-20T18:33:18.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.910+0000 7fae5b5ef740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603d11813b0 0x5603d1576f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:18.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.910+0000 7fae5b5ef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603d1189590 0x5603d15a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:18.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.910+0000 7fae5b5ef740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5603d11aa270 0x5603d156fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:18.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.910+0000 7fae5b5ef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c201160 con 0x5603d1189590 2024-05-20T18:33:18.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.910+0000 7fae5b5ef740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c0ca4c0 con 0x5603d11813b0 2024-05-20T18:33:18.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.910+0000 7fae5b5ef740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fae4c200b10 con 0x5603d11aa270 2024-05-20T18:33:18.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.910+0000 7fae51c07640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603d1189590 0x5603d15a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:18.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.910+0000 7fae51c07640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603d1189590 0x5603d15a2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59046/0 (socket says 172.21.15.5:59046) 2024-05-20T18:33:18.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.910+0000 7fae51c07640 1 -- 172.21.15.5:0/1576815364 learned_addr learned my addr 172.21.15.5:0/1576815364 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:18.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.910+0000 7fae52c09640 1 --2- 172.21.15.5:0/1576815364 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5603d11aa270 0x5603d156fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:18.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.910+0000 7fae52408640 1 --2- 172.21.15.5:0/1576815364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603d11813b0 0x5603d1576f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:18.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.910+0000 7fae51c07640 1 -- 172.21.15.5:0/1576815364 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5603d11aa270 msgr2=0x5603d156fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:18.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.910+0000 7fae51c07640 1 --2- 172.21.15.5:0/1576815364 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5603d11aa270 0x5603d156fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.910+0000 7fae51c07640 1 -- 172.21.15.5:0/1576815364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603d11813b0 msgr2=0x5603d1576f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:18.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.910+0000 7fae51c07640 1 --2- 172.21.15.5:0/1576815364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603d11813b0 0x5603d1576f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.910+0000 7fae51c07640 1 -- 172.21.15.5:0/1576815364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603d1544220 con 0x5603d1189590 2024-05-20T18:33:18.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.910+0000 7fae52408640 1 --2- 172.21.15.5:0/1576815364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603d11813b0 0x5603d1576f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:18.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.911+0000 7fae51c07640 1 --2- 172.21.15.5:0/1576815364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603d1189590 0x5603d15a2ae0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fae3c007fd0 tx=0x7fae3c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ff470015203c95 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:18.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.911+0000 7fae2f7fe640 1 -- 172.21.15.5:0/1576815364 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c005c80 con 0x5603d1189590 2024-05-20T18:33:18.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.911+0000 7fae2f7fe640 1 -- 172.21.15.5:0/1576815364 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae3c02a020 con 0x5603d1189590 2024-05-20T18:33:18.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.911+0000 7fae2f7fe640 1 -- 172.21.15.5:0/1576815364 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c005e20 con 0x5603d1189590 2024-05-20T18:33:18.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.911+0000 7fae5b5ef740 1 -- 172.21.15.5:0/1576815364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603d1189590 msgr2=0x5603d15a2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:18.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.911+0000 7fae5b5ef740 1 --2- 172.21.15.5:0/1576815364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603d1189590 0x5603d15a2ae0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fae3c007fd0 tx=0x7fae3c011000 comp rx=0 tx=0).stop 2024-05-20T18:33:18.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.911+0000 7fae5b5ef740 1 -- 172.21.15.5:0/1576815364 shutdown_connections 2024-05-20T18:33:18.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.911+0000 7fae5b5ef740 1 --2- 172.21.15.5:0/1576815364 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5603d11aa270 0x5603d156fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.911+0000 7fae5b5ef740 1 --2- 172.21.15.5:0/1576815364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603d1189590 0x5603d15a2ae0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.911+0000 7fae5b5ef740 1 --2- 172.21.15.5:0/1576815364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603d11813b0 0x5603d1576f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.911+0000 7fae5b5ef740 1 -- 172.21.15.5:0/1576815364 >> 172.21.15.5:0/1576815364 conn(0x5603d119e3c0 msgr2=0x5603d154c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:18.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.912+0000 7fae5b5ef740 1 -- 172.21.15.5:0/1576815364 shutdown_connections 2024-05-20T18:33:18.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.912+0000 7fae5b5ef740 1 -- 172.21.15.5:0/1576815364 wait complete. 2024-05-20T18:33:18.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.912+0000 7fae5b5ef740 1 -- start start 2024-05-20T18:33:18.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.912+0000 7fae5b5ef740 1 Processor -- start 2024-05-20T18:33:18.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.914+0000 7fae5b5ef740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5603d11813b0 0x5603d1593340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:18.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.914+0000 7fae5b5ef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603d1189590 0x5603d1593880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:18.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.914+0000 7fae5b5ef740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603d11aa270 0x5603d1591840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:18.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.914+0000 7fae5b5ef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5603d14ae4d0 con 0x5603d1189590 2024-05-20T18:33:18.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.914+0000 7fae5b5ef740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5603d14698e0 con 0x5603d11aa270 2024-05-20T18:33:18.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.914+0000 7fae5b5ef740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5603d15707f0 con 0x5603d11813b0 2024-05-20T18:33:18.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.914+0000 7fae5b5ef740 10 client.0 _ll_register_callbacks cb 0x7fae4807e020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:33:18.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.914+0000 7fae51c07640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603d1189590 0x5603d1593880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:18.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.914+0000 7fae51c07640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603d1189590 0x5603d1593880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59050/0 (socket says 172.21.15.5:59050) 2024-05-20T18:33:18.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.914+0000 7fae51c07640 1 -- 172.21.15.5:0/1815998196 learned_addr learned my addr 172.21.15.5:0/1815998196 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:18.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.914+0000 7fae52408640 1 --2- 172.21.15.5:0/1815998196 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5603d11813b0 0x5603d1593340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:18.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.914+0000 7fae52c09640 1 --2- 172.21.15.5:0/1815998196 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603d11aa270 0x5603d1591840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:18.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.914+0000 7fae51c07640 1 -- 172.21.15.5:0/1815998196 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5603d11813b0 msgr2=0x5603d1593340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:18.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.914+0000 7fae51c07640 1 --2- 172.21.15.5:0/1815998196 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5603d11813b0 0x5603d1593340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.914+0000 7fae51c07640 1 -- 172.21.15.5:0/1815998196 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603d11aa270 msgr2=0x5603d1591840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:18.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.914+0000 7fae51c07640 1 --2- 172.21.15.5:0/1815998196 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603d11aa270 0x5603d1591840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.914+0000 7fae51c07640 1 -- 172.21.15.5:0/1815998196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603d15712b0 con 0x5603d1189590 2024-05-20T18:33:18.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.914+0000 7fae52c09640 1 --2- 172.21.15.5:0/1815998196 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603d11aa270 0x5603d1591840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:18.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.915+0000 7fae51c07640 1 --2- 172.21.15.5:0/1815998196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603d1189590 0x5603d1593880 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fae3c0075c0 tx=0x7fae3c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ded7ba5ccfd4f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:18.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.915+0000 7fae2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:33:18.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.915+0000 7fae2d7fa640 1 -- 172.21.15.5:0/1815998196 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c029070 con 0x5603d1189590 2024-05-20T18:33:18.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.915+0000 7fae2d7fa640 1 -- 172.21.15.5:0/1815998196 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6101+0+0 (secure 0 0 0) 0x7fae3c00db70 con 0x5603d1189590 2024-05-20T18:33:18.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.915+0000 7fae5b5ef740 1 -- 172.21.15.5:0/1815998196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae3c02a020 con 0x5603d1189590 2024-05-20T18:33:18.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.915+0000 7fae2d7fa640 1 -- 172.21.15.5:0/1815998196 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fae3c00dd70 con 0x5603d1189590 2024-05-20T18:33:18.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.915+0000 7fae5b5ef740 10 client.7593 fetch_fsmap learned FSMap version 60 2024-05-20T18:33:18.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.915+0000 7fae5b5ef740 1 -- 172.21.15.5:0/1815998196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) -- 0x5603d15927d0 con 0x5603d1189590 2024-05-20T18:33:18.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.915+0000 7fae2d7fa640 1 -- 172.21.15.5:0/1815998196 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae3c02a020 con 0x5603d1189590 2024-05-20T18:33:18.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.915+0000 7fae2d7fa640 1 -- 172.21.15.5:0/1815998196 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 60) ==== 3494+0+0 (secure 0 0 0) 0x7fae3c041080 con 0x5603d1189590 2024-05-20T18:33:18.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.915+0000 7fae5b5ef740 10 client.7593 fetch_fsmap finished waiting for FSMap version 60 2024-05-20T18:33:18.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.915+0000 7fae5b5ef740 10 client.7593 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-20T18:33:18.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.915+0000 7fae5b5ef740 20 client.7593 populate_metadata read hostname 'smithi005' 2024-05-20T18:33:18.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.915+0000 7fae5b5ef740 1 --2- 172.21.15.5:0/1815998196 >> [v2:172.21.15.63:6832/2759396130,v1:172.21.15.63:6834/2759396130] conn(0x5603d15921b0 0x5603d15a12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:18.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.915+0000 7fae5b5ef740 4 client.7593 mds_command: new command op to 7449 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T18:33:18.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.915+0000 7fae5b5ef740 1 -- 172.21.15.5:0/1815998196 --> [v2:172.21.15.63:6832/2759396130,v1:172.21.15.63:6834/2759396130] -- command(tid 0: {"prefix": "status"}) -- 0x5603d15778a0 con 0x5603d15921b0 2024-05-20T18:33:18.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.916+0000 7fae52408640 1 --2- 172.21.15.5:0/1815998196 >> [v2:172.21.15.63:6832/2759396130,v1:172.21.15.63:6834/2759396130] conn(0x5603d15921b0 0x5603d15a12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:18.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.916+0000 7fae2d7fa640 10 client.7593 ms_handle_connect on v2:172.21.15.63:6832/2759396130 2024-05-20T18:33:18.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.916+0000 7fae52408640 1 --2- 172.21.15.5:0/1815998196 >> [v2:172.21.15.63:6832/2759396130,v1:172.21.15.63:6834/2759396130] conn(0x5603d15921b0 0x5603d15a12e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f089125e2c093209 server_cookie=b747e9edc78131fd in_seq=0 out_seq=0 2024-05-20T18:33:18.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.916+0000 7fae2d7fa640 1 -- 172.21.15.5:0/1815998196 <== mds.0 v2:172.21.15.63:6832/2759396130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5603d15778a0 con 0x5603d15921b0 2024-05-20T18:33:18.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.916+0000 7fae2d7fa640 10 client.7593 handle_command_reply: tid=0 2024-05-20T18:33:18.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.916+0000 7fae5b5ef740 1 client.7593 shutdown 2024-05-20T18:33:18.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.916+0000 7fae5b5ef740 1 -- 172.21.15.5:0/1815998196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603d1189590 msgr2=0x5603d1593880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:18.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.916+0000 7fae5b5ef740 1 --2- 172.21.15.5:0/1815998196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603d1189590 0x5603d1593880 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fae3c0075c0 tx=0x7fae3c010000 comp rx=0 tx=0).stop 2024-05-20T18:33:18.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.917+0000 7fae5b5ef740 1 -- 172.21.15.5:0/1815998196 shutdown_connections 2024-05-20T18:33:18.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.917+0000 7fae5b5ef740 1 --2- 172.21.15.5:0/1815998196 >> [v2:172.21.15.63:6832/2759396130,v1:172.21.15.63:6834/2759396130] conn(0x5603d15921b0 0x5603d15a12e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.917+0000 7fae5b5ef740 1 --2- 172.21.15.5:0/1815998196 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5603d11aa270 0x5603d1591840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.917+0000 7fae5b5ef740 1 --2- 172.21.15.5:0/1815998196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603d1189590 0x5603d1593880 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.917+0000 7fae2d7fa640 0 client.7593 ms_handle_reset on v2:172.21.15.63:6832/2759396130 2024-05-20T18:33:18.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.917+0000 7fae5b5ef740 1 --2- 172.21.15.5:0/1815998196 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5603d11813b0 0x5603d1593340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.917+0000 7fae5b5ef740 1 -- 172.21.15.5:0/1815998196 >> 172.21.15.5:0/1815998196 conn(0x5603d119e3c0 msgr2=0x5603d154c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:18.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.917+0000 7fae5b5ef740 1 -- 172.21.15.5:0/1815998196 shutdown_connections 2024-05-20T18:33:18.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.917+0000 7fae5b5ef740 1 -- 172.21.15.5:0/1815998196 wait complete. 2024-05-20T18:33:18.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.917+0000 7fae5b5ef740 20 client.7593 trim_cache size 0 max 16384 2024-05-20T18:33:18.923 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:33:18.923 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "399ab250-fd3c-47fa-a1c8-25e8149c9cac", 2024-05-20T18:33:18.923 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-20T18:33:18.923 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 7449, 2024-05-20T18:33:18.923 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-20T18:33:18.923 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-20T18:33:18.924 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-20T18:33:18.924 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 5.8227479280000001, 2024-05-20T18:33:18.924 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 60, 2024-05-20T18:33:18.924 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 90, 2024-05-20T18:33:18.924 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 90, 2024-05-20T18:33:18.924 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 11.763513477 2024-05-20T18:33:18.924 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:33:18.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.917+0000 7fae5466c640 1 -- 172.21.15.5:0/3398860820 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae3006be30 msgr2=0x7fae3006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:18.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.917+0000 7fae5466c640 1 --2- 172.21.15.5:0/3398860820 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae3006be30 0x7fae3006e340 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fae4c14d770 tx=0x7fae3c009040 comp rx=0 tx=0).stop 2024-05-20T18:33:18.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.917+0000 7fae5466c640 1 -- 172.21.15.5:0/3398860820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae4c0b3e80 msgr2=0x7fae4c0ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:18.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.917+0000 7fae5466c640 1 --2- 172.21.15.5:0/3398860820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae4c0b3e80 0x7fae4c0ccdc0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fae48000c00 tx=0x7fae480046c0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.917+0000 7fae5466c640 1 -- 172.21.15.5:0/3398860820 shutdown_connections 2024-05-20T18:33:18.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.917+0000 7fae5466c640 1 --2- 172.21.15.5:0/3398860820 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae3006be30 0x7fae3006e340 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.917+0000 7fae5466c640 1 --2- 172.21.15.5:0/3398860820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae4c0cd9f0 0x7fae4c0cde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.917+0000 7fae5466c640 1 --2- 172.21.15.5:0/3398860820 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae4c162820 0x7fae4c14cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.917+0000 7fae5466c640 1 --2- 172.21.15.5:0/3398860820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae4c0b3e80 0x7fae4c0ccdc0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:18.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.917+0000 7fae5466c640 1 -- 172.21.15.5:0/3398860820 >> 172.21.15.5:0/3398860820 conn(0x7fae4c0d4bd0 msgr2=0x7fae4c15e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:18.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.917+0000 7fae5466c640 1 -- 172.21.15.5:0/3398860820 shutdown_connections 2024-05-20T18:33:18.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:18.917+0000 7fae5466c640 1 -- 172.21.15.5:0/3398860820 wait complete. 2024-05-20T18:33:18.929 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-20T18:33:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.043+0000 7fd461b3e640 1 Processor -- start 2024-05-20T18:33:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.043+0000 7fd461b3e640 1 -- start start 2024-05-20T18:33:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.045+0000 7fd461b3e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd45c162820 0x7fd45c0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.045+0000 7fd461b3e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd45c0b3e80 0x7fd45c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.045+0000 7fd461b3e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd45c06dd20 con 0x7fd45c06d910 2024-05-20T18:33:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.045+0000 7fd461b3e640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd45c0d0630 con 0x7fd45c0b3e80 2024-05-20T18:33:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.045+0000 7fd461b3e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd45c0d39c0 con 0x7fd45c162820 2024-05-20T18:33:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.045+0000 7fd45b7fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd45c162820 0x7fd45c0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.045+0000 7fd45affd640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd45c0b3e80 0x7fd45c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.045+0000 7fd45b7fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd45c162820 0x7fd45c0b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:39198/0 (socket says 172.21.15.5:39198) 2024-05-20T18:33:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.045+0000 7fd45b7fe640 1 -- 172.21.15.5:0/1056753930 learned_addr learned my addr 172.21.15.5:0/1056753930 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.045+0000 7fd45affd640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd45c0b3e80 0x7fd45c06d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:34134/0 (socket says 172.21.15.5:34134) 2024-05-20T18:33:19.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.046+0000 7fd45b7fe640 1 -- 172.21.15.5:0/1056753930 >> v2:172.21.15.63:3300/0 conn(0x7fd45c0b3e80 msgr2=0x7fd45c06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:19.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.046+0000 7fd45b7fe640 1 --2- 172.21.15.5:0/1056753930 >> v2:172.21.15.63:3300/0 conn(0x7fd45c0b3e80 0x7fd45c06d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:19.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.046+0000 7fd45b7fe640 1 -- 172.21.15.5:0/1056753930 >> v1:172.21.15.63:6789/0 conn(0x7fd45c06d910 legacy=0x7fd45c0d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:19.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.046+0000 7fd45b7fe640 1 -- 172.21.15.5:0/1056753930 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd45c0d3ca0 con 0x7fd45c162820 2024-05-20T18:33:19.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.046+0000 7fd45b7fe640 1 --2- 172.21.15.5:0/1056753930 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd45c162820 0x7fd45c0b3940 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd44c009870 tx=0x7fd44c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae3e90dee23abd62 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:19.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.047+0000 7fd45a7fc640 1 -- 172.21.15.5:0/1056753930 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c037070 con 0x7fd45c162820 2024-05-20T18:33:19.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.047+0000 7fd45a7fc640 1 -- 172.21.15.5:0/1056753930 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd44c032440 con 0x7fd45c162820 2024-05-20T18:33:19.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.047+0000 7fd45a7fc640 1 -- 172.21.15.5:0/1056753930 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c032810 con 0x7fd45c162820 2024-05-20T18:33:19.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.047+0000 7fd461b3e640 1 -- 172.21.15.5:0/1056753930 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd45c162820 msgr2=0x7fd45c0b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:19.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.047+0000 7fd461b3e640 1 --2- 172.21.15.5:0/1056753930 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd45c162820 0x7fd45c0b3940 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd44c009870 tx=0x7fd44c02a270 comp rx=0 tx=0).stop 2024-05-20T18:33:19.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.047+0000 7fd461b3e640 1 -- 172.21.15.5:0/1056753930 shutdown_connections 2024-05-20T18:33:19.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.047+0000 7fd461b3e640 1 --2- 172.21.15.5:0/1056753930 >> v2:172.21.15.63:3300/0 conn(0x7fd45c0b3e80 0x7fd45c06d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:19.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.047+0000 7fd461b3e640 1 --2- 172.21.15.5:0/1056753930 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd45c162820 0x7fd45c0b3940 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:19.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.047+0000 7fd461b3e640 1 -- 172.21.15.5:0/1056753930 >> 172.21.15.5:0/1056753930 conn(0x7fd45c0d4bd0 msgr2=0x7fd45c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:19.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.047+0000 7fd461b3e640 1 -- 172.21.15.5:0/1056753930 shutdown_connections 2024-05-20T18:33:19.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.047+0000 7fd461b3e640 1 -- 172.21.15.5:0/1056753930 wait complete. 2024-05-20T18:33:19.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.049+0000 7fd461b3e640 1 Processor -- start 2024-05-20T18:33:19.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.049+0000 7fd461b3e640 1 -- start start 2024-05-20T18:33:19.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.051+0000 7fd461b3e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd45c06d910 0x7fd45c14ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.051+0000 7fd461b3e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd45c0b3e80 0x7fd45c0c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.051+0000 7fd461b3e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd45c162820 0x7fd45c0c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.051+0000 7fd461b3e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd45c0d0300 con 0x7fd45c06d910 2024-05-20T18:33:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.051+0000 7fd461b3e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd45c0d4260 con 0x7fd45c0b3e80 2024-05-20T18:33:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.051+0000 7fd461b3e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd45c0b91a0 con 0x7fd45c162820 2024-05-20T18:33:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.051+0000 7fd45b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd45c06d910 0x7fd45c14ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.051+0000 7fd45b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd45c06d910 0x7fd45c14ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59066/0 (socket says 172.21.15.5:59066) 2024-05-20T18:33:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.051+0000 7fd45b7fe640 1 -- 172.21.15.5:0/911473144 learned_addr learned my addr 172.21.15.5:0/911473144 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.051+0000 7fd45affd640 1 --2- 172.21.15.5:0/911473144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd45c0b3e80 0x7fd45c0c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.051+0000 7fd45bfff640 1 --2- 172.21.15.5:0/911473144 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd45c162820 0x7fd45c0c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.051+0000 7fd45b7fe640 1 -- 172.21.15.5:0/911473144 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd45c162820 msgr2=0x7fd45c0c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.051+0000 7fd45b7fe640 1 --2- 172.21.15.5:0/911473144 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd45c162820 0x7fd45c0c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.051+0000 7fd45b7fe640 1 -- 172.21.15.5:0/911473144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd45c0b3e80 msgr2=0x7fd45c0c93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:19.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.051+0000 7fd45b7fe640 1 --2- 172.21.15.5:0/911473144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd45c0b3e80 0x7fd45c0c93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:19.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.051+0000 7fd45b7fe640 1 -- 172.21.15.5:0/911473144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd45c14d480 con 0x7fd45c06d910 2024-05-20T18:33:19.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.051+0000 7fd45b7fe640 1 --2- 172.21.15.5:0/911473144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd45c06d910 0x7fd45c14ce30 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fd44c02a2b0 tx=0x7fd44c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:19.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.051+0000 7fd443fff640 1 -- 172.21.15.5:0/911473144 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c037040 con 0x7fd45c06d910 2024-05-20T18:33:19.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.052+0000 7fd443fff640 1 -- 172.21.15.5:0/911473144 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd44c042070 con 0x7fd45c06d910 2024-05-20T18:33:19.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.052+0000 7fd461b3e640 1 -- 172.21.15.5:0/911473144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd45c0ca150 con 0x7fd45c06d910 2024-05-20T18:33:19.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.052+0000 7fd443fff640 1 -- 172.21.15.5:0/911473144 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c030880 con 0x7fd45c06d910 2024-05-20T18:33:19.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.052+0000 7fd461b3e640 1 -- 172.21.15.5:0/911473144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd45c14da00 con 0x7fd45c06d910 2024-05-20T18:33:19.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.053+0000 7fd461b3e640 1 -- 172.21.15.5:0/911473144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd45c075b00 con 0x7fd45c06d910 2024-05-20T18:33:19.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.053+0000 7fd443fff640 1 -- 172.21.15.5:0/911473144 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd44c030a20 con 0x7fd45c06d910 2024-05-20T18:33:19.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.054+0000 7fd443fff640 1 --2- 172.21.15.5:0/911473144 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd44806be60 0x7fd44806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:19.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.054+0000 7fd45affd640 1 --2- 172.21.15.5:0/911473144 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd44806be60 0x7fd44806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:19.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.054+0000 7fd443fff640 1 -- 172.21.15.5:0/911473144 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6101+0+0 (secure 0 0 0) 0x7fd44c0ab6d0 con 0x7fd45c06d910 2024-05-20T18:33:19.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.054+0000 7fd45affd640 1 --2- 172.21.15.5:0/911473144 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd44806be60 0x7fd44806e370 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd45c15d2a0 tx=0x7fd450009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:19.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.059+0000 7fd443fff640 1 -- 172.21.15.5:0/911473144 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd44c0798a0 con 0x7fd45c06d910 2024-05-20T18:33:19.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.283+0000 7fd461b3e640 1 -- 172.21.15.5:0/911473144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd45c0d52c0 con 0x7fd45c06d910 2024-05-20T18:33:19.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.283+0000 7fd443fff640 1 -- 172.21.15.5:0/911473144 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v60) ==== 53+0+73 (secure 0 0 0) 0x7fd44c0800c0 con 0x7fd45c06d910 2024-05-20T18:33:19.284 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-20T18:33:19.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.286+0000 7fd461b3e640 1 -- 172.21.15.5:0/911473144 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd44806be60 msgr2=0x7fd44806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:19.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.286+0000 7fd461b3e640 1 --2- 172.21.15.5:0/911473144 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd44806be60 0x7fd44806e370 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd45c15d2a0 tx=0x7fd450009190 comp rx=0 tx=0).stop 2024-05-20T18:33:19.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.286+0000 7fd461b3e640 1 -- 172.21.15.5:0/911473144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd45c06d910 msgr2=0x7fd45c14ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:19.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.286+0000 7fd461b3e640 1 --2- 172.21.15.5:0/911473144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd45c06d910 0x7fd45c14ce30 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fd44c02a2b0 tx=0x7fd44c0046c0 comp rx=0 tx=0).stop 2024-05-20T18:33:19.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.286+0000 7fd461b3e640 1 -- 172.21.15.5:0/911473144 shutdown_connections 2024-05-20T18:33:19.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.286+0000 7fd461b3e640 1 --2- 172.21.15.5:0/911473144 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd44806be60 0x7fd44806e370 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:19.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.286+0000 7fd461b3e640 1 --2- 172.21.15.5:0/911473144 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd45c162820 0x7fd45c0c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:19.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.286+0000 7fd461b3e640 1 --2- 172.21.15.5:0/911473144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd45c0b3e80 0x7fd45c0c93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:19.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.286+0000 7fd461b3e640 1 --2- 172.21.15.5:0/911473144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd45c06d910 0x7fd45c14ce30 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:19.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.286+0000 7fd461b3e640 1 -- 172.21.15.5:0/911473144 >> 172.21.15.5:0/911473144 conn(0x7fd45c0d4bd0 msgr2=0x7fd45c0b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:19.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.286+0000 7fd461b3e640 1 -- 172.21.15.5:0/911473144 shutdown_connections 2024-05-20T18:33:19.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19.286+0000 7fd461b3e640 1 -- 172.21.15.5:0/911473144 wait complete. 2024-05-20T18:33:19.298 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-20T18:33:19.298 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-20T18:33:19.298 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-05-20T18:33:19.298 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-20T18:33:19.298 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-20T18:33:19.298 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-20T18:33:19.299 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-20T18:33:19.299 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-20T18:33:19.299 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:33:19.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:33:19.299 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-05-20T18:33:19.316 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T18:33:19.316 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T18:33:19.316 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T18:33:19.316 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:33:19.316 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-20T18:33:19.316 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:33:19.316 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:33:19.317 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-05-20T18:33:19.317 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-05-20T18:33:19.317 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:33:19.317 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-05-20T18:33:19.317 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-05-20T18:33:19.317 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-20T18:33:19.317 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:33:19.317 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-05-20T18:33:19.317 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-20T18:33:19.317 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42299sec preferred_lft 42299sec 2024-05-20T18:33:19.317 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-05-20T18:33:19.318 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:33:19.318 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:33:19.318 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:33:19.318 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-05-20T18:33:19.318 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-20T18:33:19.318 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:33:19.318 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-20T18:33:19.318 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:33:19.318 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::4474:eaff:fe07:4f9a/64 scope link 2024-05-20T18:33:19.318 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:33:19.319 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:33:19.319 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T18:33:19.319 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ce4:f4ff:fe5e:7906/64 scope link 2024-05-20T18:33:19.319 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:33:19.319 INFO:teuthology.orchestra.run.smithi005.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:33:19.319 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether da:26:04:0e:95:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T18:33:19.319 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::d826:4ff:fe0e:950a/64 scope link 2024-05-20T18:33:19.319 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:33:19.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:33:19.320 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-20T18:33:19.380 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-20T18:33:19.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:33:19.382 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-05-20T18:33:19.440 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-05-20T18:33:19.440 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-20T18:33:19.440 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-20T18:33:19.440 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-20T18:33:19.440 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-20T18:33:19.440 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:33:19.440 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-20T18:33:19.441 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-20T18:33:19.441 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-20T18:33:19.441 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-20T18:33:19.441 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-20T18:33:19.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:33:19.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:33:19.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:33:19.570 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-20T18:33:19.587 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-20T18:33:19.587 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-20T18:33:19.587 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:33:19.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:33:19.587 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-05-20T18:33:19.654 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:33:19.654 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-20T18:33:19.710 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:33:19.710 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:33:19.711 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:33:19.766 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-20T18:33:19.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:33:19.766 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:33:19.862 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:33:19.862 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:33:19.862 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:33:19.873 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:33:19.873 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:33:19.873 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:33:19.873 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:33:19.873 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:33:19.893 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:33:19.893 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:33:19.893 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:33:19.893 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:33:19.893 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:33:19.893 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790" 2024-05-20T18:33:19.893 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790:/ 2024-05-20T18:33:19.893 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=399ab250-fd3c-47fa-a1c8-25e8149c9cac" will pass to kernel 2024-05-20T18:33:19.894 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-20T18:33:19.894 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-05-20T18:33:19.894 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-20T18:33:19.894 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-05-20T18:33:19.894 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-20T18:33:19.895 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-20T18:33:19.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:33:19.895 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:33:19.923 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-20T18:33:20.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T18:33:20.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:33:20.028 INFO:teuthology.orchestra.run.smithi005.stdout:client7599 2024-05-20T18:33:20.028 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:33:20.028 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:33:20.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:33:20.046 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir) 2024-05-20T18:33:20.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:33:20.110 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir/file) 2024-05-20T18:33:20.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:33:20.168 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-05-20T18:33:20.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:33:20.226 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir/file testdir/hardlink) 2024-05-20T18:33:20.287 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:33:20.288 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:33:20.288 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:33:20.288 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:33:20.288 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir").st_ino) 2024-05-20T18:33:20.288 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:33:20.395 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627776 2024-05-20T18:33:20.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:33:20.404 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:33:20.404 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:33:20.462 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:33:20.462 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:33:20.518 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:33:20.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.673+0000 7fd6106e9640 1 Processor -- start 2024-05-20T18:33:20.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.673+0000 7fd6106e9640 1 -- start start 2024-05-20T18:33:20.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.674+0000 7fd6106e9640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd6080b4010 0x7fd6080d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:20.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.674+0000 7fd6106e9640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6080d0370 con 0x7fd608163a10 2024-05-20T18:33:20.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.674+0000 7fd6106e9640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6080d0650 con 0x7fd6080d14b0 2024-05-20T18:33:20.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.674+0000 7fd6106e9640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd6080d3ca0 con 0x7fd6080b4010 2024-05-20T18:33:20.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.674+0000 7fd60e485640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd608163a10 0x7fd6080cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35488/0 (socket says 172.21.15.5:35488) 2024-05-20T18:33:20.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.674+0000 7fd60e485640 1 -- 172.21.15.5:0/245330771 learned_addr learned my addr 172.21.15.5:0/245330771 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:33:20.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.675+0000 7fd60d483640 1 -- 172.21.15.5:0/245330771 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2419279085 0 0) 0x7fd6080d0370 con 0x7fd608163a10 2024-05-20T18:33:20.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.675+0000 7fd60d483640 1 -- 172.21.15.5:0/245330771 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5f4003520 con 0x7fd608163a10 2024-05-20T18:33:20.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.675+0000 7fd60dc84640 1 --2- 172.21.15.5:0/245330771 >> v2:172.21.15.63:3300/0 conn(0x7fd6080b4010 0x7fd6080d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:20.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.675+0000 7fd60d483640 1 -- 172.21.15.5:0/245330771 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4242331303 0 0) 0x7fd5f4003520 con 0x7fd608163a10 2024-05-20T18:33:20.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.675+0000 7fd60d483640 1 -- 172.21.15.5:0/245330771 >> v2:172.21.15.63:3300/0 conn(0x7fd6080b4010 msgr2=0x7fd6080d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:20.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.675+0000 7fd60d483640 1 --2- 172.21.15.5:0/245330771 >> v2:172.21.15.63:3300/0 conn(0x7fd6080b4010 0x7fd6080d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:20.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.675+0000 7fd60d483640 1 -- 172.21.15.5:0/245330771 >> v1:172.21.15.63:6789/0 conn(0x7fd6080d14b0 legacy=0x7fd6080d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:20.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.675+0000 7fd60d483640 1 -- 172.21.15.5:0/245330771 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6080d3650 con 0x7fd608163a10 2024-05-20T18:33:20.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.675+0000 7fd60d483640 1 -- 172.21.15.5:0/245330771 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7fd604001670 con 0x7fd608163a10 2024-05-20T18:33:20.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.675+0000 7fd60d483640 1 -- 172.21.15.5:0/245330771 >> v1:172.21.15.5:6789/0 conn(0x7fd608163a10 legacy=0x7fd6080cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:20.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.675+0000 7fd60d483640 1 --2- 172.21.15.5:0/245330771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5f40039b0 0x7fd5f4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:20.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.675+0000 7fd60d483640 1 --2- 172.21.15.5:0/245330771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5f4004800 0x7fd5f4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:20.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.675+0000 7fd60d483640 1 --2- 172.21.15.5:0/245330771 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5f40051c0 0x7fd5f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:20.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.675+0000 7fd60d483640 1 -- 172.21.15.5:0/245330771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd6080d3ca0 con 0x7fd5f4004800 2024-05-20T18:33:20.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.675+0000 7fd60d483640 1 -- 172.21.15.5:0/245330771 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f400c160 con 0x7fd5f40039b0 2024-05-20T18:33:20.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.675+0000 7fd60d483640 1 -- 172.21.15.5:0/245330771 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd5f400c360 con 0x7fd5f40051c0 2024-05-20T18:33:20.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.675+0000 7fd60dc84640 1 --2- 172.21.15.5:0/245330771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5f4004800 0x7fd5f4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:20.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.675+0000 7fd60e485640 1 --2- 172.21.15.5:0/245330771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5f40039b0 0x7fd5f4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:20.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.675+0000 7fd60ec86640 1 --2- 172.21.15.5:0/245330771 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5f40051c0 0x7fd5f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:20.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.675+0000 7fd60dc84640 1 -- 172.21.15.5:0/245330771 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5f40051c0 msgr2=0x7fd5f400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:20.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.675+0000 7fd60dc84640 1 --2- 172.21.15.5:0/245330771 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5f40051c0 0x7fd5f400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:20.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.675+0000 7fd60dc84640 1 -- 172.21.15.5:0/245330771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5f40039b0 msgr2=0x7fd5f4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:20.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.675+0000 7fd60dc84640 1 --2- 172.21.15.5:0/245330771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5f40039b0 0x7fd5f4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:20.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.676+0000 7fd60dc84640 1 -- 172.21.15.5:0/245330771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f4003520 con 0x7fd5f4004800 2024-05-20T18:33:20.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.676+0000 7fd60ec86640 1 -- 172.21.15.5:0/245330771 reap_dead start 2024-05-20T18:33:20.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.676+0000 7fd60e485640 1 --2- 172.21.15.5:0/245330771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5f40039b0 0x7fd5f4003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:20.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.676+0000 7fd60dc84640 1 --2- 172.21.15.5:0/245330771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5f4004800 0x7fd5f4004c10 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009ac0 tx=0x7fd5f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69feb8d6fc39281e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:20.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.676+0000 7fd60d483640 1 -- 172.21.15.5:0/245330771 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8036070 con 0x7fd5f4004800 2024-05-20T18:33:20.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.676+0000 7fd60d483640 1 -- 172.21.15.5:0/245330771 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5f8031400 con 0x7fd5f4004800 2024-05-20T18:33:20.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.676+0000 7fd60d483640 1 -- 172.21.15.5:0/245330771 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8009690 con 0x7fd5f4004800 2024-05-20T18:33:20.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.677+0000 7fd6106e9640 1 -- 172.21.15.5:0/245330771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5f4004800 msgr2=0x7fd5f4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:20.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.677+0000 7fd6106e9640 1 --2- 172.21.15.5:0/245330771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5f4004800 0x7fd5f4004c10 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009ac0 tx=0x7fd5f4005830 comp rx=0 tx=0).stop 2024-05-20T18:33:20.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.677+0000 7fd6106e9640 1 -- 172.21.15.5:0/245330771 shutdown_connections 2024-05-20T18:33:20.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.677+0000 7fd6106e9640 1 --2- 172.21.15.5:0/245330771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5f4004800 0x7fd5f4004c10 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:20.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.677+0000 7fd6106e9640 1 -- 172.21.15.5:0/245330771 >> 172.21.15.5:0/245330771 conn(0x7fd60806d240 msgr2=0x7fd60806d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:20.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.677+0000 7fd6106e9640 1 -- 172.21.15.5:0/245330771 shutdown_connections 2024-05-20T18:33:20.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.677+0000 7fd6106e9640 1 -- 172.21.15.5:0/245330771 wait complete. 2024-05-20T18:33:20.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.679+0000 7fd6106e9640 1 Processor -- start 2024-05-20T18:33:20.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.679+0000 7fd6106e9640 1 -- start start 2024-05-20T18:33:20.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.681+0000 7fd6106e9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5f4004800 0x7fd608153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:20.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.681+0000 7fd6106e9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6081580d0 0x7fd608156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:20.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.681+0000 7fd6106e9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6081570e0 0x7fd6081574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:20.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.681+0000 7fd6106e9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd6080d42b0 con 0x7fd6081570e0 2024-05-20T18:33:20.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.681+0000 7fd6106e9640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd6080d4130 con 0x7fd5f4004800 2024-05-20T18:33:20.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.681+0000 7fd6106e9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd6080d4430 con 0x7fd6081580d0 2024-05-20T18:33:20.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.681+0000 7fd60ec86640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6081570e0 0x7fd6081574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:20.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.681+0000 7fd60ec86640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6081570e0 0x7fd6081574f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59084/0 (socket says 172.21.15.5:59084) 2024-05-20T18:33:20.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.681+0000 7fd60ec86640 1 -- 172.21.15.5:0/1476043688 learned_addr learned my addr 172.21.15.5:0/1476043688 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:20.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.681+0000 7fd60e485640 1 --2- 172.21.15.5:0/1476043688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5f4004800 0x7fd608153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:20.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.681+0000 7fd60dc84640 1 --2- 172.21.15.5:0/1476043688 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6081580d0 0x7fd608156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:20.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.681+0000 7fd60ec86640 1 -- 172.21.15.5:0/1476043688 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6081580d0 msgr2=0x7fd608156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:20.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.681+0000 7fd60ec86640 1 --2- 172.21.15.5:0/1476043688 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6081580d0 0x7fd608156ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:20.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.681+0000 7fd60ec86640 1 -- 172.21.15.5:0/1476043688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5f4004800 msgr2=0x7fd608153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:20.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.681+0000 7fd60ec86640 1 --2- 172.21.15.5:0/1476043688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5f4004800 0x7fd608153940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:20.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.681+0000 7fd60ec86640 1 -- 172.21.15.5:0/1476043688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd608153e80 con 0x7fd6081570e0 2024-05-20T18:33:20.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.681+0000 7fd60dc84640 1 --2- 172.21.15.5:0/1476043688 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6081580d0 0x7fd608156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:20.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.681+0000 7fd60ec86640 1 --2- 172.21.15.5:0/1476043688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6081570e0 0x7fd6081574f0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fd608068090 tx=0x7fd60000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:20.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.681+0000 7fd5feffd640 1 -- 172.21.15.5:0/1476043688 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600003970 con 0x7fd6081570e0 2024-05-20T18:33:20.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.681+0000 7fd5feffd640 1 -- 172.21.15.5:0/1476043688 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd600018070 con 0x7fd6081570e0 2024-05-20T18:33:20.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.681+0000 7fd6106e9640 1 -- 172.21.15.5:0/1476043688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd608155d70 con 0x7fd6081570e0 2024-05-20T18:33:20.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.681+0000 7fd5feffd640 1 -- 172.21.15.5:0/1476043688 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600003d70 con 0x7fd6081570e0 2024-05-20T18:33:20.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.682+0000 7fd6106e9640 1 -- 172.21.15.5:0/1476043688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd608156330 con 0x7fd6081570e0 2024-05-20T18:33:20.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.683+0000 7fd6106e9640 1 -- 172.21.15.5:0/1476043688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd608075b00 con 0x7fd6081570e0 2024-05-20T18:33:20.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.683+0000 7fd5feffd640 1 -- 172.21.15.5:0/1476043688 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd6000050d0 con 0x7fd6081570e0 2024-05-20T18:33:20.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.684+0000 7fd5feffd640 1 --2- 172.21.15.5:0/1476043688 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd5ec06be60 0x7fd5ec06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:20.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.684+0000 7fd5feffd640 1 -- 172.21.15.5:0/1476043688 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6101+0+0 (secure 0 0 0) 0x7fd60008abb0 con 0x7fd6081570e0 2024-05-20T18:33:20.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.684+0000 7fd60e485640 1 --2- 172.21.15.5:0/1476043688 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd5ec06be60 0x7fd5ec06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:20.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.684+0000 7fd60e485640 1 --2- 172.21.15.5:0/1476043688 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd5ec06be60 0x7fd5ec06e370 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd604001cd0 tx=0x7fd604007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:20.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.689+0000 7fd5feffd640 1 -- 172.21.15.5:0/1476043688 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd6000593d0 con 0x7fd6081570e0 2024-05-20T18:33:20.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.876+0000 7fd6106e9640 1 -- 172.21.15.5:0/1476043688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6080d3050 con 0x7fd6081570e0 2024-05-20T18:33:20.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.877+0000 7fd5feffd640 1 -- 172.21.15.5:0/1476043688 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+16765 (secure 0 0 0) 0x7fd60008e090 con 0x7fd6081570e0 2024-05-20T18:33:20.878 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:33:20.878 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":90,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:33:14.088906+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:33:10.184948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-20T18:33:11.453356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:33:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.879+0000 7fd6106e9640 1 -- 172.21.15.5:0/1476043688 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd5ec06be60 msgr2=0x7fd5ec06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.879+0000 7fd6106e9640 1 --2- 172.21.15.5:0/1476043688 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd5ec06be60 0x7fd5ec06e370 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd604001cd0 tx=0x7fd604007040 comp rx=0 tx=0).stop 2024-05-20T18:33:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.880+0000 7fd6106e9640 1 -- 172.21.15.5:0/1476043688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6081570e0 msgr2=0x7fd6081574f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.880+0000 7fd6106e9640 1 --2- 172.21.15.5:0/1476043688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6081570e0 0x7fd6081574f0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fd608068090 tx=0x7fd60000ec10 comp rx=0 tx=0).stop 2024-05-20T18:33:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.880+0000 7fd6106e9640 1 -- 172.21.15.5:0/1476043688 shutdown_connections 2024-05-20T18:33:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.880+0000 7fd6106e9640 1 --2- 172.21.15.5:0/1476043688 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd5ec06be60 0x7fd5ec06e370 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.880+0000 7fd6106e9640 1 --2- 172.21.15.5:0/1476043688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6081570e0 0x7fd6081574f0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.880+0000 7fd6106e9640 1 --2- 172.21.15.5:0/1476043688 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6081580d0 0x7fd608156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.880+0000 7fd6106e9640 1 --2- 172.21.15.5:0/1476043688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5f4004800 0x7fd608153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.880+0000 7fd6106e9640 1 -- 172.21.15.5:0/1476043688 >> 172.21.15.5:0/1476043688 conn(0x7fd60806d240 msgr2=0x7fd60815f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.880+0000 7fd6106e9640 1 -- 172.21.15.5:0/1476043688 shutdown_connections 2024-05-20T18:33:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:20.880+0000 7fd6106e9640 1 -- 172.21.15.5:0/1476043688 wait complete. 2024-05-20T18:33:20.892 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:33:10.184948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:33:11.453356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:33:20.893 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:33:21.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.009+0000 7fb2a88fe640 1 Processor -- start 2024-05-20T18:33:21.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.010+0000 7fb2a88fe640 1 -- start start 2024-05-20T18:33:21.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.011+0000 7fb2a88fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb2a406d3d0 0x7fb2a406d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:21.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.011+0000 7fb2a88fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb2a40b8190 0x7fb2a40b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:21.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.011+0000 7fb2a88fe640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2a406dd20 con 0x7fb2a4162820 2024-05-20T18:33:21.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.011+0000 7fb2a88fe640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fb2a40d0630 con 0x7fb2a40b8190 2024-05-20T18:33:21.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.011+0000 7fb2a88fe640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb2a40d39c0 con 0x7fb2a406d3d0 2024-05-20T18:33:21.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.011+0000 7fb2a2d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb2a40b8190 0x7fb2a40b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:21.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.011+0000 7fb2a259c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb2a4162820 0x7fb2a40b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35500/0 (socket says 172.21.15.5:35500) 2024-05-20T18:33:21.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.011+0000 7fb2a259c640 1 -- 172.21.15.5:0/87646356 learned_addr learned my addr 172.21.15.5:0/87646356 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:33:21.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.011+0000 7fb2a1d9b640 1 --2- 172.21.15.5:0/87646356 >> v2:172.21.15.63:3300/0 conn(0x7fb2a406d3d0 0x7fb2a406d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:21.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.012+0000 7fb2a2d9d640 1 -- 172.21.15.5:0/87646356 >> v2:172.21.15.63:3300/0 conn(0x7fb2a406d3d0 msgr2=0x7fb2a406d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:21.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.012+0000 7fb2a2d9d640 1 --2- 172.21.15.5:0/87646356 >> v2:172.21.15.63:3300/0 conn(0x7fb2a406d3d0 0x7fb2a406d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:21.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.012+0000 7fb2a2d9d640 1 -- 172.21.15.5:0/87646356 >> v1:172.21.15.5:6789/0 conn(0x7fb2a4162820 legacy=0x7fb2a40b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:21.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.012+0000 7fb2a2d9d640 1 -- 172.21.15.5:0/87646356 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a40d3ca0 con 0x7fb2a40b8190 2024-05-20T18:33:21.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.012+0000 7fb2a159a640 1 -- 172.21.15.5:0/87646356 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 75626668 0 0) 0x7fb294001110 con 0x7fb2a4162820 2024-05-20T18:33:21.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.012+0000 7fb2a1d9b640 1 --2- 172.21.15.5:0/87646356 >> v2:172.21.15.63:3300/0 conn(0x7fb2a406d3d0 0x7fb2a406d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:33:21.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.012+0000 7fb2a2d9d640 1 --2- 172.21.15.5:0/87646356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2a40b8190 0x7fb2a40b85a0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fb2a40d1230 tx=0x7fb29c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02ebaa0b705cbfc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:21.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.012+0000 7fb2a159a640 1 -- 172.21.15.5:0/87646356 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c007740 con 0x7fb2a40b8190 2024-05-20T18:33:21.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.012+0000 7fb2a159a640 1 -- 172.21.15.5:0/87646356 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb29c0078e0 con 0x7fb2a40b8190 2024-05-20T18:33:21.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.012+0000 7fb2a159a640 1 -- 172.21.15.5:0/87646356 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c007cb0 con 0x7fb2a40b8190 2024-05-20T18:33:21.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.013+0000 7fb2a88fe640 1 -- 172.21.15.5:0/87646356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2a40b8190 msgr2=0x7fb2a40b85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:21.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.013+0000 7fb2a88fe640 1 --2- 172.21.15.5:0/87646356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2a40b8190 0x7fb2a40b85a0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fb2a40d1230 tx=0x7fb29c02a680 comp rx=0 tx=0).stop 2024-05-20T18:33:21.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.013+0000 7fb2a88fe640 1 -- 172.21.15.5:0/87646356 shutdown_connections 2024-05-20T18:33:21.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.013+0000 7fb2a88fe640 1 --2- 172.21.15.5:0/87646356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2a40b8190 0x7fb2a40b85a0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:21.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.013+0000 7fb2a88fe640 1 --2- 172.21.15.5:0/87646356 >> v2:172.21.15.63:3300/0 conn(0x7fb2a406d3d0 0x7fb2a406d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:21.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.013+0000 7fb2a88fe640 1 -- 172.21.15.5:0/87646356 >> 172.21.15.5:0/87646356 conn(0x7fb2a40d4bd0 msgr2=0x7fb2a40d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:21.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.013+0000 7fb2a88fe640 1 -- 172.21.15.5:0/87646356 shutdown_connections 2024-05-20T18:33:21.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.013+0000 7fb2a88fe640 1 -- 172.21.15.5:0/87646356 wait complete. 2024-05-20T18:33:21.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.015+0000 7fb2a88fe640 1 Processor -- start 2024-05-20T18:33:21.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.015+0000 7fb2a88fe640 1 -- start start 2024-05-20T18:33:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.016+0000 7fb2a88fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2a406d3d0 0x7fb2a40ce110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.017+0000 7fb2a88fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2a40b8190 0x7fb2a414cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.017+0000 7fb2a88fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2a4162820 0x7fb2a40b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.017+0000 7fb2a88fe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a40d0300 con 0x7fb2a4162820 2024-05-20T18:33:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.017+0000 7fb2a88fe640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a40d4260 con 0x7fb2a406d3d0 2024-05-20T18:33:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.017+0000 7fb2a88fe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb2a40b4050 con 0x7fb2a40b8190 2024-05-20T18:33:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.017+0000 7fb2a2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2a4162820 0x7fb2a40b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.017+0000 7fb2a2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2a4162820 0x7fb2a40b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59104/0 (socket says 172.21.15.5:59104) 2024-05-20T18:33:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.017+0000 7fb2a2d9d640 1 -- 172.21.15.5:0/2666597872 learned_addr learned my addr 172.21.15.5:0/2666597872 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.017+0000 7fb2a259c640 1 --2- 172.21.15.5:0/2666597872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2a406d3d0 0x7fb2a40ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.017+0000 7fb2a1d9b640 1 --2- 172.21.15.5:0/2666597872 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2a40b8190 0x7fb2a414cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.017+0000 7fb2a2d9d640 1 -- 172.21.15.5:0/2666597872 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2a40b8190 msgr2=0x7fb2a414cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.017+0000 7fb2a2d9d640 1 --2- 172.21.15.5:0/2666597872 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2a40b8190 0x7fb2a414cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.017+0000 7fb2a2d9d640 1 -- 172.21.15.5:0/2666597872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2a406d3d0 msgr2=0x7fb2a40ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:21.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.017+0000 7fb2a2d9d640 1 --2- 172.21.15.5:0/2666597872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2a406d3d0 0x7fb2a40ce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:21.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.017+0000 7fb2a2d9d640 1 -- 172.21.15.5:0/2666597872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a414d8f0 con 0x7fb2a4162820 2024-05-20T18:33:21.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.017+0000 7fb2a1d9b640 1 --2- 172.21.15.5:0/2666597872 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2a40b8190 0x7fb2a414cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:21.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.017+0000 7fb2a2d9d640 1 --2- 172.21.15.5:0/2666597872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2a4162820 0x7fb2a40b6390 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fb29c009e80 tx=0x7fb29c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:21.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.017+0000 7fb292ffd640 1 -- 172.21.15.5:0/2666597872 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c03f070 con 0x7fb2a4162820 2024-05-20T18:33:21.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.017+0000 7fb292ffd640 1 -- 172.21.15.5:0/2666597872 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb29c031c40 con 0x7fb2a4162820 2024-05-20T18:33:21.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.017+0000 7fb2a88fe640 1 -- 172.21.15.5:0/2666597872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a41f8a40 con 0x7fb2a4162820 2024-05-20T18:33:21.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.017+0000 7fb292ffd640 1 -- 172.21.15.5:0/2666597872 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c003b90 con 0x7fb2a4162820 2024-05-20T18:33:21.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.018+0000 7fb2a88fe640 1 -- 172.21.15.5:0/2666597872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a414b6e0 con 0x7fb2a4162820 2024-05-20T18:33:21.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.018+0000 7fb2a88fe640 1 -- 172.21.15.5:0/2666597872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2a414a850 con 0x7fb2a4162820 2024-05-20T18:33:21.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.019+0000 7fb292ffd640 1 -- 172.21.15.5:0/2666597872 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb29c004030 con 0x7fb2a4162820 2024-05-20T18:33:21.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.020+0000 7fb292ffd640 1 --2- 172.21.15.5:0/2666597872 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb28406be30 0x7fb28406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:21.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.020+0000 7fb2a259c640 1 --2- 172.21.15.5:0/2666597872 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb28406be30 0x7fb28406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:21.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.020+0000 7fb292ffd640 1 -- 172.21.15.5:0/2666597872 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6101+0+0 (secure 0 0 0) 0x7fb29c0ac270 con 0x7fb2a4162820 2024-05-20T18:33:21.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.020+0000 7fb2a259c640 1 --2- 172.21.15.5:0/2666597872 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb28406be30 0x7fb28406e340 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb294007fd0 tx=0x7fb294007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:21.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.025+0000 7fb292ffd640 1 -- 172.21.15.5:0/2666597872 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb2a414a850 con 0x7fb2a4162820 2024-05-20T18:33:21.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.212+0000 7fb2a88fe640 1 -- 172.21.15.5:0/2666597872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb2a414b940 con 0x7fb2a4162820 2024-05-20T18:33:21.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.213+0000 7fb292ffd640 1 -- 172.21.15.5:0/2666597872 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7fb29c07a3b0 con 0x7fb2a4162820 2024-05-20T18:33:21.214 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:33:21.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.215+0000 7fb2a88fe640 1 -- 172.21.15.5:0/2666597872 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb28406be30 msgr2=0x7fb28406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:21.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.215+0000 7fb2a88fe640 1 --2- 172.21.15.5:0/2666597872 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb28406be30 0x7fb28406e340 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb294007fd0 tx=0x7fb294007de0 comp rx=0 tx=0).stop 2024-05-20T18:33:21.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.215+0000 7fb2a88fe640 1 -- 172.21.15.5:0/2666597872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2a4162820 msgr2=0x7fb2a40b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:21.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.215+0000 7fb2a88fe640 1 --2- 172.21.15.5:0/2666597872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2a4162820 0x7fb2a40b6390 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fb29c009e80 tx=0x7fb29c004a40 comp rx=0 tx=0).stop 2024-05-20T18:33:21.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.215+0000 7fb2a88fe640 1 -- 172.21.15.5:0/2666597872 shutdown_connections 2024-05-20T18:33:21.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.215+0000 7fb2a88fe640 1 --2- 172.21.15.5:0/2666597872 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb28406be30 0x7fb28406e340 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:21.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.215+0000 7fb2a88fe640 1 --2- 172.21.15.5:0/2666597872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2a4162820 0x7fb2a40b6390 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:21.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.215+0000 7fb2a88fe640 1 --2- 172.21.15.5:0/2666597872 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2a40b8190 0x7fb2a414cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:21.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.215+0000 7fb2a88fe640 1 --2- 172.21.15.5:0/2666597872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2a406d3d0 0x7fb2a40ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:21.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.215+0000 7fb2a88fe640 1 -- 172.21.15.5:0/2666597872 >> 172.21.15.5:0/2666597872 conn(0x7fb2a40d4bd0 msgr2=0x7fb2a40ba5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:21.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.215+0000 7fb2a88fe640 1 -- 172.21.15.5:0/2666597872 shutdown_connections 2024-05-20T18:33:21.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.215+0000 7fb2a88fe640 1 -- 172.21.15.5:0/2666597872 wait complete. 2024-05-20T18:33:21.227 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:33:21.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.344+0000 7f83bf8e7640 1 Processor -- start 2024-05-20T18:33:21.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.345+0000 7f83bf8e7640 1 -- start start 2024-05-20T18:33:21.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.346+0000 7f83bf8e7640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f83b806d190 0x7f83b8169cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:21.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.346+0000 7f83bf8e7640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83b8160430 con 0x7f83b815f700 2024-05-20T18:33:21.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.346+0000 7f83bf8e7640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83b806b6a0 con 0x7f83b80af030 2024-05-20T18:33:21.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.346+0000 7f83bf8e7640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f83b806bcf0 con 0x7f83b806d190 2024-05-20T18:33:21.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.346+0000 7f83bde84640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f83b806d190 0x7f83b8169cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:21.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.346+0000 7f83bce82640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f83b815f700 0x7f83b815fb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35510/0 (socket says 172.21.15.5:35510) 2024-05-20T18:33:21.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.346+0000 7f83bce82640 1 -- 172.21.15.5:0/580511321 learned_addr learned my addr 172.21.15.5:0/580511321 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:33:21.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.346+0000 7f83bde84640 1 -- 172.21.15.5:0/580511321 >> v1:172.21.15.63:6789/0 conn(0x7f83b80af030 legacy=0x7f83b80af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:21.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.346+0000 7f83bde84640 1 -- 172.21.15.5:0/580511321 >> v1:172.21.15.5:6789/0 conn(0x7f83b815f700 legacy=0x7f83b815fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:21.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.346+0000 7f83bde84640 1 -- 172.21.15.5:0/580511321 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b806b980 con 0x7f83b806d190 2024-05-20T18:33:21.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.347+0000 7f83bde84640 1 --2- 172.21.15.5:0/580511321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83b806d190 0x7f83b8169cc0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f83b00097a0 tx=0x7f83b002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f3c5425668c75d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:21.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.347+0000 7f83b7fff640 1 -- 172.21.15.5:0/580511321 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b0037070 con 0x7f83b806d190 2024-05-20T18:33:21.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.347+0000 7f83b7fff640 1 -- 172.21.15.5:0/580511321 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83b0032440 con 0x7f83b806d190 2024-05-20T18:33:21.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.347+0000 7f83b7fff640 1 -- 172.21.15.5:0/580511321 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b0032810 con 0x7f83b806d190 2024-05-20T18:33:21.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.348+0000 7f83bf8e7640 1 -- 172.21.15.5:0/580511321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83b806d190 msgr2=0x7f83b8169cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:21.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.348+0000 7f83bf8e7640 1 --2- 172.21.15.5:0/580511321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83b806d190 0x7f83b8169cc0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f83b00097a0 tx=0x7f83b002a1e0 comp rx=0 tx=0).stop 2024-05-20T18:33:21.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.348+0000 7f83bf8e7640 1 -- 172.21.15.5:0/580511321 shutdown_connections 2024-05-20T18:33:21.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.348+0000 7f83bf8e7640 1 --2- 172.21.15.5:0/580511321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83b806d190 0x7f83b8169cc0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:21.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.348+0000 7f83bf8e7640 1 -- 172.21.15.5:0/580511321 >> 172.21.15.5:0/580511321 conn(0x7f83b80b6620 msgr2=0x7f83b80b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:21.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.348+0000 7f83bf8e7640 1 -- 172.21.15.5:0/580511321 shutdown_connections 2024-05-20T18:33:21.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.348+0000 7f83bf8e7640 1 -- 172.21.15.5:0/580511321 wait complete. 2024-05-20T18:33:21.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.350+0000 7f83bf8e7640 1 Processor -- start 2024-05-20T18:33:21.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.350+0000 7f83bf8e7640 1 -- start start 2024-05-20T18:33:21.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.351+0000 7f83bf8e7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83b806d190 0x7f83b815bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:21.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.351+0000 7f83bf8e7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83b80af030 0x7f83b8161780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:21.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.351+0000 7f83bf8e7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83b815f700 0x7f83b8161cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:21.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.351+0000 7f83bf8e7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f83b816a380 con 0x7f83b806d190 2024-05-20T18:33:21.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.351+0000 7f83bf8e7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f83b816a200 con 0x7f83b80af030 2024-05-20T18:33:21.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.351+0000 7f83bf8e7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f83b816a500 con 0x7f83b815f700 2024-05-20T18:33:21.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.351+0000 7f83bd683640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83b806d190 0x7f83b815bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:21.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.351+0000 7f83bd683640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83b806d190 0x7f83b815bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59134/0 (socket says 172.21.15.5:59134) 2024-05-20T18:33:21.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.351+0000 7f83bd683640 1 -- 172.21.15.5:0/2546950265 learned_addr learned my addr 172.21.15.5:0/2546950265 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:21.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.352+0000 7f83bce82640 1 --2- 172.21.15.5:0/2546950265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83b80af030 0x7f83b8161780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:21.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.352+0000 7f83bde84640 1 --2- 172.21.15.5:0/2546950265 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83b815f700 0x7f83b8161cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:21.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.352+0000 7f83bd683640 1 -- 172.21.15.5:0/2546950265 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83b815f700 msgr2=0x7f83b8161cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:21.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.352+0000 7f83bd683640 1 --2- 172.21.15.5:0/2546950265 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83b815f700 0x7f83b8161cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:21.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.352+0000 7f83bd683640 1 -- 172.21.15.5:0/2546950265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83b80af030 msgr2=0x7f83b8161780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:21.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.352+0000 7f83bd683640 1 --2- 172.21.15.5:0/2546950265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83b80af030 0x7f83b8161780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:21.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.352+0000 7f83bd683640 1 -- 172.21.15.5:0/2546950265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b8150050 con 0x7f83b806d190 2024-05-20T18:33:21.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.352+0000 7f83bd683640 1 --2- 172.21.15.5:0/2546950265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83b806d190 0x7f83b815bea0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f83a800b9f0 tx=0x7f83a800beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:21.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.352+0000 7f83b5ffb640 1 -- 172.21.15.5:0/2546950265 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a800cbd0 con 0x7f83b806d190 2024-05-20T18:33:21.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.352+0000 7f83b5ffb640 1 -- 172.21.15.5:0/2546950265 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83a800cd70 con 0x7f83b806d190 2024-05-20T18:33:21.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.352+0000 7f83bf8e7640 1 -- 172.21.15.5:0/2546950265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83b8150330 con 0x7f83b806d190 2024-05-20T18:33:21.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.352+0000 7f83b5ffb640 1 -- 172.21.15.5:0/2546950265 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a8003bb0 con 0x7f83b806d190 2024-05-20T18:33:21.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.352+0000 7f83bf8e7640 1 -- 172.21.15.5:0/2546950265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83b81508f0 con 0x7f83b806d190 2024-05-20T18:33:21.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.353+0000 7f83bf8e7640 1 -- 172.21.15.5:0/2546950265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83b8075b00 con 0x7f83b806d190 2024-05-20T18:33:21.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.354+0000 7f83b5ffb640 1 -- 172.21.15.5:0/2546950265 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f83a8014440 con 0x7f83b806d190 2024-05-20T18:33:21.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.354+0000 7f83b5ffb640 1 --2- 172.21.15.5:0/2546950265 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f83a406be30 0x7f83a406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:21.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.355+0000 7f83b5ffb640 1 -- 172.21.15.5:0/2546950265 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6101+0+0 (secure 0 0 0) 0x7f83a808af00 con 0x7f83b806d190 2024-05-20T18:33:21.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.355+0000 7f83bce82640 1 --2- 172.21.15.5:0/2546950265 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f83a406be30 0x7f83a406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:21.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.355+0000 7f83bce82640 1 --2- 172.21.15.5:0/2546950265 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f83a406be30 0x7f83a406e340 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f83b815c840 tx=0x7f83ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:21.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.359+0000 7f83b5ffb640 1 -- 172.21.15.5:0/2546950265 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f83a8059690 con 0x7f83b806d190 2024-05-20T18:33:21.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.542+0000 7f83bf8e7640 1 -- 172.21.15.5:0/2546950265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f83b80d1630 con 0x7f83b806d190 2024-05-20T18:33:21.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.543+0000 7f83b5ffb640 1 -- 172.21.15.5:0/2546950265 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f83a8058f90 con 0x7f83b806d190 2024-05-20T18:33:21.543 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:33:21.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.545+0000 7f83bf8e7640 1 -- 172.21.15.5:0/2546950265 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f83a406be30 msgr2=0x7f83a406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.545+0000 7f83bf8e7640 1 --2- 172.21.15.5:0/2546950265 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f83a406be30 0x7f83a406e340 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f83b815c840 tx=0x7f83ac009040 comp rx=0 tx=0).stop 2024-05-20T18:33:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.545+0000 7f83bf8e7640 1 -- 172.21.15.5:0/2546950265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83b806d190 msgr2=0x7f83b815bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.545+0000 7f83bf8e7640 1 --2- 172.21.15.5:0/2546950265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83b806d190 0x7f83b815bea0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f83a800b9f0 tx=0x7f83a800beb0 comp rx=0 tx=0).stop 2024-05-20T18:33:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.545+0000 7f83bf8e7640 1 -- 172.21.15.5:0/2546950265 shutdown_connections 2024-05-20T18:33:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.545+0000 7f83bf8e7640 1 --2- 172.21.15.5:0/2546950265 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f83a406be30 0x7f83a406e340 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.545+0000 7f83bf8e7640 1 --2- 172.21.15.5:0/2546950265 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83b815f700 0x7f83b8161cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.545+0000 7f83bf8e7640 1 --2- 172.21.15.5:0/2546950265 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83b80af030 0x7f83b8161780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.545+0000 7f83bf8e7640 1 --2- 172.21.15.5:0/2546950265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83b806d190 0x7f83b815bea0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.545+0000 7f83bf8e7640 1 -- 172.21.15.5:0/2546950265 >> 172.21.15.5:0/2546950265 conn(0x7f83b80b6620 msgr2=0x7f83b80af440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.545+0000 7f83bf8e7640 1 -- 172.21.15.5:0/2546950265 shutdown_connections 2024-05-20T18:33:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.545+0000 7f83bf8e7640 1 -- 172.21.15.5:0/2546950265 wait complete. 2024-05-20T18:33:21.557 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:33:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.673+0000 7f392f3cc640 1 Processor -- start 2024-05-20T18:33:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.674+0000 7f392f3cc640 1 -- start start 2024-05-20T18:33:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.675+0000 7f392f3cc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f39280d10e0 0x7f39280cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.675+0000 7f392f3cc640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39280ce2f0 con 0x7f39280b7210 2024-05-20T18:33:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.675+0000 7f392f3cc640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39280d1ac0 con 0x7f39280d14f0 2024-05-20T18:33:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.675+0000 7f392f3cc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f39280d1da0 con 0x7f39280d10e0 2024-05-20T18:33:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.675+0000 7f392d969640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f39280b7210 0x7f39280cdbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35514/0 (socket says 172.21.15.5:35514) 2024-05-20T18:33:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.675+0000 7f392d969640 1 -- 172.21.15.5:0/3873759083 learned_addr learned my addr 172.21.15.5:0/3873759083 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:33:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.675+0000 7f392d168640 1 --2- 172.21.15.5:0/3873759083 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f39280d10e0 0x7f39280cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.675+0000 7f391ffff640 1 -- 172.21.15.5:0/3873759083 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1930065571 0 0) 0x7f39280ce2f0 con 0x7f39280b7210 2024-05-20T18:33:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.676+0000 7f391ffff640 1 -- 172.21.15.5:0/3873759083 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3914003520 con 0x7f39280b7210 2024-05-20T18:33:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.676+0000 7f391ffff640 1 -- 172.21.15.5:0/3873759083 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1658528053 0 0) 0x7f39280d1ac0 con 0x7f39280d14f0 2024-05-20T18:33:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.676+0000 7f391ffff640 1 -- 172.21.15.5:0/3873759083 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39280ce2f0 con 0x7f39280d14f0 2024-05-20T18:33:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.676+0000 7f391ffff640 1 -- 172.21.15.5:0/3873759083 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 27808158 0 0) 0x7f3914003520 con 0x7f39280b7210 2024-05-20T18:33:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.676+0000 7f391ffff640 1 -- 172.21.15.5:0/3873759083 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f39280d10e0 msgr2=0x7f39280cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.676+0000 7f391ffff640 1 --2- 172.21.15.5:0/3873759083 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f39280d10e0 0x7f39280cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.676+0000 7f391ffff640 1 -- 172.21.15.5:0/3873759083 >> v1:172.21.15.63:6789/0 conn(0x7f39280d14f0 legacy=0x7f39280cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.676+0000 7f391ffff640 1 -- 172.21.15.5:0/3873759083 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39280d2080 con 0x7f39280b7210 2024-05-20T18:33:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.676+0000 7f391ffff640 1 -- 172.21.15.5:0/3873759083 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f39200019f0 con 0x7f39280b7210 2024-05-20T18:33:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.676+0000 7f391ffff640 1 -- 172.21.15.5:0/3873759083 >> v1:172.21.15.5:6789/0 conn(0x7f39280b7210 legacy=0x7f39280cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.676+0000 7f391ffff640 1 --2- 172.21.15.5:0/3873759083 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f39280d10e0 0x7f3914003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.676+0000 7f391ffff640 1 --2- 172.21.15.5:0/3873759083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39140040d0 0x7f3914004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.676+0000 7f391ffff640 1 --2- 172.21.15.5:0/3873759083 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3914004d40 0x7f3914005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.676+0000 7f391ffff640 1 -- 172.21.15.5:0/3873759083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39280d1da0 con 0x7f39140040d0 2024-05-20T18:33:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.676+0000 7f391ffff640 1 -- 172.21.15.5:0/3873759083 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f391400bb20 con 0x7f3914004d40 2024-05-20T18:33:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.676+0000 7f391ffff640 1 -- 172.21.15.5:0/3873759083 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f391400bd20 con 0x7f39280d10e0 2024-05-20T18:33:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.676+0000 7f392c967640 1 --2- 172.21.15.5:0/3873759083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39140040d0 0x7f3914004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.676+0000 7f392d168640 1 --2- 172.21.15.5:0/3873759083 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f39280d10e0 0x7f3914003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.676+0000 7f392d969640 1 --2- 172.21.15.5:0/3873759083 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3914004d40 0x7f3914005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.676+0000 7f392c967640 1 -- 172.21.15.5:0/3873759083 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f39280d10e0 msgr2=0x7f3914003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.676+0000 7f392c967640 1 --2- 172.21.15.5:0/3873759083 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f39280d10e0 0x7f3914003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.676+0000 7f392c967640 1 -- 172.21.15.5:0/3873759083 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3914004d40 msgr2=0x7f3914005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.676+0000 7f392c967640 1 --2- 172.21.15.5:0/3873759083 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3914004d40 0x7f3914005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.676+0000 7f392c967640 1 -- 172.21.15.5:0/3873759083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3914003520 con 0x7f39140040d0 2024-05-20T18:33:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.677+0000 7f392c967640 1 --2- 172.21.15.5:0/3873759083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39140040d0 0x7f3914004800 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f392808bee0 tx=0x7f3914005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70eb4b930be9735 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.677+0000 7f391ffff640 1 -- 172.21.15.5:0/3873759083 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39180014f0 con 0x7f39140040d0 2024-05-20T18:33:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.677+0000 7f391ffff640 1 -- 172.21.15.5:0/3873759083 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39180052d0 con 0x7f39140040d0 2024-05-20T18:33:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.677+0000 7f391ffff640 1 -- 172.21.15.5:0/3873759083 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391802abe0 con 0x7f39140040d0 2024-05-20T18:33:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.678+0000 7f392f3cc640 1 -- 172.21.15.5:0/3873759083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39140040d0 msgr2=0x7f3914004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.678+0000 7f392f3cc640 1 --2- 172.21.15.5:0/3873759083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39140040d0 0x7f3914004800 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f392808bee0 tx=0x7f3914005830 comp rx=0 tx=0).stop 2024-05-20T18:33:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.678+0000 7f392d969640 1 -- 172.21.15.5:0/3873759083 reap_dead start 2024-05-20T18:33:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.678+0000 7f392f3cc640 1 -- 172.21.15.5:0/3873759083 shutdown_connections 2024-05-20T18:33:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.678+0000 7f392f3cc640 1 -- 172.21.15.5:0/3873759083 >> 172.21.15.5:0/3873759083 conn(0x7f39280b44d0 msgr2=0x7f39280b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.678+0000 7f392f3cc640 1 -- 172.21.15.5:0/3873759083 shutdown_connections 2024-05-20T18:33:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.678+0000 7f392f3cc640 1 -- 172.21.15.5:0/3873759083 wait complete. 2024-05-20T18:33:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.680+0000 7f392f3cc640 1 Processor -- start 2024-05-20T18:33:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.680+0000 7f392f3cc640 1 -- start start 2024-05-20T18:33:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.681+0000 7f392f3cc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f392814b330 0x7f3928157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.681+0000 7f392f3cc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392814df40 0x7f39280cb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.681+0000 7f392f3cc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f392814e350 0x7f392814a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.681+0000 7f392f3cc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39280b82c0 con 0x7f392814df40 2024-05-20T18:33:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.681+0000 7f392f3cc640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f39280d2640 con 0x7f392814e350 2024-05-20T18:33:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.681+0000 7f392f3cc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f39280b8440 con 0x7f392814b330 2024-05-20T18:33:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.681+0000 7f392c967640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392814df40 0x7f39280cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.681+0000 7f392c967640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392814df40 0x7f39280cb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59144/0 (socket says 172.21.15.5:59144) 2024-05-20T18:33:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.681+0000 7f392c967640 1 -- 172.21.15.5:0/126055951 learned_addr learned my addr 172.21.15.5:0/126055951 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.681+0000 7f392d168640 1 --2- 172.21.15.5:0/126055951 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f392814b330 0x7f3928157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.682+0000 7f392d969640 1 --2- 172.21.15.5:0/126055951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f392814e350 0x7f392814a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.682+0000 7f392c967640 1 -- 172.21.15.5:0/126055951 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f392814b330 msgr2=0x7f3928157a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.682+0000 7f392c967640 1 --2- 172.21.15.5:0/126055951 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f392814b330 0x7f3928157a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.682+0000 7f392c967640 1 -- 172.21.15.5:0/126055951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f392814e350 msgr2=0x7f392814a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.682+0000 7f392c967640 1 --2- 172.21.15.5:0/126055951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f392814e350 0x7f392814a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.682+0000 7f392c967640 1 -- 172.21.15.5:0/126055951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39280c8fb0 con 0x7f392814df40 2024-05-20T18:33:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.682+0000 7f392c967640 1 --2- 172.21.15.5:0/126055951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392814df40 0x7f39280cb920 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f3928160bd0 tx=0x7f39180012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.682+0000 7f391dffb640 1 -- 172.21.15.5:0/126055951 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391803f020 con 0x7f392814df40 2024-05-20T18:33:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.682+0000 7f391dffb640 1 -- 172.21.15.5:0/126055951 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3918006e50 con 0x7f392814df40 2024-05-20T18:33:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.682+0000 7f392f3cc640 1 -- 172.21.15.5:0/126055951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39280cb170 con 0x7f392814df40 2024-05-20T18:33:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.682+0000 7f391dffb640 1 -- 172.21.15.5:0/126055951 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3918037040 con 0x7f392814df40 2024-05-20T18:33:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.682+0000 7f392f3cc640 1 -- 172.21.15.5:0/126055951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f392815a7f0 con 0x7f392814df40 2024-05-20T18:33:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.683+0000 7f392f3cc640 1 -- 172.21.15.5:0/126055951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3928075b00 con 0x7f392814df40 2024-05-20T18:33:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.684+0000 7f391dffb640 1 -- 172.21.15.5:0/126055951 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3918042020 con 0x7f392814df40 2024-05-20T18:33:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.685+0000 7f391dffb640 1 --2- 172.21.15.5:0/126055951 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f390c06be30 0x7f390c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.685+0000 7f391dffb640 1 -- 172.21.15.5:0/126055951 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6101+0+0 (secure 0 0 0) 0x7f39180aacc0 con 0x7f392814df40 2024-05-20T18:33:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.685+0000 7f392d168640 1 --2- 172.21.15.5:0/126055951 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f390c06be30 0x7f390c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.685+0000 7f392d168640 1 --2- 172.21.15.5:0/126055951 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f390c06be30 0x7f390c06e340 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3924009160 tx=0x7f3924008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.689+0000 7f391dffb640 1 -- 172.21.15.5:0/126055951 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f39180b1080 con 0x7f392814df40 2024-05-20T18:33:21.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.872+0000 7f392f3cc640 1 -- 172.21.15.5:0/126055951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f39280c9320 con 0x7f392814df40 2024-05-20T18:33:21.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.873+0000 7f391dffb640 1 -- 172.21.15.5:0/126055951 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f3918049030 con 0x7f392814df40 2024-05-20T18:33:21.874 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:33:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.875+0000 7f392f3cc640 1 -- 172.21.15.5:0/126055951 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f390c06be30 msgr2=0x7f390c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.875+0000 7f392f3cc640 1 --2- 172.21.15.5:0/126055951 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f390c06be30 0x7f390c06e340 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3924009160 tx=0x7f3924008040 comp rx=0 tx=0).stop 2024-05-20T18:33:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.875+0000 7f392f3cc640 1 -- 172.21.15.5:0/126055951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392814df40 msgr2=0x7f39280cb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.875+0000 7f392f3cc640 1 --2- 172.21.15.5:0/126055951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392814df40 0x7f39280cb920 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f3928160bd0 tx=0x7f39180012d0 comp rx=0 tx=0).stop 2024-05-20T18:33:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.875+0000 7f392f3cc640 1 -- 172.21.15.5:0/126055951 shutdown_connections 2024-05-20T18:33:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.875+0000 7f392f3cc640 1 --2- 172.21.15.5:0/126055951 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f390c06be30 0x7f390c06e340 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.875+0000 7f392f3cc640 1 --2- 172.21.15.5:0/126055951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f392814e350 0x7f392814a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.875+0000 7f392f3cc640 1 --2- 172.21.15.5:0/126055951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392814df40 0x7f39280cb920 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:21.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.875+0000 7f392f3cc640 1 --2- 172.21.15.5:0/126055951 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f392814b330 0x7f3928157a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:21.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.875+0000 7f392f3cc640 1 -- 172.21.15.5:0/126055951 >> 172.21.15.5:0/126055951 conn(0x7f39280b44d0 msgr2=0x7f39280b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:21.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.875+0000 7f392f3cc640 1 -- 172.21.15.5:0/126055951 shutdown_connections 2024-05-20T18:33:21.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:21.875+0000 7f392f3cc640 1 -- 172.21.15.5:0/126055951 wait complete. 2024-05-20T18:33:21.888 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T18:33:22.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.004+0000 7f0efa63d640 1 Processor -- start 2024-05-20T18:33:22.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.004+0000 7f0efa63d640 1 -- start start 2024-05-20T18:33:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.006+0000 7f0efa63d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0ef40cf250 0x7f0ef415f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.006+0000 7f0efa63d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0ef40b6680 0x7f0ef40b6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.006+0000 7f0efa63d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ef4163020 0x7f0ef4163430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.006+0000 7f0efa63d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f0ef415f9a0 con 0x7f0ef40cf250 2024-05-20T18:33:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.006+0000 7f0efa63d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0ef40d0b10 con 0x7f0ef40b6680 2024-05-20T18:33:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.006+0000 7f0efa63d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0ef40d27d0 con 0x7f0ef4163020 2024-05-20T18:33:22.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.006+0000 7f0ef3fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0ef40cf250 0x7f0ef415f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:22.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.006+0000 7f0ef3fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0ef40cf250 0x7f0ef415f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59160/0 (socket says 172.21.15.5:59160) 2024-05-20T18:33:22.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.006+0000 7f0ef3fff640 1 -- 172.21.15.5:0/552162539 learned_addr learned my addr 172.21.15.5:0/552162539 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:22.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.006+0000 7f0ef8bda640 1 --2- 172.21.15.5:0/552162539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ef4163020 0x7f0ef4163430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:22.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.006+0000 7f0eebfff640 1 --2- 172.21.15.5:0/552162539 >> v2:172.21.15.63:3300/0 conn(0x7f0ef40b6680 0x7f0ef40b6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:22.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.006+0000 7f0ef3fff640 1 -- 172.21.15.5:0/552162539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ef4163020 msgr2=0x7f0ef4163430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:22.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.006+0000 7f0ef3fff640 1 --2- 172.21.15.5:0/552162539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ef4163020 0x7f0ef4163430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:22.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.006+0000 7f0ef3fff640 1 -- 172.21.15.5:0/552162539 >> v2:172.21.15.63:3300/0 conn(0x7f0ef40b6680 msgr2=0x7f0ef40b6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:22.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.006+0000 7f0ef3fff640 1 --2- 172.21.15.5:0/552162539 >> v2:172.21.15.63:3300/0 conn(0x7f0ef40b6680 0x7f0ef40b6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:22.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.006+0000 7f0ef3fff640 1 -- 172.21.15.5:0/552162539 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef40b6fd0 con 0x7f0ef40cf250 2024-05-20T18:33:22.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.006+0000 7f0ef8bda640 1 --2- 172.21.15.5:0/552162539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ef4163020 0x7f0ef4163430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:22.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.007+0000 7f0ef3fff640 1 --2- 172.21.15.5:0/552162539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef40cf250 0x7f0ef415f460 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f0ef40d4bb0 tx=0x7f0ee402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651778d83bb463 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.007+0000 7f0ef37fe640 1 -- 172.21.15.5:0/552162539 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4037070 con 0x7f0ef40cf250 2024-05-20T18:33:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.007+0000 7f0ef37fe640 1 -- 172.21.15.5:0/552162539 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ee4032440 con 0x7f0ef40cf250 2024-05-20T18:33:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.007+0000 7f0ef37fe640 1 -- 172.21.15.5:0/552162539 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4032810 con 0x7f0ef40cf250 2024-05-20T18:33:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.008+0000 7f0efa63d640 1 -- 172.21.15.5:0/552162539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef40cf250 msgr2=0x7f0ef415f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.008+0000 7f0efa63d640 1 --2- 172.21.15.5:0/552162539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef40cf250 0x7f0ef415f460 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f0ef40d4bb0 tx=0x7f0ee402a1e0 comp rx=0 tx=0).stop 2024-05-20T18:33:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.008+0000 7f0efa63d640 1 -- 172.21.15.5:0/552162539 shutdown_connections 2024-05-20T18:33:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.008+0000 7f0efa63d640 1 --2- 172.21.15.5:0/552162539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ef4163020 0x7f0ef4163430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.008+0000 7f0efa63d640 1 --2- 172.21.15.5:0/552162539 >> v2:172.21.15.63:3300/0 conn(0x7f0ef40b6680 0x7f0ef40b6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.008+0000 7f0efa63d640 1 --2- 172.21.15.5:0/552162539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef40cf250 0x7f0ef415f460 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.008+0000 7f0efa63d640 1 -- 172.21.15.5:0/552162539 >> 172.21.15.5:0/552162539 conn(0x7f0ef40d1860 msgr2=0x7f0ef40d1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.008+0000 7f0efa63d640 1 -- 172.21.15.5:0/552162539 shutdown_connections 2024-05-20T18:33:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.008+0000 7f0efa63d640 1 -- 172.21.15.5:0/552162539 wait complete. 2024-05-20T18:33:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.010+0000 7f0efa63d640 1 Processor -- start 2024-05-20T18:33:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.010+0000 7f0efa63d640 1 -- start start 2024-05-20T18:33:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.011+0000 7f0efa63d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ef40b6680 0x7f0ef4158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.011+0000 7f0efa63d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ef40cf250 0x7f0ef41e97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.011+0000 7f0efa63d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef4163020 0x7f0ef41f57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.011+0000 7f0efa63d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef40d2a20 con 0x7f0ef4163020 2024-05-20T18:33:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.011+0000 7f0efa63d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef40d0830 con 0x7f0ef40cf250 2024-05-20T18:33:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.011+0000 7f0efa63d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0ef40d2d00 con 0x7f0ef40b6680 2024-05-20T18:33:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.011+0000 7f0ef8bda640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef4163020 0x7f0ef41f57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.011+0000 7f0ef8bda640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef4163020 0x7f0ef41f57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59162/0 (socket says 172.21.15.5:59162) 2024-05-20T18:33:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.011+0000 7f0ef8bda640 1 -- 172.21.15.5:0/1923899142 learned_addr learned my addr 172.21.15.5:0/1923899142 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.012+0000 7f0ef8bda640 1 -- 172.21.15.5:0/1923899142 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ef40b6680 msgr2=0x7f0ef4158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.012+0000 7f0ef8bda640 1 --2- 172.21.15.5:0/1923899142 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ef40b6680 0x7f0ef4158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:22.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.012+0000 7f0ef8bda640 1 -- 172.21.15.5:0/1923899142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ef40cf250 msgr2=0x7f0ef41e97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:22.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.012+0000 7f0ef8bda640 1 --2- 172.21.15.5:0/1923899142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ef40cf250 0x7f0ef41e97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:22.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.012+0000 7f0ef8bda640 1 -- 172.21.15.5:0/1923899142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef4146c70 con 0x7f0ef4163020 2024-05-20T18:33:22.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.012+0000 7f0ef8bda640 1 --2- 172.21.15.5:0/1923899142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef4163020 0x7f0ef41f57d0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f0eec007ea0 tx=0x7f0eec007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:22.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.012+0000 7f0ef17fa640 1 -- 172.21.15.5:0/1923899142 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec01a070 con 0x7f0ef4163020 2024-05-20T18:33:22.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.012+0000 7f0ef17fa640 1 -- 172.21.15.5:0/1923899142 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eec003bb0 con 0x7f0ef4163020 2024-05-20T18:33:22.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.012+0000 7f0ef17fa640 1 -- 172.21.15.5:0/1923899142 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec00de70 con 0x7f0ef4163020 2024-05-20T18:33:22.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.012+0000 7f0efa63d640 1 -- 172.21.15.5:0/1923899142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef41ea480 con 0x7f0ef4163020 2024-05-20T18:33:22.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.012+0000 7f0efa63d640 1 -- 172.21.15.5:0/1923899142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef4148760 con 0x7f0ef4163020 2024-05-20T18:33:22.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.013+0000 7f0efa63d640 1 -- 172.21.15.5:0/1923899142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ef40b6a90 con 0x7f0ef4163020 2024-05-20T18:33:22.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.014+0000 7f0ef17fa640 1 -- 172.21.15.5:0/1923899142 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0eec01d070 con 0x7f0ef4163020 2024-05-20T18:33:22.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.014+0000 7f0ef17fa640 1 --2- 172.21.15.5:0/1923899142 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0ed406be30 0x7f0ed406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:22.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.014+0000 7f0ef17fa640 1 -- 172.21.15.5:0/1923899142 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6101+0+0 (secure 0 0 0) 0x7f0eec08b3a0 con 0x7f0ef4163020 2024-05-20T18:33:22.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.014+0000 7f0ef3fff640 1 --2- 172.21.15.5:0/1923899142 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0ed406be30 0x7f0ed406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:22.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.015+0000 7f0ef3fff640 1 --2- 172.21.15.5:0/1923899142 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0ed406be30 0x7f0ed406e340 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4034000 tx=0x7f0ee402c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:22.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.019+0000 7f0ef17fa640 1 -- 172.21.15.5:0/1923899142 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0ef40b6a90 con 0x7f0ef4163020 2024-05-20T18:33:22.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.215+0000 7f0efa63d640 1 -- 172.21.15.5:0/1923899142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0ef4075b00 con 0x7f0ef4163020 2024-05-20T18:33:22.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.216+0000 7f0ef17fa640 1 -- 172.21.15.5:0/1923899142 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v90) ==== 80+0+0 (secure 0 0 0) 0x7f0eec010070 con 0x7f0ef4163020 2024-05-20T18:33:22.216 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-05-20T18:33:22.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.217+0000 7f0efa63d640 1 -- 172.21.15.5:0/1923899142 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0ed406be30 msgr2=0x7f0ed406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:22.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.217+0000 7f0efa63d640 1 --2- 172.21.15.5:0/1923899142 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0ed406be30 0x7f0ed406e340 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4034000 tx=0x7f0ee402c000 comp rx=0 tx=0).stop 2024-05-20T18:33:22.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.217+0000 7f0efa63d640 1 -- 172.21.15.5:0/1923899142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef4163020 msgr2=0x7f0ef41f57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:22.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.217+0000 7f0efa63d640 1 --2- 172.21.15.5:0/1923899142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef4163020 0x7f0ef41f57d0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f0eec007ea0 tx=0x7f0eec007f90 comp rx=0 tx=0).stop 2024-05-20T18:33:22.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.217+0000 7f0efa63d640 1 -- 172.21.15.5:0/1923899142 shutdown_connections 2024-05-20T18:33:22.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.217+0000 7f0efa63d640 1 --2- 172.21.15.5:0/1923899142 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0ed406be30 0x7f0ed406e340 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:22.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.217+0000 7f0efa63d640 1 --2- 172.21.15.5:0/1923899142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef4163020 0x7f0ef41f57d0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:22.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.217+0000 7f0efa63d640 1 --2- 172.21.15.5:0/1923899142 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ef40cf250 0x7f0ef41e97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:22.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.218+0000 7f0efa63d640 1 --2- 172.21.15.5:0/1923899142 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0ef40b6680 0x7f0ef4158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:22.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.218+0000 7f0efa63d640 1 -- 172.21.15.5:0/1923899142 >> 172.21.15.5:0/1923899142 conn(0x7f0ef40d1860 msgr2=0x7f0ef4067a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:22.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.218+0000 7f0efa63d640 1 -- 172.21.15.5:0/1923899142 shutdown_connections 2024-05-20T18:33:22.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.218+0000 7f0efa63d640 1 -- 172.21.15.5:0/1923899142 wait complete. 2024-05-20T18:33:22.230 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-20T18:33:22.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:33:22.230 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-20T18:33:22.279 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:33:22.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:33:22.279 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-20T18:33:22.309 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:33:22.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.464+0000 7f014acf7640 1 Processor -- start 2024-05-20T18:33:22.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.465+0000 7f014acf7640 1 -- start start 2024-05-20T18:33:22.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.466+0000 7f014acf7640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f01440b47b0 0x7f01440afad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:22.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.466+0000 7f014acf7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f01440b58f0 0x7f014406f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:22.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.466+0000 7f014acf7640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01440b39f0 con 0x7f014415d060 2024-05-20T18:33:22.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.466+0000 7f014acf7640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f01440b3cd0 con 0x7f01440b47b0 2024-05-20T18:33:22.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.466+0000 7f014acf7640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f01440b3fb0 con 0x7f01440b58f0 2024-05-20T18:33:22.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.466+0000 7f0148a93640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f01440b47b0 0x7f01440afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:22.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.466+0000 7f0148a93640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f01440b47b0 0x7f01440afad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59168/0 (socket says 172.21.15.5:59168) 2024-05-20T18:33:22.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.466+0000 7f0148a93640 1 -- 172.21.15.5:0/175721223 learned_addr learned my addr 172.21.15.5:0/175721223 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:22.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.466+0000 7f0149294640 1 --2- 172.21.15.5:0/175721223 >> v2:172.21.15.63:3300/0 conn(0x7f01440b58f0 0x7f014406f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:22.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.466+0000 7f0148a93640 1 -- 172.21.15.5:0/175721223 >> v2:172.21.15.63:3300/0 conn(0x7f01440b58f0 msgr2=0x7f014406f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:22.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.466+0000 7f0148a93640 1 --2- 172.21.15.5:0/175721223 >> v2:172.21.15.63:3300/0 conn(0x7f01440b58f0 0x7f014406f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:22.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.466+0000 7f0148a93640 1 -- 172.21.15.5:0/175721223 >> v1:172.21.15.63:6789/0 conn(0x7f014415d060 legacy=0x7f01440b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:22.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.467+0000 7f0148a93640 1 -- 172.21.15.5:0/175721223 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01440b4290 con 0x7f01440b47b0 2024-05-20T18:33:22.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.467+0000 7f0148a93640 1 --2- 172.21.15.5:0/175721223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01440b47b0 0x7f01440afad0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f0138009870 tx=0x7f013802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7942b034300284 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:22.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.467+0000 7f01437fe640 1 -- 172.21.15.5:0/175721223 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0138037070 con 0x7f01440b47b0 2024-05-20T18:33:22.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.467+0000 7f01437fe640 1 -- 172.21.15.5:0/175721223 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0138032440 con 0x7f01440b47b0 2024-05-20T18:33:22.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.467+0000 7f01437fe640 1 -- 172.21.15.5:0/175721223 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0138032810 con 0x7f01440b47b0 2024-05-20T18:33:22.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.468+0000 7f014acf7640 1 -- 172.21.15.5:0/175721223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01440b47b0 msgr2=0x7f01440afad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:22.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.468+0000 7f014acf7640 1 --2- 172.21.15.5:0/175721223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01440b47b0 0x7f01440afad0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f0138009870 tx=0x7f013802a210 comp rx=0 tx=0).stop 2024-05-20T18:33:22.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.468+0000 7f014acf7640 1 -- 172.21.15.5:0/175721223 shutdown_connections 2024-05-20T18:33:22.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.468+0000 7f014acf7640 1 --2- 172.21.15.5:0/175721223 >> v2:172.21.15.63:3300/0 conn(0x7f01440b58f0 0x7f014406f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:22.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.468+0000 7f014acf7640 1 --2- 172.21.15.5:0/175721223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01440b47b0 0x7f01440afad0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:22.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.468+0000 7f014acf7640 1 -- 172.21.15.5:0/175721223 >> 172.21.15.5:0/175721223 conn(0x7f01440cd2a0 msgr2=0x7f01440cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:22.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.468+0000 7f014acf7640 1 -- 172.21.15.5:0/175721223 shutdown_connections 2024-05-20T18:33:22.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.468+0000 7f014acf7640 1 -- 172.21.15.5:0/175721223 wait complete. 2024-05-20T18:33:22.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.470+0000 7f014acf7640 1 Processor -- start 2024-05-20T18:33:22.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.470+0000 7f014acf7640 1 -- start start 2024-05-20T18:33:22.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.471+0000 7f014acf7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f014415d060 0x7f0144145320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:22.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.471+0000 7f014acf7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01440af2b0 0x7f014415d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:22.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.471+0000 7f014acf7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01440b1370 0x7f01440b1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:22.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.471+0000 7f014acf7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0144159de0 con 0x7f01440b1370 2024-05-20T18:33:22.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.471+0000 7f014acf7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0144159c60 con 0x7f01440af2b0 2024-05-20T18:33:22.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.471+0000 7f014acf7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f014406f910 con 0x7f014415d060 2024-05-20T18:33:22.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.472+0000 7f0149294640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01440b1370 0x7f01440b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:22.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.472+0000 7f0149294640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01440b1370 0x7f01440b1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59182/0 (socket says 172.21.15.5:59182) 2024-05-20T18:33:22.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.472+0000 7f0149294640 1 -- 172.21.15.5:0/1691762613 learned_addr learned my addr 172.21.15.5:0/1691762613 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:22.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.472+0000 7f0143fff640 1 --2- 172.21.15.5:0/1691762613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01440af2b0 0x7f014415d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:22.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.472+0000 7f0148a93640 1 --2- 172.21.15.5:0/1691762613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f014415d060 0x7f0144145320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:22.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.472+0000 7f0149294640 1 -- 172.21.15.5:0/1691762613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f014415d060 msgr2=0x7f0144145320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:22.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.472+0000 7f0149294640 1 --2- 172.21.15.5:0/1691762613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f014415d060 0x7f0144145320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:22.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.472+0000 7f0149294640 1 -- 172.21.15.5:0/1691762613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01440af2b0 msgr2=0x7f014415d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:22.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.472+0000 7f0149294640 1 --2- 172.21.15.5:0/1691762613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01440af2b0 0x7f014415d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:22.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.472+0000 7f0149294640 1 -- 172.21.15.5:0/1691762613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01440b1cc0 con 0x7f01440b1370 2024-05-20T18:33:22.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.472+0000 7f0149294640 1 --2- 172.21.15.5:0/1691762613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01440b1370 0x7f01440b1780 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f013c007ec0 tx=0x7f013c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:22.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.472+0000 7f01417fa640 1 -- 172.21.15.5:0/1691762613 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c014420 con 0x7f01440b1370 2024-05-20T18:33:22.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.472+0000 7f01417fa640 1 -- 172.21.15.5:0/1691762613 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f013c019070 con 0x7f01440b1370 2024-05-20T18:33:22.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.472+0000 7f014acf7640 1 -- 172.21.15.5:0/1691762613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f014415da80 con 0x7f01440b1370 2024-05-20T18:33:22.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.472+0000 7f01417fa640 1 -- 172.21.15.5:0/1691762613 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c014820 con 0x7f01440b1370 2024-05-20T18:33:22.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.472+0000 7f014acf7640 1 -- 172.21.15.5:0/1691762613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f014415fce0 con 0x7f01440b1370 2024-05-20T18:33:22.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.474+0000 7f014acf7640 1 -- 172.21.15.5:0/1691762613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0144075b00 con 0x7f01440b1370 2024-05-20T18:33:22.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.474+0000 7f01417fa640 1 -- 172.21.15.5:0/1691762613 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f013c0149c0 con 0x7f01440b1370 2024-05-20T18:33:22.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.474+0000 7f01417fa640 1 --2- 172.21.15.5:0/1691762613 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f012806be60 0x7f012806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:22.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.474+0000 7f0148a93640 1 --2- 172.21.15.5:0/1691762613 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f012806be60 0x7f012806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:22.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.474+0000 7f01417fa640 1 -- 172.21.15.5:0/1691762613 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6101+0+0 (secure 0 0 0) 0x7f013c08b680 con 0x7f01440b1370 2024-05-20T18:33:22.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.475+0000 7f0148a93640 1 --2- 172.21.15.5:0/1691762613 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f012806be60 0x7f012806e370 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0138034000 tx=0x7f013802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:22.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.479+0000 7f01417fa640 1 -- 172.21.15.5:0/1691762613 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f01440b1cc0 con 0x7f01440b1370 2024-05-20T18:33:22.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.701+0000 7f014acf7640 1 -- 172.21.15.5:0/1691762613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01440af720 con 0x7f01440b1370 2024-05-20T18:33:22.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.701+0000 7f01417fa640 1 -- 172.21.15.5:0/1691762613 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+5174 (secure 0 0 0) 0x7f013c0597c0 con 0x7f01440b1370 2024-05-20T18:33:22.702 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:33:22.703 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3705772553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3705772553},{"type":"v1","addr":"172.21.15.63:6835","nonce":3705772553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":7434,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/776188138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":776188138},{"type":"v1","addr":"172.21.15.5:6835","nonce":776188138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":7437,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3556492262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3556492262},{"type":"v1","addr":"172.21.15.5:6837","nonce":3556492262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:33:12.457716+0000","modified":"2024-05-20T18:33:15.489024+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"d","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6834/2759396130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":2759396130},{"type":"v1","addr":"172.21.15.63:6834","nonce":2759396130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7449,"qdb_cluster":[7449]},"id":6}]} 2024-05-20T18:33:22.703 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 60 2024-05-20T18:33:22.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.703+0000 7f014acf7640 1 -- 172.21.15.5:0/1691762613 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f012806be60 msgr2=0x7f012806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:22.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.703+0000 7f014acf7640 1 --2- 172.21.15.5:0/1691762613 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f012806be60 0x7f012806e370 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0138034000 tx=0x7f013802e000 comp rx=0 tx=0).stop 2024-05-20T18:33:22.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.703+0000 7f014acf7640 1 -- 172.21.15.5:0/1691762613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01440b1370 msgr2=0x7f01440b1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:22.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.703+0000 7f014acf7640 1 --2- 172.21.15.5:0/1691762613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01440b1370 0x7f01440b1780 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f013c007ec0 tx=0x7f013c00e4b0 comp rx=0 tx=0).stop 2024-05-20T18:33:22.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.704+0000 7f014acf7640 1 -- 172.21.15.5:0/1691762613 shutdown_connections 2024-05-20T18:33:22.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.704+0000 7f014acf7640 1 --2- 172.21.15.5:0/1691762613 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f012806be60 0x7f012806e370 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:22.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.704+0000 7f014acf7640 1 --2- 172.21.15.5:0/1691762613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01440b1370 0x7f01440b1780 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:22.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.704+0000 7f014acf7640 1 --2- 172.21.15.5:0/1691762613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01440af2b0 0x7f014415d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:22.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.704+0000 7f014acf7640 1 --2- 172.21.15.5:0/1691762613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f014415d060 0x7f0144145320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:22.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.704+0000 7f014acf7640 1 -- 172.21.15.5:0/1691762613 >> 172.21.15.5:0/1691762613 conn(0x7f01440cd2a0 msgr2=0x7f014415c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:22.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.704+0000 7f014acf7640 1 -- 172.21.15.5:0/1691762613 shutdown_connections 2024-05-20T18:33:22.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.704+0000 7f014acf7640 1 -- 172.21.15.5:0/1691762613 wait complete. 2024-05-20T18:33:22.716 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-05-20T18:33:22.873 INFO:teuthology.orchestra.run.smithi063.stdout:{"message":"","return_code":0} 2024-05-20T18:33:22.873 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-20T18:33:22.874 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-20T18:33:22.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.990+0000 7fa98ebd5640 1 Processor -- start 2024-05-20T18:33:22.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.991+0000 7fa98ebd5640 1 -- start start 2024-05-20T18:33:22.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.992+0000 7fa98ebd5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa9880cd680 0x7fa98815f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:22.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.992+0000 7fa98ebd5640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa9880cfae0 0x7fa98806cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:22.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.992+0000 7fa98ebd5640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa98806d0f0 con 0x7fa98806c6c0 2024-05-20T18:33:22.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.992+0000 7fa98ebd5640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fa9880cda90 con 0x7fa9880cd680 2024-05-20T18:33:22.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.992+0000 7fa98ebd5640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa98815f760 con 0x7fa9880cfae0 2024-05-20T18:33:22.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.992+0000 7fa98c971640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa9880cd680 0x7fa98815f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:22.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.992+0000 7fa987fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa98806c6c0 0x7fa9880cf3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35522/0 (socket says 172.21.15.5:35522) 2024-05-20T18:33:22.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.992+0000 7fa987fff640 1 -- 172.21.15.5:0/2633873283 learned_addr learned my addr 172.21.15.5:0/2633873283 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:33:22.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.992+0000 7fa98c971640 1 -- 172.21.15.5:0/2633873283 >> v2:172.21.15.63:3300/0 conn(0x7fa9880cfae0 msgr2=0x7fa98806cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:22.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.992+0000 7fa98d172640 1 --2- 172.21.15.5:0/2633873283 >> v2:172.21.15.63:3300/0 conn(0x7fa9880cfae0 0x7fa98806cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:22.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.992+0000 7fa98c971640 1 --2- 172.21.15.5:0/2633873283 >> v2:172.21.15.63:3300/0 conn(0x7fa9880cfae0 0x7fa98806cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:22.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.992+0000 7fa9877fe640 1 -- 172.21.15.5:0/2633873283 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1914301879 0 0) 0x7fa98806d0f0 con 0x7fa98806c6c0 2024-05-20T18:33:22.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.992+0000 7fa98c971640 1 -- 172.21.15.5:0/2633873283 >> v1:172.21.15.5:6789/0 conn(0x7fa98806c6c0 legacy=0x7fa9880cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:22.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.992+0000 7fa98c971640 1 -- 172.21.15.5:0/2633873283 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa98806d3d0 con 0x7fa9880cd680 2024-05-20T18:33:22.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.992+0000 7fa98d172640 1 --2- 172.21.15.5:0/2633873283 >> v2:172.21.15.63:3300/0 conn(0x7fa9880cfae0 0x7fa98806cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:33:22.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.993+0000 7fa98c971640 1 --2- 172.21.15.5:0/2633873283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9880cd680 0x7fa98815f220 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa9880cd1c0 tx=0x7fa97802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4dfbae145513700 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:22.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.993+0000 7fa9877fe640 1 -- 172.21.15.5:0/2633873283 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa978037070 con 0x7fa9880cd680 2024-05-20T18:33:22.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.993+0000 7fa9877fe640 1 -- 172.21.15.5:0/2633873283 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa97802adc0 con 0x7fa9880cd680 2024-05-20T18:33:22.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.993+0000 7fa9877fe640 1 -- 172.21.15.5:0/2633873283 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa978032660 con 0x7fa9880cd680 2024-05-20T18:33:22.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.994+0000 7fa98ebd5640 1 -- 172.21.15.5:0/2633873283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9880cd680 msgr2=0x7fa98815f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:22.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.994+0000 7fa98ebd5640 1 --2- 172.21.15.5:0/2633873283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9880cd680 0x7fa98815f220 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa9880cd1c0 tx=0x7fa97802a0b0 comp rx=0 tx=0).stop 2024-05-20T18:33:22.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.994+0000 7fa98ebd5640 1 -- 172.21.15.5:0/2633873283 shutdown_connections 2024-05-20T18:33:22.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.994+0000 7fa98ebd5640 1 --2- 172.21.15.5:0/2633873283 >> v2:172.21.15.63:3300/0 conn(0x7fa9880cfae0 0x7fa98806cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:22.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.994+0000 7fa98ebd5640 1 --2- 172.21.15.5:0/2633873283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9880cd680 0x7fa98815f220 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:22.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.994+0000 7fa98ebd5640 1 -- 172.21.15.5:0/2633873283 >> 172.21.15.5:0/2633873283 conn(0x7fa9880b44d0 msgr2=0x7fa9880b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:22.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.994+0000 7fa98ebd5640 1 -- 172.21.15.5:0/2633873283 shutdown_connections 2024-05-20T18:33:22.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.994+0000 7fa98ebd5640 1 -- 172.21.15.5:0/2633873283 wait complete. 2024-05-20T18:33:22.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.996+0000 7fa98ebd5640 1 Processor -- start 2024-05-20T18:33:22.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.996+0000 7fa98ebd5640 1 -- start start 2024-05-20T18:33:22.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.997+0000 7fa98ebd5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa98806c6c0 0x7fa98815bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:22.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.997+0000 7fa98ebd5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9880cd680 0x7fa9880c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:22.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.997+0000 7fa98ebd5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9880b7780 0x7fa9880b7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:22.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.997+0000 7fa98ebd5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa98806d990 con 0x7fa9880cd680 2024-05-20T18:33:22.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.997+0000 7fa98ebd5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa98806d750 con 0x7fa98806c6c0 2024-05-20T18:33:23.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.997+0000 7fa98ebd5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa9880d2460 con 0x7fa9880b7780 2024-05-20T18:33:23.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.997+0000 7fa987fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9880cd680 0x7fa9880c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.998+0000 7fa987fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9880cd680 0x7fa9880c8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59208/0 (socket says 172.21.15.5:59208) 2024-05-20T18:33:23.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.998+0000 7fa987fff640 1 -- 172.21.15.5:0/1220082399 learned_addr learned my addr 172.21.15.5:0/1220082399 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:23.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.998+0000 7fa98c971640 1 --2- 172.21.15.5:0/1220082399 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa98806c6c0 0x7fa98815bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.998+0000 7fa98d172640 1 --2- 172.21.15.5:0/1220082399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9880b7780 0x7fa9880b7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.998+0000 7fa987fff640 1 -- 172.21.15.5:0/1220082399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9880b7780 msgr2=0x7fa9880b7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:23.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.998+0000 7fa987fff640 1 --2- 172.21.15.5:0/1220082399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9880b7780 0x7fa9880b7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.998+0000 7fa987fff640 1 -- 172.21.15.5:0/1220082399 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa98806c6c0 msgr2=0x7fa98815bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:23.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.998+0000 7fa987fff640 1 --2- 172.21.15.5:0/1220082399 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa98806c6c0 0x7fa98815bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.998+0000 7fa987fff640 1 -- 172.21.15.5:0/1220082399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9880b80d0 con 0x7fa9880cd680 2024-05-20T18:33:23.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.998+0000 7fa987fff640 1 --2- 172.21.15.5:0/1220082399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9880cd680 0x7fa9880c8360 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00c460 tx=0x7fa97c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:23.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.998+0000 7fa9857fa640 1 -- 172.21.15.5:0/1220082399 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c018070 con 0x7fa9880cd680 2024-05-20T18:33:23.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.998+0000 7fa9857fa640 1 -- 172.21.15.5:0/1220082399 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa97c007500 con 0x7fa9880cd680 2024-05-20T18:33:23.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.998+0000 7fa98ebd5640 1 -- 172.21.15.5:0/1220082399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9881f5080 con 0x7fa9880cd680 2024-05-20T18:33:23.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.998+0000 7fa9857fa640 1 -- 172.21.15.5:0/1220082399 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c004a20 con 0x7fa9880cd680 2024-05-20T18:33:23.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.998+0000 7fa98ebd5640 1 -- 172.21.15.5:0/1220082399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9881f5600 con 0x7fa9880cd680 2024-05-20T18:33:23.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:22.999+0000 7fa98ebd5640 1 -- 172.21.15.5:0/1220082399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa988075b00 con 0x7fa9880cd680 2024-05-20T18:33:23.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.000+0000 7fa9857fa640 1 -- 172.21.15.5:0/1220082399 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa97c01b070 con 0x7fa9880cd680 2024-05-20T18:33:23.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.000+0000 7fa9857fa640 1 --2- 172.21.15.5:0/1220082399 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa97406be30 0x7fa97406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.001+0000 7fa9857fa640 1 -- 172.21.15.5:0/1220082399 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6101+0+0 (secure 0 0 0) 0x7fa97c08a350 con 0x7fa9880cd680 2024-05-20T18:33:23.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.001+0000 7fa98c971640 1 --2- 172.21.15.5:0/1220082399 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa97406be30 0x7fa97406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.001+0000 7fa98c971640 1 --2- 172.21.15.5:0/1220082399 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa97406be30 0x7fa97406e340 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa978034000 tx=0x7fa97802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:23.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.005+0000 7fa9857fa640 1 -- 172.21.15.5:0/1220082399 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa9880b80d0 con 0x7fa9880cd680 2024-05-20T18:33:23.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.231+0000 7fa98ebd5640 1 -- 172.21.15.5:0/1220082399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa9880b8940 con 0x7fa9880cd680 2024-05-20T18:33:23.232 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-20T18:33:23.232+0000 7fd1cf53a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-20T18:33:23.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.233+0000 7fa9857fa640 1 -- 172.21.15.5:0/1220082399 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v61) ==== 161+0+0 (secure 0 0 0) 0x7fa97c0583f0 con 0x7fa9880cd680 2024-05-20T18:33:23.234 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-20T18:33:23.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.236+0000 7fa98ebd5640 1 -- 172.21.15.5:0/1220082399 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa97406be30 msgr2=0x7fa97406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:23.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.236+0000 7fa98ebd5640 1 --2- 172.21.15.5:0/1220082399 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa97406be30 0x7fa97406e340 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa978034000 tx=0x7fa97802e000 comp rx=0 tx=0).stop 2024-05-20T18:33:23.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.236+0000 7fa98ebd5640 1 -- 172.21.15.5:0/1220082399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9880cd680 msgr2=0x7fa9880c8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:23.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.236+0000 7fa98ebd5640 1 --2- 172.21.15.5:0/1220082399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9880cd680 0x7fa9880c8360 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00c460 tx=0x7fa97c00c920 comp rx=0 tx=0).stop 2024-05-20T18:33:23.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.236+0000 7fa98ebd5640 1 -- 172.21.15.5:0/1220082399 shutdown_connections 2024-05-20T18:33:23.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.236+0000 7fa98ebd5640 1 --2- 172.21.15.5:0/1220082399 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa97406be30 0x7fa97406e340 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.236+0000 7fa98ebd5640 1 --2- 172.21.15.5:0/1220082399 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9880b7780 0x7fa9880b7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.236+0000 7fa98ebd5640 1 --2- 172.21.15.5:0/1220082399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9880cd680 0x7fa9880c8360 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.236+0000 7fa98ebd5640 1 --2- 172.21.15.5:0/1220082399 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa98806c6c0 0x7fa98815bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.236+0000 7fa98ebd5640 1 -- 172.21.15.5:0/1220082399 >> 172.21.15.5:0/1220082399 conn(0x7fa9880b44d0 msgr2=0x7fa9880cddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:23.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.236+0000 7fa98ebd5640 1 -- 172.21.15.5:0/1220082399 shutdown_connections 2024-05-20T18:33:23.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.236+0000 7fa98ebd5640 1 -- 172.21.15.5:0/1220082399 wait complete. 2024-05-20T18:33:23.239 INFO:tasks.ceph.mds.d.smithi063.stderr: -4963> 2024-05-20T18:33:18.064+0000 7f4d9b212640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T18:33:23.249 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-05-20T18:33:23.298 INFO:tasks.ceph.mds.d.smithi063.stdout:starting mds.d at 2024-05-20T18:33:23.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.304+0000 7fcd3bc90f40 1 Processor -- start 2024-05-20T18:33:23.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.304+0000 7fcd3bc90f40 1 -- start start 2024-05-20T18:33:23.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.305+0000 7fcd3bc90f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab3b0da970 0x55ab3b163cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.305+0000 7fcd3bc90f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ab3b0dc5f0 0x55ab3b0910e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.305+0000 7fcd3bc90f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3b091620 0x55ab3b1694b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.305+0000 7fcd3bc90f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ab3b0c2550 con 0x55ab3b0da970 2024-05-20T18:33:23.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.305+0000 7fcd3bc90f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ab3b03ac80 con 0x55ab3b091620 2024-05-20T18:33:23.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.305+0000 7fcd3bc90f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55ab3b0c1400 con 0x55ab3b0dc5f0 2024-05-20T18:33:23.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.305+0000 7fcd3575f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab3b0da970 0x55ab3b163cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.305+0000 7fcd3575f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab3b0da970 0x55ab3b163cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59234/0 (socket says 172.21.15.5:59234) 2024-05-20T18:33:23.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.305+0000 7fcd3575f640 1 -- 172.21.15.5:0/349277509 learned_addr learned my addr 172.21.15.5:0/349277509 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:23.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.305+0000 7fcd35f60640 1 --2- 172.21.15.5:0/349277509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3b091620 0x55ab3b1694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.305+0000 7fcd2ffff640 1 --2- 172.21.15.5:0/349277509 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ab3b0dc5f0 0x55ab3b0910e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.305+0000 7fcd3575f640 1 -- 172.21.15.5:0/349277509 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ab3b0dc5f0 msgr2=0x55ab3b0910e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:23.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.305+0000 7fcd3575f640 1 --2- 172.21.15.5:0/349277509 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ab3b0dc5f0 0x55ab3b0910e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.305+0000 7fcd3575f640 1 -- 172.21.15.5:0/349277509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3b091620 msgr2=0x55ab3b1694b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:23.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.305+0000 7fcd3575f640 1 --2- 172.21.15.5:0/349277509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3b091620 0x55ab3b1694b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.305+0000 7fcd3575f640 1 -- 172.21.15.5:0/349277509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab3afbfcb0 con 0x55ab3b0da970 2024-05-20T18:33:23.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.305+0000 7fcd2ffff640 1 --2- 172.21.15.5:0/349277509 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ab3b0dc5f0 0x55ab3b0910e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:33:23.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.305+0000 7fcd3575f640 1 --2- 172.21.15.5:0/349277509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab3b0da970 0x55ab3b163cf0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fcd28001ce0 tx=0x7fcd2800b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208ee9998ad9b791 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:23.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.306+0000 7fcd2f7fe640 1 -- 172.21.15.5:0/349277509 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2800bd20 con 0x55ab3b0da970 2024-05-20T18:33:23.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.306+0000 7fcd2f7fe640 1 -- 172.21.15.5:0/349277509 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd2800f400 con 0x55ab3b0da970 2024-05-20T18:33:23.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.306+0000 7fcd2f7fe640 1 -- 172.21.15.5:0/349277509 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2800f7f0 con 0x55ab3b0da970 2024-05-20T18:33:23.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.306+0000 7fcd3bc90f40 1 -- 172.21.15.5:0/349277509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab3b0da970 msgr2=0x55ab3b163cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:23.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.306+0000 7fcd3bc90f40 1 --2- 172.21.15.5:0/349277509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab3b0da970 0x55ab3b163cf0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fcd28001ce0 tx=0x7fcd2800b5b0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.306+0000 7fcd3bc90f40 1 -- 172.21.15.5:0/349277509 shutdown_connections 2024-05-20T18:33:23.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.306+0000 7fcd3bc90f40 1 --2- 172.21.15.5:0/349277509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3b091620 0x55ab3b1694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.306+0000 7fcd3bc90f40 1 --2- 172.21.15.5:0/349277509 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ab3b0dc5f0 0x55ab3b0910e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.306+0000 7fcd3bc90f40 1 --2- 172.21.15.5:0/349277509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab3b0da970 0x55ab3b163cf0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.306+0000 7fcd3bc90f40 1 -- 172.21.15.5:0/349277509 >> 172.21.15.5:0/349277509 conn(0x55ab3afb8ac0 msgr2=0x55ab3b167800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:23.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.306+0000 7fcd3bc90f40 1 -- 172.21.15.5:0/349277509 shutdown_connections 2024-05-20T18:33:23.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.306+0000 7fcd3bc90f40 1 -- 172.21.15.5:0/349277509 wait complete. 2024-05-20T18:33:23.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.307+0000 7fcd3bc90f40 1 Processor -- start 2024-05-20T18:33:23.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.307+0000 7fcd3bc90f40 1 -- start start 2024-05-20T18:33:23.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.308+0000 7fcd3bc90f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ab3b091620 0x55ab3b1619d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.308+0000 7fcd3bc90f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab3b0da970 0x55ab3b1f6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.308+0000 7fcd3bc90f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3b0dc5f0 0x55ab3b1f7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.308+0000 7fcd3bc90f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ab3b165540 con 0x55ab3b0da970 2024-05-20T18:33:23.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.308+0000 7fcd3bc90f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ab3b1642e0 con 0x55ab3b0dc5f0 2024-05-20T18:33:23.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.308+0000 7fcd3bc90f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55ab3b1583f0 con 0x55ab3b091620 2024-05-20T18:33:23.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.309+0000 7fcd2ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab3b0da970 0x55ab3b1f6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.309+0000 7fcd2ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab3b0da970 0x55ab3b1f6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59242/0 (socket says 172.21.15.5:59242) 2024-05-20T18:33:23.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.309+0000 7fcd2ffff640 1 -- 172.21.15.5:0/407574540 learned_addr learned my addr 172.21.15.5:0/407574540 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:23.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.309+0000 7fcd3575f640 1 --2- 172.21.15.5:0/407574540 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ab3b091620 0x55ab3b1619d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.309+0000 7fcd35f60640 1 --2- 172.21.15.5:0/407574540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3b0dc5f0 0x55ab3b1f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.309+0000 7fcd2ffff640 1 -- 172.21.15.5:0/407574540 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ab3b091620 msgr2=0x55ab3b1619d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:23.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.309+0000 7fcd2ffff640 1 --2- 172.21.15.5:0/407574540 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ab3b091620 0x55ab3b1619d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.309+0000 7fcd2ffff640 1 -- 172.21.15.5:0/407574540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3b0dc5f0 msgr2=0x55ab3b1f7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:23.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.309+0000 7fcd2ffff640 1 --2- 172.21.15.5:0/407574540 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ab3b0dc5f0 0x55ab3b1f7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.309+0000 7fcd2ffff640 1 -- 172.21.15.5:0/407574540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab3b1575c0 con 0x55ab3b0da970 2024-05-20T18:33:23.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.309+0000 7fcd2ffff640 1 --2- 172.21.15.5:0/407574540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab3b0da970 0x55ab3b1f6fd0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fcd20009f50 tx=0x7fcd200084c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:23.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.309+0000 7fcd2e7fc640 1 -- 172.21.15.5:0/407574540 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd20008dc0 con 0x55ab3b0da970 2024-05-20T18:33:23.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.309+0000 7fcd2e7fc640 1 -- 172.21.15.5:0/407574540 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd200143f0 con 0x55ab3b0da970 2024-05-20T18:33:23.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.309+0000 7fcd2e7fc640 1 -- 172.21.15.5:0/407574540 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd200147e0 con 0x55ab3b0da970 2024-05-20T18:33:23.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.309+0000 7fcd3bc90f40 1 -- 172.21.15.5:0/407574540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ab3b163560 con 0x55ab3b0da970 2024-05-20T18:33:23.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.309+0000 7fcd3bc90f40 1 -- 172.21.15.5:0/407574540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab3b169bb0 con 0x55ab3b0da970 2024-05-20T18:33:23.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.311+0000 7fcd2e7fc640 1 -- 172.21.15.5:0/407574540 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcd2001b020 con 0x55ab3b0da970 2024-05-20T18:33:23.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.311+0000 7fcd2e7fc640 1 --2- 172.21.15.5:0/407574540 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcd1806a840 0x7fcd1806cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.311+0000 7fcd2e7fc640 1 -- 172.21.15.5:0/407574540 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6187+0+0 (secure 0 0 0) 0x7fcd200895a0 con 0x55ab3b0da970 2024-05-20T18:33:23.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.311+0000 7fcd3575f640 1 --2- 172.21.15.5:0/407574540 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcd1806a840 0x7fcd1806cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.312+0000 7fcd3bc90f40 1 --2- 172.21.15.5:0/407574540 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55ab3b202e70 0x55ab3b203280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.312+0000 7fcd3bc90f40 1 -- 172.21.15.5:0/407574540 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e91) -- 0x55ab3b15b4a0 con 0x55ab3b202e70 2024-05-20T18:33:23.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.312+0000 7fcd35f60640 1 --2- 172.21.15.5:0/407574540 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55ab3b202e70 0x55ab3b203280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.312+0000 7fcd3575f640 1 --2- 172.21.15.5:0/407574540 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcd1806a840 0x7fcd1806cd50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fcd28003040 tx=0x7fcd28009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:23.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.312+0000 7fcd35f60640 1 --2- 172.21.15.5:0/407574540 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55ab3b202e70 0x55ab3b203280 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:23.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.315+0000 7fcd35f60640 1 -- 172.21.15.5:0/407574540 <== osd.1 v2:172.21.15.5:6815/550664725 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v91'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fcd30051950 con 0x55ab3b202e70 2024-05-20T18:33:23.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.315+0000 7fcd3bc90f40 1 -- 172.21.15.5:0/407574540 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55ab3b202e70 msgr2=0x55ab3b203280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.315+0000 7fcd3bc90f40 1 --2- 172.21.15.5:0/407574540 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55ab3b202e70 0x55ab3b203280 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.315+0000 7fcd3bc90f40 1 -- 172.21.15.5:0/407574540 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcd1806a840 msgr2=0x7fcd1806cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.315+0000 7fcd3bc90f40 1 --2- 172.21.15.5:0/407574540 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcd1806a840 0x7fcd1806cd50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fcd28003040 tx=0x7fcd28009000 comp rx=0 tx=0).stop 2024-05-20T18:33:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.315+0000 7fcd3bc90f40 1 -- 172.21.15.5:0/407574540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab3b0da970 msgr2=0x55ab3b1f6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.315+0000 7fcd3bc90f40 1 --2- 172.21.15.5:0/407574540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab3b0da970 0x55ab3b1f6fd0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fcd20009f50 tx=0x7fcd200084c0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.315+0000 7fcd35f60640 1 -- 172.21.15.5:0/407574540 reap_dead start 2024-05-20T18:33:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.316+0000 7fcd3bc90f40 1 -- 172.21.15.5:0/407574540 shutdown_connections 2024-05-20T18:33:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.316+0000 7fcd3bc90f40 1 -- 172.21.15.5:0/407574540 >> 172.21.15.5:0/407574540 conn(0x55ab3afb8ac0 msgr2=0x55ab3b090f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.316+0000 7fcd3bc90f40 1 -- 172.21.15.5:0/407574540 shutdown_connections 2024-05-20T18:33:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.316+0000 7fcd3bc90f40 1 -- 172.21.15.5:0/407574540 wait complete. 2024-05-20T18:33:23.321 DEBUG:teuthology.orchestra.run.smithi005:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset 2024-05-20T18:33:23.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.373+0000 7f0a89723280 1 -- start start 2024-05-20T18:33:23.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.373+0000 7f0a89723280 1 Processor -- start 2024-05-20T18:33:23.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.374+0000 7f0a89723280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db23413680 0x55db234f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.374+0000 7f0a89723280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db23413b00 0x55db234f4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.374+0000 7f0a89723280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55db23414400 0x55db234f5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.374+0000 7f0a89723280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55db24146180 con 0x55db23413b00 2024-05-20T18:33:23.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.374+0000 7f0a89723280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55db24146300 con 0x55db23413680 2024-05-20T18:33:23.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.374+0000 7f0a89723280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55db24146480 con 0x55db23414400 2024-05-20T18:33:23.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.374+0000 7f0a86e4d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db23413b00 0x55db234f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.375+0000 7f0a86e4d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db23413b00 0x55db234f4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59258/0 (socket says 172.21.15.5:59258) 2024-05-20T18:33:23.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.375+0000 7f0a86e4d640 1 -- 172.21.15.5:0/5059369 learned_addr learned my addr 172.21.15.5:0/5059369 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:23.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.375+0000 7f0a8764e640 1 --2- 172.21.15.5:0/5059369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db23413680 0x55db234f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.375+0000 7f0a87e4f640 1 --2- 172.21.15.5:0/5059369 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55db23414400 0x55db234f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.375+0000 7f0a86e4d640 1 -- 172.21.15.5:0/5059369 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55db23414400 msgr2=0x55db234f5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:23.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.375+0000 7f0a86e4d640 1 --2- 172.21.15.5:0/5059369 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55db23414400 0x55db234f5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.375+0000 7f0a86e4d640 1 -- 172.21.15.5:0/5059369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db23413680 msgr2=0x55db234f4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:23.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.375+0000 7f0a86e4d640 1 --2- 172.21.15.5:0/5059369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db23413680 0x55db234f4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.375+0000 7f0a86e4d640 1 -- 172.21.15.5:0/5059369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db1f370780 con 0x55db23413b00 2024-05-20T18:33:23.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.375+0000 7f0a86e4d640 1 --2- 172.21.15.5:0/5059369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db23413b00 0x55db234f4580 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x55db2418aae0 tx=0x55db23460b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8cff82b181f2c50 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:23.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.375+0000 7f0a84e49640 1 -- 172.21.15.5:0/5059369 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55db1f3609c0 con 0x55db23413b00 2024-05-20T18:33:23.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.375+0000 7f0a84e49640 1 -- 172.21.15.5:0/5059369 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55db1f366fc0 con 0x55db23413b00 2024-05-20T18:33:23.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.375+0000 7f0a84e49640 1 -- 172.21.15.5:0/5059369 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55db1f360680 con 0x55db23413b00 2024-05-20T18:33:23.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.375+0000 7f0a89723280 1 -- 172.21.15.5:0/5059369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db1f370000 con 0x55db23413b00 2024-05-20T18:33:23.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.375+0000 7f0a89723280 1 -- 172.21.15.5:0/5059369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55db1f3703c0 con 0x55db23413b00 2024-05-20T18:33:23.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.375+0000 7f0a89723280 4 waiting for MDS map... 2024-05-20T18:33:23.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.375+0000 7f0a84e49640 1 -- 172.21.15.5:0/5059369 <== mon.0 v2:172.21.15.5:3300/0 4 ==== osd_map(91..91 src has 1..91) ==== 6187+0+0 (secure 0 0 0) 0x55db24246600 con 0x55db23413b00 2024-05-20T18:33:23.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.375+0000 7f0a84e49640 1 -- 172.21.15.5:0/5059369 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 61) ==== 2885+0+0 (secure 0 0 0) 0x55db233c5180 con 0x55db23413b00 2024-05-20T18:33:23.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.375+0000 7f0a89723280 4 Got MDS map 61 2024-05-20T18:33:23.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.376+0000 7f0a89723280 10 main: JournalTool::main 2024-05-20T18:33:23.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.376+0000 7f0a89723280 4 main: JournalTool: connecting to RADOS... 2024-05-20T18:33:23.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.376+0000 7f0a89723280 1 Processor -- start 2024-05-20T18:33:23.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.377+0000 7f0a89723280 1 -- start start 2024-05-20T18:33:23.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.378+0000 7f0a89723280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db23412d80 0x55db234f5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.378+0000 7f0a89723280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55db23412900 0x55db234f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.378+0000 7f0a89723280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db23412480 0x55db234f6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.378+0000 7f0a89723280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55db24146600 con 0x55db23412480 2024-05-20T18:33:23.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.378+0000 7f0a89723280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55db24146780 con 0x55db23412d80 2024-05-20T18:33:23.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.378+0000 7f0a89723280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55db24146900 con 0x55db23412900 2024-05-20T18:33:23.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.378+0000 7f0a86e4d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db23412480 0x55db234f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.378+0000 7f0a86e4d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db23412480 0x55db234f6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59268/0 (socket says 172.21.15.5:59268) 2024-05-20T18:33:23.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.378+0000 7f0a86e4d640 1 -- 172.21.15.5:0/2202991494 learned_addr learned my addr 172.21.15.5:0/2202991494 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:23.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.378+0000 7f0a8764e640 1 --2- 172.21.15.5:0/2202991494 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55db23412900 0x55db234f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.378+0000 7f0a87e4f640 1 --2- 172.21.15.5:0/2202991494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db23412d80 0x55db234f5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.378+0000 7f0a86e4d640 1 -- 172.21.15.5:0/2202991494 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55db23412900 msgr2=0x55db234f5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:23.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.378+0000 7f0a86e4d640 1 --2- 172.21.15.5:0/2202991494 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55db23412900 0x55db234f5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.378+0000 7f0a86e4d640 1 -- 172.21.15.5:0/2202991494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db23412d80 msgr2=0x55db234f5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:23.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.378+0000 7f0a86e4d640 1 --2- 172.21.15.5:0/2202991494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db23412d80 0x55db234f5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.378+0000 7f0a86e4d640 1 -- 172.21.15.5:0/2202991494 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db1f370960 con 0x55db23412480 2024-05-20T18:33:23.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.378+0000 7f0a86e4d640 1 --2- 172.21.15.5:0/2202991494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db23412480 0x55db234f6100 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x55db241421b0 tx=0x55db234606c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64319f5d5ae656e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:23.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.378+0000 7f0a82e45640 1 -- 172.21.15.5:0/2202991494 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55db1f360b60 con 0x55db23412480 2024-05-20T18:33:23.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.378+0000 7f0a82e45640 1 -- 172.21.15.5:0/2202991494 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55db1f366e00 con 0x55db23412480 2024-05-20T18:33:23.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.378+0000 7f0a82e45640 1 -- 172.21.15.5:0/2202991494 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55db1f360820 con 0x55db23412480 2024-05-20T18:33:23.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.379+0000 7f0a89723280 1 -- 172.21.15.5:0/2202991494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db23412480 msgr2=0x55db234f6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:23.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.379+0000 7f0a89723280 1 --2- 172.21.15.5:0/2202991494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db23412480 0x55db234f6100 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x55db241421b0 tx=0x55db234606c0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.379+0000 7f0a89723280 1 -- 172.21.15.5:0/2202991494 shutdown_connections 2024-05-20T18:33:23.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.379+0000 7f0a89723280 1 --2- 172.21.15.5:0/2202991494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db23412480 0x55db234f6100 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.379+0000 7f0a89723280 1 --2- 172.21.15.5:0/2202991494 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55db23412900 0x55db234f5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.379+0000 7f0a89723280 1 --2- 172.21.15.5:0/2202991494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db23412d80 0x55db234f5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.379+0000 7f0a89723280 1 -- 172.21.15.5:0/2202991494 >> 172.21.15.5:0/2202991494 conn(0x55db23413200 msgr2=0x55db23485800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:23.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.379+0000 7f0a89723280 1 -- 172.21.15.5:0/2202991494 shutdown_connections 2024-05-20T18:33:23.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.379+0000 7f0a89723280 1 -- 172.21.15.5:0/2202991494 wait complete. 2024-05-20T18:33:23.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.380+0000 7f0a89723280 1 Processor -- start 2024-05-20T18:33:23.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.380+0000 7f0a89723280 1 -- start start 2024-05-20T18:33:23.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.381+0000 7f0a89723280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db23412480 0x55db234f7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.381+0000 7f0a89723280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db23412900 0x55db234f6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.381+0000 7f0a89723280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55db23412d80 0x55db234f6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.381+0000 7f0a89723280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55db24147500 con 0x55db23412480 2024-05-20T18:33:23.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.381+0000 7f0a89723280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55db24146a80 con 0x55db23412900 2024-05-20T18:33:23.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.381+0000 7f0a89723280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55db24147680 con 0x55db23412d80 2024-05-20T18:33:23.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.381+0000 7f0a87e4f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db23412480 0x55db234f7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.381+0000 7f0a87e4f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db23412480 0x55db234f7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59272/0 (socket says 172.21.15.5:59272) 2024-05-20T18:33:23.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.381+0000 7f0a87e4f640 1 -- 172.21.15.5:0/703409029 learned_addr learned my addr 172.21.15.5:0/703409029 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:23.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.381+0000 7f0a8764e640 1 --2- 172.21.15.5:0/703409029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db23412900 0x55db234f6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.381+0000 7f0a86e4d640 1 --2- 172.21.15.5:0/703409029 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55db23412d80 0x55db234f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.381+0000 7f0a87e4f640 1 -- 172.21.15.5:0/703409029 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55db23412d80 msgr2=0x55db234f6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:23.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.381+0000 7f0a87e4f640 1 --2- 172.21.15.5:0/703409029 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55db23412d80 0x55db234f6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.381+0000 7f0a87e4f640 1 -- 172.21.15.5:0/703409029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db23412900 msgr2=0x55db234f6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:23.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.381+0000 7f0a87e4f640 1 --2- 172.21.15.5:0/703409029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db23412900 0x55db234f6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.381+0000 7f0a87e4f640 1 -- 172.21.15.5:0/703409029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db1f370780 con 0x55db23412480 2024-05-20T18:33:23.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.381+0000 7f0a86e4d640 1 --2- 172.21.15.5:0/703409029 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55db23412d80 0x55db234f6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:33:23.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.382+0000 7f0a87e4f640 1 --2- 172.21.15.5:0/703409029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db23412480 0x55db234f7180 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x55db24143ad0 tx=0x55db24145800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:23.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.382+0000 7f0a81642640 1 -- 172.21.15.5:0/703409029 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55db1f360b60 con 0x55db23412480 2024-05-20T18:33:23.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.382+0000 7f0a81642640 1 -- 172.21.15.5:0/703409029 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55db1f366a80 con 0x55db23412480 2024-05-20T18:33:23.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.382+0000 7f0a89723280 1 -- 172.21.15.5:0/703409029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55db1f370000 con 0x55db23412480 2024-05-20T18:33:23.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.382+0000 7f0a81642640 1 -- 172.21.15.5:0/703409029 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55db1f360820 con 0x55db23412480 2024-05-20T18:33:23.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.382+0000 7f0a89723280 1 -- 172.21.15.5:0/703409029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db1f3703c0 con 0x55db23412480 2024-05-20T18:33:23.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.382+0000 7f0a89723280 4 main: JournalTool: resolving pool 13 2024-05-20T18:33:23.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.383+0000 7f0a81642640 1 -- 172.21.15.5:0/703409029 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x55db2429ac00 con 0x55db23412480 2024-05-20T18:33:23.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.384+0000 7f0a81642640 1 --2- 172.21.15.5:0/703409029 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x55db23415180 0x55db234f7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.384+0000 7f0a81642640 1 -- 172.21.15.5:0/703409029 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6187+0+0 (secure 0 0 0) 0x55db24291000 con 0x55db23412480 2024-05-20T18:33:23.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.384+0000 7f0a89723280 4 main: JournalTool: creating IoCtx.. 2024-05-20T18:33:23.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.384+0000 7f0a89723280 4 main: Executing for rank 0 2024-05-20T18:33:23.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.384+0000 7f0a8764e640 1 --2- 172.21.15.5:0/703409029 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x55db23415180 0x55db234f7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.384+0000 7f0a8764e640 1 --2- 172.21.15.5:0/703409029 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x55db23415180 0x55db234f7700 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x55db2344a6f0 tx=0x55db236cce40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:23.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.385+0000 7f0a89723280 1 -- start start 2024-05-20T18:33:23.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.385+0000 7f0a89723280 1 Processor -- start 2024-05-20T18:33:23.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.386+0000 7f0a89723280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db23414880 0x55db234f5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.386+0000 7f0a89723280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db23414d00 0x55db234f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.386+0000 7f0a89723280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55db23415600 0x55db234f6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.386+0000 7f0a89723280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55db24147980 con 0x55db23414d00 2024-05-20T18:33:23.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.386+0000 7f0a89723280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55db24147b00 con 0x55db23414880 2024-05-20T18:33:23.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.386+0000 7f0a89723280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55db24147c80 con 0x55db23415600 2024-05-20T18:33:23.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.386+0000 7f0a8764e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db23414d00 0x55db234f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.386+0000 7f0a8764e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db23414d00 0x55db234f5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59280/0 (socket says 172.21.15.5:59280) 2024-05-20T18:33:23.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.386+0000 7f0a8764e640 1 -- 172.21.15.5:0/3180916834 learned_addr learned my addr 172.21.15.5:0/3180916834 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:23.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.386+0000 7f0a86e4d640 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55db23415600 0x55db234f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.386+0000 7f0a87e4f640 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db23414880 0x55db234f5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.386+0000 7f0a8764e640 1 -- 172.21.15.5:0/3180916834 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55db23415600 msgr2=0x55db234f6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:23.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.386+0000 7f0a8764e640 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55db23415600 0x55db234f6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.386+0000 7f0a8764e640 1 -- 172.21.15.5:0/3180916834 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db23414880 msgr2=0x55db234f5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:23.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.386+0000 7f0a8764e640 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db23414880 0x55db234f5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.386+0000 7f0a8764e640 1 -- 172.21.15.5:0/3180916834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db1f370960 con 0x55db23414d00 2024-05-20T18:33:23.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.386+0000 7f0a86e4d640 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55db23415600 0x55db234f6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:33:23.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.386+0000 7f0a87e4f640 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db23414880 0x55db234f5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:33:23.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.387+0000 7f0a8764e640 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db23414d00 0x55db234f5b80 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x55db242d1020 tx=0x55db23460960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e71c462c46230f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:23.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.387+0000 7f0a7e63c640 1 -- 172.21.15.5:0/3180916834 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55db242df380 con 0x55db23414d00 2024-05-20T18:33:23.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.387+0000 7f0a7e63c640 1 -- 172.21.15.5:0/3180916834 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55db1f366c40 con 0x55db23414d00 2024-05-20T18:33:23.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.387+0000 7f0a7e63c640 1 -- 172.21.15.5:0/3180916834 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55db236c24e0 con 0x55db23414d00 2024-05-20T18:33:23.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.387+0000 7f0a89723280 1 -- 172.21.15.5:0/3180916834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db1f370780 con 0x55db23414d00 2024-05-20T18:33:23.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.387+0000 7f0a89723280 1 -- 172.21.15.5:0/3180916834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55db1f370000 con 0x55db23414d00 2024-05-20T18:33:23.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.387+0000 7f0a89723280 4 waiting for MDS map... 2024-05-20T18:33:23.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.387+0000 7f0a7e63c640 1 -- 172.21.15.5:0/3180916834 <== mon.0 v2:172.21.15.5:3300/0 4 ==== osd_map(91..91 src has 1..91) ==== 6187+0+0 (secure 0 0 0) 0x55db2379a600 con 0x55db23414d00 2024-05-20T18:33:23.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.387+0000 7f0a7e63c640 1 -- 172.21.15.5:0/3180916834 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 61) ==== 2885+0+0 (secure 0 0 0) 0x55db233c5500 con 0x55db23414d00 2024-05-20T18:33:23.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.387+0000 7f0a89723280 4 Got MDS map 61 2024-05-20T18:33:23.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.387+0000 7f0a89723280 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x55db23415a80 0x55db237ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.387+0000 7f0a89723280 1 -- 172.21.15.5:0/3180916834 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55db237b2000 con 0x55db23415a80 2024-05-20T18:33:23.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.387+0000 7f0a87e4f640 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x55db23415a80 0x55db237ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.388+0000 7f0a87e4f640 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x55db23415a80 0x55db237ae000 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=f97a8bb1d2ec8b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:23.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.388+0000 7f0a87e4f640 1 -- 172.21.15.5:0/3180916834 <== osd.6 v2:172.21.15.63:6816/3384565140 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x55db23498000 con 0x55db23415a80 2024-05-20T18:33:23.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.388+0000 7f0a89723280 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x55db237b8000 0x55db237ae580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.388+0000 7f0a89723280 1 -- 172.21.15.5:0/3180916834 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:2 13.14 13:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) -- 0x55db237b2400 con 0x55db237b8000 2024-05-20T18:33:23.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.388+0000 7f0a86e4d640 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x55db237b8000 0x55db237ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.389+0000 7f0a86e4d640 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x55db237b8000 0x55db237ae580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=4e1dbfb74db5a845 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:23.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.391+0000 7f0a86e4d640 1 -- 172.21.15.5:0/3180916834 <== osd.0 v2:172.21.15.5:6808/688093005 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv20 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x55db23498500 con 0x55db237b8000 2024-05-20T18:33:23.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.391+0000 7f0a7ce39640 1 -- 172.21.15.5:0/3180916834 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:3 13.14 13:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e91) -- 0x55db237b2800 con 0x55db237b8000 2024-05-20T18:33:23.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.391+0000 7f0a7ce39640 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x55db237bb600 0x55db237b0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.391+0000 7f0a7ce39640 1 -- 172.21.15.5:0/3180916834 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:4 13.1b 13:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e91) -- 0x55db237b2c00 con 0x55db237bb600 2024-05-20T18:33:23.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.392+0000 7f0a86e4d640 1 -- 172.21.15.5:0/3180916834 <== osd.0 v2:172.21.15.5:6808/688093005 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv17 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x55db23498500 con 0x55db237b8000 2024-05-20T18:33:23.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.392+0000 7f0a87e4f640 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x55db237bb600 0x55db237b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.392+0000 7f0a87e4f640 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x55db237bb600 0x55db237b0680 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=56bc9f5b181300df server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:23.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.392+0000 7f0a87e4f640 1 -- 172.21.15.5:0/3180916834 <== osd.4 v2:172.21.15.63:6800/3673324406 1 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55db23498000 con 0x55db237bb600 2024-05-20T18:33:23.396 INFO:teuthology.orchestra.run.smithi005.stdout:old journal was 4212724~2605 2024-05-20T18:33:23.396 INFO:teuthology.orchestra.run.smithi005.stdout:new journal start will be 8388608 (4173279 bytes past old end) 2024-05-20T18:33:23.396 INFO:teuthology.orchestra.run.smithi005.stdout:writing journal head 2024-05-20T18:33:23.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.392+0000 7f0a89723280 1 -- 172.21.15.5:0/3180916834 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:5 13.14 13:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55db237b3400 con 0x55db237b8000 2024-05-20T18:33:23.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.394+0000 7f0a86e4d640 1 -- 172.21.15.5:0/3180916834 <== osd.0 v2:172.21.15.5:6808/688093005 3 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v91'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55db23498500 con 0x55db237b8000 2024-05-20T18:33:23.397 INFO:teuthology.orchestra.run.smithi005.stdout:writing EResetJournal entry 2024-05-20T18:33:23.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.394+0000 7f0a89723280 1 -- 172.21.15.5:0/3180916834 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:6 13.1b 13:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55db237b3800 con 0x55db237bb600 2024-05-20T18:33:23.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.394+0000 7f0a89723280 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x55db237b8480 0x55db237aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.394+0000 7f0a89723280 1 -- 172.21.15.5:0/3180916834 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:7 13.b 13:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55db237b3c00 con 0x55db237b8480 2024-05-20T18:33:23.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.394+0000 7f0a89723280 1 -- 172.21.15.5:0/3180916834 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:8 13.1f 13:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55db237d0000 con 0x55db23415a80 2024-05-20T18:33:23.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.394+0000 7f0a89723280 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55db237b8900 0x55db237af080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.394+0000 7f0a89723280 1 -- 172.21.15.5:0/3180916834 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:9 13.1a 13:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55db237d0400 con 0x55db237b8900 2024-05-20T18:33:23.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.394+0000 7f0a89723280 1 -- 172.21.15.5:0/3180916834 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:10 13.1e 13:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55db237d0800 con 0x55db237bb600 2024-05-20T18:33:23.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.394+0000 7f0a89723280 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x55db237b8d80 0x55db237af600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.394+0000 7f0a89723280 1 -- 172.21.15.5:0/3180916834 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:11 13.1d 13:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55db237d0c00 con 0x55db237b8d80 2024-05-20T18:33:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.394+0000 7f0a86e4d640 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55db237b8900 0x55db237af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.394+0000 7f0a8764e640 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x55db237b8480 0x55db237aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.394+0000 7f0a87e4f640 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x55db237b8d80 0x55db237af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.394+0000 7f0a86e4d640 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55db237b8900 0x55db237af080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=cca90e072b0f0571 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.395+0000 7f0a8764e640 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x55db237b8480 0x55db237aeb00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=310d9707a2373448 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.395+0000 7f0a87e4f640 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x55db237b8d80 0x55db237af600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=9e71a81575e004c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.395+0000 7f0a87e4f640 1 -- 172.21.15.5:0/3180916834 <== osd.6 v2:172.21.15.63:6816/3384565140 2 ==== osd_op_reply(8 200.00000004 [delete] v91'7 uv6 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55db23498000 con 0x55db23415a80 2024-05-20T18:33:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.395+0000 7f0a87e4f640 1 -- 172.21.15.5:0/3180916834 <== osd.4 v2:172.21.15.63:6800/3673324406 2 ==== osd_op_reply(6 200.00000002 [delete] v91'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55db23498000 con 0x55db237bb600 2024-05-20T18:33:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.395+0000 7f0a87e4f640 1 -- 172.21.15.5:0/3180916834 <== osd.4 v2:172.21.15.63:6800/3673324406 3 ==== osd_op_reply(10 200.00000006 [delete] v91'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55db23498000 con 0x55db237bb600 2024-05-20T18:33:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.395+0000 7f0a86e4d640 1 -- 172.21.15.5:0/3180916834 <== osd.1 v2:172.21.15.5:6815/550664725 1 ==== osd_op_reply(9 200.00000005 [delete] v91'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55db23498500 con 0x55db237b8900 2024-05-20T18:33:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.398+0000 7f0a8764e640 1 -- 172.21.15.5:0/3180916834 <== osd.5 v2:172.21.15.63:6808/1616066286 1 ==== osd_op_reply(7 200.00000003 [delete] v91'4 uv3 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55db23498780 con 0x55db237b8480 2024-05-20T18:33:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.398+0000 7f0a7ce39640 1 -- 172.21.15.5:0/3180916834 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:12 13.1b 13:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) -- 0x55db237b3000 con 0x55db237bb600 2024-05-20T18:33:23.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.398+0000 7f0a87e4f640 1 -- 172.21.15.5:0/3180916834 <== osd.7 v2:172.21.15.63:6824/4136124451 1 ==== osd_op_reply(11 200.00000007 [delete] v91'3 uv2 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55db23498000 con 0x55db237b8d80 2024-05-20T18:33:23.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.399+0000 7f0a87e4f640 1 -- 172.21.15.5:0/3180916834 <== osd.4 v2:172.21.15.63:6800/3673324406 4 ==== osd_op_reply(12 200.00000002 [write 0~48] v91'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55db23498000 con 0x55db237bb600 2024-05-20T18:33:23.401 INFO:teuthology.orchestra.run.smithi005.stdout:done 2024-05-20T18:33:23.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.399+0000 7f0a89723280 1 -- 172.21.15.5:0/3180916834 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x55db237b8000 msgr2=0x55db237ae580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:23.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.399+0000 7f0a89723280 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x55db237b8000 0x55db237ae580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.399+0000 7f0a89723280 1 -- 172.21.15.5:0/3180916834 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55db237b8900 msgr2=0x55db237af080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:23.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.399+0000 7f0a89723280 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55db237b8900 0x55db237af080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.399+0000 7f0a89723280 1 -- 172.21.15.5:0/3180916834 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x55db237bb600 msgr2=0x55db237b0680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:23.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.399+0000 7f0a89723280 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x55db237bb600 0x55db237b0680 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.399+0000 7f0a89723280 1 -- 172.21.15.5:0/3180916834 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x55db237b8480 msgr2=0x55db237aeb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:23.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.399+0000 7f0a89723280 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x55db237b8480 0x55db237aeb00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.399+0000 7f0a89723280 1 -- 172.21.15.5:0/3180916834 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x55db23415a80 msgr2=0x55db237ae000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:23.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.399+0000 7f0a89723280 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x55db23415a80 0x55db237ae000 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.399+0000 7f0a89723280 1 -- 172.21.15.5:0/3180916834 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x55db237b8d80 msgr2=0x55db237af600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:23.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.399+0000 7f0a89723280 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x55db237b8d80 0x55db237af600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.399+0000 7f0a89723280 1 -- 172.21.15.5:0/3180916834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db23414d00 msgr2=0x55db234f5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:23.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.399+0000 7f0a89723280 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db23414d00 0x55db234f5b80 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x55db242d1020 tx=0x55db23460960 comp rx=0 tx=0).stop 2024-05-20T18:33:23.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.399+0000 7f0a86e4d640 1 -- 172.21.15.5:0/3180916834 reap_dead start 2024-05-20T18:33:23.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.399+0000 7f0a89723280 1 -- 172.21.15.5:0/3180916834 shutdown_connections 2024-05-20T18:33:23.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.399+0000 7f0a89723280 1 --2- 172.21.15.5:0/3180916834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db23414d00 0x55db234f5b80 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.399+0000 7f0a89723280 1 -- 172.21.15.5:0/3180916834 >> 172.21.15.5:0/3180916834 conn(0x55db23412000 msgr2=0x55db23740800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:23.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.399+0000 7f0a89723280 1 -- 172.21.15.5:0/3180916834 shutdown_connections 2024-05-20T18:33:23.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.399+0000 7f0a89723280 1 -- 172.21.15.5:0/3180916834 wait complete. 2024-05-20T18:33:23.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.400+0000 7f0a89723280 1 -- 172.21.15.5:0/703409029 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x55db23415180 msgr2=0x55db234f7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:23.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.400+0000 7f0a89723280 1 --2- 172.21.15.5:0/703409029 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x55db23415180 0x55db234f7700 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x55db2344a6f0 tx=0x55db236cce40 comp rx=0 tx=0).stop 2024-05-20T18:33:23.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.400+0000 7f0a89723280 1 -- 172.21.15.5:0/703409029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db23412480 msgr2=0x55db234f7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:23.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.400+0000 7f0a89723280 1 --2- 172.21.15.5:0/703409029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db23412480 0x55db234f7180 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x55db24143ad0 tx=0x55db24145800 comp rx=0 tx=0).stop 2024-05-20T18:33:23.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.400+0000 7f0a89723280 1 -- 172.21.15.5:0/703409029 shutdown_connections 2024-05-20T18:33:23.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.400+0000 7f0a89723280 1 --2- 172.21.15.5:0/703409029 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x55db23415180 0x55db234f7700 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.400+0000 7f0a89723280 1 --2- 172.21.15.5:0/703409029 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55db23412d80 0x55db234f6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.400+0000 7f0a89723280 1 --2- 172.21.15.5:0/703409029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db23412900 0x55db234f6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.400+0000 7f0a89723280 1 --2- 172.21.15.5:0/703409029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db23412480 0x55db234f7180 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.400+0000 7f0a89723280 1 -- 172.21.15.5:0/703409029 >> 172.21.15.5:0/703409029 conn(0x55db23413200 msgr2=0x55db23485800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:23.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.400+0000 7f0a89723280 1 -- 172.21.15.5:0/703409029 shutdown_connections 2024-05-20T18:33:23.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.400+0000 7f0a89723280 1 -- 172.21.15.5:0/703409029 wait complete. 2024-05-20T18:33:23.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.400+0000 7f0a89723280 1 -- 172.21.15.5:0/5059369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db23413b00 msgr2=0x55db234f4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:23.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.400+0000 7f0a89723280 1 --2- 172.21.15.5:0/5059369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db23413b00 0x55db234f4580 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x55db2418aae0 tx=0x55db23460b40 comp rx=0 tx=0).stop 2024-05-20T18:33:23.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.400+0000 7f0a89723280 1 -- 172.21.15.5:0/5059369 shutdown_connections 2024-05-20T18:33:23.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.400+0000 7f0a89723280 1 --2- 172.21.15.5:0/5059369 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55db23414400 0x55db234f5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.400+0000 7f0a89723280 1 --2- 172.21.15.5:0/5059369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db23413b00 0x55db234f4580 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.400+0000 7f0a89723280 1 --2- 172.21.15.5:0/5059369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55db23413680 0x55db234f4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.400+0000 7f0a89723280 1 -- 172.21.15.5:0/5059369 >> 172.21.15.5:0/5059369 conn(0x55db23413f80 msgr2=0x55db23485000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:23.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.400+0000 7f0a89723280 1 -- 172.21.15.5:0/5059369 shutdown_connections 2024-05-20T18:33:23.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.400+0000 7f0a89723280 1 -- 172.21.15.5:0/5059369 wait complete. 2024-05-20T18:33:23.407 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset'] in time 0:00:00.086088, result: old journal was 4212724~2605 new journal start will be 8388608 (4173279 bytes past old end) writing journal head writing EResetJournal entry done 2024-05-20T18:33:23.407 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-20T18:33:23.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.524+0000 7f3c6f28b640 1 Processor -- start 2024-05-20T18:33:23.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.524+0000 7f3c6f28b640 1 -- start start 2024-05-20T18:33:23.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.525+0000 7f3c6f28b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3c68163a10 0x7f3c680d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.525+0000 7f3c6f28b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c680d14b0 0x7f3c680d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.525+0000 7f3c6f28b640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c680d0370 con 0x7f3c680b4010 2024-05-20T18:33:23.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.525+0000 7f3c6f28b640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f3c680d3ca0 con 0x7f3c68163a10 2024-05-20T18:33:23.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.525+0000 7f3c6f28b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3c680d0650 con 0x7f3c680d14b0 2024-05-20T18:33:23.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.526+0000 7f3c6d027640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3c68163a10 0x7f3c680d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.526+0000 7f3c6d828640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c680d14b0 0x7f3c680d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.526+0000 7f3c6d027640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3c68163a10 0x7f3c680d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:34348/0 (socket says 172.21.15.5:34348) 2024-05-20T18:33:23.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.526+0000 7f3c6d027640 1 -- 172.21.15.5:0/2199245774 learned_addr learned my addr 172.21.15.5:0/2199245774 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:23.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.526+0000 7f3c6d828640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c680d14b0 0x7f3c680d18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:39414/0 (socket says 172.21.15.5:39414) 2024-05-20T18:33:23.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.526+0000 7f3c6d027640 1 -- 172.21.15.5:0/2199245774 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c680d14b0 msgr2=0x7f3c680d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:23.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.526+0000 7f3c6d027640 1 --2- 172.21.15.5:0/2199245774 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c680d14b0 0x7f3c680d18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.526+0000 7f3c6d027640 1 -- 172.21.15.5:0/2199245774 >> v1:172.21.15.63:6789/0 conn(0x7f3c680b4010 legacy=0x7f3c680cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:23.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.526+0000 7f3c6d027640 1 -- 172.21.15.5:0/2199245774 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c680d3650 con 0x7f3c68163a10 2024-05-20T18:33:23.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.526+0000 7f3c6d828640 1 --2- 172.21.15.5:0/2199245774 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c680d14b0 0x7f3c680d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:33:23.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.527+0000 7f3c6d027640 1 --2- 172.21.15.5:0/2199245774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c68163a10 0x7f3c680d4130 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009870 tx=0x7f3c6402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e2460f554230eb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:23.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.527+0000 7f3c5ffff640 1 -- 172.21.15.5:0/2199245774 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c64037070 con 0x7f3c68163a10 2024-05-20T18:33:23.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.527+0000 7f3c5ffff640 1 -- 172.21.15.5:0/2199245774 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c64032440 con 0x7f3c68163a10 2024-05-20T18:33:23.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.527+0000 7f3c5ffff640 1 -- 172.21.15.5:0/2199245774 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c64032810 con 0x7f3c68163a10 2024-05-20T18:33:23.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.528+0000 7f3c6f28b640 1 -- 172.21.15.5:0/2199245774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c68163a10 msgr2=0x7f3c680d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:23.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.528+0000 7f3c6f28b640 1 --2- 172.21.15.5:0/2199245774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c68163a10 0x7f3c680d4130 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009870 tx=0x7f3c6402a250 comp rx=0 tx=0).stop 2024-05-20T18:33:23.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.528+0000 7f3c6f28b640 1 -- 172.21.15.5:0/2199245774 shutdown_connections 2024-05-20T18:33:23.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.528+0000 7f3c6f28b640 1 --2- 172.21.15.5:0/2199245774 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c680d14b0 0x7f3c680d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.528+0000 7f3c6f28b640 1 --2- 172.21.15.5:0/2199245774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c68163a10 0x7f3c680d4130 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.528+0000 7f3c6f28b640 1 -- 172.21.15.5:0/2199245774 >> 172.21.15.5:0/2199245774 conn(0x7f3c6806d240 msgr2=0x7f3c6806d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:23.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.528+0000 7f3c6f28b640 1 -- 172.21.15.5:0/2199245774 shutdown_connections 2024-05-20T18:33:23.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.528+0000 7f3c6f28b640 1 -- 172.21.15.5:0/2199245774 wait complete. 2024-05-20T18:33:23.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.530+0000 7f3c6f28b640 1 Processor -- start 2024-05-20T18:33:23.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.530+0000 7f3c6f28b640 1 -- start start 2024-05-20T18:33:23.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.531+0000 7f3c6f28b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c680b4010 0x7f3c681617e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.531+0000 7f3c6f28b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c680d14b0 0x7f3c68155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.531+0000 7f3c6f28b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c68163a10 0x7f3c68156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.531+0000 7f3c6f28b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3c680b8210 con 0x7f3c680d14b0 2024-05-20T18:33:23.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.531+0000 7f3c6f28b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3c680d1e00 con 0x7f3c680b4010 2024-05-20T18:33:23.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.531+0000 7f3c6f28b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3c680b8390 con 0x7f3c68163a10 2024-05-20T18:33:23.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.531+0000 7f3c6c826640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c680d14b0 0x7f3c68155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.531+0000 7f3c6c826640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c680d14b0 0x7f3c68155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59296/0 (socket says 172.21.15.5:59296) 2024-05-20T18:33:23.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.531+0000 7f3c6c826640 1 -- 172.21.15.5:0/2920788167 learned_addr learned my addr 172.21.15.5:0/2920788167 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:23.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.531+0000 7f3c6d027640 1 --2- 172.21.15.5:0/2920788167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c680b4010 0x7f3c681617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.531+0000 7f3c6d828640 1 --2- 172.21.15.5:0/2920788167 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c68163a10 0x7f3c68156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.532+0000 7f3c6c826640 1 -- 172.21.15.5:0/2920788167 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c68163a10 msgr2=0x7f3c68156330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:23.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.532+0000 7f3c6c826640 1 --2- 172.21.15.5:0/2920788167 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c68163a10 0x7f3c68156330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.532+0000 7f3c6c826640 1 -- 172.21.15.5:0/2920788167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c680b4010 msgr2=0x7f3c681617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:23.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.532+0000 7f3c6c826640 1 --2- 172.21.15.5:0/2920788167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c680b4010 0x7f3c681617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:23.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.532+0000 7f3c6c826640 1 -- 172.21.15.5:0/2920788167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c68156870 con 0x7f3c680d14b0 2024-05-20T18:33:23.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.532+0000 7f3c6d027640 1 --2- 172.21.15.5:0/2920788167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c680b4010 0x7f3c681617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:33:23.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.532+0000 7f3c6c826640 1 --2- 172.21.15.5:0/2920788167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c680d14b0 0x7f3c68155df0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f3c5800c3f0 tx=0x7f3c5800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:23.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.532+0000 7f3c5dffb640 1 -- 172.21.15.5:0/2920788167 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c580052d0 con 0x7f3c680d14b0 2024-05-20T18:33:23.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.532+0000 7f3c5dffb640 1 -- 172.21.15.5:0/2920788167 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c58005490 con 0x7f3c680d14b0 2024-05-20T18:33:23.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.532+0000 7f3c6f28b640 1 -- 172.21.15.5:0/2920788167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c68155030 con 0x7f3c680d14b0 2024-05-20T18:33:23.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.532+0000 7f3c5dffb640 1 -- 172.21.15.5:0/2920788167 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c58007260 con 0x7f3c680d14b0 2024-05-20T18:33:23.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.532+0000 7f3c6f28b640 1 -- 172.21.15.5:0/2920788167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c681555f0 con 0x7f3c680d14b0 2024-05-20T18:33:23.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.533+0000 7f3c6f28b640 1 -- 172.21.15.5:0/2920788167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c68161ea0 con 0x7f3c680d14b0 2024-05-20T18:33:23.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.534+0000 7f3c5dffb640 1 -- 172.21.15.5:0/2920788167 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3c58007400 con 0x7f3c680d14b0 2024-05-20T18:33:23.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.534+0000 7f3c5dffb640 1 --2- 172.21.15.5:0/2920788167 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3c4c06be60 0x7f3c4c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:23.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.535+0000 7f3c5dffb640 1 -- 172.21.15.5:0/2920788167 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6187+0+0 (secure 0 0 0) 0x7f3c5808a360 con 0x7f3c680d14b0 2024-05-20T18:33:23.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.535+0000 7f3c6d027640 1 --2- 172.21.15.5:0/2920788167 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3c4c06be60 0x7f3c4c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:23.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.535+0000 7f3c6d027640 1 --2- 172.21.15.5:0/2920788167 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3c4c06be60 0x7f3c4c06e370 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3c64034000 tx=0x7f3c6402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:23.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.539+0000 7f3c5dffb640 1 -- 172.21.15.5:0/2920788167 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3c580584e0 con 0x7f3c680d14b0 2024-05-20T18:33:23.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:23.761+0000 7f3c6f28b640 1 -- 172.21.15.5:0/2920788167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f3c68075b00 con 0x7f3c680d14b0 2024-05-20T18:33:24.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.235+0000 7f3c5dffb640 1 -- 172.21.15.5:0/2920788167 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v62) ==== 162+0+0 (secure 0 0 0) 0x7f3c5805c7e0 con 0x7f3c680d14b0 2024-05-20T18:33:24.250 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-20T18:33:24.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.239+0000 7f3c6f28b640 1 -- 172.21.15.5:0/2920788167 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3c4c06be60 msgr2=0x7f3c4c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:24.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.239+0000 7f3c6f28b640 1 --2- 172.21.15.5:0/2920788167 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3c4c06be60 0x7f3c4c06e370 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3c64034000 tx=0x7f3c6402e000 comp rx=0 tx=0).stop 2024-05-20T18:33:24.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.239+0000 7f3c6f28b640 1 -- 172.21.15.5:0/2920788167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c680d14b0 msgr2=0x7f3c68155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:24.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.239+0000 7f3c6f28b640 1 --2- 172.21.15.5:0/2920788167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c680d14b0 0x7f3c68155df0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f3c5800c3f0 tx=0x7f3c5800c7f0 comp rx=0 tx=0).stop 2024-05-20T18:33:24.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.239+0000 7f3c6f28b640 1 -- 172.21.15.5:0/2920788167 shutdown_connections 2024-05-20T18:33:24.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.239+0000 7f3c6f28b640 1 --2- 172.21.15.5:0/2920788167 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3c4c06be60 0x7f3c4c06e370 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:24.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.239+0000 7f3c6f28b640 1 --2- 172.21.15.5:0/2920788167 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c68163a10 0x7f3c68156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:24.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.239+0000 7f3c6f28b640 1 --2- 172.21.15.5:0/2920788167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c680d14b0 0x7f3c68155df0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:24.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.239+0000 7f3c6f28b640 1 --2- 172.21.15.5:0/2920788167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c680b4010 0x7f3c681617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:24.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.239+0000 7f3c6f28b640 1 -- 172.21.15.5:0/2920788167 >> 172.21.15.5:0/2920788167 conn(0x7f3c6806d240 msgr2=0x7f3c6815b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:24.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.239+0000 7f3c6f28b640 1 -- 172.21.15.5:0/2920788167 shutdown_connections 2024-05-20T18:33:24.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.239+0000 7f3c6f28b640 1 -- 172.21.15.5:0/2920788167 wait complete. 2024-05-20T18:33:24.256 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:33:24.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.378+0000 7ff93ba51640 1 Processor -- start 2024-05-20T18:33:24.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.378+0000 7ff93ba51640 1 -- start start 2024-05-20T18:33:24.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.379+0000 7ff93ba51640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff93415f220 0x7ff9340cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:24.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.379+0000 7ff93ba51640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9340cfd20 con 0x7ff9340cd680 2024-05-20T18:33:24.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.379+0000 7ff93ba51640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff93406d0f0 con 0x7ff9340cf910 2024-05-20T18:33:24.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.379+0000 7ff93ba51640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff9340cda90 con 0x7ff93415f220 2024-05-20T18:33:24.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.379+0000 7ff9397ed640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff9340cd680 0x7ff93406c6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35534/0 (socket says 172.21.15.5:35534) 2024-05-20T18:33:24.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.379+0000 7ff9397ed640 1 -- 172.21.15.5:0/293943836 learned_addr learned my addr 172.21.15.5:0/293943836 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:33:24.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.380+0000 7ff938fec640 1 --2- 172.21.15.5:0/293943836 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff93415f220 0x7ff9340cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:24.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.380+0000 7ff92bfff640 1 -- 172.21.15.5:0/293943836 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3055704172 0 0) 0x7ff930001110 con 0x7ff9340cd680 2024-05-20T18:33:24.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.380+0000 7ff92bfff640 1 -- 172.21.15.5:0/293943836 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff920003520 con 0x7ff9340cd680 2024-05-20T18:33:24.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.380+0000 7ff92bfff640 1 -- 172.21.15.5:0/293943836 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2655607339 0 0) 0x7ff920003520 con 0x7ff9340cd680 2024-05-20T18:33:24.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.380+0000 7ff92bfff640 1 -- 172.21.15.5:0/293943836 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff93415f220 msgr2=0x7ff9340cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:24.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.380+0000 7ff92bfff640 1 --2- 172.21.15.5:0/293943836 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff93415f220 0x7ff9340cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:24.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.380+0000 7ff92bfff640 1 -- 172.21.15.5:0/293943836 >> v1:172.21.15.63:6789/0 conn(0x7ff9340cf910 legacy=0x7ff9340d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:24.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.380+0000 7ff92bfff640 1 -- 172.21.15.5:0/293943836 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff93406d3d0 con 0x7ff9340cd680 2024-05-20T18:33:24.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.380+0000 7ff92bfff640 1 -- 172.21.15.5:0/293943836 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7ff930001840 con 0x7ff9340cd680 2024-05-20T18:33:24.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.380+0000 7ff92bfff640 1 -- 172.21.15.5:0/293943836 >> v1:172.21.15.5:6789/0 conn(0x7ff9340cd680 legacy=0x7ff93406c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:24.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.380+0000 7ff92bfff640 1 --2- 172.21.15.5:0/293943836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9200039e0 0x7ff920003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:24.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.380+0000 7ff92bfff640 1 --2- 172.21.15.5:0/293943836 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff93415f220 0x7ff920004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:24.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.380+0000 7ff92bfff640 1 --2- 172.21.15.5:0/293943836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff920004d40 0x7ff920005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:24.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.380+0000 7ff92bfff640 1 -- 172.21.15.5:0/293943836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff92000bb50 con 0x7ff9200039e0 2024-05-20T18:33:24.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.380+0000 7ff92bfff640 1 -- 172.21.15.5:0/293943836 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff92000bcd0 con 0x7ff920004d40 2024-05-20T18:33:24.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.380+0000 7ff92bfff640 1 -- 172.21.15.5:0/293943836 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff92000bea0 con 0x7ff93415f220 2024-05-20T18:33:24.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.380+0000 7ff9397ed640 1 --2- 172.21.15.5:0/293943836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9200039e0 0x7ff920003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:24.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.380+0000 7ff938fec640 1 --2- 172.21.15.5:0/293943836 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff93415f220 0x7ff920004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:24.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.380+0000 7ff939fee640 1 --2- 172.21.15.5:0/293943836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff920004d40 0x7ff920005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:24.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.381+0000 7ff9397ed640 1 -- 172.21.15.5:0/293943836 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff93415f220 msgr2=0x7ff920004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:24.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.381+0000 7ff9397ed640 1 --2- 172.21.15.5:0/293943836 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff93415f220 0x7ff920004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:24.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.381+0000 7ff9397ed640 1 -- 172.21.15.5:0/293943836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff920004d40 msgr2=0x7ff920005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:24.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.381+0000 7ff9397ed640 1 --2- 172.21.15.5:0/293943836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff920004d40 0x7ff920005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:24.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.381+0000 7ff9397ed640 1 -- 172.21.15.5:0/293943836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff920003520 con 0x7ff9200039e0 2024-05-20T18:33:24.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.381+0000 7ff939fee640 1 --2- 172.21.15.5:0/293943836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff920004d40 0x7ff920005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:24.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.381+0000 7ff9397ed640 1 --2- 172.21.15.5:0/293943836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9200039e0 0x7ff920003e10 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7ff93000c2f0 tx=0x7ff920005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bfbb3a823b2efa1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:24.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.382+0000 7ff92bfff640 1 -- 172.21.15.5:0/293943836 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff930035070 con 0x7ff9200039e0 2024-05-20T18:33:24.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.382+0000 7ff92bfff640 1 -- 172.21.15.5:0/293943836 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9300058d0 con 0x7ff9200039e0 2024-05-20T18:33:24.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.382+0000 7ff92bfff640 1 -- 172.21.15.5:0/293943836 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff930007e60 con 0x7ff9200039e0 2024-05-20T18:33:24.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.383+0000 7ff93ba51640 1 -- 172.21.15.5:0/293943836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9200039e0 msgr2=0x7ff920003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:24.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.383+0000 7ff93ba51640 1 --2- 172.21.15.5:0/293943836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9200039e0 0x7ff920003e10 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7ff93000c2f0 tx=0x7ff920005830 comp rx=0 tx=0).stop 2024-05-20T18:33:24.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.383+0000 7ff939fee640 1 -- 172.21.15.5:0/293943836 reap_dead start 2024-05-20T18:33:24.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.383+0000 7ff93ba51640 1 -- 172.21.15.5:0/293943836 shutdown_connections 2024-05-20T18:33:24.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.383+0000 7ff93ba51640 1 -- 172.21.15.5:0/293943836 >> 172.21.15.5:0/293943836 conn(0x7ff9340b44d0 msgr2=0x7ff9340b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:24.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.383+0000 7ff93ba51640 1 -- 172.21.15.5:0/293943836 shutdown_connections 2024-05-20T18:33:24.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.383+0000 7ff93ba51640 1 -- 172.21.15.5:0/293943836 wait complete. 2024-05-20T18:33:24.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.385+0000 7ff93ba51640 1 Processor -- start 2024-05-20T18:33:24.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.385+0000 7ff93ba51640 1 -- start start 2024-05-20T18:33:24.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.386+0000 7ff93ba51640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9200039e0 0x7ff9340b7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:24.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.386+0000 7ff93ba51640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9341579d0 0x7ff9340d15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:24.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.386+0000 7ff93ba51640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff934157de0 0x7ff9340d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:24.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.386+0000 7ff93ba51640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff93406d990 con 0x7ff934157de0 2024-05-20T18:33:24.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.386+0000 7ff93ba51640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff9340d0b40 con 0x7ff9200039e0 2024-05-20T18:33:24.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.386+0000 7ff93ba51640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff93406cdd0 con 0x7ff9341579d0 2024-05-20T18:33:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.386+0000 7ff939fee640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff934157de0 0x7ff9340d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.386+0000 7ff939fee640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff934157de0 0x7ff9340d1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59318/0 (socket says 172.21.15.5:59318) 2024-05-20T18:33:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.386+0000 7ff939fee640 1 -- 172.21.15.5:0/327316638 learned_addr learned my addr 172.21.15.5:0/327316638 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.386+0000 7ff938fec640 1 --2- 172.21.15.5:0/327316638 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9341579d0 0x7ff9340d15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.386+0000 7ff9397ed640 1 --2- 172.21.15.5:0/327316638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9200039e0 0x7ff9340b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.386+0000 7ff939fee640 1 -- 172.21.15.5:0/327316638 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9341579d0 msgr2=0x7ff9340d15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.386+0000 7ff939fee640 1 --2- 172.21.15.5:0/327316638 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9341579d0 0x7ff9340d15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.386+0000 7ff939fee640 1 -- 172.21.15.5:0/327316638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9200039e0 msgr2=0x7ff9340b7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.386+0000 7ff939fee640 1 --2- 172.21.15.5:0/327316638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9200039e0 0x7ff9340b7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.386+0000 7ff939fee640 1 -- 172.21.15.5:0/327316638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9341623b0 con 0x7ff934157de0 2024-05-20T18:33:24.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.386+0000 7ff938fec640 1 --2- 172.21.15.5:0/327316638 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9341579d0 0x7ff9340d15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:24.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.387+0000 7ff939fee640 1 --2- 172.21.15.5:0/327316638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff934157de0 0x7ff9340d1af0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7ff93406e3e0 tx=0x7ff92c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:24.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.387+0000 7ff929ffb640 1 -- 172.21.15.5:0/327316638 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c003b80 con 0x7ff934157de0 2024-05-20T18:33:24.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.387+0000 7ff929ffb640 1 -- 172.21.15.5:0/327316638 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff92c003d40 con 0x7ff934157de0 2024-05-20T18:33:24.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.387+0000 7ff929ffb640 1 -- 172.21.15.5:0/327316638 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c006b50 con 0x7ff934157de0 2024-05-20T18:33:24.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.387+0000 7ff93ba51640 1 -- 172.21.15.5:0/327316638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9341f5470 con 0x7ff934157de0 2024-05-20T18:33:24.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.387+0000 7ff93ba51640 1 -- 172.21.15.5:0/327316638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9341f5a10 con 0x7ff934157de0 2024-05-20T18:33:24.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.388+0000 7ff93ba51640 1 -- 172.21.15.5:0/327316638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9340c5ef0 con 0x7ff934157de0 2024-05-20T18:33:24.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.389+0000 7ff929ffb640 1 -- 172.21.15.5:0/327316638 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff92c01a020 con 0x7ff934157de0 2024-05-20T18:33:24.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.389+0000 7ff929ffb640 1 --2- 172.21.15.5:0/327316638 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff91806be30 0x7ff91806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:24.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.389+0000 7ff929ffb640 1 -- 172.21.15.5:0/327316638 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6187+0+0 (secure 0 0 0) 0x7ff92c08a930 con 0x7ff934157de0 2024-05-20T18:33:24.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.389+0000 7ff9397ed640 1 --2- 172.21.15.5:0/327316638 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff91806be30 0x7ff91806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:24.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.390+0000 7ff9397ed640 1 --2- 172.21.15.5:0/327316638 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff91806be30 0x7ff91806e340 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff930032000 tx=0x7ff930009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:24.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.394+0000 7ff929ffb640 1 -- 172.21.15.5:0/327316638 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff9340c5ef0 con 0x7ff934157de0 2024-05-20T18:33:24.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.613+0000 7ff93ba51640 1 -- 172.21.15.5:0/327316638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9341f63b0 con 0x7ff934157de0 2024-05-20T18:33:24.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.614+0000 7ff929ffb640 1 -- 172.21.15.5:0/327316638 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+5165 (secure 0 0 0) 0x7ff9341623b0 con 0x7ff934157de0 2024-05-20T18:33:24.615 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:33:24.615 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/337248865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":337248865},{"type":"v1","addr":"172.21.15.63:6834","nonce":337248865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3705772553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3705772553},{"type":"v1","addr":"172.21.15.63:6835","nonce":3705772553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":7434,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/776188138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":776188138},{"type":"v1","addr":"172.21.15.5:6835","nonce":776188138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:33:12.457716+0000","modified":"2024-05-20T18:33:24.240050+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"c","rank":0,"incarnation":63,"state":"up:replay","state_seq":1,"addr":"172.21.15.5:6837/3556492262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3556492262},{"type":"v1","addr":"172.21.15.5:6837","nonce":3556492262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-05-20T18:33:24.615 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 63 2024-05-20T18:33:24.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.616+0000 7ff93ba51640 1 -- 172.21.15.5:0/327316638 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff91806be30 msgr2=0x7ff91806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:24.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.616+0000 7ff93ba51640 1 --2- 172.21.15.5:0/327316638 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff91806be30 0x7ff91806e340 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff930032000 tx=0x7ff930009030 comp rx=0 tx=0).stop 2024-05-20T18:33:24.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.616+0000 7ff93ba51640 1 -- 172.21.15.5:0/327316638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff934157de0 msgr2=0x7ff9340d1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:24.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.616+0000 7ff93ba51640 1 --2- 172.21.15.5:0/327316638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff934157de0 0x7ff9340d1af0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7ff93406e3e0 tx=0x7ff92c00ecd0 comp rx=0 tx=0).stop 2024-05-20T18:33:24.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.616+0000 7ff93ba51640 1 -- 172.21.15.5:0/327316638 shutdown_connections 2024-05-20T18:33:24.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.616+0000 7ff93ba51640 1 --2- 172.21.15.5:0/327316638 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff91806be30 0x7ff91806e340 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:24.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.616+0000 7ff93ba51640 1 --2- 172.21.15.5:0/327316638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff934157de0 0x7ff9340d1af0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:24.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.616+0000 7ff93ba51640 1 --2- 172.21.15.5:0/327316638 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9341579d0 0x7ff9340d15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:24.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.616+0000 7ff93ba51640 1 --2- 172.21.15.5:0/327316638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9200039e0 0x7ff9340b7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:24.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.616+0000 7ff93ba51640 1 -- 172.21.15.5:0/327316638 >> 172.21.15.5:0/327316638 conn(0x7ff9340b44d0 msgr2=0x7ff9340b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:24.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.616+0000 7ff93ba51640 1 -- 172.21.15.5:0/327316638 shutdown_connections 2024-05-20T18:33:24.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:24.616+0000 7ff93ba51640 1 -- 172.21.15.5:0/327316638 wait complete. 2024-05-20T18:33:25.030 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 63, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T18:33:12.457716+0000', 'modified': '2024-05-20T18:33:24.240050+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 91, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7437}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7437': {'gid': 7437, 'name': 'c', 'rank': 0, 'incarnation': 63, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.5:6837/3556492262', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6836', 'nonce': 3556492262}, {'type': 'v1', 'addr': '172.21.15.5:6837', 'nonce': 3556492262}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-20T18:33:25.031 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7437:up:replay 2024-05-20T18:33:26.031 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:33:26.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.160+0000 7f604a3e3640 1 Processor -- start 2024-05-20T18:33:26.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.160+0000 7f604a3e3640 1 -- start start 2024-05-20T18:33:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.162+0000 7f604a3e3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6044162820 0x7f60440b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.162+0000 7f604a3e3640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60440d0630 con 0x7f60440b3e80 2024-05-20T18:33:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.162+0000 7f604a3e3640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60440d39c0 con 0x7f604406dae0 2024-05-20T18:33:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.162+0000 7f604a3e3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f60440d3ca0 con 0x7f6044162820 2024-05-20T18:33:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.162+0000 7f60437fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f60440b3e80 0x7f604406d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47400/0 (socket says 172.21.15.5:47400) 2024-05-20T18:33:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.162+0000 7f60437fe640 1 -- 172.21.15.5:0/1114742723 learned_addr learned my addr 172.21.15.5:0/1114742723 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:33:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.162+0000 7f6043fff640 1 --2- 172.21.15.5:0/1114742723 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6044162820 0x7f60440b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.162+0000 7f6042ffd640 1 -- 172.21.15.5:0/1114742723 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1648102370 0 0) 0x7f60440d0630 con 0x7f60440b3e80 2024-05-20T18:33:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.162+0000 7f6042ffd640 1 -- 172.21.15.5:0/1114742723 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f602c003520 con 0x7f60440b3e80 2024-05-20T18:33:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.163+0000 7f6042ffd640 1 -- 172.21.15.5:0/1114742723 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3548047012 0 0) 0x7f60440d39c0 con 0x7f604406dae0 2024-05-20T18:33:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.163+0000 7f6042ffd640 1 -- 172.21.15.5:0/1114742723 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60440d0630 con 0x7f604406dae0 2024-05-20T18:33:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.163+0000 7f6042ffd640 1 -- 172.21.15.5:0/1114742723 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3728093123 0 0) 0x7f602c003520 con 0x7f60440b3e80 2024-05-20T18:33:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.163+0000 7f6042ffd640 1 -- 172.21.15.5:0/1114742723 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6044162820 msgr2=0x7f60440b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.163+0000 7f6042ffd640 1 --2- 172.21.15.5:0/1114742723 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6044162820 0x7f60440b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.163+0000 7f6042ffd640 1 -- 172.21.15.5:0/1114742723 >> v1:172.21.15.63:6789/0 conn(0x7f604406dae0 legacy=0x7f60440b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.163+0000 7f6042ffd640 1 -- 172.21.15.5:0/1114742723 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60440d3f80 con 0x7f60440b3e80 2024-05-20T18:33:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.163+0000 7f6042ffd640 1 -- 172.21.15.5:0/1114742723 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f6038001670 con 0x7f60440b3e80 2024-05-20T18:33:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.163+0000 7f6042ffd640 1 -- 172.21.15.5:0/1114742723 >> v1:172.21.15.5:6789/0 conn(0x7f60440b3e80 legacy=0x7f604406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.163+0000 7f6042ffd640 1 --2- 172.21.15.5:0/1114742723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f602c003b70 0x7f602c003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.163+0000 7f6042ffd640 1 --2- 172.21.15.5:0/1114742723 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6044162820 0x7f602c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.163+0000 7f6042ffd640 1 --2- 172.21.15.5:0/1114742723 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f602c004d40 0x7f602c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.163+0000 7f6042ffd640 1 -- 172.21.15.5:0/1114742723 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f60440d3ca0 con 0x7f602c003b70 2024-05-20T18:33:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.163+0000 7f6042ffd640 1 -- 172.21.15.5:0/1114742723 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f602c00bb50 con 0x7f602c004d40 2024-05-20T18:33:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.163+0000 7f6042ffd640 1 -- 172.21.15.5:0/1114742723 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f602c00bd20 con 0x7f6044162820 2024-05-20T18:33:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.163+0000 7f6043fff640 1 --2- 172.21.15.5:0/1114742723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f602c003b70 0x7f602c003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.163+0000 7f60437fe640 1 --2- 172.21.15.5:0/1114742723 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6044162820 0x7f602c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.163+0000 7f6048980640 1 --2- 172.21.15.5:0/1114742723 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f602c004d40 0x7f602c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.163+0000 7f6043fff640 1 -- 172.21.15.5:0/1114742723 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6044162820 msgr2=0x7f602c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.163+0000 7f6043fff640 1 --2- 172.21.15.5:0/1114742723 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6044162820 0x7f602c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.163+0000 7f6043fff640 1 -- 172.21.15.5:0/1114742723 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f602c004d40 msgr2=0x7f602c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.163+0000 7f6043fff640 1 --2- 172.21.15.5:0/1114742723 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f602c004d40 0x7f602c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.163+0000 7f6043fff640 1 -- 172.21.15.5:0/1114742723 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f602c003520 con 0x7f602c003b70 2024-05-20T18:33:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.163+0000 7f60437fe640 1 --2- 172.21.15.5:0/1114742723 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6044162820 0x7f602c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.164+0000 7f6043fff640 1 --2- 172.21.15.5:0/1114742723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f602c003b70 0x7f602c003fc0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f6034009ac0 tx=0x7f602c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0d2eb470b837da server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.164+0000 7f6042ffd640 1 -- 172.21.15.5:0/1114742723 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6034036070 con 0x7f602c003b70 2024-05-20T18:33:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.164+0000 7f6042ffd640 1 -- 172.21.15.5:0/1114742723 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6034031400 con 0x7f602c003b70 2024-05-20T18:33:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.164+0000 7f6042ffd640 1 -- 172.21.15.5:0/1114742723 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6034009690 con 0x7f602c003b70 2024-05-20T18:33:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.165+0000 7f604a3e3640 1 -- 172.21.15.5:0/1114742723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f602c003b70 msgr2=0x7f602c003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.165+0000 7f604a3e3640 1 --2- 172.21.15.5:0/1114742723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f602c003b70 0x7f602c003fc0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f6034009ac0 tx=0x7f602c005830 comp rx=0 tx=0).stop 2024-05-20T18:33:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.165+0000 7f6048980640 1 -- 172.21.15.5:0/1114742723 reap_dead start 2024-05-20T18:33:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.165+0000 7f604a3e3640 1 -- 172.21.15.5:0/1114742723 shutdown_connections 2024-05-20T18:33:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.165+0000 7f604a3e3640 1 -- 172.21.15.5:0/1114742723 >> 172.21.15.5:0/1114742723 conn(0x7f60440d4bd0 msgr2=0x7f60440d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.165+0000 7f604a3e3640 1 -- 172.21.15.5:0/1114742723 shutdown_connections 2024-05-20T18:33:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.165+0000 7f604a3e3640 1 -- 172.21.15.5:0/1114742723 wait complete. 2024-05-20T18:33:26.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.167+0000 7f604a3e3640 1 Processor -- start 2024-05-20T18:33:26.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.167+0000 7f604a3e3640 1 -- start start 2024-05-20T18:33:26.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.168+0000 7f604a3e3640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f602c003b70 0x7f6044155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:26.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.168+0000 7f604a3e3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60440b7030 0x7f6044157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:26.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.168+0000 7f604a3e3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6044157960 0x7f60440b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:26.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.168+0000 7f604a3e3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f60440d4780 con 0x7f60440b7030 2024-05-20T18:33:26.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.168+0000 7f604a3e3640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f60440d0300 con 0x7f602c003b70 2024-05-20T18:33:26.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.168+0000 7f604a3e3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f60440d4900 con 0x7f6044157960 2024-05-20T18:33:26.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.168+0000 7f60437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60440b7030 0x7f6044157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:26.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.169+0000 7f60437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60440b7030 0x7f6044157420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54856/0 (socket says 172.21.15.5:54856) 2024-05-20T18:33:26.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.169+0000 7f60437fe640 1 -- 172.21.15.5:0/1859233430 learned_addr learned my addr 172.21.15.5:0/1859233430 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:26.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.169+0000 7f6048980640 1 --2- 172.21.15.5:0/1859233430 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6044157960 0x7f60440b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:26.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.169+0000 7f6043fff640 1 --2- 172.21.15.5:0/1859233430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f602c003b70 0x7f6044155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:26.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.169+0000 7f60437fe640 1 -- 172.21.15.5:0/1859233430 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6044157960 msgr2=0x7f60440b46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.169+0000 7f60437fe640 1 --2- 172.21.15.5:0/1859233430 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6044157960 0x7f60440b46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.169+0000 7f60437fe640 1 -- 172.21.15.5:0/1859233430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f602c003b70 msgr2=0x7f6044155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.169+0000 7f60437fe640 1 --2- 172.21.15.5:0/1859233430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f602c003b70 0x7f6044155400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.169+0000 7f60437fe640 1 -- 172.21.15.5:0/1859233430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60440cc3e0 con 0x7f60440b7030 2024-05-20T18:33:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.169+0000 7f6043fff640 1 --2- 172.21.15.5:0/1859233430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f602c003b70 0x7f6044155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.169+0000 7f60437fe640 1 --2- 172.21.15.5:0/1859233430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60440b7030 0x7f6044157420 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f603800e760 tx=0x7f603800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.169+0000 7f6040ff9640 1 -- 172.21.15.5:0/1859233430 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603800ee70 con 0x7f60440b7030 2024-05-20T18:33:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.169+0000 7f6040ff9640 1 -- 172.21.15.5:0/1859233430 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6038018070 con 0x7f60440b7030 2024-05-20T18:33:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.169+0000 7f604a3e3640 1 -- 172.21.15.5:0/1859233430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6044158530 con 0x7f60440b7030 2024-05-20T18:33:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.169+0000 7f6040ff9640 1 -- 172.21.15.5:0/1859233430 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603800ca70 con 0x7f60440b7030 2024-05-20T18:33:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.169+0000 7f604a3e3640 1 -- 172.21.15.5:0/1859233430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6044158af0 con 0x7f60440b7030 2024-05-20T18:33:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.171+0000 7f604a3e3640 1 -- 172.21.15.5:0/1859233430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6044158cf0 con 0x7f60440b7030 2024-05-20T18:33:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.171+0000 7f6040ff9640 1 -- 172.21.15.5:0/1859233430 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f603800cc10 con 0x7f60440b7030 2024-05-20T18:33:26.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.171+0000 7f6040ff9640 1 --2- 172.21.15.5:0/1859233430 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f603006be30 0x7f603006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:26.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.171+0000 7f6040ff9640 1 -- 172.21.15.5:0/1859233430 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6187+0+0 (secure 0 0 0) 0x7f603808add0 con 0x7f60440b7030 2024-05-20T18:33:26.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.172+0000 7f6043fff640 1 --2- 172.21.15.5:0/1859233430 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f603006be30 0x7f603006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:26.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.172+0000 7f6043fff640 1 --2- 172.21.15.5:0/1859233430 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f603006be30 0x7f603006e340 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6034033000 tx=0x7f6034008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:26.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.176+0000 7f6040ff9640 1 -- 172.21.15.5:0/1859233430 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f60380595b0 con 0x7f60440b7030 2024-05-20T18:33:26.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.400+0000 7f604a3e3640 1 -- 172.21.15.5:0/1859233430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60441582c0 con 0x7f60440b7030 2024-05-20T18:33:26.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.401+0000 7f6040ff9640 1 -- 172.21.15.5:0/1859233430 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5165 (secure 0 0 0) 0x7f603808c090 con 0x7f60440b7030 2024-05-20T18:33:26.401 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:33:26.402 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/337248865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":337248865},{"type":"v1","addr":"172.21.15.63:6834","nonce":337248865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3705772553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3705772553},{"type":"v1","addr":"172.21.15.63:6835","nonce":3705772553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":7434,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/776188138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":776188138},{"type":"v1","addr":"172.21.15.5:6835","nonce":776188138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:33:12.457716+0000","modified":"2024-05-20T18:33:25.250124+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"c","rank":0,"incarnation":63,"state":"up:rejoin","state_seq":7,"addr":"172.21.15.5:6837/3556492262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3556492262},{"type":"v1","addr":"172.21.15.5:6837","nonce":3556492262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-05-20T18:33:26.402 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 65 2024-05-20T18:33:26.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.402+0000 7f604a3e3640 1 -- 172.21.15.5:0/1859233430 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f603006be30 msgr2=0x7f603006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:26.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.402+0000 7f604a3e3640 1 --2- 172.21.15.5:0/1859233430 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f603006be30 0x7f603006e340 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6034033000 tx=0x7f6034008030 comp rx=0 tx=0).stop 2024-05-20T18:33:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.402+0000 7f604a3e3640 1 -- 172.21.15.5:0/1859233430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60440b7030 msgr2=0x7f6044157420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.402+0000 7f604a3e3640 1 --2- 172.21.15.5:0/1859233430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60440b7030 0x7f6044157420 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f603800e760 tx=0x7f603800ec20 comp rx=0 tx=0).stop 2024-05-20T18:33:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.403+0000 7f604a3e3640 1 -- 172.21.15.5:0/1859233430 shutdown_connections 2024-05-20T18:33:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.403+0000 7f604a3e3640 1 --2- 172.21.15.5:0/1859233430 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f603006be30 0x7f603006e340 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.403+0000 7f604a3e3640 1 --2- 172.21.15.5:0/1859233430 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6044157960 0x7f60440b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.403+0000 7f604a3e3640 1 --2- 172.21.15.5:0/1859233430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60440b7030 0x7f6044157420 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.403+0000 7f604a3e3640 1 --2- 172.21.15.5:0/1859233430 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f602c003b70 0x7f6044155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.403+0000 7f604a3e3640 1 -- 172.21.15.5:0/1859233430 >> 172.21.15.5:0/1859233430 conn(0x7f60440d4bd0 msgr2=0x7f60441e7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.403+0000 7f604a3e3640 1 -- 172.21.15.5:0/1859233430 shutdown_connections 2024-05-20T18:33:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:26.403+0000 7f604a3e3640 1 -- 172.21.15.5:0/1859233430 wait complete. 2024-05-20T18:33:26.817 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T18:33:12.457716+0000', 'modified': '2024-05-20T18:33:25.250124+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 91, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7437}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7437': {'gid': 7437, 'name': 'c', 'rank': 0, 'incarnation': 63, 'state': 'up:rejoin', 'state_seq': 7, 'addr': '172.21.15.5:6837/3556492262', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6836', 'nonce': 3556492262}, {'type': 'v1', 'addr': '172.21.15.5:6837', 'nonce': 3556492262}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-20T18:33:26.818 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7437:up:rejoin 2024-05-20T18:33:27.819 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:33:27.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.948+0000 7fc9cba61640 1 Processor -- start 2024-05-20T18:33:27.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.948+0000 7fc9cba61640 1 -- start start 2024-05-20T18:33:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.949+0000 7fc9cba61640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9c415f220 0x7fc9c40cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:27.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.949+0000 7fc9cba61640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9c40cfd20 con 0x7fc9c40cd680 2024-05-20T18:33:27.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.949+0000 7fc9cba61640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9c406d0f0 con 0x7fc9c40cf910 2024-05-20T18:33:27.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.949+0000 7fc9cba61640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc9c40cda90 con 0x7fc9c415f220 2024-05-20T18:33:27.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.949+0000 7fc9c97fd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc9c40cd680 0x7fc9c406c6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47404/0 (socket says 172.21.15.5:47404) 2024-05-20T18:33:27.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.949+0000 7fc9c97fd640 1 -- 172.21.15.5:0/1388141812 learned_addr learned my addr 172.21.15.5:0/1388141812 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:33:27.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.950+0000 7fc9c8ffc640 1 --2- 172.21.15.5:0/1388141812 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9c415f220 0x7fc9c40cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:27.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.950+0000 7fc9bbfff640 1 -- 172.21.15.5:0/1388141812 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2258516140 0 0) 0x7fc9c0001110 con 0x7fc9c40cd680 2024-05-20T18:33:27.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.950+0000 7fc9bbfff640 1 -- 172.21.15.5:0/1388141812 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9b0003520 con 0x7fc9c40cd680 2024-05-20T18:33:27.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.950+0000 7fc9bbfff640 1 -- 172.21.15.5:0/1388141812 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 804574634 0 0) 0x7fc9b0003520 con 0x7fc9c40cd680 2024-05-20T18:33:27.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.950+0000 7fc9bbfff640 1 -- 172.21.15.5:0/1388141812 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9c415f220 msgr2=0x7fc9c40cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:27.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.950+0000 7fc9bbfff640 1 --2- 172.21.15.5:0/1388141812 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9c415f220 0x7fc9c40cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.950+0000 7fc9bbfff640 1 -- 172.21.15.5:0/1388141812 >> v1:172.21.15.63:6789/0 conn(0x7fc9c40cf910 legacy=0x7fc9c40d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.950+0000 7fc9bbfff640 1 -- 172.21.15.5:0/1388141812 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c406d3d0 con 0x7fc9c40cd680 2024-05-20T18:33:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.950+0000 7fc9bbfff640 1 -- 172.21.15.5:0/1388141812 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7fc9c0001840 con 0x7fc9c40cd680 2024-05-20T18:33:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.950+0000 7fc9bbfff640 1 -- 172.21.15.5:0/1388141812 >> v1:172.21.15.5:6789/0 conn(0x7fc9c40cd680 legacy=0x7fc9c406c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.950+0000 7fc9bbfff640 1 --2- 172.21.15.5:0/1388141812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b00039e0 0x7fc9b0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.950+0000 7fc9bbfff640 1 --2- 172.21.15.5:0/1388141812 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9c415f220 0x7fc9b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.950+0000 7fc9bbfff640 1 --2- 172.21.15.5:0/1388141812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9b0004d40 0x7fc9b0005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.950+0000 7fc9bbfff640 1 -- 172.21.15.5:0/1388141812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b000bb50 con 0x7fc9b00039e0 2024-05-20T18:33:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.950+0000 7fc9bbfff640 1 -- 172.21.15.5:0/1388141812 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b000bcd0 con 0x7fc9b0004d40 2024-05-20T18:33:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.950+0000 7fc9bbfff640 1 -- 172.21.15.5:0/1388141812 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc9b000bea0 con 0x7fc9c415f220 2024-05-20T18:33:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.950+0000 7fc9c97fd640 1 --2- 172.21.15.5:0/1388141812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b00039e0 0x7fc9b0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.950+0000 7fc9c8ffc640 1 --2- 172.21.15.5:0/1388141812 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9c415f220 0x7fc9b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.950+0000 7fc9c9ffe640 1 --2- 172.21.15.5:0/1388141812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9b0004d40 0x7fc9b0005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:27.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.951+0000 7fc9c97fd640 1 -- 172.21.15.5:0/1388141812 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9c415f220 msgr2=0x7fc9b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:27.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.951+0000 7fc9c97fd640 1 --2- 172.21.15.5:0/1388141812 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9c415f220 0x7fc9b0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:27.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.951+0000 7fc9c97fd640 1 -- 172.21.15.5:0/1388141812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9b0004d40 msgr2=0x7fc9b0005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:27.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.951+0000 7fc9c97fd640 1 --2- 172.21.15.5:0/1388141812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9b0004d40 0x7fc9b0005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:27.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.951+0000 7fc9c97fd640 1 -- 172.21.15.5:0/1388141812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b0003520 con 0x7fc9b00039e0 2024-05-20T18:33:27.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.951+0000 7fc9c9ffe640 1 --2- 172.21.15.5:0/1388141812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9b0004d40 0x7fc9b0005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.951+0000 7fc9c97fd640 1 --2- 172.21.15.5:0/1388141812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b00039e0 0x7fc9b0003e10 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000c490 tx=0x7fc9b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b0e51dea613ea2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.952+0000 7fc9bbfff640 1 -- 172.21.15.5:0/1388141812 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c000c0e0 con 0x7fc9b00039e0 2024-05-20T18:33:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.952+0000 7fc9bbfff640 1 -- 172.21.15.5:0/1388141812 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9c0034070 con 0x7fc9b00039e0 2024-05-20T18:33:27.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.952+0000 7fc9bbfff640 1 -- 172.21.15.5:0/1388141812 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0005d10 con 0x7fc9b00039e0 2024-05-20T18:33:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.952+0000 7fc9cba61640 1 -- 172.21.15.5:0/1388141812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b00039e0 msgr2=0x7fc9b0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.952+0000 7fc9cba61640 1 --2- 172.21.15.5:0/1388141812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b00039e0 0x7fc9b0003e10 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000c490 tx=0x7fc9b0005830 comp rx=0 tx=0).stop 2024-05-20T18:33:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.952+0000 7fc9c9ffe640 1 -- 172.21.15.5:0/1388141812 reap_dead start 2024-05-20T18:33:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.952+0000 7fc9cba61640 1 -- 172.21.15.5:0/1388141812 shutdown_connections 2024-05-20T18:33:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.952+0000 7fc9cba61640 1 -- 172.21.15.5:0/1388141812 >> 172.21.15.5:0/1388141812 conn(0x7fc9c40b44d0 msgr2=0x7fc9c40b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.952+0000 7fc9cba61640 1 -- 172.21.15.5:0/1388141812 shutdown_connections 2024-05-20T18:33:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.952+0000 7fc9cba61640 1 -- 172.21.15.5:0/1388141812 wait complete. 2024-05-20T18:33:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.954+0000 7fc9cba61640 1 Processor -- start 2024-05-20T18:33:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.954+0000 7fc9cba61640 1 -- start start 2024-05-20T18:33:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.955+0000 7fc9cba61640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9b00039e0 0x7fc9c4157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.956+0000 7fc9cba61640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9c415fb90 0x7fc9c415ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.956+0000 7fc9cba61640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9c41604e0 0x7fc9c40c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.956+0000 7fc9cba61640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c406d990 con 0x7fc9c41604e0 2024-05-20T18:33:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.956+0000 7fc9cba61640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c40d0b40 con 0x7fc9c415fb90 2024-05-20T18:33:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.956+0000 7fc9cba61640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc9c406cdd0 con 0x7fc9b00039e0 2024-05-20T18:33:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.956+0000 7fc9c9ffe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9c41604e0 0x7fc9c40c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.956+0000 7fc9c9ffe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9c41604e0 0x7fc9c40c7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54872/0 (socket says 172.21.15.5:54872) 2024-05-20T18:33:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.956+0000 7fc9c9ffe640 1 -- 172.21.15.5:0/3430490447 learned_addr learned my addr 172.21.15.5:0/3430490447 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.956+0000 7fc9c97fd640 1 --2- 172.21.15.5:0/3430490447 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9b00039e0 0x7fc9c4157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.956+0000 7fc9c8ffc640 1 --2- 172.21.15.5:0/3430490447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9c415fb90 0x7fc9c415ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.956+0000 7fc9c9ffe640 1 -- 172.21.15.5:0/3430490447 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9b00039e0 msgr2=0x7fc9c4157f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.956+0000 7fc9c9ffe640 1 --2- 172.21.15.5:0/3430490447 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9b00039e0 0x7fc9c4157f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.956+0000 7fc9c9ffe640 1 -- 172.21.15.5:0/3430490447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9c415fb90 msgr2=0x7fc9c415ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.956+0000 7fc9c9ffe640 1 --2- 172.21.15.5:0/3430490447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9c415fb90 0x7fc9c415ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.956+0000 7fc9c9ffe640 1 -- 172.21.15.5:0/3430490447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c40b74d0 con 0x7fc9c41604e0 2024-05-20T18:33:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.956+0000 7fc9c8ffc640 1 --2- 172.21.15.5:0/3430490447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9c415fb90 0x7fc9c415ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.956+0000 7fc9c9ffe640 1 --2- 172.21.15.5:0/3430490447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9c41604e0 0x7fc9c40c7c50 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fc9c40cf320 tx=0x7fc9bc00ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:27.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.956+0000 7fc9b9ffb640 1 -- 172.21.15.5:0/3430490447 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc019070 con 0x7fc9c41604e0 2024-05-20T18:33:27.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.956+0000 7fc9b9ffb640 1 -- 172.21.15.5:0/3430490447 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9bc006b50 con 0x7fc9c41604e0 2024-05-20T18:33:27.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.956+0000 7fc9cba61640 1 -- 172.21.15.5:0/3430490447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9c40b77b0 con 0x7fc9c41604e0 2024-05-20T18:33:27.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.957+0000 7fc9b9ffb640 1 -- 172.21.15.5:0/3430490447 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc003970 con 0x7fc9c41604e0 2024-05-20T18:33:27.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.957+0000 7fc9cba61640 1 -- 172.21.15.5:0/3430490447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9c40b7d70 con 0x7fc9c41604e0 2024-05-20T18:33:27.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.958+0000 7fc9cba61640 1 -- 172.21.15.5:0/3430490447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9c40b7f70 con 0x7fc9c41604e0 2024-05-20T18:33:27.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.958+0000 7fc9b9ffb640 1 -- 172.21.15.5:0/3430490447 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc9bc01c070 con 0x7fc9c41604e0 2024-05-20T18:33:27.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.959+0000 7fc9b9ffb640 1 --2- 172.21.15.5:0/3430490447 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc9a806be60 0x7fc9a806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:27.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.959+0000 7fc9b9ffb640 1 -- 172.21.15.5:0/3430490447 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6187+0+0 (secure 0 0 0) 0x7fc9bc08a300 con 0x7fc9c41604e0 2024-05-20T18:33:27.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.959+0000 7fc9c97fd640 1 --2- 172.21.15.5:0/3430490447 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc9a806be60 0x7fc9a806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:27.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.959+0000 7fc9c97fd640 1 --2- 172.21.15.5:0/3430490447 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc9a806be60 0x7fc9a806e370 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0033000 tx=0x7fc9c0009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:27.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:27.963+0000 7fc9b9ffb640 1 -- 172.21.15.5:0/3430490447 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc9bc058480 con 0x7fc9c41604e0 2024-05-20T18:33:28.064 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-20T18:33:28.063+0000 7fce2507a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T18:33:28.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.188+0000 7fc9cba61640 1 -- 172.21.15.5:0/3430490447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9c4075b00 con 0x7fc9c41604e0 2024-05-20T18:33:28.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.189+0000 7fc9b9ffb640 1 -- 172.21.15.5:0/3430490447 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5172 (secure 0 0 0) 0x7fc9bc05c780 con 0x7fc9c41604e0 2024-05-20T18:33:28.189 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:33:28.190 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":66,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/337248865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":337248865},{"type":"v1","addr":"172.21.15.63:6834","nonce":337248865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3705772553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3705772553},{"type":"v1","addr":"172.21.15.63:6835","nonce":3705772553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":7434,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/776188138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":776188138},{"type":"v1","addr":"172.21.15.5:6835","nonce":776188138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:33:12.457716+0000","modified":"2024-05-20T18:33:27.255910+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":8,"addr":"172.21.15.5:6837/3556492262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3556492262},{"type":"v1","addr":"172.21.15.5:6837","nonce":3556492262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7437,"qdb_cluster":[7437]},"id":6}]} 2024-05-20T18:33:28.190 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 66 2024-05-20T18:33:28.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.191+0000 7fc9cba61640 1 -- 172.21.15.5:0/3430490447 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc9a806be60 msgr2=0x7fc9a806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:28.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.191+0000 7fc9cba61640 1 --2- 172.21.15.5:0/3430490447 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc9a806be60 0x7fc9a806e370 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0033000 tx=0x7fc9c0009030 comp rx=0 tx=0).stop 2024-05-20T18:33:28.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.191+0000 7fc9cba61640 1 -- 172.21.15.5:0/3430490447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9c41604e0 msgr2=0x7fc9c40c7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:28.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.191+0000 7fc9cba61640 1 --2- 172.21.15.5:0/3430490447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9c41604e0 0x7fc9c40c7c50 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fc9c40cf320 tx=0x7fc9bc00ebe0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.191+0000 7fc9cba61640 1 -- 172.21.15.5:0/3430490447 shutdown_connections 2024-05-20T18:33:28.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.191+0000 7fc9cba61640 1 --2- 172.21.15.5:0/3430490447 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc9a806be60 0x7fc9a806e370 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.191+0000 7fc9cba61640 1 --2- 172.21.15.5:0/3430490447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9c41604e0 0x7fc9c40c7c50 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.191+0000 7fc9cba61640 1 --2- 172.21.15.5:0/3430490447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9c415fb90 0x7fc9c415ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.191+0000 7fc9cba61640 1 --2- 172.21.15.5:0/3430490447 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9b00039e0 0x7fc9c4157f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.191+0000 7fc9cba61640 1 -- 172.21.15.5:0/3430490447 >> 172.21.15.5:0/3430490447 conn(0x7fc9c40b44d0 msgr2=0x7fc9c40ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:28.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.191+0000 7fc9cba61640 1 -- 172.21.15.5:0/3430490447 shutdown_connections 2024-05-20T18:33:28.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.191+0000 7fc9cba61640 1 -- 172.21.15.5:0/3430490447 wait complete. 2024-05-20T18:33:28.607 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T18:33:12.457716+0000', 'modified': '2024-05-20T18:33:27.255910+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 91, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7437}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7437': {'gid': 7437, 'name': 'c', 'rank': 0, 'incarnation': 63, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.5:6837/3556492262', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6836', 'nonce': 3556492262}, {'type': 'v1', 'addr': '172.21.15.5:6837', 'nonce': 3556492262}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7437, 'qdb_cluster': [7437]} 2024-05-20T18:33:28.607 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-20T18:33:28.607 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-20T18:33:28.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.726+0000 7ff63df38640 1 Processor -- start 2024-05-20T18:33:28.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.726+0000 7ff63df38640 1 -- start start 2024-05-20T18:33:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.728+0000 7ff63df38640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff63806d3d0 0x7ff63806d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.728+0000 7ff63df38640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff63806dd20 con 0x7ff638162820 2024-05-20T18:33:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.728+0000 7ff63df38640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6380d0630 con 0x7ff6380b8190 2024-05-20T18:33:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.728+0000 7ff63df38640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff6380d39c0 con 0x7ff63806d3d0 2024-05-20T18:33:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.728+0000 7ff6377fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff638162820 0x7ff6380b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47414/0 (socket says 172.21.15.5:47414) 2024-05-20T18:33:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.728+0000 7ff6377fe640 1 -- 172.21.15.5:0/1287233707 learned_addr learned my addr 172.21.15.5:0/1287233707 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:33:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.728+0000 7ff636ffd640 1 --2- 172.21.15.5:0/1287233707 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff63806d3d0 0x7ff63806d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.728+0000 7ff6367fc640 1 -- 172.21.15.5:0/1287233707 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3851343799 0 0) 0x7ff628001110 con 0x7ff638162820 2024-05-20T18:33:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.728+0000 7ff6367fc640 1 -- 172.21.15.5:0/1287233707 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff620003520 con 0x7ff638162820 2024-05-20T18:33:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.729+0000 7ff6367fc640 1 -- 172.21.15.5:0/1287233707 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 646616108 0 0) 0x7ff620003520 con 0x7ff638162820 2024-05-20T18:33:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.729+0000 7ff6367fc640 1 -- 172.21.15.5:0/1287233707 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff63806d3d0 msgr2=0x7ff63806d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.729+0000 7ff6367fc640 1 --2- 172.21.15.5:0/1287233707 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff63806d3d0 0x7ff63806d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.729+0000 7ff6367fc640 1 -- 172.21.15.5:0/1287233707 >> v1:172.21.15.63:6789/0 conn(0x7ff6380b8190 legacy=0x7ff6380b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.729+0000 7ff6367fc640 1 -- 172.21.15.5:0/1287233707 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6380d3ca0 con 0x7ff638162820 2024-05-20T18:33:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.729+0000 7ff6367fc640 1 -- 172.21.15.5:0/1287233707 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7ff628001840 con 0x7ff638162820 2024-05-20T18:33:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.729+0000 7ff6367fc640 1 -- 172.21.15.5:0/1287233707 >> v1:172.21.15.5:6789/0 conn(0x7ff638162820 legacy=0x7ff6380b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.729+0000 7ff6367fc640 1 --2- 172.21.15.5:0/1287233707 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff63806d3d0 0x7ff6200039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.729+0000 7ff6367fc640 1 --2- 172.21.15.5:0/1287233707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff620003f60 0x7ff620004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.729+0000 7ff6367fc640 1 --2- 172.21.15.5:0/1287233707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff620004d40 0x7ff620005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.729+0000 7ff6367fc640 1 -- 172.21.15.5:0/1287233707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff6380d39c0 con 0x7ff620003f60 2024-05-20T18:33:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.729+0000 7ff6367fc640 1 -- 172.21.15.5:0/1287233707 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff62000bb50 con 0x7ff620004d40 2024-05-20T18:33:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.729+0000 7ff6367fc640 1 -- 172.21.15.5:0/1287233707 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff62000bd50 con 0x7ff63806d3d0 2024-05-20T18:33:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.729+0000 7ff636ffd640 1 --2- 172.21.15.5:0/1287233707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff620003f60 0x7ff620004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.729+0000 7ff6377fe640 1 --2- 172.21.15.5:0/1287233707 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff63806d3d0 0x7ff6200039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.729+0000 7ff637fff640 1 --2- 172.21.15.5:0/1287233707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff620004d40 0x7ff620005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.729+0000 7ff636ffd640 1 -- 172.21.15.5:0/1287233707 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff63806d3d0 msgr2=0x7ff6200039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.729+0000 7ff636ffd640 1 --2- 172.21.15.5:0/1287233707 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff63806d3d0 0x7ff6200039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.729+0000 7ff636ffd640 1 -- 172.21.15.5:0/1287233707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff620004d40 msgr2=0x7ff620005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.729+0000 7ff636ffd640 1 --2- 172.21.15.5:0/1287233707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff620004d40 0x7ff620005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.729+0000 7ff636ffd640 1 -- 172.21.15.5:0/1287233707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff620003520 con 0x7ff620003f60 2024-05-20T18:33:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.729+0000 7ff637fff640 1 --2- 172.21.15.5:0/1287233707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff620004d40 0x7ff620005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.730+0000 7ff636ffd640 1 --2- 172.21.15.5:0/1287233707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff620003f60 0x7ff620004800 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009ac0 tx=0x7ff620005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e59b3660cd1ad5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.730+0000 7ff6367fc640 1 -- 172.21.15.5:0/1287233707 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c036070 con 0x7ff620003f60 2024-05-20T18:33:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.730+0000 7ff6367fc640 1 -- 172.21.15.5:0/1287233707 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff62c031400 con 0x7ff620003f60 2024-05-20T18:33:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.730+0000 7ff6367fc640 1 -- 172.21.15.5:0/1287233707 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c009690 con 0x7ff620003f60 2024-05-20T18:33:28.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.731+0000 7ff63df38640 1 -- 172.21.15.5:0/1287233707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff620003f60 msgr2=0x7ff620004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:28.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.731+0000 7ff63df38640 1 --2- 172.21.15.5:0/1287233707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff620003f60 0x7ff620004800 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009ac0 tx=0x7ff620005830 comp rx=0 tx=0).stop 2024-05-20T18:33:28.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.731+0000 7ff637fff640 1 -- 172.21.15.5:0/1287233707 reap_dead start 2024-05-20T18:33:28.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.731+0000 7ff63df38640 1 -- 172.21.15.5:0/1287233707 shutdown_connections 2024-05-20T18:33:28.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.731+0000 7ff63df38640 1 -- 172.21.15.5:0/1287233707 >> 172.21.15.5:0/1287233707 conn(0x7ff6380d4bd0 msgr2=0x7ff6380d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:28.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.731+0000 7ff63df38640 1 -- 172.21.15.5:0/1287233707 shutdown_connections 2024-05-20T18:33:28.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.731+0000 7ff63df38640 1 -- 172.21.15.5:0/1287233707 wait complete. 2024-05-20T18:33:28.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.733+0000 7ff63df38640 1 Processor -- start 2024-05-20T18:33:28.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.733+0000 7ff63df38640 1 -- start start 2024-05-20T18:33:28.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.734+0000 7ff63df38640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff620003f60 0x7ff63814be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.734+0000 7ff63df38640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff63815ab20 0x7ff63815af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.734+0000 7ff63df38640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff63814c800 0x7ff63814cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.734+0000 7ff63df38640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff6380d0300 con 0x7ff63815ab20 2024-05-20T18:33:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.734+0000 7ff63df38640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff6380d4260 con 0x7ff620003f60 2024-05-20T18:33:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.734+0000 7ff63df38640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff6380b4050 con 0x7ff63814c800 2024-05-20T18:33:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.734+0000 7ff636ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff63815ab20 0x7ff63815af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.734+0000 7ff636ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff63815ab20 0x7ff63815af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54890/0 (socket says 172.21.15.5:54890) 2024-05-20T18:33:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.734+0000 7ff636ffd640 1 -- 172.21.15.5:0/3497831561 learned_addr learned my addr 172.21.15.5:0/3497831561 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.735+0000 7ff637fff640 1 --2- 172.21.15.5:0/3497831561 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff63814c800 0x7ff63814cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.735+0000 7ff6377fe640 1 --2- 172.21.15.5:0/3497831561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff620003f60 0x7ff63814be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.735+0000 7ff636ffd640 1 -- 172.21.15.5:0/3497831561 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff63814c800 msgr2=0x7ff63814cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.735+0000 7ff636ffd640 1 --2- 172.21.15.5:0/3497831561 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff63814c800 0x7ff63814cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.735+0000 7ff636ffd640 1 -- 172.21.15.5:0/3497831561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff620003f60 msgr2=0x7ff63814be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.735+0000 7ff636ffd640 1 --2- 172.21.15.5:0/3497831561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff620003f60 0x7ff63814be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.735+0000 7ff636ffd640 1 -- 172.21.15.5:0/3497831561 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63815b8e0 con 0x7ff63815ab20 2024-05-20T18:33:28.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.735+0000 7ff637fff640 1 --2- 172.21.15.5:0/3497831561 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff63814c800 0x7ff63814cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:28.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.735+0000 7ff6377fe640 1 --2- 172.21.15.5:0/3497831561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff620003f60 0x7ff63814be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:28.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.735+0000 7ff636ffd640 1 --2- 172.21.15.5:0/3497831561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff63815ab20 0x7ff63815af30 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7ff62c034040 tx=0x7ff62c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:28.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.735+0000 7ff61ffff640 1 -- 172.21.15.5:0/3497831561 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c040020 con 0x7ff63815ab20 2024-05-20T18:33:28.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.735+0000 7ff61ffff640 1 -- 172.21.15.5:0/3497831561 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff62c003b40 con 0x7ff63815ab20 2024-05-20T18:33:28.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.735+0000 7ff63df38640 1 -- 172.21.15.5:0/3497831561 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff63815bbc0 con 0x7ff63815ab20 2024-05-20T18:33:28.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.735+0000 7ff61ffff640 1 -- 172.21.15.5:0/3497831561 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c036040 con 0x7ff63815ab20 2024-05-20T18:33:28.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.735+0000 7ff63df38640 1 -- 172.21.15.5:0/3497831561 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff63815c180 con 0x7ff63815ab20 2024-05-20T18:33:28.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.737+0000 7ff61ffff640 1 -- 172.21.15.5:0/3497831561 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff62c043020 con 0x7ff63815ab20 2024-05-20T18:33:28.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.737+0000 7ff61ffff640 1 --2- 172.21.15.5:0/3497831561 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff62406be30 0x7ff62406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:28.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.738+0000 7ff61ffff640 1 -- 172.21.15.5:0/3497831561 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6187+0+0 (secure 0 0 0) 0x7ff62c0ab430 con 0x7ff63815ab20 2024-05-20T18:33:28.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.738+0000 7ff6377fe640 1 --2- 172.21.15.5:0/3497831561 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff62406be30 0x7ff62406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:28.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.738+0000 7ff6377fe640 1 --2- 172.21.15.5:0/3497831561 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff62406be30 0x7ff62406e340 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff6280016c0 tx=0x7ff628009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:28.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.738+0000 7ff644de4740 1 Processor -- start 2024-05-20T18:33:28.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.738+0000 7ff644de4740 1 -- start start 2024-05-20T18:33:28.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.740+0000 7ff644de4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e28a357270 0x55e28a722700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.740+0000 7ff644de4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e28a32e3b0 0x55e28a71f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.740+0000 7ff644de4740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e28a336590 0x55e28a6f8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.740+0000 7ff644de4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e28a65b4d0 con 0x55e28a32e3b0 2024-05-20T18:33:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.740+0000 7ff644de4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e28a6033e0 con 0x55e28a357270 2024-05-20T18:33:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.740+0000 7ff644de4740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55e28a657490 con 0x55e28a336590 2024-05-20T18:33:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.740+0000 7ff636ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e28a32e3b0 0x55e28a71f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.740+0000 7ff636ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e28a32e3b0 0x55e28a71f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54892/0 (socket says 172.21.15.5:54892) 2024-05-20T18:33:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.740+0000 7ff636ffd640 1 -- 172.21.15.5:0/3476373396 learned_addr learned my addr 172.21.15.5:0/3476373396 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.740+0000 7ff636ffd640 1 -- 172.21.15.5:0/3476373396 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e28a336590 msgr2=0x55e28a6f8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.740+0000 7ff637fff640 1 --2- 172.21.15.5:0/3476373396 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e28a336590 0x55e28a6f8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.740+0000 7ff6377fe640 1 --2- 172.21.15.5:0/3476373396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e28a357270 0x55e28a722700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.740+0000 7ff636ffd640 1 --2- 172.21.15.5:0/3476373396 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e28a336590 0x55e28a6f8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.740+0000 7ff636ffd640 1 -- 172.21.15.5:0/3476373396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e28a357270 msgr2=0x55e28a722700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:28.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.740+0000 7ff636ffd640 1 --2- 172.21.15.5:0/3476373396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e28a357270 0x55e28a722700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.740+0000 7ff636ffd640 1 -- 172.21.15.5:0/3476373396 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e28a36f2d0 con 0x55e28a32e3b0 2024-05-20T18:33:28.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.740+0000 7ff636ffd640 1 --2- 172.21.15.5:0/3476373396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e28a32e3b0 0x55e28a71f0a0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7ff62c081bb0 tx=0x7ff62c0790e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2c755264e58d16 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:28.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.741+0000 7ff61d7fa640 1 -- 172.21.15.5:0/3476373396 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c0820e0 con 0x55e28a32e3b0 2024-05-20T18:33:28.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.741+0000 7ff61d7fa640 1 -- 172.21.15.5:0/3476373396 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff62c077770 con 0x55e28a32e3b0 2024-05-20T18:33:28.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.741+0000 7ff61d7fa640 1 -- 172.21.15.5:0/3476373396 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c0ab070 con 0x55e28a32e3b0 2024-05-20T18:33:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.741+0000 7ff644de4740 1 -- 172.21.15.5:0/3476373396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e28a32e3b0 msgr2=0x55e28a71f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.741+0000 7ff644de4740 1 --2- 172.21.15.5:0/3476373396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e28a32e3b0 0x55e28a71f0a0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7ff62c081bb0 tx=0x7ff62c0790e0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.742+0000 7ff644de4740 1 -- 172.21.15.5:0/3476373396 shutdown_connections 2024-05-20T18:33:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.742+0000 7ff644de4740 1 --2- 172.21.15.5:0/3476373396 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e28a336590 0x55e28a6f8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.742+0000 7ff644de4740 1 --2- 172.21.15.5:0/3476373396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e28a32e3b0 0x55e28a71f0a0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.742+0000 7ff644de4740 1 --2- 172.21.15.5:0/3476373396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e28a357270 0x55e28a722700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.742+0000 7ff644de4740 1 -- 172.21.15.5:0/3476373396 >> 172.21.15.5:0/3476373396 conn(0x55e28a34b3c0 msgr2=0x55e28a6f7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.742+0000 7ff644de4740 1 -- 172.21.15.5:0/3476373396 shutdown_connections 2024-05-20T18:33:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.742+0000 7ff644de4740 1 -- 172.21.15.5:0/3476373396 wait complete. 2024-05-20T18:33:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.742+0000 7ff644de4740 1 -- start start 2024-05-20T18:33:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.743+0000 7ff644de4740 1 Processor -- start 2024-05-20T18:33:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.744+0000 7ff644de4740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e28a32e3b0 0x55e28a725bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.744+0000 7ff644de4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e28a336590 0x55e28a71f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.744+0000 7ff644de4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e28a357270 0x55e28a71f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.744+0000 7ff644de4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e28a657490 con 0x55e28a357270 2024-05-20T18:33:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.744+0000 7ff644de4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e28a6168e0 con 0x55e28a336590 2024-05-20T18:33:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.744+0000 7ff644de4740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55e28a65b4d0 con 0x55e28a32e3b0 2024-05-20T18:33:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.744+0000 7ff644de4740 10 client.0 _ll_register_callbacks cb 0x55e28a5e39c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:33:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.744+0000 7ff637fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e28a357270 0x55e28a71f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.744+0000 7ff637fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e28a357270 0x55e28a71f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54906/0 (socket says 172.21.15.5:54906) 2024-05-20T18:33:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.744+0000 7ff637fff640 1 -- 172.21.15.5:0/1429332006 learned_addr learned my addr 172.21.15.5:0/1429332006 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.744+0000 7ff636ffd640 1 --2- 172.21.15.5:0/1429332006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e28a336590 0x55e28a71f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.744+0000 7ff6377fe640 1 --2- 172.21.15.5:0/1429332006 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e28a32e3b0 0x55e28a725bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.744+0000 7ff637fff640 1 -- 172.21.15.5:0/1429332006 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e28a32e3b0 msgr2=0x55e28a725bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.744+0000 7ff637fff640 1 --2- 172.21.15.5:0/1429332006 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e28a32e3b0 0x55e28a725bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.744+0000 7ff637fff640 1 -- 172.21.15.5:0/1429332006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e28a336590 msgr2=0x55e28a71f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.744+0000 7ff637fff640 1 --2- 172.21.15.5:0/1429332006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e28a336590 0x55e28a71f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.744+0000 7ff637fff640 1 -- 172.21.15.5:0/1429332006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e28a4396b0 con 0x55e28a357270 2024-05-20T18:33:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.745+0000 7ff637fff640 1 --2- 172.21.15.5:0/1429332006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e28a357270 0x55e28a71f550 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7ff630000f80 tx=0x7ff630001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912d61612ff5297f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.745+0000 7ff6167fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:33:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.745+0000 7ff6167fc640 1 -- 172.21.15.5:0/1429332006 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff630010040 con 0x55e28a357270 2024-05-20T18:33:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.745+0000 7ff6167fc640 1 -- 172.21.15.5:0/1429332006 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6187+0+0 (secure 0 0 0) 0x7ff630015610 con 0x55e28a357270 2024-05-20T18:33:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.745+0000 7ff644de4740 1 -- 172.21.15.5:0/1429332006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff62c077770 con 0x55e28a357270 2024-05-20T18:33:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.745+0000 7ff6167fc640 1 -- 172.21.15.5:0/1429332006 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7ff630021050 con 0x55e28a357270 2024-05-20T18:33:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.745+0000 7ff644de4740 10 client.7719 fetch_fsmap learned FSMap version 66 2024-05-20T18:33:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.745+0000 7ff644de4740 1 -- 172.21.15.5:0/1429332006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55e28a724780 con 0x55e28a357270 2024-05-20T18:33:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.745+0000 7ff6167fc640 1 -- 172.21.15.5:0/1429332006 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff62c077770 con 0x55e28a357270 2024-05-20T18:33:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.745+0000 7ff6167fc640 1 -- 172.21.15.5:0/1429332006 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7ff630024020 con 0x55e28a357270 2024-05-20T18:33:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.745+0000 7ff644de4740 10 client.7719 fetch_fsmap finished waiting for FSMap version 66 2024-05-20T18:33:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.745+0000 7ff644de4740 10 client.7719 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-20T18:33:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.745+0000 7ff644de4740 20 client.7719 populate_metadata read hostname 'smithi005' 2024-05-20T18:33:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.745+0000 7ff644de4740 1 --2- 172.21.15.5:0/1429332006 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55e28a315e40 0x55e28a74c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.745+0000 7ff644de4740 4 client.7719 mds_command: new command op to 7437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:33:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.745+0000 7ff644de4740 1 -- 172.21.15.5:0/1429332006 --> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e28a668c00 con 0x55e28a315e40 2024-05-20T18:33:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.745+0000 7ff6377fe640 1 --2- 172.21.15.5:0/1429332006 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55e28a315e40 0x55e28a74c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.746+0000 7ff6167fc640 10 client.7719 ms_handle_connect on v2:172.21.15.5:6836/3556492262 2024-05-20T18:33:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.746+0000 7ff6377fe640 1 --2- 172.21.15.5:0/1429332006 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55e28a315e40 0x55e28a74c640 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=606bb7817e0bb96f server_cookie=dcb5d78459dc8caf in_seq=0 out_seq=0 2024-05-20T18:33:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.746+0000 7ff6167fc640 1 -- 172.21.15.5:0/1429332006 <== mds.0 v2:172.21.15.5:6836/3556492262 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e28a668c00 con 0x55e28a315e40 2024-05-20T18:33:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.747+0000 7ff6167fc640 10 client.7719 handle_command_reply: tid=0 2024-05-20T18:33:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.747+0000 7ff644de4740 1 client.7719 shutdown 2024-05-20T18:33:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.747+0000 7ff644de4740 1 -- 172.21.15.5:0/1429332006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e28a357270 msgr2=0x55e28a71f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.747+0000 7ff644de4740 1 --2- 172.21.15.5:0/1429332006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e28a357270 0x55e28a71f550 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7ff630000f80 tx=0x7ff630001ac0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.747+0000 7ff644de4740 1 -- 172.21.15.5:0/1429332006 shutdown_connections 2024-05-20T18:33:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.747+0000 7ff644de4740 1 --2- 172.21.15.5:0/1429332006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e28a357270 0x55e28a71f550 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.747+0000 7ff644de4740 1 --2- 172.21.15.5:0/1429332006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e28a336590 0x55e28a71f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.747+0000 7ff644de4740 1 --2- 172.21.15.5:0/1429332006 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55e28a315e40 0x55e28a74c640 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.747+0000 7ff644de4740 1 --2- 172.21.15.5:0/1429332006 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e28a32e3b0 0x55e28a725bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.747+0000 7ff644de4740 1 -- 172.21.15.5:0/1429332006 >> 172.21.15.5:0/1429332006 conn(0x55e28a34b3c0 msgr2=0x55e28a6f7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.747+0000 7ff6167fc640 0 client.7719 ms_handle_reset on v2:172.21.15.5:6836/3556492262 2024-05-20T18:33:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.747+0000 7ff644de4740 1 -- 172.21.15.5:0/1429332006 shutdown_connections 2024-05-20T18:33:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.747+0000 7ff644de4740 1 -- 172.21.15.5:0/1429332006 wait complete. 2024-05-20T18:33:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.747+0000 7ff644de4740 20 client.7719 trim_cache size 0 max 16384 2024-05-20T18:33:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.766+0000 7ff644de4740 1 Processor -- start 2024-05-20T18:33:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.766+0000 7ff644de4740 1 -- start start 2024-05-20T18:33:28.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.767+0000 7ff644de4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e28a32e3b0 0x55e28a73ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:28.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.767+0000 7ff644de4740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e28a336590 0x55e28a73f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:28.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.767+0000 7ff644de4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e28a357270 0x55e28a74c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:28.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.767+0000 7ff644de4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff638201e50 con 0x55e28a357270 2024-05-20T18:33:28.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.767+0000 7ff644de4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff6382011b0 con 0x55e28a32e3b0 2024-05-20T18:33:28.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.767+0000 7ff644de4740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff638201800 con 0x55e28a336590 2024-05-20T18:33:28.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.768+0000 7ff637fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e28a357270 0x55e28a74c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:28.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.768+0000 7ff637fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e28a357270 0x55e28a74c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54920/0 (socket says 172.21.15.5:54920) 2024-05-20T18:33:28.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.768+0000 7ff637fff640 1 -- 172.21.15.5:0/3118239512 learned_addr learned my addr 172.21.15.5:0/3118239512 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:28.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.768+0000 7ff636ffd640 1 --2- 172.21.15.5:0/3118239512 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e28a336590 0x55e28a73f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:28.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.768+0000 7ff637fff640 1 -- 172.21.15.5:0/3118239512 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e28a336590 msgr2=0x55e28a73f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:28.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.768+0000 7ff6377fe640 1 --2- 172.21.15.5:0/3118239512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e28a32e3b0 0x55e28a73ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:28.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.768+0000 7ff637fff640 1 --2- 172.21.15.5:0/3118239512 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e28a336590 0x55e28a73f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.768+0000 7ff637fff640 1 -- 172.21.15.5:0/3118239512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e28a32e3b0 msgr2=0x55e28a73ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:28.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.768+0000 7ff637fff640 1 --2- 172.21.15.5:0/3118239512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e28a32e3b0 0x55e28a73ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.768+0000 7ff637fff640 1 -- 172.21.15.5:0/3118239512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e28a6f1220 con 0x55e28a357270 2024-05-20T18:33:28.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.768+0000 7ff636ffd640 1 --2- 172.21.15.5:0/3118239512 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e28a336590 0x55e28a73f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:33:28.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.768+0000 7ff637fff640 1 --2- 172.21.15.5:0/3118239512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e28a357270 0x55e28a74c640 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7ff630016e10 tx=0x7ff63000c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e7ee25b7ebc75a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:28.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.768+0000 7ff61cff9640 1 -- 172.21.15.5:0/3118239512 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63002a070 con 0x55e28a357270 2024-05-20T18:33:28.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.768+0000 7ff61cff9640 1 -- 172.21.15.5:0/3118239512 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6300155e0 con 0x55e28a357270 2024-05-20T18:33:28.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.768+0000 7ff61cff9640 1 -- 172.21.15.5:0/3118239512 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6300253f0 con 0x55e28a357270 2024-05-20T18:33:28.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.769+0000 7ff644de4740 1 -- 172.21.15.5:0/3118239512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e28a357270 msgr2=0x55e28a74c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:28.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.769+0000 7ff644de4740 1 --2- 172.21.15.5:0/3118239512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e28a357270 0x55e28a74c640 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7ff630016e10 tx=0x7ff63000c000 comp rx=0 tx=0).stop 2024-05-20T18:33:28.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.769+0000 7ff644de4740 1 -- 172.21.15.5:0/3118239512 shutdown_connections 2024-05-20T18:33:28.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.769+0000 7ff644de4740 1 --2- 172.21.15.5:0/3118239512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e28a357270 0x55e28a74c640 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.769+0000 7ff644de4740 1 --2- 172.21.15.5:0/3118239512 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e28a336590 0x55e28a73f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.769+0000 7ff644de4740 1 --2- 172.21.15.5:0/3118239512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e28a32e3b0 0x55e28a73ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.769+0000 7ff644de4740 1 -- 172.21.15.5:0/3118239512 >> 172.21.15.5:0/3118239512 conn(0x55e28a34b3c0 msgr2=0x55e28a6ed940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.769+0000 7ff644de4740 1 -- 172.21.15.5:0/3118239512 shutdown_connections 2024-05-20T18:33:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.769+0000 7ff644de4740 1 -- 172.21.15.5:0/3118239512 wait complete. 2024-05-20T18:33:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.770+0000 7ff644de4740 1 -- start start 2024-05-20T18:33:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.770+0000 7ff644de4740 1 Processor -- start 2024-05-20T18:33:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.771+0000 7ff644de4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e28a32e3b0 0x55e28a74c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.771+0000 7ff644de4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e28a336590 0x55e28a74cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.771+0000 7ff644de4740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e28a357270 0x55e28a71e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.771+0000 7ff644de4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e28a74d620 con 0x55e28a32e3b0 2024-05-20T18:33:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.771+0000 7ff644de4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e28a6033e0 con 0x55e28a336590 2024-05-20T18:33:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.771+0000 7ff644de4740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55e28a73c320 con 0x55e28a357270 2024-05-20T18:33:28.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.771+0000 7ff644de4740 10 client.0 _ll_register_callbacks cb 0x7ff6300052f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:33:28.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.771+0000 7ff6377fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e28a32e3b0 0x55e28a74c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:28.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.771+0000 7ff6377fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e28a32e3b0 0x55e28a74c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54934/0 (socket says 172.21.15.5:54934) 2024-05-20T18:33:28.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.772+0000 7ff6377fe640 1 -- 172.21.15.5:0/3891161661 learned_addr learned my addr 172.21.15.5:0/3891161661 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:28.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.772+0000 7ff6377fe640 1 -- 172.21.15.5:0/3891161661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e28a357270 msgr2=0x55e28a71e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:28.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.772+0000 7ff6377fe640 1 --2- 172.21.15.5:0/3891161661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e28a357270 0x55e28a71e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.772+0000 7ff6377fe640 1 -- 172.21.15.5:0/3891161661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e28a336590 msgr2=0x55e28a74cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:28.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.772+0000 7ff6377fe640 1 --2- 172.21.15.5:0/3891161661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e28a336590 0x55e28a74cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.772+0000 7ff6377fe640 1 -- 172.21.15.5:0/3891161661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e28a6f32e0 con 0x55e28a32e3b0 2024-05-20T18:33:28.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.772+0000 7ff6377fe640 1 --2- 172.21.15.5:0/3891161661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e28a32e3b0 0x55e28a74c640 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x55e28a71fef0 tx=0x7ff6380b7a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a1b8298c00895 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:28.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.772+0000 7ff6167fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:33:28.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.772+0000 7ff6167fc640 1 -- 172.21.15.5:0/3891161661 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628015640 con 0x55e28a32e3b0 2024-05-20T18:33:28.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.772+0000 7ff6167fc640 1 -- 172.21.15.5:0/3891161661 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6187+0+0 (secure 0 0 0) 0x7ff62802c9e0 con 0x55e28a32e3b0 2024-05-20T18:33:28.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.772+0000 7ff644de4740 1 -- 172.21.15.5:0/3891161661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6300155e0 con 0x55e28a32e3b0 2024-05-20T18:33:28.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.772+0000 7ff6167fc640 1 -- 172.21.15.5:0/3891161661 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7ff62802cbe0 con 0x55e28a32e3b0 2024-05-20T18:33:28.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.772+0000 7ff644de4740 10 client.7725 fetch_fsmap learned FSMap version 66 2024-05-20T18:33:28.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.772+0000 7ff644de4740 1 -- 172.21.15.5:0/3891161661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55e28a7b9230 con 0x55e28a32e3b0 2024-05-20T18:33:28.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.773+0000 7ff6167fc640 1 -- 172.21.15.5:0/3891161661 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6300155e0 con 0x55e28a32e3b0 2024-05-20T18:33:28.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.773+0000 7ff6167fc640 1 -- 172.21.15.5:0/3891161661 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7ff6280157e0 con 0x55e28a32e3b0 2024-05-20T18:33:28.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.773+0000 7ff644de4740 10 client.7725 fetch_fsmap finished waiting for FSMap version 66 2024-05-20T18:33:28.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.773+0000 7ff644de4740 10 client.7725 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-20T18:33:28.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.773+0000 7ff644de4740 20 client.7725 populate_metadata read hostname 'smithi005' 2024-05-20T18:33:28.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.773+0000 7ff644de4740 1 --2- 172.21.15.5:0/3891161661 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55e28a7bc7d0 0x55e28a7bcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:28.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.773+0000 7ff644de4740 4 client.7725 mds_command: new command op to 7437 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T18:33:28.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.773+0000 7ff644de4740 1 -- 172.21.15.5:0/3891161661 --> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] -- command(tid 0: {"prefix": "status"}) -- 0x55e28a741910 con 0x55e28a7bc7d0 2024-05-20T18:33:28.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.773+0000 7ff636ffd640 1 --2- 172.21.15.5:0/3891161661 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55e28a7bc7d0 0x55e28a7bcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:28.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.773+0000 7ff6167fc640 10 client.7725 ms_handle_connect on v2:172.21.15.5:6836/3556492262 2024-05-20T18:33:28.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.773+0000 7ff636ffd640 1 --2- 172.21.15.5:0/3891161661 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55e28a7bc7d0 0x55e28a7bcbe0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9700b0570071e495 server_cookie=8d03ad3624b50fcc in_seq=0 out_seq=0 2024-05-20T18:33:28.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.773+0000 7ff6167fc640 1 -- 172.21.15.5:0/3891161661 <== mds.0 v2:172.21.15.5:6836/3556492262 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55e28a741910 con 0x55e28a7bc7d0 2024-05-20T18:33:28.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.773+0000 7ff6167fc640 10 client.7725 handle_command_reply: tid=0 2024-05-20T18:33:28.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.773+0000 7ff644de4740 1 client.7725 shutdown 2024-05-20T18:33:28.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.774+0000 7ff644de4740 1 -- 172.21.15.5:0/3891161661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e28a32e3b0 msgr2=0x55e28a74c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:28.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.774+0000 7ff644de4740 1 --2- 172.21.15.5:0/3891161661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e28a32e3b0 0x55e28a74c640 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x55e28a71fef0 tx=0x7ff6380b7a20 comp rx=0 tx=0).stop 2024-05-20T18:33:28.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.774+0000 7ff644de4740 1 -- 172.21.15.5:0/3891161661 shutdown_connections 2024-05-20T18:33:28.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.774+0000 7ff644de4740 1 --2- 172.21.15.5:0/3891161661 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55e28a7bc7d0 0x55e28a7bcbe0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.774+0000 7ff644de4740 1 --2- 172.21.15.5:0/3891161661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e28a357270 0x55e28a71e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.774+0000 7ff644de4740 1 --2- 172.21.15.5:0/3891161661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e28a336590 0x55e28a74cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.774+0000 7ff644de4740 1 --2- 172.21.15.5:0/3891161661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e28a32e3b0 0x55e28a74c640 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.774+0000 7ff6167fc640 0 client.7725 ms_handle_reset on v2:172.21.15.5:6836/3556492262 2024-05-20T18:33:28.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.774+0000 7ff644de4740 1 -- 172.21.15.5:0/3891161661 >> 172.21.15.5:0/3891161661 conn(0x55e28a34b3c0 msgr2=0x55e28a73d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:28.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.774+0000 7ff644de4740 1 -- 172.21.15.5:0/3891161661 shutdown_connections 2024-05-20T18:33:28.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.774+0000 7ff644de4740 1 -- 172.21.15.5:0/3891161661 wait complete. 2024-05-20T18:33:28.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.774+0000 7ff644de4740 20 client.7725 trim_cache size 0 max 16384 2024-05-20T18:33:28.780 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:33:28.780 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "399ab250-fd3c-47fa-a1c8-25e8149c9cac", 2024-05-20T18:33:28.780 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-20T18:33:28.780 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 7437, 2024-05-20T18:33:28.780 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-20T18:33:28.780 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-20T18:33:28.780 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-20T18:33:28.780 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 4.5300835240000001, 2024-05-20T18:33:28.781 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 66, 2024-05-20T18:33:28.781 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 91, 2024-05-20T18:33:28.781 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 91, 2024-05-20T18:33:28.781 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 21.632107249000001 2024-05-20T18:33:28.781 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:33:28.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.774+0000 7ff63df38640 1 -- 172.21.15.5:0/3497831561 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff62406be30 msgr2=0x7ff62406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:28.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.774+0000 7ff63df38640 1 --2- 172.21.15.5:0/3497831561 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff62406be30 0x7ff62406e340 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff6280016c0 tx=0x7ff628009040 comp rx=0 tx=0).stop 2024-05-20T18:33:28.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.774+0000 7ff63df38640 1 -- 172.21.15.5:0/3497831561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff63815ab20 msgr2=0x7ff63815af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:28.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.774+0000 7ff63df38640 1 --2- 172.21.15.5:0/3497831561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff63815ab20 0x7ff63815af30 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7ff62c034040 tx=0x7ff62c004730 comp rx=0 tx=0).stop 2024-05-20T18:33:28.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.774+0000 7ff63df38640 1 -- 172.21.15.5:0/3497831561 shutdown_connections 2024-05-20T18:33:28.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.774+0000 7ff63df38640 1 --2- 172.21.15.5:0/3497831561 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff62406be30 0x7ff62406e340 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.774+0000 7ff63df38640 1 --2- 172.21.15.5:0/3497831561 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff63814c800 0x7ff63814cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.774+0000 7ff63df38640 1 --2- 172.21.15.5:0/3497831561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff63815ab20 0x7ff63815af30 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.774+0000 7ff63df38640 1 --2- 172.21.15.5:0/3497831561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff620003f60 0x7ff63814be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.774+0000 7ff63df38640 1 -- 172.21.15.5:0/3497831561 >> 172.21.15.5:0/3497831561 conn(0x7ff6380d4bd0 msgr2=0x7ff6380d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:28.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.775+0000 7ff63df38640 1 -- 172.21.15.5:0/3497831561 shutdown_connections 2024-05-20T18:33:28.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.775+0000 7ff63df38640 1 -- 172.21.15.5:0/3497831561 wait complete. 2024-05-20T18:33:28.787 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-20T18:33:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.900+0000 7f4790537640 1 Processor -- start 2024-05-20T18:33:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.901+0000 7f4790537640 1 -- start start 2024-05-20T18:33:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.902+0000 7f4790537640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4788162820 0x7f47880b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.902+0000 7f4790537640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47880d0630 con 0x7f478806dae0 2024-05-20T18:33:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.902+0000 7f4790537640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47880d39c0 con 0x7f47880b3e80 2024-05-20T18:33:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.902+0000 7f4790537640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f47880d3ca0 con 0x7f4788162820 2024-05-20T18:33:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.902+0000 7f478ead4640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f478806dae0 0x7f47880b8190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47430/0 (socket says 172.21.15.5:47430) 2024-05-20T18:33:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.902+0000 7f478ead4640 1 -- 172.21.15.5:0/4182938999 learned_addr learned my addr 172.21.15.5:0/4182938999 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:33:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.903+0000 7f478e2d3640 1 --2- 172.21.15.5:0/4182938999 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4788162820 0x7f47880b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.903+0000 7f478d2d1640 1 -- 172.21.15.5:0/4182938999 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3849629447 0 0) 0x7f47880d0630 con 0x7f478806dae0 2024-05-20T18:33:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.903+0000 7f478d2d1640 1 -- 172.21.15.5:0/4182938999 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4774003520 con 0x7f478806dae0 2024-05-20T18:33:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.903+0000 7f478d2d1640 1 -- 172.21.15.5:0/4182938999 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2634092692 0 0) 0x7f4774003520 con 0x7f478806dae0 2024-05-20T18:33:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.903+0000 7f478d2d1640 1 -- 172.21.15.5:0/4182938999 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4788162820 msgr2=0x7f47880b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.903+0000 7f478d2d1640 1 --2- 172.21.15.5:0/4182938999 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4788162820 0x7f47880b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.903+0000 7f478d2d1640 1 -- 172.21.15.5:0/4182938999 >> v1:172.21.15.63:6789/0 conn(0x7f47880b3e80 legacy=0x7f478806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.903+0000 7f478d2d1640 1 -- 172.21.15.5:0/4182938999 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47880d3f80 con 0x7f478806dae0 2024-05-20T18:33:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.903+0000 7f478d2d1640 1 -- 172.21.15.5:0/4182938999 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f47800019f0 con 0x7f478806dae0 2024-05-20T18:33:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.903+0000 7f478d2d1640 1 -- 172.21.15.5:0/4182938999 >> v1:172.21.15.5:6789/0 conn(0x7f478806dae0 legacy=0x7f47880b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.903+0000 7f478d2d1640 1 --2- 172.21.15.5:0/4182938999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47740039e0 0x7f4774003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.903+0000 7f478d2d1640 1 --2- 172.21.15.5:0/4182938999 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4788162820 0x7f4774004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.903+0000 7f478d2d1640 1 --2- 172.21.15.5:0/4182938999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4774004d40 0x7f4774005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.903+0000 7f478d2d1640 1 -- 172.21.15.5:0/4182938999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f47880d3ca0 con 0x7f4774004d40 2024-05-20T18:33:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.903+0000 7f478d2d1640 1 -- 172.21.15.5:0/4182938999 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f477400ba70 con 0x7f47740039e0 2024-05-20T18:33:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.903+0000 7f478d2d1640 1 -- 172.21.15.5:0/4182938999 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f477400bc70 con 0x7f4788162820 2024-05-20T18:33:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.903+0000 7f478ead4640 1 --2- 172.21.15.5:0/4182938999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4774004d40 0x7f4774005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.903+0000 7f478e2d3640 1 --2- 172.21.15.5:0/4182938999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47740039e0 0x7f4774003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.903+0000 7f478dad2640 1 --2- 172.21.15.5:0/4182938999 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4788162820 0x7f4774004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.903+0000 7f478ead4640 1 -- 172.21.15.5:0/4182938999 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4788162820 msgr2=0x7f4774004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.903+0000 7f478ead4640 1 --2- 172.21.15.5:0/4182938999 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4788162820 0x7f4774004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.903+0000 7f478ead4640 1 -- 172.21.15.5:0/4182938999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47740039e0 msgr2=0x7f4774003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.903+0000 7f478ead4640 1 --2- 172.21.15.5:0/4182938999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47740039e0 0x7f4774003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.903+0000 7f478ead4640 1 -- 172.21.15.5:0/4182938999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4774003520 con 0x7f4774004d40 2024-05-20T18:33:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.903+0000 7f478e2d3640 1 --2- 172.21.15.5:0/4182938999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47740039e0 0x7f4774003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.904+0000 7f478ead4640 1 --2- 172.21.15.5:0/4182938999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4774004d40 0x7f4774005150 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f478000a7c0 tx=0x7f4774005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e05c4a4c16e148 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:28.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.904+0000 7f478d2d1640 1 -- 172.21.15.5:0/4182938999 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478000a410 con 0x7f4774004d40 2024-05-20T18:33:28.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.904+0000 7f478d2d1640 1 -- 172.21.15.5:0/4182938999 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f478002f070 con 0x7f4774004d40 2024-05-20T18:33:28.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.904+0000 7f478d2d1640 1 -- 172.21.15.5:0/4182938999 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478002bc00 con 0x7f4774004d40 2024-05-20T18:33:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.905+0000 7f4790537640 1 -- 172.21.15.5:0/4182938999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4774004d40 msgr2=0x7f4774005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.905+0000 7f4790537640 1 --2- 172.21.15.5:0/4182938999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4774004d40 0x7f4774005150 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f478000a7c0 tx=0x7f4774005830 comp rx=0 tx=0).stop 2024-05-20T18:33:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.905+0000 7f478ead4640 1 -- 172.21.15.5:0/4182938999 reap_dead start 2024-05-20T18:33:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.905+0000 7f4790537640 1 -- 172.21.15.5:0/4182938999 shutdown_connections 2024-05-20T18:33:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.905+0000 7f4790537640 1 -- 172.21.15.5:0/4182938999 >> 172.21.15.5:0/4182938999 conn(0x7f47880d4bd0 msgr2=0x7f47880d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.905+0000 7f4790537640 1 -- 172.21.15.5:0/4182938999 shutdown_connections 2024-05-20T18:33:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.905+0000 7f4790537640 1 -- 172.21.15.5:0/4182938999 wait complete. 2024-05-20T18:33:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.907+0000 7f4790537640 1 Processor -- start 2024-05-20T18:33:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.907+0000 7f4790537640 1 -- start start 2024-05-20T18:33:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.908+0000 7f4790537640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4774004d40 0x7f4788155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.908+0000 7f4790537640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f478815be10 0x7f47881540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.908+0000 7f4790537640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4788158180 0x7f4788158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.908+0000 7f4790537640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f47880d4780 con 0x7f4774004d40 2024-05-20T18:33:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.908+0000 7f4790537640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f47880d0300 con 0x7f4788158180 2024-05-20T18:33:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.908+0000 7f4790537640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f47880d4900 con 0x7f478815be10 2024-05-20T18:33:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.908+0000 7f478e2d3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4774004d40 0x7f4788155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.908+0000 7f478e2d3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4774004d40 0x7f4788155fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54938/0 (socket says 172.21.15.5:54938) 2024-05-20T18:33:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.908+0000 7f478e2d3640 1 -- 172.21.15.5:0/2774605509 learned_addr learned my addr 172.21.15.5:0/2774605509 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.909+0000 7f478ead4640 1 --2- 172.21.15.5:0/2774605509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4788158180 0x7f4788158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.909+0000 7f478dad2640 1 --2- 172.21.15.5:0/2774605509 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f478815be10 0x7f47881540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.909+0000 7f478e2d3640 1 -- 172.21.15.5:0/2774605509 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f478815be10 msgr2=0x7f47881540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.909+0000 7f478e2d3640 1 --2- 172.21.15.5:0/2774605509 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f478815be10 0x7f47881540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.909+0000 7f478e2d3640 1 -- 172.21.15.5:0/2774605509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4788158180 msgr2=0x7f4788158590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.909+0000 7f478e2d3640 1 --2- 172.21.15.5:0/2774605509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4788158180 0x7f4788158590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.909+0000 7f478e2d3640 1 -- 172.21.15.5:0/2774605509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4788158ad0 con 0x7f4774004d40 2024-05-20T18:33:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.909+0000 7f478e2d3640 1 --2- 172.21.15.5:0/2774605509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4774004d40 0x7f4788155fa0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f47840116d0 tx=0x7f4784011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.909+0000 7f477effd640 1 -- 172.21.15.5:0/2774605509 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478400f9c0 con 0x7f4774004d40 2024-05-20T18:33:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.909+0000 7f477effd640 1 -- 172.21.15.5:0/2774605509 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f478400fb60 con 0x7f4774004d40 2024-05-20T18:33:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.909+0000 7f4790537640 1 -- 172.21.15.5:0/2774605509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47880b77c0 con 0x7f4774004d40 2024-05-20T18:33:28.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.909+0000 7f477effd640 1 -- 172.21.15.5:0/2774605509 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4784015460 con 0x7f4774004d40 2024-05-20T18:33:28.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.909+0000 7f4790537640 1 -- 172.21.15.5:0/2774605509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47881568a0 con 0x7f4774004d40 2024-05-20T18:33:28.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.910+0000 7f4790537640 1 -- 172.21.15.5:0/2774605509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4788075b00 con 0x7f4774004d40 2024-05-20T18:33:28.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.911+0000 7f477effd640 1 -- 172.21.15.5:0/2774605509 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4784015600 con 0x7f4774004d40 2024-05-20T18:33:28.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.911+0000 7f477effd640 1 --2- 172.21.15.5:0/2774605509 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f476c06be30 0x7f476c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.912+0000 7f478dad2640 1 --2- 172.21.15.5:0/2774605509 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f476c06be30 0x7f476c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.912+0000 7f477effd640 1 -- 172.21.15.5:0/2774605509 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6187+0+0 (secure 0 0 0) 0x7f478408ba10 con 0x7f4774004d40 2024-05-20T18:33:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.912+0000 7f478dad2640 1 --2- 172.21.15.5:0/2774605509 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f476c06be30 0x7f476c06e340 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f478815c8b0 tx=0x7f4778009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:28.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:28.916+0000 7f477effd640 1 -- 172.21.15.5:0/2774605509 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4788158ad0 con 0x7f4774004d40 2024-05-20T18:33:29.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:29.143+0000 7f4790537640 1 -- 172.21.15.5:0/2774605509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f47880b37d0 con 0x7f4774004d40 2024-05-20T18:33:29.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:29.143+0000 7f477effd640 1 -- 172.21.15.5:0/2774605509 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v66) ==== 53+0+73 (secure 0 0 0) 0x7f4784019030 con 0x7f4774004d40 2024-05-20T18:33:29.144 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-20T18:33:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:29.145+0000 7f4790537640 1 -- 172.21.15.5:0/2774605509 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f476c06be30 msgr2=0x7f476c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:29.145+0000 7f4790537640 1 --2- 172.21.15.5:0/2774605509 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f476c06be30 0x7f476c06e340 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f478815c8b0 tx=0x7f4778009040 comp rx=0 tx=0).stop 2024-05-20T18:33:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:29.145+0000 7f4790537640 1 -- 172.21.15.5:0/2774605509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4774004d40 msgr2=0x7f4788155fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:29.145+0000 7f4790537640 1 --2- 172.21.15.5:0/2774605509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4774004d40 0x7f4788155fa0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f47840116d0 tx=0x7f4784011b90 comp rx=0 tx=0).stop 2024-05-20T18:33:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:29.145+0000 7f4790537640 1 -- 172.21.15.5:0/2774605509 shutdown_connections 2024-05-20T18:33:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:29.145+0000 7f4790537640 1 --2- 172.21.15.5:0/2774605509 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f476c06be30 0x7f476c06e340 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:29.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:29.145+0000 7f4790537640 1 --2- 172.21.15.5:0/2774605509 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4788158180 0x7f4788158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:29.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:29.145+0000 7f4790537640 1 --2- 172.21.15.5:0/2774605509 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f478815be10 0x7f47881540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:29.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:29.145+0000 7f4790537640 1 --2- 172.21.15.5:0/2774605509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4774004d40 0x7f4788155fa0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:29.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:29.145+0000 7f4790537640 1 -- 172.21.15.5:0/2774605509 >> 172.21.15.5:0/2774605509 conn(0x7f47880d4bd0 msgr2=0x7f478806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:29.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:29.145+0000 7f4790537640 1 -- 172.21.15.5:0/2774605509 shutdown_connections 2024-05-20T18:33:29.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:29.145+0000 7f4790537640 1 -- 172.21.15.5:0/2774605509 wait complete. 2024-05-20T18:33:29.158 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-20T18:33:29.158 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-20T18:33:29.158 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-05-20T18:33:29.158 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-20T18:33:29.158 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-20T18:33:29.158 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-20T18:33:29.158 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-20T18:33:29.158 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-20T18:33:29.158 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:33:29.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:33:29.159 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-05-20T18:33:29.177 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T18:33:29.177 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T18:33:29.177 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T18:33:29.177 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:33:29.177 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-20T18:33:29.177 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:33:29.177 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:33:29.177 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-05-20T18:33:29.177 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-05-20T18:33:29.178 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:33:29.178 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-05-20T18:33:29.178 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-05-20T18:33:29.178 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-20T18:33:29.178 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:33:29.178 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-05-20T18:33:29.178 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-20T18:33:29.178 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42289sec preferred_lft 42289sec 2024-05-20T18:33:29.178 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-05-20T18:33:29.178 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:33:29.178 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:33:29.178 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:33:29.178 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-05-20T18:33:29.179 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-20T18:33:29.179 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:33:29.179 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-20T18:33:29.179 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:33:29.179 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::4474:eaff:fe07:4f9a/64 scope link 2024-05-20T18:33:29.179 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:33:29.179 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:33:29.179 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T18:33:29.179 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ce4:f4ff:fe5e:7906/64 scope link 2024-05-20T18:33:29.179 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:33:29.179 INFO:teuthology.orchestra.run.smithi005.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:33:29.179 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether da:26:04:0e:95:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T18:33:29.179 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::d826:4ff:fe0e:950a/64 scope link 2024-05-20T18:33:29.179 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:33:29.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:33:29.180 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-20T18:33:29.243 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-20T18:33:29.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:33:29.245 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-05-20T18:33:29.302 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-05-20T18:33:29.302 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-20T18:33:29.302 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-20T18:33:29.302 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-20T18:33:29.302 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-20T18:33:29.302 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:33:29.302 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-20T18:33:29.303 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-20T18:33:29.303 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-20T18:33:29.303 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-20T18:33:29.303 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-20T18:33:29.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:33:29.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:33:29.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:33:29.432 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-20T18:33:29.448 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-20T18:33:29.448 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-20T18:33:29.449 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:33:29.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:33:29.449 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-05-20T18:33:29.514 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:33:29.515 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-20T18:33:29.570 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:33:29.571 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:33:29.571 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:33:29.626 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-20T18:33:29.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:33:29.626 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:33:29.722 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:33:29.723 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:33:29.723 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:33:29.733 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:33:29.733 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:33:29.733 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:33:29.734 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:33:29.734 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:33:29.739 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:33:29.739 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:33:29.739 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:33:29.739 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:33:29.739 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:33:29.739 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790" 2024-05-20T18:33:29.739 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790:/ 2024-05-20T18:33:29.739 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=399ab250-fd3c-47fa-a1c8-25e8149c9cac" will pass to kernel 2024-05-20T18:33:29.740 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-20T18:33:29.740 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-05-20T18:33:29.740 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-20T18:33:29.740 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-05-20T18:33:29.740 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-20T18:33:29.742 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-20T18:33:29.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:33:29.742 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:33:29.768 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-20T18:33:29.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T18:33:29.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:33:29.866 INFO:teuthology.orchestra.run.smithi005.stdout:client7734 2024-05-20T18:33:29.866 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:33:29.867 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:33:29.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:33:29.883 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink) 2024-05-20T18:33:29.941 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-20T18:33:29.941+0000 7fce21072640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-05-20T18:33:29.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T18:33:29.943 INFO:teuthology.orchestra.run.smithi005.stderr:stat: cannot statx 'testdir/hardlink': Input/output error 2024-05-20T18:33:29.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:33:29.943 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive 2024-05-20T18:33:30.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.102+0000 7fd11c273640 1 Processor -- start 2024-05-20T18:33:30.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.102+0000 7fd11c273640 1 -- start start 2024-05-20T18:33:30.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.103+0000 7fd11c273640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1140b6db0 0x7fd1140d3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.103+0000 7fd11c273640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd114160410 con 0x7fd1140d3a90 2024-05-20T18:33:30.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.103+0000 7fd11c273640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd114162e70 con 0x7fd114163950 2024-05-20T18:33:30.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.103+0000 7fd11c273640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd1141606f0 con 0x7fd1140b6db0 2024-05-20T18:33:30.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.103+0000 7fd11a810640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd1140d3a90 0x7fd11415f160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47446/0 (socket says 172.21.15.5:47446) 2024-05-20T18:33:30.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.103+0000 7fd11a810640 1 -- 172.21.15.5:0/3335822644 learned_addr learned my addr 172.21.15.5:0/3335822644 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:33:30.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.103+0000 7fd11980e640 1 --2- 172.21.15.5:0/3335822644 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1140b6db0 0x7fd1140d3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.104+0000 7fd11900d640 1 -- 172.21.15.5:0/3335822644 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2410024339 0 0) 0x7fd114160410 con 0x7fd1140d3a90 2024-05-20T18:33:30.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.104+0000 7fd11900d640 1 -- 172.21.15.5:0/3335822644 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd100003520 con 0x7fd1140d3a90 2024-05-20T18:33:30.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.104+0000 7fd11900d640 1 -- 172.21.15.5:0/3335822644 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1153521131 0 0) 0x7fd100003520 con 0x7fd1140d3a90 2024-05-20T18:33:30.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.104+0000 7fd11900d640 1 -- 172.21.15.5:0/3335822644 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1140b6db0 msgr2=0x7fd1140d3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:30.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.104+0000 7fd11900d640 1 --2- 172.21.15.5:0/3335822644 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1140b6db0 0x7fd1140d3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.104+0000 7fd11900d640 1 -- 172.21.15.5:0/3335822644 >> v1:172.21.15.63:6789/0 conn(0x7fd114163950 legacy=0x7fd1140b66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:30.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.104+0000 7fd11900d640 1 -- 172.21.15.5:0/3335822644 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd114163150 con 0x7fd1140d3a90 2024-05-20T18:33:30.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.104+0000 7fd11900d640 1 -- 172.21.15.5:0/3335822644 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7fd10c0019f0 con 0x7fd1140d3a90 2024-05-20T18:33:30.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.104+0000 7fd11900d640 1 -- 172.21.15.5:0/3335822644 >> v1:172.21.15.5:6789/0 conn(0x7fd1140d3a90 legacy=0x7fd11415f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:30.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.104+0000 7fd11900d640 1 --2- 172.21.15.5:0/3335822644 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1000039e0 0x7fd100003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.104+0000 7fd11900d640 1 --2- 172.21.15.5:0/3335822644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd100004350 0x7fd100004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.104+0000 7fd11900d640 1 --2- 172.21.15.5:0/3335822644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd100005030 0x7fd10000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.104+0000 7fd11900d640 1 -- 172.21.15.5:0/3335822644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd10000bf00 con 0x7fd100005030 2024-05-20T18:33:30.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.104+0000 7fd11900d640 1 -- 172.21.15.5:0/3335822644 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd10000c080 con 0x7fd100004350 2024-05-20T18:33:30.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.104+0000 7fd11900d640 1 -- 172.21.15.5:0/3335822644 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd10000c280 con 0x7fd1000039e0 2024-05-20T18:33:30.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.104+0000 7fd11a810640 1 --2- 172.21.15.5:0/3335822644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd100005030 0x7fd10000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.104+0000 7fd11a00f640 1 --2- 172.21.15.5:0/3335822644 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1000039e0 0x7fd100003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.104+0000 7fd11980e640 1 --2- 172.21.15.5:0/3335822644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd100004350 0x7fd100004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.104+0000 7fd11a810640 1 -- 172.21.15.5:0/3335822644 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1000039e0 msgr2=0x7fd100003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.104+0000 7fd11a810640 1 --2- 172.21.15.5:0/3335822644 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1000039e0 0x7fd100003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.104+0000 7fd11a810640 1 -- 172.21.15.5:0/3335822644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd100004350 msgr2=0x7fd100004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.104+0000 7fd11a810640 1 --2- 172.21.15.5:0/3335822644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd100004350 0x7fd100004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.104+0000 7fd11a810640 1 -- 172.21.15.5:0/3335822644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd100003520 con 0x7fd100005030 2024-05-20T18:33:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.105+0000 7fd11a810640 1 --2- 172.21.15.5:0/3335822644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd100005030 0x7fd10000b9c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fd11408e070 tx=0x7fd100005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d46d59e560753c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.105+0000 7fd11900d640 1 -- 172.21.15.5:0/3335822644 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c00a4a0 con 0x7fd100005030 2024-05-20T18:33:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.105+0000 7fd11900d640 1 -- 172.21.15.5:0/3335822644 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd10c02f070 con 0x7fd100005030 2024-05-20T18:33:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.105+0000 7fd11900d640 1 -- 172.21.15.5:0/3335822644 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c02bc30 con 0x7fd100005030 2024-05-20T18:33:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.106+0000 7fd11c273640 1 -- 172.21.15.5:0/3335822644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd100005030 msgr2=0x7fd10000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.106+0000 7fd11c273640 1 --2- 172.21.15.5:0/3335822644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd100005030 0x7fd10000b9c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fd11408e070 tx=0x7fd100005830 comp rx=0 tx=0).stop 2024-05-20T18:33:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.106+0000 7fd11a810640 1 -- 172.21.15.5:0/3335822644 reap_dead start 2024-05-20T18:33:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.106+0000 7fd11c273640 1 -- 172.21.15.5:0/3335822644 shutdown_connections 2024-05-20T18:33:30.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.106+0000 7fd11c273640 1 -- 172.21.15.5:0/3335822644 >> 172.21.15.5:0/3335822644 conn(0x7fd1140d16a0 msgr2=0x7fd1140d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:30.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.106+0000 7fd11c273640 1 -- 172.21.15.5:0/3335822644 shutdown_connections 2024-05-20T18:33:30.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.106+0000 7fd11c273640 1 -- 172.21.15.5:0/3335822644 wait complete. 2024-05-20T18:33:30.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.108+0000 7fd11c273640 1 Processor -- start 2024-05-20T18:33:30.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.108+0000 7fd11c273640 1 -- start start 2024-05-20T18:33:30.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.109+0000 7fd11c273640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd100005030 0x7fd1140ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.109+0000 7fd11c273640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd11415cf00 0x7fd1140c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.109+0000 7fd11c273640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1140c8c70 0x7fd1141f8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.109+0000 7fd11c273640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1140b6f30 con 0x7fd11415cf00 2024-05-20T18:33:30.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.109+0000 7fd11c273640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd1140b6db0 con 0x7fd100005030 2024-05-20T18:33:30.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.109+0000 7fd11c273640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd1140d3550 con 0x7fd1140c8c70 2024-05-20T18:33:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.109+0000 7fd11980e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd11415cf00 0x7fd1140c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.109+0000 7fd11980e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd11415cf00 0x7fd1140c8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54948/0 (socket says 172.21.15.5:54948) 2024-05-20T18:33:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.109+0000 7fd11980e640 1 -- 172.21.15.5:0/3469093342 learned_addr learned my addr 172.21.15.5:0/3469093342 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.110+0000 7fd11a810640 1 --2- 172.21.15.5:0/3469093342 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1140c8c70 0x7fd1141f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.110+0000 7fd11a00f640 1 --2- 172.21.15.5:0/3469093342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd100005030 0x7fd1140ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.110+0000 7fd11980e640 1 -- 172.21.15.5:0/3469093342 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1140c8c70 msgr2=0x7fd1141f8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.110+0000 7fd11980e640 1 --2- 172.21.15.5:0/3469093342 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1140c8c70 0x7fd1141f8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.110+0000 7fd11980e640 1 -- 172.21.15.5:0/3469093342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd100005030 msgr2=0x7fd1140ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.110+0000 7fd11980e640 1 --2- 172.21.15.5:0/3469093342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd100005030 0x7fd1140ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.110+0000 7fd11980e640 1 -- 172.21.15.5:0/3469093342 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1140c9080 con 0x7fd11415cf00 2024-05-20T18:33:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.110+0000 7fd11a00f640 1 --2- 172.21.15.5:0/3469093342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd100005030 0x7fd1140ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:33:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.110+0000 7fd11980e640 1 --2- 172.21.15.5:0/3469093342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd11415cf00 0x7fd1140c8730 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fd1040119a0 tx=0x7fd104011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.110+0000 7fd10affd640 1 -- 172.21.15.5:0/3469093342 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10400faf0 con 0x7fd11415cf00 2024-05-20T18:33:30.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.110+0000 7fd10affd640 1 -- 172.21.15.5:0/3469093342 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd10400fc90 con 0x7fd11415cf00 2024-05-20T18:33:30.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.110+0000 7fd11c273640 1 -- 172.21.15.5:0/3469093342 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd11415d8a0 con 0x7fd11415cf00 2024-05-20T18:33:30.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.110+0000 7fd10affd640 1 -- 172.21.15.5:0/3469093342 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd104014440 con 0x7fd11415cf00 2024-05-20T18:33:30.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.110+0000 7fd11c273640 1 -- 172.21.15.5:0/3469093342 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1141f6380 con 0x7fd11415cf00 2024-05-20T18:33:30.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.112+0000 7fd10affd640 1 -- 172.21.15.5:0/3469093342 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd104012040 con 0x7fd11415cf00 2024-05-20T18:33:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.112+0000 7fd10affd640 1 --2- 172.21.15.5:0/3469093342 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd0f806be30 0x7fd0f806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.112+0000 7fd10affd640 1 -- 172.21.15.5:0/3469093342 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6187+0+0 (secure 0 0 0) 0x7fd10408b820 con 0x7fd11415cf00 2024-05-20T18:33:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.112+0000 7fd11a00f640 1 --2- 172.21.15.5:0/3469093342 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd0f806be30 0x7fd0f806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.113+0000 7fd11a00f640 1 --2- 172.21.15.5:0/3469093342 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd0f806be30 0x7fd0f806e340 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd110003700 tx=0x7fd110009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.113+0000 7fd1231f6740 1 Processor -- start 2024-05-20T18:33:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.114+0000 7fd1231f6740 1 -- start start 2024-05-20T18:33:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.115+0000 7fd1231f6740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ca38193340 0x55ca3855f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.115+0000 7fd1231f6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca3816a480 0x55ca385348b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.115+0000 7fd1231f6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca38172660 0x55ca3857b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.115+0000 7fd1231f6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ca38497880 con 0x55ca3816a480 2024-05-20T18:33:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.115+0000 7fd1231f6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ca3843f520 con 0x55ca38172660 2024-05-20T18:33:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.115+0000 7fd1231f6740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55ca38493840 con 0x55ca38193340 2024-05-20T18:33:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.115+0000 7fd11980e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca3816a480 0x55ca385348b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.115+0000 7fd11980e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca3816a480 0x55ca385348b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54950/0 (socket says 172.21.15.5:54950) 2024-05-20T18:33:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.115+0000 7fd11980e640 1 -- 172.21.15.5:0/3847917499 learned_addr learned my addr 172.21.15.5:0/3847917499 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.115+0000 7fd11a00f640 1 --2- 172.21.15.5:0/3847917499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ca38193340 0x55ca3855f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.115+0000 7fd11a810640 1 --2- 172.21.15.5:0/3847917499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca38172660 0x55ca3857b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.115+0000 7fd11980e640 1 -- 172.21.15.5:0/3847917499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ca38193340 msgr2=0x55ca3855f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.115+0000 7fd11980e640 1 --2- 172.21.15.5:0/3847917499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ca38193340 0x55ca3855f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.115+0000 7fd11980e640 1 -- 172.21.15.5:0/3847917499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca38172660 msgr2=0x55ca3857b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.115+0000 7fd11980e640 1 --2- 172.21.15.5:0/3847917499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca38172660 0x55ca3857b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.115+0000 7fd11980e640 1 -- 172.21.15.5:0/3847917499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca38433b40 con 0x55ca3816a480 2024-05-20T18:33:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.115+0000 7fd11a810640 1 --2- 172.21.15.5:0/3847917499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca38172660 0x55ca3857b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:33:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.116+0000 7fd11980e640 1 --2- 172.21.15.5:0/3847917499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca3816a480 0x55ca385348b0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fd104089a10 tx=0x7fd10405dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ab2da8d3da18a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.116+0000 7fd0f7fff640 1 -- 172.21.15.5:0/3847917499 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd104057c00 con 0x55ca3816a480 2024-05-20T18:33:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.116+0000 7fd0f7fff640 1 -- 172.21.15.5:0/3847917499 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd104055f70 con 0x55ca3816a480 2024-05-20T18:33:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.116+0000 7fd0f7fff640 1 -- 172.21.15.5:0/3847917499 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1040621c0 con 0x55ca3816a480 2024-05-20T18:33:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.117+0000 7fd1231f6740 1 -- 172.21.15.5:0/3847917499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca3816a480 msgr2=0x55ca385348b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.117+0000 7fd1231f6740 1 --2- 172.21.15.5:0/3847917499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca3816a480 0x55ca385348b0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fd104089a10 tx=0x7fd10405dc00 comp rx=0 tx=0).stop 2024-05-20T18:33:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.117+0000 7fd1231f6740 1 -- 172.21.15.5:0/3847917499 shutdown_connections 2024-05-20T18:33:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.117+0000 7fd1231f6740 1 --2- 172.21.15.5:0/3847917499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca38172660 0x55ca3857b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.117+0000 7fd1231f6740 1 --2- 172.21.15.5:0/3847917499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca3816a480 0x55ca385348b0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.117+0000 7fd1231f6740 1 --2- 172.21.15.5:0/3847917499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ca38193340 0x55ca3855f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.117+0000 7fd1231f6740 1 -- 172.21.15.5:0/3847917499 >> 172.21.15.5:0/3847917499 conn(0x55ca38187490 msgr2=0x55ca38533e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.117+0000 7fd1231f6740 1 -- 172.21.15.5:0/3847917499 shutdown_connections 2024-05-20T18:33:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.117+0000 7fd1231f6740 1 -- 172.21.15.5:0/3847917499 wait complete. 2024-05-20T18:33:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.118+0000 7fd1231f6740 1 -- start start 2024-05-20T18:33:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.118+0000 7fd1231f6740 1 Processor -- start 2024-05-20T18:33:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.119+0000 7fd1231f6740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ca3816a480 0x55ca38559da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.119+0000 7fd1231f6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca38172660 0x55ca3857dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.119+0000 7fd1231f6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca38193340 0x55ca3857e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.119+0000 7fd1231f6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ca38452ab0 con 0x55ca38193340 2024-05-20T18:33:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.119+0000 7fd1231f6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ca3855ede0 con 0x55ca38172660 2024-05-20T18:33:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.119+0000 7fd1231f6740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55ca38553530 con 0x55ca3816a480 2024-05-20T18:33:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.119+0000 7fd1231f6740 10 client.0 _ll_register_callbacks cb 0x55ca381aab90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:33:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.119+0000 7fd11a810640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca38193340 0x55ca3857e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.119+0000 7fd11a810640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca38193340 0x55ca3857e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54962/0 (socket says 172.21.15.5:54962) 2024-05-20T18:33:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.119+0000 7fd11a810640 1 -- 172.21.15.5:0/1217534899 learned_addr learned my addr 172.21.15.5:0/1217534899 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.120+0000 7fd11a00f640 1 --2- 172.21.15.5:0/1217534899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ca3816a480 0x55ca38559da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.120+0000 7fd11980e640 1 --2- 172.21.15.5:0/1217534899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca38172660 0x55ca3857dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.120+0000 7fd11a810640 1 -- 172.21.15.5:0/1217534899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ca3816a480 msgr2=0x55ca38559da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.120+0000 7fd11a810640 1 --2- 172.21.15.5:0/1217534899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ca3816a480 0x55ca38559da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.120+0000 7fd11a810640 1 -- 172.21.15.5:0/1217534899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca38172660 msgr2=0x55ca3857dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.120+0000 7fd11a810640 1 --2- 172.21.15.5:0/1217534899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca38172660 0x55ca3857dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.120+0000 7fd11a810640 1 -- 172.21.15.5:0/1217534899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca382777b0 con 0x55ca38193340 2024-05-20T18:33:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.120+0000 7fd11980e640 1 --2- 172.21.15.5:0/1217534899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca38172660 0x55ca3857dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.120+0000 7fd11a810640 1 --2- 172.21.15.5:0/1217534899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca38193340 0x55ca3857e150 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fd10c001cf0 tx=0x7fd114163040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e39cb9570102e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.120+0000 7fd0f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:33:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.120+0000 7fd0f57fa640 1 -- 172.21.15.5:0/1217534899 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c045060 con 0x55ca38193340 2024-05-20T18:33:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.120+0000 7fd0f57fa640 1 -- 172.21.15.5:0/1217534899 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6187+0+0 (secure 0 0 0) 0x7fd10c02cb10 con 0x55ca38193340 2024-05-20T18:33:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.120+0000 7fd1231f6740 1 -- 172.21.15.5:0/1217534899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd104055f70 con 0x55ca38193340 2024-05-20T18:33:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.120+0000 7fd0f57fa640 1 -- 172.21.15.5:0/1217534899 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fd10c04b050 con 0x55ca38193340 2024-05-20T18:33:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.120+0000 7fd1231f6740 10 client.7746 fetch_fsmap learned FSMap version 66 2024-05-20T18:33:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.120+0000 7fd1231f6740 1 -- 172.21.15.5:0/1217534899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55ca382d7f40 con 0x55ca38193340 2024-05-20T18:33:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.120+0000 7fd0f57fa640 1 -- 172.21.15.5:0/1217534899 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd104055f70 con 0x55ca38193340 2024-05-20T18:33:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.121+0000 7fd0f57fa640 1 -- 172.21.15.5:0/1217534899 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7fd10c04e020 con 0x55ca38193340 2024-05-20T18:33:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.121+0000 7fd1231f6740 10 client.7746 fetch_fsmap finished waiting for FSMap version 66 2024-05-20T18:33:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.121+0000 7fd1231f6740 10 client.7746 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-20T18:33:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.121+0000 7fd1231f6740 20 client.7746 populate_metadata read hostname 'smithi005' 2024-05-20T18:33:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.121+0000 7fd1231f6740 1 --2- 172.21.15.5:0/1217534899 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55ca38152120 0x55ca385525a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.121+0000 7fd1231f6740 4 client.7746 mds_command: new command op to 7437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:33:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.121+0000 7fd1231f6740 1 -- 172.21.15.5:0/1217534899 --> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca384a4fb0 con 0x55ca38152120 2024-05-20T18:33:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.121+0000 7fd11a00f640 1 --2- 172.21.15.5:0/1217534899 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55ca38152120 0x55ca385525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.121+0000 7fd0f57fa640 10 client.7746 ms_handle_connect on v2:172.21.15.5:6836/3556492262 2024-05-20T18:33:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.121+0000 7fd11a00f640 1 --2- 172.21.15.5:0/1217534899 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55ca38152120 0x55ca385525a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=983d9c7b88e5594 server_cookie=a50d9de12dbc7c88 in_seq=0 out_seq=0 2024-05-20T18:33:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.122+0000 7fd0f57fa640 1 -- 172.21.15.5:0/1217534899 <== mds.0 v2:172.21.15.5:6836/3556492262 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ca384a4fb0 con 0x55ca38152120 2024-05-20T18:33:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.122+0000 7fd0f57fa640 10 client.7746 handle_command_reply: tid=0 2024-05-20T18:33:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.122+0000 7fd1231f6740 1 client.7746 shutdown 2024-05-20T18:33:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.122+0000 7fd1231f6740 1 -- 172.21.15.5:0/1217534899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca38193340 msgr2=0x55ca3857e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.122+0000 7fd1231f6740 1 --2- 172.21.15.5:0/1217534899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca38193340 0x55ca3857e150 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fd10c001cf0 tx=0x7fd114163040 comp rx=0 tx=0).stop 2024-05-20T18:33:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.122+0000 7fd1231f6740 1 -- 172.21.15.5:0/1217534899 shutdown_connections 2024-05-20T18:33:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.122+0000 7fd1231f6740 1 --2- 172.21.15.5:0/1217534899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca38193340 0x55ca3857e150 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.122+0000 7fd1231f6740 1 --2- 172.21.15.5:0/1217534899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca38172660 0x55ca3857dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.122+0000 7fd1231f6740 1 --2- 172.21.15.5:0/1217534899 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55ca38152120 0x55ca385525a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.122+0000 7fd1231f6740 1 --2- 172.21.15.5:0/1217534899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ca3816a480 0x55ca38559da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.122+0000 7fd1231f6740 1 -- 172.21.15.5:0/1217534899 >> 172.21.15.5:0/1217534899 conn(0x55ca38187490 msgr2=0x55ca38533e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.122+0000 7fd0f57fa640 0 client.7746 ms_handle_reset on v2:172.21.15.5:6836/3556492262 2024-05-20T18:33:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.122+0000 7fd1231f6740 1 -- 172.21.15.5:0/1217534899 shutdown_connections 2024-05-20T18:33:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.122+0000 7fd1231f6740 1 -- 172.21.15.5:0/1217534899 wait complete. 2024-05-20T18:33:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.122+0000 7fd1231f6740 20 client.7746 trim_cache size 0 max 16384 2024-05-20T18:33:30.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.141+0000 7fd1231f6740 1 Processor -- start 2024-05-20T18:33:30.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.141+0000 7fd1231f6740 1 -- start start 2024-05-20T18:33:30.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.142+0000 7fd1231f6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca38187490 0x55ca38562290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.142+0000 7fd1231f6740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ca3816a480 0x55ca3852e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.142+0000 7fd1231f6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca38172660 0x55ca3857c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.142+0000 7fd1231f6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1142023b0 con 0x55ca38187490 2024-05-20T18:33:30.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.142+0000 7fd1231f6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd1142013b0 con 0x55ca38172660 2024-05-20T18:33:30.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.142+0000 7fd1231f6740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd114200710 con 0x55ca3816a480 2024-05-20T18:33:30.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.142+0000 7fd11a00f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca38187490 0x55ca38562290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.142+0000 7fd11a00f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca38187490 0x55ca38562290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54968/0 (socket says 172.21.15.5:54968) 2024-05-20T18:33:30.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.142+0000 7fd11a00f640 1 -- 172.21.15.5:0/1904789904 learned_addr learned my addr 172.21.15.5:0/1904789904 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:30.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.142+0000 7fd11980e640 1 --2- 172.21.15.5:0/1904789904 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ca3816a480 0x55ca3852e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.142+0000 7fd11a810640 1 --2- 172.21.15.5:0/1904789904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca38172660 0x55ca3857c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.143+0000 7fd11a00f640 1 -- 172.21.15.5:0/1904789904 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ca3816a480 msgr2=0x55ca3852e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:30.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.143+0000 7fd11a00f640 1 --2- 172.21.15.5:0/1904789904 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ca3816a480 0x55ca3852e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.143+0000 7fd11a00f640 1 -- 172.21.15.5:0/1904789904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca38172660 msgr2=0x55ca3857c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:30.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.143+0000 7fd11a00f640 1 --2- 172.21.15.5:0/1904789904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca38172660 0x55ca3857c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.143+0000 7fd11a00f640 1 -- 172.21.15.5:0/1904789904 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca3816b0a0 con 0x55ca38187490 2024-05-20T18:33:30.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.143+0000 7fd11980e640 1 --2- 172.21.15.5:0/1904789904 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ca3816a480 0x55ca3852e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:30.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.143+0000 7fd11a00f640 1 --2- 172.21.15.5:0/1904789904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca38187490 0x55ca38562290 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fd110001710 tx=0x7fd110013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33bce4de9df582e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:30.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.143+0000 7fd0f77fe640 1 -- 172.21.15.5:0/1904789904 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd110024a00 con 0x55ca38187490 2024-05-20T18:33:30.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.143+0000 7fd0f77fe640 1 -- 172.21.15.5:0/1904789904 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd110024ba0 con 0x55ca38187490 2024-05-20T18:33:30.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.143+0000 7fd0f77fe640 1 -- 172.21.15.5:0/1904789904 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1100291d0 con 0x55ca38187490 2024-05-20T18:33:30.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.144+0000 7fd1231f6740 1 -- 172.21.15.5:0/1904789904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca38187490 msgr2=0x55ca38562290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:30.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.144+0000 7fd1231f6740 1 --2- 172.21.15.5:0/1904789904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca38187490 0x55ca38562290 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fd110001710 tx=0x7fd110013000 comp rx=0 tx=0).stop 2024-05-20T18:33:30.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.144+0000 7fd1231f6740 1 -- 172.21.15.5:0/1904789904 shutdown_connections 2024-05-20T18:33:30.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.144+0000 7fd1231f6740 1 --2- 172.21.15.5:0/1904789904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca38172660 0x55ca3857c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.144+0000 7fd1231f6740 1 --2- 172.21.15.5:0/1904789904 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ca3816a480 0x55ca3852e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.144+0000 7fd1231f6740 1 --2- 172.21.15.5:0/1904789904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca38187490 0x55ca38562290 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.144+0000 7fd1231f6740 1 -- 172.21.15.5:0/1904789904 >> 172.21.15.5:0/1904789904 conn(0x55ca3855f3a0 msgr2=0x55ca38533e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:30.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.144+0000 7fd1231f6740 1 -- 172.21.15.5:0/1904789904 shutdown_connections 2024-05-20T18:33:30.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.144+0000 7fd1231f6740 1 -- 172.21.15.5:0/1904789904 wait complete. 2024-05-20T18:33:30.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.145+0000 7fd1231f6740 1 -- start start 2024-05-20T18:33:30.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.145+0000 7fd1231f6740 1 Processor -- start 2024-05-20T18:33:30.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.146+0000 7fd1231f6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca3816a480 0x55ca385f61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.146+0000 7fd1231f6740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ca38172660 0x55ca38561340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.146+0000 7fd1231f6740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca38187490 0x55ca3857c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.146+0000 7fd1231f6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd114200d60 con 0x55ca3816a480 2024-05-20T18:33:30.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.146+0000 7fd1231f6740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ca3856ff50 con 0x55ca38187490 2024-05-20T18:33:30.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.146+0000 7fd1231f6740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55ca385514c0 con 0x55ca38172660 2024-05-20T18:33:30.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.146+0000 7fd1231f6740 10 client.0 _ll_register_callbacks cb 0x7fd1000036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:33:30.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.146+0000 7fd11a00f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca3816a480 0x55ca385f61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.146+0000 7fd11a00f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca3816a480 0x55ca385f61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54972/0 (socket says 172.21.15.5:54972) 2024-05-20T18:33:30.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.146+0000 7fd11a00f640 1 -- 172.21.15.5:0/3419082768 learned_addr learned my addr 172.21.15.5:0/3419082768 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:30.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.147+0000 7fd11a810640 1 --2- 172.21.15.5:0/3419082768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca38187490 0x55ca3857c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.147+0000 7fd11980e640 1 --2- 172.21.15.5:0/3419082768 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ca38172660 0x55ca38561340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.147+0000 7fd11a00f640 1 -- 172.21.15.5:0/3419082768 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ca38172660 msgr2=0x55ca38561340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:30.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.147+0000 7fd11a00f640 1 --2- 172.21.15.5:0/3419082768 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ca38172660 0x55ca38561340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.147+0000 7fd11a00f640 1 -- 172.21.15.5:0/3419082768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca38187490 msgr2=0x55ca3857c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:30.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.147+0000 7fd11a00f640 1 --2- 172.21.15.5:0/3419082768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca38187490 0x55ca3857c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.147+0000 7fd11a00f640 1 -- 172.21.15.5:0/3419082768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca38551710 con 0x55ca3816a480 2024-05-20T18:33:30.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.147+0000 7fd11a810640 1 --2- 172.21.15.5:0/3419082768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca38187490 0x55ca3857c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:33:30.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.147+0000 7fd11980e640 1 --2- 172.21.15.5:0/3419082768 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ca38172660 0x55ca38561340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:33:30.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.147+0000 7fd11a00f640 1 --2- 172.21.15.5:0/3419082768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca3816a480 0x55ca385f61d0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fd110009010 tx=0x7fd110001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a367e35768ea33 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:30.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.147+0000 7fd0f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:33:30.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.147+0000 7fd0f57fa640 1 -- 172.21.15.5:0/3419082768 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd110039070 con 0x55ca3816a480 2024-05-20T18:33:30.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.147+0000 7fd0f57fa640 1 -- 172.21.15.5:0/3419082768 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6187+0+0 (secure 0 0 0) 0x7fd1100293d0 con 0x55ca3816a480 2024-05-20T18:33:30.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.147+0000 7fd1231f6740 1 -- 172.21.15.5:0/3419082768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd110024ba0 con 0x55ca3816a480 2024-05-20T18:33:30.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.147+0000 7fd0f57fa640 1 -- 172.21.15.5:0/3419082768 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fd110044050 con 0x55ca3816a480 2024-05-20T18:33:30.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.147+0000 7fd1231f6740 10 client.7752 fetch_fsmap learned FSMap version 66 2024-05-20T18:33:30.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.147+0000 7fd1231f6740 1 -- 172.21.15.5:0/3419082768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55ca38561880 con 0x55ca3816a480 2024-05-20T18:33:30.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.147+0000 7fd0f57fa640 1 -- 172.21.15.5:0/3419082768 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd110024ba0 con 0x55ca3816a480 2024-05-20T18:33:30.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.148+0000 7fd0f57fa640 1 -- 172.21.15.5:0/3419082768 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7fd110047020 con 0x55ca3816a480 2024-05-20T18:33:30.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.148+0000 7fd1231f6740 10 client.7752 fetch_fsmap finished waiting for FSMap version 66 2024-05-20T18:33:30.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.148+0000 7fd1231f6740 10 client.7752 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-20T18:33:30.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.148+0000 7fd1231f6740 20 client.7752 populate_metadata read hostname 'smithi005' 2024-05-20T18:33:30.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.148+0000 7fd1231f6740 1 --2- 172.21.15.5:0/3419082768 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55ca38127560 0x55ca3857c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.148+0000 7fd1231f6740 4 client.7752 mds_command: new command op to 7437 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}] 2024-05-20T18:33:30.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.148+0000 7fd1231f6740 1 -- 172.21.15.5:0/3419082768 --> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}) -- 0x55ca3849f300 con 0x55ca38127560 2024-05-20T18:33:30.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.148+0000 7fd11980e640 1 --2- 172.21.15.5:0/3419082768 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55ca38127560 0x55ca3857c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.148+0000 7fd0f57fa640 10 client.7752 ms_handle_connect on v2:172.21.15.5:6836/3556492262 2024-05-20T18:33:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.148+0000 7fd11980e640 1 --2- 172.21.15.5:0/3419082768 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55ca38127560 0x55ca3857c540 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bd222cd928fe4ed server_cookie=8a0ac1dee853809a in_seq=0 out_seq=0 2024-05-20T18:33:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.149+0000 7fd0f57fa640 1 -- 172.21.15.5:0/3419082768 <== mds.0 v2:172.21.15.5:6836/3556492262 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ca3849f300 con 0x55ca38127560 2024-05-20T18:33:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.149+0000 7fd0f57fa640 10 client.7752 handle_command_reply: tid=0 2024-05-20T18:33:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.149+0000 7fd1231f6740 1 client.7752 shutdown 2024-05-20T18:33:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.149+0000 7fd1231f6740 1 -- 172.21.15.5:0/3419082768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca3816a480 msgr2=0x55ca385f61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.149+0000 7fd1231f6740 1 --2- 172.21.15.5:0/3419082768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca3816a480 0x55ca385f61d0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fd110009010 tx=0x7fd110001230 comp rx=0 tx=0).stop 2024-05-20T18:33:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.149+0000 7fd1231f6740 1 -- 172.21.15.5:0/3419082768 shutdown_connections 2024-05-20T18:33:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.149+0000 7fd1231f6740 1 --2- 172.21.15.5:0/3419082768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ca38187490 0x55ca3857c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.149+0000 7fd1231f6740 1 --2- 172.21.15.5:0/3419082768 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55ca38127560 0x55ca3857c540 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.149+0000 7fd1231f6740 1 --2- 172.21.15.5:0/3419082768 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ca38172660 0x55ca38561340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.149+0000 7fd1231f6740 1 --2- 172.21.15.5:0/3419082768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca3816a480 0x55ca385f61d0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.149+0000 7fd0f57fa640 0 client.7752 ms_handle_reset on v2:172.21.15.5:6836/3556492262 2024-05-20T18:33:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.149+0000 7fd1231f6740 1 -- 172.21.15.5:0/3419082768 >> 172.21.15.5:0/3419082768 conn(0x55ca3855f3a0 msgr2=0x55ca38544570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:30.155 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-20T18:33:30.149+0000 7fce21072640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-05-20T18:33:20.165942+0000 1=1+0) n(v0 rc2024-05-20T18:33:20.285940+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x5555f6279b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-05-20T18:33:20.165942+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-05-20T18:33:20.285940+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-05-20T18:33:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.149+0000 7fd1231f6740 1 -- 172.21.15.5:0/3419082768 shutdown_connections 2024-05-20T18:33:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.149+0000 7fd1231f6740 1 -- 172.21.15.5:0/3419082768 wait complete. 2024-05-20T18:33:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.149+0000 7fd1231f6740 20 client.7752 trim_cache size 0 max 16384 2024-05-20T18:33:30.156 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:33:30.156 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-20T18:33:30.156 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "1f442c1f-44a4-44d6-8681-aded211e589d", 2024-05-20T18:33:30.156 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-20T18:33:30.156 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:33:30.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.150+0000 7fd11c273640 1 -- 172.21.15.5:0/3469093342 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd0f806be30 msgr2=0x7fd0f806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:30.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.150+0000 7fd11c273640 1 --2- 172.21.15.5:0/3469093342 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd0f806be30 0x7fd0f806e340 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd110003700 tx=0x7fd110009040 comp rx=0 tx=0).stop 2024-05-20T18:33:30.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.150+0000 7fd11c273640 1 -- 172.21.15.5:0/3469093342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd11415cf00 msgr2=0x7fd1140c8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:30.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.150+0000 7fd11c273640 1 --2- 172.21.15.5:0/3469093342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd11415cf00 0x7fd1140c8730 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fd1040119a0 tx=0x7fd104011e60 comp rx=0 tx=0).stop 2024-05-20T18:33:30.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.150+0000 7fd11c273640 1 -- 172.21.15.5:0/3469093342 shutdown_connections 2024-05-20T18:33:30.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.150+0000 7fd11c273640 1 --2- 172.21.15.5:0/3469093342 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd0f806be30 0x7fd0f806e340 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.150+0000 7fd11c273640 1 --2- 172.21.15.5:0/3469093342 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1140c8c70 0x7fd1141f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.150+0000 7fd11c273640 1 --2- 172.21.15.5:0/3469093342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd11415cf00 0x7fd1140c8730 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.150+0000 7fd11c273640 1 --2- 172.21.15.5:0/3469093342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd100005030 0x7fd1140ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.150+0000 7fd11c273640 1 -- 172.21.15.5:0/3469093342 >> 172.21.15.5:0/3469093342 conn(0x7fd1140d16a0 msgr2=0x7fd1140b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:30.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.150+0000 7fd11c273640 1 -- 172.21.15.5:0/3469093342 shutdown_connections 2024-05-20T18:33:30.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.150+0000 7fd11c273640 1 -- 172.21.15.5:0/3469093342 wait complete. 2024-05-20T18:33:30.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:33:30.162 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-20T18:33:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.279+0000 7fea123a3640 1 Processor -- start 2024-05-20T18:33:30.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.279+0000 7fea123a3640 1 -- start start 2024-05-20T18:33:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.280+0000 7fea123a3640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fea0c0b1e10 0x7fea0c15aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.280+0000 7fea123a3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea0c070850 0x7fea0c070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.280+0000 7fea123a3640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea0c0cd580 con 0x7fea0c15d530 2024-05-20T18:33:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.280+0000 7fea123a3640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fea0c0ce190 con 0x7fea0c0b1e10 2024-05-20T18:33:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.280+0000 7fea123a3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fea0c0cd860 con 0x7fea0c070850 2024-05-20T18:33:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.280+0000 7fea0bfff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fea0c15d530 0x7fea0c0b1700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47460/0 (socket says 172.21.15.5:47460) 2024-05-20T18:33:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.280+0000 7fea0bfff640 1 -- 172.21.15.5:0/2702838063 learned_addr learned my addr 172.21.15.5:0/2702838063 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:33:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.280+0000 7fea10940640 1 --2- 172.21.15.5:0/2702838063 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea0c070850 0x7fea0c070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.280+0000 7fea0b7fe640 1 --2- 172.21.15.5:0/2702838063 >> v2:172.21.15.63:3300/0 conn(0x7fea0c0b1e10 0x7fea0c15aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.281+0000 7fea0affd640 1 -- 172.21.15.5:0/2702838063 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2231648924 0 0) 0x7fea0c0cd580 con 0x7fea0c15d530 2024-05-20T18:33:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.281+0000 7fea0affd640 1 -- 172.21.15.5:0/2702838063 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9f4003520 con 0x7fea0c15d530 2024-05-20T18:33:30.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.281+0000 7fea0affd640 1 -- 172.21.15.5:0/2702838063 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3980843259 0 0) 0x7fe9f4003520 con 0x7fea0c15d530 2024-05-20T18:33:30.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.281+0000 7fea0affd640 1 -- 172.21.15.5:0/2702838063 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea0c070850 msgr2=0x7fea0c070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:30.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.281+0000 7fea0affd640 1 --2- 172.21.15.5:0/2702838063 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea0c070850 0x7fea0c070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.281+0000 7fea0affd640 1 -- 172.21.15.5:0/2702838063 >> v2:172.21.15.63:3300/0 conn(0x7fea0c0b1e10 msgr2=0x7fea0c15aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:30.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.281+0000 7fea0affd640 1 --2- 172.21.15.5:0/2702838063 >> v2:172.21.15.63:3300/0 conn(0x7fea0c0b1e10 0x7fea0c15aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.281+0000 7fea0affd640 1 -- 172.21.15.5:0/2702838063 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea0c0cdb40 con 0x7fea0c15d530 2024-05-20T18:33:30.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.281+0000 7fea0affd640 1 -- 172.21.15.5:0/2702838063 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7fe9fc001670 con 0x7fea0c15d530 2024-05-20T18:33:30.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.281+0000 7fea10940640 1 --2- 172.21.15.5:0/2702838063 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea0c070850 0x7fea0c070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:33:30.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.281+0000 7fea0affd640 1 -- 172.21.15.5:0/2702838063 >> v1:172.21.15.5:6789/0 conn(0x7fea0c15d530 legacy=0x7fea0c0b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:30.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.281+0000 7fea0affd640 1 --2- 172.21.15.5:0/2702838063 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea0c070850 0x7fe9f40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.281+0000 7fea0affd640 1 --2- 172.21.15.5:0/2702838063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9f4003f10 0x7fe9f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.281+0000 7fea0affd640 1 --2- 172.21.15.5:0/2702838063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9f4004d40 0x7fe9f4005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.281+0000 7fea0affd640 1 -- 172.21.15.5:0/2702838063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c0ce190 con 0x7fe9f4003f10 2024-05-20T18:33:30.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.281+0000 7fea0affd640 1 -- 172.21.15.5:0/2702838063 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c0cd860 con 0x7fe9f4004d40 2024-05-20T18:33:30.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.281+0000 7fea0affd640 1 -- 172.21.15.5:0/2702838063 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe9f400b9f0 con 0x7fea0c070850 2024-05-20T18:33:30.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.281+0000 7fea0b7fe640 1 --2- 172.21.15.5:0/2702838063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9f4003f10 0x7fe9f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.281+0000 7fea0bfff640 1 --2- 172.21.15.5:0/2702838063 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea0c070850 0x7fe9f40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.281+0000 7fea10940640 1 --2- 172.21.15.5:0/2702838063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9f4004d40 0x7fe9f4005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.281+0000 7fea0b7fe640 1 -- 172.21.15.5:0/2702838063 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea0c070850 msgr2=0x7fe9f40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:30.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.281+0000 7fea0b7fe640 1 --2- 172.21.15.5:0/2702838063 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea0c070850 0x7fe9f40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.281+0000 7fea0b7fe640 1 -- 172.21.15.5:0/2702838063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9f4004d40 msgr2=0x7fe9f4005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:30.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.281+0000 7fea0b7fe640 1 --2- 172.21.15.5:0/2702838063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9f4004d40 0x7fe9f4005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.281+0000 7fea0b7fe640 1 -- 172.21.15.5:0/2702838063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f4003520 con 0x7fe9f4003f10 2024-05-20T18:33:30.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.282+0000 7fea0b7fe640 1 --2- 172.21.15.5:0/2702838063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9f4003f10 0x7fe9f4004800 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fea00009ac0 tx=0x7fe9f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e439da1393543d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:30.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.282+0000 7fea0affd640 1 -- 172.21.15.5:0/2702838063 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00036070 con 0x7fe9f4003f10 2024-05-20T18:33:30.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.282+0000 7fea0affd640 1 -- 172.21.15.5:0/2702838063 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea00031400 con 0x7fe9f4003f10 2024-05-20T18:33:30.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.282+0000 7fea0affd640 1 -- 172.21.15.5:0/2702838063 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00009690 con 0x7fe9f4003f10 2024-05-20T18:33:30.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.283+0000 7fea123a3640 1 -- 172.21.15.5:0/2702838063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9f4003f10 msgr2=0x7fe9f4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:30.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.283+0000 7fea123a3640 1 --2- 172.21.15.5:0/2702838063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9f4003f10 0x7fe9f4004800 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fea00009ac0 tx=0x7fe9f4005830 comp rx=0 tx=0).stop 2024-05-20T18:33:30.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.283+0000 7fea10940640 1 -- 172.21.15.5:0/2702838063 reap_dead start 2024-05-20T18:33:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.283+0000 7fea123a3640 1 -- 172.21.15.5:0/2702838063 shutdown_connections 2024-05-20T18:33:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.283+0000 7fea123a3640 1 -- 172.21.15.5:0/2702838063 >> 172.21.15.5:0/2702838063 conn(0x7fea0c0d29d0 msgr2=0x7fea0c0d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.283+0000 7fea123a3640 1 -- 172.21.15.5:0/2702838063 shutdown_connections 2024-05-20T18:33:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.283+0000 7fea123a3640 1 -- 172.21.15.5:0/2702838063 wait complete. 2024-05-20T18:33:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.285+0000 7fea123a3640 1 Processor -- start 2024-05-20T18:33:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.285+0000 7fea123a3640 1 -- start start 2024-05-20T18:33:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.286+0000 7fea123a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9f4003f10 0x7fea0c14dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.286+0000 7fea123a3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea0c0c6e30 0x7fea0c0c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.286+0000 7fea123a3640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea0c15aaa0 0x7fea0c15aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.286+0000 7fea123a3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c070c60 con 0x7fe9f4003f10 2024-05-20T18:33:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.286+0000 7fea123a3640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c0cd230 con 0x7fea0c15aaa0 2024-05-20T18:33:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.286+0000 7fea123a3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fea0c070de0 con 0x7fea0c0c6e30 2024-05-20T18:33:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.287+0000 7fea0bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9f4003f10 0x7fea0c14dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.287+0000 7fea0bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9f4003f10 0x7fea0c14dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54978/0 (socket says 172.21.15.5:54978) 2024-05-20T18:33:30.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.287+0000 7fea0bfff640 1 -- 172.21.15.5:0/1837472925 learned_addr learned my addr 172.21.15.5:0/1837472925 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:30.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.287+0000 7fea10940640 1 --2- 172.21.15.5:0/1837472925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea0c15aaa0 0x7fea0c15aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.287+0000 7fea0b7fe640 1 --2- 172.21.15.5:0/1837472925 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea0c0c6e30 0x7fea0c0c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.287+0000 7fea0bfff640 1 -- 172.21.15.5:0/1837472925 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea0c0c6e30 msgr2=0x7fea0c0c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:30.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.287+0000 7fea0bfff640 1 --2- 172.21.15.5:0/1837472925 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea0c0c6e30 0x7fea0c0c7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.287+0000 7fea0bfff640 1 -- 172.21.15.5:0/1837472925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea0c15aaa0 msgr2=0x7fea0c15aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:30.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.287+0000 7fea0bfff640 1 --2- 172.21.15.5:0/1837472925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea0c15aaa0 0x7fea0c15aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.287+0000 7fea0bfff640 1 -- 172.21.15.5:0/1837472925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea0c14e240 con 0x7fe9f4003f10 2024-05-20T18:33:30.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.287+0000 7fea0bfff640 1 --2- 172.21.15.5:0/1837472925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9f4003f10 0x7fea0c14dd00 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc004f70 tx=0x7fe9fc0074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:30.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.287+0000 7fea08ff9640 1 -- 172.21.15.5:0/1837472925 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc00f750 con 0x7fe9f4003f10 2024-05-20T18:33:30.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.287+0000 7fea08ff9640 1 -- 172.21.15.5:0/1837472925 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9fc018070 con 0x7fe9f4003f10 2024-05-20T18:33:30.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.287+0000 7fea123a3640 1 -- 172.21.15.5:0/1837472925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea0c15bd90 con 0x7fe9f4003f10 2024-05-20T18:33:30.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.287+0000 7fea08ff9640 1 -- 172.21.15.5:0/1837472925 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc00fb30 con 0x7fe9f4003f10 2024-05-20T18:33:30.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.287+0000 7fea123a3640 1 -- 172.21.15.5:0/1837472925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea0c15c350 con 0x7fe9f4003f10 2024-05-20T18:33:30.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.289+0000 7fea08ff9640 1 -- 172.21.15.5:0/1837472925 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe9fc00fcd0 con 0x7fe9f4003f10 2024-05-20T18:33:30.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.289+0000 7fea08ff9640 1 --2- 172.21.15.5:0/1837472925 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe9f806be30 0x7fe9f806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.289+0000 7fea08ff9640 1 -- 172.21.15.5:0/1837472925 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6187+0+0 (secure 0 0 0) 0x7fe9fc08b0f0 con 0x7fe9f4003f10 2024-05-20T18:33:30.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.289+0000 7fea0b7fe640 1 --2- 172.21.15.5:0/1837472925 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe9f806be30 0x7fe9f806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.290+0000 7fea0b7fe640 1 --2- 172.21.15.5:0/1837472925 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe9f806be30 0x7fe9f806e340 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fea00033000 tx=0x7fea00008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:30.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.290+0000 7fea19245740 1 Processor -- start 2024-05-20T18:33:30.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.291+0000 7fea19245740 1 -- start start 2024-05-20T18:33:30.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.292+0000 7fea19245740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f741f0e390 0x55f7422d8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.292+0000 7fea19245740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f741ee54d0 0x55f7422f3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.292+0000 7fea19245740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f741eed6b0 0x55f7422f3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.292+0000 7fea19245740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f742212d00 con 0x55f741f0e390 2024-05-20T18:33:30.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.292+0000 7fea19245740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f7421ba4c0 con 0x55f741eed6b0 2024-05-20T18:33:30.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.292+0000 7fea19245740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55f74220ecc0 con 0x55f741ee54d0 2024-05-20T18:33:30.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.292+0000 7fea0bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f741f0e390 0x55f7422d8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.292+0000 7fea0bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f741f0e390 0x55f7422d8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54980/0 (socket says 172.21.15.5:54980) 2024-05-20T18:33:30.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.292+0000 7fea0bfff640 1 -- 172.21.15.5:0/2586483787 learned_addr learned my addr 172.21.15.5:0/2586483787 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:30.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.292+0000 7fea10940640 1 --2- 172.21.15.5:0/2586483787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f741eed6b0 0x55f7422f3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.292+0000 7fea0b7fe640 1 --2- 172.21.15.5:0/2586483787 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f741ee54d0 0x55f7422f3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.292+0000 7fea0bfff640 1 -- 172.21.15.5:0/2586483787 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f741ee54d0 msgr2=0x55f7422f3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:30.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.292+0000 7fea0bfff640 1 --2- 172.21.15.5:0/2586483787 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f741ee54d0 0x55f7422f3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.292+0000 7fea0bfff640 1 -- 172.21.15.5:0/2586483787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f741eed6b0 msgr2=0x55f7422f3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:30.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.292+0000 7fea0bfff640 1 --2- 172.21.15.5:0/2586483787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f741eed6b0 0x55f7422f3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.292+0000 7fea0bfff640 1 -- 172.21.15.5:0/2586483787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f741ff07d0 con 0x55f741f0e390 2024-05-20T18:33:30.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.292+0000 7fea0bfff640 1 --2- 172.21.15.5:0/2586483787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f741f0e390 0x55f7422d8b60 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc055630 tx=0x7fe9fc0591d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6145cb4f5537dddb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:30.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.292+0000 7fe9f1ffb640 1 -- 172.21.15.5:0/2586483787 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc061d00 con 0x55f741f0e390 2024-05-20T18:33:30.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.293+0000 7fe9f1ffb640 1 -- 172.21.15.5:0/2586483787 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9fc0574d0 con 0x55f741f0e390 2024-05-20T18:33:30.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.293+0000 7fe9f1ffb640 1 -- 172.21.15.5:0/2586483787 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc055840 con 0x55f741f0e390 2024-05-20T18:33:30.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.293+0000 7fea19245740 1 -- 172.21.15.5:0/2586483787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f741f0e390 msgr2=0x55f7422d8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:30.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.293+0000 7fea19245740 1 --2- 172.21.15.5:0/2586483787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f741f0e390 0x55f7422d8b60 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc055630 tx=0x7fe9fc0591d0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.293+0000 7fea19245740 1 -- 172.21.15.5:0/2586483787 shutdown_connections 2024-05-20T18:33:30.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.293+0000 7fea19245740 1 --2- 172.21.15.5:0/2586483787 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f741eed6b0 0x55f7422f3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.293+0000 7fea19245740 1 --2- 172.21.15.5:0/2586483787 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f741ee54d0 0x55f7422f3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.293+0000 7fea19245740 1 --2- 172.21.15.5:0/2586483787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f741f0e390 0x55f7422d8b60 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.293+0000 7fea19245740 1 -- 172.21.15.5:0/2586483787 >> 172.21.15.5:0/2586483787 conn(0x55f741f024e0 msgr2=0x55f7422ae1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:30.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.293+0000 7fea19245740 1 -- 172.21.15.5:0/2586483787 shutdown_connections 2024-05-20T18:33:30.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.293+0000 7fea19245740 1 -- 172.21.15.5:0/2586483787 wait complete. 2024-05-20T18:33:30.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.294+0000 7fea19245740 1 -- start start 2024-05-20T18:33:30.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.294+0000 7fea19245740 1 Processor -- start 2024-05-20T18:33:30.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.296+0000 7fea19245740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f741ee54d0 0x55f7422db750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.296+0000 7fea19245740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f741eed6b0 0x55f7422dbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.296+0000 7fea19245740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f741f0e390 0x55f7422eb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.296+0000 7fea19245740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f7421cdb90 con 0x55f741ee54d0 2024-05-20T18:33:30.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.296+0000 7fea19245740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f7422dc1d0 con 0x55f741eed6b0 2024-05-20T18:33:30.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.296+0000 7fea19245740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55f7422dc630 con 0x55f741f0e390 2024-05-20T18:33:30.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.296+0000 7fea19245740 10 client.0 _ll_register_callbacks cb 0x55f742209da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:33:30.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.296+0000 7fea0bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f741ee54d0 0x55f7422db750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.296+0000 7fea0bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f741ee54d0 0x55f7422db750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54992/0 (socket says 172.21.15.5:54992) 2024-05-20T18:33:30.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.296+0000 7fea0bfff640 1 -- 172.21.15.5:0/1968611476 learned_addr learned my addr 172.21.15.5:0/1968611476 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:30.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.296+0000 7fea0b7fe640 1 --2- 172.21.15.5:0/1968611476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f741eed6b0 0x55f7422dbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.296+0000 7fea10940640 1 --2- 172.21.15.5:0/1968611476 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f741f0e390 0x55f7422eb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.296+0000 7fea0bfff640 1 -- 172.21.15.5:0/1968611476 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f741f0e390 msgr2=0x55f7422eb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:30.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.296+0000 7fea0bfff640 1 --2- 172.21.15.5:0/1968611476 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f741f0e390 0x55f7422eb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.296+0000 7fea0bfff640 1 -- 172.21.15.5:0/1968611476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f741eed6b0 msgr2=0x55f7422dbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:30.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.296+0000 7fea0bfff640 1 --2- 172.21.15.5:0/1968611476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f741eed6b0 0x55f7422dbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.296+0000 7fea0bfff640 1 -- 172.21.15.5:0/1968611476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f741f263f0 con 0x55f741ee54d0 2024-05-20T18:33:30.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.296+0000 7fea0b7fe640 1 --2- 172.21.15.5:0/1968611476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f741eed6b0 0x55f7422dbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:33:30.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.296+0000 7fea0bfff640 1 --2- 172.21.15.5:0/1968611476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f741ee54d0 0x55f7422db750 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc00f5a0 tx=0x7fe9fc01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa560b6664b8920c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:30.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.296+0000 7fe9eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:33:30.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.297+0000 7fe9eaffd640 1 -- 172.21.15.5:0/1968611476 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc058620 con 0x55f741ee54d0 2024-05-20T18:33:30.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.297+0000 7fe9eaffd640 1 -- 172.21.15.5:0/1968611476 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6187+0+0 (secure 0 0 0) 0x7fe9fc0587c0 con 0x55f741ee54d0 2024-05-20T18:33:30.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.297+0000 7fea19245740 1 -- 172.21.15.5:0/1968611476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9fc0574d0 con 0x55f741ee54d0 2024-05-20T18:33:30.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.297+0000 7fe9eaffd640 1 -- 172.21.15.5:0/1968611476 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fe9fc077050 con 0x55f741ee54d0 2024-05-20T18:33:30.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.297+0000 7fea19245740 10 client.7764 fetch_fsmap learned FSMap version 66 2024-05-20T18:33:30.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.297+0000 7fea19245740 1 -- 172.21.15.5:0/1968611476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55f741ee60f0 con 0x55f741ee54d0 2024-05-20T18:33:30.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.297+0000 7fe9eaffd640 1 -- 172.21.15.5:0/1968611476 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9fc0574d0 con 0x55f741ee54d0 2024-05-20T18:33:30.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.297+0000 7fe9eaffd640 1 -- 172.21.15.5:0/1968611476 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7fe9fc054f20 con 0x55f741ee54d0 2024-05-20T18:33:30.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.297+0000 7fea19245740 10 client.7764 fetch_fsmap finished waiting for FSMap version 66 2024-05-20T18:33:30.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.297+0000 7fea19245740 10 client.7764 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-20T18:33:30.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.297+0000 7fea19245740 20 client.7764 populate_metadata read hostname 'smithi005' 2024-05-20T18:33:30.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.297+0000 7fea19245740 1 --2- 172.21.15.5:0/1968611476 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55f741eccf60 0x55f7422eba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.297+0000 7fea19245740 4 client.7764 mds_command: new command op to 7437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:33:30.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.297+0000 7fea19245740 1 -- 172.21.15.5:0/1968611476 --> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f742220430 con 0x55f741eccf60 2024-05-20T18:33:30.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.297+0000 7fea0b7fe640 1 --2- 172.21.15.5:0/1968611476 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55f741eccf60 0x55f7422eba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.298+0000 7fe9eaffd640 10 client.7764 ms_handle_connect on v2:172.21.15.5:6836/3556492262 2024-05-20T18:33:30.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.298+0000 7fea0b7fe640 1 --2- 172.21.15.5:0/1968611476 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55f741eccf60 0x55f7422eba10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e72ce086ca0459e server_cookie=ee9a2e2b5e3561f5 in_seq=0 out_seq=0 2024-05-20T18:33:30.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.298+0000 7fe9eaffd640 1 -- 172.21.15.5:0/1968611476 <== mds.0 v2:172.21.15.5:6836/3556492262 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f742220430 con 0x55f741eccf60 2024-05-20T18:33:30.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.298+0000 7fe9eaffd640 10 client.7764 handle_command_reply: tid=0 2024-05-20T18:33:30.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.298+0000 7fea19245740 1 client.7764 shutdown 2024-05-20T18:33:30.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.299+0000 7fea19245740 1 -- 172.21.15.5:0/1968611476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f741ee54d0 msgr2=0x55f7422db750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:30.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.299+0000 7fea19245740 1 --2- 172.21.15.5:0/1968611476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f741ee54d0 0x55f7422db750 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc00f5a0 tx=0x7fe9fc01d000 comp rx=0 tx=0).stop 2024-05-20T18:33:30.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.299+0000 7fea19245740 1 -- 172.21.15.5:0/1968611476 shutdown_connections 2024-05-20T18:33:30.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.299+0000 7fea19245740 1 --2- 172.21.15.5:0/1968611476 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55f741eccf60 0x55f7422eba10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.299+0000 7fea19245740 1 --2- 172.21.15.5:0/1968611476 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f741f0e390 0x55f7422eb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.299+0000 7fea19245740 1 --2- 172.21.15.5:0/1968611476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f741eed6b0 0x55f7422dbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.299+0000 7fea19245740 1 --2- 172.21.15.5:0/1968611476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f741ee54d0 0x55f7422db750 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.299+0000 7fe9eaffd640 0 client.7764 ms_handle_reset on v2:172.21.15.5:6836/3556492262 2024-05-20T18:33:30.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.299+0000 7fea19245740 1 -- 172.21.15.5:0/1968611476 >> 172.21.15.5:0/1968611476 conn(0x55f741f024e0 msgr2=0x55f7422ae1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:30.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.299+0000 7fea19245740 1 -- 172.21.15.5:0/1968611476 shutdown_connections 2024-05-20T18:33:30.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.299+0000 7fea19245740 1 -- 172.21.15.5:0/1968611476 wait complete. 2024-05-20T18:33:30.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.299+0000 7fea19245740 20 client.7764 trim_cache size 0 max 16384 2024-05-20T18:33:30.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.317+0000 7fea19245740 1 Processor -- start 2024-05-20T18:33:30.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.317+0000 7fea19245740 1 -- start start 2024-05-20T18:33:30.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.318+0000 7fea19245740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f741ee54d0 0x55f742366dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.318+0000 7fea19245740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f741eed6b0 0x55f7422d4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.318+0000 7fea19245740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f741f0e390 0x55f7422d4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.318+0000 7fea19245740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f7422f45a0 con 0x55f741eed6b0 2024-05-20T18:33:30.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.318+0000 7fea19245740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c2007d0 con 0x55f741f0e390 2024-05-20T18:33:30.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.318+0000 7fea19245740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fea0c1ff7d0 con 0x55f741ee54d0 2024-05-20T18:33:30.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.319+0000 7fea0b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f741eed6b0 0x55f7422d4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.319+0000 7fea0b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f741eed6b0 0x55f7422d4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55008/0 (socket says 172.21.15.5:55008) 2024-05-20T18:33:30.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.319+0000 7fea0b7fe640 1 -- 172.21.15.5:0/2241471920 learned_addr learned my addr 172.21.15.5:0/2241471920 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:30.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.319+0000 7fea0b7fe640 1 -- 172.21.15.5:0/2241471920 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f741ee54d0 msgr2=0x55f742366dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:30.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.319+0000 7fea0b7fe640 1 --2- 172.21.15.5:0/2241471920 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f741ee54d0 0x55f742366dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.319+0000 7fea0b7fe640 1 -- 172.21.15.5:0/2241471920 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f741f0e390 msgr2=0x55f7422d4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:30.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.319+0000 7fea0b7fe640 1 --2- 172.21.15.5:0/2241471920 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f741f0e390 0x55f7422d4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.319+0000 7fea0b7fe640 1 -- 172.21.15.5:0/2241471920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7422d9630 con 0x55f741eed6b0 2024-05-20T18:33:30.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.319+0000 7fea0b7fe640 1 --2- 172.21.15.5:0/2241471920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f741eed6b0 0x55f7422d4860 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fea0002abc0 tx=0x7fea0002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509fef4c25b06944 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:30.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.319+0000 7fe9f17fa640 1 -- 172.21.15.5:0/2241471920 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00055070 con 0x55f741eed6b0 2024-05-20T18:33:30.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.319+0000 7fe9f17fa640 1 -- 172.21.15.5:0/2241471920 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea00037040 con 0x55f741eed6b0 2024-05-20T18:33:30.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.319+0000 7fe9f17fa640 1 -- 172.21.15.5:0/2241471920 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00036050 con 0x55f741eed6b0 2024-05-20T18:33:30.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.320+0000 7fea19245740 1 -- 172.21.15.5:0/2241471920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f741eed6b0 msgr2=0x55f7422d4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:30.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.320+0000 7fea19245740 1 --2- 172.21.15.5:0/2241471920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f741eed6b0 0x55f7422d4860 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fea0002abc0 tx=0x7fea0002e000 comp rx=0 tx=0).stop 2024-05-20T18:33:30.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.320+0000 7fea19245740 1 -- 172.21.15.5:0/2241471920 shutdown_connections 2024-05-20T18:33:30.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.320+0000 7fea19245740 1 --2- 172.21.15.5:0/2241471920 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f741f0e390 0x55f7422d4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.320+0000 7fea19245740 1 --2- 172.21.15.5:0/2241471920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f741eed6b0 0x55f7422d4860 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.320+0000 7fea19245740 1 --2- 172.21.15.5:0/2241471920 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f741ee54d0 0x55f742366dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.320+0000 7fea19245740 1 -- 172.21.15.5:0/2241471920 >> 172.21.15.5:0/2241471920 conn(0x55f741f024e0 msgr2=0x55f7422f5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:30.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.320+0000 7fea19245740 1 -- 172.21.15.5:0/2241471920 shutdown_connections 2024-05-20T18:33:30.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.320+0000 7fea19245740 1 -- 172.21.15.5:0/2241471920 wait complete. 2024-05-20T18:33:30.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.321+0000 7fea19245740 1 -- start start 2024-05-20T18:33:30.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.321+0000 7fea19245740 1 Processor -- start 2024-05-20T18:33:30.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.322+0000 7fea19245740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f741ee54d0 0x55f7423658d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.322+0000 7fea19245740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f741eed6b0 0x55f74230d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.322+0000 7fea19245740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f741f0e390 0x55f7422ae890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.322+0000 7fea19245740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c1ff180 con 0x55f741f0e390 2024-05-20T18:33:30.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.322+0000 7fea19245740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c0cab80 con 0x55f741eed6b0 2024-05-20T18:33:30.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.322+0000 7fea19245740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55f7422f45a0 con 0x55f741ee54d0 2024-05-20T18:33:30.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.322+0000 7fea19245740 10 client.0 _ll_register_callbacks cb 0x7fe9fc058da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:33:30.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.323+0000 7fea10940640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f741f0e390 0x55f7422ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.323+0000 7fea10940640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f741f0e390 0x55f7422ae890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55014/0 (socket says 172.21.15.5:55014) 2024-05-20T18:33:30.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.323+0000 7fea10940640 1 -- 172.21.15.5:0/1699752846 learned_addr learned my addr 172.21.15.5:0/1699752846 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:33:30.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.323+0000 7fea0bfff640 1 --2- 172.21.15.5:0/1699752846 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f741ee54d0 0x55f7423658d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.323+0000 7fea0b7fe640 1 --2- 172.21.15.5:0/1699752846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f741eed6b0 0x55f74230d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.323+0000 7fea10940640 1 -- 172.21.15.5:0/1699752846 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f741ee54d0 msgr2=0x55f7423658d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:30.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.323+0000 7fea10940640 1 --2- 172.21.15.5:0/1699752846 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f741ee54d0 0x55f7423658d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.323+0000 7fea10940640 1 -- 172.21.15.5:0/1699752846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f741eed6b0 msgr2=0x55f74230d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:33:30.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.323+0000 7fea10940640 1 --2- 172.21.15.5:0/1699752846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f741eed6b0 0x55f74230d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.323+0000 7fea10940640 1 -- 172.21.15.5:0/1699752846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f74230d8f0 con 0x55f741f0e390 2024-05-20T18:33:30.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.323+0000 7fea10940640 1 --2- 172.21.15.5:0/1699752846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f741f0e390 0x55f7422ae890 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fea0c0d3610 tx=0x7fea04015700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92dd8b5b525c7d24 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:33:30.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.323+0000 7fe9eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:33:30.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.323+0000 7fe9eaffd640 1 -- 172.21.15.5:0/1699752846 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea04023480 con 0x55f741f0e390 2024-05-20T18:33:30.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.323+0000 7fe9eaffd640 1 -- 172.21.15.5:0/1699752846 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6187+0+0 (secure 0 0 0) 0x7fea04023620 con 0x55f741f0e390 2024-05-20T18:33:30.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.323+0000 7fea19245740 1 -- 172.21.15.5:0/1699752846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea00037040 con 0x55f741f0e390 2024-05-20T18:33:30.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.323+0000 7fe9eaffd640 1 -- 172.21.15.5:0/1699752846 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fea0402b050 con 0x55f741f0e390 2024-05-20T18:33:30.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.323+0000 7fea19245740 10 client.7770 fetch_fsmap learned FSMap version 66 2024-05-20T18:33:30.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.323+0000 7fea19245740 1 -- 172.21.15.5:0/1699752846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55f7422a8800 con 0x55f741f0e390 2024-05-20T18:33:30.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.324+0000 7fe9eaffd640 1 -- 172.21.15.5:0/1699752846 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea00037040 con 0x55f741f0e390 2024-05-20T18:33:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.324+0000 7fe9eaffd640 1 -- 172.21.15.5:0/1699752846 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7fea0402e020 con 0x55f741f0e390 2024-05-20T18:33:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.324+0000 7fea19245740 10 client.7770 fetch_fsmap finished waiting for FSMap version 66 2024-05-20T18:33:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.324+0000 7fea19245740 10 client.7770 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-20T18:33:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.324+0000 7fea19245740 20 client.7770 populate_metadata read hostname 'smithi005' 2024-05-20T18:33:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.324+0000 7fea19245740 1 --2- 172.21.15.5:0/1699752846 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55f7423059e0 0x55f742305df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:33:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.324+0000 7fea19245740 4 client.7770 mds_command: new command op to 7437 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T18:33:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.324+0000 7fea19245740 1 -- 172.21.15.5:0/1699752846 --> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f74221a780 con 0x55f7423059e0 2024-05-20T18:33:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.324+0000 7fea0bfff640 1 --2- 172.21.15.5:0/1699752846 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55f7423059e0 0x55f742305df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:33:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.324+0000 7fe9eaffd640 10 client.7770 ms_handle_connect on v2:172.21.15.5:6836/3556492262 2024-05-20T18:33:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.324+0000 7fea0bfff640 1 --2- 172.21.15.5:0/1699752846 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55f7423059e0 0x55f742305df0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69f8002b89c80404 server_cookie=b10b18cb5ae8101e in_seq=0 out_seq=0 2024-05-20T18:33:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.324+0000 7fe9eaffd640 1 -- 172.21.15.5:0/1699752846 <== mds.0 v2:172.21.15.5:6836/3556492262 1 ==== command_reply(tid 0: 0 ) ==== 8+0+256 (crc 0 0 0) 0x55f74221a780 con 0x55f7423059e0 2024-05-20T18:33:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.324+0000 7fe9eaffd640 10 client.7770 handle_command_reply: tid=0 2024-05-20T18:33:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.325+0000 7fea19245740 1 client.7770 shutdown 2024-05-20T18:33:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.325+0000 7fea19245740 1 -- 172.21.15.5:0/1699752846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f741f0e390 msgr2=0x55f7422ae890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:30.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.325+0000 7fea19245740 1 --2- 172.21.15.5:0/1699752846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f741f0e390 0x55f7422ae890 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fea0c0d3610 tx=0x7fea04015700 comp rx=0 tx=0).stop 2024-05-20T18:33:30.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.325+0000 7fea19245740 1 -- 172.21.15.5:0/1699752846 shutdown_connections 2024-05-20T18:33:30.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.325+0000 7fea19245740 1 --2- 172.21.15.5:0/1699752846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f741f0e390 0x55f7422ae890 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.325+0000 7fea19245740 1 --2- 172.21.15.5:0/1699752846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f741eed6b0 0x55f74230d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.325+0000 7fea19245740 1 --2- 172.21.15.5:0/1699752846 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55f7423059e0 0x55f742305df0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.325+0000 7fea19245740 1 --2- 172.21.15.5:0/1699752846 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f741ee54d0 0x55f7423658d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.325+0000 7fe9eaffd640 0 client.7770 ms_handle_reset on v2:172.21.15.5:6836/3556492262 2024-05-20T18:33:30.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.325+0000 7fea19245740 1 -- 172.21.15.5:0/1699752846 >> 172.21.15.5:0/1699752846 conn(0x55f741f024e0 msgr2=0x55f7422f5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:30.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.325+0000 7fea19245740 1 -- 172.21.15.5:0/1699752846 shutdown_connections 2024-05-20T18:33:30.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.325+0000 7fea19245740 1 -- 172.21.15.5:0/1699752846 wait complete. 2024-05-20T18:33:30.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.325+0000 7fea19245740 20 client.7770 trim_cache size 0 max 16384 2024-05-20T18:33:30.331 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:33:30.331 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-20T18:33:30.331 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-20T18:33:30.331 INFO:teuthology.orchestra.run.smithi005.stdout: "1f442c1f-44a4-44d6-8681-aded211e589d": { 2024-05-20T18:33:30.331 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/dir", 2024-05-20T18:33:30.331 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "1f442c1f-44a4-44d6-8681-aded211e589d", 2024-05-20T18:33:30.331 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive" 2024-05-20T18:33:30.331 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:33:30.331 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:33:30.331 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:33:30.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.325+0000 7fea123a3640 1 -- 172.21.15.5:0/1837472925 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe9f806be30 msgr2=0x7fe9f806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:30.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.325+0000 7fea123a3640 1 --2- 172.21.15.5:0/1837472925 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe9f806be30 0x7fe9f806e340 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fea00033000 tx=0x7fea00008030 comp rx=0 tx=0).stop 2024-05-20T18:33:30.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.325+0000 7fea123a3640 1 -- 172.21.15.5:0/1837472925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9f4003f10 msgr2=0x7fea0c14dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:33:30.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.325+0000 7fea123a3640 1 --2- 172.21.15.5:0/1837472925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9f4003f10 0x7fea0c14dd00 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc004f70 tx=0x7fe9fc0074f0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.326+0000 7fea123a3640 1 -- 172.21.15.5:0/1837472925 shutdown_connections 2024-05-20T18:33:30.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.326+0000 7fea123a3640 1 --2- 172.21.15.5:0/1837472925 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe9f806be30 0x7fe9f806e340 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.326+0000 7fea123a3640 1 --2- 172.21.15.5:0/1837472925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea0c15aaa0 0x7fea0c15aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.326+0000 7fea123a3640 1 --2- 172.21.15.5:0/1837472925 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea0c0c6e30 0x7fea0c0c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.326+0000 7fea123a3640 1 --2- 172.21.15.5:0/1837472925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9f4003f10 0x7fea0c14dd00 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:33:30.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.326+0000 7fea123a3640 1 -- 172.21.15.5:0/1837472925 >> 172.21.15.5:0/1837472925 conn(0x7fea0c0d29d0 msgr2=0x7fea0c0b3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:33:30.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.326+0000 7fea123a3640 1 -- 172.21.15.5:0/1837472925 shutdown_connections 2024-05-20T18:33:30.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:33:30.326+0000 7fea123a3640 1 -- 172.21.15.5:0/1837472925 wait complete. 2024-05-20T18:33:30.338 INFO:tasks.cephfs.filesystem:scrub status for tag:1f442c1f-44a4-44d6-8681-aded211e589d - {'path': '/dir', 'tag': '1f442c1f-44a4-44d6-8681-aded211e589d', 'options': 'recursive'} 2024-05-20T18:33:34.281 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-20T18:33:34.280+0000 7fd1d1d3f640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-20T18:33:41.520 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:33:41.523 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:34:00.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:34:00.339 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-20T18:34:00.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.471+0000 7fd3c6a26640 1 Processor -- start 2024-05-20T18:34:00.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.471+0000 7fd3c6a26640 1 -- start start 2024-05-20T18:34:00.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.472+0000 7fd3c6a26640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3c00b4010 0x7fd3c00d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:00.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.472+0000 7fd3c6a26640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd3c00d14b0 0x7fd3c00d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:00.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.472+0000 7fd3c6a26640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3c00d0370 con 0x7fd3c0163a10 2024-05-20T18:34:00.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.472+0000 7fd3c6a26640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd3c00d3ca0 con 0x7fd3c00d14b0 2024-05-20T18:34:00.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.472+0000 7fd3c6a26640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c00d0650 con 0x7fd3c00b4010 2024-05-20T18:34:00.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.473+0000 7fd3c4fc3640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd3c00d14b0 0x7fd3c00d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:00.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.473+0000 7fd3bf7fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3c00b4010 0x7fd3c00d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:00.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.473+0000 7fd3bf7fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3c00b4010 0x7fd3c00d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:42730/0 (socket says 172.21.15.5:42730) 2024-05-20T18:34:00.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.473+0000 7fd3bf7fe640 1 -- 172.21.15.5:0/1539043827 learned_addr learned my addr 172.21.15.5:0/1539043827 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:00.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.473+0000 7fd3c4fc3640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd3c00d14b0 0x7fd3c00d18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:40254/0 (socket says 172.21.15.5:40254) 2024-05-20T18:34:00.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.473+0000 7fd3c4fc3640 1 -- 172.21.15.5:0/1539043827 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3c00b4010 msgr2=0x7fd3c00d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:00.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.473+0000 7fd3c4fc3640 1 --2- 172.21.15.5:0/1539043827 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3c00b4010 0x7fd3c00d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.473+0000 7fd3c4fc3640 1 -- 172.21.15.5:0/1539043827 >> v1:172.21.15.63:6789/0 conn(0x7fd3c0163a10 legacy=0x7fd3c00cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:00.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.473+0000 7fd3c4fc3640 1 -- 172.21.15.5:0/1539043827 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c00d3650 con 0x7fd3c00d14b0 2024-05-20T18:34:00.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.473+0000 7fd3bf7fe640 1 --2- 172.21.15.5:0/1539043827 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3c00b4010 0x7fd3c00d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:34:00.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.474+0000 7fd3c4fc3640 1 --2- 172.21.15.5:0/1539043827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3c00d14b0 0x7fd3c00d18c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8009d90 tx=0x7fd3b802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b93a08ea3d604b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:00.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.474+0000 7fd3beffd640 1 -- 172.21.15.5:0/1539043827 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8007740 con 0x7fd3c00d14b0 2024-05-20T18:34:00.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.474+0000 7fd3beffd640 1 -- 172.21.15.5:0/1539043827 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b80078e0 con 0x7fd3c00d14b0 2024-05-20T18:34:00.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.474+0000 7fd3beffd640 1 -- 172.21.15.5:0/1539043827 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8007cb0 con 0x7fd3c00d14b0 2024-05-20T18:34:00.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.475+0000 7fd3c6a26640 1 -- 172.21.15.5:0/1539043827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3c00d14b0 msgr2=0x7fd3c00d18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:00.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.475+0000 7fd3c6a26640 1 --2- 172.21.15.5:0/1539043827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3c00d14b0 0x7fd3c00d18c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8009d90 tx=0x7fd3b802a660 comp rx=0 tx=0).stop 2024-05-20T18:34:00.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.475+0000 7fd3c6a26640 1 -- 172.21.15.5:0/1539043827 shutdown_connections 2024-05-20T18:34:00.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.475+0000 7fd3c6a26640 1 --2- 172.21.15.5:0/1539043827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3c00d14b0 0x7fd3c00d18c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.475+0000 7fd3c6a26640 1 --2- 172.21.15.5:0/1539043827 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3c00b4010 0x7fd3c00d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.475+0000 7fd3c6a26640 1 -- 172.21.15.5:0/1539043827 >> 172.21.15.5:0/1539043827 conn(0x7fd3c006d240 msgr2=0x7fd3c006d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:00.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.475+0000 7fd3c6a26640 1 -- 172.21.15.5:0/1539043827 shutdown_connections 2024-05-20T18:34:00.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.475+0000 7fd3c6a26640 1 -- 172.21.15.5:0/1539043827 wait complete. 2024-05-20T18:34:00.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.477+0000 7fd3c6a26640 1 Processor -- start 2024-05-20T18:34:00.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.477+0000 7fd3c6a26640 1 -- start start 2024-05-20T18:34:00.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.479+0000 7fd3c6a26640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3c00b4010 0x7fd3c01f5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:00.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.479+0000 7fd3c6a26640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3c00d14b0 0x7fd3c01ea890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:00.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.479+0000 7fd3c6a26640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3c0163a10 0x7fd3c01549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:00.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.479+0000 7fd3c6a26640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c00b8210 con 0x7fd3c00b4010 2024-05-20T18:34:00.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.479+0000 7fd3c6a26640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c00d1e00 con 0x7fd3c00d14b0 2024-05-20T18:34:00.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.479+0000 7fd3c6a26640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c00b8390 con 0x7fd3c0163a10 2024-05-20T18:34:00.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.479+0000 7fd3bffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3c00b4010 0x7fd3c01f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:00.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.479+0000 7fd3bffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3c00b4010 0x7fd3c01f5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56550/0 (socket says 172.21.15.5:56550) 2024-05-20T18:34:00.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.479+0000 7fd3bffff640 1 -- 172.21.15.5:0/2939348452 learned_addr learned my addr 172.21.15.5:0/2939348452 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:00.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.479+0000 7fd3c4fc3640 1 --2- 172.21.15.5:0/2939348452 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3c0163a10 0x7fd3c01549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:00.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.479+0000 7fd3bf7fe640 1 --2- 172.21.15.5:0/2939348452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3c00d14b0 0x7fd3c01ea890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:00.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.479+0000 7fd3bffff640 1 -- 172.21.15.5:0/2939348452 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3c0163a10 msgr2=0x7fd3c01549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:00.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.479+0000 7fd3bffff640 1 --2- 172.21.15.5:0/2939348452 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3c0163a10 0x7fd3c01549a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.479+0000 7fd3bffff640 1 -- 172.21.15.5:0/2939348452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3c00d14b0 msgr2=0x7fd3c01ea890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:00.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.479+0000 7fd3bffff640 1 --2- 172.21.15.5:0/2939348452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3c00d14b0 0x7fd3c01ea890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.479+0000 7fd3bffff640 1 -- 172.21.15.5:0/2939348452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c0161470 con 0x7fd3c00b4010 2024-05-20T18:34:00.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.479+0000 7fd3bffff640 1 --2- 172.21.15.5:0/2939348452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3c00b4010 0x7fd3c01f5420 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fd3b000c440 tx=0x7fd3b000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:00.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.480+0000 7fd3bcff9640 1 -- 172.21.15.5:0/2939348452 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b00052d0 con 0x7fd3c00b4010 2024-05-20T18:34:00.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.480+0000 7fd3bcff9640 1 -- 172.21.15.5:0/2939348452 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b0005470 con 0x7fd3c00b4010 2024-05-20T18:34:00.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.480+0000 7fd3c6a26640 1 -- 172.21.15.5:0/2939348452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3c0161710 con 0x7fd3c00b4010 2024-05-20T18:34:00.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.480+0000 7fd3bcff9640 1 -- 172.21.15.5:0/2939348452 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0007260 con 0x7fd3c00b4010 2024-05-20T18:34:00.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.480+0000 7fd3c6a26640 1 -- 172.21.15.5:0/2939348452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3c0161cd0 con 0x7fd3c00b4010 2024-05-20T18:34:00.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.482+0000 7fd3bcff9640 1 -- 172.21.15.5:0/2939348452 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd3b00074c0 con 0x7fd3c00b4010 2024-05-20T18:34:00.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.482+0000 7fd3bcff9640 1 --2- 172.21.15.5:0/2939348452 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd3ac06be30 0x7fd3ac06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:00.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.482+0000 7fd3bcff9640 1 -- 172.21.15.5:0/2939348452 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7fd3b008a490 con 0x7fd3c00b4010 2024-05-20T18:34:00.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.482+0000 7fd3bf7fe640 1 --2- 172.21.15.5:0/2939348452 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd3ac06be30 0x7fd3ac06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:00.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.483+0000 7fd3bf7fe640 1 --2- 172.21.15.5:0/2939348452 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd3ac06be30 0x7fd3ac06e340 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4009630 tx=0x7fd3b4009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:00.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.483+0000 7fd3cd9a9740 1 Processor -- start 2024-05-20T18:34:00.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.483+0000 7fd3cd9a9740 1 -- start start 2024-05-20T18:34:00.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.484+0000 7fd3cd9a9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbb5626390 0x55bbb59f0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:00.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.484+0000 7fd3cd9a9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbb55fd4d0 0x55bbb5a0b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:00.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.484+0000 7fd3cd9a9740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bbb56056b0 0x55bbb5a0bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:00.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.484+0000 7fd3cd9a9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bbb592ad00 con 0x55bbb5626390 2024-05-20T18:34:00.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.484+0000 7fd3cd9a9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bbb58d24c0 con 0x55bbb55fd4d0 2024-05-20T18:34:00.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.484+0000 7fd3cd9a9740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55bbb5926cc0 con 0x55bbb56056b0 2024-05-20T18:34:00.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.484+0000 7fd3bffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbb5626390 0x55bbb59f0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:00.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.484+0000 7fd3bffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbb5626390 0x55bbb59f0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56562/0 (socket says 172.21.15.5:56562) 2024-05-20T18:34:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.484+0000 7fd3bffff640 1 -- 172.21.15.5:0/2600094649 learned_addr learned my addr 172.21.15.5:0/2600094649 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.485+0000 7fd3bf7fe640 1 --2- 172.21.15.5:0/2600094649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbb55fd4d0 0x55bbb5a0b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.485+0000 7fd3c4fc3640 1 --2- 172.21.15.5:0/2600094649 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bbb56056b0 0x55bbb5a0bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.485+0000 7fd3bffff640 1 -- 172.21.15.5:0/2600094649 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bbb56056b0 msgr2=0x55bbb5a0bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.485+0000 7fd3bffff640 1 --2- 172.21.15.5:0/2600094649 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bbb56056b0 0x55bbb5a0bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.485+0000 7fd3bffff640 1 -- 172.21.15.5:0/2600094649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbb55fd4d0 msgr2=0x55bbb5a0b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.485+0000 7fd3bffff640 1 --2- 172.21.15.5:0/2600094649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbb55fd4d0 0x55bbb5a0b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.485+0000 7fd3bffff640 1 -- 172.21.15.5:0/2600094649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbb57087d0 con 0x55bbb5626390 2024-05-20T18:34:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.485+0000 7fd3c4fc3640 1 --2- 172.21.15.5:0/2600094649 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bbb56056b0 0x55bbb5a0bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:34:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.485+0000 7fd3bf7fe640 1 --2- 172.21.15.5:0/2600094649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbb55fd4d0 0x55bbb5a0b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:34:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.485+0000 7fd3bffff640 1 --2- 172.21.15.5:0/2600094649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbb5626390 0x55bbb59f0b60 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fd3b00886e0 tx=0x7fd3b0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3769e9f037546d29 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.485+0000 7fd3a5ffb640 1 -- 172.21.15.5:0/2600094649 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b00568a0 con 0x55bbb5626390 2024-05-20T18:34:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.485+0000 7fd3a5ffb640 1 -- 172.21.15.5:0/2600094649 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b0054c10 con 0x55bbb5626390 2024-05-20T18:34:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.485+0000 7fd3a5ffb640 1 -- 172.21.15.5:0/2600094649 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0088b90 con 0x55bbb5626390 2024-05-20T18:34:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.486+0000 7fd3cd9a9740 1 -- 172.21.15.5:0/2600094649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbb5626390 msgr2=0x55bbb59f0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:00.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.486+0000 7fd3cd9a9740 1 --2- 172.21.15.5:0/2600094649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbb5626390 0x55bbb59f0b60 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fd3b00886e0 tx=0x7fd3b0011000 comp rx=0 tx=0).stop 2024-05-20T18:34:00.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.486+0000 7fd3cd9a9740 1 -- 172.21.15.5:0/2600094649 shutdown_connections 2024-05-20T18:34:00.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.486+0000 7fd3cd9a9740 1 --2- 172.21.15.5:0/2600094649 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bbb56056b0 0x55bbb5a0bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.486+0000 7fd3cd9a9740 1 --2- 172.21.15.5:0/2600094649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbb55fd4d0 0x55bbb5a0b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.486+0000 7fd3cd9a9740 1 --2- 172.21.15.5:0/2600094649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbb5626390 0x55bbb59f0b60 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.486+0000 7fd3cd9a9740 1 -- 172.21.15.5:0/2600094649 >> 172.21.15.5:0/2600094649 conn(0x55bbb561a4e0 msgr2=0x55bbb59c61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:00.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.486+0000 7fd3cd9a9740 1 -- 172.21.15.5:0/2600094649 shutdown_connections 2024-05-20T18:34:00.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.486+0000 7fd3cd9a9740 1 -- 172.21.15.5:0/2600094649 wait complete. 2024-05-20T18:34:00.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.487+0000 7fd3cd9a9740 1 -- start start 2024-05-20T18:34:00.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.487+0000 7fd3cd9a9740 1 Processor -- start 2024-05-20T18:34:00.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.488+0000 7fd3cd9a9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbb55fd4d0 0x55bbb59f4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.488+0000 7fd3cd9a9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbb56056b0 0x55bbb5a1b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.488+0000 7fd3cd9a9740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bbb5626390 0x55bbb5a065f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.488+0000 7fd3cd9a9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bbb58e5b90 con 0x55bbb56056b0 2024-05-20T18:34:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.488+0000 7fd3cd9a9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bbb5a04310 con 0x55bbb55fd4d0 2024-05-20T18:34:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.488+0000 7fd3cd9a9740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55bbb59f2e90 con 0x55bbb5626390 2024-05-20T18:34:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.489+0000 7fd3cd9a9740 10 client.0 _ll_register_callbacks cb 0x55bbb5921da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:34:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.489+0000 7fd3bf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbb56056b0 0x55bbb5a1b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.489+0000 7fd3bf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbb56056b0 0x55bbb5a1b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56574/0 (socket says 172.21.15.5:56574) 2024-05-20T18:34:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.489+0000 7fd3bf7fe640 1 -- 172.21.15.5:0/3663584718 learned_addr learned my addr 172.21.15.5:0/3663584718 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.489+0000 7fd3bffff640 1 --2- 172.21.15.5:0/3663584718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbb55fd4d0 0x55bbb59f4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.489+0000 7fd3c4fc3640 1 --2- 172.21.15.5:0/3663584718 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bbb5626390 0x55bbb5a065f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.489+0000 7fd3bf7fe640 1 -- 172.21.15.5:0/3663584718 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bbb5626390 msgr2=0x55bbb5a065f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.489+0000 7fd3bf7fe640 1 --2- 172.21.15.5:0/3663584718 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bbb5626390 0x55bbb5a065f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.489+0000 7fd3bf7fe640 1 -- 172.21.15.5:0/3663584718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbb55fd4d0 msgr2=0x55bbb59f4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.489+0000 7fd3bf7fe640 1 --2- 172.21.15.5:0/3663584718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbb55fd4d0 0x55bbb59f4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.489+0000 7fd3bf7fe640 1 -- 172.21.15.5:0/3663584718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbb563e3f0 con 0x55bbb56056b0 2024-05-20T18:34:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.489+0000 7fd3c4fc3640 1 --2- 172.21.15.5:0/3663584718 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bbb5626390 0x55bbb5a065f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:34:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.489+0000 7fd3bf7fe640 1 --2- 172.21.15.5:0/3663584718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbb56056b0 0x55bbb5a1b2f0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4006ca0 tx=0x7fd3b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c1e26162e71fd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.489+0000 7fd39effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:34:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.489+0000 7fd39effd640 1 -- 172.21.15.5:0/3663584718 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4026030 con 0x55bbb56056b0 2024-05-20T18:34:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.489+0000 7fd39effd640 1 -- 172.21.15.5:0/3663584718 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7fd3b400f040 con 0x55bbb56056b0 2024-05-20T18:34:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.489+0000 7fd3cd9a9740 1 -- 172.21.15.5:0/3663584718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3b0054c10 con 0x55bbb56056b0 2024-05-20T18:34:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.490+0000 7fd39effd640 1 -- 172.21.15.5:0/3663584718 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fd3b4022620 con 0x55bbb56056b0 2024-05-20T18:34:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.490+0000 7fd3cd9a9740 10 client.7779 fetch_fsmap learned FSMap version 67 2024-05-20T18:34:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.490+0000 7fd3cd9a9740 1 -- 172.21.15.5:0/3663584718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55bbb55fe0f0 con 0x55bbb56056b0 2024-05-20T18:34:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.490+0000 7fd39effd640 1 -- 172.21.15.5:0/3663584718 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b0054c10 con 0x55bbb56056b0 2024-05-20T18:34:00.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.490+0000 7fd39effd640 1 -- 172.21.15.5:0/3663584718 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7fd3b4035080 con 0x55bbb56056b0 2024-05-20T18:34:00.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.490+0000 7fd3cd9a9740 10 client.7779 fetch_fsmap finished waiting for FSMap version 67 2024-05-20T18:34:00.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.490+0000 7fd3cd9a9740 10 client.7779 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-20T18:34:00.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.490+0000 7fd3cd9a9740 20 client.7779 populate_metadata read hostname 'smithi005' 2024-05-20T18:34:00.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.490+0000 7fd3cd9a9740 1 --2- 172.21.15.5:0/3663584718 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55bbb55e4f60 0x55bbb5a0beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:00.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.490+0000 7fd3cd9a9740 4 client.7779 mds_command: new command op to 7437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:34:00.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.490+0000 7fd3cd9a9740 1 -- 172.21.15.5:0/3663584718 --> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbb5938430 con 0x55bbb55e4f60 2024-05-20T18:34:00.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.490+0000 7fd3bffff640 1 --2- 172.21.15.5:0/3663584718 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55bbb55e4f60 0x55bbb5a0beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:00.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.490+0000 7fd39effd640 10 client.7779 ms_handle_connect on v2:172.21.15.5:6836/3556492262 2024-05-20T18:34:00.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.491+0000 7fd3bffff640 1 --2- 172.21.15.5:0/3663584718 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55bbb55e4f60 0x55bbb5a0beb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adc18e81c5e2355a server_cookie=d269171899359ee6 in_seq=0 out_seq=0 2024-05-20T18:34:00.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.491+0000 7fd39effd640 1 -- 172.21.15.5:0/3663584718 <== mds.0 v2:172.21.15.5:6836/3556492262 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bbb5938430 con 0x55bbb55e4f60 2024-05-20T18:34:00.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.491+0000 7fd39effd640 10 client.7779 handle_command_reply: tid=0 2024-05-20T18:34:00.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.491+0000 7fd3cd9a9740 1 client.7779 shutdown 2024-05-20T18:34:00.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.491+0000 7fd3cd9a9740 1 -- 172.21.15.5:0/3663584718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbb56056b0 msgr2=0x55bbb5a1b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:00.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.491+0000 7fd3cd9a9740 1 --2- 172.21.15.5:0/3663584718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbb56056b0 0x55bbb5a1b2f0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4006ca0 tx=0x7fd3b40046c0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.491+0000 7fd3cd9a9740 1 -- 172.21.15.5:0/3663584718 shutdown_connections 2024-05-20T18:34:00.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.491+0000 7fd3cd9a9740 1 --2- 172.21.15.5:0/3663584718 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55bbb55e4f60 0x55bbb5a0beb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.491+0000 7fd3cd9a9740 1 --2- 172.21.15.5:0/3663584718 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bbb5626390 0x55bbb5a065f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.491+0000 7fd3cd9a9740 1 --2- 172.21.15.5:0/3663584718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbb56056b0 0x55bbb5a1b2f0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.491+0000 7fd3cd9a9740 1 --2- 172.21.15.5:0/3663584718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbb55fd4d0 0x55bbb59f4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.491+0000 7fd39effd640 0 client.7779 ms_handle_reset on v2:172.21.15.5:6836/3556492262 2024-05-20T18:34:00.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.491+0000 7fd3cd9a9740 1 -- 172.21.15.5:0/3663584718 >> 172.21.15.5:0/3663584718 conn(0x55bbb561a4e0 msgr2=0x55bbb59c61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:00.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.492+0000 7fd3cd9a9740 1 -- 172.21.15.5:0/3663584718 shutdown_connections 2024-05-20T18:34:00.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.492+0000 7fd3cd9a9740 1 -- 172.21.15.5:0/3663584718 wait complete. 2024-05-20T18:34:00.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.492+0000 7fd3cd9a9740 20 client.7779 trim_cache size 0 max 16384 2024-05-20T18:34:00.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.510+0000 7fd3cd9a9740 1 Processor -- start 2024-05-20T18:34:00.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.510+0000 7fd3cd9a9740 1 -- start start 2024-05-20T18:34:00.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.511+0000 7fd3cd9a9740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bbb55fd4d0 0x55bbb5a02730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:00.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.511+0000 7fd3cd9a9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbb56056b0 0x55bbb5a02c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:00.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.511+0000 7fd3cd9a9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbb5626390 0x55bbb59ecfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:00.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.511+0000 7fd3cd9a9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c0202850 con 0x55bbb5626390 2024-05-20T18:34:00.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.511+0000 7fd3cd9a9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c0201870 con 0x55bbb56056b0 2024-05-20T18:34:00.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.511+0000 7fd3cd9a9740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c0200c30 con 0x55bbb55fd4d0 2024-05-20T18:34:00.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.512+0000 7fd3c4fc3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbb5626390 0x55bbb59ecfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:00.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.512+0000 7fd3c4fc3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbb5626390 0x55bbb59ecfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56580/0 (socket says 172.21.15.5:56580) 2024-05-20T18:34:00.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.512+0000 7fd3c4fc3640 1 -- 172.21.15.5:0/404472905 learned_addr learned my addr 172.21.15.5:0/404472905 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:00.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.512+0000 7fd3bf7fe640 1 --2- 172.21.15.5:0/404472905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbb56056b0 0x55bbb5a02c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:00.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.512+0000 7fd3bffff640 1 --2- 172.21.15.5:0/404472905 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bbb55fd4d0 0x55bbb5a02730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:00.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.512+0000 7fd3c4fc3640 1 -- 172.21.15.5:0/404472905 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bbb55fd4d0 msgr2=0x55bbb5a02730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:00.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.512+0000 7fd3c4fc3640 1 --2- 172.21.15.5:0/404472905 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bbb55fd4d0 0x55bbb5a02730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.512+0000 7fd3c4fc3640 1 -- 172.21.15.5:0/404472905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbb56056b0 msgr2=0x55bbb5a02c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:00.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.512+0000 7fd3c4fc3640 1 --2- 172.21.15.5:0/404472905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbb56056b0 0x55bbb5a02c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.512+0000 7fd3c4fc3640 1 -- 172.21.15.5:0/404472905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbb5a14b40 con 0x55bbb5626390 2024-05-20T18:34:00.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.512+0000 7fd3c4fc3640 1 --2- 172.21.15.5:0/404472905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbb5626390 0x55bbb59ecfa0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8009eb0 tx=0x7fd3b802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcedd843526ddcff server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:00.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.512+0000 7fd3a57fa640 1 -- 172.21.15.5:0/404472905 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b804b070 con 0x55bbb5626390 2024-05-20T18:34:00.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.512+0000 7fd3a57fa640 1 -- 172.21.15.5:0/404472905 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b8007b30 con 0x55bbb5626390 2024-05-20T18:34:00.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.512+0000 7fd3a57fa640 1 -- 172.21.15.5:0/404472905 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b803b410 con 0x55bbb5626390 2024-05-20T18:34:00.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.513+0000 7fd3cd9a9740 1 -- 172.21.15.5:0/404472905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbb5626390 msgr2=0x55bbb59ecfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:00.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.513+0000 7fd3cd9a9740 1 --2- 172.21.15.5:0/404472905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbb5626390 0x55bbb59ecfa0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8009eb0 tx=0x7fd3b802e000 comp rx=0 tx=0).stop 2024-05-20T18:34:00.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.513+0000 7fd3cd9a9740 1 -- 172.21.15.5:0/404472905 shutdown_connections 2024-05-20T18:34:00.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.513+0000 7fd3cd9a9740 1 --2- 172.21.15.5:0/404472905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbb5626390 0x55bbb59ecfa0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.513+0000 7fd3cd9a9740 1 --2- 172.21.15.5:0/404472905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbb56056b0 0x55bbb5a02c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.513+0000 7fd3cd9a9740 1 --2- 172.21.15.5:0/404472905 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bbb55fd4d0 0x55bbb5a02730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.513+0000 7fd3cd9a9740 1 -- 172.21.15.5:0/404472905 >> 172.21.15.5:0/404472905 conn(0x55bbb561a4e0 msgr2=0x55bbb5a2e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:00.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.513+0000 7fd3cd9a9740 1 -- 172.21.15.5:0/404472905 shutdown_connections 2024-05-20T18:34:00.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.513+0000 7fd3cd9a9740 1 -- 172.21.15.5:0/404472905 wait complete. 2024-05-20T18:34:00.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.514+0000 7fd3cd9a9740 1 -- start start 2024-05-20T18:34:00.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.514+0000 7fd3cd9a9740 1 Processor -- start 2024-05-20T18:34:00.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.516+0000 7fd3cd9a9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbb55fd4d0 0x55bbb5a045a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:00.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.516+0000 7fd3cd9a9740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bbb56056b0 0x55bbb5a06620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:00.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.516+0000 7fd3cd9a9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbb5626390 0x55bbb5a0a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:00.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.516+0000 7fd3cd9a9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bbb5a03bd0 con 0x55bbb5626390 2024-05-20T18:34:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.516+0000 7fd3cd9a9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c0201220 con 0x55bbb55fd4d0 2024-05-20T18:34:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.516+0000 7fd3cd9a9740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55bbb58e0df0 con 0x55bbb56056b0 2024-05-20T18:34:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.516+0000 7fd3cd9a9740 10 client.0 _ll_register_callbacks cb 0x7fd3b4009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:34:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.516+0000 7fd3c4fc3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbb5626390 0x55bbb5a0a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.516+0000 7fd3c4fc3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbb5626390 0x55bbb5a0a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56592/0 (socket says 172.21.15.5:56592) 2024-05-20T18:34:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.516+0000 7fd3c4fc3640 1 -- 172.21.15.5:0/2460227805 learned_addr learned my addr 172.21.15.5:0/2460227805 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.516+0000 7fd3bffff640 1 --2- 172.21.15.5:0/2460227805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbb55fd4d0 0x55bbb5a045a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.516+0000 7fd3bf7fe640 1 --2- 172.21.15.5:0/2460227805 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bbb56056b0 0x55bbb5a06620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.516+0000 7fd3c4fc3640 1 -- 172.21.15.5:0/2460227805 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bbb56056b0 msgr2=0x55bbb5a06620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.516+0000 7fd3c4fc3640 1 --2- 172.21.15.5:0/2460227805 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bbb56056b0 0x55bbb5a06620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.516+0000 7fd3c4fc3640 1 -- 172.21.15.5:0/2460227805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbb55fd4d0 msgr2=0x55bbb5a045a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.516+0000 7fd3c4fc3640 1 --2- 172.21.15.5:0/2460227805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbb55fd4d0 0x55bbb5a045a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.516+0000 7fd3c4fc3640 1 -- 172.21.15.5:0/2460227805 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbb59f27b0 con 0x55bbb5626390 2024-05-20T18:34:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.516+0000 7fd3c4fc3640 1 --2- 172.21.15.5:0/2460227805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbb5626390 0x55bbb5a0a250 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fd3b802ac50 tx=0x7fd3b8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce7806251a138c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.516+0000 7fd39effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:34:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.516+0000 7fd39effd640 1 -- 172.21.15.5:0/2460227805 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b804b070 con 0x55bbb5626390 2024-05-20T18:34:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.516+0000 7fd39effd640 1 -- 172.21.15.5:0/2460227805 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7fd3b8003da0 con 0x55bbb5626390 2024-05-20T18:34:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.517+0000 7fd3cd9a9740 1 -- 172.21.15.5:0/2460227805 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3b8007b30 con 0x55bbb5626390 2024-05-20T18:34:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.517+0000 7fd39effd640 1 -- 172.21.15.5:0/2460227805 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fd3b8004040 con 0x55bbb5626390 2024-05-20T18:34:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.517+0000 7fd3cd9a9740 10 client.7785 fetch_fsmap learned FSMap version 67 2024-05-20T18:34:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.517+0000 7fd3cd9a9740 1 -- 172.21.15.5:0/2460227805 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55bbb59ed540 con 0x55bbb5626390 2024-05-20T18:34:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.517+0000 7fd39effd640 1 -- 172.21.15.5:0/2460227805 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b8007b30 con 0x55bbb5626390 2024-05-20T18:34:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.517+0000 7fd39effd640 1 -- 172.21.15.5:0/2460227805 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7fd3b8045070 con 0x55bbb5626390 2024-05-20T18:34:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.517+0000 7fd3cd9a9740 10 client.7785 fetch_fsmap finished waiting for FSMap version 67 2024-05-20T18:34:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.517+0000 7fd3cd9a9740 10 client.7785 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-20T18:34:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.517+0000 7fd3cd9a9740 20 client.7785 populate_metadata read hostname 'smithi005' 2024-05-20T18:34:00.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.517+0000 7fd3cd9a9740 1 --2- 172.21.15.5:0/2460227805 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55bbb5a0beb0 0x55bbb5a0a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:00.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.517+0000 7fd3cd9a9740 4 client.7785 mds_command: new command op to 7437 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T18:34:00.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.517+0000 7fd3cd9a9740 1 -- 172.21.15.5:0/2460227805 --> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bbb5932780 con 0x55bbb5a0beb0 2024-05-20T18:34:00.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.517+0000 7fd3bffff640 1 --2- 172.21.15.5:0/2460227805 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55bbb5a0beb0 0x55bbb5a0a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:00.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.517+0000 7fd39effd640 10 client.7785 ms_handle_connect on v2:172.21.15.5:6836/3556492262 2024-05-20T18:34:00.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.518+0000 7fd3bffff640 1 --2- 172.21.15.5:0/2460227805 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55bbb5a0beb0 0x55bbb5a0a790 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2727a83b1c8d7d84 server_cookie=ebafa9b284a128fd in_seq=0 out_seq=0 2024-05-20T18:34:00.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.518+0000 7fd39effd640 1 -- 172.21.15.5:0/2460227805 <== mds.0 v2:172.21.15.5:6836/3556492262 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bbb5932780 con 0x55bbb5a0beb0 2024-05-20T18:34:00.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.518+0000 7fd39effd640 10 client.7785 handle_command_reply: tid=0 2024-05-20T18:34:00.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.518+0000 7fd3cd9a9740 1 client.7785 shutdown 2024-05-20T18:34:00.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.518+0000 7fd3cd9a9740 1 -- 172.21.15.5:0/2460227805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbb5626390 msgr2=0x55bbb5a0a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:00.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.518+0000 7fd3cd9a9740 1 --2- 172.21.15.5:0/2460227805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbb5626390 0x55bbb5a0a250 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fd3b802ac50 tx=0x7fd3b8004ab0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.518+0000 7fd3cd9a9740 1 -- 172.21.15.5:0/2460227805 shutdown_connections 2024-05-20T18:34:00.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.518+0000 7fd3cd9a9740 1 --2- 172.21.15.5:0/2460227805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbb5626390 0x55bbb5a0a250 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.518+0000 7fd3cd9a9740 1 --2- 172.21.15.5:0/2460227805 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55bbb5a0beb0 0x55bbb5a0a790 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.518+0000 7fd3cd9a9740 1 --2- 172.21.15.5:0/2460227805 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bbb56056b0 0x55bbb5a06620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.518+0000 7fd3cd9a9740 1 --2- 172.21.15.5:0/2460227805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bbb55fd4d0 0x55bbb5a045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.518+0000 7fd39effd640 0 client.7785 ms_handle_reset on v2:172.21.15.5:6836/3556492262 2024-05-20T18:34:00.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.518+0000 7fd3cd9a9740 1 -- 172.21.15.5:0/2460227805 >> 172.21.15.5:0/2460227805 conn(0x55bbb561a4e0 msgr2=0x55bbb5a1a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:00.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.518+0000 7fd3cd9a9740 1 -- 172.21.15.5:0/2460227805 shutdown_connections 2024-05-20T18:34:00.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.518+0000 7fd3cd9a9740 1 -- 172.21.15.5:0/2460227805 wait complete. 2024-05-20T18:34:00.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.518+0000 7fd3cd9a9740 20 client.7785 trim_cache size 0 max 16384 2024-05-20T18:34:00.524 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:34:00.524 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-20T18:34:00.524 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-20T18:34:00.525 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:34:00.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.519+0000 7fd3c6a26640 1 -- 172.21.15.5:0/2939348452 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd3ac06be30 msgr2=0x7fd3ac06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:00.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.519+0000 7fd3c6a26640 1 --2- 172.21.15.5:0/2939348452 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd3ac06be30 0x7fd3ac06e340 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4009630 tx=0x7fd3b4009240 comp rx=0 tx=0).stop 2024-05-20T18:34:00.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.519+0000 7fd3c6a26640 1 -- 172.21.15.5:0/2939348452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3c00b4010 msgr2=0x7fd3c01f5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:00.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.519+0000 7fd3c6a26640 1 --2- 172.21.15.5:0/2939348452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3c00b4010 0x7fd3c01f5420 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fd3b000c440 tx=0x7fd3b000c900 comp rx=0 tx=0).stop 2024-05-20T18:34:00.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.519+0000 7fd3c6a26640 1 -- 172.21.15.5:0/2939348452 shutdown_connections 2024-05-20T18:34:00.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.519+0000 7fd3c6a26640 1 --2- 172.21.15.5:0/2939348452 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd3ac06be30 0x7fd3ac06e340 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.519+0000 7fd3c6a26640 1 --2- 172.21.15.5:0/2939348452 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd3c0163a10 0x7fd3c01549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.519+0000 7fd3c6a26640 1 --2- 172.21.15.5:0/2939348452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3c00d14b0 0x7fd3c01ea890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.519+0000 7fd3c6a26640 1 --2- 172.21.15.5:0/2939348452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3c00b4010 0x7fd3c01f5420 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.519+0000 7fd3c6a26640 1 -- 172.21.15.5:0/2939348452 >> 172.21.15.5:0/2939348452 conn(0x7fd3c006d240 msgr2=0x7fd3c00ba590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:00.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.519+0000 7fd3c6a26640 1 -- 172.21.15.5:0/2939348452 shutdown_connections 2024-05-20T18:34:00.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.519+0000 7fd3c6a26640 1 -- 172.21.15.5:0/2939348452 wait complete. 2024-05-20T18:34:00.531 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T18:34:00.531 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:34:00.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.645+0000 7f6ee192f640 1 Processor -- start 2024-05-20T18:34:00.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.646+0000 7f6ee192f640 1 -- start start 2024-05-20T18:34:00.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.647+0000 7f6ee192f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6edc162820 0x7f6edc0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:00.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.647+0000 7f6ee192f640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6edc0d0630 con 0x7f6edc06dae0 2024-05-20T18:34:00.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.647+0000 7f6ee192f640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6edc0d39c0 con 0x7f6edc0b3e80 2024-05-20T18:34:00.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.647+0000 7f6ee192f640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f6edc0d3ca0 con 0x7f6edc162820 2024-05-20T18:34:00.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.647+0000 7f6edaffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6edc162820 0x7f6edc0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:00.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.647+0000 7f6edb7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f6edc06dae0 0x7f6edc0b8190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35430/0 (socket says 172.21.15.5:35430) 2024-05-20T18:34:00.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.647+0000 7f6edaffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6edc162820 0x7f6edc0b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56594/0 (socket says 172.21.15.5:56594) 2024-05-20T18:34:00.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.647+0000 7f6edb7fe640 1 -- 172.21.15.5:0/1638929133 learned_addr learned my addr 172.21.15.5:0/1638929133 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:34:00.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.647+0000 7f6edaffd640 1 -- 172.21.15.5:0/1638929133 >> v1:172.21.15.63:6789/0 conn(0x7f6edc0b3e80 legacy=0x7f6edc06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:00.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.647+0000 7f6edaffd640 1 -- 172.21.15.5:0/1638929133 >> v1:172.21.15.5:6789/0 conn(0x7f6edc06dae0 legacy=0x7f6edc0b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:00.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.647+0000 7f6edaffd640 1 -- 172.21.15.5:0/1638929133 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6edc0d3f80 con 0x7f6edc162820 2024-05-20T18:34:00.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.647+0000 7f6ed9ffb640 1 -- 172.21.15.5:0/1638929133 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1969479585 0 0) 0x7f6edc0d0630 con 0x7f6edc06dae0 2024-05-20T18:34:00.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.648+0000 7f6edaffd640 1 --2- 172.21.15.5:0/1638929133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6edc162820 0x7f6edc0b3940 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc0097a0 tx=0x7f6ecc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=889c89f797dbdfb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:00.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.648+0000 7f6ed9ffb640 1 -- 172.21.15.5:0/1638929133 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc037070 con 0x7f6edc162820 2024-05-20T18:34:00.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.648+0000 7f6ed9ffb640 1 -- 172.21.15.5:0/1638929133 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ecc032440 con 0x7f6edc162820 2024-05-20T18:34:00.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.648+0000 7f6ed9ffb640 1 -- 172.21.15.5:0/1638929133 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc032810 con 0x7f6edc162820 2024-05-20T18:34:00.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.649+0000 7f6ee192f640 1 -- 172.21.15.5:0/1638929133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6edc162820 msgr2=0x7f6edc0b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:00.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.649+0000 7f6ee192f640 1 --2- 172.21.15.5:0/1638929133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6edc162820 0x7f6edc0b3940 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc0097a0 tx=0x7f6ecc02a230 comp rx=0 tx=0).stop 2024-05-20T18:34:00.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.649+0000 7f6ee192f640 1 -- 172.21.15.5:0/1638929133 shutdown_connections 2024-05-20T18:34:00.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.649+0000 7f6ee192f640 1 --2- 172.21.15.5:0/1638929133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6edc162820 0x7f6edc0b3940 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.649+0000 7f6ee192f640 1 -- 172.21.15.5:0/1638929133 >> 172.21.15.5:0/1638929133 conn(0x7f6edc0d4bd0 msgr2=0x7f6edc0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:00.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.649+0000 7f6ee192f640 1 -- 172.21.15.5:0/1638929133 shutdown_connections 2024-05-20T18:34:00.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.649+0000 7f6ee192f640 1 -- 172.21.15.5:0/1638929133 wait complete. 2024-05-20T18:34:00.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.651+0000 7f6ee192f640 1 Processor -- start 2024-05-20T18:34:00.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.651+0000 7f6ee192f640 1 -- start start 2024-05-20T18:34:00.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.652+0000 7f6ee192f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6edc0b3e80 0x7f6edc0c7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:00.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.652+0000 7f6ee192f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6edc162820 0x7f6edc0cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:00.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.652+0000 7f6ee192f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6edc0cbc80 0x7f6edc0cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:00.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.652+0000 7f6ee192f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6edc0d4780 con 0x7f6edc0b3e80 2024-05-20T18:34:00.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.652+0000 7f6ee192f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6edc0d0300 con 0x7f6edc0cbc80 2024-05-20T18:34:00.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.652+0000 7f6ee192f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6edc0d4900 con 0x7f6edc162820 2024-05-20T18:34:00.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.653+0000 7f6edaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6edc0b3e80 0x7f6edc0c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:00.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.653+0000 7f6edaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6edc0b3e80 0x7f6edc0c7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56600/0 (socket says 172.21.15.5:56600) 2024-05-20T18:34:00.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.653+0000 7f6edaffd640 1 -- 172.21.15.5:0/2708431363 learned_addr learned my addr 172.21.15.5:0/2708431363 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:00.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.653+0000 7f6edb7fe640 1 --2- 172.21.15.5:0/2708431363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6edc0cbc80 0x7f6edc0cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:00.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.653+0000 7f6eda7fc640 1 --2- 172.21.15.5:0/2708431363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6edc162820 0x7f6edc0cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:00.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.653+0000 7f6edaffd640 1 -- 172.21.15.5:0/2708431363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6edc162820 msgr2=0x7f6edc0cdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:00.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.653+0000 7f6edaffd640 1 --2- 172.21.15.5:0/2708431363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6edc162820 0x7f6edc0cdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.653+0000 7f6edaffd640 1 -- 172.21.15.5:0/2708431363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6edc0cbc80 msgr2=0x7f6edc0cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:00.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.653+0000 7f6edaffd640 1 --2- 172.21.15.5:0/2708431363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6edc0cbc80 0x7f6edc0cc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.653+0000 7f6edaffd640 1 -- 172.21.15.5:0/2708431363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6edc0cc5d0 con 0x7f6edc0b3e80 2024-05-20T18:34:00.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.653+0000 7f6edaffd640 1 --2- 172.21.15.5:0/2708431363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6edc0b3e80 0x7f6edc0c7790 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc0097a0 tx=0x7f6ecc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:00.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.653+0000 7f6ec3fff640 1 -- 172.21.15.5:0/2708431363 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc037050 con 0x7f6edc0b3e80 2024-05-20T18:34:00.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.653+0000 7f6ec3fff640 1 -- 172.21.15.5:0/2708431363 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ecc038040 con 0x7f6edc0b3e80 2024-05-20T18:34:00.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.653+0000 7f6ee192f640 1 -- 172.21.15.5:0/2708431363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6edc0ce6b0 con 0x7f6edc0b3e80 2024-05-20T18:34:00.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.653+0000 7f6ec3fff640 1 -- 172.21.15.5:0/2708431363 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc004490 con 0x7f6edc0b3e80 2024-05-20T18:34:00.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.653+0000 7f6ee192f640 1 -- 172.21.15.5:0/2708431363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6edc0b44c0 con 0x7f6edc0b3e80 2024-05-20T18:34:00.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.654+0000 7f6ee192f640 1 -- 172.21.15.5:0/2708431363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6edc075b00 con 0x7f6edc0b3e80 2024-05-20T18:34:00.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.655+0000 7f6ec3fff640 1 -- 172.21.15.5:0/2708431363 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6ecc030900 con 0x7f6edc0b3e80 2024-05-20T18:34:00.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.656+0000 7f6ec3fff640 1 --2- 172.21.15.5:0/2708431363 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6ec806be30 0x7f6ec806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:00.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.656+0000 7f6ec3fff640 1 -- 172.21.15.5:0/2708431363 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7f6ecc0abb50 con 0x7f6edc0b3e80 2024-05-20T18:34:00.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.656+0000 7f6eda7fc640 1 --2- 172.21.15.5:0/2708431363 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6ec806be30 0x7f6ec806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:00.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.656+0000 7f6eda7fc640 1 --2- 172.21.15.5:0/2708431363 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6ec806be30 0x7f6ec806e340 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6edc1e8c80 tx=0x7f6ed0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:00.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.660+0000 7f6ec3fff640 1 -- 172.21.15.5:0/2708431363 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6ecc04c020 con 0x7f6edc0b3e80 2024-05-20T18:34:00.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.892+0000 7f6ee192f640 1 -- 172.21.15.5:0/2708431363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6edc0b3620 con 0x7f6edc0b3e80 2024-05-20T18:34:00.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.892+0000 7f6ec3fff640 1 -- 172.21.15.5:0/2708431363 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5173 (secure 0 0 0) 0x7f6ecc0ab020 con 0x7f6edc0b3e80 2024-05-20T18:34:00.893 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:34:00.893 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/337248865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":337248865},{"type":"v1","addr":"172.21.15.63:6834","nonce":337248865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3705772553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3705772553},{"type":"v1","addr":"172.21.15.63:6835","nonce":3705772553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":7434,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/776188138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":776188138},{"type":"v1","addr":"172.21.15.5:6835","nonce":776188138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:33:12.457716+0000","modified":"2024-05-20T18:33:34.256059+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":10,"addr":"172.21.15.5:6837/3556492262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3556492262},{"type":"v1","addr":"172.21.15.5:6837","nonce":3556492262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7437,"qdb_cluster":[7437]},"id":6}]} 2024-05-20T18:34:00.893 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 67 2024-05-20T18:34:00.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.895+0000 7f6ee192f640 1 -- 172.21.15.5:0/2708431363 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6ec806be30 msgr2=0x7f6ec806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:00.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.895+0000 7f6ee192f640 1 --2- 172.21.15.5:0/2708431363 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6ec806be30 0x7f6ec806e340 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6edc1e8c80 tx=0x7f6ed0009040 comp rx=0 tx=0).stop 2024-05-20T18:34:00.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.895+0000 7f6ee192f640 1 -- 172.21.15.5:0/2708431363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6edc0b3e80 msgr2=0x7f6edc0c7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:00.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.895+0000 7f6ee192f640 1 --2- 172.21.15.5:0/2708431363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6edc0b3e80 0x7f6edc0c7790 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc0097a0 tx=0x7f6ecc0046c0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.895+0000 7f6ee192f640 1 -- 172.21.15.5:0/2708431363 shutdown_connections 2024-05-20T18:34:00.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.895+0000 7f6ee192f640 1 --2- 172.21.15.5:0/2708431363 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6ec806be30 0x7f6ec806e340 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.895+0000 7f6ee192f640 1 --2- 172.21.15.5:0/2708431363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6edc0cbc80 0x7f6edc0cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.895+0000 7f6ee192f640 1 --2- 172.21.15.5:0/2708431363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6edc162820 0x7f6edc0cdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.895+0000 7f6ee192f640 1 --2- 172.21.15.5:0/2708431363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6edc0b3e80 0x7f6edc0c7790 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:00.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.895+0000 7f6ee192f640 1 -- 172.21.15.5:0/2708431363 >> 172.21.15.5:0/2708431363 conn(0x7f6edc0d4bd0 msgr2=0x7f6edc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:00.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.895+0000 7f6ee192f640 1 -- 172.21.15.5:0/2708431363 shutdown_connections 2024-05-20T18:34:00.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:00.895+0000 7f6ee192f640 1 -- 172.21.15.5:0/2708431363 wait complete. 2024-05-20T18:34:00.907 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-05-20T18:34:01.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.022+0000 7feb46977640 1 Processor -- start 2024-05-20T18:34:01.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.022+0000 7feb46977640 1 -- start start 2024-05-20T18:34:01.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.023+0000 7feb46977640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7feb400cf250 0x7feb4015f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.023+0000 7feb46977640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7feb400b6680 0x7feb400b6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.023+0000 7feb46977640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb400b6fd0 con 0x7feb40163020 2024-05-20T18:34:01.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.023+0000 7feb46977640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7feb4015f9a0 con 0x7feb400b6680 2024-05-20T18:34:01.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.023+0000 7feb46977640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7feb400d0b10 con 0x7feb400cf250 2024-05-20T18:34:01.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.023+0000 7feb3f7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7feb400b6680 0x7feb400b6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.023+0000 7feb44f14640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7feb40163020 0x7feb40163430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35446/0 (socket says 172.21.15.5:35446) 2024-05-20T18:34:01.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.023+0000 7feb44f14640 1 -- 172.21.15.5:0/3222984736 learned_addr learned my addr 172.21.15.5:0/3222984736 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:34:01.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.024+0000 7feb3ffff640 1 --2- 172.21.15.5:0/3222984736 >> v2:172.21.15.63:3300/0 conn(0x7feb400cf250 0x7feb4015f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.024+0000 7feb3f7fe640 1 -- 172.21.15.5:0/3222984736 >> v2:172.21.15.63:3300/0 conn(0x7feb400cf250 msgr2=0x7feb4015f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:01.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.024+0000 7feb3f7fe640 1 --2- 172.21.15.5:0/3222984736 >> v2:172.21.15.63:3300/0 conn(0x7feb400cf250 0x7feb4015f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.024+0000 7feb3f7fe640 1 -- 172.21.15.5:0/3222984736 >> v1:172.21.15.5:6789/0 conn(0x7feb40163020 legacy=0x7feb40163430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.024+0000 7feb3f7fe640 1 -- 172.21.15.5:0/3222984736 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb400d0550 con 0x7feb400b6680 2024-05-20T18:34:01.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.024+0000 7feb3ffff640 1 --2- 172.21.15.5:0/3222984736 >> v2:172.21.15.63:3300/0 conn(0x7feb400cf250 0x7feb4015f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:34:01.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.025+0000 7feb3f7fe640 1 --2- 172.21.15.5:0/3222984736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb400b6680 0x7feb400b6a90 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7feb4008c360 tx=0x7feb3402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f467b6d5bb0335 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:01.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.025+0000 7feb3effd640 1 -- 172.21.15.5:0/3222984736 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb34037070 con 0x7feb400b6680 2024-05-20T18:34:01.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.025+0000 7feb3effd640 1 -- 172.21.15.5:0/3222984736 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb34032440 con 0x7feb400b6680 2024-05-20T18:34:01.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.025+0000 7feb3effd640 1 -- 172.21.15.5:0/3222984736 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb34032810 con 0x7feb400b6680 2024-05-20T18:34:01.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.025+0000 7feb46977640 1 -- 172.21.15.5:0/3222984736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb400b6680 msgr2=0x7feb400b6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.025+0000 7feb46977640 1 --2- 172.21.15.5:0/3222984736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb400b6680 0x7feb400b6a90 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7feb4008c360 tx=0x7feb3402a1a0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.025+0000 7feb46977640 1 -- 172.21.15.5:0/3222984736 shutdown_connections 2024-05-20T18:34:01.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.025+0000 7feb46977640 1 --2- 172.21.15.5:0/3222984736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb400b6680 0x7feb400b6a90 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.025+0000 7feb46977640 1 --2- 172.21.15.5:0/3222984736 >> v2:172.21.15.63:3300/0 conn(0x7feb400cf250 0x7feb4015f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.025+0000 7feb46977640 1 -- 172.21.15.5:0/3222984736 >> 172.21.15.5:0/3222984736 conn(0x7feb400d1860 msgr2=0x7feb400d1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:01.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.026+0000 7feb46977640 1 -- 172.21.15.5:0/3222984736 shutdown_connections 2024-05-20T18:34:01.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.026+0000 7feb46977640 1 -- 172.21.15.5:0/3222984736 wait complete. 2024-05-20T18:34:01.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.027+0000 7feb46977640 1 Processor -- start 2024-05-20T18:34:01.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.027+0000 7feb46977640 1 -- start start 2024-05-20T18:34:01.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.029+0000 7feb46977640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb400b6680 0x7feb40158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.029+0000 7feb46977640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb400cf250 0x7feb4014f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.029+0000 7feb46977640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb40163020 0x7feb400b9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.029+0000 7feb46977640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feb400d2a20 con 0x7feb400cf250 2024-05-20T18:34:01.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.029+0000 7feb46977640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feb400d27d0 con 0x7feb400b6680 2024-05-20T18:34:01.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.029+0000 7feb46977640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7feb400d2d00 con 0x7feb40163020 2024-05-20T18:34:01.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.029+0000 7feb3f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb400cf250 0x7feb4014f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.029+0000 7feb3f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb400cf250 0x7feb4014f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56608/0 (socket says 172.21.15.5:56608) 2024-05-20T18:34:01.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.029+0000 7feb3f7fe640 1 -- 172.21.15.5:0/3881985244 learned_addr learned my addr 172.21.15.5:0/3881985244 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:01.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.029+0000 7feb3ffff640 1 --2- 172.21.15.5:0/3881985244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb400b6680 0x7feb40158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.029+0000 7feb44f14640 1 --2- 172.21.15.5:0/3881985244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb40163020 0x7feb400b9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.029+0000 7feb3f7fe640 1 -- 172.21.15.5:0/3881985244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb40163020 msgr2=0x7feb400b9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.029+0000 7feb3f7fe640 1 --2- 172.21.15.5:0/3881985244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb40163020 0x7feb400b9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.029+0000 7feb3f7fe640 1 -- 172.21.15.5:0/3881985244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb400b6680 msgr2=0x7feb40158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.029+0000 7feb3f7fe640 1 --2- 172.21.15.5:0/3881985244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb400b6680 0x7feb40158cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.029+0000 7feb3f7fe640 1 -- 172.21.15.5:0/3881985244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb401591f0 con 0x7feb400cf250 2024-05-20T18:34:01.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.029+0000 7feb44f14640 1 --2- 172.21.15.5:0/3881985244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb40163020 0x7feb400b9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:34:01.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.029+0000 7feb3f7fe640 1 --2- 172.21.15.5:0/3881985244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb400cf250 0x7feb4014f690 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7feb34009890 tx=0x7feb340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:01.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.030+0000 7feb3cff9640 1 -- 172.21.15.5:0/3881985244 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb34037040 con 0x7feb400cf250 2024-05-20T18:34:01.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.030+0000 7feb3cff9640 1 -- 172.21.15.5:0/3881985244 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb34042070 con 0x7feb400cf250 2024-05-20T18:34:01.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.030+0000 7feb46977640 1 -- 172.21.15.5:0/3881985244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb401e9d70 con 0x7feb400cf250 2024-05-20T18:34:01.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.030+0000 7feb3cff9640 1 -- 172.21.15.5:0/3881985244 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb34030a50 con 0x7feb400cf250 2024-05-20T18:34:01.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.030+0000 7feb46977640 1 -- 172.21.15.5:0/3881985244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb40151de0 con 0x7feb400cf250 2024-05-20T18:34:01.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.031+0000 7feb3cff9640 1 -- 172.21.15.5:0/3881985244 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feb34030bf0 con 0x7feb400cf250 2024-05-20T18:34:01.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.032+0000 7feb3cff9640 1 --2- 172.21.15.5:0/3881985244 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feb2c06be30 0x7feb2c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.032+0000 7feb3cff9640 1 -- 172.21.15.5:0/3881985244 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7feb340aba50 con 0x7feb400cf250 2024-05-20T18:34:01.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.032+0000 7feb3ffff640 1 --2- 172.21.15.5:0/3881985244 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feb2c06be30 0x7feb2c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.032+0000 7feb3ffff640 1 --2- 172.21.15.5:0/3881985244 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feb2c06be30 0x7feb2c06e340 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7feb3000aac0 tx=0x7feb30008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:01.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.033+0000 7feb4d824740 1 Processor -- start 2024-05-20T18:34:01.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.033+0000 7feb4d824740 1 -- start start 2024-05-20T18:34:01.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.034+0000 7feb4d824740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd0d60c350 0x55dd0d9d7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.034+0000 7feb4d824740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd0d5e3490 0x55dd0d9f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.034+0000 7feb4d824740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd0d5eb670 0x55dd0d9f28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.034+0000 7feb4d824740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dd0d910a30 con 0x55dd0d5eb670 2024-05-20T18:34:01.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.034+0000 7feb4d824740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dd0d8ae590 con 0x55dd0d5e3490 2024-05-20T18:34:01.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.034+0000 7feb4d824740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55dd0d90c9f0 con 0x55dd0d60c350 2024-05-20T18:34:01.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.034+0000 7feb44f14640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd0d5eb670 0x55dd0d9f28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.035+0000 7feb44f14640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd0d5eb670 0x55dd0d9f28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56624/0 (socket says 172.21.15.5:56624) 2024-05-20T18:34:01.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.035+0000 7feb44f14640 1 -- 172.21.15.5:0/2092579036 learned_addr learned my addr 172.21.15.5:0/2092579036 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:01.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.035+0000 7feb3f7fe640 1 --2- 172.21.15.5:0/2092579036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd0d5e3490 0x55dd0d9f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.035+0000 7feb44f14640 1 -- 172.21.15.5:0/2092579036 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd0d60c350 msgr2=0x55dd0d9d7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:01.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.035+0000 7feb3ffff640 1 --2- 172.21.15.5:0/2092579036 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd0d60c350 0x55dd0d9d7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.035+0000 7feb44f14640 1 --2- 172.21.15.5:0/2092579036 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd0d60c350 0x55dd0d9d7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.035+0000 7feb44f14640 1 -- 172.21.15.5:0/2092579036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd0d5e3490 msgr2=0x55dd0d9f23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:01.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.035+0000 7feb44f14640 1 --2- 172.21.15.5:0/2092579036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd0d5e3490 0x55dd0d9f23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.035+0000 7feb44f14640 1 -- 172.21.15.5:0/2092579036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd0d6f07c0 con 0x55dd0d5eb670 2024-05-20T18:34:01.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.035+0000 7feb44f14640 1 --2- 172.21.15.5:0/2092579036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd0d5eb670 0x55dd0d9f28e0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7feb38002070 tx=0x7feb3800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bcd324f0a105c99 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:01.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.035+0000 7feb25ffb640 1 -- 172.21.15.5:0/2092579036 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb38019070 con 0x55dd0d5eb670 2024-05-20T18:34:01.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.035+0000 7feb25ffb640 1 -- 172.21.15.5:0/2092579036 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb38014480 con 0x55dd0d5eb670 2024-05-20T18:34:01.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.035+0000 7feb25ffb640 1 -- 172.21.15.5:0/2092579036 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb38014830 con 0x55dd0d5eb670 2024-05-20T18:34:01.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.036+0000 7feb4d824740 1 -- 172.21.15.5:0/2092579036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd0d5eb670 msgr2=0x55dd0d9f28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.036+0000 7feb4d824740 1 --2- 172.21.15.5:0/2092579036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd0d5eb670 0x55dd0d9f28e0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7feb38002070 tx=0x7feb3800e530 comp rx=0 tx=0).stop 2024-05-20T18:34:01.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.036+0000 7feb4d824740 1 -- 172.21.15.5:0/2092579036 shutdown_connections 2024-05-20T18:34:01.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.036+0000 7feb4d824740 1 --2- 172.21.15.5:0/2092579036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd0d5eb670 0x55dd0d9f28e0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.036+0000 7feb4d824740 1 --2- 172.21.15.5:0/2092579036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd0d5e3490 0x55dd0d9f23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.036+0000 7feb4d824740 1 --2- 172.21.15.5:0/2092579036 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd0d60c350 0x55dd0d9d7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.036+0000 7feb4d824740 1 -- 172.21.15.5:0/2092579036 >> 172.21.15.5:0/2092579036 conn(0x55dd0d6004a0 msgr2=0x55dd0d9accd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:01.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.036+0000 7feb4d824740 1 -- 172.21.15.5:0/2092579036 shutdown_connections 2024-05-20T18:34:01.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.036+0000 7feb4d824740 1 -- 172.21.15.5:0/2092579036 wait complete. 2024-05-20T18:34:01.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.037+0000 7feb4d824740 1 -- start start 2024-05-20T18:34:01.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.037+0000 7feb4d824740 1 Processor -- start 2024-05-20T18:34:01.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.038+0000 7feb4d824740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd0d5e3490 0x55dd0d9ed340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.038+0000 7feb4d824740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd0d5eb670 0x55dd0d9da650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.038+0000 7feb4d824740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd0d60c350 0x55dd0da01920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.038+0000 7feb4d824740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dd0d8ac060 con 0x55dd0d5eb670 2024-05-20T18:34:01.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.038+0000 7feb4d824740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dd0da02440 con 0x55dd0d60c350 2024-05-20T18:34:01.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.038+0000 7feb4d824740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55dd0d9ed920 con 0x55dd0d5e3490 2024-05-20T18:34:01.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.039+0000 7feb4d824740 10 client.0 _ll_register_callbacks cb 0x55dd0d8d0dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:34:01.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.039+0000 7feb3f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd0d5eb670 0x55dd0d9da650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.039+0000 7feb3f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd0d5eb670 0x55dd0d9da650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56630/0 (socket says 172.21.15.5:56630) 2024-05-20T18:34:01.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.039+0000 7feb3f7fe640 1 -- 172.21.15.5:0/745111420 learned_addr learned my addr 172.21.15.5:0/745111420 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:01.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.039+0000 7feb3ffff640 1 --2- 172.21.15.5:0/745111420 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd0d5e3490 0x55dd0d9ed340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.039+0000 7feb44f14640 1 --2- 172.21.15.5:0/745111420 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd0d60c350 0x55dd0da01920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.039+0000 7feb3f7fe640 1 -- 172.21.15.5:0/745111420 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd0d5e3490 msgr2=0x55dd0d9ed340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:01.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.039+0000 7feb3f7fe640 1 --2- 172.21.15.5:0/745111420 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd0d5e3490 0x55dd0d9ed340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.039+0000 7feb3f7fe640 1 -- 172.21.15.5:0/745111420 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd0d60c350 msgr2=0x55dd0da01920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:01.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.039+0000 7feb3f7fe640 1 --2- 172.21.15.5:0/745111420 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd0d60c350 0x55dd0da01920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.039+0000 7feb3f7fe640 1 -- 172.21.15.5:0/745111420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd0d750f50 con 0x55dd0d5eb670 2024-05-20T18:34:01.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.039+0000 7feb44f14640 1 --2- 172.21.15.5:0/745111420 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd0d60c350 0x55dd0da01920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:34:01.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.039+0000 7feb3f7fe640 1 --2- 172.21.15.5:0/745111420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd0d5eb670 0x55dd0d9da650 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7feb340a9cd0 tx=0x7feb34079390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf900d4e1678a81 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:01.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.039+0000 7feb1effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:34:01.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.039+0000 7feb1effd640 1 -- 172.21.15.5:0/745111420 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb34096060 con 0x55dd0d5eb670 2024-05-20T18:34:01.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.039+0000 7feb1effd640 1 -- 172.21.15.5:0/745111420 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7feb34079c90 con 0x55dd0d5eb670 2024-05-20T18:34:01.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.039+0000 7feb4d824740 1 -- 172.21.15.5:0/745111420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb38014480 con 0x55dd0d5eb670 2024-05-20T18:34:01.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.039+0000 7feb1effd640 1 -- 172.21.15.5:0/745111420 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7feb3407dde0 con 0x55dd0d5eb670 2024-05-20T18:34:01.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.039+0000 7feb4d824740 10 client.7809 fetch_fsmap learned FSMap version 67 2024-05-20T18:34:01.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.039+0000 7feb4d824740 1 -- 172.21.15.5:0/745111420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55dd0d5e40b0 con 0x55dd0d5eb670 2024-05-20T18:34:01.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.039+0000 7feb1effd640 1 -- 172.21.15.5:0/745111420 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb34098e50 con 0x55dd0d5eb670 2024-05-20T18:34:01.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.040+0000 7feb1effd640 1 -- 172.21.15.5:0/745111420 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7feb3403a030 con 0x55dd0d5eb670 2024-05-20T18:34:01.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.040+0000 7feb4d824740 10 client.7809 fetch_fsmap finished waiting for FSMap version 67 2024-05-20T18:34:01.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.040+0000 7feb4d824740 10 client.7809 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-20T18:34:01.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.040+0000 7feb4d824740 20 client.7809 populate_metadata read hostname 'smithi005' 2024-05-20T18:34:01.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.040+0000 7feb4d824740 1 --2- 172.21.15.5:0/745111420 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55dd0d5cb130 0x55dd0d9d8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.040+0000 7feb4d824740 4 client.7809 mds_command: new command op to 7437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:34:01.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.040+0000 7feb4d824740 1 -- 172.21.15.5:0/745111420 --> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd0d91e160 con 0x55dd0d5cb130 2024-05-20T18:34:01.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.040+0000 7feb3ffff640 1 --2- 172.21.15.5:0/745111420 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55dd0d5cb130 0x55dd0d9d8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.040+0000 7feb1effd640 10 client.7809 ms_handle_connect on v2:172.21.15.5:6836/3556492262 2024-05-20T18:34:01.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.040+0000 7feb3ffff640 1 --2- 172.21.15.5:0/745111420 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55dd0d5cb130 0x55dd0d9d8be0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7a30e19855cf005 server_cookie=d28c6fa4b9bd1d28 in_seq=0 out_seq=0 2024-05-20T18:34:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.041+0000 7feb1effd640 1 -- 172.21.15.5:0/745111420 <== mds.0 v2:172.21.15.5:6836/3556492262 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dd0d91e160 con 0x55dd0d5cb130 2024-05-20T18:34:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.041+0000 7feb1effd640 10 client.7809 handle_command_reply: tid=0 2024-05-20T18:34:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.041+0000 7feb4d824740 1 client.7809 shutdown 2024-05-20T18:34:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.041+0000 7feb4d824740 1 -- 172.21.15.5:0/745111420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd0d5eb670 msgr2=0x55dd0d9da650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.041+0000 7feb4d824740 1 --2- 172.21.15.5:0/745111420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd0d5eb670 0x55dd0d9da650 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7feb340a9cd0 tx=0x7feb34079390 comp rx=0 tx=0).stop 2024-05-20T18:34:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.041+0000 7feb4d824740 1 -- 172.21.15.5:0/745111420 shutdown_connections 2024-05-20T18:34:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.041+0000 7feb4d824740 1 --2- 172.21.15.5:0/745111420 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd0d60c350 0x55dd0da01920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.041+0000 7feb4d824740 1 --2- 172.21.15.5:0/745111420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd0d5eb670 0x55dd0d9da650 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.041+0000 7feb4d824740 1 --2- 172.21.15.5:0/745111420 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55dd0d5cb130 0x55dd0d9d8be0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.041+0000 7feb4d824740 1 --2- 172.21.15.5:0/745111420 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd0d5e3490 0x55dd0d9ed340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.041+0000 7feb1effd640 0 client.7809 ms_handle_reset on v2:172.21.15.5:6836/3556492262 2024-05-20T18:34:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.041+0000 7feb4d824740 1 -- 172.21.15.5:0/745111420 >> 172.21.15.5:0/745111420 conn(0x55dd0d6004a0 msgr2=0x55dd0d9accd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.041+0000 7feb4d824740 1 -- 172.21.15.5:0/745111420 shutdown_connections 2024-05-20T18:34:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.041+0000 7feb4d824740 1 -- 172.21.15.5:0/745111420 wait complete. 2024-05-20T18:34:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.041+0000 7feb4d824740 20 client.7809 trim_cache size 0 max 16384 2024-05-20T18:34:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.055+0000 7feb4d824740 1 Processor -- start 2024-05-20T18:34:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.055+0000 7feb4d824740 1 -- start start 2024-05-20T18:34:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.057+0000 7feb4d824740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd0d5e3490 0x55dd0d9c6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.057+0000 7feb4d824740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd0d5eb670 0x55dd0d9f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.057+0000 7feb4d824740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd0d60c350 0x55dd0d9f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.057+0000 7feb4d824740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feb40201630 con 0x55dd0d5eb670 2024-05-20T18:34:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.057+0000 7feb4d824740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feb40150d00 con 0x55dd0d5e3490 2024-05-20T18:34:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.057+0000 7feb4d824740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7feb40156ee0 con 0x55dd0d60c350 2024-05-20T18:34:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.057+0000 7feb3f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd0d5eb670 0x55dd0d9f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.057+0000 7feb3f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd0d5eb670 0x55dd0d9f3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56642/0 (socket says 172.21.15.5:56642) 2024-05-20T18:34:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.057+0000 7feb3f7fe640 1 -- 172.21.15.5:0/100216186 learned_addr learned my addr 172.21.15.5:0/100216186 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.057+0000 7feb3f7fe640 1 -- 172.21.15.5:0/100216186 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd0d60c350 msgr2=0x55dd0d9f4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.057+0000 7feb3f7fe640 1 --2- 172.21.15.5:0/100216186 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd0d60c350 0x55dd0d9f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.057+0000 7feb3f7fe640 1 -- 172.21.15.5:0/100216186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd0d5e3490 msgr2=0x55dd0d9c6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.057+0000 7feb3f7fe640 1 --2- 172.21.15.5:0/100216186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd0d5e3490 0x55dd0d9c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.057+0000 7feb3f7fe640 1 -- 172.21.15.5:0/100216186 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd0d9aad20 con 0x55dd0d5eb670 2024-05-20T18:34:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.057+0000 7feb3f7fe640 1 --2- 172.21.15.5:0/100216186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd0d5eb670 0x55dd0d9f3d30 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7feb3407dfa0 tx=0x7feb3408f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905f6c699011b2a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.057+0000 7feb257fa640 1 -- 172.21.15.5:0/100216186 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb34096030 con 0x55dd0d5eb670 2024-05-20T18:34:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.057+0000 7feb257fa640 1 -- 172.21.15.5:0/100216186 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb38014480 con 0x55dd0d5eb670 2024-05-20T18:34:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.057+0000 7feb257fa640 1 -- 172.21.15.5:0/100216186 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb34098e50 con 0x55dd0d5eb670 2024-05-20T18:34:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.058+0000 7feb4d824740 1 -- 172.21.15.5:0/100216186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd0d5eb670 msgr2=0x55dd0d9f3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.058+0000 7feb4d824740 1 --2- 172.21.15.5:0/100216186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd0d5eb670 0x55dd0d9f3d30 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7feb3407dfa0 tx=0x7feb3408f000 comp rx=0 tx=0).stop 2024-05-20T18:34:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.058+0000 7feb4d824740 1 -- 172.21.15.5:0/100216186 shutdown_connections 2024-05-20T18:34:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.058+0000 7feb4d824740 1 --2- 172.21.15.5:0/100216186 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd0d60c350 0x55dd0d9f4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.058+0000 7feb4d824740 1 --2- 172.21.15.5:0/100216186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd0d5eb670 0x55dd0d9f3d30 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.058+0000 7feb4d824740 1 --2- 172.21.15.5:0/100216186 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd0d5e3490 0x55dd0d9c6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.058+0000 7feb4d824740 1 -- 172.21.15.5:0/100216186 >> 172.21.15.5:0/100216186 conn(0x55dd0d6004a0 msgr2=0x55dd0d9ebe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.058+0000 7feb4d824740 1 -- 172.21.15.5:0/100216186 shutdown_connections 2024-05-20T18:34:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.058+0000 7feb4d824740 1 -- 172.21.15.5:0/100216186 wait complete. 2024-05-20T18:34:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.059+0000 7feb4d824740 1 -- start start 2024-05-20T18:34:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.059+0000 7feb4d824740 1 Processor -- start 2024-05-20T18:34:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.060+0000 7feb4d824740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd0d5e3490 0x55dd0d9d4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.060+0000 7feb4d824740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd0d5eb670 0x55dd0d9d73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.060+0000 7feb4d824740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd0d60c350 0x55dd0d9d7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.060+0000 7feb4d824740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dd0d9f5b80 con 0x55dd0d5e3490 2024-05-20T18:34:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.060+0000 7feb4d824740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dd0da01e60 con 0x55dd0d5eb670 2024-05-20T18:34:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.060+0000 7feb4d824740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55dd0d9f61d0 con 0x55dd0d60c350 2024-05-20T18:34:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.061+0000 7feb4d824740 10 client.0 _ll_register_callbacks cb 0x7feb34030ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:34:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.061+0000 7feb3ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd0d5e3490 0x55dd0d9d4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.061+0000 7feb3ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd0d5e3490 0x55dd0d9d4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56652/0 (socket says 172.21.15.5:56652) 2024-05-20T18:34:01.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.061+0000 7feb3ffff640 1 -- 172.21.15.5:0/4151543034 learned_addr learned my addr 172.21.15.5:0/4151543034 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:01.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.061+0000 7feb3f7fe640 1 --2- 172.21.15.5:0/4151543034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd0d5eb670 0x55dd0d9d73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.061+0000 7feb44f14640 1 --2- 172.21.15.5:0/4151543034 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd0d60c350 0x55dd0d9d7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.061+0000 7feb3ffff640 1 -- 172.21.15.5:0/4151543034 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd0d60c350 msgr2=0x55dd0d9d7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:01.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.061+0000 7feb3ffff640 1 --2- 172.21.15.5:0/4151543034 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd0d60c350 0x55dd0d9d7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.061+0000 7feb3ffff640 1 -- 172.21.15.5:0/4151543034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd0d5eb670 msgr2=0x55dd0d9d73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:01.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.061+0000 7feb3ffff640 1 --2- 172.21.15.5:0/4151543034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd0d5eb670 0x55dd0d9d73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.061+0000 7feb3ffff640 1 -- 172.21.15.5:0/4151543034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd0da15720 con 0x55dd0d5e3490 2024-05-20T18:34:01.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.061+0000 7feb3f7fe640 1 --2- 172.21.15.5:0/4151543034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd0d5eb670 0x55dd0d9d73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:34:01.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.061+0000 7feb3ffff640 1 --2- 172.21.15.5:0/4151543034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd0d5e3490 0x55dd0d9d4420 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x55dd0d9d8730 tx=0x7feb300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c5c385de39235 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:01.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.061+0000 7feb1effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:34:01.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.061+0000 7feb1effd640 1 -- 172.21.15.5:0/4151543034 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb30027310 con 0x55dd0d5e3490 2024-05-20T18:34:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.061+0000 7feb1effd640 1 -- 172.21.15.5:0/4151543034 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7feb300274b0 con 0x55dd0d5e3490 2024-05-20T18:34:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.061+0000 7feb4d824740 1 -- 172.21.15.5:0/4151543034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb38014480 con 0x55dd0d5e3490 2024-05-20T18:34:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.061+0000 7feb1effd640 1 -- 172.21.15.5:0/4151543034 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7feb30035050 con 0x55dd0d5e3490 2024-05-20T18:34:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.061+0000 7feb4d824740 10 client.7815 fetch_fsmap learned FSMap version 67 2024-05-20T18:34:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.061+0000 7feb4d824740 1 -- 172.21.15.5:0/4151543034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55dd0da0a7b0 con 0x55dd0d5e3490 2024-05-20T18:34:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.062+0000 7feb1effd640 1 -- 172.21.15.5:0/4151543034 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb38014480 con 0x55dd0d5e3490 2024-05-20T18:34:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.062+0000 7feb1effd640 1 -- 172.21.15.5:0/4151543034 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7feb30038020 con 0x55dd0d5e3490 2024-05-20T18:34:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.062+0000 7feb4d824740 10 client.7815 fetch_fsmap finished waiting for FSMap version 67 2024-05-20T18:34:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.062+0000 7feb4d824740 10 client.7815 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-20T18:34:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.062+0000 7feb4d824740 20 client.7815 populate_metadata read hostname 'smithi005' 2024-05-20T18:34:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.062+0000 7feb4d824740 1 --2- 172.21.15.5:0/4151543034 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55dd0d9f56e0 0x55dd0d9c5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.062+0000 7feb4d824740 4 client.7815 mds_command: new command op to 7437 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-20T18:34:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.062+0000 7feb4d824740 1 -- 172.21.15.5:0/4151543034 --> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55dd0d9184b0 con 0x55dd0d9f56e0 2024-05-20T18:34:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.062+0000 7feb3f7fe640 1 --2- 172.21.15.5:0/4151543034 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55dd0d9f56e0 0x55dd0d9c5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.062+0000 7feb1effd640 10 client.7815 ms_handle_connect on v2:172.21.15.5:6836/3556492262 2024-05-20T18:34:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.062+0000 7feb3f7fe640 1 --2- 172.21.15.5:0/4151543034 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55dd0d9f56e0 0x55dd0d9c5100 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cbde116d9132af7 server_cookie=85ae53efd95a2665 in_seq=0 out_seq=0 2024-05-20T18:34:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.062+0000 7feb1effd640 1 -- 172.21.15.5:0/4151543034 <== mds.0 v2:172.21.15.5:6836/3556492262 1 ==== command_reply(tid 0: 0 ) ==== 8+0+420 (crc 0 0 0) 0x55dd0d9184b0 con 0x55dd0d9f56e0 2024-05-20T18:34:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.062+0000 7feb1effd640 10 client.7815 handle_command_reply: tid=0 2024-05-20T18:34:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.062+0000 7feb4d824740 1 client.7815 shutdown 2024-05-20T18:34:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.063+0000 7feb4d824740 1 -- 172.21.15.5:0/4151543034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd0d5e3490 msgr2=0x55dd0d9d4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.063+0000 7feb4d824740 1 --2- 172.21.15.5:0/4151543034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd0d5e3490 0x55dd0d9d4420 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x55dd0d9d8730 tx=0x7feb300046c0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.063+0000 7feb4d824740 1 -- 172.21.15.5:0/4151543034 shutdown_connections 2024-05-20T18:34:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.063+0000 7feb4d824740 1 --2- 172.21.15.5:0/4151543034 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55dd0d9f56e0 0x55dd0d9c5100 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.063+0000 7feb4d824740 1 --2- 172.21.15.5:0/4151543034 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd0d60c350 0x55dd0d9d7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.063+0000 7feb4d824740 1 --2- 172.21.15.5:0/4151543034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd0d5eb670 0x55dd0d9d73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.063+0000 7feb1effd640 0 client.7815 ms_handle_reset on v2:172.21.15.5:6836/3556492262 2024-05-20T18:34:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.063+0000 7feb4d824740 1 --2- 172.21.15.5:0/4151543034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd0d5e3490 0x55dd0d9d4420 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.063+0000 7feb4d824740 1 -- 172.21.15.5:0/4151543034 >> 172.21.15.5:0/4151543034 conn(0x55dd0d6004a0 msgr2=0x55dd0d9a2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.063+0000 7feb4d824740 1 -- 172.21.15.5:0/4151543034 shutdown_connections 2024-05-20T18:34:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.063+0000 7feb4d824740 1 -- 172.21.15.5:0/4151543034 wait complete. 2024-05-20T18:34:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.063+0000 7feb4d824740 20 client.7815 trim_cache size 0 max 16384 2024-05-20T18:34:01.069 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:34:01.069 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-05-20T18:34:01.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:34:01.069 INFO:teuthology.orchestra.run.smithi005.stdout: "damage_type": "dir_frag", 2024-05-20T18:34:01.069 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 1174955014, 2024-05-20T18:34:01.070 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 1099511627776, 2024-05-20T18:34:01.070 INFO:teuthology.orchestra.run.smithi005.stdout: "frag": "*", 2024-05-20T18:34:01.070 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/dir" 2024-05-20T18:34:01.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:34:01.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:34:01.070 INFO:teuthology.orchestra.run.smithi005.stdout: "damage_type": "backtrace", 2024-05-20T18:34:01.070 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 2268836207, 2024-05-20T18:34:01.070 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 1099511627776, 2024-05-20T18:34:01.070 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/dir" 2024-05-20T18:34:01.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:34:01.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:34:01.070 INFO:teuthology.orchestra.run.smithi005.stdout: "damage_type": "backtrace", 2024-05-20T18:34:01.070 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 3953132379, 2024-05-20T18:34:01.070 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 1099511627777, 2024-05-20T18:34:01.071 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/testdir/hardlink" 2024-05-20T18:34:01.071 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:34:01.071 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-05-20T18:34:01.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.063+0000 7feb46977640 1 -- 172.21.15.5:0/3881985244 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feb2c06be30 msgr2=0x7feb2c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.063+0000 7feb46977640 1 --2- 172.21.15.5:0/3881985244 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feb2c06be30 0x7feb2c06e340 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7feb3000aac0 tx=0x7feb30008040 comp rx=0 tx=0).stop 2024-05-20T18:34:01.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.063+0000 7feb46977640 1 -- 172.21.15.5:0/3881985244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb400cf250 msgr2=0x7feb4014f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.063+0000 7feb46977640 1 --2- 172.21.15.5:0/3881985244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb400cf250 0x7feb4014f690 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7feb34009890 tx=0x7feb340046c0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.063+0000 7feb46977640 1 -- 172.21.15.5:0/3881985244 shutdown_connections 2024-05-20T18:34:01.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.063+0000 7feb46977640 1 --2- 172.21.15.5:0/3881985244 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feb2c06be30 0x7feb2c06e340 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.063+0000 7feb46977640 1 --2- 172.21.15.5:0/3881985244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb40163020 0x7feb400b9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.063+0000 7feb46977640 1 --2- 172.21.15.5:0/3881985244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb400cf250 0x7feb4014f690 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.063+0000 7feb46977640 1 --2- 172.21.15.5:0/3881985244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb400b6680 0x7feb40158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.063+0000 7feb46977640 1 -- 172.21.15.5:0/3881985244 >> 172.21.15.5:0/3881985244 conn(0x7feb400d1860 msgr2=0x7feb400b4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:01.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.064+0000 7feb46977640 1 -- 172.21.15.5:0/3881985244 shutdown_connections 2024-05-20T18:34:01.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.064+0000 7feb46977640 1 -- 172.21.15.5:0/3881985244 wait complete. 2024-05-20T18:34:01.075 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-20T18:34:01.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.190+0000 7f7a8a0bc640 1 Processor -- start 2024-05-20T18:34:01.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.190+0000 7f7a8a0bc640 1 -- start start 2024-05-20T18:34:01.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.192+0000 7f7a8a0bc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a840d0560 0x7f7a8406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.192+0000 7f7a8a0bc640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a8406dd20 con 0x7f7a8406d910 2024-05-20T18:34:01.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.192+0000 7f7a8a0bc640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a8415fa90 con 0x7f7a840b4070 2024-05-20T18:34:01.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.192+0000 7f7a8a0bc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7a840b8490 con 0x7f7a840d0560 2024-05-20T18:34:01.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.192+0000 7f7a83fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7a8406d910 0x7f7a840b8920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35456/0 (socket says 172.21.15.5:35456) 2024-05-20T18:34:01.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.192+0000 7f7a83fff640 1 -- 172.21.15.5:0/1289777568 learned_addr learned my addr 172.21.15.5:0/1289777568 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:34:01.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.192+0000 7f7a82ffd640 1 --2- 172.21.15.5:0/1289777568 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a840d0560 0x7f7a8406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.192+0000 7f7a827fc640 1 -- 172.21.15.5:0/1289777568 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4092028489 0 0) 0x7f7a7c001490 con 0x7f7a8406d910 2024-05-20T18:34:01.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.192+0000 7f7a827fc640 1 -- 172.21.15.5:0/1289777568 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a6c003520 con 0x7f7a8406d910 2024-05-20T18:34:01.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.192+0000 7f7a827fc640 1 -- 172.21.15.5:0/1289777568 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2202412739 0 0) 0x7f7a8415fa90 con 0x7f7a840b4070 2024-05-20T18:34:01.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.192+0000 7f7a827fc640 1 -- 172.21.15.5:0/1289777568 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a7c001490 con 0x7f7a840b4070 2024-05-20T18:34:01.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.192+0000 7f7a827fc640 1 -- 172.21.15.5:0/1289777568 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 316228681 0 0) 0x7f7a6c003520 con 0x7f7a8406d910 2024-05-20T18:34:01.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.192+0000 7f7a82ffd640 1 -- 172.21.15.5:0/1289777568 >> v1:172.21.15.63:6789/0 conn(0x7f7a840b4070 legacy=0x7f7a840ba5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.192+0000 7f7a82ffd640 1 -- 172.21.15.5:0/1289777568 >> v1:172.21.15.5:6789/0 conn(0x7f7a8406d910 legacy=0x7f7a840b8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.192+0000 7f7a82ffd640 1 -- 172.21.15.5:0/1289777568 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a840d3f00 con 0x7f7a840d0560 2024-05-20T18:34:01.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.193+0000 7f7a82ffd640 1 --2- 172.21.15.5:0/1289777568 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a840d0560 0x7f7a8406d3d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7a780097d0 tx=0x7f7a7802a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=548d6e9e507e72a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:01.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.193+0000 7f7a827fc640 1 -- 172.21.15.5:0/1289777568 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78037070 con 0x7f7a840d0560 2024-05-20T18:34:01.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.194+0000 7f7a827fc640 1 -- 172.21.15.5:0/1289777568 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a7802ac30 con 0x7f7a840d0560 2024-05-20T18:34:01.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.194+0000 7f7a827fc640 1 -- 172.21.15.5:0/1289777568 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c001bc0 con 0x7f7a840d0560 2024-05-20T18:34:01.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.194+0000 7f7a8a0bc640 1 -- 172.21.15.5:0/1289777568 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a840d0560 msgr2=0x7f7a8406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.194+0000 7f7a8a0bc640 1 --2- 172.21.15.5:0/1289777568 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a840d0560 0x7f7a8406d3d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7a780097d0 tx=0x7f7a7802a1a0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.194+0000 7f7a8a0bc640 1 -- 172.21.15.5:0/1289777568 shutdown_connections 2024-05-20T18:34:01.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.194+0000 7f7a8a0bc640 1 --2- 172.21.15.5:0/1289777568 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a840d0560 0x7f7a8406d3d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.194+0000 7f7a8a0bc640 1 -- 172.21.15.5:0/1289777568 >> 172.21.15.5:0/1289777568 conn(0x7f7a840d4c10 msgr2=0x7f7a840d5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:01.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.194+0000 7f7a8a0bc640 1 -- 172.21.15.5:0/1289777568 shutdown_connections 2024-05-20T18:34:01.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.194+0000 7f7a8a0bc640 1 -- 172.21.15.5:0/1289777568 wait complete. 2024-05-20T18:34:01.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.196+0000 7f7a8a0bc640 1 Processor -- start 2024-05-20T18:34:01.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.196+0000 7f7a8a0bc640 1 -- start start 2024-05-20T18:34:01.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.198+0000 7f7a8a0bc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a8406d910 0x7f7a840c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.198+0000 7f7a8a0bc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a840b4070 0x7f7a84154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.198+0000 7f7a8a0bc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a840d0560 0x7f7a840ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.198+0000 7f7a8a0bc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7a840d47a0 con 0x7f7a8406d910 2024-05-20T18:34:01.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.198+0000 7f7a8a0bc640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7a840d44c0 con 0x7f7a840d0560 2024-05-20T18:34:01.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.198+0000 7f7a8a0bc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7a840d3b20 con 0x7f7a840b4070 2024-05-20T18:34:01.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.198+0000 7f7a837fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a8406d910 0x7f7a840c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.198+0000 7f7a837fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a8406d910 0x7f7a840c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56666/0 (socket says 172.21.15.5:56666) 2024-05-20T18:34:01.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.198+0000 7f7a837fe640 1 -- 172.21.15.5:0/1110051125 learned_addr learned my addr 172.21.15.5:0/1110051125 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:01.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.198+0000 7f7a82ffd640 1 --2- 172.21.15.5:0/1110051125 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a840b4070 0x7f7a84154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.198+0000 7f7a83fff640 1 --2- 172.21.15.5:0/1110051125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a840d0560 0x7f7a840ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.198+0000 7f7a837fe640 1 -- 172.21.15.5:0/1110051125 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a840b4070 msgr2=0x7f7a84154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.198+0000 7f7a837fe640 1 --2- 172.21.15.5:0/1110051125 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a840b4070 0x7f7a84154780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.198+0000 7f7a837fe640 1 -- 172.21.15.5:0/1110051125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a840d0560 msgr2=0x7f7a840ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.198+0000 7f7a837fe640 1 --2- 172.21.15.5:0/1110051125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a840d0560 0x7f7a840ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.198+0000 7f7a837fe640 1 -- 172.21.15.5:0/1110051125 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a84160920 con 0x7f7a8406d910 2024-05-20T18:34:01.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.198+0000 7f7a83fff640 1 --2- 172.21.15.5:0/1110051125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a840d0560 0x7f7a840ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:34:01.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.198+0000 7f7a837fe640 1 --2- 172.21.15.5:0/1110051125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a8406d910 0x7f7a840c8db0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400c460 tx=0x7f7a7400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:01.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.198+0000 7f7a6bfff640 1 -- 172.21.15.5:0/1110051125 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a740052d0 con 0x7f7a8406d910 2024-05-20T18:34:01.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.198+0000 7f7a6bfff640 1 -- 172.21.15.5:0/1110051125 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a74005490 con 0x7f7a8406d910 2024-05-20T18:34:01.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.198+0000 7f7a8a0bc640 1 -- 172.21.15.5:0/1110051125 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a84160c00 con 0x7f7a8406d910 2024-05-20T18:34:01.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.198+0000 7f7a6bfff640 1 -- 172.21.15.5:0/1110051125 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a74012870 con 0x7f7a8406d910 2024-05-20T18:34:01.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.198+0000 7f7a8a0bc640 1 -- 172.21.15.5:0/1110051125 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a841611c0 con 0x7f7a8406d910 2024-05-20T18:34:01.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.199+0000 7f7a8a0bc640 1 -- 172.21.15.5:0/1110051125 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a840690b0 con 0x7f7a8406d910 2024-05-20T18:34:01.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.200+0000 7f7a6bfff640 1 -- 172.21.15.5:0/1110051125 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7a74012a10 con 0x7f7a8406d910 2024-05-20T18:34:01.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.201+0000 7f7a6bfff640 1 --2- 172.21.15.5:0/1110051125 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7a7006be30 0x7f7a7006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.201+0000 7f7a6bfff640 1 -- 172.21.15.5:0/1110051125 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7f7a7408ac30 con 0x7f7a8406d910 2024-05-20T18:34:01.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.201+0000 7f7a82ffd640 1 --2- 172.21.15.5:0/1110051125 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7a7006be30 0x7f7a7006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.201+0000 7f7a82ffd640 1 --2- 172.21.15.5:0/1110051125 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7a7006be30 0x7f7a7006e340 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7a78034000 tx=0x7f7a7802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:01.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.205+0000 7f7a6bfff640 1 -- 172.21.15.5:0/1110051125 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7a74059390 con 0x7f7a8406d910 2024-05-20T18:34:01.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.429+0000 7f7a8a0bc640 1 -- 172.21.15.5:0/1110051125 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f7a840b78d0 con 0x7f7a8406d910 2024-05-20T18:34:01.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.429+0000 7f7a6bfff640 1 -- 172.21.15.5:0/1110051125 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f7a74058c90 con 0x7f7a8406d910 2024-05-20T18:34:01.430 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:34:01.431 INFO:teuthology.orchestra.run.smithi005.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-05-20T18:34:01.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.431+0000 7f7a8a0bc640 1 -- 172.21.15.5:0/1110051125 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7a7006be30 msgr2=0x7f7a7006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.431+0000 7f7a8a0bc640 1 --2- 172.21.15.5:0/1110051125 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7a7006be30 0x7f7a7006e340 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7a78034000 tx=0x7f7a7802e000 comp rx=0 tx=0).stop 2024-05-20T18:34:01.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.431+0000 7f7a8a0bc640 1 -- 172.21.15.5:0/1110051125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a8406d910 msgr2=0x7f7a840c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.431+0000 7f7a8a0bc640 1 --2- 172.21.15.5:0/1110051125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a8406d910 0x7f7a840c8db0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400c460 tx=0x7f7a7400c920 comp rx=0 tx=0).stop 2024-05-20T18:34:01.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.431+0000 7f7a8a0bc640 1 -- 172.21.15.5:0/1110051125 shutdown_connections 2024-05-20T18:34:01.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.431+0000 7f7a8a0bc640 1 --2- 172.21.15.5:0/1110051125 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7a7006be30 0x7f7a7006e340 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.431+0000 7f7a8a0bc640 1 --2- 172.21.15.5:0/1110051125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a840d0560 0x7f7a840ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.431+0000 7f7a8a0bc640 1 --2- 172.21.15.5:0/1110051125 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a840b4070 0x7f7a84154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.431+0000 7f7a8a0bc640 1 --2- 172.21.15.5:0/1110051125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a8406d910 0x7f7a840c8db0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.431+0000 7f7a8a0bc640 1 -- 172.21.15.5:0/1110051125 >> 172.21.15.5:0/1110051125 conn(0x7f7a840d4c10 msgr2=0x7f7a840d27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:01.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.432+0000 7f7a8a0bc640 1 -- 172.21.15.5:0/1110051125 shutdown_connections 2024-05-20T18:34:01.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.432+0000 7f7a8a0bc640 1 -- 172.21.15.5:0/1110051125 wait complete. 2024-05-20T18:34:01.444 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-20T18:34:01.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:34:01.444 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-05-20T18:34:01.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.561+0000 7f9bc1f2c640 1 Processor -- start 2024-05-20T18:34:01.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.561+0000 7f9bc1f2c640 1 -- start start 2024-05-20T18:34:01.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.563+0000 7f9bc1f2c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9bbc0b3860 0x7f9bbc0d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.563+0000 7f9bc1f2c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9bbc06d870 0x7f9bbc15f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.563+0000 7f9bc1f2c640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9bbc08bd20 0x7f9bbc08c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.563+0000 7f9bc1f2c640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f9bbc08c670 con 0x7f9bbc06d870 2024-05-20T18:34:01.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.563+0000 7f9bc1f2c640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f9bbc0d0450 con 0x7f9bbc08bd20 2024-05-20T18:34:01.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.563+0000 7f9bc1f2c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9bbc0d3b10 con 0x7f9bbc0b3860 2024-05-20T18:34:01.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.563+0000 7f9bbaffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9bbc06d870 0x7f9bbc15f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.563+0000 7f9bbaffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9bbc06d870 0x7f9bbc15f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56680/0 (socket says 172.21.15.5:56680) 2024-05-20T18:34:01.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.563+0000 7f9bbaffd640 1 -- 172.21.15.5:0/1956436185 learned_addr learned my addr 172.21.15.5:0/1956436185 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:01.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.563+0000 7f9bbb7fe640 1 --2- 172.21.15.5:0/1956436185 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9bbc0b3860 0x7f9bbc0d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.563+0000 7f9bbbfff640 1 --2- 172.21.15.5:0/1956436185 >> v2:172.21.15.63:3300/0 conn(0x7f9bbc08bd20 0x7f9bbc08c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.563+0000 7f9bbaffd640 1 -- 172.21.15.5:0/1956436185 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9bbc0b3860 msgr2=0x7f9bbc0d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:01.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.563+0000 7f9bbaffd640 1 --2- 172.21.15.5:0/1956436185 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9bbc0b3860 0x7f9bbc0d45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.563+0000 7f9bbaffd640 1 -- 172.21.15.5:0/1956436185 >> v2:172.21.15.63:3300/0 conn(0x7f9bbc08bd20 msgr2=0x7f9bbc08c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:01.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.563+0000 7f9bbaffd640 1 --2- 172.21.15.5:0/1956436185 >> v2:172.21.15.63:3300/0 conn(0x7f9bbc08bd20 0x7f9bbc08c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.563+0000 7f9bbaffd640 1 -- 172.21.15.5:0/1956436185 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bbc0d3df0 con 0x7f9bbc06d870 2024-05-20T18:34:01.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.564+0000 7f9bbaffd640 1 --2- 172.21.15.5:0/1956436185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bbc06d870 0x7f9bbc15f230 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc06e310 tx=0x7f9bb002a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aab4dcfde5751ba server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:01.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.564+0000 7f9bba7fc640 1 -- 172.21.15.5:0/1956436185 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0037070 con 0x7f9bbc06d870 2024-05-20T18:34:01.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.564+0000 7f9bba7fc640 1 -- 172.21.15.5:0/1956436185 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bb002ae00 con 0x7f9bbc06d870 2024-05-20T18:34:01.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.564+0000 7f9bba7fc640 1 -- 172.21.15.5:0/1956436185 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0032660 con 0x7f9bbc06d870 2024-05-20T18:34:01.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.565+0000 7f9bc1f2c640 1 -- 172.21.15.5:0/1956436185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bbc06d870 msgr2=0x7f9bbc15f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.565+0000 7f9bc1f2c640 1 --2- 172.21.15.5:0/1956436185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bbc06d870 0x7f9bbc15f230 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc06e310 tx=0x7f9bb002a0b0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.565+0000 7f9bc1f2c640 1 -- 172.21.15.5:0/1956436185 shutdown_connections 2024-05-20T18:34:01.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.565+0000 7f9bc1f2c640 1 --2- 172.21.15.5:0/1956436185 >> v2:172.21.15.63:3300/0 conn(0x7f9bbc08bd20 0x7f9bbc08c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.565+0000 7f9bc1f2c640 1 --2- 172.21.15.5:0/1956436185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bbc06d870 0x7f9bbc15f230 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.565+0000 7f9bc1f2c640 1 --2- 172.21.15.5:0/1956436185 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9bbc0b3860 0x7f9bbc0d45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.565+0000 7f9bc1f2c640 1 -- 172.21.15.5:0/1956436185 >> 172.21.15.5:0/1956436185 conn(0x7f9bbc0b6d10 msgr2=0x7f9bbc06d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:01.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.565+0000 7f9bc1f2c640 1 -- 172.21.15.5:0/1956436185 shutdown_connections 2024-05-20T18:34:01.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.565+0000 7f9bc1f2c640 1 -- 172.21.15.5:0/1956436185 wait complete. 2024-05-20T18:34:01.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.567+0000 7f9bc1f2c640 1 Processor -- start 2024-05-20T18:34:01.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.567+0000 7f9bc1f2c640 1 -- start start 2024-05-20T18:34:01.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.568+0000 7f9bc1f2c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9bbc06d870 0x7f9bbc1f4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.568+0000 7f9bc1f2c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bbc08bd20 0x7f9bbc0c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.568+0000 7f9bc1f2c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9bbc0b3860 0x7f9bbc0c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.568+0000 7f9bc1f2c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc15f770 con 0x7f9bbc08bd20 2024-05-20T18:34:01.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.568+0000 7f9bc1f2c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc0d0730 con 0x7f9bbc0b3860 2024-05-20T18:34:01.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.568+0000 7f9bc1f2c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9bbc0d27c0 con 0x7f9bbc06d870 2024-05-20T18:34:01.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.568+0000 7f9bbaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bbc08bd20 0x7f9bbc0c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.568+0000 7f9bbaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bbc08bd20 0x7f9bbc0c92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56686/0 (socket says 172.21.15.5:56686) 2024-05-20T18:34:01.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.568+0000 7f9bbaffd640 1 -- 172.21.15.5:0/1278212362 learned_addr learned my addr 172.21.15.5:0/1278212362 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:01.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.568+0000 7f9bbbfff640 1 --2- 172.21.15.5:0/1278212362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9bbc0b3860 0x7f9bbc0c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.568+0000 7f9bbb7fe640 1 --2- 172.21.15.5:0/1278212362 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9bbc06d870 0x7f9bbc1f4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.569+0000 7f9bbaffd640 1 -- 172.21.15.5:0/1278212362 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9bbc06d870 msgr2=0x7f9bbc1f4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.569+0000 7f9bbaffd640 1 --2- 172.21.15.5:0/1278212362 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9bbc06d870 0x7f9bbc1f4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.569+0000 7f9bbaffd640 1 -- 172.21.15.5:0/1278212362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9bbc0b3860 msgr2=0x7f9bbc0c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.569+0000 7f9bbaffd640 1 --2- 172.21.15.5:0/1278212362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9bbc0b3860 0x7f9bbc0c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.569+0000 7f9bbaffd640 1 -- 172.21.15.5:0/1278212362 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bbc0ca060 con 0x7f9bbc08bd20 2024-05-20T18:34:01.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.569+0000 7f9bbaffd640 1 --2- 172.21.15.5:0/1278212362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bbc08bd20 0x7f9bbc0c92c0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f9bb002a0f0 tx=0x7f9bb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:01.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.569+0000 7f9ba3fff640 1 -- 172.21.15.5:0/1278212362 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0041070 con 0x7f9bbc08bd20 2024-05-20T18:34:01.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.569+0000 7f9ba3fff640 1 -- 172.21.15.5:0/1278212362 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bb003c460 con 0x7f9bbc08bd20 2024-05-20T18:34:01.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.569+0000 7f9bc1f2c640 1 -- 172.21.15.5:0/1278212362 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bbc14a8a0 con 0x7f9bbc08bd20 2024-05-20T18:34:01.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.569+0000 7f9ba3fff640 1 -- 172.21.15.5:0/1278212362 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0037040 con 0x7f9bbc08bd20 2024-05-20T18:34:01.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.569+0000 7f9bc1f2c640 1 -- 172.21.15.5:0/1278212362 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bbc14ae60 con 0x7f9bbc08bd20 2024-05-20T18:34:01.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.571+0000 7f9ba3fff640 1 -- 172.21.15.5:0/1278212362 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9bb003a070 con 0x7f9bbc08bd20 2024-05-20T18:34:01.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.571+0000 7f9ba3fff640 1 --2- 172.21.15.5:0/1278212362 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9ba806be60 0x7f9ba806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.572+0000 7f9ba3fff640 1 -- 172.21.15.5:0/1278212362 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7f9bb002ae00 con 0x7f9bbc08bd20 2024-05-20T18:34:01.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.572+0000 7f9bbb7fe640 1 --2- 172.21.15.5:0/1278212362 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9ba806be60 0x7f9ba806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.572+0000 7f9bbb7fe640 1 --2- 172.21.15.5:0/1278212362 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9ba806be60 0x7f9ba806e370 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc0d15d0 tx=0x7f9bac009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:01.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.573+0000 7f9bc8dcd740 1 Processor -- start 2024-05-20T18:34:01.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.573+0000 7f9bc8dcd740 1 -- start start 2024-05-20T18:34:01.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.574+0000 7f9bc8dcd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d1493f410 0x557d14d0a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.574+0000 7f9bc8dcd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d14916550 0x557d14d25160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.574+0000 7f9bc8dcd740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557d1491e730 0x557d14d256a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.574+0000 7f9bc8dcd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557d14c43990 con 0x557d14916550 2024-05-20T18:34:01.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.574+0000 7f9bc8dcd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557d14beb5b0 con 0x557d1493f410 2024-05-20T18:34:01.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.574+0000 7f9bc8dcd740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x557d14c3f950 con 0x557d1491e730 2024-05-20T18:34:01.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.574+0000 7f9bbaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d14916550 0x557d14d25160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.574+0000 7f9bbaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d14916550 0x557d14d25160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56702/0 (socket says 172.21.15.5:56702) 2024-05-20T18:34:01.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.574+0000 7f9bbaffd640 1 -- 172.21.15.5:0/2234166524 learned_addr learned my addr 172.21.15.5:0/2234166524 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:01.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.574+0000 7f9bbb7fe640 1 --2- 172.21.15.5:0/2234166524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d1493f410 0x557d14d0a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.574+0000 7f9bbbfff640 1 --2- 172.21.15.5:0/2234166524 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557d1491e730 0x557d14d256a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.574+0000 7f9bbaffd640 1 -- 172.21.15.5:0/2234166524 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557d1491e730 msgr2=0x557d14d256a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:01.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.574+0000 7f9bbaffd640 1 --2- 172.21.15.5:0/2234166524 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557d1491e730 0x557d14d256a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.574+0000 7f9bbaffd640 1 -- 172.21.15.5:0/2234166524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d1493f410 msgr2=0x557d14d0a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:01.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.574+0000 7f9bbaffd640 1 --2- 172.21.15.5:0/2234166524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d1493f410 0x557d14d0a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.574+0000 7f9bbaffd640 1 -- 172.21.15.5:0/2234166524 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d14a1f7e0 con 0x557d14916550 2024-05-20T18:34:01.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.575+0000 7f9bbaffd640 1 --2- 172.21.15.5:0/2234166524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d14916550 0x557d14d25160 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f9bb00a9260 tx=0x7f9bb0078cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960c46eaa7d92663 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:01.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.575+0000 7f9ba17fa640 1 -- 172.21.15.5:0/2234166524 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb004a040 con 0x557d14916550 2024-05-20T18:34:01.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.575+0000 7f9ba17fa640 1 -- 172.21.15.5:0/2234166524 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bb0081c80 con 0x557d14916550 2024-05-20T18:34:01.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.575+0000 7f9ba17fa640 1 -- 172.21.15.5:0/2234166524 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0075730 con 0x557d14916550 2024-05-20T18:34:01.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.576+0000 7f9bc8dcd740 1 -- 172.21.15.5:0/2234166524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d14916550 msgr2=0x557d14d25160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.576+0000 7f9bc8dcd740 1 --2- 172.21.15.5:0/2234166524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d14916550 0x557d14d25160 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f9bb00a9260 tx=0x7f9bb0078cc0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.576+0000 7f9bc8dcd740 1 -- 172.21.15.5:0/2234166524 shutdown_connections 2024-05-20T18:34:01.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.576+0000 7f9bc8dcd740 1 --2- 172.21.15.5:0/2234166524 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557d1491e730 0x557d14d256a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.576+0000 7f9bc8dcd740 1 --2- 172.21.15.5:0/2234166524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d14916550 0x557d14d25160 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.576+0000 7f9bc8dcd740 1 --2- 172.21.15.5:0/2234166524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d1493f410 0x557d14d0a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.576+0000 7f9bc8dcd740 1 -- 172.21.15.5:0/2234166524 >> 172.21.15.5:0/2234166524 conn(0x557d14933560 msgr2=0x557d14cdfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:01.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.576+0000 7f9bc8dcd740 1 -- 172.21.15.5:0/2234166524 shutdown_connections 2024-05-20T18:34:01.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.576+0000 7f9bc8dcd740 1 -- 172.21.15.5:0/2234166524 wait complete. 2024-05-20T18:34:01.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.577+0000 7f9bc8dcd740 1 -- start start 2024-05-20T18:34:01.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.577+0000 7f9bc8dcd740 1 Processor -- start 2024-05-20T18:34:01.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.578+0000 7f9bc8dcd740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557d14916550 0x557d14d275b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.578+0000 7f9bc8dcd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d1491e730 0x557d14d343b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.578+0000 7f9bc8dcd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d1493f410 0x557d14d348f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.578+0000 7f9bc8dcd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557d14beb5b0 con 0x557d1491e730 2024-05-20T18:34:01.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.578+0000 7f9bc8dcd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557d14bfeb40 con 0x557d1493f410 2024-05-20T18:34:01.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.578+0000 7f9bc8dcd740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x557d14c43990 con 0x557d14916550 2024-05-20T18:34:01.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.578+0000 7f9bc8dcd740 10 client.0 _ll_register_callbacks cb 0x557d14bf30f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:34:01.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.578+0000 7f9bbaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d1491e730 0x557d14d343b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.578+0000 7f9bbaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d1491e730 0x557d14d343b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56714/0 (socket says 172.21.15.5:56714) 2024-05-20T18:34:01.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.578+0000 7f9bbaffd640 1 -- 172.21.15.5:0/1488564353 learned_addr learned my addr 172.21.15.5:0/1488564353 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:01.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.578+0000 7f9bbaffd640 1 -- 172.21.15.5:0/1488564353 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557d14916550 msgr2=0x557d14d275b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:01.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.578+0000 7f9bbaffd640 1 --2- 172.21.15.5:0/1488564353 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557d14916550 0x557d14d275b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.578+0000 7f9bbbfff640 1 --2- 172.21.15.5:0/1488564353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d1493f410 0x557d14d348f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.578+0000 7f9bbaffd640 1 -- 172.21.15.5:0/1488564353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d1493f410 msgr2=0x557d14d348f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:01.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.579+0000 7f9bbaffd640 1 --2- 172.21.15.5:0/1488564353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d1493f410 0x557d14d348f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.579+0000 7f9bbaffd640 1 -- 172.21.15.5:0/1488564353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d14a7ff70 con 0x557d1491e730 2024-05-20T18:34:01.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.579+0000 7f9bbaffd640 1 --2- 172.21.15.5:0/1488564353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d1491e730 0x557d14d343b0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0009160 tx=0x7f9bb0008070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e20073239256bc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:01.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.579+0000 7f9b9a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:34:01.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.579+0000 7f9b9a7fc640 1 -- 172.21.15.5:0/1488564353 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0098060 con 0x557d1491e730 2024-05-20T18:34:01.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.579+0000 7f9b9a7fc640 1 -- 172.21.15.5:0/1488564353 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7f9bb00819f0 con 0x557d1491e730 2024-05-20T18:34:01.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.579+0000 7f9bc8dcd740 1 -- 172.21.15.5:0/1488564353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bb0081c80 con 0x557d1491e730 2024-05-20T18:34:01.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.579+0000 7f9b9a7fc640 1 -- 172.21.15.5:0/1488564353 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f9bb009e050 con 0x557d1491e730 2024-05-20T18:34:01.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.579+0000 7f9bc8dcd740 10 client.7833 fetch_fsmap learned FSMap version 67 2024-05-20T18:34:01.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.579+0000 7f9bc8dcd740 1 -- 172.21.15.5:0/1488564353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x557d14917170 con 0x557d1491e730 2024-05-20T18:34:01.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.579+0000 7f9b9a7fc640 1 -- 172.21.15.5:0/1488564353 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bb0081c80 con 0x557d1491e730 2024-05-20T18:34:01.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.579+0000 7f9b9a7fc640 1 -- 172.21.15.5:0/1488564353 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f9bb00a1020 con 0x557d1491e730 2024-05-20T18:34:01.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.579+0000 7f9bc8dcd740 10 client.7833 fetch_fsmap finished waiting for FSMap version 67 2024-05-20T18:34:01.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.579+0000 7f9bc8dcd740 10 client.7833 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-20T18:34:01.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.579+0000 7f9bc8dcd740 20 client.7833 populate_metadata read hostname 'smithi005' 2024-05-20T18:34:01.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.579+0000 7f9bc8dcd740 1 --2- 172.21.15.5:0/1488564353 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x557d148fe1f0 0x557d14d0daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.579+0000 7f9bc8dcd740 4 client.7833 mds_command: new command op to 7437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:34:01.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.579+0000 7f9bc8dcd740 1 -- 172.21.15.5:0/1488564353 --> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d14c510c0 con 0x557d148fe1f0 2024-05-20T18:34:01.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.580+0000 7f9bbb7fe640 1 --2- 172.21.15.5:0/1488564353 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x557d148fe1f0 0x557d14d0daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.580+0000 7f9b9a7fc640 10 client.7833 ms_handle_connect on v2:172.21.15.5:6836/3556492262 2024-05-20T18:34:01.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.580+0000 7f9bbb7fe640 1 --2- 172.21.15.5:0/1488564353 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x557d148fe1f0 0x557d14d0daf0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d2f3bc35b3d1312 server_cookie=38e3ffb11dc07b77 in_seq=0 out_seq=0 2024-05-20T18:34:01.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.581+0000 7f9b9a7fc640 1 -- 172.21.15.5:0/1488564353 <== mds.0 v2:172.21.15.5:6836/3556492262 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557d14c510c0 con 0x557d148fe1f0 2024-05-20T18:34:01.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.581+0000 7f9b9a7fc640 10 client.7833 handle_command_reply: tid=0 2024-05-20T18:34:01.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.581+0000 7f9bc8dcd740 1 client.7833 shutdown 2024-05-20T18:34:01.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.581+0000 7f9bc8dcd740 1 -- 172.21.15.5:0/1488564353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d1491e730 msgr2=0x557d14d343b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.581+0000 7f9bc8dcd740 1 --2- 172.21.15.5:0/1488564353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d1491e730 0x557d14d343b0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0009160 tx=0x7f9bb0008070 comp rx=0 tx=0).stop 2024-05-20T18:34:01.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.581+0000 7f9bc8dcd740 1 -- 172.21.15.5:0/1488564353 shutdown_connections 2024-05-20T18:34:01.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.581+0000 7f9bc8dcd740 1 --2- 172.21.15.5:0/1488564353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d1493f410 0x557d14d348f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.581+0000 7f9bc8dcd740 1 --2- 172.21.15.5:0/1488564353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d1491e730 0x557d14d343b0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.581+0000 7f9bc8dcd740 1 --2- 172.21.15.5:0/1488564353 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x557d148fe1f0 0x557d14d0daf0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.581+0000 7f9bc8dcd740 1 --2- 172.21.15.5:0/1488564353 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557d14916550 0x557d14d275b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.581+0000 7f9b9a7fc640 0 client.7833 ms_handle_reset on v2:172.21.15.5:6836/3556492262 2024-05-20T18:34:01.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.581+0000 7f9bc8dcd740 1 -- 172.21.15.5:0/1488564353 >> 172.21.15.5:0/1488564353 conn(0x557d14933560 msgr2=0x557d14cdfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:01.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.581+0000 7f9bc8dcd740 1 -- 172.21.15.5:0/1488564353 shutdown_connections 2024-05-20T18:34:01.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.581+0000 7f9bc8dcd740 1 -- 172.21.15.5:0/1488564353 wait complete. 2024-05-20T18:34:01.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.581+0000 7f9bc8dcd740 20 client.7833 trim_cache size 0 max 16384 2024-05-20T18:34:01.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.600+0000 7f9bc8dcd740 1 Processor -- start 2024-05-20T18:34:01.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.600+0000 7f9bc8dcd740 1 -- start start 2024-05-20T18:34:01.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.601+0000 7f9bc8dcd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d14916550 0x557d14d02110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.601+0000 7f9bc8dcd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d1491e730 0x557d14cdfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.601+0000 7f9bc8dcd740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557d1493f410 0x557d14ce01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.601+0000 7f9bc8dcd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc2027a0 con 0x557d1491e730 2024-05-20T18:34:01.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.601+0000 7f9bc8dcd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc201780 con 0x557d14916550 2024-05-20T18:34:01.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.601+0000 7f9bc8dcd740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9bbc14c340 con 0x557d1493f410 2024-05-20T18:34:01.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.601+0000 7f9bbaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d1491e730 0x557d14cdfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.601+0000 7f9bbaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d1491e730 0x557d14cdfbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56720/0 (socket says 172.21.15.5:56720) 2024-05-20T18:34:01.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.601+0000 7f9bbaffd640 1 -- 172.21.15.5:0/1671986886 learned_addr learned my addr 172.21.15.5:0/1671986886 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:01.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.602+0000 7f9bbbfff640 1 --2- 172.21.15.5:0/1671986886 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557d1493f410 0x557d14ce01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.602+0000 7f9bbb7fe640 1 --2- 172.21.15.5:0/1671986886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d14916550 0x557d14d02110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.602+0000 7f9bbaffd640 1 -- 172.21.15.5:0/1671986886 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557d1493f410 msgr2=0x557d14ce01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:01.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.602+0000 7f9bbaffd640 1 --2- 172.21.15.5:0/1671986886 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557d1493f410 0x557d14ce01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.602+0000 7f9bbaffd640 1 -- 172.21.15.5:0/1671986886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d14916550 msgr2=0x557d14d02110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:01.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.602+0000 7f9bbaffd640 1 --2- 172.21.15.5:0/1671986886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d14916550 0x557d14d02110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.602+0000 7f9bbaffd640 1 -- 172.21.15.5:0/1671986886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d14d47010 con 0x557d1491e730 2024-05-20T18:34:01.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.602+0000 7f9bbaffd640 1 --2- 172.21.15.5:0/1671986886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d1491e730 0x557d14cdfbb0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0075520 tx=0x7f9bb00a4000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e352a45b0001ce9e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:01.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.602+0000 7f9ba0ff9640 1 -- 172.21.15.5:0/1671986886 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb004a040 con 0x557d1491e730 2024-05-20T18:34:01.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.602+0000 7f9ba0ff9640 1 -- 172.21.15.5:0/1671986886 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bb0081c80 con 0x557d1491e730 2024-05-20T18:34:01.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.602+0000 7f9ba0ff9640 1 -- 172.21.15.5:0/1671986886 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb00817f0 con 0x557d1491e730 2024-05-20T18:34:01.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.603+0000 7f9bc8dcd740 1 -- 172.21.15.5:0/1671986886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d1491e730 msgr2=0x557d14cdfbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.603+0000 7f9bc8dcd740 1 --2- 172.21.15.5:0/1671986886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d1491e730 0x557d14cdfbb0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0075520 tx=0x7f9bb00a4000 comp rx=0 tx=0).stop 2024-05-20T18:34:01.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.603+0000 7f9bc8dcd740 1 -- 172.21.15.5:0/1671986886 shutdown_connections 2024-05-20T18:34:01.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.603+0000 7f9bc8dcd740 1 --2- 172.21.15.5:0/1671986886 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557d1493f410 0x557d14ce01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.603+0000 7f9bc8dcd740 1 --2- 172.21.15.5:0/1671986886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d1491e730 0x557d14cdfbb0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.603+0000 7f9bc8dcd740 1 --2- 172.21.15.5:0/1671986886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d14916550 0x557d14d02110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.603+0000 7f9bc8dcd740 1 -- 172.21.15.5:0/1671986886 >> 172.21.15.5:0/1671986886 conn(0x557d14933560 msgr2=0x557d14d0daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:01.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.603+0000 7f9bc8dcd740 1 -- 172.21.15.5:0/1671986886 shutdown_connections 2024-05-20T18:34:01.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.603+0000 7f9bc8dcd740 1 -- 172.21.15.5:0/1671986886 wait complete. 2024-05-20T18:34:01.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.604+0000 7f9bc8dcd740 1 -- start start 2024-05-20T18:34:01.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.604+0000 7f9bc8dcd740 1 Processor -- start 2024-05-20T18:34:01.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.605+0000 7f9bc8dcd740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557d14916550 0x557d14d1c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.605+0000 7f9bc8dcd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d1491e730 0x557d14d3d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.605+0000 7f9bc8dcd740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d1493f410 0x557d14ce01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.605+0000 7f9bc8dcd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557d14ce1120 con 0x557d1491e730 2024-05-20T18:34:01.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.605+0000 7f9bc8dcd740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc2010d0 con 0x557d1493f410 2024-05-20T18:34:01.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.605+0000 7f9bc8dcd740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x557d14cdf7d0 con 0x557d14916550 2024-05-20T18:34:01.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.605+0000 7f9bc8dcd740 10 client.0 _ll_register_callbacks cb 0x7f9bb0078e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:34:01.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.605+0000 7f9bbaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d1491e730 0x557d14d3d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.605+0000 7f9bbaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d1491e730 0x557d14d3d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56726/0 (socket says 172.21.15.5:56726) 2024-05-20T18:34:01.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.605+0000 7f9bbaffd640 1 -- 172.21.15.5:0/934336480 learned_addr learned my addr 172.21.15.5:0/934336480 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:01.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.605+0000 7f9bbb7fe640 1 --2- 172.21.15.5:0/934336480 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557d14916550 0x557d14d1c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.605+0000 7f9bbbfff640 1 --2- 172.21.15.5:0/934336480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d1493f410 0x557d14ce01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.606+0000 7f9bbaffd640 1 -- 172.21.15.5:0/934336480 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557d14916550 msgr2=0x557d14d1c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:01.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.606+0000 7f9bbaffd640 1 --2- 172.21.15.5:0/934336480 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557d14916550 0x557d14d1c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.606+0000 7f9bbaffd640 1 -- 172.21.15.5:0/934336480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d1493f410 msgr2=0x557d14ce01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:01.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.606+0000 7f9bbaffd640 1 --2- 172.21.15.5:0/934336480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d1493f410 0x557d14ce01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.606+0000 7f9bbaffd640 1 -- 172.21.15.5:0/934336480 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d14d1ccd0 con 0x557d1491e730 2024-05-20T18:34:01.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.606+0000 7f9bbbfff640 1 --2- 172.21.15.5:0/934336480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d1493f410 0x557d14ce01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:34:01.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.606+0000 7f9bbaffd640 1 --2- 172.21.15.5:0/934336480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d1491e730 0x557d14d3d8e0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0079280 tx=0x7f9bb00094d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5293d8c286a98b06 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:01.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.606+0000 7f9b9a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:34:01.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.606+0000 7f9b9a7fc640 1 -- 172.21.15.5:0/934336480 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb004a070 con 0x557d1491e730 2024-05-20T18:34:01.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.606+0000 7f9b9a7fc640 1 -- 172.21.15.5:0/934336480 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7f9bb0075020 con 0x557d1491e730 2024-05-20T18:34:01.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.606+0000 7f9bc8dcd740 1 -- 172.21.15.5:0/934336480 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bb0081c80 con 0x557d1491e730 2024-05-20T18:34:01.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.606+0000 7f9b9a7fc640 1 -- 172.21.15.5:0/934336480 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f9bb008a050 con 0x557d1491e730 2024-05-20T18:34:01.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.606+0000 7f9bc8dcd740 10 client.7839 fetch_fsmap learned FSMap version 67 2024-05-20T18:34:01.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.606+0000 7f9bc8dcd740 1 -- 172.21.15.5:0/934336480 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x557d14d1ccd0 con 0x557d1491e730 2024-05-20T18:34:01.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.606+0000 7f9b9a7fc640 1 -- 172.21.15.5:0/934336480 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bb0081c80 con 0x557d1491e730 2024-05-20T18:34:01.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.606+0000 7f9b9a7fc640 1 -- 172.21.15.5:0/934336480 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f9bb008d020 con 0x557d1491e730 2024-05-20T18:34:01.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.606+0000 7f9bc8dcd740 10 client.7839 fetch_fsmap finished waiting for FSMap version 67 2024-05-20T18:34:01.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.606+0000 7f9bc8dcd740 10 client.7839 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-20T18:34:01.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.606+0000 7f9bc8dcd740 20 client.7839 populate_metadata read hostname 'smithi005' 2024-05-20T18:34:01.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.606+0000 7f9bc8dcd740 1 --2- 172.21.15.5:0/934336480 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x557d14d25740 0x557d14ce0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.606+0000 7f9bc8dcd740 4 client.7839 mds_command: new command op to 7437 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-05-20T18:34:01.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.606+0000 7f9bc8dcd740 1 -- 172.21.15.5:0/934336480 --> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x557d14c4b410 con 0x557d14d25740 2024-05-20T18:34:01.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.607+0000 7f9bbb7fe640 1 --2- 172.21.15.5:0/934336480 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x557d14d25740 0x557d14ce0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.607+0000 7f9b9a7fc640 10 client.7839 ms_handle_connect on v2:172.21.15.5:6836/3556492262 2024-05-20T18:34:01.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.607+0000 7f9bbb7fe640 1 --2- 172.21.15.5:0/934336480 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x557d14d25740 0x557d14ce0710 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77cca25e583f0df2 server_cookie=8ca5ceda80c9f679 in_seq=0 out_seq=0 2024-05-20T18:34:01.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.608+0000 7f9b9a7fc640 1 -- 172.21.15.5:0/934336480 <== mds.0 v2:172.21.15.5:6836/3556492262 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557d14c4b410 con 0x557d14d25740 2024-05-20T18:34:01.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.608+0000 7f9b9a7fc640 10 client.7839 handle_command_reply: tid=0 2024-05-20T18:34:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.608+0000 7f9bc8dcd740 1 client.7839 shutdown 2024-05-20T18:34:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.608+0000 7f9bc8dcd740 1 -- 172.21.15.5:0/934336480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d1491e730 msgr2=0x557d14d3d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.608+0000 7f9bc8dcd740 1 --2- 172.21.15.5:0/934336480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d1491e730 0x557d14d3d8e0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0079280 tx=0x7f9bb00094d0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.608+0000 7f9bc8dcd740 1 -- 172.21.15.5:0/934336480 shutdown_connections 2024-05-20T18:34:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.608+0000 7f9bc8dcd740 1 --2- 172.21.15.5:0/934336480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557d1493f410 0x557d14ce01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.608+0000 7f9bc8dcd740 1 --2- 172.21.15.5:0/934336480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d1491e730 0x557d14d3d8e0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.608+0000 7f9bc8dcd740 1 --2- 172.21.15.5:0/934336480 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x557d14d25740 0x557d14ce0710 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.608+0000 7f9bc8dcd740 1 --2- 172.21.15.5:0/934336480 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557d14916550 0x557d14d1c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.608+0000 7f9bc8dcd740 1 -- 172.21.15.5:0/934336480 >> 172.21.15.5:0/934336480 conn(0x557d14933560 msgr2=0x557d14d0aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.608+0000 7f9b9a7fc640 0 client.7839 ms_handle_reset on v2:172.21.15.5:6836/3556492262 2024-05-20T18:34:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.608+0000 7f9bc8dcd740 1 -- 172.21.15.5:0/934336480 shutdown_connections 2024-05-20T18:34:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.608+0000 7f9bc8dcd740 1 -- 172.21.15.5:0/934336480 wait complete. 2024-05-20T18:34:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.608+0000 7f9bc8dcd740 20 client.7839 trim_cache size 0 max 16384 2024-05-20T18:34:01.614 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:34:01.614 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-20T18:34:01.614 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "e5e74c9f-4d8c-4d3c-b9de-5746d84fffcb", 2024-05-20T18:34:01.614 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-20T18:34:01.614 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:34:01.614 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-20T18:34:01.608+0000 7fce21072640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv20 ap=3 DIRTYPARENT f(v0 m2024-05-20T18:33:20.165942+0000 1=1+0)->f(v1 m2024-05-20T18:33:20.165942+0000) n(v0 rc2024-05-20T18:33:20.285940+0000 2=1+1)->n(v0 rc2024-05-20T18:33:20.285940+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x5555f6279b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-05-20T18:33:20.165942+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-05-20T18:33:20.285940+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-05-20T18:34:01.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.609+0000 7f9bc1f2c640 1 -- 172.21.15.5:0/1278212362 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9ba806be60 msgr2=0x7f9ba806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.609+0000 7f9bc1f2c640 1 --2- 172.21.15.5:0/1278212362 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9ba806be60 0x7f9ba806e370 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc0d15d0 tx=0x7f9bac009190 comp rx=0 tx=0).stop 2024-05-20T18:34:01.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.609+0000 7f9bc1f2c640 1 -- 172.21.15.5:0/1278212362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bbc08bd20 msgr2=0x7f9bbc0c92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.609+0000 7f9bc1f2c640 1 --2- 172.21.15.5:0/1278212362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bbc08bd20 0x7f9bbc0c92c0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f9bb002a0f0 tx=0x7f9bb00046c0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.609+0000 7f9bc1f2c640 1 -- 172.21.15.5:0/1278212362 shutdown_connections 2024-05-20T18:34:01.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.609+0000 7f9bc1f2c640 1 --2- 172.21.15.5:0/1278212362 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9ba806be60 0x7f9ba806e370 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.609+0000 7f9bc1f2c640 1 --2- 172.21.15.5:0/1278212362 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9bbc0b3860 0x7f9bbc0c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.609+0000 7f9bc1f2c640 1 --2- 172.21.15.5:0/1278212362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bbc08bd20 0x7f9bbc0c92c0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.609+0000 7f9bc1f2c640 1 --2- 172.21.15.5:0/1278212362 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9bbc06d870 0x7f9bbc1f4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.609+0000 7f9bc1f2c640 1 -- 172.21.15.5:0/1278212362 >> 172.21.15.5:0/1278212362 conn(0x7f9bbc0b6d10 msgr2=0x7f9bbc0b9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:01.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.609+0000 7f9bc1f2c640 1 -- 172.21.15.5:0/1278212362 shutdown_connections 2024-05-20T18:34:01.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.609+0000 7f9bc1f2c640 1 -- 172.21.15.5:0/1278212362 wait complete. 2024-05-20T18:34:01.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:34:01.622 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-20T18:34:01.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.738+0000 7f37caede640 1 Processor -- start 2024-05-20T18:34:01.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.738+0000 7f37caede640 1 -- start start 2024-05-20T18:34:01.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.740+0000 7f37caede640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f37c415d460 0x7f37c406b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.740+0000 7f37caede640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f37c40b17f0 0x7f37c40b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.740+0000 7f37caede640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37c415d870 con 0x7f37c40d1780 2024-05-20T18:34:01.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.740+0000 7f37caede640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f37c415dae0 con 0x7f37c40b17f0 2024-05-20T18:34:01.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.740+0000 7f37caede640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f37c415ddc0 con 0x7f37c415d460 2024-05-20T18:34:01.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.740+0000 7f37c947b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f37c40b17f0 0x7f37c40b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.740+0000 7f37c947b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f37c40b17f0 0x7f37c40b1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56728/0 (socket says 172.21.15.5:56728) 2024-05-20T18:34:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.740+0000 7f37c947b640 1 -- 172.21.15.5:0/59319722 learned_addr learned my addr 172.21.15.5:0/59319722 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.740+0000 7f37c8c7a640 1 --2- 172.21.15.5:0/59319722 >> v2:172.21.15.63:3300/0 conn(0x7f37c415d460 0x7f37c406b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.740+0000 7f37c947b640 1 -- 172.21.15.5:0/59319722 >> v2:172.21.15.63:3300/0 conn(0x7f37c415d460 msgr2=0x7f37c406b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.740+0000 7f37c947b640 1 --2- 172.21.15.5:0/59319722 >> v2:172.21.15.63:3300/0 conn(0x7f37c415d460 0x7f37c406b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.740+0000 7f37c947b640 1 -- 172.21.15.5:0/59319722 >> v1:172.21.15.63:6789/0 conn(0x7f37c40d1780 legacy=0x7f37c40d1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.740+0000 7f37c947b640 1 -- 172.21.15.5:0/59319722 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c415e0a0 con 0x7f37c40b17f0 2024-05-20T18:34:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.741+0000 7f37c947b640 1 --2- 172.21.15.5:0/59319722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37c40b17f0 0x7f37c40b1c00 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f37c40b6ca0 tx=0x7f37bc02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abba41b9f37cf1c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.741+0000 7f37c37fe640 1 -- 172.21.15.5:0/59319722 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc036070 con 0x7f37c40b17f0 2024-05-20T18:34:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.741+0000 7f37c37fe640 1 -- 172.21.15.5:0/59319722 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37bc031600 con 0x7f37c40b17f0 2024-05-20T18:34:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.741+0000 7f37c37fe640 1 -- 172.21.15.5:0/59319722 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc0319d0 con 0x7f37c40b17f0 2024-05-20T18:34:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.742+0000 7f37caede640 1 -- 172.21.15.5:0/59319722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37c40b17f0 msgr2=0x7f37c40b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.742+0000 7f37caede640 1 --2- 172.21.15.5:0/59319722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37c40b17f0 0x7f37c40b1c00 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f37c40b6ca0 tx=0x7f37bc02a730 comp rx=0 tx=0).stop 2024-05-20T18:34:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.742+0000 7f37caede640 1 -- 172.21.15.5:0/59319722 shutdown_connections 2024-05-20T18:34:01.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.742+0000 7f37caede640 1 --2- 172.21.15.5:0/59319722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37c40b17f0 0x7f37c40b1c00 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.742+0000 7f37caede640 1 --2- 172.21.15.5:0/59319722 >> v2:172.21.15.63:3300/0 conn(0x7f37c415d460 0x7f37c406b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.742+0000 7f37caede640 1 -- 172.21.15.5:0/59319722 >> 172.21.15.5:0/59319722 conn(0x7f37c40b6620 msgr2=0x7f37c40b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:01.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.742+0000 7f37caede640 1 -- 172.21.15.5:0/59319722 shutdown_connections 2024-05-20T18:34:01.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.742+0000 7f37caede640 1 -- 172.21.15.5:0/59319722 wait complete. 2024-05-20T18:34:01.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.744+0000 7f37caede640 1 Processor -- start 2024-05-20T18:34:01.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.744+0000 7f37caede640 1 -- start start 2024-05-20T18:34:01.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.745+0000 7f37caede640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37c40b17f0 0x7f37c409f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.745+0000 7f37caede640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37c40d1780 0x7f37c415c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.745+0000 7f37caede640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37c415d460 0x7f37c41f3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.745+0000 7f37caede640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37c415e380 con 0x7f37c40b17f0 2024-05-20T18:34:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.745+0000 7f37caede640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f37c406bdb0 con 0x7f37c415d460 2024-05-20T18:34:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.745+0000 7f37caede640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f37c40b2140 con 0x7f37c40d1780 2024-05-20T18:34:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.745+0000 7f37c8c7a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37c40b17f0 0x7f37c409f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.745+0000 7f37c8c7a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37c40b17f0 0x7f37c409f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56730/0 (socket says 172.21.15.5:56730) 2024-05-20T18:34:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.745+0000 7f37c8c7a640 1 -- 172.21.15.5:0/162898009 learned_addr learned my addr 172.21.15.5:0/162898009 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.746+0000 7f37c3fff640 1 --2- 172.21.15.5:0/162898009 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37c40d1780 0x7f37c415c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.746+0000 7f37c947b640 1 --2- 172.21.15.5:0/162898009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37c415d460 0x7f37c41f3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.746+0000 7f37c8c7a640 1 -- 172.21.15.5:0/162898009 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37c40d1780 msgr2=0x7f37c415c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.746+0000 7f37c8c7a640 1 --2- 172.21.15.5:0/162898009 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37c40d1780 0x7f37c415c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.746+0000 7f37c8c7a640 1 -- 172.21.15.5:0/162898009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37c415d460 msgr2=0x7f37c41f3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.746+0000 7f37c8c7a640 1 --2- 172.21.15.5:0/162898009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37c415d460 0x7f37c41f3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.746+0000 7f37c8c7a640 1 -- 172.21.15.5:0/162898009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c409e360 con 0x7f37c40b17f0 2024-05-20T18:34:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.746+0000 7f37c3fff640 1 --2- 172.21.15.5:0/162898009 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37c40d1780 0x7f37c415c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:34:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.746+0000 7f37c947b640 1 --2- 172.21.15.5:0/162898009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37c415d460 0x7f37c41f3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:34:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.746+0000 7f37c8c7a640 1 --2- 172.21.15.5:0/162898009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37c40b17f0 0x7f37c409f400 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f37b800e870 tx=0x7f37b800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.746+0000 7f37c17fa640 1 -- 172.21.15.5:0/162898009 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8017020 con 0x7f37c40b17f0 2024-05-20T18:34:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.746+0000 7f37c17fa640 1 -- 172.21.15.5:0/162898009 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37b800ee60 con 0x7f37c40b17f0 2024-05-20T18:34:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.746+0000 7f37c17fa640 1 -- 172.21.15.5:0/162898009 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8004460 con 0x7f37c40b17f0 2024-05-20T18:34:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.746+0000 7f37caede640 1 -- 172.21.15.5:0/162898009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c409e640 con 0x7f37c40b17f0 2024-05-20T18:34:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.746+0000 7f37caede640 1 -- 172.21.15.5:0/162898009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c409ec00 con 0x7f37c40b17f0 2024-05-20T18:34:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.748+0000 7f37c17fa640 1 -- 172.21.15.5:0/162898009 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f37b801a020 con 0x7f37c40b17f0 2024-05-20T18:34:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.749+0000 7f37c17fa640 1 --2- 172.21.15.5:0/162898009 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f37a806be60 0x7f37a806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.749+0000 7f37c17fa640 1 -- 172.21.15.5:0/162898009 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7f37b808b270 con 0x7f37c40b17f0 2024-05-20T18:34:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.749+0000 7f37c3fff640 1 --2- 172.21.15.5:0/162898009 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f37a806be60 0x7f37a806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.749+0000 7f37c3fff640 1 --2- 172.21.15.5:0/162898009 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f37a806be60 0x7f37a806e370 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f37c41e7250 tx=0x7f37b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:01.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.750+0000 7f37d1d80740 1 Processor -- start 2024-05-20T18:34:01.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.750+0000 7f37d1d80740 1 -- start start 2024-05-20T18:34:01.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.751+0000 7f37d1d80740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cff1ca390 0x560cff594b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.751+0000 7f37d1d80740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cff1a14d0 0x560cff5af890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.751+0000 7f37d1d80740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cff1a96b0 0x560cff5afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.751+0000 7f37d1d80740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560cff4ced00 con 0x560cff1a96b0 2024-05-20T18:34:01.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.751+0000 7f37d1d80740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560cff4764c0 con 0x560cff1ca390 2024-05-20T18:34:01.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.751+0000 7f37d1d80740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x560cff4cacc0 con 0x560cff1a14d0 2024-05-20T18:34:01.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.751+0000 7f37c947b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cff1a96b0 0x560cff5afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.751+0000 7f37c947b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cff1a96b0 0x560cff5afdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56736/0 (socket says 172.21.15.5:56736) 2024-05-20T18:34:01.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.751+0000 7f37c947b640 1 -- 172.21.15.5:0/1439223526 learned_addr learned my addr 172.21.15.5:0/1439223526 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:01.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.751+0000 7f37c8c7a640 1 --2- 172.21.15.5:0/1439223526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cff1ca390 0x560cff594b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.751+0000 7f37c3fff640 1 --2- 172.21.15.5:0/1439223526 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cff1a14d0 0x560cff5af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.751+0000 7f37c947b640 1 -- 172.21.15.5:0/1439223526 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cff1a14d0 msgr2=0x560cff5af890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:01.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.751+0000 7f37c947b640 1 --2- 172.21.15.5:0/1439223526 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cff1a14d0 0x560cff5af890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.751+0000 7f37c947b640 1 -- 172.21.15.5:0/1439223526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cff1ca390 msgr2=0x560cff594b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:01.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.751+0000 7f37c947b640 1 --2- 172.21.15.5:0/1439223526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cff1ca390 0x560cff594b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.751+0000 7f37c947b640 1 -- 172.21.15.5:0/1439223526 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cff2ac7d0 con 0x560cff1a96b0 2024-05-20T18:34:01.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.751+0000 7f37c947b640 1 --2- 172.21.15.5:0/1439223526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cff1a96b0 0x560cff5afdd0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009f40 tx=0x7f37bc02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b471693831ac13d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:01.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.752+0000 7f37ae7fc640 1 -- 172.21.15.5:0/1439223526 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc036040 con 0x560cff1a96b0 2024-05-20T18:34:01.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.752+0000 7f37ae7fc640 1 -- 172.21.15.5:0/1439223526 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37bc041070 con 0x560cff1a96b0 2024-05-20T18:34:01.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.752+0000 7f37ae7fc640 1 -- 172.21.15.5:0/1439223526 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc03c870 con 0x560cff1a96b0 2024-05-20T18:34:01.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.752+0000 7f37d1d80740 1 -- 172.21.15.5:0/1439223526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cff1a96b0 msgr2=0x560cff5afdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.752+0000 7f37d1d80740 1 --2- 172.21.15.5:0/1439223526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cff1a96b0 0x560cff5afdd0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009f40 tx=0x7f37bc02e000 comp rx=0 tx=0).stop 2024-05-20T18:34:01.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.752+0000 7f37d1d80740 1 -- 172.21.15.5:0/1439223526 shutdown_connections 2024-05-20T18:34:01.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.752+0000 7f37d1d80740 1 --2- 172.21.15.5:0/1439223526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cff1a96b0 0x560cff5afdd0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.752+0000 7f37d1d80740 1 --2- 172.21.15.5:0/1439223526 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cff1a14d0 0x560cff5af890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.752+0000 7f37d1d80740 1 --2- 172.21.15.5:0/1439223526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cff1ca390 0x560cff594b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.752+0000 7f37d1d80740 1 -- 172.21.15.5:0/1439223526 >> 172.21.15.5:0/1439223526 conn(0x560cff1be4e0 msgr2=0x560cff56a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:01.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.752+0000 7f37d1d80740 1 -- 172.21.15.5:0/1439223526 shutdown_connections 2024-05-20T18:34:01.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.752+0000 7f37d1d80740 1 -- 172.21.15.5:0/1439223526 wait complete. 2024-05-20T18:34:01.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.753+0000 7f37d1d80740 1 -- start start 2024-05-20T18:34:01.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.754+0000 7f37d1d80740 1 Processor -- start 2024-05-20T18:34:01.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.755+0000 7f37d1d80740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cff1a14d0 0x560cff598ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.755+0000 7f37d1d80740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cff1a96b0 0x560cff585160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.755+0000 7f37d1d80740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cff1ca390 0x560cff5ae110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.755+0000 7f37d1d80740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560cff489b90 con 0x560cff1a96b0 2024-05-20T18:34:01.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.755+0000 7f37d1d80740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560cff5aeb20 con 0x560cff1a14d0 2024-05-20T18:34:01.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.755+0000 7f37d1d80740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x560cff5a9eb0 con 0x560cff1ca390 2024-05-20T18:34:01.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.755+0000 7f37d1d80740 10 client.0 _ll_register_callbacks cb 0x560cff4c5da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:34:01.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.755+0000 7f37c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cff1a96b0 0x560cff585160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.755+0000 7f37c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cff1a96b0 0x560cff585160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56746/0 (socket says 172.21.15.5:56746) 2024-05-20T18:34:01.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.755+0000 7f37c3fff640 1 -- 172.21.15.5:0/3211405611 learned_addr learned my addr 172.21.15.5:0/3211405611 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:01.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.755+0000 7f37c8c7a640 1 --2- 172.21.15.5:0/3211405611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cff1a14d0 0x560cff598ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.755+0000 7f37c947b640 1 --2- 172.21.15.5:0/3211405611 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cff1ca390 0x560cff5ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.755+0000 7f37c3fff640 1 -- 172.21.15.5:0/3211405611 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cff1ca390 msgr2=0x560cff5ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:01.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.755+0000 7f37c3fff640 1 --2- 172.21.15.5:0/3211405611 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cff1ca390 0x560cff5ae110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.755+0000 7f37c3fff640 1 -- 172.21.15.5:0/3211405611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cff1a14d0 msgr2=0x560cff598ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:01.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.755+0000 7f37c3fff640 1 --2- 172.21.15.5:0/3211405611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cff1a14d0 0x560cff598ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.755+0000 7f37c3fff640 1 -- 172.21.15.5:0/3211405611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cff1e23f0 con 0x560cff1a96b0 2024-05-20T18:34:01.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.755+0000 7f37c947b640 1 --2- 172.21.15.5:0/3211405611 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cff1ca390 0x560cff5ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:34:01.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.755+0000 7f37c3fff640 1 --2- 172.21.15.5:0/3211405611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cff1a96b0 0x560cff585160 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f37b0003890 tx=0x7f37b0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8bba18969f048c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:01.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.756+0000 7f37a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:34:01.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.756+0000 7f37a37fe640 1 -- 172.21.15.5:0/3211405611 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b000ddb0 con 0x560cff1a96b0 2024-05-20T18:34:01.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.756+0000 7f37a37fe640 1 -- 172.21.15.5:0/3211405611 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7f37b0023d00 con 0x560cff1a96b0 2024-05-20T18:34:01.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.756+0000 7f37d1d80740 1 -- 172.21.15.5:0/3211405611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37bc041070 con 0x560cff1a96b0 2024-05-20T18:34:01.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.756+0000 7f37a37fe640 1 -- 172.21.15.5:0/3211405611 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f37b0003e10 con 0x560cff1a96b0 2024-05-20T18:34:01.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.756+0000 7f37d1d80740 10 client.7851 fetch_fsmap learned FSMap version 67 2024-05-20T18:34:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.756+0000 7f37d1d80740 1 -- 172.21.15.5:0/3211405611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x560cff1a20f0 con 0x560cff1a96b0 2024-05-20T18:34:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.756+0000 7f37a37fe640 1 -- 172.21.15.5:0/3211405611 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37bc041070 con 0x560cff1a96b0 2024-05-20T18:34:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.756+0000 7f37a37fe640 1 -- 172.21.15.5:0/3211405611 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f37b002a050 con 0x560cff1a96b0 2024-05-20T18:34:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.756+0000 7f37d1d80740 10 client.7851 fetch_fsmap finished waiting for FSMap version 67 2024-05-20T18:34:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.756+0000 7f37d1d80740 10 client.7851 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-20T18:34:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.756+0000 7f37d1d80740 20 client.7851 populate_metadata read hostname 'smithi005' 2024-05-20T18:34:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.756+0000 7f37d1d80740 1 --2- 172.21.15.5:0/3211405611 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x560cff188f60 0x560cff5ac110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.756+0000 7f37d1d80740 4 client.7851 mds_command: new command op to 7437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:34:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.756+0000 7f37d1d80740 1 -- 172.21.15.5:0/3211405611 --> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cff4dc430 con 0x560cff188f60 2024-05-20T18:34:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.756+0000 7f37c8c7a640 1 --2- 172.21.15.5:0/3211405611 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x560cff188f60 0x560cff5ac110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.757+0000 7f37a37fe640 10 client.7851 ms_handle_connect on v2:172.21.15.5:6836/3556492262 2024-05-20T18:34:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.757+0000 7f37c8c7a640 1 --2- 172.21.15.5:0/3211405611 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x560cff188f60 0x560cff5ac110 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b605367d48703db server_cookie=6a81b04d26b612c0 in_seq=0 out_seq=0 2024-05-20T18:34:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.757+0000 7f37a37fe640 1 -- 172.21.15.5:0/3211405611 <== mds.0 v2:172.21.15.5:6836/3556492262 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560cff4dc430 con 0x560cff188f60 2024-05-20T18:34:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.757+0000 7f37a37fe640 10 client.7851 handle_command_reply: tid=0 2024-05-20T18:34:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.757+0000 7f37d1d80740 1 client.7851 shutdown 2024-05-20T18:34:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.758+0000 7f37d1d80740 1 -- 172.21.15.5:0/3211405611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cff1a96b0 msgr2=0x560cff585160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.758+0000 7f37d1d80740 1 --2- 172.21.15.5:0/3211405611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cff1a96b0 0x560cff585160 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f37b0003890 tx=0x7f37b0000ed0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.758+0000 7f37d1d80740 1 -- 172.21.15.5:0/3211405611 shutdown_connections 2024-05-20T18:34:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.758+0000 7f37d1d80740 1 --2- 172.21.15.5:0/3211405611 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x560cff188f60 0x560cff5ac110 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.758+0000 7f37d1d80740 1 --2- 172.21.15.5:0/3211405611 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cff1ca390 0x560cff5ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.758+0000 7f37d1d80740 1 --2- 172.21.15.5:0/3211405611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cff1a96b0 0x560cff585160 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.758+0000 7f37d1d80740 1 --2- 172.21.15.5:0/3211405611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cff1a14d0 0x560cff598ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.758+0000 7f37a37fe640 0 client.7851 ms_handle_reset on v2:172.21.15.5:6836/3556492262 2024-05-20T18:34:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.758+0000 7f37d1d80740 1 -- 172.21.15.5:0/3211405611 >> 172.21.15.5:0/3211405611 conn(0x560cff1be4e0 msgr2=0x560cff594b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.758+0000 7f37d1d80740 1 -- 172.21.15.5:0/3211405611 shutdown_connections 2024-05-20T18:34:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.758+0000 7f37d1d80740 1 -- 172.21.15.5:0/3211405611 wait complete. 2024-05-20T18:34:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.758+0000 7f37d1d80740 20 client.7851 trim_cache size 0 max 16384 2024-05-20T18:34:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.776+0000 7f37d1d80740 1 Processor -- start 2024-05-20T18:34:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.776+0000 7f37d1d80740 1 -- start start 2024-05-20T18:34:01.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.777+0000 7f37d1d80740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cff1a14d0 0x560cff564780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.777+0000 7f37d1d80740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cff1a96b0 0x560cff594d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.777+0000 7f37d1d80740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cff1ca390 0x560cff568670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.777+0000 7f37d1d80740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37c42004b0 con 0x560cff1a14d0 2024-05-20T18:34:01.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.777+0000 7f37d1d80740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f37c41ff4d0 con 0x560cff1a96b0 2024-05-20T18:34:01.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.777+0000 7f37d1d80740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f37c41e62d0 con 0x560cff1ca390 2024-05-20T18:34:01.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.777+0000 7f37c8c7a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cff1a14d0 0x560cff564780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.777+0000 7f37c8c7a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cff1a14d0 0x560cff564780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56762/0 (socket says 172.21.15.5:56762) 2024-05-20T18:34:01.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.777+0000 7f37c8c7a640 1 -- 172.21.15.5:0/282988046 learned_addr learned my addr 172.21.15.5:0/282988046 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:01.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.778+0000 7f37c3fff640 1 --2- 172.21.15.5:0/282988046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cff1a96b0 0x560cff594d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.778+0000 7f37c947b640 1 --2- 172.21.15.5:0/282988046 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cff1ca390 0x560cff568670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.778+0000 7f37c8c7a640 1 -- 172.21.15.5:0/282988046 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cff1ca390 msgr2=0x560cff568670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:01.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.778+0000 7f37c8c7a640 1 --2- 172.21.15.5:0/282988046 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cff1ca390 0x560cff568670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.778+0000 7f37c8c7a640 1 -- 172.21.15.5:0/282988046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cff1a96b0 msgr2=0x560cff594d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:01.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.778+0000 7f37c8c7a640 1 --2- 172.21.15.5:0/282988046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cff1a96b0 0x560cff594d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.778+0000 7f37c8c7a640 1 -- 172.21.15.5:0/282988046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cff5569a0 con 0x560cff1a14d0 2024-05-20T18:34:01.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.778+0000 7f37c8c7a640 1 --2- 172.21.15.5:0/282988046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cff1a14d0 0x560cff564780 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f37b800c520 tx=0x7f37b8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b415dcfb3e69591d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:01.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.778+0000 7f37adffb640 1 -- 172.21.15.5:0/282988046 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8087070 con 0x560cff1a14d0 2024-05-20T18:34:01.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.778+0000 7f37adffb640 1 -- 172.21.15.5:0/282988046 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37b80711e0 con 0x560cff1a14d0 2024-05-20T18:34:01.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.778+0000 7f37adffb640 1 -- 172.21.15.5:0/282988046 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b80558a0 con 0x560cff1a14d0 2024-05-20T18:34:01.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.779+0000 7f37d1d80740 1 -- 172.21.15.5:0/282988046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cff1a14d0 msgr2=0x560cff564780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.779+0000 7f37d1d80740 1 --2- 172.21.15.5:0/282988046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cff1a14d0 0x560cff564780 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f37b800c520 tx=0x7f37b8008040 comp rx=0 tx=0).stop 2024-05-20T18:34:01.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.779+0000 7f37d1d80740 1 -- 172.21.15.5:0/282988046 shutdown_connections 2024-05-20T18:34:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.779+0000 7f37d1d80740 1 --2- 172.21.15.5:0/282988046 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cff1ca390 0x560cff568670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.779+0000 7f37d1d80740 1 --2- 172.21.15.5:0/282988046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cff1a96b0 0x560cff594d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.779+0000 7f37d1d80740 1 --2- 172.21.15.5:0/282988046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cff1a14d0 0x560cff564780 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.779+0000 7f37d1d80740 1 -- 172.21.15.5:0/282988046 >> 172.21.15.5:0/282988046 conn(0x560cff1be4e0 msgr2=0x560cff5988e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.779+0000 7f37d1d80740 1 -- 172.21.15.5:0/282988046 shutdown_connections 2024-05-20T18:34:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.779+0000 7f37d1d80740 1 -- 172.21.15.5:0/282988046 wait complete. 2024-05-20T18:34:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.780+0000 7f37d1d80740 1 -- start start 2024-05-20T18:34:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.780+0000 7f37d1d80740 1 Processor -- start 2024-05-20T18:34:01.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.781+0000 7f37d1d80740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cff1a14d0 0x560cff62c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.781+0000 7f37d1d80740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cff1a96b0 0x560cff5c1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.781+0000 7f37d1d80740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cff1ca390 0x560cff5b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.781+0000 7f37d1d80740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560cff56a820 con 0x560cff1ca390 2024-05-20T18:34:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.781+0000 7f37d1d80740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f37c41fee80 con 0x560cff1a96b0 2024-05-20T18:34:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.781+0000 7f37d1d80740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x560cff5c1fa0 con 0x560cff1a14d0 2024-05-20T18:34:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.781+0000 7f37d1d80740 10 client.0 _ll_register_callbacks cb 0x7f37b0001380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:34:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.781+0000 7f37c947b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cff1ca390 0x560cff5b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.781+0000 7f37c947b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cff1ca390 0x560cff5b1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56766/0 (socket says 172.21.15.5:56766) 2024-05-20T18:34:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.781+0000 7f37c947b640 1 -- 172.21.15.5:0/4074090609 learned_addr learned my addr 172.21.15.5:0/4074090609 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.782+0000 7f37c8c7a640 1 --2- 172.21.15.5:0/4074090609 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cff1a14d0 0x560cff62c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.782+0000 7f37c3fff640 1 --2- 172.21.15.5:0/4074090609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cff1a96b0 0x560cff5c1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.782+0000 7f37c947b640 1 -- 172.21.15.5:0/4074090609 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cff1a14d0 msgr2=0x560cff62c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.782+0000 7f37c947b640 1 --2- 172.21.15.5:0/4074090609 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cff1a14d0 0x560cff62c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.782+0000 7f37c947b640 1 -- 172.21.15.5:0/4074090609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cff1a96b0 msgr2=0x560cff5c1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.782+0000 7f37c947b640 1 --2- 172.21.15.5:0/4074090609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cff1a96b0 0x560cff5c1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.782+0000 7f37c947b640 1 -- 172.21.15.5:0/4074090609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cff5b1460 con 0x560cff1ca390 2024-05-20T18:34:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.782+0000 7f37c947b640 1 --2- 172.21.15.5:0/4074090609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cff1ca390 0x560cff5b1b20 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009f40 tx=0x7f37bc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b39d298157c5e2b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.782+0000 7f37a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:34:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.782+0000 7f37a37fe640 1 -- 172.21.15.5:0/4074090609 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc036040 con 0x560cff1ca390 2024-05-20T18:34:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.782+0000 7f37a37fe640 1 -- 172.21.15.5:0/4074090609 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7f37bc004240 con 0x560cff1ca390 2024-05-20T18:34:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.782+0000 7f37d1d80740 1 -- 172.21.15.5:0/4074090609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37b80711e0 con 0x560cff1ca390 2024-05-20T18:34:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.782+0000 7f37a37fe640 1 -- 172.21.15.5:0/4074090609 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f37bc02adb0 con 0x560cff1ca390 2024-05-20T18:34:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.782+0000 7f37d1d80740 10 client.7857 fetch_fsmap learned FSMap version 67 2024-05-20T18:34:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.782+0000 7f37d1d80740 1 -- 172.21.15.5:0/4074090609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x560cff5b20c0 con 0x560cff1ca390 2024-05-20T18:34:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.782+0000 7f37a37fe640 1 -- 172.21.15.5:0/4074090609 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37b80711e0 con 0x560cff1ca390 2024-05-20T18:34:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.783+0000 7f37a37fe640 1 -- 172.21.15.5:0/4074090609 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f37bc004440 con 0x560cff1ca390 2024-05-20T18:34:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.783+0000 7f37d1d80740 10 client.7857 fetch_fsmap finished waiting for FSMap version 67 2024-05-20T18:34:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.783+0000 7f37d1d80740 10 client.7857 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-20T18:34:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.783+0000 7f37d1d80740 20 client.7857 populate_metadata read hostname 'smithi005' 2024-05-20T18:34:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.783+0000 7f37d1d80740 1 --2- 172.21.15.5:0/4074090609 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x560cff5d1740 0x560cff5d1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.783+0000 7f37d1d80740 4 client.7857 mds_command: new command op to 7437 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T18:34:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.783+0000 7f37d1d80740 1 -- 172.21.15.5:0/4074090609 --> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560cff4d6780 con 0x560cff5d1740 2024-05-20T18:34:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.783+0000 7f37c8c7a640 1 --2- 172.21.15.5:0/4074090609 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x560cff5d1740 0x560cff5d1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.783+0000 7f37a37fe640 10 client.7857 ms_handle_connect on v2:172.21.15.5:6836/3556492262 2024-05-20T18:34:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.783+0000 7f37c8c7a640 1 --2- 172.21.15.5:0/4074090609 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x560cff5d1740 0x560cff5d1b50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29bb4ec369737d1e server_cookie=53b276da04694577 in_seq=0 out_seq=0 2024-05-20T18:34:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.783+0000 7f37a37fe640 1 -- 172.21.15.5:0/4074090609 <== mds.0 v2:172.21.15.5:6836/3556492262 1 ==== command_reply(tid 0: 0 ) ==== 8+0+263 (crc 0 0 0) 0x560cff4d6780 con 0x560cff5d1740 2024-05-20T18:34:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.783+0000 7f37a37fe640 10 client.7857 handle_command_reply: tid=0 2024-05-20T18:34:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.783+0000 7f37d1d80740 1 client.7857 shutdown 2024-05-20T18:34:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.783+0000 7f37d1d80740 1 -- 172.21.15.5:0/4074090609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cff1ca390 msgr2=0x560cff5b1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.783+0000 7f37d1d80740 1 --2- 172.21.15.5:0/4074090609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cff1ca390 0x560cff5b1b20 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009f40 tx=0x7f37bc004ab0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.784+0000 7f37d1d80740 1 -- 172.21.15.5:0/4074090609 shutdown_connections 2024-05-20T18:34:01.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.784+0000 7f37d1d80740 1 --2- 172.21.15.5:0/4074090609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cff1ca390 0x560cff5b1b20 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.784+0000 7f37d1d80740 1 --2- 172.21.15.5:0/4074090609 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560cff1a96b0 0x560cff5c1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.784+0000 7f37d1d80740 1 --2- 172.21.15.5:0/4074090609 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x560cff5d1740 0x560cff5d1b50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.784+0000 7f37d1d80740 1 --2- 172.21.15.5:0/4074090609 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560cff1a14d0 0x560cff62c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.784+0000 7f37a37fe640 0 client.7857 ms_handle_reset on v2:172.21.15.5:6836/3556492262 2024-05-20T18:34:01.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.784+0000 7f37d1d80740 1 -- 172.21.15.5:0/4074090609 >> 172.21.15.5:0/4074090609 conn(0x560cff1be4e0 msgr2=0x560cff5988c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:01.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.784+0000 7f37d1d80740 1 -- 172.21.15.5:0/4074090609 shutdown_connections 2024-05-20T18:34:01.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.784+0000 7f37d1d80740 1 -- 172.21.15.5:0/4074090609 wait complete. 2024-05-20T18:34:01.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.784+0000 7f37d1d80740 20 client.7857 trim_cache size 0 max 16384 2024-05-20T18:34:01.789 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:34:01.790 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-20T18:34:01.790 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-20T18:34:01.790 INFO:teuthology.orchestra.run.smithi005.stdout: "e5e74c9f-4d8c-4d3c-b9de-5746d84fffcb": { 2024-05-20T18:34:01.790 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/dir", 2024-05-20T18:34:01.790 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "e5e74c9f-4d8c-4d3c-b9de-5746d84fffcb", 2024-05-20T18:34:01.790 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,repair" 2024-05-20T18:34:01.790 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:34:01.790 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:34:01.790 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:34:01.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.784+0000 7f37caede640 1 -- 172.21.15.5:0/162898009 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f37a806be60 msgr2=0x7f37a806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.784+0000 7f37caede640 1 --2- 172.21.15.5:0/162898009 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f37a806be60 0x7f37a806e370 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f37c41e7250 tx=0x7f37b0009040 comp rx=0 tx=0).stop 2024-05-20T18:34:01.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.784+0000 7f37caede640 1 -- 172.21.15.5:0/162898009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37c40b17f0 msgr2=0x7f37c409f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:01.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.784+0000 7f37caede640 1 --2- 172.21.15.5:0/162898009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37c40b17f0 0x7f37c409f400 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f37b800e870 tx=0x7f37b800ed30 comp rx=0 tx=0).stop 2024-05-20T18:34:01.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.784+0000 7f37caede640 1 -- 172.21.15.5:0/162898009 shutdown_connections 2024-05-20T18:34:01.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.784+0000 7f37caede640 1 --2- 172.21.15.5:0/162898009 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f37a806be60 0x7f37a806e370 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.784+0000 7f37caede640 1 --2- 172.21.15.5:0/162898009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f37c415d460 0x7f37c41f3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.784+0000 7f37caede640 1 --2- 172.21.15.5:0/162898009 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f37c40d1780 0x7f37c415c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.784+0000 7f37caede640 1 --2- 172.21.15.5:0/162898009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37c40b17f0 0x7f37c409f400 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:01.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.784+0000 7f37caede640 1 -- 172.21.15.5:0/162898009 >> 172.21.15.5:0/162898009 conn(0x7f37c40b6620 msgr2=0x7f37c4141580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:01.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.784+0000 7f37caede640 1 -- 172.21.15.5:0/162898009 shutdown_connections 2024-05-20T18:34:01.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:01.784+0000 7f37caede640 1 -- 172.21.15.5:0/162898009 wait complete. 2024-05-20T18:34:01.796 INFO:tasks.cephfs.filesystem:scrub status for tag:e5e74c9f-4d8c-4d3c-b9de-5746d84fffcb - {'path': '/dir', 'tag': 'e5e74c9f-4d8c-4d3c-b9de-5746d84fffcb', 'options': 'recursive,repair'} 2024-05-20T18:34:11.586 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:34:11.589 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:34:31.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:34:31.798 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-20T18:34:31.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.929+0000 7fde27c91640 1 Processor -- start 2024-05-20T18:34:31.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.929+0000 7fde27c91640 1 -- start start 2024-05-20T18:34:31.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.931+0000 7fde27c91640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fde20162820 0x7fde200b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:31.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.931+0000 7fde27c91640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fde200b3e80 0x7fde2006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:31.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.931+0000 7fde27c91640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde2006dd20 con 0x7fde2006d910 2024-05-20T18:34:31.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.931+0000 7fde27c91640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fde200d0630 con 0x7fde200b3e80 2024-05-20T18:34:31.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.931+0000 7fde27c91640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fde200d39c0 con 0x7fde20162820 2024-05-20T18:34:31.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.931+0000 7fde2522c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fde200b3e80 0x7fde2006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:31.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.931+0000 7fde2522c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fde200b3e80 0x7fde2006d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56560/0 (socket says 172.21.15.5:56560) 2024-05-20T18:34:31.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.931+0000 7fde2522c640 1 -- 172.21.15.5:0/971801808 learned_addr learned my addr 172.21.15.5:0/971801808 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:31.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.931+0000 7fde25a2d640 1 --2- 172.21.15.5:0/971801808 >> v2:172.21.15.63:3300/0 conn(0x7fde20162820 0x7fde200b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:31.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.931+0000 7fde2522c640 1 -- 172.21.15.5:0/971801808 >> v2:172.21.15.63:3300/0 conn(0x7fde20162820 msgr2=0x7fde200b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:31.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.931+0000 7fde2522c640 1 --2- 172.21.15.5:0/971801808 >> v2:172.21.15.63:3300/0 conn(0x7fde20162820 0x7fde200b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.931+0000 7fde2522c640 1 -- 172.21.15.5:0/971801808 >> v1:172.21.15.63:6789/0 conn(0x7fde2006d910 legacy=0x7fde200d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:31.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.931+0000 7fde2522c640 1 -- 172.21.15.5:0/971801808 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde200d3ca0 con 0x7fde200b3e80 2024-05-20T18:34:31.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.932+0000 7fde2522c640 1 --2- 172.21.15.5:0/971801808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde200b3e80 0x7fde2006d3d0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fde200b7740 tx=0x7fde1002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4583e2b69555351 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:31.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.932+0000 7fde24a2b640 1 -- 172.21.15.5:0/971801808 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde10037070 con 0x7fde200b3e80 2024-05-20T18:34:31.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.932+0000 7fde24a2b640 1 -- 172.21.15.5:0/971801808 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde10032440 con 0x7fde200b3e80 2024-05-20T18:34:31.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.932+0000 7fde24a2b640 1 -- 172.21.15.5:0/971801808 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde10032810 con 0x7fde200b3e80 2024-05-20T18:34:31.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.933+0000 7fde27c91640 1 -- 172.21.15.5:0/971801808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde200b3e80 msgr2=0x7fde2006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:31.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.933+0000 7fde27c91640 1 --2- 172.21.15.5:0/971801808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde200b3e80 0x7fde2006d3d0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fde200b7740 tx=0x7fde1002a270 comp rx=0 tx=0).stop 2024-05-20T18:34:31.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.933+0000 7fde27c91640 1 -- 172.21.15.5:0/971801808 shutdown_connections 2024-05-20T18:34:31.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.933+0000 7fde27c91640 1 --2- 172.21.15.5:0/971801808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde200b3e80 0x7fde2006d3d0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.933+0000 7fde27c91640 1 --2- 172.21.15.5:0/971801808 >> v2:172.21.15.63:3300/0 conn(0x7fde20162820 0x7fde200b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.933+0000 7fde27c91640 1 -- 172.21.15.5:0/971801808 >> 172.21.15.5:0/971801808 conn(0x7fde200d4bd0 msgr2=0x7fde200d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:31.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.933+0000 7fde27c91640 1 -- 172.21.15.5:0/971801808 shutdown_connections 2024-05-20T18:34:31.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.933+0000 7fde27c91640 1 -- 172.21.15.5:0/971801808 wait complete. 2024-05-20T18:34:31.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.935+0000 7fde27c91640 1 Processor -- start 2024-05-20T18:34:31.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.935+0000 7fde27c91640 1 -- start start 2024-05-20T18:34:31.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.936+0000 7fde27c91640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde2006d910 0x7fde201539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:31.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.936+0000 7fde27c91640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde200b3e80 0x7fde200c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:31.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.936+0000 7fde27c91640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde20162820 0x7fde200b6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:31.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.936+0000 7fde27c91640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fde200d0300 con 0x7fde2006d910 2024-05-20T18:34:31.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.936+0000 7fde27c91640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fde200d4260 con 0x7fde20162820 2024-05-20T18:34:31.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.936+0000 7fde27c91640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fde200b91a0 con 0x7fde200b3e80 2024-05-20T18:34:31.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.936+0000 7fde25a2d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde2006d910 0x7fde201539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:31.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.936+0000 7fde25a2d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde2006d910 0x7fde201539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56576/0 (socket says 172.21.15.5:56576) 2024-05-20T18:34:31.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.936+0000 7fde25a2d640 1 -- 172.21.15.5:0/3575734888 learned_addr learned my addr 172.21.15.5:0/3575734888 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:31.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.937+0000 7fde2522c640 1 --2- 172.21.15.5:0/3575734888 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde200b3e80 0x7fde200c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:31.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.937+0000 7fde2622e640 1 --2- 172.21.15.5:0/3575734888 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde20162820 0x7fde200b6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:31.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.937+0000 7fde25a2d640 1 -- 172.21.15.5:0/3575734888 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde200b3e80 msgr2=0x7fde200c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:31.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.937+0000 7fde25a2d640 1 --2- 172.21.15.5:0/3575734888 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde200b3e80 0x7fde200c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.937+0000 7fde25a2d640 1 -- 172.21.15.5:0/3575734888 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde20162820 msgr2=0x7fde200b6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:31.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.937+0000 7fde25a2d640 1 --2- 172.21.15.5:0/3575734888 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde20162820 0x7fde200b6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.937+0000 7fde25a2d640 1 -- 172.21.15.5:0/3575734888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde2015ac30 con 0x7fde2006d910 2024-05-20T18:34:31.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.937+0000 7fde2622e640 1 --2- 172.21.15.5:0/3575734888 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde20162820 0x7fde200b6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:34:31.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.937+0000 7fde2522c640 1 --2- 172.21.15.5:0/3575734888 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde200b3e80 0x7fde200c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:34:31.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.937+0000 7fde25a2d640 1 --2- 172.21.15.5:0/3575734888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde2006d910 0x7fde201539c0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fde1c00e870 tx=0x7fde1c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:31.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.937+0000 7fde167fc640 1 -- 172.21.15.5:0/3575734888 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c017070 con 0x7fde2006d910 2024-05-20T18:34:31.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.937+0000 7fde167fc640 1 -- 172.21.15.5:0/3575734888 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde1c00c940 con 0x7fde2006d910 2024-05-20T18:34:31.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.937+0000 7fde27c91640 1 -- 172.21.15.5:0/3575734888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde2015af10 con 0x7fde2006d910 2024-05-20T18:34:31.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.937+0000 7fde167fc640 1 -- 172.21.15.5:0/3575734888 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c00cd30 con 0x7fde2006d910 2024-05-20T18:34:31.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.937+0000 7fde27c91640 1 -- 172.21.15.5:0/3575734888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde200b5e20 con 0x7fde2006d910 2024-05-20T18:34:31.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.940+0000 7fde167fc640 1 -- 172.21.15.5:0/3575734888 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fde1c01a070 con 0x7fde2006d910 2024-05-20T18:34:31.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.940+0000 7fde167fc640 1 --2- 172.21.15.5:0/3575734888 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fde0406be30 0x7fde0406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:31.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.940+0000 7fde167fc640 1 -- 172.21.15.5:0/3575734888 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7fde1c08b320 con 0x7fde2006d910 2024-05-20T18:34:31.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.940+0000 7fde2522c640 1 --2- 172.21.15.5:0/3575734888 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fde0406be30 0x7fde0406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:31.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.941+0000 7fde2522c640 1 --2- 172.21.15.5:0/3575734888 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fde0406be30 0x7fde0406e340 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fde10034000 tx=0x7fde1002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:31.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.941+0000 7fde2ec10740 1 Processor -- start 2024-05-20T18:34:31.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.941+0000 7fde2ec10740 1 -- start start 2024-05-20T18:34:31.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.943+0000 7fde2ec10740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555d88a77390 0x555d88e41b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:31.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.943+0000 7fde2ec10740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d88a4e4d0 0x555d88e5c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:31.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.943+0000 7fde2ec10740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d88a566b0 0x555d88e5cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:31.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.943+0000 7fde2ec10740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555d88d7bd00 con 0x555d88a4e4d0 2024-05-20T18:34:31.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.943+0000 7fde2ec10740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555d88d234c0 con 0x555d88a566b0 2024-05-20T18:34:31.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.943+0000 7fde2ec10740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x555d88d77cc0 con 0x555d88a77390 2024-05-20T18:34:31.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.943+0000 7fde2522c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d88a4e4d0 0x555d88e5c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:31.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.943+0000 7fde2522c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d88a4e4d0 0x555d88e5c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56588/0 (socket says 172.21.15.5:56588) 2024-05-20T18:34:31.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.943+0000 7fde2522c640 1 -- 172.21.15.5:0/253151437 learned_addr learned my addr 172.21.15.5:0/253151437 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:31.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.943+0000 7fde25a2d640 1 --2- 172.21.15.5:0/253151437 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555d88a77390 0x555d88e41b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:31.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.943+0000 7fde2622e640 1 --2- 172.21.15.5:0/253151437 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d88a566b0 0x555d88e5cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:31.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.943+0000 7fde2522c640 1 -- 172.21.15.5:0/253151437 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555d88a77390 msgr2=0x555d88e41b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:31.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.943+0000 7fde2522c640 1 --2- 172.21.15.5:0/253151437 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555d88a77390 0x555d88e41b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.943+0000 7fde2522c640 1 -- 172.21.15.5:0/253151437 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d88a566b0 msgr2=0x555d88e5cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:31.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.943+0000 7fde2522c640 1 --2- 172.21.15.5:0/253151437 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d88a566b0 0x555d88e5cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.943+0000 7fde2522c640 1 -- 172.21.15.5:0/253151437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d88b597d0 con 0x555d88a4e4d0 2024-05-20T18:34:31.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.943+0000 7fde25a2d640 1 --2- 172.21.15.5:0/253151437 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555d88a77390 0x555d88e41b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:34:31.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.943+0000 7fde2522c640 1 --2- 172.21.15.5:0/253151437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d88a4e4d0 0x555d88e5c890 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fde10032790 tx=0x7fde1002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0346d3d232689f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:31.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.943+0000 7fde037fe640 1 -- 172.21.15.5:0/253151437 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde10037040 con 0x555d88a4e4d0 2024-05-20T18:34:31.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.944+0000 7fde037fe640 1 -- 172.21.15.5:0/253151437 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde1004c070 con 0x555d88a4e4d0 2024-05-20T18:34:31.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.944+0000 7fde037fe640 1 -- 172.21.15.5:0/253151437 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1003d440 con 0x555d88a4e4d0 2024-05-20T18:34:31.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.944+0000 7fde2ec10740 1 -- 172.21.15.5:0/253151437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d88a4e4d0 msgr2=0x555d88e5c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:31.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.944+0000 7fde2ec10740 1 --2- 172.21.15.5:0/253151437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d88a4e4d0 0x555d88e5c890 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fde10032790 tx=0x7fde1002c000 comp rx=0 tx=0).stop 2024-05-20T18:34:31.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.944+0000 7fde2ec10740 1 -- 172.21.15.5:0/253151437 shutdown_connections 2024-05-20T18:34:31.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.944+0000 7fde2ec10740 1 --2- 172.21.15.5:0/253151437 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d88a566b0 0x555d88e5cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.944+0000 7fde2ec10740 1 --2- 172.21.15.5:0/253151437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d88a4e4d0 0x555d88e5c890 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.944+0000 7fde2ec10740 1 --2- 172.21.15.5:0/253151437 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555d88a77390 0x555d88e41b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.944+0000 7fde2ec10740 1 -- 172.21.15.5:0/253151437 >> 172.21.15.5:0/253151437 conn(0x555d88a6b4e0 msgr2=0x555d88e171c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:31.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.944+0000 7fde2ec10740 1 -- 172.21.15.5:0/253151437 shutdown_connections 2024-05-20T18:34:31.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.944+0000 7fde2ec10740 1 -- 172.21.15.5:0/253151437 wait complete. 2024-05-20T18:34:31.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.945+0000 7fde2ec10740 1 -- start start 2024-05-20T18:34:31.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.945+0000 7fde2ec10740 1 Processor -- start 2024-05-20T18:34:31.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.947+0000 7fde2ec10740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d88a4e4d0 0x555d88e3ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:31.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.947+0000 7fde2ec10740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d88a566b0 0x555d88e44350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:31.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.947+0000 7fde2ec10740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555d88a77390 0x555d88e5e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:31.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.947+0000 7fde2ec10740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555d88d36b90 con 0x555d88a4e4d0 2024-05-20T18:34:31.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.947+0000 7fde2ec10740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555d88e55530 con 0x555d88a566b0 2024-05-20T18:34:31.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.947+0000 7fde2ec10740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x555d88e44ac0 con 0x555d88a77390 2024-05-20T18:34:31.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.947+0000 7fde2ec10740 10 client.0 _ll_register_callbacks cb 0x555d88d72da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:34:31.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.947+0000 7fde25a2d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d88a4e4d0 0x555d88e3ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:31.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.947+0000 7fde25a2d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d88a4e4d0 0x555d88e3ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56606/0 (socket says 172.21.15.5:56606) 2024-05-20T18:34:31.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.947+0000 7fde25a2d640 1 -- 172.21.15.5:0/1207889629 learned_addr learned my addr 172.21.15.5:0/1207889629 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:31.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.947+0000 7fde2522c640 1 --2- 172.21.15.5:0/1207889629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d88a566b0 0x555d88e44350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:31.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.947+0000 7fde2622e640 1 --2- 172.21.15.5:0/1207889629 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555d88a77390 0x555d88e5e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:31.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.947+0000 7fde25a2d640 1 -- 172.21.15.5:0/1207889629 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555d88a77390 msgr2=0x555d88e5e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:31.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.947+0000 7fde25a2d640 1 --2- 172.21.15.5:0/1207889629 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555d88a77390 0x555d88e5e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.947+0000 7fde25a2d640 1 -- 172.21.15.5:0/1207889629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d88a566b0 msgr2=0x555d88e44350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:31.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.947+0000 7fde25a2d640 1 --2- 172.21.15.5:0/1207889629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d88a566b0 0x555d88e44350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.947+0000 7fde25a2d640 1 -- 172.21.15.5:0/1207889629 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d88a8f3f0 con 0x555d88a4e4d0 2024-05-20T18:34:31.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.947+0000 7fde2622e640 1 --2- 172.21.15.5:0/1207889629 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555d88a77390 0x555d88e5e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:34:31.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.948+0000 7fde25a2d640 1 --2- 172.21.15.5:0/1207889629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d88a4e4d0 0x555d88e3ac30 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fde1c00e840 tx=0x7fde1c059bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0e4fa198e60c02 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:31.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.948+0000 7fde00ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:34:31.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.948+0000 7fde00ff9640 1 -- 172.21.15.5:0/1207889629 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c021040 con 0x555d88a4e4d0 2024-05-20T18:34:31.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.948+0000 7fde00ff9640 1 -- 172.21.15.5:0/1207889629 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7fde1c0711b0 con 0x555d88a4e4d0 2024-05-20T18:34:31.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.948+0000 7fde2ec10740 1 -- 172.21.15.5:0/1207889629 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde1004c070 con 0x555d88a4e4d0 2024-05-20T18:34:31.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.948+0000 7fde00ff9640 1 -- 172.21.15.5:0/1207889629 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fde1c0713d0 con 0x555d88a4e4d0 2024-05-20T18:34:31.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.948+0000 7fde2ec10740 10 client.7869 fetch_fsmap learned FSMap version 67 2024-05-20T18:34:31.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.948+0000 7fde2ec10740 1 -- 172.21.15.5:0/1207889629 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x555d88a4f0f0 con 0x555d88a4e4d0 2024-05-20T18:34:31.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.948+0000 7fde00ff9640 1 -- 172.21.15.5:0/1207889629 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde1004c070 con 0x555d88a4e4d0 2024-05-20T18:34:31.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.948+0000 7fde00ff9640 1 -- 172.21.15.5:0/1207889629 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7fde1c0587d0 con 0x555d88a4e4d0 2024-05-20T18:34:31.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.948+0000 7fde2ec10740 10 client.7869 fetch_fsmap finished waiting for FSMap version 67 2024-05-20T18:34:31.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.948+0000 7fde2ec10740 10 client.7869 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-20T18:34:31.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.948+0000 7fde2ec10740 20 client.7869 populate_metadata read hostname 'smithi005' 2024-05-20T18:34:31.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.948+0000 7fde2ec10740 1 --2- 172.21.15.5:0/1207889629 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x555d88a35f60 0x555d88e54580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:31.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.948+0000 7fde2ec10740 4 client.7869 mds_command: new command op to 7437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:34:31.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.948+0000 7fde2ec10740 1 -- 172.21.15.5:0/1207889629 --> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d88d89430 con 0x555d88a35f60 2024-05-20T18:34:31.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.948+0000 7fde2522c640 1 --2- 172.21.15.5:0/1207889629 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x555d88a35f60 0x555d88e54580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:31.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.949+0000 7fde2522c640 1 --2- 172.21.15.5:0/1207889629 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x555d88a35f60 0x555d88e54580 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51bf8094a02947e server_cookie=90a483c87f00115e in_seq=0 out_seq=0 2024-05-20T18:34:31.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.949+0000 7fde00ff9640 10 client.7869 ms_handle_connect on v2:172.21.15.5:6836/3556492262 2024-05-20T18:34:31.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.949+0000 7fde00ff9640 1 -- 172.21.15.5:0/1207889629 <== mds.0 v2:172.21.15.5:6836/3556492262 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555d88d89430 con 0x555d88a35f60 2024-05-20T18:34:31.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.949+0000 7fde00ff9640 10 client.7869 handle_command_reply: tid=0 2024-05-20T18:34:31.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.949+0000 7fde2ec10740 1 client.7869 shutdown 2024-05-20T18:34:31.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.950+0000 7fde2ec10740 1 -- 172.21.15.5:0/1207889629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d88a4e4d0 msgr2=0x555d88e3ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:31.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.950+0000 7fde2ec10740 1 --2- 172.21.15.5:0/1207889629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d88a4e4d0 0x555d88e3ac30 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fde1c00e840 tx=0x7fde1c059bc0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.950+0000 7fde2ec10740 1 -- 172.21.15.5:0/1207889629 shutdown_connections 2024-05-20T18:34:31.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.950+0000 7fde2ec10740 1 --2- 172.21.15.5:0/1207889629 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x555d88a35f60 0x555d88e54580 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.950+0000 7fde2ec10740 1 --2- 172.21.15.5:0/1207889629 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555d88a77390 0x555d88e5e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.950+0000 7fde2ec10740 1 --2- 172.21.15.5:0/1207889629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d88a566b0 0x555d88e44350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.950+0000 7fde00ff9640 0 client.7869 ms_handle_reset on v2:172.21.15.5:6836/3556492262 2024-05-20T18:34:31.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.950+0000 7fde2ec10740 1 --2- 172.21.15.5:0/1207889629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d88a4e4d0 0x555d88e3ac30 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.950+0000 7fde2ec10740 1 -- 172.21.15.5:0/1207889629 >> 172.21.15.5:0/1207889629 conn(0x555d88a6b4e0 msgr2=0x555d88e15670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:31.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.950+0000 7fde2ec10740 1 -- 172.21.15.5:0/1207889629 shutdown_connections 2024-05-20T18:34:31.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.950+0000 7fde2ec10740 1 -- 172.21.15.5:0/1207889629 wait complete. 2024-05-20T18:34:31.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.950+0000 7fde2ec10740 20 client.7869 trim_cache size 0 max 16384 2024-05-20T18:34:31.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.968+0000 7fde2ec10740 1 Processor -- start 2024-05-20T18:34:31.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.969+0000 7fde2ec10740 1 -- start start 2024-05-20T18:34:31.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.970+0000 7fde2ec10740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d88a4e4d0 0x555d88e3d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:31.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.970+0000 7fde2ec10740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d88a566b0 0x555d88e3d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:31.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.970+0000 7fde2ec10740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555d88a77390 0x555d88e15670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:31.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.970+0000 7fde2ec10740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fde202026d0 con 0x555d88a566b0 2024-05-20T18:34:31.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.970+0000 7fde2ec10740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fde202016b0 con 0x555d88a4e4d0 2024-05-20T18:34:31.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.970+0000 7fde2ec10740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fde20200b90 con 0x555d88a77390 2024-05-20T18:34:31.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.970+0000 7fde2522c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d88a566b0 0x555d88e3d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:31.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.970+0000 7fde2522c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d88a566b0 0x555d88e3d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56618/0 (socket says 172.21.15.5:56618) 2024-05-20T18:34:31.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.970+0000 7fde2522c640 1 -- 172.21.15.5:0/3898039118 learned_addr learned my addr 172.21.15.5:0/3898039118 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:31.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.970+0000 7fde25a2d640 1 --2- 172.21.15.5:0/3898039118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d88a4e4d0 0x555d88e3d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:31.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.970+0000 7fde2622e640 1 --2- 172.21.15.5:0/3898039118 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555d88a77390 0x555d88e15670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:31.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.970+0000 7fde2522c640 1 -- 172.21.15.5:0/3898039118 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555d88a77390 msgr2=0x555d88e15670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:31.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.970+0000 7fde2522c640 1 --2- 172.21.15.5:0/3898039118 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555d88a77390 0x555d88e15670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.970+0000 7fde2522c640 1 -- 172.21.15.5:0/3898039118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d88a4e4d0 msgr2=0x555d88e3d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:31.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.970+0000 7fde2522c640 1 --2- 172.21.15.5:0/3898039118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d88a4e4d0 0x555d88e3d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.970+0000 7fde2522c640 1 -- 172.21.15.5:0/3898039118 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d88e15210 con 0x555d88a566b0 2024-05-20T18:34:31.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.970+0000 7fde25a2d640 1 --2- 172.21.15.5:0/3898039118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d88a4e4d0 0x555d88e3d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:34:31.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.970+0000 7fde2522c640 1 --2- 172.21.15.5:0/3898039118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d88a566b0 0x555d88e3d610 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fde10009630 tx=0x7fde10039000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6942b61f922001 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:31.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.971+0000 7fde02ffd640 1 -- 172.21.15.5:0/3898039118 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde10037040 con 0x555d88a566b0 2024-05-20T18:34:31.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.971+0000 7fde02ffd640 1 -- 172.21.15.5:0/3898039118 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde1004c040 con 0x555d88a566b0 2024-05-20T18:34:31.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.971+0000 7fde02ffd640 1 -- 172.21.15.5:0/3898039118 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde10003610 con 0x555d88a566b0 2024-05-20T18:34:31.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.971+0000 7fde2ec10740 1 -- 172.21.15.5:0/3898039118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d88a566b0 msgr2=0x555d88e3d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:31.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.971+0000 7fde2ec10740 1 --2- 172.21.15.5:0/3898039118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d88a566b0 0x555d88e3d610 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fde10009630 tx=0x7fde10039000 comp rx=0 tx=0).stop 2024-05-20T18:34:31.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.971+0000 7fde2ec10740 1 -- 172.21.15.5:0/3898039118 shutdown_connections 2024-05-20T18:34:31.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.971+0000 7fde2ec10740 1 --2- 172.21.15.5:0/3898039118 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555d88a77390 0x555d88e15670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.971+0000 7fde2ec10740 1 --2- 172.21.15.5:0/3898039118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d88a566b0 0x555d88e3d610 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.971+0000 7fde2ec10740 1 --2- 172.21.15.5:0/3898039118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d88a4e4d0 0x555d88e3d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.971+0000 7fde2ec10740 1 -- 172.21.15.5:0/3898039118 >> 172.21.15.5:0/3898039118 conn(0x555d88a6b4e0 msgr2=0x555d88e3aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:31.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.971+0000 7fde2ec10740 1 -- 172.21.15.5:0/3898039118 shutdown_connections 2024-05-20T18:34:31.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.971+0000 7fde2ec10740 1 -- 172.21.15.5:0/3898039118 wait complete. 2024-05-20T18:34:31.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.972+0000 7fde2ec10740 1 -- start start 2024-05-20T18:34:31.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.972+0000 7fde2ec10740 1 Processor -- start 2024-05-20T18:34:31.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.974+0000 7fde2ec10740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d88a4e4d0 0x555d88e52dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:31.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.974+0000 7fde2ec10740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555d88a566b0 0x555d88e3e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:31.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.974+0000 7fde2ec10740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d88a77390 0x555d88e15a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:31.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.974+0000 7fde2ec10740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555d88e7fca0 con 0x555d88a4e4d0 2024-05-20T18:34:31.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.974+0000 7fde2ec10740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fde20201080 con 0x555d88a77390 2024-05-20T18:34:31.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.974+0000 7fde2ec10740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x555d88e17410 con 0x555d88a566b0 2024-05-20T18:34:31.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.974+0000 7fde2ec10740 10 client.0 _ll_register_callbacks cb 0x7fde1c009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:34:31.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.974+0000 7fde25a2d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d88a4e4d0 0x555d88e52dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:31.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.974+0000 7fde25a2d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d88a4e4d0 0x555d88e52dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56632/0 (socket says 172.21.15.5:56632) 2024-05-20T18:34:31.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.974+0000 7fde25a2d640 1 -- 172.21.15.5:0/2521194128 learned_addr learned my addr 172.21.15.5:0/2521194128 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:31.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.974+0000 7fde2622e640 1 --2- 172.21.15.5:0/2521194128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d88a77390 0x555d88e15a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:31.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.974+0000 7fde2522c640 1 --2- 172.21.15.5:0/2521194128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555d88a566b0 0x555d88e3e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:31.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.974+0000 7fde25a2d640 1 -- 172.21.15.5:0/2521194128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555d88a566b0 msgr2=0x555d88e3e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:31.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.974+0000 7fde25a2d640 1 --2- 172.21.15.5:0/2521194128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555d88a566b0 0x555d88e3e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.974+0000 7fde25a2d640 1 -- 172.21.15.5:0/2521194128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d88a77390 msgr2=0x555d88e15a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:31.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.974+0000 7fde25a2d640 1 --2- 172.21.15.5:0/2521194128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d88a77390 0x555d88e15a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.974+0000 7fde25a2d640 1 -- 172.21.15.5:0/2521194128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d88ed3390 con 0x555d88a4e4d0 2024-05-20T18:34:31.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.974+0000 7fde25a2d640 1 --2- 172.21.15.5:0/2521194128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d88a4e4d0 0x555d88e52dc0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fde1c004770 tx=0x7fde1c0894d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23cf1fa85712c697 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:31.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.974+0000 7fde00ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:34:31.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.975+0000 7fde00ff9640 1 -- 172.21.15.5:0/2521194128 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c0552d0 con 0x555d88a4e4d0 2024-05-20T18:34:31.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.975+0000 7fde00ff9640 1 -- 172.21.15.5:0/2521194128 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7fde1c058240 con 0x555d88a4e4d0 2024-05-20T18:34:31.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.975+0000 7fde2ec10740 1 -- 172.21.15.5:0/2521194128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde1004c040 con 0x555d88a4e4d0 2024-05-20T18:34:31.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.975+0000 7fde00ff9640 1 -- 172.21.15.5:0/2521194128 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fde1c0713c0 con 0x555d88a4e4d0 2024-05-20T18:34:31.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.975+0000 7fde2ec10740 10 client.7875 fetch_fsmap learned FSMap version 67 2024-05-20T18:34:31.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.975+0000 7fde2ec10740 1 -- 172.21.15.5:0/2521194128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x555d88ed3630 con 0x555d88a4e4d0 2024-05-20T18:34:31.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.975+0000 7fde00ff9640 1 -- 172.21.15.5:0/2521194128 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde1004c040 con 0x555d88a4e4d0 2024-05-20T18:34:31.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.975+0000 7fde00ff9640 1 -- 172.21.15.5:0/2521194128 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7fde1c058440 con 0x555d88a4e4d0 2024-05-20T18:34:31.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.975+0000 7fde2ec10740 10 client.7875 fetch_fsmap finished waiting for FSMap version 67 2024-05-20T18:34:31.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.975+0000 7fde2ec10740 10 client.7875 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-20T18:34:31.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.975+0000 7fde2ec10740 20 client.7875 populate_metadata read hostname 'smithi005' 2024-05-20T18:34:31.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.975+0000 7fde2ec10740 1 --2- 172.21.15.5:0/2521194128 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x555d88e15fb0 0x555d88e7e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:31.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.975+0000 7fde2ec10740 4 client.7875 mds_command: new command op to 7437 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T18:34:31.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.975+0000 7fde2ec10740 1 -- 172.21.15.5:0/2521194128 --> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555d88d83780 con 0x555d88e15fb0 2024-05-20T18:34:31.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.975+0000 7fde2522c640 1 --2- 172.21.15.5:0/2521194128 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x555d88e15fb0 0x555d88e7e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:31.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.976+0000 7fde00ff9640 10 client.7875 ms_handle_connect on v2:172.21.15.5:6836/3556492262 2024-05-20T18:34:31.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.976+0000 7fde2522c640 1 --2- 172.21.15.5:0/2521194128 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x555d88e15fb0 0x555d88e7e520 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aacf68c38298721 server_cookie=c81fb94614fdbe00 in_seq=0 out_seq=0 2024-05-20T18:34:31.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.976+0000 7fde00ff9640 1 -- 172.21.15.5:0/2521194128 <== mds.0 v2:172.21.15.5:6836/3556492262 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555d88d83780 con 0x555d88e15fb0 2024-05-20T18:34:31.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.976+0000 7fde00ff9640 10 client.7875 handle_command_reply: tid=0 2024-05-20T18:34:31.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.976+0000 7fde2ec10740 1 client.7875 shutdown 2024-05-20T18:34:31.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.976+0000 7fde2ec10740 1 -- 172.21.15.5:0/2521194128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d88a4e4d0 msgr2=0x555d88e52dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:31.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.976+0000 7fde2ec10740 1 --2- 172.21.15.5:0/2521194128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d88a4e4d0 0x555d88e52dc0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fde1c004770 tx=0x7fde1c0894d0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.976+0000 7fde2ec10740 1 -- 172.21.15.5:0/2521194128 shutdown_connections 2024-05-20T18:34:31.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.976+0000 7fde2ec10740 1 --2- 172.21.15.5:0/2521194128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555d88a77390 0x555d88e15a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.976+0000 7fde2ec10740 1 --2- 172.21.15.5:0/2521194128 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x555d88e15fb0 0x555d88e7e520 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.976+0000 7fde2ec10740 1 --2- 172.21.15.5:0/2521194128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555d88a566b0 0x555d88e3e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.976+0000 7fde2ec10740 1 --2- 172.21.15.5:0/2521194128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d88a4e4d0 0x555d88e52dc0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.976+0000 7fde00ff9640 0 client.7875 ms_handle_reset on v2:172.21.15.5:6836/3556492262 2024-05-20T18:34:31.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.976+0000 7fde2ec10740 1 -- 172.21.15.5:0/2521194128 >> 172.21.15.5:0/2521194128 conn(0x555d88a6b4e0 msgr2=0x555d88e3aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:31.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.976+0000 7fde2ec10740 1 -- 172.21.15.5:0/2521194128 shutdown_connections 2024-05-20T18:34:31.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.976+0000 7fde2ec10740 1 -- 172.21.15.5:0/2521194128 wait complete. 2024-05-20T18:34:31.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.976+0000 7fde2ec10740 20 client.7875 trim_cache size 0 max 16384 2024-05-20T18:34:31.984 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:34:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-20T18:34:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-20T18:34:31.985 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:34:31.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.977+0000 7fde27c91640 1 -- 172.21.15.5:0/3575734888 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fde0406be30 msgr2=0x7fde0406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.977+0000 7fde27c91640 1 --2- 172.21.15.5:0/3575734888 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fde0406be30 0x7fde0406e340 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fde10034000 tx=0x7fde1002e000 comp rx=0 tx=0).stop 2024-05-20T18:34:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.977+0000 7fde27c91640 1 -- 172.21.15.5:0/3575734888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde2006d910 msgr2=0x7fde201539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.977+0000 7fde27c91640 1 --2- 172.21.15.5:0/3575734888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde2006d910 0x7fde201539c0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fde1c00e870 tx=0x7fde1c00ed30 comp rx=0 tx=0).stop 2024-05-20T18:34:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.977+0000 7fde27c91640 1 -- 172.21.15.5:0/3575734888 shutdown_connections 2024-05-20T18:34:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.977+0000 7fde27c91640 1 --2- 172.21.15.5:0/3575734888 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fde0406be30 0x7fde0406e340 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.977+0000 7fde27c91640 1 --2- 172.21.15.5:0/3575734888 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde20162820 0x7fde200b6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.977+0000 7fde27c91640 1 --2- 172.21.15.5:0/3575734888 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde200b3e80 0x7fde200c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.977+0000 7fde27c91640 1 --2- 172.21.15.5:0/3575734888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde2006d910 0x7fde201539c0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:31.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.977+0000 7fde27c91640 1 -- 172.21.15.5:0/3575734888 >> 172.21.15.5:0/3575734888 conn(0x7fde200d4bd0 msgr2=0x7fde200ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:31.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.977+0000 7fde27c91640 1 -- 172.21.15.5:0/3575734888 shutdown_connections 2024-05-20T18:34:31.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:31.977+0000 7fde27c91640 1 -- 172.21.15.5:0/3575734888 wait complete. 2024-05-20T18:34:31.990 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T18:34:31.991 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:34:32.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.105+0000 7f7758073640 1 Processor -- start 2024-05-20T18:34:32.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.106+0000 7f7758073640 1 -- start start 2024-05-20T18:34:32.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.107+0000 7f7758073640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7750163a10 0x7f77500d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.107+0000 7f7758073640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f77500b4010 0x7f77500cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.107+0000 7f7758073640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77500d0370 con 0x7f77500d14b0 2024-05-20T18:34:32.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.107+0000 7f7758073640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f77500d3ca0 con 0x7f77500b4010 2024-05-20T18:34:32.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.107+0000 7f7758073640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f77500d0650 con 0x7f7750163a10 2024-05-20T18:34:32.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.107+0000 7f775560e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f77500b4010 0x7f77500cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.107+0000 7f7756610640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f77500d14b0 0x7f77500d18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:49346/0 (socket says 172.21.15.5:49346) 2024-05-20T18:34:32.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.107+0000 7f7756610640 1 -- 172.21.15.5:0/342993131 learned_addr learned my addr 172.21.15.5:0/342993131 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:34:32.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.107+0000 7f7755e0f640 1 --2- 172.21.15.5:0/342993131 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7750163a10 0x7f77500d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.108+0000 7f775560e640 1 -- 172.21.15.5:0/342993131 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7750163a10 msgr2=0x7f77500d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:32.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.108+0000 7f775560e640 1 --2- 172.21.15.5:0/342993131 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7750163a10 0x7f77500d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.108+0000 7f775560e640 1 -- 172.21.15.5:0/342993131 >> v1:172.21.15.5:6789/0 conn(0x7f77500d14b0 legacy=0x7f77500d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:32.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.108+0000 7f775560e640 1 -- 172.21.15.5:0/342993131 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77500d3650 con 0x7f77500b4010 2024-05-20T18:34:32.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.108+0000 7f7754e0d640 1 -- 172.21.15.5:0/342993131 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1236828107 0 0) 0x7f77500d0370 con 0x7f77500d14b0 2024-05-20T18:34:32.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.108+0000 7f775560e640 1 --2- 172.21.15.5:0/342993131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77500b4010 0x7f77500cfb00 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f77400097a0 tx=0x7f774002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02b04b16fd9f84c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:32.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.108+0000 7f7754e0d640 1 -- 172.21.15.5:0/342993131 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7740037070 con 0x7f77500b4010 2024-05-20T18:34:32.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.109+0000 7f7754e0d640 1 -- 172.21.15.5:0/342993131 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7740032440 con 0x7f77500b4010 2024-05-20T18:34:32.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.109+0000 7f7754e0d640 1 -- 172.21.15.5:0/342993131 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7740032810 con 0x7f77500b4010 2024-05-20T18:34:32.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.109+0000 7f7758073640 1 -- 172.21.15.5:0/342993131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77500b4010 msgr2=0x7f77500cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:32.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.109+0000 7f7758073640 1 --2- 172.21.15.5:0/342993131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77500b4010 0x7f77500cfb00 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f77400097a0 tx=0x7f774002a180 comp rx=0 tx=0).stop 2024-05-20T18:34:32.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.109+0000 7f7758073640 1 -- 172.21.15.5:0/342993131 shutdown_connections 2024-05-20T18:34:32.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.109+0000 7f7758073640 1 --2- 172.21.15.5:0/342993131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77500b4010 0x7f77500cfb00 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.109+0000 7f7758073640 1 --2- 172.21.15.5:0/342993131 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7750163a10 0x7f77500d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.109+0000 7f7758073640 1 -- 172.21.15.5:0/342993131 >> 172.21.15.5:0/342993131 conn(0x7f775006d240 msgr2=0x7f775006d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:32.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.109+0000 7f7758073640 1 -- 172.21.15.5:0/342993131 shutdown_connections 2024-05-20T18:34:32.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.109+0000 7f7758073640 1 -- 172.21.15.5:0/342993131 wait complete. 2024-05-20T18:34:32.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.111+0000 7f7758073640 1 Processor -- start 2024-05-20T18:34:32.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.111+0000 7f7758073640 1 -- start start 2024-05-20T18:34:32.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.112+0000 7f7758073640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77500b4010 0x7f77500ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.112+0000 7f7758073640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77500d14b0 0x7f77501e98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.112+0000 7f7758073640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7750163a10 0x7f77501e9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.112+0000 7f7758073640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f77500b8210 con 0x7f7750163a10 2024-05-20T18:34:32.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.112+0000 7f7758073640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f77500d0040 con 0x7f77500b4010 2024-05-20T18:34:32.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.112+0000 7f7758073640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f77500b8390 con 0x7f77500d14b0 2024-05-20T18:34:32.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.113+0000 7f7756610640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7750163a10 0x7f77501e9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.113+0000 7f7756610640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7750163a10 0x7f77501e9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56646/0 (socket says 172.21.15.5:56646) 2024-05-20T18:34:32.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.113+0000 7f7756610640 1 -- 172.21.15.5:0/1463432227 learned_addr learned my addr 172.21.15.5:0/1463432227 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:32.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.113+0000 7f7755e0f640 1 --2- 172.21.15.5:0/1463432227 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77500b4010 0x7f77500ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.113+0000 7f775560e640 1 --2- 172.21.15.5:0/1463432227 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77500d14b0 0x7f77501e98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.113+0000 7f7756610640 1 -- 172.21.15.5:0/1463432227 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77500d14b0 msgr2=0x7f77501e98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.113+0000 7f7756610640 1 --2- 172.21.15.5:0/1463432227 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77500d14b0 0x7f77501e98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.113+0000 7f7756610640 1 -- 172.21.15.5:0/1463432227 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77500b4010 msgr2=0x7f77500ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.113+0000 7f7756610640 1 --2- 172.21.15.5:0/1463432227 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77500b4010 0x7f77500ce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.113+0000 7f7756610640 1 -- 172.21.15.5:0/1463432227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77500ceaf0 con 0x7f7750163a10 2024-05-20T18:34:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.113+0000 7f7756610640 1 --2- 172.21.15.5:0/1463432227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7750163a10 0x7f77501e9e00 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f774800c700 tx=0x7f774800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.113+0000 7f77467fc640 1 -- 172.21.15.5:0/1463432227 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7748018070 con 0x7f7750163a10 2024-05-20T18:34:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.113+0000 7f77467fc640 1 -- 172.21.15.5:0/1463432227 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f774800ce10 con 0x7f7750163a10 2024-05-20T18:34:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.113+0000 7f7758073640 1 -- 172.21.15.5:0/1463432227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77501ea690 con 0x7f7750163a10 2024-05-20T18:34:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.113+0000 7f77467fc640 1 -- 172.21.15.5:0/1463432227 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774800a820 con 0x7f7750163a10 2024-05-20T18:34:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.113+0000 7f7758073640 1 -- 172.21.15.5:0/1463432227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7750144ed0 con 0x7f7750163a10 2024-05-20T18:34:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.114+0000 7f7758073640 1 -- 172.21.15.5:0/1463432227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7750158110 con 0x7f7750163a10 2024-05-20T18:34:32.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.115+0000 7f77467fc640 1 -- 172.21.15.5:0/1463432227 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f774800a9c0 con 0x7f7750163a10 2024-05-20T18:34:32.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.115+0000 7f77467fc640 1 --2- 172.21.15.5:0/1463432227 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f773406be60 0x7f773406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.116+0000 7f77467fc640 1 -- 172.21.15.5:0/1463432227 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7f774808a630 con 0x7f7750163a10 2024-05-20T18:34:32.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.116+0000 7f7755e0f640 1 --2- 172.21.15.5:0/1463432227 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f773406be60 0x7f773406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.116+0000 7f7755e0f640 1 --2- 172.21.15.5:0/1463432227 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f773406be60 0x7f773406e370 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f77500d3020 tx=0x7f774c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:32.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.120+0000 7f77467fc640 1 -- 172.21.15.5:0/1463432227 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7748058590 con 0x7f7750163a10 2024-05-20T18:34:32.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.343+0000 7f7758073640 1 -- 172.21.15.5:0/1463432227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7750075b00 con 0x7f7750163a10 2024-05-20T18:34:32.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.343+0000 7f77467fc640 1 -- 172.21.15.5:0/1463432227 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5173 (secure 0 0 0) 0x7f774805c890 con 0x7f7750163a10 2024-05-20T18:34:32.344 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:34:32.345 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/337248865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":337248865},{"type":"v1","addr":"172.21.15.63:6834","nonce":337248865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3705772553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3705772553},{"type":"v1","addr":"172.21.15.63:6835","nonce":3705772553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":7434,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/776188138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":776188138},{"type":"v1","addr":"172.21.15.5:6835","nonce":776188138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:33:12.457716+0000","modified":"2024-05-20T18:33:34.256059+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":10,"addr":"172.21.15.5:6837/3556492262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3556492262},{"type":"v1","addr":"172.21.15.5:6837","nonce":3556492262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7437,"qdb_cluster":[7437]},"id":6}]} 2024-05-20T18:34:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 67 2024-05-20T18:34:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.345+0000 7f7758073640 1 -- 172.21.15.5:0/1463432227 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f773406be60 msgr2=0x7f773406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:32.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.345+0000 7f7758073640 1 --2- 172.21.15.5:0/1463432227 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f773406be60 0x7f773406e370 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f77500d3020 tx=0x7f774c009190 comp rx=0 tx=0).stop 2024-05-20T18:34:32.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.345+0000 7f7758073640 1 -- 172.21.15.5:0/1463432227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7750163a10 msgr2=0x7f77501e9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:32.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.345+0000 7f7758073640 1 --2- 172.21.15.5:0/1463432227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7750163a10 0x7f77501e9e00 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f774800c700 tx=0x7f774800cbc0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.345+0000 7f7758073640 1 -- 172.21.15.5:0/1463432227 shutdown_connections 2024-05-20T18:34:32.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.345+0000 7f7758073640 1 --2- 172.21.15.5:0/1463432227 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f773406be60 0x7f773406e370 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.345+0000 7f7758073640 1 --2- 172.21.15.5:0/1463432227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7750163a10 0x7f77501e9e00 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.345+0000 7f7758073640 1 --2- 172.21.15.5:0/1463432227 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77500d14b0 0x7f77501e98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.345+0000 7f7758073640 1 --2- 172.21.15.5:0/1463432227 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77500b4010 0x7f77500ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.345+0000 7f7758073640 1 -- 172.21.15.5:0/1463432227 >> 172.21.15.5:0/1463432227 conn(0x7f775006d240 msgr2=0x7f77500d4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:32.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.346+0000 7f7758073640 1 -- 172.21.15.5:0/1463432227 shutdown_connections 2024-05-20T18:34:32.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.346+0000 7f7758073640 1 -- 172.21.15.5:0/1463432227 wait complete. 2024-05-20T18:34:32.358 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-05-20T18:34:32.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.474+0000 7fcd58565640 1 Processor -- start 2024-05-20T18:34:32.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.474+0000 7fcd58565640 1 -- start start 2024-05-20T18:34:32.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.475+0000 7fcd58565640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd5015d530 0x7fcd5015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.475+0000 7fcd58565640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd500cd580 con 0x7fcd50070850 2024-05-20T18:34:32.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.475+0000 7fcd58565640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd500cd860 con 0x7fcd500b1700 2024-05-20T18:34:32.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.475+0000 7fcd58565640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcd500ce190 con 0x7fcd5015d530 2024-05-20T18:34:32.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.475+0000 7fcd56b02640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fcd50070850 0x7fcd50070c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:49360/0 (socket says 172.21.15.5:49360) 2024-05-20T18:34:32.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.475+0000 7fcd56b02640 1 -- 172.21.15.5:0/34861600 learned_addr learned my addr 172.21.15.5:0/34861600 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:34:32.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.476+0000 7fcd56301640 1 --2- 172.21.15.5:0/34861600 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd5015d530 0x7fcd5015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.476+0000 7fcd552ff640 1 -- 172.21.15.5:0/34861600 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1789171767 0 0) 0x7fcd500cd580 con 0x7fcd50070850 2024-05-20T18:34:32.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.476+0000 7fcd552ff640 1 -- 172.21.15.5:0/34861600 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd3c003520 con 0x7fcd50070850 2024-05-20T18:34:32.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.476+0000 7fcd552ff640 1 -- 172.21.15.5:0/34861600 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3644991111 0 0) 0x7fcd500cd860 con 0x7fcd500b1700 2024-05-20T18:34:32.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.476+0000 7fcd552ff640 1 -- 172.21.15.5:0/34861600 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd500cd580 con 0x7fcd500b1700 2024-05-20T18:34:32.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.476+0000 7fcd56301640 1 -- 172.21.15.5:0/34861600 >> v1:172.21.15.63:6789/0 conn(0x7fcd500b1700 legacy=0x7fcd500b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:32.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.476+0000 7fcd56301640 1 -- 172.21.15.5:0/34861600 >> v1:172.21.15.5:6789/0 conn(0x7fcd50070850 legacy=0x7fcd50070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:32.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.476+0000 7fcd56301640 1 -- 172.21.15.5:0/34861600 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd500cdb40 con 0x7fcd5015d530 2024-05-20T18:34:32.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.477+0000 7fcd56301640 1 --2- 172.21.15.5:0/34861600 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd5015d530 0x7fcd5015aaa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c0097d0 tx=0x7fcd4c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76ac04fbd44826d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:32.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.477+0000 7fcd552ff640 1 -- 172.21.15.5:0/34861600 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c037070 con 0x7fcd5015d530 2024-05-20T18:34:32.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.477+0000 7fcd552ff640 1 -- 172.21.15.5:0/34861600 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd4c032440 con 0x7fcd5015d530 2024-05-20T18:34:32.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.477+0000 7fcd552ff640 1 -- 172.21.15.5:0/34861600 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c032810 con 0x7fcd5015d530 2024-05-20T18:34:32.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.478+0000 7fcd58565640 1 -- 172.21.15.5:0/34861600 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd5015d530 msgr2=0x7fcd5015aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:32.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.478+0000 7fcd58565640 1 --2- 172.21.15.5:0/34861600 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd5015d530 0x7fcd5015aaa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c0097d0 tx=0x7fcd4c02a210 comp rx=0 tx=0).stop 2024-05-20T18:34:32.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.478+0000 7fcd58565640 1 -- 172.21.15.5:0/34861600 shutdown_connections 2024-05-20T18:34:32.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.478+0000 7fcd58565640 1 --2- 172.21.15.5:0/34861600 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd5015d530 0x7fcd5015aaa0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.478+0000 7fcd58565640 1 -- 172.21.15.5:0/34861600 >> 172.21.15.5:0/34861600 conn(0x7fcd500d29d0 msgr2=0x7fcd500d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:32.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.478+0000 7fcd58565640 1 -- 172.21.15.5:0/34861600 shutdown_connections 2024-05-20T18:34:32.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.478+0000 7fcd58565640 1 -- 172.21.15.5:0/34861600 wait complete. 2024-05-20T18:34:32.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.480+0000 7fcd58565640 1 Processor -- start 2024-05-20T18:34:32.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.480+0000 7fcd58565640 1 -- start start 2024-05-20T18:34:32.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.481+0000 7fcd58565640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd50070850 0x7fcd5006e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.481+0000 7fcd58565640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd500b1700 0x7fcd500c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.481+0000 7fcd58565640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd5015d530 0x7fcd5006d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.481+0000 7fcd58565640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcd50071370 con 0x7fcd500b1700 2024-05-20T18:34:32.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.481+0000 7fcd58565640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcd500cd230 con 0x7fcd5015d530 2024-05-20T18:34:32.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.481+0000 7fcd58565640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcd500714f0 con 0x7fcd50070850 2024-05-20T18:34:32.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.481+0000 7fcd55b00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd500b1700 0x7fcd500c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.481+0000 7fcd55b00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd500b1700 0x7fcd500c8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56654/0 (socket says 172.21.15.5:56654) 2024-05-20T18:34:32.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.481+0000 7fcd55b00640 1 -- 172.21.15.5:0/2027613587 learned_addr learned my addr 172.21.15.5:0/2027613587 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:32.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.481+0000 7fcd56301640 1 --2- 172.21.15.5:0/2027613587 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd50070850 0x7fcd5006e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.481+0000 7fcd56b02640 1 --2- 172.21.15.5:0/2027613587 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd5015d530 0x7fcd5006d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.481+0000 7fcd55b00640 1 -- 172.21.15.5:0/2027613587 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd50070850 msgr2=0x7fcd5006e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:32.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.481+0000 7fcd55b00640 1 --2- 172.21.15.5:0/2027613587 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd50070850 0x7fcd5006e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.481+0000 7fcd55b00640 1 -- 172.21.15.5:0/2027613587 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd5015d530 msgr2=0x7fcd5006d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:32.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.481+0000 7fcd55b00640 1 --2- 172.21.15.5:0/2027613587 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd5015d530 0x7fcd5006d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.481+0000 7fcd55b00640 1 -- 172.21.15.5:0/2027613587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd500c7e80 con 0x7fcd500b1700 2024-05-20T18:34:32.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.482+0000 7fcd56301640 1 --2- 172.21.15.5:0/2027613587 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd50070850 0x7fcd5006e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:34:32.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.482+0000 7fcd56b02640 1 --2- 172.21.15.5:0/2027613587 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd5015d530 0x7fcd5006d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:34:32.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.482+0000 7fcd55b00640 1 --2- 172.21.15.5:0/2027613587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd500b1700 0x7fcd500c8d60 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fcd4000c3c0 tx=0x7fcd4000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:32.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.482+0000 7fcd46ffd640 1 -- 172.21.15.5:0/2027613587 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd40017070 con 0x7fcd500b1700 2024-05-20T18:34:32.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.482+0000 7fcd46ffd640 1 -- 172.21.15.5:0/2027613587 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd400052f0 con 0x7fcd500b1700 2024-05-20T18:34:32.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.482+0000 7fcd58565640 1 -- 172.21.15.5:0/2027613587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd5006c760 con 0x7fcd500b1700 2024-05-20T18:34:32.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.482+0000 7fcd46ffd640 1 -- 172.21.15.5:0/2027613587 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd40003830 con 0x7fcd500b1700 2024-05-20T18:34:32.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.482+0000 7fcd58565640 1 -- 172.21.15.5:0/2027613587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd5006cd20 con 0x7fcd500b1700 2024-05-20T18:34:32.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.484+0000 7fcd46ffd640 1 -- 172.21.15.5:0/2027613587 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcd4001a070 con 0x7fcd500b1700 2024-05-20T18:34:32.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.484+0000 7fcd46ffd640 1 --2- 172.21.15.5:0/2027613587 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcd3406be60 0x7fcd3406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.484+0000 7fcd46ffd640 1 -- 172.21.15.5:0/2027613587 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7fcd4008a450 con 0x7fcd500b1700 2024-05-20T18:34:32.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.484+0000 7fcd56301640 1 --2- 172.21.15.5:0/2027613587 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcd3406be60 0x7fcd3406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.485+0000 7fcd56301640 1 --2- 172.21.15.5:0/2027613587 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcd3406be60 0x7fcd3406e370 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c034000 tx=0x7fcd4c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:32.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.486+0000 7fcd5f414740 1 Processor -- start 2024-05-20T18:34:32.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.486+0000 7fcd5f414740 1 -- start start 2024-05-20T18:34:32.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.487+0000 7fcd5f414740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b98a2cf350 0x55b98a69a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.487+0000 7fcd5f414740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b98a2a6490 0x55b98a6b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.487+0000 7fcd5f414740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b98a2ae670 0x55b98a6b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.487+0000 7fcd5f414740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b98a5d3a30 con 0x55b98a2a6490 2024-05-20T18:34:32.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.487+0000 7fcd5f414740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b98a571590 con 0x55b98a2cf350 2024-05-20T18:34:32.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.487+0000 7fcd5f414740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55b98a5cf9f0 con 0x55b98a2ae670 2024-05-20T18:34:32.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.487+0000 7fcd55b00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b98a2a6490 0x55b98a6b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.487+0000 7fcd55b00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b98a2a6490 0x55b98a6b53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56662/0 (socket says 172.21.15.5:56662) 2024-05-20T18:34:32.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.487+0000 7fcd55b00640 1 -- 172.21.15.5:0/530379797 learned_addr learned my addr 172.21.15.5:0/530379797 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:32.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.487+0000 7fcd56301640 1 --2- 172.21.15.5:0/530379797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b98a2cf350 0x55b98a69a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.487+0000 7fcd55b00640 1 -- 172.21.15.5:0/530379797 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b98a2ae670 msgr2=0x55b98a6b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:32.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.487+0000 7fcd55b00640 1 --2- 172.21.15.5:0/530379797 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b98a2ae670 0x55b98a6b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.487+0000 7fcd55b00640 1 -- 172.21.15.5:0/530379797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b98a2cf350 msgr2=0x55b98a69a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:32.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.487+0000 7fcd55b00640 1 --2- 172.21.15.5:0/530379797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b98a2cf350 0x55b98a69a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.487+0000 7fcd55b00640 1 -- 172.21.15.5:0/530379797 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b98a3b37c0 con 0x55b98a2a6490 2024-05-20T18:34:32.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.488+0000 7fcd55b00640 1 --2- 172.21.15.5:0/530379797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b98a2a6490 0x55b98a6b53a0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fcd40001eb0 tx=0x7fcd4005c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a86f64c330c5a354 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:32.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.488+0000 7fcd33fff640 1 -- 172.21.15.5:0/530379797 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd40054b30 con 0x55b98a2a6490 2024-05-20T18:34:32.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.488+0000 7fcd33fff640 1 -- 172.21.15.5:0/530379797 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd4008a0b0 con 0x55b98a2a6490 2024-05-20T18:34:32.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.488+0000 7fcd33fff640 1 -- 172.21.15.5:0/530379797 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd40054cd0 con 0x55b98a2a6490 2024-05-20T18:34:32.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.489+0000 7fcd5f414740 1 -- 172.21.15.5:0/530379797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b98a2a6490 msgr2=0x55b98a6b53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:32.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.489+0000 7fcd5f414740 1 --2- 172.21.15.5:0/530379797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b98a2a6490 0x55b98a6b53a0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fcd40001eb0 tx=0x7fcd4005c930 comp rx=0 tx=0).stop 2024-05-20T18:34:32.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.489+0000 7fcd5f414740 1 -- 172.21.15.5:0/530379797 shutdown_connections 2024-05-20T18:34:32.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.489+0000 7fcd5f414740 1 --2- 172.21.15.5:0/530379797 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b98a2ae670 0x55b98a6b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.489+0000 7fcd5f414740 1 --2- 172.21.15.5:0/530379797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b98a2a6490 0x55b98a6b53a0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.489+0000 7fcd5f414740 1 --2- 172.21.15.5:0/530379797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b98a2cf350 0x55b98a69a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.489+0000 7fcd5f414740 1 -- 172.21.15.5:0/530379797 >> 172.21.15.5:0/530379797 conn(0x55b98a2c34a0 msgr2=0x55b98a66fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:32.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.489+0000 7fcd5f414740 1 -- 172.21.15.5:0/530379797 shutdown_connections 2024-05-20T18:34:32.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.489+0000 7fcd5f414740 1 -- 172.21.15.5:0/530379797 wait complete. 2024-05-20T18:34:32.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.490+0000 7fcd5f414740 1 -- start start 2024-05-20T18:34:32.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.490+0000 7fcd5f414740 1 Processor -- start 2024-05-20T18:34:32.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.491+0000 7fcd5f414740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b98a2a6490 0x55b98a696190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.491+0000 7fcd5f414740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b98a2ae670 0x55b98a6b95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.491+0000 7fcd5f414740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b98a2cf350 0x55b98a6b9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.491+0000 7fcd5f414740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b98a56f060 con 0x55b98a2cf350 2024-05-20T18:34:32.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.491+0000 7fcd5f414740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b98a6c5a70 con 0x55b98a2a6490 2024-05-20T18:34:32.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.491+0000 7fcd5f414740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55b98a69b610 con 0x55b98a2ae670 2024-05-20T18:34:32.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.491+0000 7fcd5f414740 10 client.0 _ll_register_callbacks cb 0x55b98a593dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:34:32.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.491+0000 7fcd56b02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b98a2cf350 0x55b98a6b9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.491+0000 7fcd56b02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b98a2cf350 0x55b98a6b9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56670/0 (socket says 172.21.15.5:56670) 2024-05-20T18:34:32.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.491+0000 7fcd56b02640 1 -- 172.21.15.5:0/3526846753 learned_addr learned my addr 172.21.15.5:0/3526846753 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:32.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.492+0000 7fcd56301640 1 --2- 172.21.15.5:0/3526846753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b98a2a6490 0x55b98a696190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.492+0000 7fcd55b00640 1 --2- 172.21.15.5:0/3526846753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b98a2ae670 0x55b98a6b95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.492+0000 7fcd56b02640 1 -- 172.21.15.5:0/3526846753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b98a2ae670 msgr2=0x55b98a6b95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:32.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.492+0000 7fcd56b02640 1 --2- 172.21.15.5:0/3526846753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b98a2ae670 0x55b98a6b95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.492+0000 7fcd56b02640 1 -- 172.21.15.5:0/3526846753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b98a2a6490 msgr2=0x55b98a696190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:32.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.492+0000 7fcd56b02640 1 --2- 172.21.15.5:0/3526846753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b98a2a6490 0x55b98a696190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.492+0000 7fcd56b02640 1 -- 172.21.15.5:0/3526846753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b98a413f50 con 0x55b98a2cf350 2024-05-20T18:34:32.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.492+0000 7fcd56b02640 1 --2- 172.21.15.5:0/3526846753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b98a2cf350 0x55b98a6b9b30 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fcd4800d770 tx=0x7fcd4800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1986b1d94e548b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:32.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.492+0000 7fcd317fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:34:32.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.492+0000 7fcd317fa640 1 -- 172.21.15.5:0/3526846753 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd48005dd0 con 0x55b98a2cf350 2024-05-20T18:34:32.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.492+0000 7fcd5f414740 1 -- 172.21.15.5:0/3526846753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd4008a0b0 con 0x55b98a2cf350 2024-05-20T18:34:32.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.492+0000 7fcd317fa640 1 -- 172.21.15.5:0/3526846753 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7fcd4800b6b0 con 0x55b98a2cf350 2024-05-20T18:34:32.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.492+0000 7fcd317fa640 1 -- 172.21.15.5:0/3526846753 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fcd4801f050 con 0x55b98a2cf350 2024-05-20T18:34:32.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.492+0000 7fcd5f414740 10 client.7896 fetch_fsmap learned FSMap version 67 2024-05-20T18:34:32.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.492+0000 7fcd5f414740 1 -- 172.21.15.5:0/3526846753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55b98a2a70b0 con 0x55b98a2cf350 2024-05-20T18:34:32.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.492+0000 7fcd317fa640 1 -- 172.21.15.5:0/3526846753 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd4008a0b0 con 0x55b98a2cf350 2024-05-20T18:34:32.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.493+0000 7fcd317fa640 1 -- 172.21.15.5:0/3526846753 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7fcd48022020 con 0x55b98a2cf350 2024-05-20T18:34:32.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.493+0000 7fcd5f414740 10 client.7896 fetch_fsmap finished waiting for FSMap version 67 2024-05-20T18:34:32.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.493+0000 7fcd5f414740 10 client.7896 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-20T18:34:32.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.493+0000 7fcd5f414740 20 client.7896 populate_metadata read hostname 'smithi005' 2024-05-20T18:34:32.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.493+0000 7fcd5f414740 1 --2- 172.21.15.5:0/3526846753 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55b98a28e130 0x55b98a694c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.493+0000 7fcd5f414740 4 client.7896 mds_command: new command op to 7437 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:34:32.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.493+0000 7fcd5f414740 1 -- 172.21.15.5:0/3526846753 --> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b98a5e1160 con 0x55b98a28e130 2024-05-20T18:34:32.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.493+0000 7fcd56301640 1 --2- 172.21.15.5:0/3526846753 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55b98a28e130 0x55b98a694c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.493+0000 7fcd317fa640 10 client.7896 ms_handle_connect on v2:172.21.15.5:6836/3556492262 2024-05-20T18:34:32.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.493+0000 7fcd56301640 1 --2- 172.21.15.5:0/3526846753 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55b98a28e130 0x55b98a694c20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46c407a74495fa75 server_cookie=1f366b245e8f07b8 in_seq=0 out_seq=0 2024-05-20T18:34:32.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.494+0000 7fcd317fa640 1 -- 172.21.15.5:0/3526846753 <== mds.0 v2:172.21.15.5:6836/3556492262 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b98a5e1160 con 0x55b98a28e130 2024-05-20T18:34:32.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.494+0000 7fcd317fa640 10 client.7896 handle_command_reply: tid=0 2024-05-20T18:34:32.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.494+0000 7fcd5f414740 1 client.7896 shutdown 2024-05-20T18:34:32.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.494+0000 7fcd5f414740 1 -- 172.21.15.5:0/3526846753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b98a2cf350 msgr2=0x55b98a6b9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:32.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.494+0000 7fcd5f414740 1 --2- 172.21.15.5:0/3526846753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b98a2cf350 0x55b98a6b9b30 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fcd4800d770 tx=0x7fcd4800dc30 comp rx=0 tx=0).stop 2024-05-20T18:34:32.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.494+0000 7fcd5f414740 1 -- 172.21.15.5:0/3526846753 shutdown_connections 2024-05-20T18:34:32.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.494+0000 7fcd5f414740 1 --2- 172.21.15.5:0/3526846753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b98a2cf350 0x55b98a6b9b30 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.494+0000 7fcd5f414740 1 --2- 172.21.15.5:0/3526846753 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55b98a28e130 0x55b98a694c20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.494+0000 7fcd5f414740 1 --2- 172.21.15.5:0/3526846753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b98a2ae670 0x55b98a6b95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.494+0000 7fcd5f414740 1 --2- 172.21.15.5:0/3526846753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b98a2a6490 0x55b98a696190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.494+0000 7fcd317fa640 0 client.7896 ms_handle_reset on v2:172.21.15.5:6836/3556492262 2024-05-20T18:34:32.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.494+0000 7fcd5f414740 1 -- 172.21.15.5:0/3526846753 >> 172.21.15.5:0/3526846753 conn(0x55b98a2c34a0 msgr2=0x55b98a66fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:32.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.494+0000 7fcd5f414740 1 -- 172.21.15.5:0/3526846753 shutdown_connections 2024-05-20T18:34:32.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.494+0000 7fcd5f414740 1 -- 172.21.15.5:0/3526846753 wait complete. 2024-05-20T18:34:32.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.494+0000 7fcd5f414740 20 client.7896 trim_cache size 0 max 16384 2024-05-20T18:34:32.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.508+0000 7fcd5f414740 1 Processor -- start 2024-05-20T18:34:32.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.509+0000 7fcd5f414740 1 -- start start 2024-05-20T18:34:32.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.510+0000 7fcd5f414740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b98a2a6490 0x55b98a6b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.510+0000 7fcd5f414740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b98a2ae670 0x55b98a6b8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.510+0000 7fcd5f414740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b98a2cf350 0x55b98a696f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.510+0000 7fcd5f414740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcd50201580 con 0x55b98a2ae670 2024-05-20T18:34:32.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.510+0000 7fcd5f414740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcd501f9590 con 0x55b98a2cf350 2024-05-20T18:34:32.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.510+0000 7fcd5f414740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcd50200f90 con 0x55b98a2a6490 2024-05-20T18:34:32.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.510+0000 7fcd55b00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b98a2ae670 0x55b98a6b8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.510+0000 7fcd55b00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b98a2ae670 0x55b98a6b8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56684/0 (socket says 172.21.15.5:56684) 2024-05-20T18:34:32.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.510+0000 7fcd55b00640 1 -- 172.21.15.5:0/3386621550 learned_addr learned my addr 172.21.15.5:0/3386621550 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:32.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.510+0000 7fcd56b02640 1 --2- 172.21.15.5:0/3386621550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b98a2cf350 0x55b98a696f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.510+0000 7fcd56301640 1 --2- 172.21.15.5:0/3386621550 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b98a2a6490 0x55b98a6b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.510+0000 7fcd55b00640 1 -- 172.21.15.5:0/3386621550 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b98a2a6490 msgr2=0x55b98a6b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:32.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.510+0000 7fcd55b00640 1 --2- 172.21.15.5:0/3386621550 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b98a2a6490 0x55b98a6b83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.510+0000 7fcd55b00640 1 -- 172.21.15.5:0/3386621550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b98a2cf350 msgr2=0x55b98a696f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:32.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.510+0000 7fcd55b00640 1 --2- 172.21.15.5:0/3386621550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b98a2cf350 0x55b98a696f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.510+0000 7fcd55b00640 1 -- 172.21.15.5:0/3386621550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b98a66dd20 con 0x55b98a2ae670 2024-05-20T18:34:32.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.510+0000 7fcd56b02640 1 --2- 172.21.15.5:0/3386621550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b98a2cf350 0x55b98a696f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:34:32.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.510+0000 7fcd55b00640 1 --2- 172.21.15.5:0/3386621550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b98a2ae670 0x55b98a6b8910 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fcd40060b70 tx=0x7fcd4001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8639499c1d79370 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:32.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.510+0000 7fcd337fe640 1 -- 172.21.15.5:0/3386621550 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd40054b30 con 0x55b98a2ae670 2024-05-20T18:34:32.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.511+0000 7fcd337fe640 1 -- 172.21.15.5:0/3386621550 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd4008a0b0 con 0x55b98a2ae670 2024-05-20T18:34:32.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.511+0000 7fcd337fe640 1 -- 172.21.15.5:0/3386621550 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd40054cd0 con 0x55b98a2ae670 2024-05-20T18:34:32.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.511+0000 7fcd5f414740 1 -- 172.21.15.5:0/3386621550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b98a2ae670 msgr2=0x55b98a6b8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:32.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.511+0000 7fcd5f414740 1 --2- 172.21.15.5:0/3386621550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b98a2ae670 0x55b98a6b8910 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fcd40060b70 tx=0x7fcd4001d000 comp rx=0 tx=0).stop 2024-05-20T18:34:32.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.511+0000 7fcd5f414740 1 -- 172.21.15.5:0/3386621550 shutdown_connections 2024-05-20T18:34:32.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.511+0000 7fcd5f414740 1 --2- 172.21.15.5:0/3386621550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b98a2cf350 0x55b98a696f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.511+0000 7fcd5f414740 1 --2- 172.21.15.5:0/3386621550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b98a2ae670 0x55b98a6b8910 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.511+0000 7fcd5f414740 1 --2- 172.21.15.5:0/3386621550 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b98a2a6490 0x55b98a6b83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.511+0000 7fcd5f414740 1 -- 172.21.15.5:0/3386621550 >> 172.21.15.5:0/3386621550 conn(0x55b98a2c34a0 msgr2=0x55b98a693530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:32.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.511+0000 7fcd5f414740 1 -- 172.21.15.5:0/3386621550 shutdown_connections 2024-05-20T18:34:32.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.511+0000 7fcd5f414740 1 -- 172.21.15.5:0/3386621550 wait complete. 2024-05-20T18:34:32.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.512+0000 7fcd5f414740 1 -- start start 2024-05-20T18:34:32.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.512+0000 7fcd5f414740 1 Processor -- start 2024-05-20T18:34:32.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.514+0000 7fcd5f414740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b98a2a6490 0x55b98a6b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.514+0000 7fcd5f414740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b98a2ae670 0x55b98a69d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.514+0000 7fcd5f414740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b98a2cf350 0x55b98a669d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.514+0000 7fcd5f414740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b98a6c4ba0 con 0x55b98a2cf350 2024-05-20T18:34:32.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.514+0000 7fcd5f414740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b98a693cc0 con 0x55b98a2a6490 2024-05-20T18:34:32.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.514+0000 7fcd5f414740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55b98a6c51c0 con 0x55b98a2ae670 2024-05-20T18:34:32.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.514+0000 7fcd5f414740 10 client.0 _ll_register_callbacks cb 0x7fcd48001880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:34:32.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.514+0000 7fcd56b02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b98a2cf350 0x55b98a669d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.514+0000 7fcd56b02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b98a2cf350 0x55b98a669d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56696/0 (socket says 172.21.15.5:56696) 2024-05-20T18:34:32.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.514+0000 7fcd56b02640 1 -- 172.21.15.5:0/1341421052 learned_addr learned my addr 172.21.15.5:0/1341421052 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:32.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.514+0000 7fcd55b00640 1 --2- 172.21.15.5:0/1341421052 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b98a2ae670 0x55b98a69d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.514+0000 7fcd56301640 1 --2- 172.21.15.5:0/1341421052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b98a2a6490 0x55b98a6b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.514+0000 7fcd56b02640 1 -- 172.21.15.5:0/1341421052 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b98a2ae670 msgr2=0x55b98a69d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:32.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.514+0000 7fcd56b02640 1 --2- 172.21.15.5:0/1341421052 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b98a2ae670 0x55b98a69d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.514+0000 7fcd56b02640 1 -- 172.21.15.5:0/1341421052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b98a2a6490 msgr2=0x55b98a6b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:32.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.514+0000 7fcd56b02640 1 --2- 172.21.15.5:0/1341421052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b98a2a6490 0x55b98a6b4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.514+0000 7fcd56b02640 1 -- 172.21.15.5:0/1341421052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b98a69d580 con 0x55b98a2cf350 2024-05-20T18:34:32.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.514+0000 7fcd56b02640 1 --2- 172.21.15.5:0/1341421052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b98a2cf350 0x55b98a669d30 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x55b98a69d010 tx=0x7fcd480014d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b2857be7ebc51a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:32.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.514+0000 7fcd317fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:34:32.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.515+0000 7fcd317fa640 1 -- 172.21.15.5:0/1341421052 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4801a350 con 0x55b98a2cf350 2024-05-20T18:34:32.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.515+0000 7fcd317fa640 1 -- 172.21.15.5:0/1341421052 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7fcd4802cd10 con 0x55b98a2cf350 2024-05-20T18:34:32.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.515+0000 7fcd5f414740 1 -- 172.21.15.5:0/1341421052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd4008a0b0 con 0x55b98a2cf350 2024-05-20T18:34:32.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.515+0000 7fcd317fa640 1 -- 172.21.15.5:0/1341421052 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fcd4801a4f0 con 0x55b98a2cf350 2024-05-20T18:34:32.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.515+0000 7fcd5f414740 10 client.7902 fetch_fsmap learned FSMap version 67 2024-05-20T18:34:32.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.515+0000 7fcd5f414740 1 -- 172.21.15.5:0/1341421052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55b98a69d790 con 0x55b98a2cf350 2024-05-20T18:34:32.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.515+0000 7fcd317fa640 1 -- 172.21.15.5:0/1341421052 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd4008a0b0 con 0x55b98a2cf350 2024-05-20T18:34:32.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.515+0000 7fcd317fa640 1 -- 172.21.15.5:0/1341421052 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7fcd48022050 con 0x55b98a2cf350 2024-05-20T18:34:32.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.515+0000 7fcd5f414740 10 client.7902 fetch_fsmap finished waiting for FSMap version 67 2024-05-20T18:34:32.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.515+0000 7fcd5f414740 10 client.7902 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-20T18:34:32.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.515+0000 7fcd5f414740 20 client.7902 populate_metadata read hostname 'smithi005' 2024-05-20T18:34:32.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.515+0000 7fcd5f414740 1 --2- 172.21.15.5:0/1341421052 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55b98a731ce0 0x55b98a692270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.515+0000 7fcd5f414740 4 client.7902 mds_command: new command op to 7437 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-20T18:34:32.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.515+0000 7fcd5f414740 1 -- 172.21.15.5:0/1341421052 --> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55b98a5db4b0 con 0x55b98a731ce0 2024-05-20T18:34:32.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.515+0000 7fcd56301640 1 --2- 172.21.15.5:0/1341421052 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55b98a731ce0 0x55b98a692270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.516+0000 7fcd317fa640 10 client.7902 ms_handle_connect on v2:172.21.15.5:6836/3556492262 2024-05-20T18:34:32.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.516+0000 7fcd56301640 1 --2- 172.21.15.5:0/1341421052 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55b98a731ce0 0x55b98a692270 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=712091d65e0ca7ca server_cookie=942648a76a411a6d in_seq=0 out_seq=0 2024-05-20T18:34:32.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.516+0000 7fcd317fa640 1 -- 172.21.15.5:0/1341421052 <== mds.0 v2:172.21.15.5:6836/3556492262 1 ==== command_reply(tid 0: 0 ) ==== 8+0+144 (crc 0 0 0) 0x7fcd4c0550a0 con 0x55b98a731ce0 2024-05-20T18:34:32.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.516+0000 7fcd317fa640 10 client.7902 handle_command_reply: tid=0 2024-05-20T18:34:32.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.516+0000 7fcd5f414740 1 client.7902 shutdown 2024-05-20T18:34:32.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.516+0000 7fcd5f414740 1 -- 172.21.15.5:0/1341421052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b98a2cf350 msgr2=0x55b98a669d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:32.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.516+0000 7fcd5f414740 1 --2- 172.21.15.5:0/1341421052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b98a2cf350 0x55b98a669d30 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x55b98a69d010 tx=0x7fcd480014d0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.516+0000 7fcd5f414740 1 -- 172.21.15.5:0/1341421052 shutdown_connections 2024-05-20T18:34:32.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.516+0000 7fcd5f414740 1 --2- 172.21.15.5:0/1341421052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b98a2cf350 0x55b98a669d30 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.516+0000 7fcd5f414740 1 --2- 172.21.15.5:0/1341421052 >> [v2:172.21.15.5:6836/3556492262,v1:172.21.15.5:6837/3556492262] conn(0x55b98a731ce0 0x55b98a692270 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.516+0000 7fcd5f414740 1 --2- 172.21.15.5:0/1341421052 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b98a2ae670 0x55b98a69d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.516+0000 7fcd5f414740 1 --2- 172.21.15.5:0/1341421052 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b98a2a6490 0x55b98a6b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.516+0000 7fcd5f414740 1 -- 172.21.15.5:0/1341421052 >> 172.21.15.5:0/1341421052 conn(0x55b98a2c34a0 msgr2=0x55b98a6afee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:32.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.516+0000 7fcd317fa640 0 client.7902 ms_handle_reset on v2:172.21.15.5:6836/3556492262 2024-05-20T18:34:32.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.516+0000 7fcd5f414740 1 -- 172.21.15.5:0/1341421052 shutdown_connections 2024-05-20T18:34:32.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.516+0000 7fcd5f414740 1 -- 172.21.15.5:0/1341421052 wait complete. 2024-05-20T18:34:32.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.516+0000 7fcd5f414740 20 client.7902 trim_cache size 0 max 16384 2024-05-20T18:34:32.523 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:34:32.523 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-05-20T18:34:32.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:34:32.524 INFO:teuthology.orchestra.run.smithi005.stdout: "damage_type": "backtrace", 2024-05-20T18:34:32.524 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 3953132379, 2024-05-20T18:34:32.524 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 1099511627777, 2024-05-20T18:34:32.524 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/testdir/hardlink" 2024-05-20T18:34:32.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:34:32.524 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-05-20T18:34:32.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.517+0000 7fcd58565640 1 -- 172.21.15.5:0/2027613587 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcd3406be60 msgr2=0x7fcd3406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:32.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.517+0000 7fcd58565640 1 --2- 172.21.15.5:0/2027613587 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcd3406be60 0x7fcd3406e370 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c034000 tx=0x7fcd4c02e000 comp rx=0 tx=0).stop 2024-05-20T18:34:32.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.517+0000 7fcd58565640 1 -- 172.21.15.5:0/2027613587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd500b1700 msgr2=0x7fcd500c8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:32.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.517+0000 7fcd58565640 1 --2- 172.21.15.5:0/2027613587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd500b1700 0x7fcd500c8d60 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fcd4000c3c0 tx=0x7fcd4000c880 comp rx=0 tx=0).stop 2024-05-20T18:34:32.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.517+0000 7fcd58565640 1 -- 172.21.15.5:0/2027613587 shutdown_connections 2024-05-20T18:34:32.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.517+0000 7fcd58565640 1 --2- 172.21.15.5:0/2027613587 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcd3406be60 0x7fcd3406e370 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.517+0000 7fcd58565640 1 --2- 172.21.15.5:0/2027613587 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd5015d530 0x7fcd5006d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.517+0000 7fcd58565640 1 --2- 172.21.15.5:0/2027613587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd500b1700 0x7fcd500c8d60 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.517+0000 7fcd58565640 1 --2- 172.21.15.5:0/2027613587 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd50070850 0x7fcd5006e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.517+0000 7fcd58565640 1 -- 172.21.15.5:0/2027613587 >> 172.21.15.5:0/2027613587 conn(0x7fcd500d29d0 msgr2=0x7fcd5015e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:32.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.517+0000 7fcd58565640 1 -- 172.21.15.5:0/2027613587 shutdown_connections 2024-05-20T18:34:32.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.517+0000 7fcd58565640 1 -- 172.21.15.5:0/2027613587 wait complete. 2024-05-20T18:34:32.529 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:34:32.529 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:34:32.545 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:34:32.546 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:34:32.602 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:34:32.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.757+0000 7fd2d6606640 1 Processor -- start 2024-05-20T18:34:32.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.757+0000 7fd2d6606640 1 -- start start 2024-05-20T18:34:32.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.758+0000 7fd2d6606640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd2d00b47b0 0x7fd2d00afad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.758+0000 7fd2d6606640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd2d015d060 0x7fd2d00b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.758+0000 7fd2d6606640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2d00b5720 0x7fd2d015fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.758+0000 7fd2d6606640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fd2d00b5b30 con 0x7fd2d00b47b0 2024-05-20T18:34:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.758+0000 7fd2d6606640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd2d00b39f0 con 0x7fd2d015d060 2024-05-20T18:34:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.758+0000 7fd2d6606640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd2d00b3cd0 con 0x7fd2d00b5720 2024-05-20T18:34:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.758+0000 7fd2cffff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd2d00b47b0 0x7fd2d00afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.758+0000 7fd2cffff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd2d00b47b0 0x7fd2d00afad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56704/0 (socket says 172.21.15.5:56704) 2024-05-20T18:34:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.758+0000 7fd2cffff640 1 -- 172.21.15.5:0/1031281892 learned_addr learned my addr 172.21.15.5:0/1031281892 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.758+0000 7fd2c7fff640 1 --2- 172.21.15.5:0/1031281892 >> v2:172.21.15.63:3300/0 conn(0x7fd2d015d060 0x7fd2d00b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.758+0000 7fd2d4ba3640 1 --2- 172.21.15.5:0/1031281892 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2d00b5720 0x7fd2d015fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.759+0000 7fd2cffff640 1 -- 172.21.15.5:0/1031281892 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2d00b5720 msgr2=0x7fd2d015fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.759+0000 7fd2cffff640 1 --2- 172.21.15.5:0/1031281892 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2d00b5720 0x7fd2d015fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.759+0000 7fd2cffff640 1 -- 172.21.15.5:0/1031281892 >> v2:172.21.15.63:3300/0 conn(0x7fd2d015d060 msgr2=0x7fd2d00b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:32.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.759+0000 7fd2cffff640 1 --2- 172.21.15.5:0/1031281892 >> v2:172.21.15.63:3300/0 conn(0x7fd2d015d060 0x7fd2d00b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.759+0000 7fd2cffff640 1 -- 172.21.15.5:0/1031281892 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d00b3fb0 con 0x7fd2d00b47b0 2024-05-20T18:34:32.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.759+0000 7fd2c7fff640 1 --2- 172.21.15.5:0/1031281892 >> v2:172.21.15.63:3300/0 conn(0x7fd2d015d060 0x7fd2d00b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:34:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.759+0000 7fd2cffff640 1 --2- 172.21.15.5:0/1031281892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d00b47b0 0x7fd2d00afad0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fd2d006b980 tx=0x7fd2c002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbdc971faf1bdd24 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.760+0000 7fd2cf7fe640 1 -- 172.21.15.5:0/1031281892 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0037070 con 0x7fd2d00b47b0 2024-05-20T18:34:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.760+0000 7fd2cf7fe640 1 -- 172.21.15.5:0/1031281892 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c0032440 con 0x7fd2d00b47b0 2024-05-20T18:34:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.760+0000 7fd2cf7fe640 1 -- 172.21.15.5:0/1031281892 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0032810 con 0x7fd2d00b47b0 2024-05-20T18:34:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.760+0000 7fd2d6606640 1 -- 172.21.15.5:0/1031281892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d00b47b0 msgr2=0x7fd2d00afad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.760+0000 7fd2d6606640 1 --2- 172.21.15.5:0/1031281892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d00b47b0 0x7fd2d00afad0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fd2d006b980 tx=0x7fd2c002a140 comp rx=0 tx=0).stop 2024-05-20T18:34:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.760+0000 7fd2d6606640 1 -- 172.21.15.5:0/1031281892 shutdown_connections 2024-05-20T18:34:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.760+0000 7fd2d6606640 1 --2- 172.21.15.5:0/1031281892 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2d00b5720 0x7fd2d015fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.760+0000 7fd2d6606640 1 --2- 172.21.15.5:0/1031281892 >> v2:172.21.15.63:3300/0 conn(0x7fd2d015d060 0x7fd2d00b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.760+0000 7fd2d6606640 1 --2- 172.21.15.5:0/1031281892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d00b47b0 0x7fd2d00afad0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.760+0000 7fd2d6606640 1 -- 172.21.15.5:0/1031281892 >> 172.21.15.5:0/1031281892 conn(0x7fd2d00cd2a0 msgr2=0x7fd2d00cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:32.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.760+0000 7fd2d6606640 1 -- 172.21.15.5:0/1031281892 shutdown_connections 2024-05-20T18:34:32.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.760+0000 7fd2d6606640 1 -- 172.21.15.5:0/1031281892 wait complete. 2024-05-20T18:34:32.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.762+0000 7fd2d6606640 1 Processor -- start 2024-05-20T18:34:32.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.762+0000 7fd2d6606640 1 -- start start 2024-05-20T18:34:32.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.763+0000 7fd2d6606640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2d00b5720 0x7fd2d014f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.763+0000 7fd2d6606640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2d015d060 0x7fd2d00b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.763+0000 7fd2d6606640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d0154d50 0x7fd2d00b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.763+0000 7fd2d6606640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d0159c60 con 0x7fd2d0154d50 2024-05-20T18:34:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.763+0000 7fd2d6606640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d00b0010 con 0x7fd2d00b5720 2024-05-20T18:34:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.763+0000 7fd2d6606640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd2d0159de0 con 0x7fd2d015d060 2024-05-20T18:34:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.764+0000 7fd2d4ba3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d0154d50 0x7fd2d00b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.764+0000 7fd2d4ba3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d0154d50 0x7fd2d00b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56718/0 (socket says 172.21.15.5:56718) 2024-05-20T18:34:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.764+0000 7fd2d4ba3640 1 -- 172.21.15.5:0/3000961590 learned_addr learned my addr 172.21.15.5:0/3000961590 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.764+0000 7fd2c7fff640 1 --2- 172.21.15.5:0/3000961590 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2d015d060 0x7fd2d00b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.764+0000 7fd2cffff640 1 --2- 172.21.15.5:0/3000961590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2d00b5720 0x7fd2d014f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.764+0000 7fd2d4ba3640 1 -- 172.21.15.5:0/3000961590 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2d015d060 msgr2=0x7fd2d00b0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.764+0000 7fd2d4ba3640 1 --2- 172.21.15.5:0/3000961590 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2d015d060 0x7fd2d00b0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.764+0000 7fd2d4ba3640 1 -- 172.21.15.5:0/3000961590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2d00b5720 msgr2=0x7fd2d014f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.764+0000 7fd2d4ba3640 1 --2- 172.21.15.5:0/3000961590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2d00b5720 0x7fd2d014f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.764+0000 7fd2d4ba3640 1 -- 172.21.15.5:0/3000961590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d0156a10 con 0x7fd2d0154d50 2024-05-20T18:34:32.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.764+0000 7fd2c7fff640 1 --2- 172.21.15.5:0/3000961590 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2d015d060 0x7fd2d00b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:34:32.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.764+0000 7fd2d4ba3640 1 --2- 172.21.15.5:0/3000961590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d0154d50 0x7fd2d00b0860 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8007d10 tx=0x7fd2c800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:32.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.764+0000 7fd2cd7fa640 1 -- 172.21.15.5:0/3000961590 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8014420 con 0x7fd2d0154d50 2024-05-20T18:34:32.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.764+0000 7fd2cd7fa640 1 -- 172.21.15.5:0/3000961590 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c8019070 con 0x7fd2d0154d50 2024-05-20T18:34:32.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.764+0000 7fd2d6606640 1 -- 172.21.15.5:0/3000961590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2d01517d0 con 0x7fd2d0154d50 2024-05-20T18:34:32.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.764+0000 7fd2cd7fa640 1 -- 172.21.15.5:0/3000961590 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8014800 con 0x7fd2d0154d50 2024-05-20T18:34:32.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.764+0000 7fd2d6606640 1 -- 172.21.15.5:0/3000961590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2d0151d90 con 0x7fd2d0154d50 2024-05-20T18:34:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.765+0000 7fd2d6606640 1 -- 172.21.15.5:0/3000961590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2d0075b00 con 0x7fd2d0154d50 2024-05-20T18:34:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.766+0000 7fd2cd7fa640 1 -- 172.21.15.5:0/3000961590 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd2c80149a0 con 0x7fd2d0154d50 2024-05-20T18:34:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.766+0000 7fd2cd7fa640 1 --2- 172.21.15.5:0/3000961590 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd2b006be60 0x7fd2b006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.766+0000 7fd2cd7fa640 1 -- 172.21.15.5:0/3000961590 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7fd2c808b7a0 con 0x7fd2d0154d50 2024-05-20T18:34:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.766+0000 7fd2cffff640 1 --2- 172.21.15.5:0/3000961590 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd2b006be60 0x7fd2b006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.767+0000 7fd2cffff640 1 --2- 172.21.15.5:0/3000961590 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd2b006be60 0x7fd2b006e370 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0034000 tx=0x7fd2c002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:32.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.771+0000 7fd2cd7fa640 1 -- 172.21.15.5:0/3000961590 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd2c8059ef0 con 0x7fd2d0154d50 2024-05-20T18:34:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.959+0000 7fd2d6606640 1 -- 172.21.15.5:0/3000961590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2d015b030 con 0x7fd2d0154d50 2024-05-20T18:34:32.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.961+0000 7fd2cd7fa640 1 -- 172.21.15.5:0/3000961590 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+17088 (secure 0 0 0) 0x7fd2c80600c0 con 0x7fd2d0154d50 2024-05-20T18:34:32.962 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:34:32.962 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":93,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:33:57.328628+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:33:10.184948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-20T18:33:11.453356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.1e","mappings":[{"from":1,"to":5}]},{"pgid":"14.29","mappings":[{"from":1,"to":7}]},{"pgid":"14.2a","mappings":[{"from":1,"to":2}]},{"pgid":"14.2c","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6834/2759396130":"2024-05-21T18:33:23.233035+0000","172.21.15.63:6832/2759396130":"2024-05-21T18:33:23.233035+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:34:32.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.963+0000 7fd2d6606640 1 -- 172.21.15.5:0/3000961590 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd2b006be60 msgr2=0x7fd2b006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:32.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.963+0000 7fd2d6606640 1 --2- 172.21.15.5:0/3000961590 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd2b006be60 0x7fd2b006e370 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0034000 tx=0x7fd2c002e000 comp rx=0 tx=0).stop 2024-05-20T18:34:32.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.963+0000 7fd2d6606640 1 -- 172.21.15.5:0/3000961590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d0154d50 msgr2=0x7fd2d00b0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:32.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.963+0000 7fd2d6606640 1 --2- 172.21.15.5:0/3000961590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d0154d50 0x7fd2d00b0860 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8007d10 tx=0x7fd2c800e4b0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.963+0000 7fd2d6606640 1 -- 172.21.15.5:0/3000961590 shutdown_connections 2024-05-20T18:34:32.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.963+0000 7fd2d6606640 1 --2- 172.21.15.5:0/3000961590 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd2b006be60 0x7fd2b006e370 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.963+0000 7fd2d6606640 1 --2- 172.21.15.5:0/3000961590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d0154d50 0x7fd2d00b0860 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.963+0000 7fd2d6606640 1 --2- 172.21.15.5:0/3000961590 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2d015d060 0x7fd2d00b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.963+0000 7fd2d6606640 1 --2- 172.21.15.5:0/3000961590 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2d00b5720 0x7fd2d014f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:32.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.963+0000 7fd2d6606640 1 -- 172.21.15.5:0/3000961590 >> 172.21.15.5:0/3000961590 conn(0x7fd2d00cd2a0 msgr2=0x7fd2d0159fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:32.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.963+0000 7fd2d6606640 1 -- 172.21.15.5:0/3000961590 shutdown_connections 2024-05-20T18:34:32.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:32.963+0000 7fd2d6606640 1 -- 172.21.15.5:0/3000961590 wait complete. 2024-05-20T18:34:32.975 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:33:10.184948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:33:11.453356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:34:32.976 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:34:33.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.093+0000 7f5205091640 1 Processor -- start 2024-05-20T18:34:33.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.093+0000 7f5205091640 1 -- start start 2024-05-20T18:34:33.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.095+0000 7f5205091640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5200162820 0x7f52000b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:33.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.095+0000 7f5205091640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f520006dae0 0x7f52000b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:33.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.095+0000 7f5205091640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52000d0630 con 0x7f52000b3e80 2024-05-20T18:34:33.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.095+0000 7f5205091640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f52000d39c0 con 0x7f520006dae0 2024-05-20T18:34:33.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.095+0000 7f5205091640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f52000d3ca0 con 0x7f5200162820 2024-05-20T18:34:33.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.095+0000 7f51ff59e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f520006dae0 0x7f52000b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:33.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.095+0000 7f51ff59e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f520006dae0 0x7f52000b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56732/0 (socket says 172.21.15.5:56732) 2024-05-20T18:34:33.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.095+0000 7f51ff59e640 1 -- 172.21.15.5:0/1609383268 learned_addr learned my addr 172.21.15.5:0/1609383268 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:33.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.095+0000 7f51fed9d640 1 --2- 172.21.15.5:0/1609383268 >> v2:172.21.15.63:3300/0 conn(0x7f5200162820 0x7f52000b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:33.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.095+0000 7f51ff59e640 1 -- 172.21.15.5:0/1609383268 >> v2:172.21.15.63:3300/0 conn(0x7f5200162820 msgr2=0x7f52000b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:33.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.095+0000 7f51ff59e640 1 --2- 172.21.15.5:0/1609383268 >> v2:172.21.15.63:3300/0 conn(0x7f5200162820 0x7f52000b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:33.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.095+0000 7f51ff59e640 1 -- 172.21.15.5:0/1609383268 >> v1:172.21.15.63:6789/0 conn(0x7f52000b3e80 legacy=0x7f520006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:33.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.095+0000 7f51ff59e640 1 -- 172.21.15.5:0/1609383268 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52000d3f80 con 0x7f520006dae0 2024-05-20T18:34:33.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.096+0000 7f51ff59e640 1 --2- 172.21.15.5:0/1609383268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f520006dae0 0x7f52000b8190 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f52000b7740 tx=0x7f51f802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2e79cc8dc38b3b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:33.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.096+0000 7f51fdd9b640 1 -- 172.21.15.5:0/1609383268 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8036070 con 0x7f520006dae0 2024-05-20T18:34:33.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.096+0000 7f51fdd9b640 1 -- 172.21.15.5:0/1609383268 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f8031600 con 0x7f520006dae0 2024-05-20T18:34:33.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.096+0000 7f51fdd9b640 1 -- 172.21.15.5:0/1609383268 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f80319d0 con 0x7f520006dae0 2024-05-20T18:34:33.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.097+0000 7f5205091640 1 -- 172.21.15.5:0/1609383268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f520006dae0 msgr2=0x7f52000b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:33.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.097+0000 7f5205091640 1 --2- 172.21.15.5:0/1609383268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f520006dae0 0x7f52000b8190 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f52000b7740 tx=0x7f51f802a730 comp rx=0 tx=0).stop 2024-05-20T18:34:33.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.097+0000 7f5205091640 1 -- 172.21.15.5:0/1609383268 shutdown_connections 2024-05-20T18:34:33.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.097+0000 7f5205091640 1 --2- 172.21.15.5:0/1609383268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f520006dae0 0x7f52000b8190 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:33.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.097+0000 7f5205091640 1 --2- 172.21.15.5:0/1609383268 >> v2:172.21.15.63:3300/0 conn(0x7f5200162820 0x7f52000b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:33.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.097+0000 7f5205091640 1 -- 172.21.15.5:0/1609383268 >> 172.21.15.5:0/1609383268 conn(0x7f52000d4bd0 msgr2=0x7f52000d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:33.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.097+0000 7f5205091640 1 -- 172.21.15.5:0/1609383268 shutdown_connections 2024-05-20T18:34:33.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.097+0000 7f5205091640 1 -- 172.21.15.5:0/1609383268 wait complete. 2024-05-20T18:34:33.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.099+0000 7f5205091640 1 Processor -- start 2024-05-20T18:34:33.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.099+0000 7f5205091640 1 -- start start 2024-05-20T18:34:33.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.101+0000 7f5205091640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52000b3e80 0x7f520014c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:33.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.101+0000 7f5205091640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5200162820 0x7f5200158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:33.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.101+0000 7f5205091640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f52000b64f0 0x7f52000b6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:33.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.101+0000 7f5205091640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f52000d4780 con 0x7f5200162820 2024-05-20T18:34:33.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.101+0000 7f5205091640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f52000d0300 con 0x7f52000b3e80 2024-05-20T18:34:33.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.101+0000 7f5205091640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f52000d4900 con 0x7f52000b64f0 2024-05-20T18:34:33.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.101+0000 7f51fe59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5200162820 0x7f5200158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:33.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.101+0000 7f51fe59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5200162820 0x7f5200158920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56734/0 (socket says 172.21.15.5:56734) 2024-05-20T18:34:33.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.101+0000 7f51fe59c640 1 -- 172.21.15.5:0/4288160182 learned_addr learned my addr 172.21.15.5:0/4288160182 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:33.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.101+0000 7f51fed9d640 1 --2- 172.21.15.5:0/4288160182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52000b3e80 0x7f520014c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:33.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.101+0000 7f51ff59e640 1 --2- 172.21.15.5:0/4288160182 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f52000b64f0 0x7f52000b6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:33.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.101+0000 7f51fe59c640 1 -- 172.21.15.5:0/4288160182 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f52000b64f0 msgr2=0x7f52000b6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:33.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.101+0000 7f51fe59c640 1 --2- 172.21.15.5:0/4288160182 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f52000b64f0 0x7f52000b6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:33.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.101+0000 7f51fe59c640 1 -- 172.21.15.5:0/4288160182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52000b3e80 msgr2=0x7f520014c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:33.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.101+0000 7f51fe59c640 1 --2- 172.21.15.5:0/4288160182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52000b3e80 0x7f520014c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:33.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.102+0000 7f51fe59c640 1 -- 172.21.15.5:0/4288160182 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52000b6e40 con 0x7f5200162820 2024-05-20T18:34:33.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.102+0000 7f51fe59c640 1 --2- 172.21.15.5:0/4288160182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5200162820 0x7f5200158920 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f51f400c3f0 tx=0x7f51f400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:33.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.102+0000 7f51e77fe640 1 -- 172.21.15.5:0/4288160182 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f40052d0 con 0x7f5200162820 2024-05-20T18:34:33.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.102+0000 7f51e77fe640 1 -- 172.21.15.5:0/4288160182 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f4005490 con 0x7f5200162820 2024-05-20T18:34:33.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.102+0000 7f5205091640 1 -- 172.21.15.5:0/4288160182 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5200158e60 con 0x7f5200162820 2024-05-20T18:34:33.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.102+0000 7f5205091640 1 -- 172.21.15.5:0/4288160182 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52001596e0 con 0x7f5200162820 2024-05-20T18:34:33.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.102+0000 7f51e77fe640 1 -- 172.21.15.5:0/4288160182 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4007260 con 0x7f5200162820 2024-05-20T18:34:33.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.103+0000 7f5205091640 1 -- 172.21.15.5:0/4288160182 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51e0005290 con 0x7f5200162820 2024-05-20T18:34:33.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.104+0000 7f51e77fe640 1 -- 172.21.15.5:0/4288160182 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f51f4007400 con 0x7f5200162820 2024-05-20T18:34:33.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.105+0000 7f51e77fe640 1 --2- 172.21.15.5:0/4288160182 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f51ec06be30 0x7f51ec06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:33.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.105+0000 7f51e77fe640 1 -- 172.21.15.5:0/4288160182 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7f51f408a480 con 0x7f5200162820 2024-05-20T18:34:33.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.105+0000 7f51fed9d640 1 --2- 172.21.15.5:0/4288160182 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f51ec06be30 0x7f51ec06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:33.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.105+0000 7f51fed9d640 1 --2- 172.21.15.5:0/4288160182 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f51ec06be30 0x7f51ec06e340 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009aa0 tx=0x7f51f0009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:33.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.110+0000 7f51e77fe640 1 -- 172.21.15.5:0/4288160182 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f52000b6e40 con 0x7f5200162820 2024-05-20T18:34:33.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.292+0000 7f5205091640 1 -- 172.21.15.5:0/4288160182 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f51e0005850 con 0x7f5200162820 2024-05-20T18:34:33.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.293+0000 7f51e77fe640 1 -- 172.21.15.5:0/4288160182 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f51f4058590 con 0x7f5200162820 2024-05-20T18:34:33.294 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:34:33.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.294+0000 7f5205091640 1 -- 172.21.15.5:0/4288160182 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f51ec06be30 msgr2=0x7f51ec06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:33.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.294+0000 7f5205091640 1 --2- 172.21.15.5:0/4288160182 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f51ec06be30 0x7f51ec06e340 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009aa0 tx=0x7f51f0009190 comp rx=0 tx=0).stop 2024-05-20T18:34:33.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.294+0000 7f5205091640 1 -- 172.21.15.5:0/4288160182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5200162820 msgr2=0x7f5200158920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:33.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.294+0000 7f5205091640 1 --2- 172.21.15.5:0/4288160182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5200162820 0x7f5200158920 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f51f400c3f0 tx=0x7f51f400c8b0 comp rx=0 tx=0).stop 2024-05-20T18:34:33.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.295+0000 7f5205091640 1 -- 172.21.15.5:0/4288160182 shutdown_connections 2024-05-20T18:34:33.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.295+0000 7f5205091640 1 --2- 172.21.15.5:0/4288160182 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f51ec06be30 0x7f51ec06e340 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:33.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.295+0000 7f5205091640 1 --2- 172.21.15.5:0/4288160182 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f52000b64f0 0x7f52000b6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:33.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.295+0000 7f5205091640 1 --2- 172.21.15.5:0/4288160182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5200162820 0x7f5200158920 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:33.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.295+0000 7f5205091640 1 --2- 172.21.15.5:0/4288160182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52000b3e80 0x7f520014c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:33.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.295+0000 7f5205091640 1 -- 172.21.15.5:0/4288160182 >> 172.21.15.5:0/4288160182 conn(0x7f52000d4bd0 msgr2=0x7f52000ba540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:33.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.295+0000 7f5205091640 1 -- 172.21.15.5:0/4288160182 shutdown_connections 2024-05-20T18:34:33.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.295+0000 7f5205091640 1 -- 172.21.15.5:0/4288160182 wait complete. 2024-05-20T18:34:33.307 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:34:33.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.424+0000 7f95f5c14640 1 Processor -- start 2024-05-20T18:34:33.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.424+0000 7f95f5c14640 1 -- start start 2024-05-20T18:34:33.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.425+0000 7f95f5c14640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f95f015d530 0x7f95f015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:33.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.425+0000 7f95f5c14640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f95f0070850 0x7f95f0070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:33.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.425+0000 7f95f5c14640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95f00cd580 con 0x7f95f00b1700 2024-05-20T18:34:33.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.425+0000 7f95f5c14640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f95f00ce190 con 0x7f95f0070850 2024-05-20T18:34:33.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.425+0000 7f95f5c14640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f95f00cd860 con 0x7f95f015d530 2024-05-20T18:34:33.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.425+0000 7f95effff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f95f0070850 0x7f95f0070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:33.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.425+0000 7f95e7fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f95f00b1700 0x7f95f00b1b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:49364/0 (socket says 172.21.15.5:49364) 2024-05-20T18:34:33.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.425+0000 7f95e7fff640 1 -- 172.21.15.5:0/1516555140 learned_addr learned my addr 172.21.15.5:0/1516555140 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:34:33.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.426+0000 7f95ef7fe640 1 --2- 172.21.15.5:0/1516555140 >> v2:172.21.15.63:3300/0 conn(0x7f95f015d530 0x7f95f015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:33.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.426+0000 7f95effff640 1 -- 172.21.15.5:0/1516555140 >> v2:172.21.15.63:3300/0 conn(0x7f95f015d530 msgr2=0x7f95f015aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:33.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.426+0000 7f95effff640 1 --2- 172.21.15.5:0/1516555140 >> v2:172.21.15.63:3300/0 conn(0x7f95f015d530 0x7f95f015aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:33.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.426+0000 7f95effff640 1 -- 172.21.15.5:0/1516555140 >> v1:172.21.15.5:6789/0 conn(0x7f95f00b1700 legacy=0x7f95f00b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:33.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.426+0000 7f95effff640 1 -- 172.21.15.5:0/1516555140 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f00cdb40 con 0x7f95f0070850 2024-05-20T18:34:33.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.426+0000 7f95eeffd640 1 -- 172.21.15.5:0/1516555140 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3494065818 0 0) 0x7f95f00cd580 con 0x7f95f00b1700 2024-05-20T18:34:33.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.427+0000 7f95effff640 1 --2- 172.21.15.5:0/1516555140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95f0070850 0x7f95f0070c60 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f95f004f790 tx=0x7f95e802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19dc6c04c4fe24a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:33.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.427+0000 7f95eeffd640 1 -- 172.21.15.5:0/1516555140 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e8007740 con 0x7f95f0070850 2024-05-20T18:34:33.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.427+0000 7f95eeffd640 1 -- 172.21.15.5:0/1516555140 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95e80078e0 con 0x7f95f0070850 2024-05-20T18:34:33.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.427+0000 7f95eeffd640 1 -- 172.21.15.5:0/1516555140 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e8007cb0 con 0x7f95f0070850 2024-05-20T18:34:33.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.427+0000 7f95f5c14640 1 -- 172.21.15.5:0/1516555140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95f0070850 msgr2=0x7f95f0070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:33.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.427+0000 7f95f5c14640 1 --2- 172.21.15.5:0/1516555140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95f0070850 0x7f95f0070c60 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f95f004f790 tx=0x7f95e802a680 comp rx=0 tx=0).stop 2024-05-20T18:34:33.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.428+0000 7f95f5c14640 1 -- 172.21.15.5:0/1516555140 shutdown_connections 2024-05-20T18:34:33.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.428+0000 7f95f5c14640 1 --2- 172.21.15.5:0/1516555140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95f0070850 0x7f95f0070c60 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:33.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.428+0000 7f95f5c14640 1 --2- 172.21.15.5:0/1516555140 >> v2:172.21.15.63:3300/0 conn(0x7f95f015d530 0x7f95f015aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:33.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.428+0000 7f95f5c14640 1 -- 172.21.15.5:0/1516555140 >> 172.21.15.5:0/1516555140 conn(0x7f95f00d29d0 msgr2=0x7f95f00d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:33.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.428+0000 7f95f5c14640 1 -- 172.21.15.5:0/1516555140 shutdown_connections 2024-05-20T18:34:33.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.428+0000 7f95f5c14640 1 -- 172.21.15.5:0/1516555140 wait complete. 2024-05-20T18:34:33.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.429+0000 7f95f5c14640 1 Processor -- start 2024-05-20T18:34:33.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.429+0000 7f95f5c14640 1 -- start start 2024-05-20T18:34:33.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.431+0000 7f95f5c14640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f95f0070850 0x7f95f00cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:33.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.431+0000 7f95f5c14640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95f00b1700 0x7f95f00ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:33.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.431+0000 7f95f5c14640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95f015d530 0x7f95f00cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:33.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.431+0000 7f95f5c14640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f95f00711a0 con 0x7f95f015d530 2024-05-20T18:34:33.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.431+0000 7f95f5c14640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f95f00cd230 con 0x7f95f00b1700 2024-05-20T18:34:33.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.431+0000 7f95f5c14640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f95f0071320 con 0x7f95f0070850 2024-05-20T18:34:33.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.431+0000 7f95effff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95f015d530 0x7f95f00cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:33.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.431+0000 7f95effff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95f015d530 0x7f95f00cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56758/0 (socket says 172.21.15.5:56758) 2024-05-20T18:34:33.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.431+0000 7f95effff640 1 -- 172.21.15.5:0/3418052816 learned_addr learned my addr 172.21.15.5:0/3418052816 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:33.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.431+0000 7f95e7fff640 1 --2- 172.21.15.5:0/3418052816 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95f00b1700 0x7f95f00ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:33.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.431+0000 7f95ef7fe640 1 --2- 172.21.15.5:0/3418052816 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f95f0070850 0x7f95f00cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:33.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.431+0000 7f95effff640 1 -- 172.21.15.5:0/3418052816 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f95f0070850 msgr2=0x7f95f00cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:33.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.431+0000 7f95effff640 1 --2- 172.21.15.5:0/3418052816 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f95f0070850 0x7f95f00cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:33.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.431+0000 7f95effff640 1 -- 172.21.15.5:0/3418052816 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95f00b1700 msgr2=0x7f95f00ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:33.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.431+0000 7f95effff640 1 --2- 172.21.15.5:0/3418052816 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95f00b1700 0x7f95f00ca8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:33.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.431+0000 7f95effff640 1 -- 172.21.15.5:0/3418052816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f00cccf0 con 0x7f95f015d530 2024-05-20T18:34:33.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.431+0000 7f95effff640 1 --2- 172.21.15.5:0/3418052816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95f015d530 0x7f95f00cae20 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f95e802ac70 tx=0x7f95e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:33.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.431+0000 7f95ecff9640 1 -- 172.21.15.5:0/3418052816 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e803f070 con 0x7f95f015d530 2024-05-20T18:34:33.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.431+0000 7f95ecff9640 1 -- 172.21.15.5:0/3418052816 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95e8031c90 con 0x7f95f015d530 2024-05-20T18:34:33.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.431+0000 7f95f5c14640 1 -- 172.21.15.5:0/3418052816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f00c9b20 con 0x7f95f015d530 2024-05-20T18:34:33.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.432+0000 7f95ecff9640 1 -- 172.21.15.5:0/3418052816 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e8044680 con 0x7f95f015d530 2024-05-20T18:34:33.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.432+0000 7f95f5c14640 1 -- 172.21.15.5:0/3418052816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f00ca0e0 con 0x7f95f015d530 2024-05-20T18:34:33.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.433+0000 7f95f5c14640 1 -- 172.21.15.5:0/3418052816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95f0075b00 con 0x7f95f015d530 2024-05-20T18:34:33.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.433+0000 7f95ecff9640 1 -- 172.21.15.5:0/3418052816 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f95e803b410 con 0x7f95f015d530 2024-05-20T18:34:33.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.434+0000 7f95ecff9640 1 --2- 172.21.15.5:0/3418052816 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f95d006be60 0x7f95d006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:33.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.434+0000 7f95ecff9640 1 -- 172.21.15.5:0/3418052816 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7f95e80aca40 con 0x7f95f015d530 2024-05-20T18:34:33.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.434+0000 7f95ef7fe640 1 --2- 172.21.15.5:0/3418052816 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f95d006be60 0x7f95d006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:33.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.434+0000 7f95ef7fe640 1 --2- 172.21.15.5:0/3418052816 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f95d006be60 0x7f95d006e370 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f95e0006fd0 tx=0x7f95e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:33.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.439+0000 7f95ecff9640 1 -- 172.21.15.5:0/3418052816 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f95e807ac10 con 0x7f95f015d530 2024-05-20T18:34:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.619+0000 7f95f5c14640 1 -- 172.21.15.5:0/3418052816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f95f00d1ee0 con 0x7f95f015d530 2024-05-20T18:34:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.620+0000 7f95ecff9640 1 -- 172.21.15.5:0/3418052816 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f95e807ef10 con 0x7f95f015d530 2024-05-20T18:34:33.621 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:34:33.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.622+0000 7f95f5c14640 1 -- 172.21.15.5:0/3418052816 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f95d006be60 msgr2=0x7f95d006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:33.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.622+0000 7f95f5c14640 1 --2- 172.21.15.5:0/3418052816 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f95d006be60 0x7f95d006e370 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f95e0006fd0 tx=0x7f95e0008040 comp rx=0 tx=0).stop 2024-05-20T18:34:33.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.622+0000 7f95f5c14640 1 -- 172.21.15.5:0/3418052816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95f015d530 msgr2=0x7f95f00cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:33.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.622+0000 7f95f5c14640 1 --2- 172.21.15.5:0/3418052816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95f015d530 0x7f95f00cae20 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f95e802ac70 tx=0x7f95e8004a40 comp rx=0 tx=0).stop 2024-05-20T18:34:33.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.622+0000 7f95f5c14640 1 -- 172.21.15.5:0/3418052816 shutdown_connections 2024-05-20T18:34:33.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.622+0000 7f95f5c14640 1 --2- 172.21.15.5:0/3418052816 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f95d006be60 0x7f95d006e370 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:33.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.622+0000 7f95f5c14640 1 --2- 172.21.15.5:0/3418052816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95f015d530 0x7f95f00cae20 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:33.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.622+0000 7f95f5c14640 1 --2- 172.21.15.5:0/3418052816 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95f00b1700 0x7f95f00ca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:33.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.622+0000 7f95f5c14640 1 --2- 172.21.15.5:0/3418052816 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f95f0070850 0x7f95f00cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:33.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.622+0000 7f95f5c14640 1 -- 172.21.15.5:0/3418052816 >> 172.21.15.5:0/3418052816 conn(0x7f95f00d29d0 msgr2=0x7f95f015d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:33.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.622+0000 7f95f5c14640 1 -- 172.21.15.5:0/3418052816 shutdown_connections 2024-05-20T18:34:33.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.622+0000 7f95f5c14640 1 -- 172.21.15.5:0/3418052816 wait complete. 2024-05-20T18:34:33.634 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:34:33.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.751+0000 7fc58ad02640 1 Processor -- start 2024-05-20T18:34:33.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.751+0000 7fc58ad02640 1 -- start start 2024-05-20T18:34:33.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.753+0000 7fc58ad02640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc58415f220 0x7fc5840d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:33.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.753+0000 7fc58ad02640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc5840cfae0 0x7fc5840ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:33.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.753+0000 7fc58ad02640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5840cda20 con 0x7fc5840d0e00 2024-05-20T18:34:33.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.753+0000 7fc58ad02640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc5840cdd00 con 0x7fc5840cfae0 2024-05-20T18:34:33.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.753+0000 7fc58ad02640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc5840cdfe0 con 0x7fc58415f220 2024-05-20T18:34:33.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.753+0000 7fc58929f640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc5840cfae0 0x7fc5840ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:33.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.753+0000 7fc588a9e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc58415f220 0x7fc5840d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:33.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.753+0000 7fc58929f640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc5840cfae0 0x7fc5840ce7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:56030/0 (socket says 172.21.15.5:56030) 2024-05-20T18:34:33.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.753+0000 7fc58929f640 1 -- 172.21.15.5:0/3862485750 learned_addr learned my addr 172.21.15.5:0/3862485750 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:33.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.754+0000 7fc58929f640 1 -- 172.21.15.5:0/3862485750 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc58415f220 msgr2=0x7fc5840d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:33.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.754+0000 7fc58929f640 1 --2- 172.21.15.5:0/3862485750 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc58415f220 0x7fc5840d08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:33.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.754+0000 7fc58929f640 1 -- 172.21.15.5:0/3862485750 >> v1:172.21.15.63:6789/0 conn(0x7fc5840d0e00 legacy=0x7fc5840cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:33.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.754+0000 7fc58929f640 1 -- 172.21.15.5:0/3862485750 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5840ce2c0 con 0x7fc5840cfae0 2024-05-20T18:34:33.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.754+0000 7fc588a9e640 1 --2- 172.21.15.5:0/3862485750 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc58415f220 0x7fc5840d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:34:33.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.754+0000 7fc58929f640 1 --2- 172.21.15.5:0/3862485750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5840cfae0 0x7fc5840ce7e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009e50 tx=0x7fc57c02a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10a04026cf26d2fb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:33.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.755+0000 7fc5837fe640 1 -- 172.21.15.5:0/3862485750 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c036070 con 0x7fc5840cfae0 2024-05-20T18:34:33.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.755+0000 7fc5837fe640 1 -- 172.21.15.5:0/3862485750 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc57c031600 con 0x7fc5840cfae0 2024-05-20T18:34:33.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.755+0000 7fc5837fe640 1 -- 172.21.15.5:0/3862485750 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c0319d0 con 0x7fc5840cfae0 2024-05-20T18:34:33.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.755+0000 7fc58ad02640 1 -- 172.21.15.5:0/3862485750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5840cfae0 msgr2=0x7fc5840ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:33.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.755+0000 7fc58ad02640 1 --2- 172.21.15.5:0/3862485750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5840cfae0 0x7fc5840ce7e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009e50 tx=0x7fc57c02a680 comp rx=0 tx=0).stop 2024-05-20T18:34:33.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.755+0000 7fc58ad02640 1 -- 172.21.15.5:0/3862485750 shutdown_connections 2024-05-20T18:34:33.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.755+0000 7fc58ad02640 1 --2- 172.21.15.5:0/3862485750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5840cfae0 0x7fc5840ce7e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:33.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.755+0000 7fc58ad02640 1 --2- 172.21.15.5:0/3862485750 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc58415f220 0x7fc5840d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:33.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.755+0000 7fc58ad02640 1 -- 172.21.15.5:0/3862485750 >> 172.21.15.5:0/3862485750 conn(0x7fc5840b44d0 msgr2=0x7fc5840b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:33.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.755+0000 7fc58ad02640 1 -- 172.21.15.5:0/3862485750 shutdown_connections 2024-05-20T18:34:33.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.755+0000 7fc58ad02640 1 -- 172.21.15.5:0/3862485750 wait complete. 2024-05-20T18:34:33.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.757+0000 7fc58ad02640 1 Processor -- start 2024-05-20T18:34:33.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.757+0000 7fc58ad02640 1 -- start start 2024-05-20T18:34:33.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.759+0000 7fc58ad02640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5840d0e00 0x7fc5840b6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:33.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.759+0000 7fc58ad02640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc58415f220 0x7fc5840b6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:33.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.759+0000 7fc58ad02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5840b72a0 0x7fc5840c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:33.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.759+0000 7fc58ad02640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc58415f7b0 con 0x7fc5840b72a0 2024-05-20T18:34:33.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.759+0000 7fc58ad02640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc58415f630 con 0x7fc5840d0e00 2024-05-20T18:34:33.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.759+0000 7fc58ad02640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc5840ced20 con 0x7fc58415f220 2024-05-20T18:34:33.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.759+0000 7fc58929f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5840b72a0 0x7fc5840c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:33.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.759+0000 7fc588a9e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5840d0e00 0x7fc5840b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:33.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.759+0000 7fc58929f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5840b72a0 0x7fc5840c69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56772/0 (socket says 172.21.15.5:56772) 2024-05-20T18:34:33.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.759+0000 7fc58929f640 1 -- 172.21.15.5:0/1149356225 learned_addr learned my addr 172.21.15.5:0/1149356225 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:33.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.759+0000 7fc583fff640 1 --2- 172.21.15.5:0/1149356225 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc58415f220 0x7fc5840b6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:33.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.759+0000 7fc58929f640 1 -- 172.21.15.5:0/1149356225 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc58415f220 msgr2=0x7fc5840b6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:33.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.759+0000 7fc58929f640 1 --2- 172.21.15.5:0/1149356225 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc58415f220 0x7fc5840b6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:33.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.759+0000 7fc58929f640 1 -- 172.21.15.5:0/1149356225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5840d0e00 msgr2=0x7fc5840b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:33.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.759+0000 7fc58929f640 1 --2- 172.21.15.5:0/1149356225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5840d0e00 0x7fc5840b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:33.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.759+0000 7fc58929f640 1 -- 172.21.15.5:0/1149356225 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5840c86d0 con 0x7fc5840b72a0 2024-05-20T18:34:33.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.759+0000 7fc58929f640 1 --2- 172.21.15.5:0/1149356225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5840b72a0 0x7fc5840c69c0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fc57c034040 tx=0x7fc57c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:33.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.759+0000 7fc5817fa640 1 -- 172.21.15.5:0/1149356225 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c036050 con 0x7fc5840b72a0 2024-05-20T18:34:33.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.760+0000 7fc5817fa640 1 -- 172.21.15.5:0/1149356225 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc57c042070 con 0x7fc5840b72a0 2024-05-20T18:34:33.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.760+0000 7fc58ad02640 1 -- 172.21.15.5:0/1149356225 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5840c9270 con 0x7fc5840b72a0 2024-05-20T18:34:33.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.760+0000 7fc5817fa640 1 -- 172.21.15.5:0/1149356225 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c009510 con 0x7fc5840b72a0 2024-05-20T18:34:33.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.760+0000 7fc58ad02640 1 -- 172.21.15.5:0/1149356225 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5840c8c20 con 0x7fc5840b72a0 2024-05-20T18:34:33.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.761+0000 7fc58ad02640 1 -- 172.21.15.5:0/1149356225 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc584075b00 con 0x7fc5840b72a0 2024-05-20T18:34:33.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.761+0000 7fc5817fa640 1 -- 172.21.15.5:0/1149356225 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc57c007bf0 con 0x7fc5840b72a0 2024-05-20T18:34:33.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.762+0000 7fc5817fa640 1 --2- 172.21.15.5:0/1149356225 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc56406be60 0x7fc56406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:33.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.762+0000 7fc5817fa640 1 -- 172.21.15.5:0/1149356225 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7fc57c0abdf0 con 0x7fc5840b72a0 2024-05-20T18:34:33.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.762+0000 7fc588a9e640 1 --2- 172.21.15.5:0/1149356225 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc56406be60 0x7fc56406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:33.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.762+0000 7fc588a9e640 1 --2- 172.21.15.5:0/1149356225 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc56406be60 0x7fc56406e370 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc574009560 tx=0x7fc574009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:33.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.766+0000 7fc5817fa640 1 -- 172.21.15.5:0/1149356225 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc57c079fa0 con 0x7fc5840b72a0 2024-05-20T18:34:33.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.956+0000 7fc58ad02640 1 -- 172.21.15.5:0/1149356225 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc5840c67a0 con 0x7fc5840b72a0 2024-05-20T18:34:33.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.956+0000 7fc5817fa640 1 -- 172.21.15.5:0/1149356225 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7fc57c07e2a0 con 0x7fc5840b72a0 2024-05-20T18:34:33.957 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:34:33.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.958+0000 7fc58ad02640 1 -- 172.21.15.5:0/1149356225 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc56406be60 msgr2=0x7fc56406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.958+0000 7fc58ad02640 1 --2- 172.21.15.5:0/1149356225 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc56406be60 0x7fc56406e370 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc574009560 tx=0x7fc574009190 comp rx=0 tx=0).stop 2024-05-20T18:34:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.958+0000 7fc58ad02640 1 -- 172.21.15.5:0/1149356225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5840b72a0 msgr2=0x7fc5840c69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.958+0000 7fc58ad02640 1 --2- 172.21.15.5:0/1149356225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5840b72a0 0x7fc5840c69c0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fc57c034040 tx=0x7fc57c004a40 comp rx=0 tx=0).stop 2024-05-20T18:34:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.958+0000 7fc58ad02640 1 -- 172.21.15.5:0/1149356225 shutdown_connections 2024-05-20T18:34:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.958+0000 7fc58ad02640 1 --2- 172.21.15.5:0/1149356225 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc56406be60 0x7fc56406e370 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.958+0000 7fc58ad02640 1 --2- 172.21.15.5:0/1149356225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5840b72a0 0x7fc5840c69c0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.958+0000 7fc58ad02640 1 --2- 172.21.15.5:0/1149356225 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc58415f220 0x7fc5840b6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.958+0000 7fc58ad02640 1 --2- 172.21.15.5:0/1149356225 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5840d0e00 0x7fc5840b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.958+0000 7fc58ad02640 1 -- 172.21.15.5:0/1149356225 >> 172.21.15.5:0/1149356225 conn(0x7fc5840b44d0 msgr2=0x7fc5840b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.959+0000 7fc58ad02640 1 -- 172.21.15.5:0/1149356225 shutdown_connections 2024-05-20T18:34:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:33.959+0000 7fc58ad02640 1 -- 172.21.15.5:0/1149356225 wait complete. 2024-05-20T18:34:33.971 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T18:34:34.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.086+0000 7fbd0b3c3640 1 Processor -- start 2024-05-20T18:34:34.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.086+0000 7fbd0b3c3640 1 -- start start 2024-05-20T18:34:34.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.087+0000 7fbd0b3c3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd0406da80 0x7fbd041602f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:34.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.087+0000 7fbd0b3c3640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbd040b1c80 0x7fbd0409a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:34.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.087+0000 7fbd0b3c3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbd040b2090 0x7fbd0409aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:34.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.087+0000 7fbd0b3c3640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fbd04159da0 con 0x7fbd040b2090 2024-05-20T18:34:34.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.087+0000 7fbd0b3c3640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fbd040af1c0 con 0x7fbd040b1c80 2024-05-20T18:34:34.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.087+0000 7fbd0b3c3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbd040d17b0 con 0x7fbd0406da80 2024-05-20T18:34:34.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.087+0000 7fbd09960640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbd040b2090 0x7fbd0409aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:34.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.087+0000 7fbd09960640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbd040b2090 0x7fbd0409aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56778/0 (socket says 172.21.15.5:56778) 2024-05-20T18:34:34.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.087+0000 7fbd09960640 1 -- 172.21.15.5:0/1062160001 learned_addr learned my addr 172.21.15.5:0/1062160001 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:34.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.088+0000 7fbd0915f640 1 --2- 172.21.15.5:0/1062160001 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd0406da80 0x7fbd041602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:34.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.088+0000 7fbd0895e640 1 --2- 172.21.15.5:0/1062160001 >> v2:172.21.15.63:3300/0 conn(0x7fbd040b1c80 0x7fbd0409a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:34.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.088+0000 7fbd09960640 1 -- 172.21.15.5:0/1062160001 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd0406da80 msgr2=0x7fbd041602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:34.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.088+0000 7fbd09960640 1 --2- 172.21.15.5:0/1062160001 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd0406da80 0x7fbd041602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:34.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.088+0000 7fbd09960640 1 -- 172.21.15.5:0/1062160001 >> v2:172.21.15.63:3300/0 conn(0x7fbd040b1c80 msgr2=0x7fbd0409a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:34.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.088+0000 7fbd09960640 1 --2- 172.21.15.5:0/1062160001 >> v2:172.21.15.63:3300/0 conn(0x7fbd040b1c80 0x7fbd0409a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:34.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.088+0000 7fbd09960640 1 -- 172.21.15.5:0/1062160001 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd040af9d0 con 0x7fbd040b2090 2024-05-20T18:34:34.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.088+0000 7fbd0895e640 1 --2- 172.21.15.5:0/1062160001 >> v2:172.21.15.63:3300/0 conn(0x7fbd040b1c80 0x7fbd0409a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:34:34.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.088+0000 7fbd0915f640 1 --2- 172.21.15.5:0/1062160001 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd0406da80 0x7fbd041602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:34:34.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.088+0000 7fbd09960640 1 --2- 172.21.15.5:0/1062160001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd040b2090 0x7fbd0409aa30 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fbd04074db0 tx=0x7fbcfc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc2ca053bb15373 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:34.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.089+0000 7fbd03fff640 1 -- 172.21.15.5:0/1062160001 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc037070 con 0x7fbd040b2090 2024-05-20T18:34:34.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.089+0000 7fbd03fff640 1 -- 172.21.15.5:0/1062160001 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcfc032440 con 0x7fbd040b2090 2024-05-20T18:34:34.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.089+0000 7fbd03fff640 1 -- 172.21.15.5:0/1062160001 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc032810 con 0x7fbd040b2090 2024-05-20T18:34:34.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.089+0000 7fbd0b3c3640 1 -- 172.21.15.5:0/1062160001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd040b2090 msgr2=0x7fbd0409aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:34.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.089+0000 7fbd0b3c3640 1 --2- 172.21.15.5:0/1062160001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd040b2090 0x7fbd0409aa30 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fbd04074db0 tx=0x7fbcfc02a140 comp rx=0 tx=0).stop 2024-05-20T18:34:34.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.089+0000 7fbd0b3c3640 1 -- 172.21.15.5:0/1062160001 shutdown_connections 2024-05-20T18:34:34.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.089+0000 7fbd0b3c3640 1 --2- 172.21.15.5:0/1062160001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd040b2090 0x7fbd0409aa30 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:34.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.089+0000 7fbd0b3c3640 1 --2- 172.21.15.5:0/1062160001 >> v2:172.21.15.63:3300/0 conn(0x7fbd040b1c80 0x7fbd0409a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:34.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.089+0000 7fbd0b3c3640 1 --2- 172.21.15.5:0/1062160001 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd0406da80 0x7fbd041602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:34.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.089+0000 7fbd0b3c3640 1 -- 172.21.15.5:0/1062160001 >> 172.21.15.5:0/1062160001 conn(0x7fbd040b6670 msgr2=0x7fbd040b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:34.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.090+0000 7fbd0b3c3640 1 -- 172.21.15.5:0/1062160001 shutdown_connections 2024-05-20T18:34:34.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.090+0000 7fbd0b3c3640 1 -- 172.21.15.5:0/1062160001 wait complete. 2024-05-20T18:34:34.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.091+0000 7fbd0b3c3640 1 Processor -- start 2024-05-20T18:34:34.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.091+0000 7fbd0b3c3640 1 -- start start 2024-05-20T18:34:34.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.093+0000 7fbd0b3c3640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd0406da80 0x7fbd04152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:34.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.093+0000 7fbd0b3c3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd040b1c80 0x7fbd04144030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:34.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.093+0000 7fbd0b3c3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd040b2090 0x7fbd04155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:34.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.093+0000 7fbd0b3c3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0409b260 con 0x7fbd040b2090 2024-05-20T18:34:34.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.093+0000 7fbd0b3c3640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbd040af6f0 con 0x7fbd0406da80 2024-05-20T18:34:34.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.093+0000 7fbd0b3c3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbd040d1c40 con 0x7fbd040b1c80 2024-05-20T18:34:34.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.093+0000 7fbd09960640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd040b2090 0x7fbd04155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:34.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.093+0000 7fbd09960640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd040b2090 0x7fbd04155220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56788/0 (socket says 172.21.15.5:56788) 2024-05-20T18:34:34.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.093+0000 7fbd09960640 1 -- 172.21.15.5:0/2721014630 learned_addr learned my addr 172.21.15.5:0/2721014630 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:34.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.093+0000 7fbd0915f640 1 --2- 172.21.15.5:0/2721014630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd0406da80 0x7fbd04152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:34.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.093+0000 7fbd0895e640 1 --2- 172.21.15.5:0/2721014630 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd040b1c80 0x7fbd04144030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:34.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.093+0000 7fbd09960640 1 -- 172.21.15.5:0/2721014630 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd040b1c80 msgr2=0x7fbd04144030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:34.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.093+0000 7fbd09960640 1 --2- 172.21.15.5:0/2721014630 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd040b1c80 0x7fbd04144030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:34.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.093+0000 7fbd09960640 1 -- 172.21.15.5:0/2721014630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd0406da80 msgr2=0x7fbd04152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:34.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.093+0000 7fbd09960640 1 --2- 172.21.15.5:0/2721014630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd0406da80 0x7fbd04152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:34.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.093+0000 7fbd09960640 1 -- 172.21.15.5:0/2721014630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd04144570 con 0x7fbd040b2090 2024-05-20T18:34:34.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.093+0000 7fbd09960640 1 --2- 172.21.15.5:0/2721014630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd040b2090 0x7fbd04155220 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc02a180 tx=0x7fbcfc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:34.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.093+0000 7fbd01ffb640 1 -- 172.21.15.5:0/2721014630 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc041020 con 0x7fbd040b2090 2024-05-20T18:34:34.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.094+0000 7fbd01ffb640 1 -- 172.21.15.5:0/2721014630 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcfc0306c0 con 0x7fbd040b2090 2024-05-20T18:34:34.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.094+0000 7fbd0b3c3640 1 -- 172.21.15.5:0/2721014630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd04155760 con 0x7fbd040b2090 2024-05-20T18:34:34.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.094+0000 7fbd01ffb640 1 -- 172.21.15.5:0/2721014630 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc037040 con 0x7fbd040b2090 2024-05-20T18:34:34.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.094+0000 7fbd0b3c3640 1 -- 172.21.15.5:0/2721014630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd04155cf0 con 0x7fbd040b2090 2024-05-20T18:34:34.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.095+0000 7fbd0b3c3640 1 -- 172.21.15.5:0/2721014630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd04075b00 con 0x7fbd040b2090 2024-05-20T18:34:34.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.095+0000 7fbd01ffb640 1 -- 172.21.15.5:0/2721014630 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbcfc044020 con 0x7fbd040b2090 2024-05-20T18:34:34.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.096+0000 7fbd01ffb640 1 --2- 172.21.15.5:0/2721014630 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbcf006be60 0x7fbcf006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:34.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.096+0000 7fbd01ffb640 1 -- 172.21.15.5:0/2721014630 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7fbcfc0ab7f0 con 0x7fbd040b2090 2024-05-20T18:34:34.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.096+0000 7fbd0915f640 1 --2- 172.21.15.5:0/2721014630 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbcf006be60 0x7fbcf006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:34.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.097+0000 7fbd0915f640 1 --2- 172.21.15.5:0/2721014630 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbcf006be60 0x7fbcf006e370 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fbd040b6ca0 tx=0x7fbcf40097f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:34.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.101+0000 7fbd01ffb640 1 -- 172.21.15.5:0/2721014630 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbcfc04b030 con 0x7fbd040b2090 2024-05-20T18:34:34.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.298+0000 7fbd0b3c3640 1 -- 172.21.15.5:0/2721014630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbd040ae950 con 0x7fbd040b2090 2024-05-20T18:34:34.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.298+0000 7fbd01ffb640 1 -- 172.21.15.5:0/2721014630 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v93) ==== 80+0+122 (secure 0 0 0) 0x7fbcfc079870 con 0x7fbd040b2090 2024-05-20T18:34:34.300 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.63:6834/2759396130 2024-05-21T18:33:23.233035+0000 2024-05-20T18:34:34.300 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.63:6832/2759396130 2024-05-21T18:33:23.233035+0000 2024-05-20T18:34:34.301 INFO:teuthology.orchestra.run.smithi005.stderr:listed 2 entries 2024-05-20T18:34:34.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.300+0000 7fbd0b3c3640 1 -- 172.21.15.5:0/2721014630 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbcf006be60 msgr2=0x7fbcf006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:34.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.300+0000 7fbd0b3c3640 1 --2- 172.21.15.5:0/2721014630 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbcf006be60 0x7fbcf006e370 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fbd040b6ca0 tx=0x7fbcf40097f0 comp rx=0 tx=0).stop 2024-05-20T18:34:34.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.300+0000 7fbd0b3c3640 1 -- 172.21.15.5:0/2721014630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd040b2090 msgr2=0x7fbd04155220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:34.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.300+0000 7fbd0b3c3640 1 --2- 172.21.15.5:0/2721014630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd040b2090 0x7fbd04155220 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc02a180 tx=0x7fbcfc0046c0 comp rx=0 tx=0).stop 2024-05-20T18:34:34.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.301+0000 7fbd0b3c3640 1 -- 172.21.15.5:0/2721014630 shutdown_connections 2024-05-20T18:34:34.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.301+0000 7fbd0b3c3640 1 --2- 172.21.15.5:0/2721014630 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbcf006be60 0x7fbcf006e370 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:34.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.301+0000 7fbd0b3c3640 1 --2- 172.21.15.5:0/2721014630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd040b2090 0x7fbd04155220 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:34.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.301+0000 7fbd0b3c3640 1 --2- 172.21.15.5:0/2721014630 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd040b1c80 0x7fbd04144030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:34.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.301+0000 7fbd0b3c3640 1 --2- 172.21.15.5:0/2721014630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd0406da80 0x7fbd04152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:34.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.301+0000 7fbd0b3c3640 1 -- 172.21.15.5:0/2721014630 >> 172.21.15.5:0/2721014630 conn(0x7fbd040b6670 msgr2=0x7fbd04067490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:34.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.301+0000 7fbd0b3c3640 1 -- 172.21.15.5:0/2721014630 shutdown_connections 2024-05-20T18:34:34.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.301+0000 7fbd0b3c3640 1 -- 172.21.15.5:0/2721014630 wait complete. 2024-05-20T18:34:34.313 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-20T18:34:34.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:34:34.314 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-20T18:34:34.344 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:34:34.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:34:34.344 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-20T18:34:34.400 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:34:34.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.554+0000 7ff481022640 1 Processor -- start 2024-05-20T18:34:34.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.554+0000 7ff481022640 1 -- start start 2024-05-20T18:34:34.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.556+0000 7ff481022640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff47c0cf3d0 0x7ff47c0cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:34.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.556+0000 7ff481022640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff47c0ce7e0 0x7ff47c0cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:34.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.556+0000 7ff481022640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff47c0cfd20 con 0x7ff47c15f220 2024-05-20T18:34:34.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.556+0000 7ff481022640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7ff47c0cf130 con 0x7ff47c0ce7e0 2024-05-20T18:34:34.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.556+0000 7ff481022640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff47c0cda20 con 0x7ff47c0cf3d0 2024-05-20T18:34:34.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.556+0000 7ff47b59e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff47c0ce7e0 0x7ff47c0cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:34.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.556+0000 7ff47b59e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff47c0ce7e0 0x7ff47c0cebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56790/0 (socket says 172.21.15.5:56790) 2024-05-20T18:34:34.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.556+0000 7ff47b59e640 1 -- 172.21.15.5:0/3582288468 learned_addr learned my addr 172.21.15.5:0/3582288468 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:34.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.556+0000 7ff47a59c640 1 --2- 172.21.15.5:0/3582288468 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff47c0cf3d0 0x7ff47c0cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:34.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.556+0000 7ff47b59e640 1 -- 172.21.15.5:0/3582288468 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff47c0cf3d0 msgr2=0x7ff47c0cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:34.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.556+0000 7ff47b59e640 1 --2- 172.21.15.5:0/3582288468 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff47c0cf3d0 0x7ff47c0cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:34.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.556+0000 7ff47b59e640 1 -- 172.21.15.5:0/3582288468 >> v1:172.21.15.63:6789/0 conn(0x7ff47c15f220 legacy=0x7ff47c0d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:34.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.556+0000 7ff47b59e640 1 -- 172.21.15.5:0/3582288468 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff47c0cdd00 con 0x7ff47c0ce7e0 2024-05-20T18:34:34.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.557+0000 7ff47b59e640 1 --2- 172.21.15.5:0/3582288468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff47c0ce7e0 0x7ff47c0cebf0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7ff474009870 tx=0x7ff47402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=206e6eef62b102a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:34.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.557+0000 7ff479d9b640 1 -- 172.21.15.5:0/3582288468 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff474037070 con 0x7ff47c0ce7e0 2024-05-20T18:34:34.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.557+0000 7ff479d9b640 1 -- 172.21.15.5:0/3582288468 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff474032440 con 0x7ff47c0ce7e0 2024-05-20T18:34:34.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.557+0000 7ff479d9b640 1 -- 172.21.15.5:0/3582288468 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff474032810 con 0x7ff47c0ce7e0 2024-05-20T18:34:34.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.558+0000 7ff481022640 1 -- 172.21.15.5:0/3582288468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff47c0ce7e0 msgr2=0x7ff47c0cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:34.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.558+0000 7ff481022640 1 --2- 172.21.15.5:0/3582288468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff47c0ce7e0 0x7ff47c0cebf0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7ff474009870 tx=0x7ff47402a1c0 comp rx=0 tx=0).stop 2024-05-20T18:34:34.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.558+0000 7ff481022640 1 -- 172.21.15.5:0/3582288468 shutdown_connections 2024-05-20T18:34:34.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.558+0000 7ff481022640 1 --2- 172.21.15.5:0/3582288468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff47c0ce7e0 0x7ff47c0cebf0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:34.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.558+0000 7ff481022640 1 --2- 172.21.15.5:0/3582288468 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff47c0cf3d0 0x7ff47c0cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:34.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.558+0000 7ff481022640 1 -- 172.21.15.5:0/3582288468 >> 172.21.15.5:0/3582288468 conn(0x7ff47c0b44d0 msgr2=0x7ff47c0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:34.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.558+0000 7ff481022640 1 -- 172.21.15.5:0/3582288468 shutdown_connections 2024-05-20T18:34:34.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.558+0000 7ff481022640 1 -- 172.21.15.5:0/3582288468 wait complete. 2024-05-20T18:34:34.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.560+0000 7ff481022640 1 Processor -- start 2024-05-20T18:34:34.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.560+0000 7ff481022640 1 -- start start 2024-05-20T18:34:34.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.561+0000 7ff481022640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff47c0ce7e0 0x7ff47c15f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:34.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.561+0000 7ff481022640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff47c0cf3d0 0x7ff47c0c64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:34.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.561+0000 7ff481022640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff47c15f220 0x7ff47c1f5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:34.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.561+0000 7ff481022640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c0d0fd0 con 0x7ff47c0ce7e0 2024-05-20T18:34:34.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.561+0000 7ff481022640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff47c0ce2c0 con 0x7ff47c0cf3d0 2024-05-20T18:34:34.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.561+0000 7ff481022640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff47c0cd680 con 0x7ff47c15f220 2024-05-20T18:34:34.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.561+0000 7ff47ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff47c0ce7e0 0x7ff47c15f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:34.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.562+0000 7ff47ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff47c0ce7e0 0x7ff47c15f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56802/0 (socket says 172.21.15.5:56802) 2024-05-20T18:34:34.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.562+0000 7ff47ad9d640 1 -- 172.21.15.5:0/1556092740 learned_addr learned my addr 172.21.15.5:0/1556092740 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:34.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.562+0000 7ff47a59c640 1 --2- 172.21.15.5:0/1556092740 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff47c0cf3d0 0x7ff47c0c64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:34.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.562+0000 7ff47b59e640 1 --2- 172.21.15.5:0/1556092740 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff47c15f220 0x7ff47c1f5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:34.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.562+0000 7ff47ad9d640 1 -- 172.21.15.5:0/1556092740 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff47c15f220 msgr2=0x7ff47c1f5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:34.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.562+0000 7ff47ad9d640 1 --2- 172.21.15.5:0/1556092740 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff47c15f220 0x7ff47c1f5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:34.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.562+0000 7ff47ad9d640 1 -- 172.21.15.5:0/1556092740 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff47c0cf3d0 msgr2=0x7ff47c0c64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:34.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.562+0000 7ff47ad9d640 1 --2- 172.21.15.5:0/1556092740 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff47c0cf3d0 0x7ff47c0c64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:34.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.562+0000 7ff47ad9d640 1 -- 172.21.15.5:0/1556092740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff47c1611a0 con 0x7ff47c0ce7e0 2024-05-20T18:34:34.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.562+0000 7ff47ad9d640 1 --2- 172.21.15.5:0/1556092740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff47c0ce7e0 0x7ff47c15f630 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00c6e0 tx=0x7ff46c00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:34.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.562+0000 7ff4637fe640 1 -- 172.21.15.5:0/1556092740 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c016020 con 0x7ff47c0ce7e0 2024-05-20T18:34:34.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.562+0000 7ff4637fe640 1 -- 172.21.15.5:0/1556092740 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff46c006d50 con 0x7ff47c0ce7e0 2024-05-20T18:34:34.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.562+0000 7ff4637fe640 1 -- 172.21.15.5:0/1556092740 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c003bb0 con 0x7ff47c0ce7e0 2024-05-20T18:34:34.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.562+0000 7ff481022640 1 -- 172.21.15.5:0/1556092740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff47c161480 con 0x7ff47c0ce7e0 2024-05-20T18:34:34.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.562+0000 7ff481022640 1 -- 172.21.15.5:0/1556092740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff47c158180 con 0x7ff47c0ce7e0 2024-05-20T18:34:34.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.563+0000 7ff481022640 1 -- 172.21.15.5:0/1556092740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff47c075b00 con 0x7ff47c0ce7e0 2024-05-20T18:34:34.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.564+0000 7ff4637fe640 1 -- 172.21.15.5:0/1556092740 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff46c0050d0 con 0x7ff47c0ce7e0 2024-05-20T18:34:34.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.564+0000 7ff4637fe640 1 --2- 172.21.15.5:0/1556092740 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff46806be60 0x7ff46806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:34.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.565+0000 7ff4637fe640 1 -- 172.21.15.5:0/1556092740 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7ff46c08ab10 con 0x7ff47c0ce7e0 2024-05-20T18:34:34.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.565+0000 7ff47a59c640 1 --2- 172.21.15.5:0/1556092740 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff46806be60 0x7ff46806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:34.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.565+0000 7ff47a59c640 1 --2- 172.21.15.5:0/1556092740 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff46806be60 0x7ff46806e370 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0b1f80 tx=0x7ff470008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:34.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.569+0000 7ff4637fe640 1 -- 172.21.15.5:0/1556092740 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff46c020020 con 0x7ff47c0ce7e0 2024-05-20T18:34:34.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.789+0000 7ff481022640 1 -- 172.21.15.5:0/1556092740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff47c162c60 con 0x7ff47c0ce7e0 2024-05-20T18:34:34.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.790+0000 7ff4637fe640 1 -- 172.21.15.5:0/1556092740 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5173 (secure 0 0 0) 0x7ff47c0cfd20 con 0x7ff47c0ce7e0 2024-05-20T18:34:34.791 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:34:34.791 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/337248865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":337248865},{"type":"v1","addr":"172.21.15.63:6834","nonce":337248865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3705772553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3705772553},{"type":"v1","addr":"172.21.15.63:6835","nonce":3705772553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":7434,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/776188138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":776188138},{"type":"v1","addr":"172.21.15.5:6835","nonce":776188138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:33:12.457716+0000","modified":"2024-05-20T18:33:34.256059+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":10,"addr":"172.21.15.5:6837/3556492262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3556492262},{"type":"v1","addr":"172.21.15.5:6837","nonce":3556492262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7437,"qdb_cluster":[7437]},"id":6}]} 2024-05-20T18:34:34.791 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 67 2024-05-20T18:34:34.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.792+0000 7ff481022640 1 -- 172.21.15.5:0/1556092740 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff46806be60 msgr2=0x7ff46806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:34.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.792+0000 7ff481022640 1 --2- 172.21.15.5:0/1556092740 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff46806be60 0x7ff46806e370 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0b1f80 tx=0x7ff470008040 comp rx=0 tx=0).stop 2024-05-20T18:34:34.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.792+0000 7ff481022640 1 -- 172.21.15.5:0/1556092740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff47c0ce7e0 msgr2=0x7ff47c15f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:34.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.792+0000 7ff481022640 1 --2- 172.21.15.5:0/1556092740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff47c0ce7e0 0x7ff47c15f630 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00c6e0 tx=0x7ff46c00cba0 comp rx=0 tx=0).stop 2024-05-20T18:34:34.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.792+0000 7ff481022640 1 -- 172.21.15.5:0/1556092740 shutdown_connections 2024-05-20T18:34:34.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.792+0000 7ff481022640 1 --2- 172.21.15.5:0/1556092740 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff46806be60 0x7ff46806e370 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:34.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.792+0000 7ff481022640 1 --2- 172.21.15.5:0/1556092740 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff47c15f220 0x7ff47c1f5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:34.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.792+0000 7ff481022640 1 --2- 172.21.15.5:0/1556092740 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff47c0cf3d0 0x7ff47c0c64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:34.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.792+0000 7ff481022640 1 --2- 172.21.15.5:0/1556092740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff47c0ce7e0 0x7ff47c15f630 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:34.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.792+0000 7ff481022640 1 -- 172.21.15.5:0/1556092740 >> 172.21.15.5:0/1556092740 conn(0x7ff47c0b44d0 msgr2=0x7ff47c0b3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:34.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.792+0000 7ff481022640 1 -- 172.21.15.5:0/1556092740 shutdown_connections 2024-05-20T18:34:34.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:34.793+0000 7ff481022640 1 -- 172.21.15.5:0/1556092740 wait complete. 2024-05-20T18:34:34.805 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-05-20T18:34:34.944 INFO:teuthology.orchestra.run.smithi005.stdout:{"message":"","return_code":0} 2024-05-20T18:34:34.945 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-20T18:34:34.945 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-20T18:34:35.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.061+0000 7f1ad2157640 1 Processor -- start 2024-05-20T18:34:35.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.061+0000 7f1ad2157640 1 -- start start 2024-05-20T18:34:35.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.062+0000 7f1ad2157640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1acc163a10 0x7f1acc0d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.062+0000 7f1ad2157640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1acc0b4010 0x7f1acc0cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.062+0000 7f1ad2157640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1acc0d14b0 0x7f1acc0d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.062+0000 7f1ad2157640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f1acc0d3ca0 con 0x7f1acc0b4010 2024-05-20T18:34:35.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.062+0000 7f1ad2157640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1acc0d0370 con 0x7f1acc0d14b0 2024-05-20T18:34:35.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.062+0000 7f1ad2157640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1acc0d0650 con 0x7f1acc163a10 2024-05-20T18:34:35.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.062+0000 7f1ac2ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1acc0b4010 0x7f1acc0cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.062+0000 7f1ac2ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1acc0b4010 0x7f1acc0cfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38274/0 (socket says 172.21.15.5:38274) 2024-05-20T18:34:35.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.062+0000 7f1ac2ffd640 1 -- 172.21.15.5:0/313327698 learned_addr learned my addr 172.21.15.5:0/313327698 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:35.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.063+0000 7f1acb7fe640 1 --2- 172.21.15.5:0/313327698 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1acc163a10 0x7f1acc0d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.063+0000 7f1acbfff640 1 --2- 172.21.15.5:0/313327698 >> v2:172.21.15.63:3300/0 conn(0x7f1acc0d14b0 0x7f1acc0d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.063+0000 7f1ac2ffd640 1 -- 172.21.15.5:0/313327698 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1acc163a10 msgr2=0x7f1acc0d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:35.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.063+0000 7f1ac2ffd640 1 --2- 172.21.15.5:0/313327698 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1acc163a10 0x7f1acc0d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.063+0000 7f1ac2ffd640 1 -- 172.21.15.5:0/313327698 >> v2:172.21.15.63:3300/0 conn(0x7f1acc0d14b0 msgr2=0x7f1acc0d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:35.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.063+0000 7f1ac2ffd640 1 --2- 172.21.15.5:0/313327698 >> v2:172.21.15.63:3300/0 conn(0x7f1acc0d14b0 0x7f1acc0d18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.063+0000 7f1ac2ffd640 1 -- 172.21.15.5:0/313327698 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1acc0d3650 con 0x7f1acc0b4010 2024-05-20T18:34:35.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.063+0000 7f1acbfff640 1 --2- 172.21.15.5:0/313327698 >> v2:172.21.15.63:3300/0 conn(0x7f1acc0d14b0 0x7f1acc0d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:34:35.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.063+0000 7f1ac2ffd640 1 --2- 172.21.15.5:0/313327698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1acc0b4010 0x7f1acc0cfb00 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f1acc0b4590 tx=0x7f1ab402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7801432155e35de server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.064+0000 7f1acaffd640 1 -- 172.21.15.5:0/313327698 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4037070 con 0x7f1acc0b4010 2024-05-20T18:34:35.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.064+0000 7f1acaffd640 1 -- 172.21.15.5:0/313327698 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab4032440 con 0x7f1acc0b4010 2024-05-20T18:34:35.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.064+0000 7f1acaffd640 1 -- 172.21.15.5:0/313327698 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4032810 con 0x7f1acc0b4010 2024-05-20T18:34:35.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.064+0000 7f1ad2157640 1 -- 172.21.15.5:0/313327698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1acc0b4010 msgr2=0x7f1acc0cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.064+0000 7f1ad2157640 1 --2- 172.21.15.5:0/313327698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1acc0b4010 0x7f1acc0cfb00 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f1acc0b4590 tx=0x7f1ab402a1e0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.064+0000 7f1ad2157640 1 -- 172.21.15.5:0/313327698 shutdown_connections 2024-05-20T18:34:35.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.064+0000 7f1ad2157640 1 --2- 172.21.15.5:0/313327698 >> v2:172.21.15.63:3300/0 conn(0x7f1acc0d14b0 0x7f1acc0d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.064+0000 7f1ad2157640 1 --2- 172.21.15.5:0/313327698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1acc0b4010 0x7f1acc0cfb00 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.064+0000 7f1ad2157640 1 --2- 172.21.15.5:0/313327698 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1acc163a10 0x7f1acc0d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.064+0000 7f1ad2157640 1 -- 172.21.15.5:0/313327698 >> 172.21.15.5:0/313327698 conn(0x7f1acc06d240 msgr2=0x7f1acc06d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:35.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.064+0000 7f1ad2157640 1 -- 172.21.15.5:0/313327698 shutdown_connections 2024-05-20T18:34:35.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.064+0000 7f1ad2157640 1 -- 172.21.15.5:0/313327698 wait complete. 2024-05-20T18:34:35.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.066+0000 7f1ad2157640 1 Processor -- start 2024-05-20T18:34:35.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.066+0000 7f1ad2157640 1 -- start start 2024-05-20T18:34:35.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.068+0000 7f1ad2157640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1acc0b4010 0x7f1acc15ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.068+0000 7f1ad2157640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1acc0d14b0 0x7f1acc159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.068+0000 7f1ad2157640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1acc163a10 0x7f1acc0ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.068+0000 7f1ad2157640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc0d1e00 con 0x7f1acc163a10 2024-05-20T18:34:35.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.068+0000 7f1ad2157640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc0d0040 con 0x7f1acc0b4010 2024-05-20T18:34:35.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.068+0000 7f1ad2157640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1acc0b8210 con 0x7f1acc0d14b0 2024-05-20T18:34:35.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.068+0000 7f1acbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1acc163a10 0x7f1acc0ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.068+0000 7f1acbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1acc163a10 0x7f1acc0ce5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38284/0 (socket says 172.21.15.5:38284) 2024-05-20T18:34:35.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.068+0000 7f1acbfff640 1 -- 172.21.15.5:0/305372984 learned_addr learned my addr 172.21.15.5:0/305372984 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:35.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.068+0000 7f1acb7fe640 1 --2- 172.21.15.5:0/305372984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1acc0b4010 0x7f1acc15ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.068+0000 7f1ac2ffd640 1 --2- 172.21.15.5:0/305372984 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1acc0d14b0 0x7f1acc159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.068+0000 7f1acbfff640 1 -- 172.21.15.5:0/305372984 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1acc0d14b0 msgr2=0x7f1acc159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.068+0000 7f1acbfff640 1 --2- 172.21.15.5:0/305372984 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1acc0d14b0 0x7f1acc159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.068+0000 7f1acbfff640 1 -- 172.21.15.5:0/305372984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1acc0b4010 msgr2=0x7f1acc15ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.068+0000 7f1acbfff640 1 --2- 172.21.15.5:0/305372984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1acc0b4010 0x7f1acc15ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.068+0000 7f1acbfff640 1 -- 172.21.15.5:0/305372984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1acc0ceaf0 con 0x7f1acc163a10 2024-05-20T18:34:35.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.068+0000 7f1acbfff640 1 --2- 172.21.15.5:0/305372984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1acc163a10 0x7f1acc0ce5b0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4007db0 tx=0x7f1ac400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.069+0000 7f1ac8ff9640 1 -- 172.21.15.5:0/305372984 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac401a070 con 0x7f1acc163a10 2024-05-20T18:34:35.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.069+0000 7f1ac8ff9640 1 -- 172.21.15.5:0/305372984 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ac4003990 con 0x7f1acc163a10 2024-05-20T18:34:35.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.069+0000 7f1ac8ff9640 1 -- 172.21.15.5:0/305372984 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4003d60 con 0x7f1acc163a10 2024-05-20T18:34:35.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.069+0000 7f1ad2157640 1 -- 172.21.15.5:0/305372984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1acc158e50 con 0x7f1acc163a10 2024-05-20T18:34:35.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.069+0000 7f1ad2157640 1 -- 172.21.15.5:0/305372984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1acc159410 con 0x7f1acc163a10 2024-05-20T18:34:35.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.070+0000 7f1ad2157640 1 -- 172.21.15.5:0/305372984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1acc075b00 con 0x7f1acc163a10 2024-05-20T18:34:35.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.070+0000 7f1ac8ff9640 1 -- 172.21.15.5:0/305372984 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1ac401d070 con 0x7f1acc163a10 2024-05-20T18:34:35.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.071+0000 7f1ac8ff9640 1 --2- 172.21.15.5:0/305372984 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1aac06be30 0x7f1aac06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.071+0000 7f1acb7fe640 1 --2- 172.21.15.5:0/305372984 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1aac06be30 0x7f1aac06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.071+0000 7f1ac8ff9640 1 -- 172.21.15.5:0/305372984 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6303+0+0 (secure 0 0 0) 0x7f1ac408b230 con 0x7f1acc163a10 2024-05-20T18:34:35.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.072+0000 7f1acb7fe640 1 --2- 172.21.15.5:0/305372984 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1aac06be30 0x7f1aac06e340 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f1acc0d3020 tx=0x7f1abc009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.076+0000 7f1ac8ff9640 1 -- 172.21.15.5:0/305372984 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1ac4010070 con 0x7f1acc163a10 2024-05-20T18:34:35.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.298+0000 7f1ad2157640 1 -- 172.21.15.5:0/305372984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1acc0d3050 con 0x7f1acc163a10 2024-05-20T18:34:35.299 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-20T18:34:35.299+0000 7fd1cf53a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-20T18:34:35.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.300+0000 7f1ac8ff9640 1 -- 172.21.15.5:0/305372984 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v68) ==== 161+0+0 (secure 0 0 0) 0x7f1ac4059340 con 0x7f1acc163a10 2024-05-20T18:34:35.301 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-20T18:34:35.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.303+0000 7f1ad2157640 1 -- 172.21.15.5:0/305372984 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1aac06be30 msgr2=0x7f1aac06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.303+0000 7f1ad2157640 1 --2- 172.21.15.5:0/305372984 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1aac06be30 0x7f1aac06e340 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f1acc0d3020 tx=0x7f1abc009190 comp rx=0 tx=0).stop 2024-05-20T18:34:35.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.303+0000 7f1ad2157640 1 -- 172.21.15.5:0/305372984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1acc163a10 msgr2=0x7f1acc0ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.303+0000 7f1ad2157640 1 --2- 172.21.15.5:0/305372984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1acc163a10 0x7f1acc0ce5b0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4007db0 tx=0x7f1ac400d4a0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.303+0000 7f1ad2157640 1 -- 172.21.15.5:0/305372984 shutdown_connections 2024-05-20T18:34:35.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.304+0000 7f1ad2157640 1 --2- 172.21.15.5:0/305372984 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1aac06be30 0x7f1aac06e340 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.304+0000 7f1ad2157640 1 --2- 172.21.15.5:0/305372984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1acc163a10 0x7f1acc0ce5b0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.304+0000 7f1ad2157640 1 --2- 172.21.15.5:0/305372984 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1acc0d14b0 0x7f1acc159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.304+0000 7f1ad2157640 1 --2- 172.21.15.5:0/305372984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1acc0b4010 0x7f1acc15ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.304+0000 7f1ad2157640 1 -- 172.21.15.5:0/305372984 >> 172.21.15.5:0/305372984 conn(0x7f1acc06d240 msgr2=0x7f1acc15dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:35.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.304+0000 7f1ad2157640 1 -- 172.21.15.5:0/305372984 shutdown_connections 2024-05-20T18:34:35.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.304+0000 7f1ad2157640 1 -- 172.21.15.5:0/305372984 wait complete. 2024-05-20T18:34:35.307 INFO:tasks.ceph.mds.c.smithi005.stderr: -4731> 2024-05-20T18:33:28.063+0000 7fce2507a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T18:34:35.307 INFO:tasks.ceph.mds.c.smithi005.stderr: -4262> 2024-05-20T18:33:29.941+0000 7fce21072640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-05-20T18:34:35.308 INFO:tasks.ceph.mds.c.smithi005.stderr: -4089> 2024-05-20T18:33:30.149+0000 7fce21072640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-05-20T18:33:20.165942+0000 1=1+0) n(v0 rc2024-05-20T18:33:20.285940+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x5555f6279b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-05-20T18:33:20.165942+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-05-20T18:33:20.285940+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-05-20T18:34:35.310 INFO:tasks.ceph.mds.c.smithi005.stderr: -2228> 2024-05-20T18:34:01.608+0000 7fce21072640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv20 ap=3 DIRTYPARENT f(v0 m2024-05-20T18:33:20.165942+0000 1=1+0)->f(v1 m2024-05-20T18:33:20.165942+0000) n(v0 rc2024-05-20T18:33:20.285940+0000 2=1+1)->n(v0 rc2024-05-20T18:33:20.285940+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x5555f6279b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-05-20T18:33:20.165942+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-05-20T18:33:20.285940+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-05-20T18:34:35.317 DEBUG:teuthology.orchestra.run.smithi005:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-05-20T18:34:35.365 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-05-20T18:34:35.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.371+0000 7fa203231740 1 -- start start 2024-05-20T18:34:35.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.371+0000 7fa203231740 1 Processor -- start 2024-05-20T18:34:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.372+0000 7fa203231740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fa557f7680 0x55fa557ca580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.372+0000 7fa203231740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa557f7b00 0x55fa557cab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.372+0000 7fa203231740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fa557f7f80 0x55fa557cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.372+0000 7fa203231740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fa56406600 con 0x55fa557f7b00 2024-05-20T18:34:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.372+0000 7fa203231740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fa56406480 con 0x55fa557f7f80 2024-05-20T18:34:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.372+0000 7fa203231740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55fa56406300 con 0x55fa557f7680 2024-05-20T18:34:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.372+0000 7fa20033e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa557f7b00 0x55fa557cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.372+0000 7fa20033e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa557f7b00 0x55fa557cab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38318/0 (socket says 172.21.15.5:38318) 2024-05-20T18:34:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.372+0000 7fa20033e640 1 -- 172.21.15.5:0/1600958899 learned_addr learned my addr 172.21.15.5:0/1600958899 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.372+0000 7fa201340640 1 --2- 172.21.15.5:0/1600958899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fa557f7f80 0x55fa557cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.372+0000 7fa200b3f640 1 --2- 172.21.15.5:0/1600958899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fa557f7680 0x55fa557ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.373+0000 7fa20033e640 1 -- 172.21.15.5:0/1600958899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fa557f7680 msgr2=0x55fa557ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.373+0000 7fa20033e640 1 --2- 172.21.15.5:0/1600958899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fa557f7680 0x55fa557ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.373+0000 7fa20033e640 1 -- 172.21.15.5:0/1600958899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fa557f7f80 msgr2=0x55fa557cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.373+0000 7fa20033e640 1 --2- 172.21.15.5:0/1600958899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fa557f7f80 0x55fa557cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.373+0000 7fa20033e640 1 -- 172.21.15.5:0/1600958899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa51639680 con 0x55fa557f7b00 2024-05-20T18:34:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.373+0000 7fa200b3f640 1 --2- 172.21.15.5:0/1600958899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fa557f7680 0x55fa557ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:34:35.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.373+0000 7fa201340640 1 --2- 172.21.15.5:0/1600958899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fa557f7f80 0x55fa557cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:34:35.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.373+0000 7fa20033e640 1 --2- 172.21.15.5:0/1600958899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa557f7b00 0x55fa557cab00 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x55fa56441170 tx=0x55fa55732840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3a2ba3f727521a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.373+0000 7fa1fe33a640 1 -- 172.21.15.5:0/1600958899 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55fa516329c0 con 0x55fa557f7b00 2024-05-20T18:34:35.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.373+0000 7fa1fe33a640 1 -- 172.21.15.5:0/1600958899 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55fa55686a80 con 0x55fa557f7b00 2024-05-20T18:34:35.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.373+0000 7fa1fe33a640 1 -- 172.21.15.5:0/1600958899 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55fa51632680 con 0x55fa557f7b00 2024-05-20T18:34:35.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.373+0000 7fa203231740 1 -- 172.21.15.5:0/1600958899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa51639860 con 0x55fa557f7b00 2024-05-20T18:34:35.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.373+0000 7fa203231740 1 -- 172.21.15.5:0/1600958899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55fa516392c0 con 0x55fa557f7b00 2024-05-20T18:34:35.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.373+0000 7fa203231740 4 waiting for MDS map... 2024-05-20T18:34:35.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.373+0000 7fa1fe33a640 1 -- 172.21.15.5:0/1600958899 <== mon.0 v2:172.21.15.5:3300/0 4 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x55fa56510200 con 0x55fa557f7b00 2024-05-20T18:34:35.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.373+0000 7fa1fe33a640 1 -- 172.21.15.5:0/1600958899 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 68) ==== 2885+0+0 (secure 0 0 0) 0x55fa55695180 con 0x55fa557f7b00 2024-05-20T18:34:35.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.373+0000 7fa203231740 4 Got MDS map 68 2024-05-20T18:34:35.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.374+0000 7fa203231740 4 datascan.main: Using metadata pool output 2024-05-20T18:34:35.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.374+0000 7fa203231740 4 datascan.main: connecting to RADOS... 2024-05-20T18:34:35.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.374+0000 7fa203231740 1 Processor -- start 2024-05-20T18:34:35.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.374+0000 7fa203231740 1 -- start start 2024-05-20T18:34:35.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.375+0000 7fa203231740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fa557f6d80 0x55fa557cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.375+0000 7fa203231740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa557f6900 0x55fa557cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.375+0000 7fa203231740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fa557f8880 0x55fa557cc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.375+0000 7fa203231740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fa56406180 con 0x55fa557f6900 2024-05-20T18:34:35.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.375+0000 7fa203231740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fa5569c480 con 0x55fa557f6d80 2024-05-20T18:34:35.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.375+0000 7fa203231740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55fa5569c900 con 0x55fa557f8880 2024-05-20T18:34:35.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.376+0000 7fa200b3f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa557f6900 0x55fa557cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.376+0000 7fa200b3f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa557f6900 0x55fa557cbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38324/0 (socket says 172.21.15.5:38324) 2024-05-20T18:34:35.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.376+0000 7fa200b3f640 1 -- 172.21.15.5:0/2504543023 learned_addr learned my addr 172.21.15.5:0/2504543023 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:35.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.376+0000 7fa201340640 1 --2- 172.21.15.5:0/2504543023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fa557f6d80 0x55fa557cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.376+0000 7fa20033e640 1 --2- 172.21.15.5:0/2504543023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fa557f8880 0x55fa557cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.376+0000 7fa200b3f640 1 -- 172.21.15.5:0/2504543023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fa557f8880 msgr2=0x55fa557cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:35.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.376+0000 7fa200b3f640 1 --2- 172.21.15.5:0/2504543023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fa557f8880 0x55fa557cc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.376+0000 7fa200b3f640 1 -- 172.21.15.5:0/2504543023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fa557f6d80 msgr2=0x55fa557cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:35.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.376+0000 7fa200b3f640 1 --2- 172.21.15.5:0/2504543023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fa557f6d80 0x55fa557cb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.376+0000 7fa200b3f640 1 -- 172.21.15.5:0/2504543023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa516394a0 con 0x55fa557f6900 2024-05-20T18:34:35.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.376+0000 7fa201340640 1 --2- 172.21.15.5:0/2504543023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fa557f6d80 0x55fa557cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:34:35.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.376+0000 7fa200b3f640 1 --2- 172.21.15.5:0/2504543023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa557f6900 0x55fa557cbb80 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x55fa56420ff0 tx=0x55fa56425800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc7d224b24f3110 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.376+0000 7fa1fc336640 1 -- 172.21.15.5:0/2504543023 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55fa516329c0 con 0x55fa557f6900 2024-05-20T18:34:35.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.376+0000 7fa1fc336640 1 -- 172.21.15.5:0/2504543023 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55fa55686540 con 0x55fa557f6900 2024-05-20T18:34:35.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.376+0000 7fa1fc336640 1 -- 172.21.15.5:0/2504543023 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55fa51632680 con 0x55fa557f6900 2024-05-20T18:34:35.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.377+0000 7fa203231740 1 -- 172.21.15.5:0/2504543023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa557f6900 msgr2=0x55fa557cbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.377+0000 7fa203231740 1 --2- 172.21.15.5:0/2504543023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa557f6900 0x55fa557cbb80 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x55fa56420ff0 tx=0x55fa56425800 comp rx=0 tx=0).stop 2024-05-20T18:34:35.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.377+0000 7fa203231740 1 -- 172.21.15.5:0/2504543023 shutdown_connections 2024-05-20T18:34:35.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.377+0000 7fa203231740 1 --2- 172.21.15.5:0/2504543023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fa557f8880 0x55fa557cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.377+0000 7fa203231740 1 --2- 172.21.15.5:0/2504543023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa557f6900 0x55fa557cbb80 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.377+0000 7fa203231740 1 --2- 172.21.15.5:0/2504543023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fa557f6d80 0x55fa557cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.377+0000 7fa203231740 1 -- 172.21.15.5:0/2504543023 >> 172.21.15.5:0/2504543023 conn(0x55fa557f7200 msgr2=0x55fa55762000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:35.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.377+0000 7fa203231740 1 -- 172.21.15.5:0/2504543023 shutdown_connections 2024-05-20T18:34:35.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.377+0000 7fa203231740 1 -- 172.21.15.5:0/2504543023 wait complete. 2024-05-20T18:34:35.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.378+0000 7fa203231740 1 Processor -- start 2024-05-20T18:34:35.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.378+0000 7fa203231740 1 -- start start 2024-05-20T18:34:35.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.379+0000 7fa203231740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa557f6900 0x55fa557cd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.379+0000 7fa203231740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fa557f6d80 0x55fa557ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.379+0000 7fa203231740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fa557f8880 0x55fa557cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.379+0000 7fa203231740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fa56407380 con 0x55fa557f6900 2024-05-20T18:34:35.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.379+0000 7fa203231740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fa56406180 con 0x55fa557f8880 2024-05-20T18:34:35.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.379+0000 7fa203231740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55fa56407500 con 0x55fa557f6d80 2024-05-20T18:34:35.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.379+0000 7fa201340640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa557f6900 0x55fa557cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.379+0000 7fa201340640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa557f6900 0x55fa557cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38340/0 (socket says 172.21.15.5:38340) 2024-05-20T18:34:35.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.379+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 learned_addr learned my addr 172.21.15.5:0/1194001475 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:35.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.379+0000 7fa200b3f640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fa557f6d80 0x55fa557ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.379+0000 7fa20033e640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fa557f8880 0x55fa557cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.379+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fa557f6d80 msgr2=0x55fa557ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.379+0000 7fa201340640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fa557f6d80 0x55fa557ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.379+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fa557f8880 msgr2=0x55fa557cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.379+0000 7fa201340640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fa557f8880 0x55fa557cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.379+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa516390e0 con 0x55fa557f6900 2024-05-20T18:34:35.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.379+0000 7fa20033e640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fa557f8880 0x55fa557cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:34:35.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.379+0000 7fa200b3f640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fa557f6d80 0x55fa557ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:34:35.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.379+0000 7fa201340640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa557f6900 0x55fa557cd180 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x55fa56421d70 tx=0x55fa56425920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.380+0000 7fa1fab33640 1 -- 172.21.15.5:0/1194001475 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55fa516329c0 con 0x55fa557f6900 2024-05-20T18:34:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.380+0000 7fa1fab33640 1 -- 172.21.15.5:0/1194001475 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55fa556861c0 con 0x55fa557f6900 2024-05-20T18:34:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.380+0000 7fa203231740 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fa51639a40 con 0x55fa557f6900 2024-05-20T18:34:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.380+0000 7fa1fab33640 1 -- 172.21.15.5:0/1194001475 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55fa51632680 con 0x55fa557f6900 2024-05-20T18:34:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.380+0000 7fa203231740 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa516383c0 con 0x55fa557f6900 2024-05-20T18:34:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.380+0000 7fa203231740 4 datascan.init: resolving metadata pool 13 2024-05-20T18:34:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.381+0000 7fa1fab33640 1 -- 172.21.15.5:0/1194001475 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x55fa5657cc00 con 0x55fa557f6900 2024-05-20T18:34:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.381+0000 7fa1fab33640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x55fa557f9a80 0x55fa557cd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.381+0000 7fa1fab33640 1 -- 172.21.15.5:0/1194001475 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x55fa5655da00 con 0x55fa557f6900 2024-05-20T18:34:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.381+0000 7fa203231740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-20T18:34:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.381+0000 7fa203231740 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-05-20T18:34:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.381+0000 7fa203231740 4 datascan.main: opening data pool 'cephfs_data' 2024-05-20T18:34:35.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.381+0000 7fa200b3f640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x55fa557f9a80 0x55fa557cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.382+0000 7fa203231740 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x55fa557f8d00 0x55fa557cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.382+0000 7fa203231740 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:1 14.0 14:00000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa56580c00 con 0x55fa557f8d00 2024-05-20T18:34:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.382+0000 7fa20033e640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x55fa557f8d00 0x55fa557cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.382+0000 7fa200b3f640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x55fa557f9a80 0x55fa557cd700 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x55fa56420ff0 tx=0x55fa55736240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.382+0000 7fa20033e640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x55fa557f8d00 0x55fa557cc100 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.384+0000 7fa20033e640 1 -- 172.21.15.5:0/1194001475 <== osd.2 v2:172.21.15.5:6800/921679764 1 ==== osd_op_reply(1 [pgnls start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a000 con 0x55fa557f8d00 2024-05-20T18:34:35.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.384+0000 7fa1fbb35640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x55fa55a18000 0x55fa55a20000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.384+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:2 14.20 14:04000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa56581400 con 0x55fa55a18000 2024-05-20T18:34:35.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.384+0000 7fa201340640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x55fa55a18000 0x55fa55a20000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.385+0000 7fa201340640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x55fa55a18000 0x55fa55a20000 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.385+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 <== osd.0 v2:172.21.15.5:6808/688093005 1 ==== osd_op_reply(2 [pgnls start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a500 con 0x55fa55a18000 2024-05-20T18:34:35.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.385+0000 7fa1fc336640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55fa55a18480 0x55fa55a20580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.385+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:3 14.10 14:08000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa56581800 con 0x55fa55a18480 2024-05-20T18:34:35.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.385+0000 7fa200b3f640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55fa55a18480 0x55fa55a20580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.385+0000 7fa200b3f640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55fa55a18480 0x55fa55a20580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.387+0000 7fa200b3f640 1 -- 172.21.15.5:0/1194001475 <== osd.1 v2:172.21.15.5:6815/550664725 1 ==== osd_op_reply(3 [pgnls start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a780 con 0x55fa55a18480 2024-05-20T18:34:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.387+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:4 14.30 14:0c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa56581c00 con 0x55fa55a18000 2024-05-20T18:34:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.388+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 <== osd.0 v2:172.21.15.5:6808/688093005 2 ==== osd_op_reply(4 [pgnls start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a500 con 0x55fa55a18000 2024-05-20T18:34:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.388+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:5 14.8 14:10000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a2c000 con 0x55fa55a18000 2024-05-20T18:34:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.388+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 <== osd.0 v2:172.21.15.5:6808/688093005 3 ==== osd_op_reply(5 [pgnls start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a500 con 0x55fa55a18000 2024-05-20T18:34:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.388+0000 7fa1fbb35640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x55fa55a18900 0x55fa55a20b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.388+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:6 14.28 14:14000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a2c800 con 0x55fa55a18900 2024-05-20T18:34:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.388+0000 7fa20033e640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x55fa55a18900 0x55fa55a20b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.389+0000 7fa20033e640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x55fa55a18900 0x55fa55a20b00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.391+0000 7fa20033e640 1 -- 172.21.15.5:0/1194001475 <== osd.3 v2:172.21.15.5:6824/1164130599 1 ==== osd_op_reply(6 [pgnls start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a000 con 0x55fa55a18900 2024-05-20T18:34:35.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.391+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:7 14.18 14:18000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a2c400 con 0x55fa55a18480 2024-05-20T18:34:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.391+0000 7fa200b3f640 1 -- 172.21.15.5:0/1194001475 <== osd.1 v2:172.21.15.5:6815/550664725 2 ==== osd_op_reply(7 [pgnls start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a780 con 0x55fa55a18480 2024-05-20T18:34:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.391+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:8 14.38 14:1c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a2cc00 con 0x55fa55a18480 2024-05-20T18:34:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.391+0000 7fa200b3f640 1 -- 172.21.15.5:0/1194001475 <== osd.1 v2:172.21.15.5:6815/550664725 3 ==== osd_op_reply(8 [pgnls start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a780 con 0x55fa55a18480 2024-05-20T18:34:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.391+0000 7fa1fc336640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x55fa55a19200 0x55fa55a21600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.391+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:9 14.4 14:20000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a2c400 con 0x55fa55a19200 2024-05-20T18:34:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.392+0000 7fa201340640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x55fa55a19200 0x55fa55a21600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.392+0000 7fa201340640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x55fa55a19200 0x55fa55a21600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.392+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 <== osd.6 v2:172.21.15.63:6816/3384565140 1 ==== osd_op_reply(9 [pgnls start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a500 con 0x55fa55a19200 2024-05-20T18:34:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.392+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:10 14.24 14:24000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a2d000 con 0x55fa55a18480 2024-05-20T18:34:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.393+0000 7fa200b3f640 1 -- 172.21.15.5:0/1194001475 <== osd.1 v2:172.21.15.5:6815/550664725 4 ==== osd_op_reply(10 [pgnls start_epoch 94 out=49b] v90'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a780 con 0x55fa55a18480 2024-05-20T18:34:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.393+0000 7fa1fc336640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x55fa55a19680 0x55fa55a21b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.393+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:11 14.14 14:28000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa56581800 con 0x55fa55a19680 2024-05-20T18:34:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.393+0000 7fa200b3f640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x55fa55a19680 0x55fa55a21b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.393+0000 7fa200b3f640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x55fa55a19680 0x55fa55a21b80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.394+0000 7fa200b3f640 1 -- 172.21.15.5:0/1194001475 <== osd.4 v2:172.21.15.63:6800/3673324406 1 ==== osd_op_reply(11 [pgnls start_epoch 94 out=49b] v90'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a780 con 0x55fa55a19680 2024-05-20T18:34:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.394+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:12 14.34 14:2c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a2c000 con 0x55fa557f8d00 2024-05-20T18:34:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.394+0000 7fa20033e640 1 -- 172.21.15.5:0/1194001475 <== osd.2 v2:172.21.15.5:6800/921679764 2 ==== osd_op_reply(12 [pgnls start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a000 con 0x55fa557f8d00 2024-05-20T18:34:35.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.394+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:13 14.c 14:30000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa56513c00 con 0x55fa55a18000 2024-05-20T18:34:35.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.394+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 <== osd.0 v2:172.21.15.5:6808/688093005 4 ==== osd_op_reply(13 [pgnls start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a500 con 0x55fa55a18000 2024-05-20T18:34:35.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.394+0000 7fa1fbb35640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x55fa55a18d80 0x55fa55a21080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.394+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:14 14.2c 14:34000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a2c400 con 0x55fa55a18d80 2024-05-20T18:34:35.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.395+0000 7fa20033e640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x55fa55a18d80 0x55fa55a21080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.395+0000 7fa20033e640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x55fa55a18d80 0x55fa55a21080 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.398+0000 7fa20033e640 1 -- 172.21.15.5:0/1194001475 <== osd.7 v2:172.21.15.63:6824/4136124451 1 ==== osd_op_reply(14 [pgnls start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a000 con 0x55fa55a18d80 2024-05-20T18:34:35.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.398+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:15 14.1c 14:38000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a2c800 con 0x55fa55a18000 2024-05-20T18:34:35.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.398+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 <== osd.0 v2:172.21.15.5:6808/688093005 5 ==== osd_op_reply(15 [pgnls start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a500 con 0x55fa55a18000 2024-05-20T18:34:35.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.398+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:16 14.3c 14:3c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa56581400 con 0x55fa55a18480 2024-05-20T18:34:35.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.398+0000 7fa200b3f640 1 -- 172.21.15.5:0/1194001475 <== osd.1 v2:172.21.15.5:6815/550664725 5 ==== osd_op_reply(16 [pgnls start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a780 con 0x55fa55a18480 2024-05-20T18:34:35.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.398+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:17 14.2 14:40000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa56580c00 con 0x55fa55a19200 2024-05-20T18:34:35.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.399+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 <== osd.6 v2:172.21.15.63:6816/3384565140 2 ==== osd_op_reply(17 [pgnls start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a500 con 0x55fa55a19200 2024-05-20T18:34:35.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.399+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:18 14.22 14:44000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa56581c00 con 0x55fa55a19680 2024-05-20T18:34:35.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.399+0000 7fa200b3f640 1 -- 172.21.15.5:0/1194001475 <== osd.4 v2:172.21.15.63:6800/3673324406 2 ==== osd_op_reply(18 [pgnls start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a780 con 0x55fa55a19680 2024-05-20T18:34:35.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.399+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:19 14.12 14:48000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa56512c00 con 0x55fa557f8d00 2024-05-20T18:34:35.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.399+0000 7fa20033e640 1 -- 172.21.15.5:0/1194001475 <== osd.2 v2:172.21.15.5:6800/921679764 3 ==== osd_op_reply(19 [pgnls start_epoch 94 out=49b] v90'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a000 con 0x55fa557f8d00 2024-05-20T18:34:35.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.399+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:20 14.32 14:4c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a2c000 con 0x55fa55a19680 2024-05-20T18:34:35.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.400+0000 7fa200b3f640 1 -- 172.21.15.5:0/1194001475 <== osd.4 v2:172.21.15.63:6800/3673324406 3 ==== osd_op_reply(20 [pgnls start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a780 con 0x55fa55a19680 2024-05-20T18:34:35.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.400+0000 7fa1fc336640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x55fa55a19b00 0x55fa55a22100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.400+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:21 14.a 14:50000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa56512800 con 0x55fa55a19b00 2024-05-20T18:34:35.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.400+0000 7fa201340640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x55fa55a19b00 0x55fa55a22100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.400+0000 7fa201340640 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x55fa55a19b00 0x55fa55a22100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.403+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 <== osd.5 v2:172.21.15.63:6808/1616066286 1 ==== osd_op_reply(21 [pgnls start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a500 con 0x55fa55a19b00 2024-05-20T18:34:35.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.403+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:22 14.2a 14:54000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a2c400 con 0x55fa55a19680 2024-05-20T18:34:35.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.403+0000 7fa200b3f640 1 -- 172.21.15.5:0/1194001475 <== osd.4 v2:172.21.15.63:6800/3673324406 4 ==== osd_op_reply(22 [pgnls start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a780 con 0x55fa55a19680 2024-05-20T18:34:35.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.403+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:23 14.1a 14:58000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a2d000 con 0x55fa55a18000 2024-05-20T18:34:35.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.403+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 <== osd.0 v2:172.21.15.5:6808/688093005 6 ==== osd_op_reply(23 [pgnls start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a500 con 0x55fa55a18000 2024-05-20T18:34:35.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.404+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:24 14.3a 14:5c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a2cc00 con 0x55fa55a18000 2024-05-20T18:34:35.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.404+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 <== osd.0 v2:172.21.15.5:6808/688093005 7 ==== osd_op_reply(24 [pgnls start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a500 con 0x55fa55a18000 2024-05-20T18:34:35.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.404+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:25 14.6 14:60000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a2c000 con 0x55fa55a18000 2024-05-20T18:34:35.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.404+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 <== osd.0 v2:172.21.15.5:6808/688093005 8 ==== osd_op_reply(25 [pgnls start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a500 con 0x55fa55a18000 2024-05-20T18:34:35.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.404+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:26 14.26 14:64000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a2d400 con 0x55fa55a19680 2024-05-20T18:34:35.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.404+0000 7fa200b3f640 1 -- 172.21.15.5:0/1194001475 <== osd.4 v2:172.21.15.63:6800/3673324406 5 ==== osd_op_reply(26 [pgnls start_epoch 94 out=49b] v90'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a780 con 0x55fa55a19680 2024-05-20T18:34:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.404+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:27 14.16 14:68000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa56581800 con 0x55fa55a18900 2024-05-20T18:34:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.405+0000 7fa20033e640 1 -- 172.21.15.5:0/1194001475 <== osd.3 v2:172.21.15.5:6824/1164130599 2 ==== osd_op_reply(27 [pgnls start_epoch 94 out=49b] v90'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a000 con 0x55fa55a18900 2024-05-20T18:34:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.405+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:28 14.36 14:6c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a2d800 con 0x55fa557f8d00 2024-05-20T18:34:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.405+0000 7fa20033e640 1 -- 172.21.15.5:0/1194001475 <== osd.2 v2:172.21.15.5:6800/921679764 4 ==== osd_op_reply(28 [pgnls start_epoch 94 out=49b] v90'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a000 con 0x55fa557f8d00 2024-05-20T18:34:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.405+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:29 14.e 14:70000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa56581400 con 0x55fa55a19680 2024-05-20T18:34:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.405+0000 7fa200b3f640 1 -- 172.21.15.5:0/1194001475 <== osd.4 v2:172.21.15.63:6800/3673324406 6 ==== osd_op_reply(29 [pgnls start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a780 con 0x55fa55a19680 2024-05-20T18:34:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.405+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:30 14.2e 14:74000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a2c800 con 0x55fa55a18900 2024-05-20T18:34:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.406+0000 7fa20033e640 1 -- 172.21.15.5:0/1194001475 <== osd.3 v2:172.21.15.5:6824/1164130599 3 ==== osd_op_reply(30 [pgnls start_epoch 94 out=49b] v90'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a000 con 0x55fa55a18900 2024-05-20T18:34:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.406+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:31 14.1e 14:78000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa56581c00 con 0x55fa55a18d80 2024-05-20T18:34:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.406+0000 7fa20033e640 1 -- 172.21.15.5:0/1194001475 <== osd.7 v2:172.21.15.63:6824/4136124451 2 ==== osd_op_reply(31 [pgnls start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a000 con 0x55fa55a18d80 2024-05-20T18:34:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.406+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:32 14.3e 14:7c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a2d000 con 0x55fa55a19b00 2024-05-20T18:34:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.407+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 <== osd.5 v2:172.21.15.63:6808/1616066286 2 ==== osd_op_reply(32 [pgnls start_epoch 94 out=49b] v90'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a500 con 0x55fa55a19b00 2024-05-20T18:34:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.407+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:33 14.1 14:80000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a54000 con 0x55fa55a18900 2024-05-20T18:34:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.407+0000 7fa20033e640 1 -- 172.21.15.5:0/1194001475 <== osd.3 v2:172.21.15.5:6824/1164130599 4 ==== osd_op_reply(33 [pgnls start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a000 con 0x55fa55a18900 2024-05-20T18:34:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.407+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:34 14.21 14:84000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a2dc00 con 0x55fa55a19b00 2024-05-20T18:34:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.407+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 <== osd.5 v2:172.21.15.63:6808/1616066286 3 ==== osd_op_reply(34 [pgnls start_epoch 94 out=49b] v90'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a500 con 0x55fa55a19b00 2024-05-20T18:34:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.407+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:35 14.11 14:88000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a54400 con 0x55fa55a19680 2024-05-20T18:34:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.408+0000 7fa200b3f640 1 -- 172.21.15.5:0/1194001475 <== osd.4 v2:172.21.15.63:6800/3673324406 7 ==== osd_op_reply(35 [pgnls start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a780 con 0x55fa55a19680 2024-05-20T18:34:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.408+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:36 14.31 14:8c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa56513c00 con 0x55fa55a18480 2024-05-20T18:34:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.408+0000 7fa200b3f640 1 -- 172.21.15.5:0/1194001475 <== osd.1 v2:172.21.15.5:6815/550664725 6 ==== osd_op_reply(36 [pgnls start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a780 con 0x55fa55a18480 2024-05-20T18:34:35.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.408+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:37 14.9 14:90000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a54800 con 0x55fa55a19b00 2024-05-20T18:34:35.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.408+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 <== osd.5 v2:172.21.15.63:6808/1616066286 4 ==== osd_op_reply(37 [pgnls start_epoch 94 out=49b] v90'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a500 con 0x55fa55a19b00 2024-05-20T18:34:35.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.408+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:38 14.29 14:94000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa56512800 con 0x55fa55a19200 2024-05-20T18:34:35.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.409+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 <== osd.6 v2:172.21.15.63:6816/3384565140 3 ==== osd_op_reply(38 [pgnls start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a500 con 0x55fa55a19200 2024-05-20T18:34:35.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.409+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:39 14.19 14:98000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a54c00 con 0x55fa55a19200 2024-05-20T18:34:35.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.409+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 <== osd.6 v2:172.21.15.63:6816/3384565140 4 ==== osd_op_reply(39 [pgnls start_epoch 94 out=49b] v90'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a500 con 0x55fa55a19200 2024-05-20T18:34:35.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.409+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:40 14.39 14:9c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa56580c00 con 0x55fa55a18d80 2024-05-20T18:34:35.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.410+0000 7fa20033e640 1 -- 172.21.15.5:0/1194001475 <== osd.7 v2:172.21.15.63:6824/4136124451 3 ==== osd_op_reply(40 [pgnls start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a000 con 0x55fa55a18d80 2024-05-20T18:34:35.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.410+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:41 14.5 14:a0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a55000 con 0x55fa55a18000 2024-05-20T18:34:35.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.410+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 <== osd.0 v2:172.21.15.5:6808/688093005 9 ==== osd_op_reply(41 [pgnls start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a500 con 0x55fa55a18000 2024-05-20T18:34:35.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.410+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:42 14.25 14:a4000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a2d800 con 0x55fa557f8d00 2024-05-20T18:34:35.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.410+0000 7fa20033e640 1 -- 172.21.15.5:0/1194001475 <== osd.2 v2:172.21.15.5:6800/921679764 5 ==== osd_op_reply(42 [pgnls start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a000 con 0x55fa557f8d00 2024-05-20T18:34:35.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.410+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:43 14.15 14:a8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a55400 con 0x55fa55a18d80 2024-05-20T18:34:35.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.411+0000 7fa20033e640 1 -- 172.21.15.5:0/1194001475 <== osd.7 v2:172.21.15.63:6824/4136124451 4 ==== osd_op_reply(43 [pgnls start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a000 con 0x55fa55a18d80 2024-05-20T18:34:35.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.411+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:44 14.35 14:ac000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a2c800 con 0x55fa55a19680 2024-05-20T18:34:35.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.411+0000 7fa200b3f640 1 -- 172.21.15.5:0/1194001475 <== osd.4 v2:172.21.15.63:6800/3673324406 8 ==== osd_op_reply(44 [pgnls start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a780 con 0x55fa55a19680 2024-05-20T18:34:35.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.411+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:45 14.d 14:b0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa56513c00 con 0x55fa55a19200 2024-05-20T18:34:35.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.412+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 <== osd.6 v2:172.21.15.63:6816/3384565140 5 ==== osd_op_reply(45 [pgnls start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a500 con 0x55fa55a19200 2024-05-20T18:34:35.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.412+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:46 14.2d 14:b4000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a2cc00 con 0x55fa55a19680 2024-05-20T18:34:35.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.412+0000 7fa200b3f640 1 -- 172.21.15.5:0/1194001475 <== osd.4 v2:172.21.15.63:6800/3673324406 9 ==== osd_op_reply(46 [pgnls start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a780 con 0x55fa55a19680 2024-05-20T18:34:35.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.412+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:47 14.1d 14:b8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a2c400 con 0x55fa55a18000 2024-05-20T18:34:35.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.412+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 <== osd.0 v2:172.21.15.5:6808/688093005 10 ==== osd_op_reply(47 [pgnls start_epoch 94 out=49b] v90'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a500 con 0x55fa55a18000 2024-05-20T18:34:35.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.412+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:48 14.3d 14:bc000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a2c000 con 0x55fa55a18900 2024-05-20T18:34:35.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.413+0000 7fa20033e640 1 -- 172.21.15.5:0/1194001475 <== osd.3 v2:172.21.15.5:6824/1164130599 5 ==== osd_op_reply(48 [pgnls start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a000 con 0x55fa55a18900 2024-05-20T18:34:35.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.413+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:49 14.3 14:c0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a2d400 con 0x55fa55a18000 2024-05-20T18:34:35.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.413+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 <== osd.0 v2:172.21.15.5:6808/688093005 11 ==== osd_op_reply(49 [pgnls start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a500 con 0x55fa55a18000 2024-05-20T18:34:35.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.413+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:50 14.23 14:c4000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa56581800 con 0x55fa55a18d80 2024-05-20T18:34:35.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.413+0000 7fa20033e640 1 -- 172.21.15.5:0/1194001475 <== osd.7 v2:172.21.15.63:6824/4136124451 5 ==== osd_op_reply(50 [pgnls start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a000 con 0x55fa55a18d80 2024-05-20T18:34:35.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.413+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:51 14.13 14:c8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a2c800 con 0x55fa557f8d00 2024-05-20T18:34:35.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.414+0000 7fa20033e640 1 -- 172.21.15.5:0/1194001475 <== osd.2 v2:172.21.15.5:6800/921679764 6 ==== osd_op_reply(51 [pgnls start_epoch 94 out=49b] v90'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a000 con 0x55fa557f8d00 2024-05-20T18:34:35.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.414+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:52 14.33 14:cc000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa56581c00 con 0x55fa55a18480 2024-05-20T18:34:35.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.414+0000 7fa200b3f640 1 -- 172.21.15.5:0/1194001475 <== osd.1 v2:172.21.15.5:6815/550664725 7 ==== osd_op_reply(52 [pgnls start_epoch 94 out=49b] v90'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a780 con 0x55fa55a18480 2024-05-20T18:34:35.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.414+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:53 14.b 14:d0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a2d000 con 0x55fa55a18000 2024-05-20T18:34:35.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.414+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 <== osd.0 v2:172.21.15.5:6808/688093005 12 ==== osd_op_reply(53 [pgnls start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a500 con 0x55fa55a18000 2024-05-20T18:34:35.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.414+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:54 14.2b 14:d4000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa56580c00 con 0x55fa55a19680 2024-05-20T18:34:35.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.415+0000 7fa200b3f640 1 -- 172.21.15.5:0/1194001475 <== osd.4 v2:172.21.15.63:6800/3673324406 10 ==== osd_op_reply(54 [pgnls start_epoch 94 out=49b] v90'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a780 con 0x55fa55a19680 2024-05-20T18:34:35.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.415+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:55 14.1b 14:d8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a2dc00 con 0x55fa55a18480 2024-05-20T18:34:35.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.415+0000 7fa200b3f640 1 -- 172.21.15.5:0/1194001475 <== osd.1 v2:172.21.15.5:6815/550664725 8 ==== osd_op_reply(55 [pgnls start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a780 con 0x55fa55a18480 2024-05-20T18:34:35.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.415+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:56 14.3b 14:dc000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a54400 con 0x55fa55a19680 2024-05-20T18:34:35.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.415+0000 7fa200b3f640 1 -- 172.21.15.5:0/1194001475 <== osd.4 v2:172.21.15.63:6800/3673324406 11 ==== osd_op_reply(56 [pgnls start_epoch 94 out=49b] v90'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a780 con 0x55fa55a19680 2024-05-20T18:34:35.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.415+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:57 14.7 14:e0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa56581400 con 0x55fa55a19b00 2024-05-20T18:34:35.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.416+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 <== osd.5 v2:172.21.15.63:6808/1616066286 5 ==== osd_op_reply(57 [pgnls start_epoch 94 out=81b] v90'6 uv6 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55fa5576a500 con 0x55fa55a19b00 2024-05-20T18:34:35.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.416+0000 7fa203231740 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:58 14.7 14:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x55fa56581000 con 0x55fa55a19b00 2024-05-20T18:34:35.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.416+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 <== osd.5 v2:172.21.15.63:6808/1616066286 6 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55fa5576a500 con 0x55fa55a19b00 2024-05-20T18:34:35.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.416+0000 7fa203231740 10 datascan.operator(): handling object 1099511627777.0 2024-05-20T18:34:35.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.416+0000 7fa203231740 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:59 14.7 14:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x55fa56513c00 con 0x55fa55a19b00 2024-05-20T18:34:35.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.418+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 <== osd.5 v2:172.21.15.63:6808/1616066286 7 ==== osd_op_reply(59 10000000001.00000000 [call] v94'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55fa5576a500 con 0x55fa55a19b00 2024-05-20T18:34:35.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.418+0000 7fa203231740 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:60 14.27 14:e4000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a54c00 con 0x55fa55a19b00 2024-05-20T18:34:35.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.418+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 <== osd.5 v2:172.21.15.63:6808/1616066286 8 ==== osd_op_reply(60 [pgnls start_epoch 94 out=49b] v90'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a500 con 0x55fa55a19b00 2024-05-20T18:34:35.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.419+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:61 14.17 14:e8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa56581400 con 0x55fa55a18480 2024-05-20T18:34:35.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.419+0000 7fa200b3f640 1 -- 172.21.15.5:0/1194001475 <== osd.1 v2:172.21.15.5:6815/550664725 9 ==== osd_op_reply(61 [pgnls start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a780 con 0x55fa55a18480 2024-05-20T18:34:35.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.419+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:62 14.37 14:ec000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a54000 con 0x55fa55a19680 2024-05-20T18:34:35.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.419+0000 7fa200b3f640 1 -- 172.21.15.5:0/1194001475 <== osd.4 v2:172.21.15.63:6800/3673324406 12 ==== osd_op_reply(62 [pgnls start_epoch 94 out=49b] v90'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a780 con 0x55fa55a19680 2024-05-20T18:34:35.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.419+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:63 14.f 14:f0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a2c400 con 0x55fa55a18d80 2024-05-20T18:34:35.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.420+0000 7fa20033e640 1 -- 172.21.15.5:0/1194001475 <== osd.7 v2:172.21.15.63:6824/4136124451 6 ==== osd_op_reply(63 [pgnls start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a000 con 0x55fa55a18d80 2024-05-20T18:34:35.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.420+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:64 14.2f 14:f4000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a54800 con 0x55fa55a18000 2024-05-20T18:34:35.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.420+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 <== osd.0 v2:172.21.15.5:6808/688093005 13 ==== osd_op_reply(64 [pgnls start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a500 con 0x55fa55a18000 2024-05-20T18:34:35.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.420+0000 7fa1fc336640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:65 14.1f 14:f8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a2d400 con 0x55fa55a18000 2024-05-20T18:34:35.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.420+0000 7fa201340640 1 -- 172.21.15.5:0/1194001475 <== osd.0 v2:172.21.15.5:6808/688093005 14 ==== osd_op_reply(65 [pgnls start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a500 con 0x55fa55a18000 2024-05-20T18:34:35.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.420+0000 7fa1fbb35640 1 -- 172.21.15.5:0/1194001475 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:66 14.3f 14:fc000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55fa55a55800 con 0x55fa55a18900 2024-05-20T18:34:35.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.421+0000 7fa20033e640 1 -- 172.21.15.5:0/1194001475 <== osd.3 v2:172.21.15.5:6824/1164130599 6 ==== osd_op_reply(66 [pgnls start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55fa5576a000 con 0x55fa55a18900 2024-05-20T18:34:35.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.421+0000 7fa203231740 1 -- 172.21.15.5:0/1194001475 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x55fa55a18000 msgr2=0x55fa55a20000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.421+0000 7fa203231740 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x55fa55a18000 0x55fa55a20000 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.421+0000 7fa203231740 1 -- 172.21.15.5:0/1194001475 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55fa55a18480 msgr2=0x55fa55a20580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.421+0000 7fa203231740 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55fa55a18480 0x55fa55a20580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.421+0000 7fa203231740 1 -- 172.21.15.5:0/1194001475 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x55fa557f8d00 msgr2=0x55fa557cc100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.421+0000 7fa203231740 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x55fa557f8d00 0x55fa557cc100 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.421+0000 7fa203231740 1 -- 172.21.15.5:0/1194001475 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x55fa55a18900 msgr2=0x55fa55a20b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.421+0000 7fa203231740 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x55fa55a18900 0x55fa55a20b00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.421+0000 7fa203231740 1 -- 172.21.15.5:0/1194001475 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x55fa55a19680 msgr2=0x55fa55a21b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.421+0000 7fa203231740 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x55fa55a19680 0x55fa55a21b80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.421+0000 7fa203231740 1 -- 172.21.15.5:0/1194001475 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x55fa55a19b00 msgr2=0x55fa55a22100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.421+0000 7fa203231740 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x55fa55a19b00 0x55fa55a22100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.421+0000 7fa203231740 1 -- 172.21.15.5:0/1194001475 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x55fa55a19200 msgr2=0x55fa55a21600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.421+0000 7fa203231740 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x55fa55a19200 0x55fa55a21600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.421+0000 7fa203231740 1 -- 172.21.15.5:0/1194001475 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x55fa55a18d80 msgr2=0x55fa55a21080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.421+0000 7fa203231740 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x55fa55a18d80 0x55fa55a21080 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.421+0000 7fa200b3f640 1 -- 172.21.15.5:0/1194001475 reap_dead start 2024-05-20T18:34:35.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.421+0000 7fa203231740 1 -- 172.21.15.5:0/1194001475 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x55fa557f9a80 msgr2=0x55fa557cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.421+0000 7fa203231740 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x55fa557f9a80 0x55fa557cd700 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x55fa56420ff0 tx=0x55fa55736240 comp rx=0 tx=0).stop 2024-05-20T18:34:35.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.421+0000 7fa200b3f640 1 -- 172.21.15.5:0/1194001475 reap_dead start 2024-05-20T18:34:35.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.421+0000 7fa203231740 1 -- 172.21.15.5:0/1194001475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa557f6900 msgr2=0x55fa557cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.421+0000 7fa203231740 1 --2- 172.21.15.5:0/1194001475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa557f6900 0x55fa557cd180 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x55fa56421d70 tx=0x55fa56425920 comp rx=0 tx=0).stop 2024-05-20T18:34:35.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.421+0000 7fa200b3f640 1 -- 172.21.15.5:0/1194001475 reap_dead start 2024-05-20T18:34:35.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.421+0000 7fa203231740 1 -- 172.21.15.5:0/1194001475 shutdown_connections 2024-05-20T18:34:35.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.421+0000 7fa203231740 1 -- 172.21.15.5:0/1194001475 >> 172.21.15.5:0/1194001475 conn(0x55fa557f7200 msgr2=0x55fa55762000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:35.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.421+0000 7fa203231740 1 -- 172.21.15.5:0/1194001475 shutdown_connections 2024-05-20T18:34:35.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.421+0000 7fa203231740 1 -- 172.21.15.5:0/1194001475 wait complete. 2024-05-20T18:34:35.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.422+0000 7fa203231740 1 -- 172.21.15.5:0/1600958899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa557f7b00 msgr2=0x55fa557cab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.422+0000 7fa203231740 1 --2- 172.21.15.5:0/1600958899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa557f7b00 0x55fa557cab00 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x55fa56441170 tx=0x55fa55732840 comp rx=0 tx=0).stop 2024-05-20T18:34:35.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.422+0000 7fa203231740 1 -- 172.21.15.5:0/1600958899 shutdown_connections 2024-05-20T18:34:35.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.422+0000 7fa203231740 1 --2- 172.21.15.5:0/1600958899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fa557f7f80 0x55fa557cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.422+0000 7fa203231740 1 --2- 172.21.15.5:0/1600958899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa557f7b00 0x55fa557cab00 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.422+0000 7fa203231740 1 --2- 172.21.15.5:0/1600958899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fa557f7680 0x55fa557ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.422+0000 7fa203231740 1 -- 172.21.15.5:0/1600958899 >> 172.21.15.5:0/1600958899 conn(0x55fa557f8400 msgr2=0x55fa55763000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:35.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.422+0000 7fa203231740 1 -- 172.21.15.5:0/1600958899 shutdown_connections 2024-05-20T18:34:35.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.422+0000 7fa203231740 1 -- 172.21.15.5:0/1600958899 wait complete. 2024-05-20T18:34:35.428 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.111366, result: 2024-05-20T18:34:35.429 DEBUG:teuthology.orchestra.run.smithi005:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes cephfs_data 2024-05-20T18:34:35.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.480+0000 7f3f19734740 1 -- start start 2024-05-20T18:34:35.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.480+0000 7f3f19734740 1 Processor -- start 2024-05-20T18:34:35.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.481+0000 7f3f19734740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5648b2b0b680 0x5648b2adeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.481+0000 7f3f19734740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648b2b0bb00 0x5648b2ade580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.481+0000 7f3f19734740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b2b0bf80 0x5648b2adf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.481+0000 7f3f19734740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5648b3730780 con 0x5648b2b0bb00 2024-05-20T18:34:35.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.481+0000 7f3f19734740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5648b3730600 con 0x5648b2b0bf80 2024-05-20T18:34:35.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.481+0000 7f3f19734740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5648b3730480 con 0x5648b2b0b680 2024-05-20T18:34:35.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.481+0000 7f3f16f6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648b2b0bb00 0x5648b2ade580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.481+0000 7f3f16f6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648b2b0bb00 0x5648b2ade580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38348/0 (socket says 172.21.15.5:38348) 2024-05-20T18:34:35.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.481+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3430690832 learned_addr learned my addr 172.21.15.5:0/3430690832 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:35.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.481+0000 7f3f1776c640 1 --2- 172.21.15.5:0/3430690832 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5648b2b0b680 0x5648b2adeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.481+0000 7f3f17f6d640 1 --2- 172.21.15.5:0/3430690832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b2b0bf80 0x5648b2adf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.481+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3430690832 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5648b2b0b680 msgr2=0x5648b2adeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:35.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.481+0000 7f3f16f6b640 1 --2- 172.21.15.5:0/3430690832 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5648b2b0b680 0x5648b2adeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.481+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3430690832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b2b0bf80 msgr2=0x5648b2adf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:35.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.481+0000 7f3f16f6b640 1 --2- 172.21.15.5:0/3430690832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b2b0bf80 0x5648b2adf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.481+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3430690832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648ae94d680 con 0x5648b2b0bb00 2024-05-20T18:34:35.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.482+0000 7f3f16f6b640 1 --2- 172.21.15.5:0/3430690832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648b2b0bb00 0x5648b2ade580 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x5648b3767d40 tx=0x5648b2a4a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2978e768dff0c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.482+0000 7f3f14f67640 1 -- 172.21.15.5:0/3430690832 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5648ae946680 con 0x5648b2b0bb00 2024-05-20T18:34:35.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.482+0000 7f3f14f67640 1 -- 172.21.15.5:0/3430690832 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5648b299ae00 con 0x5648b2b0bb00 2024-05-20T18:34:35.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.482+0000 7f3f14f67640 1 -- 172.21.15.5:0/3430690832 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5648ae946b60 con 0x5648b2b0bb00 2024-05-20T18:34:35.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.482+0000 7f3f19734740 1 -- 172.21.15.5:0/3430690832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648ae94d860 con 0x5648b2b0bb00 2024-05-20T18:34:35.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.482+0000 7f3f19734740 1 -- 172.21.15.5:0/3430690832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5648ae94d2c0 con 0x5648b2b0bb00 2024-05-20T18:34:35.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.482+0000 7f3f19734740 4 waiting for MDS map... 2024-05-20T18:34:35.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.482+0000 7f3f14f67640 1 -- 172.21.15.5:0/3430690832 <== mon.0 v2:172.21.15.5:3300/0 4 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x5648b3832a00 con 0x5648b2b0bb00 2024-05-20T18:34:35.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.482+0000 7f3f14f67640 1 -- 172.21.15.5:0/3430690832 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 68) ==== 2885+0+0 (secure 0 0 0) 0x5648b29a9180 con 0x5648b2b0bb00 2024-05-20T18:34:35.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.482+0000 7f3f19734740 4 Got MDS map 68 2024-05-20T18:34:35.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.482+0000 7f3f19734740 4 datascan.main: Using metadata pool output 2024-05-20T18:34:35.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.482+0000 7f3f19734740 4 datascan.main: connecting to RADOS... 2024-05-20T18:34:35.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.483+0000 7f3f19734740 1 Processor -- start 2024-05-20T18:34:35.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.483+0000 7f3f19734740 1 -- start start 2024-05-20T18:34:35.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.484+0000 7f3f19734740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b2b0ad80 0x5648b2adf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.484+0000 7f3f19734740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5648b2b0a900 0x5648b2adfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.484+0000 7f3f19734740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648b2b0c880 0x5648b2ae0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.484+0000 7f3f19734740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5648b3730300 con 0x5648b2b0c880 2024-05-20T18:34:35.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.484+0000 7f3f19734740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5648b3730000 con 0x5648b2b0ad80 2024-05-20T18:34:35.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.484+0000 7f3f19734740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5648b29b0900 con 0x5648b2b0a900 2024-05-20T18:34:35.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.484+0000 7f3f16f6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648b2b0c880 0x5648b2ae0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.484+0000 7f3f16f6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648b2b0c880 0x5648b2ae0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38364/0 (socket says 172.21.15.5:38364) 2024-05-20T18:34:35.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.484+0000 7f3f16f6b640 1 -- 172.21.15.5:0/2428432018 learned_addr learned my addr 172.21.15.5:0/2428432018 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:35.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.484+0000 7f3f1776c640 1 --2- 172.21.15.5:0/2428432018 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5648b2b0a900 0x5648b2adfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.484+0000 7f3f17f6d640 1 --2- 172.21.15.5:0/2428432018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b2b0ad80 0x5648b2adf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.485+0000 7f3f16f6b640 1 -- 172.21.15.5:0/2428432018 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5648b2b0a900 msgr2=0x5648b2adfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:35.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.485+0000 7f3f16f6b640 1 --2- 172.21.15.5:0/2428432018 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5648b2b0a900 0x5648b2adfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.485+0000 7f3f16f6b640 1 -- 172.21.15.5:0/2428432018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b2b0ad80 msgr2=0x5648b2adf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:35.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.485+0000 7f3f16f6b640 1 --2- 172.21.15.5:0/2428432018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b2b0ad80 0x5648b2adf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.485+0000 7f3f16f6b640 1 -- 172.21.15.5:0/2428432018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648ae94d4a0 con 0x5648b2b0c880 2024-05-20T18:34:35.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.485+0000 7f3f16f6b640 1 --2- 172.21.15.5:0/2428432018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648b2b0c880 0x5648b2ae0100 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x5648b3820b70 tx=0x5648b2a46ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8585b4b6045c6ca server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.485+0000 7f3f12f63640 1 -- 172.21.15.5:0/2428432018 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5648ae946820 con 0x5648b2b0c880 2024-05-20T18:34:35.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.485+0000 7f3f12f63640 1 -- 172.21.15.5:0/2428432018 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5648b299ac40 con 0x5648b2b0c880 2024-05-20T18:34:35.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.485+0000 7f3f12f63640 1 -- 172.21.15.5:0/2428432018 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5648ae946680 con 0x5648b2b0c880 2024-05-20T18:34:35.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.486+0000 7f3f19734740 1 -- 172.21.15.5:0/2428432018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648b2b0c880 msgr2=0x5648b2ae0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.486+0000 7f3f19734740 1 --2- 172.21.15.5:0/2428432018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648b2b0c880 0x5648b2ae0100 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x5648b3820b70 tx=0x5648b2a46ba0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.486+0000 7f3f19734740 1 -- 172.21.15.5:0/2428432018 shutdown_connections 2024-05-20T18:34:35.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.486+0000 7f3f19734740 1 --2- 172.21.15.5:0/2428432018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648b2b0c880 0x5648b2ae0100 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.486+0000 7f3f19734740 1 --2- 172.21.15.5:0/2428432018 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5648b2b0a900 0x5648b2adfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.486+0000 7f3f19734740 1 --2- 172.21.15.5:0/2428432018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b2b0ad80 0x5648b2adf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.486+0000 7f3f19734740 1 -- 172.21.15.5:0/2428432018 >> 172.21.15.5:0/2428432018 conn(0x5648b2b0b200 msgr2=0x5648b2a79800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:35.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.486+0000 7f3f19734740 1 -- 172.21.15.5:0/2428432018 shutdown_connections 2024-05-20T18:34:35.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.486+0000 7f3f19734740 1 -- 172.21.15.5:0/2428432018 wait complete. 2024-05-20T18:34:35.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.487+0000 7f3f19734740 1 Processor -- start 2024-05-20T18:34:35.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.487+0000 7f3f19734740 1 -- start start 2024-05-20T18:34:35.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.488+0000 7f3f19734740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5648b2b0ad80 0x5648b2ae1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.488+0000 7f3f19734740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648b2b0c880 0x5648b2ae0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.488+0000 7f3f19734740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b2b0cd00 0x5648b2ae0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.488+0000 7f3f19734740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5648b3731680 con 0x5648b2b0c880 2024-05-20T18:34:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.488+0000 7f3f19734740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5648b3731800 con 0x5648b2b0cd00 2024-05-20T18:34:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.488+0000 7f3f19734740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5648b3731500 con 0x5648b2b0ad80 2024-05-20T18:34:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.488+0000 7f3f1776c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648b2b0c880 0x5648b2ae0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.488+0000 7f3f1776c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648b2b0c880 0x5648b2ae0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38366/0 (socket says 172.21.15.5:38366) 2024-05-20T18:34:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.488+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 learned_addr learned my addr 172.21.15.5:0/3937564835 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.488+0000 7f3f17f6d640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5648b2b0ad80 0x5648b2ae1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.488+0000 7f3f16f6b640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b2b0cd00 0x5648b2ae0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.488+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5648b2b0ad80 msgr2=0x5648b2ae1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.488+0000 7f3f1776c640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5648b2b0ad80 0x5648b2ae1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.488+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b2b0cd00 msgr2=0x5648b2ae0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.488+0000 7f3f1776c640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b2b0cd00 0x5648b2ae0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.488+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648ae94d680 con 0x5648b2b0c880 2024-05-20T18:34:35.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.488+0000 7f3f17f6d640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5648b2b0ad80 0x5648b2ae1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:34:35.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.488+0000 7f3f16f6b640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b2b0cd00 0x5648b2ae0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:34:35.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.488+0000 7f3f1776c640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648b2b0c880 0x5648b2ae0c00 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x5648b38ae030 tx=0x5648b3845a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.489+0000 7f3f11760640 1 -- 172.21.15.5:0/3937564835 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5648ae946820 con 0x5648b2b0c880 2024-05-20T18:34:35.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.489+0000 7f3f11760640 1 -- 172.21.15.5:0/3937564835 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5648b299a540 con 0x5648b2b0c880 2024-05-20T18:34:35.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.489+0000 7f3f19734740 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5648ae94d860 con 0x5648b2b0c880 2024-05-20T18:34:35.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.489+0000 7f3f11760640 1 -- 172.21.15.5:0/3937564835 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5648ae946680 con 0x5648b2b0c880 2024-05-20T18:34:35.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.489+0000 7f3f19734740 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648ae94d0e0 con 0x5648b2b0c880 2024-05-20T18:34:35.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.489+0000 7f3f19734740 4 datascan.init: resolving metadata pool 13 2024-05-20T18:34:35.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.490+0000 7f3f11760640 1 -- 172.21.15.5:0/3937564835 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x5648b389cc00 con 0x5648b2b0c880 2024-05-20T18:34:35.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.490+0000 7f3f11760640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x5648b2b0b200 0x5648b2ae1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.490+0000 7f3f11760640 1 -- 172.21.15.5:0/3937564835 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x5648b38aca00 con 0x5648b2b0c880 2024-05-20T18:34:35.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.490+0000 7f3f17f6d640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x5648b2b0b200 0x5648b2ae1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.490+0000 7f3f19734740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-20T18:34:35.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.490+0000 7f3f19734740 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-05-20T18:34:35.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.490+0000 7f3f19734740 4 datascan.main: opening data pool 'cephfs_data' 2024-05-20T18:34:35.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.490+0000 7f3f19734740 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x5648b2b0d180 0x5648b2ae0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.490+0000 7f3f19734740 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5648b38b0800 con 0x5648b2b0d180 2024-05-20T18:34:35.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.491+0000 7f3f16f6b640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x5648b2b0d180 0x5648b2ae0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.491+0000 7f3f17f6d640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x5648b2b0b200 0x5648b2ae1700 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x5648b2a48690 tx=0x5648b2a46960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.491+0000 7f3f16f6b640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x5648b2b0d180 0x5648b2ae0100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.491+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.0 v2:172.21.15.5:6808/688093005 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv25 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2b0d180 2024-05-20T18:34:35.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.491+0000 7f3f19734740 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x5648b2b0d600 0x5648b2adf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.491+0000 7f3f19734740 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:2 13.1f 13:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5648b38b0c00 con 0x5648b2b0d600 2024-05-20T18:34:35.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.492+0000 7f3f17f6d640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x5648b2b0d600 0x5648b2adf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.492+0000 7f3f17f6d640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x5648b2b0d600 0x5648b2adf600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.492+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.6 v2:172.21.15.63:6816/3384565140 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2b0d600 2024-05-20T18:34:35.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.492+0000 7f3f19734740 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x5648b2b0da80 0x5648b2adfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.492+0000 7f3f19734740 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:3 14.0 14:00000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b38b1000 con 0x5648b2b0da80 2024-05-20T18:34:35.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.493+0000 7f3f1776c640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x5648b2b0da80 0x5648b2adfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.493+0000 7f3f1776c640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x5648b2b0da80 0x5648b2adfb80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.493+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.2 v2:172.21.15.5:6800/921679764 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2b0da80 2024-05-20T18:34:35.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.493+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:4 14.20 14:04000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b38b1c00 con 0x5648b2b0d180 2024-05-20T18:34:35.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.494+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.0 v2:172.21.15.5:6808/688093005 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2b0d180 2024-05-20T18:34:35.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.494+0000 7f3f12f63640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x5648b2d50000 0x5648b2d58000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.494+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:5 14.10 14:08000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d5c000 con 0x5648b2d50000 2024-05-20T18:34:35.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.494+0000 7f3f16f6b640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x5648b2d50000 0x5648b2d58000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.494+0000 7f3f16f6b640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x5648b2d50000 0x5648b2d58000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.494+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.1 v2:172.21.15.5:6815/550664725 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d50000 2024-05-20T18:34:35.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.494+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:6 14.30 14:0c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b382d000 con 0x5648b2b0d180 2024-05-20T18:34:35.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.495+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.0 v2:172.21.15.5:6808/688093005 3 ==== osd_op_reply(6 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2b0d180 2024-05-20T18:34:35.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.495+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:7 14.8 14:10000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b38b0800 con 0x5648b2b0d180 2024-05-20T18:34:35.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.495+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.0 v2:172.21.15.5:6808/688093005 4 ==== osd_op_reply(7 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2b0d180 2024-05-20T18:34:35.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.495+0000 7f3f12762640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x5648b2d50480 0x5648b2d58580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.495+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:8 14.28 14:14000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b382d400 con 0x5648b2d50480 2024-05-20T18:34:35.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.495+0000 7f3f17f6d640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x5648b2d50480 0x5648b2d58580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.495+0000 7f3f17f6d640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x5648b2d50480 0x5648b2d58580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.496+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.3 v2:172.21.15.5:6824/1164130599 1 ==== osd_op_reply(8 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2d50480 2024-05-20T18:34:35.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.496+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:9 14.18 14:18000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b38b1c00 con 0x5648b2d50000 2024-05-20T18:34:35.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.496+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.1 v2:172.21.15.5:6815/550664725 2 ==== osd_op_reply(9 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d50000 2024-05-20T18:34:35.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.496+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:10 14.38 14:1c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d60000 con 0x5648b2d50000 2024-05-20T18:34:35.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.496+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.1 v2:172.21.15.5:6815/550664725 3 ==== osd_op_reply(10 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d50000 2024-05-20T18:34:35.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.496+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:11 14.4 14:20000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d60c00 con 0x5648b2b0d600 2024-05-20T18:34:35.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.497+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.6 v2:172.21.15.63:6816/3384565140 2 ==== osd_op_reply(11 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2b0d600 2024-05-20T18:34:35.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.497+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:12 14.24 14:24000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d60400 con 0x5648b2d50000 2024-05-20T18:34:35.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.497+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.1 v2:172.21.15.5:6815/550664725 4 ==== osd_op_reply(12 [pgnls-filter start_epoch 94 out=49b] v90'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d50000 2024-05-20T18:34:35.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.497+0000 7f3f12f63640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x5648b2d50900 0x5648b2d58b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.497+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:13 14.14 14:28000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d61000 con 0x5648b2d50900 2024-05-20T18:34:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.497+0000 7f3f1776c640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x5648b2d50900 0x5648b2d58b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.498+0000 7f3f1776c640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x5648b2d50900 0x5648b2d58b00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.498+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.4 v2:172.21.15.63:6800/3673324406 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 94 out=49b] v90'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2d50900 2024-05-20T18:34:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.498+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:14 14.34 14:2c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d60800 con 0x5648b2b0da80 2024-05-20T18:34:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.499+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.2 v2:172.21.15.5:6800/921679764 2 ==== osd_op_reply(14 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2b0da80 2024-05-20T18:34:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.499+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:15 14.c 14:30000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d61400 con 0x5648b2b0d180 2024-05-20T18:34:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.499+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.0 v2:172.21.15.5:6808/688093005 5 ==== osd_op_reply(15 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2b0d180 2024-05-20T18:34:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.499+0000 7f3f12762640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x5648b2d51200 0x5648b2d59600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.499+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:16 14.2c 14:34000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d60000 con 0x5648b2d51200 2024-05-20T18:34:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.499+0000 7f3f16f6b640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x5648b2d51200 0x5648b2d59600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.500+0000 7f3f16f6b640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x5648b2d51200 0x5648b2d59600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.500+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.7 v2:172.21.15.63:6824/4136124451 1 ==== osd_op_reply(16 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d51200 2024-05-20T18:34:35.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.500+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:17 14.1c 14:38000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d60400 con 0x5648b2b0d180 2024-05-20T18:34:35.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.500+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.0 v2:172.21.15.5:6808/688093005 6 ==== osd_op_reply(17 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2b0d180 2024-05-20T18:34:35.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.500+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:18 14.3c 14:3c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b38b0800 con 0x5648b2d50000 2024-05-20T18:34:35.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.501+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.1 v2:172.21.15.5:6815/550664725 5 ==== osd_op_reply(18 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d50000 2024-05-20T18:34:35.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.501+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:19 14.2 14:40000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d61400 con 0x5648b2b0d600 2024-05-20T18:34:35.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.501+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.6 v2:172.21.15.63:6816/3384565140 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2b0d600 2024-05-20T18:34:35.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.501+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:20 14.22 14:44000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b38b1c00 con 0x5648b2d50900 2024-05-20T18:34:35.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.501+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.4 v2:172.21.15.63:6800/3673324406 2 ==== osd_op_reply(20 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2d50900 2024-05-20T18:34:35.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.501+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:21 14.12 14:48000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d61800 con 0x5648b2b0da80 2024-05-20T18:34:35.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.502+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.2 v2:172.21.15.5:6800/921679764 3 ==== osd_op_reply(21 [pgnls-filter start_epoch 94 out=49b] v90'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2b0da80 2024-05-20T18:34:35.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.502+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:22 14.32 14:4c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b382d000 con 0x5648b2d50900 2024-05-20T18:34:35.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.502+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.4 v2:172.21.15.63:6800/3673324406 3 ==== osd_op_reply(22 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2d50900 2024-05-20T18:34:35.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.502+0000 7f3f12f63640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x5648b2d50d80 0x5648b2d59080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.502+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:23 14.a 14:50000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d61c00 con 0x5648b2d50d80 2024-05-20T18:34:35.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.502+0000 7f3f17f6d640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x5648b2d50d80 0x5648b2d59080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.503+0000 7f3f17f6d640 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x5648b2d50d80 0x5648b2d59080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.503+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.5 v2:172.21.15.63:6808/1616066286 1 ==== osd_op_reply(23 [pgnls-filter start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2d50d80 2024-05-20T18:34:35.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.503+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:24 14.2a 14:54000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d60c00 con 0x5648b2d50900 2024-05-20T18:34:35.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.504+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.4 v2:172.21.15.63:6800/3673324406 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2d50900 2024-05-20T18:34:35.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.504+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:25 14.1a 14:58000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b38b0c00 con 0x5648b2b0d180 2024-05-20T18:34:35.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.504+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.0 v2:172.21.15.5:6808/688093005 7 ==== osd_op_reply(25 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2b0d180 2024-05-20T18:34:35.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.504+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:26 14.3a 14:5c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d61400 con 0x5648b2b0d180 2024-05-20T18:34:35.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.504+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.0 v2:172.21.15.5:6808/688093005 8 ==== osd_op_reply(26 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2b0d180 2024-05-20T18:34:35.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.504+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:27 14.6 14:60000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b38b1000 con 0x5648b2b0d180 2024-05-20T18:34:35.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.504+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.0 v2:172.21.15.5:6808/688093005 9 ==== osd_op_reply(27 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2b0d180 2024-05-20T18:34:35.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.504+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:28 14.26 14:64000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d61c00 con 0x5648b2d50900 2024-05-20T18:34:35.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.505+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.4 v2:172.21.15.63:6800/3673324406 5 ==== osd_op_reply(28 [pgnls-filter start_epoch 94 out=49b] v90'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2d50900 2024-05-20T18:34:35.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.505+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:29 14.16 14:68000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b38b1c00 con 0x5648b2d50480 2024-05-20T18:34:35.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.505+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.3 v2:172.21.15.5:6824/1164130599 2 ==== osd_op_reply(29 [pgnls-filter start_epoch 94 out=49b] v90'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2d50480 2024-05-20T18:34:35.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.505+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:30 14.36 14:6c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d61000 con 0x5648b2b0da80 2024-05-20T18:34:35.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.505+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.2 v2:172.21.15.5:6800/921679764 4 ==== osd_op_reply(30 [pgnls-filter start_epoch 94 out=49b] v90'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2b0da80 2024-05-20T18:34:35.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.506+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:31 14.e 14:70000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d60800 con 0x5648b2d50900 2024-05-20T18:34:35.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.506+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.4 v2:172.21.15.63:6800/3673324406 6 ==== osd_op_reply(31 [pgnls-filter start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2d50900 2024-05-20T18:34:35.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.506+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:32 14.2e 14:74000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b382d400 con 0x5648b2d50480 2024-05-20T18:34:35.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.506+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.3 v2:172.21.15.5:6824/1164130599 3 ==== osd_op_reply(32 [pgnls-filter start_epoch 94 out=49b] v90'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2d50480 2024-05-20T18:34:35.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.506+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:33 14.1e 14:78000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d61800 con 0x5648b2d51200 2024-05-20T18:34:35.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.507+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.7 v2:172.21.15.63:6824/4136124451 2 ==== osd_op_reply(33 [pgnls-filter start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d51200 2024-05-20T18:34:35.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.507+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:34 14.3e 14:7c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d60000 con 0x5648b2d50d80 2024-05-20T18:34:35.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.507+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.5 v2:172.21.15.63:6808/1616066286 2 ==== osd_op_reply(34 [pgnls-filter start_epoch 94 out=49b] v90'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2d50d80 2024-05-20T18:34:35.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.507+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:35 14.1 14:80000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d60c00 con 0x5648b2d50480 2024-05-20T18:34:35.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.507+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.3 v2:172.21.15.5:6824/1164130599 4 ==== osd_op_reply(35 [pgnls-filter start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2d50480 2024-05-20T18:34:35.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.507+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:36 14.21 14:84000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d60400 con 0x5648b2d50d80 2024-05-20T18:34:35.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.508+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.5 v2:172.21.15.63:6808/1616066286 3 ==== osd_op_reply(36 [pgnls-filter start_epoch 94 out=49b] v90'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2d50d80 2024-05-20T18:34:35.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.508+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:37 14.11 14:88000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d61c00 con 0x5648b2d50900 2024-05-20T18:34:35.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.508+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.4 v2:172.21.15.63:6800/3673324406 7 ==== osd_op_reply(37 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2d50900 2024-05-20T18:34:35.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.508+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:38 14.31 14:8c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d61400 con 0x5648b2d50000 2024-05-20T18:34:35.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.509+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.1 v2:172.21.15.5:6815/550664725 6 ==== osd_op_reply(38 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d50000 2024-05-20T18:34:35.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.509+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:39 14.9 14:90000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d61000 con 0x5648b2d50d80 2024-05-20T18:34:35.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.509+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.5 v2:172.21.15.63:6808/1616066286 4 ==== osd_op_reply(39 [pgnls-filter start_epoch 94 out=49b] v90'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2d50d80 2024-05-20T18:34:35.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.509+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:40 14.29 14:94000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b38b0800 con 0x5648b2b0d600 2024-05-20T18:34:35.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.509+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.6 v2:172.21.15.63:6816/3384565140 4 ==== osd_op_reply(40 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2b0d600 2024-05-20T18:34:35.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.510+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:41 14.19 14:98000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b382d000 con 0x5648b2b0d600 2024-05-20T18:34:35.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.510+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.6 v2:172.21.15.63:6816/3384565140 5 ==== osd_op_reply(41 [pgnls-filter start_epoch 94 out=49b] v90'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2b0d600 2024-05-20T18:34:35.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.510+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:42 14.39 14:9c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b38b0c00 con 0x5648b2d51200 2024-05-20T18:34:35.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.510+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.7 v2:172.21.15.63:6824/4136124451 3 ==== osd_op_reply(42 [pgnls-filter start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d51200 2024-05-20T18:34:35.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.510+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:43 14.5 14:a0000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d60000 con 0x5648b2b0d180 2024-05-20T18:34:35.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.511+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.0 v2:172.21.15.5:6808/688093005 10 ==== osd_op_reply(43 [pgnls-filter start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2b0d180 2024-05-20T18:34:35.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.511+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:44 14.25 14:a4000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b38b1000 con 0x5648b2b0da80 2024-05-20T18:34:35.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.511+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.2 v2:172.21.15.5:6800/921679764 5 ==== osd_op_reply(44 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2b0da80 2024-05-20T18:34:35.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.511+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:45 14.15 14:a8000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d60c00 con 0x5648b2d51200 2024-05-20T18:34:35.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.511+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.7 v2:172.21.15.63:6824/4136124451 4 ==== osd_op_reply(45 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d51200 2024-05-20T18:34:35.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.511+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:46 14.35 14:ac000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b374a000 con 0x5648b2d50900 2024-05-20T18:34:35.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.512+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.4 v2:172.21.15.63:6800/3673324406 8 ==== osd_op_reply(46 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2d50900 2024-05-20T18:34:35.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.512+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:47 14.d 14:b0000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d60400 con 0x5648b2b0d600 2024-05-20T18:34:35.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.512+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.6 v2:172.21.15.63:6816/3384565140 6 ==== osd_op_reply(47 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2b0d600 2024-05-20T18:34:35.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.512+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:48 14.2d 14:b4000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b374a400 con 0x5648b2d50900 2024-05-20T18:34:35.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.513+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.4 v2:172.21.15.63:6800/3673324406 9 ==== osd_op_reply(48 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2d50900 2024-05-20T18:34:35.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.513+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:49 14.1d 14:b8000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d61000 con 0x5648b2b0d180 2024-05-20T18:34:35.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.513+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.0 v2:172.21.15.5:6808/688093005 11 ==== osd_op_reply(49 [pgnls-filter start_epoch 94 out=49b] v90'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2b0d180 2024-05-20T18:34:35.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.513+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:50 14.3d 14:bc000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b374a800 con 0x5648b2d50480 2024-05-20T18:34:35.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.513+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.3 v2:172.21.15.5:6824/1164130599 5 ==== osd_op_reply(50 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2d50480 2024-05-20T18:34:35.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.514+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:51 14.3 14:c0000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d61800 con 0x5648b2b0d180 2024-05-20T18:34:35.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.514+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.0 v2:172.21.15.5:6808/688093005 12 ==== osd_op_reply(51 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2b0d180 2024-05-20T18:34:35.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.514+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:52 14.23 14:c4000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b374ac00 con 0x5648b2d51200 2024-05-20T18:34:35.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.514+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.7 v2:172.21.15.63:6824/4136124451 5 ==== osd_op_reply(52 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d51200 2024-05-20T18:34:35.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.514+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:53 14.13 14:c8000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b382d400 con 0x5648b2b0da80 2024-05-20T18:34:35.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.514+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.2 v2:172.21.15.5:6800/921679764 6 ==== osd_op_reply(53 [pgnls-filter start_epoch 94 out=49b] v90'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2b0da80 2024-05-20T18:34:35.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.515+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:54 14.33 14:cc000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b374b000 con 0x5648b2d50000 2024-05-20T18:34:35.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.515+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.1 v2:172.21.15.5:6815/550664725 7 ==== osd_op_reply(54 [pgnls-filter start_epoch 94 out=49b] v90'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d50000 2024-05-20T18:34:35.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.515+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:55 14.b 14:d0000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b38b1c00 con 0x5648b2b0d180 2024-05-20T18:34:35.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.515+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.0 v2:172.21.15.5:6808/688093005 13 ==== osd_op_reply(55 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2b0d180 2024-05-20T18:34:35.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.515+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:56 14.2b 14:d4000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b374b400 con 0x5648b2d50900 2024-05-20T18:34:35.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.516+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.4 v2:172.21.15.63:6800/3673324406 10 ==== osd_op_reply(56 [pgnls-filter start_epoch 94 out=49b] v90'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2d50900 2024-05-20T18:34:35.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.516+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:57 14.1b 14:d8000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b38b0c00 con 0x5648b2d50000 2024-05-20T18:34:35.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.516+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.1 v2:172.21.15.5:6815/550664725 8 ==== osd_op_reply(57 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d50000 2024-05-20T18:34:35.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.516+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:58 14.3b 14:dc000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b374b800 con 0x5648b2d50900 2024-05-20T18:34:35.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.516+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.4 v2:172.21.15.63:6800/3673324406 11 ==== osd_op_reply(58 [pgnls-filter start_epoch 94 out=49b] v90'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2d50900 2024-05-20T18:34:35.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.516+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:59 14.7 14:e0000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d61400 con 0x5648b2d50d80 2024-05-20T18:34:35.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.517+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.5 v2:172.21.15.63:6808/1616066286 5 ==== osd_op_reply(59 [pgnls-filter start_epoch 94 out=81b] v94'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2d50d80 2024-05-20T18:34:35.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.517+0000 7f3f19734740 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:60 14.0 14:00000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b38b1400 con 0x5648b2b0da80 2024-05-20T18:34:35.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.517+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.2 v2:172.21.15.5:6800/921679764 7 ==== osd_op_reply(60 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2b0da80 2024-05-20T18:34:35.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.517+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:61 14.20 14:04000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d60000 con 0x5648b2b0d180 2024-05-20T18:34:35.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.517+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.0 v2:172.21.15.5:6808/688093005 14 ==== osd_op_reply(61 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2b0d180 2024-05-20T18:34:35.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.517+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:62 14.10 14:08000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b374ac00 con 0x5648b2d50000 2024-05-20T18:34:35.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.518+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.1 v2:172.21.15.5:6815/550664725 9 ==== osd_op_reply(62 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d50000 2024-05-20T18:34:35.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.518+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:63 14.30 14:0c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d60c00 con 0x5648b2b0d180 2024-05-20T18:34:35.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.518+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.0 v2:172.21.15.5:6808/688093005 15 ==== osd_op_reply(63 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2b0d180 2024-05-20T18:34:35.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.518+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:64 14.8 14:10000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b374b000 con 0x5648b2b0d180 2024-05-20T18:34:35.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.518+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.0 v2:172.21.15.5:6808/688093005 16 ==== osd_op_reply(64 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2b0d180 2024-05-20T18:34:35.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.518+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:65 14.28 14:14000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d61000 con 0x5648b2d50480 2024-05-20T18:34:35.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.518+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.3 v2:172.21.15.5:6824/1164130599 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2d50480 2024-05-20T18:34:35.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.518+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:66 14.18 14:18000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b374a000 con 0x5648b2d50000 2024-05-20T18:34:35.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.519+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.1 v2:172.21.15.5:6815/550664725 10 ==== osd_op_reply(66 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d50000 2024-05-20T18:34:35.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.519+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:67 14.38 14:1c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d61800 con 0x5648b2d50000 2024-05-20T18:34:35.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.519+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.1 v2:172.21.15.5:6815/550664725 11 ==== osd_op_reply(67 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d50000 2024-05-20T18:34:35.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.519+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:68 14.4 14:20000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b374a400 con 0x5648b2b0d600 2024-05-20T18:34:35.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.520+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.6 v2:172.21.15.63:6816/3384565140 7 ==== osd_op_reply(68 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2b0d600 2024-05-20T18:34:35.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.520+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:69 14.24 14:24000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d60800 con 0x5648b2d50000 2024-05-20T18:34:35.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.520+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.1 v2:172.21.15.5:6815/550664725 12 ==== osd_op_reply(69 [pgnls-filter start_epoch 94 out=49b] v90'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d50000 2024-05-20T18:34:35.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.520+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:70 14.14 14:28000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b374bc00 con 0x5648b2d50900 2024-05-20T18:34:35.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.520+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.4 v2:172.21.15.63:6800/3673324406 12 ==== osd_op_reply(70 [pgnls-filter start_epoch 94 out=49b] v90'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2d50900 2024-05-20T18:34:35.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.520+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:71 14.34 14:2c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d61c00 con 0x5648b2b0da80 2024-05-20T18:34:35.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.521+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.2 v2:172.21.15.5:6800/921679764 8 ==== osd_op_reply(71 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2b0da80 2024-05-20T18:34:35.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.521+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:72 14.c 14:30000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b382d400 con 0x5648b2b0d180 2024-05-20T18:34:35.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.521+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.0 v2:172.21.15.5:6808/688093005 17 ==== osd_op_reply(72 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2b0d180 2024-05-20T18:34:35.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.521+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:73 14.2c 14:34000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d60000 con 0x5648b2d51200 2024-05-20T18:34:35.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.521+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.7 v2:172.21.15.63:6824/4136124451 6 ==== osd_op_reply(73 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d51200 2024-05-20T18:34:35.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.521+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:74 14.1c 14:38000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b38b1000 con 0x5648b2b0d180 2024-05-20T18:34:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.522+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.0 v2:172.21.15.5:6808/688093005 18 ==== osd_op_reply(74 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2b0d180 2024-05-20T18:34:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.522+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:75 14.3c 14:3c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d60c00 con 0x5648b2d50000 2024-05-20T18:34:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.522+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.1 v2:172.21.15.5:6815/550664725 13 ==== osd_op_reply(75 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d50000 2024-05-20T18:34:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.522+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:76 14.2 14:40000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b374ac00 con 0x5648b2b0d600 2024-05-20T18:34:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.522+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.6 v2:172.21.15.63:6816/3384565140 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2b0d600 2024-05-20T18:34:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.522+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:77 14.22 14:44000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d61800 con 0x5648b2d50900 2024-05-20T18:34:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.523+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.4 v2:172.21.15.63:6800/3673324406 13 ==== osd_op_reply(77 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2d50900 2024-05-20T18:34:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.523+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:78 14.12 14:48000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b374b000 con 0x5648b2b0da80 2024-05-20T18:34:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.523+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.2 v2:172.21.15.5:6800/921679764 9 ==== osd_op_reply(78 [pgnls-filter start_epoch 94 out=49b] v90'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2b0da80 2024-05-20T18:34:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.523+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:79 14.32 14:4c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d60800 con 0x5648b2d50900 2024-05-20T18:34:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.524+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.4 v2:172.21.15.63:6800/3673324406 14 ==== osd_op_reply(79 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2d50900 2024-05-20T18:34:35.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.524+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:80 14.a 14:50000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b374a000 con 0x5648b2d50d80 2024-05-20T18:34:35.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.524+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.5 v2:172.21.15.63:6808/1616066286 6 ==== osd_op_reply(80 [pgnls-filter start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2d50d80 2024-05-20T18:34:35.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.524+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:81 14.2a 14:54000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d60400 con 0x5648b2d50900 2024-05-20T18:34:35.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.525+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.4 v2:172.21.15.63:6800/3673324406 15 ==== osd_op_reply(81 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2d50900 2024-05-20T18:34:35.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.525+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:82 14.1a 14:58000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b374a800 con 0x5648b2b0d180 2024-05-20T18:34:35.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.525+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.0 v2:172.21.15.5:6808/688093005 19 ==== osd_op_reply(82 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2b0d180 2024-05-20T18:34:35.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.525+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:83 14.3a 14:5c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b382d000 con 0x5648b2b0d180 2024-05-20T18:34:35.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.525+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.0 v2:172.21.15.5:6808/688093005 20 ==== osd_op_reply(83 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2b0d180 2024-05-20T18:34:35.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.525+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:84 14.6 14:60000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b374a400 con 0x5648b2b0d180 2024-05-20T18:34:35.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.525+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.0 v2:172.21.15.5:6808/688093005 21 ==== osd_op_reply(84 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2b0d180 2024-05-20T18:34:35.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.526+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:85 14.26 14:64000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b38b1c00 con 0x5648b2d50900 2024-05-20T18:34:35.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.526+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.4 v2:172.21.15.63:6800/3673324406 16 ==== osd_op_reply(85 [pgnls-filter start_epoch 94 out=49b] v90'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2d50900 2024-05-20T18:34:35.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.526+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:86 14.16 14:68000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d61400 con 0x5648b2d50480 2024-05-20T18:34:35.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.526+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.3 v2:172.21.15.5:6824/1164130599 7 ==== osd_op_reply(86 [pgnls-filter start_epoch 94 out=49b] v90'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2d50480 2024-05-20T18:34:35.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.526+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:87 14.36 14:6c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b38b0c00 con 0x5648b2b0da80 2024-05-20T18:34:35.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.526+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.2 v2:172.21.15.5:6800/921679764 10 ==== osd_op_reply(87 [pgnls-filter start_epoch 94 out=49b] v90'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2b0da80 2024-05-20T18:34:35.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.526+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:88 14.e 14:70000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d61000 con 0x5648b2d50900 2024-05-20T18:34:35.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.527+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.4 v2:172.21.15.63:6800/3673324406 17 ==== osd_op_reply(88 [pgnls-filter start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2d50900 2024-05-20T18:34:35.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.527+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:89 14.2e 14:74000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b38b0800 con 0x5648b2d50480 2024-05-20T18:34:35.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.527+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.3 v2:172.21.15.5:6824/1164130599 8 ==== osd_op_reply(89 [pgnls-filter start_epoch 94 out=49b] v90'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2d50480 2024-05-20T18:34:35.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.527+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:90 14.1e 14:78000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b374b400 con 0x5648b2d51200 2024-05-20T18:34:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.528+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.7 v2:172.21.15.63:6824/4136124451 7 ==== osd_op_reply(90 [pgnls-filter start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d51200 2024-05-20T18:34:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.528+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:91 14.3e 14:7c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d82000 con 0x5648b2d50d80 2024-05-20T18:34:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.528+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.5 v2:172.21.15.63:6808/1616066286 7 ==== osd_op_reply(91 [pgnls-filter start_epoch 94 out=49b] v90'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2d50d80 2024-05-20T18:34:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.528+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:92 14.1 14:80000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b374b800 con 0x5648b2d50480 2024-05-20T18:34:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.528+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.3 v2:172.21.15.5:6824/1164130599 9 ==== osd_op_reply(92 [pgnls-filter start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2d50480 2024-05-20T18:34:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.528+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:93 14.21 14:84000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d82400 con 0x5648b2d50d80 2024-05-20T18:34:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.529+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.5 v2:172.21.15.63:6808/1616066286 8 ==== osd_op_reply(93 [pgnls-filter start_epoch 94 out=49b] v90'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2d50d80 2024-05-20T18:34:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.529+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:94 14.11 14:88000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b374bc00 con 0x5648b2d50900 2024-05-20T18:34:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.529+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.4 v2:172.21.15.63:6800/3673324406 18 ==== osd_op_reply(94 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2d50900 2024-05-20T18:34:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.529+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:95 14.31 14:8c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d82800 con 0x5648b2d50000 2024-05-20T18:34:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.529+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.1 v2:172.21.15.5:6815/550664725 14 ==== osd_op_reply(95 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d50000 2024-05-20T18:34:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.529+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:96 14.9 14:90000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b374b000 con 0x5648b2d50d80 2024-05-20T18:34:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.530+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.5 v2:172.21.15.63:6808/1616066286 9 ==== osd_op_reply(96 [pgnls-filter start_epoch 94 out=49b] v90'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2d50d80 2024-05-20T18:34:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.530+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:97 14.29 14:94000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d82c00 con 0x5648b2b0d600 2024-05-20T18:34:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.530+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.6 v2:172.21.15.63:6816/3384565140 9 ==== osd_op_reply(97 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2b0d600 2024-05-20T18:34:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.530+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:98 14.19 14:98000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d61c00 con 0x5648b2b0d600 2024-05-20T18:34:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.530+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.6 v2:172.21.15.63:6816/3384565140 10 ==== osd_op_reply(98 [pgnls-filter start_epoch 94 out=49b] v90'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2b0d600 2024-05-20T18:34:35.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.530+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:99 14.39 14:9c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d83000 con 0x5648b2d51200 2024-05-20T18:34:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.531+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.7 v2:172.21.15.63:6824/4136124451 8 ==== osd_op_reply(99 [pgnls-filter start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d51200 2024-05-20T18:34:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.531+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:100 14.5 14:a0000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d61800 con 0x5648b2b0d180 2024-05-20T18:34:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.531+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.0 v2:172.21.15.5:6808/688093005 22 ==== osd_op_reply(100 [pgnls-filter start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2b0d180 2024-05-20T18:34:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.531+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:101 14.25 14:a4000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d83400 con 0x5648b2b0da80 2024-05-20T18:34:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.531+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.2 v2:172.21.15.5:6800/921679764 11 ==== osd_op_reply(101 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2b0da80 2024-05-20T18:34:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.531+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:102 14.15 14:a8000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b38b1400 con 0x5648b2d51200 2024-05-20T18:34:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.532+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.7 v2:172.21.15.63:6824/4136124451 9 ==== osd_op_reply(102 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d51200 2024-05-20T18:34:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.532+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:103 14.35 14:ac000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d83800 con 0x5648b2d50900 2024-05-20T18:34:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.532+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.4 v2:172.21.15.63:6800/3673324406 19 ==== osd_op_reply(103 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2d50900 2024-05-20T18:34:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.532+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:104 14.d 14:b0000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b38b1c00 con 0x5648b2b0d600 2024-05-20T18:34:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.532+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.6 v2:172.21.15.63:6816/3384565140 11 ==== osd_op_reply(104 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2b0d600 2024-05-20T18:34:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.533+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:105 14.2d 14:b4000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d83c00 con 0x5648b2d50900 2024-05-20T18:34:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.533+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.4 v2:172.21.15.63:6800/3673324406 20 ==== osd_op_reply(105 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2d50900 2024-05-20T18:34:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.533+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:106 14.1d 14:b8000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b38b0c00 con 0x5648b2b0d180 2024-05-20T18:34:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.533+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.0 v2:172.21.15.5:6808/688093005 23 ==== osd_op_reply(106 [pgnls-filter start_epoch 94 out=49b] v90'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2b0d180 2024-05-20T18:34:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.534+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:107 14.3d 14:bc000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d83400 con 0x5648b2d50480 2024-05-20T18:34:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.534+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.3 v2:172.21.15.5:6824/1164130599 10 ==== osd_op_reply(107 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2d50480 2024-05-20T18:34:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.534+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:108 14.3 14:c0000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b38b1000 con 0x5648b2b0d180 2024-05-20T18:34:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.534+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.0 v2:172.21.15.5:6808/688093005 24 ==== osd_op_reply(108 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2b0d180 2024-05-20T18:34:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.534+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:109 14.23 14:c4000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d83800 con 0x5648b2d51200 2024-05-20T18:34:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.534+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.7 v2:172.21.15.63:6824/4136124451 10 ==== osd_op_reply(109 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d51200 2024-05-20T18:34:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.534+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:110 14.13 14:c8000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d60800 con 0x5648b2b0da80 2024-05-20T18:34:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.535+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.2 v2:172.21.15.5:6800/921679764 12 ==== osd_op_reply(110 [pgnls-filter start_epoch 94 out=49b] v90'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2b0da80 2024-05-20T18:34:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.535+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:111 14.33 14:cc000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d82800 con 0x5648b2d50000 2024-05-20T18:34:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.535+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.1 v2:172.21.15.5:6815/550664725 15 ==== osd_op_reply(111 [pgnls-filter start_epoch 94 out=49b] v90'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d50000 2024-05-20T18:34:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.535+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:112 14.b 14:d0000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d60400 con 0x5648b2b0d180 2024-05-20T18:34:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.535+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.0 v2:172.21.15.5:6808/688093005 25 ==== osd_op_reply(112 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2b0d180 2024-05-20T18:34:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.535+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:113 14.2b 14:d4000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d83000 con 0x5648b2d50900 2024-05-20T18:34:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.536+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.4 v2:172.21.15.63:6800/3673324406 21 ==== osd_op_reply(113 [pgnls-filter start_epoch 94 out=49b] v90'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2d50900 2024-05-20T18:34:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.536+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:114 14.1b 14:d8000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d61000 con 0x5648b2d50000 2024-05-20T18:34:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.536+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.1 v2:172.21.15.5:6815/550664725 16 ==== osd_op_reply(114 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d50000 2024-05-20T18:34:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.536+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:115 14.3b 14:dc000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d82000 con 0x5648b2d50900 2024-05-20T18:34:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.536+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.4 v2:172.21.15.63:6800/3673324406 22 ==== osd_op_reply(115 [pgnls-filter start_epoch 94 out=49b] v90'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2d50900 2024-05-20T18:34:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.536+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:116 14.7 14:e0000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d60000 con 0x5648b2d50d80 2024-05-20T18:34:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.537+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.5 v2:172.21.15.63:6808/1616066286 10 ==== osd_op_reply(116 [pgnls-filter start_epoch 94 out=81b] v94'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2d50d80 2024-05-20T18:34:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.537+0000 7f3f19734740 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-05-20T18:34:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.537+0000 7f3f19734740 10 datascan.operator(): handling object 10000000001.0 2024-05-20T18:34:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.537+0000 7f3f19734740 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:117 14.7 14:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d8e000 con 0x5648b2d50d80 2024-05-20T18:34:35.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.537+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.5 v2:172.21.15.63:6808/1616066286 11 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) ==== 416+0+157 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2d50d80 2024-05-20T18:34:35.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.537+0000 7f3f19734740 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-05-20T18:34:35.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.537+0000 7f3f19734740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/file 2024-05-20T18:34:35.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.537+0000 7f3f19734740 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=file 2024-05-20T18:34:35.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.537+0000 7f3f19734740 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:118 13.10 13:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d8e400 con 0x5648b2d50000 2024-05-20T18:34:35.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.537+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.1 v2:172.21.15.5:6815/550664725 17 ==== osd_op_reply(118 10000000000.00000000 [getxattr (59) out=59b] v0'0 uv17 ondisk = 0) ==== 164+0+59 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d50000 2024-05-20T18:34:35.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.537+0000 7f3f19734740 20 datascan.get_frag_of: got backtrace parent 0x1/dir 2024-05-20T18:34:35.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.537+0000 7f3f19734740 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d8e800 con 0x5648b2b0d600 2024-05-20T18:34:35.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.538+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.6 v2:172.21.15.63:6816/3384565140 12 ==== osd_op_reply(119 1.00000000 [omap-get-vals-by-keys out=581b] v0'0 uv9 ondisk = 0) ==== 154+0+581 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2b0d600 2024-05-20T18:34:35.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.538+0000 7f3f19734740 20 datascan.read_dentry: oid=1.00000000 dname=dir frag=*, r=0 2024-05-20T18:34:35.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.538+0000 7f3f19734740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-05-20T18:34:35.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.538+0000 7f3f19734740 20 datascan.get_frag_of: frag is * 2024-05-20T18:34:35.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.538+0000 7f3f19734740 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:120 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d8ec00 con 0x5648b2d50000 2024-05-20T18:34:35.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.538+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.1 v2:172.21.15.5:6815/550664725 18 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d50000 2024-05-20T18:34:35.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.538+0000 7f3f19734740 10 datascan.find_or_create_dirfrag: read_version = 17 2024-05-20T18:34:35.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.538+0000 7f3f19734740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2024-05-20T18:34:35.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.538+0000 7f3f19734740 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:121 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d8f000 con 0x5648b2d50000 2024-05-20T18:34:35.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.538+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.1 v2:172.21.15.5:6815/550664725 19 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d50000 2024-05-20T18:34:35.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.538+0000 7f3f19734740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-05-20T18:34:35.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.538+0000 7f3f19734740 20 datascan.read_dentry: file_head not found in result 2024-05-20T18:34:35.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.538+0000 7f3f19734740 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/file with size=0 bytes 2024-05-20T18:34:35.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.538+0000 7f3f19734740 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:122 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d8f400 con 0x5648b2d50000 2024-05-20T18:34:35.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.540+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.1 v2:172.21.15.5:6815/550664725 20 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v94'18 uv18 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d50000 2024-05-20T18:34:35.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.540+0000 7f3f19734740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-05-20T18:34:35.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.540+0000 7f3f19734740 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:123 14.27 14:e4000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d8f800 con 0x5648b2d50d80 2024-05-20T18:34:35.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.540+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.5 v2:172.21.15.63:6808/1616066286 12 ==== osd_op_reply(123 [pgnls-filter start_epoch 94 out=49b] v90'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2d50d80 2024-05-20T18:34:35.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.540+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:124 14.17 14:e8000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d60c00 con 0x5648b2d50000 2024-05-20T18:34:35.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.540+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.1 v2:172.21.15.5:6815/550664725 21 ==== osd_op_reply(124 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d50000 2024-05-20T18:34:35.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.540+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:125 14.37 14:ec000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d82400 con 0x5648b2d50900 2024-05-20T18:34:35.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.541+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 <== osd.4 v2:172.21.15.63:6800/3673324406 23 ==== osd_op_reply(125 [pgnls-filter start_epoch 94 out=49b] v90'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e780 con 0x5648b2d50900 2024-05-20T18:34:35.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.541+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:126 14.f 14:f0000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d61400 con 0x5648b2d51200 2024-05-20T18:34:35.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.542+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.7 v2:172.21.15.63:6824/4136124451 11 ==== osd_op_reply(126 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2d51200 2024-05-20T18:34:35.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.542+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:127 14.2f 14:f4000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b38b0800 con 0x5648b2b0d180 2024-05-20T18:34:35.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.542+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.0 v2:172.21.15.5:6808/688093005 26 ==== osd_op_reply(127 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2b0d180 2024-05-20T18:34:35.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.542+0000 7f3f12762640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:128 14.1f 14:f8000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b382d400 con 0x5648b2b0d180 2024-05-20T18:34:35.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.542+0000 7f3f16f6b640 1 -- 172.21.15.5:0/3937564835 <== osd.0 v2:172.21.15.5:6808/688093005 27 ==== osd_op_reply(128 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e000 con 0x5648b2b0d180 2024-05-20T18:34:35.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.542+0000 7f3f12f63640 1 -- 172.21.15.5:0/3937564835 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:129 14.3f 14:fc000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5648b2d61c00 con 0x5648b2d50480 2024-05-20T18:34:35.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.542+0000 7f3f17f6d640 1 -- 172.21.15.5:0/3937564835 <== osd.3 v2:172.21.15.5:6824/1164130599 11 ==== osd_op_reply(129 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5648b2a7e500 con 0x5648b2d50480 2024-05-20T18:34:35.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 -- 172.21.15.5:0/3937564835 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x5648b2b0d180 msgr2=0x5648b2ae0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x5648b2b0d180 0x5648b2ae0100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 -- 172.21.15.5:0/3937564835 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x5648b2d50000 msgr2=0x5648b2d58000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x5648b2d50000 0x5648b2d58000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 -- 172.21.15.5:0/3937564835 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x5648b2b0da80 msgr2=0x5648b2adfb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x5648b2b0da80 0x5648b2adfb80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 -- 172.21.15.5:0/3937564835 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x5648b2d50480 msgr2=0x5648b2d58580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x5648b2d50480 0x5648b2d58580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 -- 172.21.15.5:0/3937564835 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x5648b2d50900 msgr2=0x5648b2d58b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x5648b2d50900 0x5648b2d58b00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 -- 172.21.15.5:0/3937564835 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x5648b2d50d80 msgr2=0x5648b2d59080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 reap_dead start 2024-05-20T18:34:35.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x5648b2d50d80 0x5648b2d59080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 -- 172.21.15.5:0/3937564835 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x5648b2b0d600 msgr2=0x5648b2adf600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x5648b2b0d600 0x5648b2adf600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 reap_dead start 2024-05-20T18:34:35.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 -- 172.21.15.5:0/3937564835 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x5648b2d51200 msgr2=0x5648b2d59600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x5648b2d51200 0x5648b2d59600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f1776c640 1 -- 172.21.15.5:0/3937564835 reap_dead start 2024-05-20T18:34:35.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 -- 172.21.15.5:0/3937564835 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x5648b2b0b200 msgr2=0x5648b2ae1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x5648b2b0b200 0x5648b2ae1700 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x5648b2a48690 tx=0x5648b2a46960 comp rx=0 tx=0).stop 2024-05-20T18:34:35.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 -- 172.21.15.5:0/3937564835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648b2b0c880 msgr2=0x5648b2ae0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648b2b0c880 0x5648b2ae0c00 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x5648b38ae030 tx=0x5648b3845a40 comp rx=0 tx=0).stop 2024-05-20T18:34:35.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 -- 172.21.15.5:0/3937564835 shutdown_connections 2024-05-20T18:34:35.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x5648b2d51200 0x5648b2d59600 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x5648b2b0b200 0x5648b2ae1700 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 --2- 172.21.15.5:0/3937564835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648b2b0c880 0x5648b2ae0c00 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 -- 172.21.15.5:0/3937564835 >> 172.21.15.5:0/3937564835 conn(0x5648b2b0a900 msgr2=0x5648b2a79800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:35.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 -- 172.21.15.5:0/3937564835 shutdown_connections 2024-05-20T18:34:35.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 -- 172.21.15.5:0/3937564835 wait complete. 2024-05-20T18:34:35.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 -- 172.21.15.5:0/3430690832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648b2b0bb00 msgr2=0x5648b2ade580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 --2- 172.21.15.5:0/3430690832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648b2b0bb00 0x5648b2ade580 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x5648b3767d40 tx=0x5648b2a4a2a0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 -- 172.21.15.5:0/3430690832 shutdown_connections 2024-05-20T18:34:35.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 --2- 172.21.15.5:0/3430690832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5648b2b0bf80 0x5648b2adf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 --2- 172.21.15.5:0/3430690832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648b2b0bb00 0x5648b2ade580 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 --2- 172.21.15.5:0/3430690832 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5648b2b0b680 0x5648b2adeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 -- 172.21.15.5:0/3430690832 >> 172.21.15.5:0/3430690832 conn(0x5648b2b0c400 msgr2=0x5648b2a79000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:35.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 -- 172.21.15.5:0/3430690832 shutdown_connections 2024-05-20T18:34:35.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.543+0000 7f3f19734740 1 -- 172.21.15.5:0/3430690832 wait complete. 2024-05-20T18:34:35.552 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', 'cephfs_data'] in time 0:00:00.123068, result: 2024-05-20T18:34:35.552 DEBUG:teuthology.orchestra.run.smithi005:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-05-20T18:34:35.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.603+0000 7fcae7e32740 1 -- start start 2024-05-20T18:34:35.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.603+0000 7fcae7e32740 1 Processor -- start 2024-05-20T18:34:35.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.604+0000 7fcae7e32740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5598a242b680 0x5598a23feb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.604+0000 7fcae7e32740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598a242bb00 0x5598a23fe580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.604+0000 7fcae7e32740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5598a242bf80 0x5598a23ff080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.604+0000 7fcae7e32740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5598a3054180 con 0x5598a242bb00 2024-05-20T18:34:35.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.604+0000 7fcae7e32740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5598a3054300 con 0x5598a242b680 2024-05-20T18:34:35.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.604+0000 7fcae7e32740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5598a3054480 con 0x5598a242bf80 2024-05-20T18:34:35.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.604+0000 7fcae4fb5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598a242bb00 0x5598a23fe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.605+0000 7fcae4fb5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598a242bb00 0x5598a23fe580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38390/0 (socket says 172.21.15.5:38390) 2024-05-20T18:34:35.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.605+0000 7fcae4fb5640 1 -- 172.21.15.5:0/1548317264 learned_addr learned my addr 172.21.15.5:0/1548317264 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:35.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.605+0000 7fcae57b6640 1 --2- 172.21.15.5:0/1548317264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5598a242b680 0x5598a23feb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.605+0000 7fcae5fb7640 1 --2- 172.21.15.5:0/1548317264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5598a242bf80 0x5598a23ff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.605+0000 7fcae4fb5640 1 -- 172.21.15.5:0/1548317264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5598a242bf80 msgr2=0x5598a23ff080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:35.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.605+0000 7fcae4fb5640 1 --2- 172.21.15.5:0/1548317264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5598a242bf80 0x5598a23ff080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.605+0000 7fcae4fb5640 1 -- 172.21.15.5:0/1548317264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5598a242b680 msgr2=0x5598a23feb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:35.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.605+0000 7fcae4fb5640 1 --2- 172.21.15.5:0/1548317264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5598a242b680 0x5598a23feb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.605+0000 7fcae4fb5640 1 -- 172.21.15.5:0/1548317264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55989e26d680 con 0x5598a242bb00 2024-05-20T18:34:35.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.605+0000 7fcae4fb5640 1 --2- 172.21.15.5:0/1548317264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598a242bb00 0x5598a23fe580 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x5598a308bd40 tx=0x5598a23682a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60814fc41160e66 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.605+0000 7fcae2fb1640 1 -- 172.21.15.5:0/1548317264 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55989e266680 con 0x5598a242bb00 2024-05-20T18:34:35.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.605+0000 7fcae2fb1640 1 -- 172.21.15.5:0/1548317264 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5598a22ba540 con 0x5598a242bb00 2024-05-20T18:34:35.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.605+0000 7fcae2fb1640 1 -- 172.21.15.5:0/1548317264 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55989e266b60 con 0x5598a242bb00 2024-05-20T18:34:35.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.605+0000 7fcae7e32740 1 -- 172.21.15.5:0/1548317264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55989e26d860 con 0x5598a242bb00 2024-05-20T18:34:35.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.605+0000 7fcae7e32740 1 -- 172.21.15.5:0/1548317264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55989e26d2c0 con 0x5598a242bb00 2024-05-20T18:34:35.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.605+0000 7fcae7e32740 4 waiting for MDS map... 2024-05-20T18:34:35.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.605+0000 7fcae2fb1640 1 -- 172.21.15.5:0/1548317264 <== mon.0 v2:172.21.15.5:3300/0 4 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x5598a3152a00 con 0x5598a242bb00 2024-05-20T18:34:35.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.606+0000 7fcae2fb1640 1 -- 172.21.15.5:0/1548317264 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 68) ==== 2885+0+0 (secure 0 0 0) 0x5598a22c9180 con 0x5598a242bb00 2024-05-20T18:34:35.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.606+0000 7fcae7e32740 4 Got MDS map 68 2024-05-20T18:34:35.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.606+0000 7fcae7e32740 4 datascan.main: Using metadata pool output 2024-05-20T18:34:35.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.606+0000 7fcae7e32740 4 datascan.main: connecting to RADOS... 2024-05-20T18:34:35.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.607+0000 7fcae7e32740 1 Processor -- start 2024-05-20T18:34:35.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.607+0000 7fcae7e32740 1 -- start start 2024-05-20T18:34:35.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.608+0000 7fcae7e32740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598a242ad80 0x5598a23ff600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.608+0000 7fcae7e32740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5598a242a900 0x5598a23ffb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.608+0000 7fcae7e32740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5598a242c880 0x5598a2400100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.608+0000 7fcae7e32740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5598a3054600 con 0x5598a242ad80 2024-05-20T18:34:35.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.608+0000 7fcae7e32740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5598a3054780 con 0x5598a242a900 2024-05-20T18:34:35.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.608+0000 7fcae7e32740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5598a3054900 con 0x5598a242c880 2024-05-20T18:34:35.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.608+0000 7fcae5fb7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598a242ad80 0x5598a23ff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.608+0000 7fcae5fb7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598a242ad80 0x5598a23ff600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38406/0 (socket says 172.21.15.5:38406) 2024-05-20T18:34:35.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.608+0000 7fcae5fb7640 1 -- 172.21.15.5:0/1113274170 learned_addr learned my addr 172.21.15.5:0/1113274170 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:35.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.608+0000 7fcae57b6640 1 --2- 172.21.15.5:0/1113274170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5598a242a900 0x5598a23ffb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.608+0000 7fcae4fb5640 1 --2- 172.21.15.5:0/1113274170 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5598a242c880 0x5598a2400100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.608+0000 7fcae5fb7640 1 -- 172.21.15.5:0/1113274170 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5598a242c880 msgr2=0x5598a2400100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:35.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.608+0000 7fcae5fb7640 1 --2- 172.21.15.5:0/1113274170 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5598a242c880 0x5598a2400100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.608+0000 7fcae5fb7640 1 -- 172.21.15.5:0/1113274170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5598a242a900 msgr2=0x5598a23ffb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:35.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.608+0000 7fcae5fb7640 1 --2- 172.21.15.5:0/1113274170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5598a242a900 0x5598a23ffb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.608+0000 7fcae5fb7640 1 -- 172.21.15.5:0/1113274170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55989e26d4a0 con 0x5598a242ad80 2024-05-20T18:34:35.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.608+0000 7fcae5fb7640 1 --2- 172.21.15.5:0/1113274170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598a242ad80 0x5598a23ff600 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x5598a308b1d0 tx=0x5598a2368180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc95144a0cb40ef server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.608+0000 7fcae0fad640 1 -- 172.21.15.5:0/1113274170 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55989e2669c0 con 0x5598a242ad80 2024-05-20T18:34:35.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.608+0000 7fcae0fad640 1 -- 172.21.15.5:0/1113274170 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5598a22bae00 con 0x5598a242ad80 2024-05-20T18:34:35.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.608+0000 7fcae0fad640 1 -- 172.21.15.5:0/1113274170 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55989e266680 con 0x5598a242ad80 2024-05-20T18:34:35.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.609+0000 7fcae7e32740 1 -- 172.21.15.5:0/1113274170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598a242ad80 msgr2=0x5598a23ff600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.609+0000 7fcae7e32740 1 --2- 172.21.15.5:0/1113274170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598a242ad80 0x5598a23ff600 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x5598a308b1d0 tx=0x5598a2368180 comp rx=0 tx=0).stop 2024-05-20T18:34:35.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.609+0000 7fcae7e32740 1 -- 172.21.15.5:0/1113274170 shutdown_connections 2024-05-20T18:34:35.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.609+0000 7fcae7e32740 1 --2- 172.21.15.5:0/1113274170 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5598a242c880 0x5598a2400100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.609+0000 7fcae7e32740 1 --2- 172.21.15.5:0/1113274170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5598a242a900 0x5598a23ffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.609+0000 7fcae7e32740 1 --2- 172.21.15.5:0/1113274170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598a242ad80 0x5598a23ff600 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.609+0000 7fcae7e32740 1 -- 172.21.15.5:0/1113274170 >> 172.21.15.5:0/1113274170 conn(0x5598a242b200 msgr2=0x5598a2399800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:35.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.609+0000 7fcae7e32740 1 -- 172.21.15.5:0/1113274170 shutdown_connections 2024-05-20T18:34:35.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.609+0000 7fcae7e32740 1 -- 172.21.15.5:0/1113274170 wait complete. 2024-05-20T18:34:35.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.610+0000 7fcae7e32740 1 Processor -- start 2024-05-20T18:34:35.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.610+0000 7fcae7e32740 1 -- start start 2024-05-20T18:34:35.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.611+0000 7fcae7e32740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598a242ad80 0x5598a2401180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.611+0000 7fcae7e32740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5598a242c880 0x5598a2400c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.611+0000 7fcae7e32740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5598a242cd00 0x5598a2400680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.611+0000 7fcae7e32740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5598a3055980 con 0x5598a242ad80 2024-05-20T18:34:35.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.611+0000 7fcae7e32740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5598a3055800 con 0x5598a242cd00 2024-05-20T18:34:35.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.611+0000 7fcae7e32740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5598a3055680 con 0x5598a242c880 2024-05-20T18:34:35.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.611+0000 7fcae5fb7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598a242ad80 0x5598a2401180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.611+0000 7fcae5fb7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598a242ad80 0x5598a2401180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38412/0 (socket says 172.21.15.5:38412) 2024-05-20T18:34:35.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.611+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 learned_addr learned my addr 172.21.15.5:0/89586744 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:35.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.611+0000 7fcae57b6640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5598a242c880 0x5598a2400c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.611+0000 7fcae4fb5640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5598a242cd00 0x5598a2400680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.612+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5598a242c880 msgr2=0x5598a2400c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.612+0000 7fcae5fb7640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5598a242c880 0x5598a2400c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.612+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5598a242cd00 msgr2=0x5598a2400680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.612+0000 7fcae5fb7640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5598a242cd00 0x5598a2400680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.612+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55989e26d680 con 0x5598a242ad80 2024-05-20T18:34:35.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.612+0000 7fcae5fb7640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598a242ad80 0x5598a2401180 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x5598a313fa70 tx=0x5598a23681e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.612+0000 7fcadf7aa640 1 -- 172.21.15.5:0/89586744 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55989e266b60 con 0x5598a242ad80 2024-05-20T18:34:35.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.612+0000 7fcadf7aa640 1 -- 172.21.15.5:0/89586744 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5598a22bac40 con 0x5598a242ad80 2024-05-20T18:34:35.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.612+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55989e26d860 con 0x5598a242ad80 2024-05-20T18:34:35.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.612+0000 7fcadf7aa640 1 -- 172.21.15.5:0/89586744 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55989e2664e0 con 0x5598a242ad80 2024-05-20T18:34:35.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.612+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55989e26d0e0 con 0x5598a242ad80 2024-05-20T18:34:35.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.612+0000 7fcae7e32740 4 datascan.init: resolving metadata pool 13 2024-05-20T18:34:35.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.613+0000 7fcadf7aa640 1 -- 172.21.15.5:0/89586744 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x5598a31bac00 con 0x5598a242ad80 2024-05-20T18:34:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.614+0000 7fcadf7aa640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x5598a242b200 0x5598a2401700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.614+0000 7fcadf7aa640 1 -- 172.21.15.5:0/89586744 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x5598a31be600 con 0x5598a242ad80 2024-05-20T18:34:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.614+0000 7fcae7e32740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-20T18:34:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.614+0000 7fcae57b6640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x5598a242b200 0x5598a2401700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.614+0000 7fcae7e32740 4 datascan.main: resolving metadata pool 13 2024-05-20T18:34:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.614+0000 7fcae7e32740 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x5598a242d180 0x5598a2400100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.614+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a242d180 2024-05-20T18:34:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.614+0000 7fcae4fb5640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x5598a242d180 0x5598a2400100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.614+0000 7fcae57b6640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x5598a242b200 0x5598a2401700 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x5598a236ae70 tx=0x5598a25daae0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.614+0000 7fcae4fb5640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x5598a242d180 0x5598a2400100 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.615+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.7 v2:172.21.15.63:6824/4136124451 1 ==== osd_op_reply(1 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e000 con 0x5598a242d180 2024-05-20T18:34:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.615+0000 7fcae4fb5640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x5598a242da80 0x5598a2654000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.615+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:2 13.10 13:08000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a242da80 2024-05-20T18:34:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.615+0000 7fcae5fb7640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x5598a242da80 0x5598a2654000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.615+0000 7fcae5fb7640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x5598a242da80 0x5598a2654000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.616+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.1 v2:172.21.15.5:6815/550664725 1 ==== osd_op_reply(2 [pgnls start_epoch 94 out=177b] v94'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x5598a239e500 con 0x5598a242da80 2024-05-20T18:34:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.616+0000 7fcae5fb7640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x5598a265c000 0x5598a2654580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.616+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:3 13.8 13:10000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a265c000 2024-05-20T18:34:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.616+0000 7fcae57b6640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x5598a265c000 0x5598a2654580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.616+0000 7fcae57b6640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x5598a265c000 0x5598a2654580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.616+0000 7fcae57b6640 1 -- 172.21.15.5:0/89586744 <== osd.2 v2:172.21.15.5:6800/921679764 1 ==== osd_op_reply(3 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e780 con 0x5598a265c000 2024-05-20T18:34:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.616+0000 7fcae57b6640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:4 13.18 13:18000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a242d180 2024-05-20T18:34:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.617+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.7 v2:172.21.15.63:6824/4136124451 2 ==== osd_op_reply(4 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e000 con 0x5598a242d180 2024-05-20T18:34:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.617+0000 7fcae4fb5640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x5598a265c480 0x5598a2654b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.617+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:5 13.4 13:20000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a265c480 2024-05-20T18:34:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.617+0000 7fcae4fb5640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x5598a265c480 0x5598a2654b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.617+0000 7fcae4fb5640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x5598a265c480 0x5598a2654b00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.618+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.6 v2:172.21.15.63:6816/3384565140 1 ==== osd_op_reply(5 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e000 con 0x5598a265c480 2024-05-20T18:34:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.618+0000 7fcae4fb5640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x5598a265c900 0x5598a2655080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.618+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:6 13.14 13:28000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a265c900 2024-05-20T18:34:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.618+0000 7fcae5fb7640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x5598a265c900 0x5598a2655080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.618+0000 7fcae5fb7640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x5598a265c900 0x5598a2655080 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.618+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.0 v2:172.21.15.5:6808/688093005 1 ==== osd_op_reply(6 [pgnls start_epoch 94 out=149b] v93'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x5598a239e500 con 0x5598a265c900 2024-05-20T18:34:35.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.618+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:7 13.c 13:30000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a242da80 2024-05-20T18:34:35.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.619+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.1 v2:172.21.15.5:6815/550664725 2 ==== osd_op_reply(7 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e500 con 0x5598a242da80 2024-05-20T18:34:35.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.619+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:8 13.1c 13:38000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a265c480 2024-05-20T18:34:35.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.619+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.6 v2:172.21.15.63:6816/3384565140 2 ==== osd_op_reply(8 [pgnls start_epoch 94 out=73b] v93'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5598a239e000 con 0x5598a265c480 2024-05-20T18:34:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.619+0000 7fcae4fb5640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x5598a265cd80 0x5598a2655600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.619+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:9 13.2 13:40000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a265cd80 2024-05-20T18:34:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.619+0000 7fcae57b6640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x5598a265cd80 0x5598a2655600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.620+0000 7fcae57b6640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x5598a265cd80 0x5598a2655600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.620+0000 7fcae57b6640 1 -- 172.21.15.5:0/89586744 <== osd.5 v2:172.21.15.63:6808/1616066286 1 ==== osd_op_reply(9 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e780 con 0x5598a265cd80 2024-05-20T18:34:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.620+0000 7fcae57b6640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:10 13.12 13:48000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a265c900 2024-05-20T18:34:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.620+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.0 v2:172.21.15.5:6808/688093005 2 ==== osd_op_reply(10 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e500 con 0x5598a265c900 2024-05-20T18:34:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.620+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:11 13.a 13:50000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a265c000 2024-05-20T18:34:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.621+0000 7fcae57b6640 1 -- 172.21.15.5:0/89586744 <== osd.2 v2:172.21.15.5:6800/921679764 2 ==== osd_op_reply(11 [pgnls start_epoch 94 out=81b] v93'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5598a239e780 con 0x5598a265c000 2024-05-20T18:34:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.621+0000 7fcae57b6640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:12 13.1a 13:58000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a242da80 2024-05-20T18:34:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.621+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.1 v2:172.21.15.5:6815/550664725 3 ==== osd_op_reply(12 [pgnls start_epoch 94 out=49b] v91'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e500 con 0x5598a242da80 2024-05-20T18:34:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.621+0000 7fcae5fb7640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x5598a265db00 0x5598a2656680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.621+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:13 13.6 13:60000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a265db00 2024-05-20T18:34:35.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.621+0000 7fcae4fb5640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x5598a265db00 0x5598a2656680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.622+0000 7fcae4fb5640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x5598a265db00 0x5598a2656680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.622+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.4 v2:172.21.15.63:6800/3673324406 1 ==== osd_op_reply(13 [pgnls start_epoch 94 out=77b] v93'6 uv6 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x5598a239e000 con 0x5598a265db00 2024-05-20T18:34:35.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.622+0000 7fcae4fb5640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x5598a265d200 0x5598a2655b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.622+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:14 13.16 13:68000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a265d200 2024-05-20T18:34:35.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.622+0000 7fcae5fb7640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x5598a265d200 0x5598a2655b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.622+0000 7fcae5fb7640 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x5598a265d200 0x5598a2655b80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.623+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.3 v2:172.21.15.5:6824/1164130599 1 ==== osd_op_reply(14 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e500 con 0x5598a265d200 2024-05-20T18:34:35.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.623+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:15 13.e 13:70000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a242d180 2024-05-20T18:34:35.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.623+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.7 v2:172.21.15.63:6824/4136124451 3 ==== osd_op_reply(15 [pgnls start_epoch 94 out=73b] v93'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5598a239e000 con 0x5598a242d180 2024-05-20T18:34:35.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.623+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:16 13.1e 13:78000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a265db00 2024-05-20T18:34:35.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.623+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.4 v2:172.21.15.63:6800/3673324406 2 ==== osd_op_reply(16 [pgnls start_epoch 94 out=49b] v91'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e000 con 0x5598a265db00 2024-05-20T18:34:35.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.623+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:17 13.1 13:80000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a265c480 2024-05-20T18:34:35.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.624+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.6 v2:172.21.15.63:6816/3384565140 3 ==== osd_op_reply(17 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e000 con 0x5598a265c480 2024-05-20T18:34:35.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.624+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:18 13.11 13:88000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a242da80 2024-05-20T18:34:35.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.624+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.1 v2:172.21.15.5:6815/550664725 4 ==== osd_op_reply(18 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e500 con 0x5598a242da80 2024-05-20T18:34:35.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.624+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:19 13.9 13:90000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a265cd80 2024-05-20T18:34:35.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.624+0000 7fcae57b6640 1 -- 172.21.15.5:0/89586744 <== osd.5 v2:172.21.15.63:6808/1616066286 2 ==== osd_op_reply(19 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e780 con 0x5598a265cd80 2024-05-20T18:34:35.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.624+0000 7fcae57b6640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:20 13.19 13:98000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a265d200 2024-05-20T18:34:35.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.625+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.3 v2:172.21.15.5:6824/1164130599 2 ==== osd_op_reply(20 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e500 con 0x5598a265d200 2024-05-20T18:34:35.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.625+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:21 13.5 13:a0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a265db00 2024-05-20T18:34:35.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.625+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.4 v2:172.21.15.63:6800/3673324406 3 ==== osd_op_reply(21 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e000 con 0x5598a265db00 2024-05-20T18:34:35.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.625+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:22 13.15 13:a8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a265db00 2024-05-20T18:34:35.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.625+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.4 v2:172.21.15.63:6800/3673324406 4 ==== osd_op_reply(22 [pgnls start_epoch 94 out=97b] v93'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x5598a239e000 con 0x5598a265db00 2024-05-20T18:34:35.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.626+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:23 13.d 13:b0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a265db00 2024-05-20T18:34:35.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.626+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.4 v2:172.21.15.63:6800/3673324406 5 ==== osd_op_reply(23 [pgnls start_epoch 94 out=74b] v89'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x5598a239e000 con 0x5598a265db00 2024-05-20T18:34:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.626+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:24 13.1d 13:b8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a242d180 2024-05-20T18:34:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.626+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.7 v2:172.21.15.63:6824/4136124451 4 ==== osd_op_reply(24 [pgnls start_epoch 94 out=73b] v93'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5598a239e000 con 0x5598a242d180 2024-05-20T18:34:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.626+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:25 13.3 13:c0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a265c900 2024-05-20T18:34:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.627+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.0 v2:172.21.15.5:6808/688093005 3 ==== osd_op_reply(25 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e500 con 0x5598a265c900 2024-05-20T18:34:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.627+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:26 13.13 13:c8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a242d180 2024-05-20T18:34:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.627+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.7 v2:172.21.15.63:6824/4136124451 5 ==== osd_op_reply(26 [pgnls start_epoch 94 out=98b] v93'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x5598a239e000 con 0x5598a242d180 2024-05-20T18:34:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.627+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:27 13.b 13:d0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a265cd80 2024-05-20T18:34:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.627+0000 7fcae57b6640 1 -- 172.21.15.5:0/89586744 <== osd.5 v2:172.21.15.63:6808/1616066286 3 ==== osd_op_reply(27 [pgnls start_epoch 94 out=76b] v93'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x5598a239e780 con 0x5598a265cd80 2024-05-20T18:34:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.628+0000 7fcae57b6640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:28 13.1b 13:d8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a265db00 2024-05-20T18:34:35.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.628+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.4 v2:172.21.15.63:6800/3673324406 6 ==== osd_op_reply(28 [pgnls start_epoch 94 out=73b] v93'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5598a239e000 con 0x5598a265db00 2024-05-20T18:34:35.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.628+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:29 13.7 13:e0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a242d180 2024-05-20T18:34:35.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.628+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.7 v2:172.21.15.63:6824/4136124451 6 ==== osd_op_reply(29 [pgnls start_epoch 94 out=73b] v93'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5598a239e000 con 0x5598a242d180 2024-05-20T18:34:35.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.628+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:30 13.17 13:e8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a265cd80 2024-05-20T18:34:35.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.629+0000 7fcae57b6640 1 -- 172.21.15.5:0/89586744 <== osd.5 v2:172.21.15.63:6808/1616066286 4 ==== osd_op_reply(30 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e780 con 0x5598a265cd80 2024-05-20T18:34:35.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.629+0000 7fcae57b6640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:31 13.f 13:f0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a265c480 2024-05-20T18:34:35.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.629+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.6 v2:172.21.15.63:6816/3384565140 4 ==== osd_op_reply(31 [pgnls start_epoch 94 out=73b] v89'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5598a239e000 con 0x5598a265c480 2024-05-20T18:34:35.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.629+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:32 13.1f 13:f8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a265c480 2024-05-20T18:34:35.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.629+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.6 v2:172.21.15.63:6816/3384565140 5 ==== osd_op_reply(32 [pgnls start_epoch 94 out=101b] v93'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x5598a239e000 con 0x5598a265c480 2024-05-20T18:34:35.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.629+0000 7fcae7e32740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-05-20T18:34:35.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.629+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:33 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e400 con 0x5598a242da80 2024-05-20T18:34:35.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.630+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.1 v2:172.21.15.5:6815/550664725 5 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5598a239e500 con 0x5598a242da80 2024-05-20T18:34:35.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.630+0000 7fcae7e32740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-05-20T18:34:35.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.630+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:34 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a264f400 con 0x5598a242da80 2024-05-20T18:34:35.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.630+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.1 v2:172.21.15.5:6815/550664725 6 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5598a239e500 con 0x5598a242da80 2024-05-20T18:34:35.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.630+0000 7fcae7e32740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-05-20T18:34:35.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.630+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:35 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a264f800 con 0x5598a242da80 2024-05-20T18:34:35.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.630+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.1 v2:172.21.15.5:6815/550664725 7 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x5598a239e500 con 0x5598a242da80 2024-05-20T18:34:35.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.630+0000 7fcae7e32740 20 datascan.scan_links: injected ino detected 2024-05-20T18:34:35.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.630+0000 7fcae7e32740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-05-20T18:34:35.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.630+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:36 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a264fc00 con 0x5598a242da80 2024-05-20T18:34:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.631+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.1 v2:172.21.15.5:6815/550664725 8 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5598a239e500 con 0x5598a242da80 2024-05-20T18:34:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.631+0000 7fcae7e32740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-05-20T18:34:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.631+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:37 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e000 con 0x5598a242da80 2024-05-20T18:34:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.631+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.1 v2:172.21.15.5:6815/550664725 9 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5598a239e500 con 0x5598a242da80 2024-05-20T18:34:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.631+0000 7fcae7e32740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-05-20T18:34:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.631+0000 7fcae7e32740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-05-20T18:34:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.631+0000 7fcae7e32740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-05-20T18:34:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.631+0000 7fcae7e32740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-05-20T18:34:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.631+0000 7fcae7e32740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-05-20T18:34:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.631+0000 7fcae7e32740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-05-20T18:34:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.631+0000 7fcae7e32740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-05-20T18:34:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.631+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:38 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e400 con 0x5598a265c900 2024-05-20T18:34:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.631+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.0 v2:172.21.15.5:6808/688093005 4 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5598a239e500 con 0x5598a265c900 2024-05-20T18:34:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.631+0000 7fcae7e32740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-05-20T18:34:35.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.631+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:39 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a265c480 2024-05-20T18:34:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.632+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.6 v2:172.21.15.63:6816/3384565140 6 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5598a239e000 con 0x5598a265c480 2024-05-20T18:34:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.632+0000 7fcae7e32740 10 datascan.scan_links: step 1: handling object 100000001f6.00000000 2024-05-20T18:34:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.632+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:40 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a267ec00 con 0x5598a265c000 2024-05-20T18:34:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.632+0000 7fcae57b6640 1 -- 172.21.15.5:0/89586744 <== osd.2 v2:172.21.15.5:6800/921679764 3 ==== osd_op_reply(40 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x5598a239e780 con 0x5598a265c000 2024-05-20T18:34:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.632+0000 7fcae7e32740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-05-20T18:34:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.632+0000 7fcae7e32740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-05-20T18:34:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.632+0000 7fcae7e32740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-05-20T18:34:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.632+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:41 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e000 con 0x5598a242d180 2024-05-20T18:34:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.632+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.7 v2:172.21.15.63:6824/4136124451 7 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5598a239e000 con 0x5598a242d180 2024-05-20T18:34:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.632+0000 7fcae7e32740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-05-20T18:34:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.632+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:42 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e400 con 0x5598a265db00 2024-05-20T18:34:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.633+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.4 v2:172.21.15.63:6800/3673324406 7 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5598a239e000 con 0x5598a265db00 2024-05-20T18:34:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.633+0000 7fcae7e32740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-05-20T18:34:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.633+0000 7fcae7e32740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-05-20T18:34:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.633+0000 7fcae7e32740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-05-20T18:34:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.633+0000 7fcae7e32740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-05-20T18:34:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.633+0000 7fcae7e32740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-05-20T18:34:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.633+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:43 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a264f400 con 0x5598a242d180 2024-05-20T18:34:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.633+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.7 v2:172.21.15.63:6824/4136124451 8 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5598a239e000 con 0x5598a242d180 2024-05-20T18:34:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.633+0000 7fcae7e32740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-05-20T18:34:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.633+0000 7fcae7e32740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-05-20T18:34:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.633+0000 7fcae7e32740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-05-20T18:34:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.633+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:44 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a264f800 con 0x5598a242d180 2024-05-20T18:34:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.634+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.7 v2:172.21.15.63:6824/4136124451 9 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x5598a239e000 con 0x5598a242d180 2024-05-20T18:34:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.634+0000 7fcae7e32740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-05-20T18:34:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.634+0000 7fcae7e32740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-05-20T18:34:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.634+0000 7fcae7e32740 10 datascan.scan_links: step 1: handling object 200.00000002 2024-05-20T18:34:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.634+0000 7fcae7e32740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-05-20T18:34:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.634+0000 7fcae7e32740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-05-20T18:34:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.634+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:45 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a264fc00 con 0x5598a242d180 2024-05-20T18:34:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.634+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.7 v2:172.21.15.63:6824/4136124451 10 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5598a239e000 con 0x5598a242d180 2024-05-20T18:34:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.634+0000 7fcae7e32740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-05-20T18:34:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.634+0000 7fcae7e32740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-05-20T18:34:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.634+0000 7fcae7e32740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-05-20T18:34:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.634+0000 7fcae7e32740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-05-20T18:34:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.634+0000 7fcae7e32740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-05-20T18:34:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.634+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:46 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a265c480 2024-05-20T18:34:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.635+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.6 v2:172.21.15.63:6816/3384565140 7 ==== osd_op_reply(46 1.00000000 [omap-get-vals out=1163b] v0'0 uv9 ondisk = 0) ==== 154+0+1163 (crc 0 0 0) 0x5598a239e000 con 0x5598a265c480 2024-05-20T18:34:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.635+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:47 13.0 13:00000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e000 con 0x5598a242d180 2024-05-20T18:34:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.635+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.7 v2:172.21.15.63:6824/4136124451 11 ==== osd_op_reply(47 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e000 con 0x5598a242d180 2024-05-20T18:34:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.635+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:48 13.10 13:08000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a242da80 2024-05-20T18:34:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.635+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.1 v2:172.21.15.5:6815/550664725 10 ==== osd_op_reply(48 [pgnls start_epoch 94 out=177b] v94'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x5598a239e500 con 0x5598a242da80 2024-05-20T18:34:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.635+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:49 13.8 13:10000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a265c000 2024-05-20T18:34:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.636+0000 7fcae57b6640 1 -- 172.21.15.5:0/89586744 <== osd.2 v2:172.21.15.5:6800/921679764 4 ==== osd_op_reply(49 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e780 con 0x5598a265c000 2024-05-20T18:34:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.636+0000 7fcae57b6640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:50 13.18 13:18000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a242d180 2024-05-20T18:34:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.636+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.7 v2:172.21.15.63:6824/4136124451 12 ==== osd_op_reply(50 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e000 con 0x5598a242d180 2024-05-20T18:34:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.636+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:51 13.4 13:20000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a265c480 2024-05-20T18:34:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.637+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.6 v2:172.21.15.63:6816/3384565140 8 ==== osd_op_reply(51 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e000 con 0x5598a265c480 2024-05-20T18:34:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.637+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:52 13.14 13:28000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a265c900 2024-05-20T18:34:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.637+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.0 v2:172.21.15.5:6808/688093005 5 ==== osd_op_reply(52 [pgnls start_epoch 94 out=149b] v93'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x5598a239e500 con 0x5598a265c900 2024-05-20T18:34:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.637+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:53 13.c 13:30000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a242da80 2024-05-20T18:34:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.637+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.1 v2:172.21.15.5:6815/550664725 11 ==== osd_op_reply(53 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e500 con 0x5598a242da80 2024-05-20T18:34:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.637+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:54 13.1c 13:38000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a265c480 2024-05-20T18:34:35.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.638+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.6 v2:172.21.15.63:6816/3384565140 9 ==== osd_op_reply(54 [pgnls start_epoch 94 out=73b] v93'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5598a239e000 con 0x5598a265c480 2024-05-20T18:34:35.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.638+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:55 13.2 13:40000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a265cd80 2024-05-20T18:34:35.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.638+0000 7fcae57b6640 1 -- 172.21.15.5:0/89586744 <== osd.5 v2:172.21.15.63:6808/1616066286 5 ==== osd_op_reply(55 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e780 con 0x5598a265cd80 2024-05-20T18:34:35.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.638+0000 7fcae57b6640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:56 13.12 13:48000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a265c900 2024-05-20T18:34:35.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.638+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.0 v2:172.21.15.5:6808/688093005 6 ==== osd_op_reply(56 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e500 con 0x5598a265c900 2024-05-20T18:34:35.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.638+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:57 13.a 13:50000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a265c000 2024-05-20T18:34:35.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.639+0000 7fcae57b6640 1 -- 172.21.15.5:0/89586744 <== osd.2 v2:172.21.15.5:6800/921679764 5 ==== osd_op_reply(57 [pgnls start_epoch 94 out=81b] v93'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5598a239e780 con 0x5598a265c000 2024-05-20T18:34:35.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.639+0000 7fcae57b6640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:58 13.1a 13:58000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a242da80 2024-05-20T18:34:35.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.639+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.1 v2:172.21.15.5:6815/550664725 12 ==== osd_op_reply(58 [pgnls start_epoch 94 out=49b] v91'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e500 con 0x5598a242da80 2024-05-20T18:34:35.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.639+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:59 13.6 13:60000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a265db00 2024-05-20T18:34:35.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.639+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.4 v2:172.21.15.63:6800/3673324406 8 ==== osd_op_reply(59 [pgnls start_epoch 94 out=77b] v93'6 uv6 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x5598a239e000 con 0x5598a265db00 2024-05-20T18:34:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.639+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:60 13.16 13:68000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a265d200 2024-05-20T18:34:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.640+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.3 v2:172.21.15.5:6824/1164130599 3 ==== osd_op_reply(60 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e500 con 0x5598a265d200 2024-05-20T18:34:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.640+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:61 13.e 13:70000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a242d180 2024-05-20T18:34:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.640+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.7 v2:172.21.15.63:6824/4136124451 13 ==== osd_op_reply(61 [pgnls start_epoch 94 out=73b] v93'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5598a239e000 con 0x5598a242d180 2024-05-20T18:34:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.640+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:62 13.1e 13:78000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a265db00 2024-05-20T18:34:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.640+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.4 v2:172.21.15.63:6800/3673324406 9 ==== osd_op_reply(62 [pgnls start_epoch 94 out=49b] v91'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e000 con 0x5598a265db00 2024-05-20T18:34:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.640+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:63 13.1 13:80000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a265c480 2024-05-20T18:34:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.641+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.6 v2:172.21.15.63:6816/3384565140 10 ==== osd_op_reply(63 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e000 con 0x5598a265c480 2024-05-20T18:34:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.641+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:64 13.11 13:88000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a242da80 2024-05-20T18:34:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.641+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.1 v2:172.21.15.5:6815/550664725 13 ==== osd_op_reply(64 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e500 con 0x5598a242da80 2024-05-20T18:34:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.641+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:65 13.9 13:90000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a265cd80 2024-05-20T18:34:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.641+0000 7fcae57b6640 1 -- 172.21.15.5:0/89586744 <== osd.5 v2:172.21.15.63:6808/1616066286 6 ==== osd_op_reply(65 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e780 con 0x5598a265cd80 2024-05-20T18:34:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.641+0000 7fcae57b6640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:66 13.19 13:98000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a265d200 2024-05-20T18:34:35.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.642+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.3 v2:172.21.15.5:6824/1164130599 4 ==== osd_op_reply(66 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e500 con 0x5598a265d200 2024-05-20T18:34:35.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.642+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:67 13.5 13:a0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a265db00 2024-05-20T18:34:35.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.642+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.4 v2:172.21.15.63:6800/3673324406 10 ==== osd_op_reply(67 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e000 con 0x5598a265db00 2024-05-20T18:34:35.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.642+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:68 13.15 13:a8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a265db00 2024-05-20T18:34:35.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.642+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.4 v2:172.21.15.63:6800/3673324406 11 ==== osd_op_reply(68 [pgnls start_epoch 94 out=97b] v93'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x5598a239e000 con 0x5598a265db00 2024-05-20T18:34:35.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.642+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:69 13.d 13:b0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a265db00 2024-05-20T18:34:35.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.643+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.4 v2:172.21.15.63:6800/3673324406 12 ==== osd_op_reply(69 [pgnls start_epoch 94 out=74b] v89'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x5598a239e000 con 0x5598a265db00 2024-05-20T18:34:35.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.643+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:70 13.1d 13:b8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a242d180 2024-05-20T18:34:35.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.643+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.7 v2:172.21.15.63:6824/4136124451 14 ==== osd_op_reply(70 [pgnls start_epoch 94 out=73b] v93'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5598a239e000 con 0x5598a242d180 2024-05-20T18:34:35.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.643+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:71 13.3 13:c0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a265c900 2024-05-20T18:34:35.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.643+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.0 v2:172.21.15.5:6808/688093005 7 ==== osd_op_reply(71 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e500 con 0x5598a265c900 2024-05-20T18:34:35.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.643+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:72 13.13 13:c8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a242d180 2024-05-20T18:34:35.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.644+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.7 v2:172.21.15.63:6824/4136124451 15 ==== osd_op_reply(72 [pgnls start_epoch 94 out=98b] v93'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x5598a239e000 con 0x5598a242d180 2024-05-20T18:34:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.644+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:73 13.b 13:d0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a265cd80 2024-05-20T18:34:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.644+0000 7fcae57b6640 1 -- 172.21.15.5:0/89586744 <== osd.5 v2:172.21.15.63:6808/1616066286 7 ==== osd_op_reply(73 [pgnls start_epoch 94 out=76b] v93'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x5598a239e780 con 0x5598a265cd80 2024-05-20T18:34:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.644+0000 7fcae57b6640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:74 13.1b 13:d8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a265db00 2024-05-20T18:34:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.644+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.4 v2:172.21.15.63:6800/3673324406 13 ==== osd_op_reply(74 [pgnls start_epoch 94 out=73b] v93'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5598a239e000 con 0x5598a265db00 2024-05-20T18:34:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.644+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:75 13.7 13:e0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a242d180 2024-05-20T18:34:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.645+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.7 v2:172.21.15.63:6824/4136124451 16 ==== osd_op_reply(75 [pgnls start_epoch 94 out=73b] v93'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5598a239e000 con 0x5598a242d180 2024-05-20T18:34:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.645+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:76 13.17 13:e8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a265cd80 2024-05-20T18:34:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.645+0000 7fcae57b6640 1 -- 172.21.15.5:0/89586744 <== osd.5 v2:172.21.15.63:6808/1616066286 8 ==== osd_op_reply(76 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598a239e780 con 0x5598a265cd80 2024-05-20T18:34:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.645+0000 7fcae57b6640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:77 13.f 13:f0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a265c480 2024-05-20T18:34:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.645+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.6 v2:172.21.15.63:6816/3384565140 11 ==== osd_op_reply(77 [pgnls start_epoch 94 out=73b] v89'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5598a239e000 con 0x5598a265c480 2024-05-20T18:34:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.645+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:78 13.1f 13:f8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e800 con 0x5598a265c480 2024-05-20T18:34:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.646+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.6 v2:172.21.15.63:6816/3384565140 12 ==== osd_op_reply(78 [pgnls start_epoch 94 out=101b] v93'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x5598a239e000 con 0x5598a265c480 2024-05-20T18:34:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.646+0000 7fcae7e32740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-05-20T18:34:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.646+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:79 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e400 con 0x5598a242da80 2024-05-20T18:34:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.646+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.1 v2:172.21.15.5:6815/550664725 14 ==== osd_op_reply(79 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5598a239e500 con 0x5598a242da80 2024-05-20T18:34:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.646+0000 7fcae7e32740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-05-20T18:34:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.646+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:80 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a242da80 2024-05-20T18:34:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.646+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.1 v2:172.21.15.5:6815/550664725 15 ==== osd_op_reply(80 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5598a239e500 con 0x5598a242da80 2024-05-20T18:34:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.646+0000 7fcae7e32740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-05-20T18:34:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.646+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:81 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a314d400 con 0x5598a242da80 2024-05-20T18:34:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.647+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.1 v2:172.21.15.5:6815/550664725 16 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x5598a239e500 con 0x5598a242da80 2024-05-20T18:34:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.647+0000 7fcae7e32740 20 datascan.scan_links: injected ino detected 2024-05-20T18:34:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.647+0000 7fcae7e32740 -1 datascan.scan_links: Bad nlink on 0x10000000001 expected 2 has 1 2024-05-20T18:34:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.647+0000 7fcae7e32740 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-05-20T18:34:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.647+0000 7fcae7e32740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-05-20T18:34:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.647+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:82 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a314d000 con 0x5598a242da80 2024-05-20T18:34:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.647+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.1 v2:172.21.15.5:6815/550664725 17 ==== osd_op_reply(82 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5598a239e500 con 0x5598a242da80 2024-05-20T18:34:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.647+0000 7fcae7e32740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-05-20T18:34:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.647+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:83 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a267f000 con 0x5598a242da80 2024-05-20T18:34:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.647+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.1 v2:172.21.15.5:6815/550664725 18 ==== osd_op_reply(83 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5598a239e500 con 0x5598a242da80 2024-05-20T18:34:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.647+0000 7fcae7e32740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-05-20T18:34:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.647+0000 7fcae7e32740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-05-20T18:34:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.647+0000 7fcae7e32740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-05-20T18:34:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.647+0000 7fcae7e32740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-05-20T18:34:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.647+0000 7fcae7e32740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-05-20T18:34:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.647+0000 7fcae7e32740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-05-20T18:34:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.647+0000 7fcae7e32740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-05-20T18:34:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.647+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:84 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a267f400 con 0x5598a265c900 2024-05-20T18:34:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.648+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.0 v2:172.21.15.5:6808/688093005 8 ==== osd_op_reply(84 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5598a239e500 con 0x5598a265c900 2024-05-20T18:34:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.648+0000 7fcae7e32740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-05-20T18:34:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.648+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:85 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a267f800 con 0x5598a265c480 2024-05-20T18:34:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.648+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.6 v2:172.21.15.63:6816/3384565140 13 ==== osd_op_reply(85 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5598a239e000 con 0x5598a265c480 2024-05-20T18:34:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.648+0000 7fcae7e32740 10 datascan.scan_links: step 2: handling object 100000001f6.00000000 2024-05-20T18:34:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.648+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:86 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a267fc00 con 0x5598a265c000 2024-05-20T18:34:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.648+0000 7fcae57b6640 1 -- 172.21.15.5:0/89586744 <== osd.2 v2:172.21.15.5:6800/921679764 6 ==== osd_op_reply(86 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x5598a239e780 con 0x5598a265c000 2024-05-20T18:34:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.648+0000 7fcae7e32740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-05-20T18:34:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.648+0000 7fcae7e32740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-05-20T18:34:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.648+0000 7fcae7e32740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-05-20T18:34:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.648+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:87 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a268a000 con 0x5598a242d180 2024-05-20T18:34:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.649+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.7 v2:172.21.15.63:6824/4136124451 17 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5598a239e000 con 0x5598a242d180 2024-05-20T18:34:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.649+0000 7fcae7e32740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-05-20T18:34:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.649+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:88 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a268a400 con 0x5598a265db00 2024-05-20T18:34:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.649+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.4 v2:172.21.15.63:6800/3673324406 14 ==== osd_op_reply(88 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5598a239e000 con 0x5598a265db00 2024-05-20T18:34:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.649+0000 7fcae7e32740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-05-20T18:34:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.649+0000 7fcae7e32740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-05-20T18:34:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.649+0000 7fcae7e32740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-05-20T18:34:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.649+0000 7fcae7e32740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-05-20T18:34:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.649+0000 7fcae7e32740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-05-20T18:34:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.649+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a268a800 con 0x5598a242d180 2024-05-20T18:34:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.649+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.7 v2:172.21.15.63:6824/4136124451 18 ==== osd_op_reply(89 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5598a239e000 con 0x5598a242d180 2024-05-20T18:34:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.649+0000 7fcae7e32740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-05-20T18:34:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.650+0000 7fcae7e32740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-05-20T18:34:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.650+0000 7fcae7e32740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-05-20T18:34:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.650+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:90 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a268ac00 con 0x5598a242d180 2024-05-20T18:34:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.650+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.7 v2:172.21.15.63:6824/4136124451 19 ==== osd_op_reply(90 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x5598a239e000 con 0x5598a242d180 2024-05-20T18:34:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.650+0000 7fcae7e32740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-05-20T18:34:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.650+0000 7fcae7e32740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-05-20T18:34:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.650+0000 7fcae7e32740 10 datascan.scan_links: step 2: handling object 200.00000002 2024-05-20T18:34:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.650+0000 7fcae7e32740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-05-20T18:34:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.650+0000 7fcae7e32740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-05-20T18:34:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.650+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:91 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a268b000 con 0x5598a242d180 2024-05-20T18:34:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.650+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.7 v2:172.21.15.63:6824/4136124451 20 ==== osd_op_reply(91 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5598a239e000 con 0x5598a242d180 2024-05-20T18:34:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.650+0000 7fcae7e32740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-05-20T18:34:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.650+0000 7fcae7e32740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-05-20T18:34:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.650+0000 7fcae7e32740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-05-20T18:34:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.650+0000 7fcae7e32740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-05-20T18:34:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.650+0000 7fcae7e32740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-05-20T18:34:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.650+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:92 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a268b400 con 0x5598a265c480 2024-05-20T18:34:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.651+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.6 v2:172.21.15.63:6816/3384565140 14 ==== osd_op_reply(92 1.00000000 [omap-get-vals out=1163b] v0'0 uv9 ondisk = 0) ==== 154+0+1163 (crc 0 0 0) 0x5598a239e000 con 0x5598a265c480 2024-05-20T18:34:35.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.651+0000 7fcae7e32740 10 datascan.scan_links: processing 0 dup_primaries, 1 remote_links 2024-05-20T18:34:35.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.651+0000 7fcae7e32740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-05-20T18:34:35.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.651+0000 7fcae7e32740 10 datascan.scan_links: processing 1 bad_nlink_inos 2024-05-20T18:34:35.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.651+0000 7fcae7e32740 10 datascan.scan_links: handling bad_nlink_ino 0x10000000001 2024-05-20T18:34:35.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.651+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:93 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a268b800 con 0x5598a242da80 2024-05-20T18:34:35.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.651+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.1 v2:172.21.15.5:6815/550664725 19 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv18 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x5598a239e500 con 0x5598a242da80 2024-05-20T18:34:35.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.651+0000 7fcae7e32740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-05-20T18:34:35.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.651+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:94 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x5598a268bc00 con 0x5598a242da80 2024-05-20T18:34:35.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.652+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.1 v2:172.21.15.5:6815/550664725 20 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v94'19 uv19 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5598a239e500 con 0x5598a242da80 2024-05-20T18:34:35.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.652+0000 7fcae7e32740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-05-20T18:34:35.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.652+0000 7fcae7e32740 10 datascan.scan_links: processing 1 injected_inos 2024-05-20T18:34:35.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.652+0000 7fcae7e32740 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-05-20T18:34:35.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.652+0000 7fcae7e32740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:file 2024-05-20T18:34:35.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.652+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:95 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a267e400 con 0x5598a242da80 2024-05-20T18:34:35.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.653+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.1 v2:172.21.15.5:6815/550664725 21 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv19 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x5598a239e500 con 0x5598a242da80 2024-05-20T18:34:35.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.653+0000 7fcae7e32740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-05-20T18:34:35.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.653+0000 7fcae7e32740 20 datascan.scan_links: first is now 2 2024-05-20T18:34:35.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.653+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:96 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x5598a267f000 con 0x5598a242da80 2024-05-20T18:34:35.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.654+0000 7fcae5fb7640 1 -- 172.21.15.5:0/89586744 <== osd.1 v2:172.21.15.5:6815/550664725 22 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v94'20 uv20 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5598a239e500 con 0x5598a242da80 2024-05-20T18:34:35.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.654+0000 7fcae7e32740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-05-20T18:34:35.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.654+0000 7fcae7e32740 10 datascan.scan_links: updating inotable 2024-05-20T18:34:35.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.654+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:97 13.13 13:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a264e000 con 0x5598a242d180 2024-05-20T18:34:35.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.655+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.7 v2:172.21.15.63:6824/4136124451 21 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x5598a239e000 con 0x5598a242d180 2024-05-20T18:34:35.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.655+0000 7fcae7e32740 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x100000001f7~0xfffffffe09] 2024-05-20T18:34:35.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.655+0000 7fcae7e32740 10 mds.0.inotable: skip_inos now [0x100000001f7~0xfffffffe09] 2024-05-20T18:34:35.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.655+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:98 13.13 13:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x5598a264fc00 con 0x5598a242d180 2024-05-20T18:34:35.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.656+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.7 v2:172.21.15.63:6824/4136124451 22 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v94'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x5598a239e000 con 0x5598a242d180 2024-05-20T18:34:35.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.656+0000 7fcae7e32740 10 datascan.scan_links: updating snaptable 2024-05-20T18:34:35.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.656+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:99 13.d 13:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5598a264f800 con 0x5598a265db00 2024-05-20T18:34:35.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae4fb5640 1 -- 172.21.15.5:0/89586744 <== osd.4 v2:172.21.15.63:6800/3673324406 15 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x5598a239e000 con 0x5598a265db00 2024-05-20T18:34:35.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x5598a265c900 msgr2=0x5598a2655080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x5598a265c900 0x5598a2655080 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x5598a242da80 msgr2=0x5598a2654000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x5598a242da80 0x5598a2654000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x5598a265c000 msgr2=0x5598a2654580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x5598a265c000 0x5598a2654580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x5598a265d200 msgr2=0x5598a2655b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x5598a265d200 0x5598a2655b80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x5598a265db00 msgr2=0x5598a2656680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x5598a265db00 0x5598a2656680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x5598a265cd80 msgr2=0x5598a2655600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae57b6640 1 -- 172.21.15.5:0/89586744 reap_dead start 2024-05-20T18:34:35.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x5598a265cd80 0x5598a2655600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x5598a265c480 msgr2=0x5598a2654b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x5598a265c480 0x5598a2654b00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x5598a242d180 msgr2=0x5598a2400100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae57b6640 1 -- 172.21.15.5:0/89586744 reap_dead start 2024-05-20T18:34:35.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x5598a242d180 0x5598a2400100 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae57b6640 1 -- 172.21.15.5:0/89586744 reap_dead start 2024-05-20T18:34:35.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x5598a242b200 msgr2=0x5598a2401700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x5598a242b200 0x5598a2401700 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x5598a236ae70 tx=0x5598a25daae0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598a242ad80 msgr2=0x5598a2401180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598a242ad80 0x5598a2401180 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x5598a313fa70 tx=0x5598a23681e0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 shutdown_connections 2024-05-20T18:34:35.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x5598a242b200 0x5598a2401700 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 --2- 172.21.15.5:0/89586744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598a242ad80 0x5598a2401180 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 >> 172.21.15.5:0/89586744 conn(0x5598a242a900 msgr2=0x5598a2399800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:35.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 shutdown_connections 2024-05-20T18:34:35.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 -- 172.21.15.5:0/89586744 wait complete. 2024-05-20T18:34:35.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 -- 172.21.15.5:0/1548317264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598a242bb00 msgr2=0x5598a23fe580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 --2- 172.21.15.5:0/1548317264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598a242bb00 0x5598a23fe580 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x5598a308bd40 tx=0x5598a23682a0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 -- 172.21.15.5:0/1548317264 shutdown_connections 2024-05-20T18:34:35.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 --2- 172.21.15.5:0/1548317264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5598a242bf80 0x5598a23ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 --2- 172.21.15.5:0/1548317264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5598a242bb00 0x5598a23fe580 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 --2- 172.21.15.5:0/1548317264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5598a242b680 0x5598a23feb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 -- 172.21.15.5:0/1548317264 >> 172.21.15.5:0/1548317264 conn(0x5598a242c400 msgr2=0x5598a2399000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:35.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 -- 172.21.15.5:0/1548317264 shutdown_connections 2024-05-20T18:34:35.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.657+0000 7fcae7e32740 1 -- 172.21.15.5:0/1548317264 wait complete. 2024-05-20T18:34:35.671 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.118709, result: 2024-05-20T18:34:35.671 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-20T18:34:35.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.788+0000 7fde99109640 1 Processor -- start 2024-05-20T18:34:35.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.788+0000 7fde99109640 1 -- start start 2024-05-20T18:34:35.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.789+0000 7fde99109640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fde94168050 0x7fde9416a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.789+0000 7fde99109640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde940d0630 con 0x7fde9406d3d0 2024-05-20T18:34:35.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.789+0000 7fde99109640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde940d39c0 con 0x7fde94162820 2024-05-20T18:34:35.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.789+0000 7fde99109640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fde940d3ca0 con 0x7fde94168050 2024-05-20T18:34:35.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.790+0000 7fde9359e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fde94168050 0x7fde9416a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.790+0000 7fde9259c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fde9406d3d0 0x7fde9406d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36446/0 (socket says 172.21.15.5:36446) 2024-05-20T18:34:35.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.790+0000 7fde9359e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fde94168050 0x7fde9416a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38424/0 (socket says 172.21.15.5:38424) 2024-05-20T18:34:35.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.790+0000 7fde9259c640 1 -- 172.21.15.5:0/191986147 learned_addr learned my addr 172.21.15.5:0/191986147 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:34:35.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.790+0000 7fde9359e640 1 -- 172.21.15.5:0/191986147 >> v1:172.21.15.63:6789/0 conn(0x7fde94162820 legacy=0x7fde940b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.790+0000 7fde9359e640 1 -- 172.21.15.5:0/191986147 >> v1:172.21.15.5:6789/0 conn(0x7fde9406d3d0 legacy=0x7fde9406d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.790+0000 7fde9359e640 1 -- 172.21.15.5:0/191986147 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde940d3f80 con 0x7fde94168050 2024-05-20T18:34:35.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.790+0000 7fde91d9b640 1 -- 172.21.15.5:0/191986147 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 946261669 0 0) 0x7fde940d0630 con 0x7fde9406d3d0 2024-05-20T18:34:35.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.791+0000 7fde9359e640 1 --2- 172.21.15.5:0/191986147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde94168050 0x7fde9416a470 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fde8c009d90 tx=0x7fde8c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484ebe20b20ee7a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.791+0000 7fde91d9b640 1 -- 172.21.15.5:0/191986147 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c007740 con 0x7fde94168050 2024-05-20T18:34:35.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.791+0000 7fde91d9b640 1 -- 172.21.15.5:0/191986147 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde8c0078e0 con 0x7fde94168050 2024-05-20T18:34:35.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.791+0000 7fde91d9b640 1 -- 172.21.15.5:0/191986147 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c007cb0 con 0x7fde94168050 2024-05-20T18:34:35.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.792+0000 7fde99109640 1 -- 172.21.15.5:0/191986147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde94168050 msgr2=0x7fde9416a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.792+0000 7fde99109640 1 --2- 172.21.15.5:0/191986147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde94168050 0x7fde9416a470 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fde8c009d90 tx=0x7fde8c02a6f0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.792+0000 7fde99109640 1 -- 172.21.15.5:0/191986147 shutdown_connections 2024-05-20T18:34:35.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.792+0000 7fde99109640 1 --2- 172.21.15.5:0/191986147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde94168050 0x7fde9416a470 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.792+0000 7fde99109640 1 -- 172.21.15.5:0/191986147 >> 172.21.15.5:0/191986147 conn(0x7fde940d4bd0 msgr2=0x7fde940d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:35.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.792+0000 7fde99109640 1 -- 172.21.15.5:0/191986147 shutdown_connections 2024-05-20T18:34:35.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.792+0000 7fde99109640 1 -- 172.21.15.5:0/191986147 wait complete. 2024-05-20T18:34:35.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.794+0000 7fde99109640 1 Processor -- start 2024-05-20T18:34:35.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.794+0000 7fde99109640 1 -- start start 2024-05-20T18:34:35.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.795+0000 7fde99109640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde9406d3d0 0x7fde940b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.795+0000 7fde99109640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde94162820 0x7fde940cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.795+0000 7fde99109640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde94168050 0x7fde940c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.795+0000 7fde99109640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fde940b91a0 con 0x7fde94162820 2024-05-20T18:34:35.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.795+0000 7fde99109640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fde940d0300 con 0x7fde94168050 2024-05-20T18:34:35.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.795+0000 7fde99109640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fde940b4050 con 0x7fde9406d3d0 2024-05-20T18:34:35.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.795+0000 7fde9259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde94162820 0x7fde940cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.795+0000 7fde9259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde94162820 0x7fde940cbc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38426/0 (socket says 172.21.15.5:38426) 2024-05-20T18:34:35.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.795+0000 7fde9259c640 1 -- 172.21.15.5:0/4042516643 learned_addr learned my addr 172.21.15.5:0/4042516643 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:35.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.796+0000 7fde9359e640 1 --2- 172.21.15.5:0/4042516643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde94168050 0x7fde940c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.796+0000 7fde92d9d640 1 --2- 172.21.15.5:0/4042516643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde9406d3d0 0x7fde940b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.796+0000 7fde9259c640 1 -- 172.21.15.5:0/4042516643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde9406d3d0 msgr2=0x7fde940b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.796+0000 7fde9259c640 1 --2- 172.21.15.5:0/4042516643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde9406d3d0 0x7fde940b42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.796+0000 7fde9259c640 1 -- 172.21.15.5:0/4042516643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde94168050 msgr2=0x7fde940c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:35.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.796+0000 7fde9259c640 1 --2- 172.21.15.5:0/4042516643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde94168050 0x7fde940c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:35.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.796+0000 7fde9259c640 1 -- 172.21.15.5:0/4042516643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9415b0b0 con 0x7fde94162820 2024-05-20T18:34:35.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.796+0000 7fde9259c640 1 --2- 172.21.15.5:0/4042516643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde94162820 0x7fde940cbc60 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fde8800c420 tx=0x7fde8800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.796+0000 7fde7b7fe640 1 -- 172.21.15.5:0/4042516643 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde880074e0 con 0x7fde94162820 2024-05-20T18:34:35.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.796+0000 7fde7b7fe640 1 -- 172.21.15.5:0/4042516643 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde880047e0 con 0x7fde94162820 2024-05-20T18:34:35.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.796+0000 7fde99109640 1 -- 172.21.15.5:0/4042516643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde9415b390 con 0x7fde94162820 2024-05-20T18:34:35.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.796+0000 7fde7b7fe640 1 -- 172.21.15.5:0/4042516643 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde88004c10 con 0x7fde94162820 2024-05-20T18:34:35.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.796+0000 7fde99109640 1 -- 172.21.15.5:0/4042516643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde9415b950 con 0x7fde94162820 2024-05-20T18:34:35.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.797+0000 7fde99109640 1 -- 172.21.15.5:0/4042516643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde9414fff0 con 0x7fde94162820 2024-05-20T18:34:35.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.798+0000 7fde7b7fe640 1 -- 172.21.15.5:0/4042516643 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fde88011070 con 0x7fde94162820 2024-05-20T18:34:35.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.798+0000 7fde7b7fe640 1 --2- 172.21.15.5:0/4042516643 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fde8006be30 0x7fde8006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:35.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.799+0000 7fde92d9d640 1 --2- 172.21.15.5:0/4042516643 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fde8006be30 0x7fde8006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:35.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.799+0000 7fde7b7fe640 1 -- 172.21.15.5:0/4042516643 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7fde8808ac00 con 0x7fde94162820 2024-05-20T18:34:35.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.799+0000 7fde92d9d640 1 --2- 172.21.15.5:0/4042516643 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fde8006be30 0x7fde8006e340 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fde84000f60 tx=0x7fde84009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:35.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:35.803+0000 7fde7b7fe640 1 -- 172.21.15.5:0/4042516643 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fde9414fff0 con 0x7fde94162820 2024-05-20T18:34:36.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.026+0000 7fde99109640 1 -- 172.21.15.5:0/4042516643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fde94075b00 con 0x7fde94162820 2024-05-20T18:34:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.302+0000 7fde7b7fe640 1 -- 172.21.15.5:0/4042516643 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v69) ==== 162+0+0 (secure 0 0 0) 0x7fde88091080 con 0x7fde94162820 2024-05-20T18:34:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-20T18:34:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.304+0000 7fde99109640 1 -- 172.21.15.5:0/4042516643 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fde8006be30 msgr2=0x7fde8006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.304+0000 7fde99109640 1 --2- 172.21.15.5:0/4042516643 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fde8006be30 0x7fde8006e340 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fde84000f60 tx=0x7fde84009040 comp rx=0 tx=0).stop 2024-05-20T18:34:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.304+0000 7fde99109640 1 -- 172.21.15.5:0/4042516643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde94162820 msgr2=0x7fde940cbc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.304+0000 7fde99109640 1 --2- 172.21.15.5:0/4042516643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde94162820 0x7fde940cbc60 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fde8800c420 tx=0x7fde8800c8e0 comp rx=0 tx=0).stop 2024-05-20T18:34:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.304+0000 7fde99109640 1 -- 172.21.15.5:0/4042516643 shutdown_connections 2024-05-20T18:34:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.304+0000 7fde99109640 1 --2- 172.21.15.5:0/4042516643 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fde8006be30 0x7fde8006e340 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.304+0000 7fde99109640 1 --2- 172.21.15.5:0/4042516643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde94168050 0x7fde940c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.304+0000 7fde99109640 1 --2- 172.21.15.5:0/4042516643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde94162820 0x7fde940cbc60 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.304+0000 7fde99109640 1 --2- 172.21.15.5:0/4042516643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde9406d3d0 0x7fde940b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.304+0000 7fde99109640 1 -- 172.21.15.5:0/4042516643 >> 172.21.15.5:0/4042516643 conn(0x7fde940d4bd0 msgr2=0x7fde94153520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.304+0000 7fde99109640 1 -- 172.21.15.5:0/4042516643 shutdown_connections 2024-05-20T18:34:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.304+0000 7fde99109640 1 -- 172.21.15.5:0/4042516643 wait complete. 2024-05-20T18:34:36.321 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:34:36.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.436+0000 7f2e03051640 1 Processor -- start 2024-05-20T18:34:36.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.436+0000 7f2e03051640 1 -- start start 2024-05-20T18:34:36.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.437+0000 7f2e03051640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2dfc0d1bc0 0x7f2dfc0d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:36.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.437+0000 7f2e03051640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dfc0d0370 con 0x7f2dfc163a10 2024-05-20T18:34:36.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.437+0000 7f2e03051640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dfc0d0650 con 0x7f2dfc0b4010 2024-05-20T18:34:36.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.437+0000 7f2e03051640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2dfc0d3ca0 con 0x7f2dfc0d1bc0 2024-05-20T18:34:36.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.437+0000 7f2e00ded640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2dfc163a10 0x7f2dfc0cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36460/0 (socket says 172.21.15.5:36460) 2024-05-20T18:34:36.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.437+0000 7f2e00ded640 1 -- 172.21.15.5:0/1483542094 learned_addr learned my addr 172.21.15.5:0/1483542094 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:34:36.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.438+0000 7f2e015ee640 1 --2- 172.21.15.5:0/1483542094 >> v2:172.21.15.63:3300/0 conn(0x7f2dfc0d1bc0 0x7f2dfc0d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:36.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.438+0000 7f2dfb7fe640 1 -- 172.21.15.5:0/1483542094 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1689190397 0 0) 0x7f2dfc0d0370 con 0x7f2dfc163a10 2024-05-20T18:34:36.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.438+0000 7f2dfb7fe640 1 -- 172.21.15.5:0/1483542094 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2de8003520 con 0x7f2dfc163a10 2024-05-20T18:34:36.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.438+0000 7f2dfb7fe640 1 -- 172.21.15.5:0/1483542094 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1553321375 0 0) 0x7f2de8003520 con 0x7f2dfc163a10 2024-05-20T18:34:36.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.438+0000 7f2dfb7fe640 1 -- 172.21.15.5:0/1483542094 >> v2:172.21.15.63:3300/0 conn(0x7f2dfc0d1bc0 msgr2=0x7f2dfc0d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:36.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.438+0000 7f2dfb7fe640 1 --2- 172.21.15.5:0/1483542094 >> v2:172.21.15.63:3300/0 conn(0x7f2dfc0d1bc0 0x7f2dfc0d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:36.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.438+0000 7f2dfb7fe640 1 -- 172.21.15.5:0/1483542094 >> v1:172.21.15.63:6789/0 conn(0x7f2dfc0b4010 legacy=0x7f2dfc0d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:36.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.438+0000 7f2dfb7fe640 1 -- 172.21.15.5:0/1483542094 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dfc0d3650 con 0x7f2dfc163a10 2024-05-20T18:34:36.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.438+0000 7f2dfb7fe640 1 -- 172.21.15.5:0/1483542094 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f2de4001670 con 0x7f2dfc163a10 2024-05-20T18:34:36.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.438+0000 7f2dfb7fe640 1 -- 172.21.15.5:0/1483542094 >> v1:172.21.15.5:6789/0 conn(0x7f2dfc163a10 legacy=0x7f2dfc0cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:36.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.438+0000 7f2dfb7fe640 1 --2- 172.21.15.5:0/1483542094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2de80039b0 0x7f2de8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:36.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.438+0000 7f2dfb7fe640 1 --2- 172.21.15.5:0/1483542094 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2de8004800 0x7f2de8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.438+0000 7f2dfb7fe640 1 --2- 172.21.15.5:0/1483542094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2de80051c0 0x7f2de800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.438+0000 7f2dfb7fe640 1 -- 172.21.15.5:0/1483542094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc0d3ca0 con 0x7f2de80051c0 2024-05-20T18:34:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.438+0000 7f2dfb7fe640 1 -- 172.21.15.5:0/1483542094 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2de800c160 con 0x7f2de80039b0 2024-05-20T18:34:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.438+0000 7f2dfb7fe640 1 -- 172.21.15.5:0/1483542094 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2de800c360 con 0x7f2de8004800 2024-05-20T18:34:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.438+0000 7f2e015ee640 1 --2- 172.21.15.5:0/1483542094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2de80051c0 0x7f2de800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.438+0000 7f2e00ded640 1 --2- 172.21.15.5:0/1483542094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2de80039b0 0x7f2de8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.438+0000 7f2dfbfff640 1 --2- 172.21.15.5:0/1483542094 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2de8004800 0x7f2de8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.439+0000 7f2e015ee640 1 -- 172.21.15.5:0/1483542094 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2de8004800 msgr2=0x7f2de8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.439+0000 7f2e015ee640 1 --2- 172.21.15.5:0/1483542094 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2de8004800 0x7f2de8004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.439+0000 7f2e015ee640 1 -- 172.21.15.5:0/1483542094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2de80039b0 msgr2=0x7f2de8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.439+0000 7f2e015ee640 1 --2- 172.21.15.5:0/1483542094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2de80039b0 0x7f2de8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.439+0000 7f2e015ee640 1 -- 172.21.15.5:0/1483542094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de8003520 con 0x7f2de80051c0 2024-05-20T18:34:36.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.439+0000 7f2e015ee640 1 -- 172.21.15.5:0/1483542094 reap_dead start 2024-05-20T18:34:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.439+0000 7f2e015ee640 1 --2- 172.21.15.5:0/1483542094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2de80051c0 0x7f2de800b9c0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f2df4009fc0 tx=0x7f2de8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c73cb98daaa80d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.439+0000 7f2dfb7fe640 1 -- 172.21.15.5:0/1483542094 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4009c10 con 0x7f2de80051c0 2024-05-20T18:34:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.440+0000 7f2dfb7fe640 1 -- 172.21.15.5:0/1483542094 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2df4003bd0 con 0x7f2de80051c0 2024-05-20T18:34:36.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.440+0000 7f2dfb7fe640 1 -- 172.21.15.5:0/1483542094 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4004020 con 0x7f2de80051c0 2024-05-20T18:34:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.440+0000 7f2e03051640 1 -- 172.21.15.5:0/1483542094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2de80051c0 msgr2=0x7f2de800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.440+0000 7f2e03051640 1 --2- 172.21.15.5:0/1483542094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2de80051c0 0x7f2de800b9c0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f2df4009fc0 tx=0x7f2de8005830 comp rx=0 tx=0).stop 2024-05-20T18:34:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.440+0000 7f2e03051640 1 -- 172.21.15.5:0/1483542094 shutdown_connections 2024-05-20T18:34:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.440+0000 7f2e03051640 1 --2- 172.21.15.5:0/1483542094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2de80051c0 0x7f2de800b9c0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.440+0000 7f2e03051640 1 -- 172.21.15.5:0/1483542094 >> 172.21.15.5:0/1483542094 conn(0x7f2dfc06d240 msgr2=0x7f2dfc06d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.440+0000 7f2e03051640 1 -- 172.21.15.5:0/1483542094 shutdown_connections 2024-05-20T18:34:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.440+0000 7f2e03051640 1 -- 172.21.15.5:0/1483542094 wait complete. 2024-05-20T18:34:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.442+0000 7f2e03051640 1 Processor -- start 2024-05-20T18:34:36.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.442+0000 7f2e03051640 1 -- start start 2024-05-20T18:34:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.443+0000 7f2e03051640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2de80051c0 0x7f2dfc0b6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.443+0000 7f2e03051640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2dfc153590 0x7f2dfc159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.443+0000 7f2e03051640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2dfc15a280 0x7f2dfc0b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.443+0000 7f2e03051640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc0d42b0 con 0x7f2dfc15a280 2024-05-20T18:34:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.444+0000 7f2e03051640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc0d4130 con 0x7f2de80051c0 2024-05-20T18:34:36.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.444+0000 7f2e03051640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2dfc0d4430 con 0x7f2dfc153590 2024-05-20T18:34:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.444+0000 7f2e015ee640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2dfc15a280 0x7f2dfc0b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.444+0000 7f2e015ee640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2dfc15a280 0x7f2dfc0b4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38442/0 (socket says 172.21.15.5:38442) 2024-05-20T18:34:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.444+0000 7f2e015ee640 1 -- 172.21.15.5:0/3564711845 learned_addr learned my addr 172.21.15.5:0/3564711845 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.444+0000 7f2e00ded640 1 --2- 172.21.15.5:0/3564711845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2de80051c0 0x7f2dfc0b6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.444+0000 7f2dfbfff640 1 --2- 172.21.15.5:0/3564711845 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2dfc153590 0x7f2dfc159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.444+0000 7f2e015ee640 1 -- 172.21.15.5:0/3564711845 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2dfc153590 msgr2=0x7f2dfc159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.444+0000 7f2e015ee640 1 --2- 172.21.15.5:0/3564711845 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2dfc153590 0x7f2dfc159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.444+0000 7f2e015ee640 1 -- 172.21.15.5:0/3564711845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2de80051c0 msgr2=0x7f2dfc0b6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.444+0000 7f2e015ee640 1 --2- 172.21.15.5:0/3564711845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2de80051c0 0x7f2dfc0b6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.444+0000 7f2e015ee640 1 -- 172.21.15.5:0/3564711845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dfc159280 con 0x7f2dfc15a280 2024-05-20T18:34:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.444+0000 7f2e015ee640 1 --2- 172.21.15.5:0/3564711845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2dfc15a280 0x7f2dfc0b4c10 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f2df402ae60 tx=0x7f2df4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.444+0000 7f2df97fa640 1 -- 172.21.15.5:0/3564711845 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4009c10 con 0x7f2dfc15a280 2024-05-20T18:34:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.444+0000 7f2df97fa640 1 -- 172.21.15.5:0/3564711845 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2df4044070 con 0x7f2dfc15a280 2024-05-20T18:34:36.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.444+0000 7f2df97fa640 1 -- 172.21.15.5:0/3564711845 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df403d430 con 0x7f2dfc15a280 2024-05-20T18:34:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.444+0000 7f2e03051640 1 -- 172.21.15.5:0/3564711845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dfc15a690 con 0x7f2dfc15a280 2024-05-20T18:34:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.444+0000 7f2e03051640 1 -- 172.21.15.5:0/3564711845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dfc1589e0 con 0x7f2dfc15a280 2024-05-20T18:34:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.445+0000 7f2e03051640 1 -- 172.21.15.5:0/3564711845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dfc075b00 con 0x7f2dfc15a280 2024-05-20T18:34:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.446+0000 7f2df97fa640 1 -- 172.21.15.5:0/3564711845 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2df403b040 con 0x7f2dfc15a280 2024-05-20T18:34:36.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.446+0000 7f2df97fa640 1 --2- 172.21.15.5:0/3564711845 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2ddc06be30 0x7f2ddc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:36.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.446+0000 7f2df97fa640 1 -- 172.21.15.5:0/3564711845 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7f2df40ab940 con 0x7f2dfc15a280 2024-05-20T18:34:36.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.447+0000 7f2e00ded640 1 --2- 172.21.15.5:0/3564711845 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2ddc06be30 0x7f2ddc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:36.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.447+0000 7f2e00ded640 1 --2- 172.21.15.5:0/3564711845 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2ddc06be30 0x7f2ddc06e340 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2de4001cd0 tx=0x7f2de4000ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:36.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.451+0000 7f2df97fa640 1 -- 172.21.15.5:0/3564711845 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2dfc15a690 con 0x7f2dfc15a280 2024-05-20T18:34:36.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.671+0000 7f2e03051640 1 -- 172.21.15.5:0/3564711845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2dfc068a00 con 0x7f2dfc15a280 2024-05-20T18:34:36.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.672+0000 7f2df97fa640 1 -- 172.21.15.5:0/3564711845 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+5165 (secure 0 0 0) 0x7f2df4079960 con 0x7f2dfc15a280 2024-05-20T18:34:36.673 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:34:36.673 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":70,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/337248865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":337248865},{"type":"v1","addr":"172.21.15.63:6834","nonce":337248865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3705772553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3705772553},{"type":"v1","addr":"172.21.15.63:6835","nonce":3705772553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":7434,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/776188138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":776188138},{"type":"v1","addr":"172.21.15.5:6835","nonce":776188138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:33:12.457716+0000","modified":"2024-05-20T18:34:36.305947+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7953":{"gid":7953,"name":"c","rank":0,"incarnation":70,"state":"up:replay","state_seq":1,"addr":"172.21.15.5:6837/3880006778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3880006778},{"type":"v1","addr":"172.21.15.5:6837","nonce":3880006778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-05-20T18:34:36.673 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 70 2024-05-20T18:34:36.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.674+0000 7f2e03051640 1 -- 172.21.15.5:0/3564711845 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2ddc06be30 msgr2=0x7f2ddc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:36.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.674+0000 7f2e03051640 1 --2- 172.21.15.5:0/3564711845 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2ddc06be30 0x7f2ddc06e340 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2de4001cd0 tx=0x7f2de4000ed0 comp rx=0 tx=0).stop 2024-05-20T18:34:36.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.674+0000 7f2e03051640 1 -- 172.21.15.5:0/3564711845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2dfc15a280 msgr2=0x7f2dfc0b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:36.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.674+0000 7f2e03051640 1 --2- 172.21.15.5:0/3564711845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2dfc15a280 0x7f2dfc0b4c10 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f2df402ae60 tx=0x7f2df4009750 comp rx=0 tx=0).stop 2024-05-20T18:34:36.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.674+0000 7f2e03051640 1 -- 172.21.15.5:0/3564711845 shutdown_connections 2024-05-20T18:34:36.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.674+0000 7f2e03051640 1 --2- 172.21.15.5:0/3564711845 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2ddc06be30 0x7f2ddc06e340 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:36.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.674+0000 7f2e03051640 1 --2- 172.21.15.5:0/3564711845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2dfc15a280 0x7f2dfc0b4c10 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:36.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.674+0000 7f2e03051640 1 --2- 172.21.15.5:0/3564711845 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2dfc153590 0x7f2dfc159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:36.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.674+0000 7f2e03051640 1 --2- 172.21.15.5:0/3564711845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2de80051c0 0x7f2dfc0b6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:36.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.674+0000 7f2e03051640 1 -- 172.21.15.5:0/3564711845 >> 172.21.15.5:0/3564711845 conn(0x7f2dfc06d240 msgr2=0x7f2dfc15eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:36.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.674+0000 7f2e03051640 1 -- 172.21.15.5:0/3564711845 shutdown_connections 2024-05-20T18:34:36.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:36.674+0000 7f2e03051640 1 -- 172.21.15.5:0/3564711845 wait complete. 2024-05-20T18:34:37.090 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 70, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T18:33:12.457716+0000', 'modified': '2024-05-20T18:34:36.305947+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 94, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7953}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7953': {'gid': 7953, 'name': 'c', 'rank': 0, 'incarnation': 70, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.5:6837/3880006778', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6836', 'nonce': 3880006778}, {'type': 'v1', 'addr': '172.21.15.5:6837', 'nonce': 3880006778}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-20T18:34:37.090 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7953:up:replay 2024-05-20T18:34:38.092 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:34:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.220+0000 7faa88fe0640 1 Processor -- start 2024-05-20T18:34:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.220+0000 7faa88fe0640 1 -- start start 2024-05-20T18:34:38.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.222+0000 7faa88fe0640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7faa8406d3d0 0x7faa8406d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:38.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.222+0000 7faa88fe0640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7faa840b8190 0x7faa840b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:38.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.222+0000 7faa88fe0640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa8406dd20 con 0x7faa84162820 2024-05-20T18:34:38.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.222+0000 7faa88fe0640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7faa840d0630 con 0x7faa840b8190 2024-05-20T18:34:38.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.222+0000 7faa88fe0640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7faa840d39c0 con 0x7faa8406d3d0 2024-05-20T18:34:38.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.222+0000 7faa82d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7faa840b8190 0x7faa840b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:38.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.222+0000 7faa8259c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7faa84162820 0x7faa840b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36476/0 (socket says 172.21.15.5:36476) 2024-05-20T18:34:38.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.222+0000 7faa8259c640 1 -- 172.21.15.5:0/516689764 learned_addr learned my addr 172.21.15.5:0/516689764 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:34:38.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.222+0000 7faa81d9b640 1 --2- 172.21.15.5:0/516689764 >> v2:172.21.15.63:3300/0 conn(0x7faa8406d3d0 0x7faa8406d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:38.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.222+0000 7faa82d9d640 1 -- 172.21.15.5:0/516689764 >> v2:172.21.15.63:3300/0 conn(0x7faa8406d3d0 msgr2=0x7faa8406d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:38.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.222+0000 7faa82d9d640 1 --2- 172.21.15.5:0/516689764 >> v2:172.21.15.63:3300/0 conn(0x7faa8406d3d0 0x7faa8406d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:38.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.222+0000 7faa82d9d640 1 -- 172.21.15.5:0/516689764 >> v1:172.21.15.5:6789/0 conn(0x7faa84162820 legacy=0x7faa840b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:38.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.222+0000 7faa82d9d640 1 -- 172.21.15.5:0/516689764 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa840d3ca0 con 0x7faa840b8190 2024-05-20T18:34:38.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.223+0000 7faa82d9d640 1 --2- 172.21.15.5:0/516689764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa840b8190 0x7faa840b85a0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7faa840d1230 tx=0x7faa7c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab2556c294335877 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:38.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.223+0000 7faa8159a640 1 -- 172.21.15.5:0/516689764 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7c007740 con 0x7faa840b8190 2024-05-20T18:34:38.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.223+0000 7faa8159a640 1 -- 172.21.15.5:0/516689764 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa7c0078e0 con 0x7faa840b8190 2024-05-20T18:34:38.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.223+0000 7faa8159a640 1 -- 172.21.15.5:0/516689764 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7c007cb0 con 0x7faa840b8190 2024-05-20T18:34:38.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.224+0000 7faa88fe0640 1 -- 172.21.15.5:0/516689764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa840b8190 msgr2=0x7faa840b85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:38.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.224+0000 7faa88fe0640 1 --2- 172.21.15.5:0/516689764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa840b8190 0x7faa840b85a0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7faa840d1230 tx=0x7faa7c02a680 comp rx=0 tx=0).stop 2024-05-20T18:34:38.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.224+0000 7faa88fe0640 1 -- 172.21.15.5:0/516689764 shutdown_connections 2024-05-20T18:34:38.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.224+0000 7faa88fe0640 1 --2- 172.21.15.5:0/516689764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa840b8190 0x7faa840b85a0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:38.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.224+0000 7faa88fe0640 1 --2- 172.21.15.5:0/516689764 >> v2:172.21.15.63:3300/0 conn(0x7faa8406d3d0 0x7faa8406d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:38.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.224+0000 7faa88fe0640 1 -- 172.21.15.5:0/516689764 >> 172.21.15.5:0/516689764 conn(0x7faa840d4bd0 msgr2=0x7faa840d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:38.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.224+0000 7faa88fe0640 1 -- 172.21.15.5:0/516689764 shutdown_connections 2024-05-20T18:34:38.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.224+0000 7faa88fe0640 1 -- 172.21.15.5:0/516689764 wait complete. 2024-05-20T18:34:38.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.226+0000 7faa88fe0640 1 Processor -- start 2024-05-20T18:34:38.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.226+0000 7faa88fe0640 1 -- start start 2024-05-20T18:34:38.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.227+0000 7faa88fe0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa8406d3d0 0x7faa840c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.227+0000 7faa88fe0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa840b8190 0x7faa841f8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.227+0000 7faa88fe0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa84162820 0x7faa841f7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.227+0000 7faa88fe0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faa840d0300 con 0x7faa840b8190 2024-05-20T18:34:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.227+0000 7faa88fe0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faa840d4260 con 0x7faa84162820 2024-05-20T18:34:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.228+0000 7faa88fe0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faa840b4050 con 0x7faa8406d3d0 2024-05-20T18:34:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.228+0000 7faa81d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa840b8190 0x7faa841f8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.228+0000 7faa81d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa840b8190 0x7faa841f8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38462/0 (socket says 172.21.15.5:38462) 2024-05-20T18:34:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.228+0000 7faa81d9b640 1 -- 172.21.15.5:0/1769674864 learned_addr learned my addr 172.21.15.5:0/1769674864 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.228+0000 7faa82d9d640 1 --2- 172.21.15.5:0/1769674864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa84162820 0x7faa841f7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.228+0000 7faa8259c640 1 --2- 172.21.15.5:0/1769674864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa8406d3d0 0x7faa840c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.228+0000 7faa81d9b640 1 -- 172.21.15.5:0/1769674864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa8406d3d0 msgr2=0x7faa840c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.228+0000 7faa81d9b640 1 --2- 172.21.15.5:0/1769674864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa8406d3d0 0x7faa840c8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.228+0000 7faa81d9b640 1 -- 172.21.15.5:0/1769674864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa84162820 msgr2=0x7faa841f7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.228+0000 7faa81d9b640 1 --2- 172.21.15.5:0/1769674864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa84162820 0x7faa841f7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.228+0000 7faa81d9b640 1 -- 172.21.15.5:0/1769674864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa840c8c40 con 0x7faa840b8190 2024-05-20T18:34:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.228+0000 7faa81d9b640 1 --2- 172.21.15.5:0/1769674864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa840b8190 0x7faa841f8a90 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7faa7800c7f0 tx=0x7faa7800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.228+0000 7faa72ffd640 1 -- 172.21.15.5:0/1769674864 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa78015070 con 0x7faa840b8190 2024-05-20T18:34:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.228+0000 7faa72ffd640 1 -- 172.21.15.5:0/1769674864 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa78004220 con 0x7faa840b8190 2024-05-20T18:34:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.228+0000 7faa88fe0640 1 -- 172.21.15.5:0/1769674864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa840b9af0 con 0x7faa840b8190 2024-05-20T18:34:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.228+0000 7faa72ffd640 1 -- 172.21.15.5:0/1769674864 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa78007620 con 0x7faa840b8190 2024-05-20T18:34:38.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.229+0000 7faa88fe0640 1 -- 172.21.15.5:0/1769674864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa840ba0b0 con 0x7faa840b8190 2024-05-20T18:34:38.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.230+0000 7faa88fe0640 1 -- 172.21.15.5:0/1769674864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa84075b00 con 0x7faa840b8190 2024-05-20T18:34:38.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.230+0000 7faa72ffd640 1 -- 172.21.15.5:0/1769674864 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faa780077c0 con 0x7faa840b8190 2024-05-20T18:34:38.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.231+0000 7faa72ffd640 1 --2- 172.21.15.5:0/1769674864 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faa6406be60 0x7faa6406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:38.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.231+0000 7faa72ffd640 1 -- 172.21.15.5:0/1769674864 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7faa7808ad50 con 0x7faa840b8190 2024-05-20T18:34:38.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.231+0000 7faa8259c640 1 --2- 172.21.15.5:0/1769674864 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faa6406be60 0x7faa6406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:38.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.231+0000 7faa8259c640 1 --2- 172.21.15.5:0/1769674864 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faa6406be60 0x7faa6406e370 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7faa74007fd0 tx=0x7faa74007e80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:38.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.235+0000 7faa72ffd640 1 -- 172.21.15.5:0/1769674864 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7faa78058e90 con 0x7faa840b8190 2024-05-20T18:34:38.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.461+0000 7faa88fe0640 1 -- 172.21.15.5:0/1769674864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa8408dea0 con 0x7faa840b8190 2024-05-20T18:34:38.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.462+0000 7faa72ffd640 1 -- 172.21.15.5:0/1769674864 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5165 (secure 0 0 0) 0x7faa7805d190 con 0x7faa840b8190 2024-05-20T18:34:38.463 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:34:38.463 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":72,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/337248865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":337248865},{"type":"v1","addr":"172.21.15.63:6834","nonce":337248865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3705772553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3705772553},{"type":"v1","addr":"172.21.15.63:6835","nonce":3705772553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":7434,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/776188138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":776188138},{"type":"v1","addr":"172.21.15.5:6835","nonce":776188138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:33:12.457716+0000","modified":"2024-05-20T18:34:37.314495+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7953":{"gid":7953,"name":"c","rank":0,"incarnation":70,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.5:6837/3880006778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3880006778},{"type":"v1","addr":"172.21.15.5:6837","nonce":3880006778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-05-20T18:34:38.463 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 72 2024-05-20T18:34:38.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.464+0000 7faa88fe0640 1 -- 172.21.15.5:0/1769674864 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faa6406be60 msgr2=0x7faa6406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:38.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.464+0000 7faa88fe0640 1 --2- 172.21.15.5:0/1769674864 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faa6406be60 0x7faa6406e370 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7faa74007fd0 tx=0x7faa74007e80 comp rx=0 tx=0).stop 2024-05-20T18:34:38.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.464+0000 7faa88fe0640 1 -- 172.21.15.5:0/1769674864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa840b8190 msgr2=0x7faa841f8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:38.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.464+0000 7faa88fe0640 1 --2- 172.21.15.5:0/1769674864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa840b8190 0x7faa841f8a90 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7faa7800c7f0 tx=0x7faa7800ccb0 comp rx=0 tx=0).stop 2024-05-20T18:34:38.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.464+0000 7faa88fe0640 1 -- 172.21.15.5:0/1769674864 shutdown_connections 2024-05-20T18:34:38.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.464+0000 7faa88fe0640 1 --2- 172.21.15.5:0/1769674864 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faa6406be60 0x7faa6406e370 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:38.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.464+0000 7faa88fe0640 1 --2- 172.21.15.5:0/1769674864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa84162820 0x7faa841f7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:38.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.464+0000 7faa88fe0640 1 --2- 172.21.15.5:0/1769674864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa840b8190 0x7faa841f8a90 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:38.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.464+0000 7faa88fe0640 1 --2- 172.21.15.5:0/1769674864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa8406d3d0 0x7faa840c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:38.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.464+0000 7faa88fe0640 1 -- 172.21.15.5:0/1769674864 >> 172.21.15.5:0/1769674864 conn(0x7faa840d4bd0 msgr2=0x7faa840d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:38.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.464+0000 7faa88fe0640 1 -- 172.21.15.5:0/1769674864 shutdown_connections 2024-05-20T18:34:38.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:38.464+0000 7faa88fe0640 1 -- 172.21.15.5:0/1769674864 wait complete. 2024-05-20T18:34:38.879 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T18:33:12.457716+0000', 'modified': '2024-05-20T18:34:37.314495+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 94, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7953}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7953': {'gid': 7953, 'name': 'c', 'rank': 0, 'incarnation': 70, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.5:6837/3880006778', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6836', 'nonce': 3880006778}, {'type': 'v1', 'addr': '172.21.15.5:6837', 'nonce': 3880006778}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-20T18:34:38.879 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7953:up:rejoin 2024-05-20T18:34:39.881 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:34:40.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.010+0000 7f2be9e34640 1 Processor -- start 2024-05-20T18:34:40.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.010+0000 7f2be9e34640 1 -- start start 2024-05-20T18:34:40.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.011+0000 7f2be9e34640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2be415f220 0x7f2be40d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.011+0000 7f2be9e34640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2be40cda20 con 0x7f2be40d0e00 2024-05-20T18:34:40.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.011+0000 7f2be9e34640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2be40cdd00 con 0x7f2be40cfae0 2024-05-20T18:34:40.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.011+0000 7f2be9e34640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2be40cdfe0 con 0x7f2be415f220 2024-05-20T18:34:40.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.012+0000 7f2be2ffd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2be40d0e00 0x7f2be40cf3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36488/0 (socket says 172.21.15.5:36488) 2024-05-20T18:34:40.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.012+0000 7f2be2ffd640 1 -- 172.21.15.5:0/278404958 learned_addr learned my addr 172.21.15.5:0/278404958 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:34:40.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.012+0000 7f2be37fe640 1 --2- 172.21.15.5:0/278404958 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2be415f220 0x7f2be40d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.012+0000 7f2be27fc640 1 -- 172.21.15.5:0/278404958 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3709122281 0 0) 0x7f2be40cda20 con 0x7f2be40d0e00 2024-05-20T18:34:40.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.012+0000 7f2be27fc640 1 -- 172.21.15.5:0/278404958 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bcc003520 con 0x7f2be40d0e00 2024-05-20T18:34:40.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.012+0000 7f2be27fc640 1 -- 172.21.15.5:0/278404958 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 816765456 0 0) 0x7f2be40cdd00 con 0x7f2be40cfae0 2024-05-20T18:34:40.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.012+0000 7f2be27fc640 1 -- 172.21.15.5:0/278404958 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2be40cda20 con 0x7f2be40cfae0 2024-05-20T18:34:40.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.012+0000 7f2be27fc640 1 -- 172.21.15.5:0/278404958 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2298559335 0 0) 0x7f2bcc003520 con 0x7f2be40d0e00 2024-05-20T18:34:40.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.012+0000 7f2be37fe640 1 -- 172.21.15.5:0/278404958 >> v1:172.21.15.63:6789/0 conn(0x7f2be40cfae0 legacy=0x7f2be40ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:40.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.012+0000 7f2be37fe640 1 -- 172.21.15.5:0/278404958 >> v1:172.21.15.5:6789/0 conn(0x7f2be40d0e00 legacy=0x7f2be40cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:40.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.012+0000 7f2be37fe640 1 -- 172.21.15.5:0/278404958 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be40ce2c0 con 0x7f2be415f220 2024-05-20T18:34:40.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.013+0000 7f2be37fe640 1 --2- 172.21.15.5:0/278404958 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2be415f220 0x7f2be40d08c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40097d0 tx=0x7f2bd402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a80e865e82f7ec64 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.013+0000 7f2be27fc640 1 -- 172.21.15.5:0/278404958 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4037070 con 0x7f2be415f220 2024-05-20T18:34:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.013+0000 7f2be27fc640 1 -- 172.21.15.5:0/278404958 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bd402ab90 con 0x7f2be415f220 2024-05-20T18:34:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.013+0000 7f2be27fc640 1 -- 172.21.15.5:0/278404958 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8001670 con 0x7f2be415f220 2024-05-20T18:34:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.014+0000 7f2be9e34640 1 -- 172.21.15.5:0/278404958 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2be415f220 msgr2=0x7f2be40d08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.014+0000 7f2be9e34640 1 --2- 172.21.15.5:0/278404958 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2be415f220 0x7f2be40d08c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40097d0 tx=0x7f2bd402a100 comp rx=0 tx=0).stop 2024-05-20T18:34:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.014+0000 7f2be9e34640 1 -- 172.21.15.5:0/278404958 shutdown_connections 2024-05-20T18:34:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.014+0000 7f2be9e34640 1 --2- 172.21.15.5:0/278404958 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2be415f220 0x7f2be40d08c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.014+0000 7f2be9e34640 1 -- 172.21.15.5:0/278404958 >> 172.21.15.5:0/278404958 conn(0x7f2be40b44d0 msgr2=0x7f2be40b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.014+0000 7f2be9e34640 1 -- 172.21.15.5:0/278404958 shutdown_connections 2024-05-20T18:34:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.014+0000 7f2be9e34640 1 -- 172.21.15.5:0/278404958 wait complete. 2024-05-20T18:34:40.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.016+0000 7f2be9e34640 1 Processor -- start 2024-05-20T18:34:40.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.016+0000 7f2be9e34640 1 -- start start 2024-05-20T18:34:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.018+0000 7f2be9e34640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2be40d0e00 0x7f2be40cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.018+0000 7f2be9e34640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2be415f220 0x7f2be41f4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.018+0000 7f2be9e34640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2be40cbd90 0x7f2be41f8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.018+0000 7f2be9e34640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2be415f7b0 con 0x7f2be415f220 2024-05-20T18:34:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.018+0000 7f2be9e34640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2be415f630 con 0x7f2be40d0e00 2024-05-20T18:34:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.018+0000 7f2be9e34640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2be40ceef0 con 0x7f2be40cbd90 2024-05-20T18:34:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.018+0000 7f2be2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2be415f220 0x7f2be41f4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.018+0000 7f2be2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2be415f220 0x7f2be41f4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38466/0 (socket says 172.21.15.5:38466) 2024-05-20T18:34:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.018+0000 7f2be2ffd640 1 -- 172.21.15.5:0/2507005709 learned_addr learned my addr 172.21.15.5:0/2507005709 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.018+0000 7f2be37fe640 1 --2- 172.21.15.5:0/2507005709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2be40d0e00 0x7f2be40cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.018+0000 7f2be3fff640 1 --2- 172.21.15.5:0/2507005709 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2be40cbd90 0x7f2be41f8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.018+0000 7f2be2ffd640 1 -- 172.21.15.5:0/2507005709 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2be40cbd90 msgr2=0x7f2be41f8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.018+0000 7f2be2ffd640 1 --2- 172.21.15.5:0/2507005709 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2be40cbd90 0x7f2be41f8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.018+0000 7f2be2ffd640 1 -- 172.21.15.5:0/2507005709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2be40d0e00 msgr2=0x7f2be40cb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.018+0000 7f2be2ffd640 1 --2- 172.21.15.5:0/2507005709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2be40d0e00 0x7f2be40cb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.018+0000 7f2be2ffd640 1 -- 172.21.15.5:0/2507005709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be4160ae0 con 0x7f2be415f220 2024-05-20T18:34:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.018+0000 7f2be2ffd640 1 --2- 172.21.15.5:0/2507005709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2be415f220 0x7f2be41f4e90 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800c5d0 tx=0x7f2bd800ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.018+0000 7f2bcbfff640 1 -- 172.21.15.5:0/2507005709 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8017070 con 0x7f2be415f220 2024-05-20T18:34:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.019+0000 7f2bcbfff640 1 -- 172.21.15.5:0/2507005709 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bd8005d00 con 0x7f2be415f220 2024-05-20T18:34:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.019+0000 7f2be9e34640 1 -- 172.21.15.5:0/2507005709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be4160d80 con 0x7f2be415f220 2024-05-20T18:34:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.019+0000 7f2bcbfff640 1 -- 172.21.15.5:0/2507005709 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd80072b0 con 0x7f2be415f220 2024-05-20T18:34:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.019+0000 7f2be9e34640 1 -- 172.21.15.5:0/2507005709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be4161340 con 0x7f2be415f220 2024-05-20T18:34:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.020+0000 7f2be9e34640 1 -- 172.21.15.5:0/2507005709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2be4075b00 con 0x7f2be415f220 2024-05-20T18:34:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.020+0000 7f2bcbfff640 1 -- 172.21.15.5:0/2507005709 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2bd801a070 con 0x7f2be415f220 2024-05-20T18:34:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.021+0000 7f2bcbfff640 1 --2- 172.21.15.5:0/2507005709 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2bd006be30 0x7f2bd006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.021+0000 7f2bcbfff640 1 -- 172.21.15.5:0/2507005709 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7f2bd808a5b0 con 0x7f2be415f220 2024-05-20T18:34:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.021+0000 7f2be37fe640 1 --2- 172.21.15.5:0/2507005709 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2bd006be30 0x7f2bd006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.021+0000 7f2be37fe640 1 --2- 172.21.15.5:0/2507005709 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2bd006be30 0x7f2bd006e340 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4034000 tx=0x7f2bd402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.025+0000 7f2bcbfff640 1 -- 172.21.15.5:0/2507005709 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2bd8058cb0 con 0x7f2be415f220 2024-05-20T18:34:40.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.249+0000 7f2be9e34640 1 -- 172.21.15.5:0/2507005709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2be4162c60 con 0x7f2be415f220 2024-05-20T18:34:40.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.249+0000 7f2bcbfff640 1 -- 172.21.15.5:0/2507005709 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5172 (secure 0 0 0) 0x7f2bd80585b0 con 0x7f2be415f220 2024-05-20T18:34:40.251 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:34:40.251 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/337248865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":337248865},{"type":"v1","addr":"172.21.15.63:6834","nonce":337248865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3705772553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3705772553},{"type":"v1","addr":"172.21.15.63:6835","nonce":3705772553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":7434,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/776188138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":776188138},{"type":"v1","addr":"172.21.15.5:6835","nonce":776188138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:33:12.457716+0000","modified":"2024-05-20T18:34:39.318498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7953":{"gid":7953,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6837/3880006778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3880006778},{"type":"v1","addr":"172.21.15.5:6837","nonce":3880006778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7953,"qdb_cluster":[7953]},"id":6}]} 2024-05-20T18:34:40.251 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 73 2024-05-20T18:34:40.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.251+0000 7f2be9e34640 1 -- 172.21.15.5:0/2507005709 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2bd006be30 msgr2=0x7f2bd006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:40.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.251+0000 7f2be9e34640 1 --2- 172.21.15.5:0/2507005709 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2bd006be30 0x7f2bd006e340 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4034000 tx=0x7f2bd402e000 comp rx=0 tx=0).stop 2024-05-20T18:34:40.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.251+0000 7f2be9e34640 1 -- 172.21.15.5:0/2507005709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2be415f220 msgr2=0x7f2be41f4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:40.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.251+0000 7f2be9e34640 1 --2- 172.21.15.5:0/2507005709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2be415f220 0x7f2be41f4e90 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800c5d0 tx=0x7f2bd800ca90 comp rx=0 tx=0).stop 2024-05-20T18:34:40.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.251+0000 7f2be9e34640 1 -- 172.21.15.5:0/2507005709 shutdown_connections 2024-05-20T18:34:40.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.251+0000 7f2be9e34640 1 --2- 172.21.15.5:0/2507005709 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2bd006be30 0x7f2bd006e340 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.251+0000 7f2be9e34640 1 --2- 172.21.15.5:0/2507005709 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2be40cbd90 0x7f2be41f8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.251+0000 7f2be9e34640 1 --2- 172.21.15.5:0/2507005709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2be415f220 0x7f2be41f4e90 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.251+0000 7f2be9e34640 1 --2- 172.21.15.5:0/2507005709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2be40d0e00 0x7f2be40cb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.251+0000 7f2be9e34640 1 -- 172.21.15.5:0/2507005709 >> 172.21.15.5:0/2507005709 conn(0x7f2be40b44d0 msgr2=0x7f2be415e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:40.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.252+0000 7f2be9e34640 1 -- 172.21.15.5:0/2507005709 shutdown_connections 2024-05-20T18:34:40.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.252+0000 7f2be9e34640 1 -- 172.21.15.5:0/2507005709 wait complete. 2024-05-20T18:34:40.667 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T18:33:12.457716+0000', 'modified': '2024-05-20T18:34:39.318498+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 94, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7953}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7953': {'gid': 7953, 'name': 'c', 'rank': 0, 'incarnation': 70, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.5:6837/3880006778', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6836', 'nonce': 3880006778}, {'type': 'v1', 'addr': '172.21.15.5:6837', 'nonce': 3880006778}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7953, 'qdb_cluster': [7953]} 2024-05-20T18:34:40.668 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-20T18:34:40.668 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-20T18:34:40.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.788+0000 7f7f4c0a2640 1 Processor -- start 2024-05-20T18:34:40.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.788+0000 7f7f4c0a2640 1 -- start start 2024-05-20T18:34:40.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.789+0000 7f7f4c0a2640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7f4415f220 0x7f7f440cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.789+0000 7f7f4c0a2640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f440cf910 0x7f7f440d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.789+0000 7f7f4c0a2640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f440cfd20 con 0x7f7f440cd680 2024-05-20T18:34:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.789+0000 7f7f4c0a2640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f7f440cda90 con 0x7f7f4415f220 2024-05-20T18:34:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.789+0000 7f7f4c0a2640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7f4406d0f0 con 0x7f7f440cf910 2024-05-20T18:34:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.790+0000 7f7f4963d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7f4415f220 0x7f7f440cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.790+0000 7f7f49e3e640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7f440cd680 0x7f7f4406c6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36504/0 (socket says 172.21.15.5:36504) 2024-05-20T18:34:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.790+0000 7f7f4963d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7f4415f220 0x7f7f440cf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38480/0 (socket says 172.21.15.5:38480) 2024-05-20T18:34:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.790+0000 7f7f49e3e640 1 -- 172.21.15.5:0/136796544 learned_addr learned my addr 172.21.15.5:0/136796544 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:34:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.790+0000 7f7f4a63f640 1 --2- 172.21.15.5:0/136796544 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f440cf910 0x7f7f440d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.790+0000 7f7f4963d640 1 -- 172.21.15.5:0/136796544 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f440cf910 msgr2=0x7f7f440d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.790+0000 7f7f4963d640 1 --2- 172.21.15.5:0/136796544 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f440cf910 0x7f7f440d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.790+0000 7f7f4963d640 1 -- 172.21.15.5:0/136796544 >> v1:172.21.15.5:6789/0 conn(0x7f7f440cd680 legacy=0x7f7f4406c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.790+0000 7f7f48e3c640 1 -- 172.21.15.5:0/136796544 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 791798979 0 0) 0x7f7f40001110 con 0x7f7f440cd680 2024-05-20T18:34:40.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.790+0000 7f7f4963d640 1 -- 172.21.15.5:0/136796544 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4406d3d0 con 0x7f7f4415f220 2024-05-20T18:34:40.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.790+0000 7f7f4a63f640 1 --2- 172.21.15.5:0/136796544 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f440cf910 0x7f7f440d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:34:40.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.791+0000 7f7f4963d640 1 --2- 172.21.15.5:0/136796544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f4415f220 0x7f7f440cf3d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f7f380097a0 tx=0x7f7f3802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45838cb8693f7a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:40.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.791+0000 7f7f48e3c640 1 -- 172.21.15.5:0/136796544 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38037070 con 0x7f7f4415f220 2024-05-20T18:34:40.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.791+0000 7f7f48e3c640 1 -- 172.21.15.5:0/136796544 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f3802ae50 con 0x7f7f4415f220 2024-05-20T18:34:40.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.791+0000 7f7f48e3c640 1 -- 172.21.15.5:0/136796544 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38032660 con 0x7f7f4415f220 2024-05-20T18:34:40.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.791+0000 7f7f4c0a2640 1 -- 172.21.15.5:0/136796544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f4415f220 msgr2=0x7f7f440cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:40.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.791+0000 7f7f4c0a2640 1 --2- 172.21.15.5:0/136796544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f4415f220 0x7f7f440cf3d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f7f380097a0 tx=0x7f7f3802a100 comp rx=0 tx=0).stop 2024-05-20T18:34:40.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.792+0000 7f7f4c0a2640 1 -- 172.21.15.5:0/136796544 shutdown_connections 2024-05-20T18:34:40.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.792+0000 7f7f4c0a2640 1 --2- 172.21.15.5:0/136796544 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f440cf910 0x7f7f440d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.792+0000 7f7f4c0a2640 1 --2- 172.21.15.5:0/136796544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f4415f220 0x7f7f440cf3d0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.792+0000 7f7f4c0a2640 1 -- 172.21.15.5:0/136796544 >> 172.21.15.5:0/136796544 conn(0x7f7f440b44d0 msgr2=0x7f7f440b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:40.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.792+0000 7f7f4c0a2640 1 -- 172.21.15.5:0/136796544 shutdown_connections 2024-05-20T18:34:40.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.792+0000 7f7f4c0a2640 1 -- 172.21.15.5:0/136796544 wait complete. 2024-05-20T18:34:40.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.793+0000 7f7f4c0a2640 1 Processor -- start 2024-05-20T18:34:40.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.793+0000 7f7f4c0a2640 1 -- start start 2024-05-20T18:34:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.795+0000 7f7f4c0a2640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f440cd680 0x7f7f44151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.795+0000 7f7f4c0a2640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f440cf910 0x7f7f440b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.795+0000 7f7f4c0a2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f4415f220 0x7f7f440b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.795+0000 7f7f4c0a2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4406d990 con 0x7f7f4415f220 2024-05-20T18:34:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.795+0000 7f7f4c0a2640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7f440d0b40 con 0x7f7f440cf910 2024-05-20T18:34:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.795+0000 7f7f4c0a2640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7f4406cdd0 con 0x7f7f440cd680 2024-05-20T18:34:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.795+0000 7f7f4a63f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f4415f220 0x7f7f440b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.795+0000 7f7f4a63f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f4415f220 0x7f7f440b30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38496/0 (socket says 172.21.15.5:38496) 2024-05-20T18:34:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.795+0000 7f7f4a63f640 1 -- 172.21.15.5:0/3196629315 learned_addr learned my addr 172.21.15.5:0/3196629315 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.795+0000 7f7f49e3e640 1 --2- 172.21.15.5:0/3196629315 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f440cd680 0x7f7f44151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.795+0000 7f7f4963d640 1 --2- 172.21.15.5:0/3196629315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f440cf910 0x7f7f440b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.795+0000 7f7f4a63f640 1 -- 172.21.15.5:0/3196629315 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f440cd680 msgr2=0x7f7f44151c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.795+0000 7f7f4a63f640 1 --2- 172.21.15.5:0/3196629315 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f440cd680 0x7f7f44151c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.795+0000 7f7f4a63f640 1 -- 172.21.15.5:0/3196629315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f440cf910 msgr2=0x7f7f440b77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:40.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.795+0000 7f7f4a63f640 1 --2- 172.21.15.5:0/3196629315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f440cf910 0x7f7f440b77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.795+0000 7f7f4a63f640 1 -- 172.21.15.5:0/3196629315 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f441f9e00 con 0x7f7f4415f220 2024-05-20T18:34:40.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.795+0000 7f7f4a63f640 1 --2- 172.21.15.5:0/3196629315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f4415f220 0x7f7f440b30f0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00cd80 tx=0x7f7f3c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:40.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.795+0000 7f7f367fc640 1 -- 172.21.15.5:0/3196629315 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c00f680 con 0x7f7f4415f220 2024-05-20T18:34:40.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.796+0000 7f7f367fc640 1 -- 172.21.15.5:0/3196629315 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f3c00f840 con 0x7f7f4415f220 2024-05-20T18:34:40.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.796+0000 7f7f367fc640 1 -- 172.21.15.5:0/3196629315 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c00fc30 con 0x7f7f4415f220 2024-05-20T18:34:40.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.796+0000 7f7f4c0a2640 1 -- 172.21.15.5:0/3196629315 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f441f9040 con 0x7f7f4415f220 2024-05-20T18:34:40.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.796+0000 7f7f4c0a2640 1 -- 172.21.15.5:0/3196629315 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f441f9600 con 0x7f7f4415f220 2024-05-20T18:34:40.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.798+0000 7f7f367fc640 1 -- 172.21.15.5:0/3196629315 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7f3c00e3e0 con 0x7f7f4415f220 2024-05-20T18:34:40.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.798+0000 7f7f367fc640 1 --2- 172.21.15.5:0/3196629315 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7f2806be30 0x7f7f2806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.798+0000 7f7f367fc640 1 -- 172.21.15.5:0/3196629315 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7f7f3c08be90 con 0x7f7f4415f220 2024-05-20T18:34:40.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.798+0000 7f7f49e3e640 1 --2- 172.21.15.5:0/3196629315 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7f2806be30 0x7f7f2806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.799+0000 7f7f49e3e640 1 --2- 172.21.15.5:0/3196629315 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7f2806be30 0x7f7f2806e340 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7f40000f60 tx=0x7f7f40007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:40.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.799+0000 7f7f53028740 1 Processor -- start 2024-05-20T18:34:40.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.800+0000 7f7f53028740 1 -- start start 2024-05-20T18:34:40.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.801+0000 7f7f53028740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647400e1270 0x5647404ac700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.801+0000 7f7f53028740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647400b83b0 0x5647404a90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.801+0000 7f7f53028740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647400c0590 0x564740482360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.801+0000 7f7f53028740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5647403e54d0 con 0x5647400c0590 2024-05-20T18:34:40.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.801+0000 7f7f53028740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56474038d3e0 con 0x5647400e1270 2024-05-20T18:34:40.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.801+0000 7f7f53028740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5647403e1490 con 0x5647400b83b0 2024-05-20T18:34:40.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.801+0000 7f7f4a63f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647400c0590 0x564740482360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.801+0000 7f7f4a63f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647400c0590 0x564740482360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38508/0 (socket says 172.21.15.5:38508) 2024-05-20T18:34:40.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.801+0000 7f7f4a63f640 1 -- 172.21.15.5:0/648535704 learned_addr learned my addr 172.21.15.5:0/648535704 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:40.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.801+0000 7f7f4963d640 1 --2- 172.21.15.5:0/648535704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647400b83b0 0x5647404a90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.801+0000 7f7f49e3e640 1 --2- 172.21.15.5:0/648535704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647400e1270 0x5647404ac700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.801+0000 7f7f4a63f640 1 -- 172.21.15.5:0/648535704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647400b83b0 msgr2=0x5647404a90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:40.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.801+0000 7f7f4a63f640 1 --2- 172.21.15.5:0/648535704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647400b83b0 0x5647404a90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.801+0000 7f7f4a63f640 1 -- 172.21.15.5:0/648535704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647400e1270 msgr2=0x5647404ac700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:40.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.801+0000 7f7f4a63f640 1 --2- 172.21.15.5:0/648535704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647400e1270 0x5647404ac700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.801+0000 7f7f4a63f640 1 -- 172.21.15.5:0/648535704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647400f92d0 con 0x5647400c0590 2024-05-20T18:34:40.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.801+0000 7f7f49e3e640 1 --2- 172.21.15.5:0/648535704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647400e1270 0x5647404ac700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:34:40.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.801+0000 7f7f4a63f640 1 --2- 172.21.15.5:0/648535704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647400c0590 0x564740482360 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c062530 tx=0x7f7f3c059e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221258cebcbffd70 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:40.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.802+0000 7f7f277fe640 1 -- 172.21.15.5:0/648535704 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c088070 con 0x5647400c0590 2024-05-20T18:34:40.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.802+0000 7f7f277fe640 1 -- 172.21.15.5:0/648535704 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f3c062a60 con 0x5647400c0590 2024-05-20T18:34:40.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.802+0000 7f7f277fe640 1 -- 172.21.15.5:0/648535704 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c01d030 con 0x5647400c0590 2024-05-20T18:34:40.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.802+0000 7f7f53028740 1 -- 172.21.15.5:0/648535704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647400c0590 msgr2=0x564740482360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:40.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.802+0000 7f7f53028740 1 --2- 172.21.15.5:0/648535704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647400c0590 0x564740482360 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c062530 tx=0x7f7f3c059e80 comp rx=0 tx=0).stop 2024-05-20T18:34:40.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.802+0000 7f7f53028740 1 -- 172.21.15.5:0/648535704 shutdown_connections 2024-05-20T18:34:40.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.802+0000 7f7f53028740 1 --2- 172.21.15.5:0/648535704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647400c0590 0x564740482360 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.803+0000 7f7f53028740 1 --2- 172.21.15.5:0/648535704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647400b83b0 0x5647404a90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.803+0000 7f7f53028740 1 --2- 172.21.15.5:0/648535704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647400e1270 0x5647404ac700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.803+0000 7f7f53028740 1 -- 172.21.15.5:0/648535704 >> 172.21.15.5:0/648535704 conn(0x5647400d53c0 msgr2=0x564740481af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:40.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.803+0000 7f7f53028740 1 -- 172.21.15.5:0/648535704 shutdown_connections 2024-05-20T18:34:40.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.803+0000 7f7f53028740 1 -- 172.21.15.5:0/648535704 wait complete. 2024-05-20T18:34:40.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.803+0000 7f7f53028740 1 -- start start 2024-05-20T18:34:40.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.804+0000 7f7f53028740 1 Processor -- start 2024-05-20T18:34:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.805+0000 7f7f53028740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647400b83b0 0x5647404a3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.805+0000 7f7f53028740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647400c0590 0x5647404a3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.805+0000 7f7f53028740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647400e1270 0x5647404a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.805+0000 7f7f53028740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5647403a08e0 con 0x5647400c0590 2024-05-20T18:34:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.805+0000 7f7f53028740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5647404afc30 con 0x5647400b83b0 2024-05-20T18:34:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.805+0000 7f7f53028740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5647404ad3a0 con 0x5647400e1270 2024-05-20T18:34:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.805+0000 7f7f53028740 10 client.0 _ll_register_callbacks cb 0x56474036d9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:34:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.805+0000 7f7f4963d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647400c0590 0x5647404a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.805+0000 7f7f4963d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647400c0590 0x5647404a3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38524/0 (socket says 172.21.15.5:38524) 2024-05-20T18:34:40.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.805+0000 7f7f4963d640 1 -- 172.21.15.5:0/2286408783 learned_addr learned my addr 172.21.15.5:0/2286408783 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:40.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.805+0000 7f7f49e3e640 1 --2- 172.21.15.5:0/2286408783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647400b83b0 0x5647404a3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.805+0000 7f7f4a63f640 1 --2- 172.21.15.5:0/2286408783 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647400e1270 0x5647404a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.806+0000 7f7f4963d640 1 -- 172.21.15.5:0/2286408783 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647400e1270 msgr2=0x5647404a3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:40.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.806+0000 7f7f4963d640 1 --2- 172.21.15.5:0/2286408783 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647400e1270 0x5647404a3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.806+0000 7f7f4963d640 1 -- 172.21.15.5:0/2286408783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647400b83b0 msgr2=0x5647404a3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:40.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.806+0000 7f7f4963d640 1 --2- 172.21.15.5:0/2286408783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647400b83b0 0x5647404a3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.806+0000 7f7f4963d640 1 -- 172.21.15.5:0/2286408783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647401c36b0 con 0x5647400c0590 2024-05-20T18:34:40.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.806+0000 7f7f49e3e640 1 --2- 172.21.15.5:0/2286408783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647400b83b0 0x5647404a3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:34:40.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.806+0000 7f7f4a63f640 1 --2- 172.21.15.5:0/2286408783 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647400e1270 0x5647404a3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:34:40.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.806+0000 7f7f4963d640 1 --2- 172.21.15.5:0/2286408783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647400c0590 0x5647404a3790 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f7f380097a0 tx=0x7f7f38004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56af9b63eb48766c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:40.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.806+0000 7f7f24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:34:40.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.806+0000 7f7f24ff9640 1 -- 172.21.15.5:0/2286408783 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38042070 con 0x5647400c0590 2024-05-20T18:34:40.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.806+0000 7f7f24ff9640 1 -- 172.21.15.5:0/2286408783 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7f7f3803e460 con 0x5647400c0590 2024-05-20T18:34:40.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.806+0000 7f7f53028740 1 -- 172.21.15.5:0/2286408783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f3c062a60 con 0x5647400c0590 2024-05-20T18:34:40.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.806+0000 7f7f24ff9640 1 -- 172.21.15.5:0/2286408783 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f7f3804d050 con 0x5647400c0590 2024-05-20T18:34:40.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.806+0000 7f7f53028740 10 client.8034 fetch_fsmap learned FSMap version 73 2024-05-20T18:34:40.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.806+0000 7f7f53028740 1 -- 172.21.15.5:0/2286408783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x5647404b0440 con 0x5647400c0590 2024-05-20T18:34:40.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.806+0000 7f7f24ff9640 1 -- 172.21.15.5:0/2286408783 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f3c062a60 con 0x5647400c0590 2024-05-20T18:34:40.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.806+0000 7f7f24ff9640 1 -- 172.21.15.5:0/2286408783 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 73) ==== 3494+0+0 (secure 0 0 0) 0x7f7f38050020 con 0x5647400c0590 2024-05-20T18:34:40.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.807+0000 7f7f53028740 10 client.8034 fetch_fsmap finished waiting for FSMap version 73 2024-05-20T18:34:40.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.807+0000 7f7f53028740 10 client.8034 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-20T18:34:40.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.807+0000 7f7f53028740 20 client.8034 populate_metadata read hostname 'smithi005' 2024-05-20T18:34:40.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.807+0000 7f7f53028740 1 --2- 172.21.15.5:0/2286408783 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x56474009fe40 0x5647404e4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.807+0000 7f7f53028740 4 client.8034 mds_command: new command op to 7953 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:34:40.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.807+0000 7f7f53028740 1 -- 172.21.15.5:0/2286408783 --> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647403f2c00 con 0x56474009fe40 2024-05-20T18:34:40.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.807+0000 7f7f49e3e640 1 --2- 172.21.15.5:0/2286408783 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x56474009fe40 0x5647404e4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.807+0000 7f7f24ff9640 10 client.8034 ms_handle_connect on v2:172.21.15.5:6836/3880006778 2024-05-20T18:34:40.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.807+0000 7f7f49e3e640 1 --2- 172.21.15.5:0/2286408783 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x56474009fe40 0x5647404e4620 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62d4d7bcfbedc141 server_cookie=48eb69520af2b919 in_seq=0 out_seq=0 2024-05-20T18:34:40.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.808+0000 7f7f24ff9640 1 -- 172.21.15.5:0/2286408783 <== mds.0 v2:172.21.15.5:6836/3880006778 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5647403f2c00 con 0x56474009fe40 2024-05-20T18:34:40.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.808+0000 7f7f24ff9640 10 client.8034 handle_command_reply: tid=0 2024-05-20T18:34:40.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.808+0000 7f7f53028740 1 client.8034 shutdown 2024-05-20T18:34:40.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.808+0000 7f7f53028740 1 -- 172.21.15.5:0/2286408783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647400c0590 msgr2=0x5647404a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:40.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.808+0000 7f7f53028740 1 --2- 172.21.15.5:0/2286408783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647400c0590 0x5647404a3790 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f7f380097a0 tx=0x7f7f38004730 comp rx=0 tx=0).stop 2024-05-20T18:34:40.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.808+0000 7f7f53028740 1 -- 172.21.15.5:0/2286408783 shutdown_connections 2024-05-20T18:34:40.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.808+0000 7f7f53028740 1 --2- 172.21.15.5:0/2286408783 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x56474009fe40 0x5647404e4620 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.808+0000 7f7f53028740 1 --2- 172.21.15.5:0/2286408783 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647400e1270 0x5647404a3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.808+0000 7f7f53028740 1 --2- 172.21.15.5:0/2286408783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647400c0590 0x5647404a3790 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.808+0000 7f7f53028740 1 --2- 172.21.15.5:0/2286408783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647400b83b0 0x5647404a3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.808+0000 7f7f24ff9640 0 client.8034 ms_handle_reset on v2:172.21.15.5:6836/3880006778 2024-05-20T18:34:40.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.808+0000 7f7f53028740 1 -- 172.21.15.5:0/2286408783 >> 172.21.15.5:0/2286408783 conn(0x5647400d53c0 msgr2=0x564740481af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:40.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.808+0000 7f7f53028740 1 -- 172.21.15.5:0/2286408783 shutdown_connections 2024-05-20T18:34:40.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.808+0000 7f7f53028740 1 -- 172.21.15.5:0/2286408783 wait complete. 2024-05-20T18:34:40.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.808+0000 7f7f53028740 20 client.8034 trim_cache size 0 max 16384 2024-05-20T18:34:40.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.827+0000 7f7f53028740 1 Processor -- start 2024-05-20T18:34:40.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.827+0000 7f7f53028740 1 -- start start 2024-05-20T18:34:40.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.828+0000 7f7f53028740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647400b83b0 0x5647404a95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.828+0000 7f7f53028740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647400c0590 0x5647404ab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.828+0000 7f7f53028740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647400e1270 0x5647404abef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.828+0000 7f7f53028740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7f44201660 con 0x5647400c0590 2024-05-20T18:34:40.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.828+0000 7f7f53028740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7f440c7fe0 con 0x5647400e1270 2024-05-20T18:34:40.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.828+0000 7f7f53028740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7f440c8630 con 0x5647400b83b0 2024-05-20T18:34:40.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.829+0000 7f7f4963d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647400c0590 0x5647404ab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.829+0000 7f7f4963d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647400c0590 0x5647404ab9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38532/0 (socket says 172.21.15.5:38532) 2024-05-20T18:34:40.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.829+0000 7f7f4963d640 1 -- 172.21.15.5:0/3467406296 learned_addr learned my addr 172.21.15.5:0/3467406296 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:40.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.829+0000 7f7f49e3e640 1 --2- 172.21.15.5:0/3467406296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647400b83b0 0x5647404a95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.829+0000 7f7f4a63f640 1 --2- 172.21.15.5:0/3467406296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647400e1270 0x5647404abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.829+0000 7f7f4963d640 1 -- 172.21.15.5:0/3467406296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647400b83b0 msgr2=0x5647404a95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:40.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.829+0000 7f7f4963d640 1 --2- 172.21.15.5:0/3467406296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647400b83b0 0x5647404a95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.829+0000 7f7f4963d640 1 -- 172.21.15.5:0/3467406296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647400e1270 msgr2=0x5647404abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:40.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.829+0000 7f7f4963d640 1 --2- 172.21.15.5:0/3467406296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647400e1270 0x5647404abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.829+0000 7f7f4963d640 1 -- 172.21.15.5:0/3467406296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56474047b220 con 0x5647400c0590 2024-05-20T18:34:40.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.829+0000 7f7f49e3e640 1 --2- 172.21.15.5:0/3467406296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647400b83b0 0x5647404a95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:34:40.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.829+0000 7f7f4a63f640 1 --2- 172.21.15.5:0/3467406296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647400e1270 0x5647404abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:34:40.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.829+0000 7f7f4963d640 1 --2- 172.21.15.5:0/3467406296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647400c0590 0x5647404ab9b0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f7f38006b00 tx=0x7f7f3803b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99dac0996a006f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:40.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.829+0000 7f7f26ffd640 1 -- 172.21.15.5:0/3467406296 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38042040 con 0x5647400c0590 2024-05-20T18:34:40.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.829+0000 7f7f26ffd640 1 -- 172.21.15.5:0/3467406296 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f38032980 con 0x5647400c0590 2024-05-20T18:34:40.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.829+0000 7f7f26ffd640 1 -- 172.21.15.5:0/3467406296 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38037040 con 0x5647400c0590 2024-05-20T18:34:40.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.830+0000 7f7f53028740 1 -- 172.21.15.5:0/3467406296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647400c0590 msgr2=0x5647404ab9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:40.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.830+0000 7f7f53028740 1 --2- 172.21.15.5:0/3467406296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647400c0590 0x5647404ab9b0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f7f38006b00 tx=0x7f7f3803b000 comp rx=0 tx=0).stop 2024-05-20T18:34:40.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.830+0000 7f7f53028740 1 -- 172.21.15.5:0/3467406296 shutdown_connections 2024-05-20T18:34:40.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.830+0000 7f7f53028740 1 --2- 172.21.15.5:0/3467406296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647400e1270 0x5647404abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.830+0000 7f7f53028740 1 --2- 172.21.15.5:0/3467406296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647400c0590 0x5647404ab9b0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.830+0000 7f7f53028740 1 --2- 172.21.15.5:0/3467406296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647400b83b0 0x5647404a95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.830+0000 7f7f53028740 1 -- 172.21.15.5:0/3467406296 >> 172.21.15.5:0/3467406296 conn(0x5647400d53c0 msgr2=0x5647404a8940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:40.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.830+0000 7f7f53028740 1 -- 172.21.15.5:0/3467406296 shutdown_connections 2024-05-20T18:34:40.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.830+0000 7f7f53028740 1 -- 172.21.15.5:0/3467406296 wait complete. 2024-05-20T18:34:40.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.831+0000 7f7f53028740 1 -- start start 2024-05-20T18:34:40.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.831+0000 7f7f53028740 1 Processor -- start 2024-05-20T18:34:40.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.832+0000 7f7f53028740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647400b83b0 0x5647404ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.832+0000 7f7f53028740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647400c0590 0x564740547bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.832+0000 7f7f53028740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647400e1270 0x564740543d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.832+0000 7f7f53028740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5647404ab590 con 0x5647400e1270 2024-05-20T18:34:40.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.832+0000 7f7f53028740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5647404a4f80 con 0x5647400b83b0 2024-05-20T18:34:40.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.832+0000 7f7f53028740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5647404c63b0 con 0x5647400c0590 2024-05-20T18:34:40.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.832+0000 7f7f53028740 10 client.0 _ll_register_callbacks cb 0x7f7f38035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:34:40.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.832+0000 7f7f4a63f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647400e1270 0x564740543d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.833+0000 7f7f4a63f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647400e1270 0x564740543d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38544/0 (socket says 172.21.15.5:38544) 2024-05-20T18:34:40.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.833+0000 7f7f4a63f640 1 -- 172.21.15.5:0/913376722 learned_addr learned my addr 172.21.15.5:0/913376722 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:40.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.833+0000 7f7f49e3e640 1 --2- 172.21.15.5:0/913376722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647400b83b0 0x5647404ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.833+0000 7f7f4963d640 1 --2- 172.21.15.5:0/913376722 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647400c0590 0x564740547bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.833+0000 7f7f4a63f640 1 -- 172.21.15.5:0/913376722 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647400c0590 msgr2=0x564740547bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:40.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.833+0000 7f7f4a63f640 1 --2- 172.21.15.5:0/913376722 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647400c0590 0x564740547bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.833+0000 7f7f4a63f640 1 -- 172.21.15.5:0/913376722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647400b83b0 msgr2=0x5647404ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:40.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.833+0000 7f7f4a63f640 1 --2- 172.21.15.5:0/913376722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647400b83b0 0x5647404ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.833+0000 7f7f4a63f640 1 -- 172.21.15.5:0/913376722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647404a3980 con 0x5647400e1270 2024-05-20T18:34:40.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.833+0000 7f7f4963d640 1 --2- 172.21.15.5:0/913376722 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647400c0590 0x564740547bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:34:40.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.833+0000 7f7f4a63f640 1 --2- 172.21.15.5:0/913376722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647400e1270 0x564740543d40 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c059ec0 tx=0x7f7f3c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311234870024f9a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:40.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.833+0000 7f7f24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:34:40.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.833+0000 7f7f24ff9640 1 -- 172.21.15.5:0/913376722 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c088040 con 0x5647400e1270 2024-05-20T18:34:40.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.833+0000 7f7f24ff9640 1 -- 172.21.15.5:0/913376722 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7f7f3c0625d0 con 0x5647400e1270 2024-05-20T18:34:40.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.833+0000 7f7f53028740 1 -- 172.21.15.5:0/913376722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f38032980 con 0x5647400e1270 2024-05-20T18:34:40.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.833+0000 7f7f24ff9640 1 -- 172.21.15.5:0/913376722 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f7f3c01d040 con 0x5647400e1270 2024-05-20T18:34:40.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.833+0000 7f7f53028740 10 client.8040 fetch_fsmap learned FSMap version 73 2024-05-20T18:34:40.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.833+0000 7f7f53028740 1 -- 172.21.15.5:0/913376722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x564740548110 con 0x5647400e1270 2024-05-20T18:34:40.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.834+0000 7f7f24ff9640 1 -- 172.21.15.5:0/913376722 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f38032980 con 0x5647400e1270 2024-05-20T18:34:40.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.834+0000 7f7f24ff9640 1 -- 172.21.15.5:0/913376722 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 73) ==== 3494+0+0 (secure 0 0 0) 0x7f7f3c0592d0 con 0x5647400e1270 2024-05-20T18:34:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.834+0000 7f7f53028740 10 client.8040 fetch_fsmap finished waiting for FSMap version 73 2024-05-20T18:34:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.834+0000 7f7f53028740 10 client.8040 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-20T18:34:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.834+0000 7f7f53028740 20 client.8040 populate_metadata read hostname 'smithi005' 2024-05-20T18:34:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.834+0000 7f7f53028740 1 --2- 172.21.15.5:0/913376722 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x5647404aca80 0x5647404d8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.834+0000 7f7f53028740 4 client.8040 mds_command: new command op to 7953 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T18:34:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.834+0000 7f7f53028740 1 -- 172.21.15.5:0/913376722 --> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] -- command(tid 0: {"prefix": "status"}) -- 0x5647403ecf50 con 0x5647404aca80 2024-05-20T18:34:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.834+0000 7f7f49e3e640 1 --2- 172.21.15.5:0/913376722 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x5647404aca80 0x5647404d8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.834+0000 7f7f24ff9640 10 client.8040 ms_handle_connect on v2:172.21.15.5:6836/3880006778 2024-05-20T18:34:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.834+0000 7f7f49e3e640 1 --2- 172.21.15.5:0/913376722 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x5647404aca80 0x5647404d8c40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e12924f023cf3678 server_cookie=7c75ea3850f265be in_seq=0 out_seq=0 2024-05-20T18:34:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.835+0000 7f7f24ff9640 1 -- 172.21.15.5:0/913376722 <== mds.0 v2:172.21.15.5:6836/3880006778 1 ==== command_reply(tid 0: 0 ) ==== 8+0+323 (crc 0 0 0) 0x5647403ecf50 con 0x5647404aca80 2024-05-20T18:34:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.835+0000 7f7f24ff9640 10 client.8040 handle_command_reply: tid=0 2024-05-20T18:34:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.835+0000 7f7f53028740 1 client.8040 shutdown 2024-05-20T18:34:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.835+0000 7f7f53028740 1 -- 172.21.15.5:0/913376722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647400e1270 msgr2=0x564740543d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:40.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.835+0000 7f7f53028740 1 --2- 172.21.15.5:0/913376722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647400e1270 0x564740543d40 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c059ec0 tx=0x7f7f3c01a000 comp rx=0 tx=0).stop 2024-05-20T18:34:40.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.835+0000 7f7f53028740 1 -- 172.21.15.5:0/913376722 shutdown_connections 2024-05-20T18:34:40.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.835+0000 7f7f53028740 1 --2- 172.21.15.5:0/913376722 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x5647404aca80 0x5647404d8c40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.835+0000 7f7f53028740 1 --2- 172.21.15.5:0/913376722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647400e1270 0x564740543d40 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.835+0000 7f7f53028740 1 --2- 172.21.15.5:0/913376722 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647400c0590 0x564740547bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.835+0000 7f7f24ff9640 0 client.8040 ms_handle_reset on v2:172.21.15.5:6836/3880006778 2024-05-20T18:34:40.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.835+0000 7f7f53028740 1 --2- 172.21.15.5:0/913376722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647400b83b0 0x5647404ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.835+0000 7f7f53028740 1 -- 172.21.15.5:0/913376722 >> 172.21.15.5:0/913376722 conn(0x5647400d53c0 msgr2=0x5647404ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:40.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.835+0000 7f7f53028740 1 -- 172.21.15.5:0/913376722 shutdown_connections 2024-05-20T18:34:40.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.835+0000 7f7f53028740 1 -- 172.21.15.5:0/913376722 wait complete. 2024-05-20T18:34:40.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.835+0000 7f7f53028740 20 client.8040 trim_cache size 0 max 16384 2024-05-20T18:34:40.842 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:34:40.842 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "399ab250-fd3c-47fa-a1c8-25e8149c9cac", 2024-05-20T18:34:40.842 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-20T18:34:40.842 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 7953, 2024-05-20T18:34:40.842 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-20T18:34:40.842 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-20T18:34:40.842 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-20T18:34:40.842 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 4.5250215779999996, 2024-05-20T18:34:40.842 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 73, 2024-05-20T18:34:40.842 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 94, 2024-05-20T18:34:40.842 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 94, 2024-05-20T18:34:40.843 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 5.46878888 2024-05-20T18:34:40.843 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:34:40.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.835+0000 7f7f4c0a2640 1 -- 172.21.15.5:0/3196629315 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7f2806be30 msgr2=0x7f7f2806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:40.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.835+0000 7f7f4c0a2640 1 --2- 172.21.15.5:0/3196629315 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7f2806be30 0x7f7f2806e340 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7f40000f60 tx=0x7f7f40007c00 comp rx=0 tx=0).stop 2024-05-20T18:34:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.835+0000 7f7f4c0a2640 1 -- 172.21.15.5:0/3196629315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f4415f220 msgr2=0x7f7f440b30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.835+0000 7f7f4c0a2640 1 --2- 172.21.15.5:0/3196629315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f4415f220 0x7f7f440b30f0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00cd80 tx=0x7f7f3c00a650 comp rx=0 tx=0).stop 2024-05-20T18:34:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.836+0000 7f7f4c0a2640 1 -- 172.21.15.5:0/3196629315 shutdown_connections 2024-05-20T18:34:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.836+0000 7f7f4c0a2640 1 --2- 172.21.15.5:0/3196629315 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7f2806be30 0x7f7f2806e340 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.836+0000 7f7f4c0a2640 1 --2- 172.21.15.5:0/3196629315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f4415f220 0x7f7f440b30f0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.836+0000 7f7f4c0a2640 1 --2- 172.21.15.5:0/3196629315 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f440cf910 0x7f7f440b77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.836+0000 7f7f4c0a2640 1 --2- 172.21.15.5:0/3196629315 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7f440cd680 0x7f7f44151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.836+0000 7f7f4c0a2640 1 -- 172.21.15.5:0/3196629315 >> 172.21.15.5:0/3196629315 conn(0x7f7f440b44d0 msgr2=0x7f7f440cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.836+0000 7f7f4c0a2640 1 -- 172.21.15.5:0/3196629315 shutdown_connections 2024-05-20T18:34:40.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.836+0000 7f7f4c0a2640 1 -- 172.21.15.5:0/3196629315 wait complete. 2024-05-20T18:34:40.848 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-20T18:34:40.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.961+0000 7f6147cca640 1 Processor -- start 2024-05-20T18:34:40.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.961+0000 7f6147cca640 1 -- start start 2024-05-20T18:34:40.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.962+0000 7f6147cca640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f614006d3d0 0x7f614006d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.962+0000 7f6147cca640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f61400b8190 0x7f61400b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.962+0000 7f6147cca640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f614006dd20 con 0x7f6140162820 2024-05-20T18:34:40.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.962+0000 7f6147cca640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f61400d0630 con 0x7f614006d3d0 2024-05-20T18:34:40.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.962+0000 7f6147cca640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f61400d39c0 con 0x7f61400b8190 2024-05-20T18:34:40.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.963+0000 7f6145265640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f614006d3d0 0x7f614006d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.963+0000 7f6145265640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f614006d3d0 0x7f614006d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38552/0 (socket says 172.21.15.5:38552) 2024-05-20T18:34:40.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.963+0000 7f6145265640 1 -- 172.21.15.5:0/1806936652 learned_addr learned my addr 172.21.15.5:0/1806936652 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:40.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.963+0000 7f6146267640 1 --2- 172.21.15.5:0/1806936652 >> v2:172.21.15.63:3300/0 conn(0x7f61400b8190 0x7f61400b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.963+0000 7f6145265640 1 -- 172.21.15.5:0/1806936652 >> v2:172.21.15.63:3300/0 conn(0x7f61400b8190 msgr2=0x7f61400b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:40.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.963+0000 7f6145265640 1 --2- 172.21.15.5:0/1806936652 >> v2:172.21.15.63:3300/0 conn(0x7f61400b8190 0x7f61400b85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.963+0000 7f6145265640 1 -- 172.21.15.5:0/1806936652 >> v1:172.21.15.63:6789/0 conn(0x7f6140162820 legacy=0x7f61400b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:40.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.963+0000 7f6145265640 1 -- 172.21.15.5:0/1806936652 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61400d3ca0 con 0x7f614006d3d0 2024-05-20T18:34:40.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.964+0000 7f6145265640 1 --2- 172.21.15.5:0/1806936652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f614006d3d0 0x7f614006d7e0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f614008e070 tx=0x7f613002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7fbabf777c32c8c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:40.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.964+0000 7f6144a64640 1 -- 172.21.15.5:0/1806936652 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6130037070 con 0x7f614006d3d0 2024-05-20T18:34:40.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.964+0000 7f6144a64640 1 -- 172.21.15.5:0/1806936652 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6130032440 con 0x7f614006d3d0 2024-05-20T18:34:40.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.964+0000 7f6144a64640 1 -- 172.21.15.5:0/1806936652 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6130032810 con 0x7f614006d3d0 2024-05-20T18:34:40.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.965+0000 7f6147cca640 1 -- 172.21.15.5:0/1806936652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f614006d3d0 msgr2=0x7f614006d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:40.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.965+0000 7f6147cca640 1 --2- 172.21.15.5:0/1806936652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f614006d3d0 0x7f614006d7e0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f614008e070 tx=0x7f613002a270 comp rx=0 tx=0).stop 2024-05-20T18:34:40.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.965+0000 7f6147cca640 1 -- 172.21.15.5:0/1806936652 shutdown_connections 2024-05-20T18:34:40.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.965+0000 7f6147cca640 1 --2- 172.21.15.5:0/1806936652 >> v2:172.21.15.63:3300/0 conn(0x7f61400b8190 0x7f61400b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.965+0000 7f6147cca640 1 --2- 172.21.15.5:0/1806936652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f614006d3d0 0x7f614006d7e0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.965+0000 7f6147cca640 1 -- 172.21.15.5:0/1806936652 >> 172.21.15.5:0/1806936652 conn(0x7f61400d4bd0 msgr2=0x7f61400d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:40.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.965+0000 7f6147cca640 1 -- 172.21.15.5:0/1806936652 shutdown_connections 2024-05-20T18:34:40.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.965+0000 7f6147cca640 1 -- 172.21.15.5:0/1806936652 wait complete. 2024-05-20T18:34:40.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.966+0000 7f6147cca640 1 Processor -- start 2024-05-20T18:34:40.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.967+0000 7f6147cca640 1 -- start start 2024-05-20T18:34:40.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.968+0000 7f6147cca640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f614006d3d0 0x7f61401f81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.968+0000 7f6147cca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61400b8190 0x7f614014eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.968+0000 7f6147cca640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6140162820 0x7f614014f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.968+0000 7f6147cca640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61400d0300 con 0x7f61400b8190 2024-05-20T18:34:40.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.968+0000 7f6147cca640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f61400d4260 con 0x7f614006d3d0 2024-05-20T18:34:40.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.968+0000 7f6147cca640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f61400b4050 con 0x7f6140162820 2024-05-20T18:34:40.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.968+0000 7f6145265640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61400b8190 0x7f614014eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.968+0000 7f6145265640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61400b8190 0x7f614014eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38558/0 (socket says 172.21.15.5:38558) 2024-05-20T18:34:40.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.968+0000 7f6145265640 1 -- 172.21.15.5:0/3884402974 learned_addr learned my addr 172.21.15.5:0/3884402974 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:40.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.968+0000 7f6146267640 1 --2- 172.21.15.5:0/3884402974 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6140162820 0x7f614014f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.968+0000 7f6145265640 1 -- 172.21.15.5:0/3884402974 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6140162820 msgr2=0x7f614014f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:40.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.968+0000 7f6145a66640 1 --2- 172.21.15.5:0/3884402974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f614006d3d0 0x7f61401f81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.968+0000 7f6145265640 1 --2- 172.21.15.5:0/3884402974 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6140162820 0x7f614014f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.968+0000 7f6145265640 1 -- 172.21.15.5:0/3884402974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f614006d3d0 msgr2=0x7f61401f81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:40.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.968+0000 7f6145265640 1 --2- 172.21.15.5:0/3884402974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f614006d3d0 0x7f61401f81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:40.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.968+0000 7f6145265640 1 -- 172.21.15.5:0/3884402974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61401f86e0 con 0x7f61400b8190 2024-05-20T18:34:40.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.968+0000 7f6145265640 1 --2- 172.21.15.5:0/3884402974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61400b8190 0x7f614014eea0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f6130000c00 tx=0x7f61300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:40.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.969+0000 7f61367fc640 1 -- 172.21.15.5:0/3884402974 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6130041030 con 0x7f61400b8190 2024-05-20T18:34:40.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.969+0000 7f61367fc640 1 -- 172.21.15.5:0/3884402974 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6130030440 con 0x7f61400b8190 2024-05-20T18:34:40.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.969+0000 7f6147cca640 1 -- 172.21.15.5:0/3884402974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f614014fa40 con 0x7f61400b8190 2024-05-20T18:34:40.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.969+0000 7f61367fc640 1 -- 172.21.15.5:0/3884402974 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6130037040 con 0x7f61400b8190 2024-05-20T18:34:40.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.969+0000 7f6147cca640 1 -- 172.21.15.5:0/3884402974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6140150000 con 0x7f61400b8190 2024-05-20T18:34:40.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.970+0000 7f6147cca640 1 -- 172.21.15.5:0/3884402974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6140075b00 con 0x7f61400b8190 2024-05-20T18:34:40.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.971+0000 7f61367fc640 1 -- 172.21.15.5:0/3884402974 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6130044070 con 0x7f61400b8190 2024-05-20T18:34:40.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.971+0000 7f61367fc640 1 --2- 172.21.15.5:0/3884402974 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f612406be60 0x7f612406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:40.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.971+0000 7f6145a66640 1 --2- 172.21.15.5:0/3884402974 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f612406be60 0x7f612406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:40.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.971+0000 7f61367fc640 1 -- 172.21.15.5:0/3884402974 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7f61300ab820 con 0x7f61400b8190 2024-05-20T18:34:40.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.972+0000 7f6145a66640 1 --2- 172.21.15.5:0/3884402974 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f612406be60 0x7f612406e370 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f613c001500 tx=0x7f613c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:40.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:40.976+0000 7f61367fc640 1 -- 172.21.15.5:0/3884402974 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6130079e90 con 0x7f61400b8190 2024-05-20T18:34:41.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:41.201+0000 7f6147cca640 1 -- 172.21.15.5:0/3884402974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f614015d590 con 0x7f61400b8190 2024-05-20T18:34:41.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:41.202+0000 7f61367fc640 1 -- 172.21.15.5:0/3884402974 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v73) ==== 53+0+73 (secure 0 0 0) 0x7f6130079790 con 0x7f61400b8190 2024-05-20T18:34:41.203 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-20T18:34:41.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:41.204+0000 7f6147cca640 1 -- 172.21.15.5:0/3884402974 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f612406be60 msgr2=0x7f612406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:41.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:41.204+0000 7f6147cca640 1 --2- 172.21.15.5:0/3884402974 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f612406be60 0x7f612406e370 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f613c001500 tx=0x7f613c009040 comp rx=0 tx=0).stop 2024-05-20T18:34:41.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:41.204+0000 7f6147cca640 1 -- 172.21.15.5:0/3884402974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61400b8190 msgr2=0x7f614014eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:41.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:41.204+0000 7f6147cca640 1 --2- 172.21.15.5:0/3884402974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61400b8190 0x7f614014eea0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f6130000c00 tx=0x7f61300046c0 comp rx=0 tx=0).stop 2024-05-20T18:34:41.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:41.204+0000 7f6147cca640 1 -- 172.21.15.5:0/3884402974 shutdown_connections 2024-05-20T18:34:41.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:41.204+0000 7f6147cca640 1 --2- 172.21.15.5:0/3884402974 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f612406be60 0x7f612406e370 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:41.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:41.204+0000 7f6147cca640 1 --2- 172.21.15.5:0/3884402974 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6140162820 0x7f614014f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:41.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:41.204+0000 7f6147cca640 1 --2- 172.21.15.5:0/3884402974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61400b8190 0x7f614014eea0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:41.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:41.204+0000 7f6147cca640 1 --2- 172.21.15.5:0/3884402974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f614006d3d0 0x7f61401f81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:41.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:41.204+0000 7f6147cca640 1 -- 172.21.15.5:0/3884402974 >> 172.21.15.5:0/3884402974 conn(0x7f61400d4bd0 msgr2=0x7f6140160eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:41.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:41.204+0000 7f6147cca640 1 -- 172.21.15.5:0/3884402974 shutdown_connections 2024-05-20T18:34:41.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:41.204+0000 7f6147cca640 1 -- 172.21.15.5:0/3884402974 wait complete. 2024-05-20T18:34:41.216 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-20T18:34:41.216 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-20T18:34:41.216 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-05-20T18:34:41.216 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-20T18:34:41.216 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-20T18:34:41.216 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-20T18:34:41.216 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-20T18:34:41.216 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-20T18:34:41.216 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:34:41.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:34:41.217 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-05-20T18:34:41.233 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T18:34:41.233 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T18:34:41.233 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T18:34:41.233 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:34:41.233 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-20T18:34:41.233 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:34:41.233 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:34:41.234 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-05-20T18:34:41.234 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-05-20T18:34:41.234 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:34:41.234 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-05-20T18:34:41.234 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-05-20T18:34:41.234 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-20T18:34:41.234 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:34:41.234 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-05-20T18:34:41.234 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-20T18:34:41.234 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42217sec preferred_lft 42217sec 2024-05-20T18:34:41.234 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-05-20T18:34:41.234 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:34:41.234 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:34:41.235 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:34:41.235 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-05-20T18:34:41.235 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-20T18:34:41.235 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:34:41.235 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-20T18:34:41.235 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:34:41.235 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::4474:eaff:fe07:4f9a/64 scope link 2024-05-20T18:34:41.235 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:34:41.235 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:34:41.235 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T18:34:41.235 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ce4:f4ff:fe5e:7906/64 scope link 2024-05-20T18:34:41.235 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:34:41.235 INFO:teuthology.orchestra.run.smithi005.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:34:41.235 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether da:26:04:0e:95:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T18:34:41.235 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::d826:4ff:fe0e:950a/64 scope link 2024-05-20T18:34:41.236 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:34:41.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:34:41.236 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-20T18:34:41.298 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-20T18:34:41.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:34:41.300 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-05-20T18:34:41.311 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-20T18:34:41.311+0000 7f6ed9e7a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T18:34:41.328 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-05-20T18:34:41.328 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-20T18:34:41.329 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-20T18:34:41.329 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-20T18:34:41.329 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-20T18:34:41.329 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:34:41.329 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-20T18:34:41.329 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-20T18:34:41.329 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-20T18:34:41.329 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-20T18:34:41.330 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-20T18:34:41.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:34:41.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:34:41.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:34:41.459 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-20T18:34:41.474 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-20T18:34:41.475 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-20T18:34:41.475 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:34:41.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:34:41.475 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-05-20T18:34:41.540 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:34:41.540 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-20T18:34:41.597 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:34:41.597 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:34:41.597 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:34:41.652 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:34:41.658 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-20T18:34:41.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:34:41.658 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:34:41.707 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:34:41.761 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:34:41.762 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:34:41.762 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:34:41.773 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:34:41.774 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:34:41.774 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:34:41.774 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:34:41.774 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:34:41.780 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:34:41.780 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:34:41.780 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:34:41.780 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:34:41.780 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:34:41.780 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790" 2024-05-20T18:34:41.780 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790:/ 2024-05-20T18:34:41.780 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=399ab250-fd3c-47fa-a1c8-25e8149c9cac" will pass to kernel 2024-05-20T18:34:41.782 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-20T18:34:41.782 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-05-20T18:34:41.782 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-20T18:34:41.782 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-05-20T18:34:41.782 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-20T18:34:41.782 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-20T18:34:41.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:34:41.783 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:34:41.857 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-20T18:34:41.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T18:34:41.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:34:41.959 INFO:teuthology.orchestra.run.smithi005.stdout:client7309 2024-05-20T18:34:41.960 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:34:41.960 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:34:41.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:34:41.977 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-05-20T18:34:42.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.135+0000 7fe0d4037640 1 Processor -- start 2024-05-20T18:34:42.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.135+0000 7fe0d4037640 1 -- start start 2024-05-20T18:34:42.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.136+0000 7fe0d4037640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe0cc0d3510 0x7fe0cc0b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.136+0000 7fe0d4037640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe0cc0cf060 0x7fe0cc0cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.136+0000 7fe0d4037640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0cc0cf9b0 0x7fe0cc1631b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.136+0000 7fe0d4037640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe0cc0b4300 con 0x7fe0cc0d3510 2024-05-20T18:34:42.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.136+0000 7fe0d4037640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe0cc0cfdc0 con 0x7fe0cc0cf060 2024-05-20T18:34:42.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.136+0000 7fe0d4037640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe0cc15fac0 con 0x7fe0cc0cf9b0 2024-05-20T18:34:42.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.136+0000 7fe0d1dd3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe0cc0d3510 0x7fe0cc0b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.136+0000 7fe0d1dd3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe0cc0d3510 0x7fe0cc0b3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38572/0 (socket says 172.21.15.5:38572) 2024-05-20T18:34:42.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.136+0000 7fe0d1dd3640 1 -- 172.21.15.5:0/1048365526 learned_addr learned my addr 172.21.15.5:0/1048365526 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:42.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.137+0000 7fe0d15d2640 1 --2- 172.21.15.5:0/1048365526 >> v2:172.21.15.63:3300/0 conn(0x7fe0cc0cf060 0x7fe0cc0cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.137+0000 7fe0d25d4640 1 --2- 172.21.15.5:0/1048365526 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0cc0cf9b0 0x7fe0cc1631b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.137+0000 7fe0d1dd3640 1 -- 172.21.15.5:0/1048365526 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0cc0cf9b0 msgr2=0x7fe0cc1631b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:42.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.137+0000 7fe0d1dd3640 1 --2- 172.21.15.5:0/1048365526 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0cc0cf9b0 0x7fe0cc1631b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.137+0000 7fe0d1dd3640 1 -- 172.21.15.5:0/1048365526 >> v2:172.21.15.63:3300/0 conn(0x7fe0cc0cf060 msgr2=0x7fe0cc0cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:42.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.137+0000 7fe0d1dd3640 1 --2- 172.21.15.5:0/1048365526 >> v2:172.21.15.63:3300/0 conn(0x7fe0cc0cf060 0x7fe0cc0cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.137+0000 7fe0d1dd3640 1 -- 172.21.15.5:0/1048365526 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0cc06e090 con 0x7fe0cc0d3510 2024-05-20T18:34:42.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.137+0000 7fe0d1dd3640 1 --2- 172.21.15.5:0/1048365526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0cc0d3510 0x7fe0cc0b3dc0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc0c64d0 tx=0x7fe0c802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218b5126fccd0ffb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:42.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.138+0000 7fe0d0dd1640 1 -- 172.21.15.5:0/1048365526 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8036070 con 0x7fe0cc0d3510 2024-05-20T18:34:42.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.138+0000 7fe0d0dd1640 1 -- 172.21.15.5:0/1048365526 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0c8031600 con 0x7fe0cc0d3510 2024-05-20T18:34:42.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.138+0000 7fe0d0dd1640 1 -- 172.21.15.5:0/1048365526 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c80319d0 con 0x7fe0cc0d3510 2024-05-20T18:34:42.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.138+0000 7fe0d4037640 1 -- 172.21.15.5:0/1048365526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0cc0d3510 msgr2=0x7fe0cc0b3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:42.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.138+0000 7fe0d4037640 1 --2- 172.21.15.5:0/1048365526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0cc0d3510 0x7fe0cc0b3dc0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc0c64d0 tx=0x7fe0c802a610 comp rx=0 tx=0).stop 2024-05-20T18:34:42.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.138+0000 7fe0d4037640 1 -- 172.21.15.5:0/1048365526 shutdown_connections 2024-05-20T18:34:42.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.138+0000 7fe0d4037640 1 --2- 172.21.15.5:0/1048365526 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0cc0cf9b0 0x7fe0cc1631b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.138+0000 7fe0d4037640 1 --2- 172.21.15.5:0/1048365526 >> v2:172.21.15.63:3300/0 conn(0x7fe0cc0cf060 0x7fe0cc0cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.138+0000 7fe0d4037640 1 --2- 172.21.15.5:0/1048365526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0cc0d3510 0x7fe0cc0b3dc0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.138+0000 7fe0d4037640 1 -- 172.21.15.5:0/1048365526 >> 172.21.15.5:0/1048365526 conn(0x7fe0cc0d29f0 msgr2=0x7fe0cc0d2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:42.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.138+0000 7fe0d4037640 1 -- 172.21.15.5:0/1048365526 shutdown_connections 2024-05-20T18:34:42.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.138+0000 7fe0d4037640 1 -- 172.21.15.5:0/1048365526 wait complete. 2024-05-20T18:34:42.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.140+0000 7fe0d4037640 1 Processor -- start 2024-05-20T18:34:42.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.140+0000 7fe0d4037640 1 -- start start 2024-05-20T18:34:42.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.142+0000 7fe0d4037640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0cc0cf060 0x7fe0cc09e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.142+0000 7fe0d4037640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0cc0cf9b0 0x7fe0cc09c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.142+0000 7fe0d4037640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0cc0d3510 0x7fe0cc14c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.142+0000 7fe0d4037640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe0cc0d1af0 con 0x7fe0cc0cf060 2024-05-20T18:34:42.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.142+0000 7fe0d4037640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe0cc0d1780 con 0x7fe0cc0cf9b0 2024-05-20T18:34:42.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.142+0000 7fe0d4037640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe0cc1636f0 con 0x7fe0cc0d3510 2024-05-20T18:34:42.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.142+0000 7fe0d1dd3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0cc0cf060 0x7fe0cc09e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.142+0000 7fe0d1dd3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0cc0cf060 0x7fe0cc09e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38586/0 (socket says 172.21.15.5:38586) 2024-05-20T18:34:42.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.142+0000 7fe0d1dd3640 1 -- 172.21.15.5:0/3193037502 learned_addr learned my addr 172.21.15.5:0/3193037502 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:42.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.142+0000 7fe0d15d2640 1 --2- 172.21.15.5:0/3193037502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0cc0cf9b0 0x7fe0cc09c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.142+0000 7fe0d25d4640 1 --2- 172.21.15.5:0/3193037502 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0cc0d3510 0x7fe0cc14c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.142+0000 7fe0d1dd3640 1 -- 172.21.15.5:0/3193037502 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0cc0d3510 msgr2=0x7fe0cc14c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:42.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.142+0000 7fe0d1dd3640 1 --2- 172.21.15.5:0/3193037502 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0cc0d3510 0x7fe0cc14c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.142+0000 7fe0d1dd3640 1 -- 172.21.15.5:0/3193037502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0cc0cf9b0 msgr2=0x7fe0cc09c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:42.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.142+0000 7fe0d1dd3640 1 --2- 172.21.15.5:0/3193037502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0cc0cf9b0 0x7fe0cc09c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.142+0000 7fe0d1dd3640 1 -- 172.21.15.5:0/3193037502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0cc14a370 con 0x7fe0cc0cf060 2024-05-20T18:34:42.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.142+0000 7fe0d1dd3640 1 --2- 172.21.15.5:0/3193037502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0cc0cf060 0x7fe0cc09e320 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8009f40 tx=0x7fe0c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:42.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.142+0000 7fe0c27fc640 1 -- 172.21.15.5:0/3193037502 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c803f020 con 0x7fe0cc0cf060 2024-05-20T18:34:42.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.142+0000 7fe0c27fc640 1 -- 172.21.15.5:0/3193037502 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0c8037040 con 0x7fe0cc0cf060 2024-05-20T18:34:42.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.142+0000 7fe0d4037640 1 -- 172.21.15.5:0/3193037502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0cc14a650 con 0x7fe0cc0cf060 2024-05-20T18:34:42.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.142+0000 7fe0c27fc640 1 -- 172.21.15.5:0/3193037502 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8036070 con 0x7fe0cc0cf060 2024-05-20T18:34:42.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.143+0000 7fe0d4037640 1 -- 172.21.15.5:0/3193037502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0cc14ac10 con 0x7fe0cc0cf060 2024-05-20T18:34:42.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.144+0000 7fe0c27fc640 1 -- 172.21.15.5:0/3193037502 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe0c8042020 con 0x7fe0cc0cf060 2024-05-20T18:34:42.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.144+0000 7fe0c27fc640 1 --2- 172.21.15.5:0/3193037502 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe0b006be30 0x7fe0b006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.145+0000 7fe0c27fc640 1 -- 172.21.15.5:0/3193037502 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7fe0c80ab600 con 0x7fe0cc0cf060 2024-05-20T18:34:42.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.145+0000 7fe0d15d2640 1 --2- 172.21.15.5:0/3193037502 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe0b006be30 0x7fe0b006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.145+0000 7fe0d15d2640 1 --2- 172.21.15.5:0/3193037502 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe0b006be30 0x7fe0b006e340 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc006fd0 tx=0x7fe0bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:42.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.146+0000 7fe0dafb2740 1 Processor -- start 2024-05-20T18:34:42.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.146+0000 7fe0dafb2740 1 -- start start 2024-05-20T18:34:42.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.147+0000 7fe0dafb2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556605c27410 0x556605ff2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.147+0000 7fe0dafb2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556605bfe550 0x55660600d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.147+0000 7fe0dafb2740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556605c06730 0x55660600d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.147+0000 7fe0dafb2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556605f2b990 con 0x556605bfe550 2024-05-20T18:34:42.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.147+0000 7fe0dafb2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556605ed35b0 con 0x556605c27410 2024-05-20T18:34:42.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.147+0000 7fe0dafb2740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x556605f27950 con 0x556605c06730 2024-05-20T18:34:42.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.147+0000 7fe0d15d2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556605bfe550 0x55660600d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.147+0000 7fe0d15d2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556605bfe550 0x55660600d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38600/0 (socket says 172.21.15.5:38600) 2024-05-20T18:34:42.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.147+0000 7fe0d15d2640 1 -- 172.21.15.5:0/1360139267 learned_addr learned my addr 172.21.15.5:0/1360139267 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:42.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.147+0000 7fe0d1dd3640 1 --2- 172.21.15.5:0/1360139267 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556605c27410 0x556605ff2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.147+0000 7fe0d25d4640 1 --2- 172.21.15.5:0/1360139267 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556605c06730 0x55660600d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.148+0000 7fe0d15d2640 1 -- 172.21.15.5:0/1360139267 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556605c06730 msgr2=0x55660600d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:42.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.148+0000 7fe0d15d2640 1 --2- 172.21.15.5:0/1360139267 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556605c06730 0x55660600d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.148+0000 7fe0d15d2640 1 -- 172.21.15.5:0/1360139267 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556605c27410 msgr2=0x556605ff2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:42.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.148+0000 7fe0d15d2640 1 --2- 172.21.15.5:0/1360139267 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556605c27410 0x556605ff2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.148+0000 7fe0d15d2640 1 -- 172.21.15.5:0/1360139267 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556605d077e0 con 0x556605bfe550 2024-05-20T18:34:42.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.148+0000 7fe0d15d2640 1 --2- 172.21.15.5:0/1360139267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556605bfe550 0x55660600d160 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc00fa10 tx=0x7fe0bc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=859898bbb4466d47 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:42.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.148+0000 7fe0af7fe640 1 -- 172.21.15.5:0/1360139267 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc01b050 con 0x556605bfe550 2024-05-20T18:34:42.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.148+0000 7fe0af7fe640 1 -- 172.21.15.5:0/1360139267 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0bc025070 con 0x556605bfe550 2024-05-20T18:34:42.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.148+0000 7fe0af7fe640 1 -- 172.21.15.5:0/1360139267 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc014e00 con 0x556605bfe550 2024-05-20T18:34:42.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.149+0000 7fe0dafb2740 1 -- 172.21.15.5:0/1360139267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556605bfe550 msgr2=0x55660600d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:42.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.149+0000 7fe0dafb2740 1 --2- 172.21.15.5:0/1360139267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556605bfe550 0x55660600d160 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc00fa10 tx=0x7fe0bc012000 comp rx=0 tx=0).stop 2024-05-20T18:34:42.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.149+0000 7fe0dafb2740 1 -- 172.21.15.5:0/1360139267 shutdown_connections 2024-05-20T18:34:42.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.149+0000 7fe0dafb2740 1 --2- 172.21.15.5:0/1360139267 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556605c06730 0x55660600d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.149+0000 7fe0dafb2740 1 --2- 172.21.15.5:0/1360139267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556605bfe550 0x55660600d160 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.149+0000 7fe0dafb2740 1 --2- 172.21.15.5:0/1360139267 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556605c27410 0x556605ff2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.149+0000 7fe0dafb2740 1 -- 172.21.15.5:0/1360139267 >> 172.21.15.5:0/1360139267 conn(0x556605c1b560 msgr2=0x556605fc7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:42.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.149+0000 7fe0dafb2740 1 -- 172.21.15.5:0/1360139267 shutdown_connections 2024-05-20T18:34:42.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.149+0000 7fe0dafb2740 1 -- 172.21.15.5:0/1360139267 wait complete. 2024-05-20T18:34:42.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.150+0000 7fe0dafb2740 1 -- start start 2024-05-20T18:34:42.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.150+0000 7fe0dafb2740 1 Processor -- start 2024-05-20T18:34:42.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.151+0000 7fe0dafb2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556605bfe550 0x556605fed0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.151+0000 7fe0dafb2740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556605c06730 0x556605fed5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.151+0000 7fe0dafb2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556605c27410 0x556605ff3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.151+0000 7fe0dafb2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556605ee6b40 con 0x556605bfe550 2024-05-20T18:34:42.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.151+0000 7fe0dafb2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55660601db60 con 0x556605c27410 2024-05-20T18:34:42.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.151+0000 7fe0dafb2740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x556606010370 con 0x556605c06730 2024-05-20T18:34:42.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.151+0000 7fe0dafb2740 10 client.0 _ll_register_callbacks cb 0x556605edb0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:34:42.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.151+0000 7fe0d1dd3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556605bfe550 0x556605fed0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.152+0000 7fe0d1dd3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556605bfe550 0x556605fed0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38608/0 (socket says 172.21.15.5:38608) 2024-05-20T18:34:42.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.152+0000 7fe0d1dd3640 1 -- 172.21.15.5:0/1107238499 learned_addr learned my addr 172.21.15.5:0/1107238499 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:42.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.152+0000 7fe0d25d4640 1 --2- 172.21.15.5:0/1107238499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556605c27410 0x556605ff3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.152+0000 7fe0d15d2640 1 --2- 172.21.15.5:0/1107238499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556605c06730 0x556605fed5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.152+0000 7fe0d1dd3640 1 -- 172.21.15.5:0/1107238499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556605c06730 msgr2=0x556605fed5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:42.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.152+0000 7fe0d1dd3640 1 --2- 172.21.15.5:0/1107238499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556605c06730 0x556605fed5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.152+0000 7fe0d1dd3640 1 -- 172.21.15.5:0/1107238499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556605c27410 msgr2=0x556605ff3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:42.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.152+0000 7fe0d1dd3640 1 --2- 172.21.15.5:0/1107238499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556605c27410 0x556605ff3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.152+0000 7fe0d1dd3640 1 -- 172.21.15.5:0/1107238499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556605d67f70 con 0x556605bfe550 2024-05-20T18:34:42.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.152+0000 7fe0d25d4640 1 --2- 172.21.15.5:0/1107238499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556605c27410 0x556605ff3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:34:42.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.152+0000 7fe0d15d2640 1 --2- 172.21.15.5:0/1107238499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556605c06730 0x556605fed5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:34:42.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.152+0000 7fe0d1dd3640 1 --2- 172.21.15.5:0/1107238499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556605bfe550 0x556605fed0b0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8075af0 tx=0x7fe0c8079ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b13b6e32a4b0ec1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:42.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.152+0000 7fe0acff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:34:42.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.152+0000 7fe0acff9640 1 -- 172.21.15.5:0/1107238499 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8081de0 con 0x556605bfe550 2024-05-20T18:34:42.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.152+0000 7fe0acff9640 1 -- 172.21.15.5:0/1107238499 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7fe0c8081f80 con 0x556605bfe550 2024-05-20T18:34:42.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.152+0000 7fe0dafb2740 1 -- 172.21.15.5:0/1107238499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0bc025070 con 0x556605bfe550 2024-05-20T18:34:42.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.152+0000 7fe0acff9640 1 -- 172.21.15.5:0/1107238499 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fe0c8078ae0 con 0x556605bfe550 2024-05-20T18:34:42.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.152+0000 7fe0dafb2740 10 client.8058 fetch_fsmap learned FSMap version 73 2024-05-20T18:34:42.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.152+0000 7fe0dafb2740 1 -- 172.21.15.5:0/1107238499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x556605bff170 con 0x556605bfe550 2024-05-20T18:34:42.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.153+0000 7fe0acff9640 1 -- 172.21.15.5:0/1107238499 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0bc025070 con 0x556605bfe550 2024-05-20T18:34:42.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.153+0000 7fe0acff9640 1 -- 172.21.15.5:0/1107238499 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 73) ==== 3494+0+0 (secure 0 0 0) 0x7fe0c8078c80 con 0x556605bfe550 2024-05-20T18:34:42.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.153+0000 7fe0dafb2740 10 client.8058 fetch_fsmap finished waiting for FSMap version 73 2024-05-20T18:34:42.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.153+0000 7fe0dafb2740 10 client.8058 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-20T18:34:42.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.153+0000 7fe0dafb2740 20 client.8058 populate_metadata read hostname 'smithi005' 2024-05-20T18:34:42.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.153+0000 7fe0dafb2740 1 --2- 172.21.15.5:0/1107238499 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x556605be61f0 0x556605ff3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.153+0000 7fe0dafb2740 4 client.8058 mds_command: new command op to 7953 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:34:42.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.153+0000 7fe0dafb2740 1 -- 172.21.15.5:0/1107238499 --> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556605f390c0 con 0x556605be61f0 2024-05-20T18:34:42.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.153+0000 7fe0d15d2640 1 --2- 172.21.15.5:0/1107238499 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x556605be61f0 0x556605ff3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.153+0000 7fe0acff9640 10 client.8058 ms_handle_connect on v2:172.21.15.5:6836/3880006778 2024-05-20T18:34:42.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.153+0000 7fe0d15d2640 1 --2- 172.21.15.5:0/1107238499 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x556605be61f0 0x556605ff3880 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a74f5073f3269ee server_cookie=413c4a82b58f9abc in_seq=0 out_seq=0 2024-05-20T18:34:42.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.154+0000 7fe0acff9640 1 -- 172.21.15.5:0/1107238499 <== mds.0 v2:172.21.15.5:6836/3880006778 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556605f390c0 con 0x556605be61f0 2024-05-20T18:34:42.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.154+0000 7fe0acff9640 10 client.8058 handle_command_reply: tid=0 2024-05-20T18:34:42.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.154+0000 7fe0dafb2740 1 client.8058 shutdown 2024-05-20T18:34:42.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.154+0000 7fe0dafb2740 1 -- 172.21.15.5:0/1107238499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556605bfe550 msgr2=0x556605fed0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:42.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.154+0000 7fe0dafb2740 1 --2- 172.21.15.5:0/1107238499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556605bfe550 0x556605fed0b0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8075af0 tx=0x7fe0c8079ed0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.154+0000 7fe0dafb2740 1 -- 172.21.15.5:0/1107238499 shutdown_connections 2024-05-20T18:34:42.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.154+0000 7fe0dafb2740 1 --2- 172.21.15.5:0/1107238499 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x556605be61f0 0x556605ff3880 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.154+0000 7fe0dafb2740 1 --2- 172.21.15.5:0/1107238499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556605c27410 0x556605ff3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.154+0000 7fe0dafb2740 1 --2- 172.21.15.5:0/1107238499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556605c06730 0x556605fed5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.154+0000 7fe0dafb2740 1 --2- 172.21.15.5:0/1107238499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556605bfe550 0x556605fed0b0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.154+0000 7fe0acff9640 0 client.8058 ms_handle_reset on v2:172.21.15.5:6836/3880006778 2024-05-20T18:34:42.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.154+0000 7fe0dafb2740 1 -- 172.21.15.5:0/1107238499 >> 172.21.15.5:0/1107238499 conn(0x556605c1b560 msgr2=0x556605fc7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:42.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.154+0000 7fe0dafb2740 1 -- 172.21.15.5:0/1107238499 shutdown_connections 2024-05-20T18:34:42.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.154+0000 7fe0dafb2740 1 -- 172.21.15.5:0/1107238499 wait complete. 2024-05-20T18:34:42.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.155+0000 7fe0dafb2740 20 client.8058 trim_cache size 0 max 16384 2024-05-20T18:34:42.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.173+0000 7fe0dafb2740 1 Processor -- start 2024-05-20T18:34:42.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.173+0000 7fe0dafb2740 1 -- start start 2024-05-20T18:34:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.174+0000 7fe0dafb2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556605bfe550 0x556605fc83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.174+0000 7fe0dafb2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556605c06730 0x55660601c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.174+0000 7fe0dafb2740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556605c27410 0x55660601cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.174+0000 7fe0dafb2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe0cc1ea550 con 0x556605bfe550 2024-05-20T18:34:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.174+0000 7fe0dafb2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe0cc162320 con 0x556605c06730 2024-05-20T18:34:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.174+0000 7fe0dafb2740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe0cc09ff10 con 0x556605c27410 2024-05-20T18:34:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.175+0000 7fe0d1dd3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556605bfe550 0x556605fc83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.175+0000 7fe0d1dd3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556605bfe550 0x556605fc83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38612/0 (socket says 172.21.15.5:38612) 2024-05-20T18:34:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.175+0000 7fe0d1dd3640 1 -- 172.21.15.5:0/2159538271 learned_addr learned my addr 172.21.15.5:0/2159538271 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.175+0000 7fe0d15d2640 1 --2- 172.21.15.5:0/2159538271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556605c06730 0x55660601c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.175+0000 7fe0d25d4640 1 --2- 172.21.15.5:0/2159538271 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556605c27410 0x55660601cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.175+0000 7fe0d1dd3640 1 -- 172.21.15.5:0/2159538271 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556605c27410 msgr2=0x55660601cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:42.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.175+0000 7fe0d1dd3640 1 --2- 172.21.15.5:0/2159538271 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556605c27410 0x55660601cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.175+0000 7fe0d1dd3640 1 -- 172.21.15.5:0/2159538271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556605c06730 msgr2=0x55660601c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:42.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.175+0000 7fe0d1dd3640 1 --2- 172.21.15.5:0/2159538271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556605c06730 0x55660601c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.175+0000 7fe0d1dd3640 1 -- 172.21.15.5:0/2159538271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556605ff5bf0 con 0x556605bfe550 2024-05-20T18:34:42.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.175+0000 7fe0d15d2640 1 --2- 172.21.15.5:0/2159538271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556605c06730 0x55660601c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:34:42.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.175+0000 7fe0d1dd3640 1 --2- 172.21.15.5:0/2159538271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556605bfe550 0x556605fc83c0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8076010 tx=0x7fe0c80ab050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24907f78830380a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:42.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.175+0000 7fe0aeffd640 1 -- 172.21.15.5:0/2159538271 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8098070 con 0x556605bfe550 2024-05-20T18:34:42.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.175+0000 7fe0aeffd640 1 -- 172.21.15.5:0/2159538271 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0c8078c80 con 0x556605bfe550 2024-05-20T18:34:42.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.175+0000 7fe0aeffd640 1 -- 172.21.15.5:0/2159538271 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8082020 con 0x556605bfe550 2024-05-20T18:34:42.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.176+0000 7fe0dafb2740 1 -- 172.21.15.5:0/2159538271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556605bfe550 msgr2=0x556605fc83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:42.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.176+0000 7fe0dafb2740 1 --2- 172.21.15.5:0/2159538271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556605bfe550 0x556605fc83c0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8076010 tx=0x7fe0c80ab050 comp rx=0 tx=0).stop 2024-05-20T18:34:42.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.176+0000 7fe0dafb2740 1 -- 172.21.15.5:0/2159538271 shutdown_connections 2024-05-20T18:34:42.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.176+0000 7fe0dafb2740 1 --2- 172.21.15.5:0/2159538271 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556605c27410 0x55660601cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.176+0000 7fe0dafb2740 1 --2- 172.21.15.5:0/2159538271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556605c06730 0x55660601c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.176+0000 7fe0dafb2740 1 --2- 172.21.15.5:0/2159538271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556605bfe550 0x556605fc83c0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.176+0000 7fe0dafb2740 1 -- 172.21.15.5:0/2159538271 >> 172.21.15.5:0/2159538271 conn(0x556605c1b560 msgr2=0x556605fc7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:42.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.176+0000 7fe0dafb2740 1 -- 172.21.15.5:0/2159538271 shutdown_connections 2024-05-20T18:34:42.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.176+0000 7fe0dafb2740 1 -- 172.21.15.5:0/2159538271 wait complete. 2024-05-20T18:34:42.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.177+0000 7fe0dafb2740 1 -- start start 2024-05-20T18:34:42.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.177+0000 7fe0dafb2740 1 Processor -- start 2024-05-20T18:34:42.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.178+0000 7fe0dafb2740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556605bfe550 0x55660601cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.178+0000 7fe0dafb2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556605c06730 0x556606089120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.178+0000 7fe0dafb2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556605c27410 0x556605fc5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.178+0000 7fe0dafb2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe0cc0a0560 con 0x556605c06730 2024-05-20T18:34:42.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.178+0000 7fe0dafb2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556605feb2a0 con 0x556605c27410 2024-05-20T18:34:42.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.178+0000 7fe0dafb2740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x556605fc6c60 con 0x556605bfe550 2024-05-20T18:34:42.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.179+0000 7fe0dafb2740 10 client.0 _ll_register_callbacks cb 0x7fe0c8078ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:34:42.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.179+0000 7fe0d15d2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556605c06730 0x556606089120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.179+0000 7fe0d15d2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556605c06730 0x556606089120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38616/0 (socket says 172.21.15.5:38616) 2024-05-20T18:34:42.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.179+0000 7fe0d15d2640 1 -- 172.21.15.5:0/1540562138 learned_addr learned my addr 172.21.15.5:0/1540562138 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:42.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.179+0000 7fe0d25d4640 1 --2- 172.21.15.5:0/1540562138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556605c27410 0x556605fc5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.179+0000 7fe0d1dd3640 1 --2- 172.21.15.5:0/1540562138 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556605bfe550 0x55660601cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.179+0000 7fe0d15d2640 1 -- 172.21.15.5:0/1540562138 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556605bfe550 msgr2=0x55660601cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:42.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.179+0000 7fe0d15d2640 1 --2- 172.21.15.5:0/1540562138 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556605bfe550 0x55660601cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.179+0000 7fe0d15d2640 1 -- 172.21.15.5:0/1540562138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556605c27410 msgr2=0x556605fc5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:42.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.179+0000 7fe0d15d2640 1 --2- 172.21.15.5:0/1540562138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556605c27410 0x556605fc5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.179+0000 7fe0d15d2640 1 -- 172.21.15.5:0/1540562138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55660601c1b0 con 0x556605c06730 2024-05-20T18:34:42.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.179+0000 7fe0d15d2640 1 --2- 172.21.15.5:0/1540562138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556605c06730 0x556606089120 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc00f9b0 tx=0x7fe0bc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d0862a06ee66c64 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:42.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.179+0000 7fe0acff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:34:42.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.179+0000 7fe0acff9640 1 -- 172.21.15.5:0/1540562138 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc03a070 con 0x556605c06730 2024-05-20T18:34:42.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.179+0000 7fe0acff9640 1 -- 172.21.15.5:0/1540562138 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7fe0bc036430 con 0x556605c06730 2024-05-20T18:34:42.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.179+0000 7fe0dafb2740 1 -- 172.21.15.5:0/1540562138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0c8078c80 con 0x556605c06730 2024-05-20T18:34:42.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.179+0000 7fe0acff9640 1 -- 172.21.15.5:0/1540562138 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fe0bc045050 con 0x556605c06730 2024-05-20T18:34:42.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.179+0000 7fe0dafb2740 10 client.8064 fetch_fsmap learned FSMap version 73 2024-05-20T18:34:42.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.179+0000 7fe0dafb2740 1 -- 172.21.15.5:0/1540562138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x556605fc9bd0 con 0x556605c06730 2024-05-20T18:34:42.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.180+0000 7fe0acff9640 1 -- 172.21.15.5:0/1540562138 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0c8078c80 con 0x556605c06730 2024-05-20T18:34:42.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.180+0000 7fe0acff9640 1 -- 172.21.15.5:0/1540562138 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 73) ==== 3494+0+0 (secure 0 0 0) 0x7fe0bc048020 con 0x556605c06730 2024-05-20T18:34:42.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.180+0000 7fe0dafb2740 10 client.8064 fetch_fsmap finished waiting for FSMap version 73 2024-05-20T18:34:42.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.180+0000 7fe0dafb2740 10 client.8064 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-20T18:34:42.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.180+0000 7fe0dafb2740 20 client.8064 populate_metadata read hostname 'smithi005' 2024-05-20T18:34:42.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.180+0000 7fe0dafb2740 1 --2- 172.21.15.5:0/1540562138 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x556605fc6480 0x556605fc13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.180+0000 7fe0dafb2740 4 client.8064 mds_command: new command op to 7953 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-05-20T18:34:42.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.180+0000 7fe0dafb2740 1 -- 172.21.15.5:0/1540562138 --> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x556605f33410 con 0x556605fc6480 2024-05-20T18:34:42.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.180+0000 7fe0d1dd3640 1 --2- 172.21.15.5:0/1540562138 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x556605fc6480 0x556605fc13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.180+0000 7fe0acff9640 10 client.8064 ms_handle_connect on v2:172.21.15.5:6836/3880006778 2024-05-20T18:34:42.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.180+0000 7fe0d1dd3640 1 --2- 172.21.15.5:0/1540562138 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x556605fc6480 0x556605fc13d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2782d2275d92c5 server_cookie=f42e8476e33efc01 in_seq=0 out_seq=0 2024-05-20T18:34:42.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.181+0000 7fe0acff9640 1 -- 172.21.15.5:0/1540562138 <== mds.0 v2:172.21.15.5:6836/3880006778 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556605f33410 con 0x556605fc6480 2024-05-20T18:34:42.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.181+0000 7fe0acff9640 10 client.8064 handle_command_reply: tid=0 2024-05-20T18:34:42.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.181+0000 7fe0dafb2740 1 client.8064 shutdown 2024-05-20T18:34:42.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.181+0000 7fe0dafb2740 1 -- 172.21.15.5:0/1540562138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556605c06730 msgr2=0x556606089120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:42.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.181+0000 7fe0dafb2740 1 --2- 172.21.15.5:0/1540562138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556605c06730 0x556606089120 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc00f9b0 tx=0x7fe0bc016000 comp rx=0 tx=0).stop 2024-05-20T18:34:42.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.181+0000 7fe0dafb2740 1 -- 172.21.15.5:0/1540562138 shutdown_connections 2024-05-20T18:34:42.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.181+0000 7fe0dafb2740 1 --2- 172.21.15.5:0/1540562138 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x556605fc6480 0x556605fc13d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.181+0000 7fe0acff9640 0 client.8064 ms_handle_reset on v2:172.21.15.5:6836/3880006778 2024-05-20T18:34:42.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.181+0000 7fe0dafb2740 1 --2- 172.21.15.5:0/1540562138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556605c27410 0x556605fc5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.181+0000 7fe0dafb2740 1 --2- 172.21.15.5:0/1540562138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556605c06730 0x556606089120 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.181+0000 7fe0dafb2740 1 --2- 172.21.15.5:0/1540562138 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556605bfe550 0x55660601cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.181+0000 7fe0dafb2740 1 -- 172.21.15.5:0/1540562138 >> 172.21.15.5:0/1540562138 conn(0x556605c1b560 msgr2=0x55660601ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:42.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.181+0000 7fe0dafb2740 1 -- 172.21.15.5:0/1540562138 shutdown_connections 2024-05-20T18:34:42.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.181+0000 7fe0dafb2740 1 -- 172.21.15.5:0/1540562138 wait complete. 2024-05-20T18:34:42.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.181+0000 7fe0dafb2740 20 client.8064 trim_cache size 0 max 16384 2024-05-20T18:34:42.187 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:34:42.187 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-20T18:34:42.187 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "a984b717-d51c-479b-85f3-4c57c856de5b", 2024-05-20T18:34:42.187 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-20T18:34:42.188 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:34:42.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.182+0000 7fe0d4037640 1 -- 172.21.15.5:0/3193037502 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe0b006be30 msgr2=0x7fe0b006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:42.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.182+0000 7fe0d4037640 1 --2- 172.21.15.5:0/3193037502 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe0b006be30 0x7fe0b006e340 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc006fd0 tx=0x7fe0bc008040 comp rx=0 tx=0).stop 2024-05-20T18:34:42.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.182+0000 7fe0d4037640 1 -- 172.21.15.5:0/3193037502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0cc0cf060 msgr2=0x7fe0cc09e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:42.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.182+0000 7fe0d4037640 1 --2- 172.21.15.5:0/3193037502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0cc0cf060 0x7fe0cc09e320 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8009f40 tx=0x7fe0c8004a40 comp rx=0 tx=0).stop 2024-05-20T18:34:42.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.182+0000 7fe0d4037640 1 -- 172.21.15.5:0/3193037502 shutdown_connections 2024-05-20T18:34:42.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.182+0000 7fe0d4037640 1 --2- 172.21.15.5:0/3193037502 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe0b006be30 0x7fe0b006e340 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.182+0000 7fe0d4037640 1 --2- 172.21.15.5:0/3193037502 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0cc0d3510 0x7fe0cc14c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.182+0000 7fe0d4037640 1 --2- 172.21.15.5:0/3193037502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0cc0cf9b0 0x7fe0cc09c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.182+0000 7fe0d4037640 1 --2- 172.21.15.5:0/3193037502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0cc0cf060 0x7fe0cc09e320 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.182+0000 7fe0d4037640 1 -- 172.21.15.5:0/3193037502 >> 172.21.15.5:0/3193037502 conn(0x7fe0cc0d29f0 msgr2=0x7fe0cc0cc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:42.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.182+0000 7fe0d4037640 1 -- 172.21.15.5:0/3193037502 shutdown_connections 2024-05-20T18:34:42.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.182+0000 7fe0d4037640 1 -- 172.21.15.5:0/3193037502 wait complete. 2024-05-20T18:34:42.189 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-20T18:34:42.183+0000 7f6ed5e72640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x557137c66b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v8>,<0x1/dir v15>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T18:34:42.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:34:42.195 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-20T18:34:42.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.310+0000 7f04dcd77640 1 Processor -- start 2024-05-20T18:34:42.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.311+0000 7f04dcd77640 1 -- start start 2024-05-20T18:34:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.312+0000 7f04dcd77640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f04d8162820 0x7f04d80b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.312+0000 7f04dcd77640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04d80b3e80 0x7f04d806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.312+0000 7f04dcd77640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04d806dd20 con 0x7f04d806d910 2024-05-20T18:34:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.312+0000 7f04dcd77640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f04d80d0630 con 0x7f04d8162820 2024-05-20T18:34:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.312+0000 7f04dcd77640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f04d80d39c0 con 0x7f04d80b3e80 2024-05-20T18:34:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.312+0000 7f04d659c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f04d8162820 0x7f04d80b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.312+0000 7f04d6d9d640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f04d806d910 0x7f04d80d27a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36520/0 (socket says 172.21.15.5:36520) 2024-05-20T18:34:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.312+0000 7f04d659c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f04d8162820 0x7f04d80b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38630/0 (socket says 172.21.15.5:38630) 2024-05-20T18:34:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.312+0000 7f04d6d9d640 1 -- 172.21.15.5:0/394020794 learned_addr learned my addr 172.21.15.5:0/394020794 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:34:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.313+0000 7f04d5d9b640 1 --2- 172.21.15.5:0/394020794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04d80b3e80 0x7f04d806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.313+0000 7f04d659c640 1 -- 172.21.15.5:0/394020794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04d80b3e80 msgr2=0x7f04d806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.313+0000 7f04d659c640 1 --2- 172.21.15.5:0/394020794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04d80b3e80 0x7f04d806d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.313+0000 7f04d659c640 1 -- 172.21.15.5:0/394020794 >> v1:172.21.15.5:6789/0 conn(0x7f04d806d910 legacy=0x7f04d80d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.313+0000 7f04d659c640 1 -- 172.21.15.5:0/394020794 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d80d3ca0 con 0x7f04d8162820 2024-05-20T18:34:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.313+0000 7f04d559a640 1 -- 172.21.15.5:0/394020794 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4003725497 0 0) 0x7f04d0001110 con 0x7f04d806d910 2024-05-20T18:34:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.313+0000 7f04d659c640 1 --2- 172.21.15.5:0/394020794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d8162820 0x7f04d80b3940 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009d90 tx=0x7f04c802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4db553a04071e10 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.314+0000 7f04d559a640 1 -- 172.21.15.5:0/394020794 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8007740 con 0x7f04d8162820 2024-05-20T18:34:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.314+0000 7f04d559a640 1 -- 172.21.15.5:0/394020794 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04c80078e0 con 0x7f04d8162820 2024-05-20T18:34:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.314+0000 7f04d559a640 1 -- 172.21.15.5:0/394020794 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8007cb0 con 0x7f04d8162820 2024-05-20T18:34:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.314+0000 7f04dcd77640 1 -- 172.21.15.5:0/394020794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d8162820 msgr2=0x7f04d80b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.314+0000 7f04dcd77640 1 --2- 172.21.15.5:0/394020794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d8162820 0x7f04d80b3940 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009d90 tx=0x7f04c802a660 comp rx=0 tx=0).stop 2024-05-20T18:34:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.314+0000 7f04dcd77640 1 -- 172.21.15.5:0/394020794 shutdown_connections 2024-05-20T18:34:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.314+0000 7f04dcd77640 1 --2- 172.21.15.5:0/394020794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04d80b3e80 0x7f04d806d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.314+0000 7f04dcd77640 1 --2- 172.21.15.5:0/394020794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d8162820 0x7f04d80b3940 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.314+0000 7f04dcd77640 1 -- 172.21.15.5:0/394020794 >> 172.21.15.5:0/394020794 conn(0x7f04d80d4bd0 msgr2=0x7f04d80d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.314+0000 7f04dcd77640 1 -- 172.21.15.5:0/394020794 shutdown_connections 2024-05-20T18:34:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.314+0000 7f04dcd77640 1 -- 172.21.15.5:0/394020794 wait complete. 2024-05-20T18:34:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.316+0000 7f04dcd77640 1 Processor -- start 2024-05-20T18:34:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.316+0000 7f04dcd77640 1 -- start start 2024-05-20T18:34:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.318+0000 7f04dcd77640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04d806d910 0x7f04d80cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.318+0000 7f04dcd77640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04d80b3e80 0x7f04d8153ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.318+0000 7f04dcd77640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d8162820 0x7f04d80ce020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.318+0000 7f04dcd77640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f04d80d0300 con 0x7f04d8162820 2024-05-20T18:34:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.318+0000 7f04dcd77640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f04d80d4260 con 0x7f04d806d910 2024-05-20T18:34:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.318+0000 7f04dcd77640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f04d80b91a0 con 0x7f04d80b3e80 2024-05-20T18:34:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.318+0000 7f04d6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d8162820 0x7f04d80ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.318+0000 7f04d6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d8162820 0x7f04d80ce020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38634/0 (socket says 172.21.15.5:38634) 2024-05-20T18:34:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.318+0000 7f04d6d9d640 1 -- 172.21.15.5:0/432642624 learned_addr learned my addr 172.21.15.5:0/432642624 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.318+0000 7f04d659c640 1 --2- 172.21.15.5:0/432642624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04d806d910 0x7f04d80cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.318+0000 7f04d5d9b640 1 --2- 172.21.15.5:0/432642624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04d80b3e80 0x7f04d8153ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.318+0000 7f04d6d9d640 1 -- 172.21.15.5:0/432642624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04d80b3e80 msgr2=0x7f04d8153ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.318+0000 7f04d6d9d640 1 --2- 172.21.15.5:0/432642624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04d80b3e80 0x7f04d8153ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.318+0000 7f04d6d9d640 1 -- 172.21.15.5:0/432642624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04d806d910 msgr2=0x7f04d80cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.318+0000 7f04d6d9d640 1 --2- 172.21.15.5:0/432642624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04d806d910 0x7f04d80cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.318+0000 7f04d6d9d640 1 -- 172.21.15.5:0/432642624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d81543e0 con 0x7f04d8162820 2024-05-20T18:34:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.318+0000 7f04d6d9d640 1 --2- 172.21.15.5:0/432642624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d8162820 0x7f04d80ce020 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f04d000c3e0 tx=0x7f04d000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.319+0000 7f04c6ffd640 1 -- 172.21.15.5:0/432642624 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d00074e0 con 0x7f04d8162820 2024-05-20T18:34:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.319+0000 7f04c6ffd640 1 -- 172.21.15.5:0/432642624 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04d00047e0 con 0x7f04d8162820 2024-05-20T18:34:42.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.319+0000 7f04dcd77640 1 -- 172.21.15.5:0/432642624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d815b120 con 0x7f04d8162820 2024-05-20T18:34:42.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.319+0000 7f04c6ffd640 1 -- 172.21.15.5:0/432642624 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0004bd0 con 0x7f04d8162820 2024-05-20T18:34:42.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.319+0000 7f04dcd77640 1 -- 172.21.15.5:0/432642624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d80b52d0 con 0x7f04d8162820 2024-05-20T18:34:42.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.321+0000 7f04c6ffd640 1 -- 172.21.15.5:0/432642624 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f04d0011070 con 0x7f04d8162820 2024-05-20T18:34:42.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.321+0000 7f04c6ffd640 1 --2- 172.21.15.5:0/432642624 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f04b806be30 0x7f04b806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.321+0000 7f04c6ffd640 1 -- 172.21.15.5:0/432642624 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7f04d008abb0 con 0x7f04d8162820 2024-05-20T18:34:42.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.321+0000 7f04d659c640 1 --2- 172.21.15.5:0/432642624 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f04b806be30 0x7f04b806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.322+0000 7f04d659c640 1 --2- 172.21.15.5:0/432642624 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f04b806be30 0x7f04b806e340 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f04c8034000 tx=0x7f04c802f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:42.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.322+0000 7f04e3c23740 1 Processor -- start 2024-05-20T18:34:42.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.322+0000 7f04e3c23740 1 -- start start 2024-05-20T18:34:42.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.324+0000 7f04e3c23740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562fa43f7390 0x562fa47c1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.324+0000 7f04e3c23740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fa43ce4d0 0x562fa47dc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.324+0000 7f04e3c23740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fa43d66b0 0x562fa47dcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.324+0000 7f04e3c23740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562fa46fbd00 con 0x562fa43d66b0 2024-05-20T18:34:42.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.324+0000 7f04e3c23740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562fa46a34c0 con 0x562fa43ce4d0 2024-05-20T18:34:42.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.324+0000 7f04e3c23740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x562fa46f7cc0 con 0x562fa43f7390 2024-05-20T18:34:42.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.324+0000 7f04d6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fa43d66b0 0x562fa47dcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.324+0000 7f04d6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fa43d66b0 0x562fa47dcdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38636/0 (socket says 172.21.15.5:38636) 2024-05-20T18:34:42.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.324+0000 7f04d6d9d640 1 -- 172.21.15.5:0/3270629562 learned_addr learned my addr 172.21.15.5:0/3270629562 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:42.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.324+0000 7f04d5d9b640 1 --2- 172.21.15.5:0/3270629562 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fa43ce4d0 0x562fa47dc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.324+0000 7f04d659c640 1 --2- 172.21.15.5:0/3270629562 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562fa43f7390 0x562fa47c1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.324+0000 7f04d6d9d640 1 -- 172.21.15.5:0/3270629562 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562fa43f7390 msgr2=0x562fa47c1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:42.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.324+0000 7f04d6d9d640 1 --2- 172.21.15.5:0/3270629562 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562fa43f7390 0x562fa47c1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.324+0000 7f04d6d9d640 1 -- 172.21.15.5:0/3270629562 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fa43ce4d0 msgr2=0x562fa47dc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:42.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.324+0000 7f04d6d9d640 1 --2- 172.21.15.5:0/3270629562 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fa43ce4d0 0x562fa47dc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.324+0000 7f04d6d9d640 1 -- 172.21.15.5:0/3270629562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fa44d97d0 con 0x562fa43d66b0 2024-05-20T18:34:42.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.324+0000 7f04d5d9b640 1 --2- 172.21.15.5:0/3270629562 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fa43ce4d0 0x562fa47dc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:34:42.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.324+0000 7f04d6d9d640 1 --2- 172.21.15.5:0/3270629562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fa43d66b0 0x562fa47dcdd0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f04d0088da0 tx=0x7f04d0058c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fdac988ea6127fe server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:42.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.324+0000 7f04b7fff640 1 -- 172.21.15.5:0/3270629562 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0061790 con 0x562fa43d66b0 2024-05-20T18:34:42.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.324+0000 7f04b7fff640 1 -- 172.21.15.5:0/3270629562 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04d0056f60 con 0x562fa43d66b0 2024-05-20T18:34:42.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.325+0000 7f04b7fff640 1 -- 172.21.15.5:0/3270629562 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d00552d0 con 0x562fa43d66b0 2024-05-20T18:34:42.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.325+0000 7f04e3c23740 1 -- 172.21.15.5:0/3270629562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fa43d66b0 msgr2=0x562fa47dcdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:42.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.325+0000 7f04e3c23740 1 --2- 172.21.15.5:0/3270629562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fa43d66b0 0x562fa47dcdd0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f04d0088da0 tx=0x7f04d0058c60 comp rx=0 tx=0).stop 2024-05-20T18:34:42.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.325+0000 7f04e3c23740 1 -- 172.21.15.5:0/3270629562 shutdown_connections 2024-05-20T18:34:42.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.325+0000 7f04e3c23740 1 --2- 172.21.15.5:0/3270629562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fa43d66b0 0x562fa47dcdd0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.325+0000 7f04e3c23740 1 --2- 172.21.15.5:0/3270629562 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fa43ce4d0 0x562fa47dc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.325+0000 7f04e3c23740 1 --2- 172.21.15.5:0/3270629562 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562fa43f7390 0x562fa47c1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.325+0000 7f04e3c23740 1 -- 172.21.15.5:0/3270629562 >> 172.21.15.5:0/3270629562 conn(0x562fa43eb4e0 msgr2=0x562fa47971c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:42.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.325+0000 7f04e3c23740 1 -- 172.21.15.5:0/3270629562 shutdown_connections 2024-05-20T18:34:42.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.325+0000 7f04e3c23740 1 -- 172.21.15.5:0/3270629562 wait complete. 2024-05-20T18:34:42.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.326+0000 7f04e3c23740 1 -- start start 2024-05-20T18:34:42.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.326+0000 7f04e3c23740 1 Processor -- start 2024-05-20T18:34:42.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.328+0000 7f04e3c23740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fa43ce4d0 0x562fa47bc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.328+0000 7f04e3c23740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562fa43d66b0 0x562fa47c5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.328+0000 7f04e3c23740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fa43f7390 0x562fa47eba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.328+0000 7f04e3c23740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562fa46b6b90 con 0x562fa43ce4d0 2024-05-20T18:34:42.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.328+0000 7f04e3c23740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562fa47ec5d0 con 0x562fa43f7390 2024-05-20T18:34:42.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.328+0000 7f04e3c23740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x562fa47c4150 con 0x562fa43d66b0 2024-05-20T18:34:42.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.328+0000 7f04e3c23740 10 client.0 _ll_register_callbacks cb 0x562fa46f2da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:34:42.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.328+0000 7f04d659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fa43ce4d0 0x562fa47bc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.328+0000 7f04d659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fa43ce4d0 0x562fa47bc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38642/0 (socket says 172.21.15.5:38642) 2024-05-20T18:34:42.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.328+0000 7f04d659c640 1 -- 172.21.15.5:0/900992198 learned_addr learned my addr 172.21.15.5:0/900992198 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:42.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.328+0000 7f04d6d9d640 1 --2- 172.21.15.5:0/900992198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fa43f7390 0x562fa47eba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.328+0000 7f04d5d9b640 1 --2- 172.21.15.5:0/900992198 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562fa43d66b0 0x562fa47c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.328+0000 7f04d659c640 1 -- 172.21.15.5:0/900992198 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562fa43d66b0 msgr2=0x562fa47c5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:42.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.328+0000 7f04d659c640 1 --2- 172.21.15.5:0/900992198 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562fa43d66b0 0x562fa47c5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.328+0000 7f04d659c640 1 -- 172.21.15.5:0/900992198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fa43f7390 msgr2=0x562fa47eba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:42.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.328+0000 7f04d659c640 1 --2- 172.21.15.5:0/900992198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fa43f7390 0x562fa47eba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.328+0000 7f04d659c640 1 -- 172.21.15.5:0/900992198 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fa440f3f0 con 0x562fa43ce4d0 2024-05-20T18:34:42.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.328+0000 7f04d6d9d640 1 --2- 172.21.15.5:0/900992198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fa43f7390 0x562fa47eba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:34:42.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.329+0000 7f04d659c640 1 --2- 172.21.15.5:0/900992198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fa43ce4d0 0x562fa47bc850 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f04c8000f80 tx=0x7f04c8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a9082eb00e7875 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:42.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.329+0000 7f04b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:34:42.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.329+0000 7f04b57fa640 1 -- 172.21.15.5:0/900992198 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8046ae0 con 0x562fa43ce4d0 2024-05-20T18:34:42.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.329+0000 7f04b57fa640 1 -- 172.21.15.5:0/900992198 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7f04c8046c80 con 0x562fa43ce4d0 2024-05-20T18:34:42.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.329+0000 7f04e3c23740 1 -- 172.21.15.5:0/900992198 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04d0056f60 con 0x562fa43ce4d0 2024-05-20T18:34:42.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.329+0000 7f04b57fa640 1 -- 172.21.15.5:0/900992198 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f04c803c420 con 0x562fa43ce4d0 2024-05-20T18:34:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.329+0000 7f04e3c23740 10 client.8079 fetch_fsmap learned FSMap version 73 2024-05-20T18:34:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.329+0000 7f04e3c23740 1 -- 172.21.15.5:0/900992198 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x562fa43cf0f0 con 0x562fa43ce4d0 2024-05-20T18:34:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.329+0000 7f04b57fa640 1 -- 172.21.15.5:0/900992198 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04d0056f60 con 0x562fa43ce4d0 2024-05-20T18:34:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.329+0000 7f04b57fa640 1 -- 172.21.15.5:0/900992198 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 73) ==== 3494+0+0 (secure 0 0 0) 0x7f04c8043040 con 0x562fa43ce4d0 2024-05-20T18:34:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.329+0000 7f04e3c23740 10 client.8079 fetch_fsmap finished waiting for FSMap version 73 2024-05-20T18:34:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.329+0000 7f04e3c23740 10 client.8079 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-20T18:34:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.329+0000 7f04e3c23740 20 client.8079 populate_metadata read hostname 'smithi005' 2024-05-20T18:34:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.329+0000 7f04e3c23740 1 --2- 172.21.15.5:0/900992198 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x562fa43b5f60 0x562fa47971c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.330+0000 7f04e3c23740 4 client.8079 mds_command: new command op to 7953 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:34:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.330+0000 7f04e3c23740 1 -- 172.21.15.5:0/900992198 --> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fa4709430 con 0x562fa43b5f60 2024-05-20T18:34:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.330+0000 7f04d5d9b640 1 --2- 172.21.15.5:0/900992198 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x562fa43b5f60 0x562fa47971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.330+0000 7f04b57fa640 10 client.8079 ms_handle_connect on v2:172.21.15.5:6836/3880006778 2024-05-20T18:34:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.330+0000 7f04d5d9b640 1 --2- 172.21.15.5:0/900992198 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x562fa43b5f60 0x562fa47971c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99d105d0ec680abc server_cookie=59f2bb2235ee2256 in_seq=0 out_seq=0 2024-05-20T18:34:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.331+0000 7f04b57fa640 1 -- 172.21.15.5:0/900992198 <== mds.0 v2:172.21.15.5:6836/3880006778 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562fa4709430 con 0x562fa43b5f60 2024-05-20T18:34:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.331+0000 7f04b57fa640 10 client.8079 handle_command_reply: tid=0 2024-05-20T18:34:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.331+0000 7f04e3c23740 1 client.8079 shutdown 2024-05-20T18:34:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.331+0000 7f04e3c23740 1 -- 172.21.15.5:0/900992198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fa43ce4d0 msgr2=0x562fa47bc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.331+0000 7f04e3c23740 1 --2- 172.21.15.5:0/900992198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fa43ce4d0 0x562fa47bc850 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f04c8000f80 tx=0x7f04c8004ab0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.331+0000 7f04e3c23740 1 -- 172.21.15.5:0/900992198 shutdown_connections 2024-05-20T18:34:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.331+0000 7f04e3c23740 1 --2- 172.21.15.5:0/900992198 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x562fa43b5f60 0x562fa47971c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.331+0000 7f04e3c23740 1 --2- 172.21.15.5:0/900992198 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fa43f7390 0x562fa47eba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.331+0000 7f04e3c23740 1 --2- 172.21.15.5:0/900992198 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562fa43d66b0 0x562fa47c5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.331+0000 7f04e3c23740 1 --2- 172.21.15.5:0/900992198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fa43ce4d0 0x562fa47bc850 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.331+0000 7f04b57fa640 0 client.8079 ms_handle_reset on v2:172.21.15.5:6836/3880006778 2024-05-20T18:34:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.331+0000 7f04e3c23740 1 -- 172.21.15.5:0/900992198 >> 172.21.15.5:0/900992198 conn(0x562fa43eb4e0 msgr2=0x562fa4795ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.331+0000 7f04e3c23740 1 -- 172.21.15.5:0/900992198 shutdown_connections 2024-05-20T18:34:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.331+0000 7f04e3c23740 1 -- 172.21.15.5:0/900992198 wait complete. 2024-05-20T18:34:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.331+0000 7f04e3c23740 20 client.8079 trim_cache size 0 max 16384 2024-05-20T18:34:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.350+0000 7f04e3c23740 1 Processor -- start 2024-05-20T18:34:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.350+0000 7f04e3c23740 1 -- start start 2024-05-20T18:34:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.352+0000 7f04e3c23740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fa43ce4d0 0x562fa48538c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.352+0000 7f04e3c23740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562fa43d66b0 0x562fa47bfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.352+0000 7f04e3c23740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fa43f7390 0x562fa47c00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.352+0000 7f04e3c23740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562fa4792080 con 0x562fa43f7390 2024-05-20T18:34:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.352+0000 7f04e3c23740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f04d82026d0 con 0x562fa43ce4d0 2024-05-20T18:34:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.352+0000 7f04e3c23740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f04d82016b0 con 0x562fa43d66b0 2024-05-20T18:34:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.352+0000 7f04d6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fa43f7390 0x562fa47c00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.352+0000 7f04d6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fa43f7390 0x562fa47c00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38646/0 (socket says 172.21.15.5:38646) 2024-05-20T18:34:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.352+0000 7f04d6d9d640 1 -- 172.21.15.5:0/1044808050 learned_addr learned my addr 172.21.15.5:0/1044808050 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.352+0000 7f04d6d9d640 1 -- 172.21.15.5:0/1044808050 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562fa43d66b0 msgr2=0x562fa47bfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.352+0000 7f04d6d9d640 1 --2- 172.21.15.5:0/1044808050 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562fa43d66b0 0x562fa47bfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.352+0000 7f04d659c640 1 --2- 172.21.15.5:0/1044808050 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fa43ce4d0 0x562fa48538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.352+0000 7f04d6d9d640 1 -- 172.21.15.5:0/1044808050 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fa43ce4d0 msgr2=0x562fa48538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.352+0000 7f04d6d9d640 1 --2- 172.21.15.5:0/1044808050 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fa43ce4d0 0x562fa48538c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.352+0000 7f04d6d9d640 1 -- 172.21.15.5:0/1044808050 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fa47ddc60 con 0x562fa43f7390 2024-05-20T18:34:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.352+0000 7f04d6d9d640 1 --2- 172.21.15.5:0/1044808050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fa43f7390 0x562fa47c00e0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f04d005b010 tx=0x7f04d0007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5408ef5adf803e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:42.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.352+0000 7f04b77fe640 1 -- 172.21.15.5:0/1044808050 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0056f60 con 0x562fa43f7390 2024-05-20T18:34:42.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.352+0000 7f04b77fe640 1 -- 172.21.15.5:0/1044808050 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04d0086020 con 0x562fa43f7390 2024-05-20T18:34:42.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.352+0000 7f04b77fe640 1 -- 172.21.15.5:0/1044808050 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d00580b0 con 0x562fa43f7390 2024-05-20T18:34:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.353+0000 7f04e3c23740 1 -- 172.21.15.5:0/1044808050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fa43f7390 msgr2=0x562fa47c00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.353+0000 7f04e3c23740 1 --2- 172.21.15.5:0/1044808050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fa43f7390 0x562fa47c00e0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f04d005b010 tx=0x7f04d0007f90 comp rx=0 tx=0).stop 2024-05-20T18:34:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.353+0000 7f04e3c23740 1 -- 172.21.15.5:0/1044808050 shutdown_connections 2024-05-20T18:34:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.353+0000 7f04e3c23740 1 --2- 172.21.15.5:0/1044808050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fa43f7390 0x562fa47c00e0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.353+0000 7f04e3c23740 1 --2- 172.21.15.5:0/1044808050 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562fa43d66b0 0x562fa47bfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.353+0000 7f04e3c23740 1 --2- 172.21.15.5:0/1044808050 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fa43ce4d0 0x562fa48538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.353+0000 7f04e3c23740 1 -- 172.21.15.5:0/1044808050 >> 172.21.15.5:0/1044808050 conn(0x562fa43eb4e0 msgr2=0x562fa47bc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.353+0000 7f04e3c23740 1 -- 172.21.15.5:0/1044808050 shutdown_connections 2024-05-20T18:34:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.353+0000 7f04e3c23740 1 -- 172.21.15.5:0/1044808050 wait complete. 2024-05-20T18:34:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.354+0000 7f04e3c23740 1 -- start start 2024-05-20T18:34:42.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.354+0000 7f04e3c23740 1 Processor -- start 2024-05-20T18:34:42.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.356+0000 7f04e3c23740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fa43ce4d0 0x562fa47bfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.356+0000 7f04e3c23740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fa43d66b0 0x562fa47c00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.356+0000 7f04e3c23740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562fa43f7390 0x562fa47c2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.356+0000 7f04e3c23740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f04d8201040 con 0x562fa43d66b0 2024-05-20T18:34:42.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.356+0000 7f04e3c23740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f04d8200a20 con 0x562fa43ce4d0 2024-05-20T18:34:42.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.356+0000 7f04e3c23740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x562fa47bba20 con 0x562fa43f7390 2024-05-20T18:34:42.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.356+0000 7f04e3c23740 10 client.0 _ll_register_callbacks cb 0x7f04c8009450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:34:42.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.356+0000 7f04d5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fa43d66b0 0x562fa47c00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.356+0000 7f04d5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fa43d66b0 0x562fa47c00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38650/0 (socket says 172.21.15.5:38650) 2024-05-20T18:34:42.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.356+0000 7f04d5d9b640 1 -- 172.21.15.5:0/3213943504 learned_addr learned my addr 172.21.15.5:0/3213943504 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:34:42.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.356+0000 7f04d659c640 1 --2- 172.21.15.5:0/3213943504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fa43ce4d0 0x562fa47bfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.356+0000 7f04d6d9d640 1 --2- 172.21.15.5:0/3213943504 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562fa43f7390 0x562fa47c2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.356+0000 7f04d5d9b640 1 -- 172.21.15.5:0/3213943504 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562fa43f7390 msgr2=0x562fa47c2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:42.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.356+0000 7f04d5d9b640 1 --2- 172.21.15.5:0/3213943504 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562fa43f7390 0x562fa47c2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.356+0000 7f04d5d9b640 1 -- 172.21.15.5:0/3213943504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fa43ce4d0 msgr2=0x562fa47bfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:34:42.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.356+0000 7f04d5d9b640 1 --2- 172.21.15.5:0/3213943504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fa43ce4d0 0x562fa47bfba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.356+0000 7f04d5d9b640 1 -- 172.21.15.5:0/3213943504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fa47f59b0 con 0x562fa43d66b0 2024-05-20T18:34:42.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.356+0000 7f04d659c640 1 --2- 172.21.15.5:0/3213943504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fa43ce4d0 0x562fa47bfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:34:42.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.356+0000 7f04d5d9b640 1 --2- 172.21.15.5:0/3213943504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fa43d66b0 0x562fa47c00e0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f04cc007a30 tx=0x7f04cc018820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecec78f7fe45a4d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:34:42.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.356+0000 7f04b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:34:42.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.356+0000 7f04b57fa640 1 -- 172.21.15.5:0/3213943504 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc01f070 con 0x562fa43d66b0 2024-05-20T18:34:42.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.357+0000 7f04b57fa640 1 -- 172.21.15.5:0/3213943504 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7f04cc004030 con 0x562fa43d66b0 2024-05-20T18:34:42.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.357+0000 7f04e3c23740 1 -- 172.21.15.5:0/3213943504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04d0086020 con 0x562fa43d66b0 2024-05-20T18:34:42.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.357+0000 7f04b57fa640 1 -- 172.21.15.5:0/3213943504 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f04cc02b050 con 0x562fa43d66b0 2024-05-20T18:34:42.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.357+0000 7f04e3c23740 10 client.8085 fetch_fsmap learned FSMap version 73 2024-05-20T18:34:42.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.357+0000 7f04e3c23740 1 -- 172.21.15.5:0/3213943504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x562fa47c4dd0 con 0x562fa43d66b0 2024-05-20T18:34:42.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.357+0000 7f04b57fa640 1 -- 172.21.15.5:0/3213943504 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04d0086020 con 0x562fa43d66b0 2024-05-20T18:34:42.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.357+0000 7f04b57fa640 1 -- 172.21.15.5:0/3213943504 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 73) ==== 3494+0+0 (secure 0 0 0) 0x7f04cc02e020 con 0x562fa43d66b0 2024-05-20T18:34:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.357+0000 7f04e3c23740 10 client.8085 fetch_fsmap finished waiting for FSMap version 73 2024-05-20T18:34:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.357+0000 7f04e3c23740 10 client.8085 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-20T18:34:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.357+0000 7f04e3c23740 20 client.8085 populate_metadata read hostname 'smithi005' 2024-05-20T18:34:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.357+0000 7f04e3c23740 1 --2- 172.21.15.5:0/3213943504 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x562fa47c5490 0x562fa47c2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:34:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.357+0000 7f04e3c23740 4 client.8085 mds_command: new command op to 7953 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T18:34:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.357+0000 7f04e3c23740 1 -- 172.21.15.5:0/3213943504 --> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562fa4703780 con 0x562fa47c5490 2024-05-20T18:34:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.357+0000 7f04d659c640 1 --2- 172.21.15.5:0/3213943504 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x562fa47c5490 0x562fa47c2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:34:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.358+0000 7f04b57fa640 10 client.8085 ms_handle_connect on v2:172.21.15.5:6836/3880006778 2024-05-20T18:34:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.358+0000 7f04d659c640 1 --2- 172.21.15.5:0/3213943504 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x562fa47c5490 0x562fa47c2fb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88cface45182fdc8 server_cookie=cf8bc6c405212609 in_seq=0 out_seq=0 2024-05-20T18:34:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.358+0000 7f04b57fa640 1 -- 172.21.15.5:0/3213943504 <== mds.0 v2:172.21.15.5:6836/3880006778 1 ==== command_reply(tid 0: 0 ) ==== 8+0+263 (crc 0 0 0) 0x562fa4703780 con 0x562fa47c5490 2024-05-20T18:34:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.358+0000 7f04b57fa640 10 client.8085 handle_command_reply: tid=0 2024-05-20T18:34:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.358+0000 7f04e3c23740 1 client.8085 shutdown 2024-05-20T18:34:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.358+0000 7f04e3c23740 1 -- 172.21.15.5:0/3213943504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fa43d66b0 msgr2=0x562fa47c00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.358+0000 7f04e3c23740 1 --2- 172.21.15.5:0/3213943504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fa43d66b0 0x562fa47c00e0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f04cc007a30 tx=0x7f04cc018820 comp rx=0 tx=0).stop 2024-05-20T18:34:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.358+0000 7f04e3c23740 1 -- 172.21.15.5:0/3213943504 shutdown_connections 2024-05-20T18:34:42.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.358+0000 7f04e3c23740 1 --2- 172.21.15.5:0/3213943504 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x562fa47c5490 0x562fa47c2fb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.358+0000 7f04e3c23740 1 --2- 172.21.15.5:0/3213943504 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562fa43f7390 0x562fa47c2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.358+0000 7f04e3c23740 1 --2- 172.21.15.5:0/3213943504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fa43d66b0 0x562fa47c00e0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.358+0000 7f04b57fa640 0 client.8085 ms_handle_reset on v2:172.21.15.5:6836/3880006778 2024-05-20T18:34:42.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.358+0000 7f04e3c23740 1 --2- 172.21.15.5:0/3213943504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562fa43ce4d0 0x562fa47bfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.358+0000 7f04e3c23740 1 -- 172.21.15.5:0/3213943504 >> 172.21.15.5:0/3213943504 conn(0x562fa43eb4e0 msgr2=0x562fa47d73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:42.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.358+0000 7f04e3c23740 1 -- 172.21.15.5:0/3213943504 shutdown_connections 2024-05-20T18:34:42.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.358+0000 7f04e3c23740 1 -- 172.21.15.5:0/3213943504 wait complete. 2024-05-20T18:34:42.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.358+0000 7f04e3c23740 20 client.8085 trim_cache size 0 max 16384 2024-05-20T18:34:42.364 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:34:42.364 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-20T18:34:42.365 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-20T18:34:42.365 INFO:teuthology.orchestra.run.smithi005.stdout: "a984b717-d51c-479b-85f3-4c57c856de5b": { 2024-05-20T18:34:42.365 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/dir", 2024-05-20T18:34:42.365 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "a984b717-d51c-479b-85f3-4c57c856de5b", 2024-05-20T18:34:42.365 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,repair" 2024-05-20T18:34:42.365 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:34:42.365 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:34:42.365 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:34:42.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.359+0000 7f04dcd77640 1 -- 172.21.15.5:0/432642624 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f04b806be30 msgr2=0x7f04b806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:42.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.359+0000 7f04dcd77640 1 --2- 172.21.15.5:0/432642624 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f04b806be30 0x7f04b806e340 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f04c8034000 tx=0x7f04c802f000 comp rx=0 tx=0).stop 2024-05-20T18:34:42.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.359+0000 7f04dcd77640 1 -- 172.21.15.5:0/432642624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d8162820 msgr2=0x7f04d80ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:34:42.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.359+0000 7f04dcd77640 1 --2- 172.21.15.5:0/432642624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d8162820 0x7f04d80ce020 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f04d000c3e0 tx=0x7f04d000c8a0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.359+0000 7f04dcd77640 1 -- 172.21.15.5:0/432642624 shutdown_connections 2024-05-20T18:34:42.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.359+0000 7f04dcd77640 1 --2- 172.21.15.5:0/432642624 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f04b806be30 0x7f04b806e340 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.359+0000 7f04dcd77640 1 --2- 172.21.15.5:0/432642624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d8162820 0x7f04d80ce020 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.359+0000 7f04dcd77640 1 --2- 172.21.15.5:0/432642624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04d80b3e80 0x7f04d8153ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.359+0000 7f04dcd77640 1 --2- 172.21.15.5:0/432642624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04d806d910 0x7f04d80cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:34:42.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.359+0000 7f04dcd77640 1 -- 172.21.15.5:0/432642624 >> 172.21.15.5:0/432642624 conn(0x7f04d80d4bd0 msgr2=0x7f04d80d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:34:42.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.359+0000 7f04dcd77640 1 -- 172.21.15.5:0/432642624 shutdown_connections 2024-05-20T18:34:42.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:34:42.359+0000 7f04dcd77640 1 -- 172.21.15.5:0/432642624 wait complete. 2024-05-20T18:34:42.371 INFO:tasks.cephfs.filesystem:scrub status for tag:a984b717-d51c-479b-85f3-4c57c856de5b - {'path': '/dir', 'tag': 'a984b717-d51c-479b-85f3-4c57c856de5b', 'options': 'recursive,repair'} 2024-05-20T18:35:11.757 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:35:11.761 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:35:12.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:35:12.373 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-20T18:35:12.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.503+0000 7f5eb6232640 1 Processor -- start 2024-05-20T18:35:12.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.503+0000 7f5eb6232640 1 -- start start 2024-05-20T18:35:12.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.504+0000 7f5eb6232640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5eb00b1e10 0x7f5eb015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:12.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.504+0000 7f5eb6232640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5eb0070850 0x7f5eb0070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:12.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.504+0000 7f5eb6232640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5eb00cd580 con 0x7f5eb015d530 2024-05-20T18:35:12.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.504+0000 7f5eb6232640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f5eb00ce190 con 0x7f5eb0070850 2024-05-20T18:35:12.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.504+0000 7f5eb6232640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f5eb00cd860 con 0x7f5eb00b1e10 2024-05-20T18:35:12.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.504+0000 7f5eaffff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5eb0070850 0x7f5eb0070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:12.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.504+0000 7f5eaffff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5eb0070850 0x7f5eb0070c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56402/0 (socket says 172.21.15.5:56402) 2024-05-20T18:35:12.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.504+0000 7f5eaffff640 1 -- 172.21.15.5:0/344081221 learned_addr learned my addr 172.21.15.5:0/344081221 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:12.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.505+0000 7f5eaeffd640 1 --2- 172.21.15.5:0/344081221 >> v2:172.21.15.63:3300/0 conn(0x7f5eb00b1e10 0x7f5eb015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:12.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.505+0000 7f5eaffff640 1 -- 172.21.15.5:0/344081221 >> v2:172.21.15.63:3300/0 conn(0x7f5eb00b1e10 msgr2=0x7f5eb015aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:12.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.505+0000 7f5eaffff640 1 --2- 172.21.15.5:0/344081221 >> v2:172.21.15.63:3300/0 conn(0x7f5eb00b1e10 0x7f5eb015aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.505+0000 7f5eaffff640 1 -- 172.21.15.5:0/344081221 >> v1:172.21.15.63:6789/0 conn(0x7f5eb015d530 legacy=0x7f5eb00b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.505+0000 7f5eaffff640 1 -- 172.21.15.5:0/344081221 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb00cdb40 con 0x7f5eb0070850 2024-05-20T18:35:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.506+0000 7f5eaffff640 1 --2- 172.21.15.5:0/344081221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5eb0070850 0x7f5eb0070c60 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00d3ad0 tx=0x7f5ea802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=998f5990be477ff9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.506+0000 7f5eae7fc640 1 -- 172.21.15.5:0/344081221 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8036070 con 0x7f5eb0070850 2024-05-20T18:35:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.506+0000 7f5eae7fc640 1 -- 172.21.15.5:0/344081221 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ea8031600 con 0x7f5eb0070850 2024-05-20T18:35:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.506+0000 7f5eae7fc640 1 -- 172.21.15.5:0/344081221 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea80319d0 con 0x7f5eb0070850 2024-05-20T18:35:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.506+0000 7f5eb6232640 1 -- 172.21.15.5:0/344081221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5eb0070850 msgr2=0x7f5eb0070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.506+0000 7f5eb6232640 1 --2- 172.21.15.5:0/344081221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5eb0070850 0x7f5eb0070c60 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00d3ad0 tx=0x7f5ea802a730 comp rx=0 tx=0).stop 2024-05-20T18:35:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.507+0000 7f5eb6232640 1 -- 172.21.15.5:0/344081221 shutdown_connections 2024-05-20T18:35:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.507+0000 7f5eb6232640 1 --2- 172.21.15.5:0/344081221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5eb0070850 0x7f5eb0070c60 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.507+0000 7f5eb6232640 1 --2- 172.21.15.5:0/344081221 >> v2:172.21.15.63:3300/0 conn(0x7f5eb00b1e10 0x7f5eb015aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.507+0000 7f5eb6232640 1 -- 172.21.15.5:0/344081221 >> 172.21.15.5:0/344081221 conn(0x7f5eb00d29d0 msgr2=0x7f5eb00d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.507+0000 7f5eb6232640 1 -- 172.21.15.5:0/344081221 shutdown_connections 2024-05-20T18:35:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.507+0000 7f5eb6232640 1 -- 172.21.15.5:0/344081221 wait complete. 2024-05-20T18:35:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.508+0000 7f5eb6232640 1 Processor -- start 2024-05-20T18:35:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.509+0000 7f5eb6232640 1 -- start start 2024-05-20T18:35:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.510+0000 7f5eb6232640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5eb0070850 0x7f5eb01f5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.510+0000 7f5eb6232640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5eb00b1e10 0x7f5eb00c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.510+0000 7f5eb6232640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5eb015d530 0x7f5eb015bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.510+0000 7f5eb6232640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb00711a0 con 0x7f5eb00b1e10 2024-05-20T18:35:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.510+0000 7f5eb6232640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb00cd230 con 0x7f5eb0070850 2024-05-20T18:35:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.510+0000 7f5eb6232640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5eb0071320 con 0x7f5eb015d530 2024-05-20T18:35:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.510+0000 7f5eaeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5eb00b1e10 0x7f5eb00c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.510+0000 7f5eaeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5eb00b1e10 0x7f5eb00c9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56416/0 (socket says 172.21.15.5:56416) 2024-05-20T18:35:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.510+0000 7f5eaeffd640 1 -- 172.21.15.5:0/1263041363 learned_addr learned my addr 172.21.15.5:0/1263041363 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.510+0000 7f5eaf7fe640 1 --2- 172.21.15.5:0/1263041363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5eb0070850 0x7f5eb01f5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.510+0000 7f5eaffff640 1 --2- 172.21.15.5:0/1263041363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5eb015d530 0x7f5eb015bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.510+0000 7f5eaeffd640 1 -- 172.21.15.5:0/1263041363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5eb015d530 msgr2=0x7f5eb015bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.510+0000 7f5eaeffd640 1 --2- 172.21.15.5:0/1263041363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5eb015d530 0x7f5eb015bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.510+0000 7f5eaeffd640 1 -- 172.21.15.5:0/1263041363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5eb0070850 msgr2=0x7f5eb01f5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.510+0000 7f5eaeffd640 1 --2- 172.21.15.5:0/1263041363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5eb0070850 0x7f5eb01f5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.511+0000 7f5eaeffd640 1 -- 172.21.15.5:0/1263041363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb00ca460 con 0x7f5eb00b1e10 2024-05-20T18:35:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.511+0000 7f5eaffff640 1 --2- 172.21.15.5:0/1263041363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5eb015d530 0x7f5eb015bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.511+0000 7f5eaeffd640 1 --2- 172.21.15.5:0/1263041363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5eb00b1e10 0x7f5eb00c9a80 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f5ea400e8a0 tx=0x7f5ea400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.511+0000 7f5e97fff640 1 -- 172.21.15.5:0/1263041363 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4017070 con 0x7f5eb00b1e10 2024-05-20T18:35:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.511+0000 7f5e97fff640 1 -- 172.21.15.5:0/1263041363 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ea400c940 con 0x7f5eb00b1e10 2024-05-20T18:35:12.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.511+0000 7f5eb6232640 1 -- 172.21.15.5:0/1263041363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb00ca740 con 0x7f5eb00b1e10 2024-05-20T18:35:12.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.511+0000 7f5e97fff640 1 -- 172.21.15.5:0/1263041363 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea400cd30 con 0x7f5eb00b1e10 2024-05-20T18:35:12.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.511+0000 7f5eb6232640 1 -- 172.21.15.5:0/1263041363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb00cad00 con 0x7f5eb00b1e10 2024-05-20T18:35:12.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.513+0000 7f5e97fff640 1 -- 172.21.15.5:0/1263041363 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5ea401a070 con 0x7f5eb00b1e10 2024-05-20T18:35:12.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.513+0000 7f5e97fff640 1 --2- 172.21.15.5:0/1263041363 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5e9c06be30 0x7f5e9c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:12.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.513+0000 7f5e97fff640 1 -- 172.21.15.5:0/1263041363 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7f5ea408b250 con 0x7f5eb00b1e10 2024-05-20T18:35:12.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.513+0000 7f5eaf7fe640 1 --2- 172.21.15.5:0/1263041363 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5e9c06be30 0x7f5e9c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:12.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.514+0000 7f5eaf7fe640 1 --2- 172.21.15.5:0/1263041363 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5e9c06be30 0x7f5e9c06e340 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00d29a0 tx=0x7f5ea0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:12.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.514+0000 7f5ebd1bb740 1 Processor -- start 2024-05-20T18:35:12.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.514+0000 7f5ebd1bb740 1 -- start start 2024-05-20T18:35:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.516+0000 7f5ebd1bb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f774404390 0x55f7747ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.516+0000 7f5ebd1bb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7743db4d0 0x55f7747e9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.516+0000 7f5ebd1bb740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f7743e36b0 0x55f7747e9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.516+0000 7f5ebd1bb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f774708d00 con 0x55f7743db4d0 2024-05-20T18:35:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.516+0000 7f5ebd1bb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f7746b04c0 con 0x55f774404390 2024-05-20T18:35:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.516+0000 7f5ebd1bb740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55f774704cc0 con 0x55f7743e36b0 2024-05-20T18:35:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.516+0000 7f5eaeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7743db4d0 0x55f7747e9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.516+0000 7f5eaeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7743db4d0 0x55f7747e9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56428/0 (socket says 172.21.15.5:56428) 2024-05-20T18:35:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.516+0000 7f5eaeffd640 1 -- 172.21.15.5:0/194689963 learned_addr learned my addr 172.21.15.5:0/194689963 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.516+0000 7f5eaf7fe640 1 --2- 172.21.15.5:0/194689963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f774404390 0x55f7747ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.516+0000 7f5eaffff640 1 --2- 172.21.15.5:0/194689963 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f7743e36b0 0x55f7747e9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.516+0000 7f5eaeffd640 1 -- 172.21.15.5:0/194689963 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f7743e36b0 msgr2=0x55f7747e9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.516+0000 7f5eaeffd640 1 --2- 172.21.15.5:0/194689963 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f7743e36b0 0x55f7747e9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.516+0000 7f5eaeffd640 1 -- 172.21.15.5:0/194689963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f774404390 msgr2=0x55f7747ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.516+0000 7f5eaeffd640 1 --2- 172.21.15.5:0/194689963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f774404390 0x55f7747ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.516+0000 7f5eaeffd640 1 -- 172.21.15.5:0/194689963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7744e67d0 con 0x55f7743db4d0 2024-05-20T18:35:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.516+0000 7f5eaeffd640 1 --2- 172.21.15.5:0/194689963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7743db4d0 0x55f7747e9890 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4061900 tx=0x7f5ea4059300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fafdde70f639f341 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.517+0000 7f5e957fa640 1 -- 172.21.15.5:0/194689963 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4021040 con 0x55f7743db4d0 2024-05-20T18:35:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.517+0000 7f5e957fa640 1 -- 172.21.15.5:0/194689963 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ea408b070 con 0x55f7743db4d0 2024-05-20T18:35:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.517+0000 7f5e957fa640 1 -- 172.21.15.5:0/194689963 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4061e30 con 0x55f7743db4d0 2024-05-20T18:35:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.517+0000 7f5ebd1bb740 1 -- 172.21.15.5:0/194689963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7743db4d0 msgr2=0x55f7747e9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.517+0000 7f5ebd1bb740 1 --2- 172.21.15.5:0/194689963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7743db4d0 0x55f7747e9890 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4061900 tx=0x7f5ea4059300 comp rx=0 tx=0).stop 2024-05-20T18:35:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.518+0000 7f5ebd1bb740 1 -- 172.21.15.5:0/194689963 shutdown_connections 2024-05-20T18:35:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.518+0000 7f5ebd1bb740 1 --2- 172.21.15.5:0/194689963 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f7743e36b0 0x55f7747e9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.518+0000 7f5ebd1bb740 1 --2- 172.21.15.5:0/194689963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7743db4d0 0x55f7747e9890 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.518+0000 7f5ebd1bb740 1 --2- 172.21.15.5:0/194689963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f774404390 0x55f7747ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.518+0000 7f5ebd1bb740 1 -- 172.21.15.5:0/194689963 >> 172.21.15.5:0/194689963 conn(0x55f7743f84e0 msgr2=0x55f7747a41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.518+0000 7f5ebd1bb740 1 -- 172.21.15.5:0/194689963 shutdown_connections 2024-05-20T18:35:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.518+0000 7f5ebd1bb740 1 -- 172.21.15.5:0/194689963 wait complete. 2024-05-20T18:35:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.519+0000 7f5ebd1bb740 1 -- start start 2024-05-20T18:35:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.519+0000 7f5ebd1bb740 1 Processor -- start 2024-05-20T18:35:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.520+0000 7f5ebd1bb740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f7743db4d0 0x55f7747d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.520+0000 7f5ebd1bb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7743e36b0 0x55f7747d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.520+0000 7f5ebd1bb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f774404390 0x55f7747f8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.520+0000 7f5ebd1bb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f7746c3b90 con 0x55f7743e36b0 2024-05-20T18:35:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.520+0000 7f5ebd1bb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f7747f97e0 con 0x55f774404390 2024-05-20T18:35:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.520+0000 7f5ebd1bb740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55f7747d1810 con 0x55f7743db4d0 2024-05-20T18:35:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.520+0000 7f5ebd1bb740 10 client.0 _ll_register_callbacks cb 0x55f7746ffda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:35:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.520+0000 7f5eaeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7743e36b0 0x55f7747d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.520+0000 7f5eaeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7743e36b0 0x55f7747d0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56434/0 (socket says 172.21.15.5:56434) 2024-05-20T18:35:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.520+0000 7f5eaeffd640 1 -- 172.21.15.5:0/902600693 learned_addr learned my addr 172.21.15.5:0/902600693 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.520+0000 7f5eaf7fe640 1 --2- 172.21.15.5:0/902600693 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f7743db4d0 0x55f7747d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.520+0000 7f5eaffff640 1 --2- 172.21.15.5:0/902600693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f774404390 0x55f7747f8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.520+0000 7f5eaeffd640 1 -- 172.21.15.5:0/902600693 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f7743db4d0 msgr2=0x55f7747d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.520+0000 7f5eaeffd640 1 --2- 172.21.15.5:0/902600693 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f7743db4d0 0x55f7747d2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.520+0000 7f5eaeffd640 1 -- 172.21.15.5:0/902600693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f774404390 msgr2=0x55f7747f8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.520+0000 7f5eaeffd640 1 --2- 172.21.15.5:0/902600693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f774404390 0x55f7747f8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.520+0000 7f5eaeffd640 1 -- 172.21.15.5:0/902600693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f77441c3f0 con 0x55f7743e36b0 2024-05-20T18:35:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.520+0000 7f5eaffff640 1 --2- 172.21.15.5:0/902600693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f774404390 0x55f7747f8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.521+0000 7f5eaeffd640 1 --2- 172.21.15.5:0/902600693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7743e36b0 0x55f7747d0410 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4055760 tx=0x7f5ea4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2982af23df66ba49 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.521+0000 7f5e8e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:35:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.521+0000 7f5e8e7fc640 1 -- 172.21.15.5:0/902600693 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4021040 con 0x55f7743e36b0 2024-05-20T18:35:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.521+0000 7f5e8e7fc640 1 -- 172.21.15.5:0/902600693 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7f5ea4058750 con 0x55f7743e36b0 2024-05-20T18:35:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.521+0000 7f5ebd1bb740 1 -- 172.21.15.5:0/902600693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ea408b070 con 0x55f7743e36b0 2024-05-20T18:35:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.521+0000 7f5e8e7fc640 1 -- 172.21.15.5:0/902600693 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f5ea4058950 con 0x55f7743e36b0 2024-05-20T18:35:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.521+0000 7f5ebd1bb740 10 client.8097 fetch_fsmap learned FSMap version 73 2024-05-20T18:35:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.521+0000 7f5ebd1bb740 1 -- 172.21.15.5:0/902600693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x55f7743dc0f0 con 0x55f7743e36b0 2024-05-20T18:35:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.521+0000 7f5e8e7fc640 1 -- 172.21.15.5:0/902600693 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ea408b070 con 0x55f7743e36b0 2024-05-20T18:35:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.521+0000 7f5e8e7fc640 1 -- 172.21.15.5:0/902600693 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 73) ==== 3494+0+0 (secure 0 0 0) 0x7f5ea40781a0 con 0x55f7743e36b0 2024-05-20T18:35:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.521+0000 7f5ebd1bb740 10 client.8097 fetch_fsmap finished waiting for FSMap version 73 2024-05-20T18:35:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.521+0000 7f5ebd1bb740 10 client.8097 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-20T18:35:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.521+0000 7f5ebd1bb740 20 client.8097 populate_metadata read hostname 'smithi005' 2024-05-20T18:35:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.521+0000 7f5ebd1bb740 1 --2- 172.21.15.5:0/902600693 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x55f7743c2f60 0x55f7747ed7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.521+0000 7f5ebd1bb740 4 client.8097 mds_command: new command op to 7953 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:35:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.521+0000 7f5ebd1bb740 1 -- 172.21.15.5:0/902600693 --> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f774716430 con 0x55f7743c2f60 2024-05-20T18:35:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.521+0000 7f5eaf7fe640 1 --2- 172.21.15.5:0/902600693 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x55f7743c2f60 0x55f7747ed7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.522+0000 7f5eaf7fe640 1 --2- 172.21.15.5:0/902600693 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x55f7743c2f60 0x55f7747ed7f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d63b71b4586c2b8 server_cookie=ad8c338d68eb0e9c in_seq=0 out_seq=0 2024-05-20T18:35:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.522+0000 7f5e8e7fc640 10 client.8097 ms_handle_connect on v2:172.21.15.5:6836/3880006778 2024-05-20T18:35:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.523+0000 7f5e8e7fc640 1 -- 172.21.15.5:0/902600693 <== mds.0 v2:172.21.15.5:6836/3880006778 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f774716430 con 0x55f7743c2f60 2024-05-20T18:35:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.523+0000 7f5e8e7fc640 10 client.8097 handle_command_reply: tid=0 2024-05-20T18:35:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.523+0000 7f5ebd1bb740 1 client.8097 shutdown 2024-05-20T18:35:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.523+0000 7f5ebd1bb740 1 -- 172.21.15.5:0/902600693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7743e36b0 msgr2=0x55f7747d0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.523+0000 7f5ebd1bb740 1 --2- 172.21.15.5:0/902600693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7743e36b0 0x55f7747d0410 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4055760 tx=0x7f5ea4000bc0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.523+0000 7f5ebd1bb740 1 -- 172.21.15.5:0/902600693 shutdown_connections 2024-05-20T18:35:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.523+0000 7f5ebd1bb740 1 --2- 172.21.15.5:0/902600693 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x55f7743c2f60 0x55f7747ed7f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.523+0000 7f5ebd1bb740 1 --2- 172.21.15.5:0/902600693 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f774404390 0x55f7747f8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.523+0000 7f5e8e7fc640 0 client.8097 ms_handle_reset on v2:172.21.15.5:6836/3880006778 2024-05-20T18:35:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.523+0000 7f5ebd1bb740 1 --2- 172.21.15.5:0/902600693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7743e36b0 0x55f7747d0410 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.523+0000 7f5ebd1bb740 1 --2- 172.21.15.5:0/902600693 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f7743db4d0 0x55f7747d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.523+0000 7f5ebd1bb740 1 -- 172.21.15.5:0/902600693 >> 172.21.15.5:0/902600693 conn(0x55f7743f84e0 msgr2=0x55f7747ceb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.523+0000 7f5ebd1bb740 1 -- 172.21.15.5:0/902600693 shutdown_connections 2024-05-20T18:35:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.523+0000 7f5ebd1bb740 1 -- 172.21.15.5:0/902600693 wait complete. 2024-05-20T18:35:12.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.523+0000 7f5ebd1bb740 20 client.8097 trim_cache size 0 max 16384 2024-05-20T18:35:12.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.541+0000 7f5ebd1bb740 1 Processor -- start 2024-05-20T18:35:12.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.542+0000 7f5ebd1bb740 1 -- start start 2024-05-20T18:35:12.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.543+0000 7f5ebd1bb740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f7743db4d0 0x55f7747d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:12.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.543+0000 7f5ebd1bb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7743e36b0 0x55f7747ede80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:12.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.543+0000 7f5ebd1bb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f774404390 0x55f7747ebc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:12.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.543+0000 7f5ebd1bb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb02005d0 con 0x55f774404390 2024-05-20T18:35:12.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.543+0000 7f5ebd1bb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb01ff5d0 con 0x55f7743e36b0 2024-05-20T18:35:12.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.543+0000 7f5ebd1bb740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5eb01fe930 con 0x55f7743db4d0 2024-05-20T18:35:12.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.543+0000 7f5eaffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f774404390 0x55f7747ebc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:12.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.543+0000 7f5eaffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f774404390 0x55f7747ebc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56446/0 (socket says 172.21.15.5:56446) 2024-05-20T18:35:12.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.543+0000 7f5eaffff640 1 -- 172.21.15.5:0/1963000511 learned_addr learned my addr 172.21.15.5:0/1963000511 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:12.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.543+0000 7f5eaf7fe640 1 --2- 172.21.15.5:0/1963000511 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f7743db4d0 0x55f7747d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.543+0000 7f5eaeffd640 1 --2- 172.21.15.5:0/1963000511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7743e36b0 0x55f7747ede80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.543+0000 7f5eaffff640 1 -- 172.21.15.5:0/1963000511 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f7743db4d0 msgr2=0x55f7747d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.543+0000 7f5eaffff640 1 --2- 172.21.15.5:0/1963000511 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f7743db4d0 0x55f7747d03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.543+0000 7f5eaffff640 1 -- 172.21.15.5:0/1963000511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7743e36b0 msgr2=0x55f7747ede80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.543+0000 7f5eaffff640 1 --2- 172.21.15.5:0/1963000511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7743e36b0 0x55f7747ede80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.543+0000 7f5eaffff640 1 -- 172.21.15.5:0/1963000511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f77479efe0 con 0x55f774404390 2024-05-20T18:35:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.543+0000 7f5eaf7fe640 1 --2- 172.21.15.5:0/1963000511 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f7743db4d0 0x55f7747d03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.544+0000 7f5eaffff640 1 --2- 172.21.15.5:0/1963000511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f774404390 0x55f7747ebc30 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f5ea802ad20 tx=0x7f5ea802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=553c44627a6b8a00 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.544+0000 7f5e94ff9640 1 -- 172.21.15.5:0/1963000511 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8041070 con 0x55f774404390 2024-05-20T18:35:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.544+0000 7f5e94ff9640 1 -- 172.21.15.5:0/1963000511 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ea803d630 con 0x55f774404390 2024-05-20T18:35:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.544+0000 7f5e94ff9640 1 -- 172.21.15.5:0/1963000511 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8036040 con 0x55f774404390 2024-05-20T18:35:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.545+0000 7f5ebd1bb740 1 -- 172.21.15.5:0/1963000511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f774404390 msgr2=0x55f7747ebc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.545+0000 7f5ebd1bb740 1 --2- 172.21.15.5:0/1963000511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f774404390 0x55f7747ebc30 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f5ea802ad20 tx=0x7f5ea802e000 comp rx=0 tx=0).stop 2024-05-20T18:35:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.545+0000 7f5ebd1bb740 1 -- 172.21.15.5:0/1963000511 shutdown_connections 2024-05-20T18:35:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.545+0000 7f5ebd1bb740 1 --2- 172.21.15.5:0/1963000511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f774404390 0x55f7747ebc30 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.545+0000 7f5ebd1bb740 1 --2- 172.21.15.5:0/1963000511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7743e36b0 0x55f7747ede80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.545+0000 7f5ebd1bb740 1 --2- 172.21.15.5:0/1963000511 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f7743db4d0 0x55f7747d03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.545+0000 7f5ebd1bb740 1 -- 172.21.15.5:0/1963000511 >> 172.21.15.5:0/1963000511 conn(0x55f7743f84e0 msgr2=0x55f77480c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.545+0000 7f5ebd1bb740 1 -- 172.21.15.5:0/1963000511 shutdown_connections 2024-05-20T18:35:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.545+0000 7f5ebd1bb740 1 -- 172.21.15.5:0/1963000511 wait complete. 2024-05-20T18:35:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.546+0000 7f5ebd1bb740 1 -- start start 2024-05-20T18:35:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.546+0000 7f5ebd1bb740 1 Processor -- start 2024-05-20T18:35:12.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.547+0000 7f5ebd1bb740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f7743db4d0 0x55f7747e3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:12.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.547+0000 7f5ebd1bb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7743e36b0 0x55f7747c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.547+0000 7f5ebd1bb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f774404390 0x55f7747a4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.547+0000 7f5ebd1bb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f7746bee80 con 0x55f774404390 2024-05-20T18:35:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.547+0000 7f5ebd1bb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb01fef80 con 0x55f7743e36b0 2024-05-20T18:35:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.547+0000 7f5ebd1bb740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55f77479e880 con 0x55f7743db4d0 2024-05-20T18:35:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.547+0000 7f5ebd1bb740 10 client.0 _ll_register_callbacks cb 0x7f5ea00013a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:35:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.547+0000 7f5eaffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f774404390 0x55f7747a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.547+0000 7f5eaffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f774404390 0x55f7747a4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56452/0 (socket says 172.21.15.5:56452) 2024-05-20T18:35:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.547+0000 7f5eaffff640 1 -- 172.21.15.5:0/1772544120 learned_addr learned my addr 172.21.15.5:0/1772544120 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.547+0000 7f5eaf7fe640 1 --2- 172.21.15.5:0/1772544120 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f7743db4d0 0x55f7747e3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.547+0000 7f5eaeffd640 1 --2- 172.21.15.5:0/1772544120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7743e36b0 0x55f7747c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.547+0000 7f5eaffff640 1 -- 172.21.15.5:0/1772544120 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f7743db4d0 msgr2=0x55f7747e3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.547+0000 7f5eaffff640 1 --2- 172.21.15.5:0/1772544120 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f7743db4d0 0x55f7747e3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.547+0000 7f5eaffff640 1 -- 172.21.15.5:0/1772544120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7743e36b0 msgr2=0x55f7747c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.547+0000 7f5eaffff640 1 --2- 172.21.15.5:0/1772544120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7743e36b0 0x55f7747c94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.547+0000 7f5eaffff640 1 -- 172.21.15.5:0/1772544120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7747e4360 con 0x55f774404390 2024-05-20T18:35:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.548+0000 7f5eaffff640 1 --2- 172.21.15.5:0/1772544120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f774404390 0x55f7747a4620 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f5ea802a770 tx=0x7f5ea8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5457c2fa4a25e906 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.548+0000 7f5e8e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:35:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.548+0000 7f5e8e7fc640 1 -- 172.21.15.5:0/1772544120 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8036050 con 0x55f774404390 2024-05-20T18:35:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.548+0000 7f5e8e7fc640 1 -- 172.21.15.5:0/1772544120 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7f5ea803d9f0 con 0x55f774404390 2024-05-20T18:35:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.548+0000 7f5ebd1bb740 1 -- 172.21.15.5:0/1772544120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ea803d630 con 0x55f774404390 2024-05-20T18:35:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.548+0000 7f5e8e7fc640 1 -- 172.21.15.5:0/1772544120 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f5ea8057050 con 0x55f774404390 2024-05-20T18:35:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.548+0000 7f5ebd1bb740 10 client.8103 fetch_fsmap learned FSMap version 73 2024-05-20T18:35:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.548+0000 7f5ebd1bb740 1 -- 172.21.15.5:0/1772544120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x55f7747e4570 con 0x55f774404390 2024-05-20T18:35:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.548+0000 7f5e8e7fc640 1 -- 172.21.15.5:0/1772544120 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ea803d630 con 0x55f774404390 2024-05-20T18:35:12.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.548+0000 7f5e8e7fc640 1 -- 172.21.15.5:0/1772544120 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 73) ==== 3494+0+0 (secure 0 0 0) 0x7f5ea805a020 con 0x55f774404390 2024-05-20T18:35:12.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.548+0000 7f5ebd1bb740 10 client.8103 fetch_fsmap finished waiting for FSMap version 73 2024-05-20T18:35:12.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.548+0000 7f5ebd1bb740 10 client.8103 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-20T18:35:12.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.548+0000 7f5ebd1bb740 20 client.8103 populate_metadata read hostname 'smithi005' 2024-05-20T18:35:12.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.548+0000 7f5ebd1bb740 1 --2- 172.21.15.5:0/1772544120 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x55f774866260 0x55f7747d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:12.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.548+0000 7f5ebd1bb740 4 client.8103 mds_command: new command op to 7953 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T18:35:12.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.548+0000 7f5ebd1bb740 1 -- 172.21.15.5:0/1772544120 --> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f774710780 con 0x55f774866260 2024-05-20T18:35:12.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.548+0000 7f5eaf7fe640 1 --2- 172.21.15.5:0/1772544120 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x55f774866260 0x55f7747d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:12.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.549+0000 7f5e8e7fc640 10 client.8103 ms_handle_connect on v2:172.21.15.5:6836/3880006778 2024-05-20T18:35:12.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.549+0000 7f5eaf7fe640 1 --2- 172.21.15.5:0/1772544120 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x55f774866260 0x55f7747d2020 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4fbdf35d40d12b server_cookie=fea227c8153ef0c9 in_seq=0 out_seq=0 2024-05-20T18:35:12.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.549+0000 7f5e8e7fc640 1 -- 172.21.15.5:0/1772544120 <== mds.0 v2:172.21.15.5:6836/3880006778 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f774710780 con 0x55f774866260 2024-05-20T18:35:12.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.549+0000 7f5e8e7fc640 10 client.8103 handle_command_reply: tid=0 2024-05-20T18:35:12.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.549+0000 7f5ebd1bb740 1 client.8103 shutdown 2024-05-20T18:35:12.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.549+0000 7f5ebd1bb740 1 -- 172.21.15.5:0/1772544120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f774404390 msgr2=0x55f7747a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:12.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.549+0000 7f5ebd1bb740 1 --2- 172.21.15.5:0/1772544120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f774404390 0x55f7747a4620 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f5ea802a770 tx=0x7f5ea8004ab0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.549+0000 7f5ebd1bb740 1 -- 172.21.15.5:0/1772544120 shutdown_connections 2024-05-20T18:35:12.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.549+0000 7f5ebd1bb740 1 --2- 172.21.15.5:0/1772544120 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x55f774866260 0x55f7747d2020 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.549+0000 7f5ebd1bb740 1 --2- 172.21.15.5:0/1772544120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f774404390 0x55f7747a4620 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.549+0000 7f5ebd1bb740 1 --2- 172.21.15.5:0/1772544120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f7743e36b0 0x55f7747c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.549+0000 7f5e8e7fc640 0 client.8103 ms_handle_reset on v2:172.21.15.5:6836/3880006778 2024-05-20T18:35:12.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.549+0000 7f5ebd1bb740 1 --2- 172.21.15.5:0/1772544120 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f7743db4d0 0x55f7747e3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.549+0000 7f5ebd1bb740 1 -- 172.21.15.5:0/1772544120 >> 172.21.15.5:0/1772544120 conn(0x55f7743f84e0 msgr2=0x55f7747e5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:12.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.549+0000 7f5ebd1bb740 1 -- 172.21.15.5:0/1772544120 shutdown_connections 2024-05-20T18:35:12.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.549+0000 7f5ebd1bb740 1 -- 172.21.15.5:0/1772544120 wait complete. 2024-05-20T18:35:12.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.549+0000 7f5ebd1bb740 20 client.8103 trim_cache size 0 max 16384 2024-05-20T18:35:12.558 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:35:12.558 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-20T18:35:12.558 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-20T18:35:12.558 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:35:12.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.550+0000 7f5eb6232640 1 -- 172.21.15.5:0/1263041363 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5e9c06be30 msgr2=0x7f5e9c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:12.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.550+0000 7f5eb6232640 1 --2- 172.21.15.5:0/1263041363 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5e9c06be30 0x7f5e9c06e340 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00d29a0 tx=0x7f5ea0009040 comp rx=0 tx=0).stop 2024-05-20T18:35:12.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.550+0000 7f5eb6232640 1 -- 172.21.15.5:0/1263041363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5eb00b1e10 msgr2=0x7f5eb00c9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:12.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.550+0000 7f5eb6232640 1 --2- 172.21.15.5:0/1263041363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5eb00b1e10 0x7f5eb00c9a80 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f5ea400e8a0 tx=0x7f5ea400ed60 comp rx=0 tx=0).stop 2024-05-20T18:35:12.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.550+0000 7f5eb6232640 1 -- 172.21.15.5:0/1263041363 shutdown_connections 2024-05-20T18:35:12.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.550+0000 7f5eb6232640 1 --2- 172.21.15.5:0/1263041363 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5e9c06be30 0x7f5e9c06e340 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.550+0000 7f5eb6232640 1 --2- 172.21.15.5:0/1263041363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5eb015d530 0x7f5eb015bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.550+0000 7f5eb6232640 1 --2- 172.21.15.5:0/1263041363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5eb00b1e10 0x7f5eb00c9a80 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.550+0000 7f5eb6232640 1 --2- 172.21.15.5:0/1263041363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5eb0070850 0x7f5eb01f5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.550+0000 7f5eb6232640 1 -- 172.21.15.5:0/1263041363 >> 172.21.15.5:0/1263041363 conn(0x7f5eb00d29d0 msgr2=0x7f5eb00b7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.550+0000 7f5eb6232640 1 -- 172.21.15.5:0/1263041363 shutdown_connections 2024-05-20T18:35:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.550+0000 7f5eb6232640 1 -- 172.21.15.5:0/1263041363 wait complete. 2024-05-20T18:35:12.562 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T18:35:12.563 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:35:12.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.676+0000 7fe1f9f7e640 1 Processor -- start 2024-05-20T18:35:12.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.676+0000 7fe1f9f7e640 1 -- start start 2024-05-20T18:35:12.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.677+0000 7fe1f9f7e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1f415d530 0x7fe1f415aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:12.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.677+0000 7fe1f9f7e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe1f4070850 0x7fe1f4070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:12.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.677+0000 7fe1f9f7e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1f40cd580 con 0x7fe1f40b1700 2024-05-20T18:35:12.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.677+0000 7fe1f9f7e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe1f40ce190 con 0x7fe1f4070850 2024-05-20T18:35:12.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.677+0000 7fe1f9f7e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe1f40cd860 con 0x7fe1f415d530 2024-05-20T18:35:12.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.678+0000 7fe1f3fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe1f4070850 0x7fe1f4070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:12.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.678+0000 7fe1f2ffd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe1f40b1700 0x7fe1f40b1b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39964/0 (socket says 172.21.15.5:39964) 2024-05-20T18:35:12.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.678+0000 7fe1f2ffd640 1 -- 172.21.15.5:0/1947866474 learned_addr learned my addr 172.21.15.5:0/1947866474 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:12.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.678+0000 7fe1f37fe640 1 --2- 172.21.15.5:0/1947866474 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1f415d530 0x7fe1f415aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:12.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.678+0000 7fe1f3fff640 1 -- 172.21.15.5:0/1947866474 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1f415d530 msgr2=0x7fe1f415aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:12.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.678+0000 7fe1f3fff640 1 --2- 172.21.15.5:0/1947866474 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1f415d530 0x7fe1f415aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.678+0000 7fe1f3fff640 1 -- 172.21.15.5:0/1947866474 >> v1:172.21.15.5:6789/0 conn(0x7fe1f40b1700 legacy=0x7fe1f40b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:12.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.678+0000 7fe1f27fc640 1 -- 172.21.15.5:0/1947866474 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 727147376 0 0) 0x7fe1f40cd580 con 0x7fe1f40b1700 2024-05-20T18:35:12.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.678+0000 7fe1f3fff640 1 -- 172.21.15.5:0/1947866474 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f40cdb40 con 0x7fe1f4070850 2024-05-20T18:35:12.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.679+0000 7fe1f3fff640 1 --2- 172.21.15.5:0/1947866474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1f4070850 0x7fe1f4070c60 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec009d90 tx=0x7fe1ec02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f04a9ad9a44eea server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:12.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.679+0000 7fe1f27fc640 1 -- 172.21.15.5:0/1947866474 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec007740 con 0x7fe1f4070850 2024-05-20T18:35:12.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.679+0000 7fe1f27fc640 1 -- 172.21.15.5:0/1947866474 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1ec0078e0 con 0x7fe1f4070850 2024-05-20T18:35:12.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.679+0000 7fe1f27fc640 1 -- 172.21.15.5:0/1947866474 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec007cb0 con 0x7fe1f4070850 2024-05-20T18:35:12.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.679+0000 7fe1f9f7e640 1 -- 172.21.15.5:0/1947866474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1f4070850 msgr2=0x7fe1f4070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:12.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.679+0000 7fe1f9f7e640 1 --2- 172.21.15.5:0/1947866474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1f4070850 0x7fe1f4070c60 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec009d90 tx=0x7fe1ec02a660 comp rx=0 tx=0).stop 2024-05-20T18:35:12.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.679+0000 7fe1f9f7e640 1 -- 172.21.15.5:0/1947866474 shutdown_connections 2024-05-20T18:35:12.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.679+0000 7fe1f9f7e640 1 --2- 172.21.15.5:0/1947866474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1f4070850 0x7fe1f4070c60 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.679+0000 7fe1f9f7e640 1 --2- 172.21.15.5:0/1947866474 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1f415d530 0x7fe1f415aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.680+0000 7fe1f9f7e640 1 -- 172.21.15.5:0/1947866474 >> 172.21.15.5:0/1947866474 conn(0x7fe1f40d29d0 msgr2=0x7fe1f40d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:12.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.680+0000 7fe1f9f7e640 1 -- 172.21.15.5:0/1947866474 shutdown_connections 2024-05-20T18:35:12.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.680+0000 7fe1f9f7e640 1 -- 172.21.15.5:0/1947866474 wait complete. 2024-05-20T18:35:12.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.681+0000 7fe1f9f7e640 1 Processor -- start 2024-05-20T18:35:12.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.682+0000 7fe1f9f7e640 1 -- start start 2024-05-20T18:35:12.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.683+0000 7fe1f9f7e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1f4070850 0x7fe1f40b0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:12.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.683+0000 7fe1f9f7e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1f40b1700 0x7fe1f40ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:12.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.683+0000 7fe1f9f7e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1f415d530 0x7fe1f414bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:12.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.683+0000 7fe1f9f7e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f40711a0 con 0x7fe1f415d530 2024-05-20T18:35:12.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.683+0000 7fe1f9f7e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f40cd230 con 0x7fe1f40b1700 2024-05-20T18:35:12.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.683+0000 7fe1f9f7e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe1f4071320 con 0x7fe1f4070850 2024-05-20T18:35:12.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.683+0000 7fe1f3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1f415d530 0x7fe1f414bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:12.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.683+0000 7fe1f3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1f415d530 0x7fe1f414bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56476/0 (socket says 172.21.15.5:56476) 2024-05-20T18:35:12.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.683+0000 7fe1f3fff640 1 -- 172.21.15.5:0/2889640004 learned_addr learned my addr 172.21.15.5:0/2889640004 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:12.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.683+0000 7fe1f37fe640 1 --2- 172.21.15.5:0/2889640004 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1f4070850 0x7fe1f40b0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:12.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.683+0000 7fe1f2ffd640 1 --2- 172.21.15.5:0/2889640004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1f40b1700 0x7fe1f40ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:12.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.683+0000 7fe1f3fff640 1 -- 172.21.15.5:0/2889640004 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1f4070850 msgr2=0x7fe1f40b0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:12.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.683+0000 7fe1f3fff640 1 --2- 172.21.15.5:0/2889640004 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1f4070850 0x7fe1f40b0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.683+0000 7fe1f3fff640 1 -- 172.21.15.5:0/2889640004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1f40b1700 msgr2=0x7fe1f40ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:12.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.683+0000 7fe1f3fff640 1 --2- 172.21.15.5:0/2889640004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1f40b1700 0x7fe1f40ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.683+0000 7fe1f3fff640 1 -- 172.21.15.5:0/2889640004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f40cb5b0 con 0x7fe1f415d530 2024-05-20T18:35:12.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.684+0000 7fe1f3fff640 1 --2- 172.21.15.5:0/2889640004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1f415d530 0x7fe1f414bb00 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec009eb0 tx=0x7fe1ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:12.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.684+0000 7fe1dbfff640 1 -- 172.21.15.5:0/2889640004 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec040070 con 0x7fe1f415d530 2024-05-20T18:35:12.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.684+0000 7fe1dbfff640 1 -- 172.21.15.5:0/2889640004 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1ec031bc0 con 0x7fe1f415d530 2024-05-20T18:35:12.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.684+0000 7fe1f9f7e640 1 -- 172.21.15.5:0/2889640004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f40cb890 con 0x7fe1f415d530 2024-05-20T18:35:12.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.684+0000 7fe1dbfff640 1 -- 172.21.15.5:0/2889640004 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec03c420 con 0x7fe1f415d530 2024-05-20T18:35:12.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.684+0000 7fe1f9f7e640 1 -- 172.21.15.5:0/2889640004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f40cbe50 con 0x7fe1f415d530 2024-05-20T18:35:12.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.685+0000 7fe1f9f7e640 1 -- 172.21.15.5:0/2889640004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1f4075b00 con 0x7fe1f415d530 2024-05-20T18:35:12.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.685+0000 7fe1dbfff640 1 -- 172.21.15.5:0/2889640004 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe1ec039040 con 0x7fe1f415d530 2024-05-20T18:35:12.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.686+0000 7fe1dbfff640 1 --2- 172.21.15.5:0/2889640004 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe1e006be60 0x7fe1e006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:12.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.686+0000 7fe1f37fe640 1 --2- 172.21.15.5:0/2889640004 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe1e006be60 0x7fe1e006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:12.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.686+0000 7fe1dbfff640 1 -- 172.21.15.5:0/2889640004 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7fe1ec0ac4d0 con 0x7fe1f415d530 2024-05-20T18:35:12.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.686+0000 7fe1f37fe640 1 --2- 172.21.15.5:0/2889640004 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe1e006be60 0x7fe1e006e370 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40d29a0 tx=0x7fe1e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:12.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.691+0000 7fe1dbfff640 1 -- 172.21.15.5:0/2889640004 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe1ec07a4f0 con 0x7fe1f415d530 2024-05-20T18:35:12.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.911+0000 7fe1f9f7e640 1 -- 172.21.15.5:0/2889640004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1f40cb300 con 0x7fe1f415d530 2024-05-20T18:35:12.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.912+0000 7fe1dbfff640 1 -- 172.21.15.5:0/2889640004 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5172 (secure 0 0 0) 0x7fe1ec07e7f0 con 0x7fe1f415d530 2024-05-20T18:35:12.913 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:12.913 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/337248865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":337248865},{"type":"v1","addr":"172.21.15.63:6834","nonce":337248865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3705772553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3705772553},{"type":"v1","addr":"172.21.15.63:6835","nonce":3705772553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":7434,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/776188138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":776188138},{"type":"v1","addr":"172.21.15.5:6835","nonce":776188138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:33:12.457716+0000","modified":"2024-05-20T18:34:39.318498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7953":{"gid":7953,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6837/3880006778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3880006778},{"type":"v1","addr":"172.21.15.5:6837","nonce":3880006778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7953,"qdb_cluster":[7953]},"id":6}]} 2024-05-20T18:35:12.913 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 73 2024-05-20T18:35:12.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.914+0000 7fe1f9f7e640 1 -- 172.21.15.5:0/2889640004 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe1e006be60 msgr2=0x7fe1e006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:12.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.914+0000 7fe1f9f7e640 1 --2- 172.21.15.5:0/2889640004 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe1e006be60 0x7fe1e006e370 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe1f40d29a0 tx=0x7fe1e4008040 comp rx=0 tx=0).stop 2024-05-20T18:35:12.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.914+0000 7fe1f9f7e640 1 -- 172.21.15.5:0/2889640004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1f415d530 msgr2=0x7fe1f414bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:12.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.914+0000 7fe1f9f7e640 1 --2- 172.21.15.5:0/2889640004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1f415d530 0x7fe1f414bb00 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec009eb0 tx=0x7fe1ec004a40 comp rx=0 tx=0).stop 2024-05-20T18:35:12.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.914+0000 7fe1f9f7e640 1 -- 172.21.15.5:0/2889640004 shutdown_connections 2024-05-20T18:35:12.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.914+0000 7fe1f9f7e640 1 --2- 172.21.15.5:0/2889640004 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe1e006be60 0x7fe1e006e370 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.914+0000 7fe1f9f7e640 1 --2- 172.21.15.5:0/2889640004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1f415d530 0x7fe1f414bb00 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.914+0000 7fe1f9f7e640 1 --2- 172.21.15.5:0/2889640004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1f40b1700 0x7fe1f40ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.914+0000 7fe1f9f7e640 1 --2- 172.21.15.5:0/2889640004 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1f4070850 0x7fe1f40b0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:12.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.914+0000 7fe1f9f7e640 1 -- 172.21.15.5:0/2889640004 >> 172.21.15.5:0/2889640004 conn(0x7fe1f40d29d0 msgr2=0x7fe1f40b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:12.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.914+0000 7fe1f9f7e640 1 -- 172.21.15.5:0/2889640004 shutdown_connections 2024-05-20T18:35:12.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:12.914+0000 7fe1f9f7e640 1 -- 172.21.15.5:0/2889640004 wait complete. 2024-05-20T18:35:12.926 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-05-20T18:35:13.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.042+0000 7fdd3bb02640 1 Processor -- start 2024-05-20T18:35:13.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.042+0000 7fdd3bb02640 1 -- start start 2024-05-20T18:35:13.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.043+0000 7fdd3bb02640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fdd3406da80 0x7fdd341602f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.043+0000 7fdd3bb02640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdd340b1c80 0x7fdd3409a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.043+0000 7fdd3bb02640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd340af9d0 con 0x7fdd340b2090 2024-05-20T18:35:13.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.044+0000 7fdd3bb02640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fdd34159da0 con 0x7fdd340b1c80 2024-05-20T18:35:13.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.044+0000 7fdd3bb02640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fdd340af1c0 con 0x7fdd3406da80 2024-05-20T18:35:13.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.044+0000 7fdd3909d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdd340b1c80 0x7fdd3409a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:13.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.044+0000 7fdd3909d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdd340b1c80 0x7fdd3409a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56482/0 (socket says 172.21.15.5:56482) 2024-05-20T18:35:13.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.044+0000 7fdd3909d640 1 -- 172.21.15.5:0/179158849 learned_addr learned my addr 172.21.15.5:0/179158849 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:13.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.044+0000 7fdd3989e640 1 --2- 172.21.15.5:0/179158849 >> v2:172.21.15.63:3300/0 conn(0x7fdd3406da80 0x7fdd341602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:13.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.044+0000 7fdd3909d640 1 -- 172.21.15.5:0/179158849 >> v2:172.21.15.63:3300/0 conn(0x7fdd3406da80 msgr2=0x7fdd341602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:13.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.044+0000 7fdd3909d640 1 --2- 172.21.15.5:0/179158849 >> v2:172.21.15.63:3300/0 conn(0x7fdd3406da80 0x7fdd341602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.044+0000 7fdd3909d640 1 -- 172.21.15.5:0/179158849 >> v1:172.21.15.63:6789/0 conn(0x7fdd340b2090 legacy=0x7fdd340d1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:13.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.044+0000 7fdd3909d640 1 -- 172.21.15.5:0/179158849 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd340af410 con 0x7fdd340b1c80 2024-05-20T18:35:13.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.044+0000 7fdd3989e640 1 --2- 172.21.15.5:0/179158849 >> v2:172.21.15.63:3300/0 conn(0x7fdd3406da80 0x7fdd341602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:13.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.045+0000 7fdd3909d640 1 --2- 172.21.15.5:0/179158849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd340b1c80 0x7fdd3409a4f0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fdd340729c0 tx=0x7fdd2402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d25fbf95d269bb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:13.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.045+0000 7fdd3889c640 1 -- 172.21.15.5:0/179158849 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd24004030 con 0x7fdd340b1c80 2024-05-20T18:35:13.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.045+0000 7fdd3889c640 1 -- 172.21.15.5:0/179158849 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd240041d0 con 0x7fdd340b1c80 2024-05-20T18:35:13.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.045+0000 7fdd3889c640 1 -- 172.21.15.5:0/179158849 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd24003610 con 0x7fdd340b1c80 2024-05-20T18:35:13.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.046+0000 7fdd3bb02640 1 -- 172.21.15.5:0/179158849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd340b1c80 msgr2=0x7fdd3409a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:13.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.046+0000 7fdd3bb02640 1 --2- 172.21.15.5:0/179158849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd340b1c80 0x7fdd3409a4f0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fdd340729c0 tx=0x7fdd2402a1c0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.046+0000 7fdd3bb02640 1 -- 172.21.15.5:0/179158849 shutdown_connections 2024-05-20T18:35:13.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.046+0000 7fdd3bb02640 1 --2- 172.21.15.5:0/179158849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd340b1c80 0x7fdd3409a4f0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.046+0000 7fdd3bb02640 1 --2- 172.21.15.5:0/179158849 >> v2:172.21.15.63:3300/0 conn(0x7fdd3406da80 0x7fdd341602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.046+0000 7fdd3bb02640 1 -- 172.21.15.5:0/179158849 >> 172.21.15.5:0/179158849 conn(0x7fdd340b6670 msgr2=0x7fdd340b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:13.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.046+0000 7fdd3bb02640 1 -- 172.21.15.5:0/179158849 shutdown_connections 2024-05-20T18:35:13.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.046+0000 7fdd3bb02640 1 -- 172.21.15.5:0/179158849 wait complete. 2024-05-20T18:35:13.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.047+0000 7fdd3bb02640 1 Processor -- start 2024-05-20T18:35:13.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.048+0000 7fdd3bb02640 1 -- start start 2024-05-20T18:35:13.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.049+0000 7fdd3bb02640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd3406da80 0x7fdd34147230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.049+0000 7fdd3bb02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd340b1c80 0x7fdd341547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.049+0000 7fdd3bb02640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd340b2090 0x7fdd3409d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.049+0000 7fdd3bb02640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3409b260 con 0x7fdd340b1c80 2024-05-20T18:35:13.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.049+0000 7fdd3bb02640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdd340d17b0 con 0x7fdd3406da80 2024-05-20T18:35:13.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.049+0000 7fdd3bb02640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdd340d2350 con 0x7fdd340b2090 2024-05-20T18:35:13.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.049+0000 7fdd3909d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd340b1c80 0x7fdd341547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:13.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.049+0000 7fdd3909d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd340b1c80 0x7fdd341547c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56486/0 (socket says 172.21.15.5:56486) 2024-05-20T18:35:13.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.049+0000 7fdd3909d640 1 -- 172.21.15.5:0/47863999 learned_addr learned my addr 172.21.15.5:0/47863999 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:13.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.049+0000 7fdd3989e640 1 --2- 172.21.15.5:0/47863999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd3406da80 0x7fdd34147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:13.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.049+0000 7fdd3a09f640 1 --2- 172.21.15.5:0/47863999 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd340b2090 0x7fdd3409d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:13.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.049+0000 7fdd3909d640 1 -- 172.21.15.5:0/47863999 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd340b2090 msgr2=0x7fdd3409d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:13.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.049+0000 7fdd3909d640 1 --2- 172.21.15.5:0/47863999 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd340b2090 0x7fdd3409d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.049+0000 7fdd3909d640 1 -- 172.21.15.5:0/47863999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd3406da80 msgr2=0x7fdd34147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:13.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.049+0000 7fdd3909d640 1 --2- 172.21.15.5:0/47863999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd3406da80 0x7fdd34147230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.049+0000 7fdd3909d640 1 -- 172.21.15.5:0/47863999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd34147770 con 0x7fdd340b1c80 2024-05-20T18:35:13.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.049+0000 7fdd3a09f640 1 --2- 172.21.15.5:0/47863999 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd340b2090 0x7fdd3409d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:13.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.049+0000 7fdd3909d640 1 --2- 172.21.15.5:0/47863999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd340b1c80 0x7fdd341547c0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fdd24000c00 tx=0x7fdd240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:13.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.050+0000 7fdd2a7fc640 1 -- 172.21.15.5:0/47863999 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd24041020 con 0x7fdd340b1c80 2024-05-20T18:35:13.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.050+0000 7fdd2a7fc640 1 -- 172.21.15.5:0/47863999 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd24003c50 con 0x7fdd340b1c80 2024-05-20T18:35:13.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.050+0000 7fdd3bb02640 1 -- 172.21.15.5:0/47863999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd341551f0 con 0x7fdd340b1c80 2024-05-20T18:35:13.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.050+0000 7fdd2a7fc640 1 -- 172.21.15.5:0/47863999 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd24004030 con 0x7fdd340b1c80 2024-05-20T18:35:13.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.050+0000 7fdd3bb02640 1 -- 172.21.15.5:0/47863999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd341557b0 con 0x7fdd340b1c80 2024-05-20T18:35:13.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.051+0000 7fdd2a7fc640 1 -- 172.21.15.5:0/47863999 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdd24044020 con 0x7fdd340b1c80 2024-05-20T18:35:13.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.052+0000 7fdd2a7fc640 1 --2- 172.21.15.5:0/47863999 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdd1806be60 0x7fdd1806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.052+0000 7fdd2a7fc640 1 -- 172.21.15.5:0/47863999 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7fdd240abdc0 con 0x7fdd340b1c80 2024-05-20T18:35:13.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.052+0000 7fdd3989e640 1 --2- 172.21.15.5:0/47863999 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdd1806be60 0x7fdd1806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:13.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.053+0000 7fdd3989e640 1 --2- 172.21.15.5:0/47863999 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdd1806be60 0x7fdd1806e370 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fdd30006fd0 tx=0x7fdd30008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:13.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.053+0000 7fdd429aa740 1 Processor -- start 2024-05-20T18:35:13.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.053+0000 7fdd429aa740 1 -- start start 2024-05-20T18:35:13.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.055+0000 7fdd429aa740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f0e1e49350 0x55f0e2214670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.055+0000 7fdd429aa740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f0e1e20490 0x55f0e222f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.055+0000 7fdd429aa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0e1e28670 0x55f0e222f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.055+0000 7fdd429aa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f0e214da30 con 0x55f0e1e28670 2024-05-20T18:35:13.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.055+0000 7fdd429aa740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f0e20eb590 con 0x55f0e1e20490 2024-05-20T18:35:13.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.055+0000 7fdd429aa740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55f0e21499f0 con 0x55f0e1e49350 2024-05-20T18:35:13.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.055+0000 7fdd3a09f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0e1e28670 0x55f0e222f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:13.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.055+0000 7fdd3a09f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0e1e28670 0x55f0e222f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56502/0 (socket says 172.21.15.5:56502) 2024-05-20T18:35:13.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.055+0000 7fdd3a09f640 1 -- 172.21.15.5:0/3027398822 learned_addr learned my addr 172.21.15.5:0/3027398822 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:13.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.055+0000 7fdd3989e640 1 --2- 172.21.15.5:0/3027398822 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f0e1e49350 0x55f0e2214670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:13.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.055+0000 7fdd3909d640 1 --2- 172.21.15.5:0/3027398822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f0e1e20490 0x55f0e222f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:13.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.055+0000 7fdd3a09f640 1 -- 172.21.15.5:0/3027398822 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f0e1e49350 msgr2=0x55f0e2214670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:13.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.055+0000 7fdd3a09f640 1 --2- 172.21.15.5:0/3027398822 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f0e1e49350 0x55f0e2214670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.055+0000 7fdd3a09f640 1 -- 172.21.15.5:0/3027398822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f0e1e20490 msgr2=0x55f0e222f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:13.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.055+0000 7fdd3a09f640 1 --2- 172.21.15.5:0/3027398822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f0e1e20490 0x55f0e222f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.055+0000 7fdd3a09f640 1 -- 172.21.15.5:0/3027398822 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0e1f2d7c0 con 0x55f0e1e28670 2024-05-20T18:35:13.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.055+0000 7fdd3a09f640 1 --2- 172.21.15.5:0/3027398822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0e1e28670 0x55f0e222f8e0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fdd340b1260 tx=0x7fdd2c002070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af7994997032420e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:13.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.056+0000 7fdd177fe640 1 -- 172.21.15.5:0/3027398822 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c019070 con 0x55f0e1e28670 2024-05-20T18:35:13.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.056+0000 7fdd177fe640 1 -- 172.21.15.5:0/3027398822 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd2c003b80 con 0x55f0e1e28670 2024-05-20T18:35:13.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.056+0000 7fdd177fe640 1 -- 172.21.15.5:0/3027398822 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c00e800 con 0x55f0e1e28670 2024-05-20T18:35:13.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.056+0000 7fdd429aa740 1 -- 172.21.15.5:0/3027398822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0e1e28670 msgr2=0x55f0e222f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:13.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.056+0000 7fdd429aa740 1 --2- 172.21.15.5:0/3027398822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0e1e28670 0x55f0e222f8e0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fdd340b1260 tx=0x7fdd2c002070 comp rx=0 tx=0).stop 2024-05-20T18:35:13.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.056+0000 7fdd429aa740 1 -- 172.21.15.5:0/3027398822 shutdown_connections 2024-05-20T18:35:13.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.056+0000 7fdd429aa740 1 --2- 172.21.15.5:0/3027398822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0e1e28670 0x55f0e222f8e0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.056+0000 7fdd429aa740 1 --2- 172.21.15.5:0/3027398822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f0e1e20490 0x55f0e222f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.056+0000 7fdd429aa740 1 --2- 172.21.15.5:0/3027398822 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f0e1e49350 0x55f0e2214670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.056+0000 7fdd429aa740 1 -- 172.21.15.5:0/3027398822 >> 172.21.15.5:0/3027398822 conn(0x55f0e1e3d4a0 msgr2=0x55f0e21e9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:13.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.057+0000 7fdd429aa740 1 -- 172.21.15.5:0/3027398822 shutdown_connections 2024-05-20T18:35:13.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.057+0000 7fdd429aa740 1 -- 172.21.15.5:0/3027398822 wait complete. 2024-05-20T18:35:13.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.057+0000 7fdd429aa740 1 -- start start 2024-05-20T18:35:13.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.058+0000 7fdd429aa740 1 Processor -- start 2024-05-20T18:35:13.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.059+0000 7fdd429aa740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f0e1e20490 0x55f0e22102b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.059+0000 7fdd429aa740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f0e1e28670 0x55f0e22161b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.059+0000 7fdd429aa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0e1e49350 0x55f0e222c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.059+0000 7fdd429aa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f0e20e9060 con 0x55f0e1e49350 2024-05-20T18:35:13.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.059+0000 7fdd429aa740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f0e222d130 con 0x55f0e1e20490 2024-05-20T18:35:13.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.059+0000 7fdd429aa740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55f0e222acb0 con 0x55f0e1e28670 2024-05-20T18:35:13.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.059+0000 7fdd429aa740 10 client.0 _ll_register_callbacks cb 0x55f0e210ddd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:35:13.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.059+0000 7fdd3a09f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0e1e49350 0x55f0e222c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:13.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.059+0000 7fdd3a09f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0e1e49350 0x55f0e222c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56518/0 (socket says 172.21.15.5:56518) 2024-05-20T18:35:13.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.059+0000 7fdd3a09f640 1 -- 172.21.15.5:0/1715760851 learned_addr learned my addr 172.21.15.5:0/1715760851 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.059+0000 7fdd3989e640 1 --2- 172.21.15.5:0/1715760851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f0e1e20490 0x55f0e22102b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.059+0000 7fdd3909d640 1 --2- 172.21.15.5:0/1715760851 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f0e1e28670 0x55f0e22161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.059+0000 7fdd3a09f640 1 -- 172.21.15.5:0/1715760851 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f0e1e28670 msgr2=0x55f0e22161b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.060+0000 7fdd3a09f640 1 --2- 172.21.15.5:0/1715760851 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f0e1e28670 0x55f0e22161b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.060+0000 7fdd3a09f640 1 -- 172.21.15.5:0/1715760851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f0e1e20490 msgr2=0x55f0e22102b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.060+0000 7fdd3a09f640 1 --2- 172.21.15.5:0/1715760851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f0e1e20490 0x55f0e22102b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.060+0000 7fdd3a09f640 1 -- 172.21.15.5:0/1715760851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0e1f8df50 con 0x55f0e1e49350 2024-05-20T18:35:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.060+0000 7fdd3a09f640 1 --2- 172.21.15.5:0/1715760851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0e1e49350 0x55f0e222c7c0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c017040 tx=0x7fdd2c001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba7382750164630 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.060+0000 7fdd14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:35:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.060+0000 7fdd14ff9640 1 -- 172.21.15.5:0/1715760851 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c023060 con 0x55f0e1e49350 2024-05-20T18:35:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.060+0000 7fdd14ff9640 1 -- 172.21.15.5:0/1715760851 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7fdd2c006d50 con 0x55f0e1e49350 2024-05-20T18:35:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.060+0000 7fdd429aa740 1 -- 172.21.15.5:0/1715760851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd2c003b80 con 0x55f0e1e49350 2024-05-20T18:35:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.060+0000 7fdd14ff9640 1 -- 172.21.15.5:0/1715760851 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fdd2c029050 con 0x55f0e1e49350 2024-05-20T18:35:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.060+0000 7fdd429aa740 10 client.8124 fetch_fsmap learned FSMap version 73 2024-05-20T18:35:13.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.060+0000 7fdd429aa740 1 -- 172.21.15.5:0/1715760851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x55f0e1e210b0 con 0x55f0e1e49350 2024-05-20T18:35:13.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.060+0000 7fdd14ff9640 1 -- 172.21.15.5:0/1715760851 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd2c003b80 con 0x55f0e1e49350 2024-05-20T18:35:13.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.060+0000 7fdd14ff9640 1 -- 172.21.15.5:0/1715760851 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 73) ==== 3494+0+0 (secure 0 0 0) 0x7fdd2c02c020 con 0x55f0e1e49350 2024-05-20T18:35:13.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.060+0000 7fdd429aa740 10 client.8124 fetch_fsmap finished waiting for FSMap version 73 2024-05-20T18:35:13.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.060+0000 7fdd429aa740 10 client.8124 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-20T18:35:13.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.060+0000 7fdd429aa740 20 client.8124 populate_metadata read hostname 'smithi005' 2024-05-20T18:35:13.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.060+0000 7fdd429aa740 1 --2- 172.21.15.5:0/1715760851 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x55f0e1e08130 0x55f0e2241f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.060+0000 7fdd429aa740 4 client.8124 mds_command: new command op to 7953 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:35:13.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.060+0000 7fdd429aa740 1 -- 172.21.15.5:0/1715760851 --> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0e215b160 con 0x55f0e1e08130 2024-05-20T18:35:13.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.061+0000 7fdd3989e640 1 --2- 172.21.15.5:0/1715760851 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x55f0e1e08130 0x55f0e2241f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:13.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.061+0000 7fdd14ff9640 10 client.8124 ms_handle_connect on v2:172.21.15.5:6836/3880006778 2024-05-20T18:35:13.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.061+0000 7fdd3989e640 1 --2- 172.21.15.5:0/1715760851 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x55f0e1e08130 0x55f0e2241f50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27c2f542c1dc98ce server_cookie=96e384df8ac52d6c in_seq=0 out_seq=0 2024-05-20T18:35:13.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.062+0000 7fdd14ff9640 1 -- 172.21.15.5:0/1715760851 <== mds.0 v2:172.21.15.5:6836/3880006778 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f0e215b160 con 0x55f0e1e08130 2024-05-20T18:35:13.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.062+0000 7fdd14ff9640 10 client.8124 handle_command_reply: tid=0 2024-05-20T18:35:13.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.062+0000 7fdd429aa740 1 client.8124 shutdown 2024-05-20T18:35:13.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.062+0000 7fdd429aa740 1 -- 172.21.15.5:0/1715760851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0e1e49350 msgr2=0x55f0e222c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:13.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.062+0000 7fdd429aa740 1 --2- 172.21.15.5:0/1715760851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0e1e49350 0x55f0e222c7c0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c017040 tx=0x7fdd2c001250 comp rx=0 tx=0).stop 2024-05-20T18:35:13.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.062+0000 7fdd429aa740 1 -- 172.21.15.5:0/1715760851 shutdown_connections 2024-05-20T18:35:13.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.062+0000 7fdd429aa740 1 --2- 172.21.15.5:0/1715760851 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x55f0e1e08130 0x55f0e2241f50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.062+0000 7fdd429aa740 1 --2- 172.21.15.5:0/1715760851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0e1e49350 0x55f0e222c7c0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.062+0000 7fdd429aa740 1 --2- 172.21.15.5:0/1715760851 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f0e1e28670 0x55f0e22161b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.062+0000 7fdd14ff9640 0 client.8124 ms_handle_reset on v2:172.21.15.5:6836/3880006778 2024-05-20T18:35:13.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.062+0000 7fdd429aa740 1 --2- 172.21.15.5:0/1715760851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f0e1e20490 0x55f0e22102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.062+0000 7fdd429aa740 1 -- 172.21.15.5:0/1715760851 >> 172.21.15.5:0/1715760851 conn(0x55f0e1e3d4a0 msgr2=0x55f0e2214670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:13.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.062+0000 7fdd429aa740 1 -- 172.21.15.5:0/1715760851 shutdown_connections 2024-05-20T18:35:13.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.062+0000 7fdd429aa740 1 -- 172.21.15.5:0/1715760851 wait complete. 2024-05-20T18:35:13.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.062+0000 7fdd429aa740 20 client.8124 trim_cache size 0 max 16384 2024-05-20T18:35:13.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.076+0000 7fdd429aa740 1 Processor -- start 2024-05-20T18:35:13.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.076+0000 7fdd429aa740 1 -- start start 2024-05-20T18:35:13.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.077+0000 7fdd429aa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0e1e20490 0x55f0e222f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.077+0000 7fdd429aa740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f0e1e28670 0x55f0e222f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.077+0000 7fdd429aa740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f0e1e49350 0x55f0e21e8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.077+0000 7fdd429aa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdd342017c0 con 0x55f0e1e20490 2024-05-20T18:35:13.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.077+0000 7fdd429aa740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdd34200ba0 con 0x55f0e1e49350 2024-05-20T18:35:13.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.077+0000 7fdd429aa740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdd34201170 con 0x55f0e1e28670 2024-05-20T18:35:13.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.078+0000 7fdd3989e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0e1e20490 0x55f0e222f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:13.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.078+0000 7fdd3989e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0e1e20490 0x55f0e222f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56528/0 (socket says 172.21.15.5:56528) 2024-05-20T18:35:13.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.078+0000 7fdd3989e640 1 -- 172.21.15.5:0/682515089 learned_addr learned my addr 172.21.15.5:0/682515089 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:13.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.078+0000 7fdd3909d640 1 --2- 172.21.15.5:0/682515089 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f0e1e28670 0x55f0e222f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:13.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.078+0000 7fdd3989e640 1 -- 172.21.15.5:0/682515089 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f0e1e28670 msgr2=0x55f0e222f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:13.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.078+0000 7fdd3989e640 1 --2- 172.21.15.5:0/682515089 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f0e1e28670 0x55f0e222f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.078+0000 7fdd3a09f640 1 --2- 172.21.15.5:0/682515089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f0e1e49350 0x55f0e21e8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:13.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.078+0000 7fdd3989e640 1 -- 172.21.15.5:0/682515089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f0e1e49350 msgr2=0x55f0e21e8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:13.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.078+0000 7fdd3989e640 1 --2- 172.21.15.5:0/682515089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f0e1e49350 0x55f0e21e8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.078+0000 7fdd3989e640 1 -- 172.21.15.5:0/682515089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0e2218290 con 0x55f0e1e20490 2024-05-20T18:35:13.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.078+0000 7fdd3909d640 1 --2- 172.21.15.5:0/682515089 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f0e1e28670 0x55f0e222f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:13.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.078+0000 7fdd3a09f640 1 --2- 172.21.15.5:0/682515089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f0e1e49350 0x55f0e21e8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:13.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.078+0000 7fdd3989e640 1 --2- 172.21.15.5:0/682515089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0e1e20490 0x55f0e222f2d0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x55f0e21e94e0 tx=0x7fdd30003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f74de7aa5a204f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:13.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.078+0000 7fdd16ffd640 1 -- 172.21.15.5:0/682515089 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3001c040 con 0x55f0e1e20490 2024-05-20T18:35:13.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.078+0000 7fdd16ffd640 1 -- 172.21.15.5:0/682515089 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd3002d070 con 0x55f0e1e20490 2024-05-20T18:35:13.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.078+0000 7fdd16ffd640 1 -- 172.21.15.5:0/682515089 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd30016bd0 con 0x55f0e1e20490 2024-05-20T18:35:13.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.079+0000 7fdd429aa740 1 -- 172.21.15.5:0/682515089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0e1e20490 msgr2=0x55f0e222f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:13.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.079+0000 7fdd429aa740 1 --2- 172.21.15.5:0/682515089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0e1e20490 0x55f0e222f2d0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x55f0e21e94e0 tx=0x7fdd30003f90 comp rx=0 tx=0).stop 2024-05-20T18:35:13.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.079+0000 7fdd429aa740 1 -- 172.21.15.5:0/682515089 shutdown_connections 2024-05-20T18:35:13.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.079+0000 7fdd429aa740 1 --2- 172.21.15.5:0/682515089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f0e1e49350 0x55f0e21e8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.079+0000 7fdd429aa740 1 --2- 172.21.15.5:0/682515089 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f0e1e28670 0x55f0e222f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.079+0000 7fdd429aa740 1 --2- 172.21.15.5:0/682515089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0e1e20490 0x55f0e222f2d0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.079+0000 7fdd429aa740 1 -- 172.21.15.5:0/682515089 >> 172.21.15.5:0/682515089 conn(0x55f0e1e3d4a0 msgr2=0x55f0e21dfc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:13.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.079+0000 7fdd429aa740 1 -- 172.21.15.5:0/682515089 shutdown_connections 2024-05-20T18:35:13.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.079+0000 7fdd429aa740 1 -- 172.21.15.5:0/682515089 wait complete. 2024-05-20T18:35:13.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.080+0000 7fdd429aa740 1 -- start start 2024-05-20T18:35:13.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.080+0000 7fdd429aa740 1 Processor -- start 2024-05-20T18:35:13.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.081+0000 7fdd429aa740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f0e1e20490 0x55f0e22ab620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.081+0000 7fdd429aa740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f0e1e28670 0x55f0e2215f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.081+0000 7fdd429aa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0e1e49350 0x55f0e220e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.081+0000 7fdd429aa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f0e2218470 con 0x55f0e1e49350 2024-05-20T18:35:13.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.081+0000 7fdd429aa740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f0e222ced0 con 0x55f0e1e28670 2024-05-20T18:35:13.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.081+0000 7fdd429aa740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55f0e21e4230 con 0x55f0e1e20490 2024-05-20T18:35:13.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.082+0000 7fdd429aa740 10 client.0 _ll_register_callbacks cb 0x7fdd2c017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:35:13.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.082+0000 7fdd3a09f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0e1e49350 0x55f0e220e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:13.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.082+0000 7fdd3a09f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0e1e49350 0x55f0e220e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56536/0 (socket says 172.21.15.5:56536) 2024-05-20T18:35:13.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.082+0000 7fdd3a09f640 1 -- 172.21.15.5:0/3995069263 learned_addr learned my addr 172.21.15.5:0/3995069263 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:13.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.082+0000 7fdd3909d640 1 --2- 172.21.15.5:0/3995069263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f0e1e28670 0x55f0e2215f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:13.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.082+0000 7fdd3989e640 1 --2- 172.21.15.5:0/3995069263 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f0e1e20490 0x55f0e22ab620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:13.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.082+0000 7fdd3a09f640 1 -- 172.21.15.5:0/3995069263 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f0e1e20490 msgr2=0x55f0e22ab620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:13.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.082+0000 7fdd3a09f640 1 --2- 172.21.15.5:0/3995069263 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f0e1e20490 0x55f0e22ab620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.082+0000 7fdd3a09f640 1 -- 172.21.15.5:0/3995069263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f0e1e28670 msgr2=0x55f0e2215f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:13.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.082+0000 7fdd3a09f640 1 --2- 172.21.15.5:0/3995069263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f0e1e28670 0x55f0e2215f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.082+0000 7fdd3a09f640 1 -- 172.21.15.5:0/3995069263 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0e2216480 con 0x55f0e1e49350 2024-05-20T18:35:13.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.082+0000 7fdd3a09f640 1 --2- 172.21.15.5:0/3995069263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0e1e49350 0x55f0e220e360 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c000f80 tx=0x7fdd2c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a243673f67324d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:13.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.082+0000 7fdd14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:35:13.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.082+0000 7fdd14ff9640 1 -- 172.21.15.5:0/3995069263 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c019070 con 0x55f0e1e49350 2024-05-20T18:35:13.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.082+0000 7fdd14ff9640 1 -- 172.21.15.5:0/3995069263 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7fdd2c005770 con 0x55f0e1e49350 2024-05-20T18:35:13.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.082+0000 7fdd429aa740 1 -- 172.21.15.5:0/3995069263 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd3002d070 con 0x55f0e1e49350 2024-05-20T18:35:13.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.083+0000 7fdd14ff9640 1 -- 172.21.15.5:0/3995069263 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fdd2c03b050 con 0x55f0e1e49350 2024-05-20T18:35:13.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.083+0000 7fdd429aa740 10 client.8130 fetch_fsmap learned FSMap version 73 2024-05-20T18:35:13.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.083+0000 7fdd429aa740 1 -- 172.21.15.5:0/3995069263 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x55f0e2215a20 con 0x55f0e1e49350 2024-05-20T18:35:13.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.083+0000 7fdd14ff9640 1 -- 172.21.15.5:0/3995069263 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd3002d070 con 0x55f0e1e49350 2024-05-20T18:35:13.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.083+0000 7fdd14ff9640 1 -- 172.21.15.5:0/3995069263 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 73) ==== 3494+0+0 (secure 0 0 0) 0x7fdd2c03e020 con 0x55f0e1e49350 2024-05-20T18:35:13.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.083+0000 7fdd429aa740 10 client.8130 fetch_fsmap finished waiting for FSMap version 73 2024-05-20T18:35:13.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.083+0000 7fdd429aa740 10 client.8130 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-20T18:35:13.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.083+0000 7fdd429aa740 20 client.8130 populate_metadata read hostname 'smithi005' 2024-05-20T18:35:13.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.083+0000 7fdd429aa740 1 --2- 172.21.15.5:0/3995069263 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x55f0e2214990 0x55f0e22af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.083+0000 7fdd429aa740 4 client.8130 mds_command: new command op to 7953 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-20T18:35:13.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.083+0000 7fdd429aa740 1 -- 172.21.15.5:0/3995069263 --> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55f0e21554b0 con 0x55f0e2214990 2024-05-20T18:35:13.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.083+0000 7fdd3989e640 1 --2- 172.21.15.5:0/3995069263 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x55f0e2214990 0x55f0e22af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:13.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.083+0000 7fdd14ff9640 10 client.8130 ms_handle_connect on v2:172.21.15.5:6836/3880006778 2024-05-20T18:35:13.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.084+0000 7fdd3989e640 1 --2- 172.21.15.5:0/3995069263 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x55f0e2214990 0x55f0e22af1c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abee5adbab67537f server_cookie=d46c5118d4d2a69c in_seq=0 out_seq=0 2024-05-20T18:35:13.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.084+0000 7fdd14ff9640 1 -- 172.21.15.5:0/3995069263 <== mds.0 v2:172.21.15.5:6836/3880006778 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f0e21554b0 con 0x55f0e2214990 2024-05-20T18:35:13.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.084+0000 7fdd14ff9640 10 client.8130 handle_command_reply: tid=0 2024-05-20T18:35:13.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.084+0000 7fdd429aa740 1 client.8130 shutdown 2024-05-20T18:35:13.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.084+0000 7fdd429aa740 1 -- 172.21.15.5:0/3995069263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0e1e49350 msgr2=0x55f0e220e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:13.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.084+0000 7fdd429aa740 1 --2- 172.21.15.5:0/3995069263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0e1e49350 0x55f0e220e360 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c000f80 tx=0x7fdd2c009040 comp rx=0 tx=0).stop 2024-05-20T18:35:13.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.084+0000 7fdd429aa740 1 -- 172.21.15.5:0/3995069263 shutdown_connections 2024-05-20T18:35:13.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.084+0000 7fdd429aa740 1 --2- 172.21.15.5:0/3995069263 >> [v2:172.21.15.5:6836/3880006778,v1:172.21.15.5:6837/3880006778] conn(0x55f0e2214990 0x55f0e22af1c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.084+0000 7fdd429aa740 1 --2- 172.21.15.5:0/3995069263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0e1e49350 0x55f0e220e360 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.084+0000 7fdd429aa740 1 --2- 172.21.15.5:0/3995069263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f0e1e28670 0x55f0e2215f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.084+0000 7fdd429aa740 1 --2- 172.21.15.5:0/3995069263 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f0e1e20490 0x55f0e22ab620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.084+0000 7fdd429aa740 1 -- 172.21.15.5:0/3995069263 >> 172.21.15.5:0/3995069263 conn(0x55f0e1e3d4a0 msgr2=0x55f0e22307f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:13.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.084+0000 7fdd14ff9640 0 client.8130 ms_handle_reset on v2:172.21.15.5:6836/3880006778 2024-05-20T18:35:13.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.084+0000 7fdd429aa740 1 -- 172.21.15.5:0/3995069263 shutdown_connections 2024-05-20T18:35:13.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.084+0000 7fdd429aa740 1 -- 172.21.15.5:0/3995069263 wait complete. 2024-05-20T18:35:13.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.084+0000 7fdd429aa740 20 client.8130 trim_cache size 0 max 16384 2024-05-20T18:35:13.090 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:13.090 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-05-20T18:35:13.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.085+0000 7fdd3bb02640 1 -- 172.21.15.5:0/47863999 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdd1806be60 msgr2=0x7fdd1806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:13.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.085+0000 7fdd3bb02640 1 --2- 172.21.15.5:0/47863999 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdd1806be60 0x7fdd1806e370 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fdd30006fd0 tx=0x7fdd30008040 comp rx=0 tx=0).stop 2024-05-20T18:35:13.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.085+0000 7fdd3bb02640 1 -- 172.21.15.5:0/47863999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd340b1c80 msgr2=0x7fdd341547c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:13.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.085+0000 7fdd3bb02640 1 --2- 172.21.15.5:0/47863999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd340b1c80 0x7fdd341547c0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fdd24000c00 tx=0x7fdd240046c0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.085+0000 7fdd3bb02640 1 -- 172.21.15.5:0/47863999 shutdown_connections 2024-05-20T18:35:13.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.085+0000 7fdd3bb02640 1 --2- 172.21.15.5:0/47863999 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdd1806be60 0x7fdd1806e370 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.085+0000 7fdd3bb02640 1 --2- 172.21.15.5:0/47863999 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd340b2090 0x7fdd3409d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.085+0000 7fdd3bb02640 1 --2- 172.21.15.5:0/47863999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd340b1c80 0x7fdd341547c0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.085+0000 7fdd3bb02640 1 --2- 172.21.15.5:0/47863999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd3406da80 0x7fdd34147230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.085+0000 7fdd3bb02640 1 -- 172.21.15.5:0/47863999 >> 172.21.15.5:0/47863999 conn(0x7fdd340b6670 msgr2=0x7fdd34161e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:13.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.085+0000 7fdd3bb02640 1 -- 172.21.15.5:0/47863999 shutdown_connections 2024-05-20T18:35:13.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.085+0000 7fdd3bb02640 1 -- 172.21.15.5:0/47863999 wait complete. 2024-05-20T18:35:13.097 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-20T18:35:13.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.211+0000 7ff0d2b77640 1 Processor -- start 2024-05-20T18:35:13.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.211+0000 7ff0d2b77640 1 -- start start 2024-05-20T18:35:13.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.213+0000 7ff0d2b77640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0cc15ff40 0x7ff0cc0b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.213+0000 7ff0d2b77640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0cc15ca90 con 0x7ff0cc15bd30 2024-05-20T18:35:13.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.213+0000 7ff0d2b77640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0cc15f460 con 0x7ff0cc06bdb0 2024-05-20T18:35:13.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.213+0000 7ff0d2b77640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff0cc15c850 con 0x7ff0cc15ff40 2024-05-20T18:35:13.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.213+0000 7ff0d1114640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff0cc15bd30 0x7ff0cc15c140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39968/0 (socket says 172.21.15.5:39968) 2024-05-20T18:35:13.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.213+0000 7ff0d1114640 1 -- 172.21.15.5:0/3658940171 learned_addr learned my addr 172.21.15.5:0/3658940171 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:13.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.213+0000 7ff0d0913640 1 --2- 172.21.15.5:0/3658940171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0cc15ff40 0x7ff0cc0b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:13.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.213+0000 7ff0cb7fe640 1 -- 172.21.15.5:0/3658940171 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 496799064 0 0) 0x7ff0cc15ca90 con 0x7ff0cc15bd30 2024-05-20T18:35:13.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.213+0000 7ff0cb7fe640 1 -- 172.21.15.5:0/3658940171 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0b4005730 con 0x7ff0cc15bd30 2024-05-20T18:35:13.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.213+0000 7ff0cb7fe640 1 -- 172.21.15.5:0/3658940171 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1061798482 0 0) 0x7ff0cc15f460 con 0x7ff0cc06bdb0 2024-05-20T18:35:13.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.214+0000 7ff0cb7fe640 1 -- 172.21.15.5:0/3658940171 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0cc15ca90 con 0x7ff0cc06bdb0 2024-05-20T18:35:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.214+0000 7ff0d0913640 1 -- 172.21.15.5:0/3658940171 >> v1:172.21.15.63:6789/0 conn(0x7ff0cc06bdb0 legacy=0x7ff0cc06c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.214+0000 7ff0cb7fe640 1 -- 172.21.15.5:0/3658940171 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1711751184 0 0) 0x7ff0b4005730 con 0x7ff0cc15bd30 2024-05-20T18:35:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.214+0000 7ff0d0913640 1 -- 172.21.15.5:0/3658940171 >> v1:172.21.15.5:6789/0 conn(0x7ff0cc15bd30 legacy=0x7ff0cc15c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.214+0000 7ff0d0913640 1 -- 172.21.15.5:0/3658940171 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0cc15f740 con 0x7ff0cc15ff40 2024-05-20T18:35:13.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.215+0000 7ff0d0913640 1 --2- 172.21.15.5:0/3658940171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0cc15ff40 0x7ff0cc0b50c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc0097d0 tx=0x7ff0bc02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=215fd91031c89428 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:13.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.215+0000 7ff0cb7fe640 1 -- 172.21.15.5:0/3658940171 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc037070 con 0x7ff0cc15ff40 2024-05-20T18:35:13.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.215+0000 7ff0cb7fe640 1 -- 172.21.15.5:0/3658940171 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0bc02ac30 con 0x7ff0cc15ff40 2024-05-20T18:35:13.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.215+0000 7ff0cb7fe640 1 -- 172.21.15.5:0/3658940171 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c40019f0 con 0x7ff0cc15ff40 2024-05-20T18:35:13.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.215+0000 7ff0d2b77640 1 -- 172.21.15.5:0/3658940171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0cc15ff40 msgr2=0x7ff0cc0b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:13.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.215+0000 7ff0d2b77640 1 --2- 172.21.15.5:0/3658940171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0cc15ff40 0x7ff0cc0b50c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc0097d0 tx=0x7ff0bc02a1a0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.216+0000 7ff0d2b77640 1 -- 172.21.15.5:0/3658940171 shutdown_connections 2024-05-20T18:35:13.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.216+0000 7ff0d2b77640 1 --2- 172.21.15.5:0/3658940171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0cc15ff40 0x7ff0cc0b50c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.216+0000 7ff0d2b77640 1 -- 172.21.15.5:0/3658940171 >> 172.21.15.5:0/3658940171 conn(0x7ff0cc0cd2a0 msgr2=0x7ff0cc0cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:13.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.216+0000 7ff0d2b77640 1 -- 172.21.15.5:0/3658940171 shutdown_connections 2024-05-20T18:35:13.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.216+0000 7ff0d2b77640 1 -- 172.21.15.5:0/3658940171 wait complete. 2024-05-20T18:35:13.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.217+0000 7ff0d2b77640 1 Processor -- start 2024-05-20T18:35:13.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.217+0000 7ff0d2b77640 1 -- start start 2024-05-20T18:35:13.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.219+0000 7ff0d2b77640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0cc06bdb0 0x7ff0cc09a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.219+0000 7ff0d2b77640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0cc15bd30 0x7ff0cc14b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.219+0000 7ff0d2b77640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0cc15ff40 0x7ff0cc0b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.219+0000 7ff0d2b77640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc06b970 con 0x7ff0cc15ff40 2024-05-20T18:35:13.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.219+0000 7ff0d2b77640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc06b7f0 con 0x7ff0cc06bdb0 2024-05-20T18:35:13.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.219+0000 7ff0d2b77640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff0cc06baf0 con 0x7ff0cc15bd30 2024-05-20T18:35:13.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.219+0000 7ff0d1114640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0cc15ff40 0x7ff0cc0b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.219+0000 7ff0d1114640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0cc15ff40 0x7ff0cc0b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56546/0 (socket says 172.21.15.5:56546) 2024-05-20T18:35:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.219+0000 7ff0cbfff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0cc15bd30 0x7ff0cc14b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.219+0000 7ff0d1114640 1 -- 172.21.15.5:0/2673084031 learned_addr learned my addr 172.21.15.5:0/2673084031 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.219+0000 7ff0d0913640 1 --2- 172.21.15.5:0/2673084031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0cc06bdb0 0x7ff0cc09a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.219+0000 7ff0d1114640 1 -- 172.21.15.5:0/2673084031 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0cc15bd30 msgr2=0x7ff0cc14b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.219+0000 7ff0d1114640 1 --2- 172.21.15.5:0/2673084031 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0cc15bd30 0x7ff0cc14b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.219+0000 7ff0d1114640 1 -- 172.21.15.5:0/2673084031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0cc06bdb0 msgr2=0x7ff0cc09a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.219+0000 7ff0d1114640 1 --2- 172.21.15.5:0/2673084031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0cc06bdb0 0x7ff0cc09a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.219+0000 7ff0d1114640 1 -- 172.21.15.5:0/2673084031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0cc15e880 con 0x7ff0cc15ff40 2024-05-20T18:35:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.220+0000 7ff0d1114640 1 --2- 172.21.15.5:0/2673084031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0cc15ff40 0x7ff0cc0b04c0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4001350 tx=0x7ff0c400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.220+0000 7ff0c97fa640 1 -- 172.21.15.5:0/2673084031 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4017070 con 0x7ff0cc15ff40 2024-05-20T18:35:13.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.220+0000 7ff0c97fa640 1 -- 172.21.15.5:0/2673084031 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0c400ad50 con 0x7ff0cc15ff40 2024-05-20T18:35:13.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.220+0000 7ff0d2b77640 1 -- 172.21.15.5:0/2673084031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0cc0aed60 con 0x7ff0cc15ff40 2024-05-20T18:35:13.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.220+0000 7ff0c97fa640 1 -- 172.21.15.5:0/2673084031 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4012640 con 0x7ff0cc15ff40 2024-05-20T18:35:13.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.220+0000 7ff0d2b77640 1 -- 172.21.15.5:0/2673084031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0cc15d6e0 con 0x7ff0cc15ff40 2024-05-20T18:35:13.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.221+0000 7ff0d2b77640 1 -- 172.21.15.5:0/2673084031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0cc09e8f0 con 0x7ff0cc15ff40 2024-05-20T18:35:13.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.222+0000 7ff0c97fa640 1 -- 172.21.15.5:0/2673084031 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff0c401a070 con 0x7ff0cc15ff40 2024-05-20T18:35:13.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.222+0000 7ff0c97fa640 1 --2- 172.21.15.5:0/2673084031 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff0b806be30 0x7ff0b806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.222+0000 7ff0c97fa640 1 -- 172.21.15.5:0/2673084031 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7ff0c408b000 con 0x7ff0cc15ff40 2024-05-20T18:35:13.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.222+0000 7ff0d0913640 1 --2- 172.21.15.5:0/2673084031 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff0b806be30 0x7ff0b806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:13.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.223+0000 7ff0d0913640 1 --2- 172.21.15.5:0/2673084031 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff0b806be30 0x7ff0b806e340 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc034000 tx=0x7ff0bc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:13.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.227+0000 7ff0c97fa640 1 -- 172.21.15.5:0/2673084031 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff0cc09e8f0 con 0x7ff0cc15ff40 2024-05-20T18:35:13.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.448+0000 7ff0d2b77640 1 -- 172.21.15.5:0/2673084031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff0cc15ea50 con 0x7ff0cc15ff40 2024-05-20T18:35:13.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.448+0000 7ff0c97fa640 1 -- 172.21.15.5:0/2673084031 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7ff0c4059020 con 0x7ff0cc15ff40 2024-05-20T18:35:13.449 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:13.450 INFO:teuthology.orchestra.run.smithi005.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-20T18:35:13.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.450+0000 7ff0d2b77640 1 -- 172.21.15.5:0/2673084031 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff0b806be30 msgr2=0x7ff0b806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:13.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.450+0000 7ff0d2b77640 1 --2- 172.21.15.5:0/2673084031 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff0b806be30 0x7ff0b806e340 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc034000 tx=0x7ff0bc02e000 comp rx=0 tx=0).stop 2024-05-20T18:35:13.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.450+0000 7ff0d2b77640 1 -- 172.21.15.5:0/2673084031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0cc15ff40 msgr2=0x7ff0cc0b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:13.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.450+0000 7ff0d2b77640 1 --2- 172.21.15.5:0/2673084031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0cc15ff40 0x7ff0cc0b04c0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4001350 tx=0x7ff0c400a430 comp rx=0 tx=0).stop 2024-05-20T18:35:13.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.450+0000 7ff0d2b77640 1 -- 172.21.15.5:0/2673084031 shutdown_connections 2024-05-20T18:35:13.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.450+0000 7ff0d2b77640 1 --2- 172.21.15.5:0/2673084031 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff0b806be30 0x7ff0b806e340 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.450+0000 7ff0d2b77640 1 --2- 172.21.15.5:0/2673084031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0cc15ff40 0x7ff0cc0b04c0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.450+0000 7ff0d2b77640 1 --2- 172.21.15.5:0/2673084031 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0cc15bd30 0x7ff0cc14b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.450+0000 7ff0d2b77640 1 --2- 172.21.15.5:0/2673084031 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0cc06bdb0 0x7ff0cc09a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.450+0000 7ff0d2b77640 1 -- 172.21.15.5:0/2673084031 >> 172.21.15.5:0/2673084031 conn(0x7ff0cc0cd2a0 msgr2=0x7ff0cc156640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:13.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.450+0000 7ff0d2b77640 1 -- 172.21.15.5:0/2673084031 shutdown_connections 2024-05-20T18:35:13.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.450+0000 7ff0d2b77640 1 -- 172.21.15.5:0/2673084031 wait complete. 2024-05-20T18:35:13.463 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-20T18:35:13.463 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-20T18:35:13.463 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T18:35:13.520 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T18:35:13.578 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-20T18:35:13.578 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T18:35:13.578 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:35:13.578 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:35:13.595 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:35:13.595 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:35:13.651 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:35:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.804+0000 7fbf63a28640 1 Processor -- start 2024-05-20T18:35:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.804+0000 7fbf63a28640 1 -- start start 2024-05-20T18:35:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.806+0000 7fbf63a28640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbf5c15f220 0x7fbf5c0cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.806+0000 7fbf63a28640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbf5c0cf910 0x7fbf5c0d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.806+0000 7fbf63a28640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf5c0cfd20 con 0x7fbf5c0cd680 2024-05-20T18:35:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.806+0000 7fbf63a28640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fbf5c0cda90 con 0x7fbf5c15f220 2024-05-20T18:35:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.806+0000 7fbf63a28640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fbf5c06d0f0 con 0x7fbf5c0cf910 2024-05-20T18:35:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.806+0000 7fbf60fc3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbf5c15f220 0x7fbf5c0cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.806+0000 7fbf60fc3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbf5c15f220 0x7fbf5c0cf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56562/0 (socket says 172.21.15.5:56562) 2024-05-20T18:35:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.806+0000 7fbf60fc3640 1 -- 172.21.15.5:0/4037254854 learned_addr learned my addr 172.21.15.5:0/4037254854 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.806+0000 7fbf61fc5640 1 --2- 172.21.15.5:0/4037254854 >> v2:172.21.15.63:3300/0 conn(0x7fbf5c0cf910 0x7fbf5c0d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.806+0000 7fbf60fc3640 1 -- 172.21.15.5:0/4037254854 >> v2:172.21.15.63:3300/0 conn(0x7fbf5c0cf910 msgr2=0x7fbf5c0d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.806+0000 7fbf60fc3640 1 --2- 172.21.15.5:0/4037254854 >> v2:172.21.15.63:3300/0 conn(0x7fbf5c0cf910 0x7fbf5c0d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.806+0000 7fbf60fc3640 1 -- 172.21.15.5:0/4037254854 >> v1:172.21.15.63:6789/0 conn(0x7fbf5c0cd680 legacy=0x7fbf5c06c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.806+0000 7fbf60fc3640 1 -- 172.21.15.5:0/4037254854 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf5c06d3d0 con 0x7fbf5c15f220 2024-05-20T18:35:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.806+0000 7fbf61fc5640 1 --2- 172.21.15.5:0/4037254854 >> v2:172.21.15.63:3300/0 conn(0x7fbf5c0cf910 0x7fbf5c0d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.807+0000 7fbf60fc3640 1 --2- 172.21.15.5:0/4037254854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5c15f220 0x7fbf5c0cf3d0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0cd1c0 tx=0x7fbf4c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a34759bab603d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.807+0000 7fbf53fff640 1 -- 172.21.15.5:0/4037254854 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c037070 con 0x7fbf5c15f220 2024-05-20T18:35:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.807+0000 7fbf53fff640 1 -- 172.21.15.5:0/4037254854 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf4c02ae20 con 0x7fbf5c15f220 2024-05-20T18:35:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.807+0000 7fbf53fff640 1 -- 172.21.15.5:0/4037254854 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c032660 con 0x7fbf5c15f220 2024-05-20T18:35:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.808+0000 7fbf63a28640 1 -- 172.21.15.5:0/4037254854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5c15f220 msgr2=0x7fbf5c0cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.808+0000 7fbf63a28640 1 --2- 172.21.15.5:0/4037254854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5c15f220 0x7fbf5c0cf3d0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0cd1c0 tx=0x7fbf4c02a110 comp rx=0 tx=0).stop 2024-05-20T18:35:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.808+0000 7fbf63a28640 1 -- 172.21.15.5:0/4037254854 shutdown_connections 2024-05-20T18:35:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.808+0000 7fbf63a28640 1 --2- 172.21.15.5:0/4037254854 >> v2:172.21.15.63:3300/0 conn(0x7fbf5c0cf910 0x7fbf5c0d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.808+0000 7fbf63a28640 1 --2- 172.21.15.5:0/4037254854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5c15f220 0x7fbf5c0cf3d0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.808+0000 7fbf63a28640 1 -- 172.21.15.5:0/4037254854 >> 172.21.15.5:0/4037254854 conn(0x7fbf5c0b44d0 msgr2=0x7fbf5c0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.808+0000 7fbf63a28640 1 -- 172.21.15.5:0/4037254854 shutdown_connections 2024-05-20T18:35:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.808+0000 7fbf63a28640 1 -- 172.21.15.5:0/4037254854 wait complete. 2024-05-20T18:35:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.810+0000 7fbf63a28640 1 Processor -- start 2024-05-20T18:35:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.810+0000 7fbf63a28640 1 -- start start 2024-05-20T18:35:13.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.811+0000 7fbf63a28640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5c0cd680 0x7fbf5c0c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.811+0000 7fbf63a28640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf5c0cf910 0x7fbf5c0b7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.811+0000 7fbf63a28640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf5c15f220 0x7fbf5c15f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.811+0000 7fbf63a28640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c06d990 con 0x7fbf5c0cd680 2024-05-20T18:35:13.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.811+0000 7fbf63a28640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c0d0b80 con 0x7fbf5c15f220 2024-05-20T18:35:13.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.811+0000 7fbf63a28640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbf5c06cdd0 con 0x7fbf5c0cf910 2024-05-20T18:35:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.811+0000 7fbf617c4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5c0cd680 0x7fbf5c0c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.811+0000 7fbf617c4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5c0cd680 0x7fbf5c0c8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56564/0 (socket says 172.21.15.5:56564) 2024-05-20T18:35:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.811+0000 7fbf617c4640 1 -- 172.21.15.5:0/3163921737 learned_addr learned my addr 172.21.15.5:0/3163921737 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.812+0000 7fbf617c4640 1 -- 172.21.15.5:0/3163921737 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf5c0cf910 msgr2=0x7fbf5c0b7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.812+0000 7fbf617c4640 1 --2- 172.21.15.5:0/3163921737 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf5c0cf910 0x7fbf5c0b7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.812+0000 7fbf617c4640 1 -- 172.21.15.5:0/3163921737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf5c15f220 msgr2=0x7fbf5c15f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.812+0000 7fbf617c4640 1 --2- 172.21.15.5:0/3163921737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf5c15f220 0x7fbf5c15f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.812+0000 7fbf617c4640 1 -- 172.21.15.5:0/3163921737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf5c0c8850 con 0x7fbf5c0cd680 2024-05-20T18:35:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.812+0000 7fbf617c4640 1 --2- 172.21.15.5:0/3163921737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5c0cd680 0x7fbf5c0c8310 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800b9c0 tx=0x7fbf5800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.812+0000 7fbf51ffb640 1 -- 172.21.15.5:0/3163921737 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5800cba0 con 0x7fbf5c0cd680 2024-05-20T18:35:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.812+0000 7fbf51ffb640 1 -- 172.21.15.5:0/3163921737 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf5800cd40 con 0x7fbf5c0cd680 2024-05-20T18:35:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.812+0000 7fbf63a28640 1 -- 172.21.15.5:0/3163921737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf5c0c8ca0 con 0x7fbf5c0cd680 2024-05-20T18:35:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.812+0000 7fbf51ffb640 1 -- 172.21.15.5:0/3163921737 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf58003bb0 con 0x7fbf5c0cd680 2024-05-20T18:35:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.812+0000 7fbf63a28640 1 -- 172.21.15.5:0/3163921737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf5c0c7830 con 0x7fbf5c0cd680 2024-05-20T18:35:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.813+0000 7fbf63a28640 1 -- 172.21.15.5:0/3163921737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf5c075b00 con 0x7fbf5c0cd680 2024-05-20T18:35:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.814+0000 7fbf51ffb640 1 -- 172.21.15.5:0/3163921737 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbf58014440 con 0x7fbf5c0cd680 2024-05-20T18:35:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.814+0000 7fbf51ffb640 1 --2- 172.21.15.5:0/3163921737 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf4006be30 0x7fbf4006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.814+0000 7fbf60fc3640 1 --2- 172.21.15.5:0/3163921737 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf4006be30 0x7fbf4006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.814+0000 7fbf51ffb640 1 -- 172.21.15.5:0/3163921737 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7fbf5808b030 con 0x7fbf5c0cd680 2024-05-20T18:35:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.815+0000 7fbf60fc3640 1 --2- 172.21.15.5:0/3163921737 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf4006be30 0x7fbf4006e340 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c034000 tx=0x7fbf4c02c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:13.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:13.819+0000 7fbf51ffb640 1 -- 172.21.15.5:0/3163921737 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbf580596a0 con 0x7fbf5c0cd680 2024-05-20T18:35:14.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.004+0000 7fbf63a28640 1 -- 172.21.15.5:0/3163921737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf5c1513d0 con 0x7fbf5c0cd680 2024-05-20T18:35:14.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.005+0000 7fbf51ffb640 1 -- 172.21.15.5:0/3163921737 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+17216 (secure 0 0 0) 0x7fbf58058fa0 con 0x7fbf5c0cd680 2024-05-20T18:35:14.006 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:14.007 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":94,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:34:35.299736+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:33:10.184948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-20T18:33:11.453356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.1e","mappings":[{"from":1,"to":5}]},{"pgid":"14.29","mappings":[{"from":1,"to":7}]},{"pgid":"14.2a","mappings":[{"from":1,"to":2}]},{"pgid":"14.2c","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6837/3556492262":"2024-05-21T18:34:35.299705+0000","172.21.15.5:6836/3556492262":"2024-05-21T18:34:35.299705+0000","172.21.15.63:6834/2759396130":"2024-05-21T18:33:23.233035+0000","172.21.15.63:6832/2759396130":"2024-05-21T18:33:23.233035+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:35:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.007+0000 7fbf63a28640 1 -- 172.21.15.5:0/3163921737 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf4006be30 msgr2=0x7fbf4006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.007+0000 7fbf63a28640 1 --2- 172.21.15.5:0/3163921737 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf4006be30 0x7fbf4006e340 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c034000 tx=0x7fbf4c02c000 comp rx=0 tx=0).stop 2024-05-20T18:35:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.007+0000 7fbf63a28640 1 -- 172.21.15.5:0/3163921737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5c0cd680 msgr2=0x7fbf5c0c8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.007+0000 7fbf63a28640 1 --2- 172.21.15.5:0/3163921737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5c0cd680 0x7fbf5c0c8310 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800b9c0 tx=0x7fbf5800be80 comp rx=0 tx=0).stop 2024-05-20T18:35:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.007+0000 7fbf63a28640 1 -- 172.21.15.5:0/3163921737 shutdown_connections 2024-05-20T18:35:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.007+0000 7fbf63a28640 1 --2- 172.21.15.5:0/3163921737 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbf4006be30 0x7fbf4006e340 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.007+0000 7fbf63a28640 1 --2- 172.21.15.5:0/3163921737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf5c15f220 0x7fbf5c15f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.007+0000 7fbf63a28640 1 --2- 172.21.15.5:0/3163921737 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf5c0cf910 0x7fbf5c0b7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.007+0000 7fbf63a28640 1 --2- 172.21.15.5:0/3163921737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5c0cd680 0x7fbf5c0c8310 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.007+0000 7fbf63a28640 1 -- 172.21.15.5:0/3163921737 >> 172.21.15.5:0/3163921737 conn(0x7fbf5c0b44d0 msgr2=0x7fbf5c151cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.007+0000 7fbf63a28640 1 -- 172.21.15.5:0/3163921737 shutdown_connections 2024-05-20T18:35:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.007+0000 7fbf63a28640 1 -- 172.21.15.5:0/3163921737 wait complete. 2024-05-20T18:35:14.020 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:33:10.184948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:33:11.453356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:35:14.020 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:35:14.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.137+0000 7fbc2666c640 1 Processor -- start 2024-05-20T18:35:14.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.137+0000 7fbc2666c640 1 -- start start 2024-05-20T18:35:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.138+0000 7fbc2666c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbc20071630 0x7fbc2015f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.138+0000 7fbc2666c640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc200ce2b0 con 0x7fbc20070850 2024-05-20T18:35:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.138+0000 7fbc2666c640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc2006cce0 con 0x7fbc200cd1b0 2024-05-20T18:35:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.138+0000 7fbc2666c640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fbc2015f790 con 0x7fbc20071630 2024-05-20T18:35:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.138+0000 7fbc1ffff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbc20071630 0x7fbc2015f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.138+0000 7fbc24c09640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbc20070850 0x7fbc20070c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39980/0 (socket says 172.21.15.5:39980) 2024-05-20T18:35:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.138+0000 7fbc1ffff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbc20071630 0x7fbc2015f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56570/0 (socket says 172.21.15.5:56570) 2024-05-20T18:35:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.138+0000 7fbc24c09640 1 -- 172.21.15.5:0/3445089138 learned_addr learned my addr 172.21.15.5:0/3445089138 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.139+0000 7fbc1ffff640 1 -- 172.21.15.5:0/3445089138 >> v1:172.21.15.63:6789/0 conn(0x7fbc200cd1b0 legacy=0x7fbc2006d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.139+0000 7fbc1ffff640 1 -- 172.21.15.5:0/3445089138 >> v1:172.21.15.5:6789/0 conn(0x7fbc20070850 legacy=0x7fbc20070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.139+0000 7fbc1ffff640 1 -- 172.21.15.5:0/3445089138 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2006cfc0 con 0x7fbc20071630 2024-05-20T18:35:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.140+0000 7fbc1ffff640 1 --2- 172.21.15.5:0/3445089138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc20071630 0x7fbc2015f250 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fbc100097a0 tx=0x7fbc1002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899a694aca85e7b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.140+0000 7fbc1effd640 1 -- 172.21.15.5:0/3445089138 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc10037070 con 0x7fbc20071630 2024-05-20T18:35:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.140+0000 7fbc1effd640 1 -- 172.21.15.5:0/3445089138 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc10032440 con 0x7fbc20071630 2024-05-20T18:35:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.140+0000 7fbc1effd640 1 -- 172.21.15.5:0/3445089138 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc10032810 con 0x7fbc20071630 2024-05-20T18:35:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.140+0000 7fbc2666c640 1 -- 172.21.15.5:0/3445089138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc20071630 msgr2=0x7fbc2015f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.140+0000 7fbc2666c640 1 --2- 172.21.15.5:0/3445089138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc20071630 0x7fbc2015f250 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fbc100097a0 tx=0x7fbc1002a1a0 comp rx=0 tx=0).stop 2024-05-20T18:35:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.140+0000 7fbc2666c640 1 -- 172.21.15.5:0/3445089138 shutdown_connections 2024-05-20T18:35:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.140+0000 7fbc2666c640 1 --2- 172.21.15.5:0/3445089138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc20071630 0x7fbc2015f250 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.140+0000 7fbc2666c640 1 -- 172.21.15.5:0/3445089138 >> 172.21.15.5:0/3445089138 conn(0x7fbc200d29d0 msgr2=0x7fbc200d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.141+0000 7fbc2666c640 1 -- 172.21.15.5:0/3445089138 shutdown_connections 2024-05-20T18:35:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.141+0000 7fbc2666c640 1 -- 172.21.15.5:0/3445089138 wait complete. 2024-05-20T18:35:14.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.142+0000 7fbc2666c640 1 Processor -- start 2024-05-20T18:35:14.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.143+0000 7fbc2666c640 1 -- start start 2024-05-20T18:35:14.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.144+0000 7fbc2666c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc20070850 0x7fbc2015f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:14.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.144+0000 7fbc2666c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc20071630 0x7fbc200b6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:14.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.144+0000 7fbc2666c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbc200cd1b0 0x7fbc201f8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:14.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.144+0000 7fbc2666c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2006ded0 con 0x7fbc20071630 2024-05-20T18:35:14.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.144+0000 7fbc2666c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2006c990 con 0x7fbc20070850 2024-05-20T18:35:14.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.144+0000 7fbc2666c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbc200b68a0 con 0x7fbc200cd1b0 2024-05-20T18:35:14.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.144+0000 7fbc1f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc20071630 0x7fbc200b6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:14.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.144+0000 7fbc1f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc20071630 0x7fbc200b6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56578/0 (socket says 172.21.15.5:56578) 2024-05-20T18:35:14.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.144+0000 7fbc1f7fe640 1 -- 172.21.15.5:0/3956273319 learned_addr learned my addr 172.21.15.5:0/3956273319 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:14.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.144+0000 7fbc1ffff640 1 --2- 172.21.15.5:0/3956273319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc20070850 0x7fbc2015f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:14.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.144+0000 7fbc24c09640 1 --2- 172.21.15.5:0/3956273319 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbc200cd1b0 0x7fbc201f8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:14.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.144+0000 7fbc1f7fe640 1 -- 172.21.15.5:0/3956273319 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbc200cd1b0 msgr2=0x7fbc201f8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:14.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.144+0000 7fbc1f7fe640 1 --2- 172.21.15.5:0/3956273319 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbc200cd1b0 0x7fbc201f8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:14.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.144+0000 7fbc1f7fe640 1 -- 172.21.15.5:0/3956273319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc20070850 msgr2=0x7fbc2015f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:14.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.144+0000 7fbc1f7fe640 1 --2- 172.21.15.5:0/3956273319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc20070850 0x7fbc2015f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:14.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.144+0000 7fbc1f7fe640 1 -- 172.21.15.5:0/3956273319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc201534d0 con 0x7fbc20071630 2024-05-20T18:35:14.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.144+0000 7fbc1ffff640 1 --2- 172.21.15.5:0/3956273319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc20070850 0x7fbc2015f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:14.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.144+0000 7fbc1f7fe640 1 --2- 172.21.15.5:0/3956273319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc20071630 0x7fbc200b6e40 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400c3c0 tx=0x7fbc1400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:14.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.145+0000 7fbc1cff9640 1 -- 172.21.15.5:0/3956273319 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc140052d0 con 0x7fbc20071630 2024-05-20T18:35:14.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.145+0000 7fbc1cff9640 1 -- 172.21.15.5:0/3956273319 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc14005490 con 0x7fbc20071630 2024-05-20T18:35:14.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.145+0000 7fbc1cff9640 1 -- 172.21.15.5:0/3956273319 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc14007260 con 0x7fbc20071630 2024-05-20T18:35:14.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.145+0000 7fbc2666c640 1 -- 172.21.15.5:0/3956273319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc201537b0 con 0x7fbc20071630 2024-05-20T18:35:14.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.145+0000 7fbc2666c640 1 -- 172.21.15.5:0/3956273319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc20153d70 con 0x7fbc20071630 2024-05-20T18:35:14.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.146+0000 7fbc2666c640 1 -- 172.21.15.5:0/3956273319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc2015f550 con 0x7fbc20071630 2024-05-20T18:35:14.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.146+0000 7fbc1cff9640 1 -- 172.21.15.5:0/3956273319 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbc14007400 con 0x7fbc20071630 2024-05-20T18:35:14.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.147+0000 7fbc1cff9640 1 --2- 172.21.15.5:0/3956273319 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbc0c06be60 0x7fbc0c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:14.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.147+0000 7fbc1cff9640 1 -- 172.21.15.5:0/3956273319 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7fbc1408a670 con 0x7fbc20071630 2024-05-20T18:35:14.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.147+0000 7fbc1ffff640 1 --2- 172.21.15.5:0/3956273319 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbc0c06be60 0x7fbc0c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:14.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.147+0000 7fbc1ffff640 1 --2- 172.21.15.5:0/3956273319 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbc0c06be60 0x7fbc0c06e370 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbc10034000 tx=0x7fbc1002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:14.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.152+0000 7fbc1cff9640 1 -- 172.21.15.5:0/3956273319 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbc2015f550 con 0x7fbc20071630 2024-05-20T18:35:14.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.334+0000 7fbc2666c640 1 -- 172.21.15.5:0/3956273319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbc20075b00 con 0x7fbc20071630 2024-05-20T18:35:14.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.334+0000 7fbc1cff9640 1 -- 172.21.15.5:0/3956273319 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fbc14058c60 con 0x7fbc20071630 2024-05-20T18:35:14.335 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:35:14.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.337+0000 7fbc2666c640 1 -- 172.21.15.5:0/3956273319 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbc0c06be60 msgr2=0x7fbc0c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:14.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.337+0000 7fbc2666c640 1 --2- 172.21.15.5:0/3956273319 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbc0c06be60 0x7fbc0c06e370 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbc10034000 tx=0x7fbc1002e000 comp rx=0 tx=0).stop 2024-05-20T18:35:14.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.337+0000 7fbc2666c640 1 -- 172.21.15.5:0/3956273319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc20071630 msgr2=0x7fbc200b6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:14.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.337+0000 7fbc2666c640 1 --2- 172.21.15.5:0/3956273319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc20071630 0x7fbc200b6e40 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400c3c0 tx=0x7fbc1400c880 comp rx=0 tx=0).stop 2024-05-20T18:35:14.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.337+0000 7fbc2666c640 1 -- 172.21.15.5:0/3956273319 shutdown_connections 2024-05-20T18:35:14.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.337+0000 7fbc2666c640 1 --2- 172.21.15.5:0/3956273319 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbc0c06be60 0x7fbc0c06e370 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:14.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.337+0000 7fbc2666c640 1 --2- 172.21.15.5:0/3956273319 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbc200cd1b0 0x7fbc201f8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:14.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.337+0000 7fbc2666c640 1 --2- 172.21.15.5:0/3956273319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc20071630 0x7fbc200b6e40 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:14.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.337+0000 7fbc2666c640 1 --2- 172.21.15.5:0/3956273319 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc20070850 0x7fbc2015f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:14.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.337+0000 7fbc2666c640 1 -- 172.21.15.5:0/3956273319 >> 172.21.15.5:0/3956273319 conn(0x7fbc200d29d0 msgr2=0x7fbc200d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:14.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.337+0000 7fbc2666c640 1 -- 172.21.15.5:0/3956273319 shutdown_connections 2024-05-20T18:35:14.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.337+0000 7fbc2666c640 1 -- 172.21.15.5:0/3956273319 wait complete. 2024-05-20T18:35:14.349 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:35:14.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.466+0000 7fb7a4856640 1 Processor -- start 2024-05-20T18:35:14.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.466+0000 7fb7a4856640 1 -- start start 2024-05-20T18:35:14.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.468+0000 7fb7a4856640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb7a0162820 0x7fb7a00b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:14.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.468+0000 7fb7a4856640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb7a00b3e80 0x7fb7a006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:14.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.468+0000 7fb7a4856640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7a006dd20 con 0x7fb7a006d910 2024-05-20T18:35:14.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.468+0000 7fb7a4856640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fb7a00d0630 con 0x7fb7a0162820 2024-05-20T18:35:14.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.468+0000 7fb7a4856640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb7a00d39c0 con 0x7fb7a00b3e80 2024-05-20T18:35:14.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.468+0000 7fb79e59c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb7a0162820 0x7fb7a00b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:14.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.468+0000 7fb79e59c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb7a0162820 0x7fb7a00b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56592/0 (socket says 172.21.15.5:56592) 2024-05-20T18:35:14.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.468+0000 7fb79e59c640 1 -- 172.21.15.5:0/560606050 learned_addr learned my addr 172.21.15.5:0/560606050 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:14.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.468+0000 7fb79dd9b640 1 --2- 172.21.15.5:0/560606050 >> v2:172.21.15.63:3300/0 conn(0x7fb7a00b3e80 0x7fb7a006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:14.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.468+0000 7fb79e59c640 1 -- 172.21.15.5:0/560606050 >> v2:172.21.15.63:3300/0 conn(0x7fb7a00b3e80 msgr2=0x7fb7a006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:14.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.468+0000 7fb79e59c640 1 --2- 172.21.15.5:0/560606050 >> v2:172.21.15.63:3300/0 conn(0x7fb7a00b3e80 0x7fb7a006d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:14.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.468+0000 7fb79e59c640 1 -- 172.21.15.5:0/560606050 >> v1:172.21.15.63:6789/0 conn(0x7fb7a006d910 legacy=0x7fb7a00d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:14.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.468+0000 7fb79e59c640 1 -- 172.21.15.5:0/560606050 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a00d3ca0 con 0x7fb7a0162820 2024-05-20T18:35:14.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.469+0000 7fb79e59c640 1 --2- 172.21.15.5:0/560606050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7a0162820 0x7fb7a00b3940 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fb7a00d1230 tx=0x7fb79802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6705210b6b784d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:14.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.469+0000 7fb79d59a640 1 -- 172.21.15.5:0/560606050 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb798037070 con 0x7fb7a0162820 2024-05-20T18:35:14.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.469+0000 7fb79d59a640 1 -- 172.21.15.5:0/560606050 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb798032440 con 0x7fb7a0162820 2024-05-20T18:35:14.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.469+0000 7fb79d59a640 1 -- 172.21.15.5:0/560606050 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb798032810 con 0x7fb7a0162820 2024-05-20T18:35:14.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.470+0000 7fb7a4856640 1 -- 172.21.15.5:0/560606050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7a0162820 msgr2=0x7fb7a00b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:14.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.470+0000 7fb7a4856640 1 --2- 172.21.15.5:0/560606050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7a0162820 0x7fb7a00b3940 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fb7a00d1230 tx=0x7fb79802a270 comp rx=0 tx=0).stop 2024-05-20T18:35:14.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.470+0000 7fb7a4856640 1 -- 172.21.15.5:0/560606050 shutdown_connections 2024-05-20T18:35:14.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.470+0000 7fb7a4856640 1 --2- 172.21.15.5:0/560606050 >> v2:172.21.15.63:3300/0 conn(0x7fb7a00b3e80 0x7fb7a006d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:14.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.470+0000 7fb7a4856640 1 --2- 172.21.15.5:0/560606050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7a0162820 0x7fb7a00b3940 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:14.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.470+0000 7fb7a4856640 1 -- 172.21.15.5:0/560606050 >> 172.21.15.5:0/560606050 conn(0x7fb7a00d4bd0 msgr2=0x7fb7a00d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:14.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.470+0000 7fb7a4856640 1 -- 172.21.15.5:0/560606050 shutdown_connections 2024-05-20T18:35:14.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.470+0000 7fb7a4856640 1 -- 172.21.15.5:0/560606050 wait complete. 2024-05-20T18:35:14.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.472+0000 7fb7a4856640 1 Processor -- start 2024-05-20T18:35:14.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.472+0000 7fb7a4856640 1 -- start start 2024-05-20T18:35:14.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.474+0000 7fb7a4856640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7a006d910 0x7fb7a014aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:14.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.474+0000 7fb7a4856640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7a00b3e80 0x7fb7a014b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:14.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.474+0000 7fb7a4856640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7a0162820 0x7fb7a01f8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:14.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.474+0000 7fb7a4856640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a00d0300 con 0x7fb7a006d910 2024-05-20T18:35:14.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.474+0000 7fb7a4856640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a00d4260 con 0x7fb7a0162820 2024-05-20T18:35:14.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.474+0000 7fb7a4856640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb7a00b91a0 con 0x7fb7a00b3e80 2024-05-20T18:35:14.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.474+0000 7fb79e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7a006d910 0x7fb7a014aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:14.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.474+0000 7fb79e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7a006d910 0x7fb7a014aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56600/0 (socket says 172.21.15.5:56600) 2024-05-20T18:35:14.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.474+0000 7fb79e59c640 1 -- 172.21.15.5:0/2954249862 learned_addr learned my addr 172.21.15.5:0/2954249862 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:14.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.474+0000 7fb79ed9d640 1 --2- 172.21.15.5:0/2954249862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7a0162820 0x7fb7a01f8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:14.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.474+0000 7fb79dd9b640 1 --2- 172.21.15.5:0/2954249862 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7a00b3e80 0x7fb7a014b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:14.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.474+0000 7fb79e59c640 1 -- 172.21.15.5:0/2954249862 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7a00b3e80 msgr2=0x7fb7a014b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:14.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.474+0000 7fb79e59c640 1 --2- 172.21.15.5:0/2954249862 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7a00b3e80 0x7fb7a014b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:14.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.474+0000 7fb79e59c640 1 -- 172.21.15.5:0/2954249862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7a0162820 msgr2=0x7fb7a01f8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:14.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.474+0000 7fb79e59c640 1 --2- 172.21.15.5:0/2954249862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7a0162820 0x7fb7a01f8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:14.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.474+0000 7fb79e59c640 1 -- 172.21.15.5:0/2954249862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a00c99f0 con 0x7fb7a006d910 2024-05-20T18:35:14.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.474+0000 7fb79dd9b640 1 --2- 172.21.15.5:0/2954249862 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7a00b3e80 0x7fb7a014b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:14.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.474+0000 7fb79e59c640 1 --2- 172.21.15.5:0/2954249862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7a006d910 0x7fb7a014aba0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fb79802a2b0 tx=0x7fb7980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:14.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.474+0000 7fb796ffd640 1 -- 172.21.15.5:0/2954249862 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb798037040 con 0x7fb7a006d910 2024-05-20T18:35:14.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.474+0000 7fb796ffd640 1 -- 172.21.15.5:0/2954249862 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb798042070 con 0x7fb7a006d910 2024-05-20T18:35:14.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.474+0000 7fb7a4856640 1 -- 172.21.15.5:0/2954249862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7a00c9cd0 con 0x7fb7a006d910 2024-05-20T18:35:14.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.474+0000 7fb796ffd640 1 -- 172.21.15.5:0/2954249862 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb798030880 con 0x7fb7a006d910 2024-05-20T18:35:14.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.475+0000 7fb7a4856640 1 -- 172.21.15.5:0/2954249862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7a00cbd20 con 0x7fb7a006d910 2024-05-20T18:35:14.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.476+0000 7fb7a4856640 1 -- 172.21.15.5:0/2954249862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7a0075b00 con 0x7fb7a006d910 2024-05-20T18:35:14.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.476+0000 7fb796ffd640 1 -- 172.21.15.5:0/2954249862 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb798030a20 con 0x7fb7a006d910 2024-05-20T18:35:14.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.477+0000 7fb796ffd640 1 --2- 172.21.15.5:0/2954249862 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb78006be30 0x7fb78006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:14.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.477+0000 7fb796ffd640 1 -- 172.21.15.5:0/2954249862 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7fb7980aba00 con 0x7fb7a006d910 2024-05-20T18:35:14.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.477+0000 7fb79dd9b640 1 --2- 172.21.15.5:0/2954249862 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb78006be30 0x7fb78006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:14.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.477+0000 7fb79dd9b640 1 --2- 172.21.15.5:0/2954249862 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb78006be30 0x7fb78006e340 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb7a01f86d0 tx=0x7fb788008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:14.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.481+0000 7fb796ffd640 1 -- 172.21.15.5:0/2954249862 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb79807a060 con 0x7fb7a006d910 2024-05-20T18:35:14.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.664+0000 7fb7a4856640 1 -- 172.21.15.5:0/2954249862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb7a0160a90 con 0x7fb7a006d910 2024-05-20T18:35:14.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.664+0000 7fb796ffd640 1 -- 172.21.15.5:0/2954249862 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fb798079960 con 0x7fb7a006d910 2024-05-20T18:35:14.666 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:35:14.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.666+0000 7fb7a4856640 1 -- 172.21.15.5:0/2954249862 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb78006be30 msgr2=0x7fb78006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:14.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.666+0000 7fb7a4856640 1 --2- 172.21.15.5:0/2954249862 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb78006be30 0x7fb78006e340 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb7a01f86d0 tx=0x7fb788008040 comp rx=0 tx=0).stop 2024-05-20T18:35:14.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.666+0000 7fb7a4856640 1 -- 172.21.15.5:0/2954249862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7a006d910 msgr2=0x7fb7a014aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:14.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.666+0000 7fb7a4856640 1 --2- 172.21.15.5:0/2954249862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7a006d910 0x7fb7a014aba0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fb79802a2b0 tx=0x7fb7980046c0 comp rx=0 tx=0).stop 2024-05-20T18:35:14.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.666+0000 7fb7a4856640 1 -- 172.21.15.5:0/2954249862 shutdown_connections 2024-05-20T18:35:14.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.666+0000 7fb7a4856640 1 --2- 172.21.15.5:0/2954249862 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb78006be30 0x7fb78006e340 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:14.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.666+0000 7fb7a4856640 1 --2- 172.21.15.5:0/2954249862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7a0162820 0x7fb7a01f8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:14.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.666+0000 7fb7a4856640 1 --2- 172.21.15.5:0/2954249862 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7a00b3e80 0x7fb7a014b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:14.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.666+0000 7fb7a4856640 1 --2- 172.21.15.5:0/2954249862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7a006d910 0x7fb7a014aba0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:14.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.666+0000 7fb7a4856640 1 -- 172.21.15.5:0/2954249862 >> 172.21.15.5:0/2954249862 conn(0x7fb7a00d4bd0 msgr2=0x7fb7a00b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:14.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.667+0000 7fb7a4856640 1 -- 172.21.15.5:0/2954249862 shutdown_connections 2024-05-20T18:35:14.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.667+0000 7fb7a4856640 1 -- 172.21.15.5:0/2954249862 wait complete. 2024-05-20T18:35:14.679 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:35:14.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.796+0000 7f7ffca32640 1 Processor -- start 2024-05-20T18:35:14.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.796+0000 7f7ffca32640 1 -- start start 2024-05-20T18:35:14.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.798+0000 7f7ffca32640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7ff8168050 0x7f7ff816a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:14.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.798+0000 7f7ffca32640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ff80d0630 con 0x7f7ff806d3d0 2024-05-20T18:35:14.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.798+0000 7f7ffca32640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ff80d39c0 con 0x7f7ff8162820 2024-05-20T18:35:14.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.798+0000 7f7ffca32640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f7ff80d3ca0 con 0x7f7ff8168050 2024-05-20T18:35:14.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.798+0000 7f7ff5d9b640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7ff806d3d0 0x7f7ff806d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47296/0 (socket says 172.21.15.5:47296) 2024-05-20T18:35:14.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.798+0000 7f7ff5d9b640 1 -- 172.21.15.5:0/3393235890 learned_addr learned my addr 172.21.15.5:0/3393235890 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:14.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.798+0000 7f7ff6d9d640 1 --2- 172.21.15.5:0/3393235890 >> v2:172.21.15.5:3300/0 conn(0x7f7ff8168050 0x7f7ff816a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:14.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.798+0000 7f7ff559a640 1 -- 172.21.15.5:0/3393235890 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2681873210 0 0) 0x7f7ff80d0630 con 0x7f7ff806d3d0 2024-05-20T18:35:14.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.798+0000 7f7ff559a640 1 -- 172.21.15.5:0/3393235890 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fe4003520 con 0x7f7ff806d3d0 2024-05-20T18:35:14.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.798+0000 7f7ff6d9d640 1 -- 172.21.15.5:0/3393235890 >> v1:172.21.15.63:6789/0 conn(0x7f7ff8162820 legacy=0x7f7ff80b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:14.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.798+0000 7f7ff6d9d640 1 -- 172.21.15.5:0/3393235890 >> v1:172.21.15.5:6789/0 conn(0x7f7ff806d3d0 legacy=0x7f7ff806d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:14.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.798+0000 7f7ff6d9d640 1 -- 172.21.15.5:0/3393235890 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff80d3f80 con 0x7f7ff8168050 2024-05-20T18:35:14.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.799+0000 7f7ff6d9d640 1 --2- 172.21.15.5:0/3393235890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ff8168050 0x7f7ff816a470 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f7ff00097a0 tx=0x7f7ff002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b58bc106fe484a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:14.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.799+0000 7f7ff559a640 1 -- 172.21.15.5:0/3393235890 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0037070 con 0x7f7ff8168050 2024-05-20T18:35:14.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.799+0000 7f7ff559a640 1 -- 172.21.15.5:0/3393235890 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff0032440 con 0x7f7ff8168050 2024-05-20T18:35:14.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.799+0000 7f7ff559a640 1 -- 172.21.15.5:0/3393235890 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0032810 con 0x7f7ff8168050 2024-05-20T18:35:14.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.800+0000 7f7ffca32640 1 -- 172.21.15.5:0/3393235890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ff8168050 msgr2=0x7f7ff816a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:14.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.800+0000 7f7ffca32640 1 --2- 172.21.15.5:0/3393235890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ff8168050 0x7f7ff816a470 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f7ff00097a0 tx=0x7f7ff002a230 comp rx=0 tx=0).stop 2024-05-20T18:35:14.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.800+0000 7f7ffca32640 1 -- 172.21.15.5:0/3393235890 shutdown_connections 2024-05-20T18:35:14.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.800+0000 7f7ffca32640 1 --2- 172.21.15.5:0/3393235890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ff8168050 0x7f7ff816a470 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:14.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.800+0000 7f7ffca32640 1 -- 172.21.15.5:0/3393235890 >> 172.21.15.5:0/3393235890 conn(0x7f7ff80d4bd0 msgr2=0x7f7ff80d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:14.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.800+0000 7f7ffca32640 1 -- 172.21.15.5:0/3393235890 shutdown_connections 2024-05-20T18:35:14.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.800+0000 7f7ffca32640 1 -- 172.21.15.5:0/3393235890 wait complete. 2024-05-20T18:35:14.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.802+0000 7f7ffca32640 1 Processor -- start 2024-05-20T18:35:14.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.802+0000 7f7ffca32640 1 -- start start 2024-05-20T18:35:14.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.803+0000 7f7ffca32640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ff806d3d0 0x7f7ff815e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:14.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.803+0000 7f7ffca32640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ff8162820 0x7f7ff80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:14.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.803+0000 7f7ffca32640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ff8168050 0x7f7ff8154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:14.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.803+0000 7f7ffca32640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff80b91a0 con 0x7f7ff8168050 2024-05-20T18:35:14.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.803+0000 7f7ffca32640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff80d0300 con 0x7f7ff806d3d0 2024-05-20T18:35:14.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.803+0000 7f7ffca32640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7ff80b4050 con 0x7f7ff8162820 2024-05-20T18:35:14.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.804+0000 7f7ff6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ff8168050 0x7f7ff8154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:14.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.804+0000 7f7ff6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ff8168050 0x7f7ff8154c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55484/0 (socket says 172.21.15.5:55484) 2024-05-20T18:35:14.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.804+0000 7f7ff6d9d640 1 -- 172.21.15.5:0/839982019 learned_addr learned my addr 172.21.15.5:0/839982019 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:14.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.804+0000 7f7ff659c640 1 --2- 172.21.15.5:0/839982019 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ff806d3d0 0x7f7ff815e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:14.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.804+0000 7f7ff5d9b640 1 --2- 172.21.15.5:0/839982019 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ff8162820 0x7f7ff80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:14.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.804+0000 7f7ff6d9d640 1 -- 172.21.15.5:0/839982019 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ff8162820 msgr2=0x7f7ff80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:14.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.804+0000 7f7ff6d9d640 1 --2- 172.21.15.5:0/839982019 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ff8162820 0x7f7ff80b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:14.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.804+0000 7f7ff6d9d640 1 -- 172.21.15.5:0/839982019 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ff806d3d0 msgr2=0x7f7ff815e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:14.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.804+0000 7f7ff6d9d640 1 --2- 172.21.15.5:0/839982019 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ff806d3d0 0x7f7ff815e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:14.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.804+0000 7f7ff6d9d640 1 -- 172.21.15.5:0/839982019 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff815f3b0 con 0x7f7ff8168050 2024-05-20T18:35:14.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.804+0000 7f7ff6d9d640 1 --2- 172.21.15.5:0/839982019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ff8168050 0x7f7ff8154c20 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0035040 tx=0x7f7ff00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:14.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.804+0000 7f7feeffd640 1 -- 172.21.15.5:0/839982019 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0037040 con 0x7f7ff8168050 2024-05-20T18:35:14.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.804+0000 7f7feeffd640 1 -- 172.21.15.5:0/839982019 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff0042070 con 0x7f7ff8168050 2024-05-20T18:35:14.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.804+0000 7f7ffca32640 1 -- 172.21.15.5:0/839982019 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ff815dd70 con 0x7f7ff8168050 2024-05-20T18:35:14.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.804+0000 7f7feeffd640 1 -- 172.21.15.5:0/839982019 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff00308a0 con 0x7f7ff8168050 2024-05-20T18:35:14.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.804+0000 7f7ffca32640 1 -- 172.21.15.5:0/839982019 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ff80b6480 con 0x7f7ff8168050 2024-05-20T18:35:14.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.805+0000 7f7ffca32640 1 -- 172.21.15.5:0/839982019 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ff8075b00 con 0x7f7ff8168050 2024-05-20T18:35:14.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.806+0000 7f7feeffd640 1 -- 172.21.15.5:0/839982019 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7ff0030a40 con 0x7f7ff8168050 2024-05-20T18:35:14.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.807+0000 7f7feeffd640 1 --2- 172.21.15.5:0/839982019 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7fd806be60 0x7f7fd806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:14.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.807+0000 7f7feeffd640 1 -- 172.21.15.5:0/839982019 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7f7ff00aba10 con 0x7f7ff8168050 2024-05-20T18:35:14.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.807+0000 7f7ff659c640 1 --2- 172.21.15.5:0/839982019 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7fd806be60 0x7f7fd806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:14.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.807+0000 7f7ff659c640 1 --2- 172.21.15.5:0/839982019 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7fd806be60 0x7f7fd806e370 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8001ca0 tx=0x7f7fe8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:14.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.811+0000 7f7feeffd640 1 -- 172.21.15.5:0/839982019 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7ff007a110 con 0x7f7ff8168050 2024-05-20T18:35:14.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.993+0000 7f7ffca32640 1 -- 172.21.15.5:0/839982019 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7ff815d4e0 con 0x7f7ff8168050 2024-05-20T18:35:14.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.994+0000 7f7feeffd640 1 -- 172.21.15.5:0/839982019 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f7ff0079a10 con 0x7f7ff8168050 2024-05-20T18:35:14.995 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:35:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.996+0000 7f7ffca32640 1 -- 172.21.15.5:0/839982019 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7fd806be60 msgr2=0x7f7fd806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.996+0000 7f7ffca32640 1 --2- 172.21.15.5:0/839982019 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7fd806be60 0x7f7fd806e370 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8001ca0 tx=0x7f7fe8009040 comp rx=0 tx=0).stop 2024-05-20T18:35:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.996+0000 7f7ffca32640 1 -- 172.21.15.5:0/839982019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ff8168050 msgr2=0x7f7ff8154c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.996+0000 7f7ffca32640 1 --2- 172.21.15.5:0/839982019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ff8168050 0x7f7ff8154c20 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0035040 tx=0x7f7ff00046c0 comp rx=0 tx=0).stop 2024-05-20T18:35:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.996+0000 7f7ffca32640 1 -- 172.21.15.5:0/839982019 shutdown_connections 2024-05-20T18:35:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.996+0000 7f7ffca32640 1 --2- 172.21.15.5:0/839982019 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7fd806be60 0x7f7fd806e370 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.996+0000 7f7ffca32640 1 --2- 172.21.15.5:0/839982019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ff8168050 0x7f7ff8154c20 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.996+0000 7f7ffca32640 1 --2- 172.21.15.5:0/839982019 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7ff8162820 0x7f7ff80b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.996+0000 7f7ffca32640 1 --2- 172.21.15.5:0/839982019 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7ff806d3d0 0x7f7ff815e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.996+0000 7f7ffca32640 1 -- 172.21.15.5:0/839982019 >> 172.21.15.5:0/839982019 conn(0x7f7ff80d4bd0 msgr2=0x7f7ff8161be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:14.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.996+0000 7f7ffca32640 1 -- 172.21.15.5:0/839982019 shutdown_connections 2024-05-20T18:35:14.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:14.996+0000 7f7ffca32640 1 -- 172.21.15.5:0/839982019 wait complete. 2024-05-20T18:35:15.009 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T18:35:15.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.124+0000 7fef1f759640 1 Processor -- start 2024-05-20T18:35:15.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.124+0000 7fef1f759640 1 -- start start 2024-05-20T18:35:15.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.126+0000 7fef1f759640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef18159ca0 0x7fef180af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:15.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.126+0000 7fef1f759640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fef180b1650 0x7fef180b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:15.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.126+0000 7fef1f759640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef1809a740 con 0x7fef1806c630 2024-05-20T18:35:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.126+0000 7fef1f759640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fef1809a4f0 con 0x7fef180b1650 2024-05-20T18:35:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.126+0000 7fef1f759640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fef1809b350 con 0x7fef18159ca0 2024-05-20T18:35:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.126+0000 7fef1ccf4640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fef180b1650 0x7fef180b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.126+0000 7fef1ccf4640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fef180b1650 0x7fef180b1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55492/0 (socket says 172.21.15.5:55492) 2024-05-20T18:35:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.126+0000 7fef1ccf4640 1 -- 172.21.15.5:0/1189014735 learned_addr learned my addr 172.21.15.5:0/1189014735 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.126+0000 7fef1d4f5640 1 --2- 172.21.15.5:0/1189014735 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef18159ca0 0x7fef180af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.126+0000 7fef1ccf4640 1 -- 172.21.15.5:0/1189014735 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef18159ca0 msgr2=0x7fef180af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.126+0000 7fef1ccf4640 1 --2- 172.21.15.5:0/1189014735 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef18159ca0 0x7fef180af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.126+0000 7fef1ccf4640 1 -- 172.21.15.5:0/1189014735 >> v1:172.21.15.63:6789/0 conn(0x7fef1806c630 legacy=0x7fef1815c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.126+0000 7fef1ccf4640 1 -- 172.21.15.5:0/1189014735 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef1809aa20 con 0x7fef180b1650 2024-05-20T18:35:15.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.127+0000 7fef1ccf4640 1 --2- 172.21.15.5:0/1189014735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef180b1650 0x7fef180b1ce0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fef08009870 tx=0x7fef0802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e7fcd4e3706897 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:15.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.127+0000 7fef0ffff640 1 -- 172.21.15.5:0/1189014735 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef08037070 con 0x7fef180b1650 2024-05-20T18:35:15.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.127+0000 7fef0ffff640 1 -- 172.21.15.5:0/1189014735 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef08032440 con 0x7fef180b1650 2024-05-20T18:35:15.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.127+0000 7fef0ffff640 1 -- 172.21.15.5:0/1189014735 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef08032810 con 0x7fef180b1650 2024-05-20T18:35:15.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.128+0000 7fef1f759640 1 -- 172.21.15.5:0/1189014735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef180b1650 msgr2=0x7fef180b1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:15.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.128+0000 7fef1f759640 1 --2- 172.21.15.5:0/1189014735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef180b1650 0x7fef180b1ce0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fef08009870 tx=0x7fef0802a1c0 comp rx=0 tx=0).stop 2024-05-20T18:35:15.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.128+0000 7fef1f759640 1 -- 172.21.15.5:0/1189014735 shutdown_connections 2024-05-20T18:35:15.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.128+0000 7fef1f759640 1 --2- 172.21.15.5:0/1189014735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef180b1650 0x7fef180b1ce0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:15.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.128+0000 7fef1f759640 1 --2- 172.21.15.5:0/1189014735 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef18159ca0 0x7fef180af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.128+0000 7fef1f759640 1 -- 172.21.15.5:0/1189014735 >> 172.21.15.5:0/1189014735 conn(0x7fef180b6620 msgr2=0x7fef180b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.128+0000 7fef1f759640 1 -- 172.21.15.5:0/1189014735 shutdown_connections 2024-05-20T18:35:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.128+0000 7fef1f759640 1 -- 172.21.15.5:0/1189014735 wait complete. 2024-05-20T18:35:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.130+0000 7fef1f759640 1 Processor -- start 2024-05-20T18:35:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.130+0000 7fef1f759640 1 -- start start 2024-05-20T18:35:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.131+0000 7fef1f759640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef1806c630 0x7fef1806b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.131+0000 7fef1f759640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef180b1650 0x7fef1809b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.131+0000 7fef1f759640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef18159ca0 0x7fef1809be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.131+0000 7fef1f759640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fef1809afe0 con 0x7fef1806c630 2024-05-20T18:35:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.131+0000 7fef1f759640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fef180af770 con 0x7fef180b1650 2024-05-20T18:35:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.131+0000 7fef1f759640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fef180b2220 con 0x7fef18159ca0 2024-05-20T18:35:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.131+0000 7fef1d4f5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef1806c630 0x7fef1806b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.131+0000 7fef1d4f5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef1806c630 0x7fef1806b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55504/0 (socket says 172.21.15.5:55504) 2024-05-20T18:35:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.131+0000 7fef1d4f5640 1 -- 172.21.15.5:0/478500803 learned_addr learned my addr 172.21.15.5:0/478500803 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.132+0000 7fef1ccf4640 1 --2- 172.21.15.5:0/478500803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef180b1650 0x7fef1809b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.132+0000 7fef1dcf6640 1 --2- 172.21.15.5:0/478500803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef18159ca0 0x7fef1809be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.132+0000 7fef1d4f5640 1 -- 172.21.15.5:0/478500803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef18159ca0 msgr2=0x7fef1809be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.132+0000 7fef1d4f5640 1 --2- 172.21.15.5:0/478500803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef18159ca0 0x7fef1809be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.132+0000 7fef1d4f5640 1 -- 172.21.15.5:0/478500803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef180b1650 msgr2=0x7fef1809b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.132+0000 7fef1d4f5640 1 --2- 172.21.15.5:0/478500803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef180b1650 0x7fef1809b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.132+0000 7fef1d4f5640 1 -- 172.21.15.5:0/478500803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef1809c5d0 con 0x7fef1806c630 2024-05-20T18:35:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.132+0000 7fef1ccf4640 1 --2- 172.21.15.5:0/478500803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef180b1650 0x7fef1809b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.132+0000 7fef1d4f5640 1 --2- 172.21.15.5:0/478500803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef1806c630 0x7fef1806b700 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fef1400e8a0 tx=0x7fef1400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.132+0000 7fef0dffb640 1 -- 172.21.15.5:0/478500803 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14017070 con 0x7fef1806c630 2024-05-20T18:35:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.132+0000 7fef0dffb640 1 -- 172.21.15.5:0/478500803 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef1400c940 con 0x7fef1806c630 2024-05-20T18:35:15.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.132+0000 7fef1f759640 1 -- 172.21.15.5:0/478500803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef1809c8b0 con 0x7fef1806c630 2024-05-20T18:35:15.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.132+0000 7fef0dffb640 1 -- 172.21.15.5:0/478500803 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1400cd30 con 0x7fef1806c630 2024-05-20T18:35:15.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.132+0000 7fef1f759640 1 -- 172.21.15.5:0/478500803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef1809ce70 con 0x7fef1806c630 2024-05-20T18:35:15.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.133+0000 7fef1f759640 1 -- 172.21.15.5:0/478500803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef18075b00 con 0x7fef1806c630 2024-05-20T18:35:15.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.134+0000 7fef0dffb640 1 -- 172.21.15.5:0/478500803 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fef1401a070 con 0x7fef1806c630 2024-05-20T18:35:15.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.134+0000 7fef0dffb640 1 --2- 172.21.15.5:0/478500803 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feefc06be30 0x7feefc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:15.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.135+0000 7fef0dffb640 1 -- 172.21.15.5:0/478500803 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7fef1408b490 con 0x7fef1806c630 2024-05-20T18:35:15.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.135+0000 7fef1ccf4640 1 --2- 172.21.15.5:0/478500803 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feefc06be30 0x7feefc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:15.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.135+0000 7fef1ccf4640 1 --2- 172.21.15.5:0/478500803 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feefc06be30 0x7feefc06e340 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fef08034000 tx=0x7fef0802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.139+0000 7fef0dffb640 1 -- 172.21.15.5:0/478500803 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fef14059a60 con 0x7fef1806c630 2024-05-20T18:35:15.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.333+0000 7fef1f759640 1 -- 172.21.15.5:0/478500803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fef18067df0 con 0x7fef1806c630 2024-05-20T18:35:15.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.333+0000 7fef0dffb640 1 -- 172.21.15.5:0/478500803 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v94) ==== 80+0+242 (secure 0 0 0) 0x7fef14059360 con 0x7fef1806c630 2024-05-20T18:35:15.334 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6837/3556492262 2024-05-21T18:34:35.299705+0000 2024-05-20T18:35:15.335 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6836/3556492262 2024-05-21T18:34:35.299705+0000 2024-05-20T18:35:15.335 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.63:6834/2759396130 2024-05-21T18:33:23.233035+0000 2024-05-20T18:35:15.335 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.63:6832/2759396130 2024-05-21T18:33:23.233035+0000 2024-05-20T18:35:15.335 INFO:teuthology.orchestra.run.smithi005.stderr:listed 4 entries 2024-05-20T18:35:15.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.335+0000 7fef1f759640 1 -- 172.21.15.5:0/478500803 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feefc06be30 msgr2=0x7feefc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:15.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.335+0000 7fef1f759640 1 --2- 172.21.15.5:0/478500803 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feefc06be30 0x7feefc06e340 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fef08034000 tx=0x7fef0802e000 comp rx=0 tx=0).stop 2024-05-20T18:35:15.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.335+0000 7fef1f759640 1 -- 172.21.15.5:0/478500803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef1806c630 msgr2=0x7fef1806b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:15.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.335+0000 7fef1f759640 1 --2- 172.21.15.5:0/478500803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef1806c630 0x7fef1806b700 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fef1400e8a0 tx=0x7fef1400ed60 comp rx=0 tx=0).stop 2024-05-20T18:35:15.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.335+0000 7fef1f759640 1 -- 172.21.15.5:0/478500803 shutdown_connections 2024-05-20T18:35:15.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.335+0000 7fef1f759640 1 --2- 172.21.15.5:0/478500803 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feefc06be30 0x7feefc06e340 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:15.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.335+0000 7fef1f759640 1 --2- 172.21.15.5:0/478500803 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef18159ca0 0x7fef1809be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:15.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.335+0000 7fef1f759640 1 --2- 172.21.15.5:0/478500803 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef180b1650 0x7fef1809b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:15.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.335+0000 7fef1f759640 1 --2- 172.21.15.5:0/478500803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef1806c630 0x7fef1806b700 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:15.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.335+0000 7fef1f759640 1 -- 172.21.15.5:0/478500803 >> 172.21.15.5:0/478500803 conn(0x7fef180b6620 msgr2=0x7fef180af920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:15.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.335+0000 7fef1f759640 1 -- 172.21.15.5:0/478500803 shutdown_connections 2024-05-20T18:35:15.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.335+0000 7fef1f759640 1 -- 172.21.15.5:0/478500803 wait complete. 2024-05-20T18:35:15.348 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-20T18:35:15.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:35:15.348 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-20T18:35:15.378 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:35:15.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:35:15.378 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-20T18:35:15.434 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T18:35:15.434 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:35:15.434 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:35:15.455 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19732 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T18:35:15.455 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T18:35:15.455 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:35:15.455 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:35:15.490 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15663 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T18:35:15.490 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T18:35:15.490 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:35:15.490 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:35:15.515 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19732 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T18:35:15.516 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:35:15.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.643+0000 7fb9c41fe640 1 Processor -- start 2024-05-20T18:35:15.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.643+0000 7fb9c41fe640 1 -- start start 2024-05-20T18:35:15.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.644+0000 7fb9c41fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9bc15d530 0x7fb9bc15aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:15.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.644+0000 7fb9c41fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb9bc0b1700 0x7fb9bc0b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:15.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.644+0000 7fb9c41fe640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9bc0cd580 con 0x7fb9bc070850 2024-05-20T18:35:15.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.644+0000 7fb9c41fe640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fb9bc0ce190 con 0x7fb9bc0b1700 2024-05-20T18:35:15.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.644+0000 7fb9c41fe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb9bc0cd860 con 0x7fb9bc15d530 2024-05-20T18:35:15.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.645+0000 7fb9c1799640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb9bc0b1700 0x7fb9bc0b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:15.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.645+0000 7fb9c279b640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb9bc070850 0x7fb9bc070c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47312/0 (socket says 172.21.15.5:47312) 2024-05-20T18:35:15.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.645+0000 7fb9c279b640 1 -- 172.21.15.5:0/3119998262 learned_addr learned my addr 172.21.15.5:0/3119998262 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:15.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.645+0000 7fb9c1799640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb9bc0b1700 0x7fb9bc0b1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55508/0 (socket says 172.21.15.5:55508) 2024-05-20T18:35:15.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.645+0000 7fb9c1f9a640 1 --2- 172.21.15.5:0/3119998262 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9bc15d530 0x7fb9bc15aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:15.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.645+0000 7fb9c1799640 1 -- 172.21.15.5:0/3119998262 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9bc15d530 msgr2=0x7fb9bc15aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:15.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.645+0000 7fb9c1799640 1 --2- 172.21.15.5:0/3119998262 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9bc15d530 0x7fb9bc15aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:15.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.645+0000 7fb9c1799640 1 -- 172.21.15.5:0/3119998262 >> v1:172.21.15.5:6789/0 conn(0x7fb9bc070850 legacy=0x7fb9bc070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:15.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.645+0000 7fb9c1799640 1 -- 172.21.15.5:0/3119998262 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9bc0cdb40 con 0x7fb9bc0b1700 2024-05-20T18:35:15.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.645+0000 7fb9c0f98640 1 -- 172.21.15.5:0/3119998262 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3899770547 0 0) 0x7fb9bc0cd580 con 0x7fb9bc070850 2024-05-20T18:35:15.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.645+0000 7fb9c1f9a640 1 --2- 172.21.15.5:0/3119998262 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9bc15d530 0x7fb9bc15aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:15.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.646+0000 7fb9c1799640 1 --2- 172.21.15.5:0/3119998262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9bc0b1700 0x7fb9bc0b1b10 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0097a0 tx=0x7fb9ac02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4030eb051668c6e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:15.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.646+0000 7fb9c0f98640 1 -- 172.21.15.5:0/3119998262 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac037070 con 0x7fb9bc0b1700 2024-05-20T18:35:15.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.646+0000 7fb9c0f98640 1 -- 172.21.15.5:0/3119998262 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9ac032440 con 0x7fb9bc0b1700 2024-05-20T18:35:15.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.646+0000 7fb9c0f98640 1 -- 172.21.15.5:0/3119998262 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac032810 con 0x7fb9bc0b1700 2024-05-20T18:35:15.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.647+0000 7fb9c41fe640 1 -- 172.21.15.5:0/3119998262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9bc0b1700 msgr2=0x7fb9bc0b1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:15.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.647+0000 7fb9c41fe640 1 --2- 172.21.15.5:0/3119998262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9bc0b1700 0x7fb9bc0b1b10 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0097a0 tx=0x7fb9ac02a180 comp rx=0 tx=0).stop 2024-05-20T18:35:15.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.647+0000 7fb9c41fe640 1 -- 172.21.15.5:0/3119998262 shutdown_connections 2024-05-20T18:35:15.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.647+0000 7fb9c41fe640 1 --2- 172.21.15.5:0/3119998262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9bc0b1700 0x7fb9bc0b1b10 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:15.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.647+0000 7fb9c41fe640 1 --2- 172.21.15.5:0/3119998262 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9bc15d530 0x7fb9bc15aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:15.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.647+0000 7fb9c41fe640 1 -- 172.21.15.5:0/3119998262 >> 172.21.15.5:0/3119998262 conn(0x7fb9bc0d29d0 msgr2=0x7fb9bc0d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:15.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.647+0000 7fb9c41fe640 1 -- 172.21.15.5:0/3119998262 shutdown_connections 2024-05-20T18:35:15.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.647+0000 7fb9c41fe640 1 -- 172.21.15.5:0/3119998262 wait complete. 2024-05-20T18:35:15.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.649+0000 7fb9c41fe640 1 Processor -- start 2024-05-20T18:35:15.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.649+0000 7fb9c41fe640 1 -- start start 2024-05-20T18:35:15.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.650+0000 7fb9c41fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9bc070850 0x7fb9bc158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:15.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.650+0000 7fb9c41fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9bc0b1700 0x7fb9bc0c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:15.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.650+0000 7fb9c41fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9bc15d530 0x7fb9bc1f87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:15.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.650+0000 7fb9c41fe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc0b2050 con 0x7fb9bc0b1700 2024-05-20T18:35:15.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.650+0000 7fb9c41fe640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc0cd230 con 0x7fb9bc070850 2024-05-20T18:35:15.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.650+0000 7fb9c41fe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb9bc071370 con 0x7fb9bc15d530 2024-05-20T18:35:15.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.650+0000 7fb9c1799640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9bc0b1700 0x7fb9bc0c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:15.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.650+0000 7fb9c1799640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9bc0b1700 0x7fb9bc0c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55522/0 (socket says 172.21.15.5:55522) 2024-05-20T18:35:15.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.650+0000 7fb9c1799640 1 -- 172.21.15.5:0/3687694234 learned_addr learned my addr 172.21.15.5:0/3687694234 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:15.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.650+0000 7fb9c1f9a640 1 --2- 172.21.15.5:0/3687694234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9bc070850 0x7fb9bc158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:15.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.650+0000 7fb9c279b640 1 --2- 172.21.15.5:0/3687694234 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9bc15d530 0x7fb9bc1f87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:15.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.650+0000 7fb9c1799640 1 -- 172.21.15.5:0/3687694234 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9bc15d530 msgr2=0x7fb9bc1f87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:15.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.650+0000 7fb9c1799640 1 --2- 172.21.15.5:0/3687694234 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9bc15d530 0x7fb9bc1f87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:15.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.650+0000 7fb9c1799640 1 -- 172.21.15.5:0/3687694234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9bc070850 msgr2=0x7fb9bc158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:15.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.650+0000 7fb9c1799640 1 --2- 172.21.15.5:0/3687694234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9bc070850 0x7fb9bc158380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:15.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.650+0000 7fb9c1799640 1 -- 172.21.15.5:0/3687694234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9bc0c70b0 con 0x7fb9bc0b1700 2024-05-20T18:35:15.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.650+0000 7fb9c279b640 1 --2- 172.21.15.5:0/3687694234 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9bc15d530 0x7fb9bc1f87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:15.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.651+0000 7fb9c1799640 1 --2- 172.21.15.5:0/3687694234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9bc0b1700 0x7fb9bc0c8cc0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0098c0 tx=0x7fb9ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:15.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.651+0000 7fb9b27fc640 1 -- 172.21.15.5:0/3687694234 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac037040 con 0x7fb9bc0b1700 2024-05-20T18:35:15.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.651+0000 7fb9b27fc640 1 -- 172.21.15.5:0/3687694234 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9ac042070 con 0x7fb9bc0b1700 2024-05-20T18:35:15.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.651+0000 7fb9b27fc640 1 -- 172.21.15.5:0/3687694234 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac030a70 con 0x7fb9bc0b1700 2024-05-20T18:35:15.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.651+0000 7fb9c41fe640 1 -- 172.21.15.5:0/3687694234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9bc0c7390 con 0x7fb9bc0b1700 2024-05-20T18:35:15.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.651+0000 7fb9c41fe640 1 -- 172.21.15.5:0/3687694234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9bc0b64b0 con 0x7fb9bc0b1700 2024-05-20T18:35:15.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.652+0000 7fb9c41fe640 1 -- 172.21.15.5:0/3687694234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9bc075b00 con 0x7fb9bc0b1700 2024-05-20T18:35:15.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.653+0000 7fb9b27fc640 1 -- 172.21.15.5:0/3687694234 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb9ac030c10 con 0x7fb9bc0b1700 2024-05-20T18:35:15.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.653+0000 7fb9b27fc640 1 --2- 172.21.15.5:0/3687694234 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb9a006be30 0x7fb9a006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:15.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.653+0000 7fb9b27fc640 1 -- 172.21.15.5:0/3687694234 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7fb9ac0ab960 con 0x7fb9bc0b1700 2024-05-20T18:35:15.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.653+0000 7fb9c1f9a640 1 --2- 172.21.15.5:0/3687694234 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb9a006be30 0x7fb9a006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:15.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.654+0000 7fb9c1f9a640 1 --2- 172.21.15.5:0/3687694234 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb9a006be30 0x7fb9a006e340 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc0d29a0 tx=0x7fb9b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:15.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.658+0000 7fb9b27fc640 1 -- 172.21.15.5:0/3687694234 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb9ac04d060 con 0x7fb9bc0b1700 2024-05-20T18:35:15.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.882+0000 7fb9c41fe640 1 -- 172.21.15.5:0/3687694234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9bc067ce0 con 0x7fb9bc0b1700 2024-05-20T18:35:15.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.883+0000 7fb9b27fc640 1 -- 172.21.15.5:0/3687694234 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5172 (secure 0 0 0) 0x7fb9ac045360 con 0x7fb9bc0b1700 2024-05-20T18:35:15.884 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:15.884 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/337248865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":337248865},{"type":"v1","addr":"172.21.15.63:6834","nonce":337248865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3705772553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3705772553},{"type":"v1","addr":"172.21.15.63:6835","nonce":3705772553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":7434,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/776188138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":776188138},{"type":"v1","addr":"172.21.15.5:6835","nonce":776188138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:33:12.457716+0000","modified":"2024-05-20T18:34:39.318498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7953":{"gid":7953,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6837/3880006778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3880006778},{"type":"v1","addr":"172.21.15.5:6837","nonce":3880006778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7953,"qdb_cluster":[7953]},"id":6}]} 2024-05-20T18:35:15.884 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 73 2024-05-20T18:35:15.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.885+0000 7fb9c41fe640 1 -- 172.21.15.5:0/3687694234 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb9a006be30 msgr2=0x7fb9a006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:15.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.885+0000 7fb9c41fe640 1 --2- 172.21.15.5:0/3687694234 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb9a006be30 0x7fb9a006e340 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc0d29a0 tx=0x7fb9b8008040 comp rx=0 tx=0).stop 2024-05-20T18:35:15.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.885+0000 7fb9c41fe640 1 -- 172.21.15.5:0/3687694234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9bc0b1700 msgr2=0x7fb9bc0c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:15.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.885+0000 7fb9c41fe640 1 --2- 172.21.15.5:0/3687694234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9bc0b1700 0x7fb9bc0c8cc0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0098c0 tx=0x7fb9ac0046c0 comp rx=0 tx=0).stop 2024-05-20T18:35:15.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.885+0000 7fb9c41fe640 1 -- 172.21.15.5:0/3687694234 shutdown_connections 2024-05-20T18:35:15.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.885+0000 7fb9c41fe640 1 --2- 172.21.15.5:0/3687694234 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb9a006be30 0x7fb9a006e340 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:15.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.885+0000 7fb9c41fe640 1 --2- 172.21.15.5:0/3687694234 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9bc15d530 0x7fb9bc1f87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:15.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.885+0000 7fb9c41fe640 1 --2- 172.21.15.5:0/3687694234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9bc0b1700 0x7fb9bc0c8cc0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:15.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.885+0000 7fb9c41fe640 1 --2- 172.21.15.5:0/3687694234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9bc070850 0x7fb9bc158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:15.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.885+0000 7fb9c41fe640 1 -- 172.21.15.5:0/3687694234 >> 172.21.15.5:0/3687694234 conn(0x7fb9bc0d29d0 msgr2=0x7fb9bc15fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:15.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.885+0000 7fb9c41fe640 1 -- 172.21.15.5:0/3687694234 shutdown_connections 2024-05-20T18:35:15.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:15.885+0000 7fb9c41fe640 1 -- 172.21.15.5:0/3687694234 wait complete. 2024-05-20T18:35:15.897 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:35:16.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.011+0000 7fad64fb5640 1 Processor -- start 2024-05-20T18:35:16.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.011+0000 7fad64fb5640 1 -- start start 2024-05-20T18:35:16.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.013+0000 7fad64fb5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fad600af030 0x7fad600af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:16.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.013+0000 7fad64fb5640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fad6006c180 0x7fad6006c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:16.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.013+0000 7fad64fb5640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad600af980 con 0x7fad6015f700 2024-05-20T18:35:16.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.013+0000 7fad64fb5640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fad6006bcf0 con 0x7fad600af030 2024-05-20T18:35:16.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.013+0000 7fad64fb5640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fad60160430 con 0x7fad6006c180 2024-05-20T18:35:16.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.013+0000 7fad5e59c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fad600af030 0x7fad600af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:16.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.013+0000 7fad5e59c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fad600af030 0x7fad600af440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55524/0 (socket says 172.21.15.5:55524) 2024-05-20T18:35:16.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.013+0000 7fad5e59c640 1 -- 172.21.15.5:0/3174532115 learned_addr learned my addr 172.21.15.5:0/3174532115 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:16.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.013+0000 7fad5dd9b640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fad6015f700 0x7fad6015fb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47328/0 (socket says 172.21.15.5:47328) 2024-05-20T18:35:16.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.013+0000 7fad5ed9d640 1 --2- 172.21.15.5:0/3174532115 >> v2:172.21.15.63:3300/0 conn(0x7fad6006c180 0x7fad6006c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:16.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.013+0000 7fad5e59c640 1 -- 172.21.15.5:0/3174532115 >> v2:172.21.15.63:3300/0 conn(0x7fad6006c180 msgr2=0x7fad6006c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:16.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.013+0000 7fad5e59c640 1 --2- 172.21.15.5:0/3174532115 >> v2:172.21.15.63:3300/0 conn(0x7fad6006c180 0x7fad6006c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.013+0000 7fad5e59c640 1 -- 172.21.15.5:0/3174532115 >> v1:172.21.15.5:6789/0 conn(0x7fad6015f700 legacy=0x7fad6015fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:16.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.014+0000 7fad5d59a640 1 -- 172.21.15.5:0/3174532115 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 793907807 0 0) 0x7fad48001110 con 0x7fad6015f700 2024-05-20T18:35:16.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.014+0000 7fad5e59c640 1 -- 172.21.15.5:0/3174532115 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6006b6a0 con 0x7fad600af030 2024-05-20T18:35:16.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.014+0000 7fad5ed9d640 1 --2- 172.21.15.5:0/3174532115 >> v2:172.21.15.63:3300/0 conn(0x7fad6006c180 0x7fad6006c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.014+0000 7fad5e59c640 1 --2- 172.21.15.5:0/3174532115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad600af030 0x7fad600af440 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fad60072cd0 tx=0x7fad5802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a601d7aa6de74e42 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.014+0000 7fad5d59a640 1 -- 172.21.15.5:0/3174532115 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad58036070 con 0x7fad600af030 2024-05-20T18:35:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.015+0000 7fad5d59a640 1 -- 172.21.15.5:0/3174532115 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad58031600 con 0x7fad600af030 2024-05-20T18:35:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.015+0000 7fad5d59a640 1 -- 172.21.15.5:0/3174532115 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad580319d0 con 0x7fad600af030 2024-05-20T18:35:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.015+0000 7fad64fb5640 1 -- 172.21.15.5:0/3174532115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad600af030 msgr2=0x7fad600af440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:16.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.015+0000 7fad64fb5640 1 --2- 172.21.15.5:0/3174532115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad600af030 0x7fad600af440 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fad60072cd0 tx=0x7fad5802a730 comp rx=0 tx=0).stop 2024-05-20T18:35:16.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.015+0000 7fad64fb5640 1 -- 172.21.15.5:0/3174532115 shutdown_connections 2024-05-20T18:35:16.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.015+0000 7fad64fb5640 1 --2- 172.21.15.5:0/3174532115 >> v2:172.21.15.63:3300/0 conn(0x7fad6006c180 0x7fad6006c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.015+0000 7fad64fb5640 1 --2- 172.21.15.5:0/3174532115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad600af030 0x7fad600af440 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.015+0000 7fad64fb5640 1 -- 172.21.15.5:0/3174532115 >> 172.21.15.5:0/3174532115 conn(0x7fad600b6620 msgr2=0x7fad600b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:16.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.015+0000 7fad64fb5640 1 -- 172.21.15.5:0/3174532115 shutdown_connections 2024-05-20T18:35:16.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.015+0000 7fad64fb5640 1 -- 172.21.15.5:0/3174532115 wait complete. 2024-05-20T18:35:16.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.017+0000 7fad64fb5640 1 Processor -- start 2024-05-20T18:35:16.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.017+0000 7fad64fb5640 1 -- start start 2024-05-20T18:35:16.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.019+0000 7fad64fb5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad6006c180 0x7fad60156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:16.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.019+0000 7fad64fb5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad600af030 0x7fad6009f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:16.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.019+0000 7fad64fb5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad6015f700 0x7fad60157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:16.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.019+0000 7fad64fb5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fad6006cad0 con 0x7fad600af030 2024-05-20T18:35:16.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.019+0000 7fad64fb5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fad60159aa0 con 0x7fad6015f700 2024-05-20T18:35:16.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.019+0000 7fad64fb5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fad6006cc50 con 0x7fad6006c180 2024-05-20T18:35:16.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.019+0000 7fad5dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad600af030 0x7fad6009f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:16.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.019+0000 7fad5dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad600af030 0x7fad6009f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55536/0 (socket says 172.21.15.5:55536) 2024-05-20T18:35:16.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.019+0000 7fad5dd9b640 1 -- 172.21.15.5:0/609494565 learned_addr learned my addr 172.21.15.5:0/609494565 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:16.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.019+0000 7fad5e59c640 1 --2- 172.21.15.5:0/609494565 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad6006c180 0x7fad60156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:16.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.019+0000 7fad5ed9d640 1 --2- 172.21.15.5:0/609494565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad6015f700 0x7fad60157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:16.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.019+0000 7fad5dd9b640 1 -- 172.21.15.5:0/609494565 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad6006c180 msgr2=0x7fad60156260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:16.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.019+0000 7fad5dd9b640 1 --2- 172.21.15.5:0/609494565 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad6006c180 0x7fad60156260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.019+0000 7fad5dd9b640 1 -- 172.21.15.5:0/609494565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad6015f700 msgr2=0x7fad60157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:16.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.019+0000 7fad5dd9b640 1 --2- 172.21.15.5:0/609494565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad6015f700 0x7fad60157c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.019+0000 7fad5dd9b640 1 -- 172.21.15.5:0/609494565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad601567a0 con 0x7fad600af030 2024-05-20T18:35:16.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.019+0000 7fad5dd9b640 1 --2- 172.21.15.5:0/609494565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad600af030 0x7fad6009f260 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fad4800c410 tx=0x7fad4800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:16.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.019+0000 7fad56ffd640 1 -- 172.21.15.5:0/609494565 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad480074e0 con 0x7fad600af030 2024-05-20T18:35:16.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.019+0000 7fad56ffd640 1 -- 172.21.15.5:0/609494565 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad48007680 con 0x7fad600af030 2024-05-20T18:35:16.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.019+0000 7fad64fb5640 1 -- 172.21.15.5:0/609494565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad60159230 con 0x7fad600af030 2024-05-20T18:35:16.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.019+0000 7fad56ffd640 1 -- 172.21.15.5:0/609494565 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48004a00 con 0x7fad600af030 2024-05-20T18:35:16.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.019+0000 7fad64fb5640 1 -- 172.21.15.5:0/609494565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad6009df90 con 0x7fad600af030 2024-05-20T18:35:16.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.020+0000 7fad64fb5640 1 -- 172.21.15.5:0/609494565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad6009da00 con 0x7fad600af030 2024-05-20T18:35:16.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.021+0000 7fad56ffd640 1 -- 172.21.15.5:0/609494565 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fad4801a020 con 0x7fad600af030 2024-05-20T18:35:16.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.022+0000 7fad56ffd640 1 --2- 172.21.15.5:0/609494565 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fad4006be30 0x7fad4006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:16.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.022+0000 7fad5e59c640 1 --2- 172.21.15.5:0/609494565 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fad4006be30 0x7fad4006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:16.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.022+0000 7fad56ffd640 1 -- 172.21.15.5:0/609494565 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7fad4808a970 con 0x7fad600af030 2024-05-20T18:35:16.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.022+0000 7fad5e59c640 1 --2- 172.21.15.5:0/609494565 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fad4006be30 0x7fad4006e340 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fad58033000 tx=0x7fad58009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:16.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.026+0000 7fad56ffd640 1 -- 172.21.15.5:0/609494565 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fad60159230 con 0x7fad600af030 2024-05-20T18:35:16.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.214+0000 7fad64fb5640 1 -- 172.21.15.5:0/609494565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad60075b00 con 0x7fad600af030 2024-05-20T18:35:16.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.215+0000 7fad56ffd640 1 -- 172.21.15.5:0/609494565 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+17216 (secure 0 0 0) 0x7fad600af980 con 0x7fad600af030 2024-05-20T18:35:16.216 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:16.217 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":94,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:34:35.299736+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:33:10.184948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-20T18:33:11.453356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.1e","mappings":[{"from":1,"to":5}]},{"pgid":"14.29","mappings":[{"from":1,"to":7}]},{"pgid":"14.2a","mappings":[{"from":1,"to":2}]},{"pgid":"14.2c","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6837/3556492262":"2024-05-21T18:34:35.299705+0000","172.21.15.5:6836/3556492262":"2024-05-21T18:34:35.299705+0000","172.21.15.63:6834/2759396130":"2024-05-21T18:33:23.233035+0000","172.21.15.63:6832/2759396130":"2024-05-21T18:33:23.233035+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:35:16.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.217+0000 7fad64fb5640 1 -- 172.21.15.5:0/609494565 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fad4006be30 msgr2=0x7fad4006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:16.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.217+0000 7fad64fb5640 1 --2- 172.21.15.5:0/609494565 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fad4006be30 0x7fad4006e340 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fad58033000 tx=0x7fad58009000 comp rx=0 tx=0).stop 2024-05-20T18:35:16.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.217+0000 7fad64fb5640 1 -- 172.21.15.5:0/609494565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad600af030 msgr2=0x7fad6009f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:16.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.217+0000 7fad64fb5640 1 --2- 172.21.15.5:0/609494565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad600af030 0x7fad6009f260 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fad4800c410 tx=0x7fad4800c8d0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.217+0000 7fad64fb5640 1 -- 172.21.15.5:0/609494565 shutdown_connections 2024-05-20T18:35:16.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.217+0000 7fad64fb5640 1 --2- 172.21.15.5:0/609494565 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fad4006be30 0x7fad4006e340 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.217+0000 7fad64fb5640 1 --2- 172.21.15.5:0/609494565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad6015f700 0x7fad60157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.217+0000 7fad64fb5640 1 --2- 172.21.15.5:0/609494565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad600af030 0x7fad6009f260 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.217+0000 7fad64fb5640 1 --2- 172.21.15.5:0/609494565 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad6006c180 0x7fad60156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.217+0000 7fad64fb5640 1 -- 172.21.15.5:0/609494565 >> 172.21.15.5:0/609494565 conn(0x7fad600b6620 msgr2=0x7fad600b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:16.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.217+0000 7fad64fb5640 1 -- 172.21.15.5:0/609494565 shutdown_connections 2024-05-20T18:35:16.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.218+0000 7fad64fb5640 1 -- 172.21.15.5:0/609494565 wait complete. 2024-05-20T18:35:16.230 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:33:10.184948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:33:11.453356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:35:16.231 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:35:16.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.348+0000 7f810c81a640 1 Processor -- start 2024-05-20T18:35:16.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.348+0000 7f810c81a640 1 -- start start 2024-05-20T18:35:16.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.350+0000 7f810c81a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f81080d4400 0x7f810815f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:16.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.350+0000 7f810c81a640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f810808d2a0 con 0x7f81080d2780 2024-05-20T18:35:16.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.350+0000 7f810c81a640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81080d4810 con 0x7f810806cf00 2024-05-20T18:35:16.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.350+0000 7f810c81a640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f81080d3f70 con 0x7f81080d4400 2024-05-20T18:35:16.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.350+0000 7f8106d9d640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f81080d2780 0x7f81080d2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47344/0 (socket says 172.21.15.5:47344) 2024-05-20T18:35:16.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.350+0000 7f8106d9d640 1 -- 172.21.15.5:0/3689494110 learned_addr learned my addr 172.21.15.5:0/3689494110 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:16.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.350+0000 7f810659c640 1 --2- 172.21.15.5:0/3689494110 >> v2:172.21.15.63:3300/0 conn(0x7f81080d4400 0x7f810815f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:16.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.350+0000 7f810559a640 1 -- 172.21.15.5:0/3689494110 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3255600879 0 0) 0x7f80f8001110 con 0x7f81080d2780 2024-05-20T18:35:16.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.350+0000 7f810559a640 1 -- 172.21.15.5:0/3689494110 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80f4003520 con 0x7f81080d2780 2024-05-20T18:35:16.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.350+0000 7f810559a640 1 -- 172.21.15.5:0/3689494110 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 891787516 0 0) 0x7f80f4003520 con 0x7f81080d2780 2024-05-20T18:35:16.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.350+0000 7f810559a640 1 -- 172.21.15.5:0/3689494110 >> v2:172.21.15.63:3300/0 conn(0x7f81080d4400 msgr2=0x7f810815f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:16.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.350+0000 7f810559a640 1 --2- 172.21.15.5:0/3689494110 >> v2:172.21.15.63:3300/0 conn(0x7f81080d4400 0x7f810815f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.350+0000 7f810559a640 1 -- 172.21.15.5:0/3689494110 >> v1:172.21.15.63:6789/0 conn(0x7f810806cf00 legacy=0x7f810806d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:16.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.351+0000 7f810559a640 1 -- 172.21.15.5:0/3689494110 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81080d3360 con 0x7f81080d2780 2024-05-20T18:35:16.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.351+0000 7f810559a640 1 -- 172.21.15.5:0/3689494110 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f80f8001840 con 0x7f81080d2780 2024-05-20T18:35:16.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.351+0000 7f810559a640 1 -- 172.21.15.5:0/3689494110 >> v1:172.21.15.5:6789/0 conn(0x7f81080d2780 legacy=0x7f81080d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:16.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.351+0000 7f810559a640 1 --2- 172.21.15.5:0/3689494110 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80f40039b0 0x7f80f4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:16.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.351+0000 7f810559a640 1 --2- 172.21.15.5:0/3689494110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80f4004800 0x7f80f4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:16.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.351+0000 7f810559a640 1 --2- 172.21.15.5:0/3689494110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80f4005270 0x7f80f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:16.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.351+0000 7f810559a640 1 -- 172.21.15.5:0/3689494110 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81080d3f70 con 0x7f80f4005270 2024-05-20T18:35:16.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.351+0000 7f810559a640 1 -- 172.21.15.5:0/3689494110 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f80f400c180 con 0x7f80f4004800 2024-05-20T18:35:16.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.351+0000 7f810559a640 1 -- 172.21.15.5:0/3689494110 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f80f400c380 con 0x7f80f40039b0 2024-05-20T18:35:16.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.351+0000 7f8106d9d640 1 --2- 172.21.15.5:0/3689494110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80f4005270 0x7f80f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:16.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.351+0000 7f810659c640 1 --2- 172.21.15.5:0/3689494110 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80f40039b0 0x7f80f4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:16.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.351+0000 7f8105d9b640 1 --2- 172.21.15.5:0/3689494110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80f4004800 0x7f80f4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:16.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.351+0000 7f8106d9d640 1 -- 172.21.15.5:0/3689494110 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80f40039b0 msgr2=0x7f80f4003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:16.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.351+0000 7f8106d9d640 1 --2- 172.21.15.5:0/3689494110 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80f40039b0 0x7f80f4003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.351+0000 7f8106d9d640 1 -- 172.21.15.5:0/3689494110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80f4004800 msgr2=0x7f80f4004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:16.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.351+0000 7f8106d9d640 1 --2- 172.21.15.5:0/3689494110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80f4004800 0x7f80f4004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.351+0000 7f8106d9d640 1 -- 172.21.15.5:0/3689494110 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f4003520 con 0x7f80f4005270 2024-05-20T18:35:16.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.352+0000 7f8106d9d640 1 -- 172.21.15.5:0/3689494110 reap_dead start 2024-05-20T18:35:16.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.352+0000 7f8106d9d640 1 --2- 172.21.15.5:0/3689494110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80f4005270 0x7f80f400b9c0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f80f800c4c0 tx=0x7f80f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22592f13911d8d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:16.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.352+0000 7f810559a640 1 -- 172.21.15.5:0/3689494110 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f800c0e0 con 0x7f80f4005270 2024-05-20T18:35:16.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.352+0000 7f810559a640 1 -- 172.21.15.5:0/3689494110 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80f80058d0 con 0x7f80f4005270 2024-05-20T18:35:16.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.352+0000 7f810559a640 1 -- 172.21.15.5:0/3689494110 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f8005ce0 con 0x7f80f4005270 2024-05-20T18:35:16.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.353+0000 7f810c81a640 1 -- 172.21.15.5:0/3689494110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80f4005270 msgr2=0x7f80f400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:16.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.353+0000 7f810c81a640 1 --2- 172.21.15.5:0/3689494110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80f4005270 0x7f80f400b9c0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f80f800c4c0 tx=0x7f80f4005830 comp rx=0 tx=0).stop 2024-05-20T18:35:16.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.353+0000 7f810c81a640 1 -- 172.21.15.5:0/3689494110 shutdown_connections 2024-05-20T18:35:16.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.353+0000 7f810c81a640 1 --2- 172.21.15.5:0/3689494110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80f4005270 0x7f80f400b9c0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.353+0000 7f810c81a640 1 -- 172.21.15.5:0/3689494110 >> 172.21.15.5:0/3689494110 conn(0x7f81080d4bb0 msgr2=0x7f81080d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:16.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.353+0000 7f810c81a640 1 -- 172.21.15.5:0/3689494110 shutdown_connections 2024-05-20T18:35:16.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.353+0000 7f810c81a640 1 -- 172.21.15.5:0/3689494110 wait complete. 2024-05-20T18:35:16.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.355+0000 7f810c81a640 1 Processor -- start 2024-05-20T18:35:16.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.355+0000 7f810c81a640 1 -- start start 2024-05-20T18:35:16.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.356+0000 7f810c81a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80f4005270 0x7f810815d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:16.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.356+0000 7f810c81a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f810815b360 0x7f8108150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:16.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.356+0000 7f810c81a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81080cb020 0x7f81080cb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:16.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.356+0000 7f810c81a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81080d3c00 con 0x7f810815b360 2024-05-20T18:35:16.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.356+0000 7f810c81a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f81080d3920 con 0x7f81080cb020 2024-05-20T18:35:16.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.356+0000 7f810c81a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f81080d2b90 con 0x7f80f4005270 2024-05-20T18:35:16.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.356+0000 7f8105d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f810815b360 0x7f8108150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:16.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.356+0000 7f8105d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f810815b360 0x7f8108150ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55568/0 (socket says 172.21.15.5:55568) 2024-05-20T18:35:16.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.356+0000 7f8105d9b640 1 -- 172.21.15.5:0/85820189 learned_addr learned my addr 172.21.15.5:0/85820189 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:16.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.356+0000 7f810659c640 1 --2- 172.21.15.5:0/85820189 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80f4005270 0x7f810815d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:16.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.356+0000 7f8106d9d640 1 --2- 172.21.15.5:0/85820189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81080cb020 0x7f81080cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:16.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.357+0000 7f8105d9b640 1 -- 172.21.15.5:0/85820189 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80f4005270 msgr2=0x7f810815d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:16.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.357+0000 7f8105d9b640 1 --2- 172.21.15.5:0/85820189 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80f4005270 0x7f810815d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.357+0000 7f8105d9b640 1 -- 172.21.15.5:0/85820189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81080cb020 msgr2=0x7f81080cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:16.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.357+0000 7f8105d9b640 1 --2- 172.21.15.5:0/85820189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81080cb020 0x7f81080cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.357+0000 7f8105d9b640 1 -- 172.21.15.5:0/85820189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81080cb970 con 0x7f810815b360 2024-05-20T18:35:16.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.357+0000 7f8105d9b640 1 --2- 172.21.15.5:0/85820189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f810815b360 0x7f8108150ff0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f80f000f620 tx=0x7f80f000fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:16.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.357+0000 7f80feffd640 1 -- 172.21.15.5:0/85820189 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f00052e0 con 0x7f810815b360 2024-05-20T18:35:16.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.357+0000 7f80feffd640 1 -- 172.21.15.5:0/85820189 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80f0018070 con 0x7f810815b360 2024-05-20T18:35:16.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.357+0000 7f810c81a640 1 -- 172.21.15.5:0/85820189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f810815bb40 con 0x7f810815b360 2024-05-20T18:35:16.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.357+0000 7f80feffd640 1 -- 172.21.15.5:0/85820189 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f0003810 con 0x7f810815b360 2024-05-20T18:35:16.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.357+0000 7f810c81a640 1 -- 172.21.15.5:0/85820189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f810815c100 con 0x7f810815b360 2024-05-20T18:35:16.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.358+0000 7f810c81a640 1 -- 172.21.15.5:0/85820189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81080735a0 con 0x7f810815b360 2024-05-20T18:35:16.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.359+0000 7f80feffd640 1 -- 172.21.15.5:0/85820189 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f80f00039b0 con 0x7f810815b360 2024-05-20T18:35:16.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.359+0000 7f80feffd640 1 --2- 172.21.15.5:0/85820189 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f80e806be30 0x7f80e806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:16.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.359+0000 7f80feffd640 1 -- 172.21.15.5:0/85820189 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7f80f008adb0 con 0x7f810815b360 2024-05-20T18:35:16.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.359+0000 7f810659c640 1 --2- 172.21.15.5:0/85820189 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f80e806be30 0x7f80e806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:16.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.360+0000 7f810659c640 1 --2- 172.21.15.5:0/85820189 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f80e806be30 0x7f80e806e340 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8100014450 tx=0x7f810000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:16.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.364+0000 7f80feffd640 1 -- 172.21.15.5:0/85820189 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f81080735a0 con 0x7f810815b360 2024-05-20T18:35:16.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.545+0000 7f810c81a640 1 -- 172.21.15.5:0/85820189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f810815b8d0 con 0x7f810815b360 2024-05-20T18:35:16.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.545+0000 7f80feffd640 1 -- 172.21.15.5:0/85820189 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f81080cb970 con 0x7f810815b360 2024-05-20T18:35:16.546 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:35:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.547+0000 7f810c81a640 1 -- 172.21.15.5:0/85820189 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f80e806be30 msgr2=0x7f80e806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.547+0000 7f810c81a640 1 --2- 172.21.15.5:0/85820189 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f80e806be30 0x7f80e806e340 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8100014450 tx=0x7f810000b040 comp rx=0 tx=0).stop 2024-05-20T18:35:16.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.547+0000 7f810c81a640 1 -- 172.21.15.5:0/85820189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f810815b360 msgr2=0x7f8108150ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:16.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.547+0000 7f810c81a640 1 --2- 172.21.15.5:0/85820189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f810815b360 0x7f8108150ff0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f80f000f620 tx=0x7f80f000fae0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.547+0000 7f810c81a640 1 -- 172.21.15.5:0/85820189 shutdown_connections 2024-05-20T18:35:16.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.547+0000 7f810c81a640 1 --2- 172.21.15.5:0/85820189 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f80e806be30 0x7f80e806e340 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.547+0000 7f810c81a640 1 --2- 172.21.15.5:0/85820189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81080cb020 0x7f81080cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.547+0000 7f810c81a640 1 --2- 172.21.15.5:0/85820189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f810815b360 0x7f8108150ff0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.547+0000 7f810c81a640 1 --2- 172.21.15.5:0/85820189 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80f4005270 0x7f810815d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.547+0000 7f810c81a640 1 -- 172.21.15.5:0/85820189 >> 172.21.15.5:0/85820189 conn(0x7f81080d4bb0 msgr2=0x7f810815e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:16.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.547+0000 7f810c81a640 1 -- 172.21.15.5:0/85820189 shutdown_connections 2024-05-20T18:35:16.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.547+0000 7f810c81a640 1 -- 172.21.15.5:0/85820189 wait complete. 2024-05-20T18:35:16.560 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:35:16.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.678+0000 7feeb592f640 1 Processor -- start 2024-05-20T18:35:16.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.678+0000 7feeb592f640 1 -- start start 2024-05-20T18:35:16.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.679+0000 7feeb592f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feeb015d530 0x7feeb015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:16.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.679+0000 7feeb592f640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7feeb00b1700 0x7feeb00b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.679+0000 7feeb592f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7feeb0070850 0x7feeb0070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.679+0000 7feeb592f640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7feeb00ce190 con 0x7feeb0070850 2024-05-20T18:35:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.679+0000 7feeb592f640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7feeb00cd580 con 0x7feeb00b1700 2024-05-20T18:35:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.679+0000 7feeb592f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7feeb00cd860 con 0x7feeb015d530 2024-05-20T18:35:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.679+0000 7feeaf7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7feeb0070850 0x7feeb0070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.679+0000 7feeaf7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7feeb0070850 0x7feeb0070c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55582/0 (socket says 172.21.15.5:55582) 2024-05-20T18:35:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.679+0000 7feeaf7fe640 1 -- 172.21.15.5:0/3833496958 learned_addr learned my addr 172.21.15.5:0/3833496958 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.679+0000 7feeaeffd640 1 --2- 172.21.15.5:0/3833496958 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feeb015d530 0x7feeb015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.679+0000 7feeae7fc640 1 --2- 172.21.15.5:0/3833496958 >> v2:172.21.15.63:3300/0 conn(0x7feeb00b1700 0x7feeb00b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.680+0000 7feeaf7fe640 1 -- 172.21.15.5:0/3833496958 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feeb015d530 msgr2=0x7feeb015aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:16.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.680+0000 7feeaf7fe640 1 --2- 172.21.15.5:0/3833496958 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feeb015d530 0x7feeb015aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.680+0000 7feeaf7fe640 1 -- 172.21.15.5:0/3833496958 >> v2:172.21.15.63:3300/0 conn(0x7feeb00b1700 msgr2=0x7feeb00b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:16.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.680+0000 7feeaf7fe640 1 --2- 172.21.15.5:0/3833496958 >> v2:172.21.15.63:3300/0 conn(0x7feeb00b1700 0x7feeb00b1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.680+0000 7feeaf7fe640 1 -- 172.21.15.5:0/3833496958 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb00cdb40 con 0x7feeb0070850 2024-05-20T18:35:16.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.680+0000 7feeae7fc640 1 --2- 172.21.15.5:0/3833496958 >> v2:172.21.15.63:3300/0 conn(0x7feeb00b1700 0x7feeb00b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:16.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.680+0000 7feeaf7fe640 1 --2- 172.21.15.5:0/3833496958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feeb0070850 0x7feeb0070c60 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7feeb006c2f0 tx=0x7feea002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4f6b1b68c662e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:16.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.680+0000 7feeadffb640 1 -- 172.21.15.5:0/3833496958 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea0036070 con 0x7feeb0070850 2024-05-20T18:35:16.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.680+0000 7feeadffb640 1 -- 172.21.15.5:0/3833496958 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feea0031600 con 0x7feeb0070850 2024-05-20T18:35:16.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.680+0000 7feeadffb640 1 -- 172.21.15.5:0/3833496958 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea00319d0 con 0x7feeb0070850 2024-05-20T18:35:16.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.681+0000 7feeb592f640 1 -- 172.21.15.5:0/3833496958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feeb0070850 msgr2=0x7feeb0070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.681+0000 7feeb592f640 1 --2- 172.21.15.5:0/3833496958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feeb0070850 0x7feeb0070c60 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7feeb006c2f0 tx=0x7feea002a6a0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.681+0000 7feeb592f640 1 -- 172.21.15.5:0/3833496958 shutdown_connections 2024-05-20T18:35:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.681+0000 7feeb592f640 1 --2- 172.21.15.5:0/3833496958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feeb0070850 0x7feeb0070c60 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.681+0000 7feeb592f640 1 --2- 172.21.15.5:0/3833496958 >> v2:172.21.15.63:3300/0 conn(0x7feeb00b1700 0x7feeb00b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.681+0000 7feeb592f640 1 --2- 172.21.15.5:0/3833496958 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feeb015d530 0x7feeb015aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.681+0000 7feeb592f640 1 -- 172.21.15.5:0/3833496958 >> 172.21.15.5:0/3833496958 conn(0x7feeb00d29d0 msgr2=0x7feeb00d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.681+0000 7feeb592f640 1 -- 172.21.15.5:0/3833496958 shutdown_connections 2024-05-20T18:35:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.681+0000 7feeb592f640 1 -- 172.21.15.5:0/3833496958 wait complete. 2024-05-20T18:35:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.683+0000 7feeb592f640 1 Processor -- start 2024-05-20T18:35:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.683+0000 7feeb592f640 1 -- start start 2024-05-20T18:35:16.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.684+0000 7feeb592f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feeb0070850 0x7feeb014bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:16.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.684+0000 7feeb592f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feeb00b1700 0x7feeb00cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:16.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.685+0000 7feeb592f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feeb015d530 0x7feeb00b7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:16.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.685+0000 7feeb592f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feeb00b2050 con 0x7feeb015d530 2024-05-20T18:35:16.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.685+0000 7feeb592f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feeb00cd230 con 0x7feeb0070850 2024-05-20T18:35:16.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.685+0000 7feeb592f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7feeb00711a0 con 0x7feeb00b1700 2024-05-20T18:35:16.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.685+0000 7feeaf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feeb015d530 0x7feeb00b7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:16.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.685+0000 7feeaf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feeb015d530 0x7feeb00b7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55598/0 (socket says 172.21.15.5:55598) 2024-05-20T18:35:16.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.685+0000 7feeaf7fe640 1 -- 172.21.15.5:0/1740633316 learned_addr learned my addr 172.21.15.5:0/1740633316 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:16.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.685+0000 7feeaeffd640 1 --2- 172.21.15.5:0/1740633316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feeb0070850 0x7feeb014bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:16.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.685+0000 7feeae7fc640 1 --2- 172.21.15.5:0/1740633316 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feeb00b1700 0x7feeb00cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:16.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.685+0000 7feeaf7fe640 1 -- 172.21.15.5:0/1740633316 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feeb00b1700 msgr2=0x7feeb00cbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:16.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.685+0000 7feeaf7fe640 1 --2- 172.21.15.5:0/1740633316 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feeb00b1700 0x7feeb00cbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.685+0000 7feeaf7fe640 1 -- 172.21.15.5:0/1740633316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feeb0070850 msgr2=0x7feeb014bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:16.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.685+0000 7feeaf7fe640 1 --2- 172.21.15.5:0/1740633316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feeb0070850 0x7feeb014bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.685+0000 7feeaf7fe640 1 -- 172.21.15.5:0/1740633316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb01f3430 con 0x7feeb015d530 2024-05-20T18:35:16.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.685+0000 7feeae7fc640 1 --2- 172.21.15.5:0/1740633316 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feeb00b1700 0x7feeb00cbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:16.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.685+0000 7feeaeffd640 1 --2- 172.21.15.5:0/1740633316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feeb0070850 0x7feeb014bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:16.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.685+0000 7feeaf7fe640 1 --2- 172.21.15.5:0/1740633316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feeb015d530 0x7feeb00b7c60 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7feea0009f40 tx=0x7feea0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:16.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.685+0000 7feea7fff640 1 -- 172.21.15.5:0/1740633316 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea0036050 con 0x7feeb015d530 2024-05-20T18:35:16.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.685+0000 7feea7fff640 1 -- 172.21.15.5:0/1740633316 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feea0042070 con 0x7feeb015d530 2024-05-20T18:35:16.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.685+0000 7feeb592f640 1 -- 172.21.15.5:0/1740633316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeb01f3710 con 0x7feeb015d530 2024-05-20T18:35:16.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.685+0000 7feea7fff640 1 -- 172.21.15.5:0/1740633316 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea0009510 con 0x7feeb015d530 2024-05-20T18:35:16.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.685+0000 7feeb592f640 1 -- 172.21.15.5:0/1740633316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeb01f3cd0 con 0x7feeb015d530 2024-05-20T18:35:16.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.687+0000 7feeb592f640 1 -- 172.21.15.5:0/1740633316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feeb0075b00 con 0x7feeb015d530 2024-05-20T18:35:16.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.687+0000 7feea7fff640 1 -- 172.21.15.5:0/1740633316 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feea0007bf0 con 0x7feeb015d530 2024-05-20T18:35:16.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.688+0000 7feea7fff640 1 --2- 172.21.15.5:0/1740633316 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fee9006be30 0x7fee9006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:16.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.688+0000 7feea7fff640 1 -- 172.21.15.5:0/1740633316 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7feea00abf60 con 0x7feeb015d530 2024-05-20T18:35:16.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.688+0000 7feeaeffd640 1 --2- 172.21.15.5:0/1740633316 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fee9006be30 0x7fee9006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:16.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.688+0000 7feeaeffd640 1 --2- 172.21.15.5:0/1740633316 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fee9006be30 0x7fee9006e340 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7feeb00d29a0 tx=0x7feea8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:16.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.692+0000 7feea7fff640 1 -- 172.21.15.5:0/1740633316 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7feea007a660 con 0x7feeb015d530 2024-05-20T18:35:16.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.881+0000 7feeb592f640 1 -- 172.21.15.5:0/1740633316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7feeb015ea10 con 0x7feeb015d530 2024-05-20T18:35:16.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.881+0000 7feea7fff640 1 -- 172.21.15.5:0/1740633316 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7feea0079f60 con 0x7feeb015d530 2024-05-20T18:35:16.882 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:35:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.883+0000 7feeb592f640 1 -- 172.21.15.5:0/1740633316 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fee9006be30 msgr2=0x7fee9006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.883+0000 7feeb592f640 1 --2- 172.21.15.5:0/1740633316 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fee9006be30 0x7fee9006e340 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7feeb00d29a0 tx=0x7feea8008040 comp rx=0 tx=0).stop 2024-05-20T18:35:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.883+0000 7feeb592f640 1 -- 172.21.15.5:0/1740633316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feeb015d530 msgr2=0x7feeb00b7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.883+0000 7feeb592f640 1 --2- 172.21.15.5:0/1740633316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feeb015d530 0x7feeb00b7c60 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7feea0009f40 tx=0x7feea0004a40 comp rx=0 tx=0).stop 2024-05-20T18:35:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.883+0000 7feeb592f640 1 -- 172.21.15.5:0/1740633316 shutdown_connections 2024-05-20T18:35:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.883+0000 7feeb592f640 1 --2- 172.21.15.5:0/1740633316 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fee9006be30 0x7fee9006e340 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.883+0000 7feeb592f640 1 --2- 172.21.15.5:0/1740633316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feeb015d530 0x7feeb00b7c60 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.883+0000 7feeb592f640 1 --2- 172.21.15.5:0/1740633316 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feeb00b1700 0x7feeb00cbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.883+0000 7feeb592f640 1 --2- 172.21.15.5:0/1740633316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feeb0070850 0x7feeb014bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.883+0000 7feeb592f640 1 -- 172.21.15.5:0/1740633316 >> 172.21.15.5:0/1740633316 conn(0x7feeb00d29d0 msgr2=0x7feeb015fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:16.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.883+0000 7feeb592f640 1 -- 172.21.15.5:0/1740633316 shutdown_connections 2024-05-20T18:35:16.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:16.883+0000 7feeb592f640 1 -- 172.21.15.5:0/1740633316 wait complete. 2024-05-20T18:35:16.896 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:35:17.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.013+0000 7f46eef79640 1 Processor -- start 2024-05-20T18:35:17.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.013+0000 7f46eef79640 1 -- start start 2024-05-20T18:35:17.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.014+0000 7f46eef79640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46e8162820 0x7f46e80b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:17.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.014+0000 7f46eef79640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f46e80b3e80 0x7f46e806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:17.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.014+0000 7f46eef79640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46e806dd20 con 0x7f46e806d910 2024-05-20T18:35:17.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.014+0000 7f46eef79640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f46e80d0630 con 0x7f46e80b3e80 2024-05-20T18:35:17.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.014+0000 7f46eef79640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f46e80d39c0 con 0x7f46e8162820 2024-05-20T18:35:17.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.014+0000 7f46e7fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f46e80b3e80 0x7f46e806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:17.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.014+0000 7f46e7fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f46e80b3e80 0x7f46e806d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55608/0 (socket says 172.21.15.5:55608) 2024-05-20T18:35:17.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.014+0000 7f46e7fff640 1 -- 172.21.15.5:0/1756842050 learned_addr learned my addr 172.21.15.5:0/1756842050 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:17.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.015+0000 7f46ecd15640 1 --2- 172.21.15.5:0/1756842050 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46e8162820 0x7f46e80b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:17.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.015+0000 7f46e7fff640 1 -- 172.21.15.5:0/1756842050 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46e8162820 msgr2=0x7f46e80b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:17.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.015+0000 7f46e7fff640 1 --2- 172.21.15.5:0/1756842050 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46e8162820 0x7f46e80b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:17.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.015+0000 7f46e7fff640 1 -- 172.21.15.5:0/1756842050 >> v1:172.21.15.63:6789/0 conn(0x7f46e806d910 legacy=0x7f46e80d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:17.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.015+0000 7f46e7fff640 1 -- 172.21.15.5:0/1756842050 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e80d3ca0 con 0x7f46e80b3e80 2024-05-20T18:35:17.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.016+0000 7f46e7fff640 1 --2- 172.21.15.5:0/1756842050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46e80b3e80 0x7f46e806d3d0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f46d4009870 tx=0x7f46d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc45cf3a653c48d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:17.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.016+0000 7f46e77fe640 1 -- 172.21.15.5:0/1756842050 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d4037070 con 0x7f46e80b3e80 2024-05-20T18:35:17.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.016+0000 7f46e77fe640 1 -- 172.21.15.5:0/1756842050 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46d4032440 con 0x7f46e80b3e80 2024-05-20T18:35:17.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.016+0000 7f46e77fe640 1 -- 172.21.15.5:0/1756842050 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d4032810 con 0x7f46e80b3e80 2024-05-20T18:35:17.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.016+0000 7f46eef79640 1 -- 172.21.15.5:0/1756842050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46e80b3e80 msgr2=0x7f46e806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:17.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.016+0000 7f46eef79640 1 --2- 172.21.15.5:0/1756842050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46e80b3e80 0x7f46e806d3d0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f46d4009870 tx=0x7f46d402a270 comp rx=0 tx=0).stop 2024-05-20T18:35:17.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.016+0000 7f46eef79640 1 -- 172.21.15.5:0/1756842050 shutdown_connections 2024-05-20T18:35:17.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.016+0000 7f46eef79640 1 --2- 172.21.15.5:0/1756842050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46e80b3e80 0x7f46e806d3d0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:17.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.016+0000 7f46eef79640 1 --2- 172.21.15.5:0/1756842050 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46e8162820 0x7f46e80b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:17.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.016+0000 7f46eef79640 1 -- 172.21.15.5:0/1756842050 >> 172.21.15.5:0/1756842050 conn(0x7f46e80d4bd0 msgr2=0x7f46e80d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:17.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.016+0000 7f46eef79640 1 -- 172.21.15.5:0/1756842050 shutdown_connections 2024-05-20T18:35:17.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.016+0000 7f46eef79640 1 -- 172.21.15.5:0/1756842050 wait complete. 2024-05-20T18:35:17.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.018+0000 7f46eef79640 1 Processor -- start 2024-05-20T18:35:17.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.018+0000 7f46eef79640 1 -- start start 2024-05-20T18:35:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.020+0000 7f46eef79640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46e806d910 0x7f46e80ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.020+0000 7f46eef79640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e80b3e80 0x7f46e815bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.020+0000 7f46eef79640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46e8162820 0x7f46e81f5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.020+0000 7f46eef79640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f46e80d0300 con 0x7f46e806d910 2024-05-20T18:35:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.020+0000 7f46eef79640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f46e80d4260 con 0x7f46e80b3e80 2024-05-20T18:35:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.020+0000 7f46eef79640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f46e80b91a0 con 0x7f46e8162820 2024-05-20T18:35:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.020+0000 7f46ecd15640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46e806d910 0x7f46e80ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.020+0000 7f46ecd15640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46e806d910 0x7f46e80ce7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55616/0 (socket says 172.21.15.5:55616) 2024-05-20T18:35:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.020+0000 7f46ecd15640 1 -- 172.21.15.5:0/2033227687 learned_addr learned my addr 172.21.15.5:0/2033227687 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.020+0000 7f46e7fff640 1 --2- 172.21.15.5:0/2033227687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e80b3e80 0x7f46e815bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.020+0000 7f46ed516640 1 --2- 172.21.15.5:0/2033227687 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46e8162820 0x7f46e81f5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.020+0000 7f46ecd15640 1 -- 172.21.15.5:0/2033227687 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46e8162820 msgr2=0x7f46e81f5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.020+0000 7f46ecd15640 1 --2- 172.21.15.5:0/2033227687 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46e8162820 0x7f46e81f5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.020+0000 7f46ecd15640 1 -- 172.21.15.5:0/2033227687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e80b3e80 msgr2=0x7f46e815bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.020+0000 7f46ecd15640 1 --2- 172.21.15.5:0/2033227687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e80b3e80 0x7f46e815bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.020+0000 7f46ecd15640 1 -- 172.21.15.5:0/2033227687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e81f6280 con 0x7f46e806d910 2024-05-20T18:35:17.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.020+0000 7f46ecd15640 1 --2- 172.21.15.5:0/2033227687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46e806d910 0x7f46e80ce7c0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f46dc00e870 tx=0x7f46dc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:17.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.020+0000 7f46e57fa640 1 -- 172.21.15.5:0/2033227687 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc017070 con 0x7f46e806d910 2024-05-20T18:35:17.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.021+0000 7f46e57fa640 1 -- 172.21.15.5:0/2033227687 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46dc00c940 con 0x7f46e806d910 2024-05-20T18:35:17.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.021+0000 7f46eef79640 1 -- 172.21.15.5:0/2033227687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46e80ced00 con 0x7f46e806d910 2024-05-20T18:35:17.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.021+0000 7f46e57fa640 1 -- 172.21.15.5:0/2033227687 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc00cd30 con 0x7f46e806d910 2024-05-20T18:35:17.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.021+0000 7f46eef79640 1 -- 172.21.15.5:0/2033227687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46e80cc270 con 0x7f46e806d910 2024-05-20T18:35:17.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.022+0000 7f46eef79640 1 -- 172.21.15.5:0/2033227687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46e8075b00 con 0x7f46e806d910 2024-05-20T18:35:17.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.022+0000 7f46e57fa640 1 -- 172.21.15.5:0/2033227687 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f46dc01a070 con 0x7f46e806d910 2024-05-20T18:35:17.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.023+0000 7f46e57fa640 1 --2- 172.21.15.5:0/2033227687 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f46cc06be60 0x7f46cc06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:17.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.023+0000 7f46e7fff640 1 --2- 172.21.15.5:0/2033227687 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f46cc06be60 0x7f46cc06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:17.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.023+0000 7f46e57fa640 1 -- 172.21.15.5:0/2033227687 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7f46dc08afd0 con 0x7f46e806d910 2024-05-20T18:35:17.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.023+0000 7f46e7fff640 1 --2- 172.21.15.5:0/2033227687 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f46cc06be60 0x7f46cc06e370 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f46d4034000 tx=0x7f46d402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:17.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.028+0000 7f46e57fa640 1 -- 172.21.15.5:0/2033227687 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f46e81f6280 con 0x7f46e806d910 2024-05-20T18:35:17.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.213+0000 7f46eef79640 1 -- 172.21.15.5:0/2033227687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f46e8068a90 con 0x7f46e806d910 2024-05-20T18:35:17.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.213+0000 7f46e57fa640 1 -- 172.21.15.5:0/2033227687 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f46dc058fe0 con 0x7f46e806d910 2024-05-20T18:35:17.215 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:35:17.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.215+0000 7f46eef79640 1 -- 172.21.15.5:0/2033227687 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f46cc06be60 msgr2=0x7f46cc06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:17.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.215+0000 7f46eef79640 1 --2- 172.21.15.5:0/2033227687 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f46cc06be60 0x7f46cc06e370 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f46d4034000 tx=0x7f46d402e000 comp rx=0 tx=0).stop 2024-05-20T18:35:17.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.215+0000 7f46eef79640 1 -- 172.21.15.5:0/2033227687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46e806d910 msgr2=0x7f46e80ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:17.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.215+0000 7f46eef79640 1 --2- 172.21.15.5:0/2033227687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46e806d910 0x7f46e80ce7c0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f46dc00e870 tx=0x7f46dc00ed30 comp rx=0 tx=0).stop 2024-05-20T18:35:17.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.216+0000 7f46eef79640 1 -- 172.21.15.5:0/2033227687 shutdown_connections 2024-05-20T18:35:17.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.216+0000 7f46eef79640 1 --2- 172.21.15.5:0/2033227687 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f46cc06be60 0x7f46cc06e370 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:17.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.216+0000 7f46eef79640 1 --2- 172.21.15.5:0/2033227687 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46e8162820 0x7f46e81f5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:17.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.216+0000 7f46eef79640 1 --2- 172.21.15.5:0/2033227687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46e80b3e80 0x7f46e815bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:17.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.216+0000 7f46eef79640 1 --2- 172.21.15.5:0/2033227687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46e806d910 0x7f46e80ce7c0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:17.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.216+0000 7f46eef79640 1 -- 172.21.15.5:0/2033227687 >> 172.21.15.5:0/2033227687 conn(0x7f46e80d4bd0 msgr2=0x7f46e81608f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:17.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.216+0000 7f46eef79640 1 -- 172.21.15.5:0/2033227687 shutdown_connections 2024-05-20T18:35:17.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.216+0000 7f46eef79640 1 -- 172.21.15.5:0/2033227687 wait complete. 2024-05-20T18:35:17.228 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:35:17.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.342+0000 7f6c06261640 1 Processor -- start 2024-05-20T18:35:17.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.342+0000 7f6c06261640 1 -- start start 2024-05-20T18:35:17.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.344+0000 7f6c06261640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c000d10e0 0x7f6c000cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:17.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.344+0000 7f6c06261640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6c000d14f0 0x7f6c000cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:17.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.344+0000 7f6c06261640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c000d1ac0 con 0x7f6c000cd4d0 2024-05-20T18:35:17.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.344+0000 7f6c06261640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f6c000d1da0 con 0x7f6c000d14f0 2024-05-20T18:35:17.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.344+0000 7f6c06261640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6c000d2080 con 0x7f6c000d10e0 2024-05-20T18:35:17.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.344+0000 7f6bfeffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6c000d14f0 0x7f6c000cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:17.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.344+0000 7f6bfeffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6c000d14f0 0x7f6c000cf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55626/0 (socket says 172.21.15.5:55626) 2024-05-20T18:35:17.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.344+0000 7f6bfeffd640 1 -- 172.21.15.5:0/3717629705 learned_addr learned my addr 172.21.15.5:0/3717629705 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:17.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.344+0000 7f6bff7fe640 1 --2- 172.21.15.5:0/3717629705 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c000d10e0 0x7f6c000cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:17.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.344+0000 7f6bfeffd640 1 -- 172.21.15.5:0/3717629705 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c000d10e0 msgr2=0x7f6c000cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:17.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.344+0000 7f6bfeffd640 1 --2- 172.21.15.5:0/3717629705 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c000d10e0 0x7f6c000cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:17.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.344+0000 7f6bfeffd640 1 -- 172.21.15.5:0/3717629705 >> v1:172.21.15.63:6789/0 conn(0x7f6c000cd4d0 legacy=0x7f6c000cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:17.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.344+0000 7f6bfeffd640 1 -- 172.21.15.5:0/3717629705 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c000d2360 con 0x7f6c000d14f0 2024-05-20T18:35:17.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.345+0000 7f6bfeffd640 1 --2- 172.21.15.5:0/3717629705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c000d14f0 0x7f6c000cf680 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4009870 tx=0x7f6bf402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed5d11513cd3478 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:17.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.345+0000 7f6bfe7fc640 1 -- 172.21.15.5:0/3717629705 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4037070 con 0x7f6c000d14f0 2024-05-20T18:35:17.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.345+0000 7f6bfe7fc640 1 -- 172.21.15.5:0/3717629705 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bf4032440 con 0x7f6c000d14f0 2024-05-20T18:35:17.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.345+0000 7f6bfe7fc640 1 -- 172.21.15.5:0/3717629705 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4032810 con 0x7f6c000d14f0 2024-05-20T18:35:17.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.346+0000 7f6c06261640 1 -- 172.21.15.5:0/3717629705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c000d14f0 msgr2=0x7f6c000cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:17.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.346+0000 7f6c06261640 1 --2- 172.21.15.5:0/3717629705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c000d14f0 0x7f6c000cf680 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4009870 tx=0x7f6bf402a270 comp rx=0 tx=0).stop 2024-05-20T18:35:17.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.346+0000 7f6c06261640 1 -- 172.21.15.5:0/3717629705 shutdown_connections 2024-05-20T18:35:17.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.346+0000 7f6c06261640 1 --2- 172.21.15.5:0/3717629705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c000d14f0 0x7f6c000cf680 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:17.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.346+0000 7f6c06261640 1 --2- 172.21.15.5:0/3717629705 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c000d10e0 0x7f6c000cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:17.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.346+0000 7f6c06261640 1 -- 172.21.15.5:0/3717629705 >> 172.21.15.5:0/3717629705 conn(0x7f6c000b44d0 msgr2=0x7f6c000b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:17.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.346+0000 7f6c06261640 1 -- 172.21.15.5:0/3717629705 shutdown_connections 2024-05-20T18:35:17.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.346+0000 7f6c06261640 1 -- 172.21.15.5:0/3717629705 wait complete. 2024-05-20T18:35:17.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.348+0000 7f6c06261640 1 Processor -- start 2024-05-20T18:35:17.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.348+0000 7f6c06261640 1 -- start start 2024-05-20T18:35:17.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.349+0000 7f6c06261640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c000cd4d0 0x7f6c000b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:17.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.349+0000 7f6c06261640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c000d10e0 0x7f6c000aa600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:17.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.349+0000 7f6c06261640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c000d14f0 0x7f6c000aab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:17.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.349+0000 7f6c06261640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6c000b8440 con 0x7f6c000d14f0 2024-05-20T18:35:17.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.349+0000 7f6c06261640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6c000b82c0 con 0x7f6c000cd4d0 2024-05-20T18:35:17.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.349+0000 7f6c06261640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6c000709f0 con 0x7f6c000d10e0 2024-05-20T18:35:17.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.350+0000 7f6bfffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c000d14f0 0x7f6c000aab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:17.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.350+0000 7f6bfffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c000d14f0 0x7f6c000aab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55628/0 (socket says 172.21.15.5:55628) 2024-05-20T18:35:17.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.350+0000 7f6bfffff640 1 -- 172.21.15.5:0/2155018648 learned_addr learned my addr 172.21.15.5:0/2155018648 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:17.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.350+0000 7f6bff7fe640 1 --2- 172.21.15.5:0/2155018648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c000cd4d0 0x7f6c000b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:17.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.350+0000 7f6bfeffd640 1 --2- 172.21.15.5:0/2155018648 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c000d10e0 0x7f6c000aa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:17.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.350+0000 7f6bfffff640 1 -- 172.21.15.5:0/2155018648 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c000d10e0 msgr2=0x7f6c000aa600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:17.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.350+0000 7f6bfffff640 1 --2- 172.21.15.5:0/2155018648 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c000d10e0 0x7f6c000aa600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:17.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.350+0000 7f6bfffff640 1 -- 172.21.15.5:0/2155018648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c000cd4d0 msgr2=0x7f6c000b0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:17.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.350+0000 7f6bfffff640 1 --2- 172.21.15.5:0/2155018648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c000cd4d0 0x7f6c000b0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:17.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.350+0000 7f6bfffff640 1 -- 172.21.15.5:0/2155018648 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c001e65d0 con 0x7f6c000d14f0 2024-05-20T18:35:17.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.350+0000 7f6bfeffd640 1 --2- 172.21.15.5:0/2155018648 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c000d10e0 0x7f6c000aa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:17.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.350+0000 7f6bff7fe640 1 --2- 172.21.15.5:0/2155018648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c000cd4d0 0x7f6c000b0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:17.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.350+0000 7f6bfffff640 1 --2- 172.21.15.5:0/2155018648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c000d14f0 0x7f6c000aab40 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f6bf800c760 tx=0x7f6bf800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:17.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.350+0000 7f6be7fff640 1 -- 172.21.15.5:0/2155018648 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8003b70 con 0x7f6c000d14f0 2024-05-20T18:35:17.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.350+0000 7f6be7fff640 1 -- 172.21.15.5:0/2155018648 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bf8003d10 con 0x7f6c000d14f0 2024-05-20T18:35:17.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.350+0000 7f6c06261640 1 -- 172.21.15.5:0/2155018648 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c001e68b0 con 0x7f6c000d14f0 2024-05-20T18:35:17.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.350+0000 7f6be7fff640 1 -- 172.21.15.5:0/2155018648 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf800aa90 con 0x7f6c000d14f0 2024-05-20T18:35:17.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.350+0000 7f6c06261640 1 -- 172.21.15.5:0/2155018648 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c001e6e70 con 0x7f6c000d14f0 2024-05-20T18:35:17.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.351+0000 7f6c06261640 1 -- 172.21.15.5:0/2155018648 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c00075b00 con 0x7f6c000d14f0 2024-05-20T18:35:17.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.352+0000 7f6be7fff640 1 -- 172.21.15.5:0/2155018648 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6bf800ac30 con 0x7f6c000d14f0 2024-05-20T18:35:17.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.352+0000 7f6be7fff640 1 --2- 172.21.15.5:0/2155018648 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6bec06be30 0x7f6bec06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:17.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.353+0000 7f6be7fff640 1 -- 172.21.15.5:0/2155018648 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7f6bf808afe0 con 0x7f6c000d14f0 2024-05-20T18:35:17.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.353+0000 7f6bff7fe640 1 --2- 172.21.15.5:0/2155018648 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6bec06be30 0x7f6bec06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:17.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.353+0000 7f6bff7fe640 1 --2- 172.21.15.5:0/2155018648 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6bec06be30 0x7f6bec06e340 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0009a20 tx=0x7f6bf0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:17.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.357+0000 7f6be7fff640 1 -- 172.21.15.5:0/2155018648 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6bf8059650 con 0x7f6c000d14f0 2024-05-20T18:35:17.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.580+0000 7f6c06261640 1 -- 172.21.15.5:0/2155018648 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c001e6360 con 0x7f6c000d14f0 2024-05-20T18:35:17.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.581+0000 7f6be7fff640 1 -- 172.21.15.5:0/2155018648 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5172 (secure 0 0 0) 0x7f6bf8058f50 con 0x7f6c000d14f0 2024-05-20T18:35:17.583 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:17.583 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/337248865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":337248865},{"type":"v1","addr":"172.21.15.63:6834","nonce":337248865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3705772553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3705772553},{"type":"v1","addr":"172.21.15.63:6835","nonce":3705772553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":7434,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/776188138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":776188138},{"type":"v1","addr":"172.21.15.5:6835","nonce":776188138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:33:12.457716+0000","modified":"2024-05-20T18:34:39.318498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7953":{"gid":7953,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6837/3880006778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3880006778},{"type":"v1","addr":"172.21.15.5:6837","nonce":3880006778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7953,"qdb_cluster":[7953]},"id":6}]} 2024-05-20T18:35:17.583 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 73 2024-05-20T18:35:17.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.583+0000 7f6c06261640 1 -- 172.21.15.5:0/2155018648 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6bec06be30 msgr2=0x7f6bec06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:17.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.583+0000 7f6c06261640 1 --2- 172.21.15.5:0/2155018648 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6bec06be30 0x7f6bec06e340 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0009a20 tx=0x7f6bf0008040 comp rx=0 tx=0).stop 2024-05-20T18:35:17.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.583+0000 7f6c06261640 1 -- 172.21.15.5:0/2155018648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c000d14f0 msgr2=0x7f6c000aab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:17.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.583+0000 7f6c06261640 1 --2- 172.21.15.5:0/2155018648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c000d14f0 0x7f6c000aab40 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f6bf800c760 tx=0x7f6bf800cc20 comp rx=0 tx=0).stop 2024-05-20T18:35:17.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.583+0000 7f6c06261640 1 -- 172.21.15.5:0/2155018648 shutdown_connections 2024-05-20T18:35:17.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.583+0000 7f6c06261640 1 --2- 172.21.15.5:0/2155018648 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6bec06be30 0x7f6bec06e340 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:17.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.583+0000 7f6c06261640 1 --2- 172.21.15.5:0/2155018648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c000d14f0 0x7f6c000aab40 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:17.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.583+0000 7f6c06261640 1 --2- 172.21.15.5:0/2155018648 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c000d10e0 0x7f6c000aa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:17.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.583+0000 7f6c06261640 1 --2- 172.21.15.5:0/2155018648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c000cd4d0 0x7f6c000b0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:17.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.583+0000 7f6c06261640 1 -- 172.21.15.5:0/2155018648 >> 172.21.15.5:0/2155018648 conn(0x7f6c000b44d0 msgr2=0x7f6c000b1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:17.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.583+0000 7f6c06261640 1 -- 172.21.15.5:0/2155018648 shutdown_connections 2024-05-20T18:35:17.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.583+0000 7f6c06261640 1 -- 172.21.15.5:0/2155018648 wait complete. 2024-05-20T18:35:17.596 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:35:17.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.709+0000 7f4f5bd57640 1 Processor -- start 2024-05-20T18:35:17.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.709+0000 7f4f5bd57640 1 -- start start 2024-05-20T18:35:17.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.711+0000 7f4f5bd57640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4f54162820 0x7f4f540b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:17.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.711+0000 7f4f5bd57640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f4f540b3e80 0x7f4f5406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:17.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.711+0000 7f4f5bd57640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f5406dd20 con 0x7f4f5406d910 2024-05-20T18:35:17.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.711+0000 7f4f5bd57640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f4f540d0630 con 0x7f4f54162820 2024-05-20T18:35:17.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.711+0000 7f4f5bd57640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f4f540d39c0 con 0x7f4f540b3e80 2024-05-20T18:35:17.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.711+0000 7f4f59af3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4f54162820 0x7f4f540b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:17.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.711+0000 7f4f59af3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4f54162820 0x7f4f540b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55642/0 (socket says 172.21.15.5:55642) 2024-05-20T18:35:17.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.711+0000 7f4f59af3640 1 -- 172.21.15.5:0/3647076736 learned_addr learned my addr 172.21.15.5:0/3647076736 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:17.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.711+0000 7f4f592f2640 1 --2- 172.21.15.5:0/3647076736 >> v2:172.21.15.63:3300/0 conn(0x7f4f540b3e80 0x7f4f5406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:17.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.711+0000 7f4f59af3640 1 -- 172.21.15.5:0/3647076736 >> v2:172.21.15.63:3300/0 conn(0x7f4f540b3e80 msgr2=0x7f4f5406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:17.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.711+0000 7f4f59af3640 1 --2- 172.21.15.5:0/3647076736 >> v2:172.21.15.63:3300/0 conn(0x7f4f540b3e80 0x7f4f5406d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:17.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.711+0000 7f4f59af3640 1 -- 172.21.15.5:0/3647076736 >> v1:172.21.15.63:6789/0 conn(0x7f4f5406d910 legacy=0x7f4f540d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:17.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.711+0000 7f4f59af3640 1 -- 172.21.15.5:0/3647076736 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f540d3ca0 con 0x7f4f54162820 2024-05-20T18:35:17.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.712+0000 7f4f59af3640 1 --2- 172.21.15.5:0/3647076736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f54162820 0x7f4f540b3940 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f4f540d1230 tx=0x7f4f5002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3bf951b7eb6965 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:17.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.712+0000 7f4f58af1640 1 -- 172.21.15.5:0/3647076736 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50037070 con 0x7f4f54162820 2024-05-20T18:35:17.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.712+0000 7f4f58af1640 1 -- 172.21.15.5:0/3647076736 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f50032440 con 0x7f4f54162820 2024-05-20T18:35:17.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.712+0000 7f4f58af1640 1 -- 172.21.15.5:0/3647076736 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50032810 con 0x7f4f54162820 2024-05-20T18:35:17.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.713+0000 7f4f5bd57640 1 -- 172.21.15.5:0/3647076736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f54162820 msgr2=0x7f4f540b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:17.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.713+0000 7f4f5bd57640 1 --2- 172.21.15.5:0/3647076736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f54162820 0x7f4f540b3940 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f4f540d1230 tx=0x7f4f5002a270 comp rx=0 tx=0).stop 2024-05-20T18:35:17.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.713+0000 7f4f5bd57640 1 -- 172.21.15.5:0/3647076736 shutdown_connections 2024-05-20T18:35:17.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.713+0000 7f4f5bd57640 1 --2- 172.21.15.5:0/3647076736 >> v2:172.21.15.63:3300/0 conn(0x7f4f540b3e80 0x7f4f5406d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:17.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.713+0000 7f4f5bd57640 1 --2- 172.21.15.5:0/3647076736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f54162820 0x7f4f540b3940 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:17.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.713+0000 7f4f5bd57640 1 -- 172.21.15.5:0/3647076736 >> 172.21.15.5:0/3647076736 conn(0x7f4f540d4bd0 msgr2=0x7f4f540d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:17.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.713+0000 7f4f5bd57640 1 -- 172.21.15.5:0/3647076736 shutdown_connections 2024-05-20T18:35:17.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.713+0000 7f4f5bd57640 1 -- 172.21.15.5:0/3647076736 wait complete. 2024-05-20T18:35:17.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.715+0000 7f4f5bd57640 1 Processor -- start 2024-05-20T18:35:17.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.715+0000 7f4f5bd57640 1 -- start start 2024-05-20T18:35:17.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.716+0000 7f4f5bd57640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f5406d910 0x7f4f5414df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:17.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.716+0000 7f4f5bd57640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f540b3e80 0x7f4f541f8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:17.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.716+0000 7f4f5bd57640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f54162820 0x7f4f541f8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:17.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.716+0000 7f4f5bd57640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f540d0300 con 0x7f4f54162820 2024-05-20T18:35:17.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.716+0000 7f4f5bd57640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4f540d4260 con 0x7f4f540b3e80 2024-05-20T18:35:17.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.716+0000 7f4f5bd57640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4f540b91a0 con 0x7f4f5406d910 2024-05-20T18:35:17.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.716+0000 7f4f5a2f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f54162820 0x7f4f541f8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:17.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.716+0000 7f4f5a2f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f54162820 0x7f4f541f8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55652/0 (socket says 172.21.15.5:55652) 2024-05-20T18:35:17.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.716+0000 7f4f5a2f4640 1 -- 172.21.15.5:0/183835301 learned_addr learned my addr 172.21.15.5:0/183835301 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:17.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.716+0000 7f4f59af3640 1 --2- 172.21.15.5:0/183835301 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f5406d910 0x7f4f5414df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:17.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.717+0000 7f4f592f2640 1 --2- 172.21.15.5:0/183835301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f540b3e80 0x7f4f541f8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:17.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.717+0000 7f4f5a2f4640 1 -- 172.21.15.5:0/183835301 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f5406d910 msgr2=0x7f4f5414df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:17.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.717+0000 7f4f5a2f4640 1 --2- 172.21.15.5:0/183835301 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f5406d910 0x7f4f5414df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:17.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.717+0000 7f4f5a2f4640 1 -- 172.21.15.5:0/183835301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f540b3e80 msgr2=0x7f4f541f8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:17.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.717+0000 7f4f5a2f4640 1 --2- 172.21.15.5:0/183835301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f540b3e80 0x7f4f541f8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:17.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.717+0000 7f4f5a2f4640 1 -- 172.21.15.5:0/183835301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f541f93a0 con 0x7f4f54162820 2024-05-20T18:35:17.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.717+0000 7f4f5a2f4640 1 --2- 172.21.15.5:0/183835301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f54162820 0x7f4f541f8b90 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00c6e0 tx=0x7f4f4c00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:17.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.717+0000 7f4f4a7fc640 1 -- 172.21.15.5:0/183835301 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c003b70 con 0x7f4f54162820 2024-05-20T18:35:17.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.717+0000 7f4f4a7fc640 1 -- 172.21.15.5:0/183835301 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f4c003d30 con 0x7f4f54162820 2024-05-20T18:35:17.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.717+0000 7f4f5bd57640 1 -- 172.21.15.5:0/183835301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f541f52b0 con 0x7f4f54162820 2024-05-20T18:35:17.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.717+0000 7f4f4a7fc640 1 -- 172.21.15.5:0/183835301 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c005310 con 0x7f4f54162820 2024-05-20T18:35:17.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.717+0000 7f4f5bd57640 1 -- 172.21.15.5:0/183835301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f541f4f80 con 0x7f4f54162820 2024-05-20T18:35:17.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.718+0000 7f4f5bd57640 1 -- 172.21.15.5:0/183835301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f54068a10 con 0x7f4f54162820 2024-05-20T18:35:17.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.719+0000 7f4f4a7fc640 1 -- 172.21.15.5:0/183835301 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4f4c0054b0 con 0x7f4f54162820 2024-05-20T18:35:17.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.719+0000 7f4f4a7fc640 1 --2- 172.21.15.5:0/183835301 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4f3806be30 0x7f4f3806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:17.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.719+0000 7f4f4a7fc640 1 -- 172.21.15.5:0/183835301 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7f4f4c08af60 con 0x7f4f54162820 2024-05-20T18:35:17.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.719+0000 7f4f59af3640 1 --2- 172.21.15.5:0/183835301 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4f3806be30 0x7f4f3806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:17.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.720+0000 7f4f59af3640 1 --2- 172.21.15.5:0/183835301 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4f3806be30 0x7f4f3806e340 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4f50034000 tx=0x7f4f5002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:17.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.724+0000 7f4f4a7fc640 1 -- 172.21.15.5:0/183835301 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4f54068a10 con 0x7f4f54162820 2024-05-20T18:35:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.907+0000 7f4f5bd57640 1 -- 172.21.15.5:0/183835301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f54075b00 con 0x7f4f54162820 2024-05-20T18:35:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.909+0000 7f4f4a7fc640 1 -- 172.21.15.5:0/183835301 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+17216 (secure 0 0 0) 0x7f4f5406dd20 con 0x7f4f54162820 2024-05-20T18:35:17.910 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:17.910 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":94,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:34:35.299736+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:33:10.184948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-20T18:33:11.453356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.1e","mappings":[{"from":1,"to":5}]},{"pgid":"14.29","mappings":[{"from":1,"to":7}]},{"pgid":"14.2a","mappings":[{"from":1,"to":2}]},{"pgid":"14.2c","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6837/3556492262":"2024-05-21T18:34:35.299705+0000","172.21.15.5:6836/3556492262":"2024-05-21T18:34:35.299705+0000","172.21.15.63:6834/2759396130":"2024-05-21T18:33:23.233035+0000","172.21.15.63:6832/2759396130":"2024-05-21T18:33:23.233035+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:35:17.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.911+0000 7f4f5bd57640 1 -- 172.21.15.5:0/183835301 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4f3806be30 msgr2=0x7f4f3806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:17.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.911+0000 7f4f5bd57640 1 --2- 172.21.15.5:0/183835301 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4f3806be30 0x7f4f3806e340 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4f50034000 tx=0x7f4f5002e000 comp rx=0 tx=0).stop 2024-05-20T18:35:17.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.911+0000 7f4f5bd57640 1 -- 172.21.15.5:0/183835301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f54162820 msgr2=0x7f4f541f8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:17.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.911+0000 7f4f5bd57640 1 --2- 172.21.15.5:0/183835301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f54162820 0x7f4f541f8b90 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00c6e0 tx=0x7f4f4c00cba0 comp rx=0 tx=0).stop 2024-05-20T18:35:17.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.911+0000 7f4f5bd57640 1 -- 172.21.15.5:0/183835301 shutdown_connections 2024-05-20T18:35:17.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.911+0000 7f4f5bd57640 1 --2- 172.21.15.5:0/183835301 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4f3806be30 0x7f4f3806e340 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:17.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.911+0000 7f4f5bd57640 1 --2- 172.21.15.5:0/183835301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f54162820 0x7f4f541f8b90 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:17.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.911+0000 7f4f5bd57640 1 --2- 172.21.15.5:0/183835301 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f540b3e80 0x7f4f541f8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:17.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.911+0000 7f4f5bd57640 1 --2- 172.21.15.5:0/183835301 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f5406d910 0x7f4f5414df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:17.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.911+0000 7f4f5bd57640 1 -- 172.21.15.5:0/183835301 >> 172.21.15.5:0/183835301 conn(0x7f4f540d4bd0 msgr2=0x7f4f540d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:17.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.911+0000 7f4f5bd57640 1 -- 172.21.15.5:0/183835301 shutdown_connections 2024-05-20T18:35:17.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:17.911+0000 7f4f5bd57640 1 -- 172.21.15.5:0/183835301 wait complete. 2024-05-20T18:35:17.924 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-20T18:35:17.924 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:35:18.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.038+0000 7fd203456640 1 Processor -- start 2024-05-20T18:35:18.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.038+0000 7fd203456640 1 -- start start 2024-05-20T18:35:18.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.040+0000 7fd203456640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd1fc06d3d0 0x7fd1fc06d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.040+0000 7fd203456640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1fc0b8190 0x7fd1fc0b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.040+0000 7fd203456640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1fc06dd20 con 0x7fd1fc162820 2024-05-20T18:35:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.040+0000 7fd203456640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd1fc0d0630 con 0x7fd1fc06d3d0 2024-05-20T18:35:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.040+0000 7fd203456640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd1fc0d39c0 con 0x7fd1fc0b8190 2024-05-20T18:35:18.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.040+0000 7fd2019f3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1fc0b8190 0x7fd1fc0b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:18.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.040+0000 7fd2009f1640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd1fc06d3d0 0x7fd1fc06d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:18.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.040+0000 7fd2011f2640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fd1fc162820 0x7fd1fc0b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:47750/0 (socket says 172.21.15.5:47750) 2024-05-20T18:35:18.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.040+0000 7fd2019f3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1fc0b8190 0x7fd1fc0b85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:34400/0 (socket says 172.21.15.5:34400) 2024-05-20T18:35:18.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.040+0000 7fd2011f2640 1 -- 172.21.15.5:0/2265107908 learned_addr learned my addr 172.21.15.5:0/2265107908 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:18.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.041+0000 7fd2009f1640 1 -- 172.21.15.5:0/2265107908 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1fc0b8190 msgr2=0x7fd1fc0b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:18.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.041+0000 7fd2009f1640 1 --2- 172.21.15.5:0/2265107908 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1fc0b8190 0x7fd1fc0b85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.041+0000 7fd2009f1640 1 -- 172.21.15.5:0/2265107908 >> v1:172.21.15.63:6789/0 conn(0x7fd1fc162820 legacy=0x7fd1fc0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:18.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.041+0000 7fd2009f1640 1 -- 172.21.15.5:0/2265107908 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1fc0d3ca0 con 0x7fd1fc06d3d0 2024-05-20T18:35:18.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.041+0000 7fd2009f1640 1 --2- 172.21.15.5:0/2265107908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1fc06d3d0 0x7fd1fc06d7e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0097a0 tx=0x7fd1ec02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ebced553010d367 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.042+0000 7fd1f3fff640 1 -- 172.21.15.5:0/2265107908 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec037070 con 0x7fd1fc06d3d0 2024-05-20T18:35:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.042+0000 7fd1f3fff640 1 -- 172.21.15.5:0/2265107908 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1ec032440 con 0x7fd1fc06d3d0 2024-05-20T18:35:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.042+0000 7fd1f3fff640 1 -- 172.21.15.5:0/2265107908 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec032810 con 0x7fd1fc06d3d0 2024-05-20T18:35:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.042+0000 7fd203456640 1 -- 172.21.15.5:0/2265107908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1fc06d3d0 msgr2=0x7fd1fc06d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.042+0000 7fd203456640 1 --2- 172.21.15.5:0/2265107908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1fc06d3d0 0x7fd1fc06d7e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0097a0 tx=0x7fd1ec02a180 comp rx=0 tx=0).stop 2024-05-20T18:35:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.042+0000 7fd203456640 1 -- 172.21.15.5:0/2265107908 shutdown_connections 2024-05-20T18:35:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.042+0000 7fd203456640 1 --2- 172.21.15.5:0/2265107908 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1fc0b8190 0x7fd1fc0b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.042+0000 7fd203456640 1 --2- 172.21.15.5:0/2265107908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1fc06d3d0 0x7fd1fc06d7e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.042+0000 7fd203456640 1 -- 172.21.15.5:0/2265107908 >> 172.21.15.5:0/2265107908 conn(0x7fd1fc0d4bd0 msgr2=0x7fd1fc0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.043+0000 7fd203456640 1 -- 172.21.15.5:0/2265107908 shutdown_connections 2024-05-20T18:35:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.043+0000 7fd203456640 1 -- 172.21.15.5:0/2265107908 wait complete. 2024-05-20T18:35:18.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.044+0000 7fd203456640 1 Processor -- start 2024-05-20T18:35:18.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.044+0000 7fd203456640 1 -- start start 2024-05-20T18:35:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.046+0000 7fd203456640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1fc06d3d0 0x7fd1fc0ce930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.046+0000 7fd203456640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1fc0b8190 0x7fd1fc0c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.046+0000 7fd203456640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1fc162820 0x7fd1fc0c9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.046+0000 7fd203456640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc0d0300 con 0x7fd1fc162820 2024-05-20T18:35:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.046+0000 7fd203456640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc0d4260 con 0x7fd1fc0b8190 2024-05-20T18:35:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.046+0000 7fd203456640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd1fc0b4050 con 0x7fd1fc06d3d0 2024-05-20T18:35:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.046+0000 7fd2019f3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1fc162820 0x7fd1fc0c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.046+0000 7fd2019f3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1fc162820 0x7fd1fc0c9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55668/0 (socket says 172.21.15.5:55668) 2024-05-20T18:35:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.046+0000 7fd2019f3640 1 -- 172.21.15.5:0/2308125011 learned_addr learned my addr 172.21.15.5:0/2308125011 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.046+0000 7fd2011f2640 1 --2- 172.21.15.5:0/2308125011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1fc06d3d0 0x7fd1fc0ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.046+0000 7fd2009f1640 1 --2- 172.21.15.5:0/2308125011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1fc0b8190 0x7fd1fc0c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.046+0000 7fd2011f2640 1 -- 172.21.15.5:0/2308125011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1fc0b8190 msgr2=0x7fd1fc0c8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.046+0000 7fd2011f2640 1 --2- 172.21.15.5:0/2308125011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1fc0b8190 0x7fd1fc0c8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.046+0000 7fd2011f2640 1 -- 172.21.15.5:0/2308125011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1fc162820 msgr2=0x7fd1fc0c9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.046+0000 7fd2011f2640 1 --2- 172.21.15.5:0/2308125011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1fc162820 0x7fd1fc0c9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.046+0000 7fd2011f2640 1 -- 172.21.15.5:0/2308125011 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1fc0cee70 con 0x7fd1fc06d3d0 2024-05-20T18:35:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.046+0000 7fd2019f3640 1 --2- 172.21.15.5:0/2308125011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1fc162820 0x7fd1fc0c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:35:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.046+0000 7fd2011f2640 1 --2- 172.21.15.5:0/2308125011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1fc06d3d0 0x7fd1fc0ce930 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800c390 tx=0x7fd1f800c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.047+0000 7fd1f1ffb640 1 -- 172.21.15.5:0/2308125011 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8017070 con 0x7fd1fc06d3d0 2024-05-20T18:35:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.047+0000 7fd1f1ffb640 1 -- 172.21.15.5:0/2308125011 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1f8005ce0 con 0x7fd1fc06d3d0 2024-05-20T18:35:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.047+0000 7fd203456640 1 -- 172.21.15.5:0/2308125011 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1fc0cc230 con 0x7fd1fc06d3d0 2024-05-20T18:35:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.047+0000 7fd1f1ffb640 1 -- 172.21.15.5:0/2308125011 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8007310 con 0x7fd1fc06d3d0 2024-05-20T18:35:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.047+0000 7fd203456640 1 -- 172.21.15.5:0/2308125011 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1fc0cc7f0 con 0x7fd1fc06d3d0 2024-05-20T18:35:18.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.048+0000 7fd203456640 1 -- 172.21.15.5:0/2308125011 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1fc075b00 con 0x7fd1fc06d3d0 2024-05-20T18:35:18.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.049+0000 7fd1f1ffb640 1 -- 172.21.15.5:0/2308125011 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd1f801a070 con 0x7fd1fc06d3d0 2024-05-20T18:35:18.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.049+0000 7fd1f1ffb640 1 --2- 172.21.15.5:0/2308125011 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd1e006be30 0x7fd1e006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:18.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.049+0000 7fd1f1ffb640 1 -- 172.21.15.5:0/2308125011 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7fd1f808a420 con 0x7fd1fc06d3d0 2024-05-20T18:35:18.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.049+0000 7fd2009f1640 1 --2- 172.21.15.5:0/2308125011 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd1e006be30 0x7fd1e006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:18.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.050+0000 7fd2009f1640 1 --2- 172.21.15.5:0/2308125011 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd1e006be30 0x7fd1e006e340 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec034000 tx=0x7fd1ec02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:18.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.055+0000 7fd1f1ffb640 1 -- 172.21.15.5:0/2308125011 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd1fc06dd20 con 0x7fd1fc06d3d0 2024-05-20T18:35:18.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.280+0000 7fd203456640 1 -- 172.21.15.5:0/2308125011 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1fc15e9a0 con 0x7fd1fc06d3d0 2024-05-20T18:35:18.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.281+0000 7fd1f1ffb640 1 -- 172.21.15.5:0/2308125011 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5172 (secure 0 0 0) 0x7fd1f80583c0 con 0x7fd1fc06d3d0 2024-05-20T18:35:18.283 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:18.283 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/337248865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":337248865},{"type":"v1","addr":"172.21.15.63:6834","nonce":337248865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3705772553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3705772553},{"type":"v1","addr":"172.21.15.63:6835","nonce":3705772553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":7434,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/776188138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":776188138},{"type":"v1","addr":"172.21.15.5:6835","nonce":776188138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:33:12.457716+0000","modified":"2024-05-20T18:34:39.318498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7953":{"gid":7953,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6837/3880006778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3880006778},{"type":"v1","addr":"172.21.15.5:6837","nonce":3880006778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7953,"qdb_cluster":[7953]},"id":6}]} 2024-05-20T18:35:18.283 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 73 2024-05-20T18:35:18.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.283+0000 7fd203456640 1 -- 172.21.15.5:0/2308125011 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd1e006be30 msgr2=0x7fd1e006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:18.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.283+0000 7fd203456640 1 --2- 172.21.15.5:0/2308125011 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd1e006be30 0x7fd1e006e340 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec034000 tx=0x7fd1ec02e000 comp rx=0 tx=0).stop 2024-05-20T18:35:18.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.283+0000 7fd203456640 1 -- 172.21.15.5:0/2308125011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1fc06d3d0 msgr2=0x7fd1fc0ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:18.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.283+0000 7fd203456640 1 --2- 172.21.15.5:0/2308125011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1fc06d3d0 0x7fd1fc0ce930 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800c390 tx=0x7fd1f800c790 comp rx=0 tx=0).stop 2024-05-20T18:35:18.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.283+0000 7fd203456640 1 -- 172.21.15.5:0/2308125011 shutdown_connections 2024-05-20T18:35:18.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.283+0000 7fd203456640 1 --2- 172.21.15.5:0/2308125011 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd1e006be30 0x7fd1e006e340 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.283+0000 7fd203456640 1 --2- 172.21.15.5:0/2308125011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1fc162820 0x7fd1fc0c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.283+0000 7fd203456640 1 --2- 172.21.15.5:0/2308125011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd1fc0b8190 0x7fd1fc0c8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.283+0000 7fd203456640 1 --2- 172.21.15.5:0/2308125011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd1fc06d3d0 0x7fd1fc0ce930 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.283+0000 7fd203456640 1 -- 172.21.15.5:0/2308125011 >> 172.21.15.5:0/2308125011 conn(0x7fd1fc0d4bd0 msgr2=0x7fd1fc15d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:18.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.284+0000 7fd203456640 1 -- 172.21.15.5:0/2308125011 shutdown_connections 2024-05-20T18:35:18.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.284+0000 7fd203456640 1 -- 172.21.15.5:0/2308125011 wait complete. 2024-05-20T18:35:18.296 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:35:18.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.410+0000 7f76c3a32640 1 Processor -- start 2024-05-20T18:35:18.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.410+0000 7f76c3a32640 1 -- start start 2024-05-20T18:35:18.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.411+0000 7f76c3a32640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76bc15d530 0x7f76bc15aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:18.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.411+0000 7f76c3a32640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f76bc0b1700 0x7f76bc0b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:18.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.411+0000 7f76c3a32640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76bc0cd580 con 0x7f76bc070850 2024-05-20T18:35:18.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.411+0000 7f76c3a32640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f76bc0ce190 con 0x7f76bc0b1700 2024-05-20T18:35:18.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.411+0000 7f76c3a32640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f76bc0cd860 con 0x7f76bc15d530 2024-05-20T18:35:18.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.412+0000 7f76c17ce640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76bc15d530 0x7f76bc15aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:18.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.412+0000 7f76c0fcd640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f76bc0b1700 0x7f76bc0b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:18.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.412+0000 7f76c17ce640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76bc15d530 0x7f76bc15aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:34426/0 (socket says 172.21.15.5:34426) 2024-05-20T18:35:18.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.412+0000 7f76c17ce640 1 -- 172.21.15.5:0/1127105057 learned_addr learned my addr 172.21.15.5:0/1127105057 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:18.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.412+0000 7f76c17ce640 1 -- 172.21.15.5:0/1127105057 >> v2:172.21.15.63:3300/0 conn(0x7f76bc0b1700 msgr2=0x7f76bc0b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:18.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.412+0000 7f76c17ce640 1 --2- 172.21.15.5:0/1127105057 >> v2:172.21.15.63:3300/0 conn(0x7f76bc0b1700 0x7f76bc0b1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.412+0000 7f76c17ce640 1 -- 172.21.15.5:0/1127105057 >> v1:172.21.15.63:6789/0 conn(0x7f76bc070850 legacy=0x7f76bc070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:18.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.412+0000 7f76c17ce640 1 -- 172.21.15.5:0/1127105057 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76bc0cdb40 con 0x7f76bc15d530 2024-05-20T18:35:18.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.412+0000 7f76c0fcd640 1 --2- 172.21.15.5:0/1127105057 >> v2:172.21.15.63:3300/0 conn(0x7f76bc0b1700 0x7f76bc0b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:35:18.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.413+0000 7f76c17ce640 1 --2- 172.21.15.5:0/1127105057 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76bc15d530 0x7f76bc15aaa0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009870 tx=0x7f76b802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1472fe574164a26c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:18.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.413+0000 7f76b3fff640 1 -- 172.21.15.5:0/1127105057 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b8037070 con 0x7f76bc15d530 2024-05-20T18:35:18.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.413+0000 7f76b3fff640 1 -- 172.21.15.5:0/1127105057 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76b8032440 con 0x7f76bc15d530 2024-05-20T18:35:18.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.413+0000 7f76b3fff640 1 -- 172.21.15.5:0/1127105057 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b8032810 con 0x7f76bc15d530 2024-05-20T18:35:18.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.414+0000 7f76c3a32640 1 -- 172.21.15.5:0/1127105057 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76bc15d530 msgr2=0x7f76bc15aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:18.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.414+0000 7f76c3a32640 1 --2- 172.21.15.5:0/1127105057 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76bc15d530 0x7f76bc15aaa0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009870 tx=0x7f76b802a270 comp rx=0 tx=0).stop 2024-05-20T18:35:18.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.414+0000 7f76c3a32640 1 -- 172.21.15.5:0/1127105057 shutdown_connections 2024-05-20T18:35:18.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.414+0000 7f76c3a32640 1 --2- 172.21.15.5:0/1127105057 >> v2:172.21.15.63:3300/0 conn(0x7f76bc0b1700 0x7f76bc0b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.414+0000 7f76c3a32640 1 --2- 172.21.15.5:0/1127105057 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76bc15d530 0x7f76bc15aaa0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.414+0000 7f76c3a32640 1 -- 172.21.15.5:0/1127105057 >> 172.21.15.5:0/1127105057 conn(0x7f76bc0d29d0 msgr2=0x7f76bc0d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:18.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.414+0000 7f76c3a32640 1 -- 172.21.15.5:0/1127105057 shutdown_connections 2024-05-20T18:35:18.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.414+0000 7f76c3a32640 1 -- 172.21.15.5:0/1127105057 wait complete. 2024-05-20T18:35:18.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.416+0000 7f76c3a32640 1 Processor -- start 2024-05-20T18:35:18.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.416+0000 7f76c3a32640 1 -- start start 2024-05-20T18:35:18.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.417+0000 7f76c3a32640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76bc070850 0x7f76bc0c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:18.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.417+0000 7f76c3a32640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76bc0b1700 0x7f76bc0b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:18.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.417+0000 7f76c3a32640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76bc15d530 0x7f76bc0c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:18.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.417+0000 7f76c3a32640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc0b2050 con 0x7f76bc070850 2024-05-20T18:35:18.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.417+0000 7f76c3a32640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc0cd230 con 0x7f76bc15d530 2024-05-20T18:35:18.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.417+0000 7f76c3a32640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f76bc071370 con 0x7f76bc0b1700 2024-05-20T18:35:18.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.417+0000 7f76c17ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76bc070850 0x7f76bc0c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.417+0000 7f76c17ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76bc070850 0x7f76bc0c88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55672/0 (socket says 172.21.15.5:55672) 2024-05-20T18:35:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.417+0000 7f76c17ce640 1 -- 172.21.15.5:0/1635963492 learned_addr learned my addr 172.21.15.5:0/1635963492 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.417+0000 7f76c0fcd640 1 --2- 172.21.15.5:0/1635963492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76bc0b1700 0x7f76bc0b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.417+0000 7f76c1fcf640 1 --2- 172.21.15.5:0/1635963492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76bc15d530 0x7f76bc0c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.417+0000 7f76c17ce640 1 -- 172.21.15.5:0/1635963492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76bc0b1700 msgr2=0x7f76bc0b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.417+0000 7f76c17ce640 1 --2- 172.21.15.5:0/1635963492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76bc0b1700 0x7f76bc0b36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.417+0000 7f76c17ce640 1 -- 172.21.15.5:0/1635963492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76bc15d530 msgr2=0x7f76bc0c73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.417+0000 7f76c17ce640 1 --2- 172.21.15.5:0/1635963492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76bc15d530 0x7f76bc0c73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.418+0000 7f76c17ce640 1 -- 172.21.15.5:0/1635963492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76bc0c6630 con 0x7f76bc070850 2024-05-20T18:35:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.418+0000 7f76c17ce640 1 --2- 172.21.15.5:0/1635963492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76bc070850 0x7f76bc0c88e0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f76b80090b0 tx=0x7f76b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.418+0000 7f76b1ffb640 1 -- 172.21.15.5:0/1635963492 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b8041020 con 0x7f76bc070850 2024-05-20T18:35:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.418+0000 7f76b1ffb640 1 -- 172.21.15.5:0/1635963492 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76b8030660 con 0x7f76bc070850 2024-05-20T18:35:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.418+0000 7f76b1ffb640 1 -- 172.21.15.5:0/1635963492 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b8037040 con 0x7f76bc070850 2024-05-20T18:35:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.418+0000 7f76c3a32640 1 -- 172.21.15.5:0/1635963492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76bc0c6910 con 0x7f76bc070850 2024-05-20T18:35:18.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.418+0000 7f76c3a32640 1 -- 172.21.15.5:0/1635963492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76bc0cbcc0 con 0x7f76bc070850 2024-05-20T18:35:18.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.419+0000 7f76c3a32640 1 -- 172.21.15.5:0/1635963492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76bc075b00 con 0x7f76bc070850 2024-05-20T18:35:18.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.420+0000 7f76b1ffb640 1 -- 172.21.15.5:0/1635963492 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f76b8044020 con 0x7f76bc070850 2024-05-20T18:35:18.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.420+0000 7f76b1ffb640 1 --2- 172.21.15.5:0/1635963492 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f76a006be30 0x7f76a006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:18.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.421+0000 7f76b1ffb640 1 -- 172.21.15.5:0/1635963492 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7f76b80ab880 con 0x7f76bc070850 2024-05-20T18:35:18.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.421+0000 7f76c0fcd640 1 --2- 172.21.15.5:0/1635963492 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f76a006be30 0x7f76a006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:18.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.421+0000 7f76c0fcd640 1 --2- 172.21.15.5:0/1635963492 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f76a006be30 0x7f76a006e340 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f76bc15a5d0 tx=0x7f76ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:18.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.425+0000 7f76b1ffb640 1 -- 172.21.15.5:0/1635963492 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f76b804b030 con 0x7f76bc070850 2024-05-20T18:35:18.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.651+0000 7f76c3a32640 1 -- 172.21.15.5:0/1635963492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76bc15f820 con 0x7f76bc070850 2024-05-20T18:35:18.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.652+0000 7f76b1ffb640 1 -- 172.21.15.5:0/1635963492 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5172 (secure 0 0 0) 0x7f76b8079f80 con 0x7f76bc070850 2024-05-20T18:35:18.653 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:18.653 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/337248865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":337248865},{"type":"v1","addr":"172.21.15.63:6834","nonce":337248865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3705772553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3705772553},{"type":"v1","addr":"172.21.15.63:6835","nonce":3705772553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":7434,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/776188138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":776188138},{"type":"v1","addr":"172.21.15.5:6835","nonce":776188138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:33:12.457716+0000","modified":"2024-05-20T18:34:39.318498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7953":{"gid":7953,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6837/3880006778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3880006778},{"type":"v1","addr":"172.21.15.5:6837","nonce":3880006778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7953,"qdb_cluster":[7953]},"id":6}]} 2024-05-20T18:35:18.653 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 73 2024-05-20T18:35:18.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.654+0000 7f76c3a32640 1 -- 172.21.15.5:0/1635963492 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f76a006be30 msgr2=0x7f76a006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:18.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.654+0000 7f76c3a32640 1 --2- 172.21.15.5:0/1635963492 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f76a006be30 0x7f76a006e340 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f76bc15a5d0 tx=0x7f76ac008040 comp rx=0 tx=0).stop 2024-05-20T18:35:18.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.654+0000 7f76c3a32640 1 -- 172.21.15.5:0/1635963492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76bc070850 msgr2=0x7f76bc0c88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:18.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.654+0000 7f76c3a32640 1 --2- 172.21.15.5:0/1635963492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76bc070850 0x7f76bc0c88e0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f76b80090b0 tx=0x7f76b80046c0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.654+0000 7f76c3a32640 1 -- 172.21.15.5:0/1635963492 shutdown_connections 2024-05-20T18:35:18.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.654+0000 7f76c3a32640 1 --2- 172.21.15.5:0/1635963492 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f76a006be30 0x7f76a006e340 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.654+0000 7f76c3a32640 1 --2- 172.21.15.5:0/1635963492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76bc15d530 0x7f76bc0c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.654+0000 7f76c3a32640 1 --2- 172.21.15.5:0/1635963492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76bc0b1700 0x7f76bc0b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.654+0000 7f76c3a32640 1 --2- 172.21.15.5:0/1635963492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76bc070850 0x7f76bc0c88e0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.654+0000 7f76c3a32640 1 -- 172.21.15.5:0/1635963492 >> 172.21.15.5:0/1635963492 conn(0x7f76bc0d29d0 msgr2=0x7f76bc0d1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:18.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.654+0000 7f76c3a32640 1 -- 172.21.15.5:0/1635963492 shutdown_connections 2024-05-20T18:35:18.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.654+0000 7f76c3a32640 1 -- 172.21.15.5:0/1635963492 wait complete. 2024-05-20T18:35:18.666 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:35:18.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.780+0000 7feb89e44640 1 Processor -- start 2024-05-20T18:35:18.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.780+0000 7feb89e44640 1 -- start start 2024-05-20T18:35:18.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.782+0000 7feb89e44640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7feb84162820 0x7feb840b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:18.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.782+0000 7feb89e44640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb840b3e80 0x7feb8406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:18.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.782+0000 7feb89e44640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7feb8406d910 0x7feb840d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:18.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.782+0000 7feb89e44640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7feb8406dd20 con 0x7feb8406d910 2024-05-20T18:35:18.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.782+0000 7feb89e44640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7feb840d0630 con 0x7feb84162820 2024-05-20T18:35:18.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.782+0000 7feb89e44640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7feb840d39c0 con 0x7feb840b3e80 2024-05-20T18:35:18.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.782+0000 7feb83fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7feb8406d910 0x7feb840d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:18.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.782+0000 7feb83fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7feb8406d910 0x7feb840d27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55676/0 (socket says 172.21.15.5:55676) 2024-05-20T18:35:18.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.782+0000 7feb83fff640 1 -- 172.21.15.5:0/4039839733 learned_addr learned my addr 172.21.15.5:0/4039839733 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:18.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.782+0000 7feb82ffd640 1 --2- 172.21.15.5:0/4039839733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb840b3e80 0x7feb8406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:18.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.782+0000 7feb837fe640 1 --2- 172.21.15.5:0/4039839733 >> v2:172.21.15.63:3300/0 conn(0x7feb84162820 0x7feb840b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:18.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.782+0000 7feb83fff640 1 -- 172.21.15.5:0/4039839733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb840b3e80 msgr2=0x7feb8406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:18.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.782+0000 7feb83fff640 1 --2- 172.21.15.5:0/4039839733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb840b3e80 0x7feb8406d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.782+0000 7feb83fff640 1 -- 172.21.15.5:0/4039839733 >> v2:172.21.15.63:3300/0 conn(0x7feb84162820 msgr2=0x7feb840b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:18.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.782+0000 7feb83fff640 1 --2- 172.21.15.5:0/4039839733 >> v2:172.21.15.63:3300/0 conn(0x7feb84162820 0x7feb840b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.782+0000 7feb83fff640 1 -- 172.21.15.5:0/4039839733 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb840d3ca0 con 0x7feb8406d910 2024-05-20T18:35:18.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.783+0000 7feb83fff640 1 --2- 172.21.15.5:0/4039839733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb8406d910 0x7feb840d27a0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7feb840b7740 tx=0x7feb7c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d56109e0562076a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:18.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.783+0000 7feb827fc640 1 -- 172.21.15.5:0/4039839733 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c036070 con 0x7feb8406d910 2024-05-20T18:35:18.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.783+0000 7feb827fc640 1 -- 172.21.15.5:0/4039839733 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb7c031600 con 0x7feb8406d910 2024-05-20T18:35:18.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.783+0000 7feb827fc640 1 -- 172.21.15.5:0/4039839733 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c0319d0 con 0x7feb8406d910 2024-05-20T18:35:18.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.784+0000 7feb89e44640 1 -- 172.21.15.5:0/4039839733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb8406d910 msgr2=0x7feb840d27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:18.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.784+0000 7feb89e44640 1 --2- 172.21.15.5:0/4039839733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb8406d910 0x7feb840d27a0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7feb840b7740 tx=0x7feb7c02a6a0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.784+0000 7feb89e44640 1 -- 172.21.15.5:0/4039839733 shutdown_connections 2024-05-20T18:35:18.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.784+0000 7feb89e44640 1 --2- 172.21.15.5:0/4039839733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb8406d910 0x7feb840d27a0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.784+0000 7feb89e44640 1 --2- 172.21.15.5:0/4039839733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb840b3e80 0x7feb8406d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.784+0000 7feb89e44640 1 --2- 172.21.15.5:0/4039839733 >> v2:172.21.15.63:3300/0 conn(0x7feb84162820 0x7feb840b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.784+0000 7feb89e44640 1 -- 172.21.15.5:0/4039839733 >> 172.21.15.5:0/4039839733 conn(0x7feb840d4bd0 msgr2=0x7feb840d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:18.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.784+0000 7feb89e44640 1 -- 172.21.15.5:0/4039839733 shutdown_connections 2024-05-20T18:35:18.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.784+0000 7feb89e44640 1 -- 172.21.15.5:0/4039839733 wait complete. 2024-05-20T18:35:18.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.786+0000 7feb89e44640 1 Processor -- start 2024-05-20T18:35:18.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.786+0000 7feb89e44640 1 -- start start 2024-05-20T18:35:18.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.787+0000 7feb89e44640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb8406d910 0x7feb8414e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:18.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.787+0000 7feb89e44640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb840b3e80 0x7feb8414eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:18.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.787+0000 7feb89e44640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb84162820 0x7feb840c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:18.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.787+0000 7feb89e44640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feb840d0300 con 0x7feb84162820 2024-05-20T18:35:18.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.787+0000 7feb89e44640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feb840d4260 con 0x7feb840b3e80 2024-05-20T18:35:18.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.787+0000 7feb89e44640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7feb840d4780 con 0x7feb8406d910 2024-05-20T18:35:18.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.787+0000 7feb83fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb84162820 0x7feb840c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:18.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.787+0000 7feb83fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb84162820 0x7feb840c9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55682/0 (socket says 172.21.15.5:55682) 2024-05-20T18:35:18.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.787+0000 7feb83fff640 1 -- 172.21.15.5:0/3823647788 learned_addr learned my addr 172.21.15.5:0/3823647788 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:18.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.787+0000 7feb837fe640 1 --2- 172.21.15.5:0/3823647788 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb8406d910 0x7feb8414e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:18.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.787+0000 7feb82ffd640 1 --2- 172.21.15.5:0/3823647788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb840b3e80 0x7feb8414eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:18.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.788+0000 7feb83fff640 1 -- 172.21.15.5:0/3823647788 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb8406d910 msgr2=0x7feb8414e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:18.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.788+0000 7feb83fff640 1 --2- 172.21.15.5:0/3823647788 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb8406d910 0x7feb8414e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.788+0000 7feb83fff640 1 -- 172.21.15.5:0/3823647788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb840b3e80 msgr2=0x7feb8414eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:18.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.788+0000 7feb83fff640 1 --2- 172.21.15.5:0/3823647788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb840b3e80 0x7feb8414eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.788+0000 7feb83fff640 1 -- 172.21.15.5:0/3823647788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb8414f2e0 con 0x7feb84162820 2024-05-20T18:35:18.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.788+0000 7feb83fff640 1 --2- 172.21.15.5:0/3823647788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb84162820 0x7feb840c9a40 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7feb7c02ac90 tx=0x7feb7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:18.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.788+0000 7feb7bfff640 1 -- 172.21.15.5:0/3823647788 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c03f020 con 0x7feb84162820 2024-05-20T18:35:18.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.788+0000 7feb7bfff640 1 -- 172.21.15.5:0/3823647788 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb7c037040 con 0x7feb84162820 2024-05-20T18:35:18.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.788+0000 7feb7bfff640 1 -- 172.21.15.5:0/3823647788 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c036070 con 0x7feb84162820 2024-05-20T18:35:18.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.788+0000 7feb89e44640 1 -- 172.21.15.5:0/3823647788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb840c89a0 con 0x7feb84162820 2024-05-20T18:35:18.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.788+0000 7feb89e44640 1 -- 172.21.15.5:0/3823647788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb840c8f60 con 0x7feb84162820 2024-05-20T18:35:18.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.789+0000 7feb89e44640 1 -- 172.21.15.5:0/3823647788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb840c8700 con 0x7feb84162820 2024-05-20T18:35:18.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.790+0000 7feb7bfff640 1 -- 172.21.15.5:0/3823647788 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feb7c042020 con 0x7feb84162820 2024-05-20T18:35:18.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.790+0000 7feb7bfff640 1 --2- 172.21.15.5:0/3823647788 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feb6406be60 0x7feb6406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:18.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.790+0000 7feb7bfff640 1 -- 172.21.15.5:0/3823647788 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7feb7c0ab7f0 con 0x7feb84162820 2024-05-20T18:35:18.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.790+0000 7feb837fe640 1 --2- 172.21.15.5:0/3823647788 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feb6406be60 0x7feb6406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:18.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.791+0000 7feb837fe640 1 --2- 172.21.15.5:0/3823647788 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feb6406be60 0x7feb6406e370 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7feb740079a0 tx=0x7feb74008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:18.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.795+0000 7feb7bfff640 1 -- 172.21.15.5:0/3823647788 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7feb840c8700 con 0x7feb84162820 2024-05-20T18:35:18.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.979+0000 7feb89e44640 1 -- 172.21.15.5:0/3823647788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb84075b00 con 0x7feb84162820 2024-05-20T18:35:18.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.980+0000 7feb7bfff640 1 -- 172.21.15.5:0/3823647788 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+17216 (secure 0 0 0) 0x7feb7c049020 con 0x7feb84162820 2024-05-20T18:35:18.981 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:18.981 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":94,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:34:35.299736+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:33:10.184948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-20T18:33:11.453356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.1e","mappings":[{"from":1,"to":5}]},{"pgid":"14.29","mappings":[{"from":1,"to":7}]},{"pgid":"14.2a","mappings":[{"from":1,"to":2}]},{"pgid":"14.2c","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6837/3556492262":"2024-05-21T18:34:35.299705+0000","172.21.15.5:6836/3556492262":"2024-05-21T18:34:35.299705+0000","172.21.15.63:6834/2759396130":"2024-05-21T18:33:23.233035+0000","172.21.15.63:6832/2759396130":"2024-05-21T18:33:23.233035+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:35:18.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.982+0000 7feb89e44640 1 -- 172.21.15.5:0/3823647788 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feb6406be60 msgr2=0x7feb6406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:18.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.982+0000 7feb89e44640 1 --2- 172.21.15.5:0/3823647788 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feb6406be60 0x7feb6406e370 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7feb740079a0 tx=0x7feb74008040 comp rx=0 tx=0).stop 2024-05-20T18:35:18.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.982+0000 7feb89e44640 1 -- 172.21.15.5:0/3823647788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb84162820 msgr2=0x7feb840c9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:18.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.982+0000 7feb89e44640 1 --2- 172.21.15.5:0/3823647788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb84162820 0x7feb840c9a40 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7feb7c02ac90 tx=0x7feb7c004a40 comp rx=0 tx=0).stop 2024-05-20T18:35:18.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.982+0000 7feb89e44640 1 -- 172.21.15.5:0/3823647788 shutdown_connections 2024-05-20T18:35:18.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.982+0000 7feb89e44640 1 --2- 172.21.15.5:0/3823647788 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feb6406be60 0x7feb6406e370 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.982+0000 7feb89e44640 1 --2- 172.21.15.5:0/3823647788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb84162820 0x7feb840c9a40 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.982+0000 7feb89e44640 1 --2- 172.21.15.5:0/3823647788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb840b3e80 0x7feb8414eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.982+0000 7feb89e44640 1 --2- 172.21.15.5:0/3823647788 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb8406d910 0x7feb8414e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:18.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.982+0000 7feb89e44640 1 -- 172.21.15.5:0/3823647788 >> 172.21.15.5:0/3823647788 conn(0x7feb840d4bd0 msgr2=0x7feb840b8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:18.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.982+0000 7feb89e44640 1 -- 172.21.15.5:0/3823647788 shutdown_connections 2024-05-20T18:35:18.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:18.982+0000 7feb89e44640 1 -- 172.21.15.5:0/3823647788 wait complete. 2024-05-20T18:35:18.995 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-20T18:35:19.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.110+0000 7f08318e9640 1 Processor -- start 2024-05-20T18:35:19.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.110+0000 7f08318e9640 1 -- start start 2024-05-20T18:35:19.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.111+0000 7f08318e9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f082c163a10 0x7f082c0d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:19.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.111+0000 7f08318e9640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f082c0d14b0 0x7f082c0d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:19.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.111+0000 7f08318e9640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082c0d0370 con 0x7f082c0b4010 2024-05-20T18:35:19.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.111+0000 7f08318e9640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f082c0d3ca0 con 0x7f082c163a10 2024-05-20T18:35:19.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.111+0000 7f08318e9640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f082c0d0650 con 0x7f082c0d14b0 2024-05-20T18:35:19.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.111+0000 7f082affd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f082c163a10 0x7f082c0d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:19.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.111+0000 7f082a7fc640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f082c0b4010 0x7f082c0cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47346/0 (socket says 172.21.15.5:47346) 2024-05-20T18:35:19.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.111+0000 7f082a7fc640 1 -- 172.21.15.5:0/4262055098 learned_addr learned my addr 172.21.15.5:0/4262055098 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:19.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.111+0000 7f082b7fe640 1 --2- 172.21.15.5:0/4262055098 >> v2:172.21.15.63:3300/0 conn(0x7f082c0d14b0 0x7f082c0d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:19.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.112+0000 7f082affd640 1 -- 172.21.15.5:0/4262055098 >> v2:172.21.15.63:3300/0 conn(0x7f082c0d14b0 msgr2=0x7f082c0d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:19.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.112+0000 7f082affd640 1 --2- 172.21.15.5:0/4262055098 >> v2:172.21.15.63:3300/0 conn(0x7f082c0d14b0 0x7f082c0d18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:19.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.112+0000 7f082affd640 1 -- 172.21.15.5:0/4262055098 >> v1:172.21.15.5:6789/0 conn(0x7f082c0b4010 legacy=0x7f082c0cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:19.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.112+0000 7f0829ffb640 1 -- 172.21.15.5:0/4262055098 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1683043721 0 0) 0x7f082c0d0370 con 0x7f082c0b4010 2024-05-20T18:35:19.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.112+0000 7f082affd640 1 -- 172.21.15.5:0/4262055098 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f082c0d3650 con 0x7f082c163a10 2024-05-20T18:35:19.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.112+0000 7f082affd640 1 --2- 172.21.15.5:0/4262055098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c163a10 0x7f082c0d4130 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f082c0b4590 tx=0x7f082402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afea9268e2bccefd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:19.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.112+0000 7f0829ffb640 1 -- 172.21.15.5:0/4262055098 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0824037070 con 0x7f082c163a10 2024-05-20T18:35:19.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.113+0000 7f0829ffb640 1 -- 172.21.15.5:0/4262055098 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0824032440 con 0x7f082c163a10 2024-05-20T18:35:19.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.113+0000 7f0829ffb640 1 -- 172.21.15.5:0/4262055098 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0824032810 con 0x7f082c163a10 2024-05-20T18:35:19.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.113+0000 7f08318e9640 1 -- 172.21.15.5:0/4262055098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c163a10 msgr2=0x7f082c0d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:19.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.113+0000 7f08318e9640 1 --2- 172.21.15.5:0/4262055098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c163a10 0x7f082c0d4130 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f082c0b4590 tx=0x7f082402a1a0 comp rx=0 tx=0).stop 2024-05-20T18:35:19.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.113+0000 7f08318e9640 1 -- 172.21.15.5:0/4262055098 shutdown_connections 2024-05-20T18:35:19.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.113+0000 7f08318e9640 1 --2- 172.21.15.5:0/4262055098 >> v2:172.21.15.63:3300/0 conn(0x7f082c0d14b0 0x7f082c0d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:19.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.113+0000 7f08318e9640 1 --2- 172.21.15.5:0/4262055098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c163a10 0x7f082c0d4130 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:19.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.113+0000 7f08318e9640 1 -- 172.21.15.5:0/4262055098 >> 172.21.15.5:0/4262055098 conn(0x7f082c06d240 msgr2=0x7f082c06d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:19.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.113+0000 7f08318e9640 1 -- 172.21.15.5:0/4262055098 shutdown_connections 2024-05-20T18:35:19.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.113+0000 7f08318e9640 1 -- 172.21.15.5:0/4262055098 wait complete. 2024-05-20T18:35:19.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.115+0000 7f08318e9640 1 Processor -- start 2024-05-20T18:35:19.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.115+0000 7f08318e9640 1 -- start start 2024-05-20T18:35:19.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.116+0000 7f08318e9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f082c0b4010 0x7f082c0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:19.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.116+0000 7f08318e9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c0d14b0 0x7f082c14a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:19.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.117+0000 7f08318e9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f082c163a10 0x7f082c15fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:19.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.117+0000 7f08318e9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f082c0b8210 con 0x7f082c0d14b0 2024-05-20T18:35:19.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.117+0000 7f08318e9640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f082c0d1e00 con 0x7f082c163a10 2024-05-20T18:35:19.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.117+0000 7f08318e9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f082c0b8390 con 0x7f082c0b4010 2024-05-20T18:35:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.117+0000 7f082a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c0d14b0 0x7f082c14a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.117+0000 7f082a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c0d14b0 0x7f082c14a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55714/0 (socket says 172.21.15.5:55714) 2024-05-20T18:35:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.117+0000 7f082a7fc640 1 -- 172.21.15.5:0/3253121247 learned_addr learned my addr 172.21.15.5:0/3253121247 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.117+0000 7f082affd640 1 --2- 172.21.15.5:0/3253121247 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f082c0b4010 0x7f082c0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.117+0000 7f082b7fe640 1 --2- 172.21.15.5:0/3253121247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f082c163a10 0x7f082c15fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.117+0000 7f082a7fc640 1 -- 172.21.15.5:0/3253121247 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f082c0b4010 msgr2=0x7f082c0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.117+0000 7f082a7fc640 1 --2- 172.21.15.5:0/3253121247 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f082c0b4010 0x7f082c0cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.117+0000 7f082a7fc640 1 -- 172.21.15.5:0/3253121247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f082c163a10 msgr2=0x7f082c15fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.117+0000 7f082a7fc640 1 --2- 172.21.15.5:0/3253121247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f082c163a10 0x7f082c15fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.117+0000 7f082a7fc640 1 -- 172.21.15.5:0/3253121247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f082c14b720 con 0x7f082c0d14b0 2024-05-20T18:35:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.117+0000 7f082a7fc640 1 --2- 172.21.15.5:0/3253121247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c0d14b0 0x7f082c14a9b0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f081400c430 tx=0x7f081400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.117+0000 7f0823fff640 1 -- 172.21.15.5:0/3253121247 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08140074e0 con 0x7f082c0d14b0 2024-05-20T18:35:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.117+0000 7f0823fff640 1 -- 172.21.15.5:0/3253121247 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08140047e0 con 0x7f082c0d14b0 2024-05-20T18:35:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.117+0000 7f08318e9640 1 -- 172.21.15.5:0/3253121247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f082c148840 con 0x7f082c0d14b0 2024-05-20T18:35:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.117+0000 7f0823fff640 1 -- 172.21.15.5:0/3253121247 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814004bd0 con 0x7f082c0d14b0 2024-05-20T18:35:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.117+0000 7f08318e9640 1 -- 172.21.15.5:0/3253121247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f082c148e00 con 0x7f082c0d14b0 2024-05-20T18:35:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.119+0000 7f08318e9640 1 -- 172.21.15.5:0/3253121247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f082c14b490 con 0x7f082c0d14b0 2024-05-20T18:35:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.119+0000 7f0823fff640 1 -- 172.21.15.5:0/3253121247 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0814011070 con 0x7f082c0d14b0 2024-05-20T18:35:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.119+0000 7f0823fff640 1 --2- 172.21.15.5:0/3253121247 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f080c06be30 0x7f080c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.120+0000 7f0823fff640 1 -- 172.21.15.5:0/3253121247 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6389+0+0 (secure 0 0 0) 0x7f081408ad60 con 0x7f082c0d14b0 2024-05-20T18:35:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.120+0000 7f082affd640 1 --2- 172.21.15.5:0/3253121247 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f080c06be30 0x7f080c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.120+0000 7f082affd640 1 --2- 172.21.15.5:0/3253121247 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f080c06be30 0x7f080c06e340 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0824034000 tx=0x7f082402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:19.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.124+0000 7f0823fff640 1 -- 172.21.15.5:0/3253121247 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0814058ce0 con 0x7f082c0d14b0 2024-05-20T18:35:19.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.345+0000 7f08318e9640 1 -- 172.21.15.5:0/3253121247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f082c075b00 con 0x7f082c0d14b0 2024-05-20T18:35:19.346 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-20T18:35:19.346+0000 7fd1cf53a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-20T18:35:19.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.347+0000 7f0823fff640 1 -- 172.21.15.5:0/3253121247 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v74) ==== 161+0+0 (secure 0 0 0) 0x7f081405cfe0 con 0x7f082c0d14b0 2024-05-20T18:35:19.348 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-20T18:35:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.350+0000 7f08318e9640 1 -- 172.21.15.5:0/3253121247 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f080c06be30 msgr2=0x7f080c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.350+0000 7f08318e9640 1 --2- 172.21.15.5:0/3253121247 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f080c06be30 0x7f080c06e340 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0824034000 tx=0x7f082402e000 comp rx=0 tx=0).stop 2024-05-20T18:35:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.350+0000 7f08318e9640 1 -- 172.21.15.5:0/3253121247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c0d14b0 msgr2=0x7f082c14a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.350+0000 7f08318e9640 1 --2- 172.21.15.5:0/3253121247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c0d14b0 0x7f082c14a9b0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f081400c430 tx=0x7f081400c8f0 comp rx=0 tx=0).stop 2024-05-20T18:35:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.350+0000 7f08318e9640 1 -- 172.21.15.5:0/3253121247 shutdown_connections 2024-05-20T18:35:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.350+0000 7f08318e9640 1 --2- 172.21.15.5:0/3253121247 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f080c06be30 0x7f080c06e340 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.350+0000 7f08318e9640 1 --2- 172.21.15.5:0/3253121247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f082c163a10 0x7f082c15fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.350+0000 7f08318e9640 1 --2- 172.21.15.5:0/3253121247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c0d14b0 0x7f082c14a9b0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.350+0000 7f08318e9640 1 --2- 172.21.15.5:0/3253121247 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f082c0b4010 0x7f082c0cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.350+0000 7f08318e9640 1 -- 172.21.15.5:0/3253121247 >> 172.21.15.5:0/3253121247 conn(0x7f082c06d240 msgr2=0x7f082c15eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.350+0000 7f08318e9640 1 -- 172.21.15.5:0/3253121247 shutdown_connections 2024-05-20T18:35:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.350+0000 7f08318e9640 1 -- 172.21.15.5:0/3253121247 wait complete. 2024-05-20T18:35:19.353 INFO:tasks.ceph.mds.c.smithi005.stderr: -2406> 2024-05-20T18:34:41.311+0000 7f6ed9e7a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T18:35:19.353 INFO:tasks.ceph.mds.c.smithi005.stderr: -1951> 2024-05-20T18:34:42.183+0000 7f6ed5e72640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x557137c66b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v8>,<0x1/dir v15>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-20T18:35:19.365 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-20T18:35:19.412 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-05-20T18:35:19.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.485+0000 7f099bf38640 1 Processor -- start 2024-05-20T18:35:19.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.485+0000 7f099bf38640 1 -- start start 2024-05-20T18:35:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.486+0000 7f099bf38640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f09940d10e0 0x7f09940cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.486+0000 7f099bf38640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f09940b7210 0x7f09940cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.486+0000 7f099bf38640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09940d1ac0 con 0x7f09940d14f0 2024-05-20T18:35:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.486+0000 7f099bf38640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f09940d1da0 con 0x7f09940b7210 2024-05-20T18:35:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.486+0000 7f099bf38640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f09940d2080 con 0x7f09940d10e0 2024-05-20T18:35:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.487+0000 7f099a4d5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f09940b7210 0x7f09940cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.487+0000 7f099a4d5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f09940b7210 0x7f09940cf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55738/0 (socket says 172.21.15.5:55738) 2024-05-20T18:35:19.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.487+0000 7f099a4d5640 1 -- 172.21.15.5:0/1291013764 learned_addr learned my addr 172.21.15.5:0/1291013764 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:19.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.487+0000 7f0999cd4640 1 --2- 172.21.15.5:0/1291013764 >> v2:172.21.15.63:3300/0 conn(0x7f09940d10e0 0x7f09940cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:19.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.487+0000 7f099a4d5640 1 -- 172.21.15.5:0/1291013764 >> v2:172.21.15.63:3300/0 conn(0x7f09940d10e0 msgr2=0x7f09940cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:19.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.487+0000 7f099a4d5640 1 --2- 172.21.15.5:0/1291013764 >> v2:172.21.15.63:3300/0 conn(0x7f09940d10e0 0x7f09940cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:19.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.487+0000 7f099a4d5640 1 -- 172.21.15.5:0/1291013764 >> v1:172.21.15.63:6789/0 conn(0x7f09940d14f0 legacy=0x7f09940cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:19.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.487+0000 7f099a4d5640 1 -- 172.21.15.5:0/1291013764 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09940d2360 con 0x7f09940b7210 2024-05-20T18:35:19.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.487+0000 7f0999cd4640 1 --2- 172.21.15.5:0/1291013764 >> v2:172.21.15.63:3300/0 conn(0x7f09940d10e0 0x7f09940cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:19.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.488+0000 7f099a4d5640 1 --2- 172.21.15.5:0/1291013764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09940b7210 0x7f09940cf680 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f09940ccfc0 tx=0x7f098c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb586b50667d0ab9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:19.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.488+0000 7f0998cd2640 1 -- 172.21.15.5:0/1291013764 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c036070 con 0x7f09940b7210 2024-05-20T18:35:19.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.488+0000 7f0998cd2640 1 -- 172.21.15.5:0/1291013764 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f098c031600 con 0x7f09940b7210 2024-05-20T18:35:19.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.488+0000 7f0998cd2640 1 -- 172.21.15.5:0/1291013764 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c0319d0 con 0x7f09940b7210 2024-05-20T18:35:19.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.489+0000 7f099bf38640 1 -- 172.21.15.5:0/1291013764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09940b7210 msgr2=0x7f09940cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:19.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.489+0000 7f099bf38640 1 --2- 172.21.15.5:0/1291013764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09940b7210 0x7f09940cf680 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f09940ccfc0 tx=0x7f098c02a730 comp rx=0 tx=0).stop 2024-05-20T18:35:19.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.489+0000 7f099bf38640 1 -- 172.21.15.5:0/1291013764 shutdown_connections 2024-05-20T18:35:19.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.489+0000 7f099bf38640 1 --2- 172.21.15.5:0/1291013764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09940b7210 0x7f09940cf680 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:19.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.489+0000 7f099bf38640 1 --2- 172.21.15.5:0/1291013764 >> v2:172.21.15.63:3300/0 conn(0x7f09940d10e0 0x7f09940cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:19.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.489+0000 7f099bf38640 1 -- 172.21.15.5:0/1291013764 >> 172.21.15.5:0/1291013764 conn(0x7f09940b44d0 msgr2=0x7f09940b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:19.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.489+0000 7f099bf38640 1 -- 172.21.15.5:0/1291013764 shutdown_connections 2024-05-20T18:35:19.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.489+0000 7f099bf38640 1 -- 172.21.15.5:0/1291013764 wait complete. 2024-05-20T18:35:19.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.491+0000 7f099bf38640 1 Processor -- start 2024-05-20T18:35:19.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.491+0000 7f099bf38640 1 -- start start 2024-05-20T18:35:19.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.492+0000 7f099bf38640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09940b7210 0x7f099414ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:19.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.492+0000 7f099bf38640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09940d10e0 0x7f099414dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:19.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.492+0000 7f099bf38640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09940d14f0 0x7f09940cab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:19.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.492+0000 7f099bf38640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f09940b8440 con 0x7f09940b7210 2024-05-20T18:35:19.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.492+0000 7f099bf38640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f09940b82c0 con 0x7f09940d14f0 2024-05-20T18:35:19.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.492+0000 7f099bf38640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f09940709f0 con 0x7f09940d10e0 2024-05-20T18:35:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.492+0000 7f0999cd4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09940b7210 0x7f099414ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.492+0000 7f0999cd4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09940b7210 0x7f099414ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55740/0 (socket says 172.21.15.5:55740) 2024-05-20T18:35:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.492+0000 7f0999cd4640 1 -- 172.21.15.5:0/2898753458 learned_addr learned my addr 172.21.15.5:0/2898753458 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.492+0000 7f099a4d5640 1 --2- 172.21.15.5:0/2898753458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09940d14f0 0x7f09940cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.492+0000 7f09994d3640 1 --2- 172.21.15.5:0/2898753458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09940d10e0 0x7f099414dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.492+0000 7f0999cd4640 1 -- 172.21.15.5:0/2898753458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09940d10e0 msgr2=0x7f099414dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.492+0000 7f0999cd4640 1 --2- 172.21.15.5:0/2898753458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09940d10e0 0x7f099414dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.492+0000 7f0999cd4640 1 -- 172.21.15.5:0/2898753458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09940d14f0 msgr2=0x7f09940cab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.492+0000 7f0999cd4640 1 --2- 172.21.15.5:0/2898753458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09940d14f0 0x7f09940cab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.492+0000 7f0999cd4640 1 -- 172.21.15.5:0/2898753458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09940c9af0 con 0x7f09940b7210 2024-05-20T18:35:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.493+0000 7f0999cd4640 1 --2- 172.21.15.5:0/2898753458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09940b7210 0x7f099414ebb0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f099000d7d0 tx=0x7f099000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.493+0000 7f098a7fc640 1 -- 172.21.15.5:0/2898753458 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0990004030 con 0x7f09940b7210 2024-05-20T18:35:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.493+0000 7f098a7fc640 1 -- 172.21.15.5:0/2898753458 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09900041d0 con 0x7f09940b7210 2024-05-20T18:35:19.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.493+0000 7f099bf38640 1 -- 172.21.15.5:0/2898753458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09940c9dd0 con 0x7f09940b7210 2024-05-20T18:35:19.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.493+0000 7f098a7fc640 1 -- 172.21.15.5:0/2898753458 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099000b6c0 con 0x7f09940b7210 2024-05-20T18:35:19.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.493+0000 7f099bf38640 1 -- 172.21.15.5:0/2898753458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09940ca390 con 0x7f09940b7210 2024-05-20T18:35:19.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.494+0000 7f099bf38640 1 -- 172.21.15.5:0/2898753458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0994075b00 con 0x7f09940b7210 2024-05-20T18:35:19.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.494+0000 7f098a7fc640 1 -- 172.21.15.5:0/2898753458 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f099000b860 con 0x7f09940b7210 2024-05-20T18:35:19.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.495+0000 7f098a7fc640 1 --2- 172.21.15.5:0/2898753458 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f097806be30 0x7f097806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:19.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.495+0000 7f09994d3640 1 --2- 172.21.15.5:0/2898753458 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f097806be30 0x7f097806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:19.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.495+0000 7f098a7fc640 1 -- 172.21.15.5:0/2898753458 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6475+0+0 (secure 0 0 0) 0x7f099008b400 con 0x7f09940b7210 2024-05-20T18:35:19.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.496+0000 7f09994d3640 1 --2- 172.21.15.5:0/2898753458 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f097806be30 0x7f097806e340 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f09940cdf80 tx=0x7f0984009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:19.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.500+0000 7f098a7fc640 1 -- 172.21.15.5:0/2898753458 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0990059b40 con 0x7f09940b7210 2024-05-20T18:35:19.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:19.724+0000 7f099bf38640 1 -- 172.21.15.5:0/2898753458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f09940b1a40 con 0x7f09940b7210 2024-05-20T18:35:20.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.348+0000 7f098a7fc640 1 -- 172.21.15.5:0/2898753458 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v75) ==== 104+0+0 (secure 0 0 0) 0x7f0990059440 con 0x7f09940b7210 2024-05-20T18:35:20.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.351+0000 7f099bf38640 1 -- 172.21.15.5:0/2898753458 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f097806be30 msgr2=0x7f097806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:20.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.351+0000 7f099bf38640 1 --2- 172.21.15.5:0/2898753458 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f097806be30 0x7f097806e340 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f09940cdf80 tx=0x7f0984009040 comp rx=0 tx=0).stop 2024-05-20T18:35:20.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.351+0000 7f099bf38640 1 -- 172.21.15.5:0/2898753458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09940b7210 msgr2=0x7f099414ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:20.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.351+0000 7f099bf38640 1 --2- 172.21.15.5:0/2898753458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09940b7210 0x7f099414ebb0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f099000d7d0 tx=0x7f099000dc90 comp rx=0 tx=0).stop 2024-05-20T18:35:20.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.351+0000 7f099bf38640 1 -- 172.21.15.5:0/2898753458 shutdown_connections 2024-05-20T18:35:20.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.351+0000 7f099bf38640 1 --2- 172.21.15.5:0/2898753458 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f097806be30 0x7f097806e340 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:20.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.351+0000 7f099bf38640 1 --2- 172.21.15.5:0/2898753458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09940d14f0 0x7f09940cab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:20.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.351+0000 7f099bf38640 1 --2- 172.21.15.5:0/2898753458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09940d10e0 0x7f099414dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:20.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.351+0000 7f099bf38640 1 --2- 172.21.15.5:0/2898753458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09940b7210 0x7f099414ebb0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:20.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.351+0000 7f099bf38640 1 -- 172.21.15.5:0/2898753458 >> 172.21.15.5:0/2898753458 conn(0x7f09940b44d0 msgr2=0x7f099415cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:20.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.351+0000 7f099bf38640 1 -- 172.21.15.5:0/2898753458 shutdown_connections 2024-05-20T18:35:20.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.351+0000 7f099bf38640 1 -- 172.21.15.5:0/2898753458 wait complete. 2024-05-20T18:35:20.366 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-20T18:35:20.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.488+0000 7f425359e640 1 Processor -- start 2024-05-20T18:35:20.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.488+0000 7f425359e640 1 -- start start 2024-05-20T18:35:20.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.490+0000 7f425359e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42540cf670 0x7f425415fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:20.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.490+0000 7f425359e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f42540b6440 0x7f42540b6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:20.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.490+0000 7f425359e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f425415f6e0 con 0x7f42540b3d10 2024-05-20T18:35:20.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.490+0000 7f425359e640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f4254160400 con 0x7f42540b6440 2024-05-20T18:35:20.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.490+0000 7f425359e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f42540d3c50 con 0x7f42540cf670 2024-05-20T18:35:20.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.490+0000 7f4252d9d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f42540b6440 0x7f42540b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:20.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.490+0000 7f425259c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42540cf670 0x7f425415fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:20.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.490+0000 7f4252d9d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f42540b6440 0x7f42540b6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:44580/0 (socket says 172.21.15.5:44580) 2024-05-20T18:35:20.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.490+0000 7f4252d9d640 1 -- 172.21.15.5:0/186506073 learned_addr learned my addr 172.21.15.5:0/186506073 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:20.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.491+0000 7f4252d9d640 1 -- 172.21.15.5:0/186506073 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42540cf670 msgr2=0x7f425415fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:20.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.491+0000 7f4252d9d640 1 --2- 172.21.15.5:0/186506073 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42540cf670 0x7f425415fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:20.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.491+0000 7f4252d9d640 1 -- 172.21.15.5:0/186506073 >> v1:172.21.15.63:6789/0 conn(0x7f42540b3d10 legacy=0x7f42540d3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:20.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.491+0000 7f4252d9d640 1 -- 172.21.15.5:0/186506073 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42540b6d90 con 0x7f42540b6440 2024-05-20T18:35:20.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.491+0000 7f425259c640 1 --2- 172.21.15.5:0/186506073 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42540cf670 0x7f425415fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:35:20.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.492+0000 7f4252d9d640 1 --2- 172.21.15.5:0/186506073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42540b6440 0x7f42540b6850 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f424c009e80 tx=0x7f424c02a750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=695a0c67886240ba server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:20.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.492+0000 7f425159a640 1 -- 172.21.15.5:0/186506073 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424c036070 con 0x7f42540b6440 2024-05-20T18:35:20.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.492+0000 7f425159a640 1 -- 172.21.15.5:0/186506073 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f424c031600 con 0x7f42540b6440 2024-05-20T18:35:20.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.492+0000 7f425159a640 1 -- 172.21.15.5:0/186506073 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424c0319d0 con 0x7f42540b6440 2024-05-20T18:35:20.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.492+0000 7f425359e640 1 -- 172.21.15.5:0/186506073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42540b6440 msgr2=0x7f42540b6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:20.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.492+0000 7f425359e640 1 --2- 172.21.15.5:0/186506073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42540b6440 0x7f42540b6850 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f424c009e80 tx=0x7f424c02a750 comp rx=0 tx=0).stop 2024-05-20T18:35:20.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.493+0000 7f425359e640 1 -- 172.21.15.5:0/186506073 shutdown_connections 2024-05-20T18:35:20.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.493+0000 7f425359e640 1 --2- 172.21.15.5:0/186506073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42540b6440 0x7f42540b6850 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:20.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.493+0000 7f425359e640 1 --2- 172.21.15.5:0/186506073 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42540cf670 0x7f425415fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:20.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.493+0000 7f425359e640 1 -- 172.21.15.5:0/186506073 >> 172.21.15.5:0/186506073 conn(0x7f42540d49c0 msgr2=0x7f42540d4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:20.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.493+0000 7f425359e640 1 -- 172.21.15.5:0/186506073 shutdown_connections 2024-05-20T18:35:20.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.493+0000 7f425359e640 1 -- 172.21.15.5:0/186506073 wait complete. 2024-05-20T18:35:20.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.495+0000 7f425359e640 1 Processor -- start 2024-05-20T18:35:20.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.495+0000 7f425359e640 1 -- start start 2024-05-20T18:35:20.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.496+0000 7f425359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42540b3d10 0x7f4254159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:20.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.496+0000 7f425359e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42540b6440 0x7f4254155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:20.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.496+0000 7f425359e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42540cf670 0x7f42541e96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:20.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.496+0000 7f425359e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f425415f9a0 con 0x7f42540b3d10 2024-05-20T18:35:20.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.496+0000 7f425359e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f42540d4190 con 0x7f42540cf670 2024-05-20T18:35:20.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.496+0000 7f425359e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f42540b4120 con 0x7f42540b6440 2024-05-20T18:35:20.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.496+0000 7f425259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42540b3d10 0x7f4254159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:20.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.496+0000 7f425259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42540b3d10 0x7f4254159ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55746/0 (socket says 172.21.15.5:55746) 2024-05-20T18:35:20.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.496+0000 7f425259c640 1 -- 172.21.15.5:0/3917959727 learned_addr learned my addr 172.21.15.5:0/3917959727 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:20.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.496+0000 7f4251d9b640 1 --2- 172.21.15.5:0/3917959727 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42540b6440 0x7f4254155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:20.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.496+0000 7f4252d9d640 1 --2- 172.21.15.5:0/3917959727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42540cf670 0x7f42541e96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:20.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.496+0000 7f425259c640 1 -- 172.21.15.5:0/3917959727 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42540b6440 msgr2=0x7f4254155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:20.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.496+0000 7f425259c640 1 --2- 172.21.15.5:0/3917959727 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42540b6440 0x7f4254155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:20.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.496+0000 7f425259c640 1 -- 172.21.15.5:0/3917959727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42540cf670 msgr2=0x7f42541e96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:20.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.496+0000 7f425259c640 1 --2- 172.21.15.5:0/3917959727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42540cf670 0x7f42541e96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.496+0000 7f425259c640 1 -- 172.21.15.5:0/3917959727 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f425415a270 con 0x7f42540b3d10 2024-05-20T18:35:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.497+0000 7f425259c640 1 --2- 172.21.15.5:0/3917959727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42540b3d10 0x7f4254159ab0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f424400e960 tx=0x7f424400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.497+0000 7f424affd640 1 -- 172.21.15.5:0/3917959727 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4244017070 con 0x7f42540b3d10 2024-05-20T18:35:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.497+0000 7f424affd640 1 -- 172.21.15.5:0/3917959727 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f424400ca00 con 0x7f42540b3d10 2024-05-20T18:35:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.497+0000 7f424affd640 1 -- 172.21.15.5:0/3917959727 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424400cdd0 con 0x7f42540b3d10 2024-05-20T18:35:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.497+0000 7f425359e640 1 -- 172.21.15.5:0/3917959727 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f425415a550 con 0x7f42540b3d10 2024-05-20T18:35:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.497+0000 7f425359e640 1 -- 172.21.15.5:0/3917959727 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f425415ab10 con 0x7f42540b3d10 2024-05-20T18:35:20.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.498+0000 7f425359e640 1 -- 172.21.15.5:0/3917959727 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4254075980 con 0x7f42540b3d10 2024-05-20T18:35:20.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.499+0000 7f424affd640 1 -- 172.21.15.5:0/3917959727 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f424401a070 con 0x7f42540b3d10 2024-05-20T18:35:20.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.499+0000 7f424affd640 1 --2- 172.21.15.5:0/3917959727 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f423406be30 0x7f423406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:20.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.499+0000 7f4251d9b640 1 --2- 172.21.15.5:0/3917959727 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f423406be30 0x7f423406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:20.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.499+0000 7f424affd640 1 -- 172.21.15.5:0/3917959727 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6475+0+0 (secure 0 0 0) 0x7f424408b3d0 con 0x7f42540b3d10 2024-05-20T18:35:20.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.500+0000 7f4251d9b640 1 --2- 172.21.15.5:0/3917959727 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f423406be30 0x7f423406e340 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f423c007fd0 tx=0x7f423c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:20.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.504+0000 7f424affd640 1 -- 172.21.15.5:0/3917959727 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4244059a80 con 0x7f42540b3d10 2024-05-20T18:35:20.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:20.689+0000 7f425359e640 1 -- 172.21.15.5:0/3917959727 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f42540cad40 con 0x7f42540b3d10 2024-05-20T18:35:21.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.353+0000 7f424affd640 1 -- 172.21.15.5:0/3917959727 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v96) ==== 181+0+0 (secure 0 0 0) 0x7f4244059380 con 0x7f42540b3d10 2024-05-20T18:35:21.354 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-05-20T18:35:21.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.356+0000 7f425359e640 1 -- 172.21.15.5:0/3917959727 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f423406be30 msgr2=0x7f423406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:21.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.356+0000 7f425359e640 1 --2- 172.21.15.5:0/3917959727 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f423406be30 0x7f423406e340 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f423c007fd0 tx=0x7f423c009040 comp rx=0 tx=0).stop 2024-05-20T18:35:21.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.357+0000 7f425359e640 1 -- 172.21.15.5:0/3917959727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42540b3d10 msgr2=0x7f4254159ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:21.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.357+0000 7f425359e640 1 --2- 172.21.15.5:0/3917959727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42540b3d10 0x7f4254159ab0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f424400e960 tx=0x7f424400ee20 comp rx=0 tx=0).stop 2024-05-20T18:35:21.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.357+0000 7f425359e640 1 -- 172.21.15.5:0/3917959727 shutdown_connections 2024-05-20T18:35:21.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.357+0000 7f425359e640 1 --2- 172.21.15.5:0/3917959727 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f423406be30 0x7f423406e340 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:21.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.357+0000 7f425359e640 1 --2- 172.21.15.5:0/3917959727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42540cf670 0x7f42541e96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:21.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.357+0000 7f425359e640 1 --2- 172.21.15.5:0/3917959727 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42540b6440 0x7f4254155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:21.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.357+0000 7f425359e640 1 --2- 172.21.15.5:0/3917959727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42540b3d10 0x7f4254159ab0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:21.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.357+0000 7f425359e640 1 -- 172.21.15.5:0/3917959727 >> 172.21.15.5:0/3917959727 conn(0x7f42540d49c0 msgr2=0x7f42540d4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:21.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.358+0000 7f425359e640 1 -- 172.21.15.5:0/3917959727 shutdown_connections 2024-05-20T18:35:21.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.358+0000 7f425359e640 1 -- 172.21.15.5:0/3917959727 wait complete. 2024-05-20T18:35:21.374 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-20T18:35:21.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.498+0000 7f020d2f9640 1 Processor -- start 2024-05-20T18:35:21.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.498+0000 7f020d2f9640 1 -- start start 2024-05-20T18:35:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.500+0000 7f020d2f9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02080b1df0 0x7f02080d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.500+0000 7f020d2f9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f02080d14a0 0x7f02080b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.500+0000 7f020d2f9640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f02080b3280 0x7f02080b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.500+0000 7f020d2f9640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f020809bc20 con 0x7f02080d14a0 2024-05-20T18:35:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.500+0000 7f020d2f9640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f020809c350 con 0x7f02080b3280 2024-05-20T18:35:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.500+0000 7f020d2f9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f02080cedb0 con 0x7f02080b1df0 2024-05-20T18:35:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.500+0000 7f02067fc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f02080d14a0 0x7f02080b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.500+0000 7f02067fc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f02080d14a0 0x7f02080b2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55754/0 (socket says 172.21.15.5:55754) 2024-05-20T18:35:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.500+0000 7f02067fc640 1 -- 172.21.15.5:0/3164889849 learned_addr learned my addr 172.21.15.5:0/3164889849 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.500+0000 7f02067fc640 1 -- 172.21.15.5:0/3164889849 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02080b1df0 msgr2=0x7f02080d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:21.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.500+0000 7f0206ffd640 1 --2- 172.21.15.5:0/3164889849 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02080b1df0 0x7f02080d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:21.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.500+0000 7f02077fe640 1 --2- 172.21.15.5:0/3164889849 >> v2:172.21.15.63:3300/0 conn(0x7f02080b3280 0x7f02080b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:21.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.500+0000 7f02067fc640 1 --2- 172.21.15.5:0/3164889849 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02080b1df0 0x7f02080d0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:21.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.500+0000 7f02067fc640 1 -- 172.21.15.5:0/3164889849 >> v2:172.21.15.63:3300/0 conn(0x7f02080b3280 msgr2=0x7f02080b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:21.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.500+0000 7f02067fc640 1 --2- 172.21.15.5:0/3164889849 >> v2:172.21.15.63:3300/0 conn(0x7f02080b3280 0x7f02080b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:21.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.500+0000 7f02067fc640 1 -- 172.21.15.5:0/3164889849 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02080d1a50 con 0x7f02080d14a0 2024-05-20T18:35:21.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.500+0000 7f0206ffd640 1 --2- 172.21.15.5:0/3164889849 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02080b1df0 0x7f02080d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:21.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.500+0000 7f02077fe640 1 --2- 172.21.15.5:0/3164889849 >> v2:172.21.15.63:3300/0 conn(0x7f02080b3280 0x7f02080b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:21.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.501+0000 7f02067fc640 1 --2- 172.21.15.5:0/3164889849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02080d14a0 0x7f02080b2d40 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f020808bb30 tx=0x7f01f002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acce367cf9807594 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:21.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.501+0000 7f0205ffb640 1 -- 172.21.15.5:0/3164889849 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f0037070 con 0x7f02080d14a0 2024-05-20T18:35:21.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.501+0000 7f0205ffb640 1 -- 172.21.15.5:0/3164889849 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01f0032440 con 0x7f02080d14a0 2024-05-20T18:35:21.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.501+0000 7f0205ffb640 1 -- 172.21.15.5:0/3164889849 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f0032810 con 0x7f02080d14a0 2024-05-20T18:35:21.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.502+0000 7f020d2f9640 1 -- 172.21.15.5:0/3164889849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02080d14a0 msgr2=0x7f02080b2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:21.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.502+0000 7f020d2f9640 1 --2- 172.21.15.5:0/3164889849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02080d14a0 0x7f02080b2d40 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f020808bb30 tx=0x7f01f002a140 comp rx=0 tx=0).stop 2024-05-20T18:35:21.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.502+0000 7f020d2f9640 1 -- 172.21.15.5:0/3164889849 shutdown_connections 2024-05-20T18:35:21.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.502+0000 7f020d2f9640 1 --2- 172.21.15.5:0/3164889849 >> v2:172.21.15.63:3300/0 conn(0x7f02080b3280 0x7f02080b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:21.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.502+0000 7f020d2f9640 1 --2- 172.21.15.5:0/3164889849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02080d14a0 0x7f02080b2d40 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:21.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.502+0000 7f020d2f9640 1 --2- 172.21.15.5:0/3164889849 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02080b1df0 0x7f02080d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:21.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.502+0000 7f020d2f9640 1 -- 172.21.15.5:0/3164889849 >> 172.21.15.5:0/3164889849 conn(0x7f02080cf450 msgr2=0x7f02080cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:21.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.502+0000 7f020d2f9640 1 -- 172.21.15.5:0/3164889849 shutdown_connections 2024-05-20T18:35:21.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.502+0000 7f020d2f9640 1 -- 172.21.15.5:0/3164889849 wait complete. 2024-05-20T18:35:21.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.504+0000 7f020d2f9640 1 Processor -- start 2024-05-20T18:35:21.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.504+0000 7f020d2f9640 1 -- start start 2024-05-20T18:35:21.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.505+0000 7f020d2f9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02080b1df0 0x7f02081d9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:21.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.505+0000 7f020d2f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02080b3280 0x7f02081da680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.505+0000 7f020d2f9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02080d14a0 0x7f020806df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.505+0000 7f020d2f9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f02080d07d0 con 0x7f02080b3280 2024-05-20T18:35:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.505+0000 7f020d2f9640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f02080d0650 con 0x7f02080b1df0 2024-05-20T18:35:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.505+0000 7f020d2f9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f02080d0180 con 0x7f02080d14a0 2024-05-20T18:35:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.505+0000 7f02067fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02080b3280 0x7f02081da680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.506+0000 7f02067fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02080b3280 0x7f02081da680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55766/0 (socket says 172.21.15.5:55766) 2024-05-20T18:35:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.506+0000 7f02067fc640 1 -- 172.21.15.5:0/1142038518 learned_addr learned my addr 172.21.15.5:0/1142038518 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.506+0000 7f0206ffd640 1 --2- 172.21.15.5:0/1142038518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02080b1df0 0x7f02081d9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.506+0000 7f02077fe640 1 --2- 172.21.15.5:0/1142038518 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02080d14a0 0x7f020806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.506+0000 7f02067fc640 1 -- 172.21.15.5:0/1142038518 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02080d14a0 msgr2=0x7f020806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.506+0000 7f02067fc640 1 --2- 172.21.15.5:0/1142038518 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02080d14a0 0x7f020806df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.506+0000 7f02067fc640 1 -- 172.21.15.5:0/1142038518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02080b1df0 msgr2=0x7f02081d9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.506+0000 7f02067fc640 1 --2- 172.21.15.5:0/1142038518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02080b1df0 0x7f02081d9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.506+0000 7f02067fc640 1 -- 172.21.15.5:0/1142038518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02080a05e0 con 0x7f02080b3280 2024-05-20T18:35:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.506+0000 7f02067fc640 1 --2- 172.21.15.5:0/1142038518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02080b3280 0x7f02081da680 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f01f002a180 tx=0x7f01f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.506+0000 7f01ff7fe640 1 -- 172.21.15.5:0/1142038518 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f0037040 con 0x7f02080b3280 2024-05-20T18:35:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.506+0000 7f01ff7fe640 1 -- 172.21.15.5:0/1142038518 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01f0042070 con 0x7f02080b3280 2024-05-20T18:35:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.506+0000 7f01ff7fe640 1 -- 172.21.15.5:0/1142038518 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f0030bb0 con 0x7f02080b3280 2024-05-20T18:35:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.506+0000 7f020d2f9640 1 -- 172.21.15.5:0/1142038518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f020806cef0 con 0x7f02080b3280 2024-05-20T18:35:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.506+0000 7f020d2f9640 1 -- 172.21.15.5:0/1142038518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f020806d4b0 con 0x7f02080b3280 2024-05-20T18:35:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.508+0000 7f020d2f9640 1 -- 172.21.15.5:0/1142038518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02080690b0 con 0x7f02080b3280 2024-05-20T18:35:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.508+0000 7f01ff7fe640 1 -- 172.21.15.5:0/1142038518 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f01f0047460 con 0x7f02080b3280 2024-05-20T18:35:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.509+0000 7f01ff7fe640 1 --2- 172.21.15.5:0/1142038518 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f01e806be60 0x7f01e806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.509+0000 7f01ff7fe640 1 -- 172.21.15.5:0/1142038518 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6037+0+0 (secure 0 0 0) 0x7f01f00abc60 con 0x7f02080b3280 2024-05-20T18:35:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.509+0000 7f0206ffd640 1 --2- 172.21.15.5:0/1142038518 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f01e806be60 0x7f01e806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.509+0000 7f0206ffd640 1 --2- 172.21.15.5:0/1142038518 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f01e806be60 0x7f01e806e370 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f01f8007130 tx=0x7f01f8007970 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:21.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.513+0000 7f01ff7fe640 1 -- 172.21.15.5:0/1142038518 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f02080a05e0 con 0x7f02080b3280 2024-05-20T18:35:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:21.705+0000 7f020d2f9640 1 -- 172.21.15.5:0/1142038518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f02080c5f50 con 0x7f02080b3280 2024-05-20T18:35:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.355+0000 7f01ff7fe640 1 -- 172.21.15.5:0/1142038518 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v97) ==== 169+0+0 (secure 0 0 0) 0x7f01f007a430 con 0x7f02080b3280 2024-05-20T18:35:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-05-20T18:35:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.359+0000 7f020d2f9640 1 -- 172.21.15.5:0/1142038518 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f01e806be60 msgr2=0x7f01e806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.359+0000 7f020d2f9640 1 --2- 172.21.15.5:0/1142038518 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f01e806be60 0x7f01e806e370 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f01f8007130 tx=0x7f01f8007970 comp rx=0 tx=0).stop 2024-05-20T18:35:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.359+0000 7f020d2f9640 1 -- 172.21.15.5:0/1142038518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02080b3280 msgr2=0x7f02081da680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.359+0000 7f020d2f9640 1 --2- 172.21.15.5:0/1142038518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02080b3280 0x7f02081da680 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f01f002a180 tx=0x7f01f00046c0 comp rx=0 tx=0).stop 2024-05-20T18:35:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.360+0000 7f020d2f9640 1 -- 172.21.15.5:0/1142038518 shutdown_connections 2024-05-20T18:35:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.360+0000 7f020d2f9640 1 --2- 172.21.15.5:0/1142038518 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f01e806be60 0x7f01e806e370 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.360+0000 7f020d2f9640 1 --2- 172.21.15.5:0/1142038518 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02080d14a0 0x7f020806df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.360+0000 7f020d2f9640 1 --2- 172.21.15.5:0/1142038518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02080b3280 0x7f02081da680 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.360+0000 7f020d2f9640 1 --2- 172.21.15.5:0/1142038518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02080b1df0 0x7f02081d9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.360+0000 7f020d2f9640 1 -- 172.21.15.5:0/1142038518 >> 172.21.15.5:0/1142038518 conn(0x7f02080cf450 msgr2=0x7f02080b1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.360+0000 7f020d2f9640 1 -- 172.21.15.5:0/1142038518 shutdown_connections 2024-05-20T18:35:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.360+0000 7f020d2f9640 1 -- 172.21.15.5:0/1142038518 wait complete. 2024-05-20T18:35:22.377 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-20T18:35:22.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.497+0000 7f3af932f640 1 Processor -- start 2024-05-20T18:35:22.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.498+0000 7f3af932f640 1 -- start start 2024-05-20T18:35:22.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.499+0000 7f3af932f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3af40b6620 0x7f3af40ba3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:22.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.499+0000 7f3af932f640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3af40d3bf0 0x7f3af40d4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:22.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.499+0000 7f3af932f640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3af40d4540 con 0x7f3af4162ce0 2024-05-20T18:35:22.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.499+0000 7f3af932f640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f3af40d3480 con 0x7f3af40d3bf0 2024-05-20T18:35:22.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.499+0000 7f3af932f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3af41634a0 con 0x7f3af40b6620 2024-05-20T18:35:22.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.499+0000 7f3af37fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3af40d3bf0 0x7f3af40d4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:22.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.499+0000 7f3af2ffd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3af40b6620 0x7f3af40ba3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:22.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.499+0000 7f3af37fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3af40d3bf0 0x7f3af40d4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:44622/0 (socket says 172.21.15.5:44622) 2024-05-20T18:35:22.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.499+0000 7f3af37fe640 1 -- 172.21.15.5:0/1516867820 learned_addr learned my addr 172.21.15.5:0/1516867820 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:22.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.499+0000 7f3af2ffd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3af40b6620 0x7f3af40ba3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:34540/0 (socket says 172.21.15.5:34540) 2024-05-20T18:35:22.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.500+0000 7f3af2ffd640 1 -- 172.21.15.5:0/1516867820 >> v2:172.21.15.63:3300/0 conn(0x7f3af40d3bf0 msgr2=0x7f3af40d4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:22.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.500+0000 7f3af2ffd640 1 --2- 172.21.15.5:0/1516867820 >> v2:172.21.15.63:3300/0 conn(0x7f3af40d3bf0 0x7f3af40d4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:22.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.500+0000 7f3af2ffd640 1 -- 172.21.15.5:0/1516867820 >> v1:172.21.15.63:6789/0 conn(0x7f3af4162ce0 legacy=0x7f3af40b8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:22.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.500+0000 7f3af2ffd640 1 -- 172.21.15.5:0/1516867820 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af4163780 con 0x7f3af40b6620 2024-05-20T18:35:22.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.500+0000 7f3af37fe640 1 --2- 172.21.15.5:0/1516867820 >> v2:172.21.15.63:3300/0 conn(0x7f3af40d3bf0 0x7f3af40d4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:35:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.501+0000 7f3af2ffd640 1 --2- 172.21.15.5:0/1516867820 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3af40b6620 0x7f3af40ba3c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40097a0 tx=0x7f3ae402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=395c01c62981d504 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.501+0000 7f3af1ffb640 1 -- 172.21.15.5:0/1516867820 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4037070 con 0x7f3af40b6620 2024-05-20T18:35:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.501+0000 7f3af1ffb640 1 -- 172.21.15.5:0/1516867820 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae4032440 con 0x7f3af40b6620 2024-05-20T18:35:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.501+0000 7f3af1ffb640 1 -- 172.21.15.5:0/1516867820 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4032810 con 0x7f3af40b6620 2024-05-20T18:35:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.501+0000 7f3af932f640 1 -- 172.21.15.5:0/1516867820 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3af40b6620 msgr2=0x7f3af40ba3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.501+0000 7f3af932f640 1 --2- 172.21.15.5:0/1516867820 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3af40b6620 0x7f3af40ba3c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40097a0 tx=0x7f3ae402a180 comp rx=0 tx=0).stop 2024-05-20T18:35:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.501+0000 7f3af932f640 1 -- 172.21.15.5:0/1516867820 shutdown_connections 2024-05-20T18:35:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.501+0000 7f3af932f640 1 --2- 172.21.15.5:0/1516867820 >> v2:172.21.15.63:3300/0 conn(0x7f3af40d3bf0 0x7f3af40d4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.501+0000 7f3af932f640 1 --2- 172.21.15.5:0/1516867820 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3af40b6620 0x7f3af40ba3c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.501+0000 7f3af932f640 1 -- 172.21.15.5:0/1516867820 >> 172.21.15.5:0/1516867820 conn(0x7f3af406d240 msgr2=0x7f3af406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.502+0000 7f3af932f640 1 -- 172.21.15.5:0/1516867820 shutdown_connections 2024-05-20T18:35:22.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.502+0000 7f3af932f640 1 -- 172.21.15.5:0/1516867820 wait complete. 2024-05-20T18:35:22.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.503+0000 7f3af932f640 1 Processor -- start 2024-05-20T18:35:22.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.504+0000 7f3af932f640 1 -- start start 2024-05-20T18:35:22.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.505+0000 7f3af932f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3af40b6620 0x7f3af4158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:22.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.505+0000 7f3af932f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3af40d3bf0 0x7f3af415e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:22.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.505+0000 7f3af932f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3af4162ce0 0x7f3af415a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:22.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.505+0000 7f3af932f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3af41630f0 con 0x7f3af40d3bf0 2024-05-20T18:35:22.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.505+0000 7f3af932f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3af40d36d0 con 0x7f3af40b6620 2024-05-20T18:35:22.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.505+0000 7f3af932f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3af41f6590 con 0x7f3af4162ce0 2024-05-20T18:35:22.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.505+0000 7f3af27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3af40d3bf0 0x7f3af415e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:22.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.505+0000 7f3af27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3af40d3bf0 0x7f3af415e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55768/0 (socket says 172.21.15.5:55768) 2024-05-20T18:35:22.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.505+0000 7f3af2ffd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3af40b6620 0x7f3af4158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:22.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.505+0000 7f3af27fc640 1 -- 172.21.15.5:0/697386162 learned_addr learned my addr 172.21.15.5:0/697386162 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:22.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.505+0000 7f3af37fe640 1 --2- 172.21.15.5:0/697386162 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3af4162ce0 0x7f3af415a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:22.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.505+0000 7f3af27fc640 1 -- 172.21.15.5:0/697386162 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3af4162ce0 msgr2=0x7f3af415a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:22.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.505+0000 7f3af27fc640 1 --2- 172.21.15.5:0/697386162 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3af4162ce0 0x7f3af415a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:22.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.505+0000 7f3af27fc640 1 -- 172.21.15.5:0/697386162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3af40b6620 msgr2=0x7f3af4158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:22.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.505+0000 7f3af27fc640 1 --2- 172.21.15.5:0/697386162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3af40b6620 0x7f3af4158f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:22.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.505+0000 7f3af27fc640 1 -- 172.21.15.5:0/697386162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af415d190 con 0x7f3af40d3bf0 2024-05-20T18:35:22.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.505+0000 7f3af2ffd640 1 --2- 172.21.15.5:0/697386162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3af40b6620 0x7f3af4158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:35:22.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.506+0000 7f3af27fc640 1 --2- 172.21.15.5:0/697386162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3af40d3bf0 0x7f3af415e230 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800c3f0 tx=0x7f3ae800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:22.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.506+0000 7f3adb7fe640 1 -- 172.21.15.5:0/697386162 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae80052d0 con 0x7f3af40d3bf0 2024-05-20T18:35:22.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.506+0000 7f3adb7fe640 1 -- 172.21.15.5:0/697386162 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae8005490 con 0x7f3af40d3bf0 2024-05-20T18:35:22.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.506+0000 7f3af932f640 1 -- 172.21.15.5:0/697386162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3af415d470 con 0x7f3af40d3bf0 2024-05-20T18:35:22.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.506+0000 7f3adb7fe640 1 -- 172.21.15.5:0/697386162 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8007260 con 0x7f3af40d3bf0 2024-05-20T18:35:22.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.506+0000 7f3af932f640 1 -- 172.21.15.5:0/697386162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3af415da30 con 0x7f3af40d3bf0 2024-05-20T18:35:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.507+0000 7f3af932f640 1 -- 172.21.15.5:0/697386162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3af4075b00 con 0x7f3af40d3bf0 2024-05-20T18:35:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.508+0000 7f3adb7fe640 1 -- 172.21.15.5:0/697386162 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3ae8007400 con 0x7f3af40d3bf0 2024-05-20T18:35:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.508+0000 7f3adb7fe640 1 --2- 172.21.15.5:0/697386162 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3ae006be30 0x7f3ae006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.508+0000 7f3adb7fe640 1 -- 172.21.15.5:0/697386162 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5507+0+0 (secure 0 0 0) 0x7f3ae808a250 con 0x7f3af40d3bf0 2024-05-20T18:35:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.508+0000 7f3af2ffd640 1 --2- 172.21.15.5:0/697386162 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3ae006be30 0x7f3ae006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.509+0000 7f3af2ffd640 1 --2- 172.21.15.5:0/697386162 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3ae006be30 0x7f3ae006e340 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4034000 tx=0x7f3ae402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:22.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.513+0000 7f3adb7fe640 1 -- 172.21.15.5:0/697386162 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3af40d4540 con 0x7f3af40d3bf0 2024-05-20T18:35:22.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.705+0000 7f3af932f640 1 -- 172.21.15.5:0/697386162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f3af40d2fa0 con 0x7f3af40d3bf0 2024-05-20T18:35:22.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.708+0000 7f3adb7fe640 1 -- 172.21.15.5:0/697386162 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f3ae8058560 con 0x7f3af40d3bf0 2024-05-20T18:35:22.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.710+0000 7f3af932f640 1 -- 172.21.15.5:0/697386162 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3ae006be30 msgr2=0x7f3ae006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:22.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.710+0000 7f3af932f640 1 --2- 172.21.15.5:0/697386162 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3ae006be30 0x7f3ae006e340 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4034000 tx=0x7f3ae402e000 comp rx=0 tx=0).stop 2024-05-20T18:35:22.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.710+0000 7f3af932f640 1 -- 172.21.15.5:0/697386162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3af40d3bf0 msgr2=0x7f3af415e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:22.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.710+0000 7f3af932f640 1 --2- 172.21.15.5:0/697386162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3af40d3bf0 0x7f3af415e230 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800c3f0 tx=0x7f3ae800c7f0 comp rx=0 tx=0).stop 2024-05-20T18:35:22.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.710+0000 7f3af932f640 1 -- 172.21.15.5:0/697386162 shutdown_connections 2024-05-20T18:35:22.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.710+0000 7f3af932f640 1 --2- 172.21.15.5:0/697386162 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3ae006be30 0x7f3ae006e340 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:22.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.710+0000 7f3af932f640 1 --2- 172.21.15.5:0/697386162 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3af4162ce0 0x7f3af415a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:22.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.710+0000 7f3af932f640 1 --2- 172.21.15.5:0/697386162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3af40d3bf0 0x7f3af415e230 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:22.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.710+0000 7f3af932f640 1 --2- 172.21.15.5:0/697386162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3af40b6620 0x7f3af4158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:22.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.710+0000 7f3af932f640 1 -- 172.21.15.5:0/697386162 >> 172.21.15.5:0/697386162 conn(0x7f3af406d240 msgr2=0x7f3af40cf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:22.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.711+0000 7f3af932f640 1 -- 172.21.15.5:0/697386162 shutdown_connections 2024-05-20T18:35:22.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.711+0000 7f3af932f640 1 -- 172.21.15.5:0/697386162 wait complete. 2024-05-20T18:35:22.723 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-05-20T18:35:22.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.838+0000 7f36b6cf7640 1 Processor -- start 2024-05-20T18:35:22.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.838+0000 7f36b6cf7640 1 -- start start 2024-05-20T18:35:22.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.839+0000 7f36b6cf7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f36b015d530 0x7f36b015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:22.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.839+0000 7f36b6cf7640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f36b00b1700 0x7f36b00b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:22.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.839+0000 7f36b6cf7640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36b00cd580 con 0x7f36b0070850 2024-05-20T18:35:22.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.839+0000 7f36b6cf7640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f36b00ce190 con 0x7f36b00b1700 2024-05-20T18:35:22.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.839+0000 7f36b6cf7640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f36b00cd860 con 0x7f36b015d530 2024-05-20T18:35:22.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.839+0000 7f36affff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f36b00b1700 0x7f36b00b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:22.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.840+0000 7f36affff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f36b00b1700 0x7f36b00b1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55782/0 (socket says 172.21.15.5:55782) 2024-05-20T18:35:22.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.840+0000 7f36affff640 1 -- 172.21.15.5:0/1317249588 learned_addr learned my addr 172.21.15.5:0/1317249588 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:22.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.840+0000 7f36affff640 1 -- 172.21.15.5:0/1317249588 >> v2:172.21.15.63:3300/0 conn(0x7f36b015d530 msgr2=0x7f36b015aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:22.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.840+0000 7f36b4a93640 1 --2- 172.21.15.5:0/1317249588 >> v2:172.21.15.63:3300/0 conn(0x7f36b015d530 0x7f36b015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:22.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.840+0000 7f36affff640 1 --2- 172.21.15.5:0/1317249588 >> v2:172.21.15.63:3300/0 conn(0x7f36b015d530 0x7f36b015aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:22.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.840+0000 7f36affff640 1 -- 172.21.15.5:0/1317249588 >> v1:172.21.15.63:6789/0 conn(0x7f36b0070850 legacy=0x7f36b0070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:22.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.840+0000 7f36affff640 1 -- 172.21.15.5:0/1317249588 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b00cdb40 con 0x7f36b00b1700 2024-05-20T18:35:22.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.841+0000 7f36affff640 1 --2- 172.21.15.5:0/1317249588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36b00b1700 0x7f36b00b1b10 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f36b004f790 tx=0x7f36a002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4653c260b0aee021 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:22.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.841+0000 7f36af7fe640 1 -- 172.21.15.5:0/1317249588 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0037070 con 0x7f36b00b1700 2024-05-20T18:35:22.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.841+0000 7f36af7fe640 1 -- 172.21.15.5:0/1317249588 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36a0032440 con 0x7f36b00b1700 2024-05-20T18:35:22.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.841+0000 7f36af7fe640 1 -- 172.21.15.5:0/1317249588 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0032810 con 0x7f36b00b1700 2024-05-20T18:35:22.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.841+0000 7f36b6cf7640 1 -- 172.21.15.5:0/1317249588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36b00b1700 msgr2=0x7f36b00b1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:22.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.841+0000 7f36b6cf7640 1 --2- 172.21.15.5:0/1317249588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36b00b1700 0x7f36b00b1b10 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f36b004f790 tx=0x7f36a002a230 comp rx=0 tx=0).stop 2024-05-20T18:35:22.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.842+0000 7f36b6cf7640 1 -- 172.21.15.5:0/1317249588 shutdown_connections 2024-05-20T18:35:22.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.842+0000 7f36b6cf7640 1 --2- 172.21.15.5:0/1317249588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36b00b1700 0x7f36b00b1b10 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:22.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.842+0000 7f36b6cf7640 1 --2- 172.21.15.5:0/1317249588 >> v2:172.21.15.63:3300/0 conn(0x7f36b015d530 0x7f36b015aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:22.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.842+0000 7f36b6cf7640 1 -- 172.21.15.5:0/1317249588 >> 172.21.15.5:0/1317249588 conn(0x7f36b00d29d0 msgr2=0x7f36b00d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:22.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.842+0000 7f36b6cf7640 1 -- 172.21.15.5:0/1317249588 shutdown_connections 2024-05-20T18:35:22.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.842+0000 7f36b6cf7640 1 -- 172.21.15.5:0/1317249588 wait complete. 2024-05-20T18:35:22.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.843+0000 7f36b6cf7640 1 Processor -- start 2024-05-20T18:35:22.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.843+0000 7f36b6cf7640 1 -- start start 2024-05-20T18:35:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.845+0000 7f36b6cf7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36b0070850 0x7f36b014aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.845+0000 7f36b6cf7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36b00b1700 0x7f36b00c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.845+0000 7f36b6cf7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36b015d530 0x7f36b01476b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.845+0000 7f36b6cf7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f36b00b2050 con 0x7f36b015d530 2024-05-20T18:35:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.845+0000 7f36b6cf7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f36b00cd230 con 0x7f36b00b1700 2024-05-20T18:35:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.845+0000 7f36b6cf7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f36b0071370 con 0x7f36b0070850 2024-05-20T18:35:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.845+0000 7f36b5294640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36b015d530 0x7f36b01476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.845+0000 7f36b5294640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36b015d530 0x7f36b01476b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55786/0 (socket says 172.21.15.5:55786) 2024-05-20T18:35:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.845+0000 7f36b5294640 1 -- 172.21.15.5:0/2185998642 learned_addr learned my addr 172.21.15.5:0/2185998642 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.845+0000 7f36affff640 1 --2- 172.21.15.5:0/2185998642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36b00b1700 0x7f36b00c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.845+0000 7f36b4a93640 1 --2- 172.21.15.5:0/2185998642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36b0070850 0x7f36b014aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.845+0000 7f36b5294640 1 -- 172.21.15.5:0/2185998642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36b0070850 msgr2=0x7f36b014aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.845+0000 7f36b5294640 1 --2- 172.21.15.5:0/2185998642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36b0070850 0x7f36b014aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.845+0000 7f36b5294640 1 -- 172.21.15.5:0/2185998642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36b00b1700 msgr2=0x7f36b00c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.845+0000 7f36b5294640 1 --2- 172.21.15.5:0/2185998642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36b00b1700 0x7f36b00c6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.845+0000 7f36b5294640 1 -- 172.21.15.5:0/2185998642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b014b000 con 0x7f36b015d530 2024-05-20T18:35:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.846+0000 7f36b5294640 1 --2- 172.21.15.5:0/2185998642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36b015d530 0x7f36b01476b0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f369800b410 tx=0x7f369800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.846+0000 7f36ad7fa640 1 -- 172.21.15.5:0/2185998642 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698015020 con 0x7f36b015d530 2024-05-20T18:35:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.846+0000 7f36ad7fa640 1 -- 172.21.15.5:0/2185998642 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3698011830 con 0x7f36b015d530 2024-05-20T18:35:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.846+0000 7f36b6cf7640 1 -- 172.21.15.5:0/2185998642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36b00c6a40 con 0x7f36b015d530 2024-05-20T18:35:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.846+0000 7f36ad7fa640 1 -- 172.21.15.5:0/2185998642 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698011c20 con 0x7f36b015d530 2024-05-20T18:35:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.846+0000 7f36b6cf7640 1 -- 172.21.15.5:0/2185998642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36b01488c0 con 0x7f36b015d530 2024-05-20T18:35:22.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.847+0000 7f36b6cf7640 1 -- 172.21.15.5:0/2185998642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36b0075b00 con 0x7f36b015d530 2024-05-20T18:35:22.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.847+0000 7f36ad7fa640 1 -- 172.21.15.5:0/2185998642 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3698018020 con 0x7f36b015d530 2024-05-20T18:35:22.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.848+0000 7f36ad7fa640 1 --2- 172.21.15.5:0/2185998642 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f369006be30 0x7f369006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:22.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.848+0000 7f36b4a93640 1 --2- 172.21.15.5:0/2185998642 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f369006be30 0x7f369006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:22.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.848+0000 7f36ad7fa640 1 -- 172.21.15.5:0/2185998642 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5507+0+0 (secure 0 0 0) 0x7f369808aaf0 con 0x7f36b015d530 2024-05-20T18:35:22.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.849+0000 7f36b4a93640 1 --2- 172.21.15.5:0/2185998642 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f369006be30 0x7f369006e340 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f36b00d29a0 tx=0x7f36a8007730 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:22.853+0000 7f36ad7fa640 1 -- 172.21.15.5:0/2185998642 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f369801f020 con 0x7f36b015d530 2024-05-20T18:35:23.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.106+0000 7f36b6cf7640 1 -- 172.21.15.5:0/2185998642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7f36b00d0bd0 con 0x7f36b015d530 2024-05-20T18:35:23.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.360+0000 7f36ad7fa640 1 -- 172.21.15.5:0/2185998642 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v398) ==== 167+0+0 (secure 0 0 0) 0x7f36b014b000 con 0x7f36b015d530 2024-05-20T18:35:23.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.362+0000 7f36b6cf7640 1 -- 172.21.15.5:0/2185998642 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f369006be30 msgr2=0x7f369006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:23.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.362+0000 7f36b6cf7640 1 --2- 172.21.15.5:0/2185998642 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f369006be30 0x7f369006e340 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f36b00d29a0 tx=0x7f36a8007730 comp rx=0 tx=0).stop 2024-05-20T18:35:23.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.362+0000 7f36b6cf7640 1 -- 172.21.15.5:0/2185998642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36b015d530 msgr2=0x7f36b01476b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:23.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.362+0000 7f36b6cf7640 1 --2- 172.21.15.5:0/2185998642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36b015d530 0x7f36b01476b0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f369800b410 tx=0x7f369800b8d0 comp rx=0 tx=0).stop 2024-05-20T18:35:23.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.362+0000 7f36b6cf7640 1 -- 172.21.15.5:0/2185998642 shutdown_connections 2024-05-20T18:35:23.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.362+0000 7f36b6cf7640 1 --2- 172.21.15.5:0/2185998642 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f369006be30 0x7f369006e340 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:23.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.362+0000 7f36b6cf7640 1 --2- 172.21.15.5:0/2185998642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36b015d530 0x7f36b01476b0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:23.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.362+0000 7f36b6cf7640 1 --2- 172.21.15.5:0/2185998642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36b00b1700 0x7f36b00c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:23.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.362+0000 7f36b6cf7640 1 --2- 172.21.15.5:0/2185998642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36b0070850 0x7f36b014aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:23.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.362+0000 7f36b6cf7640 1 -- 172.21.15.5:0/2185998642 >> 172.21.15.5:0/2185998642 conn(0x7f36b00d29d0 msgr2=0x7f36b015f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:23.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.362+0000 7f36b6cf7640 1 -- 172.21.15.5:0/2185998642 shutdown_connections 2024-05-20T18:35:23.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.362+0000 7f36b6cf7640 1 -- 172.21.15.5:0/2185998642 wait complete. 2024-05-20T18:35:23.375 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-05-20T18:35:23.375 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-20T18:35:23.375 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:35:23.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.490+0000 7f14af16b640 1 Processor -- start 2024-05-20T18:35:23.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.490+0000 7f14af16b640 1 -- start start 2024-05-20T18:35:23.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.491+0000 7f14af16b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f14a80d10e0 0x7f14a80cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:23.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.491+0000 7f14af16b640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14a80ce2f0 con 0x7f14a80d14f0 2024-05-20T18:35:23.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.491+0000 7f14af16b640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14a80d1ac0 con 0x7f14a80b7210 2024-05-20T18:35:23.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.491+0000 7f14af16b640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f14a80d1da0 con 0x7f14a80d10e0 2024-05-20T18:35:23.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.491+0000 7f14a7fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f14a80d14f0 0x7f14a80cd4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47362/0 (socket says 172.21.15.5:47362) 2024-05-20T18:35:23.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.491+0000 7f14a7fff640 1 -- 172.21.15.5:0/473091982 learned_addr learned my addr 172.21.15.5:0/473091982 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:23.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.492+0000 7f14a77fe640 1 -- 172.21.15.5:0/473091982 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1982470908 0 0) 0x7f14a80ce2f0 con 0x7f14a80d14f0 2024-05-20T18:35:23.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.492+0000 7f14acf07640 1 --2- 172.21.15.5:0/473091982 >> v2:172.21.15.63:3300/0 conn(0x7f14a80d10e0 0x7f14a80cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:23.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.492+0000 7f14a77fe640 1 -- 172.21.15.5:0/473091982 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1490003520 con 0x7f14a80d14f0 2024-05-20T18:35:23.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.492+0000 7f14a77fe640 1 -- 172.21.15.5:0/473091982 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 547888015 0 0) 0x7f1490003520 con 0x7f14a80d14f0 2024-05-20T18:35:23.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.492+0000 7f14a77fe640 1 -- 172.21.15.5:0/473091982 >> v2:172.21.15.63:3300/0 conn(0x7f14a80d10e0 msgr2=0x7f14a80cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:23.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.492+0000 7f14a77fe640 1 --2- 172.21.15.5:0/473091982 >> v2:172.21.15.63:3300/0 conn(0x7f14a80d10e0 0x7f14a80cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:23.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.492+0000 7f14a77fe640 1 -- 172.21.15.5:0/473091982 >> v1:172.21.15.63:6789/0 conn(0x7f14a80b7210 legacy=0x7f14a80cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:23.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.492+0000 7f14a77fe640 1 -- 172.21.15.5:0/473091982 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a80d2080 con 0x7f14a80d14f0 2024-05-20T18:35:23.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.492+0000 7f14a77fe640 1 -- 172.21.15.5:0/473091982 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f149c001670 con 0x7f14a80d14f0 2024-05-20T18:35:23.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.492+0000 7f14a77fe640 1 -- 172.21.15.5:0/473091982 >> v1:172.21.15.5:6789/0 conn(0x7f14a80d14f0 legacy=0x7f14a80cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:23.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.492+0000 7f14a77fe640 1 --2- 172.21.15.5:0/473091982 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14900039b0 0x7f1490003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:23.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.492+0000 7f14a77fe640 1 --2- 172.21.15.5:0/473091982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1490004800 0x7f1490004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:23.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.492+0000 7f14a77fe640 1 --2- 172.21.15.5:0/473091982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14900051c0 0x7f149000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:23.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.492+0000 7f14a77fe640 1 -- 172.21.15.5:0/473091982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f14a80d1da0 con 0x7f14900051c0 2024-05-20T18:35:23.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.492+0000 7f14a77fe640 1 -- 172.21.15.5:0/473091982 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f149000c160 con 0x7f1490004800 2024-05-20T18:35:23.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.492+0000 7f14a77fe640 1 -- 172.21.15.5:0/473091982 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f149000c360 con 0x7f14900039b0 2024-05-20T18:35:23.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.492+0000 7f14ad708640 1 --2- 172.21.15.5:0/473091982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14900051c0 0x7f149000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:23.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.492+0000 7f14a7fff640 1 --2- 172.21.15.5:0/473091982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1490004800 0x7f1490004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:23.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.492+0000 7f14acf07640 1 --2- 172.21.15.5:0/473091982 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14900039b0 0x7f1490003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:23.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.492+0000 7f14ad708640 1 -- 172.21.15.5:0/473091982 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14900039b0 msgr2=0x7f1490003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:23.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.492+0000 7f14ad708640 1 --2- 172.21.15.5:0/473091982 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14900039b0 0x7f1490003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:23.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.492+0000 7f14ad708640 1 -- 172.21.15.5:0/473091982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1490004800 msgr2=0x7f1490004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:23.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.492+0000 7f14ad708640 1 --2- 172.21.15.5:0/473091982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1490004800 0x7f1490004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:23.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.492+0000 7f14ad708640 1 -- 172.21.15.5:0/473091982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1490003520 con 0x7f14900051c0 2024-05-20T18:35:23.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.492+0000 7f14acf07640 1 --2- 172.21.15.5:0/473091982 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14900039b0 0x7f1490003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:23.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.493+0000 7f14ad708640 1 -- 172.21.15.5:0/473091982 reap_dead start 2024-05-20T18:35:23.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.493+0000 7f14ad708640 1 --2- 172.21.15.5:0/473091982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14900051c0 0x7f149000b9c0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f14a0001e20 tx=0x7f1490005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa7b2d236fd3389 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:23.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.493+0000 7f14a77fe640 1 -- 172.21.15.5:0/473091982 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a0001a70 con 0x7f14900051c0 2024-05-20T18:35:23.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.493+0000 7f14a77fe640 1 -- 172.21.15.5:0/473091982 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a002ad70 con 0x7f14900051c0 2024-05-20T18:35:23.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.493+0000 7f14a77fe640 1 -- 172.21.15.5:0/473091982 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a0003bb0 con 0x7f14900051c0 2024-05-20T18:35:23.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.494+0000 7f14af16b640 1 -- 172.21.15.5:0/473091982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14900051c0 msgr2=0x7f149000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:23.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.494+0000 7f14af16b640 1 --2- 172.21.15.5:0/473091982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14900051c0 0x7f149000b9c0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f14a0001e20 tx=0x7f1490005830 comp rx=0 tx=0).stop 2024-05-20T18:35:23.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.494+0000 7f14af16b640 1 -- 172.21.15.5:0/473091982 shutdown_connections 2024-05-20T18:35:23.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.494+0000 7f14af16b640 1 --2- 172.21.15.5:0/473091982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14900051c0 0x7f149000b9c0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:23.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.494+0000 7f14af16b640 1 -- 172.21.15.5:0/473091982 >> 172.21.15.5:0/473091982 conn(0x7f14a80b44d0 msgr2=0x7f14a80b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:23.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.494+0000 7f14af16b640 1 -- 172.21.15.5:0/473091982 shutdown_connections 2024-05-20T18:35:23.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.494+0000 7f14af16b640 1 -- 172.21.15.5:0/473091982 wait complete. 2024-05-20T18:35:23.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.496+0000 7f14af16b640 1 Processor -- start 2024-05-20T18:35:23.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.496+0000 7f14af16b640 1 -- start start 2024-05-20T18:35:23.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.497+0000 7f14af16b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14900051c0 0x7f14a815a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:23.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.497+0000 7f14af16b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14a80cb7b0 0x7f14a80cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:23.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.497+0000 7f14af16b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14a8158740 0x7f14a8158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:23.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.497+0000 7f14af16b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f14a80b82c0 con 0x7f14a80cb7b0 2024-05-20T18:35:23.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.497+0000 7f14af16b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f14a80d2640 con 0x7f14900051c0 2024-05-20T18:35:23.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.497+0000 7f14af16b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f14a80b8440 con 0x7f14a8158740 2024-05-20T18:35:23.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.497+0000 7f14a7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14a80cb7b0 0x7f14a80cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:23.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.497+0000 7f14a7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14a80cb7b0 0x7f14a80cbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55808/0 (socket says 172.21.15.5:55808) 2024-05-20T18:35:23.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.497+0000 7f14a7fff640 1 -- 172.21.15.5:0/4194819252 learned_addr learned my addr 172.21.15.5:0/4194819252 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:23.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.498+0000 7f14acf07640 1 --2- 172.21.15.5:0/4194819252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14900051c0 0x7f14a815a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:23.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.498+0000 7f14ad708640 1 --2- 172.21.15.5:0/4194819252 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14a8158740 0x7f14a8158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:23.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.498+0000 7f14a7fff640 1 -- 172.21.15.5:0/4194819252 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14a8158740 msgr2=0x7f14a8158b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:23.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.498+0000 7f14a7fff640 1 --2- 172.21.15.5:0/4194819252 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14a8158740 0x7f14a8158b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:23.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.498+0000 7f14a7fff640 1 -- 172.21.15.5:0/4194819252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14900051c0 msgr2=0x7f14a815a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:23.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.498+0000 7f14a7fff640 1 --2- 172.21.15.5:0/4194819252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14900051c0 0x7f14a815a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:23.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.498+0000 7f14a7fff640 1 -- 172.21.15.5:0/4194819252 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a8157980 con 0x7f14a80cb7b0 2024-05-20T18:35:23.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.498+0000 7f14a7fff640 1 --2- 172.21.15.5:0/4194819252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14a80cb7b0 0x7f14a80cbbc0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f149c007370 tx=0x7f149c007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:23.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.498+0000 7f14a57fa640 1 -- 172.21.15.5:0/4194819252 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c01a070 con 0x7f14a80cb7b0 2024-05-20T18:35:23.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.498+0000 7f14a57fa640 1 -- 172.21.15.5:0/4194819252 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f149c00f9c0 con 0x7f14a80cb7b0 2024-05-20T18:35:23.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.498+0000 7f14af16b640 1 -- 172.21.15.5:0/4194819252 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14a8157c60 con 0x7f14a80cb7b0 2024-05-20T18:35:23.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.498+0000 7f14a57fa640 1 -- 172.21.15.5:0/4194819252 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c00fd90 con 0x7f14a80cb7b0 2024-05-20T18:35:23.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.498+0000 7f14af16b640 1 -- 172.21.15.5:0/4194819252 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14a8158220 con 0x7f14a80cb7b0 2024-05-20T18:35:23.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.499+0000 7f14af16b640 1 -- 172.21.15.5:0/4194819252 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14a8075b00 con 0x7f14a80cb7b0 2024-05-20T18:35:23.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.500+0000 7f14a57fa640 1 -- 172.21.15.5:0/4194819252 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f149c01d070 con 0x7f14a80cb7b0 2024-05-20T18:35:23.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.500+0000 7f14a57fa640 1 --2- 172.21.15.5:0/4194819252 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f149406be60 0x7f149406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:23.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.500+0000 7f14a57fa640 1 -- 172.21.15.5:0/4194819252 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5507+0+0 (secure 0 0 0) 0x7f149c08a740 con 0x7f14a80cb7b0 2024-05-20T18:35:23.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.500+0000 7f14acf07640 1 --2- 172.21.15.5:0/4194819252 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f149406be60 0x7f149406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:23.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.501+0000 7f14acf07640 1 --2- 172.21.15.5:0/4194819252 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f149406be60 0x7f149406e370 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1498006f80 tx=0x7f1498008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:23.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.505+0000 7f14a57fa640 1 -- 172.21.15.5:0/4194819252 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f149c058ad0 con 0x7f14a80cb7b0 2024-05-20T18:35:23.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.691+0000 7f14af16b640 1 -- 172.21.15.5:0/4194819252 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f14a80cc230 con 0x7f14a80cb7b0 2024-05-20T18:35:23.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.692+0000 7f14a57fa640 1 -- 172.21.15.5:0/4194819252 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+13430 (secure 0 0 0) 0x7f149c05cdd0 con 0x7f14a80cb7b0 2024-05-20T18:35:23.693 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:23.693 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":97,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:35:22.354604+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6837/3880006778":"2024-05-21T18:35:19.346999+0000","172.21.15.5:6836/3880006778":"2024-05-21T18:35:19.346999+0000","172.21.15.5:6837/3556492262":"2024-05-21T18:34:35.299705+0000","172.21.15.5:6836/3556492262":"2024-05-21T18:34:35.299705+0000","172.21.15.63:6834/2759396130":"2024-05-21T18:33:23.233035+0000","172.21.15.63:6832/2759396130":"2024-05-21T18:33:23.233035+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:35:23.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.694+0000 7f14af16b640 1 -- 172.21.15.5:0/4194819252 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f149406be60 msgr2=0x7f149406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:23.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.694+0000 7f14af16b640 1 --2- 172.21.15.5:0/4194819252 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f149406be60 0x7f149406e370 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1498006f80 tx=0x7f1498008040 comp rx=0 tx=0).stop 2024-05-20T18:35:23.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.694+0000 7f14af16b640 1 -- 172.21.15.5:0/4194819252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14a80cb7b0 msgr2=0x7f14a80cbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:23.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.694+0000 7f14af16b640 1 --2- 172.21.15.5:0/4194819252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14a80cb7b0 0x7f14a80cbbc0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f149c007370 tx=0x7f149c007830 comp rx=0 tx=0).stop 2024-05-20T18:35:23.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.694+0000 7f14af16b640 1 -- 172.21.15.5:0/4194819252 shutdown_connections 2024-05-20T18:35:23.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.694+0000 7f14af16b640 1 --2- 172.21.15.5:0/4194819252 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f149406be60 0x7f149406e370 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:23.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.694+0000 7f14af16b640 1 --2- 172.21.15.5:0/4194819252 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14a8158740 0x7f14a8158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:23.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.694+0000 7f14af16b640 1 --2- 172.21.15.5:0/4194819252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14a80cb7b0 0x7f14a80cbbc0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:23.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.694+0000 7f14af16b640 1 --2- 172.21.15.5:0/4194819252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14900051c0 0x7f14a815a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:23.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.694+0000 7f14af16b640 1 -- 172.21.15.5:0/4194819252 >> 172.21.15.5:0/4194819252 conn(0x7f14a80b44d0 msgr2=0x7f14a80b1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:23.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.694+0000 7f14af16b640 1 -- 172.21.15.5:0/4194819252 shutdown_connections 2024-05-20T18:35:23.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.694+0000 7f14af16b640 1 -- 172.21.15.5:0/4194819252 wait complete. 2024-05-20T18:35:23.706 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:35:23.707 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:35:23.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.823+0000 7f1ff397e640 1 Processor -- start 2024-05-20T18:35:23.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.823+0000 7f1ff397e640 1 -- start start 2024-05-20T18:35:23.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.825+0000 7f1ff397e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1fec0b58f0 0x7f1fec15ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:23.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.825+0000 7f1ff397e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1fec0afee0 0x7f1fec15cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:23.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.825+0000 7f1ff397e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fec0b3aa0 con 0x7f1fec0afad0 2024-05-20T18:35:23.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.825+0000 7f1ff397e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f1fec0b3850 con 0x7f1fec0b58f0 2024-05-20T18:35:23.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.825+0000 7f1ff397e640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1fec0703e0 con 0x7f1fec0afee0 2024-05-20T18:35:23.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.825+0000 7f1ff0f19640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1fec0b58f0 0x7f1fec15ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:23.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.825+0000 7f1ff0f19640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1fec0b58f0 0x7f1fec15ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55822/0 (socket says 172.21.15.5:55822) 2024-05-20T18:35:23.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.825+0000 7f1ff0f19640 1 -- 172.21.15.5:0/3140315580 learned_addr learned my addr 172.21.15.5:0/3140315580 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:23.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.825+0000 7f1ff1f1b640 1 --2- 172.21.15.5:0/3140315580 >> v2:172.21.15.63:3300/0 conn(0x7f1fec0afee0 0x7f1fec15cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:23.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.825+0000 7f1ff0f19640 1 -- 172.21.15.5:0/3140315580 >> v2:172.21.15.63:3300/0 conn(0x7f1fec0afee0 msgr2=0x7f1fec15cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:23.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.825+0000 7f1ff0f19640 1 --2- 172.21.15.5:0/3140315580 >> v2:172.21.15.63:3300/0 conn(0x7f1fec0afee0 0x7f1fec15cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:23.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.825+0000 7f1ff0f19640 1 -- 172.21.15.5:0/3140315580 >> v1:172.21.15.5:6789/0 conn(0x7f1fec0afad0 legacy=0x7f1fec0b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:23.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.825+0000 7f1ff0f19640 1 -- 172.21.15.5:0/3140315580 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fec0b3d80 con 0x7f1fec0b58f0 2024-05-20T18:35:23.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.825+0000 7f1febfff640 1 -- 172.21.15.5:0/3140315580 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2683924275 0 0) 0x7f1fec0b3aa0 con 0x7f1fec0afad0 2024-05-20T18:35:23.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.825+0000 7f1ff1f1b640 1 --2- 172.21.15.5:0/3140315580 >> v2:172.21.15.63:3300/0 conn(0x7f1fec0afee0 0x7f1fec15cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:23.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.826+0000 7f1ff0f19640 1 --2- 172.21.15.5:0/3140315580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fec0b58f0 0x7f1fec15ca50 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f1fec06e390 tx=0x7f1fe002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f522ffdfc629af server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:23.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.826+0000 7f1febfff640 1 -- 172.21.15.5:0/3140315580 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0037070 con 0x7f1fec0b58f0 2024-05-20T18:35:23.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.826+0000 7f1febfff640 1 -- 172.21.15.5:0/3140315580 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fe0032440 con 0x7f1fec0b58f0 2024-05-20T18:35:23.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.826+0000 7f1febfff640 1 -- 172.21.15.5:0/3140315580 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0032810 con 0x7f1fec0b58f0 2024-05-20T18:35:23.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.827+0000 7f1ff397e640 1 -- 172.21.15.5:0/3140315580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fec0b58f0 msgr2=0x7f1fec15ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:23.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.827+0000 7f1ff397e640 1 --2- 172.21.15.5:0/3140315580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fec0b58f0 0x7f1fec15ca50 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f1fec06e390 tx=0x7f1fe002a1e0 comp rx=0 tx=0).stop 2024-05-20T18:35:23.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.827+0000 7f1ff397e640 1 -- 172.21.15.5:0/3140315580 shutdown_connections 2024-05-20T18:35:23.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.827+0000 7f1ff397e640 1 --2- 172.21.15.5:0/3140315580 >> v2:172.21.15.63:3300/0 conn(0x7f1fec0afee0 0x7f1fec15cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:23.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.827+0000 7f1ff397e640 1 --2- 172.21.15.5:0/3140315580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fec0b58f0 0x7f1fec15ca50 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:23.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.827+0000 7f1ff397e640 1 -- 172.21.15.5:0/3140315580 >> 172.21.15.5:0/3140315580 conn(0x7f1fec0cd2a0 msgr2=0x7f1fec0cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:23.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.827+0000 7f1ff397e640 1 -- 172.21.15.5:0/3140315580 shutdown_connections 2024-05-20T18:35:23.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.827+0000 7f1ff397e640 1 -- 172.21.15.5:0/3140315580 wait complete. 2024-05-20T18:35:23.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.829+0000 7f1ff397e640 1 Processor -- start 2024-05-20T18:35:23.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.829+0000 7f1ff397e640 1 -- start start 2024-05-20T18:35:23.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.830+0000 7f1ff397e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fec0afad0 0x7f1fec14a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:23.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.830+0000 7f1ff397e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fec0afee0 0x7f1fec15d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:23.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.830+0000 7f1ff397e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1fec15b430 0x7f1fec15fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:23.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.830+0000 7f1ff397e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec0b4860 con 0x7f1fec0afee0 2024-05-20T18:35:23.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.830+0000 7f1ff397e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec0b4340 con 0x7f1fec0afad0 2024-05-20T18:35:23.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.830+0000 7f1ff397e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1fec15e0b0 con 0x7f1fec15b430 2024-05-20T18:35:23.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.830+0000 7f1ff0f19640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fec0afee0 0x7f1fec15d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:23.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.830+0000 7f1ff0f19640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fec0afee0 0x7f1fec15d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55826/0 (socket says 172.21.15.5:55826) 2024-05-20T18:35:23.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.830+0000 7f1ff0f19640 1 -- 172.21.15.5:0/342270618 learned_addr learned my addr 172.21.15.5:0/342270618 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:23.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.831+0000 7f1ff171a640 1 --2- 172.21.15.5:0/342270618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fec0afad0 0x7f1fec14a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:23.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.831+0000 7f1ff1f1b640 1 --2- 172.21.15.5:0/342270618 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1fec15b430 0x7f1fec15fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:23.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.831+0000 7f1ff0f19640 1 -- 172.21.15.5:0/342270618 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1fec15b430 msgr2=0x7f1fec15fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:23.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.831+0000 7f1ff0f19640 1 --2- 172.21.15.5:0/342270618 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1fec15b430 0x7f1fec15fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:23.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.831+0000 7f1ff0f19640 1 -- 172.21.15.5:0/342270618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fec0afad0 msgr2=0x7f1fec14a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:23.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.831+0000 7f1ff0f19640 1 --2- 172.21.15.5:0/342270618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fec0afad0 0x7f1fec14a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:23.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.831+0000 7f1ff0f19640 1 -- 172.21.15.5:0/342270618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fec15fa10 con 0x7f1fec0afee0 2024-05-20T18:35:23.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.831+0000 7f1ff171a640 1 --2- 172.21.15.5:0/342270618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fec0afad0 0x7f1fec14a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:23.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.831+0000 7f1ff0f19640 1 --2- 172.21.15.5:0/342270618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fec0afee0 0x7f1fec15d5c0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f1fe00090b0 tx=0x7f1fe00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:23.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.831+0000 7f1fe9ffb640 1 -- 172.21.15.5:0/342270618 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0037040 con 0x7f1fec0afee0 2024-05-20T18:35:23.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.831+0000 7f1fe9ffb640 1 -- 172.21.15.5:0/342270618 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fe0004230 con 0x7f1fec0afee0 2024-05-20T18:35:23.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.831+0000 7f1ff397e640 1 -- 172.21.15.5:0/342270618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fec15a390 con 0x7f1fec0afee0 2024-05-20T18:35:23.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.831+0000 7f1fe9ffb640 1 -- 172.21.15.5:0/342270618 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0003610 con 0x7f1fec0afee0 2024-05-20T18:35:23.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.831+0000 7f1ff397e640 1 -- 172.21.15.5:0/342270618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fec15a950 con 0x7f1fec0afee0 2024-05-20T18:35:23.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.832+0000 7f1ff397e640 1 -- 172.21.15.5:0/342270618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fec075b00 con 0x7f1fec0afee0 2024-05-20T18:35:23.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.833+0000 7f1fe9ffb640 1 -- 172.21.15.5:0/342270618 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1fe00037b0 con 0x7f1fec0afee0 2024-05-20T18:35:23.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.833+0000 7f1fe9ffb640 1 --2- 172.21.15.5:0/342270618 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1fd806be30 0x7f1fd806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:23.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.834+0000 7f1fe9ffb640 1 -- 172.21.15.5:0/342270618 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5507+0+0 (secure 0 0 0) 0x7f1fe00abd90 con 0x7f1fec0afee0 2024-05-20T18:35:23.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.834+0000 7f1ff171a640 1 --2- 172.21.15.5:0/342270618 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1fd806be30 0x7f1fd806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:23.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.834+0000 7f1ff171a640 1 --2- 172.21.15.5:0/342270618 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1fd806be30 0x7f1fd806e340 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4000f60 tx=0x7f1fe4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:23.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:23.838+0000 7f1fe9ffb640 1 -- 172.21.15.5:0/342270618 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1fec15fa10 con 0x7f1fec0afee0 2024-05-20T18:35:24.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.023+0000 7f1ff397e640 1 -- 172.21.15.5:0/342270618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1fec15a120 con 0x7f1fec0afee0 2024-05-20T18:35:24.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.023+0000 7f1fe9ffb640 1 -- 172.21.15.5:0/342270618 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f1fe007a770 con 0x7f1fec0afee0 2024-05-20T18:35:24.024 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:35:24.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.025+0000 7f1ff397e640 1 -- 172.21.15.5:0/342270618 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1fd806be30 msgr2=0x7f1fd806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:24.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.025+0000 7f1ff397e640 1 --2- 172.21.15.5:0/342270618 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1fd806be30 0x7f1fd806e340 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4000f60 tx=0x7f1fe4009040 comp rx=0 tx=0).stop 2024-05-20T18:35:24.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.025+0000 7f1ff397e640 1 -- 172.21.15.5:0/342270618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fec0afee0 msgr2=0x7f1fec15d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:24.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.025+0000 7f1ff397e640 1 --2- 172.21.15.5:0/342270618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fec0afee0 0x7f1fec15d5c0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f1fe00090b0 tx=0x7f1fe00046c0 comp rx=0 tx=0).stop 2024-05-20T18:35:24.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.025+0000 7f1ff397e640 1 -- 172.21.15.5:0/342270618 shutdown_connections 2024-05-20T18:35:24.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.025+0000 7f1ff397e640 1 --2- 172.21.15.5:0/342270618 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1fd806be30 0x7f1fd806e340 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:24.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.025+0000 7f1ff397e640 1 --2- 172.21.15.5:0/342270618 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1fec15b430 0x7f1fec15fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:24.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.025+0000 7f1ff397e640 1 --2- 172.21.15.5:0/342270618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fec0afee0 0x7f1fec15d5c0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:24.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.025+0000 7f1ff397e640 1 --2- 172.21.15.5:0/342270618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1fec0afad0 0x7f1fec14a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:24.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.025+0000 7f1ff397e640 1 -- 172.21.15.5:0/342270618 >> 172.21.15.5:0/342270618 conn(0x7f1fec0cd2a0 msgr2=0x7f1fec06c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:24.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.025+0000 7f1ff397e640 1 -- 172.21.15.5:0/342270618 shutdown_connections 2024-05-20T18:35:24.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.025+0000 7f1ff397e640 1 -- 172.21.15.5:0/342270618 wait complete. 2024-05-20T18:35:24.038 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-05-20T18:35:24.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.151+0000 7f03193e0640 1 Processor -- start 2024-05-20T18:35:24.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.152+0000 7f03193e0640 1 -- start start 2024-05-20T18:35:24.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.153+0000 7f03193e0640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f03140af030 0x7f03140af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:24.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.153+0000 7f03193e0640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03140af980 con 0x7f031406c180 2024-05-20T18:35:24.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.153+0000 7f03193e0640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0314160430 con 0x7f031415f700 2024-05-20T18:35:24.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.153+0000 7f03193e0640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f031406bcf0 con 0x7f03140af030 2024-05-20T18:35:24.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.153+0000 7f0312ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f03140af030 0x7f03140af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:24.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.153+0000 7f03137fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f031406c180 0x7f031406c590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47382/0 (socket says 172.21.15.5:47382) 2024-05-20T18:35:24.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.153+0000 7f03137fe640 1 -- 172.21.15.5:0/2907892073 learned_addr learned my addr 172.21.15.5:0/2907892073 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:24.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.153+0000 7f0312ffd640 1 -- 172.21.15.5:0/2907892073 >> v1:172.21.15.63:6789/0 conn(0x7f031415f700 legacy=0x7f031415fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:24.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.153+0000 7f0312ffd640 1 -- 172.21.15.5:0/2907892073 >> v1:172.21.15.5:6789/0 conn(0x7f031406c180 legacy=0x7f031406c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:24.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.153+0000 7f0311ffb640 1 -- 172.21.15.5:0/2907892073 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1282388611 0 0) 0x7f030c001490 con 0x7f031406c180 2024-05-20T18:35:24.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.153+0000 7f0312ffd640 1 -- 172.21.15.5:0/2907892073 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f031406b6a0 con 0x7f03140af030 2024-05-20T18:35:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.154+0000 7f0312ffd640 1 --2- 172.21.15.5:0/2907892073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03140af030 0x7f03140af440 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f03040097a0 tx=0x7f030402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ecfe33c4967347 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.154+0000 7f0311ffb640 1 -- 172.21.15.5:0/2907892073 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0304037070 con 0x7f03140af030 2024-05-20T18:35:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.154+0000 7f0311ffb640 1 -- 172.21.15.5:0/2907892073 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0304032440 con 0x7f03140af030 2024-05-20T18:35:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.154+0000 7f0311ffb640 1 -- 172.21.15.5:0/2907892073 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0304032810 con 0x7f03140af030 2024-05-20T18:35:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.155+0000 7f03193e0640 1 -- 172.21.15.5:0/2907892073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03140af030 msgr2=0x7f03140af440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.155+0000 7f03193e0640 1 --2- 172.21.15.5:0/2907892073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03140af030 0x7f03140af440 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f03040097a0 tx=0x7f030402a230 comp rx=0 tx=0).stop 2024-05-20T18:35:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.155+0000 7f03193e0640 1 -- 172.21.15.5:0/2907892073 shutdown_connections 2024-05-20T18:35:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.155+0000 7f03193e0640 1 --2- 172.21.15.5:0/2907892073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03140af030 0x7f03140af440 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.155+0000 7f03193e0640 1 -- 172.21.15.5:0/2907892073 >> 172.21.15.5:0/2907892073 conn(0x7f03140b6620 msgr2=0x7f03140b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.155+0000 7f03193e0640 1 -- 172.21.15.5:0/2907892073 shutdown_connections 2024-05-20T18:35:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.155+0000 7f03193e0640 1 -- 172.21.15.5:0/2907892073 wait complete. 2024-05-20T18:35:24.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.157+0000 7f03193e0640 1 Processor -- start 2024-05-20T18:35:24.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.157+0000 7f03193e0640 1 -- start start 2024-05-20T18:35:24.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.158+0000 7f03193e0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f031406c180 0x7f031415dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:24.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.158+0000 7f03193e0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03140af030 0x7f031415c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:24.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.159+0000 7f03193e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031415f700 0x7f03141499c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:24.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.159+0000 7f03193e0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f031406cca0 con 0x7f031415f700 2024-05-20T18:35:24.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.159+0000 7f03193e0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0314159aa0 con 0x7f03140af030 2024-05-20T18:35:24.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.159+0000 7f03193e0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f031406ce20 con 0x7f031406c180 2024-05-20T18:35:24.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.159+0000 7f03137fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031415f700 0x7f03141499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:24.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.159+0000 7f03137fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031415f700 0x7f03141499c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55834/0 (socket says 172.21.15.5:55834) 2024-05-20T18:35:24.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.159+0000 7f03137fe640 1 -- 172.21.15.5:0/490383238 learned_addr learned my addr 172.21.15.5:0/490383238 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:24.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.159+0000 7f0312ffd640 1 --2- 172.21.15.5:0/490383238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f031406c180 0x7f031415dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:24.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.159+0000 7f03127fc640 1 --2- 172.21.15.5:0/490383238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03140af030 0x7f031415c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:24.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.159+0000 7f03137fe640 1 -- 172.21.15.5:0/490383238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f031406c180 msgr2=0x7f031415dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:24.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.159+0000 7f03137fe640 1 --2- 172.21.15.5:0/490383238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f031406c180 0x7f031415dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:24.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.159+0000 7f03137fe640 1 -- 172.21.15.5:0/490383238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03140af030 msgr2=0x7f031415c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:24.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.159+0000 7f03137fe640 1 --2- 172.21.15.5:0/490383238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03140af030 0x7f031415c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:24.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.159+0000 7f03137fe640 1 -- 172.21.15.5:0/490383238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f031415ebd0 con 0x7f031415f700 2024-05-20T18:35:24.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.159+0000 7f03127fc640 1 --2- 172.21.15.5:0/490383238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03140af030 0x7f031415c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:24.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.159+0000 7f03137fe640 1 --2- 172.21.15.5:0/490383238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031415f700 0x7f03141499c0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f030c00cb20 tx=0x7f030c002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:24.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.159+0000 7f02fb7fe640 1 -- 172.21.15.5:0/490383238 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c00abd0 con 0x7f031415f700 2024-05-20T18:35:24.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.159+0000 7f02fb7fe640 1 -- 172.21.15.5:0/490383238 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f030c00ad90 con 0x7f031415f700 2024-05-20T18:35:24.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.159+0000 7f03193e0640 1 -- 172.21.15.5:0/490383238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f031415e610 con 0x7f031415f700 2024-05-20T18:35:24.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.159+0000 7f02fb7fe640 1 -- 172.21.15.5:0/490383238 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c012640 con 0x7f031415f700 2024-05-20T18:35:24.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.159+0000 7f03193e0640 1 -- 172.21.15.5:0/490383238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03141479d0 con 0x7f031415f700 2024-05-20T18:35:24.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.161+0000 7f03193e0640 1 -- 172.21.15.5:0/490383238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0314075b00 con 0x7f031415f700 2024-05-20T18:35:24.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.161+0000 7f02fb7fe640 1 -- 172.21.15.5:0/490383238 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f030c0127e0 con 0x7f031415f700 2024-05-20T18:35:24.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.162+0000 7f02fb7fe640 1 --2- 172.21.15.5:0/490383238 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f030006be60 0x7f030006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:24.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.162+0000 7f0312ffd640 1 --2- 172.21.15.5:0/490383238 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f030006be60 0x7f030006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:24.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.162+0000 7f02fb7fe640 1 -- 172.21.15.5:0/490383238 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5507+0+0 (secure 0 0 0) 0x7f030c08b230 con 0x7f031415f700 2024-05-20T18:35:24.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.162+0000 7f0312ffd640 1 --2- 172.21.15.5:0/490383238 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f030006be60 0x7f030006e370 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0304034000 tx=0x7f030402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:24.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.167+0000 7f02fb7fe640 1 -- 172.21.15.5:0/490383238 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f03140af980 con 0x7f031415f700 2024-05-20T18:35:24.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:24.422+0000 7f03193e0640 1 -- 172.21.15.5:0/490383238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7f03140d1df0 con 0x7f031415f700 2024-05-20T18:35:25.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.366+0000 7f02fb7fe640 1 -- 172.21.15.5:0/490383238 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v400) ==== 149+0+0 (secure 0 0 0) 0x7f030c059ca0 con 0x7f031415f700 2024-05-20T18:35:25.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.368+0000 7f03193e0640 1 -- 172.21.15.5:0/490383238 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f030006be60 msgr2=0x7f030006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:25.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.368+0000 7f03193e0640 1 --2- 172.21.15.5:0/490383238 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f030006be60 0x7f030006e370 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0304034000 tx=0x7f030402e000 comp rx=0 tx=0).stop 2024-05-20T18:35:25.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.368+0000 7f03193e0640 1 -- 172.21.15.5:0/490383238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031415f700 msgr2=0x7f03141499c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:25.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.368+0000 7f03193e0640 1 --2- 172.21.15.5:0/490383238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031415f700 0x7f03141499c0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f030c00cb20 tx=0x7f030c002190 comp rx=0 tx=0).stop 2024-05-20T18:35:25.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.369+0000 7f03193e0640 1 -- 172.21.15.5:0/490383238 shutdown_connections 2024-05-20T18:35:25.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.369+0000 7f03193e0640 1 --2- 172.21.15.5:0/490383238 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f030006be60 0x7f030006e370 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:25.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.369+0000 7f03193e0640 1 --2- 172.21.15.5:0/490383238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031415f700 0x7f03141499c0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:25.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.369+0000 7f03193e0640 1 --2- 172.21.15.5:0/490383238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03140af030 0x7f031415c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:25.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.369+0000 7f03193e0640 1 --2- 172.21.15.5:0/490383238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f031406c180 0x7f031415dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:25.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.369+0000 7f03193e0640 1 -- 172.21.15.5:0/490383238 >> 172.21.15.5:0/490383238 conn(0x7f03140b6620 msgr2=0x7f0314158f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:25.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.369+0000 7f03193e0640 1 -- 172.21.15.5:0/490383238 shutdown_connections 2024-05-20T18:35:25.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.369+0000 7f03193e0640 1 -- 172.21.15.5:0/490383238 wait complete. 2024-05-20T18:35:25.384 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-20T18:35:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.508+0000 7fd24951d640 1 Processor -- start 2024-05-20T18:35:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.509+0000 7fd24951d640 1 -- start start 2024-05-20T18:35:25.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.510+0000 7fd24951d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd244162820 0x7fd2440b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:25.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.510+0000 7fd24951d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd2440b3e80 0x7fd24406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:25.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.510+0000 7fd24951d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd24406dd20 con 0x7fd24406d910 2024-05-20T18:35:25.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.510+0000 7fd24951d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fd2440d0630 con 0x7fd244162820 2024-05-20T18:35:25.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.510+0000 7fd24951d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd2440d39c0 con 0x7fd2440b3e80 2024-05-20T18:35:25.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.510+0000 7fd242ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd244162820 0x7fd2440b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:25.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.510+0000 7fd242ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd244162820 0x7fd2440b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37688/0 (socket says 172.21.15.5:37688) 2024-05-20T18:35:25.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.510+0000 7fd242ffd640 1 -- 172.21.15.5:0/600864056 learned_addr learned my addr 172.21.15.5:0/600864056 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:25.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.510+0000 7fd2437fe640 1 --1- 172.21.15.5:0/600864056 >> v1:172.21.15.5:6789/0 conn(0x7fd24406d910 0x7fd2440d27a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36080/0 (socket says 172.21.15.5:36080) 2024-05-20T18:35:25.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.510+0000 7fd2427fc640 1 --2- 172.21.15.5:0/600864056 >> v2:172.21.15.63:3300/0 conn(0x7fd2440b3e80 0x7fd24406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:25.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.510+0000 7fd242ffd640 1 -- 172.21.15.5:0/600864056 >> v2:172.21.15.63:3300/0 conn(0x7fd2440b3e80 msgr2=0x7fd24406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:25.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.510+0000 7fd242ffd640 1 --2- 172.21.15.5:0/600864056 >> v2:172.21.15.63:3300/0 conn(0x7fd2440b3e80 0x7fd24406d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:25.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.510+0000 7fd242ffd640 1 -- 172.21.15.5:0/600864056 >> v1:172.21.15.5:6789/0 conn(0x7fd24406d910 legacy=0x7fd2440d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:25.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.510+0000 7fd242ffd640 1 -- 172.21.15.5:0/600864056 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2440d3ca0 con 0x7fd244162820 2024-05-20T18:35:25.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.511+0000 7fd242ffd640 1 --2- 172.21.15.5:0/600864056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd244162820 0x7fd2440b3940 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fd2440d1230 tx=0x7fd23402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f529b35e34db026e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:25.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.511+0000 7fd241ffb640 1 -- 172.21.15.5:0/600864056 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd234036070 con 0x7fd244162820 2024-05-20T18:35:25.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.511+0000 7fd241ffb640 1 -- 172.21.15.5:0/600864056 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd234031600 con 0x7fd244162820 2024-05-20T18:35:25.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.511+0000 7fd241ffb640 1 -- 172.21.15.5:0/600864056 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2340319d0 con 0x7fd244162820 2024-05-20T18:35:25.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.512+0000 7fd24951d640 1 -- 172.21.15.5:0/600864056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd244162820 msgr2=0x7fd2440b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:25.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.512+0000 7fd24951d640 1 --2- 172.21.15.5:0/600864056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd244162820 0x7fd2440b3940 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fd2440d1230 tx=0x7fd23402a730 comp rx=0 tx=0).stop 2024-05-20T18:35:25.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.512+0000 7fd24951d640 1 -- 172.21.15.5:0/600864056 shutdown_connections 2024-05-20T18:35:25.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.512+0000 7fd24951d640 1 --2- 172.21.15.5:0/600864056 >> v2:172.21.15.63:3300/0 conn(0x7fd2440b3e80 0x7fd24406d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:25.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.512+0000 7fd24951d640 1 --2- 172.21.15.5:0/600864056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd244162820 0x7fd2440b3940 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:25.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.512+0000 7fd24951d640 1 -- 172.21.15.5:0/600864056 >> 172.21.15.5:0/600864056 conn(0x7fd2440d4bd0 msgr2=0x7fd2440d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:25.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.512+0000 7fd24951d640 1 -- 172.21.15.5:0/600864056 shutdown_connections 2024-05-20T18:35:25.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.512+0000 7fd24951d640 1 -- 172.21.15.5:0/600864056 wait complete. 2024-05-20T18:35:25.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.514+0000 7fd24951d640 1 Processor -- start 2024-05-20T18:35:25.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.514+0000 7fd24951d640 1 -- start start 2024-05-20T18:35:25.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.515+0000 7fd24951d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd24406d910 0x7fd24414aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:25.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.516+0000 7fd24951d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2440b3e80 0x7fd24414b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:25.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.516+0000 7fd24951d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd244162820 0x7fd2441f8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.516+0000 7fd24951d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd2440d0300 con 0x7fd24406d910 2024-05-20T18:35:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.516+0000 7fd24951d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd2440d4260 con 0x7fd2440b3e80 2024-05-20T18:35:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.516+0000 7fd24951d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd2440b91a0 con 0x7fd244162820 2024-05-20T18:35:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.516+0000 7fd242ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd24406d910 0x7fd24414aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.516+0000 7fd242ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd24406d910 0x7fd24414aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37702/0 (socket says 172.21.15.5:37702) 2024-05-20T18:35:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.516+0000 7fd242ffd640 1 -- 172.21.15.5:0/778416710 learned_addr learned my addr 172.21.15.5:0/778416710 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.516+0000 7fd2427fc640 1 --2- 172.21.15.5:0/778416710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2440b3e80 0x7fd24414b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.516+0000 7fd2437fe640 1 --2- 172.21.15.5:0/778416710 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd244162820 0x7fd2441f8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.516+0000 7fd242ffd640 1 -- 172.21.15.5:0/778416710 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd244162820 msgr2=0x7fd2441f8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.516+0000 7fd242ffd640 1 --2- 172.21.15.5:0/778416710 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd244162820 0x7fd2441f8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:25.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.516+0000 7fd242ffd640 1 -- 172.21.15.5:0/778416710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2440b3e80 msgr2=0x7fd24414b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:25.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.516+0000 7fd242ffd640 1 --2- 172.21.15.5:0/778416710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2440b3e80 0x7fd24414b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:25.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.516+0000 7fd242ffd640 1 -- 172.21.15.5:0/778416710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2440c99f0 con 0x7fd24406d910 2024-05-20T18:35:25.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.516+0000 7fd2437fe640 1 --2- 172.21.15.5:0/778416710 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd244162820 0x7fd2441f8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:25.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.516+0000 7fd242ffd640 1 --2- 172.21.15.5:0/778416710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd24406d910 0x7fd24414aba0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fd234009f40 tx=0x7fd234004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:25.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.516+0000 7fd23b7fe640 1 -- 172.21.15.5:0/778416710 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23403f020 con 0x7fd24406d910 2024-05-20T18:35:25.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.516+0000 7fd23b7fe640 1 -- 172.21.15.5:0/778416710 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd234037040 con 0x7fd24406d910 2024-05-20T18:35:25.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.516+0000 7fd23b7fe640 1 -- 172.21.15.5:0/778416710 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd234036070 con 0x7fd24406d910 2024-05-20T18:35:25.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.516+0000 7fd24951d640 1 -- 172.21.15.5:0/778416710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2440c9cd0 con 0x7fd24406d910 2024-05-20T18:35:25.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.517+0000 7fd24951d640 1 -- 172.21.15.5:0/778416710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2440cbd20 con 0x7fd24406d910 2024-05-20T18:35:25.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.518+0000 7fd24951d640 1 -- 172.21.15.5:0/778416710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd244075b00 con 0x7fd24406d910 2024-05-20T18:35:25.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.518+0000 7fd23b7fe640 1 -- 172.21.15.5:0/778416710 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd234042020 con 0x7fd24406d910 2024-05-20T18:35:25.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.519+0000 7fd23b7fe640 1 --2- 172.21.15.5:0/778416710 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd22406be30 0x7fd22406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:25.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.519+0000 7fd23b7fe640 1 -- 172.21.15.5:0/778416710 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5507+0+0 (secure 0 0 0) 0x7fd2340ab310 con 0x7fd24406d910 2024-05-20T18:35:25.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.519+0000 7fd2427fc640 1 --2- 172.21.15.5:0/778416710 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd22406be30 0x7fd22406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:25.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.519+0000 7fd2427fc640 1 --2- 172.21.15.5:0/778416710 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd22406be30 0x7fd22406e340 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd22c006fd0 tx=0x7fd22c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:25.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.523+0000 7fd23b7fe640 1 -- 172.21.15.5:0/778416710 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd234049030 con 0x7fd24406d910 2024-05-20T18:35:25.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.710+0000 7fd24951d640 1 -- 172.21.15.5:0/778416710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fd244160a90 con 0x7fd24406d910 2024-05-20T18:35:25.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.714+0000 7fd23b7fe640 1 -- 172.21.15.5:0/778416710 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7fd234079d80 con 0x7fd24406d910 2024-05-20T18:35:25.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.716+0000 7fd24951d640 1 -- 172.21.15.5:0/778416710 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd22406be30 msgr2=0x7fd22406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:25.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.716+0000 7fd24951d640 1 --2- 172.21.15.5:0/778416710 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd22406be30 0x7fd22406e340 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd22c006fd0 tx=0x7fd22c008040 comp rx=0 tx=0).stop 2024-05-20T18:35:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.716+0000 7fd24951d640 1 -- 172.21.15.5:0/778416710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd24406d910 msgr2=0x7fd24414aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.716+0000 7fd24951d640 1 --2- 172.21.15.5:0/778416710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd24406d910 0x7fd24414aba0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fd234009f40 tx=0x7fd234004a40 comp rx=0 tx=0).stop 2024-05-20T18:35:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.716+0000 7fd24951d640 1 -- 172.21.15.5:0/778416710 shutdown_connections 2024-05-20T18:35:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.716+0000 7fd24951d640 1 --2- 172.21.15.5:0/778416710 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd22406be30 0x7fd22406e340 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.716+0000 7fd24951d640 1 --2- 172.21.15.5:0/778416710 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd244162820 0x7fd2441f8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.716+0000 7fd24951d640 1 --2- 172.21.15.5:0/778416710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2440b3e80 0x7fd24414b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.716+0000 7fd24951d640 1 --2- 172.21.15.5:0/778416710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd24406d910 0x7fd24414aba0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.716+0000 7fd24951d640 1 -- 172.21.15.5:0/778416710 >> 172.21.15.5:0/778416710 conn(0x7fd2440d4bd0 msgr2=0x7fd2440b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.716+0000 7fd24951d640 1 -- 172.21.15.5:0/778416710 shutdown_connections 2024-05-20T18:35:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:25.716+0000 7fd24951d640 1 -- 172.21.15.5:0/778416710 wait complete. 2024-05-20T18:35:25.729 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-20T18:35:25.729 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T18:35:25.766 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T18:35:25.819 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-20T18:35:25.819 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:35:25.819 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:35:25.836 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15663 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T18:35:25.836 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:35:25.836 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:35:25.855 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19732 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T18:35:25.855 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:35:25.855 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:35:25.892 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15663 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T18:35:25.892 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:35:25.892 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:35:25.913 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19732 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T18:35:25.913 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-20T18:35:25.914 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:35:26.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.045+0000 7ffa46b2f640 1 Processor -- start 2024-05-20T18:35:26.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.045+0000 7ffa46b2f640 1 -- start start 2024-05-20T18:35:26.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.046+0000 7ffa46b2f640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ffa4015d530 0x7ffa4015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:26.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.046+0000 7ffa46b2f640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa400cd580 con 0x7ffa40070850 2024-05-20T18:35:26.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.046+0000 7ffa46b2f640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa400cd860 con 0x7ffa400b1700 2024-05-20T18:35:26.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.046+0000 7ffa46b2f640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ffa400ce190 con 0x7ffa4015d530 2024-05-20T18:35:26.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.046+0000 7ffa450cc640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ffa40070850 0x7ffa40070c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36088/0 (socket says 172.21.15.5:36088) 2024-05-20T18:35:26.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.046+0000 7ffa450cc640 1 -- 172.21.15.5:0/3256065629 learned_addr learned my addr 172.21.15.5:0/3256065629 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:26.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.047+0000 7ffa448cb640 1 --2- 172.21.15.5:0/3256065629 >> v2:172.21.15.63:3300/0 conn(0x7ffa4015d530 0x7ffa4015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:26.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.047+0000 7ffa3f7fe640 1 -- 172.21.15.5:0/3256065629 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3320940758 0 0) 0x7ffa400cd580 con 0x7ffa40070850 2024-05-20T18:35:26.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.047+0000 7ffa3f7fe640 1 -- 172.21.15.5:0/3256065629 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa30003520 con 0x7ffa40070850 2024-05-20T18:35:26.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.047+0000 7ffa3f7fe640 1 -- 172.21.15.5:0/3256065629 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2351844194 0 0) 0x7ffa30003520 con 0x7ffa40070850 2024-05-20T18:35:26.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.047+0000 7ffa3f7fe640 1 -- 172.21.15.5:0/3256065629 >> v2:172.21.15.63:3300/0 conn(0x7ffa4015d530 msgr2=0x7ffa4015aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:26.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.047+0000 7ffa3f7fe640 1 --2- 172.21.15.5:0/3256065629 >> v2:172.21.15.63:3300/0 conn(0x7ffa4015d530 0x7ffa4015aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:26.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.047+0000 7ffa3f7fe640 1 -- 172.21.15.5:0/3256065629 >> v1:172.21.15.63:6789/0 conn(0x7ffa400b1700 legacy=0x7ffa400b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:26.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.047+0000 7ffa3f7fe640 1 -- 172.21.15.5:0/3256065629 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa400cdb40 con 0x7ffa40070850 2024-05-20T18:35:26.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.047+0000 7ffa3f7fe640 1 -- 172.21.15.5:0/3256065629 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7ffa380019f0 con 0x7ffa40070850 2024-05-20T18:35:26.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.047+0000 7ffa3f7fe640 1 -- 172.21.15.5:0/3256065629 >> v1:172.21.15.5:6789/0 conn(0x7ffa40070850 legacy=0x7ffa40070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:26.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.047+0000 7ffa3f7fe640 1 --2- 172.21.15.5:0/3256065629 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa300039b0 0x7ffa30003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:26.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.047+0000 7ffa3f7fe640 1 --2- 172.21.15.5:0/3256065629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa30004800 0x7ffa30004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:26.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.047+0000 7ffa3f7fe640 1 --2- 172.21.15.5:0/3256065629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa300051c0 0x7ffa3000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:26.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.047+0000 7ffa3f7fe640 1 -- 172.21.15.5:0/3256065629 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa400ce190 con 0x7ffa300051c0 2024-05-20T18:35:26.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.047+0000 7ffa3f7fe640 1 -- 172.21.15.5:0/3256065629 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3000c160 con 0x7ffa30004800 2024-05-20T18:35:26.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.047+0000 7ffa3f7fe640 1 -- 172.21.15.5:0/3256065629 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffa3000c360 con 0x7ffa300039b0 2024-05-20T18:35:26.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.047+0000 7ffa450cc640 1 --2- 172.21.15.5:0/3256065629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa300051c0 0x7ffa3000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:26.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.047+0000 7ffa3ffff640 1 --2- 172.21.15.5:0/3256065629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa30004800 0x7ffa30004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:26.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.047+0000 7ffa448cb640 1 --2- 172.21.15.5:0/3256065629 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa300039b0 0x7ffa30003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:26.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.048+0000 7ffa450cc640 1 -- 172.21.15.5:0/3256065629 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa300039b0 msgr2=0x7ffa30003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:26.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.048+0000 7ffa450cc640 1 --2- 172.21.15.5:0/3256065629 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa300039b0 0x7ffa30003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:26.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.048+0000 7ffa450cc640 1 -- 172.21.15.5:0/3256065629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa30004800 msgr2=0x7ffa30004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:26.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.048+0000 7ffa450cc640 1 --2- 172.21.15.5:0/3256065629 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa30004800 0x7ffa30004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:26.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.048+0000 7ffa450cc640 1 -- 172.21.15.5:0/3256065629 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa30003520 con 0x7ffa300051c0 2024-05-20T18:35:26.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.048+0000 7ffa448cb640 1 --2- 172.21.15.5:0/3256065629 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa300039b0 0x7ffa30003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:26.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.048+0000 7ffa450cc640 1 -- 172.21.15.5:0/3256065629 reap_dead start 2024-05-20T18:35:26.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.048+0000 7ffa450cc640 1 --2- 172.21.15.5:0/3256065629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa300051c0 0x7ffa3000b9c0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800a830 tx=0x7ffa30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6011d01ae4f4bc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:26.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.049+0000 7ffa3f7fe640 1 -- 172.21.15.5:0/3256065629 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3800a480 con 0x7ffa300051c0 2024-05-20T18:35:26.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.049+0000 7ffa3f7fe640 1 -- 172.21.15.5:0/3256065629 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa3802f070 con 0x7ffa300051c0 2024-05-20T18:35:26.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.049+0000 7ffa3f7fe640 1 -- 172.21.15.5:0/3256065629 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3802bbc0 con 0x7ffa300051c0 2024-05-20T18:35:26.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.049+0000 7ffa46b2f640 1 -- 172.21.15.5:0/3256065629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa300051c0 msgr2=0x7ffa3000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:26.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.049+0000 7ffa46b2f640 1 --2- 172.21.15.5:0/3256065629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa300051c0 0x7ffa3000b9c0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800a830 tx=0x7ffa30005830 comp rx=0 tx=0).stop 2024-05-20T18:35:26.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.049+0000 7ffa46b2f640 1 -- 172.21.15.5:0/3256065629 shutdown_connections 2024-05-20T18:35:26.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.049+0000 7ffa46b2f640 1 --2- 172.21.15.5:0/3256065629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa300051c0 0x7ffa3000b9c0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:26.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.049+0000 7ffa46b2f640 1 -- 172.21.15.5:0/3256065629 >> 172.21.15.5:0/3256065629 conn(0x7ffa400d29d0 msgr2=0x7ffa400d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:26.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.049+0000 7ffa46b2f640 1 -- 172.21.15.5:0/3256065629 shutdown_connections 2024-05-20T18:35:26.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.049+0000 7ffa46b2f640 1 -- 172.21.15.5:0/3256065629 wait complete. 2024-05-20T18:35:26.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.051+0000 7ffa46b2f640 1 Processor -- start 2024-05-20T18:35:26.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.051+0000 7ffa46b2f640 1 -- start start 2024-05-20T18:35:26.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.053+0000 7ffa46b2f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa300051c0 0x7ffa4014bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.053+0000 7ffa46b2f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa4015bb60 0x7ffa4015bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.053+0000 7ffa46b2f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa400c6500 0x7ffa400c6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.053+0000 7ffa46b2f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4015aaa0 con 0x7ffa400c6500 2024-05-20T18:35:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.053+0000 7ffa46b2f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffa400cd230 con 0x7ffa4015bb60 2024-05-20T18:35:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.053+0000 7ffa46b2f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffa4015ac20 con 0x7ffa300051c0 2024-05-20T18:35:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.053+0000 7ffa450cc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa400c6500 0x7ffa400c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.053+0000 7ffa450cc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa400c6500 0x7ffa400c6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37718/0 (socket says 172.21.15.5:37718) 2024-05-20T18:35:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.053+0000 7ffa450cc640 1 -- 172.21.15.5:0/3439925767 learned_addr learned my addr 172.21.15.5:0/3439925767 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.053+0000 7ffa448cb640 1 --2- 172.21.15.5:0/3439925767 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa300051c0 0x7ffa4014bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.053+0000 7ffa3ffff640 1 --2- 172.21.15.5:0/3439925767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa4015bb60 0x7ffa4015bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.053+0000 7ffa450cc640 1 -- 172.21.15.5:0/3439925767 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa300051c0 msgr2=0x7ffa4014bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.053+0000 7ffa450cc640 1 --2- 172.21.15.5:0/3439925767 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa300051c0 0x7ffa4014bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.053+0000 7ffa450cc640 1 -- 172.21.15.5:0/3439925767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa4015bb60 msgr2=0x7ffa4015bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:26.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.053+0000 7ffa450cc640 1 --2- 172.21.15.5:0/3439925767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa4015bb60 0x7ffa4015bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:26.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.053+0000 7ffa450cc640 1 -- 172.21.15.5:0/3439925767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa400c6e50 con 0x7ffa400c6500 2024-05-20T18:35:26.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.053+0000 7ffa3ffff640 1 --2- 172.21.15.5:0/3439925767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa4015bb60 0x7ffa4015bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:26.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.053+0000 7ffa450cc640 1 --2- 172.21.15.5:0/3439925767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa400c6500 0x7ffa400c6910 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800a890 tx=0x7ffa400cd750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:26.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.053+0000 7ffa3d7fa640 1 -- 172.21.15.5:0/3439925767 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38004d70 con 0x7ffa400c6500 2024-05-20T18:35:26.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.054+0000 7ffa3d7fa640 1 -- 172.21.15.5:0/3439925767 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa3802b780 con 0x7ffa400c6500 2024-05-20T18:35:26.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.054+0000 7ffa3d7fa640 1 -- 172.21.15.5:0/3439925767 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3802bb30 con 0x7ffa400c6500 2024-05-20T18:35:26.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.054+0000 7ffa46b2f640 1 -- 172.21.15.5:0/3439925767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa4015c8e0 con 0x7ffa400c6500 2024-05-20T18:35:26.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.054+0000 7ffa46b2f640 1 -- 172.21.15.5:0/3439925767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa401564f0 con 0x7ffa400c6500 2024-05-20T18:35:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.054+0000 7ffa46b2f640 1 -- 172.21.15.5:0/3439925767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa4015c650 con 0x7ffa400c6500 2024-05-20T18:35:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.055+0000 7ffa3d7fa640 1 -- 172.21.15.5:0/3439925767 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffa3802bcd0 con 0x7ffa400c6500 2024-05-20T18:35:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.056+0000 7ffa3d7fa640 1 --2- 172.21.15.5:0/3439925767 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffa2406be60 0x7ffa2406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.056+0000 7ffa3d7fa640 1 -- 172.21.15.5:0/3439925767 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5507+0+0 (secure 0 0 0) 0x7ffa380aa860 con 0x7ffa400c6500 2024-05-20T18:35:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.056+0000 7ffa448cb640 1 --2- 172.21.15.5:0/3439925767 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffa2406be60 0x7ffa2406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:26.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.056+0000 7ffa448cb640 1 --2- 172.21.15.5:0/3439925767 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffa2406be60 0x7ffa2406e370 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ffa400d29a0 tx=0x7ffa34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:26.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.060+0000 7ffa3d7fa640 1 -- 172.21.15.5:0/3439925767 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ffa38047030 con 0x7ffa400c6500 2024-05-20T18:35:26.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.285+0000 7ffa46b2f640 1 -- 172.21.15.5:0/3439925767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa400b42f0 con 0x7ffa400c6500 2024-05-20T18:35:26.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.285+0000 7ffa3d7fa640 1 -- 172.21.15.5:0/3439925767 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+3857 (secure 0 0 0) 0x7ffa400b42f0 con 0x7ffa400c6500 2024-05-20T18:35:26.287 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:26.287 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":75,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/337248865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":337248865},{"type":"v1","addr":"172.21.15.63:6834","nonce":337248865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/3705772553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":3705772553},{"type":"v1","addr":"172.21.15.63:6835","nonce":3705772553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":7434,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/776188138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":776188138},{"type":"v1","addr":"172.21.15.5:6835","nonce":776188138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":8250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3925173727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3925173727},{"type":"v1","addr":"172.21.15.5:6837","nonce":3925173727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[]} 2024-05-20T18:35:26.287 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 75 2024-05-20T18:35:26.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.287+0000 7ffa46b2f640 1 -- 172.21.15.5:0/3439925767 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffa2406be60 msgr2=0x7ffa2406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:26.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.287+0000 7ffa46b2f640 1 --2- 172.21.15.5:0/3439925767 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffa2406be60 0x7ffa2406e370 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ffa400d29a0 tx=0x7ffa34008040 comp rx=0 tx=0).stop 2024-05-20T18:35:26.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.287+0000 7ffa46b2f640 1 -- 172.21.15.5:0/3439925767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa400c6500 msgr2=0x7ffa400c6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:26.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.287+0000 7ffa46b2f640 1 --2- 172.21.15.5:0/3439925767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa400c6500 0x7ffa400c6910 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800a890 tx=0x7ffa400cd750 comp rx=0 tx=0).stop 2024-05-20T18:35:26.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.287+0000 7ffa46b2f640 1 -- 172.21.15.5:0/3439925767 shutdown_connections 2024-05-20T18:35:26.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.287+0000 7ffa46b2f640 1 --2- 172.21.15.5:0/3439925767 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ffa2406be60 0x7ffa2406e370 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:26.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.287+0000 7ffa46b2f640 1 --2- 172.21.15.5:0/3439925767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa400c6500 0x7ffa400c6910 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:26.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.287+0000 7ffa46b2f640 1 --2- 172.21.15.5:0/3439925767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffa4015bb60 0x7ffa4015bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:26.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.287+0000 7ffa46b2f640 1 --2- 172.21.15.5:0/3439925767 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffa300051c0 0x7ffa4014bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:26.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.287+0000 7ffa46b2f640 1 -- 172.21.15.5:0/3439925767 >> 172.21.15.5:0/3439925767 conn(0x7ffa400d29d0 msgr2=0x7ffa400b3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:26.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.288+0000 7ffa46b2f640 1 -- 172.21.15.5:0/3439925767 shutdown_connections 2024-05-20T18:35:26.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.288+0000 7ffa46b2f640 1 -- 172.21.15.5:0/3439925767 wait complete. 2024-05-20T18:35:26.300 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-20T18:35:26.300 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-20T18:35:26.300 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-20T18:35:26.300 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:35:26.301 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-20T18:35:26.301 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-20T18:35:26.301 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-20T18:35:26.301 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:35:26.301 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-20T18:35:26.301 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-20T18:35:26.302 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-20T18:35:26.302 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:35:26.302 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-20T18:35:26.302 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-20T18:35:26.302 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-20T18:35:26.302 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:35:26.311 INFO:tasks.ceph.mds.a:Stopped 2024-05-20T18:35:26.311 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-20T18:35:26.314 INFO:tasks.ceph.mds.c:Stopped 2024-05-20T18:35:26.314 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-20T18:35:26.315 INFO:tasks.ceph.mds.b:Stopped 2024-05-20T18:35:26.315 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-20T18:35:26.316 INFO:tasks.ceph.mds.d:Stopped 2024-05-20T18:35:26.317 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-20T18:35:26.356 INFO:tasks.ceph.mds.a:Started 2024-05-20T18:35:26.357 DEBUG:teuthology.parallel:result is None 2024-05-20T18:35:26.358 INFO:tasks.ceph.mds.c:Started 2024-05-20T18:35:26.358 DEBUG:teuthology.parallel:result is None 2024-05-20T18:35:26.359 INFO:tasks.ceph.mds.b:Started 2024-05-20T18:35:26.359 DEBUG:teuthology.parallel:result is None 2024-05-20T18:35:26.360 INFO:tasks.ceph.mds.d:Started 2024-05-20T18:35:26.361 DEBUG:teuthology.parallel:result is None 2024-05-20T18:35:26.361 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-20T18:35:26.446 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-05-20T18:35:26.448 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-05-20T18:35:26.459 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-20T18:35:26.459 INFO:tasks.ceph.mds.d.smithi063.stdout:starting mds.d at 2024-05-20T18:35:26.460 INFO:tasks.ceph.mds.b.smithi063.stderr:2024-05-20T18:35:26.459+0000 7f458268e640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-05-20T18:35:26.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.482+0000 7fd8c003d640 1 Processor -- start 2024-05-20T18:35:26.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.483+0000 7fd8c003d640 1 -- start start 2024-05-20T18:35:26.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.484+0000 7fd8c003d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd8b81602f0 0x7fd8b809a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:26.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.484+0000 7fd8c003d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8b809aa30 0x7fd8b80d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:26.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.484+0000 7fd8c003d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8b8160700 con 0x7fd8b806da80 2024-05-20T18:35:26.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.484+0000 7fd8c003d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fd8b8159da0 con 0x7fd8b81602f0 2024-05-20T18:35:26.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.484+0000 7fd8bddd9640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd8b806da80 0x7fd8b80b1c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36100/0 (socket says 172.21.15.5:36100) 2024-05-20T18:35:26.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.484+0000 7fd8bddd9640 1 -- 172.21.15.5:0/2705839400 learned_addr learned my addr 172.21.15.5:0/2705839400 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.484+0000 7fd8c003d640 1 -- 172.21.15.5:0/2705839400 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd8b80af1c0 con 0x7fd8b809aa30 2024-05-20T18:35:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.484+0000 7fd8bd5d8640 1 --2- 172.21.15.5:0/2705839400 >> v2:172.21.15.5:3300/0 conn(0x7fd8b81602f0 0x7fd8b809a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.484+0000 7fd8be5da640 1 --2- 172.21.15.5:0/2705839400 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8b809aa30 0x7fd8b80d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.484+0000 7fd8bcdd7640 1 -- 172.21.15.5:0/2705839400 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 238015714 0 0) 0x7fd8b4001110 con 0x7fd8b806da80 2024-05-20T18:35:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.485+0000 7fd8bcdd7640 1 -- 172.21.15.5:0/2705839400 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8a4003520 con 0x7fd8b806da80 2024-05-20T18:35:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.485+0000 7fd8bd5d8640 1 -- 172.21.15.5:0/2705839400 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8b809aa30 msgr2=0x7fd8b80d1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.485+0000 7fd8bd5d8640 1 --2- 172.21.15.5:0/2705839400 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8b809aa30 0x7fd8b80d1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.485+0000 7fd8bd5d8640 1 -- 172.21.15.5:0/2705839400 >> v1:172.21.15.5:6789/0 conn(0x7fd8b806da80 legacy=0x7fd8b80b1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.485+0000 7fd8bd5d8640 1 -- 172.21.15.5:0/2705839400 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b809ae40 con 0x7fd8b81602f0 2024-05-20T18:35:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.485+0000 7fd8bd5d8640 1 --2- 172.21.15.5:0/2705839400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8b81602f0 0x7fd8b809a4f0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8009790 tx=0x7fd8a802a0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ba7454736e49cb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.485+0000 7fd8bcdd7640 1 -- 172.21.15.5:0/2705839400 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8037070 con 0x7fd8b81602f0 2024-05-20T18:35:26.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.485+0000 7fd8bcdd7640 1 -- 172.21.15.5:0/2705839400 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a802ae30 con 0x7fd8b81602f0 2024-05-20T18:35:26.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.485+0000 7fd8bcdd7640 1 -- 172.21.15.5:0/2705839400 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8032640 con 0x7fd8b81602f0 2024-05-20T18:35:26.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.486+0000 7fd8c003d640 1 -- 172.21.15.5:0/2705839400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8b81602f0 msgr2=0x7fd8b809a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:26.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.486+0000 7fd8c003d640 1 --2- 172.21.15.5:0/2705839400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8b81602f0 0x7fd8b809a4f0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8009790 tx=0x7fd8a802a0e0 comp rx=0 tx=0).stop 2024-05-20T18:35:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.486+0000 7fd8c003d640 1 -- 172.21.15.5:0/2705839400 shutdown_connections 2024-05-20T18:35:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.486+0000 7fd8c003d640 1 --2- 172.21.15.5:0/2705839400 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8b809aa30 0x7fd8b80d1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.486+0000 7fd8c003d640 1 --2- 172.21.15.5:0/2705839400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8b81602f0 0x7fd8b809a4f0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.486+0000 7fd8c003d640 1 -- 172.21.15.5:0/2705839400 >> 172.21.15.5:0/2705839400 conn(0x7fd8b80b6670 msgr2=0x7fd8b80b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.486+0000 7fd8c003d640 1 -- 172.21.15.5:0/2705839400 shutdown_connections 2024-05-20T18:35:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.486+0000 7fd8c003d640 1 -- 172.21.15.5:0/2705839400 wait complete. 2024-05-20T18:35:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.488+0000 7fd8c003d640 1 Processor -- start 2024-05-20T18:35:26.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.488+0000 7fd8c003d640 1 -- start start 2024-05-20T18:35:26.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.489+0000 7fd8c003d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8b806da80 0x7fd8b8155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:26.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.489+0000 7fd8c003d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8b809aa30 0x7fd8b81582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:26.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.490+0000 7fd8c003d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8b81602f0 0x7fd8b8158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:26.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.490+0000 7fd8c003d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b80b2390 con 0x7fd8b806da80 2024-05-20T18:35:26.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.490+0000 7fd8c003d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b80d17b0 con 0x7fd8b81602f0 2024-05-20T18:35:26.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.490+0000 7fd8c003d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd8b80af410 con 0x7fd8b809aa30 2024-05-20T18:35:26.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.490+0000 7fd8bddd9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8b806da80 0x7fd8b8155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:26.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.490+0000 7fd8bddd9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8b806da80 0x7fd8b8155940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37780/0 (socket says 172.21.15.5:37780) 2024-05-20T18:35:26.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.490+0000 7fd8bddd9640 1 -- 172.21.15.5:0/715878386 learned_addr learned my addr 172.21.15.5:0/715878386 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:26.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.490+0000 7fd8be5da640 1 --2- 172.21.15.5:0/715878386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8b81602f0 0x7fd8b8158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.490+0000 7fd8bd5d8640 1 --2- 172.21.15.5:0/715878386 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8b809aa30 0x7fd8b81582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.490+0000 7fd8bddd9640 1 -- 172.21.15.5:0/715878386 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8b809aa30 msgr2=0x7fd8b81582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.490+0000 7fd8bddd9640 1 --2- 172.21.15.5:0/715878386 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8b809aa30 0x7fd8b81582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.490+0000 7fd8bddd9640 1 -- 172.21.15.5:0/715878386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8b81602f0 msgr2=0x7fd8b8158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.490+0000 7fd8bddd9640 1 --2- 172.21.15.5:0/715878386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8b81602f0 0x7fd8b8158830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.490+0000 7fd8bddd9640 1 -- 172.21.15.5:0/715878386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b809d4b0 con 0x7fd8b806da80 2024-05-20T18:35:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.490+0000 7fd8bddd9640 1 --2- 172.21.15.5:0/715878386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8b806da80 0x7fd8b8155940 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fd8b400c430 tx=0x7fd8b400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.490+0000 7fd8ae7fc640 1 -- 172.21.15.5:0/715878386 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4017070 con 0x7fd8b806da80 2024-05-20T18:35:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.490+0000 7fd8ae7fc640 1 -- 172.21.15.5:0/715878386 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8b4005ce0 con 0x7fd8b806da80 2024-05-20T18:35:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.490+0000 7fd8ae7fc640 1 -- 172.21.15.5:0/715878386 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b40072d0 con 0x7fd8b806da80 2024-05-20T18:35:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.490+0000 7fd8c003d640 1 -- 172.21.15.5:0/715878386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8b809d790 con 0x7fd8b806da80 2024-05-20T18:35:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.490+0000 7fd8c003d640 1 -- 172.21.15.5:0/715878386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8b809dff0 con 0x7fd8b806da80 2024-05-20T18:35:26.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.491+0000 7fd8c003d640 1 -- 172.21.15.5:0/715878386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8b8075b00 con 0x7fd8b806da80 2024-05-20T18:35:26.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.492+0000 7fd8ae7fc640 1 -- 172.21.15.5:0/715878386 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd8b401a070 con 0x7fd8b806da80 2024-05-20T18:35:26.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.493+0000 7fd8ae7fc640 1 --2- 172.21.15.5:0/715878386 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd89c06be30 0x7fd89c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:26.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.493+0000 7fd8ae7fc640 1 -- 172.21.15.5:0/715878386 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5507+0+0 (secure 0 0 0) 0x7fd8b408a090 con 0x7fd8b806da80 2024-05-20T18:35:26.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.493+0000 7fd8bd5d8640 1 --2- 172.21.15.5:0/715878386 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd89c06be30 0x7fd89c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:26.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.493+0000 7fd8bd5d8640 1 --2- 172.21.15.5:0/715878386 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd89c06be30 0x7fd89c06e340 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8034000 tx=0x7fd8a802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:26.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.497+0000 7fd8ae7fc640 1 -- 172.21.15.5:0/715878386 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd8b8160700 con 0x7fd8b806da80 2024-05-20T18:35:26.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:26.693+0000 7fd8c003d640 1 -- 172.21.15.5:0/715878386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fd8b8067530 con 0x7fd8b806da80 2024-05-20T18:35:27.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.377+0000 7fd8ae7fc640 1 -- 172.21.15.5:0/715878386 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v99) ==== 97+0+0 (secure 0 0 0) 0x7fd8b4058410 con 0x7fd8b806da80 2024-05-20T18:35:27.379 INFO:teuthology.orchestra.run.smithi005.stderr: removed all blocklist entries 2024-05-20T18:35:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.381+0000 7fd8c003d640 1 -- 172.21.15.5:0/715878386 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd89c06be30 msgr2=0x7fd89c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.381+0000 7fd8c003d640 1 --2- 172.21.15.5:0/715878386 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd89c06be30 0x7fd89c06e340 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8034000 tx=0x7fd8a802e000 comp rx=0 tx=0).stop 2024-05-20T18:35:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.381+0000 7fd8c003d640 1 -- 172.21.15.5:0/715878386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8b806da80 msgr2=0x7fd8b8155940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.381+0000 7fd8c003d640 1 --2- 172.21.15.5:0/715878386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8b806da80 0x7fd8b8155940 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fd8b400c430 tx=0x7fd8b400c8f0 comp rx=0 tx=0).stop 2024-05-20T18:35:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.381+0000 7fd8c003d640 1 -- 172.21.15.5:0/715878386 shutdown_connections 2024-05-20T18:35:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.381+0000 7fd8c003d640 1 --2- 172.21.15.5:0/715878386 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd89c06be30 0x7fd89c06e340 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.381+0000 7fd8c003d640 1 --2- 172.21.15.5:0/715878386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8b81602f0 0x7fd8b8158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.381+0000 7fd8c003d640 1 --2- 172.21.15.5:0/715878386 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd8b809aa30 0x7fd8b81582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.381+0000 7fd8c003d640 1 --2- 172.21.15.5:0/715878386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8b806da80 0x7fd8b8155940 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.381+0000 7fd8c003d640 1 -- 172.21.15.5:0/715878386 >> 172.21.15.5:0/715878386 conn(0x7fd8b80b6670 msgr2=0x7fd8b81568d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.381+0000 7fd8c003d640 1 -- 172.21.15.5:0/715878386 shutdown_connections 2024-05-20T18:35:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.381+0000 7fd8c003d640 1 -- 172.21.15.5:0/715878386 wait complete. 2024-05-20T18:35:27.399 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-20T18:35:27.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.521+0000 7f241b284640 1 Processor -- start 2024-05-20T18:35:27.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.521+0000 7f241b284640 1 -- start start 2024-05-20T18:35:27.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.522+0000 7f241b284640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f24140af030 0x7f24140af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:27.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.522+0000 7f241b284640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f241406c180 0x7f241406c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:27.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.522+0000 7f241b284640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24140af980 con 0x7f241415f700 2024-05-20T18:35:27.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.523+0000 7f241b284640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f241406bcf0 con 0x7f241406c180 2024-05-20T18:35:27.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.523+0000 7f241b284640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2414160430 con 0x7f24140af030 2024-05-20T18:35:27.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.523+0000 7f241881f640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f241415f700 0x7f241415fb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36104/0 (socket says 172.21.15.5:36104) 2024-05-20T18:35:27.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.523+0000 7f2419821640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f241406c180 0x7f241406c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:27.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.523+0000 7f241881f640 1 -- 172.21.15.5:0/2686764308 learned_addr learned my addr 172.21.15.5:0/2686764308 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:27.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.523+0000 7f2419020640 1 --2- 172.21.15.5:0/2686764308 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f24140af030 0x7f24140af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:27.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.523+0000 7f2419821640 1 -- 172.21.15.5:0/2686764308 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f24140af030 msgr2=0x7f24140af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:27.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.523+0000 7f2419821640 1 --2- 172.21.15.5:0/2686764308 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f24140af030 0x7f24140af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:27.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.523+0000 7f2419821640 1 -- 172.21.15.5:0/2686764308 >> v1:172.21.15.5:6789/0 conn(0x7f241415f700 legacy=0x7f241415fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:27.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.523+0000 7f2419821640 1 -- 172.21.15.5:0/2686764308 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241406b6a0 con 0x7f241406c180 2024-05-20T18:35:27.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.524+0000 7f2419821640 1 --2- 172.21.15.5:0/2686764308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f241406c180 0x7f241406c590 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f240c009d90 tx=0x7f240c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa5fe2a770c9015 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:27.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.524+0000 7f240bfff640 1 -- 172.21.15.5:0/2686764308 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c007740 con 0x7f241406c180 2024-05-20T18:35:27.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.524+0000 7f240bfff640 1 -- 172.21.15.5:0/2686764308 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f240c0078e0 con 0x7f241406c180 2024-05-20T18:35:27.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.524+0000 7f240bfff640 1 -- 172.21.15.5:0/2686764308 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c007cb0 con 0x7f241406c180 2024-05-20T18:35:27.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.525+0000 7f241b284640 1 -- 172.21.15.5:0/2686764308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f241406c180 msgr2=0x7f241406c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:27.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.525+0000 7f241b284640 1 --2- 172.21.15.5:0/2686764308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f241406c180 0x7f241406c590 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f240c009d90 tx=0x7f240c02a660 comp rx=0 tx=0).stop 2024-05-20T18:35:27.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.525+0000 7f241b284640 1 -- 172.21.15.5:0/2686764308 shutdown_connections 2024-05-20T18:35:27.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.525+0000 7f241b284640 1 --2- 172.21.15.5:0/2686764308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f241406c180 0x7f241406c590 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:27.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.525+0000 7f241b284640 1 --2- 172.21.15.5:0/2686764308 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f24140af030 0x7f24140af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:27.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.525+0000 7f241b284640 1 -- 172.21.15.5:0/2686764308 >> 172.21.15.5:0/2686764308 conn(0x7f24140b6620 msgr2=0x7f24140b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:27.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.525+0000 7f241b284640 1 -- 172.21.15.5:0/2686764308 shutdown_connections 2024-05-20T18:35:27.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.525+0000 7f241b284640 1 -- 172.21.15.5:0/2686764308 wait complete. 2024-05-20T18:35:27.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.527+0000 7f241b284640 1 Processor -- start 2024-05-20T18:35:27.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.527+0000 7f241b284640 1 -- start start 2024-05-20T18:35:27.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.528+0000 7f241b284640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f241406c180 0x7f2414157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:27.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.528+0000 7f241b284640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24140af030 0x7f241409c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:27.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.528+0000 7f241b284640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f241415f700 0x7f241409e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:27.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.528+0000 7f241b284640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f241406cad0 con 0x7f241406c180 2024-05-20T18:35:27.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.528+0000 7f241b284640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2414159aa0 con 0x7f24140af030 2024-05-20T18:35:27.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.528+0000 7f241b284640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f241406cc50 con 0x7f241415f700 2024-05-20T18:35:27.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.528+0000 7f2419020640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f241406c180 0x7f2414157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:27.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.528+0000 7f2419020640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f241406c180 0x7f2414157250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37796/0 (socket says 172.21.15.5:37796) 2024-05-20T18:35:27.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.528+0000 7f2419020640 1 -- 172.21.15.5:0/2736700899 learned_addr learned my addr 172.21.15.5:0/2736700899 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:27.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.528+0000 7f2419020640 1 -- 172.21.15.5:0/2736700899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f241415f700 msgr2=0x7f241409e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:27.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.528+0000 7f2419020640 1 --2- 172.21.15.5:0/2736700899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f241415f700 0x7f241409e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:27.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.528+0000 7f2419020640 1 -- 172.21.15.5:0/2736700899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24140af030 msgr2=0x7f241409c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:27.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.528+0000 7f2419020640 1 --2- 172.21.15.5:0/2736700899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24140af030 0x7f241409c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:27.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.528+0000 7f2419020640 1 -- 172.21.15.5:0/2736700899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241415a870 con 0x7f241406c180 2024-05-20T18:35:27.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.529+0000 7f2419020640 1 --2- 172.21.15.5:0/2736700899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f241406c180 0x7f2414157250 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f241000d5d0 tx=0x7f241000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:27.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.529+0000 7f2409ffb640 1 -- 172.21.15.5:0/2736700899 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2410004240 con 0x7f241406c180 2024-05-20T18:35:27.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.529+0000 7f2409ffb640 1 -- 172.21.15.5:0/2736700899 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24100043e0 con 0x7f241406c180 2024-05-20T18:35:27.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.529+0000 7f241b284640 1 -- 172.21.15.5:0/2736700899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f241409f500 con 0x7f241406c180 2024-05-20T18:35:27.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.529+0000 7f2409ffb640 1 -- 172.21.15.5:0/2736700899 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2410003830 con 0x7f241406c180 2024-05-20T18:35:27.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.529+0000 7f241b284640 1 -- 172.21.15.5:0/2736700899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f241409f7e0 con 0x7f241406c180 2024-05-20T18:35:27.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.530+0000 7f241b284640 1 -- 172.21.15.5:0/2736700899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2414075b00 con 0x7f241406c180 2024-05-20T18:35:27.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.531+0000 7f2409ffb640 1 -- 172.21.15.5:0/2736700899 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f24100039d0 con 0x7f241406c180 2024-05-20T18:35:27.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.531+0000 7f2409ffb640 1 --2- 172.21.15.5:0/2736700899 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f23f806be60 0x7f23f806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:27.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.532+0000 7f241881f640 1 --2- 172.21.15.5:0/2736700899 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f23f806be60 0x7f23f806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:27.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.532+0000 7f2409ffb640 1 -- 172.21.15.5:0/2736700899 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5249+0+0 (secure 0 0 0) 0x7f241008ae70 con 0x7f241406c180 2024-05-20T18:35:27.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.532+0000 7f241881f640 1 --2- 172.21.15.5:0/2736700899 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f23f806be60 0x7f23f806e370 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2404000f60 tx=0x7f2404007a80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:27.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.536+0000 7f2409ffb640 1 -- 172.21.15.5:0/2736700899 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2410059a70 con 0x7f241406c180 2024-05-20T18:35:27.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.788+0000 7f241b284640 1 -- 172.21.15.5:0/2736700899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f24140d1630 con 0x7f241406c180 2024-05-20T18:35:27.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.789+0000 7f2409ffb640 1 -- 172.21.15.5:0/2736700899 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7f2410059370 con 0x7f241406c180 2024-05-20T18:35:27.790 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:27.790 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:35:27.790 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_dump": [ 2024-05-20T18:35:27.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:35:27.790 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.a", 2024-05-20T18:35:27.790 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtm9lSeCBAA/PpiEqojk6ueJo60tU4/iA==", 2024-05-20T18:35:27.791 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:35:27.791 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:35:27.791 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:35:27.791 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:35:27.791 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:35:27.791 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:35:27.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:35:27.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:35:27.791 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.b", 2024-05-20T18:35:27.791 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmdov0FRAACkWAR/0zaCDjgnkHnoHwgw==", 2024-05-20T18:35:27.791 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:35:27.791 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:35:27.791 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:35:27.792 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:35:27.792 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:35:27.792 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:35:27.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:35:27.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:35:27.792 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.c", 2024-05-20T18:35:27.792 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmpEBlEBAAdgxDXj9x38VSOizUgdFpYA==", 2024-05-20T18:35:27.792 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:35:27.792 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:35:27.792 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:35:27.792 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:35:27.792 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:35:27.792 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:35:27.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:35:27.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:35:27.793 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.d", 2024-05-20T18:35:27.793 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtm0Gy9HRAAAkPqcBLwxTwuWmCi7aad1A==", 2024-05-20T18:35:27.793 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:35:27.793 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:35:27.793 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:35:27.793 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:35:27.793 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:35:27.793 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:35:27.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:35:27.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:35:27.793 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.0", 2024-05-20T18:35:27.793 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDHlUtmWFGzBRAA81qN37dlHf6QDZ1FgwbJ9w==", 2024-05-20T18:35:27.793 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:35:27.793 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:35:27.794 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:35:27.794 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:35:27.794 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:35:27.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:35:27.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:35:27.794 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.1", 2024-05-20T18:35:27.794 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDKlUtmBMKsBBAAD4tTTvetME6oxwugTubSKQ==", 2024-05-20T18:35:27.794 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:35:27.794 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:35:27.794 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:35:27.794 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:35:27.794 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:35:27.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:35:27.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:35:27.795 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.2", 2024-05-20T18:35:27.795 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDNlUtm4Rm/AxAAIGWTUxb+LII09Rd+h5cJ8Q==", 2024-05-20T18:35:27.795 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:35:27.795 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:35:27.795 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:35:27.795 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:35:27.795 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:35:27.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:35:27.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:35:27.795 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.3", 2024-05-20T18:35:27.795 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDQlUtmWVxZAxAAB/w6k4WJ58ulEr9zatCRfw==", 2024-05-20T18:35:27.795 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:35:27.795 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:35:27.796 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:35:27.796 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:35:27.796 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:35:27.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:35:27.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:35:27.796 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.4", 2024-05-20T18:35:27.796 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDWlUtmIyokCRAAolOiYVCYaNTQZb1PjtR6fg==", 2024-05-20T18:35:27.796 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:35:27.796 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:35:27.796 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:35:27.796 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:35:27.796 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:35:27.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:35:27.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:35:27.796 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.5", 2024-05-20T18:35:27.797 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZlUtmSf6mCBAAw73nwq/zzt9kIXZymZh93A==", 2024-05-20T18:35:27.797 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:35:27.797 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:35:27.797 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:35:27.797 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:35:27.797 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:35:27.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:35:27.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:35:27.797 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.6", 2024-05-20T18:35:27.797 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDclUtmwwhGCBAA94OBEtj9h7cmCAddYMonaA==", 2024-05-20T18:35:27.797 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:35:27.797 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:35:27.797 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:35:27.797 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:35:27.798 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:35:27.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:35:27.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:35:27.798 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.7", 2024-05-20T18:35:27.798 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDflUtm9wefBxAAvCZTBzRu05koFhc+pH5I8w==", 2024-05-20T18:35:27.798 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:35:27.798 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:35:27.798 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:35:27.798 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:35:27.798 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:35:27.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:35:27.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:35:27.798 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.0", 2024-05-20T18:35:27.798 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmevaaIhAATIwyXzr0vGT38ZmcMqR2gA==", 2024-05-20T18:35:27.799 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:35:27.799 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:35:27.799 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow r", 2024-05-20T18:35:27.799 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-20T18:35:27.799 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:35:27.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:35:27.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:35:27.799 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.1", 2024-05-20T18:35:27.799 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmK7NfKhAADaXl0uVP1Q1gUOlPsstnzg==", 2024-05-20T18:35:27.799 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:35:27.799 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:35:27.799 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-20T18:35:27.799 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-20T18:35:27.799 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-20T18:35:27.800 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:35:27.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:35:27.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:35:27.800 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.2", 2024-05-20T18:35:27.800 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmMJF1JhAAeM+HseqvgvKgG8EnIsZuhg==", 2024-05-20T18:35:27.800 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:35:27.800 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:35:27.800 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-20T18:35:27.800 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-20T18:35:27.800 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-20T18:35:27.800 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:35:27.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:35:27.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:35:27.800 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.3", 2024-05-20T18:35:27.801 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmodA7LhAAUI2v0npB6HHs2n1g3KdTXA==", 2024-05-20T18:35:27.801 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:35:27.801 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:35:27.801 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-20T18:35:27.801 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-20T18:35:27.801 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-20T18:35:27.801 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:35:27.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:35:27.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:35:27.801 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.admin", 2024-05-20T18:35:27.801 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDElUtmZacGJRAAoyr9XNPBr9r0DSU53qCHSQ==", 2024-05-20T18:35:27.801 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:35:27.801 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-20T18:35:27.801 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:35:27.801 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:35:27.802 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:35:27.802 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:35:27.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:35:27.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:35:27.802 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.x", 2024-05-20T18:35:27.802 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmPgq0BBAAAljNeQDsSau0wH2Lh2KdBw==", 2024-05-20T18:35:27.802 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:35:27.802 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-20T18:35:27.802 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-05-20T18:35:27.802 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:35:27.802 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:35:27.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:35:27.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:35:27.803 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.y", 2024-05-20T18:35:27.803 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDElUtm6Xv3OhAA0doOWTC73jy5rOYm7Vf68Q==", 2024-05-20T18:35:27.803 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:35:27.803 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-20T18:35:27.803 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-05-20T18:35:27.803 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:35:27.803 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:35:27.803 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:35:27.803 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-20T18:35:27.803 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:35:27.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.791+0000 7f241b284640 1 -- 172.21.15.5:0/2736700899 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f23f806be60 msgr2=0x7f23f806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:27.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.791+0000 7f241b284640 1 --2- 172.21.15.5:0/2736700899 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f23f806be60 0x7f23f806e370 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2404000f60 tx=0x7f2404007a80 comp rx=0 tx=0).stop 2024-05-20T18:35:27.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.791+0000 7f241b284640 1 -- 172.21.15.5:0/2736700899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f241406c180 msgr2=0x7f2414157250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:27.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.791+0000 7f241b284640 1 --2- 172.21.15.5:0/2736700899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f241406c180 0x7f2414157250 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f241000d5d0 tx=0x7f241000da90 comp rx=0 tx=0).stop 2024-05-20T18:35:27.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.791+0000 7f241b284640 1 -- 172.21.15.5:0/2736700899 shutdown_connections 2024-05-20T18:35:27.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.791+0000 7f241b284640 1 --2- 172.21.15.5:0/2736700899 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f23f806be60 0x7f23f806e370 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:27.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.791+0000 7f241b284640 1 --2- 172.21.15.5:0/2736700899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f241415f700 0x7f241409e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:27.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.791+0000 7f241b284640 1 --2- 172.21.15.5:0/2736700899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24140af030 0x7f241409c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:27.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.791+0000 7f241b284640 1 --2- 172.21.15.5:0/2736700899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f241406c180 0x7f2414157250 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:27.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.791+0000 7f241b284640 1 -- 172.21.15.5:0/2736700899 >> 172.21.15.5:0/2736700899 conn(0x7f24140b6620 msgr2=0x7f24140702b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:27.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.791+0000 7f241b284640 1 -- 172.21.15.5:0/2736700899 shutdown_connections 2024-05-20T18:35:27.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.791+0000 7f241b284640 1 -- 172.21.15.5:0/2736700899 wait complete. 2024-05-20T18:35:27.806 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:35:27.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.920+0000 7f5975efe640 1 Processor -- start 2024-05-20T18:35:27.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.920+0000 7f5975efe640 1 -- start start 2024-05-20T18:35:27.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.921+0000 7f5975efe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f597015d530 0x7f597015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:27.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.921+0000 7f5975efe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f59700b1700 0x7f59700b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:27.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.921+0000 7f5975efe640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59700cd580 con 0x7f5970070850 2024-05-20T18:35:27.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.921+0000 7f5975efe640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f59700ce190 con 0x7f59700b1700 2024-05-20T18:35:27.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.921+0000 7f5975efe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f59700cd860 con 0x7f597015d530 2024-05-20T18:35:27.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.921+0000 7f596ffff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5970070850 0x7f5970070c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36108/0 (socket says 172.21.15.5:36108) 2024-05-20T18:35:27.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.921+0000 7f596ffff640 1 -- 172.21.15.5:0/3817041717 learned_addr learned my addr 172.21.15.5:0/3817041717 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:27.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.922+0000 7f596effd640 1 -- 172.21.15.5:0/3817041717 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1346782873 0 0) 0x7f59700cd580 con 0x7f5970070850 2024-05-20T18:35:27.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.922+0000 7f5967fff640 1 --2- 172.21.15.5:0/3817041717 >> v2:172.21.15.63:3300/0 conn(0x7f59700b1700 0x7f59700b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:27.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.922+0000 7f596f7fe640 1 --2- 172.21.15.5:0/3817041717 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f597015d530 0x7f597015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:27.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.922+0000 7f596effd640 1 -- 172.21.15.5:0/3817041717 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f595c003520 con 0x7f5970070850 2024-05-20T18:35:27.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.922+0000 7f596effd640 1 -- 172.21.15.5:0/3817041717 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 151648658 0 0) 0x7f595c003520 con 0x7f5970070850 2024-05-20T18:35:27.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.922+0000 7f596effd640 1 -- 172.21.15.5:0/3817041717 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f597015d530 msgr2=0x7f597015aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:27.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.922+0000 7f596effd640 1 --2- 172.21.15.5:0/3817041717 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f597015d530 0x7f597015aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:27.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.922+0000 7f596effd640 1 -- 172.21.15.5:0/3817041717 >> v2:172.21.15.63:3300/0 conn(0x7f59700b1700 msgr2=0x7f59700b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:27.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.922+0000 7f596effd640 1 --2- 172.21.15.5:0/3817041717 >> v2:172.21.15.63:3300/0 conn(0x7f59700b1700 0x7f59700b1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:27.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.922+0000 7f596effd640 1 -- 172.21.15.5:0/3817041717 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59700cdb40 con 0x7f5970070850 2024-05-20T18:35:27.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.922+0000 7f596effd640 1 -- 172.21.15.5:0/3817041717 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f59680019f0 con 0x7f5970070850 2024-05-20T18:35:27.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.922+0000 7f596effd640 1 -- 172.21.15.5:0/3817041717 >> v1:172.21.15.5:6789/0 conn(0x7f5970070850 legacy=0x7f5970070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:27.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.922+0000 7f596effd640 1 --2- 172.21.15.5:0/3817041717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f595c0039b0 0x7f595c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:27.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.922+0000 7f596effd640 1 --2- 172.21.15.5:0/3817041717 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f597015d530 0x7f595c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:27.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.922+0000 7f596effd640 1 --2- 172.21.15.5:0/3817041717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f595c004d40 0x7f595c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:27.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.922+0000 7f596effd640 1 -- 172.21.15.5:0/3817041717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f59700ce190 con 0x7f595c004d40 2024-05-20T18:35:27.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.922+0000 7f596effd640 1 -- 172.21.15.5:0/3817041717 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f59700cd860 con 0x7f595c0039b0 2024-05-20T18:35:27.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.922+0000 7f596effd640 1 -- 172.21.15.5:0/3817041717 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f595c00b9c0 con 0x7f597015d530 2024-05-20T18:35:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.922+0000 7f596ffff640 1 --2- 172.21.15.5:0/3817041717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f595c004d40 0x7f595c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.922+0000 7f596f7fe640 1 --2- 172.21.15.5:0/3817041717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f595c0039b0 0x7f595c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.922+0000 7f5967fff640 1 --2- 172.21.15.5:0/3817041717 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f597015d530 0x7f595c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.923+0000 7f596ffff640 1 -- 172.21.15.5:0/3817041717 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f597015d530 msgr2=0x7f595c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.923+0000 7f596ffff640 1 --2- 172.21.15.5:0/3817041717 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f597015d530 0x7f595c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.923+0000 7f596ffff640 1 -- 172.21.15.5:0/3817041717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f595c0039b0 msgr2=0x7f595c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.923+0000 7f596ffff640 1 --2- 172.21.15.5:0/3817041717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f595c0039b0 0x7f595c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.923+0000 7f596ffff640 1 -- 172.21.15.5:0/3817041717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595c003520 con 0x7f595c004d40 2024-05-20T18:35:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.923+0000 7f596ffff640 1 --2- 172.21.15.5:0/3817041717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f595c004d40 0x7f595c005150 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f59700d3610 tx=0x7f595c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ecde70d64c78d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.924+0000 7f596effd640 1 -- 172.21.15.5:0/3817041717 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596800a4c0 con 0x7f595c004d40 2024-05-20T18:35:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.924+0000 7f596effd640 1 -- 172.21.15.5:0/3817041717 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f596802f070 con 0x7f595c004d40 2024-05-20T18:35:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.924+0000 7f596effd640 1 -- 172.21.15.5:0/3817041717 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596802bc30 con 0x7f595c004d40 2024-05-20T18:35:27.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.924+0000 7f5975efe640 1 -- 172.21.15.5:0/3817041717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f595c004d40 msgr2=0x7f595c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:27.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.924+0000 7f5975efe640 1 --2- 172.21.15.5:0/3817041717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f595c004d40 0x7f595c005150 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f59700d3610 tx=0x7f595c005830 comp rx=0 tx=0).stop 2024-05-20T18:35:27.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.924+0000 7f596ffff640 1 -- 172.21.15.5:0/3817041717 reap_dead start 2024-05-20T18:35:27.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.924+0000 7f5975efe640 1 -- 172.21.15.5:0/3817041717 shutdown_connections 2024-05-20T18:35:27.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.924+0000 7f5975efe640 1 -- 172.21.15.5:0/3817041717 >> 172.21.15.5:0/3817041717 conn(0x7f59700d29d0 msgr2=0x7f59700d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:27.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.924+0000 7f5975efe640 1 -- 172.21.15.5:0/3817041717 shutdown_connections 2024-05-20T18:35:27.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.924+0000 7f5975efe640 1 -- 172.21.15.5:0/3817041717 wait complete. 2024-05-20T18:35:27.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.926+0000 7f5975efe640 1 Processor -- start 2024-05-20T18:35:27.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.926+0000 7f5975efe640 1 -- start start 2024-05-20T18:35:27.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.928+0000 7f5975efe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f595c004d40 0x7f59700c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:27.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.928+0000 7f5975efe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59700cbf20 0x7f5970148d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:27.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.928+0000 7f5975efe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59700b37f0 0x7f59700b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:27.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.928+0000 7f5975efe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f59700b2050 con 0x7f59700cbf20 2024-05-20T18:35:27.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.928+0000 7f5975efe640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f59700cd230 con 0x7f595c004d40 2024-05-20T18:35:27.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.928+0000 7f5975efe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5970071370 con 0x7f59700b37f0 2024-05-20T18:35:27.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.928+0000 7f5967fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59700cbf20 0x7f5970148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:27.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.928+0000 7f5967fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59700cbf20 0x7f5970148d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37812/0 (socket says 172.21.15.5:37812) 2024-05-20T18:35:27.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.928+0000 7f5967fff640 1 -- 172.21.15.5:0/298182126 learned_addr learned my addr 172.21.15.5:0/298182126 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:27.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.928+0000 7f596ffff640 1 --2- 172.21.15.5:0/298182126 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59700b37f0 0x7f59700b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:27.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.928+0000 7f5967fff640 1 -- 172.21.15.5:0/298182126 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59700b37f0 msgr2=0x7f59700b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:27.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.928+0000 7f596f7fe640 1 --2- 172.21.15.5:0/298182126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f595c004d40 0x7f59700c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:27.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.928+0000 7f5967fff640 1 --2- 172.21.15.5:0/298182126 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59700b37f0 0x7f59700b3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:27.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.928+0000 7f5967fff640 1 -- 172.21.15.5:0/298182126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f595c004d40 msgr2=0x7f59700c9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:27.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.928+0000 7f5967fff640 1 --2- 172.21.15.5:0/298182126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f595c004d40 0x7f59700c9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.928+0000 7f5967fff640 1 -- 172.21.15.5:0/298182126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59700b4140 con 0x7f59700cbf20 2024-05-20T18:35:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.928+0000 7f596f7fe640 1 --2- 172.21.15.5:0/298182126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f595c004d40 0x7f59700c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.928+0000 7f596ffff640 1 --2- 172.21.15.5:0/298182126 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59700b37f0 0x7f59700b3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.928+0000 7f5967fff640 1 --2- 172.21.15.5:0/298182126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59700cbf20 0x7f5970148d40 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f5958011990 tx=0x7f5958011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.928+0000 7f596cff9640 1 -- 172.21.15.5:0/298182126 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5958019070 con 0x7f59700cbf20 2024-05-20T18:35:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.928+0000 7f596cff9640 1 -- 172.21.15.5:0/298182126 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5958004240 con 0x7f59700cbf20 2024-05-20T18:35:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.928+0000 7f596cff9640 1 -- 172.21.15.5:0/298182126 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595800f860 con 0x7f59700cbf20 2024-05-20T18:35:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.928+0000 7f5975efe640 1 -- 172.21.15.5:0/298182126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59700cc6c0 con 0x7f59700cbf20 2024-05-20T18:35:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.928+0000 7f5975efe640 1 -- 172.21.15.5:0/298182126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59700c6c90 con 0x7f59700cbf20 2024-05-20T18:35:27.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.929+0000 7f5975efe640 1 -- 172.21.15.5:0/298182126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59700c67a0 con 0x7f59700cbf20 2024-05-20T18:35:27.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.930+0000 7f596cff9640 1 -- 172.21.15.5:0/298182126 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f595801c070 con 0x7f59700cbf20 2024-05-20T18:35:27.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.931+0000 7f596cff9640 1 --2- 172.21.15.5:0/298182126 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f595006be60 0x7f595006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:27.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.931+0000 7f596cff9640 1 -- 172.21.15.5:0/298182126 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5249+0+0 (secure 0 0 0) 0x7f595808b040 con 0x7f59700cbf20 2024-05-20T18:35:27.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.931+0000 7f596f7fe640 1 --2- 172.21.15.5:0/298182126 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f595006be60 0x7f595006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:27.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.931+0000 7f596f7fe640 1 --2- 172.21.15.5:0/298182126 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f595006be60 0x7f595006e370 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f59700d29a0 tx=0x7f5960008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:27.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:27.935+0000 7f596cff9640 1 -- 172.21.15.5:0/298182126 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f59700b4140 con 0x7f59700cbf20 2024-05-20T18:35:28.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.119+0000 7f5975efe640 1 -- 172.21.15.5:0/298182126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5970075b00 con 0x7f59700cbf20 2024-05-20T18:35:28.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.120+0000 7f596cff9640 1 -- 172.21.15.5:0/298182126 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+13046 (secure 0 0 0) 0x7f5958059c40 con 0x7f59700cbf20 2024-05-20T18:35:28.121 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:28.121 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":101,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:35:27.387842+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:35:28.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.122+0000 7f5975efe640 1 -- 172.21.15.5:0/298182126 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f595006be60 msgr2=0x7f595006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:28.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.122+0000 7f5975efe640 1 --2- 172.21.15.5:0/298182126 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f595006be60 0x7f595006e370 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f59700d29a0 tx=0x7f5960008040 comp rx=0 tx=0).stop 2024-05-20T18:35:28.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.122+0000 7f5975efe640 1 -- 172.21.15.5:0/298182126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59700cbf20 msgr2=0x7f5970148d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:28.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.122+0000 7f5975efe640 1 --2- 172.21.15.5:0/298182126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59700cbf20 0x7f5970148d40 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f5958011990 tx=0x7f5958011e50 comp rx=0 tx=0).stop 2024-05-20T18:35:28.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.122+0000 7f5975efe640 1 -- 172.21.15.5:0/298182126 shutdown_connections 2024-05-20T18:35:28.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.122+0000 7f5975efe640 1 --2- 172.21.15.5:0/298182126 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f595006be60 0x7f595006e370 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:28.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.122+0000 7f5975efe640 1 --2- 172.21.15.5:0/298182126 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59700b37f0 0x7f59700b3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:28.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.122+0000 7f5975efe640 1 --2- 172.21.15.5:0/298182126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59700cbf20 0x7f5970148d40 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:28.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.122+0000 7f5975efe640 1 --2- 172.21.15.5:0/298182126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f595c004d40 0x7f59700c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:28.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.122+0000 7f5975efe640 1 -- 172.21.15.5:0/298182126 >> 172.21.15.5:0/298182126 conn(0x7f59700d29d0 msgr2=0x7f59700c5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:28.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.122+0000 7f5975efe640 1 -- 172.21.15.5:0/298182126 shutdown_connections 2024-05-20T18:35:28.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.122+0000 7f5975efe640 1 -- 172.21.15.5:0/298182126 wait complete. 2024-05-20T18:35:28.135 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:35:28.135 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:35:28.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.252+0000 7f3aee3ea640 1 Processor -- start 2024-05-20T18:35:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.252+0000 7f3aee3ea640 1 -- start start 2024-05-20T18:35:28.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.254+0000 7f3aee3ea640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3ae806d3d0 0x7f3ae806d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:28.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.254+0000 7f3aee3ea640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ae806dd20 con 0x7f3ae80b8190 2024-05-20T18:35:28.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.254+0000 7f3aee3ea640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ae80d0630 con 0x7f3ae8162820 2024-05-20T18:35:28.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.254+0000 7f3aee3ea640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f3ae80d39c0 con 0x7f3ae806d3d0 2024-05-20T18:35:28.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.254+0000 7f3aec987640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3ae80b8190 0x7f3ae80b85a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36112/0 (socket says 172.21.15.5:36112) 2024-05-20T18:35:28.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.254+0000 7f3aec987640 1 -- 172.21.15.5:0/1072721279 learned_addr learned my addr 172.21.15.5:0/1072721279 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:28.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.254+0000 7f3ae77fe640 1 --2- 172.21.15.5:0/1072721279 >> v2:172.21.15.63:3300/0 conn(0x7f3ae806d3d0 0x7f3ae806d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:28.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.254+0000 7f3ae6ffd640 1 -- 172.21.15.5:0/1072721279 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2002759882 0 0) 0x7f3ae0001490 con 0x7f3ae80b8190 2024-05-20T18:35:28.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.254+0000 7f3ae6ffd640 1 -- 172.21.15.5:0/1072721279 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ad0003520 con 0x7f3ae80b8190 2024-05-20T18:35:28.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.254+0000 7f3ae77fe640 1 -- 172.21.15.5:0/1072721279 >> v1:172.21.15.63:6789/0 conn(0x7f3ae8162820 legacy=0x7f3ae80b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:28.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.255+0000 7f3ae77fe640 1 -- 172.21.15.5:0/1072721279 >> v1:172.21.15.5:6789/0 conn(0x7f3ae80b8190 legacy=0x7f3ae80b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:28.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.255+0000 7f3ae77fe640 1 -- 172.21.15.5:0/1072721279 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae80d3ca0 con 0x7f3ae806d3d0 2024-05-20T18:35:28.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.255+0000 7f3ae77fe640 1 --2- 172.21.15.5:0/1072721279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ae806d3d0 0x7f3ae806d7e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3adc0097a0 tx=0x7f3adc02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42f95f6ef7ecae62 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:28.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.256+0000 7f3ae6ffd640 1 -- 172.21.15.5:0/1072721279 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3adc037070 con 0x7f3ae806d3d0 2024-05-20T18:35:28.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.256+0000 7f3ae6ffd640 1 -- 172.21.15.5:0/1072721279 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3adc032440 con 0x7f3ae806d3d0 2024-05-20T18:35:28.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.256+0000 7f3ae6ffd640 1 -- 172.21.15.5:0/1072721279 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3adc0327f0 con 0x7f3ae806d3d0 2024-05-20T18:35:28.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.256+0000 7f3aee3ea640 1 -- 172.21.15.5:0/1072721279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ae806d3d0 msgr2=0x7f3ae806d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:28.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.256+0000 7f3aee3ea640 1 --2- 172.21.15.5:0/1072721279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ae806d3d0 0x7f3ae806d7e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3adc0097a0 tx=0x7f3adc02a230 comp rx=0 tx=0).stop 2024-05-20T18:35:28.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.256+0000 7f3aee3ea640 1 -- 172.21.15.5:0/1072721279 shutdown_connections 2024-05-20T18:35:28.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.256+0000 7f3aee3ea640 1 --2- 172.21.15.5:0/1072721279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ae806d3d0 0x7f3ae806d7e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:28.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.256+0000 7f3aee3ea640 1 -- 172.21.15.5:0/1072721279 >> 172.21.15.5:0/1072721279 conn(0x7f3ae80d4bd0 msgr2=0x7f3ae80d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:28.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.256+0000 7f3aee3ea640 1 -- 172.21.15.5:0/1072721279 shutdown_connections 2024-05-20T18:35:28.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.256+0000 7f3aee3ea640 1 -- 172.21.15.5:0/1072721279 wait complete. 2024-05-20T18:35:28.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.258+0000 7f3aee3ea640 1 Processor -- start 2024-05-20T18:35:28.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.258+0000 7f3aee3ea640 1 -- start start 2024-05-20T18:35:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.260+0000 7f3aee3ea640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ae806d3d0 0x7f3ae815bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.260+0000 7f3aee3ea640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3ae80b8190 0x7f3ae81e7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.260+0000 7f3aee3ea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ae8162820 0x7f3ae81e7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.260+0000 7f3aee3ea640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae80d0300 con 0x7f3ae8162820 2024-05-20T18:35:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.260+0000 7f3aee3ea640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae80d42e0 con 0x7f3ae806d3d0 2024-05-20T18:35:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.260+0000 7f3aee3ea640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3ae80b4050 con 0x7f3ae80b8190 2024-05-20T18:35:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.260+0000 7f3aec987640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ae8162820 0x7f3ae81e7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.260+0000 7f3aec987640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ae8162820 0x7f3ae81e7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37828/0 (socket says 172.21.15.5:37828) 2024-05-20T18:35:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.260+0000 7f3aec987640 1 -- 172.21.15.5:0/2336912986 learned_addr learned my addr 172.21.15.5:0/2336912986 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.260+0000 7f3ae7fff640 1 --2- 172.21.15.5:0/2336912986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ae806d3d0 0x7f3ae815bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.260+0000 7f3ae77fe640 1 --2- 172.21.15.5:0/2336912986 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3ae80b8190 0x7f3ae81e7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.260+0000 7f3aec987640 1 -- 172.21.15.5:0/2336912986 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3ae80b8190 msgr2=0x7f3ae81e7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.260+0000 7f3aec987640 1 --2- 172.21.15.5:0/2336912986 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3ae80b8190 0x7f3ae81e7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.260+0000 7f3aec987640 1 -- 172.21.15.5:0/2336912986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ae806d3d0 msgr2=0x7f3ae815bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.260+0000 7f3aec987640 1 --2- 172.21.15.5:0/2336912986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ae806d3d0 0x7f3ae815bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.260+0000 7f3aec987640 1 -- 172.21.15.5:0/2336912986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae80cdf20 con 0x7f3ae8162820 2024-05-20T18:35:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.260+0000 7f3aec987640 1 --2- 172.21.15.5:0/2336912986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ae8162820 0x7f3ae81e7e80 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f3ae000cb10 tx=0x7f3ae000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.260+0000 7f3ae4ff9640 1 -- 172.21.15.5:0/2336912986 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0017070 con 0x7f3ae8162820 2024-05-20T18:35:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.260+0000 7f3ae4ff9640 1 -- 172.21.15.5:0/2336912986 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae000ac10 con 0x7f3ae8162820 2024-05-20T18:35:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.260+0000 7f3aee3ea640 1 -- 172.21.15.5:0/2336912986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae80cee20 con 0x7f3ae8162820 2024-05-20T18:35:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.260+0000 7f3ae4ff9640 1 -- 172.21.15.5:0/2336912986 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0012400 con 0x7f3ae8162820 2024-05-20T18:35:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.261+0000 7f3aee3ea640 1 -- 172.21.15.5:0/2336912986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae80cca10 con 0x7f3ae8162820 2024-05-20T18:35:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.262+0000 7f3aee3ea640 1 -- 172.21.15.5:0/2336912986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae815fe00 con 0x7f3ae8162820 2024-05-20T18:35:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.262+0000 7f3ae4ff9640 1 -- 172.21.15.5:0/2336912986 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3ae001a070 con 0x7f3ae8162820 2024-05-20T18:35:28.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.263+0000 7f3ae4ff9640 1 --2- 172.21.15.5:0/2336912986 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3ad406be30 0x7f3ad406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:28.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.263+0000 7f3ae4ff9640 1 -- 172.21.15.5:0/2336912986 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5249+0+0 (secure 0 0 0) 0x7f3ae008aa60 con 0x7f3ae8162820 2024-05-20T18:35:28.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.263+0000 7f3ae7fff640 1 --2- 172.21.15.5:0/2336912986 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3ad406be30 0x7f3ad406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:28.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.263+0000 7f3ae7fff640 1 --2- 172.21.15.5:0/2336912986 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3ad406be30 0x7f3ad406e340 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8000f60 tx=0x7f3ad8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:28.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.267+0000 7f3ae4ff9640 1 -- 172.21.15.5:0/2336912986 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3ae815fe00 con 0x7f3ae8162820 2024-05-20T18:35:28.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.451+0000 7f3aee3ea640 1 -- 172.21.15.5:0/2336912986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3ae8075b00 con 0x7f3ae8162820 2024-05-20T18:35:28.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.451+0000 7f3ae4ff9640 1 -- 172.21.15.5:0/2336912986 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) ==== 93+0+10 (secure 0 0 0) 0x7f3ae806dd20 con 0x7f3ae8162820 2024-05-20T18:35:28.453 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:35:28.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.453+0000 7f3aee3ea640 1 -- 172.21.15.5:0/2336912986 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3ad406be30 msgr2=0x7f3ad406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:28.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.453+0000 7f3aee3ea640 1 --2- 172.21.15.5:0/2336912986 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3ad406be30 0x7f3ad406e340 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8000f60 tx=0x7f3ad8009040 comp rx=0 tx=0).stop 2024-05-20T18:35:28.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.453+0000 7f3aee3ea640 1 -- 172.21.15.5:0/2336912986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ae8162820 msgr2=0x7f3ae81e7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:28.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.453+0000 7f3aee3ea640 1 --2- 172.21.15.5:0/2336912986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ae8162820 0x7f3ae81e7e80 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f3ae000cb10 tx=0x7f3ae000a060 comp rx=0 tx=0).stop 2024-05-20T18:35:28.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.453+0000 7f3aee3ea640 1 -- 172.21.15.5:0/2336912986 shutdown_connections 2024-05-20T18:35:28.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.453+0000 7f3aee3ea640 1 --2- 172.21.15.5:0/2336912986 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3ad406be30 0x7f3ad406e340 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:28.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.453+0000 7f3aee3ea640 1 --2- 172.21.15.5:0/2336912986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ae8162820 0x7f3ae81e7e80 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:28.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.453+0000 7f3aee3ea640 1 --2- 172.21.15.5:0/2336912986 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3ae80b8190 0x7f3ae81e7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:28.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.453+0000 7f3aee3ea640 1 --2- 172.21.15.5:0/2336912986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ae806d3d0 0x7f3ae815bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:28.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.453+0000 7f3aee3ea640 1 -- 172.21.15.5:0/2336912986 >> 172.21.15.5:0/2336912986 conn(0x7f3ae80d4bd0 msgr2=0x7f3ae815f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:28.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.453+0000 7f3aee3ea640 1 -- 172.21.15.5:0/2336912986 shutdown_connections 2024-05-20T18:35:28.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.453+0000 7f3aee3ea640 1 -- 172.21.15.5:0/2336912986 wait complete. 2024-05-20T18:35:28.466 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-20T18:35:28.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.580+0000 7f09ab59e640 1 Processor -- start 2024-05-20T18:35:28.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.580+0000 7f09ab59e640 1 -- start start 2024-05-20T18:35:28.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.581+0000 7f09ab59e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09ac0b7060 0x7f09ac162aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:28.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.581+0000 7f09ab59e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f09ac162fe0 0x7f09ac0b7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:28.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.581+0000 7f09ab59e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09ac06cff0 con 0x7f09ac0d3e20 2024-05-20T18:35:28.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.581+0000 7f09ab59e640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f09ac1633f0 con 0x7f09ac162fe0 2024-05-20T18:35:28.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.581+0000 7f09ab59e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f09ac163680 con 0x7f09ac0b7060 2024-05-20T18:35:28.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.582+0000 7f09aa59c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f09ac0d3e20 0x7f09ac0b6950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36118/0 (socket says 172.21.15.5:36118) 2024-05-20T18:35:28.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.582+0000 7f09aa59c640 1 -- 172.21.15.5:0/2974032492 learned_addr learned my addr 172.21.15.5:0/2974032492 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:28.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.582+0000 7f09aad9d640 1 --2- 172.21.15.5:0/2974032492 >> v2:172.21.15.63:3300/0 conn(0x7f09ac162fe0 0x7f09ac0b7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:28.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.582+0000 7f09a9d9b640 1 --2- 172.21.15.5:0/2974032492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09ac0b7060 0x7f09ac162aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:28.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.582+0000 7f09a959a640 1 -- 172.21.15.5:0/2974032492 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 882084778 0 0) 0x7f099c001110 con 0x7f09ac0d3e20 2024-05-20T18:35:28.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.582+0000 7f09a959a640 1 -- 172.21.15.5:0/2974032492 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0994003520 con 0x7f09ac0d3e20 2024-05-20T18:35:28.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.582+0000 7f09a959a640 1 -- 172.21.15.5:0/2974032492 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1802193686 0 0) 0x7f0994003520 con 0x7f09ac0d3e20 2024-05-20T18:35:28.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.582+0000 7f09a959a640 1 -- 172.21.15.5:0/2974032492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09ac0b7060 msgr2=0x7f09ac162aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:28.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.582+0000 7f09a959a640 1 --2- 172.21.15.5:0/2974032492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09ac0b7060 0x7f09ac162aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:28.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.582+0000 7f09a959a640 1 -- 172.21.15.5:0/2974032492 >> v2:172.21.15.63:3300/0 conn(0x7f09ac162fe0 msgr2=0x7f09ac0b7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:28.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.582+0000 7f09a959a640 1 --2- 172.21.15.5:0/2974032492 >> v2:172.21.15.63:3300/0 conn(0x7f09ac162fe0 0x7f09ac0b7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:28.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.582+0000 7f09a959a640 1 -- 172.21.15.5:0/2974032492 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09ac0d3340 con 0x7f09ac0d3e20 2024-05-20T18:35:28.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.582+0000 7f09a959a640 1 -- 172.21.15.5:0/2974032492 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f099c001840 con 0x7f09ac0d3e20 2024-05-20T18:35:28.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.582+0000 7f09a959a640 1 -- 172.21.15.5:0/2974032492 >> v1:172.21.15.5:6789/0 conn(0x7f09ac0d3e20 legacy=0x7f09ac0b6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:28.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.582+0000 7f09a959a640 1 --2- 172.21.15.5:0/2974032492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09940039b0 0x7f0994003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:28.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.582+0000 7f09a959a640 1 --2- 172.21.15.5:0/2974032492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0994004340 0x7f0994004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:28.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.582+0000 7f09a959a640 1 --2- 172.21.15.5:0/2974032492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0994004ed0 0x7f099400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:28.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.582+0000 7f09a959a640 1 -- 172.21.15.5:0/2974032492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac1633f0 con 0x7f0994004340 2024-05-20T18:35:28.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.582+0000 7f09a959a640 1 -- 172.21.15.5:0/2974032492 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac163680 con 0x7f0994004ed0 2024-05-20T18:35:28.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.582+0000 7f09a959a640 1 -- 172.21.15.5:0/2974032492 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f099400bf00 con 0x7f09940039b0 2024-05-20T18:35:28.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.582+0000 7f09a9d9b640 1 --2- 172.21.15.5:0/2974032492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0994004340 0x7f0994004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:28.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.582+0000 7f09aa59c640 1 --2- 172.21.15.5:0/2974032492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09940039b0 0x7f0994003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:28.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.582+0000 7f09aad9d640 1 --2- 172.21.15.5:0/2974032492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0994004ed0 0x7f099400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:28.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.583+0000 7f09a9d9b640 1 -- 172.21.15.5:0/2974032492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09940039b0 msgr2=0x7f0994003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:28.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.583+0000 7f09a9d9b640 1 --2- 172.21.15.5:0/2974032492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09940039b0 0x7f0994003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:28.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.583+0000 7f09a9d9b640 1 -- 172.21.15.5:0/2974032492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0994004ed0 msgr2=0x7f099400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:28.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.583+0000 7f09a9d9b640 1 --2- 172.21.15.5:0/2974032492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0994004ed0 0x7f099400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:28.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.583+0000 7f09a9d9b640 1 -- 172.21.15.5:0/2974032492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0994003520 con 0x7f0994004340 2024-05-20T18:35:28.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.583+0000 7f09a9d9b640 1 --2- 172.21.15.5:0/2974032492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0994004340 0x7f0994004860 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009ac0 tx=0x7f0994005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33605aee2862cacb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:28.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.584+0000 7f09a959a640 1 -- 172.21.15.5:0/2974032492 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0036070 con 0x7f0994004340 2024-05-20T18:35:28.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.584+0000 7f09a959a640 1 -- 172.21.15.5:0/2974032492 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09a0031400 con 0x7f0994004340 2024-05-20T18:35:28.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.584+0000 7f09a959a640 1 -- 172.21.15.5:0/2974032492 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0009690 con 0x7f0994004340 2024-05-20T18:35:28.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.584+0000 7f09ab59e640 1 -- 172.21.15.5:0/2974032492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0994004340 msgr2=0x7f0994004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:28.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.584+0000 7f09ab59e640 1 --2- 172.21.15.5:0/2974032492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0994004340 0x7f0994004860 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009ac0 tx=0x7f0994005830 comp rx=0 tx=0).stop 2024-05-20T18:35:28.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.584+0000 7f09aad9d640 1 -- 172.21.15.5:0/2974032492 reap_dead start 2024-05-20T18:35:28.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.584+0000 7f09ab59e640 1 -- 172.21.15.5:0/2974032492 shutdown_connections 2024-05-20T18:35:28.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.584+0000 7f09ab59e640 1 -- 172.21.15.5:0/2974032492 >> 172.21.15.5:0/2974032492 conn(0x7f09ac0d19f0 msgr2=0x7f09ac0d1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:28.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.584+0000 7f09ab59e640 1 -- 172.21.15.5:0/2974032492 shutdown_connections 2024-05-20T18:35:28.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.584+0000 7f09ab59e640 1 -- 172.21.15.5:0/2974032492 wait complete. 2024-05-20T18:35:28.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.586+0000 7f09ab59e640 1 Processor -- start 2024-05-20T18:35:28.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.586+0000 7f09ab59e640 1 -- start start 2024-05-20T18:35:28.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.588+0000 7f09ab59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0994004340 0x7f09ac1f94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:28.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.588+0000 7f09ab59e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09ac1536f0 0x7f09ac1f8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:28.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.588+0000 7f09ab59e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09ac1f8e90 0x7f09ac0ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:28.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.588+0000 7f09ab59e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac163960 con 0x7f0994004340 2024-05-20T18:35:28.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.588+0000 7f09ab59e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac0d3900 con 0x7f09ac1f8e90 2024-05-20T18:35:28.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.588+0000 7f09ab59e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f09ac0d4230 con 0x7f09ac1536f0 2024-05-20T18:35:28.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.588+0000 7f09aa59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0994004340 0x7f09ac1f94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:28.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.588+0000 7f09aa59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0994004340 0x7f09ac1f94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37856/0 (socket says 172.21.15.5:37856) 2024-05-20T18:35:28.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.588+0000 7f09aa59c640 1 -- 172.21.15.5:0/2248244276 learned_addr learned my addr 172.21.15.5:0/2248244276 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:28.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.588+0000 7f09a9d9b640 1 --2- 172.21.15.5:0/2248244276 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09ac1536f0 0x7f09ac1f8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:28.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.588+0000 7f09aad9d640 1 --2- 172.21.15.5:0/2248244276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09ac1f8e90 0x7f09ac0ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:28.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.588+0000 7f09aa59c640 1 -- 172.21.15.5:0/2248244276 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09ac1536f0 msgr2=0x7f09ac1f8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:28.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.588+0000 7f09aa59c640 1 --2- 172.21.15.5:0/2248244276 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09ac1536f0 0x7f09ac1f8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:28.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.588+0000 7f09aa59c640 1 -- 172.21.15.5:0/2248244276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09ac1f8e90 msgr2=0x7f09ac0ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:28.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.588+0000 7f09aa59c640 1 --2- 172.21.15.5:0/2248244276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09ac1f8e90 0x7f09ac0ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:28.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.588+0000 7f09aa59c640 1 -- 172.21.15.5:0/2248244276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09ac1f5370 con 0x7f0994004340 2024-05-20T18:35:28.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.588+0000 7f09aa59c640 1 --2- 172.21.15.5:0/2248244276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0994004340 0x7f09ac1f94f0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f099c007340 tx=0x7f099c007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:28.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.588+0000 7f099affd640 1 -- 172.21.15.5:0/2248244276 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c00f9c0 con 0x7f0994004340 2024-05-20T18:35:28.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.589+0000 7f099affd640 1 -- 172.21.15.5:0/2248244276 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f099c018070 con 0x7f0994004340 2024-05-20T18:35:28.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.589+0000 7f099affd640 1 -- 172.21.15.5:0/2248244276 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c00fd80 con 0x7f0994004340 2024-05-20T18:35:28.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.589+0000 7f09ab59e640 1 -- 172.21.15.5:0/2248244276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09ac1f92a0 con 0x7f0994004340 2024-05-20T18:35:28.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.589+0000 7f09ab59e640 1 -- 172.21.15.5:0/2248244276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09ac0cc2f0 con 0x7f0994004340 2024-05-20T18:35:28.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.590+0000 7f09ab59e640 1 -- 172.21.15.5:0/2248244276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09ac0cad40 con 0x7f0994004340 2024-05-20T18:35:28.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.590+0000 7f099affd640 1 -- 172.21.15.5:0/2248244276 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f099c01d460 con 0x7f0994004340 2024-05-20T18:35:28.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.591+0000 7f099affd640 1 --2- 172.21.15.5:0/2248244276 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f098c06be60 0x7f098c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:28.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.591+0000 7f099affd640 1 -- 172.21.15.5:0/2248244276 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5249+0+0 (secure 0 0 0) 0x7f099c08ae20 con 0x7f0994004340 2024-05-20T18:35:28.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.591+0000 7f09a9d9b640 1 --2- 172.21.15.5:0/2248244276 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f098c06be60 0x7f098c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:28.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.591+0000 7f09a9d9b640 1 --2- 172.21.15.5:0/2248244276 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f098c06be60 0x7f098c06e370 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f09a0033000 tx=0x7f09a0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:28.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.595+0000 7f099affd640 1 -- 172.21.15.5:0/2248244276 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f09ac1f92a0 con 0x7f0994004340 2024-05-20T18:35:28.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.782+0000 7f09ab59e640 1 -- 172.21.15.5:0/2248244276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f09ac075980 con 0x7f0994004340 2024-05-20T18:35:28.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.782+0000 7f099affd640 1 -- 172.21.15.5:0/2248244276 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v101) ==== 84+0+65 (secure 0 0 0) 0x7f09ac06cff0 con 0x7f0994004340 2024-05-20T18:35:28.783 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:28.783 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-05-20T18:35:28.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:35:28.784 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-05-20T18:35:28.784 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-05-20T18:35:28.784 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:35:28.784 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-05-20T18:35:28.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.784+0000 7f09ab59e640 1 -- 172.21.15.5:0/2248244276 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f098c06be60 msgr2=0x7f098c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:28.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.784+0000 7f09ab59e640 1 --2- 172.21.15.5:0/2248244276 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f098c06be60 0x7f098c06e370 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f09a0033000 tx=0x7f09a0008030 comp rx=0 tx=0).stop 2024-05-20T18:35:28.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.784+0000 7f09ab59e640 1 -- 172.21.15.5:0/2248244276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0994004340 msgr2=0x7f09ac1f94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:28.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.784+0000 7f09ab59e640 1 --2- 172.21.15.5:0/2248244276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0994004340 0x7f09ac1f94f0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f099c007340 tx=0x7f099c007800 comp rx=0 tx=0).stop 2024-05-20T18:35:28.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.784+0000 7f09ab59e640 1 -- 172.21.15.5:0/2248244276 shutdown_connections 2024-05-20T18:35:28.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.784+0000 7f09ab59e640 1 --2- 172.21.15.5:0/2248244276 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f098c06be60 0x7f098c06e370 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:28.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.784+0000 7f09ab59e640 1 --2- 172.21.15.5:0/2248244276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09ac1f8e90 0x7f09ac0ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:28.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.784+0000 7f09ab59e640 1 --2- 172.21.15.5:0/2248244276 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f09ac1536f0 0x7f09ac1f8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:28.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.784+0000 7f09ab59e640 1 --2- 172.21.15.5:0/2248244276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0994004340 0x7f09ac1f94f0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:28.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.784+0000 7f09ab59e640 1 -- 172.21.15.5:0/2248244276 >> 172.21.15.5:0/2248244276 conn(0x7f09ac0d19f0 msgr2=0x7f09ac0b80a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:28.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.784+0000 7f09ab59e640 1 -- 172.21.15.5:0/2248244276 shutdown_connections 2024-05-20T18:35:28.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.784+0000 7f09ab59e640 1 -- 172.21.15.5:0/2248244276 wait complete. 2024-05-20T18:35:28.797 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-20T18:35:28.797 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-20T18:35:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.914+0000 7fb58680d640 1 Processor -- start 2024-05-20T18:35:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.914+0000 7fb58680d640 1 -- start start 2024-05-20T18:35:28.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.916+0000 7fb58680d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5800d2330 0x7fb58006d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:28.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.916+0000 7fb58680d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb58006d660 0x7fb5800cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:28.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.916+0000 7fb58680d640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5800d2740 con 0x7fb5800cdf70 2024-05-20T18:35:28.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.916+0000 7fb58680d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fb58009c3b0 con 0x7fb58006d660 2024-05-20T18:35:28.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.916+0000 7fb58680d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb58006dda0 con 0x7fb5800d2330 2024-05-20T18:35:28.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.916+0000 7fb577fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb58006d660 0x7fb5800cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:28.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.916+0000 7fb577fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb58006d660 0x7fb5800cda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37872/0 (socket says 172.21.15.5:37872) 2024-05-20T18:35:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.916+0000 7fb577fff640 1 -- 172.21.15.5:0/3867831840 learned_addr learned my addr 172.21.15.5:0/3867831840 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.916+0000 7fb57ffff640 1 --2- 172.21.15.5:0/3867831840 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5800d2330 0x7fb58006d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.916+0000 7fb577fff640 1 -- 172.21.15.5:0/3867831840 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5800d2330 msgr2=0x7fb58006d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.916+0000 7fb577fff640 1 --2- 172.21.15.5:0/3867831840 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5800d2330 0x7fb58006d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.916+0000 7fb577fff640 1 -- 172.21.15.5:0/3867831840 >> v1:172.21.15.63:6789/0 conn(0x7fb5800cdf70 legacy=0x7fb580163770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.916+0000 7fb577fff640 1 -- 172.21.15.5:0/3867831840 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5800b81a0 con 0x7fb58006d660 2024-05-20T18:35:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.916+0000 7fb57ffff640 1 --2- 172.21.15.5:0/3867831840 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5800d2330 0x7fb58006d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.917+0000 7fb577fff640 1 --2- 172.21.15.5:0/3867831840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb58006d660 0x7fb5800cda30 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fb568009870 tx=0x7fb56802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a546462abc3abe81 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.917+0000 7fb57f7fe640 1 -- 172.21.15.5:0/3867831840 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb568037070 con 0x7fb58006d660 2024-05-20T18:35:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.917+0000 7fb57f7fe640 1 -- 172.21.15.5:0/3867831840 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb568032440 con 0x7fb58006d660 2024-05-20T18:35:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.917+0000 7fb57f7fe640 1 -- 172.21.15.5:0/3867831840 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb568032810 con 0x7fb58006d660 2024-05-20T18:35:28.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.918+0000 7fb58680d640 1 -- 172.21.15.5:0/3867831840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb58006d660 msgr2=0x7fb5800cda30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:28.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.918+0000 7fb58680d640 1 --2- 172.21.15.5:0/3867831840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb58006d660 0x7fb5800cda30 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fb568009870 tx=0x7fb56802a210 comp rx=0 tx=0).stop 2024-05-20T18:35:28.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.918+0000 7fb58680d640 1 -- 172.21.15.5:0/3867831840 shutdown_connections 2024-05-20T18:35:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.918+0000 7fb58680d640 1 --2- 172.21.15.5:0/3867831840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb58006d660 0x7fb5800cda30 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.918+0000 7fb58680d640 1 --2- 172.21.15.5:0/3867831840 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5800d2330 0x7fb58006d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.918+0000 7fb58680d640 1 -- 172.21.15.5:0/3867831840 >> 172.21.15.5:0/3867831840 conn(0x7fb5800cf450 msgr2=0x7fb5800cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.918+0000 7fb58680d640 1 -- 172.21.15.5:0/3867831840 shutdown_connections 2024-05-20T18:35:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.918+0000 7fb58680d640 1 -- 172.21.15.5:0/3867831840 wait complete. 2024-05-20T18:35:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.920+0000 7fb58680d640 1 Processor -- start 2024-05-20T18:35:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.920+0000 7fb58680d640 1 -- start start 2024-05-20T18:35:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.921+0000 7fb58680d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb58006d660 0x7fb5800b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.921+0000 7fb58680d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5800cdf70 0x7fb580152b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.921+0000 7fb58680d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5800d2330 0x7fb580153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.921+0000 7fb58680d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb5800d3800 con 0x7fb58006d660 2024-05-20T18:35:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.921+0000 7fb58680d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb58006da70 con 0x7fb5800d2330 2024-05-20T18:35:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.921+0000 7fb58680d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb5800d3980 con 0x7fb5800cdf70 2024-05-20T18:35:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.921+0000 7fb57ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb58006d660 0x7fb5800b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.921+0000 7fb57ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb58006d660 0x7fb5800b35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37888/0 (socket says 172.21.15.5:37888) 2024-05-20T18:35:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.921+0000 7fb57ffff640 1 -- 172.21.15.5:0/1271620296 learned_addr learned my addr 172.21.15.5:0/1271620296 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.922+0000 7fb577fff640 1 --2- 172.21.15.5:0/1271620296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5800cdf70 0x7fb580152b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.922+0000 7fb584daa640 1 --2- 172.21.15.5:0/1271620296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5800d2330 0x7fb580153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.922+0000 7fb57ffff640 1 -- 172.21.15.5:0/1271620296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5800cdf70 msgr2=0x7fb580152b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.922+0000 7fb57ffff640 1 --2- 172.21.15.5:0/1271620296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5800cdf70 0x7fb580152b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.922+0000 7fb57ffff640 1 -- 172.21.15.5:0/1271620296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5800d2330 msgr2=0x7fb580153070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.922+0000 7fb57ffff640 1 --2- 172.21.15.5:0/1271620296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5800d2330 0x7fb580153070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.922+0000 7fb57ffff640 1 -- 172.21.15.5:0/1271620296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5801538d0 con 0x7fb58006d660 2024-05-20T18:35:28.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.922+0000 7fb584daa640 1 --2- 172.21.15.5:0/1271620296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5800d2330 0x7fb580153070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:28.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.922+0000 7fb57ffff640 1 --2- 172.21.15.5:0/1271620296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb58006d660 0x7fb5800b35f0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fb57000c820 tx=0x7fb57000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:28.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.922+0000 7fb57d7fa640 1 -- 172.21.15.5:0/1271620296 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb570004220 con 0x7fb58006d660 2024-05-20T18:35:28.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.922+0000 7fb57d7fa640 1 -- 172.21.15.5:0/1271620296 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5700043c0 con 0x7fb58006d660 2024-05-20T18:35:28.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.922+0000 7fb58680d640 1 -- 172.21.15.5:0/1271620296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb580150cb0 con 0x7fb58006d660 2024-05-20T18:35:28.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.922+0000 7fb57d7fa640 1 -- 172.21.15.5:0/1271620296 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb570005bb0 con 0x7fb58006d660 2024-05-20T18:35:28.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.922+0000 7fb58680d640 1 -- 172.21.15.5:0/1271620296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb580152810 con 0x7fb58006d660 2024-05-20T18:35:28.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.923+0000 7fb58680d640 1 -- 172.21.15.5:0/1271620296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5800690b0 con 0x7fb58006d660 2024-05-20T18:35:28.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.924+0000 7fb57d7fa640 1 -- 172.21.15.5:0/1271620296 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb570007830 con 0x7fb58006d660 2024-05-20T18:35:28.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.924+0000 7fb57d7fa640 1 --2- 172.21.15.5:0/1271620296 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb56006be60 0x7fb56006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:28.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.924+0000 7fb57d7fa640 1 -- 172.21.15.5:0/1271620296 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5249+0+0 (secure 0 0 0) 0x7fb57008b0f0 con 0x7fb58006d660 2024-05-20T18:35:28.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.924+0000 7fb577fff640 1 --2- 172.21.15.5:0/1271620296 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb56006be60 0x7fb56006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:28.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.925+0000 7fb577fff640 1 --2- 172.21.15.5:0/1271620296 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb56006be60 0x7fb56006e370 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb568034000 tx=0x7fb56802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:28.929+0000 7fb57d7fa640 1 -- 172.21.15.5:0/1271620296 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb57001e020 con 0x7fb58006d660 2024-05-20T18:35:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.114+0000 7fb58680d640 1 -- 172.21.15.5:0/1271620296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb5800cd420 con 0x7fb58006d660 2024-05-20T18:35:29.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.383+0000 7fb57d7fa640 1 -- 172.21.15.5:0/1271620296 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v102) ==== 138+0+0 (secure 0 0 0) 0x7fb570015030 con 0x7fb58006d660 2024-05-20T18:35:29.384 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-05-20T18:35:29.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.386+0000 7fb58680d640 1 -- 172.21.15.5:0/1271620296 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb56006be60 msgr2=0x7fb56006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:29.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.386+0000 7fb58680d640 1 --2- 172.21.15.5:0/1271620296 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb56006be60 0x7fb56006e370 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb568034000 tx=0x7fb56802e000 comp rx=0 tx=0).stop 2024-05-20T18:35:29.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.386+0000 7fb58680d640 1 -- 172.21.15.5:0/1271620296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb58006d660 msgr2=0x7fb5800b35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:29.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.386+0000 7fb58680d640 1 --2- 172.21.15.5:0/1271620296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb58006d660 0x7fb5800b35f0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fb57000c820 tx=0x7fb57000cce0 comp rx=0 tx=0).stop 2024-05-20T18:35:29.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.386+0000 7fb58680d640 1 -- 172.21.15.5:0/1271620296 shutdown_connections 2024-05-20T18:35:29.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.386+0000 7fb58680d640 1 --2- 172.21.15.5:0/1271620296 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb56006be60 0x7fb56006e370 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:29.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.386+0000 7fb58680d640 1 --2- 172.21.15.5:0/1271620296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5800d2330 0x7fb580153070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:29.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.386+0000 7fb58680d640 1 --2- 172.21.15.5:0/1271620296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5800cdf70 0x7fb580152b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:29.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.386+0000 7fb58680d640 1 --2- 172.21.15.5:0/1271620296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb58006d660 0x7fb5800b35f0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:29.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.386+0000 7fb58680d640 1 -- 172.21.15.5:0/1271620296 >> 172.21.15.5:0/1271620296 conn(0x7fb5800cf450 msgr2=0x7fb580162de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:29.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.386+0000 7fb58680d640 1 -- 172.21.15.5:0/1271620296 shutdown_connections 2024-05-20T18:35:29.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.386+0000 7fb58680d640 1 -- 172.21.15.5:0/1271620296 wait complete. 2024-05-20T18:35:29.402 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-20T18:35:29.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.523+0000 7fcfabb85640 1 Processor -- start 2024-05-20T18:35:29.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.523+0000 7fcfabb85640 1 -- start start 2024-05-20T18:35:29.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.524+0000 7fcfabb85640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fcfa40adf40 0x7fcfa40af6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:29.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.524+0000 7fcfabb85640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfa4153210 con 0x7fcfa4152e00 2024-05-20T18:35:29.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.524+0000 7fcfabb85640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfa415c5d0 con 0x7fcfa406cc40 2024-05-20T18:35:29.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.524+0000 7fcfabb85640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fcfa415c380 con 0x7fcfa40adf40 2024-05-20T18:35:29.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.524+0000 7fcfa9921640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fcfa4152e00 0x7fcfa406c530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36132/0 (socket says 172.21.15.5:36132) 2024-05-20T18:35:29.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.525+0000 7fcfa9921640 1 -- 172.21.15.5:0/2148338044 learned_addr learned my addr 172.21.15.5:0/2148338044 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:29.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.525+0000 7fcfa891f640 1 -- 172.21.15.5:0/2148338044 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 793084295 0 0) 0x7fcfa4153210 con 0x7fcfa4152e00 2024-05-20T18:35:29.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.525+0000 7fcfaa122640 1 --2- 172.21.15.5:0/2148338044 >> v2:172.21.15.63:3300/0 conn(0x7fcfa40adf40 0x7fcfa40af6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:29.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.525+0000 7fcfa891f640 1 -- 172.21.15.5:0/2148338044 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf90003520 con 0x7fcfa4152e00 2024-05-20T18:35:29.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.525+0000 7fcfa891f640 1 -- 172.21.15.5:0/2148338044 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 150452259 0 0) 0x7fcf90003520 con 0x7fcfa4152e00 2024-05-20T18:35:29.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.525+0000 7fcfa891f640 1 -- 172.21.15.5:0/2148338044 >> v2:172.21.15.63:3300/0 conn(0x7fcfa40adf40 msgr2=0x7fcfa40af6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:29.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.525+0000 7fcfa891f640 1 --2- 172.21.15.5:0/2148338044 >> v2:172.21.15.63:3300/0 conn(0x7fcfa40adf40 0x7fcfa40af6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:29.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.525+0000 7fcfa891f640 1 -- 172.21.15.5:0/2148338044 >> v1:172.21.15.63:6789/0 conn(0x7fcfa406cc40 legacy=0x7fcfa40ad830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:29.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.525+0000 7fcfa891f640 1 -- 172.21.15.5:0/2148338044 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa415c8b0 con 0x7fcfa4152e00 2024-05-20T18:35:29.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.525+0000 7fcfa891f640 1 -- 172.21.15.5:0/2148338044 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7fcfa0001690 con 0x7fcfa4152e00 2024-05-20T18:35:29.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.525+0000 7fcfa891f640 1 -- 172.21.15.5:0/2148338044 >> v1:172.21.15.5:6789/0 conn(0x7fcfa4152e00 legacy=0x7fcfa406c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:29.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.525+0000 7fcfa891f640 1 --2- 172.21.15.5:0/2148338044 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf900039b0 0x7fcf90003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:29.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.525+0000 7fcfa891f640 1 --2- 172.21.15.5:0/2148338044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf90004800 0x7fcf90004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:29.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.525+0000 7fcfa891f640 1 --2- 172.21.15.5:0/2148338044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf900051c0 0x7fcf9000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:29.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.525+0000 7fcfa891f640 1 -- 172.21.15.5:0/2148338044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa415c380 con 0x7fcf90004800 2024-05-20T18:35:29.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.525+0000 7fcfa891f640 1 -- 172.21.15.5:0/2148338044 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9000c160 con 0x7fcf900051c0 2024-05-20T18:35:29.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.525+0000 7fcfa891f640 1 -- 172.21.15.5:0/2148338044 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcf9000c360 con 0x7fcf900039b0 2024-05-20T18:35:29.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.525+0000 7fcfa9120640 1 --2- 172.21.15.5:0/2148338044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf90004800 0x7fcf90004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:29.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.525+0000 7fcfaa122640 1 --2- 172.21.15.5:0/2148338044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf900051c0 0x7fcf9000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:29.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.526+0000 7fcfa9921640 1 --2- 172.21.15.5:0/2148338044 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf900039b0 0x7fcf90003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:29.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.526+0000 7fcfa9120640 1 -- 172.21.15.5:0/2148338044 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf900039b0 msgr2=0x7fcf90003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:29.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.526+0000 7fcfa9120640 1 --2- 172.21.15.5:0/2148338044 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf900039b0 0x7fcf90003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:29.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.526+0000 7fcfa9120640 1 -- 172.21.15.5:0/2148338044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf900051c0 msgr2=0x7fcf9000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:29.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.526+0000 7fcfa9120640 1 --2- 172.21.15.5:0/2148338044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf900051c0 0x7fcf9000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:29.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.526+0000 7fcfa9120640 1 -- 172.21.15.5:0/2148338044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf90003520 con 0x7fcf90004800 2024-05-20T18:35:29.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.526+0000 7fcfaa122640 1 -- 172.21.15.5:0/2148338044 reap_dead start 2024-05-20T18:35:29.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.527+0000 7fcfa9120640 1 --2- 172.21.15.5:0/2148338044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf90004800 0x7fcf90004c10 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fcf94001750 tx=0x7fcf90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfbef1dd42f9a81d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:29.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.527+0000 7fcfa891f640 1 -- 172.21.15.5:0/2148338044 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf94001370 con 0x7fcf90004800 2024-05-20T18:35:29.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.527+0000 7fcfa891f640 1 -- 172.21.15.5:0/2148338044 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf940052d0 con 0x7fcf90004800 2024-05-20T18:35:29.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.527+0000 7fcfa891f640 1 -- 172.21.15.5:0/2148338044 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9402abc0 con 0x7fcf90004800 2024-05-20T18:35:29.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.527+0000 7fcfabb85640 1 -- 172.21.15.5:0/2148338044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf90004800 msgr2=0x7fcf90004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:29.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.527+0000 7fcfabb85640 1 --2- 172.21.15.5:0/2148338044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf90004800 0x7fcf90004c10 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fcf94001750 tx=0x7fcf90005830 comp rx=0 tx=0).stop 2024-05-20T18:35:29.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.527+0000 7fcfabb85640 1 -- 172.21.15.5:0/2148338044 shutdown_connections 2024-05-20T18:35:29.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.527+0000 7fcfabb85640 1 --2- 172.21.15.5:0/2148338044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf90004800 0x7fcf90004c10 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:29.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.527+0000 7fcfabb85640 1 -- 172.21.15.5:0/2148338044 >> 172.21.15.5:0/2148338044 conn(0x7fcfa40723a0 msgr2=0x7fcfa40727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:29.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.528+0000 7fcfabb85640 1 -- 172.21.15.5:0/2148338044 shutdown_connections 2024-05-20T18:35:29.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.528+0000 7fcfabb85640 1 -- 172.21.15.5:0/2148338044 wait complete. 2024-05-20T18:35:29.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.529+0000 7fcfabb85640 1 Processor -- start 2024-05-20T18:35:29.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.529+0000 7fcfabb85640 1 -- start start 2024-05-20T18:35:29.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.531+0000 7fcfabb85640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf90004800 0x7fcfa4153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:29.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.531+0000 7fcfabb85640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa41561e0 0x7fcfa41441d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:29.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.531+0000 7fcfabb85640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfa4158bc0 0x7fcfa4158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:29.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.531+0000 7fcfabb85640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa415ce70 con 0x7fcfa4158bc0 2024-05-20T18:35:29.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.531+0000 7fcfabb85640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa415d1e0 con 0x7fcf90004800 2024-05-20T18:35:29.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.531+0000 7fcfabb85640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcfa40ad830 con 0x7fcfa41561e0 2024-05-20T18:35:29.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.531+0000 7fcfaa122640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfa4158bc0 0x7fcfa4158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:29.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.531+0000 7fcfaa122640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfa4158bc0 0x7fcfa4158fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37918/0 (socket says 172.21.15.5:37918) 2024-05-20T18:35:29.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.531+0000 7fcfaa122640 1 -- 172.21.15.5:0/2930057288 learned_addr learned my addr 172.21.15.5:0/2930057288 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:29.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.531+0000 7fcfa9921640 1 --2- 172.21.15.5:0/2930057288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf90004800 0x7fcfa4153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:29.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.531+0000 7fcfa9120640 1 --2- 172.21.15.5:0/2930057288 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa41561e0 0x7fcfa41441d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:29.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.531+0000 7fcfaa122640 1 -- 172.21.15.5:0/2930057288 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa41561e0 msgr2=0x7fcfa41441d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:29.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.531+0000 7fcfaa122640 1 --2- 172.21.15.5:0/2930057288 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa41561e0 0x7fcfa41441d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:29.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.531+0000 7fcfaa122640 1 -- 172.21.15.5:0/2930057288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf90004800 msgr2=0x7fcfa4153410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:29.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.531+0000 7fcfaa122640 1 --2- 172.21.15.5:0/2930057288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf90004800 0x7fcfa4153410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:29.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.531+0000 7fcfaa122640 1 -- 172.21.15.5:0/2930057288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa4159510 con 0x7fcfa4158bc0 2024-05-20T18:35:29.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.531+0000 7fcfaa122640 1 --2- 172.21.15.5:0/2930057288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfa4158bc0 0x7fcfa4158fd0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40ae370 tx=0x7fcf9c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:29.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.532+0000 7fcf9a7fc640 1 -- 172.21.15.5:0/2930057288 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c00fdb0 con 0x7fcfa4158bc0 2024-05-20T18:35:29.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.532+0000 7fcf9a7fc640 1 -- 172.21.15.5:0/2930057288 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf9c01d070 con 0x7fcfa4158bc0 2024-05-20T18:35:29.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.532+0000 7fcf9a7fc640 1 -- 172.21.15.5:0/2930057288 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c015670 con 0x7fcfa4158bc0 2024-05-20T18:35:29.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.532+0000 7fcfabb85640 1 -- 172.21.15.5:0/2930057288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa4144710 con 0x7fcfa4158bc0 2024-05-20T18:35:29.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.532+0000 7fcfabb85640 1 -- 172.21.15.5:0/2930057288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa4144fa0 con 0x7fcfa4158bc0 2024-05-20T18:35:29.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.533+0000 7fcfabb85640 1 -- 172.21.15.5:0/2930057288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfa406fcc0 con 0x7fcfa4158bc0 2024-05-20T18:35:29.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.533+0000 7fcf9a7fc640 1 -- 172.21.15.5:0/2930057288 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcf9c015810 con 0x7fcfa4158bc0 2024-05-20T18:35:29.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.534+0000 7fcf9a7fc640 1 --2- 172.21.15.5:0/2930057288 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcf8806be60 0x7fcf8806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:29.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.534+0000 7fcf9a7fc640 1 -- 172.21.15.5:0/2930057288 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5619+0+0 (secure 0 0 0) 0x7fcf9c08c2d0 con 0x7fcfa4158bc0 2024-05-20T18:35:29.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.534+0000 7fcfa9921640 1 --2- 172.21.15.5:0/2930057288 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcf8806be60 0x7fcf8806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:29.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.535+0000 7fcfa9921640 1 --2- 172.21.15.5:0/2930057288 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcf8806be60 0x7fcf8806e370 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fcfa41533e0 tx=0x7fcfa0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:29.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.538+0000 7fcf9a7fc640 1 -- 172.21.15.5:0/2930057288 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fcfa406fcc0 con 0x7fcfa4158bc0 2024-05-20T18:35:29.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:29.722+0000 7fcfabb85640 1 -- 172.21.15.5:0/2930057288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fcfa41539f0 con 0x7fcfa4158bc0 2024-05-20T18:35:30.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.385+0000 7fcf9a7fc640 1 -- 172.21.15.5:0/2930057288 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v103) ==== 170+0+0 (secure 0 0 0) 0x7fcfa4159510 con 0x7fcfa4158bc0 2024-05-20T18:35:30.387 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-05-20T18:35:30.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.389+0000 7fcfabb85640 1 -- 172.21.15.5:0/2930057288 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcf8806be60 msgr2=0x7fcf8806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:30.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.389+0000 7fcfabb85640 1 --2- 172.21.15.5:0/2930057288 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcf8806be60 0x7fcf8806e370 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fcfa41533e0 tx=0x7fcfa0009040 comp rx=0 tx=0).stop 2024-05-20T18:35:30.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.389+0000 7fcfabb85640 1 -- 172.21.15.5:0/2930057288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfa4158bc0 msgr2=0x7fcfa4158fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:30.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.389+0000 7fcfabb85640 1 --2- 172.21.15.5:0/2930057288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfa4158bc0 0x7fcfa4158fd0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40ae370 tx=0x7fcf9c00f580 comp rx=0 tx=0).stop 2024-05-20T18:35:30.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.390+0000 7fcfabb85640 1 -- 172.21.15.5:0/2930057288 shutdown_connections 2024-05-20T18:35:30.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.390+0000 7fcfabb85640 1 --2- 172.21.15.5:0/2930057288 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcf8806be60 0x7fcf8806e370 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:30.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.390+0000 7fcfabb85640 1 --2- 172.21.15.5:0/2930057288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfa4158bc0 0x7fcfa4158fd0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:30.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.390+0000 7fcfabb85640 1 --2- 172.21.15.5:0/2930057288 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfa41561e0 0x7fcfa41441d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:30.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.390+0000 7fcfabb85640 1 --2- 172.21.15.5:0/2930057288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf90004800 0x7fcfa4153410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:30.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.390+0000 7fcfabb85640 1 -- 172.21.15.5:0/2930057288 >> 172.21.15.5:0/2930057288 conn(0x7fcfa40723a0 msgr2=0x7fcfa40b4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:30.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.390+0000 7fcfabb85640 1 -- 172.21.15.5:0/2930057288 shutdown_connections 2024-05-20T18:35:30.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.390+0000 7fcfabb85640 1 -- 172.21.15.5:0/2930057288 wait complete. 2024-05-20T18:35:30.417 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-20T18:35:30.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.540+0000 7f1c33f45640 1 Processor -- start 2024-05-20T18:35:30.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.540+0000 7f1c33f45640 1 -- start start 2024-05-20T18:35:30.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.542+0000 7f1c33f45640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1c2c0cd680 0x7f1c2c15f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:30.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.542+0000 7f1c33f45640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1c2c0cfae0 0x7f1c2c06cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:30.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.542+0000 7f1c33f45640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c2c06d0f0 con 0x7f1c2c06c6c0 2024-05-20T18:35:30.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.542+0000 7f1c33f45640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f1c2c0cda90 con 0x7f1c2c0cfae0 2024-05-20T18:35:30.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.542+0000 7f1c33f45640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1c2c15f760 con 0x7f1c2c0cd680 2024-05-20T18:35:30.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.542+0000 7f1c324e2640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1c2c0cfae0 0x7f1c2c06cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:30.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.542+0000 7f1c324e2640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1c2c0cfae0 0x7f1c2c06cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37930/0 (socket says 172.21.15.5:37930) 2024-05-20T18:35:30.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.542+0000 7f1c324e2640 1 -- 172.21.15.5:0/211968123 learned_addr learned my addr 172.21.15.5:0/211968123 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:30.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.542+0000 7f1c31ce1640 1 --2- 172.21.15.5:0/211968123 >> v2:172.21.15.63:3300/0 conn(0x7f1c2c0cd680 0x7f1c2c15f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:30.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.542+0000 7f1c324e2640 1 -- 172.21.15.5:0/211968123 >> v2:172.21.15.63:3300/0 conn(0x7f1c2c0cd680 msgr2=0x7f1c2c15f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:30.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.542+0000 7f1c324e2640 1 --2- 172.21.15.5:0/211968123 >> v2:172.21.15.63:3300/0 conn(0x7f1c2c0cd680 0x7f1c2c15f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:30.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.542+0000 7f1c324e2640 1 -- 172.21.15.5:0/211968123 >> v1:172.21.15.63:6789/0 conn(0x7f1c2c06c6c0 legacy=0x7f1c2c0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:30.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.542+0000 7f1c324e2640 1 -- 172.21.15.5:0/211968123 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2c06d3d0 con 0x7f1c2c0cfae0 2024-05-20T18:35:30.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.543+0000 7f1c324e2640 1 --2- 172.21.15.5:0/211968123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2c0cfae0 0x7f1c2c06cad0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c08baf0 tx=0x7f1c2402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf7b5644b704050 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:30.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.543+0000 7f1c30cdf640 1 -- 172.21.15.5:0/211968123 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24036070 con 0x7f1c2c0cfae0 2024-05-20T18:35:30.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.543+0000 7f1c30cdf640 1 -- 172.21.15.5:0/211968123 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c24031600 con 0x7f1c2c0cfae0 2024-05-20T18:35:30.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.543+0000 7f1c30cdf640 1 -- 172.21.15.5:0/211968123 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c240319d0 con 0x7f1c2c0cfae0 2024-05-20T18:35:30.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.544+0000 7f1c33f45640 1 -- 172.21.15.5:0/211968123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2c0cfae0 msgr2=0x7f1c2c06cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:30.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.544+0000 7f1c33f45640 1 --2- 172.21.15.5:0/211968123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2c0cfae0 0x7f1c2c06cad0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c08baf0 tx=0x7f1c2402a5f0 comp rx=0 tx=0).stop 2024-05-20T18:35:30.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.544+0000 7f1c33f45640 1 -- 172.21.15.5:0/211968123 shutdown_connections 2024-05-20T18:35:30.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.544+0000 7f1c33f45640 1 --2- 172.21.15.5:0/211968123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2c0cfae0 0x7f1c2c06cad0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:30.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.544+0000 7f1c33f45640 1 --2- 172.21.15.5:0/211968123 >> v2:172.21.15.63:3300/0 conn(0x7f1c2c0cd680 0x7f1c2c15f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:30.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.544+0000 7f1c33f45640 1 -- 172.21.15.5:0/211968123 >> 172.21.15.5:0/211968123 conn(0x7f1c2c0b44d0 msgr2=0x7f1c2c0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:30.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.544+0000 7f1c33f45640 1 -- 172.21.15.5:0/211968123 shutdown_connections 2024-05-20T18:35:30.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.544+0000 7f1c33f45640 1 -- 172.21.15.5:0/211968123 wait complete. 2024-05-20T18:35:30.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.546+0000 7f1c33f45640 1 Processor -- start 2024-05-20T18:35:30.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.546+0000 7f1c33f45640 1 -- start start 2024-05-20T18:35:30.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.547+0000 7f1c33f45640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c2c06c6c0 0x7f1c2c14f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:30.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.547+0000 7f1c33f45640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c2c0cd680 0x7f1c2c1618a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:30.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.547+0000 7f1c33f45640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2c0c7110 0x7f1c2c0d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:30.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.547+0000 7f1c33f45640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c06d990 con 0x7f1c2c0c7110 2024-05-20T18:35:30.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.547+0000 7f1c33f45640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c06d750 con 0x7f1c2c0cd680 2024-05-20T18:35:30.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.547+0000 7f1c33f45640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1c2c0d2460 con 0x7f1c2c06c6c0 2024-05-20T18:35:30.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.547+0000 7f1c324e2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2c0c7110 0x7f1c2c0d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:30.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.547+0000 7f1c324e2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2c0c7110 0x7f1c2c0d0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37944/0 (socket says 172.21.15.5:37944) 2024-05-20T18:35:30.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.547+0000 7f1c324e2640 1 -- 172.21.15.5:0/255305989 learned_addr learned my addr 172.21.15.5:0/255305989 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:30.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.547+0000 7f1c31ce1640 1 --2- 172.21.15.5:0/255305989 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c2c06c6c0 0x7f1c2c14f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:30.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.547+0000 7f1c314e0640 1 --2- 172.21.15.5:0/255305989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c2c0cd680 0x7f1c2c1618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:30.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.548+0000 7f1c324e2640 1 -- 172.21.15.5:0/255305989 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c2c06c6c0 msgr2=0x7f1c2c14f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:30.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.548+0000 7f1c324e2640 1 --2- 172.21.15.5:0/255305989 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c2c06c6c0 0x7f1c2c14f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:30.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.548+0000 7f1c324e2640 1 -- 172.21.15.5:0/255305989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c2c0cd680 msgr2=0x7f1c2c1618a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:30.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.548+0000 7f1c324e2640 1 --2- 172.21.15.5:0/255305989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c2c0cd680 0x7f1c2c1618a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:30.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.548+0000 7f1c324e2640 1 -- 172.21.15.5:0/255305989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2c0c6670 con 0x7f1c2c0c7110 2024-05-20T18:35:30.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.548+0000 7f1c314e0640 1 --2- 172.21.15.5:0/255305989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c2c0cd680 0x7f1c2c1618a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:30.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.548+0000 7f1c324e2640 1 --2- 172.21.15.5:0/255305989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2c0c7110 0x7f1c2c0d0f60 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f1c24009f70 tx=0x7f1c24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:30.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.548+0000 7f1c227fc640 1 -- 172.21.15.5:0/255305989 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24036050 con 0x7f1c2c0c7110 2024-05-20T18:35:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.548+0000 7f1c227fc640 1 -- 172.21.15.5:0/255305989 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c24042070 con 0x7f1c2c0c7110 2024-05-20T18:35:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.548+0000 7f1c33f45640 1 -- 172.21.15.5:0/255305989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c2c0c7f70 con 0x7f1c2c0c7110 2024-05-20T18:35:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.548+0000 7f1c227fc640 1 -- 172.21.15.5:0/255305989 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24009510 con 0x7f1c2c0c7110 2024-05-20T18:35:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.548+0000 7f1c33f45640 1 -- 172.21.15.5:0/255305989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c2c0c6910 con 0x7f1c2c0c7110 2024-05-20T18:35:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.549+0000 7f1c33f45640 1 -- 172.21.15.5:0/255305989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c2c0b24d0 con 0x7f1c2c0c7110 2024-05-20T18:35:30.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.550+0000 7f1c227fc640 1 -- 172.21.15.5:0/255305989 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c24007bd0 con 0x7f1c2c0c7110 2024-05-20T18:35:30.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.550+0000 7f1c227fc640 1 --2- 172.21.15.5:0/255305989 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c1006be60 0x7f1c1006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:30.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.550+0000 7f1c227fc640 1 -- 172.21.15.5:0/255305989 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6001+0+0 (secure 0 0 0) 0x7f1c240abf80 con 0x7f1c2c0c7110 2024-05-20T18:35:30.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.550+0000 7f1c31ce1640 1 --2- 172.21.15.5:0/255305989 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c1006be60 0x7f1c1006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:30.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.551+0000 7f1c31ce1640 1 --2- 172.21.15.5:0/255305989 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c1006be60 0x7f1c1006e370 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1c28006fd0 tx=0x7f1c28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:30.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.555+0000 7f1c227fc640 1 -- 172.21.15.5:0/255305989 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1c2c0b24d0 con 0x7f1c2c0c7110 2024-05-20T18:35:30.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:30.777+0000 7f1c33f45640 1 -- 172.21.15.5:0/255305989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1c2c075b00 con 0x7f1c2c0c7110 2024-05-20T18:35:31.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.398+0000 7f1c227fc640 1 -- 172.21.15.5:0/255305989 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v78) ==== 174+0+0 (secure 0 0 0) 0x7f1c2407a800 con 0x7f1c2c0c7110 2024-05-20T18:35:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 15 and data pool 16 2024-05-20T18:35:31.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.402+0000 7f1c33f45640 1 -- 172.21.15.5:0/255305989 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c1006be60 msgr2=0x7f1c1006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:31.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.402+0000 7f1c33f45640 1 --2- 172.21.15.5:0/255305989 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c1006be60 0x7f1c1006e370 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1c28006fd0 tx=0x7f1c28008040 comp rx=0 tx=0).stop 2024-05-20T18:35:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.402+0000 7f1c33f45640 1 -- 172.21.15.5:0/255305989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2c0c7110 msgr2=0x7f1c2c0d0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.402+0000 7f1c33f45640 1 --2- 172.21.15.5:0/255305989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2c0c7110 0x7f1c2c0d0f60 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f1c24009f70 tx=0x7f1c24004a40 comp rx=0 tx=0).stop 2024-05-20T18:35:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.402+0000 7f1c33f45640 1 -- 172.21.15.5:0/255305989 shutdown_connections 2024-05-20T18:35:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.402+0000 7f1c33f45640 1 --2- 172.21.15.5:0/255305989 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1c1006be60 0x7f1c1006e370 secure :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1c28006fd0 tx=0x7f1c28008040 comp rx=0 tx=0).stop 2024-05-20T18:35:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.402+0000 7f1c33f45640 1 --2- 172.21.15.5:0/255305989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2c0c7110 0x7f1c2c0d0f60 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.402+0000 7f1c33f45640 1 --2- 172.21.15.5:0/255305989 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c2c0cd680 0x7f1c2c1618a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.402+0000 7f1c33f45640 1 --2- 172.21.15.5:0/255305989 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c2c06c6c0 0x7f1c2c14f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.402+0000 7f1c33f45640 1 -- 172.21.15.5:0/255305989 >> 172.21.15.5:0/255305989 conn(0x7f1c2c0b44d0 msgr2=0x7f1c2c0cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:31.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.404+0000 7f1c33f45640 1 -- 172.21.15.5:0/255305989 shutdown_connections 2024-05-20T18:35:31.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.406+0000 7f1c33f45640 1 -- 172.21.15.5:0/255305989 wait complete. 2024-05-20T18:35:31.437 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:35:31.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.553+0000 7f66011c3640 1 Processor -- start 2024-05-20T18:35:31.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.554+0000 7f66011c3640 1 -- start start 2024-05-20T18:35:31.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.555+0000 7f66011c3640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f65fc15f700 0x7f65fc15fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:31.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.555+0000 7f66011c3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f65fc06c180 0x7f65fc06c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:31.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.555+0000 7f66011c3640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65fc160430 con 0x7f65fc0af030 2024-05-20T18:35:31.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.555+0000 7f66011c3640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f65fc06bcf0 con 0x7f65fc06c180 2024-05-20T18:35:31.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.555+0000 7f66011c3640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f65fc06b6a0 con 0x7f65fc15f700 2024-05-20T18:35:31.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.555+0000 7f65fad9d640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f65fc0af030 0x7f65fc0af440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36134/0 (socket says 172.21.15.5:36134) 2024-05-20T18:35:31.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.555+0000 7f65fad9d640 1 -- 172.21.15.5:0/2865839531 learned_addr learned my addr 172.21.15.5:0/2865839531 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:31.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.555+0000 7f65fb59e640 1 --2- 172.21.15.5:0/2865839531 >> v2:172.21.15.5:3300/0 conn(0x7f65fc06c180 0x7f65fc06c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:31.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.555+0000 7f65fa59c640 1 --2- 172.21.15.5:0/2865839531 >> v2:172.21.15.63:3300/0 conn(0x7f65fc15f700 0x7f65fc15fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:31.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.555+0000 7f65fb59e640 1 -- 172.21.15.5:0/2865839531 >> v2:172.21.15.63:3300/0 conn(0x7f65fc15f700 msgr2=0x7f65fc15fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:31.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.555+0000 7f65fb59e640 1 --2- 172.21.15.5:0/2865839531 >> v2:172.21.15.63:3300/0 conn(0x7f65fc15f700 0x7f65fc15fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:31.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.555+0000 7f65fb59e640 1 -- 172.21.15.5:0/2865839531 >> v1:172.21.15.5:6789/0 conn(0x7f65fc0af030 legacy=0x7f65fc0af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:31.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.556+0000 7f65fb59e640 1 -- 172.21.15.5:0/2865839531 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65fc06b980 con 0x7f65fc06c180 2024-05-20T18:35:31.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.556+0000 7f65f9d9b640 1 -- 172.21.15.5:0/2865839531 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3044073921 0 0) 0x7f65fc160430 con 0x7f65fc0af030 2024-05-20T18:35:31.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.556+0000 7f65fb59e640 1 --2- 172.21.15.5:0/2865839531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65fc06c180 0x7f65fc06c590 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f65fc0aea40 tx=0x7f65f402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5eba6fe0ccfb05 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:31.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.556+0000 7f65f9d9b640 1 -- 172.21.15.5:0/2865839531 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f4007740 con 0x7f65fc06c180 2024-05-20T18:35:31.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.556+0000 7f65f9d9b640 1 -- 172.21.15.5:0/2865839531 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65f40078e0 con 0x7f65fc06c180 2024-05-20T18:35:31.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.556+0000 7f65f9d9b640 1 -- 172.21.15.5:0/2865839531 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f4007cb0 con 0x7f65fc06c180 2024-05-20T18:35:31.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.557+0000 7f66011c3640 1 -- 172.21.15.5:0/2865839531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65fc06c180 msgr2=0x7f65fc06c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:31.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.557+0000 7f66011c3640 1 --2- 172.21.15.5:0/2865839531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65fc06c180 0x7f65fc06c590 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f65fc0aea40 tx=0x7f65f402a680 comp rx=0 tx=0).stop 2024-05-20T18:35:31.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.557+0000 7f66011c3640 1 -- 172.21.15.5:0/2865839531 shutdown_connections 2024-05-20T18:35:31.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.557+0000 7f66011c3640 1 --2- 172.21.15.5:0/2865839531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65fc06c180 0x7f65fc06c590 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:31.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.557+0000 7f66011c3640 1 --2- 172.21.15.5:0/2865839531 >> v2:172.21.15.63:3300/0 conn(0x7f65fc15f700 0x7f65fc15fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:31.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.557+0000 7f66011c3640 1 -- 172.21.15.5:0/2865839531 >> 172.21.15.5:0/2865839531 conn(0x7f65fc0b6620 msgr2=0x7f65fc0b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:31.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.557+0000 7f66011c3640 1 -- 172.21.15.5:0/2865839531 shutdown_connections 2024-05-20T18:35:31.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.557+0000 7f66011c3640 1 -- 172.21.15.5:0/2865839531 wait complete. 2024-05-20T18:35:31.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.559+0000 7f66011c3640 1 Processor -- start 2024-05-20T18:35:31.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.559+0000 7f66011c3640 1 -- start start 2024-05-20T18:35:31.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.561+0000 7f66011c3640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65fc06c180 0x7f65fc15a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:31.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.561+0000 7f66011c3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65fc0af030 0x7f65fc1f6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:31.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.561+0000 7f66011c3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65fc15f700 0x7f65fc15e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:31.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.561+0000 7f66011c3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f65fc1601d0 con 0x7f65fc0af030 2024-05-20T18:35:31.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.561+0000 7f66011c3640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f65fc160050 con 0x7f65fc06c180 2024-05-20T18:35:31.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.561+0000 7f66011c3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f65fc06cad0 con 0x7f65fc15f700 2024-05-20T18:35:31.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.561+0000 7f65fa59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65fc0af030 0x7f65fc1f6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:31.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.561+0000 7f65fa59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65fc0af030 0x7f65fc1f6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37958/0 (socket says 172.21.15.5:37958) 2024-05-20T18:35:31.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.561+0000 7f65fa59c640 1 -- 172.21.15.5:0/4109643642 learned_addr learned my addr 172.21.15.5:0/4109643642 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:31.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.561+0000 7f65fad9d640 1 --2- 172.21.15.5:0/4109643642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65fc06c180 0x7f65fc15a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:31.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.561+0000 7f65fb59e640 1 --2- 172.21.15.5:0/4109643642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65fc15f700 0x7f65fc15e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:31.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.561+0000 7f65fa59c640 1 -- 172.21.15.5:0/4109643642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65fc15f700 msgr2=0x7f65fc15e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:31.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.561+0000 7f65fa59c640 1 --2- 172.21.15.5:0/4109643642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65fc15f700 0x7f65fc15e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:31.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.561+0000 7f65fa59c640 1 -- 172.21.15.5:0/4109643642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65fc06c180 msgr2=0x7f65fc15a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:31.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.561+0000 7f65fa59c640 1 --2- 172.21.15.5:0/4109643642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65fc06c180 0x7f65fc15a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:31.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.561+0000 7f65fa59c640 1 -- 172.21.15.5:0/4109643642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65fc15a8f0 con 0x7f65fc0af030 2024-05-20T18:35:31.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.561+0000 7f65fad9d640 1 --2- 172.21.15.5:0/4109643642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65fc06c180 0x7f65fc15a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:31.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.561+0000 7f65fa59c640 1 --2- 172.21.15.5:0/4109643642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65fc0af030 0x7f65fc1f6780 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f65f000e8a0 tx=0x7f65f000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:31.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.561+0000 7f65e37fe640 1 -- 172.21.15.5:0/4109643642 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f0017070 con 0x7f65fc0af030 2024-05-20T18:35:31.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.561+0000 7f65e37fe640 1 -- 172.21.15.5:0/4109643642 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65f000c940 con 0x7f65fc0af030 2024-05-20T18:35:31.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.562+0000 7f66011c3640 1 -- 172.21.15.5:0/4109643642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65fc15d4c0 con 0x7f65fc0af030 2024-05-20T18:35:31.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.562+0000 7f65e37fe640 1 -- 172.21.15.5:0/4109643642 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f000cd30 con 0x7f65fc0af030 2024-05-20T18:35:31.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.562+0000 7f66011c3640 1 -- 172.21.15.5:0/4109643642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65fc15da80 con 0x7f65fc0af030 2024-05-20T18:35:31.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.563+0000 7f66011c3640 1 -- 172.21.15.5:0/4109643642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65fc075b00 con 0x7f65fc0af030 2024-05-20T18:35:31.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.563+0000 7f65e37fe640 1 -- 172.21.15.5:0/4109643642 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f65f001a070 con 0x7f65fc0af030 2024-05-20T18:35:31.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.564+0000 7f65e37fe640 1 --2- 172.21.15.5:0/4109643642 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f65e806be60 0x7f65e806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:31.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.564+0000 7f65fad9d640 1 --2- 172.21.15.5:0/4109643642 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f65e806be60 0x7f65e806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:31.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.564+0000 7f65e37fe640 1 -- 172.21.15.5:0/4109643642 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f65f008af70 con 0x7f65fc0af030 2024-05-20T18:35:31.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.564+0000 7f65fad9d640 1 --2- 172.21.15.5:0/4109643642 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f65e806be60 0x7f65e806e370 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f65ec000f60 tx=0x7f65ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:31.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.569+0000 7f65e37fe640 1 -- 172.21.15.5:0/4109643642 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f65f0059680 con 0x7f65fc0af030 2024-05-20T18:35:31.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.753+0000 7f66011c3640 1 -- 172.21.15.5:0/4109643642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f65fc0b3b60 con 0x7f65fc0af030 2024-05-20T18:35:31.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.754+0000 7f65e37fe640 1 -- 172.21.15.5:0/4109643642 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+16777 (secure 0 0 0) 0x7f65f0058f80 con 0x7f65fc0af030 2024-05-20T18:35:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:31.756 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":104,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:35:30.778581+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:35:29.146063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-20T18:35:29.754813+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:35:31.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.756+0000 7f66011c3640 1 -- 172.21.15.5:0/4109643642 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f65e806be60 msgr2=0x7f65e806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:31.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.756+0000 7f66011c3640 1 --2- 172.21.15.5:0/4109643642 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f65e806be60 0x7f65e806e370 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f65ec000f60 tx=0x7f65ec009040 comp rx=0 tx=0).stop 2024-05-20T18:35:31.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.756+0000 7f66011c3640 1 -- 172.21.15.5:0/4109643642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65fc0af030 msgr2=0x7f65fc1f6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:31.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.756+0000 7f66011c3640 1 --2- 172.21.15.5:0/4109643642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65fc0af030 0x7f65fc1f6780 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f65f000e8a0 tx=0x7f65f000ed60 comp rx=0 tx=0).stop 2024-05-20T18:35:31.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.756+0000 7f66011c3640 1 -- 172.21.15.5:0/4109643642 shutdown_connections 2024-05-20T18:35:31.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.756+0000 7f66011c3640 1 --2- 172.21.15.5:0/4109643642 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f65e806be60 0x7f65e806e370 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:31.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.756+0000 7f66011c3640 1 --2- 172.21.15.5:0/4109643642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65fc15f700 0x7f65fc15e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:31.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.756+0000 7f66011c3640 1 --2- 172.21.15.5:0/4109643642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65fc0af030 0x7f65fc1f6780 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:31.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.756+0000 7f66011c3640 1 --2- 172.21.15.5:0/4109643642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65fc06c180 0x7f65fc15a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:31.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.756+0000 7f66011c3640 1 -- 172.21.15.5:0/4109643642 >> 172.21.15.5:0/4109643642 conn(0x7f65fc0b6620 msgr2=0x7f65fc0b6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:31.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.756+0000 7f66011c3640 1 -- 172.21.15.5:0/4109643642 shutdown_connections 2024-05-20T18:35:31.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.756+0000 7f66011c3640 1 -- 172.21.15.5:0/4109643642 wait complete. 2024-05-20T18:35:31.769 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:35:31.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.883+0000 7f44f234c640 1 Processor -- start 2024-05-20T18:35:31.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.883+0000 7f44f234c640 1 -- start start 2024-05-20T18:35:31.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.884+0000 7f44f234c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f44ec15f220 0x7f44ec0d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:31.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.884+0000 7f44f234c640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44ec0cda20 con 0x7f44ec0cfae0 2024-05-20T18:35:31.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.884+0000 7f44f234c640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44ec0cdd00 con 0x7f44ec0d0e00 2024-05-20T18:35:31.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.884+0000 7f44f234c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f44ec0cdfe0 con 0x7f44ec15f220 2024-05-20T18:35:31.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.885+0000 7f44f08e9640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f44ec0cfae0 0x7f44ec0ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36150/0 (socket says 172.21.15.5:36150) 2024-05-20T18:35:31.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.885+0000 7f44f08e9640 1 -- 172.21.15.5:0/3689039114 learned_addr learned my addr 172.21.15.5:0/3689039114 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:31.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.885+0000 7f44ebfff640 1 --2- 172.21.15.5:0/3689039114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f44ec15f220 0x7f44ec0d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:31.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.885+0000 7f44eaffd640 1 -- 172.21.15.5:0/3689039114 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1576143802 0 0) 0x7f44ec0cda20 con 0x7f44ec0cfae0 2024-05-20T18:35:31.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.885+0000 7f44eaffd640 1 -- 172.21.15.5:0/3689039114 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44d4003520 con 0x7f44ec0cfae0 2024-05-20T18:35:31.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.885+0000 7f44eaffd640 1 -- 172.21.15.5:0/3689039114 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 544937191 0 0) 0x7f44d4003520 con 0x7f44ec0cfae0 2024-05-20T18:35:31.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.885+0000 7f44eaffd640 1 -- 172.21.15.5:0/3689039114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f44ec15f220 msgr2=0x7f44ec0d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:31.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.885+0000 7f44eaffd640 1 --2- 172.21.15.5:0/3689039114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f44ec15f220 0x7f44ec0d08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:31.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.885+0000 7f44eaffd640 1 -- 172.21.15.5:0/3689039114 >> v1:172.21.15.63:6789/0 conn(0x7f44ec0d0e00 legacy=0x7f44ec0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:31.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.885+0000 7f44eaffd640 1 -- 172.21.15.5:0/3689039114 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ec0ce2c0 con 0x7f44ec0cfae0 2024-05-20T18:35:31.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.885+0000 7f44eaffd640 1 -- 172.21.15.5:0/3689039114 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f44e40019f0 con 0x7f44ec0cfae0 2024-05-20T18:35:31.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.885+0000 7f44eaffd640 1 -- 172.21.15.5:0/3689039114 >> v1:172.21.15.5:6789/0 conn(0x7f44ec0cfae0 legacy=0x7f44ec0ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:31.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.885+0000 7f44eaffd640 1 --2- 172.21.15.5:0/3689039114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f44ec15f220 0x7f44d40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:31.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.885+0000 7f44eaffd640 1 --2- 172.21.15.5:0/3689039114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44d4003f40 0x7f44d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:31.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.885+0000 7f44eaffd640 1 --2- 172.21.15.5:0/3689039114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4004d40 0x7f44d4005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:31.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.885+0000 7f44eaffd640 1 -- 172.21.15.5:0/3689039114 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec0cdfe0 con 0x7f44d4004d40 2024-05-20T18:35:31.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.885+0000 7f44eaffd640 1 -- 172.21.15.5:0/3689039114 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f44d400bb80 con 0x7f44d4003f40 2024-05-20T18:35:31.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.885+0000 7f44eaffd640 1 -- 172.21.15.5:0/3689039114 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f44d400bd80 con 0x7f44ec15f220 2024-05-20T18:35:31.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.885+0000 7f44f08e9640 1 --2- 172.21.15.5:0/3689039114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4004d40 0x7f44d4005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:31.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.886+0000 7f44eb7fe640 1 --2- 172.21.15.5:0/3689039114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44d4003f40 0x7f44d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:31.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.886+0000 7f44ebfff640 1 --2- 172.21.15.5:0/3689039114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f44ec15f220 0x7f44d40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:31.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.886+0000 7f44f08e9640 1 -- 172.21.15.5:0/3689039114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f44ec15f220 msgr2=0x7f44d40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:31.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.886+0000 7f44f08e9640 1 --2- 172.21.15.5:0/3689039114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f44ec15f220 0x7f44d40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:31.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.886+0000 7f44f08e9640 1 -- 172.21.15.5:0/3689039114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44d4003f40 msgr2=0x7f44d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:31.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.886+0000 7f44f08e9640 1 --2- 172.21.15.5:0/3689039114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44d4003f40 0x7f44d4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:31.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.886+0000 7f44f08e9640 1 -- 172.21.15.5:0/3689039114 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d4003520 con 0x7f44d4004d40 2024-05-20T18:35:31.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.887+0000 7f44f08e9640 1 --2- 172.21.15.5:0/3689039114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4004d40 0x7f44d4005150 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f44ec070140 tx=0x7f44d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dafcf9c0c11447 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:31.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.887+0000 7f44eaffd640 1 -- 172.21.15.5:0/3689039114 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e400a410 con 0x7f44d4004d40 2024-05-20T18:35:31.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.887+0000 7f44eaffd640 1 -- 172.21.15.5:0/3689039114 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44e402f070 con 0x7f44d4004d40 2024-05-20T18:35:31.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.887+0000 7f44eaffd640 1 -- 172.21.15.5:0/3689039114 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e402bbd0 con 0x7f44d4004d40 2024-05-20T18:35:31.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.887+0000 7f44f234c640 1 -- 172.21.15.5:0/3689039114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4004d40 msgr2=0x7f44d4005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:31.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.887+0000 7f44f234c640 1 --2- 172.21.15.5:0/3689039114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44d4004d40 0x7f44d4005150 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f44ec070140 tx=0x7f44d4005830 comp rx=0 tx=0).stop 2024-05-20T18:35:31.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.887+0000 7f44f08e9640 1 -- 172.21.15.5:0/3689039114 reap_dead start 2024-05-20T18:35:31.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.887+0000 7f44f234c640 1 -- 172.21.15.5:0/3689039114 shutdown_connections 2024-05-20T18:35:31.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.887+0000 7f44f234c640 1 -- 172.21.15.5:0/3689039114 >> 172.21.15.5:0/3689039114 conn(0x7f44ec0b44d0 msgr2=0x7f44ec0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:31.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.887+0000 7f44f234c640 1 -- 172.21.15.5:0/3689039114 shutdown_connections 2024-05-20T18:35:31.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.887+0000 7f44f234c640 1 -- 172.21.15.5:0/3689039114 wait complete. 2024-05-20T18:35:31.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.889+0000 7f44f234c640 1 Processor -- start 2024-05-20T18:35:31.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.889+0000 7f44f234c640 1 -- start start 2024-05-20T18:35:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.891+0000 7f44f234c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f44ec0c78b0 0x7f44ec0c64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.891+0000 7f44f234c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44ec150db0 0x7f44ec161cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.891+0000 7f44f234c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44ec1511c0 0x7f44ec162210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.891+0000 7f44f234c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec15f7b0 con 0x7f44ec1511c0 2024-05-20T18:35:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.891+0000 7f44f234c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec15f630 con 0x7f44ec150db0 2024-05-20T18:35:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.891+0000 7f44f234c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f44ec0ceef0 con 0x7f44ec0c78b0 2024-05-20T18:35:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.891+0000 7f44f08e9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44ec1511c0 0x7f44ec162210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.891+0000 7f44f08e9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44ec1511c0 0x7f44ec162210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37980/0 (socket says 172.21.15.5:37980) 2024-05-20T18:35:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.891+0000 7f44f08e9640 1 -- 172.21.15.5:0/2222243260 learned_addr learned my addr 172.21.15.5:0/2222243260 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.891+0000 7f44ebfff640 1 --2- 172.21.15.5:0/2222243260 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f44ec0c78b0 0x7f44ec0c64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.891+0000 7f44eb7fe640 1 --2- 172.21.15.5:0/2222243260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44ec150db0 0x7f44ec161cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:31.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.891+0000 7f44f08e9640 1 -- 172.21.15.5:0/2222243260 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f44ec0c78b0 msgr2=0x7f44ec0c64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:31.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.891+0000 7f44f08e9640 1 --2- 172.21.15.5:0/2222243260 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f44ec0c78b0 0x7f44ec0c64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:31.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.891+0000 7f44f08e9640 1 -- 172.21.15.5:0/2222243260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44ec150db0 msgr2=0x7f44ec161cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:31.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.891+0000 7f44f08e9640 1 --2- 172.21.15.5:0/2222243260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44ec150db0 0x7f44ec161cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:31.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.891+0000 7f44f08e9640 1 -- 172.21.15.5:0/2222243260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ec0c7420 con 0x7f44ec1511c0 2024-05-20T18:35:31.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.891+0000 7f44eb7fe640 1 --2- 172.21.15.5:0/2222243260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44ec150db0 0x7f44ec161cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:31.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.891+0000 7f44f08e9640 1 --2- 172.21.15.5:0/2222243260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44ec1511c0 0x7f44ec162210 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f44e400a880 tx=0x7f44ec0cded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:31.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.892+0000 7f44e8ff9640 1 -- 172.21.15.5:0/2222243260 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e4001f50 con 0x7f44ec1511c0 2024-05-20T18:35:31.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.892+0000 7f44e8ff9640 1 -- 172.21.15.5:0/2222243260 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44e400a3f0 con 0x7f44ec1511c0 2024-05-20T18:35:31.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.892+0000 7f44f234c640 1 -- 172.21.15.5:0/2222243260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44ec0c6b80 con 0x7f44ec1511c0 2024-05-20T18:35:31.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.892+0000 7f44e8ff9640 1 -- 172.21.15.5:0/2222243260 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e402be00 con 0x7f44ec1511c0 2024-05-20T18:35:31.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.892+0000 7f44f234c640 1 -- 172.21.15.5:0/2222243260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44ec0c7140 con 0x7f44ec1511c0 2024-05-20T18:35:31.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.893+0000 7f44f234c640 1 -- 172.21.15.5:0/2222243260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44ec075b00 con 0x7f44ec1511c0 2024-05-20T18:35:31.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.893+0000 7f44e8ff9640 1 -- 172.21.15.5:0/2222243260 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f44e403e030 con 0x7f44ec1511c0 2024-05-20T18:35:31.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.894+0000 7f44e8ff9640 1 --2- 172.21.15.5:0/2222243260 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f44d806be30 0x7f44d806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:31.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.894+0000 7f44e8ff9640 1 -- 172.21.15.5:0/2222243260 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f44e40aa300 con 0x7f44ec1511c0 2024-05-20T18:35:31.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.894+0000 7f44ebfff640 1 --2- 172.21.15.5:0/2222243260 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f44d806be30 0x7f44d806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:31.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.894+0000 7f44ebfff640 1 --2- 172.21.15.5:0/2222243260 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f44d806be30 0x7f44d806e340 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f44ec158050 tx=0x7f44dc009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:31.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:31.898+0000 7f44e8ff9640 1 -- 172.21.15.5:0/2222243260 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f44e4078c70 con 0x7f44ec1511c0 2024-05-20T18:35:32.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.084+0000 7f44f234c640 1 -- 172.21.15.5:0/2222243260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f44ec0b2690 con 0x7f44ec1511c0 2024-05-20T18:35:32.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.085+0000 7f44e8ff9640 1 -- 172.21.15.5:0/2222243260 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+16777 (secure 0 0 0) 0x7f44e4078570 con 0x7f44ec1511c0 2024-05-20T18:35:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:32.087 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":104,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:35:30.778581+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:35:29.146063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-20T18:35:29.754813+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:35:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.087+0000 7f44f234c640 1 -- 172.21.15.5:0/2222243260 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f44d806be30 msgr2=0x7f44d806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.087+0000 7f44f234c640 1 --2- 172.21.15.5:0/2222243260 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f44d806be30 0x7f44d806e340 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f44ec158050 tx=0x7f44dc009190 comp rx=0 tx=0).stop 2024-05-20T18:35:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.087+0000 7f44f234c640 1 -- 172.21.15.5:0/2222243260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44ec1511c0 msgr2=0x7f44ec162210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.087+0000 7f44f234c640 1 --2- 172.21.15.5:0/2222243260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44ec1511c0 0x7f44ec162210 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f44e400a880 tx=0x7f44ec0cded0 comp rx=0 tx=0).stop 2024-05-20T18:35:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.088+0000 7f44f234c640 1 -- 172.21.15.5:0/2222243260 shutdown_connections 2024-05-20T18:35:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.088+0000 7f44f234c640 1 --2- 172.21.15.5:0/2222243260 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f44d806be30 0x7f44d806e340 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.088+0000 7f44f234c640 1 --2- 172.21.15.5:0/2222243260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44ec1511c0 0x7f44ec162210 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.088+0000 7f44f234c640 1 --2- 172.21.15.5:0/2222243260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44ec150db0 0x7f44ec161cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.088+0000 7f44f234c640 1 --2- 172.21.15.5:0/2222243260 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f44ec0c78b0 0x7f44ec0c64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.088+0000 7f44f234c640 1 -- 172.21.15.5:0/2222243260 >> 172.21.15.5:0/2222243260 conn(0x7f44ec0b44d0 msgr2=0x7f44ec0d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.088+0000 7f44f234c640 1 -- 172.21.15.5:0/2222243260 shutdown_connections 2024-05-20T18:35:32.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.088+0000 7f44f234c640 1 -- 172.21.15.5:0/2222243260 wait complete. 2024-05-20T18:35:32.101 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-20T18:35:32.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.218+0000 7fb75b006640 1 Processor -- start 2024-05-20T18:35:32.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.218+0000 7fb75b006640 1 -- start start 2024-05-20T18:35:32.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.219+0000 7fb75b006640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb7540cd2c0 0x7fb7540b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:32.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.219+0000 7fb75b006640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7540cd6d0 con 0x7fb754160c10 2024-05-20T18:35:32.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.219+0000 7fb75b006640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7540cd950 con 0x7fb7540b4be0 2024-05-20T18:35:32.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.219+0000 7fb75b006640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb7540b52c0 con 0x7fb7540cd2c0 2024-05-20T18:35:32.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.219+0000 7fb758da2640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb754160c10 0x7fb7540b44d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36160/0 (socket says 172.21.15.5:36160) 2024-05-20T18:35:32.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.219+0000 7fb758da2640 1 -- 172.21.15.5:0/1900375343 learned_addr learned my addr 172.21.15.5:0/1900375343 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.219+0000 7fb7595a3640 1 --2- 172.21.15.5:0/1900375343 >> v2:172.21.15.63:3300/0 conn(0x7fb7540cd2c0 0x7fb7540b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.220+0000 7fb7537fe640 1 -- 172.21.15.5:0/1900375343 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2964787862 0 0) 0x7fb74c001110 con 0x7fb754160c10 2024-05-20T18:35:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.220+0000 7fb7537fe640 1 -- 172.21.15.5:0/1900375343 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb740003520 con 0x7fb754160c10 2024-05-20T18:35:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.220+0000 7fb7595a3640 1 -- 172.21.15.5:0/1900375343 >> v1:172.21.15.63:6789/0 conn(0x7fb7540b4be0 legacy=0x7fb7540d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.220+0000 7fb7537fe640 1 -- 172.21.15.5:0/1900375343 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2580898809 0 0) 0x7fb740003520 con 0x7fb754160c10 2024-05-20T18:35:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.220+0000 7fb7595a3640 1 -- 172.21.15.5:0/1900375343 >> v1:172.21.15.5:6789/0 conn(0x7fb754160c10 legacy=0x7fb7540b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:32.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.220+0000 7fb7595a3640 1 -- 172.21.15.5:0/1900375343 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7540cdc30 con 0x7fb7540cd2c0 2024-05-20T18:35:32.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.221+0000 7fb7595a3640 1 --2- 172.21.15.5:0/1900375343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7540cd2c0 0x7fb7540b5cd0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb73c0097a0 tx=0x7fb73c02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62456237695235f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:32.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.221+0000 7fb7537fe640 1 -- 172.21.15.5:0/1900375343 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73c037070 con 0x7fb7540cd2c0 2024-05-20T18:35:32.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.221+0000 7fb7537fe640 1 -- 172.21.15.5:0/1900375343 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb73c02abc0 con 0x7fb7540cd2c0 2024-05-20T18:35:32.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.221+0000 7fb7537fe640 1 -- 172.21.15.5:0/1900375343 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c001840 con 0x7fb7540cd2c0 2024-05-20T18:35:32.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.222+0000 7fb75b006640 1 -- 172.21.15.5:0/1900375343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7540cd2c0 msgr2=0x7fb7540b5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:32.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.222+0000 7fb75b006640 1 --2- 172.21.15.5:0/1900375343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7540cd2c0 0x7fb7540b5cd0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb73c0097a0 tx=0x7fb73c02a130 comp rx=0 tx=0).stop 2024-05-20T18:35:32.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.222+0000 7fb75b006640 1 -- 172.21.15.5:0/1900375343 shutdown_connections 2024-05-20T18:35:32.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.222+0000 7fb75b006640 1 --2- 172.21.15.5:0/1900375343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7540cd2c0 0x7fb7540b5cd0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:32.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.222+0000 7fb75b006640 1 -- 172.21.15.5:0/1900375343 >> 172.21.15.5:0/1900375343 conn(0x7fb7540d29d0 msgr2=0x7fb7540d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:32.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.222+0000 7fb75b006640 1 -- 172.21.15.5:0/1900375343 shutdown_connections 2024-05-20T18:35:32.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.222+0000 7fb75b006640 1 -- 172.21.15.5:0/1900375343 wait complete. 2024-05-20T18:35:32.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.224+0000 7fb75b006640 1 Processor -- start 2024-05-20T18:35:32.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.224+0000 7fb75b006640 1 -- start start 2024-05-20T18:35:32.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.225+0000 7fb75b006640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7540cd2c0 0x7fb7541e6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:32.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.225+0000 7fb75b006640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb754160c10 0x7fb75414ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:32.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.225+0000 7fb75b006640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb754159ff0 0x7fb7540cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:32.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.225+0000 7fb75b006640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb7540ce1f0 con 0x7fb7540cd2c0 2024-05-20T18:35:32.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.225+0000 7fb75b006640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb7540d14f0 con 0x7fb754159ff0 2024-05-20T18:35:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.225+0000 7fb75b006640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb75415d010 con 0x7fb754160c10 2024-05-20T18:35:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.225+0000 7fb758da2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7540cd2c0 0x7fb7541e6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.225+0000 7fb758da2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7540cd2c0 0x7fb7541e6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37982/0 (socket says 172.21.15.5:37982) 2024-05-20T18:35:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.225+0000 7fb758da2640 1 -- 172.21.15.5:0/2877005521 learned_addr learned my addr 172.21.15.5:0/2877005521 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.225+0000 7fb7595a3640 1 --2- 172.21.15.5:0/2877005521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb754159ff0 0x7fb7540cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.225+0000 7fb753fff640 1 --2- 172.21.15.5:0/2877005521 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb754160c10 0x7fb75414ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.226+0000 7fb758da2640 1 -- 172.21.15.5:0/2877005521 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb754160c10 msgr2=0x7fb75414ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.226+0000 7fb758da2640 1 --2- 172.21.15.5:0/2877005521 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb754160c10 0x7fb75414ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.226+0000 7fb758da2640 1 -- 172.21.15.5:0/2877005521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb754159ff0 msgr2=0x7fb7540cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.226+0000 7fb758da2640 1 --2- 172.21.15.5:0/2877005521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb754159ff0 0x7fb7540cbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.226+0000 7fb758da2640 1 -- 172.21.15.5:0/2877005521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75414d3c0 con 0x7fb7540cd2c0 2024-05-20T18:35:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.226+0000 7fb758da2640 1 --2- 172.21.15.5:0/2877005521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7540cd2c0 0x7fb7541e6a30 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00c4e0 tx=0x7fb74c00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.226+0000 7fb7517fa640 1 -- 172.21.15.5:0/2877005521 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c017070 con 0x7fb7540cd2c0 2024-05-20T18:35:32.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.226+0000 7fb7517fa640 1 -- 172.21.15.5:0/2877005521 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb74c005ce0 con 0x7fb7540cd2c0 2024-05-20T18:35:32.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.226+0000 7fb75b006640 1 -- 172.21.15.5:0/2877005521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7541e7330 con 0x7fb7540cd2c0 2024-05-20T18:35:32.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.226+0000 7fb7517fa640 1 -- 172.21.15.5:0/2877005521 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c0072b0 con 0x7fb7540cd2c0 2024-05-20T18:35:32.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.226+0000 7fb75b006640 1 -- 172.21.15.5:0/2877005521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7541e78f0 con 0x7fb7540cd2c0 2024-05-20T18:35:32.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.227+0000 7fb75b006640 1 -- 172.21.15.5:0/2877005521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb754075b00 con 0x7fb7540cd2c0 2024-05-20T18:35:32.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.228+0000 7fb7517fa640 1 -- 172.21.15.5:0/2877005521 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb74c01a070 con 0x7fb7540cd2c0 2024-05-20T18:35:32.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.228+0000 7fb7517fa640 1 --2- 172.21.15.5:0/2877005521 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb73406be30 0x7fb73406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:32.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.228+0000 7fb7517fa640 1 -- 172.21.15.5:0/2877005521 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fb74c08a410 con 0x7fb7540cd2c0 2024-05-20T18:35:32.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.228+0000 7fb753fff640 1 --2- 172.21.15.5:0/2877005521 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb73406be30 0x7fb73406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:32.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.229+0000 7fb753fff640 1 --2- 172.21.15.5:0/2877005521 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb73406be30 0x7fb73406e340 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb7540b4c10 tx=0x7fb744009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:32.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.233+0000 7fb7517fa640 1 -- 172.21.15.5:0/2877005521 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb7540cd6d0 con 0x7fb7540cd2c0 2024-05-20T18:35:32.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:32.457+0000 7fb75b006640 1 -- 172.21.15.5:0/2877005521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb75415c850 con 0x7fb7540cd2c0 2024-05-20T18:35:33.383 INFO:tasks.ceph.mds.b.smithi063.stderr:2024-05-20T18:35:33.383+0000 7f457c682640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T18:35:33.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.427+0000 7fb7517fa640 1 -- 172.21.15.5:0/2877005521 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v81) ==== 118+0+0 (secure 0 0 0) 0x7fb74c058550 con 0x7fb7540cd2c0 2024-05-20T18:35:33.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.430+0000 7fb75b006640 1 -- 172.21.15.5:0/2877005521 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb73406be30 msgr2=0x7fb73406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:33.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.430+0000 7fb75b006640 1 --2- 172.21.15.5:0/2877005521 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb73406be30 0x7fb73406e340 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb7540b4c10 tx=0x7fb744009040 comp rx=0 tx=0).stop 2024-05-20T18:35:33.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.430+0000 7fb75b006640 1 -- 172.21.15.5:0/2877005521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7540cd2c0 msgr2=0x7fb7541e6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:33.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.430+0000 7fb75b006640 1 --2- 172.21.15.5:0/2877005521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7540cd2c0 0x7fb7541e6a30 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00c4e0 tx=0x7fb74c00c9a0 comp rx=0 tx=0).stop 2024-05-20T18:35:33.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.430+0000 7fb75b006640 1 -- 172.21.15.5:0/2877005521 shutdown_connections 2024-05-20T18:35:33.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.430+0000 7fb75b006640 1 --2- 172.21.15.5:0/2877005521 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb73406be30 0x7fb73406e340 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:33.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.430+0000 7fb75b006640 1 --2- 172.21.15.5:0/2877005521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb754159ff0 0x7fb7540cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:33.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.430+0000 7fb75b006640 1 --2- 172.21.15.5:0/2877005521 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb754160c10 0x7fb75414ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:33.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.430+0000 7fb75b006640 1 --2- 172.21.15.5:0/2877005521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7540cd2c0 0x7fb7541e6a30 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:33.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.430+0000 7fb75b006640 1 -- 172.21.15.5:0/2877005521 >> 172.21.15.5:0/2877005521 conn(0x7fb7540d29d0 msgr2=0x7fb7540708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:33.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.430+0000 7fb75b006640 1 -- 172.21.15.5:0/2877005521 shutdown_connections 2024-05-20T18:35:33.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.430+0000 7fb75b006640 1 -- 172.21.15.5:0/2877005521 wait complete. 2024-05-20T18:35:33.445 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-20T18:35:33.446 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-20T18:35:33.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.569+0000 7fb8866a2640 1 Processor -- start 2024-05-20T18:35:33.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.569+0000 7fb8866a2640 1 -- start start 2024-05-20T18:35:33.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.570+0000 7fb8866a2640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb8801599a0 0x7fb8800b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:33.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.570+0000 7fb8866a2640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb88006d980 0x7fb88009a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:33.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.570+0000 7fb8866a2640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb88009aa30 0x7fb8800d18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:33.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.570+0000 7fb8866a2640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fb88009ae40 con 0x7fb8801599a0 2024-05-20T18:35:33.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.570+0000 7fb8866a2640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb8801607a0 con 0x7fb88009aa30 2024-05-20T18:35:33.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.570+0000 7fb8866a2640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb88009b2d0 con 0x7fb88006d980 2024-05-20T18:35:33.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.571+0000 7fb87ffff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb8801599a0 0x7fb8800b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:33.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.571+0000 7fb87ffff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb8801599a0 0x7fb8800b1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37994/0 (socket says 172.21.15.5:37994) 2024-05-20T18:35:33.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.571+0000 7fb87ffff640 1 -- 172.21.15.5:0/1015096749 learned_addr learned my addr 172.21.15.5:0/1015096749 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:33.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.571+0000 7fb87f7fe640 1 --2- 172.21.15.5:0/1015096749 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb88006d980 0x7fb88009a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:33.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.571+0000 7fb884c3f640 1 --2- 172.21.15.5:0/1015096749 >> v2:172.21.15.63:3300/0 conn(0x7fb88009aa30 0x7fb8800d18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:33.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.571+0000 7fb87ffff640 1 -- 172.21.15.5:0/1015096749 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb88006d980 msgr2=0x7fb88009a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:33.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.571+0000 7fb87ffff640 1 --2- 172.21.15.5:0/1015096749 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb88006d980 0x7fb88009a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:33.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.571+0000 7fb87ffff640 1 -- 172.21.15.5:0/1015096749 >> v2:172.21.15.63:3300/0 conn(0x7fb88009aa30 msgr2=0x7fb8800d18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:33.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.571+0000 7fb87ffff640 1 --2- 172.21.15.5:0/1015096749 >> v2:172.21.15.63:3300/0 conn(0x7fb88009aa30 0x7fb8800d18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:33.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.571+0000 7fb87ffff640 1 -- 172.21.15.5:0/1015096749 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88015fc20 con 0x7fb8801599a0 2024-05-20T18:35:33.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.572+0000 7fb87ffff640 1 --2- 172.21.15.5:0/1015096749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8801599a0 0x7fb8800b1cc0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fb8800b4880 tx=0x7fb87002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a74e71b5837764 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:33.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.572+0000 7fb87effd640 1 -- 172.21.15.5:0/1015096749 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870037070 con 0x7fb8801599a0 2024-05-20T18:35:33.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.572+0000 7fb87effd640 1 -- 172.21.15.5:0/1015096749 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb870032440 con 0x7fb8801599a0 2024-05-20T18:35:33.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.572+0000 7fb87effd640 1 -- 172.21.15.5:0/1015096749 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870032810 con 0x7fb8801599a0 2024-05-20T18:35:33.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.573+0000 7fb8866a2640 1 -- 172.21.15.5:0/1015096749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8801599a0 msgr2=0x7fb8800b1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:33.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.573+0000 7fb8866a2640 1 --2- 172.21.15.5:0/1015096749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8801599a0 0x7fb8800b1cc0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fb8800b4880 tx=0x7fb87002a140 comp rx=0 tx=0).stop 2024-05-20T18:35:33.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.573+0000 7fb8866a2640 1 -- 172.21.15.5:0/1015096749 shutdown_connections 2024-05-20T18:35:33.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.573+0000 7fb8866a2640 1 --2- 172.21.15.5:0/1015096749 >> v2:172.21.15.63:3300/0 conn(0x7fb88009aa30 0x7fb8800d18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:33.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.573+0000 7fb8866a2640 1 --2- 172.21.15.5:0/1015096749 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb88006d980 0x7fb88009a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:33.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.573+0000 7fb8866a2640 1 --2- 172.21.15.5:0/1015096749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8801599a0 0x7fb8800b1cc0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:33.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.573+0000 7fb8866a2640 1 -- 172.21.15.5:0/1015096749 >> 172.21.15.5:0/1015096749 conn(0x7fb8800b6670 msgr2=0x7fb8800b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:33.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.573+0000 7fb8866a2640 1 -- 172.21.15.5:0/1015096749 shutdown_connections 2024-05-20T18:35:33.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.573+0000 7fb8866a2640 1 -- 172.21.15.5:0/1015096749 wait complete. 2024-05-20T18:35:33.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.574+0000 7fb8866a2640 1 Processor -- start 2024-05-20T18:35:33.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.575+0000 7fb8866a2640 1 -- start start 2024-05-20T18:35:33.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.576+0000 7fb8866a2640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb88006d980 0x7fb880159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:33.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.576+0000 7fb8866a2640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb88009aa30 0x7fb88015acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:33.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.576+0000 7fb8866a2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8801599a0 0x7fb880149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:33.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.576+0000 7fb8866a2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb8801604c0 con 0x7fb8801599a0 2024-05-20T18:35:33.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.576+0000 7fb8866a2640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb8801601e0 con 0x7fb88006d980 2024-05-20T18:35:33.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.576+0000 7fb8866a2640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb88006dd90 con 0x7fb88009aa30 2024-05-20T18:35:33.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.576+0000 7fb884c3f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8801599a0 0x7fb880149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:33.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.576+0000 7fb884c3f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8801599a0 0x7fb880149dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37996/0 (socket says 172.21.15.5:37996) 2024-05-20T18:35:33.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.576+0000 7fb884c3f640 1 -- 172.21.15.5:0/3447290377 learned_addr learned my addr 172.21.15.5:0/3447290377 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:33.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.576+0000 7fb87f7fe640 1 --2- 172.21.15.5:0/3447290377 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb88009aa30 0x7fb88015acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:33.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.576+0000 7fb87ffff640 1 --2- 172.21.15.5:0/3447290377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb88006d980 0x7fb880159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:33.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.577+0000 7fb884c3f640 1 -- 172.21.15.5:0/3447290377 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb88009aa30 msgr2=0x7fb88015acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:33.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.577+0000 7fb884c3f640 1 --2- 172.21.15.5:0/3447290377 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb88009aa30 0x7fb88015acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:33.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.577+0000 7fb884c3f640 1 -- 172.21.15.5:0/3447290377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb88006d980 msgr2=0x7fb880159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:33.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.577+0000 7fb884c3f640 1 --2- 172.21.15.5:0/3447290377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb88006d980 0x7fb880159db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:33.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.577+0000 7fb884c3f640 1 -- 172.21.15.5:0/3447290377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88015a2f0 con 0x7fb8801599a0 2024-05-20T18:35:33.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.577+0000 7fb87ffff640 1 --2- 172.21.15.5:0/3447290377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb88006d980 0x7fb880159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:33.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.577+0000 7fb884c3f640 1 --2- 172.21.15.5:0/3447290377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8801599a0 0x7fb880149dc0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fb878007d40 tx=0x7fb87800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:33.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.577+0000 7fb87cff9640 1 -- 172.21.15.5:0/3447290377 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb878014420 con 0x7fb8801599a0 2024-05-20T18:35:33.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.577+0000 7fb87cff9640 1 -- 172.21.15.5:0/3447290377 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb878019070 con 0x7fb8801599a0 2024-05-20T18:35:33.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.577+0000 7fb87cff9640 1 -- 172.21.15.5:0/3447290377 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb878014820 con 0x7fb8801599a0 2024-05-20T18:35:33.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.577+0000 7fb8866a2640 1 -- 172.21.15.5:0/3447290377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb88015edf0 con 0x7fb8801599a0 2024-05-20T18:35:33.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.577+0000 7fb8866a2640 1 -- 172.21.15.5:0/3447290377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb88015f3b0 con 0x7fb8801599a0 2024-05-20T18:35:33.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.578+0000 7fb8866a2640 1 -- 172.21.15.5:0/3447290377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb88015eb60 con 0x7fb8801599a0 2024-05-20T18:35:33.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.579+0000 7fb87cff9640 1 -- 172.21.15.5:0/3447290377 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb8780149c0 con 0x7fb8801599a0 2024-05-20T18:35:33.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.579+0000 7fb87cff9640 1 --2- 172.21.15.5:0/3447290377 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb86c06be30 0x7fb86c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:33.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.579+0000 7fb87cff9640 1 -- 172.21.15.5:0/3447290377 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7fb87808b870 con 0x7fb8801599a0 2024-05-20T18:35:33.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.580+0000 7fb87ffff640 1 --2- 172.21.15.5:0/3447290377 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb86c06be30 0x7fb86c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:33.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.580+0000 7fb87ffff640 1 --2- 172.21.15.5:0/3447290377 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb86c06be30 0x7fb86c06e340 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb870034000 tx=0x7fb87002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:33.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.584+0000 7fb87cff9640 1 -- 172.21.15.5:0/3447290377 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb878059f70 con 0x7fb8801599a0 2024-05-20T18:35:33.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:33.804+0000 7fb8866a2640 1 -- 172.21.15.5:0/3447290377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb88015e8f0 con 0x7fb8801599a0 2024-05-20T18:35:34.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.428+0000 7fb87cff9640 1 -- 172.21.15.5:0/3447290377 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v82) ==== 122+0+0 (secure 0 0 0) 0x7fb878059870 con 0x7fb8801599a0 2024-05-20T18:35:34.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.431+0000 7fb8866a2640 1 -- 172.21.15.5:0/3447290377 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb86c06be30 msgr2=0x7fb86c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:34.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.431+0000 7fb8866a2640 1 --2- 172.21.15.5:0/3447290377 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb86c06be30 0x7fb86c06e340 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb870034000 tx=0x7fb87002e000 comp rx=0 tx=0).stop 2024-05-20T18:35:34.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.431+0000 7fb8866a2640 1 -- 172.21.15.5:0/3447290377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8801599a0 msgr2=0x7fb880149dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:34.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.431+0000 7fb8866a2640 1 --2- 172.21.15.5:0/3447290377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8801599a0 0x7fb880149dc0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fb878007d40 tx=0x7fb87800e4b0 comp rx=0 tx=0).stop 2024-05-20T18:35:34.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.431+0000 7fb8866a2640 1 -- 172.21.15.5:0/3447290377 shutdown_connections 2024-05-20T18:35:34.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.431+0000 7fb8866a2640 1 --2- 172.21.15.5:0/3447290377 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb86c06be30 0x7fb86c06e340 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:34.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.431+0000 7fb8866a2640 1 --2- 172.21.15.5:0/3447290377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8801599a0 0x7fb880149dc0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:34.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.431+0000 7fb8866a2640 1 --2- 172.21.15.5:0/3447290377 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb88009aa30 0x7fb88015acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:34.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.431+0000 7fb8866a2640 1 --2- 172.21.15.5:0/3447290377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb88006d980 0x7fb880159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:34.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.431+0000 7fb8866a2640 1 -- 172.21.15.5:0/3447290377 >> 172.21.15.5:0/3447290377 conn(0x7fb8800b6670 msgr2=0x7fb8800b34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:34.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.431+0000 7fb8866a2640 1 -- 172.21.15.5:0/3447290377 shutdown_connections 2024-05-20T18:35:34.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.431+0000 7fb8866a2640 1 -- 172.21.15.5:0/3447290377 wait complete. 2024-05-20T18:35:34.446 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-20T18:35:34.447 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:35:34.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.568+0000 7fb474644640 1 Processor -- start 2024-05-20T18:35:34.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.568+0000 7fb474644640 1 -- start start 2024-05-20T18:35:34.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.569+0000 7fb474644640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb46c0cf3d0 0x7fb46c0cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:34.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.569+0000 7fb474644640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb46c0cfd20 con 0x7fb46c0ce7e0 2024-05-20T18:35:34.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.569+0000 7fb474644640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb46c0cda20 con 0x7fb46c15f220 2024-05-20T18:35:34.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.569+0000 7fb474644640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb46c0cdd00 con 0x7fb46c0cf3d0 2024-05-20T18:35:34.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.570+0000 7fb472be1640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb46c0ce7e0 0x7fb46c0cebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36174/0 (socket says 172.21.15.5:36174) 2024-05-20T18:35:34.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.570+0000 7fb472be1640 1 -- 172.21.15.5:0/3573249630 learned_addr learned my addr 172.21.15.5:0/3573249630 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:34.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.570+0000 7fb471bdf640 1 --2- 172.21.15.5:0/3573249630 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb46c0cf3d0 0x7fb46c0cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:34.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.570+0000 7fb4713de640 1 -- 172.21.15.5:0/3573249630 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1902268357 0 0) 0x7fb464001490 con 0x7fb46c0ce7e0 2024-05-20T18:35:34.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.570+0000 7fb4713de640 1 -- 172.21.15.5:0/3573249630 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb458003520 con 0x7fb46c0ce7e0 2024-05-20T18:35:34.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.570+0000 7fb4713de640 1 -- 172.21.15.5:0/3573249630 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3942691101 0 0) 0x7fb46c0cda20 con 0x7fb46c15f220 2024-05-20T18:35:34.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.570+0000 7fb4713de640 1 -- 172.21.15.5:0/3573249630 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb464001490 con 0x7fb46c15f220 2024-05-20T18:35:34.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.570+0000 7fb471bdf640 1 -- 172.21.15.5:0/3573249630 >> v1:172.21.15.63:6789/0 conn(0x7fb46c15f220 legacy=0x7fb46c0d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:34.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.570+0000 7fb471bdf640 1 -- 172.21.15.5:0/3573249630 >> v1:172.21.15.5:6789/0 conn(0x7fb46c0ce7e0 legacy=0x7fb46c0cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:34.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.570+0000 7fb4713de640 1 -- 172.21.15.5:0/3573249630 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 483716969 0 0) 0x7fb458003520 con 0x7fb46c0ce7e0 2024-05-20T18:35:34.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.570+0000 7fb471bdf640 1 -- 172.21.15.5:0/3573249630 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46c0cdfe0 con 0x7fb46c0cf3d0 2024-05-20T18:35:34.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.571+0000 7fb471bdf640 1 --2- 172.21.15.5:0/3573249630 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb46c0cf3d0 0x7fb46c0cf7e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0097d0 tx=0x7fb45c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f580b297d29c7e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:34.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.571+0000 7fb4713de640 1 -- 172.21.15.5:0/3573249630 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c037070 con 0x7fb46c0cf3d0 2024-05-20T18:35:34.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.571+0000 7fb4713de640 1 -- 172.21.15.5:0/3573249630 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb45c02ab90 con 0x7fb46c0cf3d0 2024-05-20T18:35:34.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.571+0000 7fb4713de640 1 -- 172.21.15.5:0/3573249630 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb464001bc0 con 0x7fb46c0cf3d0 2024-05-20T18:35:34.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.572+0000 7fb474644640 1 -- 172.21.15.5:0/3573249630 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb46c0cf3d0 msgr2=0x7fb46c0cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:34.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.572+0000 7fb474644640 1 --2- 172.21.15.5:0/3573249630 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb46c0cf3d0 0x7fb46c0cf7e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0097d0 tx=0x7fb45c02a100 comp rx=0 tx=0).stop 2024-05-20T18:35:34.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.572+0000 7fb474644640 1 -- 172.21.15.5:0/3573249630 shutdown_connections 2024-05-20T18:35:34.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.572+0000 7fb474644640 1 --2- 172.21.15.5:0/3573249630 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb46c0cf3d0 0x7fb46c0cf7e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:34.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.572+0000 7fb474644640 1 -- 172.21.15.5:0/3573249630 >> 172.21.15.5:0/3573249630 conn(0x7fb46c0b44d0 msgr2=0x7fb46c0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:34.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.572+0000 7fb474644640 1 -- 172.21.15.5:0/3573249630 shutdown_connections 2024-05-20T18:35:34.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.572+0000 7fb474644640 1 -- 172.21.15.5:0/3573249630 wait complete. 2024-05-20T18:35:34.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.574+0000 7fb474644640 1 Processor -- start 2024-05-20T18:35:34.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.574+0000 7fb474644640 1 -- start start 2024-05-20T18:35:34.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.575+0000 7fb474644640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb46c0ce7e0 0x7fb46c147eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:34.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.575+0000 7fb474644640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c0cf3d0 0x7fb46c14b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:34.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.575+0000 7fb474644640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb46c15f220 0x7fb46c15d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:34.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.575+0000 7fb474644640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c0cd680 con 0x7fb46c0cf3d0 2024-05-20T18:35:34.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.575+0000 7fb474644640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c0d0fd0 con 0x7fb46c15f220 2024-05-20T18:35:34.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.575+0000 7fb474644640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb46c0cd800 con 0x7fb46c0ce7e0 2024-05-20T18:35:34.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.575+0000 7fb471bdf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c0cf3d0 0x7fb46c14b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:34.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.575+0000 7fb471bdf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c0cf3d0 0x7fb46c14b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38006/0 (socket says 172.21.15.5:38006) 2024-05-20T18:35:34.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.575+0000 7fb471bdf640 1 -- 172.21.15.5:0/2012217915 learned_addr learned my addr 172.21.15.5:0/2012217915 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:34.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.576+0000 7fb472be1640 1 --2- 172.21.15.5:0/2012217915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb46c15f220 0x7fb46c15d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:34.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.576+0000 7fb4723e0640 1 --2- 172.21.15.5:0/2012217915 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb46c0ce7e0 0x7fb46c147eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:34.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.576+0000 7fb471bdf640 1 -- 172.21.15.5:0/2012217915 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb46c0ce7e0 msgr2=0x7fb46c147eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:34.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.576+0000 7fb471bdf640 1 --2- 172.21.15.5:0/2012217915 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb46c0ce7e0 0x7fb46c147eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:34.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.576+0000 7fb471bdf640 1 -- 172.21.15.5:0/2012217915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb46c15f220 msgr2=0x7fb46c15d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:34.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.576+0000 7fb471bdf640 1 --2- 172.21.15.5:0/2012217915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb46c15f220 0x7fb46c15d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:34.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.576+0000 7fb471bdf640 1 -- 172.21.15.5:0/2012217915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46c0b5b50 con 0x7fb46c0cf3d0 2024-05-20T18:35:34.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.576+0000 7fb471bdf640 1 --2- 172.21.15.5:0/2012217915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c0cf3d0 0x7fb46c14b070 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0097d0 tx=0x7fb45c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:34.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.576+0000 7fb462ffd640 1 -- 172.21.15.5:0/2012217915 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c042020 con 0x7fb46c0cf3d0 2024-05-20T18:35:34.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.576+0000 7fb462ffd640 1 -- 172.21.15.5:0/2012217915 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb45c03d670 con 0x7fb46c0cf3d0 2024-05-20T18:35:34.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.576+0000 7fb474644640 1 -- 172.21.15.5:0/2012217915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb46c0b5e30 con 0x7fb46c0cf3d0 2024-05-20T18:35:34.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.576+0000 7fb462ffd640 1 -- 172.21.15.5:0/2012217915 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c037040 con 0x7fb46c0cf3d0 2024-05-20T18:35:34.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.576+0000 7fb474644640 1 -- 172.21.15.5:0/2012217915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb46c0b63f0 con 0x7fb46c0cf3d0 2024-05-20T18:35:34.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.577+0000 7fb474644640 1 -- 172.21.15.5:0/2012217915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb46c06d5c0 con 0x7fb46c0cf3d0 2024-05-20T18:35:34.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.578+0000 7fb462ffd640 1 -- 172.21.15.5:0/2012217915 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb45c045020 con 0x7fb46c0cf3d0 2024-05-20T18:35:34.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.578+0000 7fb462ffd640 1 --2- 172.21.15.5:0/2012217915 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb45006be30 0x7fb45006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:34.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.579+0000 7fb4723e0640 1 --2- 172.21.15.5:0/2012217915 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb45006be30 0x7fb45006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:34.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.579+0000 7fb462ffd640 1 -- 172.21.15.5:0/2012217915 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7fb45c0ab8d0 con 0x7fb46c0cf3d0 2024-05-20T18:35:34.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.579+0000 7fb4723e0640 1 --2- 172.21.15.5:0/2012217915 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb45006be30 0x7fb45006e340 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb4680011e0 tx=0x7fb468009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:34.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.583+0000 7fb462ffd640 1 -- 172.21.15.5:0/2012217915 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb46c06d5c0 con 0x7fb46c0cf3d0 2024-05-20T18:35:34.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.806+0000 7fb474644640 1 -- 172.21.15.5:0/2012217915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb46c075b00 con 0x7fb46c0cf3d0 2024-05-20T18:35:34.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.806+0000 7fb462ffd640 1 -- 172.21.15.5:0/2012217915 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5177 (secure 0 0 0) 0x7fb45c04c030 con 0x7fb46c0cf3d0 2024-05-20T18:35:34.808 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:34.808 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6758,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/4163145158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":4163145158},{"type":"v1","addr":"172.21.15.63:6834","nonce":4163145158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7528,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3049443532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3049443532},{"type":"v1","addr":"172.21.15.5:6837","nonce":3049443532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3552410668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3552410668},{"type":"v1","addr":"172.21.15.5:6835","nonce":3552410668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:35:30.778930+0000","modified":"2024-05-20T18:35:33.806381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8343},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8343":{"gid":8343,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/2054897241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":2054897241},{"type":"v1","addr":"172.21.15.63:6835","nonce":2054897241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8343,"qdb_cluster":[8343]},"id":7}]} 2024-05-20T18:35:34.808 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 82 2024-05-20T18:35:34.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.808+0000 7fb474644640 1 -- 172.21.15.5:0/2012217915 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb45006be30 msgr2=0x7fb45006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:34.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.808+0000 7fb474644640 1 --2- 172.21.15.5:0/2012217915 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb45006be30 0x7fb45006e340 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb4680011e0 tx=0x7fb468009040 comp rx=0 tx=0).stop 2024-05-20T18:35:34.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.808+0000 7fb474644640 1 -- 172.21.15.5:0/2012217915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c0cf3d0 msgr2=0x7fb46c14b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:34.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.808+0000 7fb474644640 1 --2- 172.21.15.5:0/2012217915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c0cf3d0 0x7fb46c14b070 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0097d0 tx=0x7fb45c0046c0 comp rx=0 tx=0).stop 2024-05-20T18:35:34.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.808+0000 7fb474644640 1 -- 172.21.15.5:0/2012217915 shutdown_connections 2024-05-20T18:35:34.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.808+0000 7fb474644640 1 --2- 172.21.15.5:0/2012217915 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb45006be30 0x7fb45006e340 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:34.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.808+0000 7fb474644640 1 --2- 172.21.15.5:0/2012217915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb46c15f220 0x7fb46c15d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:34.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.808+0000 7fb474644640 1 --2- 172.21.15.5:0/2012217915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c0cf3d0 0x7fb46c14b070 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:34.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.808+0000 7fb474644640 1 --2- 172.21.15.5:0/2012217915 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb46c0ce7e0 0x7fb46c147eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:34.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.808+0000 7fb474644640 1 -- 172.21.15.5:0/2012217915 >> 172.21.15.5:0/2012217915 conn(0x7fb46c0b44d0 msgr2=0x7fb46c0d08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:34.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.809+0000 7fb474644640 1 -- 172.21.15.5:0/2012217915 shutdown_connections 2024-05-20T18:35:34.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.809+0000 7fb474644640 1 -- 172.21.15.5:0/2012217915 wait complete. 2024-05-20T18:35:34.821 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:35:34.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.935+0000 7ff5e0c5a640 1 Processor -- start 2024-05-20T18:35:34.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.935+0000 7ff5e0c5a640 1 -- start start 2024-05-20T18:35:34.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.936+0000 7ff5e0c5a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff5dc0d17f0 0x7ff5dc15aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:34.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.936+0000 7ff5e0c5a640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5dc0cd580 con 0x7ff5dc0b1e10 2024-05-20T18:35:34.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.936+0000 7ff5e0c5a640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5dc0cd860 con 0x7ff5dc15d530 2024-05-20T18:35:34.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.936+0000 7ff5e0c5a640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff5dc0ce190 con 0x7ff5dc0d17f0 2024-05-20T18:35:34.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.936+0000 7ff5d9d9b640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff5dc0b1e10 0x7ff5dc070850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46698/0 (socket says 172.21.15.5:46698) 2024-05-20T18:35:34.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.936+0000 7ff5d9d9b640 1 -- 172.21.15.5:0/4289071170 learned_addr learned my addr 172.21.15.5:0/4289071170 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:34.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.937+0000 7ff5dad9d640 1 --2- 172.21.15.5:0/4289071170 >> v2:172.21.15.63:3300/0 conn(0x7ff5dc0d17f0 0x7ff5dc15aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:34.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.937+0000 7ff5d959a640 1 -- 172.21.15.5:0/4289071170 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2403435783 0 0) 0x7ff5dc0cd580 con 0x7ff5dc0b1e10 2024-05-20T18:35:34.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.937+0000 7ff5d959a640 1 -- 172.21.15.5:0/4289071170 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5c8003520 con 0x7ff5dc0b1e10 2024-05-20T18:35:34.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.937+0000 7ff5dad9d640 1 -- 172.21.15.5:0/4289071170 >> v1:172.21.15.63:6789/0 conn(0x7ff5dc15d530 legacy=0x7ff5dc0b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:34.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.937+0000 7ff5dad9d640 1 -- 172.21.15.5:0/4289071170 >> v1:172.21.15.5:6789/0 conn(0x7ff5dc0b1e10 legacy=0x7ff5dc070850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:34.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.937+0000 7ff5dad9d640 1 -- 172.21.15.5:0/4289071170 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5dc0cdb40 con 0x7ff5dc0d17f0 2024-05-20T18:35:34.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.938+0000 7ff5dad9d640 1 --2- 172.21.15.5:0/4289071170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5dc0d17f0 0x7ff5dc15aaa0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff5d40097a0 tx=0x7ff5d402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67e8d601aed253c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:34.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.938+0000 7ff5d959a640 1 -- 172.21.15.5:0/4289071170 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4037070 con 0x7ff5dc0d17f0 2024-05-20T18:35:34.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.938+0000 7ff5d959a640 1 -- 172.21.15.5:0/4289071170 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5d4032440 con 0x7ff5dc0d17f0 2024-05-20T18:35:34.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.938+0000 7ff5d959a640 1 -- 172.21.15.5:0/4289071170 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4032810 con 0x7ff5dc0d17f0 2024-05-20T18:35:34.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.939+0000 7ff5e0c5a640 1 -- 172.21.15.5:0/4289071170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5dc0d17f0 msgr2=0x7ff5dc15aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:34.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.939+0000 7ff5e0c5a640 1 --2- 172.21.15.5:0/4289071170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5dc0d17f0 0x7ff5dc15aaa0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff5d40097a0 tx=0x7ff5d402a230 comp rx=0 tx=0).stop 2024-05-20T18:35:34.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.939+0000 7ff5e0c5a640 1 -- 172.21.15.5:0/4289071170 shutdown_connections 2024-05-20T18:35:34.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.939+0000 7ff5e0c5a640 1 --2- 172.21.15.5:0/4289071170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5dc0d17f0 0x7ff5dc15aaa0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:34.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.939+0000 7ff5e0c5a640 1 -- 172.21.15.5:0/4289071170 >> 172.21.15.5:0/4289071170 conn(0x7ff5dc0d29d0 msgr2=0x7ff5dc0d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:34.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.939+0000 7ff5e0c5a640 1 -- 172.21.15.5:0/4289071170 shutdown_connections 2024-05-20T18:35:34.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.939+0000 7ff5e0c5a640 1 -- 172.21.15.5:0/4289071170 wait complete. 2024-05-20T18:35:34.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.941+0000 7ff5e0c5a640 1 Processor -- start 2024-05-20T18:35:34.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.941+0000 7ff5e0c5a640 1 -- start start 2024-05-20T18:35:34.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.942+0000 7ff5e0c5a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5dc0b1e10 0x7ff5dc0c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:34.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.942+0000 7ff5e0c5a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5dc0d17f0 0x7ff5dc14eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:34.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.942+0000 7ff5e0c5a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff5dc15d530 0x7ff5dc14c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:34.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.942+0000 7ff5e0c5a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5dc070f60 con 0x7ff5dc0b1e10 2024-05-20T18:35:34.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.942+0000 7ff5e0c5a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff5dc0cd230 con 0x7ff5dc0d17f0 2024-05-20T18:35:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.942+0000 7ff5e0c5a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff5dc0710e0 con 0x7ff5dc15d530 2024-05-20T18:35:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.942+0000 7ff5da59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5dc0b1e10 0x7ff5dc0c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.942+0000 7ff5da59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5dc0b1e10 0x7ff5dc0c8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49580/0 (socket says 172.21.15.5:49580) 2024-05-20T18:35:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.942+0000 7ff5da59c640 1 -- 172.21.15.5:0/3665279359 learned_addr learned my addr 172.21.15.5:0/3665279359 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.943+0000 7ff5d9d9b640 1 --2- 172.21.15.5:0/3665279359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5dc0d17f0 0x7ff5dc14eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.943+0000 7ff5dad9d640 1 --2- 172.21.15.5:0/3665279359 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff5dc15d530 0x7ff5dc14c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.943+0000 7ff5da59c640 1 -- 172.21.15.5:0/3665279359 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff5dc15d530 msgr2=0x7ff5dc14c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.943+0000 7ff5da59c640 1 --2- 172.21.15.5:0/3665279359 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff5dc15d530 0x7ff5dc14c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.943+0000 7ff5da59c640 1 -- 172.21.15.5:0/3665279359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5dc0d17f0 msgr2=0x7ff5dc14eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.943+0000 7ff5da59c640 1 --2- 172.21.15.5:0/3665279359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5dc0d17f0 0x7ff5dc14eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.943+0000 7ff5da59c640 1 -- 172.21.15.5:0/3665279359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5dc0cb6c0 con 0x7ff5dc0b1e10 2024-05-20T18:35:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.943+0000 7ff5da59c640 1 --2- 172.21.15.5:0/3665279359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5dc0b1e10 0x7ff5dc0c8e80 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc00c6e0 tx=0x7ff5cc00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.943+0000 7ff5d2ffd640 1 -- 172.21.15.5:0/3665279359 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc016020 con 0x7ff5dc0b1e10 2024-05-20T18:35:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.943+0000 7ff5d2ffd640 1 -- 172.21.15.5:0/3665279359 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5cc006d50 con 0x7ff5dc0b1e10 2024-05-20T18:35:34.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.943+0000 7ff5d2ffd640 1 -- 172.21.15.5:0/3665279359 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc003bb0 con 0x7ff5dc0b1e10 2024-05-20T18:35:34.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.943+0000 7ff5e0c5a640 1 -- 172.21.15.5:0/3665279359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5dc0cb100 con 0x7ff5dc0b1e10 2024-05-20T18:35:34.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.943+0000 7ff5e0c5a640 1 -- 172.21.15.5:0/3665279359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5dc0c6fb0 con 0x7ff5dc0b1e10 2024-05-20T18:35:34.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.944+0000 7ff5e0c5a640 1 -- 172.21.15.5:0/3665279359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5dc075b00 con 0x7ff5dc0b1e10 2024-05-20T18:35:34.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.945+0000 7ff5d2ffd640 1 -- 172.21.15.5:0/3665279359 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff5cc0050d0 con 0x7ff5dc0b1e10 2024-05-20T18:35:34.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.945+0000 7ff5d2ffd640 1 --2- 172.21.15.5:0/3665279359 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff5bc06be30 0x7ff5bc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:34.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.945+0000 7ff5d2ffd640 1 -- 172.21.15.5:0/3665279359 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7ff5cc08ab40 con 0x7ff5dc0b1e10 2024-05-20T18:35:34.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.945+0000 7ff5d9d9b640 1 --2- 172.21.15.5:0/3665279359 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff5bc06be30 0x7ff5bc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:34.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.946+0000 7ff5d9d9b640 1 --2- 172.21.15.5:0/3665279359 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff5bc06be30 0x7ff5bc06e340 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40011e0 tx=0x7ff5c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:34.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:34.950+0000 7ff5d2ffd640 1 -- 172.21.15.5:0/3665279359 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff5cc020020 con 0x7ff5dc0b1e10 2024-05-20T18:35:35.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.137+0000 7ff5e0c5a640 1 -- 172.21.15.5:0/3665279359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff5dc0709f0 con 0x7ff5dc0b1e10 2024-05-20T18:35:35.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.138+0000 7ff5d2ffd640 1 -- 172.21.15.5:0/3665279359 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+16764 (secure 0 0 0) 0x7ff5cc0593f0 con 0x7ff5dc0b1e10 2024-05-20T18:35:35.139 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:35.139 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":105,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:35:32.410574+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:35:29.146063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-20T18:35:29.754813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:35:35.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.140+0000 7ff5e0c5a640 1 -- 172.21.15.5:0/3665279359 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff5bc06be30 msgr2=0x7ff5bc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:35.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.140+0000 7ff5e0c5a640 1 --2- 172.21.15.5:0/3665279359 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff5bc06be30 0x7ff5bc06e340 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40011e0 tx=0x7ff5c4009040 comp rx=0 tx=0).stop 2024-05-20T18:35:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.140+0000 7ff5e0c5a640 1 -- 172.21.15.5:0/3665279359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5dc0b1e10 msgr2=0x7ff5dc0c8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.140+0000 7ff5e0c5a640 1 --2- 172.21.15.5:0/3665279359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5dc0b1e10 0x7ff5dc0c8e80 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc00c6e0 tx=0x7ff5cc00cba0 comp rx=0 tx=0).stop 2024-05-20T18:35:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.140+0000 7ff5e0c5a640 1 -- 172.21.15.5:0/3665279359 shutdown_connections 2024-05-20T18:35:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.140+0000 7ff5e0c5a640 1 --2- 172.21.15.5:0/3665279359 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff5bc06be30 0x7ff5bc06e340 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.140+0000 7ff5e0c5a640 1 --2- 172.21.15.5:0/3665279359 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff5dc15d530 0x7ff5dc14c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.140+0000 7ff5e0c5a640 1 --2- 172.21.15.5:0/3665279359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5dc0d17f0 0x7ff5dc14eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.140+0000 7ff5e0c5a640 1 --2- 172.21.15.5:0/3665279359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5dc0b1e10 0x7ff5dc0c8e80 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.140+0000 7ff5e0c5a640 1 -- 172.21.15.5:0/3665279359 >> 172.21.15.5:0/3665279359 conn(0x7ff5dc0d29d0 msgr2=0x7ff5dc15eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.140+0000 7ff5e0c5a640 1 -- 172.21.15.5:0/3665279359 shutdown_connections 2024-05-20T18:35:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.140+0000 7ff5e0c5a640 1 -- 172.21.15.5:0/3665279359 wait complete. 2024-05-20T18:35:35.153 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-20T18:35:35.153 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T18:35:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.267+0000 7fe83bd5e640 1 Processor -- start 2024-05-20T18:35:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.267+0000 7fe83bd5e640 1 -- start start 2024-05-20T18:35:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.268+0000 7fe83bd5e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe834162820 0x7fe8340b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.268+0000 7fe83bd5e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe83406dae0 0x7fe8340b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.268+0000 7fe83bd5e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8340d0630 con 0x7fe8340b3e80 2024-05-20T18:35:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.268+0000 7fe83bd5e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe8340d39c0 con 0x7fe83406dae0 2024-05-20T18:35:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.268+0000 7fe83bd5e640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe8340d3ca0 con 0x7fe834162820 2024-05-20T18:35:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.268+0000 7fe83a2fb640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe83406dae0 0x7fe8340b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.268+0000 7fe8392f9640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe8340b3e80 0x7fe83406d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46714/0 (socket says 172.21.15.5:46714) 2024-05-20T18:35:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.268+0000 7fe8392f9640 1 -- 172.21.15.5:0/3253962361 learned_addr learned my addr 172.21.15.5:0/3253962361 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:35.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.269+0000 7fe839afa640 1 --2- 172.21.15.5:0/3253962361 >> v2:172.21.15.63:3300/0 conn(0x7fe834162820 0x7fe8340b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:35.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.269+0000 7fe83a2fb640 1 -- 172.21.15.5:0/3253962361 >> v2:172.21.15.63:3300/0 conn(0x7fe834162820 msgr2=0x7fe8340b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:35.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.269+0000 7fe83a2fb640 1 --2- 172.21.15.5:0/3253962361 >> v2:172.21.15.63:3300/0 conn(0x7fe834162820 0x7fe8340b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:35.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.269+0000 7fe83a2fb640 1 -- 172.21.15.5:0/3253962361 >> v1:172.21.15.5:6789/0 conn(0x7fe8340b3e80 legacy=0x7fe83406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:35.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.269+0000 7fe83a2fb640 1 -- 172.21.15.5:0/3253962361 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8340d3f80 con 0x7fe83406dae0 2024-05-20T18:35:35.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.269+0000 7fe838af8640 1 -- 172.21.15.5:0/3253962361 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4038747770 0 0) 0x7fe8340d0630 con 0x7fe8340b3e80 2024-05-20T18:35:35.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.270+0000 7fe83a2fb640 1 --2- 172.21.15.5:0/3253962361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe83406dae0 0x7fe8340b8190 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fe8340b7740 tx=0x7fe82c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1e32cb5ebc44a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:35.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.270+0000 7fe838af8640 1 -- 172.21.15.5:0/3253962361 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c007740 con 0x7fe83406dae0 2024-05-20T18:35:35.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.270+0000 7fe838af8640 1 -- 172.21.15.5:0/3253962361 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe82c0078e0 con 0x7fe83406dae0 2024-05-20T18:35:35.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.270+0000 7fe838af8640 1 -- 172.21.15.5:0/3253962361 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c007cb0 con 0x7fe83406dae0 2024-05-20T18:35:35.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.270+0000 7fe83bd5e640 1 -- 172.21.15.5:0/3253962361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe83406dae0 msgr2=0x7fe8340b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:35.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.270+0000 7fe83bd5e640 1 --2- 172.21.15.5:0/3253962361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe83406dae0 0x7fe8340b8190 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fe8340b7740 tx=0x7fe82c02a680 comp rx=0 tx=0).stop 2024-05-20T18:35:35.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.270+0000 7fe83bd5e640 1 -- 172.21.15.5:0/3253962361 shutdown_connections 2024-05-20T18:35:35.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.270+0000 7fe83bd5e640 1 --2- 172.21.15.5:0/3253962361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe83406dae0 0x7fe8340b8190 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:35.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.270+0000 7fe83bd5e640 1 --2- 172.21.15.5:0/3253962361 >> v2:172.21.15.63:3300/0 conn(0x7fe834162820 0x7fe8340b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:35.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.271+0000 7fe83bd5e640 1 -- 172.21.15.5:0/3253962361 >> 172.21.15.5:0/3253962361 conn(0x7fe8340d4bd0 msgr2=0x7fe8340d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:35.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.271+0000 7fe83bd5e640 1 -- 172.21.15.5:0/3253962361 shutdown_connections 2024-05-20T18:35:35.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.271+0000 7fe83bd5e640 1 -- 172.21.15.5:0/3253962361 wait complete. 2024-05-20T18:35:35.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.272+0000 7fe83bd5e640 1 Processor -- start 2024-05-20T18:35:35.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.272+0000 7fe83bd5e640 1 -- start start 2024-05-20T18:35:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.274+0000 7fe83bd5e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8340b3e80 0x7fe8341598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.274+0000 7fe83bd5e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe834162820 0x7fe834158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.274+0000 7fe83bd5e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe834158740 0x7fe834146df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.274+0000 7fe83bd5e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe8340d4780 con 0x7fe834162820 2024-05-20T18:35:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.274+0000 7fe83bd5e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe8340d0300 con 0x7fe834158740 2024-05-20T18:35:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.274+0000 7fe83bd5e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe8340d4900 con 0x7fe8340b3e80 2024-05-20T18:35:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.274+0000 7fe8392f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe834162820 0x7fe834158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.274+0000 7fe8392f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe834162820 0x7fe834158200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49596/0 (socket says 172.21.15.5:49596) 2024-05-20T18:35:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.274+0000 7fe8392f9640 1 -- 172.21.15.5:0/1846484159 learned_addr learned my addr 172.21.15.5:0/1846484159 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.274+0000 7fe83a2fb640 1 --2- 172.21.15.5:0/1846484159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe834158740 0x7fe834146df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.274+0000 7fe839afa640 1 --2- 172.21.15.5:0/1846484159 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8340b3e80 0x7fe8341598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.274+0000 7fe8392f9640 1 -- 172.21.15.5:0/1846484159 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8340b3e80 msgr2=0x7fe8341598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.274+0000 7fe8392f9640 1 --2- 172.21.15.5:0/1846484159 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8340b3e80 0x7fe8341598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.274+0000 7fe8392f9640 1 -- 172.21.15.5:0/1846484159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe834158740 msgr2=0x7fe834146df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.274+0000 7fe8392f9640 1 --2- 172.21.15.5:0/1846484159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe834158740 0x7fe834146df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.274+0000 7fe8392f9640 1 -- 172.21.15.5:0/1846484159 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8340b6c70 con 0x7fe834162820 2024-05-20T18:35:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.274+0000 7fe8392f9640 1 --2- 172.21.15.5:0/1846484159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe834162820 0x7fe834158200 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fe82400c460 tx=0x7fe82400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.275+0000 7fe82a7fc640 1 -- 172.21.15.5:0/1846484159 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe824018070 con 0x7fe834162820 2024-05-20T18:35:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.275+0000 7fe82a7fc640 1 -- 172.21.15.5:0/1846484159 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe824007500 con 0x7fe834162820 2024-05-20T18:35:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.275+0000 7fe83bd5e640 1 -- 172.21.15.5:0/1846484159 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8340b5eb0 con 0x7fe834162820 2024-05-20T18:35:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.275+0000 7fe82a7fc640 1 -- 172.21.15.5:0/1846484159 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe824004a20 con 0x7fe834162820 2024-05-20T18:35:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.275+0000 7fe83bd5e640 1 -- 172.21.15.5:0/1846484159 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8340b6470 con 0x7fe834162820 2024-05-20T18:35:35.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.276+0000 7fe83bd5e640 1 -- 172.21.15.5:0/1846484159 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe834075b00 con 0x7fe834162820 2024-05-20T18:35:35.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.276+0000 7fe82a7fc640 1 -- 172.21.15.5:0/1846484159 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe82401b070 con 0x7fe834162820 2024-05-20T18:35:35.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.277+0000 7fe82a7fc640 1 --2- 172.21.15.5:0/1846484159 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe81806be60 0x7fe81806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.277+0000 7fe82a7fc640 1 -- 172.21.15.5:0/1846484159 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7fe82408a230 con 0x7fe834162820 2024-05-20T18:35:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.277+0000 7fe839afa640 1 --2- 172.21.15.5:0/1846484159 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe81806be60 0x7fe81806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.277+0000 7fe839afa640 1 --2- 172.21.15.5:0/1846484159 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe81806be60 0x7fe81806e370 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe830009aa0 tx=0x7fe830009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:35.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.282+0000 7fe82a7fc640 1 -- 172.21.15.5:0/1846484159 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe824058400 con 0x7fe834162820 2024-05-20T18:35:35.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.465+0000 7fe83bd5e640 1 -- 172.21.15.5:0/1846484159 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe834160b40 con 0x7fe81806be60 2024-05-20T18:35:35.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.471+0000 7fe82a7fc640 1 -- 172.21.15.5:0/1846484159 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247091 (secure 0 0 0) 0x7fe834160b40 con 0x7fe81806be60 2024-05-20T18:35:35.476 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:35.477 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-20T18:35:35.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.474+0000 7fe83bd5e640 1 -- 172.21.15.5:0/1846484159 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe81806be60 msgr2=0x7fe81806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:35.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.474+0000 7fe83bd5e640 1 --2- 172.21.15.5:0/1846484159 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe81806be60 0x7fe81806e370 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe830009aa0 tx=0x7fe830009190 comp rx=0 tx=0).stop 2024-05-20T18:35:35.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.474+0000 7fe83bd5e640 1 -- 172.21.15.5:0/1846484159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe834162820 msgr2=0x7fe834158200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:35.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.474+0000 7fe83bd5e640 1 --2- 172.21.15.5:0/1846484159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe834162820 0x7fe834158200 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fe82400c460 tx=0x7fe82400c920 comp rx=0 tx=0).stop 2024-05-20T18:35:35.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.474+0000 7fe83bd5e640 1 -- 172.21.15.5:0/1846484159 shutdown_connections 2024-05-20T18:35:35.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.474+0000 7fe83bd5e640 1 --2- 172.21.15.5:0/1846484159 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe81806be60 0x7fe81806e370 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:35.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.474+0000 7fe83bd5e640 1 --2- 172.21.15.5:0/1846484159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe834158740 0x7fe834146df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:35.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.474+0000 7fe83bd5e640 1 --2- 172.21.15.5:0/1846484159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe834162820 0x7fe834158200 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:35.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.474+0000 7fe83bd5e640 1 --2- 172.21.15.5:0/1846484159 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8340b3e80 0x7fe8341598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:35.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.474+0000 7fe83bd5e640 1 -- 172.21.15.5:0/1846484159 >> 172.21.15.5:0/1846484159 conn(0x7fe8340d4bd0 msgr2=0x7fe83415c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:35.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.474+0000 7fe83bd5e640 1 -- 172.21.15.5:0/1846484159 shutdown_connections 2024-05-20T18:35:35.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.474+0000 7fe83bd5e640 1 -- 172.21.15.5:0/1846484159 wait complete. 2024-05-20T18:35:35.487 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-05-20T18:35:34.704565+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":49,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260564,"kb_used_data":11108,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714156,"statfs":{"total":773094113280,"available":772827295744,"internally_reserved":0,"allocated":11374592,"data_stored":5886047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.319617"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.401556+0000","last_change":"2024-05-20T18:35:30.401556+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.401556+0000","last_undegraded":"2024-05-20T18:35:30.401556+0000","last_fullsized":"2024-05-20T18:35:30.401556+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395067+0000","last_change":"2024-05-20T18:35:30.395067+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395067+0000","last_undegraded":"2024-05-20T18:35:30.395067+0000","last_fullsized":"2024-05-20T18:35:30.395067+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.400834+0000","last_change":"2024-05-20T18:35:30.400834+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.400834+0000","last_undegraded":"2024-05-20T18:35:30.400834+0000","last_fullsized":"2024-05-20T18:35:30.400834+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.396143+0000","last_change":"2024-05-20T18:35:30.396143+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.396143+0000","last_undegraded":"2024-05-20T18:35:30.396143+0000","last_fullsized":"2024-05-20T18:35:30.396143+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.396956+0000","last_change":"2024-05-20T18:35:30.396956+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.396956+0000","last_undegraded":"2024-05-20T18:35:30.396956+0000","last_fullsized":"2024-05-20T18:35:30.396956+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.393816+0000","last_change":"2024-05-20T18:35:30.393816+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.393816+0000","last_undegraded":"2024-05-20T18:35:30.393816+0000","last_fullsized":"2024-05-20T18:35:30.393816+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.394799+0000","last_change":"2024-05-20T18:35:30.394799+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.394799+0000","last_undegraded":"2024-05-20T18:35:30.394799+0000","last_fullsized":"2024-05-20T18:35:30.394799+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.401809+0000","last_change":"2024-05-20T18:35:30.401809+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.401809+0000","last_undegraded":"2024-05-20T18:35:30.401809+0000","last_fullsized":"2024-05-20T18:35:30.401809+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.394202+0000","last_change":"2024-05-20T18:35:30.394202+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.394202+0000","last_undegraded":"2024-05-20T18:35:30.394202+0000","last_fullsized":"2024-05-20T18:35:30.394202+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395453+0000","last_change":"2024-05-20T18:35:30.395453+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395453+0000","last_undegraded":"2024-05-20T18:35:30.395453+0000","last_fullsized":"2024-05-20T18:35:30.395453+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.397481+0000","last_change":"2024-05-20T18:35:30.397481+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.397481+0000","last_undegraded":"2024-05-20T18:35:30.397481+0000","last_fullsized":"2024-05-20T18:35:30.397481+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.391506+0000","last_change":"2024-05-20T18:35:30.391506+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.391506+0000","last_undegraded":"2024-05-20T18:35:30.391506+0000","last_fullsized":"2024-05-20T18:35:30.391506+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.394975+0000","last_change":"2024-05-20T18:35:30.394975+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.394975+0000","last_undegraded":"2024-05-20T18:35:30.394975+0000","last_fullsized":"2024-05-20T18:35:30.394975+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.399091+0000","last_change":"2024-05-20T18:35:30.399091+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.399091+0000","last_undegraded":"2024-05-20T18:35:30.399091+0000","last_fullsized":"2024-05-20T18:35:30.399091+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.398451+0000","last_change":"2024-05-20T18:35:30.398451+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.398451+0000","last_undegraded":"2024-05-20T18:35:30.398451+0000","last_fullsized":"2024-05-20T18:35:30.398451+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.399063+0000","last_change":"2024-05-20T18:35:30.399063+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.399063+0000","last_undegraded":"2024-05-20T18:35:30.399063+0000","last_fullsized":"2024-05-20T18:35:30.399063+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.400625+0000","last_change":"2024-05-20T18:35:30.400625+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.400625+0000","last_undegraded":"2024-05-20T18:35:30.400625+0000","last_fullsized":"2024-05-20T18:35:30.400625+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.394511+0000","last_change":"2024-05-20T18:35:30.394511+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.394511+0000","last_undegraded":"2024-05-20T18:35:30.394511+0000","last_fullsized":"2024-05-20T18:35:30.394511+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.400130+0000","last_change":"2024-05-20T18:35:30.400130+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.400130+0000","last_undegraded":"2024-05-20T18:35:30.400130+0000","last_fullsized":"2024-05-20T18:35:30.400130+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.390388+0000","last_change":"2024-05-20T18:35:30.390388+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.390388+0000","last_undegraded":"2024-05-20T18:35:30.390388+0000","last_fullsized":"2024-05-20T18:35:30.390388+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.401295+0000","last_change":"2024-05-20T18:35:30.401295+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.401295+0000","last_undegraded":"2024-05-20T18:35:30.401295+0000","last_fullsized":"2024-05-20T18:35:30.401295+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395921+0000","last_change":"2024-05-20T18:35:30.395921+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395921+0000","last_undegraded":"2024-05-20T18:35:30.395921+0000","last_fullsized":"2024-05-20T18:35:30.395921+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.400238+0000","last_change":"2024-05-20T18:35:30.400238+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.400238+0000","last_undegraded":"2024-05-20T18:35:30.400238+0000","last_fullsized":"2024-05-20T18:35:30.400238+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.396399+0000","last_change":"2024-05-20T18:35:30.396399+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.396399+0000","last_undegraded":"2024-05-20T18:35:30.396399+0000","last_fullsized":"2024-05-20T18:35:30.396399+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.391428+0000","last_change":"2024-05-20T18:35:30.391428+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.391428+0000","last_undegraded":"2024-05-20T18:35:30.391428+0000","last_fullsized":"2024-05-20T18:35:30.391428+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.391347+0000","last_change":"2024-05-20T18:35:30.391347+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.391347+0000","last_undegraded":"2024-05-20T18:35:30.391347+0000","last_fullsized":"2024-05-20T18:35:30.391347+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.396899+0000","last_change":"2024-05-20T18:35:30.396899+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.396899+0000","last_undegraded":"2024-05-20T18:35:30.396899+0000","last_fullsized":"2024-05-20T18:35:30.396899+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395384+0000","last_change":"2024-05-20T18:35:30.395384+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395384+0000","last_undegraded":"2024-05-20T18:35:30.395384+0000","last_fullsized":"2024-05-20T18:35:30.395384+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.400522+0000","last_change":"2024-05-20T18:35:30.400522+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.400522+0000","last_undegraded":"2024-05-20T18:35:30.400522+0000","last_fullsized":"2024-05-20T18:35:30.400522+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.392369+0000","last_change":"2024-05-20T18:35:30.392369+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.392369+0000","last_undegraded":"2024-05-20T18:35:30.392369+0000","last_fullsized":"2024-05-20T18:35:30.392369+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.394219+0000","last_change":"2024-05-20T18:35:30.394219+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.394219+0000","last_undegraded":"2024-05-20T18:35:30.394219+0000","last_fullsized":"2024-05-20T18:35:30.394219+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.399575+0000","last_change":"2024-05-20T18:35:30.399575+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.399575+0000","last_undegraded":"2024-05-20T18:35:30.399575+0000","last_fullsized":"2024-05-20T18:35:30.399575+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.393568+0000","last_change":"2024-05-20T18:35:30.393568+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.393568+0000","last_undegraded":"2024-05-20T18:35:30.393568+0000","last_fullsized":"2024-05-20T18:35:30.393568+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.410235+0000","last_change":"2024-05-20T18:35:30.407837+0000","last_active":"2024-05-20T18:35:30.410235+0000","last_peered":"2024-05-20T18:35:30.410235+0000","last_clean":"2024-05-20T18:35:30.410235+0000","last_became_active":"2024-05-20T18:35:30.407539+0000","last_became_peered":"2024-05-20T18:35:30.407539+0000","last_unstale":"2024-05-20T18:35:30.410235+0000","last_undegraded":"2024-05-20T18:35:30.410235+0000","last_fullsized":"2024-05-20T18:35:30.410235+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:31:20.340446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395875+0000","last_change":"2024-05-20T18:35:30.395875+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395875+0000","last_undegraded":"2024-05-20T18:35:30.395875+0000","last_fullsized":"2024-05-20T18:35:30.395875+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.409396+0000","last_change":"2024-05-20T18:35:30.406366+0000","last_active":"2024-05-20T18:35:30.409396+0000","last_peered":"2024-05-20T18:35:30.409396+0000","last_clean":"2024-05-20T18:35:30.409396+0000","last_became_active":"2024-05-20T18:35:30.405770+0000","last_became_peered":"2024-05-20T18:35:30.405770+0000","last_unstale":"2024-05-20T18:35:30.409396+0000","last_undegraded":"2024-05-20T18:35:30.409396+0000","last_fullsized":"2024-05-20T18:35:30.409396+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:50:24.603292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.390568+0000","last_change":"2024-05-20T18:35:30.390568+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.390568+0000","last_undegraded":"2024-05-20T18:35:30.390568+0000","last_fullsized":"2024-05-20T18:35:30.390568+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.412922+0000","last_change":"2024-05-20T18:35:30.409253+0000","last_active":"2024-05-20T18:35:30.412922+0000","last_peered":"2024-05-20T18:35:30.412922+0000","last_clean":"2024-05-20T18:35:30.412922+0000","last_became_active":"2024-05-20T18:35:30.408847+0000","last_became_peered":"2024-05-20T18:35:30.408847+0000","last_unstale":"2024-05-20T18:35:30.412922+0000","last_undegraded":"2024-05-20T18:35:30.412922+0000","last_fullsized":"2024-05-20T18:35:30.412922+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:13:02.199994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.396581+0000","last_change":"2024-05-20T18:35:30.396581+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.396581+0000","last_undegraded":"2024-05-20T18:35:30.396581+0000","last_fullsized":"2024-05-20T18:35:30.396581+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.406960+0000","last_change":"2024-05-20T18:35:30.402307+0000","last_active":"2024-05-20T18:35:30.406960+0000","last_peered":"2024-05-20T18:35:30.406960+0000","last_clean":"2024-05-20T18:35:30.406960+0000","last_became_active":"2024-05-20T18:35:30.401952+0000","last_became_peered":"2024-05-20T18:35:30.401952+0000","last_unstale":"2024-05-20T18:35:30.406960+0000","last_undegraded":"2024-05-20T18:35:30.406960+0000","last_fullsized":"2024-05-20T18:35:30.406960+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:47:28.651899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395611+0000","last_change":"2024-05-20T18:35:30.395611+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395611+0000","last_undegraded":"2024-05-20T18:35:30.395611+0000","last_fullsized":"2024-05-20T18:35:30.395611+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.407964+0000","last_change":"2024-05-20T18:35:30.405866+0000","last_active":"2024-05-20T18:35:30.407964+0000","last_peered":"2024-05-20T18:35:30.407964+0000","last_clean":"2024-05-20T18:35:30.407964+0000","last_became_active":"2024-05-20T18:35:30.405503+0000","last_became_peered":"2024-05-20T18:35:30.405503+0000","last_unstale":"2024-05-20T18:35:30.407964+0000","last_undegraded":"2024-05-20T18:35:30.407964+0000","last_fullsized":"2024-05-20T18:35:30.407964+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:16:21.553268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.393963+0000","last_change":"2024-05-20T18:35:30.393963+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.393963+0000","last_undegraded":"2024-05-20T18:35:30.393963+0000","last_fullsized":"2024-05-20T18:35:30.393963+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.414268+0000","last_change":"2024-05-20T18:35:30.412485+0000","last_active":"2024-05-20T18:35:30.414268+0000","last_peered":"2024-05-20T18:35:30.414268+0000","last_clean":"2024-05-20T18:35:30.414268+0000","last_became_active":"2024-05-20T18:35:30.412162+0000","last_became_peered":"2024-05-20T18:35:30.412162+0000","last_unstale":"2024-05-20T18:35:30.414268+0000","last_undegraded":"2024-05-20T18:35:30.414268+0000","last_fullsized":"2024-05-20T18:35:30.414268+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:31:19.619870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.393425+0000","last_change":"2024-05-20T18:35:30.393425+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.393425+0000","last_undegraded":"2024-05-20T18:35:30.393425+0000","last_fullsized":"2024-05-20T18:35:30.393425+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.412482+0000","last_change":"2024-05-20T18:35:30.409374+0000","last_active":"2024-05-20T18:35:30.412482+0000","last_peered":"2024-05-20T18:35:30.412482+0000","last_clean":"2024-05-20T18:35:30.412482+0000","last_became_active":"2024-05-20T18:35:30.408970+0000","last_became_peered":"2024-05-20T18:35:30.408970+0000","last_unstale":"2024-05-20T18:35:30.412482+0000","last_undegraded":"2024-05-20T18:35:30.412482+0000","last_fullsized":"2024-05-20T18:35:30.412482+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:19:27.219675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395456+0000","last_change":"2024-05-20T18:35:30.395456+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395456+0000","last_undegraded":"2024-05-20T18:35:30.395456+0000","last_fullsized":"2024-05-20T18:35:30.395456+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.408240+0000","last_change":"2024-05-20T18:35:30.406064+0000","last_active":"2024-05-20T18:35:30.408240+0000","last_peered":"2024-05-20T18:35:30.408240+0000","last_clean":"2024-05-20T18:35:30.408240+0000","last_became_active":"2024-05-20T18:35:30.405710+0000","last_became_peered":"2024-05-20T18:35:30.405710+0000","last_unstale":"2024-05-20T18:35:30.408240+0000","last_undegraded":"2024-05-20T18:35:30.408240+0000","last_fullsized":"2024-05-20T18:35:30.408240+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:23:16.072920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.394555+0000","last_change":"2024-05-20T18:35:30.394555+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.394555+0000","last_undegraded":"2024-05-20T18:35:30.394555+0000","last_fullsized":"2024-05-20T18:35:30.394555+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.412758+0000","last_change":"2024-05-20T18:35:30.409880+0000","last_active":"2024-05-20T18:35:30.412758+0000","last_peered":"2024-05-20T18:35:30.412758+0000","last_clean":"2024-05-20T18:35:30.412758+0000","last_became_active":"2024-05-20T18:35:30.409517+0000","last_became_peered":"2024-05-20T18:35:30.409517+0000","last_unstale":"2024-05-20T18:35:30.412758+0000","last_undegraded":"2024-05-20T18:35:30.412758+0000","last_fullsized":"2024-05-20T18:35:30.412758+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:33:58.473775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.393400+0000","last_change":"2024-05-20T18:35:30.393400+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.393400+0000","last_undegraded":"2024-05-20T18:35:30.393400+0000","last_fullsized":"2024-05-20T18:35:30.393400+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.408544+0000","last_change":"2024-05-20T18:35:30.403374+0000","last_active":"2024-05-20T18:35:30.408544+0000","last_peered":"2024-05-20T18:35:30.408544+0000","last_clean":"2024-05-20T18:35:30.408544+0000","last_became_active":"2024-05-20T18:35:30.403012+0000","last_became_peered":"2024-05-20T18:35:30.403012+0000","last_unstale":"2024-05-20T18:35:30.408544+0000","last_undegraded":"2024-05-20T18:35:30.408544+0000","last_fullsized":"2024-05-20T18:35:30.408544+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:45:42.189012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.397246+0000","last_change":"2024-05-20T18:35:30.397246+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.397246+0000","last_undegraded":"2024-05-20T18:35:30.397246+0000","last_fullsized":"2024-05-20T18:35:30.397246+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.411337+0000","last_change":"2024-05-20T18:35:30.410506+0000","last_active":"2024-05-20T18:35:30.411337+0000","last_peered":"2024-05-20T18:35:30.411337+0000","last_clean":"2024-05-20T18:35:30.411337+0000","last_became_active":"2024-05-20T18:35:30.410168+0000","last_became_peered":"2024-05-20T18:35:30.410168+0000","last_unstale":"2024-05-20T18:35:30.411337+0000","last_undegraded":"2024-05-20T18:35:30.411337+0000","last_fullsized":"2024-05-20T18:35:30.411337+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:19:34.158678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.394072+0000","last_change":"2024-05-20T18:35:30.394072+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.394072+0000","last_undegraded":"2024-05-20T18:35:30.394072+0000","last_fullsized":"2024-05-20T18:35:30.394072+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.413578+0000","last_change":"2024-05-20T18:35:30.412065+0000","last_active":"2024-05-20T18:35:30.413578+0000","last_peered":"2024-05-20T18:35:30.413578+0000","last_clean":"2024-05-20T18:35:30.413578+0000","last_became_active":"2024-05-20T18:35:30.410006+0000","last_became_peered":"2024-05-20T18:35:30.410006+0000","last_unstale":"2024-05-20T18:35:30.413578+0000","last_undegraded":"2024-05-20T18:35:30.413578+0000","last_fullsized":"2024-05-20T18:35:30.413578+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:47:28.651899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395562+0000","last_change":"2024-05-20T18:35:30.395562+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395562+0000","last_undegraded":"2024-05-20T18:35:30.395562+0000","last_fullsized":"2024-05-20T18:35:30.395562+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.408708+0000","last_change":"2024-05-20T18:35:30.406644+0000","last_active":"2024-05-20T18:35:30.408708+0000","last_peered":"2024-05-20T18:35:30.408708+0000","last_clean":"2024-05-20T18:35:30.408708+0000","last_became_active":"2024-05-20T18:35:30.406155+0000","last_became_peered":"2024-05-20T18:35:30.406155+0000","last_unstale":"2024-05-20T18:35:30.408708+0000","last_undegraded":"2024-05-20T18:35:30.408708+0000","last_fullsized":"2024-05-20T18:35:30.408708+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:24:43.933783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.392885+0000","last_change":"2024-05-20T18:35:30.392885+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.392885+0000","last_undegraded":"2024-05-20T18:35:30.392885+0000","last_fullsized":"2024-05-20T18:35:30.392885+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.408224+0000","last_change":"2024-05-20T18:35:30.404725+0000","last_active":"2024-05-20T18:35:30.408224+0000","last_peered":"2024-05-20T18:35:30.408224+0000","last_clean":"2024-05-20T18:35:30.408224+0000","last_became_active":"2024-05-20T18:35:30.404404+0000","last_became_peered":"2024-05-20T18:35:30.404404+0000","last_unstale":"2024-05-20T18:35:30.408224+0000","last_undegraded":"2024-05-20T18:35:30.408224+0000","last_fullsized":"2024-05-20T18:35:30.408224+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:22:09.796441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.396812+0000","last_change":"2024-05-20T18:35:30.396812+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.396812+0000","last_undegraded":"2024-05-20T18:35:30.396812+0000","last_fullsized":"2024-05-20T18:35:30.396812+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.415211+0000","last_change":"2024-05-20T18:35:30.414408+0000","last_active":"2024-05-20T18:35:30.415211+0000","last_peered":"2024-05-20T18:35:30.415211+0000","last_clean":"2024-05-20T18:35:30.415211+0000","last_became_active":"2024-05-20T18:35:30.414025+0000","last_became_peered":"2024-05-20T18:35:30.414025+0000","last_unstale":"2024-05-20T18:35:30.415211+0000","last_undegraded":"2024-05-20T18:35:30.415211+0000","last_fullsized":"2024-05-20T18:35:30.415211+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:52:17.777292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395135+0000","last_change":"2024-05-20T18:35:30.395135+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395135+0000","last_undegraded":"2024-05-20T18:35:30.395135+0000","last_fullsized":"2024-05-20T18:35:30.395135+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.412446+0000","last_change":"2024-05-20T18:35:30.408547+0000","last_active":"2024-05-20T18:35:30.412446+0000","last_peered":"2024-05-20T18:35:30.412446+0000","last_clean":"2024-05-20T18:35:30.412446+0000","last_became_active":"2024-05-20T18:35:30.408205+0000","last_became_peered":"2024-05-20T18:35:30.408205+0000","last_unstale":"2024-05-20T18:35:30.412446+0000","last_undegraded":"2024-05-20T18:35:30.412446+0000","last_fullsized":"2024-05-20T18:35:30.412446+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:59:20.346125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.397543+0000","last_change":"2024-05-20T18:35:30.397543+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.397543+0000","last_undegraded":"2024-05-20T18:35:30.397543+0000","last_fullsized":"2024-05-20T18:35:30.397543+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.412005+0000","last_change":"2024-05-20T18:35:30.408259+0000","last_active":"2024-05-20T18:35:30.412005+0000","last_peered":"2024-05-20T18:35:30.412005+0000","last_clean":"2024-05-20T18:35:30.412005+0000","last_became_active":"2024-05-20T18:35:30.407911+0000","last_became_peered":"2024-05-20T18:35:30.407911+0000","last_unstale":"2024-05-20T18:35:30.412005+0000","last_undegraded":"2024-05-20T18:35:30.412005+0000","last_fullsized":"2024-05-20T18:35:30.412005+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:30:06.656452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.392761+0000","last_change":"2024-05-20T18:35:30.392761+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.392761+0000","last_undegraded":"2024-05-20T18:35:30.392761+0000","last_fullsized":"2024-05-20T18:35:30.392761+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.413312+0000","last_change":"2024-05-20T18:35:30.412088+0000","last_active":"2024-05-20T18:35:30.413312+0000","last_peered":"2024-05-20T18:35:30.413312+0000","last_clean":"2024-05-20T18:35:30.413312+0000","last_became_active":"2024-05-20T18:35:30.411708+0000","last_became_peered":"2024-05-20T18:35:30.411708+0000","last_unstale":"2024-05-20T18:35:30.413312+0000","last_undegraded":"2024-05-20T18:35:30.413312+0000","last_fullsized":"2024-05-20T18:35:30.413312+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:26:29.213682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.394854+0000","last_change":"2024-05-20T18:35:30.394854+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.394854+0000","last_undegraded":"2024-05-20T18:35:30.394854+0000","last_fullsized":"2024-05-20T18:35:30.394854+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.407629+0000","last_change":"2024-05-20T18:35:30.406195+0000","last_active":"2024-05-20T18:35:30.407629+0000","last_peered":"2024-05-20T18:35:30.407629+0000","last_clean":"2024-05-20T18:35:30.407629+0000","last_became_active":"2024-05-20T18:35:30.405856+0000","last_became_peered":"2024-05-20T18:35:30.405856+0000","last_unstale":"2024-05-20T18:35:30.407629+0000","last_undegraded":"2024-05-20T18:35:30.407629+0000","last_fullsized":"2024-05-20T18:35:30.407629+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:33:58.473775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395697+0000","last_change":"2024-05-20T18:35:30.395697+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395697+0000","last_undegraded":"2024-05-20T18:35:30.395697+0000","last_fullsized":"2024-05-20T18:35:30.395697+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.413399+0000","last_change":"2024-05-20T18:35:30.411351+0000","last_active":"2024-05-20T18:35:30.413399+0000","last_peered":"2024-05-20T18:35:30.413399+0000","last_clean":"2024-05-20T18:35:30.413399+0000","last_became_active":"2024-05-20T18:35:30.410938+0000","last_became_peered":"2024-05-20T18:35:30.410938+0000","last_unstale":"2024-05-20T18:35:30.413399+0000","last_undegraded":"2024-05-20T18:35:30.413399+0000","last_fullsized":"2024-05-20T18:35:30.413399+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:45:26.959959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.396251+0000","last_change":"2024-05-20T18:35:30.396251+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.396251+0000","last_undegraded":"2024-05-20T18:35:30.396251+0000","last_fullsized":"2024-05-20T18:35:30.396251+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.413171+0000","last_change":"2024-05-20T18:35:30.410136+0000","last_active":"2024-05-20T18:35:30.413171+0000","last_peered":"2024-05-20T18:35:30.413171+0000","last_clean":"2024-05-20T18:35:30.413171+0000","last_became_active":"2024-05-20T18:35:30.409778+0000","last_became_peered":"2024-05-20T18:35:30.409778+0000","last_unstale":"2024-05-20T18:35:30.413171+0000","last_undegraded":"2024-05-20T18:35:30.413171+0000","last_fullsized":"2024-05-20T18:35:30.413171+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:37:58.981434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.397965+0000","last_change":"2024-05-20T18:35:30.397965+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.397965+0000","last_undegraded":"2024-05-20T18:35:30.397965+0000","last_fullsized":"2024-05-20T18:35:30.397965+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.411835+0000","last_change":"2024-05-20T18:35:30.407327+0000","last_active":"2024-05-20T18:35:30.411835+0000","last_peered":"2024-05-20T18:35:30.411835+0000","last_clean":"2024-05-20T18:35:30.411835+0000","last_became_active":"2024-05-20T18:35:30.406994+0000","last_became_peered":"2024-05-20T18:35:30.406994+0000","last_unstale":"2024-05-20T18:35:30.411835+0000","last_undegraded":"2024-05-20T18:35:30.411835+0000","last_fullsized":"2024-05-20T18:35:30.411835+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:22:23.532551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.398473+0000","last_change":"2024-05-20T18:35:30.398473+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.398473+0000","last_undegraded":"2024-05-20T18:35:30.398473+0000","last_fullsized":"2024-05-20T18:35:30.398473+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.409293+0000","last_change":"2024-05-20T18:35:30.404056+0000","last_active":"2024-05-20T18:35:30.409293+0000","last_peered":"2024-05-20T18:35:30.409293+0000","last_clean":"2024-05-20T18:35:30.409293+0000","last_became_active":"2024-05-20T18:35:30.403695+0000","last_became_peered":"2024-05-20T18:35:30.403695+0000","last_unstale":"2024-05-20T18:35:30.409293+0000","last_undegraded":"2024-05-20T18:35:30.409293+0000","last_fullsized":"2024-05-20T18:35:30.409293+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:06:34.052366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.399419+0000","last_change":"2024-05-20T18:35:30.399419+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.399419+0000","last_undegraded":"2024-05-20T18:35:30.399419+0000","last_fullsized":"2024-05-20T18:35:30.399419+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.413484+0000","last_change":"2024-05-20T18:35:30.412633+0000","last_active":"2024-05-20T18:35:30.413484+0000","last_peered":"2024-05-20T18:35:30.413484+0000","last_clean":"2024-05-20T18:35:30.413484+0000","last_became_active":"2024-05-20T18:35:30.412307+0000","last_became_peered":"2024-05-20T18:35:30.412307+0000","last_unstale":"2024-05-20T18:35:30.413484+0000","last_undegraded":"2024-05-20T18:35:30.413484+0000","last_fullsized":"2024-05-20T18:35:30.413484+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:59:07.724149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395296+0000","last_change":"2024-05-20T18:35:30.395296+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395296+0000","last_undegraded":"2024-05-20T18:35:30.395296+0000","last_fullsized":"2024-05-20T18:35:30.395296+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.411785+0000","last_change":"2024-05-20T18:35:30.411059+0000","last_active":"2024-05-20T18:35:30.411785+0000","last_peered":"2024-05-20T18:35:30.411785+0000","last_clean":"2024-05-20T18:35:30.411785+0000","last_became_active":"2024-05-20T18:35:30.410688+0000","last_became_peered":"2024-05-20T18:35:30.410688+0000","last_unstale":"2024-05-20T18:35:30.411785+0000","last_undegraded":"2024-05-20T18:35:30.411785+0000","last_fullsized":"2024-05-20T18:35:30.411785+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:41:03.388138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395640+0000","last_change":"2024-05-20T18:35:30.395640+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395640+0000","last_undegraded":"2024-05-20T18:35:30.395640+0000","last_fullsized":"2024-05-20T18:35:30.395640+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.408141+0000","last_change":"2024-05-20T18:35:30.406122+0000","last_active":"2024-05-20T18:35:30.408141+0000","last_peered":"2024-05-20T18:35:30.408141+0000","last_clean":"2024-05-20T18:35:30.408141+0000","last_became_active":"2024-05-20T18:35:30.405784+0000","last_became_peered":"2024-05-20T18:35:30.405784+0000","last_unstale":"2024-05-20T18:35:30.408141+0000","last_undegraded":"2024-05-20T18:35:30.408141+0000","last_fullsized":"2024-05-20T18:35:30.408141+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:24:08.103767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.397137+0000","last_change":"2024-05-20T18:35:30.397137+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.397137+0000","last_undegraded":"2024-05-20T18:35:30.397137+0000","last_fullsized":"2024-05-20T18:35:30.397137+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.404066+0000","last_change":"2024-05-20T18:35:30.400456+0000","last_active":"2024-05-20T18:35:30.404066+0000","last_peered":"2024-05-20T18:35:30.404066+0000","last_clean":"2024-05-20T18:35:30.404066+0000","last_became_active":"2024-05-20T18:35:30.399296+0000","last_became_peered":"2024-05-20T18:35:30.399296+0000","last_unstale":"2024-05-20T18:35:30.404066+0000","last_undegraded":"2024-05-20T18:35:30.404066+0000","last_fullsized":"2024-05-20T18:35:30.404066+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:30:06.656452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.394593+0000","last_change":"2024-05-20T18:35:30.394593+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.394593+0000","last_undegraded":"2024-05-20T18:35:30.394593+0000","last_fullsized":"2024-05-20T18:35:30.394593+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.411083+0000","last_change":"2024-05-20T18:35:30.408084+0000","last_active":"2024-05-20T18:35:30.411083+0000","last_peered":"2024-05-20T18:35:30.411083+0000","last_clean":"2024-05-20T18:35:30.411083+0000","last_became_active":"2024-05-20T18:35:30.407701+0000","last_became_peered":"2024-05-20T18:35:30.407701+0000","last_unstale":"2024-05-20T18:35:30.411083+0000","last_undegraded":"2024-05-20T18:35:30.411083+0000","last_fullsized":"2024-05-20T18:35:30.411083+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:59:53.740452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.399317+0000","last_change":"2024-05-20T18:35:30.399317+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.399317+0000","last_undegraded":"2024-05-20T18:35:30.399317+0000","last_fullsized":"2024-05-20T18:35:30.399317+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.415376+0000","last_change":"2024-05-20T18:35:30.414600+0000","last_active":"2024-05-20T18:35:30.415376+0000","last_peered":"2024-05-20T18:35:30.415376+0000","last_clean":"2024-05-20T18:35:30.415376+0000","last_became_active":"2024-05-20T18:35:30.414218+0000","last_became_peered":"2024-05-20T18:35:30.414218+0000","last_unstale":"2024-05-20T18:35:30.415376+0000","last_undegraded":"2024-05-20T18:35:30.415376+0000","last_fullsized":"2024-05-20T18:35:30.415376+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:58:38.814053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.399261+0000","last_change":"2024-05-20T18:35:30.399261+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.399261+0000","last_undegraded":"2024-05-20T18:35:30.399261+0000","last_fullsized":"2024-05-20T18:35:30.399261+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.408199+0000","last_change":"2024-05-20T18:35:30.406555+0000","last_active":"2024-05-20T18:35:30.408199+0000","last_peered":"2024-05-20T18:35:30.408199+0000","last_clean":"2024-05-20T18:35:30.408199+0000","last_became_active":"2024-05-20T18:35:30.405500+0000","last_became_peered":"2024-05-20T18:35:30.405500+0000","last_unstale":"2024-05-20T18:35:30.408199+0000","last_undegraded":"2024-05-20T18:35:30.408199+0000","last_fullsized":"2024-05-20T18:35:30.408199+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:33:57.462736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.398630+0000","last_change":"2024-05-20T18:35:30.398630+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.398630+0000","last_undegraded":"2024-05-20T18:35:30.398630+0000","last_fullsized":"2024-05-20T18:35:30.398630+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.413623+0000","last_change":"2024-05-20T18:35:30.412896+0000","last_active":"2024-05-20T18:35:30.413623+0000","last_peered":"2024-05-20T18:35:30.413623+0000","last_clean":"2024-05-20T18:35:30.413623+0000","last_became_active":"2024-05-20T18:35:30.412388+0000","last_became_peered":"2024-05-20T18:35:30.412388+0000","last_unstale":"2024-05-20T18:35:30.413623+0000","last_undegraded":"2024-05-20T18:35:30.413623+0000","last_fullsized":"2024-05-20T18:35:30.413623+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:38:14.297654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395713+0000","last_change":"2024-05-20T18:35:30.395713+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395713+0000","last_undegraded":"2024-05-20T18:35:30.395713+0000","last_fullsized":"2024-05-20T18:35:30.395713+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.413668+0000","last_change":"2024-05-20T18:35:30.411370+0000","last_active":"2024-05-20T18:35:30.413668+0000","last_peered":"2024-05-20T18:35:30.413668+0000","last_clean":"2024-05-20T18:35:30.413668+0000","last_became_active":"2024-05-20T18:35:30.411022+0000","last_became_peered":"2024-05-20T18:35:30.411022+0000","last_unstale":"2024-05-20T18:35:30.413668+0000","last_undegraded":"2024-05-20T18:35:30.413668+0000","last_fullsized":"2024-05-20T18:35:30.413668+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:46:23.446345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":322,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.399749+0000","last_change":"2024-05-20T18:27:03.765694+0000","last_active":"2024-05-20T18:35:30.399749+0000","last_peered":"2024-05-20T18:35:30.399749+0000","last_clean":"2024-05-20T18:35:30.399749+0000","last_became_active":"2024-05-20T18:27:03.764695+0000","last_became_peered":"2024-05-20T18:27:03.764695+0000","last_unstale":"2024-05-20T18:35:30.399749+0000","last_undegraded":"2024-05-20T18:35:30.399749+0000","last_fullsized":"2024-05-20T18:35:30.399749+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_clean_scrub_stamp":"2024-05-20T18:27:02.750853+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:30:04.407487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574954,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32996,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338844,"statfs":{"total":96636764160,"available":96602976256,"internally_reserved":0,"allocated":1859584,"data_stored":1174365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1277952,"data_stored":590664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574952,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1277952,"data_stored":590664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1277952,"data_stored":590664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574954,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1277952,"data_stored":590664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1277952,"data_stored":590664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32424,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339416,"statfs":{"total":96636764160,"available":96603561984,"internally_reserved":0,"allocated":1265664,"data_stored":583997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574953,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33000,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338840,"statfs":{"total":96636764160,"available":96602972160,"internally_reserved":0,"allocated":1859584,"data_stored":1174365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T18:35:35.492 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T18:35:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.641+0000 7f339d8f9640 1 Processor -- start 2024-05-20T18:35:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.641+0000 7f339d8f9640 1 -- start start 2024-05-20T18:35:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.642+0000 7f339d8f9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3398162820 0x7f33980b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.642+0000 7f339d8f9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f339806dae0 0x7f33980b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.642+0000 7f339d8f9640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33980d0630 con 0x7f33980b3e80 2024-05-20T18:35:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.642+0000 7f339d8f9640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f33980d39c0 con 0x7f3398162820 2024-05-20T18:35:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.642+0000 7f339d8f9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f33980d3ca0 con 0x7f339806dae0 2024-05-20T18:35:35.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.643+0000 7f3396ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3398162820 0x7f33980b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:35.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.643+0000 7f3396ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3398162820 0x7f33980b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49598/0 (socket says 172.21.15.5:49598) 2024-05-20T18:35:35.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.643+0000 7f3396ffd640 1 -- 172.21.15.5:0/1077675986 learned_addr learned my addr 172.21.15.5:0/1077675986 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:35.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.643+0000 7f33977fe640 1 --2- 172.21.15.5:0/1077675986 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f339806dae0 0x7f33980b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:35.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.643+0000 7f3396ffd640 1 -- 172.21.15.5:0/1077675986 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f339806dae0 msgr2=0x7f33980b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:35.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.643+0000 7f3396ffd640 1 --2- 172.21.15.5:0/1077675986 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f339806dae0 0x7f33980b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:35.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.643+0000 7f3396ffd640 1 -- 172.21.15.5:0/1077675986 >> v1:172.21.15.63:6789/0 conn(0x7f33980b3e80 legacy=0x7f339806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:35.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.643+0000 7f3396ffd640 1 -- 172.21.15.5:0/1077675986 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33980d3f80 con 0x7f3398162820 2024-05-20T18:35:35.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.644+0000 7f3396ffd640 1 --2- 172.21.15.5:0/1077675986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3398162820 0x7f33980b3940 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f3388009870 tx=0x7f338802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c954c9bc95c687a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:35.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.644+0000 7f3395ffb640 1 -- 172.21.15.5:0/1077675986 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388037070 con 0x7f3398162820 2024-05-20T18:35:35.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.644+0000 7f3395ffb640 1 -- 172.21.15.5:0/1077675986 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3388032440 con 0x7f3398162820 2024-05-20T18:35:35.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.644+0000 7f3395ffb640 1 -- 172.21.15.5:0/1077675986 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388032810 con 0x7f3398162820 2024-05-20T18:35:35.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.645+0000 7f339d8f9640 1 -- 172.21.15.5:0/1077675986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3398162820 msgr2=0x7f33980b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:35.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.645+0000 7f339d8f9640 1 --2- 172.21.15.5:0/1077675986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3398162820 0x7f33980b3940 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f3388009870 tx=0x7f338802a210 comp rx=0 tx=0).stop 2024-05-20T18:35:35.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.645+0000 7f339d8f9640 1 -- 172.21.15.5:0/1077675986 shutdown_connections 2024-05-20T18:35:35.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.645+0000 7f339d8f9640 1 --2- 172.21.15.5:0/1077675986 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f339806dae0 0x7f33980b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:35.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.645+0000 7f339d8f9640 1 --2- 172.21.15.5:0/1077675986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3398162820 0x7f33980b3940 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:35.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.645+0000 7f339d8f9640 1 -- 172.21.15.5:0/1077675986 >> 172.21.15.5:0/1077675986 conn(0x7f33980d4bd0 msgr2=0x7f33980d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:35.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.645+0000 7f339d8f9640 1 -- 172.21.15.5:0/1077675986 shutdown_connections 2024-05-20T18:35:35.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.645+0000 7f339d8f9640 1 -- 172.21.15.5:0/1077675986 wait complete. 2024-05-20T18:35:35.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.646+0000 7f339d8f9640 1 Processor -- start 2024-05-20T18:35:35.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.647+0000 7f339d8f9640 1 -- start start 2024-05-20T18:35:35.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.648+0000 7f339d8f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33980b3e80 0x7f339815b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:35.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.648+0000 7f339d8f9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3398162820 0x7f33980cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:35.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.648+0000 7f339d8f9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33981e83a0 0x7f33981e87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.648+0000 7f339d8f9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f33980d4780 con 0x7f33980b3e80 2024-05-20T18:35:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.648+0000 7f339d8f9640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f33980d0300 con 0x7f3398162820 2024-05-20T18:35:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.648+0000 7f339d8f9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f33980d4900 con 0x7f33981e83a0 2024-05-20T18:35:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.648+0000 7f3396ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33980b3e80 0x7f339815b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.648+0000 7f3396ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33980b3e80 0x7f339815b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49602/0 (socket says 172.21.15.5:49602) 2024-05-20T18:35:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.648+0000 7f3396ffd640 1 -- 172.21.15.5:0/3866567783 learned_addr learned my addr 172.21.15.5:0/3866567783 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.648+0000 7f3396ffd640 1 -- 172.21.15.5:0/3866567783 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33981e83a0 msgr2=0x7f33981e87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.648+0000 7f3396ffd640 1 --2- 172.21.15.5:0/3866567783 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33981e83a0 0x7f33981e87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.648+0000 7f3396ffd640 1 -- 172.21.15.5:0/3866567783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3398162820 msgr2=0x7f33980cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.648+0000 7f33967fc640 1 --2- 172.21.15.5:0/3866567783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3398162820 0x7f33980cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.648+0000 7f3396ffd640 1 --2- 172.21.15.5:0/3866567783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3398162820 0x7f33980cb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.648+0000 7f3396ffd640 1 -- 172.21.15.5:0/3866567783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33981e8cf0 con 0x7f33980b3e80 2024-05-20T18:35:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.648+0000 7f33967fc640 1 --2- 172.21.15.5:0/3866567783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3398162820 0x7f33980cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:35.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.649+0000 7f3396ffd640 1 --2- 172.21.15.5:0/3866567783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33980b3e80 0x7f339815b460 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f3388000c00 tx=0x7f33880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:35.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.649+0000 7f337ffff640 1 -- 172.21.15.5:0/3866567783 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388041020 con 0x7f33980b3e80 2024-05-20T18:35:35.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.649+0000 7f337ffff640 1 -- 172.21.15.5:0/3866567783 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33880306c0 con 0x7f33980b3e80 2024-05-20T18:35:35.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.649+0000 7f339d8f9640 1 -- 172.21.15.5:0/3866567783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f339815b9a0 con 0x7f33980b3e80 2024-05-20T18:35:35.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.649+0000 7f337ffff640 1 -- 172.21.15.5:0/3866567783 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388037040 con 0x7f33980b3e80 2024-05-20T18:35:35.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.649+0000 7f339d8f9640 1 -- 172.21.15.5:0/3866567783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33980cd540 con 0x7f33980b3e80 2024-05-20T18:35:35.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.650+0000 7f339d8f9640 1 -- 172.21.15.5:0/3866567783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3398075b00 con 0x7f33980b3e80 2024-05-20T18:35:35.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.651+0000 7f337ffff640 1 -- 172.21.15.5:0/3866567783 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3388044020 con 0x7f33980b3e80 2024-05-20T18:35:35.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.651+0000 7f337ffff640 1 --2- 172.21.15.5:0/3866567783 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f338406be30 0x7f338406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:35.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.651+0000 7f337ffff640 1 -- 172.21.15.5:0/3866567783 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7f33880ab930 con 0x7f33980b3e80 2024-05-20T18:35:35.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.651+0000 7f33967fc640 1 --2- 172.21.15.5:0/3866567783 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f338406be30 0x7f338406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:35.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.652+0000 7f33967fc640 1 --2- 172.21.15.5:0/3866567783 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f338406be30 0x7f338406e340 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f3398162e80 tx=0x7f338c0073a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.656+0000 7f337ffff640 1 -- 172.21.15.5:0/3866567783 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f338804b030 con 0x7f33980b3e80 2024-05-20T18:35:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.836+0000 7f339d8f9640 1 -- 172.21.15.5:0/3866567783 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f33981622f0 con 0x7f338406be30 2024-05-20T18:35:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.842+0000 7f337ffff640 1 -- 172.21.15.5:0/3866567783 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247091 (secure 0 0 0) 0x7f33981622f0 con 0x7f338406be30 2024-05-20T18:35:35.846 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:35.847 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-20T18:35:35.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.845+0000 7f339d8f9640 1 -- 172.21.15.5:0/3866567783 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f338406be30 msgr2=0x7f338406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:35.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.845+0000 7f339d8f9640 1 --2- 172.21.15.5:0/3866567783 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f338406be30 0x7f338406e340 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f3398162e80 tx=0x7f338c0073a0 comp rx=0 tx=0).stop 2024-05-20T18:35:35.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.845+0000 7f339d8f9640 1 -- 172.21.15.5:0/3866567783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33980b3e80 msgr2=0x7f339815b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:35.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.845+0000 7f339d8f9640 1 --2- 172.21.15.5:0/3866567783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33980b3e80 0x7f339815b460 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f3388000c00 tx=0x7f33880046c0 comp rx=0 tx=0).stop 2024-05-20T18:35:35.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.845+0000 7f339d8f9640 1 -- 172.21.15.5:0/3866567783 shutdown_connections 2024-05-20T18:35:35.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.845+0000 7f339d8f9640 1 --2- 172.21.15.5:0/3866567783 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f338406be30 0x7f338406e340 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:35.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.845+0000 7f339d8f9640 1 --2- 172.21.15.5:0/3866567783 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33981e83a0 0x7f33981e87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:35.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.845+0000 7f339d8f9640 1 --2- 172.21.15.5:0/3866567783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3398162820 0x7f33980cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:35.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.845+0000 7f339d8f9640 1 --2- 172.21.15.5:0/3866567783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33980b3e80 0x7f339815b460 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:35.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.845+0000 7f339d8f9640 1 -- 172.21.15.5:0/3866567783 >> 172.21.15.5:0/3866567783 conn(0x7f33980d4bd0 msgr2=0x7f339806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:35.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.846+0000 7f339d8f9640 1 -- 172.21.15.5:0/3866567783 shutdown_connections 2024-05-20T18:35:35.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:35.846+0000 7f339d8f9640 1 -- 172.21.15.5:0/3866567783 wait complete. 2024-05-20T18:35:35.858 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-05-20T18:35:34.704565+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":49,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260564,"kb_used_data":11108,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714156,"statfs":{"total":773094113280,"available":772827295744,"internally_reserved":0,"allocated":11374592,"data_stored":5886047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.319617"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.401556+0000","last_change":"2024-05-20T18:35:30.401556+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.401556+0000","last_undegraded":"2024-05-20T18:35:30.401556+0000","last_fullsized":"2024-05-20T18:35:30.401556+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395067+0000","last_change":"2024-05-20T18:35:30.395067+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395067+0000","last_undegraded":"2024-05-20T18:35:30.395067+0000","last_fullsized":"2024-05-20T18:35:30.395067+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.400834+0000","last_change":"2024-05-20T18:35:30.400834+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.400834+0000","last_undegraded":"2024-05-20T18:35:30.400834+0000","last_fullsized":"2024-05-20T18:35:30.400834+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.396143+0000","last_change":"2024-05-20T18:35:30.396143+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.396143+0000","last_undegraded":"2024-05-20T18:35:30.396143+0000","last_fullsized":"2024-05-20T18:35:30.396143+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.396956+0000","last_change":"2024-05-20T18:35:30.396956+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.396956+0000","last_undegraded":"2024-05-20T18:35:30.396956+0000","last_fullsized":"2024-05-20T18:35:30.396956+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.393816+0000","last_change":"2024-05-20T18:35:30.393816+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.393816+0000","last_undegraded":"2024-05-20T18:35:30.393816+0000","last_fullsized":"2024-05-20T18:35:30.393816+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.394799+0000","last_change":"2024-05-20T18:35:30.394799+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.394799+0000","last_undegraded":"2024-05-20T18:35:30.394799+0000","last_fullsized":"2024-05-20T18:35:30.394799+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.401809+0000","last_change":"2024-05-20T18:35:30.401809+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.401809+0000","last_undegraded":"2024-05-20T18:35:30.401809+0000","last_fullsized":"2024-05-20T18:35:30.401809+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.394202+0000","last_change":"2024-05-20T18:35:30.394202+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.394202+0000","last_undegraded":"2024-05-20T18:35:30.394202+0000","last_fullsized":"2024-05-20T18:35:30.394202+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395453+0000","last_change":"2024-05-20T18:35:30.395453+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395453+0000","last_undegraded":"2024-05-20T18:35:30.395453+0000","last_fullsized":"2024-05-20T18:35:30.395453+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.397481+0000","last_change":"2024-05-20T18:35:30.397481+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.397481+0000","last_undegraded":"2024-05-20T18:35:30.397481+0000","last_fullsized":"2024-05-20T18:35:30.397481+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.391506+0000","last_change":"2024-05-20T18:35:30.391506+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.391506+0000","last_undegraded":"2024-05-20T18:35:30.391506+0000","last_fullsized":"2024-05-20T18:35:30.391506+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.394975+0000","last_change":"2024-05-20T18:35:30.394975+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.394975+0000","last_undegraded":"2024-05-20T18:35:30.394975+0000","last_fullsized":"2024-05-20T18:35:30.394975+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.399091+0000","last_change":"2024-05-20T18:35:30.399091+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.399091+0000","last_undegraded":"2024-05-20T18:35:30.399091+0000","last_fullsized":"2024-05-20T18:35:30.399091+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.398451+0000","last_change":"2024-05-20T18:35:30.398451+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.398451+0000","last_undegraded":"2024-05-20T18:35:30.398451+0000","last_fullsized":"2024-05-20T18:35:30.398451+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.399063+0000","last_change":"2024-05-20T18:35:30.399063+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.399063+0000","last_undegraded":"2024-05-20T18:35:30.399063+0000","last_fullsized":"2024-05-20T18:35:30.399063+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.400625+0000","last_change":"2024-05-20T18:35:30.400625+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.400625+0000","last_undegraded":"2024-05-20T18:35:30.400625+0000","last_fullsized":"2024-05-20T18:35:30.400625+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.394511+0000","last_change":"2024-05-20T18:35:30.394511+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.394511+0000","last_undegraded":"2024-05-20T18:35:30.394511+0000","last_fullsized":"2024-05-20T18:35:30.394511+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.400130+0000","last_change":"2024-05-20T18:35:30.400130+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.400130+0000","last_undegraded":"2024-05-20T18:35:30.400130+0000","last_fullsized":"2024-05-20T18:35:30.400130+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.390388+0000","last_change":"2024-05-20T18:35:30.390388+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.390388+0000","last_undegraded":"2024-05-20T18:35:30.390388+0000","last_fullsized":"2024-05-20T18:35:30.390388+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.401295+0000","last_change":"2024-05-20T18:35:30.401295+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.401295+0000","last_undegraded":"2024-05-20T18:35:30.401295+0000","last_fullsized":"2024-05-20T18:35:30.401295+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395921+0000","last_change":"2024-05-20T18:35:30.395921+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395921+0000","last_undegraded":"2024-05-20T18:35:30.395921+0000","last_fullsized":"2024-05-20T18:35:30.395921+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.400238+0000","last_change":"2024-05-20T18:35:30.400238+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.400238+0000","last_undegraded":"2024-05-20T18:35:30.400238+0000","last_fullsized":"2024-05-20T18:35:30.400238+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.396399+0000","last_change":"2024-05-20T18:35:30.396399+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.396399+0000","last_undegraded":"2024-05-20T18:35:30.396399+0000","last_fullsized":"2024-05-20T18:35:30.396399+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.391428+0000","last_change":"2024-05-20T18:35:30.391428+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.391428+0000","last_undegraded":"2024-05-20T18:35:30.391428+0000","last_fullsized":"2024-05-20T18:35:30.391428+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.391347+0000","last_change":"2024-05-20T18:35:30.391347+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.391347+0000","last_undegraded":"2024-05-20T18:35:30.391347+0000","last_fullsized":"2024-05-20T18:35:30.391347+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.396899+0000","last_change":"2024-05-20T18:35:30.396899+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.396899+0000","last_undegraded":"2024-05-20T18:35:30.396899+0000","last_fullsized":"2024-05-20T18:35:30.396899+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395384+0000","last_change":"2024-05-20T18:35:30.395384+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395384+0000","last_undegraded":"2024-05-20T18:35:30.395384+0000","last_fullsized":"2024-05-20T18:35:30.395384+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.400522+0000","last_change":"2024-05-20T18:35:30.400522+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.400522+0000","last_undegraded":"2024-05-20T18:35:30.400522+0000","last_fullsized":"2024-05-20T18:35:30.400522+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.392369+0000","last_change":"2024-05-20T18:35:30.392369+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.392369+0000","last_undegraded":"2024-05-20T18:35:30.392369+0000","last_fullsized":"2024-05-20T18:35:30.392369+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.394219+0000","last_change":"2024-05-20T18:35:30.394219+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.394219+0000","last_undegraded":"2024-05-20T18:35:30.394219+0000","last_fullsized":"2024-05-20T18:35:30.394219+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.399575+0000","last_change":"2024-05-20T18:35:30.399575+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.399575+0000","last_undegraded":"2024-05-20T18:35:30.399575+0000","last_fullsized":"2024-05-20T18:35:30.399575+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.393568+0000","last_change":"2024-05-20T18:35:30.393568+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.393568+0000","last_undegraded":"2024-05-20T18:35:30.393568+0000","last_fullsized":"2024-05-20T18:35:30.393568+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.410235+0000","last_change":"2024-05-20T18:35:30.407837+0000","last_active":"2024-05-20T18:35:30.410235+0000","last_peered":"2024-05-20T18:35:30.410235+0000","last_clean":"2024-05-20T18:35:30.410235+0000","last_became_active":"2024-05-20T18:35:30.407539+0000","last_became_peered":"2024-05-20T18:35:30.407539+0000","last_unstale":"2024-05-20T18:35:30.410235+0000","last_undegraded":"2024-05-20T18:35:30.410235+0000","last_fullsized":"2024-05-20T18:35:30.410235+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:31:20.340446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395875+0000","last_change":"2024-05-20T18:35:30.395875+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395875+0000","last_undegraded":"2024-05-20T18:35:30.395875+0000","last_fullsized":"2024-05-20T18:35:30.395875+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.409396+0000","last_change":"2024-05-20T18:35:30.406366+0000","last_active":"2024-05-20T18:35:30.409396+0000","last_peered":"2024-05-20T18:35:30.409396+0000","last_clean":"2024-05-20T18:35:30.409396+0000","last_became_active":"2024-05-20T18:35:30.405770+0000","last_became_peered":"2024-05-20T18:35:30.405770+0000","last_unstale":"2024-05-20T18:35:30.409396+0000","last_undegraded":"2024-05-20T18:35:30.409396+0000","last_fullsized":"2024-05-20T18:35:30.409396+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:50:24.603292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.390568+0000","last_change":"2024-05-20T18:35:30.390568+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.390568+0000","last_undegraded":"2024-05-20T18:35:30.390568+0000","last_fullsized":"2024-05-20T18:35:30.390568+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.412922+0000","last_change":"2024-05-20T18:35:30.409253+0000","last_active":"2024-05-20T18:35:30.412922+0000","last_peered":"2024-05-20T18:35:30.412922+0000","last_clean":"2024-05-20T18:35:30.412922+0000","last_became_active":"2024-05-20T18:35:30.408847+0000","last_became_peered":"2024-05-20T18:35:30.408847+0000","last_unstale":"2024-05-20T18:35:30.412922+0000","last_undegraded":"2024-05-20T18:35:30.412922+0000","last_fullsized":"2024-05-20T18:35:30.412922+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:13:02.199994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.396581+0000","last_change":"2024-05-20T18:35:30.396581+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.396581+0000","last_undegraded":"2024-05-20T18:35:30.396581+0000","last_fullsized":"2024-05-20T18:35:30.396581+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.406960+0000","last_change":"2024-05-20T18:35:30.402307+0000","last_active":"2024-05-20T18:35:30.406960+0000","last_peered":"2024-05-20T18:35:30.406960+0000","last_clean":"2024-05-20T18:35:30.406960+0000","last_became_active":"2024-05-20T18:35:30.401952+0000","last_became_peered":"2024-05-20T18:35:30.401952+0000","last_unstale":"2024-05-20T18:35:30.406960+0000","last_undegraded":"2024-05-20T18:35:30.406960+0000","last_fullsized":"2024-05-20T18:35:30.406960+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:47:28.651899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395611+0000","last_change":"2024-05-20T18:35:30.395611+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395611+0000","last_undegraded":"2024-05-20T18:35:30.395611+0000","last_fullsized":"2024-05-20T18:35:30.395611+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.407964+0000","last_change":"2024-05-20T18:35:30.405866+0000","last_active":"2024-05-20T18:35:30.407964+0000","last_peered":"2024-05-20T18:35:30.407964+0000","last_clean":"2024-05-20T18:35:30.407964+0000","last_became_active":"2024-05-20T18:35:30.405503+0000","last_became_peered":"2024-05-20T18:35:30.405503+0000","last_unstale":"2024-05-20T18:35:30.407964+0000","last_undegraded":"2024-05-20T18:35:30.407964+0000","last_fullsized":"2024-05-20T18:35:30.407964+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:16:21.553268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.393963+0000","last_change":"2024-05-20T18:35:30.393963+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.393963+0000","last_undegraded":"2024-05-20T18:35:30.393963+0000","last_fullsized":"2024-05-20T18:35:30.393963+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.414268+0000","last_change":"2024-05-20T18:35:30.412485+0000","last_active":"2024-05-20T18:35:30.414268+0000","last_peered":"2024-05-20T18:35:30.414268+0000","last_clean":"2024-05-20T18:35:30.414268+0000","last_became_active":"2024-05-20T18:35:30.412162+0000","last_became_peered":"2024-05-20T18:35:30.412162+0000","last_unstale":"2024-05-20T18:35:30.414268+0000","last_undegraded":"2024-05-20T18:35:30.414268+0000","last_fullsized":"2024-05-20T18:35:30.414268+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:31:19.619870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.393425+0000","last_change":"2024-05-20T18:35:30.393425+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.393425+0000","last_undegraded":"2024-05-20T18:35:30.393425+0000","last_fullsized":"2024-05-20T18:35:30.393425+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.412482+0000","last_change":"2024-05-20T18:35:30.409374+0000","last_active":"2024-05-20T18:35:30.412482+0000","last_peered":"2024-05-20T18:35:30.412482+0000","last_clean":"2024-05-20T18:35:30.412482+0000","last_became_active":"2024-05-20T18:35:30.408970+0000","last_became_peered":"2024-05-20T18:35:30.408970+0000","last_unstale":"2024-05-20T18:35:30.412482+0000","last_undegraded":"2024-05-20T18:35:30.412482+0000","last_fullsized":"2024-05-20T18:35:30.412482+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:19:27.219675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395456+0000","last_change":"2024-05-20T18:35:30.395456+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395456+0000","last_undegraded":"2024-05-20T18:35:30.395456+0000","last_fullsized":"2024-05-20T18:35:30.395456+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.408240+0000","last_change":"2024-05-20T18:35:30.406064+0000","last_active":"2024-05-20T18:35:30.408240+0000","last_peered":"2024-05-20T18:35:30.408240+0000","last_clean":"2024-05-20T18:35:30.408240+0000","last_became_active":"2024-05-20T18:35:30.405710+0000","last_became_peered":"2024-05-20T18:35:30.405710+0000","last_unstale":"2024-05-20T18:35:30.408240+0000","last_undegraded":"2024-05-20T18:35:30.408240+0000","last_fullsized":"2024-05-20T18:35:30.408240+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:23:16.072920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.394555+0000","last_change":"2024-05-20T18:35:30.394555+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.394555+0000","last_undegraded":"2024-05-20T18:35:30.394555+0000","last_fullsized":"2024-05-20T18:35:30.394555+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.412758+0000","last_change":"2024-05-20T18:35:30.409880+0000","last_active":"2024-05-20T18:35:30.412758+0000","last_peered":"2024-05-20T18:35:30.412758+0000","last_clean":"2024-05-20T18:35:30.412758+0000","last_became_active":"2024-05-20T18:35:30.409517+0000","last_became_peered":"2024-05-20T18:35:30.409517+0000","last_unstale":"2024-05-20T18:35:30.412758+0000","last_undegraded":"2024-05-20T18:35:30.412758+0000","last_fullsized":"2024-05-20T18:35:30.412758+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:33:58.473775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.393400+0000","last_change":"2024-05-20T18:35:30.393400+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.393400+0000","last_undegraded":"2024-05-20T18:35:30.393400+0000","last_fullsized":"2024-05-20T18:35:30.393400+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.408544+0000","last_change":"2024-05-20T18:35:30.403374+0000","last_active":"2024-05-20T18:35:30.408544+0000","last_peered":"2024-05-20T18:35:30.408544+0000","last_clean":"2024-05-20T18:35:30.408544+0000","last_became_active":"2024-05-20T18:35:30.403012+0000","last_became_peered":"2024-05-20T18:35:30.403012+0000","last_unstale":"2024-05-20T18:35:30.408544+0000","last_undegraded":"2024-05-20T18:35:30.408544+0000","last_fullsized":"2024-05-20T18:35:30.408544+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:45:42.189012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.397246+0000","last_change":"2024-05-20T18:35:30.397246+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.397246+0000","last_undegraded":"2024-05-20T18:35:30.397246+0000","last_fullsized":"2024-05-20T18:35:30.397246+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.411337+0000","last_change":"2024-05-20T18:35:30.410506+0000","last_active":"2024-05-20T18:35:30.411337+0000","last_peered":"2024-05-20T18:35:30.411337+0000","last_clean":"2024-05-20T18:35:30.411337+0000","last_became_active":"2024-05-20T18:35:30.410168+0000","last_became_peered":"2024-05-20T18:35:30.410168+0000","last_unstale":"2024-05-20T18:35:30.411337+0000","last_undegraded":"2024-05-20T18:35:30.411337+0000","last_fullsized":"2024-05-20T18:35:30.411337+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:19:34.158678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.394072+0000","last_change":"2024-05-20T18:35:30.394072+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.394072+0000","last_undegraded":"2024-05-20T18:35:30.394072+0000","last_fullsized":"2024-05-20T18:35:30.394072+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.413578+0000","last_change":"2024-05-20T18:35:30.412065+0000","last_active":"2024-05-20T18:35:30.413578+0000","last_peered":"2024-05-20T18:35:30.413578+0000","last_clean":"2024-05-20T18:35:30.413578+0000","last_became_active":"2024-05-20T18:35:30.410006+0000","last_became_peered":"2024-05-20T18:35:30.410006+0000","last_unstale":"2024-05-20T18:35:30.413578+0000","last_undegraded":"2024-05-20T18:35:30.413578+0000","last_fullsized":"2024-05-20T18:35:30.413578+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:47:28.651899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395562+0000","last_change":"2024-05-20T18:35:30.395562+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395562+0000","last_undegraded":"2024-05-20T18:35:30.395562+0000","last_fullsized":"2024-05-20T18:35:30.395562+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.408708+0000","last_change":"2024-05-20T18:35:30.406644+0000","last_active":"2024-05-20T18:35:30.408708+0000","last_peered":"2024-05-20T18:35:30.408708+0000","last_clean":"2024-05-20T18:35:30.408708+0000","last_became_active":"2024-05-20T18:35:30.406155+0000","last_became_peered":"2024-05-20T18:35:30.406155+0000","last_unstale":"2024-05-20T18:35:30.408708+0000","last_undegraded":"2024-05-20T18:35:30.408708+0000","last_fullsized":"2024-05-20T18:35:30.408708+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:24:43.933783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.392885+0000","last_change":"2024-05-20T18:35:30.392885+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.392885+0000","last_undegraded":"2024-05-20T18:35:30.392885+0000","last_fullsized":"2024-05-20T18:35:30.392885+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.408224+0000","last_change":"2024-05-20T18:35:30.404725+0000","last_active":"2024-05-20T18:35:30.408224+0000","last_peered":"2024-05-20T18:35:30.408224+0000","last_clean":"2024-05-20T18:35:30.408224+0000","last_became_active":"2024-05-20T18:35:30.404404+0000","last_became_peered":"2024-05-20T18:35:30.404404+0000","last_unstale":"2024-05-20T18:35:30.408224+0000","last_undegraded":"2024-05-20T18:35:30.408224+0000","last_fullsized":"2024-05-20T18:35:30.408224+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:22:09.796441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.396812+0000","last_change":"2024-05-20T18:35:30.396812+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.396812+0000","last_undegraded":"2024-05-20T18:35:30.396812+0000","last_fullsized":"2024-05-20T18:35:30.396812+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.415211+0000","last_change":"2024-05-20T18:35:30.414408+0000","last_active":"2024-05-20T18:35:30.415211+0000","last_peered":"2024-05-20T18:35:30.415211+0000","last_clean":"2024-05-20T18:35:30.415211+0000","last_became_active":"2024-05-20T18:35:30.414025+0000","last_became_peered":"2024-05-20T18:35:30.414025+0000","last_unstale":"2024-05-20T18:35:30.415211+0000","last_undegraded":"2024-05-20T18:35:30.415211+0000","last_fullsized":"2024-05-20T18:35:30.415211+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:52:17.777292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395135+0000","last_change":"2024-05-20T18:35:30.395135+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395135+0000","last_undegraded":"2024-05-20T18:35:30.395135+0000","last_fullsized":"2024-05-20T18:35:30.395135+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.412446+0000","last_change":"2024-05-20T18:35:30.408547+0000","last_active":"2024-05-20T18:35:30.412446+0000","last_peered":"2024-05-20T18:35:30.412446+0000","last_clean":"2024-05-20T18:35:30.412446+0000","last_became_active":"2024-05-20T18:35:30.408205+0000","last_became_peered":"2024-05-20T18:35:30.408205+0000","last_unstale":"2024-05-20T18:35:30.412446+0000","last_undegraded":"2024-05-20T18:35:30.412446+0000","last_fullsized":"2024-05-20T18:35:30.412446+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:59:20.346125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.397543+0000","last_change":"2024-05-20T18:35:30.397543+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.397543+0000","last_undegraded":"2024-05-20T18:35:30.397543+0000","last_fullsized":"2024-05-20T18:35:30.397543+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.412005+0000","last_change":"2024-05-20T18:35:30.408259+0000","last_active":"2024-05-20T18:35:30.412005+0000","last_peered":"2024-05-20T18:35:30.412005+0000","last_clean":"2024-05-20T18:35:30.412005+0000","last_became_active":"2024-05-20T18:35:30.407911+0000","last_became_peered":"2024-05-20T18:35:30.407911+0000","last_unstale":"2024-05-20T18:35:30.412005+0000","last_undegraded":"2024-05-20T18:35:30.412005+0000","last_fullsized":"2024-05-20T18:35:30.412005+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:30:06.656452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.392761+0000","last_change":"2024-05-20T18:35:30.392761+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.392761+0000","last_undegraded":"2024-05-20T18:35:30.392761+0000","last_fullsized":"2024-05-20T18:35:30.392761+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.413312+0000","last_change":"2024-05-20T18:35:30.412088+0000","last_active":"2024-05-20T18:35:30.413312+0000","last_peered":"2024-05-20T18:35:30.413312+0000","last_clean":"2024-05-20T18:35:30.413312+0000","last_became_active":"2024-05-20T18:35:30.411708+0000","last_became_peered":"2024-05-20T18:35:30.411708+0000","last_unstale":"2024-05-20T18:35:30.413312+0000","last_undegraded":"2024-05-20T18:35:30.413312+0000","last_fullsized":"2024-05-20T18:35:30.413312+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:26:29.213682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.394854+0000","last_change":"2024-05-20T18:35:30.394854+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.394854+0000","last_undegraded":"2024-05-20T18:35:30.394854+0000","last_fullsized":"2024-05-20T18:35:30.394854+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.407629+0000","last_change":"2024-05-20T18:35:30.406195+0000","last_active":"2024-05-20T18:35:30.407629+0000","last_peered":"2024-05-20T18:35:30.407629+0000","last_clean":"2024-05-20T18:35:30.407629+0000","last_became_active":"2024-05-20T18:35:30.405856+0000","last_became_peered":"2024-05-20T18:35:30.405856+0000","last_unstale":"2024-05-20T18:35:30.407629+0000","last_undegraded":"2024-05-20T18:35:30.407629+0000","last_fullsized":"2024-05-20T18:35:30.407629+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:33:58.473775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395697+0000","last_change":"2024-05-20T18:35:30.395697+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395697+0000","last_undegraded":"2024-05-20T18:35:30.395697+0000","last_fullsized":"2024-05-20T18:35:30.395697+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.413399+0000","last_change":"2024-05-20T18:35:30.411351+0000","last_active":"2024-05-20T18:35:30.413399+0000","last_peered":"2024-05-20T18:35:30.413399+0000","last_clean":"2024-05-20T18:35:30.413399+0000","last_became_active":"2024-05-20T18:35:30.410938+0000","last_became_peered":"2024-05-20T18:35:30.410938+0000","last_unstale":"2024-05-20T18:35:30.413399+0000","last_undegraded":"2024-05-20T18:35:30.413399+0000","last_fullsized":"2024-05-20T18:35:30.413399+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:45:26.959959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.396251+0000","last_change":"2024-05-20T18:35:30.396251+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.396251+0000","last_undegraded":"2024-05-20T18:35:30.396251+0000","last_fullsized":"2024-05-20T18:35:30.396251+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.413171+0000","last_change":"2024-05-20T18:35:30.410136+0000","last_active":"2024-05-20T18:35:30.413171+0000","last_peered":"2024-05-20T18:35:30.413171+0000","last_clean":"2024-05-20T18:35:30.413171+0000","last_became_active":"2024-05-20T18:35:30.409778+0000","last_became_peered":"2024-05-20T18:35:30.409778+0000","last_unstale":"2024-05-20T18:35:30.413171+0000","last_undegraded":"2024-05-20T18:35:30.413171+0000","last_fullsized":"2024-05-20T18:35:30.413171+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:37:58.981434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.397965+0000","last_change":"2024-05-20T18:35:30.397965+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.397965+0000","last_undegraded":"2024-05-20T18:35:30.397965+0000","last_fullsized":"2024-05-20T18:35:30.397965+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.411835+0000","last_change":"2024-05-20T18:35:30.407327+0000","last_active":"2024-05-20T18:35:30.411835+0000","last_peered":"2024-05-20T18:35:30.411835+0000","last_clean":"2024-05-20T18:35:30.411835+0000","last_became_active":"2024-05-20T18:35:30.406994+0000","last_became_peered":"2024-05-20T18:35:30.406994+0000","last_unstale":"2024-05-20T18:35:30.411835+0000","last_undegraded":"2024-05-20T18:35:30.411835+0000","last_fullsized":"2024-05-20T18:35:30.411835+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:22:23.532551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.398473+0000","last_change":"2024-05-20T18:35:30.398473+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.398473+0000","last_undegraded":"2024-05-20T18:35:30.398473+0000","last_fullsized":"2024-05-20T18:35:30.398473+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.409293+0000","last_change":"2024-05-20T18:35:30.404056+0000","last_active":"2024-05-20T18:35:30.409293+0000","last_peered":"2024-05-20T18:35:30.409293+0000","last_clean":"2024-05-20T18:35:30.409293+0000","last_became_active":"2024-05-20T18:35:30.403695+0000","last_became_peered":"2024-05-20T18:35:30.403695+0000","last_unstale":"2024-05-20T18:35:30.409293+0000","last_undegraded":"2024-05-20T18:35:30.409293+0000","last_fullsized":"2024-05-20T18:35:30.409293+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:06:34.052366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.399419+0000","last_change":"2024-05-20T18:35:30.399419+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.399419+0000","last_undegraded":"2024-05-20T18:35:30.399419+0000","last_fullsized":"2024-05-20T18:35:30.399419+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.413484+0000","last_change":"2024-05-20T18:35:30.412633+0000","last_active":"2024-05-20T18:35:30.413484+0000","last_peered":"2024-05-20T18:35:30.413484+0000","last_clean":"2024-05-20T18:35:30.413484+0000","last_became_active":"2024-05-20T18:35:30.412307+0000","last_became_peered":"2024-05-20T18:35:30.412307+0000","last_unstale":"2024-05-20T18:35:30.413484+0000","last_undegraded":"2024-05-20T18:35:30.413484+0000","last_fullsized":"2024-05-20T18:35:30.413484+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:59:07.724149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395296+0000","last_change":"2024-05-20T18:35:30.395296+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395296+0000","last_undegraded":"2024-05-20T18:35:30.395296+0000","last_fullsized":"2024-05-20T18:35:30.395296+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.411785+0000","last_change":"2024-05-20T18:35:30.411059+0000","last_active":"2024-05-20T18:35:30.411785+0000","last_peered":"2024-05-20T18:35:30.411785+0000","last_clean":"2024-05-20T18:35:30.411785+0000","last_became_active":"2024-05-20T18:35:30.410688+0000","last_became_peered":"2024-05-20T18:35:30.410688+0000","last_unstale":"2024-05-20T18:35:30.411785+0000","last_undegraded":"2024-05-20T18:35:30.411785+0000","last_fullsized":"2024-05-20T18:35:30.411785+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:41:03.388138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395640+0000","last_change":"2024-05-20T18:35:30.395640+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395640+0000","last_undegraded":"2024-05-20T18:35:30.395640+0000","last_fullsized":"2024-05-20T18:35:30.395640+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.408141+0000","last_change":"2024-05-20T18:35:30.406122+0000","last_active":"2024-05-20T18:35:30.408141+0000","last_peered":"2024-05-20T18:35:30.408141+0000","last_clean":"2024-05-20T18:35:30.408141+0000","last_became_active":"2024-05-20T18:35:30.405784+0000","last_became_peered":"2024-05-20T18:35:30.405784+0000","last_unstale":"2024-05-20T18:35:30.408141+0000","last_undegraded":"2024-05-20T18:35:30.408141+0000","last_fullsized":"2024-05-20T18:35:30.408141+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:24:08.103767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.397137+0000","last_change":"2024-05-20T18:35:30.397137+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.397137+0000","last_undegraded":"2024-05-20T18:35:30.397137+0000","last_fullsized":"2024-05-20T18:35:30.397137+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.404066+0000","last_change":"2024-05-20T18:35:30.400456+0000","last_active":"2024-05-20T18:35:30.404066+0000","last_peered":"2024-05-20T18:35:30.404066+0000","last_clean":"2024-05-20T18:35:30.404066+0000","last_became_active":"2024-05-20T18:35:30.399296+0000","last_became_peered":"2024-05-20T18:35:30.399296+0000","last_unstale":"2024-05-20T18:35:30.404066+0000","last_undegraded":"2024-05-20T18:35:30.404066+0000","last_fullsized":"2024-05-20T18:35:30.404066+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:30:06.656452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.394593+0000","last_change":"2024-05-20T18:35:30.394593+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.394593+0000","last_undegraded":"2024-05-20T18:35:30.394593+0000","last_fullsized":"2024-05-20T18:35:30.394593+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.411083+0000","last_change":"2024-05-20T18:35:30.408084+0000","last_active":"2024-05-20T18:35:30.411083+0000","last_peered":"2024-05-20T18:35:30.411083+0000","last_clean":"2024-05-20T18:35:30.411083+0000","last_became_active":"2024-05-20T18:35:30.407701+0000","last_became_peered":"2024-05-20T18:35:30.407701+0000","last_unstale":"2024-05-20T18:35:30.411083+0000","last_undegraded":"2024-05-20T18:35:30.411083+0000","last_fullsized":"2024-05-20T18:35:30.411083+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:59:53.740452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.399317+0000","last_change":"2024-05-20T18:35:30.399317+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.399317+0000","last_undegraded":"2024-05-20T18:35:30.399317+0000","last_fullsized":"2024-05-20T18:35:30.399317+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.415376+0000","last_change":"2024-05-20T18:35:30.414600+0000","last_active":"2024-05-20T18:35:30.415376+0000","last_peered":"2024-05-20T18:35:30.415376+0000","last_clean":"2024-05-20T18:35:30.415376+0000","last_became_active":"2024-05-20T18:35:30.414218+0000","last_became_peered":"2024-05-20T18:35:30.414218+0000","last_unstale":"2024-05-20T18:35:30.415376+0000","last_undegraded":"2024-05-20T18:35:30.415376+0000","last_fullsized":"2024-05-20T18:35:30.415376+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:58:38.814053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.399261+0000","last_change":"2024-05-20T18:35:30.399261+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.399261+0000","last_undegraded":"2024-05-20T18:35:30.399261+0000","last_fullsized":"2024-05-20T18:35:30.399261+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.408199+0000","last_change":"2024-05-20T18:35:30.406555+0000","last_active":"2024-05-20T18:35:30.408199+0000","last_peered":"2024-05-20T18:35:30.408199+0000","last_clean":"2024-05-20T18:35:30.408199+0000","last_became_active":"2024-05-20T18:35:30.405500+0000","last_became_peered":"2024-05-20T18:35:30.405500+0000","last_unstale":"2024-05-20T18:35:30.408199+0000","last_undegraded":"2024-05-20T18:35:30.408199+0000","last_fullsized":"2024-05-20T18:35:30.408199+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:33:57.462736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.398630+0000","last_change":"2024-05-20T18:35:30.398630+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.398630+0000","last_undegraded":"2024-05-20T18:35:30.398630+0000","last_fullsized":"2024-05-20T18:35:30.398630+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.413623+0000","last_change":"2024-05-20T18:35:30.412896+0000","last_active":"2024-05-20T18:35:30.413623+0000","last_peered":"2024-05-20T18:35:30.413623+0000","last_clean":"2024-05-20T18:35:30.413623+0000","last_became_active":"2024-05-20T18:35:30.412388+0000","last_became_peered":"2024-05-20T18:35:30.412388+0000","last_unstale":"2024-05-20T18:35:30.413623+0000","last_undegraded":"2024-05-20T18:35:30.413623+0000","last_fullsized":"2024-05-20T18:35:30.413623+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:38:14.297654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395713+0000","last_change":"2024-05-20T18:35:30.395713+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395713+0000","last_undegraded":"2024-05-20T18:35:30.395713+0000","last_fullsized":"2024-05-20T18:35:30.395713+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.413668+0000","last_change":"2024-05-20T18:35:30.411370+0000","last_active":"2024-05-20T18:35:30.413668+0000","last_peered":"2024-05-20T18:35:30.413668+0000","last_clean":"2024-05-20T18:35:30.413668+0000","last_became_active":"2024-05-20T18:35:30.411022+0000","last_became_peered":"2024-05-20T18:35:30.411022+0000","last_unstale":"2024-05-20T18:35:30.413668+0000","last_undegraded":"2024-05-20T18:35:30.413668+0000","last_fullsized":"2024-05-20T18:35:30.413668+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:46:23.446345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":322,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.399749+0000","last_change":"2024-05-20T18:27:03.765694+0000","last_active":"2024-05-20T18:35:30.399749+0000","last_peered":"2024-05-20T18:35:30.399749+0000","last_clean":"2024-05-20T18:35:30.399749+0000","last_became_active":"2024-05-20T18:27:03.764695+0000","last_became_peered":"2024-05-20T18:27:03.764695+0000","last_unstale":"2024-05-20T18:35:30.399749+0000","last_undegraded":"2024-05-20T18:35:30.399749+0000","last_fullsized":"2024-05-20T18:35:30.399749+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_clean_scrub_stamp":"2024-05-20T18:27:02.750853+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:30:04.407487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574954,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32996,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338844,"statfs":{"total":96636764160,"available":96602976256,"internally_reserved":0,"allocated":1859584,"data_stored":1174365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1277952,"data_stored":590664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574952,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1277952,"data_stored":590664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1277952,"data_stored":590664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574954,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1277952,"data_stored":590664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1277952,"data_stored":590664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32424,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339416,"statfs":{"total":96636764160,"available":96603561984,"internally_reserved":0,"allocated":1265664,"data_stored":583997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574953,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33000,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338840,"statfs":{"total":96636764160,"available":96602972160,"internally_reserved":0,"allocated":1859584,"data_stored":1174365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T18:35:35.862 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T18:35:36.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.013+0000 7f43757d5640 1 Processor -- start 2024-05-20T18:35:36.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.013+0000 7f43757d5640 1 -- start start 2024-05-20T18:35:36.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.014+0000 7f43757d5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4370156690 0x7f437015cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:36.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.014+0000 7f43757d5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43700d1610 0x7f43700d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:36.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.014+0000 7f43757d5640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f437006d890 con 0x7f437009a4f0 2024-05-20T18:35:36.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.014+0000 7f43757d5640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f43700b29b0 con 0x7f4370156690 2024-05-20T18:35:36.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.014+0000 7f43757d5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f43700b1910 con 0x7f43700d1610 2024-05-20T18:35:36.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.014+0000 7f436effd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4370156690 0x7f437015cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:36.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.014+0000 7f436f7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f437009a4f0 0x7f437009a900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46720/0 (socket says 172.21.15.5:46720) 2024-05-20T18:35:36.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.014+0000 7f436f7fe640 1 -- 172.21.15.5:0/976136656 learned_addr learned my addr 172.21.15.5:0/976136656 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:36.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.014+0000 7f436e7fc640 1 --2- 172.21.15.5:0/976136656 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43700d1610 0x7f43700d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:36.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.015+0000 7f436effd640 1 -- 172.21.15.5:0/976136656 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43700d1610 msgr2=0x7f43700d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:36.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.015+0000 7f436effd640 1 --2- 172.21.15.5:0/976136656 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43700d1610 0x7f43700d1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.015+0000 7f436effd640 1 -- 172.21.15.5:0/976136656 >> v1:172.21.15.5:6789/0 conn(0x7f437009a4f0 legacy=0x7f437009a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:36.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.015+0000 7f436dffb640 1 -- 172.21.15.5:0/976136656 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 318431366 0 0) 0x7f437006d890 con 0x7f437009a4f0 2024-05-20T18:35:36.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.015+0000 7f436effd640 1 -- 172.21.15.5:0/976136656 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43700b1bf0 con 0x7f4370156690 2024-05-20T18:35:36.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.015+0000 7f436effd640 1 --2- 172.21.15.5:0/976136656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4370156690 0x7f437015cd20 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f43600097a0 tx=0x7f436002a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c3df35f1d88421 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:36.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.015+0000 7f436dffb640 1 -- 172.21.15.5:0/976136656 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360037070 con 0x7f4370156690 2024-05-20T18:35:36.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.016+0000 7f436dffb640 1 -- 172.21.15.5:0/976136656 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f436002adb0 con 0x7f4370156690 2024-05-20T18:35:36.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.016+0000 7f436dffb640 1 -- 172.21.15.5:0/976136656 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360032660 con 0x7f4370156690 2024-05-20T18:35:36.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.016+0000 7f43757d5640 1 -- 172.21.15.5:0/976136656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4370156690 msgr2=0x7f437015cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:36.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.016+0000 7f43757d5640 1 --2- 172.21.15.5:0/976136656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4370156690 0x7f437015cd20 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f43600097a0 tx=0x7f436002a060 comp rx=0 tx=0).stop 2024-05-20T18:35:36.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.016+0000 7f43757d5640 1 -- 172.21.15.5:0/976136656 shutdown_connections 2024-05-20T18:35:36.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.016+0000 7f43757d5640 1 --2- 172.21.15.5:0/976136656 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43700d1610 0x7f43700d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.016+0000 7f43757d5640 1 --2- 172.21.15.5:0/976136656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4370156690 0x7f437015cd20 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.016+0000 7f43757d5640 1 -- 172.21.15.5:0/976136656 >> 172.21.15.5:0/976136656 conn(0x7f43700b6670 msgr2=0x7f43700b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:36.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.016+0000 7f43757d5640 1 -- 172.21.15.5:0/976136656 shutdown_connections 2024-05-20T18:35:36.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.016+0000 7f43757d5640 1 -- 172.21.15.5:0/976136656 wait complete. 2024-05-20T18:35:36.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.018+0000 7f43757d5640 1 Processor -- start 2024-05-20T18:35:36.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.018+0000 7f43757d5640 1 -- start start 2024-05-20T18:35:36.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.019+0000 7f43757d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f437009a4f0 0x7f43701fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:36.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.019+0000 7f43757d5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43700d1610 0x7f437015dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:36.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.019+0000 7f43757d5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4370156690 0x7f437015e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:36.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.019+0000 7f43757d5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f43700b2490 con 0x7f437009a4f0 2024-05-20T18:35:36.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.019+0000 7f43757d5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f43700b21b0 con 0x7f4370156690 2024-05-20T18:35:36.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.019+0000 7f43757d5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f43700d1f60 con 0x7f43700d1610 2024-05-20T18:35:36.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.020+0000 7f436effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f437009a4f0 0x7f43701fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:36.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.020+0000 7f436effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f437009a4f0 0x7f43701fc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49614/0 (socket says 172.21.15.5:49614) 2024-05-20T18:35:36.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.020+0000 7f436effd640 1 -- 172.21.15.5:0/4143849703 learned_addr learned my addr 172.21.15.5:0/4143849703 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:36.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.020+0000 7f436f7fe640 1 --2- 172.21.15.5:0/4143849703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4370156690 0x7f437015e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:36.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.020+0000 7f436e7fc640 1 --2- 172.21.15.5:0/4143849703 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43700d1610 0x7f437015dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:36.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.020+0000 7f436effd640 1 -- 172.21.15.5:0/4143849703 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43700d1610 msgr2=0x7f437015dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:36.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.020+0000 7f436effd640 1 --2- 172.21.15.5:0/4143849703 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43700d1610 0x7f437015dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.020+0000 7f436effd640 1 -- 172.21.15.5:0/4143849703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4370156690 msgr2=0x7f437015e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:36.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.020+0000 7f436effd640 1 --2- 172.21.15.5:0/4143849703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4370156690 0x7f437015e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.020+0000 7f436effd640 1 -- 172.21.15.5:0/4143849703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4370153650 con 0x7f437009a4f0 2024-05-20T18:35:36.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.020+0000 7f436f7fe640 1 --2- 172.21.15.5:0/4143849703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4370156690 0x7f437015e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:36.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.020+0000 7f436effd640 1 --2- 172.21.15.5:0/4143849703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f437009a4f0 0x7f43701fc690 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f43600098c0 tx=0x7f43600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:36.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.020+0000 7f43577fe640 1 -- 172.21.15.5:0/4143849703 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360042070 con 0x7f437009a4f0 2024-05-20T18:35:36.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.020+0000 7f43577fe640 1 -- 172.21.15.5:0/4143849703 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f436003e670 con 0x7f437009a4f0 2024-05-20T18:35:36.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.020+0000 7f43577fe640 1 -- 172.21.15.5:0/4143849703 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360037040 con 0x7f437009a4f0 2024-05-20T18:35:36.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.020+0000 7f43757d5640 1 -- 172.21.15.5:0/4143849703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43701538f0 con 0x7f437009a4f0 2024-05-20T18:35:36.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.021+0000 7f43757d5640 1 -- 172.21.15.5:0/4143849703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4370151320 con 0x7f437009a4f0 2024-05-20T18:35:36.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.022+0000 7f43757d5640 1 -- 172.21.15.5:0/4143849703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4370075b00 con 0x7f437009a4f0 2024-05-20T18:35:36.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.022+0000 7f43577fe640 1 -- 172.21.15.5:0/4143849703 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4360045070 con 0x7f437009a4f0 2024-05-20T18:35:36.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.023+0000 7f43577fe640 1 --2- 172.21.15.5:0/4143849703 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f435c06be30 0x7f435c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:36.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.023+0000 7f43577fe640 1 -- 172.21.15.5:0/4143849703 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7f43600ab880 con 0x7f437009a4f0 2024-05-20T18:35:36.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.023+0000 7f436e7fc640 1 --2- 172.21.15.5:0/4143849703 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f435c06be30 0x7f435c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:36.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.023+0000 7f436e7fc640 1 --2- 172.21.15.5:0/4143849703 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f435c06be30 0x7f435c06e340 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4370156150 tx=0x7f4364008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:36.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.027+0000 7f43577fe640 1 -- 172.21.15.5:0/4143849703 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f436007a020 con 0x7f437009a4f0 2024-05-20T18:35:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.211+0000 7f43757d5640 1 -- 172.21.15.5:0/4143849703 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f43701faf10 con 0x7f435c06be30 2024-05-20T18:35:36.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.217+0000 7f43577fe640 1 -- 172.21.15.5:0/4143849703 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247091 (secure 0 0 0) 0x7f43701faf10 con 0x7f435c06be30 2024-05-20T18:35:36.221 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:36.223 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-20T18:35:36.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.221+0000 7f43757d5640 1 -- 172.21.15.5:0/4143849703 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f435c06be30 msgr2=0x7f435c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:36.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.221+0000 7f43757d5640 1 --2- 172.21.15.5:0/4143849703 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f435c06be30 0x7f435c06e340 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4370156150 tx=0x7f4364008040 comp rx=0 tx=0).stop 2024-05-20T18:35:36.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.221+0000 7f43757d5640 1 -- 172.21.15.5:0/4143849703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f437009a4f0 msgr2=0x7f43701fc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:36.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.221+0000 7f43757d5640 1 --2- 172.21.15.5:0/4143849703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f437009a4f0 0x7f43701fc690 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f43600098c0 tx=0x7f43600046c0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.221+0000 7f43757d5640 1 -- 172.21.15.5:0/4143849703 shutdown_connections 2024-05-20T18:35:36.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.221+0000 7f43757d5640 1 --2- 172.21.15.5:0/4143849703 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f435c06be30 0x7f435c06e340 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.221+0000 7f43757d5640 1 --2- 172.21.15.5:0/4143849703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4370156690 0x7f437015e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.221+0000 7f43757d5640 1 --2- 172.21.15.5:0/4143849703 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43700d1610 0x7f437015dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.221+0000 7f43757d5640 1 --2- 172.21.15.5:0/4143849703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f437009a4f0 0x7f43701fc690 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.221+0000 7f43757d5640 1 -- 172.21.15.5:0/4143849703 >> 172.21.15.5:0/4143849703 conn(0x7f43700b6670 msgr2=0x7f437006f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:36.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.221+0000 7f43757d5640 1 -- 172.21.15.5:0/4143849703 shutdown_connections 2024-05-20T18:35:36.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.221+0000 7f43757d5640 1 -- 172.21.15.5:0/4143849703 wait complete. 2024-05-20T18:35:36.233 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-05-20T18:35:34.704565+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":49,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260564,"kb_used_data":11108,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714156,"statfs":{"total":773094113280,"available":772827295744,"internally_reserved":0,"allocated":11374592,"data_stored":5886047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.319617"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.401556+0000","last_change":"2024-05-20T18:35:30.401556+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.401556+0000","last_undegraded":"2024-05-20T18:35:30.401556+0000","last_fullsized":"2024-05-20T18:35:30.401556+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395067+0000","last_change":"2024-05-20T18:35:30.395067+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395067+0000","last_undegraded":"2024-05-20T18:35:30.395067+0000","last_fullsized":"2024-05-20T18:35:30.395067+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.400834+0000","last_change":"2024-05-20T18:35:30.400834+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.400834+0000","last_undegraded":"2024-05-20T18:35:30.400834+0000","last_fullsized":"2024-05-20T18:35:30.400834+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.396143+0000","last_change":"2024-05-20T18:35:30.396143+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.396143+0000","last_undegraded":"2024-05-20T18:35:30.396143+0000","last_fullsized":"2024-05-20T18:35:30.396143+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.396956+0000","last_change":"2024-05-20T18:35:30.396956+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.396956+0000","last_undegraded":"2024-05-20T18:35:30.396956+0000","last_fullsized":"2024-05-20T18:35:30.396956+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.393816+0000","last_change":"2024-05-20T18:35:30.393816+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.393816+0000","last_undegraded":"2024-05-20T18:35:30.393816+0000","last_fullsized":"2024-05-20T18:35:30.393816+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.394799+0000","last_change":"2024-05-20T18:35:30.394799+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.394799+0000","last_undegraded":"2024-05-20T18:35:30.394799+0000","last_fullsized":"2024-05-20T18:35:30.394799+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.401809+0000","last_change":"2024-05-20T18:35:30.401809+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.401809+0000","last_undegraded":"2024-05-20T18:35:30.401809+0000","last_fullsized":"2024-05-20T18:35:30.401809+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.394202+0000","last_change":"2024-05-20T18:35:30.394202+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.394202+0000","last_undegraded":"2024-05-20T18:35:30.394202+0000","last_fullsized":"2024-05-20T18:35:30.394202+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395453+0000","last_change":"2024-05-20T18:35:30.395453+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395453+0000","last_undegraded":"2024-05-20T18:35:30.395453+0000","last_fullsized":"2024-05-20T18:35:30.395453+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.397481+0000","last_change":"2024-05-20T18:35:30.397481+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.397481+0000","last_undegraded":"2024-05-20T18:35:30.397481+0000","last_fullsized":"2024-05-20T18:35:30.397481+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.391506+0000","last_change":"2024-05-20T18:35:30.391506+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.391506+0000","last_undegraded":"2024-05-20T18:35:30.391506+0000","last_fullsized":"2024-05-20T18:35:30.391506+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.394975+0000","last_change":"2024-05-20T18:35:30.394975+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.394975+0000","last_undegraded":"2024-05-20T18:35:30.394975+0000","last_fullsized":"2024-05-20T18:35:30.394975+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.399091+0000","last_change":"2024-05-20T18:35:30.399091+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.399091+0000","last_undegraded":"2024-05-20T18:35:30.399091+0000","last_fullsized":"2024-05-20T18:35:30.399091+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.398451+0000","last_change":"2024-05-20T18:35:30.398451+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.398451+0000","last_undegraded":"2024-05-20T18:35:30.398451+0000","last_fullsized":"2024-05-20T18:35:30.398451+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.399063+0000","last_change":"2024-05-20T18:35:30.399063+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.399063+0000","last_undegraded":"2024-05-20T18:35:30.399063+0000","last_fullsized":"2024-05-20T18:35:30.399063+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.400625+0000","last_change":"2024-05-20T18:35:30.400625+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.400625+0000","last_undegraded":"2024-05-20T18:35:30.400625+0000","last_fullsized":"2024-05-20T18:35:30.400625+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.394511+0000","last_change":"2024-05-20T18:35:30.394511+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.394511+0000","last_undegraded":"2024-05-20T18:35:30.394511+0000","last_fullsized":"2024-05-20T18:35:30.394511+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.400130+0000","last_change":"2024-05-20T18:35:30.400130+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.400130+0000","last_undegraded":"2024-05-20T18:35:30.400130+0000","last_fullsized":"2024-05-20T18:35:30.400130+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.390388+0000","last_change":"2024-05-20T18:35:30.390388+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.390388+0000","last_undegraded":"2024-05-20T18:35:30.390388+0000","last_fullsized":"2024-05-20T18:35:30.390388+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.401295+0000","last_change":"2024-05-20T18:35:30.401295+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.401295+0000","last_undegraded":"2024-05-20T18:35:30.401295+0000","last_fullsized":"2024-05-20T18:35:30.401295+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395921+0000","last_change":"2024-05-20T18:35:30.395921+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395921+0000","last_undegraded":"2024-05-20T18:35:30.395921+0000","last_fullsized":"2024-05-20T18:35:30.395921+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.400238+0000","last_change":"2024-05-20T18:35:30.400238+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.400238+0000","last_undegraded":"2024-05-20T18:35:30.400238+0000","last_fullsized":"2024-05-20T18:35:30.400238+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.396399+0000","last_change":"2024-05-20T18:35:30.396399+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.396399+0000","last_undegraded":"2024-05-20T18:35:30.396399+0000","last_fullsized":"2024-05-20T18:35:30.396399+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.391428+0000","last_change":"2024-05-20T18:35:30.391428+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.391428+0000","last_undegraded":"2024-05-20T18:35:30.391428+0000","last_fullsized":"2024-05-20T18:35:30.391428+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.391347+0000","last_change":"2024-05-20T18:35:30.391347+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.391347+0000","last_undegraded":"2024-05-20T18:35:30.391347+0000","last_fullsized":"2024-05-20T18:35:30.391347+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.396899+0000","last_change":"2024-05-20T18:35:30.396899+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.396899+0000","last_undegraded":"2024-05-20T18:35:30.396899+0000","last_fullsized":"2024-05-20T18:35:30.396899+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395384+0000","last_change":"2024-05-20T18:35:30.395384+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395384+0000","last_undegraded":"2024-05-20T18:35:30.395384+0000","last_fullsized":"2024-05-20T18:35:30.395384+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.400522+0000","last_change":"2024-05-20T18:35:30.400522+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.400522+0000","last_undegraded":"2024-05-20T18:35:30.400522+0000","last_fullsized":"2024-05-20T18:35:30.400522+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.392369+0000","last_change":"2024-05-20T18:35:30.392369+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.392369+0000","last_undegraded":"2024-05-20T18:35:30.392369+0000","last_fullsized":"2024-05-20T18:35:30.392369+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.394219+0000","last_change":"2024-05-20T18:35:30.394219+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.394219+0000","last_undegraded":"2024-05-20T18:35:30.394219+0000","last_fullsized":"2024-05-20T18:35:30.394219+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.399575+0000","last_change":"2024-05-20T18:35:30.399575+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.399575+0000","last_undegraded":"2024-05-20T18:35:30.399575+0000","last_fullsized":"2024-05-20T18:35:30.399575+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.393568+0000","last_change":"2024-05-20T18:35:30.393568+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.393568+0000","last_undegraded":"2024-05-20T18:35:30.393568+0000","last_fullsized":"2024-05-20T18:35:30.393568+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.410235+0000","last_change":"2024-05-20T18:35:30.407837+0000","last_active":"2024-05-20T18:35:30.410235+0000","last_peered":"2024-05-20T18:35:30.410235+0000","last_clean":"2024-05-20T18:35:30.410235+0000","last_became_active":"2024-05-20T18:35:30.407539+0000","last_became_peered":"2024-05-20T18:35:30.407539+0000","last_unstale":"2024-05-20T18:35:30.410235+0000","last_undegraded":"2024-05-20T18:35:30.410235+0000","last_fullsized":"2024-05-20T18:35:30.410235+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:31:20.340446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395875+0000","last_change":"2024-05-20T18:35:30.395875+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395875+0000","last_undegraded":"2024-05-20T18:35:30.395875+0000","last_fullsized":"2024-05-20T18:35:30.395875+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.409396+0000","last_change":"2024-05-20T18:35:30.406366+0000","last_active":"2024-05-20T18:35:30.409396+0000","last_peered":"2024-05-20T18:35:30.409396+0000","last_clean":"2024-05-20T18:35:30.409396+0000","last_became_active":"2024-05-20T18:35:30.405770+0000","last_became_peered":"2024-05-20T18:35:30.405770+0000","last_unstale":"2024-05-20T18:35:30.409396+0000","last_undegraded":"2024-05-20T18:35:30.409396+0000","last_fullsized":"2024-05-20T18:35:30.409396+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:50:24.603292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.390568+0000","last_change":"2024-05-20T18:35:30.390568+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.390568+0000","last_undegraded":"2024-05-20T18:35:30.390568+0000","last_fullsized":"2024-05-20T18:35:30.390568+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.412922+0000","last_change":"2024-05-20T18:35:30.409253+0000","last_active":"2024-05-20T18:35:30.412922+0000","last_peered":"2024-05-20T18:35:30.412922+0000","last_clean":"2024-05-20T18:35:30.412922+0000","last_became_active":"2024-05-20T18:35:30.408847+0000","last_became_peered":"2024-05-20T18:35:30.408847+0000","last_unstale":"2024-05-20T18:35:30.412922+0000","last_undegraded":"2024-05-20T18:35:30.412922+0000","last_fullsized":"2024-05-20T18:35:30.412922+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:13:02.199994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.396581+0000","last_change":"2024-05-20T18:35:30.396581+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.396581+0000","last_undegraded":"2024-05-20T18:35:30.396581+0000","last_fullsized":"2024-05-20T18:35:30.396581+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.406960+0000","last_change":"2024-05-20T18:35:30.402307+0000","last_active":"2024-05-20T18:35:30.406960+0000","last_peered":"2024-05-20T18:35:30.406960+0000","last_clean":"2024-05-20T18:35:30.406960+0000","last_became_active":"2024-05-20T18:35:30.401952+0000","last_became_peered":"2024-05-20T18:35:30.401952+0000","last_unstale":"2024-05-20T18:35:30.406960+0000","last_undegraded":"2024-05-20T18:35:30.406960+0000","last_fullsized":"2024-05-20T18:35:30.406960+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:47:28.651899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395611+0000","last_change":"2024-05-20T18:35:30.395611+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395611+0000","last_undegraded":"2024-05-20T18:35:30.395611+0000","last_fullsized":"2024-05-20T18:35:30.395611+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.407964+0000","last_change":"2024-05-20T18:35:30.405866+0000","last_active":"2024-05-20T18:35:30.407964+0000","last_peered":"2024-05-20T18:35:30.407964+0000","last_clean":"2024-05-20T18:35:30.407964+0000","last_became_active":"2024-05-20T18:35:30.405503+0000","last_became_peered":"2024-05-20T18:35:30.405503+0000","last_unstale":"2024-05-20T18:35:30.407964+0000","last_undegraded":"2024-05-20T18:35:30.407964+0000","last_fullsized":"2024-05-20T18:35:30.407964+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:16:21.553268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.393963+0000","last_change":"2024-05-20T18:35:30.393963+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.393963+0000","last_undegraded":"2024-05-20T18:35:30.393963+0000","last_fullsized":"2024-05-20T18:35:30.393963+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.414268+0000","last_change":"2024-05-20T18:35:30.412485+0000","last_active":"2024-05-20T18:35:30.414268+0000","last_peered":"2024-05-20T18:35:30.414268+0000","last_clean":"2024-05-20T18:35:30.414268+0000","last_became_active":"2024-05-20T18:35:30.412162+0000","last_became_peered":"2024-05-20T18:35:30.412162+0000","last_unstale":"2024-05-20T18:35:30.414268+0000","last_undegraded":"2024-05-20T18:35:30.414268+0000","last_fullsized":"2024-05-20T18:35:30.414268+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:31:19.619870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.393425+0000","last_change":"2024-05-20T18:35:30.393425+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.393425+0000","last_undegraded":"2024-05-20T18:35:30.393425+0000","last_fullsized":"2024-05-20T18:35:30.393425+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.412482+0000","last_change":"2024-05-20T18:35:30.409374+0000","last_active":"2024-05-20T18:35:30.412482+0000","last_peered":"2024-05-20T18:35:30.412482+0000","last_clean":"2024-05-20T18:35:30.412482+0000","last_became_active":"2024-05-20T18:35:30.408970+0000","last_became_peered":"2024-05-20T18:35:30.408970+0000","last_unstale":"2024-05-20T18:35:30.412482+0000","last_undegraded":"2024-05-20T18:35:30.412482+0000","last_fullsized":"2024-05-20T18:35:30.412482+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:19:27.219675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395456+0000","last_change":"2024-05-20T18:35:30.395456+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395456+0000","last_undegraded":"2024-05-20T18:35:30.395456+0000","last_fullsized":"2024-05-20T18:35:30.395456+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.408240+0000","last_change":"2024-05-20T18:35:30.406064+0000","last_active":"2024-05-20T18:35:30.408240+0000","last_peered":"2024-05-20T18:35:30.408240+0000","last_clean":"2024-05-20T18:35:30.408240+0000","last_became_active":"2024-05-20T18:35:30.405710+0000","last_became_peered":"2024-05-20T18:35:30.405710+0000","last_unstale":"2024-05-20T18:35:30.408240+0000","last_undegraded":"2024-05-20T18:35:30.408240+0000","last_fullsized":"2024-05-20T18:35:30.408240+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:23:16.072920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.394555+0000","last_change":"2024-05-20T18:35:30.394555+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.394555+0000","last_undegraded":"2024-05-20T18:35:30.394555+0000","last_fullsized":"2024-05-20T18:35:30.394555+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.412758+0000","last_change":"2024-05-20T18:35:30.409880+0000","last_active":"2024-05-20T18:35:30.412758+0000","last_peered":"2024-05-20T18:35:30.412758+0000","last_clean":"2024-05-20T18:35:30.412758+0000","last_became_active":"2024-05-20T18:35:30.409517+0000","last_became_peered":"2024-05-20T18:35:30.409517+0000","last_unstale":"2024-05-20T18:35:30.412758+0000","last_undegraded":"2024-05-20T18:35:30.412758+0000","last_fullsized":"2024-05-20T18:35:30.412758+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:33:58.473775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.393400+0000","last_change":"2024-05-20T18:35:30.393400+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.393400+0000","last_undegraded":"2024-05-20T18:35:30.393400+0000","last_fullsized":"2024-05-20T18:35:30.393400+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.408544+0000","last_change":"2024-05-20T18:35:30.403374+0000","last_active":"2024-05-20T18:35:30.408544+0000","last_peered":"2024-05-20T18:35:30.408544+0000","last_clean":"2024-05-20T18:35:30.408544+0000","last_became_active":"2024-05-20T18:35:30.403012+0000","last_became_peered":"2024-05-20T18:35:30.403012+0000","last_unstale":"2024-05-20T18:35:30.408544+0000","last_undegraded":"2024-05-20T18:35:30.408544+0000","last_fullsized":"2024-05-20T18:35:30.408544+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:45:42.189012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.397246+0000","last_change":"2024-05-20T18:35:30.397246+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.397246+0000","last_undegraded":"2024-05-20T18:35:30.397246+0000","last_fullsized":"2024-05-20T18:35:30.397246+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.411337+0000","last_change":"2024-05-20T18:35:30.410506+0000","last_active":"2024-05-20T18:35:30.411337+0000","last_peered":"2024-05-20T18:35:30.411337+0000","last_clean":"2024-05-20T18:35:30.411337+0000","last_became_active":"2024-05-20T18:35:30.410168+0000","last_became_peered":"2024-05-20T18:35:30.410168+0000","last_unstale":"2024-05-20T18:35:30.411337+0000","last_undegraded":"2024-05-20T18:35:30.411337+0000","last_fullsized":"2024-05-20T18:35:30.411337+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:19:34.158678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.394072+0000","last_change":"2024-05-20T18:35:30.394072+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.394072+0000","last_undegraded":"2024-05-20T18:35:30.394072+0000","last_fullsized":"2024-05-20T18:35:30.394072+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.413578+0000","last_change":"2024-05-20T18:35:30.412065+0000","last_active":"2024-05-20T18:35:30.413578+0000","last_peered":"2024-05-20T18:35:30.413578+0000","last_clean":"2024-05-20T18:35:30.413578+0000","last_became_active":"2024-05-20T18:35:30.410006+0000","last_became_peered":"2024-05-20T18:35:30.410006+0000","last_unstale":"2024-05-20T18:35:30.413578+0000","last_undegraded":"2024-05-20T18:35:30.413578+0000","last_fullsized":"2024-05-20T18:35:30.413578+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:47:28.651899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395562+0000","last_change":"2024-05-20T18:35:30.395562+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395562+0000","last_undegraded":"2024-05-20T18:35:30.395562+0000","last_fullsized":"2024-05-20T18:35:30.395562+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.408708+0000","last_change":"2024-05-20T18:35:30.406644+0000","last_active":"2024-05-20T18:35:30.408708+0000","last_peered":"2024-05-20T18:35:30.408708+0000","last_clean":"2024-05-20T18:35:30.408708+0000","last_became_active":"2024-05-20T18:35:30.406155+0000","last_became_peered":"2024-05-20T18:35:30.406155+0000","last_unstale":"2024-05-20T18:35:30.408708+0000","last_undegraded":"2024-05-20T18:35:30.408708+0000","last_fullsized":"2024-05-20T18:35:30.408708+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:24:43.933783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.392885+0000","last_change":"2024-05-20T18:35:30.392885+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.392885+0000","last_undegraded":"2024-05-20T18:35:30.392885+0000","last_fullsized":"2024-05-20T18:35:30.392885+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.408224+0000","last_change":"2024-05-20T18:35:30.404725+0000","last_active":"2024-05-20T18:35:30.408224+0000","last_peered":"2024-05-20T18:35:30.408224+0000","last_clean":"2024-05-20T18:35:30.408224+0000","last_became_active":"2024-05-20T18:35:30.404404+0000","last_became_peered":"2024-05-20T18:35:30.404404+0000","last_unstale":"2024-05-20T18:35:30.408224+0000","last_undegraded":"2024-05-20T18:35:30.408224+0000","last_fullsized":"2024-05-20T18:35:30.408224+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:22:09.796441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.396812+0000","last_change":"2024-05-20T18:35:30.396812+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.396812+0000","last_undegraded":"2024-05-20T18:35:30.396812+0000","last_fullsized":"2024-05-20T18:35:30.396812+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.415211+0000","last_change":"2024-05-20T18:35:30.414408+0000","last_active":"2024-05-20T18:35:30.415211+0000","last_peered":"2024-05-20T18:35:30.415211+0000","last_clean":"2024-05-20T18:35:30.415211+0000","last_became_active":"2024-05-20T18:35:30.414025+0000","last_became_peered":"2024-05-20T18:35:30.414025+0000","last_unstale":"2024-05-20T18:35:30.415211+0000","last_undegraded":"2024-05-20T18:35:30.415211+0000","last_fullsized":"2024-05-20T18:35:30.415211+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:52:17.777292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395135+0000","last_change":"2024-05-20T18:35:30.395135+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395135+0000","last_undegraded":"2024-05-20T18:35:30.395135+0000","last_fullsized":"2024-05-20T18:35:30.395135+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.412446+0000","last_change":"2024-05-20T18:35:30.408547+0000","last_active":"2024-05-20T18:35:30.412446+0000","last_peered":"2024-05-20T18:35:30.412446+0000","last_clean":"2024-05-20T18:35:30.412446+0000","last_became_active":"2024-05-20T18:35:30.408205+0000","last_became_peered":"2024-05-20T18:35:30.408205+0000","last_unstale":"2024-05-20T18:35:30.412446+0000","last_undegraded":"2024-05-20T18:35:30.412446+0000","last_fullsized":"2024-05-20T18:35:30.412446+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:59:20.346125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.397543+0000","last_change":"2024-05-20T18:35:30.397543+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.397543+0000","last_undegraded":"2024-05-20T18:35:30.397543+0000","last_fullsized":"2024-05-20T18:35:30.397543+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.412005+0000","last_change":"2024-05-20T18:35:30.408259+0000","last_active":"2024-05-20T18:35:30.412005+0000","last_peered":"2024-05-20T18:35:30.412005+0000","last_clean":"2024-05-20T18:35:30.412005+0000","last_became_active":"2024-05-20T18:35:30.407911+0000","last_became_peered":"2024-05-20T18:35:30.407911+0000","last_unstale":"2024-05-20T18:35:30.412005+0000","last_undegraded":"2024-05-20T18:35:30.412005+0000","last_fullsized":"2024-05-20T18:35:30.412005+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:30:06.656452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.392761+0000","last_change":"2024-05-20T18:35:30.392761+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.392761+0000","last_undegraded":"2024-05-20T18:35:30.392761+0000","last_fullsized":"2024-05-20T18:35:30.392761+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.413312+0000","last_change":"2024-05-20T18:35:30.412088+0000","last_active":"2024-05-20T18:35:30.413312+0000","last_peered":"2024-05-20T18:35:30.413312+0000","last_clean":"2024-05-20T18:35:30.413312+0000","last_became_active":"2024-05-20T18:35:30.411708+0000","last_became_peered":"2024-05-20T18:35:30.411708+0000","last_unstale":"2024-05-20T18:35:30.413312+0000","last_undegraded":"2024-05-20T18:35:30.413312+0000","last_fullsized":"2024-05-20T18:35:30.413312+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:26:29.213682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.394854+0000","last_change":"2024-05-20T18:35:30.394854+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.394854+0000","last_undegraded":"2024-05-20T18:35:30.394854+0000","last_fullsized":"2024-05-20T18:35:30.394854+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.407629+0000","last_change":"2024-05-20T18:35:30.406195+0000","last_active":"2024-05-20T18:35:30.407629+0000","last_peered":"2024-05-20T18:35:30.407629+0000","last_clean":"2024-05-20T18:35:30.407629+0000","last_became_active":"2024-05-20T18:35:30.405856+0000","last_became_peered":"2024-05-20T18:35:30.405856+0000","last_unstale":"2024-05-20T18:35:30.407629+0000","last_undegraded":"2024-05-20T18:35:30.407629+0000","last_fullsized":"2024-05-20T18:35:30.407629+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:33:58.473775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395697+0000","last_change":"2024-05-20T18:35:30.395697+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395697+0000","last_undegraded":"2024-05-20T18:35:30.395697+0000","last_fullsized":"2024-05-20T18:35:30.395697+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.413399+0000","last_change":"2024-05-20T18:35:30.411351+0000","last_active":"2024-05-20T18:35:30.413399+0000","last_peered":"2024-05-20T18:35:30.413399+0000","last_clean":"2024-05-20T18:35:30.413399+0000","last_became_active":"2024-05-20T18:35:30.410938+0000","last_became_peered":"2024-05-20T18:35:30.410938+0000","last_unstale":"2024-05-20T18:35:30.413399+0000","last_undegraded":"2024-05-20T18:35:30.413399+0000","last_fullsized":"2024-05-20T18:35:30.413399+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:45:26.959959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.396251+0000","last_change":"2024-05-20T18:35:30.396251+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.396251+0000","last_undegraded":"2024-05-20T18:35:30.396251+0000","last_fullsized":"2024-05-20T18:35:30.396251+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.413171+0000","last_change":"2024-05-20T18:35:30.410136+0000","last_active":"2024-05-20T18:35:30.413171+0000","last_peered":"2024-05-20T18:35:30.413171+0000","last_clean":"2024-05-20T18:35:30.413171+0000","last_became_active":"2024-05-20T18:35:30.409778+0000","last_became_peered":"2024-05-20T18:35:30.409778+0000","last_unstale":"2024-05-20T18:35:30.413171+0000","last_undegraded":"2024-05-20T18:35:30.413171+0000","last_fullsized":"2024-05-20T18:35:30.413171+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:37:58.981434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.397965+0000","last_change":"2024-05-20T18:35:30.397965+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.397965+0000","last_undegraded":"2024-05-20T18:35:30.397965+0000","last_fullsized":"2024-05-20T18:35:30.397965+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.411835+0000","last_change":"2024-05-20T18:35:30.407327+0000","last_active":"2024-05-20T18:35:30.411835+0000","last_peered":"2024-05-20T18:35:30.411835+0000","last_clean":"2024-05-20T18:35:30.411835+0000","last_became_active":"2024-05-20T18:35:30.406994+0000","last_became_peered":"2024-05-20T18:35:30.406994+0000","last_unstale":"2024-05-20T18:35:30.411835+0000","last_undegraded":"2024-05-20T18:35:30.411835+0000","last_fullsized":"2024-05-20T18:35:30.411835+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:22:23.532551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.398473+0000","last_change":"2024-05-20T18:35:30.398473+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.398473+0000","last_undegraded":"2024-05-20T18:35:30.398473+0000","last_fullsized":"2024-05-20T18:35:30.398473+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.409293+0000","last_change":"2024-05-20T18:35:30.404056+0000","last_active":"2024-05-20T18:35:30.409293+0000","last_peered":"2024-05-20T18:35:30.409293+0000","last_clean":"2024-05-20T18:35:30.409293+0000","last_became_active":"2024-05-20T18:35:30.403695+0000","last_became_peered":"2024-05-20T18:35:30.403695+0000","last_unstale":"2024-05-20T18:35:30.409293+0000","last_undegraded":"2024-05-20T18:35:30.409293+0000","last_fullsized":"2024-05-20T18:35:30.409293+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:06:34.052366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.399419+0000","last_change":"2024-05-20T18:35:30.399419+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.399419+0000","last_undegraded":"2024-05-20T18:35:30.399419+0000","last_fullsized":"2024-05-20T18:35:30.399419+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.413484+0000","last_change":"2024-05-20T18:35:30.412633+0000","last_active":"2024-05-20T18:35:30.413484+0000","last_peered":"2024-05-20T18:35:30.413484+0000","last_clean":"2024-05-20T18:35:30.413484+0000","last_became_active":"2024-05-20T18:35:30.412307+0000","last_became_peered":"2024-05-20T18:35:30.412307+0000","last_unstale":"2024-05-20T18:35:30.413484+0000","last_undegraded":"2024-05-20T18:35:30.413484+0000","last_fullsized":"2024-05-20T18:35:30.413484+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:59:07.724149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395296+0000","last_change":"2024-05-20T18:35:30.395296+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395296+0000","last_undegraded":"2024-05-20T18:35:30.395296+0000","last_fullsized":"2024-05-20T18:35:30.395296+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.411785+0000","last_change":"2024-05-20T18:35:30.411059+0000","last_active":"2024-05-20T18:35:30.411785+0000","last_peered":"2024-05-20T18:35:30.411785+0000","last_clean":"2024-05-20T18:35:30.411785+0000","last_became_active":"2024-05-20T18:35:30.410688+0000","last_became_peered":"2024-05-20T18:35:30.410688+0000","last_unstale":"2024-05-20T18:35:30.411785+0000","last_undegraded":"2024-05-20T18:35:30.411785+0000","last_fullsized":"2024-05-20T18:35:30.411785+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:41:03.388138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395640+0000","last_change":"2024-05-20T18:35:30.395640+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395640+0000","last_undegraded":"2024-05-20T18:35:30.395640+0000","last_fullsized":"2024-05-20T18:35:30.395640+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.408141+0000","last_change":"2024-05-20T18:35:30.406122+0000","last_active":"2024-05-20T18:35:30.408141+0000","last_peered":"2024-05-20T18:35:30.408141+0000","last_clean":"2024-05-20T18:35:30.408141+0000","last_became_active":"2024-05-20T18:35:30.405784+0000","last_became_peered":"2024-05-20T18:35:30.405784+0000","last_unstale":"2024-05-20T18:35:30.408141+0000","last_undegraded":"2024-05-20T18:35:30.408141+0000","last_fullsized":"2024-05-20T18:35:30.408141+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:24:08.103767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.397137+0000","last_change":"2024-05-20T18:35:30.397137+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.397137+0000","last_undegraded":"2024-05-20T18:35:30.397137+0000","last_fullsized":"2024-05-20T18:35:30.397137+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.404066+0000","last_change":"2024-05-20T18:35:30.400456+0000","last_active":"2024-05-20T18:35:30.404066+0000","last_peered":"2024-05-20T18:35:30.404066+0000","last_clean":"2024-05-20T18:35:30.404066+0000","last_became_active":"2024-05-20T18:35:30.399296+0000","last_became_peered":"2024-05-20T18:35:30.399296+0000","last_unstale":"2024-05-20T18:35:30.404066+0000","last_undegraded":"2024-05-20T18:35:30.404066+0000","last_fullsized":"2024-05-20T18:35:30.404066+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:30:06.656452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.394593+0000","last_change":"2024-05-20T18:35:30.394593+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.394593+0000","last_undegraded":"2024-05-20T18:35:30.394593+0000","last_fullsized":"2024-05-20T18:35:30.394593+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.411083+0000","last_change":"2024-05-20T18:35:30.408084+0000","last_active":"2024-05-20T18:35:30.411083+0000","last_peered":"2024-05-20T18:35:30.411083+0000","last_clean":"2024-05-20T18:35:30.411083+0000","last_became_active":"2024-05-20T18:35:30.407701+0000","last_became_peered":"2024-05-20T18:35:30.407701+0000","last_unstale":"2024-05-20T18:35:30.411083+0000","last_undegraded":"2024-05-20T18:35:30.411083+0000","last_fullsized":"2024-05-20T18:35:30.411083+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:59:53.740452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.399317+0000","last_change":"2024-05-20T18:35:30.399317+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.399317+0000","last_undegraded":"2024-05-20T18:35:30.399317+0000","last_fullsized":"2024-05-20T18:35:30.399317+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.415376+0000","last_change":"2024-05-20T18:35:30.414600+0000","last_active":"2024-05-20T18:35:30.415376+0000","last_peered":"2024-05-20T18:35:30.415376+0000","last_clean":"2024-05-20T18:35:30.415376+0000","last_became_active":"2024-05-20T18:35:30.414218+0000","last_became_peered":"2024-05-20T18:35:30.414218+0000","last_unstale":"2024-05-20T18:35:30.415376+0000","last_undegraded":"2024-05-20T18:35:30.415376+0000","last_fullsized":"2024-05-20T18:35:30.415376+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:58:38.814053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.399261+0000","last_change":"2024-05-20T18:35:30.399261+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.399261+0000","last_undegraded":"2024-05-20T18:35:30.399261+0000","last_fullsized":"2024-05-20T18:35:30.399261+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.408199+0000","last_change":"2024-05-20T18:35:30.406555+0000","last_active":"2024-05-20T18:35:30.408199+0000","last_peered":"2024-05-20T18:35:30.408199+0000","last_clean":"2024-05-20T18:35:30.408199+0000","last_became_active":"2024-05-20T18:35:30.405500+0000","last_became_peered":"2024-05-20T18:35:30.405500+0000","last_unstale":"2024-05-20T18:35:30.408199+0000","last_undegraded":"2024-05-20T18:35:30.408199+0000","last_fullsized":"2024-05-20T18:35:30.408199+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:33:57.462736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.398630+0000","last_change":"2024-05-20T18:35:30.398630+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.398630+0000","last_undegraded":"2024-05-20T18:35:30.398630+0000","last_fullsized":"2024-05-20T18:35:30.398630+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.413623+0000","last_change":"2024-05-20T18:35:30.412896+0000","last_active":"2024-05-20T18:35:30.413623+0000","last_peered":"2024-05-20T18:35:30.413623+0000","last_clean":"2024-05-20T18:35:30.413623+0000","last_became_active":"2024-05-20T18:35:30.412388+0000","last_became_peered":"2024-05-20T18:35:30.412388+0000","last_unstale":"2024-05-20T18:35:30.413623+0000","last_undegraded":"2024-05-20T18:35:30.413623+0000","last_fullsized":"2024-05-20T18:35:30.413623+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:38:14.297654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-05-20T18:35:30.395713+0000","last_change":"2024-05-20T18:35:30.395713+0000","last_active":"2024-05-20T18:35:30.384431+0000","last_peered":"2024-05-20T18:35:30.384431+0000","last_clean":"2024-05-20T18:35:30.384431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-20T18:35:30.395713+0000","last_undegraded":"2024-05-20T18:35:30.395713+0000","last_fullsized":"2024-05-20T18:35:30.395713+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.413668+0000","last_change":"2024-05-20T18:35:30.411370+0000","last_active":"2024-05-20T18:35:30.413668+0000","last_peered":"2024-05-20T18:35:30.413668+0000","last_clean":"2024-05-20T18:35:30.413668+0000","last_became_active":"2024-05-20T18:35:30.411022+0000","last_became_peered":"2024-05-20T18:35:30.411022+0000","last_unstale":"2024-05-20T18:35:30.413668+0000","last_undegraded":"2024-05-20T18:35:30.413668+0000","last_fullsized":"2024-05-20T18:35:30.413668+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:46:23.446345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":322,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-05-20T18:35:30.399749+0000","last_change":"2024-05-20T18:27:03.765694+0000","last_active":"2024-05-20T18:35:30.399749+0000","last_peered":"2024-05-20T18:35:30.399749+0000","last_clean":"2024-05-20T18:35:30.399749+0000","last_became_active":"2024-05-20T18:27:03.764695+0000","last_became_peered":"2024-05-20T18:27:03.764695+0000","last_unstale":"2024-05-20T18:35:30.399749+0000","last_undegraded":"2024-05-20T18:35:30.399749+0000","last_fullsized":"2024-05-20T18:35:30.399749+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_clean_scrub_stamp":"2024-05-20T18:27:02.750853+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:30:04.407487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574954,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32996,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338844,"statfs":{"total":96636764160,"available":96602976256,"internally_reserved":0,"allocated":1859584,"data_stored":1174365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1277952,"data_stored":590664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574952,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1277952,"data_stored":590664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1277952,"data_stored":590664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574954,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1277952,"data_stored":590664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1277952,"data_stored":590664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32424,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339416,"statfs":{"total":96636764160,"available":96603561984,"internally_reserved":0,"allocated":1265664,"data_stored":583997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574953,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33000,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338840,"statfs":{"total":96636764160,"available":96602972160,"internally_reserved":0,"allocated":1859584,"data_stored":1174365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T18:35:36.237 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3f is not active+clean 2024-05-20T18:35:36.237 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.401556+0000', 'last_change': '2024-05-20T18:35:30.401556+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.401556+0000', 'last_undegraded': '2024-05-20T18:35:30.401556+0000', 'last_fullsized': '2024-05-20T18:35:30.401556+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-20T18:35:36.237 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3e is not active+clean 2024-05-20T18:35:36.237 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.395067+0000', 'last_change': '2024-05-20T18:35:30.395067+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.395067+0000', 'last_undegraded': '2024-05-20T18:35:30.395067+0000', 'last_fullsized': '2024-05-20T18:35:30.395067+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-20T18:35:36.237 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3d is not active+clean 2024-05-20T18:35:36.237 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.400834+0000', 'last_change': '2024-05-20T18:35:30.400834+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.400834+0000', 'last_undegraded': '2024-05-20T18:35:30.400834+0000', 'last_fullsized': '2024-05-20T18:35:30.400834+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-20T18:35:36.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3c is not active+clean 2024-05-20T18:35:36.238 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.396143+0000', 'last_change': '2024-05-20T18:35:30.396143+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.396143+0000', 'last_undegraded': '2024-05-20T18:35:30.396143+0000', 'last_fullsized': '2024-05-20T18:35:30.396143+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-20T18:35:36.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3b is not active+clean 2024-05-20T18:35:36.238 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.396956+0000', 'last_change': '2024-05-20T18:35:30.396956+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.396956+0000', 'last_undegraded': '2024-05-20T18:35:30.396956+0000', 'last_fullsized': '2024-05-20T18:35:30.396956+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-20T18:35:36.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3a is not active+clean 2024-05-20T18:35:36.238 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.393816+0000', 'last_change': '2024-05-20T18:35:30.393816+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.393816+0000', 'last_undegraded': '2024-05-20T18:35:30.393816+0000', 'last_fullsized': '2024-05-20T18:35:30.393816+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-20T18:35:36.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.39 is not active+clean 2024-05-20T18:35:36.238 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.394799+0000', 'last_change': '2024-05-20T18:35:30.394799+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.394799+0000', 'last_undegraded': '2024-05-20T18:35:30.394799+0000', 'last_fullsized': '2024-05-20T18:35:30.394799+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-20T18:35:36.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.38 is not active+clean 2024-05-20T18:35:36.238 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.401809+0000', 'last_change': '2024-05-20T18:35:30.401809+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.401809+0000', 'last_undegraded': '2024-05-20T18:35:30.401809+0000', 'last_fullsized': '2024-05-20T18:35:30.401809+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-20T18:35:36.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.37 is not active+clean 2024-05-20T18:35:36.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.394202+0000', 'last_change': '2024-05-20T18:35:30.394202+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.394202+0000', 'last_undegraded': '2024-05-20T18:35:30.394202+0000', 'last_fullsized': '2024-05-20T18:35:30.394202+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-20T18:35:36.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.36 is not active+clean 2024-05-20T18:35:36.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.395453+0000', 'last_change': '2024-05-20T18:35:30.395453+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.395453+0000', 'last_undegraded': '2024-05-20T18:35:30.395453+0000', 'last_fullsized': '2024-05-20T18:35:30.395453+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-20T18:35:36.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.35 is not active+clean 2024-05-20T18:35:36.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.397481+0000', 'last_change': '2024-05-20T18:35:30.397481+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.397481+0000', 'last_undegraded': '2024-05-20T18:35:30.397481+0000', 'last_fullsized': '2024-05-20T18:35:30.397481+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-20T18:35:36.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.34 is not active+clean 2024-05-20T18:35:36.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.391506+0000', 'last_change': '2024-05-20T18:35:30.391506+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.391506+0000', 'last_undegraded': '2024-05-20T18:35:30.391506+0000', 'last_fullsized': '2024-05-20T18:35:30.391506+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-20T18:35:36.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.33 is not active+clean 2024-05-20T18:35:36.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.394975+0000', 'last_change': '2024-05-20T18:35:30.394975+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.394975+0000', 'last_undegraded': '2024-05-20T18:35:30.394975+0000', 'last_fullsized': '2024-05-20T18:35:30.394975+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-20T18:35:36.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.32 is not active+clean 2024-05-20T18:35:36.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.399091+0000', 'last_change': '2024-05-20T18:35:30.399091+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.399091+0000', 'last_undegraded': '2024-05-20T18:35:30.399091+0000', 'last_fullsized': '2024-05-20T18:35:30.399091+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-20T18:35:36.240 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.31 is not active+clean 2024-05-20T18:35:36.240 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.398451+0000', 'last_change': '2024-05-20T18:35:30.398451+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.398451+0000', 'last_undegraded': '2024-05-20T18:35:30.398451+0000', 'last_fullsized': '2024-05-20T18:35:30.398451+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-20T18:35:36.240 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.30 is not active+clean 2024-05-20T18:35:36.240 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.399063+0000', 'last_change': '2024-05-20T18:35:30.399063+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.399063+0000', 'last_undegraded': '2024-05-20T18:35:30.399063+0000', 'last_fullsized': '2024-05-20T18:35:30.399063+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-20T18:35:36.240 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2f is not active+clean 2024-05-20T18:35:36.240 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.400625+0000', 'last_change': '2024-05-20T18:35:30.400625+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.400625+0000', 'last_undegraded': '2024-05-20T18:35:30.400625+0000', 'last_fullsized': '2024-05-20T18:35:30.400625+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-20T18:35:36.240 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2e is not active+clean 2024-05-20T18:35:36.240 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.394511+0000', 'last_change': '2024-05-20T18:35:30.394511+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.394511+0000', 'last_undegraded': '2024-05-20T18:35:30.394511+0000', 'last_fullsized': '2024-05-20T18:35:30.394511+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-20T18:35:36.240 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2d is not active+clean 2024-05-20T18:35:36.240 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.400130+0000', 'last_change': '2024-05-20T18:35:30.400130+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.400130+0000', 'last_undegraded': '2024-05-20T18:35:30.400130+0000', 'last_fullsized': '2024-05-20T18:35:30.400130+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-20T18:35:36.240 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2c is not active+clean 2024-05-20T18:35:36.240 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.390388+0000', 'last_change': '2024-05-20T18:35:30.390388+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.390388+0000', 'last_undegraded': '2024-05-20T18:35:30.390388+0000', 'last_fullsized': '2024-05-20T18:35:30.390388+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-20T18:35:36.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2b is not active+clean 2024-05-20T18:35:36.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.401295+0000', 'last_change': '2024-05-20T18:35:30.401295+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.401295+0000', 'last_undegraded': '2024-05-20T18:35:30.401295+0000', 'last_fullsized': '2024-05-20T18:35:30.401295+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-20T18:35:36.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2a is not active+clean 2024-05-20T18:35:36.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.395921+0000', 'last_change': '2024-05-20T18:35:30.395921+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.395921+0000', 'last_undegraded': '2024-05-20T18:35:30.395921+0000', 'last_fullsized': '2024-05-20T18:35:30.395921+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-20T18:35:36.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.29 is not active+clean 2024-05-20T18:35:36.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.400238+0000', 'last_change': '2024-05-20T18:35:30.400238+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.400238+0000', 'last_undegraded': '2024-05-20T18:35:30.400238+0000', 'last_fullsized': '2024-05-20T18:35:30.400238+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-20T18:35:36.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.28 is not active+clean 2024-05-20T18:35:36.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.396399+0000', 'last_change': '2024-05-20T18:35:30.396399+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.396399+0000', 'last_undegraded': '2024-05-20T18:35:30.396399+0000', 'last_fullsized': '2024-05-20T18:35:30.396399+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-20T18:35:36.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.27 is not active+clean 2024-05-20T18:35:36.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.391428+0000', 'last_change': '2024-05-20T18:35:30.391428+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.391428+0000', 'last_undegraded': '2024-05-20T18:35:30.391428+0000', 'last_fullsized': '2024-05-20T18:35:30.391428+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-20T18:35:36.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.26 is not active+clean 2024-05-20T18:35:36.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.391347+0000', 'last_change': '2024-05-20T18:35:30.391347+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.391347+0000', 'last_undegraded': '2024-05-20T18:35:30.391347+0000', 'last_fullsized': '2024-05-20T18:35:30.391347+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-20T18:35:36.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.25 is not active+clean 2024-05-20T18:35:36.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.396899+0000', 'last_change': '2024-05-20T18:35:30.396899+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.396899+0000', 'last_undegraded': '2024-05-20T18:35:30.396899+0000', 'last_fullsized': '2024-05-20T18:35:30.396899+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-20T18:35:36.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.24 is not active+clean 2024-05-20T18:35:36.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.395384+0000', 'last_change': '2024-05-20T18:35:30.395384+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.395384+0000', 'last_undegraded': '2024-05-20T18:35:30.395384+0000', 'last_fullsized': '2024-05-20T18:35:30.395384+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-20T18:35:36.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.23 is not active+clean 2024-05-20T18:35:36.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.400522+0000', 'last_change': '2024-05-20T18:35:30.400522+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.400522+0000', 'last_undegraded': '2024-05-20T18:35:30.400522+0000', 'last_fullsized': '2024-05-20T18:35:30.400522+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-20T18:35:36.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.22 is not active+clean 2024-05-20T18:35:36.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.392369+0000', 'last_change': '2024-05-20T18:35:30.392369+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.392369+0000', 'last_undegraded': '2024-05-20T18:35:30.392369+0000', 'last_fullsized': '2024-05-20T18:35:30.392369+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-20T18:35:36.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.21 is not active+clean 2024-05-20T18:35:36.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.394219+0000', 'last_change': '2024-05-20T18:35:30.394219+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.394219+0000', 'last_undegraded': '2024-05-20T18:35:30.394219+0000', 'last_fullsized': '2024-05-20T18:35:30.394219+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-20T18:35:36.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.20 is not active+clean 2024-05-20T18:35:36.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.399575+0000', 'last_change': '2024-05-20T18:35:30.399575+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.399575+0000', 'last_undegraded': '2024-05-20T18:35:30.399575+0000', 'last_fullsized': '2024-05-20T18:35:30.399575+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-20T18:35:36.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.0 is not active+clean 2024-05-20T18:35:36.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.393568+0000', 'last_change': '2024-05-20T18:35:30.393568+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.393568+0000', 'last_undegraded': '2024-05-20T18:35:30.393568+0000', 'last_fullsized': '2024-05-20T18:35:30.393568+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-20T18:35:36.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1 is not active+clean 2024-05-20T18:35:36.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.395875+0000', 'last_change': '2024-05-20T18:35:30.395875+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.395875+0000', 'last_undegraded': '2024-05-20T18:35:30.395875+0000', 'last_fullsized': '2024-05-20T18:35:30.395875+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-20T18:35:36.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2 is not active+clean 2024-05-20T18:35:36.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.390568+0000', 'last_change': '2024-05-20T18:35:30.390568+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.390568+0000', 'last_undegraded': '2024-05-20T18:35:30.390568+0000', 'last_fullsized': '2024-05-20T18:35:30.390568+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-20T18:35:36.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3 is not active+clean 2024-05-20T18:35:36.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.396581+0000', 'last_change': '2024-05-20T18:35:30.396581+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.396581+0000', 'last_undegraded': '2024-05-20T18:35:30.396581+0000', 'last_fullsized': '2024-05-20T18:35:30.396581+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-20T18:35:36.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.4 is not active+clean 2024-05-20T18:35:36.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.395611+0000', 'last_change': '2024-05-20T18:35:30.395611+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.395611+0000', 'last_undegraded': '2024-05-20T18:35:30.395611+0000', 'last_fullsized': '2024-05-20T18:35:30.395611+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-20T18:35:36.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.5 is not active+clean 2024-05-20T18:35:36.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.393963+0000', 'last_change': '2024-05-20T18:35:30.393963+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.393963+0000', 'last_undegraded': '2024-05-20T18:35:30.393963+0000', 'last_fullsized': '2024-05-20T18:35:30.393963+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-20T18:35:36.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.6 is not active+clean 2024-05-20T18:35:36.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.393425+0000', 'last_change': '2024-05-20T18:35:30.393425+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.393425+0000', 'last_undegraded': '2024-05-20T18:35:30.393425+0000', 'last_fullsized': '2024-05-20T18:35:30.393425+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-20T18:35:36.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.7 is not active+clean 2024-05-20T18:35:36.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.395456+0000', 'last_change': '2024-05-20T18:35:30.395456+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.395456+0000', 'last_undegraded': '2024-05-20T18:35:30.395456+0000', 'last_fullsized': '2024-05-20T18:35:30.395456+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-20T18:35:36.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.8 is not active+clean 2024-05-20T18:35:36.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.394555+0000', 'last_change': '2024-05-20T18:35:30.394555+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.394555+0000', 'last_undegraded': '2024-05-20T18:35:30.394555+0000', 'last_fullsized': '2024-05-20T18:35:30.394555+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-20T18:35:36.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.9 is not active+clean 2024-05-20T18:35:36.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.393400+0000', 'last_change': '2024-05-20T18:35:30.393400+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.393400+0000', 'last_undegraded': '2024-05-20T18:35:30.393400+0000', 'last_fullsized': '2024-05-20T18:35:30.393400+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-20T18:35:36.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.a is not active+clean 2024-05-20T18:35:36.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.397246+0000', 'last_change': '2024-05-20T18:35:30.397246+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.397246+0000', 'last_undegraded': '2024-05-20T18:35:30.397246+0000', 'last_fullsized': '2024-05-20T18:35:30.397246+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-20T18:35:36.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.b is not active+clean 2024-05-20T18:35:36.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.394072+0000', 'last_change': '2024-05-20T18:35:30.394072+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.394072+0000', 'last_undegraded': '2024-05-20T18:35:30.394072+0000', 'last_fullsized': '2024-05-20T18:35:30.394072+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-20T18:35:36.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.c is not active+clean 2024-05-20T18:35:36.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.395562+0000', 'last_change': '2024-05-20T18:35:30.395562+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.395562+0000', 'last_undegraded': '2024-05-20T18:35:30.395562+0000', 'last_fullsized': '2024-05-20T18:35:30.395562+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-20T18:35:36.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.d is not active+clean 2024-05-20T18:35:36.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.392885+0000', 'last_change': '2024-05-20T18:35:30.392885+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.392885+0000', 'last_undegraded': '2024-05-20T18:35:30.392885+0000', 'last_fullsized': '2024-05-20T18:35:30.392885+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-20T18:35:36.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.e is not active+clean 2024-05-20T18:35:36.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.396812+0000', 'last_change': '2024-05-20T18:35:30.396812+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.396812+0000', 'last_undegraded': '2024-05-20T18:35:30.396812+0000', 'last_fullsized': '2024-05-20T18:35:30.396812+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-20T18:35:36.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.f is not active+clean 2024-05-20T18:35:36.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.395135+0000', 'last_change': '2024-05-20T18:35:30.395135+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.395135+0000', 'last_undegraded': '2024-05-20T18:35:30.395135+0000', 'last_fullsized': '2024-05-20T18:35:30.395135+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-20T18:35:36.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.10 is not active+clean 2024-05-20T18:35:36.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.397543+0000', 'last_change': '2024-05-20T18:35:30.397543+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.397543+0000', 'last_undegraded': '2024-05-20T18:35:30.397543+0000', 'last_fullsized': '2024-05-20T18:35:30.397543+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-20T18:35:36.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.12 is not active+clean 2024-05-20T18:35:36.245 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.392761+0000', 'last_change': '2024-05-20T18:35:30.392761+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.392761+0000', 'last_undegraded': '2024-05-20T18:35:30.392761+0000', 'last_fullsized': '2024-05-20T18:35:30.392761+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-20T18:35:36.245 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.13 is not active+clean 2024-05-20T18:35:36.246 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.394854+0000', 'last_change': '2024-05-20T18:35:30.394854+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.394854+0000', 'last_undegraded': '2024-05-20T18:35:30.394854+0000', 'last_fullsized': '2024-05-20T18:35:30.394854+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-20T18:35:36.246 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.14 is not active+clean 2024-05-20T18:35:36.246 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.395697+0000', 'last_change': '2024-05-20T18:35:30.395697+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.395697+0000', 'last_undegraded': '2024-05-20T18:35:30.395697+0000', 'last_fullsized': '2024-05-20T18:35:30.395697+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-20T18:35:36.246 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.15 is not active+clean 2024-05-20T18:35:36.246 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.396251+0000', 'last_change': '2024-05-20T18:35:30.396251+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.396251+0000', 'last_undegraded': '2024-05-20T18:35:30.396251+0000', 'last_fullsized': '2024-05-20T18:35:30.396251+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-20T18:35:36.246 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.16 is not active+clean 2024-05-20T18:35:36.246 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.397965+0000', 'last_change': '2024-05-20T18:35:30.397965+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.397965+0000', 'last_undegraded': '2024-05-20T18:35:30.397965+0000', 'last_fullsized': '2024-05-20T18:35:30.397965+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-20T18:35:36.246 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.17 is not active+clean 2024-05-20T18:35:36.246 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.398473+0000', 'last_change': '2024-05-20T18:35:30.398473+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.398473+0000', 'last_undegraded': '2024-05-20T18:35:30.398473+0000', 'last_fullsized': '2024-05-20T18:35:30.398473+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-20T18:35:36.246 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.18 is not active+clean 2024-05-20T18:35:36.247 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.399419+0000', 'last_change': '2024-05-20T18:35:30.399419+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.399419+0000', 'last_undegraded': '2024-05-20T18:35:30.399419+0000', 'last_fullsized': '2024-05-20T18:35:30.399419+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-20T18:35:36.247 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.19 is not active+clean 2024-05-20T18:35:36.247 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.395296+0000', 'last_change': '2024-05-20T18:35:30.395296+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.395296+0000', 'last_undegraded': '2024-05-20T18:35:30.395296+0000', 'last_fullsized': '2024-05-20T18:35:30.395296+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-20T18:35:36.247 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1a is not active+clean 2024-05-20T18:35:36.247 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.395640+0000', 'last_change': '2024-05-20T18:35:30.395640+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.395640+0000', 'last_undegraded': '2024-05-20T18:35:30.395640+0000', 'last_fullsized': '2024-05-20T18:35:30.395640+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-20T18:35:36.247 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1b is not active+clean 2024-05-20T18:35:36.247 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.397137+0000', 'last_change': '2024-05-20T18:35:30.397137+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.397137+0000', 'last_undegraded': '2024-05-20T18:35:30.397137+0000', 'last_fullsized': '2024-05-20T18:35:30.397137+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-20T18:35:36.247 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1c is not active+clean 2024-05-20T18:35:36.247 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.394593+0000', 'last_change': '2024-05-20T18:35:30.394593+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.394593+0000', 'last_undegraded': '2024-05-20T18:35:30.394593+0000', 'last_fullsized': '2024-05-20T18:35:30.394593+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-20T18:35:36.247 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1d is not active+clean 2024-05-20T18:35:36.247 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.399317+0000', 'last_change': '2024-05-20T18:35:30.399317+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.399317+0000', 'last_undegraded': '2024-05-20T18:35:30.399317+0000', 'last_fullsized': '2024-05-20T18:35:30.399317+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-20T18:35:36.247 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1e is not active+clean 2024-05-20T18:35:36.248 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.399261+0000', 'last_change': '2024-05-20T18:35:30.399261+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.399261+0000', 'last_undegraded': '2024-05-20T18:35:30.399261+0000', 'last_fullsized': '2024-05-20T18:35:30.399261+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-20T18:35:36.248 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1f is not active+clean 2024-05-20T18:35:36.248 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.398630+0000', 'last_change': '2024-05-20T18:35:30.398630+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.398630+0000', 'last_undegraded': '2024-05-20T18:35:30.398630+0000', 'last_fullsized': '2024-05-20T18:35:30.398630+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-20T18:35:36.248 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.11 is not active+clean 2024-05-20T18:35:36.248 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 103, 'state': 'creating+peering', 'last_fresh': '2024-05-20T18:35:30.395713+0000', 'last_change': '2024-05-20T18:35:30.395713+0000', 'last_active': '2024-05-20T18:35:30.384431+0000', 'last_peered': '2024-05-20T18:35:30.384431+0000', 'last_clean': '2024-05-20T18:35:30.384431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-20T18:35:30.395713+0000', 'last_undegraded': '2024-05-20T18:35:30.395713+0000', 'last_fullsized': '2024-05-20T18:35:30.395713+0000', 'mapping_epoch': 103, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 103, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'last_clean_scrub_stamp': '2024-05-20T18:35:30.384431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-20T18:35:36.248 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-20T18:35:36.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.387+0000 7f46eed36640 1 Processor -- start 2024-05-20T18:35:36.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.387+0000 7f46eed36640 1 -- start start 2024-05-20T18:35:36.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.388+0000 7f46eed36640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f46e80b5e10 0x7f46e80b6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:36.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.388+0000 7f46eed36640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46e80d1740 0x7f46e80d1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:36.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.388+0000 7f46eed36640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46e80b4860 con 0x7f46e815d040 2024-05-20T18:35:36.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.388+0000 7f46eed36640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f46e80b4b40 con 0x7f46e80b5e10 2024-05-20T18:35:36.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.388+0000 7f46eed36640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f46e80b4e20 con 0x7f46e80d1740 2024-05-20T18:35:36.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.389+0000 7f46ecad2640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f46e815d040 0x7f46e80712e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46724/0 (socket says 172.21.15.5:46724) 2024-05-20T18:35:36.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.389+0000 7f46ecad2640 1 -- 172.21.15.5:0/2077832802 learned_addr learned my addr 172.21.15.5:0/2077832802 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:36.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.389+0000 7f46e7fff640 1 --2- 172.21.15.5:0/2077832802 >> v2:172.21.15.63:3300/0 conn(0x7f46e80b5e10 0x7f46e80b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:36.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.389+0000 7f46ed2d3640 1 --2- 172.21.15.5:0/2077832802 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46e80d1740 0x7f46e80d1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:36.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.389+0000 7f46e77fe640 1 -- 172.21.15.5:0/2077832802 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 613398235 0 0) 0x7f46e80b4860 con 0x7f46e815d040 2024-05-20T18:35:36.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.389+0000 7f46e77fe640 1 -- 172.21.15.5:0/2077832802 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46d8003520 con 0x7f46e815d040 2024-05-20T18:35:36.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.389+0000 7f46e77fe640 1 -- 172.21.15.5:0/2077832802 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 513229869 0 0) 0x7f46d8003520 con 0x7f46e815d040 2024-05-20T18:35:36.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.389+0000 7f46e77fe640 1 -- 172.21.15.5:0/2077832802 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46e80d1740 msgr2=0x7f46e80d1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:36.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.389+0000 7f46e77fe640 1 --2- 172.21.15.5:0/2077832802 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46e80d1740 0x7f46e80d1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.389+0000 7f46e77fe640 1 -- 172.21.15.5:0/2077832802 >> v2:172.21.15.63:3300/0 conn(0x7f46e80b5e10 msgr2=0x7f46e80b6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:36.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.389+0000 7f46e77fe640 1 --2- 172.21.15.5:0/2077832802 >> v2:172.21.15.63:3300/0 conn(0x7f46e80b5e10 0x7f46e80b6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.389+0000 7f46e77fe640 1 -- 172.21.15.5:0/2077832802 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e80d1220 con 0x7f46e815d040 2024-05-20T18:35:36.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.389+0000 7f46e77fe640 1 -- 172.21.15.5:0/2077832802 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f46dc001670 con 0x7f46e815d040 2024-05-20T18:35:36.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.389+0000 7f46e77fe640 1 -- 172.21.15.5:0/2077832802 >> v1:172.21.15.5:6789/0 conn(0x7f46e815d040 legacy=0x7f46e80712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:36.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.389+0000 7f46e77fe640 1 --2- 172.21.15.5:0/2077832802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46d80039b0 0x7f46d8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:36.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.389+0000 7f46e77fe640 1 --2- 172.21.15.5:0/2077832802 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46d8004800 0x7f46d8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:36.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.389+0000 7f46e77fe640 1 --2- 172.21.15.5:0/2077832802 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46e80d1740 0x7f46d8005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:36.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.389+0000 7f46e77fe640 1 -- 172.21.15.5:0/2077832802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f46e80b4b40 con 0x7f46d80039b0 2024-05-20T18:35:36.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.389+0000 7f46e77fe640 1 -- 172.21.15.5:0/2077832802 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f46e80b4e20 con 0x7f46d8004800 2024-05-20T18:35:36.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.389+0000 7f46e77fe640 1 -- 172.21.15.5:0/2077832802 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f46d800b9c0 con 0x7f46e80d1740 2024-05-20T18:35:36.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.389+0000 7f46ecad2640 1 --2- 172.21.15.5:0/2077832802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46d80039b0 0x7f46d8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:36.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.390+0000 7f46e7fff640 1 --2- 172.21.15.5:0/2077832802 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46d8004800 0x7f46d8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:36.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.390+0000 7f46ed2d3640 1 --2- 172.21.15.5:0/2077832802 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46e80d1740 0x7f46d8005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:36.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.390+0000 7f46ecad2640 1 -- 172.21.15.5:0/2077832802 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46e80d1740 msgr2=0x7f46d8005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:36.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.390+0000 7f46ecad2640 1 --2- 172.21.15.5:0/2077832802 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46e80d1740 0x7f46d8005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.390+0000 7f46ecad2640 1 -- 172.21.15.5:0/2077832802 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46d8004800 msgr2=0x7f46d8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:36.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.390+0000 7f46ecad2640 1 --2- 172.21.15.5:0/2077832802 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46d8004800 0x7f46d8004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.390+0000 7f46ecad2640 1 -- 172.21.15.5:0/2077832802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d8003520 con 0x7f46d80039b0 2024-05-20T18:35:36.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.390+0000 7f46ecad2640 1 --2- 172.21.15.5:0/2077832802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46d80039b0 0x7f46d8003e20 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f46dc007fd0 tx=0x7f46d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155ef66dc58ddd5f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:36.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.391+0000 7f46e77fe640 1 -- 172.21.15.5:0/2077832802 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc007d50 con 0x7f46d80039b0 2024-05-20T18:35:36.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.391+0000 7f46e77fe640 1 -- 172.21.15.5:0/2077832802 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46dc005ad0 con 0x7f46d80039b0 2024-05-20T18:35:36.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.391+0000 7f46e77fe640 1 -- 172.21.15.5:0/2077832802 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc007070 con 0x7f46d80039b0 2024-05-20T18:35:36.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.391+0000 7f46eed36640 1 -- 172.21.15.5:0/2077832802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46d80039b0 msgr2=0x7f46d8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:36.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.391+0000 7f46eed36640 1 --2- 172.21.15.5:0/2077832802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46d80039b0 0x7f46d8003e20 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f46dc007fd0 tx=0x7f46d8005830 comp rx=0 tx=0).stop 2024-05-20T18:35:36.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.391+0000 7f46ed2d3640 1 -- 172.21.15.5:0/2077832802 reap_dead start 2024-05-20T18:35:36.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.391+0000 7f46eed36640 1 -- 172.21.15.5:0/2077832802 shutdown_connections 2024-05-20T18:35:36.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.391+0000 7f46eed36640 1 -- 172.21.15.5:0/2077832802 >> 172.21.15.5:0/2077832802 conn(0x7f46e80cf480 msgr2=0x7f46e80cf890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:36.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.391+0000 7f46eed36640 1 -- 172.21.15.5:0/2077832802 shutdown_connections 2024-05-20T18:35:36.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.391+0000 7f46eed36640 1 -- 172.21.15.5:0/2077832802 wait complete. 2024-05-20T18:35:36.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.393+0000 7f46eed36640 1 Processor -- start 2024-05-20T18:35:36.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.393+0000 7f46eed36640 1 -- start start 2024-05-20T18:35:36.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.395+0000 7f46eed36640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46d80039b0 0x7f46e81f52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:36.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.395+0000 7f46eed36640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46e81e8550 0x7f46e80cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:36.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.395+0000 7f46eed36640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46e80cb8e0 0x7f46e81e6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:36.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.395+0000 7f46eed36640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f46e80b4600 con 0x7f46e81e8550 2024-05-20T18:35:36.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.395+0000 7f46eed36640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f46e80b4480 con 0x7f46d80039b0 2024-05-20T18:35:36.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.395+0000 7f46eed36640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f46e80d1b50 con 0x7f46e80cb8e0 2024-05-20T18:35:36.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.395+0000 7f46e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46e81e8550 0x7f46e80cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:36.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.395+0000 7f46e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46e81e8550 0x7f46e80cb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49644/0 (socket says 172.21.15.5:49644) 2024-05-20T18:35:36.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.395+0000 7f46e7fff640 1 -- 172.21.15.5:0/620594321 learned_addr learned my addr 172.21.15.5:0/620594321 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:36.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.395+0000 7f46ecad2640 1 --2- 172.21.15.5:0/620594321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46d80039b0 0x7f46e81f52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:36.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.395+0000 7f46ed2d3640 1 --2- 172.21.15.5:0/620594321 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46e80cb8e0 0x7f46e81e6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:36.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.395+0000 7f46e7fff640 1 -- 172.21.15.5:0/620594321 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46e80cb8e0 msgr2=0x7f46e81e6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:36.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.395+0000 7f46e7fff640 1 --2- 172.21.15.5:0/620594321 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46e80cb8e0 0x7f46e81e6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.395+0000 7f46e7fff640 1 -- 172.21.15.5:0/620594321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46d80039b0 msgr2=0x7f46e81f52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:36.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.395+0000 7f46e7fff640 1 --2- 172.21.15.5:0/620594321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46d80039b0 0x7f46e81f52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.395+0000 7f46e7fff640 1 -- 172.21.15.5:0/620594321 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e80cbcf0 con 0x7f46e81e8550 2024-05-20T18:35:36.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.395+0000 7f46e7fff640 1 --2- 172.21.15.5:0/620594321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46e81e8550 0x7f46e80cb3a0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f46d4011910 tx=0x7f46d4011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:36.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.395+0000 7f46e57fa640 1 -- 172.21.15.5:0/620594321 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d400faf0 con 0x7f46e81e8550 2024-05-20T18:35:36.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.395+0000 7f46e57fa640 1 -- 172.21.15.5:0/620594321 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46d400fc90 con 0x7f46e81e8550 2024-05-20T18:35:36.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.396+0000 7f46eed36640 1 -- 172.21.15.5:0/620594321 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46e81f57f0 con 0x7f46e81e8550 2024-05-20T18:35:36.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.396+0000 7f46e57fa640 1 -- 172.21.15.5:0/620594321 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d4014440 con 0x7f46e81e8550 2024-05-20T18:35:36.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.396+0000 7f46eed36640 1 -- 172.21.15.5:0/620594321 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46e81f91a0 con 0x7f46e81e8550 2024-05-20T18:35:36.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.397+0000 7f46eed36640 1 -- 172.21.15.5:0/620594321 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46e80690b0 con 0x7f46e81e8550 2024-05-20T18:35:36.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.397+0000 7f46e57fa640 1 -- 172.21.15.5:0/620594321 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f46d4012040 con 0x7f46e81e8550 2024-05-20T18:35:36.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.397+0000 7f46e57fa640 1 --2- 172.21.15.5:0/620594321 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f46cc06be30 0x7f46cc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:36.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.398+0000 7f46e57fa640 1 -- 172.21.15.5:0/620594321 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7f46d408b590 con 0x7f46e81e8550 2024-05-20T18:35:36.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.398+0000 7f46ecad2640 1 --2- 172.21.15.5:0/620594321 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f46cc06be30 0x7f46cc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:36.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.398+0000 7f46ecad2640 1 --2- 172.21.15.5:0/620594321 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f46cc06be30 0x7f46cc06e340 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f46dc033000 tx=0x7f46dc009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:36.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.402+0000 7f46e57fa640 1 -- 172.21.15.5:0/620594321 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f46e80cbcf0 con 0x7f46e81e8550 2024-05-20T18:35:36.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.655+0000 7f46eed36640 1 -- 172.21.15.5:0/620594321 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f46e80c9570 con 0x7f46e81e8550 2024-05-20T18:35:36.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.656+0000 7f46e57fa640 1 -- 172.21.15.5:0/620594321 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1197 (secure 0 0 0) 0x7f46d4019030 con 0x7f46e81e8550 2024-05-20T18:35:36.657 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:36.657 INFO:teuthology.orchestra.run.smithi005.stdout:{"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":526,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":105,"num_osds":8,"num_up_osds":8,"osd_up_since":1716229620,"num_in_osds":8,"osd_in_since":1716229612,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":266817536,"bytes_avail":772827295744,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":82,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"b","status":"up:active","gid":8343}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-05-20T18:34:56.696011+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-20T18:35:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.657+0000 7f46eed36640 1 -- 172.21.15.5:0/620594321 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f46cc06be30 msgr2=0x7f46cc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.657+0000 7f46eed36640 1 --2- 172.21.15.5:0/620594321 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f46cc06be30 0x7f46cc06e340 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f46dc033000 tx=0x7f46dc009030 comp rx=0 tx=0).stop 2024-05-20T18:35:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.657+0000 7f46eed36640 1 -- 172.21.15.5:0/620594321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46e81e8550 msgr2=0x7f46e80cb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.657+0000 7f46eed36640 1 --2- 172.21.15.5:0/620594321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46e81e8550 0x7f46e80cb3a0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f46d4011910 tx=0x7f46d4011dd0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.658+0000 7f46eed36640 1 -- 172.21.15.5:0/620594321 shutdown_connections 2024-05-20T18:35:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.658+0000 7f46eed36640 1 --2- 172.21.15.5:0/620594321 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f46cc06be30 0x7f46cc06e340 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.658+0000 7f46eed36640 1 --2- 172.21.15.5:0/620594321 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46e80cb8e0 0x7f46e81e6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.658+0000 7f46eed36640 1 --2- 172.21.15.5:0/620594321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46e81e8550 0x7f46e80cb3a0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.658+0000 7f46eed36640 1 --2- 172.21.15.5:0/620594321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46d80039b0 0x7f46e81f52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.658+0000 7f46eed36640 1 -- 172.21.15.5:0/620594321 >> 172.21.15.5:0/620594321 conn(0x7f46e80cf480 msgr2=0x7f46e80710d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.658+0000 7f46eed36640 1 -- 172.21.15.5:0/620594321 shutdown_connections 2024-05-20T18:35:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.658+0000 7f46eed36640 1 -- 172.21.15.5:0/620594321 wait complete. 2024-05-20T18:35:36.670 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-05-20T18:35:36.670 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T18:35:36.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.785+0000 7fe714a28640 1 Processor -- start 2024-05-20T18:35:36.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.785+0000 7fe714a28640 1 -- start start 2024-05-20T18:35:36.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.786+0000 7fe714a28640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe710162820 0x7fe7100b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:36.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.786+0000 7fe714a28640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe7100b3e80 0x7fe71006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:36.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.786+0000 7fe714a28640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe71006d910 0x7fe7100d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:36.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.786+0000 7fe714a28640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe71006dd20 con 0x7fe710162820 2024-05-20T18:35:36.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.786+0000 7fe714a28640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe7100d0630 con 0x7fe7100b3e80 2024-05-20T18:35:36.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.786+0000 7fe714a28640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe7100d39c0 con 0x7fe71006d910 2024-05-20T18:35:36.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.786+0000 7fe70e59c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe710162820 0x7fe7100b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:36.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.786+0000 7fe70e59c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe710162820 0x7fe7100b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49646/0 (socket says 172.21.15.5:49646) 2024-05-20T18:35:36.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.786+0000 7fe70e59c640 1 -- 172.21.15.5:0/694049972 learned_addr learned my addr 172.21.15.5:0/694049972 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:36.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.787+0000 7fe70dd9b640 1 --2- 172.21.15.5:0/694049972 >> v2:172.21.15.63:3300/0 conn(0x7fe7100b3e80 0x7fe71006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:36.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.787+0000 7fe70ed9d640 1 --2- 172.21.15.5:0/694049972 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe71006d910 0x7fe7100d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:36.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.787+0000 7fe70e59c640 1 -- 172.21.15.5:0/694049972 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe71006d910 msgr2=0x7fe7100d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:36.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.787+0000 7fe70e59c640 1 --2- 172.21.15.5:0/694049972 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe71006d910 0x7fe7100d27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.787+0000 7fe70e59c640 1 -- 172.21.15.5:0/694049972 >> v2:172.21.15.63:3300/0 conn(0x7fe7100b3e80 msgr2=0x7fe71006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:36.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.787+0000 7fe70e59c640 1 --2- 172.21.15.5:0/694049972 >> v2:172.21.15.63:3300/0 conn(0x7fe7100b3e80 0x7fe71006d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.787+0000 7fe70e59c640 1 -- 172.21.15.5:0/694049972 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7100d3ca0 con 0x7fe710162820 2024-05-20T18:35:36.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.787+0000 7fe70dd9b640 1 --2- 172.21.15.5:0/694049972 >> v2:172.21.15.63:3300/0 conn(0x7fe7100b3e80 0x7fe71006d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:36.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.788+0000 7fe70e59c640 1 --2- 172.21.15.5:0/694049972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe710162820 0x7fe7100b3940 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fe7100d1230 tx=0x7fe70002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cfe9717e8ed4df server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:36.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.788+0000 7fe70d59a640 1 -- 172.21.15.5:0/694049972 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe700037070 con 0x7fe710162820 2024-05-20T18:35:36.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.788+0000 7fe70d59a640 1 -- 172.21.15.5:0/694049972 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe700032440 con 0x7fe710162820 2024-05-20T18:35:36.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.788+0000 7fe70d59a640 1 -- 172.21.15.5:0/694049972 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe700032810 con 0x7fe710162820 2024-05-20T18:35:36.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.788+0000 7fe714a28640 1 -- 172.21.15.5:0/694049972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe710162820 msgr2=0x7fe7100b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:36.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.788+0000 7fe714a28640 1 --2- 172.21.15.5:0/694049972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe710162820 0x7fe7100b3940 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fe7100d1230 tx=0x7fe70002a1e0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.789+0000 7fe714a28640 1 -- 172.21.15.5:0/694049972 shutdown_connections 2024-05-20T18:35:36.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.789+0000 7fe714a28640 1 --2- 172.21.15.5:0/694049972 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe71006d910 0x7fe7100d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.789+0000 7fe714a28640 1 --2- 172.21.15.5:0/694049972 >> v2:172.21.15.63:3300/0 conn(0x7fe7100b3e80 0x7fe71006d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.789+0000 7fe714a28640 1 --2- 172.21.15.5:0/694049972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe710162820 0x7fe7100b3940 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.789+0000 7fe714a28640 1 -- 172.21.15.5:0/694049972 >> 172.21.15.5:0/694049972 conn(0x7fe7100d4bd0 msgr2=0x7fe7100d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:36.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.789+0000 7fe714a28640 1 -- 172.21.15.5:0/694049972 shutdown_connections 2024-05-20T18:35:36.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.789+0000 7fe714a28640 1 -- 172.21.15.5:0/694049972 wait complete. 2024-05-20T18:35:36.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.791+0000 7fe714a28640 1 Processor -- start 2024-05-20T18:35:36.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.791+0000 7fe714a28640 1 -- start start 2024-05-20T18:35:36.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.792+0000 7fe714a28640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe71006d910 0x7fe710153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:36.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.792+0000 7fe714a28640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7100b3e80 0x7fe7100c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:36.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.792+0000 7fe714a28640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe710162820 0x7fe7100caf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:36.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.792+0000 7fe714a28640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe7100d0300 con 0x7fe7100b3e80 2024-05-20T18:35:36.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.792+0000 7fe714a28640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe7100d4260 con 0x7fe71006d910 2024-05-20T18:35:36.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.792+0000 7fe714a28640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe7100d4780 con 0x7fe710162820 2024-05-20T18:35:36.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.792+0000 7fe70dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7100b3e80 0x7fe7100c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:36.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.792+0000 7fe70dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7100b3e80 0x7fe7100c8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49648/0 (socket says 172.21.15.5:49648) 2024-05-20T18:35:36.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.792+0000 7fe70dd9b640 1 -- 172.21.15.5:0/432295226 learned_addr learned my addr 172.21.15.5:0/432295226 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:36.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.792+0000 7fe70e59c640 1 --2- 172.21.15.5:0/432295226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe71006d910 0x7fe710153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:36.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.792+0000 7fe70ed9d640 1 --2- 172.21.15.5:0/432295226 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe710162820 0x7fe7100caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:36.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.792+0000 7fe70dd9b640 1 -- 172.21.15.5:0/432295226 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe710162820 msgr2=0x7fe7100caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:36.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.792+0000 7fe70dd9b640 1 --2- 172.21.15.5:0/432295226 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe710162820 0x7fe7100caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.792+0000 7fe70dd9b640 1 -- 172.21.15.5:0/432295226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe71006d910 msgr2=0x7fe710153f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:36.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.792+0000 7fe70dd9b640 1 --2- 172.21.15.5:0/432295226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe71006d910 0x7fe710153f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.792+0000 7fe70dd9b640 1 -- 172.21.15.5:0/432295226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7100c94e0 con 0x7fe7100b3e80 2024-05-20T18:35:36.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.792+0000 7fe70ed9d640 1 --2- 172.21.15.5:0/432295226 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe710162820 0x7fe7100caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:36.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.793+0000 7fe70dd9b640 1 --2- 172.21.15.5:0/432295226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7100b3e80 0x7fe7100c8860 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fe6f800d7d0 tx=0x7fe6f800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:36.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.793+0000 7fe706ffd640 1 -- 172.21.15.5:0/432295226 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8004030 con 0x7fe7100b3e80 2024-05-20T18:35:36.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.793+0000 7fe706ffd640 1 -- 172.21.15.5:0/432295226 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6f80041d0 con 0x7fe7100b3e80 2024-05-20T18:35:36.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.793+0000 7fe714a28640 1 -- 172.21.15.5:0/432295226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7100c97c0 con 0x7fe7100b3e80 2024-05-20T18:35:36.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.793+0000 7fe706ffd640 1 -- 172.21.15.5:0/432295226 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f80096a0 con 0x7fe7100b3e80 2024-05-20T18:35:36.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.793+0000 7fe714a28640 1 -- 172.21.15.5:0/432295226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7100c9d80 con 0x7fe7100b3e80 2024-05-20T18:35:36.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.794+0000 7fe714a28640 1 -- 172.21.15.5:0/432295226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe710075b00 con 0x7fe7100b3e80 2024-05-20T18:35:36.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.794+0000 7fe706ffd640 1 -- 172.21.15.5:0/432295226 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe6f8009840 con 0x7fe7100b3e80 2024-05-20T18:35:36.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.795+0000 7fe706ffd640 1 --2- 172.21.15.5:0/432295226 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe6f006be30 0x7fe6f006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:36.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.795+0000 7fe706ffd640 1 -- 172.21.15.5:0/432295226 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7fe6f808b310 con 0x7fe7100b3e80 2024-05-20T18:35:36.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.795+0000 7fe70e59c640 1 --2- 172.21.15.5:0/432295226 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe6f006be30 0x7fe6f006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:36.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.795+0000 7fe70e59c640 1 --2- 172.21.15.5:0/432295226 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe6f006be30 0x7fe6f006e340 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe700034000 tx=0x7fe70002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:36.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.799+0000 7fe706ffd640 1 -- 172.21.15.5:0/432295226 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe6f8059a00 con 0x7fe7100b3e80 2024-05-20T18:35:36.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.978+0000 7fe714a28640 1 -- 172.21.15.5:0/432295226 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe710068810 con 0x7fe6f006be30 2024-05-20T18:35:36.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.985+0000 7fe706ffd640 1 -- 172.21.15.5:0/432295226 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252486 (secure 0 0 0) 0x7fe710068810 con 0x7fe6f006be30 2024-05-20T18:35:36.988 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:36.989 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-20T18:35:36.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.987+0000 7fe714a28640 1 -- 172.21.15.5:0/432295226 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe6f006be30 msgr2=0x7fe6f006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:36.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.987+0000 7fe714a28640 1 --2- 172.21.15.5:0/432295226 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe6f006be30 0x7fe6f006e340 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe700034000 tx=0x7fe70002e000 comp rx=0 tx=0).stop 2024-05-20T18:35:36.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.987+0000 7fe714a28640 1 -- 172.21.15.5:0/432295226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7100b3e80 msgr2=0x7fe7100c8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:36.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.987+0000 7fe714a28640 1 --2- 172.21.15.5:0/432295226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7100b3e80 0x7fe7100c8860 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fe6f800d7d0 tx=0x7fe6f800dc90 comp rx=0 tx=0).stop 2024-05-20T18:35:36.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.988+0000 7fe714a28640 1 -- 172.21.15.5:0/432295226 shutdown_connections 2024-05-20T18:35:36.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.988+0000 7fe714a28640 1 --2- 172.21.15.5:0/432295226 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe6f006be30 0x7fe6f006e340 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.988+0000 7fe714a28640 1 --2- 172.21.15.5:0/432295226 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe710162820 0x7fe7100caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.988+0000 7fe714a28640 1 --2- 172.21.15.5:0/432295226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7100b3e80 0x7fe7100c8860 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.988+0000 7fe714a28640 1 --2- 172.21.15.5:0/432295226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe71006d910 0x7fe710153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:36.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.988+0000 7fe714a28640 1 -- 172.21.15.5:0/432295226 >> 172.21.15.5:0/432295226 conn(0x7fe7100d4bd0 msgr2=0x7fe7100b8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:36.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.988+0000 7fe714a28640 1 -- 172.21.15.5:0/432295226 shutdown_connections 2024-05-20T18:35:36.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:36.988+0000 7fe714a28640 1 -- 172.21.15.5:0/432295226 wait complete. 2024-05-20T18:35:37.000 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2024-05-20T18:35:36.704901+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260876,"kb_used_data":11400,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713844,"statfs":{"total":773094113280,"available":772826976256,"internally_reserved":0,"allocated":11673600,"data_stored":6021740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.317375"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.418005+0000","last_change":"2024-05-20T18:35:31.424894+0000","last_active":"2024-05-20T18:35:32.418005+0000","last_peered":"2024-05-20T18:35:32.418005+0000","last_clean":"2024-05-20T18:35:32.418005+0000","last_became_active":"2024-05-20T18:35:31.424577+0000","last_became_peered":"2024-05-20T18:35:31.424577+0000","last_unstale":"2024-05-20T18:35:32.418005+0000","last_undegraded":"2024-05-20T18:35:32.418005+0000","last_fullsized":"2024-05-20T18:35:32.418005+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:41:32.611483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.421808+0000","last_change":"2024-05-20T18:35:31.421351+0000","last_active":"2024-05-20T18:35:32.421808+0000","last_peered":"2024-05-20T18:35:32.421808+0000","last_clean":"2024-05-20T18:35:32.421808+0000","last_became_active":"2024-05-20T18:35:31.420985+0000","last_became_peered":"2024-05-20T18:35:31.420985+0000","last_unstale":"2024-05-20T18:35:32.421808+0000","last_undegraded":"2024-05-20T18:35:32.421808+0000","last_fullsized":"2024-05-20T18:35:32.421808+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:51:25.489802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.417921+0000","last_change":"2024-05-20T18:35:31.431529+0000","last_active":"2024-05-20T18:35:32.417921+0000","last_peered":"2024-05-20T18:35:32.417921+0000","last_clean":"2024-05-20T18:35:32.417921+0000","last_became_active":"2024-05-20T18:35:31.431235+0000","last_became_peered":"2024-05-20T18:35:31.431235+0000","last_unstale":"2024-05-20T18:35:32.417921+0000","last_undegraded":"2024-05-20T18:35:32.417921+0000","last_fullsized":"2024-05-20T18:35:32.417921+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:57:55.678295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238773+0000","last_change":"2024-05-20T18:35:31.426986+0000","last_active":"2024-05-20T18:35:33.238773+0000","last_peered":"2024-05-20T18:35:33.238773+0000","last_clean":"2024-05-20T18:35:33.238773+0000","last_became_active":"2024-05-20T18:35:31.426700+0000","last_became_peered":"2024-05-20T18:35:31.426700+0000","last_unstale":"2024-05-20T18:35:33.238773+0000","last_undegraded":"2024-05-20T18:35:33.238773+0000","last_fullsized":"2024-05-20T18:35:33.238773+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:29:47.972518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.237987+0000","last_change":"2024-05-20T18:35:31.415028+0000","last_active":"2024-05-20T18:35:33.237987+0000","last_peered":"2024-05-20T18:35:33.237987+0000","last_clean":"2024-05-20T18:35:33.237987+0000","last_became_active":"2024-05-20T18:35:31.414706+0000","last_became_peered":"2024-05-20T18:35:31.414706+0000","last_unstale":"2024-05-20T18:35:33.237987+0000","last_undegraded":"2024-05-20T18:35:33.237987+0000","last_fullsized":"2024-05-20T18:35:33.237987+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:52:36.277484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.420891+0000","last_change":"2024-05-20T18:35:31.427490+0000","last_active":"2024-05-20T18:35:32.420891+0000","last_peered":"2024-05-20T18:35:32.420891+0000","last_clean":"2024-05-20T18:35:32.420891+0000","last_became_active":"2024-05-20T18:35:31.427174+0000","last_became_peered":"2024-05-20T18:35:31.427174+0000","last_unstale":"2024-05-20T18:35:32.420891+0000","last_undegraded":"2024-05-20T18:35:32.420891+0000","last_fullsized":"2024-05-20T18:35:32.420891+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:16:22.557898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.420233+0000","last_change":"2024-05-20T18:35:31.413562+0000","last_active":"2024-05-20T18:35:32.420233+0000","last_peered":"2024-05-20T18:35:32.420233+0000","last_clean":"2024-05-20T18:35:32.420233+0000","last_became_active":"2024-05-20T18:35:31.413252+0000","last_became_peered":"2024-05-20T18:35:31.413252+0000","last_unstale":"2024-05-20T18:35:32.420233+0000","last_undegraded":"2024-05-20T18:35:32.420233+0000","last_fullsized":"2024-05-20T18:35:32.420233+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:35:00.678467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.574807+0000","last_change":"2024-05-20T18:35:31.420510+0000","last_active":"2024-05-20T18:35:32.574807+0000","last_peered":"2024-05-20T18:35:32.574807+0000","last_clean":"2024-05-20T18:35:32.574807+0000","last_became_active":"2024-05-20T18:35:31.420004+0000","last_became_peered":"2024-05-20T18:35:31.420004+0000","last_unstale":"2024-05-20T18:35:32.574807+0000","last_undegraded":"2024-05-20T18:35:32.574807+0000","last_fullsized":"2024-05-20T18:35:32.574807+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:26:18.458203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.420786+0000","last_change":"2024-05-20T18:35:31.422857+0000","last_active":"2024-05-20T18:35:32.420786+0000","last_peered":"2024-05-20T18:35:32.420786+0000","last_clean":"2024-05-20T18:35:32.420786+0000","last_became_active":"2024-05-20T18:35:31.422519+0000","last_became_peered":"2024-05-20T18:35:31.422519+0000","last_unstale":"2024-05-20T18:35:32.420786+0000","last_undegraded":"2024-05-20T18:35:32.420786+0000","last_fullsized":"2024-05-20T18:35:32.420786+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:02:05.636153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238524+0000","last_change":"2024-05-20T18:35:31.428547+0000","last_active":"2024-05-20T18:35:33.238524+0000","last_peered":"2024-05-20T18:35:33.238524+0000","last_clean":"2024-05-20T18:35:33.238524+0000","last_became_active":"2024-05-20T18:35:31.428234+0000","last_became_peered":"2024-05-20T18:35:31.428234+0000","last_unstale":"2024-05-20T18:35:33.238524+0000","last_undegraded":"2024-05-20T18:35:33.238524+0000","last_fullsized":"2024-05-20T18:35:33.238524+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:19:35.163308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.237950+0000","last_change":"2024-05-20T18:35:31.428591+0000","last_active":"2024-05-20T18:35:33.237950+0000","last_peered":"2024-05-20T18:35:33.237950+0000","last_clean":"2024-05-20T18:35:33.237950+0000","last_became_active":"2024-05-20T18:35:31.428289+0000","last_became_peered":"2024-05-20T18:35:31.428289+0000","last_unstale":"2024-05-20T18:35:33.237950+0000","last_undegraded":"2024-05-20T18:35:33.237950+0000","last_fullsized":"2024-05-20T18:35:33.237950+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:51:25.489802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.421357+0000","last_change":"2024-05-20T18:35:31.426873+0000","last_active":"2024-05-20T18:35:32.421357+0000","last_peered":"2024-05-20T18:35:32.421357+0000","last_clean":"2024-05-20T18:35:32.421357+0000","last_became_active":"2024-05-20T18:35:31.426575+0000","last_became_peered":"2024-05-20T18:35:31.426575+0000","last_unstale":"2024-05-20T18:35:32.421357+0000","last_undegraded":"2024-05-20T18:35:32.421357+0000","last_fullsized":"2024-05-20T18:35:32.421357+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:48.866975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238238+0000","last_change":"2024-05-20T18:35:31.428552+0000","last_active":"2024-05-20T18:35:33.238238+0000","last_peered":"2024-05-20T18:35:33.238238+0000","last_clean":"2024-05-20T18:35:33.238238+0000","last_became_active":"2024-05-20T18:35:31.428231+0000","last_became_peered":"2024-05-20T18:35:31.428231+0000","last_unstale":"2024-05-20T18:35:33.238238+0000","last_undegraded":"2024-05-20T18:35:33.238238+0000","last_fullsized":"2024-05-20T18:35:33.238238+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:45:43.193642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.576996+0000","last_change":"2024-05-20T18:35:31.420831+0000","last_active":"2024-05-20T18:35:32.576996+0000","last_peered":"2024-05-20T18:35:32.576996+0000","last_clean":"2024-05-20T18:35:32.576996+0000","last_became_active":"2024-05-20T18:35:31.420499+0000","last_became_peered":"2024-05-20T18:35:31.420499+0000","last_unstale":"2024-05-20T18:35:32.576996+0000","last_undegraded":"2024-05-20T18:35:32.576996+0000","last_fullsized":"2024-05-20T18:35:32.576996+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:22:59.144571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.576952+0000","last_change":"2024-05-20T18:35:31.428171+0000","last_active":"2024-05-20T18:35:32.576952+0000","last_peered":"2024-05-20T18:35:32.576952+0000","last_clean":"2024-05-20T18:35:32.576952+0000","last_became_active":"2024-05-20T18:35:31.427857+0000","last_became_peered":"2024-05-20T18:35:31.427857+0000","last_unstale":"2024-05-20T18:35:32.576952+0000","last_undegraded":"2024-05-20T18:35:32.576952+0000","last_fullsized":"2024-05-20T18:35:32.576952+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:58:19.834727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.576929+0000","last_change":"2024-05-20T18:35:31.419900+0000","last_active":"2024-05-20T18:35:32.576929+0000","last_peered":"2024-05-20T18:35:32.576929+0000","last_clean":"2024-05-20T18:35:32.576929+0000","last_became_active":"2024-05-20T18:35:31.419541+0000","last_became_peered":"2024-05-20T18:35:31.419541+0000","last_unstale":"2024-05-20T18:35:32.576929+0000","last_undegraded":"2024-05-20T18:35:32.576929+0000","last_fullsized":"2024-05-20T18:35:32.576929+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:12:16.222008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.417916+0000","last_change":"2024-05-20T18:35:31.430878+0000","last_active":"2024-05-20T18:35:32.417916+0000","last_peered":"2024-05-20T18:35:32.417916+0000","last_clean":"2024-05-20T18:35:32.417916+0000","last_became_active":"2024-05-20T18:35:31.430531+0000","last_became_peered":"2024-05-20T18:35:31.430531+0000","last_unstale":"2024-05-20T18:35:32.417916+0000","last_undegraded":"2024-05-20T18:35:32.417916+0000","last_fullsized":"2024-05-20T18:35:32.417916+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:55:12.845763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.422037+0000","last_change":"2024-05-20T18:35:31.427821+0000","last_active":"2024-05-20T18:35:32.422037+0000","last_peered":"2024-05-20T18:35:32.422037+0000","last_clean":"2024-05-20T18:35:32.422037+0000","last_became_active":"2024-05-20T18:35:31.427009+0000","last_became_peered":"2024-05-20T18:35:31.427009+0000","last_unstale":"2024-05-20T18:35:32.422037+0000","last_undegraded":"2024-05-20T18:35:32.422037+0000","last_fullsized":"2024-05-20T18:35:32.422037+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:48.866975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.418110+0000","last_change":"2024-05-20T18:35:31.414035+0000","last_active":"2024-05-20T18:35:32.418110+0000","last_peered":"2024-05-20T18:35:32.418110+0000","last_clean":"2024-05-20T18:35:32.418110+0000","last_became_active":"2024-05-20T18:35:31.413520+0000","last_became_peered":"2024-05-20T18:35:31.413520+0000","last_unstale":"2024-05-20T18:35:32.418110+0000","last_undegraded":"2024-05-20T18:35:32.418110+0000","last_fullsized":"2024-05-20T18:35:32.418110+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:22:02.551585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.421574+0000","last_change":"2024-05-20T18:35:31.422500+0000","last_active":"2024-05-20T18:35:32.421574+0000","last_peered":"2024-05-20T18:35:32.421574+0000","last_clean":"2024-05-20T18:35:32.421574+0000","last_became_active":"2024-05-20T18:35:31.422196+0000","last_became_peered":"2024-05-20T18:35:31.422196+0000","last_unstale":"2024-05-20T18:35:32.421574+0000","last_undegraded":"2024-05-20T18:35:32.421574+0000","last_fullsized":"2024-05-20T18:35:32.421574+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:04.213768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.575028+0000","last_change":"2024-05-20T18:35:31.432624+0000","last_active":"2024-05-20T18:35:32.575028+0000","last_peered":"2024-05-20T18:35:32.575028+0000","last_clean":"2024-05-20T18:35:32.575028+0000","last_became_active":"2024-05-20T18:35:31.432296+0000","last_became_peered":"2024-05-20T18:35:31.432296+0000","last_unstale":"2024-05-20T18:35:32.575028+0000","last_undegraded":"2024-05-20T18:35:32.575028+0000","last_fullsized":"2024-05-20T18:35:32.575028+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:19:21.865495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238389+0000","last_change":"2024-05-20T18:35:31.431224+0000","last_active":"2024-05-20T18:35:33.238389+0000","last_peered":"2024-05-20T18:35:33.238389+0000","last_clean":"2024-05-20T18:35:33.238389+0000","last_became_active":"2024-05-20T18:35:31.430907+0000","last_became_peered":"2024-05-20T18:35:31.430907+0000","last_unstale":"2024-05-20T18:35:33.238389+0000","last_undegraded":"2024-05-20T18:35:33.238389+0000","last_fullsized":"2024-05-20T18:35:33.238389+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:16:22.557898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.417577+0000","last_change":"2024-05-20T18:35:31.429270+0000","last_active":"2024-05-20T18:35:32.417577+0000","last_peered":"2024-05-20T18:35:32.417577+0000","last_clean":"2024-05-20T18:35:32.417577+0000","last_became_active":"2024-05-20T18:35:31.428933+0000","last_became_peered":"2024-05-20T18:35:31.428933+0000","last_unstale":"2024-05-20T18:35:32.417577+0000","last_undegraded":"2024-05-20T18:35:32.417577+0000","last_fullsized":"2024-05-20T18:35:32.417577+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:59:21.885118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.237901+0000","last_change":"2024-05-20T18:35:31.421744+0000","last_active":"2024-05-20T18:35:33.237901+0000","last_peered":"2024-05-20T18:35:33.237901+0000","last_clean":"2024-05-20T18:35:33.237901+0000","last_became_active":"2024-05-20T18:35:31.421411+0000","last_became_peered":"2024-05-20T18:35:31.421411+0000","last_unstale":"2024-05-20T18:35:33.237901+0000","last_undegraded":"2024-05-20T18:35:33.237901+0000","last_fullsized":"2024-05-20T18:35:33.237901+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:57:42.416971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.421559+0000","last_change":"2024-05-20T18:35:31.431228+0000","last_active":"2024-05-20T18:35:32.421559+0000","last_peered":"2024-05-20T18:35:32.421559+0000","last_clean":"2024-05-20T18:35:32.421559+0000","last_became_active":"2024-05-20T18:35:31.430609+0000","last_became_peered":"2024-05-20T18:35:31.430609+0000","last_unstale":"2024-05-20T18:35:32.421559+0000","last_undegraded":"2024-05-20T18:35:32.421559+0000","last_fullsized":"2024-05-20T18:35:32.421559+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:46:50.437839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.421069+0000","last_change":"2024-05-20T18:35:31.412446+0000","last_active":"2024-05-20T18:35:32.421069+0000","last_peered":"2024-05-20T18:35:32.421069+0000","last_clean":"2024-05-20T18:35:32.421069+0000","last_became_active":"2024-05-20T18:35:31.412118+0000","last_became_peered":"2024-05-20T18:35:31.412118+0000","last_unstale":"2024-05-20T18:35:32.421069+0000","last_undegraded":"2024-05-20T18:35:32.421069+0000","last_fullsized":"2024-05-20T18:35:32.421069+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:51:25.489802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238280+0000","last_change":"2024-05-20T18:35:31.430873+0000","last_active":"2024-05-20T18:35:33.238280+0000","last_peered":"2024-05-20T18:35:33.238280+0000","last_clean":"2024-05-20T18:35:33.238280+0000","last_became_active":"2024-05-20T18:35:31.430452+0000","last_became_peered":"2024-05-20T18:35:31.430452+0000","last_unstale":"2024-05-20T18:35:33.238280+0000","last_undegraded":"2024-05-20T18:35:33.238280+0000","last_fullsized":"2024-05-20T18:35:33.238280+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:46:50.437839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.421697+0000","last_change":"2024-05-20T18:35:31.428946+0000","last_active":"2024-05-20T18:35:32.421697+0000","last_peered":"2024-05-20T18:35:32.421697+0000","last_clean":"2024-05-20T18:35:32.421697+0000","last_became_active":"2024-05-20T18:35:31.428590+0000","last_became_peered":"2024-05-20T18:35:31.428590+0000","last_unstale":"2024-05-20T18:35:32.421697+0000","last_undegraded":"2024-05-20T18:35:32.421697+0000","last_fullsized":"2024-05-20T18:35:32.421697+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:00:14.038942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.575195+0000","last_change":"2024-05-20T18:35:31.431023+0000","last_active":"2024-05-20T18:35:32.575195+0000","last_peered":"2024-05-20T18:35:32.575195+0000","last_clean":"2024-05-20T18:35:32.575195+0000","last_became_active":"2024-05-20T18:35:31.426870+0000","last_became_peered":"2024-05-20T18:35:31.426870+0000","last_unstale":"2024-05-20T18:35:32.575195+0000","last_undegraded":"2024-05-20T18:35:32.575195+0000","last_fullsized":"2024-05-20T18:35:32.575195+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:07:20.710926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.420637+0000","last_change":"2024-05-20T18:35:31.428685+0000","last_active":"2024-05-20T18:35:32.420637+0000","last_peered":"2024-05-20T18:35:32.420637+0000","last_clean":"2024-05-20T18:35:32.420637+0000","last_became_active":"2024-05-20T18:35:31.428366+0000","last_became_peered":"2024-05-20T18:35:31.428366+0000","last_unstale":"2024-05-20T18:35:32.420637+0000","last_undegraded":"2024-05-20T18:35:32.420637+0000","last_fullsized":"2024-05-20T18:35:32.420637+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:26:30.218312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.420441+0000","last_change":"2024-05-20T18:35:31.412871+0000","last_active":"2024-05-20T18:35:32.420441+0000","last_peered":"2024-05-20T18:35:32.420441+0000","last_clean":"2024-05-20T18:35:32.420441+0000","last_became_active":"2024-05-20T18:35:31.412480+0000","last_became_peered":"2024-05-20T18:35:31.412480+0000","last_unstale":"2024-05-20T18:35:32.420441+0000","last_undegraded":"2024-05-20T18:35:32.420441+0000","last_fullsized":"2024-05-20T18:35:32.420441+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:57:42.416971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.574887+0000","last_change":"2024-05-20T18:35:31.427123+0000","last_active":"2024-05-20T18:35:32.574887+0000","last_peered":"2024-05-20T18:35:32.574887+0000","last_clean":"2024-05-20T18:35:32.574887+0000","last_became_active":"2024-05-20T18:35:31.426800+0000","last_became_peered":"2024-05-20T18:35:31.426800+0000","last_unstale":"2024-05-20T18:35:32.574887+0000","last_undegraded":"2024-05-20T18:35:32.574887+0000","last_fullsized":"2024-05-20T18:35:32.574887+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:24:09.108397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.417954+0000","last_change":"2024-05-20T18:35:31.414346+0000","last_active":"2024-05-20T18:35:32.417954+0000","last_peered":"2024-05-20T18:35:32.417954+0000","last_clean":"2024-05-20T18:35:32.417954+0000","last_became_active":"2024-05-20T18:35:31.412899+0000","last_became_peered":"2024-05-20T18:35:31.412899+0000","last_unstale":"2024-05-20T18:35:32.417954+0000","last_undegraded":"2024-05-20T18:35:32.417954+0000","last_fullsized":"2024-05-20T18:35:32.417954+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:44:07.390703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"104'4","reported_seq":18,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238671+0000","last_change":"2024-05-20T18:35:30.407837+0000","last_active":"2024-05-20T18:35:33.238671+0000","last_peered":"2024-05-20T18:35:33.238671+0000","last_clean":"2024-05-20T18:35:33.238671+0000","last_became_active":"2024-05-20T18:35:30.407539+0000","last_became_peered":"2024-05-20T18:35:30.407539+0000","last_unstale":"2024-05-20T18:35:33.238671+0000","last_undegraded":"2024-05-20T18:35:33.238671+0000","last_fullsized":"2024-05-20T18:35:33.238671+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:17:55.050005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.574852+0000","last_change":"2024-05-20T18:35:31.417545+0000","last_active":"2024-05-20T18:35:32.574852+0000","last_peered":"2024-05-20T18:35:32.574852+0000","last_clean":"2024-05-20T18:35:32.574852+0000","last_became_active":"2024-05-20T18:35:31.417145+0000","last_became_peered":"2024-05-20T18:35:31.417145+0000","last_unstale":"2024-05-20T18:35:32.574852+0000","last_undegraded":"2024-05-20T18:35:32.574852+0000","last_fullsized":"2024-05-20T18:35:32.574852+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:44:59.743100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.575366+0000","last_change":"2024-05-20T18:35:30.406366+0000","last_active":"2024-05-20T18:35:32.575366+0000","last_peered":"2024-05-20T18:35:32.575366+0000","last_clean":"2024-05-20T18:35:32.575366+0000","last_became_active":"2024-05-20T18:35:30.405770+0000","last_became_peered":"2024-05-20T18:35:30.405770+0000","last_unstale":"2024-05-20T18:35:32.575366+0000","last_undegraded":"2024-05-20T18:35:32.575366+0000","last_fullsized":"2024-05-20T18:35:32.575366+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:04:30.447517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.420834+0000","last_change":"2024-05-20T18:35:31.424665+0000","last_active":"2024-05-20T18:35:32.420834+0000","last_peered":"2024-05-20T18:35:32.420834+0000","last_clean":"2024-05-20T18:35:32.420834+0000","last_became_active":"2024-05-20T18:35:31.424324+0000","last_became_peered":"2024-05-20T18:35:31.424324+0000","last_unstale":"2024-05-20T18:35:32.420834+0000","last_undegraded":"2024-05-20T18:35:32.420834+0000","last_fullsized":"2024-05-20T18:35:32.420834+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:00:14.038942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.421055+0000","last_change":"2024-05-20T18:35:30.409253+0000","last_active":"2024-05-20T18:35:32.421055+0000","last_peered":"2024-05-20T18:35:32.421055+0000","last_clean":"2024-05-20T18:35:32.421055+0000","last_became_active":"2024-05-20T18:35:30.408847+0000","last_became_peered":"2024-05-20T18:35:30.408847+0000","last_unstale":"2024-05-20T18:35:32.421055+0000","last_undegraded":"2024-05-20T18:35:32.421055+0000","last_fullsized":"2024-05-20T18:35:32.421055+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:28:15.030130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.237962+0000","last_change":"2024-05-20T18:35:31.436546+0000","last_active":"2024-05-20T18:35:33.237962+0000","last_peered":"2024-05-20T18:35:33.237962+0000","last_clean":"2024-05-20T18:35:33.237962+0000","last_became_active":"2024-05-20T18:35:31.436258+0000","last_became_peered":"2024-05-20T18:35:31.436258+0000","last_unstale":"2024-05-20T18:35:33.237962+0000","last_undegraded":"2024-05-20T18:35:33.237962+0000","last_fullsized":"2024-05-20T18:35:33.237962+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:00:14.038942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238141+0000","last_change":"2024-05-20T18:35:30.402307+0000","last_active":"2024-05-20T18:35:33.238141+0000","last_peered":"2024-05-20T18:35:33.238141+0000","last_clean":"2024-05-20T18:35:33.238141+0000","last_became_active":"2024-05-20T18:35:30.401952+0000","last_became_peered":"2024-05-20T18:35:30.401952+0000","last_unstale":"2024-05-20T18:35:33.238141+0000","last_undegraded":"2024-05-20T18:35:33.238141+0000","last_fullsized":"2024-05-20T18:35:33.238141+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:13:02.199994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.239027+0000","last_change":"2024-05-20T18:35:31.420523+0000","last_active":"2024-05-20T18:35:33.239027+0000","last_peered":"2024-05-20T18:35:33.239027+0000","last_clean":"2024-05-20T18:35:33.239027+0000","last_became_active":"2024-05-20T18:35:31.420237+0000","last_became_peered":"2024-05-20T18:35:31.420237+0000","last_unstale":"2024-05-20T18:35:33.239027+0000","last_undegraded":"2024-05-20T18:35:33.239027+0000","last_fullsized":"2024-05-20T18:35:33.239027+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:32:04.226629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.577120+0000","last_change":"2024-05-20T18:35:30.405866+0000","last_active":"2024-05-20T18:35:32.577120+0000","last_peered":"2024-05-20T18:35:32.577120+0000","last_clean":"2024-05-20T18:35:32.577120+0000","last_became_active":"2024-05-20T18:35:30.405503+0000","last_became_peered":"2024-05-20T18:35:30.405503+0000","last_unstale":"2024-05-20T18:35:32.577120+0000","last_undegraded":"2024-05-20T18:35:32.577120+0000","last_fullsized":"2024-05-20T18:35:32.577120+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:00:13.034312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.418334+0000","last_change":"2024-05-20T18:35:31.413854+0000","last_active":"2024-05-20T18:35:32.418334+0000","last_peered":"2024-05-20T18:35:32.418334+0000","last_clean":"2024-05-20T18:35:32.418334+0000","last_became_active":"2024-05-20T18:35:31.413104+0000","last_became_peered":"2024-05-20T18:35:31.413104+0000","last_unstale":"2024-05-20T18:35:32.418334+0000","last_undegraded":"2024-05-20T18:35:32.418334+0000","last_fullsized":"2024-05-20T18:35:32.418334+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:46:10.206099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.237663+0000","last_change":"2024-05-20T18:35:30.412485+0000","last_active":"2024-05-20T18:35:33.237663+0000","last_peered":"2024-05-20T18:35:33.237663+0000","last_clean":"2024-05-20T18:35:33.237663+0000","last_became_active":"2024-05-20T18:35:30.412162+0000","last_became_peered":"2024-05-20T18:35:30.412162+0000","last_unstale":"2024-05-20T18:35:33.237663+0000","last_undegraded":"2024-05-20T18:35:33.237663+0000","last_fullsized":"2024-05-20T18:35:33.237663+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:28:15.030130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.422066+0000","last_change":"2024-05-20T18:35:31.429341+0000","last_active":"2024-05-20T18:35:32.422066+0000","last_peered":"2024-05-20T18:35:32.422066+0000","last_clean":"2024-05-20T18:35:32.422066+0000","last_became_active":"2024-05-20T18:35:31.428508+0000","last_became_peered":"2024-05-20T18:35:31.428508+0000","last_unstale":"2024-05-20T18:35:32.422066+0000","last_undegraded":"2024-05-20T18:35:32.422066+0000","last_fullsized":"2024-05-20T18:35:32.422066+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:46:50.437839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.420309+0000","last_change":"2024-05-20T18:35:30.409374+0000","last_active":"2024-05-20T18:35:32.420309+0000","last_peered":"2024-05-20T18:35:32.420309+0000","last_clean":"2024-05-20T18:35:32.420309+0000","last_became_active":"2024-05-20T18:35:30.408970+0000","last_became_peered":"2024-05-20T18:35:30.408970+0000","last_unstale":"2024-05-20T18:35:32.420309+0000","last_undegraded":"2024-05-20T18:35:32.420309+0000","last_fullsized":"2024-05-20T18:35:32.420309+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:08:29.264517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238790+0000","last_change":"2024-05-20T18:35:31.416735+0000","last_active":"2024-05-20T18:35:33.238790+0000","last_peered":"2024-05-20T18:35:33.238790+0000","last_clean":"2024-05-20T18:35:33.238790+0000","last_became_active":"2024-05-20T18:35:31.416413+0000","last_became_peered":"2024-05-20T18:35:31.416413+0000","last_unstale":"2024-05-20T18:35:33.238790+0000","last_undegraded":"2024-05-20T18:35:33.238790+0000","last_fullsized":"2024-05-20T18:35:33.238790+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:41:04.392769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238378+0000","last_change":"2024-05-20T18:35:30.406064+0000","last_active":"2024-05-20T18:35:33.238378+0000","last_peered":"2024-05-20T18:35:33.238378+0000","last_clean":"2024-05-20T18:35:33.238378+0000","last_became_active":"2024-05-20T18:35:30.405710+0000","last_became_peered":"2024-05-20T18:35:30.405710+0000","last_unstale":"2024-05-20T18:35:33.238378+0000","last_undegraded":"2024-05-20T18:35:33.238378+0000","last_fullsized":"2024-05-20T18:35:33.238378+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:19:06.313009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.417559+0000","last_change":"2024-05-20T18:35:31.414458+0000","last_active":"2024-05-20T18:35:32.417559+0000","last_peered":"2024-05-20T18:35:32.417559+0000","last_clean":"2024-05-20T18:35:32.417559+0000","last_became_active":"2024-05-20T18:35:31.413331+0000","last_became_peered":"2024-05-20T18:35:31.413331+0000","last_unstale":"2024-05-20T18:35:32.417559+0000","last_undegraded":"2024-05-20T18:35:32.417559+0000","last_fullsized":"2024-05-20T18:35:32.417559+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:37:38.515946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.421697+0000","last_change":"2024-05-20T18:35:30.409880+0000","last_active":"2024-05-20T18:35:32.421697+0000","last_peered":"2024-05-20T18:35:32.421697+0000","last_clean":"2024-05-20T18:35:32.421697+0000","last_became_active":"2024-05-20T18:35:30.409517+0000","last_became_peered":"2024-05-20T18:35:30.409517+0000","last_unstale":"2024-05-20T18:35:32.421697+0000","last_undegraded":"2024-05-20T18:35:32.421697+0000","last_fullsized":"2024-05-20T18:35:32.421697+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:14:56.573180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.237994+0000","last_change":"2024-05-20T18:35:31.415635+0000","last_active":"2024-05-20T18:35:33.237994+0000","last_peered":"2024-05-20T18:35:33.237994+0000","last_clean":"2024-05-20T18:35:33.237994+0000","last_became_active":"2024-05-20T18:35:31.415324+0000","last_became_peered":"2024-05-20T18:35:31.415324+0000","last_unstale":"2024-05-20T18:35:33.237994+0000","last_undegraded":"2024-05-20T18:35:33.237994+0000","last_fullsized":"2024-05-20T18:35:33.237994+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:06:54.343178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.417642+0000","last_change":"2024-05-20T18:35:30.403374+0000","last_active":"2024-05-20T18:35:32.417642+0000","last_peered":"2024-05-20T18:35:32.417642+0000","last_clean":"2024-05-20T18:35:32.417642+0000","last_became_active":"2024-05-20T18:35:30.403012+0000","last_became_peered":"2024-05-20T18:35:30.403012+0000","last_unstale":"2024-05-20T18:35:32.417642+0000","last_undegraded":"2024-05-20T18:35:32.417642+0000","last_fullsized":"2024-05-20T18:35:32.417642+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:29:46.967888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.574928+0000","last_change":"2024-05-20T18:35:31.424248+0000","last_active":"2024-05-20T18:35:32.574928+0000","last_peered":"2024-05-20T18:35:32.574928+0000","last_clean":"2024-05-20T18:35:32.574928+0000","last_became_active":"2024-05-20T18:35:31.423928+0000","last_became_peered":"2024-05-20T18:35:31.423928+0000","last_unstale":"2024-05-20T18:35:32.574928+0000","last_undegraded":"2024-05-20T18:35:32.574928+0000","last_fullsized":"2024-05-20T18:35:32.574928+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:16:52.263863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"104'3","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.577117+0000","last_change":"2024-05-20T18:35:30.410506+0000","last_active":"2024-05-20T18:35:32.577117+0000","last_peered":"2024-05-20T18:35:32.577117+0000","last_clean":"2024-05-20T18:35:32.577117+0000","last_became_active":"2024-05-20T18:35:30.410168+0000","last_became_peered":"2024-05-20T18:35:30.410168+0000","last_unstale":"2024-05-20T18:35:32.577117+0000","last_undegraded":"2024-05-20T18:35:32.577117+0000","last_fullsized":"2024-05-20T18:35:32.577117+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:51:24.485172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238307+0000","last_change":"2024-05-20T18:35:31.416784+0000","last_active":"2024-05-20T18:35:33.238307+0000","last_peered":"2024-05-20T18:35:33.238307+0000","last_clean":"2024-05-20T18:35:33.238307+0000","last_became_active":"2024-05-20T18:35:31.416462+0000","last_became_peered":"2024-05-20T18:35:31.416462+0000","last_unstale":"2024-05-20T18:35:33.238307+0000","last_undegraded":"2024-05-20T18:35:33.238307+0000","last_fullsized":"2024-05-20T18:35:33.238307+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:02:05.636153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"104'6","reported_seq":20,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.421928+0000","last_change":"2024-05-20T18:35:30.412065+0000","last_active":"2024-05-20T18:35:32.421928+0000","last_peered":"2024-05-20T18:35:32.421928+0000","last_clean":"2024-05-20T18:35:32.421928+0000","last_became_active":"2024-05-20T18:35:30.410006+0000","last_became_peered":"2024-05-20T18:35:30.410006+0000","last_unstale":"2024-05-20T18:35:32.421928+0000","last_undegraded":"2024-05-20T18:35:32.421928+0000","last_fullsized":"2024-05-20T18:35:32.421928+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:08:57.415190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.237822+0000","last_change":"2024-05-20T18:35:31.435493+0000","last_active":"2024-05-20T18:35:33.237822+0000","last_peered":"2024-05-20T18:35:33.237822+0000","last_clean":"2024-05-20T18:35:33.237822+0000","last_became_active":"2024-05-20T18:35:31.435145+0000","last_became_peered":"2024-05-20T18:35:31.435145+0000","last_unstale":"2024-05-20T18:35:33.237822+0000","last_undegraded":"2024-05-20T18:35:33.237822+0000","last_fullsized":"2024-05-20T18:35:33.237822+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:19:35.163308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238285+0000","last_change":"2024-05-20T18:35:30.406644+0000","last_active":"2024-05-20T18:35:33.238285+0000","last_peered":"2024-05-20T18:35:33.238285+0000","last_clean":"2024-05-20T18:35:33.238285+0000","last_became_active":"2024-05-20T18:35:30.406155+0000","last_became_peered":"2024-05-20T18:35:30.406155+0000","last_unstale":"2024-05-20T18:35:33.238285+0000","last_undegraded":"2024-05-20T18:35:33.238285+0000","last_fullsized":"2024-05-20T18:35:33.238285+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:59:20.346125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.421348+0000","last_change":"2024-05-20T18:35:31.423927+0000","last_active":"2024-05-20T18:35:32.421348+0000","last_peered":"2024-05-20T18:35:32.421348+0000","last_clean":"2024-05-20T18:35:32.421348+0000","last_became_active":"2024-05-20T18:35:31.423640+0000","last_became_peered":"2024-05-20T18:35:31.423640+0000","last_unstale":"2024-05-20T18:35:32.421348+0000","last_undegraded":"2024-05-20T18:35:32.421348+0000","last_fullsized":"2024-05-20T18:35:32.421348+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:26:30.218312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238942+0000","last_change":"2024-05-20T18:35:30.404725+0000","last_active":"2024-05-20T18:35:33.238942+0000","last_peered":"2024-05-20T18:35:33.238942+0000","last_clean":"2024-05-20T18:35:33.238942+0000","last_became_active":"2024-05-20T18:35:30.404404+0000","last_became_peered":"2024-05-20T18:35:30.404404+0000","last_unstale":"2024-05-20T18:35:33.238942+0000","last_undegraded":"2024-05-20T18:35:33.238942+0000","last_fullsized":"2024-05-20T18:35:33.238942+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:20:38.890394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.575110+0000","last_change":"2024-05-20T18:35:31.426565+0000","last_active":"2024-05-20T18:35:32.575110+0000","last_peered":"2024-05-20T18:35:32.575110+0000","last_clean":"2024-05-20T18:35:32.575110+0000","last_became_active":"2024-05-20T18:35:31.426223+0000","last_became_peered":"2024-05-20T18:35:31.426223+0000","last_unstale":"2024-05-20T18:35:32.575110+0000","last_undegraded":"2024-05-20T18:35:32.575110+0000","last_fullsized":"2024-05-20T18:35:32.575110+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:33:58.467366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.574851+0000","last_change":"2024-05-20T18:35:30.414408+0000","last_active":"2024-05-20T18:35:32.574851+0000","last_peered":"2024-05-20T18:35:32.574851+0000","last_clean":"2024-05-20T18:35:32.574851+0000","last_became_active":"2024-05-20T18:35:30.414025+0000","last_became_peered":"2024-05-20T18:35:30.414025+0000","last_unstale":"2024-05-20T18:35:32.574851+0000","last_undegraded":"2024-05-20T18:35:32.574851+0000","last_fullsized":"2024-05-20T18:35:32.574851+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:35:47.133654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.418171+0000","last_change":"2024-05-20T18:35:31.430837+0000","last_active":"2024-05-20T18:35:32.418171+0000","last_peered":"2024-05-20T18:35:32.418171+0000","last_clean":"2024-05-20T18:35:32.418171+0000","last_became_active":"2024-05-20T18:35:31.430520+0000","last_became_peered":"2024-05-20T18:35:31.430520+0000","last_unstale":"2024-05-20T18:35:32.418171+0000","last_undegraded":"2024-05-20T18:35:32.418171+0000","last_fullsized":"2024-05-20T18:35:32.418171+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:58:09.429427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"104'8","reported_seq":22,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.420126+0000","last_change":"2024-05-20T18:35:30.408547+0000","last_active":"2024-05-20T18:35:32.420126+0000","last_peered":"2024-05-20T18:35:32.420126+0000","last_clean":"2024-05-20T18:35:32.420126+0000","last_became_active":"2024-05-20T18:35:30.408205+0000","last_became_peered":"2024-05-20T18:35:30.408205+0000","last_unstale":"2024-05-20T18:35:32.420126+0000","last_undegraded":"2024-05-20T18:35:32.420126+0000","last_fullsized":"2024-05-20T18:35:32.420126+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:31:19.619870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.575221+0000","last_change":"2024-05-20T18:35:31.416565+0000","last_active":"2024-05-20T18:35:32.575221+0000","last_peered":"2024-05-20T18:35:32.575221+0000","last_clean":"2024-05-20T18:35:32.575221+0000","last_became_active":"2024-05-20T18:35:31.416255+0000","last_became_peered":"2024-05-20T18:35:31.416255+0000","last_unstale":"2024-05-20T18:35:32.575221+0000","last_undegraded":"2024-05-20T18:35:32.575221+0000","last_fullsized":"2024-05-20T18:35:32.575221+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:10:49.033115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.421415+0000","last_change":"2024-05-20T18:35:30.408259+0000","last_active":"2024-05-20T18:35:32.421415+0000","last_peered":"2024-05-20T18:35:32.421415+0000","last_clean":"2024-05-20T18:35:32.421415+0000","last_became_active":"2024-05-20T18:35:30.407911+0000","last_became_peered":"2024-05-20T18:35:30.407911+0000","last_unstale":"2024-05-20T18:35:32.421415+0000","last_undegraded":"2024-05-20T18:35:32.421415+0000","last_fullsized":"2024-05-20T18:35:32.421415+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:59:07.724149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.421174+0000","last_change":"2024-05-20T18:35:31.415197+0000","last_active":"2024-05-20T18:35:32.421174+0000","last_peered":"2024-05-20T18:35:32.421174+0000","last_clean":"2024-05-20T18:35:32.421174+0000","last_became_active":"2024-05-20T18:35:31.414725+0000","last_became_peered":"2024-05-20T18:35:31.414725+0000","last_unstale":"2024-05-20T18:35:32.421174+0000","last_undegraded":"2024-05-20T18:35:32.421174+0000","last_fullsized":"2024-05-20T18:35:32.421174+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:19:35.163308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.577260+0000","last_change":"2024-05-20T18:35:30.412088+0000","last_active":"2024-05-20T18:35:32.577260+0000","last_peered":"2024-05-20T18:35:32.577260+0000","last_clean":"2024-05-20T18:35:32.577260+0000","last_became_active":"2024-05-20T18:35:30.411708+0000","last_became_peered":"2024-05-20T18:35:30.411708+0000","last_unstale":"2024-05-20T18:35:32.577260+0000","last_undegraded":"2024-05-20T18:35:32.577260+0000","last_fullsized":"2024-05-20T18:35:32.577260+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:47.862345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238110+0000","last_change":"2024-05-20T18:35:31.420523+0000","last_active":"2024-05-20T18:35:33.238110+0000","last_peered":"2024-05-20T18:35:33.238110+0000","last_clean":"2024-05-20T18:35:33.238110+0000","last_became_active":"2024-05-20T18:35:31.420169+0000","last_became_peered":"2024-05-20T18:35:31.420169+0000","last_unstale":"2024-05-20T18:35:33.238110+0000","last_undegraded":"2024-05-20T18:35:33.238110+0000","last_fullsized":"2024-05-20T18:35:33.238110+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:35:00.678467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.237979+0000","last_change":"2024-05-20T18:35:30.406195+0000","last_active":"2024-05-20T18:35:33.237979+0000","last_peered":"2024-05-20T18:35:33.237979+0000","last_clean":"2024-05-20T18:35:33.237979+0000","last_became_active":"2024-05-20T18:35:30.405856+0000","last_became_peered":"2024-05-20T18:35:30.405856+0000","last_unstale":"2024-05-20T18:35:33.237979+0000","last_undegraded":"2024-05-20T18:35:33.237979+0000","last_fullsized":"2024-05-20T18:35:33.237979+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:08:29.264517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.418073+0000","last_change":"2024-05-20T18:35:31.425634+0000","last_active":"2024-05-20T18:35:32.418073+0000","last_peered":"2024-05-20T18:35:32.418073+0000","last_clean":"2024-05-20T18:35:32.418073+0000","last_became_active":"2024-05-20T18:35:31.425326+0000","last_became_peered":"2024-05-20T18:35:31.425326+0000","last_unstale":"2024-05-20T18:35:32.418073+0000","last_undegraded":"2024-05-20T18:35:32.418073+0000","last_fullsized":"2024-05-20T18:35:32.418073+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:59:31.379489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.575311+0000","last_change":"2024-05-20T18:35:30.411351+0000","last_active":"2024-05-20T18:35:32.575311+0000","last_peered":"2024-05-20T18:35:32.575311+0000","last_clean":"2024-05-20T18:35:32.575311+0000","last_became_active":"2024-05-20T18:35:30.410938+0000","last_became_peered":"2024-05-20T18:35:30.410938+0000","last_unstale":"2024-05-20T18:35:32.575311+0000","last_undegraded":"2024-05-20T18:35:32.575311+0000","last_fullsized":"2024-05-20T18:35:32.575311+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:41:27.932466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.418194+0000","last_change":"2024-05-20T18:35:31.418822+0000","last_active":"2024-05-20T18:35:32.418194+0000","last_peered":"2024-05-20T18:35:32.418194+0000","last_clean":"2024-05-20T18:35:32.418194+0000","last_became_active":"2024-05-20T18:35:31.418530+0000","last_became_peered":"2024-05-20T18:35:31.418530+0000","last_unstale":"2024-05-20T18:35:32.418194+0000","last_undegraded":"2024-05-20T18:35:32.418194+0000","last_fullsized":"2024-05-20T18:35:32.418194+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:31:51.789823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.574940+0000","last_change":"2024-05-20T18:35:30.410136+0000","last_active":"2024-05-20T18:35:32.574940+0000","last_peered":"2024-05-20T18:35:32.574940+0000","last_clean":"2024-05-20T18:35:32.574940+0000","last_became_active":"2024-05-20T18:35:30.409778+0000","last_became_peered":"2024-05-20T18:35:30.409778+0000","last_unstale":"2024-05-20T18:35:32.574940+0000","last_undegraded":"2024-05-20T18:35:32.574940+0000","last_fullsized":"2024-05-20T18:35:32.574940+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:37:49.781033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.239028+0000","last_change":"2024-05-20T18:35:31.420081+0000","last_active":"2024-05-20T18:35:33.239028+0000","last_peered":"2024-05-20T18:35:33.239028+0000","last_clean":"2024-05-20T18:35:33.239028+0000","last_became_active":"2024-05-20T18:35:31.419505+0000","last_became_peered":"2024-05-20T18:35:31.419505+0000","last_unstale":"2024-05-20T18:35:33.239028+0000","last_undegraded":"2024-05-20T18:35:33.239028+0000","last_fullsized":"2024-05-20T18:35:33.239028+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:50:25.744663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.420570+0000","last_change":"2024-05-20T18:35:30.407327+0000","last_active":"2024-05-20T18:35:32.420570+0000","last_peered":"2024-05-20T18:35:32.420570+0000","last_clean":"2024-05-20T18:35:32.420570+0000","last_became_active":"2024-05-20T18:35:30.406994+0000","last_became_peered":"2024-05-20T18:35:30.406994+0000","last_unstale":"2024-05-20T18:35:32.420570+0000","last_undegraded":"2024-05-20T18:35:32.420570+0000","last_fullsized":"2024-05-20T18:35:32.420570+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:59:53.740452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.575183+0000","last_change":"2024-05-20T18:35:31.418822+0000","last_active":"2024-05-20T18:35:32.575183+0000","last_peered":"2024-05-20T18:35:32.575183+0000","last_clean":"2024-05-20T18:35:32.575183+0000","last_became_active":"2024-05-20T18:35:31.418512+0000","last_became_peered":"2024-05-20T18:35:31.418512+0000","last_unstale":"2024-05-20T18:35:32.575183+0000","last_undegraded":"2024-05-20T18:35:32.575183+0000","last_fullsized":"2024-05-20T18:35:32.575183+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:22:06.783328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.575437+0000","last_change":"2024-05-20T18:35:30.404056+0000","last_active":"2024-05-20T18:35:32.575437+0000","last_peered":"2024-05-20T18:35:32.575437+0000","last_clean":"2024-05-20T18:35:32.575437+0000","last_became_active":"2024-05-20T18:35:30.403695+0000","last_became_peered":"2024-05-20T18:35:30.403695+0000","last_unstale":"2024-05-20T18:35:32.575437+0000","last_undegraded":"2024-05-20T18:35:32.575437+0000","last_fullsized":"2024-05-20T18:35:32.575437+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:10:39.186470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.417744+0000","last_change":"2024-05-20T18:35:31.428550+0000","last_active":"2024-05-20T18:35:32.417744+0000","last_peered":"2024-05-20T18:35:32.417744+0000","last_clean":"2024-05-20T18:35:32.417744+0000","last_became_active":"2024-05-20T18:35:31.423158+0000","last_became_peered":"2024-05-20T18:35:31.423158+0000","last_unstale":"2024-05-20T18:35:32.417744+0000","last_undegraded":"2024-05-20T18:35:32.417744+0000","last_fullsized":"2024-05-20T18:35:32.417744+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:58:57.880090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238687+0000","last_change":"2024-05-20T18:35:30.412633+0000","last_active":"2024-05-20T18:35:33.238687+0000","last_peered":"2024-05-20T18:35:33.238687+0000","last_clean":"2024-05-20T18:35:33.238687+0000","last_became_active":"2024-05-20T18:35:30.412307+0000","last_became_peered":"2024-05-20T18:35:30.412307+0000","last_unstale":"2024-05-20T18:35:33.238687+0000","last_undegraded":"2024-05-20T18:35:33.238687+0000","last_fullsized":"2024-05-20T18:35:33.238687+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:59:53.740452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.420246+0000","last_change":"2024-05-20T18:35:31.425567+0000","last_active":"2024-05-20T18:35:32.420246+0000","last_peered":"2024-05-20T18:35:32.420246+0000","last_clean":"2024-05-20T18:35:32.420246+0000","last_became_active":"2024-05-20T18:35:31.425213+0000","last_became_peered":"2024-05-20T18:35:31.425213+0000","last_unstale":"2024-05-20T18:35:32.420246+0000","last_undegraded":"2024-05-20T18:35:32.420246+0000","last_fullsized":"2024-05-20T18:35:32.420246+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:19:35.163308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"105'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.417740+0000","last_change":"2024-05-20T18:35:30.411059+0000","last_active":"2024-05-20T18:35:32.417740+0000","last_peered":"2024-05-20T18:35:32.417740+0000","last_clean":"2024-05-20T18:35:32.417740+0000","last_became_active":"2024-05-20T18:35:30.410688+0000","last_became_peered":"2024-05-20T18:35:30.410688+0000","last_unstale":"2024-05-20T18:35:32.417740+0000","last_undegraded":"2024-05-20T18:35:32.417740+0000","last_fullsized":"2024-05-20T18:35:32.417740+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:32:03.221999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238766+0000","last_change":"2024-05-20T18:35:31.416928+0000","last_active":"2024-05-20T18:35:33.238766+0000","last_peered":"2024-05-20T18:35:33.238766+0000","last_clean":"2024-05-20T18:35:33.238766+0000","last_became_active":"2024-05-20T18:35:31.416613+0000","last_became_peered":"2024-05-20T18:35:31.416613+0000","last_unstale":"2024-05-20T18:35:33.238766+0000","last_undegraded":"2024-05-20T18:35:33.238766+0000","last_fullsized":"2024-05-20T18:35:33.238766+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:25:42.090152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.421142+0000","last_change":"2024-05-20T18:35:30.406122+0000","last_active":"2024-05-20T18:35:32.421142+0000","last_peered":"2024-05-20T18:35:32.421142+0000","last_clean":"2024-05-20T18:35:32.421142+0000","last_became_active":"2024-05-20T18:35:30.405784+0000","last_became_peered":"2024-05-20T18:35:30.405784+0000","last_unstale":"2024-05-20T18:35:32.421142+0000","last_undegraded":"2024-05-20T18:35:32.421142+0000","last_fullsized":"2024-05-20T18:35:32.421142+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:24:43.933783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.237873+0000","last_change":"2024-05-20T18:35:31.430996+0000","last_active":"2024-05-20T18:35:33.237873+0000","last_peered":"2024-05-20T18:35:33.237873+0000","last_clean":"2024-05-20T18:35:33.237873+0000","last_became_active":"2024-05-20T18:35:31.430691+0000","last_became_peered":"2024-05-20T18:35:31.430691+0000","last_unstale":"2024-05-20T18:35:33.237873+0000","last_undegraded":"2024-05-20T18:35:33.237873+0000","last_fullsized":"2024-05-20T18:35:33.237873+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:26:30.218312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238281+0000","last_change":"2024-05-20T18:35:30.400456+0000","last_active":"2024-05-20T18:35:33.238281+0000","last_peered":"2024-05-20T18:35:33.238281+0000","last_clean":"2024-05-20T18:35:33.238281+0000","last_became_active":"2024-05-20T18:35:30.399296+0000","last_became_peered":"2024-05-20T18:35:30.399296+0000","last_unstale":"2024-05-20T18:35:33.238281+0000","last_undegraded":"2024-05-20T18:35:33.238281+0000","last_fullsized":"2024-05-20T18:35:33.238281+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:19:27.219675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.575002+0000","last_change":"2024-05-20T18:35:31.423203+0000","last_active":"2024-05-20T18:35:32.575002+0000","last_peered":"2024-05-20T18:35:32.575002+0000","last_clean":"2024-05-20T18:35:32.575002+0000","last_became_active":"2024-05-20T18:35:31.422831+0000","last_became_peered":"2024-05-20T18:35:31.422831+0000","last_unstale":"2024-05-20T18:35:32.575002+0000","last_undegraded":"2024-05-20T18:35:32.575002+0000","last_fullsized":"2024-05-20T18:35:32.575002+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:57:23.130788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.420646+0000","last_change":"2024-05-20T18:35:30.408084+0000","last_active":"2024-05-20T18:35:32.420646+0000","last_peered":"2024-05-20T18:35:32.420646+0000","last_clean":"2024-05-20T18:35:32.420646+0000","last_became_active":"2024-05-20T18:35:30.407701+0000","last_became_peered":"2024-05-20T18:35:30.407701+0000","last_unstale":"2024-05-20T18:35:32.420646+0000","last_undegraded":"2024-05-20T18:35:32.420646+0000","last_fullsized":"2024-05-20T18:35:32.420646+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:22:04.089340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.575027+0000","last_change":"2024-05-20T18:35:31.422299+0000","last_active":"2024-05-20T18:35:32.575027+0000","last_peered":"2024-05-20T18:35:32.575027+0000","last_clean":"2024-05-20T18:35:32.575027+0000","last_became_active":"2024-05-20T18:35:31.421946+0000","last_became_peered":"2024-05-20T18:35:31.421946+0000","last_unstale":"2024-05-20T18:35:32.575027+0000","last_undegraded":"2024-05-20T18:35:32.575027+0000","last_fullsized":"2024-05-20T18:35:32.575027+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:59:54.966570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.575072+0000","last_change":"2024-05-20T18:35:30.414600+0000","last_active":"2024-05-20T18:35:32.575072+0000","last_peered":"2024-05-20T18:35:32.575072+0000","last_clean":"2024-05-20T18:35:32.575072+0000","last_became_active":"2024-05-20T18:35:30.414218+0000","last_became_peered":"2024-05-20T18:35:30.414218+0000","last_unstale":"2024-05-20T18:35:32.575072+0000","last_undegraded":"2024-05-20T18:35:32.575072+0000","last_fullsized":"2024-05-20T18:35:32.575072+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:56:58.143092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.577332+0000","last_change":"2024-05-20T18:35:31.429051+0000","last_active":"2024-05-20T18:35:32.577332+0000","last_peered":"2024-05-20T18:35:32.577332+0000","last_clean":"2024-05-20T18:35:32.577332+0000","last_became_active":"2024-05-20T18:35:31.428724+0000","last_became_peered":"2024-05-20T18:35:31.428724+0000","last_unstale":"2024-05-20T18:35:32.577332+0000","last_undegraded":"2024-05-20T18:35:32.577332+0000","last_fullsized":"2024-05-20T18:35:32.577332+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:45:44.257378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.420685+0000","last_change":"2024-05-20T18:35:30.406555+0000","last_active":"2024-05-20T18:35:32.420685+0000","last_peered":"2024-05-20T18:35:32.420685+0000","last_clean":"2024-05-20T18:35:32.420685+0000","last_became_active":"2024-05-20T18:35:30.405500+0000","last_became_peered":"2024-05-20T18:35:30.405500+0000","last_unstale":"2024-05-20T18:35:32.420685+0000","last_undegraded":"2024-05-20T18:35:32.420685+0000","last_fullsized":"2024-05-20T18:35:32.420685+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:59:20.346125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238265+0000","last_change":"2024-05-20T18:35:31.430305+0000","last_active":"2024-05-20T18:35:33.238265+0000","last_peered":"2024-05-20T18:35:33.238265+0000","last_clean":"2024-05-20T18:35:33.238265+0000","last_became_active":"2024-05-20T18:35:31.429852+0000","last_became_peered":"2024-05-20T18:35:31.429852+0000","last_unstale":"2024-05-20T18:35:33.238265+0000","last_undegraded":"2024-05-20T18:35:33.238265+0000","last_fullsized":"2024-05-20T18:35:33.238265+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:48.866975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.575542+0000","last_change":"2024-05-20T18:35:30.412896+0000","last_active":"2024-05-20T18:35:32.575542+0000","last_peered":"2024-05-20T18:35:32.575542+0000","last_clean":"2024-05-20T18:35:32.575542+0000","last_became_active":"2024-05-20T18:35:30.412388+0000","last_became_peered":"2024-05-20T18:35:30.412388+0000","last_unstale":"2024-05-20T18:35:32.575542+0000","last_undegraded":"2024-05-20T18:35:32.575542+0000","last_fullsized":"2024-05-20T18:35:32.575542+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:35:42.001777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.237708+0000","last_change":"2024-05-20T18:35:31.434643+0000","last_active":"2024-05-20T18:35:33.237708+0000","last_peered":"2024-05-20T18:35:33.237708+0000","last_clean":"2024-05-20T18:35:33.237708+0000","last_became_active":"2024-05-20T18:35:31.434223+0000","last_became_peered":"2024-05-20T18:35:31.434223+0000","last_unstale":"2024-05-20T18:35:33.237708+0000","last_undegraded":"2024-05-20T18:35:33.237708+0000","last_fullsized":"2024-05-20T18:35:33.237708+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:16:22.557898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.574958+0000","last_change":"2024-05-20T18:35:30.411370+0000","last_active":"2024-05-20T18:35:32.574958+0000","last_peered":"2024-05-20T18:35:32.574958+0000","last_clean":"2024-05-20T18:35:32.574958+0000","last_became_active":"2024-05-20T18:35:30.411022+0000","last_became_peered":"2024-05-20T18:35:30.411022+0000","last_unstale":"2024-05-20T18:35:32.574958+0000","last_undegraded":"2024-05-20T18:35:32.574958+0000","last_fullsized":"2024-05-20T18:35:32.574958+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:49:00.568944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":326,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.576840+0000","last_change":"2024-05-20T18:27:03.765694+0000","last_active":"2024-05-20T18:35:32.576840+0000","last_peered":"2024-05-20T18:35:32.576840+0000","last_clean":"2024-05-20T18:35:32.576840+0000","last_became_active":"2024-05-20T18:27:03.764695+0000","last_became_peered":"2024-05-20T18:27:03.764695+0000","last_unstale":"2024-05-20T18:35:32.576840+0000","last_undegraded":"2024-05-20T18:35:32.576840+0000","last_fullsized":"2024-05-20T18:35:32.576840+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_clean_scrub_stamp":"2024-05-20T18:27:02.750853+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:30:04.407487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574955,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33044,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338796,"statfs":{"total":96636764160,"available":96602927104,"internally_reserved":0,"allocated":1904640,"data_stored":1195051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574953,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1323008,"data_stored":606739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574953,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1302528,"data_stored":604523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574953,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1310720,"data_stored":604615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574955,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1306624,"data_stored":605105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574954,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1306624,"data_stored":604557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574955,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1306624,"data_stored":604557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574954,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33052,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338788,"statfs":{"total":96636764160,"available":96602918912,"internally_reserved":0,"allocated":1912832,"data_stored":1196593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T18:35:40.004 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T18:35:40.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.134+0000 7f914a92c640 1 Processor -- start 2024-05-20T18:35:40.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.134+0000 7f914a92c640 1 -- start start 2024-05-20T18:35:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.135+0000 7f914a92c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f91440af030 0x7f91440af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.135+0000 7f914a92c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f914415f700 0x7f914415fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.135+0000 7f914a92c640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91440af980 con 0x7f914409a4f0 2024-05-20T18:35:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.135+0000 7f914a92c640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f914406bcf0 con 0x7f91440af030 2024-05-20T18:35:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.135+0000 7f914a92c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9144160430 con 0x7f914415f700 2024-05-20T18:35:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.135+0000 7f9143fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f91440af030 0x7f91440af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.135+0000 7f9143fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f91440af030 0x7f91440af440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49654/0 (socket says 172.21.15.5:49654) 2024-05-20T18:35:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.135+0000 7f9143fff640 1 -- 172.21.15.5:0/583789749 learned_addr learned my addr 172.21.15.5:0/583789749 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.135+0000 7f91437fe640 1 --2- 172.21.15.5:0/583789749 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f914415f700 0x7f914415fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.136+0000 7f9143fff640 1 -- 172.21.15.5:0/583789749 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f914415f700 msgr2=0x7f914415fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.136+0000 7f9143fff640 1 --2- 172.21.15.5:0/583789749 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f914415f700 0x7f914415fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.136+0000 7f9143fff640 1 -- 172.21.15.5:0/583789749 >> v1:172.21.15.63:6789/0 conn(0x7f914409a4f0 legacy=0x7f914409a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.136+0000 7f9143fff640 1 -- 172.21.15.5:0/583789749 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f914406b6a0 con 0x7f91440af030 2024-05-20T18:35:40.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.136+0000 7f9143fff640 1 --2- 172.21.15.5:0/583789749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91440af030 0x7f91440af440 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f9134009870 tx=0x7f913402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5882a090ededfe4d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:40.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.137+0000 7f9142ffd640 1 -- 172.21.15.5:0/583789749 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134037070 con 0x7f91440af030 2024-05-20T18:35:40.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.137+0000 7f9142ffd640 1 -- 172.21.15.5:0/583789749 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9134032440 con 0x7f91440af030 2024-05-20T18:35:40.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.137+0000 7f9142ffd640 1 -- 172.21.15.5:0/583789749 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134032810 con 0x7f91440af030 2024-05-20T18:35:40.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.137+0000 7f914a92c640 1 -- 172.21.15.5:0/583789749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91440af030 msgr2=0x7f91440af440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:40.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.137+0000 7f914a92c640 1 --2- 172.21.15.5:0/583789749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91440af030 0x7f91440af440 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f9134009870 tx=0x7f913402a270 comp rx=0 tx=0).stop 2024-05-20T18:35:40.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.137+0000 7f914a92c640 1 -- 172.21.15.5:0/583789749 shutdown_connections 2024-05-20T18:35:40.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.137+0000 7f914a92c640 1 --2- 172.21.15.5:0/583789749 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f914415f700 0x7f914415fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:40.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.137+0000 7f914a92c640 1 --2- 172.21.15.5:0/583789749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91440af030 0x7f91440af440 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:40.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.137+0000 7f914a92c640 1 -- 172.21.15.5:0/583789749 >> 172.21.15.5:0/583789749 conn(0x7f91440b6620 msgr2=0x7f91440b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:40.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.137+0000 7f914a92c640 1 -- 172.21.15.5:0/583789749 shutdown_connections 2024-05-20T18:35:40.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.137+0000 7f914a92c640 1 -- 172.21.15.5:0/583789749 wait complete. 2024-05-20T18:35:40.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.139+0000 7f914a92c640 1 Processor -- start 2024-05-20T18:35:40.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.139+0000 7f914a92c640 1 -- start start 2024-05-20T18:35:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.141+0000 7f914a92c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f914409a4f0 0x7f91441f9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.141+0000 7f914a92c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91440af030 0x7f91441f8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.141+0000 7f914a92c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f914415f700 0x7f9144161840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.141+0000 7f914a92c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9144160050 con 0x7f91440af030 2024-05-20T18:35:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.141+0000 7f914a92c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9144159aa0 con 0x7f914415f700 2024-05-20T18:35:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.141+0000 7f914a92c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f91441601d0 con 0x7f914409a4f0 2024-05-20T18:35:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.141+0000 7f91437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91440af030 0x7f91441f8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.141+0000 7f91437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91440af030 0x7f91441f8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49666/0 (socket says 172.21.15.5:49666) 2024-05-20T18:35:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.141+0000 7f91437fe640 1 -- 172.21.15.5:0/3980727401 learned_addr learned my addr 172.21.15.5:0/3980727401 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.141+0000 7f9143fff640 1 --2- 172.21.15.5:0/3980727401 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f914409a4f0 0x7f91441f9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.141+0000 7f9148ec9640 1 --2- 172.21.15.5:0/3980727401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f914415f700 0x7f9144161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.141+0000 7f91437fe640 1 -- 172.21.15.5:0/3980727401 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f914409a4f0 msgr2=0x7f91441f9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.141+0000 7f91437fe640 1 --2- 172.21.15.5:0/3980727401 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f914409a4f0 0x7f91441f9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.141+0000 7f91437fe640 1 -- 172.21.15.5:0/3980727401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f914415f700 msgr2=0x7f9144161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.141+0000 7f91437fe640 1 --2- 172.21.15.5:0/3980727401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f914415f700 0x7f9144161840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.141+0000 7f91437fe640 1 -- 172.21.15.5:0/3980727401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9144145150 con 0x7f91440af030 2024-05-20T18:35:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.141+0000 7f91437fe640 1 --2- 172.21.15.5:0/3980727401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91440af030 0x7f91441f8030 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f913800d600 tx=0x7f913800dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.142+0000 7f9140ff9640 1 -- 172.21.15.5:0/3980727401 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91380125f0 con 0x7f91440af030 2024-05-20T18:35:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.142+0000 7f9140ff9640 1 -- 172.21.15.5:0/3980727401 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9138019070 con 0x7f91440af030 2024-05-20T18:35:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.142+0000 7f914a92c640 1 -- 172.21.15.5:0/3980727401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9144145430 con 0x7f91440af030 2024-05-20T18:35:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.142+0000 7f9140ff9640 1 -- 172.21.15.5:0/3980727401 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91380129f0 con 0x7f91440af030 2024-05-20T18:35:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.142+0000 7f914a92c640 1 -- 172.21.15.5:0/3980727401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91441f8570 con 0x7f91440af030 2024-05-20T18:35:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.143+0000 7f914a92c640 1 -- 172.21.15.5:0/3980727401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9144075b00 con 0x7f91440af030 2024-05-20T18:35:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.143+0000 7f9140ff9640 1 -- 172.21.15.5:0/3980727401 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9138012b90 con 0x7f91440af030 2024-05-20T18:35:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.144+0000 7f9140ff9640 1 --2- 172.21.15.5:0/3980727401 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f913006be30 0x7f913006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.144+0000 7f9140ff9640 1 -- 172.21.15.5:0/3980727401 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7f913808b410 con 0x7f91440af030 2024-05-20T18:35:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.144+0000 7f9143fff640 1 --2- 172.21.15.5:0/3980727401 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f913006be30 0x7f913006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.144+0000 7f9143fff640 1 --2- 172.21.15.5:0/3980727401 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f913006be30 0x7f913006e340 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9134034000 tx=0x7f913402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:40.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.149+0000 7f9140ff9640 1 -- 172.21.15.5:0/3980727401 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9138059c30 con 0x7f91440af030 2024-05-20T18:35:40.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.328+0000 7f914a92c640 1 -- 172.21.15.5:0/3980727401 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f91440d1630 con 0x7f913006be30 2024-05-20T18:35:40.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.334+0000 7f9140ff9640 1 -- 172.21.15.5:0/3980727401 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252486 (secure 0 0 0) 0x7f91440d1630 con 0x7f913006be30 2024-05-20T18:35:40.338 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:40.339 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-20T18:35:40.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.337+0000 7f914a92c640 1 -- 172.21.15.5:0/3980727401 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f913006be30 msgr2=0x7f913006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:40.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.337+0000 7f914a92c640 1 --2- 172.21.15.5:0/3980727401 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f913006be30 0x7f913006e340 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9134034000 tx=0x7f913402e000 comp rx=0 tx=0).stop 2024-05-20T18:35:40.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.337+0000 7f914a92c640 1 -- 172.21.15.5:0/3980727401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91440af030 msgr2=0x7f91441f8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:40.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.337+0000 7f914a92c640 1 --2- 172.21.15.5:0/3980727401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91440af030 0x7f91441f8030 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f913800d600 tx=0x7f913800dac0 comp rx=0 tx=0).stop 2024-05-20T18:35:40.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.337+0000 7f914a92c640 1 -- 172.21.15.5:0/3980727401 shutdown_connections 2024-05-20T18:35:40.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.337+0000 7f914a92c640 1 --2- 172.21.15.5:0/3980727401 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f913006be30 0x7f913006e340 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:40.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.337+0000 7f914a92c640 1 --2- 172.21.15.5:0/3980727401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f914415f700 0x7f9144161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:40.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.337+0000 7f914a92c640 1 --2- 172.21.15.5:0/3980727401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91440af030 0x7f91441f8030 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:40.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.337+0000 7f914a92c640 1 --2- 172.21.15.5:0/3980727401 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f914409a4f0 0x7f91441f9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:40.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.338+0000 7f914a92c640 1 -- 172.21.15.5:0/3980727401 >> 172.21.15.5:0/3980727401 conn(0x7f91440b6620 msgr2=0x7f9144070310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:40.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.338+0000 7f914a92c640 1 -- 172.21.15.5:0/3980727401 shutdown_connections 2024-05-20T18:35:40.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.338+0000 7f914a92c640 1 -- 172.21.15.5:0/3980727401 wait complete. 2024-05-20T18:35:40.350 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-05-20T18:35:38.705532+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260876,"kb_used_data":11400,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713844,"statfs":{"total":773094113280,"available":772826976256,"internally_reserved":0,"allocated":11673600,"data_stored":6021740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001731"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.418005+0000","last_change":"2024-05-20T18:35:31.424894+0000","last_active":"2024-05-20T18:35:32.418005+0000","last_peered":"2024-05-20T18:35:32.418005+0000","last_clean":"2024-05-20T18:35:32.418005+0000","last_became_active":"2024-05-20T18:35:31.424577+0000","last_became_peered":"2024-05-20T18:35:31.424577+0000","last_unstale":"2024-05-20T18:35:32.418005+0000","last_undegraded":"2024-05-20T18:35:32.418005+0000","last_fullsized":"2024-05-20T18:35:32.418005+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:41:32.611483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.421808+0000","last_change":"2024-05-20T18:35:31.421351+0000","last_active":"2024-05-20T18:35:32.421808+0000","last_peered":"2024-05-20T18:35:32.421808+0000","last_clean":"2024-05-20T18:35:32.421808+0000","last_became_active":"2024-05-20T18:35:31.420985+0000","last_became_peered":"2024-05-20T18:35:31.420985+0000","last_unstale":"2024-05-20T18:35:32.421808+0000","last_undegraded":"2024-05-20T18:35:32.421808+0000","last_fullsized":"2024-05-20T18:35:32.421808+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:51:25.489802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.417921+0000","last_change":"2024-05-20T18:35:31.431529+0000","last_active":"2024-05-20T18:35:32.417921+0000","last_peered":"2024-05-20T18:35:32.417921+0000","last_clean":"2024-05-20T18:35:32.417921+0000","last_became_active":"2024-05-20T18:35:31.431235+0000","last_became_peered":"2024-05-20T18:35:31.431235+0000","last_unstale":"2024-05-20T18:35:32.417921+0000","last_undegraded":"2024-05-20T18:35:32.417921+0000","last_fullsized":"2024-05-20T18:35:32.417921+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:57:55.678295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238773+0000","last_change":"2024-05-20T18:35:31.426986+0000","last_active":"2024-05-20T18:35:33.238773+0000","last_peered":"2024-05-20T18:35:33.238773+0000","last_clean":"2024-05-20T18:35:33.238773+0000","last_became_active":"2024-05-20T18:35:31.426700+0000","last_became_peered":"2024-05-20T18:35:31.426700+0000","last_unstale":"2024-05-20T18:35:33.238773+0000","last_undegraded":"2024-05-20T18:35:33.238773+0000","last_fullsized":"2024-05-20T18:35:33.238773+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:29:47.972518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.237987+0000","last_change":"2024-05-20T18:35:31.415028+0000","last_active":"2024-05-20T18:35:33.237987+0000","last_peered":"2024-05-20T18:35:33.237987+0000","last_clean":"2024-05-20T18:35:33.237987+0000","last_became_active":"2024-05-20T18:35:31.414706+0000","last_became_peered":"2024-05-20T18:35:31.414706+0000","last_unstale":"2024-05-20T18:35:33.237987+0000","last_undegraded":"2024-05-20T18:35:33.237987+0000","last_fullsized":"2024-05-20T18:35:33.237987+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:52:36.277484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.420891+0000","last_change":"2024-05-20T18:35:31.427490+0000","last_active":"2024-05-20T18:35:32.420891+0000","last_peered":"2024-05-20T18:35:32.420891+0000","last_clean":"2024-05-20T18:35:32.420891+0000","last_became_active":"2024-05-20T18:35:31.427174+0000","last_became_peered":"2024-05-20T18:35:31.427174+0000","last_unstale":"2024-05-20T18:35:32.420891+0000","last_undegraded":"2024-05-20T18:35:32.420891+0000","last_fullsized":"2024-05-20T18:35:32.420891+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:16:22.557898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.420233+0000","last_change":"2024-05-20T18:35:31.413562+0000","last_active":"2024-05-20T18:35:32.420233+0000","last_peered":"2024-05-20T18:35:32.420233+0000","last_clean":"2024-05-20T18:35:32.420233+0000","last_became_active":"2024-05-20T18:35:31.413252+0000","last_became_peered":"2024-05-20T18:35:31.413252+0000","last_unstale":"2024-05-20T18:35:32.420233+0000","last_undegraded":"2024-05-20T18:35:32.420233+0000","last_fullsized":"2024-05-20T18:35:32.420233+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:35:00.678467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.574807+0000","last_change":"2024-05-20T18:35:31.420510+0000","last_active":"2024-05-20T18:35:32.574807+0000","last_peered":"2024-05-20T18:35:32.574807+0000","last_clean":"2024-05-20T18:35:32.574807+0000","last_became_active":"2024-05-20T18:35:31.420004+0000","last_became_peered":"2024-05-20T18:35:31.420004+0000","last_unstale":"2024-05-20T18:35:32.574807+0000","last_undegraded":"2024-05-20T18:35:32.574807+0000","last_fullsized":"2024-05-20T18:35:32.574807+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:26:18.458203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.420786+0000","last_change":"2024-05-20T18:35:31.422857+0000","last_active":"2024-05-20T18:35:32.420786+0000","last_peered":"2024-05-20T18:35:32.420786+0000","last_clean":"2024-05-20T18:35:32.420786+0000","last_became_active":"2024-05-20T18:35:31.422519+0000","last_became_peered":"2024-05-20T18:35:31.422519+0000","last_unstale":"2024-05-20T18:35:32.420786+0000","last_undegraded":"2024-05-20T18:35:32.420786+0000","last_fullsized":"2024-05-20T18:35:32.420786+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:02:05.636153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238524+0000","last_change":"2024-05-20T18:35:31.428547+0000","last_active":"2024-05-20T18:35:33.238524+0000","last_peered":"2024-05-20T18:35:33.238524+0000","last_clean":"2024-05-20T18:35:33.238524+0000","last_became_active":"2024-05-20T18:35:31.428234+0000","last_became_peered":"2024-05-20T18:35:31.428234+0000","last_unstale":"2024-05-20T18:35:33.238524+0000","last_undegraded":"2024-05-20T18:35:33.238524+0000","last_fullsized":"2024-05-20T18:35:33.238524+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:19:35.163308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.237950+0000","last_change":"2024-05-20T18:35:31.428591+0000","last_active":"2024-05-20T18:35:33.237950+0000","last_peered":"2024-05-20T18:35:33.237950+0000","last_clean":"2024-05-20T18:35:33.237950+0000","last_became_active":"2024-05-20T18:35:31.428289+0000","last_became_peered":"2024-05-20T18:35:31.428289+0000","last_unstale":"2024-05-20T18:35:33.237950+0000","last_undegraded":"2024-05-20T18:35:33.237950+0000","last_fullsized":"2024-05-20T18:35:33.237950+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:51:25.489802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.421357+0000","last_change":"2024-05-20T18:35:31.426873+0000","last_active":"2024-05-20T18:35:32.421357+0000","last_peered":"2024-05-20T18:35:32.421357+0000","last_clean":"2024-05-20T18:35:32.421357+0000","last_became_active":"2024-05-20T18:35:31.426575+0000","last_became_peered":"2024-05-20T18:35:31.426575+0000","last_unstale":"2024-05-20T18:35:32.421357+0000","last_undegraded":"2024-05-20T18:35:32.421357+0000","last_fullsized":"2024-05-20T18:35:32.421357+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:48.866975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238238+0000","last_change":"2024-05-20T18:35:31.428552+0000","last_active":"2024-05-20T18:35:33.238238+0000","last_peered":"2024-05-20T18:35:33.238238+0000","last_clean":"2024-05-20T18:35:33.238238+0000","last_became_active":"2024-05-20T18:35:31.428231+0000","last_became_peered":"2024-05-20T18:35:31.428231+0000","last_unstale":"2024-05-20T18:35:33.238238+0000","last_undegraded":"2024-05-20T18:35:33.238238+0000","last_fullsized":"2024-05-20T18:35:33.238238+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:45:43.193642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.576996+0000","last_change":"2024-05-20T18:35:31.420831+0000","last_active":"2024-05-20T18:35:32.576996+0000","last_peered":"2024-05-20T18:35:32.576996+0000","last_clean":"2024-05-20T18:35:32.576996+0000","last_became_active":"2024-05-20T18:35:31.420499+0000","last_became_peered":"2024-05-20T18:35:31.420499+0000","last_unstale":"2024-05-20T18:35:32.576996+0000","last_undegraded":"2024-05-20T18:35:32.576996+0000","last_fullsized":"2024-05-20T18:35:32.576996+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:22:59.144571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.576952+0000","last_change":"2024-05-20T18:35:31.428171+0000","last_active":"2024-05-20T18:35:32.576952+0000","last_peered":"2024-05-20T18:35:32.576952+0000","last_clean":"2024-05-20T18:35:32.576952+0000","last_became_active":"2024-05-20T18:35:31.427857+0000","last_became_peered":"2024-05-20T18:35:31.427857+0000","last_unstale":"2024-05-20T18:35:32.576952+0000","last_undegraded":"2024-05-20T18:35:32.576952+0000","last_fullsized":"2024-05-20T18:35:32.576952+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:58:19.834727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.576929+0000","last_change":"2024-05-20T18:35:31.419900+0000","last_active":"2024-05-20T18:35:32.576929+0000","last_peered":"2024-05-20T18:35:32.576929+0000","last_clean":"2024-05-20T18:35:32.576929+0000","last_became_active":"2024-05-20T18:35:31.419541+0000","last_became_peered":"2024-05-20T18:35:31.419541+0000","last_unstale":"2024-05-20T18:35:32.576929+0000","last_undegraded":"2024-05-20T18:35:32.576929+0000","last_fullsized":"2024-05-20T18:35:32.576929+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:12:16.222008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.417916+0000","last_change":"2024-05-20T18:35:31.430878+0000","last_active":"2024-05-20T18:35:32.417916+0000","last_peered":"2024-05-20T18:35:32.417916+0000","last_clean":"2024-05-20T18:35:32.417916+0000","last_became_active":"2024-05-20T18:35:31.430531+0000","last_became_peered":"2024-05-20T18:35:31.430531+0000","last_unstale":"2024-05-20T18:35:32.417916+0000","last_undegraded":"2024-05-20T18:35:32.417916+0000","last_fullsized":"2024-05-20T18:35:32.417916+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:55:12.845763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.422037+0000","last_change":"2024-05-20T18:35:31.427821+0000","last_active":"2024-05-20T18:35:32.422037+0000","last_peered":"2024-05-20T18:35:32.422037+0000","last_clean":"2024-05-20T18:35:32.422037+0000","last_became_active":"2024-05-20T18:35:31.427009+0000","last_became_peered":"2024-05-20T18:35:31.427009+0000","last_unstale":"2024-05-20T18:35:32.422037+0000","last_undegraded":"2024-05-20T18:35:32.422037+0000","last_fullsized":"2024-05-20T18:35:32.422037+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:48.866975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.418110+0000","last_change":"2024-05-20T18:35:31.414035+0000","last_active":"2024-05-20T18:35:32.418110+0000","last_peered":"2024-05-20T18:35:32.418110+0000","last_clean":"2024-05-20T18:35:32.418110+0000","last_became_active":"2024-05-20T18:35:31.413520+0000","last_became_peered":"2024-05-20T18:35:31.413520+0000","last_unstale":"2024-05-20T18:35:32.418110+0000","last_undegraded":"2024-05-20T18:35:32.418110+0000","last_fullsized":"2024-05-20T18:35:32.418110+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:22:02.551585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.421574+0000","last_change":"2024-05-20T18:35:31.422500+0000","last_active":"2024-05-20T18:35:32.421574+0000","last_peered":"2024-05-20T18:35:32.421574+0000","last_clean":"2024-05-20T18:35:32.421574+0000","last_became_active":"2024-05-20T18:35:31.422196+0000","last_became_peered":"2024-05-20T18:35:31.422196+0000","last_unstale":"2024-05-20T18:35:32.421574+0000","last_undegraded":"2024-05-20T18:35:32.421574+0000","last_fullsized":"2024-05-20T18:35:32.421574+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:04.213768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.575028+0000","last_change":"2024-05-20T18:35:31.432624+0000","last_active":"2024-05-20T18:35:32.575028+0000","last_peered":"2024-05-20T18:35:32.575028+0000","last_clean":"2024-05-20T18:35:32.575028+0000","last_became_active":"2024-05-20T18:35:31.432296+0000","last_became_peered":"2024-05-20T18:35:31.432296+0000","last_unstale":"2024-05-20T18:35:32.575028+0000","last_undegraded":"2024-05-20T18:35:32.575028+0000","last_fullsized":"2024-05-20T18:35:32.575028+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:19:21.865495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238389+0000","last_change":"2024-05-20T18:35:31.431224+0000","last_active":"2024-05-20T18:35:33.238389+0000","last_peered":"2024-05-20T18:35:33.238389+0000","last_clean":"2024-05-20T18:35:33.238389+0000","last_became_active":"2024-05-20T18:35:31.430907+0000","last_became_peered":"2024-05-20T18:35:31.430907+0000","last_unstale":"2024-05-20T18:35:33.238389+0000","last_undegraded":"2024-05-20T18:35:33.238389+0000","last_fullsized":"2024-05-20T18:35:33.238389+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:16:22.557898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.417577+0000","last_change":"2024-05-20T18:35:31.429270+0000","last_active":"2024-05-20T18:35:32.417577+0000","last_peered":"2024-05-20T18:35:32.417577+0000","last_clean":"2024-05-20T18:35:32.417577+0000","last_became_active":"2024-05-20T18:35:31.428933+0000","last_became_peered":"2024-05-20T18:35:31.428933+0000","last_unstale":"2024-05-20T18:35:32.417577+0000","last_undegraded":"2024-05-20T18:35:32.417577+0000","last_fullsized":"2024-05-20T18:35:32.417577+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:59:21.885118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.237901+0000","last_change":"2024-05-20T18:35:31.421744+0000","last_active":"2024-05-20T18:35:33.237901+0000","last_peered":"2024-05-20T18:35:33.237901+0000","last_clean":"2024-05-20T18:35:33.237901+0000","last_became_active":"2024-05-20T18:35:31.421411+0000","last_became_peered":"2024-05-20T18:35:31.421411+0000","last_unstale":"2024-05-20T18:35:33.237901+0000","last_undegraded":"2024-05-20T18:35:33.237901+0000","last_fullsized":"2024-05-20T18:35:33.237901+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:57:42.416971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.421559+0000","last_change":"2024-05-20T18:35:31.431228+0000","last_active":"2024-05-20T18:35:32.421559+0000","last_peered":"2024-05-20T18:35:32.421559+0000","last_clean":"2024-05-20T18:35:32.421559+0000","last_became_active":"2024-05-20T18:35:31.430609+0000","last_became_peered":"2024-05-20T18:35:31.430609+0000","last_unstale":"2024-05-20T18:35:32.421559+0000","last_undegraded":"2024-05-20T18:35:32.421559+0000","last_fullsized":"2024-05-20T18:35:32.421559+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:46:50.437839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.421069+0000","last_change":"2024-05-20T18:35:31.412446+0000","last_active":"2024-05-20T18:35:32.421069+0000","last_peered":"2024-05-20T18:35:32.421069+0000","last_clean":"2024-05-20T18:35:32.421069+0000","last_became_active":"2024-05-20T18:35:31.412118+0000","last_became_peered":"2024-05-20T18:35:31.412118+0000","last_unstale":"2024-05-20T18:35:32.421069+0000","last_undegraded":"2024-05-20T18:35:32.421069+0000","last_fullsized":"2024-05-20T18:35:32.421069+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:51:25.489802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238280+0000","last_change":"2024-05-20T18:35:31.430873+0000","last_active":"2024-05-20T18:35:33.238280+0000","last_peered":"2024-05-20T18:35:33.238280+0000","last_clean":"2024-05-20T18:35:33.238280+0000","last_became_active":"2024-05-20T18:35:31.430452+0000","last_became_peered":"2024-05-20T18:35:31.430452+0000","last_unstale":"2024-05-20T18:35:33.238280+0000","last_undegraded":"2024-05-20T18:35:33.238280+0000","last_fullsized":"2024-05-20T18:35:33.238280+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:46:50.437839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.421697+0000","last_change":"2024-05-20T18:35:31.428946+0000","last_active":"2024-05-20T18:35:32.421697+0000","last_peered":"2024-05-20T18:35:32.421697+0000","last_clean":"2024-05-20T18:35:32.421697+0000","last_became_active":"2024-05-20T18:35:31.428590+0000","last_became_peered":"2024-05-20T18:35:31.428590+0000","last_unstale":"2024-05-20T18:35:32.421697+0000","last_undegraded":"2024-05-20T18:35:32.421697+0000","last_fullsized":"2024-05-20T18:35:32.421697+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:00:14.038942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.575195+0000","last_change":"2024-05-20T18:35:31.431023+0000","last_active":"2024-05-20T18:35:32.575195+0000","last_peered":"2024-05-20T18:35:32.575195+0000","last_clean":"2024-05-20T18:35:32.575195+0000","last_became_active":"2024-05-20T18:35:31.426870+0000","last_became_peered":"2024-05-20T18:35:31.426870+0000","last_unstale":"2024-05-20T18:35:32.575195+0000","last_undegraded":"2024-05-20T18:35:32.575195+0000","last_fullsized":"2024-05-20T18:35:32.575195+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:07:20.710926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.420637+0000","last_change":"2024-05-20T18:35:31.428685+0000","last_active":"2024-05-20T18:35:32.420637+0000","last_peered":"2024-05-20T18:35:32.420637+0000","last_clean":"2024-05-20T18:35:32.420637+0000","last_became_active":"2024-05-20T18:35:31.428366+0000","last_became_peered":"2024-05-20T18:35:31.428366+0000","last_unstale":"2024-05-20T18:35:32.420637+0000","last_undegraded":"2024-05-20T18:35:32.420637+0000","last_fullsized":"2024-05-20T18:35:32.420637+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:26:30.218312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.420441+0000","last_change":"2024-05-20T18:35:31.412871+0000","last_active":"2024-05-20T18:35:32.420441+0000","last_peered":"2024-05-20T18:35:32.420441+0000","last_clean":"2024-05-20T18:35:32.420441+0000","last_became_active":"2024-05-20T18:35:31.412480+0000","last_became_peered":"2024-05-20T18:35:31.412480+0000","last_unstale":"2024-05-20T18:35:32.420441+0000","last_undegraded":"2024-05-20T18:35:32.420441+0000","last_fullsized":"2024-05-20T18:35:32.420441+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:57:42.416971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.574887+0000","last_change":"2024-05-20T18:35:31.427123+0000","last_active":"2024-05-20T18:35:32.574887+0000","last_peered":"2024-05-20T18:35:32.574887+0000","last_clean":"2024-05-20T18:35:32.574887+0000","last_became_active":"2024-05-20T18:35:31.426800+0000","last_became_peered":"2024-05-20T18:35:31.426800+0000","last_unstale":"2024-05-20T18:35:32.574887+0000","last_undegraded":"2024-05-20T18:35:32.574887+0000","last_fullsized":"2024-05-20T18:35:32.574887+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:24:09.108397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.417954+0000","last_change":"2024-05-20T18:35:31.414346+0000","last_active":"2024-05-20T18:35:32.417954+0000","last_peered":"2024-05-20T18:35:32.417954+0000","last_clean":"2024-05-20T18:35:32.417954+0000","last_became_active":"2024-05-20T18:35:31.412899+0000","last_became_peered":"2024-05-20T18:35:31.412899+0000","last_unstale":"2024-05-20T18:35:32.417954+0000","last_undegraded":"2024-05-20T18:35:32.417954+0000","last_fullsized":"2024-05-20T18:35:32.417954+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:44:07.390703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"104'4","reported_seq":18,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238671+0000","last_change":"2024-05-20T18:35:30.407837+0000","last_active":"2024-05-20T18:35:33.238671+0000","last_peered":"2024-05-20T18:35:33.238671+0000","last_clean":"2024-05-20T18:35:33.238671+0000","last_became_active":"2024-05-20T18:35:30.407539+0000","last_became_peered":"2024-05-20T18:35:30.407539+0000","last_unstale":"2024-05-20T18:35:33.238671+0000","last_undegraded":"2024-05-20T18:35:33.238671+0000","last_fullsized":"2024-05-20T18:35:33.238671+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:17:55.050005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.574852+0000","last_change":"2024-05-20T18:35:31.417545+0000","last_active":"2024-05-20T18:35:32.574852+0000","last_peered":"2024-05-20T18:35:32.574852+0000","last_clean":"2024-05-20T18:35:32.574852+0000","last_became_active":"2024-05-20T18:35:31.417145+0000","last_became_peered":"2024-05-20T18:35:31.417145+0000","last_unstale":"2024-05-20T18:35:32.574852+0000","last_undegraded":"2024-05-20T18:35:32.574852+0000","last_fullsized":"2024-05-20T18:35:32.574852+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:44:59.743100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.575366+0000","last_change":"2024-05-20T18:35:30.406366+0000","last_active":"2024-05-20T18:35:32.575366+0000","last_peered":"2024-05-20T18:35:32.575366+0000","last_clean":"2024-05-20T18:35:32.575366+0000","last_became_active":"2024-05-20T18:35:30.405770+0000","last_became_peered":"2024-05-20T18:35:30.405770+0000","last_unstale":"2024-05-20T18:35:32.575366+0000","last_undegraded":"2024-05-20T18:35:32.575366+0000","last_fullsized":"2024-05-20T18:35:32.575366+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:04:30.447517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.420834+0000","last_change":"2024-05-20T18:35:31.424665+0000","last_active":"2024-05-20T18:35:32.420834+0000","last_peered":"2024-05-20T18:35:32.420834+0000","last_clean":"2024-05-20T18:35:32.420834+0000","last_became_active":"2024-05-20T18:35:31.424324+0000","last_became_peered":"2024-05-20T18:35:31.424324+0000","last_unstale":"2024-05-20T18:35:32.420834+0000","last_undegraded":"2024-05-20T18:35:32.420834+0000","last_fullsized":"2024-05-20T18:35:32.420834+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:00:14.038942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.421055+0000","last_change":"2024-05-20T18:35:30.409253+0000","last_active":"2024-05-20T18:35:32.421055+0000","last_peered":"2024-05-20T18:35:32.421055+0000","last_clean":"2024-05-20T18:35:32.421055+0000","last_became_active":"2024-05-20T18:35:30.408847+0000","last_became_peered":"2024-05-20T18:35:30.408847+0000","last_unstale":"2024-05-20T18:35:32.421055+0000","last_undegraded":"2024-05-20T18:35:32.421055+0000","last_fullsized":"2024-05-20T18:35:32.421055+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:28:15.030130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.237962+0000","last_change":"2024-05-20T18:35:31.436546+0000","last_active":"2024-05-20T18:35:33.237962+0000","last_peered":"2024-05-20T18:35:33.237962+0000","last_clean":"2024-05-20T18:35:33.237962+0000","last_became_active":"2024-05-20T18:35:31.436258+0000","last_became_peered":"2024-05-20T18:35:31.436258+0000","last_unstale":"2024-05-20T18:35:33.237962+0000","last_undegraded":"2024-05-20T18:35:33.237962+0000","last_fullsized":"2024-05-20T18:35:33.237962+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:00:14.038942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238141+0000","last_change":"2024-05-20T18:35:30.402307+0000","last_active":"2024-05-20T18:35:33.238141+0000","last_peered":"2024-05-20T18:35:33.238141+0000","last_clean":"2024-05-20T18:35:33.238141+0000","last_became_active":"2024-05-20T18:35:30.401952+0000","last_became_peered":"2024-05-20T18:35:30.401952+0000","last_unstale":"2024-05-20T18:35:33.238141+0000","last_undegraded":"2024-05-20T18:35:33.238141+0000","last_fullsized":"2024-05-20T18:35:33.238141+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:13:02.199994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.239027+0000","last_change":"2024-05-20T18:35:31.420523+0000","last_active":"2024-05-20T18:35:33.239027+0000","last_peered":"2024-05-20T18:35:33.239027+0000","last_clean":"2024-05-20T18:35:33.239027+0000","last_became_active":"2024-05-20T18:35:31.420237+0000","last_became_peered":"2024-05-20T18:35:31.420237+0000","last_unstale":"2024-05-20T18:35:33.239027+0000","last_undegraded":"2024-05-20T18:35:33.239027+0000","last_fullsized":"2024-05-20T18:35:33.239027+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:32:04.226629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.577120+0000","last_change":"2024-05-20T18:35:30.405866+0000","last_active":"2024-05-20T18:35:32.577120+0000","last_peered":"2024-05-20T18:35:32.577120+0000","last_clean":"2024-05-20T18:35:32.577120+0000","last_became_active":"2024-05-20T18:35:30.405503+0000","last_became_peered":"2024-05-20T18:35:30.405503+0000","last_unstale":"2024-05-20T18:35:32.577120+0000","last_undegraded":"2024-05-20T18:35:32.577120+0000","last_fullsized":"2024-05-20T18:35:32.577120+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:00:13.034312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.418334+0000","last_change":"2024-05-20T18:35:31.413854+0000","last_active":"2024-05-20T18:35:32.418334+0000","last_peered":"2024-05-20T18:35:32.418334+0000","last_clean":"2024-05-20T18:35:32.418334+0000","last_became_active":"2024-05-20T18:35:31.413104+0000","last_became_peered":"2024-05-20T18:35:31.413104+0000","last_unstale":"2024-05-20T18:35:32.418334+0000","last_undegraded":"2024-05-20T18:35:32.418334+0000","last_fullsized":"2024-05-20T18:35:32.418334+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:46:10.206099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.237663+0000","last_change":"2024-05-20T18:35:30.412485+0000","last_active":"2024-05-20T18:35:33.237663+0000","last_peered":"2024-05-20T18:35:33.237663+0000","last_clean":"2024-05-20T18:35:33.237663+0000","last_became_active":"2024-05-20T18:35:30.412162+0000","last_became_peered":"2024-05-20T18:35:30.412162+0000","last_unstale":"2024-05-20T18:35:33.237663+0000","last_undegraded":"2024-05-20T18:35:33.237663+0000","last_fullsized":"2024-05-20T18:35:33.237663+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:28:15.030130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.422066+0000","last_change":"2024-05-20T18:35:31.429341+0000","last_active":"2024-05-20T18:35:32.422066+0000","last_peered":"2024-05-20T18:35:32.422066+0000","last_clean":"2024-05-20T18:35:32.422066+0000","last_became_active":"2024-05-20T18:35:31.428508+0000","last_became_peered":"2024-05-20T18:35:31.428508+0000","last_unstale":"2024-05-20T18:35:32.422066+0000","last_undegraded":"2024-05-20T18:35:32.422066+0000","last_fullsized":"2024-05-20T18:35:32.422066+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:46:50.437839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.420309+0000","last_change":"2024-05-20T18:35:30.409374+0000","last_active":"2024-05-20T18:35:32.420309+0000","last_peered":"2024-05-20T18:35:32.420309+0000","last_clean":"2024-05-20T18:35:32.420309+0000","last_became_active":"2024-05-20T18:35:30.408970+0000","last_became_peered":"2024-05-20T18:35:30.408970+0000","last_unstale":"2024-05-20T18:35:32.420309+0000","last_undegraded":"2024-05-20T18:35:32.420309+0000","last_fullsized":"2024-05-20T18:35:32.420309+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:08:29.264517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238790+0000","last_change":"2024-05-20T18:35:31.416735+0000","last_active":"2024-05-20T18:35:33.238790+0000","last_peered":"2024-05-20T18:35:33.238790+0000","last_clean":"2024-05-20T18:35:33.238790+0000","last_became_active":"2024-05-20T18:35:31.416413+0000","last_became_peered":"2024-05-20T18:35:31.416413+0000","last_unstale":"2024-05-20T18:35:33.238790+0000","last_undegraded":"2024-05-20T18:35:33.238790+0000","last_fullsized":"2024-05-20T18:35:33.238790+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:41:04.392769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238378+0000","last_change":"2024-05-20T18:35:30.406064+0000","last_active":"2024-05-20T18:35:33.238378+0000","last_peered":"2024-05-20T18:35:33.238378+0000","last_clean":"2024-05-20T18:35:33.238378+0000","last_became_active":"2024-05-20T18:35:30.405710+0000","last_became_peered":"2024-05-20T18:35:30.405710+0000","last_unstale":"2024-05-20T18:35:33.238378+0000","last_undegraded":"2024-05-20T18:35:33.238378+0000","last_fullsized":"2024-05-20T18:35:33.238378+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:19:06.313009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.417559+0000","last_change":"2024-05-20T18:35:31.414458+0000","last_active":"2024-05-20T18:35:32.417559+0000","last_peered":"2024-05-20T18:35:32.417559+0000","last_clean":"2024-05-20T18:35:32.417559+0000","last_became_active":"2024-05-20T18:35:31.413331+0000","last_became_peered":"2024-05-20T18:35:31.413331+0000","last_unstale":"2024-05-20T18:35:32.417559+0000","last_undegraded":"2024-05-20T18:35:32.417559+0000","last_fullsized":"2024-05-20T18:35:32.417559+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:37:38.515946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.421697+0000","last_change":"2024-05-20T18:35:30.409880+0000","last_active":"2024-05-20T18:35:32.421697+0000","last_peered":"2024-05-20T18:35:32.421697+0000","last_clean":"2024-05-20T18:35:32.421697+0000","last_became_active":"2024-05-20T18:35:30.409517+0000","last_became_peered":"2024-05-20T18:35:30.409517+0000","last_unstale":"2024-05-20T18:35:32.421697+0000","last_undegraded":"2024-05-20T18:35:32.421697+0000","last_fullsized":"2024-05-20T18:35:32.421697+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:14:56.573180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.237994+0000","last_change":"2024-05-20T18:35:31.415635+0000","last_active":"2024-05-20T18:35:33.237994+0000","last_peered":"2024-05-20T18:35:33.237994+0000","last_clean":"2024-05-20T18:35:33.237994+0000","last_became_active":"2024-05-20T18:35:31.415324+0000","last_became_peered":"2024-05-20T18:35:31.415324+0000","last_unstale":"2024-05-20T18:35:33.237994+0000","last_undegraded":"2024-05-20T18:35:33.237994+0000","last_fullsized":"2024-05-20T18:35:33.237994+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:06:54.343178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.417642+0000","last_change":"2024-05-20T18:35:30.403374+0000","last_active":"2024-05-20T18:35:32.417642+0000","last_peered":"2024-05-20T18:35:32.417642+0000","last_clean":"2024-05-20T18:35:32.417642+0000","last_became_active":"2024-05-20T18:35:30.403012+0000","last_became_peered":"2024-05-20T18:35:30.403012+0000","last_unstale":"2024-05-20T18:35:32.417642+0000","last_undegraded":"2024-05-20T18:35:32.417642+0000","last_fullsized":"2024-05-20T18:35:32.417642+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:29:46.967888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.574928+0000","last_change":"2024-05-20T18:35:31.424248+0000","last_active":"2024-05-20T18:35:32.574928+0000","last_peered":"2024-05-20T18:35:32.574928+0000","last_clean":"2024-05-20T18:35:32.574928+0000","last_became_active":"2024-05-20T18:35:31.423928+0000","last_became_peered":"2024-05-20T18:35:31.423928+0000","last_unstale":"2024-05-20T18:35:32.574928+0000","last_undegraded":"2024-05-20T18:35:32.574928+0000","last_fullsized":"2024-05-20T18:35:32.574928+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:16:52.263863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"104'3","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.577117+0000","last_change":"2024-05-20T18:35:30.410506+0000","last_active":"2024-05-20T18:35:32.577117+0000","last_peered":"2024-05-20T18:35:32.577117+0000","last_clean":"2024-05-20T18:35:32.577117+0000","last_became_active":"2024-05-20T18:35:30.410168+0000","last_became_peered":"2024-05-20T18:35:30.410168+0000","last_unstale":"2024-05-20T18:35:32.577117+0000","last_undegraded":"2024-05-20T18:35:32.577117+0000","last_fullsized":"2024-05-20T18:35:32.577117+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:51:24.485172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238307+0000","last_change":"2024-05-20T18:35:31.416784+0000","last_active":"2024-05-20T18:35:33.238307+0000","last_peered":"2024-05-20T18:35:33.238307+0000","last_clean":"2024-05-20T18:35:33.238307+0000","last_became_active":"2024-05-20T18:35:31.416462+0000","last_became_peered":"2024-05-20T18:35:31.416462+0000","last_unstale":"2024-05-20T18:35:33.238307+0000","last_undegraded":"2024-05-20T18:35:33.238307+0000","last_fullsized":"2024-05-20T18:35:33.238307+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:02:05.636153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"104'6","reported_seq":20,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.421928+0000","last_change":"2024-05-20T18:35:30.412065+0000","last_active":"2024-05-20T18:35:32.421928+0000","last_peered":"2024-05-20T18:35:32.421928+0000","last_clean":"2024-05-20T18:35:32.421928+0000","last_became_active":"2024-05-20T18:35:30.410006+0000","last_became_peered":"2024-05-20T18:35:30.410006+0000","last_unstale":"2024-05-20T18:35:32.421928+0000","last_undegraded":"2024-05-20T18:35:32.421928+0000","last_fullsized":"2024-05-20T18:35:32.421928+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:08:57.415190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.237822+0000","last_change":"2024-05-20T18:35:31.435493+0000","last_active":"2024-05-20T18:35:33.237822+0000","last_peered":"2024-05-20T18:35:33.237822+0000","last_clean":"2024-05-20T18:35:33.237822+0000","last_became_active":"2024-05-20T18:35:31.435145+0000","last_became_peered":"2024-05-20T18:35:31.435145+0000","last_unstale":"2024-05-20T18:35:33.237822+0000","last_undegraded":"2024-05-20T18:35:33.237822+0000","last_fullsized":"2024-05-20T18:35:33.237822+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:19:35.163308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238285+0000","last_change":"2024-05-20T18:35:30.406644+0000","last_active":"2024-05-20T18:35:33.238285+0000","last_peered":"2024-05-20T18:35:33.238285+0000","last_clean":"2024-05-20T18:35:33.238285+0000","last_became_active":"2024-05-20T18:35:30.406155+0000","last_became_peered":"2024-05-20T18:35:30.406155+0000","last_unstale":"2024-05-20T18:35:33.238285+0000","last_undegraded":"2024-05-20T18:35:33.238285+0000","last_fullsized":"2024-05-20T18:35:33.238285+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:59:20.346125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.421348+0000","last_change":"2024-05-20T18:35:31.423927+0000","last_active":"2024-05-20T18:35:32.421348+0000","last_peered":"2024-05-20T18:35:32.421348+0000","last_clean":"2024-05-20T18:35:32.421348+0000","last_became_active":"2024-05-20T18:35:31.423640+0000","last_became_peered":"2024-05-20T18:35:31.423640+0000","last_unstale":"2024-05-20T18:35:32.421348+0000","last_undegraded":"2024-05-20T18:35:32.421348+0000","last_fullsized":"2024-05-20T18:35:32.421348+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:26:30.218312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238942+0000","last_change":"2024-05-20T18:35:30.404725+0000","last_active":"2024-05-20T18:35:33.238942+0000","last_peered":"2024-05-20T18:35:33.238942+0000","last_clean":"2024-05-20T18:35:33.238942+0000","last_became_active":"2024-05-20T18:35:30.404404+0000","last_became_peered":"2024-05-20T18:35:30.404404+0000","last_unstale":"2024-05-20T18:35:33.238942+0000","last_undegraded":"2024-05-20T18:35:33.238942+0000","last_fullsized":"2024-05-20T18:35:33.238942+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:20:38.890394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.575110+0000","last_change":"2024-05-20T18:35:31.426565+0000","last_active":"2024-05-20T18:35:32.575110+0000","last_peered":"2024-05-20T18:35:32.575110+0000","last_clean":"2024-05-20T18:35:32.575110+0000","last_became_active":"2024-05-20T18:35:31.426223+0000","last_became_peered":"2024-05-20T18:35:31.426223+0000","last_unstale":"2024-05-20T18:35:32.575110+0000","last_undegraded":"2024-05-20T18:35:32.575110+0000","last_fullsized":"2024-05-20T18:35:32.575110+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:33:58.467366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.574851+0000","last_change":"2024-05-20T18:35:30.414408+0000","last_active":"2024-05-20T18:35:32.574851+0000","last_peered":"2024-05-20T18:35:32.574851+0000","last_clean":"2024-05-20T18:35:32.574851+0000","last_became_active":"2024-05-20T18:35:30.414025+0000","last_became_peered":"2024-05-20T18:35:30.414025+0000","last_unstale":"2024-05-20T18:35:32.574851+0000","last_undegraded":"2024-05-20T18:35:32.574851+0000","last_fullsized":"2024-05-20T18:35:32.574851+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:35:47.133654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.418171+0000","last_change":"2024-05-20T18:35:31.430837+0000","last_active":"2024-05-20T18:35:32.418171+0000","last_peered":"2024-05-20T18:35:32.418171+0000","last_clean":"2024-05-20T18:35:32.418171+0000","last_became_active":"2024-05-20T18:35:31.430520+0000","last_became_peered":"2024-05-20T18:35:31.430520+0000","last_unstale":"2024-05-20T18:35:32.418171+0000","last_undegraded":"2024-05-20T18:35:32.418171+0000","last_fullsized":"2024-05-20T18:35:32.418171+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:58:09.429427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"104'8","reported_seq":22,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.420126+0000","last_change":"2024-05-20T18:35:30.408547+0000","last_active":"2024-05-20T18:35:32.420126+0000","last_peered":"2024-05-20T18:35:32.420126+0000","last_clean":"2024-05-20T18:35:32.420126+0000","last_became_active":"2024-05-20T18:35:30.408205+0000","last_became_peered":"2024-05-20T18:35:30.408205+0000","last_unstale":"2024-05-20T18:35:32.420126+0000","last_undegraded":"2024-05-20T18:35:32.420126+0000","last_fullsized":"2024-05-20T18:35:32.420126+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:31:19.619870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.575221+0000","last_change":"2024-05-20T18:35:31.416565+0000","last_active":"2024-05-20T18:35:32.575221+0000","last_peered":"2024-05-20T18:35:32.575221+0000","last_clean":"2024-05-20T18:35:32.575221+0000","last_became_active":"2024-05-20T18:35:31.416255+0000","last_became_peered":"2024-05-20T18:35:31.416255+0000","last_unstale":"2024-05-20T18:35:32.575221+0000","last_undegraded":"2024-05-20T18:35:32.575221+0000","last_fullsized":"2024-05-20T18:35:32.575221+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:10:49.033115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.421415+0000","last_change":"2024-05-20T18:35:30.408259+0000","last_active":"2024-05-20T18:35:32.421415+0000","last_peered":"2024-05-20T18:35:32.421415+0000","last_clean":"2024-05-20T18:35:32.421415+0000","last_became_active":"2024-05-20T18:35:30.407911+0000","last_became_peered":"2024-05-20T18:35:30.407911+0000","last_unstale":"2024-05-20T18:35:32.421415+0000","last_undegraded":"2024-05-20T18:35:32.421415+0000","last_fullsized":"2024-05-20T18:35:32.421415+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:59:07.724149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.421174+0000","last_change":"2024-05-20T18:35:31.415197+0000","last_active":"2024-05-20T18:35:32.421174+0000","last_peered":"2024-05-20T18:35:32.421174+0000","last_clean":"2024-05-20T18:35:32.421174+0000","last_became_active":"2024-05-20T18:35:31.414725+0000","last_became_peered":"2024-05-20T18:35:31.414725+0000","last_unstale":"2024-05-20T18:35:32.421174+0000","last_undegraded":"2024-05-20T18:35:32.421174+0000","last_fullsized":"2024-05-20T18:35:32.421174+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:19:35.163308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.577260+0000","last_change":"2024-05-20T18:35:30.412088+0000","last_active":"2024-05-20T18:35:32.577260+0000","last_peered":"2024-05-20T18:35:32.577260+0000","last_clean":"2024-05-20T18:35:32.577260+0000","last_became_active":"2024-05-20T18:35:30.411708+0000","last_became_peered":"2024-05-20T18:35:30.411708+0000","last_unstale":"2024-05-20T18:35:32.577260+0000","last_undegraded":"2024-05-20T18:35:32.577260+0000","last_fullsized":"2024-05-20T18:35:32.577260+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:47.862345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238110+0000","last_change":"2024-05-20T18:35:31.420523+0000","last_active":"2024-05-20T18:35:33.238110+0000","last_peered":"2024-05-20T18:35:33.238110+0000","last_clean":"2024-05-20T18:35:33.238110+0000","last_became_active":"2024-05-20T18:35:31.420169+0000","last_became_peered":"2024-05-20T18:35:31.420169+0000","last_unstale":"2024-05-20T18:35:33.238110+0000","last_undegraded":"2024-05-20T18:35:33.238110+0000","last_fullsized":"2024-05-20T18:35:33.238110+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:35:00.678467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.237979+0000","last_change":"2024-05-20T18:35:30.406195+0000","last_active":"2024-05-20T18:35:33.237979+0000","last_peered":"2024-05-20T18:35:33.237979+0000","last_clean":"2024-05-20T18:35:33.237979+0000","last_became_active":"2024-05-20T18:35:30.405856+0000","last_became_peered":"2024-05-20T18:35:30.405856+0000","last_unstale":"2024-05-20T18:35:33.237979+0000","last_undegraded":"2024-05-20T18:35:33.237979+0000","last_fullsized":"2024-05-20T18:35:33.237979+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:08:29.264517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.418073+0000","last_change":"2024-05-20T18:35:31.425634+0000","last_active":"2024-05-20T18:35:32.418073+0000","last_peered":"2024-05-20T18:35:32.418073+0000","last_clean":"2024-05-20T18:35:32.418073+0000","last_became_active":"2024-05-20T18:35:31.425326+0000","last_became_peered":"2024-05-20T18:35:31.425326+0000","last_unstale":"2024-05-20T18:35:32.418073+0000","last_undegraded":"2024-05-20T18:35:32.418073+0000","last_fullsized":"2024-05-20T18:35:32.418073+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:59:31.379489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.575311+0000","last_change":"2024-05-20T18:35:30.411351+0000","last_active":"2024-05-20T18:35:32.575311+0000","last_peered":"2024-05-20T18:35:32.575311+0000","last_clean":"2024-05-20T18:35:32.575311+0000","last_became_active":"2024-05-20T18:35:30.410938+0000","last_became_peered":"2024-05-20T18:35:30.410938+0000","last_unstale":"2024-05-20T18:35:32.575311+0000","last_undegraded":"2024-05-20T18:35:32.575311+0000","last_fullsized":"2024-05-20T18:35:32.575311+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:41:27.932466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.418194+0000","last_change":"2024-05-20T18:35:31.418822+0000","last_active":"2024-05-20T18:35:32.418194+0000","last_peered":"2024-05-20T18:35:32.418194+0000","last_clean":"2024-05-20T18:35:32.418194+0000","last_became_active":"2024-05-20T18:35:31.418530+0000","last_became_peered":"2024-05-20T18:35:31.418530+0000","last_unstale":"2024-05-20T18:35:32.418194+0000","last_undegraded":"2024-05-20T18:35:32.418194+0000","last_fullsized":"2024-05-20T18:35:32.418194+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:31:51.789823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.574940+0000","last_change":"2024-05-20T18:35:30.410136+0000","last_active":"2024-05-20T18:35:32.574940+0000","last_peered":"2024-05-20T18:35:32.574940+0000","last_clean":"2024-05-20T18:35:32.574940+0000","last_became_active":"2024-05-20T18:35:30.409778+0000","last_became_peered":"2024-05-20T18:35:30.409778+0000","last_unstale":"2024-05-20T18:35:32.574940+0000","last_undegraded":"2024-05-20T18:35:32.574940+0000","last_fullsized":"2024-05-20T18:35:32.574940+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:37:49.781033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.239028+0000","last_change":"2024-05-20T18:35:31.420081+0000","last_active":"2024-05-20T18:35:33.239028+0000","last_peered":"2024-05-20T18:35:33.239028+0000","last_clean":"2024-05-20T18:35:33.239028+0000","last_became_active":"2024-05-20T18:35:31.419505+0000","last_became_peered":"2024-05-20T18:35:31.419505+0000","last_unstale":"2024-05-20T18:35:33.239028+0000","last_undegraded":"2024-05-20T18:35:33.239028+0000","last_fullsized":"2024-05-20T18:35:33.239028+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:50:25.744663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.420570+0000","last_change":"2024-05-20T18:35:30.407327+0000","last_active":"2024-05-20T18:35:32.420570+0000","last_peered":"2024-05-20T18:35:32.420570+0000","last_clean":"2024-05-20T18:35:32.420570+0000","last_became_active":"2024-05-20T18:35:30.406994+0000","last_became_peered":"2024-05-20T18:35:30.406994+0000","last_unstale":"2024-05-20T18:35:32.420570+0000","last_undegraded":"2024-05-20T18:35:32.420570+0000","last_fullsized":"2024-05-20T18:35:32.420570+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:59:53.740452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.575183+0000","last_change":"2024-05-20T18:35:31.418822+0000","last_active":"2024-05-20T18:35:32.575183+0000","last_peered":"2024-05-20T18:35:32.575183+0000","last_clean":"2024-05-20T18:35:32.575183+0000","last_became_active":"2024-05-20T18:35:31.418512+0000","last_became_peered":"2024-05-20T18:35:31.418512+0000","last_unstale":"2024-05-20T18:35:32.575183+0000","last_undegraded":"2024-05-20T18:35:32.575183+0000","last_fullsized":"2024-05-20T18:35:32.575183+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:22:06.783328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.575437+0000","last_change":"2024-05-20T18:35:30.404056+0000","last_active":"2024-05-20T18:35:32.575437+0000","last_peered":"2024-05-20T18:35:32.575437+0000","last_clean":"2024-05-20T18:35:32.575437+0000","last_became_active":"2024-05-20T18:35:30.403695+0000","last_became_peered":"2024-05-20T18:35:30.403695+0000","last_unstale":"2024-05-20T18:35:32.575437+0000","last_undegraded":"2024-05-20T18:35:32.575437+0000","last_fullsized":"2024-05-20T18:35:32.575437+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:10:39.186470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.417744+0000","last_change":"2024-05-20T18:35:31.428550+0000","last_active":"2024-05-20T18:35:32.417744+0000","last_peered":"2024-05-20T18:35:32.417744+0000","last_clean":"2024-05-20T18:35:32.417744+0000","last_became_active":"2024-05-20T18:35:31.423158+0000","last_became_peered":"2024-05-20T18:35:31.423158+0000","last_unstale":"2024-05-20T18:35:32.417744+0000","last_undegraded":"2024-05-20T18:35:32.417744+0000","last_fullsized":"2024-05-20T18:35:32.417744+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:58:57.880090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238687+0000","last_change":"2024-05-20T18:35:30.412633+0000","last_active":"2024-05-20T18:35:33.238687+0000","last_peered":"2024-05-20T18:35:33.238687+0000","last_clean":"2024-05-20T18:35:33.238687+0000","last_became_active":"2024-05-20T18:35:30.412307+0000","last_became_peered":"2024-05-20T18:35:30.412307+0000","last_unstale":"2024-05-20T18:35:33.238687+0000","last_undegraded":"2024-05-20T18:35:33.238687+0000","last_fullsized":"2024-05-20T18:35:33.238687+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:59:53.740452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.420246+0000","last_change":"2024-05-20T18:35:31.425567+0000","last_active":"2024-05-20T18:35:32.420246+0000","last_peered":"2024-05-20T18:35:32.420246+0000","last_clean":"2024-05-20T18:35:32.420246+0000","last_became_active":"2024-05-20T18:35:31.425213+0000","last_became_peered":"2024-05-20T18:35:31.425213+0000","last_unstale":"2024-05-20T18:35:32.420246+0000","last_undegraded":"2024-05-20T18:35:32.420246+0000","last_fullsized":"2024-05-20T18:35:32.420246+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:19:35.163308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"105'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.417740+0000","last_change":"2024-05-20T18:35:30.411059+0000","last_active":"2024-05-20T18:35:32.417740+0000","last_peered":"2024-05-20T18:35:32.417740+0000","last_clean":"2024-05-20T18:35:32.417740+0000","last_became_active":"2024-05-20T18:35:30.410688+0000","last_became_peered":"2024-05-20T18:35:30.410688+0000","last_unstale":"2024-05-20T18:35:32.417740+0000","last_undegraded":"2024-05-20T18:35:32.417740+0000","last_fullsized":"2024-05-20T18:35:32.417740+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:32:03.221999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238766+0000","last_change":"2024-05-20T18:35:31.416928+0000","last_active":"2024-05-20T18:35:33.238766+0000","last_peered":"2024-05-20T18:35:33.238766+0000","last_clean":"2024-05-20T18:35:33.238766+0000","last_became_active":"2024-05-20T18:35:31.416613+0000","last_became_peered":"2024-05-20T18:35:31.416613+0000","last_unstale":"2024-05-20T18:35:33.238766+0000","last_undegraded":"2024-05-20T18:35:33.238766+0000","last_fullsized":"2024-05-20T18:35:33.238766+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:25:42.090152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.421142+0000","last_change":"2024-05-20T18:35:30.406122+0000","last_active":"2024-05-20T18:35:32.421142+0000","last_peered":"2024-05-20T18:35:32.421142+0000","last_clean":"2024-05-20T18:35:32.421142+0000","last_became_active":"2024-05-20T18:35:30.405784+0000","last_became_peered":"2024-05-20T18:35:30.405784+0000","last_unstale":"2024-05-20T18:35:32.421142+0000","last_undegraded":"2024-05-20T18:35:32.421142+0000","last_fullsized":"2024-05-20T18:35:32.421142+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:24:43.933783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.237873+0000","last_change":"2024-05-20T18:35:31.430996+0000","last_active":"2024-05-20T18:35:33.237873+0000","last_peered":"2024-05-20T18:35:33.237873+0000","last_clean":"2024-05-20T18:35:33.237873+0000","last_became_active":"2024-05-20T18:35:31.430691+0000","last_became_peered":"2024-05-20T18:35:31.430691+0000","last_unstale":"2024-05-20T18:35:33.237873+0000","last_undegraded":"2024-05-20T18:35:33.237873+0000","last_fullsized":"2024-05-20T18:35:33.237873+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:26:30.218312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238281+0000","last_change":"2024-05-20T18:35:30.400456+0000","last_active":"2024-05-20T18:35:33.238281+0000","last_peered":"2024-05-20T18:35:33.238281+0000","last_clean":"2024-05-20T18:35:33.238281+0000","last_became_active":"2024-05-20T18:35:30.399296+0000","last_became_peered":"2024-05-20T18:35:30.399296+0000","last_unstale":"2024-05-20T18:35:33.238281+0000","last_undegraded":"2024-05-20T18:35:33.238281+0000","last_fullsized":"2024-05-20T18:35:33.238281+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:19:27.219675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.575002+0000","last_change":"2024-05-20T18:35:31.423203+0000","last_active":"2024-05-20T18:35:32.575002+0000","last_peered":"2024-05-20T18:35:32.575002+0000","last_clean":"2024-05-20T18:35:32.575002+0000","last_became_active":"2024-05-20T18:35:31.422831+0000","last_became_peered":"2024-05-20T18:35:31.422831+0000","last_unstale":"2024-05-20T18:35:32.575002+0000","last_undegraded":"2024-05-20T18:35:32.575002+0000","last_fullsized":"2024-05-20T18:35:32.575002+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:57:23.130788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.420646+0000","last_change":"2024-05-20T18:35:30.408084+0000","last_active":"2024-05-20T18:35:32.420646+0000","last_peered":"2024-05-20T18:35:32.420646+0000","last_clean":"2024-05-20T18:35:32.420646+0000","last_became_active":"2024-05-20T18:35:30.407701+0000","last_became_peered":"2024-05-20T18:35:30.407701+0000","last_unstale":"2024-05-20T18:35:32.420646+0000","last_undegraded":"2024-05-20T18:35:32.420646+0000","last_fullsized":"2024-05-20T18:35:32.420646+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:22:04.089340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.575027+0000","last_change":"2024-05-20T18:35:31.422299+0000","last_active":"2024-05-20T18:35:32.575027+0000","last_peered":"2024-05-20T18:35:32.575027+0000","last_clean":"2024-05-20T18:35:32.575027+0000","last_became_active":"2024-05-20T18:35:31.421946+0000","last_became_peered":"2024-05-20T18:35:31.421946+0000","last_unstale":"2024-05-20T18:35:32.575027+0000","last_undegraded":"2024-05-20T18:35:32.575027+0000","last_fullsized":"2024-05-20T18:35:32.575027+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:59:54.966570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.575072+0000","last_change":"2024-05-20T18:35:30.414600+0000","last_active":"2024-05-20T18:35:32.575072+0000","last_peered":"2024-05-20T18:35:32.575072+0000","last_clean":"2024-05-20T18:35:32.575072+0000","last_became_active":"2024-05-20T18:35:30.414218+0000","last_became_peered":"2024-05-20T18:35:30.414218+0000","last_unstale":"2024-05-20T18:35:32.575072+0000","last_undegraded":"2024-05-20T18:35:32.575072+0000","last_fullsized":"2024-05-20T18:35:32.575072+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:56:58.143092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.577332+0000","last_change":"2024-05-20T18:35:31.429051+0000","last_active":"2024-05-20T18:35:32.577332+0000","last_peered":"2024-05-20T18:35:32.577332+0000","last_clean":"2024-05-20T18:35:32.577332+0000","last_became_active":"2024-05-20T18:35:31.428724+0000","last_became_peered":"2024-05-20T18:35:31.428724+0000","last_unstale":"2024-05-20T18:35:32.577332+0000","last_undegraded":"2024-05-20T18:35:32.577332+0000","last_fullsized":"2024-05-20T18:35:32.577332+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:45:44.257378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.420685+0000","last_change":"2024-05-20T18:35:30.406555+0000","last_active":"2024-05-20T18:35:32.420685+0000","last_peered":"2024-05-20T18:35:32.420685+0000","last_clean":"2024-05-20T18:35:32.420685+0000","last_became_active":"2024-05-20T18:35:30.405500+0000","last_became_peered":"2024-05-20T18:35:30.405500+0000","last_unstale":"2024-05-20T18:35:32.420685+0000","last_undegraded":"2024-05-20T18:35:32.420685+0000","last_fullsized":"2024-05-20T18:35:32.420685+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:59:20.346125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.238265+0000","last_change":"2024-05-20T18:35:31.430305+0000","last_active":"2024-05-20T18:35:33.238265+0000","last_peered":"2024-05-20T18:35:33.238265+0000","last_clean":"2024-05-20T18:35:33.238265+0000","last_became_active":"2024-05-20T18:35:31.429852+0000","last_became_peered":"2024-05-20T18:35:31.429852+0000","last_unstale":"2024-05-20T18:35:33.238265+0000","last_undegraded":"2024-05-20T18:35:33.238265+0000","last_fullsized":"2024-05-20T18:35:33.238265+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:48.866975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.575542+0000","last_change":"2024-05-20T18:35:30.412896+0000","last_active":"2024-05-20T18:35:32.575542+0000","last_peered":"2024-05-20T18:35:32.575542+0000","last_clean":"2024-05-20T18:35:32.575542+0000","last_became_active":"2024-05-20T18:35:30.412388+0000","last_became_peered":"2024-05-20T18:35:30.412388+0000","last_unstale":"2024-05-20T18:35:32.575542+0000","last_undegraded":"2024-05-20T18:35:32.575542+0000","last_fullsized":"2024-05-20T18:35:32.575542+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:35:42.001777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:33.237708+0000","last_change":"2024-05-20T18:35:31.434643+0000","last_active":"2024-05-20T18:35:33.237708+0000","last_peered":"2024-05-20T18:35:33.237708+0000","last_clean":"2024-05-20T18:35:33.237708+0000","last_became_active":"2024-05-20T18:35:31.434223+0000","last_became_peered":"2024-05-20T18:35:31.434223+0000","last_unstale":"2024-05-20T18:35:33.237708+0000","last_undegraded":"2024-05-20T18:35:33.237708+0000","last_fullsized":"2024-05-20T18:35:33.237708+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:30.384431+0000","last_clean_scrub_stamp":"2024-05-20T18:35:30.384431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:16:22.557898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.574958+0000","last_change":"2024-05-20T18:35:30.411370+0000","last_active":"2024-05-20T18:35:32.574958+0000","last_peered":"2024-05-20T18:35:32.574958+0000","last_clean":"2024-05-20T18:35:32.574958+0000","last_became_active":"2024-05-20T18:35:30.411022+0000","last_became_peered":"2024-05-20T18:35:30.411022+0000","last_unstale":"2024-05-20T18:35:32.574958+0000","last_undegraded":"2024-05-20T18:35:32.574958+0000","last_fullsized":"2024-05-20T18:35:32.574958+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:35:29.379801+0000","last_clean_scrub_stamp":"2024-05-20T18:35:29.379801+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:49:00.568944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":326,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-05-20T18:35:32.576840+0000","last_change":"2024-05-20T18:27:03.765694+0000","last_active":"2024-05-20T18:35:32.576840+0000","last_peered":"2024-05-20T18:35:32.576840+0000","last_clean":"2024-05-20T18:35:32.576840+0000","last_became_active":"2024-05-20T18:27:03.764695+0000","last_became_peered":"2024-05-20T18:27:03.764695+0000","last_unstale":"2024-05-20T18:35:32.576840+0000","last_undegraded":"2024-05-20T18:35:32.576840+0000","last_fullsized":"2024-05-20T18:35:32.576840+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_clean_scrub_stamp":"2024-05-20T18:27:02.750853+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:30:04.407487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574955,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33044,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338796,"statfs":{"total":96636764160,"available":96602927104,"internally_reserved":0,"allocated":1904640,"data_stored":1195051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574953,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1323008,"data_stored":606739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574953,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1302528,"data_stored":604523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574953,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1310720,"data_stored":604615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574955,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1306624,"data_stored":605105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574954,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1306624,"data_stored":604557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574955,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1306624,"data_stored":604557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574954,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33052,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338788,"statfs":{"total":96636764160,"available":96602918912,"internally_reserved":0,"allocated":1912832,"data_stored":1196593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T18:35:40.354 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-20T18:35:40.355 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-20T18:35:40.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.511+0000 7f2ad2dce640 1 Processor -- start 2024-05-20T18:35:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.511+0000 7f2ad2dce640 1 -- start start 2024-05-20T18:35:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.513+0000 7f2ad2dce640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2acc163a10 0x7f2acc0d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.513+0000 7f2ad2dce640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2acc0b4010 0x7f2acc0cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.513+0000 7f2ad2dce640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2acc0d0370 con 0x7f2acc0d14b0 2024-05-20T18:35:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.513+0000 7f2ad2dce640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f2acc0d3ca0 con 0x7f2acc0b4010 2024-05-20T18:35:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.513+0000 7f2ad2dce640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2acc0d0650 con 0x7f2acc163a10 2024-05-20T18:35:40.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.513+0000 7f2acbfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2acc0b4010 0x7f2acc0cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:40.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.513+0000 7f2ad136b640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2acc0d14b0 0x7f2acc0d18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46728/0 (socket says 172.21.15.5:46728) 2024-05-20T18:35:40.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.513+0000 7f2ad136b640 1 -- 172.21.15.5:0/3177015704 learned_addr learned my addr 172.21.15.5:0/3177015704 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:40.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.513+0000 7f2ad0b6a640 1 --2- 172.21.15.5:0/3177015704 >> v2:172.21.15.63:3300/0 conn(0x7f2acc163a10 0x7f2acc0d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:40.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.513+0000 7f2acbfff640 1 -- 172.21.15.5:0/3177015704 >> v2:172.21.15.63:3300/0 conn(0x7f2acc163a10 msgr2=0x7f2acc0d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:40.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.513+0000 7f2acbfff640 1 --2- 172.21.15.5:0/3177015704 >> v2:172.21.15.63:3300/0 conn(0x7f2acc163a10 0x7f2acc0d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:40.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.513+0000 7f2acbfff640 1 -- 172.21.15.5:0/3177015704 >> v1:172.21.15.5:6789/0 conn(0x7f2acc0d14b0 legacy=0x7f2acc0d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:40.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.513+0000 7f2acbfff640 1 -- 172.21.15.5:0/3177015704 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2acc0d3650 con 0x7f2acc0b4010 2024-05-20T18:35:40.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.514+0000 7f2acbfff640 1 --2- 172.21.15.5:0/3177015704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2acc0b4010 0x7f2acc0cfb00 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f2acc0cf1c0 tx=0x7f2ac002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4f13b6a5f2b697 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:40.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.514+0000 7f2acb7fe640 1 -- 172.21.15.5:0/3177015704 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0037070 con 0x7f2acc0b4010 2024-05-20T18:35:40.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.514+0000 7f2acb7fe640 1 -- 172.21.15.5:0/3177015704 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac0032440 con 0x7f2acc0b4010 2024-05-20T18:35:40.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.514+0000 7f2acb7fe640 1 -- 172.21.15.5:0/3177015704 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0032810 con 0x7f2acc0b4010 2024-05-20T18:35:40.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.515+0000 7f2ad2dce640 1 -- 172.21.15.5:0/3177015704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2acc0b4010 msgr2=0x7f2acc0cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:40.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.515+0000 7f2ad2dce640 1 --2- 172.21.15.5:0/3177015704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2acc0b4010 0x7f2acc0cfb00 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f2acc0cf1c0 tx=0x7f2ac002a1a0 comp rx=0 tx=0).stop 2024-05-20T18:35:40.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.515+0000 7f2ad2dce640 1 -- 172.21.15.5:0/3177015704 shutdown_connections 2024-05-20T18:35:40.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.515+0000 7f2ad2dce640 1 --2- 172.21.15.5:0/3177015704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2acc0b4010 0x7f2acc0cfb00 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:40.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.515+0000 7f2ad2dce640 1 --2- 172.21.15.5:0/3177015704 >> v2:172.21.15.63:3300/0 conn(0x7f2acc163a10 0x7f2acc0d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:40.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.515+0000 7f2ad2dce640 1 -- 172.21.15.5:0/3177015704 >> 172.21.15.5:0/3177015704 conn(0x7f2acc06d240 msgr2=0x7f2acc06d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:40.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.515+0000 7f2ad2dce640 1 -- 172.21.15.5:0/3177015704 shutdown_connections 2024-05-20T18:35:40.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.515+0000 7f2ad2dce640 1 -- 172.21.15.5:0/3177015704 wait complete. 2024-05-20T18:35:40.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.517+0000 7f2ad2dce640 1 Processor -- start 2024-05-20T18:35:40.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.517+0000 7f2ad2dce640 1 -- start start 2024-05-20T18:35:40.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.518+0000 7f2ad2dce640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2acc0b4010 0x7f2acc158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:40.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.518+0000 7f2ad2dce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2acc0d14b0 0x7f2acc15a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:40.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.518+0000 7f2ad2dce640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2acc163a10 0x7f2acc0ce140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:40.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.518+0000 7f2ad2dce640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2acc0b8210 con 0x7f2acc0d14b0 2024-05-20T18:35:40.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.518+0000 7f2ad2dce640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2acc0d0040 con 0x7f2acc163a10 2024-05-20T18:35:40.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.518+0000 7f2ad2dce640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2acc0b8390 con 0x7f2acc0b4010 2024-05-20T18:35:40.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.518+0000 7f2acbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2acc0d14b0 0x7f2acc15a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:40.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.519+0000 7f2acbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2acc0d14b0 0x7f2acc15a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49686/0 (socket says 172.21.15.5:49686) 2024-05-20T18:35:40.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.519+0000 7f2acbfff640 1 -- 172.21.15.5:0/3096566384 learned_addr learned my addr 172.21.15.5:0/3096566384 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:40.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.519+0000 7f2ad136b640 1 --2- 172.21.15.5:0/3096566384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2acc163a10 0x7f2acc0ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:40.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.519+0000 7f2ad0b6a640 1 --2- 172.21.15.5:0/3096566384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2acc0b4010 0x7f2acc158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:40.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.519+0000 7f2acbfff640 1 -- 172.21.15.5:0/3096566384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2acc0b4010 msgr2=0x7f2acc158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:40.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.519+0000 7f2acbfff640 1 --2- 172.21.15.5:0/3096566384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2acc0b4010 0x7f2acc158f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:40.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.519+0000 7f2acbfff640 1 -- 172.21.15.5:0/3096566384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2acc163a10 msgr2=0x7f2acc0ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:40.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.519+0000 7f2acbfff640 1 --2- 172.21.15.5:0/3096566384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2acc163a10 0x7f2acc0ce140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:40.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.519+0000 7f2acbfff640 1 -- 172.21.15.5:0/3096566384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2acc157a50 con 0x7f2acc0d14b0 2024-05-20T18:35:40.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.519+0000 7f2ad0b6a640 1 --2- 172.21.15.5:0/3096566384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2acc0b4010 0x7f2acc158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:40.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.519+0000 7f2ad136b640 1 --2- 172.21.15.5:0/3096566384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2acc163a10 0x7f2acc0ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:40.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.519+0000 7f2acbfff640 1 --2- 172.21.15.5:0/3096566384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2acc0d14b0 0x7f2acc15a760 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0000c00 tx=0x7f2ac00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:40.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.519+0000 7f2ac97fa640 1 -- 172.21.15.5:0/3096566384 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0037040 con 0x7f2acc0d14b0 2024-05-20T18:35:40.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.519+0000 7f2ac97fa640 1 -- 172.21.15.5:0/3096566384 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac0042070 con 0x7f2acc0d14b0 2024-05-20T18:35:40.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.519+0000 7f2ad2dce640 1 -- 172.21.15.5:0/3096566384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2acc157ec0 con 0x7f2acc0d14b0 2024-05-20T18:35:40.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.519+0000 7f2ac97fa640 1 -- 172.21.15.5:0/3096566384 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0030bd0 con 0x7f2acc0d14b0 2024-05-20T18:35:40.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.519+0000 7f2ad2dce640 1 -- 172.21.15.5:0/3096566384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2acc158480 con 0x7f2acc0d14b0 2024-05-20T18:35:40.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.520+0000 7f2ad2dce640 1 -- 172.21.15.5:0/3096566384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2acc157c20 con 0x7f2acc0d14b0 2024-05-20T18:35:40.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.521+0000 7f2ac97fa640 1 -- 172.21.15.5:0/3096566384 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2ac0047460 con 0x7f2acc0d14b0 2024-05-20T18:35:40.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.521+0000 7f2ac97fa640 1 --2- 172.21.15.5:0/3096566384 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2ab806be60 0x7f2ab806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:40.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.522+0000 7f2ac97fa640 1 -- 172.21.15.5:0/3096566384 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7f2ac00ab9a0 con 0x7f2acc0d14b0 2024-05-20T18:35:40.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.522+0000 7f2ad0b6a640 1 --2- 172.21.15.5:0/3096566384 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2ab806be60 0x7f2ab806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:40.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.522+0000 7f2ad0b6a640 1 --2- 172.21.15.5:0/3096566384 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2ab806be60 0x7f2ab806e370 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2abc006fd0 tx=0x7f2abc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.526+0000 7f2ac97fa640 1 -- 172.21.15.5:0/3096566384 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2acc157c20 con 0x7f2acc0d14b0 2024-05-20T18:35:40.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.784+0000 7f2ad2dce640 1 -- 172.21.15.5:0/3096566384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f2acc075b00 con 0x7f2acc0d14b0 2024-05-20T18:35:40.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.787+0000 7f2ac97fa640 1 -- 172.21.15.5:0/3096566384 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7f2ac004d060 con 0x7f2acc0d14b0 2024-05-20T18:35:40.788 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2024-05-20T18:35:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDFlUtmevaaIhAATIwyXzr0vGT38ZmcMqR2gA== 2024-05-20T18:35:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: caps mds = "allow" 2024-05-20T18:35:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: caps mon = "allow r" 2024-05-20T18:35:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-20T18:35:40.788 INFO:teuthology.orchestra.run.smithi005.stderr:updated caps for client.0 2024-05-20T18:35:40.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.789+0000 7f2ad2dce640 1 -- 172.21.15.5:0/3096566384 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2ab806be60 msgr2=0x7f2ab806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:40.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.789+0000 7f2ad2dce640 1 --2- 172.21.15.5:0/3096566384 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2ab806be60 0x7f2ab806e370 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2abc006fd0 tx=0x7f2abc008040 comp rx=0 tx=0).stop 2024-05-20T18:35:40.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.789+0000 7f2ad2dce640 1 -- 172.21.15.5:0/3096566384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2acc0d14b0 msgr2=0x7f2acc15a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:40.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.789+0000 7f2ad2dce640 1 --2- 172.21.15.5:0/3096566384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2acc0d14b0 0x7f2acc15a760 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0000c00 tx=0x7f2ac00046c0 comp rx=0 tx=0).stop 2024-05-20T18:35:40.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.789+0000 7f2ad2dce640 1 -- 172.21.15.5:0/3096566384 shutdown_connections 2024-05-20T18:35:40.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.789+0000 7f2ad2dce640 1 --2- 172.21.15.5:0/3096566384 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2ab806be60 0x7f2ab806e370 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:40.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.789+0000 7f2ad2dce640 1 --2- 172.21.15.5:0/3096566384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2acc163a10 0x7f2acc0ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:40.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.789+0000 7f2ad2dce640 1 --2- 172.21.15.5:0/3096566384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2acc0d14b0 0x7f2acc15a760 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:40.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.789+0000 7f2ad2dce640 1 --2- 172.21.15.5:0/3096566384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2acc0b4010 0x7f2acc158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:40.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.789+0000 7f2ad2dce640 1 -- 172.21.15.5:0/3096566384 >> 172.21.15.5:0/3096566384 conn(0x7f2acc06d240 msgr2=0x7f2acc15f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:40.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.789+0000 7f2ad2dce640 1 -- 172.21.15.5:0/3096566384 shutdown_connections 2024-05-20T18:35:40.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.789+0000 7f2ad2dce640 1 -- 172.21.15.5:0/3096566384 wait complete. 2024-05-20T18:35:40.801 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:35:40.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.914+0000 7ff5c50c3640 1 Processor -- start 2024-05-20T18:35:40.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.914+0000 7ff5c50c3640 1 -- start start 2024-05-20T18:35:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.916+0000 7ff5c50c3640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff5c015f220 0x7ff5c00cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.916+0000 7ff5c50c3640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5c00cfd20 con 0x7ff5c00cd680 2024-05-20T18:35:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.916+0000 7ff5c50c3640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5c006d0f0 con 0x7ff5c00cf910 2024-05-20T18:35:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.916+0000 7ff5c50c3640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff5c00cda90 con 0x7ff5c015f220 2024-05-20T18:35:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.916+0000 7ff5bed9d640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff5c00cd680 0x7ff5c006c6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46734/0 (socket says 172.21.15.5:46734) 2024-05-20T18:35:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.916+0000 7ff5bed9d640 1 -- 172.21.15.5:0/2042676483 learned_addr learned my addr 172.21.15.5:0/2042676483 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.916+0000 7ff5bdd9b640 1 -- 172.21.15.5:0/2042676483 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4886853 0 0) 0x7ff5b0001110 con 0x7ff5c00cd680 2024-05-20T18:35:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.916+0000 7ff5bdd9b640 1 -- 172.21.15.5:0/2042676483 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5a8003520 con 0x7ff5c00cd680 2024-05-20T18:35:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.916+0000 7ff5be59c640 1 --2- 172.21.15.5:0/2042676483 >> v2:172.21.15.63:3300/0 conn(0x7ff5c015f220 0x7ff5c00cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.916+0000 7ff5bdd9b640 1 -- 172.21.15.5:0/2042676483 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 78924281 0 0) 0x7ff5a8003520 con 0x7ff5c00cd680 2024-05-20T18:35:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.916+0000 7ff5bdd9b640 1 -- 172.21.15.5:0/2042676483 >> v2:172.21.15.63:3300/0 conn(0x7ff5c015f220 msgr2=0x7ff5c00cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.916+0000 7ff5bdd9b640 1 --2- 172.21.15.5:0/2042676483 >> v2:172.21.15.63:3300/0 conn(0x7ff5c015f220 0x7ff5c00cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.916+0000 7ff5bdd9b640 1 -- 172.21.15.5:0/2042676483 >> v1:172.21.15.63:6789/0 conn(0x7ff5c00cf910 legacy=0x7ff5c00d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.916+0000 7ff5bdd9b640 1 -- 172.21.15.5:0/2042676483 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c006d3d0 con 0x7ff5c00cd680 2024-05-20T18:35:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.916+0000 7ff5bdd9b640 1 -- 172.21.15.5:0/2042676483 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7ff5b0001840 con 0x7ff5c00cd680 2024-05-20T18:35:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.916+0000 7ff5bdd9b640 1 -- 172.21.15.5:0/2042676483 >> v1:172.21.15.5:6789/0 conn(0x7ff5c00cd680 legacy=0x7ff5c006c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.917+0000 7ff5bdd9b640 1 --2- 172.21.15.5:0/2042676483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5a80039b0 0x7ff5a8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.917+0000 7ff5bdd9b640 1 --2- 172.21.15.5:0/2042676483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff5a8004800 0x7ff5a8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.917+0000 7ff5bdd9b640 1 --2- 172.21.15.5:0/2042676483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5a8005350 0x7ff5a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.917+0000 7ff5bdd9b640 1 -- 172.21.15.5:0/2042676483 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a800c250 con 0x7ff5a8005350 2024-05-20T18:35:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.917+0000 7ff5bdd9b640 1 -- 172.21.15.5:0/2042676483 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a800c400 con 0x7ff5a80039b0 2024-05-20T18:35:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.917+0000 7ff5bdd9b640 1 -- 172.21.15.5:0/2042676483 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff5a800c600 con 0x7ff5a8004800 2024-05-20T18:35:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.917+0000 7ff5bf59e640 1 --2- 172.21.15.5:0/2042676483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5a8005350 0x7ff5a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.917+0000 7ff5bed9d640 1 --2- 172.21.15.5:0/2042676483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5a80039b0 0x7ff5a8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.917+0000 7ff5be59c640 1 --2- 172.21.15.5:0/2042676483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff5a8004800 0x7ff5a8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.917+0000 7ff5bf59e640 1 -- 172.21.15.5:0/2042676483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff5a8004800 msgr2=0x7ff5a8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.917+0000 7ff5bf59e640 1 --2- 172.21.15.5:0/2042676483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff5a8004800 0x7ff5a8004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.917+0000 7ff5bf59e640 1 -- 172.21.15.5:0/2042676483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5a80039b0 msgr2=0x7ff5a8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.917+0000 7ff5bf59e640 1 --2- 172.21.15.5:0/2042676483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5a80039b0 0x7ff5a8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:40.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.917+0000 7ff5bf59e640 1 -- 172.21.15.5:0/2042676483 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a8003520 con 0x7ff5a8005350 2024-05-20T18:35:40.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.918+0000 7ff5bf59e640 1 -- 172.21.15.5:0/2042676483 reap_dead start 2024-05-20T18:35:40.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.918+0000 7ff5bf59e640 1 --2- 172.21.15.5:0/2042676483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5a8005350 0x7ff5a800b9c0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8001e50 tx=0x7ff5a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e72ae045c34f876 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:40.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.918+0000 7ff5bdd9b640 1 -- 172.21.15.5:0/2042676483 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8001a70 con 0x7ff5a8005350 2024-05-20T18:35:40.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.918+0000 7ff5bdd9b640 1 -- 172.21.15.5:0/2042676483 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b8003970 con 0x7ff5a8005350 2024-05-20T18:35:40.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.918+0000 7ff5bdd9b640 1 -- 172.21.15.5:0/2042676483 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8003d60 con 0x7ff5a8005350 2024-05-20T18:35:40.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.918+0000 7ff5c50c3640 1 -- 172.21.15.5:0/2042676483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5a8005350 msgr2=0x7ff5a800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:40.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.918+0000 7ff5c50c3640 1 --2- 172.21.15.5:0/2042676483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5a8005350 0x7ff5a800b9c0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8001e50 tx=0x7ff5a8005830 comp rx=0 tx=0).stop 2024-05-20T18:35:40.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.919+0000 7ff5c50c3640 1 -- 172.21.15.5:0/2042676483 shutdown_connections 2024-05-20T18:35:40.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.919+0000 7ff5c50c3640 1 --2- 172.21.15.5:0/2042676483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5a8005350 0x7ff5a800b9c0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:40.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.919+0000 7ff5c50c3640 1 -- 172.21.15.5:0/2042676483 >> 172.21.15.5:0/2042676483 conn(0x7ff5c00b44d0 msgr2=0x7ff5c00b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:40.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.919+0000 7ff5c50c3640 1 -- 172.21.15.5:0/2042676483 shutdown_connections 2024-05-20T18:35:40.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.919+0000 7ff5c50c3640 1 -- 172.21.15.5:0/2042676483 wait complete. 2024-05-20T18:35:40.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.921+0000 7ff5c50c3640 1 Processor -- start 2024-05-20T18:35:40.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.921+0000 7ff5c50c3640 1 -- start start 2024-05-20T18:35:40.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.922+0000 7ff5c50c3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5a8005350 0x7ff5c0154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:40.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.922+0000 7ff5c50c3640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5c00b7340 0x7ff5c00b7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:40.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.922+0000 7ff5c50c3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff5c0162c60 0x7ff5c0163070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.922+0000 7ff5c50c3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c006d990 con 0x7ff5a8005350 2024-05-20T18:35:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.922+0000 7ff5c50c3640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c00d0b40 con 0x7ff5c00b7340 2024-05-20T18:35:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.922+0000 7ff5c50c3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c015f630 con 0x7ff5c0162c60 2024-05-20T18:35:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.922+0000 7ff5bed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5a8005350 0x7ff5c0154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.922+0000 7ff5bed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5a8005350 0x7ff5c0154770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49706/0 (socket says 172.21.15.5:49706) 2024-05-20T18:35:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.922+0000 7ff5bed9d640 1 -- 172.21.15.5:0/1596337309 learned_addr learned my addr 172.21.15.5:0/1596337309 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.922+0000 7ff5be59c640 1 --2- 172.21.15.5:0/1596337309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5c00b7340 0x7ff5c00b7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.922+0000 7ff5bf59e640 1 --2- 172.21.15.5:0/1596337309 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff5c0162c60 0x7ff5c0163070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.923+0000 7ff5bed9d640 1 -- 172.21.15.5:0/1596337309 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff5c0162c60 msgr2=0x7ff5c0163070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.923+0000 7ff5bed9d640 1 --2- 172.21.15.5:0/1596337309 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff5c0162c60 0x7ff5c0163070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.923+0000 7ff5bed9d640 1 -- 172.21.15.5:0/1596337309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5c00b7340 msgr2=0x7ff5c00b7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.923+0000 7ff5bed9d640 1 --2- 172.21.15.5:0/1596337309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5c00b7340 0x7ff5c00b7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.923+0000 7ff5bed9d640 1 -- 172.21.15.5:0/1596337309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c0154cb0 con 0x7ff5a8005350 2024-05-20T18:35:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.923+0000 7ff5be59c640 1 --2- 172.21.15.5:0/1596337309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5c00b7340 0x7ff5c00b7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.923+0000 7ff5bf59e640 1 --2- 172.21.15.5:0/1596337309 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff5c0162c60 0x7ff5c0163070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.923+0000 7ff5bed9d640 1 --2- 172.21.15.5:0/1596337309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5a8005350 0x7ff5c0154770 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0007340 tx=0x7ff5b0007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.923+0000 7ff5a77fe640 1 -- 172.21.15.5:0/1596337309 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b000f8c0 con 0x7ff5a8005350 2024-05-20T18:35:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.923+0000 7ff5a77fe640 1 -- 172.21.15.5:0/1596337309 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b0018070 con 0x7ff5a8005350 2024-05-20T18:35:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.923+0000 7ff5c50c3640 1 -- 172.21.15.5:0/1596337309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c01635b0 con 0x7ff5a8005350 2024-05-20T18:35:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.923+0000 7ff5a77fe640 1 -- 172.21.15.5:0/1596337309 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b000fce0 con 0x7ff5a8005350 2024-05-20T18:35:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.923+0000 7ff5c50c3640 1 -- 172.21.15.5:0/1596337309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c0160970 con 0x7ff5a8005350 2024-05-20T18:35:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.924+0000 7ff5c50c3640 1 -- 172.21.15.5:0/1596337309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c0075b00 con 0x7ff5a8005350 2024-05-20T18:35:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.925+0000 7ff5a77fe640 1 -- 172.21.15.5:0/1596337309 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff5b001d460 con 0x7ff5a8005350 2024-05-20T18:35:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.925+0000 7ff5a77fe640 1 --2- 172.21.15.5:0/1596337309 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff5ac06be30 0x7ff5ac06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.925+0000 7ff5be59c640 1 --2- 172.21.15.5:0/1596337309 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff5ac06be30 0x7ff5ac06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.925+0000 7ff5a77fe640 1 -- 172.21.15.5:0/1596337309 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7ff5b008b090 con 0x7ff5a8005350 2024-05-20T18:35:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.926+0000 7ff5be59c640 1 --2- 172.21.15.5:0/1596337309 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff5ac06be30 0x7ff5ac06e340 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0162980 tx=0x7ff5b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:40.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:40.930+0000 7ff5a77fe640 1 -- 172.21.15.5:0/1596337309 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff5c01635b0 con 0x7ff5a8005350 2024-05-20T18:35:41.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.152+0000 7ff5c50c3640 1 -- 172.21.15.5:0/1596337309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5c015b4e0 con 0x7ff5a8005350 2024-05-20T18:35:41.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.153+0000 7ff5a77fe640 1 -- 172.21.15.5:0/1596337309 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5177 (secure 0 0 0) 0x7ff5c00cfd20 con 0x7ff5a8005350 2024-05-20T18:35:41.154 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:41.154 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6758,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/4163145158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":4163145158},{"type":"v1","addr":"172.21.15.63:6834","nonce":4163145158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7528,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3049443532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3049443532},{"type":"v1","addr":"172.21.15.5:6837","nonce":3049443532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3552410668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3552410668},{"type":"v1","addr":"172.21.15.5:6835","nonce":3552410668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:35:30.778930+0000","modified":"2024-05-20T18:35:33.806381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8343},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8343":{"gid":8343,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/2054897241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":2054897241},{"type":"v1","addr":"172.21.15.63:6835","nonce":2054897241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8343,"qdb_cluster":[8343]},"id":7}]} 2024-05-20T18:35:41.154 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 82 2024-05-20T18:35:41.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.155+0000 7ff5c50c3640 1 -- 172.21.15.5:0/1596337309 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff5ac06be30 msgr2=0x7ff5ac06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:41.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.155+0000 7ff5c50c3640 1 --2- 172.21.15.5:0/1596337309 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff5ac06be30 0x7ff5ac06e340 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0162980 tx=0x7ff5b4008040 comp rx=0 tx=0).stop 2024-05-20T18:35:41.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.155+0000 7ff5c50c3640 1 -- 172.21.15.5:0/1596337309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5a8005350 msgr2=0x7ff5c0154770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:41.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.155+0000 7ff5c50c3640 1 --2- 172.21.15.5:0/1596337309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5a8005350 0x7ff5c0154770 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0007340 tx=0x7ff5b0007800 comp rx=0 tx=0).stop 2024-05-20T18:35:41.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.155+0000 7ff5c50c3640 1 -- 172.21.15.5:0/1596337309 shutdown_connections 2024-05-20T18:35:41.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.155+0000 7ff5c50c3640 1 --2- 172.21.15.5:0/1596337309 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff5ac06be30 0x7ff5ac06e340 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.155+0000 7ff5c50c3640 1 --2- 172.21.15.5:0/1596337309 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff5c0162c60 0x7ff5c0163070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.155+0000 7ff5c50c3640 1 --2- 172.21.15.5:0/1596337309 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff5c00b7340 0x7ff5c00b7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.155+0000 7ff5c50c3640 1 --2- 172.21.15.5:0/1596337309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5a8005350 0x7ff5c0154770 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.155+0000 7ff5c50c3640 1 -- 172.21.15.5:0/1596337309 >> 172.21.15.5:0/1596337309 conn(0x7ff5c00b44d0 msgr2=0x7ff5c00b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:41.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.155+0000 7ff5c50c3640 1 -- 172.21.15.5:0/1596337309 shutdown_connections 2024-05-20T18:35:41.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.155+0000 7ff5c50c3640 1 -- 172.21.15.5:0/1596337309 wait complete. 2024-05-20T18:35:41.570 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 82, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T18:35:30.778930+0000', 'modified': '2024-05-20T18:35:33.806381+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8343}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8343': {'gid': 8343, 'name': 'b', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.63:6835/2054897241', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6833', 'nonce': 2054897241}, {'type': 'v1', 'addr': '172.21.15.63:6835', 'nonce': 2054897241}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8343, 'qdb_cluster': [8343]} 2024-05-20T18:35:41.570 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-20T18:35:41.571 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-20T18:35:41.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.689+0000 7f8a3e606640 1 Processor -- start 2024-05-20T18:35:41.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.689+0000 7f8a3e606640 1 -- start start 2024-05-20T18:35:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.691+0000 7f8a3e606640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8a38162820 0x7f8a380b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.691+0000 7f8a3e606640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8a3806dae0 0x7f8a380b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:41.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.691+0000 7f8a3e606640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a380d0630 con 0x7f8a380b3e80 2024-05-20T18:35:41.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.691+0000 7f8a3e606640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f8a380d39c0 con 0x7f8a38162820 2024-05-20T18:35:41.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.691+0000 7f8a3e606640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8a380d3ca0 con 0x7f8a3806dae0 2024-05-20T18:35:41.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.691+0000 7f8a37fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8a38162820 0x7f8a380b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:41.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.691+0000 7f8a37fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8a38162820 0x7f8a380b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49722/0 (socket says 172.21.15.5:49722) 2024-05-20T18:35:41.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.691+0000 7f8a37fff640 1 -- 172.21.15.5:0/101256477 learned_addr learned my addr 172.21.15.5:0/101256477 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:41.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.691+0000 7f8a3cba3640 1 --2- 172.21.15.5:0/101256477 >> v2:172.21.15.63:3300/0 conn(0x7f8a3806dae0 0x7f8a380b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:41.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.691+0000 7f8a37fff640 1 -- 172.21.15.5:0/101256477 >> v2:172.21.15.63:3300/0 conn(0x7f8a3806dae0 msgr2=0x7f8a380b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:41.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.691+0000 7f8a37fff640 1 --2- 172.21.15.5:0/101256477 >> v2:172.21.15.63:3300/0 conn(0x7f8a3806dae0 0x7f8a380b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.691+0000 7f8a37fff640 1 -- 172.21.15.5:0/101256477 >> v1:172.21.15.63:6789/0 conn(0x7f8a380b3e80 legacy=0x7f8a3806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:41.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.691+0000 7f8a37fff640 1 -- 172.21.15.5:0/101256477 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a380d3f80 con 0x7f8a38162820 2024-05-20T18:35:41.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.691+0000 7f8a3cba3640 1 --2- 172.21.15.5:0/101256477 >> v2:172.21.15.63:3300/0 conn(0x7f8a3806dae0 0x7f8a380b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:41.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.692+0000 7f8a37fff640 1 --2- 172.21.15.5:0/101256477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a38162820 0x7f8a380b3940 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f8a3808e070 tx=0x7f8a2802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5eb84a61b1cff3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:41.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.692+0000 7f8a377fe640 1 -- 172.21.15.5:0/101256477 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a28037070 con 0x7f8a38162820 2024-05-20T18:35:41.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.692+0000 7f8a377fe640 1 -- 172.21.15.5:0/101256477 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a28032440 con 0x7f8a38162820 2024-05-20T18:35:41.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.692+0000 7f8a377fe640 1 -- 172.21.15.5:0/101256477 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a28032810 con 0x7f8a38162820 2024-05-20T18:35:41.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.693+0000 7f8a3e606640 1 -- 172.21.15.5:0/101256477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a38162820 msgr2=0x7f8a380b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:41.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.693+0000 7f8a3e606640 1 --2- 172.21.15.5:0/101256477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a38162820 0x7f8a380b3940 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f8a3808e070 tx=0x7f8a2802a230 comp rx=0 tx=0).stop 2024-05-20T18:35:41.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.693+0000 7f8a3e606640 1 -- 172.21.15.5:0/101256477 shutdown_connections 2024-05-20T18:35:41.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.693+0000 7f8a3e606640 1 --2- 172.21.15.5:0/101256477 >> v2:172.21.15.63:3300/0 conn(0x7f8a3806dae0 0x7f8a380b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.693+0000 7f8a3e606640 1 --2- 172.21.15.5:0/101256477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a38162820 0x7f8a380b3940 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.693+0000 7f8a3e606640 1 -- 172.21.15.5:0/101256477 >> 172.21.15.5:0/101256477 conn(0x7f8a380d4bd0 msgr2=0x7f8a380d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:41.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.693+0000 7f8a3e606640 1 -- 172.21.15.5:0/101256477 shutdown_connections 2024-05-20T18:35:41.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.693+0000 7f8a3e606640 1 -- 172.21.15.5:0/101256477 wait complete. 2024-05-20T18:35:41.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.695+0000 7f8a3e606640 1 Processor -- start 2024-05-20T18:35:41.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.695+0000 7f8a3e606640 1 -- start start 2024-05-20T18:35:41.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.696+0000 7f8a3e606640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a380b3e80 0x7f8a380cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:41.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.696+0000 7f8a3e606640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a38162820 0x7f8a3814bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:41.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.696+0000 7f8a3e606640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a3806d3d0 0x7f8a3806d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:41.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.696+0000 7f8a3e606640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8a380d4780 con 0x7f8a38162820 2024-05-20T18:35:41.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.696+0000 7f8a3e606640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8a380d0300 con 0x7f8a380b3e80 2024-05-20T18:35:41.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.696+0000 7f8a3e606640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8a380d4900 con 0x7f8a3806d3d0 2024-05-20T18:35:41.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.696+0000 7f8a2ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a38162820 0x7f8a3814bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:41.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.696+0000 7f8a2ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a38162820 0x7f8a3814bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49726/0 (socket says 172.21.15.5:49726) 2024-05-20T18:35:41.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.696+0000 7f8a2ffff640 1 -- 172.21.15.5:0/273742048 learned_addr learned my addr 172.21.15.5:0/273742048 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:41.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.696+0000 7f8a37fff640 1 --2- 172.21.15.5:0/273742048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a380b3e80 0x7f8a380cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:41.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.696+0000 7f8a3cba3640 1 --2- 172.21.15.5:0/273742048 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a3806d3d0 0x7f8a3806d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:41.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.697+0000 7f8a2ffff640 1 -- 172.21.15.5:0/273742048 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a3806d3d0 msgr2=0x7f8a3806d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:41.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.697+0000 7f8a2ffff640 1 --2- 172.21.15.5:0/273742048 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a3806d3d0 0x7f8a3806d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.697+0000 7f8a2ffff640 1 -- 172.21.15.5:0/273742048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a380b3e80 msgr2=0x7f8a380cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:41.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.697+0000 7f8a2ffff640 1 --2- 172.21.15.5:0/273742048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a380b3e80 0x7f8a380cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.697+0000 7f8a2ffff640 1 -- 172.21.15.5:0/273742048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3806dd20 con 0x7f8a38162820 2024-05-20T18:35:41.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.697+0000 7f8a2ffff640 1 --2- 172.21.15.5:0/273742048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a38162820 0x7f8a3814bdd0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000b3f0 tx=0x7f8a2000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:41.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.697+0000 7f8a357fa640 1 -- 172.21.15.5:0/273742048 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a20015020 con 0x7f8a38162820 2024-05-20T18:35:41.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.697+0000 7f8a357fa640 1 -- 172.21.15.5:0/273742048 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a20011830 con 0x7f8a38162820 2024-05-20T18:35:41.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.697+0000 7f8a3e606640 1 -- 172.21.15.5:0/273742048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a3814c890 con 0x7f8a38162820 2024-05-20T18:35:41.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.697+0000 7f8a357fa640 1 -- 172.21.15.5:0/273742048 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a20011c20 con 0x7f8a38162820 2024-05-20T18:35:41.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.697+0000 7f8a3e606640 1 -- 172.21.15.5:0/273742048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a38149f50 con 0x7f8a38162820 2024-05-20T18:35:41.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.699+0000 7f8a357fa640 1 -- 172.21.15.5:0/273742048 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8a20018020 con 0x7f8a38162820 2024-05-20T18:35:41.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.699+0000 7f8a357fa640 1 --2- 172.21.15.5:0/273742048 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8a1806be60 0x7f8a1806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:41.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.699+0000 7f8a357fa640 1 -- 172.21.15.5:0/273742048 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7f8a2008ac10 con 0x7f8a38162820 2024-05-20T18:35:41.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.699+0000 7f8a37fff640 1 --2- 172.21.15.5:0/273742048 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8a1806be60 0x7f8a1806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:41.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.700+0000 7f8a37fff640 1 --2- 172.21.15.5:0/273742048 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8a1806be60 0x7f8a1806e370 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8a28034000 tx=0x7f8a2802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:41.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.700+0000 7f8a45586740 1 Processor -- start 2024-05-20T18:35:41.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.700+0000 7f8a45586740 1 -- start start 2024-05-20T18:35:41.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.702+0000 7f8a45586740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f06bcd270 0x564f06f98700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:41.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.702+0000 7f8a45586740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f06ba43b0 0x564f06f950a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:41.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.702+0000 7f8a45586740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f06bac590 0x564f06f6e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:41.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.702+0000 7f8a45586740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564f06ed14d0 con 0x564f06ba43b0 2024-05-20T18:35:41.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.702+0000 7f8a45586740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564f06e793e0 con 0x564f06bcd270 2024-05-20T18:35:41.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.702+0000 7f8a45586740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564f06ecd490 con 0x564f06bac590 2024-05-20T18:35:41.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.702+0000 7f8a2ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f06ba43b0 0x564f06f950a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:41.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.702+0000 7f8a2ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f06ba43b0 0x564f06f950a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49732/0 (socket says 172.21.15.5:49732) 2024-05-20T18:35:41.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.702+0000 7f8a2ffff640 1 -- 172.21.15.5:0/47386660 learned_addr learned my addr 172.21.15.5:0/47386660 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:41.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.702+0000 7f8a37fff640 1 --2- 172.21.15.5:0/47386660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f06bcd270 0x564f06f98700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:41.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.702+0000 7f8a3cba3640 1 --2- 172.21.15.5:0/47386660 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f06bac590 0x564f06f6e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:41.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.702+0000 7f8a2ffff640 1 -- 172.21.15.5:0/47386660 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f06bac590 msgr2=0x564f06f6e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:41.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.702+0000 7f8a2ffff640 1 --2- 172.21.15.5:0/47386660 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f06bac590 0x564f06f6e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.702+0000 7f8a2ffff640 1 -- 172.21.15.5:0/47386660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f06bcd270 msgr2=0x564f06f98700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:41.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.702+0000 7f8a2ffff640 1 --2- 172.21.15.5:0/47386660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f06bcd270 0x564f06f98700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.702+0000 7f8a2ffff640 1 -- 172.21.15.5:0/47386660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f06be52d0 con 0x564f06ba43b0 2024-05-20T18:35:41.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.702+0000 7f8a37fff640 1 --2- 172.21.15.5:0/47386660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f06bcd270 0x564f06f98700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:41.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.702+0000 7f8a3cba3640 1 --2- 172.21.15.5:0/47386660 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f06bac590 0x564f06f6e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:41.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.702+0000 7f8a2ffff640 1 --2- 172.21.15.5:0/47386660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f06ba43b0 0x564f06f950a0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f8a20004730 tx=0x7f8a200594e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4e3c08e358c1e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:41.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.702+0000 7f8a2dffb640 1 -- 172.21.15.5:0/47386660 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a20056d40 con 0x564f06ba43b0 2024-05-20T18:35:41.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.702+0000 7f8a2dffb640 1 -- 172.21.15.5:0/47386660 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a20011dc0 con 0x564f06ba43b0 2024-05-20T18:35:41.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.702+0000 7f8a2dffb640 1 -- 172.21.15.5:0/47386660 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a20056fa0 con 0x564f06ba43b0 2024-05-20T18:35:41.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.703+0000 7f8a45586740 1 -- 172.21.15.5:0/47386660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f06ba43b0 msgr2=0x564f06f950a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:41.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.703+0000 7f8a45586740 1 --2- 172.21.15.5:0/47386660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f06ba43b0 0x564f06f950a0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f8a20004730 tx=0x7f8a200594e0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.703+0000 7f8a45586740 1 -- 172.21.15.5:0/47386660 shutdown_connections 2024-05-20T18:35:41.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.703+0000 7f8a45586740 1 --2- 172.21.15.5:0/47386660 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f06bac590 0x564f06f6e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.703+0000 7f8a45586740 1 --2- 172.21.15.5:0/47386660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f06ba43b0 0x564f06f950a0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.703+0000 7f8a45586740 1 --2- 172.21.15.5:0/47386660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f06bcd270 0x564f06f98700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.703+0000 7f8a45586740 1 -- 172.21.15.5:0/47386660 >> 172.21.15.5:0/47386660 conn(0x564f06bc13c0 msgr2=0x564f06f6daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:41.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.703+0000 7f8a45586740 1 -- 172.21.15.5:0/47386660 shutdown_connections 2024-05-20T18:35:41.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.703+0000 7f8a45586740 1 -- 172.21.15.5:0/47386660 wait complete. 2024-05-20T18:35:41.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.704+0000 7f8a45586740 1 -- start start 2024-05-20T18:35:41.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.704+0000 7f8a45586740 1 Processor -- start 2024-05-20T18:35:41.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.706+0000 7f8a45586740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f06ba43b0 0x564f06fc3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:41.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.706+0000 7f8a45586740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f06bac590 0x564f06fb2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:41.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.706+0000 7f8a45586740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f06bcd270 0x564f06f6e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.706+0000 7f8a45586740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564f06e8c8e0 con 0x564f06bcd270 2024-05-20T18:35:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.706+0000 7f8a45586740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564f06f9ba00 con 0x564f06ba43b0 2024-05-20T18:35:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.706+0000 7f8a45586740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564f06fb33b0 con 0x564f06bac590 2024-05-20T18:35:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.706+0000 7f8a45586740 10 client.0 _ll_register_callbacks cb 0x564f06e599c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:35:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.706+0000 7f8a3cba3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f06bcd270 0x564f06f6e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.706+0000 7f8a3cba3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f06bcd270 0x564f06f6e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49738/0 (socket says 172.21.15.5:49738) 2024-05-20T18:35:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.706+0000 7f8a3cba3640 1 -- 172.21.15.5:0/589599281 learned_addr learned my addr 172.21.15.5:0/589599281 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.706+0000 7f8a37fff640 1 --2- 172.21.15.5:0/589599281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f06ba43b0 0x564f06fc3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.706+0000 7f8a2ffff640 1 --2- 172.21.15.5:0/589599281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f06bac590 0x564f06fb2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.706+0000 7f8a3cba3640 1 -- 172.21.15.5:0/589599281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f06bac590 msgr2=0x564f06fb2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.706+0000 7f8a3cba3640 1 --2- 172.21.15.5:0/589599281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f06bac590 0x564f06fb2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.706+0000 7f8a3cba3640 1 -- 172.21.15.5:0/589599281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f06ba43b0 msgr2=0x564f06fc3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.706+0000 7f8a3cba3640 1 --2- 172.21.15.5:0/589599281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f06ba43b0 0x564f06fc3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.706+0000 7f8a3cba3640 1 -- 172.21.15.5:0/589599281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f06caf6b0 con 0x564f06bcd270 2024-05-20T18:35:41.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.707+0000 7f8a3cba3640 1 --2- 172.21.15.5:0/589599281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f06bcd270 0x564f06f6e340 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f8a380c8490 tx=0x7f8a30010f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c64aee83ac7c16 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:41.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.707+0000 7f8a16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:35:41.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.707+0000 7f8a16ffd640 1 -- 172.21.15.5:0/589599281 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3001a3f0 con 0x564f06bcd270 2024-05-20T18:35:41.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.707+0000 7f8a16ffd640 1 -- 172.21.15.5:0/589599281 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7f8a3001a590 con 0x564f06bcd270 2024-05-20T18:35:41.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.707+0000 7f8a45586740 1 -- 172.21.15.5:0/589599281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a20011dc0 con 0x564f06bcd270 2024-05-20T18:35:41.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.707+0000 7f8a16ffd640 1 -- 172.21.15.5:0/589599281 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f8a30024050 con 0x564f06bcd270 2024-05-20T18:35:41.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.707+0000 7f8a45586740 10 client.8505 fetch_fsmap learned FSMap version 82 2024-05-20T18:35:41.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.707+0000 7f8a45586740 1 -- 172.21.15.5:0/589599281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x564f06fb15c0 con 0x564f06bcd270 2024-05-20T18:35:41.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.707+0000 7f8a16ffd640 1 -- 172.21.15.5:0/589599281 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a20011dc0 con 0x564f06bcd270 2024-05-20T18:35:41.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.707+0000 7f8a16ffd640 1 -- 172.21.15.5:0/589599281 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f8a30027020 con 0x564f06bcd270 2024-05-20T18:35:41.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.707+0000 7f8a45586740 10 client.8505 fetch_fsmap finished waiting for FSMap version 82 2024-05-20T18:35:41.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.707+0000 7f8a45586740 10 client.8505 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-20T18:35:41.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.707+0000 7f8a45586740 20 client.8505 populate_metadata read hostname 'smithi005' 2024-05-20T18:35:41.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.707+0000 7f8a45586740 1 --2- 172.21.15.5:0/589599281 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x564f06b8be40 0x564f06f9ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:41.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.707+0000 7f8a45586740 4 client.8505 mds_command: new command op to 8343 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:35:41.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.707+0000 7f8a45586740 1 -- 172.21.15.5:0/589599281 --> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f06edec00 con 0x564f06b8be40 2024-05-20T18:35:41.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.708+0000 7f8a37fff640 1 --2- 172.21.15.5:0/589599281 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x564f06b8be40 0x564f06f9ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:41.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.708+0000 7f8a16ffd640 10 client.8505 ms_handle_connect on v2:172.21.15.63:6833/2054897241 2024-05-20T18:35:41.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.708+0000 7f8a37fff640 1 --2- 172.21.15.5:0/589599281 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x564f06b8be40 0x564f06f9ad80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77b542cc5b95b7a6 server_cookie=b72d499b1fc460e3 in_seq=0 out_seq=0 2024-05-20T18:35:41.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.709+0000 7f8a16ffd640 1 -- 172.21.15.5:0/589599281 <== mds.0 v2:172.21.15.63:6833/2054897241 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564f06edec00 con 0x564f06b8be40 2024-05-20T18:35:41.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.709+0000 7f8a16ffd640 10 client.8505 handle_command_reply: tid=0 2024-05-20T18:35:41.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.709+0000 7f8a45586740 1 client.8505 shutdown 2024-05-20T18:35:41.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.709+0000 7f8a45586740 1 -- 172.21.15.5:0/589599281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f06bcd270 msgr2=0x564f06f6e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:41.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.709+0000 7f8a45586740 1 --2- 172.21.15.5:0/589599281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f06bcd270 0x564f06f6e340 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f8a380c8490 tx=0x7f8a30010f00 comp rx=0 tx=0).stop 2024-05-20T18:35:41.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.709+0000 7f8a45586740 1 -- 172.21.15.5:0/589599281 shutdown_connections 2024-05-20T18:35:41.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.709+0000 7f8a45586740 1 --2- 172.21.15.5:0/589599281 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x564f06b8be40 0x564f06f9ad80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.709+0000 7f8a45586740 1 --2- 172.21.15.5:0/589599281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f06bcd270 0x564f06f6e340 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.709+0000 7f8a45586740 1 --2- 172.21.15.5:0/589599281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f06bac590 0x564f06fb2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.709+0000 7f8a16ffd640 0 client.8505 ms_handle_reset on v2:172.21.15.63:6833/2054897241 2024-05-20T18:35:41.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.709+0000 7f8a45586740 1 --2- 172.21.15.5:0/589599281 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f06ba43b0 0x564f06fc3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.709+0000 7f8a45586740 1 -- 172.21.15.5:0/589599281 >> 172.21.15.5:0/589599281 conn(0x564f06bc13c0 msgr2=0x564f06f6daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:41.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.709+0000 7f8a45586740 1 -- 172.21.15.5:0/589599281 shutdown_connections 2024-05-20T18:35:41.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.709+0000 7f8a45586740 1 -- 172.21.15.5:0/589599281 wait complete. 2024-05-20T18:35:41.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.710+0000 7f8a45586740 20 client.8505 trim_cache size 0 max 16384 2024-05-20T18:35:41.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.728+0000 7f8a45586740 1 Processor -- start 2024-05-20T18:35:41.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.728+0000 7f8a45586740 1 -- start start 2024-05-20T18:35:41.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.730+0000 7f8a45586740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f06ba43b0 0x564f06f6dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:41.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.730+0000 7f8a45586740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f06bac590 0x564f06f988f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:41.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.730+0000 7f8a45586740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f06bcd270 0x564f06fa8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:41.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.730+0000 7f8a45586740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8a382013d0 con 0x564f06bac590 2024-05-20T18:35:41.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.730+0000 7f8a45586740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8a382007b0 con 0x564f06ba43b0 2024-05-20T18:35:41.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.730+0000 7f8a45586740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8a38200d80 con 0x564f06bcd270 2024-05-20T18:35:41.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.730+0000 7f8a2ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f06bac590 0x564f06f988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:41.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.730+0000 7f8a2ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f06bac590 0x564f06f988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49740/0 (socket says 172.21.15.5:49740) 2024-05-20T18:35:41.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.730+0000 7f8a2ffff640 1 -- 172.21.15.5:0/1698442455 learned_addr learned my addr 172.21.15.5:0/1698442455 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:41.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.730+0000 7f8a37fff640 1 --2- 172.21.15.5:0/1698442455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f06ba43b0 0x564f06f6dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:41.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.730+0000 7f8a3cba3640 1 --2- 172.21.15.5:0/1698442455 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f06bcd270 0x564f06fa8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:41.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.730+0000 7f8a2ffff640 1 -- 172.21.15.5:0/1698442455 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f06bcd270 msgr2=0x564f06fa8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:41.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.730+0000 7f8a2ffff640 1 --2- 172.21.15.5:0/1698442455 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f06bcd270 0x564f06fa8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.730+0000 7f8a2ffff640 1 -- 172.21.15.5:0/1698442455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f06ba43b0 msgr2=0x564f06f6dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:41.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.730+0000 7f8a2ffff640 1 --2- 172.21.15.5:0/1698442455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f06ba43b0 0x564f06f6dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.730+0000 7f8a2ffff640 1 -- 172.21.15.5:0/1698442455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f06fc21d0 con 0x564f06bac590 2024-05-20T18:35:41.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.730+0000 7f8a37fff640 1 --2- 172.21.15.5:0/1698442455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f06ba43b0 0x564f06f6dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:41.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.730+0000 7f8a3cba3640 1 --2- 172.21.15.5:0/1698442455 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f06bcd270 0x564f06fa8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:41.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.730+0000 7f8a2ffff640 1 --2- 172.21.15.5:0/1698442455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f06bac590 0x564f06f988f0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f8a20058e60 tx=0x7f8a20070000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2383eecac77e973 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:41.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.730+0000 7f8a2d7fa640 1 -- 172.21.15.5:0/1698442455 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2007a020 con 0x564f06bac590 2024-05-20T18:35:41.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.730+0000 7f8a2d7fa640 1 -- 172.21.15.5:0/1698442455 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a20056d40 con 0x564f06bac590 2024-05-20T18:35:41.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.730+0000 7f8a2d7fa640 1 -- 172.21.15.5:0/1698442455 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a20056fa0 con 0x564f06bac590 2024-05-20T18:35:41.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.731+0000 7f8a45586740 1 -- 172.21.15.5:0/1698442455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f06bac590 msgr2=0x564f06f988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:41.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.731+0000 7f8a45586740 1 --2- 172.21.15.5:0/1698442455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f06bac590 0x564f06f988f0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f8a20058e60 tx=0x7f8a20070000 comp rx=0 tx=0).stop 2024-05-20T18:35:41.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.731+0000 7f8a45586740 1 -- 172.21.15.5:0/1698442455 shutdown_connections 2024-05-20T18:35:41.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.731+0000 7f8a45586740 1 --2- 172.21.15.5:0/1698442455 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f06bcd270 0x564f06fa8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.731+0000 7f8a45586740 1 --2- 172.21.15.5:0/1698442455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f06bac590 0x564f06f988f0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.731+0000 7f8a45586740 1 --2- 172.21.15.5:0/1698442455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f06ba43b0 0x564f06f6dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.731+0000 7f8a45586740 1 -- 172.21.15.5:0/1698442455 >> 172.21.15.5:0/1698442455 conn(0x564f06bc13c0 msgr2=0x564f06fc2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:41.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.731+0000 7f8a45586740 1 -- 172.21.15.5:0/1698442455 shutdown_connections 2024-05-20T18:35:41.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.731+0000 7f8a45586740 1 -- 172.21.15.5:0/1698442455 wait complete. 2024-05-20T18:35:41.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.732+0000 7f8a45586740 1 -- start start 2024-05-20T18:35:41.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.732+0000 7f8a45586740 1 Processor -- start 2024-05-20T18:35:41.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.733+0000 7f8a45586740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f06ba43b0 0x564f06f6dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:41.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.733+0000 7f8a45586740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f06bac590 0x564f06f6e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:41.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.734+0000 7f8a45586740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f06bcd270 0x564f06fa8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:41.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.734+0000 7f8a45586740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564f06fc5ec0 con 0x564f06bac590 2024-05-20T18:35:41.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.734+0000 7f8a45586740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564f06fb1dc0 con 0x564f06bcd270 2024-05-20T18:35:41.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.734+0000 7f8a45586740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564f06fc64b0 con 0x564f06ba43b0 2024-05-20T18:35:41.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.734+0000 7f8a45586740 10 client.0 _ll_register_callbacks cb 0x7f8a30007640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:35:41.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.734+0000 7f8a2ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f06bac590 0x564f06f6e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:41.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.734+0000 7f8a2ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f06bac590 0x564f06f6e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49754/0 (socket says 172.21.15.5:49754) 2024-05-20T18:35:41.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.734+0000 7f8a2ffff640 1 -- 172.21.15.5:0/371906078 learned_addr learned my addr 172.21.15.5:0/371906078 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:41.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.734+0000 7f8a37fff640 1 --2- 172.21.15.5:0/371906078 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f06ba43b0 0x564f06f6dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:41.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.734+0000 7f8a3cba3640 1 --2- 172.21.15.5:0/371906078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f06bcd270 0x564f06fa8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:41.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.734+0000 7f8a2ffff640 1 -- 172.21.15.5:0/371906078 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f06ba43b0 msgr2=0x564f06f6dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:41.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.734+0000 7f8a2ffff640 1 --2- 172.21.15.5:0/371906078 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f06ba43b0 0x564f06f6dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.734+0000 7f8a2ffff640 1 -- 172.21.15.5:0/371906078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f06bcd270 msgr2=0x564f06fa8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:41.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.734+0000 7f8a2ffff640 1 --2- 172.21.15.5:0/371906078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f06bcd270 0x564f06fa8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.734+0000 7f8a2ffff640 1 -- 172.21.15.5:0/371906078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f06fb0d90 con 0x564f06bac590 2024-05-20T18:35:41.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.734+0000 7f8a2ffff640 1 --2- 172.21.15.5:0/371906078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f06bac590 0x564f06f6e010 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f8a20089000 tx=0x7f8a20004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb1f68cbc9d85f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:41.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.734+0000 7f8a16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:35:41.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.734+0000 7f8a16ffd640 1 -- 172.21.15.5:0/371906078 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2007a050 con 0x564f06bac590 2024-05-20T18:35:41.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.734+0000 7f8a16ffd640 1 -- 172.21.15.5:0/371906078 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7f8a20055310 con 0x564f06bac590 2024-05-20T18:35:41.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.734+0000 7f8a45586740 1 -- 172.21.15.5:0/371906078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a20056d40 con 0x564f06bac590 2024-05-20T18:35:41.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.734+0000 7f8a16ffd640 1 -- 172.21.15.5:0/371906078 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f8a20066050 con 0x564f06bac590 2024-05-20T18:35:41.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.734+0000 7f8a45586740 10 client.8511 fetch_fsmap learned FSMap version 82 2024-05-20T18:35:41.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.734+0000 7f8a45586740 1 -- 172.21.15.5:0/371906078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x564f07031510 con 0x564f06bac590 2024-05-20T18:35:41.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.735+0000 7f8a16ffd640 1 -- 172.21.15.5:0/371906078 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a20056d40 con 0x564f06bac590 2024-05-20T18:35:41.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.735+0000 7f8a16ffd640 1 -- 172.21.15.5:0/371906078 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f8a20069020 con 0x564f06bac590 2024-05-20T18:35:41.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.735+0000 7f8a45586740 10 client.8511 fetch_fsmap finished waiting for FSMap version 82 2024-05-20T18:35:41.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.735+0000 7f8a45586740 10 client.8511 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-20T18:35:41.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.735+0000 7f8a45586740 20 client.8511 populate_metadata read hostname 'smithi005' 2024-05-20T18:35:41.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.735+0000 7f8a45586740 1 --2- 172.21.15.5:0/371906078 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x564f06fa9520 0x564f06f94670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:41.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.735+0000 7f8a45586740 4 client.8511 mds_command: new command op to 8343 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T18:35:41.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.735+0000 7f8a45586740 1 -- 172.21.15.5:0/371906078 --> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] -- command(tid 0: {"prefix": "status"}) -- 0x564f06ed8f50 con 0x564f06fa9520 2024-05-20T18:35:41.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.735+0000 7f8a37fff640 1 --2- 172.21.15.5:0/371906078 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x564f06fa9520 0x564f06f94670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:41.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.735+0000 7f8a16ffd640 10 client.8511 ms_handle_connect on v2:172.21.15.63:6833/2054897241 2024-05-20T18:35:41.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.735+0000 7f8a37fff640 1 --2- 172.21.15.5:0/371906078 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x564f06fa9520 0x564f06f94670 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdf56e59806c8269 server_cookie=3dec0d1830f01566 in_seq=0 out_seq=0 2024-05-20T18:35:41.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.736+0000 7f8a16ffd640 1 -- 172.21.15.5:0/371906078 <== mds.0 v2:172.21.15.63:6833/2054897241 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x564f06ed8f50 con 0x564f06fa9520 2024-05-20T18:35:41.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.736+0000 7f8a16ffd640 10 client.8511 handle_command_reply: tid=0 2024-05-20T18:35:41.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.736+0000 7f8a45586740 1 client.8511 shutdown 2024-05-20T18:35:41.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.736+0000 7f8a45586740 1 -- 172.21.15.5:0/371906078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f06bac590 msgr2=0x564f06f6e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:41.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.736+0000 7f8a45586740 1 --2- 172.21.15.5:0/371906078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f06bac590 0x564f06f6e010 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f8a20089000 tx=0x7f8a20004680 comp rx=0 tx=0).stop 2024-05-20T18:35:41.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.736+0000 7f8a45586740 1 -- 172.21.15.5:0/371906078 shutdown_connections 2024-05-20T18:35:41.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.736+0000 7f8a45586740 1 --2- 172.21.15.5:0/371906078 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x564f06fa9520 0x564f06f94670 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.736+0000 7f8a45586740 1 --2- 172.21.15.5:0/371906078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564f06bcd270 0x564f06fa8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.736+0000 7f8a45586740 1 --2- 172.21.15.5:0/371906078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f06bac590 0x564f06f6e010 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.736+0000 7f8a16ffd640 0 client.8511 ms_handle_reset on v2:172.21.15.63:6833/2054897241 2024-05-20T18:35:41.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.736+0000 7f8a45586740 1 --2- 172.21.15.5:0/371906078 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564f06ba43b0 0x564f06f6dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.736+0000 7f8a45586740 1 -- 172.21.15.5:0/371906078 >> 172.21.15.5:0/371906078 conn(0x564f06bc13c0 msgr2=0x564f06fa9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:41.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.736+0000 7f8a45586740 1 -- 172.21.15.5:0/371906078 shutdown_connections 2024-05-20T18:35:41.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.736+0000 7f8a45586740 1 -- 172.21.15.5:0/371906078 wait complete. 2024-05-20T18:35:41.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.736+0000 7f8a45586740 20 client.8511 trim_cache size 0 max 16384 2024-05-20T18:35:41.742 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:35:41.743 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "399ab250-fd3c-47fa-a1c8-25e8149c9cac", 2024-05-20T18:35:41.743 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-20T18:35:41.743 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 8343, 2024-05-20T18:35:41.743 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-20T18:35:41.743 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-20T18:35:41.743 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-20T18:35:41.743 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 10.336619869, 2024-05-20T18:35:41.743 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 82, 2024-05-20T18:35:41.743 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 105, 2024-05-20T18:35:41.743 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 105, 2024-05-20T18:35:41.743 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 15.274987496 2024-05-20T18:35:41.743 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:35:41.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.736+0000 7f8a3e606640 1 -- 172.21.15.5:0/273742048 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8a1806be60 msgr2=0x7f8a1806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:41.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.737+0000 7f8a3e606640 1 --2- 172.21.15.5:0/273742048 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8a1806be60 0x7f8a1806e370 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8a28034000 tx=0x7f8a2802e000 comp rx=0 tx=0).stop 2024-05-20T18:35:41.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.737+0000 7f8a3e606640 1 -- 172.21.15.5:0/273742048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a38162820 msgr2=0x7f8a3814bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:41.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.737+0000 7f8a3e606640 1 --2- 172.21.15.5:0/273742048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a38162820 0x7f8a3814bdd0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000b3f0 tx=0x7f8a2000b8b0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.737+0000 7f8a3e606640 1 -- 172.21.15.5:0/273742048 shutdown_connections 2024-05-20T18:35:41.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.737+0000 7f8a3e606640 1 --2- 172.21.15.5:0/273742048 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8a1806be60 0x7f8a1806e370 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.737+0000 7f8a3e606640 1 --2- 172.21.15.5:0/273742048 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a3806d3d0 0x7f8a3806d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.737+0000 7f8a3e606640 1 --2- 172.21.15.5:0/273742048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a38162820 0x7f8a3814bdd0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.737+0000 7f8a3e606640 1 --2- 172.21.15.5:0/273742048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a380b3e80 0x7f8a380cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.737+0000 7f8a3e606640 1 -- 172.21.15.5:0/273742048 >> 172.21.15.5:0/273742048 conn(0x7f8a380d4bd0 msgr2=0x7f8a3815d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:41.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.737+0000 7f8a3e606640 1 -- 172.21.15.5:0/273742048 shutdown_connections 2024-05-20T18:35:41.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.737+0000 7f8a3e606640 1 -- 172.21.15.5:0/273742048 wait complete. 2024-05-20T18:35:41.749 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-20T18:35:41.840 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:35:41.842 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:35:41.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.865+0000 7f5bedc92640 1 Processor -- start 2024-05-20T18:35:41.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.865+0000 7f5bedc92640 1 -- start start 2024-05-20T18:35:41.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.866+0000 7f5bedc92640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80d14e0 0x7f5be806b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:41.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.866+0000 7f5bedc92640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5be806cce0 con 0x7f5be806ba60 2024-05-20T18:35:41.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.866+0000 7f5bedc92640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5be809ad20 con 0x7f5be80b17c0 2024-05-20T18:35:41.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.866+0000 7f5bedc92640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5be806cab0 con 0x7f5be80d14e0 2024-05-20T18:35:41.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.866+0000 7f5be7fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5be806ba60 0x7f5be806be70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46738/0 (socket says 172.21.15.5:46738) 2024-05-20T18:35:41.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.866+0000 7f5be7fff640 1 -- 172.21.15.5:0/1391990373 learned_addr learned my addr 172.21.15.5:0/1391990373 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:41.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.866+0000 7f5bdeffd640 1 --2- 172.21.15.5:0/1391990373 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80d14e0 0x7f5be806b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:41.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.866+0000 7f5be6ffd640 1 -- 172.21.15.5:0/1391990373 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2613427328 0 0) 0x7f5be806cce0 con 0x7f5be806ba60 2024-05-20T18:35:41.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.867+0000 7f5be6ffd640 1 -- 172.21.15.5:0/1391990373 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bd4003520 con 0x7f5be806ba60 2024-05-20T18:35:41.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.867+0000 7f5be6ffd640 1 -- 172.21.15.5:0/1391990373 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3113488849 0 0) 0x7f5bd4003520 con 0x7f5be806ba60 2024-05-20T18:35:41.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.867+0000 7f5be6ffd640 1 -- 172.21.15.5:0/1391990373 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80d14e0 msgr2=0x7f5be806b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:41.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.867+0000 7f5be6ffd640 1 --2- 172.21.15.5:0/1391990373 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80d14e0 0x7f5be806b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.867+0000 7f5be6ffd640 1 -- 172.21.15.5:0/1391990373 >> v1:172.21.15.63:6789/0 conn(0x7f5be80b17c0 legacy=0x7f5be80d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:41.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.867+0000 7f5be6ffd640 1 -- 172.21.15.5:0/1391990373 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be80d1f50 con 0x7f5be806ba60 2024-05-20T18:35:41.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.867+0000 7f5be6ffd640 1 -- 172.21.15.5:0/1391990373 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f5be0001a10 con 0x7f5be806ba60 2024-05-20T18:35:41.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.867+0000 7f5be6ffd640 1 -- 172.21.15.5:0/1391990373 >> v1:172.21.15.5:6789/0 conn(0x7f5be806ba60 legacy=0x7f5be806be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:41.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.867+0000 7f5be6ffd640 1 --2- 172.21.15.5:0/1391990373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5bd40039e0 0x7f5bd4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:41.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.867+0000 7f5be6ffd640 1 --2- 172.21.15.5:0/1391990373 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80d14e0 0x7f5bd4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:41.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.867+0000 7f5be6ffd640 1 --2- 172.21.15.5:0/1391990373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bd4004d40 0x7f5bd4005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:41.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.867+0000 7f5be6ffd640 1 -- 172.21.15.5:0/1391990373 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd400ba70 con 0x7f5bd4004d40 2024-05-20T18:35:41.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.867+0000 7f5be6ffd640 1 -- 172.21.15.5:0/1391990373 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd400bbf0 con 0x7f5bd40039e0 2024-05-20T18:35:41.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.867+0000 7f5be6ffd640 1 -- 172.21.15.5:0/1391990373 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5bd400bdf0 con 0x7f5be80d14e0 2024-05-20T18:35:41.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.867+0000 7f5be7fff640 1 --2- 172.21.15.5:0/1391990373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bd4004d40 0x7f5bd4005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:41.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.867+0000 7f5bdeffd640 1 --2- 172.21.15.5:0/1391990373 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80d14e0 0x7f5bd4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:41.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.867+0000 7f5be77fe640 1 --2- 172.21.15.5:0/1391990373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5bd40039e0 0x7f5bd4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:41.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.867+0000 7f5be7fff640 1 -- 172.21.15.5:0/1391990373 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80d14e0 msgr2=0x7f5bd4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:41.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.867+0000 7f5be7fff640 1 --2- 172.21.15.5:0/1391990373 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80d14e0 0x7f5bd4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.867+0000 7f5be7fff640 1 -- 172.21.15.5:0/1391990373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5bd40039e0 msgr2=0x7f5bd4003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:41.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.867+0000 7f5be7fff640 1 --2- 172.21.15.5:0/1391990373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5bd40039e0 0x7f5bd4003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.867+0000 7f5be7fff640 1 -- 172.21.15.5:0/1391990373 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd4003520 con 0x7f5bd4004d40 2024-05-20T18:35:41.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.868+0000 7f5be7fff640 1 --2- 172.21.15.5:0/1391990373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bd4004d40 0x7f5bd4005150 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f5be806ceb0 tx=0x7f5bd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d82f61f03cd3cf43 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:41.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.868+0000 7f5be6ffd640 1 -- 172.21.15.5:0/1391990373 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be000a450 con 0x7f5bd4004d40 2024-05-20T18:35:41.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.868+0000 7f5be6ffd640 1 -- 172.21.15.5:0/1391990373 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5be002f070 con 0x7f5bd4004d40 2024-05-20T18:35:41.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.868+0000 7f5be6ffd640 1 -- 172.21.15.5:0/1391990373 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be002bc00 con 0x7f5bd4004d40 2024-05-20T18:35:41.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.869+0000 7f5bedc92640 1 -- 172.21.15.5:0/1391990373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bd4004d40 msgr2=0x7f5bd4005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:41.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.869+0000 7f5bedc92640 1 --2- 172.21.15.5:0/1391990373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bd4004d40 0x7f5bd4005150 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f5be806ceb0 tx=0x7f5bd4005830 comp rx=0 tx=0).stop 2024-05-20T18:35:41.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.869+0000 7f5be7fff640 1 -- 172.21.15.5:0/1391990373 reap_dead start 2024-05-20T18:35:41.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.869+0000 7f5bedc92640 1 -- 172.21.15.5:0/1391990373 shutdown_connections 2024-05-20T18:35:41.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.869+0000 7f5bedc92640 1 -- 172.21.15.5:0/1391990373 >> 172.21.15.5:0/1391990373 conn(0x7f5be8072ed0 msgr2=0x7f5be80b6080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:41.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.869+0000 7f5bedc92640 1 -- 172.21.15.5:0/1391990373 shutdown_connections 2024-05-20T18:35:41.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.869+0000 7f5bedc92640 1 -- 172.21.15.5:0/1391990373 wait complete. 2024-05-20T18:35:41.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.871+0000 7f5bedc92640 1 Processor -- start 2024-05-20T18:35:41.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.871+0000 7f5bedc92640 1 -- start start 2024-05-20T18:35:41.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.872+0000 7f5bedc92640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5bd4004d40 0x7f5be80cd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:41.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.872+0000 7f5bedc92640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be815eb30 0x7f5be8149820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:41.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.872+0000 7f5bedc92640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5be81f8710 0x7f5be81f8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:41.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.872+0000 7f5bedc92640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5be80d1ac0 con 0x7f5be81f8710 2024-05-20T18:35:41.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.872+0000 7f5bedc92640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5be8074d70 con 0x7f5bd4004d40 2024-05-20T18:35:41.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.872+0000 7f5bedc92640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5be809aa40 con 0x7f5be815eb30 2024-05-20T18:35:41.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.872+0000 7f5be7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5be81f8710 0x7f5be81f8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:41.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.873+0000 7f5be7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5be81f8710 0x7f5be81f8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49770/0 (socket says 172.21.15.5:49770) 2024-05-20T18:35:41.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.873+0000 7f5be7fff640 1 -- 172.21.15.5:0/4233529814 learned_addr learned my addr 172.21.15.5:0/4233529814 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:41.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.873+0000 7f5be77fe640 1 --2- 172.21.15.5:0/4233529814 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5bd4004d40 0x7f5be80cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:41.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.873+0000 7f5bdeffd640 1 --2- 172.21.15.5:0/4233529814 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be815eb30 0x7f5be8149820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:41.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.873+0000 7f5be7fff640 1 -- 172.21.15.5:0/4233529814 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be815eb30 msgr2=0x7f5be8149820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:41.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.873+0000 7f5be7fff640 1 --2- 172.21.15.5:0/4233529814 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be815eb30 0x7f5be8149820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.873+0000 7f5be7fff640 1 -- 172.21.15.5:0/4233529814 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5bd4004d40 msgr2=0x7f5be80cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:41.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.873+0000 7f5be7fff640 1 --2- 172.21.15.5:0/4233529814 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5bd4004d40 0x7f5be80cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:41.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.873+0000 7f5be7fff640 1 -- 172.21.15.5:0/4233529814 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be81f9060 con 0x7f5be81f8710 2024-05-20T18:35:41.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.873+0000 7f5be7fff640 1 --2- 172.21.15.5:0/4233529814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5be81f8710 0x7f5be81f8b20 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f5be002b470 tx=0x7f5be002d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:41.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.873+0000 7f5be4ff9640 1 -- 172.21.15.5:0/4233529814 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be000a450 con 0x7f5be81f8710 2024-05-20T18:35:41.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.873+0000 7f5be4ff9640 1 -- 172.21.15.5:0/4233529814 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5be0001250 con 0x7f5be81f8710 2024-05-20T18:35:41.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.873+0000 7f5bedc92640 1 -- 172.21.15.5:0/4233529814 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be8149d60 con 0x7f5be81f8710 2024-05-20T18:35:41.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.873+0000 7f5be4ff9640 1 -- 172.21.15.5:0/4233529814 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be0034670 con 0x7f5be81f8710 2024-05-20T18:35:41.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.873+0000 7f5bedc92640 1 -- 172.21.15.5:0/4233529814 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be80d00d0 con 0x7f5be81f8710 2024-05-20T18:35:41.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.874+0000 7f5bedc92640 1 -- 172.21.15.5:0/4233529814 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5be8075b00 con 0x7f5be81f8710 2024-05-20T18:35:41.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.875+0000 7f5be4ff9640 1 -- 172.21.15.5:0/4233529814 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5be0034810 con 0x7f5be81f8710 2024-05-20T18:35:41.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.875+0000 7f5be4ff9640 1 --2- 172.21.15.5:0/4233529814 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5bc806be60 0x7f5bc806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:41.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.875+0000 7f5be4ff9640 1 -- 172.21.15.5:0/4233529814 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7f5be00aafc0 con 0x7f5be81f8710 2024-05-20T18:35:41.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.876+0000 7f5be77fe640 1 --2- 172.21.15.5:0/4233529814 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5bc806be60 0x7f5bc806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:41.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.876+0000 7f5be77fe640 1 --2- 172.21.15.5:0/4233529814 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5bc806be60 0x7f5bc806e370 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5bd80018d0 tx=0x7f5bd8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:41.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:41.880+0000 7f5be4ff9640 1 -- 172.21.15.5:0/4233529814 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5be81f9060 con 0x7f5be81f8710 2024-05-20T18:35:42.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:42.102+0000 7f5bedc92640 1 -- 172.21.15.5:0/4233529814 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5be806fd80 con 0x7f5be81f8710 2024-05-20T18:35:42.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:42.103+0000 7f5be4ff9640 1 -- 172.21.15.5:0/4233529814 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) ==== 53+0+73 (secure 0 0 0) 0x7f5be0079810 con 0x7f5be81f8710 2024-05-20T18:35:42.104 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-20T18:35:42.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:42.104+0000 7f5bedc92640 1 -- 172.21.15.5:0/4233529814 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5bc806be60 msgr2=0x7f5bc806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:42.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:42.104+0000 7f5bedc92640 1 --2- 172.21.15.5:0/4233529814 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5bc806be60 0x7f5bc806e370 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5bd80018d0 tx=0x7f5bd8009040 comp rx=0 tx=0).stop 2024-05-20T18:35:42.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:42.104+0000 7f5bedc92640 1 -- 172.21.15.5:0/4233529814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5be81f8710 msgr2=0x7f5be81f8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:42.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:42.104+0000 7f5bedc92640 1 --2- 172.21.15.5:0/4233529814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5be81f8710 0x7f5be81f8b20 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f5be002b470 tx=0x7f5be002d070 comp rx=0 tx=0).stop 2024-05-20T18:35:42.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:42.105+0000 7f5bedc92640 1 -- 172.21.15.5:0/4233529814 shutdown_connections 2024-05-20T18:35:42.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:42.105+0000 7f5bedc92640 1 --2- 172.21.15.5:0/4233529814 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5bc806be60 0x7f5bc806e370 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:42.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:42.105+0000 7f5bedc92640 1 --2- 172.21.15.5:0/4233529814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5be81f8710 0x7f5be81f8b20 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:42.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:42.105+0000 7f5bedc92640 1 --2- 172.21.15.5:0/4233529814 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be815eb30 0x7f5be8149820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:42.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:42.105+0000 7f5bedc92640 1 --2- 172.21.15.5:0/4233529814 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5bd4004d40 0x7f5be80cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:42.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:42.105+0000 7f5bedc92640 1 -- 172.21.15.5:0/4233529814 >> 172.21.15.5:0/4233529814 conn(0x7f5be8072ed0 msgr2=0x7f5be8067be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:42.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:42.105+0000 7f5bedc92640 1 -- 172.21.15.5:0/4233529814 shutdown_connections 2024-05-20T18:35:42.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:42.105+0000 7f5bedc92640 1 -- 172.21.15.5:0/4233529814 wait complete. 2024-05-20T18:35:42.117 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-20T18:35:42.117 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-20T18:35:42.118 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-05-20T18:35:42.118 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-20T18:35:42.118 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-20T18:35:42.118 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-20T18:35:42.118 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-20T18:35:42.118 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-20T18:35:42.118 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:35:42.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:35:42.118 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-05-20T18:35:42.135 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T18:35:42.135 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T18:35:42.135 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T18:35:42.135 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:35:42.135 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-20T18:35:42.135 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:35:42.135 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:35:42.135 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-05-20T18:35:42.135 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-05-20T18:35:42.136 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:35:42.136 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-05-20T18:35:42.136 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-05-20T18:35:42.136 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-20T18:35:42.136 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:35:42.136 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-05-20T18:35:42.136 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-20T18:35:42.136 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42156sec preferred_lft 42156sec 2024-05-20T18:35:42.136 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-05-20T18:35:42.136 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:35:42.136 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:35:42.136 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:35:42.136 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-05-20T18:35:42.136 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-20T18:35:42.137 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:35:42.137 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-20T18:35:42.137 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:35:42.137 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::4474:eaff:fe07:4f9a/64 scope link 2024-05-20T18:35:42.137 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:35:42.137 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:35:42.137 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T18:35:42.137 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ce4:f4ff:fe5e:7906/64 scope link 2024-05-20T18:35:42.137 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:35:42.137 INFO:teuthology.orchestra.run.smithi005.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:35:42.137 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether da:26:04:0e:95:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T18:35:42.137 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::d826:4ff:fe0e:950a/64 scope link 2024-05-20T18:35:42.137 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:35:42.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:35:42.138 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-20T18:35:42.199 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-20T18:35:42.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:35:42.201 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-05-20T18:35:42.257 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-05-20T18:35:42.258 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-20T18:35:42.258 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-20T18:35:42.258 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-20T18:35:42.258 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-20T18:35:42.258 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:35:42.258 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-20T18:35:42.258 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-20T18:35:42.258 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-20T18:35:42.258 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-20T18:35:42.259 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-20T18:35:42.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:35:42.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:35:42.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:35:42.389 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-20T18:35:42.405 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-20T18:35:42.405 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-20T18:35:42.405 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:35:42.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:35:42.405 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-05-20T18:35:42.472 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:35:42.472 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-20T18:35:42.527 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:35:42.528 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:35:42.528 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:35:42.583 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-20T18:35:42.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:35:42.583 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:35:42.680 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:35:42.680 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:35:42.680 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:35:42.691 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:35:42.691 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:35:42.691 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:35:42.691 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:35:42.691 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:35:42.699 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:35:42.699 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:35:42.699 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:35:42.699 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:35:42.699 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:35:42.699 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790" 2024-05-20T18:35:42.699 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790:/ 2024-05-20T18:35:42.699 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=399ab250-fd3c-47fa-a1c8-25e8149c9cac" will pass to kernel 2024-05-20T18:35:42.700 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-20T18:35:42.700 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-05-20T18:35:42.700 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-20T18:35:42.700 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-05-20T18:35:42.700 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-20T18:35:42.702 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-20T18:35:42.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:35:42.702 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:35:42.729 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-20T18:35:42.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:42 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T18:35:42.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:35:42.834 INFO:teuthology.orchestra.run.smithi005.stdout:client7669 2024-05-20T18:35:42.834 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:35:42.834 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:35:42.851 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:35:42.851 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:35:42.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:35:42.907 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-05-20T18:35:43.005 INFO:teuthology.orchestra.run.smithi005.stderr:6+0 records in 2024-05-20T18:35:43.005 INFO:teuthology.orchestra.run.smithi005.stderr:6+0 records out 2024-05-20T18:35:43.005 INFO:teuthology.orchestra.run.smithi005.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0365604 s, 172 MB/s 2024-05-20T18:35:43.006 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:35:43.006 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:35:43.022 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:35:43.022 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:35:43.079 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:35:43.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.233+0000 7fe78ef8b640 1 Processor -- start 2024-05-20T18:35:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.233+0000 7fe78ef8b640 1 -- start start 2024-05-20T18:35:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.235+0000 7fe78ef8b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe788162820 0x7fe7880b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.235+0000 7fe78ef8b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe7880b3e80 0x7fe78806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.235+0000 7fe78ef8b640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe78806dd20 con 0x7fe78806d910 2024-05-20T18:35:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.235+0000 7fe78ef8b640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe7880d0630 con 0x7fe788162820 2024-05-20T18:35:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.235+0000 7fe78ef8b640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe7880d39c0 con 0x7fe7880b3e80 2024-05-20T18:35:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.235+0000 7fe78cd27640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe788162820 0x7fe7880b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.235+0000 7fe78cd27640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe788162820 0x7fe7880b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49784/0 (socket says 172.21.15.5:49784) 2024-05-20T18:35:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.235+0000 7fe78cd27640 1 -- 172.21.15.5:0/594030495 learned_addr learned my addr 172.21.15.5:0/594030495 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.235+0000 7fe78d528640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe78806d910 0x7fe7880d27a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46742/0 (socket says 172.21.15.5:46742) 2024-05-20T18:35:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.235+0000 7fe78cd27640 1 -- 172.21.15.5:0/594030495 >> v2:172.21.15.63:3300/0 conn(0x7fe7880b3e80 msgr2=0x7fe78806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.235+0000 7fe787fff640 1 --2- 172.21.15.5:0/594030495 >> v2:172.21.15.63:3300/0 conn(0x7fe7880b3e80 0x7fe78806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.235+0000 7fe7877fe640 1 -- 172.21.15.5:0/594030495 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 23146410 0 0) 0x7fe780001490 con 0x7fe78806d910 2024-05-20T18:35:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.235+0000 7fe78cd27640 1 --2- 172.21.15.5:0/594030495 >> v2:172.21.15.63:3300/0 conn(0x7fe7880b3e80 0x7fe78806d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.235+0000 7fe78cd27640 1 -- 172.21.15.5:0/594030495 >> v1:172.21.15.5:6789/0 conn(0x7fe78806d910 legacy=0x7fe7880d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.235+0000 7fe78cd27640 1 -- 172.21.15.5:0/594030495 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7880d3ca0 con 0x7fe788162820 2024-05-20T18:35:43.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.235+0000 7fe787fff640 1 --2- 172.21.15.5:0/594030495 >> v2:172.21.15.63:3300/0 conn(0x7fe7880b3e80 0x7fe78806d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.236+0000 7fe78cd27640 1 --2- 172.21.15.5:0/594030495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe788162820 0x7fe7880b3940 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fe7880d1230 tx=0x7fe77802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa56de24d0e13ae server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.236+0000 7fe7877fe640 1 -- 172.21.15.5:0/594030495 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe778037070 con 0x7fe788162820 2024-05-20T18:35:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.236+0000 7fe7877fe640 1 -- 172.21.15.5:0/594030495 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe778032440 con 0x7fe788162820 2024-05-20T18:35:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.236+0000 7fe7877fe640 1 -- 172.21.15.5:0/594030495 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe778032810 con 0x7fe788162820 2024-05-20T18:35:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.237+0000 7fe78ef8b640 1 -- 172.21.15.5:0/594030495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe788162820 msgr2=0x7fe7880b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.237+0000 7fe78ef8b640 1 --2- 172.21.15.5:0/594030495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe788162820 0x7fe7880b3940 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fe7880d1230 tx=0x7fe77802a270 comp rx=0 tx=0).stop 2024-05-20T18:35:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.237+0000 7fe78ef8b640 1 -- 172.21.15.5:0/594030495 shutdown_connections 2024-05-20T18:35:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.237+0000 7fe78ef8b640 1 --2- 172.21.15.5:0/594030495 >> v2:172.21.15.63:3300/0 conn(0x7fe7880b3e80 0x7fe78806d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.237+0000 7fe78ef8b640 1 --2- 172.21.15.5:0/594030495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe788162820 0x7fe7880b3940 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.237+0000 7fe78ef8b640 1 -- 172.21.15.5:0/594030495 >> 172.21.15.5:0/594030495 conn(0x7fe7880d4bd0 msgr2=0x7fe7880d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.237+0000 7fe78ef8b640 1 -- 172.21.15.5:0/594030495 shutdown_connections 2024-05-20T18:35:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.237+0000 7fe78ef8b640 1 -- 172.21.15.5:0/594030495 wait complete. 2024-05-20T18:35:43.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.239+0000 7fe78ef8b640 1 Processor -- start 2024-05-20T18:35:43.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.239+0000 7fe78ef8b640 1 -- start start 2024-05-20T18:35:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.240+0000 7fe78ef8b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe78806d910 0x7fe7880c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.240+0000 7fe78ef8b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7880b3e80 0x7fe7881e7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.240+0000 7fe78ef8b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe788162820 0x7fe78814ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.240+0000 7fe78ef8b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe7880d0300 con 0x7fe788162820 2024-05-20T18:35:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.240+0000 7fe78ef8b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe7880d4260 con 0x7fe78806d910 2024-05-20T18:35:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.240+0000 7fe78ef8b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe7880b91a0 con 0x7fe7880b3e80 2024-05-20T18:35:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.240+0000 7fe78d528640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe788162820 0x7fe78814ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.240+0000 7fe78d528640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe788162820 0x7fe78814ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49794/0 (socket says 172.21.15.5:49794) 2024-05-20T18:35:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.240+0000 7fe78d528640 1 -- 172.21.15.5:0/205937704 learned_addr learned my addr 172.21.15.5:0/205937704 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.240+0000 7fe78cd27640 1 --2- 172.21.15.5:0/205937704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe78806d910 0x7fe7880c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.240+0000 7fe787fff640 1 --2- 172.21.15.5:0/205937704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7880b3e80 0x7fe7881e7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.241+0000 7fe78d528640 1 -- 172.21.15.5:0/205937704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7880b3e80 msgr2=0x7fe7881e7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.241+0000 7fe78d528640 1 --2- 172.21.15.5:0/205937704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7880b3e80 0x7fe7881e7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.241+0000 7fe78d528640 1 -- 172.21.15.5:0/205937704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe78806d910 msgr2=0x7fe7880c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.241+0000 7fe78d528640 1 --2- 172.21.15.5:0/205937704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe78806d910 0x7fe7880c88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.241+0000 7fe78d528640 1 -- 172.21.15.5:0/205937704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7881e8600 con 0x7fe788162820 2024-05-20T18:35:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.241+0000 7fe78cd27640 1 --2- 172.21.15.5:0/205937704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe78806d910 0x7fe7880c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.241+0000 7fe78d528640 1 --2- 172.21.15.5:0/205937704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe788162820 0x7fe78814ddf0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fe78000c700 tx=0x7fe78000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.241+0000 7fe7857fa640 1 -- 172.21.15.5:0/205937704 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe780018070 con 0x7fe788162820 2024-05-20T18:35:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.241+0000 7fe7857fa640 1 -- 172.21.15.5:0/205937704 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe78000ce10 con 0x7fe788162820 2024-05-20T18:35:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.241+0000 7fe78ef8b640 1 -- 172.21.15.5:0/205937704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7881e88e0 con 0x7fe788162820 2024-05-20T18:35:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.241+0000 7fe7857fa640 1 -- 172.21.15.5:0/205937704 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78000a880 con 0x7fe788162820 2024-05-20T18:35:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.241+0000 7fe78ef8b640 1 -- 172.21.15.5:0/205937704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7881e8ea0 con 0x7fe788162820 2024-05-20T18:35:43.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.242+0000 7fe78ef8b640 1 -- 172.21.15.5:0/205937704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7881e8370 con 0x7fe788162820 2024-05-20T18:35:43.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.243+0000 7fe7857fa640 1 -- 172.21.15.5:0/205937704 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe78000aa20 con 0x7fe788162820 2024-05-20T18:35:43.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.243+0000 7fe7857fa640 1 --2- 172.21.15.5:0/205937704 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe77406be60 0x7fe77406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:43.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.243+0000 7fe7857fa640 1 -- 172.21.15.5:0/205937704 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7fe78008a5b0 con 0x7fe788162820 2024-05-20T18:35:43.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.243+0000 7fe78cd27640 1 --2- 172.21.15.5:0/205937704 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe77406be60 0x7fe77406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:43.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.244+0000 7fe78cd27640 1 --2- 172.21.15.5:0/205937704 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe77406be60 0x7fe77406e370 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe778034000 tx=0x7fe77802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:43.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.248+0000 7fe7857fa640 1 -- 172.21.15.5:0/205937704 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe78806dd20 con 0x7fe788162820 2024-05-20T18:35:43.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.435+0000 7fe78ef8b640 1 -- 172.21.15.5:0/205937704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe7881e9280 con 0x7fe788162820 2024-05-20T18:35:43.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.436+0000 7fe7857fa640 1 -- 172.21.15.5:0/205937704 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+16764 (secure 0 0 0) 0x7fe780058cc0 con 0x7fe788162820 2024-05-20T18:35:43.438 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:43.438 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":105,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:35:32.410574+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:35:29.146063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-20T18:35:29.754813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:35:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.438+0000 7fe78ef8b640 1 -- 172.21.15.5:0/205937704 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe77406be60 msgr2=0x7fe77406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.438+0000 7fe78ef8b640 1 --2- 172.21.15.5:0/205937704 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe77406be60 0x7fe77406e370 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe778034000 tx=0x7fe77802e000 comp rx=0 tx=0).stop 2024-05-20T18:35:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.438+0000 7fe78ef8b640 1 -- 172.21.15.5:0/205937704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe788162820 msgr2=0x7fe78814ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.438+0000 7fe78ef8b640 1 --2- 172.21.15.5:0/205937704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe788162820 0x7fe78814ddf0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fe78000c700 tx=0x7fe78000cbc0 comp rx=0 tx=0).stop 2024-05-20T18:35:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.439+0000 7fe78ef8b640 1 -- 172.21.15.5:0/205937704 shutdown_connections 2024-05-20T18:35:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.439+0000 7fe78ef8b640 1 --2- 172.21.15.5:0/205937704 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe77406be60 0x7fe77406e370 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.439+0000 7fe78ef8b640 1 --2- 172.21.15.5:0/205937704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe788162820 0x7fe78814ddf0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.439+0000 7fe78ef8b640 1 --2- 172.21.15.5:0/205937704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7880b3e80 0x7fe7881e7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.439+0000 7fe78ef8b640 1 --2- 172.21.15.5:0/205937704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe78806d910 0x7fe7880c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.439+0000 7fe78ef8b640 1 -- 172.21.15.5:0/205937704 >> 172.21.15.5:0/205937704 conn(0x7fe7880d4bd0 msgr2=0x7fe7880b4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:43.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.439+0000 7fe78ef8b640 1 -- 172.21.15.5:0/205937704 shutdown_connections 2024-05-20T18:35:43.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.439+0000 7fe78ef8b640 1 -- 172.21.15.5:0/205937704 wait complete. 2024-05-20T18:35:43.452 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:35:29.146063+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:35:29.754813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:35:43.452 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:35:43.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.569+0000 7fe87a4e2640 1 Processor -- start 2024-05-20T18:35:43.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.569+0000 7fe87a4e2640 1 -- start start 2024-05-20T18:35:43.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.570+0000 7fe87a4e2640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe874168050 0x7fe87416a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:43.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.570+0000 7fe87a4e2640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8740d0630 con 0x7fe874162820 2024-05-20T18:35:43.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.570+0000 7fe87a4e2640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8740d39c0 con 0x7fe87406d3d0 2024-05-20T18:35:43.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.570+0000 7fe87a4e2640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe8740d3ca0 con 0x7fe874168050 2024-05-20T18:35:43.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.570+0000 7fe873fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe874162820 0x7fe8740b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46758/0 (socket says 172.21.15.5:46758) 2024-05-20T18:35:43.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.570+0000 7fe873fff640 1 -- 172.21.15.5:0/3688870626 learned_addr learned my addr 172.21.15.5:0/3688870626 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:43.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.570+0000 7fe878a7f640 1 --2- 172.21.15.5:0/3688870626 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe874168050 0x7fe87416a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:43.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.570+0000 7fe872ffd640 1 -- 172.21.15.5:0/3688870626 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1153336645 0 0) 0x7fe8740d0630 con 0x7fe874162820 2024-05-20T18:35:43.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.570+0000 7fe872ffd640 1 -- 172.21.15.5:0/3688870626 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe85c003520 con 0x7fe874162820 2024-05-20T18:35:43.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.571+0000 7fe872ffd640 1 -- 172.21.15.5:0/3688870626 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 410635366 0 0) 0x7fe85c003520 con 0x7fe874162820 2024-05-20T18:35:43.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.571+0000 7fe872ffd640 1 -- 172.21.15.5:0/3688870626 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe874168050 msgr2=0x7fe87416a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:43.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.571+0000 7fe872ffd640 1 --2- 172.21.15.5:0/3688870626 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe874168050 0x7fe87416a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:43.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.571+0000 7fe872ffd640 1 -- 172.21.15.5:0/3688870626 >> v1:172.21.15.63:6789/0 conn(0x7fe87406d3d0 legacy=0x7fe87406d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:43.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.571+0000 7fe872ffd640 1 -- 172.21.15.5:0/3688870626 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8740d3f80 con 0x7fe874162820 2024-05-20T18:35:43.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.571+0000 7fe872ffd640 1 -- 172.21.15.5:0/3688870626 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7fe864001670 con 0x7fe874162820 2024-05-20T18:35:43.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.571+0000 7fe872ffd640 1 -- 172.21.15.5:0/3688870626 >> v1:172.21.15.5:6789/0 conn(0x7fe874162820 legacy=0x7fe8740b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:43.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.571+0000 7fe872ffd640 1 --2- 172.21.15.5:0/3688870626 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe874168050 0x7fe85c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:43.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.571+0000 7fe872ffd640 1 --2- 172.21.15.5:0/3688870626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe85c003f40 0x7fe85c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:43.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.571+0000 7fe872ffd640 1 --2- 172.21.15.5:0/3688870626 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe85c004d40 0x7fe85c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:43.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.571+0000 7fe872ffd640 1 -- 172.21.15.5:0/3688870626 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe8740d3ca0 con 0x7fe85c003f40 2024-05-20T18:35:43.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.571+0000 7fe872ffd640 1 -- 172.21.15.5:0/3688870626 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c00bad0 con 0x7fe85c004d40 2024-05-20T18:35:43.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.571+0000 7fe872ffd640 1 -- 172.21.15.5:0/3688870626 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe85c00bcd0 con 0x7fe874168050 2024-05-20T18:35:43.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.571+0000 7fe8737fe640 1 --2- 172.21.15.5:0/3688870626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe85c003f40 0x7fe85c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:43.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.571+0000 7fe873fff640 1 --2- 172.21.15.5:0/3688870626 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe874168050 0x7fe85c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:43.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.571+0000 7fe878a7f640 1 --2- 172.21.15.5:0/3688870626 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe85c004d40 0x7fe85c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:43.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.571+0000 7fe8737fe640 1 -- 172.21.15.5:0/3688870626 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe874168050 msgr2=0x7fe85c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:43.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.571+0000 7fe8737fe640 1 --2- 172.21.15.5:0/3688870626 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe874168050 0x7fe85c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:43.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.571+0000 7fe8737fe640 1 -- 172.21.15.5:0/3688870626 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe85c004d40 msgr2=0x7fe85c005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:43.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.571+0000 7fe8737fe640 1 --2- 172.21.15.5:0/3688870626 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe85c004d40 0x7fe85c005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:43.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.571+0000 7fe8737fe640 1 -- 172.21.15.5:0/3688870626 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe85c003520 con 0x7fe85c003f40 2024-05-20T18:35:43.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.571+0000 7fe878a7f640 1 --2- 172.21.15.5:0/3688870626 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe85c004d40 0x7fe85c005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:43.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.572+0000 7fe8737fe640 1 --2- 172.21.15.5:0/3688870626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe85c003f40 0x7fe85c004800 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fe868001860 tx=0x7fe85c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54ecef05cb1863 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:43.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.572+0000 7fe872ffd640 1 -- 172.21.15.5:0/3688870626 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe868037070 con 0x7fe85c003f40 2024-05-20T18:35:43.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.572+0000 7fe872ffd640 1 -- 172.21.15.5:0/3688870626 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe86802abf0 con 0x7fe85c003f40 2024-05-20T18:35:43.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.572+0000 7fe872ffd640 1 -- 172.21.15.5:0/3688870626 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe868001430 con 0x7fe85c003f40 2024-05-20T18:35:43.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.573+0000 7fe87a4e2640 1 -- 172.21.15.5:0/3688870626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe85c003f40 msgr2=0x7fe85c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:43.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.573+0000 7fe87a4e2640 1 --2- 172.21.15.5:0/3688870626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe85c003f40 0x7fe85c004800 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fe868001860 tx=0x7fe85c005830 comp rx=0 tx=0).stop 2024-05-20T18:35:43.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.573+0000 7fe878a7f640 1 -- 172.21.15.5:0/3688870626 reap_dead start 2024-05-20T18:35:43.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.573+0000 7fe87a4e2640 1 -- 172.21.15.5:0/3688870626 shutdown_connections 2024-05-20T18:35:43.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.573+0000 7fe87a4e2640 1 -- 172.21.15.5:0/3688870626 >> 172.21.15.5:0/3688870626 conn(0x7fe8740d4bd0 msgr2=0x7fe8740d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:43.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.573+0000 7fe87a4e2640 1 -- 172.21.15.5:0/3688870626 shutdown_connections 2024-05-20T18:35:43.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.573+0000 7fe87a4e2640 1 -- 172.21.15.5:0/3688870626 wait complete. 2024-05-20T18:35:43.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.575+0000 7fe87a4e2640 1 Processor -- start 2024-05-20T18:35:43.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.575+0000 7fe87a4e2640 1 -- start start 2024-05-20T18:35:43.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.576+0000 7fe87a4e2640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe85c003f40 0x7fe874157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:43.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.576+0000 7fe87a4e2640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe874157760 0x7fe8740ce360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:43.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.576+0000 7fe87a4e2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8740ce8a0 0x7fe8740cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:43.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.576+0000 7fe87a4e2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe8740b91a0 con 0x7fe8740ce8a0 2024-05-20T18:35:43.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.576+0000 7fe87a4e2640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe8740d0300 con 0x7fe85c003f40 2024-05-20T18:35:43.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.576+0000 7fe87a4e2640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe8740b4050 con 0x7fe874157760 2024-05-20T18:35:43.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.576+0000 7fe878a7f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8740ce8a0 0x7fe8740cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:43.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.576+0000 7fe878a7f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8740ce8a0 0x7fe8740cc9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49802/0 (socket says 172.21.15.5:49802) 2024-05-20T18:35:43.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.576+0000 7fe878a7f640 1 -- 172.21.15.5:0/775009635 learned_addr learned my addr 172.21.15.5:0/775009635 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:43.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.577+0000 7fe873fff640 1 --2- 172.21.15.5:0/775009635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe85c003f40 0x7fe874157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:43.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.577+0000 7fe8737fe640 1 --2- 172.21.15.5:0/775009635 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe874157760 0x7fe8740ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:43.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.577+0000 7fe878a7f640 1 -- 172.21.15.5:0/775009635 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe874157760 msgr2=0x7fe8740ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:43.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.577+0000 7fe878a7f640 1 --2- 172.21.15.5:0/775009635 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe874157760 0x7fe8740ce360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:43.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.577+0000 7fe878a7f640 1 -- 172.21.15.5:0/775009635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe85c003f40 msgr2=0x7fe874157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:43.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.577+0000 7fe878a7f640 1 --2- 172.21.15.5:0/775009635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe85c003f40 0x7fe874157220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:43.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.577+0000 7fe878a7f640 1 -- 172.21.15.5:0/775009635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe874157b70 con 0x7fe8740ce8a0 2024-05-20T18:35:43.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.577+0000 7fe878a7f640 1 --2- 172.21.15.5:0/775009635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8740ce8a0 0x7fe8740cc9f0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fe87408e090 tx=0x7fe86c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:43.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.577+0000 7fe870ff9640 1 -- 172.21.15.5:0/775009635 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c011e60 con 0x7fe8740ce8a0 2024-05-20T18:35:43.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.577+0000 7fe870ff9640 1 -- 172.21.15.5:0/775009635 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe86c01c070 con 0x7fe8740ce8a0 2024-05-20T18:35:43.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.577+0000 7fe87a4e2640 1 -- 172.21.15.5:0/775009635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8741591a0 con 0x7fe8740ce8a0 2024-05-20T18:35:43.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.577+0000 7fe870ff9640 1 -- 172.21.15.5:0/775009635 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c0166b0 con 0x7fe8740ce8a0 2024-05-20T18:35:43.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.577+0000 7fe87a4e2640 1 -- 172.21.15.5:0/775009635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe874159760 con 0x7fe8740ce8a0 2024-05-20T18:35:43.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.578+0000 7fe87a4e2640 1 -- 172.21.15.5:0/775009635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe874075b00 con 0x7fe8740ce8a0 2024-05-20T18:35:43.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.579+0000 7fe870ff9640 1 -- 172.21.15.5:0/775009635 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe86c005d10 con 0x7fe8740ce8a0 2024-05-20T18:35:43.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.579+0000 7fe870ff9640 1 --2- 172.21.15.5:0/775009635 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe86006be30 0x7fe86006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:43.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.579+0000 7fe870ff9640 1 -- 172.21.15.5:0/775009635 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7fe86c08b7c0 con 0x7fe8740ce8a0 2024-05-20T18:35:43.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.579+0000 7fe873fff640 1 --2- 172.21.15.5:0/775009635 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe86006be30 0x7fe86006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:43.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.580+0000 7fe873fff640 1 --2- 172.21.15.5:0/775009635 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe86006be30 0x7fe86006e340 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe864007cb0 tx=0x7fe864009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:43.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.584+0000 7fe870ff9640 1 -- 172.21.15.5:0/775009635 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe874157b70 con 0x7fe8740ce8a0 2024-05-20T18:35:43.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.772+0000 7fe87a4e2640 1 -- 172.21.15.5:0/775009635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe8740b3620 con 0x7fe8740ce8a0 2024-05-20T18:35:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.774+0000 7fe870ff9640 1 -- 172.21.15.5:0/775009635 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) ==== 93+0+10 (secure 0 0 0) 0x7fe86c059f50 con 0x7fe8740ce8a0 2024-05-20T18:35:43.778 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:35:43.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.777+0000 7fe87a4e2640 1 -- 172.21.15.5:0/775009635 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe86006be30 msgr2=0x7fe86006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:43.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.777+0000 7fe87a4e2640 1 --2- 172.21.15.5:0/775009635 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe86006be30 0x7fe86006e340 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe864007cb0 tx=0x7fe864009040 comp rx=0 tx=0).stop 2024-05-20T18:35:43.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.777+0000 7fe87a4e2640 1 -- 172.21.15.5:0/775009635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8740ce8a0 msgr2=0x7fe8740cc9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:43.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.777+0000 7fe87a4e2640 1 --2- 172.21.15.5:0/775009635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8740ce8a0 0x7fe8740cc9f0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fe87408e090 tx=0x7fe86c009f90 comp rx=0 tx=0).stop 2024-05-20T18:35:43.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.777+0000 7fe87a4e2640 1 -- 172.21.15.5:0/775009635 shutdown_connections 2024-05-20T18:35:43.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.777+0000 7fe87a4e2640 1 --2- 172.21.15.5:0/775009635 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe86006be30 0x7fe86006e340 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:43.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.777+0000 7fe87a4e2640 1 --2- 172.21.15.5:0/775009635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8740ce8a0 0x7fe8740cc9f0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:43.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.777+0000 7fe87a4e2640 1 --2- 172.21.15.5:0/775009635 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe874157760 0x7fe8740ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:43.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.777+0000 7fe87a4e2640 1 --2- 172.21.15.5:0/775009635 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe85c003f40 0x7fe874157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:43.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.777+0000 7fe87a4e2640 1 -- 172.21.15.5:0/775009635 >> 172.21.15.5:0/775009635 conn(0x7fe8740d4bd0 msgr2=0x7fe87415eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:43.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.777+0000 7fe87a4e2640 1 -- 172.21.15.5:0/775009635 shutdown_connections 2024-05-20T18:35:43.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.777+0000 7fe87a4e2640 1 -- 172.21.15.5:0/775009635 wait complete. 2024-05-20T18:35:43.832 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:35:43.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.949+0000 7f32df123640 1 Processor -- start 2024-05-20T18:35:43.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.949+0000 7f32df123640 1 -- start start 2024-05-20T18:35:43.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.950+0000 7f32df123640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f32d80af330 0x7f32d80af740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:43.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.950+0000 7f32df123640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32d806c840 con 0x7f32d80d1610 2024-05-20T18:35:43.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.950+0000 7f32df123640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32d80d20d0 con 0x7f32d80b2560 2024-05-20T18:35:43.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.950+0000 7f32df123640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f32d806cae0 con 0x7f32d80af330 2024-05-20T18:35:43.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.950+0000 7f32d7fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f32d80d1610 0x7f32d80b1e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46770/0 (socket says 172.21.15.5:46770) 2024-05-20T18:35:43.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.950+0000 7f32d7fff640 1 -- 172.21.15.5:0/1496946856 learned_addr learned my addr 172.21.15.5:0/1496946856 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:43.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.951+0000 7f32dcebf640 1 --2- 172.21.15.5:0/1496946856 >> v2:172.21.15.63:3300/0 conn(0x7f32d80af330 0x7f32d80af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:43.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.951+0000 7f32d77fe640 1 -- 172.21.15.5:0/1496946856 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1354064948 0 0) 0x7f32d806c840 con 0x7f32d80d1610 2024-05-20T18:35:43.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.951+0000 7f32d77fe640 1 -- 172.21.15.5:0/1496946856 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32c4003520 con 0x7f32d80d1610 2024-05-20T18:35:43.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.951+0000 7f32d77fe640 1 -- 172.21.15.5:0/1496946856 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1483131298 0 0) 0x7f32c4003520 con 0x7f32d80d1610 2024-05-20T18:35:43.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.951+0000 7f32d77fe640 1 -- 172.21.15.5:0/1496946856 >> v2:172.21.15.63:3300/0 conn(0x7f32d80af330 msgr2=0x7f32d80af740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:43.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.951+0000 7f32d77fe640 1 --2- 172.21.15.5:0/1496946856 >> v2:172.21.15.63:3300/0 conn(0x7f32d80af330 0x7f32d80af740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:43.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.951+0000 7f32d77fe640 1 -- 172.21.15.5:0/1496946856 >> v1:172.21.15.63:6789/0 conn(0x7f32d80b2560 legacy=0x7f32d806d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:43.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.951+0000 7f32d77fe640 1 -- 172.21.15.5:0/1496946856 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d806cdc0 con 0x7f32d80d1610 2024-05-20T18:35:43.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.951+0000 7f32d77fe640 1 -- 172.21.15.5:0/1496946856 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f32c0001a10 con 0x7f32d80d1610 2024-05-20T18:35:43.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.951+0000 7f32d77fe640 1 -- 172.21.15.5:0/1496946856 >> v1:172.21.15.5:6789/0 conn(0x7f32d80d1610 legacy=0x7f32d80b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:43.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.951+0000 7f32d77fe640 1 --2- 172.21.15.5:0/1496946856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32c40039b0 0x7f32c4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:43.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.951+0000 7f32d77fe640 1 --2- 172.21.15.5:0/1496946856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32c4004800 0x7f32c4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:43.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.951+0000 7f32d77fe640 1 --2- 172.21.15.5:0/1496946856 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32c4005250 0x7f32c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:43.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.951+0000 7f32d77fe640 1 -- 172.21.15.5:0/1496946856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f32d806cae0 con 0x7f32c40039b0 2024-05-20T18:35:43.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.951+0000 7f32d77fe640 1 -- 172.21.15.5:0/1496946856 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f32c400c1e0 con 0x7f32c4004800 2024-05-20T18:35:43.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.951+0000 7f32d77fe640 1 -- 172.21.15.5:0/1496946856 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f32c400c3e0 con 0x7f32c4005250 2024-05-20T18:35:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.951+0000 7f32dcebf640 1 --2- 172.21.15.5:0/1496946856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32c40039b0 0x7f32c4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.951+0000 7f32d7fff640 1 --2- 172.21.15.5:0/1496946856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32c4004800 0x7f32c4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.951+0000 7f32dd6c0640 1 --2- 172.21.15.5:0/1496946856 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32c4005250 0x7f32c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.951+0000 7f32dcebf640 1 -- 172.21.15.5:0/1496946856 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32c4005250 msgr2=0x7f32c400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.951+0000 7f32dcebf640 1 --2- 172.21.15.5:0/1496946856 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32c4005250 0x7f32c400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.951+0000 7f32dcebf640 1 -- 172.21.15.5:0/1496946856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32c4004800 msgr2=0x7f32c4004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.951+0000 7f32dcebf640 1 --2- 172.21.15.5:0/1496946856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32c4004800 0x7f32c4004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.951+0000 7f32dcebf640 1 -- 172.21.15.5:0/1496946856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c4003520 con 0x7f32c40039b0 2024-05-20T18:35:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.951+0000 7f32dd6c0640 1 -- 172.21.15.5:0/1496946856 reap_dead start 2024-05-20T18:35:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.951+0000 7f32d7fff640 1 --2- 172.21.15.5:0/1496946856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32c4004800 0x7f32c4004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.952+0000 7f32dcebf640 1 --2- 172.21.15.5:0/1496946856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32c40039b0 0x7f32c4003e20 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f32d809b570 tx=0x7f32c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9baf2628212a2c72 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.952+0000 7f32d77fe640 1 -- 172.21.15.5:0/1496946856 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d0036070 con 0x7f32c40039b0 2024-05-20T18:35:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.952+0000 7f32d77fe640 1 -- 172.21.15.5:0/1496946856 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32d0031400 con 0x7f32c40039b0 2024-05-20T18:35:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.952+0000 7f32d77fe640 1 -- 172.21.15.5:0/1496946856 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d0009690 con 0x7f32c40039b0 2024-05-20T18:35:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.953+0000 7f32df123640 1 -- 172.21.15.5:0/1496946856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32c40039b0 msgr2=0x7f32c4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:43.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.953+0000 7f32df123640 1 --2- 172.21.15.5:0/1496946856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32c40039b0 0x7f32c4003e20 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f32d809b570 tx=0x7f32c4005830 comp rx=0 tx=0).stop 2024-05-20T18:35:43.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.953+0000 7f32df123640 1 -- 172.21.15.5:0/1496946856 shutdown_connections 2024-05-20T18:35:43.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.953+0000 7f32df123640 1 --2- 172.21.15.5:0/1496946856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32c40039b0 0x7f32c4003e20 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:43.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.953+0000 7f32df123640 1 -- 172.21.15.5:0/1496946856 >> 172.21.15.5:0/1496946856 conn(0x7f32d80b6670 msgr2=0x7f32d80b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:43.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.953+0000 7f32df123640 1 -- 172.21.15.5:0/1496946856 shutdown_connections 2024-05-20T18:35:43.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.953+0000 7f32df123640 1 -- 172.21.15.5:0/1496946856 wait complete. 2024-05-20T18:35:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.955+0000 7f32df123640 1 Processor -- start 2024-05-20T18:35:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.955+0000 7f32df123640 1 -- start start 2024-05-20T18:35:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.956+0000 7f32df123640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32c40039b0 0x7f32d81500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.957+0000 7f32df123640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32d814c130 0x7f32d80cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.957+0000 7f32df123640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32d814c540 0x7f32d815b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.957+0000 7f32df123640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f32d8159d30 con 0x7f32c40039b0 2024-05-20T18:35:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.957+0000 7f32df123640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f32d815d350 con 0x7f32d814c540 2024-05-20T18:35:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.957+0000 7f32df123640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f32d8159eb0 con 0x7f32d814c130 2024-05-20T18:35:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.957+0000 7f32dcebf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32c40039b0 0x7f32d81500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.957+0000 7f32dcebf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32c40039b0 0x7f32d81500b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49830/0 (socket says 172.21.15.5:49830) 2024-05-20T18:35:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.957+0000 7f32dcebf640 1 -- 172.21.15.5:0/3990556117 learned_addr learned my addr 172.21.15.5:0/3990556117 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.957+0000 7f32d7fff640 1 --2- 172.21.15.5:0/3990556117 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32d814c130 0x7f32d80cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.957+0000 7f32dd6c0640 1 --2- 172.21.15.5:0/3990556117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32d814c540 0x7f32d815b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.957+0000 7f32dcebf640 1 -- 172.21.15.5:0/3990556117 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32d814c130 msgr2=0x7f32d80cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.957+0000 7f32dcebf640 1 --2- 172.21.15.5:0/3990556117 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32d814c130 0x7f32d80cc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.957+0000 7f32dcebf640 1 -- 172.21.15.5:0/3990556117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32d814c540 msgr2=0x7f32d815b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.957+0000 7f32dcebf640 1 --2- 172.21.15.5:0/3990556117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32d814c540 0x7f32d815b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.957+0000 7f32dcebf640 1 -- 172.21.15.5:0/3990556117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d815a4b0 con 0x7f32c40039b0 2024-05-20T18:35:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.957+0000 7f32dcebf640 1 --2- 172.21.15.5:0/3990556117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32c40039b0 0x7f32d81500b0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f32d0000c00 tx=0x7f32d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.957+0000 7f32d57fa640 1 -- 172.21.15.5:0/3990556117 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d0040030 con 0x7f32c40039b0 2024-05-20T18:35:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.957+0000 7f32d57fa640 1 -- 172.21.15.5:0/3990556117 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32d003b460 con 0x7f32c40039b0 2024-05-20T18:35:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.957+0000 7f32d57fa640 1 -- 172.21.15.5:0/3990556117 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d0036040 con 0x7f32c40039b0 2024-05-20T18:35:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.957+0000 7f32df123640 1 -- 172.21.15.5:0/3990556117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32d815a790 con 0x7f32c40039b0 2024-05-20T18:35:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.957+0000 7f32df123640 1 -- 172.21.15.5:0/3990556117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32d815ad50 con 0x7f32c40039b0 2024-05-20T18:35:43.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.958+0000 7f32df123640 1 -- 172.21.15.5:0/3990556117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32d8075b00 con 0x7f32c40039b0 2024-05-20T18:35:43.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.959+0000 7f32d57fa640 1 -- 172.21.15.5:0/3990556117 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f32d0043070 con 0x7f32c40039b0 2024-05-20T18:35:43.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.960+0000 7f32d57fa640 1 --2- 172.21.15.5:0/3990556117 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f32b806be30 0x7f32b806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:43.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.960+0000 7f32d7fff640 1 --2- 172.21.15.5:0/3990556117 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f32b806be30 0x7f32b806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:43.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.960+0000 7f32d57fa640 1 -- 172.21.15.5:0/3990556117 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7f32d00aadc0 con 0x7f32c40039b0 2024-05-20T18:35:43.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.960+0000 7f32d7fff640 1 --2- 172.21.15.5:0/3990556117 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f32b806be30 0x7f32b806e340 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f32d81f8920 tx=0x7f32c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:43.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:43.965+0000 7f32d57fa640 1 -- 172.21.15.5:0/3990556117 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f32d0078f90 con 0x7f32c40039b0 2024-05-20T18:35:44.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.151+0000 7f32df123640 1 -- 172.21.15.5:0/3990556117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f32d80b1e00 con 0x7f32c40039b0 2024-05-20T18:35:44.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.151+0000 7f32d57fa640 1 -- 172.21.15.5:0/3990556117 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) ==== 104+0+11 (secure 0 0 0) 0x7f32d007d290 con 0x7f32c40039b0 2024-05-20T18:35:44.152 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:35:44.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.153+0000 7f32df123640 1 -- 172.21.15.5:0/3990556117 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f32b806be30 msgr2=0x7f32b806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:44.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.153+0000 7f32df123640 1 --2- 172.21.15.5:0/3990556117 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f32b806be30 0x7f32b806e340 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f32d81f8920 tx=0x7f32c0009040 comp rx=0 tx=0).stop 2024-05-20T18:35:44.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.153+0000 7f32df123640 1 -- 172.21.15.5:0/3990556117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32c40039b0 msgr2=0x7f32d81500b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:44.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.153+0000 7f32df123640 1 --2- 172.21.15.5:0/3990556117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32c40039b0 0x7f32d81500b0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f32d0000c00 tx=0x7f32d0004730 comp rx=0 tx=0).stop 2024-05-20T18:35:44.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.153+0000 7f32df123640 1 -- 172.21.15.5:0/3990556117 shutdown_connections 2024-05-20T18:35:44.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.153+0000 7f32df123640 1 --2- 172.21.15.5:0/3990556117 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f32b806be30 0x7f32b806e340 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:44.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.153+0000 7f32df123640 1 --2- 172.21.15.5:0/3990556117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32d814c540 0x7f32d815b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:44.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.153+0000 7f32df123640 1 --2- 172.21.15.5:0/3990556117 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32d814c130 0x7f32d80cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:44.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.153+0000 7f32df123640 1 --2- 172.21.15.5:0/3990556117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32c40039b0 0x7f32d81500b0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:44.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.153+0000 7f32df123640 1 -- 172.21.15.5:0/3990556117 >> 172.21.15.5:0/3990556117 conn(0x7f32d80b6670 msgr2=0x7f32d806ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:44.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.153+0000 7f32df123640 1 -- 172.21.15.5:0/3990556117 shutdown_connections 2024-05-20T18:35:44.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.153+0000 7f32df123640 1 -- 172.21.15.5:0/3990556117 wait complete. 2024-05-20T18:35:44.166 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:35:44.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.283+0000 7f40d427a640 1 Processor -- start 2024-05-20T18:35:44.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.283+0000 7f40d427a640 1 -- start start 2024-05-20T18:35:44.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.284+0000 7f40d427a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f40cc0d4400 0x7f40cc15f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:44.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.285+0000 7f40d427a640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40cc08d2a0 con 0x7f40cc0d2780 2024-05-20T18:35:44.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.285+0000 7f40d427a640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40cc0d4810 con 0x7f40cc06cf00 2024-05-20T18:35:44.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.285+0000 7f40d427a640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f40cc0d3f70 con 0x7f40cc0d4400 2024-05-20T18:35:44.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.285+0000 7f40d2817640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f40cc0d2780 0x7f40cc0d2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46774/0 (socket says 172.21.15.5:46774) 2024-05-20T18:35:44.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.285+0000 7f40d2817640 1 -- 172.21.15.5:0/2834330701 learned_addr learned my addr 172.21.15.5:0/2834330701 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:44.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.285+0000 7f40d2016640 1 --2- 172.21.15.5:0/2834330701 >> v2:172.21.15.63:3300/0 conn(0x7f40cc0d4400 0x7f40cc15f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:44.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.285+0000 7f40d1014640 1 -- 172.21.15.5:0/2834330701 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3236211921 0 0) 0x7f40c4001490 con 0x7f40cc0d2780 2024-05-20T18:35:44.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.285+0000 7f40d1014640 1 -- 172.21.15.5:0/2834330701 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40b8003520 con 0x7f40cc0d2780 2024-05-20T18:35:44.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.285+0000 7f40d2016640 1 -- 172.21.15.5:0/2834330701 >> v1:172.21.15.63:6789/0 conn(0x7f40cc06cf00 legacy=0x7f40cc06d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:44.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.285+0000 7f40d2016640 1 -- 172.21.15.5:0/2834330701 >> v1:172.21.15.5:6789/0 conn(0x7f40cc0d2780 legacy=0x7f40cc0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:44.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.285+0000 7f40d2016640 1 -- 172.21.15.5:0/2834330701 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40cc0d3360 con 0x7f40cc0d4400 2024-05-20T18:35:44.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.286+0000 7f40d2016640 1 --2- 172.21.15.5:0/2834330701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40cc0d4400 0x7f40cc15f440 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0d40f0 tx=0x7f40c802a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5456bdf4c448a4ee server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:44.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.286+0000 7f40d1014640 1 -- 172.21.15.5:0/2834330701 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c8037070 con 0x7f40cc0d4400 2024-05-20T18:35:44.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.286+0000 7f40d1014640 1 -- 172.21.15.5:0/2834330701 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40c8032640 con 0x7f40cc0d4400 2024-05-20T18:35:44.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.286+0000 7f40d1014640 1 -- 172.21.15.5:0/2834330701 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c8032a10 con 0x7f40cc0d4400 2024-05-20T18:35:44.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.287+0000 7f40d427a640 1 -- 172.21.15.5:0/2834330701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40cc0d4400 msgr2=0x7f40cc15f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:44.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.287+0000 7f40d427a640 1 --2- 172.21.15.5:0/2834330701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40cc0d4400 0x7f40cc15f440 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0d40f0 tx=0x7f40c802a2c0 comp rx=0 tx=0).stop 2024-05-20T18:35:44.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.287+0000 7f40d427a640 1 -- 172.21.15.5:0/2834330701 shutdown_connections 2024-05-20T18:35:44.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.287+0000 7f40d427a640 1 --2- 172.21.15.5:0/2834330701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40cc0d4400 0x7f40cc15f440 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:44.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.287+0000 7f40d427a640 1 -- 172.21.15.5:0/2834330701 >> 172.21.15.5:0/2834330701 conn(0x7f40cc0d4bb0 msgr2=0x7f40cc0d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:44.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.287+0000 7f40d427a640 1 -- 172.21.15.5:0/2834330701 shutdown_connections 2024-05-20T18:35:44.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.287+0000 7f40d427a640 1 -- 172.21.15.5:0/2834330701 wait complete. 2024-05-20T18:35:44.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.289+0000 7f40d427a640 1 Processor -- start 2024-05-20T18:35:44.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.289+0000 7f40d427a640 1 -- start start 2024-05-20T18:35:44.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.290+0000 7f40d427a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40cc06cf00 0x7f40cc15e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:44.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.290+0000 7f40d427a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40cc0d2780 0x7f40cc159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:44.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.290+0000 7f40d427a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f40cc0d4400 0x7f40cc1f8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:44.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.290+0000 7f40d427a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc0d3c00 con 0x7f40cc06cf00 2024-05-20T18:35:44.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.290+0000 7f40d427a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc0d39a0 con 0x7f40cc0d2780 2024-05-20T18:35:44.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.290+0000 7f40d427a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f40cc15f980 con 0x7f40cc0d4400 2024-05-20T18:35:44.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.290+0000 7f40d2016640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40cc06cf00 0x7f40cc15e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:44.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.290+0000 7f40d2016640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40cc06cf00 0x7f40cc15e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49842/0 (socket says 172.21.15.5:49842) 2024-05-20T18:35:44.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.290+0000 7f40d2016640 1 -- 172.21.15.5:0/2581918638 learned_addr learned my addr 172.21.15.5:0/2581918638 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:44.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.290+0000 7f40d1815640 1 --2- 172.21.15.5:0/2581918638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40cc0d2780 0x7f40cc159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:44.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.290+0000 7f40d2817640 1 --2- 172.21.15.5:0/2581918638 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f40cc0d4400 0x7f40cc1f8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:44.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.291+0000 7f40d2016640 1 -- 172.21.15.5:0/2581918638 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f40cc0d4400 msgr2=0x7f40cc1f8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:44.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.291+0000 7f40d2016640 1 --2- 172.21.15.5:0/2581918638 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f40cc0d4400 0x7f40cc1f8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:44.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.291+0000 7f40d2016640 1 -- 172.21.15.5:0/2581918638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40cc0d2780 msgr2=0x7f40cc159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:44.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.291+0000 7f40d2016640 1 --2- 172.21.15.5:0/2581918638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40cc0d2780 0x7f40cc159e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:44.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.291+0000 7f40d2016640 1 -- 172.21.15.5:0/2581918638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40cc1e7bc0 con 0x7f40cc06cf00 2024-05-20T18:35:44.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.291+0000 7f40d2817640 1 --2- 172.21.15.5:0/2581918638 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f40cc0d4400 0x7f40cc1f8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:44.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.291+0000 7f40d2016640 1 --2- 172.21.15.5:0/2581918638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40cc06cf00 0x7f40cc15e250 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f40c8000c00 tx=0x7f40c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:44.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.291+0000 7f40c2ffd640 1 -- 172.21.15.5:0/2581918638 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c8037040 con 0x7f40cc06cf00 2024-05-20T18:35:44.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.291+0000 7f40c2ffd640 1 -- 172.21.15.5:0/2581918638 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40c8042070 con 0x7f40cc06cf00 2024-05-20T18:35:44.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.291+0000 7f40d427a640 1 -- 172.21.15.5:0/2581918638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40cc15d780 con 0x7f40cc06cf00 2024-05-20T18:35:44.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.291+0000 7f40c2ffd640 1 -- 172.21.15.5:0/2581918638 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c8030990 con 0x7f40cc06cf00 2024-05-20T18:35:44.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.291+0000 7f40d427a640 1 -- 172.21.15.5:0/2581918638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40cc15dd40 con 0x7f40cc06cf00 2024-05-20T18:35:44.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.292+0000 7f40d427a640 1 -- 172.21.15.5:0/2581918638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40cc0735a0 con 0x7f40cc06cf00 2024-05-20T18:35:44.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.293+0000 7f40c2ffd640 1 -- 172.21.15.5:0/2581918638 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f40c8030b30 con 0x7f40cc06cf00 2024-05-20T18:35:44.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.293+0000 7f40c2ffd640 1 --2- 172.21.15.5:0/2581918638 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f40b006be30 0x7f40b006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:44.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.293+0000 7f40d1815640 1 --2- 172.21.15.5:0/2581918638 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f40b006be30 0x7f40b006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:44.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.293+0000 7f40c2ffd640 1 -- 172.21.15.5:0/2581918638 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7f40c80ab6c0 con 0x7f40cc06cf00 2024-05-20T18:35:44.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.294+0000 7f40d1815640 1 --2- 172.21.15.5:0/2581918638 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f40b006be30 0x7f40b006e340 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0c8570 tx=0x7f40bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:44.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.298+0000 7f40c2ffd640 1 -- 172.21.15.5:0/2581918638 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f40cc0735a0 con 0x7f40cc06cf00 2024-05-20T18:35:44.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.480+0000 7f40d427a640 1 -- 172.21.15.5:0/2581918638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f40cc075b00 con 0x7f40cc06cf00 2024-05-20T18:35:44.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.480+0000 7f40c2ffd640 1 -- 172.21.15.5:0/2581918638 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) ==== 100+0+11 (secure 0 0 0) 0x7f40c804d060 con 0x7f40cc06cf00 2024-05-20T18:35:44.482 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:35:44.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.482+0000 7f40d427a640 1 -- 172.21.15.5:0/2581918638 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f40b006be30 msgr2=0x7f40b006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:44.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.482+0000 7f40d427a640 1 --2- 172.21.15.5:0/2581918638 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f40b006be30 0x7f40b006e340 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0c8570 tx=0x7f40bc009040 comp rx=0 tx=0).stop 2024-05-20T18:35:44.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.482+0000 7f40d427a640 1 -- 172.21.15.5:0/2581918638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40cc06cf00 msgr2=0x7f40cc15e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:44.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.482+0000 7f40d427a640 1 --2- 172.21.15.5:0/2581918638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40cc06cf00 0x7f40cc15e250 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f40c8000c00 tx=0x7f40c80046c0 comp rx=0 tx=0).stop 2024-05-20T18:35:44.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.482+0000 7f40d427a640 1 -- 172.21.15.5:0/2581918638 shutdown_connections 2024-05-20T18:35:44.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.482+0000 7f40d427a640 1 --2- 172.21.15.5:0/2581918638 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f40b006be30 0x7f40b006e340 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:44.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.482+0000 7f40d427a640 1 --2- 172.21.15.5:0/2581918638 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f40cc0d4400 0x7f40cc1f8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:44.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.482+0000 7f40d427a640 1 --2- 172.21.15.5:0/2581918638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40cc0d2780 0x7f40cc159e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:44.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.482+0000 7f40d427a640 1 --2- 172.21.15.5:0/2581918638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40cc06cf00 0x7f40cc15e250 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:44.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.482+0000 7f40d427a640 1 -- 172.21.15.5:0/2581918638 >> 172.21.15.5:0/2581918638 conn(0x7f40cc0d4bb0 msgr2=0x7f40cc0cdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:44.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.482+0000 7f40d427a640 1 -- 172.21.15.5:0/2581918638 shutdown_connections 2024-05-20T18:35:44.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.482+0000 7f40d427a640 1 -- 172.21.15.5:0/2581918638 wait complete. 2024-05-20T18:35:44.495 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T18:35:44.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.611+0000 7fbd9df1a640 1 Processor -- start 2024-05-20T18:35:44.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.611+0000 7fbd9df1a640 1 -- start start 2024-05-20T18:35:44.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.612+0000 7fbd9df1a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbd98163a10 0x7fbd980d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:44.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.612+0000 7fbd9df1a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd980d14b0 0x7fbd980d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:44.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.612+0000 7fbd9df1a640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd980d0370 con 0x7fbd980b4010 2024-05-20T18:35:44.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.612+0000 7fbd9df1a640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fbd980d3ca0 con 0x7fbd98163a10 2024-05-20T18:35:44.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.612+0000 7fbd9df1a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbd980d0650 con 0x7fbd980d14b0 2024-05-20T18:35:44.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.612+0000 7fbd977fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbd98163a10 0x7fbd980d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:44.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.612+0000 7fbd96ffd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbd980b4010 0x7fbd980cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46776/0 (socket says 172.21.15.5:46776) 2024-05-20T18:35:44.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.612+0000 7fbd96ffd640 1 -- 172.21.15.5:0/3592662431 learned_addr learned my addr 172.21.15.5:0/3592662431 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:44.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.612+0000 7fbd97fff640 1 --2- 172.21.15.5:0/3592662431 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd980d14b0 0x7fbd980d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:44.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.613+0000 7fbd977fe640 1 -- 172.21.15.5:0/3592662431 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd980d14b0 msgr2=0x7fbd980d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:44.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.613+0000 7fbd977fe640 1 --2- 172.21.15.5:0/3592662431 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd980d14b0 0x7fbd980d18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:44.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.613+0000 7fbd977fe640 1 -- 172.21.15.5:0/3592662431 >> v1:172.21.15.5:6789/0 conn(0x7fbd980b4010 legacy=0x7fbd980cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:44.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.613+0000 7fbd967fc640 1 -- 172.21.15.5:0/3592662431 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1750640796 0 0) 0x7fbd980d0370 con 0x7fbd980b4010 2024-05-20T18:35:44.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.613+0000 7fbd977fe640 1 -- 172.21.15.5:0/3592662431 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd980d3650 con 0x7fbd98163a10 2024-05-20T18:35:44.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.613+0000 7fbd977fe640 1 --2- 172.21.15.5:0/3592662431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd98163a10 0x7fbd980d4130 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fbd880097a0 tx=0x7fbd8802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4474e37f9a8072 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:44.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.614+0000 7fbd967fc640 1 -- 172.21.15.5:0/3592662431 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88037070 con 0x7fbd98163a10 2024-05-20T18:35:44.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.614+0000 7fbd967fc640 1 -- 172.21.15.5:0/3592662431 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd88032440 con 0x7fbd98163a10 2024-05-20T18:35:44.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.614+0000 7fbd967fc640 1 -- 172.21.15.5:0/3592662431 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88032810 con 0x7fbd98163a10 2024-05-20T18:35:44.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.614+0000 7fbd9df1a640 1 -- 172.21.15.5:0/3592662431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd98163a10 msgr2=0x7fbd980d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:44.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.614+0000 7fbd9df1a640 1 --2- 172.21.15.5:0/3592662431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd98163a10 0x7fbd980d4130 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fbd880097a0 tx=0x7fbd8802a180 comp rx=0 tx=0).stop 2024-05-20T18:35:44.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.614+0000 7fbd9df1a640 1 -- 172.21.15.5:0/3592662431 shutdown_connections 2024-05-20T18:35:44.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.614+0000 7fbd9df1a640 1 --2- 172.21.15.5:0/3592662431 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd980d14b0 0x7fbd980d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:44.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.614+0000 7fbd9df1a640 1 --2- 172.21.15.5:0/3592662431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd98163a10 0x7fbd980d4130 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:44.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.614+0000 7fbd9df1a640 1 -- 172.21.15.5:0/3592662431 >> 172.21.15.5:0/3592662431 conn(0x7fbd9806d240 msgr2=0x7fbd9806d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:44.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.614+0000 7fbd9df1a640 1 -- 172.21.15.5:0/3592662431 shutdown_connections 2024-05-20T18:35:44.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.614+0000 7fbd9df1a640 1 -- 172.21.15.5:0/3592662431 wait complete. 2024-05-20T18:35:44.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.616+0000 7fbd9df1a640 1 Processor -- start 2024-05-20T18:35:44.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.616+0000 7fbd9df1a640 1 -- start start 2024-05-20T18:35:44.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.618+0000 7fbd9df1a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd980b4010 0x7fbd980ab580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:44.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.618+0000 7fbd9df1a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd980d14b0 0x7fbd980cde90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:44.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.618+0000 7fbd9df1a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd98163a10 0x7fbd980ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:44.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.618+0000 7fbd9df1a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd980b8210 con 0x7fbd98163a10 2024-05-20T18:35:44.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.618+0000 7fbd9df1a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbd980d1e00 con 0x7fbd980d14b0 2024-05-20T18:35:44.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.618+0000 7fbd9df1a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbd980b8390 con 0x7fbd980b4010 2024-05-20T18:35:44.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.618+0000 7fbd97fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd98163a10 0x7fbd980ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:44.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.618+0000 7fbd97fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd98163a10 0x7fbd980ce3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49852/0 (socket says 172.21.15.5:49852) 2024-05-20T18:35:44.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.618+0000 7fbd97fff640 1 -- 172.21.15.5:0/2486127021 learned_addr learned my addr 172.21.15.5:0/2486127021 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:44.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.618+0000 7fbd977fe640 1 --2- 172.21.15.5:0/2486127021 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd980b4010 0x7fbd980ab580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:44.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.618+0000 7fbd96ffd640 1 --2- 172.21.15.5:0/2486127021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd980d14b0 0x7fbd980cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:44.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.618+0000 7fbd97fff640 1 -- 172.21.15.5:0/2486127021 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd980b4010 msgr2=0x7fbd980ab580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:44.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.618+0000 7fbd97fff640 1 --2- 172.21.15.5:0/2486127021 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd980b4010 0x7fbd980ab580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:44.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.618+0000 7fbd97fff640 1 -- 172.21.15.5:0/2486127021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd980d14b0 msgr2=0x7fbd980cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:44.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.618+0000 7fbd97fff640 1 --2- 172.21.15.5:0/2486127021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd980d14b0 0x7fbd980cde90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:44.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.618+0000 7fbd97fff640 1 -- 172.21.15.5:0/2486127021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd980abac0 con 0x7fbd98163a10 2024-05-20T18:35:44.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.618+0000 7fbd97fff640 1 --2- 172.21.15.5:0/2486127021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd98163a10 0x7fbd980ce3d0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000efc0 tx=0x7fbd9000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:44.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.619+0000 7fbd7ffff640 1 -- 172.21.15.5:0/2486127021 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd90018020 con 0x7fbd98163a10 2024-05-20T18:35:44.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.619+0000 7fbd7ffff640 1 -- 172.21.15.5:0/2486127021 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd90014480 con 0x7fbd98163a10 2024-05-20T18:35:44.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.619+0000 7fbd9df1a640 1 -- 172.21.15.5:0/2486127021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd980abfd0 con 0x7fbd98163a10 2024-05-20T18:35:44.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.619+0000 7fbd7ffff640 1 -- 172.21.15.5:0/2486127021 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd90009e60 con 0x7fbd98163a10 2024-05-20T18:35:44.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.619+0000 7fbd9df1a640 1 -- 172.21.15.5:0/2486127021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd980ac590 con 0x7fbd98163a10 2024-05-20T18:35:44.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.620+0000 7fbd9df1a640 1 -- 172.21.15.5:0/2486127021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd980abd40 con 0x7fbd98163a10 2024-05-20T18:35:44.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.620+0000 7fbd7ffff640 1 -- 172.21.15.5:0/2486127021 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbd90005d10 con 0x7fbd98163a10 2024-05-20T18:35:44.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.621+0000 7fbd7ffff640 1 --2- 172.21.15.5:0/2486127021 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbd8406be30 0x7fbd8406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:44.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.621+0000 7fbd977fe640 1 --2- 172.21.15.5:0/2486127021 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbd8406be30 0x7fbd8406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:44.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.621+0000 7fbd7ffff640 1 -- 172.21.15.5:0/2486127021 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7fbd9008bb00 con 0x7fbd98163a10 2024-05-20T18:35:44.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.622+0000 7fbd977fe640 1 --2- 172.21.15.5:0/2486127021 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbd8406be30 0x7fbd8406e340 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fbd88034000 tx=0x7fbd8802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.626+0000 7fbd7ffff640 1 -- 172.21.15.5:0/2486127021 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbd90022030 con 0x7fbd98163a10 2024-05-20T18:35:44.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.823+0000 7fbd9df1a640 1 -- 172.21.15.5:0/2486127021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbd98075b00 con 0x7fbd98163a10 2024-05-20T18:35:44.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.824+0000 7fbd7ffff640 1 -- 172.21.15.5:0/2486127021 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v105) ==== 80+0+0 (secure 0 0 0) 0x7fbd90010040 con 0x7fbd98163a10 2024-05-20T18:35:44.825 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-05-20T18:35:44.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.825+0000 7fbd9df1a640 1 -- 172.21.15.5:0/2486127021 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbd8406be30 msgr2=0x7fbd8406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:44.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.826+0000 7fbd9df1a640 1 --2- 172.21.15.5:0/2486127021 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbd8406be30 0x7fbd8406e340 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fbd88034000 tx=0x7fbd8802e000 comp rx=0 tx=0).stop 2024-05-20T18:35:44.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.826+0000 7fbd9df1a640 1 -- 172.21.15.5:0/2486127021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd98163a10 msgr2=0x7fbd980ce3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:44.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.826+0000 7fbd9df1a640 1 --2- 172.21.15.5:0/2486127021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd98163a10 0x7fbd980ce3d0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000efc0 tx=0x7fbd9000c490 comp rx=0 tx=0).stop 2024-05-20T18:35:44.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.826+0000 7fbd9df1a640 1 -- 172.21.15.5:0/2486127021 shutdown_connections 2024-05-20T18:35:44.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.826+0000 7fbd9df1a640 1 --2- 172.21.15.5:0/2486127021 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbd8406be30 0x7fbd8406e340 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:44.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.826+0000 7fbd9df1a640 1 --2- 172.21.15.5:0/2486127021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd98163a10 0x7fbd980ce3d0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:44.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.826+0000 7fbd9df1a640 1 --2- 172.21.15.5:0/2486127021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd980d14b0 0x7fbd980cde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:44.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.826+0000 7fbd9df1a640 1 --2- 172.21.15.5:0/2486127021 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd980b4010 0x7fbd980ab580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:44.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.826+0000 7fbd9df1a640 1 -- 172.21.15.5:0/2486127021 >> 172.21.15.5:0/2486127021 conn(0x7fbd9806d240 msgr2=0x7fbd980b4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:44.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.826+0000 7fbd9df1a640 1 -- 172.21.15.5:0/2486127021 shutdown_connections 2024-05-20T18:35:44.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:44.826+0000 7fbd9df1a640 1 -- 172.21.15.5:0/2486127021 wait complete. 2024-05-20T18:35:44.839 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-20T18:35:44.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:35:44.839 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-20T18:35:44.871 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:35:44.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:35:44.871 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-20T18:35:44.928 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:35:45.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.083+0000 7f6666afe640 1 Processor -- start 2024-05-20T18:35:45.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.083+0000 7f6666afe640 1 -- start start 2024-05-20T18:35:45.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.084+0000 7f6666afe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f66601576d0 0x7f666006c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:45.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.084+0000 7f6666afe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f66600cedd0 0x7f666006e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:45.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.084+0000 7f6666afe640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6660157ae0 con 0x7f666015d5f0 2024-05-20T18:35:45.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.084+0000 7f6666afe640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f666006cee0 con 0x7f66600cedd0 2024-05-20T18:35:45.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.084+0000 7f6666afe640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f666006b900 con 0x7f66601576d0 2024-05-20T18:35:45.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.084+0000 7f666489a640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f666015d5f0 0x7f66600ce6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33590/0 (socket says 172.21.15.5:33590) 2024-05-20T18:35:45.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.084+0000 7f666489a640 1 -- 172.21.15.5:0/253043290 learned_addr learned my addr 172.21.15.5:0/253043290 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:45.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.084+0000 7f666509b640 1 --2- 172.21.15.5:0/253043290 >> v2:172.21.15.5:3300/0 conn(0x7f66600cedd0 0x7f666006e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:45.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.084+0000 7f665ffff640 1 --2- 172.21.15.5:0/253043290 >> v2:172.21.15.63:3300/0 conn(0x7f66601576d0 0x7f666006c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:45.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.084+0000 7f665f7fe640 1 -- 172.21.15.5:0/253043290 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1008826764 0 0) 0x7f6660157ae0 con 0x7f666015d5f0 2024-05-20T18:35:45.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.085+0000 7f665f7fe640 1 -- 172.21.15.5:0/253043290 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f664c003520 con 0x7f666015d5f0 2024-05-20T18:35:45.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.085+0000 7f666509b640 1 -- 172.21.15.5:0/253043290 >> v2:172.21.15.63:3300/0 conn(0x7f66601576d0 msgr2=0x7f666006c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:45.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.085+0000 7f666509b640 1 --2- 172.21.15.5:0/253043290 >> v2:172.21.15.63:3300/0 conn(0x7f66601576d0 0x7f666006c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:45.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.085+0000 7f666509b640 1 -- 172.21.15.5:0/253043290 >> v1:172.21.15.5:6789/0 conn(0x7f666015d5f0 legacy=0x7f66600ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:45.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.085+0000 7f666509b640 1 -- 172.21.15.5:0/253043290 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666006bbe0 con 0x7f66600cedd0 2024-05-20T18:35:45.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.085+0000 7f666509b640 1 --2- 172.21.15.5:0/253043290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66600cedd0 0x7f666006e3a0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f66600708f0 tx=0x7f665802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c962161667ab4510 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:45.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.086+0000 7f665f7fe640 1 -- 172.21.15.5:0/253043290 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6658007740 con 0x7f66600cedd0 2024-05-20T18:35:45.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.086+0000 7f665f7fe640 1 -- 172.21.15.5:0/253043290 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66580078e0 con 0x7f66600cedd0 2024-05-20T18:35:45.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.086+0000 7f665f7fe640 1 -- 172.21.15.5:0/253043290 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6658007cb0 con 0x7f66600cedd0 2024-05-20T18:35:45.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.086+0000 7f6666afe640 1 -- 172.21.15.5:0/253043290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66600cedd0 msgr2=0x7f666006e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:45.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.086+0000 7f6666afe640 1 --2- 172.21.15.5:0/253043290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66600cedd0 0x7f666006e3a0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f66600708f0 tx=0x7f665802a5f0 comp rx=0 tx=0).stop 2024-05-20T18:35:45.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.086+0000 7f6666afe640 1 -- 172.21.15.5:0/253043290 shutdown_connections 2024-05-20T18:35:45.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.086+0000 7f6666afe640 1 --2- 172.21.15.5:0/253043290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66600cedd0 0x7f666006e3a0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:45.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.086+0000 7f6666afe640 1 --2- 172.21.15.5:0/253043290 >> v2:172.21.15.63:3300/0 conn(0x7f66601576d0 0x7f666006c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:45.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.086+0000 7f6666afe640 1 -- 172.21.15.5:0/253043290 >> 172.21.15.5:0/253043290 conn(0x7f66600cd2a0 msgr2=0x7f66600cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:45.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.086+0000 7f6666afe640 1 -- 172.21.15.5:0/253043290 shutdown_connections 2024-05-20T18:35:45.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.086+0000 7f6666afe640 1 -- 172.21.15.5:0/253043290 wait complete. 2024-05-20T18:35:45.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.088+0000 7f6666afe640 1 Processor -- start 2024-05-20T18:35:45.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.088+0000 7f6666afe640 1 -- start start 2024-05-20T18:35:45.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.089+0000 7f6666afe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f66600cedd0 0x7f6660156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:45.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.089+0000 7f6666afe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f66601576d0 0x7f6660154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:45.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.089+0000 7f6666afe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f666015d5f0 0x7f66601529d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:45.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.090+0000 7f6666afe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f666006c480 con 0x7f666015d5f0 2024-05-20T18:35:45.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.090+0000 7f6666afe640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f666006c1a0 con 0x7f66601576d0 2024-05-20T18:35:45.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.090+0000 7f6666afe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f66600cf1e0 con 0x7f66600cedd0 2024-05-20T18:35:45.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.090+0000 7f666509b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f666015d5f0 0x7f66601529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:45.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.090+0000 7f666509b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f666015d5f0 0x7f66601529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37824/0 (socket says 172.21.15.5:37824) 2024-05-20T18:35:45.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.090+0000 7f666509b640 1 -- 172.21.15.5:0/4110575575 learned_addr learned my addr 172.21.15.5:0/4110575575 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:45.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.090+0000 7f665ffff640 1 --2- 172.21.15.5:0/4110575575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f66601576d0 0x7f6660154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:45.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.090+0000 7f666489a640 1 --2- 172.21.15.5:0/4110575575 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f66600cedd0 0x7f6660156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:45.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.090+0000 7f666509b640 1 -- 172.21.15.5:0/4110575575 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f66600cedd0 msgr2=0x7f6660156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:45.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.090+0000 7f666509b640 1 --2- 172.21.15.5:0/4110575575 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f66600cedd0 0x7f6660156870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:45.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.090+0000 7f666509b640 1 -- 172.21.15.5:0/4110575575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f66601576d0 msgr2=0x7f6660154d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:45.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.090+0000 7f666509b640 1 --2- 172.21.15.5:0/4110575575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f66601576d0 0x7f6660154d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:45.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.090+0000 7f666509b640 1 -- 172.21.15.5:0/4110575575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66601557d0 con 0x7f666015d5f0 2024-05-20T18:35:45.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.090+0000 7f666509b640 1 --2- 172.21.15.5:0/4110575575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f666015d5f0 0x7f66601529d0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f6658009e50 tx=0x7f6658004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:45.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.090+0000 7f665d7fa640 1 -- 172.21.15.5:0/4110575575 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665803f070 con 0x7f666015d5f0 2024-05-20T18:35:45.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.090+0000 7f665d7fa640 1 -- 172.21.15.5:0/4110575575 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6658031bf0 con 0x7f666015d5f0 2024-05-20T18:35:45.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.090+0000 7f6666afe640 1 -- 172.21.15.5:0/4110575575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6660155ab0 con 0x7f666015d5f0 2024-05-20T18:35:45.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.090+0000 7f665d7fa640 1 -- 172.21.15.5:0/4110575575 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6658009450 con 0x7f666015d5f0 2024-05-20T18:35:45.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.090+0000 7f6666afe640 1 -- 172.21.15.5:0/4110575575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6660156070 con 0x7f666015d5f0 2024-05-20T18:35:45.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.091+0000 7f6666afe640 1 -- 172.21.15.5:0/4110575575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6660155540 con 0x7f666015d5f0 2024-05-20T18:35:45.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.092+0000 7f665d7fa640 1 -- 172.21.15.5:0/4110575575 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f665803b410 con 0x7f666015d5f0 2024-05-20T18:35:45.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.093+0000 7f665d7fa640 1 --2- 172.21.15.5:0/4110575575 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f664006be60 0x7f664006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:45.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.093+0000 7f666489a640 1 --2- 172.21.15.5:0/4110575575 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f664006be60 0x7f664006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:45.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.093+0000 7f665d7fa640 1 -- 172.21.15.5:0/4110575575 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7f66580ac820 con 0x7f666015d5f0 2024-05-20T18:35:45.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.094+0000 7f666489a640 1 --2- 172.21.15.5:0/4110575575 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f664006be60 0x7f664006e370 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6648005880 tx=0x7f6648009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:45.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.098+0000 7f665d7fa640 1 -- 172.21.15.5:0/4110575575 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f665807a8e0 con 0x7f666015d5f0 2024-05-20T18:35:45.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.321+0000 7f6666afe640 1 -- 172.21.15.5:0/4110575575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6660075b00 con 0x7f666015d5f0 2024-05-20T18:35:45.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.321+0000 7f665d7fa640 1 -- 172.21.15.5:0/4110575575 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5177 (secure 0 0 0) 0x7f665807ebe0 con 0x7f666015d5f0 2024-05-20T18:35:45.322 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:45.323 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6758,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/4163145158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":4163145158},{"type":"v1","addr":"172.21.15.63:6834","nonce":4163145158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7528,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3049443532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3049443532},{"type":"v1","addr":"172.21.15.5:6837","nonce":3049443532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3552410668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3552410668},{"type":"v1","addr":"172.21.15.5:6835","nonce":3552410668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:35:30.778930+0000","modified":"2024-05-20T18:35:33.806381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8343},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8343":{"gid":8343,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/2054897241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":2054897241},{"type":"v1","addr":"172.21.15.63:6835","nonce":2054897241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8343,"qdb_cluster":[8343]},"id":7}]} 2024-05-20T18:35:45.323 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 82 2024-05-20T18:35:45.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.323+0000 7f6666afe640 1 -- 172.21.15.5:0/4110575575 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f664006be60 msgr2=0x7f664006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:45.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.323+0000 7f6666afe640 1 --2- 172.21.15.5:0/4110575575 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f664006be60 0x7f664006e370 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6648005880 tx=0x7f6648009040 comp rx=0 tx=0).stop 2024-05-20T18:35:45.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.323+0000 7f6666afe640 1 -- 172.21.15.5:0/4110575575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f666015d5f0 msgr2=0x7f66601529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:45.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.323+0000 7f6666afe640 1 --2- 172.21.15.5:0/4110575575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f666015d5f0 0x7f66601529d0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f6658009e50 tx=0x7f6658004a40 comp rx=0 tx=0).stop 2024-05-20T18:35:45.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.323+0000 7f6666afe640 1 -- 172.21.15.5:0/4110575575 shutdown_connections 2024-05-20T18:35:45.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.323+0000 7f6666afe640 1 --2- 172.21.15.5:0/4110575575 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f664006be60 0x7f664006e370 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:45.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.323+0000 7f6666afe640 1 --2- 172.21.15.5:0/4110575575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f666015d5f0 0x7f66601529d0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:45.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.323+0000 7f6666afe640 1 --2- 172.21.15.5:0/4110575575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f66601576d0 0x7f6660154d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:45.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.323+0000 7f6666afe640 1 --2- 172.21.15.5:0/4110575575 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f66600cedd0 0x7f6660156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:45.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.323+0000 7f6666afe640 1 -- 172.21.15.5:0/4110575575 >> 172.21.15.5:0/4110575575 conn(0x7f66600cd2a0 msgr2=0x7f66600cfe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:45.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.324+0000 7f6666afe640 1 -- 172.21.15.5:0/4110575575 shutdown_connections 2024-05-20T18:35:45.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.324+0000 7f6666afe640 1 -- 172.21.15.5:0/4110575575 wait complete. 2024-05-20T18:35:45.337 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-05-20T18:35:45.488 INFO:teuthology.orchestra.run.smithi063.stdout:{"message":"","return_code":0} 2024-05-20T18:35:45.489 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-20T18:35:45.489 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:35:45.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.603+0000 7f72a6b7e640 1 Processor -- start 2024-05-20T18:35:45.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.603+0000 7f72a6b7e640 1 -- start start 2024-05-20T18:35:45.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.605+0000 7f72a6b7e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72a0168050 0x7f72a016a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:45.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.605+0000 7f72a6b7e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72a00d0630 con 0x7f72a006d3d0 2024-05-20T18:35:45.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.605+0000 7f72a6b7e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72a00d39c0 con 0x7f72a0162820 2024-05-20T18:35:45.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.605+0000 7f72a6b7e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f72a00d3ca0 con 0x7f72a0168050 2024-05-20T18:35:45.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.605+0000 7f729ffff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f72a006d3d0 0x7f72a006d7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33596/0 (socket says 172.21.15.5:33596) 2024-05-20T18:35:45.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.605+0000 7f729ffff640 1 -- 172.21.15.5:0/678753284 learned_addr learned my addr 172.21.15.5:0/678753284 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:45.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.605+0000 7f729f7fe640 1 -- 172.21.15.5:0/678753284 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 19425533 0 0) 0x7f72a00d0630 con 0x7f72a006d3d0 2024-05-20T18:35:45.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.605+0000 7f729f7fe640 1 -- 172.21.15.5:0/678753284 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7288003520 con 0x7f72a006d3d0 2024-05-20T18:35:45.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.605+0000 7f72a511b640 1 --2- 172.21.15.5:0/678753284 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72a0168050 0x7f72a016a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:45.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.605+0000 7f729f7fe640 1 -- 172.21.15.5:0/678753284 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 278987469 0 0) 0x7f7288003520 con 0x7f72a006d3d0 2024-05-20T18:35:45.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.605+0000 7f729f7fe640 1 -- 172.21.15.5:0/678753284 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72a0168050 msgr2=0x7f72a016a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:45.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.605+0000 7f729f7fe640 1 --2- 172.21.15.5:0/678753284 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72a0168050 0x7f72a016a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:45.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.605+0000 7f729f7fe640 1 -- 172.21.15.5:0/678753284 >> v1:172.21.15.63:6789/0 conn(0x7f72a0162820 legacy=0x7f72a00b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:45.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.605+0000 7f729f7fe640 1 -- 172.21.15.5:0/678753284 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a00d3f80 con 0x7f72a006d3d0 2024-05-20T18:35:45.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.605+0000 7f729f7fe640 1 -- 172.21.15.5:0/678753284 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f7294001670 con 0x7f72a006d3d0 2024-05-20T18:35:45.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.606+0000 7f729f7fe640 1 -- 172.21.15.5:0/678753284 >> v1:172.21.15.5:6789/0 conn(0x7f72a006d3d0 legacy=0x7f72a006d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:45.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.606+0000 7f729f7fe640 1 --2- 172.21.15.5:0/678753284 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72880039e0 0x7f7288003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:45.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.606+0000 7f729f7fe640 1 --2- 172.21.15.5:0/678753284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7288004800 0x7f7288004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:45.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.606+0000 7f729f7fe640 1 --2- 172.21.15.5:0/678753284 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72a0168050 0x7f7288005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:45.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.606+0000 7f729f7fe640 1 -- 172.21.15.5:0/678753284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f72a00d3ca0 con 0x7f7288004800 2024-05-20T18:35:45.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.606+0000 7f729f7fe640 1 -- 172.21.15.5:0/678753284 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f728800baf0 con 0x7f72880039e0 2024-05-20T18:35:45.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.606+0000 7f729f7fe640 1 -- 172.21.15.5:0/678753284 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f728800bcf0 con 0x7f72a0168050 2024-05-20T18:35:45.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.606+0000 7f729ffff640 1 --2- 172.21.15.5:0/678753284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7288004800 0x7f7288004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:45.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.606+0000 7f72a491a640 1 --2- 172.21.15.5:0/678753284 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72880039e0 0x7f7288003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:45.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.606+0000 7f72a511b640 1 --2- 172.21.15.5:0/678753284 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72a0168050 0x7f7288005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:45.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.606+0000 7f729ffff640 1 -- 172.21.15.5:0/678753284 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72a0168050 msgr2=0x7f7288005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:45.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.606+0000 7f729ffff640 1 --2- 172.21.15.5:0/678753284 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72a0168050 0x7f7288005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:45.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.606+0000 7f729ffff640 1 -- 172.21.15.5:0/678753284 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72880039e0 msgr2=0x7f7288003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:45.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.606+0000 7f729ffff640 1 --2- 172.21.15.5:0/678753284 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72880039e0 0x7f7288003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:45.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.606+0000 7f729ffff640 1 -- 172.21.15.5:0/678753284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7288003520 con 0x7f7288004800 2024-05-20T18:35:45.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.607+0000 7f729ffff640 1 --2- 172.21.15.5:0/678753284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7288004800 0x7f7288004c10 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f729400c210 tx=0x7f7288005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c54239dfe8ce6da server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:45.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.607+0000 7f729f7fe640 1 -- 172.21.15.5:0/678753284 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7294007cf0 con 0x7f7288004800 2024-05-20T18:35:45.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.607+0000 7f729f7fe640 1 -- 172.21.15.5:0/678753284 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7294005af0 con 0x7f7288004800 2024-05-20T18:35:45.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.607+0000 7f729f7fe640 1 -- 172.21.15.5:0/678753284 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7294007070 con 0x7f7288004800 2024-05-20T18:35:45.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.608+0000 7f72a6b7e640 1 -- 172.21.15.5:0/678753284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7288004800 msgr2=0x7f7288004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:45.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.608+0000 7f72a6b7e640 1 --2- 172.21.15.5:0/678753284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7288004800 0x7f7288004c10 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f729400c210 tx=0x7f7288005830 comp rx=0 tx=0).stop 2024-05-20T18:35:45.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.608+0000 7f72a511b640 1 -- 172.21.15.5:0/678753284 reap_dead start 2024-05-20T18:35:45.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.608+0000 7f72a6b7e640 1 -- 172.21.15.5:0/678753284 shutdown_connections 2024-05-20T18:35:45.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.608+0000 7f72a6b7e640 1 -- 172.21.15.5:0/678753284 >> 172.21.15.5:0/678753284 conn(0x7f72a00d4bd0 msgr2=0x7f72a00d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:45.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.608+0000 7f72a6b7e640 1 -- 172.21.15.5:0/678753284 shutdown_connections 2024-05-20T18:35:45.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.608+0000 7f72a6b7e640 1 -- 172.21.15.5:0/678753284 wait complete. 2024-05-20T18:35:45.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.610+0000 7f72a6b7e640 1 Processor -- start 2024-05-20T18:35:45.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.610+0000 7f72a6b7e640 1 -- start start 2024-05-20T18:35:45.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.611+0000 7f72a6b7e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7288004800 0x7f72a01f8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:45.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.611+0000 7f72a6b7e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72a00ca7e0 0x7f72a015ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:45.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.611+0000 7f72a6b7e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72a00cabf0 0x7f72a015d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:45.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.611+0000 7f72a6b7e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f72a00b91a0 con 0x7f72a00ca7e0 2024-05-20T18:35:45.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.611+0000 7f72a6b7e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f72a00d0300 con 0x7f72a00cabf0 2024-05-20T18:35:45.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.611+0000 7f72a6b7e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f72a00b4050 con 0x7f7288004800 2024-05-20T18:35:45.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.611+0000 7f729ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72a00ca7e0 0x7f72a015ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:45.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.611+0000 7f729ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72a00ca7e0 0x7f72a015ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37830/0 (socket says 172.21.15.5:37830) 2024-05-20T18:35:45.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.611+0000 7f729ffff640 1 -- 172.21.15.5:0/53445940 learned_addr learned my addr 172.21.15.5:0/53445940 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:45.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.611+0000 7f72a511b640 1 --2- 172.21.15.5:0/53445940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72a00cabf0 0x7f72a015d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:45.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.611+0000 7f72a491a640 1 --2- 172.21.15.5:0/53445940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7288004800 0x7f72a01f8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:45.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.611+0000 7f729ffff640 1 -- 172.21.15.5:0/53445940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7288004800 msgr2=0x7f72a01f8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:45.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.611+0000 7f729ffff640 1 --2- 172.21.15.5:0/53445940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7288004800 0x7f72a01f8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:45.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.611+0000 7f729ffff640 1 -- 172.21.15.5:0/53445940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72a00cabf0 msgr2=0x7f72a015d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:45.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.611+0000 7f729ffff640 1 --2- 172.21.15.5:0/53445940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72a00cabf0 0x7f72a015d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:45.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.611+0000 7f729ffff640 1 -- 172.21.15.5:0/53445940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a01f9530 con 0x7f72a00ca7e0 2024-05-20T18:35:45.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.612+0000 7f729ffff640 1 --2- 172.21.15.5:0/53445940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72a00ca7e0 0x7f72a015ce30 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f7294000f60 tx=0x7f72a00b7a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:45.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.612+0000 7f729d7fa640 1 -- 172.21.15.5:0/53445940 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7294001630 con 0x7f72a00ca7e0 2024-05-20T18:35:45.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.612+0000 7f729d7fa640 1 -- 172.21.15.5:0/53445940 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7294042070 con 0x7f72a00ca7e0 2024-05-20T18:35:45.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.612+0000 7f72a6b7e640 1 -- 172.21.15.5:0/53445940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a01f9810 con 0x7f72a00ca7e0 2024-05-20T18:35:45.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.612+0000 7f729d7fa640 1 -- 172.21.15.5:0/53445940 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7294007290 con 0x7f72a00ca7e0 2024-05-20T18:35:45.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.612+0000 7f72a6b7e640 1 -- 172.21.15.5:0/53445940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a00cc100 con 0x7f72a00ca7e0 2024-05-20T18:35:45.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.613+0000 7f72a6b7e640 1 -- 172.21.15.5:0/53445940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72a0075b00 con 0x7f72a00ca7e0 2024-05-20T18:35:45.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.613+0000 7f729d7fa640 1 -- 172.21.15.5:0/53445940 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7294007430 con 0x7f72a00ca7e0 2024-05-20T18:35:45.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.614+0000 7f729d7fa640 1 --2- 172.21.15.5:0/53445940 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f728c06be30 0x7f728c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:45.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.614+0000 7f729d7fa640 1 -- 172.21.15.5:0/53445940 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7f72940aafb0 con 0x7f72a00ca7e0 2024-05-20T18:35:45.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.614+0000 7f72a491a640 1 --2- 172.21.15.5:0/53445940 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f728c06be30 0x7f728c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:45.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.615+0000 7f72a491a640 1 --2- 172.21.15.5:0/53445940 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f728c06be30 0x7f728c06e340 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7290001760 tx=0x7f7290009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:45.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.619+0000 7f729d7fa640 1 -- 172.21.15.5:0/53445940 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7294005ac0 con 0x7f72a00ca7e0 2024-05-20T18:35:45.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.841+0000 7f72a6b7e640 1 -- 172.21.15.5:0/53445940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72a00b3620 con 0x7f72a00ca7e0 2024-05-20T18:35:45.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.842+0000 7f729d7fa640 1 -- 172.21.15.5:0/53445940 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5177 (secure 0 0 0) 0x7f7294079180 con 0x7f72a00ca7e0 2024-05-20T18:35:45.843 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:45.843 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6758,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/4163145158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":4163145158},{"type":"v1","addr":"172.21.15.63:6834","nonce":4163145158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7528,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3049443532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3049443532},{"type":"v1","addr":"172.21.15.5:6837","nonce":3049443532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3552410668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3552410668},{"type":"v1","addr":"172.21.15.5:6835","nonce":3552410668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:35:30.778930+0000","modified":"2024-05-20T18:35:33.806381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8343},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8343":{"gid":8343,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/2054897241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":2054897241},{"type":"v1","addr":"172.21.15.63:6835","nonce":2054897241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8343,"qdb_cluster":[8343]},"id":7}]} 2024-05-20T18:35:45.844 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 82 2024-05-20T18:35:45.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.844+0000 7f72a6b7e640 1 -- 172.21.15.5:0/53445940 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f728c06be30 msgr2=0x7f728c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:45.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.844+0000 7f72a6b7e640 1 --2- 172.21.15.5:0/53445940 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f728c06be30 0x7f728c06e340 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7290001760 tx=0x7f7290009040 comp rx=0 tx=0).stop 2024-05-20T18:35:45.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.844+0000 7f72a6b7e640 1 -- 172.21.15.5:0/53445940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72a00ca7e0 msgr2=0x7f72a015ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:45.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.844+0000 7f72a6b7e640 1 --2- 172.21.15.5:0/53445940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72a00ca7e0 0x7f72a015ce30 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f7294000f60 tx=0x7f72a00b7a20 comp rx=0 tx=0).stop 2024-05-20T18:35:45.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.844+0000 7f72a6b7e640 1 -- 172.21.15.5:0/53445940 shutdown_connections 2024-05-20T18:35:45.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.844+0000 7f72a6b7e640 1 --2- 172.21.15.5:0/53445940 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f728c06be30 0x7f728c06e340 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:45.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.844+0000 7f72a6b7e640 1 --2- 172.21.15.5:0/53445940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72a00cabf0 0x7f72a015d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:45.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.844+0000 7f72a6b7e640 1 --2- 172.21.15.5:0/53445940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72a00ca7e0 0x7f72a015ce30 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:45.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.844+0000 7f72a6b7e640 1 --2- 172.21.15.5:0/53445940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7288004800 0x7f72a01f8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:45.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.844+0000 7f72a6b7e640 1 -- 172.21.15.5:0/53445940 >> 172.21.15.5:0/53445940 conn(0x7f72a00d4bd0 msgr2=0x7f72a00b9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:45.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.844+0000 7f72a6b7e640 1 -- 172.21.15.5:0/53445940 shutdown_connections 2024-05-20T18:35:45.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.844+0000 7f72a6b7e640 1 -- 172.21.15.5:0/53445940 wait complete. 2024-05-20T18:35:45.857 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2024-05-20T18:35:45.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.912+0000 7fbe11c0bf40 1 Processor -- start 2024-05-20T18:35:45.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.912+0000 7fbe11c0bf40 1 -- start start 2024-05-20T18:35:45.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.914+0000 7fbe11c0bf40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555a43128760 0x555a431b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:45.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.914+0000 7fbe11c0bf40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a431b8ab0 0x555a431b8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:45.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.914+0000 7fbe11c0bf40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555a4312b5f0 0x555a4312ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:45.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.914+0000 7fbe11c0bf40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555a43089c80 con 0x555a431b8ab0 2024-05-20T18:35:45.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.914+0000 7fbe11c0bf40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555a4308dcc0 con 0x555a4312b5f0 2024-05-20T18:35:45.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.914+0000 7fbe11c0bf40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x555a43110400 con 0x555a43128760 2024-05-20T18:35:45.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.914+0000 7fbe0a1cb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a431b8ab0 0x555a431b8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:45.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.914+0000 7fbe0a1cb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a431b8ab0 0x555a431b8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37840/0 (socket says 172.21.15.5:37840) 2024-05-20T18:35:45.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.914+0000 7fbe0a1cb640 1 -- 172.21.15.5:0/3836976772 learned_addr learned my addr 172.21.15.5:0/3836976772 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:45.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.914+0000 7fbe0a9cc640 1 --2- 172.21.15.5:0/3836976772 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555a43128760 0x555a431b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:45.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.914+0000 7fbe0b1cd640 1 --2- 172.21.15.5:0/3836976772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555a4312b5f0 0x555a4312ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:45.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.914+0000 7fbe0a1cb640 1 -- 172.21.15.5:0/3836976772 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555a43128760 msgr2=0x555a431b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:45.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.914+0000 7fbe0a1cb640 1 --2- 172.21.15.5:0/3836976772 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555a43128760 0x555a431b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:45.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.914+0000 7fbe0a1cb640 1 -- 172.21.15.5:0/3836976772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555a4312b5f0 msgr2=0x555a4312ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:45.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.914+0000 7fbe0a1cb640 1 --2- 172.21.15.5:0/3836976772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555a4312b5f0 0x555a4312ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:45.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.914+0000 7fbe0a1cb640 1 -- 172.21.15.5:0/3836976772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a4312bf40 con 0x555a431b8ab0 2024-05-20T18:35:45.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.914+0000 7fbe0a9cc640 1 --2- 172.21.15.5:0/3836976772 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555a43128760 0x555a431b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:45.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.914+0000 7fbe0b1cd640 1 --2- 172.21.15.5:0/3836976772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555a4312b5f0 0x555a4312ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:45.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.914+0000 7fbe0a1cb640 1 --2- 172.21.15.5:0/3836976772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a431b8ab0 0x555a431b8ec0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4009590 tx=0x7fbdf4009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3651f71da526420b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:45.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.914+0000 7fbe091c9640 1 -- 172.21.15.5:0/3836976772 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf400e860 con 0x555a431b8ab0 2024-05-20T18:35:45.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.914+0000 7fbe091c9640 1 -- 172.21.15.5:0/3836976772 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdf4013030 con 0x555a431b8ab0 2024-05-20T18:35:45.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.914+0000 7fbe091c9640 1 -- 172.21.15.5:0/3836976772 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf400ec20 con 0x555a431b8ab0 2024-05-20T18:35:45.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.915+0000 7fbe11c0bf40 1 -- 172.21.15.5:0/3836976772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a431b8ab0 msgr2=0x555a431b8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:45.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.915+0000 7fbe11c0bf40 1 --2- 172.21.15.5:0/3836976772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a431b8ab0 0x555a431b8ec0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4009590 tx=0x7fbdf4009a50 comp rx=0 tx=0).stop 2024-05-20T18:35:45.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.915+0000 7fbe11c0bf40 1 -- 172.21.15.5:0/3836976772 shutdown_connections 2024-05-20T18:35:45.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.915+0000 7fbe11c0bf40 1 --2- 172.21.15.5:0/3836976772 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555a4312b5f0 0x555a4312ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:45.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.915+0000 7fbe11c0bf40 1 --2- 172.21.15.5:0/3836976772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a431b8ab0 0x555a431b8ec0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:45.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.915+0000 7fbe11c0bf40 1 --2- 172.21.15.5:0/3836976772 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555a43128760 0x555a431b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:45.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.915+0000 7fbe11c0bf40 1 -- 172.21.15.5:0/3836976772 >> 172.21.15.5:0/3836976772 conn(0x555a43007ac0 msgr2=0x555a431b9560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:45.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.915+0000 7fbe11c0bf40 1 -- 172.21.15.5:0/3836976772 shutdown_connections 2024-05-20T18:35:45.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.915+0000 7fbe11c0bf40 1 -- 172.21.15.5:0/3836976772 wait complete. 2024-05-20T18:35:45.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.916+0000 7fbe11c0bf40 1 Processor -- start 2024-05-20T18:35:45.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.916+0000 7fbe11c0bf40 1 -- start start 2024-05-20T18:35:45.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.917+0000 7fbe11c0bf40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555a43128760 0x555a431c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:45.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.917+0000 7fbe11c0bf40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a4312b5f0 0x555a431262f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:45.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.917+0000 7fbe11c0bf40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555a431b8ab0 0x555a43128ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:45.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.917+0000 7fbe11c0bf40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555a431b73f0 con 0x555a4312b5f0 2024-05-20T18:35:45.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.917+0000 7fbe11c0bf40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555a431c5ee0 con 0x555a43128760 2024-05-20T18:35:45.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.917+0000 7fbe11c0bf40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x555a431b7880 con 0x555a431b8ab0 2024-05-20T18:35:45.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.917+0000 7fbe0a1cb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a4312b5f0 0x555a431262f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:45.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.917+0000 7fbe0a1cb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a4312b5f0 0x555a431262f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37854/0 (socket says 172.21.15.5:37854) 2024-05-20T18:35:45.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.917+0000 7fbe0a1cb640 1 -- 172.21.15.5:0/2184154025 learned_addr learned my addr 172.21.15.5:0/2184154025 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:45.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.918+0000 7fbe0a9cc640 1 --2- 172.21.15.5:0/2184154025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555a43128760 0x555a431c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:45.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.918+0000 7fbe0b1cd640 1 --2- 172.21.15.5:0/2184154025 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555a431b8ab0 0x555a43128ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:45.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.918+0000 7fbe0a1cb640 1 -- 172.21.15.5:0/2184154025 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555a431b8ab0 msgr2=0x555a43128ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:45.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.918+0000 7fbe0a1cb640 1 --2- 172.21.15.5:0/2184154025 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555a431b8ab0 0x555a43128ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:45.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.918+0000 7fbe0a1cb640 1 -- 172.21.15.5:0/2184154025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555a43128760 msgr2=0x555a431c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:45.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.918+0000 7fbe0a1cb640 1 --2- 172.21.15.5:0/2184154025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555a43128760 0x555a431c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:45.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.918+0000 7fbe0a1cb640 1 -- 172.21.15.5:0/2184154025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a431c3650 con 0x555a4312b5f0 2024-05-20T18:35:45.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.918+0000 7fbe0a9cc640 1 --2- 172.21.15.5:0/2184154025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555a43128760 0x555a431c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:45.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.918+0000 7fbe0a1cb640 1 --2- 172.21.15.5:0/2184154025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a4312b5f0 0x555a431262f0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fbdf40018f0 tx=0x7fbdf4001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:45.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.918+0000 7fbe03fff640 1 -- 172.21.15.5:0/2184154025 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf400f860 con 0x555a4312b5f0 2024-05-20T18:35:45.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.918+0000 7fbe03fff640 1 -- 172.21.15.5:0/2184154025 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdf400fa00 con 0x555a4312b5f0 2024-05-20T18:35:45.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.918+0000 7fbe03fff640 1 -- 172.21.15.5:0/2184154025 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf400fdf0 con 0x555a4312b5f0 2024-05-20T18:35:45.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.918+0000 7fbe11c0bf40 1 -- 172.21.15.5:0/2184154025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555a431c3ba0 con 0x555a4312b5f0 2024-05-20T18:35:45.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.918+0000 7fbe11c0bf40 1 -- 172.21.15.5:0/2184154025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a431c52c0 con 0x555a4312b5f0 2024-05-20T18:35:45.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.920+0000 7fbe03fff640 1 -- 172.21.15.5:0/2184154025 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbdf4026450 con 0x555a4312b5f0 2024-05-20T18:35:45.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.920+0000 7fbe03fff640 1 --2- 172.21.15.5:0/2184154025 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbdec06a810 0x7fbdec06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:45.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.920+0000 7fbe03fff640 1 -- 172.21.15.5:0/2184154025 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7fbdf408c3d0 con 0x555a4312b5f0 2024-05-20T18:35:45.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.920+0000 7fbe0a9cc640 1 --2- 172.21.15.5:0/2184154025 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbdec06a810 0x7fbdec06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:45.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.920+0000 7fbe11c0bf40 1 --2- 172.21.15.5:0/2184154025 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x555a43126830 0x555a432547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:45.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.920+0000 7fbe11c0bf40 1 -- 172.21.15.5:0/2184154025 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e105) -- 0x555a43129430 con 0x555a43126830 2024-05-20T18:35:45.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.921+0000 7fbe0b1cd640 1 --2- 172.21.15.5:0/2184154025 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x555a43126830 0x555a432547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:45.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.921+0000 7fbe0a9cc640 1 --2- 172.21.15.5:0/2184154025 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbdec06a810 0x7fbdec06cd20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fbe04000f60 tx=0x7fbe04000ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:45.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.921+0000 7fbe0b1cd640 1 --2- 172.21.15.5:0/2184154025 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x555a43126830 0x555a432547a0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:45.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.921+0000 7fbe0b1cd640 1 -- 172.21.15.5:0/2184154025 <== osd.1 v2:172.21.15.5:6815/550664725 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x7fbdfc005040 con 0x555a43126830 2024-05-20T18:35:45.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.921+0000 7fbe11c0bf40 1 -- 172.21.15.5:0/2184154025 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x555a43126830 msgr2=0x555a432547a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:45.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.921+0000 7fbe11c0bf40 1 --2- 172.21.15.5:0/2184154025 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x555a43126830 0x555a432547a0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:45.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.921+0000 7fbe11c0bf40 1 -- 172.21.15.5:0/2184154025 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbdec06a810 msgr2=0x7fbdec06cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:45.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.921+0000 7fbe11c0bf40 1 --2- 172.21.15.5:0/2184154025 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fbdec06a810 0x7fbdec06cd20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fbe04000f60 tx=0x7fbe04000ed0 comp rx=0 tx=0).stop 2024-05-20T18:35:45.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.921+0000 7fbe11c0bf40 1 -- 172.21.15.5:0/2184154025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a4312b5f0 msgr2=0x555a431262f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:45.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.921+0000 7fbe11c0bf40 1 --2- 172.21.15.5:0/2184154025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a4312b5f0 0x555a431262f0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fbdf40018f0 tx=0x7fbdf4001a30 comp rx=0 tx=0).stop 2024-05-20T18:35:45.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.921+0000 7fbe0b1cd640 1 -- 172.21.15.5:0/2184154025 reap_dead start 2024-05-20T18:35:45.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.921+0000 7fbe11c0bf40 1 -- 172.21.15.5:0/2184154025 shutdown_connections 2024-05-20T18:35:45.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.921+0000 7fbe11c0bf40 1 -- 172.21.15.5:0/2184154025 >> 172.21.15.5:0/2184154025 conn(0x555a43007ac0 msgr2=0x555a431b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:45.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.922+0000 7fbe11c0bf40 1 -- 172.21.15.5:0/2184154025 shutdown_connections 2024-05-20T18:35:45.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:45.922+0000 7fbe11c0bf40 1 -- 172.21.15.5:0/2184154025 wait complete. 2024-05-20T18:35:45.927 INFO:teuthology.orchestra.run.smithi005.stdout:����� 2024-05-20T18:35:45.927 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-20T18:35:46.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.041+0000 7f6cfae92640 1 Processor -- start 2024-05-20T18:35:46.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.041+0000 7f6cfae92640 1 -- start start 2024-05-20T18:35:46.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.042+0000 7f6cfae92640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6cf4162820 0x7f6cf40b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:46.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.042+0000 7f6cfae92640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6cf40b3e80 0x7f6cf406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:46.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.042+0000 7f6cfae92640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cf406dd20 con 0x7f6cf406d910 2024-05-20T18:35:46.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.042+0000 7f6cfae92640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f6cf40d0630 con 0x7f6cf4162820 2024-05-20T18:35:46.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.042+0000 7f6cfae92640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f6cf40d39c0 con 0x7f6cf40b3e80 2024-05-20T18:35:46.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.042+0000 7f6cf8c2e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6cf4162820 0x7f6cf40b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:46.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.042+0000 7f6cf942f640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f6cf406d910 0x7f6cf40d27a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33604/0 (socket says 172.21.15.5:33604) 2024-05-20T18:35:46.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.042+0000 7f6cf942f640 1 -- 172.21.15.5:0/3630128354 learned_addr learned my addr 172.21.15.5:0/3630128354 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:46.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.042+0000 7f6cf3fff640 1 --2- 172.21.15.5:0/3630128354 >> v2:172.21.15.63:3300/0 conn(0x7f6cf40b3e80 0x7f6cf406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:46.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.043+0000 7f6cf8c2e640 1 -- 172.21.15.5:0/3630128354 >> v2:172.21.15.63:3300/0 conn(0x7f6cf40b3e80 msgr2=0x7f6cf406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:46.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.043+0000 7f6cf8c2e640 1 --2- 172.21.15.5:0/3630128354 >> v2:172.21.15.63:3300/0 conn(0x7f6cf40b3e80 0x7f6cf406d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:46.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.043+0000 7f6cf8c2e640 1 -- 172.21.15.5:0/3630128354 >> v1:172.21.15.5:6789/0 conn(0x7f6cf406d910 legacy=0x7f6cf40d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:46.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.043+0000 7f6cf8c2e640 1 -- 172.21.15.5:0/3630128354 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf40d3ca0 con 0x7f6cf4162820 2024-05-20T18:35:46.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.043+0000 7f6cf37fe640 1 -- 172.21.15.5:0/3630128354 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1354547471 0 0) 0x7f6cec001490 con 0x7f6cf406d910 2024-05-20T18:35:46.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.043+0000 7f6cf8c2e640 1 --2- 172.21.15.5:0/3630128354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cf4162820 0x7f6cf40b3940 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40d1230 tx=0x7f6ce402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1029e9ae1ed9a81 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:46.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.043+0000 7f6cf37fe640 1 -- 172.21.15.5:0/3630128354 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4037070 con 0x7f6cf4162820 2024-05-20T18:35:46.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.044+0000 7f6cf37fe640 1 -- 172.21.15.5:0/3630128354 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ce4032440 con 0x7f6cf4162820 2024-05-20T18:35:46.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.044+0000 7f6cf37fe640 1 -- 172.21.15.5:0/3630128354 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4032810 con 0x7f6cf4162820 2024-05-20T18:35:46.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.044+0000 7f6cfae92640 1 -- 172.21.15.5:0/3630128354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cf4162820 msgr2=0x7f6cf40b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:46.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.044+0000 7f6cfae92640 1 --2- 172.21.15.5:0/3630128354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cf4162820 0x7f6cf40b3940 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40d1230 tx=0x7f6ce402a1a0 comp rx=0 tx=0).stop 2024-05-20T18:35:46.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.044+0000 7f6cfae92640 1 -- 172.21.15.5:0/3630128354 shutdown_connections 2024-05-20T18:35:46.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.044+0000 7f6cfae92640 1 --2- 172.21.15.5:0/3630128354 >> v2:172.21.15.63:3300/0 conn(0x7f6cf40b3e80 0x7f6cf406d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:46.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.044+0000 7f6cfae92640 1 --2- 172.21.15.5:0/3630128354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cf4162820 0x7f6cf40b3940 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:46.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.044+0000 7f6cfae92640 1 -- 172.21.15.5:0/3630128354 >> 172.21.15.5:0/3630128354 conn(0x7f6cf40d4bd0 msgr2=0x7f6cf40d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:46.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.044+0000 7f6cfae92640 1 -- 172.21.15.5:0/3630128354 shutdown_connections 2024-05-20T18:35:46.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.044+0000 7f6cfae92640 1 -- 172.21.15.5:0/3630128354 wait complete. 2024-05-20T18:35:46.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.046+0000 7f6cfae92640 1 Processor -- start 2024-05-20T18:35:46.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.046+0000 7f6cfae92640 1 -- start start 2024-05-20T18:35:46.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.048+0000 7f6cfae92640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cf406d910 0x7f6cf414d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:46.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.048+0000 7f6cfae92640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cf40b3e80 0x7f6cf41f5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:46.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.048+0000 7f6cfae92640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6cf4162820 0x7f6cf40b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:46.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.048+0000 7f6cfae92640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf40d0300 con 0x7f6cf40b3e80 2024-05-20T18:35:46.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.048+0000 7f6cfae92640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf40d4260 con 0x7f6cf4162820 2024-05-20T18:35:46.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.048+0000 7f6cfae92640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6cf40b91a0 con 0x7f6cf406d910 2024-05-20T18:35:46.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.048+0000 7f6cf3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cf40b3e80 0x7f6cf41f5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:46.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.048+0000 7f6cf3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cf40b3e80 0x7f6cf41f5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37866/0 (socket says 172.21.15.5:37866) 2024-05-20T18:35:46.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.048+0000 7f6cf3fff640 1 -- 172.21.15.5:0/1001225115 learned_addr learned my addr 172.21.15.5:0/1001225115 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:46.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.048+0000 7f6cf8c2e640 1 --2- 172.21.15.5:0/1001225115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cf406d910 0x7f6cf414d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:46.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.048+0000 7f6cf942f640 1 --2- 172.21.15.5:0/1001225115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6cf4162820 0x7f6cf40b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:46.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.048+0000 7f6cf3fff640 1 -- 172.21.15.5:0/1001225115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cf406d910 msgr2=0x7f6cf414d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:46.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.048+0000 7f6cf3fff640 1 --2- 172.21.15.5:0/1001225115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cf406d910 0x7f6cf414d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:46.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.048+0000 7f6cf3fff640 1 -- 172.21.15.5:0/1001225115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6cf4162820 msgr2=0x7f6cf40b68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:46.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.048+0000 7f6cf3fff640 1 --2- 172.21.15.5:0/1001225115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6cf4162820 0x7f6cf40b68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:46.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.048+0000 7f6cf3fff640 1 -- 172.21.15.5:0/1001225115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf41f9930 con 0x7f6cf40b3e80 2024-05-20T18:35:46.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.048+0000 7f6cf3fff640 1 --2- 172.21.15.5:0/1001225115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cf40b3e80 0x7f6cf41f5720 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800e8a0 tx=0x7f6ce800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:46.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.049+0000 7f6cf17fa640 1 -- 172.21.15.5:0/1001225115 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8017070 con 0x7f6cf40b3e80 2024-05-20T18:35:46.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.049+0000 7f6cf17fa640 1 -- 172.21.15.5:0/1001225115 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ce800c8e0 con 0x7f6cf40b3e80 2024-05-20T18:35:46.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.049+0000 7f6cf17fa640 1 -- 172.21.15.5:0/1001225115 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce800ccd0 con 0x7f6cf40b3e80 2024-05-20T18:35:46.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.049+0000 7f6cfae92640 1 -- 172.21.15.5:0/1001225115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cf414d920 con 0x7f6cf40b3e80 2024-05-20T18:35:46.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.049+0000 7f6cfae92640 1 -- 172.21.15.5:0/1001225115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cf414dee0 con 0x7f6cf40b3e80 2024-05-20T18:35:46.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.050+0000 7f6cfae92640 1 -- 172.21.15.5:0/1001225115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cf40ba390 con 0x7f6cf40b3e80 2024-05-20T18:35:46.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.050+0000 7f6cf17fa640 1 -- 172.21.15.5:0/1001225115 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6ce801a070 con 0x7f6cf40b3e80 2024-05-20T18:35:46.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.051+0000 7f6cf17fa640 1 --2- 172.21.15.5:0/1001225115 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6ce006be60 0x7f6ce006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:46.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.051+0000 7f6cf17fa640 1 -- 172.21.15.5:0/1001225115 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7f6ce808b170 con 0x7f6cf40b3e80 2024-05-20T18:35:46.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.051+0000 7f6cf8c2e640 1 --2- 172.21.15.5:0/1001225115 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6ce006be60 0x7f6ce006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:46.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.051+0000 7f6cf8c2e640 1 --2- 172.21.15.5:0/1001225115 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6ce006be60 0x7f6ce006e370 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4034000 tx=0x7f6ce402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:46.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.055+0000 7f6cf17fa640 1 -- 172.21.15.5:0/1001225115 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6cf40ba390 con 0x7f6cf40b3e80 2024-05-20T18:35:46.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.277+0000 7f6cfae92640 1 -- 172.21.15.5:0/1001225115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6cf4075b00 con 0x7f6cf40b3e80 2024-05-20T18:35:46.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.277+0000 7f6cf17fa640 1 -- 172.21.15.5:0/1001225115 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) ==== 53+0+73 (secure 0 0 0) 0x7f6ce80592a0 con 0x7f6cf40b3e80 2024-05-20T18:35:46.278 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-20T18:35:46.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.280+0000 7f6cfae92640 1 -- 172.21.15.5:0/1001225115 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6ce006be60 msgr2=0x7f6ce006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:46.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.280+0000 7f6cfae92640 1 --2- 172.21.15.5:0/1001225115 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6ce006be60 0x7f6ce006e370 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4034000 tx=0x7f6ce402e000 comp rx=0 tx=0).stop 2024-05-20T18:35:46.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.280+0000 7f6cfae92640 1 -- 172.21.15.5:0/1001225115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cf40b3e80 msgr2=0x7f6cf41f5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:46.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.280+0000 7f6cfae92640 1 --2- 172.21.15.5:0/1001225115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cf40b3e80 0x7f6cf41f5720 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800e8a0 tx=0x7f6ce800ed60 comp rx=0 tx=0).stop 2024-05-20T18:35:46.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.280+0000 7f6cfae92640 1 -- 172.21.15.5:0/1001225115 shutdown_connections 2024-05-20T18:35:46.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.280+0000 7f6cfae92640 1 --2- 172.21.15.5:0/1001225115 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6ce006be60 0x7f6ce006e370 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:46.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.280+0000 7f6cfae92640 1 --2- 172.21.15.5:0/1001225115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6cf4162820 0x7f6cf40b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:46.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.280+0000 7f6cfae92640 1 --2- 172.21.15.5:0/1001225115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cf40b3e80 0x7f6cf41f5720 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:46.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.280+0000 7f6cfae92640 1 --2- 172.21.15.5:0/1001225115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cf406d910 0x7f6cf414d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:46.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.280+0000 7f6cfae92640 1 -- 172.21.15.5:0/1001225115 >> 172.21.15.5:0/1001225115 conn(0x7f6cf40d4bd0 msgr2=0x7f6cf4160eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:46.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.280+0000 7f6cfae92640 1 -- 172.21.15.5:0/1001225115 shutdown_connections 2024-05-20T18:35:46.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46.280+0000 7f6cfae92640 1 -- 172.21.15.5:0/1001225115 wait complete. 2024-05-20T18:35:46.292 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-20T18:35:46.292 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-20T18:35:46.292 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-05-20T18:35:46.292 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-20T18:35:46.292 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-20T18:35:46.292 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-20T18:35:46.292 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-20T18:35:46.292 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-20T18:35:46.292 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:35:46.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:35:46.293 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-05-20T18:35:46.309 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T18:35:46.309 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T18:35:46.310 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T18:35:46.310 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:35:46.310 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-20T18:35:46.310 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:35:46.310 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:35:46.310 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-05-20T18:35:46.310 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-05-20T18:35:46.310 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:35:46.310 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-05-20T18:35:46.310 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-05-20T18:35:46.310 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-20T18:35:46.310 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:35:46.310 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-05-20T18:35:46.311 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-20T18:35:46.311 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42152sec preferred_lft 42152sec 2024-05-20T18:35:46.311 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-05-20T18:35:46.311 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:35:46.311 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:35:46.311 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:35:46.311 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-05-20T18:35:46.311 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-20T18:35:46.311 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:35:46.311 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-20T18:35:46.311 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:35:46.311 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::4474:eaff:fe07:4f9a/64 scope link 2024-05-20T18:35:46.311 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:35:46.312 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:35:46.312 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T18:35:46.312 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ce4:f4ff:fe5e:7906/64 scope link 2024-05-20T18:35:46.312 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:35:46.312 INFO:teuthology.orchestra.run.smithi005.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:35:46.312 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether da:26:04:0e:95:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T18:35:46.312 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::d826:4ff:fe0e:950a/64 scope link 2024-05-20T18:35:46.312 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:35:46.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:35:46.312 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-20T18:35:46.374 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-20T18:35:46.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:35:46.376 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-05-20T18:35:46.432 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-05-20T18:35:46.432 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-20T18:35:46.432 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-20T18:35:46.432 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-20T18:35:46.432 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-20T18:35:46.433 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:35:46.433 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-20T18:35:46.433 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-20T18:35:46.433 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-20T18:35:46.433 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-20T18:35:46.433 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-20T18:35:46.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:35:46.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:35:46.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:35:46.563 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-20T18:35:46.579 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-20T18:35:46.579 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-20T18:35:46.579 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:35:46.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:35:46.579 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-05-20T18:35:46.645 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:35:46.645 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-20T18:35:46.700 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:35:46.701 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:35:46.701 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:35:46.756 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-20T18:35:46.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:35:46.757 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:35:46.853 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:35:46.853 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:35:46.853 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:35:46.864 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:35:46.864 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:35:46.864 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:35:46.864 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:35:46.864 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:35:46.882 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:35:46.883 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:35:46.883 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:35:46.883 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:35:46.883 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:35:46.883 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790" 2024-05-20T18:35:46.883 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790:/ 2024-05-20T18:35:46.883 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=399ab250-fd3c-47fa-a1c8-25e8149c9cac" will pass to kernel 2024-05-20T18:35:46.883 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-20T18:35:46.884 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-05-20T18:35:46.884 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-20T18:35:46.884 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-05-20T18:35:46.884 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-20T18:35:46.885 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-20T18:35:46.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:35:46.885 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:35:46.911 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-20T18:35:47.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T18:35:47.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:35:47.009 INFO:teuthology.orchestra.run.smithi005.stdout:client7726 2024-05-20T18:35:47.010 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:35:47.010 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:35:47.026 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:35:47.026 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:35:47.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:35:47.083 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-05-20T18:35:47.172 INFO:teuthology.orchestra.run.smithi005.stderr:6+0 records in 2024-05-20T18:35:47.172 INFO:teuthology.orchestra.run.smithi005.stderr:6+0 records out 2024-05-20T18:35:47.172 INFO:teuthology.orchestra.run.smithi005.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0299524 s, 210 MB/s 2024-05-20T18:35:47.172 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:35:47.172 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:35:47.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:35:47.189 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-05-20T18:35:47.279 INFO:teuthology.orchestra.run.smithi005.stderr:6+0 records in 2024-05-20T18:35:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:6+0 records out 2024-05-20T18:35:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0330732 s, 190 MB/s 2024-05-20T18:35:47.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:35:47.281 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-20T18:35:47.299 INFO:teuthology.orchestra.run.smithi005.stdout:./ 2024-05-20T18:35:47.299 INFO:teuthology.orchestra.run.smithi005.stdout:./file2_sixmegs 2024-05-20T18:35:47.299 INFO:teuthology.orchestra.run.smithi005.stdout:./file3_sixmegs 2024-05-20T18:35:47.299 INFO:teuthology.orchestra.run.smithi005.stdout:./file1_sixmegs 2024-05-20T18:35:47.299 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:35:47.300 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:35:47.300 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:35:47.300 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:35:47.300 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-20T18:35:47.300 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:35:47.408 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-20T18:35:47.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:35:47.417 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:35:47.418 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:35:47.418 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:35:47.418 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:35:47.418 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2024-05-20T18:35:47.418 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:35:47.526 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627777 2024-05-20T18:35:47.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:35:47.534 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:35:47.535 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:35:47.535 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:35:47.535 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:35:47.535 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2024-05-20T18:35:47.535 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:35:47.643 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627778 2024-05-20T18:35:47.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:35:47.651 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:35:47.652 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:35:47.652 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:35:47.652 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:35:47.652 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2024-05-20T18:35:47.652 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:35:47.759 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627776 2024-05-20T18:35:47.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:35:47.769 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:35:47.769 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:35:47.826 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:35:47.826 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:35:47.883 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:35:48.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.038+0000 7fdc91480640 1 Processor -- start 2024-05-20T18:35:48.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.038+0000 7fdc91480640 1 -- start start 2024-05-20T18:35:48.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.040+0000 7fdc91480640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdc8c162820 0x7fdc8c0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:48.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.040+0000 7fdc91480640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdc8c06dae0 0x7fdc8c0b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:48.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.040+0000 7fdc91480640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc8c0d0630 con 0x7fdc8c0b3e80 2024-05-20T18:35:48.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.040+0000 7fdc91480640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fdc8c0d39c0 con 0x7fdc8c162820 2024-05-20T18:35:48.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.040+0000 7fdc91480640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdc8c0d3ca0 con 0x7fdc8c06dae0 2024-05-20T18:35:48.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.040+0000 7fdc8affd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdc8c162820 0x7fdc8c0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:48.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.040+0000 7fdc8a7fc640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fdc8c0b3e80 0x7fdc8c06d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33608/0 (socket says 172.21.15.5:33608) 2024-05-20T18:35:48.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.040+0000 7fdc8a7fc640 1 -- 172.21.15.5:0/987852976 learned_addr learned my addr 172.21.15.5:0/987852976 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:48.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.040+0000 7fdc8affd640 1 -- 172.21.15.5:0/987852976 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdc8c06dae0 msgr2=0x7fdc8c0b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:48.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.040+0000 7fdc8affd640 1 --2- 172.21.15.5:0/987852976 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdc8c06dae0 0x7fdc8c0b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:48.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.040+0000 7fdc8affd640 1 -- 172.21.15.5:0/987852976 >> v1:172.21.15.5:6789/0 conn(0x7fdc8c0b3e80 legacy=0x7fdc8c06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:48.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.040+0000 7fdc89ffb640 1 -- 172.21.15.5:0/987852976 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1426951520 0 0) 0x7fdc8c0d0630 con 0x7fdc8c0b3e80 2024-05-20T18:35:48.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.040+0000 7fdc8affd640 1 -- 172.21.15.5:0/987852976 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc8c0d3f80 con 0x7fdc8c162820 2024-05-20T18:35:48.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.041+0000 7fdc8affd640 1 --2- 172.21.15.5:0/987852976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc8c162820 0x7fdc8c0b3940 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c009d90 tx=0x7fdc7c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725173bdb9c76c6e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:48.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.041+0000 7fdc89ffb640 1 -- 172.21.15.5:0/987852976 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c007740 con 0x7fdc8c162820 2024-05-20T18:35:48.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.041+0000 7fdc89ffb640 1 -- 172.21.15.5:0/987852976 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc7c0078e0 con 0x7fdc8c162820 2024-05-20T18:35:48.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.041+0000 7fdc89ffb640 1 -- 172.21.15.5:0/987852976 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c007cb0 con 0x7fdc8c162820 2024-05-20T18:35:48.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.042+0000 7fdc91480640 1 -- 172.21.15.5:0/987852976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc8c162820 msgr2=0x7fdc8c0b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:48.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.042+0000 7fdc91480640 1 --2- 172.21.15.5:0/987852976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc8c162820 0x7fdc8c0b3940 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c009d90 tx=0x7fdc7c02a680 comp rx=0 tx=0).stop 2024-05-20T18:35:48.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.042+0000 7fdc91480640 1 -- 172.21.15.5:0/987852976 shutdown_connections 2024-05-20T18:35:48.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.042+0000 7fdc91480640 1 --2- 172.21.15.5:0/987852976 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdc8c06dae0 0x7fdc8c0b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:48.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.042+0000 7fdc91480640 1 --2- 172.21.15.5:0/987852976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc8c162820 0x7fdc8c0b3940 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:48.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.042+0000 7fdc91480640 1 -- 172.21.15.5:0/987852976 >> 172.21.15.5:0/987852976 conn(0x7fdc8c0d4bd0 msgr2=0x7fdc8c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:48.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.042+0000 7fdc91480640 1 -- 172.21.15.5:0/987852976 shutdown_connections 2024-05-20T18:35:48.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.042+0000 7fdc91480640 1 -- 172.21.15.5:0/987852976 wait complete. 2024-05-20T18:35:48.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.044+0000 7fdc91480640 1 Processor -- start 2024-05-20T18:35:48.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.044+0000 7fdc91480640 1 -- start start 2024-05-20T18:35:48.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.045+0000 7fdc91480640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc8c0b3e80 0x7fdc8c0cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:48.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.045+0000 7fdc91480640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc8c162820 0x7fdc8c1f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:48.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.045+0000 7fdc91480640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdc8c0b6e00 0x7fdc8c0cded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:48.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.045+0000 7fdc91480640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c0d4780 con 0x7fdc8c162820 2024-05-20T18:35:48.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.045+0000 7fdc91480640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c0d0300 con 0x7fdc8c0b3e80 2024-05-20T18:35:48.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.045+0000 7fdc91480640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdc8c0d4900 con 0x7fdc8c0b6e00 2024-05-20T18:35:48.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.045+0000 7fdc8a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc8c162820 0x7fdc8c1f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:48.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.046+0000 7fdc8a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc8c162820 0x7fdc8c1f4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37892/0 (socket says 172.21.15.5:37892) 2024-05-20T18:35:48.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.046+0000 7fdc8a7fc640 1 -- 172.21.15.5:0/2029342970 learned_addr learned my addr 172.21.15.5:0/2029342970 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:48.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.046+0000 7fdc8affd640 1 --2- 172.21.15.5:0/2029342970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc8c0b3e80 0x7fdc8c0cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:48.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.046+0000 7fdc8b7fe640 1 --2- 172.21.15.5:0/2029342970 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdc8c0b6e00 0x7fdc8c0cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:48.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.046+0000 7fdc8a7fc640 1 -- 172.21.15.5:0/2029342970 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdc8c0b6e00 msgr2=0x7fdc8c0cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:48.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.046+0000 7fdc8a7fc640 1 --2- 172.21.15.5:0/2029342970 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdc8c0b6e00 0x7fdc8c0cded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:48.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.046+0000 7fdc8a7fc640 1 -- 172.21.15.5:0/2029342970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc8c0b3e80 msgr2=0x7fdc8c0cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:48.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.046+0000 7fdc8a7fc640 1 --2- 172.21.15.5:0/2029342970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc8c0b3e80 0x7fdc8c0cc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:48.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.046+0000 7fdc8a7fc640 1 -- 172.21.15.5:0/2029342970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc8c1f6080 con 0x7fdc8c162820 2024-05-20T18:35:48.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.046+0000 7fdc8affd640 1 --2- 172.21.15.5:0/2029342970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc8c0b3e80 0x7fdc8c0cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:48.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.046+0000 7fdc8a7fc640 1 --2- 172.21.15.5:0/2029342970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc8c162820 0x7fdc8c1f4e80 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fdc7400c430 tx=0x7fdc7400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:48.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.046+0000 7fdc837fe640 1 -- 172.21.15.5:0/2029342970 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc740074e0 con 0x7fdc8c162820 2024-05-20T18:35:48.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.046+0000 7fdc837fe640 1 -- 172.21.15.5:0/2029342970 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc740047e0 con 0x7fdc8c162820 2024-05-20T18:35:48.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.046+0000 7fdc837fe640 1 -- 172.21.15.5:0/2029342970 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc74004bd0 con 0x7fdc8c162820 2024-05-20T18:35:48.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.046+0000 7fdc91480640 1 -- 172.21.15.5:0/2029342970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc8c1f6360 con 0x7fdc8c162820 2024-05-20T18:35:48.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.046+0000 7fdc91480640 1 -- 172.21.15.5:0/2029342970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc8c0b9830 con 0x7fdc8c162820 2024-05-20T18:35:48.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.047+0000 7fdc91480640 1 -- 172.21.15.5:0/2029342970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc8c0cbdd0 con 0x7fdc8c162820 2024-05-20T18:35:48.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.048+0000 7fdc837fe640 1 -- 172.21.15.5:0/2029342970 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdc74011070 con 0x7fdc8c162820 2024-05-20T18:35:48.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.048+0000 7fdc837fe640 1 --2- 172.21.15.5:0/2029342970 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdc6c06be30 0x7fdc6c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:48.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.049+0000 7fdc837fe640 1 -- 172.21.15.5:0/2029342970 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7fdc7408ac00 con 0x7fdc8c162820 2024-05-20T18:35:48.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.049+0000 7fdc8affd640 1 --2- 172.21.15.5:0/2029342970 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdc6c06be30 0x7fdc6c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:48.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.049+0000 7fdc8affd640 1 --2- 172.21.15.5:0/2029342970 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdc6c06be30 0x7fdc6c06e340 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c034000 tx=0x7fdc7c02f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:48.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.053+0000 7fdc837fe640 1 -- 172.21.15.5:0/2029342970 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fdc74091080 con 0x7fdc8c162820 2024-05-20T18:35:48.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.238+0000 7fdc91480640 1 -- 172.21.15.5:0/2029342970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdc8c15efb0 con 0x7fdc8c162820 2024-05-20T18:35:48.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.240+0000 7fdc837fe640 1 -- 172.21.15.5:0/2029342970 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+16764 (secure 0 0 0) 0x7fdc8c15efb0 con 0x7fdc8c162820 2024-05-20T18:35:48.240 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:48.240 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":105,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:35:32.410574+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:35:29.146063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-20T18:35:29.754813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:35:48.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.242+0000 7fdc91480640 1 -- 172.21.15.5:0/2029342970 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdc6c06be30 msgr2=0x7fdc6c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:48.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.242+0000 7fdc91480640 1 --2- 172.21.15.5:0/2029342970 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdc6c06be30 0x7fdc6c06e340 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c034000 tx=0x7fdc7c02f000 comp rx=0 tx=0).stop 2024-05-20T18:35:48.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.242+0000 7fdc91480640 1 -- 172.21.15.5:0/2029342970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc8c162820 msgr2=0x7fdc8c1f4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:48.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.242+0000 7fdc91480640 1 --2- 172.21.15.5:0/2029342970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc8c162820 0x7fdc8c1f4e80 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fdc7400c430 tx=0x7fdc7400c8f0 comp rx=0 tx=0).stop 2024-05-20T18:35:48.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.242+0000 7fdc91480640 1 -- 172.21.15.5:0/2029342970 shutdown_connections 2024-05-20T18:35:48.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.242+0000 7fdc91480640 1 --2- 172.21.15.5:0/2029342970 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdc6c06be30 0x7fdc6c06e340 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:48.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.242+0000 7fdc91480640 1 --2- 172.21.15.5:0/2029342970 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdc8c0b6e00 0x7fdc8c0cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:48.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.242+0000 7fdc91480640 1 --2- 172.21.15.5:0/2029342970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc8c162820 0x7fdc8c1f4e80 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:48.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.242+0000 7fdc91480640 1 --2- 172.21.15.5:0/2029342970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc8c0b3e80 0x7fdc8c0cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:48.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.242+0000 7fdc91480640 1 -- 172.21.15.5:0/2029342970 >> 172.21.15.5:0/2029342970 conn(0x7fdc8c0d4bd0 msgr2=0x7fdc8c15dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:48.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.242+0000 7fdc91480640 1 -- 172.21.15.5:0/2029342970 shutdown_connections 2024-05-20T18:35:48.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.242+0000 7fdc91480640 1 -- 172.21.15.5:0/2029342970 wait complete. 2024-05-20T18:35:48.254 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:35:29.146063+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:35:29.754813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:35:48.254 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:35:48.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.371+0000 7f64ba7ea640 1 Processor -- start 2024-05-20T18:35:48.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.371+0000 7f64ba7ea640 1 -- start start 2024-05-20T18:35:48.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.373+0000 7f64ba7ea640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f64b406d3d0 0x7f64b406d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:48.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.373+0000 7f64ba7ea640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f64b40b8190 0x7f64b40b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:48.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.373+0000 7f64ba7ea640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64b406dd20 con 0x7f64b4162820 2024-05-20T18:35:48.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.373+0000 7f64ba7ea640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f64b40d0630 con 0x7f64b40b8190 2024-05-20T18:35:48.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.373+0000 7f64ba7ea640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f64b40d39c0 con 0x7f64b406d3d0 2024-05-20T18:35:48.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.373+0000 7f64b8d87640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f64b40b8190 0x7f64b40b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:48.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.373+0000 7f64b8d87640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f64b40b8190 0x7f64b40b85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37908/0 (socket says 172.21.15.5:37908) 2024-05-20T18:35:48.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.373+0000 7f64b8d87640 1 -- 172.21.15.5:0/2372766014 learned_addr learned my addr 172.21.15.5:0/2372766014 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:48.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.373+0000 7f64b37fe640 1 --2- 172.21.15.5:0/2372766014 >> v2:172.21.15.63:3300/0 conn(0x7f64b406d3d0 0x7f64b406d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:48.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.373+0000 7f64b8d87640 1 -- 172.21.15.5:0/2372766014 >> v2:172.21.15.63:3300/0 conn(0x7f64b406d3d0 msgr2=0x7f64b406d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:48.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.373+0000 7f64b8d87640 1 --2- 172.21.15.5:0/2372766014 >> v2:172.21.15.63:3300/0 conn(0x7f64b406d3d0 0x7f64b406d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:48.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.373+0000 7f64b8d87640 1 -- 172.21.15.5:0/2372766014 >> v1:172.21.15.63:6789/0 conn(0x7f64b4162820 legacy=0x7f64b40b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:48.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.373+0000 7f64b8d87640 1 -- 172.21.15.5:0/2372766014 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64b40d3ca0 con 0x7f64b40b8190 2024-05-20T18:35:48.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.374+0000 7f64b8d87640 1 --2- 172.21.15.5:0/2372766014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64b40b8190 0x7f64b40b85a0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f64b40d1230 tx=0x7f64ac02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f861009f6f68559 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:48.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.374+0000 7f64b2ffd640 1 -- 172.21.15.5:0/2372766014 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac036070 con 0x7f64b40b8190 2024-05-20T18:35:48.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.374+0000 7f64b2ffd640 1 -- 172.21.15.5:0/2372766014 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64ac031600 con 0x7f64b40b8190 2024-05-20T18:35:48.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.374+0000 7f64b2ffd640 1 -- 172.21.15.5:0/2372766014 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac0319d0 con 0x7f64b40b8190 2024-05-20T18:35:48.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.375+0000 7f64ba7ea640 1 -- 172.21.15.5:0/2372766014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64b40b8190 msgr2=0x7f64b40b85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:48.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.375+0000 7f64ba7ea640 1 --2- 172.21.15.5:0/2372766014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64b40b8190 0x7f64b40b85a0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f64b40d1230 tx=0x7f64ac02a730 comp rx=0 tx=0).stop 2024-05-20T18:35:48.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.375+0000 7f64ba7ea640 1 -- 172.21.15.5:0/2372766014 shutdown_connections 2024-05-20T18:35:48.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.375+0000 7f64ba7ea640 1 --2- 172.21.15.5:0/2372766014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64b40b8190 0x7f64b40b85a0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:48.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.375+0000 7f64ba7ea640 1 --2- 172.21.15.5:0/2372766014 >> v2:172.21.15.63:3300/0 conn(0x7f64b406d3d0 0x7f64b406d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:48.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.375+0000 7f64ba7ea640 1 -- 172.21.15.5:0/2372766014 >> 172.21.15.5:0/2372766014 conn(0x7f64b40d4bd0 msgr2=0x7f64b40d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:48.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.375+0000 7f64ba7ea640 1 -- 172.21.15.5:0/2372766014 shutdown_connections 2024-05-20T18:35:48.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.375+0000 7f64ba7ea640 1 -- 172.21.15.5:0/2372766014 wait complete. 2024-05-20T18:35:48.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.377+0000 7f64ba7ea640 1 Processor -- start 2024-05-20T18:35:48.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.377+0000 7f64ba7ea640 1 -- start start 2024-05-20T18:35:48.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.378+0000 7f64ba7ea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64b406d3d0 0x7f64b41f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:48.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.378+0000 7f64ba7ea640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64b40b8190 0x7f64b4147c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:48.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.378+0000 7f64ba7ea640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64b4162820 0x7f64b40b9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:48.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.378+0000 7f64ba7ea640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f64b40d0300 con 0x7f64b406d3d0 2024-05-20T18:35:48.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.378+0000 7f64ba7ea640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f64b40d4260 con 0x7f64b4162820 2024-05-20T18:35:48.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.378+0000 7f64ba7ea640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f64b40b4050 con 0x7f64b40b8190 2024-05-20T18:35:48.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.378+0000 7f64b3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64b406d3d0 0x7f64b41f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:48.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.378+0000 7f64b3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64b406d3d0 0x7f64b41f5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37922/0 (socket says 172.21.15.5:37922) 2024-05-20T18:35:48.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.378+0000 7f64b3fff640 1 -- 172.21.15.5:0/2604162087 learned_addr learned my addr 172.21.15.5:0/2604162087 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:48.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.379+0000 7f64b8d87640 1 --2- 172.21.15.5:0/2604162087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64b4162820 0x7f64b40b9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:48.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.379+0000 7f64b37fe640 1 --2- 172.21.15.5:0/2604162087 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64b40b8190 0x7f64b4147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:48.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.379+0000 7f64b3fff640 1 -- 172.21.15.5:0/2604162087 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64b40b8190 msgr2=0x7f64b4147c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:48.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.379+0000 7f64b3fff640 1 --2- 172.21.15.5:0/2604162087 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64b40b8190 0x7f64b4147c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:48.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.379+0000 7f64b3fff640 1 -- 172.21.15.5:0/2604162087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64b4162820 msgr2=0x7f64b40b9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:48.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.379+0000 7f64b3fff640 1 --2- 172.21.15.5:0/2604162087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64b4162820 0x7f64b40b9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:48.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.379+0000 7f64b3fff640 1 -- 172.21.15.5:0/2604162087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64b41e7940 con 0x7f64b406d3d0 2024-05-20T18:35:48.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.379+0000 7f64b8d87640 1 --2- 172.21.15.5:0/2604162087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64b4162820 0x7f64b40b9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:48.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.379+0000 7f64b37fe640 1 --2- 172.21.15.5:0/2604162087 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64b40b8190 0x7f64b4147c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:48.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.379+0000 7f64b3fff640 1 --2- 172.21.15.5:0/2604162087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64b406d3d0 0x7f64b41f5f20 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f64a400c3f0 tx=0x7f64a400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:48.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.379+0000 7f64b0ff9640 1 -- 172.21.15.5:0/2604162087 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a40052d0 con 0x7f64b406d3d0 2024-05-20T18:35:48.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.379+0000 7f64b0ff9640 1 -- 172.21.15.5:0/2604162087 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64a4005490 con 0x7f64b406d3d0 2024-05-20T18:35:48.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.379+0000 7f64ba7ea640 1 -- 172.21.15.5:0/2604162087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64b41e7db0 con 0x7f64b406d3d0 2024-05-20T18:35:48.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.379+0000 7f64b0ff9640 1 -- 172.21.15.5:0/2604162087 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a4007260 con 0x7f64b406d3d0 2024-05-20T18:35:48.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.379+0000 7f64ba7ea640 1 -- 172.21.15.5:0/2604162087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64b41e8370 con 0x7f64b406d3d0 2024-05-20T18:35:48.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.380+0000 7f64ba7ea640 1 -- 172.21.15.5:0/2604162087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64b4075b00 con 0x7f64b406d3d0 2024-05-20T18:35:48.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.381+0000 7f64b0ff9640 1 -- 172.21.15.5:0/2604162087 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f64a4007400 con 0x7f64b406d3d0 2024-05-20T18:35:48.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.381+0000 7f64b0ff9640 1 --2- 172.21.15.5:0/2604162087 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f64a006be30 0x7f64a006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:48.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.381+0000 7f64b0ff9640 1 -- 172.21.15.5:0/2604162087 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7f64a408a540 con 0x7f64b406d3d0 2024-05-20T18:35:48.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.381+0000 7f64b37fe640 1 --2- 172.21.15.5:0/2604162087 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f64a006be30 0x7f64a006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:48.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.382+0000 7f64b37fe640 1 --2- 172.21.15.5:0/2604162087 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f64a006be30 0x7f64a006e340 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f64b40ba2c0 tx=0x7f64a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:48.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.386+0000 7f64b0ff9640 1 -- 172.21.15.5:0/2604162087 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f64b406dd20 con 0x7f64b406d3d0 2024-05-20T18:35:48.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.568+0000 7f64ba7ea640 1 -- 172.21.15.5:0/2604162087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f64b406e1c0 con 0x7f64b406d3d0 2024-05-20T18:35:48.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.568+0000 7f64b0ff9640 1 -- 172.21.15.5:0/2604162087 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) ==== 93+0+10 (secure 0 0 0) 0x7f64a4058cd0 con 0x7f64b406d3d0 2024-05-20T18:35:48.568 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:35:48.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.570+0000 7f64ba7ea640 1 -- 172.21.15.5:0/2604162087 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f64a006be30 msgr2=0x7f64a006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:48.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.570+0000 7f64ba7ea640 1 --2- 172.21.15.5:0/2604162087 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f64a006be30 0x7f64a006e340 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f64b40ba2c0 tx=0x7f64a8008040 comp rx=0 tx=0).stop 2024-05-20T18:35:48.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.570+0000 7f64ba7ea640 1 -- 172.21.15.5:0/2604162087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64b406d3d0 msgr2=0x7f64b41f5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:48.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.570+0000 7f64ba7ea640 1 --2- 172.21.15.5:0/2604162087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64b406d3d0 0x7f64b41f5f20 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f64a400c3f0 tx=0x7f64a400c8b0 comp rx=0 tx=0).stop 2024-05-20T18:35:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.570+0000 7f64ba7ea640 1 -- 172.21.15.5:0/2604162087 shutdown_connections 2024-05-20T18:35:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.570+0000 7f64ba7ea640 1 --2- 172.21.15.5:0/2604162087 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f64a006be30 0x7f64a006e340 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.570+0000 7f64ba7ea640 1 --2- 172.21.15.5:0/2604162087 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64b4162820 0x7f64b40b9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.570+0000 7f64ba7ea640 1 --2- 172.21.15.5:0/2604162087 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f64b40b8190 0x7f64b4147c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.570+0000 7f64ba7ea640 1 --2- 172.21.15.5:0/2604162087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64b406d3d0 0x7f64b41f5f20 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.570+0000 7f64ba7ea640 1 -- 172.21.15.5:0/2604162087 >> 172.21.15.5:0/2604162087 conn(0x7f64b40d4bd0 msgr2=0x7f64b415f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.570+0000 7f64ba7ea640 1 -- 172.21.15.5:0/2604162087 shutdown_connections 2024-05-20T18:35:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.570+0000 7f64ba7ea640 1 -- 172.21.15.5:0/2604162087 wait complete. 2024-05-20T18:35:48.582 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:35:48.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.699+0000 7f8b7cd02640 1 Processor -- start 2024-05-20T18:35:48.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.699+0000 7f8b7cd02640 1 -- start start 2024-05-20T18:35:48.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.701+0000 7f8b7cd02640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8b7806ca80 0x7f8b7815f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:48.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.701+0000 7f8b7cd02640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b780ae320 0x7f8b7806bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:48.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.701+0000 7f8b7cd02640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b780b1b80 con 0x7f8b780ae730 2024-05-20T18:35:48.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.701+0000 7f8b7cd02640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8b7806b750 con 0x7f8b7806ca80 2024-05-20T18:35:48.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.701+0000 7f8b7cd02640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8b780b12e0 con 0x7f8b780ae320 2024-05-20T18:35:48.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.702+0000 7f8b7659c640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8b7806ca80 0x7f8b7815f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:48.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.702+0000 7f8b76d9d640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f8b780ae730 0x7f8b7806c120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:48182/0 (socket says 172.21.15.5:48182) 2024-05-20T18:35:48.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.702+0000 7f8b75d9b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b780ae320 0x7f8b7806bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:48.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.702+0000 7f8b76d9d640 1 -- 172.21.15.5:0/3763368733 learned_addr learned my addr 172.21.15.5:0/3763368733 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:48.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.702+0000 7f8b7659c640 1 -- 172.21.15.5:0/3763368733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b780ae320 msgr2=0x7f8b7806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:48.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.702+0000 7f8b7659c640 1 --2- 172.21.15.5:0/3763368733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b780ae320 0x7f8b7806bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:48.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.702+0000 7f8b7659c640 1 -- 172.21.15.5:0/3763368733 >> v1:172.21.15.63:6789/0 conn(0x7f8b780ae730 legacy=0x7f8b7806c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:48.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.702+0000 7f8b7659c640 1 -- 172.21.15.5:0/3763368733 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b780b15c0 con 0x7f8b7806ca80 2024-05-20T18:35:48.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.703+0000 7f8b7659c640 1 --2- 172.21.15.5:0/3763368733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b7806ca80 0x7f8b7815f280 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009d90 tx=0x7f8b6802a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53793124252efca0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:48.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.703+0000 7f8b7559a640 1 -- 172.21.15.5:0/3763368733 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b68007740 con 0x7f8b7806ca80 2024-05-20T18:35:48.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.703+0000 7f8b7559a640 1 -- 172.21.15.5:0/3763368733 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b680078e0 con 0x7f8b7806ca80 2024-05-20T18:35:48.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.703+0000 7f8b7559a640 1 -- 172.21.15.5:0/3763368733 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b68007cb0 con 0x7f8b7806ca80 2024-05-20T18:35:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.704+0000 7f8b7cd02640 1 -- 172.21.15.5:0/3763368733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b7806ca80 msgr2=0x7f8b7815f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.704+0000 7f8b7cd02640 1 --2- 172.21.15.5:0/3763368733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b7806ca80 0x7f8b7815f280 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009d90 tx=0x7f8b6802a560 comp rx=0 tx=0).stop 2024-05-20T18:35:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.704+0000 7f8b7cd02640 1 -- 172.21.15.5:0/3763368733 shutdown_connections 2024-05-20T18:35:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.704+0000 7f8b7cd02640 1 --2- 172.21.15.5:0/3763368733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b780ae320 0x7f8b7806bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.704+0000 7f8b7cd02640 1 --2- 172.21.15.5:0/3763368733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b7806ca80 0x7f8b7815f280 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.704+0000 7f8b7cd02640 1 -- 172.21.15.5:0/3763368733 >> 172.21.15.5:0/3763368733 conn(0x7f8b78072ed0 msgr2=0x7f8b780b6080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.704+0000 7f8b7cd02640 1 -- 172.21.15.5:0/3763368733 shutdown_connections 2024-05-20T18:35:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.704+0000 7f8b7cd02640 1 -- 172.21.15.5:0/3763368733 wait complete. 2024-05-20T18:35:48.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.706+0000 7f8b7cd02640 1 Processor -- start 2024-05-20T18:35:48.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.706+0000 7f8b7cd02640 1 -- start start 2024-05-20T18:35:48.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.707+0000 7f8b7cd02640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b7806ca80 0x7f8b781516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.707+0000 7f8b7cd02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b780ae320 0x7f8b780ce750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.707+0000 7f8b7cd02640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b780ae730 0x7f8b780cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.707+0000 7f8b7cd02640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8b780d0c10 con 0x7f8b780ae320 2024-05-20T18:35:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.707+0000 7f8b7cd02640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8b78074e00 con 0x7f8b7806ca80 2024-05-20T18:35:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.707+0000 7f8b7cd02640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8b780d0d90 con 0x7f8b780ae730 2024-05-20T18:35:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.707+0000 7f8b75d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b780ae320 0x7f8b780ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.707+0000 7f8b75d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b780ae320 0x7f8b780ce750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37928/0 (socket says 172.21.15.5:37928) 2024-05-20T18:35:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.707+0000 7f8b75d9b640 1 -- 172.21.15.5:0/2107986239 learned_addr learned my addr 172.21.15.5:0/2107986239 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.708+0000 7f8b7659c640 1 --2- 172.21.15.5:0/2107986239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b7806ca80 0x7f8b781516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.708+0000 7f8b76d9d640 1 --2- 172.21.15.5:0/2107986239 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b780ae730 0x7f8b780cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.708+0000 7f8b75d9b640 1 -- 172.21.15.5:0/2107986239 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b780ae730 msgr2=0x7f8b780cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.708+0000 7f8b75d9b640 1 --2- 172.21.15.5:0/2107986239 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b780ae730 0x7f8b780cfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.708+0000 7f8b75d9b640 1 -- 172.21.15.5:0/2107986239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b7806ca80 msgr2=0x7f8b781516f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:48.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.708+0000 7f8b75d9b640 1 --2- 172.21.15.5:0/2107986239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b7806ca80 0x7f8b781516f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:48.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.708+0000 7f8b75d9b640 1 -- 172.21.15.5:0/2107986239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b780cf3d0 con 0x7f8b780ae320 2024-05-20T18:35:48.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.708+0000 7f8b76d9d640 1 --2- 172.21.15.5:0/2107986239 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b780ae730 0x7f8b780cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:48.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.708+0000 7f8b7659c640 1 --2- 172.21.15.5:0/2107986239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b7806ca80 0x7f8b781516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:48.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.708+0000 7f8b75d9b640 1 --2- 172.21.15.5:0/2107986239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b780ae320 0x7f8b780ce750 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c00e8a0 tx=0x7f8b6c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:48.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.708+0000 7f8b66ffd640 1 -- 172.21.15.5:0/2107986239 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c017070 con 0x7f8b780ae320 2024-05-20T18:35:48.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.708+0000 7f8b66ffd640 1 -- 172.21.15.5:0/2107986239 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b6c00c940 con 0x7f8b780ae320 2024-05-20T18:35:48.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.708+0000 7f8b7cd02640 1 -- 172.21.15.5:0/2107986239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b78160b20 con 0x7f8b780ae320 2024-05-20T18:35:48.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.708+0000 7f8b66ffd640 1 -- 172.21.15.5:0/2107986239 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c00cd30 con 0x7f8b780ae320 2024-05-20T18:35:48.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.708+0000 7f8b7cd02640 1 -- 172.21.15.5:0/2107986239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b781610e0 con 0x7f8b780ae320 2024-05-20T18:35:48.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.709+0000 7f8b7cd02640 1 -- 172.21.15.5:0/2107986239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b78075b00 con 0x7f8b780ae320 2024-05-20T18:35:48.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.710+0000 7f8b66ffd640 1 -- 172.21.15.5:0/2107986239 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8b6c01a070 con 0x7f8b780ae320 2024-05-20T18:35:48.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.710+0000 7f8b66ffd640 1 --2- 172.21.15.5:0/2107986239 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8b5806be60 0x7f8b5806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:48.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.710+0000 7f8b7659c640 1 --2- 172.21.15.5:0/2107986239 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8b5806be60 0x7f8b5806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:48.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.710+0000 7f8b66ffd640 1 -- 172.21.15.5:0/2107986239 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7f8b6c08af00 con 0x7f8b780ae320 2024-05-20T18:35:48.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.711+0000 7f8b7659c640 1 --2- 172.21.15.5:0/2107986239 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8b5806be60 0x7f8b5806e370 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8b68034000 tx=0x7f8b68033040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:48.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.715+0000 7f8b66ffd640 1 -- 172.21.15.5:0/2107986239 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8b6c059690 con 0x7f8b780ae320 2024-05-20T18:35:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.898+0000 7f8b7cd02640 1 -- 172.21.15.5:0/2107986239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8b78162920 con 0x7f8b780ae320 2024-05-20T18:35:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.899+0000 7f8b66ffd640 1 -- 172.21.15.5:0/2107986239 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) ==== 104+0+11 (secure 0 0 0) 0x7f8b6c058f90 con 0x7f8b780ae320 2024-05-20T18:35:48.899 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:35:48.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.901+0000 7f8b7cd02640 1 -- 172.21.15.5:0/2107986239 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8b5806be60 msgr2=0x7f8b5806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:48.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.901+0000 7f8b7cd02640 1 --2- 172.21.15.5:0/2107986239 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8b5806be60 0x7f8b5806e370 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8b68034000 tx=0x7f8b68033040 comp rx=0 tx=0).stop 2024-05-20T18:35:48.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.901+0000 7f8b7cd02640 1 -- 172.21.15.5:0/2107986239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b780ae320 msgr2=0x7f8b780ce750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:48.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.901+0000 7f8b7cd02640 1 --2- 172.21.15.5:0/2107986239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b780ae320 0x7f8b780ce750 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c00e8a0 tx=0x7f8b6c00ed60 comp rx=0 tx=0).stop 2024-05-20T18:35:48.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.901+0000 7f8b7cd02640 1 -- 172.21.15.5:0/2107986239 shutdown_connections 2024-05-20T18:35:48.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.901+0000 7f8b7cd02640 1 --2- 172.21.15.5:0/2107986239 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8b5806be60 0x7f8b5806e370 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:48.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.901+0000 7f8b7cd02640 1 --2- 172.21.15.5:0/2107986239 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b780ae730 0x7f8b780cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:48.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.901+0000 7f8b7cd02640 1 --2- 172.21.15.5:0/2107986239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b780ae320 0x7f8b780ce750 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:48.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.901+0000 7f8b7cd02640 1 --2- 172.21.15.5:0/2107986239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b7806ca80 0x7f8b781516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:48.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.901+0000 7f8b7cd02640 1 -- 172.21.15.5:0/2107986239 >> 172.21.15.5:0/2107986239 conn(0x7f8b78072ed0 msgr2=0x7f8b78158880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:48.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.901+0000 7f8b7cd02640 1 -- 172.21.15.5:0/2107986239 shutdown_connections 2024-05-20T18:35:48.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:48.901+0000 7f8b7cd02640 1 -- 172.21.15.5:0/2107986239 wait complete. 2024-05-20T18:35:48.912 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:35:49.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.030+0000 7f1abcf02640 1 Processor -- start 2024-05-20T18:35:49.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.030+0000 7f1abcf02640 1 -- start start 2024-05-20T18:35:49.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.032+0000 7f1abcf02640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1ab8162820 0x7f1ab80b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:49.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.032+0000 7f1abcf02640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1ab806dae0 0x7f1ab80b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:49.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.032+0000 7f1abcf02640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ab80d0630 con 0x7f1ab80b3e80 2024-05-20T18:35:49.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.032+0000 7f1abcf02640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f1ab80d39c0 con 0x7f1ab806dae0 2024-05-20T18:35:49.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.032+0000 7f1abcf02640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1ab80d3ca0 con 0x7f1ab8162820 2024-05-20T18:35:49.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.032+0000 7f1ab6d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1ab806dae0 0x7f1ab80b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:49.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.032+0000 7f1ab5d9b640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1ab80b3e80 0x7f1ab806d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33610/0 (socket says 172.21.15.5:33610) 2024-05-20T18:35:49.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.032+0000 7f1ab6d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1ab806dae0 0x7f1ab80b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37932/0 (socket says 172.21.15.5:37932) 2024-05-20T18:35:49.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.032+0000 7f1ab5d9b640 1 -- 172.21.15.5:0/4170553245 learned_addr learned my addr 172.21.15.5:0/4170553245 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:49.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.032+0000 7f1ab659c640 1 --2- 172.21.15.5:0/4170553245 >> v2:172.21.15.63:3300/0 conn(0x7f1ab8162820 0x7f1ab80b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:49.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.032+0000 7f1ab6d9d640 1 -- 172.21.15.5:0/4170553245 >> v2:172.21.15.63:3300/0 conn(0x7f1ab8162820 msgr2=0x7f1ab80b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:49.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.032+0000 7f1ab6d9d640 1 --2- 172.21.15.5:0/4170553245 >> v2:172.21.15.63:3300/0 conn(0x7f1ab8162820 0x7f1ab80b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:49.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.032+0000 7f1ab6d9d640 1 -- 172.21.15.5:0/4170553245 >> v1:172.21.15.5:6789/0 conn(0x7f1ab80b3e80 legacy=0x7f1ab806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:49.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.032+0000 7f1ab6d9d640 1 -- 172.21.15.5:0/4170553245 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab80d3f80 con 0x7f1ab806dae0 2024-05-20T18:35:49.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.032+0000 7f1ab559a640 1 -- 172.21.15.5:0/4170553245 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1784166758 0 0) 0x7f1ab80d0630 con 0x7f1ab80b3e80 2024-05-20T18:35:49.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.032+0000 7f1ab659c640 1 --2- 172.21.15.5:0/4170553245 >> v2:172.21.15.63:3300/0 conn(0x7f1ab8162820 0x7f1ab80b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:49.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.033+0000 7f1ab6d9d640 1 --2- 172.21.15.5:0/4170553245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab806dae0 0x7f1ab80b8190 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f1ab80b7740 tx=0x7f1ab002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf314c2cde64443 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.033+0000 7f1ab559a640 1 -- 172.21.15.5:0/4170553245 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0007740 con 0x7f1ab806dae0 2024-05-20T18:35:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.033+0000 7f1ab559a640 1 -- 172.21.15.5:0/4170553245 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab00078e0 con 0x7f1ab806dae0 2024-05-20T18:35:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.033+0000 7f1ab559a640 1 -- 172.21.15.5:0/4170553245 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0007cb0 con 0x7f1ab806dae0 2024-05-20T18:35:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.034+0000 7f1abcf02640 1 -- 172.21.15.5:0/4170553245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab806dae0 msgr2=0x7f1ab80b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:49.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.034+0000 7f1abcf02640 1 --2- 172.21.15.5:0/4170553245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab806dae0 0x7f1ab80b8190 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f1ab80b7740 tx=0x7f1ab002a680 comp rx=0 tx=0).stop 2024-05-20T18:35:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.034+0000 7f1abcf02640 1 -- 172.21.15.5:0/4170553245 shutdown_connections 2024-05-20T18:35:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.034+0000 7f1abcf02640 1 --2- 172.21.15.5:0/4170553245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab806dae0 0x7f1ab80b8190 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.034+0000 7f1abcf02640 1 --2- 172.21.15.5:0/4170553245 >> v2:172.21.15.63:3300/0 conn(0x7f1ab8162820 0x7f1ab80b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.034+0000 7f1abcf02640 1 -- 172.21.15.5:0/4170553245 >> 172.21.15.5:0/4170553245 conn(0x7f1ab80d4bd0 msgr2=0x7f1ab80d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.034+0000 7f1abcf02640 1 -- 172.21.15.5:0/4170553245 shutdown_connections 2024-05-20T18:35:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.034+0000 7f1abcf02640 1 -- 172.21.15.5:0/4170553245 wait complete. 2024-05-20T18:35:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.036+0000 7f1abcf02640 1 Processor -- start 2024-05-20T18:35:49.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.036+0000 7f1abcf02640 1 -- start start 2024-05-20T18:35:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.037+0000 7f1abcf02640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ab80b3e80 0x7f1ab81f4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.037+0000 7f1abcf02640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ab8162820 0x7f1ab8158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.037+0000 7f1abcf02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab81593b0 0x7f1ab8156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.037+0000 7f1abcf02640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab80d4780 con 0x7f1ab81593b0 2024-05-20T18:35:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.037+0000 7f1abcf02640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab80d0300 con 0x7f1ab8162820 2024-05-20T18:35:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.037+0000 7f1abcf02640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1ab80d4900 con 0x7f1ab80b3e80 2024-05-20T18:35:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.037+0000 7f1ab6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab81593b0 0x7f1ab8156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.037+0000 7f1ab6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab81593b0 0x7f1ab8156b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37934/0 (socket says 172.21.15.5:37934) 2024-05-20T18:35:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.037+0000 7f1ab6d9d640 1 -- 172.21.15.5:0/708843974 learned_addr learned my addr 172.21.15.5:0/708843974 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.038+0000 7f1ab659c640 1 --2- 172.21.15.5:0/708843974 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ab80b3e80 0x7f1ab81f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.038+0000 7f1ab5d9b640 1 --2- 172.21.15.5:0/708843974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ab8162820 0x7f1ab8158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:49.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.038+0000 7f1ab6d9d640 1 -- 172.21.15.5:0/708843974 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ab80b3e80 msgr2=0x7f1ab81f4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.038+0000 7f1ab6d9d640 1 --2- 172.21.15.5:0/708843974 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ab80b3e80 0x7f1ab81f4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.038+0000 7f1ab6d9d640 1 -- 172.21.15.5:0/708843974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ab8162820 msgr2=0x7f1ab8158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.038+0000 7f1ab6d9d640 1 --2- 172.21.15.5:0/708843974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ab8162820 0x7f1ab8158e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.038+0000 7f1ab6d9d640 1 -- 172.21.15.5:0/708843974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab8159ec0 con 0x7f1ab81593b0 2024-05-20T18:35:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.038+0000 7f1ab5d9b640 1 --2- 172.21.15.5:0/708843974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ab8162820 0x7f1ab8158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.038+0000 7f1ab659c640 1 --2- 172.21.15.5:0/708843974 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ab80b3e80 0x7f1ab81f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.038+0000 7f1ab6d9d640 1 --2- 172.21.15.5:0/708843974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab81593b0 0x7f1ab8156b60 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f1ab002ac70 tx=0x7f1ab0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.038+0000 7f1aa6ffd640 1 -- 172.21.15.5:0/708843974 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab003f070 con 0x7f1ab81593b0 2024-05-20T18:35:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.038+0000 7f1aa6ffd640 1 -- 172.21.15.5:0/708843974 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab0031c90 con 0x7f1ab81593b0 2024-05-20T18:35:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.038+0000 7f1abcf02640 1 -- 172.21.15.5:0/708843974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab8159be0 con 0x7f1ab81593b0 2024-05-20T18:35:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.038+0000 7f1aa6ffd640 1 -- 172.21.15.5:0/708843974 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0044680 con 0x7f1ab81593b0 2024-05-20T18:35:49.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.038+0000 7f1abcf02640 1 -- 172.21.15.5:0/708843974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab80b85a0 con 0x7f1ab81593b0 2024-05-20T18:35:49.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.039+0000 7f1abcf02640 1 -- 172.21.15.5:0/708843974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab815d660 con 0x7f1ab81593b0 2024-05-20T18:35:49.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.040+0000 7f1aa6ffd640 1 -- 172.21.15.5:0/708843974 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1ab003b410 con 0x7f1ab81593b0 2024-05-20T18:35:49.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.040+0000 7f1aa6ffd640 1 --2- 172.21.15.5:0/708843974 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1a9806be30 0x7f1a9806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:49.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.040+0000 7f1aa6ffd640 1 -- 172.21.15.5:0/708843974 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7f1ab00ac9f0 con 0x7f1ab81593b0 2024-05-20T18:35:49.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.040+0000 7f1ab659c640 1 --2- 172.21.15.5:0/708843974 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1a9806be30 0x7f1a9806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:49.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.041+0000 7f1ab659c640 1 --2- 172.21.15.5:0/708843974 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1a9806be30 0x7f1a9806e340 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800a990 tx=0x7f1aa8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:49.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.045+0000 7f1aa6ffd640 1 -- 172.21.15.5:0/708843974 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1ab007ac80 con 0x7f1ab81593b0 2024-05-20T18:35:49.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.228+0000 7f1abcf02640 1 -- 172.21.15.5:0/708843974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1ab8075b00 con 0x7f1ab81593b0 2024-05-20T18:35:49.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.229+0000 7f1aa6ffd640 1 -- 172.21.15.5:0/708843974 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) ==== 100+0+11 (secure 0 0 0) 0x7f1ab00ac070 con 0x7f1ab81593b0 2024-05-20T18:35:49.229 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:35:49.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.231+0000 7f1abcf02640 1 -- 172.21.15.5:0/708843974 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1a9806be30 msgr2=0x7f1a9806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:49.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.231+0000 7f1abcf02640 1 --2- 172.21.15.5:0/708843974 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1a9806be30 0x7f1a9806e340 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800a990 tx=0x7f1aa8008040 comp rx=0 tx=0).stop 2024-05-20T18:35:49.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.231+0000 7f1abcf02640 1 -- 172.21.15.5:0/708843974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab81593b0 msgr2=0x7f1ab8156b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:49.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.231+0000 7f1abcf02640 1 --2- 172.21.15.5:0/708843974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab81593b0 0x7f1ab8156b60 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f1ab002ac70 tx=0x7f1ab0004a40 comp rx=0 tx=0).stop 2024-05-20T18:35:49.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.231+0000 7f1abcf02640 1 -- 172.21.15.5:0/708843974 shutdown_connections 2024-05-20T18:35:49.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.231+0000 7f1abcf02640 1 --2- 172.21.15.5:0/708843974 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1a9806be30 0x7f1a9806e340 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:49.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.231+0000 7f1abcf02640 1 --2- 172.21.15.5:0/708843974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab81593b0 0x7f1ab8156b60 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:49.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.231+0000 7f1abcf02640 1 --2- 172.21.15.5:0/708843974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ab8162820 0x7f1ab8158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:49.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.231+0000 7f1abcf02640 1 --2- 172.21.15.5:0/708843974 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ab80b3e80 0x7f1ab81f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:49.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.231+0000 7f1abcf02640 1 -- 172.21.15.5:0/708843974 >> 172.21.15.5:0/708843974 conn(0x7f1ab80d4bd0 msgr2=0x7f1ab8156230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:49.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.231+0000 7f1abcf02640 1 -- 172.21.15.5:0/708843974 shutdown_connections 2024-05-20T18:35:49.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.231+0000 7f1abcf02640 1 -- 172.21.15.5:0/708843974 wait complete. 2024-05-20T18:35:49.243 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T18:35:49.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.358+0000 7f47dec98640 1 Processor -- start 2024-05-20T18:35:49.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.358+0000 7f47dec98640 1 -- start start 2024-05-20T18:35:49.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.359+0000 7f47dec98640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f47d8163a10 0x7f47d80d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:49.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.359+0000 7f47dec98640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47d80b4010 0x7f47d80cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:49.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.359+0000 7f47dec98640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47d80d0370 con 0x7f47d80d14b0 2024-05-20T18:35:49.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.359+0000 7f47dec98640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f47d80d3ca0 con 0x7f47d8163a10 2024-05-20T18:35:49.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.359+0000 7f47dec98640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f47d80d0650 con 0x7f47d80b4010 2024-05-20T18:35:49.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.360+0000 7f47d7fff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47d80b4010 0x7f47d80cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:49.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.360+0000 7f47dca34640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f47d8163a10 0x7f47d80d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:49.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.360+0000 7f47dca34640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f47d8163a10 0x7f47d80d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:52000/0 (socket says 172.21.15.5:52000) 2024-05-20T18:35:49.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.360+0000 7f47dd235640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f47d80d14b0 0x7f47d80d18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:48190/0 (socket says 172.21.15.5:48190) 2024-05-20T18:35:49.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.360+0000 7f47d7fff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47d80b4010 0x7f47d80cfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:47568/0 (socket says 172.21.15.5:47568) 2024-05-20T18:35:49.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.360+0000 7f47dca34640 1 -- 172.21.15.5:0/1225593866 learned_addr learned my addr 172.21.15.5:0/1225593866 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:49.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.360+0000 7f47d7fff640 1 -- 172.21.15.5:0/1225593866 >> v2:172.21.15.63:3300/0 conn(0x7f47d8163a10 msgr2=0x7f47d80d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:49.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.360+0000 7f47d7fff640 1 --2- 172.21.15.5:0/1225593866 >> v2:172.21.15.63:3300/0 conn(0x7f47d8163a10 0x7f47d80d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:49.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.360+0000 7f47d7fff640 1 -- 172.21.15.5:0/1225593866 >> v1:172.21.15.63:6789/0 conn(0x7f47d80d14b0 legacy=0x7f47d80d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:49.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.360+0000 7f47d7fff640 1 -- 172.21.15.5:0/1225593866 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d80d3650 con 0x7f47d80b4010 2024-05-20T18:35:49.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.360+0000 7f47dca34640 1 --2- 172.21.15.5:0/1225593866 >> v2:172.21.15.63:3300/0 conn(0x7f47d8163a10 0x7f47d80d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:35:49.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.361+0000 7f47d7fff640 1 --2- 172.21.15.5:0/1225593866 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47d80b4010 0x7f47d80cfb00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009d90 tx=0x7f47c802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35b3ed531874cd60 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:49.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.361+0000 7f47d77fe640 1 -- 172.21.15.5:0/1225593866 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c8007740 con 0x7f47d80b4010 2024-05-20T18:35:49.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.361+0000 7f47d77fe640 1 -- 172.21.15.5:0/1225593866 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47c80078e0 con 0x7f47d80b4010 2024-05-20T18:35:49.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.361+0000 7f47d77fe640 1 -- 172.21.15.5:0/1225593866 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c8007cb0 con 0x7f47d80b4010 2024-05-20T18:35:49.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.362+0000 7f47dec98640 1 -- 172.21.15.5:0/1225593866 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47d80b4010 msgr2=0x7f47d80cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:49.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.362+0000 7f47dec98640 1 --2- 172.21.15.5:0/1225593866 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47d80b4010 0x7f47d80cfb00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009d90 tx=0x7f47c802a660 comp rx=0 tx=0).stop 2024-05-20T18:35:49.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.362+0000 7f47dec98640 1 -- 172.21.15.5:0/1225593866 shutdown_connections 2024-05-20T18:35:49.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.362+0000 7f47dec98640 1 --2- 172.21.15.5:0/1225593866 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47d80b4010 0x7f47d80cfb00 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:49.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.362+0000 7f47dec98640 1 --2- 172.21.15.5:0/1225593866 >> v2:172.21.15.63:3300/0 conn(0x7f47d8163a10 0x7f47d80d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:49.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.362+0000 7f47dec98640 1 -- 172.21.15.5:0/1225593866 >> 172.21.15.5:0/1225593866 conn(0x7f47d806d240 msgr2=0x7f47d806d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:49.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.362+0000 7f47dec98640 1 -- 172.21.15.5:0/1225593866 shutdown_connections 2024-05-20T18:35:49.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.362+0000 7f47dec98640 1 -- 172.21.15.5:0/1225593866 wait complete. 2024-05-20T18:35:49.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.364+0000 7f47dec98640 1 Processor -- start 2024-05-20T18:35:49.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.364+0000 7f47dec98640 1 -- start start 2024-05-20T18:35:49.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.365+0000 7f47dec98640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47d80b4010 0x7f47d81f8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:49.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.365+0000 7f47dec98640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47d80d14b0 0x7f47d81f92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:49.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.365+0000 7f47dec98640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47d8163a10 0x7f47d81e8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:49.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.365+0000 7f47dec98640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f47d80b8210 con 0x7f47d80d14b0 2024-05-20T18:35:49.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.365+0000 7f47dec98640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f47d80d0040 con 0x7f47d80b4010 2024-05-20T18:35:49.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.365+0000 7f47dec98640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f47d80b8390 con 0x7f47d8163a10 2024-05-20T18:35:49.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.365+0000 7f47d7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47d80d14b0 0x7f47d81f92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:49.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.365+0000 7f47d7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47d80d14b0 0x7f47d81f92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37936/0 (socket says 172.21.15.5:37936) 2024-05-20T18:35:49.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.365+0000 7f47d7fff640 1 -- 172.21.15.5:0/3099104869 learned_addr learned my addr 172.21.15.5:0/3099104869 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:49.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.366+0000 7f47dd235640 1 --2- 172.21.15.5:0/3099104869 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47d8163a10 0x7f47d81e8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:49.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.366+0000 7f47dca34640 1 --2- 172.21.15.5:0/3099104869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47d80b4010 0x7f47d81f8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:49.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.366+0000 7f47d7fff640 1 -- 172.21.15.5:0/3099104869 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47d8163a10 msgr2=0x7f47d81e8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:49.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.366+0000 7f47d7fff640 1 --2- 172.21.15.5:0/3099104869 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47d8163a10 0x7f47d81e8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:49.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.366+0000 7f47d7fff640 1 -- 172.21.15.5:0/3099104869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47d80b4010 msgr2=0x7f47d81f8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:49.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.366+0000 7f47d7fff640 1 --2- 172.21.15.5:0/3099104869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47d80b4010 0x7f47d81f8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:49.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.366+0000 7f47d7fff640 1 -- 172.21.15.5:0/3099104869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d81e9860 con 0x7f47d80d14b0 2024-05-20T18:35:49.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.366+0000 7f47d7fff640 1 --2- 172.21.15.5:0/3099104869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47d80d14b0 0x7f47d81f92c0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009eb0 tx=0x7f47c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:49.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.366+0000 7f47d57fa640 1 -- 172.21.15.5:0/3099104869 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c803f070 con 0x7f47d80d14b0 2024-05-20T18:35:49.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.366+0000 7f47d57fa640 1 -- 172.21.15.5:0/3099104869 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47c8031c40 con 0x7f47d80d14b0 2024-05-20T18:35:49.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.366+0000 7f47dec98640 1 -- 172.21.15.5:0/3099104869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47d81e9b40 con 0x7f47d80d14b0 2024-05-20T18:35:49.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.366+0000 7f47d57fa640 1 -- 172.21.15.5:0/3099104869 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c8044460 con 0x7f47d80d14b0 2024-05-20T18:35:49.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.366+0000 7f47dec98640 1 -- 172.21.15.5:0/3099104869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47d81ea100 con 0x7f47d80d14b0 2024-05-20T18:35:49.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.367+0000 7f47dec98640 1 -- 172.21.15.5:0/3099104869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47d8075b00 con 0x7f47d80d14b0 2024-05-20T18:35:49.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.368+0000 7f47d57fa640 1 -- 172.21.15.5:0/3099104869 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f47c803b410 con 0x7f47d80d14b0 2024-05-20T18:35:49.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.368+0000 7f47d57fa640 1 --2- 172.21.15.5:0/3099104869 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f47b806be60 0x7f47b806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:49.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.369+0000 7f47dca34640 1 --2- 172.21.15.5:0/3099104869 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f47b806be60 0x7f47b806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:49.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.369+0000 7f47d57fa640 1 -- 172.21.15.5:0/3099104869 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7f47c80ac5a0 con 0x7f47d80d14b0 2024-05-20T18:35:49.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.369+0000 7f47dca34640 1 --2- 172.21.15.5:0/3099104869 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f47b806be60 0x7f47b806e370 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f47d80d3020 tx=0x7f47d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:49.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.373+0000 7f47d57fa640 1 -- 172.21.15.5:0/3099104869 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f47c8049060 con 0x7f47d80d14b0 2024-05-20T18:35:49.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.568+0000 7f47dec98640 1 -- 172.21.15.5:0/3099104869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f47d80d3050 con 0x7f47d80d14b0 2024-05-20T18:35:49.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.568+0000 7f47d57fa640 1 -- 172.21.15.5:0/3099104869 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v105) ==== 80+0+0 (secure 0 0 0) 0x7f47c807a6d0 con 0x7f47d80d14b0 2024-05-20T18:35:49.569 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-05-20T18:35:49.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.570+0000 7f47dec98640 1 -- 172.21.15.5:0/3099104869 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f47b806be60 msgr2=0x7f47b806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:49.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.570+0000 7f47dec98640 1 --2- 172.21.15.5:0/3099104869 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f47b806be60 0x7f47b806e370 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f47d80d3020 tx=0x7f47d0008040 comp rx=0 tx=0).stop 2024-05-20T18:35:49.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.570+0000 7f47dec98640 1 -- 172.21.15.5:0/3099104869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47d80d14b0 msgr2=0x7f47d81f92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:49.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.570+0000 7f47dec98640 1 --2- 172.21.15.5:0/3099104869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47d80d14b0 0x7f47d81f92c0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009eb0 tx=0x7f47c8004a40 comp rx=0 tx=0).stop 2024-05-20T18:35:49.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.570+0000 7f47dec98640 1 -- 172.21.15.5:0/3099104869 shutdown_connections 2024-05-20T18:35:49.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.570+0000 7f47dec98640 1 --2- 172.21.15.5:0/3099104869 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f47b806be60 0x7f47b806e370 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:49.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.570+0000 7f47dec98640 1 --2- 172.21.15.5:0/3099104869 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47d8163a10 0x7f47d81e8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:49.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.570+0000 7f47dec98640 1 --2- 172.21.15.5:0/3099104869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47d80d14b0 0x7f47d81f92c0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:49.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.570+0000 7f47dec98640 1 --2- 172.21.15.5:0/3099104869 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47d80b4010 0x7f47d81f8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:49.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.570+0000 7f47dec98640 1 -- 172.21.15.5:0/3099104869 >> 172.21.15.5:0/3099104869 conn(0x7f47d806d240 msgr2=0x7f47d80ba4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:49.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.570+0000 7f47dec98640 1 -- 172.21.15.5:0/3099104869 shutdown_connections 2024-05-20T18:35:49.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.570+0000 7f47dec98640 1 -- 172.21.15.5:0/3099104869 wait complete. 2024-05-20T18:35:49.583 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-20T18:35:49.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:35:49.583 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-20T18:35:49.632 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:35:49.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:35:49.632 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-20T18:35:49.659 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:35:49.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.813+0000 7f8e8c4ca640 1 Processor -- start 2024-05-20T18:35:49.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.813+0000 7f8e8c4ca640 1 -- start start 2024-05-20T18:35:49.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.815+0000 7f8e8c4ca640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8e8415f220 0x7f8e840d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:49.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.815+0000 7f8e8c4ca640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8e840cfae0 0x7f8e840ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:49.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.815+0000 7f8e8c4ca640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e840cda20 con 0x7f8e840d0e00 2024-05-20T18:35:49.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.815+0000 7f8e8c4ca640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f8e840cdd00 con 0x7f8e8415f220 2024-05-20T18:35:49.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.815+0000 7f8e8c4ca640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8e840cdfe0 con 0x7f8e840cfae0 2024-05-20T18:35:49.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.815+0000 7f8e8a266640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8e8415f220 0x7f8e840d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:49.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.815+0000 7f8e89a65640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8e840d0e00 0x7f8e840cf3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33618/0 (socket says 172.21.15.5:33618) 2024-05-20T18:35:49.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.815+0000 7f8e89a65640 1 -- 172.21.15.5:0/1420832216 learned_addr learned my addr 172.21.15.5:0/1420832216 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:49.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.815+0000 7f8e89264640 1 -- 172.21.15.5:0/1420832216 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3568953864 0 0) 0x7f8e840cda20 con 0x7f8e840d0e00 2024-05-20T18:35:49.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.815+0000 7f8e89264640 1 -- 172.21.15.5:0/1420832216 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e70003520 con 0x7f8e840d0e00 2024-05-20T18:35:49.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.815+0000 7f8e8aa67640 1 --2- 172.21.15.5:0/1420832216 >> v2:172.21.15.63:3300/0 conn(0x7f8e840cfae0 0x7f8e840ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:49.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.815+0000 7f8e8a266640 1 -- 172.21.15.5:0/1420832216 >> v2:172.21.15.63:3300/0 conn(0x7f8e840cfae0 msgr2=0x7f8e840ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:49.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.815+0000 7f8e8a266640 1 --2- 172.21.15.5:0/1420832216 >> v2:172.21.15.63:3300/0 conn(0x7f8e840cfae0 0x7f8e840ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:49.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.815+0000 7f8e8a266640 1 -- 172.21.15.5:0/1420832216 >> v1:172.21.15.5:6789/0 conn(0x7f8e840d0e00 legacy=0x7f8e840cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:49.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.815+0000 7f8e8a266640 1 -- 172.21.15.5:0/1420832216 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e840ce2c0 con 0x7f8e8415f220 2024-05-20T18:35:49.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.815+0000 7f8e8aa67640 1 --2- 172.21.15.5:0/1420832216 >> v2:172.21.15.63:3300/0 conn(0x7f8e840cfae0 0x7f8e840ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:49.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.816+0000 7f8e8a266640 1 --2- 172.21.15.5:0/1420832216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e8415f220 0x7f8e840d08c0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f8e8406e3e0 tx=0x7f8e8002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576499e93dd85a3d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:49.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.816+0000 7f8e89264640 1 -- 172.21.15.5:0/1420832216 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e80037070 con 0x7f8e8415f220 2024-05-20T18:35:49.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.816+0000 7f8e89264640 1 -- 172.21.15.5:0/1420832216 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e80032440 con 0x7f8e8415f220 2024-05-20T18:35:49.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.816+0000 7f8e89264640 1 -- 172.21.15.5:0/1420832216 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8002ae60 con 0x7f8e8415f220 2024-05-20T18:35:49.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.817+0000 7f8e8c4ca640 1 -- 172.21.15.5:0/1420832216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e8415f220 msgr2=0x7f8e840d08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:49.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.817+0000 7f8e8c4ca640 1 --2- 172.21.15.5:0/1420832216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e8415f220 0x7f8e840d08c0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f8e8406e3e0 tx=0x7f8e8002a110 comp rx=0 tx=0).stop 2024-05-20T18:35:49.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.817+0000 7f8e8c4ca640 1 -- 172.21.15.5:0/1420832216 shutdown_connections 2024-05-20T18:35:49.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.817+0000 7f8e8c4ca640 1 --2- 172.21.15.5:0/1420832216 >> v2:172.21.15.63:3300/0 conn(0x7f8e840cfae0 0x7f8e840ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:49.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.817+0000 7f8e8c4ca640 1 --2- 172.21.15.5:0/1420832216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e8415f220 0x7f8e840d08c0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:49.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.817+0000 7f8e8c4ca640 1 -- 172.21.15.5:0/1420832216 >> 172.21.15.5:0/1420832216 conn(0x7f8e840b44d0 msgr2=0x7f8e840b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:49.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.817+0000 7f8e8c4ca640 1 -- 172.21.15.5:0/1420832216 shutdown_connections 2024-05-20T18:35:49.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.817+0000 7f8e8c4ca640 1 -- 172.21.15.5:0/1420832216 wait complete. 2024-05-20T18:35:49.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.819+0000 7f8e8c4ca640 1 Processor -- start 2024-05-20T18:35:49.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.819+0000 7f8e8c4ca640 1 -- start start 2024-05-20T18:35:49.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.820+0000 7f8e8c4ca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e840d0e00 0x7f8e8414d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:49.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.820+0000 7f8e8c4ca640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e8415f220 0x7f8e84151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:49.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.820+0000 7f8e8c4ca640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e840c9920 0x7f8e8415c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:49.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.820+0000 7f8e8c4ca640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8415f7b0 con 0x7f8e840d0e00 2024-05-20T18:35:49.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.820+0000 7f8e8c4ca640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8415f630 con 0x7f8e8415f220 2024-05-20T18:35:49.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.820+0000 7f8e8c4ca640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8e840ced20 con 0x7f8e840c9920 2024-05-20T18:35:49.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.820+0000 7f8e8a266640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e840d0e00 0x7f8e8414d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:49.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.820+0000 7f8e8a266640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e840d0e00 0x7f8e8414d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37960/0 (socket says 172.21.15.5:37960) 2024-05-20T18:35:49.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.820+0000 7f8e8a266640 1 -- 172.21.15.5:0/191012409 learned_addr learned my addr 172.21.15.5:0/191012409 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:49.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.821+0000 7f8e89a65640 1 --2- 172.21.15.5:0/191012409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e8415f220 0x7f8e84151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:49.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.821+0000 7f8e8aa67640 1 --2- 172.21.15.5:0/191012409 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e840c9920 0x7f8e8415c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:49.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.821+0000 7f8e8a266640 1 -- 172.21.15.5:0/191012409 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e840c9920 msgr2=0x7f8e8415c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:49.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.821+0000 7f8e8a266640 1 --2- 172.21.15.5:0/191012409 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e840c9920 0x7f8e8415c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:49.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.821+0000 7f8e8a266640 1 -- 172.21.15.5:0/191012409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e8415f220 msgr2=0x7f8e84151490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:49.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.821+0000 7f8e8a266640 1 --2- 172.21.15.5:0/191012409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e8415f220 0x7f8e84151490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:49.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.821+0000 7f8e8a266640 1 -- 172.21.15.5:0/191012409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e840cc810 con 0x7f8e840d0e00 2024-05-20T18:35:49.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.821+0000 7f8e89a65640 1 --2- 172.21.15.5:0/191012409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e8415f220 0x7f8e84151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:49.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.821+0000 7f8e8aa67640 1 --2- 172.21.15.5:0/191012409 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e840c9920 0x7f8e8415c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:49.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.821+0000 7f8e8a266640 1 --2- 172.21.15.5:0/191012409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e840d0e00 0x7f8e8414d0c0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009890 tx=0x7f8e800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:49.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.821+0000 7f8e7affd640 1 -- 172.21.15.5:0/191012409 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e80037040 con 0x7f8e840d0e00 2024-05-20T18:35:49.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.821+0000 7f8e7affd640 1 -- 172.21.15.5:0/191012409 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e80042070 con 0x7f8e840d0e00 2024-05-20T18:35:49.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.821+0000 7f8e8c4ca640 1 -- 172.21.15.5:0/191012409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e840cbf70 con 0x7f8e840d0e00 2024-05-20T18:35:49.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.821+0000 7f8e7affd640 1 -- 172.21.15.5:0/191012409 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e80030960 con 0x7f8e840d0e00 2024-05-20T18:35:49.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.821+0000 7f8e8c4ca640 1 -- 172.21.15.5:0/191012409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e840cc530 con 0x7f8e840d0e00 2024-05-20T18:35:49.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.822+0000 7f8e8c4ca640 1 -- 172.21.15.5:0/191012409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e84075b00 con 0x7f8e840d0e00 2024-05-20T18:35:49.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.823+0000 7f8e7affd640 1 -- 172.21.15.5:0/191012409 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8e80030b00 con 0x7f8e840d0e00 2024-05-20T18:35:49.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.823+0000 7f8e7affd640 1 --2- 172.21.15.5:0/191012409 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8e6806be30 0x7f8e6806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:49.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.823+0000 7f8e7affd640 1 -- 172.21.15.5:0/191012409 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7f8e800aba10 con 0x7f8e840d0e00 2024-05-20T18:35:49.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.823+0000 7f8e89a65640 1 --2- 172.21.15.5:0/191012409 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8e6806be30 0x7f8e6806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:49.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.824+0000 7f8e89a65640 1 --2- 172.21.15.5:0/191012409 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8e6806be30 0x7f8e6806e340 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8e74007fd0 tx=0x7f8e74009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:49.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:49.828+0000 7f8e7affd640 1 -- 172.21.15.5:0/191012409 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8e80045360 con 0x7f8e840d0e00 2024-05-20T18:35:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.050+0000 7f8e8c4ca640 1 -- 172.21.15.5:0/191012409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e84162c60 con 0x7f8e840d0e00 2024-05-20T18:35:50.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.050+0000 7f8e7affd640 1 -- 172.21.15.5:0/191012409 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5177 (secure 0 0 0) 0x7f8e8007a1a0 con 0x7f8e840d0e00 2024-05-20T18:35:50.051 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:50.052 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6758,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/4163145158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":4163145158},{"type":"v1","addr":"172.21.15.63:6834","nonce":4163145158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7528,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3049443532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3049443532},{"type":"v1","addr":"172.21.15.5:6837","nonce":3049443532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3552410668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3552410668},{"type":"v1","addr":"172.21.15.5:6835","nonce":3552410668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:35:30.778930+0000","modified":"2024-05-20T18:35:33.806381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8343},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8343":{"gid":8343,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/2054897241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":2054897241},{"type":"v1","addr":"172.21.15.63:6835","nonce":2054897241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8343,"qdb_cluster":[8343]},"id":7}]} 2024-05-20T18:35:50.052 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 82 2024-05-20T18:35:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.052+0000 7f8e8c4ca640 1 -- 172.21.15.5:0/191012409 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8e6806be30 msgr2=0x7f8e6806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.052+0000 7f8e8c4ca640 1 --2- 172.21.15.5:0/191012409 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8e6806be30 0x7f8e6806e340 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8e74007fd0 tx=0x7f8e74009040 comp rx=0 tx=0).stop 2024-05-20T18:35:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.052+0000 7f8e8c4ca640 1 -- 172.21.15.5:0/191012409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e840d0e00 msgr2=0x7f8e8414d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.052+0000 7f8e8c4ca640 1 --2- 172.21.15.5:0/191012409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e840d0e00 0x7f8e8414d0c0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009890 tx=0x7f8e800046c0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.052+0000 7f8e8c4ca640 1 -- 172.21.15.5:0/191012409 shutdown_connections 2024-05-20T18:35:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.052+0000 7f8e8c4ca640 1 --2- 172.21.15.5:0/191012409 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8e6806be30 0x7f8e6806e340 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.052+0000 7f8e8c4ca640 1 --2- 172.21.15.5:0/191012409 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e840c9920 0x7f8e8415c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.052+0000 7f8e8c4ca640 1 --2- 172.21.15.5:0/191012409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e8415f220 0x7f8e84151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.052+0000 7f8e8c4ca640 1 --2- 172.21.15.5:0/191012409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e840d0e00 0x7f8e8414d0c0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.052+0000 7f8e8c4ca640 1 -- 172.21.15.5:0/191012409 >> 172.21.15.5:0/191012409 conn(0x7f8e840b44d0 msgr2=0x7f8e8406cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.053+0000 7f8e8c4ca640 1 -- 172.21.15.5:0/191012409 shutdown_connections 2024-05-20T18:35:50.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.053+0000 7f8e8c4ca640 1 -- 172.21.15.5:0/191012409 wait complete. 2024-05-20T18:35:50.065 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-05-20T18:35:50.217 INFO:teuthology.orchestra.run.smithi063.stdout:{"message":"","return_code":0} 2024-05-20T18:35:50.217 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-20T18:35:50.218 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:35:50.218 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:35:50.235 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15663 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T18:35:50.235 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:35:50.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:35:50.235 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-20T18:35:50.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T18:35:50.292 INFO:teuthology.orchestra.run.smithi005.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-05-20T18:35:50.292 DEBUG:teuthology.orchestra.run.smithi005:> sudo daemon-helper kill ceph --cluster ceph -w 2024-05-20T18:35:50.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:35:50.334 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-05-20T18:35:50.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.453+0000 7f0b9907a640 1 Processor -- start 2024-05-20T18:35:50.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.453+0000 7f0b9907a640 1 -- start start 2024-05-20T18:35:50.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.455+0000 7f0b9907a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0b9415d2e0 0x7f0b940cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.455+0000 7f0b9907a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b940cd780 0x7f0b940b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.455+0000 7f0b9907a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0b940b3920 0x7f0b940d1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.455+0000 7f0b9907a640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f0b940b18a0 con 0x7f0b940b3920 2024-05-20T18:35:50.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.455+0000 7f0b9907a640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0b940b3d30 con 0x7f0b9415d2e0 2024-05-20T18:35:50.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.455+0000 7f0b9907a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0b940cdc90 con 0x7f0b940cd780 2024-05-20T18:35:50.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.455+0000 7f0b9359e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0b940b3920 0x7f0b940d1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.455+0000 7f0b9359e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0b940b3920 0x7f0b940d1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37968/0 (socket says 172.21.15.5:37968) 2024-05-20T18:35:50.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.455+0000 7f0b9359e640 1 -- 172.21.15.5:0/451827199 learned_addr learned my addr 172.21.15.5:0/451827199 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:50.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.455+0000 7f0b92d9d640 1 --2- 172.21.15.5:0/451827199 >> v2:172.21.15.63:3300/0 conn(0x7f0b9415d2e0 0x7f0b940cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.455+0000 7f0b9259c640 1 --2- 172.21.15.5:0/451827199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b940cd780 0x7f0b940b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.455+0000 7f0b9359e640 1 -- 172.21.15.5:0/451827199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b940cd780 msgr2=0x7f0b940b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:50.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.455+0000 7f0b9359e640 1 --2- 172.21.15.5:0/451827199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b940cd780 0x7f0b940b33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.455+0000 7f0b9359e640 1 -- 172.21.15.5:0/451827199 >> v2:172.21.15.63:3300/0 conn(0x7f0b9415d2e0 msgr2=0x7f0b940cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:50.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.455+0000 7f0b9359e640 1 --2- 172.21.15.5:0/451827199 >> v2:172.21.15.63:3300/0 conn(0x7f0b9415d2e0 0x7f0b940cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.455+0000 7f0b9359e640 1 -- 172.21.15.5:0/451827199 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b940cdf70 con 0x7f0b940b3920 2024-05-20T18:35:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.456+0000 7f0b9359e640 1 --2- 172.21.15.5:0/451827199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b940b3920 0x7f0b940d1820 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f0b940b4030 tx=0x7f0b8c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f532d298e278f8e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.456+0000 7f0b91d9b640 1 -- 172.21.15.5:0/451827199 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c036070 con 0x7f0b940b3920 2024-05-20T18:35:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.456+0000 7f0b91d9b640 1 -- 172.21.15.5:0/451827199 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b8c031600 con 0x7f0b940b3920 2024-05-20T18:35:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.456+0000 7f0b91d9b640 1 -- 172.21.15.5:0/451827199 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c0319d0 con 0x7f0b940b3920 2024-05-20T18:35:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.457+0000 7f0b9907a640 1 -- 172.21.15.5:0/451827199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b940b3920 msgr2=0x7f0b940d1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.457+0000 7f0b9907a640 1 --2- 172.21.15.5:0/451827199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b940b3920 0x7f0b940d1820 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f0b940b4030 tx=0x7f0b8c02a6a0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.457+0000 7f0b9907a640 1 -- 172.21.15.5:0/451827199 shutdown_connections 2024-05-20T18:35:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.457+0000 7f0b9907a640 1 --2- 172.21.15.5:0/451827199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b940b3920 0x7f0b940d1820 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.457+0000 7f0b9907a640 1 --2- 172.21.15.5:0/451827199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b940cd780 0x7f0b940b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.457+0000 7f0b9907a640 1 --2- 172.21.15.5:0/451827199 >> v2:172.21.15.63:3300/0 conn(0x7f0b9415d2e0 0x7f0b940cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.457+0000 7f0b9907a640 1 -- 172.21.15.5:0/451827199 >> 172.21.15.5:0/451827199 conn(0x7f0b940b6630 msgr2=0x7f0b940b6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.457+0000 7f0b9907a640 1 -- 172.21.15.5:0/451827199 shutdown_connections 2024-05-20T18:35:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.457+0000 7f0b9907a640 1 -- 172.21.15.5:0/451827199 wait complete. 2024-05-20T18:35:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.459+0000 7f0b9907a640 1 Processor -- start 2024-05-20T18:35:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.459+0000 7f0b9907a640 1 -- start start 2024-05-20T18:35:50.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.461+0000 7f0b9907a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b940b3920 0x7f0b9409cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.461+0000 7f0b9907a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b940cd780 0x7f0b940cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.461+0000 7f0b9907a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b9415d2e0 0x7f0b94151580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.461+0000 7f0b9907a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0b940cf020 con 0x7f0b9415d2e0 2024-05-20T18:35:50.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.461+0000 7f0b9907a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0b940b1530 con 0x7f0b940b3920 2024-05-20T18:35:50.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.461+0000 7f0b9907a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0b940cf1a0 con 0x7f0b940cd780 2024-05-20T18:35:50.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.461+0000 7f0b9359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b9415d2e0 0x7f0b94151580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.461+0000 7f0b9359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b9415d2e0 0x7f0b94151580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37978/0 (socket says 172.21.15.5:37978) 2024-05-20T18:35:50.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.461+0000 7f0b9359e640 1 -- 172.21.15.5:0/3122452417 learned_addr learned my addr 172.21.15.5:0/3122452417 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:50.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.461+0000 7f0b92d9d640 1 --2- 172.21.15.5:0/3122452417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b940b3920 0x7f0b9409cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.461+0000 7f0b9259c640 1 --2- 172.21.15.5:0/3122452417 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b940cd780 0x7f0b940cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.461+0000 7f0b9359e640 1 -- 172.21.15.5:0/3122452417 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b940cd780 msgr2=0x7f0b940cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:50.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.461+0000 7f0b9359e640 1 --2- 172.21.15.5:0/3122452417 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b940cd780 0x7f0b940cb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.461+0000 7f0b9359e640 1 -- 172.21.15.5:0/3122452417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b940b3920 msgr2=0x7f0b9409cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:50.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.461+0000 7f0b9359e640 1 --2- 172.21.15.5:0/3122452417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b940b3920 0x7f0b9409cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.461+0000 7f0b9359e640 1 -- 172.21.15.5:0/3122452417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b9409d270 con 0x7f0b9415d2e0 2024-05-20T18:35:50.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.461+0000 7f0b92d9d640 1 --2- 172.21.15.5:0/3122452417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b940b3920 0x7f0b9409cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:35:50.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.461+0000 7f0b9259c640 1 --2- 172.21.15.5:0/3122452417 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b940cd780 0x7f0b940cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:35:50.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.461+0000 7f0b9359e640 1 --2- 172.21.15.5:0/3122452417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b9415d2e0 0x7f0b94151580 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c034040 tx=0x7f0b8c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:50.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.462+0000 7f0b8b7fe640 1 -- 172.21.15.5:0/3122452417 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c036050 con 0x7f0b9415d2e0 2024-05-20T18:35:50.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.462+0000 7f0b8b7fe640 1 -- 172.21.15.5:0/3122452417 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b8c042070 con 0x7f0b9415d2e0 2024-05-20T18:35:50.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.462+0000 7f0b9907a640 1 -- 172.21.15.5:0/3122452417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b94159530 con 0x7f0b9415d2e0 2024-05-20T18:35:50.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.462+0000 7f0b8b7fe640 1 -- 172.21.15.5:0/3122452417 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c009510 con 0x7f0b9415d2e0 2024-05-20T18:35:50.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.462+0000 7f0b9907a640 1 -- 172.21.15.5:0/3122452417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b9409bf70 con 0x7f0b9415d2e0 2024-05-20T18:35:50.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.464+0000 7f0b8b7fe640 1 -- 172.21.15.5:0/3122452417 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0b8c007bf0 con 0x7f0b9415d2e0 2024-05-20T18:35:50.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.464+0000 7f0b8b7fe640 1 --2- 172.21.15.5:0/3122452417 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0b7406be30 0x7f0b7406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.464+0000 7f0b8b7fe640 1 -- 172.21.15.5:0/3122452417 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7f0b8c0abce0 con 0x7f0b9415d2e0 2024-05-20T18:35:50.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.464+0000 7f0b92d9d640 1 --2- 172.21.15.5:0/3122452417 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0b7406be30 0x7f0b7406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.465+0000 7f0b92d9d640 1 --2- 172.21.15.5:0/3122452417 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0b7406be30 0x7f0b7406e340 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0b84009e10 tx=0x7f0b84009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:50.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.465+0000 7f0ba0002740 1 Processor -- start 2024-05-20T18:35:50.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.465+0000 7f0ba0002740 1 -- start start 2024-05-20T18:35:50.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.467+0000 7f0ba0002740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d42a55410 0x556d42e20430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.467+0000 7f0ba0002740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556d42a2c550 0x556d42e3b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.467+0000 7f0ba0002740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d42a34730 0x556d42e3b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.467+0000 7f0ba0002740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556d42d59990 con 0x556d42a55410 2024-05-20T18:35:50.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.467+0000 7f0ba0002740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556d42d015b0 con 0x556d42a34730 2024-05-20T18:35:50.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.467+0000 7f0ba0002740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x556d42d55950 con 0x556d42a2c550 2024-05-20T18:35:50.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.467+0000 7f0b92d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d42a55410 0x556d42e20430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.467+0000 7f0b9359e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d42a34730 0x556d42e3b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.467+0000 7f0b92d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d42a55410 0x556d42e20430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37984/0 (socket says 172.21.15.5:37984) 2024-05-20T18:35:50.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.467+0000 7f0b9259c640 1 --2- 172.21.15.5:0/1300482657 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556d42a2c550 0x556d42e3b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.467+0000 7f0b92d9d640 1 -- 172.21.15.5:0/1300482657 learned_addr learned my addr 172.21.15.5:0/1300482657 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:50.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.467+0000 7f0b9259c640 1 -- 172.21.15.5:0/1300482657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d42a34730 msgr2=0x556d42e3b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:50.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.467+0000 7f0b9259c640 1 --2- 172.21.15.5:0/1300482657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d42a34730 0x556d42e3b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.467+0000 7f0b9259c640 1 -- 172.21.15.5:0/1300482657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d42a55410 msgr2=0x556d42e20430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:50.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.467+0000 7f0b9259c640 1 --2- 172.21.15.5:0/1300482657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d42a55410 0x556d42e20430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.467+0000 7f0b9259c640 1 -- 172.21.15.5:0/1300482657 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d42b357e0 con 0x556d42a2c550 2024-05-20T18:35:50.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.467+0000 7f0b9359e640 1 --2- 172.21.15.5:0/1300482657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d42a34730 0x556d42e3b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:35:50.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.467+0000 7f0b9259c640 1 --2- 172.21.15.5:0/1300482657 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556d42a2c550 0x556d42e3b160 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009ea0 tx=0x7f0b7c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30f37a6abe06d971 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:50.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.468+0000 7f0b88ff9640 1 -- 172.21.15.5:0/1300482657 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c019070 con 0x556d42a2c550 2024-05-20T18:35:50.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.468+0000 7f0b88ff9640 1 -- 172.21.15.5:0/1300482657 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b7c004240 con 0x556d42a2c550 2024-05-20T18:35:50.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.468+0000 7f0b88ff9640 1 -- 172.21.15.5:0/1300482657 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c003610 con 0x556d42a2c550 2024-05-20T18:35:50.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.468+0000 7f0ba0002740 1 -- 172.21.15.5:0/1300482657 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556d42a2c550 msgr2=0x556d42e3b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:50.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.468+0000 7f0ba0002740 1 --2- 172.21.15.5:0/1300482657 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556d42a2c550 0x556d42e3b160 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009ea0 tx=0x7f0b7c009f90 comp rx=0 tx=0).stop 2024-05-20T18:35:50.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.469+0000 7f0ba0002740 1 -- 172.21.15.5:0/1300482657 shutdown_connections 2024-05-20T18:35:50.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.469+0000 7f0ba0002740 1 --2- 172.21.15.5:0/1300482657 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d42a34730 0x556d42e3b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.469+0000 7f0ba0002740 1 --2- 172.21.15.5:0/1300482657 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556d42a2c550 0x556d42e3b160 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.469+0000 7f0ba0002740 1 --2- 172.21.15.5:0/1300482657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d42a55410 0x556d42e20430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.469+0000 7f0ba0002740 1 -- 172.21.15.5:0/1300482657 >> 172.21.15.5:0/1300482657 conn(0x556d42a49560 msgr2=0x556d42df5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:50.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.469+0000 7f0ba0002740 1 -- 172.21.15.5:0/1300482657 shutdown_connections 2024-05-20T18:35:50.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.469+0000 7f0ba0002740 1 -- 172.21.15.5:0/1300482657 wait complete. 2024-05-20T18:35:50.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.470+0000 7f0ba0002740 1 -- start start 2024-05-20T18:35:50.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.470+0000 7f0ba0002740 1 Processor -- start 2024-05-20T18:35:50.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.471+0000 7f0ba0002740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d42a2c550 0x556d42e332c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.471+0000 7f0ba0002740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d42a34730 0x556d42e39c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.471+0000 7f0ba0002740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556d42a55410 0x556d42e3a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.471+0000 7f0ba0002740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556d42d14b40 con 0x556d42a2c550 2024-05-20T18:35:50.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.471+0000 7f0ba0002740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x556d42e33800 con 0x556d42a34730 2024-05-20T18:35:50.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.471+0000 7f0ba0002740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x556d42e35790 con 0x556d42a55410 2024-05-20T18:35:50.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.471+0000 7f0ba0002740 10 client.0 _ll_register_callbacks cb 0x556d42d090f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:35:50.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.471+0000 7f0b92d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d42a2c550 0x556d42e332c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.471+0000 7f0b92d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d42a2c550 0x556d42e332c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37986/0 (socket says 172.21.15.5:37986) 2024-05-20T18:35:50.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.471+0000 7f0b92d9d640 1 -- 172.21.15.5:0/3259678217 learned_addr learned my addr 172.21.15.5:0/3259678217 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:50.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.471+0000 7f0b9259c640 1 --2- 172.21.15.5:0/3259678217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d42a34730 0x556d42e39c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.471+0000 7f0b9359e640 1 --2- 172.21.15.5:0/3259678217 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556d42a55410 0x556d42e3a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.471+0000 7f0b92d9d640 1 -- 172.21.15.5:0/3259678217 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556d42a55410 msgr2=0x556d42e3a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:50.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.471+0000 7f0b92d9d640 1 --2- 172.21.15.5:0/3259678217 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556d42a55410 0x556d42e3a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.472+0000 7f0b92d9d640 1 -- 172.21.15.5:0/3259678217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d42a34730 msgr2=0x556d42e39c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:50.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.472+0000 7f0b92d9d640 1 --2- 172.21.15.5:0/3259678217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d42a34730 0x556d42e39c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.472+0000 7f0b92d9d640 1 -- 172.21.15.5:0/3259678217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d42b95f70 con 0x556d42a2c550 2024-05-20T18:35:50.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.472+0000 7f0b9259c640 1 --2- 172.21.15.5:0/3259678217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d42a34730 0x556d42e39c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:50.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.472+0000 7f0b9359e640 1 --2- 172.21.15.5:0/3259678217 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556d42a55410 0x556d42e3a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:50.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.472+0000 7f0b92d9d640 1 --2- 172.21.15.5:0/3259678217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d42a2c550 0x556d42e332c0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f0b84003bc0 tx=0x7f0b840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e280cab36286af server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:50.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.472+0000 7f0b71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:35:50.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.472+0000 7f0b71ffb640 1 -- 172.21.15.5:0/3259678217 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b84027070 con 0x556d42a2c550 2024-05-20T18:35:50.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.472+0000 7f0ba0002740 1 -- 172.21.15.5:0/3259678217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b7c004240 con 0x556d42a2c550 2024-05-20T18:35:50.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.472+0000 7f0b71ffb640 1 -- 172.21.15.5:0/3259678217 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7f0b84014940 con 0x556d42a2c550 2024-05-20T18:35:50.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.472+0000 7f0b71ffb640 1 -- 172.21.15.5:0/3259678217 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f0b84032050 con 0x556d42a2c550 2024-05-20T18:35:50.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.472+0000 7f0ba0002740 10 client.8637 fetch_fsmap learned FSMap version 82 2024-05-20T18:35:50.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.472+0000 7f0ba0002740 1 -- 172.21.15.5:0/3259678217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x556d42a2d170 con 0x556d42a2c550 2024-05-20T18:35:50.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.472+0000 7f0b71ffb640 1 -- 172.21.15.5:0/3259678217 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b7c004240 con 0x556d42a2c550 2024-05-20T18:35:50.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.472+0000 7f0b71ffb640 1 -- 172.21.15.5:0/3259678217 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f0b84035020 con 0x556d42a2c550 2024-05-20T18:35:50.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.473+0000 7f0ba0002740 10 client.8637 fetch_fsmap finished waiting for FSMap version 82 2024-05-20T18:35:50.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.473+0000 7f0ba0002740 10 client.8637 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-05-20T18:35:50.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.473+0000 7f0ba0002740 20 client.8637 populate_metadata read hostname 'smithi005' 2024-05-20T18:35:50.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.473+0000 7f0ba0002740 1 --2- 172.21.15.5:0/3259678217 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x556d42a141f0 0x556d42e57050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.473+0000 7f0ba0002740 4 client.8637 mds_command: new command op to 8343 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:35:50.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.473+0000 7f0ba0002740 1 -- 172.21.15.5:0/3259678217 --> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d42d670c0 con 0x556d42a141f0 2024-05-20T18:35:50.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.473+0000 7f0b9259c640 1 --2- 172.21.15.5:0/3259678217 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x556d42a141f0 0x556d42e57050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.473+0000 7f0b71ffb640 10 client.8637 ms_handle_connect on v2:172.21.15.63:6833/2054897241 2024-05-20T18:35:50.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.473+0000 7f0b9259c640 1 --2- 172.21.15.5:0/3259678217 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x556d42a141f0 0x556d42e57050 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcd42f8aa9acae10 server_cookie=b1e8a18477e8417b in_seq=0 out_seq=0 2024-05-20T18:35:50.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.474+0000 7f0b71ffb640 1 -- 172.21.15.5:0/3259678217 <== mds.0 v2:172.21.15.63:6833/2054897241 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556d42d670c0 con 0x556d42a141f0 2024-05-20T18:35:50.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.474+0000 7fe7c4e0d640 1 Processor -- start 2024-05-20T18:35:50.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.474+0000 7f0b71ffb640 10 client.8637 handle_command_reply: tid=0 2024-05-20T18:35:50.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.474+0000 7f0ba0002740 1 client.8637 shutdown 2024-05-20T18:35:50.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.474+0000 7fe7c4e0d640 1 -- start start 2024-05-20T18:35:50.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.474+0000 7f0ba0002740 1 -- 172.21.15.5:0/3259678217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d42a2c550 msgr2=0x556d42e332c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:50.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.474+0000 7f0ba0002740 1 --2- 172.21.15.5:0/3259678217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d42a2c550 0x556d42e332c0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f0b84003bc0 tx=0x7f0b840046c0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.474+0000 7f0ba0002740 1 -- 172.21.15.5:0/3259678217 shutdown_connections 2024-05-20T18:35:50.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.474+0000 7f0ba0002740 1 --2- 172.21.15.5:0/3259678217 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x556d42a141f0 0x556d42e57050 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.474+0000 7f0ba0002740 1 --2- 172.21.15.5:0/3259678217 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556d42a55410 0x556d42e3a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.474+0000 7f0ba0002740 1 --2- 172.21.15.5:0/3259678217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d42a34730 0x556d42e39c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.474+0000 7f0ba0002740 1 --2- 172.21.15.5:0/3259678217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d42a2c550 0x556d42e332c0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.474+0000 7f0ba0002740 1 -- 172.21.15.5:0/3259678217 >> 172.21.15.5:0/3259678217 conn(0x556d42a49560 msgr2=0x556d42e20430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:50.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.474+0000 7f0b71ffb640 0 client.8637 ms_handle_reset on v2:172.21.15.63:6833/2054897241 2024-05-20T18:35:50.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.474+0000 7f0ba0002740 1 -- 172.21.15.5:0/3259678217 shutdown_connections 2024-05-20T18:35:50.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.474+0000 7f0ba0002740 1 -- 172.21.15.5:0/3259678217 wait complete. 2024-05-20T18:35:50.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.474+0000 7f0ba0002740 20 client.8637 trim_cache size 0 max 16384 2024-05-20T18:35:50.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.476+0000 7fe7c4e0d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe7c00b3730 0x7fe7c00b66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.476+0000 7fe7c4e0d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe7c00b6c20 0x7fe7c00d3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.477+0000 7fe7c4e0d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7c00b7030 con 0x7fe7c015f230 2024-05-20T18:35:50.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.477+0000 7fe7c4e0d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe7c0162d10 con 0x7fe7c00b6c20 2024-05-20T18:35:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.477+0000 7fe7c4e0d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe7c01635b0 con 0x7fe7c00b3730 2024-05-20T18:35:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.477+0000 7fe7bed9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe7c00b6c20 0x7fe7c00d3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.477+0000 7fe7be59c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe7c015f230 0x7fe7c00d0260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33630/0 (socket says 172.21.15.5:33630) 2024-05-20T18:35:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.477+0000 7fe7be59c640 1 -- 172.21.15.5:0/3243187241 learned_addr learned my addr 172.21.15.5:0/3243187241 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.477+0000 7fe7bdd9b640 1 --2- 172.21.15.5:0/3243187241 >> v2:172.21.15.63:3300/0 conn(0x7fe7c00b3730 0x7fe7c00b66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.477+0000 7fe7bed9d640 1 -- 172.21.15.5:0/3243187241 >> v2:172.21.15.63:3300/0 conn(0x7fe7c00b3730 msgr2=0x7fe7c00b66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.477+0000 7fe7bed9d640 1 --2- 172.21.15.5:0/3243187241 >> v2:172.21.15.63:3300/0 conn(0x7fe7c00b3730 0x7fe7c00b66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.477+0000 7fe7bd59a640 1 -- 172.21.15.5:0/3243187241 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3888652016 0 0) 0x7fe7b0001110 con 0x7fe7c015f230 2024-05-20T18:35:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.477+0000 7fe7bed9d640 1 -- 172.21.15.5:0/3243187241 >> v1:172.21.15.5:6789/0 conn(0x7fe7c015f230 legacy=0x7fe7c00d0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.477+0000 7fe7bed9d640 1 -- 172.21.15.5:0/3243187241 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c00b3b40 con 0x7fe7c00b6c20 2024-05-20T18:35:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.477+0000 7fe7bdd9b640 1 --2- 172.21.15.5:0/3243187241 >> v2:172.21.15.63:3300/0 conn(0x7fe7c00b3730 0x7fe7c00b66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:35:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.478+0000 7fe7bed9d640 1 --2- 172.21.15.5:0/3243187241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7c00b6c20 0x7fe7c00d3490 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fe7c008d930 tx=0x7fe7b802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=353861273c0405cf server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.478+0000 7fe7bd59a640 1 -- 172.21.15.5:0/3243187241 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8007740 con 0x7fe7c00b6c20 2024-05-20T18:35:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.478+0000 7fe7bd59a640 1 -- 172.21.15.5:0/3243187241 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7b80078e0 con 0x7fe7c00b6c20 2024-05-20T18:35:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.478+0000 7fe7bd59a640 1 -- 172.21.15.5:0/3243187241 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8007cb0 con 0x7fe7c00b6c20 2024-05-20T18:35:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.479+0000 7fe7c4e0d640 1 -- 172.21.15.5:0/3243187241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7c00b6c20 msgr2=0x7fe7c00d3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.479+0000 7fe7c4e0d640 1 --2- 172.21.15.5:0/3243187241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7c00b6c20 0x7fe7c00d3490 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fe7c008d930 tx=0x7fe7b802a5f0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.479+0000 7fe7c4e0d640 1 -- 172.21.15.5:0/3243187241 shutdown_connections 2024-05-20T18:35:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.479+0000 7fe7c4e0d640 1 --2- 172.21.15.5:0/3243187241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7c00b6c20 0x7fe7c00d3490 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.479+0000 7fe7c4e0d640 1 --2- 172.21.15.5:0/3243187241 >> v2:172.21.15.63:3300/0 conn(0x7fe7c00b3730 0x7fe7c00b66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.479+0000 7fe7c4e0d640 1 -- 172.21.15.5:0/3243187241 >> 172.21.15.5:0/3243187241 conn(0x7fe7c00d18c0 msgr2=0x7fe7c00d1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.479+0000 7fe7c4e0d640 1 -- 172.21.15.5:0/3243187241 shutdown_connections 2024-05-20T18:35:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.479+0000 7fe7c4e0d640 1 -- 172.21.15.5:0/3243187241 wait complete. 2024-05-20T18:35:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.481+0000 7fe7c4e0d640 1 Processor -- start 2024-05-20T18:35:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.481+0000 7fe7c4e0d640 1 -- start start 2024-05-20T18:35:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.482+0000 7fe7c4e0d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7c00b3730 0x7fe7c01e8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.482+0000 7fe7c4e0d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7c00b6c20 0x7fe7c015fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.482+0000 7fe7c4e0d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7c015f230 0x7fe7c0160140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.482+0000 7fe7c4e0d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c0162f60 con 0x7fe7c00b6c20 2024-05-20T18:35:50.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.482+0000 7fe7c4e0d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c00b3e10 con 0x7fe7c00b3730 2024-05-20T18:35:50.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.482+0000 7fe7c4e0d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe7c0163240 con 0x7fe7c015f230 2024-05-20T18:35:50.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.483+0000 7fe7bdd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7c00b6c20 0x7fe7c015fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.483+0000 7fe7bdd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7c00b6c20 0x7fe7c015fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38002/0 (socket says 172.21.15.5:38002) 2024-05-20T18:35:50.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.483+0000 7fe7bdd9b640 1 -- 172.21.15.5:0/1376524015 learned_addr learned my addr 172.21.15.5:0/1376524015 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:50.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.483+0000 7fe7bed9d640 1 --2- 172.21.15.5:0/1376524015 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7c015f230 0x7fe7c0160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.483+0000 7fe7be59c640 1 --2- 172.21.15.5:0/1376524015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7c00b3730 0x7fe7c01e8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.483+0000 7fe7bdd9b640 1 -- 172.21.15.5:0/1376524015 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7c015f230 msgr2=0x7fe7c0160140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:50.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.483+0000 7fe7bdd9b640 1 --2- 172.21.15.5:0/1376524015 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7c015f230 0x7fe7c0160140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.483+0000 7fe7bdd9b640 1 -- 172.21.15.5:0/1376524015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7c00b3730 msgr2=0x7fe7c01e8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:50.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.483+0000 7fe7bdd9b640 1 --2- 172.21.15.5:0/1376524015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7c00b3730 0x7fe7c01e8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.483+0000 7fe7bdd9b640 1 -- 172.21.15.5:0/1376524015 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c01e9f20 con 0x7fe7c00b6c20 2024-05-20T18:35:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.483+0000 7fe7bdd9b640 1 --2- 172.21.15.5:0/1376524015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7c00b6c20 0x7fe7c015fc00 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fe7a800e960 tx=0x7fe7a800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.483+0000 7fe7b6ffd640 1 -- 172.21.15.5:0/1376524015 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8017070 con 0x7fe7c00b6c20 2024-05-20T18:35:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.483+0000 7fe7b6ffd640 1 -- 172.21.15.5:0/1376524015 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7a800ca00 con 0x7fe7c00b6c20 2024-05-20T18:35:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.483+0000 7fe7c4e0d640 1 -- 172.21.15.5:0/1376524015 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7c01e9680 con 0x7fe7c00b6c20 2024-05-20T18:35:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.483+0000 7fe7b6ffd640 1 -- 172.21.15.5:0/1376524015 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a800cdd0 con 0x7fe7c00b6c20 2024-05-20T18:35:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.483+0000 7fe7c4e0d640 1 -- 172.21.15.5:0/1376524015 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7c01e9c40 con 0x7fe7c00b6c20 2024-05-20T18:35:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.484+0000 7fe7c4e0d640 1 -- 172.21.15.5:0/1376524015 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fe7c00690b0 con 0x7fe7c00b6c20 2024-05-20T18:35:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.485+0000 7fe7b6ffd640 1 -- 172.21.15.5:0/1376524015 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe7a801a070 con 0x7fe7c00b6c20 2024-05-20T18:35:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.485+0000 7fe7b6ffd640 1 --2- 172.21.15.5:0/1376524015 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe7a006be60 0x7fe7a006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.486+0000 7fe7b6ffd640 1 -- 172.21.15.5:0/1376524015 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7fe7a808add0 con 0x7fe7c00b6c20 2024-05-20T18:35:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.486+0000 7fe7be59c640 1 --2- 172.21.15.5:0/1376524015 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe7a006be60 0x7fe7a006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.486+0000 7fe7b6ffd640 1 -- 172.21.15.5:0/1376524015 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+512 (secure 0 0 0) 0x7fe7a808d370 con 0x7fe7c00b6c20 2024-05-20T18:35:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.486+0000 7fe7c4e0d640 1 -- 172.21.15.5:0/1376524015 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7fe7c00d3040 con 0x7fe7c00b6c20 2024-05-20T18:35:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.486+0000 7fe7be59c640 1 --2- 172.21.15.5:0/1376524015 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe7a006be60 0x7fe7a006e370 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0007fd0 tx=0x7fe7b0007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:50.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.495+0000 7f0ba0002740 1 Processor -- start 2024-05-20T18:35:50.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.495+0000 7f0ba0002740 1 -- start start 2024-05-20T18:35:50.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.496+0000 7f0ba0002740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d42a2c550 0x556d42e5cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.496+0000 7f0ba0002740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556d42a34730 0x556d42e1f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.496+0000 7f0ba0002740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d42a55410 0x556d42e1f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.496+0000 7f0ba0002740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0b94200090 con 0x556d42a2c550 2024-05-20T18:35:50.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.496+0000 7f0ba0002740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0b941ff0f0 con 0x556d42a55410 2024-05-20T18:35:50.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.496+0000 7f0ba0002740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0b940ca790 con 0x556d42a34730 2024-05-20T18:35:50.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.497+0000 7f0b92d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d42a2c550 0x556d42e5cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.497+0000 7f0b92d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d42a2c550 0x556d42e5cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38016/0 (socket says 172.21.15.5:38016) 2024-05-20T18:35:50.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.497+0000 7f0b92d9d640 1 -- 172.21.15.5:0/2355269648 learned_addr learned my addr 172.21.15.5:0/2355269648 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:50.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.497+0000 7f0b9359e640 1 --2- 172.21.15.5:0/2355269648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d42a55410 0x556d42e1f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.497+0000 7f0b9259c640 1 --2- 172.21.15.5:0/2355269648 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556d42a34730 0x556d42e1f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.497+0000 7f0b92d9d640 1 -- 172.21.15.5:0/2355269648 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556d42a34730 msgr2=0x556d42e1f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:50.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.497+0000 7f0b92d9d640 1 --2- 172.21.15.5:0/2355269648 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556d42a34730 0x556d42e1f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.497+0000 7f0b92d9d640 1 -- 172.21.15.5:0/2355269648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d42a55410 msgr2=0x556d42e1f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:50.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.497+0000 7f0b92d9d640 1 --2- 172.21.15.5:0/2355269648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d42a55410 0x556d42e1f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.497+0000 7f0b92d9d640 1 -- 172.21.15.5:0/2355269648 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d42e1ac50 con 0x556d42a2c550 2024-05-20T18:35:50.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.497+0000 7f0b9359e640 1 --2- 172.21.15.5:0/2355269648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d42a55410 0x556d42e1f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:50.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.497+0000 7f0b9259c640 1 --2- 172.21.15.5:0/2355269648 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556d42a34730 0x556d42e1f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:35:50.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.497+0000 7f0b92d9d640 1 --2- 172.21.15.5:0/2355269648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d42a2c550 0x556d42e5cca0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f0b84011c40 tx=0x7f0b84024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1755cb79a7a778 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:50.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.497+0000 7f0b73fff640 1 -- 172.21.15.5:0/2355269648 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b84027070 con 0x556d42a2c550 2024-05-20T18:35:50.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.497+0000 7f0b73fff640 1 -- 172.21.15.5:0/2355269648 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b84014a10 con 0x556d42a2c550 2024-05-20T18:35:50.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.497+0000 7f0b73fff640 1 -- 172.21.15.5:0/2355269648 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8402c640 con 0x556d42a2c550 2024-05-20T18:35:50.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.498+0000 7f0ba0002740 1 -- 172.21.15.5:0/2355269648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d42a2c550 msgr2=0x556d42e5cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:50.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.498+0000 7f0ba0002740 1 --2- 172.21.15.5:0/2355269648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d42a2c550 0x556d42e5cca0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f0b84011c40 tx=0x7f0b84024000 comp rx=0 tx=0).stop 2024-05-20T18:35:50.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.498+0000 7f0ba0002740 1 -- 172.21.15.5:0/2355269648 shutdown_connections 2024-05-20T18:35:50.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.498+0000 7f0ba0002740 1 --2- 172.21.15.5:0/2355269648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d42a55410 0x556d42e1f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.498+0000 7f0ba0002740 1 --2- 172.21.15.5:0/2355269648 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556d42a34730 0x556d42e1f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.498+0000 7f0ba0002740 1 --2- 172.21.15.5:0/2355269648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d42a2c550 0x556d42e5cca0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.498+0000 7f0ba0002740 1 -- 172.21.15.5:0/2355269648 >> 172.21.15.5:0/2355269648 conn(0x556d42a49560 msgr2=0x556d42df1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:50.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.498+0000 7f0ba0002740 1 -- 172.21.15.5:0/2355269648 shutdown_connections 2024-05-20T18:35:50.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.498+0000 7f0ba0002740 1 -- 172.21.15.5:0/2355269648 wait complete. 2024-05-20T18:35:50.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.499+0000 7f0ba0002740 1 -- start start 2024-05-20T18:35:50.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.499+0000 7f0ba0002740 1 Processor -- start 2024-05-20T18:35:50.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.500+0000 7f0ba0002740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d42a2c550 0x556d42eb73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.500+0000 7f0ba0002740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d42a34730 0x556d42df44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.500+0000 7f0ba0002740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556d42a55410 0x556d42e37650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.500+0000 7f0ba0002740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556d42e3b230 con 0x556d42a34730 2024-05-20T18:35:50.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.500+0000 7f0ba0002740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0b940afef0 con 0x556d42a2c550 2024-05-20T18:35:50.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.500+0000 7f0ba0002740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x556d42e35de0 con 0x556d42a55410 2024-05-20T18:35:50.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.500+0000 7f0ba0002740 10 client.0 _ll_register_callbacks cb 0x7f0b84011f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:35:50.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.501+0000 7f0b9259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d42a34730 0x556d42df44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.501+0000 7f0b9259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d42a34730 0x556d42df44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38032/0 (socket says 172.21.15.5:38032) 2024-05-20T18:35:50.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.501+0000 7f0b9259c640 1 -- 172.21.15.5:0/3601693594 learned_addr learned my addr 172.21.15.5:0/3601693594 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:50.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.501+0000 7f0b92d9d640 1 --2- 172.21.15.5:0/3601693594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d42a2c550 0x556d42eb73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.501+0000 7f0b9359e640 1 --2- 172.21.15.5:0/3601693594 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556d42a55410 0x556d42e37650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.501+0000 7f0b9259c640 1 -- 172.21.15.5:0/3601693594 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556d42a55410 msgr2=0x556d42e37650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:50.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.501+0000 7f0b9259c640 1 --2- 172.21.15.5:0/3601693594 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556d42a55410 0x556d42e37650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.501+0000 7f0b9259c640 1 -- 172.21.15.5:0/3601693594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d42a2c550 msgr2=0x556d42eb73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:50.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.501+0000 7f0b9259c640 1 --2- 172.21.15.5:0/3601693594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d42a2c550 0x556d42eb73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.501+0000 7f0b9259c640 1 -- 172.21.15.5:0/3601693594 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d42e49f30 con 0x556d42a34730 2024-05-20T18:35:50.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.501+0000 7f0b9359e640 1 --2- 172.21.15.5:0/3601693594 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556d42a55410 0x556d42e37650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:50.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.501+0000 7f0b9259c640 1 --2- 172.21.15.5:0/3601693594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d42a34730 0x556d42df44e0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c018000 tx=0x7f0b7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f774d1284f5c87 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:50.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.501+0000 7f0b71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:35:50.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.501+0000 7f0b71ffb640 1 -- 172.21.15.5:0/3601693594 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c019040 con 0x556d42a34730 2024-05-20T18:35:50.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.501+0000 7f0b71ffb640 1 -- 172.21.15.5:0/3601693594 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7f0b7c003810 con 0x556d42a34730 2024-05-20T18:35:50.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.501+0000 7f0ba0002740 1 -- 172.21.15.5:0/3601693594 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b84014a10 con 0x556d42a34730 2024-05-20T18:35:50.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.501+0000 7f0b71ffb640 1 -- 172.21.15.5:0/3601693594 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f0b7c03b050 con 0x556d42a34730 2024-05-20T18:35:50.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.501+0000 7f0ba0002740 10 client.8652 fetch_fsmap learned FSMap version 82 2024-05-20T18:35:50.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.501+0000 7f0ba0002740 1 -- 172.21.15.5:0/3601693594 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x556d42e37b90 con 0x556d42a34730 2024-05-20T18:35:50.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.502+0000 7f0b71ffb640 1 -- 172.21.15.5:0/3601693594 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b84014a10 con 0x556d42a34730 2024-05-20T18:35:50.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.502+0000 7f0b71ffb640 1 -- 172.21.15.5:0/3601693594 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f0b7c03e020 con 0x556d42a34730 2024-05-20T18:35:50.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.502+0000 7f0ba0002740 10 client.8652 fetch_fsmap finished waiting for FSMap version 82 2024-05-20T18:35:50.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.502+0000 7f0ba0002740 10 client.8652 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-05-20T18:35:50.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.502+0000 7f0ba0002740 20 client.8652 populate_metadata read hostname 'smithi005' 2024-05-20T18:35:50.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.502+0000 7f0ba0002740 1 --2- 172.21.15.5:0/3601693594 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x556d42df5780 0x556d42df5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.502+0000 7f0ba0002740 4 client.8652 mds_command: new command op to 8343 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-05-20T18:35:50.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.502+0000 7f0ba0002740 1 -- 172.21.15.5:0/3601693594 --> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x556d42d61410 con 0x556d42df5780 2024-05-20T18:35:50.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.502+0000 7f0b92d9d640 1 --2- 172.21.15.5:0/3601693594 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x556d42df5780 0x556d42df5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.502+0000 7f0b92d9d640 1 --2- 172.21.15.5:0/3601693594 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x556d42df5780 0x556d42df5b90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc03673b96d35ffc server_cookie=49b9e6ae70c94c2a in_seq=0 out_seq=0 2024-05-20T18:35:50.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.502+0000 7f0b71ffb640 10 client.8652 ms_handle_connect on v2:172.21.15.63:6833/2054897241 2024-05-20T18:35:50.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.503+0000 7f0b71ffb640 1 -- 172.21.15.5:0/3601693594 <== mds.0 v2:172.21.15.63:6833/2054897241 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556d42d61410 con 0x556d42df5780 2024-05-20T18:35:50.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.503+0000 7f0b71ffb640 10 client.8652 handle_command_reply: tid=0 2024-05-20T18:35:50.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.503+0000 7f0ba0002740 1 client.8652 shutdown 2024-05-20T18:35:50.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.503+0000 7f0ba0002740 1 -- 172.21.15.5:0/3601693594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d42a34730 msgr2=0x556d42df44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:50.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.503+0000 7f0ba0002740 1 --2- 172.21.15.5:0/3601693594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d42a34730 0x556d42df44e0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c018000 tx=0x7f0b7c004730 comp rx=0 tx=0).stop 2024-05-20T18:35:50.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.503+0000 7f0ba0002740 1 -- 172.21.15.5:0/3601693594 shutdown_connections 2024-05-20T18:35:50.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.503+0000 7f0ba0002740 1 --2- 172.21.15.5:0/3601693594 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x556d42df5780 0x556d42df5b90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.503+0000 7f0b71ffb640 0 client.8652 ms_handle_reset on v2:172.21.15.63:6833/2054897241 2024-05-20T18:35:50.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.503+0000 7f0ba0002740 1 --2- 172.21.15.5:0/3601693594 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556d42a55410 0x556d42e37650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.503+0000 7f0ba0002740 1 --2- 172.21.15.5:0/3601693594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d42a34730 0x556d42df44e0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.503+0000 7f0ba0002740 1 --2- 172.21.15.5:0/3601693594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556d42a2c550 0x556d42eb73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.503+0000 7f0ba0002740 1 -- 172.21.15.5:0/3601693594 >> 172.21.15.5:0/3601693594 conn(0x556d42a49560 msgr2=0x556d42e343e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:50.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.503+0000 7f0ba0002740 1 -- 172.21.15.5:0/3601693594 shutdown_connections 2024-05-20T18:35:50.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.503+0000 7f0ba0002740 1 -- 172.21.15.5:0/3601693594 wait complete. 2024-05-20T18:35:50.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.504+0000 7f0ba0002740 20 client.8652 trim_cache size 0 max 16384 2024-05-20T18:35:50.509 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:35:50.510 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-20T18:35:50.510 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "6c211f6d-2e81-47ae-9e81-478bb3882d33", 2024-05-20T18:35:50.510 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-20T18:35:50.510 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:35:50.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.504+0000 7f0b9907a640 1 -- 172.21.15.5:0/3122452417 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0b7406be30 msgr2=0x7f0b7406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:50.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.504+0000 7f0b9907a640 1 --2- 172.21.15.5:0/3122452417 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0b7406be30 0x7f0b7406e340 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0b84009e10 tx=0x7f0b84009190 comp rx=0 tx=0).stop 2024-05-20T18:35:50.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.504+0000 7f0b9907a640 1 -- 172.21.15.5:0/3122452417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b9415d2e0 msgr2=0x7f0b94151580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:50.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.504+0000 7f0b9907a640 1 --2- 172.21.15.5:0/3122452417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b9415d2e0 0x7f0b94151580 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c034040 tx=0x7f0b8c004a40 comp rx=0 tx=0).stop 2024-05-20T18:35:50.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.504+0000 7f0b9907a640 1 -- 172.21.15.5:0/3122452417 shutdown_connections 2024-05-20T18:35:50.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.504+0000 7f0b9907a640 1 --2- 172.21.15.5:0/3122452417 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0b7406be30 0x7f0b7406e340 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.504+0000 7f0b9907a640 1 --2- 172.21.15.5:0/3122452417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b9415d2e0 0x7f0b94151580 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.504+0000 7f0b9907a640 1 --2- 172.21.15.5:0/3122452417 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b940cd780 0x7f0b940cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.504+0000 7f0b9907a640 1 --2- 172.21.15.5:0/3122452417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b940b3920 0x7f0b9409cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.504+0000 7f0b9907a640 1 -- 172.21.15.5:0/3122452417 >> 172.21.15.5:0/3122452417 conn(0x7f0b940b6630 msgr2=0x7f0b9406fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:50.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.504+0000 7f0b9907a640 1 -- 172.21.15.5:0/3122452417 shutdown_connections 2024-05-20T18:35:50.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.504+0000 7f0b9907a640 1 -- 172.21.15.5:0/3122452417 wait complete. 2024-05-20T18:35:50.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:35:50.519 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-20T18:35:50.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.635+0000 7fc76c03d640 1 Processor -- start 2024-05-20T18:35:50.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.635+0000 7fc76c03d640 1 -- start start 2024-05-20T18:35:50.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.636+0000 7fc76c03d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7640cd680 0x7fc76415f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.636+0000 7fc76c03d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc76406d0f0 con 0x7fc7640cfae0 2024-05-20T18:35:50.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.636+0000 7fc76c03d640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc76406d3d0 con 0x7fc76406c6c0 2024-05-20T18:35:50.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.636+0000 7fc76c03d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc7640cda90 con 0x7fc7640cd680 2024-05-20T18:35:50.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.636+0000 7fc76a5da640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc7640cfae0 0x7fc7640d1060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33644/0 (socket says 172.21.15.5:33644) 2024-05-20T18:35:50.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.636+0000 7fc76a5da640 1 -- 172.21.15.5:0/4289521010 learned_addr learned my addr 172.21.15.5:0/4289521010 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:35:50.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.637+0000 7fc769dd9640 1 --2- 172.21.15.5:0/4289521010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7640cd680 0x7fc76415f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.637+0000 7fc768dd7640 1 -- 172.21.15.5:0/4289521010 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1650930863 0 0) 0x7fc76406d0f0 con 0x7fc7640cfae0 2024-05-20T18:35:50.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.637+0000 7fc768dd7640 1 -- 172.21.15.5:0/4289521010 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc750003520 con 0x7fc7640cfae0 2024-05-20T18:35:50.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.637+0000 7fc768dd7640 1 -- 172.21.15.5:0/4289521010 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3905017799 0 0) 0x7fc750003520 con 0x7fc7640cfae0 2024-05-20T18:35:50.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.637+0000 7fc768dd7640 1 -- 172.21.15.5:0/4289521010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7640cd680 msgr2=0x7fc76415f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:50.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.637+0000 7fc768dd7640 1 --2- 172.21.15.5:0/4289521010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7640cd680 0x7fc76415f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.637+0000 7fc768dd7640 1 -- 172.21.15.5:0/4289521010 >> v1:172.21.15.63:6789/0 conn(0x7fc76406c6c0 legacy=0x7fc7640cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:50.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.637+0000 7fc768dd7640 1 -- 172.21.15.5:0/4289521010 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76406d6b0 con 0x7fc7640cfae0 2024-05-20T18:35:50.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.637+0000 7fc768dd7640 1 -- 172.21.15.5:0/4289521010 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7fc75c0019f0 con 0x7fc7640cfae0 2024-05-20T18:35:50.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.637+0000 7fc768dd7640 1 -- 172.21.15.5:0/4289521010 >> v1:172.21.15.5:6789/0 conn(0x7fc7640cfae0 legacy=0x7fc7640d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:50.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.637+0000 7fc768dd7640 1 --2- 172.21.15.5:0/4289521010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7500039e0 0x7fc750003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.637+0000 7fc768dd7640 1 --2- 172.21.15.5:0/4289521010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7640cd680 0x7fc750004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.637+0000 7fc768dd7640 1 --2- 172.21.15.5:0/4289521010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc750004d40 0x7fc750005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.637+0000 7fc768dd7640 1 -- 172.21.15.5:0/4289521010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc75000bb50 con 0x7fc750004d40 2024-05-20T18:35:50.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.637+0000 7fc768dd7640 1 -- 172.21.15.5:0/4289521010 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc75000bcd0 con 0x7fc7500039e0 2024-05-20T18:35:50.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.637+0000 7fc768dd7640 1 -- 172.21.15.5:0/4289521010 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc75000be80 con 0x7fc7640cd680 2024-05-20T18:35:50.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.637+0000 7fc769dd9640 1 --2- 172.21.15.5:0/4289521010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7500039e0 0x7fc750003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.637+0000 7fc76a5da640 1 --2- 172.21.15.5:0/4289521010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc750004d40 0x7fc750005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.637+0000 7fc7695d8640 1 --2- 172.21.15.5:0/4289521010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7640cd680 0x7fc750004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.638+0000 7fc769dd9640 1 -- 172.21.15.5:0/4289521010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7640cd680 msgr2=0x7fc750004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:50.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.638+0000 7fc769dd9640 1 --2- 172.21.15.5:0/4289521010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7640cd680 0x7fc750004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.638+0000 7fc769dd9640 1 -- 172.21.15.5:0/4289521010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc750004d40 msgr2=0x7fc750005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:50.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.638+0000 7fc769dd9640 1 --2- 172.21.15.5:0/4289521010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc750004d40 0x7fc750005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.638+0000 7fc769dd9640 1 -- 172.21.15.5:0/4289521010 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc750003520 con 0x7fc7500039e0 2024-05-20T18:35:50.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.638+0000 7fc7695d8640 1 --2- 172.21.15.5:0/4289521010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7640cd680 0x7fc750004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:35:50.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.638+0000 7fc76a5da640 1 --2- 172.21.15.5:0/4289521010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc750004d40 0x7fc750005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:35:50.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.639+0000 7fc769dd9640 1 --2- 172.21.15.5:0/4289521010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7500039e0 0x7fc750003e10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc760009a90 tx=0x7fc750005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b110cf8ac912e1ff server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:50.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.639+0000 7fc768dd7640 1 -- 172.21.15.5:0/4289521010 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc760036070 con 0x7fc7500039e0 2024-05-20T18:35:50.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.639+0000 7fc768dd7640 1 -- 172.21.15.5:0/4289521010 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc760031400 con 0x7fc7500039e0 2024-05-20T18:35:50.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.639+0000 7fc768dd7640 1 -- 172.21.15.5:0/4289521010 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc760009690 con 0x7fc7500039e0 2024-05-20T18:35:50.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.639+0000 7fc76c03d640 1 -- 172.21.15.5:0/4289521010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7500039e0 msgr2=0x7fc750003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:50.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.639+0000 7fc76c03d640 1 --2- 172.21.15.5:0/4289521010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7500039e0 0x7fc750003e10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc760009a90 tx=0x7fc750005830 comp rx=0 tx=0).stop 2024-05-20T18:35:50.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.639+0000 7fc76a5da640 1 -- 172.21.15.5:0/4289521010 reap_dead start 2024-05-20T18:35:50.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.640+0000 7fc76c03d640 1 -- 172.21.15.5:0/4289521010 shutdown_connections 2024-05-20T18:35:50.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.640+0000 7fc76c03d640 1 -- 172.21.15.5:0/4289521010 >> 172.21.15.5:0/4289521010 conn(0x7fc7640b44d0 msgr2=0x7fc7640b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:50.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.640+0000 7fc76c03d640 1 -- 172.21.15.5:0/4289521010 shutdown_connections 2024-05-20T18:35:50.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.640+0000 7fc76c03d640 1 -- 172.21.15.5:0/4289521010 wait complete. 2024-05-20T18:35:50.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.641+0000 7fc76c03d640 1 Processor -- start 2024-05-20T18:35:50.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.641+0000 7fc76c03d640 1 -- start start 2024-05-20T18:35:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.643+0000 7fc76c03d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7500039e0 0x7fc7641497e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.643+0000 7fc76c03d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7640b1270 0x7fc764147a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.643+0000 7fc76c03d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7640b1680 0x7fc7640af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.643+0000 7fc76c03d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc7640d2460 con 0x7fc7640b1270 2024-05-20T18:35:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.643+0000 7fc76c03d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc76406da10 con 0x7fc7500039e0 2024-05-20T18:35:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.643+0000 7fc76c03d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc7640d25e0 con 0x7fc7640b1680 2024-05-20T18:35:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.643+0000 7fc7695d8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7640b1270 0x7fc764147a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.643+0000 7fc7695d8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7640b1270 0x7fc764147a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38052/0 (socket says 172.21.15.5:38052) 2024-05-20T18:35:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.643+0000 7fc7695d8640 1 -- 172.21.15.5:0/1720119497 learned_addr learned my addr 172.21.15.5:0/1720119497 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.643+0000 7fc769dd9640 1 --2- 172.21.15.5:0/1720119497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7500039e0 0x7fc7641497e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.643+0000 7fc76a5da640 1 --2- 172.21.15.5:0/1720119497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7640b1680 0x7fc7640af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.643+0000 7fc7695d8640 1 -- 172.21.15.5:0/1720119497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7640b1680 msgr2=0x7fc7640af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.643+0000 7fc7695d8640 1 --2- 172.21.15.5:0/1720119497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7640b1680 0x7fc7640af420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.643+0000 7fc7695d8640 1 -- 172.21.15.5:0/1720119497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7500039e0 msgr2=0x7fc7641497e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.643+0000 7fc7695d8640 1 --2- 172.21.15.5:0/1720119497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7500039e0 0x7fc7641497e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.643+0000 7fc7695d8640 1 -- 172.21.15.5:0/1720119497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc764148740 con 0x7fc7640b1270 2024-05-20T18:35:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.643+0000 7fc769dd9640 1 --2- 172.21.15.5:0/1720119497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7500039e0 0x7fc7641497e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:35:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.643+0000 7fc7695d8640 1 --2- 172.21.15.5:0/1720119497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7640b1270 0x7fc764147a10 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fc75400fc10 tx=0x7fc75400d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.644+0000 7fc75a7fc640 1 -- 172.21.15.5:0/1720119497 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75400dc30 con 0x7fc7640b1270 2024-05-20T18:35:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.644+0000 7fc75a7fc640 1 -- 172.21.15.5:0/1720119497 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc754018070 con 0x7fc7640b1270 2024-05-20T18:35:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.644+0000 7fc76c03d640 1 -- 172.21.15.5:0/1720119497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc764148a20 con 0x7fc7640b1270 2024-05-20T18:35:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.644+0000 7fc75a7fc640 1 -- 172.21.15.5:0/1720119497 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7540050d0 con 0x7fc7640b1270 2024-05-20T18:35:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.644+0000 7fc76c03d640 1 -- 172.21.15.5:0/1720119497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc764148fe0 con 0x7fc7640b1270 2024-05-20T18:35:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.646+0000 7fc75a7fc640 1 -- 172.21.15.5:0/1720119497 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc754005270 con 0x7fc7640b1270 2024-05-20T18:35:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.646+0000 7fc75a7fc640 1 --2- 172.21.15.5:0/1720119497 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc74806be60 0x7fc74806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.647+0000 7fc75a7fc640 1 -- 172.21.15.5:0/1720119497 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7fc75408b050 con 0x7fc7640b1270 2024-05-20T18:35:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.647+0000 7fc769dd9640 1 --2- 172.21.15.5:0/1720119497 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc74806be60 0x7fc74806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.647+0000 7fc769dd9640 1 --2- 172.21.15.5:0/1720119497 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc74806be60 0x7fc74806e370 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc760033000 tx=0x7fc760008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.647+0000 7fc772fc3740 1 Processor -- start 2024-05-20T18:35:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.648+0000 7fc772fc3740 1 -- start start 2024-05-20T18:35:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.649+0000 7fc772fc3740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5586cd068390 0x5586cd432b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.649+0000 7fc772fc3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586cd03f4d0 0x5586cd44d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.649+0000 7fc772fc3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586cd0476b0 0x5586cd44ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.649+0000 7fc772fc3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5586cd36cd00 con 0x5586cd0476b0 2024-05-20T18:35:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.649+0000 7fc772fc3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5586cd3144c0 con 0x5586cd03f4d0 2024-05-20T18:35:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.649+0000 7fc772fc3740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5586cd368cc0 con 0x5586cd068390 2024-05-20T18:35:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.649+0000 7fc76a5da640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586cd0476b0 0x5586cd44ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.649+0000 7fc76a5da640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586cd0476b0 0x5586cd44ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38062/0 (socket says 172.21.15.5:38062) 2024-05-20T18:35:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.649+0000 7fc76a5da640 1 -- 172.21.15.5:0/3779844621 learned_addr learned my addr 172.21.15.5:0/3779844621 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.649+0000 7fc769dd9640 1 --2- 172.21.15.5:0/3779844621 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5586cd068390 0x5586cd432b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.649+0000 7fc7695d8640 1 --2- 172.21.15.5:0/3779844621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586cd03f4d0 0x5586cd44d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.649+0000 7fc76a5da640 1 -- 172.21.15.5:0/3779844621 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5586cd068390 msgr2=0x5586cd432b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.649+0000 7fc76a5da640 1 --2- 172.21.15.5:0/3779844621 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5586cd068390 0x5586cd432b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.649+0000 7fc76a5da640 1 -- 172.21.15.5:0/3779844621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586cd03f4d0 msgr2=0x5586cd44d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.649+0000 7fc76a5da640 1 --2- 172.21.15.5:0/3779844621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586cd03f4d0 0x5586cd44d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.649+0000 7fc76a5da640 1 -- 172.21.15.5:0/3779844621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586cd14a7d0 con 0x5586cd0476b0 2024-05-20T18:35:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.649+0000 7fc76a5da640 1 --2- 172.21.15.5:0/3779844621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586cd0476b0 0x5586cd44ddd0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fc75c002230 tx=0x7fc75c0021a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ebc533bc10a31b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.650+0000 7fc7477fe640 1 -- 172.21.15.5:0/3779844621 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c00a410 con 0x5586cd0476b0 2024-05-20T18:35:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.650+0000 7fc7477fe640 1 -- 172.21.15.5:0/3779844621 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc75c00adb0 con 0x5586cd0476b0 2024-05-20T18:35:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.650+0000 7fc7477fe640 1 -- 172.21.15.5:0/3779844621 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7540014e0 con 0x5586cd0476b0 2024-05-20T18:35:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.650+0000 7fc772fc3740 1 -- 172.21.15.5:0/3779844621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586cd0476b0 msgr2=0x5586cd44ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.650+0000 7fc772fc3740 1 --2- 172.21.15.5:0/3779844621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586cd0476b0 0x5586cd44ddd0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fc75c002230 tx=0x7fc75c0021a0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.650+0000 7fc772fc3740 1 -- 172.21.15.5:0/3779844621 shutdown_connections 2024-05-20T18:35:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.650+0000 7fc772fc3740 1 --2- 172.21.15.5:0/3779844621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586cd0476b0 0x5586cd44ddd0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.650+0000 7fc772fc3740 1 --2- 172.21.15.5:0/3779844621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586cd03f4d0 0x5586cd44d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.650+0000 7fc772fc3740 1 --2- 172.21.15.5:0/3779844621 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5586cd068390 0x5586cd432b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.650+0000 7fc772fc3740 1 -- 172.21.15.5:0/3779844621 >> 172.21.15.5:0/3779844621 conn(0x5586cd05c4e0 msgr2=0x5586cd4081c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.650+0000 7fc772fc3740 1 -- 172.21.15.5:0/3779844621 shutdown_connections 2024-05-20T18:35:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.650+0000 7fc772fc3740 1 -- 172.21.15.5:0/3779844621 wait complete. 2024-05-20T18:35:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.651+0000 7fc772fc3740 1 -- start start 2024-05-20T18:35:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.651+0000 7fc772fc3740 1 Processor -- start 2024-05-20T18:35:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.653+0000 7fc772fc3740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5586cd03f4d0 0x5586cd42e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.653+0000 7fc772fc3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586cd0476b0 0x5586cd42e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.653+0000 7fc772fc3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586cd068390 0x5586cd451430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.653+0000 7fc772fc3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5586cd327b90 con 0x5586cd0476b0 2024-05-20T18:35:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.653+0000 7fc772fc3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5586cd42a6e0 con 0x5586cd068390 2024-05-20T18:35:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.653+0000 7fc772fc3740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5586cd42a4d0 con 0x5586cd03f4d0 2024-05-20T18:35:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.653+0000 7fc772fc3740 10 client.0 _ll_register_callbacks cb 0x5586cd363da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:35:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.653+0000 7fc7695d8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586cd0476b0 0x5586cd42e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.653+0000 7fc7695d8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586cd0476b0 0x5586cd42e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38068/0 (socket says 172.21.15.5:38068) 2024-05-20T18:35:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.653+0000 7fc7695d8640 1 -- 172.21.15.5:0/877528395 learned_addr learned my addr 172.21.15.5:0/877528395 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.653+0000 7fc769dd9640 1 --2- 172.21.15.5:0/877528395 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5586cd03f4d0 0x5586cd42e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.653+0000 7fc76a5da640 1 --2- 172.21.15.5:0/877528395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586cd068390 0x5586cd451430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.653+0000 7fc7695d8640 1 -- 172.21.15.5:0/877528395 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5586cd03f4d0 msgr2=0x5586cd42e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.653+0000 7fc7695d8640 1 --2- 172.21.15.5:0/877528395 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5586cd03f4d0 0x5586cd42e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.653+0000 7fc7695d8640 1 -- 172.21.15.5:0/877528395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586cd068390 msgr2=0x5586cd451430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.653+0000 7fc7695d8640 1 --2- 172.21.15.5:0/877528395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586cd068390 0x5586cd451430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.653+0000 7fc7695d8640 1 -- 172.21.15.5:0/877528395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586cd0803f0 con 0x5586cd0476b0 2024-05-20T18:35:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.654+0000 7fc7695d8640 1 --2- 172.21.15.5:0/877528395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586cd0476b0 0x5586cd42e710 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fc7540050a0 tx=0x7fc7540014e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc3b85ccdeb506d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.654+0000 7fc744ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:35:50.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.654+0000 7fc744ff9640 1 -- 172.21.15.5:0/877528395 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc754086070 con 0x5586cd0476b0 2024-05-20T18:35:50.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.654+0000 7fc744ff9640 1 -- 172.21.15.5:0/877528395 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7fc7540618c0 con 0x5586cd0476b0 2024-05-20T18:35:50.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.654+0000 7fc772fc3740 1 -- 172.21.15.5:0/877528395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc75c00adb0 con 0x5586cd0476b0 2024-05-20T18:35:50.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.654+0000 7fc744ff9640 1 -- 172.21.15.5:0/877528395 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fc75405d490 con 0x5586cd0476b0 2024-05-20T18:35:50.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.654+0000 7fc772fc3740 10 client.8664 fetch_fsmap learned FSMap version 82 2024-05-20T18:35:50.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.654+0000 7fc772fc3740 1 -- 172.21.15.5:0/877528395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x5586cd0400f0 con 0x5586cd0476b0 2024-05-20T18:35:50.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.654+0000 7fc744ff9640 1 -- 172.21.15.5:0/877528395 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc75c00adb0 con 0x5586cd0476b0 2024-05-20T18:35:50.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.654+0000 7fc744ff9640 1 -- 172.21.15.5:0/877528395 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7fc7540585e0 con 0x5586cd0476b0 2024-05-20T18:35:50.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.654+0000 7fc772fc3740 10 client.8664 fetch_fsmap finished waiting for FSMap version 82 2024-05-20T18:35:50.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.654+0000 7fc772fc3740 10 client.8664 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-05-20T18:35:50.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.654+0000 7fc772fc3740 20 client.8664 populate_metadata read hostname 'smithi005' 2024-05-20T18:35:50.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.654+0000 7fc772fc3740 1 --2- 172.21.15.5:0/877528395 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x5586cd026f60 0x5586cd433ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.654+0000 7fc772fc3740 4 client.8664 mds_command: new command op to 8343 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:35:50.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.654+0000 7fc772fc3740 1 -- 172.21.15.5:0/877528395 --> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586cd37a430 con 0x5586cd026f60 2024-05-20T18:35:50.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.654+0000 7fc769dd9640 1 --2- 172.21.15.5:0/877528395 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x5586cd026f60 0x5586cd433ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.655+0000 7fc744ff9640 10 client.8664 ms_handle_connect on v2:172.21.15.63:6833/2054897241 2024-05-20T18:35:50.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.655+0000 7fc769dd9640 1 --2- 172.21.15.5:0/877528395 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x5586cd026f60 0x5586cd433ca0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e76f53c6de2a2f4 server_cookie=f573d9f03c06deee in_seq=0 out_seq=0 2024-05-20T18:35:50.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.655+0000 7fc744ff9640 1 -- 172.21.15.5:0/877528395 <== mds.0 v2:172.21.15.63:6833/2054897241 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5586cd37a430 con 0x5586cd026f60 2024-05-20T18:35:50.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.655+0000 7fc744ff9640 10 client.8664 handle_command_reply: tid=0 2024-05-20T18:35:50.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.656+0000 7fc772fc3740 1 client.8664 shutdown 2024-05-20T18:35:50.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.656+0000 7fc772fc3740 1 -- 172.21.15.5:0/877528395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586cd0476b0 msgr2=0x5586cd42e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:50.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.656+0000 7fc772fc3740 1 --2- 172.21.15.5:0/877528395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586cd0476b0 0x5586cd42e710 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fc7540050a0 tx=0x7fc7540014e0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.656+0000 7fc772fc3740 1 -- 172.21.15.5:0/877528395 shutdown_connections 2024-05-20T18:35:50.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.656+0000 7fc772fc3740 1 --2- 172.21.15.5:0/877528395 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x5586cd026f60 0x5586cd433ca0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.656+0000 7fc772fc3740 1 --2- 172.21.15.5:0/877528395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586cd068390 0x5586cd451430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.656+0000 7fc772fc3740 1 --2- 172.21.15.5:0/877528395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586cd0476b0 0x5586cd42e710 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.656+0000 7fc744ff9640 0 client.8664 ms_handle_reset on v2:172.21.15.63:6833/2054897241 2024-05-20T18:35:50.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.656+0000 7fc772fc3740 1 --2- 172.21.15.5:0/877528395 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5586cd03f4d0 0x5586cd42e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.656+0000 7fc772fc3740 1 -- 172.21.15.5:0/877528395 >> 172.21.15.5:0/877528395 conn(0x5586cd05c4e0 msgr2=0x5586cd432b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:50.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.656+0000 7fc772fc3740 1 -- 172.21.15.5:0/877528395 shutdown_connections 2024-05-20T18:35:50.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.656+0000 7fc772fc3740 1 -- 172.21.15.5:0/877528395 wait complete. 2024-05-20T18:35:50.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.656+0000 7fc772fc3740 20 client.8664 trim_cache size 0 max 16384 2024-05-20T18:35:50.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.675+0000 7fc772fc3740 1 Processor -- start 2024-05-20T18:35:50.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.675+0000 7fc772fc3740 1 -- start start 2024-05-20T18:35:50.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.676+0000 7fc772fc3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586cd03f4d0 0x5586cd44dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.676+0000 7fc772fc3740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5586cd0476b0 0x5586cd403830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.676+0000 7fc772fc3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586cd068390 0x5586cd402ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.676+0000 7fc772fc3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc764202170 con 0x5586cd03f4d0 2024-05-20T18:35:50.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.676+0000 7fc772fc3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc7642012d0 con 0x5586cd068390 2024-05-20T18:35:50.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.676+0000 7fc772fc3740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc7640cb730 con 0x5586cd0476b0 2024-05-20T18:35:50.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.676+0000 7fc769dd9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586cd03f4d0 0x5586cd44dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.676+0000 7fc769dd9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586cd03f4d0 0x5586cd44dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38070/0 (socket says 172.21.15.5:38070) 2024-05-20T18:35:50.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.676+0000 7fc769dd9640 1 -- 172.21.15.5:0/2422479115 learned_addr learned my addr 172.21.15.5:0/2422479115 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:50.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.676+0000 7fc7695d8640 1 --2- 172.21.15.5:0/2422479115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5586cd0476b0 0x5586cd403830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.676+0000 7fc76a5da640 1 --2- 172.21.15.5:0/2422479115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586cd068390 0x5586cd402ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.676+0000 7fc769dd9640 1 -- 172.21.15.5:0/2422479115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5586cd0476b0 msgr2=0x5586cd403830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:50.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.676+0000 7fc769dd9640 1 --2- 172.21.15.5:0/2422479115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5586cd0476b0 0x5586cd403830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.676+0000 7fc769dd9640 1 -- 172.21.15.5:0/2422479115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586cd068390 msgr2=0x5586cd402ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.676+0000 7fc769dd9640 1 --2- 172.21.15.5:0/2422479115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586cd068390 0x5586cd402ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.676+0000 7fc769dd9640 1 -- 172.21.15.5:0/2422479115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586cd45c540 con 0x5586cd03f4d0 2024-05-20T18:35:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.676+0000 7fc76a5da640 1 --2- 172.21.15.5:0/2422479115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586cd068390 0x5586cd402ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.676+0000 7fc7695d8640 1 --2- 172.21.15.5:0/2422479115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5586cd0476b0 0x5586cd403830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:35:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.677+0000 7fc769dd9640 1 --2- 172.21.15.5:0/2422479115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586cd03f4d0 0x5586cd44dd30 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fc760009b80 tx=0x7fc76002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f20a931163b683 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.677+0000 7fc746ffd640 1 -- 172.21.15.5:0/2422479115 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc760041040 con 0x5586cd03f4d0 2024-05-20T18:35:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.677+0000 7fc746ffd640 1 -- 172.21.15.5:0/2422479115 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc760058070 con 0x5586cd03f4d0 2024-05-20T18:35:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.677+0000 7fc746ffd640 1 -- 172.21.15.5:0/2422479115 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc760036040 con 0x5586cd03f4d0 2024-05-20T18:35:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.677+0000 7fc772fc3740 1 -- 172.21.15.5:0/2422479115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586cd03f4d0 msgr2=0x5586cd44dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.677+0000 7fc772fc3740 1 --2- 172.21.15.5:0/2422479115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586cd03f4d0 0x5586cd44dd30 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fc760009b80 tx=0x7fc76002e000 comp rx=0 tx=0).stop 2024-05-20T18:35:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.677+0000 7fc772fc3740 1 -- 172.21.15.5:0/2422479115 shutdown_connections 2024-05-20T18:35:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.677+0000 7fc772fc3740 1 --2- 172.21.15.5:0/2422479115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586cd068390 0x5586cd402ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.677+0000 7fc772fc3740 1 --2- 172.21.15.5:0/2422479115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5586cd0476b0 0x5586cd403830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.677+0000 7fc772fc3740 1 --2- 172.21.15.5:0/2422479115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586cd03f4d0 0x5586cd44dd30 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.677+0000 7fc772fc3740 1 -- 172.21.15.5:0/2422479115 >> 172.21.15.5:0/2422479115 conn(0x5586cd05c4e0 msgr2=0x5586cd447320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:50.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.678+0000 7fc772fc3740 1 -- 172.21.15.5:0/2422479115 shutdown_connections 2024-05-20T18:35:50.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.678+0000 7fc772fc3740 1 -- 172.21.15.5:0/2422479115 wait complete. 2024-05-20T18:35:50.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.678+0000 7fc772fc3740 1 -- start start 2024-05-20T18:35:50.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.678+0000 7fc772fc3740 1 Processor -- start 2024-05-20T18:35:50.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.680+0000 7fc772fc3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586cd03f4d0 0x5586cd4504a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.680+0000 7fc772fc3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586cd0476b0 0x5586cd4509e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.680+0000 7fc772fc3740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5586cd068390 0x5586cd42f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.680+0000 7fc772fc3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc7640cbd80 con 0x5586cd0476b0 2024-05-20T18:35:50.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.680+0000 7fc772fc3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5586cd4294f0 con 0x5586cd03f4d0 2024-05-20T18:35:50.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.680+0000 7fc772fc3740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5586cd44d070 con 0x5586cd068390 2024-05-20T18:35:50.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.680+0000 7fc772fc3740 10 client.0 _ll_register_callbacks cb 0x7fc754058f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:35:50.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.680+0000 7fc7695d8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586cd0476b0 0x5586cd4509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.680+0000 7fc7695d8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586cd0476b0 0x5586cd4509e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38074/0 (socket says 172.21.15.5:38074) 2024-05-20T18:35:50.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.680+0000 7fc7695d8640 1 -- 172.21.15.5:0/1956522308 learned_addr learned my addr 172.21.15.5:0/1956522308 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:35:50.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.680+0000 7fc76a5da640 1 --2- 172.21.15.5:0/1956522308 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5586cd068390 0x5586cd42f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.680+0000 7fc769dd9640 1 --2- 172.21.15.5:0/1956522308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586cd03f4d0 0x5586cd4504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.680+0000 7fc7695d8640 1 -- 172.21.15.5:0/1956522308 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5586cd068390 msgr2=0x5586cd42f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:50.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.680+0000 7fc7695d8640 1 --2- 172.21.15.5:0/1956522308 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5586cd068390 0x5586cd42f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.680+0000 7fc7695d8640 1 -- 172.21.15.5:0/1956522308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586cd03f4d0 msgr2=0x5586cd4504a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:35:50.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.680+0000 7fc7695d8640 1 --2- 172.21.15.5:0/1956522308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586cd03f4d0 0x5586cd4504a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.680+0000 7fc7695d8640 1 -- 172.21.15.5:0/1956522308 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586cd44ded0 con 0x5586cd0476b0 2024-05-20T18:35:50.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.681+0000 7fc7695d8640 1 --2- 172.21.15.5:0/1956522308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586cd0476b0 0x5586cd4509e0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fc754059120 tx=0x7fc7540055e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71add0574ddcbca5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:35:50.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.681+0000 7fc744ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:35:50.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.681+0000 7fc744ff9640 1 -- 172.21.15.5:0/1956522308 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc754086040 con 0x5586cd0476b0 2024-05-20T18:35:50.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.681+0000 7fc744ff9640 1 -- 172.21.15.5:0/1956522308 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7fc754054ee0 con 0x5586cd0476b0 2024-05-20T18:35:50.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.681+0000 7fc772fc3740 1 -- 172.21.15.5:0/1956522308 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc760058070 con 0x5586cd0476b0 2024-05-20T18:35:50.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.681+0000 7fc744ff9640 1 -- 172.21.15.5:0/1956522308 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fc754055430 con 0x5586cd0476b0 2024-05-20T18:35:50.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.681+0000 7fc772fc3740 10 client.8670 fetch_fsmap learned FSMap version 82 2024-05-20T18:35:50.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.681+0000 7fc772fc3740 1 -- 172.21.15.5:0/1956522308 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x5586cd428f40 con 0x5586cd0476b0 2024-05-20T18:35:50.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.681+0000 7fc744ff9640 1 -- 172.21.15.5:0/1956522308 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc760058070 con 0x5586cd0476b0 2024-05-20T18:35:50.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.681+0000 7fc744ff9640 1 -- 172.21.15.5:0/1956522308 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7fc7540550e0 con 0x5586cd0476b0 2024-05-20T18:35:50.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.681+0000 7fc772fc3740 10 client.8670 fetch_fsmap finished waiting for FSMap version 82 2024-05-20T18:35:50.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.681+0000 7fc772fc3740 10 client.8670 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-05-20T18:35:50.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.681+0000 7fc772fc3740 20 client.8670 populate_metadata read hostname 'smithi005' 2024-05-20T18:35:50.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.681+0000 7fc772fc3740 1 --2- 172.21.15.5:0/1956522308 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x5586cd434ed0 0x5586cd42f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:35:50.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.681+0000 7fc772fc3740 4 client.8670 mds_command: new command op to 8343 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T18:35:50.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.681+0000 7fc772fc3740 1 -- 172.21.15.5:0/1956522308 --> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5586cd374780 con 0x5586cd434ed0 2024-05-20T18:35:50.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.681+0000 7fc769dd9640 1 --2- 172.21.15.5:0/1956522308 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x5586cd434ed0 0x5586cd42f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:35:50.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.682+0000 7fc744ff9640 10 client.8670 ms_handle_connect on v2:172.21.15.63:6833/2054897241 2024-05-20T18:35:50.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.682+0000 7fc769dd9640 1 --2- 172.21.15.5:0/1956522308 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x5586cd434ed0 0x5586cd42f660 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=292e473815d2ebba server_cookie=2df624143d9d5c39 in_seq=0 out_seq=0 2024-05-20T18:35:50.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.682+0000 7fc744ff9640 1 -- 172.21.15.5:0/1956522308 <== mds.0 v2:172.21.15.63:6833/2054897241 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x5586cd374780 con 0x5586cd434ed0 2024-05-20T18:35:50.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.682+0000 7fc744ff9640 10 client.8670 handle_command_reply: tid=0 2024-05-20T18:35:50.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.682+0000 7fc772fc3740 1 client.8670 shutdown 2024-05-20T18:35:50.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.682+0000 7fc772fc3740 1 -- 172.21.15.5:0/1956522308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586cd0476b0 msgr2=0x5586cd4509e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:50.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.682+0000 7fc772fc3740 1 --2- 172.21.15.5:0/1956522308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586cd0476b0 0x5586cd4509e0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fc754059120 tx=0x7fc7540055e0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.682+0000 7fc772fc3740 1 -- 172.21.15.5:0/1956522308 shutdown_connections 2024-05-20T18:35:50.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.682+0000 7fc772fc3740 1 --2- 172.21.15.5:0/1956522308 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x5586cd434ed0 0x5586cd42f660 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.682+0000 7fc772fc3740 1 --2- 172.21.15.5:0/1956522308 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5586cd068390 0x5586cd42f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.682+0000 7fc744ff9640 0 client.8670 ms_handle_reset on v2:172.21.15.63:6833/2054897241 2024-05-20T18:35:50.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.682+0000 7fc772fc3740 1 --2- 172.21.15.5:0/1956522308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586cd0476b0 0x5586cd4509e0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.682+0000 7fc772fc3740 1 --2- 172.21.15.5:0/1956522308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5586cd03f4d0 0x5586cd4504a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.682+0000 7fc772fc3740 1 -- 172.21.15.5:0/1956522308 >> 172.21.15.5:0/1956522308 conn(0x5586cd05c4e0 msgr2=0x5586cd403830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:50.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.682+0000 7fc772fc3740 1 -- 172.21.15.5:0/1956522308 shutdown_connections 2024-05-20T18:35:50.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.682+0000 7fc772fc3740 1 -- 172.21.15.5:0/1956522308 wait complete. 2024-05-20T18:35:50.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.682+0000 7fc772fc3740 20 client.8670 trim_cache size 0 max 16384 2024-05-20T18:35:50.689 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:35:50.690 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-20T18:35:50.690 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-20T18:35:50.690 INFO:teuthology.orchestra.run.smithi005.stdout: "6c211f6d-2e81-47ae-9e81-478bb3882d33": { 2024-05-20T18:35:50.690 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-05-20T18:35:50.690 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "6c211f6d-2e81-47ae-9e81-478bb3882d33", 2024-05-20T18:35:50.690 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,repair" 2024-05-20T18:35:50.690 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:35:50.690 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:35:50.690 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:35:50.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.683+0000 7fc76c03d640 1 -- 172.21.15.5:0/1720119497 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc74806be60 msgr2=0x7fc74806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:50.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.683+0000 7fc76c03d640 1 --2- 172.21.15.5:0/1720119497 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc74806be60 0x7fc74806e370 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc760033000 tx=0x7fc760008030 comp rx=0 tx=0).stop 2024-05-20T18:35:50.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.683+0000 7fc76c03d640 1 -- 172.21.15.5:0/1720119497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7640b1270 msgr2=0x7fc764147a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:35:50.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.683+0000 7fc76c03d640 1 --2- 172.21.15.5:0/1720119497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7640b1270 0x7fc764147a10 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fc75400fc10 tx=0x7fc75400d520 comp rx=0 tx=0).stop 2024-05-20T18:35:50.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.683+0000 7fc76c03d640 1 -- 172.21.15.5:0/1720119497 shutdown_connections 2024-05-20T18:35:50.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.683+0000 7fc76c03d640 1 --2- 172.21.15.5:0/1720119497 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc74806be60 0x7fc74806e370 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.683+0000 7fc76c03d640 1 --2- 172.21.15.5:0/1720119497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7640b1680 0x7fc7640af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.683+0000 7fc76c03d640 1 --2- 172.21.15.5:0/1720119497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7640b1270 0x7fc764147a10 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.683+0000 7fc76c03d640 1 --2- 172.21.15.5:0/1720119497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7500039e0 0x7fc7641497e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:35:50.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.683+0000 7fc76c03d640 1 -- 172.21.15.5:0/1720119497 >> 172.21.15.5:0/1720119497 conn(0x7fc7640b44d0 msgr2=0x7fc7640b3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:35:50.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.683+0000 7fc76c03d640 1 -- 172.21.15.5:0/1720119497 shutdown_connections 2024-05-20T18:35:50.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:50.683+0000 7fc76c03d640 1 -- 172.21.15.5:0/1720119497 wait complete. 2024-05-20T18:35:50.695 INFO:tasks.cephfs.filesystem:scrub status for tag:6c211f6d-2e81-47ae-9e81-478bb3882d33 - {'path': '/', 'tag': '6c211f6d-2e81-47ae-9e81-478bb3882d33', 'options': 'recursive,repair'} 2024-05-20T18:35:51.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:51.804+0000 7fe7b6ffd640 1 -- 172.21.15.5:0/1376524015 <== mon.0 v2:172.21.15.5:3300/0 7 ==== log(5 entries from seq 1 at 2024-05-20T18:35:50.504070+0000) ==== 846+0+0 (secure 0 0 0) 0x7fe7a80093e0 con 0x7fe7c00b6c20 2024-05-20T18:35:51.806 INFO:teuthology.orchestra.run.smithi005.stdout: cluster: 2024-05-20T18:35:51.806 INFO:teuthology.orchestra.run.smithi005.stdout: id: 399ab250-fd3c-47fa-a1c8-25e8149c9cac 2024-05-20T18:35:51.806 INFO:teuthology.orchestra.run.smithi005.stdout: health: HEALTH_OK 2024-05-20T18:35:51.806 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:51.806 INFO:teuthology.orchestra.run.smithi005.stdout: services: 2024-05-20T18:35:51.807 INFO:teuthology.orchestra.run.smithi005.stdout: mon: 3 daemons, quorum a,b,c (age 9m) 2024-05-20T18:35:51.807 INFO:teuthology.orchestra.run.smithi005.stdout: mgr: y(active, since 8m), standbys: x 2024-05-20T18:35:51.807 INFO:teuthology.orchestra.run.smithi005.stdout: mds: 1/1 daemons up, 3 standby 2024-05-20T18:35:51.807 INFO:teuthology.orchestra.run.smithi005.stdout: osd: 8 osds: 8 up (since 8m), 8 in (since 8m) 2024-05-20T18:35:51.807 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:51.807 INFO:teuthology.orchestra.run.smithi005.stdout: data: 2024-05-20T18:35:51.807 INFO:teuthology.orchestra.run.smithi005.stdout: volumes: 1/1 healthy 2024-05-20T18:35:51.807 INFO:teuthology.orchestra.run.smithi005.stdout: pools: 3 pools, 97 pgs 2024-05-20T18:35:51.807 INFO:teuthology.orchestra.run.smithi005.stdout: objects: 26 objects, 6.6 MiB 2024-05-20T18:35:51.807 INFO:teuthology.orchestra.run.smithi005.stdout: usage: 267 MiB used, 720 GiB / 720 GiB avail 2024-05-20T18:35:51.807 INFO:teuthology.orchestra.run.smithi005.stdout: pgs: 97 active+clean 2024-05-20T18:35:51.807 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:51.807 INFO:teuthology.orchestra.run.smithi005.stdout: io: 2024-05-20T18:35:51.807 INFO:teuthology.orchestra.run.smithi005.stdout: client: 85 B/s rd, 514 KiB/s wr, 0 op/s rd, 2 op/s wr 2024-05-20T18:35:51.807 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:51.808 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:35:51.808 INFO:teuthology.orchestra.run.smithi005.stdout:2024-05-20T18:35:50.504070+0000 mds.b [INF] scrub queued for path: / 2024-05-20T18:35:51.808 INFO:teuthology.orchestra.run.smithi005.stdout:2024-05-20T18:35:50.504073+0000 mds.b [INF] scrub summary: idle+waiting paths [/] 2024-05-20T18:35:51.808 INFO:teuthology.orchestra.run.smithi005.stdout:2024-05-20T18:35:50.504079+0000 mds.b [INF] scrub summary: active paths [/] 2024-05-20T18:35:51.808 INFO:teuthology.orchestra.run.smithi005.stdout:2024-05-20T18:35:50.505542+0000 mds.b [INF] scrub summary: idle+waiting paths [/] 2024-05-20T18:35:51.808 INFO:teuthology.orchestra.run.smithi005.stdout:2024-05-20T18:35:51.464636+0000 mds.b [INF] scrub summary: idle 2024-05-20T18:35:56.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:56.812+0000 7fe7b6ffd640 1 -- 172.21.15.5:0/1376524015 <== mon.0 v2:172.21.15.5:3300/0 8 ==== log(2 entries from seq 1178 at 2024-05-20T18:35:56.346821+0000) ==== 603+0+0 (secure 0 0 0) 0x7fe7a8057030 con 0x7fe7c00b6c20 2024-05-20T18:35:57.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:35:57.826+0000 7fe7b6ffd640 1 -- 172.21.15.5:0/1376524015 <== mon.0 v2:172.21.15.5:3300/0 9 ==== log(2 entries from seq 1180 at 2024-05-20T18:35:56.818828+0000) ==== 607+0+0 (secure 0 0 0) 0x7fe7a8057290 con 0x7fe7c00b6c20 2024-05-20T18:36:11.907 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:36:11.911 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:36:20.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:36:20.696 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-20T18:36:20.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.827+0000 7f17debd5640 1 Processor -- start 2024-05-20T18:36:20.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.828+0000 7f17debd5640 1 -- start start 2024-05-20T18:36:20.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.829+0000 7f17debd5640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f17d80b1e10 0x7f17d815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:20.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.829+0000 7f17debd5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f17d8070850 0x7f17d8070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:20.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.829+0000 7f17debd5640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17d80cd580 con 0x7f17d815d530 2024-05-20T18:36:20.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.829+0000 7f17debd5640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f17d80ce190 con 0x7f17d8070850 2024-05-20T18:36:20.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.829+0000 7f17debd5640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f17d80cd860 con 0x7f17d80b1e10 2024-05-20T18:36:20.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.829+0000 7f17dd172640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f17d8070850 0x7f17d8070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:20.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.829+0000 7f17dd172640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f17d8070850 0x7f17d8070c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57598/0 (socket says 172.21.15.5:57598) 2024-05-20T18:36:20.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.829+0000 7f17dd172640 1 -- 172.21.15.5:0/1289636625 learned_addr learned my addr 172.21.15.5:0/1289636625 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:20.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.829+0000 7f17d7fff640 1 --2- 172.21.15.5:0/1289636625 >> v2:172.21.15.63:3300/0 conn(0x7f17d80b1e10 0x7f17d815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:20.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.829+0000 7f17dd172640 1 -- 172.21.15.5:0/1289636625 >> v2:172.21.15.63:3300/0 conn(0x7f17d80b1e10 msgr2=0x7f17d815aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:20.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.830+0000 7f17dd172640 1 --2- 172.21.15.5:0/1289636625 >> v2:172.21.15.63:3300/0 conn(0x7f17d80b1e10 0x7f17d815aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.830+0000 7f17dd172640 1 -- 172.21.15.5:0/1289636625 >> v1:172.21.15.5:6789/0 conn(0x7f17d815d530 legacy=0x7f17d80b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:20.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.830+0000 7f17dd172640 1 -- 172.21.15.5:0/1289636625 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d80cdb40 con 0x7f17d8070850 2024-05-20T18:36:20.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.830+0000 7f17d7fff640 1 --2- 172.21.15.5:0/1289636625 >> v2:172.21.15.63:3300/0 conn(0x7f17d80b1e10 0x7f17d815aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:36:20.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.830+0000 7f17dd172640 1 --2- 172.21.15.5:0/1289636625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d8070850 0x7f17d8070c60 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f17d80d3ad0 tx=0x7f17d002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f47f4613cf608c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:20.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.830+0000 7f17d77fe640 1 -- 172.21.15.5:0/1289636625 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d0036070 con 0x7f17d8070850 2024-05-20T18:36:20.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.831+0000 7f17d77fe640 1 -- 172.21.15.5:0/1289636625 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17d0031600 con 0x7f17d8070850 2024-05-20T18:36:20.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.831+0000 7f17d77fe640 1 -- 172.21.15.5:0/1289636625 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d00319d0 con 0x7f17d8070850 2024-05-20T18:36:20.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.831+0000 7f17debd5640 1 -- 172.21.15.5:0/1289636625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d8070850 msgr2=0x7f17d8070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:20.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.831+0000 7f17debd5640 1 --2- 172.21.15.5:0/1289636625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d8070850 0x7f17d8070c60 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f17d80d3ad0 tx=0x7f17d002a730 comp rx=0 tx=0).stop 2024-05-20T18:36:20.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.831+0000 7f17debd5640 1 -- 172.21.15.5:0/1289636625 shutdown_connections 2024-05-20T18:36:20.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.831+0000 7f17debd5640 1 --2- 172.21.15.5:0/1289636625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d8070850 0x7f17d8070c60 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.831+0000 7f17debd5640 1 --2- 172.21.15.5:0/1289636625 >> v2:172.21.15.63:3300/0 conn(0x7f17d80b1e10 0x7f17d815aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.831+0000 7f17debd5640 1 -- 172.21.15.5:0/1289636625 >> 172.21.15.5:0/1289636625 conn(0x7f17d80d29d0 msgr2=0x7f17d80d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:20.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.831+0000 7f17debd5640 1 -- 172.21.15.5:0/1289636625 shutdown_connections 2024-05-20T18:36:20.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.831+0000 7f17debd5640 1 -- 172.21.15.5:0/1289636625 wait complete. 2024-05-20T18:36:20.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.833+0000 7f17debd5640 1 Processor -- start 2024-05-20T18:36:20.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.833+0000 7f17debd5640 1 -- start start 2024-05-20T18:36:20.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.835+0000 7f17debd5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17d8070850 0x7f17d80c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:20.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.835+0000 7f17debd5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d80b1e10 0x7f17d815f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:20.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.835+0000 7f17debd5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17d815d530 0x7f17d81555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:20.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.835+0000 7f17debd5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f17d80711a0 con 0x7f17d80b1e10 2024-05-20T18:36:20.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.835+0000 7f17debd5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f17d80cd230 con 0x7f17d815d530 2024-05-20T18:36:20.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.835+0000 7f17debd5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f17d8071320 con 0x7f17d8070850 2024-05-20T18:36:20.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.835+0000 7f17d7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d80b1e10 0x7f17d815f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:20.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.835+0000 7f17d7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d80b1e10 0x7f17d815f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57608/0 (socket says 172.21.15.5:57608) 2024-05-20T18:36:20.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.835+0000 7f17d7fff640 1 -- 172.21.15.5:0/2129416926 learned_addr learned my addr 172.21.15.5:0/2129416926 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.835+0000 7f17dd172640 1 --2- 172.21.15.5:0/2129416926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17d815d530 0x7f17d81555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.835+0000 7f17dc971640 1 --2- 172.21.15.5:0/2129416926 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17d8070850 0x7f17d80c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.835+0000 7f17d7fff640 1 -- 172.21.15.5:0/2129416926 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17d8070850 msgr2=0x7f17d80c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.835+0000 7f17d7fff640 1 --2- 172.21.15.5:0/2129416926 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17d8070850 0x7f17d80c6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.835+0000 7f17d7fff640 1 -- 172.21.15.5:0/2129416926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17d815d530 msgr2=0x7f17d81555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.835+0000 7f17d7fff640 1 --2- 172.21.15.5:0/2129416926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17d815d530 0x7f17d81555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.835+0000 7f17d7fff640 1 -- 172.21.15.5:0/2129416926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d80c6d50 con 0x7f17d80b1e10 2024-05-20T18:36:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.835+0000 7f17dc971640 1 --2- 172.21.15.5:0/2129416926 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17d8070850 0x7f17d80c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:36:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.835+0000 7f17d7fff640 1 --2- 172.21.15.5:0/2129416926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d80b1e10 0x7f17d815f8a0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f17c000d7d0 tx=0x7f17c000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.836+0000 7f17d57fa640 1 -- 172.21.15.5:0/2129416926 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0004030 con 0x7f17d80b1e10 2024-05-20T18:36:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.836+0000 7f17d57fa640 1 -- 172.21.15.5:0/2129416926 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17c00041d0 con 0x7f17d80b1e10 2024-05-20T18:36:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.836+0000 7f17d57fa640 1 -- 172.21.15.5:0/2129416926 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c00096a0 con 0x7f17d80b1e10 2024-05-20T18:36:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.836+0000 7f17debd5640 1 -- 172.21.15.5:0/2129416926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d80c7030 con 0x7f17d80b1e10 2024-05-20T18:36:20.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.836+0000 7f17debd5640 1 -- 172.21.15.5:0/2129416926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d80c75f0 con 0x7f17d80b1e10 2024-05-20T18:36:20.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.838+0000 7f17d57fa640 1 -- 172.21.15.5:0/2129416926 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f17c0009840 con 0x7f17d80b1e10 2024-05-20T18:36:20.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.838+0000 7f17d57fa640 1 --2- 172.21.15.5:0/2129416926 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f17b806be60 0x7f17b806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:20.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.838+0000 7f17d57fa640 1 -- 172.21.15.5:0/2129416926 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6159+0+0 (secure 0 0 0) 0x7f17c008b160 con 0x7f17d80b1e10 2024-05-20T18:36:20.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.838+0000 7f17dc971640 1 --2- 172.21.15.5:0/2129416926 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f17b806be60 0x7f17b806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:20.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.839+0000 7f17dc971640 1 --2- 172.21.15.5:0/2129416926 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f17b806be60 0x7f17b806e370 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f17d80d29a0 tx=0x7f17c8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:20.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.839+0000 7f17e5a84740 1 Processor -- start 2024-05-20T18:36:20.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.839+0000 7f17e5a84740 1 -- start start 2024-05-20T18:36:20.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.840+0000 7f17e5a84740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd3699390 0x564fd3a63b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:20.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.840+0000 7f17e5a84740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564fd36704d0 0x564fd3a7e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:20.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.840+0000 7f17e5a84740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fd36786b0 0x564fd3a7edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:20.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.840+0000 7f17e5a84740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564fd399dd00 con 0x564fd3699390 2024-05-20T18:36:20.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.841+0000 7f17e5a84740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564fd39454c0 con 0x564fd36786b0 2024-05-20T18:36:20.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.841+0000 7f17e5a84740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564fd3999cc0 con 0x564fd36704d0 2024-05-20T18:36:20.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.841+0000 7f17dc971640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd3699390 0x564fd3a63b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:20.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.841+0000 7f17dc971640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd3699390 0x564fd3a63b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57618/0 (socket says 172.21.15.5:57618) 2024-05-20T18:36:20.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.841+0000 7f17dc971640 1 -- 172.21.15.5:0/3582534384 learned_addr learned my addr 172.21.15.5:0/3582534384 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:20.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.841+0000 7f17d7fff640 1 --2- 172.21.15.5:0/3582534384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564fd36704d0 0x564fd3a7e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:20.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.841+0000 7f17dd172640 1 --2- 172.21.15.5:0/3582534384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fd36786b0 0x564fd3a7edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:20.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.841+0000 7f17dc971640 1 -- 172.21.15.5:0/3582534384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564fd36704d0 msgr2=0x564fd3a7e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:20.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.841+0000 7f17dc971640 1 --2- 172.21.15.5:0/3582534384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564fd36704d0 0x564fd3a7e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.841+0000 7f17dc971640 1 -- 172.21.15.5:0/3582534384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fd36786b0 msgr2=0x564fd3a7edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:20.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.841+0000 7f17dc971640 1 --2- 172.21.15.5:0/3582534384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fd36786b0 0x564fd3a7edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.841+0000 7f17dc971640 1 -- 172.21.15.5:0/3582534384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fd377b7d0 con 0x564fd3699390 2024-05-20T18:36:20.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.841+0000 7f17dc971640 1 --2- 172.21.15.5:0/3582534384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd3699390 0x564fd3a63b60 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f17c80052d0 tx=0x7f17c8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3531947dbf7ebf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:20.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.841+0000 7f17ce7fc640 1 -- 172.21.15.5:0/3582534384 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c8005480 con 0x564fd3699390 2024-05-20T18:36:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.841+0000 7f17ce7fc640 1 -- 172.21.15.5:0/3582534384 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17c800e740 con 0x564fd3699390 2024-05-20T18:36:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.841+0000 7f17ce7fc640 1 -- 172.21.15.5:0/3582534384 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c800eaf0 con 0x564fd3699390 2024-05-20T18:36:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.842+0000 7f17e5a84740 1 -- 172.21.15.5:0/3582534384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd3699390 msgr2=0x564fd3a63b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.842+0000 7f17e5a84740 1 --2- 172.21.15.5:0/3582534384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd3699390 0x564fd3a63b60 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f17c80052d0 tx=0x7f17c8011000 comp rx=0 tx=0).stop 2024-05-20T18:36:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.842+0000 7f17e5a84740 1 -- 172.21.15.5:0/3582534384 shutdown_connections 2024-05-20T18:36:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.842+0000 7f17e5a84740 1 --2- 172.21.15.5:0/3582534384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fd36786b0 0x564fd3a7edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.842+0000 7f17e5a84740 1 --2- 172.21.15.5:0/3582534384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564fd36704d0 0x564fd3a7e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.842+0000 7f17e5a84740 1 --2- 172.21.15.5:0/3582534384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd3699390 0x564fd3a63b60 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.842+0000 7f17e5a84740 1 -- 172.21.15.5:0/3582534384 >> 172.21.15.5:0/3582534384 conn(0x564fd368d4e0 msgr2=0x564fd3a391c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.842+0000 7f17e5a84740 1 -- 172.21.15.5:0/3582534384 shutdown_connections 2024-05-20T18:36:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.842+0000 7f17e5a84740 1 -- 172.21.15.5:0/3582534384 wait complete. 2024-05-20T18:36:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.843+0000 7f17e5a84740 1 -- start start 2024-05-20T18:36:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.843+0000 7f17e5a84740 1 Processor -- start 2024-05-20T18:36:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.845+0000 7f17e5a84740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd36704d0 0x564fd3a797a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.845+0000 7f17e5a84740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564fd36786b0 0x564fd3a77a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.845+0000 7f17e5a84740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fd3699390 0x564fd3a77fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.845+0000 7f17e5a84740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564fd3958b90 con 0x564fd36704d0 2024-05-20T18:36:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.845+0000 7f17e5a84740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564fd3a8e400 con 0x564fd3699390 2024-05-20T18:36:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.845+0000 7f17e5a84740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564fd3a7ec90 con 0x564fd36786b0 2024-05-20T18:36:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.845+0000 7f17e5a84740 10 client.0 _ll_register_callbacks cb 0x564fd3994da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:36:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.845+0000 7f17dc971640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd36704d0 0x564fd3a797a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.845+0000 7f17dc971640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd36704d0 0x564fd3a797a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57634/0 (socket says 172.21.15.5:57634) 2024-05-20T18:36:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.845+0000 7f17dc971640 1 -- 172.21.15.5:0/513347574 learned_addr learned my addr 172.21.15.5:0/513347574 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.845+0000 7f17d7fff640 1 --2- 172.21.15.5:0/513347574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564fd36786b0 0x564fd3a77a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.845+0000 7f17dd172640 1 --2- 172.21.15.5:0/513347574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fd3699390 0x564fd3a77fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.845+0000 7f17dc971640 1 -- 172.21.15.5:0/513347574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564fd36786b0 msgr2=0x564fd3a77a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.845+0000 7f17dc971640 1 --2- 172.21.15.5:0/513347574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564fd36786b0 0x564fd3a77a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.845+0000 7f17dc971640 1 -- 172.21.15.5:0/513347574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fd3699390 msgr2=0x564fd3a77fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.845+0000 7f17dc971640 1 --2- 172.21.15.5:0/513347574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fd3699390 0x564fd3a77fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.845+0000 7f17dc971640 1 -- 172.21.15.5:0/513347574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fd36b13f0 con 0x564fd36704d0 2024-05-20T18:36:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.846+0000 7f17dc971640 1 --2- 172.21.15.5:0/513347574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd36704d0 0x564fd3a797a0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x564fd3a35560 tx=0x7f17c8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab82434abc195026 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.846+0000 7f17b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:36:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.846+0000 7f17b77fe640 1 -- 172.21.15.5:0/513347574 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c800e8f0 con 0x564fd36704d0 2024-05-20T18:36:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.846+0000 7f17b77fe640 1 -- 172.21.15.5:0/513347574 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 6159+0+0 (secure 0 0 0) 0x7f17c800ea90 con 0x564fd36704d0 2024-05-20T18:36:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.846+0000 7f17e5a84740 1 -- 172.21.15.5:0/513347574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17c800e740 con 0x564fd36704d0 2024-05-20T18:36:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.846+0000 7f17b77fe640 1 -- 172.21.15.5:0/513347574 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f17c8033050 con 0x564fd36704d0 2024-05-20T18:36:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.846+0000 7f17e5a84740 10 client.8685 fetch_fsmap learned FSMap version 82 2024-05-20T18:36:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.846+0000 7f17e5a84740 1 -- 172.21.15.5:0/513347574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x564fd36710f0 con 0x564fd36704d0 2024-05-20T18:36:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.846+0000 7f17b77fe640 1 -- 172.21.15.5:0/513347574 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17c800e740 con 0x564fd36704d0 2024-05-20T18:36:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.846+0000 7f17b77fe640 1 -- 172.21.15.5:0/513347574 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f17c8036020 con 0x564fd36704d0 2024-05-20T18:36:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.846+0000 7f17e5a84740 10 client.8685 fetch_fsmap finished waiting for FSMap version 82 2024-05-20T18:36:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.846+0000 7f17e5a84740 10 client.8685 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-05-20T18:36:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.846+0000 7f17e5a84740 20 client.8685 populate_metadata read hostname 'smithi005' 2024-05-20T18:36:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.846+0000 7f17e5a84740 1 --2- 172.21.15.5:0/513347574 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x564fd3657f60 0x564fd3a66ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.846+0000 7f17e5a84740 4 client.8685 mds_command: new command op to 8343 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:36:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.846+0000 7f17e5a84740 1 -- 172.21.15.5:0/513347574 --> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fd39ab430 con 0x564fd3657f60 2024-05-20T18:36:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.847+0000 7f17d7fff640 1 --2- 172.21.15.5:0/513347574 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x564fd3657f60 0x564fd3a66ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.847+0000 7f17d7fff640 1 --2- 172.21.15.5:0/513347574 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x564fd3657f60 0x564fd3a66ef0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b339fdf07bb83e0d server_cookie=23a040d44d7fe11f in_seq=0 out_seq=0 2024-05-20T18:36:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.847+0000 7f17b77fe640 10 client.8685 ms_handle_connect on v2:172.21.15.63:6833/2054897241 2024-05-20T18:36:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.848+0000 7f17b77fe640 1 -- 172.21.15.5:0/513347574 <== mds.0 v2:172.21.15.63:6833/2054897241 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564fd39ab430 con 0x564fd3657f60 2024-05-20T18:36:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.848+0000 7f17b77fe640 10 client.8685 handle_command_reply: tid=0 2024-05-20T18:36:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.848+0000 7f17e5a84740 1 client.8685 shutdown 2024-05-20T18:36:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.848+0000 7f17e5a84740 1 -- 172.21.15.5:0/513347574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd36704d0 msgr2=0x564fd3a797a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.848+0000 7f17e5a84740 1 --2- 172.21.15.5:0/513347574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd36704d0 0x564fd3a797a0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x564fd3a35560 tx=0x7f17c8000bc0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.848+0000 7f17e5a84740 1 -- 172.21.15.5:0/513347574 shutdown_connections 2024-05-20T18:36:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.848+0000 7f17e5a84740 1 --2- 172.21.15.5:0/513347574 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x564fd3657f60 0x564fd3a66ef0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.848+0000 7f17e5a84740 1 --2- 172.21.15.5:0/513347574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fd3699390 0x564fd3a77fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.848+0000 7f17e5a84740 1 --2- 172.21.15.5:0/513347574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564fd36786b0 0x564fd3a77a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.848+0000 7f17b77fe640 0 client.8685 ms_handle_reset on v2:172.21.15.63:6833/2054897241 2024-05-20T18:36:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.848+0000 7f17e5a84740 1 --2- 172.21.15.5:0/513347574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd36704d0 0x564fd3a797a0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.848+0000 7f17e5a84740 1 -- 172.21.15.5:0/513347574 >> 172.21.15.5:0/513347574 conn(0x564fd368d4e0 msgr2=0x564fd3a391c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.848+0000 7f17e5a84740 1 -- 172.21.15.5:0/513347574 shutdown_connections 2024-05-20T18:36:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.848+0000 7f17e5a84740 1 -- 172.21.15.5:0/513347574 wait complete. 2024-05-20T18:36:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.849+0000 7f17e5a84740 20 client.8685 trim_cache size 0 max 16384 2024-05-20T18:36:20.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.869+0000 7f17e5a84740 1 Processor -- start 2024-05-20T18:36:20.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.869+0000 7f17e5a84740 1 -- start start 2024-05-20T18:36:20.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.870+0000 7f17e5a84740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fd36704d0 0x564fd3a64110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:20.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.870+0000 7f17e5a84740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564fd36786b0 0x564fd3a76670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:20.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.870+0000 7f17e5a84740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd3699390 0x564fd3a76bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:20.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.870+0000 7f17e5a84740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f17d8200260 con 0x564fd3699390 2024-05-20T18:36:20.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.870+0000 7f17e5a84740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f17d80ca080 con 0x564fd36704d0 2024-05-20T18:36:20.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.870+0000 7f17e5a84740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f17d81f4300 con 0x564fd36786b0 2024-05-20T18:36:20.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.870+0000 7f17dd172640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd3699390 0x564fd3a76bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:20.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.870+0000 7f17dd172640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd3699390 0x564fd3a76bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57644/0 (socket says 172.21.15.5:57644) 2024-05-20T18:36:20.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.870+0000 7f17dd172640 1 -- 172.21.15.5:0/220865369 learned_addr learned my addr 172.21.15.5:0/220865369 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:20.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.870+0000 7f17d7fff640 1 --2- 172.21.15.5:0/220865369 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564fd36786b0 0x564fd3a76670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:20.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.870+0000 7f17dc971640 1 --2- 172.21.15.5:0/220865369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fd36704d0 0x564fd3a64110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:20.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.870+0000 7f17dd172640 1 -- 172.21.15.5:0/220865369 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564fd36786b0 msgr2=0x564fd3a76670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:20.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.870+0000 7f17dd172640 1 --2- 172.21.15.5:0/220865369 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564fd36786b0 0x564fd3a76670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.870+0000 7f17dd172640 1 -- 172.21.15.5:0/220865369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fd36704d0 msgr2=0x564fd3a64110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:20.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.870+0000 7f17dd172640 1 --2- 172.21.15.5:0/220865369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fd36704d0 0x564fd3a64110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.871+0000 7f17dd172640 1 -- 172.21.15.5:0/220865369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fd3a64650 con 0x564fd3699390 2024-05-20T18:36:20.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.871+0000 7f17dd172640 1 --2- 172.21.15.5:0/220865369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd3699390 0x564fd3a76bb0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f17d002ad20 tx=0x7f17d002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c84704303fa727 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:20.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.871+0000 7f17cdffb640 1 -- 172.21.15.5:0/220865369 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d0036040 con 0x564fd3699390 2024-05-20T18:36:20.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.871+0000 7f17cdffb640 1 -- 172.21.15.5:0/220865369 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17d0037040 con 0x564fd3699390 2024-05-20T18:36:20.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.871+0000 7f17cdffb640 1 -- 172.21.15.5:0/220865369 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d0031a20 con 0x564fd3699390 2024-05-20T18:36:20.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.872+0000 7f17e5a84740 1 -- 172.21.15.5:0/220865369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd3699390 msgr2=0x564fd3a76bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:20.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.872+0000 7f17e5a84740 1 --2- 172.21.15.5:0/220865369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd3699390 0x564fd3a76bb0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f17d002ad20 tx=0x7f17d002e000 comp rx=0 tx=0).stop 2024-05-20T18:36:20.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.872+0000 7f17e5a84740 1 -- 172.21.15.5:0/220865369 shutdown_connections 2024-05-20T18:36:20.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.872+0000 7f17e5a84740 1 --2- 172.21.15.5:0/220865369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd3699390 0x564fd3a76bb0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.872+0000 7f17e5a84740 1 --2- 172.21.15.5:0/220865369 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564fd36786b0 0x564fd3a76670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.872+0000 7f17e5a84740 1 --2- 172.21.15.5:0/220865369 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fd36704d0 0x564fd3a64110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.872+0000 7f17e5a84740 1 -- 172.21.15.5:0/220865369 >> 172.21.15.5:0/220865369 conn(0x564fd368d4e0 msgr2=0x564fd3a39a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:20.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.872+0000 7f17e5a84740 1 -- 172.21.15.5:0/220865369 shutdown_connections 2024-05-20T18:36:20.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.872+0000 7f17e5a84740 1 -- 172.21.15.5:0/220865369 wait complete. 2024-05-20T18:36:20.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.873+0000 7f17e5a84740 1 -- start start 2024-05-20T18:36:20.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.873+0000 7f17e5a84740 1 Processor -- start 2024-05-20T18:36:20.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.874+0000 7f17e5a84740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fd36704d0 0x564fd3a76650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:20.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.874+0000 7f17e5a84740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564fd36786b0 0x564fd3a76b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.874+0000 7f17e5a84740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd3699390 0x564fd3afb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.874+0000 7f17e5a84740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f17d80c9a60 con 0x564fd3699390 2024-05-20T18:36:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.874+0000 7f17e5a84740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564fd3a35c90 con 0x564fd36704d0 2024-05-20T18:36:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.874+0000 7f17e5a84740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564fd3aa0610 con 0x564fd36786b0 2024-05-20T18:36:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.874+0000 7f17e5a84740 10 client.0 _ll_register_callbacks cb 0x7f17c8005170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:36:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.874+0000 7f17dd172640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd3699390 0x564fd3afb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.874+0000 7f17dd172640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd3699390 0x564fd3afb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57656/0 (socket says 172.21.15.5:57656) 2024-05-20T18:36:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.874+0000 7f17dd172640 1 -- 172.21.15.5:0/3707620639 learned_addr learned my addr 172.21.15.5:0/3707620639 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.875+0000 7f17dc971640 1 --2- 172.21.15.5:0/3707620639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fd36704d0 0x564fd3a76650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.875+0000 7f17d7fff640 1 --2- 172.21.15.5:0/3707620639 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564fd36786b0 0x564fd3a76b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.875+0000 7f17dd172640 1 -- 172.21.15.5:0/3707620639 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564fd36786b0 msgr2=0x564fd3a76b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.875+0000 7f17dd172640 1 --2- 172.21.15.5:0/3707620639 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564fd36786b0 0x564fd3a76b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.875+0000 7f17dd172640 1 -- 172.21.15.5:0/3707620639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fd36704d0 msgr2=0x564fd3a76650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.875+0000 7f17dd172640 1 --2- 172.21.15.5:0/3707620639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fd36704d0 0x564fd3a76650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.875+0000 7f17dd172640 1 -- 172.21.15.5:0/3707620639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fd3aa0140 con 0x564fd3699390 2024-05-20T18:36:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.875+0000 7f17d7fff640 1 --2- 172.21.15.5:0/3707620639 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564fd36786b0 0x564fd3a76b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:36:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.875+0000 7f17dd172640 1 --2- 172.21.15.5:0/3707620639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd3699390 0x564fd3afb270 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f17d00093e0 tx=0x7f17d0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e333abdf1ea1fc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.875+0000 7f17b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:36:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.875+0000 7f17b77fe640 1 -- 172.21.15.5:0/3707620639 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d0054060 con 0x564fd3699390 2024-05-20T18:36:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.875+0000 7f17b77fe640 1 -- 172.21.15.5:0/3707620639 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 6159+0+0 (secure 0 0 0) 0x7f17d00319d0 con 0x564fd3699390 2024-05-20T18:36:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.875+0000 7f17e5a84740 1 -- 172.21.15.5:0/3707620639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17d0037040 con 0x564fd3699390 2024-05-20T18:36:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.875+0000 7f17b77fe640 1 -- 172.21.15.5:0/3707620639 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f17d0036040 con 0x564fd3699390 2024-05-20T18:36:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.875+0000 7f17e5a84740 10 client.8691 fetch_fsmap learned FSMap version 82 2024-05-20T18:36:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.875+0000 7f17e5a84740 1 -- 172.21.15.5:0/3707620639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x564fd3aff0c0 con 0x564fd3699390 2024-05-20T18:36:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.875+0000 7f17b77fe640 1 -- 172.21.15.5:0/3707620639 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17d0037040 con 0x564fd3699390 2024-05-20T18:36:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.876+0000 7f17b77fe640 1 -- 172.21.15.5:0/3707620639 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 82) ==== 3494+0+0 (secure 0 0 0) 0x7f17d0031bd0 con 0x564fd3699390 2024-05-20T18:36:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.876+0000 7f17e5a84740 10 client.8691 fetch_fsmap finished waiting for FSMap version 82 2024-05-20T18:36:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.876+0000 7f17e5a84740 10 client.8691 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-05-20T18:36:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.876+0000 7f17e5a84740 20 client.8691 populate_metadata read hostname 'smithi005' 2024-05-20T18:36:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.876+0000 7f17e5a84740 1 --2- 172.21.15.5:0/3707620639 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x564fd3a74de0 0x564fd3afb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.876+0000 7f17e5a84740 4 client.8691 mds_command: new command op to 8343 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T18:36:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.876+0000 7f17e5a84740 1 -- 172.21.15.5:0/3707620639 --> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564fd39a5780 con 0x564fd3a74de0 2024-05-20T18:36:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.876+0000 7f17dc971640 1 --2- 172.21.15.5:0/3707620639 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x564fd3a74de0 0x564fd3afb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.876+0000 7f17b77fe640 10 client.8691 ms_handle_connect on v2:172.21.15.63:6833/2054897241 2024-05-20T18:36:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.876+0000 7f17dc971640 1 --2- 172.21.15.5:0/3707620639 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x564fd3a74de0 0x564fd3afb7b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8437405e7e1f9a5 server_cookie=b194605fc237bf18 in_seq=0 out_seq=0 2024-05-20T18:36:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.877+0000 7f17b77fe640 1 -- 172.21.15.5:0/3707620639 <== mds.0 v2:172.21.15.63:6833/2054897241 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564fd39a5780 con 0x564fd3a74de0 2024-05-20T18:36:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.877+0000 7f17b77fe640 10 client.8691 handle_command_reply: tid=0 2024-05-20T18:36:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.877+0000 7f17e5a84740 1 client.8691 shutdown 2024-05-20T18:36:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.877+0000 7f17e5a84740 1 -- 172.21.15.5:0/3707620639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd3699390 msgr2=0x564fd3afb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.877+0000 7f17e5a84740 1 --2- 172.21.15.5:0/3707620639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd3699390 0x564fd3afb270 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f17d00093e0 tx=0x7f17d0004ab0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.877+0000 7f17e5a84740 1 -- 172.21.15.5:0/3707620639 shutdown_connections 2024-05-20T18:36:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.877+0000 7f17e5a84740 1 --2- 172.21.15.5:0/3707620639 >> [v2:172.21.15.63:6833/2054897241,v1:172.21.15.63:6835/2054897241] conn(0x564fd3a74de0 0x564fd3afb7b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.877+0000 7f17e5a84740 1 --2- 172.21.15.5:0/3707620639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd3699390 0x564fd3afb270 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.877+0000 7f17e5a84740 1 --2- 172.21.15.5:0/3707620639 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564fd36786b0 0x564fd3a76b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.877+0000 7f17b77fe640 0 client.8691 ms_handle_reset on v2:172.21.15.63:6833/2054897241 2024-05-20T18:36:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.877+0000 7f17e5a84740 1 --2- 172.21.15.5:0/3707620639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564fd36704d0 0x564fd3a76650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.877+0000 7f17e5a84740 1 -- 172.21.15.5:0/3707620639 >> 172.21.15.5:0/3707620639 conn(0x564fd368d4e0 msgr2=0x564fd3a77c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.877+0000 7f17e5a84740 1 -- 172.21.15.5:0/3707620639 shutdown_connections 2024-05-20T18:36:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.877+0000 7f17e5a84740 1 -- 172.21.15.5:0/3707620639 wait complete. 2024-05-20T18:36:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.877+0000 7f17e5a84740 20 client.8691 trim_cache size 0 max 16384 2024-05-20T18:36:20.884 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:36:20.884 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-20T18:36:20.884 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-20T18:36:20.884 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:36:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.878+0000 7f17debd5640 1 -- 172.21.15.5:0/2129416926 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f17b806be60 msgr2=0x7f17b806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.878+0000 7f17debd5640 1 --2- 172.21.15.5:0/2129416926 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f17b806be60 0x7f17b806e370 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f17d80d29a0 tx=0x7f17c8009040 comp rx=0 tx=0).stop 2024-05-20T18:36:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.878+0000 7f17debd5640 1 -- 172.21.15.5:0/2129416926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d80b1e10 msgr2=0x7f17d815f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.878+0000 7f17debd5640 1 --2- 172.21.15.5:0/2129416926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d80b1e10 0x7f17d815f8a0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f17c000d7d0 tx=0x7f17c000dc90 comp rx=0 tx=0).stop 2024-05-20T18:36:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.878+0000 7f17debd5640 1 -- 172.21.15.5:0/2129416926 shutdown_connections 2024-05-20T18:36:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.878+0000 7f17debd5640 1 --2- 172.21.15.5:0/2129416926 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f17b806be60 0x7f17b806e370 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.878+0000 7f17debd5640 1 --2- 172.21.15.5:0/2129416926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17d815d530 0x7f17d81555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.878+0000 7f17debd5640 1 --2- 172.21.15.5:0/2129416926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d80b1e10 0x7f17d815f8a0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.878+0000 7f17debd5640 1 --2- 172.21.15.5:0/2129416926 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17d8070850 0x7f17d80c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.878+0000 7f17debd5640 1 -- 172.21.15.5:0/2129416926 >> 172.21.15.5:0/2129416926 conn(0x7f17d80d29d0 msgr2=0x7f17d80d1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.878+0000 7f17debd5640 1 -- 172.21.15.5:0/2129416926 shutdown_connections 2024-05-20T18:36:20.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:20.878+0000 7f17debd5640 1 -- 172.21.15.5:0/2129416926 wait complete. 2024-05-20T18:36:20.890 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T18:36:20.992 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-20T18:36:21.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.106+0000 7f5c5a873640 1 Processor -- start 2024-05-20T18:36:21.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.107+0000 7f5c5a873640 1 -- start start 2024-05-20T18:36:21.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.108+0000 7f5c5a873640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c54168050 0x7f5c5416a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.108+0000 7f5c5a873640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c540d0630 con 0x7f5c54162820 2024-05-20T18:36:21.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.108+0000 7f5c5a873640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c540d39c0 con 0x7f5c5406d3d0 2024-05-20T18:36:21.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.108+0000 7f5c5a873640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5c540d3ca0 con 0x7f5c54168050 2024-05-20T18:36:21.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.108+0000 7f5c53fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5c54162820 0x7f5c540b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50586/0 (socket says 172.21.15.5:50586) 2024-05-20T18:36:21.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.108+0000 7f5c53fff640 1 -- 172.21.15.5:0/2754134980 learned_addr learned my addr 172.21.15.5:0/2754134980 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:36:21.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.108+0000 7f5c58e10640 1 --2- 172.21.15.5:0/2754134980 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c54168050 0x7f5c5416a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.108+0000 7f5c52ffd640 1 -- 172.21.15.5:0/2754134980 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 775080620 0 0) 0x7f5c540d0630 con 0x7f5c54162820 2024-05-20T18:36:21.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.108+0000 7f5c52ffd640 1 -- 172.21.15.5:0/2754134980 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c3c003520 con 0x7f5c54162820 2024-05-20T18:36:21.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.109+0000 7f5c52ffd640 1 -- 172.21.15.5:0/2754134980 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2125847614 0 0) 0x7f5c3c003520 con 0x7f5c54162820 2024-05-20T18:36:21.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.109+0000 7f5c52ffd640 1 -- 172.21.15.5:0/2754134980 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c54168050 msgr2=0x7f5c5416a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:21.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.109+0000 7f5c52ffd640 1 --2- 172.21.15.5:0/2754134980 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c54168050 0x7f5c5416a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.109+0000 7f5c52ffd640 1 -- 172.21.15.5:0/2754134980 >> v1:172.21.15.63:6789/0 conn(0x7f5c5406d3d0 legacy=0x7f5c5406d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.109+0000 7f5c52ffd640 1 -- 172.21.15.5:0/2754134980 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c540d3f80 con 0x7f5c54162820 2024-05-20T18:36:21.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.109+0000 7f5c52ffd640 1 -- 172.21.15.5:0/2754134980 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f5c44001670 con 0x7f5c54162820 2024-05-20T18:36:21.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.109+0000 7f5c52ffd640 1 -- 172.21.15.5:0/2754134980 >> v1:172.21.15.5:6789/0 conn(0x7f5c54162820 legacy=0x7f5c540b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.109+0000 7f5c52ffd640 1 --2- 172.21.15.5:0/2754134980 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c54168050 0x7f5c3c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.109+0000 7f5c52ffd640 1 --2- 172.21.15.5:0/2754134980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c3c003f40 0x7f5c3c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.109+0000 7f5c52ffd640 1 --2- 172.21.15.5:0/2754134980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c3c004d40 0x7f5c3c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.109+0000 7f5c52ffd640 1 -- 172.21.15.5:0/2754134980 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5c540d3ca0 con 0x7f5c3c003f40 2024-05-20T18:36:21.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.109+0000 7f5c52ffd640 1 -- 172.21.15.5:0/2754134980 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c00bad0 con 0x7f5c3c004d40 2024-05-20T18:36:21.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.109+0000 7f5c52ffd640 1 -- 172.21.15.5:0/2754134980 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5c3c00bcd0 con 0x7f5c54168050 2024-05-20T18:36:21.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.109+0000 7f5c537fe640 1 --2- 172.21.15.5:0/2754134980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c3c003f40 0x7f5c3c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.109+0000 7f5c53fff640 1 --2- 172.21.15.5:0/2754134980 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c54168050 0x7f5c3c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.109+0000 7f5c58e10640 1 --2- 172.21.15.5:0/2754134980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c3c004d40 0x7f5c3c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.109+0000 7f5c537fe640 1 -- 172.21.15.5:0/2754134980 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c54168050 msgr2=0x7f5c3c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:21.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.109+0000 7f5c537fe640 1 --2- 172.21.15.5:0/2754134980 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c54168050 0x7f5c3c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.109+0000 7f5c537fe640 1 -- 172.21.15.5:0/2754134980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c3c004d40 msgr2=0x7f5c3c005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:21.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.109+0000 7f5c537fe640 1 --2- 172.21.15.5:0/2754134980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c3c004d40 0x7f5c3c005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.109+0000 7f5c537fe640 1 -- 172.21.15.5:0/2754134980 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c3c003520 con 0x7f5c3c003f40 2024-05-20T18:36:21.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.110+0000 7f5c537fe640 1 --2- 172.21.15.5:0/2754134980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c3c003f40 0x7f5c3c004800 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f5c48001860 tx=0x7f5c3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca43e900b7dffb9b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:21.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.110+0000 7f5c52ffd640 1 -- 172.21.15.5:0/2754134980 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48037070 con 0x7f5c3c003f40 2024-05-20T18:36:21.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.110+0000 7f5c52ffd640 1 -- 172.21.15.5:0/2754134980 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c4802abf0 con 0x7f5c3c003f40 2024-05-20T18:36:21.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.110+0000 7f5c52ffd640 1 -- 172.21.15.5:0/2754134980 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48001430 con 0x7f5c3c003f40 2024-05-20T18:36:21.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.111+0000 7f5c5a873640 1 -- 172.21.15.5:0/2754134980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c3c003f40 msgr2=0x7f5c3c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.111+0000 7f5c5a873640 1 --2- 172.21.15.5:0/2754134980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c3c003f40 0x7f5c3c004800 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f5c48001860 tx=0x7f5c3c005830 comp rx=0 tx=0).stop 2024-05-20T18:36:21.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.111+0000 7f5c58e10640 1 -- 172.21.15.5:0/2754134980 reap_dead start 2024-05-20T18:36:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.111+0000 7f5c5a873640 1 -- 172.21.15.5:0/2754134980 shutdown_connections 2024-05-20T18:36:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.111+0000 7f5c5a873640 1 -- 172.21.15.5:0/2754134980 >> 172.21.15.5:0/2754134980 conn(0x7f5c540d4bd0 msgr2=0x7f5c540d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.111+0000 7f5c5a873640 1 -- 172.21.15.5:0/2754134980 shutdown_connections 2024-05-20T18:36:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.111+0000 7f5c5a873640 1 -- 172.21.15.5:0/2754134980 wait complete. 2024-05-20T18:36:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.113+0000 7f5c5a873640 1 Processor -- start 2024-05-20T18:36:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.113+0000 7f5c5a873640 1 -- start start 2024-05-20T18:36:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.114+0000 7f5c5a873640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c3c003f40 0x7f5c540b5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.114+0000 7f5c5a873640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c540b5bb0 0x7f5c5415e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.114+0000 7f5c5a873640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c54154790 0x7f5c5415e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.114+0000 7f5c5a873640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5c540b91a0 con 0x7f5c540b5bb0 2024-05-20T18:36:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.114+0000 7f5c5a873640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5c540d0300 con 0x7f5c54154790 2024-05-20T18:36:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.114+0000 7f5c5a873640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5c540b4050 con 0x7f5c3c003f40 2024-05-20T18:36:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.114+0000 7f5c537fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c540b5bb0 0x7f5c5415e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.114+0000 7f5c537fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c540b5bb0 0x7f5c5415e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57680/0 (socket says 172.21.15.5:57680) 2024-05-20T18:36:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.114+0000 7f5c537fe640 1 -- 172.21.15.5:0/1331123864 learned_addr learned my addr 172.21.15.5:0/1331123864 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.115+0000 7f5c53fff640 1 --2- 172.21.15.5:0/1331123864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c3c003f40 0x7f5c540b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.115+0000 7f5c537fe640 1 -- 172.21.15.5:0/1331123864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c3c003f40 msgr2=0x7f5c540b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.115+0000 7f5c537fe640 1 --2- 172.21.15.5:0/1331123864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c3c003f40 0x7f5c540b5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.115+0000 7f5c537fe640 1 -- 172.21.15.5:0/1331123864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c54154790 msgr2=0x7f5c5415e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.115+0000 7f5c58e10640 1 --2- 172.21.15.5:0/1331123864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c54154790 0x7f5c5415e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.115+0000 7f5c537fe640 1 --2- 172.21.15.5:0/1331123864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c54154790 0x7f5c5415e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.115+0000 7f5c537fe640 1 -- 172.21.15.5:0/1331123864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c54155000 con 0x7f5c540b5bb0 2024-05-20T18:36:21.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.115+0000 7f5c537fe640 1 --2- 172.21.15.5:0/1331123864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c540b5bb0 0x7f5c5415e200 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f5c48001980 tx=0x7f5c48001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:21.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.115+0000 7f5c50ff9640 1 -- 172.21.15.5:0/1331123864 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4803f020 con 0x7f5c540b5bb0 2024-05-20T18:36:21.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.115+0000 7f5c50ff9640 1 -- 172.21.15.5:0/1331123864 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c48005270 con 0x7f5c540b5bb0 2024-05-20T18:36:21.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.115+0000 7f5c50ff9640 1 -- 172.21.15.5:0/1331123864 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48037040 con 0x7f5c540b5bb0 2024-05-20T18:36:21.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.115+0000 7f5c5a873640 1 -- 172.21.15.5:0/1331123864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c54160740 con 0x7f5c540b5bb0 2024-05-20T18:36:21.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.115+0000 7f5c5a873640 1 -- 172.21.15.5:0/1331123864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c540cd040 con 0x7f5c540b5bb0 2024-05-20T18:36:21.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.116+0000 7f5c5a873640 1 -- 172.21.15.5:0/1331123864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c540ccb00 con 0x7f5c540b5bb0 2024-05-20T18:36:21.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.117+0000 7f5c50ff9640 1 -- 172.21.15.5:0/1331123864 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5c48042020 con 0x7f5c540b5bb0 2024-05-20T18:36:21.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.117+0000 7f5c50ff9640 1 --2- 172.21.15.5:0/1331123864 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5c4006be30 0x7f5c4006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.117+0000 7f5c50ff9640 1 -- 172.21.15.5:0/1331123864 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6159+0+0 (secure 0 0 0) 0x7f5c480aafa0 con 0x7f5c540b5bb0 2024-05-20T18:36:21.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.117+0000 7f5c53fff640 1 --2- 172.21.15.5:0/1331123864 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5c4006be30 0x7f5c4006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.118+0000 7f5c53fff640 1 --2- 172.21.15.5:0/1331123864 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5c4006be30 0x7f5c4006e340 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f5c44001770 tx=0x7f5c44009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:21.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.122+0000 7f5c50ff9640 1 -- 172.21.15.5:0/1331123864 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5c48049030 con 0x7f5c540b5bb0 2024-05-20T18:36:21.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.344+0000 7f5c5a873640 1 -- 172.21.15.5:0/1331123864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5c54075b00 con 0x7f5c540b5bb0 2024-05-20T18:36:21.346 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-20T18:36:21.345+0000 7fd1cf53a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-20T18:36:21.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.346+0000 7f5c50ff9640 1 -- 172.21.15.5:0/1331123864 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v83) ==== 161+0+0 (secure 0 0 0) 0x7f5c48079660 con 0x7f5c540b5bb0 2024-05-20T18:36:21.348 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-20T18:36:21.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.350+0000 7f5c5a873640 1 -- 172.21.15.5:0/1331123864 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5c4006be30 msgr2=0x7f5c4006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.350+0000 7f5c5a873640 1 --2- 172.21.15.5:0/1331123864 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5c4006be30 0x7f5c4006e340 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f5c44001770 tx=0x7f5c44009040 comp rx=0 tx=0).stop 2024-05-20T18:36:21.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.350+0000 7f5c5a873640 1 -- 172.21.15.5:0/1331123864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c540b5bb0 msgr2=0x7f5c5415e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.350+0000 7f5c5a873640 1 --2- 172.21.15.5:0/1331123864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c540b5bb0 0x7f5c5415e200 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f5c48001980 tx=0x7f5c48001230 comp rx=0 tx=0).stop 2024-05-20T18:36:21.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.350+0000 7f5c5a873640 1 -- 172.21.15.5:0/1331123864 shutdown_connections 2024-05-20T18:36:21.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.350+0000 7f5c5a873640 1 --2- 172.21.15.5:0/1331123864 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5c4006be30 0x7f5c4006e340 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.350+0000 7f5c5a873640 1 --2- 172.21.15.5:0/1331123864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c54154790 0x7f5c5415e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.350+0000 7f5c5a873640 1 --2- 172.21.15.5:0/1331123864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c540b5bb0 0x7f5c5415e200 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.350+0000 7f5c5a873640 1 --2- 172.21.15.5:0/1331123864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c3c003f40 0x7f5c540b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.350+0000 7f5c5a873640 1 -- 172.21.15.5:0/1331123864 >> 172.21.15.5:0/1331123864 conn(0x7f5c540d4bd0 msgr2=0x7f5c54161ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:21.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.350+0000 7f5c5a873640 1 -- 172.21.15.5:0/1331123864 shutdown_connections 2024-05-20T18:36:21.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.350+0000 7f5c5a873640 1 -- 172.21.15.5:0/1331123864 wait complete. 2024-05-20T18:36:21.354 INFO:tasks.ceph.mds.b.smithi063.stderr: -8507> 2024-05-20T18:35:26.459+0000 7f458268e640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-05-20T18:36:21.356 INFO:tasks.ceph.mds.b.smithi063.stderr: -7368> 2024-05-20T18:35:33.383+0000 7f457c682640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T18:36:21.363 DEBUG:teuthology.orchestra.run.smithi005:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2024-05-20T18:36:21.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.417+0000 7f1ef0d23280 1 -- start start 2024-05-20T18:36:21.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.417+0000 7f1ef0d23280 1 Processor -- start 2024-05-20T18:36:21.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.419+0000 7f1ef0d23280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d40fdc1680 0x55d40fea2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.419+0000 7f1ef0d23280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d40fdc1b00 0x55d40fea2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.419+0000 7f1ef0d23280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d40fdc2400 0x55d40fea3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.419+0000 7f1ef0d23280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d410afc180 con 0x55d40fdc2400 2024-05-20T18:36:21.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.419+0000 7f1ef0d23280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d410afc300 con 0x55d40fdc1b00 2024-05-20T18:36:21.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.419+0000 7f1ef0d23280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55d410afc480 con 0x55d40fdc1680 2024-05-20T18:36:21.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.419+0000 7f1eef3f3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d40fdc2400 0x55d40fea3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.419+0000 7f1eef3f3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d40fdc2400 0x55d40fea3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57698/0 (socket says 172.21.15.5:57698) 2024-05-20T18:36:21.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.419+0000 7f1eef3f3640 1 -- 172.21.15.5:0/3934823614 learned_addr learned my addr 172.21.15.5:0/3934823614 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:21.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.419+0000 7f1eee3f1640 1 --2- 172.21.15.5:0/3934823614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d40fdc1b00 0x55d40fea2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.419+0000 7f1eeebf2640 1 --2- 172.21.15.5:0/3934823614 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d40fdc1680 0x55d40fea2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.419+0000 7f1eef3f3640 1 -- 172.21.15.5:0/3934823614 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d40fdc1680 msgr2=0x55d40fea2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:21.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.419+0000 7f1eef3f3640 1 --2- 172.21.15.5:0/3934823614 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d40fdc1680 0x55d40fea2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.419+0000 7f1eef3f3640 1 -- 172.21.15.5:0/3934823614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d40fdc1b00 msgr2=0x55d40fea2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:21.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.419+0000 7f1eef3f3640 1 --2- 172.21.15.5:0/3934823614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d40fdc1b00 0x55d40fea2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.419+0000 7f1eef3f3640 1 -- 172.21.15.5:0/3934823614 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d40bd1e780 con 0x55d40fdc2400 2024-05-20T18:36:21.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.419+0000 7f1eef3f3640 1 --2- 172.21.15.5:0/3934823614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d40fdc2400 0x55d40fea3080 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x55d410b3b410 tx=0x55d40fe102a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5903cf2eae20527e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:21.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.419+0000 7f1eec3ed640 1 -- 172.21.15.5:0/3934823614 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d40bd0e680 con 0x55d40fdc2400 2024-05-20T18:36:21.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.420+0000 7f1eec3ed640 1 -- 172.21.15.5:0/3934823614 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55d40bd14fc0 con 0x55d40fdc2400 2024-05-20T18:36:21.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.420+0000 7f1eec3ed640 1 -- 172.21.15.5:0/3934823614 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d40bd0eb60 con 0x55d40fdc2400 2024-05-20T18:36:21.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.420+0000 7f1ef0d23280 1 -- 172.21.15.5:0/3934823614 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d40bd1f4a0 con 0x55d40fdc2400 2024-05-20T18:36:21.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.420+0000 7f1ef0d23280 1 -- 172.21.15.5:0/3934823614 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55d40bd1e000 con 0x55d40fdc2400 2024-05-20T18:36:21.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.420+0000 7f1ef0d23280 4 waiting for MDS map... 2024-05-20T18:36:21.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.420+0000 7f1eec3ed640 1 -- 172.21.15.5:0/3934823614 <== mon.0 v2:172.21.15.5:3300/0 4 ==== osd_map(108..108 src has 1..108) ==== 6245+0+0 (secure 0 0 0) 0x55d410bfca00 con 0x55d40fdc2400 2024-05-20T18:36:21.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.420+0000 7f1eec3ed640 1 -- 172.21.15.5:0/3934823614 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 83) ==== 2885+0+0 (secure 0 0 0) 0x55d40fd73180 con 0x55d40fdc2400 2024-05-20T18:36:21.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.420+0000 7f1ef0d23280 4 Got MDS map 83 2024-05-20T18:36:21.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.420+0000 7f1ef0d23280 10 main: JournalTool::main 2024-05-20T18:36:21.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.420+0000 7f1ef0d23280 4 main: JournalTool: connecting to RADOS... 2024-05-20T18:36:21.424 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-20T18:36:21.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.421+0000 7f1ef0d23280 1 Processor -- start 2024-05-20T18:36:21.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.421+0000 7f1ef0d23280 1 -- start start 2024-05-20T18:36:21.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.422+0000 7f1ef0d23280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d40fdc0d80 0x55d40fea3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.422+0000 7f1ef0d23280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d40fdc0900 0x55d40fea3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.422+0000 7f1ef0d23280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d40fdc0480 0x55d40fea4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.422+0000 7f1ef0d23280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d410afc600 con 0x55d40fdc0480 2024-05-20T18:36:21.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.422+0000 7f1ef0d23280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d410afc780 con 0x55d40fdc0d80 2024-05-20T18:36:21.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.422+0000 7f1ef0d23280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55d410afc900 con 0x55d40fdc0900 2024-05-20T18:36:21.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.422+0000 7f1eee3f1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d40fdc0480 0x55d40fea4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.422+0000 7f1eee3f1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d40fdc0480 0x55d40fea4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57700/0 (socket says 172.21.15.5:57700) 2024-05-20T18:36:21.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.422+0000 7f1eee3f1640 1 -- 172.21.15.5:0/3467485521 learned_addr learned my addr 172.21.15.5:0/3467485521 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:21.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.422+0000 7f1eeebf2640 1 --2- 172.21.15.5:0/3467485521 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d40fdc0900 0x55d40fea3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.422+0000 7f1eee3f1640 1 --2- 172.21.15.5:0/3467485521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d40fdc0d80 0x55d40fea3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.422+0000 7f1eee3f1640 1 -- 172.21.15.5:0/3467485521 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d40fdc0900 msgr2=0x55d40fea3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:21.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.422+0000 7f1eee3f1640 1 --2- 172.21.15.5:0/3467485521 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d40fdc0900 0x55d40fea3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.423+0000 7f1eee3f1640 1 -- 172.21.15.5:0/3467485521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d40fdc0d80 msgr2=0x55d40fea3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:21.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.423+0000 7f1eee3f1640 1 --2- 172.21.15.5:0/3467485521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d40fdc0d80 0x55d40fea3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.423+0000 7f1eee3f1640 1 -- 172.21.15.5:0/3467485521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d40bd1e3c0 con 0x55d40fdc0480 2024-05-20T18:36:21.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.423+0000 7f1eee3f1640 1 --2- 172.21.15.5:0/3467485521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d40fdc0480 0x55d40fea4100 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x55d410bdc6c0 tx=0x55d410bdf8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb3f759979a062b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:21.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.423+0000 7f1eea3e9640 1 -- 172.21.15.5:0/3467485521 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d40bd0e680 con 0x55d40fdc0480 2024-05-20T18:36:21.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.423+0000 7f1eea3e9640 1 -- 172.21.15.5:0/3467485521 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55d40bd14a80 con 0x55d40fdc0480 2024-05-20T18:36:21.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.423+0000 7f1eea3e9640 1 -- 172.21.15.5:0/3467485521 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d40bd0eb60 con 0x55d40fdc0480 2024-05-20T18:36:21.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.423+0000 7f1ef0d23280 1 -- 172.21.15.5:0/3467485521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d40fdc0480 msgr2=0x55d40fea4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.423+0000 7f1ef0d23280 1 --2- 172.21.15.5:0/3467485521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d40fdc0480 0x55d40fea4100 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x55d410bdc6c0 tx=0x55d410bdf8c0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.423+0000 7f1ef0d23280 1 -- 172.21.15.5:0/3467485521 shutdown_connections 2024-05-20T18:36:21.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.423+0000 7f1ef0d23280 1 --2- 172.21.15.5:0/3467485521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d40fdc0480 0x55d40fea4100 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.423+0000 7f1ef0d23280 1 --2- 172.21.15.5:0/3467485521 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d40fdc0900 0x55d40fea3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.423+0000 7f1ef0d23280 1 --2- 172.21.15.5:0/3467485521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d40fdc0d80 0x55d40fea3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.423+0000 7f1ef0d23280 1 -- 172.21.15.5:0/3467485521 >> 172.21.15.5:0/3467485521 conn(0x55d40fdc1200 msgr2=0x55d40fe33800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:21.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.424+0000 7f1ef0d23280 1 -- 172.21.15.5:0/3467485521 shutdown_connections 2024-05-20T18:36:21.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.424+0000 7f1ef0d23280 1 -- 172.21.15.5:0/3467485521 wait complete. 2024-05-20T18:36:21.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.424+0000 7f1ef0d23280 1 Processor -- start 2024-05-20T18:36:21.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.424+0000 7f1ef0d23280 1 -- start start 2024-05-20T18:36:21.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.425+0000 7f1ef0d23280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d40fdc0900 0x55d40fea5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.425+0000 7f1ef0d23280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d40fdc0d80 0x55d40fea4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.426+0000 7f1ef0d23280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d40fdc0000 0x55d40fea4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.426+0000 7f1ef0d23280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d410afdb00 con 0x55d40fdc0000 2024-05-20T18:36:21.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.426+0000 7f1ef0d23280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d410afd980 con 0x55d40fdc0d80 2024-05-20T18:36:21.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.426+0000 7f1ef0d23280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55d410afd800 con 0x55d40fdc0900 2024-05-20T18:36:21.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.426+0000 7f1eee3f1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d40fdc0000 0x55d40fea4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.426+0000 7f1eee3f1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d40fdc0000 0x55d40fea4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57702/0 (socket says 172.21.15.5:57702) 2024-05-20T18:36:21.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.426+0000 7f1eeebf2640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d40fdc0d80 0x55d40fea4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.426+0000 7f1eee3f1640 1 -- 172.21.15.5:0/4273834146 learned_addr learned my addr 172.21.15.5:0/4273834146 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:21.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.426+0000 7f1eee3f1640 1 --2- 172.21.15.5:0/4273834146 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d40fdc0900 0x55d40fea5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.426+0000 7f1eee3f1640 1 -- 172.21.15.5:0/4273834146 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d40fdc0900 msgr2=0x55d40fea5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.426+0000 7f1eee3f1640 1 --2- 172.21.15.5:0/4273834146 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d40fdc0900 0x55d40fea5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.426+0000 7f1eee3f1640 1 -- 172.21.15.5:0/4273834146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d40fdc0d80 msgr2=0x55d40fea4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.426+0000 7f1eee3f1640 1 --2- 172.21.15.5:0/4273834146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d40fdc0d80 0x55d40fea4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.426+0000 7f1eee3f1640 1 -- 172.21.15.5:0/4273834146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d40bd1e780 con 0x55d40fdc0000 2024-05-20T18:36:21.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.426+0000 7f1eeebf2640 1 --2- 172.21.15.5:0/4273834146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d40fdc0d80 0x55d40fea4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:36:21.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.426+0000 7f1eee3f1640 1 --2- 172.21.15.5:0/4273834146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d40fdc0000 0x55d40fea4680 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x55d410bdd980 tx=0x55d40fe10180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:21.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.426+0000 7f1ee8be6640 1 -- 172.21.15.5:0/4273834146 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d40bd0e4e0 con 0x55d40fdc0000 2024-05-20T18:36:21.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.426+0000 7f1ee8be6640 1 -- 172.21.15.5:0/4273834146 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55d40bd14a80 con 0x55d40fdc0000 2024-05-20T18:36:21.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.426+0000 7f1ee8be6640 1 -- 172.21.15.5:0/4273834146 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d40bd0e680 con 0x55d40fdc0000 2024-05-20T18:36:21.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.426+0000 7f1ef0d23280 1 -- 172.21.15.5:0/4273834146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d40bd1f4a0 con 0x55d40fdc0000 2024-05-20T18:36:21.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.426+0000 7f1ef0d23280 1 -- 172.21.15.5:0/4273834146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d40bd1e960 con 0x55d40fdc0000 2024-05-20T18:36:21.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.426+0000 7f1ef0d23280 4 main: JournalTool: resolving pool 15 2024-05-20T18:36:21.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.428+0000 7f1ee8be6640 1 -- 172.21.15.5:0/4273834146 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x55d410b12c00 con 0x55d40fdc0000 2024-05-20T18:36:21.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.428+0000 7f1ee8be6640 1 --2- 172.21.15.5:0/4273834146 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x55d40fdc1200 0x55d40fea5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.428+0000 7f1ee8be6640 1 -- 172.21.15.5:0/4273834146 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6245+0+0 (secure 0 0 0) 0x55d410b10a00 con 0x55d40fdc0000 2024-05-20T18:36:21.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.428+0000 7f1eeebf2640 1 --2- 172.21.15.5:0/4273834146 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x55d40fdc1200 0x55d40fea5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.428+0000 7f1ef0d23280 4 main: JournalTool: creating IoCtx.. 2024-05-20T18:36:21.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.428+0000 7f1ef0d23280 4 main: Executing for rank 0 2024-05-20T18:36:21.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.428+0000 7f1ef0d23280 4 Filtering by inode '1099511627777' 2024-05-20T18:36:21.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.428+0000 7f1ef0d23280 1 --2- 172.21.15.5:0/4273834146 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x55d40fdc2880 0x55d40fea3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.428+0000 7f1ef0d23280 1 -- 172.21.15.5:0/4273834146 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) -- 0x55d410b0e800 con 0x55d40fdc2880 2024-05-20T18:36:21.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.429+0000 7f1eee3f1640 1 --2- 172.21.15.5:0/4273834146 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x55d40fdc2880 0x55d40fea3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.429+0000 7f1eeebf2640 1 --2- 172.21.15.5:0/4273834146 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x55d40fdc1200 0x55d40fea5700 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x55d40bd1be00 tx=0x55d40fe0e4e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:21.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.429+0000 7f1eee3f1640 1 --2- 172.21.15.5:0/4273834146 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x55d40fdc2880 0x55d40fea3600 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:21.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.430+0000 7f1eee3f1640 1 -- 172.21.15.5:0/4273834146 <== osd.6 v2:172.21.15.63:6816/3384565140 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x55d40fe46000 con 0x55d40fdc2880 2024-05-20T18:36:21.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.430+0000 7f1ef0d23280 4 Pointer 400.00000000 is readable 2024-05-20T18:36:21.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.430+0000 7f1ef0d23280 4 JournalScanner::scan: reading header object '200.00000000' 2024-05-20T18:36:21.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.430+0000 7f1ef0d23280 1 -- 172.21.15.5:0/4273834146 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) -- 0x55d410b0ec00 con 0x55d40fdc2880 2024-05-20T18:36:21.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.430+0000 7f1eee3f1640 1 -- 172.21.15.5:0/4273834146 <== osd.6 v2:172.21.15.63:6816/3384565140 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x55d40fe46000 con 0x55d40fdc2880 2024-05-20T18:36:21.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.430+0000 7f1ef0d23280 10 Header 0x400000 0x407d21 0x40809d 2024-05-20T18:36:21.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.430+0000 7f1ef0d23280 10 Starting journal scan from offset 0x407d21 2024-05-20T18:36:21.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.430+0000 7f1ef0d23280 1 -- 172.21.15.5:0/4273834146 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 32033~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) -- 0x55d410104000 con 0x55d40fdc2880 2024-05-20T18:36:21.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.431+0000 7f1eee3f1640 1 -- 172.21.15.5:0/4273834146 <== osd.6 v2:172.21.15.63:6816/3384565140 3 ==== osd_op_reply(3 200.00000001 [read 32033~892 out=892b] v0'0 uv28 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x55d40fe46000 con 0x55d40fdc2880 2024-05-20T18:36:21.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.431+0000 7f1ef0d23280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-05-20T18:36:21.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.431+0000 7f1ef0d23280 10 Parsing data, 0x37c bytes available 2024-05-20T18:36:21.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.431+0000 7f1ef0d23280 10 Attempting decode at 0x407d21 2024-05-20T18:36:21.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.431+0000 7f1ef0d23280 10 Consumed 0x37c bytes 2024-05-20T18:36:21.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.431+0000 7f1ef0d23280 10 Valid entry at 0x407d21 2024-05-20T18:36:21.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.431+0000 7f1ef0d23280 1 --2- 172.21.15.5:0/4273834146 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x55d40fdc2d00 0x55d40fea3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.431+0000 7f1ef0d23280 1 -- 172.21.15.5:0/4273834146 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) -- 0x55d410104400 con 0x55d40fdc2d00 2024-05-20T18:36:21.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.431+0000 7f1eef3f3640 1 --2- 172.21.15.5:0/4273834146 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x55d40fdc2d00 0x55d40fea3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.431+0000 7f1eef3f3640 1 --2- 172.21.15.5:0/4273834146 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x55d40fdc2d00 0x55d40fea3b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:21.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.432+0000 7f1eef3f3640 1 -- 172.21.15.5:0/4273834146 <== osd.7 v2:172.21.15.63:6824/4136124451 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55d40fe46500 con 0x55d40fdc2d00 2024-05-20T18:36:21.437 INFO:teuthology.orchestra.run.smithi005.stdout:Events by type: 2024-05-20T18:36:21.438 INFO:teuthology.orchestra.run.smithi005.stdout:Errors: 0 2024-05-20T18:36:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.432+0000 7f1ef0d23280 4 Reached end of journal objects 2024-05-20T18:36:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.432+0000 7f1ef0d23280 4 Scanned objects, 0 missing, 1 valid 2024-05-20T18:36:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.432+0000 7f1ef0d23280 4 Events scanned, 0 gaps 2024-05-20T18:36:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.432+0000 7f1ef0d23280 4 Found 1 valid events 2024-05-20T18:36:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.432+0000 7f1ef0d23280 4 Selected 0 events events for processing 2024-05-20T18:36:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.432+0000 7f1ef0d23280 4 0 events 2024-05-20T18:36:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.432+0000 7f1ef0d23280 1 -- 172.21.15.5:0/4273834146 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x55d40fdc2880 msgr2=0x55d40fea3600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.432+0000 7f1ef0d23280 1 --2- 172.21.15.5:0/4273834146 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x55d40fdc2880 0x55d40fea3600 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.432+0000 7f1ef0d23280 1 -- 172.21.15.5:0/4273834146 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x55d40fdc2d00 msgr2=0x55d40fea3b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.432+0000 7f1ef0d23280 1 --2- 172.21.15.5:0/4273834146 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x55d40fdc2d00 0x55d40fea3b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.432+0000 7f1ef0d23280 1 -- 172.21.15.5:0/4273834146 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x55d40fdc1200 msgr2=0x55d40fea5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.432+0000 7f1ef0d23280 1 --2- 172.21.15.5:0/4273834146 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x55d40fdc1200 0x55d40fea5700 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x55d40bd1be00 tx=0x55d40fe0e4e0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.432+0000 7f1ef0d23280 1 -- 172.21.15.5:0/4273834146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d40fdc0000 msgr2=0x55d40fea4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.432+0000 7f1ef0d23280 1 --2- 172.21.15.5:0/4273834146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d40fdc0000 0x55d40fea4680 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x55d410bdd980 tx=0x55d40fe10180 comp rx=0 tx=0).stop 2024-05-20T18:36:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.432+0000 7f1eeebf2640 1 -- 172.21.15.5:0/4273834146 reap_dead start 2024-05-20T18:36:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.432+0000 7f1ef0d23280 1 -- 172.21.15.5:0/4273834146 shutdown_connections 2024-05-20T18:36:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.432+0000 7f1ef0d23280 1 -- 172.21.15.5:0/4273834146 >> 172.21.15.5:0/4273834146 conn(0x55d40fdc0480 msgr2=0x55d40fe33800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.432+0000 7f1eeebf2640 1 -- 172.21.15.5:0/4273834146 reap_dead start 2024-05-20T18:36:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.432+0000 7f1ef0d23280 1 -- 172.21.15.5:0/4273834146 shutdown_connections 2024-05-20T18:36:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.432+0000 7f1ef0d23280 1 -- 172.21.15.5:0/4273834146 wait complete. 2024-05-20T18:36:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.432+0000 7f1ef0d23280 1 -- 172.21.15.5:0/3934823614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d40fdc2400 msgr2=0x55d40fea3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.432+0000 7f1ef0d23280 1 --2- 172.21.15.5:0/3934823614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d40fdc2400 0x55d40fea3080 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x55d410b3b410 tx=0x55d40fe102a0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.432+0000 7f1ef0d23280 1 -- 172.21.15.5:0/3934823614 shutdown_connections 2024-05-20T18:36:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.432+0000 7f1ef0d23280 1 --2- 172.21.15.5:0/3934823614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d40fdc2400 0x55d40fea3080 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.432+0000 7f1ef0d23280 1 --2- 172.21.15.5:0/3934823614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d40fdc1b00 0x55d40fea2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.432+0000 7f1ef0d23280 1 --2- 172.21.15.5:0/3934823614 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d40fdc1680 0x55d40fea2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.432+0000 7f1ef0d23280 1 -- 172.21.15.5:0/3934823614 >> 172.21.15.5:0/3934823614 conn(0x55d40fdc1f80 msgr2=0x55d40fe33000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:21.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.432+0000 7f1ef0d23280 1 -- 172.21.15.5:0/3934823614 shutdown_connections 2024-05-20T18:36:21.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.432+0000 7f1ef0d23280 1 -- 172.21.15.5:0/3934823614 wait complete. 2024-05-20T18:36:21.442 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.079353, result: Events by type: Errors: 0 2024-05-20T18:36:21.442 DEBUG:teuthology.orchestra.run.smithi005:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2024-05-20T18:36:21.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.497+0000 7f8405d29280 1 -- start start 2024-05-20T18:36:21.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.498+0000 7f8405d29280 1 Processor -- start 2024-05-20T18:36:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.499+0000 7f8405d29280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636606c5680 0x5636607a7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.499+0000 7f8405d29280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5636606c5b00 0x5636607a7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.499+0000 7f8405d29280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636606c6400 0x5636607a6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.499+0000 7f8405d29280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563661400180 con 0x5636606c5680 2024-05-20T18:36:21.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.499+0000 7f8405d29280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56366067e900 con 0x5636606c6400 2024-05-20T18:36:21.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.499+0000 7f8405d29280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x56366067e480 con 0x5636606c5b00 2024-05-20T18:36:21.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.499+0000 7f84044d7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636606c5680 0x5636607a7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.499+0000 7f84044d7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636606c5680 0x5636607a7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57722/0 (socket says 172.21.15.5:57722) 2024-05-20T18:36:21.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.499+0000 7f84044d7640 1 -- 172.21.15.5:0/1166135405 learned_addr learned my addr 172.21.15.5:0/1166135405 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:21.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.499+0000 7f8403cd6640 1 --2- 172.21.15.5:0/1166135405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5636606c5b00 0x5636607a7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.499+0000 7f8404cd8640 1 --2- 172.21.15.5:0/1166135405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636606c6400 0x5636607a6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.499+0000 7f84044d7640 1 -- 172.21.15.5:0/1166135405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5636606c5b00 msgr2=0x5636607a7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:21.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.499+0000 7f84044d7640 1 --2- 172.21.15.5:0/1166135405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5636606c5b00 0x5636607a7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.499+0000 7f84044d7640 1 -- 172.21.15.5:0/1166135405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636606c6400 msgr2=0x5636607a6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:21.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.499+0000 7f84044d7640 1 --2- 172.21.15.5:0/1166135405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636606c6400 0x5636607a6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.499+0000 7f84044d7640 1 -- 172.21.15.5:0/1166135405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56365c622780 con 0x5636606c5680 2024-05-20T18:36:21.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.499+0000 7f84044d7640 1 --2- 172.21.15.5:0/1166135405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636606c5680 0x5636607a7b80 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x563661449410 tx=0x5636607140c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d95b51d08f480f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:21.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.500+0000 7f8401cd2640 1 -- 172.21.15.5:0/1166135405 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56365c612680 con 0x5636606c5680 2024-05-20T18:36:21.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.500+0000 7f8401cd2640 1 -- 172.21.15.5:0/1166135405 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56365c618700 con 0x5636606c5680 2024-05-20T18:36:21.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.500+0000 7f8401cd2640 1 -- 172.21.15.5:0/1166135405 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56365c612b60 con 0x5636606c5680 2024-05-20T18:36:21.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.500+0000 7f8405d29280 1 -- 172.21.15.5:0/1166135405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56365c6234a0 con 0x5636606c5680 2024-05-20T18:36:21.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.500+0000 7f8405d29280 1 -- 172.21.15.5:0/1166135405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=0}) -- 0x56365c622000 con 0x5636606c5680 2024-05-20T18:36:21.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.500+0000 7f8405d29280 4 waiting for MDS map... 2024-05-20T18:36:21.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.500+0000 7f8401cd2640 1 -- 172.21.15.5:0/1166135405 <== mon.0 v2:172.21.15.5:3300/0 4 ==== osd_map(108..108 src has 1..108) ==== 6245+0+0 (secure 0 0 0) 0x5636614eea00 con 0x5636606c5680 2024-05-20T18:36:21.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.500+0000 7f8401cd2640 1 -- 172.21.15.5:0/1166135405 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 83) ==== 2885+0+0 (secure 0 0 0) 0x563660677180 con 0x5636606c5680 2024-05-20T18:36:21.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.500+0000 7f8405d29280 4 Got MDS map 83 2024-05-20T18:36:21.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.500+0000 7f8405d29280 10 main: JournalTool::main 2024-05-20T18:36:21.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.500+0000 7f8405d29280 4 main: JournalTool: connecting to RADOS... 2024-05-20T18:36:21.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.501+0000 7f8405d29280 1 Processor -- start 2024-05-20T18:36:21.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.501+0000 7f8405d29280 1 -- start start 2024-05-20T18:36:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.502+0000 7f8405d29280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636606c4d80 0x5636607a6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.502+0000 7f8405d29280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636606c4900 0x5636607a7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.502+0000 7f8405d29280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5636606c4480 0x5636607a8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.502+0000 7f8405d29280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56366067e180 con 0x5636606c4900 2024-05-20T18:36:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.502+0000 7f8405d29280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563661400d80 con 0x5636606c4d80 2024-05-20T18:36:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.502+0000 7f8405d29280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x563661400f00 con 0x5636606c4480 2024-05-20T18:36:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.502+0000 7f84044d7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636606c4900 0x5636607a7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.502+0000 7f84044d7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636606c4900 0x5636607a7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57724/0 (socket says 172.21.15.5:57724) 2024-05-20T18:36:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.502+0000 7f84044d7640 1 -- 172.21.15.5:0/1851146668 learned_addr learned my addr 172.21.15.5:0/1851146668 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.502+0000 7f84044d7640 1 -- 172.21.15.5:0/1851146668 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5636606c4480 msgr2=0x5636607a8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.502+0000 7f84044d7640 1 --2- 172.21.15.5:0/1851146668 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5636606c4480 0x5636607a8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.502+0000 7f84044d7640 1 -- 172.21.15.5:0/1851146668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636606c4d80 msgr2=0x5636607a6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.502+0000 7f84044d7640 1 --2- 172.21.15.5:0/1851146668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636606c4d80 0x5636607a6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.503+0000 7f84044d7640 1 -- 172.21.15.5:0/1851146668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56365c6223c0 con 0x5636606c4900 2024-05-20T18:36:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.503+0000 7f84044d7640 1 --2- 172.21.15.5:0/1851146668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636606c4900 0x5636607a7080 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x563661502660 tx=0x563660712960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f4867e83856b84 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.503+0000 7f83ffcce640 1 -- 172.21.15.5:0/1851146668 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56365c612820 con 0x5636606c4900 2024-05-20T18:36:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.503+0000 7f83ffcce640 1 -- 172.21.15.5:0/1851146668 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56365c618fc0 con 0x5636606c4900 2024-05-20T18:36:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.503+0000 7f83ffcce640 1 -- 172.21.15.5:0/1851146668 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56365c612680 con 0x5636606c4900 2024-05-20T18:36:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.503+0000 7f8405d29280 1 -- 172.21.15.5:0/1851146668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636606c4900 msgr2=0x5636607a7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.503+0000 7f8405d29280 1 --2- 172.21.15.5:0/1851146668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636606c4900 0x5636607a7080 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x563661502660 tx=0x563660712960 comp rx=0 tx=0).stop 2024-05-20T18:36:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.503+0000 7f8405d29280 1 -- 172.21.15.5:0/1851146668 shutdown_connections 2024-05-20T18:36:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.503+0000 7f8405d29280 1 --2- 172.21.15.5:0/1851146668 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5636606c4480 0x5636607a8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.503+0000 7f8405d29280 1 --2- 172.21.15.5:0/1851146668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636606c4900 0x5636607a7080 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.503+0000 7f8405d29280 1 --2- 172.21.15.5:0/1851146668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636606c4d80 0x5636607a6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.503+0000 7f8405d29280 1 -- 172.21.15.5:0/1851146668 >> 172.21.15.5:0/1851146668 conn(0x5636606c5200 msgr2=0x563660737800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.503+0000 7f8405d29280 1 -- 172.21.15.5:0/1851146668 shutdown_connections 2024-05-20T18:36:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.503+0000 7f8405d29280 1 -- 172.21.15.5:0/1851146668 wait complete. 2024-05-20T18:36:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.504+0000 7f8405d29280 1 Processor -- start 2024-05-20T18:36:21.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.504+0000 7f8405d29280 1 -- start start 2024-05-20T18:36:21.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.505+0000 7f8405d29280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636606c4900 0x5636607a8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.505+0000 7f8405d29280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5636606c4d80 0x5636607a7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.505+0000 7f8405d29280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636606c4000 0x5636607a6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.505+0000 7f8405d29280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563661401200 con 0x5636606c4000 2024-05-20T18:36:21.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.505+0000 7f8405d29280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563661401080 con 0x5636606c4900 2024-05-20T18:36:21.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.505+0000 7f8405d29280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x563661401380 con 0x5636606c4d80 2024-05-20T18:36:21.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.506+0000 7f8403cd6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636606c4000 0x5636607a6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.506+0000 7f8403cd6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636606c4000 0x5636607a6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57738/0 (socket says 172.21.15.5:57738) 2024-05-20T18:36:21.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.506+0000 7f8403cd6640 1 -- 172.21.15.5:0/1625216234 learned_addr learned my addr 172.21.15.5:0/1625216234 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:21.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.506+0000 7f8404cd8640 1 --2- 172.21.15.5:0/1625216234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636606c4900 0x5636607a8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.506+0000 7f84044d7640 1 --2- 172.21.15.5:0/1625216234 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5636606c4d80 0x5636607a7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.506+0000 7f8403cd6640 1 -- 172.21.15.5:0/1625216234 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5636606c4d80 msgr2=0x5636607a7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.506+0000 7f8403cd6640 1 --2- 172.21.15.5:0/1625216234 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5636606c4d80 0x5636607a7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.506+0000 7f8403cd6640 1 -- 172.21.15.5:0/1625216234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636606c4900 msgr2=0x5636607a8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.506+0000 7f8403cd6640 1 --2- 172.21.15.5:0/1625216234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636606c4900 0x5636607a8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.506+0000 7f8403cd6640 1 -- 172.21.15.5:0/1625216234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56365c622780 con 0x5636606c4000 2024-05-20T18:36:21.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.506+0000 7f8404cd8640 1 --2- 172.21.15.5:0/1625216234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636606c4900 0x5636607a8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:36:21.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.506+0000 7f8403cd6640 1 --2- 172.21.15.5:0/1625216234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636606c4000 0x5636607a6b00 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x563661503980 tx=0x563661505a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:21.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.506+0000 7f83fe4cb640 1 -- 172.21.15.5:0/1625216234 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56365c612820 con 0x5636606c4000 2024-05-20T18:36:21.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.506+0000 7f83fe4cb640 1 -- 172.21.15.5:0/1625216234 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56365c618a80 con 0x5636606c4000 2024-05-20T18:36:21.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.506+0000 7f8405d29280 1 -- 172.21.15.5:0/1625216234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56365c6234a0 con 0x5636606c4000 2024-05-20T18:36:21.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.506+0000 7f83fe4cb640 1 -- 172.21.15.5:0/1625216234 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56365c612680 con 0x5636606c4000 2024-05-20T18:36:21.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.506+0000 7f8405d29280 1 -- 172.21.15.5:0/1625216234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56365c622960 con 0x5636606c4000 2024-05-20T18:36:21.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.506+0000 7f8405d29280 4 main: JournalTool: resolving pool 15 2024-05-20T18:36:21.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.508+0000 7f83fe4cb640 1 -- 172.21.15.5:0/1625216234 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x56366156ac00 con 0x5636606c4000 2024-05-20T18:36:21.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.508+0000 7f83fe4cb640 1 --2- 172.21.15.5:0/1625216234 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x5636606c5200 0x563660a0a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.508+0000 7f83fe4cb640 1 -- 172.21.15.5:0/1625216234 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6245+0+0 (secure 0 0 0) 0x563661574a00 con 0x5636606c4000 2024-05-20T18:36:21.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.508+0000 7f8405d29280 4 main: JournalTool: creating IoCtx.. 2024-05-20T18:36:21.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.508+0000 7f8405d29280 4 main: Executing for rank 0 2024-05-20T18:36:21.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.508+0000 7f8405d29280 4 Filtering by inode '1099511627778' 2024-05-20T18:36:21.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.508+0000 7f8404cd8640 1 --2- 172.21.15.5:0/1625216234 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x5636606c5200 0x563660a0a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.508+0000 7f8405d29280 1 --2- 172.21.15.5:0/1625216234 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x5636606c6880 0x5636607a8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.508+0000 7f8405d29280 1 -- 172.21.15.5:0/1625216234 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) -- 0x563661576800 con 0x5636606c6880 2024-05-20T18:36:21.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.508+0000 7f84044d7640 1 --2- 172.21.15.5:0/1625216234 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x5636606c6880 0x5636607a8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.508+0000 7f8404cd8640 1 --2- 172.21.15.5:0/1625216234 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x5636606c5200 0x563660a0a000 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x5636606fc060 tx=0x5636607124e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:21.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.509+0000 7f84044d7640 1 --2- 172.21.15.5:0/1625216234 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x5636606c6880 0x5636607a8680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:21.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.509+0000 7f84044d7640 1 -- 172.21.15.5:0/1625216234 <== osd.6 v2:172.21.15.63:6816/3384565140 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x56366074a000 con 0x5636606c6880 2024-05-20T18:36:21.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.509+0000 7f8405d29280 4 Pointer 400.00000000 is readable 2024-05-20T18:36:21.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.509+0000 7f8405d29280 4 JournalScanner::scan: reading header object '200.00000000' 2024-05-20T18:36:21.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.509+0000 7f8405d29280 1 -- 172.21.15.5:0/1625216234 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) -- 0x563661576c00 con 0x5636606c6880 2024-05-20T18:36:21.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.509+0000 7f84044d7640 1 -- 172.21.15.5:0/1625216234 <== osd.6 v2:172.21.15.63:6816/3384565140 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x56366074a000 con 0x5636606c6880 2024-05-20T18:36:21.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.509+0000 7f8405d29280 10 Header 0x400000 0x407d21 0x40809d 2024-05-20T18:36:21.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.509+0000 7f8405d29280 10 Starting journal scan from offset 0x407d21 2024-05-20T18:36:21.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.509+0000 7f8405d29280 1 -- 172.21.15.5:0/1625216234 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 32033~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) -- 0x563661577000 con 0x5636606c6880 2024-05-20T18:36:21.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.510+0000 7f84044d7640 1 -- 172.21.15.5:0/1625216234 <== osd.6 v2:172.21.15.63:6816/3384565140 3 ==== osd_op_reply(3 200.00000001 [read 32033~892 out=892b] v0'0 uv28 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x56366074a000 con 0x5636606c6880 2024-05-20T18:36:21.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.510+0000 7f8405d29280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-05-20T18:36:21.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.510+0000 7f8405d29280 10 Parsing data, 0x37c bytes available 2024-05-20T18:36:21.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.510+0000 7f8405d29280 10 Attempting decode at 0x407d21 2024-05-20T18:36:21.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.510+0000 7f8405d29280 10 Consumed 0x37c bytes 2024-05-20T18:36:21.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.510+0000 7f8405d29280 10 Valid entry at 0x407d21 2024-05-20T18:36:21.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.510+0000 7f8405d29280 1 --2- 172.21.15.5:0/1625216234 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x5636606c6d00 0x5636607a8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.510+0000 7f8405d29280 1 -- 172.21.15.5:0/1625216234 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) -- 0x563661577400 con 0x5636606c6d00 2024-05-20T18:36:21.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.510+0000 7f8404cd8640 1 --2- 172.21.15.5:0/1625216234 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x5636606c6d00 0x5636607a8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.510+0000 7f8404cd8640 1 --2- 172.21.15.5:0/1625216234 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x5636606c6d00 0x5636607a8c00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:21.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.511+0000 7f8404cd8640 1 -- 172.21.15.5:0/1625216234 <== osd.7 v2:172.21.15.63:6824/4136124451 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56366074a500 con 0x5636606c6d00 2024-05-20T18:36:21.517 INFO:teuthology.orchestra.run.smithi005.stdout:Events by type: 2024-05-20T18:36:21.517 INFO:teuthology.orchestra.run.smithi005.stdout:Errors: 0 2024-05-20T18:36:21.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.511+0000 7f8405d29280 4 Reached end of journal objects 2024-05-20T18:36:21.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.511+0000 7f8405d29280 4 Scanned objects, 0 missing, 1 valid 2024-05-20T18:36:21.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.511+0000 7f8405d29280 4 Events scanned, 0 gaps 2024-05-20T18:36:21.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.511+0000 7f8405d29280 4 Found 1 valid events 2024-05-20T18:36:21.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.511+0000 7f8405d29280 4 Selected 0 events events for processing 2024-05-20T18:36:21.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.511+0000 7f8405d29280 4 0 events 2024-05-20T18:36:21.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.511+0000 7f8405d29280 1 -- 172.21.15.5:0/1625216234 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x5636606c6880 msgr2=0x5636607a8680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.511+0000 7f8405d29280 1 --2- 172.21.15.5:0/1625216234 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x5636606c6880 0x5636607a8680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.511+0000 7f8405d29280 1 -- 172.21.15.5:0/1625216234 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x5636606c6d00 msgr2=0x5636607a8c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.511+0000 7f8405d29280 1 --2- 172.21.15.5:0/1625216234 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x5636606c6d00 0x5636607a8c00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.511+0000 7f8405d29280 1 -- 172.21.15.5:0/1625216234 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x5636606c5200 msgr2=0x563660a0a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.511+0000 7f8405d29280 1 --2- 172.21.15.5:0/1625216234 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x5636606c5200 0x563660a0a000 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x5636606fc060 tx=0x5636607124e0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.511+0000 7f8405d29280 1 -- 172.21.15.5:0/1625216234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636606c4000 msgr2=0x5636607a6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.511+0000 7f8405d29280 1 --2- 172.21.15.5:0/1625216234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636606c4000 0x5636607a6b00 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x563661503980 tx=0x563661505a40 comp rx=0 tx=0).stop 2024-05-20T18:36:21.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.511+0000 7f8403cd6640 1 -- 172.21.15.5:0/1625216234 reap_dead start 2024-05-20T18:36:21.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.511+0000 7f8403cd6640 1 -- 172.21.15.5:0/1625216234 reap_dead start 2024-05-20T18:36:21.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.511+0000 7f8405d29280 1 -- 172.21.15.5:0/1625216234 shutdown_connections 2024-05-20T18:36:21.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.511+0000 7f8405d29280 1 -- 172.21.15.5:0/1625216234 >> 172.21.15.5:0/1625216234 conn(0x5636606c4480 msgr2=0x563660737800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:21.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.511+0000 7f8405d29280 1 -- 172.21.15.5:0/1625216234 shutdown_connections 2024-05-20T18:36:21.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.511+0000 7f8405d29280 1 -- 172.21.15.5:0/1625216234 wait complete. 2024-05-20T18:36:21.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.511+0000 7f8405d29280 1 -- 172.21.15.5:0/1166135405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636606c5680 msgr2=0x5636607a7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.511+0000 7f8405d29280 1 --2- 172.21.15.5:0/1166135405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636606c5680 0x5636607a7b80 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x563661449410 tx=0x5636607140c0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.511+0000 7f8405d29280 1 -- 172.21.15.5:0/1166135405 shutdown_connections 2024-05-20T18:36:21.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.511+0000 7f8405d29280 1 --2- 172.21.15.5:0/1166135405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5636606c6400 0x5636607a6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.511+0000 7f8405d29280 1 --2- 172.21.15.5:0/1166135405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5636606c5b00 0x5636607a7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.511+0000 7f8405d29280 1 --2- 172.21.15.5:0/1166135405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636606c5680 0x5636607a7b80 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.511+0000 7f8405d29280 1 -- 172.21.15.5:0/1166135405 >> 172.21.15.5:0/1166135405 conn(0x5636606c5f80 msgr2=0x563660737000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:21.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.511+0000 7f8405d29280 1 -- 172.21.15.5:0/1166135405 shutdown_connections 2024-05-20T18:36:21.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.511+0000 7f8405d29280 1 -- 172.21.15.5:0/1166135405 wait complete. 2024-05-20T18:36:21.522 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.079628, result: Events by type: Errors: 0 2024-05-20T18:36:21.522 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-05-20T18:36:21.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.577+0000 7f45442f6f40 1 Processor -- start 2024-05-20T18:36:21.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.578+0000 7f45442f6f40 1 -- start start 2024-05-20T18:36:21.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.579+0000 7f45442f6f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583833a2970 0x55838335a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.579+0000 7f45442f6f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55838342fa30 0x5583833868e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.579+0000 7f45442f6f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558383433010 0x558383387310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.579+0000 7f45442f6f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55838338a550 con 0x558383433010 2024-05-20T18:36:21.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.579+0000 7f45442f6f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558383302c80 con 0x55838342fa30 2024-05-20T18:36:21.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.579+0000 7f45442f6f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x558383433960 con 0x5583833a2970 2024-05-20T18:36:21.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.579+0000 7f453e5c8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558383433010 0x558383387310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.579+0000 7f453ddc7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583833a2970 0x55838335a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.579+0000 7f453e5c8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558383433010 0x558383387310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57748/0 (socket says 172.21.15.5:57748) 2024-05-20T18:36:21.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.579+0000 7f453d5c6640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55838342fa30 0x5583833868e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.579+0000 7f453e5c8640 1 -- 172.21.15.5:0/1268423231 learned_addr learned my addr 172.21.15.5:0/1268423231 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:21.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.579+0000 7f453e5c8640 1 -- 172.21.15.5:0/1268423231 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583833a2970 msgr2=0x55838335a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:21.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.579+0000 7f453e5c8640 1 --2- 172.21.15.5:0/1268423231 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583833a2970 0x55838335a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.579+0000 7f453e5c8640 1 -- 172.21.15.5:0/1268423231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55838342fa30 msgr2=0x5583833868e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:21.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.579+0000 7f453e5c8640 1 --2- 172.21.15.5:0/1268423231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55838342fa30 0x5583833868e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.579+0000 7f453e5c8640 1 -- 172.21.15.5:0/1268423231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558383287cb0 con 0x558383433010 2024-05-20T18:36:21.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.579+0000 7f453d5c6640 1 --2- 172.21.15.5:0/1268423231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55838342fa30 0x5583833868e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:36:21.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.579+0000 7f453e5c8640 1 --2- 172.21.15.5:0/1268423231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558383433010 0x558383387310 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f45300015e0 tx=0x7f45300019e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d47bfcf15e4a1cc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:21.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.580+0000 7f4537fff640 1 -- 172.21.15.5:0/1268423231 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4530013620 con 0x558383433010 2024-05-20T18:36:21.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.580+0000 7f4537fff640 1 -- 172.21.15.5:0/1268423231 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45300137c0 con 0x558383433010 2024-05-20T18:36:21.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.580+0000 7f4537fff640 1 -- 172.21.15.5:0/1268423231 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4530013bb0 con 0x558383433010 2024-05-20T18:36:21.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.580+0000 7f45442f6f40 1 -- 172.21.15.5:0/1268423231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558383433010 msgr2=0x558383387310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.580+0000 7f45442f6f40 1 --2- 172.21.15.5:0/1268423231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558383433010 0x558383387310 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f45300015e0 tx=0x7f45300019e0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.580+0000 7f45442f6f40 1 -- 172.21.15.5:0/1268423231 shutdown_connections 2024-05-20T18:36:21.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.580+0000 7f45442f6f40 1 --2- 172.21.15.5:0/1268423231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558383433010 0x558383387310 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.580+0000 7f45442f6f40 1 --2- 172.21.15.5:0/1268423231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55838342fa30 0x5583833868e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.580+0000 7f45442f6f40 1 --2- 172.21.15.5:0/1268423231 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583833a2970 0x55838335a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.580+0000 7f45442f6f40 1 -- 172.21.15.5:0/1268423231 >> 172.21.15.5:0/1268423231 conn(0x558383280ac0 msgr2=0x5583833a2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.580+0000 7f45442f6f40 1 -- 172.21.15.5:0/1268423231 shutdown_connections 2024-05-20T18:36:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.580+0000 7f45442f6f40 1 -- 172.21.15.5:0/1268423231 wait complete. 2024-05-20T18:36:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.581+0000 7f45442f6f40 1 Processor -- start 2024-05-20T18:36:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.581+0000 7f45442f6f40 1 -- start start 2024-05-20T18:36:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.582+0000 7f45442f6f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583833a2970 0x558383435890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.582+0000 7f45442f6f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55838342fa30 0x5583833872c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.582+0000 7f45442f6f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558383433010 0x5583834ca420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.582+0000 7f45442f6f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55838335ac00 con 0x558383433010 2024-05-20T18:36:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.582+0000 7f45442f6f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558383385080 con 0x5583833a2970 2024-05-20T18:36:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.583+0000 7f45442f6f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x558383437bf0 con 0x55838342fa30 2024-05-20T18:36:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.583+0000 7f453e5c8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558383433010 0x5583834ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.583+0000 7f453e5c8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558383433010 0x5583834ca420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57758/0 (socket says 172.21.15.5:57758) 2024-05-20T18:36:21.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.583+0000 7f453e5c8640 1 -- 172.21.15.5:0/776086528 learned_addr learned my addr 172.21.15.5:0/776086528 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:21.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.583+0000 7f453d5c6640 1 --2- 172.21.15.5:0/776086528 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55838342fa30 0x5583833872c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.583+0000 7f453ddc7640 1 --2- 172.21.15.5:0/776086528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583833a2970 0x558383435890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.583+0000 7f453e5c8640 1 -- 172.21.15.5:0/776086528 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55838342fa30 msgr2=0x5583833872c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.583+0000 7f453e5c8640 1 --2- 172.21.15.5:0/776086528 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55838342fa30 0x5583833872c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.583+0000 7f453e5c8640 1 -- 172.21.15.5:0/776086528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583833a2970 msgr2=0x558383435890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.583+0000 7f453e5c8640 1 --2- 172.21.15.5:0/776086528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583833a2970 0x558383435890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.583+0000 7f453e5c8640 1 -- 172.21.15.5:0/776086528 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558383386e00 con 0x558383433010 2024-05-20T18:36:21.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.583+0000 7f453e5c8640 1 --2- 172.21.15.5:0/776086528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558383433010 0x5583834ca420 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f45300012e0 tx=0x7f4530001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:21.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.583+0000 7f4536ffd640 1 -- 172.21.15.5:0/776086528 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4530021020 con 0x558383433010 2024-05-20T18:36:21.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.583+0000 7f4536ffd640 1 -- 172.21.15.5:0/776086528 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4530003450 con 0x558383433010 2024-05-20T18:36:21.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.583+0000 7f4536ffd640 1 -- 172.21.15.5:0/776086528 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4530018040 con 0x558383433010 2024-05-20T18:36:21.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.583+0000 7f45442f6f40 1 -- 172.21.15.5:0/776086528 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5583834216b0 con 0x558383433010 2024-05-20T18:36:21.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.583+0000 7f45442f6f40 1 -- 172.21.15.5:0/776086528 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558383437670 con 0x558383433010 2024-05-20T18:36:21.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.585+0000 7f4536ffd640 1 -- 172.21.15.5:0/776086528 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4530024020 con 0x558383433010 2024-05-20T18:36:21.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.585+0000 7f4536ffd640 1 --2- 172.21.15.5:0/776086528 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f452006a810 0x7f452006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.586+0000 7f4536ffd640 1 -- 172.21.15.5:0/776086528 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6245+0+0 (secure 0 0 0) 0x7f453008b8e0 con 0x558383433010 2024-05-20T18:36:21.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.586+0000 7f453ddc7640 1 --2- 172.21.15.5:0/776086528 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f452006a810 0x7f452006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.586+0000 7f45442f6f40 1 --2- 172.21.15.5:0/776086528 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x558383435210 0x558383433e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.586+0000 7f45442f6f40 1 -- 172.21.15.5:0/776086528 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x558383437840 con 0x558383435210 2024-05-20T18:36:21.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.586+0000 7f453d5c6640 1 --2- 172.21.15.5:0/776086528 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x558383435210 0x558383433e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.586+0000 7f453ddc7640 1 --2- 172.21.15.5:0/776086528 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f452006a810 0x7f452006cd20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f4538056000 tx=0x7f4538050000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:21.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.586+0000 7f453d5c6640 1 --2- 172.21.15.5:0/776086528 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x558383435210 0x558383433e20 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:21.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.588+0000 7f453d5c6640 1 -- 172.21.15.5:0/776086528 <== osd.1 v2:172.21.15.5:6815/550664725 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v108'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4528009ad0 con 0x558383435210 2024-05-20T18:36:21.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.588+0000 7f45442f6f40 1 -- 172.21.15.5:0/776086528 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x558383435210 msgr2=0x558383433e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.588+0000 7f45442f6f40 1 --2- 172.21.15.5:0/776086528 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x558383435210 0x558383433e20 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.588+0000 7f45442f6f40 1 -- 172.21.15.5:0/776086528 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f452006a810 msgr2=0x7f452006cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.588+0000 7f45442f6f40 1 --2- 172.21.15.5:0/776086528 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f452006a810 0x7f452006cd20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f4538056000 tx=0x7f4538050000 comp rx=0 tx=0).stop 2024-05-20T18:36:21.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.588+0000 7f45442f6f40 1 -- 172.21.15.5:0/776086528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558383433010 msgr2=0x5583834ca420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.588+0000 7f45442f6f40 1 --2- 172.21.15.5:0/776086528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558383433010 0x5583834ca420 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f45300012e0 tx=0x7f4530001250 comp rx=0 tx=0).stop 2024-05-20T18:36:21.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.588+0000 7f453e5c8640 1 -- 172.21.15.5:0/776086528 reap_dead start 2024-05-20T18:36:21.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.588+0000 7f45442f6f40 1 -- 172.21.15.5:0/776086528 shutdown_connections 2024-05-20T18:36:21.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.588+0000 7f45442f6f40 1 -- 172.21.15.5:0/776086528 >> 172.21.15.5:0/776086528 conn(0x558383280ac0 msgr2=0x55838342c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:21.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.588+0000 7f45442f6f40 1 -- 172.21.15.5:0/776086528 shutdown_connections 2024-05-20T18:36:21.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.588+0000 7f45442f6f40 1 -- 172.21.15.5:0/776086528 wait complete. 2024-05-20T18:36:21.595 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-20T18:36:21.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.749+0000 7fa42f902640 1 Processor -- start 2024-05-20T18:36:21.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.750+0000 7fa42f902640 1 -- start start 2024-05-20T18:36:21.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.751+0000 7fa42f902640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa4280cf3d0 0x7fa4280cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.751+0000 7fa42f902640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4280ce7e0 0x7fa4280cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.751+0000 7fa42f902640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4280cfd20 con 0x7fa42815f220 2024-05-20T18:36:21.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.751+0000 7fa42f902640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fa4280cf130 con 0x7fa4280cf3d0 2024-05-20T18:36:21.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.751+0000 7fa42f902640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa4280cda20 con 0x7fa4280ce7e0 2024-05-20T18:36:21.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.751+0000 7fa42ce9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa4280cf3d0 0x7fa4280cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.751+0000 7fa42ce9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa4280cf3d0 0x7fa4280cf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57760/0 (socket says 172.21.15.5:57760) 2024-05-20T18:36:21.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.751+0000 7fa42ce9d640 1 -- 172.21.15.5:0/1921240155 learned_addr learned my addr 172.21.15.5:0/1921240155 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:21.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.751+0000 7fa42de9f640 1 --2- 172.21.15.5:0/1921240155 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4280ce7e0 0x7fa4280cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.751+0000 7fa42ce9d640 1 -- 172.21.15.5:0/1921240155 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4280ce7e0 msgr2=0x7fa4280cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:21.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.751+0000 7fa42ce9d640 1 --2- 172.21.15.5:0/1921240155 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4280ce7e0 0x7fa4280cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.751+0000 7fa42ce9d640 1 -- 172.21.15.5:0/1921240155 >> v1:172.21.15.63:6789/0 conn(0x7fa42815f220 legacy=0x7fa4280d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.752+0000 7fa42ce9d640 1 -- 172.21.15.5:0/1921240155 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4280cdd00 con 0x7fa4280cf3d0 2024-05-20T18:36:21.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.752+0000 7fa42ce9d640 1 --2- 172.21.15.5:0/1921240155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4280cf3d0 0x7fa4280cf7e0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fa41c009870 tx=0x7fa41c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5babb5ed5dbc83c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:21.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.752+0000 7fa427fff640 1 -- 172.21.15.5:0/1921240155 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c037070 con 0x7fa4280cf3d0 2024-05-20T18:36:21.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.752+0000 7fa427fff640 1 -- 172.21.15.5:0/1921240155 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa41c032440 con 0x7fa4280cf3d0 2024-05-20T18:36:21.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.752+0000 7fa427fff640 1 -- 172.21.15.5:0/1921240155 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c032810 con 0x7fa4280cf3d0 2024-05-20T18:36:21.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.753+0000 7fa42f902640 1 -- 172.21.15.5:0/1921240155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4280cf3d0 msgr2=0x7fa4280cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.753+0000 7fa42f902640 1 --2- 172.21.15.5:0/1921240155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4280cf3d0 0x7fa4280cf7e0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fa41c009870 tx=0x7fa41c02a1c0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.753+0000 7fa42f902640 1 -- 172.21.15.5:0/1921240155 shutdown_connections 2024-05-20T18:36:21.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.753+0000 7fa42f902640 1 --2- 172.21.15.5:0/1921240155 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4280ce7e0 0x7fa4280cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.753+0000 7fa42f902640 1 --2- 172.21.15.5:0/1921240155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4280cf3d0 0x7fa4280cf7e0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.753+0000 7fa42f902640 1 -- 172.21.15.5:0/1921240155 >> 172.21.15.5:0/1921240155 conn(0x7fa4280b44d0 msgr2=0x7fa4280b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:21.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.753+0000 7fa42f902640 1 -- 172.21.15.5:0/1921240155 shutdown_connections 2024-05-20T18:36:21.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.753+0000 7fa42f902640 1 -- 172.21.15.5:0/1921240155 wait complete. 2024-05-20T18:36:21.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.755+0000 7fa42f902640 1 Processor -- start 2024-05-20T18:36:21.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.755+0000 7fa42f902640 1 -- start start 2024-05-20T18:36:21.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.756+0000 7fa42f902640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4280ce7e0 0x7fa4281f53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.756+0000 7fa42f902640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4280cf3d0 0x7fa4280b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.756+0000 7fa42f902640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa42815f220 0x7fa4280c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.756+0000 7fa42f902640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4280d0fd0 con 0x7fa4280cf3d0 2024-05-20T18:36:21.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.756+0000 7fa42f902640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa4280ce2c0 con 0x7fa42815f220 2024-05-20T18:36:21.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.756+0000 7fa42f902640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa4280cd680 con 0x7fa4280ce7e0 2024-05-20T18:36:21.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.757+0000 7fa42ce9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4280cf3d0 0x7fa4280b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.757+0000 7fa42ce9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4280cf3d0 0x7fa4280b55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57764/0 (socket says 172.21.15.5:57764) 2024-05-20T18:36:21.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.757+0000 7fa42ce9d640 1 -- 172.21.15.5:0/2192907303 learned_addr learned my addr 172.21.15.5:0/2192907303 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:21.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.757+0000 7fa42d69e640 1 --2- 172.21.15.5:0/2192907303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4280ce7e0 0x7fa4281f53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.757+0000 7fa42de9f640 1 --2- 172.21.15.5:0/2192907303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa42815f220 0x7fa4280c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.757+0000 7fa42ce9d640 1 -- 172.21.15.5:0/2192907303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4280ce7e0 msgr2=0x7fa4281f53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.757+0000 7fa42ce9d640 1 --2- 172.21.15.5:0/2192907303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4280ce7e0 0x7fa4281f53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.757+0000 7fa42ce9d640 1 -- 172.21.15.5:0/2192907303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa42815f220 msgr2=0x7fa4280c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:21.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.757+0000 7fa42ce9d640 1 --2- 172.21.15.5:0/2192907303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa42815f220 0x7fa4280c86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:21.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.757+0000 7fa42ce9d640 1 -- 172.21.15.5:0/2192907303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4281f69c0 con 0x7fa4280cf3d0 2024-05-20T18:36:21.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.757+0000 7fa42d69e640 1 --2- 172.21.15.5:0/2192907303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4280ce7e0 0x7fa4281f53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:36:21.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.757+0000 7fa42ce9d640 1 --2- 172.21.15.5:0/2192907303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4280cf3d0 0x7fa4280b55e0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fa41c000c00 tx=0x7fa41c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:21.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.757+0000 7fa425ffb640 1 -- 172.21.15.5:0/2192907303 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c037040 con 0x7fa4280cf3d0 2024-05-20T18:36:21.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.757+0000 7fa425ffb640 1 -- 172.21.15.5:0/2192907303 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa41c042070 con 0x7fa4280cf3d0 2024-05-20T18:36:21.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.757+0000 7fa42f902640 1 -- 172.21.15.5:0/2192907303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4281f66e0 con 0x7fa4280cf3d0 2024-05-20T18:36:21.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.757+0000 7fa425ffb640 1 -- 172.21.15.5:0/2192907303 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c030880 con 0x7fa4280cf3d0 2024-05-20T18:36:21.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.758+0000 7fa42f902640 1 -- 172.21.15.5:0/2192907303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4280b7540 con 0x7fa4280cf3d0 2024-05-20T18:36:21.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.759+0000 7fa42f902640 1 -- 172.21.15.5:0/2192907303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4281f6430 con 0x7fa4280cf3d0 2024-05-20T18:36:21.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.759+0000 7fa425ffb640 1 -- 172.21.15.5:0/2192907303 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa41c02ad10 con 0x7fa4280cf3d0 2024-05-20T18:36:21.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.760+0000 7fa425ffb640 1 --2- 172.21.15.5:0/2192907303 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa41406be60 0x7fa41406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:21.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.760+0000 7fa425ffb640 1 -- 172.21.15.5:0/2192907303 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6245+0+0 (secure 0 0 0) 0x7fa41c0ab5e0 con 0x7fa4280cf3d0 2024-05-20T18:36:21.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.760+0000 7fa42d69e640 1 --2- 172.21.15.5:0/2192907303 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa41406be60 0x7fa41406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:21.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.760+0000 7fa42d69e640 1 --2- 172.21.15.5:0/2192907303 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa41406be60 0x7fa41406e370 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa4280c64c0 tx=0x7fa418009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:21.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.764+0000 7fa425ffb640 1 -- 172.21.15.5:0/2192907303 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa41c079ce0 con 0x7fa4280cf3d0 2024-05-20T18:36:21.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:21.987+0000 7fa42f902640 1 -- 172.21.15.5:0/2192907303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fa428075b00 con 0x7fa4280cf3d0 2024-05-20T18:36:22.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.348+0000 7fa425ffb640 1 -- 172.21.15.5:0/2192907303 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v84) ==== 162+0+0 (secure 0 0 0) 0x7fa41c0795e0 con 0x7fa4280cf3d0 2024-05-20T18:36:22.350 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-20T18:36:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.351+0000 7fa42f902640 1 -- 172.21.15.5:0/2192907303 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa41406be60 msgr2=0x7fa41406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.351+0000 7fa42f902640 1 --2- 172.21.15.5:0/2192907303 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa41406be60 0x7fa41406e370 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa4280c64c0 tx=0x7fa418009040 comp rx=0 tx=0).stop 2024-05-20T18:36:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.351+0000 7fa42f902640 1 -- 172.21.15.5:0/2192907303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4280cf3d0 msgr2=0x7fa4280b55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.351+0000 7fa42f902640 1 --2- 172.21.15.5:0/2192907303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4280cf3d0 0x7fa4280b55e0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fa41c000c00 tx=0x7fa41c0046c0 comp rx=0 tx=0).stop 2024-05-20T18:36:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.351+0000 7fa42f902640 1 -- 172.21.15.5:0/2192907303 shutdown_connections 2024-05-20T18:36:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.351+0000 7fa42f902640 1 --2- 172.21.15.5:0/2192907303 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa41406be60 0x7fa41406e370 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.351+0000 7fa42f902640 1 --2- 172.21.15.5:0/2192907303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa42815f220 0x7fa4280c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.351+0000 7fa42f902640 1 --2- 172.21.15.5:0/2192907303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4280cf3d0 0x7fa4280b55e0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.351+0000 7fa42f902640 1 --2- 172.21.15.5:0/2192907303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4280ce7e0 0x7fa4281f53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.351+0000 7fa42f902640 1 -- 172.21.15.5:0/2192907303 >> 172.21.15.5:0/2192907303 conn(0x7fa4280b44d0 msgr2=0x7fa4280d11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.351+0000 7fa42f902640 1 -- 172.21.15.5:0/2192907303 shutdown_connections 2024-05-20T18:36:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.351+0000 7fa42f902640 1 -- 172.21.15.5:0/2192907303 wait complete. 2024-05-20T18:36:22.367 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:36:22.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.484+0000 7f4210b8f640 1 Processor -- start 2024-05-20T18:36:22.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.484+0000 7f4210b8f640 1 -- start start 2024-05-20T18:36:22.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.486+0000 7f4210b8f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f420c0d10e0 0x7f420c0cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:22.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.486+0000 7f4210b8f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f420c0b7210 0x7f420c0cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:22.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.486+0000 7f4210b8f640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f420c0d1ac0 con 0x7f420c0d14f0 2024-05-20T18:36:22.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.486+0000 7f4210b8f640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f420c0d1da0 con 0x7f420c0b7210 2024-05-20T18:36:22.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.486+0000 7f4210b8f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f420c0d2080 con 0x7f420c0d10e0 2024-05-20T18:36:22.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.486+0000 7f420ad9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f420c0b7210 0x7f420c0cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:22.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.486+0000 7f420ad9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f420c0b7210 0x7f420c0cf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57780/0 (socket says 172.21.15.5:57780) 2024-05-20T18:36:22.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.486+0000 7f420ad9d640 1 -- 172.21.15.5:0/3047206838 learned_addr learned my addr 172.21.15.5:0/3047206838 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:22.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.486+0000 7f420a59c640 1 --2- 172.21.15.5:0/3047206838 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f420c0d10e0 0x7f420c0cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:22.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.486+0000 7f420ad9d640 1 -- 172.21.15.5:0/3047206838 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f420c0d10e0 msgr2=0x7f420c0cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:22.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.486+0000 7f420ad9d640 1 --2- 172.21.15.5:0/3047206838 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f420c0d10e0 0x7f420c0cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:22.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.486+0000 7f420ad9d640 1 -- 172.21.15.5:0/3047206838 >> v1:172.21.15.63:6789/0 conn(0x7f420c0d14f0 legacy=0x7f420c0cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:22.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.486+0000 7f420ad9d640 1 -- 172.21.15.5:0/3047206838 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f420c0d2360 con 0x7f420c0b7210 2024-05-20T18:36:22.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.487+0000 7f420ad9d640 1 --2- 172.21.15.5:0/3047206838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f420c0b7210 0x7f420c0cf680 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f4204009870 tx=0x7f420402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67458c5bf0ef58c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:22.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.487+0000 7f420959a640 1 -- 172.21.15.5:0/3047206838 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4204037070 con 0x7f420c0b7210 2024-05-20T18:36:22.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.487+0000 7f420959a640 1 -- 172.21.15.5:0/3047206838 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4204032440 con 0x7f420c0b7210 2024-05-20T18:36:22.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.487+0000 7f420959a640 1 -- 172.21.15.5:0/3047206838 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4204032810 con 0x7f420c0b7210 2024-05-20T18:36:22.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.488+0000 7f4210b8f640 1 -- 172.21.15.5:0/3047206838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f420c0b7210 msgr2=0x7f420c0cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:22.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.488+0000 7f4210b8f640 1 --2- 172.21.15.5:0/3047206838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f420c0b7210 0x7f420c0cf680 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f4204009870 tx=0x7f420402a270 comp rx=0 tx=0).stop 2024-05-20T18:36:22.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.488+0000 7f4210b8f640 1 -- 172.21.15.5:0/3047206838 shutdown_connections 2024-05-20T18:36:22.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.488+0000 7f4210b8f640 1 --2- 172.21.15.5:0/3047206838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f420c0b7210 0x7f420c0cf680 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:22.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.488+0000 7f4210b8f640 1 --2- 172.21.15.5:0/3047206838 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f420c0d10e0 0x7f420c0cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:22.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.488+0000 7f4210b8f640 1 -- 172.21.15.5:0/3047206838 >> 172.21.15.5:0/3047206838 conn(0x7f420c0b44d0 msgr2=0x7f420c0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:22.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.488+0000 7f4210b8f640 1 -- 172.21.15.5:0/3047206838 shutdown_connections 2024-05-20T18:36:22.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.488+0000 7f4210b8f640 1 -- 172.21.15.5:0/3047206838 wait complete. 2024-05-20T18:36:22.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.490+0000 7f4210b8f640 1 Processor -- start 2024-05-20T18:36:22.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.490+0000 7f4210b8f640 1 -- start start 2024-05-20T18:36:22.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.491+0000 7f4210b8f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f420c0b7210 0x7f420c0b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:22.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.491+0000 7f4210b8f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f420c0d10e0 0x7f420c0ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:22.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.491+0000 7f4210b8f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f420c0d14f0 0x7f420c1580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:22.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.491+0000 7f4210b8f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f420c0b8440 con 0x7f420c0d14f0 2024-05-20T18:36:22.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.491+0000 7f4210b8f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f420c0b82c0 con 0x7f420c0d10e0 2024-05-20T18:36:22.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.491+0000 7f4210b8f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f420c0709f0 con 0x7f420c0b7210 2024-05-20T18:36:22.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.491+0000 7f420ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f420c0d14f0 0x7f420c1580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:22.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.491+0000 7f420ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f420c0d14f0 0x7f420c1580b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57788/0 (socket says 172.21.15.5:57788) 2024-05-20T18:36:22.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.491+0000 7f420ad9d640 1 -- 172.21.15.5:0/3434850996 learned_addr learned my addr 172.21.15.5:0/3434850996 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:22.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.492+0000 7f420a59c640 1 --2- 172.21.15.5:0/3434850996 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f420c0b7210 0x7f420c0b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:22.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.492+0000 7f4209d9b640 1 --2- 172.21.15.5:0/3434850996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f420c0d10e0 0x7f420c0ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:22.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.492+0000 7f420ad9d640 1 -- 172.21.15.5:0/3434850996 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f420c0b7210 msgr2=0x7f420c0b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:22.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.492+0000 7f420ad9d640 1 --2- 172.21.15.5:0/3434850996 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f420c0b7210 0x7f420c0b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:22.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.492+0000 7f420ad9d640 1 -- 172.21.15.5:0/3434850996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f420c0d10e0 msgr2=0x7f420c0ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:22.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.492+0000 7f420ad9d640 1 --2- 172.21.15.5:0/3434850996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f420c0d10e0 0x7f420c0ca7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:22.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.492+0000 7f420ad9d640 1 -- 172.21.15.5:0/3434850996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f420c0cb380 con 0x7f420c0d14f0 2024-05-20T18:36:22.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.492+0000 7f4209d9b640 1 --2- 172.21.15.5:0/3434850996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f420c0d10e0 0x7f420c0ca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:36:22.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.492+0000 7f420ad9d640 1 --2- 172.21.15.5:0/3434850996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f420c0d14f0 0x7f420c1580b0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f420402a2b0 tx=0x7f42040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:22.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.492+0000 7f41faffd640 1 -- 172.21.15.5:0/3434850996 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4204041020 con 0x7f420c0d14f0 2024-05-20T18:36:22.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.492+0000 7f41faffd640 1 -- 172.21.15.5:0/3434850996 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42040306c0 con 0x7f420c0d14f0 2024-05-20T18:36:22.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.492+0000 7f41faffd640 1 -- 172.21.15.5:0/3434850996 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4204037040 con 0x7f420c0d14f0 2024-05-20T18:36:22.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.492+0000 7f4210b8f640 1 -- 172.21.15.5:0/3434850996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f420c0cb660 con 0x7f420c0d14f0 2024-05-20T18:36:22.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.492+0000 7f4210b8f640 1 -- 172.21.15.5:0/3434850996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f420c0cbc20 con 0x7f420c0d14f0 2024-05-20T18:36:22.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.493+0000 7f4210b8f640 1 -- 172.21.15.5:0/3434850996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f420c0cb0f0 con 0x7f420c0d14f0 2024-05-20T18:36:22.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.494+0000 7f41faffd640 1 -- 172.21.15.5:0/3434850996 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4204044020 con 0x7f420c0d14f0 2024-05-20T18:36:22.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.494+0000 7f41faffd640 1 --2- 172.21.15.5:0/3434850996 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f41ec06be60 0x7f41ec06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:22.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.494+0000 7f41faffd640 1 -- 172.21.15.5:0/3434850996 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6245+0+0 (secure 0 0 0) 0x7f42040ab910 con 0x7f420c0d14f0 2024-05-20T18:36:22.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.494+0000 7f420a59c640 1 --2- 172.21.15.5:0/3434850996 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f41ec06be60 0x7f41ec06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:22.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.495+0000 7f420a59c640 1 --2- 172.21.15.5:0/3434850996 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f41ec06be60 0x7f41ec06e370 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f41fc0099c0 tx=0x7f41fc003990 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:22.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.499+0000 7f41faffd640 1 -- 172.21.15.5:0/3434850996 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f420404b030 con 0x7f420c0d14f0 2024-05-20T18:36:22.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.720+0000 7f4210b8f640 1 -- 172.21.15.5:0/3434850996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f420c0b5260 con 0x7f420c0d14f0 2024-05-20T18:36:22.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.720+0000 7f41faffd640 1 -- 172.21.15.5:0/3434850996 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5172 (secure 0 0 0) 0x7f420407a000 con 0x7f420c0d14f0 2024-05-20T18:36:22.721 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:36:22.722 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":85,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6758,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/4163145158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":4163145158},{"type":"v1","addr":"172.21.15.63:6834","nonce":4163145158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7528,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3049443532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3049443532},{"type":"v1","addr":"172.21.15.5:6837","nonce":3049443532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7828,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/4180308082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":4180308082},{"type":"v1","addr":"172.21.15.63:6835","nonce":4180308082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:35:30.778930+0000","modified":"2024-05-20T18:36:22.352501+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8325},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8325":{"gid":8325,"name":"c","rank":0,"incarnation":85,"state":"up:replay","state_seq":1,"addr":"172.21.15.5:6835/3552410668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3552410668},{"type":"v1","addr":"172.21.15.5:6835","nonce":3552410668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-20T18:36:22.722 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 85 2024-05-20T18:36:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.722+0000 7f4210b8f640 1 -- 172.21.15.5:0/3434850996 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f41ec06be60 msgr2=0x7f41ec06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.722+0000 7f4210b8f640 1 --2- 172.21.15.5:0/3434850996 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f41ec06be60 0x7f41ec06e370 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f41fc0099c0 tx=0x7f41fc003990 comp rx=0 tx=0).stop 2024-05-20T18:36:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.722+0000 7f4210b8f640 1 -- 172.21.15.5:0/3434850996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f420c0d14f0 msgr2=0x7f420c1580b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.722+0000 7f4210b8f640 1 --2- 172.21.15.5:0/3434850996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f420c0d14f0 0x7f420c1580b0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f420402a2b0 tx=0x7f42040046c0 comp rx=0 tx=0).stop 2024-05-20T18:36:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.722+0000 7f4210b8f640 1 -- 172.21.15.5:0/3434850996 shutdown_connections 2024-05-20T18:36:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.722+0000 7f4210b8f640 1 --2- 172.21.15.5:0/3434850996 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f41ec06be60 0x7f41ec06e370 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.722+0000 7f4210b8f640 1 --2- 172.21.15.5:0/3434850996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f420c0d14f0 0x7f420c1580b0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.722+0000 7f4210b8f640 1 --2- 172.21.15.5:0/3434850996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f420c0d10e0 0x7f420c0ca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.722+0000 7f4210b8f640 1 --2- 172.21.15.5:0/3434850996 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f420c0b7210 0x7f420c0b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.722+0000 7f4210b8f640 1 -- 172.21.15.5:0/3434850996 >> 172.21.15.5:0/3434850996 conn(0x7f420c0b44d0 msgr2=0x7f420c0ccef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.722+0000 7f4210b8f640 1 -- 172.21.15.5:0/3434850996 shutdown_connections 2024-05-20T18:36:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:22.722+0000 7f4210b8f640 1 -- 172.21.15.5:0/3434850996 wait complete. 2024-05-20T18:36:23.138 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 85, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T18:35:30.778930+0000', 'modified': '2024-05-20T18:36:22.352501+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 108, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8325}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8325': {'gid': 8325, 'name': 'c', 'rank': 0, 'incarnation': 85, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.5:6835/3552410668', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6834', 'nonce': 3552410668}, {'type': 'v1', 'addr': '172.21.15.5:6835', 'nonce': 3552410668}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-20T18:36:23.138 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8325:up:replay 2024-05-20T18:36:24.139 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:36:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.267+0000 7f8c3c7c7640 1 Processor -- start 2024-05-20T18:36:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.267+0000 7f8c3c7c7640 1 -- start start 2024-05-20T18:36:24.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.269+0000 7f8c3c7c7640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8c3415f220 0x7f8c340d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.269+0000 7f8c3c7c7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c340d0e00 0x7f8c340cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.269+0000 7f8c3c7c7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8c340cf910 0x7f8c340ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.269+0000 7f8c3c7c7640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f8c340cfd20 con 0x7f8c3415f220 2024-05-20T18:36:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.269+0000 7f8c3c7c7640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8c340cda20 con 0x7f8c340cf910 2024-05-20T18:36:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.269+0000 7f8c3c7c7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8c340cdd00 con 0x7f8c340d0e00 2024-05-20T18:36:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.269+0000 7f8c3a563640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8c3415f220 0x7f8c340d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.269+0000 7f8c3a563640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8c3415f220 0x7f8c340d08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57792/0 (socket says 172.21.15.5:57792) 2024-05-20T18:36:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.269+0000 7f8c3a563640 1 -- 172.21.15.5:0/2167557612 learned_addr learned my addr 172.21.15.5:0/2167557612 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.269+0000 7f8c3ad64640 1 --2- 172.21.15.5:0/2167557612 >> v2:172.21.15.63:3300/0 conn(0x7f8c340cf910 0x7f8c340ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.269+0000 7f8c39d62640 1 --2- 172.21.15.5:0/2167557612 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c340d0e00 0x7f8c340cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.269+0000 7f8c3a563640 1 -- 172.21.15.5:0/2167557612 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c340d0e00 msgr2=0x7f8c340cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.269+0000 7f8c3a563640 1 --2- 172.21.15.5:0/2167557612 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c340d0e00 0x7f8c340cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.269+0000 7f8c3a563640 1 -- 172.21.15.5:0/2167557612 >> v2:172.21.15.63:3300/0 conn(0x7f8c340cf910 msgr2=0x7f8c340ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.269+0000 7f8c3a563640 1 --2- 172.21.15.5:0/2167557612 >> v2:172.21.15.63:3300/0 conn(0x7f8c340cf910 0x7f8c340ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.269+0000 7f8c3a563640 1 -- 172.21.15.5:0/2167557612 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c340cdfe0 con 0x7f8c3415f220 2024-05-20T18:36:24.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.270+0000 7f8c3a563640 1 --2- 172.21.15.5:0/2167557612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c3415f220 0x7f8c340d08c0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f8c3406ce40 tx=0x7f8c3002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32525c1030eb838a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:24.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.270+0000 7f8c39561640 1 -- 172.21.15.5:0/2167557612 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c30037070 con 0x7f8c3415f220 2024-05-20T18:36:24.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.270+0000 7f8c39561640 1 -- 172.21.15.5:0/2167557612 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c30032440 con 0x7f8c3415f220 2024-05-20T18:36:24.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.270+0000 7f8c39561640 1 -- 172.21.15.5:0/2167557612 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c30032810 con 0x7f8c3415f220 2024-05-20T18:36:24.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.271+0000 7f8c3c7c7640 1 -- 172.21.15.5:0/2167557612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c3415f220 msgr2=0x7f8c340d08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:24.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.271+0000 7f8c3c7c7640 1 --2- 172.21.15.5:0/2167557612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c3415f220 0x7f8c340d08c0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f8c3406ce40 tx=0x7f8c3002a140 comp rx=0 tx=0).stop 2024-05-20T18:36:24.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.271+0000 7f8c3c7c7640 1 -- 172.21.15.5:0/2167557612 shutdown_connections 2024-05-20T18:36:24.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.271+0000 7f8c3c7c7640 1 --2- 172.21.15.5:0/2167557612 >> v2:172.21.15.63:3300/0 conn(0x7f8c340cf910 0x7f8c340ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:24.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.271+0000 7f8c3c7c7640 1 --2- 172.21.15.5:0/2167557612 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c340d0e00 0x7f8c340cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:24.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.271+0000 7f8c3c7c7640 1 --2- 172.21.15.5:0/2167557612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c3415f220 0x7f8c340d08c0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:24.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.271+0000 7f8c3c7c7640 1 -- 172.21.15.5:0/2167557612 >> 172.21.15.5:0/2167557612 conn(0x7f8c340b44d0 msgr2=0x7f8c340b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:24.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.271+0000 7f8c3c7c7640 1 -- 172.21.15.5:0/2167557612 shutdown_connections 2024-05-20T18:36:24.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.271+0000 7f8c3c7c7640 1 -- 172.21.15.5:0/2167557612 wait complete. 2024-05-20T18:36:24.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.273+0000 7f8c3c7c7640 1 Processor -- start 2024-05-20T18:36:24.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.273+0000 7f8c3c7c7640 1 -- start start 2024-05-20T18:36:24.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.274+0000 7f8c3c7c7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c340cf910 0x7f8c3415f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:24.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.274+0000 7f8c3c7c7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c340d0e00 0x7f8c3414de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:24.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.274+0000 7f8c3c7c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c3415f220 0x7f8c34150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:24.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.274+0000 7f8c3c7c7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8c340cd800 con 0x7f8c3415f220 2024-05-20T18:36:24.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.274+0000 7f8c3c7c7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8c340cd680 con 0x7f8c340cf910 2024-05-20T18:36:24.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.274+0000 7f8c3c7c7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8c3415f630 con 0x7f8c340d0e00 2024-05-20T18:36:24.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.274+0000 7f8c3ad64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c3415f220 0x7f8c34150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:24.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.275+0000 7f8c3ad64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c3415f220 0x7f8c34150f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57802/0 (socket says 172.21.15.5:57802) 2024-05-20T18:36:24.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.275+0000 7f8c3ad64640 1 -- 172.21.15.5:0/1030573703 learned_addr learned my addr 172.21.15.5:0/1030573703 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:24.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.275+0000 7f8c39d62640 1 --2- 172.21.15.5:0/1030573703 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c340d0e00 0x7f8c3414de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:24.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.275+0000 7f8c3a563640 1 --2- 172.21.15.5:0/1030573703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c340cf910 0x7f8c3415f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:24.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.275+0000 7f8c3ad64640 1 -- 172.21.15.5:0/1030573703 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c340d0e00 msgr2=0x7f8c3414de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:24.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.275+0000 7f8c3ad64640 1 --2- 172.21.15.5:0/1030573703 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c340d0e00 0x7f8c3414de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:24.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.275+0000 7f8c3ad64640 1 -- 172.21.15.5:0/1030573703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c340cf910 msgr2=0x7f8c3415f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:24.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.275+0000 7f8c3ad64640 1 --2- 172.21.15.5:0/1030573703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c340cf910 0x7f8c3415f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:24.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.275+0000 7f8c3ad64640 1 -- 172.21.15.5:0/1030573703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c34160240 con 0x7f8c3415f220 2024-05-20T18:36:24.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.275+0000 7f8c3a563640 1 --2- 172.21.15.5:0/1030573703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c340cf910 0x7f8c3415f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:36:24.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.275+0000 7f8c39d62640 1 --2- 172.21.15.5:0/1030573703 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c340d0e00 0x7f8c3414de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:36:24.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.275+0000 7f8c3ad64640 1 --2- 172.21.15.5:0/1030573703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c3415f220 0x7f8c34150f80 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c007d40 tx=0x7f8c2c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:24.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.275+0000 7f8c2affd640 1 -- 172.21.15.5:0/1030573703 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c014420 con 0x7f8c3415f220 2024-05-20T18:36:24.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.275+0000 7f8c2affd640 1 -- 172.21.15.5:0/1030573703 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c2c019070 con 0x7f8c3415f220 2024-05-20T18:36:24.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.275+0000 7f8c3c7c7640 1 -- 172.21.15.5:0/1030573703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c34160520 con 0x7f8c3415f220 2024-05-20T18:36:24.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.275+0000 7f8c2affd640 1 -- 172.21.15.5:0/1030573703 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c014800 con 0x7f8c3415f220 2024-05-20T18:36:24.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.275+0000 7f8c3c7c7640 1 -- 172.21.15.5:0/1030573703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c34160ae0 con 0x7f8c3415f220 2024-05-20T18:36:24.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.276+0000 7f8c3c7c7640 1 -- 172.21.15.5:0/1030573703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c340d2670 con 0x7f8c3415f220 2024-05-20T18:36:24.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.277+0000 7f8c2affd640 1 -- 172.21.15.5:0/1030573703 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8c2c0149a0 con 0x7f8c3415f220 2024-05-20T18:36:24.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.277+0000 7f8c2affd640 1 --2- 172.21.15.5:0/1030573703 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8c1806be30 0x7f8c1806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:24.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.277+0000 7f8c2affd640 1 -- 172.21.15.5:0/1030573703 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6245+0+0 (secure 0 0 0) 0x7f8c2c08b960 con 0x7f8c3415f220 2024-05-20T18:36:24.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.278+0000 7f8c3a563640 1 --2- 172.21.15.5:0/1030573703 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8c1806be30 0x7f8c1806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:24.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.278+0000 7f8c3a563640 1 --2- 172.21.15.5:0/1030573703 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8c1806be30 0x7f8c1806e340 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8c30034000 tx=0x7f8c3002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:24.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.282+0000 7f8c2affd640 1 -- 172.21.15.5:0/1030573703 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8c340d2670 con 0x7f8c3415f220 2024-05-20T18:36:24.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.504+0000 7f8c3c7c7640 1 -- 172.21.15.5:0/1030573703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c34075b00 con 0x7f8c3415f220 2024-05-20T18:36:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.505+0000 7f8c2affd640 1 -- 172.21.15.5:0/1030573703 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+5173 (secure 0 0 0) 0x7f8c2c059970 con 0x7f8c3415f220 2024-05-20T18:36:24.506 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:36:24.506 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":87,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6758,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/4163145158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":4163145158},{"type":"v1","addr":"172.21.15.63:6834","nonce":4163145158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7528,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3049443532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3049443532},{"type":"v1","addr":"172.21.15.5:6837","nonce":3049443532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7828,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/4180308082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":4180308082},{"type":"v1","addr":"172.21.15.63:6835","nonce":4180308082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:35:30.778930+0000","modified":"2024-05-20T18:36:23.364733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8325},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8325":{"gid":8325,"name":"c","rank":0,"incarnation":85,"state":"up:rejoin","state_seq":16,"addr":"172.21.15.5:6835/3552410668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3552410668},{"type":"v1","addr":"172.21.15.5:6835","nonce":3552410668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-20T18:36:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 87 2024-05-20T18:36:24.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.507+0000 7f8c3c7c7640 1 -- 172.21.15.5:0/1030573703 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8c1806be30 msgr2=0x7f8c1806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:24.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.507+0000 7f8c3c7c7640 1 --2- 172.21.15.5:0/1030573703 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8c1806be30 0x7f8c1806e340 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8c30034000 tx=0x7f8c3002e000 comp rx=0 tx=0).stop 2024-05-20T18:36:24.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.507+0000 7f8c3c7c7640 1 -- 172.21.15.5:0/1030573703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c3415f220 msgr2=0x7f8c34150f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:24.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.507+0000 7f8c3c7c7640 1 --2- 172.21.15.5:0/1030573703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c3415f220 0x7f8c34150f80 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c007d40 tx=0x7f8c2c00e4b0 comp rx=0 tx=0).stop 2024-05-20T18:36:24.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.507+0000 7f8c3c7c7640 1 -- 172.21.15.5:0/1030573703 shutdown_connections 2024-05-20T18:36:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.507+0000 7f8c3c7c7640 1 --2- 172.21.15.5:0/1030573703 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8c1806be30 0x7f8c1806e340 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.507+0000 7f8c3c7c7640 1 --2- 172.21.15.5:0/1030573703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c3415f220 0x7f8c34150f80 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.507+0000 7f8c3c7c7640 1 --2- 172.21.15.5:0/1030573703 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c340d0e00 0x7f8c3414de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.507+0000 7f8c3c7c7640 1 --2- 172.21.15.5:0/1030573703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c340cf910 0x7f8c3415f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.507+0000 7f8c3c7c7640 1 -- 172.21.15.5:0/1030573703 >> 172.21.15.5:0/1030573703 conn(0x7f8c340b44d0 msgr2=0x7f8c340b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.507+0000 7f8c3c7c7640 1 -- 172.21.15.5:0/1030573703 shutdown_connections 2024-05-20T18:36:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:24.507+0000 7f8c3c7c7640 1 -- 172.21.15.5:0/1030573703 wait complete. 2024-05-20T18:36:24.922 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 87, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T18:35:30.778930+0000', 'modified': '2024-05-20T18:36:23.364733+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 108, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8325}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8325': {'gid': 8325, 'name': 'c', 'rank': 0, 'incarnation': 85, 'state': 'up:rejoin', 'state_seq': 16, 'addr': '172.21.15.5:6835/3552410668', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6834', 'nonce': 3552410668}, {'type': 'v1', 'addr': '172.21.15.5:6835', 'nonce': 3552410668}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-20T18:36:24.922 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8325:up:rejoin 2024-05-20T18:36:25.924 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:36:26.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.052+0000 7fa411f77640 1 Processor -- start 2024-05-20T18:36:26.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.052+0000 7fa411f77640 1 -- start start 2024-05-20T18:36:26.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.054+0000 7fa411f77640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa40c0d4400 0x7fa40c15f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:26.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.054+0000 7fa411f77640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa40c06cf00 0x7fa40c06d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:26.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.054+0000 7fa411f77640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa40c0d2780 0x7fa40c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.054+0000 7fa411f77640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fa40c0d3f70 con 0x7fa40c0d2780 2024-05-20T18:36:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.054+0000 7fa411f77640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa40c08d2a0 con 0x7fa40c06cf00 2024-05-20T18:36:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.054+0000 7fa411f77640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa40c06d850 con 0x7fa40c0d4400 2024-05-20T18:36:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.054+0000 7fa40bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa40c0d2780 0x7fa40c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.054+0000 7fa40bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa40c0d2780 0x7fa40c0d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59832/0 (socket says 172.21.15.5:59832) 2024-05-20T18:36:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.054+0000 7fa40bfff640 1 -- 172.21.15.5:0/4125514432 learned_addr learned my addr 172.21.15.5:0/4125514432 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.054+0000 7fa40affd640 1 --2- 172.21.15.5:0/4125514432 >> v2:172.21.15.63:3300/0 conn(0x7fa40c06cf00 0x7fa40c06d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.054+0000 7fa40b7fe640 1 --2- 172.21.15.5:0/4125514432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa40c0d4400 0x7fa40c15f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.054+0000 7fa40bfff640 1 -- 172.21.15.5:0/4125514432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa40c0d4400 msgr2=0x7fa40c15f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.054+0000 7fa40bfff640 1 --2- 172.21.15.5:0/4125514432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa40c0d4400 0x7fa40c15f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.054+0000 7fa40bfff640 1 -- 172.21.15.5:0/4125514432 >> v2:172.21.15.63:3300/0 conn(0x7fa40c06cf00 msgr2=0x7fa40c06d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.054+0000 7fa40bfff640 1 --2- 172.21.15.5:0/4125514432 >> v2:172.21.15.63:3300/0 conn(0x7fa40c06cf00 0x7fa40c06d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.054+0000 7fa40bfff640 1 -- 172.21.15.5:0/4125514432 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40c0d30d0 con 0x7fa40c0d2780 2024-05-20T18:36:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.054+0000 7fa40b7fe640 1 --2- 172.21.15.5:0/4125514432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa40c0d4400 0x7fa40c15f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:36:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.055+0000 7fa40bfff640 1 --2- 172.21.15.5:0/4125514432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa40c0d2780 0x7fa40c0d2b90 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0b77a0 tx=0x7fa40402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2636d7a11c66b26 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.055+0000 7fa40a7fc640 1 -- 172.21.15.5:0/4125514432 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404036070 con 0x7fa40c0d2780 2024-05-20T18:36:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.055+0000 7fa40a7fc640 1 -- 172.21.15.5:0/4125514432 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa404031600 con 0x7fa40c0d2780 2024-05-20T18:36:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.055+0000 7fa40a7fc640 1 -- 172.21.15.5:0/4125514432 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4040319d0 con 0x7fa40c0d2780 2024-05-20T18:36:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.056+0000 7fa411f77640 1 -- 172.21.15.5:0/4125514432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa40c0d2780 msgr2=0x7fa40c0d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.056+0000 7fa411f77640 1 --2- 172.21.15.5:0/4125514432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa40c0d2780 0x7fa40c0d2b90 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0b77a0 tx=0x7fa40402a730 comp rx=0 tx=0).stop 2024-05-20T18:36:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.056+0000 7fa411f77640 1 -- 172.21.15.5:0/4125514432 shutdown_connections 2024-05-20T18:36:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.056+0000 7fa411f77640 1 --2- 172.21.15.5:0/4125514432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa40c0d2780 0x7fa40c0d2b90 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.056+0000 7fa411f77640 1 --2- 172.21.15.5:0/4125514432 >> v2:172.21.15.63:3300/0 conn(0x7fa40c06cf00 0x7fa40c06d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.056+0000 7fa411f77640 1 --2- 172.21.15.5:0/4125514432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa40c0d4400 0x7fa40c15f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.056+0000 7fa411f77640 1 -- 172.21.15.5:0/4125514432 >> 172.21.15.5:0/4125514432 conn(0x7fa40c0d4bb0 msgr2=0x7fa40c0d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.056+0000 7fa411f77640 1 -- 172.21.15.5:0/4125514432 shutdown_connections 2024-05-20T18:36:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.056+0000 7fa411f77640 1 -- 172.21.15.5:0/4125514432 wait complete. 2024-05-20T18:36:26.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.058+0000 7fa411f77640 1 Processor -- start 2024-05-20T18:36:26.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.058+0000 7fa411f77640 1 -- start start 2024-05-20T18:36:26.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.059+0000 7fa411f77640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa40c06cf00 0x7fa40c155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.059+0000 7fa411f77640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa40c0d2780 0x7fa40c158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.059+0000 7fa411f77640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa40c0d4400 0x7fa40c158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.059+0000 7fa411f77640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c0d3640 con 0x7fa40c06cf00 2024-05-20T18:36:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.059+0000 7fa411f77640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c0d4840 con 0x7fa40c0d4400 2024-05-20T18:36:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.059+0000 7fa411f77640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa40c0d3920 con 0x7fa40c0d2780 2024-05-20T18:36:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.059+0000 7fa40b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa40c06cf00 0x7fa40c155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.059+0000 7fa40b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa40c06cf00 0x7fa40c155de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59844/0 (socket says 172.21.15.5:59844) 2024-05-20T18:36:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.059+0000 7fa40b7fe640 1 -- 172.21.15.5:0/3412777172 learned_addr learned my addr 172.21.15.5:0/3412777172 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.060+0000 7fa40bfff640 1 --2- 172.21.15.5:0/3412777172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa40c0d4400 0x7fa40c158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.060+0000 7fa40affd640 1 --2- 172.21.15.5:0/3412777172 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa40c0d2780 0x7fa40c158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.060+0000 7fa40b7fe640 1 -- 172.21.15.5:0/3412777172 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa40c0d2780 msgr2=0x7fa40c158500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.060+0000 7fa40b7fe640 1 --2- 172.21.15.5:0/3412777172 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa40c0d2780 0x7fa40c158500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.060+0000 7fa40b7fe640 1 -- 172.21.15.5:0/3412777172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa40c0d4400 msgr2=0x7fa40c158a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:26.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.060+0000 7fa40b7fe640 1 --2- 172.21.15.5:0/3412777172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa40c0d4400 0x7fa40c158a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.060+0000 7fa40b7fe640 1 -- 172.21.15.5:0/3412777172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40c156860 con 0x7fa40c06cf00 2024-05-20T18:36:26.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.060+0000 7fa40bfff640 1 --2- 172.21.15.5:0/3412777172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa40c0d4400 0x7fa40c158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:36:26.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.060+0000 7fa40b7fe640 1 --2- 172.21.15.5:0/3412777172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa40c06cf00 0x7fa40c155de0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc00c7f0 tx=0x7fa3fc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:26.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.060+0000 7fa3f3fff640 1 -- 172.21.15.5:0/3412777172 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc015070 con 0x7fa40c06cf00 2024-05-20T18:36:26.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.060+0000 7fa3f3fff640 1 -- 172.21.15.5:0/3412777172 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3fc007820 con 0x7fa40c06cf00 2024-05-20T18:36:26.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.060+0000 7fa411f77640 1 -- 172.21.15.5:0/3412777172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa40c156b40 con 0x7fa40c06cf00 2024-05-20T18:36:26.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.060+0000 7fa3f3fff640 1 -- 172.21.15.5:0/3412777172 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc007bd0 con 0x7fa40c06cf00 2024-05-20T18:36:26.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.060+0000 7fa411f77640 1 -- 172.21.15.5:0/3412777172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa40c157100 con 0x7fa40c06cf00 2024-05-20T18:36:26.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.061+0000 7fa411f77640 1 -- 172.21.15.5:0/3412777172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa40c0735a0 con 0x7fa40c06cf00 2024-05-20T18:36:26.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.062+0000 7fa3f3fff640 1 -- 172.21.15.5:0/3412777172 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa3fc018070 con 0x7fa40c06cf00 2024-05-20T18:36:26.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.062+0000 7fa3f3fff640 1 --2- 172.21.15.5:0/3412777172 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa3f806be30 0x7fa3f806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:26.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.062+0000 7fa3f3fff640 1 -- 172.21.15.5:0/3412777172 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6245+0+0 (secure 0 0 0) 0x7fa3fc08b230 con 0x7fa40c06cf00 2024-05-20T18:36:26.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.063+0000 7fa40affd640 1 --2- 172.21.15.5:0/3412777172 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa3f806be30 0x7fa3f806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:26.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.063+0000 7fa40affd640 1 --2- 172.21.15.5:0/3412777172 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa3f806be30 0x7fa3f806e340 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa400006fd0 tx=0x7fa400008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:26.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.067+0000 7fa3f3fff640 1 -- 172.21.15.5:0/3412777172 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa40c0735a0 con 0x7fa40c06cf00 2024-05-20T18:36:26.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.290+0000 7fa411f77640 1 -- 172.21.15.5:0/3412777172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa40c075b00 con 0x7fa40c06cf00 2024-05-20T18:36:26.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.290+0000 7fa3f3fff640 1 -- 172.21.15.5:0/3412777172 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+5180 (secure 0 0 0) 0x7fa3fc059930 con 0x7fa40c06cf00 2024-05-20T18:36:26.292 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:36:26.292 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":88,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6758,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/4163145158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":4163145158},{"type":"v1","addr":"172.21.15.63:6834","nonce":4163145158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7528,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3049443532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3049443532},{"type":"v1","addr":"172.21.15.5:6837","nonce":3049443532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7828,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/4180308082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":4180308082},{"type":"v1","addr":"172.21.15.63:6835","nonce":4180308082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:35:30.778930+0000","modified":"2024-05-20T18:36:25.364533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8325},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8325":{"gid":8325,"name":"c","rank":0,"incarnation":85,"state":"up:active","state_seq":17,"addr":"172.21.15.5:6835/3552410668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3552410668},{"type":"v1","addr":"172.21.15.5:6835","nonce":3552410668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8325,"qdb_cluster":[8325]},"id":7}]} 2024-05-20T18:36:26.292 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 88 2024-05-20T18:36:26.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.292+0000 7fa411f77640 1 -- 172.21.15.5:0/3412777172 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa3f806be30 msgr2=0x7fa3f806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:26.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.292+0000 7fa411f77640 1 --2- 172.21.15.5:0/3412777172 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa3f806be30 0x7fa3f806e340 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa400006fd0 tx=0x7fa400008040 comp rx=0 tx=0).stop 2024-05-20T18:36:26.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.292+0000 7fa411f77640 1 -- 172.21.15.5:0/3412777172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa40c06cf00 msgr2=0x7fa40c155de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:26.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.292+0000 7fa411f77640 1 --2- 172.21.15.5:0/3412777172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa40c06cf00 0x7fa40c155de0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc00c7f0 tx=0x7fa3fc00ccb0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.292+0000 7fa411f77640 1 -- 172.21.15.5:0/3412777172 shutdown_connections 2024-05-20T18:36:26.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.292+0000 7fa411f77640 1 --2- 172.21.15.5:0/3412777172 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa3f806be30 0x7fa3f806e340 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.292+0000 7fa411f77640 1 --2- 172.21.15.5:0/3412777172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa40c0d4400 0x7fa40c158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.292+0000 7fa411f77640 1 --2- 172.21.15.5:0/3412777172 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa40c0d2780 0x7fa40c158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.292+0000 7fa411f77640 1 --2- 172.21.15.5:0/3412777172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa40c06cf00 0x7fa40c155de0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.292+0000 7fa411f77640 1 -- 172.21.15.5:0/3412777172 >> 172.21.15.5:0/3412777172 conn(0x7fa40c0d4bb0 msgr2=0x7fa40c15e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:26.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.292+0000 7fa411f77640 1 -- 172.21.15.5:0/3412777172 shutdown_connections 2024-05-20T18:36:26.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.292+0000 7fa411f77640 1 -- 172.21.15.5:0/3412777172 wait complete. 2024-05-20T18:36:26.708 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 88, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T18:35:30.778930+0000', 'modified': '2024-05-20T18:36:25.364533+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 108, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8325}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8325': {'gid': 8325, 'name': 'c', 'rank': 0, 'incarnation': 85, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.5:6835/3552410668', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6834', 'nonce': 3552410668}, {'type': 'v1', 'addr': '172.21.15.5:6835', 'nonce': 3552410668}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8325, 'qdb_cluster': [8325]} 2024-05-20T18:36:26.708 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-20T18:36:26.708 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-20T18:36:26.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.826+0000 7f79b073e640 1 Processor -- start 2024-05-20T18:36:26.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.826+0000 7f79b073e640 1 -- start start 2024-05-20T18:36:26.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.827+0000 7f79b073e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79a80cf850 0x7f79a80b6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:26.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.828+0000 7f79b073e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79a80d1ac0 con 0x7f79a80d14f0 2024-05-20T18:36:26.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.828+0000 7f79b073e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79a80d1da0 con 0x7f79a80d10e0 2024-05-20T18:36:26.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.828+0000 7f79b073e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f79a80d2080 con 0x7f79a80cf850 2024-05-20T18:36:26.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.828+0000 7f79aecdb640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f79a80d14f0 0x7f79a80b6740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36596/0 (socket says 172.21.15.5:36596) 2024-05-20T18:36:26.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.828+0000 7f79aecdb640 1 -- 172.21.15.5:0/3823760231 learned_addr learned my addr 172.21.15.5:0/3823760231 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:36:26.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.828+0000 7f79adcd9640 1 --2- 172.21.15.5:0/3823760231 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79a80cf850 0x7f79a80b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:26.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.828+0000 7f79ad4d8640 1 -- 172.21.15.5:0/3823760231 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3191894029 0 0) 0x7f79a80d1ac0 con 0x7f79a80d14f0 2024-05-20T18:36:26.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.828+0000 7f79ad4d8640 1 -- 172.21.15.5:0/3823760231 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7994003520 con 0x7f79a80d14f0 2024-05-20T18:36:26.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.828+0000 7f79ad4d8640 1 -- 172.21.15.5:0/3823760231 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3843990328 0 0) 0x7f79a80d1da0 con 0x7f79a80d10e0 2024-05-20T18:36:26.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.828+0000 7f79ad4d8640 1 -- 172.21.15.5:0/3823760231 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79a80d1ac0 con 0x7f79a80d10e0 2024-05-20T18:36:26.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.828+0000 7f79ad4d8640 1 -- 172.21.15.5:0/3823760231 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3388453946 0 0) 0x7f7994003520 con 0x7f79a80d14f0 2024-05-20T18:36:26.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.828+0000 7f79ad4d8640 1 -- 172.21.15.5:0/3823760231 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79a80cf850 msgr2=0x7f79a80b6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:26.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.828+0000 7f79ad4d8640 1 --2- 172.21.15.5:0/3823760231 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79a80cf850 0x7f79a80b6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.828+0000 7f79ad4d8640 1 -- 172.21.15.5:0/3823760231 >> v1:172.21.15.63:6789/0 conn(0x7f79a80d10e0 legacy=0x7f79a80cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:26.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.828+0000 7f79ad4d8640 1 -- 172.21.15.5:0/3823760231 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a80d2360 con 0x7f79a80d14f0 2024-05-20T18:36:26.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.828+0000 7f79ad4d8640 1 -- 172.21.15.5:0/3823760231 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f79a00019f0 con 0x7f79a80d14f0 2024-05-20T18:36:26.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.828+0000 7f79ad4d8640 1 -- 172.21.15.5:0/3823760231 >> v1:172.21.15.5:6789/0 conn(0x7f79a80d14f0 legacy=0x7f79a80b6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:26.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.828+0000 7f79ad4d8640 1 --2- 172.21.15.5:0/3823760231 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79a80cf850 0x7f7994003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:26.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.828+0000 7f79ad4d8640 1 --2- 172.21.15.5:0/3823760231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79940040d0 0x7f7994004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:26.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.828+0000 7f79ad4d8640 1 --2- 172.21.15.5:0/3823760231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7994004d40 0x7f7994005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:26.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.828+0000 7f79ad4d8640 1 -- 172.21.15.5:0/3823760231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f79a80d2080 con 0x7f79940040d0 2024-05-20T18:36:26.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.828+0000 7f79ad4d8640 1 -- 172.21.15.5:0/3823760231 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f799400bb20 con 0x7f7994004d40 2024-05-20T18:36:26.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.828+0000 7f79ad4d8640 1 -- 172.21.15.5:0/3823760231 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f799400bd20 con 0x7f79a80cf850 2024-05-20T18:36:26.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.829+0000 7f79adcd9640 1 --2- 172.21.15.5:0/3823760231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79940040d0 0x7f7994004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:26.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.829+0000 7f79ae4da640 1 --2- 172.21.15.5:0/3823760231 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79a80cf850 0x7f7994003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:26.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.829+0000 7f79aecdb640 1 --2- 172.21.15.5:0/3823760231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7994004d40 0x7f7994005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:26.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.829+0000 7f79adcd9640 1 -- 172.21.15.5:0/3823760231 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79a80cf850 msgr2=0x7f7994003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:26.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.829+0000 7f79adcd9640 1 --2- 172.21.15.5:0/3823760231 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79a80cf850 0x7f7994003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.829+0000 7f79adcd9640 1 -- 172.21.15.5:0/3823760231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7994004d40 msgr2=0x7f7994005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:26.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.829+0000 7f79adcd9640 1 --2- 172.21.15.5:0/3823760231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7994004d40 0x7f7994005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.829+0000 7f79adcd9640 1 -- 172.21.15.5:0/3823760231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7994003520 con 0x7f79940040d0 2024-05-20T18:36:26.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.830+0000 7f79adcd9640 1 --2- 172.21.15.5:0/3823760231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79940040d0 0x7f7994004800 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f79a4009ac0 tx=0x7f7994005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d443601d55ac4b3f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:26.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.830+0000 7f79ad4d8640 1 -- 172.21.15.5:0/3823760231 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a4036070 con 0x7f79940040d0 2024-05-20T18:36:26.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.830+0000 7f79ad4d8640 1 -- 172.21.15.5:0/3823760231 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79a4031400 con 0x7f79940040d0 2024-05-20T18:36:26.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.830+0000 7f79ad4d8640 1 -- 172.21.15.5:0/3823760231 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a4009690 con 0x7f79940040d0 2024-05-20T18:36:26.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.830+0000 7f79b073e640 1 -- 172.21.15.5:0/3823760231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79940040d0 msgr2=0x7f7994004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:26.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.830+0000 7f79b073e640 1 --2- 172.21.15.5:0/3823760231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79940040d0 0x7f7994004800 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f79a4009ac0 tx=0x7f7994005830 comp rx=0 tx=0).stop 2024-05-20T18:36:26.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.831+0000 7f79aecdb640 1 -- 172.21.15.5:0/3823760231 reap_dead start 2024-05-20T18:36:26.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.831+0000 7f79b073e640 1 -- 172.21.15.5:0/3823760231 shutdown_connections 2024-05-20T18:36:26.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.831+0000 7f79b073e640 1 -- 172.21.15.5:0/3823760231 >> 172.21.15.5:0/3823760231 conn(0x7f79a80b44d0 msgr2=0x7f79a80b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:26.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.831+0000 7f79b073e640 1 -- 172.21.15.5:0/3823760231 shutdown_connections 2024-05-20T18:36:26.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.831+0000 7f79b073e640 1 -- 172.21.15.5:0/3823760231 wait complete. 2024-05-20T18:36:26.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.832+0000 7f79b073e640 1 Processor -- start 2024-05-20T18:36:26.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.833+0000 7f79b073e640 1 -- start start 2024-05-20T18:36:26.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.834+0000 7f79b073e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79940040d0 0x7f79a814de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:26.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.834+0000 7f79b073e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79a81513b0 0x7f79a81f8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:26.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.834+0000 7f79b073e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79a81f8ba0 0x7f79a815f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:26.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.834+0000 7f79b073e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f79a80b8440 con 0x7f79a81f8ba0 2024-05-20T18:36:26.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.834+0000 7f79b073e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f79a80b82c0 con 0x7f79940040d0 2024-05-20T18:36:26.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.834+0000 7f79b073e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f79a80709f0 con 0x7f79a81513b0 2024-05-20T18:36:26.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.834+0000 7f79aecdb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79a81f8ba0 0x7f79a815f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:26.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.834+0000 7f79aecdb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79a81f8ba0 0x7f79a815f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59872/0 (socket says 172.21.15.5:59872) 2024-05-20T18:36:26.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.834+0000 7f79aecdb640 1 -- 172.21.15.5:0/15525992 learned_addr learned my addr 172.21.15.5:0/15525992 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:26.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.834+0000 7f79adcd9640 1 --2- 172.21.15.5:0/15525992 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79a81513b0 0x7f79a81f8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:26.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.834+0000 7f79ae4da640 1 --2- 172.21.15.5:0/15525992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79940040d0 0x7f79a814de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:26.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.834+0000 7f79aecdb640 1 -- 172.21.15.5:0/15525992 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79a81513b0 msgr2=0x7f79a81f8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:26.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.834+0000 7f79aecdb640 1 --2- 172.21.15.5:0/15525992 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79a81513b0 0x7f79a81f8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.834+0000 7f79aecdb640 1 -- 172.21.15.5:0/15525992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79940040d0 msgr2=0x7f79a814de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:26.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.834+0000 7f79aecdb640 1 --2- 172.21.15.5:0/15525992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79940040d0 0x7f79a814de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.834+0000 7f79aecdb640 1 -- 172.21.15.5:0/15525992 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a8150650 con 0x7f79a81f8ba0 2024-05-20T18:36:26.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.834+0000 7f79adcd9640 1 --2- 172.21.15.5:0/15525992 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79a81513b0 0x7f79a81f8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:36:26.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.835+0000 7f79aecdb640 1 --2- 172.21.15.5:0/15525992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79a81f8ba0 0x7f79a815f910 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f79a80b58c0 tx=0x7f79a0007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:26.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.835+0000 7f799affd640 1 -- 172.21.15.5:0/15525992 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a000abf0 con 0x7f79a81f8ba0 2024-05-20T18:36:26.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.835+0000 7f799affd640 1 -- 172.21.15.5:0/15525992 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79a0018070 con 0x7f79a81f8ba0 2024-05-20T18:36:26.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.835+0000 7f79b073e640 1 -- 172.21.15.5:0/15525992 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a81f9390 con 0x7f79a81f8ba0 2024-05-20T18:36:26.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.835+0000 7f799affd640 1 -- 172.21.15.5:0/15525992 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a00133f0 con 0x7f79a81f8ba0 2024-05-20T18:36:26.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.835+0000 7f79b073e640 1 -- 172.21.15.5:0/15525992 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a8150bb0 con 0x7f79a81f8ba0 2024-05-20T18:36:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.836+0000 7f799affd640 1 -- 172.21.15.5:0/15525992 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f79a0013590 con 0x7f79a81f8ba0 2024-05-20T18:36:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.837+0000 7f799affd640 1 --2- 172.21.15.5:0/15525992 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f798c06be30 0x7f798c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.837+0000 7f79ae4da640 1 --2- 172.21.15.5:0/15525992 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f798c06be30 0x7f798c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.837+0000 7f799affd640 1 -- 172.21.15.5:0/15525992 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6245+0+0 (secure 0 0 0) 0x7f79a008b6b0 con 0x7f79a81f8ba0 2024-05-20T18:36:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.838+0000 7f79ae4da640 1 --2- 172.21.15.5:0/15525992 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f798c06be30 0x7f798c06e340 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f799c0055b0 tx=0x7f799c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.838+0000 7f79b75ed740 1 Processor -- start 2024-05-20T18:36:26.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.839+0000 7f79b75ed740 1 -- start start 2024-05-20T18:36:26.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.840+0000 7f79b75ed740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c8880270 0x5590c8c4b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:26.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.840+0000 7f79b75ed740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5590c88573b0 0x5590c8c480a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:26.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.840+0000 7f79b75ed740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590c885f590 0x5590c8c21360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:26.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.840+0000 7f79b75ed740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5590c8b844d0 con 0x5590c885f590 2024-05-20T18:36:26.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.840+0000 7f79b75ed740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5590c8b2c3e0 con 0x5590c8880270 2024-05-20T18:36:26.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.840+0000 7f79b75ed740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5590c8b80490 con 0x5590c88573b0 2024-05-20T18:36:26.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.840+0000 7f79aecdb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590c885f590 0x5590c8c21360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:26.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.840+0000 7f79aecdb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590c885f590 0x5590c8c21360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59880/0 (socket says 172.21.15.5:59880) 2024-05-20T18:36:26.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.840+0000 7f79aecdb640 1 -- 172.21.15.5:0/2264657145 learned_addr learned my addr 172.21.15.5:0/2264657145 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:26.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.840+0000 7f79ae4da640 1 --2- 172.21.15.5:0/2264657145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c8880270 0x5590c8c4b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:26.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.840+0000 7f79adcd9640 1 --2- 172.21.15.5:0/2264657145 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5590c88573b0 0x5590c8c480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:26.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.840+0000 7f79aecdb640 1 -- 172.21.15.5:0/2264657145 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5590c88573b0 msgr2=0x5590c8c480a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:26.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.840+0000 7f79aecdb640 1 --2- 172.21.15.5:0/2264657145 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5590c88573b0 0x5590c8c480a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.840+0000 7f79aecdb640 1 -- 172.21.15.5:0/2264657145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c8880270 msgr2=0x5590c8c4b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:26.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.840+0000 7f79aecdb640 1 --2- 172.21.15.5:0/2264657145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c8880270 0x5590c8c4b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.840+0000 7f79aecdb640 1 -- 172.21.15.5:0/2264657145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590c88982d0 con 0x5590c885f590 2024-05-20T18:36:26.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.840+0000 7f79ae4da640 1 --2- 172.21.15.5:0/2264657145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c8880270 0x5590c8c4b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:36:26.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.840+0000 7f79aecdb640 1 --2- 172.21.15.5:0/2264657145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590c885f590 0x5590c8c21360 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f79a0055ba0 tx=0x7f79a0059740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ab78e4c4719a31 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:26.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.840+0000 7f798bfff640 1 -- 172.21.15.5:0/2264657145 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a0062320 con 0x5590c885f590 2024-05-20T18:36:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.841+0000 7f798bfff640 1 -- 172.21.15.5:0/2264657145 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79a0057a40 con 0x5590c885f590 2024-05-20T18:36:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.841+0000 7f798bfff640 1 -- 172.21.15.5:0/2264657145 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a006dbd0 con 0x5590c885f590 2024-05-20T18:36:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.841+0000 7f79b75ed740 1 -- 172.21.15.5:0/2264657145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590c885f590 msgr2=0x5590c8c21360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.841+0000 7f79b75ed740 1 --2- 172.21.15.5:0/2264657145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590c885f590 0x5590c8c21360 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f79a0055ba0 tx=0x7f79a0059740 comp rx=0 tx=0).stop 2024-05-20T18:36:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.841+0000 7f79b75ed740 1 -- 172.21.15.5:0/2264657145 shutdown_connections 2024-05-20T18:36:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.841+0000 7f79b75ed740 1 --2- 172.21.15.5:0/2264657145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590c885f590 0x5590c8c21360 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.841+0000 7f79b75ed740 1 --2- 172.21.15.5:0/2264657145 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5590c88573b0 0x5590c8c480a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.841+0000 7f79b75ed740 1 --2- 172.21.15.5:0/2264657145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c8880270 0x5590c8c4b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.841+0000 7f79b75ed740 1 -- 172.21.15.5:0/2264657145 >> 172.21.15.5:0/2264657145 conn(0x5590c88743c0 msgr2=0x5590c8c20af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.841+0000 7f79b75ed740 1 -- 172.21.15.5:0/2264657145 shutdown_connections 2024-05-20T18:36:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.842+0000 7f79b75ed740 1 -- 172.21.15.5:0/2264657145 wait complete. 2024-05-20T18:36:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.842+0000 7f79b75ed740 1 -- start start 2024-05-20T18:36:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.842+0000 7f79b75ed740 1 Processor -- start 2024-05-20T18:36:26.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.844+0000 7f79b75ed740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c88573b0 0x5590c8c75080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:26.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.844+0000 7f79b75ed740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5590c885f590 0x5590c8c669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:26.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.844+0000 7f79b75ed740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590c8880270 0x5590c8c78b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.844+0000 7f79b75ed740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5590c8b3f8e0 con 0x5590c8880270 2024-05-20T18:36:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.844+0000 7f79b75ed740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5590c8c4d560 con 0x5590c88573b0 2024-05-20T18:36:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.844+0000 7f79b75ed740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5590c8c4b180 con 0x5590c885f590 2024-05-20T18:36:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.844+0000 7f79b75ed740 10 client.0 _ll_register_callbacks cb 0x5590c8b0c9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:36:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.844+0000 7f79aecdb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590c8880270 0x5590c8c78b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.844+0000 7f79aecdb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590c8880270 0x5590c8c78b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59884/0 (socket says 172.21.15.5:59884) 2024-05-20T18:36:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.844+0000 7f79aecdb640 1 -- 172.21.15.5:0/328470221 learned_addr learned my addr 172.21.15.5:0/328470221 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.844+0000 7f79ae4da640 1 --2- 172.21.15.5:0/328470221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c88573b0 0x5590c8c75080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.844+0000 7f79adcd9640 1 --2- 172.21.15.5:0/328470221 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5590c885f590 0x5590c8c669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.844+0000 7f79aecdb640 1 -- 172.21.15.5:0/328470221 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5590c885f590 msgr2=0x5590c8c669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.844+0000 7f79aecdb640 1 --2- 172.21.15.5:0/328470221 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5590c885f590 0x5590c8c669b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.844+0000 7f79aecdb640 1 -- 172.21.15.5:0/328470221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c88573b0 msgr2=0x5590c8c75080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:26.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.844+0000 7f79aecdb640 1 --2- 172.21.15.5:0/328470221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c88573b0 0x5590c8c75080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.844+0000 7f79aecdb640 1 -- 172.21.15.5:0/328470221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590c89626b0 con 0x5590c8880270 2024-05-20T18:36:26.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.845+0000 7f79aecdb640 1 --2- 172.21.15.5:0/328470221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590c8880270 0x5590c8c78b50 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f79a0059440 tx=0x7f79a0001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94ffe1e202ebe3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:26.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.845+0000 7f79897fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:36:26.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.845+0000 7f79897fa640 1 -- 172.21.15.5:0/328470221 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a0071070 con 0x5590c8880270 2024-05-20T18:36:26.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.845+0000 7f79897fa640 1 -- 172.21.15.5:0/328470221 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 6245+0+0 (secure 0 0 0) 0x7f79a0061e90 con 0x5590c8880270 2024-05-20T18:36:26.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.845+0000 7f79b75ed740 1 -- 172.21.15.5:0/328470221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79a0057a40 con 0x5590c8880270 2024-05-20T18:36:26.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.845+0000 7f79897fa640 1 -- 172.21.15.5:0/328470221 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f79a0078050 con 0x5590c8880270 2024-05-20T18:36:26.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.845+0000 7f79b75ed740 10 client.8772 fetch_fsmap learned FSMap version 88 2024-05-20T18:36:26.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.845+0000 7f79b75ed740 1 -- 172.21.15.5:0/328470221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x5590c8c4d2a0 con 0x5590c8880270 2024-05-20T18:36:26.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.845+0000 7f79897fa640 1 -- 172.21.15.5:0/328470221 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79a0057a40 con 0x5590c8880270 2024-05-20T18:36:26.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.845+0000 7f79897fa640 1 -- 172.21.15.5:0/328470221 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7f79a0058b90 con 0x5590c8880270 2024-05-20T18:36:26.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.845+0000 7f79b75ed740 10 client.8772 fetch_fsmap finished waiting for FSMap version 88 2024-05-20T18:36:26.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.845+0000 7f79b75ed740 10 client.8772 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-20T18:36:26.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.845+0000 7f79b75ed740 20 client.8772 populate_metadata read hostname 'smithi005' 2024-05-20T18:36:26.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.845+0000 7f79b75ed740 1 --2- 172.21.15.5:0/328470221 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x5590c883ee40 0x5590c8c40290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:26.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.845+0000 7f79b75ed740 4 client.8772 mds_command: new command op to 8325 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:36:26.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.845+0000 7f79b75ed740 1 -- 172.21.15.5:0/328470221 --> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590c8b91c00 con 0x5590c883ee40 2024-05-20T18:36:26.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.845+0000 7f79ae4da640 1 --2- 172.21.15.5:0/328470221 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x5590c883ee40 0x5590c8c40290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:26.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.846+0000 7f79ae4da640 1 --2- 172.21.15.5:0/328470221 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x5590c883ee40 0x5590c8c40290 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=630bca83aa0b825a server_cookie=ea24b5c3f7a398f7 in_seq=0 out_seq=0 2024-05-20T18:36:26.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.846+0000 7f79897fa640 10 client.8772 ms_handle_connect on v2:172.21.15.5:6834/3552410668 2024-05-20T18:36:26.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.846+0000 7f79897fa640 1 -- 172.21.15.5:0/328470221 <== mds.0 v2:172.21.15.5:6834/3552410668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5590c8b91c00 con 0x5590c883ee40 2024-05-20T18:36:26.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.846+0000 7f79897fa640 10 client.8772 handle_command_reply: tid=0 2024-05-20T18:36:26.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.846+0000 7f79b75ed740 1 client.8772 shutdown 2024-05-20T18:36:26.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.847+0000 7f79b75ed740 1 -- 172.21.15.5:0/328470221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590c8880270 msgr2=0x5590c8c78b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:26.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.847+0000 7f79b75ed740 1 --2- 172.21.15.5:0/328470221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590c8880270 0x5590c8c78b50 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f79a0059440 tx=0x7f79a0001360 comp rx=0 tx=0).stop 2024-05-20T18:36:26.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.847+0000 7f79b75ed740 1 -- 172.21.15.5:0/328470221 shutdown_connections 2024-05-20T18:36:26.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.847+0000 7f79b75ed740 1 --2- 172.21.15.5:0/328470221 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x5590c883ee40 0x5590c8c40290 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.847+0000 7f79b75ed740 1 --2- 172.21.15.5:0/328470221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590c8880270 0x5590c8c78b50 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.847+0000 7f79b75ed740 1 --2- 172.21.15.5:0/328470221 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5590c885f590 0x5590c8c669b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.847+0000 7f79b75ed740 1 --2- 172.21.15.5:0/328470221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c88573b0 0x5590c8c75080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.847+0000 7f79897fa640 0 client.8772 ms_handle_reset on v2:172.21.15.5:6834/3552410668 2024-05-20T18:36:26.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.847+0000 7f79b75ed740 1 -- 172.21.15.5:0/328470221 >> 172.21.15.5:0/328470221 conn(0x5590c88743c0 msgr2=0x5590c8c20af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:26.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.847+0000 7f79b75ed740 1 -- 172.21.15.5:0/328470221 shutdown_connections 2024-05-20T18:36:26.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.847+0000 7f79b75ed740 1 -- 172.21.15.5:0/328470221 wait complete. 2024-05-20T18:36:26.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.847+0000 7f79b75ed740 20 client.8772 trim_cache size 0 max 16384 2024-05-20T18:36:26.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.866+0000 7f79b75ed740 1 Processor -- start 2024-05-20T18:36:26.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.866+0000 7f79b75ed740 1 -- start start 2024-05-20T18:36:26.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.867+0000 7f79b75ed740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590c88573b0 0x5590c8c4b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:26.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.867+0000 7f79b75ed740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c885f590 0x5590c8c4bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:26.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.867+0000 7f79b75ed740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5590c8880270 0x5590c8c4a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:26.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.867+0000 7f79b75ed740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f79a8201980 con 0x5590c88573b0 2024-05-20T18:36:26.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.867+0000 7f79b75ed740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f79a8200ce0 con 0x5590c885f590 2024-05-20T18:36:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.867+0000 7f79b75ed740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f79a8201330 con 0x5590c8880270 2024-05-20T18:36:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.868+0000 7f79ae4da640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590c88573b0 0x5590c8c4b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.868+0000 7f79ae4da640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590c88573b0 0x5590c8c4b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59900/0 (socket says 172.21.15.5:59900) 2024-05-20T18:36:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.868+0000 7f79ae4da640 1 -- 172.21.15.5:0/3418158872 learned_addr learned my addr 172.21.15.5:0/3418158872 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.868+0000 7f79aecdb640 1 --2- 172.21.15.5:0/3418158872 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5590c8880270 0x5590c8c4a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.868+0000 7f79adcd9640 1 --2- 172.21.15.5:0/3418158872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c885f590 0x5590c8c4bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.868+0000 7f79ae4da640 1 -- 172.21.15.5:0/3418158872 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5590c8880270 msgr2=0x5590c8c4a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.868+0000 7f79ae4da640 1 --2- 172.21.15.5:0/3418158872 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5590c8880270 0x5590c8c4a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.868+0000 7f79ae4da640 1 -- 172.21.15.5:0/3418158872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c885f590 msgr2=0x5590c8c4bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.868+0000 7f79ae4da640 1 --2- 172.21.15.5:0/3418158872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c885f590 0x5590c8c4bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.868+0000 7f79ae4da640 1 -- 172.21.15.5:0/3418158872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590c8c48800 con 0x5590c88573b0 2024-05-20T18:36:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.868+0000 7f79adcd9640 1 --2- 172.21.15.5:0/3418158872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c885f590 0x5590c8c4bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:36:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.868+0000 7f79ae4da640 1 --2- 172.21.15.5:0/3418158872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590c88573b0 0x5590c8c4b700 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f799c0055e0 tx=0x7f799c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=288036d4d11f43c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.868+0000 7f798b7fe640 1 -- 172.21.15.5:0/3418158872 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c01b040 con 0x5590c88573b0 2024-05-20T18:36:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.868+0000 7f798b7fe640 1 -- 172.21.15.5:0/3418158872 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f799c01c040 con 0x5590c88573b0 2024-05-20T18:36:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.868+0000 7f798b7fe640 1 -- 172.21.15.5:0/3418158872 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c016b40 con 0x5590c88573b0 2024-05-20T18:36:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.869+0000 7f79b75ed740 1 -- 172.21.15.5:0/3418158872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590c88573b0 msgr2=0x5590c8c4b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.869+0000 7f79b75ed740 1 --2- 172.21.15.5:0/3418158872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590c88573b0 0x5590c8c4b700 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f799c0055e0 tx=0x7f799c014000 comp rx=0 tx=0).stop 2024-05-20T18:36:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.869+0000 7f79b75ed740 1 -- 172.21.15.5:0/3418158872 shutdown_connections 2024-05-20T18:36:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.869+0000 7f79b75ed740 1 --2- 172.21.15.5:0/3418158872 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5590c8880270 0x5590c8c4a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.869+0000 7f79b75ed740 1 --2- 172.21.15.5:0/3418158872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c885f590 0x5590c8c4bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.869+0000 7f79b75ed740 1 --2- 172.21.15.5:0/3418158872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590c88573b0 0x5590c8c4b700 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.869+0000 7f79b75ed740 1 -- 172.21.15.5:0/3418158872 >> 172.21.15.5:0/3418158872 conn(0x5590c88743c0 msgr2=0x5590c8c74e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.869+0000 7f79b75ed740 1 -- 172.21.15.5:0/3418158872 shutdown_connections 2024-05-20T18:36:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.869+0000 7f79b75ed740 1 -- 172.21.15.5:0/3418158872 wait complete. 2024-05-20T18:36:26.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.870+0000 7f79b75ed740 1 -- start start 2024-05-20T18:36:26.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.870+0000 7f79b75ed740 1 Processor -- start 2024-05-20T18:36:26.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.871+0000 7f79b75ed740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c88573b0 0x5590c8c4a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:26.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.871+0000 7f79b75ed740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5590c885f590 0x5590c8c4a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:26.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.871+0000 7f79b75ed740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590c8880270 0x5590c8c779b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:26.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.871+0000 7f79b75ed740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5590c8c4ae50 con 0x5590c8880270 2024-05-20T18:36:26.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.871+0000 7f79b75ed740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5590c8b3f8e0 con 0x5590c88573b0 2024-05-20T18:36:26.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.871+0000 7f79b75ed740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5590c8c4c650 con 0x5590c885f590 2024-05-20T18:36:26.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.871+0000 7f79b75ed740 10 client.0 _ll_register_callbacks cb 0x7f79a0059130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:36:26.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.872+0000 7f79aecdb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590c8880270 0x5590c8c779b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:26.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.872+0000 7f79aecdb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590c8880270 0x5590c8c779b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59914/0 (socket says 172.21.15.5:59914) 2024-05-20T18:36:26.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.872+0000 7f79aecdb640 1 -- 172.21.15.5:0/2700974812 learned_addr learned my addr 172.21.15.5:0/2700974812 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:26.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.872+0000 7f79adcd9640 1 --2- 172.21.15.5:0/2700974812 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5590c885f590 0x5590c8c4a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:26.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.872+0000 7f79ae4da640 1 --2- 172.21.15.5:0/2700974812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c88573b0 0x5590c8c4a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:26.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.872+0000 7f79aecdb640 1 -- 172.21.15.5:0/2700974812 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5590c885f590 msgr2=0x5590c8c4a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:26.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.872+0000 7f79aecdb640 1 --2- 172.21.15.5:0/2700974812 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5590c885f590 0x5590c8c4a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.872+0000 7f79aecdb640 1 -- 172.21.15.5:0/2700974812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c88573b0 msgr2=0x5590c8c4a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.872+0000 7f79aecdb640 1 --2- 172.21.15.5:0/2700974812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c88573b0 0x5590c8c4a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.872+0000 7f79aecdb640 1 -- 172.21.15.5:0/2700974812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590c8c4e0b0 con 0x5590c8880270 2024-05-20T18:36:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.872+0000 7f79aecdb640 1 --2- 172.21.15.5:0/2700974812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590c8880270 0x5590c8c779b0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f79a005dcb0 tx=0x7f79a0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5f0e672bdaea4a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.872+0000 7f79897fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:36:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.872+0000 7f79897fa640 1 -- 172.21.15.5:0/2700974812 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a00559e0 con 0x5590c8880270 2024-05-20T18:36:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.872+0000 7f79897fa640 1 -- 172.21.15.5:0/2700974812 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 6245+0+0 (secure 0 0 0) 0x7f79a0078ae0 con 0x5590c8880270 2024-05-20T18:36:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.872+0000 7f79b75ed740 1 -- 172.21.15.5:0/2700974812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f799c01c040 con 0x5590c8880270 2024-05-20T18:36:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.872+0000 7f79897fa640 1 -- 172.21.15.5:0/2700974812 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f79a0078ce0 con 0x5590c8880270 2024-05-20T18:36:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.872+0000 7f79b75ed740 10 client.8778 fetch_fsmap learned FSMap version 88 2024-05-20T18:36:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.872+0000 7f79b75ed740 1 -- 172.21.15.5:0/2700974812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x5590c8c1a630 con 0x5590c8880270 2024-05-20T18:36:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.872+0000 7f79897fa640 1 -- 172.21.15.5:0/2700974812 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f799c01c040 con 0x5590c8880270 2024-05-20T18:36:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.873+0000 7f79897fa640 1 -- 172.21.15.5:0/2700974812 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7f79a0055490 con 0x5590c8880270 2024-05-20T18:36:26.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.873+0000 7f79b75ed740 10 client.8778 fetch_fsmap finished waiting for FSMap version 88 2024-05-20T18:36:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.873+0000 7f79b75ed740 10 client.8778 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-20T18:36:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.873+0000 7f79b75ed740 20 client.8778 populate_metadata read hostname 'smithi005' 2024-05-20T18:36:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.873+0000 7f79b75ed740 1 --2- 172.21.15.5:0/2700974812 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x5590c8c782a0 0x5590c8c67210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.873+0000 7f79b75ed740 4 client.8778 mds_command: new command op to 8325 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T18:36:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.873+0000 7f79b75ed740 1 -- 172.21.15.5:0/2700974812 --> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] -- command(tid 0: {"prefix": "status"}) -- 0x5590c8c48dc0 con 0x5590c8c782a0 2024-05-20T18:36:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.873+0000 7f79ae4da640 1 --2- 172.21.15.5:0/2700974812 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x5590c8c782a0 0x5590c8c67210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.873+0000 7f79897fa640 10 client.8778 ms_handle_connect on v2:172.21.15.5:6834/3552410668 2024-05-20T18:36:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.873+0000 7f79ae4da640 1 --2- 172.21.15.5:0/2700974812 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x5590c8c782a0 0x5590c8c67210 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9abeb4145148f5b3 server_cookie=d7377ec99bef2ffe in_seq=0 out_seq=0 2024-05-20T18:36:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.873+0000 7f79897fa640 1 -- 172.21.15.5:0/2700974812 <== mds.0 v2:172.21.15.5:6834/3552410668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x5590c8c48dc0 con 0x5590c8c782a0 2024-05-20T18:36:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.873+0000 7f79897fa640 10 client.8778 handle_command_reply: tid=0 2024-05-20T18:36:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.873+0000 7f79b75ed740 1 client.8778 shutdown 2024-05-20T18:36:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.874+0000 7f79b75ed740 1 -- 172.21.15.5:0/2700974812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590c8880270 msgr2=0x5590c8c779b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:26.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.874+0000 7f79b75ed740 1 --2- 172.21.15.5:0/2700974812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590c8880270 0x5590c8c779b0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f79a005dcb0 tx=0x7f79a0009040 comp rx=0 tx=0).stop 2024-05-20T18:36:26.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.874+0000 7f79b75ed740 1 -- 172.21.15.5:0/2700974812 shutdown_connections 2024-05-20T18:36:26.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.874+0000 7f79b75ed740 1 --2- 172.21.15.5:0/2700974812 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x5590c8c782a0 0x5590c8c67210 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.874+0000 7f79b75ed740 1 --2- 172.21.15.5:0/2700974812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590c8880270 0x5590c8c779b0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.874+0000 7f79b75ed740 1 --2- 172.21.15.5:0/2700974812 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5590c885f590 0x5590c8c4a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.874+0000 7f79b75ed740 1 --2- 172.21.15.5:0/2700974812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5590c88573b0 0x5590c8c4a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.874+0000 7f79897fa640 0 client.8778 ms_handle_reset on v2:172.21.15.5:6834/3552410668 2024-05-20T18:36:26.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.874+0000 7f79b75ed740 1 -- 172.21.15.5:0/2700974812 >> 172.21.15.5:0/2700974812 conn(0x5590c88743c0 msgr2=0x5590c8c4ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:26.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.874+0000 7f79b75ed740 1 -- 172.21.15.5:0/2700974812 shutdown_connections 2024-05-20T18:36:26.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.874+0000 7f79b75ed740 1 -- 172.21.15.5:0/2700974812 wait complete. 2024-05-20T18:36:26.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.874+0000 7f79b75ed740 20 client.8778 trim_cache size 0 max 16384 2024-05-20T18:36:26.881 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:36:26.882 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "399ab250-fd3c-47fa-a1c8-25e8149c9cac", 2024-05-20T18:36:26.882 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-20T18:36:26.882 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 8325, 2024-05-20T18:36:26.882 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-20T18:36:26.882 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-20T18:36:26.882 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-20T18:36:26.882 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 4.5194062700000002, 2024-05-20T18:36:26.882 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 88, 2024-05-20T18:36:26.882 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 108, 2024-05-20T18:36:26.882 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 108, 2024-05-20T18:36:26.882 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 60.425958346999998 2024-05-20T18:36:26.882 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:36:26.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.874+0000 7f79b073e640 1 -- 172.21.15.5:0/15525992 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f798c06be30 msgr2=0x7f798c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:26.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.874+0000 7f79b073e640 1 --2- 172.21.15.5:0/15525992 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f798c06be30 0x7f798c06e340 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f799c0055b0 tx=0x7f799c009040 comp rx=0 tx=0).stop 2024-05-20T18:36:26.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.874+0000 7f79b073e640 1 -- 172.21.15.5:0/15525992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79a81f8ba0 msgr2=0x7f79a815f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:26.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.874+0000 7f79b073e640 1 --2- 172.21.15.5:0/15525992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79a81f8ba0 0x7f79a815f910 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f79a80b58c0 tx=0x7f79a0007bd0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.874+0000 7f79b073e640 1 -- 172.21.15.5:0/15525992 shutdown_connections 2024-05-20T18:36:26.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.874+0000 7f79b073e640 1 --2- 172.21.15.5:0/15525992 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f798c06be30 0x7f798c06e340 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.874+0000 7f79b073e640 1 --2- 172.21.15.5:0/15525992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79a81f8ba0 0x7f79a815f910 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.874+0000 7f79b073e640 1 --2- 172.21.15.5:0/15525992 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79a81513b0 0x7f79a81f8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.874+0000 7f79b073e640 1 --2- 172.21.15.5:0/15525992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79940040d0 0x7f79a814de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:26.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.874+0000 7f79b073e640 1 -- 172.21.15.5:0/15525992 >> 172.21.15.5:0/15525992 conn(0x7f79a80b44d0 msgr2=0x7f79a80b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:26.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.874+0000 7f79b073e640 1 -- 172.21.15.5:0/15525992 shutdown_connections 2024-05-20T18:36:26.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:26.874+0000 7f79b073e640 1 -- 172.21.15.5:0/15525992 wait complete. 2024-05-20T18:36:26.887 DEBUG:teuthology.orchestra.run.smithi005:> sudo daemon-helper kill ceph --cluster ceph -w 2024-05-20T18:36:26.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:36:26.889 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-05-20T18:36:27.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.008+0000 7f705a94c640 1 Processor -- start 2024-05-20T18:36:27.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.008+0000 7f705a94c640 1 -- start start 2024-05-20T18:36:27.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.010+0000 7f705a94c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70540d3510 0x7f70540b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.010+0000 7f705a94c640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f70540d1830 0x7f70540ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.010+0000 7f705a94c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f70540cea20 0x7f70540b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.010+0000 7f705a94c640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f70540d0160 con 0x7f70540cea20 2024-05-20T18:36:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.010+0000 7f705a94c640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f70540cda00 con 0x7f70540d1830 2024-05-20T18:36:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.010+0000 7f705a94c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f70540cdce0 con 0x7f70540d3510 2024-05-20T18:36:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.010+0000 7f7058ee9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f70540cea20 0x7f70540b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.010+0000 7f7058ee9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f70540cea20 0x7f70540b21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59916/0 (socket says 172.21.15.5:59916) 2024-05-20T18:36:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.010+0000 7f7058ee9640 1 -- 172.21.15.5:0/2563485116 learned_addr learned my addr 172.21.15.5:0/2563485116 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.010+0000 7f7053fff640 1 --2- 172.21.15.5:0/2563485116 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70540d3510 0x7f70540b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.010+0000 7f70537fe640 1 --2- 172.21.15.5:0/2563485116 >> v2:172.21.15.63:3300/0 conn(0x7f70540d1830 0x7f70540ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.010+0000 7f7058ee9640 1 -- 172.21.15.5:0/2563485116 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70540d3510 msgr2=0x7f70540b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:27.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.010+0000 7f7058ee9640 1 --2- 172.21.15.5:0/2563485116 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70540d3510 0x7f70540b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.010+0000 7f7058ee9640 1 -- 172.21.15.5:0/2563485116 >> v2:172.21.15.63:3300/0 conn(0x7f70540d1830 msgr2=0x7f70540ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:27.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.010+0000 7f7058ee9640 1 --2- 172.21.15.5:0/2563485116 >> v2:172.21.15.63:3300/0 conn(0x7f70540d1830 0x7f70540ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.010+0000 7f7058ee9640 1 -- 172.21.15.5:0/2563485116 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70540cdfc0 con 0x7f70540cea20 2024-05-20T18:36:27.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.011+0000 7f7058ee9640 1 --2- 172.21.15.5:0/2563485116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70540cea20 0x7f70540b21d0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f7054074eb0 tx=0x7f704c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240e0ab31b895906 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:27.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.011+0000 7f7052ffd640 1 -- 172.21.15.5:0/2563485116 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c036070 con 0x7f70540cea20 2024-05-20T18:36:27.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.011+0000 7f7052ffd640 1 -- 172.21.15.5:0/2563485116 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f704c031600 con 0x7f70540cea20 2024-05-20T18:36:27.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.011+0000 7f7052ffd640 1 -- 172.21.15.5:0/2563485116 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c0319d0 con 0x7f70540cea20 2024-05-20T18:36:27.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.012+0000 7f705a94c640 1 -- 172.21.15.5:0/2563485116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70540cea20 msgr2=0x7f70540b21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:27.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.012+0000 7f705a94c640 1 --2- 172.21.15.5:0/2563485116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70540cea20 0x7f70540b21d0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f7054074eb0 tx=0x7f704c02a610 comp rx=0 tx=0).stop 2024-05-20T18:36:27.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.012+0000 7f705a94c640 1 -- 172.21.15.5:0/2563485116 shutdown_connections 2024-05-20T18:36:27.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.012+0000 7f705a94c640 1 --2- 172.21.15.5:0/2563485116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70540cea20 0x7f70540b21d0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.012+0000 7f705a94c640 1 --2- 172.21.15.5:0/2563485116 >> v2:172.21.15.63:3300/0 conn(0x7f70540d1830 0x7f70540ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.012+0000 7f705a94c640 1 --2- 172.21.15.5:0/2563485116 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70540d3510 0x7f70540b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.012+0000 7f705a94c640 1 -- 172.21.15.5:0/2563485116 >> 172.21.15.5:0/2563485116 conn(0x7f70540d29f0 msgr2=0x7f70540d2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:27.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.012+0000 7f705a94c640 1 -- 172.21.15.5:0/2563485116 shutdown_connections 2024-05-20T18:36:27.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.012+0000 7f705a94c640 1 -- 172.21.15.5:0/2563485116 wait complete. 2024-05-20T18:36:27.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.014+0000 7f705a94c640 1 Processor -- start 2024-05-20T18:36:27.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.014+0000 7f705a94c640 1 -- start start 2024-05-20T18:36:27.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.015+0000 7f705a94c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70540cea20 0x7f705414deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.015+0000 7f705a94c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70540d1830 0x7f70540c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.016+0000 7f705a94c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70540d3510 0x7f70540c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.016+0000 7f705a94c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f70540cff00 con 0x7f70540cea20 2024-05-20T18:36:27.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.016+0000 7f705a94c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f70540cfd80 con 0x7f70540d1830 2024-05-20T18:36:27.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.016+0000 7f705a94c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f70540b1e20 con 0x7f70540d3510 2024-05-20T18:36:27.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.016+0000 7f7053fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70540cea20 0x7f705414deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.016+0000 7f7053fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70540cea20 0x7f705414deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59918/0 (socket says 172.21.15.5:59918) 2024-05-20T18:36:27.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.016+0000 7f7053fff640 1 -- 172.21.15.5:0/3759247792 learned_addr learned my addr 172.21.15.5:0/3759247792 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:27.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.016+0000 7f70537fe640 1 --2- 172.21.15.5:0/3759247792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70540d1830 0x7f70540c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.016+0000 7f7058ee9640 1 --2- 172.21.15.5:0/3759247792 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70540d3510 0x7f70540c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.016+0000 7f7053fff640 1 -- 172.21.15.5:0/3759247792 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70540d3510 msgr2=0x7f70540c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:27.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.016+0000 7f7053fff640 1 --2- 172.21.15.5:0/3759247792 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70540d3510 0x7f70540c6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.016+0000 7f7053fff640 1 -- 172.21.15.5:0/3759247792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70540d1830 msgr2=0x7f70540c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:27.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.016+0000 7f7053fff640 1 --2- 172.21.15.5:0/3759247792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70540d1830 0x7f70540c8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.016+0000 7f7053fff640 1 -- 172.21.15.5:0/3759247792 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705414e3f0 con 0x7f70540cea20 2024-05-20T18:36:27.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.016+0000 7f7058ee9640 1 --2- 172.21.15.5:0/3759247792 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70540d3510 0x7f70540c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:36:27.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.016+0000 7f7053fff640 1 --2- 172.21.15.5:0/3759247792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70540cea20 0x7f705414deb0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f704400e870 tx=0x7f704400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:27.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.016+0000 7f7050ff9640 1 -- 172.21.15.5:0/3759247792 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7044017070 con 0x7f70540cea20 2024-05-20T18:36:27.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.016+0000 7f705a94c640 1 -- 172.21.15.5:0/3759247792 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70541f6b70 con 0x7f70540cea20 2024-05-20T18:36:27.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.017+0000 7f7050ff9640 1 -- 172.21.15.5:0/3759247792 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f704400c940 con 0x7f70540cea20 2024-05-20T18:36:27.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.017+0000 7f7050ff9640 1 -- 172.21.15.5:0/3759247792 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704400cd30 con 0x7f70540cea20 2024-05-20T18:36:27.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.017+0000 7f705a94c640 1 -- 172.21.15.5:0/3759247792 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70541f7130 con 0x7f70540cea20 2024-05-20T18:36:27.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.018+0000 7f7050ff9640 1 -- 172.21.15.5:0/3759247792 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f704401a070 con 0x7f70540cea20 2024-05-20T18:36:27.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.019+0000 7f7050ff9640 1 --2- 172.21.15.5:0/3759247792 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f704006be30 0x7f704006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.019+0000 7f7050ff9640 1 -- 172.21.15.5:0/3759247792 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6245+0+0 (secure 0 0 0) 0x7f704408b340 con 0x7f70540cea20 2024-05-20T18:36:27.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.019+0000 7f70537fe640 1 --2- 172.21.15.5:0/3759247792 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f704006be30 0x7f704006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.020+0000 7f70537fe640 1 --2- 172.21.15.5:0/3759247792 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f704006be30 0x7f704006e340 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f70541588b0 tx=0x7f7048009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:27.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.020+0000 7f70617f7740 1 Processor -- start 2024-05-20T18:36:27.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.020+0000 7f70617f7740 1 -- start start 2024-05-20T18:36:27.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.021+0000 7f70617f7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641213ea410 0x5641217b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.021+0000 7f70617f7740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641213c1550 0x5641217d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.021+0000 7f70617f7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641213c9730 0x5641217d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.021+0000 7f70617f7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5641216ee990 con 0x5641213c9730 2024-05-20T18:36:27.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.021+0000 7f70617f7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5641216965b0 con 0x5641213ea410 2024-05-20T18:36:27.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.021+0000 7f70617f7740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5641216ea950 con 0x5641213c1550 2024-05-20T18:36:27.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.021+0000 7f7058ee9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641213c9730 0x5641217d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.022+0000 7f7058ee9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641213c9730 0x5641217d06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59926/0 (socket says 172.21.15.5:59926) 2024-05-20T18:36:27.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.022+0000 7f7058ee9640 1 -- 172.21.15.5:0/948461704 learned_addr learned my addr 172.21.15.5:0/948461704 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:27.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.022+0000 7f70537fe640 1 --2- 172.21.15.5:0/948461704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641213c1550 0x5641217d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.022+0000 7f7053fff640 1 --2- 172.21.15.5:0/948461704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641213ea410 0x5641217b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.022+0000 7f7058ee9640 1 -- 172.21.15.5:0/948461704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641213c1550 msgr2=0x5641217d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:27.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.022+0000 7f7058ee9640 1 --2- 172.21.15.5:0/948461704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641213c1550 0x5641217d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.022+0000 7f7058ee9640 1 -- 172.21.15.5:0/948461704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641213ea410 msgr2=0x5641217b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:27.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.022+0000 7f7058ee9640 1 --2- 172.21.15.5:0/948461704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641213ea410 0x5641217b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.022+0000 7f7058ee9640 1 -- 172.21.15.5:0/948461704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641214ca7e0 con 0x5641213c9730 2024-05-20T18:36:27.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.022+0000 7f7053fff640 1 --2- 172.21.15.5:0/948461704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641213ea410 0x5641217b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:36:27.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.022+0000 7f7058ee9640 1 --2- 172.21.15.5:0/948461704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641213c9730 0x5641217d06a0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f704c02ac00 tx=0x7f704c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1832a53f69339d54 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:27.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.022+0000 7f7039ffb640 1 -- 172.21.15.5:0/948461704 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c041070 con 0x5641213c9730 2024-05-20T18:36:27.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.022+0000 7f7039ffb640 1 -- 172.21.15.5:0/948461704 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f704c03c640 con 0x5641213c9730 2024-05-20T18:36:27.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.022+0000 7f7039ffb640 1 -- 172.21.15.5:0/948461704 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c036040 con 0x5641213c9730 2024-05-20T18:36:27.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.023+0000 7f70617f7740 1 -- 172.21.15.5:0/948461704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641213c9730 msgr2=0x5641217d06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:27.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.023+0000 7f70617f7740 1 --2- 172.21.15.5:0/948461704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641213c9730 0x5641217d06a0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f704c02ac00 tx=0x7f704c02e000 comp rx=0 tx=0).stop 2024-05-20T18:36:27.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.023+0000 7f70617f7740 1 -- 172.21.15.5:0/948461704 shutdown_connections 2024-05-20T18:36:27.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.023+0000 7f70617f7740 1 --2- 172.21.15.5:0/948461704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641213c9730 0x5641217d06a0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.023+0000 7f70617f7740 1 --2- 172.21.15.5:0/948461704 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641213c1550 0x5641217d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.023+0000 7f70617f7740 1 --2- 172.21.15.5:0/948461704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641213ea410 0x5641217b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.023+0000 7f70617f7740 1 -- 172.21.15.5:0/948461704 >> 172.21.15.5:0/948461704 conn(0x5641213de560 msgr2=0x56412178aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:27.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.023+0000 7f70617f7740 1 -- 172.21.15.5:0/948461704 shutdown_connections 2024-05-20T18:36:27.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.023+0000 7f70617f7740 1 -- 172.21.15.5:0/948461704 wait complete. 2024-05-20T18:36:27.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.024+0000 7f70617f7740 1 -- start start 2024-05-20T18:36:27.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.025+0000 7f70617f7740 1 Processor -- start 2024-05-20T18:36:27.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.026+0000 7f70617f7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641213c1550 0x5641217afe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.026+0000 7f70617f7740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641213c9730 0x5641217dfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.026+0000 7f70617f7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641213ea410 0x5641217e0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.026+0000 7f70617f7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5641216a9b40 con 0x5641213c1550 2024-05-20T18:36:27.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.026+0000 7f70617f7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5641217e1720 con 0x5641213ea410 2024-05-20T18:36:27.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.026+0000 7f70617f7740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5641217e02e0 con 0x5641213c9730 2024-05-20T18:36:27.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.026+0000 7f70617f7740 10 client.0 _ll_register_callbacks cb 0x56412169e0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:36:27.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.026+0000 7f7053fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641213c1550 0x5641217afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.026+0000 7f7053fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641213c1550 0x5641217afe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59936/0 (socket says 172.21.15.5:59936) 2024-05-20T18:36:27.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.026+0000 7f7053fff640 1 -- 172.21.15.5:0/3998820124 learned_addr learned my addr 172.21.15.5:0/3998820124 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:27.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.026+0000 7f70537fe640 1 --2- 172.21.15.5:0/3998820124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641213c9730 0x5641217dfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.026+0000 7f7058ee9640 1 --2- 172.21.15.5:0/3998820124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641213ea410 0x5641217e0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.026+0000 7f7053fff640 1 -- 172.21.15.5:0/3998820124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641213c9730 msgr2=0x5641217dfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:27.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.026+0000 7f7053fff640 1 --2- 172.21.15.5:0/3998820124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641213c9730 0x5641217dfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.026+0000 7f7053fff640 1 -- 172.21.15.5:0/3998820124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641213ea410 msgr2=0x5641217e0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:27.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.026+0000 7f7053fff640 1 --2- 172.21.15.5:0/3998820124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641213ea410 0x5641217e0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.026+0000 7f7053fff640 1 -- 172.21.15.5:0/3998820124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56412152af70 con 0x5641213c1550 2024-05-20T18:36:27.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.027+0000 7f7053fff640 1 --2- 172.21.15.5:0/3998820124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641213c1550 0x5641217afe20 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f7044059300 tx=0x7f7044058d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72615083dc568fba server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:27.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.027+0000 7f7032ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:36:27.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.027+0000 7f7032ffd640 1 -- 172.21.15.5:0/3998820124 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7044071020 con 0x5641213c1550 2024-05-20T18:36:27.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.027+0000 7f7032ffd640 1 -- 172.21.15.5:0/3998820124 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 6245+0+0 (secure 0 0 0) 0x7f70440711c0 con 0x5641213c1550 2024-05-20T18:36:27.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.027+0000 7f70617f7740 1 -- 172.21.15.5:0/3998820124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f704c03c640 con 0x5641213c1550 2024-05-20T18:36:27.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.027+0000 7f7032ffd640 1 -- 172.21.15.5:0/3998820124 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f7044021040 con 0x5641213c1550 2024-05-20T18:36:27.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.027+0000 7f70617f7740 10 client.8790 fetch_fsmap learned FSMap version 88 2024-05-20T18:36:27.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.027+0000 7f70617f7740 1 -- 172.21.15.5:0/3998820124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x5641213c2170 con 0x5641213c1550 2024-05-20T18:36:27.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.027+0000 7f7032ffd640 1 -- 172.21.15.5:0/3998820124 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f704c03c640 con 0x5641213c1550 2024-05-20T18:36:27.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.027+0000 7f7032ffd640 1 -- 172.21.15.5:0/3998820124 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7f704405d170 con 0x5641213c1550 2024-05-20T18:36:27.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.027+0000 7f70617f7740 10 client.8790 fetch_fsmap finished waiting for FSMap version 88 2024-05-20T18:36:27.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.027+0000 7f70617f7740 10 client.8790 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-20T18:36:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.027+0000 7f70617f7740 20 client.8790 populate_metadata read hostname 'smithi005' 2024-05-20T18:36:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.027+0000 7f70617f7740 1 --2- 172.21.15.5:0/3998820124 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x5641213a91f0 0x5641217b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.027+0000 7f70617f7740 4 client.8790 mds_command: new command op to 8325 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:36:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.027+0000 7f70617f7740 1 -- 172.21.15.5:0/3998820124 --> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641216fc0c0 con 0x5641213a91f0 2024-05-20T18:36:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.028+0000 7f70537fe640 1 --2- 172.21.15.5:0/3998820124 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x5641213a91f0 0x5641217b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.028+0000 7f7032ffd640 10 client.8790 ms_handle_connect on v2:172.21.15.5:6834/3552410668 2024-05-20T18:36:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.028+0000 7f70537fe640 1 --2- 172.21.15.5:0/3998820124 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x5641213a91f0 0x5641217b7b80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=577130363265338c server_cookie=ef792e163ba05c22 in_seq=0 out_seq=0 2024-05-20T18:36:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.028+0000 7f7c87702640 1 Processor -- start 2024-05-20T18:36:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.029+0000 7f7032ffd640 1 -- 172.21.15.5:0/3998820124 <== mds.0 v2:172.21.15.5:6834/3552410668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5641216fc0c0 con 0x5641213a91f0 2024-05-20T18:36:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.029+0000 7f7032ffd640 10 client.8790 handle_command_reply: tid=0 2024-05-20T18:36:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.029+0000 7f70617f7740 1 client.8790 shutdown 2024-05-20T18:36:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.029+0000 7f7c87702640 1 -- start start 2024-05-20T18:36:27.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.029+0000 7f70617f7740 1 -- 172.21.15.5:0/3998820124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641213c1550 msgr2=0x5641217afe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:27.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.029+0000 7f70617f7740 1 --2- 172.21.15.5:0/3998820124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641213c1550 0x5641217afe20 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f7044059300 tx=0x7f7044058d50 comp rx=0 tx=0).stop 2024-05-20T18:36:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.029+0000 7f70617f7740 1 -- 172.21.15.5:0/3998820124 shutdown_connections 2024-05-20T18:36:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.029+0000 7f70617f7740 1 --2- 172.21.15.5:0/3998820124 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x5641213a91f0 0x5641217b7b80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.029+0000 7f70617f7740 1 --2- 172.21.15.5:0/3998820124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641213ea410 0x5641217e0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.029+0000 7f70617f7740 1 --2- 172.21.15.5:0/3998820124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641213c9730 0x5641217dfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.029+0000 7f70617f7740 1 --2- 172.21.15.5:0/3998820124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641213c1550 0x5641217afe20 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.029+0000 7f7032ffd640 0 client.8790 ms_handle_reset on v2:172.21.15.5:6834/3552410668 2024-05-20T18:36:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.029+0000 7f70617f7740 1 -- 172.21.15.5:0/3998820124 >> 172.21.15.5:0/3998820124 conn(0x5641213de560 msgr2=0x5641217b5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.029+0000 7f70617f7740 1 -- 172.21.15.5:0/3998820124 shutdown_connections 2024-05-20T18:36:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.029+0000 7f70617f7740 1 -- 172.21.15.5:0/3998820124 wait complete. 2024-05-20T18:36:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.029+0000 7f70617f7740 20 client.8790 trim_cache size 0 max 16384 2024-05-20T18:36:27.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.030+0000 7f7c87702640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c8009ae00 0x7f7c800cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.030+0000 7f7c87702640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7c8009b210 0x7f7c800ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.030+0000 7f7c87702640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c800d09c0 con 0x7f7c800d1180 2024-05-20T18:36:27.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.030+0000 7f7c87702640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7c800ce7a0 con 0x7f7c8009b210 2024-05-20T18:36:27.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.030+0000 7f7c87702640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7c8015f7d0 con 0x7f7c8009ae00 2024-05-20T18:36:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.030+0000 7f7c85c9f640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7c800d1180 0x7f7c800d1590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36612/0 (socket says 172.21.15.5:36612) 2024-05-20T18:36:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.030+0000 7f7c85c9f640 1 -- 172.21.15.5:0/1911228674 learned_addr learned my addr 172.21.15.5:0/1911228674 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:36:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.031+0000 7f7c84c9d640 1 --2- 172.21.15.5:0/1911228674 >> v2:172.21.15.63:3300/0 conn(0x7f7c8009b210 0x7f7c800ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.031+0000 7f7c8549e640 1 --2- 172.21.15.5:0/1911228674 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c8009ae00 0x7f7c800cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.031+0000 7f7c77fff640 1 -- 172.21.15.5:0/1911228674 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4253739714 0 0) 0x7f7c800d09c0 con 0x7f7c800d1180 2024-05-20T18:36:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.031+0000 7f7c77fff640 1 -- 172.21.15.5:0/1911228674 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c6c003520 con 0x7f7c800d1180 2024-05-20T18:36:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.031+0000 7f7c84c9d640 1 -- 172.21.15.5:0/1911228674 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c8009ae00 msgr2=0x7f7c800cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.031+0000 7f7c84c9d640 1 --2- 172.21.15.5:0/1911228674 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c8009ae00 0x7f7c800cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.031+0000 7f7c84c9d640 1 -- 172.21.15.5:0/1911228674 >> v1:172.21.15.5:6789/0 conn(0x7f7c800d1180 legacy=0x7f7c800d1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.031+0000 7f7c84c9d640 1 -- 172.21.15.5:0/1911228674 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c800cd480 con 0x7f7c8009b210 2024-05-20T18:36:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.032+0000 7f7c84c9d640 1 --2- 172.21.15.5:0/1911228674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c8009b210 0x7f7c800ce260 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7c700097a0 tx=0x7f7c7002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ebf3720dc7d8c62 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.032+0000 7f7c77fff640 1 -- 172.21.15.5:0/1911228674 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c70037070 con 0x7f7c8009b210 2024-05-20T18:36:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.032+0000 7f7c77fff640 1 -- 172.21.15.5:0/1911228674 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c7002ae50 con 0x7f7c8009b210 2024-05-20T18:36:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.032+0000 7f7c77fff640 1 -- 172.21.15.5:0/1911228674 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c70032660 con 0x7f7c8009b210 2024-05-20T18:36:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.033+0000 7f7c87702640 1 -- 172.21.15.5:0/1911228674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c8009b210 msgr2=0x7f7c800ce260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.033+0000 7f7c87702640 1 --2- 172.21.15.5:0/1911228674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c8009b210 0x7f7c800ce260 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7c700097a0 tx=0x7f7c7002a100 comp rx=0 tx=0).stop 2024-05-20T18:36:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.033+0000 7f7c87702640 1 -- 172.21.15.5:0/1911228674 shutdown_connections 2024-05-20T18:36:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.033+0000 7f7c87702640 1 --2- 172.21.15.5:0/1911228674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c8009b210 0x7f7c800ce260 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.033+0000 7f7c87702640 1 --2- 172.21.15.5:0/1911228674 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c8009ae00 0x7f7c800cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.033+0000 7f7c87702640 1 -- 172.21.15.5:0/1911228674 >> 172.21.15.5:0/1911228674 conn(0x7f7c800b6670 msgr2=0x7f7c800b6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.033+0000 7f7c87702640 1 -- 172.21.15.5:0/1911228674 shutdown_connections 2024-05-20T18:36:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.033+0000 7f7c87702640 1 -- 172.21.15.5:0/1911228674 wait complete. 2024-05-20T18:36:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.035+0000 7f7c87702640 1 Processor -- start 2024-05-20T18:36:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.035+0000 7f7c87702640 1 -- start start 2024-05-20T18:36:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.036+0000 7f7c87702640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c8009ae00 0x7f7c800c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.036+0000 7f7c87702640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c8009b210 0x7f7c8015dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.036+0000 7f7c87702640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c800d1180 0x7f7c8015e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.036+0000 7f7c87702640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7c800cda40 con 0x7f7c8009b210 2024-05-20T18:36:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.036+0000 7f7c87702640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7c800cd760 con 0x7f7c800d1180 2024-05-20T18:36:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.036+0000 7f7c87702640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7c800b2780 con 0x7f7c8009ae00 2024-05-20T18:36:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.036+0000 7f7c84c9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c8009b210 0x7f7c8015dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.036+0000 7f7c84c9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c8009b210 0x7f7c8015dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59942/0 (socket says 172.21.15.5:59942) 2024-05-20T18:36:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.036+0000 7f7c84c9d640 1 -- 172.21.15.5:0/1014737986 learned_addr learned my addr 172.21.15.5:0/1014737986 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.036+0000 7f7c8549e640 1 --2- 172.21.15.5:0/1014737986 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c8009ae00 0x7f7c800c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.036+0000 7f7c85c9f640 1 --2- 172.21.15.5:0/1014737986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c800d1180 0x7f7c8015e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.037+0000 7f7c84c9d640 1 -- 172.21.15.5:0/1014737986 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c8009ae00 msgr2=0x7f7c800c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.037+0000 7f7c84c9d640 1 --2- 172.21.15.5:0/1014737986 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c8009ae00 0x7f7c800c7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.037+0000 7f7c84c9d640 1 -- 172.21.15.5:0/1014737986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c800d1180 msgr2=0x7f7c8015e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.037+0000 7f7c84c9d640 1 --2- 172.21.15.5:0/1014737986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c800d1180 0x7f7c8015e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.037+0000 7f7c84c9d640 1 -- 172.21.15.5:0/1014737986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c800c6160 con 0x7f7c8009b210 2024-05-20T18:36:27.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.037+0000 7f7c84c9d640 1 --2- 172.21.15.5:0/1014737986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c8009b210 0x7f7c8015dff0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f7c70000c00 tx=0x7f7c700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:27.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.037+0000 7f7c75ffb640 1 -- 172.21.15.5:0/1014737986 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c70041070 con 0x7f7c8009b210 2024-05-20T18:36:27.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.037+0000 7f7c75ffb640 1 -- 172.21.15.5:0/1014737986 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c7003c460 con 0x7f7c8009b210 2024-05-20T18:36:27.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.037+0000 7f7c87702640 1 -- 172.21.15.5:0/1014737986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c80161a80 con 0x7f7c8009b210 2024-05-20T18:36:27.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.037+0000 7f7c75ffb640 1 -- 172.21.15.5:0/1014737986 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c70037040 con 0x7f7c8009b210 2024-05-20T18:36:27.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.037+0000 7f7c87702640 1 -- 172.21.15.5:0/1014737986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c80162040 con 0x7f7c8009b210 2024-05-20T18:36:27.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.038+0000 7f7c87702640 1 -- 172.21.15.5:0/1014737986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f7c800690b0 con 0x7f7c8009b210 2024-05-20T18:36:27.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.039+0000 7f7c75ffb640 1 -- 172.21.15.5:0/1014737986 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7c70044070 con 0x7f7c8009b210 2024-05-20T18:36:27.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.040+0000 7f7c75ffb640 1 --2- 172.21.15.5:0/1014737986 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7c6406be60 0x7f7c6406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.040+0000 7f7c75ffb640 1 -- 172.21.15.5:0/1014737986 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6245+0+0 (secure 0 0 0) 0x7f7c700ab7d0 con 0x7f7c8009b210 2024-05-20T18:36:27.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.040+0000 7f7c8549e640 1 --2- 172.21.15.5:0/1014737986 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7c6406be60 0x7f7c6406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.040+0000 7f7c75ffb640 1 -- 172.21.15.5:0/1014737986 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+443 (secure 0 0 0) 0x7f7c700ae1d0 con 0x7f7c8009b210 2024-05-20T18:36:27.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.040+0000 7f7c87702640 1 -- 172.21.15.5:0/1014737986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f7c801e9990 con 0x7f7c8009b210 2024-05-20T18:36:27.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.040+0000 7f7c8549e640 1 --2- 172.21.15.5:0/1014737986 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7c6406be60 0x7f7c6406e370 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7c800b4590 tx=0x7f7c7c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:27.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.041+0000 7f7c75ffb640 1 -- 172.21.15.5:0/1014737986 <== mon.0 v2:172.21.15.5:3300/0 7 ==== log(2 entries from seq 1205 at 2024-05-20T18:36:25.364594+0000) ==== 389+0+0 (secure 0 0 0) 0x7f7c70077920 con 0x7f7c8009b210 2024-05-20T18:36:27.045 INFO:teuthology.orchestra.run.smithi005.stdout: cluster: 2024-05-20T18:36:27.045 INFO:teuthology.orchestra.run.smithi005.stdout: id: 399ab250-fd3c-47fa-a1c8-25e8149c9cac 2024-05-20T18:36:27.045 INFO:teuthology.orchestra.run.smithi005.stdout: health: HEALTH_OK 2024-05-20T18:36:27.045 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:36:27.045 INFO:teuthology.orchestra.run.smithi005.stdout: services: 2024-05-20T18:36:27.045 INFO:teuthology.orchestra.run.smithi005.stdout: mon: 3 daemons, quorum a,b,c (age 9m) 2024-05-20T18:36:27.045 INFO:teuthology.orchestra.run.smithi005.stdout: mgr: y(active, since 9m), standbys: x 2024-05-20T18:36:27.046 INFO:teuthology.orchestra.run.smithi005.stdout: mds: 1/1 daemons up, 3 standby 2024-05-20T18:36:27.046 INFO:teuthology.orchestra.run.smithi005.stdout: osd: 8 osds: 8 up (since 9m), 8 in (since 9m) 2024-05-20T18:36:27.046 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:36:27.046 INFO:teuthology.orchestra.run.smithi005.stdout: data: 2024-05-20T18:36:27.046 INFO:teuthology.orchestra.run.smithi005.stdout: volumes: 1/1 healthy 2024-05-20T18:36:27.046 INFO:teuthology.orchestra.run.smithi005.stdout: pools: 3 pools, 97 pgs 2024-05-20T18:36:27.046 INFO:teuthology.orchestra.run.smithi005.stdout: objects: 30 objects, 19 MiB 2024-05-20T18:36:27.046 INFO:teuthology.orchestra.run.smithi005.stdout: usage: 291 MiB used, 720 GiB / 720 GiB avail 2024-05-20T18:36:27.046 INFO:teuthology.orchestra.run.smithi005.stdout: pgs: 97 active+clean 2024-05-20T18:36:27.046 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:36:27.046 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:36:27.046 INFO:teuthology.orchestra.run.smithi005.stdout:2024-05-20T18:36:25.364594+0000 mon.a [INF] Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-20T18:36:27.046 INFO:teuthology.orchestra.run.smithi005.stdout:2024-05-20T18:36:25.364621+0000 mon.a [INF] Cluster is now healthy 2024-05-20T18:36:27.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.050+0000 7f70617f7740 1 Processor -- start 2024-05-20T18:36:27.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.050+0000 7f70617f7740 1 -- start start 2024-05-20T18:36:27.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.051+0000 7f70617f7740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641213a91f0 0x5641217b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.051+0000 7f70617f7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641213c1550 0x564121788f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.051+0000 7f70617f7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641213c9730 0x564121789480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.051+0000 7f70617f7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7054200320 con 0x5641213c1550 2024-05-20T18:36:27.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.051+0000 7f70617f7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f70541ff340 con 0x5641213c9730 2024-05-20T18:36:27.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.051+0000 7f70617f7740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f70540b7370 con 0x5641213a91f0 2024-05-20T18:36:27.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.051+0000 7f70537fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641213c1550 0x564121788f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.051+0000 7f70537fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641213c1550 0x564121788f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59944/0 (socket says 172.21.15.5:59944) 2024-05-20T18:36:27.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.051+0000 7f70537fe640 1 -- 172.21.15.5:0/4092600181 learned_addr learned my addr 172.21.15.5:0/4092600181 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:27.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.051+0000 7f7058ee9640 1 --2- 172.21.15.5:0/4092600181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641213c9730 0x564121789480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.051+0000 7f7053fff640 1 --2- 172.21.15.5:0/4092600181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641213a91f0 0x5641217b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.052+0000 7f70537fe640 1 -- 172.21.15.5:0/4092600181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641213a91f0 msgr2=0x5641217b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:27.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.052+0000 7f70537fe640 1 --2- 172.21.15.5:0/4092600181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641213a91f0 0x5641217b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.052+0000 7f70537fe640 1 -- 172.21.15.5:0/4092600181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641213c9730 msgr2=0x564121789480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:27.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.052+0000 7f70537fe640 1 --2- 172.21.15.5:0/4092600181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641213c9730 0x564121789480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.052+0000 7f70537fe640 1 -- 172.21.15.5:0/4092600181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564121788ae0 con 0x5641213c1550 2024-05-20T18:36:27.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.052+0000 7f70537fe640 1 --2- 172.21.15.5:0/4092600181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641213c1550 0x564121788f40 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f7048004370 tx=0x7f7048018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e6d8467567f697 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:27.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.052+0000 7f70397fa640 1 -- 172.21.15.5:0/4092600181 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704801d070 con 0x5641213c1550 2024-05-20T18:36:27.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.052+0000 7f70397fa640 1 -- 172.21.15.5:0/4092600181 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f704801e040 con 0x5641213c1550 2024-05-20T18:36:27.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.052+0000 7f70397fa640 1 -- 172.21.15.5:0/4092600181 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704802c560 con 0x5641213c1550 2024-05-20T18:36:27.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.053+0000 7f70617f7740 1 -- 172.21.15.5:0/4092600181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641213c1550 msgr2=0x564121788f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:27.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.053+0000 7f70617f7740 1 --2- 172.21.15.5:0/4092600181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641213c1550 0x564121788f40 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f7048004370 tx=0x7f7048018000 comp rx=0 tx=0).stop 2024-05-20T18:36:27.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.053+0000 7f70617f7740 1 -- 172.21.15.5:0/4092600181 shutdown_connections 2024-05-20T18:36:27.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.053+0000 7f70617f7740 1 --2- 172.21.15.5:0/4092600181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641213c9730 0x564121789480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.053+0000 7f70617f7740 1 --2- 172.21.15.5:0/4092600181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641213c1550 0x564121788f40 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.053+0000 7f70617f7740 1 --2- 172.21.15.5:0/4092600181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641213a91f0 0x5641217b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.053+0000 7f70617f7740 1 -- 172.21.15.5:0/4092600181 >> 172.21.15.5:0/4092600181 conn(0x5641213de560 msgr2=0x5641217858e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:27.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.053+0000 7f70617f7740 1 -- 172.21.15.5:0/4092600181 shutdown_connections 2024-05-20T18:36:27.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.053+0000 7f70617f7740 1 -- 172.21.15.5:0/4092600181 wait complete. 2024-05-20T18:36:27.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.054+0000 7f70617f7740 1 -- start start 2024-05-20T18:36:27.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.054+0000 7f70617f7740 1 Processor -- start 2024-05-20T18:36:27.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.055+0000 7f70617f7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641213a91f0 0x5641217f3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.055+0000 7f70617f7740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641213c1550 0x56412184b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.055+0000 7f70617f7740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641213c9730 0x564121788e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.055+0000 7f70617f7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f70541fed20 con 0x5641213a91f0 2024-05-20T18:36:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.055+0000 7f70617f7740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5641217c9d70 con 0x5641213c9730 2024-05-20T18:36:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.055+0000 7f70617f7740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x56412184e900 con 0x5641213c1550 2024-05-20T18:36:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.055+0000 7f70617f7740 10 client.0 _ll_register_callbacks cb 0x7f704400ced0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:36:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.055+0000 7f7053fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641213a91f0 0x5641217f3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.055+0000 7f7053fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641213a91f0 0x5641217f3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59948/0 (socket says 172.21.15.5:59948) 2024-05-20T18:36:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.055+0000 7f7053fff640 1 -- 172.21.15.5:0/2342301329 learned_addr learned my addr 172.21.15.5:0/2342301329 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.056+0000 7f7053fff640 1 -- 172.21.15.5:0/2342301329 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641213c1550 msgr2=0x56412184b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.056+0000 7f7053fff640 1 --2- 172.21.15.5:0/2342301329 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641213c1550 0x56412184b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.056+0000 7f7058ee9640 1 --2- 172.21.15.5:0/2342301329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641213c9730 0x564121788e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.056+0000 7f7053fff640 1 -- 172.21.15.5:0/2342301329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641213c9730 msgr2=0x564121788e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.056+0000 7f7053fff640 1 --2- 172.21.15.5:0/2342301329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641213c9730 0x564121788e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.056+0000 7f7053fff640 1 -- 172.21.15.5:0/2342301329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641217df1b0 con 0x5641213a91f0 2024-05-20T18:36:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.056+0000 7f7053fff640 1 --2- 172.21.15.5:0/2342301329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641213a91f0 0x5641217f3320 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f7044061a00 tx=0x7f7044089520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ccbdbc0c461b864 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.056+0000 7f7032ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:36:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.056+0000 7f7032ffd640 1 -- 172.21.15.5:0/2342301329 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7044071040 con 0x5641213a91f0 2024-05-20T18:36:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.056+0000 7f7032ffd640 1 -- 172.21.15.5:0/2342301329 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 6245+0+0 (secure 0 0 0) 0x7f70440587c0 con 0x5641213a91f0 2024-05-20T18:36:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.056+0000 7f70617f7740 1 -- 172.21.15.5:0/2342301329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f704801e040 con 0x5641213a91f0 2024-05-20T18:36:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.056+0000 7f7032ffd640 1 -- 172.21.15.5:0/2342301329 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f704407f050 con 0x5641213a91f0 2024-05-20T18:36:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.056+0000 7f70617f7740 10 client.8802 fetch_fsmap learned FSMap version 88 2024-05-20T18:36:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.056+0000 7f70617f7740 1 -- 172.21.15.5:0/2342301329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x5641217b20f0 con 0x5641213a91f0 2024-05-20T18:36:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.056+0000 7f7032ffd640 1 -- 172.21.15.5:0/2342301329 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7044063080 con 0x5641213a91f0 2024-05-20T18:36:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.056+0000 7f7032ffd640 1 -- 172.21.15.5:0/2342301329 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7f7044082020 con 0x5641213a91f0 2024-05-20T18:36:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.057+0000 7f70617f7740 10 client.8802 fetch_fsmap finished waiting for FSMap version 88 2024-05-20T18:36:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.057+0000 7f70617f7740 10 client.8802 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-20T18:36:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.057+0000 7f70617f7740 20 client.8802 populate_metadata read hostname 'smithi005' 2024-05-20T18:36:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.057+0000 7f70617f7740 1 --2- 172.21.15.5:0/2342301329 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x56412137e630 0x564121789380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.057+0000 7f70617f7740 4 client.8802 mds_command: new command op to 8325 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-05-20T18:36:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.057+0000 7f70617f7740 1 -- 172.21.15.5:0/2342301329 --> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x5641216f6410 con 0x56412137e630 2024-05-20T18:36:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.057+0000 7f70537fe640 1 --2- 172.21.15.5:0/2342301329 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x56412137e630 0x564121789380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.057+0000 7f7032ffd640 10 client.8802 ms_handle_connect on v2:172.21.15.5:6834/3552410668 2024-05-20T18:36:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.057+0000 7f70537fe640 1 --2- 172.21.15.5:0/2342301329 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x56412137e630 0x564121789380 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36b9f1b2f86f322d server_cookie=234d9485676da4f4 in_seq=0 out_seq=0 2024-05-20T18:36:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.058+0000 7f7032ffd640 1 -- 172.21.15.5:0/2342301329 <== mds.0 v2:172.21.15.5:6834/3552410668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5641216f6410 con 0x56412137e630 2024-05-20T18:36:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.058+0000 7f7032ffd640 10 client.8802 handle_command_reply: tid=0 2024-05-20T18:36:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.058+0000 7f70617f7740 1 client.8802 shutdown 2024-05-20T18:36:27.062 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-20T18:36:27.058+0000 7f1d34683640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-05-20T18:36:27.062 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-20T18:36:27.058+0000 7f1d34683640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-05-20T18:36:27.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.058+0000 7f70617f7740 1 -- 172.21.15.5:0/2342301329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641213a91f0 msgr2=0x5641217f3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:27.063 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-20T18:36:27.058+0000 7f1d34683640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-05-20T18:36:27.063 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-20T18:36:27.058+0000 7f1d34683640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-05-20T18:36:27.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.058+0000 7f70617f7740 1 --2- 172.21.15.5:0/2342301329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641213a91f0 0x5641217f3320 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f7044061a00 tx=0x7f7044089520 comp rx=0 tx=0).stop 2024-05-20T18:36:27.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.058+0000 7f70617f7740 1 -- 172.21.15.5:0/2342301329 shutdown_connections 2024-05-20T18:36:27.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.058+0000 7f70617f7740 1 --2- 172.21.15.5:0/2342301329 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x56412137e630 0x564121789380 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.058+0000 7f70617f7740 1 --2- 172.21.15.5:0/2342301329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641213c9730 0x564121788e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.058+0000 7f70617f7740 1 --2- 172.21.15.5:0/2342301329 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641213c1550 0x56412184b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.058+0000 7f7032ffd640 0 client.8802 ms_handle_reset on v2:172.21.15.5:6834/3552410668 2024-05-20T18:36:27.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.058+0000 7f70617f7740 1 --2- 172.21.15.5:0/2342301329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641213a91f0 0x5641217f3320 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.058+0000 7f70617f7740 1 -- 172.21.15.5:0/2342301329 >> 172.21.15.5:0/2342301329 conn(0x5641213de560 msgr2=0x5641217cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.058+0000 7f70617f7740 1 -- 172.21.15.5:0/2342301329 shutdown_connections 2024-05-20T18:36:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.058+0000 7f70617f7740 1 -- 172.21.15.5:0/2342301329 wait complete. 2024-05-20T18:36:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.059+0000 7f70617f7740 20 client.8802 trim_cache size 0 max 16384 2024-05-20T18:36:27.065 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:36:27.065 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-05-20T18:36:27.065 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "fe4defaa-27d8-48f2-a366-c6ee59fc6d8e", 2024-05-20T18:36:27.065 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-05-20T18:36:27.065 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:36:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.059+0000 7f705a94c640 1 -- 172.21.15.5:0/3759247792 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f704006be30 msgr2=0x7f704006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.059+0000 7f705a94c640 1 --2- 172.21.15.5:0/3759247792 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f704006be30 0x7f704006e340 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f70541588b0 tx=0x7f7048009190 comp rx=0 tx=0).stop 2024-05-20T18:36:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.059+0000 7f705a94c640 1 -- 172.21.15.5:0/3759247792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70540cea20 msgr2=0x7f705414deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.059+0000 7f705a94c640 1 --2- 172.21.15.5:0/3759247792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70540cea20 0x7f705414deb0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f704400e870 tx=0x7f704400ed30 comp rx=0 tx=0).stop 2024-05-20T18:36:27.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.060+0000 7f705a94c640 1 -- 172.21.15.5:0/3759247792 shutdown_connections 2024-05-20T18:36:27.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.060+0000 7f705a94c640 1 --2- 172.21.15.5:0/3759247792 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f704006be30 0x7f704006e340 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.060+0000 7f705a94c640 1 --2- 172.21.15.5:0/3759247792 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70540d3510 0x7f70540c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.060+0000 7f705a94c640 1 --2- 172.21.15.5:0/3759247792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70540d1830 0x7f70540c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.060+0000 7f705a94c640 1 --2- 172.21.15.5:0/3759247792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70540cea20 0x7f705414deb0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.060+0000 7f705a94c640 1 -- 172.21.15.5:0/3759247792 >> 172.21.15.5:0/3759247792 conn(0x7f70540d29f0 msgr2=0x7f7054161660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:27.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.060+0000 7f705a94c640 1 -- 172.21.15.5:0/3759247792 shutdown_connections 2024-05-20T18:36:27.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.060+0000 7f705a94c640 1 -- 172.21.15.5:0/3759247792 wait complete. 2024-05-20T18:36:27.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:36:27.073 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-20T18:36:27.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.188+0000 7fc108298640 1 Processor -- start 2024-05-20T18:36:27.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.188+0000 7fc108298640 1 -- start start 2024-05-20T18:36:27.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.189+0000 7fc108298640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1000b1e10 0x7fc10015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.189+0000 7fc108298640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc100070850 0x7fc100070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.189+0000 7fc108298640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1000cd580 con 0x7fc10015d530 2024-05-20T18:36:27.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.189+0000 7fc108298640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fc1000ce190 con 0x7fc100070850 2024-05-20T18:36:27.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.189+0000 7fc108298640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc1000cd860 con 0x7fc1000b1e10 2024-05-20T18:36:27.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.189+0000 7fc106835640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc100070850 0x7fc100070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.189+0000 7fc106034640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc10015d530 0x7fc1000b1700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:36620/0 (socket says 172.21.15.5:36620) 2024-05-20T18:36:27.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.189+0000 7fc106034640 1 -- 172.21.15.5:0/2935161039 learned_addr learned my addr 172.21.15.5:0/2935161039 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:36:27.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.190+0000 7fc105833640 1 --2- 172.21.15.5:0/2935161039 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1000b1e10 0x7fc10015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.190+0000 7fc106835640 1 -- 172.21.15.5:0/2935161039 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1000b1e10 msgr2=0x7fc10015aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:27.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.190+0000 7fc106835640 1 --2- 172.21.15.5:0/2935161039 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1000b1e10 0x7fc10015aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.190+0000 7fc106835640 1 -- 172.21.15.5:0/2935161039 >> v1:172.21.15.5:6789/0 conn(0x7fc10015d530 legacy=0x7fc1000b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:27.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.190+0000 7fc106835640 1 -- 172.21.15.5:0/2935161039 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1000cdb40 con 0x7fc100070850 2024-05-20T18:36:27.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.191+0000 7fc106835640 1 --2- 172.21.15.5:0/2935161039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc100070850 0x7fc100070c60 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009d90 tx=0x7fc0f802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1a0efc933596d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:27.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.191+0000 7fc105032640 1 -- 172.21.15.5:0/2935161039 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8007740 con 0x7fc100070850 2024-05-20T18:36:27.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.191+0000 7fc105032640 1 -- 172.21.15.5:0/2935161039 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0f80078e0 con 0x7fc100070850 2024-05-20T18:36:27.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.191+0000 7fc105032640 1 -- 172.21.15.5:0/2935161039 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8007cb0 con 0x7fc100070850 2024-05-20T18:36:27.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.191+0000 7fc108298640 1 -- 172.21.15.5:0/2935161039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc100070850 msgr2=0x7fc100070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:27.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.191+0000 7fc108298640 1 --2- 172.21.15.5:0/2935161039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc100070850 0x7fc100070c60 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009d90 tx=0x7fc0f802a660 comp rx=0 tx=0).stop 2024-05-20T18:36:27.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.191+0000 7fc108298640 1 -- 172.21.15.5:0/2935161039 shutdown_connections 2024-05-20T18:36:27.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.191+0000 7fc108298640 1 --2- 172.21.15.5:0/2935161039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc100070850 0x7fc100070c60 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.192+0000 7fc108298640 1 --2- 172.21.15.5:0/2935161039 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1000b1e10 0x7fc10015aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.192+0000 7fc108298640 1 -- 172.21.15.5:0/2935161039 >> 172.21.15.5:0/2935161039 conn(0x7fc1000d29d0 msgr2=0x7fc1000d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:27.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.192+0000 7fc108298640 1 -- 172.21.15.5:0/2935161039 shutdown_connections 2024-05-20T18:36:27.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.192+0000 7fc108298640 1 -- 172.21.15.5:0/2935161039 wait complete. 2024-05-20T18:36:27.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.193+0000 7fc108298640 1 Processor -- start 2024-05-20T18:36:27.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.193+0000 7fc108298640 1 -- start start 2024-05-20T18:36:27.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.195+0000 7fc108298640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc100070850 0x7fc100154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.195+0000 7fc108298640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1000b1e10 0x7fc1000c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.195+0000 7fc108298640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc10015d530 0x7fc10015c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.195+0000 7fc108298640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc1000711a0 con 0x7fc10015d530 2024-05-20T18:36:27.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.195+0000 7fc108298640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc1000cd230 con 0x7fc100070850 2024-05-20T18:36:27.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.195+0000 7fc108298640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc100071320 con 0x7fc1000b1e10 2024-05-20T18:36:27.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.195+0000 7fc106835640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc10015d530 0x7fc10015c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.195+0000 7fc106835640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc10015d530 0x7fc10015c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59964/0 (socket says 172.21.15.5:59964) 2024-05-20T18:36:27.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.195+0000 7fc106835640 1 -- 172.21.15.5:0/2221081257 learned_addr learned my addr 172.21.15.5:0/2221081257 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:27.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.195+0000 7fc105833640 1 --2- 172.21.15.5:0/2221081257 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1000b1e10 0x7fc1000c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.195+0000 7fc106034640 1 --2- 172.21.15.5:0/2221081257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc100070850 0x7fc100154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.195+0000 7fc106835640 1 -- 172.21.15.5:0/2221081257 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1000b1e10 msgr2=0x7fc1000c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:27.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.195+0000 7fc106835640 1 --2- 172.21.15.5:0/2221081257 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1000b1e10 0x7fc1000c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.195+0000 7fc106835640 1 -- 172.21.15.5:0/2221081257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc100070850 msgr2=0x7fc100154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:27.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.195+0000 7fc106835640 1 --2- 172.21.15.5:0/2221081257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc100070850 0x7fc100154010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.195+0000 7fc106835640 1 -- 172.21.15.5:0/2221081257 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1000ca6b0 con 0x7fc10015d530 2024-05-20T18:36:27.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.195+0000 7fc106835640 1 --2- 172.21.15.5:0/2221081257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc10015d530 0x7fc10015c360 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009eb0 tx=0x7fc0f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:27.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.196+0000 7fc0f6ffd640 1 -- 172.21.15.5:0/2221081257 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f803f070 con 0x7fc10015d530 2024-05-20T18:36:27.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.196+0000 7fc0f6ffd640 1 -- 172.21.15.5:0/2221081257 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0f8031bc0 con 0x7fc10015d530 2024-05-20T18:36:27.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.196+0000 7fc108298640 1 -- 172.21.15.5:0/2221081257 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1001f6d50 con 0x7fc10015d530 2024-05-20T18:36:27.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.196+0000 7fc0f6ffd640 1 -- 172.21.15.5:0/2221081257 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8009450 con 0x7fc10015d530 2024-05-20T18:36:27.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.196+0000 7fc108298640 1 -- 172.21.15.5:0/2221081257 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1001f7310 con 0x7fc10015d530 2024-05-20T18:36:27.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.197+0000 7fc0f6ffd640 1 -- 172.21.15.5:0/2221081257 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc0f803b410 con 0x7fc10015d530 2024-05-20T18:36:27.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.198+0000 7fc0f6ffd640 1 --2- 172.21.15.5:0/2221081257 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc0e406be30 0x7fc0e406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.198+0000 7fc0f6ffd640 1 -- 172.21.15.5:0/2221081257 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6245+0+0 (secure 0 0 0) 0x7fc0f80ac830 con 0x7fc10015d530 2024-05-20T18:36:27.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.198+0000 7fc106034640 1 --2- 172.21.15.5:0/2221081257 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc0e406be30 0x7fc0e406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.199+0000 7fc106034640 1 --2- 172.21.15.5:0/2221081257 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc0e406be30 0x7fc0e406e340 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc007fd0 tx=0x7fc0fc007e80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:27.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.199+0000 7fc10f216740 1 Processor -- start 2024-05-20T18:36:27.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.199+0000 7fc10f216740 1 -- start start 2024-05-20T18:36:27.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.200+0000 7fc10f216740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562eadf77390 0x562eae341b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.201+0000 7fc10f216740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562eadf4e4d0 0x562eae35c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.201+0000 7fc10f216740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eadf566b0 0x562eae35cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.201+0000 7fc10f216740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562eae27bd00 con 0x562eadf566b0 2024-05-20T18:36:27.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.201+0000 7fc10f216740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562eae2234c0 con 0x562eadf77390 2024-05-20T18:36:27.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.201+0000 7fc10f216740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x562eae277cc0 con 0x562eadf4e4d0 2024-05-20T18:36:27.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.201+0000 7fc106835640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eadf566b0 0x562eae35cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.201+0000 7fc106835640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eadf566b0 0x562eae35cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59966/0 (socket says 172.21.15.5:59966) 2024-05-20T18:36:27.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.201+0000 7fc106835640 1 -- 172.21.15.5:0/3269851982 learned_addr learned my addr 172.21.15.5:0/3269851982 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:27.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.201+0000 7fc106034640 1 --2- 172.21.15.5:0/3269851982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562eadf77390 0x562eae341b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.201+0000 7fc105833640 1 --2- 172.21.15.5:0/3269851982 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562eadf4e4d0 0x562eae35c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.201+0000 7fc106835640 1 -- 172.21.15.5:0/3269851982 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562eadf4e4d0 msgr2=0x562eae35c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:27.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.201+0000 7fc106835640 1 --2- 172.21.15.5:0/3269851982 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562eadf4e4d0 0x562eae35c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.201+0000 7fc106835640 1 -- 172.21.15.5:0/3269851982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562eadf77390 msgr2=0x562eae341b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:27.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.201+0000 7fc106835640 1 --2- 172.21.15.5:0/3269851982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562eadf77390 0x562eae341b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.201+0000 7fc106835640 1 -- 172.21.15.5:0/3269851982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eae0597d0 con 0x562eadf566b0 2024-05-20T18:36:27.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.201+0000 7fc106034640 1 --2- 172.21.15.5:0/3269851982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562eadf77390 0x562eae341b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:36:27.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.201+0000 7fc105833640 1 --2- 172.21.15.5:0/3269851982 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562eadf4e4d0 0x562eae35c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:36:27.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.201+0000 7fc106835640 1 --2- 172.21.15.5:0/3269851982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eadf566b0 0x562eae35cdd0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8082ed0 tx=0x7fc0f8031f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9529405239d933e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:27.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.201+0000 7fc0e3fff640 1 -- 172.21.15.5:0/3269851982 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8078b40 con 0x562eadf566b0 2024-05-20T18:36:27.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.201+0000 7fc0e3fff640 1 -- 172.21.15.5:0/3269851982 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0f8076eb0 con 0x562eadf566b0 2024-05-20T18:36:27.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.201+0000 7fc0e3fff640 1 -- 172.21.15.5:0/3269851982 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f807acc0 con 0x562eadf566b0 2024-05-20T18:36:27.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.202+0000 7fc10f216740 1 -- 172.21.15.5:0/3269851982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eadf566b0 msgr2=0x562eae35cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:27.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.202+0000 7fc10f216740 1 --2- 172.21.15.5:0/3269851982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eadf566b0 0x562eae35cdd0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8082ed0 tx=0x7fc0f8031f90 comp rx=0 tx=0).stop 2024-05-20T18:36:27.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.202+0000 7fc10f216740 1 -- 172.21.15.5:0/3269851982 shutdown_connections 2024-05-20T18:36:27.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.202+0000 7fc10f216740 1 --2- 172.21.15.5:0/3269851982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eadf566b0 0x562eae35cdd0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.202+0000 7fc10f216740 1 --2- 172.21.15.5:0/3269851982 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562eadf4e4d0 0x562eae35c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.202+0000 7fc10f216740 1 --2- 172.21.15.5:0/3269851982 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562eadf77390 0x562eae341b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.202+0000 7fc10f216740 1 -- 172.21.15.5:0/3269851982 >> 172.21.15.5:0/3269851982 conn(0x562eadf6b4e0 msgr2=0x562eae3171c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:27.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.202+0000 7fc10f216740 1 -- 172.21.15.5:0/3269851982 shutdown_connections 2024-05-20T18:36:27.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.202+0000 7fc10f216740 1 -- 172.21.15.5:0/3269851982 wait complete. 2024-05-20T18:36:27.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.203+0000 7fc10f216740 1 -- start start 2024-05-20T18:36:27.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.203+0000 7fc10f216740 1 Processor -- start 2024-05-20T18:36:27.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.205+0000 7fc10f216740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eadf4e4d0 0x562eae354430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.205+0000 7fc10f216740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562eadf566b0 0x562eae355a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.205+0000 7fc10f216740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562eadf77390 0x562eae355fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.205+0000 7fc10f216740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562eae2234c0 con 0x562eadf4e4d0 2024-05-20T18:36:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.205+0000 7fc10f216740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562eae236b90 con 0x562eadf77390 2024-05-20T18:36:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.205+0000 7fc10f216740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x562eae27bd00 con 0x562eadf566b0 2024-05-20T18:36:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.205+0000 7fc10f216740 10 client.0 _ll_register_callbacks cb 0x562eae272da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:36:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.205+0000 7fc106034640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eadf4e4d0 0x562eae354430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.205+0000 7fc106034640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eadf4e4d0 0x562eae354430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59982/0 (socket says 172.21.15.5:59982) 2024-05-20T18:36:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.205+0000 7fc106034640 1 -- 172.21.15.5:0/262770454 learned_addr learned my addr 172.21.15.5:0/262770454 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.205+0000 7fc105833640 1 --2- 172.21.15.5:0/262770454 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562eadf566b0 0x562eae355a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.205+0000 7fc106034640 1 -- 172.21.15.5:0/262770454 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562eadf566b0 msgr2=0x562eae355a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.205+0000 7fc106034640 1 --2- 172.21.15.5:0/262770454 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562eadf566b0 0x562eae355a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.205+0000 7fc106034640 1 -- 172.21.15.5:0/262770454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562eadf77390 msgr2=0x562eae355fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.205+0000 7fc106835640 1 --2- 172.21.15.5:0/262770454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562eadf77390 0x562eae355fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.205+0000 7fc106034640 1 --2- 172.21.15.5:0/262770454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562eadf77390 0x562eae355fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.205+0000 7fc106034640 1 -- 172.21.15.5:0/262770454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eadf8f3f0 con 0x562eadf4e4d0 2024-05-20T18:36:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.205+0000 7fc106034640 1 --2- 172.21.15.5:0/262770454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eadf4e4d0 0x562eae354430 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc001e70 tx=0x7fc0fc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3db5be3bd710196 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.205+0000 7fc0e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:36:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.206+0000 7fc0e17fa640 1 -- 172.21.15.5:0/262770454 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc026060 con 0x562eadf4e4d0 2024-05-20T18:36:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.206+0000 7fc0e17fa640 1 -- 172.21.15.5:0/262770454 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 6245+0+0 (secure 0 0 0) 0x7fc0fc021cc0 con 0x562eadf4e4d0 2024-05-20T18:36:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.206+0000 7fc10f216740 1 -- 172.21.15.5:0/262770454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0f8076eb0 con 0x562eadf4e4d0 2024-05-20T18:36:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.206+0000 7fc0e17fa640 1 -- 172.21.15.5:0/262770454 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fc0fc019070 con 0x562eadf4e4d0 2024-05-20T18:36:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.206+0000 7fc10f216740 10 client.8817 fetch_fsmap learned FSMap version 88 2024-05-20T18:36:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.206+0000 7fc10f216740 1 -- 172.21.15.5:0/262770454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x562eadf4f0f0 con 0x562eadf4e4d0 2024-05-20T18:36:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.206+0000 7fc0e17fa640 1 -- 172.21.15.5:0/262770454 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0f8076eb0 con 0x562eadf4e4d0 2024-05-20T18:36:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.206+0000 7fc0e17fa640 1 -- 172.21.15.5:0/262770454 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7fc0fc01f070 con 0x562eadf4e4d0 2024-05-20T18:36:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.206+0000 7fc10f216740 10 client.8817 fetch_fsmap finished waiting for FSMap version 88 2024-05-20T18:36:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.206+0000 7fc10f216740 10 client.8817 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-20T18:36:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.206+0000 7fc10f216740 20 client.8817 populate_metadata read hostname 'smithi005' 2024-05-20T18:36:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.206+0000 7fc10f216740 1 --2- 172.21.15.5:0/262770454 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x562eadf35f60 0x562eae343680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.206+0000 7fc10f216740 4 client.8817 mds_command: new command op to 8325 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:36:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.206+0000 7fc10f216740 1 -- 172.21.15.5:0/262770454 --> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562eae289430 con 0x562eadf35f60 2024-05-20T18:36:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.206+0000 7fc105833640 1 --2- 172.21.15.5:0/262770454 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x562eadf35f60 0x562eae343680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.207+0000 7fc0e17fa640 10 client.8817 ms_handle_connect on v2:172.21.15.5:6834/3552410668 2024-05-20T18:36:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.207+0000 7fc105833640 1 --2- 172.21.15.5:0/262770454 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x562eadf35f60 0x562eae343680 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=546dfe33b0e1743c server_cookie=9303c58c6d306275 in_seq=0 out_seq=0 2024-05-20T18:36:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.207+0000 7fc0e17fa640 1 -- 172.21.15.5:0/262770454 <== mds.0 v2:172.21.15.5:6834/3552410668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562eae289430 con 0x562eadf35f60 2024-05-20T18:36:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.207+0000 7fc0e17fa640 10 client.8817 handle_command_reply: tid=0 2024-05-20T18:36:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.207+0000 7fc10f216740 1 client.8817 shutdown 2024-05-20T18:36:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.208+0000 7fc10f216740 1 -- 172.21.15.5:0/262770454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eadf4e4d0 msgr2=0x562eae354430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.208+0000 7fc10f216740 1 --2- 172.21.15.5:0/262770454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eadf4e4d0 0x562eae354430 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc001e70 tx=0x7fc0fc000bc0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.208+0000 7fc10f216740 1 -- 172.21.15.5:0/262770454 shutdown_connections 2024-05-20T18:36:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.208+0000 7fc10f216740 1 --2- 172.21.15.5:0/262770454 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x562eadf35f60 0x562eae343680 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.208+0000 7fc10f216740 1 --2- 172.21.15.5:0/262770454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562eadf77390 0x562eae355fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.208+0000 7fc10f216740 1 --2- 172.21.15.5:0/262770454 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562eadf566b0 0x562eae355a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.208+0000 7fc10f216740 1 --2- 172.21.15.5:0/262770454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eadf4e4d0 0x562eae354430 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.208+0000 7fc0e17fa640 0 client.8817 ms_handle_reset on v2:172.21.15.5:6834/3552410668 2024-05-20T18:36:27.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.208+0000 7fc10f216740 1 -- 172.21.15.5:0/262770454 >> 172.21.15.5:0/262770454 conn(0x562eadf6b4e0 msgr2=0x562eae341b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:27.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.208+0000 7fc10f216740 1 -- 172.21.15.5:0/262770454 shutdown_connections 2024-05-20T18:36:27.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.208+0000 7fc10f216740 1 -- 172.21.15.5:0/262770454 wait complete. 2024-05-20T18:36:27.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.208+0000 7fc10f216740 20 client.8817 trim_cache size 0 max 16384 2024-05-20T18:36:27.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.226+0000 7fc10f216740 1 Processor -- start 2024-05-20T18:36:27.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.226+0000 7fc10f216740 1 -- start start 2024-05-20T18:36:27.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.228+0000 7fc10f216740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562eadf4e4d0 0x562eae3556c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.228+0000 7fc10f216740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eadf566b0 0x562eae355c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.228+0000 7fc10f216740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562eadf77390 0x562eae35bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.228+0000 7fc10f216740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc100200390 con 0x562eadf566b0 2024-05-20T18:36:27.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.228+0000 7fc10f216740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc1001ff3d0 con 0x562eadf4e4d0 2024-05-20T18:36:27.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.228+0000 7fc10f216740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc1001586e0 con 0x562eadf77390 2024-05-20T18:36:27.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.228+0000 7fc105833640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eadf566b0 0x562eae355c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.228+0000 7fc105833640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eadf566b0 0x562eae355c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59984/0 (socket says 172.21.15.5:59984) 2024-05-20T18:36:27.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.228+0000 7fc105833640 1 -- 172.21.15.5:0/1578692475 learned_addr learned my addr 172.21.15.5:0/1578692475 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:27.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.228+0000 7fc105833640 1 -- 172.21.15.5:0/1578692475 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562eadf77390 msgr2=0x562eae35bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:27.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.228+0000 7fc105833640 1 --2- 172.21.15.5:0/1578692475 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562eadf77390 0x562eae35bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.228+0000 7fc105833640 1 -- 172.21.15.5:0/1578692475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562eadf4e4d0 msgr2=0x562eae3556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:27.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.228+0000 7fc105833640 1 --2- 172.21.15.5:0/1578692475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562eadf4e4d0 0x562eae3556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.228+0000 7fc105833640 1 -- 172.21.15.5:0/1578692475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eae3550c0 con 0x562eadf566b0 2024-05-20T18:36:27.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.228+0000 7fc105833640 1 --2- 172.21.15.5:0/1578692475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eadf566b0 0x562eae355c00 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fc10015d010 tx=0x7fc0f0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0292b3cbd09daff server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:27.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.228+0000 7fc0e37fe640 1 -- 172.21.15.5:0/1578692475 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f001d250 con 0x562eadf566b0 2024-05-20T18:36:27.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.228+0000 7fc0e37fe640 1 -- 172.21.15.5:0/1578692475 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0f001d3f0 con 0x562eadf566b0 2024-05-20T18:36:27.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.228+0000 7fc0e37fe640 1 -- 172.21.15.5:0/1578692475 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f001d5a0 con 0x562eadf566b0 2024-05-20T18:36:27.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.229+0000 7fc10f216740 1 -- 172.21.15.5:0/1578692475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eadf566b0 msgr2=0x562eae355c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:27.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.229+0000 7fc10f216740 1 --2- 172.21.15.5:0/1578692475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eadf566b0 0x562eae355c00 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fc10015d010 tx=0x7fc0f0008040 comp rx=0 tx=0).stop 2024-05-20T18:36:27.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.229+0000 7fc10f216740 1 -- 172.21.15.5:0/1578692475 shutdown_connections 2024-05-20T18:36:27.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.229+0000 7fc10f216740 1 --2- 172.21.15.5:0/1578692475 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562eadf77390 0x562eae35bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.229+0000 7fc10f216740 1 --2- 172.21.15.5:0/1578692475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eadf566b0 0x562eae355c00 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.229+0000 7fc10f216740 1 --2- 172.21.15.5:0/1578692475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562eadf4e4d0 0x562eae3556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.229+0000 7fc10f216740 1 -- 172.21.15.5:0/1578692475 >> 172.21.15.5:0/1578692475 conn(0x562eadf6b4e0 msgr2=0x562eae312830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:27.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.229+0000 7fc10f216740 1 -- 172.21.15.5:0/1578692475 shutdown_connections 2024-05-20T18:36:27.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.229+0000 7fc10f216740 1 -- 172.21.15.5:0/1578692475 wait complete. 2024-05-20T18:36:27.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.230+0000 7fc10f216740 1 -- start start 2024-05-20T18:36:27.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.230+0000 7fc10f216740 1 Processor -- start 2024-05-20T18:36:27.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.231+0000 7fc10f216740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562eadf4e4d0 0x562eae33f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.231+0000 7fc10f216740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562eadf566b0 0x562eae315670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.231+0000 7fc10f216740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eadf77390 0x562eae3527c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.232+0000 7fc10f216740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc1001fef20 con 0x562eadf77390 2024-05-20T18:36:27.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.232+0000 7fc10f216740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562eae356410 con 0x562eadf566b0 2024-05-20T18:36:27.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.232+0000 7fc10f216740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x562eae3408c0 con 0x562eadf4e4d0 2024-05-20T18:36:27.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.232+0000 7fc10f216740 10 client.0 _ll_register_callbacks cb 0x7fc0fc004b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:36:27.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.232+0000 7fc106835640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eadf77390 0x562eae3527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.232+0000 7fc106835640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eadf77390 0x562eae3527c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59986/0 (socket says 172.21.15.5:59986) 2024-05-20T18:36:27.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.232+0000 7fc106835640 1 -- 172.21.15.5:0/66203293 learned_addr learned my addr 172.21.15.5:0/66203293 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:27.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.232+0000 7fc105833640 1 --2- 172.21.15.5:0/66203293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562eadf566b0 0x562eae315670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.232+0000 7fc106034640 1 --2- 172.21.15.5:0/66203293 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562eadf4e4d0 0x562eae33f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.232+0000 7fc106835640 1 -- 172.21.15.5:0/66203293 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562eadf4e4d0 msgr2=0x562eae33f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:27.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.232+0000 7fc106835640 1 --2- 172.21.15.5:0/66203293 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562eadf4e4d0 0x562eae33f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.232+0000 7fc106835640 1 -- 172.21.15.5:0/66203293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562eadf566b0 msgr2=0x562eae315670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.232+0000 7fc106835640 1 --2- 172.21.15.5:0/66203293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562eadf566b0 0x562eae315670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.232+0000 7fc106835640 1 -- 172.21.15.5:0/66203293 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eae3163c0 con 0x562eadf77390 2024-05-20T18:36:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.232+0000 7fc106835640 1 --2- 172.21.15.5:0/66203293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eadf77390 0x562eae3527c0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8078af0 tx=0x7fc0f8007fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8617a004d9b8e3f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.232+0000 7fc0e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:36:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.232+0000 7fc0e17fa640 1 -- 172.21.15.5:0/66203293 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8076ae0 con 0x562eadf77390 2024-05-20T18:36:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.232+0000 7fc0e17fa640 1 -- 172.21.15.5:0/66203293 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 6245+0+0 (secure 0 0 0) 0x7fc0f807aca0 con 0x562eadf77390 2024-05-20T18:36:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.232+0000 7fc10f216740 1 -- 172.21.15.5:0/66203293 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0f001d3f0 con 0x562eadf77390 2024-05-20T18:36:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.232+0000 7fc0e17fa640 1 -- 172.21.15.5:0/66203293 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fc0f8082f70 con 0x562eadf77390 2024-05-20T18:36:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.233+0000 7fc10f216740 10 client.8823 fetch_fsmap learned FSMap version 88 2024-05-20T18:36:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.233+0000 7fc10f216740 1 -- 172.21.15.5:0/66203293 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x562eae316100 con 0x562eadf77390 2024-05-20T18:36:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.233+0000 7fc0e17fa640 1 -- 172.21.15.5:0/66203293 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0f001d3f0 con 0x562eadf77390 2024-05-20T18:36:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.233+0000 7fc0e17fa640 1 -- 172.21.15.5:0/66203293 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7fc0f804c030 con 0x562eadf77390 2024-05-20T18:36:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.233+0000 7fc10f216740 10 client.8823 fetch_fsmap finished waiting for FSMap version 88 2024-05-20T18:36:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.233+0000 7fc10f216740 10 client.8823 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-20T18:36:27.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.233+0000 7fc10f216740 20 client.8823 populate_metadata read hostname 'smithi005' 2024-05-20T18:36:27.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.233+0000 7fc10f216740 1 --2- 172.21.15.5:0/66203293 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x562eae315bb0 0x562eae352d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:27.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.233+0000 7fc10f216740 4 client.8823 mds_command: new command op to 8325 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T18:36:27.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.233+0000 7fc10f216740 1 -- 172.21.15.5:0/66203293 --> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562eae283780 con 0x562eae315bb0 2024-05-20T18:36:27.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.233+0000 7fc106034640 1 --2- 172.21.15.5:0/66203293 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x562eae315bb0 0x562eae352d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:27.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.233+0000 7fc0e17fa640 10 client.8823 ms_handle_connect on v2:172.21.15.5:6834/3552410668 2024-05-20T18:36:27.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.234+0000 7fc106034640 1 --2- 172.21.15.5:0/66203293 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x562eae315bb0 0x562eae352d00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d16f630d89e5f1b8 server_cookie=b9075e45c161dd38 in_seq=0 out_seq=0 2024-05-20T18:36:27.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.234+0000 7fc0e17fa640 1 -- 172.21.15.5:0/66203293 <== mds.0 v2:172.21.15.5:6834/3552410668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x562eae283780 con 0x562eae315bb0 2024-05-20T18:36:27.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.234+0000 7fc0e17fa640 10 client.8823 handle_command_reply: tid=0 2024-05-20T18:36:27.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.234+0000 7fc10f216740 1 client.8823 shutdown 2024-05-20T18:36:27.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.234+0000 7fc10f216740 1 -- 172.21.15.5:0/66203293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eadf77390 msgr2=0x562eae3527c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:27.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.234+0000 7fc10f216740 1 --2- 172.21.15.5:0/66203293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eadf77390 0x562eae3527c0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8078af0 tx=0x7fc0f8007fc0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.234+0000 7fc10f216740 1 -- 172.21.15.5:0/66203293 shutdown_connections 2024-05-20T18:36:27.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.234+0000 7fc10f216740 1 --2- 172.21.15.5:0/66203293 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x562eae315bb0 0x562eae352d00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.234+0000 7fc10f216740 1 --2- 172.21.15.5:0/66203293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eadf77390 0x562eae3527c0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.234+0000 7fc10f216740 1 --2- 172.21.15.5:0/66203293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562eadf566b0 0x562eae315670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.234+0000 7fc0e17fa640 0 client.8823 ms_handle_reset on v2:172.21.15.5:6834/3552410668 2024-05-20T18:36:27.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.234+0000 7fc10f216740 1 --2- 172.21.15.5:0/66203293 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562eadf4e4d0 0x562eae33f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.234+0000 7fc10f216740 1 -- 172.21.15.5:0/66203293 >> 172.21.15.5:0/66203293 conn(0x562eadf6b4e0 msgr2=0x562eae317a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:27.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.234+0000 7fc10f216740 1 -- 172.21.15.5:0/66203293 shutdown_connections 2024-05-20T18:36:27.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.234+0000 7fc10f216740 1 -- 172.21.15.5:0/66203293 wait complete. 2024-05-20T18:36:27.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.234+0000 7fc10f216740 20 client.8823 trim_cache size 0 max 16384 2024-05-20T18:36:27.240 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:36:27.240 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-20T18:36:27.241 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-05-20T18:36:27.241 INFO:teuthology.orchestra.run.smithi005.stdout: "fe4defaa-27d8-48f2-a366-c6ee59fc6d8e": { 2024-05-20T18:36:27.241 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-05-20T18:36:27.241 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "fe4defaa-27d8-48f2-a366-c6ee59fc6d8e", 2024-05-20T18:36:27.241 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,repair" 2024-05-20T18:36:27.241 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:36:27.241 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:36:27.241 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:36:27.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.235+0000 7fc108298640 1 -- 172.21.15.5:0/2221081257 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc0e406be30 msgr2=0x7fc0e406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:27.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.235+0000 7fc108298640 1 --2- 172.21.15.5:0/2221081257 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc0e406be30 0x7fc0e406e340 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc007fd0 tx=0x7fc0fc007e80 comp rx=0 tx=0).stop 2024-05-20T18:36:27.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.235+0000 7fc108298640 1 -- 172.21.15.5:0/2221081257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc10015d530 msgr2=0x7fc10015c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:27.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.235+0000 7fc108298640 1 --2- 172.21.15.5:0/2221081257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc10015d530 0x7fc10015c360 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009eb0 tx=0x7fc0f8004a40 comp rx=0 tx=0).stop 2024-05-20T18:36:27.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.235+0000 7fc108298640 1 -- 172.21.15.5:0/2221081257 shutdown_connections 2024-05-20T18:36:27.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.235+0000 7fc108298640 1 --2- 172.21.15.5:0/2221081257 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc0e406be30 0x7fc0e406e340 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.235+0000 7fc108298640 1 --2- 172.21.15.5:0/2221081257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc10015d530 0x7fc10015c360 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.235+0000 7fc108298640 1 --2- 172.21.15.5:0/2221081257 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1000b1e10 0x7fc1000c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.235+0000 7fc108298640 1 --2- 172.21.15.5:0/2221081257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc100070850 0x7fc100154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:27.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.235+0000 7fc108298640 1 -- 172.21.15.5:0/2221081257 >> 172.21.15.5:0/2221081257 conn(0x7fc1000d29d0 msgr2=0x7fc10015e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:27.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.235+0000 7fc108298640 1 -- 172.21.15.5:0/2221081257 shutdown_connections 2024-05-20T18:36:27.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:27.235+0000 7fc108298640 1 -- 172.21.15.5:0/2221081257 wait complete. 2024-05-20T18:36:27.247 INFO:tasks.cephfs.filesystem:scrub status for tag:fe4defaa-27d8-48f2-a366-c6ee59fc6d8e - {'path': '/', 'tag': 'fe4defaa-27d8-48f2-a366-c6ee59fc6d8e', 'options': 'recursive,repair'} 2024-05-20T18:36:27.385 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-20T18:36:27.385+0000 7f1d3868b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T18:36:28.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:28.379+0000 7f7c75ffb640 1 -- 172.21.15.5:0/1014737986 <== mon.0 v2:172.21.15.5:3300/0 8 ==== log(8 entries from seq 1 at 2024-05-20T18:36:27.058948+0000) ==== 1407+0+0 (secure 0 0 0) 0x7f7c70077b80 con 0x7f7c8009b210 2024-05-20T18:36:28.381 INFO:teuthology.orchestra.run.smithi005.stdout:2024-05-20T18:36:27.058948+0000 mds.c [INF] scrub queued for path: / 2024-05-20T18:36:28.381 INFO:teuthology.orchestra.run.smithi005.stdout:2024-05-20T18:36:27.058952+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-05-20T18:36:28.381 INFO:teuthology.orchestra.run.smithi005.stdout:2024-05-20T18:36:27.058956+0000 mds.c [INF] scrub summary: active paths [/] 2024-05-20T18:36:28.381 INFO:teuthology.orchestra.run.smithi005.stdout:2024-05-20T18:36:27.059550+0000 mds.c [ERR] scrub: inode wrongly marked free: 0x10000000001 2024-05-20T18:36:28.381 INFO:teuthology.orchestra.run.smithi005.stdout:2024-05-20T18:36:27.059555+0000 mds.c [ERR] inode table repaired for inode: 0x10000000001 2024-05-20T18:36:28.381 INFO:teuthology.orchestra.run.smithi005.stdout:2024-05-20T18:36:27.059686+0000 mds.c [ERR] scrub: inode wrongly marked free: 0x10000000002 2024-05-20T18:36:28.381 INFO:teuthology.orchestra.run.smithi005.stdout:2024-05-20T18:36:27.059692+0000 mds.c [ERR] inode table repaired for inode: 0x10000000002 2024-05-20T18:36:28.381 INFO:teuthology.orchestra.run.smithi005.stdout:2024-05-20T18:36:27.060374+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-05-20T18:36:31.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:31.772+0000 7f7c75ffb640 1 -- 172.21.15.5:0/1014737986 <== mon.0 v2:172.21.15.5:3300/0 9 ==== log(1 entries from seq 9 at 2024-05-20T18:36:31.452379+0000) ==== 189+0+0 (secure 0 0 0) 0x7f7c700a9b50 con 0x7f7c8009b210 2024-05-20T18:36:31.774 INFO:teuthology.orchestra.run.smithi005.stdout:2024-05-20T18:36:31.452379+0000 mds.c [INF] scrub summary: idle 2024-05-20T18:36:41.970 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:36:41.973 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:36:57.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:36:57.248 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-20T18:36:57.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.379+0000 7f7b9b461640 1 Processor -- start 2024-05-20T18:36:57.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.380+0000 7f7b9b461640 1 -- start start 2024-05-20T18:36:57.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.381+0000 7f7b9b461640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7b940cf850 0x7f7b940b6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.381+0000 7f7b9b461640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b940d14f0 0x7f7b940b6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.381+0000 7f7b9b461640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b940d1ac0 con 0x7f7b940d10e0 2024-05-20T18:36:57.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.381+0000 7f7b9b461640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7b940d1da0 con 0x7f7b940cf850 2024-05-20T18:36:57.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.381+0000 7f7b9b461640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7b940d2080 con 0x7f7b940d14f0 2024-05-20T18:36:57.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.381+0000 7f7b989fc640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7b940cf850 0x7f7b940b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.382+0000 7f7b999fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b940d14f0 0x7f7b940b6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.382+0000 7f7b991fd640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f7b940d10e0 0x7f7b940cf140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:55504/0 (socket says 172.21.15.5:55504) 2024-05-20T18:36:57.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.382+0000 7f7b991fd640 1 -- 172.21.15.5:0/3834575143 learned_addr learned my addr 172.21.15.5:0/3834575143 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:36:57.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.382+0000 7f7b989fc640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7b940cf850 0x7f7b940b6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:58388/0 (socket says 172.21.15.5:58388) 2024-05-20T18:36:57.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.382+0000 7f7b989fc640 1 -- 172.21.15.5:0/3834575143 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b940d14f0 msgr2=0x7f7b940b6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:57.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.382+0000 7f7b989fc640 1 --2- 172.21.15.5:0/3834575143 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b940d14f0 0x7f7b940b6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.382+0000 7f7b989fc640 1 -- 172.21.15.5:0/3834575143 >> v1:172.21.15.63:6789/0 conn(0x7f7b940d10e0 legacy=0x7f7b940cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:57.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.382+0000 7f7b989fc640 1 -- 172.21.15.5:0/3834575143 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b940d2360 con 0x7f7b940cf850 2024-05-20T18:36:57.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.382+0000 7f7b93fff640 1 -- 172.21.15.5:0/3834575143 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 997783679 0 0) 0x7f7b940d1ac0 con 0x7f7b940d10e0 2024-05-20T18:36:57.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.382+0000 7f7b999fe640 1 --2- 172.21.15.5:0/3834575143 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b940d14f0 0x7f7b940b6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:36:57.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.383+0000 7f7b989fc640 1 --2- 172.21.15.5:0/3834575143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b940cf850 0x7f7b940b6200 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7b880097a0 tx=0x7f7b8802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f19f7ddf75878b16 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:57.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.383+0000 7f7b93fff640 1 -- 172.21.15.5:0/3834575143 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88037070 con 0x7f7b940cf850 2024-05-20T18:36:57.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.383+0000 7f7b93fff640 1 -- 172.21.15.5:0/3834575143 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b88032440 con 0x7f7b940cf850 2024-05-20T18:36:57.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.383+0000 7f7b93fff640 1 -- 172.21.15.5:0/3834575143 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88032810 con 0x7f7b940cf850 2024-05-20T18:36:57.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.384+0000 7f7b9b461640 1 -- 172.21.15.5:0/3834575143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b940cf850 msgr2=0x7f7b940b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:57.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.384+0000 7f7b9b461640 1 --2- 172.21.15.5:0/3834575143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b940cf850 0x7f7b940b6200 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7b880097a0 tx=0x7f7b8802a180 comp rx=0 tx=0).stop 2024-05-20T18:36:57.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.384+0000 7f7b9b461640 1 -- 172.21.15.5:0/3834575143 shutdown_connections 2024-05-20T18:36:57.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.384+0000 7f7b9b461640 1 --2- 172.21.15.5:0/3834575143 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b940d14f0 0x7f7b940b6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.384+0000 7f7b9b461640 1 --2- 172.21.15.5:0/3834575143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b940cf850 0x7f7b940b6200 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.384+0000 7f7b9b461640 1 -- 172.21.15.5:0/3834575143 >> 172.21.15.5:0/3834575143 conn(0x7f7b940b44d0 msgr2=0x7f7b940b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:57.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.384+0000 7f7b9b461640 1 -- 172.21.15.5:0/3834575143 shutdown_connections 2024-05-20T18:36:57.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.384+0000 7f7b9b461640 1 -- 172.21.15.5:0/3834575143 wait complete. 2024-05-20T18:36:57.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.386+0000 7f7b9b461640 1 Processor -- start 2024-05-20T18:36:57.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.386+0000 7f7b9b461640 1 -- start start 2024-05-20T18:36:57.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.387+0000 7f7b9b461640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b940cf850 0x7f7b941552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.387+0000 7f7b9b461640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b940d10e0 0x7f7b940caac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.387+0000 7f7b9b461640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b940d14f0 0x7f7b940ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.387+0000 7f7b9b461640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7b940b8440 con 0x7f7b940d14f0 2024-05-20T18:36:57.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.387+0000 7f7b9b461640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7b940b82c0 con 0x7f7b940cf850 2024-05-20T18:36:57.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.387+0000 7f7b9b461640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7b940709f0 con 0x7f7b940d10e0 2024-05-20T18:36:57.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.387+0000 7f7b999fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b940d14f0 0x7f7b940ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.387+0000 7f7b999fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b940d14f0 0x7f7b940ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48620/0 (socket says 172.21.15.5:48620) 2024-05-20T18:36:57.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.387+0000 7f7b999fe640 1 -- 172.21.15.5:0/2305325377 learned_addr learned my addr 172.21.15.5:0/2305325377 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:57.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.388+0000 7f7b991fd640 1 --2- 172.21.15.5:0/2305325377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b940cf850 0x7f7b941552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.388+0000 7f7b989fc640 1 --2- 172.21.15.5:0/2305325377 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b940d10e0 0x7f7b940caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.388+0000 7f7b999fe640 1 -- 172.21.15.5:0/2305325377 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b940d10e0 msgr2=0x7f7b940caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:57.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.388+0000 7f7b999fe640 1 --2- 172.21.15.5:0/2305325377 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b940d10e0 0x7f7b940caac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.388+0000 7f7b999fe640 1 -- 172.21.15.5:0/2305325377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b940cf850 msgr2=0x7f7b941552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:57.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.388+0000 7f7b999fe640 1 --2- 172.21.15.5:0/2305325377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b940cf850 0x7f7b941552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.388+0000 7f7b999fe640 1 -- 172.21.15.5:0/2305325377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b940cb000 con 0x7f7b940d14f0 2024-05-20T18:36:57.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.388+0000 7f7b991fd640 1 --2- 172.21.15.5:0/2305325377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b940cf850 0x7f7b941552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:36:57.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.388+0000 7f7b999fe640 1 --2- 172.21.15.5:0/2305325377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b940d14f0 0x7f7b940ccff0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00e890 tx=0x7f7b8c00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:57.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.388+0000 7f7b91ffb640 1 -- 172.21.15.5:0/2305325377 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c017070 con 0x7f7b940d14f0 2024-05-20T18:36:57.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.388+0000 7f7b9b461640 1 -- 172.21.15.5:0/2305325377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b940cb510 con 0x7f7b940d14f0 2024-05-20T18:36:57.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.388+0000 7f7b91ffb640 1 -- 172.21.15.5:0/2305325377 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b8c00c8e0 con 0x7f7b940d14f0 2024-05-20T18:36:57.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.388+0000 7f7b91ffb640 1 -- 172.21.15.5:0/2305325377 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c00ccb0 con 0x7f7b940d14f0 2024-05-20T18:36:57.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.388+0000 7f7b9b461640 1 -- 172.21.15.5:0/2305325377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b940cbad0 con 0x7f7b940d14f0 2024-05-20T18:36:57.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.390+0000 7f7b91ffb640 1 -- 172.21.15.5:0/2305325377 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7b8c01a070 con 0x7f7b940d14f0 2024-05-20T18:36:57.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.391+0000 7f7b91ffb640 1 --2- 172.21.15.5:0/2305325377 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7b8006be60 0x7f7b8006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.391+0000 7f7b91ffb640 1 -- 172.21.15.5:0/2305325377 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6245+0+0 (secure 0 0 0) 0x7f7b8c08b250 con 0x7f7b940d14f0 2024-05-20T18:36:57.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.391+0000 7f7b991fd640 1 --2- 172.21.15.5:0/2305325377 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7b8006be60 0x7f7b8006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.391+0000 7f7b991fd640 1 --2- 172.21.15.5:0/2305325377 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7b8006be60 0x7f7b8006e370 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7b840012e0 tx=0x7f7b84005b90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:57.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.392+0000 7f7ba23e1740 1 Processor -- start 2024-05-20T18:36:57.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.392+0000 7f7ba23e1740 1 -- start start 2024-05-20T18:36:57.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.393+0000 7f7ba23e1740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fbbb7fe390 0x55fbbbbc8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.393+0000 7f7ba23e1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fbbb7d54d0 0x55fbbbbe3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.393+0000 7f7ba23e1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fbbb7dd6b0 0x55fbbbbe3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.393+0000 7f7ba23e1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fbbbb02d00 con 0x55fbbb7d54d0 2024-05-20T18:36:57.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.393+0000 7f7ba23e1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fbbbaaa4c0 con 0x55fbbb7dd6b0 2024-05-20T18:36:57.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.393+0000 7f7ba23e1740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55fbbbafecc0 con 0x55fbbb7fe390 2024-05-20T18:36:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.393+0000 7f7b989fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fbbb7d54d0 0x55fbbbbe3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.393+0000 7f7b989fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fbbb7d54d0 0x55fbbbbe3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48636/0 (socket says 172.21.15.5:48636) 2024-05-20T18:36:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.393+0000 7f7b989fc640 1 -- 172.21.15.5:0/2634120658 learned_addr learned my addr 172.21.15.5:0/2634120658 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.394+0000 7f7b991fd640 1 --2- 172.21.15.5:0/2634120658 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fbbb7fe390 0x55fbbbbc8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.394+0000 7f7b999fe640 1 --2- 172.21.15.5:0/2634120658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fbbb7dd6b0 0x55fbbbbe3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.394+0000 7f7b989fc640 1 -- 172.21.15.5:0/2634120658 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fbbb7fe390 msgr2=0x55fbbbbc8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.394+0000 7f7b989fc640 1 --2- 172.21.15.5:0/2634120658 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fbbb7fe390 0x55fbbbbc8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.394+0000 7f7b989fc640 1 -- 172.21.15.5:0/2634120658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fbbb7dd6b0 msgr2=0x55fbbbbe3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.394+0000 7f7b989fc640 1 --2- 172.21.15.5:0/2634120658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fbbb7dd6b0 0x55fbbbbe3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.394+0000 7f7b989fc640 1 -- 172.21.15.5:0/2634120658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbbb8e07d0 con 0x55fbbb7d54d0 2024-05-20T18:36:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.394+0000 7f7b989fc640 1 --2- 172.21.15.5:0/2634120658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fbbb7d54d0 0x55fbbbbe3890 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f7b880097a0 tx=0x7f7b8802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1530e54227aa60 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.394+0000 7f7b7affd640 1 -- 172.21.15.5:0/2634120658 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88041070 con 0x55fbbb7d54d0 2024-05-20T18:36:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.394+0000 7f7b7affd640 1 -- 172.21.15.5:0/2634120658 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b88032880 con 0x55fbbb7d54d0 2024-05-20T18:36:57.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.394+0000 7f7b7affd640 1 -- 172.21.15.5:0/2634120658 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88037040 con 0x55fbbb7d54d0 2024-05-20T18:36:57.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.395+0000 7f7ba23e1740 1 -- 172.21.15.5:0/2634120658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fbbb7d54d0 msgr2=0x55fbbbbe3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:57.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.395+0000 7f7ba23e1740 1 --2- 172.21.15.5:0/2634120658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fbbb7d54d0 0x55fbbbbe3890 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f7b880097a0 tx=0x7f7b8802c000 comp rx=0 tx=0).stop 2024-05-20T18:36:57.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.395+0000 7f7ba23e1740 1 -- 172.21.15.5:0/2634120658 shutdown_connections 2024-05-20T18:36:57.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.395+0000 7f7ba23e1740 1 --2- 172.21.15.5:0/2634120658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fbbb7dd6b0 0x55fbbbbe3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.395+0000 7f7ba23e1740 1 --2- 172.21.15.5:0/2634120658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fbbb7d54d0 0x55fbbbbe3890 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.395+0000 7f7ba23e1740 1 --2- 172.21.15.5:0/2634120658 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fbbb7fe390 0x55fbbbbc8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.395+0000 7f7ba23e1740 1 -- 172.21.15.5:0/2634120658 >> 172.21.15.5:0/2634120658 conn(0x55fbbb7f24e0 msgr2=0x55fbbbb9e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:57.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.395+0000 7f7ba23e1740 1 -- 172.21.15.5:0/2634120658 shutdown_connections 2024-05-20T18:36:57.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.395+0000 7f7ba23e1740 1 -- 172.21.15.5:0/2634120658 wait complete. 2024-05-20T18:36:57.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.396+0000 7f7ba23e1740 1 -- start start 2024-05-20T18:36:57.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.396+0000 7f7ba23e1740 1 Processor -- start 2024-05-20T18:36:57.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.398+0000 7f7ba23e1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fbbb7d54d0 0x55fbbbbc28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.398+0000 7f7ba23e1740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fbbb7dd6b0 0x55fbbbbe3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.398+0000 7f7ba23e1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fbbb7fe390 0x55fbbbbe0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.398+0000 7f7ba23e1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fbbbabdb90 con 0x55fbbb7fe390 2024-05-20T18:36:57.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.398+0000 7f7ba23e1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55fbbbbcb3d0 con 0x55fbbb7d54d0 2024-05-20T18:36:57.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.398+0000 7f7ba23e1740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55fbbbbcb6b0 con 0x55fbbb7dd6b0 2024-05-20T18:36:57.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.398+0000 7f7ba23e1740 10 client.0 _ll_register_callbacks cb 0x55fbbbaf9da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:36:57.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.398+0000 7f7b999fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fbbb7fe390 0x55fbbbbe0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.398+0000 7f7b999fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fbbb7fe390 0x55fbbbbe0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48642/0 (socket says 172.21.15.5:48642) 2024-05-20T18:36:57.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.398+0000 7f7b999fe640 1 -- 172.21.15.5:0/904990558 learned_addr learned my addr 172.21.15.5:0/904990558 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:57.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.398+0000 7f7b991fd640 1 --2- 172.21.15.5:0/904990558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fbbb7d54d0 0x55fbbbbc28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.398+0000 7f7b989fc640 1 --2- 172.21.15.5:0/904990558 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fbbb7dd6b0 0x55fbbbbe3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.398+0000 7f7b999fe640 1 -- 172.21.15.5:0/904990558 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fbbb7dd6b0 msgr2=0x55fbbbbe3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:57.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.398+0000 7f7b999fe640 1 --2- 172.21.15.5:0/904990558 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fbbb7dd6b0 0x55fbbbbe3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.398+0000 7f7b999fe640 1 -- 172.21.15.5:0/904990558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fbbb7d54d0 msgr2=0x55fbbbbc28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:57.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.398+0000 7f7b999fe640 1 --2- 172.21.15.5:0/904990558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fbbb7d54d0 0x55fbbbbc28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.398+0000 7f7b999fe640 1 -- 172.21.15.5:0/904990558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbbb8163f0 con 0x55fbbb7fe390 2024-05-20T18:36:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.398+0000 7f7b999fe640 1 --2- 172.21.15.5:0/904990558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fbbb7fe390 0x55fbbbbe0350 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00e860 tx=0x7f7b8c009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d56fd5b5ef4661 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.398+0000 7f7b73fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:36:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.398+0000 7f7b73fff640 1 -- 172.21.15.5:0/904990558 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c021070 con 0x55fbbb7fe390 2024-05-20T18:36:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.398+0000 7f7b73fff640 1 -- 172.21.15.5:0/904990558 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 6245+0+0 (secure 0 0 0) 0x7f7b8c05d0d0 con 0x55fbbb7fe390 2024-05-20T18:36:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.398+0000 7f7ba23e1740 1 -- 172.21.15.5:0/904990558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b88032880 con 0x55fbbb7fe390 2024-05-20T18:36:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.399+0000 7f7b73fff640 1 -- 172.21.15.5:0/904990558 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f7b8c05d580 con 0x55fbbb7fe390 2024-05-20T18:36:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.399+0000 7f7ba23e1740 10 client.8832 fetch_fsmap learned FSMap version 88 2024-05-20T18:36:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.399+0000 7f7ba23e1740 1 -- 172.21.15.5:0/904990558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x55fbbb7d60f0 con 0x55fbbb7fe390 2024-05-20T18:36:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.399+0000 7f7b73fff640 1 -- 172.21.15.5:0/904990558 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b88032880 con 0x55fbbb7fe390 2024-05-20T18:36:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.399+0000 7f7b73fff640 1 -- 172.21.15.5:0/904990558 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7f7b8c0780b0 con 0x55fbbb7fe390 2024-05-20T18:36:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.399+0000 7f7ba23e1740 10 client.8832 fetch_fsmap finished waiting for FSMap version 88 2024-05-20T18:36:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.399+0000 7f7ba23e1740 10 client.8832 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-20T18:36:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.399+0000 7f7ba23e1740 20 client.8832 populate_metadata read hostname 'smithi005' 2024-05-20T18:36:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.399+0000 7f7ba23e1740 1 --2- 172.21.15.5:0/904990558 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x55fbbb7bcf60 0x55fbbbbe1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.399+0000 7f7ba23e1740 4 client.8832 mds_command: new command op to 8325 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:36:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.399+0000 7f7ba23e1740 1 -- 172.21.15.5:0/904990558 --> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbbbb10430 con 0x55fbbb7bcf60 2024-05-20T18:36:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.399+0000 7f7b991fd640 1 --2- 172.21.15.5:0/904990558 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x55fbbb7bcf60 0x55fbbbbe1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.399+0000 7f7b73fff640 10 client.8832 ms_handle_connect on v2:172.21.15.5:6834/3552410668 2024-05-20T18:36:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.399+0000 7f7b991fd640 1 --2- 172.21.15.5:0/904990558 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x55fbbb7bcf60 0x55fbbbbe1810 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a75e20038ba54e9 server_cookie=bd4dadd17869426d in_seq=0 out_seq=0 2024-05-20T18:36:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.400+0000 7f7b73fff640 1 -- 172.21.15.5:0/904990558 <== mds.0 v2:172.21.15.5:6834/3552410668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fbbbb10430 con 0x55fbbb7bcf60 2024-05-20T18:36:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.400+0000 7f7b73fff640 10 client.8832 handle_command_reply: tid=0 2024-05-20T18:36:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.400+0000 7f7ba23e1740 1 client.8832 shutdown 2024-05-20T18:36:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.400+0000 7f7ba23e1740 1 -- 172.21.15.5:0/904990558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fbbb7fe390 msgr2=0x55fbbbbe0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.400+0000 7f7ba23e1740 1 --2- 172.21.15.5:0/904990558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fbbb7fe390 0x55fbbbbe0350 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00e860 tx=0x7f7b8c009480 comp rx=0 tx=0).stop 2024-05-20T18:36:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.400+0000 7f7ba23e1740 1 -- 172.21.15.5:0/904990558 shutdown_connections 2024-05-20T18:36:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.400+0000 7f7ba23e1740 1 --2- 172.21.15.5:0/904990558 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x55fbbb7bcf60 0x55fbbbbe1810 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.400+0000 7f7ba23e1740 1 --2- 172.21.15.5:0/904990558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fbbb7fe390 0x55fbbbbe0350 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.400+0000 7f7ba23e1740 1 --2- 172.21.15.5:0/904990558 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fbbb7dd6b0 0x55fbbbbe3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.400+0000 7f7b73fff640 0 client.8832 ms_handle_reset on v2:172.21.15.5:6834/3552410668 2024-05-20T18:36:57.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.400+0000 7f7ba23e1740 1 --2- 172.21.15.5:0/904990558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fbbb7d54d0 0x55fbbbbc28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.400+0000 7f7ba23e1740 1 -- 172.21.15.5:0/904990558 >> 172.21.15.5:0/904990558 conn(0x55fbbb7f24e0 msgr2=0x55fbbbb9e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:57.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.401+0000 7f7ba23e1740 1 -- 172.21.15.5:0/904990558 shutdown_connections 2024-05-20T18:36:57.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.401+0000 7f7ba23e1740 1 -- 172.21.15.5:0/904990558 wait complete. 2024-05-20T18:36:57.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.401+0000 7f7ba23e1740 20 client.8832 trim_cache size 0 max 16384 2024-05-20T18:36:57.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.419+0000 7f7ba23e1740 1 Processor -- start 2024-05-20T18:36:57.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.419+0000 7f7ba23e1740 1 -- start start 2024-05-20T18:36:57.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.420+0000 7f7ba23e1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fbbb7d54d0 0x55fbbbb94590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.420+0000 7f7ba23e1740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fbbb7dd6b0 0x55fbbbc059d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.420+0000 7f7ba23e1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fbbb7fe390 0x55fbbbb9c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.420+0000 7f7ba23e1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fbbbbcc920 con 0x55fbbb7fe390 2024-05-20T18:36:57.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.420+0000 7f7ba23e1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7b94202430 con 0x55fbbb7d54d0 2024-05-20T18:36:57.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.420+0000 7f7ba23e1740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7b94201450 con 0x55fbbb7dd6b0 2024-05-20T18:36:57.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.421+0000 7f7b999fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fbbb7fe390 0x55fbbbb9c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.421+0000 7f7b999fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fbbb7fe390 0x55fbbbb9c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48658/0 (socket says 172.21.15.5:48658) 2024-05-20T18:36:57.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.421+0000 7f7b999fe640 1 -- 172.21.15.5:0/2873727664 learned_addr learned my addr 172.21.15.5:0/2873727664 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:57.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.421+0000 7f7b991fd640 1 --2- 172.21.15.5:0/2873727664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fbbb7d54d0 0x55fbbbb94590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.421+0000 7f7b999fe640 1 -- 172.21.15.5:0/2873727664 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fbbb7dd6b0 msgr2=0x55fbbbc059d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:57.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.421+0000 7f7b999fe640 1 --2- 172.21.15.5:0/2873727664 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fbbb7dd6b0 0x55fbbbc059d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.421+0000 7f7b999fe640 1 -- 172.21.15.5:0/2873727664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fbbb7d54d0 msgr2=0x55fbbbb94590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:57.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.421+0000 7f7b999fe640 1 --2- 172.21.15.5:0/2873727664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fbbb7d54d0 0x55fbbbb94590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.421+0000 7f7b999fe640 1 -- 172.21.15.5:0/2873727664 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbbbbc24b0 con 0x55fbbb7fe390 2024-05-20T18:36:57.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.421+0000 7f7b999fe640 1 --2- 172.21.15.5:0/2873727664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fbbb7fe390 0x55fbbbb9c670 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c059110 tx=0x7f7b8c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344ee64a7ea769fd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:57.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.421+0000 7f7b7a7fc640 1 -- 172.21.15.5:0/2873727664 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c061e90 con 0x55fbbb7fe390 2024-05-20T18:36:57.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.421+0000 7f7b7a7fc640 1 -- 172.21.15.5:0/2873727664 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b8c00ce50 con 0x55fbbb7fe390 2024-05-20T18:36:57.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.421+0000 7f7b7a7fc640 1 -- 172.21.15.5:0/2873727664 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c05d0d0 con 0x55fbbb7fe390 2024-05-20T18:36:57.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.422+0000 7f7ba23e1740 1 -- 172.21.15.5:0/2873727664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fbbb7fe390 msgr2=0x55fbbbb9c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:57.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.422+0000 7f7ba23e1740 1 --2- 172.21.15.5:0/2873727664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fbbb7fe390 0x55fbbbb9c670 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c059110 tx=0x7f7b8c01d000 comp rx=0 tx=0).stop 2024-05-20T18:36:57.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.422+0000 7f7ba23e1740 1 -- 172.21.15.5:0/2873727664 shutdown_connections 2024-05-20T18:36:57.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.422+0000 7f7ba23e1740 1 --2- 172.21.15.5:0/2873727664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fbbb7fe390 0x55fbbbb9c670 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.422+0000 7f7ba23e1740 1 --2- 172.21.15.5:0/2873727664 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fbbb7dd6b0 0x55fbbbc059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.422+0000 7f7ba23e1740 1 --2- 172.21.15.5:0/2873727664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fbbb7d54d0 0x55fbbbb94590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.422+0000 7f7ba23e1740 1 -- 172.21.15.5:0/2873727664 >> 172.21.15.5:0/2873727664 conn(0x55fbbb7f24e0 msgr2=0x55fbbbbdd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:57.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.422+0000 7f7ba23e1740 1 -- 172.21.15.5:0/2873727664 shutdown_connections 2024-05-20T18:36:57.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.422+0000 7f7ba23e1740 1 -- 172.21.15.5:0/2873727664 wait complete. 2024-05-20T18:36:57.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.423+0000 7f7ba23e1740 1 -- start start 2024-05-20T18:36:57.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.423+0000 7f7ba23e1740 1 Processor -- start 2024-05-20T18:36:57.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.424+0000 7f7ba23e1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fbbb7d54d0 0x55fbbbbf1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.424+0000 7f7ba23e1740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fbbb7dd6b0 0x55fbbbbbebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.424+0000 7f7ba23e1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fbbb7fe390 0x55fbbbbbf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.424+0000 7f7ba23e1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7b94200e30 con 0x55fbbb7d54d0 2024-05-20T18:36:57.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.424+0000 7f7ba23e1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7b94153770 con 0x55fbbb7fe390 2024-05-20T18:36:57.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.424+0000 7f7ba23e1740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55fbbbb98290 con 0x55fbbb7dd6b0 2024-05-20T18:36:57.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.424+0000 7f7ba23e1740 10 client.0 _ll_register_callbacks cb 0x7f7b8c059040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:36:57.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.424+0000 7f7b991fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fbbb7d54d0 0x55fbbbbf1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.424+0000 7f7b991fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fbbb7d54d0 0x55fbbbbf1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48672/0 (socket says 172.21.15.5:48672) 2024-05-20T18:36:57.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.424+0000 7f7b991fd640 1 -- 172.21.15.5:0/2937360420 learned_addr learned my addr 172.21.15.5:0/2937360420 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:57.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.425+0000 7f7b991fd640 1 -- 172.21.15.5:0/2937360420 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fbbb7dd6b0 msgr2=0x55fbbbbbebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:57.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.425+0000 7f7b991fd640 1 --2- 172.21.15.5:0/2937360420 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fbbb7dd6b0 0x55fbbbbbebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.425+0000 7f7b991fd640 1 -- 172.21.15.5:0/2937360420 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fbbb7fe390 msgr2=0x55fbbbbbf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:57.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.425+0000 7f7b991fd640 1 --2- 172.21.15.5:0/2937360420 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fbbb7fe390 0x55fbbbbbf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.425+0000 7f7b991fd640 1 -- 172.21.15.5:0/2937360420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbbbb9c210 con 0x55fbbb7d54d0 2024-05-20T18:36:57.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.425+0000 7f7b991fd640 1 --2- 172.21.15.5:0/2937360420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fbbb7d54d0 0x55fbbbbf1280 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x55fbbbbe1ee0 tx=0x7f7b84001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ef9a455b37fb19 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:57.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.425+0000 7f7b73fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:36:57.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.425+0000 7f7b73fff640 1 -- 172.21.15.5:0/2937360420 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84014880 con 0x55fbbb7d54d0 2024-05-20T18:36:57.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.425+0000 7f7b73fff640 1 -- 172.21.15.5:0/2937360420 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 6245+0+0 (secure 0 0 0) 0x7f7b84010040 con 0x55fbbb7d54d0 2024-05-20T18:36:57.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.425+0000 7f7ba23e1740 1 -- 172.21.15.5:0/2937360420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b8c00ce50 con 0x55fbbb7d54d0 2024-05-20T18:36:57.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.425+0000 7f7b73fff640 1 -- 172.21.15.5:0/2937360420 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f7b84036050 con 0x55fbbb7d54d0 2024-05-20T18:36:57.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.425+0000 7f7ba23e1740 10 client.8838 fetch_fsmap learned FSMap version 88 2024-05-20T18:36:57.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.425+0000 7f7ba23e1740 1 -- 172.21.15.5:0/2937360420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x55fbbbc05140 con 0x55fbbb7d54d0 2024-05-20T18:36:57.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.425+0000 7f7b73fff640 1 -- 172.21.15.5:0/2937360420 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b8c00ce50 con 0x55fbbb7d54d0 2024-05-20T18:36:57.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.426+0000 7f7b73fff640 1 -- 172.21.15.5:0/2937360420 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7f7b84039020 con 0x55fbbb7d54d0 2024-05-20T18:36:57.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.426+0000 7f7ba23e1740 10 client.8838 fetch_fsmap finished waiting for FSMap version 88 2024-05-20T18:36:57.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.426+0000 7f7ba23e1740 10 client.8838 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-05-20T18:36:57.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.426+0000 7f7ba23e1740 20 client.8838 populate_metadata read hostname 'smithi005' 2024-05-20T18:36:57.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.426+0000 7f7ba23e1740 1 --2- 172.21.15.5:0/2937360420 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x55fbbbb97b90 0x55fbbbbbf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.426+0000 7f7ba23e1740 4 client.8838 mds_command: new command op to 8325 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-20T18:36:57.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.426+0000 7f7ba23e1740 1 -- 172.21.15.5:0/2937360420 --> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fbbbb0a780 con 0x55fbbbb97b90 2024-05-20T18:36:57.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.426+0000 7f7b989fc640 1 --2- 172.21.15.5:0/2937360420 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x55fbbbb97b90 0x55fbbbbbf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.426+0000 7f7b73fff640 10 client.8838 ms_handle_connect on v2:172.21.15.5:6834/3552410668 2024-05-20T18:36:57.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.426+0000 7f7b989fc640 1 --2- 172.21.15.5:0/2937360420 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x55fbbbb97b90 0x55fbbbbbf640 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=175c34856f87d8b4 server_cookie=4a4c3307f8252b1f in_seq=0 out_seq=0 2024-05-20T18:36:57.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.426+0000 7f7b73fff640 1 -- 172.21.15.5:0/2937360420 <== mds.0 v2:172.21.15.5:6834/3552410668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fbbbb0a780 con 0x55fbbbb97b90 2024-05-20T18:36:57.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.426+0000 7f7b73fff640 10 client.8838 handle_command_reply: tid=0 2024-05-20T18:36:57.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.426+0000 7f7ba23e1740 1 client.8838 shutdown 2024-05-20T18:36:57.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.426+0000 7f7ba23e1740 1 -- 172.21.15.5:0/2937360420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fbbb7d54d0 msgr2=0x55fbbbbf1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:57.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.426+0000 7f7ba23e1740 1 --2- 172.21.15.5:0/2937360420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fbbb7d54d0 0x55fbbbbf1280 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x55fbbbbe1ee0 tx=0x7f7b84001250 comp rx=0 tx=0).stop 2024-05-20T18:36:57.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.427+0000 7f7ba23e1740 1 -- 172.21.15.5:0/2937360420 shutdown_connections 2024-05-20T18:36:57.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.427+0000 7f7ba23e1740 1 --2- 172.21.15.5:0/2937360420 >> [v2:172.21.15.5:6834/3552410668,v1:172.21.15.5:6835/3552410668] conn(0x55fbbbb97b90 0x55fbbbbbf640 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.427+0000 7f7ba23e1740 1 --2- 172.21.15.5:0/2937360420 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55fbbb7fe390 0x55fbbbbbf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.427+0000 7f7ba23e1740 1 --2- 172.21.15.5:0/2937360420 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55fbbb7dd6b0 0x55fbbbbbebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.427+0000 7f7b73fff640 0 client.8838 ms_handle_reset on v2:172.21.15.5:6834/3552410668 2024-05-20T18:36:57.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.427+0000 7f7ba23e1740 1 --2- 172.21.15.5:0/2937360420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fbbb7d54d0 0x55fbbbbf1280 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.427+0000 7f7ba23e1740 1 -- 172.21.15.5:0/2937360420 >> 172.21.15.5:0/2937360420 conn(0x55fbbb7f24e0 msgr2=0x55fbbbbdd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:57.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.427+0000 7f7ba23e1740 1 -- 172.21.15.5:0/2937360420 shutdown_connections 2024-05-20T18:36:57.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.427+0000 7f7ba23e1740 1 -- 172.21.15.5:0/2937360420 wait complete. 2024-05-20T18:36:57.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.427+0000 7f7ba23e1740 20 client.8838 trim_cache size 0 max 16384 2024-05-20T18:36:57.433 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:36:57.433 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-05-20T18:36:57.433 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-05-20T18:36:57.433 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:36:57.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.427+0000 7f7b9b461640 1 -- 172.21.15.5:0/2305325377 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7b8006be60 msgr2=0x7f7b8006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:57.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.427+0000 7f7b9b461640 1 --2- 172.21.15.5:0/2305325377 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7b8006be60 0x7f7b8006e370 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7b840012e0 tx=0x7f7b84005b90 comp rx=0 tx=0).stop 2024-05-20T18:36:57.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.427+0000 7f7b9b461640 1 -- 172.21.15.5:0/2305325377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b940d14f0 msgr2=0x7f7b940ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:57.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.427+0000 7f7b9b461640 1 --2- 172.21.15.5:0/2305325377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b940d14f0 0x7f7b940ccff0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00e890 tx=0x7f7b8c00ed50 comp rx=0 tx=0).stop 2024-05-20T18:36:57.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.427+0000 7f7b9b461640 1 -- 172.21.15.5:0/2305325377 shutdown_connections 2024-05-20T18:36:57.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.427+0000 7f7b9b461640 1 --2- 172.21.15.5:0/2305325377 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7b8006be60 0x7f7b8006e370 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.427+0000 7f7b9b461640 1 --2- 172.21.15.5:0/2305325377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b940d14f0 0x7f7b940ccff0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.427+0000 7f7b9b461640 1 --2- 172.21.15.5:0/2305325377 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b940d10e0 0x7f7b940caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.427+0000 7f7b9b461640 1 --2- 172.21.15.5:0/2305325377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b940cf850 0x7f7b941552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.427+0000 7f7b9b461640 1 -- 172.21.15.5:0/2305325377 >> 172.21.15.5:0/2305325377 conn(0x7f7b940b44d0 msgr2=0x7f7b940b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:57.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.428+0000 7f7b9b461640 1 -- 172.21.15.5:0/2305325377 shutdown_connections 2024-05-20T18:36:57.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.428+0000 7f7b9b461640 1 -- 172.21.15.5:0/2305325377 wait complete. 2024-05-20T18:36:57.440 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-20T18:36:57.541 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-20T18:36:57.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.656+0000 7fd96d240640 1 Processor -- start 2024-05-20T18:36:57.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.656+0000 7fd96d240640 1 -- start start 2024-05-20T18:36:57.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.658+0000 7fd96d240640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd96815d530 0x7fd96815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.658+0000 7fd96d240640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd9680b1700 0x7fd9680b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.658+0000 7fd96d240640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd968070850 0x7fd968070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.658+0000 7fd96d240640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fd9680ce190 con 0x7fd968070850 2024-05-20T18:36:57.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.658+0000 7fd96d240640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd9680cd580 con 0x7fd9680b1700 2024-05-20T18:36:57.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.658+0000 7fd96d240640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd9680cd860 con 0x7fd96815d530 2024-05-20T18:36:57.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.658+0000 7fd96759e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd968070850 0x7fd968070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.658+0000 7fd96759e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd968070850 0x7fd968070c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48680/0 (socket says 172.21.15.5:48680) 2024-05-20T18:36:57.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.658+0000 7fd96759e640 1 -- 172.21.15.5:0/736377348 learned_addr learned my addr 172.21.15.5:0/736377348 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:57.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.658+0000 7fd96659c640 1 --2- 172.21.15.5:0/736377348 >> v2:172.21.15.63:3300/0 conn(0x7fd9680b1700 0x7fd9680b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.658+0000 7fd966d9d640 1 --2- 172.21.15.5:0/736377348 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd96815d530 0x7fd96815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.658+0000 7fd96759e640 1 -- 172.21.15.5:0/736377348 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd96815d530 msgr2=0x7fd96815aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:57.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.658+0000 7fd96759e640 1 --2- 172.21.15.5:0/736377348 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd96815d530 0x7fd96815aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.658+0000 7fd96759e640 1 -- 172.21.15.5:0/736377348 >> v2:172.21.15.63:3300/0 conn(0x7fd9680b1700 msgr2=0x7fd9680b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:57.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.658+0000 7fd96759e640 1 --2- 172.21.15.5:0/736377348 >> v2:172.21.15.63:3300/0 conn(0x7fd9680b1700 0x7fd9680b1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.658+0000 7fd96759e640 1 -- 172.21.15.5:0/736377348 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9680cdb40 con 0x7fd968070850 2024-05-20T18:36:57.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.658+0000 7fd96659c640 1 --2- 172.21.15.5:0/736377348 >> v2:172.21.15.63:3300/0 conn(0x7fd9680b1700 0x7fd9680b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:36:57.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.659+0000 7fd96759e640 1 --2- 172.21.15.5:0/736377348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd968070850 0x7fd968070c60 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fd96806c2f0 tx=0x7fd96002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b173a2827798846f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:57.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.659+0000 7fd965d9b640 1 -- 172.21.15.5:0/736377348 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd960036070 con 0x7fd968070850 2024-05-20T18:36:57.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.659+0000 7fd965d9b640 1 -- 172.21.15.5:0/736377348 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd960031600 con 0x7fd968070850 2024-05-20T18:36:57.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.659+0000 7fd965d9b640 1 -- 172.21.15.5:0/736377348 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9600319d0 con 0x7fd968070850 2024-05-20T18:36:57.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.660+0000 7fd96d240640 1 -- 172.21.15.5:0/736377348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd968070850 msgr2=0x7fd968070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:57.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.660+0000 7fd96d240640 1 --2- 172.21.15.5:0/736377348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd968070850 0x7fd968070c60 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fd96806c2f0 tx=0x7fd96002a6a0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.660+0000 7fd96d240640 1 -- 172.21.15.5:0/736377348 shutdown_connections 2024-05-20T18:36:57.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.660+0000 7fd96d240640 1 --2- 172.21.15.5:0/736377348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd968070850 0x7fd968070c60 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.660+0000 7fd96d240640 1 --2- 172.21.15.5:0/736377348 >> v2:172.21.15.63:3300/0 conn(0x7fd9680b1700 0x7fd9680b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.660+0000 7fd96d240640 1 --2- 172.21.15.5:0/736377348 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd96815d530 0x7fd96815aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.660+0000 7fd96d240640 1 -- 172.21.15.5:0/736377348 >> 172.21.15.5:0/736377348 conn(0x7fd9680d29d0 msgr2=0x7fd9680d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:57.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.660+0000 7fd96d240640 1 -- 172.21.15.5:0/736377348 shutdown_connections 2024-05-20T18:36:57.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.660+0000 7fd96d240640 1 -- 172.21.15.5:0/736377348 wait complete. 2024-05-20T18:36:57.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.662+0000 7fd96d240640 1 Processor -- start 2024-05-20T18:36:57.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.662+0000 7fd96d240640 1 -- start start 2024-05-20T18:36:57.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.663+0000 7fd96d240640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd968070850 0x7fd968157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.663+0000 7fd96d240640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9680b1700 0x7fd9681f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.663+0000 7fd96d240640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd96815d530 0x7fd9680d1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.663+0000 7fd96d240640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd9680b2050 con 0x7fd968070850 2024-05-20T18:36:57.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.663+0000 7fd96d240640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd9680cd230 con 0x7fd96815d530 2024-05-20T18:36:57.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.663+0000 7fd96d240640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd9680711a0 con 0x7fd9680b1700 2024-05-20T18:36:57.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.663+0000 7fd966d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd968070850 0x7fd968157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.664+0000 7fd966d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd968070850 0x7fd968157f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48696/0 (socket says 172.21.15.5:48696) 2024-05-20T18:36:57.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.664+0000 7fd966d9d640 1 -- 172.21.15.5:0/1153324972 learned_addr learned my addr 172.21.15.5:0/1153324972 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:57.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.664+0000 7fd96759e640 1 --2- 172.21.15.5:0/1153324972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd96815d530 0x7fd9680d1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.664+0000 7fd96659c640 1 --2- 172.21.15.5:0/1153324972 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9680b1700 0x7fd9681f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.664+0000 7fd966d9d640 1 -- 172.21.15.5:0/1153324972 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9680b1700 msgr2=0x7fd9681f5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:57.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.664+0000 7fd966d9d640 1 --2- 172.21.15.5:0/1153324972 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9680b1700 0x7fd9681f5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.664+0000 7fd966d9d640 1 -- 172.21.15.5:0/1153324972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd96815d530 msgr2=0x7fd9680d1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:57.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.664+0000 7fd966d9d640 1 --2- 172.21.15.5:0/1153324972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd96815d530 0x7fd9680d1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.664+0000 7fd966d9d640 1 -- 172.21.15.5:0/1153324972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9680b7280 con 0x7fd968070850 2024-05-20T18:36:57.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.664+0000 7fd96659c640 1 --2- 172.21.15.5:0/1153324972 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9680b1700 0x7fd9681f5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:36:57.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.664+0000 7fd966d9d640 1 --2- 172.21.15.5:0/1153324972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd968070850 0x7fd968157f50 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fd95800c7f0 tx=0x7fd95800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:57.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.664+0000 7fd94f7fe640 1 -- 172.21.15.5:0/1153324972 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd958004230 con 0x7fd968070850 2024-05-20T18:36:57.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.664+0000 7fd94f7fe640 1 -- 172.21.15.5:0/1153324972 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9580043d0 con 0x7fd968070850 2024-05-20T18:36:57.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.664+0000 7fd94f7fe640 1 -- 172.21.15.5:0/1153324972 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd958007860 con 0x7fd968070850 2024-05-20T18:36:57.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.664+0000 7fd96d240640 1 -- 172.21.15.5:0/1153324972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9680b7560 con 0x7fd968070850 2024-05-20T18:36:57.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.664+0000 7fd96d240640 1 -- 172.21.15.5:0/1153324972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9680b7b20 con 0x7fd968070850 2024-05-20T18:36:57.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.665+0000 7fd96d240640 1 -- 172.21.15.5:0/1153324972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd968075b00 con 0x7fd968070850 2024-05-20T18:36:57.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.666+0000 7fd94f7fe640 1 -- 172.21.15.5:0/1153324972 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd958007c10 con 0x7fd968070850 2024-05-20T18:36:57.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.667+0000 7fd94f7fe640 1 --2- 172.21.15.5:0/1153324972 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd95406be30 0x7fd95406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.667+0000 7fd94f7fe640 1 -- 172.21.15.5:0/1153324972 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6245+0+0 (secure 0 0 0) 0x7fd95808b280 con 0x7fd968070850 2024-05-20T18:36:57.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.667+0000 7fd96659c640 1 --2- 172.21.15.5:0/1153324972 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd95406be30 0x7fd95406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.667+0000 7fd96659c640 1 --2- 172.21.15.5:0/1153324972 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd95406be30 0x7fd95406e340 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd96815ddc0 tx=0x7fd95c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:57.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.671+0000 7fd94f7fe640 1 -- 172.21.15.5:0/1153324972 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd958015020 con 0x7fd968070850 2024-05-20T18:36:57.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.895+0000 7fd96d240640 1 -- 172.21.15.5:0/1153324972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd9680b3cd0 con 0x7fd968070850 2024-05-20T18:36:57.897 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-20T18:36:57.896+0000 7fd1cf53a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-20T18:36:57.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.897+0000 7fd94f7fe640 1 -- 172.21.15.5:0/1153324972 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v89) ==== 161+0+0 (secure 0 0 0) 0x7fd958059a10 con 0x7fd968070850 2024-05-20T18:36:57.899 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-20T18:36:57.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.900+0000 7fd96d240640 1 -- 172.21.15.5:0/1153324972 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd95406be30 msgr2=0x7fd95406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:57.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.900+0000 7fd96d240640 1 --2- 172.21.15.5:0/1153324972 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd95406be30 0x7fd95406e340 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd96815ddc0 tx=0x7fd95c008040 comp rx=0 tx=0).stop 2024-05-20T18:36:57.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.900+0000 7fd96d240640 1 -- 172.21.15.5:0/1153324972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd968070850 msgr2=0x7fd968157f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:57.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.900+0000 7fd96d240640 1 --2- 172.21.15.5:0/1153324972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd968070850 0x7fd968157f50 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fd95800c7f0 tx=0x7fd95800ccb0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.901+0000 7fd96d240640 1 -- 172.21.15.5:0/1153324972 shutdown_connections 2024-05-20T18:36:57.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.901+0000 7fd96d240640 1 --2- 172.21.15.5:0/1153324972 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd95406be30 0x7fd95406e340 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.901+0000 7fd96d240640 1 --2- 172.21.15.5:0/1153324972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd96815d530 0x7fd9680d1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.901+0000 7fd96d240640 1 --2- 172.21.15.5:0/1153324972 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9680b1700 0x7fd9681f5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.901+0000 7fd96d240640 1 --2- 172.21.15.5:0/1153324972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd968070850 0x7fd968157f50 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.901+0000 7fd96d240640 1 -- 172.21.15.5:0/1153324972 >> 172.21.15.5:0/1153324972 conn(0x7fd9680d29d0 msgr2=0x7fd96815f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:57.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.901+0000 7fd96d240640 1 -- 172.21.15.5:0/1153324972 shutdown_connections 2024-05-20T18:36:57.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.901+0000 7fd96d240640 1 -- 172.21.15.5:0/1153324972 wait complete. 2024-05-20T18:36:57.905 INFO:tasks.ceph.mds.c.smithi005.stderr: -1154> 2024-05-20T18:36:27.058+0000 7f1d34683640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-05-20T18:36:57.905 INFO:tasks.ceph.mds.c.smithi005.stderr: -1151> 2024-05-20T18:36:27.058+0000 7f1d34683640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-05-20T18:36:57.905 INFO:tasks.ceph.mds.c.smithi005.stderr: -1134> 2024-05-20T18:36:27.058+0000 7f1d34683640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-05-20T18:36:57.905 INFO:tasks.ceph.mds.c.smithi005.stderr: -1131> 2024-05-20T18:36:27.058+0000 7f1d34683640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-05-20T18:36:57.905 INFO:tasks.ceph.mds.c.smithi005.stderr: -979> 2024-05-20T18:36:27.385+0000 7f1d3868b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T18:36:57.915 DEBUG:teuthology.orchestra.run.smithi005:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2024-05-20T18:36:57.958 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-05-20T18:36:57.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.967+0000 7faeb6a2c280 1 -- start start 2024-05-20T18:36:57.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.968+0000 7faeb6a2c280 1 Processor -- start 2024-05-20T18:36:57.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.969+0000 7faeb6a2c280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56427a0afa80 0x56427a080580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.969+0000 7faeb6a2c280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56427a0af180 0x56427a080b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.969+0000 7faeb6a2c280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56427adb0000 0x56427a081080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.969+0000 7faeb6a2c280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56427acd2600 con 0x56427a0afa80 2024-05-20T18:36:57.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.969+0000 7faeb6a2c280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56427acd2480 con 0x56427a0af180 2024-05-20T18:36:57.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.969+0000 7faeb6a2c280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x56427acd2300 con 0x56427adb0000 2024-05-20T18:36:57.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.969+0000 7faeb4228640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56427a0afa80 0x56427a080580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.969+0000 7faeb4228640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56427a0afa80 0x56427a080580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48740/0 (socket says 172.21.15.5:48740) 2024-05-20T18:36:57.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.969+0000 7faeb4228640 1 -- 172.21.15.5:0/1238418368 learned_addr learned my addr 172.21.15.5:0/1238418368 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:57.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.969+0000 7faeb3a27640 1 --2- 172.21.15.5:0/1238418368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56427a0af180 0x56427a080b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.969+0000 7faeb4a29640 1 --2- 172.21.15.5:0/1238418368 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56427adb0000 0x56427a081080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.969+0000 7faeb4228640 1 -- 172.21.15.5:0/1238418368 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56427adb0000 msgr2=0x56427a081080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:57.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.969+0000 7faeb4228640 1 --2- 172.21.15.5:0/1238418368 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56427adb0000 0x56427a081080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.969+0000 7faeb4228640 1 -- 172.21.15.5:0/1238418368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56427a0af180 msgr2=0x56427a080b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:57.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.969+0000 7faeb4228640 1 --2- 172.21.15.5:0/1238418368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56427a0af180 0x56427a080b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.969+0000 7faeb4228640 1 -- 172.21.15.5:0/1238418368 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564275efc780 con 0x56427a0afa80 2024-05-20T18:36:57.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.969+0000 7faeb4a29640 1 --2- 172.21.15.5:0/1238418368 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56427adb0000 0x56427a081080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:36:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.969+0000 7faeb4228640 1 --2- 172.21.15.5:0/1238418368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56427a0afa80 0x56427a080580 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x56427ad172c0 tx=0x564279fea2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=384811d7a5f76d78 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.970+0000 7faeb1a23640 1 -- 172.21.15.5:0/1238418368 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564275eec680 con 0x56427a0afa80 2024-05-20T18:36:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.970+0000 7faeb1a23640 1 -- 172.21.15.5:0/1238418368 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x564275ef2a80 con 0x56427a0afa80 2024-05-20T18:36:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.970+0000 7faeb1a23640 1 -- 172.21.15.5:0/1238418368 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564275eecb60 con 0x56427a0afa80 2024-05-20T18:36:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.970+0000 7faeb6a2c280 1 -- 172.21.15.5:0/1238418368 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564275efc3c0 con 0x56427a0afa80 2024-05-20T18:36:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.970+0000 7faeb6a2c280 1 -- 172.21.15.5:0/1238418368 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=0}) -- 0x564275efd680 con 0x56427a0afa80 2024-05-20T18:36:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.970+0000 7faeb6a2c280 4 waiting for MDS map... 2024-05-20T18:36:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.970+0000 7faeb1a23640 1 -- 172.21.15.5:0/1238418368 <== mon.0 v2:172.21.15.5:3300/0 4 ==== osd_map(109..109 src has 1..109) ==== 6331+0+0 (secure 0 0 0) 0x56427adce600 con 0x56427a0afa80 2024-05-20T18:36:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.970+0000 7faeb1a23640 1 -- 172.21.15.5:0/1238418368 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 89) ==== 2885+0+0 (secure 0 0 0) 0x564279f51180 con 0x56427a0afa80 2024-05-20T18:36:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.970+0000 7faeb6a2c280 4 Got MDS map 89 2024-05-20T18:36:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.970+0000 7faeb6a2c280 10 main: main 2024-05-20T18:36:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.970+0000 7faeb6a2c280 4 main: connecting to RADOS... 2024-05-20T18:36:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.971+0000 7faeb6a2c280 1 Processor -- start 2024-05-20T18:36:57.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.971+0000 7faeb6a2c280 1 -- start start 2024-05-20T18:36:57.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.972+0000 7faeb6a2c280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56427adb0900 0x56427a081600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.972+0000 7faeb6a2c280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56427adb0d80 0x56427a081b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.972+0000 7faeb6a2c280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56427adb1200 0x56427a082100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.972+0000 7faeb6a2c280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56427acd2180 con 0x56427adb0d80 2024-05-20T18:36:57.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.972+0000 7faeb6a2c280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564279f58480 con 0x56427adb0900 2024-05-20T18:36:57.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.972+0000 7faeb6a2c280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564279f58900 con 0x56427adb1200 2024-05-20T18:36:57.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.972+0000 7faeb4228640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56427adb0d80 0x56427a081b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.972+0000 7faeb4228640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56427adb0d80 0x56427a081b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48756/0 (socket says 172.21.15.5:48756) 2024-05-20T18:36:57.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.972+0000 7faeb4228640 1 -- 172.21.15.5:0/2181866933 learned_addr learned my addr 172.21.15.5:0/2181866933 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:57.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.972+0000 7faeb3a27640 1 --2- 172.21.15.5:0/2181866933 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56427adb1200 0x56427a082100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.972+0000 7faeb4a29640 1 --2- 172.21.15.5:0/2181866933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56427adb0900 0x56427a081600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.973+0000 7faeb4228640 1 -- 172.21.15.5:0/2181866933 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56427adb1200 msgr2=0x56427a082100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:57.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.973+0000 7faeb4228640 1 --2- 172.21.15.5:0/2181866933 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56427adb1200 0x56427a082100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.973+0000 7faeb4228640 1 -- 172.21.15.5:0/2181866933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56427adb0900 msgr2=0x56427a081600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:57.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.973+0000 7faeb4228640 1 --2- 172.21.15.5:0/2181866933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56427adb0900 0x56427a081600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.973+0000 7faeb4228640 1 -- 172.21.15.5:0/2181866933 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564275efd4a0 con 0x56427adb0d80 2024-05-20T18:36:57.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.973+0000 7faeb3a27640 1 --2- 172.21.15.5:0/2181866933 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56427adb1200 0x56427a082100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:36:57.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.973+0000 7faeb4228640 1 --2- 172.21.15.5:0/2181866933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56427adb0d80 0x56427a081b80 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x56427ae26690 tx=0x56427a075080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae98302a58a7311 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:57.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.973+0000 7faeafa1f640 1 -- 172.21.15.5:0/2181866933 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564275eec820 con 0x56427adb0d80 2024-05-20T18:36:57.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.973+0000 7faeafa1f640 1 -- 172.21.15.5:0/2181866933 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x564275ef2e00 con 0x56427adb0d80 2024-05-20T18:36:57.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.973+0000 7faeafa1f640 1 -- 172.21.15.5:0/2181866933 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564275eec680 con 0x56427adb0d80 2024-05-20T18:36:57.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.973+0000 7faeb6a2c280 1 -- 172.21.15.5:0/2181866933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56427adb0d80 msgr2=0x56427a081b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:57.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.973+0000 7faeb6a2c280 1 --2- 172.21.15.5:0/2181866933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56427adb0d80 0x56427a081b80 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x56427ae26690 tx=0x56427a075080 comp rx=0 tx=0).stop 2024-05-20T18:36:57.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.973+0000 7faeb6a2c280 1 -- 172.21.15.5:0/2181866933 shutdown_connections 2024-05-20T18:36:57.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.973+0000 7faeb6a2c280 1 --2- 172.21.15.5:0/2181866933 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56427adb1200 0x56427a082100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.973+0000 7faeb6a2c280 1 --2- 172.21.15.5:0/2181866933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56427adb0d80 0x56427a081b80 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.973+0000 7faeb6a2c280 1 --2- 172.21.15.5:0/2181866933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56427adb0900 0x56427a081600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.973+0000 7faeb6a2c280 1 -- 172.21.15.5:0/2181866933 >> 172.21.15.5:0/2181866933 conn(0x56427adb0480 msgr2=0x56427a00a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:57.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.974+0000 7faeb6a2c280 1 -- 172.21.15.5:0/2181866933 shutdown_connections 2024-05-20T18:36:57.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.974+0000 7faeb6a2c280 1 -- 172.21.15.5:0/2181866933 wait complete. 2024-05-20T18:36:57.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.974+0000 7faeb6a2c280 1 Processor -- start 2024-05-20T18:36:57.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.975+0000 7faeb6a2c280 1 -- start start 2024-05-20T18:36:57.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.976+0000 7faeb6a2c280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56427adb0900 0x56427a083180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.976+0000 7faeb6a2c280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56427adb0d80 0x56427a082c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.976+0000 7faeb6a2c280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56427adb1200 0x56427a082680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.976+0000 7faeb6a2c280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56427acd3380 con 0x56427adb0d80 2024-05-20T18:36:57.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.976+0000 7faeb6a2c280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56427acd3500 con 0x56427adb1200 2024-05-20T18:36:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.976+0000 7faeb6a2c280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x56427acd2180 con 0x56427adb0900 2024-05-20T18:36:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.976+0000 7faeb4228640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56427adb0d80 0x56427a082c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.976+0000 7faeb4228640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56427adb0d80 0x56427a082c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48772/0 (socket says 172.21.15.5:48772) 2024-05-20T18:36:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.976+0000 7faeb4228640 1 -- 172.21.15.5:0/778510159 learned_addr learned my addr 172.21.15.5:0/778510159 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.976+0000 7faeb3a27640 1 --2- 172.21.15.5:0/778510159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56427adb1200 0x56427a082680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.976+0000 7faeb4a29640 1 --2- 172.21.15.5:0/778510159 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56427adb0900 0x56427a083180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.976+0000 7faeb4228640 1 -- 172.21.15.5:0/778510159 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56427adb0900 msgr2=0x56427a083180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.976+0000 7faeb4228640 1 --2- 172.21.15.5:0/778510159 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56427adb0900 0x56427a083180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.976+0000 7faeb4228640 1 -- 172.21.15.5:0/778510159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56427adb1200 msgr2=0x56427a082680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.976+0000 7faeb4228640 1 --2- 172.21.15.5:0/778510159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56427adb1200 0x56427a082680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.976+0000 7faeb4228640 1 -- 172.21.15.5:0/778510159 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564275efc780 con 0x56427adb0d80 2024-05-20T18:36:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.976+0000 7faeb4228640 1 --2- 172.21.15.5:0/778510159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56427adb0d80 0x56427a082c00 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x56427ae26900 tx=0x56427ade7800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.976+0000 7faeae21c640 1 -- 172.21.15.5:0/778510159 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564275eecb60 con 0x56427adb0d80 2024-05-20T18:36:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.977+0000 7faeae21c640 1 -- 172.21.15.5:0/778510159 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x564275ef2e00 con 0x56427adb0d80 2024-05-20T18:36:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.977+0000 7faeb6a2c280 1 -- 172.21.15.5:0/778510159 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564275efc3c0 con 0x56427adb0d80 2024-05-20T18:36:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.977+0000 7faeae21c640 1 -- 172.21.15.5:0/778510159 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564275eec4e0 con 0x56427adb0d80 2024-05-20T18:36:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.977+0000 7faeb6a2c280 1 -- 172.21.15.5:0/778510159 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564275efd680 con 0x56427adb0d80 2024-05-20T18:36:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.977+0000 7faeb6a2c280 4 main: resolving pool 15 2024-05-20T18:36:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.978+0000 7faeae21c640 1 -- 172.21.15.5:0/778510159 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x56427adc8c00 con 0x56427adb0d80 2024-05-20T18:36:57.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.978+0000 7faeae21c640 1 --2- 172.21.15.5:0/778510159 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x56427adb3600 0x56427a083700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.978+0000 7faeae21c640 1 -- 172.21.15.5:0/778510159 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6331+0+0 (secure 0 0 0) 0x56427ae4c600 con 0x56427adb0d80 2024-05-20T18:36:57.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.978+0000 7faeb6a2c280 4 main: creating IoCtx.. 2024-05-20T18:36:57.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.978+0000 7faeb4a29640 1 --2- 172.21.15.5:0/778510159 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x56427adb3600 0x56427a083700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.978+0000 7faeb6a2c280 1 --2- 172.21.15.5:0/778510159 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x56427adb1680 0x56427a082100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:57.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.978+0000 7faeb6a2c280 1 -- 172.21.15.5:0/778510159 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109) -- 0x56427a2de000 con 0x56427adb1680 2024-05-20T18:36:57.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.979+0000 7faeb3a27640 1 --2- 172.21.15.5:0/778510159 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x56427adb1680 0x56427a082100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:57.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.979+0000 7faeb4a29640 1 --2- 172.21.15.5:0/778510159 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x56427adb3600 0x56427a083700 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x56427ad17260 tx=0x56427a268ae0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:57.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.979+0000 7faeb3a27640 1 --2- 172.21.15.5:0/778510159 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x56427adb1680 0x56427a082100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:57.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.981+0000 7faeb3a27640 1 -- 172.21.15.5:0/778510159 <== osd.1 v2:172.21.15.5:6815/550664725 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x56427a06c000 con 0x56427adb1680 2024-05-20T18:36:57.985 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:36:57.985 INFO:teuthology.orchestra.run.smithi005.stdout: "0": { 2024-05-20T18:36:57.985 INFO:teuthology.orchestra.run.smithi005.stdout: "data": { 2024-05-20T18:36:57.985 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 5, 2024-05-20T18:36:57.986 INFO:teuthology.orchestra.run.smithi005.stdout: "inotable": { 2024-05-20T18:36:57.986 INFO:teuthology.orchestra.run.smithi005.stdout: "projected_free": [ 2024-05-20T18:36:57.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:36:57.986 INFO:teuthology.orchestra.run.smithi005.stdout: "start": 1099511627779, 2024-05-20T18:36:57.986 INFO:teuthology.orchestra.run.smithi005.stdout: "len": 1099511627773 2024-05-20T18:36:57.986 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:36:57.986 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-20T18:36:57.986 INFO:teuthology.orchestra.run.smithi005.stdout: "free": [ 2024-05-20T18:36:57.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:36:57.986 INFO:teuthology.orchestra.run.smithi005.stdout: "start": 1099511627779, 2024-05-20T18:36:57.986 INFO:teuthology.orchestra.run.smithi005.stdout: "len": 1099511627773 2024-05-20T18:36:57.986 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:36:57.986 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-20T18:36:57.987 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:36:57.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:36:57.987 INFO:teuthology.orchestra.run.smithi005.stdout: "result": 0 2024-05-20T18:36:57.987 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:36:57.987 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:36:57.987 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:36:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.981+0000 7faeb6a2c280 1 -- 172.21.15.5:0/778510159 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x56427adb1680 msgr2=0x56427a082100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.981+0000 7faeb6a2c280 1 --2- 172.21.15.5:0/778510159 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x56427adb1680 0x56427a082100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.981+0000 7faeb6a2c280 1 -- 172.21.15.5:0/778510159 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x56427adb3600 msgr2=0x56427a083700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.981+0000 7faeb6a2c280 1 --2- 172.21.15.5:0/778510159 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x56427adb3600 0x56427a083700 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x56427ad17260 tx=0x56427a268ae0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.981+0000 7faeb6a2c280 1 -- 172.21.15.5:0/778510159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56427adb0d80 msgr2=0x56427a082c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.981+0000 7faeb6a2c280 1 --2- 172.21.15.5:0/778510159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56427adb0d80 0x56427a082c00 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x56427ae26900 tx=0x56427ade7800 comp rx=0 tx=0).stop 2024-05-20T18:36:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.981+0000 7faeb3a27640 1 -- 172.21.15.5:0/778510159 reap_dead start 2024-05-20T18:36:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.981+0000 7faeb6a2c280 1 -- 172.21.15.5:0/778510159 shutdown_connections 2024-05-20T18:36:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.981+0000 7faeb6a2c280 1 -- 172.21.15.5:0/778510159 >> 172.21.15.5:0/778510159 conn(0x56427adb0480 msgr2=0x56427a00a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.982+0000 7faeb6a2c280 1 -- 172.21.15.5:0/778510159 shutdown_connections 2024-05-20T18:36:57.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.982+0000 7faeb6a2c280 1 -- 172.21.15.5:0/778510159 wait complete. 2024-05-20T18:36:57.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.982+0000 7faeb6a2c280 1 -- 172.21.15.5:0/1238418368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56427a0afa80 msgr2=0x56427a080580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:57.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.982+0000 7faeb6a2c280 1 --2- 172.21.15.5:0/1238418368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56427a0afa80 0x56427a080580 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x56427ad172c0 tx=0x564279fea2a0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.982+0000 7faeb6a2c280 1 -- 172.21.15.5:0/1238418368 shutdown_connections 2024-05-20T18:36:57.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.982+0000 7faeb6a2c280 1 --2- 172.21.15.5:0/1238418368 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56427adb0000 0x56427a081080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.982+0000 7faeb6a2c280 1 --2- 172.21.15.5:0/1238418368 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56427a0af180 0x56427a080b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.982+0000 7faeb6a2c280 1 --2- 172.21.15.5:0/1238418368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56427a0afa80 0x56427a080580 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:57.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.982+0000 7faeb6a2c280 1 -- 172.21.15.5:0/1238418368 >> 172.21.15.5:0/1238418368 conn(0x56427a0af600 msgr2=0x56427a00a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:57.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.982+0000 7faeb6a2c280 1 -- 172.21.15.5:0/1238418368 shutdown_connections 2024-05-20T18:36:57.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:57.982+0000 7faeb6a2c280 1 -- 172.21.15.5:0/1238418368 wait complete. 2024-05-20T18:36:57.990 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.075585, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2024-05-20T18:36:57.991 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-20T18:36:57.991 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T18:36:58.069 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T18:36:58.112 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-20T18:36:58.112 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T18:36:58.112 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:36:58.112 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:36:58.128 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15663 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T18:36:58.128 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T18:36:58.129 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:36:58.129 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:36:58.171 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19732 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T18:36:58.171 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T18:36:58.171 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:36:58.171 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:36:58.188 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15663 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T18:36:58.188 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T18:36:58.188 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:36:58.188 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:36:58.230 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19732 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T18:36:58.230 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:36:58.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.344+0000 7f2b3ab5e640 1 Processor -- start 2024-05-20T18:36:58.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.344+0000 7f2b3ab5e640 1 -- start start 2024-05-20T18:36:58.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.345+0000 7f2b3ab5e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2b3406d3d0 0x7f2b3406d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:58.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.345+0000 7f2b3ab5e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b340b8190 0x7f2b340b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:58.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.345+0000 7f2b3ab5e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b3406dd20 con 0x7f2b34162820 2024-05-20T18:36:58.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.345+0000 7f2b3ab5e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f2b340d0630 con 0x7f2b3406d3d0 2024-05-20T18:36:58.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.345+0000 7f2b3ab5e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2b340d39c0 con 0x7f2b340b8190 2024-05-20T18:36:58.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.345+0000 7f2b33fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2b3406d3d0 0x7f2b3406d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:58.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.345+0000 7f2b388fa640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2b34162820 0x7f2b340b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51734/0 (socket says 172.21.15.5:51734) 2024-05-20T18:36:58.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.345+0000 7f2b388fa640 1 -- 172.21.15.5:0/3915708470 learned_addr learned my addr 172.21.15.5:0/3915708470 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:36:58.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.345+0000 7f2b33fff640 1 -- 172.21.15.5:0/3915708470 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b340b8190 msgr2=0x7f2b340b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:58.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.345+0000 7f2b33fff640 1 --2- 172.21.15.5:0/3915708470 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b340b8190 0x7f2b340b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:58.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.345+0000 7f2b33fff640 1 -- 172.21.15.5:0/3915708470 >> v1:172.21.15.5:6789/0 conn(0x7f2b34162820 legacy=0x7f2b340b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:58.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.345+0000 7f2b33fff640 1 -- 172.21.15.5:0/3915708470 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b340d3ca0 con 0x7f2b3406d3d0 2024-05-20T18:36:58.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.345+0000 7f2b337fe640 1 -- 172.21.15.5:0/3915708470 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1144488738 0 0) 0x7f2b24001110 con 0x7f2b34162820 2024-05-20T18:36:58.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.346+0000 7f2b33fff640 1 --2- 172.21.15.5:0/3915708470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b3406d3d0 0x7f2b3406d7e0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f2b280097a0 tx=0x7f2b2802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87173b4df01264a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:58.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.346+0000 7f2b337fe640 1 -- 172.21.15.5:0/3915708470 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b28037070 con 0x7f2b3406d3d0 2024-05-20T18:36:58.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.346+0000 7f2b337fe640 1 -- 172.21.15.5:0/3915708470 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b28032440 con 0x7f2b3406d3d0 2024-05-20T18:36:58.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.346+0000 7f2b337fe640 1 -- 172.21.15.5:0/3915708470 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b28032810 con 0x7f2b3406d3d0 2024-05-20T18:36:58.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.347+0000 7f2b3ab5e640 1 -- 172.21.15.5:0/3915708470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b3406d3d0 msgr2=0x7f2b3406d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:58.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.347+0000 7f2b3ab5e640 1 --2- 172.21.15.5:0/3915708470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b3406d3d0 0x7f2b3406d7e0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f2b280097a0 tx=0x7f2b2802a1a0 comp rx=0 tx=0).stop 2024-05-20T18:36:58.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.347+0000 7f2b3ab5e640 1 -- 172.21.15.5:0/3915708470 shutdown_connections 2024-05-20T18:36:58.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.347+0000 7f2b3ab5e640 1 --2- 172.21.15.5:0/3915708470 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b340b8190 0x7f2b340b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:58.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.347+0000 7f2b3ab5e640 1 --2- 172.21.15.5:0/3915708470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b3406d3d0 0x7f2b3406d7e0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:58.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.347+0000 7f2b3ab5e640 1 -- 172.21.15.5:0/3915708470 >> 172.21.15.5:0/3915708470 conn(0x7f2b340d4bd0 msgr2=0x7f2b340d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:58.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.347+0000 7f2b3ab5e640 1 -- 172.21.15.5:0/3915708470 shutdown_connections 2024-05-20T18:36:58.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.347+0000 7f2b3ab5e640 1 -- 172.21.15.5:0/3915708470 wait complete. 2024-05-20T18:36:58.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.349+0000 7f2b3ab5e640 1 Processor -- start 2024-05-20T18:36:58.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.349+0000 7f2b3ab5e640 1 -- start start 2024-05-20T18:36:58.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.350+0000 7f2b3ab5e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b3406d3d0 0x7f2b340cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:58.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.350+0000 7f2b3ab5e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b340b8190 0x7f2b34154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:58.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.350+0000 7f2b3ab5e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b34162820 0x7f2b340c9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:58.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.350+0000 7f2b3ab5e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2b340d0300 con 0x7f2b3406d3d0 2024-05-20T18:36:58.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.350+0000 7f2b3ab5e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2b340d4260 con 0x7f2b340b8190 2024-05-20T18:36:58.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.350+0000 7f2b3ab5e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2b340b4050 con 0x7f2b34162820 2024-05-20T18:36:58.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.351+0000 7f2b388fa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b3406d3d0 0x7f2b340cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:58.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.351+0000 7f2b388fa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b3406d3d0 0x7f2b340cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48786/0 (socket says 172.21.15.5:48786) 2024-05-20T18:36:58.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.351+0000 7f2b388fa640 1 -- 172.21.15.5:0/1636577474 learned_addr learned my addr 172.21.15.5:0/1636577474 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:58.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.351+0000 7f2b390fb640 1 --2- 172.21.15.5:0/1636577474 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b34162820 0x7f2b340c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:58.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.351+0000 7f2b33fff640 1 --2- 172.21.15.5:0/1636577474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b340b8190 0x7f2b34154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:58.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.351+0000 7f2b388fa640 1 -- 172.21.15.5:0/1636577474 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b34162820 msgr2=0x7f2b340c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:58.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.351+0000 7f2b388fa640 1 --2- 172.21.15.5:0/1636577474 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b34162820 0x7f2b340c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:58.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.351+0000 7f2b388fa640 1 -- 172.21.15.5:0/1636577474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b340b8190 msgr2=0x7f2b34154b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:58.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.351+0000 7f2b388fa640 1 --2- 172.21.15.5:0/1636577474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b340b8190 0x7f2b34154b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:58.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.351+0000 7f2b388fa640 1 -- 172.21.15.5:0/1636577474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b340ce780 con 0x7f2b3406d3d0 2024-05-20T18:36:58.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.351+0000 7f2b390fb640 1 --2- 172.21.15.5:0/1636577474 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b34162820 0x7f2b340c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:36:58.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.351+0000 7f2b388fa640 1 --2- 172.21.15.5:0/1636577474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b3406d3d0 0x7f2b340cc240 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400c460 tx=0x7f2b2400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:58.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.351+0000 7f2b317fa640 1 -- 172.21.15.5:0/1636577474 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b24018070 con 0x7f2b3406d3d0 2024-05-20T18:36:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.351+0000 7f2b317fa640 1 -- 172.21.15.5:0/1636577474 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b24007500 con 0x7f2b3406d3d0 2024-05-20T18:36:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.351+0000 7f2b3ab5e640 1 -- 172.21.15.5:0/1636577474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b340c87a0 con 0x7f2b3406d3d0 2024-05-20T18:36:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.351+0000 7f2b317fa640 1 -- 172.21.15.5:0/1636577474 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b24004a20 con 0x7f2b3406d3d0 2024-05-20T18:36:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.351+0000 7f2b3ab5e640 1 -- 172.21.15.5:0/1636577474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b340c8d60 con 0x7f2b3406d3d0 2024-05-20T18:36:58.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.353+0000 7f2b3ab5e640 1 -- 172.21.15.5:0/1636577474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b34075b00 con 0x7f2b3406d3d0 2024-05-20T18:36:58.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.353+0000 7f2b317fa640 1 -- 172.21.15.5:0/1636577474 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2b2401b070 con 0x7f2b3406d3d0 2024-05-20T18:36:58.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.353+0000 7f2b317fa640 1 --2- 172.21.15.5:0/1636577474 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2b2006be30 0x7f2b2006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:58.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.354+0000 7f2b317fa640 1 -- 172.21.15.5:0/1636577474 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6331+0+0 (secure 0 0 0) 0x7f2b2408a3e0 con 0x7f2b3406d3d0 2024-05-20T18:36:58.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.354+0000 7f2b33fff640 1 --2- 172.21.15.5:0/1636577474 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2b2006be30 0x7f2b2006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:58.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.354+0000 7f2b33fff640 1 --2- 172.21.15.5:0/1636577474 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2b2006be30 0x7f2b2006e340 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f2b28034000 tx=0x7f2b2802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:58.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.358+0000 7f2b317fa640 1 -- 172.21.15.5:0/1636577474 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2b3406dd20 con 0x7f2b3406d3d0 2024-05-20T18:36:58.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.583+0000 7f2b3ab5e640 1 -- 172.21.15.5:0/1636577474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b3415e970 con 0x7f2b3406d3d0 2024-05-20T18:36:58.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.583+0000 7f2b317fa640 1 -- 172.21.15.5:0/1636577474 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+4342 (secure 0 0 0) 0x7f2b24058a90 con 0x7f2b3406d3d0 2024-05-20T18:36:58.585 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:36:58.585 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":89,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6758,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/4163145158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":4163145158},{"type":"v1","addr":"172.21.15.63:6834","nonce":4163145158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7528,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3049443532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3049443532},{"type":"v1","addr":"172.21.15.5:6837","nonce":3049443532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7828,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/4180308082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":4180308082},{"type":"v1","addr":"172.21.15.63:6835","nonce":4180308082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":89,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:35:30.778930+0000","modified":"2024-05-20T18:36:57.897515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-20T18:36:58.585 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 89 2024-05-20T18:36:58.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.585+0000 7f2b3ab5e640 1 -- 172.21.15.5:0/1636577474 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2b2006be30 msgr2=0x7f2b2006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:58.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.585+0000 7f2b3ab5e640 1 --2- 172.21.15.5:0/1636577474 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2b2006be30 0x7f2b2006e340 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f2b28034000 tx=0x7f2b2802e000 comp rx=0 tx=0).stop 2024-05-20T18:36:58.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.585+0000 7f2b3ab5e640 1 -- 172.21.15.5:0/1636577474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b3406d3d0 msgr2=0x7f2b340cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:58.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.585+0000 7f2b3ab5e640 1 --2- 172.21.15.5:0/1636577474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b3406d3d0 0x7f2b340cc240 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400c460 tx=0x7f2b2400c920 comp rx=0 tx=0).stop 2024-05-20T18:36:58.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.585+0000 7f2b3ab5e640 1 -- 172.21.15.5:0/1636577474 shutdown_connections 2024-05-20T18:36:58.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.585+0000 7f2b3ab5e640 1 --2- 172.21.15.5:0/1636577474 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2b2006be30 0x7f2b2006e340 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:58.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.585+0000 7f2b3ab5e640 1 --2- 172.21.15.5:0/1636577474 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b34162820 0x7f2b340c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:58.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.585+0000 7f2b3ab5e640 1 --2- 172.21.15.5:0/1636577474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b340b8190 0x7f2b34154b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:58.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.585+0000 7f2b3ab5e640 1 --2- 172.21.15.5:0/1636577474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b3406d3d0 0x7f2b340cc240 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:58.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.585+0000 7f2b3ab5e640 1 -- 172.21.15.5:0/1636577474 >> 172.21.15.5:0/1636577474 conn(0x7f2b340d4bd0 msgr2=0x7f2b340d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:58.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.586+0000 7f2b3ab5e640 1 -- 172.21.15.5:0/1636577474 shutdown_connections 2024-05-20T18:36:58.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.586+0000 7f2b3ab5e640 1 -- 172.21.15.5:0/1636577474 wait complete. 2024-05-20T18:36:58.598 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:36:58.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.712+0000 7f654bb8f640 1 Processor -- start 2024-05-20T18:36:58.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.712+0000 7f654bb8f640 1 -- start start 2024-05-20T18:36:58.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.713+0000 7f654bb8f640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6544162820 0x7f65440b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:58.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.713+0000 7f654bb8f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f654406dae0 0x7f65440b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:58.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.713+0000 7f654bb8f640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65440d0630 con 0x7f65440b3e80 2024-05-20T18:36:58.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.713+0000 7f654bb8f640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f65440d39c0 con 0x7f6544162820 2024-05-20T18:36:58.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.713+0000 7f654bb8f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f65440d3ca0 con 0x7f654406dae0 2024-05-20T18:36:58.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.714+0000 7f654912a640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f65440b3e80 0x7f654406d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51742/0 (socket says 172.21.15.5:51742) 2024-05-20T18:36:58.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.714+0000 7f654912a640 1 -- 172.21.15.5:0/379996040 learned_addr learned my addr 172.21.15.5:0/379996040 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:36:58.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.714+0000 7f6548929640 1 -- 172.21.15.5:0/379996040 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2955639081 0 0) 0x7f65440d0630 con 0x7f65440b3e80 2024-05-20T18:36:58.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.714+0000 7f6548929640 1 -- 172.21.15.5:0/379996040 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f652c003520 con 0x7f65440b3e80 2024-05-20T18:36:58.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.714+0000 7f654a12c640 1 --2- 172.21.15.5:0/379996040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f654406dae0 0x7f65440b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:58.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.714+0000 7f654992b640 1 --2- 172.21.15.5:0/379996040 >> v2:172.21.15.63:3300/0 conn(0x7f6544162820 0x7f65440b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.714+0000 7f6548929640 1 -- 172.21.15.5:0/379996040 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 300590814 0 0) 0x7f652c003520 con 0x7f65440b3e80 2024-05-20T18:36:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.714+0000 7f6548929640 1 -- 172.21.15.5:0/379996040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f654406dae0 msgr2=0x7f65440b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.714+0000 7f6548929640 1 --2- 172.21.15.5:0/379996040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f654406dae0 0x7f65440b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.714+0000 7f6548929640 1 -- 172.21.15.5:0/379996040 >> v2:172.21.15.63:3300/0 conn(0x7f6544162820 msgr2=0x7f65440b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.714+0000 7f6548929640 1 --2- 172.21.15.5:0/379996040 >> v2:172.21.15.63:3300/0 conn(0x7f6544162820 0x7f65440b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.714+0000 7f6548929640 1 -- 172.21.15.5:0/379996040 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65440d3f80 con 0x7f65440b3e80 2024-05-20T18:36:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.714+0000 7f6548929640 1 -- 172.21.15.5:0/379996040 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f6538001670 con 0x7f65440b3e80 2024-05-20T18:36:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.714+0000 7f6548929640 1 -- 172.21.15.5:0/379996040 >> v1:172.21.15.5:6789/0 conn(0x7f65440b3e80 legacy=0x7f654406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.714+0000 7f6548929640 1 --2- 172.21.15.5:0/379996040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f652c0039b0 0x7f652c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.714+0000 7f6548929640 1 --2- 172.21.15.5:0/379996040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f652c004800 0x7f652c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.714+0000 7f6548929640 1 --2- 172.21.15.5:0/379996040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f652c005150 0x7f652c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.714+0000 7f6548929640 1 -- 172.21.15.5:0/379996040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f65440d39c0 con 0x7f652c0039b0 2024-05-20T18:36:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.714+0000 7f6548929640 1 -- 172.21.15.5:0/379996040 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f65440d3ca0 con 0x7f652c005150 2024-05-20T18:36:58.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.714+0000 7f6548929640 1 -- 172.21.15.5:0/379996040 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f652c00bf00 con 0x7f652c004800 2024-05-20T18:36:58.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.714+0000 7f654992b640 1 --2- 172.21.15.5:0/379996040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f652c0039b0 0x7f652c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:58.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.715+0000 7f654a12c640 1 --2- 172.21.15.5:0/379996040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f652c005150 0x7f652c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:58.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.715+0000 7f654912a640 1 --2- 172.21.15.5:0/379996040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f652c004800 0x7f652c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:58.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.715+0000 7f654992b640 1 -- 172.21.15.5:0/379996040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f652c004800 msgr2=0x7f652c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:58.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.715+0000 7f654992b640 1 --2- 172.21.15.5:0/379996040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f652c004800 0x7f652c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:58.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.715+0000 7f654992b640 1 -- 172.21.15.5:0/379996040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f652c005150 msgr2=0x7f652c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:58.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.715+0000 7f654992b640 1 --2- 172.21.15.5:0/379996040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f652c005150 0x7f652c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:58.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.715+0000 7f654992b640 1 -- 172.21.15.5:0/379996040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652c003520 con 0x7f652c0039b0 2024-05-20T18:36:58.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.716+0000 7f654992b640 1 --2- 172.21.15.5:0/379996040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f652c0039b0 0x7f652c003e20 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f653c009fc0 tx=0x7f652c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bec0d10fcef7fc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:58.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.716+0000 7f6548929640 1 -- 172.21.15.5:0/379996040 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c009c10 con 0x7f652c0039b0 2024-05-20T18:36:58.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.716+0000 7f6548929640 1 -- 172.21.15.5:0/379996040 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f653c036070 con 0x7f652c0039b0 2024-05-20T18:36:58.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.716+0000 7f6548929640 1 -- 172.21.15.5:0/379996040 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c02d070 con 0x7f652c0039b0 2024-05-20T18:36:58.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.716+0000 7f654bb8f640 1 -- 172.21.15.5:0/379996040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f652c0039b0 msgr2=0x7f652c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:58.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.716+0000 7f654bb8f640 1 --2- 172.21.15.5:0/379996040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f652c0039b0 0x7f652c003e20 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f653c009fc0 tx=0x7f652c005830 comp rx=0 tx=0).stop 2024-05-20T18:36:58.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.716+0000 7f654a12c640 1 -- 172.21.15.5:0/379996040 reap_dead start 2024-05-20T18:36:58.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.716+0000 7f654bb8f640 1 -- 172.21.15.5:0/379996040 shutdown_connections 2024-05-20T18:36:58.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.716+0000 7f654bb8f640 1 -- 172.21.15.5:0/379996040 >> 172.21.15.5:0/379996040 conn(0x7f65440d4bd0 msgr2=0x7f65440d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:58.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.716+0000 7f654bb8f640 1 -- 172.21.15.5:0/379996040 shutdown_connections 2024-05-20T18:36:58.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.716+0000 7f654bb8f640 1 -- 172.21.15.5:0/379996040 wait complete. 2024-05-20T18:36:58.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.718+0000 7f654bb8f640 1 Processor -- start 2024-05-20T18:36:58.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.718+0000 7f654bb8f640 1 -- start start 2024-05-20T18:36:58.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.720+0000 7f654bb8f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f652c0039b0 0x7f6544154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:58.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.720+0000 7f654bb8f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65441f8950 0x7f65441f8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:58.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.720+0000 7f654bb8f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65441f92a0 0x7f65440ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:58.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.720+0000 7f654bb8f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f65440d4780 con 0x7f652c0039b0 2024-05-20T18:36:58.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.720+0000 7f654bb8f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f65440d0300 con 0x7f65441f8950 2024-05-20T18:36:58.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.720+0000 7f654bb8f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f65440d4900 con 0x7f65441f92a0 2024-05-20T18:36:58.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.720+0000 7f654992b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f652c0039b0 0x7f6544154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:58.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.720+0000 7f654992b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f652c0039b0 0x7f6544154c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48802/0 (socket says 172.21.15.5:48802) 2024-05-20T18:36:58.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.720+0000 7f654992b640 1 -- 172.21.15.5:0/1987234532 learned_addr learned my addr 172.21.15.5:0/1987234532 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:58.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.720+0000 7f654912a640 1 --2- 172.21.15.5:0/1987234532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65441f8950 0x7f65441f8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:58.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.720+0000 7f654a12c640 1 --2- 172.21.15.5:0/1987234532 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65441f92a0 0x7f65440ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:58.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.720+0000 7f654992b640 1 -- 172.21.15.5:0/1987234532 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65441f92a0 msgr2=0x7f65440ccf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:58.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.720+0000 7f654992b640 1 --2- 172.21.15.5:0/1987234532 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65441f92a0 0x7f65440ccf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:58.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.720+0000 7f654992b640 1 -- 172.21.15.5:0/1987234532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65441f8950 msgr2=0x7f65441f8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:58.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.720+0000 7f654992b640 1 --2- 172.21.15.5:0/1987234532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65441f8950 0x7f65441f8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:58.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.720+0000 7f654992b640 1 -- 172.21.15.5:0/1987234532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65440b7140 con 0x7f652c0039b0 2024-05-20T18:36:58.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.720+0000 7f654992b640 1 --2- 172.21.15.5:0/1987234532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f652c0039b0 0x7f6544154c20 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f653c00e230 tx=0x7f653c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:58.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.720+0000 7f65427fc640 1 -- 172.21.15.5:0/1987234532 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c009c10 con 0x7f652c0039b0 2024-05-20T18:36:58.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.721+0000 7f65427fc640 1 -- 172.21.15.5:0/1987234532 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f653c003bb0 con 0x7f652c0039b0 2024-05-20T18:36:58.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.721+0000 7f65427fc640 1 -- 172.21.15.5:0/1987234532 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c004020 con 0x7f652c0039b0 2024-05-20T18:36:58.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.721+0000 7f654bb8f640 1 -- 172.21.15.5:0/1987234532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65440b7420 con 0x7f652c0039b0 2024-05-20T18:36:58.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.721+0000 7f654bb8f640 1 -- 172.21.15.5:0/1987234532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65440b6b00 con 0x7f652c0039b0 2024-05-20T18:36:58.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.722+0000 7f654bb8f640 1 -- 172.21.15.5:0/1987234532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6544075b00 con 0x7f652c0039b0 2024-05-20T18:36:58.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.722+0000 7f65427fc640 1 -- 172.21.15.5:0/1987234532 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f653c005d10 con 0x7f652c0039b0 2024-05-20T18:36:58.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.723+0000 7f65427fc640 1 --2- 172.21.15.5:0/1987234532 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f653006be30 0x7f653006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:58.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.723+0000 7f65427fc640 1 -- 172.21.15.5:0/1987234532 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6331+0+0 (secure 0 0 0) 0x7f653c0ac620 con 0x7f652c0039b0 2024-05-20T18:36:58.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.723+0000 7f654912a640 1 --2- 172.21.15.5:0/1987234532 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f653006be30 0x7f653006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:58.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.723+0000 7f654912a640 1 --2- 172.21.15.5:0/1987234532 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f653006be30 0x7f653006e340 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f65440ccd40 tx=0x7f65380078f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:58.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.727+0000 7f65427fc640 1 -- 172.21.15.5:0/1987234532 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f653c07ad90 con 0x7f652c0039b0 2024-05-20T18:36:58.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.916+0000 7f654bb8f640 1 -- 172.21.15.5:0/1987234532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f65440cef60 con 0x7f652c0039b0 2024-05-20T18:36:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.917+0000 7f65427fc640 1 -- 172.21.15.5:0/1987234532 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+17118 (secure 0 0 0) 0x7f653c07a690 con 0x7f652c0039b0 2024-05-20T18:36:58.919 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:36:58.919 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":109,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:36:57.897296+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:35:29.146063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-20T18:35:29.754813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.6","mappings":[{"from":5,"to":4}]},{"pgid":"16.1c","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6835/3552410668":"2024-05-21T18:36:57.897254+0000","172.21.15.5:6834/3552410668":"2024-05-21T18:36:57.897254+0000","172.21.15.63:6835/2054897241":"2024-05-21T18:36:21.346248+0000","172.21.15.63:6833/2054897241":"2024-05-21T18:36:21.346248+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:36:58.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.919+0000 7f654bb8f640 1 -- 172.21.15.5:0/1987234532 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f653006be30 msgr2=0x7f653006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:58.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.919+0000 7f654bb8f640 1 --2- 172.21.15.5:0/1987234532 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f653006be30 0x7f653006e340 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f65440ccd40 tx=0x7f65380078f0 comp rx=0 tx=0).stop 2024-05-20T18:36:58.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.919+0000 7f654bb8f640 1 -- 172.21.15.5:0/1987234532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f652c0039b0 msgr2=0x7f6544154c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:58.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.919+0000 7f654bb8f640 1 --2- 172.21.15.5:0/1987234532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f652c0039b0 0x7f6544154c20 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f653c00e230 tx=0x7f653c009750 comp rx=0 tx=0).stop 2024-05-20T18:36:58.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.919+0000 7f654bb8f640 1 -- 172.21.15.5:0/1987234532 shutdown_connections 2024-05-20T18:36:58.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.919+0000 7f654bb8f640 1 --2- 172.21.15.5:0/1987234532 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f653006be30 0x7f653006e340 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:58.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.919+0000 7f654bb8f640 1 --2- 172.21.15.5:0/1987234532 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65441f92a0 0x7f65440ccf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:58.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.919+0000 7f654bb8f640 1 --2- 172.21.15.5:0/1987234532 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65441f8950 0x7f65441f8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:58.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.919+0000 7f654bb8f640 1 --2- 172.21.15.5:0/1987234532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f652c0039b0 0x7f6544154c20 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:58.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.919+0000 7f654bb8f640 1 -- 172.21.15.5:0/1987234532 >> 172.21.15.5:0/1987234532 conn(0x7f65440d4bd0 msgr2=0x7f65440b89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:58.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.920+0000 7f654bb8f640 1 -- 172.21.15.5:0/1987234532 shutdown_connections 2024-05-20T18:36:58.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:58.920+0000 7f654bb8f640 1 -- 172.21.15.5:0/1987234532 wait complete. 2024-05-20T18:36:58.932 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:35:29.146063+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:35:29.754813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:36:58.932 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:36:59.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.049+0000 7f7d13e8b640 1 Processor -- start 2024-05-20T18:36:59.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.049+0000 7f7d13e8b640 1 -- start start 2024-05-20T18:36:59.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.051+0000 7f7d13e8b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d0c162820 0x7f7d0c0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:59.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.051+0000 7f7d13e8b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7d0c0b3e80 0x7f7d0c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:59.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.051+0000 7f7d13e8b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7d0c06d910 0x7f7d0c0d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:59.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.051+0000 7f7d13e8b640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f7d0c06dd20 con 0x7f7d0c06d910 2024-05-20T18:36:59.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.051+0000 7f7d13e8b640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7d0c0d0630 con 0x7f7d0c0b3e80 2024-05-20T18:36:59.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.051+0000 7f7d13e8b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7d0c0d39c0 con 0x7f7d0c162820 2024-05-20T18:36:59.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.051+0000 7f7d12428640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7d0c06d910 0x7f7d0c0d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:59.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.051+0000 7f7d12428640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7d0c06d910 0x7f7d0c0d27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48816/0 (socket says 172.21.15.5:48816) 2024-05-20T18:36:59.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.051+0000 7f7d12428640 1 -- 172.21.15.5:0/3651071142 learned_addr learned my addr 172.21.15.5:0/3651071142 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:59.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.051+0000 7f7d12428640 1 -- 172.21.15.5:0/3651071142 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d0c162820 msgr2=0x7f7d0c0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:59.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.051+0000 7f7d11426640 1 --2- 172.21.15.5:0/3651071142 >> v2:172.21.15.63:3300/0 conn(0x7f7d0c0b3e80 0x7f7d0c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:59.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.051+0000 7f7d11c27640 1 --2- 172.21.15.5:0/3651071142 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d0c162820 0x7f7d0c0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:59.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.051+0000 7f7d12428640 1 --2- 172.21.15.5:0/3651071142 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d0c162820 0x7f7d0c0b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.051+0000 7f7d12428640 1 -- 172.21.15.5:0/3651071142 >> v2:172.21.15.63:3300/0 conn(0x7f7d0c0b3e80 msgr2=0x7f7d0c06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:59.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.051+0000 7f7d12428640 1 --2- 172.21.15.5:0/3651071142 >> v2:172.21.15.63:3300/0 conn(0x7f7d0c0b3e80 0x7f7d0c06d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.051+0000 7f7d12428640 1 -- 172.21.15.5:0/3651071142 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0c0d3ca0 con 0x7f7d0c06d910 2024-05-20T18:36:59.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.051+0000 7f7d11426640 1 --2- 172.21.15.5:0/3651071142 >> v2:172.21.15.63:3300/0 conn(0x7f7d0c0b3e80 0x7f7d0c06d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:36:59.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.052+0000 7f7d12428640 1 --2- 172.21.15.5:0/3651071142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d0c06d910 0x7f7d0c0d27a0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c08d710 tx=0x7f7d0402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b060481442062d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:59.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.052+0000 7f7d10c25640 1 -- 172.21.15.5:0/3651071142 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d04036070 con 0x7f7d0c06d910 2024-05-20T18:36:59.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.052+0000 7f7d10c25640 1 -- 172.21.15.5:0/3651071142 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d04031600 con 0x7f7d0c06d910 2024-05-20T18:36:59.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.052+0000 7f7d10c25640 1 -- 172.21.15.5:0/3651071142 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d040319d0 con 0x7f7d0c06d910 2024-05-20T18:36:59.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.053+0000 7f7d13e8b640 1 -- 172.21.15.5:0/3651071142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d0c06d910 msgr2=0x7f7d0c0d27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:59.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.053+0000 7f7d13e8b640 1 --2- 172.21.15.5:0/3651071142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d0c06d910 0x7f7d0c0d27a0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c08d710 tx=0x7f7d0402a6a0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.053+0000 7f7d13e8b640 1 -- 172.21.15.5:0/3651071142 shutdown_connections 2024-05-20T18:36:59.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.053+0000 7f7d13e8b640 1 --2- 172.21.15.5:0/3651071142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d0c06d910 0x7f7d0c0d27a0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.053+0000 7f7d13e8b640 1 --2- 172.21.15.5:0/3651071142 >> v2:172.21.15.63:3300/0 conn(0x7f7d0c0b3e80 0x7f7d0c06d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.053+0000 7f7d13e8b640 1 --2- 172.21.15.5:0/3651071142 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d0c162820 0x7f7d0c0b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.053+0000 7f7d13e8b640 1 -- 172.21.15.5:0/3651071142 >> 172.21.15.5:0/3651071142 conn(0x7f7d0c0d4bd0 msgr2=0x7f7d0c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:59.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.053+0000 7f7d13e8b640 1 -- 172.21.15.5:0/3651071142 shutdown_connections 2024-05-20T18:36:59.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.053+0000 7f7d13e8b640 1 -- 172.21.15.5:0/3651071142 wait complete. 2024-05-20T18:36:59.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.055+0000 7f7d13e8b640 1 Processor -- start 2024-05-20T18:36:59.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.055+0000 7f7d13e8b640 1 -- start start 2024-05-20T18:36:59.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.056+0000 7f7d13e8b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d0c06d910 0x7f7d0c1f9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:59.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.056+0000 7f7d13e8b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d0c0b3e80 0x7f7d0c14dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:59.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.056+0000 7f7d13e8b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d0c162820 0x7f7d0c0c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:59.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.056+0000 7f7d13e8b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c0d0300 con 0x7f7d0c06d910 2024-05-20T18:36:59.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.056+0000 7f7d13e8b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c0d4260 con 0x7f7d0c162820 2024-05-20T18:36:59.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.056+0000 7f7d13e8b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7d0c0d4780 con 0x7f7d0c0b3e80 2024-05-20T18:36:59.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.056+0000 7f7d11c27640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d0c06d910 0x7f7d0c1f9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:59.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.056+0000 7f7d11c27640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d0c06d910 0x7f7d0c1f9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48832/0 (socket says 172.21.15.5:48832) 2024-05-20T18:36:59.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.056+0000 7f7d11c27640 1 -- 172.21.15.5:0/2735503499 learned_addr learned my addr 172.21.15.5:0/2735503499 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:59.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.057+0000 7f7d12428640 1 --2- 172.21.15.5:0/2735503499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d0c162820 0x7f7d0c0c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:59.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.057+0000 7f7d11426640 1 --2- 172.21.15.5:0/2735503499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d0c0b3e80 0x7f7d0c14dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:59.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.057+0000 7f7d11c27640 1 -- 172.21.15.5:0/2735503499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d0c0b3e80 msgr2=0x7f7d0c14dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:59.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.057+0000 7f7d11c27640 1 --2- 172.21.15.5:0/2735503499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d0c0b3e80 0x7f7d0c14dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.057+0000 7f7d11c27640 1 -- 172.21.15.5:0/2735503499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d0c162820 msgr2=0x7f7d0c0c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:59.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.057+0000 7f7d11c27640 1 --2- 172.21.15.5:0/2735503499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d0c162820 0x7f7d0c0c8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.057+0000 7f7d11c27640 1 -- 172.21.15.5:0/2735503499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0c14e270 con 0x7f7d0c06d910 2024-05-20T18:36:59.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.057+0000 7f7d11426640 1 --2- 172.21.15.5:0/2735503499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d0c0b3e80 0x7f7d0c14dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:36:59.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.057+0000 7f7d11c27640 1 --2- 172.21.15.5:0/2735503499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d0c06d910 0x7f7d0c1f9350 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f7d0800c7f0 tx=0x7f7d0800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:59.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.057+0000 7f7d027fc640 1 -- 172.21.15.5:0/2735503499 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d08015070 con 0x7f7d0c06d910 2024-05-20T18:36:59.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.057+0000 7f7d027fc640 1 -- 172.21.15.5:0/2735503499 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d08004220 con 0x7f7d0c06d910 2024-05-20T18:36:59.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.057+0000 7f7d027fc640 1 -- 172.21.15.5:0/2735503499 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d08007620 con 0x7f7d0c06d910 2024-05-20T18:36:59.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.057+0000 7f7d13e8b640 1 -- 172.21.15.5:0/2735503499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d0c0caa20 con 0x7f7d0c06d910 2024-05-20T18:36:59.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.057+0000 7f7d13e8b640 1 -- 172.21.15.5:0/2735503499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d0c0cafe0 con 0x7f7d0c06d910 2024-05-20T18:36:59.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.058+0000 7f7d13e8b640 1 -- 172.21.15.5:0/2735503499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d0c075b00 con 0x7f7d0c06d910 2024-05-20T18:36:59.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.059+0000 7f7d027fc640 1 -- 172.21.15.5:0/2735503499 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7d080079d0 con 0x7f7d0c06d910 2024-05-20T18:36:59.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.059+0000 7f7d027fc640 1 --2- 172.21.15.5:0/2735503499 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7cf006be60 0x7f7cf006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:59.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.059+0000 7f7d027fc640 1 -- 172.21.15.5:0/2735503499 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6331+0+0 (secure 0 0 0) 0x7f7d0808b0a0 con 0x7f7d0c06d910 2024-05-20T18:36:59.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.059+0000 7f7d11426640 1 --2- 172.21.15.5:0/2735503499 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7cf006be60 0x7f7cf006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:59.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.059+0000 7f7d11426640 1 --2- 172.21.15.5:0/2735503499 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7cf006be60 0x7f7cf006e370 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c0b71c0 tx=0x7f7cfc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:59.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.064+0000 7f7d027fc640 1 -- 172.21.15.5:0/2735503499 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7d080597e0 con 0x7f7d0c06d910 2024-05-20T18:36:59.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.246+0000 7f7d13e8b640 1 -- 172.21.15.5:0/2735503499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7d0c06e2a0 con 0x7f7d0c06d910 2024-05-20T18:36:59.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.246+0000 7f7d027fc640 1 -- 172.21.15.5:0/2735503499 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) ==== 93+0+10 (secure 0 0 0) 0x7f7d080590e0 con 0x7f7d0c06d910 2024-05-20T18:36:59.248 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:36:59.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.248+0000 7f7d13e8b640 1 -- 172.21.15.5:0/2735503499 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7cf006be60 msgr2=0x7f7cf006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:59.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.248+0000 7f7d13e8b640 1 --2- 172.21.15.5:0/2735503499 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7cf006be60 0x7f7cf006e370 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c0b71c0 tx=0x7f7cfc008040 comp rx=0 tx=0).stop 2024-05-20T18:36:59.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.248+0000 7f7d13e8b640 1 -- 172.21.15.5:0/2735503499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d0c06d910 msgr2=0x7f7d0c1f9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:59.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.248+0000 7f7d13e8b640 1 --2- 172.21.15.5:0/2735503499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d0c06d910 0x7f7d0c1f9350 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f7d0800c7f0 tx=0x7f7d0800ccb0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.248+0000 7f7d13e8b640 1 -- 172.21.15.5:0/2735503499 shutdown_connections 2024-05-20T18:36:59.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.248+0000 7f7d13e8b640 1 --2- 172.21.15.5:0/2735503499 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7cf006be60 0x7f7cf006e370 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.248+0000 7f7d13e8b640 1 --2- 172.21.15.5:0/2735503499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d0c162820 0x7f7d0c0c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.248+0000 7f7d13e8b640 1 --2- 172.21.15.5:0/2735503499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d0c0b3e80 0x7f7d0c14dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.248+0000 7f7d13e8b640 1 --2- 172.21.15.5:0/2735503499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d0c06d910 0x7f7d0c1f9350 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.248+0000 7f7d13e8b640 1 -- 172.21.15.5:0/2735503499 >> 172.21.15.5:0/2735503499 conn(0x7f7d0c0d4bd0 msgr2=0x7f7d0c161580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:59.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.248+0000 7f7d13e8b640 1 -- 172.21.15.5:0/2735503499 shutdown_connections 2024-05-20T18:36:59.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.248+0000 7f7d13e8b640 1 -- 172.21.15.5:0/2735503499 wait complete. 2024-05-20T18:36:59.261 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:36:59.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.377+0000 7f14c7d96640 1 Processor -- start 2024-05-20T18:36:59.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.378+0000 7f14c7d96640 1 -- start start 2024-05-20T18:36:59.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.379+0000 7f14c7d96640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14c0162820 0x7f14c00b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:59.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.379+0000 7f14c7d96640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14c00d0630 con 0x7f14c00b3e80 2024-05-20T18:36:59.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.379+0000 7f14c7d96640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14c00d39c0 con 0x7f14c006dae0 2024-05-20T18:36:59.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.379+0000 7f14c7d96640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f14c00d3ca0 con 0x7f14c0162820 2024-05-20T18:36:59.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.379+0000 7f14c5331640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f14c00b3e80 0x7f14c006d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51754/0 (socket says 172.21.15.5:51754) 2024-05-20T18:36:59.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.379+0000 7f14c5331640 1 -- 172.21.15.5:0/2590739700 learned_addr learned my addr 172.21.15.5:0/2590739700 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:36:59.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.379+0000 7f14c5b32640 1 --2- 172.21.15.5:0/2590739700 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14c0162820 0x7f14c00b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:59.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.379+0000 7f14c4b30640 1 -- 172.21.15.5:0/2590739700 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3373257903 0 0) 0x7f14c00d0630 con 0x7f14c00b3e80 2024-05-20T18:36:59.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.380+0000 7f14c4b30640 1 -- 172.21.15.5:0/2590739700 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14a8003520 con 0x7f14c00b3e80 2024-05-20T18:36:59.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.380+0000 7f14c4b30640 1 -- 172.21.15.5:0/2590739700 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3818397942 0 0) 0x7f14a8003520 con 0x7f14c00b3e80 2024-05-20T18:36:59.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.380+0000 7f14c4b30640 1 -- 172.21.15.5:0/2590739700 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14c0162820 msgr2=0x7f14c00b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:59.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.380+0000 7f14c4b30640 1 --2- 172.21.15.5:0/2590739700 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14c0162820 0x7f14c00b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.380+0000 7f14c4b30640 1 -- 172.21.15.5:0/2590739700 >> v1:172.21.15.63:6789/0 conn(0x7f14c006dae0 legacy=0x7f14c00b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:59.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.380+0000 7f14c4b30640 1 -- 172.21.15.5:0/2590739700 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c00d3f80 con 0x7f14c00b3e80 2024-05-20T18:36:59.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.380+0000 7f14c4b30640 1 -- 172.21.15.5:0/2590739700 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f14b4001670 con 0x7f14c00b3e80 2024-05-20T18:36:59.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.380+0000 7f14c4b30640 1 -- 172.21.15.5:0/2590739700 >> v1:172.21.15.5:6789/0 conn(0x7f14c00b3e80 legacy=0x7f14c006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:59.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.380+0000 7f14c4b30640 1 --2- 172.21.15.5:0/2590739700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14a80039e0 0x7f14a8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:59.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.380+0000 7f14c4b30640 1 --2- 172.21.15.5:0/2590739700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14a8004800 0x7f14a8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:59.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.380+0000 7f14c4b30640 1 --2- 172.21.15.5:0/2590739700 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14c0162820 0x7f14a8005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:59.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.380+0000 7f14c4b30640 1 -- 172.21.15.5:0/2590739700 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f14c00d3ca0 con 0x7f14a8004800 2024-05-20T18:36:59.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.380+0000 7f14c4b30640 1 -- 172.21.15.5:0/2590739700 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f14a800ba70 con 0x7f14a80039e0 2024-05-20T18:36:59.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.380+0000 7f14c4b30640 1 -- 172.21.15.5:0/2590739700 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f14a800bc70 con 0x7f14c0162820 2024-05-20T18:36:59.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.380+0000 7f14c5331640 1 --2- 172.21.15.5:0/2590739700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14a8004800 0x7f14a8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:59.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.380+0000 7f14c5b32640 1 --2- 172.21.15.5:0/2590739700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14a80039e0 0x7f14a8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:59.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.380+0000 7f14c6333640 1 --2- 172.21.15.5:0/2590739700 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14c0162820 0x7f14a8005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:59.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.380+0000 7f14c5331640 1 -- 172.21.15.5:0/2590739700 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14c0162820 msgr2=0x7f14a8005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:59.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.380+0000 7f14c5331640 1 --2- 172.21.15.5:0/2590739700 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14c0162820 0x7f14a8005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.380+0000 7f14c5331640 1 -- 172.21.15.5:0/2590739700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14a80039e0 msgr2=0x7f14a8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:59.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.380+0000 7f14c5331640 1 --2- 172.21.15.5:0/2590739700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14a80039e0 0x7f14a8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.380+0000 7f14c5331640 1 -- 172.21.15.5:0/2590739700 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a8003520 con 0x7f14a8004800 2024-05-20T18:36:59.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.381+0000 7f14c5331640 1 --2- 172.21.15.5:0/2590739700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14a8004800 0x7f14a8004c10 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f14c004f770 tx=0x7f14a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11116b82d90d7007 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:59.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.381+0000 7f14c4b30640 1 -- 172.21.15.5:0/2590739700 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4007ca0 con 0x7f14a8004800 2024-05-20T18:36:59.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.381+0000 7f14c4b30640 1 -- 172.21.15.5:0/2590739700 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14b4005ad0 con 0x7f14a8004800 2024-05-20T18:36:59.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.381+0000 7f14c4b30640 1 -- 172.21.15.5:0/2590739700 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4007070 con 0x7f14a8004800 2024-05-20T18:36:59.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.382+0000 7f14c7d96640 1 -- 172.21.15.5:0/2590739700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14a8004800 msgr2=0x7f14a8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:59.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.382+0000 7f14c7d96640 1 --2- 172.21.15.5:0/2590739700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14a8004800 0x7f14a8004c10 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f14c004f770 tx=0x7f14a8005830 comp rx=0 tx=0).stop 2024-05-20T18:36:59.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.382+0000 7f14c6333640 1 -- 172.21.15.5:0/2590739700 reap_dead start 2024-05-20T18:36:59.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.382+0000 7f14c7d96640 1 -- 172.21.15.5:0/2590739700 shutdown_connections 2024-05-20T18:36:59.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.382+0000 7f14c7d96640 1 -- 172.21.15.5:0/2590739700 >> 172.21.15.5:0/2590739700 conn(0x7f14c00d4bd0 msgr2=0x7f14c00d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:59.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.382+0000 7f14c7d96640 1 -- 172.21.15.5:0/2590739700 shutdown_connections 2024-05-20T18:36:59.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.382+0000 7f14c7d96640 1 -- 172.21.15.5:0/2590739700 wait complete. 2024-05-20T18:36:59.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.384+0000 7f14c7d96640 1 Processor -- start 2024-05-20T18:36:59.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.384+0000 7f14c7d96640 1 -- start start 2024-05-20T18:36:59.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.385+0000 7f14c7d96640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14a8004800 0x7f14c0156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:59.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.385+0000 7f14c7d96640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14c0155400 0x7f14c0156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:59.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.385+0000 7f14c7d96640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14c0158280 0x7f14c0158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:59.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.385+0000 7f14c7d96640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f14c00d4780 con 0x7f14a8004800 2024-05-20T18:36:59.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.385+0000 7f14c7d96640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f14c00d0300 con 0x7f14c0155400 2024-05-20T18:36:59.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.385+0000 7f14c7d96640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f14c00d4900 con 0x7f14c0158280 2024-05-20T18:36:59.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.385+0000 7f14c5b32640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14a8004800 0x7f14c0156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:59.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.385+0000 7f14c5b32640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14a8004800 0x7f14c0156230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48844/0 (socket says 172.21.15.5:48844) 2024-05-20T18:36:59.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.385+0000 7f14c5b32640 1 -- 172.21.15.5:0/875924001 learned_addr learned my addr 172.21.15.5:0/875924001 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:59.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.386+0000 7f14c5331640 1 --2- 172.21.15.5:0/875924001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14c0155400 0x7f14c0156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:59.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.386+0000 7f14c6333640 1 --2- 172.21.15.5:0/875924001 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14c0158280 0x7f14c0158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:59.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.386+0000 7f14c5b32640 1 -- 172.21.15.5:0/875924001 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14c0158280 msgr2=0x7f14c0158690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:59.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.386+0000 7f14c5b32640 1 --2- 172.21.15.5:0/875924001 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14c0158280 0x7f14c0158690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.386+0000 7f14c5b32640 1 -- 172.21.15.5:0/875924001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14c0155400 msgr2=0x7f14c0156770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:59.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.386+0000 7f14c5b32640 1 --2- 172.21.15.5:0/875924001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14c0155400 0x7f14c0156770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.386+0000 7f14c5b32640 1 -- 172.21.15.5:0/875924001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c0158bd0 con 0x7f14a8004800 2024-05-20T18:36:59.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.386+0000 7f14c5b32640 1 --2- 172.21.15.5:0/875924001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14a8004800 0x7f14c0156230 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f14b0011e30 tx=0x7f14b000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:59.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.386+0000 7f14be7fc640 1 -- 172.21.15.5:0/875924001 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b0015630 con 0x7f14a8004800 2024-05-20T18:36:59.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.386+0000 7f14be7fc640 1 -- 172.21.15.5:0/875924001 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14b00157d0 con 0x7f14a8004800 2024-05-20T18:36:59.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.386+0000 7f14c7d96640 1 -- 172.21.15.5:0/875924001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c015aea0 con 0x7f14a8004800 2024-05-20T18:36:59.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.386+0000 7f14be7fc640 1 -- 172.21.15.5:0/875924001 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b0015ba0 con 0x7f14a8004800 2024-05-20T18:36:59.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.386+0000 7f14c7d96640 1 -- 172.21.15.5:0/875924001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c01f4f30 con 0x7f14a8004800 2024-05-20T18:36:59.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.387+0000 7f14c7d96640 1 -- 172.21.15.5:0/875924001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14c015b070 con 0x7f14a8004800 2024-05-20T18:36:59.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.388+0000 7f14be7fc640 1 -- 172.21.15.5:0/875924001 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f14b001d400 con 0x7f14a8004800 2024-05-20T18:36:59.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.388+0000 7f14be7fc640 1 --2- 172.21.15.5:0/875924001 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f14ac06be60 0x7f14ac06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:59.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.388+0000 7f14be7fc640 1 -- 172.21.15.5:0/875924001 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6331+0+0 (secure 0 0 0) 0x7f14b008c180 con 0x7f14a8004800 2024-05-20T18:36:59.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.388+0000 7f14c5331640 1 --2- 172.21.15.5:0/875924001 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f14ac06be60 0x7f14ac06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:59.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.389+0000 7f14c5331640 1 --2- 172.21.15.5:0/875924001 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f14ac06be60 0x7f14ac06e370 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f14c004f770 tx=0x7f14b4009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:59.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.393+0000 7f14be7fc640 1 -- 172.21.15.5:0/875924001 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f14c0158bd0 con 0x7f14a8004800 2024-05-20T18:36:59.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.577+0000 7f14c7d96640 1 -- 172.21.15.5:0/875924001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f14c015ac80 con 0x7f14a8004800 2024-05-20T18:36:59.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.578+0000 7f14be7fc640 1 -- 172.21.15.5:0/875924001 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) ==== 104+0+11 (secure 0 0 0) 0x7f14b0019030 con 0x7f14a8004800 2024-05-20T18:36:59.579 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:36:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.580+0000 7f14c7d96640 1 -- 172.21.15.5:0/875924001 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f14ac06be60 msgr2=0x7f14ac06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.580+0000 7f14c7d96640 1 --2- 172.21.15.5:0/875924001 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f14ac06be60 0x7f14ac06e370 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f14c004f770 tx=0x7f14b4009030 comp rx=0 tx=0).stop 2024-05-20T18:36:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.580+0000 7f14c7d96640 1 -- 172.21.15.5:0/875924001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14a8004800 msgr2=0x7f14c0156230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.580+0000 7f14c7d96640 1 --2- 172.21.15.5:0/875924001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14a8004800 0x7f14c0156230 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f14b0011e30 tx=0x7f14b000f580 comp rx=0 tx=0).stop 2024-05-20T18:36:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.580+0000 7f14c7d96640 1 -- 172.21.15.5:0/875924001 shutdown_connections 2024-05-20T18:36:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.580+0000 7f14c7d96640 1 --2- 172.21.15.5:0/875924001 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f14ac06be60 0x7f14ac06e370 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.580+0000 7f14c7d96640 1 --2- 172.21.15.5:0/875924001 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14c0158280 0x7f14c0158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.580+0000 7f14c7d96640 1 --2- 172.21.15.5:0/875924001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14c0155400 0x7f14c0156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.580+0000 7f14c7d96640 1 --2- 172.21.15.5:0/875924001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14a8004800 0x7f14c0156230 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.580+0000 7f14c7d96640 1 -- 172.21.15.5:0/875924001 >> 172.21.15.5:0/875924001 conn(0x7f14c00d4bd0 msgr2=0x7f14c00b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.580+0000 7f14c7d96640 1 -- 172.21.15.5:0/875924001 shutdown_connections 2024-05-20T18:36:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.580+0000 7f14c7d96640 1 -- 172.21.15.5:0/875924001 wait complete. 2024-05-20T18:36:59.592 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:36:59.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.708+0000 7f052cf2f640 1 Processor -- start 2024-05-20T18:36:59.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.708+0000 7f052cf2f640 1 -- start start 2024-05-20T18:36:59.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.710+0000 7f052cf2f640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f052806d3d0 0x7f052806d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:59.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.710+0000 7f052cf2f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05280b8190 0x7f05280b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:59.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.710+0000 7f052cf2f640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f052806dd20 con 0x7f0528162820 2024-05-20T18:36:59.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.710+0000 7f052cf2f640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f05280d0630 con 0x7f052806d3d0 2024-05-20T18:36:59.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.710+0000 7f052cf2f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f05280d39c0 con 0x7f05280b8190 2024-05-20T18:36:59.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.710+0000 7f052659c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0528162820 0x7f05280b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51766/0 (socket says 172.21.15.5:51766) 2024-05-20T18:36:59.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.710+0000 7f052659c640 1 -- 172.21.15.5:0/2039123997 learned_addr learned my addr 172.21.15.5:0/2039123997 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:36:59.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.710+0000 7f0526d9d640 1 --2- 172.21.15.5:0/2039123997 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05280b8190 0x7f05280b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:59.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.710+0000 7f0525d9b640 1 --2- 172.21.15.5:0/2039123997 >> v2:172.21.15.63:3300/0 conn(0x7f052806d3d0 0x7f052806d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:59.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.710+0000 7f052559a640 1 -- 172.21.15.5:0/2039123997 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2571893731 0 0) 0x7f0518001110 con 0x7f0528162820 2024-05-20T18:36:59.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.710+0000 7f052559a640 1 -- 172.21.15.5:0/2039123997 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0510003520 con 0x7f0528162820 2024-05-20T18:36:59.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.710+0000 7f052559a640 1 -- 172.21.15.5:0/2039123997 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2441681051 0 0) 0x7f0510003520 con 0x7f0528162820 2024-05-20T18:36:59.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.710+0000 7f0525d9b640 1 -- 172.21.15.5:0/2039123997 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05280b8190 msgr2=0x7f05280b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:36:59.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.710+0000 7f0525d9b640 1 --2- 172.21.15.5:0/2039123997 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05280b8190 0x7f05280b85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.710+0000 7f0525d9b640 1 -- 172.21.15.5:0/2039123997 >> v1:172.21.15.5:6789/0 conn(0x7f0528162820 legacy=0x7f05280b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:59.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.711+0000 7f0525d9b640 1 -- 172.21.15.5:0/2039123997 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05280d3ca0 con 0x7f052806d3d0 2024-05-20T18:36:59.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.711+0000 7f0526d9d640 1 --2- 172.21.15.5:0/2039123997 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05280b8190 0x7f05280b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:36:59.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.711+0000 7f0525d9b640 1 --2- 172.21.15.5:0/2039123997 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f052806d3d0 0x7f052806d7e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f051c0097a0 tx=0x7f051c02a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efa2086baf293983 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:59.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.712+0000 7f052559a640 1 -- 172.21.15.5:0/2039123997 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c037070 con 0x7f052806d3d0 2024-05-20T18:36:59.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.712+0000 7f052559a640 1 -- 172.21.15.5:0/2039123997 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f051c02aba0 con 0x7f052806d3d0 2024-05-20T18:36:59.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.712+0000 7f052559a640 1 -- 172.21.15.5:0/2039123997 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0518001840 con 0x7f052806d3d0 2024-05-20T18:36:59.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.712+0000 7f052cf2f640 1 -- 172.21.15.5:0/2039123997 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f052806d3d0 msgr2=0x7f052806d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:59.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.712+0000 7f052cf2f640 1 --2- 172.21.15.5:0/2039123997 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f052806d3d0 0x7f052806d7e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f051c0097a0 tx=0x7f051c02a110 comp rx=0 tx=0).stop 2024-05-20T18:36:59.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.712+0000 7f052cf2f640 1 -- 172.21.15.5:0/2039123997 shutdown_connections 2024-05-20T18:36:59.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.712+0000 7f052cf2f640 1 --2- 172.21.15.5:0/2039123997 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05280b8190 0x7f05280b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.712+0000 7f052cf2f640 1 --2- 172.21.15.5:0/2039123997 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f052806d3d0 0x7f052806d7e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.712+0000 7f052cf2f640 1 -- 172.21.15.5:0/2039123997 >> 172.21.15.5:0/2039123997 conn(0x7f05280d4bd0 msgr2=0x7f05280d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:59.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.713+0000 7f052cf2f640 1 -- 172.21.15.5:0/2039123997 shutdown_connections 2024-05-20T18:36:59.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.713+0000 7f052cf2f640 1 -- 172.21.15.5:0/2039123997 wait complete. 2024-05-20T18:36:59.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.714+0000 7f052cf2f640 1 Processor -- start 2024-05-20T18:36:59.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.715+0000 7f052cf2f640 1 -- start start 2024-05-20T18:36:59.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.716+0000 7f052cf2f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052806d3d0 0x7f052814c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:59.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.716+0000 7f052cf2f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05280b8190 0x7f05280cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:59.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.716+0000 7f052cf2f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0528162820 0x7f05280b5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:59.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.716+0000 7f052cf2f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f05280d0300 con 0x7f052806d3d0 2024-05-20T18:36:59.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.716+0000 7f052cf2f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f05280d4260 con 0x7f0528162820 2024-05-20T18:36:59.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.716+0000 7f052cf2f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f05280b4050 con 0x7f05280b8190 2024-05-20T18:36:59.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.716+0000 7f052659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052806d3d0 0x7f052814c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:59.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.716+0000 7f052659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052806d3d0 0x7f052814c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48856/0 (socket says 172.21.15.5:48856) 2024-05-20T18:36:59.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.716+0000 7f052659c640 1 -- 172.21.15.5:0/510238515 learned_addr learned my addr 172.21.15.5:0/510238515 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:36:59.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.716+0000 7f0525d9b640 1 --2- 172.21.15.5:0/510238515 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05280b8190 0x7f05280cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:59.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.716+0000 7f0526d9d640 1 --2- 172.21.15.5:0/510238515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0528162820 0x7f05280b5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:59.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.716+0000 7f052659c640 1 -- 172.21.15.5:0/510238515 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05280b8190 msgr2=0x7f05280cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:59.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.716+0000 7f052659c640 1 --2- 172.21.15.5:0/510238515 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05280b8190 0x7f05280cc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.716+0000 7f052659c640 1 -- 172.21.15.5:0/510238515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0528162820 msgr2=0x7f05280b5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:59.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.716+0000 7f052659c640 1 --2- 172.21.15.5:0/510238515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0528162820 0x7f05280b5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.716+0000 7f052659c640 1 -- 172.21.15.5:0/510238515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f052814c5b0 con 0x7f052806d3d0 2024-05-20T18:36:59.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.716+0000 7f0526d9d640 1 --2- 172.21.15.5:0/510238515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0528162820 0x7f05280b5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:36:59.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.717+0000 7f052659c640 1 --2- 172.21.15.5:0/510238515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052806d3d0 0x7f052814c070 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f051800c760 tx=0x7f051800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:59.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.717+0000 7f0516ffd640 1 -- 172.21.15.5:0/510238515 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0518017070 con 0x7f052806d3d0 2024-05-20T18:36:59.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.717+0000 7f0516ffd640 1 -- 172.21.15.5:0/510238515 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0518005af0 con 0x7f052806d3d0 2024-05-20T18:36:59.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.717+0000 7f052cf2f640 1 -- 172.21.15.5:0/510238515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05280ceca0 con 0x7f052806d3d0 2024-05-20T18:36:59.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.717+0000 7f0516ffd640 1 -- 172.21.15.5:0/510238515 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0518007070 con 0x7f052806d3d0 2024-05-20T18:36:59.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.717+0000 7f052cf2f640 1 -- 172.21.15.5:0/510238515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05280cf260 con 0x7f052806d3d0 2024-05-20T18:36:59.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.718+0000 7f052cf2f640 1 -- 172.21.15.5:0/510238515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0528075b00 con 0x7f052806d3d0 2024-05-20T18:36:59.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.718+0000 7f0516ffd640 1 -- 172.21.15.5:0/510238515 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f051801a070 con 0x7f052806d3d0 2024-05-20T18:36:59.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.719+0000 7f0516ffd640 1 --2- 172.21.15.5:0/510238515 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f050806be30 0x7f050806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:36:59.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.719+0000 7f0516ffd640 1 -- 172.21.15.5:0/510238515 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6331+0+0 (secure 0 0 0) 0x7f051808a680 con 0x7f052806d3d0 2024-05-20T18:36:59.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.719+0000 7f0525d9b640 1 --2- 172.21.15.5:0/510238515 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f050806be30 0x7f050806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:36:59.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.719+0000 7f0525d9b640 1 --2- 172.21.15.5:0/510238515 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f050806be30 0x7f050806e340 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f051c034000 tx=0x7f051c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:36:59.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.723+0000 7f0516ffd640 1 -- 172.21.15.5:0/510238515 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f052814c5b0 con 0x7f052806d3d0 2024-05-20T18:36:59.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.913+0000 7f052cf2f640 1 -- 172.21.15.5:0/510238515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f052814cac0 con 0x7f052806d3d0 2024-05-20T18:36:59.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.913+0000 7f0516ffd640 1 -- 172.21.15.5:0/510238515 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) ==== 100+0+11 (secure 0 0 0) 0x7f052806dd20 con 0x7f052806d3d0 2024-05-20T18:36:59.916 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:36:59.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.915+0000 7f052cf2f640 1 -- 172.21.15.5:0/510238515 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f050806be30 msgr2=0x7f050806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:59.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.915+0000 7f052cf2f640 1 --2- 172.21.15.5:0/510238515 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f050806be30 0x7f050806e340 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f051c034000 tx=0x7f051c02e000 comp rx=0 tx=0).stop 2024-05-20T18:36:59.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.915+0000 7f052cf2f640 1 -- 172.21.15.5:0/510238515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052806d3d0 msgr2=0x7f052814c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:36:59.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.915+0000 7f052cf2f640 1 --2- 172.21.15.5:0/510238515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052806d3d0 0x7f052814c070 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f051800c760 tx=0x7f051800cc20 comp rx=0 tx=0).stop 2024-05-20T18:36:59.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.916+0000 7f052cf2f640 1 -- 172.21.15.5:0/510238515 shutdown_connections 2024-05-20T18:36:59.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.916+0000 7f052cf2f640 1 --2- 172.21.15.5:0/510238515 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f050806be30 0x7f050806e340 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.916+0000 7f052cf2f640 1 --2- 172.21.15.5:0/510238515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0528162820 0x7f05280b5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.916+0000 7f052cf2f640 1 --2- 172.21.15.5:0/510238515 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05280b8190 0x7f05280cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.916+0000 7f052cf2f640 1 --2- 172.21.15.5:0/510238515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052806d3d0 0x7f052814c070 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:36:59.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.916+0000 7f052cf2f640 1 -- 172.21.15.5:0/510238515 >> 172.21.15.5:0/510238515 conn(0x7f05280d4bd0 msgr2=0x7f0528160eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:36:59.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.916+0000 7f052cf2f640 1 -- 172.21.15.5:0/510238515 shutdown_connections 2024-05-20T18:36:59.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:36:59.916+0000 7f052cf2f640 1 -- 172.21.15.5:0/510238515 wait complete. 2024-05-20T18:36:59.928 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:37:00.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.041+0000 7f172aae7640 1 Processor -- start 2024-05-20T18:37:00.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.041+0000 7f172aae7640 1 -- start start 2024-05-20T18:37:00.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.042+0000 7f172aae7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1724163a10 0x7f17240d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:00.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.042+0000 7f172aae7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17240b4010 0x7f17240cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:00.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.042+0000 7f172aae7640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17240d0370 con 0x7f17240d14b0 2024-05-20T18:37:00.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.042+0000 7f172aae7640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f17240d3ca0 con 0x7f1724163a10 2024-05-20T18:37:00.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.042+0000 7f172aae7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f17240d0650 con 0x7f17240b4010 2024-05-20T18:37:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.043+0000 7f1728883640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1724163a10 0x7f17240d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.043+0000 7f1723fff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17240b4010 0x7f17240cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.043+0000 7f1728883640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1724163a10 0x7f17240d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:58562/0 (socket says 172.21.15.5:58562) 2024-05-20T18:37:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.043+0000 7f1728883640 1 -- 172.21.15.5:0/3157405112 learned_addr learned my addr 172.21.15.5:0/3157405112 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.043+0000 7f1728883640 1 -- 172.21.15.5:0/3157405112 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17240b4010 msgr2=0x7f17240cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.043+0000 7f1728883640 1 --2- 172.21.15.5:0/3157405112 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17240b4010 0x7f17240cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.043+0000 7f1728883640 1 -- 172.21.15.5:0/3157405112 >> v1:172.21.15.63:6789/0 conn(0x7f17240d14b0 legacy=0x7f17240d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.043+0000 7f1728883640 1 -- 172.21.15.5:0/3157405112 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17240d3650 con 0x7f1724163a10 2024-05-20T18:37:00.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.043+0000 7f17237fe640 1 -- 172.21.15.5:0/3157405112 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2148549595 0 0) 0x7f17240d0370 con 0x7f17240d14b0 2024-05-20T18:37:00.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.044+0000 7f1728883640 1 --2- 172.21.15.5:0/3157405112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1724163a10 0x7f17240d4130 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1718009870 tx=0x7f171802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=484e25f6e405af2a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.044+0000 7f17237fe640 1 -- 172.21.15.5:0/3157405112 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718037070 con 0x7f1724163a10 2024-05-20T18:37:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.044+0000 7f17237fe640 1 -- 172.21.15.5:0/3157405112 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1718032440 con 0x7f1724163a10 2024-05-20T18:37:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.044+0000 7f17237fe640 1 -- 172.21.15.5:0/3157405112 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718032810 con 0x7f1724163a10 2024-05-20T18:37:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.045+0000 7f172aae7640 1 -- 172.21.15.5:0/3157405112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1724163a10 msgr2=0x7f17240d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.045+0000 7f172aae7640 1 --2- 172.21.15.5:0/3157405112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1724163a10 0x7f17240d4130 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1718009870 tx=0x7f171802a270 comp rx=0 tx=0).stop 2024-05-20T18:37:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.045+0000 7f172aae7640 1 -- 172.21.15.5:0/3157405112 shutdown_connections 2024-05-20T18:37:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.045+0000 7f172aae7640 1 --2- 172.21.15.5:0/3157405112 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17240b4010 0x7f17240cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.045+0000 7f172aae7640 1 --2- 172.21.15.5:0/3157405112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1724163a10 0x7f17240d4130 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.045+0000 7f172aae7640 1 -- 172.21.15.5:0/3157405112 >> 172.21.15.5:0/3157405112 conn(0x7f172406d240 msgr2=0x7f172406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.045+0000 7f172aae7640 1 -- 172.21.15.5:0/3157405112 shutdown_connections 2024-05-20T18:37:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.045+0000 7f172aae7640 1 -- 172.21.15.5:0/3157405112 wait complete. 2024-05-20T18:37:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.047+0000 7f172aae7640 1 Processor -- start 2024-05-20T18:37:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.047+0000 7f172aae7640 1 -- start start 2024-05-20T18:37:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.048+0000 7f172aae7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17240b4010 0x7f17240ce780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.048+0000 7f172aae7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17240d14b0 0x7f17240ab580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.048+0000 7f172aae7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1724163a10 0x7f17240abac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.048+0000 7f172aae7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f17240b8210 con 0x7f1724163a10 2024-05-20T18:37:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.048+0000 7f172aae7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f17240d0040 con 0x7f17240b4010 2024-05-20T18:37:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.048+0000 7f172aae7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f17240b8390 con 0x7f17240d14b0 2024-05-20T18:37:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.049+0000 7f1729084640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1724163a10 0x7f17240abac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.049+0000 7f1729084640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1724163a10 0x7f17240abac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48866/0 (socket says 172.21.15.5:48866) 2024-05-20T18:37:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.049+0000 7f1729084640 1 -- 172.21.15.5:0/3048080403 learned_addr learned my addr 172.21.15.5:0/3048080403 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.049+0000 7f1728883640 1 --2- 172.21.15.5:0/3048080403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17240b4010 0x7f17240ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.049+0000 7f1723fff640 1 --2- 172.21.15.5:0/3048080403 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17240d14b0 0x7f17240ab580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.049+0000 7f1729084640 1 -- 172.21.15.5:0/3048080403 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17240d14b0 msgr2=0x7f17240ab580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.049+0000 7f1729084640 1 --2- 172.21.15.5:0/3048080403 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17240d14b0 0x7f17240ab580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.049+0000 7f1729084640 1 -- 172.21.15.5:0/3048080403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17240b4010 msgr2=0x7f17240ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.049+0000 7f1729084640 1 --2- 172.21.15.5:0/3048080403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17240b4010 0x7f17240ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.049+0000 7f1729084640 1 -- 172.21.15.5:0/3048080403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17240b2680 con 0x7f1724163a10 2024-05-20T18:37:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.049+0000 7f1729084640 1 --2- 172.21.15.5:0/3048080403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1724163a10 0x7f17240abac0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f171c00c780 tx=0x7f171c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.049+0000 7f17217fa640 1 -- 172.21.15.5:0/3048080403 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171c003b70 con 0x7f1724163a10 2024-05-20T18:37:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.049+0000 7f17217fa640 1 -- 172.21.15.5:0/3048080403 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f171c003d30 con 0x7f1724163a10 2024-05-20T18:37:00.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.049+0000 7f172aae7640 1 -- 172.21.15.5:0/3048080403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17240b2960 con 0x7f1724163a10 2024-05-20T18:37:00.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.049+0000 7f17217fa640 1 -- 172.21.15.5:0/3048080403 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171c0052b0 con 0x7f1724163a10 2024-05-20T18:37:00.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.049+0000 7f172aae7640 1 -- 172.21.15.5:0/3048080403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17240b2f20 con 0x7f1724163a10 2024-05-20T18:37:00.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.050+0000 7f172aae7640 1 -- 172.21.15.5:0/3048080403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1724075b00 con 0x7f1724163a10 2024-05-20T18:37:00.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.051+0000 7f17217fa640 1 -- 172.21.15.5:0/3048080403 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f171c005450 con 0x7f1724163a10 2024-05-20T18:37:00.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.051+0000 7f17217fa640 1 --2- 172.21.15.5:0/3048080403 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f170806be60 0x7f170806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:00.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.052+0000 7f17217fa640 1 -- 172.21.15.5:0/3048080403 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6331+0+0 (secure 0 0 0) 0x7f171c08aef0 con 0x7f1724163a10 2024-05-20T18:37:00.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.052+0000 7f1728883640 1 --2- 172.21.15.5:0/3048080403 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f170806be60 0x7f170806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:00.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.052+0000 7f1728883640 1 --2- 172.21.15.5:0/3048080403 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f170806be60 0x7f170806e370 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1718034000 tx=0x7f171802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:00.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.056+0000 7f17217fa640 1 -- 172.21.15.5:0/3048080403 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f171c058ef0 con 0x7f1724163a10 2024-05-20T18:37:00.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.277+0000 7f172aae7640 1 -- 172.21.15.5:0/3048080403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17240ae560 con 0x7f1724163a10 2024-05-20T18:37:00.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.277+0000 7f17217fa640 1 -- 172.21.15.5:0/3048080403 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5161 (secure 0 0 0) 0x7f171c05d1f0 con 0x7f1724163a10 2024-05-20T18:37:00.279 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:00.279 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6758,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/4163145158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":4163145158},{"type":"v1","addr":"172.21.15.63:6834","nonce":4163145158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7528,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3049443532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3049443532},{"type":"v1","addr":"172.21.15.5:6837","nonce":3049443532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7828,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/4180308082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":4180308082},{"type":"v1","addr":"172.21.15.63:6835","nonce":4180308082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":8850,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/888311173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":888311173},{"type":"v1","addr":"172.21.15.5:6835","nonce":888311173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":89,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:35:30.778930+0000","modified":"2024-05-20T18:36:57.897515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-20T18:37:00.279 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 90 2024-05-20T18:37:00.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.279+0000 7f172aae7640 1 -- 172.21.15.5:0/3048080403 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f170806be60 msgr2=0x7f170806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:00.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.279+0000 7f172aae7640 1 --2- 172.21.15.5:0/3048080403 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f170806be60 0x7f170806e370 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1718034000 tx=0x7f171802e000 comp rx=0 tx=0).stop 2024-05-20T18:37:00.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.279+0000 7f172aae7640 1 -- 172.21.15.5:0/3048080403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1724163a10 msgr2=0x7f17240abac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:00.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.279+0000 7f172aae7640 1 --2- 172.21.15.5:0/3048080403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1724163a10 0x7f17240abac0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f171c00c780 tx=0x7f171c00cc40 comp rx=0 tx=0).stop 2024-05-20T18:37:00.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.279+0000 7f172aae7640 1 -- 172.21.15.5:0/3048080403 shutdown_connections 2024-05-20T18:37:00.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.279+0000 7f172aae7640 1 --2- 172.21.15.5:0/3048080403 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f170806be60 0x7f170806e370 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:00.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.279+0000 7f172aae7640 1 --2- 172.21.15.5:0/3048080403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1724163a10 0x7f17240abac0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:00.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.279+0000 7f172aae7640 1 --2- 172.21.15.5:0/3048080403 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17240d14b0 0x7f17240ab580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:00.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.279+0000 7f172aae7640 1 --2- 172.21.15.5:0/3048080403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17240b4010 0x7f17240ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:00.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.279+0000 7f172aae7640 1 -- 172.21.15.5:0/3048080403 >> 172.21.15.5:0/3048080403 conn(0x7f172406d240 msgr2=0x7f172406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:00.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.279+0000 7f172aae7640 1 -- 172.21.15.5:0/3048080403 shutdown_connections 2024-05-20T18:37:00.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.279+0000 7f172aae7640 1 -- 172.21.15.5:0/3048080403 wait complete. 2024-05-20T18:37:00.292 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:37:00.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.406+0000 7fdfc42f7640 1 Processor -- start 2024-05-20T18:37:00.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.406+0000 7fdfc42f7640 1 -- start start 2024-05-20T18:37:00.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.407+0000 7fdfc42f7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fdfbc06bc30 0x7fdfbc167be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:00.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.407+0000 7fdfc42f7640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfbc06def0 con 0x7fdfbc15def0 2024-05-20T18:37:00.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.407+0000 7fdfc42f7640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfbc160c20 con 0x7fdfbc161700 2024-05-20T18:37:00.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.407+0000 7fdfc42f7640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fdfbc160f00 con 0x7fdfbc06bc30 2024-05-20T18:37:00.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.407+0000 7fdfc1892640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fdfbc15def0 0x7fdfbc06b520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51774/0 (socket says 172.21.15.5:51774) 2024-05-20T18:37:00.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.407+0000 7fdfc1892640 1 -- 172.21.15.5:0/3253062987 learned_addr learned my addr 172.21.15.5:0/3253062987 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:00.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.408+0000 7fdfc1091640 1 -- 172.21.15.5:0/3253062987 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2737734242 0 0) 0x7fdfbc06def0 con 0x7fdfbc15def0 2024-05-20T18:37:00.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.408+0000 7fdfc2894640 1 --2- 172.21.15.5:0/3253062987 >> v2:172.21.15.63:3300/0 conn(0x7fdfbc06bc30 0x7fdfbc167be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:00.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.408+0000 7fdfc1091640 1 -- 172.21.15.5:0/3253062987 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdfa4003520 con 0x7fdfbc15def0 2024-05-20T18:37:00.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.408+0000 7fdfc1091640 1 -- 172.21.15.5:0/3253062987 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2517929814 0 0) 0x7fdfa4003520 con 0x7fdfbc15def0 2024-05-20T18:37:00.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.408+0000 7fdfc1091640 1 -- 172.21.15.5:0/3253062987 >> v2:172.21.15.63:3300/0 conn(0x7fdfbc06bc30 msgr2=0x7fdfbc167be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:00.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.408+0000 7fdfc1091640 1 --2- 172.21.15.5:0/3253062987 >> v2:172.21.15.63:3300/0 conn(0x7fdfbc06bc30 0x7fdfbc167be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:00.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.408+0000 7fdfc1091640 1 -- 172.21.15.5:0/3253062987 >> v1:172.21.15.63:6789/0 conn(0x7fdfbc161700 legacy=0x7fdfbc06d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:00.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.408+0000 7fdfc1091640 1 -- 172.21.15.5:0/3253062987 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfbc1611e0 con 0x7fdfbc15def0 2024-05-20T18:37:00.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.408+0000 7fdfc1091640 1 -- 172.21.15.5:0/3253062987 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7fdfb0001670 con 0x7fdfbc15def0 2024-05-20T18:37:00.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.408+0000 7fdfc1091640 1 -- 172.21.15.5:0/3253062987 >> v1:172.21.15.5:6789/0 conn(0x7fdfbc15def0 legacy=0x7fdfbc06b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:00.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.408+0000 7fdfc1091640 1 --2- 172.21.15.5:0/3253062987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfa40039b0 0x7fdfa4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:00.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.408+0000 7fdfc1091640 1 --2- 172.21.15.5:0/3253062987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdfa4004800 0x7fdfa4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:00.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.408+0000 7fdfc1091640 1 --2- 172.21.15.5:0/3253062987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfa40051c0 0x7fdfa400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:00.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.408+0000 7fdfc1091640 1 -- 172.21.15.5:0/3253062987 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc160f00 con 0x7fdfa40051c0 2024-05-20T18:37:00.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.408+0000 7fdfc1091640 1 -- 172.21.15.5:0/3253062987 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa400c160 con 0x7fdfa40039b0 2024-05-20T18:37:00.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.408+0000 7fdfc1091640 1 -- 172.21.15.5:0/3253062987 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdfa400c360 con 0x7fdfa4004800 2024-05-20T18:37:00.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.408+0000 7fdfc2894640 1 --2- 172.21.15.5:0/3253062987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfa40051c0 0x7fdfa400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:00.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.408+0000 7fdfc2093640 1 --2- 172.21.15.5:0/3253062987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfa40039b0 0x7fdfa4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:00.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.408+0000 7fdfc1892640 1 --2- 172.21.15.5:0/3253062987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdfa4004800 0x7fdfa4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:00.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.409+0000 7fdfc2894640 1 -- 172.21.15.5:0/3253062987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdfa4004800 msgr2=0x7fdfa4004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:00.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.409+0000 7fdfc2894640 1 --2- 172.21.15.5:0/3253062987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdfa4004800 0x7fdfa4004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:00.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.409+0000 7fdfc2894640 1 -- 172.21.15.5:0/3253062987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfa40039b0 msgr2=0x7fdfa4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:00.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.409+0000 7fdfc2894640 1 --2- 172.21.15.5:0/3253062987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfa40039b0 0x7fdfa4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:00.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.409+0000 7fdfc2894640 1 -- 172.21.15.5:0/3253062987 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa4003520 con 0x7fdfa40051c0 2024-05-20T18:37:00.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.409+0000 7fdfc2894640 1 -- 172.21.15.5:0/3253062987 reap_dead start 2024-05-20T18:37:00.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.409+0000 7fdfc2894640 1 --2- 172.21.15.5:0/3253062987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfa40051c0 0x7fdfa400b9c0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4009ac0 tx=0x7fdfa4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6ae878c9de8673 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:00.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.409+0000 7fdfc1091640 1 -- 172.21.15.5:0/3253062987 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4009690 con 0x7fdfa40051c0 2024-05-20T18:37:00.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.409+0000 7fdfc1091640 1 -- 172.21.15.5:0/3253062987 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfb4004020 con 0x7fdfa40051c0 2024-05-20T18:37:00.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.409+0000 7fdfc1091640 1 -- 172.21.15.5:0/3253062987 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4004410 con 0x7fdfa40051c0 2024-05-20T18:37:00.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.410+0000 7fdfc42f7640 1 -- 172.21.15.5:0/3253062987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfa40051c0 msgr2=0x7fdfa400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:00.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.410+0000 7fdfc42f7640 1 --2- 172.21.15.5:0/3253062987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfa40051c0 0x7fdfa400b9c0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4009ac0 tx=0x7fdfa4005830 comp rx=0 tx=0).stop 2024-05-20T18:37:00.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.410+0000 7fdfc42f7640 1 -- 172.21.15.5:0/3253062987 shutdown_connections 2024-05-20T18:37:00.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.410+0000 7fdfc42f7640 1 --2- 172.21.15.5:0/3253062987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfa40051c0 0x7fdfa400b9c0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:00.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.410+0000 7fdfc42f7640 1 -- 172.21.15.5:0/3253062987 >> 172.21.15.5:0/3253062987 conn(0x7fdfbc0b6620 msgr2=0x7fdfbc0b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:00.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.410+0000 7fdfc42f7640 1 -- 172.21.15.5:0/3253062987 shutdown_connections 2024-05-20T18:37:00.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.410+0000 7fdfc42f7640 1 -- 172.21.15.5:0/3253062987 wait complete. 2024-05-20T18:37:00.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.412+0000 7fdfc42f7640 1 Processor -- start 2024-05-20T18:37:00.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.412+0000 7fdfc42f7640 1 -- start start 2024-05-20T18:37:00.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.413+0000 7fdfc42f7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfa40051c0 0x7fdfbc070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:00.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.413+0000 7fdfc42f7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdfbc0709c0 0x7fdfbc14f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:00.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.413+0000 7fdfc42f7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfbc14ff20 0x7fdfbc14f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:00.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.413+0000 7fdfc42f7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc06b6a0 con 0x7fdfbc14ff20 2024-05-20T18:37:00.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.413+0000 7fdfc42f7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc06b520 con 0x7fdfa40051c0 2024-05-20T18:37:00.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.413+0000 7fdfc42f7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdfbc06b820 con 0x7fdfbc0709c0 2024-05-20T18:37:00.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.414+0000 7fdfc2894640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfbc14ff20 0x7fdfbc14f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:00.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.414+0000 7fdfc2894640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfbc14ff20 0x7fdfbc14f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48874/0 (socket says 172.21.15.5:48874) 2024-05-20T18:37:00.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.414+0000 7fdfc2894640 1 -- 172.21.15.5:0/2889671866 learned_addr learned my addr 172.21.15.5:0/2889671866 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:00.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.414+0000 7fdfc2093640 1 --2- 172.21.15.5:0/2889671866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfa40051c0 0x7fdfbc070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:00.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.414+0000 7fdfc1892640 1 --2- 172.21.15.5:0/2889671866 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdfbc0709c0 0x7fdfbc14f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:00.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.414+0000 7fdfc2894640 1 -- 172.21.15.5:0/2889671866 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdfbc0709c0 msgr2=0x7fdfbc14f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:00.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.414+0000 7fdfc2894640 1 --2- 172.21.15.5:0/2889671866 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdfbc0709c0 0x7fdfbc14f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:00.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.414+0000 7fdfc2894640 1 -- 172.21.15.5:0/2889671866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfa40051c0 msgr2=0x7fdfbc070480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:00.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.414+0000 7fdfc2894640 1 --2- 172.21.15.5:0/2889671866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfa40051c0 0x7fdfbc070480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:00.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.414+0000 7fdfc2894640 1 -- 172.21.15.5:0/2889671866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfbc1523d0 con 0x7fdfbc14ff20 2024-05-20T18:37:00.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.414+0000 7fdfc2894640 1 --2- 172.21.15.5:0/2889671866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfbc14ff20 0x7fdfbc14f760 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fdfb402ab10 tx=0x7fdfb4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:00.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.414+0000 7fdfbaffd640 1 -- 172.21.15.5:0/2889671866 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb400f040 con 0x7fdfbc14ff20 2024-05-20T18:37:00.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.414+0000 7fdfbaffd640 1 -- 172.21.15.5:0/2889671866 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfb4040070 con 0x7fdfbc14ff20 2024-05-20T18:37:00.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.414+0000 7fdfbaffd640 1 -- 172.21.15.5:0/2889671866 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb403a6b0 con 0x7fdfbc14ff20 2024-05-20T18:37:00.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.414+0000 7fdfc42f7640 1 -- 172.21.15.5:0/2889671866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfbc1526b0 con 0x7fdfbc14ff20 2024-05-20T18:37:00.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.414+0000 7fdfc42f7640 1 -- 172.21.15.5:0/2889671866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfbc1520e0 con 0x7fdfbc14ff20 2024-05-20T18:37:00.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.415+0000 7fdfc42f7640 1 -- 172.21.15.5:0/2889671866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfbc075b00 con 0x7fdfbc14ff20 2024-05-20T18:37:00.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.416+0000 7fdfbaffd640 1 -- 172.21.15.5:0/2889671866 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdfb403a850 con 0x7fdfbc14ff20 2024-05-20T18:37:00.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.416+0000 7fdfbaffd640 1 --2- 172.21.15.5:0/2889671866 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdfa806be30 0x7fdfa806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:00.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.417+0000 7fdfbaffd640 1 -- 172.21.15.5:0/2889671866 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6331+0+0 (secure 0 0 0) 0x7fdfb40ab3d0 con 0x7fdfbc14ff20 2024-05-20T18:37:00.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.417+0000 7fdfc2093640 1 --2- 172.21.15.5:0/2889671866 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdfa806be30 0x7fdfa806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:00.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.417+0000 7fdfc2093640 1 --2- 172.21.15.5:0/2889671866 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdfa806be30 0x7fdfa806e340 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fdfac000f80 tx=0x7fdfac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:00.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.421+0000 7fdfbaffd640 1 -- 172.21.15.5:0/2889671866 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fdfb4079390 con 0x7fdfbc14ff20 2024-05-20T18:37:00.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.604+0000 7fdfc42f7640 1 -- 172.21.15.5:0/2889671866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdfbc1f7cd0 con 0x7fdfbc14ff20 2024-05-20T18:37:00.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.606+0000 7fdfbaffd640 1 -- 172.21.15.5:0/2889671866 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+17118 (secure 0 0 0) 0x7fdfb407d690 con 0x7fdfbc14ff20 2024-05-20T18:37:00.607 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:00.607 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":109,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:36:57.897296+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:35:29.146063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-20T18:35:29.754813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.6","mappings":[{"from":5,"to":4}]},{"pgid":"16.1c","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6835/3552410668":"2024-05-21T18:36:57.897254+0000","172.21.15.5:6834/3552410668":"2024-05-21T18:36:57.897254+0000","172.21.15.63:6835/2054897241":"2024-05-21T18:36:21.346248+0000","172.21.15.63:6833/2054897241":"2024-05-21T18:36:21.346248+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:37:00.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.608+0000 7fdfc42f7640 1 -- 172.21.15.5:0/2889671866 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdfa806be30 msgr2=0x7fdfa806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:00.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.608+0000 7fdfc42f7640 1 --2- 172.21.15.5:0/2889671866 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdfa806be30 0x7fdfa806e340 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fdfac000f80 tx=0x7fdfac009040 comp rx=0 tx=0).stop 2024-05-20T18:37:00.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.608+0000 7fdfc42f7640 1 -- 172.21.15.5:0/2889671866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfbc14ff20 msgr2=0x7fdfbc14f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:00.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.608+0000 7fdfc42f7640 1 --2- 172.21.15.5:0/2889671866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfbc14ff20 0x7fdfbc14f760 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fdfb402ab10 tx=0x7fdfb4004730 comp rx=0 tx=0).stop 2024-05-20T18:37:00.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.608+0000 7fdfc42f7640 1 -- 172.21.15.5:0/2889671866 shutdown_connections 2024-05-20T18:37:00.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.608+0000 7fdfc42f7640 1 --2- 172.21.15.5:0/2889671866 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdfa806be30 0x7fdfa806e340 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:00.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.608+0000 7fdfc42f7640 1 --2- 172.21.15.5:0/2889671866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfbc14ff20 0x7fdfbc14f760 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:00.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.608+0000 7fdfc42f7640 1 --2- 172.21.15.5:0/2889671866 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdfbc0709c0 0x7fdfbc14f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:00.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.608+0000 7fdfc42f7640 1 --2- 172.21.15.5:0/2889671866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfa40051c0 0x7fdfbc070480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:00.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.608+0000 7fdfc42f7640 1 -- 172.21.15.5:0/2889671866 >> 172.21.15.5:0/2889671866 conn(0x7fdfbc0b6620 msgr2=0x7fdfbc09a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:00.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.608+0000 7fdfc42f7640 1 -- 172.21.15.5:0/2889671866 shutdown_connections 2024-05-20T18:37:00.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.608+0000 7fdfc42f7640 1 -- 172.21.15.5:0/2889671866 wait complete. 2024-05-20T18:37:00.621 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-20T18:37:00.621 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:37:00.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.735+0000 7f2aa07c7640 1 Processor -- start 2024-05-20T18:37:00.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.735+0000 7f2aa07c7640 1 -- start start 2024-05-20T18:37:00.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.737+0000 7f2aa07c7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2a9806c630 0x7f2a9806ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.737+0000 7f2aa07c7640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2a980af5c0 0x7f2a980af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.737+0000 7f2aa07c7640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a9806d320 con 0x7f2a980d17c0 2024-05-20T18:37:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.737+0000 7f2aa07c7640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f2a9806df30 con 0x7f2a980af5c0 2024-05-20T18:37:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.737+0000 7f2aa07c7640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2a9806d600 con 0x7f2a9806c630 2024-05-20T18:37:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.737+0000 7f2a9ed64640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2a980af5c0 0x7f2a980af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.737+0000 7f2a9e563640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2a980d17c0 0x7f2a980d1bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51788/0 (socket says 172.21.15.5:51788) 2024-05-20T18:37:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.737+0000 7f2a9e563640 1 -- 172.21.15.5:0/1995238856 learned_addr learned my addr 172.21.15.5:0/1995238856 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.737+0000 7f2a9dd62640 1 --2- 172.21.15.5:0/1995238856 >> v2:172.21.15.63:3300/0 conn(0x7f2a9806c630 0x7f2a9806ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.737+0000 7f2a9ed64640 1 -- 172.21.15.5:0/1995238856 >> v2:172.21.15.63:3300/0 conn(0x7f2a9806c630 msgr2=0x7f2a9806ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.737+0000 7f2a9ed64640 1 --2- 172.21.15.5:0/1995238856 >> v2:172.21.15.63:3300/0 conn(0x7f2a9806c630 0x7f2a9806ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.737+0000 7f2a9ed64640 1 -- 172.21.15.5:0/1995238856 >> v1:172.21.15.5:6789/0 conn(0x7f2a980d17c0 legacy=0x7f2a980d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.737+0000 7f2a9ed64640 1 -- 172.21.15.5:0/1995238856 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9806d8e0 con 0x7f2a980af5c0 2024-05-20T18:37:00.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.737+0000 7f2a9d561640 1 -- 172.21.15.5:0/1995238856 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1698772103 0 0) 0x7f2a9806d320 con 0x7f2a980d17c0 2024-05-20T18:37:00.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.738+0000 7f2a9ed64640 1 --2- 172.21.15.5:0/1995238856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a980af5c0 0x7f2a980af9d0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f2a9815f8e0 tx=0x7f2a9002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eee9b22ea5ea2a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:00.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.738+0000 7f2a9d561640 1 -- 172.21.15.5:0/1995238856 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a90037070 con 0x7f2a980af5c0 2024-05-20T18:37:00.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.738+0000 7f2a9d561640 1 -- 172.21.15.5:0/1995238856 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a90032440 con 0x7f2a980af5c0 2024-05-20T18:37:00.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.739+0000 7f2a9d561640 1 -- 172.21.15.5:0/1995238856 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a90032810 con 0x7f2a980af5c0 2024-05-20T18:37:00.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.739+0000 7f2aa07c7640 1 -- 172.21.15.5:0/1995238856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a980af5c0 msgr2=0x7f2a980af9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:00.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.739+0000 7f2aa07c7640 1 --2- 172.21.15.5:0/1995238856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a980af5c0 0x7f2a980af9d0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f2a9815f8e0 tx=0x7f2a9002a1a0 comp rx=0 tx=0).stop 2024-05-20T18:37:00.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.739+0000 7f2aa07c7640 1 -- 172.21.15.5:0/1995238856 shutdown_connections 2024-05-20T18:37:00.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.739+0000 7f2aa07c7640 1 --2- 172.21.15.5:0/1995238856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a980af5c0 0x7f2a980af9d0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:00.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.739+0000 7f2aa07c7640 1 --2- 172.21.15.5:0/1995238856 >> v2:172.21.15.63:3300/0 conn(0x7f2a9806c630 0x7f2a9806ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:00.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.739+0000 7f2aa07c7640 1 -- 172.21.15.5:0/1995238856 >> 172.21.15.5:0/1995238856 conn(0x7f2a980b6620 msgr2=0x7f2a980b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:00.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.739+0000 7f2aa07c7640 1 -- 172.21.15.5:0/1995238856 shutdown_connections 2024-05-20T18:37:00.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.739+0000 7f2aa07c7640 1 -- 172.21.15.5:0/1995238856 wait complete. 2024-05-20T18:37:00.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.741+0000 7f2aa07c7640 1 Processor -- start 2024-05-20T18:37:00.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.741+0000 7f2aa07c7640 1 -- start start 2024-05-20T18:37:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.742+0000 7f2aa07c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a9806c630 0x7f2a980cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.742+0000 7f2aa07c7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a980af5c0 0x7f2a98140f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.742+0000 7f2aa07c7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a980d17c0 0x7f2a98141470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.743+0000 7f2aa07c7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9806d100 con 0x7f2a9806c630 2024-05-20T18:37:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.743+0000 7f2aa07c7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9806cf80 con 0x7f2a980af5c0 2024-05-20T18:37:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.743+0000 7f2aa07c7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2a9815c010 con 0x7f2a980d17c0 2024-05-20T18:37:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.743+0000 7f2a9e563640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a9806c630 0x7f2a980cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.743+0000 7f2a9e563640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a9806c630 0x7f2a980cffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48890/0 (socket says 172.21.15.5:48890) 2024-05-20T18:37:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.743+0000 7f2a9e563640 1 -- 172.21.15.5:0/2957328815 learned_addr learned my addr 172.21.15.5:0/2957328815 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.743+0000 7f2a9ed64640 1 --2- 172.21.15.5:0/2957328815 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a980d17c0 0x7f2a98141470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.743+0000 7f2a9dd62640 1 --2- 172.21.15.5:0/2957328815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a980af5c0 0x7f2a98140f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.743+0000 7f2a9e563640 1 -- 172.21.15.5:0/2957328815 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a980d17c0 msgr2=0x7f2a98141470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.743+0000 7f2a9e563640 1 --2- 172.21.15.5:0/2957328815 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a980d17c0 0x7f2a98141470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.743+0000 7f2a9e563640 1 -- 172.21.15.5:0/2957328815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a980af5c0 msgr2=0x7f2a98140f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.743+0000 7f2a9e563640 1 --2- 172.21.15.5:0/2957328815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a980af5c0 0x7f2a98140f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.743+0000 7f2a9e563640 1 -- 172.21.15.5:0/2957328815 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9813f690 con 0x7f2a9806c630 2024-05-20T18:37:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.743+0000 7f2a9ed64640 1 --2- 172.21.15.5:0/2957328815 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a980d17c0 0x7f2a98141470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.743+0000 7f2a9e563640 1 --2- 172.21.15.5:0/2957328815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a9806c630 0x7f2a980cffa0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800c7e0 tx=0x7f2a8800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.743+0000 7f2a96ffd640 1 -- 172.21.15.5:0/2957328815 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a88018070 con 0x7f2a9806c630 2024-05-20T18:37:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.743+0000 7f2a96ffd640 1 -- 172.21.15.5:0/2957328815 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a8800a640 con 0x7f2a9806c630 2024-05-20T18:37:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.743+0000 7f2aa07c7640 1 -- 172.21.15.5:0/2957328815 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a9813fb90 con 0x7f2a9806c630 2024-05-20T18:37:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.743+0000 7f2a96ffd640 1 -- 172.21.15.5:0/2957328815 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8800aa30 con 0x7f2a9806c630 2024-05-20T18:37:00.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.743+0000 7f2aa07c7640 1 -- 172.21.15.5:0/2957328815 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a98140150 con 0x7f2a9806c630 2024-05-20T18:37:00.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.745+0000 7f2aa07c7640 1 -- 172.21.15.5:0/2957328815 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a98075b00 con 0x7f2a9806c630 2024-05-20T18:37:00.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.745+0000 7f2a96ffd640 1 -- 172.21.15.5:0/2957328815 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2a8801b070 con 0x7f2a9806c630 2024-05-20T18:37:00.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.746+0000 7f2a96ffd640 1 --2- 172.21.15.5:0/2957328815 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2a8406be60 0x7f2a8406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:00.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.746+0000 7f2a96ffd640 1 -- 172.21.15.5:0/2957328815 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6331+0+0 (secure 0 0 0) 0x7f2a8808a760 con 0x7f2a9806c630 2024-05-20T18:37:00.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.746+0000 7f2a9dd62640 1 --2- 172.21.15.5:0/2957328815 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2a8406be60 0x7f2a8406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:00.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.746+0000 7f2a9dd62640 1 --2- 172.21.15.5:0/2957328815 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2a8406be60 0x7f2a8406e370 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c006fd0 tx=0x7f2a8c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:00.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.750+0000 7f2a96ffd640 1 -- 172.21.15.5:0/2957328815 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2a88058e10 con 0x7f2a9806c630 2024-05-20T18:37:00.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.974+0000 7f2aa07c7640 1 -- 172.21.15.5:0/2957328815 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a980b3bc0 con 0x7f2a9806c630 2024-05-20T18:37:00.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.974+0000 7f2a96ffd640 1 -- 172.21.15.5:0/2957328815 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5161 (secure 0 0 0) 0x7f2a88058710 con 0x7f2a9806c630 2024-05-20T18:37:00.977 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:00.977 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6758,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/4163145158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":4163145158},{"type":"v1","addr":"172.21.15.63:6834","nonce":4163145158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7528,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3049443532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3049443532},{"type":"v1","addr":"172.21.15.5:6837","nonce":3049443532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7828,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/4180308082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":4180308082},{"type":"v1","addr":"172.21.15.63:6835","nonce":4180308082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":8850,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/888311173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":888311173},{"type":"v1","addr":"172.21.15.5:6835","nonce":888311173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":89,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:35:30.778930+0000","modified":"2024-05-20T18:36:57.897515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-20T18:37:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 90 2024-05-20T18:37:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.976+0000 7f2aa07c7640 1 -- 172.21.15.5:0/2957328815 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2a8406be60 msgr2=0x7f2a8406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.976+0000 7f2aa07c7640 1 --2- 172.21.15.5:0/2957328815 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2a8406be60 0x7f2a8406e370 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c006fd0 tx=0x7f2a8c008040 comp rx=0 tx=0).stop 2024-05-20T18:37:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.976+0000 7f2aa07c7640 1 -- 172.21.15.5:0/2957328815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a9806c630 msgr2=0x7f2a980cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.976+0000 7f2aa07c7640 1 --2- 172.21.15.5:0/2957328815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a9806c630 0x7f2a980cffa0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800c7e0 tx=0x7f2a8800cca0 comp rx=0 tx=0).stop 2024-05-20T18:37:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.976+0000 7f2aa07c7640 1 -- 172.21.15.5:0/2957328815 shutdown_connections 2024-05-20T18:37:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.976+0000 7f2aa07c7640 1 --2- 172.21.15.5:0/2957328815 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2a8406be60 0x7f2a8406e370 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.976+0000 7f2aa07c7640 1 --2- 172.21.15.5:0/2957328815 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a980d17c0 0x7f2a98141470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.976+0000 7f2aa07c7640 1 --2- 172.21.15.5:0/2957328815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a980af5c0 0x7f2a98140f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.976+0000 7f2aa07c7640 1 --2- 172.21.15.5:0/2957328815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a9806c630 0x7f2a980cffa0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.976+0000 7f2aa07c7640 1 -- 172.21.15.5:0/2957328815 >> 172.21.15.5:0/2957328815 conn(0x7f2a980b6620 msgr2=0x7f2a980d1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.977+0000 7f2aa07c7640 1 -- 172.21.15.5:0/2957328815 shutdown_connections 2024-05-20T18:37:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:00.977+0000 7f2aa07c7640 1 -- 172.21.15.5:0/2957328815 wait complete. 2024-05-20T18:37:00.989 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:37:01.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.102+0000 7f2253ede640 1 Processor -- start 2024-05-20T18:37:01.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.102+0000 7f2253ede640 1 -- start start 2024-05-20T18:37:01.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.104+0000 7f2253ede640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f224c15f280 0x7f224c06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:01.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.104+0000 7f2253ede640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f224c06c120 0x7f224c0d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.104+0000 7f2253ede640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f224c0b1b80 con 0x7f224c06ca80 2024-05-20T18:37:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.104+0000 7f2253ede640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f224c06b750 con 0x7f224c06c120 2024-05-20T18:37:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.104+0000 7f2253ede640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f224c15f690 con 0x7f224c15f280 2024-05-20T18:37:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.104+0000 7f225247b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f224c06c120 0x7f224c0d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.104+0000 7f2251c7a640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f224c06ca80 0x7f224c0ae320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51794/0 (socket says 172.21.15.5:51794) 2024-05-20T18:37:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.104+0000 7f2251c7a640 1 -- 172.21.15.5:0/902093253 learned_addr learned my addr 172.21.15.5:0/902093253 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.104+0000 7f2251479640 1 --2- 172.21.15.5:0/902093253 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f224c15f280 0x7f224c06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:01.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.104+0000 7f225247b640 1 -- 172.21.15.5:0/902093253 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f224c15f280 msgr2=0x7f224c06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.104+0000 7f225247b640 1 --2- 172.21.15.5:0/902093253 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f224c15f280 0x7f224c06bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.104+0000 7f225247b640 1 -- 172.21.15.5:0/902093253 >> v1:172.21.15.5:6789/0 conn(0x7f224c06ca80 legacy=0x7f224c0ae320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.104+0000 7f225247b640 1 -- 172.21.15.5:0/902093253 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224c0b12e0 con 0x7f224c06c120 2024-05-20T18:37:01.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.104+0000 7f2250c78640 1 -- 172.21.15.5:0/902093253 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1375515560 0 0) 0x7f2248001110 con 0x7f224c06ca80 2024-05-20T18:37:01.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.105+0000 7f225247b640 1 --2- 172.21.15.5:0/902093253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f224c06c120 0x7f224c0d0c10 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f2244009d90 tx=0x7f224402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ac5e474e144ba0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:01.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.105+0000 7f2250c78640 1 -- 172.21.15.5:0/902093253 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2244007740 con 0x7f224c06c120 2024-05-20T18:37:01.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.105+0000 7f2250c78640 1 -- 172.21.15.5:0/902093253 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22440078e0 con 0x7f224c06c120 2024-05-20T18:37:01.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.105+0000 7f2250c78640 1 -- 172.21.15.5:0/902093253 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2244007cb0 con 0x7f224c06c120 2024-05-20T18:37:01.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.106+0000 7f2253ede640 1 -- 172.21.15.5:0/902093253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f224c06c120 msgr2=0x7f224c0d0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:01.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.106+0000 7f2253ede640 1 --2- 172.21.15.5:0/902093253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f224c06c120 0x7f224c0d0c10 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f2244009d90 tx=0x7f224402a660 comp rx=0 tx=0).stop 2024-05-20T18:37:01.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.106+0000 7f2253ede640 1 -- 172.21.15.5:0/902093253 shutdown_connections 2024-05-20T18:37:01.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.106+0000 7f2253ede640 1 --2- 172.21.15.5:0/902093253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f224c06c120 0x7f224c0d0c10 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:01.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.106+0000 7f2253ede640 1 --2- 172.21.15.5:0/902093253 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f224c15f280 0x7f224c06bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:01.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.106+0000 7f2253ede640 1 -- 172.21.15.5:0/902093253 >> 172.21.15.5:0/902093253 conn(0x7f224c072ed0 msgr2=0x7f224c0b6080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:01.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.106+0000 7f2253ede640 1 -- 172.21.15.5:0/902093253 shutdown_connections 2024-05-20T18:37:01.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.106+0000 7f2253ede640 1 -- 172.21.15.5:0/902093253 wait complete. 2024-05-20T18:37:01.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.108+0000 7f2253ede640 1 Processor -- start 2024-05-20T18:37:01.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.108+0000 7f2253ede640 1 -- start start 2024-05-20T18:37:01.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.109+0000 7f2253ede640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f224c06c120 0x7f224c161f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:01.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.109+0000 7f2253ede640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f224c06ca80 0x7f224c154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:01.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.109+0000 7f2253ede640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f224c15f280 0x7f224c06f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:01.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.109+0000 7f2253ede640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f224c074e00 con 0x7f224c15f280 2024-05-20T18:37:01.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.109+0000 7f2253ede640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f224c0b18a0 con 0x7f224c06ca80 2024-05-20T18:37:01.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.109+0000 7f2253ede640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f224c06c530 con 0x7f224c06c120 2024-05-20T18:37:01.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.109+0000 7f225247b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f224c15f280 0x7f224c06f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:01.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.109+0000 7f225247b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f224c15f280 0x7f224c06f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48908/0 (socket says 172.21.15.5:48908) 2024-05-20T18:37:01.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.109+0000 7f225247b640 1 -- 172.21.15.5:0/3231009661 learned_addr learned my addr 172.21.15.5:0/3231009661 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:01.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.109+0000 7f2251479640 1 --2- 172.21.15.5:0/3231009661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f224c06ca80 0x7f224c154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:01.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.109+0000 7f2251c7a640 1 --2- 172.21.15.5:0/3231009661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f224c06c120 0x7f224c161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:01.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.109+0000 7f225247b640 1 -- 172.21.15.5:0/3231009661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f224c06c120 msgr2=0x7f224c161f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:01.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.109+0000 7f225247b640 1 --2- 172.21.15.5:0/3231009661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f224c06c120 0x7f224c161f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:01.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.109+0000 7f225247b640 1 -- 172.21.15.5:0/3231009661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f224c06ca80 msgr2=0x7f224c154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:01.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.109+0000 7f225247b640 1 --2- 172.21.15.5:0/3231009661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f224c06ca80 0x7f224c154160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:01.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.109+0000 7f225247b640 1 -- 172.21.15.5:0/3231009661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224c1546a0 con 0x7f224c15f280 2024-05-20T18:37:01.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.110+0000 7f225247b640 1 --2- 172.21.15.5:0/3231009661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f224c15f280 0x7f224c06f4b0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f2244009d90 tx=0x7f2244004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:01.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.110+0000 7f22427fc640 1 -- 172.21.15.5:0/3231009661 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224403f070 con 0x7f224c15f280 2024-05-20T18:37:01.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.110+0000 7f22427fc640 1 -- 172.21.15.5:0/3231009661 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2244031bc0 con 0x7f224c15f280 2024-05-20T18:37:01.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.110+0000 7f2253ede640 1 -- 172.21.15.5:0/3231009661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f224c1603c0 con 0x7f224c15f280 2024-05-20T18:37:01.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.110+0000 7f22427fc640 1 -- 172.21.15.5:0/3231009661 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2244043410 con 0x7f224c15f280 2024-05-20T18:37:01.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.110+0000 7f2253ede640 1 -- 172.21.15.5:0/3231009661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f224c160980 con 0x7f224c15f280 2024-05-20T18:37:01.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.111+0000 7f2253ede640 1 -- 172.21.15.5:0/3231009661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f224c075b00 con 0x7f224c15f280 2024-05-20T18:37:01.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.112+0000 7f22427fc640 1 -- 172.21.15.5:0/3231009661 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2244007790 con 0x7f224c15f280 2024-05-20T18:37:01.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.112+0000 7f22427fc640 1 --2- 172.21.15.5:0/3231009661 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f223006be30 0x7f223006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:01.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.112+0000 7f22427fc640 1 -- 172.21.15.5:0/3231009661 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6331+0+0 (secure 0 0 0) 0x7f22440ac620 con 0x7f224c15f280 2024-05-20T18:37:01.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.112+0000 7f2251c7a640 1 --2- 172.21.15.5:0/3231009661 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f223006be30 0x7f223006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:01.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.113+0000 7f2251c7a640 1 --2- 172.21.15.5:0/3231009661 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f223006be30 0x7f223006e340 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2248000f60 tx=0x7f2248007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:01.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.117+0000 7f22427fc640 1 -- 172.21.15.5:0/3231009661 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f224407acd0 con 0x7f224c15f280 2024-05-20T18:37:01.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.338+0000 7f2253ede640 1 -- 172.21.15.5:0/3231009661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f224c0b0eb0 con 0x7f224c15f280 2024-05-20T18:37:01.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.339+0000 7f22427fc640 1 -- 172.21.15.5:0/3231009661 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5161 (secure 0 0 0) 0x7f224407a5d0 con 0x7f224c15f280 2024-05-20T18:37:01.340 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:01.340 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6758,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/4163145158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":4163145158},{"type":"v1","addr":"172.21.15.63:6834","nonce":4163145158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7528,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3049443532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3049443532},{"type":"v1","addr":"172.21.15.5:6837","nonce":3049443532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7828,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/4180308082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":4180308082},{"type":"v1","addr":"172.21.15.63:6835","nonce":4180308082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":8850,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/888311173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":888311173},{"type":"v1","addr":"172.21.15.5:6835","nonce":888311173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":89,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:35:30.778930+0000","modified":"2024-05-20T18:36:57.897515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-20T18:37:01.341 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 90 2024-05-20T18:37:01.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.341+0000 7f2253ede640 1 -- 172.21.15.5:0/3231009661 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f223006be30 msgr2=0x7f223006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:01.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.341+0000 7f2253ede640 1 --2- 172.21.15.5:0/3231009661 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f223006be30 0x7f223006e340 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2248000f60 tx=0x7f2248007c00 comp rx=0 tx=0).stop 2024-05-20T18:37:01.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.341+0000 7f2253ede640 1 -- 172.21.15.5:0/3231009661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f224c15f280 msgr2=0x7f224c06f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:01.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.341+0000 7f2253ede640 1 --2- 172.21.15.5:0/3231009661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f224c15f280 0x7f224c06f4b0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f2244009d90 tx=0x7f2244004a40 comp rx=0 tx=0).stop 2024-05-20T18:37:01.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.341+0000 7f2253ede640 1 -- 172.21.15.5:0/3231009661 shutdown_connections 2024-05-20T18:37:01.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.341+0000 7f2253ede640 1 --2- 172.21.15.5:0/3231009661 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f223006be30 0x7f223006e340 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:01.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.341+0000 7f2253ede640 1 --2- 172.21.15.5:0/3231009661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f224c15f280 0x7f224c06f4b0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:01.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.341+0000 7f2253ede640 1 --2- 172.21.15.5:0/3231009661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f224c06ca80 0x7f224c154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:01.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.341+0000 7f2253ede640 1 --2- 172.21.15.5:0/3231009661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f224c06c120 0x7f224c161f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:01.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.341+0000 7f2253ede640 1 -- 172.21.15.5:0/3231009661 >> 172.21.15.5:0/3231009661 conn(0x7f224c072ed0 msgr2=0x7f224c0af940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:01.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.341+0000 7f2253ede640 1 -- 172.21.15.5:0/3231009661 shutdown_connections 2024-05-20T18:37:01.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.341+0000 7f2253ede640 1 -- 172.21.15.5:0/3231009661 wait complete. 2024-05-20T18:37:01.354 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:37:01.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.467+0000 7fb8512f7640 1 Processor -- start 2024-05-20T18:37:01.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.467+0000 7fb8512f7640 1 -- start start 2024-05-20T18:37:01.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.468+0000 7fb8512f7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb84c162820 0x7fb84c0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:01.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.468+0000 7fb8512f7640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb84c0b3e80 0x7fb84c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:01.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.468+0000 7fb8512f7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb84c06d910 0x7fb84c0d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:01.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.468+0000 7fb8512f7640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fb84c06dd20 con 0x7fb84c0b3e80 2024-05-20T18:37:01.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.469+0000 7fb8512f7640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb84c0d0630 con 0x7fb84c06d910 2024-05-20T18:37:01.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.469+0000 7fb8512f7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb84c0d39c0 con 0x7fb84c162820 2024-05-20T18:37:01.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.469+0000 7fb84a7fc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb84c0b3e80 0x7fb84c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:01.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.469+0000 7fb84a7fc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb84c0b3e80 0x7fb84c06d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48924/0 (socket says 172.21.15.5:48924) 2024-05-20T18:37:01.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.469+0000 7fb84a7fc640 1 -- 172.21.15.5:0/1632569718 learned_addr learned my addr 172.21.15.5:0/1632569718 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:01.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.469+0000 7fb84affd640 1 --2- 172.21.15.5:0/1632569718 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb84c162820 0x7fb84c0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:01.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.469+0000 7fb84b7fe640 1 --2- 172.21.15.5:0/1632569718 >> v2:172.21.15.63:3300/0 conn(0x7fb84c06d910 0x7fb84c0d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:01.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.469+0000 7fb84a7fc640 1 -- 172.21.15.5:0/1632569718 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb84c162820 msgr2=0x7fb84c0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:01.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.469+0000 7fb84a7fc640 1 --2- 172.21.15.5:0/1632569718 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb84c162820 0x7fb84c0b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:01.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.469+0000 7fb84a7fc640 1 -- 172.21.15.5:0/1632569718 >> v2:172.21.15.63:3300/0 conn(0x7fb84c06d910 msgr2=0x7fb84c0d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:01.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.469+0000 7fb84a7fc640 1 --2- 172.21.15.5:0/1632569718 >> v2:172.21.15.63:3300/0 conn(0x7fb84c06d910 0x7fb84c0d27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:01.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.469+0000 7fb84a7fc640 1 -- 172.21.15.5:0/1632569718 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84c0d3ca0 con 0x7fb84c0b3e80 2024-05-20T18:37:01.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.470+0000 7fb84a7fc640 1 --2- 172.21.15.5:0/1632569718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb84c0b3e80 0x7fb84c06d3d0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fb84c06ce40 tx=0x7fb83402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6f1d0afdfecbdd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.470+0000 7fb849ffb640 1 -- 172.21.15.5:0/1632569718 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb834037070 con 0x7fb84c0b3e80 2024-05-20T18:37:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.470+0000 7fb849ffb640 1 -- 172.21.15.5:0/1632569718 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb834032440 con 0x7fb84c0b3e80 2024-05-20T18:37:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.470+0000 7fb849ffb640 1 -- 172.21.15.5:0/1632569718 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb834032810 con 0x7fb84c0b3e80 2024-05-20T18:37:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.471+0000 7fb8512f7640 1 -- 172.21.15.5:0/1632569718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb84c0b3e80 msgr2=0x7fb84c06d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.471+0000 7fb8512f7640 1 --2- 172.21.15.5:0/1632569718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb84c0b3e80 0x7fb84c06d3d0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fb84c06ce40 tx=0x7fb83402a1e0 comp rx=0 tx=0).stop 2024-05-20T18:37:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.471+0000 7fb8512f7640 1 -- 172.21.15.5:0/1632569718 shutdown_connections 2024-05-20T18:37:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.471+0000 7fb8512f7640 1 --2- 172.21.15.5:0/1632569718 >> v2:172.21.15.63:3300/0 conn(0x7fb84c06d910 0x7fb84c0d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.471+0000 7fb8512f7640 1 --2- 172.21.15.5:0/1632569718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb84c0b3e80 0x7fb84c06d3d0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.471+0000 7fb8512f7640 1 --2- 172.21.15.5:0/1632569718 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb84c162820 0x7fb84c0b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.471+0000 7fb8512f7640 1 -- 172.21.15.5:0/1632569718 >> 172.21.15.5:0/1632569718 conn(0x7fb84c0d4bd0 msgr2=0x7fb84c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.471+0000 7fb8512f7640 1 -- 172.21.15.5:0/1632569718 shutdown_connections 2024-05-20T18:37:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.471+0000 7fb8512f7640 1 -- 172.21.15.5:0/1632569718 wait complete. 2024-05-20T18:37:01.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.473+0000 7fb8512f7640 1 Processor -- start 2024-05-20T18:37:01.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.473+0000 7fb8512f7640 1 -- start start 2024-05-20T18:37:01.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.474+0000 7fb8512f7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb84c06d910 0x7fb84c157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:01.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.474+0000 7fb8512f7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb84c0b3e80 0x7fb84c1f5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:01.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.474+0000 7fb8512f7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb84c162820 0x7fb84c158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:01.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.474+0000 7fb8512f7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb84c0d0300 con 0x7fb84c06d910 2024-05-20T18:37:01.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.474+0000 7fb8512f7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb84c0d4260 con 0x7fb84c162820 2024-05-20T18:37:01.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.474+0000 7fb8512f7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb84c0d4780 con 0x7fb84c0b3e80 2024-05-20T18:37:01.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.474+0000 7fb84affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb84c06d910 0x7fb84c157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:01.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.474+0000 7fb84affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb84c06d910 0x7fb84c157760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48926/0 (socket says 172.21.15.5:48926) 2024-05-20T18:37:01.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.474+0000 7fb84affd640 1 -- 172.21.15.5:0/567008476 learned_addr learned my addr 172.21.15.5:0/567008476 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:01.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.474+0000 7fb84b7fe640 1 --2- 172.21.15.5:0/567008476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb84c162820 0x7fb84c158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:01.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.474+0000 7fb84a7fc640 1 --2- 172.21.15.5:0/567008476 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb84c0b3e80 0x7fb84c1f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.475+0000 7fb84affd640 1 -- 172.21.15.5:0/567008476 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb84c0b3e80 msgr2=0x7fb84c1f5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.475+0000 7fb84affd640 1 --2- 172.21.15.5:0/567008476 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb84c0b3e80 0x7fb84c1f5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.475+0000 7fb84affd640 1 -- 172.21.15.5:0/567008476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb84c162820 msgr2=0x7fb84c158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.475+0000 7fb84affd640 1 --2- 172.21.15.5:0/567008476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb84c162820 0x7fb84c158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.475+0000 7fb84affd640 1 -- 172.21.15.5:0/567008476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84c1f9c80 con 0x7fb84c06d910 2024-05-20T18:37:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.475+0000 7fb84b7fe640 1 --2- 172.21.15.5:0/567008476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb84c162820 0x7fb84c158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.475+0000 7fb84affd640 1 --2- 172.21.15.5:0/567008476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb84c06d910 0x7fb84c157760 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fb83c009d10 tx=0x7fb83c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.475+0000 7fb8437fe640 1 -- 172.21.15.5:0/567008476 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c018020 con 0x7fb84c06d910 2024-05-20T18:37:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.475+0000 7fb8437fe640 1 -- 172.21.15.5:0/567008476 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb83c014480 con 0x7fb84c06d910 2024-05-20T18:37:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.475+0000 7fb8512f7640 1 -- 172.21.15.5:0/567008476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb84c156cc0 con 0x7fb84c06d910 2024-05-20T18:37:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.475+0000 7fb8437fe640 1 -- 172.21.15.5:0/567008476 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c014830 con 0x7fb84c06d910 2024-05-20T18:37:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.475+0000 7fb8512f7640 1 -- 172.21.15.5:0/567008476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb84c156f60 con 0x7fb84c06d910 2024-05-20T18:37:01.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.476+0000 7fb8512f7640 1 -- 172.21.15.5:0/567008476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb84c075b00 con 0x7fb84c06d910 2024-05-20T18:37:01.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.477+0000 7fb8437fe640 1 -- 172.21.15.5:0/567008476 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb83c01b020 con 0x7fb84c06d910 2024-05-20T18:37:01.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.477+0000 7fb8437fe640 1 --2- 172.21.15.5:0/567008476 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb82c06be30 0x7fb82c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:01.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.477+0000 7fb8437fe640 1 -- 172.21.15.5:0/567008476 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6331+0+0 (secure 0 0 0) 0x7fb83c08bc80 con 0x7fb84c06d910 2024-05-20T18:37:01.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.477+0000 7fb84a7fc640 1 --2- 172.21.15.5:0/567008476 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb82c06be30 0x7fb82c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:01.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.478+0000 7fb84a7fc640 1 --2- 172.21.15.5:0/567008476 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb82c06be30 0x7fb82c06e340 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb834034000 tx=0x7fb83402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:01.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.482+0000 7fb8437fe640 1 -- 172.21.15.5:0/567008476 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb83c022030 con 0x7fb84c06d910 2024-05-20T18:37:01.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.666+0000 7fb8512f7640 1 -- 172.21.15.5:0/567008476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb84c068a90 con 0x7fb84c06d910 2024-05-20T18:37:01.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.667+0000 7fb8437fe640 1 -- 172.21.15.5:0/567008476 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+17118 (secure 0 0 0) 0x7fb83c010040 con 0x7fb84c06d910 2024-05-20T18:37:01.668 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:01.669 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":109,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:36:57.897296+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:35:29.146063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-20T18:35:29.754813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.6","mappings":[{"from":5,"to":4}]},{"pgid":"16.1c","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6835/3552410668":"2024-05-21T18:36:57.897254+0000","172.21.15.5:6834/3552410668":"2024-05-21T18:36:57.897254+0000","172.21.15.63:6835/2054897241":"2024-05-21T18:36:21.346248+0000","172.21.15.63:6833/2054897241":"2024-05-21T18:36:21.346248+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:37:01.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.669+0000 7fb8512f7640 1 -- 172.21.15.5:0/567008476 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb82c06be30 msgr2=0x7fb82c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.669+0000 7fb8512f7640 1 --2- 172.21.15.5:0/567008476 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb82c06be30 0x7fb82c06e340 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb834034000 tx=0x7fb83402e000 comp rx=0 tx=0).stop 2024-05-20T18:37:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.669+0000 7fb8512f7640 1 -- 172.21.15.5:0/567008476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb84c06d910 msgr2=0x7fb84c157760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.669+0000 7fb8512f7640 1 --2- 172.21.15.5:0/567008476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb84c06d910 0x7fb84c157760 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fb83c009d10 tx=0x7fb83c00c490 comp rx=0 tx=0).stop 2024-05-20T18:37:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.669+0000 7fb8512f7640 1 -- 172.21.15.5:0/567008476 shutdown_connections 2024-05-20T18:37:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.669+0000 7fb8512f7640 1 --2- 172.21.15.5:0/567008476 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb82c06be30 0x7fb82c06e340 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.669+0000 7fb8512f7640 1 --2- 172.21.15.5:0/567008476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb84c162820 0x7fb84c158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.669+0000 7fb8512f7640 1 --2- 172.21.15.5:0/567008476 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb84c0b3e80 0x7fb84c1f5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.669+0000 7fb8512f7640 1 --2- 172.21.15.5:0/567008476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb84c06d910 0x7fb84c157760 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.669+0000 7fb8512f7640 1 -- 172.21.15.5:0/567008476 >> 172.21.15.5:0/567008476 conn(0x7fb84c0d4bd0 msgr2=0x7fb84c0b8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.669+0000 7fb8512f7640 1 -- 172.21.15.5:0/567008476 shutdown_connections 2024-05-20T18:37:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.669+0000 7fb8512f7640 1 -- 172.21.15.5:0/567008476 wait complete. 2024-05-20T18:37:01.682 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-20T18:37:01.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.795+0000 7f002beb8640 1 Processor -- start 2024-05-20T18:37:01.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.796+0000 7f002beb8640 1 -- start start 2024-05-20T18:37:01.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.797+0000 7f002beb8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f002406d3d0 0x7f002406d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:01.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.797+0000 7f002beb8640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f002406dd20 con 0x7f0024162820 2024-05-20T18:37:01.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.797+0000 7f002beb8640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00240d0630 con 0x7f00240b8190 2024-05-20T18:37:01.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.797+0000 7f002beb8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f00240d39c0 con 0x7f002406d3d0 2024-05-20T18:37:01.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.797+0000 7f0029c54640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0024162820 0x7f00240b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51796/0 (socket says 172.21.15.5:51796) 2024-05-20T18:37:01.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.797+0000 7f0029c54640 1 -- 172.21.15.5:0/2462939281 learned_addr learned my addr 172.21.15.5:0/2462939281 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:01.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.797+0000 7f0029453640 1 --2- 172.21.15.5:0/2462939281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f002406d3d0 0x7f002406d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:01.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.797+0000 7f0028c52640 1 -- 172.21.15.5:0/2462939281 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1358565818 0 0) 0x7f0020001110 con 0x7f0024162820 2024-05-20T18:37:01.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.797+0000 7f0028c52640 1 -- 172.21.15.5:0/2462939281 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0010003520 con 0x7f0024162820 2024-05-20T18:37:01.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.798+0000 7f0028c52640 1 -- 172.21.15.5:0/2462939281 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1337036609 0 0) 0x7f00240d0630 con 0x7f00240b8190 2024-05-20T18:37:01.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.798+0000 7f0028c52640 1 -- 172.21.15.5:0/2462939281 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0020001110 con 0x7f00240b8190 2024-05-20T18:37:01.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.798+0000 7f0028c52640 1 -- 172.21.15.5:0/2462939281 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1015587874 0 0) 0x7f0010003520 con 0x7f0024162820 2024-05-20T18:37:01.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.798+0000 7f0029453640 1 -- 172.21.15.5:0/2462939281 >> v1:172.21.15.63:6789/0 conn(0x7f00240b8190 legacy=0x7f00240b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:01.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.798+0000 7f0029453640 1 -- 172.21.15.5:0/2462939281 >> v1:172.21.15.5:6789/0 conn(0x7f0024162820 legacy=0x7f00240b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:01.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.798+0000 7f0029453640 1 -- 172.21.15.5:0/2462939281 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00240d3ca0 con 0x7f002406d3d0 2024-05-20T18:37:01.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.799+0000 7f0029453640 1 --2- 172.21.15.5:0/2462939281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f002406d3d0 0x7f002406d7e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f00140097d0 tx=0x7f001402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45f12b50a0e2051c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:01.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.799+0000 7f0028c52640 1 -- 172.21.15.5:0/2462939281 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0014037070 con 0x7f002406d3d0 2024-05-20T18:37:01.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.799+0000 7f0028c52640 1 -- 172.21.15.5:0/2462939281 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f001402ac30 con 0x7f002406d3d0 2024-05-20T18:37:01.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.799+0000 7f0028c52640 1 -- 172.21.15.5:0/2462939281 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0020001840 con 0x7f002406d3d0 2024-05-20T18:37:01.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.799+0000 7f002beb8640 1 -- 172.21.15.5:0/2462939281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f002406d3d0 msgr2=0x7f002406d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:01.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.799+0000 7f002beb8640 1 --2- 172.21.15.5:0/2462939281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f002406d3d0 0x7f002406d7e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f00140097d0 tx=0x7f001402a1a0 comp rx=0 tx=0).stop 2024-05-20T18:37:01.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.799+0000 7f002beb8640 1 -- 172.21.15.5:0/2462939281 shutdown_connections 2024-05-20T18:37:01.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.799+0000 7f002beb8640 1 --2- 172.21.15.5:0/2462939281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f002406d3d0 0x7f002406d7e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:01.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.800+0000 7f002beb8640 1 -- 172.21.15.5:0/2462939281 >> 172.21.15.5:0/2462939281 conn(0x7f00240d4bd0 msgr2=0x7f00240d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:01.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.800+0000 7f002beb8640 1 -- 172.21.15.5:0/2462939281 shutdown_connections 2024-05-20T18:37:01.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.800+0000 7f002beb8640 1 -- 172.21.15.5:0/2462939281 wait complete. 2024-05-20T18:37:01.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.801+0000 7f002beb8640 1 Processor -- start 2024-05-20T18:37:01.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.801+0000 7f002beb8640 1 -- start start 2024-05-20T18:37:01.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.803+0000 7f002beb8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f002406d3d0 0x7f00241e7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:01.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.803+0000 7f002beb8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00240b8190 0x7f002414f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:01.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.803+0000 7f002beb8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0024162820 0x7f00240cea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:01.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.803+0000 7f002beb8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f00240d0300 con 0x7f00240b8190 2024-05-20T18:37:01.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.803+0000 7f002beb8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f00240d4260 con 0x7f002406d3d0 2024-05-20T18:37:01.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.803+0000 7f002beb8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f00240b4050 con 0x7f0024162820 2024-05-20T18:37:01.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.803+0000 7f0029453640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00240b8190 0x7f002414f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:01.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.803+0000 7f0029453640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00240b8190 0x7f002414f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48930/0 (socket says 172.21.15.5:48930) 2024-05-20T18:37:01.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.803+0000 7f0029453640 1 -- 172.21.15.5:0/3554407419 learned_addr learned my addr 172.21.15.5:0/3554407419 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:01.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.803+0000 7f0029c54640 1 --2- 172.21.15.5:0/3554407419 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f002406d3d0 0x7f00241e7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:01.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.803+0000 7f002a455640 1 --2- 172.21.15.5:0/3554407419 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0024162820 0x7f00240cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:01.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.803+0000 7f0029453640 1 -- 172.21.15.5:0/3554407419 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0024162820 msgr2=0x7f00240cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:01.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.803+0000 7f0029453640 1 --2- 172.21.15.5:0/3554407419 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0024162820 0x7f00240cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:01.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.803+0000 7f0029453640 1 -- 172.21.15.5:0/3554407419 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f002406d3d0 msgr2=0x7f00241e7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:01.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.803+0000 7f0029453640 1 --2- 172.21.15.5:0/3554407419 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f002406d3d0 0x7f00241e7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:01.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.803+0000 7f0029453640 1 -- 172.21.15.5:0/3554407419 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00240cd560 con 0x7f00240b8190 2024-05-20T18:37:01.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.803+0000 7f0029c54640 1 --2- 172.21.15.5:0/3554407419 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f002406d3d0 0x7f00241e7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:01.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.803+0000 7f002a455640 1 --2- 172.21.15.5:0/3554407419 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0024162820 0x7f00240cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:01.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.803+0000 7f0029453640 1 --2- 172.21.15.5:0/3554407419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00240b8190 0x7f002414f080 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f00140098f0 tx=0x7f00140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:01.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.803+0000 7f001a7fc640 1 -- 172.21.15.5:0/3554407419 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0014042020 con 0x7f00240b8190 2024-05-20T18:37:01.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.804+0000 7f001a7fc640 1 -- 172.21.15.5:0/3554407419 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0014003c40 con 0x7f00240b8190 2024-05-20T18:37:01.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.804+0000 7f002beb8640 1 -- 172.21.15.5:0/3554407419 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00240cd9d0 con 0x7f00240b8190 2024-05-20T18:37:01.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.804+0000 7f001a7fc640 1 -- 172.21.15.5:0/3554407419 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0014037040 con 0x7f00240b8190 2024-05-20T18:37:01.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.804+0000 7f002beb8640 1 -- 172.21.15.5:0/3554407419 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00240cdf90 con 0x7f00240b8190 2024-05-20T18:37:01.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.805+0000 7f002beb8640 1 -- 172.21.15.5:0/3554407419 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0024075b00 con 0x7f00240b8190 2024-05-20T18:37:01.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.805+0000 7f001a7fc640 1 -- 172.21.15.5:0/3554407419 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0014045020 con 0x7f00240b8190 2024-05-20T18:37:01.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.806+0000 7f001a7fc640 1 --2- 172.21.15.5:0/3554407419 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f000806be30 0x7f000806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:01.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.806+0000 7f001a7fc640 1 -- 172.21.15.5:0/3554407419 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6331+0+0 (secure 0 0 0) 0x7f00140ab360 con 0x7f00240b8190 2024-05-20T18:37:01.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.806+0000 7f0029c54640 1 --2- 172.21.15.5:0/3554407419 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f000806be30 0x7f000806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:01.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.806+0000 7f0029c54640 1 --2- 172.21.15.5:0/3554407419 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f000806be30 0x7f000806e340 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f00200013b0 tx=0x7f0020009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:01.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:01.811+0000 7f001a7fc640 1 -- 172.21.15.5:0/3554407419 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f001404c060 con 0x7f00240b8190 2024-05-20T18:37:02.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:02.032+0000 7f002beb8640 1 -- 172.21.15.5:0/3554407419 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f00240cd760 con 0x7f00240b8190 2024-05-20T18:37:02.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:02.922+0000 7f001a7fc640 1 -- 172.21.15.5:0/3554407419 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v91) ==== 161+0+0 (secure 0 0 0) 0x7f001403b370 con 0x7f00240b8190 2024-05-20T18:37:02.923 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-20T18:37:02.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:02.925+0000 7f002beb8640 1 -- 172.21.15.5:0/3554407419 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f000806be30 msgr2=0x7f000806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:02.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:02.925+0000 7f002beb8640 1 --2- 172.21.15.5:0/3554407419 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f000806be30 0x7f000806e340 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f00200013b0 tx=0x7f0020009040 comp rx=0 tx=0).stop 2024-05-20T18:37:02.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:02.925+0000 7f002beb8640 1 -- 172.21.15.5:0/3554407419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00240b8190 msgr2=0x7f002414f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:02.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:02.925+0000 7f002beb8640 1 --2- 172.21.15.5:0/3554407419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00240b8190 0x7f002414f080 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f00140098f0 tx=0x7f00140046c0 comp rx=0 tx=0).stop 2024-05-20T18:37:02.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:02.925+0000 7f002beb8640 1 -- 172.21.15.5:0/3554407419 shutdown_connections 2024-05-20T18:37:02.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:02.925+0000 7f002beb8640 1 --2- 172.21.15.5:0/3554407419 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f000806be30 0x7f000806e340 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:02.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:02.925+0000 7f002beb8640 1 --2- 172.21.15.5:0/3554407419 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0024162820 0x7f00240cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:02.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:02.925+0000 7f002beb8640 1 --2- 172.21.15.5:0/3554407419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00240b8190 0x7f002414f080 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:02.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:02.925+0000 7f002beb8640 1 --2- 172.21.15.5:0/3554407419 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f002406d3d0 0x7f00241e7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:02.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:02.925+0000 7f002beb8640 1 -- 172.21.15.5:0/3554407419 >> 172.21.15.5:0/3554407419 conn(0x7f00240d4bd0 msgr2=0x7f00240d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:02.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:02.925+0000 7f002beb8640 1 -- 172.21.15.5:0/3554407419 shutdown_connections 2024-05-20T18:37:02.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:02.925+0000 7f002beb8640 1 -- 172.21.15.5:0/3554407419 wait complete. 2024-05-20T18:37:02.940 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-20T18:37:03.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.063+0000 7fae1f00d640 1 Processor -- start 2024-05-20T18:37:03.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.064+0000 7fae1f00d640 1 -- start start 2024-05-20T18:37:03.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.065+0000 7fae1f00d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae18162820 0x7fae180b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:03.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.065+0000 7fae1f00d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae180d0630 con 0x7fae180b3e80 2024-05-20T18:37:03.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.065+0000 7fae1f00d640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae180d39c0 con 0x7fae1806dae0 2024-05-20T18:37:03.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.065+0000 7fae1f00d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fae180d3ca0 con 0x7fae18162820 2024-05-20T18:37:03.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.065+0000 7fae17fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fae180b3e80 0x7fae1806d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51808/0 (socket says 172.21.15.5:51808) 2024-05-20T18:37:03.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.065+0000 7fae17fff640 1 -- 172.21.15.5:0/1230544580 learned_addr learned my addr 172.21.15.5:0/1230544580 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:03.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.065+0000 7fae1cda9640 1 --2- 172.21.15.5:0/1230544580 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae18162820 0x7fae180b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:03.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.065+0000 7fae177fe640 1 -- 172.21.15.5:0/1230544580 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1787611121 0 0) 0x7fae180d0630 con 0x7fae180b3e80 2024-05-20T18:37:03.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.065+0000 7fae177fe640 1 -- 172.21.15.5:0/1230544580 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae00003520 con 0x7fae180b3e80 2024-05-20T18:37:03.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.066+0000 7fae177fe640 1 -- 172.21.15.5:0/1230544580 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2347591816 0 0) 0x7fae180d39c0 con 0x7fae1806dae0 2024-05-20T18:37:03.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.066+0000 7fae177fe640 1 -- 172.21.15.5:0/1230544580 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae180d0630 con 0x7fae1806dae0 2024-05-20T18:37:03.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.066+0000 7fae1cda9640 1 -- 172.21.15.5:0/1230544580 >> v1:172.21.15.63:6789/0 conn(0x7fae1806dae0 legacy=0x7fae180b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:03.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.066+0000 7fae1cda9640 1 -- 172.21.15.5:0/1230544580 >> v1:172.21.15.5:6789/0 conn(0x7fae180b3e80 legacy=0x7fae1806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:03.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.066+0000 7fae1cda9640 1 -- 172.21.15.5:0/1230544580 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae180d3f80 con 0x7fae18162820 2024-05-20T18:37:03.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.067+0000 7fae1cda9640 1 --2- 172.21.15.5:0/1230544580 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae18162820 0x7fae180b3940 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fae080097d0 tx=0x7fae0802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c02b306626438517 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:03.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.067+0000 7fae177fe640 1 -- 172.21.15.5:0/1230544580 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae08037070 con 0x7fae18162820 2024-05-20T18:37:03.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.067+0000 7fae177fe640 1 -- 172.21.15.5:0/1230544580 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae08032440 con 0x7fae18162820 2024-05-20T18:37:03.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.067+0000 7fae177fe640 1 -- 172.21.15.5:0/1230544580 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae08032810 con 0x7fae18162820 2024-05-20T18:37:03.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.067+0000 7fae1f00d640 1 -- 172.21.15.5:0/1230544580 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae18162820 msgr2=0x7fae180b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:03.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.067+0000 7fae1f00d640 1 --2- 172.21.15.5:0/1230544580 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae18162820 0x7fae180b3940 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fae080097d0 tx=0x7fae0802a210 comp rx=0 tx=0).stop 2024-05-20T18:37:03.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.068+0000 7fae1f00d640 1 -- 172.21.15.5:0/1230544580 shutdown_connections 2024-05-20T18:37:03.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.068+0000 7fae1f00d640 1 --2- 172.21.15.5:0/1230544580 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae18162820 0x7fae180b3940 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:03.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.068+0000 7fae1f00d640 1 -- 172.21.15.5:0/1230544580 >> 172.21.15.5:0/1230544580 conn(0x7fae180d4bd0 msgr2=0x7fae180d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:03.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.068+0000 7fae1f00d640 1 -- 172.21.15.5:0/1230544580 shutdown_connections 2024-05-20T18:37:03.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.068+0000 7fae1f00d640 1 -- 172.21.15.5:0/1230544580 wait complete. 2024-05-20T18:37:03.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.069+0000 7fae1f00d640 1 Processor -- start 2024-05-20T18:37:03.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.069+0000 7fae1f00d640 1 -- start start 2024-05-20T18:37:03.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.071+0000 7fae1f00d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae180b3e80 0x7fae1814d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:03.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.071+0000 7fae1f00d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae18162820 0x7fae180b6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:03.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.071+0000 7fae1f00d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae180b9310 0x7fae180b9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:03.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.071+0000 7fae1f00d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fae180d4780 con 0x7fae18162820 2024-05-20T18:37:03.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.071+0000 7fae1f00d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fae180d0300 con 0x7fae180b9310 2024-05-20T18:37:03.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.071+0000 7fae1f00d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fae180d4900 con 0x7fae180b3e80 2024-05-20T18:37:03.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.071+0000 7fae17fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae18162820 0x7fae180b6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:03.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.071+0000 7fae17fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae18162820 0x7fae180b6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48932/0 (socket says 172.21.15.5:48932) 2024-05-20T18:37:03.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.071+0000 7fae17fff640 1 -- 172.21.15.5:0/2770680449 learned_addr learned my addr 172.21.15.5:0/2770680449 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:03.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.071+0000 7fae1cda9640 1 --2- 172.21.15.5:0/2770680449 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae180b3e80 0x7fae1814d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:03.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.071+0000 7fae1d5aa640 1 --2- 172.21.15.5:0/2770680449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae180b9310 0x7fae180b9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:03.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.071+0000 7fae17fff640 1 -- 172.21.15.5:0/2770680449 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae180b3e80 msgr2=0x7fae1814d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:03.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.071+0000 7fae17fff640 1 --2- 172.21.15.5:0/2770680449 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae180b3e80 0x7fae1814d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:03.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.071+0000 7fae17fff640 1 -- 172.21.15.5:0/2770680449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae180b9310 msgr2=0x7fae180b9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:03.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.071+0000 7fae17fff640 1 --2- 172.21.15.5:0/2770680449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae180b9310 0x7fae180b9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:03.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.071+0000 7fae17fff640 1 -- 172.21.15.5:0/2770680449 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae1814d6c0 con 0x7fae18162820 2024-05-20T18:37:03.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.071+0000 7fae1d5aa640 1 --2- 172.21.15.5:0/2770680449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae180b9310 0x7fae180b9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:03.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.072+0000 7fae17fff640 1 --2- 172.21.15.5:0/2770680449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae18162820 0x7fae180b6d90 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00c3e0 tx=0x7fae0c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:03.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.072+0000 7fae157fa640 1 -- 172.21.15.5:0/2770680449 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0c017070 con 0x7fae18162820 2024-05-20T18:37:03.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.072+0000 7fae157fa640 1 -- 172.21.15.5:0/2770680449 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae0c005ce0 con 0x7fae18162820 2024-05-20T18:37:03.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.072+0000 7fae157fa640 1 -- 172.21.15.5:0/2770680449 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0c0072d0 con 0x7fae18162820 2024-05-20T18:37:03.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.072+0000 7fae1f00d640 1 -- 172.21.15.5:0/2770680449 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae180b9c60 con 0x7fae18162820 2024-05-20T18:37:03.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.072+0000 7fae1f00d640 1 -- 172.21.15.5:0/2770680449 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae18156f50 con 0x7fae18162820 2024-05-20T18:37:03.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.073+0000 7fae1f00d640 1 -- 172.21.15.5:0/2770680449 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae18073a00 con 0x7fae18162820 2024-05-20T18:37:03.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.074+0000 7fae157fa640 1 -- 172.21.15.5:0/2770680449 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fae0c01a070 con 0x7fae18162820 2024-05-20T18:37:03.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.074+0000 7fae157fa640 1 --2- 172.21.15.5:0/2770680449 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae0406be30 0x7fae0406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:03.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.074+0000 7fae157fa640 1 -- 172.21.15.5:0/2770680449 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6331+0+0 (secure 0 0 0) 0x7fae0c08a450 con 0x7fae18162820 2024-05-20T18:37:03.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.074+0000 7fae1cda9640 1 --2- 172.21.15.5:0/2770680449 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae0406be30 0x7fae0406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:03.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.075+0000 7fae1cda9640 1 --2- 172.21.15.5:0/2770680449 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae0406be30 0x7fae0406e340 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fae08034000 tx=0x7fae0802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:03.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.079+0000 7fae157fa640 1 -- 172.21.15.5:0/2770680449 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fae18073a00 con 0x7fae18162820 2024-05-20T18:37:03.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.301+0000 7fae1f00d640 1 -- 172.21.15.5:0/2770680449 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fae18156ce0 con 0x7fae18162820 2024-05-20T18:37:03.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.927+0000 7fae157fa640 1 -- 172.21.15.5:0/2770680449 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v92) ==== 104+0+0 (secure 0 0 0) 0x7fae180b9c60 con 0x7fae18162820 2024-05-20T18:37:03.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.930+0000 7fae1f00d640 1 -- 172.21.15.5:0/2770680449 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae0406be30 msgr2=0x7fae0406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:03.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.930+0000 7fae1f00d640 1 --2- 172.21.15.5:0/2770680449 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae0406be30 0x7fae0406e340 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fae08034000 tx=0x7fae0802e000 comp rx=0 tx=0).stop 2024-05-20T18:37:03.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.930+0000 7fae1f00d640 1 -- 172.21.15.5:0/2770680449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae18162820 msgr2=0x7fae180b6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:03.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.930+0000 7fae1f00d640 1 --2- 172.21.15.5:0/2770680449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae18162820 0x7fae180b6d90 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00c3e0 tx=0x7fae0c00c8a0 comp rx=0 tx=0).stop 2024-05-20T18:37:03.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.930+0000 7fae1f00d640 1 -- 172.21.15.5:0/2770680449 shutdown_connections 2024-05-20T18:37:03.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.930+0000 7fae1f00d640 1 --2- 172.21.15.5:0/2770680449 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fae0406be30 0x7fae0406e340 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:03.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.930+0000 7fae1f00d640 1 --2- 172.21.15.5:0/2770680449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae180b9310 0x7fae180b9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:03.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.930+0000 7fae1f00d640 1 --2- 172.21.15.5:0/2770680449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae18162820 0x7fae180b6d90 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:03.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.930+0000 7fae1f00d640 1 --2- 172.21.15.5:0/2770680449 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae180b3e80 0x7fae1814d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:03.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.930+0000 7fae1f00d640 1 -- 172.21.15.5:0/2770680449 >> 172.21.15.5:0/2770680449 conn(0x7fae180d4bd0 msgr2=0x7fae1815c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:03.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.931+0000 7fae1f00d640 1 -- 172.21.15.5:0/2770680449 shutdown_connections 2024-05-20T18:37:03.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:03.931+0000 7fae1f00d640 1 -- 172.21.15.5:0/2770680449 wait complete. 2024-05-20T18:37:03.945 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-20T18:37:04.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.069+0000 7f92517f5640 1 Processor -- start 2024-05-20T18:37:04.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.069+0000 7f92517f5640 1 -- start start 2024-05-20T18:37:04.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.070+0000 7f92517f5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f924c0afbf0 0x7f924c0b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:04.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.070+0000 7f92517f5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f924c0d0930 0x7f924c0d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:04.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.070+0000 7f92517f5640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f924c0aeb50 con 0x7f924c0b3a30 2024-05-20T18:37:04.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.070+0000 7f92517f5640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f924c15f920 con 0x7f924c0afbf0 2024-05-20T18:37:04.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.070+0000 7f92517f5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f924c0b3e40 con 0x7f924c0d0930 2024-05-20T18:37:04.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.070+0000 7f92427fc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f924c0afbf0 0x7f924c0b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:04.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.070+0000 7f92427fc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f924c0afbf0 0x7f924c0b0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48944/0 (socket says 172.21.15.5:48944) 2024-05-20T18:37:04.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.070+0000 7f92427fc640 1 -- 172.21.15.5:0/595083230 learned_addr learned my addr 172.21.15.5:0/595083230 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:04.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.071+0000 7f924b7fe640 1 --2- 172.21.15.5:0/595083230 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f924c0d0930 0x7f924c0d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:04.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.071+0000 7f92427fc640 1 -- 172.21.15.5:0/595083230 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f924c0d0930 msgr2=0x7f924c0d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:04.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.071+0000 7f92427fc640 1 --2- 172.21.15.5:0/595083230 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f924c0d0930 0x7f924c0d0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:04.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.071+0000 7f92427fc640 1 -- 172.21.15.5:0/595083230 >> v1:172.21.15.63:6789/0 conn(0x7f924c0b3a30 legacy=0x7f924c06b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:04.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.071+0000 7f92427fc640 1 -- 172.21.15.5:0/595083230 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924c0aee30 con 0x7f924c0afbf0 2024-05-20T18:37:04.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.072+0000 7f92427fc640 1 --2- 172.21.15.5:0/595083230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f924c0afbf0 0x7f924c0b0000 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9234009870 tx=0x7f923402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191b9d9de3a3c514 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:04.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.072+0000 7f924a7fc640 1 -- 172.21.15.5:0/595083230 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9234037070 con 0x7f924c0afbf0 2024-05-20T18:37:04.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.072+0000 7f924a7fc640 1 -- 172.21.15.5:0/595083230 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9234032440 con 0x7f924c0afbf0 2024-05-20T18:37:04.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.072+0000 7f924a7fc640 1 -- 172.21.15.5:0/595083230 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9234032810 con 0x7f924c0afbf0 2024-05-20T18:37:04.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.072+0000 7f92517f5640 1 -- 172.21.15.5:0/595083230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f924c0afbf0 msgr2=0x7f924c0b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:04.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.072+0000 7f92517f5640 1 --2- 172.21.15.5:0/595083230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f924c0afbf0 0x7f924c0b0000 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9234009870 tx=0x7f923402a270 comp rx=0 tx=0).stop 2024-05-20T18:37:04.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.072+0000 7f92517f5640 1 -- 172.21.15.5:0/595083230 shutdown_connections 2024-05-20T18:37:04.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.072+0000 7f92517f5640 1 --2- 172.21.15.5:0/595083230 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f924c0d0930 0x7f924c0d0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:04.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.073+0000 7f92517f5640 1 --2- 172.21.15.5:0/595083230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f924c0afbf0 0x7f924c0b0000 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:04.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.073+0000 7f92517f5640 1 -- 172.21.15.5:0/595083230 >> 172.21.15.5:0/595083230 conn(0x7f924c0b6630 msgr2=0x7f924c0b6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:04.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.073+0000 7f92517f5640 1 -- 172.21.15.5:0/595083230 shutdown_connections 2024-05-20T18:37:04.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.073+0000 7f92517f5640 1 -- 172.21.15.5:0/595083230 wait complete. 2024-05-20T18:37:04.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.074+0000 7f92517f5640 1 Processor -- start 2024-05-20T18:37:04.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.075+0000 7f92517f5640 1 -- start start 2024-05-20T18:37:04.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.076+0000 7f92517f5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f924c0afbf0 0x7f924c143920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:04.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.076+0000 7f92517f5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f924c0b3a30 0x7f924c143e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:04.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.076+0000 7f92517f5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f924c0d0930 0x7f924c15b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:04.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.076+0000 7f92517f5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f924c0af3f0 con 0x7f924c0b3a30 2024-05-20T18:37:04.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.076+0000 7f92517f5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f924c0b0540 con 0x7f924c0afbf0 2024-05-20T18:37:04.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.076+0000 7f92517f5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f924c0af6d0 con 0x7f924c0d0930 2024-05-20T18:37:04.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.076+0000 7f92427fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f924c0b3a30 0x7f924c143e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:04.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.076+0000 7f92427fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f924c0b3a30 0x7f924c143e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48952/0 (socket says 172.21.15.5:48952) 2024-05-20T18:37:04.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.076+0000 7f92427fc640 1 -- 172.21.15.5:0/865288238 learned_addr learned my addr 172.21.15.5:0/865288238 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:04.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.076+0000 7f924affd640 1 --2- 172.21.15.5:0/865288238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f924c0afbf0 0x7f924c143920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:04.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.076+0000 7f924b7fe640 1 --2- 172.21.15.5:0/865288238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f924c0d0930 0x7f924c15b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:04.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.076+0000 7f92427fc640 1 -- 172.21.15.5:0/865288238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f924c0d0930 msgr2=0x7f924c15b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:04.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.076+0000 7f92427fc640 1 --2- 172.21.15.5:0/865288238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f924c0d0930 0x7f924c15b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:04.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.076+0000 7f92427fc640 1 -- 172.21.15.5:0/865288238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f924c0afbf0 msgr2=0x7f924c143920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:04.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.076+0000 7f92427fc640 1 --2- 172.21.15.5:0/865288238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f924c0afbf0 0x7f924c143920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:04.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.076+0000 7f92427fc640 1 -- 172.21.15.5:0/865288238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924c153fd0 con 0x7f924c0b3a30 2024-05-20T18:37:04.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.076+0000 7f924b7fe640 1 --2- 172.21.15.5:0/865288238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f924c0d0930 0x7f924c15b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:04.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.077+0000 7f92427fc640 1 --2- 172.21.15.5:0/865288238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f924c0b3a30 0x7f924c143e60 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f9234000c00 tx=0x7f92340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:04.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.077+0000 7f9243fff640 1 -- 172.21.15.5:0/865288238 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9234041020 con 0x7f924c0b3a30 2024-05-20T18:37:04.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.077+0000 7f9243fff640 1 -- 172.21.15.5:0/865288238 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92340306c0 con 0x7f924c0b3a30 2024-05-20T18:37:04.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.077+0000 7f92517f5640 1 -- 172.21.15.5:0/865288238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f924c1e8e40 con 0x7f924c0b3a30 2024-05-20T18:37:04.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.077+0000 7f9243fff640 1 -- 172.21.15.5:0/865288238 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9234037040 con 0x7f924c0b3a30 2024-05-20T18:37:04.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.077+0000 7f92517f5640 1 -- 172.21.15.5:0/865288238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f924c1416b0 con 0x7f924c0b3a30 2024-05-20T18:37:04.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.078+0000 7f92517f5640 1 -- 172.21.15.5:0/865288238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f924c0690b0 con 0x7f924c0b3a30 2024-05-20T18:37:04.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.078+0000 7f9243fff640 1 -- 172.21.15.5:0/865288238 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9234044020 con 0x7f924c0b3a30 2024-05-20T18:37:04.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.079+0000 7f9243fff640 1 --2- 172.21.15.5:0/865288238 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f922c06be30 0x7f922c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:04.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.079+0000 7f9243fff640 1 -- 172.21.15.5:0/865288238 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6331+0+0 (secure 0 0 0) 0x7f92340ab680 con 0x7f924c0b3a30 2024-05-20T18:37:04.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.079+0000 7f924affd640 1 --2- 172.21.15.5:0/865288238 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f922c06be30 0x7f922c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:04.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.079+0000 7f924affd640 1 --2- 172.21.15.5:0/865288238 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f922c06be30 0x7f922c06e340 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f923c007fd0 tx=0x7f923c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:04.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.084+0000 7f9243fff640 1 -- 172.21.15.5:0/865288238 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f923404b070 con 0x7f924c0b3a30 2024-05-20T18:37:04.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.269+0000 7f92517f5640 1 -- 172.21.15.5:0/865288238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f924c144670 con 0x7f924c0b3a30 2024-05-20T18:37:04.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.932+0000 7f9243fff640 1 -- 172.21.15.5:0/865288238 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v110) ==== 181+0+0 (secure 0 0 0) 0x7f9234079d30 con 0x7f924c0b3a30 2024-05-20T18:37:04.934 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-05-20T18:37:04.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.936+0000 7f92517f5640 1 -- 172.21.15.5:0/865288238 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f922c06be30 msgr2=0x7f922c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:04.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.936+0000 7f92517f5640 1 --2- 172.21.15.5:0/865288238 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f922c06be30 0x7f922c06e340 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f923c007fd0 tx=0x7f923c009040 comp rx=0 tx=0).stop 2024-05-20T18:37:04.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.936+0000 7f92517f5640 1 -- 172.21.15.5:0/865288238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f924c0b3a30 msgr2=0x7f924c143e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:04.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.936+0000 7f92517f5640 1 --2- 172.21.15.5:0/865288238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f924c0b3a30 0x7f924c143e60 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f9234000c00 tx=0x7f92340046c0 comp rx=0 tx=0).stop 2024-05-20T18:37:04.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.937+0000 7f92517f5640 1 -- 172.21.15.5:0/865288238 shutdown_connections 2024-05-20T18:37:04.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.937+0000 7f92517f5640 1 --2- 172.21.15.5:0/865288238 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f922c06be30 0x7f922c06e340 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:04.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.937+0000 7f92517f5640 1 --2- 172.21.15.5:0/865288238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f924c0d0930 0x7f924c15b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:04.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.937+0000 7f92517f5640 1 --2- 172.21.15.5:0/865288238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f924c0b3a30 0x7f924c143e60 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:04.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.937+0000 7f92517f5640 1 --2- 172.21.15.5:0/865288238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f924c0afbf0 0x7f924c143920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:04.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.937+0000 7f92517f5640 1 -- 172.21.15.5:0/865288238 >> 172.21.15.5:0/865288238 conn(0x7f924c0b6630 msgr2=0x7f924c06fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:04.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.937+0000 7f92517f5640 1 -- 172.21.15.5:0/865288238 shutdown_connections 2024-05-20T18:37:04.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:04.937+0000 7f92517f5640 1 -- 172.21.15.5:0/865288238 wait complete. 2024-05-20T18:37:04.953 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-20T18:37:05.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.078+0000 7f2a75daa640 1 Processor -- start 2024-05-20T18:37:05.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.078+0000 7f2a75daa640 1 -- start start 2024-05-20T18:37:05.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.079+0000 7f2a75daa640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2a700b21d0 0x7f2a700b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:05.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.079+0000 7f2a75daa640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a700d0160 con 0x7f2a700d1830 2024-05-20T18:37:05.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.079+0000 7f2a75daa640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a700cda00 con 0x7f2a700d3510 2024-05-20T18:37:05.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.079+0000 7f2a75daa640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2a700b2b20 con 0x7f2a700b21d0 2024-05-20T18:37:05.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.079+0000 7f2a6effd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2a700d1830 0x7f2a700ce4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35296/0 (socket says 172.21.15.5:35296) 2024-05-20T18:37:05.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.079+0000 7f2a6effd640 1 -- 172.21.15.5:0/3303424682 learned_addr learned my addr 172.21.15.5:0/3303424682 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:05.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.080+0000 7f2a6ffff640 1 --2- 172.21.15.5:0/3303424682 >> v2:172.21.15.63:3300/0 conn(0x7f2a700b21d0 0x7f2a700b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:05.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.080+0000 7f2a6e7fc640 1 -- 172.21.15.5:0/3303424682 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3692802420 0 0) 0x7f2a700d0160 con 0x7f2a700d1830 2024-05-20T18:37:05.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.080+0000 7f2a6e7fc640 1 -- 172.21.15.5:0/3303424682 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a5c003520 con 0x7f2a700d1830 2024-05-20T18:37:05.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.080+0000 7f2a6e7fc640 1 -- 172.21.15.5:0/3303424682 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3381182653 0 0) 0x7f2a5c003520 con 0x7f2a700d1830 2024-05-20T18:37:05.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.080+0000 7f2a6e7fc640 1 -- 172.21.15.5:0/3303424682 >> v2:172.21.15.63:3300/0 conn(0x7f2a700b21d0 msgr2=0x7f2a700b25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:05.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.080+0000 7f2a6e7fc640 1 --2- 172.21.15.5:0/3303424682 >> v2:172.21.15.63:3300/0 conn(0x7f2a700b21d0 0x7f2a700b25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:05.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.080+0000 7f2a6e7fc640 1 -- 172.21.15.5:0/3303424682 >> v1:172.21.15.63:6789/0 conn(0x7f2a700d3510 legacy=0x7f2a700b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:05.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.080+0000 7f2a6e7fc640 1 -- 172.21.15.5:0/3303424682 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a700cdce0 con 0x7f2a700d1830 2024-05-20T18:37:05.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.080+0000 7f2a6e7fc640 1 -- 172.21.15.5:0/3303424682 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f2a58001670 con 0x7f2a700d1830 2024-05-20T18:37:05.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.080+0000 7f2a6e7fc640 1 -- 172.21.15.5:0/3303424682 >> v1:172.21.15.5:6789/0 conn(0x7f2a700d1830 legacy=0x7f2a700ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:05.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.080+0000 7f2a6e7fc640 1 --2- 172.21.15.5:0/3303424682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a5c0039b0 0x7f2a5c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:05.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.080+0000 7f2a6e7fc640 1 --2- 172.21.15.5:0/3303424682 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a5c004800 0x7f2a5c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:05.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.080+0000 7f2a6e7fc640 1 --2- 172.21.15.5:0/3303424682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a5c005270 0x7f2a5c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:05.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.080+0000 7f2a6e7fc640 1 -- 172.21.15.5:0/3303424682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2a700b2b20 con 0x7f2a5c0039b0 2024-05-20T18:37:05.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.080+0000 7f2a6e7fc640 1 -- 172.21.15.5:0/3303424682 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c00c200 con 0x7f2a5c005270 2024-05-20T18:37:05.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.080+0000 7f2a6e7fc640 1 -- 172.21.15.5:0/3303424682 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2a5c00c400 con 0x7f2a5c004800 2024-05-20T18:37:05.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.080+0000 7f2a6f7fe640 1 --2- 172.21.15.5:0/3303424682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a5c0039b0 0x7f2a5c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:05.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.080+0000 7f2a6effd640 1 --2- 172.21.15.5:0/3303424682 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a5c004800 0x7f2a5c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:05.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.080+0000 7f2a6ffff640 1 --2- 172.21.15.5:0/3303424682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a5c005270 0x7f2a5c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:05.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.080+0000 7f2a6f7fe640 1 -- 172.21.15.5:0/3303424682 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a5c004800 msgr2=0x7f2a5c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:05.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.080+0000 7f2a6f7fe640 1 --2- 172.21.15.5:0/3303424682 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a5c004800 0x7f2a5c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:05.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.080+0000 7f2a6f7fe640 1 -- 172.21.15.5:0/3303424682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a5c005270 msgr2=0x7f2a5c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:05.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.081+0000 7f2a6f7fe640 1 --2- 172.21.15.5:0/3303424682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a5c005270 0x7f2a5c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:05.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.081+0000 7f2a6f7fe640 1 -- 172.21.15.5:0/3303424682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5c003520 con 0x7f2a5c0039b0 2024-05-20T18:37:05.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.081+0000 7f2a6ffff640 1 --2- 172.21.15.5:0/3303424682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a5c005270 0x7f2a5c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:05.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.081+0000 7f2a6ffff640 1 -- 172.21.15.5:0/3303424682 reap_dead start 2024-05-20T18:37:05.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.081+0000 7f2a6f7fe640 1 --2- 172.21.15.5:0/3303424682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a5c0039b0 0x7f2a5c003e40 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f2a60001e50 tx=0x7f2a5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e910bcaef19fdd server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:05.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.081+0000 7f2a6e7fc640 1 -- 172.21.15.5:0/3303424682 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a60001a70 con 0x7f2a5c0039b0 2024-05-20T18:37:05.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.081+0000 7f2a6e7fc640 1 -- 172.21.15.5:0/3303424682 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a6002ae40 con 0x7f2a5c0039b0 2024-05-20T18:37:05.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.081+0000 7f2a6e7fc640 1 -- 172.21.15.5:0/3303424682 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a60003bb0 con 0x7f2a5c0039b0 2024-05-20T18:37:05.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.082+0000 7f2a75daa640 1 -- 172.21.15.5:0/3303424682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a5c0039b0 msgr2=0x7f2a5c003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:05.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.082+0000 7f2a75daa640 1 --2- 172.21.15.5:0/3303424682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a5c0039b0 0x7f2a5c003e40 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f2a60001e50 tx=0x7f2a5c005830 comp rx=0 tx=0).stop 2024-05-20T18:37:05.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.082+0000 7f2a75daa640 1 -- 172.21.15.5:0/3303424682 shutdown_connections 2024-05-20T18:37:05.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.082+0000 7f2a75daa640 1 --2- 172.21.15.5:0/3303424682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a5c0039b0 0x7f2a5c003e40 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:05.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.082+0000 7f2a75daa640 1 -- 172.21.15.5:0/3303424682 >> 172.21.15.5:0/3303424682 conn(0x7f2a700d29f0 msgr2=0x7f2a700d2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:05.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.082+0000 7f2a75daa640 1 -- 172.21.15.5:0/3303424682 shutdown_connections 2024-05-20T18:37:05.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.082+0000 7f2a75daa640 1 -- 172.21.15.5:0/3303424682 wait complete. 2024-05-20T18:37:05.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.084+0000 7f2a75daa640 1 Processor -- start 2024-05-20T18:37:05.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.084+0000 7f2a75daa640 1 -- start start 2024-05-20T18:37:05.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.085+0000 7f2a75daa640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a5c0039b0 0x7f2a70159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:05.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.085+0000 7f2a75daa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a700b67d0 0x7f2a70158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:05.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.085+0000 7f2a75daa640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a7015ab10 0x7f2a700b5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:05.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.086+0000 7f2a75daa640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2a700cfd80 con 0x7f2a700b67d0 2024-05-20T18:37:05.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.086+0000 7f2a75daa640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2a700cd6b0 con 0x7f2a7015ab10 2024-05-20T18:37:05.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.086+0000 7f2a75daa640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2a700cff00 con 0x7f2a5c0039b0 2024-05-20T18:37:05.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.086+0000 7f2a6effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a700b67d0 0x7f2a70158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:05.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.086+0000 7f2a6effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a700b67d0 0x7f2a70158100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35276/0 (socket says 172.21.15.5:35276) 2024-05-20T18:37:05.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.086+0000 7f2a6effd640 1 -- 172.21.15.5:0/4081073482 learned_addr learned my addr 172.21.15.5:0/4081073482 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:05.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.086+0000 7f2a6f7fe640 1 --2- 172.21.15.5:0/4081073482 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a5c0039b0 0x7f2a70159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:05.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.086+0000 7f2a6ffff640 1 --2- 172.21.15.5:0/4081073482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a7015ab10 0x7f2a700b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:05.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.086+0000 7f2a6effd640 1 -- 172.21.15.5:0/4081073482 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a5c0039b0 msgr2=0x7f2a70159220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:05.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.086+0000 7f2a6effd640 1 --2- 172.21.15.5:0/4081073482 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a5c0039b0 0x7f2a70159220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:05.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.086+0000 7f2a6effd640 1 -- 172.21.15.5:0/4081073482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a7015ab10 msgr2=0x7f2a700b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:05.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.086+0000 7f2a6effd640 1 --2- 172.21.15.5:0/4081073482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a7015ab10 0x7f2a700b5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:05.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.086+0000 7f2a6effd640 1 -- 172.21.15.5:0/4081073482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a70159a70 con 0x7f2a700b67d0 2024-05-20T18:37:05.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.086+0000 7f2a6effd640 1 --2- 172.21.15.5:0/4081073482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a700b67d0 0x7f2a70158100 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f2a5800e730 tx=0x7f2a5800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:05.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.086+0000 7f2a67fff640 1 -- 172.21.15.5:0/4081073482 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5800ee40 con 0x7f2a700b67d0 2024-05-20T18:37:05.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.086+0000 7f2a67fff640 1 -- 172.21.15.5:0/4081073482 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a58018070 con 0x7f2a700b67d0 2024-05-20T18:37:05.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.086+0000 7f2a67fff640 1 -- 172.21.15.5:0/4081073482 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5800caf0 con 0x7f2a700b67d0 2024-05-20T18:37:05.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.086+0000 7f2a75daa640 1 -- 172.21.15.5:0/4081073482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a70159d50 con 0x7f2a700b67d0 2024-05-20T18:37:05.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.087+0000 7f2a75daa640 1 -- 172.21.15.5:0/4081073482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a7015a310 con 0x7f2a700b67d0 2024-05-20T18:37:05.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.088+0000 7f2a75daa640 1 -- 172.21.15.5:0/4081073482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a700690b0 con 0x7f2a700b67d0 2024-05-20T18:37:05.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.088+0000 7f2a67fff640 1 -- 172.21.15.5:0/4081073482 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2a5800cc90 con 0x7f2a700b67d0 2024-05-20T18:37:05.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.088+0000 7f2a67fff640 1 --2- 172.21.15.5:0/4081073482 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2a5006be30 0x7f2a5006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:05.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.089+0000 7f2a67fff640 1 -- 172.21.15.5:0/4081073482 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5893+0+0 (secure 0 0 0) 0x7f2a5808ae50 con 0x7f2a700b67d0 2024-05-20T18:37:05.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.089+0000 7f2a6f7fe640 1 --2- 172.21.15.5:0/4081073482 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2a5006be30 0x7f2a5006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:05.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.089+0000 7f2a6f7fe640 1 --2- 172.21.15.5:0/4081073482 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2a5006be30 0x7f2a5006e340 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f2a60032000 tx=0x7f2a6002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:05.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.093+0000 7f2a67fff640 1 -- 172.21.15.5:0/4081073482 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2a580596b0 con 0x7f2a700b67d0 2024-05-20T18:37:05.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.276+0000 7f2a75daa640 1 -- 172.21.15.5:0/4081073482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2a7006bf30 con 0x7f2a700b67d0 2024-05-20T18:37:05.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.940+0000 7f2a67fff640 1 -- 172.21.15.5:0/4081073482 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v111) ==== 169+0+0 (secure 0 0 0) 0x7f2a58058fb0 con 0x7f2a700b67d0 2024-05-20T18:37:05.942 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-05-20T18:37:05.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.944+0000 7f2a75daa640 1 -- 172.21.15.5:0/4081073482 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2a5006be30 msgr2=0x7f2a5006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:05.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.944+0000 7f2a75daa640 1 --2- 172.21.15.5:0/4081073482 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2a5006be30 0x7f2a5006e340 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f2a60032000 tx=0x7f2a6002e000 comp rx=0 tx=0).stop 2024-05-20T18:37:05.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.944+0000 7f2a75daa640 1 -- 172.21.15.5:0/4081073482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a700b67d0 msgr2=0x7f2a70158100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:05.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.944+0000 7f2a75daa640 1 --2- 172.21.15.5:0/4081073482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a700b67d0 0x7f2a70158100 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f2a5800e730 tx=0x7f2a5800ebf0 comp rx=0 tx=0).stop 2024-05-20T18:37:05.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.945+0000 7f2a75daa640 1 -- 172.21.15.5:0/4081073482 shutdown_connections 2024-05-20T18:37:05.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.945+0000 7f2a75daa640 1 --2- 172.21.15.5:0/4081073482 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2a5006be30 0x7f2a5006e340 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:05.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.945+0000 7f2a75daa640 1 --2- 172.21.15.5:0/4081073482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a7015ab10 0x7f2a700b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:05.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.945+0000 7f2a75daa640 1 --2- 172.21.15.5:0/4081073482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a700b67d0 0x7f2a70158100 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:05.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.945+0000 7f2a75daa640 1 --2- 172.21.15.5:0/4081073482 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a5c0039b0 0x7f2a70159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:05.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.945+0000 7f2a75daa640 1 -- 172.21.15.5:0/4081073482 >> 172.21.15.5:0/4081073482 conn(0x7f2a700d29f0 msgr2=0x7f2a700ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:05.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.945+0000 7f2a75daa640 1 -- 172.21.15.5:0/4081073482 shutdown_connections 2024-05-20T18:37:05.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:05.945+0000 7f2a75daa640 1 -- 172.21.15.5:0/4081073482 wait complete. 2024-05-20T18:37:05.962 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-20T18:37:06.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.083+0000 7f604a675640 1 Processor -- start 2024-05-20T18:37:06.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.083+0000 7f604a675640 1 -- start start 2024-05-20T18:37:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.084+0000 7f604a675640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6044162820 0x7f60440b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.084+0000 7f604a675640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f604406dae0 0x7f60440b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.084+0000 7f604a675640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60440d0630 con 0x7f60440b3e80 2024-05-20T18:37:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.084+0000 7f604a675640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f60440d39c0 con 0x7f604406dae0 2024-05-20T18:37:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.084+0000 7f604a675640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f60440d3ca0 con 0x7f6044162820 2024-05-20T18:37:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.085+0000 7f6048c12640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f604406dae0 0x7f60440b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.085+0000 7f60437fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f60440b3e80 0x7f604406d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35302/0 (socket says 172.21.15.5:35302) 2024-05-20T18:37:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.085+0000 7f60437fe640 1 -- 172.21.15.5:0/3626579296 learned_addr learned my addr 172.21.15.5:0/3626579296 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.085+0000 7f6043fff640 1 --2- 172.21.15.5:0/3626579296 >> v2:172.21.15.63:3300/0 conn(0x7f6044162820 0x7f60440b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.085+0000 7f6048c12640 1 -- 172.21.15.5:0/3626579296 >> v2:172.21.15.63:3300/0 conn(0x7f6044162820 msgr2=0x7f60440b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.085+0000 7f6048c12640 1 --2- 172.21.15.5:0/3626579296 >> v2:172.21.15.63:3300/0 conn(0x7f6044162820 0x7f60440b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.085+0000 7f6048c12640 1 -- 172.21.15.5:0/3626579296 >> v1:172.21.15.5:6789/0 conn(0x7f60440b3e80 legacy=0x7f604406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.085+0000 7f6048c12640 1 -- 172.21.15.5:0/3626579296 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60440d3f80 con 0x7f604406dae0 2024-05-20T18:37:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.085+0000 7f6042ffd640 1 -- 172.21.15.5:0/3626579296 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2823418182 0 0) 0x7f60440d0630 con 0x7f60440b3e80 2024-05-20T18:37:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.085+0000 7f6043fff640 1 --2- 172.21.15.5:0/3626579296 >> v2:172.21.15.63:3300/0 conn(0x7f6044162820 0x7f60440b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:06.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.086+0000 7f6048c12640 1 --2- 172.21.15.5:0/3626579296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f604406dae0 0x7f60440b8190 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f60440b7740 tx=0x7f603c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f131a18e216a26 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:06.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.086+0000 7f6042ffd640 1 -- 172.21.15.5:0/3626579296 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603c007740 con 0x7f604406dae0 2024-05-20T18:37:06.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.086+0000 7f6042ffd640 1 -- 172.21.15.5:0/3626579296 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f603c0078e0 con 0x7f604406dae0 2024-05-20T18:37:06.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.086+0000 7f6042ffd640 1 -- 172.21.15.5:0/3626579296 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603c007cb0 con 0x7f604406dae0 2024-05-20T18:37:06.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.086+0000 7f604a675640 1 -- 172.21.15.5:0/3626579296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f604406dae0 msgr2=0x7f60440b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:06.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.086+0000 7f604a675640 1 --2- 172.21.15.5:0/3626579296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f604406dae0 0x7f60440b8190 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f60440b7740 tx=0x7f603c02a680 comp rx=0 tx=0).stop 2024-05-20T18:37:06.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.087+0000 7f604a675640 1 -- 172.21.15.5:0/3626579296 shutdown_connections 2024-05-20T18:37:06.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.087+0000 7f604a675640 1 --2- 172.21.15.5:0/3626579296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f604406dae0 0x7f60440b8190 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:06.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.087+0000 7f604a675640 1 --2- 172.21.15.5:0/3626579296 >> v2:172.21.15.63:3300/0 conn(0x7f6044162820 0x7f60440b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:06.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.087+0000 7f604a675640 1 -- 172.21.15.5:0/3626579296 >> 172.21.15.5:0/3626579296 conn(0x7f60440d4bd0 msgr2=0x7f60440d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:06.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.087+0000 7f604a675640 1 -- 172.21.15.5:0/3626579296 shutdown_connections 2024-05-20T18:37:06.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.087+0000 7f604a675640 1 -- 172.21.15.5:0/3626579296 wait complete. 2024-05-20T18:37:06.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.088+0000 7f604a675640 1 Processor -- start 2024-05-20T18:37:06.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.089+0000 7f604a675640 1 -- start start 2024-05-20T18:37:06.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.090+0000 7f604a675640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60440b3e80 0x7f60441f4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:06.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.090+0000 7f604a675640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6044162820 0x7f6044158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:06.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.090+0000 7f604a675640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60441593b0 0x7f6044156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:06.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.090+0000 7f604a675640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f60440d4780 con 0x7f60441593b0 2024-05-20T18:37:06.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.090+0000 7f604a675640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f60440d0300 con 0x7f6044162820 2024-05-20T18:37:06.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.090+0000 7f604a675640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f60440d4900 con 0x7f60440b3e80 2024-05-20T18:37:06.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.090+0000 7f6048c12640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60441593b0 0x7f6044156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:06.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.090+0000 7f6048c12640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60441593b0 0x7f6044156b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35292/0 (socket says 172.21.15.5:35292) 2024-05-20T18:37:06.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.090+0000 7f6048c12640 1 -- 172.21.15.5:0/1806983514 learned_addr learned my addr 172.21.15.5:0/1806983514 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:06.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.090+0000 7f60437fe640 1 --2- 172.21.15.5:0/1806983514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6044162820 0x7f6044158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:06.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.090+0000 7f6043fff640 1 --2- 172.21.15.5:0/1806983514 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60440b3e80 0x7f60441f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:06.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.090+0000 7f6048c12640 1 -- 172.21.15.5:0/1806983514 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60440b3e80 msgr2=0x7f60441f4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:06.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.090+0000 7f6048c12640 1 --2- 172.21.15.5:0/1806983514 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60440b3e80 0x7f60441f4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:06.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.090+0000 7f6048c12640 1 -- 172.21.15.5:0/1806983514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6044162820 msgr2=0x7f6044158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:06.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.090+0000 7f6048c12640 1 --2- 172.21.15.5:0/1806983514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6044162820 0x7f6044158e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:06.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.090+0000 7f6048c12640 1 -- 172.21.15.5:0/1806983514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6044159ec0 con 0x7f60441593b0 2024-05-20T18:37:06.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.090+0000 7f6043fff640 1 --2- 172.21.15.5:0/1806983514 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60440b3e80 0x7f60441f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:06.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.091+0000 7f6048c12640 1 --2- 172.21.15.5:0/1806983514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60441593b0 0x7f6044156b60 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f603c02ac70 tx=0x7f603c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:06.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.091+0000 7f6040ff9640 1 -- 172.21.15.5:0/1806983514 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603c03f070 con 0x7f60441593b0 2024-05-20T18:37:06.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.091+0000 7f6040ff9640 1 -- 172.21.15.5:0/1806983514 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f603c031c90 con 0x7f60441593b0 2024-05-20T18:37:06.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.091+0000 7f6040ff9640 1 -- 172.21.15.5:0/1806983514 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603c044680 con 0x7f60441593b0 2024-05-20T18:37:06.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.091+0000 7f604a675640 1 -- 172.21.15.5:0/1806983514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6044159be0 con 0x7f60441593b0 2024-05-20T18:37:06.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.091+0000 7f604a675640 1 -- 172.21.15.5:0/1806983514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60440b85a0 con 0x7f60441593b0 2024-05-20T18:37:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.092+0000 7f604a675640 1 -- 172.21.15.5:0/1806983514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f604415d660 con 0x7f60441593b0 2024-05-20T18:37:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.093+0000 7f6040ff9640 1 -- 172.21.15.5:0/1806983514 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f603c03b410 con 0x7f60441593b0 2024-05-20T18:37:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.093+0000 7f6040ff9640 1 --2- 172.21.15.5:0/1806983514 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f603006be30 0x7f603006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.093+0000 7f6040ff9640 1 -- 172.21.15.5:0/1806983514 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5421+0+0 (secure 0 0 0) 0x7f603c0ac5f0 con 0x7f60441593b0 2024-05-20T18:37:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.093+0000 7f6043fff640 1 --2- 172.21.15.5:0/1806983514 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f603006be30 0x7f603006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.094+0000 7f6043fff640 1 --2- 172.21.15.5:0/1806983514 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f603006be30 0x7f603006e340 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f603400a990 tx=0x7f6034008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.098+0000 7f6040ff9640 1 -- 172.21.15.5:0/1806983514 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f603c07b180 con 0x7f60441593b0 2024-05-20T18:37:06.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.280+0000 7f604a675640 1 -- 172.21.15.5:0/1806983514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f6044075b00 con 0x7f60441593b0 2024-05-20T18:37:06.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.284+0000 7f6040ff9640 1 -- 172.21.15.5:0/1806983514 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7f603c07aa80 con 0x7f60441593b0 2024-05-20T18:37:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.286+0000 7f604a675640 1 -- 172.21.15.5:0/1806983514 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f603006be30 msgr2=0x7f603006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.286+0000 7f604a675640 1 --2- 172.21.15.5:0/1806983514 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f603006be30 0x7f603006e340 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f603400a990 tx=0x7f6034008040 comp rx=0 tx=0).stop 2024-05-20T18:37:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.286+0000 7f604a675640 1 -- 172.21.15.5:0/1806983514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60441593b0 msgr2=0x7f6044156b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.286+0000 7f604a675640 1 --2- 172.21.15.5:0/1806983514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60441593b0 0x7f6044156b60 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f603c02ac70 tx=0x7f603c004a40 comp rx=0 tx=0).stop 2024-05-20T18:37:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.286+0000 7f604a675640 1 -- 172.21.15.5:0/1806983514 shutdown_connections 2024-05-20T18:37:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.286+0000 7f604a675640 1 --2- 172.21.15.5:0/1806983514 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f603006be30 0x7f603006e340 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.286+0000 7f604a675640 1 --2- 172.21.15.5:0/1806983514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60441593b0 0x7f6044156b60 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.286+0000 7f604a675640 1 --2- 172.21.15.5:0/1806983514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6044162820 0x7f6044158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.286+0000 7f604a675640 1 --2- 172.21.15.5:0/1806983514 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60440b3e80 0x7f60441f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:06.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.286+0000 7f604a675640 1 -- 172.21.15.5:0/1806983514 >> 172.21.15.5:0/1806983514 conn(0x7f60440d4bd0 msgr2=0x7f6044156230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:06.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.286+0000 7f604a675640 1 -- 172.21.15.5:0/1806983514 shutdown_connections 2024-05-20T18:37:06.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.286+0000 7f604a675640 1 -- 172.21.15.5:0/1806983514 wait complete. 2024-05-20T18:37:06.299 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-05-20T18:37:06.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.414+0000 7f76d0565640 1 Processor -- start 2024-05-20T18:37:06.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.415+0000 7f76d0565640 1 -- start start 2024-05-20T18:37:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.416+0000 7f76d0565640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f76c8162820 0x7f76c80b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.416+0000 7f76d0565640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76c80b3e80 0x7f76c806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:06.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.416+0000 7f76d0565640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f76c806d910 0x7f76c80d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:06.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.416+0000 7f76d0565640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f76c806dd20 con 0x7f76c8162820 2024-05-20T18:37:06.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.416+0000 7f76d0565640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f76c80d0630 con 0x7f76c806d910 2024-05-20T18:37:06.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.416+0000 7f76d0565640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f76c80d39c0 con 0x7f76c80b3e80 2024-05-20T18:37:06.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.416+0000 7f76ce301640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f76c8162820 0x7f76c80b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:06.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.416+0000 7f76ce301640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f76c8162820 0x7f76c80b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35306/0 (socket says 172.21.15.5:35306) 2024-05-20T18:37:06.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.416+0000 7f76ce301640 1 -- 172.21.15.5:0/1864999804 learned_addr learned my addr 172.21.15.5:0/1864999804 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:06.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.416+0000 7f76cdb00640 1 --2- 172.21.15.5:0/1864999804 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76c80b3e80 0x7f76c806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:06.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.416+0000 7f76ceb02640 1 --2- 172.21.15.5:0/1864999804 >> v2:172.21.15.63:3300/0 conn(0x7f76c806d910 0x7f76c80d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:06.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.417+0000 7f76ce301640 1 -- 172.21.15.5:0/1864999804 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76c80b3e80 msgr2=0x7f76c806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:06.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.417+0000 7f76ce301640 1 --2- 172.21.15.5:0/1864999804 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76c80b3e80 0x7f76c806d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:06.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.417+0000 7f76ce301640 1 -- 172.21.15.5:0/1864999804 >> v2:172.21.15.63:3300/0 conn(0x7f76c806d910 msgr2=0x7f76c80d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:06.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.417+0000 7f76ce301640 1 --2- 172.21.15.5:0/1864999804 >> v2:172.21.15.63:3300/0 conn(0x7f76c806d910 0x7f76c80d27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:06.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.417+0000 7f76ce301640 1 -- 172.21.15.5:0/1864999804 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c80d3ca0 con 0x7f76c8162820 2024-05-20T18:37:06.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.417+0000 7f76cdb00640 1 --2- 172.21.15.5:0/1864999804 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76c80b3e80 0x7f76c806d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:06.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.417+0000 7f76ceb02640 1 --2- 172.21.15.5:0/1864999804 >> v2:172.21.15.63:3300/0 conn(0x7f76c806d910 0x7f76c80d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:06.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.417+0000 7f76ce301640 1 --2- 172.21.15.5:0/1864999804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76c8162820 0x7f76c80b3940 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f76c808d5e0 tx=0x7f76c402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43c7d904dc1efc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:06.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.418+0000 7f76cd2ff640 1 -- 172.21.15.5:0/1864999804 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c4037070 con 0x7f76c8162820 2024-05-20T18:37:06.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.418+0000 7f76cd2ff640 1 -- 172.21.15.5:0/1864999804 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76c4032440 con 0x7f76c8162820 2024-05-20T18:37:06.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.418+0000 7f76cd2ff640 1 -- 172.21.15.5:0/1864999804 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c4032810 con 0x7f76c8162820 2024-05-20T18:37:06.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.418+0000 7f76d0565640 1 -- 172.21.15.5:0/1864999804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76c8162820 msgr2=0x7f76c80b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:06.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.418+0000 7f76d0565640 1 --2- 172.21.15.5:0/1864999804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76c8162820 0x7f76c80b3940 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f76c808d5e0 tx=0x7f76c402a1e0 comp rx=0 tx=0).stop 2024-05-20T18:37:06.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.418+0000 7f76d0565640 1 -- 172.21.15.5:0/1864999804 shutdown_connections 2024-05-20T18:37:06.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.418+0000 7f76d0565640 1 --2- 172.21.15.5:0/1864999804 >> v2:172.21.15.63:3300/0 conn(0x7f76c806d910 0x7f76c80d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:06.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.418+0000 7f76d0565640 1 --2- 172.21.15.5:0/1864999804 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76c80b3e80 0x7f76c806d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:06.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.418+0000 7f76d0565640 1 --2- 172.21.15.5:0/1864999804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76c8162820 0x7f76c80b3940 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:06.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.418+0000 7f76d0565640 1 -- 172.21.15.5:0/1864999804 >> 172.21.15.5:0/1864999804 conn(0x7f76c80d4bd0 msgr2=0x7f76c80d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:06.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.418+0000 7f76d0565640 1 -- 172.21.15.5:0/1864999804 shutdown_connections 2024-05-20T18:37:06.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.418+0000 7f76d0565640 1 -- 172.21.15.5:0/1864999804 wait complete. 2024-05-20T18:37:06.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.420+0000 7f76d0565640 1 Processor -- start 2024-05-20T18:37:06.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.420+0000 7f76d0565640 1 -- start start 2024-05-20T18:37:06.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.421+0000 7f76d0565640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76c806d910 0x7f76c80b6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:06.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.421+0000 7f76d0565640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76c80b3e80 0x7f76c81f8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:06.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.421+0000 7f76d0565640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76c8162820 0x7f76c81f93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:06.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.421+0000 7f76d0565640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f76c80d0300 con 0x7f76c8162820 2024-05-20T18:37:06.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.421+0000 7f76d0565640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f76c80d4260 con 0x7f76c80b3e80 2024-05-20T18:37:06.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.421+0000 7f76d0565640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f76c80d4780 con 0x7f76c806d910 2024-05-20T18:37:06.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.422+0000 7f76ceb02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76c8162820 0x7f76c81f93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:06.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.422+0000 7f76ceb02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76c8162820 0x7f76c81f93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35312/0 (socket says 172.21.15.5:35312) 2024-05-20T18:37:06.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.422+0000 7f76ceb02640 1 -- 172.21.15.5:0/36651550 learned_addr learned my addr 172.21.15.5:0/36651550 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:06.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.422+0000 7f76cdb00640 1 --2- 172.21.15.5:0/36651550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76c80b3e80 0x7f76c81f8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:06.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.422+0000 7f76ce301640 1 --2- 172.21.15.5:0/36651550 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76c806d910 0x7f76c80b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:06.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.422+0000 7f76ceb02640 1 -- 172.21.15.5:0/36651550 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76c806d910 msgr2=0x7f76c80b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.422+0000 7f76ceb02640 1 --2- 172.21.15.5:0/36651550 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76c806d910 0x7f76c80b6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.422+0000 7f76ceb02640 1 -- 172.21.15.5:0/36651550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76c80b3e80 msgr2=0x7f76c81f8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.422+0000 7f76ceb02640 1 --2- 172.21.15.5:0/36651550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76c80b3e80 0x7f76c81f8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.422+0000 7f76ceb02640 1 -- 172.21.15.5:0/36651550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c80cbe20 con 0x7f76c8162820 2024-05-20T18:37:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.422+0000 7f76cdb00640 1 --2- 172.21.15.5:0/36651550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76c80b3e80 0x7f76c81f8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.422+0000 7f76ce301640 1 --2- 172.21.15.5:0/36651550 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76c806d910 0x7f76c80b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.422+0000 7f76ceb02640 1 --2- 172.21.15.5:0/36651550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76c8162820 0x7f76c81f93d0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f76c000ccc0 tx=0x7f76c000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.422+0000 7f76beffd640 1 -- 172.21.15.5:0/36651550 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c000f680 con 0x7f76c8162820 2024-05-20T18:37:06.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.422+0000 7f76beffd640 1 -- 172.21.15.5:0/36651550 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76c000f820 con 0x7f76c8162820 2024-05-20T18:37:06.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.422+0000 7f76d0565640 1 -- 172.21.15.5:0/36651550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c80cc100 con 0x7f76c8162820 2024-05-20T18:37:06.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.422+0000 7f76beffd640 1 -- 172.21.15.5:0/36651550 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c000fc10 con 0x7f76c8162820 2024-05-20T18:37:06.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.422+0000 7f76d0565640 1 -- 172.21.15.5:0/36651550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c80cc6c0 con 0x7f76c8162820 2024-05-20T18:37:06.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.424+0000 7f76d0565640 1 -- 172.21.15.5:0/36651550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c8075b00 con 0x7f76c8162820 2024-05-20T18:37:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.424+0000 7f76beffd640 1 -- 172.21.15.5:0/36651550 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f76c00213f0 con 0x7f76c8162820 2024-05-20T18:37:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.425+0000 7f76beffd640 1 --2- 172.21.15.5:0/36651550 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f76ac06be30 0x7f76ac06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.425+0000 7f76ce301640 1 --2- 172.21.15.5:0/36651550 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f76ac06be30 0x7f76ac06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.425+0000 7f76beffd640 1 -- 172.21.15.5:0/36651550 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5421+0+0 (secure 0 0 0) 0x7f76c008ba80 con 0x7f76c8162820 2024-05-20T18:37:06.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.425+0000 7f76ce301640 1 --2- 172.21.15.5:0/36651550 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f76ac06be30 0x7f76ac06e340 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f76c4034000 tx=0x7f76c402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:06.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.430+0000 7f76beffd640 1 -- 172.21.15.5:0/36651550 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f76c0014030 con 0x7f76c8162820 2024-05-20T18:37:06.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.684+0000 7f76d0565640 1 -- 172.21.15.5:0/36651550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7f76c80d00e0 con 0x7f76c8162820 2024-05-20T18:37:06.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.942+0000 7f76beffd640 1 -- 172.21.15.5:0/36651550 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v472) ==== 146+0+0 (secure 0 0 0) 0x7f76c0059e70 con 0x7f76c8162820 2024-05-20T18:37:06.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.944+0000 7f76d0565640 1 -- 172.21.15.5:0/36651550 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f76ac06be30 msgr2=0x7f76ac06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:06.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.944+0000 7f76d0565640 1 --2- 172.21.15.5:0/36651550 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f76ac06be30 0x7f76ac06e340 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f76c4034000 tx=0x7f76c402e000 comp rx=0 tx=0).stop 2024-05-20T18:37:06.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.944+0000 7f76d0565640 1 -- 172.21.15.5:0/36651550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76c8162820 msgr2=0x7f76c81f93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:06.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.944+0000 7f76d0565640 1 --2- 172.21.15.5:0/36651550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76c8162820 0x7f76c81f93d0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f76c000ccc0 tx=0x7f76c000a650 comp rx=0 tx=0).stop 2024-05-20T18:37:06.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.944+0000 7f76d0565640 1 -- 172.21.15.5:0/36651550 shutdown_connections 2024-05-20T18:37:06.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.944+0000 7f76d0565640 1 --2- 172.21.15.5:0/36651550 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f76ac06be30 0x7f76ac06e340 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:06.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.944+0000 7f76d0565640 1 --2- 172.21.15.5:0/36651550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76c8162820 0x7f76c81f93d0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:06.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.944+0000 7f76d0565640 1 --2- 172.21.15.5:0/36651550 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76c80b3e80 0x7f76c81f8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:06.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.944+0000 7f76d0565640 1 --2- 172.21.15.5:0/36651550 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76c806d910 0x7f76c80b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:06.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.944+0000 7f76d0565640 1 -- 172.21.15.5:0/36651550 >> 172.21.15.5:0/36651550 conn(0x7f76c80d4bd0 msgr2=0x7f76c80ba560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:06.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.944+0000 7f76d0565640 1 -- 172.21.15.5:0/36651550 shutdown_connections 2024-05-20T18:37:06.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:06.944+0000 7f76d0565640 1 -- 172.21.15.5:0/36651550 wait complete. 2024-05-20T18:37:06.957 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-05-20T18:37:06.957 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-20T18:37:06.958 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:37:07.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.082+0000 7f2d113d5640 1 Processor -- start 2024-05-20T18:37:07.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.082+0000 7f2d113d5640 1 -- start start 2024-05-20T18:37:07.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.083+0000 7f2d113d5640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2d0c159aa0 0x7f2d0c06da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:07.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.083+0000 7f2d113d5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2d0c0d1580 0x7f2d0c0d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:07.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.083+0000 7f2d113d5640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d0c0d1ed0 con 0x7f2d0c09a4f0 2024-05-20T18:37:07.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.083+0000 7f2d113d5640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f2d0c160630 con 0x7f2d0c0d1580 2024-05-20T18:37:07.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.083+0000 7f2d113d5640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2d0c15fa20 con 0x7f2d0c159aa0 2024-05-20T18:37:07.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.083+0000 7f2d027fc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2d0c0d1580 0x7f2d0c0d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:07.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.083+0000 7f2d027fc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2d0c0d1580 0x7f2d0c0d1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35318/0 (socket says 172.21.15.5:35318) 2024-05-20T18:37:07.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.083+0000 7f2d027fc640 1 -- 172.21.15.5:0/681164119 learned_addr learned my addr 172.21.15.5:0/681164119 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:07.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.084+0000 7f2d0affd640 1 --2- 172.21.15.5:0/681164119 >> v2:172.21.15.63:3300/0 conn(0x7f2d0c159aa0 0x7f2d0c06da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:07.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.084+0000 7f2d027fc640 1 -- 172.21.15.5:0/681164119 >> v2:172.21.15.63:3300/0 conn(0x7f2d0c159aa0 msgr2=0x7f2d0c06da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:07.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.084+0000 7f2d027fc640 1 --2- 172.21.15.5:0/681164119 >> v2:172.21.15.63:3300/0 conn(0x7f2d0c159aa0 0x7f2d0c06da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:07.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.084+0000 7f2d027fc640 1 -- 172.21.15.5:0/681164119 >> v1:172.21.15.63:6789/0 conn(0x7f2d0c09a4f0 legacy=0x7f2d0c09a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:07.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.084+0000 7f2d027fc640 1 -- 172.21.15.5:0/681164119 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d0c15fd00 con 0x7f2d0c0d1580 2024-05-20T18:37:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.084+0000 7f2d027fc640 1 --2- 172.21.15.5:0/681164119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d0c0d1580 0x7f2d0c0d1990 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c072730 tx=0x7f2cf402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8919f57dea4e1f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.085+0000 7f2d0a7fc640 1 -- 172.21.15.5:0/681164119 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4037070 con 0x7f2d0c0d1580 2024-05-20T18:37:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.085+0000 7f2d0a7fc640 1 -- 172.21.15.5:0/681164119 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cf4032440 con 0x7f2d0c0d1580 2024-05-20T18:37:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.085+0000 7f2d0a7fc640 1 -- 172.21.15.5:0/681164119 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4032810 con 0x7f2d0c0d1580 2024-05-20T18:37:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.085+0000 7f2d113d5640 1 -- 172.21.15.5:0/681164119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d0c0d1580 msgr2=0x7f2d0c0d1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.085+0000 7f2d113d5640 1 --2- 172.21.15.5:0/681164119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d0c0d1580 0x7f2d0c0d1990 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c072730 tx=0x7f2cf402a270 comp rx=0 tx=0).stop 2024-05-20T18:37:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.085+0000 7f2d113d5640 1 -- 172.21.15.5:0/681164119 shutdown_connections 2024-05-20T18:37:07.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.085+0000 7f2d113d5640 1 --2- 172.21.15.5:0/681164119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d0c0d1580 0x7f2d0c0d1990 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:07.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.085+0000 7f2d113d5640 1 --2- 172.21.15.5:0/681164119 >> v2:172.21.15.63:3300/0 conn(0x7f2d0c159aa0 0x7f2d0c06da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:07.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.085+0000 7f2d113d5640 1 -- 172.21.15.5:0/681164119 >> 172.21.15.5:0/681164119 conn(0x7f2d0c0b6620 msgr2=0x7f2d0c0b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:07.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.086+0000 7f2d113d5640 1 -- 172.21.15.5:0/681164119 shutdown_connections 2024-05-20T18:37:07.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.086+0000 7f2d113d5640 1 -- 172.21.15.5:0/681164119 wait complete. 2024-05-20T18:37:07.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.087+0000 7f2d113d5640 1 Processor -- start 2024-05-20T18:37:07.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.087+0000 7f2d113d5640 1 -- start start 2024-05-20T18:37:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.089+0000 7f2d113d5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d0c09a4f0 0x7f2d0c1f4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.089+0000 7f2d113d5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2d0c0d1580 0x7f2d0c1570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.089+0000 7f2d113d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d0c159aa0 0x7f2d0c157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.089+0000 7f2d113d5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c06dfc0 con 0x7f2d0c159aa0 2024-05-20T18:37:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.089+0000 7f2d113d5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c1602c0 con 0x7f2d0c09a4f0 2024-05-20T18:37:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.089+0000 7f2d113d5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2d0c0b1420 con 0x7f2d0c0d1580 2024-05-20T18:37:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.089+0000 7f2d0b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d0c159aa0 0x7f2d0c157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.089+0000 7f2d0b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d0c159aa0 0x7f2d0c157610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35324/0 (socket says 172.21.15.5:35324) 2024-05-20T18:37:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.089+0000 7f2d0b7fe640 1 -- 172.21.15.5:0/1775248173 learned_addr learned my addr 172.21.15.5:0/1775248173 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.089+0000 7f2d0affd640 1 --2- 172.21.15.5:0/1775248173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d0c09a4f0 0x7f2d0c1f4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.089+0000 7f2d027fc640 1 --2- 172.21.15.5:0/1775248173 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2d0c0d1580 0x7f2d0c1570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.089+0000 7f2d0b7fe640 1 -- 172.21.15.5:0/1775248173 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2d0c0d1580 msgr2=0x7f2d0c1570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.089+0000 7f2d0b7fe640 1 --2- 172.21.15.5:0/1775248173 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2d0c0d1580 0x7f2d0c1570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.089+0000 7f2d0b7fe640 1 -- 172.21.15.5:0/1775248173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d0c09a4f0 msgr2=0x7f2d0c1f4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.089+0000 7f2d0b7fe640 1 --2- 172.21.15.5:0/1775248173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d0c09a4f0 0x7f2d0c1f4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.089+0000 7f2d0b7fe640 1 -- 172.21.15.5:0/1775248173 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d0c1f8cd0 con 0x7f2d0c159aa0 2024-05-20T18:37:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.090+0000 7f2d0b7fe640 1 --2- 172.21.15.5:0/1775248173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d0c159aa0 0x7f2d0c157610 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400c6b0 tx=0x7f2d0400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.090+0000 7f2d03fff640 1 -- 172.21.15.5:0/1775248173 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d04003b70 con 0x7f2d0c159aa0 2024-05-20T18:37:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.090+0000 7f2d03fff640 1 -- 172.21.15.5:0/1775248173 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d04003d10 con 0x7f2d0c159aa0 2024-05-20T18:37:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.090+0000 7f2d113d5640 1 -- 172.21.15.5:0/1775248173 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d0c1f8fb0 con 0x7f2d0c159aa0 2024-05-20T18:37:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.090+0000 7f2d03fff640 1 -- 172.21.15.5:0/1775248173 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d04005310 con 0x7f2d0c159aa0 2024-05-20T18:37:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.090+0000 7f2d113d5640 1 -- 172.21.15.5:0/1775248173 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d0c1f51f0 con 0x7f2d0c159aa0 2024-05-20T18:37:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.091+0000 7f2d113d5640 1 -- 172.21.15.5:0/1775248173 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d0c075b00 con 0x7f2d0c159aa0 2024-05-20T18:37:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.091+0000 7f2d03fff640 1 -- 172.21.15.5:0/1775248173 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2d040054b0 con 0x7f2d0c159aa0 2024-05-20T18:37:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.092+0000 7f2d03fff640 1 --2- 172.21.15.5:0/1775248173 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2cec06be60 0x7f2cec06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.092+0000 7f2d03fff640 1 -- 172.21.15.5:0/1775248173 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5421+0+0 (secure 0 0 0) 0x7f2d0408aab0 con 0x7f2d0c159aa0 2024-05-20T18:37:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.092+0000 7f2d0affd640 1 --2- 172.21.15.5:0/1775248173 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2cec06be60 0x7f2cec06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.092+0000 7f2d0affd640 1 --2- 172.21.15.5:0/1775248173 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2cec06be60 0x7f2cec06e370 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c073320 tx=0x7f2cfc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.097+0000 7f2d03fff640 1 -- 172.21.15.5:0/1775248173 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2d0c0d1ed0 con 0x7f2d0c159aa0 2024-05-20T18:37:07.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.280+0000 7f2d113d5640 1 -- 172.21.15.5:0/1775248173 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2d0c157c00 con 0x7f2d0c159aa0 2024-05-20T18:37:07.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.281+0000 7f2d03fff640 1 -- 172.21.15.5:0/1775248173 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) ==== 74+0+13311 (secure 0 0 0) 0x7f2d04058eb0 con 0x7f2d0c159aa0 2024-05-20T18:37:07.283 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:07.283 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":111,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:37:05.933946+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6835/3552410668":"2024-05-21T18:36:57.897254+0000","172.21.15.5:6834/3552410668":"2024-05-21T18:36:57.897254+0000","172.21.15.63:6835/2054897241":"2024-05-21T18:36:21.346248+0000","172.21.15.63:6833/2054897241":"2024-05-21T18:36:21.346248+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:37:07.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.283+0000 7f2d113d5640 1 -- 172.21.15.5:0/1775248173 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2cec06be60 msgr2=0x7f2cec06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:07.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.283+0000 7f2d113d5640 1 --2- 172.21.15.5:0/1775248173 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2cec06be60 0x7f2cec06e370 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c073320 tx=0x7f2cfc008040 comp rx=0 tx=0).stop 2024-05-20T18:37:07.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.283+0000 7f2d113d5640 1 -- 172.21.15.5:0/1775248173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d0c159aa0 msgr2=0x7f2d0c157610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:07.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.283+0000 7f2d113d5640 1 --2- 172.21.15.5:0/1775248173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d0c159aa0 0x7f2d0c157610 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400c6b0 tx=0x7f2d0400cb70 comp rx=0 tx=0).stop 2024-05-20T18:37:07.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.284+0000 7f2d113d5640 1 -- 172.21.15.5:0/1775248173 shutdown_connections 2024-05-20T18:37:07.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.284+0000 7f2d113d5640 1 --2- 172.21.15.5:0/1775248173 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2cec06be60 0x7f2cec06e370 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:07.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.284+0000 7f2d113d5640 1 --2- 172.21.15.5:0/1775248173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d0c159aa0 0x7f2d0c157610 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:07.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.284+0000 7f2d113d5640 1 --2- 172.21.15.5:0/1775248173 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2d0c0d1580 0x7f2d0c1570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:07.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.284+0000 7f2d113d5640 1 --2- 172.21.15.5:0/1775248173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d0c09a4f0 0x7f2d0c1f4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:07.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.284+0000 7f2d113d5640 1 -- 172.21.15.5:0/1775248173 >> 172.21.15.5:0/1775248173 conn(0x7f2d0c0b6620 msgr2=0x7f2d0c150240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:07.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.284+0000 7f2d113d5640 1 -- 172.21.15.5:0/1775248173 shutdown_connections 2024-05-20T18:37:07.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.284+0000 7f2d113d5640 1 -- 172.21.15.5:0/1775248173 wait complete. 2024-05-20T18:37:07.297 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:37:07.297 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:37:07.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.414+0000 7fcdc89b5640 1 Processor -- start 2024-05-20T18:37:07.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.414+0000 7fcdc89b5640 1 -- start start 2024-05-20T18:37:07.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.415+0000 7fcdc89b5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcdc4162820 0x7fcdc40b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:07.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.415+0000 7fcdc89b5640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fcdc406dae0 0x7fcdc40b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:07.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.415+0000 7fcdc89b5640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdc40d0630 con 0x7fcdc40b3e80 2024-05-20T18:37:07.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.415+0000 7fcdc89b5640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fcdc40d39c0 con 0x7fcdc4162820 2024-05-20T18:37:07.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.415+0000 7fcdc89b5640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fcdc40d3ca0 con 0x7fcdc406dae0 2024-05-20T18:37:07.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.415+0000 7fcdc259c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcdc4162820 0x7fcdc40b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:07.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.415+0000 7fcdc259c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcdc4162820 0x7fcdc40b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35326/0 (socket says 172.21.15.5:35326) 2024-05-20T18:37:07.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.415+0000 7fcdc259c640 1 -- 172.21.15.5:0/2189748570 learned_addr learned my addr 172.21.15.5:0/2189748570 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:07.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.416+0000 7fcdc2d9d640 1 --2- 172.21.15.5:0/2189748570 >> v2:172.21.15.63:3300/0 conn(0x7fcdc406dae0 0x7fcdc40b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:07.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.416+0000 7fcdc259c640 1 -- 172.21.15.5:0/2189748570 >> v2:172.21.15.63:3300/0 conn(0x7fcdc406dae0 msgr2=0x7fcdc40b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:07.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.416+0000 7fcdc259c640 1 --2- 172.21.15.5:0/2189748570 >> v2:172.21.15.63:3300/0 conn(0x7fcdc406dae0 0x7fcdc40b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:07.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.416+0000 7fcdc259c640 1 -- 172.21.15.5:0/2189748570 >> v1:172.21.15.63:6789/0 conn(0x7fcdc40b3e80 legacy=0x7fcdc406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:07.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.416+0000 7fcdc259c640 1 -- 172.21.15.5:0/2189748570 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc40d3f80 con 0x7fcdc4162820 2024-05-20T18:37:07.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.417+0000 7fcdc259c640 1 --2- 172.21.15.5:0/2189748570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdc4162820 0x7fcdc40b3940 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fcdc408e070 tx=0x7fcdb402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ab2e98d055d849 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:07.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.417+0000 7fcdc1d9b640 1 -- 172.21.15.5:0/2189748570 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4037070 con 0x7fcdc4162820 2024-05-20T18:37:07.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.417+0000 7fcdc1d9b640 1 -- 172.21.15.5:0/2189748570 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdb4032440 con 0x7fcdc4162820 2024-05-20T18:37:07.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.417+0000 7fcdc1d9b640 1 -- 172.21.15.5:0/2189748570 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4032810 con 0x7fcdc4162820 2024-05-20T18:37:07.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.417+0000 7fcdc89b5640 1 -- 172.21.15.5:0/2189748570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdc4162820 msgr2=0x7fcdc40b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:07.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.417+0000 7fcdc89b5640 1 --2- 172.21.15.5:0/2189748570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdc4162820 0x7fcdc40b3940 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fcdc408e070 tx=0x7fcdb402a230 comp rx=0 tx=0).stop 2024-05-20T18:37:07.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.417+0000 7fcdc89b5640 1 -- 172.21.15.5:0/2189748570 shutdown_connections 2024-05-20T18:37:07.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.417+0000 7fcdc89b5640 1 --2- 172.21.15.5:0/2189748570 >> v2:172.21.15.63:3300/0 conn(0x7fcdc406dae0 0x7fcdc40b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:07.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.417+0000 7fcdc89b5640 1 --2- 172.21.15.5:0/2189748570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdc4162820 0x7fcdc40b3940 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:07.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.417+0000 7fcdc89b5640 1 -- 172.21.15.5:0/2189748570 >> 172.21.15.5:0/2189748570 conn(0x7fcdc40d4bd0 msgr2=0x7fcdc40d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:07.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.418+0000 7fcdc89b5640 1 -- 172.21.15.5:0/2189748570 shutdown_connections 2024-05-20T18:37:07.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.418+0000 7fcdc89b5640 1 -- 172.21.15.5:0/2189748570 wait complete. 2024-05-20T18:37:07.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.420+0000 7fcdc89b5640 1 Processor -- start 2024-05-20T18:37:07.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.420+0000 7fcdc89b5640 1 -- start start 2024-05-20T18:37:07.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.421+0000 7fcdc89b5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcdc40b3e80 0x7fcdc41520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:07.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.421+0000 7fcdc89b5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdc4162820 0x7fcdc41f4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:07.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.421+0000 7fcdc89b5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcdc415b0e0 0x7fcdc40c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:07.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.421+0000 7fcdc89b5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc40d4780 con 0x7fcdc4162820 2024-05-20T18:37:07.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.421+0000 7fcdc89b5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc40d0300 con 0x7fcdc415b0e0 2024-05-20T18:37:07.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.421+0000 7fcdc89b5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcdc40d4900 con 0x7fcdc40b3e80 2024-05-20T18:37:07.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.421+0000 7fcdb9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdc4162820 0x7fcdc41f4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:07.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.421+0000 7fcdb9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdc4162820 0x7fcdc41f4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35340/0 (socket says 172.21.15.5:35340) 2024-05-20T18:37:07.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.421+0000 7fcdb9d9b640 1 -- 172.21.15.5:0/2539375280 learned_addr learned my addr 172.21.15.5:0/2539375280 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:07.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.421+0000 7fcdc259c640 1 --2- 172.21.15.5:0/2539375280 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcdc40b3e80 0x7fcdc41520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:07.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.421+0000 7fcdc2d9d640 1 --2- 172.21.15.5:0/2539375280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcdc415b0e0 0x7fcdc40c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:07.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.422+0000 7fcdb9d9b640 1 -- 172.21.15.5:0/2539375280 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcdc40b3e80 msgr2=0x7fcdc41520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:07.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.422+0000 7fcdb9d9b640 1 --2- 172.21.15.5:0/2539375280 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcdc40b3e80 0x7fcdc41520d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:07.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.422+0000 7fcdb9d9b640 1 -- 172.21.15.5:0/2539375280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcdc415b0e0 msgr2=0x7fcdc40c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:07.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.422+0000 7fcdb9d9b640 1 --2- 172.21.15.5:0/2539375280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcdc415b0e0 0x7fcdc40c99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:07.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.422+0000 7fcdb9d9b640 1 -- 172.21.15.5:0/2539375280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc40c8950 con 0x7fcdc4162820 2024-05-20T18:37:07.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.422+0000 7fcdb9d9b640 1 --2- 172.21.15.5:0/2539375280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdc4162820 0x7fcdc41f4fc0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00b420 tx=0x7fcdac00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:07.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.422+0000 7fcdbb7fe640 1 -- 172.21.15.5:0/2539375280 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac011830 con 0x7fcdc4162820 2024-05-20T18:37:07.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.422+0000 7fcdbb7fe640 1 -- 172.21.15.5:0/2539375280 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdac016070 con 0x7fcdc4162820 2024-05-20T18:37:07.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.422+0000 7fcdc89b5640 1 -- 172.21.15.5:0/2539375280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdc40c8c30 con 0x7fcdc4162820 2024-05-20T18:37:07.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.422+0000 7fcdbb7fe640 1 -- 172.21.15.5:0/2539375280 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac011c10 con 0x7fcdc4162820 2024-05-20T18:37:07.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.422+0000 7fcdc89b5640 1 -- 172.21.15.5:0/2539375280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdc40c91f0 con 0x7fcdc4162820 2024-05-20T18:37:07.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.423+0000 7fcdc89b5640 1 -- 172.21.15.5:0/2539375280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdc4075b00 con 0x7fcdc4162820 2024-05-20T18:37:07.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.424+0000 7fcdbb7fe640 1 -- 172.21.15.5:0/2539375280 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcdac01b460 con 0x7fcdc4162820 2024-05-20T18:37:07.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.424+0000 7fcdbb7fe640 1 --2- 172.21.15.5:0/2539375280 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcda406be30 0x7fcda406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:07.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.424+0000 7fcdbb7fe640 1 -- 172.21.15.5:0/2539375280 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5421+0+0 (secure 0 0 0) 0x7fcdac08a980 con 0x7fcdc4162820 2024-05-20T18:37:07.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.424+0000 7fcdc259c640 1 --2- 172.21.15.5:0/2539375280 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcda406be30 0x7fcda406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:07.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.425+0000 7fcdc259c640 1 --2- 172.21.15.5:0/2539375280 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcda406be30 0x7fcda406e340 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4034000 tx=0x7fcdb402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:07.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.429+0000 7fcdbb7fe640 1 -- 172.21.15.5:0/2539375280 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fcdac058cf0 con 0x7fcdc4162820 2024-05-20T18:37:07.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.611+0000 7fcdc89b5640 1 -- 172.21.15.5:0/2539375280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcdc41f5500 con 0x7fcdc4162820 2024-05-20T18:37:07.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.612+0000 7fcdbb7fe640 1 -- 172.21.15.5:0/2539375280 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v111) ==== 93+0+10 (secure 0 0 0) 0x7fcdac05cff0 con 0x7fcdc4162820 2024-05-20T18:37:07.614 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:37:07.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.614+0000 7fcdc89b5640 1 -- 172.21.15.5:0/2539375280 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcda406be30 msgr2=0x7fcda406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:07.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.614+0000 7fcdc89b5640 1 --2- 172.21.15.5:0/2539375280 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcda406be30 0x7fcda406e340 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4034000 tx=0x7fcdb402e000 comp rx=0 tx=0).stop 2024-05-20T18:37:07.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.614+0000 7fcdc89b5640 1 -- 172.21.15.5:0/2539375280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdc4162820 msgr2=0x7fcdc41f4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:07.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.614+0000 7fcdc89b5640 1 --2- 172.21.15.5:0/2539375280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdc4162820 0x7fcdc41f4fc0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00b420 tx=0x7fcdac00b8e0 comp rx=0 tx=0).stop 2024-05-20T18:37:07.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.614+0000 7fcdc89b5640 1 -- 172.21.15.5:0/2539375280 shutdown_connections 2024-05-20T18:37:07.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.614+0000 7fcdc89b5640 1 --2- 172.21.15.5:0/2539375280 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcda406be30 0x7fcda406e340 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:07.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.614+0000 7fcdc89b5640 1 --2- 172.21.15.5:0/2539375280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcdc415b0e0 0x7fcdc40c99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:07.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.614+0000 7fcdc89b5640 1 --2- 172.21.15.5:0/2539375280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdc4162820 0x7fcdc41f4fc0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:07.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.614+0000 7fcdc89b5640 1 --2- 172.21.15.5:0/2539375280 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcdc40b3e80 0x7fcdc41520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:07.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.614+0000 7fcdc89b5640 1 -- 172.21.15.5:0/2539375280 >> 172.21.15.5:0/2539375280 conn(0x7fcdc40d4bd0 msgr2=0x7fcdc415e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:07.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.614+0000 7fcdc89b5640 1 -- 172.21.15.5:0/2539375280 shutdown_connections 2024-05-20T18:37:07.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.614+0000 7fcdc89b5640 1 -- 172.21.15.5:0/2539375280 wait complete. 2024-05-20T18:37:07.627 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-05-20T18:37:07.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.741+0000 7f7c79eca640 1 Processor -- start 2024-05-20T18:37:07.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.741+0000 7f7c79eca640 1 -- start start 2024-05-20T18:37:07.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.742+0000 7f7c79eca640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c74162820 0x7f7c740b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:07.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.742+0000 7f7c79eca640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7c7406dae0 0x7f7c740b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:07.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.742+0000 7f7c79eca640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c740d0630 con 0x7f7c740b3e80 2024-05-20T18:37:07.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.742+0000 7f7c79eca640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7c740d39c0 con 0x7f7c7406dae0 2024-05-20T18:37:07.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.742+0000 7f7c79eca640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7c740d3ca0 con 0x7f7c74162820 2024-05-20T18:37:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.743+0000 7f7c73fff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7c7406dae0 0x7f7c740b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.743+0000 7f7c737fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c74162820 0x7f7c740b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.743+0000 7f7c737fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c74162820 0x7f7c740b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:45910/0 (socket says 172.21.15.5:45910) 2024-05-20T18:37:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.743+0000 7f7c737fe640 1 -- 172.21.15.5:0/171827940 learned_addr learned my addr 172.21.15.5:0/171827940 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.743+0000 7f7c72ffd640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f7c740b3e80 0x7f7c7406d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:39806/0 (socket says 172.21.15.5:39806) 2024-05-20T18:37:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.743+0000 7f7c73fff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7c7406dae0 0x7f7c740b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:45866/0 (socket says 172.21.15.5:45866) 2024-05-20T18:37:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.743+0000 7f7c73fff640 1 -- 172.21.15.5:0/171827940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c74162820 msgr2=0x7f7c740b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.743+0000 7f7c73fff640 1 --2- 172.21.15.5:0/171827940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c74162820 0x7f7c740b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.743+0000 7f7c73fff640 1 -- 172.21.15.5:0/171827940 >> v1:172.21.15.63:6789/0 conn(0x7f7c740b3e80 legacy=0x7f7c7406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.743+0000 7f7c73fff640 1 -- 172.21.15.5:0/171827940 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c740d3f80 con 0x7f7c7406dae0 2024-05-20T18:37:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.743+0000 7f7c737fe640 1 --2- 172.21.15.5:0/171827940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c74162820 0x7f7c740b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:37:07.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.744+0000 7f7c73fff640 1 --2- 172.21.15.5:0/171827940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c7406dae0 0x7f7c740b8190 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c009d90 tx=0x7f7c6c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8199bbc5a188a812 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:07.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.744+0000 7f7c727fc640 1 -- 172.21.15.5:0/171827940 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c007740 con 0x7f7c7406dae0 2024-05-20T18:37:07.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.744+0000 7f7c727fc640 1 -- 172.21.15.5:0/171827940 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c6c0078e0 con 0x7f7c7406dae0 2024-05-20T18:37:07.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.744+0000 7f7c727fc640 1 -- 172.21.15.5:0/171827940 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c007cb0 con 0x7f7c7406dae0 2024-05-20T18:37:07.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.745+0000 7f7c79eca640 1 -- 172.21.15.5:0/171827940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c7406dae0 msgr2=0x7f7c740b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:07.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.745+0000 7f7c79eca640 1 --2- 172.21.15.5:0/171827940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c7406dae0 0x7f7c740b8190 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c009d90 tx=0x7f7c6c02a660 comp rx=0 tx=0).stop 2024-05-20T18:37:07.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.745+0000 7f7c79eca640 1 -- 172.21.15.5:0/171827940 shutdown_connections 2024-05-20T18:37:07.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.745+0000 7f7c79eca640 1 --2- 172.21.15.5:0/171827940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c7406dae0 0x7f7c740b8190 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:07.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.745+0000 7f7c79eca640 1 --2- 172.21.15.5:0/171827940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c74162820 0x7f7c740b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:07.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.745+0000 7f7c79eca640 1 -- 172.21.15.5:0/171827940 >> 172.21.15.5:0/171827940 conn(0x7f7c740d4bd0 msgr2=0x7f7c740d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:07.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.745+0000 7f7c79eca640 1 -- 172.21.15.5:0/171827940 shutdown_connections 2024-05-20T18:37:07.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.745+0000 7f7c79eca640 1 -- 172.21.15.5:0/171827940 wait complete. 2024-05-20T18:37:07.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.747+0000 7f7c79eca640 1 Processor -- start 2024-05-20T18:37:07.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.747+0000 7f7c79eca640 1 -- start start 2024-05-20T18:37:07.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.748+0000 7f7c79eca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c740b3e80 0x7f7c7414b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:07.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.748+0000 7f7c79eca640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c74162820 0x7f7c740c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:07.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.748+0000 7f7c79eca640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c740c9d20 0x7f7c740cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:07.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.748+0000 7f7c79eca640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7c740d4780 con 0x7f7c740b3e80 2024-05-20T18:37:07.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.748+0000 7f7c79eca640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7c740d0300 con 0x7f7c740c9d20 2024-05-20T18:37:07.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.748+0000 7f7c79eca640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7c740d4900 con 0x7f7c74162820 2024-05-20T18:37:07.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.749+0000 7f7c737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c740b3e80 0x7f7c7414b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:07.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.749+0000 7f7c737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c740b3e80 0x7f7c7414b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35342/0 (socket says 172.21.15.5:35342) 2024-05-20T18:37:07.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.749+0000 7f7c737fe640 1 -- 172.21.15.5:0/355109911 learned_addr learned my addr 172.21.15.5:0/355109911 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:07.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.749+0000 7f7c73fff640 1 --2- 172.21.15.5:0/355109911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c740c9d20 0x7f7c740cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:07.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.749+0000 7f7c72ffd640 1 --2- 172.21.15.5:0/355109911 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c74162820 0x7f7c740c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:07.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.749+0000 7f7c737fe640 1 -- 172.21.15.5:0/355109911 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c74162820 msgr2=0x7f7c740c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:07.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.749+0000 7f7c737fe640 1 --2- 172.21.15.5:0/355109911 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c74162820 0x7f7c740c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:07.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.749+0000 7f7c737fe640 1 -- 172.21.15.5:0/355109911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c740c9d20 msgr2=0x7f7c740cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:07.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.749+0000 7f7c737fe640 1 --2- 172.21.15.5:0/355109911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c740c9d20 0x7f7c740cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:07.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.749+0000 7f7c737fe640 1 -- 172.21.15.5:0/355109911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c740ca5a0 con 0x7f7c740b3e80 2024-05-20T18:37:07.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.749+0000 7f7c737fe640 1 --2- 172.21.15.5:0/355109911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c740b3e80 0x7f7c7414b1f0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400e9e0 tx=0x7f7c6400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:07.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.749+0000 7f7c5bfff640 1 -- 172.21.15.5:0/355109911 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c64017070 con 0x7f7c740b3e80 2024-05-20T18:37:07.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.749+0000 7f7c79eca640 1 -- 172.21.15.5:0/355109911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c740ca880 con 0x7f7c740b3e80 2024-05-20T18:37:07.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.749+0000 7f7c5bfff640 1 -- 172.21.15.5:0/355109911 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c6400c9d0 con 0x7f7c740b3e80 2024-05-20T18:37:07.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.749+0000 7f7c5bfff640 1 -- 172.21.15.5:0/355109911 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6400cd80 con 0x7f7c740b3e80 2024-05-20T18:37:07.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.749+0000 7f7c79eca640 1 -- 172.21.15.5:0/355109911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c740cae40 con 0x7f7c740b3e80 2024-05-20T18:37:07.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.751+0000 7f7c79eca640 1 -- 172.21.15.5:0/355109911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c74075b00 con 0x7f7c740b3e80 2024-05-20T18:37:07.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.751+0000 7f7c5bfff640 1 -- 172.21.15.5:0/355109911 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7c6401a070 con 0x7f7c740b3e80 2024-05-20T18:37:07.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.751+0000 7f7c5bfff640 1 --2- 172.21.15.5:0/355109911 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7c6006be30 0x7f7c6006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:07.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.752+0000 7f7c5bfff640 1 -- 172.21.15.5:0/355109911 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5421+0+0 (secure 0 0 0) 0x7f7c6408b010 con 0x7f7c740b3e80 2024-05-20T18:37:07.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.752+0000 7f7c72ffd640 1 --2- 172.21.15.5:0/355109911 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7c6006be30 0x7f7c6006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:07.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.752+0000 7f7c72ffd640 1 --2- 172.21.15.5:0/355109911 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7c6006be30 0x7f7c6006e340 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7c740b7030 tx=0x7f7c68007ce0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:07.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:07.756+0000 7f7c5bfff640 1 -- 172.21.15.5:0/355109911 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7c64059a50 con 0x7f7c740b3e80 2024-05-20T18:37:08.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:08.012+0000 7f7c79eca640 1 -- 172.21.15.5:0/355109911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7f7c740b4ee0 con 0x7f7c740b3e80 2024-05-20T18:37:08.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:08.948+0000 7f7c5bfff640 1 -- 172.21.15.5:0/355109911 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v474) ==== 147+0+0 (secure 0 0 0) 0x7f7c64059350 con 0x7f7c740b3e80 2024-05-20T18:37:08.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:08.951+0000 7f7c79eca640 1 -- 172.21.15.5:0/355109911 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7c6006be30 msgr2=0x7f7c6006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:08.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:08.951+0000 7f7c79eca640 1 --2- 172.21.15.5:0/355109911 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7c6006be30 0x7f7c6006e340 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7c740b7030 tx=0x7f7c68007ce0 comp rx=0 tx=0).stop 2024-05-20T18:37:08.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:08.951+0000 7f7c79eca640 1 -- 172.21.15.5:0/355109911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c740b3e80 msgr2=0x7f7c7414b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:08.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:08.951+0000 7f7c79eca640 1 --2- 172.21.15.5:0/355109911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c740b3e80 0x7f7c7414b1f0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400e9e0 tx=0x7f7c6400eea0 comp rx=0 tx=0).stop 2024-05-20T18:37:08.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:08.951+0000 7f7c79eca640 1 -- 172.21.15.5:0/355109911 shutdown_connections 2024-05-20T18:37:08.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:08.951+0000 7f7c79eca640 1 --2- 172.21.15.5:0/355109911 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7c6006be30 0x7f7c6006e340 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:08.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:08.951+0000 7f7c79eca640 1 --2- 172.21.15.5:0/355109911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c740c9d20 0x7f7c740cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:08.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:08.951+0000 7f7c79eca640 1 --2- 172.21.15.5:0/355109911 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7c74162820 0x7f7c740c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:08.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:08.951+0000 7f7c79eca640 1 --2- 172.21.15.5:0/355109911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c740b3e80 0x7f7c7414b1f0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:08.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:08.951+0000 7f7c79eca640 1 -- 172.21.15.5:0/355109911 >> 172.21.15.5:0/355109911 conn(0x7f7c740d4bd0 msgr2=0x7f7c740ba460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:08.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:08.951+0000 7f7c79eca640 1 -- 172.21.15.5:0/355109911 shutdown_connections 2024-05-20T18:37:08.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:08.951+0000 7f7c79eca640 1 -- 172.21.15.5:0/355109911 wait complete. 2024-05-20T18:37:08.967 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-20T18:37:09.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.091+0000 7fb43b6bc640 1 Processor -- start 2024-05-20T18:37:09.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.092+0000 7fb43b6bc640 1 -- start start 2024-05-20T18:37:09.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.093+0000 7fb43b6bc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb43415f700 0x7fb43415fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:09.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.093+0000 7fb43b6bc640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb43406c180 0x7fb43406c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:09.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.093+0000 7fb43b6bc640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb434160430 con 0x7fb4340af030 2024-05-20T18:37:09.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.093+0000 7fb43b6bc640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fb43406bcf0 con 0x7fb43415f700 2024-05-20T18:37:09.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.093+0000 7fb43b6bc640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb43406b6a0 con 0x7fb43406c180 2024-05-20T18:37:09.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.093+0000 7fb438c57640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb43415f700 0x7fb43415fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:09.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.093+0000 7fb438c57640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb43415f700 0x7fb43415fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35354/0 (socket says 172.21.15.5:35354) 2024-05-20T18:37:09.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.093+0000 7fb438c57640 1 -- 172.21.15.5:0/1543372305 learned_addr learned my addr 172.21.15.5:0/1543372305 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:09.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.093+0000 7fb439c59640 1 --2- 172.21.15.5:0/1543372305 >> v2:172.21.15.63:3300/0 conn(0x7fb43406c180 0x7fb43406c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:09.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.094+0000 7fb438c57640 1 -- 172.21.15.5:0/1543372305 >> v2:172.21.15.63:3300/0 conn(0x7fb43406c180 msgr2=0x7fb43406c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:09.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.094+0000 7fb438c57640 1 --2- 172.21.15.5:0/1543372305 >> v2:172.21.15.63:3300/0 conn(0x7fb43406c180 0x7fb43406c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:09.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.094+0000 7fb438c57640 1 -- 172.21.15.5:0/1543372305 >> v1:172.21.15.63:6789/0 conn(0x7fb4340af030 legacy=0x7fb4340af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:09.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.094+0000 7fb438c57640 1 -- 172.21.15.5:0/1543372305 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb43406b980 con 0x7fb43415f700 2024-05-20T18:37:09.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.094+0000 7fb438c57640 1 --2- 172.21.15.5:0/1543372305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb43415f700 0x7fb43415fb10 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fb4340aea40 tx=0x7fb42802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c48a185f7a7fa0d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:09.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.094+0000 7fb433fff640 1 -- 172.21.15.5:0/1543372305 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428037070 con 0x7fb43415f700 2024-05-20T18:37:09.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.095+0000 7fb433fff640 1 -- 172.21.15.5:0/1543372305 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb428032440 con 0x7fb43415f700 2024-05-20T18:37:09.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.095+0000 7fb433fff640 1 -- 172.21.15.5:0/1543372305 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428032810 con 0x7fb43415f700 2024-05-20T18:37:09.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.095+0000 7fb43b6bc640 1 -- 172.21.15.5:0/1543372305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb43415f700 msgr2=0x7fb43415fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:09.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.095+0000 7fb43b6bc640 1 --2- 172.21.15.5:0/1543372305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb43415f700 0x7fb43415fb10 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fb4340aea40 tx=0x7fb42802a210 comp rx=0 tx=0).stop 2024-05-20T18:37:09.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.095+0000 7fb43b6bc640 1 -- 172.21.15.5:0/1543372305 shutdown_connections 2024-05-20T18:37:09.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.095+0000 7fb43b6bc640 1 --2- 172.21.15.5:0/1543372305 >> v2:172.21.15.63:3300/0 conn(0x7fb43406c180 0x7fb43406c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:09.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.095+0000 7fb43b6bc640 1 --2- 172.21.15.5:0/1543372305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb43415f700 0x7fb43415fb10 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:09.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.095+0000 7fb43b6bc640 1 -- 172.21.15.5:0/1543372305 >> 172.21.15.5:0/1543372305 conn(0x7fb4340b6620 msgr2=0x7fb4340b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:09.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.095+0000 7fb43b6bc640 1 -- 172.21.15.5:0/1543372305 shutdown_connections 2024-05-20T18:37:09.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.095+0000 7fb43b6bc640 1 -- 172.21.15.5:0/1543372305 wait complete. 2024-05-20T18:37:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.097+0000 7fb43b6bc640 1 Processor -- start 2024-05-20T18:37:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.097+0000 7fb43b6bc640 1 -- start start 2024-05-20T18:37:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.098+0000 7fb43b6bc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb43406c180 0x7fb43409d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.098+0000 7fb43b6bc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4340af030 0x7fb43415bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.098+0000 7fb43b6bc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb43415f700 0x7fb434159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.098+0000 7fb43b6bc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb4341601d0 con 0x7fb43415f700 2024-05-20T18:37:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.098+0000 7fb43b6bc640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb434160050 con 0x7fb4340af030 2024-05-20T18:37:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.098+0000 7fb43b6bc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb43406cad0 con 0x7fb43406c180 2024-05-20T18:37:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.099+0000 7fb439c59640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb43415f700 0x7fb434159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.099+0000 7fb439c59640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb43415f700 0x7fb434159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35362/0 (socket says 172.21.15.5:35362) 2024-05-20T18:37:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.099+0000 7fb439c59640 1 -- 172.21.15.5:0/1067683422 learned_addr learned my addr 172.21.15.5:0/1067683422 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.099+0000 7fb439458640 1 --2- 172.21.15.5:0/1067683422 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb43406c180 0x7fb43409d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.099+0000 7fb438c57640 1 --2- 172.21.15.5:0/1067683422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4340af030 0x7fb43415bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.099+0000 7fb439c59640 1 -- 172.21.15.5:0/1067683422 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb43406c180 msgr2=0x7fb43409d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.099+0000 7fb439c59640 1 --2- 172.21.15.5:0/1067683422 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb43406c180 0x7fb43409d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.099+0000 7fb439c59640 1 -- 172.21.15.5:0/1067683422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4340af030 msgr2=0x7fb43415bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.099+0000 7fb439c59640 1 --2- 172.21.15.5:0/1067683422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4340af030 0x7fb43415bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.099+0000 7fb439c59640 1 -- 172.21.15.5:0/1067683422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4341f5720 con 0x7fb43415f700 2024-05-20T18:37:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.099+0000 7fb439c59640 1 --2- 172.21.15.5:0/1067683422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb43415f700 0x7fb434159ee0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fb42c00d5d0 tx=0x7fb42c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.099+0000 7fb431ffb640 1 -- 172.21.15.5:0/1067683422 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c0125f0 con 0x7fb43415f700 2024-05-20T18:37:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.099+0000 7fb431ffb640 1 -- 172.21.15.5:0/1067683422 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb42c019070 con 0x7fb43415f700 2024-05-20T18:37:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.099+0000 7fb43b6bc640 1 -- 172.21.15.5:0/1067683422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb43409d790 con 0x7fb43415f700 2024-05-20T18:37:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.099+0000 7fb431ffb640 1 -- 172.21.15.5:0/1067683422 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c0129f0 con 0x7fb43415f700 2024-05-20T18:37:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.100+0000 7fb43b6bc640 1 -- 172.21.15.5:0/1067683422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb43415aef0 con 0x7fb43415f700 2024-05-20T18:37:09.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.101+0000 7fb43b6bc640 1 -- 172.21.15.5:0/1067683422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb43415a980 con 0x7fb43415f700 2024-05-20T18:37:09.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.101+0000 7fb431ffb640 1 -- 172.21.15.5:0/1067683422 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb42c012b90 con 0x7fb43415f700 2024-05-20T18:37:09.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.102+0000 7fb431ffb640 1 --2- 172.21.15.5:0/1067683422 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb42006be30 0x7fb42006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:09.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.102+0000 7fb431ffb640 1 -- 172.21.15.5:0/1067683422 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5421+0+0 (secure 0 0 0) 0x7fb42c08af70 con 0x7fb43415f700 2024-05-20T18:37:09.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.102+0000 7fb439458640 1 --2- 172.21.15.5:0/1067683422 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb42006be30 0x7fb42006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:09.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.102+0000 7fb439458640 1 --2- 172.21.15.5:0/1067683422 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb42006be30 0x7fb42006e340 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb43406e1f0 tx=0x7fb42400d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:09.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.106+0000 7fb431ffb640 1 -- 172.21.15.5:0/1067683422 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb4341f5720 con 0x7fb43415f700 2024-05-20T18:37:09.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.289+0000 7fb43b6bc640 1 -- 172.21.15.5:0/1067683422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb4340d1630 con 0x7fb43415f700 2024-05-20T18:37:09.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.293+0000 7fb431ffb640 1 -- 172.21.15.5:0/1067683422 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7fb42c059400 con 0x7fb43415f700 2024-05-20T18:37:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.295+0000 7fb43b6bc640 1 -- 172.21.15.5:0/1067683422 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb42006be30 msgr2=0x7fb42006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.295+0000 7fb43b6bc640 1 --2- 172.21.15.5:0/1067683422 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb42006be30 0x7fb42006e340 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb43406e1f0 tx=0x7fb42400d040 comp rx=0 tx=0).stop 2024-05-20T18:37:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.295+0000 7fb43b6bc640 1 -- 172.21.15.5:0/1067683422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb43415f700 msgr2=0x7fb434159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.295+0000 7fb43b6bc640 1 --2- 172.21.15.5:0/1067683422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb43415f700 0x7fb434159ee0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fb42c00d5d0 tx=0x7fb42c00da90 comp rx=0 tx=0).stop 2024-05-20T18:37:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.295+0000 7fb43b6bc640 1 -- 172.21.15.5:0/1067683422 shutdown_connections 2024-05-20T18:37:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.295+0000 7fb43b6bc640 1 --2- 172.21.15.5:0/1067683422 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb42006be30 0x7fb42006e340 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.295+0000 7fb43b6bc640 1 --2- 172.21.15.5:0/1067683422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb43415f700 0x7fb434159ee0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.295+0000 7fb43b6bc640 1 --2- 172.21.15.5:0/1067683422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb4340af030 0x7fb43415bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.295+0000 7fb43b6bc640 1 --2- 172.21.15.5:0/1067683422 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb43406c180 0x7fb43409d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.295+0000 7fb43b6bc640 1 -- 172.21.15.5:0/1067683422 >> 172.21.15.5:0/1067683422 conn(0x7fb4340b6620 msgr2=0x7fb434070760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.295+0000 7fb43b6bc640 1 -- 172.21.15.5:0/1067683422 shutdown_connections 2024-05-20T18:37:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.295+0000 7fb43b6bc640 1 -- 172.21.15.5:0/1067683422 wait complete. 2024-05-20T18:37:09.308 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-20T18:37:09.308 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T18:37:09.352 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T18:37:09.407 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-20T18:37:09.407 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:37:09.407 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:37:09.424 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15663 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T18:37:09.424 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:37:09.424 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:37:09.444 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19732 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T18:37:09.444 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:37:09.444 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:37:09.480 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15663 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T18:37:09.480 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:37:09.480 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:37:09.503 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19732 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T18:37:09.504 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-20T18:37:09.504 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:37:09.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.633+0000 7f77f6ee2640 1 Processor -- start 2024-05-20T18:37:09.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.633+0000 7f77f6ee2640 1 -- start start 2024-05-20T18:37:09.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.635+0000 7f77f6ee2640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77f0162820 0x7f77f00b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:09.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.635+0000 7f77f6ee2640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f77f00b3e80 0x7f77f006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:09.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.635+0000 7f77f6ee2640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f77f006d910 0x7f77f00d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:09.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.635+0000 7f77f6ee2640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f77f006dd20 con 0x7f77f00b3e80 2024-05-20T18:37:09.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.635+0000 7f77f6ee2640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f77f00d0630 con 0x7f77f006d910 2024-05-20T18:37:09.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.635+0000 7f77f6ee2640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f77f00d39c0 con 0x7f77f0162820 2024-05-20T18:37:09.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.635+0000 7f77effff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f77f00b3e80 0x7f77f006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:09.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.635+0000 7f77effff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f77f00b3e80 0x7f77f006d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35378/0 (socket says 172.21.15.5:35378) 2024-05-20T18:37:09.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.635+0000 7f77effff640 1 -- 172.21.15.5:0/259288992 learned_addr learned my addr 172.21.15.5:0/259288992 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:09.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.635+0000 7f77f547f640 1 --2- 172.21.15.5:0/259288992 >> v2:172.21.15.63:3300/0 conn(0x7f77f006d910 0x7f77f00d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:09.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.635+0000 7f77effff640 1 -- 172.21.15.5:0/259288992 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77f0162820 msgr2=0x7f77f00b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:09.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.635+0000 7f77effff640 1 --2- 172.21.15.5:0/259288992 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77f0162820 0x7f77f00b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:09.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.635+0000 7f77effff640 1 -- 172.21.15.5:0/259288992 >> v2:172.21.15.63:3300/0 conn(0x7f77f006d910 msgr2=0x7f77f00d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:09.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.635+0000 7f77effff640 1 --2- 172.21.15.5:0/259288992 >> v2:172.21.15.63:3300/0 conn(0x7f77f006d910 0x7f77f00d27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:09.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.635+0000 7f77effff640 1 -- 172.21.15.5:0/259288992 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f00d3ca0 con 0x7f77f00b3e80 2024-05-20T18:37:09.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.636+0000 7f77effff640 1 --2- 172.21.15.5:0/259288992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77f00b3e80 0x7f77f006d3d0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f77f006ce40 tx=0x7f77e402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e729ef87497304 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:09.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.636+0000 7f77ef7fe640 1 -- 172.21.15.5:0/259288992 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e4037070 con 0x7f77f00b3e80 2024-05-20T18:37:09.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.636+0000 7f77ef7fe640 1 -- 172.21.15.5:0/259288992 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77e4032440 con 0x7f77f00b3e80 2024-05-20T18:37:09.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.636+0000 7f77ef7fe640 1 -- 172.21.15.5:0/259288992 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e4032810 con 0x7f77f00b3e80 2024-05-20T18:37:09.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.637+0000 7f77f6ee2640 1 -- 172.21.15.5:0/259288992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77f00b3e80 msgr2=0x7f77f006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:09.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.637+0000 7f77f6ee2640 1 --2- 172.21.15.5:0/259288992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77f00b3e80 0x7f77f006d3d0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f77f006ce40 tx=0x7f77e402a1e0 comp rx=0 tx=0).stop 2024-05-20T18:37:09.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.637+0000 7f77f6ee2640 1 -- 172.21.15.5:0/259288992 shutdown_connections 2024-05-20T18:37:09.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.637+0000 7f77f6ee2640 1 --2- 172.21.15.5:0/259288992 >> v2:172.21.15.63:3300/0 conn(0x7f77f006d910 0x7f77f00d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:09.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.637+0000 7f77f6ee2640 1 --2- 172.21.15.5:0/259288992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77f00b3e80 0x7f77f006d3d0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:09.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.637+0000 7f77f6ee2640 1 --2- 172.21.15.5:0/259288992 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77f0162820 0x7f77f00b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:09.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.637+0000 7f77f6ee2640 1 -- 172.21.15.5:0/259288992 >> 172.21.15.5:0/259288992 conn(0x7f77f00d4bd0 msgr2=0x7f77f00d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:09.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.637+0000 7f77f6ee2640 1 -- 172.21.15.5:0/259288992 shutdown_connections 2024-05-20T18:37:09.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.637+0000 7f77f6ee2640 1 -- 172.21.15.5:0/259288992 wait complete. 2024-05-20T18:37:09.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.639+0000 7f77f6ee2640 1 Processor -- start 2024-05-20T18:37:09.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.639+0000 7f77f6ee2640 1 -- start start 2024-05-20T18:37:09.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.640+0000 7f77f6ee2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77f006d910 0x7f77f014af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:09.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.640+0000 7f77f6ee2640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77f00b3e80 0x7f77f00b8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:09.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.640+0000 7f77f6ee2640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77f0162820 0x7f77f00b9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:09.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.640+0000 7f77f6ee2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f77f00d0300 con 0x7f77f006d910 2024-05-20T18:37:09.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.640+0000 7f77f6ee2640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f77f00d4260 con 0x7f77f0162820 2024-05-20T18:37:09.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.640+0000 7f77f6ee2640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f77f00d4780 con 0x7f77f00b3e80 2024-05-20T18:37:09.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.640+0000 7f77f4c7e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77f006d910 0x7f77f014af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:09.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.640+0000 7f77f4c7e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77f006d910 0x7f77f014af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35380/0 (socket says 172.21.15.5:35380) 2024-05-20T18:37:09.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.640+0000 7f77f4c7e640 1 -- 172.21.15.5:0/1258812126 learned_addr learned my addr 172.21.15.5:0/1258812126 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:09.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.641+0000 7f77f547f640 1 --2- 172.21.15.5:0/1258812126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77f0162820 0x7f77f00b9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:09.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.641+0000 7f77effff640 1 --2- 172.21.15.5:0/1258812126 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77f00b3e80 0x7f77f00b8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:09.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.641+0000 7f77f4c7e640 1 -- 172.21.15.5:0/1258812126 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77f00b3e80 msgr2=0x7f77f00b8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:09.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.641+0000 7f77f4c7e640 1 --2- 172.21.15.5:0/1258812126 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77f00b3e80 0x7f77f00b8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:09.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.641+0000 7f77f4c7e640 1 -- 172.21.15.5:0/1258812126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77f0162820 msgr2=0x7f77f00b9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:09.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.641+0000 7f77f4c7e640 1 --2- 172.21.15.5:0/1258812126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77f0162820 0x7f77f00b9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:09.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.641+0000 7f77f4c7e640 1 -- 172.21.15.5:0/1258812126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f00c9110 con 0x7f77f006d910 2024-05-20T18:37:09.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.641+0000 7f77f547f640 1 --2- 172.21.15.5:0/1258812126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77f0162820 0x7f77f00b9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:09.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.641+0000 7f77effff640 1 --2- 172.21.15.5:0/1258812126 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77f00b3e80 0x7f77f00b8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:09.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.641+0000 7f77f4c7e640 1 --2- 172.21.15.5:0/1258812126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77f006d910 0x7f77f014af40 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f77e000b440 tx=0x7f77e000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:09.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.641+0000 7f77ed7fa640 1 -- 172.21.15.5:0/1258812126 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e0015020 con 0x7f77f006d910 2024-05-20T18:37:09.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.641+0000 7f77ed7fa640 1 -- 172.21.15.5:0/1258812126 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77e0011890 con 0x7f77f006d910 2024-05-20T18:37:09.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.641+0000 7f77f6ee2640 1 -- 172.21.15.5:0/1258812126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77f00c93d0 con 0x7f77f006d910 2024-05-20T18:37:09.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.641+0000 7f77ed7fa640 1 -- 172.21.15.5:0/1258812126 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e0011c80 con 0x7f77f006d910 2024-05-20T18:37:09.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.641+0000 7f77f6ee2640 1 -- 172.21.15.5:0/1258812126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77f00b5520 con 0x7f77f006d910 2024-05-20T18:37:09.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.642+0000 7f77f6ee2640 1 -- 172.21.15.5:0/1258812126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77f0075b00 con 0x7f77f006d910 2024-05-20T18:37:09.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.643+0000 7f77ed7fa640 1 -- 172.21.15.5:0/1258812126 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f77e0018020 con 0x7f77f006d910 2024-05-20T18:37:09.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.643+0000 7f77ed7fa640 1 --2- 172.21.15.5:0/1258812126 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f77dc06be60 0x7f77dc06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:09.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.644+0000 7f77ed7fa640 1 -- 172.21.15.5:0/1258812126 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5421+0+0 (secure 0 0 0) 0x7f77e008aea0 con 0x7f77f006d910 2024-05-20T18:37:09.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.644+0000 7f77effff640 1 --2- 172.21.15.5:0/1258812126 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f77dc06be60 0x7f77dc06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:09.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.644+0000 7f77effff640 1 --2- 172.21.15.5:0/1258812126 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f77dc06be60 0x7f77dc06e370 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f77e4034000 tx=0x7f77e402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:09.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.648+0000 7f77ed7fa640 1 -- 172.21.15.5:0/1258812126 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f77e001f020 con 0x7f77f006d910 2024-05-20T18:37:09.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.872+0000 7f77f6ee2640 1 -- 172.21.15.5:0/1258812126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77f014b480 con 0x7f77f006d910 2024-05-20T18:37:09.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.873+0000 7f77ed7fa640 1 -- 172.21.15.5:0/1258812126 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+3860 (secure 0 0 0) 0x7f77e000d040 con 0x7f77f006d910 2024-05-20T18:37:09.874 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:09.874 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":92,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6758,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/4163145158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":4163145158},{"type":"v1","addr":"172.21.15.63:6834","nonce":4163145158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7528,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3049443532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3049443532},{"type":"v1","addr":"172.21.15.5:6837","nonce":3049443532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7828,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/4180308082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":4180308082},{"type":"v1","addr":"172.21.15.63:6835","nonce":4180308082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":8850,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/888311173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":888311173},{"type":"v1","addr":"172.21.15.5:6835","nonce":888311173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[]} 2024-05-20T18:37:09.874 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 92 2024-05-20T18:37:09.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.875+0000 7f77f6ee2640 1 -- 172.21.15.5:0/1258812126 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f77dc06be60 msgr2=0x7f77dc06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:09.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.875+0000 7f77f6ee2640 1 --2- 172.21.15.5:0/1258812126 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f77dc06be60 0x7f77dc06e370 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f77e4034000 tx=0x7f77e402e000 comp rx=0 tx=0).stop 2024-05-20T18:37:09.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.875+0000 7f77f6ee2640 1 -- 172.21.15.5:0/1258812126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77f006d910 msgr2=0x7f77f014af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:09.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.875+0000 7f77f6ee2640 1 --2- 172.21.15.5:0/1258812126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77f006d910 0x7f77f014af40 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f77e000b440 tx=0x7f77e000b900 comp rx=0 tx=0).stop 2024-05-20T18:37:09.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.875+0000 7f77f6ee2640 1 -- 172.21.15.5:0/1258812126 shutdown_connections 2024-05-20T18:37:09.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.875+0000 7f77f6ee2640 1 --2- 172.21.15.5:0/1258812126 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f77dc06be60 0x7f77dc06e370 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:09.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.875+0000 7f77f6ee2640 1 --2- 172.21.15.5:0/1258812126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77f0162820 0x7f77f00b9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:09.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.875+0000 7f77f6ee2640 1 --2- 172.21.15.5:0/1258812126 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77f00b3e80 0x7f77f00b8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:09.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.875+0000 7f77f6ee2640 1 --2- 172.21.15.5:0/1258812126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77f006d910 0x7f77f014af40 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:09.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.875+0000 7f77f6ee2640 1 -- 172.21.15.5:0/1258812126 >> 172.21.15.5:0/1258812126 conn(0x7f77f00d4bd0 msgr2=0x7f77f015ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:09.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.875+0000 7f77f6ee2640 1 -- 172.21.15.5:0/1258812126 shutdown_connections 2024-05-20T18:37:09.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:09.875+0000 7f77f6ee2640 1 -- 172.21.15.5:0/1258812126 wait complete. 2024-05-20T18:37:09.888 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-20T18:37:09.888 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-20T18:37:09.888 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-20T18:37:09.888 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:37:09.888 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-20T18:37:09.888 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-20T18:37:09.889 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-20T18:37:09.889 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:37:09.889 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-20T18:37:09.889 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-20T18:37:09.889 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-20T18:37:09.889 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:37:09.889 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-20T18:37:09.889 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-20T18:37:09.890 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-20T18:37:09.890 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:37:09.899 INFO:tasks.ceph.mds.a:Stopped 2024-05-20T18:37:09.899 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-20T18:37:09.901 INFO:tasks.ceph.mds.c:Stopped 2024-05-20T18:37:09.902 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-20T18:37:09.903 INFO:tasks.ceph.mds.d:Stopped 2024-05-20T18:37:09.903 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-20T18:37:09.904 INFO:tasks.ceph.mds.b:Stopped 2024-05-20T18:37:09.904 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-20T18:37:09.944 INFO:tasks.ceph.mds.a:Started 2024-05-20T18:37:09.945 DEBUG:teuthology.parallel:result is None 2024-05-20T18:37:09.946 INFO:tasks.ceph.mds.c:Started 2024-05-20T18:37:09.946 DEBUG:teuthology.parallel:result is None 2024-05-20T18:37:09.946 INFO:tasks.ceph.mds.d:Started 2024-05-20T18:37:09.947 DEBUG:teuthology.parallel:result is None 2024-05-20T18:37:09.948 INFO:tasks.ceph.mds.b:Started 2024-05-20T18:37:09.948 DEBUG:teuthology.parallel:result is None 2024-05-20T18:37:09.948 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-20T18:37:10.036 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-05-20T18:37:10.037 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-05-20T18:37:10.048 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-20T18:37:10.048 INFO:tasks.ceph.mds.d.smithi063.stdout:starting mds.d at 2024-05-20T18:37:10.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.070+0000 7fe1784bc640 1 Processor -- start 2024-05-20T18:37:10.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.070+0000 7fe1784bc640 1 -- start start 2024-05-20T18:37:10.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.072+0000 7fe1784bc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe170163a10 0x7fe1700d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:10.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.072+0000 7fe1784bc640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1700d0370 con 0x7fe1700b4010 2024-05-20T18:37:10.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.072+0000 7fe1784bc640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1700d0650 con 0x7fe1700d14b0 2024-05-20T18:37:10.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.072+0000 7fe1784bc640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe1700d3ca0 con 0x7fe170163a10 2024-05-20T18:37:10.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.072+0000 7fe175a57640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe1700b4010 0x7fe1700cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35326/0 (socket says 172.21.15.5:35326) 2024-05-20T18:37:10.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.072+0000 7fe176258640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe170163a10 0x7fe1700d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:10.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.072+0000 7fe175a57640 1 -- 172.21.15.5:0/731704500 learned_addr learned my addr 172.21.15.5:0/731704500 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:10.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.072+0000 7fe176258640 1 -- 172.21.15.5:0/731704500 >> v1:172.21.15.63:6789/0 conn(0x7fe1700d14b0 legacy=0x7fe1700d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:10.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.072+0000 7fe176258640 1 -- 172.21.15.5:0/731704500 >> v1:172.21.15.5:6789/0 conn(0x7fe1700b4010 legacy=0x7fe1700cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:10.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.072+0000 7fe175256640 1 -- 172.21.15.5:0/731704500 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 577171174 0 0) 0x7fe1700d0370 con 0x7fe1700b4010 2024-05-20T18:37:10.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.072+0000 7fe176258640 1 -- 172.21.15.5:0/731704500 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1700d3650 con 0x7fe170163a10 2024-05-20T18:37:10.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.073+0000 7fe176258640 1 --2- 172.21.15.5:0/731704500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe170163a10 0x7fe1700d4130 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fe16c0097a0 tx=0x7fe16c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ea0194ec750eb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:10.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.073+0000 7fe175256640 1 -- 172.21.15.5:0/731704500 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c037070 con 0x7fe170163a10 2024-05-20T18:37:10.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.073+0000 7fe175256640 1 -- 172.21.15.5:0/731704500 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe16c032440 con 0x7fe170163a10 2024-05-20T18:37:10.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.073+0000 7fe175256640 1 -- 172.21.15.5:0/731704500 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c032810 con 0x7fe170163a10 2024-05-20T18:37:10.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.074+0000 7fe1784bc640 1 -- 172.21.15.5:0/731704500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe170163a10 msgr2=0x7fe1700d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:10.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.074+0000 7fe1784bc640 1 --2- 172.21.15.5:0/731704500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe170163a10 0x7fe1700d4130 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fe16c0097a0 tx=0x7fe16c02a230 comp rx=0 tx=0).stop 2024-05-20T18:37:10.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.074+0000 7fe1784bc640 1 -- 172.21.15.5:0/731704500 shutdown_connections 2024-05-20T18:37:10.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.074+0000 7fe1784bc640 1 --2- 172.21.15.5:0/731704500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe170163a10 0x7fe1700d4130 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:10.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.074+0000 7fe1784bc640 1 -- 172.21.15.5:0/731704500 >> 172.21.15.5:0/731704500 conn(0x7fe17006d240 msgr2=0x7fe17006d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:10.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.074+0000 7fe1784bc640 1 -- 172.21.15.5:0/731704500 shutdown_connections 2024-05-20T18:37:10.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.074+0000 7fe1784bc640 1 -- 172.21.15.5:0/731704500 wait complete. 2024-05-20T18:37:10.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.076+0000 7fe1784bc640 1 Processor -- start 2024-05-20T18:37:10.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.076+0000 7fe1784bc640 1 -- start start 2024-05-20T18:37:10.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.077+0000 7fe1784bc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1700b4010 0x7fe1701e8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:10.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.077+0000 7fe1784bc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1700d14b0 0x7fe170157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:10.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.077+0000 7fe1784bc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe170163a10 0x7fe1700ce290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:10.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.077+0000 7fe1784bc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1700b8390 con 0x7fe1700b4010 2024-05-20T18:37:10.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.077+0000 7fe1784bc640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe1700b8210 con 0x7fe1700d14b0 2024-05-20T18:37:10.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.077+0000 7fe1784bc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe1700b8510 con 0x7fe170163a10 2024-05-20T18:37:10.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.077+0000 7fe176258640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1700b4010 0x7fe1701e8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:10.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.077+0000 7fe176258640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1700b4010 0x7fe1701e8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35428/0 (socket says 172.21.15.5:35428) 2024-05-20T18:37:10.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.077+0000 7fe176258640 1 -- 172.21.15.5:0/2448079526 learned_addr learned my addr 172.21.15.5:0/2448079526 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:10.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.077+0000 7fe176a59640 1 --2- 172.21.15.5:0/2448079526 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe170163a10 0x7fe1700ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:10.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.077+0000 7fe175a57640 1 --2- 172.21.15.5:0/2448079526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1700d14b0 0x7fe170157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:10.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.077+0000 7fe176258640 1 -- 172.21.15.5:0/2448079526 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe170163a10 msgr2=0x7fe1700ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:10.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.077+0000 7fe176258640 1 --2- 172.21.15.5:0/2448079526 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe170163a10 0x7fe1700ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:10.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.077+0000 7fe176258640 1 -- 172.21.15.5:0/2448079526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1700d14b0 msgr2=0x7fe170157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:10.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.077+0000 7fe176258640 1 --2- 172.21.15.5:0/2448079526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1700d14b0 0x7fe170157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:10.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.077+0000 7fe176258640 1 -- 172.21.15.5:0/2448079526 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1701e9450 con 0x7fe1700b4010 2024-05-20T18:37:10.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.077+0000 7fe176a59640 1 --2- 172.21.15.5:0/2448079526 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe170163a10 0x7fe1700ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:10.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.077+0000 7fe175a57640 1 --2- 172.21.15.5:0/2448079526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1700d14b0 0x7fe170157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:10.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.078+0000 7fe176258640 1 --2- 172.21.15.5:0/2448079526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1700b4010 0x7fe1701e8c00 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fe16c0097a0 tx=0x7fe16c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:10.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.078+0000 7fe166ffd640 1 -- 172.21.15.5:0/2448079526 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c037050 con 0x7fe1700b4010 2024-05-20T18:37:10.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.078+0000 7fe166ffd640 1 -- 172.21.15.5:0/2448079526 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe16c038040 con 0x7fe1700b4010 2024-05-20T18:37:10.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.078+0000 7fe166ffd640 1 -- 172.21.15.5:0/2448079526 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c004490 con 0x7fe1700b4010 2024-05-20T18:37:10.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.078+0000 7fe1784bc640 1 -- 172.21.15.5:0/2448079526 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1701e9730 con 0x7fe1700b4010 2024-05-20T18:37:10.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.078+0000 7fe1784bc640 1 -- 172.21.15.5:0/2448079526 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1701e9cf0 con 0x7fe1700b4010 2024-05-20T18:37:10.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.079+0000 7fe1784bc640 1 -- 172.21.15.5:0/2448079526 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe170075b00 con 0x7fe1700b4010 2024-05-20T18:37:10.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.080+0000 7fe166ffd640 1 -- 172.21.15.5:0/2448079526 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe16c030900 con 0x7fe1700b4010 2024-05-20T18:37:10.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.080+0000 7fe166ffd640 1 --2- 172.21.15.5:0/2448079526 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe15406be60 0x7fe15406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:10.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.080+0000 7fe166ffd640 1 -- 172.21.15.5:0/2448079526 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5421+0+0 (secure 0 0 0) 0x7fe16c0ab860 con 0x7fe1700b4010 2024-05-20T18:37:10.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.080+0000 7fe175a57640 1 --2- 172.21.15.5:0/2448079526 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe15406be60 0x7fe15406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:10.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.081+0000 7fe175a57640 1 --2- 172.21.15.5:0/2448079526 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe15406be60 0x7fe15406e370 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe170160540 tx=0x7fe160009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:10.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.085+0000 7fe166ffd640 1 -- 172.21.15.5:0/2448079526 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe16c04c020 con 0x7fe1700b4010 2024-05-20T18:37:10.105 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-05-20T18:37:10.105 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-05-20T18:37:10.106 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-05-20T18:37:10.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.280+0000 7fe1784bc640 1 -- 172.21.15.5:0/2448079526 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe1700d3050 con 0x7fe1700b4010 2024-05-20T18:37:10.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.967+0000 7fe166ffd640 1 -- 172.21.15.5:0/2448079526 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v113) ==== 97+0+0 (secure 0 0 0) 0x7fe16c07a3c0 con 0x7fe1700b4010 2024-05-20T18:37:10.969 INFO:teuthology.orchestra.run.smithi005.stderr: removed all blocklist entries 2024-05-20T18:37:10.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.970+0000 7fe1784bc640 1 -- 172.21.15.5:0/2448079526 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe15406be60 msgr2=0x7fe15406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:10.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.970+0000 7fe1784bc640 1 --2- 172.21.15.5:0/2448079526 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe15406be60 0x7fe15406e370 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe170160540 tx=0x7fe160009040 comp rx=0 tx=0).stop 2024-05-20T18:37:10.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.970+0000 7fe1784bc640 1 -- 172.21.15.5:0/2448079526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1700b4010 msgr2=0x7fe1701e8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:10.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.970+0000 7fe1784bc640 1 --2- 172.21.15.5:0/2448079526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1700b4010 0x7fe1701e8c00 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fe16c0097a0 tx=0x7fe16c0046c0 comp rx=0 tx=0).stop 2024-05-20T18:37:10.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.970+0000 7fe1784bc640 1 -- 172.21.15.5:0/2448079526 shutdown_connections 2024-05-20T18:37:10.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.970+0000 7fe1784bc640 1 --2- 172.21.15.5:0/2448079526 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe15406be60 0x7fe15406e370 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:10.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.970+0000 7fe1784bc640 1 --2- 172.21.15.5:0/2448079526 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe170163a10 0x7fe1700ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:10.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.970+0000 7fe1784bc640 1 --2- 172.21.15.5:0/2448079526 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1700d14b0 0x7fe170157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:10.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.970+0000 7fe1784bc640 1 --2- 172.21.15.5:0/2448079526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1700b4010 0x7fe1701e8c00 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:10.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.970+0000 7fe1784bc640 1 -- 172.21.15.5:0/2448079526 >> 172.21.15.5:0/2448079526 conn(0x7fe17006d240 msgr2=0x7fe170162610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:10.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.970+0000 7fe1784bc640 1 -- 172.21.15.5:0/2448079526 shutdown_connections 2024-05-20T18:37:10.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:10.970+0000 7fe1784bc640 1 -- 172.21.15.5:0/2448079526 wait complete. 2024-05-20T18:37:10.987 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-20T18:37:11.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.111+0000 7f6dbbbb1640 1 Processor -- start 2024-05-20T18:37:11.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.111+0000 7f6dbbbb1640 1 -- start start 2024-05-20T18:37:11.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.112+0000 7f6dbbbb1640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6db4071630 0x7f6db415f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:11.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.112+0000 7f6dbbbb1640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6db40cd1b0 0x7f6db406d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:11.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.112+0000 7f6dbbbb1640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6db40ce2b0 con 0x7f6db4070850 2024-05-20T18:37:11.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.112+0000 7f6dbbbb1640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f6db415f790 con 0x7f6db4071630 2024-05-20T18:37:11.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.112+0000 7f6dbbbb1640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f6db406e590 con 0x7f6db40cd1b0 2024-05-20T18:37:11.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.112+0000 7f6db994d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6db4071630 0x7f6db415f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:11.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.112+0000 7f6db994d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6db4071630 0x7f6db415f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35440/0 (socket says 172.21.15.5:35440) 2024-05-20T18:37:11.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.112+0000 7f6db994d640 1 -- 172.21.15.5:0/3257984016 learned_addr learned my addr 172.21.15.5:0/3257984016 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:11.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.113+0000 7f6db914c640 1 --2- 172.21.15.5:0/3257984016 >> v2:172.21.15.63:3300/0 conn(0x7f6db40cd1b0 0x7f6db406d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:11.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.113+0000 7f6db994d640 1 -- 172.21.15.5:0/3257984016 >> v2:172.21.15.63:3300/0 conn(0x7f6db40cd1b0 msgr2=0x7f6db406d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:11.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.113+0000 7f6db994d640 1 --2- 172.21.15.5:0/3257984016 >> v2:172.21.15.63:3300/0 conn(0x7f6db40cd1b0 0x7f6db406d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:11.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.113+0000 7f6db994d640 1 -- 172.21.15.5:0/3257984016 >> v1:172.21.15.63:6789/0 conn(0x7f6db4070850 legacy=0x7f6db4070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:11.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.113+0000 7f6db994d640 1 -- 172.21.15.5:0/3257984016 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db406cce0 con 0x7f6db4071630 2024-05-20T18:37:11.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.113+0000 7f6db994d640 1 --2- 172.21.15.5:0/3257984016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6db4071630 0x7f6db415f250 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f6db40ccfa0 tx=0x7f6db002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a88beb8451ff81d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:11.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.114+0000 7f6db894b640 1 -- 172.21.15.5:0/3257984016 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db0037070 con 0x7f6db4071630 2024-05-20T18:37:11.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.114+0000 7f6db894b640 1 -- 172.21.15.5:0/3257984016 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6db0032440 con 0x7f6db4071630 2024-05-20T18:37:11.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.114+0000 7f6db894b640 1 -- 172.21.15.5:0/3257984016 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db0032810 con 0x7f6db4071630 2024-05-20T18:37:11.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.114+0000 7f6dbbbb1640 1 -- 172.21.15.5:0/3257984016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6db4071630 msgr2=0x7f6db415f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:11.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.114+0000 7f6dbbbb1640 1 --2- 172.21.15.5:0/3257984016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6db4071630 0x7f6db415f250 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f6db40ccfa0 tx=0x7f6db002a1a0 comp rx=0 tx=0).stop 2024-05-20T18:37:11.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.114+0000 7f6dbbbb1640 1 -- 172.21.15.5:0/3257984016 shutdown_connections 2024-05-20T18:37:11.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.114+0000 7f6dbbbb1640 1 --2- 172.21.15.5:0/3257984016 >> v2:172.21.15.63:3300/0 conn(0x7f6db40cd1b0 0x7f6db406d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:11.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.114+0000 7f6dbbbb1640 1 --2- 172.21.15.5:0/3257984016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6db4071630 0x7f6db415f250 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:11.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.114+0000 7f6dbbbb1640 1 -- 172.21.15.5:0/3257984016 >> 172.21.15.5:0/3257984016 conn(0x7f6db40d29d0 msgr2=0x7f6db40d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:11.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.115+0000 7f6dbbbb1640 1 -- 172.21.15.5:0/3257984016 shutdown_connections 2024-05-20T18:37:11.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.115+0000 7f6dbbbb1640 1 -- 172.21.15.5:0/3257984016 wait complete. 2024-05-20T18:37:11.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.116+0000 7f6dbbbb1640 1 Processor -- start 2024-05-20T18:37:11.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.116+0000 7f6dbbbb1640 1 -- start start 2024-05-20T18:37:11.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.118+0000 7f6dbbbb1640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6db4070850 0x7f6db4149fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:11.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.118+0000 7f6dbbbb1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6db4071630 0x7f6db40b6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:11.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.118+0000 7f6dbbbb1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6db40cd1b0 0x7f6db40b6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:11.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.118+0000 7f6dbbbb1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6db406d2a0 con 0x7f6db4071630 2024-05-20T18:37:11.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.118+0000 7f6dbbbb1640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6db406c990 con 0x7f6db4070850 2024-05-20T18:37:11.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.118+0000 7f6dbbbb1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6db406dd00 con 0x7f6db40cd1b0 2024-05-20T18:37:11.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.118+0000 7f6db914c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6db4071630 0x7f6db40b6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:11.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.118+0000 7f6db914c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6db4071630 0x7f6db40b6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35446/0 (socket says 172.21.15.5:35446) 2024-05-20T18:37:11.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.118+0000 7f6db914c640 1 -- 172.21.15.5:0/3874682728 learned_addr learned my addr 172.21.15.5:0/3874682728 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:11.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.118+0000 7f6db994d640 1 --2- 172.21.15.5:0/3874682728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6db4070850 0x7f6db4149fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:11.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.118+0000 7f6dba14e640 1 --2- 172.21.15.5:0/3874682728 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6db40cd1b0 0x7f6db40b6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:11.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.118+0000 7f6db914c640 1 -- 172.21.15.5:0/3874682728 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6db40cd1b0 msgr2=0x7f6db40b6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:11.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.118+0000 7f6db914c640 1 --2- 172.21.15.5:0/3874682728 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6db40cd1b0 0x7f6db40b6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:11.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.118+0000 7f6db914c640 1 -- 172.21.15.5:0/3874682728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6db4070850 msgr2=0x7f6db4149fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:11.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.118+0000 7f6db914c640 1 --2- 172.21.15.5:0/3874682728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6db4070850 0x7f6db4149fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:11.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.118+0000 7f6db914c640 1 -- 172.21.15.5:0/3874682728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db4162990 con 0x7f6db4071630 2024-05-20T18:37:11.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.118+0000 7f6db914c640 1 --2- 172.21.15.5:0/3874682728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6db4071630 0x7f6db40b6010 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f6da400d620 tx=0x7f6da400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:11.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.118+0000 7f6daa7fc640 1 -- 172.21.15.5:0/3874682728 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da400b620 con 0x7f6db4071630 2024-05-20T18:37:11.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.118+0000 7f6daa7fc640 1 -- 172.21.15.5:0/3874682728 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6da4019070 con 0x7f6db4071630 2024-05-20T18:37:11.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.118+0000 7f6dbbbb1640 1 -- 172.21.15.5:0/3874682728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db41f5b60 con 0x7f6db4071630 2024-05-20T18:37:11.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.118+0000 7f6daa7fc640 1 -- 172.21.15.5:0/3874682728 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da400ba00 con 0x7f6db4071630 2024-05-20T18:37:11.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.119+0000 7f6dbbbb1640 1 -- 172.21.15.5:0/3874682728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db41f6120 con 0x7f6db4071630 2024-05-20T18:37:11.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.120+0000 7f6dbbbb1640 1 -- 172.21.15.5:0/3874682728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6db4075b00 con 0x7f6db4071630 2024-05-20T18:37:11.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.120+0000 7f6daa7fc640 1 -- 172.21.15.5:0/3874682728 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6da400bba0 con 0x7f6db4071630 2024-05-20T18:37:11.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.120+0000 7f6daa7fc640 1 --2- 172.21.15.5:0/3874682728 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6d9806be30 0x7f6d9806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:11.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.121+0000 7f6db994d640 1 --2- 172.21.15.5:0/3874682728 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6d9806be30 0x7f6d9806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:11.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.121+0000 7f6daa7fc640 1 -- 172.21.15.5:0/3874682728 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5249+0+0 (secure 0 0 0) 0x7f6da408b130 con 0x7f6db4071630 2024-05-20T18:37:11.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.121+0000 7f6db994d640 1 --2- 172.21.15.5:0/3874682728 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6d9806be30 0x7f6d9806e340 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6db0034000 tx=0x7f6db002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:11.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.125+0000 7f6daa7fc640 1 -- 172.21.15.5:0/3874682728 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6da4059b70 con 0x7f6db4071630 2024-05-20T18:37:11.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.378+0000 7f6dbbbb1640 1 -- 172.21.15.5:0/3874682728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f6db4162720 con 0x7f6db4071630 2024-05-20T18:37:11.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.378+0000 7f6daa7fc640 1 -- 172.21.15.5:0/3874682728 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) ==== 80+0+5304 (secure 0 0 0) 0x7f6da4059470 con 0x7f6db4071630 2024-05-20T18:37:11.380 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:11.380 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:37:11.380 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_dump": [ 2024-05-20T18:37:11.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:37:11.380 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.a", 2024-05-20T18:37:11.380 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtm9lSeCBAA/PpiEqojk6ueJo60tU4/iA==", 2024-05-20T18:37:11.380 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:37:11.380 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:37:11.380 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:37:11.380 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:37:11.380 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:37:11.381 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:37:11.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:37:11.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:37:11.381 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.b", 2024-05-20T18:37:11.381 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmdov0FRAACkWAR/0zaCDjgnkHnoHwgw==", 2024-05-20T18:37:11.381 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:37:11.381 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:37:11.381 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:37:11.381 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:37:11.381 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:37:11.381 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:37:11.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:37:11.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:37:11.381 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.c", 2024-05-20T18:37:11.381 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmpEBlEBAAdgxDXj9x38VSOizUgdFpYA==", 2024-05-20T18:37:11.382 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:37:11.382 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:37:11.382 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:37:11.382 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:37:11.382 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:37:11.382 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:37:11.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:37:11.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:37:11.382 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.d", 2024-05-20T18:37:11.382 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtm0Gy9HRAAAkPqcBLwxTwuWmCi7aad1A==", 2024-05-20T18:37:11.382 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:37:11.382 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:37:11.382 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:37:11.383 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:37:11.383 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:37:11.383 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:37:11.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:37:11.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:37:11.383 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.0", 2024-05-20T18:37:11.383 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDHlUtmWFGzBRAA81qN37dlHf6QDZ1FgwbJ9w==", 2024-05-20T18:37:11.383 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:37:11.383 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:37:11.383 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:37:11.383 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:37:11.383 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:37:11.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:37:11.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:37:11.383 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.1", 2024-05-20T18:37:11.384 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDKlUtmBMKsBBAAD4tTTvetME6oxwugTubSKQ==", 2024-05-20T18:37:11.384 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:37:11.384 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:37:11.384 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:37:11.384 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:37:11.384 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:37:11.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:37:11.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:37:11.384 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.2", 2024-05-20T18:37:11.384 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDNlUtm4Rm/AxAAIGWTUxb+LII09Rd+h5cJ8Q==", 2024-05-20T18:37:11.384 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:37:11.384 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:37:11.384 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:37:11.384 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:37:11.384 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:37:11.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:37:11.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:37:11.385 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.3", 2024-05-20T18:37:11.385 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDQlUtmWVxZAxAAB/w6k4WJ58ulEr9zatCRfw==", 2024-05-20T18:37:11.385 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:37:11.385 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:37:11.385 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:37:11.385 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:37:11.385 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:37:11.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:37:11.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:37:11.385 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.4", 2024-05-20T18:37:11.385 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDWlUtmIyokCRAAolOiYVCYaNTQZb1PjtR6fg==", 2024-05-20T18:37:11.385 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:37:11.385 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:37:11.386 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:37:11.386 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:37:11.386 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:37:11.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:37:11.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:37:11.386 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.5", 2024-05-20T18:37:11.386 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZlUtmSf6mCBAAw73nwq/zzt9kIXZymZh93A==", 2024-05-20T18:37:11.386 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:37:11.386 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:37:11.386 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:37:11.386 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:37:11.386 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:37:11.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:37:11.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:37:11.387 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.6", 2024-05-20T18:37:11.387 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDclUtmwwhGCBAA94OBEtj9h7cmCAddYMonaA==", 2024-05-20T18:37:11.387 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:37:11.387 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:37:11.387 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:37:11.387 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:37:11.387 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:37:11.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:37:11.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:37:11.387 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.7", 2024-05-20T18:37:11.387 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDflUtm9wefBxAAvCZTBzRu05koFhc+pH5I8w==", 2024-05-20T18:37:11.387 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:37:11.387 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-05-20T18:37:11.388 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-05-20T18:37:11.388 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:37:11.388 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:37:11.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:37:11.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:37:11.388 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.0", 2024-05-20T18:37:11.388 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmevaaIhAATIwyXzr0vGT38ZmcMqR2gA==", 2024-05-20T18:37:11.388 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:37:11.388 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:37:11.388 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow r", 2024-05-20T18:37:11.388 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-20T18:37:11.388 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:37:11.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:37:11.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:37:11.388 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.1", 2024-05-20T18:37:11.389 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmK7NfKhAADaXl0uVP1Q1gUOlPsstnzg==", 2024-05-20T18:37:11.389 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:37:11.389 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:37:11.389 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-20T18:37:11.389 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-20T18:37:11.389 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-20T18:37:11.389 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:37:11.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:37:11.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:37:11.389 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.2", 2024-05-20T18:37:11.389 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmMJF1JhAAeM+HseqvgvKgG8EnIsZuhg==", 2024-05-20T18:37:11.389 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:37:11.389 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:37:11.390 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-20T18:37:11.390 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-20T18:37:11.390 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-20T18:37:11.390 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:37:11.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:37:11.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:37:11.390 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.3", 2024-05-20T18:37:11.390 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmodA7LhAAUI2v0npB6HHs2n1g3KdTXA==", 2024-05-20T18:37:11.390 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:37:11.390 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-05-20T18:37:11.390 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-05-20T18:37:11.390 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-05-20T18:37:11.391 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-05-20T18:37:11.391 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:37:11.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:37:11.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:37:11.391 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.admin", 2024-05-20T18:37:11.391 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDElUtmZacGJRAAoyr9XNPBr9r0DSU53qCHSQ==", 2024-05-20T18:37:11.391 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:37:11.391 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-20T18:37:11.391 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-05-20T18:37:11.391 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-05-20T18:37:11.391 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:37:11.391 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:37:11.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:37:11.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:37:11.392 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.x", 2024-05-20T18:37:11.392 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDFlUtmPgq0BBAAAljNeQDsSau0wH2Lh2KdBw==", 2024-05-20T18:37:11.392 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:37:11.392 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-20T18:37:11.392 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-05-20T18:37:11.392 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:37:11.392 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:37:11.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-20T18:37:11.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:37:11.392 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.y", 2024-05-20T18:37:11.392 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDElUtm6Xv3OhAA0doOWTC73jy5rOYm7Vf68Q==", 2024-05-20T18:37:11.392 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-05-20T18:37:11.392 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-05-20T18:37:11.393 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-05-20T18:37:11.393 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-05-20T18:37:11.393 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:37:11.393 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:37:11.393 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-20T18:37:11.393 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:37:11.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.380+0000 7f6dbbbb1640 1 -- 172.21.15.5:0/3874682728 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6d9806be30 msgr2=0x7f6d9806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:11.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.380+0000 7f6dbbbb1640 1 --2- 172.21.15.5:0/3874682728 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6d9806be30 0x7f6d9806e340 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6db0034000 tx=0x7f6db002e000 comp rx=0 tx=0).stop 2024-05-20T18:37:11.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.380+0000 7f6dbbbb1640 1 -- 172.21.15.5:0/3874682728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6db4071630 msgr2=0x7f6db40b6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:11.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.380+0000 7f6dbbbb1640 1 --2- 172.21.15.5:0/3874682728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6db4071630 0x7f6db40b6010 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f6da400d620 tx=0x7f6da400dae0 comp rx=0 tx=0).stop 2024-05-20T18:37:11.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.380+0000 7f6dbbbb1640 1 -- 172.21.15.5:0/3874682728 shutdown_connections 2024-05-20T18:37:11.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.380+0000 7f6dbbbb1640 1 --2- 172.21.15.5:0/3874682728 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6d9806be30 0x7f6d9806e340 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:11.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.380+0000 7f6dbbbb1640 1 --2- 172.21.15.5:0/3874682728 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6db40cd1b0 0x7f6db40b6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:11.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.380+0000 7f6dbbbb1640 1 --2- 172.21.15.5:0/3874682728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6db4071630 0x7f6db40b6010 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:11.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.380+0000 7f6dbbbb1640 1 --2- 172.21.15.5:0/3874682728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6db4070850 0x7f6db4149fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:11.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.380+0000 7f6dbbbb1640 1 -- 172.21.15.5:0/3874682728 >> 172.21.15.5:0/3874682728 conn(0x7f6db40d29d0 msgr2=0x7f6db40cf200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:11.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.380+0000 7f6dbbbb1640 1 -- 172.21.15.5:0/3874682728 shutdown_connections 2024-05-20T18:37:11.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.380+0000 7f6dbbbb1640 1 -- 172.21.15.5:0/3874682728 wait complete. 2024-05-20T18:37:11.395 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:37:11.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.509+0000 7f04746b8640 1 Processor -- start 2024-05-20T18:37:11.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.509+0000 7f04746b8640 1 -- start start 2024-05-20T18:37:11.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.510+0000 7f04746b8640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f046c0b4010 0x7f046c0d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:11.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.510+0000 7f04746b8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f046c0d14b0 0x7f046c0d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:11.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.510+0000 7f04746b8640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f046c0d0370 con 0x7f046c163a10 2024-05-20T18:37:11.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.510+0000 7f04746b8640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f046c0d3ca0 con 0x7f046c0b4010 2024-05-20T18:37:11.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.510+0000 7f04746b8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f046c0d0650 con 0x7f046c0d14b0 2024-05-20T18:37:11.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.510+0000 7f0472454640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f046c163a10 0x7f046c0cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35328/0 (socket says 172.21.15.5:35328) 2024-05-20T18:37:11.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.510+0000 7f0472454640 1 -- 172.21.15.5:0/4057764713 learned_addr learned my addr 172.21.15.5:0/4057764713 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:11.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.510+0000 7f0472c55640 1 --2- 172.21.15.5:0/4057764713 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f046c0d14b0 0x7f046c0d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:11.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.510+0000 7f0471c53640 1 --2- 172.21.15.5:0/4057764713 >> v2:172.21.15.63:3300/0 conn(0x7f046c0b4010 0x7f046c0d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:11.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.511+0000 7f0471452640 1 -- 172.21.15.5:0/4057764713 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 467743611 0 0) 0x7f046c0d0370 con 0x7f046c163a10 2024-05-20T18:37:11.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.511+0000 7f0471452640 1 -- 172.21.15.5:0/4057764713 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0454003520 con 0x7f046c163a10 2024-05-20T18:37:11.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.511+0000 7f0471452640 1 -- 172.21.15.5:0/4057764713 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 785601780 0 0) 0x7f0454003520 con 0x7f046c163a10 2024-05-20T18:37:11.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.511+0000 7f0471452640 1 -- 172.21.15.5:0/4057764713 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f046c0d14b0 msgr2=0x7f046c0d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:11.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.511+0000 7f0471452640 1 --2- 172.21.15.5:0/4057764713 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f046c0d14b0 0x7f046c0d18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:11.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.511+0000 7f0471452640 1 -- 172.21.15.5:0/4057764713 >> v2:172.21.15.63:3300/0 conn(0x7f046c0b4010 msgr2=0x7f046c0d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:11.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.511+0000 7f0471452640 1 --2- 172.21.15.5:0/4057764713 >> v2:172.21.15.63:3300/0 conn(0x7f046c0b4010 0x7f046c0d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:11.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.511+0000 7f0471452640 1 -- 172.21.15.5:0/4057764713 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f046c0d3650 con 0x7f046c163a10 2024-05-20T18:37:11.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.511+0000 7f0471452640 1 -- 172.21.15.5:0/4057764713 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f0464001670 con 0x7f046c163a10 2024-05-20T18:37:11.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.511+0000 7f0471452640 1 -- 172.21.15.5:0/4057764713 >> v1:172.21.15.5:6789/0 conn(0x7f046c163a10 legacy=0x7f046c0cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:11.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.511+0000 7f0471452640 1 --2- 172.21.15.5:0/4057764713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04540039b0 0x7f0454003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:11.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.511+0000 7f0471452640 1 --2- 172.21.15.5:0/4057764713 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0454004800 0x7f0454004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:11.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.511+0000 7f0471452640 1 --2- 172.21.15.5:0/4057764713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0454005150 0x7f045400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:11.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.511+0000 7f0471452640 1 -- 172.21.15.5:0/4057764713 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f046c0d3ca0 con 0x7f0454005150 2024-05-20T18:37:11.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.511+0000 7f0471452640 1 -- 172.21.15.5:0/4057764713 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f046c0d0650 con 0x7f04540039b0 2024-05-20T18:37:11.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.511+0000 7f0471452640 1 -- 172.21.15.5:0/4057764713 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f045400bf80 con 0x7f0454004800 2024-05-20T18:37:11.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.511+0000 7f0471c53640 1 --2- 172.21.15.5:0/4057764713 >> v2:172.21.15.63:3300/0 conn(0x7f046c0b4010 0x7f046c0d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:37:11.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.511+0000 7f0472c55640 1 --2- 172.21.15.5:0/4057764713 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f046c0d14b0 0x7f046c0d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:37:11.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.511+0000 7f0472454640 1 --2- 172.21.15.5:0/4057764713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04540039b0 0x7f0454003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:11.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.511+0000 7f0472c55640 1 --2- 172.21.15.5:0/4057764713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0454005150 0x7f045400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:11.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.511+0000 7f0471c53640 1 --2- 172.21.15.5:0/4057764713 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0454004800 0x7f0454004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:11.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.512+0000 7f0472c55640 1 -- 172.21.15.5:0/4057764713 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0454004800 msgr2=0x7f0454004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:11.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.512+0000 7f0472c55640 1 --2- 172.21.15.5:0/4057764713 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0454004800 0x7f0454004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:11.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.512+0000 7f0472c55640 1 -- 172.21.15.5:0/4057764713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04540039b0 msgr2=0x7f0454003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:11.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.512+0000 7f0472c55640 1 --2- 172.21.15.5:0/4057764713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04540039b0 0x7f0454003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:11.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.512+0000 7f0472c55640 1 -- 172.21.15.5:0/4057764713 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0454003520 con 0x7f0454005150 2024-05-20T18:37:11.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.512+0000 7f0472c55640 1 --2- 172.21.15.5:0/4057764713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0454005150 0x7f045400b9c0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f046c0bad00 tx=0x7f0454005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3b4453691beafa server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:11.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.512+0000 7f0471452640 1 -- 172.21.15.5:0/4057764713 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c037070 con 0x7f0454005150 2024-05-20T18:37:11.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.513+0000 7f0471452640 1 -- 172.21.15.5:0/4057764713 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f045c003c50 con 0x7f0454005150 2024-05-20T18:37:11.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.513+0000 7f0471452640 1 -- 172.21.15.5:0/4057764713 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c009d40 con 0x7f0454005150 2024-05-20T18:37:11.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.513+0000 7f04746b8640 1 -- 172.21.15.5:0/4057764713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0454005150 msgr2=0x7f045400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:11.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.513+0000 7f04746b8640 1 --2- 172.21.15.5:0/4057764713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0454005150 0x7f045400b9c0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f046c0bad00 tx=0x7f0454005830 comp rx=0 tx=0).stop 2024-05-20T18:37:11.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.513+0000 7f0472c55640 1 -- 172.21.15.5:0/4057764713 reap_dead start 2024-05-20T18:37:11.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.513+0000 7f04746b8640 1 -- 172.21.15.5:0/4057764713 shutdown_connections 2024-05-20T18:37:11.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.513+0000 7f04746b8640 1 -- 172.21.15.5:0/4057764713 >> 172.21.15.5:0/4057764713 conn(0x7f046c06d240 msgr2=0x7f046c06d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:11.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.513+0000 7f04746b8640 1 -- 172.21.15.5:0/4057764713 shutdown_connections 2024-05-20T18:37:11.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.513+0000 7f04746b8640 1 -- 172.21.15.5:0/4057764713 wait complete. 2024-05-20T18:37:11.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.515+0000 7f04746b8640 1 Processor -- start 2024-05-20T18:37:11.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.515+0000 7f04746b8640 1 -- start start 2024-05-20T18:37:11.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.516+0000 7f04746b8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0454005150 0x7f046c157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:11.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.516+0000 7f04746b8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f046c0cda90 0x7f046c145ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:11.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.516+0000 7f04746b8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f046c0cdea0 0x7f046c1f8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:11.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.516+0000 7f04746b8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f046c0d1a40 con 0x7f046c0cdea0 2024-05-20T18:37:11.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.516+0000 7f04746b8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f046c0d18c0 con 0x7f0454005150 2024-05-20T18:37:11.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.516+0000 7f04746b8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f046c0d1bc0 con 0x7f046c0cda90 2024-05-20T18:37:11.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.517+0000 7f0472c55640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f046c0cdea0 0x7f046c1f8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.517+0000 7f0472c55640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f046c0cdea0 0x7f046c1f8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35474/0 (socket says 172.21.15.5:35474) 2024-05-20T18:37:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.517+0000 7f0472c55640 1 -- 172.21.15.5:0/2247004460 learned_addr learned my addr 172.21.15.5:0/2247004460 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.517+0000 7f0471c53640 1 --2- 172.21.15.5:0/2247004460 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f046c0cda90 0x7f046c145ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.517+0000 7f0472454640 1 --2- 172.21.15.5:0/2247004460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0454005150 0x7f046c157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.517+0000 7f0472c55640 1 -- 172.21.15.5:0/2247004460 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f046c0cda90 msgr2=0x7f046c145ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.517+0000 7f0472c55640 1 --2- 172.21.15.5:0/2247004460 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f046c0cda90 0x7f046c145ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.517+0000 7f0472c55640 1 -- 172.21.15.5:0/2247004460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0454005150 msgr2=0x7f046c157120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.517+0000 7f0472c55640 1 --2- 172.21.15.5:0/2247004460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0454005150 0x7f046c157120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.517+0000 7f0472c55640 1 -- 172.21.15.5:0/2247004460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f046c157660 con 0x7f046c0cdea0 2024-05-20T18:37:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.517+0000 7f0471c53640 1 --2- 172.21.15.5:0/2247004460 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f046c0cda90 0x7f046c145ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.517+0000 7f0472c55640 1 --2- 172.21.15.5:0/2247004460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f046c0cdea0 0x7f046c1f8350 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f046c0d4b90 tx=0x7f045c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.517+0000 7f046affd640 1 -- 172.21.15.5:0/2247004460 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c037070 con 0x7f046c0cdea0 2024-05-20T18:37:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.517+0000 7f046affd640 1 -- 172.21.15.5:0/2247004460 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f045c044070 con 0x7f046c0cdea0 2024-05-20T18:37:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.517+0000 7f04746b8640 1 -- 172.21.15.5:0/2247004460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f046c1456f0 con 0x7f046c0cdea0 2024-05-20T18:37:11.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.517+0000 7f046affd640 1 -- 172.21.15.5:0/2247004460 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c009d40 con 0x7f046c0cdea0 2024-05-20T18:37:11.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.517+0000 7f04746b8640 1 -- 172.21.15.5:0/2247004460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f046c154bc0 con 0x7f046c0cdea0 2024-05-20T18:37:11.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.518+0000 7f04746b8640 1 -- 172.21.15.5:0/2247004460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f046c075b00 con 0x7f046c0cdea0 2024-05-20T18:37:11.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.519+0000 7f046affd640 1 -- 172.21.15.5:0/2247004460 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f045c005d10 con 0x7f046c0cdea0 2024-05-20T18:37:11.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.519+0000 7f046affd640 1 --2- 172.21.15.5:0/2247004460 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f045806be30 0x7f045806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:11.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.520+0000 7f046affd640 1 -- 172.21.15.5:0/2247004460 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5249+0+0 (secure 0 0 0) 0x7f045c0aba60 con 0x7f046c0cdea0 2024-05-20T18:37:11.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.520+0000 7f0472454640 1 --2- 172.21.15.5:0/2247004460 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f045806be30 0x7f045806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:11.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.520+0000 7f0472454640 1 --2- 172.21.15.5:0/2247004460 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f045806be30 0x7f045806e340 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f0464000c00 tx=0x7f0464009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:11.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.524+0000 7f046affd640 1 -- 172.21.15.5:0/2247004460 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f045c02e050 con 0x7f046c0cdea0 2024-05-20T18:37:11.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.709+0000 7f04746b8640 1 -- 172.21.15.5:0/2247004460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f046c075180 con 0x7f046c0cdea0 2024-05-20T18:37:11.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.710+0000 7f046affd640 1 -- 172.21.15.5:0/2247004460 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+13054 (secure 0 0 0) 0x7f045c07a540 con 0x7f046c0cdea0 2024-05-20T18:37:11.711 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:11.711 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":115,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:37:10.980434+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:37:11.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.712+0000 7f04746b8640 1 -- 172.21.15.5:0/2247004460 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f045806be30 msgr2=0x7f045806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:11.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.712+0000 7f04746b8640 1 --2- 172.21.15.5:0/2247004460 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f045806be30 0x7f045806e340 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f0464000c00 tx=0x7f0464009040 comp rx=0 tx=0).stop 2024-05-20T18:37:11.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.712+0000 7f04746b8640 1 -- 172.21.15.5:0/2247004460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f046c0cdea0 msgr2=0x7f046c1f8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:11.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.712+0000 7f04746b8640 1 --2- 172.21.15.5:0/2247004460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f046c0cdea0 0x7f046c1f8350 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f046c0d4b90 tx=0x7f045c009750 comp rx=0 tx=0).stop 2024-05-20T18:37:11.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.712+0000 7f04746b8640 1 -- 172.21.15.5:0/2247004460 shutdown_connections 2024-05-20T18:37:11.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.712+0000 7f04746b8640 1 --2- 172.21.15.5:0/2247004460 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f045806be30 0x7f045806e340 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:11.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.712+0000 7f04746b8640 1 --2- 172.21.15.5:0/2247004460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f046c0cdea0 0x7f046c1f8350 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:11.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.712+0000 7f04746b8640 1 --2- 172.21.15.5:0/2247004460 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f046c0cda90 0x7f046c145ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:11.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.712+0000 7f04746b8640 1 --2- 172.21.15.5:0/2247004460 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0454005150 0x7f046c157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:11.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.712+0000 7f04746b8640 1 -- 172.21.15.5:0/2247004460 >> 172.21.15.5:0/2247004460 conn(0x7f046c06d240 msgr2=0x7f046c15da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:11.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.712+0000 7f04746b8640 1 -- 172.21.15.5:0/2247004460 shutdown_connections 2024-05-20T18:37:11.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.712+0000 7f04746b8640 1 -- 172.21.15.5:0/2247004460 wait complete. 2024-05-20T18:37:11.725 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:37:11.725 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:37:11.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.842+0000 7f50c2606640 1 Processor -- start 2024-05-20T18:37:11.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.842+0000 7f50c2606640 1 -- start start 2024-05-20T18:37:11.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.843+0000 7f50c2606640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f50bc162820 0x7f50bc0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:11.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.843+0000 7f50c2606640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f50bc06dae0 0x7f50bc0b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:11.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.843+0000 7f50c2606640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50bc0d0630 con 0x7f50bc0b3e80 2024-05-20T18:37:11.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.843+0000 7f50c2606640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f50bc0d39c0 con 0x7f50bc162820 2024-05-20T18:37:11.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.843+0000 7f50c2606640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f50bc0d3ca0 con 0x7f50bc06dae0 2024-05-20T18:37:11.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.843+0000 7f50bbfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f50bc162820 0x7f50bc0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:11.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.843+0000 7f50bb7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f50bc0b3e80 0x7f50bc06d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35340/0 (socket says 172.21.15.5:35340) 2024-05-20T18:37:11.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.843+0000 7f50bb7fe640 1 -- 172.21.15.5:0/373692897 learned_addr learned my addr 172.21.15.5:0/373692897 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:11.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.844+0000 7f50c0ba3640 1 --2- 172.21.15.5:0/373692897 >> v2:172.21.15.63:3300/0 conn(0x7f50bc06dae0 0x7f50bc0b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:11.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.844+0000 7f50bbfff640 1 -- 172.21.15.5:0/373692897 >> v2:172.21.15.63:3300/0 conn(0x7f50bc06dae0 msgr2=0x7f50bc0b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:11.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.844+0000 7f50bbfff640 1 --2- 172.21.15.5:0/373692897 >> v2:172.21.15.63:3300/0 conn(0x7f50bc06dae0 0x7f50bc0b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:11.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.844+0000 7f50bbfff640 1 -- 172.21.15.5:0/373692897 >> v1:172.21.15.5:6789/0 conn(0x7f50bc0b3e80 legacy=0x7f50bc06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:11.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.844+0000 7f50baffd640 1 -- 172.21.15.5:0/373692897 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1328896569 0 0) 0x7f50bc0d0630 con 0x7f50bc0b3e80 2024-05-20T18:37:11.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.844+0000 7f50bbfff640 1 -- 172.21.15.5:0/373692897 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50bc0d3f80 con 0x7f50bc162820 2024-05-20T18:37:11.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.844+0000 7f50bbfff640 1 --2- 172.21.15.5:0/373692897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50bc162820 0x7f50bc0b3940 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f50bc08e070 tx=0x7f50ac02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0de93fd6613f055 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:11.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.845+0000 7f50baffd640 1 -- 172.21.15.5:0/373692897 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac037070 con 0x7f50bc162820 2024-05-20T18:37:11.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.845+0000 7f50baffd640 1 -- 172.21.15.5:0/373692897 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50ac032440 con 0x7f50bc162820 2024-05-20T18:37:11.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.845+0000 7f50baffd640 1 -- 172.21.15.5:0/373692897 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac032810 con 0x7f50bc162820 2024-05-20T18:37:11.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.845+0000 7f50c2606640 1 -- 172.21.15.5:0/373692897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50bc162820 msgr2=0x7f50bc0b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:11.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.845+0000 7f50c2606640 1 --2- 172.21.15.5:0/373692897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50bc162820 0x7f50bc0b3940 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f50bc08e070 tx=0x7f50ac02a1a0 comp rx=0 tx=0).stop 2024-05-20T18:37:11.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.845+0000 7f50c2606640 1 -- 172.21.15.5:0/373692897 shutdown_connections 2024-05-20T18:37:11.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.845+0000 7f50c2606640 1 --2- 172.21.15.5:0/373692897 >> v2:172.21.15.63:3300/0 conn(0x7f50bc06dae0 0x7f50bc0b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:11.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.845+0000 7f50c2606640 1 --2- 172.21.15.5:0/373692897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50bc162820 0x7f50bc0b3940 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:11.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.845+0000 7f50c2606640 1 -- 172.21.15.5:0/373692897 >> 172.21.15.5:0/373692897 conn(0x7f50bc0d4bd0 msgr2=0x7f50bc0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:11.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.845+0000 7f50c2606640 1 -- 172.21.15.5:0/373692897 shutdown_connections 2024-05-20T18:37:11.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.845+0000 7f50c2606640 1 -- 172.21.15.5:0/373692897 wait complete. 2024-05-20T18:37:11.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.847+0000 7f50c2606640 1 Processor -- start 2024-05-20T18:37:11.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.847+0000 7f50c2606640 1 -- start start 2024-05-20T18:37:11.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.849+0000 7f50c2606640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50bc0b3e80 0x7f50bc1564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:11.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.849+0000 7f50c2606640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50bc162820 0x7f50bc15c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:11.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.849+0000 7f50c2606640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50bc0b5050 0x7f50bc15ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:11.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.849+0000 7f50c2606640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc0d4780 con 0x7f50bc162820 2024-05-20T18:37:11.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.849+0000 7f50c2606640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc0d0300 con 0x7f50bc0b5050 2024-05-20T18:37:11.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.849+0000 7f50c2606640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f50bc0d4900 con 0x7f50bc0b3e80 2024-05-20T18:37:11.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.849+0000 7f50bb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50bc162820 0x7f50bc15c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:11.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.849+0000 7f50bb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50bc162820 0x7f50bc15c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35494/0 (socket says 172.21.15.5:35494) 2024-05-20T18:37:11.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.849+0000 7f50bb7fe640 1 -- 172.21.15.5:0/132400432 learned_addr learned my addr 172.21.15.5:0/132400432 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:11.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.849+0000 7f50bbfff640 1 --2- 172.21.15.5:0/132400432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50bc0b3e80 0x7f50bc1564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:11.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.849+0000 7f50c0ba3640 1 --2- 172.21.15.5:0/132400432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50bc0b5050 0x7f50bc15ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:11.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.849+0000 7f50bb7fe640 1 -- 172.21.15.5:0/132400432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50bc0b3e80 msgr2=0x7f50bc1564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:11.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.849+0000 7f50bb7fe640 1 --2- 172.21.15.5:0/132400432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50bc0b3e80 0x7f50bc1564d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:11.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.849+0000 7f50bb7fe640 1 -- 172.21.15.5:0/132400432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50bc0b5050 msgr2=0x7f50bc15ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:11.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.849+0000 7f50bb7fe640 1 --2- 172.21.15.5:0/132400432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50bc0b5050 0x7f50bc15ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:11.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.849+0000 7f50bb7fe640 1 -- 172.21.15.5:0/132400432 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50bc159df0 con 0x7f50bc162820 2024-05-20T18:37:11.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.849+0000 7f50bb7fe640 1 --2- 172.21.15.5:0/132400432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50bc162820 0x7f50bc15c990 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f50b000c430 tx=0x7f50b000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:11.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.849+0000 7f50b8ff9640 1 -- 172.21.15.5:0/132400432 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b00074e0 con 0x7f50bc162820 2024-05-20T18:37:11.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.849+0000 7f50b8ff9640 1 -- 172.21.15.5:0/132400432 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50b00047e0 con 0x7f50bc162820 2024-05-20T18:37:11.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.849+0000 7f50c2606640 1 -- 172.21.15.5:0/132400432 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50bc157410 con 0x7f50bc162820 2024-05-20T18:37:11.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.850+0000 7f50b8ff9640 1 -- 172.21.15.5:0/132400432 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b0004bd0 con 0x7f50bc162820 2024-05-20T18:37:11.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.850+0000 7f50c2606640 1 -- 172.21.15.5:0/132400432 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50bc1579d0 con 0x7f50bc162820 2024-05-20T18:37:11.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.851+0000 7f50c2606640 1 -- 172.21.15.5:0/132400432 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50bc075b00 con 0x7f50bc162820 2024-05-20T18:37:11.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.851+0000 7f50b8ff9640 1 -- 172.21.15.5:0/132400432 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f50b0011070 con 0x7f50bc162820 2024-05-20T18:37:11.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.852+0000 7f50b8ff9640 1 --2- 172.21.15.5:0/132400432 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f50a806be30 0x7f50a806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:11.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.852+0000 7f50b8ff9640 1 -- 172.21.15.5:0/132400432 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5249+0+0 (secure 0 0 0) 0x7f50b008a6c0 con 0x7f50bc162820 2024-05-20T18:37:11.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.852+0000 7f50bbfff640 1 --2- 172.21.15.5:0/132400432 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f50a806be30 0x7f50a806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:11.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.852+0000 7f50bbfff640 1 --2- 172.21.15.5:0/132400432 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f50a806be30 0x7f50a806e340 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f50ac034000 tx=0x7f50ac02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:11.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:11.856+0000 7f50b8ff9640 1 -- 172.21.15.5:0/132400432 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f50b0058c70 con 0x7f50bc162820 2024-05-20T18:37:12.029 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:37:12.031 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:37:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.045+0000 7f50c2606640 1 -- 172.21.15.5:0/132400432 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f50bc159b70 con 0x7f50bc162820 2024-05-20T18:37:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.046+0000 7f50b8ff9640 1 -- 172.21.15.5:0/132400432 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v115) ==== 93+0+10 (secure 0 0 0) 0x7f50bc159b70 con 0x7f50bc162820 2024-05-20T18:37:12.047 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:37:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.048+0000 7f50c2606640 1 -- 172.21.15.5:0/132400432 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f50a806be30 msgr2=0x7f50a806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.048+0000 7f50c2606640 1 --2- 172.21.15.5:0/132400432 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f50a806be30 0x7f50a806e340 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f50ac034000 tx=0x7f50ac02e000 comp rx=0 tx=0).stop 2024-05-20T18:37:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.048+0000 7f50c2606640 1 -- 172.21.15.5:0/132400432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50bc162820 msgr2=0x7f50bc15c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.048+0000 7f50c2606640 1 --2- 172.21.15.5:0/132400432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50bc162820 0x7f50bc15c990 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f50b000c430 tx=0x7f50b000c8f0 comp rx=0 tx=0).stop 2024-05-20T18:37:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.048+0000 7f50c2606640 1 -- 172.21.15.5:0/132400432 shutdown_connections 2024-05-20T18:37:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.048+0000 7f50c2606640 1 --2- 172.21.15.5:0/132400432 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f50a806be30 0x7f50a806e340 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.048+0000 7f50c2606640 1 --2- 172.21.15.5:0/132400432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50bc0b5050 0x7f50bc15ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.048+0000 7f50c2606640 1 --2- 172.21.15.5:0/132400432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50bc162820 0x7f50bc15c990 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.048+0000 7f50c2606640 1 --2- 172.21.15.5:0/132400432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50bc0b3e80 0x7f50bc1564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.048+0000 7f50c2606640 1 -- 172.21.15.5:0/132400432 >> 172.21.15.5:0/132400432 conn(0x7f50bc0d4bd0 msgr2=0x7f50bc15eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:12.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.048+0000 7f50c2606640 1 -- 172.21.15.5:0/132400432 shutdown_connections 2024-05-20T18:37:12.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.048+0000 7f50c2606640 1 -- 172.21.15.5:0/132400432 wait complete. 2024-05-20T18:37:12.062 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-20T18:37:12.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.234+0000 7fc11a365640 1 Processor -- start 2024-05-20T18:37:12.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.234+0000 7fc11a365640 1 -- start start 2024-05-20T18:37:12.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.235+0000 7fc11a365640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc114163a10 0x7fc1140d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:12.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.235+0000 7fc11a365640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1140b4010 0x7fc1140cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:12.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.235+0000 7fc11a365640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc1140d14b0 0x7fc1140d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:12.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.235+0000 7fc11a365640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fc1140d3ca0 con 0x7fc114163a10 2024-05-20T18:37:12.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.236+0000 7fc11a365640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc1140d0370 con 0x7fc1140d14b0 2024-05-20T18:37:12.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.236+0000 7fc11a365640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc1140d0650 con 0x7fc1140b4010 2024-05-20T18:37:12.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.236+0000 7fc113fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc114163a10 0x7fc1140d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:12.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.236+0000 7fc113fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc114163a10 0x7fc1140d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35502/0 (socket says 172.21.15.5:35502) 2024-05-20T18:37:12.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.236+0000 7fc113fff640 1 -- 172.21.15.5:0/3216352432 learned_addr learned my addr 172.21.15.5:0/3216352432 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:12.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.236+0000 7fc10bfff640 1 --2- 172.21.15.5:0/3216352432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1140b4010 0x7fc1140cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:12.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.236+0000 7fc118902640 1 --2- 172.21.15.5:0/3216352432 >> v2:172.21.15.63:3300/0 conn(0x7fc1140d14b0 0x7fc1140d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:12.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.236+0000 7fc113fff640 1 -- 172.21.15.5:0/3216352432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1140b4010 msgr2=0x7fc1140cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:12.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.236+0000 7fc113fff640 1 --2- 172.21.15.5:0/3216352432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1140b4010 0x7fc1140cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:12.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.236+0000 7fc113fff640 1 -- 172.21.15.5:0/3216352432 >> v2:172.21.15.63:3300/0 conn(0x7fc1140d14b0 msgr2=0x7fc1140d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:12.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.236+0000 7fc113fff640 1 --2- 172.21.15.5:0/3216352432 >> v2:172.21.15.63:3300/0 conn(0x7fc1140d14b0 0x7fc1140d18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.236+0000 7fc113fff640 1 -- 172.21.15.5:0/3216352432 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1140d3650 con 0x7fc114163a10 2024-05-20T18:37:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.237+0000 7fc113fff640 1 --2- 172.21.15.5:0/3216352432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc114163a10 0x7fc1140d4130 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fc11408dea0 tx=0x7fc10402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f7b05026d4475e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.237+0000 7fc1137fe640 1 -- 172.21.15.5:0/3216352432 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc104037070 con 0x7fc114163a10 2024-05-20T18:37:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.237+0000 7fc1137fe640 1 -- 172.21.15.5:0/3216352432 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc104032440 con 0x7fc114163a10 2024-05-20T18:37:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.237+0000 7fc1137fe640 1 -- 172.21.15.5:0/3216352432 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc104032810 con 0x7fc114163a10 2024-05-20T18:37:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.238+0000 7fc11a365640 1 -- 172.21.15.5:0/3216352432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc114163a10 msgr2=0x7fc1140d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.238+0000 7fc11a365640 1 --2- 172.21.15.5:0/3216352432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc114163a10 0x7fc1140d4130 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fc11408dea0 tx=0x7fc10402a1e0 comp rx=0 tx=0).stop 2024-05-20T18:37:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.238+0000 7fc11a365640 1 -- 172.21.15.5:0/3216352432 shutdown_connections 2024-05-20T18:37:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.238+0000 7fc11a365640 1 --2- 172.21.15.5:0/3216352432 >> v2:172.21.15.63:3300/0 conn(0x7fc1140d14b0 0x7fc1140d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.238+0000 7fc11a365640 1 --2- 172.21.15.5:0/3216352432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1140b4010 0x7fc1140cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.238+0000 7fc11a365640 1 --2- 172.21.15.5:0/3216352432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc114163a10 0x7fc1140d4130 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.238+0000 7fc11a365640 1 -- 172.21.15.5:0/3216352432 >> 172.21.15.5:0/3216352432 conn(0x7fc11406d240 msgr2=0x7fc11406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.238+0000 7fc11a365640 1 -- 172.21.15.5:0/3216352432 shutdown_connections 2024-05-20T18:37:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.238+0000 7fc11a365640 1 -- 172.21.15.5:0/3216352432 wait complete. 2024-05-20T18:37:12.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.240+0000 7fc11a365640 1 Processor -- start 2024-05-20T18:37:12.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.240+0000 7fc11a365640 1 -- start start 2024-05-20T18:37:12.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.241+0000 7fc11a365640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1140b4010 0x7fc1140ce530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:12.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.241+0000 7fc11a365640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1140d14b0 0x7fc1141f9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:12.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.241+0000 7fc11a365640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc114163a10 0x7fc11415d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:12.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.241+0000 7fc11a365640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc1140d1e00 con 0x7fc1140d14b0 2024-05-20T18:37:12.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.241+0000 7fc11a365640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc1140d0040 con 0x7fc114163a10 2024-05-20T18:37:12.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.241+0000 7fc11a365640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc1140b8210 con 0x7fc1140b4010 2024-05-20T18:37:12.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.241+0000 7fc10bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1140d14b0 0x7fc1141f9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:12.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.241+0000 7fc10bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1140d14b0 0x7fc1141f9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35506/0 (socket says 172.21.15.5:35506) 2024-05-20T18:37:12.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.241+0000 7fc10bfff640 1 -- 172.21.15.5:0/778420206 learned_addr learned my addr 172.21.15.5:0/778420206 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:12.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.241+0000 7fc113fff640 1 --2- 172.21.15.5:0/778420206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1140b4010 0x7fc1140ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:12.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.241+0000 7fc118902640 1 --2- 172.21.15.5:0/778420206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc114163a10 0x7fc11415d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:12.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.241+0000 7fc10bfff640 1 -- 172.21.15.5:0/778420206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1140b4010 msgr2=0x7fc1140ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:12.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.241+0000 7fc10bfff640 1 --2- 172.21.15.5:0/778420206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1140b4010 0x7fc1140ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:12.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.241+0000 7fc10bfff640 1 -- 172.21.15.5:0/778420206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc114163a10 msgr2=0x7fc11415d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:12.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.241+0000 7fc10bfff640 1 --2- 172.21.15.5:0/778420206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc114163a10 0x7fc11415d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:12.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.241+0000 7fc10bfff640 1 -- 172.21.15.5:0/778420206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc114147640 con 0x7fc1140d14b0 2024-05-20T18:37:12.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.242+0000 7fc10bfff640 1 --2- 172.21.15.5:0/778420206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1140d14b0 0x7fc1141f9930 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00d830 tx=0x7fc0fc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:12.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.242+0000 7fc1117fa640 1 -- 172.21.15.5:0/778420206 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc018030 con 0x7fc1140d14b0 2024-05-20T18:37:12.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.242+0000 7fc1117fa640 1 -- 172.21.15.5:0/778420206 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0fc00be10 con 0x7fc1140d14b0 2024-05-20T18:37:12.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.242+0000 7fc1117fa640 1 -- 172.21.15.5:0/778420206 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc0098b0 con 0x7fc1140d14b0 2024-05-20T18:37:12.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.242+0000 7fc11a365640 1 -- 172.21.15.5:0/778420206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1141f9e70 con 0x7fc1140d14b0 2024-05-20T18:37:12.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.242+0000 7fc11a365640 1 -- 172.21.15.5:0/778420206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc114146b60 con 0x7fc1140d14b0 2024-05-20T18:37:12.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.243+0000 7fc11a365640 1 -- 172.21.15.5:0/778420206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1140d23e0 con 0x7fc1140d14b0 2024-05-20T18:37:12.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.243+0000 7fc1117fa640 1 -- 172.21.15.5:0/778420206 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc0fc01b070 con 0x7fc1140d14b0 2024-05-20T18:37:12.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.244+0000 7fc1117fa640 1 --2- 172.21.15.5:0/778420206 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc0f406be30 0x7fc0f406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:12.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.244+0000 7fc1117fa640 1 -- 172.21.15.5:0/778420206 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5249+0+0 (secure 0 0 0) 0x7fc0fc08ad60 con 0x7fc1140d14b0 2024-05-20T18:37:12.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.244+0000 7fc113fff640 1 --2- 172.21.15.5:0/778420206 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc0f406be30 0x7fc0f406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:12.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.244+0000 7fc113fff640 1 --2- 172.21.15.5:0/778420206 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc0f406be30 0x7fc0f406e340 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc104034000 tx=0x7fc10402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:12.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.249+0000 7fc1117fa640 1 -- 172.21.15.5:0/778420206 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc1140d23e0 con 0x7fc1140d14b0 2024-05-20T18:37:12.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.433+0000 7fc11a365640 1 -- 172.21.15.5:0/778420206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc114155200 con 0x7fc1140d14b0 2024-05-20T18:37:12.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.433+0000 7fc1117fa640 1 -- 172.21.15.5:0/778420206 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v115) ==== 84+0+65 (secure 0 0 0) 0x7fc0fc0591f0 con 0x7fc1140d14b0 2024-05-20T18:37:12.435 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:12.435 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-05-20T18:37:12.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-20T18:37:12.435 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-05-20T18:37:12.435 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-05-20T18:37:12.435 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-20T18:37:12.435 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-05-20T18:37:12.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.435+0000 7fc11a365640 1 -- 172.21.15.5:0/778420206 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc0f406be30 msgr2=0x7fc0f406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:12.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.435+0000 7fc11a365640 1 --2- 172.21.15.5:0/778420206 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc0f406be30 0x7fc0f406e340 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc104034000 tx=0x7fc10402e000 comp rx=0 tx=0).stop 2024-05-20T18:37:12.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.435+0000 7fc11a365640 1 -- 172.21.15.5:0/778420206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1140d14b0 msgr2=0x7fc1141f9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:12.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.435+0000 7fc11a365640 1 --2- 172.21.15.5:0/778420206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1140d14b0 0x7fc1141f9930 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00d830 tx=0x7fc0fc00dcf0 comp rx=0 tx=0).stop 2024-05-20T18:37:12.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.435+0000 7fc11a365640 1 -- 172.21.15.5:0/778420206 shutdown_connections 2024-05-20T18:37:12.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.435+0000 7fc11a365640 1 --2- 172.21.15.5:0/778420206 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc0f406be30 0x7fc0f406e340 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:12.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.435+0000 7fc11a365640 1 --2- 172.21.15.5:0/778420206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc114163a10 0x7fc11415d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:12.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.435+0000 7fc11a365640 1 --2- 172.21.15.5:0/778420206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1140d14b0 0x7fc1141f9930 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:12.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.435+0000 7fc11a365640 1 --2- 172.21.15.5:0/778420206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1140b4010 0x7fc1140ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:12.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.435+0000 7fc11a365640 1 -- 172.21.15.5:0/778420206 >> 172.21.15.5:0/778420206 conn(0x7fc11406d240 msgr2=0x7fc1140ba5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:12.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.435+0000 7fc11a365640 1 -- 172.21.15.5:0/778420206 shutdown_connections 2024-05-20T18:37:12.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.435+0000 7fc11a365640 1 -- 172.21.15.5:0/778420206 wait complete. 2024-05-20T18:37:12.448 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-20T18:37:12.448 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-20T18:37:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.566+0000 7f04655aa640 1 Processor -- start 2024-05-20T18:37:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.566+0000 7f04655aa640 1 -- start start 2024-05-20T18:37:12.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.567+0000 7f04655aa640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f04600d04a0 0x7f046015f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:12.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.567+0000 7f04655aa640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0460162fb0 con 0x7f04600b3850 2024-05-20T18:37:12.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.568+0000 7f04655aa640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0460163290 con 0x7f04600b6d50 2024-05-20T18:37:12.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.568+0000 7f04655aa640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f0460162d60 con 0x7f04600d04a0 2024-05-20T18:37:12.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.568+0000 7f045f7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f04600d04a0 0x7f046015f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:12.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.568+0000 7f045effd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f04600b3850 0x7f04600b6640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35350/0 (socket says 172.21.15.5:35350) 2024-05-20T18:37:12.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.568+0000 7f045effd640 1 -- 172.21.15.5:0/1474804362 learned_addr learned my addr 172.21.15.5:0/1474804362 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:12.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.568+0000 7f045f7fe640 1 -- 172.21.15.5:0/1474804362 >> v1:172.21.15.63:6789/0 conn(0x7f04600b6d50 legacy=0x7f04600d3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:12.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.568+0000 7f045f7fe640 1 -- 172.21.15.5:0/1474804362 >> v1:172.21.15.5:6789/0 conn(0x7f04600b3850 legacy=0x7f04600b6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:12.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.568+0000 7f045f7fe640 1 -- 172.21.15.5:0/1474804362 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0460163570 con 0x7f04600d04a0 2024-05-20T18:37:12.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.568+0000 7f045e7fc640 1 -- 172.21.15.5:0/1474804362 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 757130634 0 0) 0x7f0460162fb0 con 0x7f04600b3850 2024-05-20T18:37:12.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.569+0000 7f045f7fe640 1 --2- 172.21.15.5:0/1474804362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04600d04a0 0x7f046015f2c0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f0460162ee0 tx=0x7f045802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb34df16ab751576 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:12.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.569+0000 7f045e7fc640 1 -- 172.21.15.5:0/1474804362 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0458031600 con 0x7f04600d04a0 2024-05-20T18:37:12.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.569+0000 7f045e7fc640 1 -- 172.21.15.5:0/1474804362 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04580317a0 con 0x7f04600d04a0 2024-05-20T18:37:12.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.569+0000 7f045e7fc640 1 -- 172.21.15.5:0/1474804362 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0458031b70 con 0x7f04600d04a0 2024-05-20T18:37:12.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.570+0000 7f04655aa640 1 -- 172.21.15.5:0/1474804362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04600d04a0 msgr2=0x7f046015f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:12.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.570+0000 7f04655aa640 1 --2- 172.21.15.5:0/1474804362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04600d04a0 0x7f046015f2c0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f0460162ee0 tx=0x7f045802a610 comp rx=0 tx=0).stop 2024-05-20T18:37:12.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.570+0000 7f04655aa640 1 -- 172.21.15.5:0/1474804362 shutdown_connections 2024-05-20T18:37:12.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.570+0000 7f04655aa640 1 --2- 172.21.15.5:0/1474804362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04600d04a0 0x7f046015f2c0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:12.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.570+0000 7f04655aa640 1 -- 172.21.15.5:0/1474804362 >> 172.21.15.5:0/1474804362 conn(0x7f04600d1820 msgr2=0x7f04600d1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:12.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.570+0000 7f04655aa640 1 -- 172.21.15.5:0/1474804362 shutdown_connections 2024-05-20T18:37:12.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.570+0000 7f04655aa640 1 -- 172.21.15.5:0/1474804362 wait complete. 2024-05-20T18:37:12.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.571+0000 7f04655aa640 1 Processor -- start 2024-05-20T18:37:12.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.572+0000 7f04655aa640 1 -- start start 2024-05-20T18:37:12.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.573+0000 7f04655aa640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04600b3850 0x7f0460161fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:12.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.573+0000 7f04655aa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04600d04a0 0x7f0460147a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:12.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.573+0000 7f04655aa640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04601f9410 0x7f04601f8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:12.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.573+0000 7f04655aa640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f04600b8300 con 0x7f04600d04a0 2024-05-20T18:37:12.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.573+0000 7f04655aa640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f04600b8180 con 0x7f04600b3850 2024-05-20T18:37:12.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.573+0000 7f04655aa640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f04600b3da0 con 0x7f04601f9410 2024-05-20T18:37:12.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.573+0000 7f0457fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04600d04a0 0x7f0460147a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:12.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.573+0000 7f0457fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04600d04a0 0x7f0460147a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35536/0 (socket says 172.21.15.5:35536) 2024-05-20T18:37:12.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.573+0000 7f0457fff640 1 -- 172.21.15.5:0/2592904912 learned_addr learned my addr 172.21.15.5:0/2592904912 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:12.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.573+0000 7f045effd640 1 --2- 172.21.15.5:0/2592904912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04600b3850 0x7f0460161fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:12.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.573+0000 7f045f7fe640 1 --2- 172.21.15.5:0/2592904912 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04601f9410 0x7f04601f8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:12.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.573+0000 7f0457fff640 1 -- 172.21.15.5:0/2592904912 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04601f9410 msgr2=0x7f04601f8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:12.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.573+0000 7f0457fff640 1 --2- 172.21.15.5:0/2592904912 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04601f9410 0x7f04601f8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:12.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.573+0000 7f0457fff640 1 -- 172.21.15.5:0/2592904912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04600b3850 msgr2=0x7f0460161fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:12.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.573+0000 7f0457fff640 1 --2- 172.21.15.5:0/2592904912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04600b3850 0x7f0460161fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:12.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.573+0000 7f0457fff640 1 -- 172.21.15.5:0/2592904912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0460162950 con 0x7f04600d04a0 2024-05-20T18:37:12.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.573+0000 7f045f7fe640 1 --2- 172.21.15.5:0/2592904912 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04601f9410 0x7f04601f8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:12.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.574+0000 7f0457fff640 1 --2- 172.21.15.5:0/2592904912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04600d04a0 0x7f0460147a90 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f044800b3f0 tx=0x7f044800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:12.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.574+0000 7f04577fe640 1 -- 172.21.15.5:0/2592904912 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0448015020 con 0x7f04600d04a0 2024-05-20T18:37:12.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.574+0000 7f04577fe640 1 -- 172.21.15.5:0/2592904912 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0448011830 con 0x7f04600d04a0 2024-05-20T18:37:12.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.574+0000 7f04577fe640 1 -- 172.21.15.5:0/2592904912 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0448011c20 con 0x7f04600d04a0 2024-05-20T18:37:12.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.574+0000 7f04655aa640 1 -- 172.21.15.5:0/2592904912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f046015f860 con 0x7f04600d04a0 2024-05-20T18:37:12.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.574+0000 7f04655aa640 1 -- 172.21.15.5:0/2592904912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f046015fe20 con 0x7f04600d04a0 2024-05-20T18:37:12.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.575+0000 7f04655aa640 1 -- 172.21.15.5:0/2592904912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04600690b0 con 0x7f04600d04a0 2024-05-20T18:37:12.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.575+0000 7f04577fe640 1 -- 172.21.15.5:0/2592904912 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0448018020 con 0x7f04600d04a0 2024-05-20T18:37:12.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.576+0000 7f04577fe640 1 --2- 172.21.15.5:0/2592904912 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f044006be60 0x7f044006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:12.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.576+0000 7f04577fe640 1 -- 172.21.15.5:0/2592904912 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5249+0+0 (secure 0 0 0) 0x7f044808a8b0 con 0x7f04600d04a0 2024-05-20T18:37:12.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.576+0000 7f045effd640 1 --2- 172.21.15.5:0/2592904912 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f044006be60 0x7f044006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:12.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.576+0000 7f045effd640 1 --2- 172.21.15.5:0/2592904912 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f044006be60 0x7f044006e370 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0450007fd0 tx=0x7f0450007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:12.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.581+0000 7f04577fe640 1 -- 172.21.15.5:0/2592904912 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f044801f030 con 0x7f04600d04a0 2024-05-20T18:37:12.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.765+0000 7f04655aa640 1 -- 172.21.15.5:0/2592904912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f04600c7e10 con 0x7f04600d04a0 2024-05-20T18:37:12.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.971+0000 7f04577fe640 1 -- 172.21.15.5:0/2592904912 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v116) ==== 138+0+0 (secure 0 0 0) 0x7f0448059390 con 0x7f04600d04a0 2024-05-20T18:37:12.972 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-05-20T18:37:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.973+0000 7f04655aa640 1 -- 172.21.15.5:0/2592904912 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f044006be60 msgr2=0x7f044006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.973+0000 7f04655aa640 1 --2- 172.21.15.5:0/2592904912 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f044006be60 0x7f044006e370 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0450007fd0 tx=0x7f0450007de0 comp rx=0 tx=0).stop 2024-05-20T18:37:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.973+0000 7f04655aa640 1 -- 172.21.15.5:0/2592904912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04600d04a0 msgr2=0x7f0460147a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.973+0000 7f04655aa640 1 --2- 172.21.15.5:0/2592904912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04600d04a0 0x7f0460147a90 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f044800b3f0 tx=0x7f044800b8b0 comp rx=0 tx=0).stop 2024-05-20T18:37:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.973+0000 7f04655aa640 1 -- 172.21.15.5:0/2592904912 shutdown_connections 2024-05-20T18:37:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.973+0000 7f04655aa640 1 --2- 172.21.15.5:0/2592904912 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f044006be60 0x7f044006e370 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.973+0000 7f04655aa640 1 --2- 172.21.15.5:0/2592904912 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04601f9410 0x7f04601f8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.973+0000 7f04655aa640 1 --2- 172.21.15.5:0/2592904912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04600d04a0 0x7f0460147a90 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:12.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.973+0000 7f04655aa640 1 --2- 172.21.15.5:0/2592904912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04600b3850 0x7f0460161fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:12.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.973+0000 7f04655aa640 1 -- 172.21.15.5:0/2592904912 >> 172.21.15.5:0/2592904912 conn(0x7f04600d1820 msgr2=0x7f04600b9ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:12.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.973+0000 7f04655aa640 1 -- 172.21.15.5:0/2592904912 shutdown_connections 2024-05-20T18:37:12.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:12.973+0000 7f04655aa640 1 -- 172.21.15.5:0/2592904912 wait complete. 2024-05-20T18:37:12.990 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-20T18:37:13.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.111+0000 7fb6c41a3640 1 Processor -- start 2024-05-20T18:37:13.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.112+0000 7fb6c41a3640 1 -- start start 2024-05-20T18:37:13.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.113+0000 7fb6c41a3640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb6bc0d20f0 0x7fb6bc0b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:13.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.113+0000 7fb6c41a3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb6bc0b4020 0x7fb6bc0b83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:13.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.113+0000 7fb6c41a3640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6bc0d3ae0 con 0x7fb6bc15f6a0 2024-05-20T18:37:13.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.113+0000 7fb6c41a3640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb6bc0d3370 con 0x7fb6bc0d20f0 2024-05-20T18:37:13.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.113+0000 7fb6c41a3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb6bc0d4680 con 0x7fb6bc0b4020 2024-05-20T18:37:13.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.113+0000 7fb6c2740640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb6bc0b4020 0x7fb6bc0b83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:13.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.113+0000 7fb6c173e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb6bc0d20f0 0x7fb6bc0b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:13.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.113+0000 7fb6c2740640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb6bc0b4020 0x7fb6bc0b83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:46048/0 (socket says 172.21.15.5:46048) 2024-05-20T18:37:13.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.113+0000 7fb6c2740640 1 -- 172.21.15.5:0/3038473740 learned_addr learned my addr 172.21.15.5:0/3038473740 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:13.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.114+0000 7fb6c173e640 1 -- 172.21.15.5:0/3038473740 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb6bc0b4020 msgr2=0x7fb6bc0b83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:13.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.114+0000 7fb6c173e640 1 --2- 172.21.15.5:0/3038473740 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb6bc0b4020 0x7fb6bc0b83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:13.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.114+0000 7fb6c173e640 1 -- 172.21.15.5:0/3038473740 >> v1:172.21.15.63:6789/0 conn(0x7fb6bc15f6a0 legacy=0x7fb6bc0d19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:13.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.114+0000 7fb6c173e640 1 -- 172.21.15.5:0/3038473740 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6bc0d3d50 con 0x7fb6bc0d20f0 2024-05-20T18:37:13.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.114+0000 7fb6c173e640 1 --2- 172.21.15.5:0/3038473740 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6bc0d20f0 0x7fb6bc0b3ae0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb6b00097a0 tx=0x7fb6b002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e97ac0854e271c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:13.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.115+0000 7fb6c0f3d640 1 -- 172.21.15.5:0/3038473740 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0037070 con 0x7fb6bc0d20f0 2024-05-20T18:37:13.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.115+0000 7fb6c0f3d640 1 -- 172.21.15.5:0/3038473740 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6b0032440 con 0x7fb6bc0d20f0 2024-05-20T18:37:13.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.115+0000 7fb6c0f3d640 1 -- 172.21.15.5:0/3038473740 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0032810 con 0x7fb6bc0d20f0 2024-05-20T18:37:13.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.115+0000 7fb6c41a3640 1 -- 172.21.15.5:0/3038473740 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6bc0d20f0 msgr2=0x7fb6bc0b3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:13.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.115+0000 7fb6c41a3640 1 --2- 172.21.15.5:0/3038473740 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6bc0d20f0 0x7fb6bc0b3ae0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb6b00097a0 tx=0x7fb6b002a180 comp rx=0 tx=0).stop 2024-05-20T18:37:13.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.115+0000 7fb6c41a3640 1 -- 172.21.15.5:0/3038473740 shutdown_connections 2024-05-20T18:37:13.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.115+0000 7fb6c41a3640 1 --2- 172.21.15.5:0/3038473740 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb6bc0b4020 0x7fb6bc0b83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:13.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.115+0000 7fb6c41a3640 1 --2- 172.21.15.5:0/3038473740 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6bc0d20f0 0x7fb6bc0b3ae0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:13.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.115+0000 7fb6c41a3640 1 -- 172.21.15.5:0/3038473740 >> 172.21.15.5:0/3038473740 conn(0x7fb6bc06d400 msgr2=0x7fb6bc06d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:13.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.116+0000 7fb6c41a3640 1 -- 172.21.15.5:0/3038473740 shutdown_connections 2024-05-20T18:37:13.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.116+0000 7fb6c41a3640 1 -- 172.21.15.5:0/3038473740 wait complete. 2024-05-20T18:37:13.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.117+0000 7fb6c41a3640 1 Processor -- start 2024-05-20T18:37:13.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.117+0000 7fb6c41a3640 1 -- start start 2024-05-20T18:37:13.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.119+0000 7fb6c41a3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb6bc0b4020 0x7fb6bc0cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:13.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.119+0000 7fb6c41a3640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6bc0d20f0 0x7fb6bc151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:13.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.119+0000 7fb6c41a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6bc15f6a0 0x7fb6bc1f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:13.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.119+0000 7fb6c41a3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb6bc0b4430 con 0x7fb6bc15f6a0 2024-05-20T18:37:13.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.119+0000 7fb6c41a3640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb6bc0d4310 con 0x7fb6bc0d20f0 2024-05-20T18:37:13.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.119+0000 7fb6c41a3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb6bc0b8920 con 0x7fb6bc0b4020 2024-05-20T18:37:13.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.119+0000 7fb6c2740640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6bc15f6a0 0x7fb6bc1f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:13.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.119+0000 7fb6c2740640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6bc15f6a0 0x7fb6bc1f81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35538/0 (socket says 172.21.15.5:35538) 2024-05-20T18:37:13.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.119+0000 7fb6c2740640 1 -- 172.21.15.5:0/4256718631 learned_addr learned my addr 172.21.15.5:0/4256718631 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:13.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.119+0000 7fb6c1f3f640 1 --2- 172.21.15.5:0/4256718631 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb6bc0b4020 0x7fb6bc0cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:13.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.119+0000 7fb6c173e640 1 --2- 172.21.15.5:0/4256718631 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6bc0d20f0 0x7fb6bc151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:13.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.119+0000 7fb6c2740640 1 -- 172.21.15.5:0/4256718631 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb6bc0b4020 msgr2=0x7fb6bc0cc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:13.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.119+0000 7fb6c2740640 1 --2- 172.21.15.5:0/4256718631 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb6bc0b4020 0x7fb6bc0cc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:13.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.119+0000 7fb6c2740640 1 -- 172.21.15.5:0/4256718631 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6bc0d20f0 msgr2=0x7fb6bc151090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:13.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.119+0000 7fb6c2740640 1 --2- 172.21.15.5:0/4256718631 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6bc0d20f0 0x7fb6bc151090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:13.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.119+0000 7fb6c2740640 1 -- 172.21.15.5:0/4256718631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6bc0cd260 con 0x7fb6bc15f6a0 2024-05-20T18:37:13.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.119+0000 7fb6c2740640 1 --2- 172.21.15.5:0/4256718631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6bc15f6a0 0x7fb6bc1f81f0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400e870 tx=0x7fb6b400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:13.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.120+0000 7fb6ba7fc640 1 -- 172.21.15.5:0/4256718631 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4017070 con 0x7fb6bc15f6a0 2024-05-20T18:37:13.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.120+0000 7fb6ba7fc640 1 -- 172.21.15.5:0/4256718631 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6b4009e20 con 0x7fb6bc15f6a0 2024-05-20T18:37:13.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.120+0000 7fb6c41a3640 1 -- 172.21.15.5:0/4256718631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6bc0cd540 con 0x7fb6bc15f6a0 2024-05-20T18:37:13.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.120+0000 7fb6ba7fc640 1 -- 172.21.15.5:0/4256718631 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b400cb80 con 0x7fb6bc15f6a0 2024-05-20T18:37:13.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.120+0000 7fb6c41a3640 1 -- 172.21.15.5:0/4256718631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6bc14ed70 con 0x7fb6bc15f6a0 2024-05-20T18:37:13.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.121+0000 7fb6c41a3640 1 -- 172.21.15.5:0/4256718631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6bc0ccfd0 con 0x7fb6bc15f6a0 2024-05-20T18:37:13.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.121+0000 7fb6ba7fc640 1 -- 172.21.15.5:0/4256718631 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb6b401a070 con 0x7fb6bc15f6a0 2024-05-20T18:37:13.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.122+0000 7fb6ba7fc640 1 --2- 172.21.15.5:0/4256718631 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb6a806be60 0x7fb6a806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:13.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.122+0000 7fb6ba7fc640 1 -- 172.21.15.5:0/4256718631 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5619+0+0 (secure 0 0 0) 0x7fb6b408af90 con 0x7fb6bc15f6a0 2024-05-20T18:37:13.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.122+0000 7fb6c1f3f640 1 --2- 172.21.15.5:0/4256718631 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb6a806be60 0x7fb6a806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:13.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.122+0000 7fb6c1f3f640 1 --2- 172.21.15.5:0/4256718631 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb6a806be60 0x7fb6a806e370 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc0d1090 tx=0x7fb6ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:13.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.127+0000 7fb6ba7fc640 1 -- 172.21.15.5:0/4256718631 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb6b40592c0 con 0x7fb6bc15f6a0 2024-05-20T18:37:13.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.314+0000 7fb6c41a3640 1 -- 172.21.15.5:0/4256718631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb6bc0690b0 con 0x7fb6bc15f6a0 2024-05-20T18:37:13.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.973+0000 7fb6ba7fc640 1 -- 172.21.15.5:0/4256718631 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v117) ==== 170+0+0 (secure 0 0 0) 0x7fb6b405d5c0 con 0x7fb6bc15f6a0 2024-05-20T18:37:13.975 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-05-20T18:37:13.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.976+0000 7fb6c41a3640 1 -- 172.21.15.5:0/4256718631 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb6a806be60 msgr2=0x7fb6a806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:13.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.976+0000 7fb6c41a3640 1 --2- 172.21.15.5:0/4256718631 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb6a806be60 0x7fb6a806e370 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc0d1090 tx=0x7fb6ac009040 comp rx=0 tx=0).stop 2024-05-20T18:37:13.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.976+0000 7fb6c41a3640 1 -- 172.21.15.5:0/4256718631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6bc15f6a0 msgr2=0x7fb6bc1f81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:13.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.976+0000 7fb6c41a3640 1 --2- 172.21.15.5:0/4256718631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6bc15f6a0 0x7fb6bc1f81f0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400e870 tx=0x7fb6b400ed30 comp rx=0 tx=0).stop 2024-05-20T18:37:13.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.976+0000 7fb6c41a3640 1 -- 172.21.15.5:0/4256718631 shutdown_connections 2024-05-20T18:37:13.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.976+0000 7fb6c41a3640 1 --2- 172.21.15.5:0/4256718631 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb6a806be60 0x7fb6a806e370 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:13.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.976+0000 7fb6c41a3640 1 --2- 172.21.15.5:0/4256718631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6bc15f6a0 0x7fb6bc1f81f0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:13.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.976+0000 7fb6c41a3640 1 --2- 172.21.15.5:0/4256718631 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6bc0d20f0 0x7fb6bc151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:13.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.976+0000 7fb6c41a3640 1 --2- 172.21.15.5:0/4256718631 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb6bc0b4020 0x7fb6bc0cc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:13.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.976+0000 7fb6c41a3640 1 -- 172.21.15.5:0/4256718631 >> 172.21.15.5:0/4256718631 conn(0x7fb6bc06d400 msgr2=0x7fb6bc0cf740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:13.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.977+0000 7fb6c41a3640 1 -- 172.21.15.5:0/4256718631 shutdown_connections 2024-05-20T18:37:13.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:13.977+0000 7fb6c41a3640 1 -- 172.21.15.5:0/4256718631 wait complete. 2024-05-20T18:37:14.005 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-20T18:37:14.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.127+0000 7fa653e75640 1 Processor -- start 2024-05-20T18:37:14.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.127+0000 7fa653e75640 1 -- start start 2024-05-20T18:37:14.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.128+0000 7fa653e75640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa64c0b1e10 0x7fa64c15aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:14.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.128+0000 7fa653e75640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa64c070850 0x7fa64c070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:14.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.128+0000 7fa653e75640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa64c0cd580 con 0x7fa64c15d530 2024-05-20T18:37:14.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.128+0000 7fa653e75640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa64c0ce190 con 0x7fa64c070850 2024-05-20T18:37:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.128+0000 7fa653e75640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa64c0cd860 con 0x7fa64c0b1e10 2024-05-20T18:37:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.128+0000 7fa651c11640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa64c15d530 0x7fa64c0b1700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35354/0 (socket says 172.21.15.5:35354) 2024-05-20T18:37:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.128+0000 7fa651c11640 1 -- 172.21.15.5:0/2444137350 learned_addr learned my addr 172.21.15.5:0/2444137350 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.128+0000 7fa652412640 1 --2- 172.21.15.5:0/2444137350 >> v2:172.21.15.63:3300/0 conn(0x7fa64c070850 0x7fa64c070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.128+0000 7fa651410640 1 --2- 172.21.15.5:0/2444137350 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa64c0b1e10 0x7fa64c15aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.129+0000 7fa650c0f640 1 -- 172.21.15.5:0/2444137350 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3579506936 0 0) 0x7fa64c0cd580 con 0x7fa64c15d530 2024-05-20T18:37:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.129+0000 7fa650c0f640 1 -- 172.21.15.5:0/2444137350 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa634003520 con 0x7fa64c15d530 2024-05-20T18:37:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.129+0000 7fa650c0f640 1 -- 172.21.15.5:0/2444137350 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2023727667 0 0) 0x7fa634003520 con 0x7fa64c15d530 2024-05-20T18:37:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.129+0000 7fa650c0f640 1 -- 172.21.15.5:0/2444137350 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa64c0b1e10 msgr2=0x7fa64c15aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.129+0000 7fa650c0f640 1 --2- 172.21.15.5:0/2444137350 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa64c0b1e10 0x7fa64c15aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.129+0000 7fa650c0f640 1 -- 172.21.15.5:0/2444137350 >> v2:172.21.15.63:3300/0 conn(0x7fa64c070850 msgr2=0x7fa64c070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.129+0000 7fa650c0f640 1 --2- 172.21.15.5:0/2444137350 >> v2:172.21.15.63:3300/0 conn(0x7fa64c070850 0x7fa64c070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.129+0000 7fa650c0f640 1 -- 172.21.15.5:0/2444137350 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64c0cdb40 con 0x7fa64c15d530 2024-05-20T18:37:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.129+0000 7fa650c0f640 1 -- 172.21.15.5:0/2444137350 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7fa644001670 con 0x7fa64c15d530 2024-05-20T18:37:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.129+0000 7fa650c0f640 1 -- 172.21.15.5:0/2444137350 >> v1:172.21.15.5:6789/0 conn(0x7fa64c15d530 legacy=0x7fa64c0b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.129+0000 7fa650c0f640 1 --2- 172.21.15.5:0/2444137350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6340039b0 0x7fa634003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.129+0000 7fa650c0f640 1 --2- 172.21.15.5:0/2444137350 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa64c0b1e10 0x7fa634004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.129+0000 7fa650c0f640 1 --2- 172.21.15.5:0/2444137350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa634004d40 0x7fa634005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.129+0000 7fa650c0f640 1 -- 172.21.15.5:0/2444137350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c0ce190 con 0x7fa6340039b0 2024-05-20T18:37:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.129+0000 7fa650c0f640 1 -- 172.21.15.5:0/2444137350 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c0cd860 con 0x7fa634004d40 2024-05-20T18:37:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.129+0000 7fa650c0f640 1 -- 172.21.15.5:0/2444137350 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa63400b9c0 con 0x7fa64c0b1e10 2024-05-20T18:37:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.129+0000 7fa651c11640 1 --2- 172.21.15.5:0/2444137350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6340039b0 0x7fa634003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.129+0000 7fa651410640 1 --2- 172.21.15.5:0/2444137350 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa64c0b1e10 0x7fa634004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.129+0000 7fa652412640 1 --2- 172.21.15.5:0/2444137350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa634004d40 0x7fa634005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.129+0000 7fa651c11640 1 -- 172.21.15.5:0/2444137350 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa64c0b1e10 msgr2=0x7fa634004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.129+0000 7fa651c11640 1 --2- 172.21.15.5:0/2444137350 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa64c0b1e10 0x7fa634004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.129+0000 7fa651c11640 1 -- 172.21.15.5:0/2444137350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa634004d40 msgr2=0x7fa634005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.129+0000 7fa651c11640 1 --2- 172.21.15.5:0/2444137350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa634004d40 0x7fa634005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.129+0000 7fa651c11640 1 -- 172.21.15.5:0/2444137350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa634003520 con 0x7fa6340039b0 2024-05-20T18:37:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.129+0000 7fa652412640 1 --2- 172.21.15.5:0/2444137350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa634004d40 0x7fa634005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:14.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.130+0000 7fa651c11640 1 --2- 172.21.15.5:0/2444137350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6340039b0 0x7fa634003e20 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fa644007fd0 tx=0x7fa634005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ee80ac18cca09f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:14.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.130+0000 7fa650c0f640 1 -- 172.21.15.5:0/2444137350 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa644007d50 con 0x7fa6340039b0 2024-05-20T18:37:14.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.130+0000 7fa650c0f640 1 -- 172.21.15.5:0/2444137350 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa644005ad0 con 0x7fa6340039b0 2024-05-20T18:37:14.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.130+0000 7fa650c0f640 1 -- 172.21.15.5:0/2444137350 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa644007070 con 0x7fa6340039b0 2024-05-20T18:37:14.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.131+0000 7fa653e75640 1 -- 172.21.15.5:0/2444137350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6340039b0 msgr2=0x7fa634003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:14.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.131+0000 7fa653e75640 1 --2- 172.21.15.5:0/2444137350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6340039b0 0x7fa634003e20 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fa644007fd0 tx=0x7fa634005830 comp rx=0 tx=0).stop 2024-05-20T18:37:14.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.131+0000 7fa652412640 1 -- 172.21.15.5:0/2444137350 reap_dead start 2024-05-20T18:37:14.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.131+0000 7fa653e75640 1 -- 172.21.15.5:0/2444137350 shutdown_connections 2024-05-20T18:37:14.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.131+0000 7fa653e75640 1 -- 172.21.15.5:0/2444137350 >> 172.21.15.5:0/2444137350 conn(0x7fa64c0d29d0 msgr2=0x7fa64c0d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:14.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.131+0000 7fa653e75640 1 -- 172.21.15.5:0/2444137350 shutdown_connections 2024-05-20T18:37:14.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.131+0000 7fa653e75640 1 -- 172.21.15.5:0/2444137350 wait complete. 2024-05-20T18:37:14.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.133+0000 7fa653e75640 1 Processor -- start 2024-05-20T18:37:14.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.133+0000 7fa653e75640 1 -- start start 2024-05-20T18:37:14.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.134+0000 7fa653e75640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6340039b0 0x7fa64c0cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:14.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.134+0000 7fa653e75640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa64c15aaa0 0x7fa64c0c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:14.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.134+0000 7fa653e75640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa64c0c8800 0x7fa64c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:14.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.134+0000 7fa653e75640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c0711a0 con 0x7fa64c15aaa0 2024-05-20T18:37:14.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.134+0000 7fa653e75640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c0cd230 con 0x7fa6340039b0 2024-05-20T18:37:14.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.134+0000 7fa653e75640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa64c071320 con 0x7fa64c0c8800 2024-05-20T18:37:14.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.134+0000 7fa651410640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa64c15aaa0 0x7fa64c0c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:14.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.134+0000 7fa651410640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa64c15aaa0 0x7fa64c0c82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35566/0 (socket says 172.21.15.5:35566) 2024-05-20T18:37:14.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.134+0000 7fa651410640 1 -- 172.21.15.5:0/648610426 learned_addr learned my addr 172.21.15.5:0/648610426 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:14.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.135+0000 7fa651c11640 1 --2- 172.21.15.5:0/648610426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6340039b0 0x7fa64c0cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:14.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.135+0000 7fa652412640 1 --2- 172.21.15.5:0/648610426 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa64c0c8800 0x7fa64c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:14.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.135+0000 7fa651410640 1 -- 172.21.15.5:0/648610426 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa64c0c8800 msgr2=0x7fa64c06c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:14.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.135+0000 7fa651410640 1 --2- 172.21.15.5:0/648610426 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa64c0c8800 0x7fa64c06c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:14.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.135+0000 7fa651410640 1 -- 172.21.15.5:0/648610426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6340039b0 msgr2=0x7fa64c0cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:14.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.135+0000 7fa651410640 1 --2- 172.21.15.5:0/648610426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6340039b0 0x7fa64c0cc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:14.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.135+0000 7fa651410640 1 -- 172.21.15.5:0/648610426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64c0c6500 con 0x7fa64c15aaa0 2024-05-20T18:37:14.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.135+0000 7fa652412640 1 --2- 172.21.15.5:0/648610426 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa64c0c8800 0x7fa64c06c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:14.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.135+0000 7fa651410640 1 --2- 172.21.15.5:0/648610426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa64c15aaa0 0x7fa64c0c82c0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fa640011600 tx=0x7fa640011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:14.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.135+0000 7fa64a7fc640 1 -- 172.21.15.5:0/648610426 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64000f920 con 0x7fa64c15aaa0 2024-05-20T18:37:14.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.135+0000 7fa64a7fc640 1 -- 172.21.15.5:0/648610426 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa64000fac0 con 0x7fa64c15aaa0 2024-05-20T18:37:14.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.135+0000 7fa653e75640 1 -- 172.21.15.5:0/648610426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa64c0c67e0 con 0x7fa64c15aaa0 2024-05-20T18:37:14.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.135+0000 7fa64a7fc640 1 -- 172.21.15.5:0/648610426 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64000fe70 con 0x7fa64c15aaa0 2024-05-20T18:37:14.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.135+0000 7fa653e75640 1 -- 172.21.15.5:0/648610426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa64c15b190 con 0x7fa64c15aaa0 2024-05-20T18:37:14.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.136+0000 7fa653e75640 1 -- 172.21.15.5:0/648610426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa64c0c9040 con 0x7fa64c15aaa0 2024-05-20T18:37:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.137+0000 7fa64a7fc640 1 -- 172.21.15.5:0/648610426 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa640015460 con 0x7fa64c15aaa0 2024-05-20T18:37:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.137+0000 7fa64a7fc640 1 --2- 172.21.15.5:0/648610426 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa63806be60 0x7fa63806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.137+0000 7fa651c11640 1 --2- 172.21.15.5:0/648610426 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa63806be60 0x7fa63806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.137+0000 7fa64a7fc640 1 -- 172.21.15.5:0/648610426 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6001+0+0 (secure 0 0 0) 0x7fa64008b9c0 con 0x7fa64c15aaa0 2024-05-20T18:37:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.138+0000 7fa651c11640 1 --2- 172.21.15.5:0/648610426 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa63806be60 0x7fa63806e370 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa644033000 tx=0x7fa644009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:14.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.142+0000 7fa64a7fc640 1 -- 172.21.15.5:0/648610426 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa64008de20 con 0x7fa64c15aaa0 2024-05-20T18:37:14.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.363+0000 7fa653e75640 1 -- 172.21.15.5:0/648610426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa64c075b00 con 0x7fa64c15aaa0 2024-05-20T18:37:14.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:14.994+0000 7fa64a7fc640 1 -- 172.21.15.5:0/648610426 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v95) ==== 174+0+0 (secure 0 0 0) 0x7fa640019030 con 0x7fa64c15aaa0 2024-05-20T18:37:14.996 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 17 and data pool 18 2024-05-20T18:37:15.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.007+0000 7fa653e75640 1 -- 172.21.15.5:0/648610426 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa63806be60 msgr2=0x7fa63806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:15.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.007+0000 7fa653e75640 1 --2- 172.21.15.5:0/648610426 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa63806be60 0x7fa63806e370 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa644033000 tx=0x7fa644009030 comp rx=0 tx=0).stop 2024-05-20T18:37:15.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.007+0000 7fa653e75640 1 -- 172.21.15.5:0/648610426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa64c15aaa0 msgr2=0x7fa64c0c82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:15.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.007+0000 7fa653e75640 1 --2- 172.21.15.5:0/648610426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa64c15aaa0 0x7fa64c0c82c0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fa640011600 tx=0x7fa640011ac0 comp rx=0 tx=0).stop 2024-05-20T18:37:15.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.008+0000 7fa653e75640 1 -- 172.21.15.5:0/648610426 shutdown_connections 2024-05-20T18:37:15.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.008+0000 7fa653e75640 1 --2- 172.21.15.5:0/648610426 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa63806be60 0x7fa63806e370 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:15.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.008+0000 7fa653e75640 1 --2- 172.21.15.5:0/648610426 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa64c0c8800 0x7fa64c06c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:15.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.008+0000 7fa653e75640 1 --2- 172.21.15.5:0/648610426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa64c15aaa0 0x7fa64c0c82c0 secure :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fa640011600 tx=0x7fa640011ac0 comp rx=0 tx=0).stop 2024-05-20T18:37:15.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.008+0000 7fa653e75640 1 --2- 172.21.15.5:0/648610426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6340039b0 0x7fa64c0cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:15.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.008+0000 7fa653e75640 1 -- 172.21.15.5:0/648610426 >> 172.21.15.5:0/648610426 conn(0x7fa64c0d29d0 msgr2=0x7fa64c0b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:15.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.010+0000 7fa653e75640 1 -- 172.21.15.5:0/648610426 shutdown_connections 2024-05-20T18:37:15.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.011+0000 7fa653e75640 1 -- 172.21.15.5:0/648610426 wait complete. 2024-05-20T18:37:15.031 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:37:15.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.148+0000 7f39dc4ca640 1 Processor -- start 2024-05-20T18:37:15.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.148+0000 7f39dc4ca640 1 -- start start 2024-05-20T18:37:15.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.149+0000 7f39dc4ca640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f39d40b4010 0x7f39d40d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:15.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.149+0000 7f39dc4ca640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39d40d0370 con 0x7f39d40d14b0 2024-05-20T18:37:15.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.149+0000 7f39dc4ca640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39d40d0650 con 0x7f39d4163a10 2024-05-20T18:37:15.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.149+0000 7f39dc4ca640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f39d40d3ca0 con 0x7f39d40b4010 2024-05-20T18:37:15.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.149+0000 7f39daa67640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f39d40d14b0 0x7f39d40d18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:40842/0 (socket says 172.21.15.5:40842) 2024-05-20T18:37:15.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.149+0000 7f39daa67640 1 -- 172.21.15.5:0/2283030434 learned_addr learned my addr 172.21.15.5:0/2283030434 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:15.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.150+0000 7f39d9a65640 1 --2- 172.21.15.5:0/2283030434 >> v2:172.21.15.63:3300/0 conn(0x7f39d40b4010 0x7f39d40d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:15.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.150+0000 7f39d9264640 1 -- 172.21.15.5:0/2283030434 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1832402906 0 0) 0x7f39d40d0370 con 0x7f39d40d14b0 2024-05-20T18:37:15.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.150+0000 7f39d9264640 1 -- 172.21.15.5:0/2283030434 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39c0003520 con 0x7f39d40d14b0 2024-05-20T18:37:15.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.150+0000 7f39d9a65640 1 -- 172.21.15.5:0/2283030434 >> v1:172.21.15.63:6789/0 conn(0x7f39d4163a10 legacy=0x7f39d40cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:15.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.150+0000 7f39d9a65640 1 -- 172.21.15.5:0/2283030434 >> v1:172.21.15.5:6789/0 conn(0x7f39d40d14b0 legacy=0x7f39d40d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:15.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.150+0000 7f39d9a65640 1 -- 172.21.15.5:0/2283030434 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d40d3650 con 0x7f39d40b4010 2024-05-20T18:37:15.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.151+0000 7f39d9a65640 1 --2- 172.21.15.5:0/2283030434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39d40b4010 0x7f39d40d4130 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f39c40097a0 tx=0x7f39c402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eeef9145a89a55d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:15.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.151+0000 7f39d9264640 1 -- 172.21.15.5:0/2283030434 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c4037070 con 0x7f39d40b4010 2024-05-20T18:37:15.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.151+0000 7f39d9264640 1 -- 172.21.15.5:0/2283030434 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39c4032440 con 0x7f39d40b4010 2024-05-20T18:37:15.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.151+0000 7f39d9264640 1 -- 172.21.15.5:0/2283030434 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c40327f0 con 0x7f39d40b4010 2024-05-20T18:37:15.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.152+0000 7f39dc4ca640 1 -- 172.21.15.5:0/2283030434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39d40b4010 msgr2=0x7f39d40d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:15.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.152+0000 7f39dc4ca640 1 --2- 172.21.15.5:0/2283030434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39d40b4010 0x7f39d40d4130 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f39c40097a0 tx=0x7f39c402a230 comp rx=0 tx=0).stop 2024-05-20T18:37:15.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.152+0000 7f39dc4ca640 1 -- 172.21.15.5:0/2283030434 shutdown_connections 2024-05-20T18:37:15.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.152+0000 7f39dc4ca640 1 --2- 172.21.15.5:0/2283030434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39d40b4010 0x7f39d40d4130 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:15.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.152+0000 7f39dc4ca640 1 -- 172.21.15.5:0/2283030434 >> 172.21.15.5:0/2283030434 conn(0x7f39d406d240 msgr2=0x7f39d406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:15.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.152+0000 7f39dc4ca640 1 -- 172.21.15.5:0/2283030434 shutdown_connections 2024-05-20T18:37:15.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.152+0000 7f39dc4ca640 1 -- 172.21.15.5:0/2283030434 wait complete. 2024-05-20T18:37:15.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.153+0000 7f39dc4ca640 1 Processor -- start 2024-05-20T18:37:15.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.154+0000 7f39dc4ca640 1 -- start start 2024-05-20T18:37:15.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.155+0000 7f39dc4ca640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f39d40b4010 0x7f39d415ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:15.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.155+0000 7f39dc4ca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39d40d14b0 0x7f39d41592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:15.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.155+0000 7f39dc4ca640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39d4163a10 0x7f39d4154830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:15.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.155+0000 7f39dc4ca640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39d40b8390 con 0x7f39d40d14b0 2024-05-20T18:37:15.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.155+0000 7f39dc4ca640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f39d40b8210 con 0x7f39d4163a10 2024-05-20T18:37:15.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.155+0000 7f39dc4ca640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f39d40b8510 con 0x7f39d40b4010 2024-05-20T18:37:15.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.155+0000 7f39d9a65640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39d40d14b0 0x7f39d41592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:15.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.155+0000 7f39d9a65640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39d40d14b0 0x7f39d41592a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34206/0 (socket says 172.21.15.5:34206) 2024-05-20T18:37:15.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.155+0000 7f39d9a65640 1 -- 172.21.15.5:0/1118227104 learned_addr learned my addr 172.21.15.5:0/1118227104 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:15.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.155+0000 7f39daa67640 1 --2- 172.21.15.5:0/1118227104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39d4163a10 0x7f39d4154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:15.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.155+0000 7f39da266640 1 --2- 172.21.15.5:0/1118227104 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f39d40b4010 0x7f39d415ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:15.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.155+0000 7f39d9a65640 1 -- 172.21.15.5:0/1118227104 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f39d40b4010 msgr2=0x7f39d415ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:15.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.155+0000 7f39d9a65640 1 --2- 172.21.15.5:0/1118227104 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f39d40b4010 0x7f39d415ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:15.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.155+0000 7f39d9a65640 1 -- 172.21.15.5:0/1118227104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39d4163a10 msgr2=0x7f39d4154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:15.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.155+0000 7f39d9a65640 1 --2- 172.21.15.5:0/1118227104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39d4163a10 0x7f39d4154830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:15.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.155+0000 7f39d9a65640 1 -- 172.21.15.5:0/1118227104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d4159c80 con 0x7f39d40d14b0 2024-05-20T18:37:15.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.156+0000 7f39d9a65640 1 --2- 172.21.15.5:0/1118227104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39d40d14b0 0x7f39d41592a0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f39c40097a0 tx=0x7f39c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:15.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.156+0000 7f39caffd640 1 -- 172.21.15.5:0/1118227104 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c4037050 con 0x7f39d40d14b0 2024-05-20T18:37:15.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.156+0000 7f39caffd640 1 -- 172.21.15.5:0/1118227104 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39c4038040 con 0x7f39d40d14b0 2024-05-20T18:37:15.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.156+0000 7f39dc4ca640 1 -- 172.21.15.5:0/1118227104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d4159f60 con 0x7f39d40d14b0 2024-05-20T18:37:15.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.156+0000 7f39caffd640 1 -- 172.21.15.5:0/1118227104 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c4004470 con 0x7f39d40d14b0 2024-05-20T18:37:15.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.156+0000 7f39dc4ca640 1 -- 172.21.15.5:0/1118227104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d415a520 con 0x7f39d40d14b0 2024-05-20T18:37:15.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.157+0000 7f39dc4ca640 1 -- 172.21.15.5:0/1118227104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39d41599f0 con 0x7f39d40d14b0 2024-05-20T18:37:15.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.157+0000 7f39caffd640 1 -- 172.21.15.5:0/1118227104 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f39c403c6d0 con 0x7f39d40d14b0 2024-05-20T18:37:15.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.158+0000 7f39caffd640 1 --2- 172.21.15.5:0/1118227104 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f39b806be30 0x7f39b806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:15.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.158+0000 7f39caffd640 1 -- 172.21.15.5:0/1118227104 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f39c40ab860 con 0x7f39d40d14b0 2024-05-20T18:37:15.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.158+0000 7f39da266640 1 --2- 172.21.15.5:0/1118227104 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f39b806be30 0x7f39b806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:15.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.158+0000 7f39da266640 1 --2- 172.21.15.5:0/1118227104 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f39b806be30 0x7f39b806e340 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f39d0000f60 tx=0x7f39d0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:15.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.163+0000 7f39caffd640 1 -- 172.21.15.5:0/1118227104 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f39c404c020 con 0x7f39d40d14b0 2024-05-20T18:37:15.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.347+0000 7f39dc4ca640 1 -- 172.21.15.5:0/1118227104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f39d4075b00 con 0x7f39d40d14b0 2024-05-20T18:37:15.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.349+0000 7f39caffd640 1 -- 172.21.15.5:0/1118227104 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16845 (secure 0 0 0) 0x7f39c407a110 con 0x7f39d40d14b0 2024-05-20T18:37:15.350 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:15.350 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":118,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:37:14.364852+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:37:12.797471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-20T18:37:13.346156+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:37:15.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.351+0000 7f39dc4ca640 1 -- 172.21.15.5:0/1118227104 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f39b806be30 msgr2=0x7f39b806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:15.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.351+0000 7f39dc4ca640 1 --2- 172.21.15.5:0/1118227104 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f39b806be30 0x7f39b806e340 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f39d0000f60 tx=0x7f39d0009040 comp rx=0 tx=0).stop 2024-05-20T18:37:15.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.351+0000 7f39dc4ca640 1 -- 172.21.15.5:0/1118227104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39d40d14b0 msgr2=0x7f39d41592a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:15.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.351+0000 7f39dc4ca640 1 --2- 172.21.15.5:0/1118227104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39d40d14b0 0x7f39d41592a0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f39c40097a0 tx=0x7f39c40046c0 comp rx=0 tx=0).stop 2024-05-20T18:37:15.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.351+0000 7f39dc4ca640 1 -- 172.21.15.5:0/1118227104 shutdown_connections 2024-05-20T18:37:15.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.351+0000 7f39dc4ca640 1 --2- 172.21.15.5:0/1118227104 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f39b806be30 0x7f39b806e340 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:15.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.351+0000 7f39dc4ca640 1 --2- 172.21.15.5:0/1118227104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f39d4163a10 0x7f39d4154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:15.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.351+0000 7f39dc4ca640 1 --2- 172.21.15.5:0/1118227104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39d40d14b0 0x7f39d41592a0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:15.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.351+0000 7f39dc4ca640 1 --2- 172.21.15.5:0/1118227104 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f39d40b4010 0x7f39d415ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:15.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.351+0000 7f39dc4ca640 1 -- 172.21.15.5:0/1118227104 >> 172.21.15.5:0/1118227104 conn(0x7f39d406d240 msgr2=0x7f39d40b6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:15.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.351+0000 7f39dc4ca640 1 -- 172.21.15.5:0/1118227104 shutdown_connections 2024-05-20T18:37:15.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.351+0000 7f39dc4ca640 1 -- 172.21.15.5:0/1118227104 wait complete. 2024-05-20T18:37:15.364 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:37:15.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.477+0000 7fc3b57c7640 1 Processor -- start 2024-05-20T18:37:15.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.477+0000 7fc3b57c7640 1 -- start start 2024-05-20T18:37:15.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.478+0000 7fc3b57c7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3b006d3d0 0x7fc3b006d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:15.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.478+0000 7fc3b57c7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc3b00b8190 0x7fc3b00b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:15.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.478+0000 7fc3b57c7640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3b006dd20 con 0x7fc3b0162820 2024-05-20T18:37:15.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.478+0000 7fc3b57c7640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc3b00d0630 con 0x7fc3b00b8190 2024-05-20T18:37:15.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.478+0000 7fc3b57c7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc3b00d39c0 con 0x7fc3b006d3d0 2024-05-20T18:37:15.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.478+0000 7fc3ae7fc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3b006d3d0 0x7fc3b006d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:15.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.478+0000 7fc3af7fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc3b00b8190 0x7fc3b00b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:15.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.479+0000 7fc3af7fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc3b00b8190 0x7fc3b00b85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:54676/0 (socket says 172.21.15.5:54676) 2024-05-20T18:37:15.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.479+0000 7fc3ae7fc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3b006d3d0 0x7fc3b006d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:52444/0 (socket says 172.21.15.5:52444) 2024-05-20T18:37:15.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.479+0000 7fc3aeffd640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fc3b0162820 0x7fc3b00b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:55652/0 (socket says 172.21.15.5:55652) 2024-05-20T18:37:15.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.479+0000 7fc3af7fe640 1 -- 172.21.15.5:0/3363363127 learned_addr learned my addr 172.21.15.5:0/3363363127 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:15.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.479+0000 7fc3adffb640 1 -- 172.21.15.5:0/3363363127 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3826691334 0 0) 0x7fc3a0001110 con 0x7fc3b0162820 2024-05-20T18:37:15.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.479+0000 7fc3adffb640 1 -- 172.21.15.5:0/3363363127 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc398003520 con 0x7fc3b0162820 2024-05-20T18:37:15.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.479+0000 7fc3ae7fc640 1 -- 172.21.15.5:0/3363363127 >> v2:172.21.15.63:3300/0 conn(0x7fc3b00b8190 msgr2=0x7fc3b00b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:15.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.479+0000 7fc3ae7fc640 1 --2- 172.21.15.5:0/3363363127 >> v2:172.21.15.63:3300/0 conn(0x7fc3b00b8190 0x7fc3b00b85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:15.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.479+0000 7fc3ae7fc640 1 -- 172.21.15.5:0/3363363127 >> v1:172.21.15.63:6789/0 conn(0x7fc3b0162820 legacy=0x7fc3b00b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:15.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.479+0000 7fc3ae7fc640 1 -- 172.21.15.5:0/3363363127 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b00d3ca0 con 0x7fc3b006d3d0 2024-05-20T18:37:15.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.479+0000 7fc3af7fe640 1 --2- 172.21.15.5:0/3363363127 >> v2:172.21.15.63:3300/0 conn(0x7fc3b00b8190 0x7fc3b00b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:37:15.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.480+0000 7fc3ae7fc640 1 --2- 172.21.15.5:0/3363363127 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3b006d3d0 0x7fc3b006d7e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40097a0 tx=0x7fc3a402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3efef6052d98b202 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:15.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.480+0000 7fc3adffb640 1 -- 172.21.15.5:0/3363363127 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a4037070 con 0x7fc3b006d3d0 2024-05-20T18:37:15.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.480+0000 7fc3adffb640 1 -- 172.21.15.5:0/3363363127 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3a4032440 con 0x7fc3b006d3d0 2024-05-20T18:37:15.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.480+0000 7fc3adffb640 1 -- 172.21.15.5:0/3363363127 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a40327f0 con 0x7fc3b006d3d0 2024-05-20T18:37:15.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.481+0000 7fc3b57c7640 1 -- 172.21.15.5:0/3363363127 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3b006d3d0 msgr2=0x7fc3b006d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:15.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.481+0000 7fc3b57c7640 1 --2- 172.21.15.5:0/3363363127 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3b006d3d0 0x7fc3b006d7e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40097a0 tx=0x7fc3a402a180 comp rx=0 tx=0).stop 2024-05-20T18:37:15.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.481+0000 7fc3b57c7640 1 -- 172.21.15.5:0/3363363127 shutdown_connections 2024-05-20T18:37:15.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.481+0000 7fc3b57c7640 1 --2- 172.21.15.5:0/3363363127 >> v2:172.21.15.63:3300/0 conn(0x7fc3b00b8190 0x7fc3b00b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:15.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.481+0000 7fc3b57c7640 1 --2- 172.21.15.5:0/3363363127 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3b006d3d0 0x7fc3b006d7e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:15.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.481+0000 7fc3b57c7640 1 -- 172.21.15.5:0/3363363127 >> 172.21.15.5:0/3363363127 conn(0x7fc3b00d4bd0 msgr2=0x7fc3b00d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:15.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.481+0000 7fc3b57c7640 1 -- 172.21.15.5:0/3363363127 shutdown_connections 2024-05-20T18:37:15.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.481+0000 7fc3b57c7640 1 -- 172.21.15.5:0/3363363127 wait complete. 2024-05-20T18:37:15.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.483+0000 7fc3b57c7640 1 Processor -- start 2024-05-20T18:37:15.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.483+0000 7fc3b57c7640 1 -- start start 2024-05-20T18:37:15.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.484+0000 7fc3b57c7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3b006d3d0 0x7fc3b015db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:15.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.484+0000 7fc3b57c7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3b00b8190 0x7fc3b00cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:15.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.484+0000 7fc3b57c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3b0162820 0x7fc3b00cd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:15.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.484+0000 7fc3b57c7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b00d0300 con 0x7fc3b0162820 2024-05-20T18:37:15.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.484+0000 7fc3b57c7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b00d4260 con 0x7fc3b00b8190 2024-05-20T18:37:15.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.484+0000 7fc3b57c7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc3b00b4050 con 0x7fc3b006d3d0 2024-05-20T18:37:15.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.484+0000 7fc3af7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3b0162820 0x7fc3b00cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:15.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.484+0000 7fc3af7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3b0162820 0x7fc3b00cd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34222/0 (socket says 172.21.15.5:34222) 2024-05-20T18:37:15.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.484+0000 7fc3af7fe640 1 -- 172.21.15.5:0/2080725882 learned_addr learned my addr 172.21.15.5:0/2080725882 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:15.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.484+0000 7fc3ae7fc640 1 --2- 172.21.15.5:0/2080725882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3b00b8190 0x7fc3b00cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:15.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.484+0000 7fc3aeffd640 1 --2- 172.21.15.5:0/2080725882 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3b006d3d0 0x7fc3b015db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:15.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.485+0000 7fc3af7fe640 1 -- 172.21.15.5:0/2080725882 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3b006d3d0 msgr2=0x7fc3b015db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:15.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.485+0000 7fc3af7fe640 1 --2- 172.21.15.5:0/2080725882 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3b006d3d0 0x7fc3b015db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:15.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.485+0000 7fc3af7fe640 1 -- 172.21.15.5:0/2080725882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3b00b8190 msgr2=0x7fc3b00cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:15.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.485+0000 7fc3af7fe640 1 --2- 172.21.15.5:0/2080725882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3b00b8190 0x7fc3b00cccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:15.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.485+0000 7fc3af7fe640 1 -- 172.21.15.5:0/2080725882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b015bf30 con 0x7fc3b0162820 2024-05-20T18:37:15.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.485+0000 7fc3af7fe640 1 --2- 172.21.15.5:0/2080725882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3b0162820 0x7fc3b00cd210 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8009e50 tx=0x7fc3a800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:15.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.485+0000 7fc3977fe640 1 -- 172.21.15.5:0/2080725882 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a800cd80 con 0x7fc3b0162820 2024-05-20T18:37:15.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.485+0000 7fc3977fe640 1 -- 172.21.15.5:0/2080725882 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3a8019070 con 0x7fc3b0162820 2024-05-20T18:37:15.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.485+0000 7fc3977fe640 1 -- 172.21.15.5:0/2080725882 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a80146c0 con 0x7fc3b0162820 2024-05-20T18:37:15.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.485+0000 7fc3b57c7640 1 -- 172.21.15.5:0/2080725882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b015c210 con 0x7fc3b0162820 2024-05-20T18:37:15.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.485+0000 7fc3b57c7640 1 -- 172.21.15.5:0/2080725882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3b015c7d0 con 0x7fc3b0162820 2024-05-20T18:37:15.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.486+0000 7fc3b57c7640 1 -- 172.21.15.5:0/2080725882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3b015d8b0 con 0x7fc3b0162820 2024-05-20T18:37:15.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.487+0000 7fc3977fe640 1 -- 172.21.15.5:0/2080725882 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc3a8005d10 con 0x7fc3b0162820 2024-05-20T18:37:15.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.487+0000 7fc3977fe640 1 --2- 172.21.15.5:0/2080725882 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc39c06be30 0x7fc39c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:15.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.487+0000 7fc3aeffd640 1 --2- 172.21.15.5:0/2080725882 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc39c06be30 0x7fc39c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:15.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.487+0000 7fc3977fe640 1 -- 172.21.15.5:0/2080725882 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fc3a808b340 con 0x7fc3b0162820 2024-05-20T18:37:15.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.488+0000 7fc3aeffd640 1 --2- 172.21.15.5:0/2080725882 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc39c06be30 0x7fc39c06e340 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fc3a00012b0 tx=0x7fc3a0007b30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:15.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.492+0000 7fc3977fe640 1 -- 172.21.15.5:0/2080725882 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc3b015d8b0 con 0x7fc3b0162820 2024-05-20T18:37:15.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.680+0000 7fc3b57c7640 1 -- 172.21.15.5:0/2080725882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3b0075b00 con 0x7fc3b0162820 2024-05-20T18:37:15.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.681+0000 7fc3977fe640 1 -- 172.21.15.5:0/2080725882 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16845 (secure 0 0 0) 0x7fc3a8010040 con 0x7fc3b0162820 2024-05-20T18:37:15.682 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:15.683 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":118,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:37:14.364852+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:37:12.797471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-20T18:37:13.346156+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:37:15.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.683+0000 7fc3b57c7640 1 -- 172.21.15.5:0/2080725882 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc39c06be30 msgr2=0x7fc39c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:15.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.683+0000 7fc3b57c7640 1 --2- 172.21.15.5:0/2080725882 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc39c06be30 0x7fc39c06e340 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fc3a00012b0 tx=0x7fc3a0007b30 comp rx=0 tx=0).stop 2024-05-20T18:37:15.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.683+0000 7fc3b57c7640 1 -- 172.21.15.5:0/2080725882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3b0162820 msgr2=0x7fc3b00cd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:15.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.683+0000 7fc3b57c7640 1 --2- 172.21.15.5:0/2080725882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3b0162820 0x7fc3b00cd210 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8009e50 tx=0x7fc3a800c490 comp rx=0 tx=0).stop 2024-05-20T18:37:15.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.683+0000 7fc3b57c7640 1 -- 172.21.15.5:0/2080725882 shutdown_connections 2024-05-20T18:37:15.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.683+0000 7fc3b57c7640 1 --2- 172.21.15.5:0/2080725882 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc39c06be30 0x7fc39c06e340 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:15.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.683+0000 7fc3b57c7640 1 --2- 172.21.15.5:0/2080725882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3b0162820 0x7fc3b00cd210 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:15.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.683+0000 7fc3b57c7640 1 --2- 172.21.15.5:0/2080725882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3b00b8190 0x7fc3b00cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:15.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.683+0000 7fc3b57c7640 1 --2- 172.21.15.5:0/2080725882 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3b006d3d0 0x7fc3b015db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:15.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.683+0000 7fc3b57c7640 1 -- 172.21.15.5:0/2080725882 >> 172.21.15.5:0/2080725882 conn(0x7fc3b00d4bd0 msgr2=0x7fc3b00b5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:15.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.683+0000 7fc3b57c7640 1 -- 172.21.15.5:0/2080725882 shutdown_connections 2024-05-20T18:37:15.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.683+0000 7fc3b57c7640 1 -- 172.21.15.5:0/2080725882 wait complete. 2024-05-20T18:37:15.697 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-20T18:37:15.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.813+0000 7f5033079640 1 Processor -- start 2024-05-20T18:37:15.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.813+0000 7f5033079640 1 -- start start 2024-05-20T18:37:15.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.814+0000 7f5033079640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f502c0d1bc0 0x7f502c0d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:15.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.814+0000 7f5033079640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f502c0d0370 con 0x7f502c0b4010 2024-05-20T18:37:15.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.814+0000 7f5033079640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f502c0d0650 con 0x7f502c163a10 2024-05-20T18:37:15.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.814+0000 7f5033079640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f502c0d3ca0 con 0x7f502c0d1bc0 2024-05-20T18:37:15.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.815+0000 7f5031616640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f502c0d1bc0 0x7f502c0d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:15.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.815+0000 7f502bfff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f502c0b4010 0x7f502c0d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:40850/0 (socket says 172.21.15.5:40850) 2024-05-20T18:37:15.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.815+0000 7f502bfff640 1 -- 172.21.15.5:0/3468838395 learned_addr learned my addr 172.21.15.5:0/3468838395 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:15.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.815+0000 7f5031616640 1 -- 172.21.15.5:0/3468838395 >> v1:172.21.15.63:6789/0 conn(0x7f502c163a10 legacy=0x7f502c0cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:15.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.815+0000 7f5031616640 1 -- 172.21.15.5:0/3468838395 >> v1:172.21.15.5:6789/0 conn(0x7f502c0b4010 legacy=0x7f502c0d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:15.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.815+0000 7f5031616640 1 -- 172.21.15.5:0/3468838395 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f502c0d3650 con 0x7f502c0d1bc0 2024-05-20T18:37:15.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.815+0000 7f502b7fe640 1 -- 172.21.15.5:0/3468838395 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 164591199 0 0) 0x7f502c0d0370 con 0x7f502c0b4010 2024-05-20T18:37:15.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.816+0000 7f5031616640 1 --2- 172.21.15.5:0/3468838395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f502c0d1bc0 0x7f502c0d4130 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f5024009d90 tx=0x7f502402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac64e39b75dbbaa server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:15.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.816+0000 7f502b7fe640 1 -- 172.21.15.5:0/3468838395 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024007740 con 0x7f502c0d1bc0 2024-05-20T18:37:15.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.816+0000 7f502b7fe640 1 -- 172.21.15.5:0/3468838395 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50240078e0 con 0x7f502c0d1bc0 2024-05-20T18:37:15.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.816+0000 7f502b7fe640 1 -- 172.21.15.5:0/3468838395 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024007cb0 con 0x7f502c0d1bc0 2024-05-20T18:37:15.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.816+0000 7f5033079640 1 -- 172.21.15.5:0/3468838395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f502c0d1bc0 msgr2=0x7f502c0d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:15.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.816+0000 7f5033079640 1 --2- 172.21.15.5:0/3468838395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f502c0d1bc0 0x7f502c0d4130 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f5024009d90 tx=0x7f502402a6f0 comp rx=0 tx=0).stop 2024-05-20T18:37:15.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.816+0000 7f5033079640 1 -- 172.21.15.5:0/3468838395 shutdown_connections 2024-05-20T18:37:15.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.816+0000 7f5033079640 1 --2- 172.21.15.5:0/3468838395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f502c0d1bc0 0x7f502c0d4130 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:15.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.816+0000 7f5033079640 1 -- 172.21.15.5:0/3468838395 >> 172.21.15.5:0/3468838395 conn(0x7f502c06d240 msgr2=0x7f502c06d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:15.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.817+0000 7f5033079640 1 -- 172.21.15.5:0/3468838395 shutdown_connections 2024-05-20T18:37:15.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.817+0000 7f5033079640 1 -- 172.21.15.5:0/3468838395 wait complete. 2024-05-20T18:37:15.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.818+0000 7f5033079640 1 Processor -- start 2024-05-20T18:37:15.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.818+0000 7f5033079640 1 -- start start 2024-05-20T18:37:15.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.820+0000 7f5033079640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f502c0b4010 0x7f502c156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:15.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.820+0000 7f5033079640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f502c0d1bc0 0x7f502c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:15.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.820+0000 7f5033079640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f502c163a10 0x7f502c0b9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:15.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.820+0000 7f5033079640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f502c0b8390 con 0x7f502c0b4010 2024-05-20T18:37:15.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.820+0000 7f5033079640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f502c0b8210 con 0x7f502c163a10 2024-05-20T18:37:15.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.820+0000 7f5033079640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f502c0b8510 con 0x7f502c0d1bc0 2024-05-20T18:37:15.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.820+0000 7f5030e15640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f502c0b4010 0x7f502c156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:15.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.820+0000 7f5030e15640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f502c0b4010 0x7f502c156850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34230/0 (socket says 172.21.15.5:34230) 2024-05-20T18:37:15.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.820+0000 7f5030e15640 1 -- 172.21.15.5:0/3881017665 learned_addr learned my addr 172.21.15.5:0/3881017665 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:15.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.820+0000 7f5031616640 1 --2- 172.21.15.5:0/3881017665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f502c163a10 0x7f502c0b9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:15.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.820+0000 7f502bfff640 1 --2- 172.21.15.5:0/3881017665 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f502c0d1bc0 0x7f502c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:15.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.820+0000 7f5030e15640 1 -- 172.21.15.5:0/3881017665 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f502c0d1bc0 msgr2=0x7f502c0b8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:15.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.820+0000 7f5030e15640 1 --2- 172.21.15.5:0/3881017665 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f502c0d1bc0 0x7f502c0b8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:15.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.820+0000 7f5030e15640 1 -- 172.21.15.5:0/3881017665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f502c163a10 msgr2=0x7f502c0b9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:15.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.820+0000 7f5030e15640 1 --2- 172.21.15.5:0/3881017665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f502c163a10 0x7f502c0b9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:15.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.820+0000 7f5030e15640 1 -- 172.21.15.5:0/3881017665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f502c157310 con 0x7f502c0b4010 2024-05-20T18:37:15.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.821+0000 7f5030e15640 1 --2- 172.21.15.5:0/3881017665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f502c0b4010 0x7f502c156850 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f502000c3c0 tx=0x7f502000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:15.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.821+0000 7f50297fa640 1 -- 172.21.15.5:0/3881017665 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5020017070 con 0x7f502c0b4010 2024-05-20T18:37:15.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.821+0000 7f50297fa640 1 -- 172.21.15.5:0/3881017665 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50200052d0 con 0x7f502c0b4010 2024-05-20T18:37:15.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.821+0000 7f5033079640 1 -- 172.21.15.5:0/3881017665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f502c1575f0 con 0x7f502c0b4010 2024-05-20T18:37:15.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.821+0000 7f50297fa640 1 -- 172.21.15.5:0/3881017665 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5020003830 con 0x7f502c0b4010 2024-05-20T18:37:15.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.821+0000 7f5033079640 1 -- 172.21.15.5:0/3881017665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f502c1e9130 con 0x7f502c0b4010 2024-05-20T18:37:15.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.822+0000 7f5033079640 1 -- 172.21.15.5:0/3881017665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f502c075b00 con 0x7f502c0b4010 2024-05-20T18:37:15.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.822+0000 7f50297fa640 1 -- 172.21.15.5:0/3881017665 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f502001a070 con 0x7f502c0b4010 2024-05-20T18:37:15.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.823+0000 7f50297fa640 1 --2- 172.21.15.5:0/3881017665 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f501006be30 0x7f501006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:15.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.823+0000 7f50297fa640 1 -- 172.21.15.5:0/3881017665 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f502008a350 con 0x7f502c0b4010 2024-05-20T18:37:15.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.823+0000 7f502bfff640 1 --2- 172.21.15.5:0/3881017665 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f501006be30 0x7f501006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:15.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.823+0000 7f502bfff640 1 --2- 172.21.15.5:0/3881017665 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f501006be30 0x7f501006e340 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f502c147920 tx=0x7f5018007e40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:15.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:15.827+0000 7f50297fa640 1 -- 172.21.15.5:0/3881017665 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5020058490 con 0x7f502c0b4010 2024-05-20T18:37:16.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:16.050+0000 7f5033079640 1 -- 172.21.15.5:0/3881017665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f502c0d3050 con 0x7f502c0b4010 2024-05-20T18:37:16.817 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-05-20T18:37:16.817 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-05-20T18:37:16.817 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-05-20T18:37:16.971 INFO:tasks.ceph.mds.d.smithi063.stderr:2024-05-20T18:37:16.970+0000 7fcea4cf1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T18:37:17.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.016+0000 7f50297fa640 1 -- 172.21.15.5:0/3881017665 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v97) ==== 118+0+0 (secure 0 0 0) 0x7f502005c790 con 0x7f502c0b4010 2024-05-20T18:37:17.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.019+0000 7f5033079640 1 -- 172.21.15.5:0/3881017665 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f501006be30 msgr2=0x7f501006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:17.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.019+0000 7f5033079640 1 --2- 172.21.15.5:0/3881017665 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f501006be30 0x7f501006e340 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f502c147920 tx=0x7f5018007e40 comp rx=0 tx=0).stop 2024-05-20T18:37:17.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.019+0000 7f5033079640 1 -- 172.21.15.5:0/3881017665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f502c0b4010 msgr2=0x7f502c156850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:17.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.019+0000 7f5033079640 1 --2- 172.21.15.5:0/3881017665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f502c0b4010 0x7f502c156850 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f502000c3c0 tx=0x7f502000c880 comp rx=0 tx=0).stop 2024-05-20T18:37:17.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.019+0000 7f5033079640 1 -- 172.21.15.5:0/3881017665 shutdown_connections 2024-05-20T18:37:17.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.019+0000 7f5033079640 1 --2- 172.21.15.5:0/3881017665 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f501006be30 0x7f501006e340 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:17.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.019+0000 7f5033079640 1 --2- 172.21.15.5:0/3881017665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f502c163a10 0x7f502c0b9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:17.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.019+0000 7f5033079640 1 --2- 172.21.15.5:0/3881017665 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f502c0d1bc0 0x7f502c0b8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:17.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.019+0000 7f5033079640 1 --2- 172.21.15.5:0/3881017665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f502c0b4010 0x7f502c156850 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:17.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.019+0000 7f5033079640 1 -- 172.21.15.5:0/3881017665 >> 172.21.15.5:0/3881017665 conn(0x7f502c06d240 msgr2=0x7f502c15f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:17.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.019+0000 7f5033079640 1 -- 172.21.15.5:0/3881017665 shutdown_connections 2024-05-20T18:37:17.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.019+0000 7f5033079640 1 -- 172.21.15.5:0/3881017665 wait complete. 2024-05-20T18:37:17.034 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-20T18:37:17.035 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-20T18:37:17.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.159+0000 7f520f765640 1 Processor -- start 2024-05-20T18:37:17.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.159+0000 7f520f765640 1 -- start start 2024-05-20T18:37:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.160+0000 7f520f765640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f52080d4400 0x7f520815f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.160+0000 7f520f765640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f52080d2780 0x7f52080d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.160+0000 7f520f765640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f520808d2a0 con 0x7f520806cf00 2024-05-20T18:37:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.160+0000 7f520f765640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f52080d3f70 con 0x7f52080d4400 2024-05-20T18:37:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.160+0000 7f520f765640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f52080d30d0 con 0x7f52080d2780 2024-05-20T18:37:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.161+0000 7f520cd00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f520806cf00 0x7f520806d310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:40860/0 (socket says 172.21.15.5:40860) 2024-05-20T18:37:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.161+0000 7f520cd00640 1 -- 172.21.15.5:0/201805574 learned_addr learned my addr 172.21.15.5:0/201805574 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.161+0000 7f51fffff640 1 -- 172.21.15.5:0/201805574 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1344363517 0 0) 0x7f51f8001110 con 0x7f520806cf00 2024-05-20T18:37:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.161+0000 7f520dd02640 1 --2- 172.21.15.5:0/201805574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f52080d2780 0x7f52080d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.161+0000 7f51fffff640 1 -- 172.21.15.5:0/201805574 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51f4003520 con 0x7f520806cf00 2024-05-20T18:37:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.161+0000 7f520d501640 1 --2- 172.21.15.5:0/201805574 >> v2:172.21.15.63:3300/0 conn(0x7f52080d4400 0x7f520815f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.161+0000 7f51fffff640 1 -- 172.21.15.5:0/201805574 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2910846699 0 0) 0x7f51f4003520 con 0x7f520806cf00 2024-05-20T18:37:17.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.161+0000 7f51fffff640 1 -- 172.21.15.5:0/201805574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f52080d2780 msgr2=0x7f52080d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:17.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.161+0000 7f51fffff640 1 --2- 172.21.15.5:0/201805574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f52080d2780 0x7f52080d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:17.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.161+0000 7f51fffff640 1 -- 172.21.15.5:0/201805574 >> v2:172.21.15.63:3300/0 conn(0x7f52080d4400 msgr2=0x7f520815f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:17.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.161+0000 7f51fffff640 1 --2- 172.21.15.5:0/201805574 >> v2:172.21.15.63:3300/0 conn(0x7f52080d4400 0x7f520815f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:17.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.161+0000 7f51fffff640 1 -- 172.21.15.5:0/201805574 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52080d4810 con 0x7f520806cf00 2024-05-20T18:37:17.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.161+0000 7f51fffff640 1 -- 172.21.15.5:0/201805574 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f51f8001840 con 0x7f520806cf00 2024-05-20T18:37:17.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.161+0000 7f51fffff640 1 -- 172.21.15.5:0/201805574 >> v1:172.21.15.5:6789/0 conn(0x7f520806cf00 legacy=0x7f520806d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:17.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.161+0000 7f51fffff640 1 --2- 172.21.15.5:0/201805574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51f40039b0 0x7f51f4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:17.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.161+0000 7f51fffff640 1 --2- 172.21.15.5:0/201805574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f52080d2780 0x7f51f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:17.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.161+0000 7f51fffff640 1 --2- 172.21.15.5:0/201805574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51f4004d40 0x7f51f4005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:17.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.161+0000 7f51fffff640 1 -- 172.21.15.5:0/201805574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f52080d3f70 con 0x7f51f4004d40 2024-05-20T18:37:17.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.161+0000 7f51fffff640 1 -- 172.21.15.5:0/201805574 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f52080d30d0 con 0x7f51f40039b0 2024-05-20T18:37:17.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.161+0000 7f51fffff640 1 -- 172.21.15.5:0/201805574 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f51f400ba40 con 0x7f52080d2780 2024-05-20T18:37:17.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.162+0000 7f520dd02640 1 --2- 172.21.15.5:0/201805574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51f4004d40 0x7f51f4005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:17.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.162+0000 7f520d501640 1 --2- 172.21.15.5:0/201805574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51f40039b0 0x7f51f4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:17.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.162+0000 7f520cd00640 1 --2- 172.21.15.5:0/201805574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f52080d2780 0x7f51f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:17.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.162+0000 7f520dd02640 1 -- 172.21.15.5:0/201805574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f52080d2780 msgr2=0x7f51f4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:17.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.162+0000 7f520dd02640 1 --2- 172.21.15.5:0/201805574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f52080d2780 0x7f51f4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:17.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.162+0000 7f520dd02640 1 -- 172.21.15.5:0/201805574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51f40039b0 msgr2=0x7f51f4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:17.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.162+0000 7f520dd02640 1 --2- 172.21.15.5:0/201805574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51f40039b0 0x7f51f4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:17.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.162+0000 7f520dd02640 1 -- 172.21.15.5:0/201805574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f4003520 con 0x7f51f4004d40 2024-05-20T18:37:17.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.163+0000 7f520dd02640 1 --2- 172.21.15.5:0/201805574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51f4004d40 0x7f51f4005150 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f520808e040 tx=0x7f51f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91718641a286b42a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:17.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.163+0000 7f51fffff640 1 -- 172.21.15.5:0/201805574 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5200009c10 con 0x7f51f4004d40 2024-05-20T18:37:17.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.163+0000 7f51fffff640 1 -- 172.21.15.5:0/201805574 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5200036070 con 0x7f51f4004d40 2024-05-20T18:37:17.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.163+0000 7f51fffff640 1 -- 172.21.15.5:0/201805574 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5200031830 con 0x7f51f4004d40 2024-05-20T18:37:17.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.163+0000 7f520f765640 1 -- 172.21.15.5:0/201805574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51f4004d40 msgr2=0x7f51f4005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:17.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.163+0000 7f520f765640 1 --2- 172.21.15.5:0/201805574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51f4004d40 0x7f51f4005150 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f520808e040 tx=0x7f51f4005830 comp rx=0 tx=0).stop 2024-05-20T18:37:17.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.163+0000 7f520dd02640 1 -- 172.21.15.5:0/201805574 reap_dead start 2024-05-20T18:37:17.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.163+0000 7f520f765640 1 -- 172.21.15.5:0/201805574 shutdown_connections 2024-05-20T18:37:17.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.163+0000 7f520f765640 1 -- 172.21.15.5:0/201805574 >> 172.21.15.5:0/201805574 conn(0x7f52080d4bb0 msgr2=0x7f52080d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:17.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.164+0000 7f520f765640 1 -- 172.21.15.5:0/201805574 shutdown_connections 2024-05-20T18:37:17.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.164+0000 7f520f765640 1 -- 172.21.15.5:0/201805574 wait complete. 2024-05-20T18:37:17.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.165+0000 7f520f765640 1 Processor -- start 2024-05-20T18:37:17.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.165+0000 7f520f765640 1 -- start start 2024-05-20T18:37:17.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.167+0000 7f520f765640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51f4004d40 0x7f52080cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:17.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.167+0000 7f520f765640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5208151110 0x7f520815e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:17.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.167+0000 7f520f765640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5208160f80 0x7f5208161390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:17.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.167+0000 7f520f765640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f52080d3920 con 0x7f5208151110 2024-05-20T18:37:17.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.167+0000 7f520f765640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f52080d3640 con 0x7f51f4004d40 2024-05-20T18:37:17.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.167+0000 7f520f765640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f52080d3c00 con 0x7f5208160f80 2024-05-20T18:37:17.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.167+0000 7f520cd00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5208151110 0x7f520815e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:17.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.167+0000 7f520cd00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5208151110 0x7f520815e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34240/0 (socket says 172.21.15.5:34240) 2024-05-20T18:37:17.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.167+0000 7f520cd00640 1 -- 172.21.15.5:0/2688247899 learned_addr learned my addr 172.21.15.5:0/2688247899 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:17.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.167+0000 7f520d501640 1 --2- 172.21.15.5:0/2688247899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51f4004d40 0x7f52080cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:17.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.167+0000 7f520dd02640 1 --2- 172.21.15.5:0/2688247899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5208160f80 0x7f5208161390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:17.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.167+0000 7f520cd00640 1 -- 172.21.15.5:0/2688247899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5208160f80 msgr2=0x7f5208161390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:17.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.167+0000 7f520cd00640 1 --2- 172.21.15.5:0/2688247899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5208160f80 0x7f5208161390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:17.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.167+0000 7f520cd00640 1 -- 172.21.15.5:0/2688247899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51f4004d40 msgr2=0x7f52080cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:17.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.167+0000 7f520cd00640 1 --2- 172.21.15.5:0/2688247899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51f4004d40 0x7f52080cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:17.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.167+0000 7f520cd00640 1 -- 172.21.15.5:0/2688247899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52081618d0 con 0x7f5208151110 2024-05-20T18:37:17.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.167+0000 7f520cd00640 1 --2- 172.21.15.5:0/2688247899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5208151110 0x7f520815e0f0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f51f80019b0 tx=0x7f51f80077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:17.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.167+0000 7f51fdffb640 1 -- 172.21.15.5:0/2688247899 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f800f9c0 con 0x7f5208151110 2024-05-20T18:37:17.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.168+0000 7f51fdffb640 1 -- 172.21.15.5:0/2688247899 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f8018070 con 0x7f5208151110 2024-05-20T18:37:17.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.168+0000 7f520f765640 1 -- 172.21.15.5:0/2688247899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52080c9fc0 con 0x7f5208151110 2024-05-20T18:37:17.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.168+0000 7f51fdffb640 1 -- 172.21.15.5:0/2688247899 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f800fda0 con 0x7f5208151110 2024-05-20T18:37:17.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.168+0000 7f520f765640 1 -- 172.21.15.5:0/2688247899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52080ca7a0 con 0x7f5208151110 2024-05-20T18:37:17.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.169+0000 7f520f765640 1 -- 172.21.15.5:0/2688247899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52080ca230 con 0x7f5208151110 2024-05-20T18:37:17.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.169+0000 7f51fdffb640 1 -- 172.21.15.5:0/2688247899 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f51f801d460 con 0x7f5208151110 2024-05-20T18:37:17.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.170+0000 7f51fdffb640 1 --2- 172.21.15.5:0/2688247899 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f51ec06be30 0x7f51ec06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:17.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.170+0000 7f51fdffb640 1 -- 172.21.15.5:0/2688247899 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7f51f808b310 con 0x7f5208151110 2024-05-20T18:37:17.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.170+0000 7f520d501640 1 --2- 172.21.15.5:0/2688247899 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f51ec06be30 0x7f51ec06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:17.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.170+0000 7f520d501640 1 --2- 172.21.15.5:0/2688247899 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f51ec06be30 0x7f51ec06e340 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5204009b00 tx=0x7f5204008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:17.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.174+0000 7f51fdffb640 1 -- 172.21.15.5:0/2688247899 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f52081618d0 con 0x7f5208151110 2024-05-20T18:37:17.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:17.396+0000 7f520f765640 1 -- 172.21.15.5:0/2688247899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f52080735a0 con 0x7f5208151110 2024-05-20T18:37:18.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.018+0000 7f51fdffb640 1 -- 172.21.15.5:0/2688247899 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v98) ==== 122+0+0 (secure 0 0 0) 0x7f52080735a0 con 0x7f5208151110 2024-05-20T18:37:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.020+0000 7f520f765640 1 -- 172.21.15.5:0/2688247899 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f51ec06be30 msgr2=0x7f51ec06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.020+0000 7f520f765640 1 --2- 172.21.15.5:0/2688247899 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f51ec06be30 0x7f51ec06e340 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5204009b00 tx=0x7f5204008040 comp rx=0 tx=0).stop 2024-05-20T18:37:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.020+0000 7f520f765640 1 -- 172.21.15.5:0/2688247899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5208151110 msgr2=0x7f520815e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.020+0000 7f520f765640 1 --2- 172.21.15.5:0/2688247899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5208151110 0x7f520815e0f0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f51f80019b0 tx=0x7f51f80077a0 comp rx=0 tx=0).stop 2024-05-20T18:37:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.020+0000 7f520f765640 1 -- 172.21.15.5:0/2688247899 shutdown_connections 2024-05-20T18:37:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.020+0000 7f520f765640 1 --2- 172.21.15.5:0/2688247899 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f51ec06be30 0x7f51ec06e340 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.020+0000 7f520f765640 1 --2- 172.21.15.5:0/2688247899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5208160f80 0x7f5208161390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.020+0000 7f520f765640 1 --2- 172.21.15.5:0/2688247899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5208151110 0x7f520815e0f0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.020+0000 7f520f765640 1 --2- 172.21.15.5:0/2688247899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51f4004d40 0x7f52080cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.020+0000 7f520f765640 1 -- 172.21.15.5:0/2688247899 >> 172.21.15.5:0/2688247899 conn(0x7f52080d4bb0 msgr2=0x7f52080ce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.021+0000 7f520f765640 1 -- 172.21.15.5:0/2688247899 shutdown_connections 2024-05-20T18:37:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.021+0000 7f520f765640 1 -- 172.21.15.5:0/2688247899 wait complete. 2024-05-20T18:37:18.036 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-20T18:37:18.036 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:37:18.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.158+0000 7f43fe08b640 1 Processor -- start 2024-05-20T18:37:18.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.158+0000 7f43fe08b640 1 -- start start 2024-05-20T18:37:18.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.159+0000 7f43fe08b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f43f815d530 0x7f43f815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:18.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.159+0000 7f43fe08b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43f80b1700 0x7f43f80b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:18.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.159+0000 7f43fe08b640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43f80cd580 con 0x7f43f8070850 2024-05-20T18:37:18.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.159+0000 7f43fe08b640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f43f80ce190 con 0x7f43f815d530 2024-05-20T18:37:18.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.159+0000 7f43fe08b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f43f80cd860 con 0x7f43f80b1700 2024-05-20T18:37:18.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.159+0000 7f43f77fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f43f815d530 0x7f43f815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:18.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.159+0000 7f43f77fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f43f815d530 0x7f43f815aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34256/0 (socket says 172.21.15.5:34256) 2024-05-20T18:37:18.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.159+0000 7f43f77fe640 1 -- 172.21.15.5:0/2012475721 learned_addr learned my addr 172.21.15.5:0/2012475721 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:18.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.160+0000 7f43f6ffd640 1 --2- 172.21.15.5:0/2012475721 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43f80b1700 0x7f43f80b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:18.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.160+0000 7f43f77fe640 1 -- 172.21.15.5:0/2012475721 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43f80b1700 msgr2=0x7f43f80b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:18.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.160+0000 7f43f77fe640 1 --2- 172.21.15.5:0/2012475721 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43f80b1700 0x7f43f80b1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:18.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.160+0000 7f43f77fe640 1 -- 172.21.15.5:0/2012475721 >> v1:172.21.15.63:6789/0 conn(0x7f43f8070850 legacy=0x7f43f8070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:18.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.160+0000 7f43f77fe640 1 -- 172.21.15.5:0/2012475721 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f80cdb40 con 0x7f43f815d530 2024-05-20T18:37:18.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.160+0000 7f43f77fe640 1 --2- 172.21.15.5:0/2012475721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43f815d530 0x7f43f815aaa0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f43e8009e50 tx=0x7f43e802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a06e545ed526ac4 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:18.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.161+0000 7f43f67fc640 1 -- 172.21.15.5:0/2012475721 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e8036070 con 0x7f43f815d530 2024-05-20T18:37:18.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.161+0000 7f43f67fc640 1 -- 172.21.15.5:0/2012475721 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43e8031600 con 0x7f43f815d530 2024-05-20T18:37:18.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.161+0000 7f43f67fc640 1 -- 172.21.15.5:0/2012475721 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e80319d0 con 0x7f43f815d530 2024-05-20T18:37:18.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.161+0000 7f43fe08b640 1 -- 172.21.15.5:0/2012475721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43f815d530 msgr2=0x7f43f815aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:18.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.161+0000 7f43fe08b640 1 --2- 172.21.15.5:0/2012475721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43f815d530 0x7f43f815aaa0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f43e8009e50 tx=0x7f43e802a710 comp rx=0 tx=0).stop 2024-05-20T18:37:18.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.161+0000 7f43fe08b640 1 -- 172.21.15.5:0/2012475721 shutdown_connections 2024-05-20T18:37:18.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.161+0000 7f43fe08b640 1 --2- 172.21.15.5:0/2012475721 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43f80b1700 0x7f43f80b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:18.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.161+0000 7f43fe08b640 1 --2- 172.21.15.5:0/2012475721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43f815d530 0x7f43f815aaa0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:18.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.161+0000 7f43fe08b640 1 -- 172.21.15.5:0/2012475721 >> 172.21.15.5:0/2012475721 conn(0x7f43f80d29d0 msgr2=0x7f43f80d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:18.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.161+0000 7f43fe08b640 1 -- 172.21.15.5:0/2012475721 shutdown_connections 2024-05-20T18:37:18.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.161+0000 7f43fe08b640 1 -- 172.21.15.5:0/2012475721 wait complete. 2024-05-20T18:37:18.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.163+0000 7f43fe08b640 1 Processor -- start 2024-05-20T18:37:18.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.163+0000 7f43fe08b640 1 -- start start 2024-05-20T18:37:18.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.165+0000 7f43fe08b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43f8070850 0x7f43f814c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:18.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.165+0000 7f43fe08b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43f80b1700 0x7f43f80ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:18.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.165+0000 7f43fe08b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43f815d530 0x7f43f815baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:18.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.165+0000 7f43fe08b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f43f80b2050 con 0x7f43f80b1700 2024-05-20T18:37:18.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.165+0000 7f43fe08b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f43f80cd230 con 0x7f43f815d530 2024-05-20T18:37:18.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.165+0000 7f43fe08b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f43f8071370 con 0x7f43f8070850 2024-05-20T18:37:18.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.165+0000 7f43f6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43f80b1700 0x7f43f80ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:18.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.165+0000 7f43f6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43f80b1700 0x7f43f80ca490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34270/0 (socket says 172.21.15.5:34270) 2024-05-20T18:37:18.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.165+0000 7f43f6ffd640 1 -- 172.21.15.5:0/519276511 learned_addr learned my addr 172.21.15.5:0/519276511 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:18.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.165+0000 7f43f77fe640 1 --2- 172.21.15.5:0/519276511 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43f8070850 0x7f43f814c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:18.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.165+0000 7f43f7fff640 1 --2- 172.21.15.5:0/519276511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43f815d530 0x7f43f815baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:18.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.165+0000 7f43f6ffd640 1 -- 172.21.15.5:0/519276511 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43f8070850 msgr2=0x7f43f814c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:18.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.165+0000 7f43f6ffd640 1 --2- 172.21.15.5:0/519276511 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43f8070850 0x7f43f814c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:18.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.165+0000 7f43f6ffd640 1 -- 172.21.15.5:0/519276511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43f815d530 msgr2=0x7f43f815baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:18.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.165+0000 7f43f6ffd640 1 --2- 172.21.15.5:0/519276511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43f815d530 0x7f43f815baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:18.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.165+0000 7f43f6ffd640 1 -- 172.21.15.5:0/519276511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f80cace0 con 0x7f43f80b1700 2024-05-20T18:37:18.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.165+0000 7f43f6ffd640 1 --2- 172.21.15.5:0/519276511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43f80b1700 0x7f43f80ca490 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f43ec00d5f0 tx=0x7f43ec00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:18.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.166+0000 7f43dffff640 1 -- 172.21.15.5:0/519276511 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec0125f0 con 0x7f43f80b1700 2024-05-20T18:37:18.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.166+0000 7f43dffff640 1 -- 172.21.15.5:0/519276511 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43ec019070 con 0x7f43f80b1700 2024-05-20T18:37:18.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.166+0000 7f43fe08b640 1 -- 172.21.15.5:0/519276511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f80cafc0 con 0x7f43f80b1700 2024-05-20T18:37:18.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.166+0000 7f43dffff640 1 -- 172.21.15.5:0/519276511 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec0129d0 con 0x7f43f80b1700 2024-05-20T18:37:18.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.166+0000 7f43fe08b640 1 -- 172.21.15.5:0/519276511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f80cb580 con 0x7f43f80b1700 2024-05-20T18:37:18.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.167+0000 7f43fe08b640 1 -- 172.21.15.5:0/519276511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43f8075b00 con 0x7f43f80b1700 2024-05-20T18:37:18.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.167+0000 7f43dffff640 1 -- 172.21.15.5:0/519276511 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f43ec012b70 con 0x7f43f80b1700 2024-05-20T18:37:18.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.168+0000 7f43dffff640 1 --2- 172.21.15.5:0/519276511 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f43e406be30 0x7f43e406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:18.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.168+0000 7f43dffff640 1 -- 172.21.15.5:0/519276511 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7f43ec08b360 con 0x7f43f80b1700 2024-05-20T18:37:18.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.168+0000 7f43f77fe640 1 --2- 172.21.15.5:0/519276511 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f43e406be30 0x7f43e406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:18.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.168+0000 7f43f77fe640 1 --2- 172.21.15.5:0/519276511 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f43e406be30 0x7f43e406e340 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f43e8033000 tx=0x7f43e8009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:18.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.172+0000 7f43dffff640 1 -- 172.21.15.5:0/519276511 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f43ec059b80 con 0x7f43f80b1700 2024-05-20T18:37:18.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.394+0000 7f43fe08b640 1 -- 172.21.15.5:0/519276511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43f80cea50 con 0x7f43f80b1700 2024-05-20T18:37:18.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.394+0000 7f43dffff640 1 -- 172.21.15.5:0/519276511 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+5174 (secure 0 0 0) 0x7f43ec059480 con 0x7f43f80b1700 2024-05-20T18:37:18.396 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:18.396 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":98,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/266630184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":266630184},{"type":"v1","addr":"172.21.15.63:6834","nonce":266630184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/1739221946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1739221946},{"type":"v1","addr":"172.21.15.5:6836","nonce":1739221946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9012,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4273429943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":4273429943},{"type":"v1","addr":"172.21.15.5:6837","nonce":4273429943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:37:14.365195+0000","modified":"2024-05-20T18:37:17.398193+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9030},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9030":{"gid":9030,"name":"d","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/2352520472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":2352520472},{"type":"v1","addr":"172.21.15.63:6835","nonce":2352520472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9030,"qdb_cluster":[9030]},"id":8}]} 2024-05-20T18:37:18.396 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 98 2024-05-20T18:37:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.396+0000 7f43fe08b640 1 -- 172.21.15.5:0/519276511 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f43e406be30 msgr2=0x7f43e406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.396+0000 7f43fe08b640 1 --2- 172.21.15.5:0/519276511 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f43e406be30 0x7f43e406e340 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f43e8033000 tx=0x7f43e8009000 comp rx=0 tx=0).stop 2024-05-20T18:37:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.396+0000 7f43fe08b640 1 -- 172.21.15.5:0/519276511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43f80b1700 msgr2=0x7f43f80ca490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.396+0000 7f43fe08b640 1 --2- 172.21.15.5:0/519276511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43f80b1700 0x7f43f80ca490 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f43ec00d5f0 tx=0x7f43ec00dab0 comp rx=0 tx=0).stop 2024-05-20T18:37:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.396+0000 7f43fe08b640 1 -- 172.21.15.5:0/519276511 shutdown_connections 2024-05-20T18:37:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.396+0000 7f43fe08b640 1 --2- 172.21.15.5:0/519276511 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f43e406be30 0x7f43e406e340 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.396+0000 7f43fe08b640 1 --2- 172.21.15.5:0/519276511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43f815d530 0x7f43f815baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.396+0000 7f43fe08b640 1 --2- 172.21.15.5:0/519276511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43f80b1700 0x7f43f80ca490 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.396+0000 7f43fe08b640 1 --2- 172.21.15.5:0/519276511 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43f8070850 0x7f43f814c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.396+0000 7f43fe08b640 1 -- 172.21.15.5:0/519276511 >> 172.21.15.5:0/519276511 conn(0x7f43f80d29d0 msgr2=0x7f43f815f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.396+0000 7f43fe08b640 1 -- 172.21.15.5:0/519276511 shutdown_connections 2024-05-20T18:37:18.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.396+0000 7f43fe08b640 1 -- 172.21.15.5:0/519276511 wait complete. 2024-05-20T18:37:18.409 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:37:18.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.523+0000 7f1cc41aa640 1 Processor -- start 2024-05-20T18:37:18.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.524+0000 7f1cc41aa640 1 -- start start 2024-05-20T18:37:18.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.525+0000 7f1cc41aa640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1cbc15f220 0x7f1cbc0d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:18.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.525+0000 7f1cc41aa640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cbc0cda20 con 0x7f1cbc0cfae0 2024-05-20T18:37:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.525+0000 7f1cc41aa640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cbc0cdd00 con 0x7f1cbc0d0e00 2024-05-20T18:37:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.525+0000 7f1cc41aa640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f1cbc0cdfe0 con 0x7f1cbc15f220 2024-05-20T18:37:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.525+0000 7f1cc1f46640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1cbc15f220 0x7f1cbc0d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.525+0000 7f1cc2747640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1cbc0cfae0 0x7f1cbc0ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:40866/0 (socket says 172.21.15.5:40866) 2024-05-20T18:37:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.525+0000 7f1cc2747640 1 -- 172.21.15.5:0/1613502877 learned_addr learned my addr 172.21.15.5:0/1613502877 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.526+0000 7f1cc1f46640 1 -- 172.21.15.5:0/1613502877 >> v1:172.21.15.63:6789/0 conn(0x7f1cbc0d0e00 legacy=0x7f1cbc0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.526+0000 7f1cc1f46640 1 -- 172.21.15.5:0/1613502877 >> v1:172.21.15.5:6789/0 conn(0x7f1cbc0cfae0 legacy=0x7f1cbc0ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.526+0000 7f1cc0f44640 1 -- 172.21.15.5:0/1613502877 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3789632878 0 0) 0x7f1cbc0cda20 con 0x7f1cbc0cfae0 2024-05-20T18:37:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.526+0000 7f1cc1f46640 1 -- 172.21.15.5:0/1613502877 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cbc0ce2c0 con 0x7f1cbc15f220 2024-05-20T18:37:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.526+0000 7f1cc1f46640 1 --2- 172.21.15.5:0/1613502877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cbc15f220 0x7f1cbc0d08c0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f1cb80097a0 tx=0x7f1cb802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9af2ee3d1a60c28 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.527+0000 7f1cc0f44640 1 -- 172.21.15.5:0/1613502877 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8037070 con 0x7f1cbc15f220 2024-05-20T18:37:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.527+0000 7f1cc0f44640 1 -- 172.21.15.5:0/1613502877 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cb8032440 con 0x7f1cbc15f220 2024-05-20T18:37:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.527+0000 7f1cc0f44640 1 -- 172.21.15.5:0/1613502877 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8032810 con 0x7f1cbc15f220 2024-05-20T18:37:18.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.527+0000 7f1cc41aa640 1 -- 172.21.15.5:0/1613502877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cbc15f220 msgr2=0x7f1cbc0d08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:18.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.527+0000 7f1cc41aa640 1 --2- 172.21.15.5:0/1613502877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cbc15f220 0x7f1cbc0d08c0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f1cb80097a0 tx=0x7f1cb802a1a0 comp rx=0 tx=0).stop 2024-05-20T18:37:18.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.527+0000 7f1cc41aa640 1 -- 172.21.15.5:0/1613502877 shutdown_connections 2024-05-20T18:37:18.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.527+0000 7f1cc41aa640 1 --2- 172.21.15.5:0/1613502877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cbc15f220 0x7f1cbc0d08c0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:18.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.527+0000 7f1cc41aa640 1 -- 172.21.15.5:0/1613502877 >> 172.21.15.5:0/1613502877 conn(0x7f1cbc0b44d0 msgr2=0x7f1cbc0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:18.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.527+0000 7f1cc41aa640 1 -- 172.21.15.5:0/1613502877 shutdown_connections 2024-05-20T18:37:18.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.527+0000 7f1cc41aa640 1 -- 172.21.15.5:0/1613502877 wait complete. 2024-05-20T18:37:18.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.529+0000 7f1cc41aa640 1 Processor -- start 2024-05-20T18:37:18.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.529+0000 7f1cc41aa640 1 -- start start 2024-05-20T18:37:18.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.531+0000 7f1cc41aa640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cbc0d0e00 0x7f1cbc0cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:18.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.531+0000 7f1cc41aa640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cbc15f220 0x7f1cbc0c9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:18.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.531+0000 7f1cc41aa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cbc0c9e60 0x7f1cbc0ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:18.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.531+0000 7f1cc41aa640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc15f7b0 con 0x7f1cbc0c9e60 2024-05-20T18:37:18.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.531+0000 7f1cc41aa640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc15f630 con 0x7f1cbc0d0e00 2024-05-20T18:37:18.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.531+0000 7f1cc41aa640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1cbc0ceef0 con 0x7f1cbc15f220 2024-05-20T18:37:18.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.531+0000 7f1cc2747640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cbc0c9e60 0x7f1cbc0ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:18.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.531+0000 7f1cc2747640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cbc0c9e60 0x7f1cbc0ca270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34286/0 (socket says 172.21.15.5:34286) 2024-05-20T18:37:18.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.531+0000 7f1cc2747640 1 -- 172.21.15.5:0/4137212132 learned_addr learned my addr 172.21.15.5:0/4137212132 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:18.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.531+0000 7f1cc1f46640 1 --2- 172.21.15.5:0/4137212132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cbc0d0e00 0x7f1cbc0cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:18.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.531+0000 7f1cc1745640 1 --2- 172.21.15.5:0/4137212132 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cbc15f220 0x7f1cbc0c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:18.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.531+0000 7f1cc2747640 1 -- 172.21.15.5:0/4137212132 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cbc15f220 msgr2=0x7f1cbc0c9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:18.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.531+0000 7f1cc2747640 1 --2- 172.21.15.5:0/4137212132 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cbc15f220 0x7f1cbc0c9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:18.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.531+0000 7f1cc2747640 1 -- 172.21.15.5:0/4137212132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cbc0d0e00 msgr2=0x7f1cbc0cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:18.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.531+0000 7f1cc2747640 1 --2- 172.21.15.5:0/4137212132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cbc0d0e00 0x7f1cbc0cb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:18.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.531+0000 7f1cc2747640 1 -- 172.21.15.5:0/4137212132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cbc0cbc90 con 0x7f1cbc0c9e60 2024-05-20T18:37:18.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.532+0000 7f1cc2747640 1 --2- 172.21.15.5:0/4137212132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cbc0c9e60 0x7f1cbc0ca270 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400c730 tx=0x7f1cb400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:18.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.532+0000 7f1cb27fc640 1 -- 172.21.15.5:0/4137212132 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4018070 con 0x7f1cbc0c9e60 2024-05-20T18:37:18.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.532+0000 7f1cb27fc640 1 -- 172.21.15.5:0/4137212132 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cb400ce40 con 0x7f1cbc0c9e60 2024-05-20T18:37:18.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.532+0000 7f1cc41aa640 1 -- 172.21.15.5:0/4137212132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cbc0cbf70 con 0x7f1cbc0c9e60 2024-05-20T18:37:18.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.532+0000 7f1cb27fc640 1 -- 172.21.15.5:0/4137212132 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb400a860 con 0x7f1cbc0c9e60 2024-05-20T18:37:18.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.532+0000 7f1cc41aa640 1 -- 172.21.15.5:0/4137212132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cbc0cc530 con 0x7f1cbc0c9e60 2024-05-20T18:37:18.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.533+0000 7f1cc41aa640 1 -- 172.21.15.5:0/4137212132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cbc075b00 con 0x7f1cbc0c9e60 2024-05-20T18:37:18.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.534+0000 7f1cb27fc640 1 -- 172.21.15.5:0/4137212132 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1cb401b070 con 0x7f1cbc0c9e60 2024-05-20T18:37:18.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.534+0000 7f1cb27fc640 1 --2- 172.21.15.5:0/4137212132 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1ca006be30 0x7f1ca006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:18.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.535+0000 7f1cc1f46640 1 --2- 172.21.15.5:0/4137212132 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1ca006be30 0x7f1ca006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:18.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.535+0000 7f1cb27fc640 1 -- 172.21.15.5:0/4137212132 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7f1cb408a580 con 0x7f1cbc0c9e60 2024-05-20T18:37:18.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.535+0000 7f1cc1f46640 1 --2- 172.21.15.5:0/4137212132 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1ca006be30 0x7f1ca006e340 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8034000 tx=0x7f1cb802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:18.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.539+0000 7f1cb27fc640 1 -- 172.21.15.5:0/4137212132 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1cb40586b0 con 0x7f1cbc0c9e60 2024-05-20T18:37:18.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.722+0000 7f1cc41aa640 1 -- 172.21.15.5:0/4137212132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1cbc0cb9e0 con 0x7f1cbc0c9e60 2024-05-20T18:37:18.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.723+0000 7f1cb27fc640 1 -- 172.21.15.5:0/4137212132 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+16832 (secure 0 0 0) 0x7f1cb405c9b0 con 0x7f1cbc0c9e60 2024-05-20T18:37:18.725 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:18.725 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":119,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:37:15.995209+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:37:12.797471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-20T18:37:13.346156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:37:18.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.725+0000 7f1cc41aa640 1 -- 172.21.15.5:0/4137212132 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1ca006be30 msgr2=0x7f1ca006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:18.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.725+0000 7f1cc41aa640 1 --2- 172.21.15.5:0/4137212132 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1ca006be30 0x7f1ca006e340 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8034000 tx=0x7f1cb802e000 comp rx=0 tx=0).stop 2024-05-20T18:37:18.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.725+0000 7f1cc41aa640 1 -- 172.21.15.5:0/4137212132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cbc0c9e60 msgr2=0x7f1cbc0ca270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:18.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.725+0000 7f1cc41aa640 1 --2- 172.21.15.5:0/4137212132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cbc0c9e60 0x7f1cbc0ca270 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400c730 tx=0x7f1cb400cbf0 comp rx=0 tx=0).stop 2024-05-20T18:37:18.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.725+0000 7f1cc41aa640 1 -- 172.21.15.5:0/4137212132 shutdown_connections 2024-05-20T18:37:18.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.725+0000 7f1cc41aa640 1 --2- 172.21.15.5:0/4137212132 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1ca006be30 0x7f1ca006e340 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:18.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.725+0000 7f1cc41aa640 1 --2- 172.21.15.5:0/4137212132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cbc0c9e60 0x7f1cbc0ca270 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:18.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.725+0000 7f1cc41aa640 1 --2- 172.21.15.5:0/4137212132 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cbc15f220 0x7f1cbc0c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:18.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.725+0000 7f1cc41aa640 1 --2- 172.21.15.5:0/4137212132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cbc0d0e00 0x7f1cbc0cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:18.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.725+0000 7f1cc41aa640 1 -- 172.21.15.5:0/4137212132 >> 172.21.15.5:0/4137212132 conn(0x7f1cbc0b44d0 msgr2=0x7f1cbc0d1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:18.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.725+0000 7f1cc41aa640 1 -- 172.21.15.5:0/4137212132 shutdown_connections 2024-05-20T18:37:18.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.725+0000 7f1cc41aa640 1 -- 172.21.15.5:0/4137212132 wait complete. 2024-05-20T18:37:18.738 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-20T18:37:18.739 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T18:37:18.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.852+0000 7f1cbef85640 1 Processor -- start 2024-05-20T18:37:18.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.853+0000 7f1cbef85640 1 -- start start 2024-05-20T18:37:18.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.854+0000 7f1cbef85640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cb80cd710 0x7f1cb8162e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:18.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.854+0000 7f1cbef85640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cb806c8c0 con 0x7f1cb806d6b0 2024-05-20T18:37:18.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.854+0000 7f1cbef85640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cb806dac0 con 0x7f1cb80cfae0 2024-05-20T18:37:18.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.854+0000 7f1cbef85640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1cb80d0ba0 con 0x7f1cb80cd710 2024-05-20T18:37:18.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.854+0000 7f1cb7fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1cb806d6b0 0x7f1cb80cf3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:40876/0 (socket says 172.21.15.5:40876) 2024-05-20T18:37:18.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.854+0000 7f1cb7fff640 1 -- 172.21.15.5:0/3596201121 learned_addr learned my addr 172.21.15.5:0/3596201121 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:18.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.854+0000 7f1cb77fe640 1 -- 172.21.15.5:0/3596201121 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2737947928 0 0) 0x7f1cb806c8c0 con 0x7f1cb806d6b0 2024-05-20T18:37:18.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.854+0000 7f1cb77fe640 1 -- 172.21.15.5:0/3596201121 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ca0003520 con 0x7f1cb806d6b0 2024-05-20T18:37:18.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.854+0000 7f1cbcd21640 1 --2- 172.21.15.5:0/3596201121 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cb80cd710 0x7f1cb8162e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:18.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.855+0000 7f1cb77fe640 1 -- 172.21.15.5:0/3596201121 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4043214386 0 0) 0x7f1ca0003520 con 0x7f1cb806d6b0 2024-05-20T18:37:18.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.855+0000 7f1cb77fe640 1 -- 172.21.15.5:0/3596201121 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cb80cd710 msgr2=0x7f1cb8162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:18.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.855+0000 7f1cb77fe640 1 --2- 172.21.15.5:0/3596201121 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cb80cd710 0x7f1cb8162e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:18.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.855+0000 7f1cb77fe640 1 -- 172.21.15.5:0/3596201121 >> v1:172.21.15.63:6789/0 conn(0x7f1cb80cfae0 legacy=0x7f1cb80d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:18.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.855+0000 7f1cb77fe640 1 -- 172.21.15.5:0/3596201121 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb81633e0 con 0x7f1cb806d6b0 2024-05-20T18:37:18.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.855+0000 7f1cb77fe640 1 -- 172.21.15.5:0/3596201121 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f1cac001670 con 0x7f1cb806d6b0 2024-05-20T18:37:18.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.855+0000 7f1cb77fe640 1 -- 172.21.15.5:0/3596201121 >> v1:172.21.15.5:6789/0 conn(0x7f1cb806d6b0 legacy=0x7f1cb80cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:18.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.855+0000 7f1cb77fe640 1 --2- 172.21.15.5:0/3596201121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ca00039e0 0x7f1ca0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:18.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.855+0000 7f1cb77fe640 1 --2- 172.21.15.5:0/3596201121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ca0004800 0x7f1ca0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:18.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.855+0000 7f1cb77fe640 1 --2- 172.21.15.5:0/3596201121 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cb80cd710 0x7f1ca0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:18.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.855+0000 7f1cb77fe640 1 -- 172.21.15.5:0/3596201121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb80d0ba0 con 0x7f1ca00039e0 2024-05-20T18:37:18.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.855+0000 7f1cb77fe640 1 -- 172.21.15.5:0/3596201121 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca000baf0 con 0x7f1ca0004800 2024-05-20T18:37:18.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.855+0000 7f1cb77fe640 1 -- 172.21.15.5:0/3596201121 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1ca000bcf0 con 0x7f1cb80cd710 2024-05-20T18:37:18.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.855+0000 7f1cbcd21640 1 --2- 172.21.15.5:0/3596201121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ca00039e0 0x7f1ca0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:18.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.855+0000 7f1cb7fff640 1 --2- 172.21.15.5:0/3596201121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ca0004800 0x7f1ca0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:18.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.855+0000 7f1cbd522640 1 --2- 172.21.15.5:0/3596201121 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cb80cd710 0x7f1ca0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:18.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.855+0000 7f1cbcd21640 1 -- 172.21.15.5:0/3596201121 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cb80cd710 msgr2=0x7f1ca0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:18.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.855+0000 7f1cbcd21640 1 --2- 172.21.15.5:0/3596201121 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cb80cd710 0x7f1ca0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:18.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.855+0000 7f1cbcd21640 1 -- 172.21.15.5:0/3596201121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ca0004800 msgr2=0x7f1ca0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:18.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.855+0000 7f1cbcd21640 1 --2- 172.21.15.5:0/3596201121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ca0004800 0x7f1ca0004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:18.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.855+0000 7f1cbcd21640 1 -- 172.21.15.5:0/3596201121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca0003520 con 0x7f1ca00039e0 2024-05-20T18:37:18.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.855+0000 7f1cb7fff640 1 --2- 172.21.15.5:0/3596201121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ca0004800 0x7f1ca0004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:18.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.856+0000 7f1cbcd21640 1 --2- 172.21.15.5:0/3596201121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ca00039e0 0x7f1ca0003e50 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8162d60 tx=0x7f1ca0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb6b41f41018de server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:18.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.856+0000 7f1cb77fe640 1 -- 172.21.15.5:0/3596201121 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8036070 con 0x7f1ca00039e0 2024-05-20T18:37:18.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.856+0000 7f1cb77fe640 1 -- 172.21.15.5:0/3596201121 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ca8031400 con 0x7f1ca00039e0 2024-05-20T18:37:18.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.856+0000 7f1cb77fe640 1 -- 172.21.15.5:0/3596201121 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8009690 con 0x7f1ca00039e0 2024-05-20T18:37:18.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.857+0000 7f1cbef85640 1 -- 172.21.15.5:0/3596201121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ca00039e0 msgr2=0x7f1ca0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:18.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.857+0000 7f1cbef85640 1 --2- 172.21.15.5:0/3596201121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ca00039e0 0x7f1ca0003e50 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8162d60 tx=0x7f1ca0005830 comp rx=0 tx=0).stop 2024-05-20T18:37:18.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.857+0000 7f1cbd522640 1 -- 172.21.15.5:0/3596201121 reap_dead start 2024-05-20T18:37:18.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.857+0000 7f1cbef85640 1 -- 172.21.15.5:0/3596201121 shutdown_connections 2024-05-20T18:37:18.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.857+0000 7f1cbef85640 1 -- 172.21.15.5:0/3596201121 >> 172.21.15.5:0/3596201121 conn(0x7f1cb80b44d0 msgr2=0x7f1cb80b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:18.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.857+0000 7f1cbef85640 1 -- 172.21.15.5:0/3596201121 shutdown_connections 2024-05-20T18:37:18.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.857+0000 7f1cbef85640 1 -- 172.21.15.5:0/3596201121 wait complete. 2024-05-20T18:37:18.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.859+0000 7f1cbef85640 1 Processor -- start 2024-05-20T18:37:18.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.859+0000 7f1cbef85640 1 -- start start 2024-05-20T18:37:18.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.860+0000 7f1cbef85640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ca00039e0 0x7f1cb8155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:18.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.860+0000 7f1cbef85640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cb815c2a0 0x7f1cb81611f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:18.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.860+0000 7f1cbef85640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cb8161730 0x7f1cb80cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:18.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.860+0000 7f1cbef85640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb8163c80 con 0x7f1cb815c2a0 2024-05-20T18:37:18.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.860+0000 7f1cbef85640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb8163760 con 0x7f1cb8161730 2024-05-20T18:37:18.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.860+0000 7f1cbef85640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1cb8162e00 con 0x7f1ca00039e0 2024-05-20T18:37:18.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.860+0000 7f1cb7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cb815c2a0 0x7f1cb81611f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:18.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.860+0000 7f1cb7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cb815c2a0 0x7f1cb81611f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34314/0 (socket says 172.21.15.5:34314) 2024-05-20T18:37:18.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.860+0000 7f1cb7fff640 1 -- 172.21.15.5:0/3578785999 learned_addr learned my addr 172.21.15.5:0/3578785999 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:18.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.860+0000 7f1cbd522640 1 --2- 172.21.15.5:0/3578785999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cb8161730 0x7f1cb80cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:18.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.860+0000 7f1cbcd21640 1 --2- 172.21.15.5:0/3578785999 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ca00039e0 0x7f1cb8155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:18.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.861+0000 7f1cb7fff640 1 -- 172.21.15.5:0/3578785999 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ca00039e0 msgr2=0x7f1cb8155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:18.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.861+0000 7f1cb7fff640 1 --2- 172.21.15.5:0/3578785999 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ca00039e0 0x7f1cb8155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:18.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.861+0000 7f1cb7fff640 1 -- 172.21.15.5:0/3578785999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cb8161730 msgr2=0x7f1cb80cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:18.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.861+0000 7f1cb7fff640 1 --2- 172.21.15.5:0/3578785999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cb8161730 0x7f1cb80cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:18.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.861+0000 7f1cb7fff640 1 -- 172.21.15.5:0/3578785999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb80ca090 con 0x7f1cb815c2a0 2024-05-20T18:37:18.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.861+0000 7f1cbd522640 1 --2- 172.21.15.5:0/3578785999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cb8161730 0x7f1cb80cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:18.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.861+0000 7f1cb7fff640 1 --2- 172.21.15.5:0/3578785999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cb815c2a0 0x7f1cb81611f0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00e790 tx=0x7f1cac00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:18.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.861+0000 7f1cb57fa640 1 -- 172.21.15.5:0/3578785999 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac00eea0 con 0x7f1cb815c2a0 2024-05-20T18:37:18.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.861+0000 7f1cb57fa640 1 -- 172.21.15.5:0/3578785999 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cac018070 con 0x7f1cb815c2a0 2024-05-20T18:37:18.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.861+0000 7f1cb57fa640 1 -- 172.21.15.5:0/3578785999 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac00cad0 con 0x7f1cb815c2a0 2024-05-20T18:37:18.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.861+0000 7f1cbef85640 1 -- 172.21.15.5:0/3578785999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb80ca330 con 0x7f1cb815c2a0 2024-05-20T18:37:18.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.861+0000 7f1cbef85640 1 -- 172.21.15.5:0/3578785999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb80ca8f0 con 0x7f1cb815c2a0 2024-05-20T18:37:18.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.862+0000 7f1cbef85640 1 -- 172.21.15.5:0/3578785999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cb8075b00 con 0x7f1cb815c2a0 2024-05-20T18:37:18.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.863+0000 7f1cb57fa640 1 -- 172.21.15.5:0/3578785999 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1cac00cc70 con 0x7f1cb815c2a0 2024-05-20T18:37:18.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.863+0000 7f1cb57fa640 1 --2- 172.21.15.5:0/3578785999 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1ca406be30 0x7f1ca406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:18.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.863+0000 7f1cb57fa640 1 -- 172.21.15.5:0/3578785999 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7f1cac08af50 con 0x7f1cb815c2a0 2024-05-20T18:37:18.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.863+0000 7f1cbcd21640 1 --2- 172.21.15.5:0/3578785999 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1ca406be30 0x7f1ca406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:18.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.864+0000 7f1cbcd21640 1 --2- 172.21.15.5:0/3578785999 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1ca406be30 0x7f1ca406e340 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8033000 tx=0x7f1ca8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:18.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:18.868+0000 7f1cb57fa640 1 -- 172.21.15.5:0/3578785999 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1cac0596e0 con 0x7f1cb815c2a0 2024-05-20T18:37:19.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.048+0000 7f1cbef85640 1 -- 172.21.15.5:0/3578785999 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1cb80b8940 con 0x7f1ca406be30 2024-05-20T18:37:19.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.054+0000 7f1cb57fa640 1 -- 172.21.15.5:0/3578785999 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252487 (secure 0 0 0) 0x7f1cb80b8940 con 0x7f1ca406be30 2024-05-20T18:37:19.059 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:19.061 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-20T18:37:19.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.058+0000 7f1cbef85640 1 -- 172.21.15.5:0/3578785999 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1ca406be30 msgr2=0x7f1ca406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:19.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.058+0000 7f1cbef85640 1 --2- 172.21.15.5:0/3578785999 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1ca406be30 0x7f1ca406e340 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8033000 tx=0x7f1ca8008030 comp rx=0 tx=0).stop 2024-05-20T18:37:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.058+0000 7f1cbef85640 1 -- 172.21.15.5:0/3578785999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cb815c2a0 msgr2=0x7f1cb81611f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.058+0000 7f1cbef85640 1 --2- 172.21.15.5:0/3578785999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cb815c2a0 0x7f1cb81611f0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00e790 tx=0x7f1cac00ec50 comp rx=0 tx=0).stop 2024-05-20T18:37:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.058+0000 7f1cbef85640 1 -- 172.21.15.5:0/3578785999 shutdown_connections 2024-05-20T18:37:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.058+0000 7f1cbef85640 1 --2- 172.21.15.5:0/3578785999 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1ca406be30 0x7f1ca406e340 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.058+0000 7f1cbef85640 1 --2- 172.21.15.5:0/3578785999 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cb8161730 0x7f1cb80cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.058+0000 7f1cbef85640 1 --2- 172.21.15.5:0/3578785999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cb815c2a0 0x7f1cb81611f0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.058+0000 7f1cbef85640 1 --2- 172.21.15.5:0/3578785999 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ca00039e0 0x7f1cb8155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.058+0000 7f1cbef85640 1 -- 172.21.15.5:0/3578785999 >> 172.21.15.5:0/3578785999 conn(0x7f1cb80b44d0 msgr2=0x7f1cb80b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.058+0000 7f1cbef85640 1 -- 172.21.15.5:0/3578785999 shutdown_connections 2024-05-20T18:37:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.058+0000 7f1cbef85640 1 -- 172.21.15.5:0/3578785999 wait complete. 2024-05-20T18:37:19.071 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":421,"stamp":"2024-05-20T18:37:18.732606+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":156,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262208,"kb_used_data":12724,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712512,"statfs":{"total":773094113280,"available":772825612288,"internally_reserved":0,"allocated":13029376,"data_stored":6674088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.759937"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.003962+0000","last_change":"2024-05-20T18:37:15.011514+0000","last_active":"2024-05-20T18:37:16.003962+0000","last_peered":"2024-05-20T18:37:16.003962+0000","last_clean":"2024-05-20T18:37:16.003962+0000","last_became_active":"2024-05-20T18:37:15.009781+0000","last_became_peered":"2024-05-20T18:37:15.009781+0000","last_unstale":"2024-05-20T18:37:16.003962+0000","last_undegraded":"2024-05-20T18:37:16.003962+0000","last_fullsized":"2024-05-20T18:37:16.003962+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:45:47.732140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180783+0000","last_change":"2024-05-20T18:37:15.010720+0000","last_active":"2024-05-20T18:37:16.180783+0000","last_peered":"2024-05-20T18:37:16.180783+0000","last_clean":"2024-05-20T18:37:16.180783+0000","last_became_active":"2024-05-20T18:37:15.008847+0000","last_became_peered":"2024-05-20T18:37:15.008847+0000","last_unstale":"2024-05-20T18:37:16.180783+0000","last_undegraded":"2024-05-20T18:37:16.180783+0000","last_fullsized":"2024-05-20T18:37:16.180783+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:12:01.989254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181073+0000","last_change":"2024-05-20T18:37:15.006116+0000","last_active":"2024-05-20T18:37:16.181073+0000","last_peered":"2024-05-20T18:37:16.181073+0000","last_clean":"2024-05-20T18:37:16.181073+0000","last_became_active":"2024-05-20T18:37:15.005765+0000","last_became_peered":"2024-05-20T18:37:15.005765+0000","last_unstale":"2024-05-20T18:37:16.181073+0000","last_undegraded":"2024-05-20T18:37:16.181073+0000","last_fullsized":"2024-05-20T18:37:16.181073+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:24:39.770496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181995+0000","last_change":"2024-05-20T18:37:15.019066+0000","last_active":"2024-05-20T18:37:16.181995+0000","last_peered":"2024-05-20T18:37:16.181995+0000","last_clean":"2024-05-20T18:37:16.181995+0000","last_became_active":"2024-05-20T18:37:15.018671+0000","last_became_peered":"2024-05-20T18:37:15.018671+0000","last_unstale":"2024-05-20T18:37:16.181995+0000","last_undegraded":"2024-05-20T18:37:16.181995+0000","last_fullsized":"2024-05-20T18:37:16.181995+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:31:21.035991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.003058+0000","last_change":"2024-05-20T18:37:15.011675+0000","last_active":"2024-05-20T18:37:16.003058+0000","last_peered":"2024-05-20T18:37:16.003058+0000","last_clean":"2024-05-20T18:37:16.003058+0000","last_became_active":"2024-05-20T18:37:15.011253+0000","last_became_peered":"2024-05-20T18:37:15.011253+0000","last_unstale":"2024-05-20T18:37:16.003058+0000","last_undegraded":"2024-05-20T18:37:16.003058+0000","last_fullsized":"2024-05-20T18:37:16.003058+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:13.723823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.005843+0000","last_change":"2024-05-20T18:37:15.019993+0000","last_active":"2024-05-20T18:37:16.005843+0000","last_peered":"2024-05-20T18:37:16.005843+0000","last_clean":"2024-05-20T18:37:16.005843+0000","last_became_active":"2024-05-20T18:37:15.019594+0000","last_became_peered":"2024-05-20T18:37:15.019594+0000","last_unstale":"2024-05-20T18:37:16.005843+0000","last_undegraded":"2024-05-20T18:37:16.005843+0000","last_fullsized":"2024-05-20T18:37:16.005843+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:24:35.377207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.005741+0000","last_change":"2024-05-20T18:37:15.010292+0000","last_active":"2024-05-20T18:37:16.005741+0000","last_peered":"2024-05-20T18:37:16.005741+0000","last_clean":"2024-05-20T18:37:16.005741+0000","last_became_active":"2024-05-20T18:37:15.009918+0000","last_became_peered":"2024-05-20T18:37:15.009918+0000","last_unstale":"2024-05-20T18:37:16.005741+0000","last_undegraded":"2024-05-20T18:37:16.005741+0000","last_fullsized":"2024-05-20T18:37:16.005741+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:50:46.214512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181084+0000","last_change":"2024-05-20T18:37:15.006577+0000","last_active":"2024-05-20T18:37:16.181084+0000","last_peered":"2024-05-20T18:37:16.181084+0000","last_clean":"2024-05-20T18:37:16.181084+0000","last_became_active":"2024-05-20T18:37:15.006256+0000","last_became_peered":"2024-05-20T18:37:15.006256+0000","last_unstale":"2024-05-20T18:37:16.181084+0000","last_undegraded":"2024-05-20T18:37:16.181084+0000","last_fullsized":"2024-05-20T18:37:16.181084+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:29:45.355146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181963+0000","last_change":"2024-05-20T18:37:15.016997+0000","last_active":"2024-05-20T18:37:16.181963+0000","last_peered":"2024-05-20T18:37:16.181963+0000","last_clean":"2024-05-20T18:37:16.181963+0000","last_became_active":"2024-05-20T18:37:15.015933+0000","last_became_peered":"2024-05-20T18:37:15.015933+0000","last_unstale":"2024-05-20T18:37:16.181963+0000","last_undegraded":"2024-05-20T18:37:16.181963+0000","last_fullsized":"2024-05-20T18:37:16.181963+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:38:47.009102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180844+0000","last_change":"2024-05-20T18:37:15.001456+0000","last_active":"2024-05-20T18:37:16.180844+0000","last_peered":"2024-05-20T18:37:16.180844+0000","last_clean":"2024-05-20T18:37:16.180844+0000","last_became_active":"2024-05-20T18:37:15.001024+0000","last_became_peered":"2024-05-20T18:37:15.001024+0000","last_unstale":"2024-05-20T18:37:16.180844+0000","last_undegraded":"2024-05-20T18:37:16.180844+0000","last_fullsized":"2024-05-20T18:37:16.180844+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:14.967187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.003632+0000","last_change":"2024-05-20T18:37:15.014038+0000","last_active":"2024-05-20T18:37:16.003632+0000","last_peered":"2024-05-20T18:37:16.003632+0000","last_clean":"2024-05-20T18:37:16.003632+0000","last_became_active":"2024-05-20T18:37:15.013678+0000","last_became_peered":"2024-05-20T18:37:15.013678+0000","last_unstale":"2024-05-20T18:37:16.003632+0000","last_undegraded":"2024-05-20T18:37:16.003632+0000","last_fullsized":"2024-05-20T18:37:16.003632+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:42:54.909774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.006022+0000","last_change":"2024-05-20T18:37:15.013567+0000","last_active":"2024-05-20T18:37:16.006022+0000","last_peered":"2024-05-20T18:37:16.006022+0000","last_clean":"2024-05-20T18:37:16.006022+0000","last_became_active":"2024-05-20T18:37:15.013252+0000","last_became_peered":"2024-05-20T18:37:15.013252+0000","last_unstale":"2024-05-20T18:37:16.006022+0000","last_undegraded":"2024-05-20T18:37:16.006022+0000","last_fullsized":"2024-05-20T18:37:16.006022+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:15:29.541315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180952+0000","last_change":"2024-05-20T18:37:14.999101+0000","last_active":"2024-05-20T18:37:16.180952+0000","last_peered":"2024-05-20T18:37:16.180952+0000","last_clean":"2024-05-20T18:37:16.180952+0000","last_became_active":"2024-05-20T18:37:14.998786+0000","last_became_peered":"2024-05-20T18:37:14.998786+0000","last_unstale":"2024-05-20T18:37:16.180952+0000","last_undegraded":"2024-05-20T18:37:16.180952+0000","last_fullsized":"2024-05-20T18:37:16.180952+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:33:35.377520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181457+0000","last_change":"2024-05-20T18:37:15.010296+0000","last_active":"2024-05-20T18:37:16.181457+0000","last_peered":"2024-05-20T18:37:16.181457+0000","last_clean":"2024-05-20T18:37:16.181457+0000","last_became_active":"2024-05-20T18:37:15.009968+0000","last_became_peered":"2024-05-20T18:37:15.009968+0000","last_unstale":"2024-05-20T18:37:16.181457+0000","last_undegraded":"2024-05-20T18:37:16.181457+0000","last_fullsized":"2024-05-20T18:37:16.181457+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:35:17.702668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.002166+0000","last_change":"2024-05-20T18:37:15.016621+0000","last_active":"2024-05-20T18:37:16.002166+0000","last_peered":"2024-05-20T18:37:16.002166+0000","last_clean":"2024-05-20T18:37:16.002166+0000","last_became_active":"2024-05-20T18:37:15.016304+0000","last_became_peered":"2024-05-20T18:37:15.016304+0000","last_unstale":"2024-05-20T18:37:16.002166+0000","last_undegraded":"2024-05-20T18:37:16.002166+0000","last_fullsized":"2024-05-20T18:37:16.002166+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:14.967187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.005525+0000","last_change":"2024-05-20T18:37:14.999598+0000","last_active":"2024-05-20T18:37:16.005525+0000","last_peered":"2024-05-20T18:37:16.005525+0000","last_clean":"2024-05-20T18:37:16.005525+0000","last_became_active":"2024-05-20T18:37:14.999289+0000","last_became_peered":"2024-05-20T18:37:14.999289+0000","last_unstale":"2024-05-20T18:37:16.005525+0000","last_undegraded":"2024-05-20T18:37:16.005525+0000","last_fullsized":"2024-05-20T18:37:16.005525+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:24:31.957377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180774+0000","last_change":"2024-05-20T18:37:15.000116+0000","last_active":"2024-05-20T18:37:16.180774+0000","last_peered":"2024-05-20T18:37:16.180774+0000","last_clean":"2024-05-20T18:37:16.180774+0000","last_became_active":"2024-05-20T18:37:14.999791+0000","last_became_peered":"2024-05-20T18:37:14.999791+0000","last_unstale":"2024-05-20T18:37:16.180774+0000","last_undegraded":"2024-05-20T18:37:16.180774+0000","last_fullsized":"2024-05-20T18:37:16.180774+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:23:46.139283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181135+0000","last_change":"2024-05-20T18:37:15.008179+0000","last_active":"2024-05-20T18:37:16.181135+0000","last_peered":"2024-05-20T18:37:16.181135+0000","last_clean":"2024-05-20T18:37:16.181135+0000","last_became_active":"2024-05-20T18:37:15.007807+0000","last_became_peered":"2024-05-20T18:37:15.007807+0000","last_unstale":"2024-05-20T18:37:16.181135+0000","last_undegraded":"2024-05-20T18:37:16.181135+0000","last_fullsized":"2024-05-20T18:37:16.181135+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:17:09.591443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.002836+0000","last_change":"2024-05-20T18:37:15.013161+0000","last_active":"2024-05-20T18:37:16.002836+0000","last_peered":"2024-05-20T18:37:16.002836+0000","last_clean":"2024-05-20T18:37:16.002836+0000","last_became_active":"2024-05-20T18:37:15.012529+0000","last_became_peered":"2024-05-20T18:37:15.012529+0000","last_unstale":"2024-05-20T18:37:16.002836+0000","last_undegraded":"2024-05-20T18:37:16.002836+0000","last_fullsized":"2024-05-20T18:37:16.002836+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:14.967187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:15.998413+0000","last_change":"2024-05-20T18:37:14.998616+0000","last_active":"2024-05-20T18:37:15.998413+0000","last_peered":"2024-05-20T18:37:15.998413+0000","last_clean":"2024-05-20T18:37:15.998413+0000","last_became_active":"2024-05-20T18:37:14.998273+0000","last_became_peered":"2024-05-20T18:37:14.998273+0000","last_unstale":"2024-05-20T18:37:15.998413+0000","last_undegraded":"2024-05-20T18:37:15.998413+0000","last_fullsized":"2024-05-20T18:37:15.998413+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:22:23.482722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.182059+0000","last_change":"2024-05-20T18:37:15.008413+0000","last_active":"2024-05-20T18:37:16.182059+0000","last_peered":"2024-05-20T18:37:16.182059+0000","last_clean":"2024-05-20T18:37:16.182059+0000","last_became_active":"2024-05-20T18:37:15.007971+0000","last_became_peered":"2024-05-20T18:37:15.007971+0000","last_unstale":"2024-05-20T18:37:16.182059+0000","last_undegraded":"2024-05-20T18:37:16.182059+0000","last_fullsized":"2024-05-20T18:37:16.182059+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:17:17.517079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.002267+0000","last_change":"2024-05-20T18:37:15.019953+0000","last_active":"2024-05-20T18:37:16.002267+0000","last_peered":"2024-05-20T18:37:16.002267+0000","last_clean":"2024-05-20T18:37:16.002267+0000","last_became_active":"2024-05-20T18:37:15.019578+0000","last_became_peered":"2024-05-20T18:37:15.019578+0000","last_unstale":"2024-05-20T18:37:16.002267+0000","last_undegraded":"2024-05-20T18:37:16.002267+0000","last_fullsized":"2024-05-20T18:37:16.002267+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:33:35.377520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181278+0000","last_change":"2024-05-20T18:37:15.012654+0000","last_active":"2024-05-20T18:37:16.181278+0000","last_peered":"2024-05-20T18:37:16.181278+0000","last_clean":"2024-05-20T18:37:16.181278+0000","last_became_active":"2024-05-20T18:37:15.012307+0000","last_became_peered":"2024-05-20T18:37:15.012307+0000","last_unstale":"2024-05-20T18:37:16.181278+0000","last_undegraded":"2024-05-20T18:37:16.181278+0000","last_fullsized":"2024-05-20T18:37:16.181278+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:14:17.389325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180661+0000","last_change":"2024-05-20T18:37:15.001101+0000","last_active":"2024-05-20T18:37:16.180661+0000","last_peered":"2024-05-20T18:37:16.180661+0000","last_clean":"2024-05-20T18:37:16.180661+0000","last_became_active":"2024-05-20T18:37:15.000773+0000","last_became_peered":"2024-05-20T18:37:15.000773+0000","last_unstale":"2024-05-20T18:37:16.180661+0000","last_undegraded":"2024-05-20T18:37:16.180661+0000","last_fullsized":"2024-05-20T18:37:16.180661+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:33:35.377520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181239+0000","last_change":"2024-05-20T18:37:15.013502+0000","last_active":"2024-05-20T18:37:16.181239+0000","last_peered":"2024-05-20T18:37:16.181239+0000","last_clean":"2024-05-20T18:37:16.181239+0000","last_became_active":"2024-05-20T18:37:15.013156+0000","last_became_peered":"2024-05-20T18:37:15.013156+0000","last_unstale":"2024-05-20T18:37:16.181239+0000","last_undegraded":"2024-05-20T18:37:16.181239+0000","last_fullsized":"2024-05-20T18:37:16.181239+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:45:47.732140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:15.998484+0000","last_change":"2024-05-20T18:37:14.999693+0000","last_active":"2024-05-20T18:37:15.998484+0000","last_peered":"2024-05-20T18:37:15.998484+0000","last_clean":"2024-05-20T18:37:15.998484+0000","last_became_active":"2024-05-20T18:37:14.999253+0000","last_became_peered":"2024-05-20T18:37:14.999253+0000","last_unstale":"2024-05-20T18:37:15.998484+0000","last_undegraded":"2024-05-20T18:37:15.998484+0000","last_fullsized":"2024-05-20T18:37:15.998484+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:20:50.905337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.003082+0000","last_change":"2024-05-20T18:37:15.012911+0000","last_active":"2024-05-20T18:37:16.003082+0000","last_peered":"2024-05-20T18:37:16.003082+0000","last_clean":"2024-05-20T18:37:16.003082+0000","last_became_active":"2024-05-20T18:37:15.012063+0000","last_became_peered":"2024-05-20T18:37:15.012063+0000","last_unstale":"2024-05-20T18:37:16.003082+0000","last_undegraded":"2024-05-20T18:37:16.003082+0000","last_fullsized":"2024-05-20T18:37:16.003082+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:33:35.377520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.006121+0000","last_change":"2024-05-20T18:37:15.006949+0000","last_active":"2024-05-20T18:37:16.006121+0000","last_peered":"2024-05-20T18:37:16.006121+0000","last_clean":"2024-05-20T18:37:16.006121+0000","last_became_active":"2024-05-20T18:37:15.006476+0000","last_became_peered":"2024-05-20T18:37:15.006476+0000","last_unstale":"2024-05-20T18:37:16.006121+0000","last_undegraded":"2024-05-20T18:37:16.006121+0000","last_fullsized":"2024-05-20T18:37:16.006121+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:12:01.175574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:15.998524+0000","last_change":"2024-05-20T18:37:15.016442+0000","last_active":"2024-05-20T18:37:15.998524+0000","last_peered":"2024-05-20T18:37:15.998524+0000","last_clean":"2024-05-20T18:37:15.998524+0000","last_became_active":"2024-05-20T18:37:15.016120+0000","last_became_peered":"2024-05-20T18:37:15.016120+0000","last_unstale":"2024-05-20T18:37:15.998524+0000","last_undegraded":"2024-05-20T18:37:15.998524+0000","last_fullsized":"2024-05-20T18:37:15.998524+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:19:39.642333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181428+0000","last_change":"2024-05-20T18:37:15.013302+0000","last_active":"2024-05-20T18:37:16.181428+0000","last_peered":"2024-05-20T18:37:16.181428+0000","last_clean":"2024-05-20T18:37:16.181428+0000","last_became_active":"2024-05-20T18:37:15.012938+0000","last_became_peered":"2024-05-20T18:37:15.012938+0000","last_unstale":"2024-05-20T18:37:16.181428+0000","last_undegraded":"2024-05-20T18:37:16.181428+0000","last_fullsized":"2024-05-20T18:37:16.181428+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:12:31.191269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180761+0000","last_change":"2024-05-20T18:37:14.999304+0000","last_active":"2024-05-20T18:37:16.180761+0000","last_peered":"2024-05-20T18:37:16.180761+0000","last_clean":"2024-05-20T18:37:16.180761+0000","last_became_active":"2024-05-20T18:37:14.998974+0000","last_became_peered":"2024-05-20T18:37:14.998974+0000","last_unstale":"2024-05-20T18:37:16.180761+0000","last_undegraded":"2024-05-20T18:37:16.180761+0000","last_fullsized":"2024-05-20T18:37:16.180761+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:47:53.793796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180816+0000","last_change":"2024-05-20T18:37:14.993720+0000","last_active":"2024-05-20T18:37:16.180816+0000","last_peered":"2024-05-20T18:37:16.180816+0000","last_clean":"2024-05-20T18:37:16.180816+0000","last_became_active":"2024-05-20T18:37:14.993330+0000","last_became_peered":"2024-05-20T18:37:14.993330+0000","last_unstale":"2024-05-20T18:37:16.180816+0000","last_undegraded":"2024-05-20T18:37:16.180816+0000","last_fullsized":"2024-05-20T18:37:16.180816+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:13.723823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180964+0000","last_change":"2024-05-20T18:37:14.998070+0000","last_active":"2024-05-20T18:37:16.180964+0000","last_peered":"2024-05-20T18:37:16.180964+0000","last_clean":"2024-05-20T18:37:16.180964+0000","last_became_active":"2024-05-20T18:37:14.997718+0000","last_became_peered":"2024-05-20T18:37:14.997718+0000","last_unstale":"2024-05-20T18:37:16.180964+0000","last_undegraded":"2024-05-20T18:37:16.180964+0000","last_fullsized":"2024-05-20T18:37:16.180964+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:47:53.793796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"118'4","reported_seq":18,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.002625+0000","last_change":"2024-05-20T18:37:13.988387+0000","last_active":"2024-05-20T18:37:16.002625+0000","last_peered":"2024-05-20T18:37:16.002625+0000","last_clean":"2024-05-20T18:37:16.002625+0000","last_became_active":"2024-05-20T18:37:13.988076+0000","last_became_peered":"2024-05-20T18:37:13.988076+0000","last_unstale":"2024-05-20T18:37:16.002625+0000","last_undegraded":"2024-05-20T18:37:16.002625+0000","last_fullsized":"2024-05-20T18:37:16.002625+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:01:04.468095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.005688+0000","last_change":"2024-05-20T18:37:15.011785+0000","last_active":"2024-05-20T18:37:16.005688+0000","last_peered":"2024-05-20T18:37:16.005688+0000","last_clean":"2024-05-20T18:37:16.005688+0000","last_became_active":"2024-05-20T18:37:15.011476+0000","last_became_peered":"2024-05-20T18:37:15.011476+0000","last_unstale":"2024-05-20T18:37:16.005688+0000","last_undegraded":"2024-05-20T18:37:16.005688+0000","last_fullsized":"2024-05-20T18:37:16.005688+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:33:02.222185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"118'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181029+0000","last_change":"2024-05-20T18:37:13.991010+0000","last_active":"2024-05-20T18:37:16.181029+0000","last_peered":"2024-05-20T18:37:16.181029+0000","last_clean":"2024-05-20T18:37:16.181029+0000","last_became_active":"2024-05-20T18:37:13.990714+0000","last_became_peered":"2024-05-20T18:37:13.990714+0000","last_unstale":"2024-05-20T18:37:16.181029+0000","last_undegraded":"2024-05-20T18:37:16.181029+0000","last_fullsized":"2024-05-20T18:37:16.181029+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:51:59.490183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181086+0000","last_change":"2024-05-20T18:37:15.005081+0000","last_active":"2024-05-20T18:37:16.181086+0000","last_peered":"2024-05-20T18:37:16.181086+0000","last_clean":"2024-05-20T18:37:16.181086+0000","last_became_active":"2024-05-20T18:37:15.004756+0000","last_became_peered":"2024-05-20T18:37:15.004756+0000","last_unstale":"2024-05-20T18:37:16.181086+0000","last_undegraded":"2024-05-20T18:37:16.181086+0000","last_fullsized":"2024-05-20T18:37:16.181086+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:58:06.602987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181976+0000","last_change":"2024-05-20T18:37:13.995516+0000","last_active":"2024-05-20T18:37:16.181976+0000","last_peered":"2024-05-20T18:37:16.181976+0000","last_clean":"2024-05-20T18:37:16.181976+0000","last_became_active":"2024-05-20T18:37:13.995175+0000","last_became_peered":"2024-05-20T18:37:13.995175+0000","last_unstale":"2024-05-20T18:37:16.181976+0000","last_undegraded":"2024-05-20T18:37:16.181976+0000","last_fullsized":"2024-05-20T18:37:16.181976+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:33:34.372799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.182073+0000","last_change":"2024-05-20T18:37:15.009642+0000","last_active":"2024-05-20T18:37:16.182073+0000","last_peered":"2024-05-20T18:37:16.182073+0000","last_clean":"2024-05-20T18:37:16.182073+0000","last_became_active":"2024-05-20T18:37:15.008987+0000","last_became_peered":"2024-05-20T18:37:15.008987+0000","last_unstale":"2024-05-20T18:37:16.182073+0000","last_undegraded":"2024-05-20T18:37:16.182073+0000","last_fullsized":"2024-05-20T18:37:16.182073+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:50:45.389018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181099+0000","last_change":"2024-05-20T18:37:13.991400+0000","last_active":"2024-05-20T18:37:16.181099+0000","last_peered":"2024-05-20T18:37:16.181099+0000","last_clean":"2024-05-20T18:37:16.181099+0000","last_became_active":"2024-05-20T18:37:13.990372+0000","last_became_peered":"2024-05-20T18:37:13.990372+0000","last_unstale":"2024-05-20T18:37:16.181099+0000","last_undegraded":"2024-05-20T18:37:16.181099+0000","last_fullsized":"2024-05-20T18:37:16.181099+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:01:42.587185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180709+0000","last_change":"2024-05-20T18:37:14.995034+0000","last_active":"2024-05-20T18:37:16.180709+0000","last_peered":"2024-05-20T18:37:16.180709+0000","last_clean":"2024-05-20T18:37:16.180709+0000","last_became_active":"2024-05-20T18:37:14.994707+0000","last_became_peered":"2024-05-20T18:37:14.994707+0000","last_unstale":"2024-05-20T18:37:16.180709+0000","last_undegraded":"2024-05-20T18:37:16.180709+0000","last_fullsized":"2024-05-20T18:37:16.180709+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:39:22.103644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"118'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.002790+0000","last_change":"2024-05-20T18:37:13.983744+0000","last_active":"2024-05-20T18:37:16.002790+0000","last_peered":"2024-05-20T18:37:16.002790+0000","last_clean":"2024-05-20T18:37:16.002790+0000","last_became_active":"2024-05-20T18:37:13.983395+0000","last_became_peered":"2024-05-20T18:37:13.983395+0000","last_unstale":"2024-05-20T18:37:16.002790+0000","last_undegraded":"2024-05-20T18:37:16.002790+0000","last_fullsized":"2024-05-20T18:37:16.002790+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:11:51.568152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.002364+0000","last_change":"2024-05-20T18:37:15.013186+0000","last_active":"2024-05-20T18:37:16.002364+0000","last_peered":"2024-05-20T18:37:16.002364+0000","last_clean":"2024-05-20T18:37:16.002364+0000","last_became_active":"2024-05-20T18:37:15.012877+0000","last_became_peered":"2024-05-20T18:37:15.012877+0000","last_unstale":"2024-05-20T18:37:16.002364+0000","last_undegraded":"2024-05-20T18:37:16.002364+0000","last_fullsized":"2024-05-20T18:37:16.002364+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:47:53.793796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"118'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.005687+0000","last_change":"2024-05-20T18:37:14.000511+0000","last_active":"2024-05-20T18:37:16.005687+0000","last_peered":"2024-05-20T18:37:16.005687+0000","last_clean":"2024-05-20T18:37:16.005687+0000","last_became_active":"2024-05-20T18:37:14.000188+0000","last_became_peered":"2024-05-20T18:37:14.000188+0000","last_unstale":"2024-05-20T18:37:16.005687+0000","last_undegraded":"2024-05-20T18:37:16.005687+0000","last_fullsized":"2024-05-20T18:37:16.005687+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:14:03.223520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181526+0000","last_change":"2024-05-20T18:37:15.010572+0000","last_active":"2024-05-20T18:37:16.181526+0000","last_peered":"2024-05-20T18:37:16.181526+0000","last_clean":"2024-05-20T18:37:16.181526+0000","last_became_active":"2024-05-20T18:37:15.009946+0000","last_became_peered":"2024-05-20T18:37:15.009946+0000","last_unstale":"2024-05-20T18:37:16.181526+0000","last_undegraded":"2024-05-20T18:37:16.181526+0000","last_fullsized":"2024-05-20T18:37:16.181526+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:00:48.335547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181385+0000","last_change":"2024-05-20T18:37:13.990980+0000","last_active":"2024-05-20T18:37:16.181385+0000","last_peered":"2024-05-20T18:37:16.181385+0000","last_clean":"2024-05-20T18:37:16.181385+0000","last_became_active":"2024-05-20T18:37:13.990675+0000","last_became_peered":"2024-05-20T18:37:13.990675+0000","last_unstale":"2024-05-20T18:37:16.181385+0000","last_undegraded":"2024-05-20T18:37:16.181385+0000","last_fullsized":"2024-05-20T18:37:16.181385+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:49:11.347324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180650+0000","last_change":"2024-05-20T18:37:15.010905+0000","last_active":"2024-05-20T18:37:16.180650+0000","last_peered":"2024-05-20T18:37:16.180650+0000","last_clean":"2024-05-20T18:37:16.180650+0000","last_became_active":"2024-05-20T18:37:15.010597+0000","last_became_peered":"2024-05-20T18:37:15.010597+0000","last_unstale":"2024-05-20T18:37:16.180650+0000","last_undegraded":"2024-05-20T18:37:16.180650+0000","last_fullsized":"2024-05-20T18:37:16.180650+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:43:16.199181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.005708+0000","last_change":"2024-05-20T18:37:14.000613+0000","last_active":"2024-05-20T18:37:16.005708+0000","last_peered":"2024-05-20T18:37:16.005708+0000","last_clean":"2024-05-20T18:37:16.005708+0000","last_became_active":"2024-05-20T18:37:13.999533+0000","last_became_peered":"2024-05-20T18:37:13.999533+0000","last_unstale":"2024-05-20T18:37:16.005708+0000","last_undegraded":"2024-05-20T18:37:16.005708+0000","last_fullsized":"2024-05-20T18:37:16.005708+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:29:23.761809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.002682+0000","last_change":"2024-05-20T18:37:15.001552+0000","last_active":"2024-05-20T18:37:16.002682+0000","last_peered":"2024-05-20T18:37:16.002682+0000","last_clean":"2024-05-20T18:37:16.002682+0000","last_became_active":"2024-05-20T18:37:15.001237+0000","last_became_peered":"2024-05-20T18:37:15.001237+0000","last_unstale":"2024-05-20T18:37:16.002682+0000","last_undegraded":"2024-05-20T18:37:16.002682+0000","last_fullsized":"2024-05-20T18:37:16.002682+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:59:53.017125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.002916+0000","last_change":"2024-05-20T18:37:14.000726+0000","last_active":"2024-05-20T18:37:16.002916+0000","last_peered":"2024-05-20T18:37:16.002916+0000","last_clean":"2024-05-20T18:37:16.002916+0000","last_became_active":"2024-05-20T18:37:14.000459+0000","last_became_peered":"2024-05-20T18:37:14.000459+0000","last_unstale":"2024-05-20T18:37:16.002916+0000","last_undegraded":"2024-05-20T18:37:16.002916+0000","last_fullsized":"2024-05-20T18:37:16.002916+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:11:51.568152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181393+0000","last_change":"2024-05-20T18:37:15.015983+0000","last_active":"2024-05-20T18:37:16.181393+0000","last_peered":"2024-05-20T18:37:16.181393+0000","last_clean":"2024-05-20T18:37:16.181393+0000","last_became_active":"2024-05-20T18:37:15.014074+0000","last_became_peered":"2024-05-20T18:37:15.014074+0000","last_unstale":"2024-05-20T18:37:16.181393+0000","last_undegraded":"2024-05-20T18:37:16.181393+0000","last_fullsized":"2024-05-20T18:37:16.181393+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:33:06.615494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180894+0000","last_change":"2024-05-20T18:37:13.991931+0000","last_active":"2024-05-20T18:37:16.180894+0000","last_peered":"2024-05-20T18:37:16.180894+0000","last_clean":"2024-05-20T18:37:16.180894+0000","last_became_active":"2024-05-20T18:37:13.991611+0000","last_became_peered":"2024-05-20T18:37:13.991611+0000","last_unstale":"2024-05-20T18:37:16.180894+0000","last_undegraded":"2024-05-20T18:37:16.180894+0000","last_fullsized":"2024-05-20T18:37:16.180894+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:01:37.695206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181174+0000","last_change":"2024-05-20T18:37:15.006954+0000","last_active":"2024-05-20T18:37:16.181174+0000","last_peered":"2024-05-20T18:37:16.181174+0000","last_clean":"2024-05-20T18:37:16.181174+0000","last_became_active":"2024-05-20T18:37:15.006639+0000","last_became_peered":"2024-05-20T18:37:15.006639+0000","last_unstale":"2024-05-20T18:37:16.181174+0000","last_undegraded":"2024-05-20T18:37:16.181174+0000","last_fullsized":"2024-05-20T18:37:16.181174+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:35:27.774093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"118'3","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181484+0000","last_change":"2024-05-20T18:37:13.991796+0000","last_active":"2024-05-20T18:37:16.181484+0000","last_peered":"2024-05-20T18:37:16.181484+0000","last_clean":"2024-05-20T18:37:16.181484+0000","last_became_active":"2024-05-20T18:37:13.991445+0000","last_became_peered":"2024-05-20T18:37:13.991445+0000","last_unstale":"2024-05-20T18:37:16.181484+0000","last_undegraded":"2024-05-20T18:37:16.181484+0000","last_fullsized":"2024-05-20T18:37:16.181484+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:14:00.046447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181420+0000","last_change":"2024-05-20T18:37:15.000159+0000","last_active":"2024-05-20T18:37:16.181420+0000","last_peered":"2024-05-20T18:37:16.181420+0000","last_clean":"2024-05-20T18:37:16.181420+0000","last_became_active":"2024-05-20T18:37:14.999842+0000","last_became_peered":"2024-05-20T18:37:14.999842+0000","last_unstale":"2024-05-20T18:37:16.181420+0000","last_undegraded":"2024-05-20T18:37:16.181420+0000","last_fullsized":"2024-05-20T18:37:16.181420+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:52:24.191074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"118'6","reported_seq":20,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181289+0000","last_change":"2024-05-20T18:37:13.990269+0000","last_active":"2024-05-20T18:37:16.181289+0000","last_peered":"2024-05-20T18:37:16.181289+0000","last_clean":"2024-05-20T18:37:16.181289+0000","last_became_active":"2024-05-20T18:37:13.989960+0000","last_became_peered":"2024-05-20T18:37:13.989960+0000","last_unstale":"2024-05-20T18:37:16.181289+0000","last_undegraded":"2024-05-20T18:37:16.181289+0000","last_fullsized":"2024-05-20T18:37:16.181289+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:14:35.884964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.002403+0000","last_change":"2024-05-20T18:37:15.004032+0000","last_active":"2024-05-20T18:37:16.002403+0000","last_peered":"2024-05-20T18:37:16.002403+0000","last_clean":"2024-05-20T18:37:16.002403+0000","last_became_active":"2024-05-20T18:37:15.003654+0000","last_became_peered":"2024-05-20T18:37:15.003654+0000","last_unstale":"2024-05-20T18:37:16.002403+0000","last_undegraded":"2024-05-20T18:37:16.002403+0000","last_fullsized":"2024-05-20T18:37:16.002403+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:23:46.139283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180943+0000","last_change":"2024-05-20T18:37:13.989571+0000","last_active":"2024-05-20T18:37:16.180943+0000","last_peered":"2024-05-20T18:37:16.180943+0000","last_clean":"2024-05-20T18:37:16.180943+0000","last_became_active":"2024-05-20T18:37:13.989147+0000","last_became_peered":"2024-05-20T18:37:13.989147+0000","last_unstale":"2024-05-20T18:37:16.180943+0000","last_undegraded":"2024-05-20T18:37:16.180943+0000","last_fullsized":"2024-05-20T18:37:16.180943+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:34:18.346275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180600+0000","last_change":"2024-05-20T18:37:15.004403+0000","last_active":"2024-05-20T18:37:16.180600+0000","last_peered":"2024-05-20T18:37:16.180600+0000","last_clean":"2024-05-20T18:37:16.180600+0000","last_became_active":"2024-05-20T18:37:15.002347+0000","last_became_peered":"2024-05-20T18:37:15.002347+0000","last_unstale":"2024-05-20T18:37:16.180600+0000","last_undegraded":"2024-05-20T18:37:16.180600+0000","last_fullsized":"2024-05-20T18:37:16.180600+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:45:50.978401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181607+0000","last_change":"2024-05-20T18:37:13.998175+0000","last_active":"2024-05-20T18:37:16.181607+0000","last_peered":"2024-05-20T18:37:16.181607+0000","last_clean":"2024-05-20T18:37:16.181607+0000","last_became_active":"2024-05-20T18:37:13.997805+0000","last_became_peered":"2024-05-20T18:37:13.997805+0000","last_unstale":"2024-05-20T18:37:16.181607+0000","last_undegraded":"2024-05-20T18:37:16.181607+0000","last_fullsized":"2024-05-20T18:37:16.181607+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:31:34.605988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:15.998261+0000","last_change":"2024-05-20T18:37:15.014162+0000","last_active":"2024-05-20T18:37:15.998261+0000","last_peered":"2024-05-20T18:37:15.998261+0000","last_clean":"2024-05-20T18:37:15.998261+0000","last_became_active":"2024-05-20T18:37:15.013796+0000","last_became_peered":"2024-05-20T18:37:15.013796+0000","last_unstale":"2024-05-20T18:37:15.998261+0000","last_undegraded":"2024-05-20T18:37:15.998261+0000","last_fullsized":"2024-05-20T18:37:15.998261+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:33:04.932773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181484+0000","last_change":"2024-05-20T18:37:13.995413+0000","last_active":"2024-05-20T18:37:16.181484+0000","last_peered":"2024-05-20T18:37:16.181484+0000","last_clean":"2024-05-20T18:37:16.181484+0000","last_became_active":"2024-05-20T18:37:13.995054+0000","last_became_peered":"2024-05-20T18:37:13.995054+0000","last_unstale":"2024-05-20T18:37:16.181484+0000","last_undegraded":"2024-05-20T18:37:16.181484+0000","last_fullsized":"2024-05-20T18:37:16.181484+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:50:44.384297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181584+0000","last_change":"2024-05-20T18:37:15.004187+0000","last_active":"2024-05-20T18:37:16.181584+0000","last_peered":"2024-05-20T18:37:16.181584+0000","last_clean":"2024-05-20T18:37:16.181584+0000","last_became_active":"2024-05-20T18:37:15.003783+0000","last_became_peered":"2024-05-20T18:37:15.003783+0000","last_unstale":"2024-05-20T18:37:16.181584+0000","last_undegraded":"2024-05-20T18:37:16.181584+0000","last_fullsized":"2024-05-20T18:37:16.181584+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:42:21.574338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"118'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180765+0000","last_change":"2024-05-20T18:37:14.001188+0000","last_active":"2024-05-20T18:37:16.180765+0000","last_peered":"2024-05-20T18:37:16.180765+0000","last_clean":"2024-05-20T18:37:16.180765+0000","last_became_active":"2024-05-20T18:37:14.000895+0000","last_became_peered":"2024-05-20T18:37:14.000895+0000","last_unstale":"2024-05-20T18:37:16.180765+0000","last_undegraded":"2024-05-20T18:37:16.180765+0000","last_fullsized":"2024-05-20T18:37:16.180765+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:24:51.857629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.003758+0000","last_change":"2024-05-20T18:37:15.019659+0000","last_active":"2024-05-20T18:37:16.003758+0000","last_peered":"2024-05-20T18:37:16.003758+0000","last_clean":"2024-05-20T18:37:16.003758+0000","last_became_active":"2024-05-20T18:37:15.019319+0000","last_became_peered":"2024-05-20T18:37:15.019319+0000","last_unstale":"2024-05-20T18:37:16.003758+0000","last_undegraded":"2024-05-20T18:37:16.003758+0000","last_fullsized":"2024-05-20T18:37:16.003758+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:58:06.602987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:15.998605+0000","last_change":"2024-05-20T18:37:14.002718+0000","last_active":"2024-05-20T18:37:15.998605+0000","last_peered":"2024-05-20T18:37:15.998605+0000","last_clean":"2024-05-20T18:37:15.998605+0000","last_became_active":"2024-05-20T18:37:14.002338+0000","last_became_peered":"2024-05-20T18:37:14.002338+0000","last_unstale":"2024-05-20T18:37:15.998605+0000","last_undegraded":"2024-05-20T18:37:15.998605+0000","last_fullsized":"2024-05-20T18:37:15.998605+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:24:41.727547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.006111+0000","last_change":"2024-05-20T18:37:15.007582+0000","last_active":"2024-05-20T18:37:16.006111+0000","last_peered":"2024-05-20T18:37:16.006111+0000","last_clean":"2024-05-20T18:37:16.006111+0000","last_became_active":"2024-05-20T18:37:15.007175+0000","last_became_peered":"2024-05-20T18:37:15.007175+0000","last_unstale":"2024-05-20T18:37:16.006111+0000","last_undegraded":"2024-05-20T18:37:16.006111+0000","last_fullsized":"2024-05-20T18:37:16.006111+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:34.231637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"118'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.182076+0000","last_change":"2024-05-20T18:37:13.995870+0000","last_active":"2024-05-20T18:37:16.182076+0000","last_peered":"2024-05-20T18:37:16.182076+0000","last_clean":"2024-05-20T18:37:16.182076+0000","last_became_active":"2024-05-20T18:37:13.995474+0000","last_became_peered":"2024-05-20T18:37:13.995474+0000","last_unstale":"2024-05-20T18:37:16.182076+0000","last_undegraded":"2024-05-20T18:37:16.182076+0000","last_fullsized":"2024-05-20T18:37:16.182076+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:47:52.789075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181630+0000","last_change":"2024-05-20T18:37:15.003118+0000","last_active":"2024-05-20T18:37:16.181630+0000","last_peered":"2024-05-20T18:37:16.181630+0000","last_clean":"2024-05-20T18:37:16.181630+0000","last_became_active":"2024-05-20T18:37:15.002565+0000","last_became_peered":"2024-05-20T18:37:15.002565+0000","last_unstale":"2024-05-20T18:37:16.181630+0000","last_undegraded":"2024-05-20T18:37:16.181630+0000","last_fullsized":"2024-05-20T18:37:16.181630+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:48:01.975705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181775+0000","last_change":"2024-05-20T18:37:14.003282+0000","last_active":"2024-05-20T18:37:16.181775+0000","last_peered":"2024-05-20T18:37:16.181775+0000","last_clean":"2024-05-20T18:37:16.181775+0000","last_became_active":"2024-05-20T18:37:14.002922+0000","last_became_peered":"2024-05-20T18:37:14.002922+0000","last_unstale":"2024-05-20T18:37:16.181775+0000","last_undegraded":"2024-05-20T18:37:16.181775+0000","last_fullsized":"2024-05-20T18:37:16.181775+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:13.962466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.005510+0000","last_change":"2024-05-20T18:37:15.005929+0000","last_active":"2024-05-20T18:37:16.005510+0000","last_peered":"2024-05-20T18:37:16.005510+0000","last_clean":"2024-05-20T18:37:16.005510+0000","last_became_active":"2024-05-20T18:37:15.005567+0000","last_became_peered":"2024-05-20T18:37:15.005567+0000","last_unstale":"2024-05-20T18:37:16.005510+0000","last_undegraded":"2024-05-20T18:37:16.005510+0000","last_fullsized":"2024-05-20T18:37:16.005510+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:21:39.843969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.005620+0000","last_change":"2024-05-20T18:37:13.998501+0000","last_active":"2024-05-20T18:37:16.005620+0000","last_peered":"2024-05-20T18:37:16.005620+0000","last_clean":"2024-05-20T18:37:16.005620+0000","last_became_active":"2024-05-20T18:37:13.998181+0000","last_became_peered":"2024-05-20T18:37:13.998181+0000","last_unstale":"2024-05-20T18:37:16.005620+0000","last_undegraded":"2024-05-20T18:37:16.005620+0000","last_fullsized":"2024-05-20T18:37:16.005620+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:40:50.875776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181353+0000","last_change":"2024-05-20T18:37:15.005993+0000","last_active":"2024-05-20T18:37:16.181353+0000","last_peered":"2024-05-20T18:37:16.181353+0000","last_clean":"2024-05-20T18:37:16.181353+0000","last_became_active":"2024-05-20T18:37:15.005664+0000","last_became_peered":"2024-05-20T18:37:15.005664+0000","last_unstale":"2024-05-20T18:37:16.181353+0000","last_undegraded":"2024-05-20T18:37:16.181353+0000","last_fullsized":"2024-05-20T18:37:16.181353+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:55:08.267509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181231+0000","last_change":"2024-05-20T18:37:13.995250+0000","last_active":"2024-05-20T18:37:16.181231+0000","last_peered":"2024-05-20T18:37:16.181231+0000","last_clean":"2024-05-20T18:37:16.181231+0000","last_became_active":"2024-05-20T18:37:13.994781+0000","last_became_peered":"2024-05-20T18:37:13.994781+0000","last_unstale":"2024-05-20T18:37:16.181231+0000","last_undegraded":"2024-05-20T18:37:16.181231+0000","last_fullsized":"2024-05-20T18:37:16.181231+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:12.719102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181617+0000","last_change":"2024-05-20T18:37:15.004979+0000","last_active":"2024-05-20T18:37:16.181617+0000","last_peered":"2024-05-20T18:37:16.181617+0000","last_clean":"2024-05-20T18:37:16.181617+0000","last_became_active":"2024-05-20T18:37:15.004648+0000","last_became_peered":"2024-05-20T18:37:15.004648+0000","last_unstale":"2024-05-20T18:37:16.181617+0000","last_undegraded":"2024-05-20T18:37:16.181617+0000","last_fullsized":"2024-05-20T18:37:16.181617+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:14:04.228241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181527+0000","last_change":"2024-05-20T18:37:14.003922+0000","last_active":"2024-05-20T18:37:16.181527+0000","last_peered":"2024-05-20T18:37:16.181527+0000","last_clean":"2024-05-20T18:37:16.181527+0000","last_became_active":"2024-05-20T18:37:14.003596+0000","last_became_peered":"2024-05-20T18:37:14.003596+0000","last_unstale":"2024-05-20T18:37:16.181527+0000","last_undegraded":"2024-05-20T18:37:16.181527+0000","last_fullsized":"2024-05-20T18:37:16.181527+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:08:07.110329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180959+0000","last_change":"2024-05-20T18:37:14.999384+0000","last_active":"2024-05-20T18:37:16.180959+0000","last_peered":"2024-05-20T18:37:16.180959+0000","last_clean":"2024-05-20T18:37:16.180959+0000","last_became_active":"2024-05-20T18:37:14.999033+0000","last_became_peered":"2024-05-20T18:37:14.999033+0000","last_unstale":"2024-05-20T18:37:16.180959+0000","last_undegraded":"2024-05-20T18:37:16.180959+0000","last_fullsized":"2024-05-20T18:37:16.180959+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:42:54.909774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180672+0000","last_change":"2024-05-20T18:37:13.995209+0000","last_active":"2024-05-20T18:37:16.180672+0000","last_peered":"2024-05-20T18:37:16.180672+0000","last_clean":"2024-05-20T18:37:16.180672+0000","last_became_active":"2024-05-20T18:37:13.994872+0000","last_became_peered":"2024-05-20T18:37:13.994872+0000","last_unstale":"2024-05-20T18:37:16.180672+0000","last_undegraded":"2024-05-20T18:37:16.180672+0000","last_fullsized":"2024-05-20T18:37:16.180672+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:23:45.134561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.006127+0000","last_change":"2024-05-20T18:37:15.003161+0000","last_active":"2024-05-20T18:37:16.006127+0000","last_peered":"2024-05-20T18:37:16.006127+0000","last_clean":"2024-05-20T18:37:16.006127+0000","last_became_active":"2024-05-20T18:37:15.002779+0000","last_became_peered":"2024-05-20T18:37:15.002779+0000","last_unstale":"2024-05-20T18:37:16.006127+0000","last_undegraded":"2024-05-20T18:37:16.006127+0000","last_fullsized":"2024-05-20T18:37:16.006127+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:08:00.924313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.005808+0000","last_change":"2024-05-20T18:37:14.000467+0000","last_active":"2024-05-20T18:37:16.005808+0000","last_peered":"2024-05-20T18:37:16.005808+0000","last_clean":"2024-05-20T18:37:16.005808+0000","last_became_active":"2024-05-20T18:37:14.000117+0000","last_became_peered":"2024-05-20T18:37:14.000117+0000","last_unstale":"2024-05-20T18:37:16.005808+0000","last_undegraded":"2024-05-20T18:37:16.005808+0000","last_fullsized":"2024-05-20T18:37:16.005808+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:35:16.697947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181035+0000","last_change":"2024-05-20T18:37:14.994215+0000","last_active":"2024-05-20T18:37:16.181035+0000","last_peered":"2024-05-20T18:37:16.181035+0000","last_clean":"2024-05-20T18:37:16.181035+0000","last_became_active":"2024-05-20T18:37:14.993772+0000","last_became_peered":"2024-05-20T18:37:14.993772+0000","last_unstale":"2024-05-20T18:37:16.181035+0000","last_undegraded":"2024-05-20T18:37:16.181035+0000","last_fullsized":"2024-05-20T18:37:16.181035+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:13.723823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"119'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.006646+0000","last_change":"2024-05-20T18:37:14.000744+0000","last_active":"2024-05-20T18:37:16.006646+0000","last_peered":"2024-05-20T18:37:16.006646+0000","last_clean":"2024-05-20T18:37:16.006646+0000","last_became_active":"2024-05-20T18:37:14.000471+0000","last_became_peered":"2024-05-20T18:37:14.000471+0000","last_unstale":"2024-05-20T18:37:16.006646+0000","last_undegraded":"2024-05-20T18:37:16.006646+0000","last_fullsized":"2024-05-20T18:37:16.006646+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:14:00.046447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181953+0000","last_change":"2024-05-20T18:37:15.012874+0000","last_active":"2024-05-20T18:37:16.181953+0000","last_peered":"2024-05-20T18:37:16.181953+0000","last_clean":"2024-05-20T18:37:16.181953+0000","last_became_active":"2024-05-20T18:37:15.012551+0000","last_became_peered":"2024-05-20T18:37:15.012551+0000","last_unstale":"2024-05-20T18:37:16.181953+0000","last_undegraded":"2024-05-20T18:37:16.181953+0000","last_fullsized":"2024-05-20T18:37:16.181953+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:52:52.789301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.003040+0000","last_change":"2024-05-20T18:37:13.999989+0000","last_active":"2024-05-20T18:37:16.003040+0000","last_peered":"2024-05-20T18:37:16.003040+0000","last_clean":"2024-05-20T18:37:16.003040+0000","last_became_active":"2024-05-20T18:37:13.999742+0000","last_became_peered":"2024-05-20T18:37:13.999742+0000","last_unstale":"2024-05-20T18:37:16.003040+0000","last_undegraded":"2024-05-20T18:37:16.003040+0000","last_fullsized":"2024-05-20T18:37:16.003040+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:00:40.463066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:15.998537+0000","last_change":"2024-05-20T18:37:15.008013+0000","last_active":"2024-05-20T18:37:15.998537+0000","last_peered":"2024-05-20T18:37:15.998537+0000","last_clean":"2024-05-20T18:37:15.998537+0000","last_became_active":"2024-05-20T18:37:15.007682+0000","last_became_peered":"2024-05-20T18:37:15.007682+0000","last_unstale":"2024-05-20T18:37:15.998537+0000","last_undegraded":"2024-05-20T18:37:15.998537+0000","last_fullsized":"2024-05-20T18:37:15.998537+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:15:09.133031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.006181+0000","last_change":"2024-05-20T18:37:14.002593+0000","last_active":"2024-05-20T18:37:16.006181+0000","last_peered":"2024-05-20T18:37:16.006181+0000","last_clean":"2024-05-20T18:37:16.006181+0000","last_became_active":"2024-05-20T18:37:14.002216+0000","last_became_peered":"2024-05-20T18:37:14.002216+0000","last_unstale":"2024-05-20T18:37:16.006181+0000","last_undegraded":"2024-05-20T18:37:16.006181+0000","last_fullsized":"2024-05-20T18:37:16.006181+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:27:34.640410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.005707+0000","last_change":"2024-05-20T18:37:15.011640+0000","last_active":"2024-05-20T18:37:16.005707+0000","last_peered":"2024-05-20T18:37:16.005707+0000","last_clean":"2024-05-20T18:37:16.005707+0000","last_became_active":"2024-05-20T18:37:15.011335+0000","last_became_peered":"2024-05-20T18:37:15.011335+0000","last_unstale":"2024-05-20T18:37:16.005707+0000","last_undegraded":"2024-05-20T18:37:16.005707+0000","last_fullsized":"2024-05-20T18:37:16.005707+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:38:08.377203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.002983+0000","last_change":"2024-05-20T18:37:13.993423+0000","last_active":"2024-05-20T18:37:16.002983+0000","last_peered":"2024-05-20T18:37:16.002983+0000","last_clean":"2024-05-20T18:37:16.002983+0000","last_became_active":"2024-05-20T18:37:13.993049+0000","last_became_peered":"2024-05-20T18:37:13.993049+0000","last_unstale":"2024-05-20T18:37:16.002983+0000","last_undegraded":"2024-05-20T18:37:16.002983+0000","last_fullsized":"2024-05-20T18:37:16.002983+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:01:04.468095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181412+0000","last_change":"2024-05-20T18:37:15.000899+0000","last_active":"2024-05-20T18:37:16.181412+0000","last_peered":"2024-05-20T18:37:16.181412+0000","last_clean":"2024-05-20T18:37:16.181412+0000","last_became_active":"2024-05-20T18:37:15.000522+0000","last_became_peered":"2024-05-20T18:37:15.000522+0000","last_unstale":"2024-05-20T18:37:16.181412+0000","last_undegraded":"2024-05-20T18:37:16.181412+0000","last_fullsized":"2024-05-20T18:37:16.181412+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:45:09.518395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.002379+0000","last_change":"2024-05-20T18:37:13.990987+0000","last_active":"2024-05-20T18:37:16.002379+0000","last_peered":"2024-05-20T18:37:16.002379+0000","last_clean":"2024-05-20T18:37:16.002379+0000","last_became_active":"2024-05-20T18:37:13.990659+0000","last_became_peered":"2024-05-20T18:37:13.990659+0000","last_unstale":"2024-05-20T18:37:16.002379+0000","last_undegraded":"2024-05-20T18:37:16.002379+0000","last_fullsized":"2024-05-20T18:37:16.002379+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:00:40.463066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181004+0000","last_change":"2024-05-20T18:37:15.009130+0000","last_active":"2024-05-20T18:37:16.181004+0000","last_peered":"2024-05-20T18:37:16.181004+0000","last_clean":"2024-05-20T18:37:16.181004+0000","last_became_active":"2024-05-20T18:37:15.008519+0000","last_became_peered":"2024-05-20T18:37:15.008519+0000","last_unstale":"2024-05-20T18:37:16.181004+0000","last_undegraded":"2024-05-20T18:37:16.181004+0000","last_fullsized":"2024-05-20T18:37:16.181004+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:05:13.899060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.005909+0000","last_change":"2024-05-20T18:37:14.001820+0000","last_active":"2024-05-20T18:37:16.005909+0000","last_peered":"2024-05-20T18:37:16.005909+0000","last_clean":"2024-05-20T18:37:16.005909+0000","last_became_active":"2024-05-20T18:37:14.001371+0000","last_became_peered":"2024-05-20T18:37:14.001371+0000","last_unstale":"2024-05-20T18:37:16.005909+0000","last_undegraded":"2024-05-20T18:37:16.005909+0000","last_fullsized":"2024-05-20T18:37:16.005909+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:00:47.330826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180832+0000","last_change":"2024-05-20T18:37:15.009930+0000","last_active":"2024-05-20T18:37:16.180832+0000","last_peered":"2024-05-20T18:37:16.180832+0000","last_clean":"2024-05-20T18:37:16.180832+0000","last_became_active":"2024-05-20T18:37:15.009512+0000","last_became_peered":"2024-05-20T18:37:15.009512+0000","last_unstale":"2024-05-20T18:37:16.180832+0000","last_undegraded":"2024-05-20T18:37:16.180832+0000","last_fullsized":"2024-05-20T18:37:16.180832+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:59:53.017125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181075+0000","last_change":"2024-05-20T18:37:13.988195+0000","last_active":"2024-05-20T18:37:16.181075+0000","last_peered":"2024-05-20T18:37:16.181075+0000","last_clean":"2024-05-20T18:37:16.181075+0000","last_became_active":"2024-05-20T18:37:13.987879+0000","last_became_peered":"2024-05-20T18:37:13.987879+0000","last_unstale":"2024-05-20T18:37:16.181075+0000","last_undegraded":"2024-05-20T18:37:16.181075+0000","last_fullsized":"2024-05-20T18:37:16.181075+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:59:52.012404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181751+0000","last_change":"2024-05-20T18:37:15.008815+0000","last_active":"2024-05-20T18:37:16.181751+0000","last_peered":"2024-05-20T18:37:16.181751+0000","last_clean":"2024-05-20T18:37:16.181751+0000","last_became_active":"2024-05-20T18:37:15.008412+0000","last_became_peered":"2024-05-20T18:37:15.008412+0000","last_unstale":"2024-05-20T18:37:16.181751+0000","last_undegraded":"2024-05-20T18:37:16.181751+0000","last_fullsized":"2024-05-20T18:37:16.181751+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:17:23.407267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"118'8","reported_seq":22,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181276+0000","last_change":"2024-05-20T18:37:13.998153+0000","last_active":"2024-05-20T18:37:16.181276+0000","last_peered":"2024-05-20T18:37:16.181276+0000","last_clean":"2024-05-20T18:37:16.181276+0000","last_became_active":"2024-05-20T18:37:13.997841+0000","last_became_peered":"2024-05-20T18:37:13.997841+0000","last_unstale":"2024-05-20T18:37:16.181276+0000","last_undegraded":"2024-05-20T18:37:16.181276+0000","last_fullsized":"2024-05-20T18:37:16.181276+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:51:55.113103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":352,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180971+0000","last_change":"2024-05-20T18:27:03.765694+0000","last_active":"2024-05-20T18:37:16.180971+0000","last_peered":"2024-05-20T18:37:16.180971+0000","last_clean":"2024-05-20T18:37:16.180971+0000","last_became_active":"2024-05-20T18:27:03.764695+0000","last_became_peered":"2024-05-20T18:27:03.764695+0000","last_unstale":"2024-05-20T18:37:16.180971+0000","last_undegraded":"2024-05-20T18:37:16.180971+0000","last_fullsized":"2024-05-20T18:37:16.180971+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_clean_scrub_stamp":"2024-05-20T18:27:02.750853+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:30:04.407487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574975,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33216,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338624,"statfs":{"total":96636764160,"available":96602750976,"internally_reserved":0,"allocated":2080768,"data_stored":1276374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574973,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1486848,"data_stored":692504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574973,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1478656,"data_stored":685390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574973,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32636,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1482752,"data_stored":686842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574975,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32640,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339200,"statfs":{"total":96636764160,"available":96603340800,"internally_reserved":0,"allocated":1486848,"data_stored":686876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1470464,"data_stored":685230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574975,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1474560,"data_stored":685252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574974,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33204,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338636,"statfs":{"total":96636764160,"available":96602763264,"internally_reserved":0,"allocated":2068480,"data_stored":1275620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T18:37:19.075 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T18:37:19.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.224+0000 7f2b46b9d640 1 Processor -- start 2024-05-20T18:37:19.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.224+0000 7f2b46b9d640 1 -- start start 2024-05-20T18:37:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.226+0000 7f2b46b9d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2b40162820 0x7f2b400b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.226+0000 7f2b46b9d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b4006dae0 0x7f2b400b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.226+0000 7f2b46b9d640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b400d0630 con 0x7f2b400b3e80 2024-05-20T18:37:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.226+0000 7f2b46b9d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2b400d39c0 con 0x7f2b40162820 2024-05-20T18:37:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.226+0000 7f2b46b9d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2b400d3ca0 con 0x7f2b4006dae0 2024-05-20T18:37:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.226+0000 7f2b44939640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2b40162820 0x7f2b400b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.226+0000 7f2b4513a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b4006dae0 0x7f2b400b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.226+0000 7f2b4513a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b4006dae0 0x7f2b400b8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:52554/0 (socket says 172.21.15.5:52554) 2024-05-20T18:37:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.226+0000 7f2b44939640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2b40162820 0x7f2b400b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:54758/0 (socket says 172.21.15.5:54758) 2024-05-20T18:37:19.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.226+0000 7f2b4513a640 1 -- 172.21.15.5:0/1234321021 learned_addr learned my addr 172.21.15.5:0/1234321021 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:19.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.227+0000 7f2b4513a640 1 -- 172.21.15.5:0/1234321021 >> v2:172.21.15.63:3300/0 conn(0x7f2b40162820 msgr2=0x7f2b400b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:19.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.227+0000 7f2b4513a640 1 --2- 172.21.15.5:0/1234321021 >> v2:172.21.15.63:3300/0 conn(0x7f2b40162820 0x7f2b400b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:19.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.227+0000 7f2b4513a640 1 -- 172.21.15.5:0/1234321021 >> v1:172.21.15.63:6789/0 conn(0x7f2b400b3e80 legacy=0x7f2b4006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:19.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.227+0000 7f2b4513a640 1 -- 172.21.15.5:0/1234321021 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b400d3f80 con 0x7f2b4006dae0 2024-05-20T18:37:19.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.228+0000 7f2b4513a640 1 --2- 172.21.15.5:0/1234321021 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b4006dae0 0x7f2b400b8190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009e50 tx=0x7f2b3802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e3ecaa07fd766a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:19.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.228+0000 7f2b3f7fe640 1 -- 172.21.15.5:0/1234321021 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b38036070 con 0x7f2b4006dae0 2024-05-20T18:37:19.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.228+0000 7f2b3f7fe640 1 -- 172.21.15.5:0/1234321021 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b38031600 con 0x7f2b4006dae0 2024-05-20T18:37:19.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.228+0000 7f2b3f7fe640 1 -- 172.21.15.5:0/1234321021 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b380319d0 con 0x7f2b4006dae0 2024-05-20T18:37:19.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.228+0000 7f2b46b9d640 1 -- 172.21.15.5:0/1234321021 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b4006dae0 msgr2=0x7f2b400b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.228+0000 7f2b46b9d640 1 --2- 172.21.15.5:0/1234321021 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b4006dae0 0x7f2b400b8190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009e50 tx=0x7f2b3802a710 comp rx=0 tx=0).stop 2024-05-20T18:37:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.228+0000 7f2b46b9d640 1 -- 172.21.15.5:0/1234321021 shutdown_connections 2024-05-20T18:37:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.228+0000 7f2b46b9d640 1 --2- 172.21.15.5:0/1234321021 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b4006dae0 0x7f2b400b8190 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.228+0000 7f2b46b9d640 1 --2- 172.21.15.5:0/1234321021 >> v2:172.21.15.63:3300/0 conn(0x7f2b40162820 0x7f2b400b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.228+0000 7f2b46b9d640 1 -- 172.21.15.5:0/1234321021 >> 172.21.15.5:0/1234321021 conn(0x7f2b400d4bd0 msgr2=0x7f2b400d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.228+0000 7f2b46b9d640 1 -- 172.21.15.5:0/1234321021 shutdown_connections 2024-05-20T18:37:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.228+0000 7f2b46b9d640 1 -- 172.21.15.5:0/1234321021 wait complete. 2024-05-20T18:37:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.230+0000 7f2b46b9d640 1 Processor -- start 2024-05-20T18:37:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.230+0000 7f2b46b9d640 1 -- start start 2024-05-20T18:37:19.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.232+0000 7f2b46b9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b400b3e80 0x7f2b4015b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:19.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.232+0000 7f2b46b9d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b40162820 0x7f2b400b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:19.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.232+0000 7f2b46b9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b400b5920 0x7f2b400cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:19.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.232+0000 7f2b46b9d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2b400d4780 con 0x7f2b400b3e80 2024-05-20T18:37:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.232+0000 7f2b46b9d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2b400d0300 con 0x7f2b400b5920 2024-05-20T18:37:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.232+0000 7f2b46b9d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2b400d4900 con 0x7f2b40162820 2024-05-20T18:37:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.232+0000 7f2b44939640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b400b3e80 0x7f2b4015b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.232+0000 7f2b44939640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b400b3e80 0x7f2b4015b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34328/0 (socket says 172.21.15.5:34328) 2024-05-20T18:37:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.232+0000 7f2b44939640 1 -- 172.21.15.5:0/518262300 learned_addr learned my addr 172.21.15.5:0/518262300 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.232+0000 7f2b4513a640 1 --2- 172.21.15.5:0/518262300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b400b5920 0x7f2b400cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.232+0000 7f2b3ffff640 1 --2- 172.21.15.5:0/518262300 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b40162820 0x7f2b400b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.232+0000 7f2b44939640 1 -- 172.21.15.5:0/518262300 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b40162820 msgr2=0x7f2b400b53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.232+0000 7f2b44939640 1 --2- 172.21.15.5:0/518262300 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b40162820 0x7f2b400b53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.232+0000 7f2b44939640 1 -- 172.21.15.5:0/518262300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b400b5920 msgr2=0x7f2b400cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.232+0000 7f2b44939640 1 --2- 172.21.15.5:0/518262300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b400b5920 0x7f2b400cb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.232+0000 7f2b44939640 1 -- 172.21.15.5:0/518262300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b400b6d20 con 0x7f2b400b3e80 2024-05-20T18:37:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.232+0000 7f2b44939640 1 --2- 172.21.15.5:0/518262300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b400b3e80 0x7f2b4015b1b0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f2b3000e8a0 tx=0x7f2b3000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:19.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.233+0000 7f2b3d7fa640 1 -- 172.21.15.5:0/518262300 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b30017070 con 0x7f2b400b3e80 2024-05-20T18:37:19.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.233+0000 7f2b3d7fa640 1 -- 172.21.15.5:0/518262300 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b3000c940 con 0x7f2b400b3e80 2024-05-20T18:37:19.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.233+0000 7f2b3d7fa640 1 -- 172.21.15.5:0/518262300 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3000cd30 con 0x7f2b400b3e80 2024-05-20T18:37:19.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.233+0000 7f2b46b9d640 1 -- 172.21.15.5:0/518262300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b400b6110 con 0x7f2b400b3e80 2024-05-20T18:37:19.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.233+0000 7f2b46b9d640 1 -- 172.21.15.5:0/518262300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b400b66d0 con 0x7f2b400b3e80 2024-05-20T18:37:19.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.234+0000 7f2b46b9d640 1 -- 172.21.15.5:0/518262300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b40075b00 con 0x7f2b400b3e80 2024-05-20T18:37:19.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.234+0000 7f2b3d7fa640 1 -- 172.21.15.5:0/518262300 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2b3001a070 con 0x7f2b400b3e80 2024-05-20T18:37:19.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.235+0000 7f2b3d7fa640 1 --2- 172.21.15.5:0/518262300 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2b2c06be30 0x7f2b2c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:19.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.235+0000 7f2b3ffff640 1 --2- 172.21.15.5:0/518262300 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2b2c06be30 0x7f2b2c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:19.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.235+0000 7f2b3d7fa640 1 -- 172.21.15.5:0/518262300 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7f2b3008b150 con 0x7f2b400b3e80 2024-05-20T18:37:19.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.235+0000 7f2b3ffff640 1 --2- 172.21.15.5:0/518262300 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2b2c06be30 0x7f2b2c06e340 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2b400cacb0 tx=0x7f2b34009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:19.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.240+0000 7f2b3d7fa640 1 -- 172.21.15.5:0/518262300 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2b400b6d20 con 0x7f2b400b3e80 2024-05-20T18:37:19.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.421+0000 7f2b46b9d640 1 -- 172.21.15.5:0/518262300 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2b40161df0 con 0x7f2b2c06be30 2024-05-20T18:37:19.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.427+0000 7f2b3d7fa640 1 -- 172.21.15.5:0/518262300 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252487 (secure 0 0 0) 0x7f2b40161df0 con 0x7f2b2c06be30 2024-05-20T18:37:19.432 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:19.433 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-20T18:37:19.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.430+0000 7f2b46b9d640 1 -- 172.21.15.5:0/518262300 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2b2c06be30 msgr2=0x7f2b2c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:19.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.430+0000 7f2b46b9d640 1 --2- 172.21.15.5:0/518262300 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2b2c06be30 0x7f2b2c06e340 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2b400cacb0 tx=0x7f2b34009040 comp rx=0 tx=0).stop 2024-05-20T18:37:19.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.430+0000 7f2b46b9d640 1 -- 172.21.15.5:0/518262300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b400b3e80 msgr2=0x7f2b4015b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:19.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.430+0000 7f2b46b9d640 1 --2- 172.21.15.5:0/518262300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b400b3e80 0x7f2b4015b1b0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f2b3000e8a0 tx=0x7f2b3000ed60 comp rx=0 tx=0).stop 2024-05-20T18:37:19.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.430+0000 7f2b46b9d640 1 -- 172.21.15.5:0/518262300 shutdown_connections 2024-05-20T18:37:19.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.430+0000 7f2b46b9d640 1 --2- 172.21.15.5:0/518262300 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2b2c06be30 0x7f2b2c06e340 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:19.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.431+0000 7f2b46b9d640 1 --2- 172.21.15.5:0/518262300 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b400b5920 0x7f2b400cb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:19.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.431+0000 7f2b46b9d640 1 --2- 172.21.15.5:0/518262300 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b40162820 0x7f2b400b53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:19.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.431+0000 7f2b46b9d640 1 --2- 172.21.15.5:0/518262300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b400b3e80 0x7f2b4015b1b0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:19.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.431+0000 7f2b46b9d640 1 -- 172.21.15.5:0/518262300 >> 172.21.15.5:0/518262300 conn(0x7f2b400d4bd0 msgr2=0x7f2b4015cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:19.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.431+0000 7f2b46b9d640 1 -- 172.21.15.5:0/518262300 shutdown_connections 2024-05-20T18:37:19.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.431+0000 7f2b46b9d640 1 -- 172.21.15.5:0/518262300 wait complete. 2024-05-20T18:37:19.443 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":421,"stamp":"2024-05-20T18:37:18.732606+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":156,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262208,"kb_used_data":12724,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712512,"statfs":{"total":773094113280,"available":772825612288,"internally_reserved":0,"allocated":13029376,"data_stored":6674088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.759937"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.003962+0000","last_change":"2024-05-20T18:37:15.011514+0000","last_active":"2024-05-20T18:37:16.003962+0000","last_peered":"2024-05-20T18:37:16.003962+0000","last_clean":"2024-05-20T18:37:16.003962+0000","last_became_active":"2024-05-20T18:37:15.009781+0000","last_became_peered":"2024-05-20T18:37:15.009781+0000","last_unstale":"2024-05-20T18:37:16.003962+0000","last_undegraded":"2024-05-20T18:37:16.003962+0000","last_fullsized":"2024-05-20T18:37:16.003962+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:45:47.732140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180783+0000","last_change":"2024-05-20T18:37:15.010720+0000","last_active":"2024-05-20T18:37:16.180783+0000","last_peered":"2024-05-20T18:37:16.180783+0000","last_clean":"2024-05-20T18:37:16.180783+0000","last_became_active":"2024-05-20T18:37:15.008847+0000","last_became_peered":"2024-05-20T18:37:15.008847+0000","last_unstale":"2024-05-20T18:37:16.180783+0000","last_undegraded":"2024-05-20T18:37:16.180783+0000","last_fullsized":"2024-05-20T18:37:16.180783+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:12:01.989254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181073+0000","last_change":"2024-05-20T18:37:15.006116+0000","last_active":"2024-05-20T18:37:16.181073+0000","last_peered":"2024-05-20T18:37:16.181073+0000","last_clean":"2024-05-20T18:37:16.181073+0000","last_became_active":"2024-05-20T18:37:15.005765+0000","last_became_peered":"2024-05-20T18:37:15.005765+0000","last_unstale":"2024-05-20T18:37:16.181073+0000","last_undegraded":"2024-05-20T18:37:16.181073+0000","last_fullsized":"2024-05-20T18:37:16.181073+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:24:39.770496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181995+0000","last_change":"2024-05-20T18:37:15.019066+0000","last_active":"2024-05-20T18:37:16.181995+0000","last_peered":"2024-05-20T18:37:16.181995+0000","last_clean":"2024-05-20T18:37:16.181995+0000","last_became_active":"2024-05-20T18:37:15.018671+0000","last_became_peered":"2024-05-20T18:37:15.018671+0000","last_unstale":"2024-05-20T18:37:16.181995+0000","last_undegraded":"2024-05-20T18:37:16.181995+0000","last_fullsized":"2024-05-20T18:37:16.181995+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:31:21.035991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.003058+0000","last_change":"2024-05-20T18:37:15.011675+0000","last_active":"2024-05-20T18:37:16.003058+0000","last_peered":"2024-05-20T18:37:16.003058+0000","last_clean":"2024-05-20T18:37:16.003058+0000","last_became_active":"2024-05-20T18:37:15.011253+0000","last_became_peered":"2024-05-20T18:37:15.011253+0000","last_unstale":"2024-05-20T18:37:16.003058+0000","last_undegraded":"2024-05-20T18:37:16.003058+0000","last_fullsized":"2024-05-20T18:37:16.003058+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:13.723823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.005843+0000","last_change":"2024-05-20T18:37:15.019993+0000","last_active":"2024-05-20T18:37:16.005843+0000","last_peered":"2024-05-20T18:37:16.005843+0000","last_clean":"2024-05-20T18:37:16.005843+0000","last_became_active":"2024-05-20T18:37:15.019594+0000","last_became_peered":"2024-05-20T18:37:15.019594+0000","last_unstale":"2024-05-20T18:37:16.005843+0000","last_undegraded":"2024-05-20T18:37:16.005843+0000","last_fullsized":"2024-05-20T18:37:16.005843+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:24:35.377207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.005741+0000","last_change":"2024-05-20T18:37:15.010292+0000","last_active":"2024-05-20T18:37:16.005741+0000","last_peered":"2024-05-20T18:37:16.005741+0000","last_clean":"2024-05-20T18:37:16.005741+0000","last_became_active":"2024-05-20T18:37:15.009918+0000","last_became_peered":"2024-05-20T18:37:15.009918+0000","last_unstale":"2024-05-20T18:37:16.005741+0000","last_undegraded":"2024-05-20T18:37:16.005741+0000","last_fullsized":"2024-05-20T18:37:16.005741+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:50:46.214512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181084+0000","last_change":"2024-05-20T18:37:15.006577+0000","last_active":"2024-05-20T18:37:16.181084+0000","last_peered":"2024-05-20T18:37:16.181084+0000","last_clean":"2024-05-20T18:37:16.181084+0000","last_became_active":"2024-05-20T18:37:15.006256+0000","last_became_peered":"2024-05-20T18:37:15.006256+0000","last_unstale":"2024-05-20T18:37:16.181084+0000","last_undegraded":"2024-05-20T18:37:16.181084+0000","last_fullsized":"2024-05-20T18:37:16.181084+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:29:45.355146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181963+0000","last_change":"2024-05-20T18:37:15.016997+0000","last_active":"2024-05-20T18:37:16.181963+0000","last_peered":"2024-05-20T18:37:16.181963+0000","last_clean":"2024-05-20T18:37:16.181963+0000","last_became_active":"2024-05-20T18:37:15.015933+0000","last_became_peered":"2024-05-20T18:37:15.015933+0000","last_unstale":"2024-05-20T18:37:16.181963+0000","last_undegraded":"2024-05-20T18:37:16.181963+0000","last_fullsized":"2024-05-20T18:37:16.181963+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:38:47.009102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180844+0000","last_change":"2024-05-20T18:37:15.001456+0000","last_active":"2024-05-20T18:37:16.180844+0000","last_peered":"2024-05-20T18:37:16.180844+0000","last_clean":"2024-05-20T18:37:16.180844+0000","last_became_active":"2024-05-20T18:37:15.001024+0000","last_became_peered":"2024-05-20T18:37:15.001024+0000","last_unstale":"2024-05-20T18:37:16.180844+0000","last_undegraded":"2024-05-20T18:37:16.180844+0000","last_fullsized":"2024-05-20T18:37:16.180844+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:14.967187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.003632+0000","last_change":"2024-05-20T18:37:15.014038+0000","last_active":"2024-05-20T18:37:16.003632+0000","last_peered":"2024-05-20T18:37:16.003632+0000","last_clean":"2024-05-20T18:37:16.003632+0000","last_became_active":"2024-05-20T18:37:15.013678+0000","last_became_peered":"2024-05-20T18:37:15.013678+0000","last_unstale":"2024-05-20T18:37:16.003632+0000","last_undegraded":"2024-05-20T18:37:16.003632+0000","last_fullsized":"2024-05-20T18:37:16.003632+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:42:54.909774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.006022+0000","last_change":"2024-05-20T18:37:15.013567+0000","last_active":"2024-05-20T18:37:16.006022+0000","last_peered":"2024-05-20T18:37:16.006022+0000","last_clean":"2024-05-20T18:37:16.006022+0000","last_became_active":"2024-05-20T18:37:15.013252+0000","last_became_peered":"2024-05-20T18:37:15.013252+0000","last_unstale":"2024-05-20T18:37:16.006022+0000","last_undegraded":"2024-05-20T18:37:16.006022+0000","last_fullsized":"2024-05-20T18:37:16.006022+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:15:29.541315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180952+0000","last_change":"2024-05-20T18:37:14.999101+0000","last_active":"2024-05-20T18:37:16.180952+0000","last_peered":"2024-05-20T18:37:16.180952+0000","last_clean":"2024-05-20T18:37:16.180952+0000","last_became_active":"2024-05-20T18:37:14.998786+0000","last_became_peered":"2024-05-20T18:37:14.998786+0000","last_unstale":"2024-05-20T18:37:16.180952+0000","last_undegraded":"2024-05-20T18:37:16.180952+0000","last_fullsized":"2024-05-20T18:37:16.180952+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:33:35.377520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181457+0000","last_change":"2024-05-20T18:37:15.010296+0000","last_active":"2024-05-20T18:37:16.181457+0000","last_peered":"2024-05-20T18:37:16.181457+0000","last_clean":"2024-05-20T18:37:16.181457+0000","last_became_active":"2024-05-20T18:37:15.009968+0000","last_became_peered":"2024-05-20T18:37:15.009968+0000","last_unstale":"2024-05-20T18:37:16.181457+0000","last_undegraded":"2024-05-20T18:37:16.181457+0000","last_fullsized":"2024-05-20T18:37:16.181457+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:35:17.702668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.002166+0000","last_change":"2024-05-20T18:37:15.016621+0000","last_active":"2024-05-20T18:37:16.002166+0000","last_peered":"2024-05-20T18:37:16.002166+0000","last_clean":"2024-05-20T18:37:16.002166+0000","last_became_active":"2024-05-20T18:37:15.016304+0000","last_became_peered":"2024-05-20T18:37:15.016304+0000","last_unstale":"2024-05-20T18:37:16.002166+0000","last_undegraded":"2024-05-20T18:37:16.002166+0000","last_fullsized":"2024-05-20T18:37:16.002166+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:14.967187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.005525+0000","last_change":"2024-05-20T18:37:14.999598+0000","last_active":"2024-05-20T18:37:16.005525+0000","last_peered":"2024-05-20T18:37:16.005525+0000","last_clean":"2024-05-20T18:37:16.005525+0000","last_became_active":"2024-05-20T18:37:14.999289+0000","last_became_peered":"2024-05-20T18:37:14.999289+0000","last_unstale":"2024-05-20T18:37:16.005525+0000","last_undegraded":"2024-05-20T18:37:16.005525+0000","last_fullsized":"2024-05-20T18:37:16.005525+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:24:31.957377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180774+0000","last_change":"2024-05-20T18:37:15.000116+0000","last_active":"2024-05-20T18:37:16.180774+0000","last_peered":"2024-05-20T18:37:16.180774+0000","last_clean":"2024-05-20T18:37:16.180774+0000","last_became_active":"2024-05-20T18:37:14.999791+0000","last_became_peered":"2024-05-20T18:37:14.999791+0000","last_unstale":"2024-05-20T18:37:16.180774+0000","last_undegraded":"2024-05-20T18:37:16.180774+0000","last_fullsized":"2024-05-20T18:37:16.180774+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:23:46.139283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181135+0000","last_change":"2024-05-20T18:37:15.008179+0000","last_active":"2024-05-20T18:37:16.181135+0000","last_peered":"2024-05-20T18:37:16.181135+0000","last_clean":"2024-05-20T18:37:16.181135+0000","last_became_active":"2024-05-20T18:37:15.007807+0000","last_became_peered":"2024-05-20T18:37:15.007807+0000","last_unstale":"2024-05-20T18:37:16.181135+0000","last_undegraded":"2024-05-20T18:37:16.181135+0000","last_fullsized":"2024-05-20T18:37:16.181135+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:17:09.591443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.002836+0000","last_change":"2024-05-20T18:37:15.013161+0000","last_active":"2024-05-20T18:37:16.002836+0000","last_peered":"2024-05-20T18:37:16.002836+0000","last_clean":"2024-05-20T18:37:16.002836+0000","last_became_active":"2024-05-20T18:37:15.012529+0000","last_became_peered":"2024-05-20T18:37:15.012529+0000","last_unstale":"2024-05-20T18:37:16.002836+0000","last_undegraded":"2024-05-20T18:37:16.002836+0000","last_fullsized":"2024-05-20T18:37:16.002836+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:14.967187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:15.998413+0000","last_change":"2024-05-20T18:37:14.998616+0000","last_active":"2024-05-20T18:37:15.998413+0000","last_peered":"2024-05-20T18:37:15.998413+0000","last_clean":"2024-05-20T18:37:15.998413+0000","last_became_active":"2024-05-20T18:37:14.998273+0000","last_became_peered":"2024-05-20T18:37:14.998273+0000","last_unstale":"2024-05-20T18:37:15.998413+0000","last_undegraded":"2024-05-20T18:37:15.998413+0000","last_fullsized":"2024-05-20T18:37:15.998413+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:22:23.482722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.182059+0000","last_change":"2024-05-20T18:37:15.008413+0000","last_active":"2024-05-20T18:37:16.182059+0000","last_peered":"2024-05-20T18:37:16.182059+0000","last_clean":"2024-05-20T18:37:16.182059+0000","last_became_active":"2024-05-20T18:37:15.007971+0000","last_became_peered":"2024-05-20T18:37:15.007971+0000","last_unstale":"2024-05-20T18:37:16.182059+0000","last_undegraded":"2024-05-20T18:37:16.182059+0000","last_fullsized":"2024-05-20T18:37:16.182059+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:17:17.517079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.002267+0000","last_change":"2024-05-20T18:37:15.019953+0000","last_active":"2024-05-20T18:37:16.002267+0000","last_peered":"2024-05-20T18:37:16.002267+0000","last_clean":"2024-05-20T18:37:16.002267+0000","last_became_active":"2024-05-20T18:37:15.019578+0000","last_became_peered":"2024-05-20T18:37:15.019578+0000","last_unstale":"2024-05-20T18:37:16.002267+0000","last_undegraded":"2024-05-20T18:37:16.002267+0000","last_fullsized":"2024-05-20T18:37:16.002267+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:33:35.377520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181278+0000","last_change":"2024-05-20T18:37:15.012654+0000","last_active":"2024-05-20T18:37:16.181278+0000","last_peered":"2024-05-20T18:37:16.181278+0000","last_clean":"2024-05-20T18:37:16.181278+0000","last_became_active":"2024-05-20T18:37:15.012307+0000","last_became_peered":"2024-05-20T18:37:15.012307+0000","last_unstale":"2024-05-20T18:37:16.181278+0000","last_undegraded":"2024-05-20T18:37:16.181278+0000","last_fullsized":"2024-05-20T18:37:16.181278+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:14:17.389325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180661+0000","last_change":"2024-05-20T18:37:15.001101+0000","last_active":"2024-05-20T18:37:16.180661+0000","last_peered":"2024-05-20T18:37:16.180661+0000","last_clean":"2024-05-20T18:37:16.180661+0000","last_became_active":"2024-05-20T18:37:15.000773+0000","last_became_peered":"2024-05-20T18:37:15.000773+0000","last_unstale":"2024-05-20T18:37:16.180661+0000","last_undegraded":"2024-05-20T18:37:16.180661+0000","last_fullsized":"2024-05-20T18:37:16.180661+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:33:35.377520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181239+0000","last_change":"2024-05-20T18:37:15.013502+0000","last_active":"2024-05-20T18:37:16.181239+0000","last_peered":"2024-05-20T18:37:16.181239+0000","last_clean":"2024-05-20T18:37:16.181239+0000","last_became_active":"2024-05-20T18:37:15.013156+0000","last_became_peered":"2024-05-20T18:37:15.013156+0000","last_unstale":"2024-05-20T18:37:16.181239+0000","last_undegraded":"2024-05-20T18:37:16.181239+0000","last_fullsized":"2024-05-20T18:37:16.181239+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:45:47.732140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:15.998484+0000","last_change":"2024-05-20T18:37:14.999693+0000","last_active":"2024-05-20T18:37:15.998484+0000","last_peered":"2024-05-20T18:37:15.998484+0000","last_clean":"2024-05-20T18:37:15.998484+0000","last_became_active":"2024-05-20T18:37:14.999253+0000","last_became_peered":"2024-05-20T18:37:14.999253+0000","last_unstale":"2024-05-20T18:37:15.998484+0000","last_undegraded":"2024-05-20T18:37:15.998484+0000","last_fullsized":"2024-05-20T18:37:15.998484+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:20:50.905337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.003082+0000","last_change":"2024-05-20T18:37:15.012911+0000","last_active":"2024-05-20T18:37:16.003082+0000","last_peered":"2024-05-20T18:37:16.003082+0000","last_clean":"2024-05-20T18:37:16.003082+0000","last_became_active":"2024-05-20T18:37:15.012063+0000","last_became_peered":"2024-05-20T18:37:15.012063+0000","last_unstale":"2024-05-20T18:37:16.003082+0000","last_undegraded":"2024-05-20T18:37:16.003082+0000","last_fullsized":"2024-05-20T18:37:16.003082+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:33:35.377520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.006121+0000","last_change":"2024-05-20T18:37:15.006949+0000","last_active":"2024-05-20T18:37:16.006121+0000","last_peered":"2024-05-20T18:37:16.006121+0000","last_clean":"2024-05-20T18:37:16.006121+0000","last_became_active":"2024-05-20T18:37:15.006476+0000","last_became_peered":"2024-05-20T18:37:15.006476+0000","last_unstale":"2024-05-20T18:37:16.006121+0000","last_undegraded":"2024-05-20T18:37:16.006121+0000","last_fullsized":"2024-05-20T18:37:16.006121+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:12:01.175574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:15.998524+0000","last_change":"2024-05-20T18:37:15.016442+0000","last_active":"2024-05-20T18:37:15.998524+0000","last_peered":"2024-05-20T18:37:15.998524+0000","last_clean":"2024-05-20T18:37:15.998524+0000","last_became_active":"2024-05-20T18:37:15.016120+0000","last_became_peered":"2024-05-20T18:37:15.016120+0000","last_unstale":"2024-05-20T18:37:15.998524+0000","last_undegraded":"2024-05-20T18:37:15.998524+0000","last_fullsized":"2024-05-20T18:37:15.998524+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:19:39.642333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181428+0000","last_change":"2024-05-20T18:37:15.013302+0000","last_active":"2024-05-20T18:37:16.181428+0000","last_peered":"2024-05-20T18:37:16.181428+0000","last_clean":"2024-05-20T18:37:16.181428+0000","last_became_active":"2024-05-20T18:37:15.012938+0000","last_became_peered":"2024-05-20T18:37:15.012938+0000","last_unstale":"2024-05-20T18:37:16.181428+0000","last_undegraded":"2024-05-20T18:37:16.181428+0000","last_fullsized":"2024-05-20T18:37:16.181428+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:12:31.191269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180761+0000","last_change":"2024-05-20T18:37:14.999304+0000","last_active":"2024-05-20T18:37:16.180761+0000","last_peered":"2024-05-20T18:37:16.180761+0000","last_clean":"2024-05-20T18:37:16.180761+0000","last_became_active":"2024-05-20T18:37:14.998974+0000","last_became_peered":"2024-05-20T18:37:14.998974+0000","last_unstale":"2024-05-20T18:37:16.180761+0000","last_undegraded":"2024-05-20T18:37:16.180761+0000","last_fullsized":"2024-05-20T18:37:16.180761+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:47:53.793796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180816+0000","last_change":"2024-05-20T18:37:14.993720+0000","last_active":"2024-05-20T18:37:16.180816+0000","last_peered":"2024-05-20T18:37:16.180816+0000","last_clean":"2024-05-20T18:37:16.180816+0000","last_became_active":"2024-05-20T18:37:14.993330+0000","last_became_peered":"2024-05-20T18:37:14.993330+0000","last_unstale":"2024-05-20T18:37:16.180816+0000","last_undegraded":"2024-05-20T18:37:16.180816+0000","last_fullsized":"2024-05-20T18:37:16.180816+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:13.723823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180964+0000","last_change":"2024-05-20T18:37:14.998070+0000","last_active":"2024-05-20T18:37:16.180964+0000","last_peered":"2024-05-20T18:37:16.180964+0000","last_clean":"2024-05-20T18:37:16.180964+0000","last_became_active":"2024-05-20T18:37:14.997718+0000","last_became_peered":"2024-05-20T18:37:14.997718+0000","last_unstale":"2024-05-20T18:37:16.180964+0000","last_undegraded":"2024-05-20T18:37:16.180964+0000","last_fullsized":"2024-05-20T18:37:16.180964+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:47:53.793796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"118'4","reported_seq":18,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.002625+0000","last_change":"2024-05-20T18:37:13.988387+0000","last_active":"2024-05-20T18:37:16.002625+0000","last_peered":"2024-05-20T18:37:16.002625+0000","last_clean":"2024-05-20T18:37:16.002625+0000","last_became_active":"2024-05-20T18:37:13.988076+0000","last_became_peered":"2024-05-20T18:37:13.988076+0000","last_unstale":"2024-05-20T18:37:16.002625+0000","last_undegraded":"2024-05-20T18:37:16.002625+0000","last_fullsized":"2024-05-20T18:37:16.002625+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:01:04.468095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.005688+0000","last_change":"2024-05-20T18:37:15.011785+0000","last_active":"2024-05-20T18:37:16.005688+0000","last_peered":"2024-05-20T18:37:16.005688+0000","last_clean":"2024-05-20T18:37:16.005688+0000","last_became_active":"2024-05-20T18:37:15.011476+0000","last_became_peered":"2024-05-20T18:37:15.011476+0000","last_unstale":"2024-05-20T18:37:16.005688+0000","last_undegraded":"2024-05-20T18:37:16.005688+0000","last_fullsized":"2024-05-20T18:37:16.005688+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:33:02.222185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"118'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181029+0000","last_change":"2024-05-20T18:37:13.991010+0000","last_active":"2024-05-20T18:37:16.181029+0000","last_peered":"2024-05-20T18:37:16.181029+0000","last_clean":"2024-05-20T18:37:16.181029+0000","last_became_active":"2024-05-20T18:37:13.990714+0000","last_became_peered":"2024-05-20T18:37:13.990714+0000","last_unstale":"2024-05-20T18:37:16.181029+0000","last_undegraded":"2024-05-20T18:37:16.181029+0000","last_fullsized":"2024-05-20T18:37:16.181029+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:51:59.490183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181086+0000","last_change":"2024-05-20T18:37:15.005081+0000","last_active":"2024-05-20T18:37:16.181086+0000","last_peered":"2024-05-20T18:37:16.181086+0000","last_clean":"2024-05-20T18:37:16.181086+0000","last_became_active":"2024-05-20T18:37:15.004756+0000","last_became_peered":"2024-05-20T18:37:15.004756+0000","last_unstale":"2024-05-20T18:37:16.181086+0000","last_undegraded":"2024-05-20T18:37:16.181086+0000","last_fullsized":"2024-05-20T18:37:16.181086+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:58:06.602987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181976+0000","last_change":"2024-05-20T18:37:13.995516+0000","last_active":"2024-05-20T18:37:16.181976+0000","last_peered":"2024-05-20T18:37:16.181976+0000","last_clean":"2024-05-20T18:37:16.181976+0000","last_became_active":"2024-05-20T18:37:13.995175+0000","last_became_peered":"2024-05-20T18:37:13.995175+0000","last_unstale":"2024-05-20T18:37:16.181976+0000","last_undegraded":"2024-05-20T18:37:16.181976+0000","last_fullsized":"2024-05-20T18:37:16.181976+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:33:34.372799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.182073+0000","last_change":"2024-05-20T18:37:15.009642+0000","last_active":"2024-05-20T18:37:16.182073+0000","last_peered":"2024-05-20T18:37:16.182073+0000","last_clean":"2024-05-20T18:37:16.182073+0000","last_became_active":"2024-05-20T18:37:15.008987+0000","last_became_peered":"2024-05-20T18:37:15.008987+0000","last_unstale":"2024-05-20T18:37:16.182073+0000","last_undegraded":"2024-05-20T18:37:16.182073+0000","last_fullsized":"2024-05-20T18:37:16.182073+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:50:45.389018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181099+0000","last_change":"2024-05-20T18:37:13.991400+0000","last_active":"2024-05-20T18:37:16.181099+0000","last_peered":"2024-05-20T18:37:16.181099+0000","last_clean":"2024-05-20T18:37:16.181099+0000","last_became_active":"2024-05-20T18:37:13.990372+0000","last_became_peered":"2024-05-20T18:37:13.990372+0000","last_unstale":"2024-05-20T18:37:16.181099+0000","last_undegraded":"2024-05-20T18:37:16.181099+0000","last_fullsized":"2024-05-20T18:37:16.181099+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:01:42.587185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180709+0000","last_change":"2024-05-20T18:37:14.995034+0000","last_active":"2024-05-20T18:37:16.180709+0000","last_peered":"2024-05-20T18:37:16.180709+0000","last_clean":"2024-05-20T18:37:16.180709+0000","last_became_active":"2024-05-20T18:37:14.994707+0000","last_became_peered":"2024-05-20T18:37:14.994707+0000","last_unstale":"2024-05-20T18:37:16.180709+0000","last_undegraded":"2024-05-20T18:37:16.180709+0000","last_fullsized":"2024-05-20T18:37:16.180709+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:39:22.103644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"118'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.002790+0000","last_change":"2024-05-20T18:37:13.983744+0000","last_active":"2024-05-20T18:37:16.002790+0000","last_peered":"2024-05-20T18:37:16.002790+0000","last_clean":"2024-05-20T18:37:16.002790+0000","last_became_active":"2024-05-20T18:37:13.983395+0000","last_became_peered":"2024-05-20T18:37:13.983395+0000","last_unstale":"2024-05-20T18:37:16.002790+0000","last_undegraded":"2024-05-20T18:37:16.002790+0000","last_fullsized":"2024-05-20T18:37:16.002790+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:11:51.568152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.002364+0000","last_change":"2024-05-20T18:37:15.013186+0000","last_active":"2024-05-20T18:37:16.002364+0000","last_peered":"2024-05-20T18:37:16.002364+0000","last_clean":"2024-05-20T18:37:16.002364+0000","last_became_active":"2024-05-20T18:37:15.012877+0000","last_became_peered":"2024-05-20T18:37:15.012877+0000","last_unstale":"2024-05-20T18:37:16.002364+0000","last_undegraded":"2024-05-20T18:37:16.002364+0000","last_fullsized":"2024-05-20T18:37:16.002364+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:47:53.793796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"118'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.005687+0000","last_change":"2024-05-20T18:37:14.000511+0000","last_active":"2024-05-20T18:37:16.005687+0000","last_peered":"2024-05-20T18:37:16.005687+0000","last_clean":"2024-05-20T18:37:16.005687+0000","last_became_active":"2024-05-20T18:37:14.000188+0000","last_became_peered":"2024-05-20T18:37:14.000188+0000","last_unstale":"2024-05-20T18:37:16.005687+0000","last_undegraded":"2024-05-20T18:37:16.005687+0000","last_fullsized":"2024-05-20T18:37:16.005687+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:14:03.223520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181526+0000","last_change":"2024-05-20T18:37:15.010572+0000","last_active":"2024-05-20T18:37:16.181526+0000","last_peered":"2024-05-20T18:37:16.181526+0000","last_clean":"2024-05-20T18:37:16.181526+0000","last_became_active":"2024-05-20T18:37:15.009946+0000","last_became_peered":"2024-05-20T18:37:15.009946+0000","last_unstale":"2024-05-20T18:37:16.181526+0000","last_undegraded":"2024-05-20T18:37:16.181526+0000","last_fullsized":"2024-05-20T18:37:16.181526+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:00:48.335547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181385+0000","last_change":"2024-05-20T18:37:13.990980+0000","last_active":"2024-05-20T18:37:16.181385+0000","last_peered":"2024-05-20T18:37:16.181385+0000","last_clean":"2024-05-20T18:37:16.181385+0000","last_became_active":"2024-05-20T18:37:13.990675+0000","last_became_peered":"2024-05-20T18:37:13.990675+0000","last_unstale":"2024-05-20T18:37:16.181385+0000","last_undegraded":"2024-05-20T18:37:16.181385+0000","last_fullsized":"2024-05-20T18:37:16.181385+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:49:11.347324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180650+0000","last_change":"2024-05-20T18:37:15.010905+0000","last_active":"2024-05-20T18:37:16.180650+0000","last_peered":"2024-05-20T18:37:16.180650+0000","last_clean":"2024-05-20T18:37:16.180650+0000","last_became_active":"2024-05-20T18:37:15.010597+0000","last_became_peered":"2024-05-20T18:37:15.010597+0000","last_unstale":"2024-05-20T18:37:16.180650+0000","last_undegraded":"2024-05-20T18:37:16.180650+0000","last_fullsized":"2024-05-20T18:37:16.180650+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:43:16.199181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.005708+0000","last_change":"2024-05-20T18:37:14.000613+0000","last_active":"2024-05-20T18:37:16.005708+0000","last_peered":"2024-05-20T18:37:16.005708+0000","last_clean":"2024-05-20T18:37:16.005708+0000","last_became_active":"2024-05-20T18:37:13.999533+0000","last_became_peered":"2024-05-20T18:37:13.999533+0000","last_unstale":"2024-05-20T18:37:16.005708+0000","last_undegraded":"2024-05-20T18:37:16.005708+0000","last_fullsized":"2024-05-20T18:37:16.005708+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:29:23.761809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.002682+0000","last_change":"2024-05-20T18:37:15.001552+0000","last_active":"2024-05-20T18:37:16.002682+0000","last_peered":"2024-05-20T18:37:16.002682+0000","last_clean":"2024-05-20T18:37:16.002682+0000","last_became_active":"2024-05-20T18:37:15.001237+0000","last_became_peered":"2024-05-20T18:37:15.001237+0000","last_unstale":"2024-05-20T18:37:16.002682+0000","last_undegraded":"2024-05-20T18:37:16.002682+0000","last_fullsized":"2024-05-20T18:37:16.002682+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:59:53.017125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.002916+0000","last_change":"2024-05-20T18:37:14.000726+0000","last_active":"2024-05-20T18:37:16.002916+0000","last_peered":"2024-05-20T18:37:16.002916+0000","last_clean":"2024-05-20T18:37:16.002916+0000","last_became_active":"2024-05-20T18:37:14.000459+0000","last_became_peered":"2024-05-20T18:37:14.000459+0000","last_unstale":"2024-05-20T18:37:16.002916+0000","last_undegraded":"2024-05-20T18:37:16.002916+0000","last_fullsized":"2024-05-20T18:37:16.002916+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:11:51.568152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181393+0000","last_change":"2024-05-20T18:37:15.015983+0000","last_active":"2024-05-20T18:37:16.181393+0000","last_peered":"2024-05-20T18:37:16.181393+0000","last_clean":"2024-05-20T18:37:16.181393+0000","last_became_active":"2024-05-20T18:37:15.014074+0000","last_became_peered":"2024-05-20T18:37:15.014074+0000","last_unstale":"2024-05-20T18:37:16.181393+0000","last_undegraded":"2024-05-20T18:37:16.181393+0000","last_fullsized":"2024-05-20T18:37:16.181393+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:33:06.615494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180894+0000","last_change":"2024-05-20T18:37:13.991931+0000","last_active":"2024-05-20T18:37:16.180894+0000","last_peered":"2024-05-20T18:37:16.180894+0000","last_clean":"2024-05-20T18:37:16.180894+0000","last_became_active":"2024-05-20T18:37:13.991611+0000","last_became_peered":"2024-05-20T18:37:13.991611+0000","last_unstale":"2024-05-20T18:37:16.180894+0000","last_undegraded":"2024-05-20T18:37:16.180894+0000","last_fullsized":"2024-05-20T18:37:16.180894+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:01:37.695206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181174+0000","last_change":"2024-05-20T18:37:15.006954+0000","last_active":"2024-05-20T18:37:16.181174+0000","last_peered":"2024-05-20T18:37:16.181174+0000","last_clean":"2024-05-20T18:37:16.181174+0000","last_became_active":"2024-05-20T18:37:15.006639+0000","last_became_peered":"2024-05-20T18:37:15.006639+0000","last_unstale":"2024-05-20T18:37:16.181174+0000","last_undegraded":"2024-05-20T18:37:16.181174+0000","last_fullsized":"2024-05-20T18:37:16.181174+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:35:27.774093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"118'3","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181484+0000","last_change":"2024-05-20T18:37:13.991796+0000","last_active":"2024-05-20T18:37:16.181484+0000","last_peered":"2024-05-20T18:37:16.181484+0000","last_clean":"2024-05-20T18:37:16.181484+0000","last_became_active":"2024-05-20T18:37:13.991445+0000","last_became_peered":"2024-05-20T18:37:13.991445+0000","last_unstale":"2024-05-20T18:37:16.181484+0000","last_undegraded":"2024-05-20T18:37:16.181484+0000","last_fullsized":"2024-05-20T18:37:16.181484+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:14:00.046447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181420+0000","last_change":"2024-05-20T18:37:15.000159+0000","last_active":"2024-05-20T18:37:16.181420+0000","last_peered":"2024-05-20T18:37:16.181420+0000","last_clean":"2024-05-20T18:37:16.181420+0000","last_became_active":"2024-05-20T18:37:14.999842+0000","last_became_peered":"2024-05-20T18:37:14.999842+0000","last_unstale":"2024-05-20T18:37:16.181420+0000","last_undegraded":"2024-05-20T18:37:16.181420+0000","last_fullsized":"2024-05-20T18:37:16.181420+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:52:24.191074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"118'6","reported_seq":20,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181289+0000","last_change":"2024-05-20T18:37:13.990269+0000","last_active":"2024-05-20T18:37:16.181289+0000","last_peered":"2024-05-20T18:37:16.181289+0000","last_clean":"2024-05-20T18:37:16.181289+0000","last_became_active":"2024-05-20T18:37:13.989960+0000","last_became_peered":"2024-05-20T18:37:13.989960+0000","last_unstale":"2024-05-20T18:37:16.181289+0000","last_undegraded":"2024-05-20T18:37:16.181289+0000","last_fullsized":"2024-05-20T18:37:16.181289+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T20:14:35.884964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.002403+0000","last_change":"2024-05-20T18:37:15.004032+0000","last_active":"2024-05-20T18:37:16.002403+0000","last_peered":"2024-05-20T18:37:16.002403+0000","last_clean":"2024-05-20T18:37:16.002403+0000","last_became_active":"2024-05-20T18:37:15.003654+0000","last_became_peered":"2024-05-20T18:37:15.003654+0000","last_unstale":"2024-05-20T18:37:16.002403+0000","last_undegraded":"2024-05-20T18:37:16.002403+0000","last_fullsized":"2024-05-20T18:37:16.002403+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:23:46.139283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180943+0000","last_change":"2024-05-20T18:37:13.989571+0000","last_active":"2024-05-20T18:37:16.180943+0000","last_peered":"2024-05-20T18:37:16.180943+0000","last_clean":"2024-05-20T18:37:16.180943+0000","last_became_active":"2024-05-20T18:37:13.989147+0000","last_became_peered":"2024-05-20T18:37:13.989147+0000","last_unstale":"2024-05-20T18:37:16.180943+0000","last_undegraded":"2024-05-20T18:37:16.180943+0000","last_fullsized":"2024-05-20T18:37:16.180943+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:34:18.346275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180600+0000","last_change":"2024-05-20T18:37:15.004403+0000","last_active":"2024-05-20T18:37:16.180600+0000","last_peered":"2024-05-20T18:37:16.180600+0000","last_clean":"2024-05-20T18:37:16.180600+0000","last_became_active":"2024-05-20T18:37:15.002347+0000","last_became_peered":"2024-05-20T18:37:15.002347+0000","last_unstale":"2024-05-20T18:37:16.180600+0000","last_undegraded":"2024-05-20T18:37:16.180600+0000","last_fullsized":"2024-05-20T18:37:16.180600+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:45:50.978401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181607+0000","last_change":"2024-05-20T18:37:13.998175+0000","last_active":"2024-05-20T18:37:16.181607+0000","last_peered":"2024-05-20T18:37:16.181607+0000","last_clean":"2024-05-20T18:37:16.181607+0000","last_became_active":"2024-05-20T18:37:13.997805+0000","last_became_peered":"2024-05-20T18:37:13.997805+0000","last_unstale":"2024-05-20T18:37:16.181607+0000","last_undegraded":"2024-05-20T18:37:16.181607+0000","last_fullsized":"2024-05-20T18:37:16.181607+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:31:34.605988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:15.998261+0000","last_change":"2024-05-20T18:37:15.014162+0000","last_active":"2024-05-20T18:37:15.998261+0000","last_peered":"2024-05-20T18:37:15.998261+0000","last_clean":"2024-05-20T18:37:15.998261+0000","last_became_active":"2024-05-20T18:37:15.013796+0000","last_became_peered":"2024-05-20T18:37:15.013796+0000","last_unstale":"2024-05-20T18:37:15.998261+0000","last_undegraded":"2024-05-20T18:37:15.998261+0000","last_fullsized":"2024-05-20T18:37:15.998261+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:33:04.932773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181484+0000","last_change":"2024-05-20T18:37:13.995413+0000","last_active":"2024-05-20T18:37:16.181484+0000","last_peered":"2024-05-20T18:37:16.181484+0000","last_clean":"2024-05-20T18:37:16.181484+0000","last_became_active":"2024-05-20T18:37:13.995054+0000","last_became_peered":"2024-05-20T18:37:13.995054+0000","last_unstale":"2024-05-20T18:37:16.181484+0000","last_undegraded":"2024-05-20T18:37:16.181484+0000","last_fullsized":"2024-05-20T18:37:16.181484+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:50:44.384297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181584+0000","last_change":"2024-05-20T18:37:15.004187+0000","last_active":"2024-05-20T18:37:16.181584+0000","last_peered":"2024-05-20T18:37:16.181584+0000","last_clean":"2024-05-20T18:37:16.181584+0000","last_became_active":"2024-05-20T18:37:15.003783+0000","last_became_peered":"2024-05-20T18:37:15.003783+0000","last_unstale":"2024-05-20T18:37:16.181584+0000","last_undegraded":"2024-05-20T18:37:16.181584+0000","last_fullsized":"2024-05-20T18:37:16.181584+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:42:21.574338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"118'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180765+0000","last_change":"2024-05-20T18:37:14.001188+0000","last_active":"2024-05-20T18:37:16.180765+0000","last_peered":"2024-05-20T18:37:16.180765+0000","last_clean":"2024-05-20T18:37:16.180765+0000","last_became_active":"2024-05-20T18:37:14.000895+0000","last_became_peered":"2024-05-20T18:37:14.000895+0000","last_unstale":"2024-05-20T18:37:16.180765+0000","last_undegraded":"2024-05-20T18:37:16.180765+0000","last_fullsized":"2024-05-20T18:37:16.180765+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:24:51.857629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.003758+0000","last_change":"2024-05-20T18:37:15.019659+0000","last_active":"2024-05-20T18:37:16.003758+0000","last_peered":"2024-05-20T18:37:16.003758+0000","last_clean":"2024-05-20T18:37:16.003758+0000","last_became_active":"2024-05-20T18:37:15.019319+0000","last_became_peered":"2024-05-20T18:37:15.019319+0000","last_unstale":"2024-05-20T18:37:16.003758+0000","last_undegraded":"2024-05-20T18:37:16.003758+0000","last_fullsized":"2024-05-20T18:37:16.003758+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:58:06.602987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:15.998605+0000","last_change":"2024-05-20T18:37:14.002718+0000","last_active":"2024-05-20T18:37:15.998605+0000","last_peered":"2024-05-20T18:37:15.998605+0000","last_clean":"2024-05-20T18:37:15.998605+0000","last_became_active":"2024-05-20T18:37:14.002338+0000","last_became_peered":"2024-05-20T18:37:14.002338+0000","last_unstale":"2024-05-20T18:37:15.998605+0000","last_undegraded":"2024-05-20T18:37:15.998605+0000","last_fullsized":"2024-05-20T18:37:15.998605+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:24:41.727547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.006111+0000","last_change":"2024-05-20T18:37:15.007582+0000","last_active":"2024-05-20T18:37:16.006111+0000","last_peered":"2024-05-20T18:37:16.006111+0000","last_clean":"2024-05-20T18:37:16.006111+0000","last_became_active":"2024-05-20T18:37:15.007175+0000","last_became_peered":"2024-05-20T18:37:15.007175+0000","last_unstale":"2024-05-20T18:37:16.006111+0000","last_undegraded":"2024-05-20T18:37:16.006111+0000","last_fullsized":"2024-05-20T18:37:16.006111+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:25:34.231637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"118'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.182076+0000","last_change":"2024-05-20T18:37:13.995870+0000","last_active":"2024-05-20T18:37:16.182076+0000","last_peered":"2024-05-20T18:37:16.182076+0000","last_clean":"2024-05-20T18:37:16.182076+0000","last_became_active":"2024-05-20T18:37:13.995474+0000","last_became_peered":"2024-05-20T18:37:13.995474+0000","last_unstale":"2024-05-20T18:37:16.182076+0000","last_undegraded":"2024-05-20T18:37:16.182076+0000","last_fullsized":"2024-05-20T18:37:16.182076+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:47:52.789075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181630+0000","last_change":"2024-05-20T18:37:15.003118+0000","last_active":"2024-05-20T18:37:16.181630+0000","last_peered":"2024-05-20T18:37:16.181630+0000","last_clean":"2024-05-20T18:37:16.181630+0000","last_became_active":"2024-05-20T18:37:15.002565+0000","last_became_peered":"2024-05-20T18:37:15.002565+0000","last_unstale":"2024-05-20T18:37:16.181630+0000","last_undegraded":"2024-05-20T18:37:16.181630+0000","last_fullsized":"2024-05-20T18:37:16.181630+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:48:01.975705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181775+0000","last_change":"2024-05-20T18:37:14.003282+0000","last_active":"2024-05-20T18:37:16.181775+0000","last_peered":"2024-05-20T18:37:16.181775+0000","last_clean":"2024-05-20T18:37:16.181775+0000","last_became_active":"2024-05-20T18:37:14.002922+0000","last_became_peered":"2024-05-20T18:37:14.002922+0000","last_unstale":"2024-05-20T18:37:16.181775+0000","last_undegraded":"2024-05-20T18:37:16.181775+0000","last_fullsized":"2024-05-20T18:37:16.181775+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:01:13.962466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.005510+0000","last_change":"2024-05-20T18:37:15.005929+0000","last_active":"2024-05-20T18:37:16.005510+0000","last_peered":"2024-05-20T18:37:16.005510+0000","last_clean":"2024-05-20T18:37:16.005510+0000","last_became_active":"2024-05-20T18:37:15.005567+0000","last_became_peered":"2024-05-20T18:37:15.005567+0000","last_unstale":"2024-05-20T18:37:16.005510+0000","last_undegraded":"2024-05-20T18:37:16.005510+0000","last_fullsized":"2024-05-20T18:37:16.005510+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:21:39.843969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.005620+0000","last_change":"2024-05-20T18:37:13.998501+0000","last_active":"2024-05-20T18:37:16.005620+0000","last_peered":"2024-05-20T18:37:16.005620+0000","last_clean":"2024-05-20T18:37:16.005620+0000","last_became_active":"2024-05-20T18:37:13.998181+0000","last_became_peered":"2024-05-20T18:37:13.998181+0000","last_unstale":"2024-05-20T18:37:16.005620+0000","last_undegraded":"2024-05-20T18:37:16.005620+0000","last_fullsized":"2024-05-20T18:37:16.005620+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:40:50.875776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181353+0000","last_change":"2024-05-20T18:37:15.005993+0000","last_active":"2024-05-20T18:37:16.181353+0000","last_peered":"2024-05-20T18:37:16.181353+0000","last_clean":"2024-05-20T18:37:16.181353+0000","last_became_active":"2024-05-20T18:37:15.005664+0000","last_became_peered":"2024-05-20T18:37:15.005664+0000","last_unstale":"2024-05-20T18:37:16.181353+0000","last_undegraded":"2024-05-20T18:37:16.181353+0000","last_fullsized":"2024-05-20T18:37:16.181353+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T18:55:08.267509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181231+0000","last_change":"2024-05-20T18:37:13.995250+0000","last_active":"2024-05-20T18:37:16.181231+0000","last_peered":"2024-05-20T18:37:16.181231+0000","last_clean":"2024-05-20T18:37:16.181231+0000","last_became_active":"2024-05-20T18:37:13.994781+0000","last_became_peered":"2024-05-20T18:37:13.994781+0000","last_unstale":"2024-05-20T18:37:16.181231+0000","last_undegraded":"2024-05-20T18:37:16.181231+0000","last_fullsized":"2024-05-20T18:37:16.181231+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:12.719102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181617+0000","last_change":"2024-05-20T18:37:15.004979+0000","last_active":"2024-05-20T18:37:16.181617+0000","last_peered":"2024-05-20T18:37:16.181617+0000","last_clean":"2024-05-20T18:37:16.181617+0000","last_became_active":"2024-05-20T18:37:15.004648+0000","last_became_peered":"2024-05-20T18:37:15.004648+0000","last_unstale":"2024-05-20T18:37:16.181617+0000","last_undegraded":"2024-05-20T18:37:16.181617+0000","last_fullsized":"2024-05-20T18:37:16.181617+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T22:14:04.228241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181527+0000","last_change":"2024-05-20T18:37:14.003922+0000","last_active":"2024-05-20T18:37:16.181527+0000","last_peered":"2024-05-20T18:37:16.181527+0000","last_clean":"2024-05-20T18:37:16.181527+0000","last_became_active":"2024-05-20T18:37:14.003596+0000","last_became_peered":"2024-05-20T18:37:14.003596+0000","last_unstale":"2024-05-20T18:37:16.181527+0000","last_undegraded":"2024-05-20T18:37:16.181527+0000","last_fullsized":"2024-05-20T18:37:16.181527+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:08:07.110329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180959+0000","last_change":"2024-05-20T18:37:14.999384+0000","last_active":"2024-05-20T18:37:16.180959+0000","last_peered":"2024-05-20T18:37:16.180959+0000","last_clean":"2024-05-20T18:37:16.180959+0000","last_became_active":"2024-05-20T18:37:14.999033+0000","last_became_peered":"2024-05-20T18:37:14.999033+0000","last_unstale":"2024-05-20T18:37:16.180959+0000","last_undegraded":"2024-05-20T18:37:16.180959+0000","last_fullsized":"2024-05-20T18:37:16.180959+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:42:54.909774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180672+0000","last_change":"2024-05-20T18:37:13.995209+0000","last_active":"2024-05-20T18:37:16.180672+0000","last_peered":"2024-05-20T18:37:16.180672+0000","last_clean":"2024-05-20T18:37:16.180672+0000","last_became_active":"2024-05-20T18:37:13.994872+0000","last_became_peered":"2024-05-20T18:37:13.994872+0000","last_unstale":"2024-05-20T18:37:16.180672+0000","last_undegraded":"2024-05-20T18:37:16.180672+0000","last_fullsized":"2024-05-20T18:37:16.180672+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:23:45.134561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.006127+0000","last_change":"2024-05-20T18:37:15.003161+0000","last_active":"2024-05-20T18:37:16.006127+0000","last_peered":"2024-05-20T18:37:16.006127+0000","last_clean":"2024-05-20T18:37:16.006127+0000","last_became_active":"2024-05-20T18:37:15.002779+0000","last_became_peered":"2024-05-20T18:37:15.002779+0000","last_unstale":"2024-05-20T18:37:16.006127+0000","last_undegraded":"2024-05-20T18:37:16.006127+0000","last_fullsized":"2024-05-20T18:37:16.006127+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T19:08:00.924313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.005808+0000","last_change":"2024-05-20T18:37:14.000467+0000","last_active":"2024-05-20T18:37:16.005808+0000","last_peered":"2024-05-20T18:37:16.005808+0000","last_clean":"2024-05-20T18:37:16.005808+0000","last_became_active":"2024-05-20T18:37:14.000117+0000","last_became_peered":"2024-05-20T18:37:14.000117+0000","last_unstale":"2024-05-20T18:37:16.005808+0000","last_undegraded":"2024-05-20T18:37:16.005808+0000","last_fullsized":"2024-05-20T18:37:16.005808+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:35:16.697947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181035+0000","last_change":"2024-05-20T18:37:14.994215+0000","last_active":"2024-05-20T18:37:16.181035+0000","last_peered":"2024-05-20T18:37:16.181035+0000","last_clean":"2024-05-20T18:37:16.181035+0000","last_became_active":"2024-05-20T18:37:14.993772+0000","last_became_peered":"2024-05-20T18:37:14.993772+0000","last_unstale":"2024-05-20T18:37:16.181035+0000","last_undegraded":"2024-05-20T18:37:16.181035+0000","last_fullsized":"2024-05-20T18:37:16.181035+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:40:13.723823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"119'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.006646+0000","last_change":"2024-05-20T18:37:14.000744+0000","last_active":"2024-05-20T18:37:16.006646+0000","last_peered":"2024-05-20T18:37:16.006646+0000","last_clean":"2024-05-20T18:37:16.006646+0000","last_became_active":"2024-05-20T18:37:14.000471+0000","last_became_peered":"2024-05-20T18:37:14.000471+0000","last_unstale":"2024-05-20T18:37:16.006646+0000","last_undegraded":"2024-05-20T18:37:16.006646+0000","last_fullsized":"2024-05-20T18:37:16.006646+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:14:00.046447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181953+0000","last_change":"2024-05-20T18:37:15.012874+0000","last_active":"2024-05-20T18:37:16.181953+0000","last_peered":"2024-05-20T18:37:16.181953+0000","last_clean":"2024-05-20T18:37:16.181953+0000","last_became_active":"2024-05-20T18:37:15.012551+0000","last_became_peered":"2024-05-20T18:37:15.012551+0000","last_unstale":"2024-05-20T18:37:16.181953+0000","last_undegraded":"2024-05-20T18:37:16.181953+0000","last_fullsized":"2024-05-20T18:37:16.181953+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:52:52.789301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.003040+0000","last_change":"2024-05-20T18:37:13.999989+0000","last_active":"2024-05-20T18:37:16.003040+0000","last_peered":"2024-05-20T18:37:16.003040+0000","last_clean":"2024-05-20T18:37:16.003040+0000","last_became_active":"2024-05-20T18:37:13.999742+0000","last_became_peered":"2024-05-20T18:37:13.999742+0000","last_unstale":"2024-05-20T18:37:16.003040+0000","last_undegraded":"2024-05-20T18:37:16.003040+0000","last_fullsized":"2024-05-20T18:37:16.003040+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:00:40.463066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:15.998537+0000","last_change":"2024-05-20T18:37:15.008013+0000","last_active":"2024-05-20T18:37:15.998537+0000","last_peered":"2024-05-20T18:37:15.998537+0000","last_clean":"2024-05-20T18:37:15.998537+0000","last_became_active":"2024-05-20T18:37:15.007682+0000","last_became_peered":"2024-05-20T18:37:15.007682+0000","last_unstale":"2024-05-20T18:37:15.998537+0000","last_undegraded":"2024-05-20T18:37:15.998537+0000","last_fullsized":"2024-05-20T18:37:15.998537+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:15:09.133031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.006181+0000","last_change":"2024-05-20T18:37:14.002593+0000","last_active":"2024-05-20T18:37:16.006181+0000","last_peered":"2024-05-20T18:37:16.006181+0000","last_clean":"2024-05-20T18:37:16.006181+0000","last_became_active":"2024-05-20T18:37:14.002216+0000","last_became_peered":"2024-05-20T18:37:14.002216+0000","last_unstale":"2024-05-20T18:37:16.006181+0000","last_undegraded":"2024-05-20T18:37:16.006181+0000","last_fullsized":"2024-05-20T18:37:16.006181+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T06:27:34.640410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.005707+0000","last_change":"2024-05-20T18:37:15.011640+0000","last_active":"2024-05-20T18:37:16.005707+0000","last_peered":"2024-05-20T18:37:16.005707+0000","last_clean":"2024-05-20T18:37:16.005707+0000","last_became_active":"2024-05-20T18:37:15.011335+0000","last_became_peered":"2024-05-20T18:37:15.011335+0000","last_unstale":"2024-05-20T18:37:16.005707+0000","last_undegraded":"2024-05-20T18:37:16.005707+0000","last_fullsized":"2024-05-20T18:37:16.005707+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T01:38:08.377203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.002983+0000","last_change":"2024-05-20T18:37:13.993423+0000","last_active":"2024-05-20T18:37:16.002983+0000","last_peered":"2024-05-20T18:37:16.002983+0000","last_clean":"2024-05-20T18:37:16.002983+0000","last_became_active":"2024-05-20T18:37:13.993049+0000","last_became_peered":"2024-05-20T18:37:13.993049+0000","last_unstale":"2024-05-20T18:37:16.002983+0000","last_undegraded":"2024-05-20T18:37:16.002983+0000","last_fullsized":"2024-05-20T18:37:16.002983+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T00:01:04.468095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181412+0000","last_change":"2024-05-20T18:37:15.000899+0000","last_active":"2024-05-20T18:37:16.181412+0000","last_peered":"2024-05-20T18:37:16.181412+0000","last_clean":"2024-05-20T18:37:16.181412+0000","last_became_active":"2024-05-20T18:37:15.000522+0000","last_became_peered":"2024-05-20T18:37:15.000522+0000","last_unstale":"2024-05-20T18:37:16.181412+0000","last_undegraded":"2024-05-20T18:37:16.181412+0000","last_fullsized":"2024-05-20T18:37:16.181412+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:45:09.518395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.002379+0000","last_change":"2024-05-20T18:37:13.990987+0000","last_active":"2024-05-20T18:37:16.002379+0000","last_peered":"2024-05-20T18:37:16.002379+0000","last_clean":"2024-05-20T18:37:16.002379+0000","last_became_active":"2024-05-20T18:37:13.990659+0000","last_became_peered":"2024-05-20T18:37:13.990659+0000","last_unstale":"2024-05-20T18:37:16.002379+0000","last_undegraded":"2024-05-20T18:37:16.002379+0000","last_fullsized":"2024-05-20T18:37:16.002379+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:00:40.463066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181004+0000","last_change":"2024-05-20T18:37:15.009130+0000","last_active":"2024-05-20T18:37:16.181004+0000","last_peered":"2024-05-20T18:37:16.181004+0000","last_clean":"2024-05-20T18:37:16.181004+0000","last_became_active":"2024-05-20T18:37:15.008519+0000","last_became_peered":"2024-05-20T18:37:15.008519+0000","last_unstale":"2024-05-20T18:37:16.181004+0000","last_undegraded":"2024-05-20T18:37:16.181004+0000","last_fullsized":"2024-05-20T18:37:16.181004+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T23:05:13.899060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.005909+0000","last_change":"2024-05-20T18:37:14.001820+0000","last_active":"2024-05-20T18:37:16.005909+0000","last_peered":"2024-05-20T18:37:16.005909+0000","last_clean":"2024-05-20T18:37:16.005909+0000","last_became_active":"2024-05-20T18:37:14.001371+0000","last_became_peered":"2024-05-20T18:37:14.001371+0000","last_unstale":"2024-05-20T18:37:16.005909+0000","last_undegraded":"2024-05-20T18:37:16.005909+0000","last_fullsized":"2024-05-20T18:37:16.005909+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:00:47.330826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180832+0000","last_change":"2024-05-20T18:37:15.009930+0000","last_active":"2024-05-20T18:37:16.180832+0000","last_peered":"2024-05-20T18:37:16.180832+0000","last_clean":"2024-05-20T18:37:16.180832+0000","last_became_active":"2024-05-20T18:37:15.009512+0000","last_became_peered":"2024-05-20T18:37:15.009512+0000","last_unstale":"2024-05-20T18:37:16.180832+0000","last_undegraded":"2024-05-20T18:37:16.180832+0000","last_fullsized":"2024-05-20T18:37:16.180832+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:59:53.017125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181075+0000","last_change":"2024-05-20T18:37:13.988195+0000","last_active":"2024-05-20T18:37:16.181075+0000","last_peered":"2024-05-20T18:37:16.181075+0000","last_clean":"2024-05-20T18:37:16.181075+0000","last_became_active":"2024-05-20T18:37:13.987879+0000","last_became_peered":"2024-05-20T18:37:13.987879+0000","last_unstale":"2024-05-20T18:37:16.181075+0000","last_undegraded":"2024-05-20T18:37:16.181075+0000","last_fullsized":"2024-05-20T18:37:16.181075+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T04:59:52.012404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181751+0000","last_change":"2024-05-20T18:37:15.008815+0000","last_active":"2024-05-20T18:37:16.181751+0000","last_peered":"2024-05-20T18:37:16.181751+0000","last_clean":"2024-05-20T18:37:16.181751+0000","last_became_active":"2024-05-20T18:37:15.008412+0000","last_became_peered":"2024-05-20T18:37:15.008412+0000","last_unstale":"2024-05-20T18:37:16.181751+0000","last_undegraded":"2024-05-20T18:37:16.181751+0000","last_fullsized":"2024-05-20T18:37:16.181751+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:13.972129+0000","last_clean_scrub_stamp":"2024-05-20T18:37:13.972129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T03:17:23.407267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"118'8","reported_seq":22,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.181276+0000","last_change":"2024-05-20T18:37:13.998153+0000","last_active":"2024-05-20T18:37:16.181276+0000","last_peered":"2024-05-20T18:37:16.181276+0000","last_clean":"2024-05-20T18:37:16.181276+0000","last_became_active":"2024-05-20T18:37:13.997841+0000","last_became_peered":"2024-05-20T18:37:13.997841+0000","last_unstale":"2024-05-20T18:37:16.181276+0000","last_undegraded":"2024-05-20T18:37:16.181276+0000","last_fullsized":"2024-05-20T18:37:16.181276+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:37:12.967408+0000","last_clean_scrub_stamp":"2024-05-20T18:37:12.967408+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T02:51:55.113103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":352,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-20T18:37:16.180971+0000","last_change":"2024-05-20T18:27:03.765694+0000","last_active":"2024-05-20T18:37:16.180971+0000","last_peered":"2024-05-20T18:37:16.180971+0000","last_clean":"2024-05-20T18:37:16.180971+0000","last_became_active":"2024-05-20T18:27:03.764695+0000","last_became_peered":"2024-05-20T18:27:03.764695+0000","last_unstale":"2024-05-20T18:37:16.180971+0000","last_undegraded":"2024-05-20T18:37:16.180971+0000","last_fullsized":"2024-05-20T18:37:16.180971+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_clean_scrub_stamp":"2024-05-20T18:27:02.750853+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:30:04.407487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574975,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33216,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338624,"statfs":{"total":96636764160,"available":96602750976,"internally_reserved":0,"allocated":2080768,"data_stored":1276374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574973,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1486848,"data_stored":692504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574973,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1478656,"data_stored":685390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574973,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32636,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1482752,"data_stored":686842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574975,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32640,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339200,"statfs":{"total":96636764160,"available":96603340800,"internally_reserved":0,"allocated":1486848,"data_stored":686876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1470464,"data_stored":685230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574975,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1474560,"data_stored":685252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574974,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33204,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338636,"statfs":{"total":96636764160,"available":96602763264,"internally_reserved":0,"allocated":2068480,"data_stored":1275620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T18:37:19.447 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-20T18:37:19.447 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-20T18:37:19.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.605+0000 7ff6f09fe640 1 Processor -- start 2024-05-20T18:37:19.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.605+0000 7ff6f09fe640 1 -- start start 2024-05-20T18:37:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.606+0000 7ff6f09fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6ec0d0340 0x7ff6ec0b6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.606+0000 7ff6f09fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff6ec0b69d0 0x7ff6ec06b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.606+0000 7ff6f09fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff6ec06ba60 0x7ff6ec0d36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.607+0000 7ff6f09fe640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7ff6ec0b8720 con 0x7ff6ec0b69d0 2024-05-20T18:37:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.607+0000 7ff6f09fe640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff6ec06be70 con 0x7ff6ec06ba60 2024-05-20T18:37:19.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.607+0000 7ff6f09fe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff6ec0b6f50 con 0x7ff6ec0d0340 2024-05-20T18:37:19.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.607+0000 7ff6ea59c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff6ec0b69d0 0x7ff6ec06b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:19.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.607+0000 7ff6ea59c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff6ec0b69d0 0x7ff6ec06b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34332/0 (socket says 172.21.15.5:34332) 2024-05-20T18:37:19.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.607+0000 7ff6ea59c640 1 -- 172.21.15.5:0/3953309179 learned_addr learned my addr 172.21.15.5:0/3953309179 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:19.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.607+0000 7ff6ead9d640 1 --2- 172.21.15.5:0/3953309179 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6ec0d0340 0x7ff6ec0b6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:19.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.607+0000 7ff6eb59e640 1 --2- 172.21.15.5:0/3953309179 >> v2:172.21.15.63:3300/0 conn(0x7ff6ec06ba60 0x7ff6ec0d36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:19.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.607+0000 7ff6ea59c640 1 -- 172.21.15.5:0/3953309179 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6ec0d0340 msgr2=0x7ff6ec0b6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:19.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.607+0000 7ff6ea59c640 1 --2- 172.21.15.5:0/3953309179 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6ec0d0340 0x7ff6ec0b6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:19.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.607+0000 7ff6ea59c640 1 -- 172.21.15.5:0/3953309179 >> v2:172.21.15.63:3300/0 conn(0x7ff6ec06ba60 msgr2=0x7ff6ec0d36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:19.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.607+0000 7ff6ea59c640 1 --2- 172.21.15.5:0/3953309179 >> v2:172.21.15.63:3300/0 conn(0x7ff6ec06ba60 0x7ff6ec0d36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:19.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.607+0000 7ff6ea59c640 1 -- 172.21.15.5:0/3953309179 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6ec0b7230 con 0x7ff6ec0b69d0 2024-05-20T18:37:19.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.608+0000 7ff6ea59c640 1 --2- 172.21.15.5:0/3953309179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6ec0b69d0 0x7ff6ec06b520 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec06dfc0 tx=0x7ff6e002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef549f33fbf168a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:19.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.608+0000 7ff6e9d9b640 1 -- 172.21.15.5:0/3953309179 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e0036070 con 0x7ff6ec0b69d0 2024-05-20T18:37:19.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.608+0000 7ff6e9d9b640 1 -- 172.21.15.5:0/3953309179 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6e0031600 con 0x7ff6ec0b69d0 2024-05-20T18:37:19.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.608+0000 7ff6e9d9b640 1 -- 172.21.15.5:0/3953309179 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e00319d0 con 0x7ff6ec0b69d0 2024-05-20T18:37:19.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.609+0000 7ff6f09fe640 1 -- 172.21.15.5:0/3953309179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6ec0b69d0 msgr2=0x7ff6ec06b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:19.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.609+0000 7ff6f09fe640 1 --2- 172.21.15.5:0/3953309179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6ec0b69d0 0x7ff6ec06b520 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec06dfc0 tx=0x7ff6e002a6a0 comp rx=0 tx=0).stop 2024-05-20T18:37:19.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.609+0000 7ff6f09fe640 1 -- 172.21.15.5:0/3953309179 shutdown_connections 2024-05-20T18:37:19.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.609+0000 7ff6f09fe640 1 --2- 172.21.15.5:0/3953309179 >> v2:172.21.15.63:3300/0 conn(0x7ff6ec06ba60 0x7ff6ec0d36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:19.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.609+0000 7ff6f09fe640 1 --2- 172.21.15.5:0/3953309179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6ec0b69d0 0x7ff6ec06b520 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:19.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.609+0000 7ff6f09fe640 1 --2- 172.21.15.5:0/3953309179 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6ec0d0340 0x7ff6ec0b6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:19.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.609+0000 7ff6f09fe640 1 -- 172.21.15.5:0/3953309179 >> 172.21.15.5:0/3953309179 conn(0x7ff6ec0d4a10 msgr2=0x7ff6ec0d4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:19.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.609+0000 7ff6f09fe640 1 -- 172.21.15.5:0/3953309179 shutdown_connections 2024-05-20T18:37:19.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.609+0000 7ff6f09fe640 1 -- 172.21.15.5:0/3953309179 wait complete. 2024-05-20T18:37:19.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.611+0000 7ff6f09fe640 1 Processor -- start 2024-05-20T18:37:19.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.611+0000 7ff6f09fe640 1 -- start start 2024-05-20T18:37:19.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.612+0000 7ff6f09fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6ec06ba60 0x7ff6ec0b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:19.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.612+0000 7ff6f09fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6ec0b69d0 0x7ff6ec0cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:19.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.612+0000 7ff6f09fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6ec0d0340 0x7ff6ec0ce080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:19.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.612+0000 7ff6f09fe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec0b7fd0 con 0x7ff6ec06ba60 2024-05-20T18:37:19.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.612+0000 7ff6f09fe640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec0d2f60 con 0x7ff6ec0d0340 2024-05-20T18:37:19.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.612+0000 7ff6f09fe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff6ec0b8150 con 0x7ff6ec0b69d0 2024-05-20T18:37:19.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.612+0000 7ff6ead9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6ec06ba60 0x7ff6ec0b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:19.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.612+0000 7ff6ead9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6ec06ba60 0x7ff6ec0b2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34348/0 (socket says 172.21.15.5:34348) 2024-05-20T18:37:19.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.612+0000 7ff6ead9d640 1 -- 172.21.15.5:0/4010669280 learned_addr learned my addr 172.21.15.5:0/4010669280 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:19.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.613+0000 7ff6ea59c640 1 --2- 172.21.15.5:0/4010669280 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6ec0b69d0 0x7ff6ec0cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.613+0000 7ff6eb59e640 1 --2- 172.21.15.5:0/4010669280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6ec0d0340 0x7ff6ec0ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.613+0000 7ff6ead9d640 1 -- 172.21.15.5:0/4010669280 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6ec0b69d0 msgr2=0x7ff6ec0cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.613+0000 7ff6ead9d640 1 --2- 172.21.15.5:0/4010669280 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6ec0b69d0 0x7ff6ec0cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.613+0000 7ff6ead9d640 1 -- 172.21.15.5:0/4010669280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6ec0d0340 msgr2=0x7ff6ec0ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.613+0000 7ff6ead9d640 1 --2- 172.21.15.5:0/4010669280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6ec0d0340 0x7ff6ec0ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.613+0000 7ff6ead9d640 1 -- 172.21.15.5:0/4010669280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6ec1e76e0 con 0x7ff6ec06ba60 2024-05-20T18:37:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.613+0000 7ff6ea59c640 1 --2- 172.21.15.5:0/4010669280 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6ec0b69d0 0x7ff6ec0cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.613+0000 7ff6ead9d640 1 --2- 172.21.15.5:0/4010669280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6ec06ba60 0x7ff6ec0b2d20 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc00e8d0 tx=0x7ff6dc00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.613+0000 7ff6d37fe640 1 -- 172.21.15.5:0/4010669280 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc017070 con 0x7ff6ec06ba60 2024-05-20T18:37:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.613+0000 7ff6d37fe640 1 -- 172.21.15.5:0/4010669280 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6dc00c940 con 0x7ff6ec06ba60 2024-05-20T18:37:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.613+0000 7ff6d37fe640 1 -- 172.21.15.5:0/4010669280 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc00cd30 con 0x7ff6ec06ba60 2024-05-20T18:37:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.613+0000 7ff6f09fe640 1 -- 172.21.15.5:0/4010669280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6ec14b080 con 0x7ff6ec06ba60 2024-05-20T18:37:19.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.613+0000 7ff6f09fe640 1 -- 172.21.15.5:0/4010669280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6ec14b600 con 0x7ff6ec06ba60 2024-05-20T18:37:19.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.614+0000 7ff6f09fe640 1 -- 172.21.15.5:0/4010669280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6ec075980 con 0x7ff6ec06ba60 2024-05-20T18:37:19.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.615+0000 7ff6d37fe640 1 -- 172.21.15.5:0/4010669280 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff6dc01a070 con 0x7ff6ec06ba60 2024-05-20T18:37:19.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.615+0000 7ff6d37fe640 1 --2- 172.21.15.5:0/4010669280 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff6d806be60 0x7ff6d806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:19.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.616+0000 7ff6d37fe640 1 -- 172.21.15.5:0/4010669280 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7ff6dc08ae50 con 0x7ff6ec06ba60 2024-05-20T18:37:19.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.616+0000 7ff6ea59c640 1 --2- 172.21.15.5:0/4010669280 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff6d806be60 0x7ff6d806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:19.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.616+0000 7ff6ea59c640 1 --2- 172.21.15.5:0/4010669280 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff6d806be60 0x7ff6d806e370 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0033000 tx=0x7ff6e0009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:19.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.620+0000 7ff6d37fe640 1 -- 172.21.15.5:0/4010669280 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff6dc059050 con 0x7ff6ec06ba60 2024-05-20T18:37:19.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.875+0000 7ff6f09fe640 1 -- 172.21.15.5:0/4010669280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7ff6ec074e00 con 0x7ff6ec06ba60 2024-05-20T18:37:19.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.878+0000 7ff6d37fe640 1 -- 172.21.15.5:0/4010669280 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7ff6dc05d350 con 0x7ff6ec06ba60 2024-05-20T18:37:19.880 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2024-05-20T18:37:19.880 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDFlUtmevaaIhAATIwyXzr0vGT38ZmcMqR2gA== 2024-05-20T18:37:19.880 INFO:teuthology.orchestra.run.smithi005.stdout: caps mds = "allow" 2024-05-20T18:37:19.880 INFO:teuthology.orchestra.run.smithi005.stdout: caps mon = "allow r" 2024-05-20T18:37:19.880 INFO:teuthology.orchestra.run.smithi005.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-20T18:37:19.880 INFO:teuthology.orchestra.run.smithi005.stderr:updated caps for client.0 2024-05-20T18:37:19.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.880+0000 7ff6f09fe640 1 -- 172.21.15.5:0/4010669280 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff6d806be60 msgr2=0x7ff6d806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:19.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.880+0000 7ff6f09fe640 1 --2- 172.21.15.5:0/4010669280 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff6d806be60 0x7ff6d806e370 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0033000 tx=0x7ff6e0009000 comp rx=0 tx=0).stop 2024-05-20T18:37:19.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.880+0000 7ff6f09fe640 1 -- 172.21.15.5:0/4010669280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6ec06ba60 msgr2=0x7ff6ec0b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:19.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.880+0000 7ff6f09fe640 1 --2- 172.21.15.5:0/4010669280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6ec06ba60 0x7ff6ec0b2d20 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc00e8d0 tx=0x7ff6dc00ed90 comp rx=0 tx=0).stop 2024-05-20T18:37:19.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.880+0000 7ff6f09fe640 1 -- 172.21.15.5:0/4010669280 shutdown_connections 2024-05-20T18:37:19.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.880+0000 7ff6f09fe640 1 --2- 172.21.15.5:0/4010669280 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff6d806be60 0x7ff6d806e370 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:19.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.880+0000 7ff6f09fe640 1 --2- 172.21.15.5:0/4010669280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6ec0d0340 0x7ff6ec0ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:19.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.880+0000 7ff6f09fe640 1 --2- 172.21.15.5:0/4010669280 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff6ec0b69d0 0x7ff6ec0cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:19.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.880+0000 7ff6f09fe640 1 --2- 172.21.15.5:0/4010669280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6ec06ba60 0x7ff6ec0b2d20 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:19.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.880+0000 7ff6f09fe640 1 -- 172.21.15.5:0/4010669280 >> 172.21.15.5:0/4010669280 conn(0x7ff6ec0d4a10 msgr2=0x7ff6ec15f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:19.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.880+0000 7ff6f09fe640 1 -- 172.21.15.5:0/4010669280 shutdown_connections 2024-05-20T18:37:19.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:19.880+0000 7ff6f09fe640 1 -- 172.21.15.5:0/4010669280 wait complete. 2024-05-20T18:37:19.893 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:37:20.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.006+0000 7feffba73640 1 Processor -- start 2024-05-20T18:37:20.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.006+0000 7feffba73640 1 -- start start 2024-05-20T18:37:20.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.008+0000 7feffba73640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7feff415f220 0x7feff40cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.008+0000 7feffba73640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feff40cf910 0x7feff40d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.008+0000 7feffba73640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feff40cfd20 con 0x7feff40cd680 2024-05-20T18:37:20.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.008+0000 7feffba73640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7feff40cda90 con 0x7feff415f220 2024-05-20T18:37:20.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.008+0000 7feffba73640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7feff406d0f0 con 0x7feff40cf910 2024-05-20T18:37:20.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.008+0000 7feff900e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7feff415f220 0x7feff40cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.008+0000 7feff900e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7feff415f220 0x7feff40cf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34358/0 (socket says 172.21.15.5:34358) 2024-05-20T18:37:20.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.008+0000 7feff900e640 1 -- 172.21.15.5:0/3518503197 learned_addr learned my addr 172.21.15.5:0/3518503197 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:20.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.008+0000 7feffa010640 1 --2- 172.21.15.5:0/3518503197 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feff40cf910 0x7feff40d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.008+0000 7feff900e640 1 -- 172.21.15.5:0/3518503197 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feff40cf910 msgr2=0x7feff40d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:20.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.008+0000 7feff900e640 1 --2- 172.21.15.5:0/3518503197 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feff40cf910 0x7feff40d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.008+0000 7feff900e640 1 -- 172.21.15.5:0/3518503197 >> v1:172.21.15.63:6789/0 conn(0x7feff40cd680 legacy=0x7feff406c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:20.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.008+0000 7feff900e640 1 -- 172.21.15.5:0/3518503197 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff406d3d0 con 0x7feff415f220 2024-05-20T18:37:20.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.008+0000 7feffa010640 1 --2- 172.21.15.5:0/3518503197 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feff40cf910 0x7feff40d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:20.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.009+0000 7feff900e640 1 --2- 172.21.15.5:0/3518503197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feff415f220 0x7feff40cf3d0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fefe4009870 tx=0x7fefe402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a7cd90f6b65b98 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:20.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.009+0000 7feff880d640 1 -- 172.21.15.5:0/3518503197 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe4037070 con 0x7feff415f220 2024-05-20T18:37:20.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.009+0000 7feff880d640 1 -- 172.21.15.5:0/3518503197 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefe4032440 con 0x7feff415f220 2024-05-20T18:37:20.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.009+0000 7feff880d640 1 -- 172.21.15.5:0/3518503197 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe4032810 con 0x7feff415f220 2024-05-20T18:37:20.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.010+0000 7feffba73640 1 -- 172.21.15.5:0/3518503197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feff415f220 msgr2=0x7feff40cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:20.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.010+0000 7feffba73640 1 --2- 172.21.15.5:0/3518503197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feff415f220 0x7feff40cf3d0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fefe4009870 tx=0x7fefe402a180 comp rx=0 tx=0).stop 2024-05-20T18:37:20.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.010+0000 7feffba73640 1 -- 172.21.15.5:0/3518503197 shutdown_connections 2024-05-20T18:37:20.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.010+0000 7feffba73640 1 --2- 172.21.15.5:0/3518503197 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feff40cf910 0x7feff40d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.010+0000 7feffba73640 1 --2- 172.21.15.5:0/3518503197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feff415f220 0x7feff40cf3d0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.010+0000 7feffba73640 1 -- 172.21.15.5:0/3518503197 >> 172.21.15.5:0/3518503197 conn(0x7feff40b44d0 msgr2=0x7feff40b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:20.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.010+0000 7feffba73640 1 -- 172.21.15.5:0/3518503197 shutdown_connections 2024-05-20T18:37:20.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.010+0000 7feffba73640 1 -- 172.21.15.5:0/3518503197 wait complete. 2024-05-20T18:37:20.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.012+0000 7feffba73640 1 Processor -- start 2024-05-20T18:37:20.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.012+0000 7feffba73640 1 -- start start 2024-05-20T18:37:20.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.013+0000 7feffba73640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feff40cd680 0x7feff4152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.013+0000 7feffba73640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feff40cf910 0x7feff40b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.013+0000 7feffba73640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feff415f220 0x7feff40b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.013+0000 7feffba73640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feff406d990 con 0x7feff415f220 2024-05-20T18:37:20.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.013+0000 7feffba73640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feff40d0b40 con 0x7feff40cf910 2024-05-20T18:37:20.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.013+0000 7feffba73640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7feff406cdd0 con 0x7feff40cd680 2024-05-20T18:37:20.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.013+0000 7feffa010640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feff415f220 0x7feff40b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.013+0000 7feffa010640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feff415f220 0x7feff40b5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34368/0 (socket says 172.21.15.5:34368) 2024-05-20T18:37:20.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.013+0000 7feffa010640 1 -- 172.21.15.5:0/649031469 learned_addr learned my addr 172.21.15.5:0/649031469 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:20.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.013+0000 7feff980f640 1 --2- 172.21.15.5:0/649031469 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feff40cd680 0x7feff4152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.013+0000 7feff900e640 1 --2- 172.21.15.5:0/649031469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feff40cf910 0x7feff40b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.014+0000 7feffa010640 1 -- 172.21.15.5:0/649031469 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feff40cd680 msgr2=0x7feff4152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:20.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.014+0000 7feffa010640 1 --2- 172.21.15.5:0/649031469 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feff40cd680 0x7feff4152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.014+0000 7feffa010640 1 -- 172.21.15.5:0/649031469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feff40cf910 msgr2=0x7feff40b55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:20.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.014+0000 7feffa010640 1 --2- 172.21.15.5:0/649031469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feff40cf910 0x7feff40b55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.014+0000 7feffa010640 1 -- 172.21.15.5:0/649031469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff40b6cc0 con 0x7feff415f220 2024-05-20T18:37:20.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.014+0000 7feffa010640 1 --2- 172.21.15.5:0/649031469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feff415f220 0x7feff40b5af0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fefec00cd50 tx=0x7fefec00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:20.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.014+0000 7fefea7fc640 1 -- 172.21.15.5:0/649031469 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec00f680 con 0x7feff415f220 2024-05-20T18:37:20.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.014+0000 7fefea7fc640 1 -- 172.21.15.5:0/649031469 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefec00f840 con 0x7feff415f220 2024-05-20T18:37:20.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.014+0000 7fefea7fc640 1 -- 172.21.15.5:0/649031469 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec00fc30 con 0x7feff415f220 2024-05-20T18:37:20.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.014+0000 7feffba73640 1 -- 172.21.15.5:0/649031469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff41f6420 con 0x7feff415f220 2024-05-20T18:37:20.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.014+0000 7feffba73640 1 -- 172.21.15.5:0/649031469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff40b7210 con 0x7feff415f220 2024-05-20T18:37:20.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.015+0000 7feffba73640 1 -- 172.21.15.5:0/649031469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feff4075b00 con 0x7feff415f220 2024-05-20T18:37:20.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.016+0000 7fefea7fc640 1 -- 172.21.15.5:0/649031469 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fefec0213f0 con 0x7feff415f220 2024-05-20T18:37:20.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.016+0000 7fefea7fc640 1 --2- 172.21.15.5:0/649031469 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fefd806be60 0x7fefd806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.016+0000 7fefea7fc640 1 -- 172.21.15.5:0/649031469 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7fefec08bc40 con 0x7feff415f220 2024-05-20T18:37:20.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.016+0000 7feff980f640 1 --2- 172.21.15.5:0/649031469 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fefd806be60 0x7fefd806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.017+0000 7feff980f640 1 --2- 172.21.15.5:0/649031469 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fefd806be60 0x7fefd806e370 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7feff0006fd0 tx=0x7feff0009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:20.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.021+0000 7fefea7fc640 1 -- 172.21.15.5:0/649031469 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fefec014030 con 0x7feff415f220 2024-05-20T18:37:20.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.243+0000 7feffba73640 1 -- 172.21.15.5:0/649031469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feff41f65f0 con 0x7feff415f220 2024-05-20T18:37:20.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.243+0000 7fefea7fc640 1 -- 172.21.15.5:0/649031469 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+5174 (secure 0 0 0) 0x7fefec059e10 con 0x7feff415f220 2024-05-20T18:37:20.244 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:20.245 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":98,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/266630184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":266630184},{"type":"v1","addr":"172.21.15.63:6834","nonce":266630184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/1739221946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1739221946},{"type":"v1","addr":"172.21.15.5:6836","nonce":1739221946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9012,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4273429943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":4273429943},{"type":"v1","addr":"172.21.15.5:6837","nonce":4273429943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:37:14.365195+0000","modified":"2024-05-20T18:37:17.398193+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9030},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9030":{"gid":9030,"name":"d","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6835/2352520472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":2352520472},{"type":"v1","addr":"172.21.15.63:6835","nonce":2352520472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9030,"qdb_cluster":[9030]},"id":8}]} 2024-05-20T18:37:20.245 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 98 2024-05-20T18:37:20.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.245+0000 7feffba73640 1 -- 172.21.15.5:0/649031469 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fefd806be60 msgr2=0x7fefd806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:20.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.245+0000 7feffba73640 1 --2- 172.21.15.5:0/649031469 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fefd806be60 0x7fefd806e370 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7feff0006fd0 tx=0x7feff0009060 comp rx=0 tx=0).stop 2024-05-20T18:37:20.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.245+0000 7feffba73640 1 -- 172.21.15.5:0/649031469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feff415f220 msgr2=0x7feff40b5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:20.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.245+0000 7feffba73640 1 --2- 172.21.15.5:0/649031469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feff415f220 0x7feff40b5af0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fefec00cd50 tx=0x7fefec00a650 comp rx=0 tx=0).stop 2024-05-20T18:37:20.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.245+0000 7feffba73640 1 -- 172.21.15.5:0/649031469 shutdown_connections 2024-05-20T18:37:20.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.245+0000 7feffba73640 1 --2- 172.21.15.5:0/649031469 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fefd806be60 0x7fefd806e370 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.245+0000 7feffba73640 1 --2- 172.21.15.5:0/649031469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feff415f220 0x7feff40b5af0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.245+0000 7feffba73640 1 --2- 172.21.15.5:0/649031469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feff40cf910 0x7feff40b55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.245+0000 7feffba73640 1 --2- 172.21.15.5:0/649031469 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feff40cd680 0x7feff4152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.245+0000 7feffba73640 1 -- 172.21.15.5:0/649031469 >> 172.21.15.5:0/649031469 conn(0x7feff40b44d0 msgr2=0x7feff40cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:20.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.245+0000 7feffba73640 1 -- 172.21.15.5:0/649031469 shutdown_connections 2024-05-20T18:37:20.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.245+0000 7feffba73640 1 -- 172.21.15.5:0/649031469 wait complete. 2024-05-20T18:37:20.660 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 98, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T18:37:14.365195+0000', 'modified': '2024-05-20T18:37:17.398193+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9030}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9030': {'gid': 9030, 'name': 'd', 'rank': 0, 'incarnation': 95, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.63:6835/2352520472', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6833', 'nonce': 2352520472}, {'type': 'v1', 'addr': '172.21.15.63:6835', 'nonce': 2352520472}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9030, 'qdb_cluster': [9030]} 2024-05-20T18:37:20.661 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-20T18:37:20.661 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-20T18:37:20.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.780+0000 7f4c93432640 1 Processor -- start 2024-05-20T18:37:20.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.780+0000 7f4c93432640 1 -- start start 2024-05-20T18:37:20.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.782+0000 7f4c93432640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4c8c0d10e0 0x7f4c8c0cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.782+0000 7f4c93432640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c8c0d14f0 0x7f4c8c0cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.782+0000 7f4c93432640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c8c0d1ac0 con 0x7f4c8c0cd4d0 2024-05-20T18:37:20.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.782+0000 7f4c93432640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f4c8c0d1da0 con 0x7f4c8c0d10e0 2024-05-20T18:37:20.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.782+0000 7f4c93432640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4c8c0d2080 con 0x7f4c8c0d14f0 2024-05-20T18:37:20.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.782+0000 7f4c911ce640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4c8c0d10e0 0x7f4c8c0cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.782+0000 7f4c911ce640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4c8c0d10e0 0x7f4c8c0cf140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34374/0 (socket says 172.21.15.5:34374) 2024-05-20T18:37:20.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.782+0000 7f4c911ce640 1 -- 172.21.15.5:0/4245038106 learned_addr learned my addr 172.21.15.5:0/4245038106 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:20.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.782+0000 7f4c911ce640 1 -- 172.21.15.5:0/4245038106 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c8c0d14f0 msgr2=0x7f4c8c0cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:20.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.782+0000 7f4c909cd640 1 --2- 172.21.15.5:0/4245038106 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c8c0d14f0 0x7f4c8c0cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.782+0000 7f4c911ce640 1 --2- 172.21.15.5:0/4245038106 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c8c0d14f0 0x7f4c8c0cf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.782+0000 7f4c911ce640 1 -- 172.21.15.5:0/4245038106 >> v1:172.21.15.63:6789/0 conn(0x7f4c8c0cd4d0 legacy=0x7f4c8c0cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:20.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.782+0000 7f4c911ce640 1 -- 172.21.15.5:0/4245038106 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c8c0d2360 con 0x7f4c8c0d10e0 2024-05-20T18:37:20.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.783+0000 7f4c911ce640 1 --2- 172.21.15.5:0/4245038106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c8c0d10e0 0x7f4c8c0cf140 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f4c88009870 tx=0x7f4c8802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a664b95da5ca05 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:20.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.783+0000 7f4c83fff640 1 -- 172.21.15.5:0/4245038106 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c88037070 con 0x7f4c8c0d10e0 2024-05-20T18:37:20.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.783+0000 7f4c83fff640 1 -- 172.21.15.5:0/4245038106 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c88032440 con 0x7f4c8c0d10e0 2024-05-20T18:37:20.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.783+0000 7f4c83fff640 1 -- 172.21.15.5:0/4245038106 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c88032810 con 0x7f4c8c0d10e0 2024-05-20T18:37:20.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.784+0000 7f4c93432640 1 -- 172.21.15.5:0/4245038106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c8c0d10e0 msgr2=0x7f4c8c0cf140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:20.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.784+0000 7f4c93432640 1 --2- 172.21.15.5:0/4245038106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c8c0d10e0 0x7f4c8c0cf140 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f4c88009870 tx=0x7f4c8802a210 comp rx=0 tx=0).stop 2024-05-20T18:37:20.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.784+0000 7f4c93432640 1 -- 172.21.15.5:0/4245038106 shutdown_connections 2024-05-20T18:37:20.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.784+0000 7f4c93432640 1 --2- 172.21.15.5:0/4245038106 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c8c0d14f0 0x7f4c8c0cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.784+0000 7f4c93432640 1 --2- 172.21.15.5:0/4245038106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c8c0d10e0 0x7f4c8c0cf140 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.784+0000 7f4c93432640 1 -- 172.21.15.5:0/4245038106 >> 172.21.15.5:0/4245038106 conn(0x7f4c8c0b44d0 msgr2=0x7f4c8c0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:20.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.784+0000 7f4c93432640 1 -- 172.21.15.5:0/4245038106 shutdown_connections 2024-05-20T18:37:20.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.784+0000 7f4c93432640 1 -- 172.21.15.5:0/4245038106 wait complete. 2024-05-20T18:37:20.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.786+0000 7f4c93432640 1 Processor -- start 2024-05-20T18:37:20.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.786+0000 7f4c93432640 1 -- start start 2024-05-20T18:37:20.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.787+0000 7f4c93432640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c8c0cd4d0 0x7f4c8c158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.787+0000 7f4c93432640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c8c0d10e0 0x7f4c8c1602d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.787+0000 7f4c93432640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c8c0d14f0 0x7f4c8c0c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.787+0000 7f4c93432640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c0b8440 con 0x7f4c8c0cd4d0 2024-05-20T18:37:20.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.787+0000 7f4c93432640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c0b82c0 con 0x7f4c8c0d10e0 2024-05-20T18:37:20.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.787+0000 7f4c93432640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4c8c0709f0 con 0x7f4c8c0d14f0 2024-05-20T18:37:20.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.787+0000 7f4c911ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c8c0cd4d0 0x7f4c8c158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.787+0000 7f4c911ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c8c0cd4d0 0x7f4c8c158c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34384/0 (socket says 172.21.15.5:34384) 2024-05-20T18:37:20.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.787+0000 7f4c911ce640 1 -- 172.21.15.5:0/3323247381 learned_addr learned my addr 172.21.15.5:0/3323247381 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:20.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.787+0000 7f4c909cd640 1 --2- 172.21.15.5:0/3323247381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c8c0d10e0 0x7f4c8c1602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.787+0000 7f4c919cf640 1 --2- 172.21.15.5:0/3323247381 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c8c0d14f0 0x7f4c8c0c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.788+0000 7f4c911ce640 1 -- 172.21.15.5:0/3323247381 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c8c0d14f0 msgr2=0x7f4c8c0c8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:20.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.788+0000 7f4c911ce640 1 --2- 172.21.15.5:0/3323247381 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c8c0d14f0 0x7f4c8c0c8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.788+0000 7f4c911ce640 1 -- 172.21.15.5:0/3323247381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c8c0d10e0 msgr2=0x7f4c8c1602d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:20.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.788+0000 7f4c911ce640 1 --2- 172.21.15.5:0/3323247381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c8c0d10e0 0x7f4c8c1602d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.788+0000 7f4c911ce640 1 -- 172.21.15.5:0/3323247381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c8c1f8f50 con 0x7f4c8c0cd4d0 2024-05-20T18:37:20.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.788+0000 7f4c909cd640 1 --2- 172.21.15.5:0/3323247381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c8c0d10e0 0x7f4c8c1602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:20.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.788+0000 7f4c911ce640 1 --2- 172.21.15.5:0/3323247381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c8c0cd4d0 0x7f4c8c158c30 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f4c88009870 tx=0x7f4c880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:20.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.788+0000 7f4c81ffb640 1 -- 172.21.15.5:0/3323247381 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c88037040 con 0x7f4c8c0cd4d0 2024-05-20T18:37:20.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.788+0000 7f4c81ffb640 1 -- 172.21.15.5:0/3323247381 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c88042070 con 0x7f4c8c0cd4d0 2024-05-20T18:37:20.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.788+0000 7f4c93432640 1 -- 172.21.15.5:0/3323247381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c8c1f9230 con 0x7f4c8c0cd4d0 2024-05-20T18:37:20.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.788+0000 7f4c81ffb640 1 -- 172.21.15.5:0/3323247381 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c88030bb0 con 0x7f4c8c0cd4d0 2024-05-20T18:37:20.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.788+0000 7f4c93432640 1 -- 172.21.15.5:0/3323247381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c8c0cbc30 con 0x7f4c8c0cd4d0 2024-05-20T18:37:20.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.790+0000 7f4c81ffb640 1 -- 172.21.15.5:0/3323247381 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4c88047460 con 0x7f4c8c0cd4d0 2024-05-20T18:37:20.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.791+0000 7f4c81ffb640 1 --2- 172.21.15.5:0/3323247381 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4c7006be30 0x7f4c7006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.791+0000 7f4c81ffb640 1 -- 172.21.15.5:0/3323247381 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7f4c880abcd0 con 0x7f4c8c0cd4d0 2024-05-20T18:37:20.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.791+0000 7f4c909cd640 1 --2- 172.21.15.5:0/3323247381 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4c7006be30 0x7f4c7006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.791+0000 7f4c909cd640 1 --2- 172.21.15.5:0/3323247381 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4c7006be30 0x7f4c7006e340 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c006fd0 tx=0x7f4c7c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:20.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.792+0000 7f4c9a3bb740 1 Processor -- start 2024-05-20T18:37:20.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.792+0000 7f4c9a3bb740 1 -- start start 2024-05-20T18:37:20.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.793+0000 7f4c9a3bb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56254cc37270 0x56254d002700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.793+0000 7f4c9a3bb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56254cc0e3b0 0x56254cfff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.793+0000 7f4c9a3bb740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56254cc16590 0x56254cfd8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.793+0000 7f4c9a3bb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56254cf3b4d0 con 0x56254cc37270 2024-05-20T18:37:20.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.793+0000 7f4c9a3bb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56254cee33e0 con 0x56254cc0e3b0 2024-05-20T18:37:20.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.793+0000 7f4c9a3bb740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x56254cf37490 con 0x56254cc16590 2024-05-20T18:37:20.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.793+0000 7f4c911ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56254cc37270 0x56254d002700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.793+0000 7f4c911ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56254cc37270 0x56254d002700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34394/0 (socket says 172.21.15.5:34394) 2024-05-20T18:37:20.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.793+0000 7f4c911ce640 1 -- 172.21.15.5:0/1320527384 learned_addr learned my addr 172.21.15.5:0/1320527384 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:20.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.793+0000 7f4c909cd640 1 --2- 172.21.15.5:0/1320527384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56254cc0e3b0 0x56254cfff0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.793+0000 7f4c919cf640 1 --2- 172.21.15.5:0/1320527384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56254cc16590 0x56254cfd8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.793+0000 7f4c911ce640 1 -- 172.21.15.5:0/1320527384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56254cc16590 msgr2=0x56254cfd8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:20.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.793+0000 7f4c911ce640 1 --2- 172.21.15.5:0/1320527384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56254cc16590 0x56254cfd8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.793+0000 7f4c911ce640 1 -- 172.21.15.5:0/1320527384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56254cc0e3b0 msgr2=0x56254cfff0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:20.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.793+0000 7f4c911ce640 1 --2- 172.21.15.5:0/1320527384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56254cc0e3b0 0x56254cfff0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.793+0000 7f4c911ce640 1 -- 172.21.15.5:0/1320527384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56254cc4f2d0 con 0x56254cc37270 2024-05-20T18:37:20.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.793+0000 7f4c909cd640 1 --2- 172.21.15.5:0/1320527384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56254cc0e3b0 0x56254cfff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:20.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.794+0000 7f4c911ce640 1 --2- 172.21.15.5:0/1320527384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56254cc37270 0x56254d002700 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f4c880a9f40 tx=0x7f4c880797e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9bc4bd0eb3d9f4e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:20.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.794+0000 7f4c76ffd640 1 -- 172.21.15.5:0/1320527384 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c88082930 con 0x56254cc37270 2024-05-20T18:37:20.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.794+0000 7f4c76ffd640 1 -- 172.21.15.5:0/1320527384 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c88078050 con 0x56254cc37270 2024-05-20T18:37:20.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.794+0000 7f4c76ffd640 1 -- 172.21.15.5:0/1320527384 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c880763c0 con 0x56254cc37270 2024-05-20T18:37:20.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.794+0000 7f4c9a3bb740 1 -- 172.21.15.5:0/1320527384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56254cc37270 msgr2=0x56254d002700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:20.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.794+0000 7f4c9a3bb740 1 --2- 172.21.15.5:0/1320527384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56254cc37270 0x56254d002700 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f4c880a9f40 tx=0x7f4c880797e0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.795+0000 7f4c9a3bb740 1 -- 172.21.15.5:0/1320527384 shutdown_connections 2024-05-20T18:37:20.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.795+0000 7f4c9a3bb740 1 --2- 172.21.15.5:0/1320527384 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56254cc16590 0x56254cfd8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.795+0000 7f4c9a3bb740 1 --2- 172.21.15.5:0/1320527384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56254cc0e3b0 0x56254cfff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.795+0000 7f4c9a3bb740 1 --2- 172.21.15.5:0/1320527384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56254cc37270 0x56254d002700 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.795+0000 7f4c9a3bb740 1 -- 172.21.15.5:0/1320527384 >> 172.21.15.5:0/1320527384 conn(0x56254cc2b3c0 msgr2=0x56254cfd7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:20.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.795+0000 7f4c9a3bb740 1 -- 172.21.15.5:0/1320527384 shutdown_connections 2024-05-20T18:37:20.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.795+0000 7f4c9a3bb740 1 -- 172.21.15.5:0/1320527384 wait complete. 2024-05-20T18:37:20.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.795+0000 7f4c9a3bb740 1 -- start start 2024-05-20T18:37:20.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.796+0000 7f4c9a3bb740 1 Processor -- start 2024-05-20T18:37:20.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.797+0000 7f4c9a3bb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56254cc0e3b0 0x56254cffcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.797+0000 7f4c9a3bb740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56254cc16590 0x56254d01b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.797+0000 7f4c9a3bb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56254cc37270 0x56254d01b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.797+0000 7f4c9a3bb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56254cf3b4d0 con 0x56254cc37270 2024-05-20T18:37:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.797+0000 7f4c9a3bb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56254cef68e0 con 0x56254cc0e3b0 2024-05-20T18:37:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.797+0000 7f4c9a3bb740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x56254d01c7d0 con 0x56254cc16590 2024-05-20T18:37:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.797+0000 7f4c9a3bb740 10 client.0 _ll_register_callbacks cb 0x56254cec39c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:37:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.797+0000 7f4c919cf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56254cc37270 0x56254d01b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.797+0000 7f4c919cf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56254cc37270 0x56254d01b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34400/0 (socket says 172.21.15.5:34400) 2024-05-20T18:37:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.797+0000 7f4c919cf640 1 -- 172.21.15.5:0/945664059 learned_addr learned my addr 172.21.15.5:0/945664059 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.797+0000 7f4c911ce640 1 --2- 172.21.15.5:0/945664059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56254cc0e3b0 0x56254cffcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.797+0000 7f4c909cd640 1 --2- 172.21.15.5:0/945664059 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56254cc16590 0x56254d01b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.797+0000 7f4c919cf640 1 -- 172.21.15.5:0/945664059 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56254cc16590 msgr2=0x56254d01b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.797+0000 7f4c919cf640 1 --2- 172.21.15.5:0/945664059 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56254cc16590 0x56254d01b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.797+0000 7f4c919cf640 1 -- 172.21.15.5:0/945664059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56254cc0e3b0 msgr2=0x56254cffcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:20.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.797+0000 7f4c919cf640 1 --2- 172.21.15.5:0/945664059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56254cc0e3b0 0x56254cffcdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.798+0000 7f4c919cf640 1 -- 172.21.15.5:0/945664059 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56254cd196b0 con 0x56254cc37270 2024-05-20T18:37:20.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.798+0000 7f4c911ce640 1 --2- 172.21.15.5:0/945664059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56254cc0e3b0 0x56254cffcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:20.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.798+0000 7f4c919cf640 1 --2- 172.21.15.5:0/945664059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56254cc37270 0x56254d01b9d0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400cab0 tx=0x7f4c8400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1822097866184f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:20.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.798+0000 7f4c6bfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:37:20.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.798+0000 7f4c6bfff640 1 -- 172.21.15.5:0/945664059 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c84009d20 con 0x56254cc37270 2024-05-20T18:37:20.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.798+0000 7f4c6bfff640 1 -- 172.21.15.5:0/945664059 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7f4c840153f0 con 0x56254cc37270 2024-05-20T18:37:20.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.798+0000 7f4c9a3bb740 1 -- 172.21.15.5:0/945664059 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c88078050 con 0x56254cc37270 2024-05-20T18:37:20.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.798+0000 7f4c6bfff640 1 -- 172.21.15.5:0/945664059 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f4c84021050 con 0x56254cc37270 2024-05-20T18:37:20.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.798+0000 7f4c9a3bb740 10 client.9156 fetch_fsmap learned FSMap version 98 2024-05-20T18:37:20.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.798+0000 7f4c9a3bb740 1 -- 172.21.15.5:0/945664059 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) -- 0x56254d01afd0 con 0x56254cc37270 2024-05-20T18:37:20.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.798+0000 7f4c6bfff640 1 -- 172.21.15.5:0/945664059 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c88078050 con 0x56254cc37270 2024-05-20T18:37:20.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.798+0000 7f4c6bfff640 1 -- 172.21.15.5:0/945664059 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 98) ==== 3494+0+0 (secure 0 0 0) 0x7f4c84024020 con 0x56254cc37270 2024-05-20T18:37:20.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.798+0000 7f4c9a3bb740 10 client.9156 fetch_fsmap finished waiting for FSMap version 98 2024-05-20T18:37:20.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.798+0000 7f4c9a3bb740 10 client.9156 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-20T18:37:20.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.798+0000 7f4c9a3bb740 20 client.9156 populate_metadata read hostname 'smithi005' 2024-05-20T18:37:20.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.798+0000 7f4c9a3bb740 1 --2- 172.21.15.5:0/945664059 >> [v2:172.21.15.63:6833/2352520472,v1:172.21.15.63:6835/2352520472] conn(0x56254cbf5e40 0x56254d039cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.798+0000 7f4c9a3bb740 4 client.9156 mds_command: new command op to 9030 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:37:20.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.798+0000 7f4c9a3bb740 1 -- 172.21.15.5:0/945664059 --> [v2:172.21.15.63:6833/2352520472,v1:172.21.15.63:6835/2352520472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56254cf48c00 con 0x56254cbf5e40 2024-05-20T18:37:20.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.799+0000 7f4c911ce640 1 --2- 172.21.15.5:0/945664059 >> [v2:172.21.15.63:6833/2352520472,v1:172.21.15.63:6835/2352520472] conn(0x56254cbf5e40 0x56254d039cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.799+0000 7f4c6bfff640 10 client.9156 ms_handle_connect on v2:172.21.15.63:6833/2352520472 2024-05-20T18:37:20.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.799+0000 7f4c911ce640 1 --2- 172.21.15.5:0/945664059 >> [v2:172.21.15.63:6833/2352520472,v1:172.21.15.63:6835/2352520472] conn(0x56254cbf5e40 0x56254d039cf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba828684b493bdf3 server_cookie=b78fd2ce287564ac in_seq=0 out_seq=0 2024-05-20T18:37:20.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.800+0000 7f4c6bfff640 1 -- 172.21.15.5:0/945664059 <== mds.0 v2:172.21.15.63:6833/2352520472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56254cf48c00 con 0x56254cbf5e40 2024-05-20T18:37:20.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.800+0000 7f4c6bfff640 10 client.9156 handle_command_reply: tid=0 2024-05-20T18:37:20.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.800+0000 7f4c9a3bb740 1 client.9156 shutdown 2024-05-20T18:37:20.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.800+0000 7f4c9a3bb740 1 -- 172.21.15.5:0/945664059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56254cc37270 msgr2=0x56254d01b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:20.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.800+0000 7f4c9a3bb740 1 --2- 172.21.15.5:0/945664059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56254cc37270 0x56254d01b9d0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400cab0 tx=0x7f4c8400cf70 comp rx=0 tx=0).stop 2024-05-20T18:37:20.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.800+0000 7f4c9a3bb740 1 -- 172.21.15.5:0/945664059 shutdown_connections 2024-05-20T18:37:20.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.800+0000 7f4c9a3bb740 1 --2- 172.21.15.5:0/945664059 >> [v2:172.21.15.63:6833/2352520472,v1:172.21.15.63:6835/2352520472] conn(0x56254cbf5e40 0x56254d039cf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.800+0000 7f4c9a3bb740 1 --2- 172.21.15.5:0/945664059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56254cc37270 0x56254d01b9d0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.800+0000 7f4c9a3bb740 1 --2- 172.21.15.5:0/945664059 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56254cc16590 0x56254d01b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.800+0000 7f4c9a3bb740 1 --2- 172.21.15.5:0/945664059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56254cc0e3b0 0x56254cffcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.800+0000 7f4c6bfff640 0 client.9156 ms_handle_reset on v2:172.21.15.63:6833/2352520472 2024-05-20T18:37:20.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.800+0000 7f4c9a3bb740 1 -- 172.21.15.5:0/945664059 >> 172.21.15.5:0/945664059 conn(0x56254cc2b3c0 msgr2=0x56254cfd7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:20.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.800+0000 7f4c9a3bb740 1 -- 172.21.15.5:0/945664059 shutdown_connections 2024-05-20T18:37:20.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.800+0000 7f4c9a3bb740 1 -- 172.21.15.5:0/945664059 wait complete. 2024-05-20T18:37:20.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.801+0000 7f4c9a3bb740 20 client.9156 trim_cache size 0 max 16384 2024-05-20T18:37:20.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.820+0000 7f4c9a3bb740 1 Processor -- start 2024-05-20T18:37:20.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.820+0000 7f4c9a3bb740 1 -- start start 2024-05-20T18:37:20.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.821+0000 7f4c9a3bb740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56254cc0e3b0 0x56254cffefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.821+0000 7f4c9a3bb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56254cc16590 0x56254d002700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.821+0000 7f4c9a3bb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56254cc37270 0x56254d002c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.821+0000 7f4c9a3bb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c201580 con 0x56254cc16590 2024-05-20T18:37:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.821+0000 7f4c9a3bb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c0cab80 con 0x56254cc37270 2024-05-20T18:37:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.821+0000 7f4c9a3bb740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4c8c200f60 con 0x56254cc0e3b0 2024-05-20T18:37:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.821+0000 7f4c909cd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56254cc16590 0x56254d002700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.821+0000 7f4c909cd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56254cc16590 0x56254d002700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34412/0 (socket says 172.21.15.5:34412) 2024-05-20T18:37:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.821+0000 7f4c909cd640 1 -- 172.21.15.5:0/2552305817 learned_addr learned my addr 172.21.15.5:0/2552305817 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.821+0000 7f4c919cf640 1 --2- 172.21.15.5:0/2552305817 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56254cc37270 0x56254d002c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.821+0000 7f4c911ce640 1 --2- 172.21.15.5:0/2552305817 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56254cc0e3b0 0x56254cffefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.821+0000 7f4c909cd640 1 -- 172.21.15.5:0/2552305817 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56254cc0e3b0 msgr2=0x56254cffefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.821+0000 7f4c909cd640 1 --2- 172.21.15.5:0/2552305817 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56254cc0e3b0 0x56254cffefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.821+0000 7f4c909cd640 1 -- 172.21.15.5:0/2552305817 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56254cc37270 msgr2=0x56254d002c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.821+0000 7f4c909cd640 1 --2- 172.21.15.5:0/2552305817 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56254cc37270 0x56254d002c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.821+0000 7f4c909cd640 1 -- 172.21.15.5:0/2552305817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56254cfff4f0 con 0x56254cc16590 2024-05-20T18:37:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.821+0000 7f4c909cd640 1 --2- 172.21.15.5:0/2552305817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56254cc16590 0x56254d002700 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00a870 tx=0x7f4c7c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37ed464c95bd981 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.822+0000 7f4c767fc640 1 -- 172.21.15.5:0/2552305817 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c013ca0 con 0x56254cc16590 2024-05-20T18:37:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.822+0000 7f4c767fc640 1 -- 172.21.15.5:0/2552305817 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c7c026070 con 0x56254cc16590 2024-05-20T18:37:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.822+0000 7f4c767fc640 1 -- 172.21.15.5:0/2552305817 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c013e40 con 0x56254cc16590 2024-05-20T18:37:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.822+0000 7f4c9a3bb740 1 -- 172.21.15.5:0/2552305817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56254cc16590 msgr2=0x56254d002700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.822+0000 7f4c9a3bb740 1 --2- 172.21.15.5:0/2552305817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56254cc16590 0x56254d002700 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00a870 tx=0x7f4c7c014000 comp rx=0 tx=0).stop 2024-05-20T18:37:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.822+0000 7f4c9a3bb740 1 -- 172.21.15.5:0/2552305817 shutdown_connections 2024-05-20T18:37:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.822+0000 7f4c9a3bb740 1 --2- 172.21.15.5:0/2552305817 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56254cc37270 0x56254d002c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.822+0000 7f4c9a3bb740 1 --2- 172.21.15.5:0/2552305817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56254cc16590 0x56254d002700 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.822+0000 7f4c9a3bb740 1 --2- 172.21.15.5:0/2552305817 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56254cc0e3b0 0x56254cffefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.822+0000 7f4c9a3bb740 1 -- 172.21.15.5:0/2552305817 >> 172.21.15.5:0/2552305817 conn(0x56254cc2b3c0 msgr2=0x56254cfd2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.822+0000 7f4c9a3bb740 1 -- 172.21.15.5:0/2552305817 shutdown_connections 2024-05-20T18:37:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.822+0000 7f4c9a3bb740 1 -- 172.21.15.5:0/2552305817 wait complete. 2024-05-20T18:37:20.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.823+0000 7f4c9a3bb740 1 -- start start 2024-05-20T18:37:20.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.823+0000 7f4c9a3bb740 1 Processor -- start 2024-05-20T18:37:20.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.825+0000 7f4c9a3bb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56254cc0e3b0 0x56254d08fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.825+0000 7f4c9a3bb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56254cc16590 0x56254d035460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.825+0000 7f4c9a3bb740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56254cc37270 0x56254cffc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.825+0000 7f4c9a3bb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56254d024f90 con 0x56254cc16590 2024-05-20T18:37:20.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.825+0000 7f4c9a3bb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56254d01c510 con 0x56254cc0e3b0 2024-05-20T18:37:20.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.825+0000 7f4c9a3bb740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x56254d005690 con 0x56254cc37270 2024-05-20T18:37:20.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.825+0000 7f4c9a3bb740 10 client.0 _ll_register_callbacks cb 0x7f4c8400a680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:37:20.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.825+0000 7f4c909cd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56254cc16590 0x56254d035460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.825+0000 7f4c909cd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56254cc16590 0x56254d035460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34420/0 (socket says 172.21.15.5:34420) 2024-05-20T18:37:20.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.825+0000 7f4c909cd640 1 -- 172.21.15.5:0/1992277593 learned_addr learned my addr 172.21.15.5:0/1992277593 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:20.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.825+0000 7f4c911ce640 1 --2- 172.21.15.5:0/1992277593 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56254cc0e3b0 0x56254d08fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.825+0000 7f4c919cf640 1 --2- 172.21.15.5:0/1992277593 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56254cc37270 0x56254cffc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.825+0000 7f4c909cd640 1 -- 172.21.15.5:0/1992277593 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56254cc37270 msgr2=0x56254cffc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:20.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.825+0000 7f4c909cd640 1 --2- 172.21.15.5:0/1992277593 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56254cc37270 0x56254cffc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.825+0000 7f4c909cd640 1 -- 172.21.15.5:0/1992277593 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56254cc0e3b0 msgr2=0x56254d08fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:20.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.825+0000 7f4c909cd640 1 --2- 172.21.15.5:0/1992277593 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56254cc0e3b0 0x56254d08fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.825+0000 7f4c909cd640 1 -- 172.21.15.5:0/1992277593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56254d01b7e0 con 0x56254cc16590 2024-05-20T18:37:20.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.825+0000 7f4c909cd640 1 --2- 172.21.15.5:0/1992277593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56254cc16590 0x56254d035460 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c008010 tx=0x7f4c7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf15d6ca093cd94c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:20.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.825+0000 7f4c6bfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:37:20.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.826+0000 7f4c6bfff640 1 -- 172.21.15.5:0/1992277593 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c02e070 con 0x56254cc16590 2024-05-20T18:37:20.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.826+0000 7f4c6bfff640 1 -- 172.21.15.5:0/1992277593 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7f4c7c005da0 con 0x56254cc16590 2024-05-20T18:37:20.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.826+0000 7f4c9a3bb740 1 -- 172.21.15.5:0/1992277593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c7c026070 con 0x56254cc16590 2024-05-20T18:37:20.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.826+0000 7f4c6bfff640 1 -- 172.21.15.5:0/1992277593 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f4c7c03a050 con 0x56254cc16590 2024-05-20T18:37:20.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.826+0000 7f4c9a3bb740 10 client.9162 fetch_fsmap learned FSMap version 98 2024-05-20T18:37:20.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.826+0000 7f4c9a3bb740 1 -- 172.21.15.5:0/1992277593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) -- 0x56254cffa5f0 con 0x56254cc16590 2024-05-20T18:37:20.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.826+0000 7f4c6bfff640 1 -- 172.21.15.5:0/1992277593 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c7c026070 con 0x56254cc16590 2024-05-20T18:37:20.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.826+0000 7f4c6bfff640 1 -- 172.21.15.5:0/1992277593 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 98) ==== 3494+0+0 (secure 0 0 0) 0x7f4c7c03d020 con 0x56254cc16590 2024-05-20T18:37:20.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.826+0000 7f4c9a3bb740 10 client.9162 fetch_fsmap finished waiting for FSMap version 98 2024-05-20T18:37:20.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.826+0000 7f4c9a3bb740 10 client.9162 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-20T18:37:20.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.826+0000 7f4c9a3bb740 20 client.9162 populate_metadata read hostname 'smithi005' 2024-05-20T18:37:20.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.826+0000 7f4c9a3bb740 1 --2- 172.21.15.5:0/1992277593 >> [v2:172.21.15.63:6833/2352520472,v1:172.21.15.63:6835/2352520472] conn(0x56254d020dd0 0x56254d0238b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.826+0000 7f4c9a3bb740 4 client.9162 mds_command: new command op to 9030 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T18:37:20.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.826+0000 7f4c9a3bb740 1 -- 172.21.15.5:0/1992277593 --> [v2:172.21.15.63:6833/2352520472,v1:172.21.15.63:6835/2352520472] -- command(tid 0: {"prefix": "status"}) -- 0x56254cf42f50 con 0x56254d020dd0 2024-05-20T18:37:20.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.827+0000 7f4c911ce640 1 --2- 172.21.15.5:0/1992277593 >> [v2:172.21.15.63:6833/2352520472,v1:172.21.15.63:6835/2352520472] conn(0x56254d020dd0 0x56254d0238b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.827+0000 7f4c911ce640 1 --2- 172.21.15.5:0/1992277593 >> [v2:172.21.15.63:6833/2352520472,v1:172.21.15.63:6835/2352520472] conn(0x56254d020dd0 0x56254d0238b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28964f5d261034d0 server_cookie=a9bbe808380fe59b in_seq=0 out_seq=0 2024-05-20T18:37:20.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.827+0000 7f4c6bfff640 10 client.9162 ms_handle_connect on v2:172.21.15.63:6833/2352520472 2024-05-20T18:37:20.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.827+0000 7f4c6bfff640 1 -- 172.21.15.5:0/1992277593 <== mds.0 v2:172.21.15.63:6833/2352520472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x56254cf42f50 con 0x56254d020dd0 2024-05-20T18:37:20.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.827+0000 7f4c6bfff640 10 client.9162 handle_command_reply: tid=0 2024-05-20T18:37:20.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.827+0000 7f4c9a3bb740 1 client.9162 shutdown 2024-05-20T18:37:20.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.827+0000 7f4c9a3bb740 1 -- 172.21.15.5:0/1992277593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56254cc16590 msgr2=0x56254d035460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:20.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.827+0000 7f4c9a3bb740 1 --2- 172.21.15.5:0/1992277593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56254cc16590 0x56254d035460 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c008010 tx=0x7f4c7c004730 comp rx=0 tx=0).stop 2024-05-20T18:37:20.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.827+0000 7f4c9a3bb740 1 -- 172.21.15.5:0/1992277593 shutdown_connections 2024-05-20T18:37:20.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.827+0000 7f4c9a3bb740 1 --2- 172.21.15.5:0/1992277593 >> [v2:172.21.15.63:6833/2352520472,v1:172.21.15.63:6835/2352520472] conn(0x56254d020dd0 0x56254d0238b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.827+0000 7f4c9a3bb740 1 --2- 172.21.15.5:0/1992277593 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56254cc37270 0x56254cffc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.827+0000 7f4c6bfff640 0 client.9162 ms_handle_reset on v2:172.21.15.63:6833/2352520472 2024-05-20T18:37:20.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.827+0000 7f4c9a3bb740 1 --2- 172.21.15.5:0/1992277593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56254cc16590 0x56254d035460 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.827+0000 7f4c9a3bb740 1 --2- 172.21.15.5:0/1992277593 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56254cc0e3b0 0x56254d08fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.827+0000 7f4c9a3bb740 1 -- 172.21.15.5:0/1992277593 >> 172.21.15.5:0/1992277593 conn(0x56254cc2b3c0 msgr2=0x56254cfd29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:20.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.828+0000 7f4c9a3bb740 1 -- 172.21.15.5:0/1992277593 shutdown_connections 2024-05-20T18:37:20.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.828+0000 7f4c9a3bb740 1 -- 172.21.15.5:0/1992277593 wait complete. 2024-05-20T18:37:20.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.828+0000 7f4c9a3bb740 20 client.9162 trim_cache size 0 max 16384 2024-05-20T18:37:20.835 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:37:20.835 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "399ab250-fd3c-47fa-a1c8-25e8149c9cac", 2024-05-20T18:37:20.835 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-20T18:37:20.836 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 9030, 2024-05-20T18:37:20.836 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-20T18:37:20.836 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-20T18:37:20.836 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-20T18:37:20.836 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 5.8314079120000004, 2024-05-20T18:37:20.836 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 98, 2024-05-20T18:37:20.836 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 119, 2024-05-20T18:37:20.836 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 119, 2024-05-20T18:37:20.836 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 10.778616985999999 2024-05-20T18:37:20.836 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:37:20.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.828+0000 7f4c93432640 1 -- 172.21.15.5:0/3323247381 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4c7006be30 msgr2=0x7f4c7006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:20.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.828+0000 7f4c93432640 1 --2- 172.21.15.5:0/3323247381 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4c7006be30 0x7f4c7006e340 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c006fd0 tx=0x7f4c7c008040 comp rx=0 tx=0).stop 2024-05-20T18:37:20.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.828+0000 7f4c93432640 1 -- 172.21.15.5:0/3323247381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c8c0cd4d0 msgr2=0x7f4c8c158c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:20.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.828+0000 7f4c93432640 1 --2- 172.21.15.5:0/3323247381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c8c0cd4d0 0x7f4c8c158c30 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f4c88009870 tx=0x7f4c880046c0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.828+0000 7f4c93432640 1 -- 172.21.15.5:0/3323247381 shutdown_connections 2024-05-20T18:37:20.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.828+0000 7f4c93432640 1 --2- 172.21.15.5:0/3323247381 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4c7006be30 0x7f4c7006e340 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.828+0000 7f4c93432640 1 --2- 172.21.15.5:0/3323247381 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c8c0d14f0 0x7f4c8c0c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.828+0000 7f4c93432640 1 --2- 172.21.15.5:0/3323247381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c8c0d10e0 0x7f4c8c1602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.828+0000 7f4c93432640 1 --2- 172.21.15.5:0/3323247381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c8c0cd4d0 0x7f4c8c158c30 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.828+0000 7f4c93432640 1 -- 172.21.15.5:0/3323247381 >> 172.21.15.5:0/3323247381 conn(0x7f4c8c0b44d0 msgr2=0x7f4c8c0b3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:20.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.828+0000 7f4c93432640 1 -- 172.21.15.5:0/3323247381 shutdown_connections 2024-05-20T18:37:20.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.829+0000 7f4c93432640 1 -- 172.21.15.5:0/3323247381 wait complete. 2024-05-20T18:37:20.841 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-20T18:37:20.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.955+0000 7f552a53e640 1 Processor -- start 2024-05-20T18:37:20.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.955+0000 7f552a53e640 1 -- start start 2024-05-20T18:37:20.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.956+0000 7f552a53e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5524162820 0x7f55240b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.956+0000 7f552a53e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f552406dae0 0x7f55240b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.956+0000 7f552a53e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55240d0630 con 0x7f55240b3e80 2024-05-20T18:37:20.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.956+0000 7f552a53e640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f55240d39c0 con 0x7f5524162820 2024-05-20T18:37:20.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.956+0000 7f552a53e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f55240d3ca0 con 0x7f552406dae0 2024-05-20T18:37:20.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.956+0000 7f5523fff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5524162820 0x7f55240b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.956+0000 7f5528adb640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f552406dae0 0x7f55240b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.956+0000 7f5523fff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5524162820 0x7f55240b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:54854/0 (socket says 172.21.15.5:54854) 2024-05-20T18:37:20.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.956+0000 7f5523fff640 1 -- 172.21.15.5:0/3352775642 learned_addr learned my addr 172.21.15.5:0/3352775642 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:20.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.956+0000 7f551bfff640 1 --1- 172.21.15.5:0/3352775642 >> v1:172.21.15.63:6789/0 conn(0x7f55240b3e80 0x7f552406d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:55726/0 (socket says 172.21.15.5:55726) 2024-05-20T18:37:20.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.957+0000 7f5523fff640 1 -- 172.21.15.5:0/3352775642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f552406dae0 msgr2=0x7f55240b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:20.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.957+0000 7f5523fff640 1 --2- 172.21.15.5:0/3352775642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f552406dae0 0x7f55240b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.957+0000 7f5523fff640 1 -- 172.21.15.5:0/3352775642 >> v1:172.21.15.63:6789/0 conn(0x7f55240b3e80 legacy=0x7f552406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:20.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.957+0000 7f5523fff640 1 -- 172.21.15.5:0/3352775642 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55240d3f80 con 0x7f5524162820 2024-05-20T18:37:20.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.958+0000 7f5523fff640 1 --2- 172.21.15.5:0/3352775642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5524162820 0x7f55240b3940 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5514009870 tx=0x7f551402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d28b125006aec52 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:20.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.958+0000 7f55237fe640 1 -- 172.21.15.5:0/3352775642 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5514037070 con 0x7f5524162820 2024-05-20T18:37:20.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.958+0000 7f55237fe640 1 -- 172.21.15.5:0/3352775642 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5514032440 con 0x7f5524162820 2024-05-20T18:37:20.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.958+0000 7f55237fe640 1 -- 172.21.15.5:0/3352775642 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5514032810 con 0x7f5524162820 2024-05-20T18:37:20.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.959+0000 7f552a53e640 1 -- 172.21.15.5:0/3352775642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5524162820 msgr2=0x7f55240b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:20.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.959+0000 7f552a53e640 1 --2- 172.21.15.5:0/3352775642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5524162820 0x7f55240b3940 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5514009870 tx=0x7f551402a270 comp rx=0 tx=0).stop 2024-05-20T18:37:20.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.959+0000 7f552a53e640 1 -- 172.21.15.5:0/3352775642 shutdown_connections 2024-05-20T18:37:20.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.959+0000 7f552a53e640 1 --2- 172.21.15.5:0/3352775642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f552406dae0 0x7f55240b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.959+0000 7f552a53e640 1 --2- 172.21.15.5:0/3352775642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5524162820 0x7f55240b3940 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.959+0000 7f552a53e640 1 -- 172.21.15.5:0/3352775642 >> 172.21.15.5:0/3352775642 conn(0x7f55240d4bd0 msgr2=0x7f55240d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:20.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.959+0000 7f552a53e640 1 -- 172.21.15.5:0/3352775642 shutdown_connections 2024-05-20T18:37:20.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.959+0000 7f552a53e640 1 -- 172.21.15.5:0/3352775642 wait complete. 2024-05-20T18:37:20.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.960+0000 7f552a53e640 1 Processor -- start 2024-05-20T18:37:20.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.960+0000 7f552a53e640 1 -- start start 2024-05-20T18:37:20.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.962+0000 7f552a53e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55240b3e80 0x7f552414f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.962+0000 7f552a53e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5524162820 0x7f55240cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.962+0000 7f552a53e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f552415c380 0x7f55240cd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.962+0000 7f552a53e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f55240d4780 con 0x7f55240b3e80 2024-05-20T18:37:20.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.962+0000 7f552a53e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f55240d0300 con 0x7f552415c380 2024-05-20T18:37:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.962+0000 7f552a53e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f55240d4900 con 0x7f5524162820 2024-05-20T18:37:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.962+0000 7f5523fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55240b3e80 0x7f552414f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.962+0000 7f5523fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55240b3e80 0x7f552414f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34436/0 (socket says 172.21.15.5:34436) 2024-05-20T18:37:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.962+0000 7f5523fff640 1 -- 172.21.15.5:0/2049243606 learned_addr learned my addr 172.21.15.5:0/2049243606 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.962+0000 7f5528adb640 1 --2- 172.21.15.5:0/2049243606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f552415c380 0x7f55240cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.962+0000 7f551bfff640 1 --2- 172.21.15.5:0/2049243606 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5524162820 0x7f55240cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.962+0000 7f5523fff640 1 -- 172.21.15.5:0/2049243606 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5524162820 msgr2=0x7f55240cbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.962+0000 7f5523fff640 1 --2- 172.21.15.5:0/2049243606 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5524162820 0x7f55240cbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.962+0000 7f5523fff640 1 -- 172.21.15.5:0/2049243606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f552415c380 msgr2=0x7f55240cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.962+0000 7f5523fff640 1 --2- 172.21.15.5:0/2049243606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f552415c380 0x7f55240cd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.962+0000 7f5523fff640 1 -- 172.21.15.5:0/2049243606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55240cc1c0 con 0x7f55240b3e80 2024-05-20T18:37:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.962+0000 7f5523fff640 1 --2- 172.21.15.5:0/2049243606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55240b3e80 0x7f552414f830 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f551402a2b0 tx=0x7f55140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:20.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.963+0000 7f55217fa640 1 -- 172.21.15.5:0/2049243606 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5514037040 con 0x7f55240b3e80 2024-05-20T18:37:20.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.963+0000 7f55217fa640 1 -- 172.21.15.5:0/2049243606 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5514042070 con 0x7f55240b3e80 2024-05-20T18:37:20.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.963+0000 7f552a53e640 1 -- 172.21.15.5:0/2049243606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55240cc490 con 0x7f55240b3e80 2024-05-20T18:37:20.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.963+0000 7f55217fa640 1 -- 172.21.15.5:0/2049243606 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5514030880 con 0x7f55240b3e80 2024-05-20T18:37:20.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.963+0000 7f552a53e640 1 -- 172.21.15.5:0/2049243606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55240cca50 con 0x7f55240b3e80 2024-05-20T18:37:20.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.964+0000 7f552a53e640 1 -- 172.21.15.5:0/2049243606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5524075b00 con 0x7f55240b3e80 2024-05-20T18:37:20.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.964+0000 7f55217fa640 1 -- 172.21.15.5:0/2049243606 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5514030a20 con 0x7f55240b3e80 2024-05-20T18:37:20.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.965+0000 7f55217fa640 1 --2- 172.21.15.5:0/2049243606 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f550406be30 0x7f550406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:20.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.965+0000 7f55217fa640 1 -- 172.21.15.5:0/2049243606 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7f55140ab6e0 con 0x7f55240b3e80 2024-05-20T18:37:20.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.965+0000 7f551bfff640 1 --2- 172.21.15.5:0/2049243606 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f550406be30 0x7f550406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:20.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.965+0000 7f551bfff640 1 --2- 172.21.15.5:0/2049243606 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f550406be30 0x7f550406e340 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f55241529c0 tx=0x7f550c000ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:20.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:20.970+0000 7f55217fa640 1 -- 172.21.15.5:0/2049243606 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f55140798b0 con 0x7f55240b3e80 2024-05-20T18:37:21.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:21.200+0000 7f552a53e640 1 -- 172.21.15.5:0/2049243606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f552415e590 con 0x7f55240b3e80 2024-05-20T18:37:21.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:21.201+0000 7f55217fa640 1 -- 172.21.15.5:0/2049243606 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v98) ==== 53+0+73 (secure 0 0 0) 0x7f551407dbb0 con 0x7f55240b3e80 2024-05-20T18:37:21.202 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-20T18:37:21.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:21.203+0000 7f552a53e640 1 -- 172.21.15.5:0/2049243606 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f550406be30 msgr2=0x7f550406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:21.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:21.203+0000 7f552a53e640 1 --2- 172.21.15.5:0/2049243606 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f550406be30 0x7f550406e340 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f55241529c0 tx=0x7f550c000ed0 comp rx=0 tx=0).stop 2024-05-20T18:37:21.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:21.203+0000 7f552a53e640 1 -- 172.21.15.5:0/2049243606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55240b3e80 msgr2=0x7f552414f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:21.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:21.203+0000 7f552a53e640 1 --2- 172.21.15.5:0/2049243606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55240b3e80 0x7f552414f830 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f551402a2b0 tx=0x7f55140046c0 comp rx=0 tx=0).stop 2024-05-20T18:37:21.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:21.203+0000 7f552a53e640 1 -- 172.21.15.5:0/2049243606 shutdown_connections 2024-05-20T18:37:21.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:21.203+0000 7f552a53e640 1 --2- 172.21.15.5:0/2049243606 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f550406be30 0x7f550406e340 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:21.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:21.203+0000 7f552a53e640 1 --2- 172.21.15.5:0/2049243606 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f552415c380 0x7f55240cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:21.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:21.203+0000 7f552a53e640 1 --2- 172.21.15.5:0/2049243606 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5524162820 0x7f55240cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:21.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:21.203+0000 7f552a53e640 1 --2- 172.21.15.5:0/2049243606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55240b3e80 0x7f552414f830 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:21.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:21.203+0000 7f552a53e640 1 -- 172.21.15.5:0/2049243606 >> 172.21.15.5:0/2049243606 conn(0x7f55240d4bd0 msgr2=0x7f552406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:21.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:21.203+0000 7f552a53e640 1 -- 172.21.15.5:0/2049243606 shutdown_connections 2024-05-20T18:37:21.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:21.203+0000 7f552a53e640 1 -- 172.21.15.5:0/2049243606 wait complete. 2024-05-20T18:37:21.216 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-20T18:37:21.216 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-20T18:37:21.216 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-05-20T18:37:21.216 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-20T18:37:21.216 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-20T18:37:21.216 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-20T18:37:21.216 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-20T18:37:21.216 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-20T18:37:21.216 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:37:21.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:21.217 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-05-20T18:37:21.234 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T18:37:21.235 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T18:37:21.235 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T18:37:21.235 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:37:21.235 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-20T18:37:21.235 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:37:21.235 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:37:21.235 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-05-20T18:37:21.235 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-05-20T18:37:21.235 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:37:21.235 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-05-20T18:37:21.235 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-05-20T18:37:21.235 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-20T18:37:21.236 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:37:21.236 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-05-20T18:37:21.236 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-20T18:37:21.236 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42057sec preferred_lft 42057sec 2024-05-20T18:37:21.236 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-05-20T18:37:21.236 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:37:21.236 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:37:21.236 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:37:21.236 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-05-20T18:37:21.236 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-20T18:37:21.236 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:37:21.236 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-20T18:37:21.237 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:37:21.237 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::4474:eaff:fe07:4f9a/64 scope link 2024-05-20T18:37:21.237 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:37:21.237 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:37:21.237 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T18:37:21.237 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ce4:f4ff:fe5e:7906/64 scope link 2024-05-20T18:37:21.237 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:37:21.237 INFO:teuthology.orchestra.run.smithi005.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:37:21.237 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether da:26:04:0e:95:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T18:37:21.237 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::d826:4ff:fe0e:950a/64 scope link 2024-05-20T18:37:21.237 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:37:21.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:21.238 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-20T18:37:21.300 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-20T18:37:21.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:21.302 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-05-20T18:37:21.358 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-05-20T18:37:21.359 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-20T18:37:21.359 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-20T18:37:21.359 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-20T18:37:21.359 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-20T18:37:21.359 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:37:21.359 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-20T18:37:21.359 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-20T18:37:21.359 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-20T18:37:21.359 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-20T18:37:21.360 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-20T18:37:21.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:37:21.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:21.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:21.489 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-20T18:37:21.505 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-20T18:37:21.505 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-20T18:37:21.505 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:37:21.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:21.506 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-05-20T18:37:21.572 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:37:21.572 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-20T18:37:21.628 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:37:21.628 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:37:21.628 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:37:21.683 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-20T18:37:21.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:21.683 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:37:21.779 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:37:21.779 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:37:21.779 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:37:21.790 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:37:21.790 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:37:21.790 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:37:21.790 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:37:21.790 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:37:21.809 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:37:21.809 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:37:21.809 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:37:21.809 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:37:21.809 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:37:21.809 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790" 2024-05-20T18:37:21.810 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790:/ 2024-05-20T18:37:21.810 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=399ab250-fd3c-47fa-a1c8-25e8149c9cac" will pass to kernel 2024-05-20T18:37:21.810 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-20T18:37:21.810 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-05-20T18:37:21.810 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-20T18:37:21.810 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-05-20T18:37:21.810 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-20T18:37:21.811 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-20T18:37:21.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:21.812 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:37:21.838 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-20T18:37:21.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T18:37:21.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:21.943 INFO:teuthology.orchestra.run.smithi005.stdout:client7280 2024-05-20T18:37:21.943 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:37:21.943 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:37:21.960 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:37:21.960 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:37:21.960 DEBUG:teuthology.orchestra.run.smithi005:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2024-05-20T18:37:21.960 DEBUG:teuthology.orchestra.run.smithi005:> parent = os.path.join(mount_point, "parent") 2024-05-20T18:37:21.960 DEBUG:teuthology.orchestra.run.smithi005:> os.mkdir(parent) 2024-05-20T18:37:21.960 DEBUG:teuthology.orchestra.run.smithi005:> flushed = os.path.join(parent, "flushed") 2024-05-20T18:37:21.961 DEBUG:teuthology.orchestra.run.smithi005:> os.mkdir(flushed) 2024-05-20T18:37:21.961 DEBUG:teuthology.orchestra.run.smithi005:> for f in ["alpha", "bravo", "charlie"]: 2024-05-20T18:37:21.961 DEBUG:teuthology.orchestra.run.smithi005:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2024-05-20T18:37:21.961 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:37:22.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:22.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:22.088 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-20T18:37:22.150 INFO:teuthology.orchestra.run.smithi005.stdout:./ 2024-05-20T18:37:22.150 INFO:teuthology.orchestra.run.smithi005.stdout:./parent 2024-05-20T18:37:22.150 INFO:teuthology.orchestra.run.smithi005.stdout:./parent/flushed 2024-05-20T18:37:22.150 INFO:teuthology.orchestra.run.smithi005.stdout:./parent/flushed/charlie 2024-05-20T18:37:22.150 INFO:teuthology.orchestra.run.smithi005.stdout:./parent/flushed/alpha 2024-05-20T18:37:22.150 INFO:teuthology.orchestra.run.smithi005.stdout:./parent/flushed/bravo 2024-05-20T18:37:22.151 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:37:22.151 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:37:22.151 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:37:22.151 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:37:22.151 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-20T18:37:22.152 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:37:22.259 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-20T18:37:22.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:22.268 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:37:22.268 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:37:22.268 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:37:22.268 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:37:22.268 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-05-20T18:37:22.268 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:37:22.376 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627776 2024-05-20T18:37:22.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:22.385 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:37:22.386 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:37:22.386 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:37:22.386 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:37:22.386 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-05-20T18:37:22.386 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:37:22.494 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627777 2024-05-20T18:37:22.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:22.503 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:37:22.504 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:37:22.504 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:37:22.504 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:37:22.504 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-05-20T18:37:22.504 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:37:22.612 INFO:teuthology.orchestra.run.smithi005.stdout:1099511628280 2024-05-20T18:37:22.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:22.621 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:37:22.621 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:37:22.621 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:37:22.621 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:37:22.621 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-05-20T18:37:22.621 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:37:22.729 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627778 2024-05-20T18:37:22.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:22.738 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:37:22.739 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:37:22.739 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:37:22.739 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:37:22.739 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-05-20T18:37:22.739 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:37:22.847 INFO:teuthology.orchestra.run.smithi005.stdout:1099511628279 2024-05-20T18:37:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:22.856 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:37:22.856 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:37:22.912 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:37:22.913 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:37:22.969 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:37:23.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.122+0000 7f104408b640 1 Processor -- start 2024-05-20T18:37:23.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.123+0000 7f104408b640 1 -- start start 2024-05-20T18:37:23.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.124+0000 7f104408b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f103c162820 0x7f103c0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:23.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.124+0000 7f104408b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f103c06dae0 0x7f103c0b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:23.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.124+0000 7f104408b640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f103c0d0630 con 0x7f103c0b3e80 2024-05-20T18:37:23.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.124+0000 7f104408b640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f103c0d39c0 con 0x7f103c162820 2024-05-20T18:37:23.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.124+0000 7f104408b640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f103c0d3ca0 con 0x7f103c06dae0 2024-05-20T18:37:23.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.124+0000 7f1041e27640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f103c162820 0x7f103c0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:23.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.124+0000 7f1041e27640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f103c162820 0x7f103c0b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34448/0 (socket says 172.21.15.5:34448) 2024-05-20T18:37:23.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.124+0000 7f1041e27640 1 -- 172.21.15.5:0/1835997137 learned_addr learned my addr 172.21.15.5:0/1835997137 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:23.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.124+0000 7f1042628640 1 --2- 172.21.15.5:0/1835997137 >> v2:172.21.15.63:3300/0 conn(0x7f103c06dae0 0x7f103c0b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:23.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.125+0000 7f1041e27640 1 -- 172.21.15.5:0/1835997137 >> v2:172.21.15.63:3300/0 conn(0x7f103c06dae0 msgr2=0x7f103c0b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:23.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.125+0000 7f1041e27640 1 --2- 172.21.15.5:0/1835997137 >> v2:172.21.15.63:3300/0 conn(0x7f103c06dae0 0x7f103c0b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:23.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.125+0000 7f1041e27640 1 -- 172.21.15.5:0/1835997137 >> v1:172.21.15.63:6789/0 conn(0x7f103c0b3e80 legacy=0x7f103c06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:23.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.125+0000 7f1041e27640 1 -- 172.21.15.5:0/1835997137 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f103c0d3f80 con 0x7f103c162820 2024-05-20T18:37:23.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.125+0000 7f1041e27640 1 --2- 172.21.15.5:0/1835997137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f103c162820 0x7f103c0b3940 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f103c08e070 tx=0x7f103802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977a59ece564ac6e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:23.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.125+0000 7f1040e25640 1 -- 172.21.15.5:0/1835997137 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038037070 con 0x7f103c162820 2024-05-20T18:37:23.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.126+0000 7f1040e25640 1 -- 172.21.15.5:0/1835997137 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1038032440 con 0x7f103c162820 2024-05-20T18:37:23.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.126+0000 7f1040e25640 1 -- 172.21.15.5:0/1835997137 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038032810 con 0x7f103c162820 2024-05-20T18:37:23.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.126+0000 7f104408b640 1 -- 172.21.15.5:0/1835997137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f103c162820 msgr2=0x7f103c0b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:23.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.126+0000 7f104408b640 1 --2- 172.21.15.5:0/1835997137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f103c162820 0x7f103c0b3940 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f103c08e070 tx=0x7f103802a270 comp rx=0 tx=0).stop 2024-05-20T18:37:23.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.126+0000 7f104408b640 1 -- 172.21.15.5:0/1835997137 shutdown_connections 2024-05-20T18:37:23.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.126+0000 7f104408b640 1 --2- 172.21.15.5:0/1835997137 >> v2:172.21.15.63:3300/0 conn(0x7f103c06dae0 0x7f103c0b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:23.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.126+0000 7f104408b640 1 --2- 172.21.15.5:0/1835997137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f103c162820 0x7f103c0b3940 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:23.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.126+0000 7f104408b640 1 -- 172.21.15.5:0/1835997137 >> 172.21.15.5:0/1835997137 conn(0x7f103c0d4bd0 msgr2=0x7f103c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:23.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.126+0000 7f104408b640 1 -- 172.21.15.5:0/1835997137 shutdown_connections 2024-05-20T18:37:23.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.126+0000 7f104408b640 1 -- 172.21.15.5:0/1835997137 wait complete. 2024-05-20T18:37:23.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.128+0000 7f104408b640 1 Processor -- start 2024-05-20T18:37:23.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.128+0000 7f104408b640 1 -- start start 2024-05-20T18:37:23.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.129+0000 7f104408b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f103c0b3e80 0x7f103c0cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:23.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.129+0000 7f104408b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f103c162820 0x7f103c15b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:23.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.129+0000 7f104408b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f103c0cbc80 0x7f103c0b64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:23.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.129+0000 7f104408b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f103c0d4780 con 0x7f103c0cbc80 2024-05-20T18:37:23.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.129+0000 7f104408b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f103c0d0300 con 0x7f103c0b3e80 2024-05-20T18:37:23.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.129+0000 7f104408b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f103c0d4900 con 0x7f103c162820 2024-05-20T18:37:23.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.129+0000 7f1042628640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f103c0cbc80 0x7f103c0b64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:23.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.130+0000 7f1042628640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f103c0cbc80 0x7f103c0b64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34450/0 (socket says 172.21.15.5:34450) 2024-05-20T18:37:23.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.130+0000 7f1042628640 1 -- 172.21.15.5:0/1542777233 learned_addr learned my addr 172.21.15.5:0/1542777233 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:23.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.130+0000 7f1041e27640 1 --2- 172.21.15.5:0/1542777233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f103c0b3e80 0x7f103c0cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:23.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.130+0000 7f1041626640 1 --2- 172.21.15.5:0/1542777233 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f103c162820 0x7f103c15b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:23.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.130+0000 7f1042628640 1 -- 172.21.15.5:0/1542777233 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f103c162820 msgr2=0x7f103c15b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:23.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.130+0000 7f1042628640 1 --2- 172.21.15.5:0/1542777233 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f103c162820 0x7f103c15b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:23.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.130+0000 7f1042628640 1 -- 172.21.15.5:0/1542777233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f103c0b3e80 msgr2=0x7f103c0cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:23.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.130+0000 7f1042628640 1 --2- 172.21.15.5:0/1542777233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f103c0b3e80 0x7f103c0cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:23.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.130+0000 7f1042628640 1 -- 172.21.15.5:0/1542777233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f103c0cc360 con 0x7f103c0cbc80 2024-05-20T18:37:23.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.130+0000 7f1041e27640 1 --2- 172.21.15.5:0/1542777233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f103c0b3e80 0x7f103c0cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:23.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.130+0000 7f1041626640 1 --2- 172.21.15.5:0/1542777233 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f103c162820 0x7f103c15b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:23.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.130+0000 7f1042628640 1 --2- 172.21.15.5:0/1542777233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f103c0cbc80 0x7f103c0b64f0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f103400efc0 tx=0x7f103400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:23.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.130+0000 7f10327fc640 1 -- 172.21.15.5:0/1542777233 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1034018020 con 0x7f103c0cbc80 2024-05-20T18:37:23.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.130+0000 7f10327fc640 1 -- 172.21.15.5:0/1542777233 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1034014480 con 0x7f103c0cbc80 2024-05-20T18:37:23.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.130+0000 7f104408b640 1 -- 172.21.15.5:0/1542777233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f103c0ceae0 con 0x7f103c0cbc80 2024-05-20T18:37:23.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.130+0000 7f10327fc640 1 -- 172.21.15.5:0/1542777233 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1034014870 con 0x7f103c0cbc80 2024-05-20T18:37:23.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.130+0000 7f104408b640 1 -- 172.21.15.5:0/1542777233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f103c0cf0a0 con 0x7f103c0cbc80 2024-05-20T18:37:23.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.131+0000 7f104408b640 1 -- 172.21.15.5:0/1542777233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f103c075b00 con 0x7f103c0cbc80 2024-05-20T18:37:23.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.132+0000 7f10327fc640 1 -- 172.21.15.5:0/1542777233 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1034005d10 con 0x7f103c0cbc80 2024-05-20T18:37:23.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.132+0000 7f10327fc640 1 --2- 172.21.15.5:0/1542777233 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f102006be60 0x7f102006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:23.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.132+0000 7f10327fc640 1 -- 172.21.15.5:0/1542777233 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7f103408bb00 con 0x7f103c0cbc80 2024-05-20T18:37:23.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.132+0000 7f1041e27640 1 --2- 172.21.15.5:0/1542777233 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f102006be60 0x7f102006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:23.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.133+0000 7f1041e27640 1 --2- 172.21.15.5:0/1542777233 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f102006be60 0x7f102006e370 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f1038034000 tx=0x7f103802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:23.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.137+0000 7f10327fc640 1 -- 172.21.15.5:0/1542777233 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1034022070 con 0x7f103c0cbc80 2024-05-20T18:37:23.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.322+0000 7f104408b640 1 -- 172.21.15.5:0/1542777233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f103c0cc0f0 con 0x7f103c0cbc80 2024-05-20T18:37:23.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.323+0000 7f10327fc640 1 -- 172.21.15.5:0/1542777233 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+16832 (secure 0 0 0) 0x7f1034010040 con 0x7f103c0cbc80 2024-05-20T18:37:23.325 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:23.325 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":119,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:37:15.995209+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:37:12.797471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-20T18:37:13.346156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:37:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.325+0000 7f104408b640 1 -- 172.21.15.5:0/1542777233 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f102006be60 msgr2=0x7f102006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.325+0000 7f104408b640 1 --2- 172.21.15.5:0/1542777233 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f102006be60 0x7f102006e370 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f1038034000 tx=0x7f103802e000 comp rx=0 tx=0).stop 2024-05-20T18:37:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.325+0000 7f104408b640 1 -- 172.21.15.5:0/1542777233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f103c0cbc80 msgr2=0x7f103c0b64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.325+0000 7f104408b640 1 --2- 172.21.15.5:0/1542777233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f103c0cbc80 0x7f103c0b64f0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f103400efc0 tx=0x7f103400c490 comp rx=0 tx=0).stop 2024-05-20T18:37:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.325+0000 7f104408b640 1 -- 172.21.15.5:0/1542777233 shutdown_connections 2024-05-20T18:37:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.325+0000 7f104408b640 1 --2- 172.21.15.5:0/1542777233 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f102006be60 0x7f102006e370 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.325+0000 7f104408b640 1 --2- 172.21.15.5:0/1542777233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f103c0cbc80 0x7f103c0b64f0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.325+0000 7f104408b640 1 --2- 172.21.15.5:0/1542777233 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f103c162820 0x7f103c15b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.325+0000 7f104408b640 1 --2- 172.21.15.5:0/1542777233 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f103c0b3e80 0x7f103c0cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:23.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.325+0000 7f104408b640 1 -- 172.21.15.5:0/1542777233 >> 172.21.15.5:0/1542777233 conn(0x7f103c0d4bd0 msgr2=0x7f103c15fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:23.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.326+0000 7f104408b640 1 -- 172.21.15.5:0/1542777233 shutdown_connections 2024-05-20T18:37:23.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.326+0000 7f104408b640 1 -- 172.21.15.5:0/1542777233 wait complete. 2024-05-20T18:37:23.338 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:37:12.797471+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:37:13.346156+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:37:23.339 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:37:23.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.455+0000 7ff0a1b84640 1 Processor -- start 2024-05-20T18:37:23.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.455+0000 7ff0a1b84640 1 -- start start 2024-05-20T18:37:23.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.456+0000 7ff0a1b84640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff09c0cf3d0 0x7ff09c0cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:23.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.456+0000 7ff0a1b84640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff09c0cfd20 con 0x7ff09c0ce7e0 2024-05-20T18:37:23.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.456+0000 7ff0a1b84640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff09c0cda20 con 0x7ff09c15f220 2024-05-20T18:37:23.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.456+0000 7ff0a1b84640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff09c0cdd00 con 0x7ff09c0cf3d0 2024-05-20T18:37:23.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.457+0000 7ff09bfff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff09c0ce7e0 0x7ff09c0cebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:40892/0 (socket says 172.21.15.5:40892) 2024-05-20T18:37:23.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.457+0000 7ff09bfff640 1 -- 172.21.15.5:0/568457164 learned_addr learned my addr 172.21.15.5:0/568457164 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:23.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.457+0000 7ff09affd640 1 --2- 172.21.15.5:0/568457164 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff09c0cf3d0 0x7ff09c0cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:23.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.457+0000 7ff09a7fc640 1 -- 172.21.15.5:0/568457164 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3926863866 0 0) 0x7ff094001490 con 0x7ff09c0ce7e0 2024-05-20T18:37:23.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.457+0000 7ff09a7fc640 1 -- 172.21.15.5:0/568457164 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff084003520 con 0x7ff09c0ce7e0 2024-05-20T18:37:23.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.457+0000 7ff09a7fc640 1 -- 172.21.15.5:0/568457164 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2910785400 0 0) 0x7ff09c0cda20 con 0x7ff09c15f220 2024-05-20T18:37:23.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.457+0000 7ff09affd640 1 -- 172.21.15.5:0/568457164 >> v1:172.21.15.63:6789/0 conn(0x7ff09c15f220 legacy=0x7ff09c0d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:23.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.457+0000 7ff09affd640 1 -- 172.21.15.5:0/568457164 >> v1:172.21.15.5:6789/0 conn(0x7ff09c0ce7e0 legacy=0x7ff09c0cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:23.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.457+0000 7ff09affd640 1 -- 172.21.15.5:0/568457164 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09c0cdfe0 con 0x7ff09c0cf3d0 2024-05-20T18:37:23.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.458+0000 7ff09affd640 1 --2- 172.21.15.5:0/568457164 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff09c0cf3d0 0x7ff09c0cf7e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff0900097d0 tx=0x7ff09002a0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68e79f4559760ccb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:23.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.458+0000 7ff09a7fc640 1 -- 172.21.15.5:0/568457164 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff090037070 con 0x7ff09c0cf3d0 2024-05-20T18:37:23.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.458+0000 7ff09a7fc640 1 -- 172.21.15.5:0/568457164 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff09002ab70 con 0x7ff09c0cf3d0 2024-05-20T18:37:23.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.458+0000 7ff09a7fc640 1 -- 172.21.15.5:0/568457164 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094001bc0 con 0x7ff09c0cf3d0 2024-05-20T18:37:23.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.459+0000 7ff0a1b84640 1 -- 172.21.15.5:0/568457164 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff09c0cf3d0 msgr2=0x7ff09c0cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:23.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.459+0000 7ff0a1b84640 1 --2- 172.21.15.5:0/568457164 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff09c0cf3d0 0x7ff09c0cf7e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff0900097d0 tx=0x7ff09002a0e0 comp rx=0 tx=0).stop 2024-05-20T18:37:23.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.459+0000 7ff0a1b84640 1 -- 172.21.15.5:0/568457164 shutdown_connections 2024-05-20T18:37:23.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.459+0000 7ff0a1b84640 1 --2- 172.21.15.5:0/568457164 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff09c0cf3d0 0x7ff09c0cf7e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:23.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.459+0000 7ff0a1b84640 1 -- 172.21.15.5:0/568457164 >> 172.21.15.5:0/568457164 conn(0x7ff09c0b44d0 msgr2=0x7ff09c0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:23.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.459+0000 7ff0a1b84640 1 -- 172.21.15.5:0/568457164 shutdown_connections 2024-05-20T18:37:23.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.459+0000 7ff0a1b84640 1 -- 172.21.15.5:0/568457164 wait complete. 2024-05-20T18:37:23.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.461+0000 7ff0a1b84640 1 Processor -- start 2024-05-20T18:37:23.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.461+0000 7ff0a1b84640 1 -- start start 2024-05-20T18:37:23.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.462+0000 7ff0a1b84640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff09c0ce7e0 0x7ff09c0b7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:23.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.462+0000 7ff0a1b84640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff09c0cf3d0 0x7ff09c1f8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:23.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.462+0000 7ff0a1b84640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff09c15f220 0x7ff09c0d2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:23.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.462+0000 7ff0a1b84640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c0cd680 con 0x7ff09c0cf3d0 2024-05-20T18:37:23.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.462+0000 7ff0a1b84640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c0d0fd0 con 0x7ff09c15f220 2024-05-20T18:37:23.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.462+0000 7ff0a1b84640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff09c0cd800 con 0x7ff09c0ce7e0 2024-05-20T18:37:23.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.462+0000 7ff09affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff09c0cf3d0 0x7ff09c1f8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:23.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.462+0000 7ff09affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff09c0cf3d0 0x7ff09c1f8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34452/0 (socket says 172.21.15.5:34452) 2024-05-20T18:37:23.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.462+0000 7ff09affd640 1 -- 172.21.15.5:0/1723762634 learned_addr learned my addr 172.21.15.5:0/1723762634 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:23.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.463+0000 7ff09b7fe640 1 --2- 172.21.15.5:0/1723762634 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff09c0ce7e0 0x7ff09c0b7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:23.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.463+0000 7ff09bfff640 1 --2- 172.21.15.5:0/1723762634 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff09c15f220 0x7ff09c0d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:23.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.463+0000 7ff09affd640 1 -- 172.21.15.5:0/1723762634 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff09c0ce7e0 msgr2=0x7ff09c0b7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:23.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.463+0000 7ff09affd640 1 --2- 172.21.15.5:0/1723762634 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff09c0ce7e0 0x7ff09c0b7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:23.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.463+0000 7ff09affd640 1 -- 172.21.15.5:0/1723762634 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff09c15f220 msgr2=0x7ff09c0d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:23.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.463+0000 7ff09affd640 1 --2- 172.21.15.5:0/1723762634 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff09c15f220 0x7ff09c0d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:23.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.463+0000 7ff09affd640 1 -- 172.21.15.5:0/1723762634 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09c0c9920 con 0x7ff09c0cf3d0 2024-05-20T18:37:23.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.463+0000 7ff09bfff640 1 --2- 172.21.15.5:0/1723762634 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff09c15f220 0x7ff09c0d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:23.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.463+0000 7ff09affd640 1 --2- 172.21.15.5:0/1723762634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff09c0cf3d0 0x7ff09c1f8a70 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7ff0900097d0 tx=0x7ff0900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:23.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.463+0000 7ff083fff640 1 -- 172.21.15.5:0/1723762634 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff090037040 con 0x7ff09c0cf3d0 2024-05-20T18:37:23.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.463+0000 7ff0a1b84640 1 -- 172.21.15.5:0/1723762634 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff09c0c9e50 con 0x7ff09c0cf3d0 2024-05-20T18:37:23.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.463+0000 7ff083fff640 1 -- 172.21.15.5:0/1723762634 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff090004030 con 0x7ff09c0cf3d0 2024-05-20T18:37:23.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.463+0000 7ff083fff640 1 -- 172.21.15.5:0/1723762634 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff090004420 con 0x7ff09c0cf3d0 2024-05-20T18:37:23.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.463+0000 7ff0a1b84640 1 -- 172.21.15.5:0/1723762634 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff09c0ca410 con 0x7ff09c0cf3d0 2024-05-20T18:37:23.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.464+0000 7ff0a1b84640 1 -- 172.21.15.5:0/1723762634 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff09c075b00 con 0x7ff09c0cf3d0 2024-05-20T18:37:23.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.465+0000 7ff083fff640 1 -- 172.21.15.5:0/1723762634 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff09003a040 con 0x7ff09c0cf3d0 2024-05-20T18:37:23.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.465+0000 7ff083fff640 1 --2- 172.21.15.5:0/1723762634 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff08806be60 0x7ff08806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:23.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.466+0000 7ff083fff640 1 -- 172.21.15.5:0/1723762634 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7ff0900ab0c0 con 0x7ff09c0cf3d0 2024-05-20T18:37:23.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.466+0000 7ff09b7fe640 1 --2- 172.21.15.5:0/1723762634 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff08806be60 0x7ff08806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:23.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.466+0000 7ff09b7fe640 1 --2- 172.21.15.5:0/1723762634 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff08806be60 0x7ff08806e370 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff08c000f60 tx=0x7ff08c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:23.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.470+0000 7ff083fff640 1 -- 172.21.15.5:0/1723762634 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff090042020 con 0x7ff09c0cf3d0 2024-05-20T18:37:23.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.653+0000 7ff0a1b84640 1 -- 172.21.15.5:0/1723762634 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff09c0cfb00 con 0x7ff09c0cf3d0 2024-05-20T18:37:23.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.654+0000 7ff083fff640 1 -- 172.21.15.5:0/1723762634 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) ==== 93+0+10 (secure 0 0 0) 0x7ff090079180 con 0x7ff09c0cf3d0 2024-05-20T18:37:23.655 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:37:23.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.656+0000 7ff0a1b84640 1 -- 172.21.15.5:0/1723762634 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff08806be60 msgr2=0x7ff08806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:23.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.656+0000 7ff0a1b84640 1 --2- 172.21.15.5:0/1723762634 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff08806be60 0x7ff08806e370 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff08c000f60 tx=0x7ff08c009040 comp rx=0 tx=0).stop 2024-05-20T18:37:23.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.656+0000 7ff0a1b84640 1 -- 172.21.15.5:0/1723762634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff09c0cf3d0 msgr2=0x7ff09c1f8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:23.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.656+0000 7ff0a1b84640 1 --2- 172.21.15.5:0/1723762634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff09c0cf3d0 0x7ff09c1f8a70 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7ff0900097d0 tx=0x7ff0900046c0 comp rx=0 tx=0).stop 2024-05-20T18:37:23.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.656+0000 7ff0a1b84640 1 -- 172.21.15.5:0/1723762634 shutdown_connections 2024-05-20T18:37:23.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.656+0000 7ff0a1b84640 1 --2- 172.21.15.5:0/1723762634 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff08806be60 0x7ff08806e370 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:23.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.656+0000 7ff0a1b84640 1 --2- 172.21.15.5:0/1723762634 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff09c15f220 0x7ff09c0d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:23.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.656+0000 7ff0a1b84640 1 --2- 172.21.15.5:0/1723762634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff09c0cf3d0 0x7ff09c1f8a70 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:23.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.656+0000 7ff0a1b84640 1 --2- 172.21.15.5:0/1723762634 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff09c0ce7e0 0x7ff09c0b7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:23.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.656+0000 7ff0a1b84640 1 -- 172.21.15.5:0/1723762634 >> 172.21.15.5:0/1723762634 conn(0x7ff09c0b44d0 msgr2=0x7ff09c0d08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:23.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.656+0000 7ff0a1b84640 1 -- 172.21.15.5:0/1723762634 shutdown_connections 2024-05-20T18:37:23.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.656+0000 7ff0a1b84640 1 -- 172.21.15.5:0/1723762634 wait complete. 2024-05-20T18:37:23.669 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:37:23.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.785+0000 7f1ab7b79640 1 Processor -- start 2024-05-20T18:37:23.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.785+0000 7f1ab7b79640 1 -- start start 2024-05-20T18:37:23.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.786+0000 7f1ab7b79640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1ab00b1e10 0x7f1ab015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:23.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.786+0000 7f1ab7b79640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1ab0070850 0x7f1ab0070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:23.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.786+0000 7f1ab7b79640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ab00cd580 con 0x7f1ab015d530 2024-05-20T18:37:23.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.786+0000 7f1ab7b79640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f1ab00ce190 con 0x7f1ab00b1e10 2024-05-20T18:37:23.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.786+0000 7f1ab7b79640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1ab00cd860 con 0x7f1ab0070850 2024-05-20T18:37:23.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.786+0000 7f1ab5114640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1ab00b1e10 0x7f1ab015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:23.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.786+0000 7f1ab5114640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1ab00b1e10 0x7f1ab015aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34464/0 (socket says 172.21.15.5:34464) 2024-05-20T18:37:23.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.786+0000 7f1ab5114640 1 -- 172.21.15.5:0/4018358969 learned_addr learned my addr 172.21.15.5:0/4018358969 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:23.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.787+0000 7f1ab6116640 1 --2- 172.21.15.5:0/4018358969 >> v2:172.21.15.63:3300/0 conn(0x7f1ab0070850 0x7f1ab0070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:23.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.787+0000 7f1ab5114640 1 -- 172.21.15.5:0/4018358969 >> v2:172.21.15.63:3300/0 conn(0x7f1ab0070850 msgr2=0x7f1ab0070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:23.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.787+0000 7f1ab5114640 1 --2- 172.21.15.5:0/4018358969 >> v2:172.21.15.63:3300/0 conn(0x7f1ab0070850 0x7f1ab0070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:23.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.787+0000 7f1ab5114640 1 -- 172.21.15.5:0/4018358969 >> v1:172.21.15.63:6789/0 conn(0x7f1ab015d530 legacy=0x7f1ab00b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:23.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.787+0000 7f1ab5114640 1 -- 172.21.15.5:0/4018358969 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab00cdb40 con 0x7f1ab00b1e10 2024-05-20T18:37:23.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.788+0000 7f1ab5114640 1 --2- 172.21.15.5:0/4018358969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab00b1e10 0x7f1ab015aaa0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00b8400 tx=0x7f1aa402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891dd79c67f7c0ee server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:23.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.788+0000 7f1ab4913640 1 -- 172.21.15.5:0/4018358969 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4037070 con 0x7f1ab00b1e10 2024-05-20T18:37:23.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.788+0000 7f1ab4913640 1 -- 172.21.15.5:0/4018358969 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aa4032440 con 0x7f1ab00b1e10 2024-05-20T18:37:23.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.788+0000 7f1ab4913640 1 -- 172.21.15.5:0/4018358969 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4032810 con 0x7f1ab00b1e10 2024-05-20T18:37:23.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.788+0000 7f1ab7b79640 1 -- 172.21.15.5:0/4018358969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab00b1e10 msgr2=0x7f1ab015aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:23.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.788+0000 7f1ab7b79640 1 --2- 172.21.15.5:0/4018358969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab00b1e10 0x7f1ab015aaa0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00b8400 tx=0x7f1aa402a270 comp rx=0 tx=0).stop 2024-05-20T18:37:23.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.788+0000 7f1ab7b79640 1 -- 172.21.15.5:0/4018358969 shutdown_connections 2024-05-20T18:37:23.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.788+0000 7f1ab7b79640 1 --2- 172.21.15.5:0/4018358969 >> v2:172.21.15.63:3300/0 conn(0x7f1ab0070850 0x7f1ab0070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:23.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.788+0000 7f1ab7b79640 1 --2- 172.21.15.5:0/4018358969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab00b1e10 0x7f1ab015aaa0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:23.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.788+0000 7f1ab7b79640 1 -- 172.21.15.5:0/4018358969 >> 172.21.15.5:0/4018358969 conn(0x7f1ab00d29d0 msgr2=0x7f1ab00d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:23.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.789+0000 7f1ab7b79640 1 -- 172.21.15.5:0/4018358969 shutdown_connections 2024-05-20T18:37:23.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.789+0000 7f1ab7b79640 1 -- 172.21.15.5:0/4018358969 wait complete. 2024-05-20T18:37:23.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.790+0000 7f1ab7b79640 1 Processor -- start 2024-05-20T18:37:23.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.790+0000 7f1ab7b79640 1 -- start start 2024-05-20T18:37:23.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.792+0000 7f1ab7b79640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ab0070850 0x7f1ab014c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:23.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.792+0000 7f1ab7b79640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ab00b1e10 0x7f1ab014e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:23.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.792+0000 7f1ab7b79640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab015d530 0x7f1ab00c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:23.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.792+0000 7f1ab7b79640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab00711a0 con 0x7f1ab015d530 2024-05-20T18:37:23.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.792+0000 7f1ab7b79640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab00cd230 con 0x7f1ab00b1e10 2024-05-20T18:37:23.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.792+0000 7f1ab7b79640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1ab0071320 con 0x7f1ab0070850 2024-05-20T18:37:23.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.792+0000 7f1ab6116640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab015d530 0x7f1ab00c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:23.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.792+0000 7f1ab6116640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab015d530 0x7f1ab00c7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34478/0 (socket says 172.21.15.5:34478) 2024-05-20T18:37:23.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.792+0000 7f1ab6116640 1 -- 172.21.15.5:0/1907120153 learned_addr learned my addr 172.21.15.5:0/1907120153 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:23.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.792+0000 7f1ab5915640 1 --2- 172.21.15.5:0/1907120153 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ab0070850 0x7f1ab014c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:23.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.792+0000 7f1ab5114640 1 --2- 172.21.15.5:0/1907120153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ab00b1e10 0x7f1ab014e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:23.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.792+0000 7f1ab6116640 1 -- 172.21.15.5:0/1907120153 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ab0070850 msgr2=0x7f1ab014c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:23.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.792+0000 7f1ab6116640 1 --2- 172.21.15.5:0/1907120153 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ab0070850 0x7f1ab014c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:23.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.792+0000 7f1ab6116640 1 -- 172.21.15.5:0/1907120153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ab00b1e10 msgr2=0x7f1ab014e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:23.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.792+0000 7f1ab6116640 1 --2- 172.21.15.5:0/1907120153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ab00b1e10 0x7f1ab014e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:23.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.792+0000 7f1ab6116640 1 -- 172.21.15.5:0/1907120153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab00c6c70 con 0x7f1ab015d530 2024-05-20T18:37:23.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.792+0000 7f1ab6116640 1 --2- 172.21.15.5:0/1907120153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab015d530 0x7f1ab00c7d10 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800d5d0 tx=0x7f1aa800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:23.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.793+0000 7f1aae7fc640 1 -- 172.21.15.5:0/1907120153 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa80125f0 con 0x7f1ab015d530 2024-05-20T18:37:23.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.793+0000 7f1aae7fc640 1 -- 172.21.15.5:0/1907120153 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aa8019070 con 0x7f1ab015d530 2024-05-20T18:37:23.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.793+0000 7f1ab7b79640 1 -- 172.21.15.5:0/1907120153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab00c6f50 con 0x7f1ab015d530 2024-05-20T18:37:23.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.793+0000 7f1aae7fc640 1 -- 172.21.15.5:0/1907120153 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa80129f0 con 0x7f1ab015d530 2024-05-20T18:37:23.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.793+0000 7f1ab7b79640 1 -- 172.21.15.5:0/1907120153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab00c7510 con 0x7f1ab015d530 2024-05-20T18:37:23.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.794+0000 7f1ab7b79640 1 -- 172.21.15.5:0/1907120153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab00c69c0 con 0x7f1ab015d530 2024-05-20T18:37:23.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.794+0000 7f1aae7fc640 1 -- 172.21.15.5:0/1907120153 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1aa8012b90 con 0x7f1ab015d530 2024-05-20T18:37:23.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.795+0000 7f1aae7fc640 1 --2- 172.21.15.5:0/1907120153 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1a9c06be60 0x7f1a9c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:23.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.795+0000 7f1aae7fc640 1 -- 172.21.15.5:0/1907120153 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7f1aa808b3d0 con 0x7f1ab015d530 2024-05-20T18:37:23.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.795+0000 7f1ab5915640 1 --2- 172.21.15.5:0/1907120153 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1a9c06be60 0x7f1a9c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:23.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.795+0000 7f1ab5915640 1 --2- 172.21.15.5:0/1907120153 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1a9c06be60 0x7f1a9c06e370 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00d29a0 tx=0x7f1aa0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:23.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.799+0000 7f1aae7fc640 1 -- 172.21.15.5:0/1907120153 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1aa8059b70 con 0x7f1ab015d530 2024-05-20T18:37:23.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.983+0000 7f1ab7b79640 1 -- 172.21.15.5:0/1907120153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1ab0075b00 con 0x7f1ab015d530 2024-05-20T18:37:23.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.984+0000 7f1aae7fc640 1 -- 172.21.15.5:0/1907120153 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v119) ==== 104+0+11 (secure 0 0 0) 0x7f1aa8059470 con 0x7f1ab015d530 2024-05-20T18:37:23.985 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:37:23.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.986+0000 7f1ab7b79640 1 -- 172.21.15.5:0/1907120153 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1a9c06be60 msgr2=0x7f1a9c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:23.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.986+0000 7f1ab7b79640 1 --2- 172.21.15.5:0/1907120153 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1a9c06be60 0x7f1a9c06e370 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00d29a0 tx=0x7f1aa0009040 comp rx=0 tx=0).stop 2024-05-20T18:37:23.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.986+0000 7f1ab7b79640 1 -- 172.21.15.5:0/1907120153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab015d530 msgr2=0x7f1ab00c7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:23.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.986+0000 7f1ab7b79640 1 --2- 172.21.15.5:0/1907120153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab015d530 0x7f1ab00c7d10 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800d5d0 tx=0x7f1aa800da90 comp rx=0 tx=0).stop 2024-05-20T18:37:23.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.986+0000 7f1ab7b79640 1 -- 172.21.15.5:0/1907120153 shutdown_connections 2024-05-20T18:37:23.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.986+0000 7f1ab7b79640 1 --2- 172.21.15.5:0/1907120153 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1a9c06be60 0x7f1a9c06e370 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:23.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.986+0000 7f1ab7b79640 1 --2- 172.21.15.5:0/1907120153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab015d530 0x7f1ab00c7d10 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:23.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.986+0000 7f1ab7b79640 1 --2- 172.21.15.5:0/1907120153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ab00b1e10 0x7f1ab014e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:23.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.986+0000 7f1ab7b79640 1 --2- 172.21.15.5:0/1907120153 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ab0070850 0x7f1ab014c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:23.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.986+0000 7f1ab7b79640 1 -- 172.21.15.5:0/1907120153 >> 172.21.15.5:0/1907120153 conn(0x7f1ab00d29d0 msgr2=0x7f1ab00d06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:23.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.986+0000 7f1ab7b79640 1 -- 172.21.15.5:0/1907120153 shutdown_connections 2024-05-20T18:37:23.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:23.986+0000 7f1ab7b79640 1 -- 172.21.15.5:0/1907120153 wait complete. 2024-05-20T18:37:23.999 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:37:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.115+0000 7fd97ace9640 1 Processor -- start 2024-05-20T18:37:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.115+0000 7fd97ace9640 1 -- start start 2024-05-20T18:37:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.116+0000 7fd97ace9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9740d10e0 0x7fd9740cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.117+0000 7fd97ace9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd9740d14f0 0x7fd9740cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.117+0000 7fd97ace9640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9740d1ac0 con 0x7fd9740cd4d0 2024-05-20T18:37:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.117+0000 7fd97ace9640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fd9740d1da0 con 0x7fd9740d14f0 2024-05-20T18:37:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.117+0000 7fd97ace9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd9740d2080 con 0x7fd9740d10e0 2024-05-20T18:37:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.117+0000 7fd973fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd9740d14f0 0x7fd9740cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.117+0000 7fd973fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd9740d14f0 0x7fd9740cf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34484/0 (socket says 172.21.15.5:34484) 2024-05-20T18:37:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.117+0000 7fd979286640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd9740cd4d0 0x7fd9740cd8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:40906/0 (socket says 172.21.15.5:40906) 2024-05-20T18:37:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.117+0000 7fd973fff640 1 -- 172.21.15.5:0/4106849763 learned_addr learned my addr 172.21.15.5:0/4106849763 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.117+0000 7fd978a85640 1 --2- 172.21.15.5:0/4106849763 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9740d10e0 0x7fd9740cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.117+0000 7fd973fff640 1 -- 172.21.15.5:0/4106849763 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9740d10e0 msgr2=0x7fd9740cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.117+0000 7fd973fff640 1 --2- 172.21.15.5:0/4106849763 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9740d10e0 0x7fd9740cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.117+0000 7fd973fff640 1 -- 172.21.15.5:0/4106849763 >> v1:172.21.15.5:6789/0 conn(0x7fd9740cd4d0 legacy=0x7fd9740cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.117+0000 7fd973fff640 1 -- 172.21.15.5:0/4106849763 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9740d2360 con 0x7fd9740d14f0 2024-05-20T18:37:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.118+0000 7fd973fff640 1 --2- 172.21.15.5:0/4106849763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9740d14f0 0x7fd9740cf680 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fd95c009870 tx=0x7fd95c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3001c5ec48cbf068 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.118+0000 7fd9737fe640 1 -- 172.21.15.5:0/4106849763 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c037070 con 0x7fd9740d14f0 2024-05-20T18:37:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.118+0000 7fd9737fe640 1 -- 172.21.15.5:0/4106849763 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd95c032440 con 0x7fd9740d14f0 2024-05-20T18:37:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.118+0000 7fd9737fe640 1 -- 172.21.15.5:0/4106849763 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c032810 con 0x7fd9740d14f0 2024-05-20T18:37:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.119+0000 7fd97ace9640 1 -- 172.21.15.5:0/4106849763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9740d14f0 msgr2=0x7fd9740cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.119+0000 7fd97ace9640 1 --2- 172.21.15.5:0/4106849763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9740d14f0 0x7fd9740cf680 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fd95c009870 tx=0x7fd95c02a270 comp rx=0 tx=0).stop 2024-05-20T18:37:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.119+0000 7fd97ace9640 1 -- 172.21.15.5:0/4106849763 shutdown_connections 2024-05-20T18:37:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.119+0000 7fd97ace9640 1 --2- 172.21.15.5:0/4106849763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9740d14f0 0x7fd9740cf680 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.119+0000 7fd97ace9640 1 --2- 172.21.15.5:0/4106849763 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9740d10e0 0x7fd9740cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.119+0000 7fd97ace9640 1 -- 172.21.15.5:0/4106849763 >> 172.21.15.5:0/4106849763 conn(0x7fd9740b44d0 msgr2=0x7fd9740b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.119+0000 7fd97ace9640 1 -- 172.21.15.5:0/4106849763 shutdown_connections 2024-05-20T18:37:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.119+0000 7fd97ace9640 1 -- 172.21.15.5:0/4106849763 wait complete. 2024-05-20T18:37:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.121+0000 7fd97ace9640 1 Processor -- start 2024-05-20T18:37:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.121+0000 7fd97ace9640 1 -- start start 2024-05-20T18:37:24.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.122+0000 7fd97ace9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9740cd4d0 0x7fd9741f27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.122+0000 7fd97ace9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9740d10e0 0x7fd97414aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.122+0000 7fd97ace9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9740d14f0 0x7fd97414b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.122+0000 7fd97ace9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd9740b8440 con 0x7fd9740d14f0 2024-05-20T18:37:24.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.122+0000 7fd97ace9640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd9740b82c0 con 0x7fd9740cd4d0 2024-05-20T18:37:24.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.122+0000 7fd97ace9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd9740709f0 con 0x7fd9740d10e0 2024-05-20T18:37:24.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.122+0000 7fd979286640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9740d14f0 0x7fd97414b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.122+0000 7fd979286640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9740d14f0 0x7fd97414b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34496/0 (socket says 172.21.15.5:34496) 2024-05-20T18:37:24.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.122+0000 7fd979286640 1 -- 172.21.15.5:0/1961090059 learned_addr learned my addr 172.21.15.5:0/1961090059 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:24.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.122+0000 7fd973fff640 1 --2- 172.21.15.5:0/1961090059 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9740d10e0 0x7fd97414aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.122+0000 7fd978a85640 1 --2- 172.21.15.5:0/1961090059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9740cd4d0 0x7fd9741f27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.123+0000 7fd979286640 1 -- 172.21.15.5:0/1961090059 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9740d10e0 msgr2=0x7fd97414aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:24.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.123+0000 7fd979286640 1 --2- 172.21.15.5:0/1961090059 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9740d10e0 0x7fd97414aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.123+0000 7fd979286640 1 -- 172.21.15.5:0/1961090059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9740cd4d0 msgr2=0x7fd9741f27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:24.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.123+0000 7fd979286640 1 --2- 172.21.15.5:0/1961090059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9740cd4d0 0x7fd9741f27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.123+0000 7fd979286640 1 -- 172.21.15.5:0/1961090059 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9741e6010 con 0x7fd9740d14f0 2024-05-20T18:37:24.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.123+0000 7fd973fff640 1 --2- 172.21.15.5:0/1961090059 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9740d10e0 0x7fd97414aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:24.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.123+0000 7fd979286640 1 --2- 172.21.15.5:0/1961090059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9740d14f0 0x7fd97414b010 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00c760 tx=0x7fd96c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:24.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.123+0000 7fd9717fa640 1 -- 172.21.15.5:0/1961090059 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c017070 con 0x7fd9740d14f0 2024-05-20T18:37:24.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.123+0000 7fd9717fa640 1 -- 172.21.15.5:0/1961090059 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd96c00ce10 con 0x7fd9740d14f0 2024-05-20T18:37:24.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.123+0000 7fd9717fa640 1 -- 172.21.15.5:0/1961090059 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c00a840 con 0x7fd9740d14f0 2024-05-20T18:37:24.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.123+0000 7fd97ace9640 1 -- 172.21.15.5:0/1961090059 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9741e62f0 con 0x7fd9740d14f0 2024-05-20T18:37:24.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.123+0000 7fd97ace9640 1 -- 172.21.15.5:0/1961090059 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9741e68b0 con 0x7fd9740d14f0 2024-05-20T18:37:24.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.124+0000 7fd97ace9640 1 -- 172.21.15.5:0/1961090059 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9741e5d80 con 0x7fd9740d14f0 2024-05-20T18:37:24.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.125+0000 7fd9717fa640 1 -- 172.21.15.5:0/1961090059 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd96c01a070 con 0x7fd9740d14f0 2024-05-20T18:37:24.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.125+0000 7fd9717fa640 1 --2- 172.21.15.5:0/1961090059 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd95406be60 0x7fd95406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.125+0000 7fd9717fa640 1 -- 172.21.15.5:0/1961090059 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7fd96c08a540 con 0x7fd9740d14f0 2024-05-20T18:37:24.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.125+0000 7fd978a85640 1 --2- 172.21.15.5:0/1961090059 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd95406be60 0x7fd95406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.126+0000 7fd978a85640 1 --2- 172.21.15.5:0/1961090059 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd95406be60 0x7fd95406e370 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd964006fd0 tx=0x7fd964008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.130+0000 7fd9717fa640 1 -- 172.21.15.5:0/1961090059 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd96c058600 con 0x7fd9740d14f0 2024-05-20T18:37:24.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.313+0000 7fd97ace9640 1 -- 172.21.15.5:0/1961090059 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd974075b00 con 0x7fd9740d14f0 2024-05-20T18:37:24.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.313+0000 7fd9717fa640 1 -- 172.21.15.5:0/1961090059 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v119) ==== 100+0+11 (secure 0 0 0) 0x7fd96c05c900 con 0x7fd9740d14f0 2024-05-20T18:37:24.315 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:37:24.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.315+0000 7fd97ace9640 1 -- 172.21.15.5:0/1961090059 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd95406be60 msgr2=0x7fd95406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:24.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.315+0000 7fd97ace9640 1 --2- 172.21.15.5:0/1961090059 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd95406be60 0x7fd95406e370 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd964006fd0 tx=0x7fd964008040 comp rx=0 tx=0).stop 2024-05-20T18:37:24.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.315+0000 7fd97ace9640 1 -- 172.21.15.5:0/1961090059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9740d14f0 msgr2=0x7fd97414b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:24.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.315+0000 7fd97ace9640 1 --2- 172.21.15.5:0/1961090059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9740d14f0 0x7fd97414b010 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00c760 tx=0x7fd96c00cc20 comp rx=0 tx=0).stop 2024-05-20T18:37:24.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.315+0000 7fd97ace9640 1 -- 172.21.15.5:0/1961090059 shutdown_connections 2024-05-20T18:37:24.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.315+0000 7fd97ace9640 1 --2- 172.21.15.5:0/1961090059 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd95406be60 0x7fd95406e370 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.315+0000 7fd97ace9640 1 --2- 172.21.15.5:0/1961090059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9740d14f0 0x7fd97414b010 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.315+0000 7fd97ace9640 1 --2- 172.21.15.5:0/1961090059 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9740d10e0 0x7fd97414aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.315+0000 7fd97ace9640 1 --2- 172.21.15.5:0/1961090059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9740cd4d0 0x7fd9741f27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.315+0000 7fd97ace9640 1 -- 172.21.15.5:0/1961090059 >> 172.21.15.5:0/1961090059 conn(0x7fd9740b44d0 msgr2=0x7fd9740b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:24.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.315+0000 7fd97ace9640 1 -- 172.21.15.5:0/1961090059 shutdown_connections 2024-05-20T18:37:24.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.315+0000 7fd97ace9640 1 -- 172.21.15.5:0/1961090059 wait complete. 2024-05-20T18:37:24.328 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T18:37:24.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.442+0000 7f4675237640 1 Processor -- start 2024-05-20T18:37:24.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.442+0000 7f4675237640 1 -- start start 2024-05-20T18:37:24.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.444+0000 7f4675237640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4670162820 0x7f46700b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.444+0000 7f4675237640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f467006dae0 0x7f46700b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.444+0000 7f4675237640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46700d0630 con 0x7f46700b3e80 2024-05-20T18:37:24.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.444+0000 7f4675237640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f46700d39c0 con 0x7f4670162820 2024-05-20T18:37:24.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.444+0000 7f4675237640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f46700d3ca0 con 0x7f467006dae0 2024-05-20T18:37:24.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.444+0000 7f466ed9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4670162820 0x7f46700b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.444+0000 7f466ed9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4670162820 0x7f46700b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34500/0 (socket says 172.21.15.5:34500) 2024-05-20T18:37:24.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.444+0000 7f466ed9d640 1 -- 172.21.15.5:0/1280176171 learned_addr learned my addr 172.21.15.5:0/1280176171 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:24.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.444+0000 7f466f59e640 1 --2- 172.21.15.5:0/1280176171 >> v2:172.21.15.63:3300/0 conn(0x7f467006dae0 0x7f46700b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.444+0000 7f466ed9d640 1 -- 172.21.15.5:0/1280176171 >> v2:172.21.15.63:3300/0 conn(0x7f467006dae0 msgr2=0x7f46700b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:24.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.444+0000 7f466ed9d640 1 --2- 172.21.15.5:0/1280176171 >> v2:172.21.15.63:3300/0 conn(0x7f467006dae0 0x7f46700b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.444+0000 7f466ed9d640 1 -- 172.21.15.5:0/1280176171 >> v1:172.21.15.63:6789/0 conn(0x7f46700b3e80 legacy=0x7f467006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:24.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.444+0000 7f466ed9d640 1 -- 172.21.15.5:0/1280176171 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46700d3f80 con 0x7f4670162820 2024-05-20T18:37:24.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.445+0000 7f466ed9d640 1 --2- 172.21.15.5:0/1280176171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4670162820 0x7f46700b3940 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f467008e070 tx=0x7f466002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a370067f91d1bab server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:24.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.445+0000 7f466dd9b640 1 -- 172.21.15.5:0/1280176171 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660037070 con 0x7f4670162820 2024-05-20T18:37:24.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.445+0000 7f466dd9b640 1 -- 172.21.15.5:0/1280176171 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4660032440 con 0x7f4670162820 2024-05-20T18:37:24.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.445+0000 7f466dd9b640 1 -- 172.21.15.5:0/1280176171 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660032810 con 0x7f4670162820 2024-05-20T18:37:24.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.446+0000 7f4675237640 1 -- 172.21.15.5:0/1280176171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4670162820 msgr2=0x7f46700b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:24.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.446+0000 7f4675237640 1 --2- 172.21.15.5:0/1280176171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4670162820 0x7f46700b3940 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f467008e070 tx=0x7f466002a270 comp rx=0 tx=0).stop 2024-05-20T18:37:24.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.446+0000 7f4675237640 1 -- 172.21.15.5:0/1280176171 shutdown_connections 2024-05-20T18:37:24.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.446+0000 7f4675237640 1 --2- 172.21.15.5:0/1280176171 >> v2:172.21.15.63:3300/0 conn(0x7f467006dae0 0x7f46700b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.446+0000 7f4675237640 1 --2- 172.21.15.5:0/1280176171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4670162820 0x7f46700b3940 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.446+0000 7f4675237640 1 -- 172.21.15.5:0/1280176171 >> 172.21.15.5:0/1280176171 conn(0x7f46700d4bd0 msgr2=0x7f46700d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:24.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.446+0000 7f4675237640 1 -- 172.21.15.5:0/1280176171 shutdown_connections 2024-05-20T18:37:24.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.446+0000 7f4675237640 1 -- 172.21.15.5:0/1280176171 wait complete. 2024-05-20T18:37:24.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.448+0000 7f4675237640 1 Processor -- start 2024-05-20T18:37:24.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.448+0000 7f4675237640 1 -- start start 2024-05-20T18:37:24.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.449+0000 7f4675237640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46700b3e80 0x7f467014cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.449+0000 7f4675237640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4670162820 0x7f46700cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.449+0000 7f4675237640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46700ce1e0 0x7f46700ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.449+0000 7f4675237640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f46700d4780 con 0x7f46700ce1e0 2024-05-20T18:37:24.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.449+0000 7f4675237640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f46700d0300 con 0x7f4670162820 2024-05-20T18:37:24.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.449+0000 7f4675237640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f46700d4900 con 0x7f46700b3e80 2024-05-20T18:37:24.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.450+0000 7f466f59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46700ce1e0 0x7f46700ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.450+0000 7f466f59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46700ce1e0 0x7f46700ce5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34512/0 (socket says 172.21.15.5:34512) 2024-05-20T18:37:24.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.450+0000 7f466f59e640 1 -- 172.21.15.5:0/3385259812 learned_addr learned my addr 172.21.15.5:0/3385259812 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:24.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.450+0000 7f466ed9d640 1 --2- 172.21.15.5:0/3385259812 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46700b3e80 0x7f467014cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.450+0000 7f466e59c640 1 --2- 172.21.15.5:0/3385259812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4670162820 0x7f46700cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.450+0000 7f466f59e640 1 -- 172.21.15.5:0/3385259812 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46700b3e80 msgr2=0x7f467014cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:24.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.450+0000 7f466f59e640 1 --2- 172.21.15.5:0/3385259812 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46700b3e80 0x7f467014cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.450+0000 7f466f59e640 1 -- 172.21.15.5:0/3385259812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4670162820 msgr2=0x7f46700cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:24.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.450+0000 7f466f59e640 1 --2- 172.21.15.5:0/3385259812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4670162820 0x7f46700cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.450+0000 7f466f59e640 1 -- 172.21.15.5:0/3385259812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4670153a30 con 0x7f46700ce1e0 2024-05-20T18:37:24.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.450+0000 7f466e59c640 1 --2- 172.21.15.5:0/3385259812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4670162820 0x7f46700cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:24.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.450+0000 7f466f59e640 1 --2- 172.21.15.5:0/3385259812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46700ce1e0 0x7f46700ce5f0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f466800efc0 tx=0x7f466800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:24.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.450+0000 7f46677fe640 1 -- 172.21.15.5:0/3385259812 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668018020 con 0x7f46700ce1e0 2024-05-20T18:37:24.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.450+0000 7f46677fe640 1 -- 172.21.15.5:0/3385259812 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4668014480 con 0x7f46700ce1e0 2024-05-20T18:37:24.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.450+0000 7f46677fe640 1 -- 172.21.15.5:0/3385259812 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668014870 con 0x7f46700ce1e0 2024-05-20T18:37:24.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.450+0000 7f4675237640 1 -- 172.21.15.5:0/3385259812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46700c96f0 con 0x7f46700ce1e0 2024-05-20T18:37:24.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.450+0000 7f4675237640 1 -- 172.21.15.5:0/3385259812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46700c9cb0 con 0x7f46700ce1e0 2024-05-20T18:37:24.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.451+0000 7f4675237640 1 -- 172.21.15.5:0/3385259812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4670075b00 con 0x7f46700ce1e0 2024-05-20T18:37:24.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.452+0000 7f46677fe640 1 -- 172.21.15.5:0/3385259812 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4668005d10 con 0x7f46700ce1e0 2024-05-20T18:37:24.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.452+0000 7f46677fe640 1 --2- 172.21.15.5:0/3385259812 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f465006be30 0x7f465006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.452+0000 7f46677fe640 1 -- 172.21.15.5:0/3385259812 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7f466808bd20 con 0x7f46700ce1e0 2024-05-20T18:37:24.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.452+0000 7f466ed9d640 1 --2- 172.21.15.5:0/3385259812 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f465006be30 0x7f465006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.453+0000 7f466ed9d640 1 --2- 172.21.15.5:0/3385259812 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f465006be30 0x7f465006e340 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f4660034000 tx=0x7f466002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:24.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.457+0000 7f46677fe640 1 -- 172.21.15.5:0/3385259812 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4668022030 con 0x7f46700ce1e0 2024-05-20T18:37:24.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.654+0000 7f4675237640 1 -- 172.21.15.5:0/3385259812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f46700ba940 con 0x7f46700ce1e0 2024-05-20T18:37:24.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.655+0000 7f46677fe640 1 -- 172.21.15.5:0/3385259812 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v119) ==== 80+0+0 (secure 0 0 0) 0x7f4668010040 con 0x7f46700ce1e0 2024-05-20T18:37:24.656 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-05-20T18:37:24.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.657+0000 7f4675237640 1 -- 172.21.15.5:0/3385259812 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f465006be30 msgr2=0x7f465006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:24.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.657+0000 7f4675237640 1 --2- 172.21.15.5:0/3385259812 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f465006be30 0x7f465006e340 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f4660034000 tx=0x7f466002e000 comp rx=0 tx=0).stop 2024-05-20T18:37:24.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.657+0000 7f4675237640 1 -- 172.21.15.5:0/3385259812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46700ce1e0 msgr2=0x7f46700ce5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:24.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.657+0000 7f4675237640 1 --2- 172.21.15.5:0/3385259812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46700ce1e0 0x7f46700ce5f0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f466800efc0 tx=0x7f466800c490 comp rx=0 tx=0).stop 2024-05-20T18:37:24.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.657+0000 7f4675237640 1 -- 172.21.15.5:0/3385259812 shutdown_connections 2024-05-20T18:37:24.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.657+0000 7f4675237640 1 --2- 172.21.15.5:0/3385259812 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f465006be30 0x7f465006e340 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.657+0000 7f4675237640 1 --2- 172.21.15.5:0/3385259812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46700ce1e0 0x7f46700ce5f0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.657+0000 7f4675237640 1 --2- 172.21.15.5:0/3385259812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4670162820 0x7f46700cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.657+0000 7f4675237640 1 --2- 172.21.15.5:0/3385259812 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f46700b3e80 0x7f467014cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.657+0000 7f4675237640 1 -- 172.21.15.5:0/3385259812 >> 172.21.15.5:0/3385259812 conn(0x7f46700d4bd0 msgr2=0x7f467015d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:24.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.657+0000 7f4675237640 1 -- 172.21.15.5:0/3385259812 shutdown_connections 2024-05-20T18:37:24.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.657+0000 7f4675237640 1 -- 172.21.15.5:0/3385259812 wait complete. 2024-05-20T18:37:24.670 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-20T18:37:24.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:24.670 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-20T18:37:24.741 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:37:24.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:24.741 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-20T18:37:24.762 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-05-20T18:37:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.918+0000 7f8561aca640 1 Processor -- start 2024-05-20T18:37:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.918+0000 7f8561aca640 1 -- start start 2024-05-20T18:37:24.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.919+0000 7f8561aca640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f855c0d4400 0x7f855c15f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.919+0000 7f8561aca640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f855c06cf00 0x7f855c06d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.919+0000 7f8561aca640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f855c0d2780 0x7f855c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.919+0000 7f8561aca640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f855c0d3f70 con 0x7f855c06cf00 2024-05-20T18:37:24.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.919+0000 7f8561aca640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f855c08d2a0 con 0x7f855c0d4400 2024-05-20T18:37:24.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.919+0000 7f8561aca640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f855c06d850 con 0x7f855c0d2780 2024-05-20T18:37:24.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.919+0000 7f8552ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f855c06cf00 0x7f855c06d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.919+0000 7f8552ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f855c06cf00 0x7f855c06d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33844/0 (socket says 172.21.15.5:33844) 2024-05-20T18:37:24.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.919+0000 7f8552ffd640 1 -- 172.21.15.5:0/1962538722 learned_addr learned my addr 172.21.15.5:0/1962538722 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:24.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.919+0000 7f855bfff640 1 --2- 172.21.15.5:0/1962538722 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f855c0d2780 0x7f855c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.919+0000 7f855b7fe640 1 --2- 172.21.15.5:0/1962538722 >> v2:172.21.15.63:3300/0 conn(0x7f855c0d4400 0x7f855c15f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.920+0000 7f8552ffd640 1 -- 172.21.15.5:0/1962538722 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f855c0d2780 msgr2=0x7f855c0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:24.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.920+0000 7f8552ffd640 1 --2- 172.21.15.5:0/1962538722 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f855c0d2780 0x7f855c0d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.920+0000 7f8552ffd640 1 -- 172.21.15.5:0/1962538722 >> v2:172.21.15.63:3300/0 conn(0x7f855c0d4400 msgr2=0x7f855c15f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:24.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.920+0000 7f8552ffd640 1 --2- 172.21.15.5:0/1962538722 >> v2:172.21.15.63:3300/0 conn(0x7f855c0d4400 0x7f855c15f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.920+0000 7f8552ffd640 1 -- 172.21.15.5:0/1962538722 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f855c0d30d0 con 0x7f855c06cf00 2024-05-20T18:37:24.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.920+0000 7f855b7fe640 1 --2- 172.21.15.5:0/1962538722 >> v2:172.21.15.63:3300/0 conn(0x7f855c0d4400 0x7f855c15f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.920+0000 7f8552ffd640 1 --2- 172.21.15.5:0/1962538722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f855c06cf00 0x7f855c06d310 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f855c0d5ab0 tx=0x7f854402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980cddd3093d7f80 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.920+0000 7f855affd640 1 -- 172.21.15.5:0/1962538722 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8544037070 con 0x7f855c06cf00 2024-05-20T18:37:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.921+0000 7f855affd640 1 -- 172.21.15.5:0/1962538722 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8544032440 con 0x7f855c06cf00 2024-05-20T18:37:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.921+0000 7f855affd640 1 -- 172.21.15.5:0/1962538722 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8544032810 con 0x7f855c06cf00 2024-05-20T18:37:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.921+0000 7f8561aca640 1 -- 172.21.15.5:0/1962538722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f855c06cf00 msgr2=0x7f855c06d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:24.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.921+0000 7f8561aca640 1 --2- 172.21.15.5:0/1962538722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f855c06cf00 0x7f855c06d310 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f855c0d5ab0 tx=0x7f854402a270 comp rx=0 tx=0).stop 2024-05-20T18:37:24.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.921+0000 7f8561aca640 1 -- 172.21.15.5:0/1962538722 shutdown_connections 2024-05-20T18:37:24.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.921+0000 7f8561aca640 1 --2- 172.21.15.5:0/1962538722 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f855c0d2780 0x7f855c0d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.921+0000 7f8561aca640 1 --2- 172.21.15.5:0/1962538722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f855c06cf00 0x7f855c06d310 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.921+0000 7f8561aca640 1 --2- 172.21.15.5:0/1962538722 >> v2:172.21.15.63:3300/0 conn(0x7f855c0d4400 0x7f855c15f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.921+0000 7f8561aca640 1 -- 172.21.15.5:0/1962538722 >> 172.21.15.5:0/1962538722 conn(0x7f855c0d4bb0 msgr2=0x7f855c0d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:24.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.921+0000 7f8561aca640 1 -- 172.21.15.5:0/1962538722 shutdown_connections 2024-05-20T18:37:24.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.921+0000 7f8561aca640 1 -- 172.21.15.5:0/1962538722 wait complete. 2024-05-20T18:37:24.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.923+0000 7f8561aca640 1 Processor -- start 2024-05-20T18:37:24.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.923+0000 7f8561aca640 1 -- start start 2024-05-20T18:37:24.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.925+0000 7f8561aca640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f855c06cf00 0x7f855c1f5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.925+0000 7f8561aca640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f855c0d2780 0x7f855c153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.925+0000 7f8561aca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f855c0d4400 0x7f855c144760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.925+0000 7f8561aca640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f855c0d3640 con 0x7f855c0d4400 2024-05-20T18:37:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.925+0000 7f8561aca640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f855c0d3360 con 0x7f855c0d2780 2024-05-20T18:37:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.925+0000 7f8561aca640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f855c0d3920 con 0x7f855c06cf00 2024-05-20T18:37:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.925+0000 7f855bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f855c0d4400 0x7f855c144760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.925+0000 7f855bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f855c0d4400 0x7f855c144760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33854/0 (socket says 172.21.15.5:33854) 2024-05-20T18:37:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.925+0000 7f855bfff640 1 -- 172.21.15.5:0/3963885664 learned_addr learned my addr 172.21.15.5:0/3963885664 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.925+0000 7f8552ffd640 1 --2- 172.21.15.5:0/3963885664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f855c0d2780 0x7f855c153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.925+0000 7f855b7fe640 1 --2- 172.21.15.5:0/3963885664 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f855c06cf00 0x7f855c1f5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.925+0000 7f855bfff640 1 -- 172.21.15.5:0/3963885664 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f855c06cf00 msgr2=0x7f855c1f5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.925+0000 7f855bfff640 1 --2- 172.21.15.5:0/3963885664 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f855c06cf00 0x7f855c1f5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.925+0000 7f855bfff640 1 -- 172.21.15.5:0/3963885664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f855c0d2780 msgr2=0x7f855c153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.925+0000 7f855bfff640 1 --2- 172.21.15.5:0/3963885664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f855c0d2780 0x7f855c153db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.925+0000 7f855bfff640 1 -- 172.21.15.5:0/3963885664 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f855c1542f0 con 0x7f855c0d4400 2024-05-20T18:37:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.925+0000 7f855b7fe640 1 --2- 172.21.15.5:0/3963885664 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f855c06cf00 0x7f855c1f5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.925+0000 7f8552ffd640 1 --2- 172.21.15.5:0/3963885664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f855c0d2780 0x7f855c153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:24.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.925+0000 7f855bfff640 1 --2- 172.21.15.5:0/3963885664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f855c0d4400 0x7f855c144760 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f8554007d10 tx=0x7f855400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:24.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.926+0000 7f8558ff9640 1 -- 172.21.15.5:0/3963885664 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8554014420 con 0x7f855c0d4400 2024-05-20T18:37:24.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.926+0000 7f8558ff9640 1 -- 172.21.15.5:0/3963885664 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8554019070 con 0x7f855c0d4400 2024-05-20T18:37:24.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.926+0000 7f8561aca640 1 -- 172.21.15.5:0/3963885664 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f855c155f60 con 0x7f855c0d4400 2024-05-20T18:37:24.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.926+0000 7f8558ff9640 1 -- 172.21.15.5:0/3963885664 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8554014800 con 0x7f855c0d4400 2024-05-20T18:37:24.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.926+0000 7f8561aca640 1 -- 172.21.15.5:0/3963885664 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f855c156520 con 0x7f855c0d4400 2024-05-20T18:37:24.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.927+0000 7f8558ff9640 1 -- 172.21.15.5:0/3963885664 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f85540149a0 con 0x7f855c0d4400 2024-05-20T18:37:24.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.928+0000 7f8558ff9640 1 --2- 172.21.15.5:0/3963885664 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f853c06be30 0x7f853c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.928+0000 7f8558ff9640 1 -- 172.21.15.5:0/3963885664 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7f855408b700 con 0x7f855c0d4400 2024-05-20T18:37:24.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.928+0000 7f855b7fe640 1 --2- 172.21.15.5:0/3963885664 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f853c06be30 0x7f853c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.928+0000 7f855b7fe640 1 --2- 172.21.15.5:0/3963885664 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f853c06be30 0x7f853c06e340 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f854c00a990 tx=0x7f854c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:24.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.929+0000 7f856896f740 1 Processor -- start 2024-05-20T18:37:24.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.929+0000 7f856896f740 1 -- start start 2024-05-20T18:37:24.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.930+0000 7f856896f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561620418390 0x5616207e2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.930+0000 7f856896f740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5616203ef4d0 0x5616207fd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.930+0000 7f856896f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616203f76b0 0x5616207fddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.930+0000 7f856896f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56162071cd00 con 0x5616203f76b0 2024-05-20T18:37:24.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.930+0000 7f856896f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5616206c44c0 con 0x561620418390 2024-05-20T18:37:24.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.930+0000 7f856896f740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x561620718cc0 con 0x5616203ef4d0 2024-05-20T18:37:24.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.931+0000 7f855bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616203f76b0 0x5616207fddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.931+0000 7f855bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616203f76b0 0x5616207fddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33862/0 (socket says 172.21.15.5:33862) 2024-05-20T18:37:24.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.931+0000 7f855bfff640 1 -- 172.21.15.5:0/2494641026 learned_addr learned my addr 172.21.15.5:0/2494641026 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:24.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.931+0000 7f8552ffd640 1 --2- 172.21.15.5:0/2494641026 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5616203ef4d0 0x5616207fd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.931+0000 7f855b7fe640 1 --2- 172.21.15.5:0/2494641026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561620418390 0x5616207e2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.931+0000 7f855bfff640 1 -- 172.21.15.5:0/2494641026 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5616203ef4d0 msgr2=0x5616207fd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:24.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.931+0000 7f855bfff640 1 --2- 172.21.15.5:0/2494641026 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5616203ef4d0 0x5616207fd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.931+0000 7f855bfff640 1 -- 172.21.15.5:0/2494641026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561620418390 msgr2=0x5616207e2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:24.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.931+0000 7f855bfff640 1 --2- 172.21.15.5:0/2494641026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561620418390 0x5616207e2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.931+0000 7f855bfff640 1 -- 172.21.15.5:0/2494641026 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616204fa7d0 con 0x5616203f76b0 2024-05-20T18:37:24.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.931+0000 7f855bfff640 1 --2- 172.21.15.5:0/2494641026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616203f76b0 0x5616207fddd0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f855400c040 tx=0x7f85540592b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663ba92f14d3a5b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:24.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.931+0000 7f85517fa640 1 -- 172.21.15.5:0/2494641026 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8554062400 con 0x5616203f76b0 2024-05-20T18:37:24.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.931+0000 7f85517fa640 1 -- 172.21.15.5:0/2494641026 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8554057b20 con 0x5616203f76b0 2024-05-20T18:37:24.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.931+0000 7f85517fa640 1 -- 172.21.15.5:0/2494641026 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855405ab50 con 0x5616203f76b0 2024-05-20T18:37:24.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.932+0000 7f856896f740 1 -- 172.21.15.5:0/2494641026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616203f76b0 msgr2=0x5616207fddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:24.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.932+0000 7f856896f740 1 --2- 172.21.15.5:0/2494641026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616203f76b0 0x5616207fddd0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f855400c040 tx=0x7f85540592b0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.932+0000 7f856896f740 1 -- 172.21.15.5:0/2494641026 shutdown_connections 2024-05-20T18:37:24.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.932+0000 7f856896f740 1 --2- 172.21.15.5:0/2494641026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616203f76b0 0x5616207fddd0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.932+0000 7f856896f740 1 --2- 172.21.15.5:0/2494641026 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5616203ef4d0 0x5616207fd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.932+0000 7f856896f740 1 --2- 172.21.15.5:0/2494641026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561620418390 0x5616207e2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.932+0000 7f856896f740 1 -- 172.21.15.5:0/2494641026 >> 172.21.15.5:0/2494641026 conn(0x56162040c4e0 msgr2=0x5616207b81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:24.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.932+0000 7f856896f740 1 -- 172.21.15.5:0/2494641026 shutdown_connections 2024-05-20T18:37:24.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.932+0000 7f856896f740 1 -- 172.21.15.5:0/2494641026 wait complete. 2024-05-20T18:37:24.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.933+0000 7f856896f740 1 -- start start 2024-05-20T18:37:24.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.933+0000 7f856896f740 1 Processor -- start 2024-05-20T18:37:24.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.934+0000 7f856896f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616203ef4d0 0x56162080e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.934+0000 7f856896f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5616203f76b0 0x5616207e5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.934+0000 7f856896f740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561620418390 0x5616207f69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.935+0000 7f856896f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5616206d7b90 con 0x5616203ef4d0 2024-05-20T18:37:24.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.935+0000 7f856896f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56162080ec60 con 0x5616203f76b0 2024-05-20T18:37:24.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.935+0000 7f856896f740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5616207e6170 con 0x561620418390 2024-05-20T18:37:24.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.935+0000 7f856896f740 10 client.0 _ll_register_callbacks cb 0x561620713da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:37:24.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.935+0000 7f855b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616203ef4d0 0x56162080e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.935+0000 7f855b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616203ef4d0 0x56162080e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33876/0 (socket says 172.21.15.5:33876) 2024-05-20T18:37:24.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.935+0000 7f855b7fe640 1 -- 172.21.15.5:0/4176559991 learned_addr learned my addr 172.21.15.5:0/4176559991 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:24.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.935+0000 7f8552ffd640 1 --2- 172.21.15.5:0/4176559991 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5616203f76b0 0x5616207e5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.935+0000 7f855bfff640 1 --2- 172.21.15.5:0/4176559991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561620418390 0x5616207f69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.935+0000 7f855b7fe640 1 -- 172.21.15.5:0/4176559991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561620418390 msgr2=0x5616207f69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:24.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.935+0000 7f855b7fe640 1 --2- 172.21.15.5:0/4176559991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561620418390 0x5616207f69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.935+0000 7f855b7fe640 1 -- 172.21.15.5:0/4176559991 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5616203f76b0 msgr2=0x5616207e5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:24.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.935+0000 7f855b7fe640 1 --2- 172.21.15.5:0/4176559991 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5616203f76b0 0x5616207e5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.935+0000 7f855b7fe640 1 -- 172.21.15.5:0/4176559991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616204303f0 con 0x5616203ef4d0 2024-05-20T18:37:24.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.935+0000 7f855b7fe640 1 --2- 172.21.15.5:0/4176559991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616203ef4d0 0x56162080e3d0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f854c0144e0 tx=0x7f854c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432fdbfa6967cfc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:24.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.935+0000 7f853a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:37:24.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.935+0000 7f853a7fc640 1 -- 172.21.15.5:0/4176559991 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854c015820 con 0x5616203ef4d0 2024-05-20T18:37:24.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.935+0000 7f853a7fc640 1 -- 172.21.15.5:0/4176559991 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7f854c0159c0 con 0x5616203ef4d0 2024-05-20T18:37:24.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.935+0000 7f856896f740 1 -- 172.21.15.5:0/4176559991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8554057b20 con 0x5616203ef4d0 2024-05-20T18:37:24.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.936+0000 7f853a7fc640 1 -- 172.21.15.5:0/4176559991 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f854c015bc0 con 0x5616203ef4d0 2024-05-20T18:37:24.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.936+0000 7f856896f740 10 client.9210 fetch_fsmap learned FSMap version 98 2024-05-20T18:37:24.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.936+0000 7f856896f740 1 -- 172.21.15.5:0/4176559991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) -- 0x5616203f00f0 con 0x5616203ef4d0 2024-05-20T18:37:24.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.936+0000 7f853a7fc640 1 -- 172.21.15.5:0/4176559991 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8554057b20 con 0x5616203ef4d0 2024-05-20T18:37:24.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.936+0000 7f853a7fc640 1 -- 172.21.15.5:0/4176559991 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 98) ==== 3494+0+0 (secure 0 0 0) 0x7f854c005990 con 0x5616203ef4d0 2024-05-20T18:37:24.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.936+0000 7f856896f740 10 client.9210 fetch_fsmap finished waiting for FSMap version 98 2024-05-20T18:37:24.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.936+0000 7f856896f740 10 client.9210 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-05-20T18:37:24.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.936+0000 7f856896f740 20 client.9210 populate_metadata read hostname 'smithi005' 2024-05-20T18:37:24.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.936+0000 7f856896f740 1 --2- 172.21.15.5:0/4176559991 >> [v2:172.21.15.63:6833/2352520472,v1:172.21.15.63:6835/2352520472] conn(0x5616203d6f60 0x5616207f8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.936+0000 7f856896f740 4 client.9210 mds_command: new command op to 9030 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:37:24.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.936+0000 7f856896f740 1 -- 172.21.15.5:0/4176559991 --> [v2:172.21.15.63:6833/2352520472,v1:172.21.15.63:6835/2352520472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56162072a430 con 0x5616203d6f60 2024-05-20T18:37:24.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.936+0000 7f8552ffd640 1 --2- 172.21.15.5:0/4176559991 >> [v2:172.21.15.63:6833/2352520472,v1:172.21.15.63:6835/2352520472] conn(0x5616203d6f60 0x5616207f8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.937+0000 7f8552ffd640 1 --2- 172.21.15.5:0/4176559991 >> [v2:172.21.15.63:6833/2352520472,v1:172.21.15.63:6835/2352520472] conn(0x5616203d6f60 0x5616207f8f80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a2d612fe9da5670 server_cookie=c1044009f802b5b8 in_seq=0 out_seq=0 2024-05-20T18:37:24.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.937+0000 7f853a7fc640 10 client.9210 ms_handle_connect on v2:172.21.15.63:6833/2352520472 2024-05-20T18:37:24.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.937+0000 7f853a7fc640 1 -- 172.21.15.5:0/4176559991 <== mds.0 v2:172.21.15.63:6833/2352520472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56162072a430 con 0x5616203d6f60 2024-05-20T18:37:24.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.937+0000 7f853a7fc640 10 client.9210 handle_command_reply: tid=0 2024-05-20T18:37:24.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.937+0000 7f856896f740 1 client.9210 shutdown 2024-05-20T18:37:24.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.938+0000 7f856896f740 1 -- 172.21.15.5:0/4176559991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616203ef4d0 msgr2=0x56162080e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:24.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.938+0000 7f856896f740 1 --2- 172.21.15.5:0/4176559991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616203ef4d0 0x56162080e3d0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f854c0144e0 tx=0x7f854c004730 comp rx=0 tx=0).stop 2024-05-20T18:37:24.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.938+0000 7f856896f740 1 -- 172.21.15.5:0/4176559991 shutdown_connections 2024-05-20T18:37:24.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.938+0000 7f856896f740 1 --2- 172.21.15.5:0/4176559991 >> [v2:172.21.15.63:6833/2352520472,v1:172.21.15.63:6835/2352520472] conn(0x5616203d6f60 0x5616207f8f80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.938+0000 7f856896f740 1 --2- 172.21.15.5:0/4176559991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561620418390 0x5616207f69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.938+0000 7f856896f740 1 --2- 172.21.15.5:0/4176559991 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5616203f76b0 0x5616207e5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.938+0000 7f853a7fc640 0 client.9210 ms_handle_reset on v2:172.21.15.63:6833/2352520472 2024-05-20T18:37:24.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.938+0000 7f856896f740 1 --2- 172.21.15.5:0/4176559991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616203ef4d0 0x56162080e3d0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.938+0000 7f856896f740 1 -- 172.21.15.5:0/4176559991 >> 172.21.15.5:0/4176559991 conn(0x56162040c4e0 msgr2=0x5616207b6670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:24.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.938+0000 7f856896f740 1 -- 172.21.15.5:0/4176559991 shutdown_connections 2024-05-20T18:37:24.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.938+0000 7f856896f740 1 -- 172.21.15.5:0/4176559991 wait complete. 2024-05-20T18:37:24.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.938+0000 7f856896f740 20 client.9210 trim_cache size 0 max 16384 2024-05-20T18:37:24.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.953+0000 7f856896f740 1 Processor -- start 2024-05-20T18:37:24.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.953+0000 7f856896f740 1 -- start start 2024-05-20T18:37:24.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.955+0000 7f856896f740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5616203ef4d0 0x5616207b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.955+0000 7f856896f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616203f76b0 0x5616207b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.955+0000 7f856896f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561620418390 0x5616207b8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.955+0000 7f856896f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5616207f6da0 con 0x5616203f76b0 2024-05-20T18:37:24.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.955+0000 7f856896f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f855c2023e0 con 0x561620418390 2024-05-20T18:37:24.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.955+0000 7f856896f740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f855c201420 con 0x5616203ef4d0 2024-05-20T18:37:24.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.955+0000 7f8552ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616203f76b0 0x5616207b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.955+0000 7f8552ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616203f76b0 0x5616207b8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33892/0 (socket says 172.21.15.5:33892) 2024-05-20T18:37:24.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.955+0000 7f8552ffd640 1 -- 172.21.15.5:0/3386055418 learned_addr learned my addr 172.21.15.5:0/3386055418 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:24.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.955+0000 7f855bfff640 1 --2- 172.21.15.5:0/3386055418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561620418390 0x5616207b8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.955+0000 7f855b7fe640 1 --2- 172.21.15.5:0/3386055418 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5616203ef4d0 0x5616207b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.955+0000 7f8552ffd640 1 -- 172.21.15.5:0/3386055418 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5616203ef4d0 msgr2=0x5616207b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:24.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.955+0000 7f8552ffd640 1 --2- 172.21.15.5:0/3386055418 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5616203ef4d0 0x5616207b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.955+0000 7f8552ffd640 1 -- 172.21.15.5:0/3386055418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561620418390 msgr2=0x5616207b8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:24.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.955+0000 7f8552ffd640 1 --2- 172.21.15.5:0/3386055418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561620418390 0x5616207b8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.955+0000 7f8552ffd640 1 -- 172.21.15.5:0/3386055418 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616207a49a0 con 0x5616203f76b0 2024-05-20T18:37:24.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.955+0000 7f855b7fe640 1 --2- 172.21.15.5:0/3386055418 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5616203ef4d0 0x5616207b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:24.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.955+0000 7f8552ffd640 1 --2- 172.21.15.5:0/3386055418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616203f76b0 0x5616207b8a10 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f8544009960 tx=0x7f854402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4499d7c00de0ea77 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:24.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.956+0000 7f8550ff9640 1 -- 172.21.15.5:0/3386055418 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8544037040 con 0x5616203f76b0 2024-05-20T18:37:24.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.956+0000 7f8550ff9640 1 -- 172.21.15.5:0/3386055418 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8544042070 con 0x5616203f76b0 2024-05-20T18:37:24.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.956+0000 7f8550ff9640 1 -- 172.21.15.5:0/3386055418 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8544032ce0 con 0x5616203f76b0 2024-05-20T18:37:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.956+0000 7f856896f740 1 -- 172.21.15.5:0/3386055418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616203f76b0 msgr2=0x5616207b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.956+0000 7f856896f740 1 --2- 172.21.15.5:0/3386055418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616203f76b0 0x5616207b8a10 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f8544009960 tx=0x7f854402e000 comp rx=0 tx=0).stop 2024-05-20T18:37:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.956+0000 7f856896f740 1 -- 172.21.15.5:0/3386055418 shutdown_connections 2024-05-20T18:37:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.956+0000 7f856896f740 1 --2- 172.21.15.5:0/3386055418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561620418390 0x5616207b8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.956+0000 7f856896f740 1 --2- 172.21.15.5:0/3386055418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616203f76b0 0x5616207b8a10 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.956+0000 7f856896f740 1 --2- 172.21.15.5:0/3386055418 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5616203ef4d0 0x5616207b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.957+0000 7f856896f740 1 -- 172.21.15.5:0/3386055418 >> 172.21.15.5:0/3386055418 conn(0x56162040c4e0 msgr2=0x5616207e2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.957+0000 7f856896f740 1 -- 172.21.15.5:0/3386055418 shutdown_connections 2024-05-20T18:37:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.957+0000 7f856896f740 1 -- 172.21.15.5:0/3386055418 wait complete. 2024-05-20T18:37:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.957+0000 7f856896f740 1 -- start start 2024-05-20T18:37:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.958+0000 7f856896f740 1 Processor -- start 2024-05-20T18:37:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.959+0000 7f856896f740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5616203ef4d0 0x5616207dfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.959+0000 7f856896f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5616203f76b0 0x5616207b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.959+0000 7f856896f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561620418390 0x56162087e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.959+0000 7f856896f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f855c152fe0 con 0x561620418390 2024-05-20T18:37:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.959+0000 7f856896f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5616207dd5e0 con 0x5616203f76b0 2024-05-20T18:37:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.959+0000 7f856896f740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5616207f99b0 con 0x5616203ef4d0 2024-05-20T18:37:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.959+0000 7f856896f740 10 client.0 _ll_register_callbacks cb 0x7f854c00a6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:37:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.959+0000 7f855bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561620418390 0x56162087e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.959+0000 7f855bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561620418390 0x56162087e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33908/0 (socket says 172.21.15.5:33908) 2024-05-20T18:37:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.959+0000 7f855bfff640 1 -- 172.21.15.5:0/2369320047 learned_addr learned my addr 172.21.15.5:0/2369320047 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.959+0000 7f8552ffd640 1 --2- 172.21.15.5:0/2369320047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5616203f76b0 0x5616207b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.959+0000 7f855b7fe640 1 --2- 172.21.15.5:0/2369320047 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5616203ef4d0 0x5616207dfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.959+0000 7f855bfff640 1 -- 172.21.15.5:0/2369320047 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5616203ef4d0 msgr2=0x5616207dfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.959+0000 7f855bfff640 1 --2- 172.21.15.5:0/2369320047 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5616203ef4d0 0x5616207dfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.959+0000 7f855bfff640 1 -- 172.21.15.5:0/2369320047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5616203f76b0 msgr2=0x5616207b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.959+0000 7f855bfff640 1 --2- 172.21.15.5:0/2369320047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5616203f76b0 0x5616207b30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.959+0000 7f855bfff640 1 -- 172.21.15.5:0/2369320047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56162087d9c0 con 0x561620418390 2024-05-20T18:37:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.959+0000 7f855bfff640 1 --2- 172.21.15.5:0/2369320047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561620418390 0x56162087e460 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f8554089af0 tx=0x7f855408b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=361c2f2390458590 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.959+0000 7f853a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:37:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.960+0000 7f853a7fc640 1 -- 172.21.15.5:0/2369320047 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8554089070 con 0x561620418390 2024-05-20T18:37:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.960+0000 7f853a7fc640 1 -- 172.21.15.5:0/2369320047 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7f8554057b20 con 0x561620418390 2024-05-20T18:37:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.960+0000 7f856896f740 1 -- 172.21.15.5:0/2369320047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8544042070 con 0x561620418390 2024-05-20T18:37:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.960+0000 7f853a7fc640 1 -- 172.21.15.5:0/2369320047 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f8554062400 con 0x561620418390 2024-05-20T18:37:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.960+0000 7f856896f740 10 client.9216 fetch_fsmap learned FSMap version 98 2024-05-20T18:37:24.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.960+0000 7f856896f740 1 -- 172.21.15.5:0/2369320047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) -- 0x561620879fa0 con 0x561620418390 2024-05-20T18:37:24.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.960+0000 7f853a7fc640 1 -- 172.21.15.5:0/2369320047 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8544042070 con 0x561620418390 2024-05-20T18:37:24.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.960+0000 7f853a7fc640 1 -- 172.21.15.5:0/2369320047 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 98) ==== 3494+0+0 (secure 0 0 0) 0x7f8554055e90 con 0x561620418390 2024-05-20T18:37:24.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.960+0000 7f856896f740 10 client.9216 fetch_fsmap finished waiting for FSMap version 98 2024-05-20T18:37:24.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.960+0000 7f856896f740 10 client.9216 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-05-20T18:37:24.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.960+0000 7f856896f740 20 client.9216 populate_metadata read hostname 'smithi005' 2024-05-20T18:37:24.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.960+0000 7f856896f740 1 --2- 172.21.15.5:0/2369320047 >> [v2:172.21.15.63:6833/2352520472,v1:172.21.15.63:6835/2352520472] conn(0x5616207dc9b0 0x5616207dcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:24.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.960+0000 7f856896f740 4 client.9216 mds_command: new command op to 9030 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T18:37:24.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.960+0000 7f856896f740 1 -- 172.21.15.5:0/2369320047 --> [v2:172.21.15.63:6833/2352520472,v1:172.21.15.63:6835/2352520472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561620724780 con 0x5616207dc9b0 2024-05-20T18:37:24.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.960+0000 7f855b7fe640 1 --2- 172.21.15.5:0/2369320047 >> [v2:172.21.15.63:6833/2352520472,v1:172.21.15.63:6835/2352520472] conn(0x5616207dc9b0 0x5616207dcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:24.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.961+0000 7f855b7fe640 1 --2- 172.21.15.5:0/2369320047 >> [v2:172.21.15.63:6833/2352520472,v1:172.21.15.63:6835/2352520472] conn(0x5616207dc9b0 0x5616207dcdc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4716bf4b30e32e server_cookie=118e9f3d8eb6d3f0 in_seq=0 out_seq=0 2024-05-20T18:37:24.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.961+0000 7f853a7fc640 10 client.9216 ms_handle_connect on v2:172.21.15.63:6833/2352520472 2024-05-20T18:37:24.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.972+0000 7f853a7fc640 1 -- 172.21.15.5:0/2369320047 <== mds.0 v2:172.21.15.63:6833/2352520472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561620724780 con 0x5616207dc9b0 2024-05-20T18:37:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.972+0000 7f853a7fc640 10 client.9216 handle_command_reply: tid=0 2024-05-20T18:37:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.972+0000 7f856896f740 1 client.9216 shutdown 2024-05-20T18:37:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.972+0000 7f856896f740 1 -- 172.21.15.5:0/2369320047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561620418390 msgr2=0x56162087e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.972+0000 7f856896f740 1 --2- 172.21.15.5:0/2369320047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561620418390 0x56162087e460 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f8554089af0 tx=0x7f855408b040 comp rx=0 tx=0).stop 2024-05-20T18:37:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.972+0000 7f856896f740 1 -- 172.21.15.5:0/2369320047 shutdown_connections 2024-05-20T18:37:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.972+0000 7f856896f740 1 --2- 172.21.15.5:0/2369320047 >> [v2:172.21.15.63:6833/2352520472,v1:172.21.15.63:6835/2352520472] conn(0x5616207dc9b0 0x5616207dcdc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.972+0000 7f856896f740 1 --2- 172.21.15.5:0/2369320047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561620418390 0x56162087e460 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.972+0000 7f856896f740 1 --2- 172.21.15.5:0/2369320047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5616203f76b0 0x5616207b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.972+0000 7f856896f740 1 --2- 172.21.15.5:0/2369320047 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5616203ef4d0 0x5616207dfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.972+0000 7f853a7fc640 0 client.9216 ms_handle_reset on v2:172.21.15.63:6833/2352520472 2024-05-20T18:37:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.972+0000 7f856896f740 1 -- 172.21.15.5:0/2369320047 >> 172.21.15.5:0/2369320047 conn(0x56162040c4e0 msgr2=0x561620820c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:24.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.972+0000 7f856896f740 1 -- 172.21.15.5:0/2369320047 shutdown_connections 2024-05-20T18:37:24.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.972+0000 7f856896f740 1 -- 172.21.15.5:0/2369320047 wait complete. 2024-05-20T18:37:24.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.972+0000 7f856896f740 20 client.9216 trim_cache size 0 max 16384 2024-05-20T18:37:24.975 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:37:24.975 INFO:teuthology.orchestra.run.smithi005.stdout: "message": "", 2024-05-20T18:37:24.975 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0 2024-05-20T18:37:24.975 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:37:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.973+0000 7f8561aca640 1 -- 172.21.15.5:0/3963885664 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f853c06be30 msgr2=0x7f853c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.973+0000 7f8561aca640 1 --2- 172.21.15.5:0/3963885664 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f853c06be30 0x7f853c06e340 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f854c00a990 tx=0x7f854c008040 comp rx=0 tx=0).stop 2024-05-20T18:37:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.973+0000 7f8561aca640 1 -- 172.21.15.5:0/3963885664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f855c0d4400 msgr2=0x7f855c144760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.973+0000 7f8561aca640 1 --2- 172.21.15.5:0/3963885664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f855c0d4400 0x7f855c144760 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f8554007d10 tx=0x7f855400e4b0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.973+0000 7f8561aca640 1 -- 172.21.15.5:0/3963885664 shutdown_connections 2024-05-20T18:37:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.973+0000 7f8561aca640 1 --2- 172.21.15.5:0/3963885664 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f853c06be30 0x7f853c06e340 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.973+0000 7f8561aca640 1 --2- 172.21.15.5:0/3963885664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f855c0d4400 0x7f855c144760 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.973+0000 7f8561aca640 1 --2- 172.21.15.5:0/3963885664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f855c0d2780 0x7f855c153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.973+0000 7f8561aca640 1 --2- 172.21.15.5:0/3963885664 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f855c06cf00 0x7f855c1f5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:24.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.973+0000 7f8561aca640 1 -- 172.21.15.5:0/3963885664 >> 172.21.15.5:0/3963885664 conn(0x7f855c0d4bb0 msgr2=0x7f855c15d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:24.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.973+0000 7f8561aca640 1 -- 172.21.15.5:0/3963885664 shutdown_connections 2024-05-20T18:37:24.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:24.973+0000 7f8561aca640 1 -- 172.21.15.5:0/3963885664 wait complete. 2024-05-20T18:37:24.985 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-20T18:37:25.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.101+0000 7f2e728a0640 1 Processor -- start 2024-05-20T18:37:25.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.102+0000 7f2e728a0640 1 -- start start 2024-05-20T18:37:25.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.103+0000 7f2e728a0640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2e6c0cd680 0x7f2e6c15f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:25.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.103+0000 7f2e728a0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e6c0cfae0 0x7f2e6c06cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:25.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.103+0000 7f2e728a0640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e6c06d0f0 con 0x7f2e6c06c6c0 2024-05-20T18:37:25.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.103+0000 7f2e728a0640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2e6c0cda90 con 0x7f2e6c0cd680 2024-05-20T18:37:25.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.103+0000 7f2e728a0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2e6c15f760 con 0x7f2e6c0cfae0 2024-05-20T18:37:25.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.103+0000 7f2e6b7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2e6c06c6c0 0x7f2e6c0cf3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38600/0 (socket says 172.21.15.5:38600) 2024-05-20T18:37:25.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.103+0000 7f2e6b7fe640 1 -- 172.21.15.5:0/1903036131 learned_addr learned my addr 172.21.15.5:0/1903036131 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:25.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.103+0000 7f2e6bfff640 1 --2- 172.21.15.5:0/1903036131 >> v2:172.21.15.63:3300/0 conn(0x7f2e6c0cd680 0x7f2e6c15f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:25.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.103+0000 7f2e6affd640 1 -- 172.21.15.5:0/1903036131 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 889692720 0 0) 0x7f2e6c06d0f0 con 0x7f2e6c06c6c0 2024-05-20T18:37:25.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.103+0000 7f2e70e3d640 1 --2- 172.21.15.5:0/1903036131 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e6c0cfae0 0x7f2e6c06cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:25.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.103+0000 7f2e6affd640 1 -- 172.21.15.5:0/1903036131 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e54003520 con 0x7f2e6c06c6c0 2024-05-20T18:37:25.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.104+0000 7f2e6affd640 1 -- 172.21.15.5:0/1903036131 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3310555697 0 0) 0x7f2e54003520 con 0x7f2e6c06c6c0 2024-05-20T18:37:25.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.104+0000 7f2e6affd640 1 -- 172.21.15.5:0/1903036131 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e6c0cfae0 msgr2=0x7f2e6c06cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:25.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.104+0000 7f2e6affd640 1 --2- 172.21.15.5:0/1903036131 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e6c0cfae0 0x7f2e6c06cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:25.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.104+0000 7f2e6affd640 1 -- 172.21.15.5:0/1903036131 >> v2:172.21.15.63:3300/0 conn(0x7f2e6c0cd680 msgr2=0x7f2e6c15f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:25.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.104+0000 7f2e6affd640 1 --2- 172.21.15.5:0/1903036131 >> v2:172.21.15.63:3300/0 conn(0x7f2e6c0cd680 0x7f2e6c15f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:25.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.104+0000 7f2e6affd640 1 -- 172.21.15.5:0/1903036131 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e6c06d3d0 con 0x7f2e6c06c6c0 2024-05-20T18:37:25.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.104+0000 7f2e6affd640 1 -- 172.21.15.5:0/1903036131 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f2e60001670 con 0x7f2e6c06c6c0 2024-05-20T18:37:25.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.104+0000 7f2e6affd640 1 -- 172.21.15.5:0/1903036131 >> v1:172.21.15.5:6789/0 conn(0x7f2e6c06c6c0 legacy=0x7f2e6c0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:25.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.104+0000 7f2e6affd640 1 --2- 172.21.15.5:0/1903036131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e540039b0 0x7f2e54003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:25.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.104+0000 7f2e6affd640 1 --2- 172.21.15.5:0/1903036131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e54004800 0x7f2e54004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:25.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.104+0000 7f2e6affd640 1 --2- 172.21.15.5:0/1903036131 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e54005150 0x7f2e5400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:25.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.104+0000 7f2e6affd640 1 -- 172.21.15.5:0/1903036131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6c15f760 con 0x7f2e54004800 2024-05-20T18:37:25.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.104+0000 7f2e6affd640 1 -- 172.21.15.5:0/1903036131 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5400bf00 con 0x7f2e540039b0 2024-05-20T18:37:25.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.104+0000 7f2e6affd640 1 -- 172.21.15.5:0/1903036131 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2e5400c080 con 0x7f2e54005150 2024-05-20T18:37:25.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.104+0000 7f2e6b7fe640 1 --2- 172.21.15.5:0/1903036131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e54004800 0x7f2e54004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.104+0000 7f2e6bfff640 1 --2- 172.21.15.5:0/1903036131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e540039b0 0x7f2e54003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.104+0000 7f2e70e3d640 1 --2- 172.21.15.5:0/1903036131 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e54005150 0x7f2e5400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.104+0000 7f2e6b7fe640 1 -- 172.21.15.5:0/1903036131 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e54005150 msgr2=0x7f2e5400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.104+0000 7f2e6b7fe640 1 --2- 172.21.15.5:0/1903036131 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e54005150 0x7f2e5400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.104+0000 7f2e6b7fe640 1 -- 172.21.15.5:0/1903036131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e540039b0 msgr2=0x7f2e54003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.104+0000 7f2e6b7fe640 1 --2- 172.21.15.5:0/1903036131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e540039b0 0x7f2e54003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.104+0000 7f2e6b7fe640 1 -- 172.21.15.5:0/1903036131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e54003520 con 0x7f2e54004800 2024-05-20T18:37:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.105+0000 7f2e6b7fe640 1 --2- 172.21.15.5:0/1903036131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e54004800 0x7f2e54004c10 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000c240 tx=0x7f2e54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=472978e44adaed7f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.105+0000 7f2e6affd640 1 -- 172.21.15.5:0/1903036131 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e60007d30 con 0x7f2e54004800 2024-05-20T18:37:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.105+0000 7f2e6affd640 1 -- 172.21.15.5:0/1903036131 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e60005af0 con 0x7f2e54004800 2024-05-20T18:37:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.105+0000 7f2e6affd640 1 -- 172.21.15.5:0/1903036131 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e60007070 con 0x7f2e54004800 2024-05-20T18:37:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.106+0000 7f2e728a0640 1 -- 172.21.15.5:0/1903036131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e54004800 msgr2=0x7f2e54004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.106+0000 7f2e728a0640 1 --2- 172.21.15.5:0/1903036131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e54004800 0x7f2e54004c10 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000c240 tx=0x7f2e54005830 comp rx=0 tx=0).stop 2024-05-20T18:37:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.106+0000 7f2e70e3d640 1 -- 172.21.15.5:0/1903036131 reap_dead start 2024-05-20T18:37:25.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.106+0000 7f2e728a0640 1 -- 172.21.15.5:0/1903036131 shutdown_connections 2024-05-20T18:37:25.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.106+0000 7f2e728a0640 1 -- 172.21.15.5:0/1903036131 >> 172.21.15.5:0/1903036131 conn(0x7f2e6c0b44d0 msgr2=0x7f2e6c0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:25.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.106+0000 7f2e728a0640 1 -- 172.21.15.5:0/1903036131 shutdown_connections 2024-05-20T18:37:25.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.106+0000 7f2e728a0640 1 -- 172.21.15.5:0/1903036131 wait complete. 2024-05-20T18:37:25.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.108+0000 7f2e728a0640 1 Processor -- start 2024-05-20T18:37:25.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.108+0000 7f2e728a0640 1 -- start start 2024-05-20T18:37:25.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.109+0000 7f2e728a0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e54004800 0x7f2e6c14c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:25.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.109+0000 7f2e728a0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e6c161d90 0x7f2e6c0c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:25.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.109+0000 7f2e728a0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e6c14baf0 0x7f2e6c14bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:25.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.109+0000 7f2e728a0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6c06d990 con 0x7f2e54004800 2024-05-20T18:37:25.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.109+0000 7f2e728a0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6c0d0b40 con 0x7f2e6c161d90 2024-05-20T18:37:25.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.109+0000 7f2e728a0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2e6c0d2460 con 0x7f2e6c14baf0 2024-05-20T18:37:25.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.109+0000 7f2e6bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e54004800 0x7f2e6c14c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:25.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.109+0000 7f2e6bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e54004800 0x7f2e6c14c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33922/0 (socket says 172.21.15.5:33922) 2024-05-20T18:37:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.109+0000 7f2e6bfff640 1 -- 172.21.15.5:0/1609907598 learned_addr learned my addr 172.21.15.5:0/1609907598 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.110+0000 7f2e6b7fe640 1 --2- 172.21.15.5:0/1609907598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e6c161d90 0x7f2e6c0c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.110+0000 7f2e70e3d640 1 --2- 172.21.15.5:0/1609907598 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e6c14baf0 0x7f2e6c14bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.110+0000 7f2e6bfff640 1 -- 172.21.15.5:0/1609907598 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e6c14baf0 msgr2=0x7f2e6c14bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.110+0000 7f2e6bfff640 1 --2- 172.21.15.5:0/1609907598 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e6c14baf0 0x7f2e6c14bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.110+0000 7f2e6bfff640 1 -- 172.21.15.5:0/1609907598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e6c161d90 msgr2=0x7f2e6c0c8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.110+0000 7f2e6bfff640 1 --2- 172.21.15.5:0/1609907598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e6c161d90 0x7f2e6c0c8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.110+0000 7f2e6bfff640 1 -- 172.21.15.5:0/1609907598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e6c0c9380 con 0x7f2e54004800 2024-05-20T18:37:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.110+0000 7f2e6b7fe640 1 --2- 172.21.15.5:0/1609907598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e6c161d90 0x7f2e6c0c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.110+0000 7f2e70e3d640 1 --2- 172.21.15.5:0/1609907598 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e6c14baf0 0x7f2e6c14bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.110+0000 7f2e6bfff640 1 --2- 172.21.15.5:0/1609907598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e54004800 0x7f2e6c14c730 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c006d40 tx=0x7f2e5c011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.110+0000 7f2e68ff9640 1 -- 172.21.15.5:0/1609907598 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c00faf0 con 0x7f2e54004800 2024-05-20T18:37:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.110+0000 7f2e68ff9640 1 -- 172.21.15.5:0/1609907598 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e5c00fc90 con 0x7f2e54004800 2024-05-20T18:37:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.110+0000 7f2e68ff9640 1 -- 172.21.15.5:0/1609907598 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c004030 con 0x7f2e54004800 2024-05-20T18:37:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.110+0000 7f2e728a0640 1 -- 172.21.15.5:0/1609907598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e6c161310 con 0x7f2e54004800 2024-05-20T18:37:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.110+0000 7f2e728a0640 1 -- 172.21.15.5:0/1609907598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e6c15aec0 con 0x7f2e54004800 2024-05-20T18:37:25.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.111+0000 7f2e728a0640 1 -- 172.21.15.5:0/1609907598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e6c163390 con 0x7f2e54004800 2024-05-20T18:37:25.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.112+0000 7f2e68ff9640 1 -- 172.21.15.5:0/1609907598 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2e5c012040 con 0x7f2e54004800 2024-05-20T18:37:25.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.112+0000 7f2e68ff9640 1 --2- 172.21.15.5:0/1609907598 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2e5806be30 0x7f2e5806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:25.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.113+0000 7f2e68ff9640 1 -- 172.21.15.5:0/1609907598 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7f2e5c08b3d0 con 0x7f2e54004800 2024-05-20T18:37:25.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.113+0000 7f2e6b7fe640 1 --2- 172.21.15.5:0/1609907598 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2e5806be30 0x7f2e5806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:25.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.113+0000 7f2e6b7fe640 1 --2- 172.21.15.5:0/1609907598 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2e5806be30 0x7f2e5806e340 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2e60033000 tx=0x7f2e60009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:25.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.117+0000 7f2e68ff9640 1 -- 172.21.15.5:0/1609907598 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2e6c163390 con 0x7f2e54004800 2024-05-20T18:37:25.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.338+0000 7f2e728a0640 1 -- 172.21.15.5:0/1609907598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2e6c075b00 con 0x7f2e54004800 2024-05-20T18:37:25.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.338+0000 7f2e68ff9640 1 -- 172.21.15.5:0/1609907598 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v98) ==== 53+0+73 (secure 0 0 0) 0x7f2e6c161310 con 0x7f2e54004800 2024-05-20T18:37:25.339 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-20T18:37:25.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.340+0000 7f2e728a0640 1 -- 172.21.15.5:0/1609907598 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2e5806be30 msgr2=0x7f2e5806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:25.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.340+0000 7f2e728a0640 1 --2- 172.21.15.5:0/1609907598 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2e5806be30 0x7f2e5806e340 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2e60033000 tx=0x7f2e60009030 comp rx=0 tx=0).stop 2024-05-20T18:37:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.340+0000 7f2e728a0640 1 -- 172.21.15.5:0/1609907598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e54004800 msgr2=0x7f2e6c14c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.340+0000 7f2e728a0640 1 --2- 172.21.15.5:0/1609907598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e54004800 0x7f2e6c14c730 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c006d40 tx=0x7f2e5c011dc0 comp rx=0 tx=0).stop 2024-05-20T18:37:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.340+0000 7f2e728a0640 1 -- 172.21.15.5:0/1609907598 shutdown_connections 2024-05-20T18:37:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.340+0000 7f2e728a0640 1 --2- 172.21.15.5:0/1609907598 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2e5806be30 0x7f2e5806e340 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.340+0000 7f2e728a0640 1 --2- 172.21.15.5:0/1609907598 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e6c14baf0 0x7f2e6c14bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.340+0000 7f2e728a0640 1 --2- 172.21.15.5:0/1609907598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e6c161d90 0x7f2e6c0c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.340+0000 7f2e728a0640 1 --2- 172.21.15.5:0/1609907598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e54004800 0x7f2e6c14c730 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.340+0000 7f2e728a0640 1 -- 172.21.15.5:0/1609907598 >> 172.21.15.5:0/1609907598 conn(0x7f2e6c0b44d0 msgr2=0x7f2e6c0ce9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.340+0000 7f2e728a0640 1 -- 172.21.15.5:0/1609907598 shutdown_connections 2024-05-20T18:37:25.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25.340+0000 7f2e728a0640 1 -- 172.21.15.5:0/1609907598 wait complete. 2024-05-20T18:37:25.353 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-20T18:37:25.353 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-20T18:37:25.353 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-05-20T18:37:25.353 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-20T18:37:25.353 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-20T18:37:25.353 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-20T18:37:25.353 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-20T18:37:25.353 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-20T18:37:25.353 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:37:25.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:25.354 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-05-20T18:37:25.370 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T18:37:25.370 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T18:37:25.370 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T18:37:25.370 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:37:25.371 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-20T18:37:25.371 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:37:25.371 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:37:25.371 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-05-20T18:37:25.371 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-05-20T18:37:25.371 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:37:25.371 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-05-20T18:37:25.371 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-05-20T18:37:25.371 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-20T18:37:25.371 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:37:25.371 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-05-20T18:37:25.371 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-20T18:37:25.371 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42053sec preferred_lft 42053sec 2024-05-20T18:37:25.371 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-05-20T18:37:25.372 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:37:25.372 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:37:25.372 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:37:25.372 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-05-20T18:37:25.372 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-20T18:37:25.372 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:37:25.372 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-20T18:37:25.372 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:37:25.372 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::4474:eaff:fe07:4f9a/64 scope link 2024-05-20T18:37:25.372 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:37:25.372 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:37:25.372 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T18:37:25.372 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ce4:f4ff:fe5e:7906/64 scope link 2024-05-20T18:37:25.373 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:37:25.373 INFO:teuthology.orchestra.run.smithi005.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:37:25.373 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether da:26:04:0e:95:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T18:37:25.373 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::d826:4ff:fe0e:950a/64 scope link 2024-05-20T18:37:25.373 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:37:25.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:25.373 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-20T18:37:25.436 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-20T18:37:25.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:25.437 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-05-20T18:37:25.494 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-05-20T18:37:25.494 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-20T18:37:25.494 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-20T18:37:25.494 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-20T18:37:25.494 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-20T18:37:25.495 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:37:25.495 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-20T18:37:25.495 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-20T18:37:25.495 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-20T18:37:25.495 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-20T18:37:25.495 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-20T18:37:25.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:37:25.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:25.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:25.625 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-20T18:37:25.641 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-20T18:37:25.641 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-20T18:37:25.641 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:37:25.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:25.641 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-05-20T18:37:25.707 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:37:25.707 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-20T18:37:25.763 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:37:25.763 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:37:25.763 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:37:25.819 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-20T18:37:25.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:25.819 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:37:25.914 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:37:25.915 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:37:25.915 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:37:25.925 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:37:25.926 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:37:25.926 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:37:25.926 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:37:25.926 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:37:25.945 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:37:25.945 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:37:25.945 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:37:25.945 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:37:25.945 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:37:25.946 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790" 2024-05-20T18:37:25.946 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790:/ 2024-05-20T18:37:25.946 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=399ab250-fd3c-47fa-a1c8-25e8149c9cac" will pass to kernel 2024-05-20T18:37:25.946 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-20T18:37:25.946 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-05-20T18:37:25.946 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-20T18:37:25.946 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-05-20T18:37:25.946 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-20T18:37:25.948 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-20T18:37:25.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:25.948 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:37:25.974 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-20T18:37:26.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T18:37:26.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:26.073 INFO:teuthology.orchestra.run.smithi005.stdout:client7319 2024-05-20T18:37:26.073 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:37:26.073 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:37:26.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:26.090 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2024-05-20T18:37:26.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:26.153 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2024-05-20T18:37:26.231 INFO:teuthology.orchestra.run.smithi005.stderr:8+0 records in 2024-05-20T18:37:26.231 INFO:teuthology.orchestra.run.smithi005.stderr:8+0 records out 2024-05-20T18:37:26.231 INFO:teuthology.orchestra.run.smithi005.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0232055 s, 361 MB/s 2024-05-20T18:37:26.232 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:37:26.232 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:37:26.232 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:37:26.232 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:37:26.232 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-05-20T18:37:26.232 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:37:26.340 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627779 2024-05-20T18:37:26.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:26.349 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:37:26.349 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:37:26.349 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:37:26.350 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:37:26.350 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-05-20T18:37:26.350 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:37:26.456 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627780 2024-05-20T18:37:26.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:26.465 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:37:26.466 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:37:26.523 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:37:26.523 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:37:26.582 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:37:26.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.736+0000 7fec90d8b640 1 Processor -- start 2024-05-20T18:37:26.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.736+0000 7fec90d8b640 1 -- start start 2024-05-20T18:37:26.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.737+0000 7fec90d8b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec8c0d1bc0 0x7fec8c0d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:26.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.737+0000 7fec90d8b640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec8c0d0370 con 0x7fec8c0b4010 2024-05-20T18:37:26.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.737+0000 7fec90d8b640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec8c0d0650 con 0x7fec8c163a10 2024-05-20T18:37:26.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.737+0000 7fec90d8b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fec8c0d3ca0 con 0x7fec8c0d1bc0 2024-05-20T18:37:26.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.737+0000 7fec81d9b640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fec8c0b4010 0x7fec8c0d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38608/0 (socket says 172.21.15.5:38608) 2024-05-20T18:37:26.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.737+0000 7fec81d9b640 1 -- 172.21.15.5:0/2048474801 learned_addr learned my addr 172.21.15.5:0/2048474801 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:26.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.738+0000 7fec8ad9d640 1 --2- 172.21.15.5:0/2048474801 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec8c0d1bc0 0x7fec8c0d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:26.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.738+0000 7fec89d9b640 1 -- 172.21.15.5:0/2048474801 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1477164818 0 0) 0x7fec8c0d0370 con 0x7fec8c0b4010 2024-05-20T18:37:26.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.738+0000 7fec89d9b640 1 -- 172.21.15.5:0/2048474801 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec78003520 con 0x7fec8c0b4010 2024-05-20T18:37:26.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.738+0000 7fec89d9b640 1 -- 172.21.15.5:0/2048474801 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3725395569 0 0) 0x7fec78003520 con 0x7fec8c0b4010 2024-05-20T18:37:26.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.738+0000 7fec89d9b640 1 -- 172.21.15.5:0/2048474801 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec8c0d1bc0 msgr2=0x7fec8c0d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:26.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.738+0000 7fec89d9b640 1 --2- 172.21.15.5:0/2048474801 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec8c0d1bc0 0x7fec8c0d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:26.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.738+0000 7fec89d9b640 1 -- 172.21.15.5:0/2048474801 >> v1:172.21.15.63:6789/0 conn(0x7fec8c163a10 legacy=0x7fec8c0cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:26.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.738+0000 7fec89d9b640 1 -- 172.21.15.5:0/2048474801 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec8c0d3650 con 0x7fec8c0b4010 2024-05-20T18:37:26.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.738+0000 7fec89d9b640 1 -- 172.21.15.5:0/2048474801 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7fec74001670 con 0x7fec8c0b4010 2024-05-20T18:37:26.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.738+0000 7fec89d9b640 1 -- 172.21.15.5:0/2048474801 >> v1:172.21.15.5:6789/0 conn(0x7fec8c0b4010 legacy=0x7fec8c0d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:26.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.738+0000 7fec89d9b640 1 --2- 172.21.15.5:0/2048474801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec780039e0 0x7fec78003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:26.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.738+0000 7fec89d9b640 1 --2- 172.21.15.5:0/2048474801 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec8c0d1bc0 0x7fec78004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:26.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.738+0000 7fec89d9b640 1 --2- 172.21.15.5:0/2048474801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec78004d40 0x7fec78005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:26.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.738+0000 7fec89d9b640 1 -- 172.21.15.5:0/2048474801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c0d3ca0 con 0x7fec780039e0 2024-05-20T18:37:26.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.738+0000 7fec89d9b640 1 -- 172.21.15.5:0/2048474801 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fec7800bb50 con 0x7fec78004d40 2024-05-20T18:37:26.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.738+0000 7fec89d9b640 1 -- 172.21.15.5:0/2048474801 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fec7800bcd0 con 0x7fec8c0d1bc0 2024-05-20T18:37:26.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.738+0000 7fec8a59c640 1 --2- 172.21.15.5:0/2048474801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec780039e0 0x7fec78003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:26.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.738+0000 7fec81d9b640 1 --2- 172.21.15.5:0/2048474801 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec8c0d1bc0 0x7fec78004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:26.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.738+0000 7fec8ad9d640 1 --2- 172.21.15.5:0/2048474801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec78004d40 0x7fec78005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:26.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.739+0000 7fec8a59c640 1 -- 172.21.15.5:0/2048474801 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec8c0d1bc0 msgr2=0x7fec78004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:26.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.739+0000 7fec8a59c640 1 --2- 172.21.15.5:0/2048474801 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec8c0d1bc0 0x7fec78004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:26.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.739+0000 7fec8a59c640 1 -- 172.21.15.5:0/2048474801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec78004d40 msgr2=0x7fec78005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:26.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.739+0000 7fec8a59c640 1 --2- 172.21.15.5:0/2048474801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec78004d40 0x7fec78005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:26.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.739+0000 7fec8a59c640 1 -- 172.21.15.5:0/2048474801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec78003520 con 0x7fec780039e0 2024-05-20T18:37:26.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.739+0000 7fec8a59c640 1 --2- 172.21.15.5:0/2048474801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec780039e0 0x7fec78003e50 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fec7c001720 tx=0x7fec78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27ca83de2232032 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:26.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.740+0000 7fec89d9b640 1 -- 172.21.15.5:0/2048474801 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c001370 con 0x7fec780039e0 2024-05-20T18:37:26.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.740+0000 7fec89d9b640 1 -- 172.21.15.5:0/2048474801 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec7c0052d0 con 0x7fec780039e0 2024-05-20T18:37:26.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.740+0000 7fec89d9b640 1 -- 172.21.15.5:0/2048474801 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c02abc0 con 0x7fec780039e0 2024-05-20T18:37:26.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.740+0000 7fec90d8b640 1 -- 172.21.15.5:0/2048474801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec780039e0 msgr2=0x7fec78003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:26.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.740+0000 7fec90d8b640 1 --2- 172.21.15.5:0/2048474801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec780039e0 0x7fec78003e50 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fec7c001720 tx=0x7fec78005830 comp rx=0 tx=0).stop 2024-05-20T18:37:26.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.740+0000 7fec8ad9d640 1 -- 172.21.15.5:0/2048474801 reap_dead start 2024-05-20T18:37:26.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.740+0000 7fec90d8b640 1 -- 172.21.15.5:0/2048474801 shutdown_connections 2024-05-20T18:37:26.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.740+0000 7fec90d8b640 1 -- 172.21.15.5:0/2048474801 >> 172.21.15.5:0/2048474801 conn(0x7fec8c06d240 msgr2=0x7fec8c06d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:26.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.740+0000 7fec90d8b640 1 -- 172.21.15.5:0/2048474801 shutdown_connections 2024-05-20T18:37:26.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.740+0000 7fec90d8b640 1 -- 172.21.15.5:0/2048474801 wait complete. 2024-05-20T18:37:26.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.742+0000 7fec90d8b640 1 Processor -- start 2024-05-20T18:37:26.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.742+0000 7fec90d8b640 1 -- start start 2024-05-20T18:37:26.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.744+0000 7fec90d8b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec780039e0 0x7fec8c0b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:26.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.744+0000 7fec90d8b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec8c1576e0 0x7fec8c1f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:26.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.744+0000 7fec90d8b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec8c0b4620 0x7fec8c1438a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:26.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.744+0000 7fec90d8b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c0d42b0 con 0x7fec8c1576e0 2024-05-20T18:37:26.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.744+0000 7fec90d8b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c0d4130 con 0x7fec8c0b4620 2024-05-20T18:37:26.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.744+0000 7fec90d8b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fec8c0d4430 con 0x7fec780039e0 2024-05-20T18:37:26.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.744+0000 7fec81d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec8c1576e0 0x7fec8c1f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:26.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.744+0000 7fec81d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec8c1576e0 0x7fec8c1f4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33940/0 (socket says 172.21.15.5:33940) 2024-05-20T18:37:26.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.744+0000 7fec81d9b640 1 -- 172.21.15.5:0/2812217299 learned_addr learned my addr 172.21.15.5:0/2812217299 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:26.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.744+0000 7fec8a59c640 1 --2- 172.21.15.5:0/2812217299 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec780039e0 0x7fec8c0b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:26.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.744+0000 7fec8ad9d640 1 --2- 172.21.15.5:0/2812217299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec8c0b4620 0x7fec8c1438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:26.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.744+0000 7fec81d9b640 1 -- 172.21.15.5:0/2812217299 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec780039e0 msgr2=0x7fec8c0b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:26.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.744+0000 7fec81d9b640 1 --2- 172.21.15.5:0/2812217299 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec780039e0 0x7fec8c0b6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:26.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.744+0000 7fec81d9b640 1 -- 172.21.15.5:0/2812217299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec8c0b4620 msgr2=0x7fec8c1438a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:26.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.744+0000 7fec81d9b640 1 --2- 172.21.15.5:0/2812217299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec8c0b4620 0x7fec8c1438a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:26.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.744+0000 7fec81d9b640 1 -- 172.21.15.5:0/2812217299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec8c0b7480 con 0x7fec8c1576e0 2024-05-20T18:37:26.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.745+0000 7fec81d9b640 1 --2- 172.21.15.5:0/2812217299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec8c1576e0 0x7fec8c1f4fd0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fec74007290 tx=0x7fec74007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:26.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.745+0000 7fec837fe640 1 -- 172.21.15.5:0/2812217299 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7400f960 con 0x7fec8c1576e0 2024-05-20T18:37:26.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.745+0000 7fec837fe640 1 -- 172.21.15.5:0/2812217299 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec74018070 con 0x7fec8c1576e0 2024-05-20T18:37:26.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.745+0000 7fec90d8b640 1 -- 172.21.15.5:0/2812217299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec8c156640 con 0x7fec8c1576e0 2024-05-20T18:37:26.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.745+0000 7fec837fe640 1 -- 172.21.15.5:0/2812217299 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7400fd40 con 0x7fec8c1576e0 2024-05-20T18:37:26.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.745+0000 7fec90d8b640 1 -- 172.21.15.5:0/2812217299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec8c156c00 con 0x7fec8c1576e0 2024-05-20T18:37:26.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.746+0000 7fec90d8b640 1 -- 172.21.15.5:0/2812217299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec8c075b00 con 0x7fec8c1576e0 2024-05-20T18:37:26.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.747+0000 7fec837fe640 1 -- 172.21.15.5:0/2812217299 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fec7401d460 con 0x7fec8c1576e0 2024-05-20T18:37:26.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.747+0000 7fec837fe640 1 --2- 172.21.15.5:0/2812217299 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fec6c06be30 0x7fec6c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:26.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.747+0000 7fec8a59c640 1 --2- 172.21.15.5:0/2812217299 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fec6c06be30 0x7fec6c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:26.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.747+0000 7fec837fe640 1 -- 172.21.15.5:0/2812217299 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7fec7408aff0 con 0x7fec8c1576e0 2024-05-20T18:37:26.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.748+0000 7fec8a59c640 1 --2- 172.21.15.5:0/2812217299 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fec6c06be30 0x7fec6c06e340 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fec7c032000 tx=0x7fec7c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:26.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.752+0000 7fec837fe640 1 -- 172.21.15.5:0/2812217299 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fec740590b0 con 0x7fec8c1576e0 2024-05-20T18:37:26.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.935+0000 7fec90d8b640 1 -- 172.21.15.5:0/2812217299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec8c0d3050 con 0x7fec8c1576e0 2024-05-20T18:37:26.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.936+0000 7fec837fe640 1 -- 172.21.15.5:0/2812217299 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+16832 (secure 0 0 0) 0x7fec7405d3b0 con 0x7fec8c1576e0 2024-05-20T18:37:26.938 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:26.938 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":119,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:37:15.995209+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:37:12.797471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-20T18:37:13.346156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:37:26.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.938+0000 7fec90d8b640 1 -- 172.21.15.5:0/2812217299 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fec6c06be30 msgr2=0x7fec6c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:26.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.938+0000 7fec90d8b640 1 --2- 172.21.15.5:0/2812217299 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fec6c06be30 0x7fec6c06e340 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fec7c032000 tx=0x7fec7c02e000 comp rx=0 tx=0).stop 2024-05-20T18:37:26.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.938+0000 7fec90d8b640 1 -- 172.21.15.5:0/2812217299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec8c1576e0 msgr2=0x7fec8c1f4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:26.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.938+0000 7fec90d8b640 1 --2- 172.21.15.5:0/2812217299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec8c1576e0 0x7fec8c1f4fd0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fec74007290 tx=0x7fec74007750 comp rx=0 tx=0).stop 2024-05-20T18:37:26.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.938+0000 7fec90d8b640 1 -- 172.21.15.5:0/2812217299 shutdown_connections 2024-05-20T18:37:26.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.938+0000 7fec90d8b640 1 --2- 172.21.15.5:0/2812217299 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fec6c06be30 0x7fec6c06e340 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:26.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.938+0000 7fec90d8b640 1 --2- 172.21.15.5:0/2812217299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec8c0b4620 0x7fec8c1438a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:26.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.938+0000 7fec90d8b640 1 --2- 172.21.15.5:0/2812217299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec8c1576e0 0x7fec8c1f4fd0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:26.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.938+0000 7fec90d8b640 1 --2- 172.21.15.5:0/2812217299 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec780039e0 0x7fec8c0b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:26.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.938+0000 7fec90d8b640 1 -- 172.21.15.5:0/2812217299 >> 172.21.15.5:0/2812217299 conn(0x7fec8c06d240 msgr2=0x7fec8c15f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:26.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.938+0000 7fec90d8b640 1 -- 172.21.15.5:0/2812217299 shutdown_connections 2024-05-20T18:37:26.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:26.938+0000 7fec90d8b640 1 -- 172.21.15.5:0/2812217299 wait complete. 2024-05-20T18:37:26.951 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:37:12.797471+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:37:13.346156+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:37:26.952 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:37:27.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.069+0000 7fdecec98640 1 Processor -- start 2024-05-20T18:37:27.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.069+0000 7fdecec98640 1 -- start start 2024-05-20T18:37:27.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.070+0000 7fdecec98640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdec80b1e10 0x7fdec815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:27.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.070+0000 7fdecec98640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdec8070850 0x7fdec8070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:27.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.070+0000 7fdecec98640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdec80cd580 con 0x7fdec815d530 2024-05-20T18:37:27.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.070+0000 7fdecec98640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fdec80ce190 con 0x7fdec8070850 2024-05-20T18:37:27.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.070+0000 7fdecec98640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdec80cd860 con 0x7fdec80b1e10 2024-05-20T18:37:27.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.070+0000 7fdecd235640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdec8070850 0x7fdec8070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:27.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.070+0000 7fdecd235640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdec8070850 0x7fdec8070c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33952/0 (socket says 172.21.15.5:33952) 2024-05-20T18:37:27.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.070+0000 7fdecd235640 1 -- 172.21.15.5:0/3802240690 learned_addr learned my addr 172.21.15.5:0/3802240690 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:27.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.071+0000 7fdec7fff640 1 --2- 172.21.15.5:0/3802240690 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdec80b1e10 0x7fdec815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:27.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.071+0000 7fdecd235640 1 -- 172.21.15.5:0/3802240690 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdec80b1e10 msgr2=0x7fdec815aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:27.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.071+0000 7fdecd235640 1 --2- 172.21.15.5:0/3802240690 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdec80b1e10 0x7fdec815aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.071+0000 7fdecd235640 1 -- 172.21.15.5:0/3802240690 >> v1:172.21.15.63:6789/0 conn(0x7fdec815d530 legacy=0x7fdec80b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:27.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.071+0000 7fdecd235640 1 -- 172.21.15.5:0/3802240690 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec80cdb40 con 0x7fdec8070850 2024-05-20T18:37:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.071+0000 7fdecd235640 1 --2- 172.21.15.5:0/3802240690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdec8070850 0x7fdec8070c60 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fdec0009e50 tx=0x7fdec002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5c6bbee7da940a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.072+0000 7fdec77fe640 1 -- 172.21.15.5:0/3802240690 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec0036070 con 0x7fdec8070850 2024-05-20T18:37:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.072+0000 7fdec77fe640 1 -- 172.21.15.5:0/3802240690 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdec0031600 con 0x7fdec8070850 2024-05-20T18:37:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.072+0000 7fdec77fe640 1 -- 172.21.15.5:0/3802240690 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec00319d0 con 0x7fdec8070850 2024-05-20T18:37:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.072+0000 7fdecec98640 1 -- 172.21.15.5:0/3802240690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdec8070850 msgr2=0x7fdec8070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.072+0000 7fdecec98640 1 --2- 172.21.15.5:0/3802240690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdec8070850 0x7fdec8070c60 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fdec0009e50 tx=0x7fdec002a710 comp rx=0 tx=0).stop 2024-05-20T18:37:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.072+0000 7fdecec98640 1 -- 172.21.15.5:0/3802240690 shutdown_connections 2024-05-20T18:37:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.072+0000 7fdecec98640 1 --2- 172.21.15.5:0/3802240690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdec8070850 0x7fdec8070c60 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.072+0000 7fdecec98640 1 --2- 172.21.15.5:0/3802240690 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdec80b1e10 0x7fdec815aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.072+0000 7fdecec98640 1 -- 172.21.15.5:0/3802240690 >> 172.21.15.5:0/3802240690 conn(0x7fdec80d29d0 msgr2=0x7fdec80d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.072+0000 7fdecec98640 1 -- 172.21.15.5:0/3802240690 shutdown_connections 2024-05-20T18:37:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.072+0000 7fdecec98640 1 -- 172.21.15.5:0/3802240690 wait complete. 2024-05-20T18:37:27.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.074+0000 7fdecec98640 1 Processor -- start 2024-05-20T18:37:27.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.074+0000 7fdecec98640 1 -- start start 2024-05-20T18:37:27.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.076+0000 7fdecec98640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdec8070850 0x7fdec80c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:27.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.076+0000 7fdecec98640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdec80b1e10 0x7fdec80c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:27.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.076+0000 7fdecec98640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdec815d530 0x7fdec815bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.076+0000 7fdecec98640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdec80711a0 con 0x7fdec80b1e10 2024-05-20T18:37:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.076+0000 7fdecec98640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdec80cd230 con 0x7fdec815d530 2024-05-20T18:37:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.076+0000 7fdecec98640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdec8071320 con 0x7fdec8070850 2024-05-20T18:37:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.076+0000 7fdec7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdec80b1e10 0x7fdec80c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.076+0000 7fdec7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdec80b1e10 0x7fdec80c9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33968/0 (socket says 172.21.15.5:33968) 2024-05-20T18:37:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.076+0000 7fdec7fff640 1 -- 172.21.15.5:0/565731067 learned_addr learned my addr 172.21.15.5:0/565731067 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.076+0000 7fdecca34640 1 --2- 172.21.15.5:0/565731067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdec8070850 0x7fdec80c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.076+0000 7fdec7fff640 1 -- 172.21.15.5:0/565731067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdec8070850 msgr2=0x7fdec80c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.076+0000 7fdec7fff640 1 --2- 172.21.15.5:0/565731067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdec8070850 0x7fdec80c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.076+0000 7fdec7fff640 1 -- 172.21.15.5:0/565731067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdec815d530 msgr2=0x7fdec815bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.076+0000 7fdec7fff640 1 --2- 172.21.15.5:0/565731067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdec815d530 0x7fdec815bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.076+0000 7fdec7fff640 1 -- 172.21.15.5:0/565731067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec80ca720 con 0x7fdec80b1e10 2024-05-20T18:37:27.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.076+0000 7fdec7fff640 1 --2- 172.21.15.5:0/565731067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdec80b1e10 0x7fdec80c9a80 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400e870 tx=0x7fdeb400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:27.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.076+0000 7fdec57fa640 1 -- 172.21.15.5:0/565731067 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4017070 con 0x7fdec80b1e10 2024-05-20T18:37:27.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.076+0000 7fdec57fa640 1 -- 172.21.15.5:0/565731067 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeb400c940 con 0x7fdec80b1e10 2024-05-20T18:37:27.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.076+0000 7fdec57fa640 1 -- 172.21.15.5:0/565731067 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb400cd30 con 0x7fdec80b1e10 2024-05-20T18:37:27.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.076+0000 7fdecec98640 1 -- 172.21.15.5:0/565731067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec80caa00 con 0x7fdec80b1e10 2024-05-20T18:37:27.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.077+0000 7fdecec98640 1 -- 172.21.15.5:0/565731067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec80cafc0 con 0x7fdec80b1e10 2024-05-20T18:37:27.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.077+0000 7fdecec98640 1 -- 172.21.15.5:0/565731067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdec8075b00 con 0x7fdec80b1e10 2024-05-20T18:37:27.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.078+0000 7fdec57fa640 1 -- 172.21.15.5:0/565731067 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdeb401a070 con 0x7fdec80b1e10 2024-05-20T18:37:27.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.078+0000 7fdec57fa640 1 --2- 172.21.15.5:0/565731067 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdeac06be30 0x7fdeac06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:27.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.079+0000 7fdec57fa640 1 -- 172.21.15.5:0/565731067 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7fdeb408af40 con 0x7fdec80b1e10 2024-05-20T18:37:27.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.079+0000 7fdecca34640 1 --2- 172.21.15.5:0/565731067 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdeac06be30 0x7fdeac06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:27.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.079+0000 7fdecca34640 1 --2- 172.21.15.5:0/565731067 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdeac06be30 0x7fdeac06e340 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fdebc007fd0 tx=0x7fdebc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:27.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.083+0000 7fdec57fa640 1 -- 172.21.15.5:0/565731067 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fdeb40596c0 con 0x7fdec80b1e10 2024-05-20T18:37:27.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.268+0000 7fdecec98640 1 -- 172.21.15.5:0/565731067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdec815e7e0 con 0x7fdec80b1e10 2024-05-20T18:37:27.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.269+0000 7fdec57fa640 1 -- 172.21.15.5:0/565731067 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) ==== 93+0+10 (secure 0 0 0) 0x7fdeb4058fc0 con 0x7fdec80b1e10 2024-05-20T18:37:27.271 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:37:27.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.271+0000 7fdecec98640 1 -- 172.21.15.5:0/565731067 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdeac06be30 msgr2=0x7fdeac06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:27.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.271+0000 7fdecec98640 1 --2- 172.21.15.5:0/565731067 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdeac06be30 0x7fdeac06e340 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fdebc007fd0 tx=0x7fdebc009040 comp rx=0 tx=0).stop 2024-05-20T18:37:27.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.271+0000 7fdecec98640 1 -- 172.21.15.5:0/565731067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdec80b1e10 msgr2=0x7fdec80c9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:27.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.271+0000 7fdecec98640 1 --2- 172.21.15.5:0/565731067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdec80b1e10 0x7fdec80c9a80 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400e870 tx=0x7fdeb400ed30 comp rx=0 tx=0).stop 2024-05-20T18:37:27.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.271+0000 7fdecec98640 1 -- 172.21.15.5:0/565731067 shutdown_connections 2024-05-20T18:37:27.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.271+0000 7fdecec98640 1 --2- 172.21.15.5:0/565731067 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdeac06be30 0x7fdeac06e340 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.271+0000 7fdecec98640 1 --2- 172.21.15.5:0/565731067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdec815d530 0x7fdec815bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.271+0000 7fdecec98640 1 --2- 172.21.15.5:0/565731067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdec80b1e10 0x7fdec80c9a80 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.271+0000 7fdecec98640 1 --2- 172.21.15.5:0/565731067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdec8070850 0x7fdec80c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.271+0000 7fdecec98640 1 -- 172.21.15.5:0/565731067 >> 172.21.15.5:0/565731067 conn(0x7fdec80d29d0 msgr2=0x7fdec80b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:27.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.271+0000 7fdecec98640 1 -- 172.21.15.5:0/565731067 shutdown_connections 2024-05-20T18:37:27.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.271+0000 7fdecec98640 1 -- 172.21.15.5:0/565731067 wait complete. 2024-05-20T18:37:27.284 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:37:27.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.400+0000 7fb715c14640 1 Processor -- start 2024-05-20T18:37:27.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.400+0000 7fb715c14640 1 -- start start 2024-05-20T18:37:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.403+0000 7fb715c14640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb71015d460 0x7fb71006b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.403+0000 7fb715c14640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb7100d1780 0x7fb7100d1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.403+0000 7fb715c14640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb71009a4f0 0x7fb71009a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.403+0000 7fb715c14640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fb71015d870 con 0x7fb7100d1780 2024-05-20T18:37:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.403+0000 7fb715c14640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb7100d20d0 con 0x7fb71009a4f0 2024-05-20T18:37:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.403+0000 7fb715c14640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb71015dae0 con 0x7fb71015d460 2024-05-20T18:37:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.403+0000 7fb70effd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb7100d1780 0x7fb7100d1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.403+0000 7fb70effd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb7100d1780 0x7fb7100d1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33976/0 (socket says 172.21.15.5:33976) 2024-05-20T18:37:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.403+0000 7fb70effd640 1 -- 172.21.15.5:0/32951780 learned_addr learned my addr 172.21.15.5:0/32951780 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.403+0000 7fb70ffff640 1 --2- 172.21.15.5:0/32951780 >> v2:172.21.15.63:3300/0 conn(0x7fb71009a4f0 0x7fb71009a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.403+0000 7fb70f7fe640 1 --2- 172.21.15.5:0/32951780 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb71015d460 0x7fb71006b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.403+0000 7fb70effd640 1 -- 172.21.15.5:0/32951780 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb71015d460 msgr2=0x7fb71006b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.403+0000 7fb70effd640 1 --2- 172.21.15.5:0/32951780 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb71015d460 0x7fb71006b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.403+0000 7fb70effd640 1 -- 172.21.15.5:0/32951780 >> v2:172.21.15.63:3300/0 conn(0x7fb71009a4f0 msgr2=0x7fb71009a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.403+0000 7fb70effd640 1 --2- 172.21.15.5:0/32951780 >> v2:172.21.15.63:3300/0 conn(0x7fb71009a4f0 0x7fb71009a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.403+0000 7fb70effd640 1 -- 172.21.15.5:0/32951780 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71015ddc0 con 0x7fb7100d1780 2024-05-20T18:37:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.403+0000 7fb70ffff640 1 --2- 172.21.15.5:0/32951780 >> v2:172.21.15.63:3300/0 conn(0x7fb71009a4f0 0x7fb71009a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.403+0000 7fb70effd640 1 --2- 172.21.15.5:0/32951780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7100d1780 0x7fb7100d1b90 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fb7100b6b30 tx=0x7fb70402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165caccc6336f134 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.403+0000 7fb70e7fc640 1 -- 172.21.15.5:0/32951780 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb704037070 con 0x7fb7100d1780 2024-05-20T18:37:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.403+0000 7fb70e7fc640 1 -- 172.21.15.5:0/32951780 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb704032440 con 0x7fb7100d1780 2024-05-20T18:37:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.403+0000 7fb70e7fc640 1 -- 172.21.15.5:0/32951780 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb704032810 con 0x7fb7100d1780 2024-05-20T18:37:27.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.404+0000 7fb715c14640 1 -- 172.21.15.5:0/32951780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7100d1780 msgr2=0x7fb7100d1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:27.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.404+0000 7fb715c14640 1 --2- 172.21.15.5:0/32951780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7100d1780 0x7fb7100d1b90 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fb7100b6b30 tx=0x7fb70402a1e0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.404+0000 7fb715c14640 1 -- 172.21.15.5:0/32951780 shutdown_connections 2024-05-20T18:37:27.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.404+0000 7fb715c14640 1 --2- 172.21.15.5:0/32951780 >> v2:172.21.15.63:3300/0 conn(0x7fb71009a4f0 0x7fb71009a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.404+0000 7fb715c14640 1 --2- 172.21.15.5:0/32951780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7100d1780 0x7fb7100d1b90 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.404+0000 7fb715c14640 1 --2- 172.21.15.5:0/32951780 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb71015d460 0x7fb71006b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.404+0000 7fb715c14640 1 -- 172.21.15.5:0/32951780 >> 172.21.15.5:0/32951780 conn(0x7fb7100b6620 msgr2=0x7fb7100b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:27.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.404+0000 7fb715c14640 1 -- 172.21.15.5:0/32951780 shutdown_connections 2024-05-20T18:37:27.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.404+0000 7fb715c14640 1 -- 172.21.15.5:0/32951780 wait complete. 2024-05-20T18:37:27.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.406+0000 7fb715c14640 1 Processor -- start 2024-05-20T18:37:27.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.406+0000 7fb715c14640 1 -- start start 2024-05-20T18:37:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.407+0000 7fb715c14640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb71009a4f0 0x7fb71013fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.407+0000 7fb715c14640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7100d1780 0x7fb7100ce740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.407+0000 7fb715c14640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb71015d460 0x7fb7100cec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.407+0000 7fb715c14640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb71006bdb0 con 0x7fb7100d1780 2024-05-20T18:37:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.407+0000 7fb715c14640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb7100b12d0 con 0x7fb71015d460 2024-05-20T18:37:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.407+0000 7fb715c14640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb71015e380 con 0x7fb71009a4f0 2024-05-20T18:37:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.408+0000 7fb70effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7100d1780 0x7fb7100ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.408+0000 7fb70effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7100d1780 0x7fb7100ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33986/0 (socket says 172.21.15.5:33986) 2024-05-20T18:37:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.408+0000 7fb70effd640 1 -- 172.21.15.5:0/3887306016 learned_addr learned my addr 172.21.15.5:0/3887306016 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.408+0000 7fb70f7fe640 1 --2- 172.21.15.5:0/3887306016 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb71009a4f0 0x7fb71013fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.408+0000 7fb70ffff640 1 --2- 172.21.15.5:0/3887306016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb71015d460 0x7fb7100cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.408+0000 7fb70effd640 1 -- 172.21.15.5:0/3887306016 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb71009a4f0 msgr2=0x7fb71013fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.408+0000 7fb70effd640 1 --2- 172.21.15.5:0/3887306016 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb71009a4f0 0x7fb71013fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.408+0000 7fb70effd640 1 -- 172.21.15.5:0/3887306016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb71015d460 msgr2=0x7fb7100cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.408+0000 7fb70effd640 1 --2- 172.21.15.5:0/3887306016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb71015d460 0x7fb7100cec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.408+0000 7fb70effd640 1 -- 172.21.15.5:0/3887306016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7100cdbb0 con 0x7fb7100d1780 2024-05-20T18:37:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.408+0000 7fb70effd640 1 --2- 172.21.15.5:0/3887306016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7100d1780 0x7fb7100ce740 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fb704000c00 tx=0x7fb7040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.408+0000 7fb6f7fff640 1 -- 172.21.15.5:0/3887306016 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb704037040 con 0x7fb7100d1780 2024-05-20T18:37:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.408+0000 7fb6f7fff640 1 -- 172.21.15.5:0/3887306016 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb704042070 con 0x7fb7100d1780 2024-05-20T18:37:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.408+0000 7fb715c14640 1 -- 172.21.15.5:0/3887306016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7100ae8e0 con 0x7fb7100d1780 2024-05-20T18:37:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.408+0000 7fb6f7fff640 1 -- 172.21.15.5:0/3887306016 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb704030860 con 0x7fb7100d1780 2024-05-20T18:37:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.408+0000 7fb715c14640 1 -- 172.21.15.5:0/3887306016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7100aeb80 con 0x7fb7100d1780 2024-05-20T18:37:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.409+0000 7fb715c14640 1 -- 172.21.15.5:0/3887306016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb710075b00 con 0x7fb7100d1780 2024-05-20T18:37:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.410+0000 7fb6f7fff640 1 -- 172.21.15.5:0/3887306016 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb704030a00 con 0x7fb7100d1780 2024-05-20T18:37:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.411+0000 7fb6f7fff640 1 --2- 172.21.15.5:0/3887306016 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb6fc06be30 0x7fb6fc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.411+0000 7fb6f7fff640 1 -- 172.21.15.5:0/3887306016 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7fb7040ac180 con 0x7fb7100d1780 2024-05-20T18:37:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.411+0000 7fb70f7fe640 1 --2- 172.21.15.5:0/3887306016 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb6fc06be30 0x7fb6fc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.411+0000 7fb70f7fe640 1 --2- 172.21.15.5:0/3887306016 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb6fc06be30 0x7fb6fc06e340 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb71006c760 tx=0x7fb700009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.415+0000 7fb6f7fff640 1 -- 172.21.15.5:0/3887306016 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb70407a430 con 0x7fb7100d1780 2024-05-20T18:37:27.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.599+0000 7fb715c14640 1 -- 172.21.15.5:0/3887306016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb7100cd660 con 0x7fb7100d1780 2024-05-20T18:37:27.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.599+0000 7fb6f7fff640 1 -- 172.21.15.5:0/3887306016 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v119) ==== 104+0+11 (secure 0 0 0) 0x7fb70407e730 con 0x7fb7100d1780 2024-05-20T18:37:27.601 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:37:27.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.601+0000 7fb715c14640 1 -- 172.21.15.5:0/3887306016 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb6fc06be30 msgr2=0x7fb6fc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:27.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.601+0000 7fb715c14640 1 --2- 172.21.15.5:0/3887306016 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb6fc06be30 0x7fb6fc06e340 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb71006c760 tx=0x7fb700009190 comp rx=0 tx=0).stop 2024-05-20T18:37:27.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.602+0000 7fb715c14640 1 -- 172.21.15.5:0/3887306016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7100d1780 msgr2=0x7fb7100ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:27.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.602+0000 7fb715c14640 1 --2- 172.21.15.5:0/3887306016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7100d1780 0x7fb7100ce740 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fb704000c00 tx=0x7fb7040046c0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.602+0000 7fb715c14640 1 -- 172.21.15.5:0/3887306016 shutdown_connections 2024-05-20T18:37:27.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.602+0000 7fb715c14640 1 --2- 172.21.15.5:0/3887306016 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb6fc06be30 0x7fb6fc06e340 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.602+0000 7fb715c14640 1 --2- 172.21.15.5:0/3887306016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb71015d460 0x7fb7100cec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.602+0000 7fb715c14640 1 --2- 172.21.15.5:0/3887306016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7100d1780 0x7fb7100ce740 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.602+0000 7fb715c14640 1 --2- 172.21.15.5:0/3887306016 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb71009a4f0 0x7fb71013fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.602+0000 7fb715c14640 1 -- 172.21.15.5:0/3887306016 >> 172.21.15.5:0/3887306016 conn(0x7fb7100b6620 msgr2=0x7fb7100b35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:27.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.602+0000 7fb715c14640 1 -- 172.21.15.5:0/3887306016 shutdown_connections 2024-05-20T18:37:27.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.602+0000 7fb715c14640 1 -- 172.21.15.5:0/3887306016 wait complete. 2024-05-20T18:37:27.615 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:37:27.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.731+0000 7f3336051640 1 Processor -- start 2024-05-20T18:37:27.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.731+0000 7f3336051640 1 -- start start 2024-05-20T18:37:27.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.733+0000 7f3336051640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f333006d3d0 0x7f333006d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:27.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.733+0000 7f3336051640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33300b8190 0x7f33300b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:27.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.733+0000 7f3336051640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f333006dd20 con 0x7f3330162820 2024-05-20T18:37:27.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.733+0000 7f3336051640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f33300d0630 con 0x7f333006d3d0 2024-05-20T18:37:27.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.733+0000 7f3336051640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f33300d39c0 con 0x7f33300b8190 2024-05-20T18:37:27.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.733+0000 7f3326ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f333006d3d0 0x7f333006d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:27.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.733+0000 7f332f7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3330162820 0x7f33300b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38618/0 (socket says 172.21.15.5:38618) 2024-05-20T18:37:27.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.733+0000 7f332f7fe640 1 -- 172.21.15.5:0/1982572258 learned_addr learned my addr 172.21.15.5:0/1982572258 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:27.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.733+0000 7f332ffff640 1 --2- 172.21.15.5:0/1982572258 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33300b8190 0x7f33300b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:27.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.733+0000 7f3326ffd640 1 -- 172.21.15.5:0/1982572258 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33300b8190 msgr2=0x7f33300b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:27.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.733+0000 7f3326ffd640 1 --2- 172.21.15.5:0/1982572258 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33300b8190 0x7f33300b85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.733+0000 7f3326ffd640 1 -- 172.21.15.5:0/1982572258 >> v1:172.21.15.5:6789/0 conn(0x7f3330162820 legacy=0x7f33300b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:27.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.733+0000 7f3326ffd640 1 -- 172.21.15.5:0/1982572258 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33300d3ca0 con 0x7f333006d3d0 2024-05-20T18:37:27.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.733+0000 7f332effd640 1 -- 172.21.15.5:0/1982572258 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2606158879 0 0) 0x7f3320001110 con 0x7f3330162820 2024-05-20T18:37:27.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.733+0000 7f3326ffd640 1 --2- 172.21.15.5:0/1982572258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f333006d3d0 0x7f333006d7e0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f33180097a0 tx=0x7f331802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb3265eb86c7606 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:27.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.733+0000 7f332effd640 1 -- 172.21.15.5:0/1982572258 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3318037070 con 0x7f333006d3d0 2024-05-20T18:37:27.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.734+0000 7f332effd640 1 -- 172.21.15.5:0/1982572258 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3318032440 con 0x7f333006d3d0 2024-05-20T18:37:27.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.734+0000 7f332effd640 1 -- 172.21.15.5:0/1982572258 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3318032810 con 0x7f333006d3d0 2024-05-20T18:37:27.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.734+0000 7f3336051640 1 -- 172.21.15.5:0/1982572258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f333006d3d0 msgr2=0x7f333006d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:27.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.734+0000 7f3336051640 1 --2- 172.21.15.5:0/1982572258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f333006d3d0 0x7f333006d7e0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f33180097a0 tx=0x7f331802a180 comp rx=0 tx=0).stop 2024-05-20T18:37:27.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.734+0000 7f3336051640 1 -- 172.21.15.5:0/1982572258 shutdown_connections 2024-05-20T18:37:27.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.734+0000 7f3336051640 1 --2- 172.21.15.5:0/1982572258 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33300b8190 0x7f33300b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.734+0000 7f3336051640 1 --2- 172.21.15.5:0/1982572258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f333006d3d0 0x7f333006d7e0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.734+0000 7f3336051640 1 -- 172.21.15.5:0/1982572258 >> 172.21.15.5:0/1982572258 conn(0x7f33300d4bd0 msgr2=0x7f33300d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:27.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.734+0000 7f3336051640 1 -- 172.21.15.5:0/1982572258 shutdown_connections 2024-05-20T18:37:27.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.734+0000 7f3336051640 1 -- 172.21.15.5:0/1982572258 wait complete. 2024-05-20T18:37:27.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.736+0000 7f3336051640 1 Processor -- start 2024-05-20T18:37:27.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.736+0000 7f3336051640 1 -- start start 2024-05-20T18:37:27.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.738+0000 7f3336051640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f333006d3d0 0x7f33301f86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:27.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.738+0000 7f3336051640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33300b8190 0x7f333014a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:27.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.738+0000 7f3336051640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3330162820 0x7f33300cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:27.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.738+0000 7f3336051640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f33300d0300 con 0x7f333006d3d0 2024-05-20T18:37:27.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.738+0000 7f3336051640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f33300d4260 con 0x7f33300b8190 2024-05-20T18:37:27.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.738+0000 7f3336051640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f33300b4050 con 0x7f3330162820 2024-05-20T18:37:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.738+0000 7f332f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f333006d3d0 0x7f33301f86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.738+0000 7f332f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f333006d3d0 0x7f33301f86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34006/0 (socket says 172.21.15.5:34006) 2024-05-20T18:37:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.738+0000 7f332f7fe640 1 -- 172.21.15.5:0/1007955818 learned_addr learned my addr 172.21.15.5:0/1007955818 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.738+0000 7f332ffff640 1 --2- 172.21.15.5:0/1007955818 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3330162820 0x7f33300cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.738+0000 7f3326ffd640 1 --2- 172.21.15.5:0/1007955818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33300b8190 0x7f333014a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.738+0000 7f332f7fe640 1 -- 172.21.15.5:0/1007955818 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3330162820 msgr2=0x7f33300cbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.738+0000 7f332f7fe640 1 --2- 172.21.15.5:0/1007955818 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3330162820 0x7f33300cbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.738+0000 7f332f7fe640 1 -- 172.21.15.5:0/1007955818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33300b8190 msgr2=0x7f333014a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.738+0000 7f332f7fe640 1 --2- 172.21.15.5:0/1007955818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33300b8190 0x7f333014a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.738+0000 7f332f7fe640 1 -- 172.21.15.5:0/1007955818 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33300b5390 con 0x7f333006d3d0 2024-05-20T18:37:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.738+0000 7f332f7fe640 1 --2- 172.21.15.5:0/1007955818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f333006d3d0 0x7f33301f86a0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f332000c430 tx=0x7f332000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.738+0000 7f332cff9640 1 -- 172.21.15.5:0/1007955818 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33200074e0 con 0x7f333006d3d0 2024-05-20T18:37:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.738+0000 7f332cff9640 1 -- 172.21.15.5:0/1007955818 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33200047e0 con 0x7f333006d3d0 2024-05-20T18:37:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.738+0000 7f3336051640 1 -- 172.21.15.5:0/1007955818 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33301f95d0 con 0x7f333006d3d0 2024-05-20T18:37:27.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.739+0000 7f332cff9640 1 -- 172.21.15.5:0/1007955818 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3320004bd0 con 0x7f333006d3d0 2024-05-20T18:37:27.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.739+0000 7f3336051640 1 -- 172.21.15.5:0/1007955818 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33301f92f0 con 0x7f333006d3d0 2024-05-20T18:37:27.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.740+0000 7f3336051640 1 -- 172.21.15.5:0/1007955818 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33300b5560 con 0x7f333006d3d0 2024-05-20T18:37:27.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.740+0000 7f332cff9640 1 -- 172.21.15.5:0/1007955818 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3320011070 con 0x7f333006d3d0 2024-05-20T18:37:27.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.741+0000 7f332cff9640 1 --2- 172.21.15.5:0/1007955818 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f331006be60 0x7f331006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:27.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.741+0000 7f3326ffd640 1 --2- 172.21.15.5:0/1007955818 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f331006be60 0x7f331006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:27.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.741+0000 7f332cff9640 1 -- 172.21.15.5:0/1007955818 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7f332008aa20 con 0x7f333006d3d0 2024-05-20T18:37:27.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.741+0000 7f3326ffd640 1 --2- 172.21.15.5:0/1007955818 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f331006be60 0x7f331006e370 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3318034000 tx=0x7f331802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:27.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.746+0000 7f332cff9640 1 -- 172.21.15.5:0/1007955818 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f333006dd20 con 0x7f333006d3d0 2024-05-20T18:37:27.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.927+0000 7f3336051640 1 -- 172.21.15.5:0/1007955818 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3330075b00 con 0x7f333006d3d0 2024-05-20T18:37:27.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.928+0000 7f332cff9640 1 -- 172.21.15.5:0/1007955818 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v119) ==== 100+0+11 (secure 0 0 0) 0x7f3320058c80 con 0x7f333006d3d0 2024-05-20T18:37:27.929 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:37:27.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.930+0000 7f3336051640 1 -- 172.21.15.5:0/1007955818 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f331006be60 msgr2=0x7f331006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:27.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.930+0000 7f3336051640 1 --2- 172.21.15.5:0/1007955818 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f331006be60 0x7f331006e370 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3318034000 tx=0x7f331802e000 comp rx=0 tx=0).stop 2024-05-20T18:37:27.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.930+0000 7f3336051640 1 -- 172.21.15.5:0/1007955818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f333006d3d0 msgr2=0x7f33301f86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.930+0000 7f3336051640 1 --2- 172.21.15.5:0/1007955818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f333006d3d0 0x7f33301f86a0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f332000c430 tx=0x7f332000c8f0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.930+0000 7f3336051640 1 -- 172.21.15.5:0/1007955818 shutdown_connections 2024-05-20T18:37:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.930+0000 7f3336051640 1 --2- 172.21.15.5:0/1007955818 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f331006be60 0x7f331006e370 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.930+0000 7f3336051640 1 --2- 172.21.15.5:0/1007955818 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3330162820 0x7f33300cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.930+0000 7f3336051640 1 --2- 172.21.15.5:0/1007955818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33300b8190 0x7f333014a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.930+0000 7f3336051640 1 --2- 172.21.15.5:0/1007955818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f333006d3d0 0x7f33301f86a0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.930+0000 7f3336051640 1 -- 172.21.15.5:0/1007955818 >> 172.21.15.5:0/1007955818 conn(0x7f33300d4bd0 msgr2=0x7f333015fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.930+0000 7f3336051640 1 -- 172.21.15.5:0/1007955818 shutdown_connections 2024-05-20T18:37:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:27.930+0000 7f3336051640 1 -- 172.21.15.5:0/1007955818 wait complete. 2024-05-20T18:37:27.943 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T18:37:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.058+0000 7fd2658fe640 1 Processor -- start 2024-05-20T18:37:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.058+0000 7fd2658fe640 1 -- start start 2024-05-20T18:37:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.059+0000 7fd2658fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2601435f0 0x7fd26006dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.059+0000 7fd2658fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd2600d1f00 0x7fd2600b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.059+0000 7fd2658fe640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd26015dcd0 con 0x7fd26015d4a0 2024-05-20T18:37:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.059+0000 7fd2658fe640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd26015dfb0 con 0x7fd2600d1f00 2024-05-20T18:37:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.059+0000 7fd2658fe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd26015e290 con 0x7fd2601435f0 2024-05-20T18:37:28.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.059+0000 7fd25effd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2601435f0 0x7fd26006dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:28.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.059+0000 7fd25f7fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd2600d1f00 0x7fd2600b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:28.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.059+0000 7fd25f7fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd2600d1f00 0x7fd2600b18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:49496/0 (socket says 172.21.15.5:49496) 2024-05-20T18:37:28.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.059+0000 7fd25effd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2601435f0 0x7fd26006dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:44404/0 (socket says 172.21.15.5:44404) 2024-05-20T18:37:28.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.059+0000 7fd25f7fe640 1 -- 172.21.15.5:0/1044756725 learned_addr learned my addr 172.21.15.5:0/1044756725 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:28.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.059+0000 7fd25e7fc640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fd26015d4a0 0x7fd2600d17f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:60156/0 (socket says 172.21.15.5:60156) 2024-05-20T18:37:28.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.060+0000 7fd25dffb640 1 -- 172.21.15.5:0/1044756725 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4022748918 0 0) 0x7fd26015dcd0 con 0x7fd26015d4a0 2024-05-20T18:37:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.060+0000 7fd25dffb640 1 -- 172.21.15.5:0/1044756725 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd248003520 con 0x7fd26015d4a0 2024-05-20T18:37:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.060+0000 7fd25effd640 1 -- 172.21.15.5:0/1044756725 >> v2:172.21.15.63:3300/0 conn(0x7fd2600d1f00 msgr2=0x7fd2600b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.060+0000 7fd25effd640 1 --2- 172.21.15.5:0/1044756725 >> v2:172.21.15.63:3300/0 conn(0x7fd2600d1f00 0x7fd2600b18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.060+0000 7fd25effd640 1 -- 172.21.15.5:0/1044756725 >> v1:172.21.15.63:6789/0 conn(0x7fd26015d4a0 legacy=0x7fd2600d17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.060+0000 7fd25effd640 1 -- 172.21.15.5:0/1044756725 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2600b1390 con 0x7fd2601435f0 2024-05-20T18:37:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.060+0000 7fd25f7fe640 1 --2- 172.21.15.5:0/1044756725 >> v2:172.21.15.63:3300/0 conn(0x7fd2600d1f00 0x7fd2600b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:37:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.061+0000 7fd25effd640 1 --2- 172.21.15.5:0/1044756725 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2601435f0 0x7fd26006dce0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd250009d90 tx=0x7fd25002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=903c958c320c3df6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.061+0000 7fd25dffb640 1 -- 172.21.15.5:0/1044756725 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250007740 con 0x7fd2601435f0 2024-05-20T18:37:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.061+0000 7fd25dffb640 1 -- 172.21.15.5:0/1044756725 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2500078e0 con 0x7fd2601435f0 2024-05-20T18:37:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.061+0000 7fd25dffb640 1 -- 172.21.15.5:0/1044756725 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250007c90 con 0x7fd2601435f0 2024-05-20T18:37:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.062+0000 7fd2658fe640 1 -- 172.21.15.5:0/1044756725 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2601435f0 msgr2=0x7fd26006dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.062+0000 7fd2658fe640 1 --2- 172.21.15.5:0/1044756725 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2601435f0 0x7fd26006dce0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd250009d90 tx=0x7fd25002a660 comp rx=0 tx=0).stop 2024-05-20T18:37:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.062+0000 7fd2658fe640 1 -- 172.21.15.5:0/1044756725 shutdown_connections 2024-05-20T18:37:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.062+0000 7fd2658fe640 1 --2- 172.21.15.5:0/1044756725 >> v2:172.21.15.63:3300/0 conn(0x7fd2600d1f00 0x7fd2600b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.062+0000 7fd2658fe640 1 --2- 172.21.15.5:0/1044756725 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2601435f0 0x7fd26006dce0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.062+0000 7fd2658fe640 1 -- 172.21.15.5:0/1044756725 >> 172.21.15.5:0/1044756725 conn(0x7fd2600b6620 msgr2=0x7fd2600b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.062+0000 7fd2658fe640 1 -- 172.21.15.5:0/1044756725 shutdown_connections 2024-05-20T18:37:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.062+0000 7fd2658fe640 1 -- 172.21.15.5:0/1044756725 wait complete. 2024-05-20T18:37:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.063+0000 7fd2658fe640 1 Processor -- start 2024-05-20T18:37:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.064+0000 7fd2658fe640 1 -- start start 2024-05-20T18:37:28.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.065+0000 7fd2658fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2600d1f00 0x7fd260160470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:28.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.065+0000 7fd2658fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd26015d4a0 0x7fd2601406f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.065+0000 7fd2658fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2601564e0 0x7fd2600afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.065+0000 7fd2658fe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd2600b1f70 con 0x7fd2600d1f00 2024-05-20T18:37:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.065+0000 7fd2658fe640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd2600b1df0 con 0x7fd2601564e0 2024-05-20T18:37:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.065+0000 7fd2658fe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd2600b20f0 con 0x7fd26015d4a0 2024-05-20T18:37:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.065+0000 7fd25effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2600d1f00 0x7fd260160470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.065+0000 7fd25effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2600d1f00 0x7fd260160470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34008/0 (socket says 172.21.15.5:34008) 2024-05-20T18:37:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.065+0000 7fd25effd640 1 -- 172.21.15.5:0/543312299 learned_addr learned my addr 172.21.15.5:0/543312299 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.065+0000 7fd25e7fc640 1 --2- 172.21.15.5:0/543312299 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd26015d4a0 0x7fd2601406f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.065+0000 7fd25f7fe640 1 --2- 172.21.15.5:0/543312299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2601564e0 0x7fd2600afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.065+0000 7fd25effd640 1 -- 172.21.15.5:0/543312299 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd26015d4a0 msgr2=0x7fd2601406f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.065+0000 7fd25effd640 1 --2- 172.21.15.5:0/543312299 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd26015d4a0 0x7fd2601406f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.065+0000 7fd25effd640 1 -- 172.21.15.5:0/543312299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2601564e0 msgr2=0x7fd2600afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.065+0000 7fd25effd640 1 --2- 172.21.15.5:0/543312299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2601564e0 0x7fd2600afea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.065+0000 7fd25effd640 1 -- 172.21.15.5:0/543312299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd260140c30 con 0x7fd2600d1f00 2024-05-20T18:37:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.066+0000 7fd25effd640 1 --2- 172.21.15.5:0/543312299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2600d1f00 0x7fd260160470 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fd250000f80 tx=0x7fd250004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.066+0000 7fd247fff640 1 -- 172.21.15.5:0/543312299 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250009450 con 0x7fd2600d1f00 2024-05-20T18:37:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.066+0000 7fd247fff640 1 -- 172.21.15.5:0/543312299 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd250004240 con 0x7fd2600d1f00 2024-05-20T18:37:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.066+0000 7fd2658fe640 1 -- 172.21.15.5:0/543312299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2600aee00 con 0x7fd2600d1f00 2024-05-20T18:37:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.066+0000 7fd247fff640 1 -- 172.21.15.5:0/543312299 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2500045f0 con 0x7fd2600d1f00 2024-05-20T18:37:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.066+0000 7fd2658fe640 1 -- 172.21.15.5:0/543312299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2600af3c0 con 0x7fd2600d1f00 2024-05-20T18:37:28.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.067+0000 7fd2658fe640 1 -- 172.21.15.5:0/543312299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd260075b00 con 0x7fd2600d1f00 2024-05-20T18:37:28.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.068+0000 7fd247fff640 1 -- 172.21.15.5:0/543312299 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd25003b410 con 0x7fd2600d1f00 2024-05-20T18:37:28.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.068+0000 7fd247fff640 1 --2- 172.21.15.5:0/543312299 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd24c06be30 0x7fd24c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:28.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.068+0000 7fd247fff640 1 -- 172.21.15.5:0/543312299 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7fd2500ac2f0 con 0x7fd2600d1f00 2024-05-20T18:37:28.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.068+0000 7fd25e7fc640 1 --2- 172.21.15.5:0/543312299 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd24c06be30 0x7fd24c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:28.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.069+0000 7fd25e7fc640 1 --2- 172.21.15.5:0/543312299 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd24c06be30 0x7fd24c06e340 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd2600d00a0 tx=0x7fd254007cf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.073+0000 7fd247fff640 1 -- 172.21.15.5:0/543312299 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd260140c30 con 0x7fd2600d1f00 2024-05-20T18:37:28.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.268+0000 7fd2658fe640 1 -- 172.21.15.5:0/543312299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd260142d10 con 0x7fd2600d1f00 2024-05-20T18:37:28.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.269+0000 7fd247fff640 1 -- 172.21.15.5:0/543312299 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v119) ==== 80+0+0 (secure 0 0 0) 0x7fd25003f030 con 0x7fd2600d1f00 2024-05-20T18:37:28.270 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-05-20T18:37:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.271+0000 7fd2658fe640 1 -- 172.21.15.5:0/543312299 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd24c06be30 msgr2=0x7fd24c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.271+0000 7fd2658fe640 1 --2- 172.21.15.5:0/543312299 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd24c06be30 0x7fd24c06e340 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd2600d00a0 tx=0x7fd254007cf0 comp rx=0 tx=0).stop 2024-05-20T18:37:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.271+0000 7fd2658fe640 1 -- 172.21.15.5:0/543312299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2600d1f00 msgr2=0x7fd260160470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.271+0000 7fd2658fe640 1 --2- 172.21.15.5:0/543312299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2600d1f00 0x7fd260160470 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fd250000f80 tx=0x7fd250004a40 comp rx=0 tx=0).stop 2024-05-20T18:37:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.271+0000 7fd2658fe640 1 -- 172.21.15.5:0/543312299 shutdown_connections 2024-05-20T18:37:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.271+0000 7fd2658fe640 1 --2- 172.21.15.5:0/543312299 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd24c06be30 0x7fd24c06e340 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.271+0000 7fd2658fe640 1 --2- 172.21.15.5:0/543312299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2601564e0 0x7fd2600afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.271+0000 7fd2658fe640 1 --2- 172.21.15.5:0/543312299 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd26015d4a0 0x7fd2601406f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.271+0000 7fd2658fe640 1 --2- 172.21.15.5:0/543312299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2600d1f00 0x7fd260160470 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.271+0000 7fd2658fe640 1 -- 172.21.15.5:0/543312299 >> 172.21.15.5:0/543312299 conn(0x7fd2600b6620 msgr2=0x7fd26006d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.271+0000 7fd2658fe640 1 -- 172.21.15.5:0/543312299 shutdown_connections 2024-05-20T18:37:28.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.271+0000 7fd2658fe640 1 -- 172.21.15.5:0/543312299 wait complete. 2024-05-20T18:37:28.284 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-20T18:37:28.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:28.284 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-20T18:37:28.377 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:37:28.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:28.377 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-20T18:37:28.395 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:37:28.395 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:37:28.452 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15663 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T18:37:28.453 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:37:28.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:28.453 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-20T18:37:28.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T18:37:28.509 INFO:teuthology.orchestra.run.smithi005.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-05-20T18:37:28.509 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-20T18:37:28.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.665+0000 7fe1de359640 1 Processor -- start 2024-05-20T18:37:28.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.665+0000 7fe1de359640 1 -- start start 2024-05-20T18:37:28.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.667+0000 7fe1de359640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1d8162820 0x7fe1d80b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:28.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.667+0000 7fe1de359640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe1d80b3e80 0x7fe1d806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:28.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.667+0000 7fe1de359640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1d806dd20 con 0x7fe1d806d910 2024-05-20T18:37:28.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.667+0000 7fe1de359640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe1d80d0630 con 0x7fe1d80b3e80 2024-05-20T18:37:28.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.667+0000 7fe1de359640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe1d80d39c0 con 0x7fe1d8162820 2024-05-20T18:37:28.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.667+0000 7fe1dc8f6640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe1d806d910 0x7fe1d80d27a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38626/0 (socket says 172.21.15.5:38626) 2024-05-20T18:37:28.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.667+0000 7fe1dc8f6640 1 -- 172.21.15.5:0/2995692799 learned_addr learned my addr 172.21.15.5:0/2995692799 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:28.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.667+0000 7fe1d7fff640 1 --2- 172.21.15.5:0/2995692799 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1d8162820 0x7fe1d80b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:28.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.667+0000 7fe1d77fe640 1 --2- 172.21.15.5:0/2995692799 >> v2:172.21.15.63:3300/0 conn(0x7fe1d80b3e80 0x7fe1d806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:28.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.667+0000 7fe1d6ffd640 1 -- 172.21.15.5:0/2995692799 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1985260765 0 0) 0x7fe1d0001490 con 0x7fe1d806d910 2024-05-20T18:37:28.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.667+0000 7fe1d6ffd640 1 -- 172.21.15.5:0/2995692799 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1c0003520 con 0x7fe1d806d910 2024-05-20T18:37:28.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.667+0000 7fe1d7fff640 1 -- 172.21.15.5:0/2995692799 >> v2:172.21.15.63:3300/0 conn(0x7fe1d80b3e80 msgr2=0x7fe1d806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:28.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.667+0000 7fe1d7fff640 1 --2- 172.21.15.5:0/2995692799 >> v2:172.21.15.63:3300/0 conn(0x7fe1d80b3e80 0x7fe1d806d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:28.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.667+0000 7fe1d7fff640 1 -- 172.21.15.5:0/2995692799 >> v1:172.21.15.5:6789/0 conn(0x7fe1d806d910 legacy=0x7fe1d80d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:28.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.667+0000 7fe1d7fff640 1 -- 172.21.15.5:0/2995692799 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d80d3ca0 con 0x7fe1d8162820 2024-05-20T18:37:28.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.667+0000 7fe1d77fe640 1 --2- 172.21.15.5:0/2995692799 >> v2:172.21.15.63:3300/0 conn(0x7fe1d80b3e80 0x7fe1d806d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:37:28.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.668+0000 7fe1d7fff640 1 --2- 172.21.15.5:0/2995692799 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1d8162820 0x7fe1d80b3940 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe1c80097a0 tx=0x7fe1c802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b50781b52b98a97 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:28.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.668+0000 7fe1d6ffd640 1 -- 172.21.15.5:0/2995692799 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8037070 con 0x7fe1d8162820 2024-05-20T18:37:28.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.668+0000 7fe1d6ffd640 1 -- 172.21.15.5:0/2995692799 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1c8032440 con 0x7fe1d8162820 2024-05-20T18:37:28.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.668+0000 7fe1d6ffd640 1 -- 172.21.15.5:0/2995692799 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8032810 con 0x7fe1d8162820 2024-05-20T18:37:28.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.669+0000 7fe1de359640 1 -- 172.21.15.5:0/2995692799 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1d8162820 msgr2=0x7fe1d80b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:28.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.669+0000 7fe1de359640 1 --2- 172.21.15.5:0/2995692799 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1d8162820 0x7fe1d80b3940 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe1c80097a0 tx=0x7fe1c802a180 comp rx=0 tx=0).stop 2024-05-20T18:37:28.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.669+0000 7fe1de359640 1 -- 172.21.15.5:0/2995692799 shutdown_connections 2024-05-20T18:37:28.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.669+0000 7fe1de359640 1 --2- 172.21.15.5:0/2995692799 >> v2:172.21.15.63:3300/0 conn(0x7fe1d80b3e80 0x7fe1d806d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:28.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.669+0000 7fe1de359640 1 --2- 172.21.15.5:0/2995692799 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1d8162820 0x7fe1d80b3940 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:28.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.669+0000 7fe1de359640 1 -- 172.21.15.5:0/2995692799 >> 172.21.15.5:0/2995692799 conn(0x7fe1d80d4bd0 msgr2=0x7fe1d80d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:28.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.669+0000 7fe1de359640 1 -- 172.21.15.5:0/2995692799 shutdown_connections 2024-05-20T18:37:28.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.669+0000 7fe1de359640 1 -- 172.21.15.5:0/2995692799 wait complete. 2024-05-20T18:37:28.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.671+0000 7fe1de359640 1 Processor -- start 2024-05-20T18:37:28.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.671+0000 7fe1de359640 1 -- start start 2024-05-20T18:37:28.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.672+0000 7fe1de359640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1d806d910 0x7fe1d80ce590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:28.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.673+0000 7fe1de359640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1d80b3e80 0x7fe1d815f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:28.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.673+0000 7fe1de359640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d8162820 0x7fe1d815fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:28.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.673+0000 7fe1de359640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d80d0300 con 0x7fe1d8162820 2024-05-20T18:37:28.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.673+0000 7fe1de359640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d80d4260 con 0x7fe1d806d910 2024-05-20T18:37:28.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.673+0000 7fe1de359640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe1d80b91a0 con 0x7fe1d80b3e80 2024-05-20T18:37:28.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.673+0000 7fe1dc8f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d8162820 0x7fe1d815fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:28.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.673+0000 7fe1dc8f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d8162820 0x7fe1d815fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34012/0 (socket says 172.21.15.5:34012) 2024-05-20T18:37:28.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.673+0000 7fe1dc8f6640 1 -- 172.21.15.5:0/2211226754 learned_addr learned my addr 172.21.15.5:0/2211226754 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:28.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.673+0000 7fe1d77fe640 1 --2- 172.21.15.5:0/2211226754 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1d80b3e80 0x7fe1d815f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:28.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.673+0000 7fe1d7fff640 1 --2- 172.21.15.5:0/2211226754 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1d806d910 0x7fe1d80ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:28.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.673+0000 7fe1dc8f6640 1 -- 172.21.15.5:0/2211226754 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1d80b3e80 msgr2=0x7fe1d815f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:28.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.673+0000 7fe1dc8f6640 1 --2- 172.21.15.5:0/2211226754 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1d80b3e80 0x7fe1d815f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:28.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.673+0000 7fe1dc8f6640 1 -- 172.21.15.5:0/2211226754 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1d806d910 msgr2=0x7fe1d80ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:28.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.673+0000 7fe1dc8f6640 1 --2- 172.21.15.5:0/2211226754 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1d806d910 0x7fe1d80ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:28.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.673+0000 7fe1dc8f6640 1 -- 172.21.15.5:0/2211226754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d81e9f30 con 0x7fe1d8162820 2024-05-20T18:37:28.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.673+0000 7fe1dc8f6640 1 --2- 172.21.15.5:0/2211226754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d8162820 0x7fe1d815fa90 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fe1d000cb40 tx=0x7fe1d000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:28.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.673+0000 7fe1d4ff9640 1 -- 172.21.15.5:0/2211226754 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0017070 con 0x7fe1d8162820 2024-05-20T18:37:28.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.674+0000 7fe1d4ff9640 1 -- 172.21.15.5:0/2211226754 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1d000ac40 con 0x7fe1d8162820 2024-05-20T18:37:28.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.674+0000 7fe1de359640 1 -- 172.21.15.5:0/2211226754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1d81ea210 con 0x7fe1d8162820 2024-05-20T18:37:28.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.674+0000 7fe1d4ff9640 1 -- 172.21.15.5:0/2211226754 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0012400 con 0x7fe1d8162820 2024-05-20T18:37:28.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.674+0000 7fe1de359640 1 -- 172.21.15.5:0/2211226754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1d81ea7d0 con 0x7fe1d8162820 2024-05-20T18:37:28.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.674+0000 7fe1de359640 1 -- 172.21.15.5:0/2211226754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1d8075b00 con 0x7fe1d8162820 2024-05-20T18:37:28.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.675+0000 7fe1d4ff9640 1 -- 172.21.15.5:0/2211226754 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe1d001a070 con 0x7fe1d8162820 2024-05-20T18:37:28.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.676+0000 7fe1d4ff9640 1 --2- 172.21.15.5:0/2211226754 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe1c406be30 0x7fe1c406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:28.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.676+0000 7fe1d7fff640 1 --2- 172.21.15.5:0/2211226754 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe1c406be30 0x7fe1c406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:28.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.676+0000 7fe1d4ff9640 1 -- 172.21.15.5:0/2211226754 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7fe1d008ad60 con 0x7fe1d8162820 2024-05-20T18:37:28.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.676+0000 7fe1d7fff640 1 --2- 172.21.15.5:0/2211226754 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe1c406be30 0x7fe1c406e340 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8034000 tx=0x7fe1c802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:28.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.681+0000 7fe1d4ff9640 1 -- 172.21.15.5:0/2211226754 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe1d806dd20 con 0x7fe1d8162820 2024-05-20T18:37:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.901+0000 7fe1de359640 1 -- 172.21.15.5:0/2211226754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe1d81e9230 con 0x7fe1d8162820 2024-05-20T18:37:28.902 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-20T18:37:28.902+0000 7fd1cf53a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-20T18:37:28.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.903+0000 7fe1d4ff9640 1 -- 172.21.15.5:0/2211226754 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v99) ==== 161+0+0 (secure 0 0 0) 0x7fe1d0058ea0 con 0x7fe1d8162820 2024-05-20T18:37:28.903 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-20T18:37:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.906+0000 7fe1de359640 1 -- 172.21.15.5:0/2211226754 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe1c406be30 msgr2=0x7fe1c406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.906+0000 7fe1de359640 1 --2- 172.21.15.5:0/2211226754 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe1c406be30 0x7fe1c406e340 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8034000 tx=0x7fe1c802e000 comp rx=0 tx=0).stop 2024-05-20T18:37:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.906+0000 7fe1de359640 1 -- 172.21.15.5:0/2211226754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d8162820 msgr2=0x7fe1d815fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.906+0000 7fe1de359640 1 --2- 172.21.15.5:0/2211226754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d8162820 0x7fe1d815fa90 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fe1d000cb40 tx=0x7fe1d000a060 comp rx=0 tx=0).stop 2024-05-20T18:37:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.906+0000 7fe1de359640 1 -- 172.21.15.5:0/2211226754 shutdown_connections 2024-05-20T18:37:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.906+0000 7fe1de359640 1 --2- 172.21.15.5:0/2211226754 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe1c406be30 0x7fe1c406e340 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.906+0000 7fe1de359640 1 --2- 172.21.15.5:0/2211226754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d8162820 0x7fe1d815fa90 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.906+0000 7fe1de359640 1 --2- 172.21.15.5:0/2211226754 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe1d80b3e80 0x7fe1d815f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.906+0000 7fe1de359640 1 --2- 172.21.15.5:0/2211226754 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1d806d910 0x7fe1d80ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.906+0000 7fe1de359640 1 -- 172.21.15.5:0/2211226754 >> 172.21.15.5:0/2211226754 conn(0x7fe1d80d4bd0 msgr2=0x7fe1d80b94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.906+0000 7fe1de359640 1 -- 172.21.15.5:0/2211226754 shutdown_connections 2024-05-20T18:37:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:28.906+0000 7fe1de359640 1 -- 172.21.15.5:0/2211226754 wait complete. 2024-05-20T18:37:28.911 INFO:tasks.ceph.mds.d.smithi063.stderr: -8550> 2024-05-20T18:37:16.970+0000 7fcea4cf1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T18:37:28.919 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-20T18:37:28.921 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-20T18:37:28.964 INFO:teuthology.orchestra.run.smithi005.stdout:[global] 2024-05-20T18:37:28.965 INFO:teuthology.orchestra.run.smithi005.stdout: chdir = "" 2024-05-20T18:37:28.965 INFO:teuthology.orchestra.run.smithi005.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-20T18:37:28.965 INFO:teuthology.orchestra.run.smithi005.stdout: auth supported = cephx 2024-05-20T18:37:28.965 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:28.965 INFO:teuthology.orchestra.run.smithi005.stdout: filestore xattr use omap = true 2024-05-20T18:37:28.965 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:28.965 INFO:teuthology.orchestra.run.smithi005.stdout: mon clock drift allowed = 1.000 2024-05-20T18:37:28.965 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:28.965 INFO:teuthology.orchestra.run.smithi005.stdout: osd crush chooseleaf type = 0 2024-05-20T18:37:28.965 INFO:teuthology.orchestra.run.smithi005.stdout: auth debug = true 2024-05-20T18:37:28.965 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:28.965 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on old message = true 2024-05-20T18:37:28.966 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on bug = true 2024-05-20T18:37:28.966 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:28.966 INFO:teuthology.orchestra.run.smithi005.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-20T18:37:28.966 INFO:teuthology.orchestra.run.smithi005.stdout: mon pg warn max object skew = 0 2024-05-20T18:37:28.966 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:28.966 INFO:teuthology.orchestra.run.smithi005.stdout: # disable pg_autoscaler by default for new pools 2024-05-20T18:37:28.966 INFO:teuthology.orchestra.run.smithi005.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-20T18:37:28.966 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:28.966 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default size = 2 2024-05-20T18:37:28.966 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:28.966 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow primary affinity = true 2024-05-20T18:37:28.966 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow pg remap = true 2024-05-20T18:37:28.967 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on legacy crush tunables = false 2024-05-20T18:37:28.967 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on crush straw calc version zero = false 2024-05-20T18:37:28.967 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on no sortbitwise = false 2024-05-20T18:37:28.967 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on osd down out interval zero = false 2024-05-20T18:37:28.967 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on too few osds = false 2024-05-20T18:37:28.967 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-20T18:37:28.967 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-20T18:37:28.967 INFO:teuthology.orchestra.run.smithi005.stdout: mon_allow_pool_size_one = true 2024-05-20T18:37:28.967 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:28.967 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-20T18:37:28.967 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:28.967 INFO:teuthology.orchestra.run.smithi005.stdout: osd default data pool replay window = 5 2024-05-20T18:37:28.967 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:28.968 INFO:teuthology.orchestra.run.smithi005.stdout: mon allow pool delete = true 2024-05-20T18:37:28.968 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:28.968 INFO:teuthology.orchestra.run.smithi005.stdout: mon cluster log file level = debug 2024-05-20T18:37:28.968 INFO:teuthology.orchestra.run.smithi005.stdout: debug asserts on shutdown = true 2024-05-20T18:37:28.968 INFO:teuthology.orchestra.run.smithi005.stdout: mon health detail to clog = false 2024-05-20T18:37:28.968 INFO:teuthology.orchestra.run.smithi005.stdout: mon host = "172.21.15.5,172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790]" 2024-05-20T18:37:28.968 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on skipped message = False 2024-05-20T18:37:28.968 INFO:teuthology.orchestra.run.smithi005.stdout: fsid = 399ab250-fd3c-47fa-a1c8-25e8149c9cac 2024-05-20T18:37:28.968 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:28.968 INFO:teuthology.orchestra.run.smithi005.stdout:[osd] 2024-05-20T18:37:28.968 INFO:teuthology.orchestra.run.smithi005.stdout: osd journal size = 100 2024-05-20T18:37:28.968 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:28.969 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub load threshold = 5.0 2024-05-20T18:37:28.969 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub max interval = 600 2024-05-20T18:37:28.969 INFO:teuthology.orchestra.run.smithi005.stdout: osd mclock profile = high_recovery_ops 2024-05-20T18:37:28.969 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:28.969 INFO:teuthology.orchestra.run.smithi005.stdout: osd recover clone overlap = true 2024-05-20T18:37:28.969 INFO:teuthology.orchestra.run.smithi005.stdout: osd recovery max chunk = 1048576 2024-05-20T18:37:28.969 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:28.969 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug shutdown = true 2024-05-20T18:37:28.969 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug op order = true 2024-05-20T18:37:28.969 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug verify stray on activate = true 2024-05-20T18:37:28.969 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug trim objects = true 2024-05-20T18:37:28.969 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:28.969 INFO:teuthology.orchestra.run.smithi005.stdout: osd open classes on start = true 2024-05-20T18:37:28.970 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug pg log writeout = true 2024-05-20T18:37:28.970 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:28.970 INFO:teuthology.orchestra.run.smithi005.stdout: osd deep scrub update digest min age = 30 2024-05-20T18:37:28.970 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:28.970 INFO:teuthology.orchestra.run.smithi005.stdout: osd map max advance = 10 2024-05-20T18:37:28.970 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:28.970 INFO:teuthology.orchestra.run.smithi005.stdout: journal zero on create = true 2024-05-20T18:37:28.970 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:28.970 INFO:teuthology.orchestra.run.smithi005.stdout: filestore ondisk finisher threads = 3 2024-05-20T18:37:28.970 INFO:teuthology.orchestra.run.smithi005.stdout: filestore apply finisher threads = 3 2024-05-20T18:37:28.970 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:28.970 INFO:teuthology.orchestra.run.smithi005.stdout: bdev debug aio = true 2024-05-20T18:37:28.970 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug misdirected ops = true 2024-05-20T18:37:28.970 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore block size = 96636764160 2024-05-20T18:37:28.971 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore fsck on mount = True 2024-05-20T18:37:28.971 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluefs = 20 2024-05-20T18:37:28.971 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluestore = 20 2024-05-20T18:37:28.971 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-20T18:37:28.971 INFO:teuthology.orchestra.run.smithi005.stdout: debug osd = 20 2024-05-20T18:37:28.971 INFO:teuthology.orchestra.run.smithi005.stdout: debug rocksdb = 10 2024-05-20T18:37:28.971 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd backfillfull_ratio = 0.85 2024-05-20T18:37:28.971 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd full ratio = 0.9 2024-05-20T18:37:28.971 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd nearfull ratio = 0.8 2024-05-20T18:37:28.971 INFO:teuthology.orchestra.run.smithi005.stdout: osd failsafe full ratio = 0.95 2024-05-20T18:37:28.971 INFO:teuthology.orchestra.run.smithi005.stdout: osd objectstore = bluestore 2024-05-20T18:37:28.971 INFO:teuthology.orchestra.run.smithi005.stdout: osd op complaint time = 180 2024-05-20T18:37:28.971 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:28.971 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr] 2024-05-20T18:37:28.971 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-20T18:37:28.971 INFO:teuthology.orchestra.run.smithi005.stdout: debug mgr = 20 2024-05-20T18:37:28.972 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2024-05-20T18:37:28.972 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2024-05-20T18:37:28.972 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2024-05-20T18:37:28.972 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2024-05-20T18:37:28.972 INFO:teuthology.orchestra.run.smithi005.stdout: mgr/telemetry/nag = false 2024-05-20T18:37:28.972 INFO:teuthology.orchestra.run.smithi005.stdout: debug client = 20 2024-05-20T18:37:28.972 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:28.972 INFO:teuthology.orchestra.run.smithi005.stdout:[mon] 2024-05-20T18:37:28.972 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-20T18:37:28.972 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2024-05-20T18:37:28.972 INFO:teuthology.orchestra.run.smithi005.stdout: debug paxos = 20 2024-05-20T18:37:28.972 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2024-05-20T18:37:28.972 INFO:teuthology.orchestra.run.smithi005.stdout: mon data avail warn = 5 2024-05-20T18:37:28.973 INFO:teuthology.orchestra.run.smithi005.stdout: mon mgr mkfs grace = 240 2024-05-20T18:37:28.973 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2024-05-20T18:37:28.973 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd reporter subtree level = osd 2024-05-20T18:37:28.973 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd prime pg temp = true 2024-05-20T18:37:28.973 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2024-05-20T18:37:28.973 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:28.973 INFO:teuthology.orchestra.run.smithi005.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-20T18:37:28.973 INFO:teuthology.orchestra.run.smithi005.stdout: auth mon ticket ttl = 660 # 11m 2024-05-20T18:37:28.973 INFO:teuthology.orchestra.run.smithi005.stdout: auth service ticket ttl = 240 # 4m 2024-05-20T18:37:28.973 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:28.973 INFO:teuthology.orchestra.run.smithi005.stdout: # don't complain about insecure global_id in the test suite 2024-05-20T18:37:28.973 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-20T18:37:28.973 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-20T18:37:28.973 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:28.973 INFO:teuthology.orchestra.run.smithi005.stdout: # 1m isn't quite enough 2024-05-20T18:37:28.974 INFO:teuthology.orchestra.run.smithi005.stdout: mon_down_mkfs_grace = 2m 2024-05-20T18:37:28.974 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:28.974 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_filestore_osds = false 2024-05-20T18:37:28.974 INFO:teuthology.orchestra.run.smithi005.stdout: mon down mkfs grace = 300 2024-05-20T18:37:28.974 INFO:teuthology.orchestra.run.smithi005.stdout: mon op complaint time = 120 2024-05-20T18:37:28.974 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:28.974 INFO:teuthology.orchestra.run.smithi005.stdout:[client] 2024-05-20T18:37:28.974 INFO:teuthology.orchestra.run.smithi005.stdout: rgw cache enabled = true 2024-05-20T18:37:28.974 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable ops log = true 2024-05-20T18:37:28.974 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable usage log = true 2024-05-20T18:37:28.974 INFO:teuthology.orchestra.run.smithi005.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-20T18:37:28.974 INFO:teuthology.orchestra.run.smithi005.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-20T18:37:28.974 INFO:teuthology.orchestra.run.smithi005.stdout: client mount timeout = 600 2024-05-20T18:37:28.975 INFO:teuthology.orchestra.run.smithi005.stdout: debug client = 20 2024-05-20T18:37:28.975 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-20T18:37:28.975 INFO:teuthology.orchestra.run.smithi005.stdout: pid file = "" 2024-05-20T18:37:28.975 INFO:teuthology.orchestra.run.smithi005.stdout: rados mon op timeout = 900 2024-05-20T18:37:28.975 INFO:teuthology.orchestra.run.smithi005.stdout: rados osd op timeout = 900 2024-05-20T18:37:28.975 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.a] 2024-05-20T18:37:28.975 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.b] 2024-05-20T18:37:28.975 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.c] 2024-05-20T18:37:28.975 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a] 2024-05-20T18:37:28.975 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.c] 2024-05-20T18:37:28.975 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b] 2024-05-20T18:37:28.975 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.d] 2024-05-20T18:37:28.975 INFO:teuthology.orchestra.run.smithi005.stdout:[mds] 2024-05-20T18:37:28.975 INFO:teuthology.orchestra.run.smithi005.stdout: debug mds = 20 2024-05-20T18:37:28.976 INFO:teuthology.orchestra.run.smithi005.stdout: debug mds balancer = 20 2024-05-20T18:37:28.976 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-20T18:37:28.976 INFO:teuthology.orchestra.run.smithi005.stdout: mds debug frag = True 2024-05-20T18:37:28.976 INFO:teuthology.orchestra.run.smithi005.stdout: mds debug scatterstat = True 2024-05-20T18:37:28.976 INFO:teuthology.orchestra.run.smithi005.stdout: mds op complaint time = 180 2024-05-20T18:37:28.976 INFO:teuthology.orchestra.run.smithi005.stdout: mds verify scatter = False 2024-05-20T18:37:28.976 INFO:teuthology.orchestra.run.smithi005.stdout: osd op complaint time = 180 2024-05-20T18:37:28.976 INFO:teuthology.orchestra.run.smithi005.stdout: rados mon op timeout = 900 2024-05-20T18:37:28.976 INFO:teuthology.orchestra.run.smithi005.stdout: rados osd op timeout = 900 2024-05-20T18:37:28.977 INFO:teuthology.orchestra.run.smithi063.stdout:[global] 2024-05-20T18:37:28.977 INFO:teuthology.orchestra.run.smithi063.stdout: chdir = "" 2024-05-20T18:37:28.977 INFO:teuthology.orchestra.run.smithi063.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-20T18:37:28.977 INFO:teuthology.orchestra.run.smithi063.stdout: auth supported = cephx 2024-05-20T18:37:28.977 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:28.977 INFO:teuthology.orchestra.run.smithi063.stdout: filestore xattr use omap = true 2024-05-20T18:37:28.977 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:28.977 INFO:teuthology.orchestra.run.smithi063.stdout: mon clock drift allowed = 1.000 2024-05-20T18:37:28.977 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:28.978 INFO:teuthology.orchestra.run.smithi063.stdout: osd crush chooseleaf type = 0 2024-05-20T18:37:28.978 INFO:teuthology.orchestra.run.smithi063.stdout: auth debug = true 2024-05-20T18:37:28.978 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:28.978 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on old message = true 2024-05-20T18:37:28.978 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on bug = true 2024-05-20T18:37:28.978 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:28.978 INFO:teuthology.orchestra.run.smithi063.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-20T18:37:28.978 INFO:teuthology.orchestra.run.smithi063.stdout: mon pg warn max object skew = 0 2024-05-20T18:37:28.978 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:28.978 INFO:teuthology.orchestra.run.smithi063.stdout: # disable pg_autoscaler by default for new pools 2024-05-20T18:37:28.978 INFO:teuthology.orchestra.run.smithi063.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-20T18:37:28.978 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:28.979 INFO:teuthology.orchestra.run.smithi063.stdout: osd pool default size = 2 2024-05-20T18:37:28.979 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:28.979 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd allow primary affinity = true 2024-05-20T18:37:28.979 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd allow pg remap = true 2024-05-20T18:37:28.979 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on legacy crush tunables = false 2024-05-20T18:37:28.979 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on crush straw calc version zero = false 2024-05-20T18:37:28.979 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on no sortbitwise = false 2024-05-20T18:37:28.979 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on osd down out interval zero = false 2024-05-20T18:37:28.979 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on too few osds = false 2024-05-20T18:37:28.979 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-20T18:37:28.979 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-20T18:37:28.979 INFO:teuthology.orchestra.run.smithi063.stdout: mon_allow_pool_size_one = true 2024-05-20T18:37:28.980 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:28.980 INFO:teuthology.orchestra.run.smithi063.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-20T18:37:28.980 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:28.980 INFO:teuthology.orchestra.run.smithi063.stdout: osd default data pool replay window = 5 2024-05-20T18:37:28.980 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:28.980 INFO:teuthology.orchestra.run.smithi063.stdout: mon allow pool delete = true 2024-05-20T18:37:28.980 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:28.980 INFO:teuthology.orchestra.run.smithi063.stdout: mon cluster log file level = debug 2024-05-20T18:37:28.980 INFO:teuthology.orchestra.run.smithi063.stdout: debug asserts on shutdown = true 2024-05-20T18:37:28.981 INFO:teuthology.orchestra.run.smithi063.stdout: mon health detail to clog = false 2024-05-20T18:37:28.981 INFO:teuthology.orchestra.run.smithi063.stdout: mon host = "172.21.15.5,172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790]" 2024-05-20T18:37:28.981 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on skipped message = False 2024-05-20T18:37:28.981 INFO:teuthology.orchestra.run.smithi063.stdout: fsid = 399ab250-fd3c-47fa-a1c8-25e8149c9cac 2024-05-20T18:37:28.981 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:28.981 INFO:teuthology.orchestra.run.smithi063.stdout:[osd] 2024-05-20T18:37:28.981 INFO:teuthology.orchestra.run.smithi063.stdout: osd journal size = 100 2024-05-20T18:37:28.981 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:28.981 INFO:teuthology.orchestra.run.smithi063.stdout: osd scrub load threshold = 5.0 2024-05-20T18:37:28.981 INFO:teuthology.orchestra.run.smithi063.stdout: osd scrub max interval = 600 2024-05-20T18:37:28.981 INFO:teuthology.orchestra.run.smithi063.stdout: osd mclock profile = high_recovery_ops 2024-05-20T18:37:28.982 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:28.982 INFO:teuthology.orchestra.run.smithi063.stdout: osd recover clone overlap = true 2024-05-20T18:37:28.982 INFO:teuthology.orchestra.run.smithi063.stdout: osd recovery max chunk = 1048576 2024-05-20T18:37:28.982 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:28.982 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug shutdown = true 2024-05-20T18:37:28.982 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug op order = true 2024-05-20T18:37:28.982 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug verify stray on activate = true 2024-05-20T18:37:28.982 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug trim objects = true 2024-05-20T18:37:28.982 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:28.982 INFO:teuthology.orchestra.run.smithi063.stdout: osd open classes on start = true 2024-05-20T18:37:28.982 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug pg log writeout = true 2024-05-20T18:37:28.982 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:28.982 INFO:teuthology.orchestra.run.smithi063.stdout: osd deep scrub update digest min age = 30 2024-05-20T18:37:28.982 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:28.983 INFO:teuthology.orchestra.run.smithi063.stdout: osd map max advance = 10 2024-05-20T18:37:28.983 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:28.983 INFO:teuthology.orchestra.run.smithi063.stdout: journal zero on create = true 2024-05-20T18:37:28.983 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:28.983 INFO:teuthology.orchestra.run.smithi063.stdout: filestore ondisk finisher threads = 3 2024-05-20T18:37:28.983 INFO:teuthology.orchestra.run.smithi063.stdout: filestore apply finisher threads = 3 2024-05-20T18:37:28.983 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:28.983 INFO:teuthology.orchestra.run.smithi063.stdout: bdev debug aio = true 2024-05-20T18:37:28.983 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug misdirected ops = true 2024-05-20T18:37:28.983 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore block size = 96636764160 2024-05-20T18:37:28.983 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore fsck on mount = True 2024-05-20T18:37:28.983 INFO:teuthology.orchestra.run.smithi063.stdout: debug bluefs = 20 2024-05-20T18:37:28.983 INFO:teuthology.orchestra.run.smithi063.stdout: debug bluestore = 20 2024-05-20T18:37:28.983 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-20T18:37:28.984 INFO:teuthology.orchestra.run.smithi063.stdout: debug osd = 20 2024-05-20T18:37:28.984 INFO:teuthology.orchestra.run.smithi063.stdout: debug rocksdb = 10 2024-05-20T18:37:28.984 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd backfillfull_ratio = 0.85 2024-05-20T18:37:28.984 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd full ratio = 0.9 2024-05-20T18:37:28.984 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd nearfull ratio = 0.8 2024-05-20T18:37:28.984 INFO:teuthology.orchestra.run.smithi063.stdout: osd failsafe full ratio = 0.95 2024-05-20T18:37:28.984 INFO:teuthology.orchestra.run.smithi063.stdout: osd objectstore = bluestore 2024-05-20T18:37:28.984 INFO:teuthology.orchestra.run.smithi063.stdout: osd op complaint time = 180 2024-05-20T18:37:28.984 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:28.984 INFO:teuthology.orchestra.run.smithi063.stdout:[mgr] 2024-05-20T18:37:28.984 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-20T18:37:28.984 INFO:teuthology.orchestra.run.smithi063.stdout: debug mgr = 20 2024-05-20T18:37:28.984 INFO:teuthology.orchestra.run.smithi063.stdout: debug mon = 20 2024-05-20T18:37:28.985 INFO:teuthology.orchestra.run.smithi063.stdout: debug auth = 20 2024-05-20T18:37:28.985 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min pgs per osd = 4 2024-05-20T18:37:28.985 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min bytes per osd = 10 2024-05-20T18:37:28.985 INFO:teuthology.orchestra.run.smithi063.stdout: mgr/telemetry/nag = false 2024-05-20T18:37:28.985 INFO:teuthology.orchestra.run.smithi063.stdout: debug client = 20 2024-05-20T18:37:28.985 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:28.985 INFO:teuthology.orchestra.run.smithi063.stdout:[mon] 2024-05-20T18:37:28.985 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-20T18:37:28.985 INFO:teuthology.orchestra.run.smithi063.stdout: debug mon = 20 2024-05-20T18:37:28.985 INFO:teuthology.orchestra.run.smithi063.stdout: debug paxos = 20 2024-05-20T18:37:28.985 INFO:teuthology.orchestra.run.smithi063.stdout: debug auth = 20 2024-05-20T18:37:28.986 INFO:teuthology.orchestra.run.smithi063.stdout: mon data avail warn = 5 2024-05-20T18:37:28.986 INFO:teuthology.orchestra.run.smithi063.stdout: mon mgr mkfs grace = 240 2024-05-20T18:37:28.986 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min pgs per osd = 4 2024-05-20T18:37:28.986 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd reporter subtree level = osd 2024-05-20T18:37:28.986 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd prime pg temp = true 2024-05-20T18:37:28.986 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min bytes per osd = 10 2024-05-20T18:37:28.986 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:28.986 INFO:teuthology.orchestra.run.smithi063.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-20T18:37:28.986 INFO:teuthology.orchestra.run.smithi063.stdout: auth mon ticket ttl = 660 # 11m 2024-05-20T18:37:28.986 INFO:teuthology.orchestra.run.smithi063.stdout: auth service ticket ttl = 240 # 4m 2024-05-20T18:37:28.986 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:28.986 INFO:teuthology.orchestra.run.smithi063.stdout: # don't complain about insecure global_id in the test suite 2024-05-20T18:37:28.986 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-20T18:37:28.986 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-20T18:37:28.987 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:28.987 INFO:teuthology.orchestra.run.smithi063.stdout: # 1m isn't quite enough 2024-05-20T18:37:28.987 INFO:teuthology.orchestra.run.smithi063.stdout: mon_down_mkfs_grace = 2m 2024-05-20T18:37:28.987 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:28.987 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_filestore_osds = false 2024-05-20T18:37:28.987 INFO:teuthology.orchestra.run.smithi063.stdout: mon down mkfs grace = 300 2024-05-20T18:37:28.987 INFO:teuthology.orchestra.run.smithi063.stdout: mon op complaint time = 120 2024-05-20T18:37:28.987 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:28.987 INFO:teuthology.orchestra.run.smithi063.stdout:[client] 2024-05-20T18:37:28.987 INFO:teuthology.orchestra.run.smithi063.stdout: rgw cache enabled = true 2024-05-20T18:37:28.987 INFO:teuthology.orchestra.run.smithi063.stdout: rgw enable ops log = true 2024-05-20T18:37:28.987 INFO:teuthology.orchestra.run.smithi063.stdout: rgw enable usage log = true 2024-05-20T18:37:28.987 INFO:teuthology.orchestra.run.smithi063.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-20T18:37:28.987 INFO:teuthology.orchestra.run.smithi063.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-20T18:37:28.988 INFO:teuthology.orchestra.run.smithi063.stdout: client mount timeout = 600 2024-05-20T18:37:28.988 INFO:teuthology.orchestra.run.smithi063.stdout: debug client = 20 2024-05-20T18:37:28.988 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-20T18:37:28.988 INFO:teuthology.orchestra.run.smithi063.stdout: pid file = "" 2024-05-20T18:37:28.988 INFO:teuthology.orchestra.run.smithi063.stdout: rados mon op timeout = 900 2024-05-20T18:37:28.988 INFO:teuthology.orchestra.run.smithi063.stdout: rados osd op timeout = 900 2024-05-20T18:37:28.988 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.a] 2024-05-20T18:37:28.988 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.b] 2024-05-20T18:37:28.988 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.c] 2024-05-20T18:37:28.988 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.a] 2024-05-20T18:37:28.988 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.c] 2024-05-20T18:37:28.988 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.b] 2024-05-20T18:37:28.988 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.d] 2024-05-20T18:37:28.989 INFO:teuthology.orchestra.run.smithi063.stdout:[mds] 2024-05-20T18:37:28.989 INFO:teuthology.orchestra.run.smithi063.stdout: debug mds = 20 2024-05-20T18:37:28.989 INFO:teuthology.orchestra.run.smithi063.stdout: debug mds balancer = 20 2024-05-20T18:37:28.989 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-20T18:37:28.989 INFO:teuthology.orchestra.run.smithi063.stdout: mds debug frag = True 2024-05-20T18:37:28.989 INFO:teuthology.orchestra.run.smithi063.stdout: mds debug scatterstat = True 2024-05-20T18:37:28.989 INFO:teuthology.orchestra.run.smithi063.stdout: mds op complaint time = 180 2024-05-20T18:37:28.989 INFO:teuthology.orchestra.run.smithi063.stdout: mds verify scatter = False 2024-05-20T18:37:28.989 INFO:teuthology.orchestra.run.smithi063.stdout: osd op complaint time = 180 2024-05-20T18:37:28.989 INFO:teuthology.orchestra.run.smithi063.stdout: rados mon op timeout = 900 2024-05-20T18:37:28.989 INFO:teuthology.orchestra.run.smithi063.stdout: rados osd op timeout = 900 2024-05-20T18:37:28.990 INFO:tasks.ceph.mds.d.smithi063.stdout:starting mds.d at 2024-05-20T18:37:28.991 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-20T18:37:29.035 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-20T18:37:29.080 INFO:teuthology.orchestra.run.smithi005.stdout:[global] 2024-05-20T18:37:29.080 INFO:teuthology.orchestra.run.smithi005.stdout: chdir = "" 2024-05-20T18:37:29.081 INFO:teuthology.orchestra.run.smithi005.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-20T18:37:29.081 INFO:teuthology.orchestra.run.smithi005.stdout: auth supported = cephx 2024-05-20T18:37:29.081 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:29.081 INFO:teuthology.orchestra.run.smithi005.stdout: filestore xattr use omap = true 2024-05-20T18:37:29.081 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:29.081 INFO:teuthology.orchestra.run.smithi005.stdout: mon clock drift allowed = 1.000 2024-05-20T18:37:29.081 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:29.081 INFO:teuthology.orchestra.run.smithi005.stdout: osd crush chooseleaf type = 0 2024-05-20T18:37:29.081 INFO:teuthology.orchestra.run.smithi005.stdout: auth debug = true 2024-05-20T18:37:29.081 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:29.081 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on old message = true 2024-05-20T18:37:29.081 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on bug = true 2024-05-20T18:37:29.082 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:29.082 INFO:teuthology.orchestra.run.smithi005.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-20T18:37:29.082 INFO:teuthology.orchestra.run.smithi005.stdout: mon pg warn max object skew = 0 2024-05-20T18:37:29.082 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:29.082 INFO:teuthology.orchestra.run.smithi005.stdout: # disable pg_autoscaler by default for new pools 2024-05-20T18:37:29.082 INFO:teuthology.orchestra.run.smithi005.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-20T18:37:29.082 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:29.082 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default size = 2 2024-05-20T18:37:29.082 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:29.082 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow primary affinity = true 2024-05-20T18:37:29.082 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow pg remap = true 2024-05-20T18:37:29.083 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on legacy crush tunables = false 2024-05-20T18:37:29.083 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on crush straw calc version zero = false 2024-05-20T18:37:29.083 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on no sortbitwise = false 2024-05-20T18:37:29.083 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on osd down out interval zero = false 2024-05-20T18:37:29.083 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on too few osds = false 2024-05-20T18:37:29.083 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-20T18:37:29.083 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-20T18:37:29.083 INFO:teuthology.orchestra.run.smithi005.stdout: mon_allow_pool_size_one = true 2024-05-20T18:37:29.083 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:29.083 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-20T18:37:29.083 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:29.083 INFO:teuthology.orchestra.run.smithi005.stdout: osd default data pool replay window = 5 2024-05-20T18:37:29.083 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:29.083 INFO:teuthology.orchestra.run.smithi005.stdout: mon allow pool delete = true 2024-05-20T18:37:29.084 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:29.084 INFO:teuthology.orchestra.run.smithi005.stdout: mon cluster log file level = debug 2024-05-20T18:37:29.084 INFO:teuthology.orchestra.run.smithi005.stdout: debug asserts on shutdown = true 2024-05-20T18:37:29.084 INFO:teuthology.orchestra.run.smithi005.stdout: mon health detail to clog = false 2024-05-20T18:37:29.084 INFO:teuthology.orchestra.run.smithi005.stdout: mon host = "172.21.15.5,172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790]" 2024-05-20T18:37:29.084 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on skipped message = False 2024-05-20T18:37:29.084 INFO:teuthology.orchestra.run.smithi005.stdout: fsid = 399ab250-fd3c-47fa-a1c8-25e8149c9cac 2024-05-20T18:37:29.084 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:29.084 INFO:teuthology.orchestra.run.smithi005.stdout:[osd] 2024-05-20T18:37:29.084 INFO:teuthology.orchestra.run.smithi005.stdout: osd journal size = 100 2024-05-20T18:37:29.084 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:29.084 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub load threshold = 5.0 2024-05-20T18:37:29.084 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub max interval = 600 2024-05-20T18:37:29.084 INFO:teuthology.orchestra.run.smithi005.stdout: osd mclock profile = high_recovery_ops 2024-05-20T18:37:29.085 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:29.085 INFO:teuthology.orchestra.run.smithi005.stdout: osd recover clone overlap = true 2024-05-20T18:37:29.085 INFO:teuthology.orchestra.run.smithi005.stdout: osd recovery max chunk = 1048576 2024-05-20T18:37:29.085 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:29.085 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug shutdown = true 2024-05-20T18:37:29.085 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug op order = true 2024-05-20T18:37:29.085 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug verify stray on activate = true 2024-05-20T18:37:29.085 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug trim objects = true 2024-05-20T18:37:29.085 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:29.085 INFO:teuthology.orchestra.run.smithi005.stdout: osd open classes on start = true 2024-05-20T18:37:29.085 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug pg log writeout = true 2024-05-20T18:37:29.085 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:29.085 INFO:teuthology.orchestra.run.smithi005.stdout: osd deep scrub update digest min age = 30 2024-05-20T18:37:29.085 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:29.086 INFO:teuthology.orchestra.run.smithi005.stdout: osd map max advance = 10 2024-05-20T18:37:29.086 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:29.086 INFO:teuthology.orchestra.run.smithi005.stdout: journal zero on create = true 2024-05-20T18:37:29.086 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:29.086 INFO:teuthology.orchestra.run.smithi005.stdout: filestore ondisk finisher threads = 3 2024-05-20T18:37:29.086 INFO:teuthology.orchestra.run.smithi005.stdout: filestore apply finisher threads = 3 2024-05-20T18:37:29.086 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:29.086 INFO:teuthology.orchestra.run.smithi005.stdout: bdev debug aio = true 2024-05-20T18:37:29.086 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug misdirected ops = true 2024-05-20T18:37:29.086 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore block size = 96636764160 2024-05-20T18:37:29.086 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore fsck on mount = True 2024-05-20T18:37:29.086 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluefs = 20 2024-05-20T18:37:29.086 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluestore = 20 2024-05-20T18:37:29.086 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-20T18:37:29.087 INFO:teuthology.orchestra.run.smithi005.stdout: debug osd = 20 2024-05-20T18:37:29.087 INFO:teuthology.orchestra.run.smithi005.stdout: debug rocksdb = 10 2024-05-20T18:37:29.087 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd backfillfull_ratio = 0.85 2024-05-20T18:37:29.087 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd full ratio = 0.9 2024-05-20T18:37:29.087 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd nearfull ratio = 0.8 2024-05-20T18:37:29.087 INFO:teuthology.orchestra.run.smithi005.stdout: osd failsafe full ratio = 0.95 2024-05-20T18:37:29.087 INFO:teuthology.orchestra.run.smithi005.stdout: osd objectstore = bluestore 2024-05-20T18:37:29.087 INFO:teuthology.orchestra.run.smithi005.stdout: osd op complaint time = 180 2024-05-20T18:37:29.087 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:29.087 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr] 2024-05-20T18:37:29.087 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-20T18:37:29.087 INFO:teuthology.orchestra.run.smithi005.stdout: debug mgr = 20 2024-05-20T18:37:29.087 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2024-05-20T18:37:29.087 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2024-05-20T18:37:29.088 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2024-05-20T18:37:29.088 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2024-05-20T18:37:29.088 INFO:teuthology.orchestra.run.smithi005.stdout: mgr/telemetry/nag = false 2024-05-20T18:37:29.088 INFO:teuthology.orchestra.run.smithi005.stdout: debug client = 20 2024-05-20T18:37:29.088 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:29.088 INFO:teuthology.orchestra.run.smithi005.stdout:[mon] 2024-05-20T18:37:29.088 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-20T18:37:29.088 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2024-05-20T18:37:29.088 INFO:teuthology.orchestra.run.smithi005.stdout: debug paxos = 20 2024-05-20T18:37:29.088 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2024-05-20T18:37:29.088 INFO:teuthology.orchestra.run.smithi005.stdout: mon data avail warn = 5 2024-05-20T18:37:29.088 INFO:teuthology.orchestra.run.smithi005.stdout: mon mgr mkfs grace = 240 2024-05-20T18:37:29.088 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2024-05-20T18:37:29.089 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd reporter subtree level = osd 2024-05-20T18:37:29.089 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd prime pg temp = true 2024-05-20T18:37:29.089 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2024-05-20T18:37:29.089 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:29.089 INFO:teuthology.orchestra.run.smithi005.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-20T18:37:29.089 INFO:teuthology.orchestra.run.smithi005.stdout: auth mon ticket ttl = 660 # 11m 2024-05-20T18:37:29.089 INFO:teuthology.orchestra.run.smithi005.stdout: auth service ticket ttl = 240 # 4m 2024-05-20T18:37:29.089 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:29.089 INFO:teuthology.orchestra.run.smithi005.stdout: # don't complain about insecure global_id in the test suite 2024-05-20T18:37:29.089 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-20T18:37:29.089 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-20T18:37:29.089 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:29.090 INFO:teuthology.orchestra.run.smithi005.stdout: # 1m isn't quite enough 2024-05-20T18:37:29.090 INFO:teuthology.orchestra.run.smithi005.stdout: mon_down_mkfs_grace = 2m 2024-05-20T18:37:29.090 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:29.090 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_filestore_osds = false 2024-05-20T18:37:29.090 INFO:teuthology.orchestra.run.smithi005.stdout: mon down mkfs grace = 300 2024-05-20T18:37:29.090 INFO:teuthology.orchestra.run.smithi005.stdout: mon op complaint time = 120 2024-05-20T18:37:29.090 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:29.090 INFO:teuthology.orchestra.run.smithi005.stdout:[client] 2024-05-20T18:37:29.090 INFO:teuthology.orchestra.run.smithi005.stdout: rgw cache enabled = true 2024-05-20T18:37:29.090 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable ops log = true 2024-05-20T18:37:29.090 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable usage log = true 2024-05-20T18:37:29.091 INFO:teuthology.orchestra.run.smithi005.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-20T18:37:29.091 INFO:teuthology.orchestra.run.smithi005.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-20T18:37:29.091 INFO:teuthology.orchestra.run.smithi005.stdout: client mount timeout = 600 2024-05-20T18:37:29.091 INFO:teuthology.orchestra.run.smithi005.stdout: debug client = 20 2024-05-20T18:37:29.091 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-20T18:37:29.091 INFO:teuthology.orchestra.run.smithi005.stdout: pid file = "" 2024-05-20T18:37:29.091 INFO:teuthology.orchestra.run.smithi005.stdout: rados mon op timeout = 900 2024-05-20T18:37:29.091 INFO:teuthology.orchestra.run.smithi005.stdout: rados osd op timeout = 900 2024-05-20T18:37:29.091 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.a] 2024-05-20T18:37:29.091 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.b] 2024-05-20T18:37:29.091 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.c] 2024-05-20T18:37:29.091 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a] 2024-05-20T18:37:29.091 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.c] 2024-05-20T18:37:29.091 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b] 2024-05-20T18:37:29.092 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.d] 2024-05-20T18:37:29.092 INFO:teuthology.orchestra.run.smithi005.stdout:[mds] 2024-05-20T18:37:29.092 INFO:teuthology.orchestra.run.smithi005.stdout: debug mds = 20 2024-05-20T18:37:29.092 INFO:teuthology.orchestra.run.smithi005.stdout: debug mds balancer = 20 2024-05-20T18:37:29.092 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-20T18:37:29.092 INFO:teuthology.orchestra.run.smithi005.stdout: mds debug frag = True 2024-05-20T18:37:29.092 INFO:teuthology.orchestra.run.smithi005.stdout: mds debug scatterstat = False 2024-05-20T18:37:29.092 INFO:teuthology.orchestra.run.smithi005.stdout: mds op complaint time = 180 2024-05-20T18:37:29.092 INFO:teuthology.orchestra.run.smithi005.stdout: mds verify scatter = False 2024-05-20T18:37:29.092 INFO:teuthology.orchestra.run.smithi005.stdout: osd op complaint time = 180 2024-05-20T18:37:29.092 INFO:teuthology.orchestra.run.smithi005.stdout: rados mon op timeout = 900 2024-05-20T18:37:29.092 INFO:teuthology.orchestra.run.smithi005.stdout: rados osd op timeout = 900 2024-05-20T18:37:29.093 INFO:teuthology.orchestra.run.smithi063.stdout:[global] 2024-05-20T18:37:29.093 INFO:teuthology.orchestra.run.smithi063.stdout: chdir = "" 2024-05-20T18:37:29.093 INFO:teuthology.orchestra.run.smithi063.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-20T18:37:29.093 INFO:teuthology.orchestra.run.smithi063.stdout: auth supported = cephx 2024-05-20T18:37:29.093 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:29.093 INFO:teuthology.orchestra.run.smithi063.stdout: filestore xattr use omap = true 2024-05-20T18:37:29.093 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:29.093 INFO:teuthology.orchestra.run.smithi063.stdout: mon clock drift allowed = 1.000 2024-05-20T18:37:29.093 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:29.093 INFO:teuthology.orchestra.run.smithi063.stdout: osd crush chooseleaf type = 0 2024-05-20T18:37:29.094 INFO:teuthology.orchestra.run.smithi063.stdout: auth debug = true 2024-05-20T18:37:29.094 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:29.094 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on old message = true 2024-05-20T18:37:29.094 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on bug = true 2024-05-20T18:37:29.094 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:29.094 INFO:teuthology.orchestra.run.smithi063.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-20T18:37:29.094 INFO:teuthology.orchestra.run.smithi063.stdout: mon pg warn max object skew = 0 2024-05-20T18:37:29.094 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:29.094 INFO:teuthology.orchestra.run.smithi063.stdout: # disable pg_autoscaler by default for new pools 2024-05-20T18:37:29.094 INFO:teuthology.orchestra.run.smithi063.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-20T18:37:29.094 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:29.094 INFO:teuthology.orchestra.run.smithi063.stdout: osd pool default size = 2 2024-05-20T18:37:29.094 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:29.095 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd allow primary affinity = true 2024-05-20T18:37:29.095 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd allow pg remap = true 2024-05-20T18:37:29.095 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on legacy crush tunables = false 2024-05-20T18:37:29.095 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on crush straw calc version zero = false 2024-05-20T18:37:29.095 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on no sortbitwise = false 2024-05-20T18:37:29.095 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on osd down out interval zero = false 2024-05-20T18:37:29.095 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on too few osds = false 2024-05-20T18:37:29.095 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-20T18:37:29.095 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-20T18:37:29.095 INFO:teuthology.orchestra.run.smithi063.stdout: mon_allow_pool_size_one = true 2024-05-20T18:37:29.095 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:29.096 INFO:teuthology.orchestra.run.smithi063.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-20T18:37:29.096 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:29.096 INFO:teuthology.orchestra.run.smithi063.stdout: osd default data pool replay window = 5 2024-05-20T18:37:29.096 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:29.096 INFO:teuthology.orchestra.run.smithi063.stdout: mon allow pool delete = true 2024-05-20T18:37:29.096 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:29.096 INFO:teuthology.orchestra.run.smithi063.stdout: mon cluster log file level = debug 2024-05-20T18:37:29.096 INFO:teuthology.orchestra.run.smithi063.stdout: debug asserts on shutdown = true 2024-05-20T18:37:29.096 INFO:teuthology.orchestra.run.smithi063.stdout: mon health detail to clog = false 2024-05-20T18:37:29.096 INFO:teuthology.orchestra.run.smithi063.stdout: mon host = "172.21.15.5,172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790]" 2024-05-20T18:37:29.096 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on skipped message = False 2024-05-20T18:37:29.096 INFO:teuthology.orchestra.run.smithi063.stdout: fsid = 399ab250-fd3c-47fa-a1c8-25e8149c9cac 2024-05-20T18:37:29.096 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:29.096 INFO:teuthology.orchestra.run.smithi063.stdout:[osd] 2024-05-20T18:37:29.097 INFO:teuthology.orchestra.run.smithi063.stdout: osd journal size = 100 2024-05-20T18:37:29.097 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:29.097 INFO:teuthology.orchestra.run.smithi063.stdout: osd scrub load threshold = 5.0 2024-05-20T18:37:29.097 INFO:teuthology.orchestra.run.smithi063.stdout: osd scrub max interval = 600 2024-05-20T18:37:29.097 INFO:teuthology.orchestra.run.smithi063.stdout: osd mclock profile = high_recovery_ops 2024-05-20T18:37:29.097 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:29.097 INFO:teuthology.orchestra.run.smithi063.stdout: osd recover clone overlap = true 2024-05-20T18:37:29.097 INFO:teuthology.orchestra.run.smithi063.stdout: osd recovery max chunk = 1048576 2024-05-20T18:37:29.097 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:29.097 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug shutdown = true 2024-05-20T18:37:29.097 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug op order = true 2024-05-20T18:37:29.097 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug verify stray on activate = true 2024-05-20T18:37:29.097 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug trim objects = true 2024-05-20T18:37:29.097 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:29.098 INFO:teuthology.orchestra.run.smithi063.stdout: osd open classes on start = true 2024-05-20T18:37:29.098 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug pg log writeout = true 2024-05-20T18:37:29.098 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:29.098 INFO:teuthology.orchestra.run.smithi063.stdout: osd deep scrub update digest min age = 30 2024-05-20T18:37:29.098 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:29.098 INFO:teuthology.orchestra.run.smithi063.stdout: osd map max advance = 10 2024-05-20T18:37:29.098 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:29.098 INFO:teuthology.orchestra.run.smithi063.stdout: journal zero on create = true 2024-05-20T18:37:29.098 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:29.098 INFO:teuthology.orchestra.run.smithi063.stdout: filestore ondisk finisher threads = 3 2024-05-20T18:37:29.098 INFO:teuthology.orchestra.run.smithi063.stdout: filestore apply finisher threads = 3 2024-05-20T18:37:29.098 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:29.098 INFO:teuthology.orchestra.run.smithi063.stdout: bdev debug aio = true 2024-05-20T18:37:29.099 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug misdirected ops = true 2024-05-20T18:37:29.099 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore block size = 96636764160 2024-05-20T18:37:29.099 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore fsck on mount = True 2024-05-20T18:37:29.099 INFO:teuthology.orchestra.run.smithi063.stdout: debug bluefs = 20 2024-05-20T18:37:29.099 INFO:teuthology.orchestra.run.smithi063.stdout: debug bluestore = 20 2024-05-20T18:37:29.099 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-20T18:37:29.099 INFO:teuthology.orchestra.run.smithi063.stdout: debug osd = 20 2024-05-20T18:37:29.099 INFO:teuthology.orchestra.run.smithi063.stdout: debug rocksdb = 10 2024-05-20T18:37:29.099 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd backfillfull_ratio = 0.85 2024-05-20T18:37:29.099 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd full ratio = 0.9 2024-05-20T18:37:29.099 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd nearfull ratio = 0.8 2024-05-20T18:37:29.099 INFO:teuthology.orchestra.run.smithi063.stdout: osd failsafe full ratio = 0.95 2024-05-20T18:37:29.099 INFO:teuthology.orchestra.run.smithi063.stdout: osd objectstore = bluestore 2024-05-20T18:37:29.100 INFO:teuthology.orchestra.run.smithi063.stdout: osd op complaint time = 180 2024-05-20T18:37:29.100 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:29.100 INFO:teuthology.orchestra.run.smithi063.stdout:[mgr] 2024-05-20T18:37:29.100 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-20T18:37:29.100 INFO:teuthology.orchestra.run.smithi063.stdout: debug mgr = 20 2024-05-20T18:37:29.100 INFO:teuthology.orchestra.run.smithi063.stdout: debug mon = 20 2024-05-20T18:37:29.100 INFO:teuthology.orchestra.run.smithi063.stdout: debug auth = 20 2024-05-20T18:37:29.100 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min pgs per osd = 4 2024-05-20T18:37:29.100 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min bytes per osd = 10 2024-05-20T18:37:29.100 INFO:teuthology.orchestra.run.smithi063.stdout: mgr/telemetry/nag = false 2024-05-20T18:37:29.100 INFO:teuthology.orchestra.run.smithi063.stdout: debug client = 20 2024-05-20T18:37:29.100 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:29.100 INFO:teuthology.orchestra.run.smithi063.stdout:[mon] 2024-05-20T18:37:29.101 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-20T18:37:29.101 INFO:teuthology.orchestra.run.smithi063.stdout: debug mon = 20 2024-05-20T18:37:29.101 INFO:teuthology.orchestra.run.smithi063.stdout: debug paxos = 20 2024-05-20T18:37:29.101 INFO:teuthology.orchestra.run.smithi063.stdout: debug auth = 20 2024-05-20T18:37:29.101 INFO:teuthology.orchestra.run.smithi063.stdout: mon data avail warn = 5 2024-05-20T18:37:29.101 INFO:teuthology.orchestra.run.smithi063.stdout: mon mgr mkfs grace = 240 2024-05-20T18:37:29.101 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min pgs per osd = 4 2024-05-20T18:37:29.101 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd reporter subtree level = osd 2024-05-20T18:37:29.101 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd prime pg temp = true 2024-05-20T18:37:29.101 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min bytes per osd = 10 2024-05-20T18:37:29.101 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:29.101 INFO:teuthology.orchestra.run.smithi063.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-20T18:37:29.101 INFO:teuthology.orchestra.run.smithi063.stdout: auth mon ticket ttl = 660 # 11m 2024-05-20T18:37:29.101 INFO:teuthology.orchestra.run.smithi063.stdout: auth service ticket ttl = 240 # 4m 2024-05-20T18:37:29.102 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:29.102 INFO:teuthology.orchestra.run.smithi063.stdout: # don't complain about insecure global_id in the test suite 2024-05-20T18:37:29.102 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-20T18:37:29.102 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-20T18:37:29.102 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:29.102 INFO:teuthology.orchestra.run.smithi063.stdout: # 1m isn't quite enough 2024-05-20T18:37:29.102 INFO:teuthology.orchestra.run.smithi063.stdout: mon_down_mkfs_grace = 2m 2024-05-20T18:37:29.102 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:29.102 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_filestore_osds = false 2024-05-20T18:37:29.102 INFO:teuthology.orchestra.run.smithi063.stdout: mon down mkfs grace = 300 2024-05-20T18:37:29.102 INFO:teuthology.orchestra.run.smithi063.stdout: mon op complaint time = 120 2024-05-20T18:37:29.102 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:29.103 INFO:teuthology.orchestra.run.smithi063.stdout:[client] 2024-05-20T18:37:29.103 INFO:teuthology.orchestra.run.smithi063.stdout: rgw cache enabled = true 2024-05-20T18:37:29.103 INFO:teuthology.orchestra.run.smithi063.stdout: rgw enable ops log = true 2024-05-20T18:37:29.103 INFO:teuthology.orchestra.run.smithi063.stdout: rgw enable usage log = true 2024-05-20T18:37:29.103 INFO:teuthology.orchestra.run.smithi063.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-20T18:37:29.103 INFO:teuthology.orchestra.run.smithi063.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-20T18:37:29.103 INFO:teuthology.orchestra.run.smithi063.stdout: client mount timeout = 600 2024-05-20T18:37:29.103 INFO:teuthology.orchestra.run.smithi063.stdout: debug client = 20 2024-05-20T18:37:29.104 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-20T18:37:29.104 INFO:teuthology.orchestra.run.smithi063.stdout: pid file = "" 2024-05-20T18:37:29.104 INFO:teuthology.orchestra.run.smithi063.stdout: rados mon op timeout = 900 2024-05-20T18:37:29.104 INFO:teuthology.orchestra.run.smithi063.stdout: rados osd op timeout = 900 2024-05-20T18:37:29.104 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.a] 2024-05-20T18:37:29.104 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.b] 2024-05-20T18:37:29.104 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.c] 2024-05-20T18:37:29.104 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.a] 2024-05-20T18:37:29.104 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.c] 2024-05-20T18:37:29.104 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.b] 2024-05-20T18:37:29.104 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.d] 2024-05-20T18:37:29.104 INFO:teuthology.orchestra.run.smithi063.stdout:[mds] 2024-05-20T18:37:29.104 INFO:teuthology.orchestra.run.smithi063.stdout: debug mds = 20 2024-05-20T18:37:29.105 INFO:teuthology.orchestra.run.smithi063.stdout: debug mds balancer = 20 2024-05-20T18:37:29.105 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-20T18:37:29.105 INFO:teuthology.orchestra.run.smithi063.stdout: mds debug frag = True 2024-05-20T18:37:29.105 INFO:teuthology.orchestra.run.smithi063.stdout: mds debug scatterstat = False 2024-05-20T18:37:29.105 INFO:teuthology.orchestra.run.smithi063.stdout: mds op complaint time = 180 2024-05-20T18:37:29.105 INFO:teuthology.orchestra.run.smithi063.stdout: mds verify scatter = False 2024-05-20T18:37:29.105 INFO:teuthology.orchestra.run.smithi063.stdout: osd op complaint time = 180 2024-05-20T18:37:29.105 INFO:teuthology.orchestra.run.smithi063.stdout: rados mon op timeout = 900 2024-05-20T18:37:29.105 INFO:teuthology.orchestra.run.smithi063.stdout: rados osd op timeout = 900 2024-05-20T18:37:29.107 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2024-05-20T18:37:29.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.162+0000 7fb928a2af40 1 Processor -- start 2024-05-20T18:37:29.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.162+0000 7fb928a2af40 1 -- start start 2024-05-20T18:37:29.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.164+0000 7fb928a2af40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570bcecb170 0x5570bceaffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:29.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.164+0000 7fb928a2af40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570bcecb580 0x5570bceb04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:29.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.164+0000 7fb928a2af40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5570bcf4f150 0x5570bcf52d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:29.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.164+0000 7fb928a2af40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5570bceccb60 con 0x5570bcecb170 2024-05-20T18:37:29.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.164+0000 7fb928a2af40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5570bce2bc80 con 0x5570bcecb580 2024-05-20T18:37:29.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.164+0000 7fb928a2af40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5570bcecce00 con 0x5570bcf4f150 2024-05-20T18:37:29.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.164+0000 7fb9224f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570bcecb170 0x5570bceaffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:29.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.164+0000 7fb9224f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570bcecb170 0x5570bceaffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34036/0 (socket says 172.21.15.5:34036) 2024-05-20T18:37:29.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.164+0000 7fb9224f9640 1 -- 172.21.15.5:0/1189603353 learned_addr learned my addr 172.21.15.5:0/1189603353 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:29.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.164+0000 7fb921cf8640 1 --2- 172.21.15.5:0/1189603353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570bcecb580 0x5570bceb04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:29.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.164+0000 7fb922cfa640 1 --2- 172.21.15.5:0/1189603353 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5570bcf4f150 0x5570bcf52d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:29.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.164+0000 7fb9224f9640 1 -- 172.21.15.5:0/1189603353 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5570bcf4f150 msgr2=0x5570bcf52d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:29.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.164+0000 7fb9224f9640 1 --2- 172.21.15.5:0/1189603353 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5570bcf4f150 0x5570bcf52d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:29.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.164+0000 7fb9224f9640 1 -- 172.21.15.5:0/1189603353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570bcecb580 msgr2=0x5570bceb04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:29.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.164+0000 7fb9224f9640 1 --2- 172.21.15.5:0/1189603353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570bcecb580 0x5570bceb04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:29.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.164+0000 7fb9224f9640 1 -- 172.21.15.5:0/1189603353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570bcdb0cb0 con 0x5570bcecb170 2024-05-20T18:37:29.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.164+0000 7fb9224f9640 1 --2- 172.21.15.5:0/1189603353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570bcecb170 0x5570bceaffb0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fb914001680 tx=0x7fb914001b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6277c4f4c069b96c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:29.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.165+0000 7fb920cf6640 1 -- 172.21.15.5:0/1189603353 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914003650 con 0x5570bcecb170 2024-05-20T18:37:29.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.165+0000 7fb920cf6640 1 -- 172.21.15.5:0/1189603353 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9140037f0 con 0x5570bcecb170 2024-05-20T18:37:29.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.165+0000 7fb920cf6640 1 -- 172.21.15.5:0/1189603353 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914003be0 con 0x5570bcecb170 2024-05-20T18:37:29.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.165+0000 7fb928a2af40 1 -- 172.21.15.5:0/1189603353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570bcecb170 msgr2=0x5570bceaffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:29.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.165+0000 7fb928a2af40 1 --2- 172.21.15.5:0/1189603353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570bcecb170 0x5570bceaffb0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fb914001680 tx=0x7fb914001b40 comp rx=0 tx=0).stop 2024-05-20T18:37:29.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.165+0000 7fb928a2af40 1 -- 172.21.15.5:0/1189603353 shutdown_connections 2024-05-20T18:37:29.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.165+0000 7fb928a2af40 1 --2- 172.21.15.5:0/1189603353 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5570bcf4f150 0x5570bcf52d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:29.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.165+0000 7fb928a2af40 1 --2- 172.21.15.5:0/1189603353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570bcecb580 0x5570bceb04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:29.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.165+0000 7fb928a2af40 1 --2- 172.21.15.5:0/1189603353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570bcecb170 0x5570bceaffb0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:29.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.165+0000 7fb928a2af40 1 -- 172.21.15.5:0/1189603353 >> 172.21.15.5:0/1189603353 conn(0x5570bcda9ac0 msgr2=0x5570bcf56d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:29.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.165+0000 7fb928a2af40 1 -- 172.21.15.5:0/1189603353 shutdown_connections 2024-05-20T18:37:29.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.165+0000 7fb928a2af40 1 -- 172.21.15.5:0/1189603353 wait complete. 2024-05-20T18:37:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.166+0000 7fb928a2af40 1 Processor -- start 2024-05-20T18:37:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.166+0000 7fb928a2af40 1 -- start start 2024-05-20T18:37:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.167+0000 7fb928a2af40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570bcecb170 0x5570bcf4bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.167+0000 7fb928a2af40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570bcecb580 0x5570bcf51300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.167+0000 7fb928a2af40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5570bcf4f150 0x5570bcf51840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.167+0000 7fb928a2af40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5570bcf55af0 con 0x5570bcecb580 2024-05-20T18:37:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.167+0000 7fb928a2af40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5570bceaede0 con 0x5570bcecb170 2024-05-20T18:37:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.167+0000 7fb928a2af40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5570bceaf0c0 con 0x5570bcf4f150 2024-05-20T18:37:29.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.168+0000 7fb921cf8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570bcecb580 0x5570bcf51300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:29.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.168+0000 7fb921cf8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570bcecb580 0x5570bcf51300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34042/0 (socket says 172.21.15.5:34042) 2024-05-20T18:37:29.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.168+0000 7fb921cf8640 1 -- 172.21.15.5:0/3304160419 learned_addr learned my addr 172.21.15.5:0/3304160419 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:29.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.168+0000 7fb9224f9640 1 --2- 172.21.15.5:0/3304160419 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570bcecb170 0x5570bcf4bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:29.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.168+0000 7fb922cfa640 1 --2- 172.21.15.5:0/3304160419 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5570bcf4f150 0x5570bcf51840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:29.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.168+0000 7fb921cf8640 1 -- 172.21.15.5:0/3304160419 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5570bcf4f150 msgr2=0x5570bcf51840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:29.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.168+0000 7fb921cf8640 1 --2- 172.21.15.5:0/3304160419 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5570bcf4f150 0x5570bcf51840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:29.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.168+0000 7fb921cf8640 1 -- 172.21.15.5:0/3304160419 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570bcecb170 msgr2=0x5570bcf4bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:29.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.168+0000 7fb921cf8640 1 --2- 172.21.15.5:0/3304160419 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5570bcecb170 0x5570bcf4bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:29.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.168+0000 7fb921cf8640 1 -- 172.21.15.5:0/3304160419 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570bce839d0 con 0x5570bcecb580 2024-05-20T18:37:29.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.168+0000 7fb921cf8640 1 --2- 172.21.15.5:0/3304160419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570bcecb580 0x5570bcf51300 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fb90c000c00 tx=0x7fb90c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:29.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.168+0000 7fb91b7fe640 1 -- 172.21.15.5:0/3304160419 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c06d070 con 0x5570bcecb580 2024-05-20T18:37:29.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.168+0000 7fb91b7fe640 1 -- 172.21.15.5:0/3304160419 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb90c05e930 con 0x5570bcecb580 2024-05-20T18:37:29.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.168+0000 7fb928a2af40 1 -- 172.21.15.5:0/3304160419 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5570bcf4c110 con 0x5570bcecb580 2024-05-20T18:37:29.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.168+0000 7fb91b7fe640 1 -- 172.21.15.5:0/3304160419 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c05ed00 con 0x5570bcecb580 2024-05-20T18:37:29.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.168+0000 7fb928a2af40 1 -- 172.21.15.5:0/3304160419 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570bcff4500 con 0x5570bcecb580 2024-05-20T18:37:29.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.170+0000 7fb91b7fe640 1 -- 172.21.15.5:0/3304160419 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb90c070070 con 0x5570bcecb580 2024-05-20T18:37:29.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.170+0000 7fb91b7fe640 1 --2- 172.21.15.5:0/3304160419 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb90406a810 0x7fb90406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:29.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.170+0000 7fb91b7fe640 1 -- 172.21.15.5:0/3304160419 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7fb90c0cdfc0 con 0x5570bcecb580 2024-05-20T18:37:29.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.170+0000 7fb9224f9640 1 --2- 172.21.15.5:0/3304160419 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb90406a810 0x7fb90406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:29.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.170+0000 7fb928a2af40 1 --2- 172.21.15.5:0/3304160419 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x5570bcf53540 0x5570bcf4dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:29.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.170+0000 7fb928a2af40 1 -- 172.21.15.5:0/3304160419 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:1 17.7 17:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5570bcf4e200 con 0x5570bcf53540 2024-05-20T18:37:29.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.171+0000 7fb922cfa640 1 --2- 172.21.15.5:0/3304160419 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x5570bcf53540 0x5570bcf4dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:29.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.171+0000 7fb9224f9640 1 --2- 172.21.15.5:0/3304160419 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb90406a810 0x7fb90406cd20 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb914001680 tx=0x7fb914004000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:29.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.171+0000 7fb922cfa640 1 --2- 172.21.15.5:0/3304160419 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x5570bcf53540 0x5570bcf4dcc0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:29.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.172+0000 7fb922cfa640 1 -- 172.21.15.5:0/3304160419 <== osd.0 v2:172.21.15.5:6808/688093005 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v120'5 uv5 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb91c009830 con 0x5570bcf53540 2024-05-20T18:37:29.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.172+0000 7fb928a2af40 1 -- 172.21.15.5:0/3304160419 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x5570bcf53540 msgr2=0x5570bcf4dcc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:29.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.172+0000 7fb928a2af40 1 --2- 172.21.15.5:0/3304160419 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x5570bcf53540 0x5570bcf4dcc0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:29.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.172+0000 7fb928a2af40 1 -- 172.21.15.5:0/3304160419 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb90406a810 msgr2=0x7fb90406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:29.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.172+0000 7fb928a2af40 1 --2- 172.21.15.5:0/3304160419 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb90406a810 0x7fb90406cd20 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb914001680 tx=0x7fb914004000 comp rx=0 tx=0).stop 2024-05-20T18:37:29.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.172+0000 7fb928a2af40 1 -- 172.21.15.5:0/3304160419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570bcecb580 msgr2=0x5570bcf51300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:29.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.172+0000 7fb928a2af40 1 --2- 172.21.15.5:0/3304160419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5570bcecb580 0x5570bcf51300 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fb90c000c00 tx=0x7fb90c002fb0 comp rx=0 tx=0).stop 2024-05-20T18:37:29.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.172+0000 7fb922cfa640 1 -- 172.21.15.5:0/3304160419 reap_dead start 2024-05-20T18:37:29.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.172+0000 7fb928a2af40 1 -- 172.21.15.5:0/3304160419 shutdown_connections 2024-05-20T18:37:29.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.172+0000 7fb928a2af40 1 -- 172.21.15.5:0/3304160419 >> 172.21.15.5:0/3304160419 conn(0x5570bcda9ac0 msgr2=0x5570bcecbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:29.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.173+0000 7fb928a2af40 1 -- 172.21.15.5:0/3304160419 shutdown_connections 2024-05-20T18:37:29.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.173+0000 7fb928a2af40 1 -- 172.21.15.5:0/3304160419 wait complete. 2024-05-20T18:37:29.176 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-20T18:37:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.293+0000 7f58c03b5640 1 Processor -- start 2024-05-20T18:37:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.293+0000 7f58c03b5640 1 -- start start 2024-05-20T18:37:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.295+0000 7f58c03b5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f58b80d4400 0x7f58b815f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.295+0000 7f58c03b5640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f58b80d2780 0x7f58b80d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.295+0000 7f58c03b5640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58b808d2a0 con 0x7f58b806cf00 2024-05-20T18:37:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.295+0000 7f58c03b5640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f58b80d3f70 con 0x7f58b80d4400 2024-05-20T18:37:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.295+0000 7f58c03b5640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f58b80d30d0 con 0x7f58b80d2780 2024-05-20T18:37:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.295+0000 7f58be151640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f58b80d4400 0x7f58b815f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.295+0000 7f58bd950640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f58b806cf00 0x7f58b806d310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38644/0 (socket says 172.21.15.5:38644) 2024-05-20T18:37:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.295+0000 7f58be151640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f58b80d4400 0x7f58b815f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34052/0 (socket says 172.21.15.5:34052) 2024-05-20T18:37:29.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.295+0000 7f58bd950640 1 -- 172.21.15.5:0/3525835464 learned_addr learned my addr 172.21.15.5:0/3525835464 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:29.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.295+0000 7f58be952640 1 --2- 172.21.15.5:0/3525835464 >> v2:172.21.15.63:3300/0 conn(0x7f58b80d2780 0x7f58b80d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:29.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.295+0000 7f58be151640 1 -- 172.21.15.5:0/3525835464 >> v2:172.21.15.63:3300/0 conn(0x7f58b80d2780 msgr2=0x7f58b80d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:29.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.295+0000 7f58be151640 1 --2- 172.21.15.5:0/3525835464 >> v2:172.21.15.63:3300/0 conn(0x7f58b80d2780 0x7f58b80d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:29.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.295+0000 7f58be151640 1 -- 172.21.15.5:0/3525835464 >> v1:172.21.15.5:6789/0 conn(0x7f58b806cf00 legacy=0x7f58b806d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:29.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.296+0000 7f58bd14f640 1 -- 172.21.15.5:0/3525835464 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2020070949 0 0) 0x7f58ac001110 con 0x7f58b806cf00 2024-05-20T18:37:29.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.296+0000 7f58be151640 1 -- 172.21.15.5:0/3525835464 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b80d4810 con 0x7f58b80d4400 2024-05-20T18:37:29.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.296+0000 7f58be952640 1 --2- 172.21.15.5:0/3525835464 >> v2:172.21.15.63:3300/0 conn(0x7f58b80d2780 0x7f58b80d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.296+0000 7f58be151640 1 --2- 172.21.15.5:0/3525835464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58b80d4400 0x7f58b815f440 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f58b808e020 tx=0x7f58a802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1527560fbe6967fa server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.296+0000 7f58bd14f640 1 -- 172.21.15.5:0/3525835464 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a8007740 con 0x7f58b80d4400 2024-05-20T18:37:29.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.296+0000 7f58bd14f640 1 -- 172.21.15.5:0/3525835464 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58a80078e0 con 0x7f58b80d4400 2024-05-20T18:37:29.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.296+0000 7f58bd14f640 1 -- 172.21.15.5:0/3525835464 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a8007cb0 con 0x7f58b80d4400 2024-05-20T18:37:29.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.297+0000 7f58c03b5640 1 -- 172.21.15.5:0/3525835464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58b80d4400 msgr2=0x7f58b815f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:29.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.297+0000 7f58c03b5640 1 --2- 172.21.15.5:0/3525835464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58b80d4400 0x7f58b815f440 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f58b808e020 tx=0x7f58a802a710 comp rx=0 tx=0).stop 2024-05-20T18:37:29.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.297+0000 7f58c03b5640 1 -- 172.21.15.5:0/3525835464 shutdown_connections 2024-05-20T18:37:29.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.297+0000 7f58c03b5640 1 --2- 172.21.15.5:0/3525835464 >> v2:172.21.15.63:3300/0 conn(0x7f58b80d2780 0x7f58b80d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:29.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.297+0000 7f58c03b5640 1 --2- 172.21.15.5:0/3525835464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58b80d4400 0x7f58b815f440 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:29.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.297+0000 7f58c03b5640 1 -- 172.21.15.5:0/3525835464 >> 172.21.15.5:0/3525835464 conn(0x7f58b80d4bb0 msgr2=0x7f58b80d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:29.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.297+0000 7f58c03b5640 1 -- 172.21.15.5:0/3525835464 shutdown_connections 2024-05-20T18:37:29.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.297+0000 7f58c03b5640 1 -- 172.21.15.5:0/3525835464 wait complete. 2024-05-20T18:37:29.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.299+0000 7f58c03b5640 1 Processor -- start 2024-05-20T18:37:29.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.299+0000 7f58c03b5640 1 -- start start 2024-05-20T18:37:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.300+0000 7f58c03b5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58b806cf00 0x7f58b815be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.300+0000 7f58c03b5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58b80d2780 0x7f58b80cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.300+0000 7f58c03b5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58b80d4400 0x7f58b8161510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.300+0000 7f58c03b5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f58b80d3920 con 0x7f58b806cf00 2024-05-20T18:37:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.300+0000 7f58c03b5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f58b80d3640 con 0x7f58b80d4400 2024-05-20T18:37:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.300+0000 7f58c03b5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f58b80d3c00 con 0x7f58b80d2780 2024-05-20T18:37:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.300+0000 7f58be151640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58b806cf00 0x7f58b815be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.300+0000 7f58be151640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58b806cf00 0x7f58b815be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34066/0 (socket says 172.21.15.5:34066) 2024-05-20T18:37:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.300+0000 7f58be151640 1 -- 172.21.15.5:0/109792575 learned_addr learned my addr 172.21.15.5:0/109792575 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.301+0000 7f58be952640 1 --2- 172.21.15.5:0/109792575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58b80d4400 0x7f58b8161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.301+0000 7f58bd950640 1 --2- 172.21.15.5:0/109792575 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58b80d2780 0x7f58b80cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.301+0000 7f58be151640 1 -- 172.21.15.5:0/109792575 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58b80d2780 msgr2=0x7f58b80cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.301+0000 7f58be151640 1 --2- 172.21.15.5:0/109792575 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58b80d2780 0x7f58b80cd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.301+0000 7f58be151640 1 -- 172.21.15.5:0/109792575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58b80d4400 msgr2=0x7f58b8161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.301+0000 7f58be151640 1 --2- 172.21.15.5:0/109792575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58b80d4400 0x7f58b8161510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.301+0000 7f58be151640 1 -- 172.21.15.5:0/109792575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b81552a0 con 0x7f58b806cf00 2024-05-20T18:37:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.301+0000 7f58be952640 1 --2- 172.21.15.5:0/109792575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58b80d4400 0x7f58b8161510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.301+0000 7f58be151640 1 --2- 172.21.15.5:0/109792575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58b806cf00 0x7f58b815be10 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f58a8009e80 tx=0x7f58a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.301+0000 7f58b6ffd640 1 -- 172.21.15.5:0/109792575 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a8040030 con 0x7f58b806cf00 2024-05-20T18:37:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.301+0000 7f58b6ffd640 1 -- 172.21.15.5:0/109792575 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58a8031a40 con 0x7f58b806cf00 2024-05-20T18:37:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.301+0000 7f58b6ffd640 1 -- 172.21.15.5:0/109792575 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a8031e30 con 0x7f58b806cf00 2024-05-20T18:37:29.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.301+0000 7f58c03b5640 1 -- 172.21.15.5:0/109792575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b815ea00 con 0x7f58b806cf00 2024-05-20T18:37:29.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.301+0000 7f58c03b5640 1 -- 172.21.15.5:0/109792575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b815c350 con 0x7f58b806cf00 2024-05-20T18:37:29.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.302+0000 7f58c03b5640 1 -- 172.21.15.5:0/109792575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58b80735a0 con 0x7f58b806cf00 2024-05-20T18:37:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.303+0000 7f58b6ffd640 1 -- 172.21.15.5:0/109792575 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f58a803b410 con 0x7f58b806cf00 2024-05-20T18:37:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.303+0000 7f58b6ffd640 1 --2- 172.21.15.5:0/109792575 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f58a406be60 0x7f58a406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.304+0000 7f58b6ffd640 1 -- 172.21.15.5:0/109792575 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7f58a80ac1b0 con 0x7f58b806cf00 2024-05-20T18:37:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.304+0000 7f58bd950640 1 --2- 172.21.15.5:0/109792575 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f58a406be60 0x7f58a406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.304+0000 7f58bd950640 1 --2- 172.21.15.5:0/109792575 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f58a406be60 0x7f58a406e370 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f58b8151870 tx=0x7f58ac007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:29.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.308+0000 7f58b6ffd640 1 -- 172.21.15.5:0/109792575 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f58b80735a0 con 0x7f58b806cf00 2024-05-20T18:37:29.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.531+0000 7f58c03b5640 1 -- 172.21.15.5:0/109792575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f58b8075b00 con 0x7f58b806cf00 2024-05-20T18:37:29.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.905+0000 7f58b6ffd640 1 -- 172.21.15.5:0/109792575 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v100) ==== 162+0+0 (secure 0 0 0) 0x7f58a807a850 con 0x7f58b806cf00 2024-05-20T18:37:29.905 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-20T18:37:29.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.907+0000 7f58c03b5640 1 -- 172.21.15.5:0/109792575 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f58a406be60 msgr2=0x7f58a406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:29.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.907+0000 7f58c03b5640 1 --2- 172.21.15.5:0/109792575 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f58a406be60 0x7f58a406e370 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f58b8151870 tx=0x7f58ac007c00 comp rx=0 tx=0).stop 2024-05-20T18:37:29.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.907+0000 7f58c03b5640 1 -- 172.21.15.5:0/109792575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58b806cf00 msgr2=0x7f58b815be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:29.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.907+0000 7f58c03b5640 1 --2- 172.21.15.5:0/109792575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58b806cf00 0x7f58b815be10 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f58a8009e80 tx=0x7f58a8004a40 comp rx=0 tx=0).stop 2024-05-20T18:37:29.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.907+0000 7f58c03b5640 1 -- 172.21.15.5:0/109792575 shutdown_connections 2024-05-20T18:37:29.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.907+0000 7f58c03b5640 1 --2- 172.21.15.5:0/109792575 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f58a406be60 0x7f58a406e370 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:29.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.907+0000 7f58c03b5640 1 --2- 172.21.15.5:0/109792575 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58b80d4400 0x7f58b8161510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:29.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.907+0000 7f58c03b5640 1 --2- 172.21.15.5:0/109792575 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58b80d2780 0x7f58b80cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:29.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.907+0000 7f58c03b5640 1 --2- 172.21.15.5:0/109792575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58b806cf00 0x7f58b815be10 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:29.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.907+0000 7f58c03b5640 1 -- 172.21.15.5:0/109792575 >> 172.21.15.5:0/109792575 conn(0x7f58b80d4bb0 msgr2=0x7f58b815a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:29.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.907+0000 7f58c03b5640 1 -- 172.21.15.5:0/109792575 shutdown_connections 2024-05-20T18:37:29.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:29.907+0000 7f58c03b5640 1 -- 172.21.15.5:0/109792575 wait complete. 2024-05-20T18:37:29.923 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:37:30.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.040+0000 7f6a26731640 1 Processor -- start 2024-05-20T18:37:30.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.040+0000 7f6a26731640 1 -- start start 2024-05-20T18:37:30.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.042+0000 7f6a26731640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a20162820 0x7f6a200b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:30.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.042+0000 7f6a26731640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6a2006dae0 0x7f6a200b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:30.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.042+0000 7f6a26731640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a200d0630 con 0x7f6a200b3e80 2024-05-20T18:37:30.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.042+0000 7f6a26731640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f6a200d39c0 con 0x7f6a2006dae0 2024-05-20T18:37:30.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.042+0000 7f6a26731640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6a200d3ca0 con 0x7f6a20162820 2024-05-20T18:37:30.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.042+0000 7f6a24cce640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6a2006dae0 0x7f6a200b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:30.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.042+0000 7f6a1f7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f6a200b3e80 0x7f6a2006d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38650/0 (socket says 172.21.15.5:38650) 2024-05-20T18:37:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.042+0000 7f6a1f7fe640 1 -- 172.21.15.5:0/2282832839 learned_addr learned my addr 172.21.15.5:0/2282832839 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.042+0000 7f6a1ffff640 1 --2- 172.21.15.5:0/2282832839 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a20162820 0x7f6a200b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.042+0000 7f6a24cce640 1 -- 172.21.15.5:0/2282832839 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a20162820 msgr2=0x7f6a200b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.042+0000 7f6a24cce640 1 --2- 172.21.15.5:0/2282832839 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a20162820 0x7f6a200b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.042+0000 7f6a24cce640 1 -- 172.21.15.5:0/2282832839 >> v1:172.21.15.5:6789/0 conn(0x7f6a200b3e80 legacy=0x7f6a2006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.042+0000 7f6a1effd640 1 -- 172.21.15.5:0/2282832839 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2538930366 0 0) 0x7f6a200d0630 con 0x7f6a200b3e80 2024-05-20T18:37:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.042+0000 7f6a24cce640 1 -- 172.21.15.5:0/2282832839 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a200d3f80 con 0x7f6a2006dae0 2024-05-20T18:37:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.043+0000 7f6a24cce640 1 --2- 172.21.15.5:0/2282832839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a2006dae0 0x7f6a200b8190 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009d90 tx=0x7f6a1802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40edd55df8efceb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.043+0000 7f6a1effd640 1 -- 172.21.15.5:0/2282832839 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a18007740 con 0x7f6a2006dae0 2024-05-20T18:37:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.043+0000 7f6a1effd640 1 -- 172.21.15.5:0/2282832839 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a180078e0 con 0x7f6a2006dae0 2024-05-20T18:37:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.043+0000 7f6a1effd640 1 -- 172.21.15.5:0/2282832839 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a18007cb0 con 0x7f6a2006dae0 2024-05-20T18:37:30.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.044+0000 7f6a26731640 1 -- 172.21.15.5:0/2282832839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a2006dae0 msgr2=0x7f6a200b8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:30.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.044+0000 7f6a26731640 1 --2- 172.21.15.5:0/2282832839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a2006dae0 0x7f6a200b8190 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009d90 tx=0x7f6a1802a660 comp rx=0 tx=0).stop 2024-05-20T18:37:30.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.044+0000 7f6a26731640 1 -- 172.21.15.5:0/2282832839 shutdown_connections 2024-05-20T18:37:30.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.044+0000 7f6a26731640 1 --2- 172.21.15.5:0/2282832839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a2006dae0 0x7f6a200b8190 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:30.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.044+0000 7f6a26731640 1 --2- 172.21.15.5:0/2282832839 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a20162820 0x7f6a200b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:30.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.044+0000 7f6a26731640 1 -- 172.21.15.5:0/2282832839 >> 172.21.15.5:0/2282832839 conn(0x7f6a200d4bd0 msgr2=0x7f6a200d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:30.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.044+0000 7f6a26731640 1 -- 172.21.15.5:0/2282832839 shutdown_connections 2024-05-20T18:37:30.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.044+0000 7f6a26731640 1 -- 172.21.15.5:0/2282832839 wait complete. 2024-05-20T18:37:30.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.046+0000 7f6a26731640 1 Processor -- start 2024-05-20T18:37:30.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.046+0000 7f6a26731640 1 -- start start 2024-05-20T18:37:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.047+0000 7f6a26731640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a200b3e80 0x7f6a2014cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.047+0000 7f6a26731640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a20162820 0x7f6a200cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.047+0000 7f6a26731640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a201f82f0 0x7f6a201f8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.047+0000 7f6a26731640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6a200d4780 con 0x7f6a20162820 2024-05-20T18:37:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.047+0000 7f6a26731640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6a200d0300 con 0x7f6a200b3e80 2024-05-20T18:37:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.047+0000 7f6a26731640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6a200d4900 con 0x7f6a201f82f0 2024-05-20T18:37:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.047+0000 7f6a1f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a20162820 0x7f6a200cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.047+0000 7f6a1f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a20162820 0x7f6a200cb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34082/0 (socket says 172.21.15.5:34082) 2024-05-20T18:37:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.047+0000 7f6a1f7fe640 1 -- 172.21.15.5:0/4006185477 learned_addr learned my addr 172.21.15.5:0/4006185477 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:30.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.048+0000 7f6a24cce640 1 --2- 172.21.15.5:0/4006185477 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a201f82f0 0x7f6a201f8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:30.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.048+0000 7f6a1ffff640 1 --2- 172.21.15.5:0/4006185477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a200b3e80 0x7f6a2014cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:30.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.048+0000 7f6a1f7fe640 1 -- 172.21.15.5:0/4006185477 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a201f82f0 msgr2=0x7f6a201f8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:30.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.048+0000 7f6a1f7fe640 1 --2- 172.21.15.5:0/4006185477 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a201f82f0 0x7f6a201f8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:30.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.048+0000 7f6a1f7fe640 1 -- 172.21.15.5:0/4006185477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a200b3e80 msgr2=0x7f6a2014cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:30.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.048+0000 7f6a1f7fe640 1 --2- 172.21.15.5:0/4006185477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a200b3e80 0x7f6a2014cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:30.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.048+0000 7f6a1f7fe640 1 -- 172.21.15.5:0/4006185477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2014d530 con 0x7f6a20162820 2024-05-20T18:37:30.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.048+0000 7f6a1ffff640 1 --2- 172.21.15.5:0/4006185477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a200b3e80 0x7f6a2014cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:30.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.048+0000 7f6a1f7fe640 1 --2- 172.21.15.5:0/4006185477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a20162820 0x7f6a200cb0f0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400c410 tx=0x7f6a1400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:30.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.048+0000 7f6a1cff9640 1 -- 172.21.15.5:0/4006185477 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a140074e0 con 0x7f6a20162820 2024-05-20T18:37:30.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.048+0000 7f6a1cff9640 1 -- 172.21.15.5:0/4006185477 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a140047e0 con 0x7f6a20162820 2024-05-20T18:37:30.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.048+0000 7f6a26731640 1 -- 172.21.15.5:0/4006185477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a201f8c40 con 0x7f6a20162820 2024-05-20T18:37:30.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.048+0000 7f6a1cff9640 1 -- 172.21.15.5:0/4006185477 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a14004bd0 con 0x7f6a20162820 2024-05-20T18:37:30.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.048+0000 7f6a26731640 1 -- 172.21.15.5:0/4006185477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a2015dc80 con 0x7f6a20162820 2024-05-20T18:37:30.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.049+0000 7f6a26731640 1 -- 172.21.15.5:0/4006185477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a20075b00 con 0x7f6a20162820 2024-05-20T18:37:30.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.050+0000 7f6a1cff9640 1 -- 172.21.15.5:0/4006185477 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6a14011070 con 0x7f6a20162820 2024-05-20T18:37:30.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.050+0000 7f6a1cff9640 1 --2- 172.21.15.5:0/4006185477 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6a0c06be30 0x7f6a0c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:30.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.050+0000 7f6a1cff9640 1 -- 172.21.15.5:0/4006185477 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7f6a1408adb0 con 0x7f6a20162820 2024-05-20T18:37:30.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.050+0000 7f6a1ffff640 1 --2- 172.21.15.5:0/4006185477 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6a0c06be30 0x7f6a0c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:30.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.051+0000 7f6a1ffff640 1 --2- 172.21.15.5:0/4006185477 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6a0c06be30 0x7f6a0c06e340 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f6a10009a20 tx=0x7f6a10008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:30.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.055+0000 7f6a1cff9640 1 -- 172.21.15.5:0/4006185477 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6a1408f070 con 0x7f6a20162820 2024-05-20T18:37:30.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.275+0000 7f6a26731640 1 -- 172.21.15.5:0/4006185477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a2015da10 con 0x7f6a20162820 2024-05-20T18:37:30.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.276+0000 7f6a1cff9640 1 -- 172.21.15.5:0/4006185477 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) ==== 95+0+5173 (secure 0 0 0) 0x7f6a201f8c40 con 0x7f6a20162820 2024-05-20T18:37:30.276 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:30.276 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":101,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/266630184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":266630184},{"type":"v1","addr":"172.21.15.63:6834","nonce":266630184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7361,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1494022697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":1494022697},{"type":"v1","addr":"172.21.15.63:6835","nonce":1494022697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":9009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/1739221946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1739221946},{"type":"v1","addr":"172.21.15.5:6836","nonce":1739221946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:37:14.365195+0000","modified":"2024-05-20T18:37:29.907939+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9012":{"gid":9012,"name":"c","rank":0,"incarnation":101,"state":"up:replay","state_seq":1,"addr":"172.21.15.5:6837/4273429943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":4273429943},{"type":"v1","addr":"172.21.15.5:6837","nonce":4273429943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-05-20T18:37:30.276 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 101 2024-05-20T18:37:30.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.277+0000 7f6a26731640 1 -- 172.21.15.5:0/4006185477 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6a0c06be30 msgr2=0x7f6a0c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:30.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.277+0000 7f6a26731640 1 --2- 172.21.15.5:0/4006185477 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6a0c06be30 0x7f6a0c06e340 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f6a10009a20 tx=0x7f6a10008040 comp rx=0 tx=0).stop 2024-05-20T18:37:30.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.277+0000 7f6a26731640 1 -- 172.21.15.5:0/4006185477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a20162820 msgr2=0x7f6a200cb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:30.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.277+0000 7f6a26731640 1 --2- 172.21.15.5:0/4006185477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a20162820 0x7f6a200cb0f0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400c410 tx=0x7f6a1400c8d0 comp rx=0 tx=0).stop 2024-05-20T18:37:30.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.278+0000 7f6a26731640 1 -- 172.21.15.5:0/4006185477 shutdown_connections 2024-05-20T18:37:30.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.278+0000 7f6a26731640 1 --2- 172.21.15.5:0/4006185477 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f6a0c06be30 0x7f6a0c06e340 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.278+0000 7f6a26731640 1 --2- 172.21.15.5:0/4006185477 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a201f82f0 0x7f6a201f8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.278+0000 7f6a26731640 1 --2- 172.21.15.5:0/4006185477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a20162820 0x7f6a200cb0f0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.278+0000 7f6a26731640 1 --2- 172.21.15.5:0/4006185477 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a200b3e80 0x7f6a2014cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.278+0000 7f6a26731640 1 -- 172.21.15.5:0/4006185477 >> 172.21.15.5:0/4006185477 conn(0x7f6a200d4bd0 msgr2=0x7f6a200ba1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.278+0000 7f6a26731640 1 -- 172.21.15.5:0/4006185477 shutdown_connections 2024-05-20T18:37:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:30.278+0000 7f6a26731640 1 -- 172.21.15.5:0/4006185477 wait complete. 2024-05-20T18:37:30.692 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 101, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T18:37:14.365195+0000', 'modified': '2024-05-20T18:37:29.907939+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 120, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9012}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9012': {'gid': 9012, 'name': 'c', 'rank': 0, 'incarnation': 101, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.5:6837/4273429943', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6835', 'nonce': 4273429943}, {'type': 'v1', 'addr': '172.21.15.5:6837', 'nonce': 4273429943}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-20T18:37:30.692 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9012:up:replay 2024-05-20T18:37:31.693 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:37:31.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.823+0000 7f4b59aca640 1 Processor -- start 2024-05-20T18:37:31.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.823+0000 7f4b59aca640 1 -- start start 2024-05-20T18:37:31.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.824+0000 7f4b59aca640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f4b54162820 0x7f4b540b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:31.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.824+0000 7f4b59aca640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b540b3e80 0x7f4b5406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:31.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.824+0000 7f4b59aca640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b5406dd20 con 0x7f4b5406d910 2024-05-20T18:37:31.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.824+0000 7f4b59aca640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f4b540d0630 con 0x7f4b54162820 2024-05-20T18:37:31.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.824+0000 7f4b59aca640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4b540d39c0 con 0x7f4b540b3e80 2024-05-20T18:37:31.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.824+0000 7f4b53fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f4b5406d910 0x7f4b540d27a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38662/0 (socket says 172.21.15.5:38662) 2024-05-20T18:37:31.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.824+0000 7f4b53fff640 1 -- 172.21.15.5:0/3402567529 learned_addr learned my addr 172.21.15.5:0/3402567529 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:31.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.824+0000 7f4b52ffd640 1 --2- 172.21.15.5:0/3402567529 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b540b3e80 0x7f4b5406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:31.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.824+0000 7f4b537fe640 1 --2- 172.21.15.5:0/3402567529 >> v2:172.21.15.63:3300/0 conn(0x7f4b54162820 0x7f4b540b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:31.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.825+0000 7f4b527fc640 1 -- 172.21.15.5:0/3402567529 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2355484177 0 0) 0x7f4b4c001110 con 0x7f4b5406d910 2024-05-20T18:37:31.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.825+0000 7f4b527fc640 1 -- 172.21.15.5:0/3402567529 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b3c003520 con 0x7f4b5406d910 2024-05-20T18:37:31.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.825+0000 7f4b527fc640 1 -- 172.21.15.5:0/3402567529 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3472922402 0 0) 0x7f4b3c003520 con 0x7f4b5406d910 2024-05-20T18:37:31.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.825+0000 7f4b527fc640 1 -- 172.21.15.5:0/3402567529 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b540b3e80 msgr2=0x7f4b5406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:31.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.825+0000 7f4b527fc640 1 --2- 172.21.15.5:0/3402567529 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b540b3e80 0x7f4b5406d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:31.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.825+0000 7f4b527fc640 1 -- 172.21.15.5:0/3402567529 >> v2:172.21.15.63:3300/0 conn(0x7f4b54162820 msgr2=0x7f4b540b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:31.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.825+0000 7f4b527fc640 1 --2- 172.21.15.5:0/3402567529 >> v2:172.21.15.63:3300/0 conn(0x7f4b54162820 0x7f4b540b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:31.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.825+0000 7f4b527fc640 1 -- 172.21.15.5:0/3402567529 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b540d3ca0 con 0x7f4b5406d910 2024-05-20T18:37:31.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.825+0000 7f4b527fc640 1 -- 172.21.15.5:0/3402567529 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f4b4c001840 con 0x7f4b5406d910 2024-05-20T18:37:31.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.825+0000 7f4b527fc640 1 -- 172.21.15.5:0/3402567529 >> v1:172.21.15.5:6789/0 conn(0x7f4b5406d910 legacy=0x7f4b540d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:31.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.825+0000 7f4b527fc640 1 --2- 172.21.15.5:0/3402567529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b3c0039b0 0x7f4b3c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:31.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.825+0000 7f4b527fc640 1 --2- 172.21.15.5:0/3402567529 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b540b3e80 0x7f4b3c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:31.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.825+0000 7f4b527fc640 1 --2- 172.21.15.5:0/3402567529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b3c004d40 0x7f4b3c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:31.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.825+0000 7f4b527fc640 1 -- 172.21.15.5:0/3402567529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4b540d0630 con 0x7f4b3c004d40 2024-05-20T18:37:31.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.825+0000 7f4b527fc640 1 -- 172.21.15.5:0/3402567529 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4b540d39c0 con 0x7f4b3c0039b0 2024-05-20T18:37:31.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.825+0000 7f4b527fc640 1 -- 172.21.15.5:0/3402567529 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4b3c00b9c0 con 0x7f4b540b3e80 2024-05-20T18:37:31.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.825+0000 7f4b537fe640 1 --2- 172.21.15.5:0/3402567529 >> v2:172.21.15.63:3300/0 conn(0x7f4b54162820 0x7f4b540b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:37:31.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.825+0000 7f4b53fff640 1 --2- 172.21.15.5:0/3402567529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b3c004d40 0x7f4b3c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:31.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.825+0000 7f4b52ffd640 1 --2- 172.21.15.5:0/3402567529 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b540b3e80 0x7f4b3c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:31.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.825+0000 7f4b537fe640 1 --2- 172.21.15.5:0/3402567529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b3c0039b0 0x7f4b3c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:31.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.826+0000 7f4b53fff640 1 -- 172.21.15.5:0/3402567529 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b540b3e80 msgr2=0x7f4b3c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:31.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.826+0000 7f4b53fff640 1 --2- 172.21.15.5:0/3402567529 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b540b3e80 0x7f4b3c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:31.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.826+0000 7f4b53fff640 1 -- 172.21.15.5:0/3402567529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b3c0039b0 msgr2=0x7f4b3c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:31.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.826+0000 7f4b53fff640 1 --2- 172.21.15.5:0/3402567529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b3c0039b0 0x7f4b3c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:31.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.826+0000 7f4b53fff640 1 -- 172.21.15.5:0/3402567529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b3c003520 con 0x7f4b3c004d40 2024-05-20T18:37:31.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.826+0000 7f4b537fe640 1 --2- 172.21.15.5:0/3402567529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b3c0039b0 0x7f4b3c003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:31.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.826+0000 7f4b53fff640 1 --2- 172.21.15.5:0/3402567529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b3c004d40 0x7f4b3c005150 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00c490 tx=0x7f4b3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04a42ff55c392e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.826+0000 7f4b527fc640 1 -- 172.21.15.5:0/3402567529 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c00c0e0 con 0x7f4b3c004d40 2024-05-20T18:37:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.826+0000 7f4b527fc640 1 -- 172.21.15.5:0/3402567529 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b4c034070 con 0x7f4b3c004d40 2024-05-20T18:37:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.826+0000 7f4b527fc640 1 -- 172.21.15.5:0/3402567529 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c005af0 con 0x7f4b3c004d40 2024-05-20T18:37:31.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.827+0000 7f4b59aca640 1 -- 172.21.15.5:0/3402567529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b3c004d40 msgr2=0x7f4b3c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:31.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.827+0000 7f4b59aca640 1 --2- 172.21.15.5:0/3402567529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b3c004d40 0x7f4b3c005150 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00c490 tx=0x7f4b3c005830 comp rx=0 tx=0).stop 2024-05-20T18:37:31.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.827+0000 7f4b53fff640 1 -- 172.21.15.5:0/3402567529 reap_dead start 2024-05-20T18:37:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.827+0000 7f4b59aca640 1 -- 172.21.15.5:0/3402567529 shutdown_connections 2024-05-20T18:37:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.827+0000 7f4b59aca640 1 -- 172.21.15.5:0/3402567529 >> 172.21.15.5:0/3402567529 conn(0x7f4b540d4bd0 msgr2=0x7f4b540d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.827+0000 7f4b59aca640 1 -- 172.21.15.5:0/3402567529 shutdown_connections 2024-05-20T18:37:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.827+0000 7f4b59aca640 1 -- 172.21.15.5:0/3402567529 wait complete. 2024-05-20T18:37:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.829+0000 7f4b59aca640 1 Processor -- start 2024-05-20T18:37:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.829+0000 7f4b59aca640 1 -- start start 2024-05-20T18:37:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.831+0000 7f4b59aca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b3c004d40 0x7f4b541f9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.831+0000 7f4b59aca640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b541f4d10 0x7f4b54161590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.831+0000 7f4b59aca640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b541f5120 0x7f4b54161ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.831+0000 7f4b59aca640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4b540d0300 con 0x7f4b3c004d40 2024-05-20T18:37:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.831+0000 7f4b59aca640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4b540d4260 con 0x7f4b541f4d10 2024-05-20T18:37:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.831+0000 7f4b59aca640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4b540b91a0 con 0x7f4b541f5120 2024-05-20T18:37:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.831+0000 7f4b537fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b3c004d40 0x7f4b541f9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.831+0000 7f4b537fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b3c004d40 0x7f4b541f9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34098/0 (socket says 172.21.15.5:34098) 2024-05-20T18:37:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.831+0000 7f4b537fe640 1 -- 172.21.15.5:0/1661265646 learned_addr learned my addr 172.21.15.5:0/1661265646 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.831+0000 7f4b52ffd640 1 --2- 172.21.15.5:0/1661265646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b541f4d10 0x7f4b54161590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.831+0000 7f4b53fff640 1 --2- 172.21.15.5:0/1661265646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b541f5120 0x7f4b54161ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.831+0000 7f4b537fe640 1 -- 172.21.15.5:0/1661265646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b541f5120 msgr2=0x7f4b54161ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.831+0000 7f4b537fe640 1 --2- 172.21.15.5:0/1661265646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b541f5120 0x7f4b54161ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.831+0000 7f4b537fe640 1 -- 172.21.15.5:0/1661265646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b541f4d10 msgr2=0x7f4b54161590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.831+0000 7f4b537fe640 1 --2- 172.21.15.5:0/1661265646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b541f4d10 0x7f4b54161590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.831+0000 7f4b537fe640 1 -- 172.21.15.5:0/1661265646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b541f9ab0 con 0x7f4b3c004d40 2024-05-20T18:37:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.831+0000 7f4b537fe640 1 --2- 172.21.15.5:0/1661265646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b3c004d40 0x7f4b541f9570 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f4b44011900 tx=0x7f4b44011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.831+0000 7f4b3bfff640 1 -- 172.21.15.5:0/1661265646 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4400fbd0 con 0x7f4b3c004d40 2024-05-20T18:37:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.831+0000 7f4b3bfff640 1 -- 172.21.15.5:0/1661265646 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b4400fd70 con 0x7f4b3c004d40 2024-05-20T18:37:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.832+0000 7f4b59aca640 1 -- 172.21.15.5:0/1661265646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b541e80d0 con 0x7f4b3c004d40 2024-05-20T18:37:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.832+0000 7f4b3bfff640 1 -- 172.21.15.5:0/1661265646 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b44015660 con 0x7f4b3c004d40 2024-05-20T18:37:31.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.832+0000 7f4b59aca640 1 -- 172.21.15.5:0/1661265646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b541e8690 con 0x7f4b3c004d40 2024-05-20T18:37:31.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.832+0000 7f4b59aca640 1 -- 172.21.15.5:0/1661265646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b54075b00 con 0x7f4b3c004d40 2024-05-20T18:37:31.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.833+0000 7f4b3bfff640 1 -- 172.21.15.5:0/1661265646 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4b44015800 con 0x7f4b3c004d40 2024-05-20T18:37:31.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.834+0000 7f4b3bfff640 1 --2- 172.21.15.5:0/1661265646 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4b4006be60 0x7f4b4006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:31.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.834+0000 7f4b3bfff640 1 -- 172.21.15.5:0/1661265646 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7f4b4408ba80 con 0x7f4b3c004d40 2024-05-20T18:37:31.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.834+0000 7f4b52ffd640 1 --2- 172.21.15.5:0/1661265646 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4b4006be60 0x7f4b4006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:31.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.834+0000 7f4b52ffd640 1 --2- 172.21.15.5:0/1661265646 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4b4006be60 0x7f4b4006e370 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4b540b6930 tx=0x7f4b4800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:31.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:31.839+0000 7f4b3bfff640 1 -- 172.21.15.5:0/1661265646 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4b541f9ab0 con 0x7f4b3c004d40 2024-05-20T18:37:32.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:32.061+0000 7f4b59aca640 1 -- 172.21.15.5:0/1661265646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b54160d20 con 0x7f4b3c004d40 2024-05-20T18:37:32.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:32.062+0000 7f4b3bfff640 1 -- 172.21.15.5:0/1661265646 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5173 (secure 0 0 0) 0x7f4b44019030 con 0x7f4b3c004d40 2024-05-20T18:37:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:32.064 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":103,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/266630184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":266630184},{"type":"v1","addr":"172.21.15.63:6834","nonce":266630184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7361,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1494022697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":1494022697},{"type":"v1","addr":"172.21.15.63:6835","nonce":1494022697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":9009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/1739221946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1739221946},{"type":"v1","addr":"172.21.15.5:6836","nonce":1739221946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:37:14.365195+0000","modified":"2024-05-20T18:37:30.916356+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9012":{"gid":9012,"name":"c","rank":0,"incarnation":101,"state":"up:rejoin","state_seq":7,"addr":"172.21.15.5:6837/4273429943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":4273429943},{"type":"v1","addr":"172.21.15.5:6837","nonce":4273429943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-05-20T18:37:32.064 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 103 2024-05-20T18:37:32.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:32.064+0000 7f4b59aca640 1 -- 172.21.15.5:0/1661265646 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4b4006be60 msgr2=0x7f4b4006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:32.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:32.064+0000 7f4b59aca640 1 --2- 172.21.15.5:0/1661265646 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4b4006be60 0x7f4b4006e370 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4b540b6930 tx=0x7f4b4800b040 comp rx=0 tx=0).stop 2024-05-20T18:37:32.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:32.064+0000 7f4b59aca640 1 -- 172.21.15.5:0/1661265646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b3c004d40 msgr2=0x7f4b541f9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:32.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:32.064+0000 7f4b59aca640 1 --2- 172.21.15.5:0/1661265646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b3c004d40 0x7f4b541f9570 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f4b44011900 tx=0x7f4b44011dc0 comp rx=0 tx=0).stop 2024-05-20T18:37:32.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:32.064+0000 7f4b59aca640 1 -- 172.21.15.5:0/1661265646 shutdown_connections 2024-05-20T18:37:32.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:32.064+0000 7f4b59aca640 1 --2- 172.21.15.5:0/1661265646 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4b4006be60 0x7f4b4006e370 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:32.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:32.064+0000 7f4b59aca640 1 --2- 172.21.15.5:0/1661265646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b541f5120 0x7f4b54161ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:32.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:32.064+0000 7f4b59aca640 1 --2- 172.21.15.5:0/1661265646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b541f4d10 0x7f4b54161590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:32.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:32.064+0000 7f4b59aca640 1 --2- 172.21.15.5:0/1661265646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b3c004d40 0x7f4b541f9570 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:32.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:32.064+0000 7f4b59aca640 1 -- 172.21.15.5:0/1661265646 >> 172.21.15.5:0/1661265646 conn(0x7f4b540d4bd0 msgr2=0x7f4b540b4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:32.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:32.064+0000 7f4b59aca640 1 -- 172.21.15.5:0/1661265646 shutdown_connections 2024-05-20T18:37:32.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:32.064+0000 7f4b59aca640 1 -- 172.21.15.5:0/1661265646 wait complete. 2024-05-20T18:37:32.480 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 103, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T18:37:14.365195+0000', 'modified': '2024-05-20T18:37:30.916356+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 120, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9012}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9012': {'gid': 9012, 'name': 'c', 'rank': 0, 'incarnation': 101, 'state': 'up:rejoin', 'state_seq': 7, 'addr': '172.21.15.5:6837/4273429943', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6835', 'nonce': 4273429943}, {'type': 'v1', 'addr': '172.21.15.5:6837', 'nonce': 4273429943}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-20T18:37:32.480 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9012:up:rejoin 2024-05-20T18:37:33.481 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:37:33.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.609+0000 7fc55ac84640 1 Processor -- start 2024-05-20T18:37:33.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.610+0000 7fc55ac84640 1 -- start start 2024-05-20T18:37:33.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.611+0000 7fc55ac84640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc5540b6620 0x7fc5540ba3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:33.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.611+0000 7fc55ac84640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5541634a0 con 0x7fc554162ce0 2024-05-20T18:37:33.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.611+0000 7fc55ac84640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc554163780 con 0x7fc5540d3bf0 2024-05-20T18:37:33.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.611+0000 7fc55ac84640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fc5540d3480 con 0x7fc5540b6620 2024-05-20T18:37:33.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.611+0000 7fc558a20640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc5540b6620 0x7fc5540ba3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.611+0000 7fc553fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc554162ce0 0x7fc5540b8070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38668/0 (socket says 172.21.15.5:38668) 2024-05-20T18:37:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.611+0000 7fc553fff640 1 -- 172.21.15.5:0/193519243 learned_addr learned my addr 172.21.15.5:0/193519243 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.611+0000 7fc558a20640 1 -- 172.21.15.5:0/193519243 >> v1:172.21.15.63:6789/0 conn(0x7fc5540d3bf0 legacy=0x7fc5540d4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.611+0000 7fc558a20640 1 -- 172.21.15.5:0/193519243 >> v1:172.21.15.5:6789/0 conn(0x7fc554162ce0 legacy=0x7fc5540b8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.612+0000 7fc558a20640 1 -- 172.21.15.5:0/193519243 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc554163a60 con 0x7fc5540b6620 2024-05-20T18:37:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.612+0000 7fc5537fe640 1 -- 172.21.15.5:0/193519243 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 970919137 0 0) 0x7fc5541634a0 con 0x7fc554162ce0 2024-05-20T18:37:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.612+0000 7fc558a20640 1 --2- 172.21.15.5:0/193519243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5540b6620 0x7fc5540ba3c0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fc5440097a0 tx=0x7fc54402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e333751081cb0375 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.612+0000 7fc5537fe640 1 -- 172.21.15.5:0/193519243 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544037070 con 0x7fc5540b6620 2024-05-20T18:37:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.613+0000 7fc5537fe640 1 -- 172.21.15.5:0/193519243 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc544032440 con 0x7fc5540b6620 2024-05-20T18:37:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.613+0000 7fc5537fe640 1 -- 172.21.15.5:0/193519243 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544032810 con 0x7fc5540b6620 2024-05-20T18:37:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.613+0000 7fc55ac84640 1 -- 172.21.15.5:0/193519243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5540b6620 msgr2=0x7fc5540ba3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.613+0000 7fc55ac84640 1 --2- 172.21.15.5:0/193519243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5540b6620 0x7fc5540ba3c0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fc5440097a0 tx=0x7fc54402a1e0 comp rx=0 tx=0).stop 2024-05-20T18:37:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.613+0000 7fc55ac84640 1 -- 172.21.15.5:0/193519243 shutdown_connections 2024-05-20T18:37:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.613+0000 7fc55ac84640 1 --2- 172.21.15.5:0/193519243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5540b6620 0x7fc5540ba3c0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.613+0000 7fc55ac84640 1 -- 172.21.15.5:0/193519243 >> 172.21.15.5:0/193519243 conn(0x7fc55406d240 msgr2=0x7fc55406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.613+0000 7fc55ac84640 1 -- 172.21.15.5:0/193519243 shutdown_connections 2024-05-20T18:37:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.613+0000 7fc55ac84640 1 -- 172.21.15.5:0/193519243 wait complete. 2024-05-20T18:37:33.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.615+0000 7fc55ac84640 1 Processor -- start 2024-05-20T18:37:33.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.615+0000 7fc55ac84640 1 -- start start 2024-05-20T18:37:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.617+0000 7fc55ac84640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5540b6620 0x7fc5540c7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.617+0000 7fc55ac84640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5540d3bf0 0x7fc554159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.617+0000 7fc55ac84640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc554162ce0 0x7fc55415be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.617+0000 7fc55ac84640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5541630f0 con 0x7fc554162ce0 2024-05-20T18:37:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.617+0000 7fc55ac84640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc5540b8780 con 0x7fc5540b6620 2024-05-20T18:37:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.617+0000 7fc55ac84640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc554149de0 con 0x7fc5540d3bf0 2024-05-20T18:37:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.617+0000 7fc559221640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc554162ce0 0x7fc55415be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.617+0000 7fc559221640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc554162ce0 0x7fc55415be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34108/0 (socket says 172.21.15.5:34108) 2024-05-20T18:37:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.617+0000 7fc559221640 1 -- 172.21.15.5:0/325412274 learned_addr learned my addr 172.21.15.5:0/325412274 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.617+0000 7fc558a20640 1 --2- 172.21.15.5:0/325412274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5540b6620 0x7fc5540c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.617+0000 7fc553fff640 1 --2- 172.21.15.5:0/325412274 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5540d3bf0 0x7fc554159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.617+0000 7fc559221640 1 -- 172.21.15.5:0/325412274 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5540d3bf0 msgr2=0x7fc554159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.617+0000 7fc559221640 1 --2- 172.21.15.5:0/325412274 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5540d3bf0 0x7fc554159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.617+0000 7fc559221640 1 -- 172.21.15.5:0/325412274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5540b6620 msgr2=0x7fc5540c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.617+0000 7fc559221640 1 --2- 172.21.15.5:0/325412274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5540b6620 0x7fc5540c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.617+0000 7fc559221640 1 -- 172.21.15.5:0/325412274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5541e8bf0 con 0x7fc554162ce0 2024-05-20T18:37:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.617+0000 7fc559221640 1 --2- 172.21.15.5:0/325412274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc554162ce0 0x7fc55415be10 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fc54c00be60 tx=0x7fc54c00bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.618+0000 7fc5517fa640 1 -- 172.21.15.5:0/325412274 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54c019070 con 0x7fc554162ce0 2024-05-20T18:37:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.618+0000 7fc5517fa640 1 -- 172.21.15.5:0/325412274 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc54c014440 con 0x7fc554162ce0 2024-05-20T18:37:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.618+0000 7fc55ac84640 1 -- 172.21.15.5:0/325412274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5541e8ed0 con 0x7fc554162ce0 2024-05-20T18:37:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.618+0000 7fc5517fa640 1 -- 172.21.15.5:0/325412274 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54c014870 con 0x7fc554162ce0 2024-05-20T18:37:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.618+0000 7fc55ac84640 1 -- 172.21.15.5:0/325412274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5541e9490 con 0x7fc554162ce0 2024-05-20T18:37:33.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.619+0000 7fc55ac84640 1 -- 172.21.15.5:0/325412274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc554075b00 con 0x7fc554162ce0 2024-05-20T18:37:33.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.619+0000 7fc5517fa640 1 -- 172.21.15.5:0/325412274 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc54c01c070 con 0x7fc554162ce0 2024-05-20T18:37:33.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.620+0000 7fc5517fa640 1 --2- 172.21.15.5:0/325412274 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc54006be30 0x7fc54006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:33.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.620+0000 7fc5517fa640 1 -- 172.21.15.5:0/325412274 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7fc54c08b010 con 0x7fc554162ce0 2024-05-20T18:37:33.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.620+0000 7fc558a20640 1 --2- 172.21.15.5:0/325412274 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc54006be30 0x7fc54006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:33.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.621+0000 7fc558a20640 1 --2- 172.21.15.5:0/325412274 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc54006be30 0x7fc54006e340 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc544034000 tx=0x7fc54402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:33.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.625+0000 7fc5517fa640 1 -- 172.21.15.5:0/325412274 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc54c059190 con 0x7fc554162ce0 2024-05-20T18:37:33.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.846+0000 7fc55ac84640 1 -- 172.21.15.5:0/325412274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc554159d80 con 0x7fc554162ce0 2024-05-20T18:37:33.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.846+0000 7fc5517fa640 1 -- 172.21.15.5:0/325412274 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+5180 (secure 0 0 0) 0x7fc54c05d490 con 0x7fc554162ce0 2024-05-20T18:37:33.848 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:33.848 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":104,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/266630184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":266630184},{"type":"v1","addr":"172.21.15.63:6834","nonce":266630184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7361,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1494022697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":1494022697},{"type":"v1","addr":"172.21.15.63:6835","nonce":1494022697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":9009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/1739221946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1739221946},{"type":"v1","addr":"172.21.15.5:6836","nonce":1739221946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:37:14.365195+0000","modified":"2024-05-20T18:37:32.914662+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9012":{"gid":9012,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":8,"addr":"172.21.15.5:6837/4273429943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":4273429943},{"type":"v1","addr":"172.21.15.5:6837","nonce":4273429943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9012,"qdb_cluster":[9012]},"id":8}]} 2024-05-20T18:37:33.848 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 104 2024-05-20T18:37:33.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.848+0000 7fc55ac84640 1 -- 172.21.15.5:0/325412274 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc54006be30 msgr2=0x7fc54006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:33.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.848+0000 7fc55ac84640 1 --2- 172.21.15.5:0/325412274 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc54006be30 0x7fc54006e340 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc544034000 tx=0x7fc54402e000 comp rx=0 tx=0).stop 2024-05-20T18:37:33.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.848+0000 7fc55ac84640 1 -- 172.21.15.5:0/325412274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc554162ce0 msgr2=0x7fc55415be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:33.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.848+0000 7fc55ac84640 1 --2- 172.21.15.5:0/325412274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc554162ce0 0x7fc55415be10 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fc54c00be60 tx=0x7fc54c00bf50 comp rx=0 tx=0).stop 2024-05-20T18:37:33.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.848+0000 7fc55ac84640 1 -- 172.21.15.5:0/325412274 shutdown_connections 2024-05-20T18:37:33.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.848+0000 7fc55ac84640 1 --2- 172.21.15.5:0/325412274 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc54006be30 0x7fc54006e340 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:33.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.848+0000 7fc55ac84640 1 --2- 172.21.15.5:0/325412274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc554162ce0 0x7fc55415be10 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:33.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.848+0000 7fc55ac84640 1 --2- 172.21.15.5:0/325412274 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5540d3bf0 0x7fc554159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:33.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.848+0000 7fc55ac84640 1 --2- 172.21.15.5:0/325412274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5540b6620 0x7fc5540c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:33.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.848+0000 7fc55ac84640 1 -- 172.21.15.5:0/325412274 >> 172.21.15.5:0/325412274 conn(0x7fc55406d240 msgr2=0x7fc554067700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:33.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.848+0000 7fc55ac84640 1 -- 172.21.15.5:0/325412274 shutdown_connections 2024-05-20T18:37:33.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:33.848+0000 7fc55ac84640 1 -- 172.21.15.5:0/325412274 wait complete. 2024-05-20T18:37:34.263 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 104, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T18:37:14.365195+0000', 'modified': '2024-05-20T18:37:32.914662+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 120, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9012}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9012': {'gid': 9012, 'name': 'c', 'rank': 0, 'incarnation': 101, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.5:6837/4273429943', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6835', 'nonce': 4273429943}, {'type': 'v1', 'addr': '172.21.15.5:6837', 'nonce': 4273429943}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9012, 'qdb_cluster': [9012]} 2024-05-20T18:37:34.264 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-20T18:37:34.264 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-20T18:37:34.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.385+0000 7fa6e2e73640 1 Processor -- start 2024-05-20T18:37:34.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.385+0000 7fa6e2e73640 1 -- start start 2024-05-20T18:37:34.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.386+0000 7fa6e2e73640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6dc0cf250 0x7fa6dc15f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:34.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.386+0000 7fa6e2e73640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa6dc0b6680 0x7fa6dc0b6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:34.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.386+0000 7fa6e2e73640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6dc0b6fd0 con 0x7fa6dc163020 2024-05-20T18:37:34.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.386+0000 7fa6e2e73640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fa6dc15f9a0 con 0x7fa6dc0b6680 2024-05-20T18:37:34.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.386+0000 7fa6e2e73640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa6dc0d0b10 con 0x7fa6dc0cf250 2024-05-20T18:37:34.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.386+0000 7fa6d3fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa6dc0b6680 0x7fa6dc0b6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:34.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.386+0000 7fa6d3fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa6dc0b6680 0x7fa6dc0b6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34120/0 (socket says 172.21.15.5:34120) 2024-05-20T18:37:34.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.386+0000 7fa6d3fff640 1 -- 172.21.15.5:0/1626864040 learned_addr learned my addr 172.21.15.5:0/1626864040 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:34.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.387+0000 7fa6e0c0f640 1 --2- 172.21.15.5:0/1626864040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6dc0cf250 0x7fa6dc15f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:34.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.387+0000 7fa6d3fff640 1 -- 172.21.15.5:0/1626864040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6dc0cf250 msgr2=0x7fa6dc15f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:34.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.387+0000 7fa6d3fff640 1 --2- 172.21.15.5:0/1626864040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6dc0cf250 0x7fa6dc15f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.387+0000 7fa6d3fff640 1 -- 172.21.15.5:0/1626864040 >> v1:172.21.15.63:6789/0 conn(0x7fa6dc163020 legacy=0x7fa6dc163430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:34.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.387+0000 7fa6d3fff640 1 -- 172.21.15.5:0/1626864040 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6dc0d0550 con 0x7fa6dc0b6680 2024-05-20T18:37:34.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.388+0000 7fa6d3fff640 1 --2- 172.21.15.5:0/1626864040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6dc0b6680 0x7fa6dc0b6a90 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc009870 tx=0x7fa6cc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69abb9496b75e062 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:34.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.388+0000 7fa6d37fe640 1 -- 172.21.15.5:0/1626864040 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc037070 con 0x7fa6dc0b6680 2024-05-20T18:37:34.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.388+0000 7fa6d37fe640 1 -- 172.21.15.5:0/1626864040 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6cc032440 con 0x7fa6dc0b6680 2024-05-20T18:37:34.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.388+0000 7fa6d37fe640 1 -- 172.21.15.5:0/1626864040 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc032810 con 0x7fa6dc0b6680 2024-05-20T18:37:34.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.388+0000 7fa6e2e73640 1 -- 172.21.15.5:0/1626864040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6dc0b6680 msgr2=0x7fa6dc0b6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:34.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.388+0000 7fa6e2e73640 1 --2- 172.21.15.5:0/1626864040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6dc0b6680 0x7fa6dc0b6a90 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc009870 tx=0x7fa6cc02a1c0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.389+0000 7fa6e2e73640 1 -- 172.21.15.5:0/1626864040 shutdown_connections 2024-05-20T18:37:34.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.389+0000 7fa6e2e73640 1 --2- 172.21.15.5:0/1626864040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6dc0b6680 0x7fa6dc0b6a90 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.389+0000 7fa6e2e73640 1 --2- 172.21.15.5:0/1626864040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6dc0cf250 0x7fa6dc15f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.389+0000 7fa6e2e73640 1 -- 172.21.15.5:0/1626864040 >> 172.21.15.5:0/1626864040 conn(0x7fa6dc0d1860 msgr2=0x7fa6dc0d1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:34.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.389+0000 7fa6e2e73640 1 -- 172.21.15.5:0/1626864040 shutdown_connections 2024-05-20T18:37:34.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.389+0000 7fa6e2e73640 1 -- 172.21.15.5:0/1626864040 wait complete. 2024-05-20T18:37:34.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.390+0000 7fa6e2e73640 1 Processor -- start 2024-05-20T18:37:34.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.390+0000 7fa6e2e73640 1 -- start start 2024-05-20T18:37:34.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.392+0000 7fa6e2e73640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6dc0b6680 0x7fa6dc1550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:34.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.392+0000 7fa6e2e73640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6dc0cf250 0x7fa6dc15d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:34.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.392+0000 7fa6e2e73640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6dc163020 0x7fa6dc15dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:34.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.392+0000 7fa6e2e73640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc0d2a20 con 0x7fa6dc0b6680 2024-05-20T18:37:34.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.392+0000 7fa6e2e73640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc0d27d0 con 0x7fa6dc163020 2024-05-20T18:37:34.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.392+0000 7fa6e2e73640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa6dc0d2d00 con 0x7fa6dc0cf250 2024-05-20T18:37:34.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.392+0000 7fa6e0c0f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6dc0b6680 0x7fa6dc1550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:34.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.392+0000 7fa6e0c0f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6dc0b6680 0x7fa6dc1550a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34130/0 (socket says 172.21.15.5:34130) 2024-05-20T18:37:34.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.392+0000 7fa6e0c0f640 1 -- 172.21.15.5:0/3516931934 learned_addr learned my addr 172.21.15.5:0/3516931934 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:34.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.392+0000 7fa6d3fff640 1 --2- 172.21.15.5:0/3516931934 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6dc0cf250 0x7fa6dc15d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:34.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.392+0000 7fa6e1410640 1 --2- 172.21.15.5:0/3516931934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6dc163020 0x7fa6dc15dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:34.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.392+0000 7fa6e0c0f640 1 -- 172.21.15.5:0/3516931934 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6dc0cf250 msgr2=0x7fa6dc15d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:34.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.392+0000 7fa6e0c0f640 1 --2- 172.21.15.5:0/3516931934 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6dc0cf250 0x7fa6dc15d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.392+0000 7fa6e0c0f640 1 -- 172.21.15.5:0/3516931934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6dc163020 msgr2=0x7fa6dc15dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:34.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.392+0000 7fa6e0c0f640 1 --2- 172.21.15.5:0/3516931934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6dc163020 0x7fa6dc15dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.392+0000 7fa6e0c0f640 1 -- 172.21.15.5:0/3516931934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6dc15e430 con 0x7fa6dc0b6680 2024-05-20T18:37:34.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.392+0000 7fa6e1410640 1 --2- 172.21.15.5:0/3516931934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6dc163020 0x7fa6dc15dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:34.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.392+0000 7fa6e0c0f640 1 --2- 172.21.15.5:0/3516931934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6dc0b6680 0x7fa6dc1550a0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800e8d0 tx=0x7fa6d800ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:34.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.393+0000 7fa6d17fa640 1 -- 172.21.15.5:0/3516931934 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8017070 con 0x7fa6dc0b6680 2024-05-20T18:37:34.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.393+0000 7fa6d17fa640 1 -- 172.21.15.5:0/3516931934 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6d800c940 con 0x7fa6dc0b6680 2024-05-20T18:37:34.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.393+0000 7fa6e2e73640 1 -- 172.21.15.5:0/3516931934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6dc15e710 con 0x7fa6dc0b6680 2024-05-20T18:37:34.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.393+0000 7fa6d17fa640 1 -- 172.21.15.5:0/3516931934 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d800cd30 con 0x7fa6dc0b6680 2024-05-20T18:37:34.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.393+0000 7fa6e2e73640 1 -- 172.21.15.5:0/3516931934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6dc162290 con 0x7fa6dc0b6680 2024-05-20T18:37:34.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.395+0000 7fa6d17fa640 1 -- 172.21.15.5:0/3516931934 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa6d801a070 con 0x7fa6dc0b6680 2024-05-20T18:37:34.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.395+0000 7fa6d17fa640 1 --2- 172.21.15.5:0/3516931934 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa6c806be60 0x7fa6c806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:34.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.395+0000 7fa6d17fa640 1 -- 172.21.15.5:0/3516931934 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7fa6d808b1b0 con 0x7fa6dc0b6680 2024-05-20T18:37:34.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.395+0000 7fa6d3fff640 1 --2- 172.21.15.5:0/3516931934 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa6c806be60 0x7fa6c806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:34.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.396+0000 7fa6d3fff640 1 --2- 172.21.15.5:0/3516931934 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa6c806be60 0x7fa6c806e370 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc034000 tx=0x7fa6cc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:34.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.396+0000 7fa6e9df8740 1 Processor -- start 2024-05-20T18:37:34.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.396+0000 7fa6e9df8740 1 -- start start 2024-05-20T18:37:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.397+0000 7fa6e9df8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3fa759270 0x55e3fab24700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.397+0000 7fa6e9df8740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e3fa7303b0 0x55e3fab210a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.397+0000 7fa6e9df8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3fa738590 0x55e3faafa360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.397+0000 7fa6e9df8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e3faa5d4d0 con 0x55e3fa738590 2024-05-20T18:37:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.397+0000 7fa6e9df8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e3faa053e0 con 0x55e3fa759270 2024-05-20T18:37:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.397+0000 7fa6e9df8740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55e3faa59490 con 0x55e3fa7303b0 2024-05-20T18:37:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.398+0000 7fa6e1410640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3fa738590 0x55e3faafa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.398+0000 7fa6e1410640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3fa738590 0x55e3faafa360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34146/0 (socket says 172.21.15.5:34146) 2024-05-20T18:37:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.398+0000 7fa6e1410640 1 -- 172.21.15.5:0/566453730 learned_addr learned my addr 172.21.15.5:0/566453730 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.398+0000 7fa6d3fff640 1 --2- 172.21.15.5:0/566453730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e3fa7303b0 0x55e3fab210a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:34.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.398+0000 7fa6e0c0f640 1 --2- 172.21.15.5:0/566453730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3fa759270 0x55e3fab24700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:34.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.398+0000 7fa6e1410640 1 -- 172.21.15.5:0/566453730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e3fa7303b0 msgr2=0x55e3fab210a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:34.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.398+0000 7fa6e1410640 1 --2- 172.21.15.5:0/566453730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e3fa7303b0 0x55e3fab210a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.398+0000 7fa6e1410640 1 -- 172.21.15.5:0/566453730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3fa759270 msgr2=0x55e3fab24700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:34.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.398+0000 7fa6e1410640 1 --2- 172.21.15.5:0/566453730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3fa759270 0x55e3fab24700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.398+0000 7fa6e1410640 1 -- 172.21.15.5:0/566453730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3fa7712d0 con 0x55e3fa738590 2024-05-20T18:37:34.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.398+0000 7fa6e0c0f640 1 --2- 172.21.15.5:0/566453730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3fa759270 0x55e3fab24700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:34.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.398+0000 7fa6e1410640 1 --2- 172.21.15.5:0/566453730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3fa738590 0x55e3faafa360 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4001c10 tx=0x7fa6d40020d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547bec43d5ba74ea server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:34.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.398+0000 7fa6c27fc640 1 -- 172.21.15.5:0/566453730 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4019070 con 0x55e3fa738590 2024-05-20T18:37:34.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.398+0000 7fa6c27fc640 1 -- 172.21.15.5:0/566453730 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6d4003b80 con 0x55e3fa738590 2024-05-20T18:37:34.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.398+0000 7fa6c27fc640 1 -- 172.21.15.5:0/566453730 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d400e7a0 con 0x55e3fa738590 2024-05-20T18:37:34.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.399+0000 7fa6e9df8740 1 -- 172.21.15.5:0/566453730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3fa738590 msgr2=0x55e3faafa360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:34.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.399+0000 7fa6e9df8740 1 --2- 172.21.15.5:0/566453730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3fa738590 0x55e3faafa360 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4001c10 tx=0x7fa6d40020d0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.399+0000 7fa6e9df8740 1 -- 172.21.15.5:0/566453730 shutdown_connections 2024-05-20T18:37:34.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.399+0000 7fa6e9df8740 1 --2- 172.21.15.5:0/566453730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3fa738590 0x55e3faafa360 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.399+0000 7fa6e9df8740 1 --2- 172.21.15.5:0/566453730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e3fa7303b0 0x55e3fab210a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.399+0000 7fa6e9df8740 1 --2- 172.21.15.5:0/566453730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3fa759270 0x55e3fab24700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.399+0000 7fa6e9df8740 1 -- 172.21.15.5:0/566453730 >> 172.21.15.5:0/566453730 conn(0x55e3fa74d3c0 msgr2=0x55e3faaf9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:34.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.399+0000 7fa6e9df8740 1 -- 172.21.15.5:0/566453730 shutdown_connections 2024-05-20T18:37:34.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.399+0000 7fa6e9df8740 1 -- 172.21.15.5:0/566453730 wait complete. 2024-05-20T18:37:34.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.400+0000 7fa6e9df8740 1 -- start start 2024-05-20T18:37:34.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.400+0000 7fa6e9df8740 1 Processor -- start 2024-05-20T18:37:34.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.402+0000 7fa6e9df8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3fa7303b0 0x55e3fab1d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:34.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.402+0000 7fa6e9df8740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e3fa738590 0x55e3fab23160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:34.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.402+0000 7fa6e9df8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3fa759270 0x55e3fab0c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:34.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.402+0000 7fa6e9df8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e3faa188e0 con 0x55e3fa7303b0 2024-05-20T18:37:34.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.402+0000 7fa6e9df8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e3fab0a650 con 0x55e3fa759270 2024-05-20T18:37:34.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.402+0000 7fa6e9df8740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55e3fab0a7f0 con 0x55e3fa738590 2024-05-20T18:37:34.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.402+0000 7fa6e9df8740 10 client.0 _ll_register_callbacks cb 0x55e3fa9e59c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:37:34.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.402+0000 7fa6e0c0f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3fa7303b0 0x55e3fab1d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:34.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.402+0000 7fa6e0c0f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3fa7303b0 0x55e3fab1d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34148/0 (socket says 172.21.15.5:34148) 2024-05-20T18:37:34.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.402+0000 7fa6e0c0f640 1 -- 172.21.15.5:0/3551976005 learned_addr learned my addr 172.21.15.5:0/3551976005 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:34.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.402+0000 7fa6d3fff640 1 --2- 172.21.15.5:0/3551976005 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e3fa738590 0x55e3fab23160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:34.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.402+0000 7fa6e1410640 1 --2- 172.21.15.5:0/3551976005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3fa759270 0x55e3fab0c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:34.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.402+0000 7fa6e0c0f640 1 -- 172.21.15.5:0/3551976005 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e3fa738590 msgr2=0x55e3fab23160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:34.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.402+0000 7fa6e0c0f640 1 --2- 172.21.15.5:0/3551976005 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e3fa738590 0x55e3fab23160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.402+0000 7fa6e0c0f640 1 -- 172.21.15.5:0/3551976005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3fa759270 msgr2=0x55e3fab0c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:34.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.402+0000 7fa6e0c0f640 1 --2- 172.21.15.5:0/3551976005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3fa759270 0x55e3fab0c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.402+0000 7fa6e0c0f640 1 -- 172.21.15.5:0/3551976005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3fa83b6b0 con 0x55e3fa7303b0 2024-05-20T18:37:34.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.402+0000 7fa6d3fff640 1 --2- 172.21.15.5:0/3551976005 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e3fa738590 0x55e3fab23160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:34.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.402+0000 7fa6e0c0f640 1 --2- 172.21.15.5:0/3551976005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3fa7303b0 0x55e3fab1d260 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800e8a0 tx=0x7fa6d8059360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf2c570453870f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:34.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.402+0000 7fa6bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:37:34.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.402+0000 7fa6bb7fe640 1 -- 172.21.15.5:0/3551976005 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8086020 con 0x55e3fa7303b0 2024-05-20T18:37:34.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.402+0000 7fa6bb7fe640 1 -- 172.21.15.5:0/3551976005 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7fa6d805d370 con 0x55e3fa7303b0 2024-05-20T18:37:34.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.402+0000 7fa6e9df8740 1 -- 172.21.15.5:0/3551976005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6d4003b80 con 0x55e3fa7303b0 2024-05-20T18:37:34.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.403+0000 7fa6bb7fe640 1 -- 172.21.15.5:0/3551976005 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7fa6d8078050 con 0x55e3fa7303b0 2024-05-20T18:37:34.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.403+0000 7fa6e9df8740 10 client.9318 fetch_fsmap learned FSMap version 104 2024-05-20T18:37:34.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.403+0000 7fa6e9df8740 1 -- 172.21.15.5:0/3551976005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) -- 0x55e3fab0cc00 con 0x55e3fa7303b0 2024-05-20T18:37:34.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.403+0000 7fa6bb7fe640 1 -- 172.21.15.5:0/3551976005 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6d4003b80 con 0x55e3fa7303b0 2024-05-20T18:37:34.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.403+0000 7fa6bb7fe640 1 -- 172.21.15.5:0/3551976005 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 104) ==== 3494+0+0 (secure 0 0 0) 0x7fa6d80587b0 con 0x55e3fa7303b0 2024-05-20T18:37:34.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.403+0000 7fa6e9df8740 10 client.9318 fetch_fsmap finished waiting for FSMap version 104 2024-05-20T18:37:34.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.403+0000 7fa6e9df8740 10 client.9318 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-20T18:37:34.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.403+0000 7fa6e9df8740 20 client.9318 populate_metadata read hostname 'smithi005' 2024-05-20T18:37:34.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.403+0000 7fa6e9df8740 1 --2- 172.21.15.5:0/3551976005 >> [v2:172.21.15.5:6835/4273429943,v1:172.21.15.5:6837/4273429943] conn(0x55e3fa717e40 0x55e3fab51cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:34.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.403+0000 7fa6e9df8740 4 client.9318 mds_command: new command op to 9012 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:37:34.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.403+0000 7fa6e9df8740 1 -- 172.21.15.5:0/3551976005 --> [v2:172.21.15.5:6835/4273429943,v1:172.21.15.5:6837/4273429943] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3faa6ac00 con 0x55e3fa717e40 2024-05-20T18:37:34.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.403+0000 7fa6d3fff640 1 --2- 172.21.15.5:0/3551976005 >> [v2:172.21.15.5:6835/4273429943,v1:172.21.15.5:6837/4273429943] conn(0x55e3fa717e40 0x55e3fab51cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:34.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.403+0000 7fa6bb7fe640 10 client.9318 ms_handle_connect on v2:172.21.15.5:6835/4273429943 2024-05-20T18:37:34.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.403+0000 7fa6d3fff640 1 --2- 172.21.15.5:0/3551976005 >> [v2:172.21.15.5:6835/4273429943,v1:172.21.15.5:6837/4273429943] conn(0x55e3fa717e40 0x55e3fab51cb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8deb49286c77499 server_cookie=6c4f086840275d67 in_seq=0 out_seq=0 2024-05-20T18:37:34.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.404+0000 7fa6bb7fe640 1 -- 172.21.15.5:0/3551976005 <== mds.0 v2:172.21.15.5:6835/4273429943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e3faa6ac00 con 0x55e3fa717e40 2024-05-20T18:37:34.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.404+0000 7fa6bb7fe640 10 client.9318 handle_command_reply: tid=0 2024-05-20T18:37:34.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.404+0000 7fa6e9df8740 1 client.9318 shutdown 2024-05-20T18:37:34.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.404+0000 7fa6e9df8740 1 -- 172.21.15.5:0/3551976005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3fa7303b0 msgr2=0x55e3fab1d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:34.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.404+0000 7fa6e9df8740 1 --2- 172.21.15.5:0/3551976005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3fa7303b0 0x55e3fab1d260 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800e8a0 tx=0x7fa6d8059360 comp rx=0 tx=0).stop 2024-05-20T18:37:34.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.404+0000 7fa6e9df8740 1 -- 172.21.15.5:0/3551976005 shutdown_connections 2024-05-20T18:37:34.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.404+0000 7fa6e9df8740 1 --2- 172.21.15.5:0/3551976005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3fa759270 0x55e3fab0c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.404+0000 7fa6e9df8740 1 --2- 172.21.15.5:0/3551976005 >> [v2:172.21.15.5:6835/4273429943,v1:172.21.15.5:6837/4273429943] conn(0x55e3fa717e40 0x55e3fab51cb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.404+0000 7fa6e9df8740 1 --2- 172.21.15.5:0/3551976005 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e3fa738590 0x55e3fab23160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.404+0000 7fa6e9df8740 1 --2- 172.21.15.5:0/3551976005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3fa7303b0 0x55e3fab1d260 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.404+0000 7fa6bb7fe640 0 client.9318 ms_handle_reset on v2:172.21.15.5:6835/4273429943 2024-05-20T18:37:34.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.404+0000 7fa6e9df8740 1 -- 172.21.15.5:0/3551976005 >> 172.21.15.5:0/3551976005 conn(0x55e3fa74d3c0 msgr2=0x55e3faaf8e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:34.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.404+0000 7fa6e9df8740 1 -- 172.21.15.5:0/3551976005 shutdown_connections 2024-05-20T18:37:34.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.405+0000 7fa6e9df8740 1 -- 172.21.15.5:0/3551976005 wait complete. 2024-05-20T18:37:34.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.405+0000 7fa6e9df8740 20 client.9318 trim_cache size 0 max 16384 2024-05-20T18:37:34.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.424+0000 7fa6e9df8740 1 Processor -- start 2024-05-20T18:37:34.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.424+0000 7fa6e9df8740 1 -- start start 2024-05-20T18:37:34.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.425+0000 7fa6e9df8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3fa7303b0 0x55e3fab62e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:34.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.425+0000 7fa6e9df8740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e3fa738590 0x55e3fab633d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:34.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.425+0000 7fa6e9df8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3fa759270 0x55e3fab3cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:34.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.425+0000 7fa6e9df8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc201730 con 0x55e3fa759270 2024-05-20T18:37:34.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.425+0000 7fa6e9df8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc1615b0 con 0x55e3fa7303b0 2024-05-20T18:37:34.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.425+0000 7fa6e9df8740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa6dc161c00 con 0x55e3fa738590 2024-05-20T18:37:34.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.425+0000 7fa6e1410640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3fa759270 0x55e3fab3cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:34.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.425+0000 7fa6e1410640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3fa759270 0x55e3fab3cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34162/0 (socket says 172.21.15.5:34162) 2024-05-20T18:37:34.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.425+0000 7fa6e1410640 1 -- 172.21.15.5:0/806151217 learned_addr learned my addr 172.21.15.5:0/806151217 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:34.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.425+0000 7fa6d3fff640 1 --2- 172.21.15.5:0/806151217 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e3fa738590 0x55e3fab633d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:34.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.425+0000 7fa6e0c0f640 1 --2- 172.21.15.5:0/806151217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3fa7303b0 0x55e3fab62e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:34.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.425+0000 7fa6e1410640 1 -- 172.21.15.5:0/806151217 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e3fa738590 msgr2=0x55e3fab633d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:34.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.425+0000 7fa6e1410640 1 --2- 172.21.15.5:0/806151217 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e3fa738590 0x55e3fab633d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.425+0000 7fa6e1410640 1 -- 172.21.15.5:0/806151217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3fa7303b0 msgr2=0x55e3fab62e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:34.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.425+0000 7fa6e1410640 1 --2- 172.21.15.5:0/806151217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3fa7303b0 0x55e3fab62e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.425+0000 7fa6e1410640 1 -- 172.21.15.5:0/806151217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3fab234d0 con 0x55e3fa759270 2024-05-20T18:37:34.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.425+0000 7fa6e1410640 1 --2- 172.21.15.5:0/806151217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3fa759270 0x55e3fab3cd30 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x55e3fab36280 tx=0x7fa6d400a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b5280285da10b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:34.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.426+0000 7fa6c1ffb640 1 -- 172.21.15.5:0/806151217 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4023070 con 0x55e3fa759270 2024-05-20T18:37:34.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.426+0000 7fa6c1ffb640 1 -- 172.21.15.5:0/806151217 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6d400ea60 con 0x55e3fa759270 2024-05-20T18:37:34.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.426+0000 7fa6c1ffb640 1 -- 172.21.15.5:0/806151217 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4019040 con 0x55e3fa759270 2024-05-20T18:37:34.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.426+0000 7fa6e9df8740 1 -- 172.21.15.5:0/806151217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3fa759270 msgr2=0x55e3fab3cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:34.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.426+0000 7fa6e9df8740 1 --2- 172.21.15.5:0/806151217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3fa759270 0x55e3fab3cd30 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x55e3fab36280 tx=0x7fa6d400a000 comp rx=0 tx=0).stop 2024-05-20T18:37:34.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.426+0000 7fa6e9df8740 1 -- 172.21.15.5:0/806151217 shutdown_connections 2024-05-20T18:37:34.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.426+0000 7fa6e9df8740 1 --2- 172.21.15.5:0/806151217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3fa759270 0x55e3fab3cd30 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.426+0000 7fa6e9df8740 1 --2- 172.21.15.5:0/806151217 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e3fa738590 0x55e3fab633d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.426+0000 7fa6e9df8740 1 --2- 172.21.15.5:0/806151217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3fa7303b0 0x55e3fab62e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.426+0000 7fa6e9df8740 1 -- 172.21.15.5:0/806151217 >> 172.21.15.5:0/806151217 conn(0x55e3fa74d3c0 msgr2=0x55e3faaef940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:34.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.426+0000 7fa6e9df8740 1 -- 172.21.15.5:0/806151217 shutdown_connections 2024-05-20T18:37:34.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.426+0000 7fa6e9df8740 1 -- 172.21.15.5:0/806151217 wait complete. 2024-05-20T18:37:34.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.427+0000 7fa6e9df8740 1 -- start start 2024-05-20T18:37:34.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.428+0000 7fa6e9df8740 1 Processor -- start 2024-05-20T18:37:34.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.429+0000 7fa6e9df8740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e3fa7303b0 0x55e3fab624d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:34.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.429+0000 7fa6e9df8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3fa738590 0x55e3fab3ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:34.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.429+0000 7fa6e9df8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3fa759270 0x55e3fab40290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:34.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.429+0000 7fa6e9df8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e3fab50a70 con 0x55e3fa738590 2024-05-20T18:37:34.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.429+0000 7fa6e9df8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e3fab0cef0 con 0x55e3fa759270 2024-05-20T18:37:34.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.429+0000 7fa6e9df8740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55e3fab51030 con 0x55e3fa7303b0 2024-05-20T18:37:34.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.429+0000 7fa6e9df8740 10 client.0 _ll_register_callbacks cb 0x7fa6d80094f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:37:34.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.429+0000 7fa6d3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3fa738590 0x55e3fab3ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:34.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.429+0000 7fa6d3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3fa738590 0x55e3fab3ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34178/0 (socket says 172.21.15.5:34178) 2024-05-20T18:37:34.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.429+0000 7fa6d3fff640 1 -- 172.21.15.5:0/3199483866 learned_addr learned my addr 172.21.15.5:0/3199483866 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:34.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.429+0000 7fa6e0c0f640 1 --2- 172.21.15.5:0/3199483866 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e3fa7303b0 0x55e3fab624d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:34.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.429+0000 7fa6e1410640 1 --2- 172.21.15.5:0/3199483866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3fa759270 0x55e3fab40290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:34.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.429+0000 7fa6d3fff640 1 -- 172.21.15.5:0/3199483866 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e3fa7303b0 msgr2=0x55e3fab624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:34.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.429+0000 7fa6d3fff640 1 --2- 172.21.15.5:0/3199483866 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e3fa7303b0 0x55e3fab624d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.429+0000 7fa6d3fff640 1 -- 172.21.15.5:0/3199483866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3fa759270 msgr2=0x55e3fab40290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:34.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.429+0000 7fa6d3fff640 1 --2- 172.21.15.5:0/3199483866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3fa759270 0x55e3fab40290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.429+0000 7fa6d3fff640 1 -- 172.21.15.5:0/3199483866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3fab62a10 con 0x55e3fa738590 2024-05-20T18:37:34.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.429+0000 7fa6e0c0f640 1 --2- 172.21.15.5:0/3199483866 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e3fa7303b0 0x55e3fab624d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:34.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.429+0000 7fa6e1410640 1 --2- 172.21.15.5:0/3199483866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3fa759270 0x55e3fab40290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:34.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.430+0000 7fa6d3fff640 1 --2- 172.21.15.5:0/3199483866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3fa738590 0x55e3fab3ed90 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc009870 tx=0x7fa6cc02c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652571eee621aa6f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:34.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.430+0000 7fa6bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:37:34.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.430+0000 7fa6bb7fe640 1 -- 172.21.15.5:0/3199483866 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc051030 con 0x55e3fa738590 2024-05-20T18:37:34.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.430+0000 7fa6bb7fe640 1 -- 172.21.15.5:0/3199483866 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7fa6cc03e5e0 con 0x55e3fa738590 2024-05-20T18:37:34.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.430+0000 7fa6e9df8740 1 -- 172.21.15.5:0/3199483866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6d400ea60 con 0x55e3fa738590 2024-05-20T18:37:34.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.430+0000 7fa6bb7fe640 1 -- 172.21.15.5:0/3199483866 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7fa6cc05d050 con 0x55e3fa738590 2024-05-20T18:37:34.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.430+0000 7fa6e9df8740 10 client.9324 fetch_fsmap learned FSMap version 104 2024-05-20T18:37:34.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.430+0000 7fa6e9df8740 1 -- 172.21.15.5:0/3199483866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) -- 0x55e3fab62c20 con 0x55e3fa738590 2024-05-20T18:37:34.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.430+0000 7fa6bb7fe640 1 -- 172.21.15.5:0/3199483866 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6d400ea60 con 0x55e3fa738590 2024-05-20T18:37:34.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.430+0000 7fa6bb7fe640 1 -- 172.21.15.5:0/3199483866 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 104) ==== 3494+0+0 (secure 0 0 0) 0x7fa6cc060020 con 0x55e3fa738590 2024-05-20T18:37:34.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.430+0000 7fa6e9df8740 10 client.9324 fetch_fsmap finished waiting for FSMap version 104 2024-05-20T18:37:34.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.430+0000 7fa6e9df8740 10 client.9324 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-20T18:37:34.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.430+0000 7fa6e9df8740 20 client.9324 populate_metadata read hostname 'smithi005' 2024-05-20T18:37:34.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.430+0000 7fa6e9df8740 1 --2- 172.21.15.5:0/3199483866 >> [v2:172.21.15.5:6835/4273429943,v1:172.21.15.5:6837/4273429943] conn(0x55e3faaf39f0 0x55e3faaf3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:34.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.430+0000 7fa6e9df8740 4 client.9324 mds_command: new command op to 9012 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T18:37:34.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.430+0000 7fa6e9df8740 1 -- 172.21.15.5:0/3199483866 --> [v2:172.21.15.5:6835/4273429943,v1:172.21.15.5:6837/4273429943] -- command(tid 0: {"prefix": "status"}) -- 0x55e3faa64f50 con 0x55e3faaf39f0 2024-05-20T18:37:34.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.430+0000 7fa6e0c0f640 1 --2- 172.21.15.5:0/3199483866 >> [v2:172.21.15.5:6835/4273429943,v1:172.21.15.5:6837/4273429943] conn(0x55e3faaf39f0 0x55e3faaf3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:34.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.431+0000 7fa6bb7fe640 10 client.9324 ms_handle_connect on v2:172.21.15.5:6835/4273429943 2024-05-20T18:37:34.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.431+0000 7fa6e0c0f640 1 --2- 172.21.15.5:0/3199483866 >> [v2:172.21.15.5:6835/4273429943,v1:172.21.15.5:6837/4273429943] conn(0x55e3faaf39f0 0x55e3faaf3e00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7fd7d5ae66f9dd1 server_cookie=ae9de1d1c4343b5c in_seq=0 out_seq=0 2024-05-20T18:37:34.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.431+0000 7fa6bb7fe640 1 -- 172.21.15.5:0/3199483866 <== mds.0 v2:172.21.15.5:6835/4273429943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55e3faa64f50 con 0x55e3faaf39f0 2024-05-20T18:37:34.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.431+0000 7fa6bb7fe640 10 client.9324 handle_command_reply: tid=0 2024-05-20T18:37:34.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.431+0000 7fa6e9df8740 1 client.9324 shutdown 2024-05-20T18:37:34.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.431+0000 7fa6e9df8740 1 -- 172.21.15.5:0/3199483866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3fa738590 msgr2=0x55e3fab3ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:34.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.431+0000 7fa6e9df8740 1 --2- 172.21.15.5:0/3199483866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3fa738590 0x55e3fab3ed90 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc009870 tx=0x7fa6cc02c070 comp rx=0 tx=0).stop 2024-05-20T18:37:34.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.431+0000 7fa6e9df8740 1 -- 172.21.15.5:0/3199483866 shutdown_connections 2024-05-20T18:37:34.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.431+0000 7fa6e9df8740 1 --2- 172.21.15.5:0/3199483866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e3fa759270 0x55e3fab40290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.431+0000 7fa6e9df8740 1 --2- 172.21.15.5:0/3199483866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3fa738590 0x55e3fab3ed90 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.431+0000 7fa6e9df8740 1 --2- 172.21.15.5:0/3199483866 >> [v2:172.21.15.5:6835/4273429943,v1:172.21.15.5:6837/4273429943] conn(0x55e3faaf39f0 0x55e3faaf3e00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.431+0000 7fa6e9df8740 1 --2- 172.21.15.5:0/3199483866 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e3fa7303b0 0x55e3fab624d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.431+0000 7fa6e9df8740 1 -- 172.21.15.5:0/3199483866 >> 172.21.15.5:0/3199483866 conn(0x55e3fa74d3c0 msgr2=0x55e3faaf6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:34.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.431+0000 7fa6bb7fe640 0 client.9324 ms_handle_reset on v2:172.21.15.5:6835/4273429943 2024-05-20T18:37:34.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.432+0000 7fa6e9df8740 1 -- 172.21.15.5:0/3199483866 shutdown_connections 2024-05-20T18:37:34.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.432+0000 7fa6e9df8740 1 -- 172.21.15.5:0/3199483866 wait complete. 2024-05-20T18:37:34.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.432+0000 7fa6e9df8740 20 client.9324 trim_cache size 0 max 16384 2024-05-20T18:37:34.440 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:37:34.440 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "399ab250-fd3c-47fa-a1c8-25e8149c9cac", 2024-05-20T18:37:34.440 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-20T18:37:34.441 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 9012, 2024-05-20T18:37:34.441 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-20T18:37:34.441 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-20T18:37:34.441 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-20T18:37:34.441 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 4.5201082980000002, 2024-05-20T18:37:34.441 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 104, 2024-05-20T18:37:34.441 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 120, 2024-05-20T18:37:34.441 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 120, 2024-05-20T18:37:34.441 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 24.394502489000001 2024-05-20T18:37:34.441 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:37:34.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.432+0000 7fa6e2e73640 1 -- 172.21.15.5:0/3516931934 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa6c806be60 msgr2=0x7fa6c806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:34.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.432+0000 7fa6e2e73640 1 --2- 172.21.15.5:0/3516931934 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa6c806be60 0x7fa6c806e370 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc034000 tx=0x7fa6cc02e000 comp rx=0 tx=0).stop 2024-05-20T18:37:34.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.432+0000 7fa6e2e73640 1 -- 172.21.15.5:0/3516931934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6dc0b6680 msgr2=0x7fa6dc1550a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:34.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.432+0000 7fa6e2e73640 1 --2- 172.21.15.5:0/3516931934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6dc0b6680 0x7fa6dc1550a0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800e8d0 tx=0x7fa6d800ed90 comp rx=0 tx=0).stop 2024-05-20T18:37:34.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.432+0000 7fa6e2e73640 1 -- 172.21.15.5:0/3516931934 shutdown_connections 2024-05-20T18:37:34.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.432+0000 7fa6e2e73640 1 --2- 172.21.15.5:0/3516931934 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa6c806be60 0x7fa6c806e370 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.432+0000 7fa6e2e73640 1 --2- 172.21.15.5:0/3516931934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6dc163020 0x7fa6dc15dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.432+0000 7fa6e2e73640 1 --2- 172.21.15.5:0/3516931934 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6dc0cf250 0x7fa6dc15d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.432+0000 7fa6e2e73640 1 --2- 172.21.15.5:0/3516931934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6dc0b6680 0x7fa6dc1550a0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.432+0000 7fa6e2e73640 1 -- 172.21.15.5:0/3516931934 >> 172.21.15.5:0/3516931934 conn(0x7fa6dc0d1860 msgr2=0x7fa6dc0b89b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:34.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.432+0000 7fa6e2e73640 1 -- 172.21.15.5:0/3516931934 shutdown_connections 2024-05-20T18:37:34.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.432+0000 7fa6e2e73640 1 -- 172.21.15.5:0/3516931934 wait complete. 2024-05-20T18:37:34.445 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-20T18:37:34.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.558+0000 7f62f5502640 1 Processor -- start 2024-05-20T18:37:34.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.558+0000 7f62f5502640 1 -- start start 2024-05-20T18:37:34.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.559+0000 7f62f5502640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62f015d060 0x7f62f00b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:34.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.559+0000 7f62f5502640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f62f00b5720 0x7f62f006f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:34.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.559+0000 7f62f5502640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62f00b5b30 con 0x7f62f00b47b0 2024-05-20T18:37:34.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.559+0000 7f62f5502640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f62f00b39f0 con 0x7f62f00b5720 2024-05-20T18:37:34.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.559+0000 7f62f5502640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f62f00b3cd0 con 0x7f62f015d060 2024-05-20T18:37:34.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.559+0000 7f62ef7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f62f00b5720 0x7f62f006f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:34.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.559+0000 7f62eeffd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f62f00b47b0 0x7f62f00afad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38682/0 (socket says 172.21.15.5:38682) 2024-05-20T18:37:34.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.559+0000 7f62eeffd640 1 -- 172.21.15.5:0/3200357943 learned_addr learned my addr 172.21.15.5:0/3200357943 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:34.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.560+0000 7f62ee7fc640 1 --2- 172.21.15.5:0/3200357943 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62f015d060 0x7f62f00b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:34.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.560+0000 7f62ef7fe640 1 -- 172.21.15.5:0/3200357943 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62f015d060 msgr2=0x7f62f00b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:34.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.560+0000 7f62ef7fe640 1 --2- 172.21.15.5:0/3200357943 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62f015d060 0x7f62f00b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.560+0000 7f62ef7fe640 1 -- 172.21.15.5:0/3200357943 >> v1:172.21.15.5:6789/0 conn(0x7f62f00b47b0 legacy=0x7f62f00afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:34.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.560+0000 7f62ef7fe640 1 -- 172.21.15.5:0/3200357943 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f00b3fb0 con 0x7f62f00b5720 2024-05-20T18:37:34.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.560+0000 7f62edffb640 1 -- 172.21.15.5:0/3200357943 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3612128985 0 0) 0x7f62e0001110 con 0x7f62f00b47b0 2024-05-20T18:37:34.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.560+0000 7f62ee7fc640 1 --2- 172.21.15.5:0/3200357943 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62f015d060 0x7f62f00b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:34.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.561+0000 7f62ef7fe640 1 --2- 172.21.15.5:0/3200357943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f00b5720 0x7f62f006f3d0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f62e800bfa0 tx=0x7f62e802c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b1a10ba2c215af server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:34.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.561+0000 7f62edffb640 1 -- 172.21.15.5:0/3200357943 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e8039070 con 0x7f62f00b5720 2024-05-20T18:37:34.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.561+0000 7f62edffb640 1 -- 172.21.15.5:0/3200357943 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62e8009540 con 0x7f62f00b5720 2024-05-20T18:37:34.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.561+0000 7f62edffb640 1 -- 172.21.15.5:0/3200357943 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e8009910 con 0x7f62f00b5720 2024-05-20T18:37:34.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.561+0000 7f62f5502640 1 -- 172.21.15.5:0/3200357943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f00b5720 msgr2=0x7f62f006f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:34.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.561+0000 7f62f5502640 1 --2- 172.21.15.5:0/3200357943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f00b5720 0x7f62f006f3d0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f62e800bfa0 tx=0x7f62e802c860 comp rx=0 tx=0).stop 2024-05-20T18:37:34.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.561+0000 7f62f5502640 1 -- 172.21.15.5:0/3200357943 shutdown_connections 2024-05-20T18:37:34.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.561+0000 7f62f5502640 1 --2- 172.21.15.5:0/3200357943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f00b5720 0x7f62f006f3d0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.561+0000 7f62f5502640 1 --2- 172.21.15.5:0/3200357943 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62f015d060 0x7f62f00b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.561+0000 7f62f5502640 1 -- 172.21.15.5:0/3200357943 >> 172.21.15.5:0/3200357943 conn(0x7f62f00cd2a0 msgr2=0x7f62f00cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:34.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.562+0000 7f62f5502640 1 -- 172.21.15.5:0/3200357943 shutdown_connections 2024-05-20T18:37:34.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.562+0000 7f62f5502640 1 -- 172.21.15.5:0/3200357943 wait complete. 2024-05-20T18:37:34.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.563+0000 7f62f5502640 1 Processor -- start 2024-05-20T18:37:34.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.564+0000 7f62f5502640 1 -- start start 2024-05-20T18:37:34.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.565+0000 7f62f5502640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62f00b5720 0x7f62f0150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:34.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.565+0000 7f62f5502640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62f015d060 0x7f62f00cb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:34.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.565+0000 7f62f5502640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f00cb7d0 0x7f62f00b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:34.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.565+0000 7f62f5502640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f62f0159de0 con 0x7f62f00cb7d0 2024-05-20T18:37:34.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.565+0000 7f62f5502640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f62f0159c60 con 0x7f62f00b5720 2024-05-20T18:37:34.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.565+0000 7f62f5502640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f62f006f910 con 0x7f62f015d060 2024-05-20T18:37:34.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.565+0000 7f62ef7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f00cb7d0 0x7f62f00b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:34.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.565+0000 7f62ef7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f00cb7d0 0x7f62f00b0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34198/0 (socket says 172.21.15.5:34198) 2024-05-20T18:37:34.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.565+0000 7f62ef7fe640 1 -- 172.21.15.5:0/2991729957 learned_addr learned my addr 172.21.15.5:0/2991729957 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:34.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.565+0000 7f62eeffd640 1 --2- 172.21.15.5:0/2991729957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62f00b5720 0x7f62f0150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:34.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.565+0000 7f62ee7fc640 1 --2- 172.21.15.5:0/2991729957 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62f015d060 0x7f62f00cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:34.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.565+0000 7f62ef7fe640 1 -- 172.21.15.5:0/2991729957 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62f015d060 msgr2=0x7f62f00cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:34.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.565+0000 7f62ef7fe640 1 --2- 172.21.15.5:0/2991729957 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62f015d060 0x7f62f00cb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.565+0000 7f62ef7fe640 1 -- 172.21.15.5:0/2991729957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62f00b5720 msgr2=0x7f62f0150bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:34.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.565+0000 7f62ef7fe640 1 --2- 172.21.15.5:0/2991729957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62f00b5720 0x7f62f0150bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.565+0000 7f62ef7fe640 1 -- 172.21.15.5:0/2991729957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f015d6c0 con 0x7f62f00cb7d0 2024-05-20T18:37:34.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.566+0000 7f62ef7fe640 1 --2- 172.21.15.5:0/2991729957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f00cb7d0 0x7f62f00b0320 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f62e800bfa0 tx=0x7f62e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:34.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.566+0000 7f62d77fe640 1 -- 172.21.15.5:0/2991729957 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e8039040 con 0x7f62f00cb7d0 2024-05-20T18:37:34.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.566+0000 7f62d77fe640 1 -- 172.21.15.5:0/2991729957 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62e80092b0 con 0x7f62f00cb7d0 2024-05-20T18:37:34.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.566+0000 7f62f5502640 1 -- 172.21.15.5:0/2991729957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f015d9a0 con 0x7f62f00cb7d0 2024-05-20T18:37:34.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.566+0000 7f62d77fe640 1 -- 172.21.15.5:0/2991729957 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e80348b0 con 0x7f62f00cb7d0 2024-05-20T18:37:34.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.566+0000 7f62f5502640 1 -- 172.21.15.5:0/2991729957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f015df60 con 0x7f62f00cb7d0 2024-05-20T18:37:34.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.567+0000 7f62f5502640 1 -- 172.21.15.5:0/2991729957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62f0075b00 con 0x7f62f00cb7d0 2024-05-20T18:37:34.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.567+0000 7f62d77fe640 1 -- 172.21.15.5:0/2991729957 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f62e8034a50 con 0x7f62f00cb7d0 2024-05-20T18:37:34.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.568+0000 7f62d77fe640 1 --2- 172.21.15.5:0/2991729957 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f62dc06be30 0x7f62dc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:34.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.568+0000 7f62d77fe640 1 -- 172.21.15.5:0/2991729957 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7f62e80abcd0 con 0x7f62f00cb7d0 2024-05-20T18:37:34.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.568+0000 7f62eeffd640 1 --2- 172.21.15.5:0/2991729957 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f62dc06be30 0x7f62dc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:34.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.569+0000 7f62eeffd640 1 --2- 172.21.15.5:0/2991729957 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f62dc06be30 0x7f62dc06e340 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f62e0007fd0 tx=0x7f62e0007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:34.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.573+0000 7f62d77fe640 1 -- 172.21.15.5:0/2991729957 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f62e8079e50 con 0x7f62f00cb7d0 2024-05-20T18:37:34.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.794+0000 7f62f5502640 1 -- 172.21.15.5:0/2991729957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f62f00b1e40 con 0x7f62f00cb7d0 2024-05-20T18:37:34.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.794+0000 7f62d77fe640 1 -- 172.21.15.5:0/2991729957 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v104) ==== 53+0+73 (secure 0 0 0) 0x7f62e807e150 con 0x7f62f00cb7d0 2024-05-20T18:37:34.796 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-20T18:37:34.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.796+0000 7f62f5502640 1 -- 172.21.15.5:0/2991729957 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f62dc06be30 msgr2=0x7f62dc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:34.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.796+0000 7f62f5502640 1 --2- 172.21.15.5:0/2991729957 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f62dc06be30 0x7f62dc06e340 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f62e0007fd0 tx=0x7f62e0007de0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.796+0000 7f62f5502640 1 -- 172.21.15.5:0/2991729957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f00cb7d0 msgr2=0x7f62f00b0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:34.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.796+0000 7f62f5502640 1 --2- 172.21.15.5:0/2991729957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f00cb7d0 0x7f62f00b0320 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f62e800bfa0 tx=0x7f62e8004a40 comp rx=0 tx=0).stop 2024-05-20T18:37:34.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.796+0000 7f62f5502640 1 -- 172.21.15.5:0/2991729957 shutdown_connections 2024-05-20T18:37:34.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.796+0000 7f62f5502640 1 --2- 172.21.15.5:0/2991729957 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f62dc06be30 0x7f62dc06e340 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.796+0000 7f62f5502640 1 --2- 172.21.15.5:0/2991729957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f00cb7d0 0x7f62f00b0320 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.796+0000 7f62f5502640 1 --2- 172.21.15.5:0/2991729957 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62f015d060 0x7f62f00cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.796+0000 7f62f5502640 1 --2- 172.21.15.5:0/2991729957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62f00b5720 0x7f62f0150bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:34.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.796+0000 7f62f5502640 1 -- 172.21.15.5:0/2991729957 >> 172.21.15.5:0/2991729957 conn(0x7f62f00cd2a0 msgr2=0x7f62f015a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:34.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.796+0000 7f62f5502640 1 -- 172.21.15.5:0/2991729957 shutdown_connections 2024-05-20T18:37:34.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:34.796+0000 7f62f5502640 1 -- 172.21.15.5:0/2991729957 wait complete. 2024-05-20T18:37:34.809 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-20T18:37:34.809 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-20T18:37:34.809 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-05-20T18:37:34.809 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-20T18:37:34.809 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-20T18:37:34.809 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-20T18:37:34.809 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-20T18:37:34.809 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-20T18:37:34.809 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:37:34.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:34.810 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-05-20T18:37:34.826 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T18:37:34.826 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T18:37:34.826 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T18:37:34.826 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:37:34.826 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-20T18:37:34.826 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:37:34.827 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:37:34.827 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-05-20T18:37:34.827 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-05-20T18:37:34.827 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:37:34.827 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-05-20T18:37:34.827 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-05-20T18:37:34.827 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-20T18:37:34.827 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:37:34.827 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-05-20T18:37:34.827 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-20T18:37:34.827 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42043sec preferred_lft 42043sec 2024-05-20T18:37:34.827 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-05-20T18:37:34.827 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:37:34.828 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:37:34.828 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:37:34.828 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-05-20T18:37:34.828 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-20T18:37:34.828 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:37:34.828 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-20T18:37:34.828 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:37:34.828 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::4474:eaff:fe07:4f9a/64 scope link 2024-05-20T18:37:34.828 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:37:34.828 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:37:34.828 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T18:37:34.828 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ce4:f4ff:fe5e:7906/64 scope link 2024-05-20T18:37:34.828 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:37:34.829 INFO:teuthology.orchestra.run.smithi005.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:37:34.829 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether da:26:04:0e:95:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T18:37:34.829 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::d826:4ff:fe0e:950a/64 scope link 2024-05-20T18:37:34.829 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:37:34.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:34.829 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-20T18:37:34.890 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-20T18:37:34.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:34.892 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-05-20T18:37:34.949 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-05-20T18:37:34.950 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-20T18:37:34.950 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-20T18:37:34.950 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-20T18:37:34.950 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-20T18:37:34.950 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:37:34.950 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-20T18:37:34.950 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-20T18:37:34.950 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-20T18:37:34.950 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-20T18:37:34.951 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-20T18:37:35.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:37:35.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:35.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:35.080 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-20T18:37:35.096 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-20T18:37:35.096 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-20T18:37:35.097 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:37:35.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:35.097 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-05-20T18:37:35.163 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:37:35.163 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-20T18:37:35.220 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:37:35.220 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:37:35.221 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:37:35.276 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-20T18:37:35.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:35.277 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:37:35.372 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:37:35.373 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:37:35.373 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:37:35.384 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:37:35.384 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:37:35.384 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:37:35.384 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:37:35.384 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:37:35.399 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:37:35.399 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:37:35.399 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:37:35.399 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:37:35.399 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:37:35.399 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790" 2024-05-20T18:37:35.400 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790:/ 2024-05-20T18:37:35.400 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=399ab250-fd3c-47fa-a1c8-25e8149c9cac" will pass to kernel 2024-05-20T18:37:35.400 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-20T18:37:35.400 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-05-20T18:37:35.400 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-20T18:37:35.400 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-05-20T18:37:35.400 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-20T18:37:35.401 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-20T18:37:35.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:35.402 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:37:35.427 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-20T18:37:35.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T18:37:35.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:35.525 INFO:teuthology.orchestra.run.smithi005.stdout:client7403 2024-05-20T18:37:35.525 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:37:35.525 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:37:35.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:35.542 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-20T18:37:35.604 INFO:teuthology.orchestra.run.smithi005.stdout:./ 2024-05-20T18:37:35.605 INFO:teuthology.orchestra.run.smithi005.stdout:./parent 2024-05-20T18:37:35.605 INFO:teuthology.orchestra.run.smithi005.stdout:./parent/flushed 2024-05-20T18:37:35.605 INFO:teuthology.orchestra.run.smithi005.stdout:./parent/flushed/charlie 2024-05-20T18:37:35.605 INFO:teuthology.orchestra.run.smithi005.stdout:./parent/flushed/alpha 2024-05-20T18:37:35.605 INFO:teuthology.orchestra.run.smithi005.stdout:./parent/unflushed 2024-05-20T18:37:35.605 INFO:teuthology.orchestra.run.smithi005.stdout:./parent/unflushed/jfile 2024-05-20T18:37:35.605 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:37:35.605 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:37:35.605 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:37:35.605 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:37:35.606 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-20T18:37:35.606 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:37:35.713 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-20T18:37:35.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:35.722 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:37:35.722 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:37:35.722 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:37:35.722 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:37:35.722 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-05-20T18:37:35.722 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:37:35.830 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627776 2024-05-20T18:37:35.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:35.840 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:37:35.840 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:37:35.840 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:37:35.840 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:37:35.840 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-05-20T18:37:35.840 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:37:35.949 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627777 2024-05-20T18:37:35.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:35.957 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:37:35.958 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:37:35.958 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:37:35.958 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:37:35.958 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-05-20T18:37:35.958 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:37:36.066 INFO:teuthology.orchestra.run.smithi005.stdout:1099511628280 2024-05-20T18:37:36.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:36.075 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:37:36.075 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:37:36.075 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:37:36.076 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:37:36.076 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-05-20T18:37:36.076 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:37:36.183 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627778 2024-05-20T18:37:36.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:36.192 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:37:36.192 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:37:36.192 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:37:36.192 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:37:36.192 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-05-20T18:37:36.192 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:37:36.300 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627779 2024-05-20T18:37:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:36.309 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:37:36.310 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:37:36.310 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:37:36.310 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:37:36.310 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-05-20T18:37:36.310 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:37:36.418 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627780 2024-05-20T18:37:36.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:36.427 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:37:36.427 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:37:36.484 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:37:36.484 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:37:36.540 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:37:36.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.693+0000 7f76144bc640 1 Processor -- start 2024-05-20T18:37:36.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.693+0000 7f76144bc640 1 -- start start 2024-05-20T18:37:36.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.694+0000 7f76144bc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f760c0cd1b0 0x7f760c15f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:36.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.694+0000 7f76144bc640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f760c070850 0x7f760c070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:36.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.694+0000 7f76144bc640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f760c0711a0 con 0x7f760c071630 2024-05-20T18:37:36.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.694+0000 7f76144bc640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f760c15f790 con 0x7f760c070850 2024-05-20T18:37:36.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.694+0000 7f76144bc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f760c06e590 con 0x7f760c0cd1b0 2024-05-20T18:37:36.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.694+0000 7f7612258640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f760c071630 0x7f760c06d7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:32826/0 (socket says 172.21.15.5:32826) 2024-05-20T18:37:36.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.694+0000 7f7612258640 1 -- 172.21.15.5:0/3227507276 learned_addr learned my addr 172.21.15.5:0/3227507276 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:36.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.695+0000 7f7611256640 1 -- 172.21.15.5:0/3227507276 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4053462657 0 0) 0x7f7608001110 con 0x7f760c071630 2024-05-20T18:37:36.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.695+0000 7f7611256640 1 -- 172.21.15.5:0/3227507276 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75f8003520 con 0x7f760c071630 2024-05-20T18:37:36.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.695+0000 7f7611a57640 1 --2- 172.21.15.5:0/3227507276 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f760c0cd1b0 0x7f760c15f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:36.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.695+0000 7f7612a59640 1 --2- 172.21.15.5:0/3227507276 >> v2:172.21.15.63:3300/0 conn(0x7f760c070850 0x7f760c070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:36.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.695+0000 7f7611256640 1 -- 172.21.15.5:0/3227507276 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2878278214 0 0) 0x7f75f8003520 con 0x7f760c071630 2024-05-20T18:37:36.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.695+0000 7f7611256640 1 -- 172.21.15.5:0/3227507276 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f760c0cd1b0 msgr2=0x7f760c15f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:36.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.695+0000 7f7611256640 1 --2- 172.21.15.5:0/3227507276 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f760c0cd1b0 0x7f760c15f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:36.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.695+0000 7f7611256640 1 -- 172.21.15.5:0/3227507276 >> v2:172.21.15.63:3300/0 conn(0x7f760c070850 msgr2=0x7f760c070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:36.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.695+0000 7f7611256640 1 --2- 172.21.15.5:0/3227507276 >> v2:172.21.15.63:3300/0 conn(0x7f760c070850 0x7f760c070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:36.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.695+0000 7f7611256640 1 -- 172.21.15.5:0/3227507276 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f760c0ce2b0 con 0x7f760c071630 2024-05-20T18:37:36.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.695+0000 7f7611256640 1 -- 172.21.15.5:0/3227507276 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f7608001840 con 0x7f760c071630 2024-05-20T18:37:36.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.695+0000 7f7611a57640 1 --2- 172.21.15.5:0/3227507276 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f760c0cd1b0 0x7f760c15f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:36.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.695+0000 7f7611256640 1 -- 172.21.15.5:0/3227507276 >> v1:172.21.15.5:6789/0 conn(0x7f760c071630 legacy=0x7f760c06d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:36.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.695+0000 7f7611256640 1 --2- 172.21.15.5:0/3227507276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f80039b0 0x7f75f8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:36.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.695+0000 7f7611256640 1 --2- 172.21.15.5:0/3227507276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75f8004800 0x7f75f8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:36.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.695+0000 7f7611256640 1 --2- 172.21.15.5:0/3227507276 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f760c0cd1b0 0x7f75f8005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:36.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.695+0000 7f7611256640 1 -- 172.21.15.5:0/3227507276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f760c15f790 con 0x7f75f80039b0 2024-05-20T18:37:36.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.695+0000 7f7611256640 1 -- 172.21.15.5:0/3227507276 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f75f800ba70 con 0x7f75f8004800 2024-05-20T18:37:36.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.695+0000 7f7611256640 1 -- 172.21.15.5:0/3227507276 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f75f800bc70 con 0x7f760c0cd1b0 2024-05-20T18:37:36.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.695+0000 7f7612258640 1 --2- 172.21.15.5:0/3227507276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f80039b0 0x7f75f8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:36.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.695+0000 7f7611a57640 1 --2- 172.21.15.5:0/3227507276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75f8004800 0x7f75f8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:36.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.695+0000 7f7612a59640 1 --2- 172.21.15.5:0/3227507276 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f760c0cd1b0 0x7f75f8005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:36.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.696+0000 7f7612258640 1 -- 172.21.15.5:0/3227507276 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f760c0cd1b0 msgr2=0x7f75f8005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:36.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.696+0000 7f7612258640 1 --2- 172.21.15.5:0/3227507276 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f760c0cd1b0 0x7f75f8005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:36.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.696+0000 7f7612258640 1 -- 172.21.15.5:0/3227507276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75f8004800 msgr2=0x7f75f8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:36.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.696+0000 7f7612258640 1 --2- 172.21.15.5:0/3227507276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75f8004800 0x7f75f8004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:36.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.696+0000 7f7612258640 1 -- 172.21.15.5:0/3227507276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f8003520 con 0x7f75f80039b0 2024-05-20T18:37:36.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.696+0000 7f7612258640 1 --2- 172.21.15.5:0/3227507276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f80039b0 0x7f75f8003e20 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f760800c3e0 tx=0x7f75f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1f34cdd70d0be6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:36.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.696+0000 7f7611256640 1 -- 172.21.15.5:0/3227507276 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7608032020 con 0x7f75f80039b0 2024-05-20T18:37:36.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.697+0000 7f7611256640 1 -- 172.21.15.5:0/3227507276 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7608005ad0 con 0x7f75f80039b0 2024-05-20T18:37:36.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.697+0000 7f7611256640 1 -- 172.21.15.5:0/3227507276 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760800c0e0 con 0x7f75f80039b0 2024-05-20T18:37:36.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.697+0000 7f76144bc640 1 -- 172.21.15.5:0/3227507276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f80039b0 msgr2=0x7f75f8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:36.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.697+0000 7f76144bc640 1 --2- 172.21.15.5:0/3227507276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f80039b0 0x7f75f8003e20 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f760800c3e0 tx=0x7f75f8005830 comp rx=0 tx=0).stop 2024-05-20T18:37:36.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.697+0000 7f7612a59640 1 -- 172.21.15.5:0/3227507276 reap_dead start 2024-05-20T18:37:36.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.697+0000 7f76144bc640 1 -- 172.21.15.5:0/3227507276 shutdown_connections 2024-05-20T18:37:36.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.697+0000 7f76144bc640 1 -- 172.21.15.5:0/3227507276 >> 172.21.15.5:0/3227507276 conn(0x7f760c0d29d0 msgr2=0x7f760c0d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:36.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.697+0000 7f76144bc640 1 -- 172.21.15.5:0/3227507276 shutdown_connections 2024-05-20T18:37:36.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.697+0000 7f76144bc640 1 -- 172.21.15.5:0/3227507276 wait complete. 2024-05-20T18:37:36.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.699+0000 7f76144bc640 1 Processor -- start 2024-05-20T18:37:36.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.699+0000 7f76144bc640 1 -- start start 2024-05-20T18:37:36.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.700+0000 7f76144bc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f80039b0 0x7f760c162570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:36.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.700+0000 7f76144bc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f760c156c60 0x7f760c15f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:36.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.700+0000 7f76144bc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f760c157070 0x7f760c15fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:36.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.700+0000 7f76144bc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f760c06ded0 con 0x7f75f80039b0 2024-05-20T18:37:36.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.700+0000 7f76144bc640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f760c06cfc0 con 0x7f760c157070 2024-05-20T18:37:36.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.700+0000 7f76144bc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f760c06d2a0 con 0x7f760c156c60 2024-05-20T18:37:36.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.701+0000 7f7612258640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f80039b0 0x7f760c162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:36.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.701+0000 7f7612258640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f80039b0 0x7f760c162570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35278/0 (socket says 172.21.15.5:35278) 2024-05-20T18:37:36.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.701+0000 7f7612258640 1 -- 172.21.15.5:0/2795499641 learned_addr learned my addr 172.21.15.5:0/2795499641 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:36.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.701+0000 7f7611a57640 1 --2- 172.21.15.5:0/2795499641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f760c156c60 0x7f760c15f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:36.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.701+0000 7f7612a59640 1 --2- 172.21.15.5:0/2795499641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f760c157070 0x7f760c15fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:36.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.701+0000 7f7612258640 1 -- 172.21.15.5:0/2795499641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f760c156c60 msgr2=0x7f760c15f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:36.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.701+0000 7f7612258640 1 --2- 172.21.15.5:0/2795499641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f760c156c60 0x7f760c15f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:36.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.701+0000 7f7612258640 1 -- 172.21.15.5:0/2795499641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f760c157070 msgr2=0x7f760c15fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:36.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.701+0000 7f7612258640 1 --2- 172.21.15.5:0/2795499641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f760c157070 0x7f760c15fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:36.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.701+0000 7f7612258640 1 -- 172.21.15.5:0/2795499641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f760c0b1600 con 0x7f75f80039b0 2024-05-20T18:37:36.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.701+0000 7f7612a59640 1 --2- 172.21.15.5:0/2795499641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f760c157070 0x7f760c15fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:36.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.701+0000 7f7612258640 1 --2- 172.21.15.5:0/2795499641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f80039b0 0x7f760c162570 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f760800c3e0 tx=0x7f760c0ce480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:36.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.701+0000 7f7602ffd640 1 -- 172.21.15.5:0/2795499641 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7608032030 con 0x7f75f80039b0 2024-05-20T18:37:36.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.701+0000 7f7602ffd640 1 -- 172.21.15.5:0/2795499641 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7608004bf0 con 0x7f75f80039b0 2024-05-20T18:37:36.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.701+0000 7f76144bc640 1 -- 172.21.15.5:0/2795499641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f760c0b18e0 con 0x7f75f80039b0 2024-05-20T18:37:36.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.701+0000 7f7602ffd640 1 -- 172.21.15.5:0/2795499641 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7608007070 con 0x7f75f80039b0 2024-05-20T18:37:36.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.701+0000 7f76144bc640 1 -- 172.21.15.5:0/2795499641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f760c0b6590 con 0x7f75f80039b0 2024-05-20T18:37:36.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.702+0000 7f76144bc640 1 -- 172.21.15.5:0/2795499641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f760c0b22a0 con 0x7f75f80039b0 2024-05-20T18:37:36.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.703+0000 7f7602ffd640 1 -- 172.21.15.5:0/2795499641 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7608035020 con 0x7f75f80039b0 2024-05-20T18:37:36.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.704+0000 7f7602ffd640 1 --2- 172.21.15.5:0/2795499641 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f75f006be30 0x7f75f006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:36.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.704+0000 7f7602ffd640 1 -- 172.21.15.5:0/2795499641 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7f76080ab800 con 0x7f75f80039b0 2024-05-20T18:37:36.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.704+0000 7f7611a57640 1 --2- 172.21.15.5:0/2795499641 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f75f006be30 0x7f75f006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:36.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.704+0000 7f7611a57640 1 --2- 172.21.15.5:0/2795499641 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f75f006be30 0x7f75f006e340 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f760c15e3e0 tx=0x7f75fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:36.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.708+0000 7f7602ffd640 1 -- 172.21.15.5:0/2795499641 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f760c0b22a0 con 0x7f75f80039b0 2024-05-20T18:37:36.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.897+0000 7f76144bc640 1 -- 172.21.15.5:0/2795499641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f760c075b00 con 0x7f75f80039b0 2024-05-20T18:37:36.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.898+0000 7f7602ffd640 1 -- 172.21.15.5:0/2795499641 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) ==== 74+0+16961 (secure 0 0 0) 0x7f760c0711a0 con 0x7f75f80039b0 2024-05-20T18:37:36.900 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:36.900 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":120,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:37:28.901970+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:37:12.797471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-20T18:37:13.346156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6835/2352520472":"2024-05-21T18:37:28.901938+0000","172.21.15.63:6833/2352520472":"2024-05-21T18:37:28.901938+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:37:36.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.900+0000 7f76144bc640 1 -- 172.21.15.5:0/2795499641 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f75f006be30 msgr2=0x7f75f006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:36.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.900+0000 7f76144bc640 1 --2- 172.21.15.5:0/2795499641 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f75f006be30 0x7f75f006e340 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f760c15e3e0 tx=0x7f75fc008040 comp rx=0 tx=0).stop 2024-05-20T18:37:36.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.900+0000 7f76144bc640 1 -- 172.21.15.5:0/2795499641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f80039b0 msgr2=0x7f760c162570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:36.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.900+0000 7f76144bc640 1 --2- 172.21.15.5:0/2795499641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f80039b0 0x7f760c162570 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f760800c3e0 tx=0x7f760c0ce480 comp rx=0 tx=0).stop 2024-05-20T18:37:36.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.900+0000 7f76144bc640 1 -- 172.21.15.5:0/2795499641 shutdown_connections 2024-05-20T18:37:36.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.900+0000 7f76144bc640 1 --2- 172.21.15.5:0/2795499641 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f75f006be30 0x7f75f006e340 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:36.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.900+0000 7f76144bc640 1 --2- 172.21.15.5:0/2795499641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f760c157070 0x7f760c15fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:36.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.900+0000 7f76144bc640 1 --2- 172.21.15.5:0/2795499641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f760c156c60 0x7f760c15f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:36.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.900+0000 7f76144bc640 1 --2- 172.21.15.5:0/2795499641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f80039b0 0x7f760c162570 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:36.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.900+0000 7f76144bc640 1 -- 172.21.15.5:0/2795499641 >> 172.21.15.5:0/2795499641 conn(0x7f760c0d29d0 msgr2=0x7f760c0b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:36.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.901+0000 7f76144bc640 1 -- 172.21.15.5:0/2795499641 shutdown_connections 2024-05-20T18:37:36.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:36.901+0000 7f76144bc640 1 -- 172.21.15.5:0/2795499641 wait complete. 2024-05-20T18:37:36.913 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:37:12.797471+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:37:13.346156+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:37:36.914 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:37:36.971 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-20T18:37:36.970+0000 7f2481efd640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T18:37:37.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.030+0000 7f03c3fe0640 1 Processor -- start 2024-05-20T18:37:37.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.031+0000 7f03c3fe0640 1 -- start start 2024-05-20T18:37:37.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.032+0000 7f03c3fe0640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f03bc06d3d0 0x7f03bc06d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:37.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.032+0000 7f03c3fe0640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03bc06dd20 con 0x7f03bc162820 2024-05-20T18:37:37.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.032+0000 7f03c3fe0640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03bc0d0630 con 0x7f03bc0b8190 2024-05-20T18:37:37.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.032+0000 7f03c3fe0640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f03bc0d39c0 con 0x7f03bc06d3d0 2024-05-20T18:37:37.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.032+0000 7f03c1d7c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f03bc162820 0x7f03bc0b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:32842/0 (socket says 172.21.15.5:32842) 2024-05-20T18:37:37.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.032+0000 7f03c1d7c640 1 -- 172.21.15.5:0/641779407 learned_addr learned my addr 172.21.15.5:0/641779407 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:37.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.032+0000 7f03c157b640 1 --2- 172.21.15.5:0/641779407 >> v2:172.21.15.63:3300/0 conn(0x7f03bc06d3d0 0x7f03bc06d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:37.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.032+0000 7f03c0d7a640 1 -- 172.21.15.5:0/641779407 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3153469469 0 0) 0x7f03b8001110 con 0x7f03bc162820 2024-05-20T18:37:37.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.032+0000 7f03c0d7a640 1 -- 172.21.15.5:0/641779407 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f03a8003520 con 0x7f03bc162820 2024-05-20T18:37:37.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.033+0000 7f03c0d7a640 1 -- 172.21.15.5:0/641779407 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3745243432 0 0) 0x7f03a8003520 con 0x7f03bc162820 2024-05-20T18:37:37.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.033+0000 7f03c0d7a640 1 -- 172.21.15.5:0/641779407 >> v2:172.21.15.63:3300/0 conn(0x7f03bc06d3d0 msgr2=0x7f03bc06d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:37.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.033+0000 7f03c0d7a640 1 --2- 172.21.15.5:0/641779407 >> v2:172.21.15.63:3300/0 conn(0x7f03bc06d3d0 0x7f03bc06d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:37.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.033+0000 7f03c0d7a640 1 -- 172.21.15.5:0/641779407 >> v1:172.21.15.63:6789/0 conn(0x7f03bc0b8190 legacy=0x7f03bc0b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:37.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.033+0000 7f03c0d7a640 1 -- 172.21.15.5:0/641779407 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03bc0d3ca0 con 0x7f03bc162820 2024-05-20T18:37:37.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.033+0000 7f03c0d7a640 1 -- 172.21.15.5:0/641779407 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f03b8001840 con 0x7f03bc162820 2024-05-20T18:37:37.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.033+0000 7f03c0d7a640 1 -- 172.21.15.5:0/641779407 >> v1:172.21.15.5:6789/0 conn(0x7f03bc162820 legacy=0x7f03bc0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:37.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.033+0000 7f03c0d7a640 1 --2- 172.21.15.5:0/641779407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03a80039b0 0x7f03a8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:37.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.033+0000 7f03c0d7a640 1 --2- 172.21.15.5:0/641779407 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03a8004800 0x7f03a8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:37.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.033+0000 7f03c0d7a640 1 --2- 172.21.15.5:0/641779407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03a80051c0 0x7f03a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:37.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.033+0000 7f03c0d7a640 1 -- 172.21.15.5:0/641779407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc0d39c0 con 0x7f03a80051c0 2024-05-20T18:37:37.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.033+0000 7f03c0d7a640 1 -- 172.21.15.5:0/641779407 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f03a800c160 con 0x7f03a80039b0 2024-05-20T18:37:37.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.033+0000 7f03c0d7a640 1 -- 172.21.15.5:0/641779407 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f03a800c360 con 0x7f03a8004800 2024-05-20T18:37:37.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.033+0000 7f03c257d640 1 --2- 172.21.15.5:0/641779407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03a80051c0 0x7f03a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:37.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.033+0000 7f03c1d7c640 1 --2- 172.21.15.5:0/641779407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03a80039b0 0x7f03a8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:37.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.033+0000 7f03c157b640 1 --2- 172.21.15.5:0/641779407 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03a8004800 0x7f03a8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:37.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.033+0000 7f03c257d640 1 -- 172.21.15.5:0/641779407 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03a8004800 msgr2=0x7f03a8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:37.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.033+0000 7f03c257d640 1 --2- 172.21.15.5:0/641779407 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03a8004800 0x7f03a8004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:37.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.033+0000 7f03c257d640 1 -- 172.21.15.5:0/641779407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03a80039b0 msgr2=0x7f03a8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:37.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.033+0000 7f03c257d640 1 --2- 172.21.15.5:0/641779407 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03a80039b0 0x7f03a8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:37.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.033+0000 7f03c257d640 1 -- 172.21.15.5:0/641779407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a8003520 con 0x7f03a80051c0 2024-05-20T18:37:37.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.034+0000 7f03c257d640 1 -- 172.21.15.5:0/641779407 reap_dead start 2024-05-20T18:37:37.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.034+0000 7f03c257d640 1 --2- 172.21.15.5:0/641779407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03a80051c0 0x7f03a800b9c0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f03b4001e50 tx=0x7f03a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51679fcd0ebb725b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:37.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.034+0000 7f03c0d7a640 1 -- 172.21.15.5:0/641779407 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4001a70 con 0x7f03a80051c0 2024-05-20T18:37:37.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.034+0000 7f03c0d7a640 1 -- 172.21.15.5:0/641779407 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03b402ade0 con 0x7f03a80051c0 2024-05-20T18:37:37.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.034+0000 7f03c0d7a640 1 -- 172.21.15.5:0/641779407 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4003bb0 con 0x7f03a80051c0 2024-05-20T18:37:37.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.035+0000 7f03c3fe0640 1 -- 172.21.15.5:0/641779407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03a80051c0 msgr2=0x7f03a800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:37.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.035+0000 7f03c3fe0640 1 --2- 172.21.15.5:0/641779407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03a80051c0 0x7f03a800b9c0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f03b4001e50 tx=0x7f03a8005830 comp rx=0 tx=0).stop 2024-05-20T18:37:37.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.035+0000 7f03c3fe0640 1 -- 172.21.15.5:0/641779407 shutdown_connections 2024-05-20T18:37:37.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.035+0000 7f03c3fe0640 1 --2- 172.21.15.5:0/641779407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03a80051c0 0x7f03a800b9c0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:37.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.035+0000 7f03c3fe0640 1 -- 172.21.15.5:0/641779407 >> 172.21.15.5:0/641779407 conn(0x7f03bc0d4bd0 msgr2=0x7f03bc0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:37.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.035+0000 7f03c3fe0640 1 -- 172.21.15.5:0/641779407 shutdown_connections 2024-05-20T18:37:37.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.035+0000 7f03c3fe0640 1 -- 172.21.15.5:0/641779407 wait complete. 2024-05-20T18:37:37.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.037+0000 7f03c3fe0640 1 Processor -- start 2024-05-20T18:37:37.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.037+0000 7f03c3fe0640 1 -- start start 2024-05-20T18:37:37.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.038+0000 7f03c3fe0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03a80051c0 0x7f03bc1f24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:37.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.038+0000 7f03c3fe0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03bc0cb090 0x7f03bc0ce020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:37.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.038+0000 7f03c3fe0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03bc0c8700 0x7f03bc0c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:37.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.038+0000 7f03c3fe0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc0d0300 con 0x7f03bc0c8700 2024-05-20T18:37:37.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.038+0000 7f03c3fe0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc0d4260 con 0x7f03a80051c0 2024-05-20T18:37:37.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.038+0000 7f03c3fe0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f03bc0d4780 con 0x7f03bc0cb090 2024-05-20T18:37:37.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.038+0000 7f03c257d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03bc0c8700 0x7f03bc0c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:37.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.038+0000 7f03c257d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03bc0c8700 0x7f03bc0c8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35302/0 (socket says 172.21.15.5:35302) 2024-05-20T18:37:37.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.038+0000 7f03c257d640 1 -- 172.21.15.5:0/1678265293 learned_addr learned my addr 172.21.15.5:0/1678265293 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:37.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.039+0000 7f03c1d7c640 1 --2- 172.21.15.5:0/1678265293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03a80051c0 0x7f03bc1f24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:37.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.039+0000 7f03c157b640 1 --2- 172.21.15.5:0/1678265293 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03bc0cb090 0x7f03bc0ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:37.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.039+0000 7f03c257d640 1 -- 172.21.15.5:0/1678265293 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03bc0cb090 msgr2=0x7f03bc0ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:37.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.039+0000 7f03c257d640 1 --2- 172.21.15.5:0/1678265293 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03bc0cb090 0x7f03bc0ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:37.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.039+0000 7f03c257d640 1 -- 172.21.15.5:0/1678265293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03a80051c0 msgr2=0x7f03bc1f24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:37.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.039+0000 7f03c257d640 1 --2- 172.21.15.5:0/1678265293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03a80051c0 0x7f03bc1f24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:37.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.039+0000 7f03c257d640 1 -- 172.21.15.5:0/1678265293 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03bc0ccf80 con 0x7f03bc0c8700 2024-05-20T18:37:37.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.039+0000 7f03c1d7c640 1 --2- 172.21.15.5:0/1678265293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03a80051c0 0x7f03bc1f24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:37.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.039+0000 7f03c157b640 1 --2- 172.21.15.5:0/1678265293 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03bc0cb090 0x7f03bc0ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:37.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.039+0000 7f03c257d640 1 --2- 172.21.15.5:0/1678265293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03bc0c8700 0x7f03bc0c8b10 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f03b402a850 tx=0x7f03b40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:37.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.039+0000 7f03b27fc640 1 -- 172.21.15.5:0/1678265293 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4001250 con 0x7f03bc0c8700 2024-05-20T18:37:37.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.039+0000 7f03b27fc640 1 -- 172.21.15.5:0/1678265293 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03b402aba0 con 0x7f03bc0c8700 2024-05-20T18:37:37.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.039+0000 7f03c3fe0640 1 -- 172.21.15.5:0/1678265293 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03bc0cd260 con 0x7f03bc0c8700 2024-05-20T18:37:37.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.039+0000 7f03b27fc640 1 -- 172.21.15.5:0/1678265293 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4005830 con 0x7f03bc0c8700 2024-05-20T18:37:37.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.039+0000 7f03c3fe0640 1 -- 172.21.15.5:0/1678265293 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03bc0cd820 con 0x7f03bc0c8700 2024-05-20T18:37:37.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.040+0000 7f03c3fe0640 1 -- 172.21.15.5:0/1678265293 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03bc075b00 con 0x7f03bc0c8700 2024-05-20T18:37:37.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.041+0000 7f03b27fc640 1 -- 172.21.15.5:0/1678265293 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f03b4032440 con 0x7f03bc0c8700 2024-05-20T18:37:37.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.041+0000 7f03b27fc640 1 --2- 172.21.15.5:0/1678265293 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f03a006c000 0x7f03a006e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:37.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.041+0000 7f03b27fc640 1 -- 172.21.15.5:0/1678265293 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7f03b40aa730 con 0x7f03bc0c8700 2024-05-20T18:37:37.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.041+0000 7f03c1d7c640 1 --2- 172.21.15.5:0/1678265293 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f03a006c000 0x7f03a006e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:37.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.042+0000 7f03c1d7c640 1 --2- 172.21.15.5:0/1678265293 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f03a006c000 0x7f03a006e510 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f03b80012b0 tx=0x7f03b8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:37.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.046+0000 7f03b27fc640 1 -- 172.21.15.5:0/1678265293 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f03b4078f00 con 0x7f03bc0c8700 2024-05-20T18:37:37.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.233+0000 7f03c3fe0640 1 -- 172.21.15.5:0/1678265293 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f03bc160840 con 0x7f03bc0c8700 2024-05-20T18:37:37.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.233+0000 7f03b27fc640 1 -- 172.21.15.5:0/1678265293 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v120) ==== 93+0+10 (secure 0 0 0) 0x7f03b407f0d0 con 0x7f03bc0c8700 2024-05-20T18:37:37.235 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:37:37.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.235+0000 7f03c3fe0640 1 -- 172.21.15.5:0/1678265293 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f03a006c000 msgr2=0x7f03a006e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:37.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.235+0000 7f03c3fe0640 1 --2- 172.21.15.5:0/1678265293 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f03a006c000 0x7f03a006e510 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f03b80012b0 tx=0x7f03b8009040 comp rx=0 tx=0).stop 2024-05-20T18:37:37.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.235+0000 7f03c3fe0640 1 -- 172.21.15.5:0/1678265293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03bc0c8700 msgr2=0x7f03bc0c8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:37.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.235+0000 7f03c3fe0640 1 --2- 172.21.15.5:0/1678265293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03bc0c8700 0x7f03bc0c8b10 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f03b402a850 tx=0x7f03b40014a0 comp rx=0 tx=0).stop 2024-05-20T18:37:37.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.235+0000 7f03c3fe0640 1 -- 172.21.15.5:0/1678265293 shutdown_connections 2024-05-20T18:37:37.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.235+0000 7f03c3fe0640 1 --2- 172.21.15.5:0/1678265293 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f03a006c000 0x7f03a006e510 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:37.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.235+0000 7f03c3fe0640 1 --2- 172.21.15.5:0/1678265293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03bc0c8700 0x7f03bc0c8b10 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:37.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.235+0000 7f03c3fe0640 1 --2- 172.21.15.5:0/1678265293 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03bc0cb090 0x7f03bc0ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:37.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.235+0000 7f03c3fe0640 1 --2- 172.21.15.5:0/1678265293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03a80051c0 0x7f03bc1f24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:37.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.235+0000 7f03c3fe0640 1 -- 172.21.15.5:0/1678265293 >> 172.21.15.5:0/1678265293 conn(0x7f03bc0d4bd0 msgr2=0x7f03bc0b5170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:37.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.235+0000 7f03c3fe0640 1 -- 172.21.15.5:0/1678265293 shutdown_connections 2024-05-20T18:37:37.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.236+0000 7f03c3fe0640 1 -- 172.21.15.5:0/1678265293 wait complete. 2024-05-20T18:37:37.249 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:37:37.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.365+0000 7f62f60ca640 1 Processor -- start 2024-05-20T18:37:37.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.365+0000 7f62f60ca640 1 -- start start 2024-05-20T18:37:37.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.367+0000 7f62f60ca640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f62f0163a10 0x7f62f00d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:37.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.367+0000 7f62f60ca640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f62f00d14b0 0x7f62f00d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:37.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.367+0000 7f62f60ca640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62f00d0370 con 0x7f62f00b4010 2024-05-20T18:37:37.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.367+0000 7f62f60ca640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f62f00d3ca0 con 0x7f62f00d14b0 2024-05-20T18:37:37.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.367+0000 7f62f60ca640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f62f00d0650 con 0x7f62f0163a10 2024-05-20T18:37:37.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.367+0000 7f62effff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f62f00d14b0 0x7f62f00d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.367+0000 7f62effff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f62f00d14b0 0x7f62f00d18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35314/0 (socket says 172.21.15.5:35314) 2024-05-20T18:37:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.367+0000 7f62effff640 1 -- 172.21.15.5:0/1595748753 learned_addr learned my addr 172.21.15.5:0/1595748753 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.367+0000 7f62ef7fe640 1 --2- 172.21.15.5:0/1595748753 >> v2:172.21.15.63:3300/0 conn(0x7f62f0163a10 0x7f62f00d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.367+0000 7f62effff640 1 -- 172.21.15.5:0/1595748753 >> v2:172.21.15.63:3300/0 conn(0x7f62f0163a10 msgr2=0x7f62f00d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.367+0000 7f62effff640 1 --2- 172.21.15.5:0/1595748753 >> v2:172.21.15.63:3300/0 conn(0x7f62f0163a10 0x7f62f00d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.367+0000 7f62effff640 1 -- 172.21.15.5:0/1595748753 >> v1:172.21.15.63:6789/0 conn(0x7f62f00b4010 legacy=0x7f62f00cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.367+0000 7f62effff640 1 -- 172.21.15.5:0/1595748753 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f00d3650 con 0x7f62f00d14b0 2024-05-20T18:37:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.368+0000 7f62effff640 1 --2- 172.21.15.5:0/1595748753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f00d14b0 0x7f62f00d18c0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f62f00cf1c0 tx=0x7f62e802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511fadc329145d2d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.368+0000 7f62ee7fc640 1 -- 172.21.15.5:0/1595748753 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e8036070 con 0x7f62f00d14b0 2024-05-20T18:37:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.368+0000 7f62ee7fc640 1 -- 172.21.15.5:0/1595748753 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62e8031600 con 0x7f62f00d14b0 2024-05-20T18:37:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.368+0000 7f62ee7fc640 1 -- 172.21.15.5:0/1595748753 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e80319d0 con 0x7f62f00d14b0 2024-05-20T18:37:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.369+0000 7f62f60ca640 1 -- 172.21.15.5:0/1595748753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f00d14b0 msgr2=0x7f62f00d18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.369+0000 7f62f60ca640 1 --2- 172.21.15.5:0/1595748753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f00d14b0 0x7f62f00d18c0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f62f00cf1c0 tx=0x7f62e802a730 comp rx=0 tx=0).stop 2024-05-20T18:37:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.369+0000 7f62f60ca640 1 -- 172.21.15.5:0/1595748753 shutdown_connections 2024-05-20T18:37:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.369+0000 7f62f60ca640 1 --2- 172.21.15.5:0/1595748753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f00d14b0 0x7f62f00d18c0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.369+0000 7f62f60ca640 1 --2- 172.21.15.5:0/1595748753 >> v2:172.21.15.63:3300/0 conn(0x7f62f0163a10 0x7f62f00d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.369+0000 7f62f60ca640 1 -- 172.21.15.5:0/1595748753 >> 172.21.15.5:0/1595748753 conn(0x7f62f006d240 msgr2=0x7f62f006d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.369+0000 7f62f60ca640 1 -- 172.21.15.5:0/1595748753 shutdown_connections 2024-05-20T18:37:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.369+0000 7f62f60ca640 1 -- 172.21.15.5:0/1595748753 wait complete. 2024-05-20T18:37:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.371+0000 7f62f60ca640 1 Processor -- start 2024-05-20T18:37:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.371+0000 7f62f60ca640 1 -- start start 2024-05-20T18:37:37.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.372+0000 7f62f60ca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f00b4010 0x7f62f01f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:37.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.372+0000 7f62f60ca640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62f00d14b0 0x7f62f01f6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:37.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.372+0000 7f62f60ca640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62f0163a10 0x7f62f01f6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:37.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.372+0000 7f62f60ca640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f62f00b8210 con 0x7f62f00b4010 2024-05-20T18:37:37.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.372+0000 7f62f60ca640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f62f00d1e00 con 0x7f62f00d14b0 2024-05-20T18:37:37.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.372+0000 7f62f60ca640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f62f00b8390 con 0x7f62f0163a10 2024-05-20T18:37:37.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.372+0000 7f62ef7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f00b4010 0x7f62f01f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:37.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.372+0000 7f62ef7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f00b4010 0x7f62f01f5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35322/0 (socket says 172.21.15.5:35322) 2024-05-20T18:37:37.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.372+0000 7f62ef7fe640 1 -- 172.21.15.5:0/2403583022 learned_addr learned my addr 172.21.15.5:0/2403583022 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.372+0000 7f62ef7fe640 1 -- 172.21.15.5:0/2403583022 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62f0163a10 msgr2=0x7f62f01f6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.372+0000 7f62effff640 1 --2- 172.21.15.5:0/2403583022 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62f0163a10 0x7f62f01f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.372+0000 7f62eeffd640 1 --2- 172.21.15.5:0/2403583022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62f00d14b0 0x7f62f01f6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.372+0000 7f62ef7fe640 1 --2- 172.21.15.5:0/2403583022 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62f0163a10 0x7f62f01f6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.372+0000 7f62ef7fe640 1 -- 172.21.15.5:0/2403583022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62f00d14b0 msgr2=0x7f62f01f6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.372+0000 7f62ef7fe640 1 --2- 172.21.15.5:0/2403583022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62f00d14b0 0x7f62f01f6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.372+0000 7f62ef7fe640 1 -- 172.21.15.5:0/2403583022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f01f5670 con 0x7f62f00b4010 2024-05-20T18:37:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.372+0000 7f62eeffd640 1 --2- 172.21.15.5:0/2403583022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62f00d14b0 0x7f62f01f6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.372+0000 7f62effff640 1 --2- 172.21.15.5:0/2403583022 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62f0163a10 0x7f62f01f6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.373+0000 7f62ef7fe640 1 --2- 172.21.15.5:0/2403583022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f00b4010 0x7f62f01f5e10 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f62e000e870 tx=0x7f62e000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.373+0000 7f62d7fff640 1 -- 172.21.15.5:0/2403583022 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e0017070 con 0x7f62f00b4010 2024-05-20T18:37:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.373+0000 7f62d7fff640 1 -- 172.21.15.5:0/2403583022 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62e000c940 con 0x7f62f00b4010 2024-05-20T18:37:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.373+0000 7f62f60ca640 1 -- 172.21.15.5:0/2403583022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f00ad690 con 0x7f62f00b4010 2024-05-20T18:37:37.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.373+0000 7f62d7fff640 1 -- 172.21.15.5:0/2403583022 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e000cd10 con 0x7f62f00b4010 2024-05-20T18:37:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.373+0000 7f62f60ca640 1 -- 172.21.15.5:0/2403583022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f00adc50 con 0x7f62f00b4010 2024-05-20T18:37:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.374+0000 7f62f60ca640 1 -- 172.21.15.5:0/2403583022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62f01f5a50 con 0x7f62f00b4010 2024-05-20T18:37:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.375+0000 7f62d7fff640 1 -- 172.21.15.5:0/2403583022 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f62e001a070 con 0x7f62f00b4010 2024-05-20T18:37:37.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.375+0000 7f62d7fff640 1 --2- 172.21.15.5:0/2403583022 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f62dc06be30 0x7f62dc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:37.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.375+0000 7f62d7fff640 1 -- 172.21.15.5:0/2403583022 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7f62e008af50 con 0x7f62f00b4010 2024-05-20T18:37:37.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.375+0000 7f62eeffd640 1 --2- 172.21.15.5:0/2403583022 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f62dc06be30 0x7f62dc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:37.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.376+0000 7f62eeffd640 1 --2- 172.21.15.5:0/2403583022 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f62dc06be30 0x7f62dc06e340 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f62f0162900 tx=0x7f62e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:37.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.380+0000 7f62d7fff640 1 -- 172.21.15.5:0/2403583022 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f62f01f5670 con 0x7f62f00b4010 2024-05-20T18:37:37.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.563+0000 7f62f60ca640 1 -- 172.21.15.5:0/2403583022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f62f00ade90 con 0x7f62f00b4010 2024-05-20T18:37:37.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.564+0000 7f62d7fff640 1 -- 172.21.15.5:0/2403583022 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v120) ==== 104+0+11 (secure 0 0 0) 0x7f62e0059690 con 0x7f62f00b4010 2024-05-20T18:37:37.565 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:37:37.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.566+0000 7f62f60ca640 1 -- 172.21.15.5:0/2403583022 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f62dc06be30 msgr2=0x7f62dc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:37.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.566+0000 7f62f60ca640 1 --2- 172.21.15.5:0/2403583022 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f62dc06be30 0x7f62dc06e340 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f62f0162900 tx=0x7f62e4009040 comp rx=0 tx=0).stop 2024-05-20T18:37:37.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.566+0000 7f62f60ca640 1 -- 172.21.15.5:0/2403583022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f00b4010 msgr2=0x7f62f01f5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:37.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.566+0000 7f62f60ca640 1 --2- 172.21.15.5:0/2403583022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f00b4010 0x7f62f01f5e10 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f62e000e870 tx=0x7f62e000ed30 comp rx=0 tx=0).stop 2024-05-20T18:37:37.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.566+0000 7f62f60ca640 1 -- 172.21.15.5:0/2403583022 shutdown_connections 2024-05-20T18:37:37.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.566+0000 7f62f60ca640 1 --2- 172.21.15.5:0/2403583022 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f62dc06be30 0x7f62dc06e340 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:37.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.566+0000 7f62f60ca640 1 --2- 172.21.15.5:0/2403583022 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62f0163a10 0x7f62f01f6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:37.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.566+0000 7f62f60ca640 1 --2- 172.21.15.5:0/2403583022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62f00d14b0 0x7f62f01f6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:37.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.566+0000 7f62f60ca640 1 --2- 172.21.15.5:0/2403583022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f00b4010 0x7f62f01f5e10 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:37.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.566+0000 7f62f60ca640 1 -- 172.21.15.5:0/2403583022 >> 172.21.15.5:0/2403583022 conn(0x7f62f006d240 msgr2=0x7f62f015f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:37.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.566+0000 7f62f60ca640 1 -- 172.21.15.5:0/2403583022 shutdown_connections 2024-05-20T18:37:37.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.566+0000 7f62f60ca640 1 -- 172.21.15.5:0/2403583022 wait complete. 2024-05-20T18:37:37.579 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:37:37.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.695+0000 7f0ee32bc640 1 Processor -- start 2024-05-20T18:37:37.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.695+0000 7f0ee32bc640 1 -- start start 2024-05-20T18:37:37.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.697+0000 7f0ee32bc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0edc15f220 0x7f0edc0cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:37.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.697+0000 7f0ee32bc640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0edc0cf910 0x7f0edc0d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:37.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.697+0000 7f0ee32bc640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0edc0cfd20 con 0x7f0edc0cd680 2024-05-20T18:37:37.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.697+0000 7f0ee32bc640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0edc0cda90 con 0x7f0edc0cf910 2024-05-20T18:37:37.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.697+0000 7f0ee32bc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0edc06d0f0 con 0x7f0edc15f220 2024-05-20T18:37:37.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.697+0000 7f0ee1859640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0edc0cf910 0x7f0edc0d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:37.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.697+0000 7f0ee0857640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0edc15f220 0x7f0edc0cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:37.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.697+0000 7f0ee1058640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f0edc0cd680 0x7f0edc06c6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:45326/0 (socket says 172.21.15.5:45326) 2024-05-20T18:37:37.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.697+0000 7f0ee0857640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0edc15f220 0x7f0edc0cf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:46728/0 (socket says 172.21.15.5:46728) 2024-05-20T18:37:37.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.697+0000 7f0ee1058640 1 -- 172.21.15.5:0/700912770 learned_addr learned my addr 172.21.15.5:0/700912770 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:37.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.697+0000 7f0ee1859640 1 --2- 172.21.15.5:0/700912770 >> v2:172.21.15.63:3300/0 conn(0x7f0edc0cf910 0x7f0edc0d1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:44934/0 (socket says 172.21.15.5:44934) 2024-05-20T18:37:37.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.697+0000 7f0ee1859640 1 -- 172.21.15.5:0/700912770 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0edc15f220 msgr2=0x7f0edc0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:37.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.697+0000 7f0ee1859640 1 --2- 172.21.15.5:0/700912770 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0edc15f220 0x7f0edc0cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:37.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.697+0000 7f0ee1859640 1 -- 172.21.15.5:0/700912770 >> v1:172.21.15.63:6789/0 conn(0x7f0edc0cd680 legacy=0x7f0edc06c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:37.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.697+0000 7f0ee1859640 1 -- 172.21.15.5:0/700912770 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0edc06d3d0 con 0x7f0edc0cf910 2024-05-20T18:37:37.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.697+0000 7f0ed3fff640 1 -- 172.21.15.5:0/700912770 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 429065274 0 0) 0x7f0ed8001490 con 0x7f0edc0cd680 2024-05-20T18:37:37.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.697+0000 7f0ee0857640 1 --2- 172.21.15.5:0/700912770 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0edc15f220 0x7f0edc0cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:37:37.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.698+0000 7f0ee1859640 1 --2- 172.21.15.5:0/700912770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0edc0cf910 0x7f0edc0d1060 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0ed40097a0 tx=0x7f0ed402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bfea0d243e74740 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:37.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.698+0000 7f0ed3fff640 1 -- 172.21.15.5:0/700912770 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4037070 con 0x7f0edc0cf910 2024-05-20T18:37:37.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.698+0000 7f0ed3fff640 1 -- 172.21.15.5:0/700912770 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ed402ae50 con 0x7f0edc0cf910 2024-05-20T18:37:37.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.698+0000 7f0ed3fff640 1 -- 172.21.15.5:0/700912770 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4032660 con 0x7f0edc0cf910 2024-05-20T18:37:37.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.699+0000 7f0ee32bc640 1 -- 172.21.15.5:0/700912770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0edc0cf910 msgr2=0x7f0edc0d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:37.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.699+0000 7f0ee32bc640 1 --2- 172.21.15.5:0/700912770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0edc0cf910 0x7f0edc0d1060 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0ed40097a0 tx=0x7f0ed402a100 comp rx=0 tx=0).stop 2024-05-20T18:37:37.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.699+0000 7f0ee32bc640 1 -- 172.21.15.5:0/700912770 shutdown_connections 2024-05-20T18:37:37.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.699+0000 7f0ee32bc640 1 --2- 172.21.15.5:0/700912770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0edc0cf910 0x7f0edc0d1060 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:37.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.699+0000 7f0ee32bc640 1 --2- 172.21.15.5:0/700912770 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0edc15f220 0x7f0edc0cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:37.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.699+0000 7f0ee32bc640 1 -- 172.21.15.5:0/700912770 >> 172.21.15.5:0/700912770 conn(0x7f0edc0b44d0 msgr2=0x7f0edc0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:37.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.699+0000 7f0ee32bc640 1 -- 172.21.15.5:0/700912770 shutdown_connections 2024-05-20T18:37:37.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.699+0000 7f0ee32bc640 1 -- 172.21.15.5:0/700912770 wait complete. 2024-05-20T18:37:37.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.701+0000 7f0ee32bc640 1 Processor -- start 2024-05-20T18:37:37.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.701+0000 7f0ee32bc640 1 -- start start 2024-05-20T18:37:37.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.702+0000 7f0ee32bc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0edc0cd680 0x7f0edc151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:37.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.702+0000 7f0ee32bc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0edc0cf910 0x7f0edc15f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:37.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.702+0000 7f0ee32bc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0edc15f220 0x7f0edc15fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:37.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.702+0000 7f0ee32bc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc06d990 con 0x7f0edc15f220 2024-05-20T18:37:37.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.702+0000 7f0ee32bc640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc0d0b40 con 0x7f0edc0cd680 2024-05-20T18:37:37.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.702+0000 7f0ee32bc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0edc06cdd0 con 0x7f0edc0cf910 2024-05-20T18:37:37.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.703+0000 7f0ee1859640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0edc15f220 0x7f0edc15fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:37.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.703+0000 7f0ee1859640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0edc15f220 0x7f0edc15fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35336/0 (socket says 172.21.15.5:35336) 2024-05-20T18:37:37.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.703+0000 7f0ee1859640 1 -- 172.21.15.5:0/2302925801 learned_addr learned my addr 172.21.15.5:0/2302925801 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:37.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.703+0000 7f0ee0857640 1 --2- 172.21.15.5:0/2302925801 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0edc0cf910 0x7f0edc15f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:37.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.703+0000 7f0ee1058640 1 --2- 172.21.15.5:0/2302925801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0edc0cd680 0x7f0edc151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:37.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.703+0000 7f0ee1859640 1 -- 172.21.15.5:0/2302925801 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0edc0cf910 msgr2=0x7f0edc15f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:37.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.703+0000 7f0ee1859640 1 --2- 172.21.15.5:0/2302925801 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0edc0cf910 0x7f0edc15f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:37.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.703+0000 7f0ee1859640 1 -- 172.21.15.5:0/2302925801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0edc0cd680 msgr2=0x7f0edc151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:37.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.703+0000 7f0ee1859640 1 --2- 172.21.15.5:0/2302925801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0edc0cd680 0x7f0edc151d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:37.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.703+0000 7f0ee1859640 1 -- 172.21.15.5:0/2302925801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0edc160720 con 0x7f0edc15f220 2024-05-20T18:37:37.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.703+0000 7f0ee1859640 1 --2- 172.21.15.5:0/2302925801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0edc15f220 0x7f0edc15fec0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f0ed40097a0 tx=0x7f0ed40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:37.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.703+0000 7f0ed1ffb640 1 -- 172.21.15.5:0/2302925801 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4037040 con 0x7f0edc15f220 2024-05-20T18:37:37.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.703+0000 7f0ed1ffb640 1 -- 172.21.15.5:0/2302925801 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ed4004030 con 0x7f0edc15f220 2024-05-20T18:37:37.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.703+0000 7f0ee32bc640 1 -- 172.21.15.5:0/2302925801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0edc160a00 con 0x7f0edc15f220 2024-05-20T18:37:37.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.703+0000 7f0ed1ffb640 1 -- 172.21.15.5:0/2302925801 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4004420 con 0x7f0edc15f220 2024-05-20T18:37:37.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.704+0000 7f0ee32bc640 1 -- 172.21.15.5:0/2302925801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0edc160fc0 con 0x7f0edc15f220 2024-05-20T18:37:37.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.705+0000 7f0ee32bc640 1 -- 172.21.15.5:0/2302925801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0edc075b00 con 0x7f0edc15f220 2024-05-20T18:37:37.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.706+0000 7f0ed1ffb640 1 -- 172.21.15.5:0/2302925801 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0ed4003610 con 0x7f0edc15f220 2024-05-20T18:37:37.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.706+0000 7f0ed1ffb640 1 --2- 172.21.15.5:0/2302925801 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0ec006be60 0x7f0ec006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:37.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.706+0000 7f0ed1ffb640 1 -- 172.21.15.5:0/2302925801 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7f0ed40abe10 con 0x7f0edc15f220 2024-05-20T18:37:37.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.706+0000 7f0ee1058640 1 --2- 172.21.15.5:0/2302925801 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0ec006be60 0x7f0ec006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:37.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.707+0000 7f0ee1058640 1 --2- 172.21.15.5:0/2302925801 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0ec006be60 0x7f0ec006e370 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8007ed0 tx=0x7f0ed8007e60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:37.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.711+0000 7f0ed1ffb640 1 -- 172.21.15.5:0/2302925801 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0ed407a560 con 0x7f0edc15f220 2024-05-20T18:37:37.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.894+0000 7f0ee32bc640 1 -- 172.21.15.5:0/2302925801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0edc1604b0 con 0x7f0edc15f220 2024-05-20T18:37:37.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.895+0000 7f0ed1ffb640 1 -- 172.21.15.5:0/2302925801 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v120) ==== 100+0+11 (secure 0 0 0) 0x7f0ed4079e60 con 0x7f0edc15f220 2024-05-20T18:37:37.896 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:37:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.897+0000 7f0ee32bc640 1 -- 172.21.15.5:0/2302925801 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0ec006be60 msgr2=0x7f0ec006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.897+0000 7f0ee32bc640 1 --2- 172.21.15.5:0/2302925801 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0ec006be60 0x7f0ec006e370 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8007ed0 tx=0x7f0ed8007e60 comp rx=0 tx=0).stop 2024-05-20T18:37:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.897+0000 7f0ee32bc640 1 -- 172.21.15.5:0/2302925801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0edc15f220 msgr2=0x7f0edc15fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.897+0000 7f0ee32bc640 1 --2- 172.21.15.5:0/2302925801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0edc15f220 0x7f0edc15fec0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f0ed40097a0 tx=0x7f0ed40046c0 comp rx=0 tx=0).stop 2024-05-20T18:37:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.897+0000 7f0ee32bc640 1 -- 172.21.15.5:0/2302925801 shutdown_connections 2024-05-20T18:37:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.897+0000 7f0ee32bc640 1 --2- 172.21.15.5:0/2302925801 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f0ec006be60 0x7f0ec006e370 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.897+0000 7f0ee32bc640 1 --2- 172.21.15.5:0/2302925801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0edc15f220 0x7f0edc15fec0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.897+0000 7f0ee32bc640 1 --2- 172.21.15.5:0/2302925801 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0edc0cf910 0x7f0edc15f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.897+0000 7f0ee32bc640 1 --2- 172.21.15.5:0/2302925801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0edc0cd680 0x7f0edc151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.897+0000 7f0ee32bc640 1 -- 172.21.15.5:0/2302925801 >> 172.21.15.5:0/2302925801 conn(0x7f0edc0b44d0 msgr2=0x7f0edc0b24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.897+0000 7f0ee32bc640 1 -- 172.21.15.5:0/2302925801 shutdown_connections 2024-05-20T18:37:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:37.897+0000 7f0ee32bc640 1 -- 172.21.15.5:0/2302925801 wait complete. 2024-05-20T18:37:37.910 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T18:37:38.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.025+0000 7feb7b4b8640 1 Processor -- start 2024-05-20T18:37:38.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.025+0000 7feb7b4b8640 1 -- start start 2024-05-20T18:37:38.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.027+0000 7feb7b4b8640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7feb740ba3c0 0x7feb740b8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:38.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.027+0000 7feb7b4b8640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7feb740b85b0 0x7feb740d3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:38.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.027+0000 7feb7b4b8640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb741634a0 con 0x7feb740b6620 2024-05-20T18:37:38.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.027+0000 7feb7b4b8640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7feb740d3480 con 0x7feb740b85b0 2024-05-20T18:37:38.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.027+0000 7feb7b4b8640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7feb74163780 con 0x7feb740ba3c0 2024-05-20T18:37:38.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.027+0000 7feb79a55640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7feb740b85b0 0x7feb740d3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:38.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.027+0000 7feb79254640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7feb740b6620 0x7feb74162ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:32846/0 (socket says 172.21.15.5:32846) 2024-05-20T18:37:38.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.027+0000 7feb79a55640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7feb740b85b0 0x7feb740d3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35348/0 (socket says 172.21.15.5:35348) 2024-05-20T18:37:38.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.027+0000 7feb79a55640 1 -- 172.21.15.5:0/3408313021 learned_addr learned my addr 172.21.15.5:0/3408313021 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:38.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.027+0000 7feb78a53640 1 --2- 172.21.15.5:0/3408313021 >> v2:172.21.15.63:3300/0 conn(0x7feb740ba3c0 0x7feb740b8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:38.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.027+0000 7feb79a55640 1 -- 172.21.15.5:0/3408313021 >> v2:172.21.15.63:3300/0 conn(0x7feb740ba3c0 msgr2=0x7feb740b8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:38.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.027+0000 7feb79a55640 1 --2- 172.21.15.5:0/3408313021 >> v2:172.21.15.63:3300/0 conn(0x7feb740ba3c0 0x7feb740b8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:38.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.027+0000 7feb79a55640 1 -- 172.21.15.5:0/3408313021 >> v1:172.21.15.5:6789/0 conn(0x7feb740b6620 legacy=0x7feb74162ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:38.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.027+0000 7feb6bfff640 1 -- 172.21.15.5:0/3408313021 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1206041037 0 0) 0x7feb741634a0 con 0x7feb740b6620 2024-05-20T18:37:38.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.027+0000 7feb79a55640 1 -- 172.21.15.5:0/3408313021 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb74163a60 con 0x7feb740b85b0 2024-05-20T18:37:38.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.027+0000 7feb78a53640 1 --2- 172.21.15.5:0/3408313021 >> v2:172.21.15.63:3300/0 conn(0x7feb740ba3c0 0x7feb740b8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:38.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.028+0000 7feb79a55640 1 --2- 172.21.15.5:0/3408313021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb740b85b0 0x7feb740d3bf0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7feb740b7790 tx=0x7feb6c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eadf6b7ab89883f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:38.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.028+0000 7feb6bfff640 1 -- 172.21.15.5:0/3408313021 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c036070 con 0x7feb740b85b0 2024-05-20T18:37:38.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.028+0000 7feb6bfff640 1 -- 172.21.15.5:0/3408313021 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb6c031600 con 0x7feb740b85b0 2024-05-20T18:37:38.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.028+0000 7feb6bfff640 1 -- 172.21.15.5:0/3408313021 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c0319d0 con 0x7feb740b85b0 2024-05-20T18:37:38.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.029+0000 7feb7b4b8640 1 -- 172.21.15.5:0/3408313021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb740b85b0 msgr2=0x7feb740d3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:38.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.029+0000 7feb7b4b8640 1 --2- 172.21.15.5:0/3408313021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb740b85b0 0x7feb740d3bf0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7feb740b7790 tx=0x7feb6c02a730 comp rx=0 tx=0).stop 2024-05-20T18:37:38.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.029+0000 7feb7b4b8640 1 -- 172.21.15.5:0/3408313021 shutdown_connections 2024-05-20T18:37:38.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.029+0000 7feb7b4b8640 1 --2- 172.21.15.5:0/3408313021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb740b85b0 0x7feb740d3bf0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:38.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.029+0000 7feb7b4b8640 1 --2- 172.21.15.5:0/3408313021 >> v2:172.21.15.63:3300/0 conn(0x7feb740ba3c0 0x7feb740b8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:38.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.029+0000 7feb7b4b8640 1 -- 172.21.15.5:0/3408313021 >> 172.21.15.5:0/3408313021 conn(0x7feb7406d240 msgr2=0x7feb7406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:38.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.029+0000 7feb7b4b8640 1 -- 172.21.15.5:0/3408313021 shutdown_connections 2024-05-20T18:37:38.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.029+0000 7feb7b4b8640 1 -- 172.21.15.5:0/3408313021 wait complete. 2024-05-20T18:37:38.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.031+0000 7feb7b4b8640 1 Processor -- start 2024-05-20T18:37:38.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.031+0000 7feb7b4b8640 1 -- start start 2024-05-20T18:37:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.032+0000 7feb7b4b8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb740b6620 0x7feb7414fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.032+0000 7feb7b4b8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb740b85b0 0x7feb740cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.032+0000 7feb7b4b8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb740ba3c0 0x7feb740ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.032+0000 7feb7b4b8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feb740d42b0 con 0x7feb740ba3c0 2024-05-20T18:37:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.032+0000 7feb7b4b8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feb740d4130 con 0x7feb740b6620 2024-05-20T18:37:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.032+0000 7feb7b4b8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7feb740d4430 con 0x7feb740b85b0 2024-05-20T18:37:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.032+0000 7feb79a55640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb740ba3c0 0x7feb740ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.032+0000 7feb79a55640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb740ba3c0 0x7feb740ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35364/0 (socket says 172.21.15.5:35364) 2024-05-20T18:37:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.032+0000 7feb79a55640 1 -- 172.21.15.5:0/182524932 learned_addr learned my addr 172.21.15.5:0/182524932 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.032+0000 7feb79254640 1 --2- 172.21.15.5:0/182524932 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb740b6620 0x7feb7414fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.032+0000 7feb78a53640 1 --2- 172.21.15.5:0/182524932 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb740b85b0 0x7feb740cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.033+0000 7feb79a55640 1 -- 172.21.15.5:0/182524932 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb740b85b0 msgr2=0x7feb740cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:38.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.033+0000 7feb79a55640 1 --2- 172.21.15.5:0/182524932 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb740b85b0 0x7feb740cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.033+0000 7feb79a55640 1 -- 172.21.15.5:0/182524932 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb740b6620 msgr2=0x7feb7414fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.033+0000 7feb79a55640 1 --2- 172.21.15.5:0/182524932 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb740b6620 0x7feb7414fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.033+0000 7feb79a55640 1 -- 172.21.15.5:0/182524932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7415d5f0 con 0x7feb740ba3c0 2024-05-20T18:37:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.033+0000 7feb78a53640 1 --2- 172.21.15.5:0/182524932 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb740b85b0 0x7feb740cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.033+0000 7feb79a55640 1 --2- 172.21.15.5:0/182524932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb740ba3c0 0x7feb740ccf60 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7feb6c02ad20 tx=0x7feb6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.033+0000 7feb69ffb640 1 -- 172.21.15.5:0/182524932 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c03f020 con 0x7feb740ba3c0 2024-05-20T18:37:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.033+0000 7feb69ffb640 1 -- 172.21.15.5:0/182524932 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb6c037040 con 0x7feb740ba3c0 2024-05-20T18:37:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.033+0000 7feb69ffb640 1 -- 172.21.15.5:0/182524932 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c036070 con 0x7feb740ba3c0 2024-05-20T18:37:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.033+0000 7feb7b4b8640 1 -- 172.21.15.5:0/182524932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb740cd5f0 con 0x7feb740ba3c0 2024-05-20T18:37:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.033+0000 7feb7b4b8640 1 -- 172.21.15.5:0/182524932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb740cdbb0 con 0x7feb740ba3c0 2024-05-20T18:37:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.034+0000 7feb7b4b8640 1 -- 172.21.15.5:0/182524932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb7415d360 con 0x7feb740ba3c0 2024-05-20T18:37:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.035+0000 7feb69ffb640 1 -- 172.21.15.5:0/182524932 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feb6c042020 con 0x7feb740ba3c0 2024-05-20T18:37:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.035+0000 7feb69ffb640 1 --2- 172.21.15.5:0/182524932 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feb5806be30 0x7feb5806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.035+0000 7feb69ffb640 1 -- 172.21.15.5:0/182524932 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7feb6c0ab980 con 0x7feb740ba3c0 2024-05-20T18:37:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.035+0000 7feb79254640 1 --2- 172.21.15.5:0/182524932 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feb5806be30 0x7feb5806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.036+0000 7feb79254640 1 --2- 172.21.15.5:0/182524932 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feb5806be30 0x7feb5806e340 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7feb70000f60 tx=0x7feb70007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:38.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.040+0000 7feb69ffb640 1 -- 172.21.15.5:0/182524932 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7feb6c049020 con 0x7feb740ba3c0 2024-05-20T18:37:38.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.236+0000 7feb7b4b8640 1 -- 172.21.15.5:0/182524932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7feb74075b00 con 0x7feb740ba3c0 2024-05-20T18:37:38.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.237+0000 7feb69ffb640 1 -- 172.21.15.5:0/182524932 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v120) ==== 80+0+122 (secure 0 0 0) 0x7feb6c07a0c0 con 0x7feb740ba3c0 2024-05-20T18:37:38.238 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.63:6835/2352520472 2024-05-21T18:37:28.901938+0000 2024-05-20T18:37:38.238 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.63:6833/2352520472 2024-05-21T18:37:28.901938+0000 2024-05-20T18:37:38.242 INFO:teuthology.orchestra.run.smithi005.stderr:listed 2 entries 2024-05-20T18:37:38.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.239+0000 7feb7b4b8640 1 -- 172.21.15.5:0/182524932 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feb5806be30 msgr2=0x7feb5806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:38.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.239+0000 7feb7b4b8640 1 --2- 172.21.15.5:0/182524932 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feb5806be30 0x7feb5806e340 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7feb70000f60 tx=0x7feb70007c00 comp rx=0 tx=0).stop 2024-05-20T18:37:38.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.239+0000 7feb7b4b8640 1 -- 172.21.15.5:0/182524932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb740ba3c0 msgr2=0x7feb740ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:38.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.239+0000 7feb7b4b8640 1 --2- 172.21.15.5:0/182524932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb740ba3c0 0x7feb740ccf60 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7feb6c02ad20 tx=0x7feb6c004a40 comp rx=0 tx=0).stop 2024-05-20T18:37:38.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.239+0000 7feb7b4b8640 1 -- 172.21.15.5:0/182524932 shutdown_connections 2024-05-20T18:37:38.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.239+0000 7feb7b4b8640 1 --2- 172.21.15.5:0/182524932 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7feb5806be30 0x7feb5806e340 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:38.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.239+0000 7feb7b4b8640 1 --2- 172.21.15.5:0/182524932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb740ba3c0 0x7feb740ccf60 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:38.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.239+0000 7feb7b4b8640 1 --2- 172.21.15.5:0/182524932 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb740b85b0 0x7feb740cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:38.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.239+0000 7feb7b4b8640 1 --2- 172.21.15.5:0/182524932 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb740b6620 0x7feb7414fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:38.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.239+0000 7feb7b4b8640 1 -- 172.21.15.5:0/182524932 >> 172.21.15.5:0/182524932 conn(0x7feb7406d240 msgr2=0x7feb740b9230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:38.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.239+0000 7feb7b4b8640 1 -- 172.21.15.5:0/182524932 shutdown_connections 2024-05-20T18:37:38.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.239+0000 7feb7b4b8640 1 -- 172.21.15.5:0/182524932 wait complete. 2024-05-20T18:37:38.252 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-20T18:37:38.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:38.252 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-20T18:37:38.281 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:37:38.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:38.281 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-20T18:37:38.338 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:37:38.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.491+0000 7f7688b1a640 1 Processor -- start 2024-05-20T18:37:38.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.491+0000 7f7688b1a640 1 -- start start 2024-05-20T18:37:38.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.492+0000 7f7688b1a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76841576d0 0x7f768406c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:38.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.492+0000 7f7688b1a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f76840cedd0 0x7f768406e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:38.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.492+0000 7f7688b1a640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7684157ae0 con 0x7f768415d5f0 2024-05-20T18:37:38.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.492+0000 7f7688b1a640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f768406cee0 con 0x7f76840cedd0 2024-05-20T18:37:38.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.492+0000 7f7688b1a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f768406b900 con 0x7f76841576d0 2024-05-20T18:37:38.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.492+0000 7f768259c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f768415d5f0 0x7f76840ce6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:32850/0 (socket says 172.21.15.5:32850) 2024-05-20T18:37:38.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.492+0000 7f768259c640 1 -- 172.21.15.5:0/3762809499 learned_addr learned my addr 172.21.15.5:0/3762809499 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:38.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.493+0000 7f7681d9b640 1 -- 172.21.15.5:0/3762809499 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3322112460 0 0) 0x7f7684157ae0 con 0x7f768415d5f0 2024-05-20T18:37:38.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.493+0000 7f7679d9b640 1 --2- 172.21.15.5:0/3762809499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76841576d0 0x7f768406c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:38.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.493+0000 7f7682d9d640 1 --2- 172.21.15.5:0/3762809499 >> v2:172.21.15.63:3300/0 conn(0x7f76840cedd0 0x7f768406e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:38.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.493+0000 7f7681d9b640 1 -- 172.21.15.5:0/3762809499 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7670003520 con 0x7f768415d5f0 2024-05-20T18:37:38.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.493+0000 7f7681d9b640 1 -- 172.21.15.5:0/3762809499 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2003911706 0 0) 0x7f7670003520 con 0x7f768415d5f0 2024-05-20T18:37:38.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.493+0000 7f7681d9b640 1 -- 172.21.15.5:0/3762809499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76841576d0 msgr2=0x7f768406c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:38.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.493+0000 7f7681d9b640 1 --2- 172.21.15.5:0/3762809499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76841576d0 0x7f768406c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:38.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.493+0000 7f7681d9b640 1 -- 172.21.15.5:0/3762809499 >> v2:172.21.15.63:3300/0 conn(0x7f76840cedd0 msgr2=0x7f768406e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:38.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.493+0000 7f7681d9b640 1 --2- 172.21.15.5:0/3762809499 >> v2:172.21.15.63:3300/0 conn(0x7f76840cedd0 0x7f768406e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:38.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.493+0000 7f7681d9b640 1 -- 172.21.15.5:0/3762809499 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768406bbe0 con 0x7f768415d5f0 2024-05-20T18:37:38.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.493+0000 7f7681d9b640 1 -- 172.21.15.5:0/3762809499 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f7674001670 con 0x7f768415d5f0 2024-05-20T18:37:38.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.493+0000 7f7681d9b640 1 -- 172.21.15.5:0/3762809499 >> v1:172.21.15.5:6789/0 conn(0x7f768415d5f0 legacy=0x7f76840ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:38.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.493+0000 7f7681d9b640 1 --2- 172.21.15.5:0/3762809499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76700039b0 0x7f7670003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:38.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.493+0000 7f7681d9b640 1 --2- 172.21.15.5:0/3762809499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7670004800 0x7f7670004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:38.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.493+0000 7f7681d9b640 1 --2- 172.21.15.5:0/3762809499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76841576d0 0x7f7670005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:38.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.493+0000 7f7681d9b640 1 -- 172.21.15.5:0/3762809499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f768406cee0 con 0x7f7670004800 2024-05-20T18:37:38.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.493+0000 7f7681d9b640 1 -- 172.21.15.5:0/3762809499 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f768406b900 con 0x7f76700039b0 2024-05-20T18:37:38.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.493+0000 7f7681d9b640 1 -- 172.21.15.5:0/3762809499 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f767000ba70 con 0x7f76841576d0 2024-05-20T18:37:38.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.493+0000 7f7679d9b640 1 --2- 172.21.15.5:0/3762809499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7670004800 0x7f7670004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:38.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.493+0000 7f768259c640 1 --2- 172.21.15.5:0/3762809499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76700039b0 0x7f7670003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:38.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.493+0000 7f7682d9d640 1 --2- 172.21.15.5:0/3762809499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76841576d0 0x7f7670005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:38.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.494+0000 7f7679d9b640 1 -- 172.21.15.5:0/3762809499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76841576d0 msgr2=0x7f7670005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:38.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.494+0000 7f7679d9b640 1 --2- 172.21.15.5:0/3762809499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76841576d0 0x7f7670005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:38.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.494+0000 7f7679d9b640 1 -- 172.21.15.5:0/3762809499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76700039b0 msgr2=0x7f7670003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:38.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.494+0000 7f7679d9b640 1 --2- 172.21.15.5:0/3762809499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76700039b0 0x7f7670003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:38.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.494+0000 7f7679d9b640 1 -- 172.21.15.5:0/3762809499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7670003520 con 0x7f7670004800 2024-05-20T18:37:38.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.494+0000 7f7679d9b640 1 --2- 172.21.15.5:0/3762809499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7670004800 0x7f7670004c10 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f766c009a90 tx=0x7f7670005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0ad302a51df379 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:38.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.494+0000 7f7681d9b640 1 -- 172.21.15.5:0/3762809499 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c036070 con 0x7f7670004800 2024-05-20T18:37:38.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.494+0000 7f7681d9b640 1 -- 172.21.15.5:0/3762809499 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f766c031400 con 0x7f7670004800 2024-05-20T18:37:38.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.495+0000 7f7681d9b640 1 -- 172.21.15.5:0/3762809499 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c009690 con 0x7f7670004800 2024-05-20T18:37:38.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.495+0000 7f7688b1a640 1 -- 172.21.15.5:0/3762809499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7670004800 msgr2=0x7f7670004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:38.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.495+0000 7f7688b1a640 1 --2- 172.21.15.5:0/3762809499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7670004800 0x7f7670004c10 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f766c009a90 tx=0x7f7670005830 comp rx=0 tx=0).stop 2024-05-20T18:37:38.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.495+0000 7f7682d9d640 1 -- 172.21.15.5:0/3762809499 reap_dead start 2024-05-20T18:37:38.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.495+0000 7f7688b1a640 1 -- 172.21.15.5:0/3762809499 shutdown_connections 2024-05-20T18:37:38.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.495+0000 7f7688b1a640 1 -- 172.21.15.5:0/3762809499 >> 172.21.15.5:0/3762809499 conn(0x7f76840cd2a0 msgr2=0x7f76840cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:38.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.495+0000 7f7688b1a640 1 -- 172.21.15.5:0/3762809499 shutdown_connections 2024-05-20T18:37:38.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.495+0000 7f7688b1a640 1 -- 172.21.15.5:0/3762809499 wait complete. 2024-05-20T18:37:38.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.497+0000 7f7688b1a640 1 Processor -- start 2024-05-20T18:37:38.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.497+0000 7f7688b1a640 1 -- start start 2024-05-20T18:37:38.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.499+0000 7f7688b1a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7670004800 0x7f7684145100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:38.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.499+0000 7f7688b1a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76841e8700 0x7f76841e8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:38.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.499+0000 7f7688b1a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f768415f7e0 0x7f768415fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:38.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.499+0000 7f7688b1a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f768406c480 con 0x7f76841e8700 2024-05-20T18:37:38.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.499+0000 7f7688b1a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f768406c1a0 con 0x7f7670004800 2024-05-20T18:37:38.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.499+0000 7f7688b1a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f76840cf1e0 con 0x7f768415f7e0 2024-05-20T18:37:38.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.499+0000 7f7679d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76841e8700 0x7f76841e8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:38.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.499+0000 7f7679d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76841e8700 0x7f76841e8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35382/0 (socket says 172.21.15.5:35382) 2024-05-20T18:37:38.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.499+0000 7f7679d9b640 1 -- 172.21.15.5:0/3304479153 learned_addr learned my addr 172.21.15.5:0/3304479153 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:38.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.499+0000 7f768259c640 1 --2- 172.21.15.5:0/3304479153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7670004800 0x7f7684145100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:38.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.499+0000 7f7682d9d640 1 --2- 172.21.15.5:0/3304479153 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f768415f7e0 0x7f768415fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:38.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.499+0000 7f7679d9b640 1 -- 172.21.15.5:0/3304479153 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f768415f7e0 msgr2=0x7f768415fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:38.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.499+0000 7f7679d9b640 1 --2- 172.21.15.5:0/3304479153 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f768415f7e0 0x7f768415fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:38.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.499+0000 7f7679d9b640 1 -- 172.21.15.5:0/3304479153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7670004800 msgr2=0x7f7684145100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:38.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.499+0000 7f7679d9b640 1 --2- 172.21.15.5:0/3304479153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7670004800 0x7f7684145100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:38.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.499+0000 7f7679d9b640 1 -- 172.21.15.5:0/3304479153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7684160130 con 0x7f76841e8700 2024-05-20T18:37:38.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.500+0000 7f7679d9b640 1 --2- 172.21.15.5:0/3304479153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76841e8700 0x7f76841e8b10 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f766c009bb0 tx=0x7f766c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:38.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.500+0000 7f767b7fe640 1 -- 172.21.15.5:0/3304479153 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c040020 con 0x7f76841e8700 2024-05-20T18:37:38.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.500+0000 7f767b7fe640 1 -- 172.21.15.5:0/3304479153 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f766c003bc0 con 0x7f76841e8700 2024-05-20T18:37:38.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.500+0000 7f7688b1a640 1 -- 172.21.15.5:0/3304479153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76841e7ca0 con 0x7f76841e8700 2024-05-20T18:37:38.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.500+0000 7f767b7fe640 1 -- 172.21.15.5:0/3304479153 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c036040 con 0x7f76841e8700 2024-05-20T18:37:38.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.500+0000 7f7688b1a640 1 -- 172.21.15.5:0/3304479153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76841e6ee0 con 0x7f76841e8700 2024-05-20T18:37:38.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.501+0000 7f7688b1a640 1 -- 172.21.15.5:0/3304479153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7684075b00 con 0x7f76841e8700 2024-05-20T18:37:38.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.501+0000 7f767b7fe640 1 -- 172.21.15.5:0/3304479153 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f766c043020 con 0x7f76841e8700 2024-05-20T18:37:38.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.502+0000 7f767b7fe640 1 --2- 172.21.15.5:0/3304479153 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f766406be60 0x7f766406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:38.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.502+0000 7f768259c640 1 --2- 172.21.15.5:0/3304479153 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f766406be60 0x7f766406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:38.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.502+0000 7f767b7fe640 1 -- 172.21.15.5:0/3304479153 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7f766c0ab110 con 0x7f76841e8700 2024-05-20T18:37:38.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.503+0000 7f768259c640 1 --2- 172.21.15.5:0/3304479153 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f766406be60 0x7f766406e370 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f7674001050 tx=0x7f7674007970 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:38.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.507+0000 7f767b7fe640 1 -- 172.21.15.5:0/3304479153 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f766c04a070 con 0x7f76841e8700 2024-05-20T18:37:38.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.728+0000 7f7688b1a640 1 -- 172.21.15.5:0/3304479153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76840b2020 con 0x7f76841e8700 2024-05-20T18:37:38.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.729+0000 7f767b7fe640 1 -- 172.21.15.5:0/3304479153 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+5180 (secure 0 0 0) 0x7f7684160130 con 0x7f76841e8700 2024-05-20T18:37:38.730 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:38.730 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":104,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/266630184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":266630184},{"type":"v1","addr":"172.21.15.63:6834","nonce":266630184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7361,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1494022697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":1494022697},{"type":"v1","addr":"172.21.15.63:6835","nonce":1494022697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":9009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/1739221946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1739221946},{"type":"v1","addr":"172.21.15.5:6836","nonce":1739221946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:37:14.365195+0000","modified":"2024-05-20T18:37:32.914662+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9012":{"gid":9012,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":8,"addr":"172.21.15.5:6837/4273429943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":4273429943},{"type":"v1","addr":"172.21.15.5:6837","nonce":4273429943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9012,"qdb_cluster":[9012]},"id":8}]} 2024-05-20T18:37:38.731 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 104 2024-05-20T18:37:38.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.731+0000 7f7688b1a640 1 -- 172.21.15.5:0/3304479153 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f766406be60 msgr2=0x7f766406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:38.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.731+0000 7f7688b1a640 1 --2- 172.21.15.5:0/3304479153 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f766406be60 0x7f766406e370 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f7674001050 tx=0x7f7674007970 comp rx=0 tx=0).stop 2024-05-20T18:37:38.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.731+0000 7f7688b1a640 1 -- 172.21.15.5:0/3304479153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76841e8700 msgr2=0x7f76841e8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:38.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.731+0000 7f7688b1a640 1 --2- 172.21.15.5:0/3304479153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76841e8700 0x7f76841e8b10 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f766c009bb0 tx=0x7f766c004730 comp rx=0 tx=0).stop 2024-05-20T18:37:38.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.731+0000 7f7688b1a640 1 -- 172.21.15.5:0/3304479153 shutdown_connections 2024-05-20T18:37:38.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.731+0000 7f7688b1a640 1 --2- 172.21.15.5:0/3304479153 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f766406be60 0x7f766406e370 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:38.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.731+0000 7f7688b1a640 1 --2- 172.21.15.5:0/3304479153 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f768415f7e0 0x7f768415fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:38.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.731+0000 7f7688b1a640 1 --2- 172.21.15.5:0/3304479153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76841e8700 0x7f76841e8b10 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:38.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.731+0000 7f7688b1a640 1 --2- 172.21.15.5:0/3304479153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7670004800 0x7f7684145100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:38.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.731+0000 7f7688b1a640 1 -- 172.21.15.5:0/3304479153 >> 172.21.15.5:0/3304479153 conn(0x7f76840cd2a0 msgr2=0x7f76840b0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:38.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.731+0000 7f7688b1a640 1 -- 172.21.15.5:0/3304479153 shutdown_connections 2024-05-20T18:37:38.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.731+0000 7f7688b1a640 1 -- 172.21.15.5:0/3304479153 wait complete. 2024-05-20T18:37:38.744 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parent mytag123 2024-05-20T18:37:38.879 INFO:teuthology.orchestra.run.smithi005.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2024-05-20T18:37:38.880 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2024-05-20T18:37:38.880 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-05-20T18:37:38.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.935+0000 7faedec14f40 1 Processor -- start 2024-05-20T18:37:38.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.935+0000 7faedec14f40 1 -- start start 2024-05-20T18:37:38.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.936+0000 7faedec14f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3bf34170 0x55eb3bf1b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:38.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.936+0000 7faedec14f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3bf34580 0x55eb3bfb7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:38.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.936+0000 7faedec14f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55eb3bf19340 0x55eb3bf1ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:38.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.936+0000 7faedec14f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55eb3bf35b60 con 0x55eb3bf34170 2024-05-20T18:37:38.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.936+0000 7faedec14f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55eb3be94c80 con 0x55eb3bf34580 2024-05-20T18:37:38.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.936+0000 7faedec14f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55eb3bf35e00 con 0x55eb3bf19340 2024-05-20T18:37:38.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.936+0000 7faed7a74640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3bf34170 0x55eb3bf1b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:38.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.936+0000 7faed7a74640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3bf34170 0x55eb3bf1b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35394/0 (socket says 172.21.15.5:35394) 2024-05-20T18:37:38.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.936+0000 7faed7a74640 1 -- 172.21.15.5:0/1046872664 learned_addr learned my addr 172.21.15.5:0/1046872664 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:38.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.937+0000 7faed7a74640 1 -- 172.21.15.5:0/1046872664 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55eb3bf19340 msgr2=0x55eb3bf1ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:38.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.937+0000 7faed7a74640 1 --2- 172.21.15.5:0/1046872664 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55eb3bf19340 0x55eb3bf1ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:38.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.937+0000 7faed7a74640 1 -- 172.21.15.5:0/1046872664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3bf34580 msgr2=0x55eb3bfb7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:38.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.937+0000 7faed7a74640 1 --2- 172.21.15.5:0/1046872664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3bf34580 0x55eb3bfb7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:38.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.937+0000 7faed7a74640 1 -- 172.21.15.5:0/1046872664 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb3be19cb0 con 0x55eb3bf34170 2024-05-20T18:37:38.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.937+0000 7faed7a74640 1 --2- 172.21.15.5:0/1046872664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3bf34170 0x55eb3bf1b170 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7faec800b2c0 tx=0x7faec800b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf4f0b255fcb04b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:38.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.937+0000 7faed6271640 1 -- 172.21.15.5:0/1046872664 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec8008070 con 0x55eb3bf34170 2024-05-20T18:37:38.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.937+0000 7faed6271640 1 -- 172.21.15.5:0/1046872664 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faec800d500 con 0x55eb3bf34170 2024-05-20T18:37:38.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.937+0000 7faed6271640 1 -- 172.21.15.5:0/1046872664 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec800d8d0 con 0x55eb3bf34170 2024-05-20T18:37:38.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.938+0000 7faedec14f40 1 -- 172.21.15.5:0/1046872664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3bf34170 msgr2=0x55eb3bf1b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:38.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.938+0000 7faedec14f40 1 --2- 172.21.15.5:0/1046872664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3bf34170 0x55eb3bf1b170 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7faec800b2c0 tx=0x7faec800b780 comp rx=0 tx=0).stop 2024-05-20T18:37:38.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.938+0000 7faedec14f40 1 -- 172.21.15.5:0/1046872664 shutdown_connections 2024-05-20T18:37:38.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.938+0000 7faedec14f40 1 --2- 172.21.15.5:0/1046872664 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55eb3bf19340 0x55eb3bf1ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:38.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.938+0000 7faedec14f40 1 --2- 172.21.15.5:0/1046872664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3bf34580 0x55eb3bfb7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:38.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.938+0000 7faedec14f40 1 --2- 172.21.15.5:0/1046872664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3bf34170 0x55eb3bf1b170 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:38.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.938+0000 7faedec14f40 1 -- 172.21.15.5:0/1046872664 >> 172.21.15.5:0/1046872664 conn(0x55eb3be12ac0 msgr2=0x55eb3bf18580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:38.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.938+0000 7faedec14f40 1 -- 172.21.15.5:0/1046872664 shutdown_connections 2024-05-20T18:37:38.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.938+0000 7faedec14f40 1 -- 172.21.15.5:0/1046872664 wait complete. 2024-05-20T18:37:38.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.939+0000 7faedec14f40 1 Processor -- start 2024-05-20T18:37:38.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.939+0000 7faedec14f40 1 -- start start 2024-05-20T18:37:38.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.940+0000 7faedec14f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3bf34170 0x55eb3bfb74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:38.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.940+0000 7faedec14f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55eb3bf34580 0x55eb3bfb3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:38.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.940+0000 7faedec14f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3bfb8f90 0x55eb3bfb93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:38.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.940+0000 7faedec14f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55eb3bf1b6b0 con 0x55eb3bfb8f90 2024-05-20T18:37:38.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.940+0000 7faedec14f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55eb3beed2e0 con 0x55eb3bf34170 2024-05-20T18:37:38.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.940+0000 7faedec14f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55eb3bfb9d20 con 0x55eb3bf34580 2024-05-20T18:37:38.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.940+0000 7faed8275640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3bfb8f90 0x55eb3bfb93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:38.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.940+0000 7faed8275640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3bfb8f90 0x55eb3bfb93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35404/0 (socket says 172.21.15.5:35404) 2024-05-20T18:37:38.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.940+0000 7faed8275640 1 -- 172.21.15.5:0/1719846970 learned_addr learned my addr 172.21.15.5:0/1719846970 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:38.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.940+0000 7faed7a74640 1 --2- 172.21.15.5:0/1719846970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3bf34170 0x55eb3bfb74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:38.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.940+0000 7faed7273640 1 --2- 172.21.15.5:0/1719846970 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55eb3bf34580 0x55eb3bfb3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:38.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.940+0000 7faed8275640 1 -- 172.21.15.5:0/1719846970 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55eb3bf34580 msgr2=0x55eb3bfb3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:38.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.940+0000 7faed8275640 1 --2- 172.21.15.5:0/1719846970 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55eb3bf34580 0x55eb3bfb3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:38.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.940+0000 7faed8275640 1 -- 172.21.15.5:0/1719846970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3bf34170 msgr2=0x55eb3bfb74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:38.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.940+0000 7faed8275640 1 --2- 172.21.15.5:0/1719846970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55eb3bf34170 0x55eb3bfb74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:38.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.940+0000 7faed8275640 1 -- 172.21.15.5:0/1719846970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb3bfb4620 con 0x55eb3bfb8f90 2024-05-20T18:37:38.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.940+0000 7faed7273640 1 --2- 172.21.15.5:0/1719846970 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55eb3bf34580 0x55eb3bfb3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:38.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.941+0000 7faed8275640 1 --2- 172.21.15.5:0/1719846970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3bfb8f90 0x55eb3bfb93a0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7faed000cfd0 tx=0x7faed000a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:38.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.941+0000 7faed526f640 1 -- 172.21.15.5:0/1719846970 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed000ae20 con 0x55eb3bfb8f90 2024-05-20T18:37:38.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.941+0000 7faed526f640 1 -- 172.21.15.5:0/1719846970 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faed0018070 con 0x55eb3bfb8f90 2024-05-20T18:37:38.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.941+0000 7faedec14f40 1 -- 172.21.15.5:0/1719846970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55eb3bf1b330 con 0x55eb3bfb8f90 2024-05-20T18:37:38.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.941+0000 7faed526f640 1 -- 172.21.15.5:0/1719846970 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed0011810 con 0x55eb3bfb8f90 2024-05-20T18:37:38.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.941+0000 7faedec14f40 1 -- 172.21.15.5:0/1719846970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb3bfb9a60 con 0x55eb3bfb8f90 2024-05-20T18:37:38.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.942+0000 7faed526f640 1 -- 172.21.15.5:0/1719846970 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faed00119b0 con 0x55eb3bfb8f90 2024-05-20T18:37:38.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.943+0000 7faed526f640 1 --2- 172.21.15.5:0/1719846970 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faec406a810 0x7faec406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:38.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.943+0000 7faed526f640 1 -- 172.21.15.5:0/1719846970 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7faed0089f30 con 0x55eb3bfb8f90 2024-05-20T18:37:38.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.943+0000 7faed7a74640 1 --2- 172.21.15.5:0/1719846970 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faec406a810 0x7faec406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:38.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.943+0000 7faedec14f40 1 --2- 172.21.15.5:0/1719846970 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55eb3beeae90 0x55eb3c0673b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:38.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.943+0000 7faedec14f40 1 -- 172.21.15.5:0/1719846970 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:1 18.36 18:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x55eb3bfc5540 con 0x55eb3beeae90 2024-05-20T18:37:38.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.943+0000 7faed7273640 1 --2- 172.21.15.5:0/1719846970 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55eb3beeae90 0x55eb3c0673b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:38.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.943+0000 7faed7a74640 1 --2- 172.21.15.5:0/1719846970 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faec406a810 0x7faec406cd20 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7faec8000c00 tx=0x7faec8008000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:38.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.944+0000 7faed7273640 1 --2- 172.21.15.5:0/1719846970 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55eb3beeae90 0x55eb3c0673b0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:38.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.944+0000 7faed7273640 1 -- 172.21.15.5:0/1719846970 <== osd.1 v2:172.21.15.5:6815/550664725 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7faecc051070 con 0x55eb3beeae90 2024-05-20T18:37:38.949 INFO:teuthology.orchestra.run.smithi005.stderr:error getting xattr cephfs_data/100000001f7.00000000/scrub_tag: (61) No data available 2024-05-20T18:37:38.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.944+0000 7faedec14f40 1 -- 172.21.15.5:0/1719846970 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55eb3beeae90 msgr2=0x55eb3c0673b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:38.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.944+0000 7faedec14f40 1 --2- 172.21.15.5:0/1719846970 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55eb3beeae90 0x55eb3c0673b0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:38.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.944+0000 7faedec14f40 1 -- 172.21.15.5:0/1719846970 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faec406a810 msgr2=0x7faec406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:38.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.944+0000 7faedec14f40 1 --2- 172.21.15.5:0/1719846970 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7faec406a810 0x7faec406cd20 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7faec8000c00 tx=0x7faec8008000 comp rx=0 tx=0).stop 2024-05-20T18:37:38.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.944+0000 7faedec14f40 1 -- 172.21.15.5:0/1719846970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3bfb8f90 msgr2=0x55eb3bfb93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:38.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.944+0000 7faedec14f40 1 --2- 172.21.15.5:0/1719846970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb3bfb8f90 0x55eb3bfb93a0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7faed000cfd0 tx=0x7faed000a5f0 comp rx=0 tx=0).stop 2024-05-20T18:37:38.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.944+0000 7faed8275640 1 -- 172.21.15.5:0/1719846970 reap_dead start 2024-05-20T18:37:38.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.944+0000 7faedec14f40 1 -- 172.21.15.5:0/1719846970 shutdown_connections 2024-05-20T18:37:38.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.944+0000 7faedec14f40 1 -- 172.21.15.5:0/1719846970 >> 172.21.15.5:0/1719846970 conn(0x55eb3be12ac0 msgr2=0x55eb3bfb7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:38.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.944+0000 7faedec14f40 1 -- 172.21.15.5:0/1719846970 shutdown_connections 2024-05-20T18:37:38.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:38.944+0000 7faedec14f40 1 -- 172.21.15.5:0/1719846970 wait complete. 2024-05-20T18:37:38.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T18:37:38.951 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-05-20T18:37:39.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.007+0000 7f2e92416f40 1 Processor -- start 2024-05-20T18:37:39.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.007+0000 7f2e92416f40 1 -- start start 2024-05-20T18:37:39.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.008+0000 7f2e92416f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3af8523a0 0x55d3af857760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.008+0000 7f2e92416f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d3af8527b0 0x55d3af8eb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.008+0000 7f2e92416f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3af907430 0x55d3af902240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.008+0000 7f2e92416f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d3af8262f0 con 0x55d3af8523a0 2024-05-20T18:37:39.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.008+0000 7f2e92416f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d3af7cfc80 con 0x55d3af907430 2024-05-20T18:37:39.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.008+0000 7f2e92416f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55d3af8fd880 con 0x55d3af8527b0 2024-05-20T18:37:39.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.008+0000 7f2e8b274640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3af8523a0 0x55d3af857760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.008+0000 7f2e8b274640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3af8523a0 0x55d3af857760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35422/0 (socket says 172.21.15.5:35422) 2024-05-20T18:37:39.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.008+0000 7f2e8b274640 1 -- 172.21.15.5:0/1268399878 learned_addr learned my addr 172.21.15.5:0/1268399878 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:39.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.008+0000 7f2e83fff640 1 --2- 172.21.15.5:0/1268399878 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d3af8527b0 0x55d3af8eb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.009+0000 7f2e8ba75640 1 --2- 172.21.15.5:0/1268399878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3af907430 0x55d3af902240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.009+0000 7f2e8b274640 1 -- 172.21.15.5:0/1268399878 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d3af8527b0 msgr2=0x55d3af8eb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:39.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.009+0000 7f2e8b274640 1 --2- 172.21.15.5:0/1268399878 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d3af8527b0 0x55d3af8eb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.009+0000 7f2e8b274640 1 -- 172.21.15.5:0/1268399878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3af907430 msgr2=0x55d3af902240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:39.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.009+0000 7f2e8b274640 1 --2- 172.21.15.5:0/1268399878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3af907430 0x55d3af902240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.009+0000 7f2e8b274640 1 -- 172.21.15.5:0/1268399878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3af754cb0 con 0x55d3af8523a0 2024-05-20T18:37:39.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.009+0000 7f2e8b274640 1 --2- 172.21.15.5:0/1268399878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3af8523a0 0x55d3af857760 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c009750 tx=0x7f2e7c005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3345fc452252f2d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.009+0000 7f2e8a272640 1 -- 172.21.15.5:0/1268399878 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c019070 con 0x55d3af8523a0 2024-05-20T18:37:39.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.009+0000 7f2e8a272640 1 -- 172.21.15.5:0/1268399878 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e7c008890 con 0x55d3af8523a0 2024-05-20T18:37:39.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.009+0000 7f2e8a272640 1 -- 172.21.15.5:0/1268399878 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c008c60 con 0x55d3af8523a0 2024-05-20T18:37:39.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.010+0000 7f2e92416f40 1 -- 172.21.15.5:0/1268399878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3af8523a0 msgr2=0x55d3af857760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.010+0000 7f2e92416f40 1 --2- 172.21.15.5:0/1268399878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3af8523a0 0x55d3af857760 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c009750 tx=0x7f2e7c005a00 comp rx=0 tx=0).stop 2024-05-20T18:37:39.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.010+0000 7f2e92416f40 1 -- 172.21.15.5:0/1268399878 shutdown_connections 2024-05-20T18:37:39.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.010+0000 7f2e92416f40 1 --2- 172.21.15.5:0/1268399878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3af907430 0x55d3af902240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.010+0000 7f2e92416f40 1 --2- 172.21.15.5:0/1268399878 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d3af8527b0 0x55d3af8eb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.010+0000 7f2e92416f40 1 --2- 172.21.15.5:0/1268399878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3af8523a0 0x55d3af857760 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.010+0000 7f2e92416f40 1 -- 172.21.15.5:0/1268399878 >> 172.21.15.5:0/1268399878 conn(0x55d3af74dac0 msgr2=0x55d3af8f7230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:39.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.010+0000 7f2e92416f40 1 -- 172.21.15.5:0/1268399878 shutdown_connections 2024-05-20T18:37:39.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.010+0000 7f2e92416f40 1 -- 172.21.15.5:0/1268399878 wait complete. 2024-05-20T18:37:39.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.011+0000 7f2e92416f40 1 Processor -- start 2024-05-20T18:37:39.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.011+0000 7f2e92416f40 1 -- start start 2024-05-20T18:37:39.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.012+0000 7f2e92416f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3af8523a0 0x55d3af8eea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.012+0000 7f2e92416f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3af8527b0 0x55d3af86dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.013+0000 7f2e92416f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d3af907430 0x55d3af86e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.013+0000 7f2e92416f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d3af8f7b60 con 0x55d3af8527b0 2024-05-20T18:37:39.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.013+0000 7f2e92416f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d3af8fa1e0 con 0x55d3af8523a0 2024-05-20T18:37:39.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.013+0000 7f2e92416f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55d3af86cd00 con 0x55d3af907430 2024-05-20T18:37:39.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.013+0000 7f2e83fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3af8527b0 0x55d3af86dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.013+0000 7f2e83fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3af8527b0 0x55d3af86dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35424/0 (socket says 172.21.15.5:35424) 2024-05-20T18:37:39.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.013+0000 7f2e83fff640 1 -- 172.21.15.5:0/475699564 learned_addr learned my addr 172.21.15.5:0/475699564 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:39.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.013+0000 7f2e8b274640 1 --2- 172.21.15.5:0/475699564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3af8523a0 0x55d3af8eea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.013+0000 7f2e8ba75640 1 --2- 172.21.15.5:0/475699564 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d3af907430 0x55d3af86e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.013+0000 7f2e83fff640 1 -- 172.21.15.5:0/475699564 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d3af907430 msgr2=0x55d3af86e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.013+0000 7f2e83fff640 1 --2- 172.21.15.5:0/475699564 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d3af907430 0x55d3af86e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.013+0000 7f2e83fff640 1 -- 172.21.15.5:0/475699564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3af8523a0 msgr2=0x55d3af8eea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.013+0000 7f2e83fff640 1 --2- 172.21.15.5:0/475699564 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d3af8523a0 0x55d3af8eea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.013+0000 7f2e83fff640 1 -- 172.21.15.5:0/475699564 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3af8eef70 con 0x55d3af8527b0 2024-05-20T18:37:39.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.013+0000 7f2e83fff640 1 --2- 172.21.15.5:0/475699564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3af8527b0 0x55d3af86dc60 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f2e7400efc0 tx=0x7f2e7400c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.013+0000 7f2e89270640 1 -- 172.21.15.5:0/475699564 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e74008070 con 0x55d3af8527b0 2024-05-20T18:37:39.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.014+0000 7f2e89270640 1 -- 172.21.15.5:0/475699564 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e74011680 con 0x55d3af8527b0 2024-05-20T18:37:39.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.014+0000 7f2e92416f40 1 -- 172.21.15.5:0/475699564 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d3af857900 con 0x55d3af8527b0 2024-05-20T18:37:39.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.014+0000 7f2e89270640 1 -- 172.21.15.5:0/475699564 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e74011a70 con 0x55d3af8527b0 2024-05-20T18:37:39.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.014+0000 7f2e92416f40 1 -- 172.21.15.5:0/475699564 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3af8f9f20 con 0x55d3af8527b0 2024-05-20T18:37:39.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.015+0000 7f2e89270640 1 -- 172.21.15.5:0/475699564 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2e74011c10 con 0x55d3af8527b0 2024-05-20T18:37:39.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.016+0000 7f2e89270640 1 --2- 172.21.15.5:0/475699564 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2e6c06a840 0x7f2e6c06cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.016+0000 7f2e8b274640 1 --2- 172.21.15.5:0/475699564 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2e6c06a840 0x7f2e6c06cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.016+0000 7f2e89270640 1 -- 172.21.15.5:0/475699564 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7f2e7408a230 con 0x55d3af8527b0 2024-05-20T18:37:39.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.016+0000 7f2e92416f40 1 --2- 172.21.15.5:0/475699564 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x55d3af871f50 0x55d3af9a26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.016+0000 7f2e92416f40 1 -- 172.21.15.5:0/475699564 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:1 18.12 18:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x55d3af873d00 con 0x55d3af871f50 2024-05-20T18:37:39.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.016+0000 7f2e8ba75640 1 --2- 172.21.15.5:0/475699564 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x55d3af871f50 0x55d3af9a26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.016+0000 7f2e8b274640 1 --2- 172.21.15.5:0/475699564 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2e6c06a840 0x7f2e6c06cd50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c017040 tx=0x7f2e7c008000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.016+0000 7f2e8ba75640 1 --2- 172.21.15.5:0/475699564 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x55d3af871f50 0x55d3af9a26b0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.017+0000 7f2e8ba75640 1 -- 172.21.15.5:0/475699564 <== osd.2 v2:172.21.15.5:6800/921679764 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f2e84052950 con 0x55d3af871f50 2024-05-20T18:37:39.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.017+0000 7f2e92416f40 1 -- 172.21.15.5:0/475699564 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x55d3af871f50 msgr2=0x55d3af9a26b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.017+0000 7f2e92416f40 1 --2- 172.21.15.5:0/475699564 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x55d3af871f50 0x55d3af9a26b0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.017+0000 7f2e92416f40 1 -- 172.21.15.5:0/475699564 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2e6c06a840 msgr2=0x7f2e6c06cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.017+0000 7f2e92416f40 1 --2- 172.21.15.5:0/475699564 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f2e6c06a840 0x7f2e6c06cd50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c017040 tx=0x7f2e7c008000 comp rx=0 tx=0).stop 2024-05-20T18:37:39.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.017+0000 7f2e92416f40 1 -- 172.21.15.5:0/475699564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3af8527b0 msgr2=0x55d3af86dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.017+0000 7f2e92416f40 1 --2- 172.21.15.5:0/475699564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3af8527b0 0x55d3af86dc60 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f2e7400efc0 tx=0x7f2e7400c710 comp rx=0 tx=0).stop 2024-05-20T18:37:39.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.017+0000 7f2e8ba75640 1 -- 172.21.15.5:0/475699564 reap_dead start 2024-05-20T18:37:39.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.017+0000 7f2e92416f40 1 -- 172.21.15.5:0/475699564 shutdown_connections 2024-05-20T18:37:39.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.017+0000 7f2e92416f40 1 -- 172.21.15.5:0/475699564 >> 172.21.15.5:0/475699564 conn(0x55d3af74dac0 msgr2=0x55d3af8f9600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:39.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.017+0000 7f2e92416f40 1 -- 172.21.15.5:0/475699564 shutdown_connections 2024-05-20T18:37:39.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.017+0000 7f2e92416f40 1 -- 172.21.15.5:0/475699564 wait complete. 2024-05-20T18:37:39.021 INFO:teuthology.orchestra.run.smithi005.stdout:mytag123 2024-05-20T18:37:39.021 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-05-20T18:37:39.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.076+0000 7f9907d78f40 1 Processor -- start 2024-05-20T18:37:39.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.076+0000 7f9907d78f40 1 -- start start 2024-05-20T18:37:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.077+0000 7f9907d78f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555be48ba970 0x555be4946480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.077+0000 7f9907d78f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555be48bc5f0 0x555be489eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.077+0000 7f9907d78f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be494b010 0x555be489f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.077+0000 7f9907d78f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555be48a1400 con 0x555be494b010 2024-05-20T18:37:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.077+0000 7f9907d78f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555be481ac80 con 0x555be48ba970 2024-05-20T18:37:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.077+0000 7f9907d78f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x555be48a16a0 con 0x555be48bc5f0 2024-05-20T18:37:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.078+0000 7f990204a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be494b010 0x555be489f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.078+0000 7f990204a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be494b010 0x555be489f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35430/0 (socket says 172.21.15.5:35430) 2024-05-20T18:37:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.078+0000 7f990204a640 1 -- 172.21.15.5:0/2779915636 learned_addr learned my addr 172.21.15.5:0/2779915636 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.078+0000 7f9901849640 1 --2- 172.21.15.5:0/2779915636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555be48ba970 0x555be4946480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.078+0000 7f9901048640 1 --2- 172.21.15.5:0/2779915636 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555be48bc5f0 0x555be489eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.078+0000 7f990204a640 1 -- 172.21.15.5:0/2779915636 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555be48bc5f0 msgr2=0x555be489eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.078+0000 7f990204a640 1 --2- 172.21.15.5:0/2779915636 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555be48bc5f0 0x555be489eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.078+0000 7f990204a640 1 -- 172.21.15.5:0/2779915636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555be48ba970 msgr2=0x555be4946480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.078+0000 7f990204a640 1 --2- 172.21.15.5:0/2779915636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555be48ba970 0x555be4946480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.078+0000 7f990204a640 1 -- 172.21.15.5:0/2779915636 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555be479fcb0 con 0x555be494b010 2024-05-20T18:37:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.078+0000 7f9901849640 1 --2- 172.21.15.5:0/2779915636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555be48ba970 0x555be4946480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:37:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.078+0000 7f990204a640 1 --2- 172.21.15.5:0/2779915636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be494b010 0x555be489f0a0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f98fc00a800 tx=0x7f98fc00acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c75d6b9c6f20115 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.078+0000 7f98f3fff640 1 -- 172.21.15.5:0/2779915636 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc016020 con 0x555be494b010 2024-05-20T18:37:39.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.078+0000 7f98f3fff640 1 -- 172.21.15.5:0/2779915636 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98fc00d960 con 0x555be494b010 2024-05-20T18:37:39.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.078+0000 7f98f3fff640 1 -- 172.21.15.5:0/2779915636 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc00dd10 con 0x555be494b010 2024-05-20T18:37:39.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.079+0000 7f9907d78f40 1 -- 172.21.15.5:0/2779915636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be494b010 msgr2=0x555be489f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.079+0000 7f9907d78f40 1 --2- 172.21.15.5:0/2779915636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be494b010 0x555be489f0a0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f98fc00a800 tx=0x7f98fc00acc0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.079+0000 7f9907d78f40 1 -- 172.21.15.5:0/2779915636 shutdown_connections 2024-05-20T18:37:39.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.079+0000 7f9907d78f40 1 --2- 172.21.15.5:0/2779915636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be494b010 0x555be489f0a0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.079+0000 7f9907d78f40 1 --2- 172.21.15.5:0/2779915636 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555be48bc5f0 0x555be489eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.079+0000 7f9907d78f40 1 --2- 172.21.15.5:0/2779915636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555be48ba970 0x555be4946480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.079+0000 7f9907d78f40 1 -- 172.21.15.5:0/2779915636 >> 172.21.15.5:0/2779915636 conn(0x555be4798ac0 msgr2=0x555be4947800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:39.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.079+0000 7f9907d78f40 1 -- 172.21.15.5:0/2779915636 shutdown_connections 2024-05-20T18:37:39.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.079+0000 7f9907d78f40 1 -- 172.21.15.5:0/2779915636 wait complete. 2024-05-20T18:37:39.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.080+0000 7f9907d78f40 1 Processor -- start 2024-05-20T18:37:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.080+0000 7f9907d78f40 1 -- start start 2024-05-20T18:37:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.081+0000 7f9907d78f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555be48ba970 0x555be4953bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.081+0000 7f9907d78f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be48bc5f0 0x555be4939a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.081+0000 7f9907d78f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555be494b010 0x555be493a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.081+0000 7f9907d78f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555be489e180 con 0x555be48bc5f0 2024-05-20T18:37:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.081+0000 7f9907d78f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555be493ae50 con 0x555be48ba970 2024-05-20T18:37:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.081+0000 7f9907d78f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x555be49420b0 con 0x555be494b010 2024-05-20T18:37:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.081+0000 7f9901048640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be48bc5f0 0x555be4939a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.081+0000 7f9901048640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be48bc5f0 0x555be4939a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35434/0 (socket says 172.21.15.5:35434) 2024-05-20T18:37:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.081+0000 7f9901048640 1 -- 172.21.15.5:0/932029312 learned_addr learned my addr 172.21.15.5:0/932029312 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.081+0000 7f9901849640 1 --2- 172.21.15.5:0/932029312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555be48ba970 0x555be4953bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.081+0000 7f990204a640 1 --2- 172.21.15.5:0/932029312 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555be494b010 0x555be493a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.082+0000 7f9901048640 1 -- 172.21.15.5:0/932029312 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555be494b010 msgr2=0x555be493a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.082+0000 7f9901048640 1 --2- 172.21.15.5:0/932029312 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555be494b010 0x555be493a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.082+0000 7f9901048640 1 -- 172.21.15.5:0/932029312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555be48ba970 msgr2=0x555be4953bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.082+0000 7f9901048640 1 --2- 172.21.15.5:0/932029312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555be48ba970 0x555be4953bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.082+0000 7f9901048640 1 -- 172.21.15.5:0/932029312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555be4954110 con 0x555be48bc5f0 2024-05-20T18:37:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.082+0000 7f9901048640 1 --2- 172.21.15.5:0/932029312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be48bc5f0 0x555be4939a30 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f98f800c330 tx=0x7f98f800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.082+0000 7f98f2ffd640 1 -- 172.21.15.5:0/932029312 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f8008070 con 0x555be48bc5f0 2024-05-20T18:37:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.082+0000 7f98f2ffd640 1 -- 172.21.15.5:0/932029312 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98f8011680 con 0x555be48bc5f0 2024-05-20T18:37:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.082+0000 7f98f2ffd640 1 -- 172.21.15.5:0/932029312 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f8011a30 con 0x555be48bc5f0 2024-05-20T18:37:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.082+0000 7f9907d78f40 1 -- 172.21.15.5:0/932029312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555be48a2730 con 0x555be48bc5f0 2024-05-20T18:37:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.082+0000 7f9907d78f40 1 -- 172.21.15.5:0/932029312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555be48a2cf0 con 0x555be48bc5f0 2024-05-20T18:37:39.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.084+0000 7f98f2ffd640 1 -- 172.21.15.5:0/932029312 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f98f8011bd0 con 0x555be48bc5f0 2024-05-20T18:37:39.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.084+0000 7f98f2ffd640 1 --2- 172.21.15.5:0/932029312 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f98e406a810 0x7f98e406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.084+0000 7f98f2ffd640 1 -- 172.21.15.5:0/932029312 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7f98f808a3c0 con 0x555be48bc5f0 2024-05-20T18:37:39.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.084+0000 7f9901849640 1 --2- 172.21.15.5:0/932029312 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f98e406a810 0x7f98e406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.085+0000 7f9907d78f40 1 --2- 172.21.15.5:0/932029312 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x555be493a710 0x555be489efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.085+0000 7f9907d78f40 1 -- 172.21.15.5:0/932029312 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:1 18.19 18:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x555be49ed550 con 0x555be493a710 2024-05-20T18:37:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.085+0000 7f990204a640 1 --2- 172.21.15.5:0/932029312 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x555be493a710 0x555be489efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.085+0000 7f9901849640 1 --2- 172.21.15.5:0/932029312 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f98e406a810 0x7f98e406cd20 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f98f4056000 tx=0x7f98f4050000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.085+0000 7f990204a640 1 --2- 172.21.15.5:0/932029312 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x555be493a710 0x555be489efe0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.086+0000 7f990204a640 1 -- 172.21.15.5:0/932029312 <== osd.4 v2:172.21.15.63:6800/3673324406 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (12) out=12b] v0'0 uv23 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f98fc00dc70 con 0x555be493a710 2024-05-20T18:37:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.086+0000 7f9907d78f40 1 -- 172.21.15.5:0/932029312 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x555be493a710 msgr2=0x555be489efe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.086+0000 7f9907d78f40 1 --2- 172.21.15.5:0/932029312 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x555be493a710 0x555be489efe0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.086+0000 7f9907d78f40 1 -- 172.21.15.5:0/932029312 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f98e406a810 msgr2=0x7f98e406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.086+0000 7f9907d78f40 1 --2- 172.21.15.5:0/932029312 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f98e406a810 0x7f98e406cd20 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f98f4056000 tx=0x7f98f4050000 comp rx=0 tx=0).stop 2024-05-20T18:37:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.086+0000 7f9907d78f40 1 -- 172.21.15.5:0/932029312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be48bc5f0 msgr2=0x555be4939a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.086+0000 7f9907d78f40 1 --2- 172.21.15.5:0/932029312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555be48bc5f0 0x555be4939a30 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f98f800c330 tx=0x7f98f800ef90 comp rx=0 tx=0).stop 2024-05-20T18:37:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.086+0000 7f990204a640 1 -- 172.21.15.5:0/932029312 reap_dead start 2024-05-20T18:37:39.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.086+0000 7f9907d78f40 1 -- 172.21.15.5:0/932029312 shutdown_connections 2024-05-20T18:37:39.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.086+0000 7f9907d78f40 1 -- 172.21.15.5:0/932029312 >> 172.21.15.5:0/932029312 conn(0x555be4798ac0 msgr2=0x555be4947800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:39.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.086+0000 7f9907d78f40 1 -- 172.21.15.5:0/932029312 shutdown_connections 2024-05-20T18:37:39.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.086+0000 7f9907d78f40 1 -- 172.21.15.5:0/932029312 wait complete. 2024-05-20T18:37:39.091 INFO:teuthology.orchestra.run.smithi005.stdout:mytag123 2024-05-20T18:37:39.091 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2024-05-20T18:37:39.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.146+0000 7f9420c85f40 1 Processor -- start 2024-05-20T18:37:39.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.146+0000 7f9420c85f40 1 -- start start 2024-05-20T18:37:39.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.147+0000 7f9420c85f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96495f40 0x562c9651dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.147+0000 7f9420c85f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562c96497170 0x562c96497ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.148+0000 7f9420c85f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c96497580 0x562c964983e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.148+0000 7f9420c85f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562c96498b60 con 0x562c96495f40 2024-05-20T18:37:39.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.148+0000 7f9420c85f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562c963f7c80 con 0x562c96497580 2024-05-20T18:37:39.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.148+0000 7f9420c85f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x562c96498e00 con 0x562c96497170 2024-05-20T18:37:39.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.148+0000 7f941a754640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96495f40 0x562c9651dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.148+0000 7f941a754640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96495f40 0x562c9651dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35454/0 (socket says 172.21.15.5:35454) 2024-05-20T18:37:39.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.148+0000 7f941a754640 1 -- 172.21.15.5:0/793266229 learned_addr learned my addr 172.21.15.5:0/793266229 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:39.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.148+0000 7f9419f53640 1 --2- 172.21.15.5:0/793266229 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562c96497170 0x562c96497ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.148+0000 7f941a754640 1 -- 172.21.15.5:0/793266229 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562c96497170 msgr2=0x562c96497ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:39.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.148+0000 7f941af55640 1 --2- 172.21.15.5:0/793266229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c96497580 0x562c964983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.148+0000 7f941a754640 1 --2- 172.21.15.5:0/793266229 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562c96497170 0x562c96497ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.148+0000 7f941a754640 1 -- 172.21.15.5:0/793266229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c96497580 msgr2=0x562c964983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:39.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.148+0000 7f941a754640 1 --2- 172.21.15.5:0/793266229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c96497580 0x562c964983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.148+0000 7f941a754640 1 -- 172.21.15.5:0/793266229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c9637ccb0 con 0x562c96495f40 2024-05-20T18:37:39.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.148+0000 7f9419f53640 1 --2- 172.21.15.5:0/793266229 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562c96497170 0x562c96497ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:39.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.148+0000 7f941af55640 1 --2- 172.21.15.5:0/793266229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c96497580 0x562c964983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:39.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.148+0000 7f941a754640 1 --2- 172.21.15.5:0/793266229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96495f40 0x562c9651dd50 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f9414009780 tx=0x7f9414005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863fb40f6790503b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.148+0000 7f9418f51640 1 -- 172.21.15.5:0/793266229 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9414008870 con 0x562c96495f40 2024-05-20T18:37:39.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.148+0000 7f9418f51640 1 -- 172.21.15.5:0/793266229 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9414018070 con 0x562c96495f40 2024-05-20T18:37:39.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.148+0000 7f9418f51640 1 -- 172.21.15.5:0/793266229 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9414008c30 con 0x562c96495f40 2024-05-20T18:37:39.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.149+0000 7f9420c85f40 1 -- 172.21.15.5:0/793266229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96495f40 msgr2=0x562c9651dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.149+0000 7f9420c85f40 1 --2- 172.21.15.5:0/793266229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96495f40 0x562c9651dd50 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f9414009780 tx=0x7f9414005a00 comp rx=0 tx=0).stop 2024-05-20T18:37:39.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.149+0000 7f9420c85f40 1 -- 172.21.15.5:0/793266229 shutdown_connections 2024-05-20T18:37:39.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.149+0000 7f9420c85f40 1 --2- 172.21.15.5:0/793266229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c96497580 0x562c964983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.149+0000 7f9420c85f40 1 --2- 172.21.15.5:0/793266229 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562c96497170 0x562c96497ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.149+0000 7f9420c85f40 1 --2- 172.21.15.5:0/793266229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96495f40 0x562c9651dd50 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.149+0000 7f9420c85f40 1 -- 172.21.15.5:0/793266229 >> 172.21.15.5:0/793266229 conn(0x562c96375ac0 msgr2=0x562c96528f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:39.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.149+0000 7f9420c85f40 1 -- 172.21.15.5:0/793266229 shutdown_connections 2024-05-20T18:37:39.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.149+0000 7f9420c85f40 1 -- 172.21.15.5:0/793266229 wait complete. 2024-05-20T18:37:39.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.150+0000 7f9420c85f40 1 Processor -- start 2024-05-20T18:37:39.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.150+0000 7f9420c85f40 1 -- start start 2024-05-20T18:37:39.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.151+0000 7f9420c85f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c96495f40 0x562c96496350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.151+0000 7f9420c85f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562c96497170 0x562c96496890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.151+0000 7f9420c85f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96497580 0x562c965273c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.151+0000 7f9420c85f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562c9647abc0 con 0x562c96497580 2024-05-20T18:37:39.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.151+0000 7f9420c85f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562c9644ff60 con 0x562c96495f40 2024-05-20T18:37:39.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.151+0000 7f9420c85f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x562c9647ae10 con 0x562c96497170 2024-05-20T18:37:39.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.151+0000 7f941af55640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96497580 0x562c965273c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.151+0000 7f941af55640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96497580 0x562c965273c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35464/0 (socket says 172.21.15.5:35464) 2024-05-20T18:37:39.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.151+0000 7f941af55640 1 -- 172.21.15.5:0/2182599476 learned_addr learned my addr 172.21.15.5:0/2182599476 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:39.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.152+0000 7f9419f53640 1 --2- 172.21.15.5:0/2182599476 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562c96497170 0x562c96496890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.152+0000 7f941a754640 1 --2- 172.21.15.5:0/2182599476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c96495f40 0x562c96496350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.152+0000 7f941af55640 1 -- 172.21.15.5:0/2182599476 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562c96497170 msgr2=0x562c96496890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.152+0000 7f941af55640 1 --2- 172.21.15.5:0/2182599476 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562c96497170 0x562c96496890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.152+0000 7f941af55640 1 -- 172.21.15.5:0/2182599476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c96495f40 msgr2=0x562c96496350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.152+0000 7f941af55640 1 --2- 172.21.15.5:0/2182599476 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562c96495f40 0x562c96496350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.152+0000 7f941af55640 1 -- 172.21.15.5:0/2182599476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c9650d7b0 con 0x562c96497580 2024-05-20T18:37:39.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.152+0000 7f941af55640 1 --2- 172.21.15.5:0/2182599476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96497580 0x562c965273c0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f940c002150 tx=0x7f940c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.152+0000 7f94137fe640 1 -- 172.21.15.5:0/2182599476 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c0062d0 con 0x562c96497580 2024-05-20T18:37:39.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.152+0000 7f94137fe640 1 -- 172.21.15.5:0/2182599476 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f940c003450 con 0x562c96497580 2024-05-20T18:37:39.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.152+0000 7f9420c85f40 1 -- 172.21.15.5:0/2182599476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562c9651fcb0 con 0x562c96497580 2024-05-20T18:37:39.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.152+0000 7f94137fe640 1 -- 172.21.15.5:0/2182599476 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c003800 con 0x562c96497580 2024-05-20T18:37:39.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.152+0000 7f9420c85f40 1 -- 172.21.15.5:0/2182599476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c96520660 con 0x562c96497580 2024-05-20T18:37:39.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.154+0000 7f94137fe640 1 -- 172.21.15.5:0/2182599476 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f940c01a020 con 0x562c96497580 2024-05-20T18:37:39.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.154+0000 7f94137fe640 1 --2- 172.21.15.5:0/2182599476 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f93fc06a840 0x7f93fc06cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.155+0000 7f941a754640 1 --2- 172.21.15.5:0/2182599476 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f93fc06a840 0x7f93fc06cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.155+0000 7f94137fe640 1 -- 172.21.15.5:0/2182599476 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7f940c0898b0 con 0x562c96497580 2024-05-20T18:37:39.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.155+0000 7f9420c85f40 1 --2- 172.21.15.5:0/2182599476 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x562c9647c390 0x562c96527900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.155+0000 7f9420c85f40 1 -- 172.21.15.5:0/2182599476 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:1 18.5 18:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x562c96527e40 con 0x562c9647c390 2024-05-20T18:37:39.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.155+0000 7f9419f53640 1 --2- 172.21.15.5:0/2182599476 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x562c9647c390 0x562c96527900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.155+0000 7f941a754640 1 --2- 172.21.15.5:0/2182599476 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f93fc06a840 0x7f93fc06cd50 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f9414015040 tx=0x7f9414012000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.155+0000 7f9419f53640 1 --2- 172.21.15.5:0/2182599476 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x562c9647c390 0x562c96527900 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.156+0000 7f9419f53640 1 -- 172.21.15.5:0/2182599476 <== osd.1 v2:172.21.15.5:6815/550664725 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv12 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f940405c040 con 0x562c9647c390 2024-05-20T18:37:39.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.156+0000 7f9420c85f40 1 -- 172.21.15.5:0/2182599476 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x562c9647c390 msgr2=0x562c96527900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.156+0000 7f9420c85f40 1 --2- 172.21.15.5:0/2182599476 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x562c9647c390 0x562c96527900 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.156+0000 7f9420c85f40 1 -- 172.21.15.5:0/2182599476 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f93fc06a840 msgr2=0x7f93fc06cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.156+0000 7f9420c85f40 1 --2- 172.21.15.5:0/2182599476 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f93fc06a840 0x7f93fc06cd50 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f9414015040 tx=0x7f9414012000 comp rx=0 tx=0).stop 2024-05-20T18:37:39.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.156+0000 7f9420c85f40 1 -- 172.21.15.5:0/2182599476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96497580 msgr2=0x562c965273c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.156+0000 7f9420c85f40 1 --2- 172.21.15.5:0/2182599476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c96497580 0x562c965273c0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f940c002150 tx=0x7f940c00c520 comp rx=0 tx=0).stop 2024-05-20T18:37:39.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.156+0000 7f941af55640 1 -- 172.21.15.5:0/2182599476 reap_dead start 2024-05-20T18:37:39.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.156+0000 7f9420c85f40 1 -- 172.21.15.5:0/2182599476 shutdown_connections 2024-05-20T18:37:39.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.156+0000 7f9420c85f40 1 -- 172.21.15.5:0/2182599476 >> 172.21.15.5:0/2182599476 conn(0x562c96375ac0 msgr2=0x562c96525200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:39.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.156+0000 7f9420c85f40 1 -- 172.21.15.5:0/2182599476 shutdown_connections 2024-05-20T18:37:39.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.156+0000 7f9420c85f40 1 -- 172.21.15.5:0/2182599476 wait complete. 2024-05-20T18:37:39.161 INFO:teuthology.orchestra.run.smithi005.stdout:mytag123 2024-05-20T18:37:39.161 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-05-20T18:37:39.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.277+0000 7f680b0ca640 1 Processor -- start 2024-05-20T18:37:39.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.277+0000 7f680b0ca640 1 -- start start 2024-05-20T18:37:39.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.278+0000 7f680b0ca640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f680415ca00 0x7f680415fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.278+0000 7f680b0ca640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68040b51c0 con 0x7f680406c4c0 2024-05-20T18:37:39.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.278+0000 7f680b0ca640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f680415bf20 con 0x7f68040b0020 2024-05-20T18:37:39.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.278+0000 7f680b0ca640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f680415c200 con 0x7f680415ca00 2024-05-20T18:37:39.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.278+0000 7f6803fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f680406c4c0 0x7f68040b3850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:32882/0 (socket says 172.21.15.5:32882) 2024-05-20T18:37:39.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.278+0000 7f6803fff640 1 -- 172.21.15.5:0/1818132540 learned_addr learned my addr 172.21.15.5:0/1818132540 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:39.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.278+0000 7f6809667640 1 --2- 172.21.15.5:0/1818132540 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f680415ca00 0x7f680415fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.278+0000 7f68037fe640 1 -- 172.21.15.5:0/1818132540 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1241556009 0 0) 0x7f68040b51c0 con 0x7f680406c4c0 2024-05-20T18:37:39.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.279+0000 7f68037fe640 1 -- 172.21.15.5:0/1818132540 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f67f0005730 con 0x7f680406c4c0 2024-05-20T18:37:39.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.279+0000 7f68037fe640 1 -- 172.21.15.5:0/1818132540 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 783604172 0 0) 0x7f680415bf20 con 0x7f68040b0020 2024-05-20T18:37:39.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.279+0000 7f6809667640 1 -- 172.21.15.5:0/1818132540 >> v1:172.21.15.63:6789/0 conn(0x7f68040b0020 legacy=0x7f680406bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.279+0000 7f6809667640 1 -- 172.21.15.5:0/1818132540 >> v1:172.21.15.5:6789/0 conn(0x7f680406c4c0 legacy=0x7f68040b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.279+0000 7f6809667640 1 -- 172.21.15.5:0/1818132540 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680415c4e0 con 0x7f680415ca00 2024-05-20T18:37:39.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.280+0000 7f6809667640 1 --2- 172.21.15.5:0/1818132540 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f680415ca00 0x7f680415fd80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009dc0 tx=0x7f67fc02a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8b9d8491f693e7c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.280+0000 7f68037fe640 1 -- 172.21.15.5:0/1818132540 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc02ae60 con 0x7f680415ca00 2024-05-20T18:37:39.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.280+0000 7f68037fe640 1 -- 172.21.15.5:0/1818132540 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67fc007740 con 0x7f680415ca00 2024-05-20T18:37:39.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.280+0000 7f68037fe640 1 -- 172.21.15.5:0/1818132540 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc007af0 con 0x7f680415ca00 2024-05-20T18:37:39.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.280+0000 7f680b0ca640 1 -- 172.21.15.5:0/1818132540 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f680415ca00 msgr2=0x7f680415fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.280+0000 7f680b0ca640 1 --2- 172.21.15.5:0/1818132540 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f680415ca00 0x7f680415fd80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009dc0 tx=0x7f67fc02a640 comp rx=0 tx=0).stop 2024-05-20T18:37:39.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.281+0000 7f680b0ca640 1 -- 172.21.15.5:0/1818132540 shutdown_connections 2024-05-20T18:37:39.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.281+0000 7f680b0ca640 1 --2- 172.21.15.5:0/1818132540 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f680415ca00 0x7f680415fd80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.281+0000 7f680b0ca640 1 -- 172.21.15.5:0/1818132540 >> 172.21.15.5:0/1818132540 conn(0x7f68040cd2a0 msgr2=0x7f68040cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:39.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.281+0000 7f680b0ca640 1 -- 172.21.15.5:0/1818132540 shutdown_connections 2024-05-20T18:37:39.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.281+0000 7f680b0ca640 1 -- 172.21.15.5:0/1818132540 wait complete. 2024-05-20T18:37:39.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.282+0000 7f680b0ca640 1 Processor -- start 2024-05-20T18:37:39.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.282+0000 7f680b0ca640 1 -- start start 2024-05-20T18:37:39.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.284+0000 7f680b0ca640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68040b0020 0x7f6804140e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.284+0000 7f680b0ca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f680415ca00 0x7f68041579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.284+0000 7f680b0ca640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68041ee0a0 0x7f680415e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.284+0000 7f680b0ca640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f68040b4c00 con 0x7f680415ca00 2024-05-20T18:37:39.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.284+0000 7f680b0ca640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f68040b54a0 con 0x7f68041ee0a0 2024-05-20T18:37:39.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.284+0000 7f680b0ca640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f68040b4d80 con 0x7f68040b0020 2024-05-20T18:37:39.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.284+0000 7f6803fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f680415ca00 0x7f68041579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.284+0000 7f6803fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f680415ca00 0x7f68041579e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35466/0 (socket says 172.21.15.5:35466) 2024-05-20T18:37:39.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.284+0000 7f6803fff640 1 -- 172.21.15.5:0/742802371 learned_addr learned my addr 172.21.15.5:0/742802371 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:39.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.284+0000 7f6808e66640 1 --2- 172.21.15.5:0/742802371 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68040b0020 0x7f6804140e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.284+0000 7f6809667640 1 --2- 172.21.15.5:0/742802371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68041ee0a0 0x7f680415e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.284+0000 7f6803fff640 1 -- 172.21.15.5:0/742802371 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68040b0020 msgr2=0x7f6804140e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.284+0000 7f6803fff640 1 --2- 172.21.15.5:0/742802371 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68040b0020 0x7f6804140e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.284+0000 7f6803fff640 1 -- 172.21.15.5:0/742802371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68041ee0a0 msgr2=0x7f680415e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.284+0000 7f6803fff640 1 --2- 172.21.15.5:0/742802371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68041ee0a0 0x7f680415e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.284+0000 7f6803fff640 1 -- 172.21.15.5:0/742802371 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68041f0cb0 con 0x7f680415ca00 2024-05-20T18:37:39.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.284+0000 7f6809667640 1 --2- 172.21.15.5:0/742802371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68041ee0a0 0x7f680415e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:39.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.285+0000 7f6803fff640 1 --2- 172.21.15.5:0/742802371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f680415ca00 0x7f68041579e0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f67ec00c400 tx=0x7f67ec00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.285+0000 7f68017fa640 1 -- 172.21.15.5:0/742802371 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec017070 con 0x7f680415ca00 2024-05-20T18:37:39.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.285+0000 7f68017fa640 1 -- 172.21.15.5:0/742802371 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67ec005d00 con 0x7f680415ca00 2024-05-20T18:37:39.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.285+0000 7f680b0ca640 1 -- 172.21.15.5:0/742802371 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68041f1b80 con 0x7f680415ca00 2024-05-20T18:37:39.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.285+0000 7f68017fa640 1 -- 172.21.15.5:0/742802371 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec007310 con 0x7f680415ca00 2024-05-20T18:37:39.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.285+0000 7f680b0ca640 1 -- 172.21.15.5:0/742802371 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68041f1250 con 0x7f680415ca00 2024-05-20T18:37:39.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.287+0000 7f68017fa640 1 -- 172.21.15.5:0/742802371 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f67ec01a070 con 0x7f680415ca00 2024-05-20T18:37:39.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.288+0000 7f68017fa640 1 --2- 172.21.15.5:0/742802371 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f67e406be60 0x7f67e406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.288+0000 7f68017fa640 1 -- 172.21.15.5:0/742802371 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7f67ec08a4a0 con 0x7f680415ca00 2024-05-20T18:37:39.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.288+0000 7f6808e66640 1 --2- 172.21.15.5:0/742802371 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f67e406be60 0x7f67e406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.288+0000 7f6808e66640 1 --2- 172.21.15.5:0/742802371 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f67e406be60 0x7f67e406e370 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f67f4007fd0 tx=0x7f67f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.289+0000 7f6811f73740 1 Processor -- start 2024-05-20T18:37:39.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.289+0000 7f6811f73740 1 -- start start 2024-05-20T18:37:39.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.290+0000 7f6811f73740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562051d68390 0x562052132b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.290+0000 7f6811f73740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562051d3f4d0 0x56205214d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.290+0000 7f6811f73740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562051d476b0 0x56205214ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.290+0000 7f6811f73740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56205206cd00 con 0x562051d68390 2024-05-20T18:37:39.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.290+0000 7f6811f73740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5620520144c0 con 0x562051d476b0 2024-05-20T18:37:39.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.290+0000 7f6811f73740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x562052068cc0 con 0x562051d3f4d0 2024-05-20T18:37:39.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.290+0000 7f6808e66640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562051d68390 0x562052132b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.290+0000 7f6808e66640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562051d68390 0x562052132b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35480/0 (socket says 172.21.15.5:35480) 2024-05-20T18:37:39.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.290+0000 7f6808e66640 1 -- 172.21.15.5:0/4128162674 learned_addr learned my addr 172.21.15.5:0/4128162674 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:39.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.290+0000 7f6803fff640 1 --2- 172.21.15.5:0/4128162674 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562051d3f4d0 0x56205214d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.290+0000 7f6809667640 1 --2- 172.21.15.5:0/4128162674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562051d476b0 0x56205214ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.290+0000 7f6808e66640 1 -- 172.21.15.5:0/4128162674 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562051d3f4d0 msgr2=0x56205214d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:39.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.290+0000 7f6808e66640 1 --2- 172.21.15.5:0/4128162674 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562051d3f4d0 0x56205214d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.290+0000 7f6808e66640 1 -- 172.21.15.5:0/4128162674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562051d476b0 msgr2=0x56205214ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:39.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.290+0000 7f6808e66640 1 --2- 172.21.15.5:0/4128162674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562051d476b0 0x56205214ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.290+0000 7f6808e66640 1 -- 172.21.15.5:0/4128162674 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562051e4a7d0 con 0x562051d68390 2024-05-20T18:37:39.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.290+0000 7f6809667640 1 --2- 172.21.15.5:0/4128162674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562051d476b0 0x56205214ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:39.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.291+0000 7f6808e66640 1 --2- 172.21.15.5:0/4128162674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562051d68390 0x562052132b60 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f67f4001e80 tx=0x7f67f4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70cb314114f23bbb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.291+0000 7f67fa7fc640 1 -- 172.21.15.5:0/4128162674 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f40035f0 con 0x562051d68390 2024-05-20T18:37:39.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.291+0000 7f67fa7fc640 1 -- 172.21.15.5:0/4128162674 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67f4003790 con 0x562051d68390 2024-05-20T18:37:39.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.291+0000 7f67fa7fc640 1 -- 172.21.15.5:0/4128162674 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f4003b40 con 0x562051d68390 2024-05-20T18:37:39.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.292+0000 7f6811f73740 1 -- 172.21.15.5:0/4128162674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562051d68390 msgr2=0x562052132b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.292+0000 7f6811f73740 1 --2- 172.21.15.5:0/4128162674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562051d68390 0x562052132b60 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f67f4001e80 tx=0x7f67f4011000 comp rx=0 tx=0).stop 2024-05-20T18:37:39.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.292+0000 7f6811f73740 1 -- 172.21.15.5:0/4128162674 shutdown_connections 2024-05-20T18:37:39.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.292+0000 7f6811f73740 1 --2- 172.21.15.5:0/4128162674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562051d476b0 0x56205214ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.292+0000 7f6811f73740 1 --2- 172.21.15.5:0/4128162674 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562051d3f4d0 0x56205214d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.292+0000 7f6811f73740 1 --2- 172.21.15.5:0/4128162674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562051d68390 0x562052132b60 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.292+0000 7f6811f73740 1 -- 172.21.15.5:0/4128162674 >> 172.21.15.5:0/4128162674 conn(0x562051d5c4e0 msgr2=0x5620521081c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:39.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.292+0000 7f6811f73740 1 -- 172.21.15.5:0/4128162674 shutdown_connections 2024-05-20T18:37:39.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.292+0000 7f6811f73740 1 -- 172.21.15.5:0/4128162674 wait complete. 2024-05-20T18:37:39.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.292+0000 7f6811f73740 1 -- start start 2024-05-20T18:37:39.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.293+0000 7f6811f73740 1 Processor -- start 2024-05-20T18:37:39.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.294+0000 7f6811f73740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562051d3f4d0 0x56205214dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.294+0000 7f6811f73740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562051d476b0 0x56205212e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.294+0000 7f6811f73740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562051d68390 0x56205212e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.294+0000 7f6811f73740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562052027b90 con 0x562051d476b0 2024-05-20T18:37:39.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.294+0000 7f6811f73740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562052136420 con 0x562051d68390 2024-05-20T18:37:39.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.294+0000 7f6811f73740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x562052144cb0 con 0x562051d3f4d0 2024-05-20T18:37:39.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.294+0000 7f6811f73740 10 client.0 _ll_register_callbacks cb 0x562052063da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:37:39.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.294+0000 7f6803fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562051d476b0 0x56205212e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.294+0000 7f6803fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562051d476b0 0x56205212e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35488/0 (socket says 172.21.15.5:35488) 2024-05-20T18:37:39.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.294+0000 7f6803fff640 1 -- 172.21.15.5:0/106561088 learned_addr learned my addr 172.21.15.5:0/106561088 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:39.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.294+0000 7f6808e66640 1 --2- 172.21.15.5:0/106561088 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562051d3f4d0 0x56205214dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.294+0000 7f6803fff640 1 -- 172.21.15.5:0/106561088 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562051d3f4d0 msgr2=0x56205214dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:39.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.294+0000 7f6803fff640 1 --2- 172.21.15.5:0/106561088 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562051d3f4d0 0x56205214dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.295+0000 7f6803fff640 1 -- 172.21.15.5:0/106561088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562051d68390 msgr2=0x56205212e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:39.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.295+0000 7f6809667640 1 --2- 172.21.15.5:0/106561088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562051d68390 0x56205212e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.295+0000 7f6803fff640 1 --2- 172.21.15.5:0/106561088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562051d68390 0x56205212e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.295+0000 7f6803fff640 1 -- 172.21.15.5:0/106561088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562051d803f0 con 0x562051d476b0 2024-05-20T18:37:39.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.295+0000 7f6803fff640 1 --2- 172.21.15.5:0/106561088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562051d476b0 0x56205212e240 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f67ec060c20 tx=0x7f67ec058d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e695f53963d9912 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.295+0000 7f67e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:37:39.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.295+0000 7f67e37fe640 1 -- 172.21.15.5:0/106561088 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec08a2c0 con 0x562051d476b0 2024-05-20T18:37:39.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.295+0000 7f67e37fe640 1 -- 172.21.15.5:0/106561088 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7f67ec0074b0 con 0x562051d476b0 2024-05-20T18:37:39.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.295+0000 7f6811f73740 1 -- 172.21.15.5:0/106561088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67f4003790 con 0x562051d476b0 2024-05-20T18:37:39.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.295+0000 7f67e37fe640 1 -- 172.21.15.5:0/106561088 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7f67ec0076b0 con 0x562051d476b0 2024-05-20T18:37:39.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.295+0000 7f6811f73740 10 client.9423 fetch_fsmap learned FSMap version 104 2024-05-20T18:37:39.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.295+0000 7f6811f73740 1 -- 172.21.15.5:0/106561088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) -- 0x562051d400f0 con 0x562051d476b0 2024-05-20T18:37:39.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.295+0000 7f67e37fe640 1 -- 172.21.15.5:0/106561088 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67f4003790 con 0x562051d476b0 2024-05-20T18:37:39.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.295+0000 7f67e37fe640 1 -- 172.21.15.5:0/106561088 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 104) ==== 3494+0+0 (secure 0 0 0) 0x7f67ec057930 con 0x562051d476b0 2024-05-20T18:37:39.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.295+0000 7f6811f73740 10 client.9423 fetch_fsmap finished waiting for FSMap version 104 2024-05-20T18:37:39.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.295+0000 7f6811f73740 10 client.9423 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-05-20T18:37:39.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.295+0000 7f6811f73740 20 client.9423 populate_metadata read hostname 'smithi005' 2024-05-20T18:37:39.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.295+0000 7f6811f73740 1 --2- 172.21.15.5:0/106561088 >> [v2:172.21.15.5:6835/4273429943,v1:172.21.15.5:6837/4273429943] conn(0x562051d26f60 0x562052160b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.295+0000 7f6811f73740 4 client.9423 mds_command: new command op to 9012 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:37:39.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.295+0000 7f6811f73740 1 -- 172.21.15.5:0/106561088 --> [v2:172.21.15.5:6835/4273429943,v1:172.21.15.5:6837/4273429943] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56205207a430 con 0x562051d26f60 2024-05-20T18:37:39.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.295+0000 7f6808e66640 1 --2- 172.21.15.5:0/106561088 >> [v2:172.21.15.5:6835/4273429943,v1:172.21.15.5:6837/4273429943] conn(0x562051d26f60 0x562052160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.296+0000 7f67e37fe640 10 client.9423 ms_handle_connect on v2:172.21.15.5:6835/4273429943 2024-05-20T18:37:39.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.296+0000 7f6808e66640 1 --2- 172.21.15.5:0/106561088 >> [v2:172.21.15.5:6835/4273429943,v1:172.21.15.5:6837/4273429943] conn(0x562051d26f60 0x562052160b10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdd0dbf3bef14705 server_cookie=9b1f5adae290bef7 in_seq=0 out_seq=0 2024-05-20T18:37:39.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.297+0000 7f67e37fe640 1 -- 172.21.15.5:0/106561088 <== mds.0 v2:172.21.15.5:6835/4273429943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56205207a430 con 0x562051d26f60 2024-05-20T18:37:39.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.297+0000 7f67e37fe640 10 client.9423 handle_command_reply: tid=0 2024-05-20T18:37:39.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.297+0000 7f6811f73740 1 client.9423 shutdown 2024-05-20T18:37:39.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.297+0000 7f6811f73740 1 -- 172.21.15.5:0/106561088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562051d476b0 msgr2=0x56205212e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.297+0000 7f6811f73740 1 --2- 172.21.15.5:0/106561088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562051d476b0 0x56205212e240 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f67ec060c20 tx=0x7f67ec058d20 comp rx=0 tx=0).stop 2024-05-20T18:37:39.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.297+0000 7f6811f73740 1 -- 172.21.15.5:0/106561088 shutdown_connections 2024-05-20T18:37:39.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.297+0000 7f6811f73740 1 --2- 172.21.15.5:0/106561088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562051d68390 0x56205212e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.297+0000 7f6811f73740 1 --2- 172.21.15.5:0/106561088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562051d476b0 0x56205212e240 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.297+0000 7f6811f73740 1 --2- 172.21.15.5:0/106561088 >> [v2:172.21.15.5:6835/4273429943,v1:172.21.15.5:6837/4273429943] conn(0x562051d26f60 0x562052160b10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.297+0000 7f6811f73740 1 --2- 172.21.15.5:0/106561088 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562051d3f4d0 0x56205214dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.297+0000 7f67e37fe640 0 client.9423 ms_handle_reset on v2:172.21.15.5:6835/4273429943 2024-05-20T18:37:39.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.297+0000 7f6811f73740 1 -- 172.21.15.5:0/106561088 >> 172.21.15.5:0/106561088 conn(0x562051d5c4e0 msgr2=0x5620521081c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:39.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.297+0000 7f6811f73740 1 -- 172.21.15.5:0/106561088 shutdown_connections 2024-05-20T18:37:39.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.297+0000 7f6811f73740 1 -- 172.21.15.5:0/106561088 wait complete. 2024-05-20T18:37:39.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.297+0000 7f6811f73740 20 client.9423 trim_cache size 0 max 16384 2024-05-20T18:37:39.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.312+0000 7f6811f73740 1 Processor -- start 2024-05-20T18:37:39.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.313+0000 7f6811f73740 1 -- start start 2024-05-20T18:37:39.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.314+0000 7f6811f73740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562051d3f4d0 0x562052107f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.314+0000 7f6811f73740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562051d476b0 0x562052145d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.314+0000 7f6811f73740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562051d68390 0x5620521462c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.314+0000 7f6811f73740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f68041fdd10 con 0x562051d68390 2024-05-20T18:37:39.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.314+0000 7f6811f73740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f68041fcda0 con 0x562051d476b0 2024-05-20T18:37:39.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.314+0000 7f6811f73740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f68040b1050 con 0x562051d3f4d0 2024-05-20T18:37:39.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.314+0000 7f6809667640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562051d68390 0x5620521462c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.314+0000 7f6809667640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562051d68390 0x5620521462c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35490/0 (socket says 172.21.15.5:35490) 2024-05-20T18:37:39.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.314+0000 7f6809667640 1 -- 172.21.15.5:0/1547224710 learned_addr learned my addr 172.21.15.5:0/1547224710 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:39.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.314+0000 7f6808e66640 1 --2- 172.21.15.5:0/1547224710 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562051d3f4d0 0x562052107f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.314+0000 7f6803fff640 1 --2- 172.21.15.5:0/1547224710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562051d476b0 0x562052145d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.314+0000 7f6809667640 1 -- 172.21.15.5:0/1547224710 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562051d3f4d0 msgr2=0x562052107f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:39.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.314+0000 7f6809667640 1 --2- 172.21.15.5:0/1547224710 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562051d3f4d0 0x562052107f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.314+0000 7f6809667640 1 -- 172.21.15.5:0/1547224710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562051d476b0 msgr2=0x562052145d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:39.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.314+0000 7f6809667640 1 --2- 172.21.15.5:0/1547224710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562051d476b0 0x562052145d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.314+0000 7f6809667640 1 -- 172.21.15.5:0/1547224710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562052106210 con 0x562051d68390 2024-05-20T18:37:39.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.314+0000 7f6809667640 1 --2- 172.21.15.5:0/1547224710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562051d68390 0x5620521462c0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f67fc02a680 tx=0x7f67fc02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592ad00aaed7263e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.315+0000 7f67f9ffb640 1 -- 172.21.15.5:0/1547224710 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc003b90 con 0x562051d68390 2024-05-20T18:37:39.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.315+0000 7f67f9ffb640 1 -- 172.21.15.5:0/1547224710 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67fc041070 con 0x562051d68390 2024-05-20T18:37:39.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.315+0000 7f67f9ffb640 1 -- 172.21.15.5:0/1547224710 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc004040 con 0x562051d68390 2024-05-20T18:37:39.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.315+0000 7f6811f73740 1 -- 172.21.15.5:0/1547224710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562051d68390 msgr2=0x5620521462c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.315+0000 7f6811f73740 1 --2- 172.21.15.5:0/1547224710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562051d68390 0x5620521462c0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f67fc02a680 tx=0x7f67fc02e000 comp rx=0 tx=0).stop 2024-05-20T18:37:39.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.315+0000 7f6811f73740 1 -- 172.21.15.5:0/1547224710 shutdown_connections 2024-05-20T18:37:39.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.315+0000 7f6811f73740 1 --2- 172.21.15.5:0/1547224710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562051d68390 0x5620521462c0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.315+0000 7f6811f73740 1 --2- 172.21.15.5:0/1547224710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562051d476b0 0x562052145d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.315+0000 7f6811f73740 1 --2- 172.21.15.5:0/1547224710 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562051d3f4d0 0x562052107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.315+0000 7f6811f73740 1 -- 172.21.15.5:0/1547224710 >> 172.21.15.5:0/1547224710 conn(0x562051d5c4e0 msgr2=0x562052108a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:39.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.315+0000 7f6811f73740 1 -- 172.21.15.5:0/1547224710 shutdown_connections 2024-05-20T18:37:39.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.315+0000 7f6811f73740 1 -- 172.21.15.5:0/1547224710 wait complete. 2024-05-20T18:37:39.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.316+0000 7f6811f73740 1 -- start start 2024-05-20T18:37:39.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.316+0000 7f6811f73740 1 Processor -- start 2024-05-20T18:37:39.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.318+0000 7f6811f73740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562051d3f4d0 0x5620521ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.318+0000 7f6811f73740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562051d476b0 0x5620521cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.318+0000 7f6811f73740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562051d68390 0x562052134fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.318+0000 7f6811f73740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f68041fc750 con 0x562051d476b0 2024-05-20T18:37:39.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.318+0000 7f6811f73740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56205214fcf0 con 0x562051d3f4d0 2024-05-20T18:37:39.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.318+0000 7f6811f73740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5620521ce6f0 con 0x562051d68390 2024-05-20T18:37:39.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.318+0000 7f6811f73740 10 client.0 _ll_register_callbacks cb 0x7f67ec054780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:37:39.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.318+0000 7f6803fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562051d476b0 0x5620521cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.318+0000 7f6803fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562051d476b0 0x5620521cddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35504/0 (socket says 172.21.15.5:35504) 2024-05-20T18:37:39.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.318+0000 7f6803fff640 1 -- 172.21.15.5:0/1023427548 learned_addr learned my addr 172.21.15.5:0/1023427548 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:39.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.318+0000 7f6809667640 1 --2- 172.21.15.5:0/1023427548 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562051d68390 0x562052134fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.318+0000 7f6808e66640 1 --2- 172.21.15.5:0/1023427548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562051d3f4d0 0x5620521ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.318+0000 7f6803fff640 1 -- 172.21.15.5:0/1023427548 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562051d68390 msgr2=0x562052134fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:39.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.318+0000 7f6803fff640 1 --2- 172.21.15.5:0/1023427548 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562051d68390 0x562052134fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.318+0000 7f6803fff640 1 -- 172.21.15.5:0/1023427548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562051d3f4d0 msgr2=0x5620521ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:39.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.318+0000 7f6803fff640 1 --2- 172.21.15.5:0/1023427548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562051d3f4d0 0x5620521ca2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.318+0000 7f6803fff640 1 -- 172.21.15.5:0/1023427548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56205214df00 con 0x562051d476b0 2024-05-20T18:37:39.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.318+0000 7f6803fff640 1 --2- 172.21.15.5:0/1023427548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562051d476b0 0x5620521cddb0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f67ec058420 tx=0x7f67ec0886c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d31f0295e89cb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.319+0000 7f67e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:37:39.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.319+0000 7f67e37fe640 1 -- 172.21.15.5:0/1023427548 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec05c890 con 0x562051d476b0 2024-05-20T18:37:39.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.319+0000 7f67e37fe640 1 -- 172.21.15.5:0/1023427548 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7f67ec08a0c0 con 0x562051d476b0 2024-05-20T18:37:39.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.319+0000 7f6811f73740 1 -- 172.21.15.5:0/1023427548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67fc041070 con 0x562051d476b0 2024-05-20T18:37:39.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.319+0000 7f67e37fe640 1 -- 172.21.15.5:0/1023427548 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7f67ec07e050 con 0x562051d476b0 2024-05-20T18:37:39.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.319+0000 7f6811f73740 10 client.9429 fetch_fsmap learned FSMap version 104 2024-05-20T18:37:39.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.319+0000 7f6811f73740 1 -- 172.21.15.5:0/1023427548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) -- 0x56205214da70 con 0x562051d476b0 2024-05-20T18:37:39.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.319+0000 7f67e37fe640 1 -- 172.21.15.5:0/1023427548 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67fc041070 con 0x562051d476b0 2024-05-20T18:37:39.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.319+0000 7f67e37fe640 1 -- 172.21.15.5:0/1023427548 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 104) ==== 3494+0+0 (secure 0 0 0) 0x7f67ec081020 con 0x562051d476b0 2024-05-20T18:37:39.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.319+0000 7f6811f73740 10 client.9429 fetch_fsmap finished waiting for FSMap version 104 2024-05-20T18:37:39.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.319+0000 7f6811f73740 10 client.9429 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-05-20T18:37:39.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.319+0000 7f6811f73740 20 client.9429 populate_metadata read hostname 'smithi005' 2024-05-20T18:37:39.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.319+0000 7f6811f73740 1 --2- 172.21.15.5:0/1023427548 >> [v2:172.21.15.5:6835/4273429943,v1:172.21.15.5:6837/4273429943] conn(0x562052135500 0x5620521341e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.319+0000 7f6811f73740 4 client.9429 mds_command: new command op to 9012 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-20T18:37:39.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.319+0000 7f6811f73740 1 -- 172.21.15.5:0/1023427548 --> [v2:172.21.15.5:6835/4273429943,v1:172.21.15.5:6837/4273429943] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562052074780 con 0x562052135500 2024-05-20T18:37:39.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.319+0000 7f6808e66640 1 --2- 172.21.15.5:0/1023427548 >> [v2:172.21.15.5:6835/4273429943,v1:172.21.15.5:6837/4273429943] conn(0x562052135500 0x5620521341e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.320+0000 7f67e37fe640 10 client.9429 ms_handle_connect on v2:172.21.15.5:6835/4273429943 2024-05-20T18:37:39.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.320+0000 7f6808e66640 1 --2- 172.21.15.5:0/1023427548 >> [v2:172.21.15.5:6835/4273429943,v1:172.21.15.5:6837/4273429943] conn(0x562052135500 0x5620521341e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fea99a7267237d8 server_cookie=68e5210c1226654f in_seq=0 out_seq=0 2024-05-20T18:37:39.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.329+0000 7f67e37fe640 1 -- 172.21.15.5:0/1023427548 <== mds.0 v2:172.21.15.5:6835/4273429943 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562052074780 con 0x562052135500 2024-05-20T18:37:39.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.329+0000 7f67e37fe640 10 client.9429 handle_command_reply: tid=0 2024-05-20T18:37:39.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.329+0000 7f6811f73740 1 client.9429 shutdown 2024-05-20T18:37:39.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.329+0000 7f6811f73740 1 -- 172.21.15.5:0/1023427548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562051d476b0 msgr2=0x5620521cddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.329+0000 7f6811f73740 1 --2- 172.21.15.5:0/1023427548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562051d476b0 0x5620521cddb0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f67ec058420 tx=0x7f67ec0886c0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.330+0000 7f6811f73740 1 -- 172.21.15.5:0/1023427548 shutdown_connections 2024-05-20T18:37:39.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.330+0000 7f6811f73740 1 --2- 172.21.15.5:0/1023427548 >> [v2:172.21.15.5:6835/4273429943,v1:172.21.15.5:6837/4273429943] conn(0x562052135500 0x5620521341e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.330+0000 7f6811f73740 1 --2- 172.21.15.5:0/1023427548 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562051d68390 0x562052134fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.330+0000 7f6811f73740 1 --2- 172.21.15.5:0/1023427548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562051d476b0 0x5620521cddb0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.330+0000 7f6811f73740 1 --2- 172.21.15.5:0/1023427548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562051d3f4d0 0x5620521ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.330+0000 7f6811f73740 1 -- 172.21.15.5:0/1023427548 >> 172.21.15.5:0/1023427548 conn(0x562051d5c4e0 msgr2=0x562052107eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:39.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.330+0000 7f67e37fe640 0 client.9429 ms_handle_reset on v2:172.21.15.5:6835/4273429943 2024-05-20T18:37:39.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.330+0000 7f6811f73740 1 -- 172.21.15.5:0/1023427548 shutdown_connections 2024-05-20T18:37:39.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.330+0000 7f6811f73740 1 -- 172.21.15.5:0/1023427548 wait complete. 2024-05-20T18:37:39.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.330+0000 7f6811f73740 20 client.9429 trim_cache size 0 max 16384 2024-05-20T18:37:39.332 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:37:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "message": "", 2024-05-20T18:37:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0 2024-05-20T18:37:39.333 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:37:39.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.330+0000 7f680b0ca640 1 -- 172.21.15.5:0/742802371 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f67e406be60 msgr2=0x7f67e406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.330+0000 7f680b0ca640 1 --2- 172.21.15.5:0/742802371 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f67e406be60 0x7f67e406e370 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f67f4007fd0 tx=0x7f67f4009040 comp rx=0 tx=0).stop 2024-05-20T18:37:39.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.330+0000 7f680b0ca640 1 -- 172.21.15.5:0/742802371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f680415ca00 msgr2=0x7f68041579e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.330+0000 7f680b0ca640 1 --2- 172.21.15.5:0/742802371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f680415ca00 0x7f68041579e0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f67ec00c400 tx=0x7f67ec00c8c0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.331+0000 7f680b0ca640 1 -- 172.21.15.5:0/742802371 shutdown_connections 2024-05-20T18:37:39.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.331+0000 7f680b0ca640 1 --2- 172.21.15.5:0/742802371 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f67e406be60 0x7f67e406e370 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.331+0000 7f680b0ca640 1 --2- 172.21.15.5:0/742802371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68041ee0a0 0x7f680415e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.331+0000 7f680b0ca640 1 --2- 172.21.15.5:0/742802371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f680415ca00 0x7f68041579e0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.331+0000 7f680b0ca640 1 --2- 172.21.15.5:0/742802371 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68040b0020 0x7f6804140e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.331+0000 7f680b0ca640 1 -- 172.21.15.5:0/742802371 >> 172.21.15.5:0/742802371 conn(0x7f68040cd2a0 msgr2=0x7f680415b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:39.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.331+0000 7f680b0ca640 1 -- 172.21.15.5:0/742802371 shutdown_connections 2024-05-20T18:37:39.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.331+0000 7f680b0ca640 1 -- 172.21.15.5:0/742802371 wait complete. 2024-05-20T18:37:39.343 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-20T18:37:39.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.458+0000 7f178f61b640 1 Processor -- start 2024-05-20T18:37:39.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.459+0000 7f178f61b640 1 -- start start 2024-05-20T18:37:39.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.460+0000 7f178f61b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1788162820 0x7f17880b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.460+0000 7f178f61b640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17880d0630 con 0x7f178806dae0 2024-05-20T18:37:39.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.460+0000 7f178f61b640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17880d39c0 con 0x7f17880b3e80 2024-05-20T18:37:39.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.460+0000 7f178f61b640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f17880d3ca0 con 0x7f1788162820 2024-05-20T18:37:39.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.460+0000 7f178d3b7640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1788162820 0x7f17880b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.460+0000 7f178dbb8640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f178806dae0 0x7f17880b8190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:32886/0 (socket says 172.21.15.5:32886) 2024-05-20T18:37:39.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.460+0000 7f178d3b7640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1788162820 0x7f17880b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35516/0 (socket says 172.21.15.5:35516) 2024-05-20T18:37:39.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.460+0000 7f178dbb8640 1 -- 172.21.15.5:0/3500162845 learned_addr learned my addr 172.21.15.5:0/3500162845 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:39.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.460+0000 7f178d3b7640 1 -- 172.21.15.5:0/3500162845 >> v1:172.21.15.63:6789/0 conn(0x7f17880b3e80 legacy=0x7f178806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.460+0000 7f178d3b7640 1 -- 172.21.15.5:0/3500162845 >> v1:172.21.15.5:6789/0 conn(0x7f178806dae0 legacy=0x7f17880b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.460+0000 7f178d3b7640 1 -- 172.21.15.5:0/3500162845 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17880d3f80 con 0x7f1788162820 2024-05-20T18:37:39.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.460+0000 7f177ffff640 1 -- 172.21.15.5:0/3500162845 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1641135035 0 0) 0x7f17880d0630 con 0x7f178806dae0 2024-05-20T18:37:39.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.461+0000 7f178d3b7640 1 --2- 172.21.15.5:0/3500162845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1788162820 0x7f17880b3940 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f17840097a0 tx=0x7f178402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f66e3793a275cac server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.461+0000 7f177ffff640 1 -- 172.21.15.5:0/3500162845 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1784037070 con 0x7f1788162820 2024-05-20T18:37:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.461+0000 7f177ffff640 1 -- 172.21.15.5:0/3500162845 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1784032440 con 0x7f1788162820 2024-05-20T18:37:39.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.461+0000 7f177ffff640 1 -- 172.21.15.5:0/3500162845 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1784032810 con 0x7f1788162820 2024-05-20T18:37:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.462+0000 7f178f61b640 1 -- 172.21.15.5:0/3500162845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1788162820 msgr2=0x7f17880b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.462+0000 7f178f61b640 1 --2- 172.21.15.5:0/3500162845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1788162820 0x7f17880b3940 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f17840097a0 tx=0x7f178402a230 comp rx=0 tx=0).stop 2024-05-20T18:37:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.462+0000 7f178f61b640 1 -- 172.21.15.5:0/3500162845 shutdown_connections 2024-05-20T18:37:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.462+0000 7f178f61b640 1 --2- 172.21.15.5:0/3500162845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1788162820 0x7f17880b3940 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.462+0000 7f178f61b640 1 -- 172.21.15.5:0/3500162845 >> 172.21.15.5:0/3500162845 conn(0x7f17880d4bd0 msgr2=0x7f17880d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.462+0000 7f178f61b640 1 -- 172.21.15.5:0/3500162845 shutdown_connections 2024-05-20T18:37:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.462+0000 7f178f61b640 1 -- 172.21.15.5:0/3500162845 wait complete. 2024-05-20T18:37:39.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.464+0000 7f178f61b640 1 Processor -- start 2024-05-20T18:37:39.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.464+0000 7f178f61b640 1 -- start start 2024-05-20T18:37:39.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.465+0000 7f178f61b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17880b3e80 0x7f1788159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.465+0000 7f178f61b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1788162820 0x7f17880c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.465+0000 7f178f61b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f178815aee0 0x7f178815b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.465+0000 7f178f61b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f17880d4780 con 0x7f178815aee0 2024-05-20T18:37:39.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.465+0000 7f178f61b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f17880d0300 con 0x7f1788162820 2024-05-20T18:37:39.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.465+0000 7f178f61b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f17880d4900 con 0x7f17880b3e80 2024-05-20T18:37:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.465+0000 7f178dbb8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f178815aee0 0x7f178815b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.465+0000 7f178dbb8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f178815aee0 0x7f178815b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35528/0 (socket says 172.21.15.5:35528) 2024-05-20T18:37:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.465+0000 7f178dbb8640 1 -- 172.21.15.5:0/4074268930 learned_addr learned my addr 172.21.15.5:0/4074268930 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.466+0000 7f178d3b7640 1 --2- 172.21.15.5:0/4074268930 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17880b3e80 0x7f1788159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.466+0000 7f178cbb6640 1 --2- 172.21.15.5:0/4074268930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1788162820 0x7f17880c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.466+0000 7f178dbb8640 1 -- 172.21.15.5:0/4074268930 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17880b3e80 msgr2=0x7f1788159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.466+0000 7f178dbb8640 1 --2- 172.21.15.5:0/4074268930 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17880b3e80 0x7f1788159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.466+0000 7f178dbb8640 1 -- 172.21.15.5:0/4074268930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1788162820 msgr2=0x7f17880c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.466+0000 7f178dbb8640 1 --2- 172.21.15.5:0/4074268930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1788162820 0x7f17880c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.466+0000 7f178dbb8640 1 -- 172.21.15.5:0/4074268930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17881f5ce0 con 0x7f178815aee0 2024-05-20T18:37:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.466+0000 7f178d3b7640 1 --2- 172.21.15.5:0/4074268930 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17880b3e80 0x7f1788159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.466+0000 7f178dbb8640 1 --2- 172.21.15.5:0/4074268930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f178815aee0 0x7f178815b2f0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f178000c700 tx=0x7f178000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.466+0000 7f177dffb640 1 -- 172.21.15.5:0/4074268930 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780018070 con 0x7f178815aee0 2024-05-20T18:37:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.466+0000 7f177dffb640 1 -- 172.21.15.5:0/4074268930 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f178000ce10 con 0x7f178815aee0 2024-05-20T18:37:39.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.466+0000 7f178f61b640 1 -- 172.21.15.5:0/4074268930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f178815b830 con 0x7f178815aee0 2024-05-20T18:37:39.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.466+0000 7f177dffb640 1 -- 172.21.15.5:0/4074268930 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178000a880 con 0x7f178815aee0 2024-05-20T18:37:39.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.466+0000 7f178f61b640 1 -- 172.21.15.5:0/4074268930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17881f87b0 con 0x7f178815aee0 2024-05-20T18:37:39.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.467+0000 7f178f61b640 1 -- 172.21.15.5:0/4074268930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1788075b00 con 0x7f178815aee0 2024-05-20T18:37:39.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.468+0000 7f177dffb640 1 -- 172.21.15.5:0/4074268930 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f178001b070 con 0x7f178815aee0 2024-05-20T18:37:39.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.468+0000 7f177dffb640 1 --2- 172.21.15.5:0/4074268930 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f176c06be30 0x7f176c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.469+0000 7f177dffb640 1 -- 172.21.15.5:0/4074268930 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7f178008a670 con 0x7f178815aee0 2024-05-20T18:37:39.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.469+0000 7f178d3b7640 1 --2- 172.21.15.5:0/4074268930 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f176c06be30 0x7f176c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.469+0000 7f178d3b7640 1 --2- 172.21.15.5:0/4074268930 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f176c06be30 0x7f176c06e340 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1784034000 tx=0x7f178402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.473+0000 7f177dffb640 1 -- 172.21.15.5:0/4074268930 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f178815b830 con 0x7f178815aee0 2024-05-20T18:37:39.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.697+0000 7f178f61b640 1 -- 172.21.15.5:0/4074268930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f178815dc70 con 0x7f178815aee0 2024-05-20T18:37:39.699 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-20T18:37:39.698+0000 7fd1cf53a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-20T18:37:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.700+0000 7f177dffb640 1 -- 172.21.15.5:0/4074268930 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v105) ==== 161+0+0 (secure 0 0 0) 0x7f17881f5ce0 con 0x7f178815aee0 2024-05-20T18:37:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-20T18:37:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.702+0000 7f178f61b640 1 -- 172.21.15.5:0/4074268930 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f176c06be30 msgr2=0x7f176c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.702+0000 7f178f61b640 1 --2- 172.21.15.5:0/4074268930 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f176c06be30 0x7f176c06e340 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1784034000 tx=0x7f178402e000 comp rx=0 tx=0).stop 2024-05-20T18:37:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.702+0000 7f178f61b640 1 -- 172.21.15.5:0/4074268930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f178815aee0 msgr2=0x7f178815b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.702+0000 7f178f61b640 1 --2- 172.21.15.5:0/4074268930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f178815aee0 0x7f178815b2f0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f178000c700 tx=0x7f178000cbc0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.703+0000 7f178f61b640 1 -- 172.21.15.5:0/4074268930 shutdown_connections 2024-05-20T18:37:39.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.703+0000 7f178f61b640 1 --2- 172.21.15.5:0/4074268930 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f176c06be30 0x7f176c06e340 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.703+0000 7f178f61b640 1 --2- 172.21.15.5:0/4074268930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f178815aee0 0x7f178815b2f0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.703+0000 7f178f61b640 1 --2- 172.21.15.5:0/4074268930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1788162820 0x7f17880c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.703+0000 7f178f61b640 1 --2- 172.21.15.5:0/4074268930 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17880b3e80 0x7f1788159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.703+0000 7f178f61b640 1 -- 172.21.15.5:0/4074268930 >> 172.21.15.5:0/4074268930 conn(0x7f17880d4bd0 msgr2=0x7f17880d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:39.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.703+0000 7f178f61b640 1 -- 172.21.15.5:0/4074268930 shutdown_connections 2024-05-20T18:37:39.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.703+0000 7f178f61b640 1 -- 172.21.15.5:0/4074268930 wait complete. 2024-05-20T18:37:39.707 INFO:tasks.ceph.mds.c.smithi005.stderr: -948> 2024-05-20T18:37:36.970+0000 7f2481efd640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T18:37:39.716 DEBUG:teuthology.orchestra.run.smithi005:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2024-05-20T18:37:39.763 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-05-20T18:37:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.773+0000 7f5361a1f280 1 -- start start 2024-05-20T18:37:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.773+0000 7f5361a1f280 1 Processor -- start 2024-05-20T18:37:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.774+0000 7f5361a1f280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d130ed680 0x564d131cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.774+0000 7f5361a1f280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d130edb00 0x564d131cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.774+0000 7f5361a1f280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d130ee400 0x564d131ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.774+0000 7f5361a1f280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564d13e30180 con 0x564d130ee400 2024-05-20T18:37:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.774+0000 7f5361a1f280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564d130a6900 con 0x564d130ed680 2024-05-20T18:37:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.774+0000 7f5361a1f280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564d130a6480 con 0x564d130edb00 2024-05-20T18:37:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.774+0000 7f53609cc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d130ee400 0x564d131ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.774+0000 7f53609cc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d130ee400 0x564d131ce580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35564/0 (socket says 172.21.15.5:35564) 2024-05-20T18:37:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.774+0000 7f53609cc640 1 -- 172.21.15.5:0/1867002457 learned_addr learned my addr 172.21.15.5:0/1867002457 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.774+0000 7f53601cb640 1 --2- 172.21.15.5:0/1867002457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d130ed680 0x564d131cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.774+0000 7f535f9ca640 1 --2- 172.21.15.5:0/1867002457 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d130edb00 0x564d131cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.774+0000 7f53609cc640 1 -- 172.21.15.5:0/1867002457 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d130edb00 msgr2=0x564d131cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.774+0000 7f53609cc640 1 --2- 172.21.15.5:0/1867002457 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d130edb00 0x564d131cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.774+0000 7f53609cc640 1 -- 172.21.15.5:0/1867002457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d130ed680 msgr2=0x564d131cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.774+0000 7f53609cc640 1 --2- 172.21.15.5:0/1867002457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d130ed680 0x564d131cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.774+0000 7f53609cc640 1 -- 172.21.15.5:0/1867002457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d0f04a780 con 0x564d130ee400 2024-05-20T18:37:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.775+0000 7f53609cc640 1 --2- 172.21.15.5:0/1867002457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d130ee400 0x564d131ce580 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x564d13e6b3e0 tx=0x564d1313a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b08be7bca3a528 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.775+0000 7f535d9c6640 1 -- 172.21.15.5:0/1867002457 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564d0f03a9c0 con 0x564d130ee400 2024-05-20T18:37:39.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.775+0000 7f535d9c6640 1 -- 172.21.15.5:0/1867002457 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x564d0f040700 con 0x564d130ee400 2024-05-20T18:37:39.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.775+0000 7f535d9c6640 1 -- 172.21.15.5:0/1867002457 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564d0f03a680 con 0x564d130ee400 2024-05-20T18:37:39.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.775+0000 7f5361a1f280 1 -- 172.21.15.5:0/1867002457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d0f04b4a0 con 0x564d130ee400 2024-05-20T18:37:39.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.775+0000 7f5361a1f280 1 -- 172.21.15.5:0/1867002457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=0}) -- 0x564d0f04a000 con 0x564d130ee400 2024-05-20T18:37:39.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.775+0000 7f5361a1f280 4 waiting for MDS map... 2024-05-20T18:37:39.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.775+0000 7f535d9c6640 1 -- 172.21.15.5:0/1867002457 <== mon.0 v2:172.21.15.5:3300/0 4 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x564d13f1a200 con 0x564d130ee400 2024-05-20T18:37:39.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.775+0000 7f535d9c6640 1 -- 172.21.15.5:0/1867002457 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 105) ==== 2885+0+0 (secure 0 0 0) 0x564d1309f180 con 0x564d130ee400 2024-05-20T18:37:39.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.775+0000 7f5361a1f280 4 Got MDS map 105 2024-05-20T18:37:39.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.775+0000 7f5361a1f280 10 main: JournalTool::main 2024-05-20T18:37:39.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.775+0000 7f5361a1f280 4 main: JournalTool: connecting to RADOS... 2024-05-20T18:37:39.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.776+0000 7f5361a1f280 1 Processor -- start 2024-05-20T18:37:39.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.776+0000 7f5361a1f280 1 -- start start 2024-05-20T18:37:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.777+0000 7f5361a1f280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d130ecd80 0x564d131ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.777+0000 7f5361a1f280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d130ec900 0x564d131cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.777+0000 7f5361a1f280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d130ec480 0x564d131d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.777+0000 7f5361a1f280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564d130a6180 con 0x564d130ec900 2024-05-20T18:37:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.777+0000 7f5361a1f280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564d13e30d80 con 0x564d130ecd80 2024-05-20T18:37:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.777+0000 7f5361a1f280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564d13e30f00 con 0x564d130ec480 2024-05-20T18:37:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.777+0000 7f53601cb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d130ec900 0x564d131cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.777+0000 7f53601cb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d130ec900 0x564d131cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35580/0 (socket says 172.21.15.5:35580) 2024-05-20T18:37:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.777+0000 7f53601cb640 1 -- 172.21.15.5:0/3707252208 learned_addr learned my addr 172.21.15.5:0/3707252208 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.777+0000 7f535f9ca640 1 --2- 172.21.15.5:0/3707252208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d130ecd80 0x564d131ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.777+0000 7f535f9ca640 1 --2- 172.21.15.5:0/3707252208 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d130ec480 0x564d131d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.778+0000 7f53601cb640 1 -- 172.21.15.5:0/3707252208 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d130ec480 msgr2=0x564d131d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.778+0000 7f53601cb640 1 --2- 172.21.15.5:0/3707252208 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d130ec480 0x564d131d0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.778+0000 7f53601cb640 1 -- 172.21.15.5:0/3707252208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d130ecd80 msgr2=0x564d131ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.778+0000 7f53601cb640 1 --2- 172.21.15.5:0/3707252208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d130ecd80 0x564d131ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.778+0000 7f53601cb640 1 -- 172.21.15.5:0/3707252208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d0f04a3c0 con 0x564d130ec900 2024-05-20T18:37:39.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.778+0000 7f535f9ca640 1 --2- 172.21.15.5:0/3707252208 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d130ec480 0x564d131d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:39.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.778+0000 7f53601cb640 1 --2- 172.21.15.5:0/3707252208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d130ec900 0x564d131cf080 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x564d13e2c870 tx=0x564d13e22d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de467dc952c1ed3c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.778+0000 7f535b9c2640 1 -- 172.21.15.5:0/3707252208 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564d0f03a9c0 con 0x564d130ec900 2024-05-20T18:37:39.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.778+0000 7f535b9c2640 1 -- 172.21.15.5:0/3707252208 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x564d0f040e00 con 0x564d130ec900 2024-05-20T18:37:39.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.778+0000 7f535b9c2640 1 -- 172.21.15.5:0/3707252208 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564d0f03a680 con 0x564d130ec900 2024-05-20T18:37:39.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.779+0000 7f5361a1f280 1 -- 172.21.15.5:0/3707252208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d130ec900 msgr2=0x564d131cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.779+0000 7f5361a1f280 1 --2- 172.21.15.5:0/3707252208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d130ec900 0x564d131cf080 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x564d13e2c870 tx=0x564d13e22d20 comp rx=0 tx=0).stop 2024-05-20T18:37:39.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.779+0000 7f5361a1f280 1 -- 172.21.15.5:0/3707252208 shutdown_connections 2024-05-20T18:37:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.779+0000 7f5361a1f280 1 --2- 172.21.15.5:0/3707252208 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d130ec480 0x564d131d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.779+0000 7f5361a1f280 1 --2- 172.21.15.5:0/3707252208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d130ec900 0x564d131cf080 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.779+0000 7f5361a1f280 1 --2- 172.21.15.5:0/3707252208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d130ecd80 0x564d131ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.779+0000 7f5361a1f280 1 -- 172.21.15.5:0/3707252208 >> 172.21.15.5:0/3707252208 conn(0x564d130ed200 msgr2=0x564d1315f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.779+0000 7f5361a1f280 1 -- 172.21.15.5:0/3707252208 shutdown_connections 2024-05-20T18:37:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.779+0000 7f5361a1f280 1 -- 172.21.15.5:0/3707252208 wait complete. 2024-05-20T18:37:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.779+0000 7f5361a1f280 1 Processor -- start 2024-05-20T18:37:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.779+0000 7f5361a1f280 1 -- start start 2024-05-20T18:37:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.781+0000 7f5361a1f280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d130ec480 0x564d131d0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.781+0000 7f5361a1f280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d130ec900 0x564d131cf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.781+0000 7f5361a1f280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d130ecd80 0x564d131ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.781+0000 7f5361a1f280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564d13e31200 con 0x564d130ec900 2024-05-20T18:37:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.781+0000 7f5361a1f280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564d13e31080 con 0x564d130ecd80 2024-05-20T18:37:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.781+0000 7f5361a1f280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564d13e31380 con 0x564d130ec480 2024-05-20T18:37:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.781+0000 7f53601cb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d130ec900 0x564d131cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.781+0000 7f53601cb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d130ec900 0x564d131cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35584/0 (socket says 172.21.15.5:35584) 2024-05-20T18:37:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.781+0000 7f53601cb640 1 -- 172.21.15.5:0/3377729129 learned_addr learned my addr 172.21.15.5:0/3377729129 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.781+0000 7f535f9ca640 1 --2- 172.21.15.5:0/3377729129 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d130ec480 0x564d131d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.781+0000 7f535f9ca640 1 --2- 172.21.15.5:0/3377729129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d130ecd80 0x564d131ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.781+0000 7f53601cb640 1 -- 172.21.15.5:0/3377729129 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d130ec480 msgr2=0x564d131d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.781+0000 7f53601cb640 1 --2- 172.21.15.5:0/3377729129 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d130ec480 0x564d131d0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.781+0000 7f53601cb640 1 -- 172.21.15.5:0/3377729129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d130ecd80 msgr2=0x564d131ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.781+0000 7f53601cb640 1 --2- 172.21.15.5:0/3377729129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d130ecd80 0x564d131ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.781+0000 7f53601cb640 1 -- 172.21.15.5:0/3377729129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d0f04a960 con 0x564d130ec900 2024-05-20T18:37:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.781+0000 7f53601cb640 1 --2- 172.21.15.5:0/3377729129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d130ec900 0x564d131cf080 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x564d13e2db30 tx=0x564d1313a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.781+0000 7f535a1bf640 1 -- 172.21.15.5:0/3377729129 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564d0f03a4e0 con 0x564d130ec900 2024-05-20T18:37:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.781+0000 7f535a1bf640 1 -- 172.21.15.5:0/3377729129 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x564d0f040a80 con 0x564d130ec900 2024-05-20T18:37:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.781+0000 7f535a1bf640 1 -- 172.21.15.5:0/3377729129 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564d0f03a9c0 con 0x564d130ec900 2024-05-20T18:37:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.781+0000 7f5361a1f280 1 -- 172.21.15.5:0/3377729129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564d0f04b680 con 0x564d130ec900 2024-05-20T18:37:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.781+0000 7f5361a1f280 1 -- 172.21.15.5:0/3377729129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d0f04b860 con 0x564d130ec900 2024-05-20T18:37:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.781+0000 7f5361a1f280 4 main: JournalTool: resolving pool 17 2024-05-20T18:37:39.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.783+0000 7f535a1bf640 1 -- 172.21.15.5:0/3377729129 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x564d13f78c00 con 0x564d130ec900 2024-05-20T18:37:39.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.783+0000 7f535a1bf640 1 --2- 172.21.15.5:0/3377729129 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x564d130ef180 0x564d13424000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.783+0000 7f535a1bf640 1 -- 172.21.15.5:0/3377729129 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x564d13f76a00 con 0x564d130ec900 2024-05-20T18:37:39.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.783+0000 7f5361a1f280 4 main: JournalTool: creating IoCtx.. 2024-05-20T18:37:39.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.783+0000 7f535f9ca640 1 --2- 172.21.15.5:0/3377729129 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x564d130ef180 0x564d13424000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.783+0000 7f5361a1f280 4 main: Executing for rank 0 2024-05-20T18:37:39.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.784+0000 7f535f9ca640 1 --2- 172.21.15.5:0/3377729129 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x564d130ef180 0x564d13424000 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x564d13124630 tx=0x564d13e23980 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.784+0000 7f5361a1f280 1 -- start start 2024-05-20T18:37:39.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.784+0000 7f5361a1f280 1 Processor -- start 2024-05-20T18:37:39.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.785+0000 7f5361a1f280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d130ee880 0x564d131d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.786+0000 7f5361a1f280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d130eed00 0x564d131d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.786+0000 7f5361a1f280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d130ef600 0x564d131d1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.786+0000 7f5361a1f280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564d13e31800 con 0x564d130ef600 2024-05-20T18:37:39.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.786+0000 7f5361a1f280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564d13e31980 con 0x564d130ee880 2024-05-20T18:37:39.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.786+0000 7f5361a1f280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564d13e31b00 con 0x564d130eed00 2024-05-20T18:37:39.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.786+0000 7f535f9ca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d130ef600 0x564d131d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.786+0000 7f535f9ca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d130ef600 0x564d131d1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35590/0 (socket says 172.21.15.5:35590) 2024-05-20T18:37:39.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.786+0000 7f535f9ca640 1 -- 172.21.15.5:0/1157016912 learned_addr learned my addr 172.21.15.5:0/1157016912 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:39.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.786+0000 7f53601cb640 1 --2- 172.21.15.5:0/1157016912 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d130eed00 0x564d131d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.786+0000 7f53609cc640 1 --2- 172.21.15.5:0/1157016912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d130ee880 0x564d131d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.786+0000 7f535f9ca640 1 -- 172.21.15.5:0/1157016912 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d130eed00 msgr2=0x564d131d0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:39.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.786+0000 7f535f9ca640 1 --2- 172.21.15.5:0/1157016912 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d130eed00 0x564d131d0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.786+0000 7f535f9ca640 1 -- 172.21.15.5:0/1157016912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d130ee880 msgr2=0x564d131d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:39.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.786+0000 7f535f9ca640 1 --2- 172.21.15.5:0/1157016912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d130ee880 0x564d131d0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.786+0000 7f535f9ca640 1 -- 172.21.15.5:0/1157016912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d0f04ba40 con 0x564d130ef600 2024-05-20T18:37:39.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.786+0000 7f535f9ca640 1 --2- 172.21.15.5:0/1157016912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d130ef600 0x564d131d1180 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x564d1335b080 tx=0x564d13e22c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50269c78ed67107 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.786+0000 7f53571b9640 1 -- 172.21.15.5:0/1157016912 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564d133696c0 con 0x564d130ef600 2024-05-20T18:37:39.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.786+0000 7f53571b9640 1 -- 172.21.15.5:0/1157016912 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x564d0f040c40 con 0x564d130ef600 2024-05-20T18:37:39.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.786+0000 7f5361a1f280 1 -- 172.21.15.5:0/1157016912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d0f04a3c0 con 0x564d130ef600 2024-05-20T18:37:39.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.786+0000 7f53571b9640 1 -- 172.21.15.5:0/1157016912 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564d133ac820 con 0x564d130ef600 2024-05-20T18:37:39.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.786+0000 7f5361a1f280 1 -- 172.21.15.5:0/1157016912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=0}) -- 0x564d0f04a960 con 0x564d130ef600 2024-05-20T18:37:39.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.786+0000 7f5361a1f280 4 waiting for MDS map... 2024-05-20T18:37:39.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.787+0000 7f53571b9640 1 -- 172.21.15.5:0/1157016912 <== mon.0 v2:172.21.15.5:3300/0 4 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x564d1348aa00 con 0x564d130ef600 2024-05-20T18:37:39.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.787+0000 7f53571b9640 1 -- 172.21.15.5:0/1157016912 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 105) ==== 2885+0+0 (secure 0 0 0) 0x564d1309f500 con 0x564d130ef600 2024-05-20T18:37:39.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.787+0000 7f5361a1f280 4 Got MDS map 105 2024-05-20T18:37:39.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.787+0000 7f5361a1f280 1 --2- 172.21.15.5:0/1157016912 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x564d130efa80 0x564d131d1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.787+0000 7f5361a1f280 1 -- 172.21.15.5:0/1157016912 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:1 17.f 17:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564d134a0000 con 0x564d130efa80 2024-05-20T18:37:39.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.787+0000 7f53609cc640 1 --2- 172.21.15.5:0/1157016912 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x564d130efa80 0x564d131d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.787+0000 7f53609cc640 1 --2- 172.21.15.5:0/1157016912 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x564d130efa80 0x564d131d1700 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=6eee0e9ee24d50fe server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.791+0000 7f53609cc640 1 -- 172.21.15.5:0/1157016912 <== osd.1 v2:172.21.15.5:6815/550664725 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v121'2 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x564d13172000 con 0x564d130efa80 2024-05-20T18:37:39.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.791+0000 7f5361a1f280 1 --2- 172.21.15.5:0/1157016912 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x564d134a6000 0x564d13424580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.791+0000 7f5361a1f280 1 -- 172.21.15.5:0/1157016912 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:2 17.14 17:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564d134a0400 con 0x564d134a6000 2024-05-20T18:37:39.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.791+0000 7f53601cb640 1 --2- 172.21.15.5:0/1157016912 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x564d134a6000 0x564d13424580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.791+0000 7f53601cb640 1 --2- 172.21.15.5:0/1157016912 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x564d134a6000 0x564d13424580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=3e8cfef6bb3adbdc server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.793+0000 7f53601cb640 1 -- 172.21.15.5:0/1157016912 <== osd.3 v2:172.21.15.5:6824/1164130599 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v121'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x564d13172500 con 0x564d134a6000 2024-05-20T18:37:39.797 INFO:teuthology.orchestra.run.smithi005.stdout:writing EResetJournal entry 2024-05-20T18:37:39.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.793+0000 7f5361a1f280 1 -- 172.21.15.5:0/1157016912 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:3 17.14 17:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564d134a0800 con 0x564d134a6000 2024-05-20T18:37:39.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.793+0000 7f5361a1f280 1 --2- 172.21.15.5:0/1157016912 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x564d134a6480 0x564d13424b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.793+0000 7f5361a1f280 1 -- 172.21.15.5:0/1157016912 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:4 17.1b 17:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564d134a0c00 con 0x564d134a6480 2024-05-20T18:37:39.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.793+0000 7f5361a1f280 1 --2- 172.21.15.5:0/1157016912 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x564d134a6900 0x564d13425080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.793+0000 7f5361a1f280 1 -- 172.21.15.5:0/1157016912 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:5 17.b 17:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564d134a1000 con 0x564d134a6900 2024-05-20T18:37:39.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.793+0000 7f5361a1f280 1 --2- 172.21.15.5:0/1157016912 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x564d134a6d80 0x564d13425600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.793+0000 7f5361a1f280 1 -- 172.21.15.5:0/1157016912 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:6 17.1f 17:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564d134a1400 con 0x564d134a6d80 2024-05-20T18:37:39.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.793+0000 7f5361a1f280 1 -- 172.21.15.5:0/1157016912 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:7 17.1a 17:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564d134a1800 con 0x564d134a6900 2024-05-20T18:37:39.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.793+0000 7f5361a1f280 1 -- 172.21.15.5:0/1157016912 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:8 17.1e 17:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564d134a1c00 con 0x564d130efa80 2024-05-20T18:37:39.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.793+0000 7f53601cb640 1 --2- 172.21.15.5:0/1157016912 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x564d134a6900 0x564d13425080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.793+0000 7f53609cc640 1 --2- 172.21.15.5:0/1157016912 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x564d134a6480 0x564d13424b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.793+0000 7f535f9ca640 1 --2- 172.21.15.5:0/1157016912 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x564d134a6d80 0x564d13425600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.794+0000 7f53609cc640 1 --2- 172.21.15.5:0/1157016912 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x564d134a6480 0x564d13424b00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=6ca4ad7bac620fe2 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.794+0000 7f53601cb640 1 --2- 172.21.15.5:0/1157016912 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x564d134a6900 0x564d13425080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=335e417d14c29d1f server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.794+0000 7f535f9ca640 1 --2- 172.21.15.5:0/1157016912 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x564d134a6d80 0x564d13425600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=c75e74b3f7093506 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.794+0000 7f53601cb640 1 -- 172.21.15.5:0/1157016912 <== osd.3 v2:172.21.15.5:6824/1164130599 2 ==== osd_op_reply(3 200.00000001 [delete] v121'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x564d13172500 con 0x564d134a6000 2024-05-20T18:37:39.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.795+0000 7f53609cc640 1 -- 172.21.15.5:0/1157016912 <== osd.1 v2:172.21.15.5:6815/550664725 2 ==== osd_op_reply(8 200.00000006 [delete] v121'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x564d13172000 con 0x564d130efa80 2024-05-20T18:37:39.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.795+0000 7f53609cc640 1 -- 172.21.15.5:0/1157016912 <== osd.4 v2:172.21.15.63:6800/3673324406 1 ==== osd_op_reply(4 200.00000002 [delete] v121'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x564d13172000 con 0x564d134a6480 2024-05-20T18:37:39.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.795+0000 7f53601cb640 1 -- 172.21.15.5:0/1157016912 <== osd.0 v2:172.21.15.5:6808/688093005 1 ==== osd_op_reply(5 200.00000003 [delete] v121'6 uv5 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x564d13172500 con 0x564d134a6900 2024-05-20T18:37:39.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.796+0000 7f53601cb640 1 -- 172.21.15.5:0/1157016912 <== osd.0 v2:172.21.15.5:6808/688093005 2 ==== osd_op_reply(7 200.00000005 [delete] v121'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x564d13172500 con 0x564d134a6900 2024-05-20T18:37:39.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.796+0000 7f53559b6640 1 -- 172.21.15.5:0/1157016912 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:9 17.14 17:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564d134a0400 con 0x564d134a6000 2024-05-20T18:37:39.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.796+0000 7f535f9ca640 1 -- 172.21.15.5:0/1157016912 <== osd.6 v2:172.21.15.63:6816/3384565140 1 ==== osd_op_reply(6 200.00000004 [delete] v121'9 uv8 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x564d13172780 con 0x564d134a6d80 2024-05-20T18:37:39.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.797+0000 7f53601cb640 1 -- 172.21.15.5:0/1157016912 <== osd.3 v2:172.21.15.5:6824/1164130599 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v121'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x564d13172500 con 0x564d134a6000 2024-05-20T18:37:39.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.797+0000 7f5361a1f280 4 Successfully wrote new journal pointer and header for rank 8:0 2024-05-20T18:37:39.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.797+0000 7f5361a1f280 1 -- 172.21.15.5:0/1157016912 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x564d134a6900 msgr2=0x564d13425080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.797+0000 7f5361a1f280 1 --2- 172.21.15.5:0/1157016912 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x564d134a6900 0x564d13425080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.797+0000 7f5361a1f280 1 -- 172.21.15.5:0/1157016912 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x564d130efa80 msgr2=0x564d131d1700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.797+0000 7f5361a1f280 1 --2- 172.21.15.5:0/1157016912 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x564d130efa80 0x564d131d1700 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.797+0000 7f5361a1f280 1 -- 172.21.15.5:0/1157016912 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x564d134a6000 msgr2=0x564d13424580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.797+0000 7f5361a1f280 1 --2- 172.21.15.5:0/1157016912 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x564d134a6000 0x564d13424580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.797+0000 7f5361a1f280 1 -- 172.21.15.5:0/1157016912 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x564d134a6480 msgr2=0x564d13424b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.797+0000 7f535f9ca640 1 -- 172.21.15.5:0/1157016912 reap_dead start 2024-05-20T18:37:39.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.797+0000 7f5361a1f280 1 --2- 172.21.15.5:0/1157016912 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x564d134a6480 0x564d13424b00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.798+0000 7f5361a1f280 1 -- 172.21.15.5:0/1157016912 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x564d134a6d80 msgr2=0x564d13425600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.798+0000 7f5361a1f280 1 --2- 172.21.15.5:0/1157016912 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x564d134a6d80 0x564d13425600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.798+0000 7f5361a1f280 1 -- 172.21.15.5:0/1157016912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d130ef600 msgr2=0x564d131d1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.798+0000 7f5361a1f280 1 --2- 172.21.15.5:0/1157016912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d130ef600 0x564d131d1180 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x564d1335b080 tx=0x564d13e22c60 comp rx=0 tx=0).stop 2024-05-20T18:37:39.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.798+0000 7f5361a1f280 1 -- 172.21.15.5:0/1157016912 shutdown_connections 2024-05-20T18:37:39.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.798+0000 7f5361a1f280 1 --2- 172.21.15.5:0/1157016912 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x564d134a6d80 0x564d13425600 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.798+0000 7f5361a1f280 1 --2- 172.21.15.5:0/1157016912 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x564d134a6480 0x564d13424b00 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.798+0000 7f5361a1f280 1 --2- 172.21.15.5:0/1157016912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d130ef600 0x564d131d1180 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.798+0000 7f5361a1f280 1 -- 172.21.15.5:0/1157016912 >> 172.21.15.5:0/1157016912 conn(0x564d130ec000 msgr2=0x564d1342e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:39.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.798+0000 7f5361a1f280 1 -- 172.21.15.5:0/1157016912 shutdown_connections 2024-05-20T18:37:39.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.798+0000 7f5361a1f280 1 -- 172.21.15.5:0/1157016912 wait complete. 2024-05-20T18:37:39.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.798+0000 7f5361a1f280 1 -- 172.21.15.5:0/3377729129 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x564d130ef180 msgr2=0x564d13424000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.798+0000 7f5361a1f280 1 --2- 172.21.15.5:0/3377729129 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x564d130ef180 0x564d13424000 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x564d13124630 tx=0x564d13e23980 comp rx=0 tx=0).stop 2024-05-20T18:37:39.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.798+0000 7f5361a1f280 1 -- 172.21.15.5:0/3377729129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d130ec900 msgr2=0x564d131cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.798+0000 7f5361a1f280 1 --2- 172.21.15.5:0/3377729129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d130ec900 0x564d131cf080 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x564d13e2db30 tx=0x564d1313a6c0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.798+0000 7f5361a1f280 1 -- 172.21.15.5:0/3377729129 shutdown_connections 2024-05-20T18:37:39.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.798+0000 7f5361a1f280 1 --2- 172.21.15.5:0/3377729129 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x564d130ef180 0x564d13424000 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.798+0000 7f5361a1f280 1 --2- 172.21.15.5:0/3377729129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d130ecd80 0x564d131ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.798+0000 7f5361a1f280 1 --2- 172.21.15.5:0/3377729129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d130ec900 0x564d131cf080 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.798+0000 7f5361a1f280 1 --2- 172.21.15.5:0/3377729129 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d130ec480 0x564d131d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.798+0000 7f5361a1f280 1 -- 172.21.15.5:0/3377729129 >> 172.21.15.5:0/3377729129 conn(0x564d130ed200 msgr2=0x564d1315f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:39.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.798+0000 7f5361a1f280 1 -- 172.21.15.5:0/3377729129 shutdown_connections 2024-05-20T18:37:39.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.798+0000 7f5361a1f280 1 -- 172.21.15.5:0/3377729129 wait complete. 2024-05-20T18:37:39.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.799+0000 7f5361a1f280 1 -- 172.21.15.5:0/1867002457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d130ee400 msgr2=0x564d131ce580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.799+0000 7f5361a1f280 1 --2- 172.21.15.5:0/1867002457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d130ee400 0x564d131ce580 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x564d13e6b3e0 tx=0x564d1313a660 comp rx=0 tx=0).stop 2024-05-20T18:37:39.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.799+0000 7f5361a1f280 1 -- 172.21.15.5:0/1867002457 shutdown_connections 2024-05-20T18:37:39.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.799+0000 7f5361a1f280 1 --2- 172.21.15.5:0/1867002457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d130ee400 0x564d131ce580 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.799+0000 7f5361a1f280 1 --2- 172.21.15.5:0/1867002457 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d130edb00 0x564d131cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.799+0000 7f5361a1f280 1 --2- 172.21.15.5:0/1867002457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d130ed680 0x564d131cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.799+0000 7f5361a1f280 1 -- 172.21.15.5:0/1867002457 >> 172.21.15.5:0/1867002457 conn(0x564d130edf80 msgr2=0x564d1315f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:39.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.799+0000 7f5361a1f280 1 -- 172.21.15.5:0/1867002457 shutdown_connections 2024-05-20T18:37:39.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.799+0000 7f5361a1f280 1 -- 172.21.15.5:0/1867002457 wait complete. 2024-05-20T18:37:39.808 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.091884, result: writing EResetJournal entry 2024-05-20T18:37:39.808 DEBUG:teuthology.orchestra.run.smithi005:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-05-20T18:37:39.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.859+0000 7f8d3c01f740 1 -- start start 2024-05-20T18:37:39.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.859+0000 7f8d3c01f740 1 Processor -- start 2024-05-20T18:37:39.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.860+0000 7f8d3c01f740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a092b25680 0x55a092af8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.861+0000 7f8d3c01f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a092b25b00 0x55a092af8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.861+0000 7f8d3c01f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a092b25f80 0x55a092af9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.861+0000 7f8d3c01f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a093746600 con 0x55a092b25f80 2024-05-20T18:37:39.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.861+0000 7f8d3c01f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a093746480 con 0x55a092b25b00 2024-05-20T18:37:39.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.861+0000 7f8d3c01f740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55a093746300 con 0x55a092b25680 2024-05-20T18:37:39.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.861+0000 7f8d3a5dc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a092b25f80 0x55a092af9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.861+0000 7f8d3a5dc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a092b25f80 0x55a092af9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35596/0 (socket says 172.21.15.5:35596) 2024-05-20T18:37:39.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.861+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/337140378 learned_addr learned my addr 172.21.15.5:0/337140378 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:39.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.861+0000 7f8d39ddb640 1 --2- 172.21.15.5:0/337140378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a092b25680 0x55a092af8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.861+0000 7f8d395da640 1 --2- 172.21.15.5:0/337140378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a092b25b00 0x55a092af8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.861+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/337140378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a092b25680 msgr2=0x55a092af8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:39.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.861+0000 7f8d3a5dc640 1 --2- 172.21.15.5:0/337140378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a092b25680 0x55a092af8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.861+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/337140378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a092b25b00 msgr2=0x55a092af8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:39.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.861+0000 7f8d3a5dc640 1 --2- 172.21.15.5:0/337140378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a092b25b00 0x55a092af8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.861+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/337140378 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a08e967680 con 0x55a092b25f80 2024-05-20T18:37:39.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.861+0000 7f8d395da640 1 --2- 172.21.15.5:0/337140378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a092b25b00 0x55a092af8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:39.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.861+0000 7f8d3a5dc640 1 --2- 172.21.15.5:0/337140378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a092b25f80 0x55a092af9080 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x55a093787d40 tx=0x55a092a64060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2cfbc24dd55c43e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.861+0000 7f8d375d6640 1 -- 172.21.15.5:0/337140378 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a08e960680 con 0x55a092b25f80 2024-05-20T18:37:39.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.861+0000 7f8d375d6640 1 -- 172.21.15.5:0/337140378 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55a0929b4c40 con 0x55a092b25f80 2024-05-20T18:37:39.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.861+0000 7f8d375d6640 1 -- 172.21.15.5:0/337140378 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a08e9609c0 con 0x55a092b25f80 2024-05-20T18:37:39.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.861+0000 7f8d3c01f740 1 -- 172.21.15.5:0/337140378 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a08e967a40 con 0x55a092b25f80 2024-05-20T18:37:39.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.862+0000 7f8d3c01f740 1 -- 172.21.15.5:0/337140378 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55a08e9672c0 con 0x55a092b25f80 2024-05-20T18:37:39.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.862+0000 7f8d3c01f740 4 waiting for MDS map... 2024-05-20T18:37:39.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.862+0000 7f8d375d6640 1 -- 172.21.15.5:0/337140378 <== mon.0 v2:172.21.15.5:3300/0 4 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x55a093840a00 con 0x55a092b25f80 2024-05-20T18:37:39.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.862+0000 7f8d375d6640 1 -- 172.21.15.5:0/337140378 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 105) ==== 2885+0+0 (secure 0 0 0) 0x55a0929c3180 con 0x55a092b25f80 2024-05-20T18:37:39.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.862+0000 7f8d3c01f740 4 Got MDS map 105 2024-05-20T18:37:39.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.862+0000 7f8d3c01f740 4 datascan.main: Using metadata pool output 2024-05-20T18:37:39.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.862+0000 7f8d3c01f740 4 datascan.main: connecting to RADOS... 2024-05-20T18:37:39.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.863+0000 7f8d3c01f740 1 Processor -- start 2024-05-20T18:37:39.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.863+0000 7f8d3c01f740 1 -- start start 2024-05-20T18:37:39.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.864+0000 7f8d3c01f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a092b24d80 0x55a092af9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.864+0000 7f8d3c01f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a092b24900 0x55a092af9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.864+0000 7f8d3c01f740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a092b26880 0x55a092afa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.864+0000 7f8d3c01f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a093746180 con 0x55a092b24d80 2024-05-20T18:37:39.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.864+0000 7f8d3c01f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a0929ca480 con 0x55a092b24900 2024-05-20T18:37:39.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.864+0000 7f8d3c01f740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55a0929ca900 con 0x55a092b26880 2024-05-20T18:37:39.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.864+0000 7f8d395da640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a092b24d80 0x55a092af9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.864+0000 7f8d395da640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a092b24d80 0x55a092af9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35610/0 (socket says 172.21.15.5:35610) 2024-05-20T18:37:39.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.864+0000 7f8d395da640 1 -- 172.21.15.5:0/259364708 learned_addr learned my addr 172.21.15.5:0/259364708 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:39.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.864+0000 7f8d39ddb640 1 --2- 172.21.15.5:0/259364708 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a092b24900 0x55a092af9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.864+0000 7f8d395da640 1 --2- 172.21.15.5:0/259364708 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a092b26880 0x55a092afa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.864+0000 7f8d395da640 1 -- 172.21.15.5:0/259364708 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a092b26880 msgr2=0x55a092afa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:39.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.864+0000 7f8d395da640 1 --2- 172.21.15.5:0/259364708 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a092b26880 0x55a092afa100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.864+0000 7f8d395da640 1 -- 172.21.15.5:0/259364708 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a092b24900 msgr2=0x55a092af9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:39.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.864+0000 7f8d395da640 1 --2- 172.21.15.5:0/259364708 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a092b24900 0x55a092af9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.864+0000 7f8d395da640 1 -- 172.21.15.5:0/259364708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a08e9674a0 con 0x55a092b24d80 2024-05-20T18:37:39.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.865+0000 7f8d395da640 1 --2- 172.21.15.5:0/259364708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a092b24d80 0x55a092af9600 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x55a093767320 tx=0x55a09384f860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed5a2315bc1f0be server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.865+0000 7f8d355d2640 1 -- 172.21.15.5:0/259364708 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a08e960680 con 0x55a092b24d80 2024-05-20T18:37:39.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.865+0000 7f8d355d2640 1 -- 172.21.15.5:0/259364708 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55a0929b41c0 con 0x55a092b24d80 2024-05-20T18:37:39.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.865+0000 7f8d355d2640 1 -- 172.21.15.5:0/259364708 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a08e9609c0 con 0x55a092b24d80 2024-05-20T18:37:39.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.865+0000 7f8d3c01f740 1 -- 172.21.15.5:0/259364708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a092b24d80 msgr2=0x55a092af9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.865+0000 7f8d3c01f740 1 --2- 172.21.15.5:0/259364708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a092b24d80 0x55a092af9600 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x55a093767320 tx=0x55a09384f860 comp rx=0 tx=0).stop 2024-05-20T18:37:39.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.865+0000 7f8d3c01f740 1 -- 172.21.15.5:0/259364708 shutdown_connections 2024-05-20T18:37:39.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.865+0000 7f8d3c01f740 1 --2- 172.21.15.5:0/259364708 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a092b26880 0x55a092afa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.865+0000 7f8d3c01f740 1 --2- 172.21.15.5:0/259364708 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a092b24900 0x55a092af9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.865+0000 7f8d3c01f740 1 --2- 172.21.15.5:0/259364708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a092b24d80 0x55a092af9600 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.865+0000 7f8d3c01f740 1 -- 172.21.15.5:0/259364708 >> 172.21.15.5:0/259364708 conn(0x55a092b25200 msgr2=0x55a092a90000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:39.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.865+0000 7f8d3c01f740 1 -- 172.21.15.5:0/259364708 shutdown_connections 2024-05-20T18:37:39.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.865+0000 7f8d3c01f740 1 -- 172.21.15.5:0/259364708 wait complete. 2024-05-20T18:37:39.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.866+0000 7f8d3c01f740 1 Processor -- start 2024-05-20T18:37:39.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.866+0000 7f8d3c01f740 1 -- start start 2024-05-20T18:37:39.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.867+0000 7f8d3c01f740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a092b24900 0x55a092afb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.867+0000 7f8d3c01f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a092b24d80 0x55a092afac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.867+0000 7f8d3c01f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a092b26880 0x55a092afa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.867+0000 7f8d3c01f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a093747380 con 0x55a092b26880 2024-05-20T18:37:39.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.867+0000 7f8d3c01f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a093746180 con 0x55a092b24d80 2024-05-20T18:37:39.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.867+0000 7f8d3c01f740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55a093747500 con 0x55a092b24900 2024-05-20T18:37:39.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.868+0000 7f8d395da640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a092b26880 0x55a092afa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.868+0000 7f8d395da640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a092b26880 0x55a092afa680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35614/0 (socket says 172.21.15.5:35614) 2024-05-20T18:37:39.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.868+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 learned_addr learned my addr 172.21.15.5:0/3072204518 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:39.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.868+0000 7f8d395da640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a092b24900 0x55a092afb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.868+0000 7f8d39ddb640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a092b24d80 0x55a092afac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.868+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a092b24900 msgr2=0x55a092afb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.868+0000 7f8d395da640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a092b24900 0x55a092afb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.868+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a092b24d80 msgr2=0x55a092afac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.868+0000 7f8d395da640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a092b24d80 0x55a092afac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.868+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a08e967860 con 0x55a092b26880 2024-05-20T18:37:39.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.868+0000 7f8d395da640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a092b26880 0x55a092afa680 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x55a0938cc000 tx=0x55a092a641e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.868+0000 7f8d33dcf640 1 -- 172.21.15.5:0/3072204518 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a08e960b60 con 0x55a092b26880 2024-05-20T18:37:39.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.868+0000 7f8d33dcf640 1 -- 172.21.15.5:0/3072204518 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55a0929b5340 con 0x55a092b26880 2024-05-20T18:37:39.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.868+0000 7f8d33dcf640 1 -- 172.21.15.5:0/3072204518 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a08e960680 con 0x55a092b26880 2024-05-20T18:37:39.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.868+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a08e967680 con 0x55a092b26880 2024-05-20T18:37:39.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.868+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a08e967a40 con 0x55a092b26880 2024-05-20T18:37:39.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.868+0000 7f8d3c01f740 4 datascan.init: resolving metadata pool 17 2024-05-20T18:37:39.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.870+0000 7f8d33dcf640 1 -- 172.21.15.5:0/3072204518 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x55a0938b6c00 con 0x55a092b26880 2024-05-20T18:37:39.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.870+0000 7f8d33dcf640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x55a092b27a80 0x55a092afb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.870+0000 7f8d33dcf640 1 -- 172.21.15.5:0/3072204518 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x55a0938b4a00 con 0x55a092b26880 2024-05-20T18:37:39.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.870+0000 7f8d39ddb640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x55a092b27a80 0x55a092afb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.870+0000 7f8d3c01f740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-20T18:37:39.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.870+0000 7f8d3c01f740 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-05-20T18:37:39.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.870+0000 7f8d3c01f740 4 datascan.main: opening data pool 'cephfs_data' 2024-05-20T18:37:39.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.870+0000 7f8d3c01f740 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x55a092b26d00 0x55a092afa100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.870+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:1 18.0 18:00000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a0938b2800 con 0x55a092b26d00 2024-05-20T18:37:39.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.870+0000 7f8d395da640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x55a092b26d00 0x55a092afa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.871+0000 7f8d39ddb640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x55a092b27a80 0x55a092afb700 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x55a0929ab650 tx=0x55a092a60b40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.871+0000 7f8d395da640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x55a092b26d00 0x55a092afa100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.871+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.0 v2:172.21.15.5:6808/688093005 1 ==== osd_op_reply(1 [pgnls start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98000 con 0x55a092b26d00 2024-05-20T18:37:39.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.871+0000 7f8d34dd1640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x55a092d60000 0x55a092d68000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.871+0000 7f8d34dd1640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:2 18.20 18:04000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a0938b3000 con 0x55a092d60000 2024-05-20T18:37:39.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.871+0000 7f8d3a5dc640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x55a092d60000 0x55a092d68000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.872+0000 7f8d3a5dc640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x55a092d60000 0x55a092d68000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.874+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.5 v2:172.21.15.63:6808/1616066286 1 ==== osd_op_reply(2 [pgnls start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98500 con 0x55a092d60000 2024-05-20T18:37:39.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.874+0000 7f8d355d2640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x55a092d60480 0x55a092d68580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.874+0000 7f8d355d2640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:3 18.10 18:08000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a0938b3400 con 0x55a092d60480 2024-05-20T18:37:39.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.875+0000 7f8d39ddb640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x55a092d60480 0x55a092d68580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.875+0000 7f8d39ddb640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x55a092d60480 0x55a092d68580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.875+0000 7f8d39ddb640 1 -- 172.21.15.5:0/3072204518 <== osd.6 v2:172.21.15.63:6816/3384565140 1 ==== osd_op_reply(3 [pgnls start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98780 con 0x55a092d60480 2024-05-20T18:37:39.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.875+0000 7f8d34dd1640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:4 18.30 18:0c000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a0938b3800 con 0x55a092b26d00 2024-05-20T18:37:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.876+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.0 v2:172.21.15.5:6808/688093005 2 ==== osd_op_reply(4 [pgnls start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98000 con 0x55a092b26d00 2024-05-20T18:37:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.876+0000 7f8d355d2640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:5 18.8 18:10000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d74000 con 0x55a092b26d00 2024-05-20T18:37:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.876+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.0 v2:172.21.15.5:6808/688093005 3 ==== osd_op_reply(5 [pgnls start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98000 con 0x55a092b26d00 2024-05-20T18:37:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.876+0000 7f8d34dd1640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:6 18.28 18:14000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a0938b3c00 con 0x55a092d60000 2024-05-20T18:37:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.876+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.5 v2:172.21.15.63:6808/1616066286 2 ==== osd_op_reply(6 [pgnls start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98500 con 0x55a092d60000 2024-05-20T18:37:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.876+0000 7f8d355d2640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55a092d60900 0x55a092d68b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.876+0000 7f8d355d2640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:7 18.18 18:18000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d74400 con 0x55a092d60900 2024-05-20T18:37:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.876+0000 7f8d395da640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55a092d60900 0x55a092d68b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.877+0000 7f8d395da640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55a092d60900 0x55a092d68b00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.877+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.1 v2:172.21.15.5:6815/550664725 1 ==== osd_op_reply(7 [pgnls start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98000 con 0x55a092d60900 2024-05-20T18:37:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.877+0000 7f8d34dd1640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x55a092d61200 0x55a092d69600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.877+0000 7f8d34dd1640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:8 18.38 18:1c000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a0938b2800 con 0x55a092d61200 2024-05-20T18:37:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.877+0000 7f8d3a5dc640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x55a092d61200 0x55a092d69600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.877+0000 7f8d3a5dc640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x55a092d61200 0x55a092d69600 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.878+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.3 v2:172.21.15.5:6824/1164130599 1 ==== osd_op_reply(8 [pgnls start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98500 con 0x55a092d61200 2024-05-20T18:37:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.878+0000 7f8d355d2640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:9 18.4 18:20000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a0938b3000 con 0x55a092b26d00 2024-05-20T18:37:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.878+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.0 v2:172.21.15.5:6808/688093005 4 ==== osd_op_reply(9 [pgnls start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98000 con 0x55a092b26d00 2024-05-20T18:37:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.878+0000 7f8d34dd1640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:10 18.24 18:24000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a0938b3800 con 0x55a092b26d00 2024-05-20T18:37:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.878+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.0 v2:172.21.15.5:6808/688093005 5 ==== osd_op_reply(10 [pgnls start_epoch 121 out=49b] v119'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98000 con 0x55a092b26d00 2024-05-20T18:37:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.878+0000 7f8d355d2640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:11 18.14 18:28000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a09383f400 con 0x55a092d60480 2024-05-20T18:37:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.879+0000 7f8d39ddb640 1 -- 172.21.15.5:0/3072204518 <== osd.6 v2:172.21.15.63:6816/3384565140 2 ==== osd_op_reply(11 [pgnls start_epoch 121 out=49b] v119'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98780 con 0x55a092d60480 2024-05-20T18:37:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.879+0000 7f8d34dd1640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:12 18.34 18:2c000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a0938b3c00 con 0x55a092b26d00 2024-05-20T18:37:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.879+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.0 v2:172.21.15.5:6808/688093005 6 ==== osd_op_reply(12 [pgnls start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98000 con 0x55a092b26d00 2024-05-20T18:37:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.879+0000 7f8d355d2640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:13 18.c 18:30000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a09383f000 con 0x55a092d60000 2024-05-20T18:37:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.879+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.5 v2:172.21.15.63:6808/1616066286 3 ==== osd_op_reply(13 [pgnls start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98500 con 0x55a092d60000 2024-05-20T18:37:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.879+0000 7f8d34dd1640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x55a092d61680 0x55a092d69b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.879+0000 7f8d34dd1640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:14 18.2c 18:34000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a0938b2800 con 0x55a092d61680 2024-05-20T18:37:39.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.880+0000 7f8d39ddb640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x55a092d61680 0x55a092d69b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.880+0000 7f8d39ddb640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x55a092d61680 0x55a092d69b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.880+0000 7f8d39ddb640 1 -- 172.21.15.5:0/3072204518 <== osd.2 v2:172.21.15.5:6800/921679764 1 ==== osd_op_reply(14 [pgnls start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98780 con 0x55a092d61680 2024-05-20T18:37:39.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.880+0000 7f8d355d2640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:15 18.1c 18:38000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d82000 con 0x55a092d60900 2024-05-20T18:37:39.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.880+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.1 v2:172.21.15.5:6815/550664725 2 ==== osd_op_reply(15 [pgnls start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98000 con 0x55a092d60900 2024-05-20T18:37:39.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.880+0000 7f8d34dd1640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x55a092d61b00 0x55a092d6a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.880+0000 7f8d34dd1640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:16 18.3c 18:3c000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d83000 con 0x55a092d61b00 2024-05-20T18:37:39.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.881+0000 7f8d395da640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x55a092d61b00 0x55a092d6a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.881+0000 7f8d395da640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x55a092d61b00 0x55a092d6a100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.881+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.7 v2:172.21.15.63:6824/4136124451 1 ==== osd_op_reply(16 [pgnls start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98000 con 0x55a092d61b00 2024-05-20T18:37:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.882+0000 7f8d355d2640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:17 18.2 18:40000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d82400 con 0x55a092d61b00 2024-05-20T18:37:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.882+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.7 v2:172.21.15.63:6824/4136124451 2 ==== osd_op_reply(17 [pgnls start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98000 con 0x55a092d61b00 2024-05-20T18:37:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.882+0000 7f8d34dd1640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:18 18.22 18:44000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d83400 con 0x55a092d61200 2024-05-20T18:37:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.882+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.3 v2:172.21.15.5:6824/1164130599 2 ==== osd_op_reply(18 [pgnls start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98500 con 0x55a092d61200 2024-05-20T18:37:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.882+0000 7f8d355d2640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:19 18.12 18:48000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d82800 con 0x55a092d61680 2024-05-20T18:37:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.883+0000 7f8d39ddb640 1 -- 172.21.15.5:0/3072204518 <== osd.2 v2:172.21.15.5:6800/921679764 2 ==== osd_op_reply(19 [pgnls start_epoch 121 out=81b] v120'22 uv22 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55a092a98780 con 0x55a092d61680 2024-05-20T18:37:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.883+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:20 18.12 18:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55a0938b2c00 con 0x55a092d61680 2024-05-20T18:37:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.883+0000 7f8d39ddb640 1 -- 172.21.15.5:0/3072204518 <== osd.2 v2:172.21.15.5:6800/921679764 3 ==== osd_op_reply(20 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55a092a98780 con 0x55a092d61680 2024-05-20T18:37:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.883+0000 7f8d3c01f740 10 datascan.operator(): handling object 1099511627778.0 2024-05-20T18:37:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.883+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:21 18.12 18:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55a0938b3400 con 0x55a092d61680 2024-05-20T18:37:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.884+0000 7f8d39ddb640 1 -- 172.21.15.5:0/3072204518 <== osd.2 v2:172.21.15.5:6800/921679764 4 ==== osd_op_reply(21 10000000002.00000000 [call] v121'23 uv23 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55a092a98780 con 0x55a092d61680 2024-05-20T18:37:39.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.884+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:22 18.32 18:4c000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a0938b2800 con 0x55a092d61200 2024-05-20T18:37:39.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.885+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.3 v2:172.21.15.5:6824/1164130599 3 ==== osd_op_reply(22 [pgnls start_epoch 121 out=49b] v119'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98500 con 0x55a092d61200 2024-05-20T18:37:39.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.885+0000 7f8d355d2640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:23 18.a 18:50000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d82c00 con 0x55a092d61200 2024-05-20T18:37:39.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.885+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.3 v2:172.21.15.5:6824/1164130599 4 ==== osd_op_reply(23 [pgnls start_epoch 121 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98500 con 0x55a092d61200 2024-05-20T18:37:39.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.885+0000 7f8d34dd1640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x55a092d61f80 0x55a092d6a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.885+0000 7f8d34dd1640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:24 18.2a 18:54000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d83800 con 0x55a092d61f80 2024-05-20T18:37:39.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.885+0000 7f8d3a5dc640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x55a092d61f80 0x55a092d6a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.886+0000 7f8d3a5dc640 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x55a092d61f80 0x55a092d6a680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.886+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.4 v2:172.21.15.63:6800/3673324406 1 ==== osd_op_reply(24 [pgnls start_epoch 121 out=49b] v119'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98500 con 0x55a092d61f80 2024-05-20T18:37:39.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.886+0000 7f8d355d2640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:25 18.1a 18:58000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d82800 con 0x55a092d61200 2024-05-20T18:37:39.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.886+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.3 v2:172.21.15.5:6824/1164130599 5 ==== osd_op_reply(25 [pgnls start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98500 con 0x55a092d61200 2024-05-20T18:37:39.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.886+0000 7f8d34dd1640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:26 18.3a 18:5c000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d83c00 con 0x55a092b26d00 2024-05-20T18:37:39.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.887+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.0 v2:172.21.15.5:6808/688093005 7 ==== osd_op_reply(26 [pgnls start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98000 con 0x55a092b26d00 2024-05-20T18:37:39.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.887+0000 7f8d355d2640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:27 18.6 18:60000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a0938b2c00 con 0x55a092d61b00 2024-05-20T18:37:39.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.887+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.7 v2:172.21.15.63:6824/4136124451 3 ==== osd_op_reply(27 [pgnls start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98000 con 0x55a092d61b00 2024-05-20T18:37:39.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.887+0000 7f8d34dd1640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:28 18.26 18:64000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d83400 con 0x55a092d61680 2024-05-20T18:37:39.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.887+0000 7f8d39ddb640 1 -- 172.21.15.5:0/3072204518 <== osd.2 v2:172.21.15.5:6800/921679764 5 ==== osd_op_reply(28 [pgnls start_epoch 121 out=49b] v119'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98780 con 0x55a092d61680 2024-05-20T18:37:39.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.887+0000 7f8d355d2640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:29 18.16 18:68000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a0938b3000 con 0x55a092d61200 2024-05-20T18:37:39.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.888+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.3 v2:172.21.15.5:6824/1164130599 6 ==== osd_op_reply(29 [pgnls start_epoch 121 out=49b] v119'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98500 con 0x55a092d61200 2024-05-20T18:37:39.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.888+0000 7f8d34dd1640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:30 18.36 18:6c000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d82c00 con 0x55a092d60900 2024-05-20T18:37:39.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.888+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.1 v2:172.21.15.5:6815/550664725 3 ==== osd_op_reply(30 [pgnls start_epoch 121 out=81b] v119'33 uv17 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55a092a98000 con 0x55a092d60900 2024-05-20T18:37:39.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.888+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:31 18.36 18:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55a09383f400 con 0x55a092d60900 2024-05-20T18:37:39.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.888+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.1 v2:172.21.15.5:6815/550664725 4 ==== osd_op_reply(31 100000001f7.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55a092a98000 con 0x55a092d60900 2024-05-20T18:37:39.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.888+0000 7f8d3c01f740 10 datascan.operator(): handling object 1099511628279.0 2024-05-20T18:37:39.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.888+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:32 18.36 18:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55a092d82000 con 0x55a092d60900 2024-05-20T18:37:39.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.889+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.1 v2:172.21.15.5:6815/550664725 5 ==== osd_op_reply(32 100000001f7.00000000 [call] v121'34 uv34 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55a092a98000 con 0x55a092d60900 2024-05-20T18:37:39.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.890+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:33 18.e 18:70000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d83000 con 0x55a092b26d00 2024-05-20T18:37:39.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.890+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.0 v2:172.21.15.5:6808/688093005 8 ==== osd_op_reply(33 [pgnls start_epoch 121 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98000 con 0x55a092b26d00 2024-05-20T18:37:39.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.890+0000 7f8d34dd1640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:34 18.2e 18:74000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d83800 con 0x55a092d61200 2024-05-20T18:37:39.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.890+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.3 v2:172.21.15.5:6824/1164130599 7 ==== osd_op_reply(34 [pgnls start_epoch 121 out=49b] v119'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98500 con 0x55a092d61200 2024-05-20T18:37:39.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.890+0000 7f8d355d2640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:35 18.1e 18:78000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a0938b3800 con 0x55a092d60000 2024-05-20T18:37:39.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.891+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.5 v2:172.21.15.63:6808/1616066286 4 ==== osd_op_reply(35 [pgnls start_epoch 121 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98500 con 0x55a092d60000 2024-05-20T18:37:39.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.891+0000 7f8d34dd1640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:36 18.3e 18:7c000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d82800 con 0x55a092d61200 2024-05-20T18:37:39.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.891+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.3 v2:172.21.15.5:6824/1164130599 8 ==== osd_op_reply(36 [pgnls start_epoch 121 out=49b] v119'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98500 con 0x55a092d61200 2024-05-20T18:37:39.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.891+0000 7f8d355d2640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:37 18.1 18:80000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a0938b3c00 con 0x55a092d61200 2024-05-20T18:37:39.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.891+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.3 v2:172.21.15.5:6824/1164130599 9 ==== osd_op_reply(37 [pgnls start_epoch 121 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98500 con 0x55a092d61200 2024-05-20T18:37:39.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.891+0000 7f8d34dd1640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:38 18.21 18:84000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a0938b2800 con 0x55a092d60000 2024-05-20T18:37:39.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.892+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.5 v2:172.21.15.63:6808/1616066286 5 ==== osd_op_reply(38 [pgnls start_epoch 121 out=49b] v119'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98500 con 0x55a092d60000 2024-05-20T18:37:39.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.892+0000 7f8d355d2640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:39 18.11 18:88000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d82400 con 0x55a092d60900 2024-05-20T18:37:39.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.892+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.1 v2:172.21.15.5:6815/550664725 6 ==== osd_op_reply(39 [pgnls start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98000 con 0x55a092d60900 2024-05-20T18:37:39.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.892+0000 7f8d34dd1640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:40 18.31 18:8c000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a09383f000 con 0x55a092d61f80 2024-05-20T18:37:39.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.892+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.4 v2:172.21.15.63:6800/3673324406 2 ==== osd_op_reply(40 [pgnls start_epoch 121 out=81b] v119'15 uv8 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55a092a98500 con 0x55a092d61f80 2024-05-20T18:37:39.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.893+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:41 18.31 18:8de29a03:::10000000004.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55a0938b3000 con 0x55a092d61f80 2024-05-20T18:37:39.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.893+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.4 v2:172.21.15.63:6800/3673324406 3 ==== osd_op_reply(41 10000000004.00000001 [stat out=16b] v0'0 uv8 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55a092a98500 con 0x55a092d61f80 2024-05-20T18:37:39.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.893+0000 7f8d3c01f740 10 datascan.operator(): handling object 1099511627780.1 2024-05-20T18:37:39.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.893+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:42 18.5 18:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55a0938b3800 con 0x55a092d60900 2024-05-20T18:37:39.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.894+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.1 v2:172.21.15.5:6815/550664725 7 ==== osd_op_reply(42 10000000004.00000000 [call] v121'14 uv14 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55a092a98000 con 0x55a092d60900 2024-05-20T18:37:39.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.894+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:43 18.9 18:90000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a0938b3c00 con 0x55a092d61200 2024-05-20T18:37:39.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.895+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.3 v2:172.21.15.5:6824/1164130599 10 ==== osd_op_reply(43 [pgnls start_epoch 121 out=49b] v119'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98500 con 0x55a092d61200 2024-05-20T18:37:39.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.895+0000 7f8d34dd1640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:44 18.29 18:94000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a09383f000 con 0x55a092d61b00 2024-05-20T18:37:39.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.895+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.7 v2:172.21.15.63:6824/4136124451 4 ==== osd_op_reply(44 [pgnls start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98000 con 0x55a092d61b00 2024-05-20T18:37:39.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.895+0000 7f8d355d2640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:45 18.19 18:98000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d83c00 con 0x55a092d61f80 2024-05-20T18:37:39.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.896+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.4 v2:172.21.15.63:6800/3673324406 4 ==== osd_op_reply(45 [pgnls start_epoch 121 out=81b] v120'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55a092a98500 con 0x55a092d61f80 2024-05-20T18:37:39.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.896+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:46 18.19 18:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55a092d83800 con 0x55a092d61f80 2024-05-20T18:37:39.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.896+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.4 v2:172.21.15.63:6800/3673324406 5 ==== osd_op_reply(46 100000001f8.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55a092a98500 con 0x55a092d61f80 2024-05-20T18:37:39.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.896+0000 7f8d3c01f740 10 datascan.operator(): handling object 1099511628280.0 2024-05-20T18:37:39.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.896+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:47 18.19 18:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55a092d82800 con 0x55a092d61f80 2024-05-20T18:37:39.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.898+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.4 v2:172.21.15.63:6800/3673324406 6 ==== osd_op_reply(47 100000001f8.00000000 [call] v121'24 uv24 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55a092a98500 con 0x55a092d61f80 2024-05-20T18:37:39.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.898+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:48 18.39 18:9c000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d96000 con 0x55a092b26d00 2024-05-20T18:37:39.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.898+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.0 v2:172.21.15.5:6808/688093005 9 ==== osd_op_reply(48 [pgnls start_epoch 121 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98000 con 0x55a092b26d00 2024-05-20T18:37:39.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.898+0000 7f8d355d2640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:49 18.5 18:a0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d82c00 con 0x55a092d60900 2024-05-20T18:37:39.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.898+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.1 v2:172.21.15.5:6815/550664725 8 ==== osd_op_reply(49 [pgnls start_epoch 121 out=81b] v121'14 uv14 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55a092a98000 con 0x55a092d60900 2024-05-20T18:37:39.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.898+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:50 18.5 18:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55a092d96400 con 0x55a092d60900 2024-05-20T18:37:39.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.898+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.1 v2:172.21.15.5:6815/550664725 9 ==== osd_op_reply(50 10000000004.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55a092a98000 con 0x55a092d60900 2024-05-20T18:37:39.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.898+0000 7f8d3c01f740 10 datascan.operator(): handling object 1099511627780.0 2024-05-20T18:37:39.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.898+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:51 18.5 18:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55a092d96800 con 0x55a092d60900 2024-05-20T18:37:39.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.900+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.1 v2:172.21.15.5:6815/550664725 10 ==== osd_op_reply(51 10000000004.00000000 [call] v121'15 uv15 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55a092a98000 con 0x55a092d60900 2024-05-20T18:37:39.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.900+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:52 18.25 18:a4000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d96c00 con 0x55a092d60480 2024-05-20T18:37:39.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.900+0000 7f8d39ddb640 1 -- 172.21.15.5:0/3072204518 <== osd.6 v2:172.21.15.63:6816/3384565140 3 ==== osd_op_reply(52 [pgnls start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98780 con 0x55a092d60480 2024-05-20T18:37:39.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.900+0000 7f8d355d2640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:53 18.15 18:a8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d82000 con 0x55a092d60000 2024-05-20T18:37:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.900+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.5 v2:172.21.15.63:6808/1616066286 6 ==== osd_op_reply(53 [pgnls start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98500 con 0x55a092d60000 2024-05-20T18:37:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.900+0000 7f8d34dd1640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:54 18.35 18:ac000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a0938b2800 con 0x55a092d61f80 2024-05-20T18:37:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.901+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.4 v2:172.21.15.63:6800/3673324406 7 ==== osd_op_reply(54 [pgnls start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98500 con 0x55a092d61f80 2024-05-20T18:37:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.901+0000 7f8d355d2640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:55 18.d 18:b0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a09383f400 con 0x55a092d61f80 2024-05-20T18:37:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.901+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.4 v2:172.21.15.63:6800/3673324406 8 ==== osd_op_reply(55 [pgnls start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98500 con 0x55a092d61f80 2024-05-20T18:37:39.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.901+0000 7f8d34dd1640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:56 18.2d 18:b4000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d83c00 con 0x55a092d60480 2024-05-20T18:37:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.902+0000 7f8d39ddb640 1 -- 172.21.15.5:0/3072204518 <== osd.6 v2:172.21.15.63:6816/3384565140 4 ==== osd_op_reply(56 [pgnls start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98780 con 0x55a092d60480 2024-05-20T18:37:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.902+0000 7f8d355d2640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:57 18.1d 18:b8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a0938b2c00 con 0x55a092b26d00 2024-05-20T18:37:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.902+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.0 v2:172.21.15.5:6808/688093005 10 ==== osd_op_reply(57 [pgnls start_epoch 121 out=49b] v119'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98000 con 0x55a092b26d00 2024-05-20T18:37:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.902+0000 7f8d34dd1640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:58 18.3d 18:bc000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d83800 con 0x55a092d61200 2024-05-20T18:37:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.902+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.3 v2:172.21.15.5:6824/1164130599 11 ==== osd_op_reply(58 [pgnls start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98500 con 0x55a092d61200 2024-05-20T18:37:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.902+0000 7f8d355d2640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:59 18.3 18:c0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d96000 con 0x55a092d60900 2024-05-20T18:37:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.902+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.1 v2:172.21.15.5:6815/550664725 11 ==== osd_op_reply(59 [pgnls start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98000 con 0x55a092d60900 2024-05-20T18:37:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.902+0000 7f8d34dd1640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:60 18.23 18:c4000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a09383f400 con 0x55a092d61680 2024-05-20T18:37:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.903+0000 7f8d39ddb640 1 -- 172.21.15.5:0/3072204518 <== osd.2 v2:172.21.15.5:6800/921679764 6 ==== osd_op_reply(60 [pgnls start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98780 con 0x55a092d61680 2024-05-20T18:37:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.903+0000 7f8d355d2640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:61 18.13 18:c8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d96400 con 0x55a092d61b00 2024-05-20T18:37:39.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.903+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.7 v2:172.21.15.63:6824/4136124451 5 ==== osd_op_reply(61 [pgnls start_epoch 121 out=49b] v119'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98000 con 0x55a092d61b00 2024-05-20T18:37:39.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.903+0000 7f8d34dd1640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:62 18.33 18:cc000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a09383f000 con 0x55a092d60900 2024-05-20T18:37:39.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.903+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.1 v2:172.21.15.5:6815/550664725 12 ==== osd_op_reply(62 [pgnls start_epoch 121 out=49b] v119'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98000 con 0x55a092d60900 2024-05-20T18:37:39.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.903+0000 7f8d355d2640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:63 18.b 18:d0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d82800 con 0x55a092d60000 2024-05-20T18:37:39.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.904+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.5 v2:172.21.15.63:6808/1616066286 7 ==== osd_op_reply(63 [pgnls start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98500 con 0x55a092d60000 2024-05-20T18:37:39.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.904+0000 7f8d34dd1640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:64 18.2b 18:d4000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a0938b3000 con 0x55a092d61b00 2024-05-20T18:37:39.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.904+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.7 v2:172.21.15.63:6824/4136124451 6 ==== osd_op_reply(64 [pgnls start_epoch 121 out=49b] v119'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98000 con 0x55a092d61b00 2024-05-20T18:37:39.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.904+0000 7f8d355d2640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:65 18.1b 18:d8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d82000 con 0x55a092d60900 2024-05-20T18:37:39.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.904+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.1 v2:172.21.15.5:6815/550664725 13 ==== osd_op_reply(65 [pgnls start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98000 con 0x55a092d60900 2024-05-20T18:37:39.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.904+0000 7f8d34dd1640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:66 18.3b 18:dc000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a0938b3c00 con 0x55a092d61f80 2024-05-20T18:37:39.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.905+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.4 v2:172.21.15.63:6800/3673324406 9 ==== osd_op_reply(66 [pgnls start_epoch 121 out=49b] v119'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98500 con 0x55a092d61f80 2024-05-20T18:37:39.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.905+0000 7f8d355d2640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:67 18.7 18:e0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d83000 con 0x55a092d61680 2024-05-20T18:37:39.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.905+0000 7f8d39ddb640 1 -- 172.21.15.5:0/3072204518 <== osd.2 v2:172.21.15.5:6800/921679764 7 ==== osd_op_reply(67 [pgnls start_epoch 121 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98780 con 0x55a092d61680 2024-05-20T18:37:39.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.905+0000 7f8d34dd1640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:68 18.27 18:e4000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a0938b2800 con 0x55a092d60000 2024-05-20T18:37:39.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.905+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.5 v2:172.21.15.63:6808/1616066286 8 ==== osd_op_reply(68 [pgnls start_epoch 121 out=49b] v119'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98500 con 0x55a092d60000 2024-05-20T18:37:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.905+0000 7f8d355d2640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:69 18.17 18:e8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d82400 con 0x55a092d60000 2024-05-20T18:37:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.906+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.5 v2:172.21.15.63:6808/1616066286 9 ==== osd_op_reply(69 [pgnls start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98500 con 0x55a092d60000 2024-05-20T18:37:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.906+0000 7f8d34dd1640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:70 18.37 18:ec000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a0938b3800 con 0x55a092d61b00 2024-05-20T18:37:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.906+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.7 v2:172.21.15.63:6824/4136124451 7 ==== osd_op_reply(70 [pgnls start_epoch 121 out=49b] v119'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98000 con 0x55a092d61b00 2024-05-20T18:37:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.906+0000 7f8d355d2640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:71 18.f 18:f0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d82c00 con 0x55a092d60000 2024-05-20T18:37:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.907+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.5 v2:172.21.15.63:6808/1616066286 10 ==== osd_op_reply(71 [pgnls start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98500 con 0x55a092d60000 2024-05-20T18:37:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.907+0000 7f8d34dd1640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:72 18.2f 18:f4000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d82000 con 0x55a092d60900 2024-05-20T18:37:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.907+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.1 v2:172.21.15.5:6815/550664725 14 ==== osd_op_reply(72 [pgnls start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98000 con 0x55a092d60900 2024-05-20T18:37:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.907+0000 7f8d355d2640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:73 18.1f 18:f8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d96800 con 0x55a092d61b00 2024-05-20T18:37:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.907+0000 7f8d395da640 1 -- 172.21.15.5:0/3072204518 <== osd.7 v2:172.21.15.63:6824/4136124451 8 ==== osd_op_reply(73 [pgnls start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98000 con 0x55a092d61b00 2024-05-20T18:37:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.907+0000 7f8d34dd1640 1 -- 172.21.15.5:0/3072204518 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:74 18.3f 18:fc000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55a092d83400 con 0x55a092d61f80 2024-05-20T18:37:39.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.908+0000 7f8d3a5dc640 1 -- 172.21.15.5:0/3072204518 <== osd.4 v2:172.21.15.63:6800/3673324406 10 ==== osd_op_reply(74 [pgnls start_epoch 121 out=49b] v119'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a092a98500 con 0x55a092d61f80 2024-05-20T18:37:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.908+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x55a092b26d00 msgr2=0x55a092afa100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.908+0000 7f8d3c01f740 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x55a092b26d00 0x55a092afa100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.908+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55a092d60900 msgr2=0x55a092d68b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.908+0000 7f8d3c01f740 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55a092d60900 0x55a092d68b00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.908+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x55a092d61680 msgr2=0x55a092d69b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.908+0000 7f8d3c01f740 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x55a092d61680 0x55a092d69b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.908+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x55a092d61200 msgr2=0x55a092d69600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.908+0000 7f8d3c01f740 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x55a092d61200 0x55a092d69600 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.908+0000 7f8d39ddb640 1 -- 172.21.15.5:0/3072204518 reap_dead start 2024-05-20T18:37:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.908+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x55a092d61f80 msgr2=0x55a092d6a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.908+0000 7f8d3c01f740 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x55a092d61f80 0x55a092d6a680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.908+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x55a092d60000 msgr2=0x55a092d68000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.908+0000 7f8d3c01f740 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x55a092d60000 0x55a092d68000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.908+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x55a092d60480 msgr2=0x55a092d68580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.908+0000 7f8d3c01f740 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x55a092d60480 0x55a092d68580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.908+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x55a092d61b00 msgr2=0x55a092d6a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.908+0000 7f8d3c01f740 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x55a092d61b00 0x55a092d6a100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.908+0000 7f8d39ddb640 1 -- 172.21.15.5:0/3072204518 reap_dead start 2024-05-20T18:37:39.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.908+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x55a092b27a80 msgr2=0x55a092afb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.908+0000 7f8d3c01f740 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x55a092b27a80 0x55a092afb700 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x55a0929ab650 tx=0x55a092a60b40 comp rx=0 tx=0).stop 2024-05-20T18:37:39.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.908+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a092b26880 msgr2=0x55a092afa680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.908+0000 7f8d3c01f740 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a092b26880 0x55a092afa680 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x55a0938cc000 tx=0x55a092a641e0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.908+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 shutdown_connections 2024-05-20T18:37:39.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.908+0000 7f8d3c01f740 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x55a092b27a80 0x55a092afb700 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.908+0000 7f8d3c01f740 1 --2- 172.21.15.5:0/3072204518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a092b26880 0x55a092afa680 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.908+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 >> 172.21.15.5:0/3072204518 conn(0x55a092b25200 msgr2=0x55a092a90000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.908+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 shutdown_connections 2024-05-20T18:37:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.908+0000 7f8d3c01f740 1 -- 172.21.15.5:0/3072204518 wait complete. 2024-05-20T18:37:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.909+0000 7f8d3c01f740 1 -- 172.21.15.5:0/337140378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a092b25f80 msgr2=0x55a092af9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.909+0000 7f8d3c01f740 1 --2- 172.21.15.5:0/337140378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a092b25f80 0x55a092af9080 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x55a093787d40 tx=0x55a092a64060 comp rx=0 tx=0).stop 2024-05-20T18:37:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.909+0000 7f8d3c01f740 1 -- 172.21.15.5:0/337140378 shutdown_connections 2024-05-20T18:37:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.909+0000 7f8d3c01f740 1 --2- 172.21.15.5:0/337140378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a092b25f80 0x55a092af9080 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.909+0000 7f8d3c01f740 1 --2- 172.21.15.5:0/337140378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a092b25b00 0x55a092af8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.909+0000 7f8d3c01f740 1 --2- 172.21.15.5:0/337140378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a092b25680 0x55a092af8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.909+0000 7f8d3c01f740 1 -- 172.21.15.5:0/337140378 >> 172.21.15.5:0/337140378 conn(0x55a092b26400 msgr2=0x55a092a91000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.909+0000 7f8d3c01f740 1 -- 172.21.15.5:0/337140378 shutdown_connections 2024-05-20T18:37:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.909+0000 7f8d3c01f740 1 -- 172.21.15.5:0/337140378 wait complete. 2024-05-20T18:37:39.915 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.106699, result: 2024-05-20T18:37:39.915 DEBUG:teuthology.orchestra.run.smithi005:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2024-05-20T18:37:39.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.969+0000 7f002cc23740 1 -- start start 2024-05-20T18:37:39.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.969+0000 7f002cc23740 1 Processor -- start 2024-05-20T18:37:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.970+0000 7f002cc23740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d542a70480 0x55d541e1fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.970+0000 7f002cc23740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d541e4b200 0x55d541e1eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.970+0000 7f002cc23740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d541e4ad80 0x55d541e1f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.970+0000 7f002cc23740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d542a9c000 con 0x55d541e4b200 2024-05-20T18:37:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.970+0000 7f002cc23740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d542a9c180 con 0x55d542a70480 2024-05-20T18:37:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.970+0000 7f002cc23740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55d542a9c300 con 0x55d541e4ad80 2024-05-20T18:37:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.970+0000 7f002a1da640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d541e4b200 0x55d541e1eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.970+0000 7f002a1da640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d541e4b200 0x55d541e1eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35638/0 (socket says 172.21.15.5:35638) 2024-05-20T18:37:39.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.970+0000 7f002a1da640 1 -- 172.21.15.5:0/1839538641 learned_addr learned my addr 172.21.15.5:0/1839538641 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:39.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.970+0000 7f002a9db640 1 --2- 172.21.15.5:0/1839538641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d542a70480 0x55d541e1fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.970+0000 7f002b1dc640 1 --2- 172.21.15.5:0/1839538641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d541e4ad80 0x55d541e1f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.971+0000 7f002a1da640 1 -- 172.21.15.5:0/1839538641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d541e4ad80 msgr2=0x55d541e1f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:39.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.971+0000 7f002a1da640 1 --2- 172.21.15.5:0/1839538641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d541e4ad80 0x55d541e1f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.971+0000 7f002a1da640 1 -- 172.21.15.5:0/1839538641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d542a70480 msgr2=0x55d541e1fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:39.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.971+0000 7f002a1da640 1 --2- 172.21.15.5:0/1839538641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d542a70480 0x55d541e1fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.971+0000 7f002a1da640 1 -- 172.21.15.5:0/1839538641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d53dc8d680 con 0x55d541e4b200 2024-05-20T18:37:39.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.971+0000 7f002a1da640 1 --2- 172.21.15.5:0/1839538641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d541e4b200 0x55d541e1eb00 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x55d542abb950 tx=0x55d541d88000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae31091371dfe43 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.971+0000 7f00281d6640 1 -- 172.21.15.5:0/1839538641 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d53dc86d00 con 0x55d541e4b200 2024-05-20T18:37:39.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.971+0000 7f00281d6640 1 -- 172.21.15.5:0/1839538641 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55d541cda540 con 0x55d541e4b200 2024-05-20T18:37:39.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.971+0000 7f00281d6640 1 -- 172.21.15.5:0/1839538641 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d53dc86680 con 0x55d541e4b200 2024-05-20T18:37:39.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.971+0000 7f002cc23740 1 -- 172.21.15.5:0/1839538641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d53dc8d2c0 con 0x55d541e4b200 2024-05-20T18:37:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.971+0000 7f002cc23740 1 -- 172.21.15.5:0/1839538641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55d53dc8cb40 con 0x55d541e4b200 2024-05-20T18:37:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.971+0000 7f002cc23740 4 waiting for MDS map... 2024-05-20T18:37:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.971+0000 7f00281d6640 1 -- 172.21.15.5:0/1839538641 <== mon.0 v2:172.21.15.5:3300/0 4 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x55d542b6a400 con 0x55d541e4b200 2024-05-20T18:37:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.971+0000 7f00281d6640 1 -- 172.21.15.5:0/1839538641 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 105) ==== 2885+0+0 (secure 0 0 0) 0x55d541ce9180 con 0x55d541e4b200 2024-05-20T18:37:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.972+0000 7f002cc23740 4 Got MDS map 105 2024-05-20T18:37:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.972+0000 7f002cc23740 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2024-05-20T18:37:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.972+0000 7f002cc23740 4 datascan.main: Using metadata pool output 2024-05-20T18:37:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.972+0000 7f002cc23740 4 datascan.main: connecting to RADOS... 2024-05-20T18:37:39.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.972+0000 7f002cc23740 1 Processor -- start 2024-05-20T18:37:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.973+0000 7f002cc23740 1 -- start start 2024-05-20T18:37:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.974+0000 7f002cc23740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d541e4c880 0x55d541e1f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.974+0000 7f002cc23740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d541e4cd00 0x55d541e20100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.974+0000 7f002cc23740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d541e4d180 0x55d541e20680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.974+0000 7f002cc23740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d542a9c480 con 0x55d541e4c880 2024-05-20T18:37:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.974+0000 7f002cc23740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d542a9c600 con 0x55d541e4d180 2024-05-20T18:37:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.974+0000 7f002cc23740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55d542a9c780 con 0x55d541e4cd00 2024-05-20T18:37:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.974+0000 7f002b1dc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d541e4c880 0x55d541e1f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.974+0000 7f002b1dc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d541e4c880 0x55d541e1f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35646/0 (socket says 172.21.15.5:35646) 2024-05-20T18:37:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.974+0000 7f002b1dc640 1 -- 172.21.15.5:0/2076542256 learned_addr learned my addr 172.21.15.5:0/2076542256 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.974+0000 7f002a9db640 1 --2- 172.21.15.5:0/2076542256 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d541e4cd00 0x55d541e20100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.974+0000 7f002a1da640 1 --2- 172.21.15.5:0/2076542256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d541e4d180 0x55d541e20680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.974+0000 7f002b1dc640 1 -- 172.21.15.5:0/2076542256 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d541e4cd00 msgr2=0x55d541e20100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.974+0000 7f002b1dc640 1 --2- 172.21.15.5:0/2076542256 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d541e4cd00 0x55d541e20100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.974+0000 7f002b1dc640 1 -- 172.21.15.5:0/2076542256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d541e4d180 msgr2=0x55d541e20680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.974+0000 7f002b1dc640 1 --2- 172.21.15.5:0/2076542256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d541e4d180 0x55d541e20680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.974+0000 7f002b1dc640 1 -- 172.21.15.5:0/2076542256 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d53dc8c3c0 con 0x55d541e4c880 2024-05-20T18:37:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.974+0000 7f002b1dc640 1 --2- 172.21.15.5:0/2076542256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d541e4c880 0x55d541e1f080 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x55d542b84de0 tx=0x55d541d87d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f73f9af304cc636 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.974+0000 7f00261d2640 1 -- 172.21.15.5:0/2076542256 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d53dc86b60 con 0x55d541e4c880 2024-05-20T18:37:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.974+0000 7f00261d2640 1 -- 172.21.15.5:0/2076542256 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55d541cdaa80 con 0x55d541e4c880 2024-05-20T18:37:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.974+0000 7f00261d2640 1 -- 172.21.15.5:0/2076542256 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d53dc86d00 con 0x55d541e4c880 2024-05-20T18:37:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.975+0000 7f002cc23740 1 -- 172.21.15.5:0/2076542256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d541e4c880 msgr2=0x55d541e1f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.975+0000 7f002cc23740 1 --2- 172.21.15.5:0/2076542256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d541e4c880 0x55d541e1f080 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x55d542b84de0 tx=0x55d541d87d40 comp rx=0 tx=0).stop 2024-05-20T18:37:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.975+0000 7f002cc23740 1 -- 172.21.15.5:0/2076542256 shutdown_connections 2024-05-20T18:37:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.975+0000 7f002cc23740 1 --2- 172.21.15.5:0/2076542256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d541e4d180 0x55d541e20680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.975+0000 7f002cc23740 1 --2- 172.21.15.5:0/2076542256 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d541e4cd00 0x55d541e20100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.975+0000 7f002cc23740 1 --2- 172.21.15.5:0/2076542256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d541e4c880 0x55d541e1f080 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.975+0000 7f002cc23740 1 -- 172.21.15.5:0/2076542256 >> 172.21.15.5:0/2076542256 conn(0x55d541e4a900 msgr2=0x55d541db9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:39.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.975+0000 7f002cc23740 1 -- 172.21.15.5:0/2076542256 shutdown_connections 2024-05-20T18:37:39.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.975+0000 7f002cc23740 1 -- 172.21.15.5:0/2076542256 wait complete. 2024-05-20T18:37:39.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.976+0000 7f002cc23740 1 Processor -- start 2024-05-20T18:37:39.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.976+0000 7f002cc23740 1 -- start start 2024-05-20T18:37:39.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.977+0000 7f002cc23740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d541e4c880 0x55d541e21700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.977+0000 7f002cc23740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d541e4cd00 0x55d541e21180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.977+0000 7f002cc23740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d541e4d180 0x55d541e20c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.977+0000 7f002cc23740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d542a9dc80 con 0x55d541e4c880 2024-05-20T18:37:39.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.977+0000 7f002cc23740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d542a9db00 con 0x55d541e4d180 2024-05-20T18:37:39.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.977+0000 7f002cc23740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55d542a9d980 con 0x55d541e4cd00 2024-05-20T18:37:39.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.977+0000 7f002b1dc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d541e4c880 0x55d541e21700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.977+0000 7f002b1dc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d541e4c880 0x55d541e21700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35662/0 (socket says 172.21.15.5:35662) 2024-05-20T18:37:39.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.977+0000 7f002b1dc640 1 -- 172.21.15.5:0/4000438286 learned_addr learned my addr 172.21.15.5:0/4000438286 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:39.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.977+0000 7f002a9db640 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d541e4cd00 0x55d541e21180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.977+0000 7f002a1da640 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d541e4d180 0x55d541e20c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.977+0000 7f002b1dc640 1 -- 172.21.15.5:0/4000438286 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d541e4cd00 msgr2=0x55d541e21180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.977+0000 7f002b1dc640 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d541e4cd00 0x55d541e21180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.977+0000 7f002b1dc640 1 -- 172.21.15.5:0/4000438286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d541e4d180 msgr2=0x55d541e20c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:39.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.977+0000 7f002b1dc640 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d541e4d180 0x55d541e20c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:39.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.977+0000 7f002b1dc640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d53dc8d860 con 0x55d541e4c880 2024-05-20T18:37:39.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.978+0000 7f002b1dc640 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d541e4c880 0x55d541e21700 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x55d542b85110 tx=0x55d541d87c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.978+0000 7f00249cf640 1 -- 172.21.15.5:0/4000438286 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d53dc86680 con 0x55d541e4c880 2024-05-20T18:37:39.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.978+0000 7f00249cf640 1 -- 172.21.15.5:0/4000438286 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55d541cdb340 con 0x55d541e4c880 2024-05-20T18:37:39.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.978+0000 7f002cc23740 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d53dc8d0e0 con 0x55d541e4c880 2024-05-20T18:37:39.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.978+0000 7f00249cf640 1 -- 172.21.15.5:0/4000438286 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d53dc86820 con 0x55d541e4c880 2024-05-20T18:37:39.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.978+0000 7f002cc23740 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d53dc8d680 con 0x55d541e4c880 2024-05-20T18:37:39.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.978+0000 7f002cc23740 4 datascan.init: resolving metadata pool 17 2024-05-20T18:37:39.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.979+0000 7f00249cf640 1 -- 172.21.15.5:0/4000438286 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x55d542bd2c00 con 0x55d541e4c880 2024-05-20T18:37:39.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.979+0000 7f00249cf640 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x55d541e4bf80 0x55d542074000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.979+0000 7f00249cf640 1 -- 172.21.15.5:0/4000438286 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x55d542a64600 con 0x55d541e4c880 2024-05-20T18:37:39.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.980+0000 7f002cc23740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-20T18:37:39.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.980+0000 7f002cc23740 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-05-20T18:37:39.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.980+0000 7f002cc23740 4 datascan.main: opening data pool 'cephfs_data' 2024-05-20T18:37:39.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.980+0000 7f002a9db640 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x55d541e4bf80 0x55d542074000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.980+0000 7f002cc23740 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x55d541e4d600 0x55d541e20680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.980+0000 7f002cc23740 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:1 17.14 17:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55d54207e000 con 0x55d541e4d600 2024-05-20T18:37:39.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.980+0000 7f002a1da640 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x55d541e4d600 0x55d541e20680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.980+0000 7f002a9db640 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x55d541e4bf80 0x55d542074000 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x55d541d8ae10 tx=0x55d542000ae0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.980+0000 7f002a1da640 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x55d541e4d600 0x55d541e20680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.980+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.3 v2:172.21.15.5:6824/1164130599 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv34 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x55d541dbe000 con 0x55d541e4d600 2024-05-20T18:37:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.980+0000 7f002cc23740 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x55d542a70000 0x55d541e20100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.980+0000 7f002cc23740 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:2 17.1f 17:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55d54207e400 con 0x55d542a70000 2024-05-20T18:37:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.981+0000 7f002b1dc640 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x55d542a70000 0x55d541e20100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.981+0000 7f002b1dc640 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x55d542a70000 0x55d541e20100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.981+0000 7f002b1dc640 1 -- 172.21.15.5:0/4000438286 <== osd.6 v2:172.21.15.63:6816/3384565140 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x55d541dbe500 con 0x55d542a70000 2024-05-20T18:37:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.981+0000 7f002cc23740 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x55d542a70900 0x55d541e1f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.981+0000 7f002cc23740 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:3 18.0 18:00000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d54207f400 con 0x55d542a70900 2024-05-20T18:37:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.982+0000 7f002a9db640 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x55d542a70900 0x55d541e1f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.982+0000 7f002a9db640 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x55d542a70900 0x55d541e1f080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.982+0000 7f002a9db640 1 -- 172.21.15.5:0/4000438286 <== osd.0 v2:172.21.15.5:6808/688093005 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe780 con 0x55d542a70900 2024-05-20T18:37:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.982+0000 7f00259d1640 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x55d542a72880 0x55d542074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.982+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:4 18.20 18:04000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d542098000 con 0x55d542a72880 2024-05-20T18:37:39.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.983+0000 7f002a1da640 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x55d542a72880 0x55d542074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.983+0000 7f002a1da640 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x55d542a72880 0x55d542074580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.983+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.5 v2:172.21.15.63:6808/1616066286 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d542a72880 2024-05-20T18:37:39.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.983+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:5 18.10 18:08000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d542098400 con 0x55d542a70000 2024-05-20T18:37:39.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.984+0000 7f002b1dc640 1 -- 172.21.15.5:0/4000438286 <== osd.6 v2:172.21.15.63:6816/3384565140 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe500 con 0x55d542a70000 2024-05-20T18:37:39.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.984+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:6 18.30 18:0c000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d542098800 con 0x55d542a70900 2024-05-20T18:37:39.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.984+0000 7f002a9db640 1 -- 172.21.15.5:0/4000438286 <== osd.0 v2:172.21.15.5:6808/688093005 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe780 con 0x55d542a70900 2024-05-20T18:37:39.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.984+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:7 18.8 18:10000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d542099000 con 0x55d542a70900 2024-05-20T18:37:39.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.984+0000 7f002a9db640 1 -- 172.21.15.5:0/4000438286 <== osd.0 v2:172.21.15.5:6808/688093005 3 ==== osd_op_reply(7 [pgnls-filter start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe780 con 0x55d542a70900 2024-05-20T18:37:39.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.984+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:8 18.28 18:14000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d542098c00 con 0x55d542a72880 2024-05-20T18:37:39.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.984+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.5 v2:172.21.15.63:6808/1616066286 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d542a72880 2024-05-20T18:37:39.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.984+0000 7f00261d2640 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55d542a72d00 0x55d542074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.984+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:9 18.18 18:18000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d542099400 con 0x55d542a72d00 2024-05-20T18:37:39.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.985+0000 7f002b1dc640 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55d542a72d00 0x55d542074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.985+0000 7f002b1dc640 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55d542a72d00 0x55d542074b00 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.985+0000 7f002b1dc640 1 -- 172.21.15.5:0/4000438286 <== osd.1 v2:172.21.15.5:6815/550664725 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe500 con 0x55d542a72d00 2024-05-20T18:37:39.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.985+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:10 18.38 18:1c000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d54207e000 con 0x55d541e4d600 2024-05-20T18:37:39.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.985+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.3 v2:172.21.15.5:6824/1164130599 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d541e4d600 2024-05-20T18:37:39.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.986+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:11 18.4 18:20000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d542098000 con 0x55d542a70900 2024-05-20T18:37:39.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.986+0000 7f002a9db640 1 -- 172.21.15.5:0/4000438286 <== osd.0 v2:172.21.15.5:6808/688093005 4 ==== osd_op_reply(11 [pgnls-filter start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe780 con 0x55d542a70900 2024-05-20T18:37:39.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.986+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:12 18.24 18:24000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d542b67400 con 0x55d542a70900 2024-05-20T18:37:39.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.986+0000 7f002a9db640 1 -- 172.21.15.5:0/4000438286 <== osd.0 v2:172.21.15.5:6808/688093005 5 ==== osd_op_reply(12 [pgnls-filter start_epoch 121 out=49b] v119'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe780 con 0x55d542a70900 2024-05-20T18:37:39.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.986+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:13 18.14 18:28000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d54207f000 con 0x55d542a70000 2024-05-20T18:37:39.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.986+0000 7f002b1dc640 1 -- 172.21.15.5:0/4000438286 <== osd.6 v2:172.21.15.63:6816/3384565140 3 ==== osd_op_reply(13 [pgnls-filter start_epoch 121 out=49b] v119'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe500 con 0x55d542a70000 2024-05-20T18:37:39.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.986+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:14 18.34 18:2c000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d542b67000 con 0x55d542a70900 2024-05-20T18:37:39.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.987+0000 7f002a9db640 1 -- 172.21.15.5:0/4000438286 <== osd.0 v2:172.21.15.5:6808/688093005 6 ==== osd_op_reply(14 [pgnls-filter start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe780 con 0x55d542a70900 2024-05-20T18:37:39.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.987+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:15 18.c 18:30000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d54207f400 con 0x55d542a72880 2024-05-20T18:37:39.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.987+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.5 v2:172.21.15.63:6808/1616066286 3 ==== osd_op_reply(15 [pgnls-filter start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d542a72880 2024-05-20T18:37:39.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.987+0000 7f00259d1640 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x55d542a73180 0x55d542075080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.987+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:16 18.2c 18:34000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d542098c00 con 0x55d542a73180 2024-05-20T18:37:39.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.988+0000 7f002a9db640 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x55d542a73180 0x55d542075080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.988+0000 7f002a9db640 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x55d542a73180 0x55d542075080 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.988+0000 7f002a9db640 1 -- 172.21.15.5:0/4000438286 <== osd.2 v2:172.21.15.5:6800/921679764 1 ==== osd_op_reply(16 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe780 con 0x55d542a73180 2024-05-20T18:37:39.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.988+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:17 18.1c 18:38000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d542099c00 con 0x55d542a72d00 2024-05-20T18:37:39.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.989+0000 7f002b1dc640 1 -- 172.21.15.5:0/4000438286 <== osd.1 v2:172.21.15.5:6815/550664725 2 ==== osd_op_reply(17 [pgnls-filter start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe500 con 0x55d542a72d00 2024-05-20T18:37:39.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.989+0000 7f00259d1640 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x55d542a73600 0x55d542075600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:39.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.989+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:18 18.3c 18:3c000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d542098800 con 0x55d542a73600 2024-05-20T18:37:39.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.989+0000 7f002a1da640 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x55d542a73600 0x55d542075600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:39.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.989+0000 7f002a1da640 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x55d542a73600 0x55d542075600 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:39.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.990+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.7 v2:172.21.15.63:6824/4136124451 1 ==== osd_op_reply(18 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d542a73600 2024-05-20T18:37:39.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.990+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:19 18.2 18:40000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420ae000 con 0x55d542a73600 2024-05-20T18:37:39.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.990+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.7 v2:172.21.15.63:6824/4136124451 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d542a73600 2024-05-20T18:37:39.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.990+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:20 18.22 18:44000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d542099000 con 0x55d541e4d600 2024-05-20T18:37:39.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.990+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.3 v2:172.21.15.5:6824/1164130599 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d541e4d600 2024-05-20T18:37:39.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.990+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:21 18.12 18:48000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420ae400 con 0x55d542a73180 2024-05-20T18:37:39.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.990+0000 7f002a9db640 1 -- 172.21.15.5:0/4000438286 <== osd.2 v2:172.21.15.5:6800/921679764 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 121 out=81b] v121'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55d541dbe780 con 0x55d542a73180 2024-05-20T18:37:39.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.991+0000 7f002cc23740 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:22 18.0 18:00000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d54207f800 con 0x55d542a70900 2024-05-20T18:37:39.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.991+0000 7f002a9db640 1 -- 172.21.15.5:0/4000438286 <== osd.0 v2:172.21.15.5:6808/688093005 7 ==== osd_op_reply(22 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe780 con 0x55d542a70900 2024-05-20T18:37:39.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.991+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:23 18.20 18:04000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420ae800 con 0x55d542a72880 2024-05-20T18:37:39.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.991+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.5 v2:172.21.15.63:6808/1616066286 4 ==== osd_op_reply(23 [pgnls-filter start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d542a72880 2024-05-20T18:37:39.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.991+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:24 18.10 18:08000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d542099800 con 0x55d542a70000 2024-05-20T18:37:39.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.992+0000 7f002b1dc640 1 -- 172.21.15.5:0/4000438286 <== osd.6 v2:172.21.15.63:6816/3384565140 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe500 con 0x55d542a70000 2024-05-20T18:37:39.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.992+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:25 18.30 18:0c000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420aec00 con 0x55d542a70900 2024-05-20T18:37:39.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.992+0000 7f002a9db640 1 -- 172.21.15.5:0/4000438286 <== osd.0 v2:172.21.15.5:6808/688093005 8 ==== osd_op_reply(25 [pgnls-filter start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe780 con 0x55d542a70900 2024-05-20T18:37:39.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.992+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:26 18.8 18:10000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d542b67400 con 0x55d542a70900 2024-05-20T18:37:39.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.992+0000 7f002a9db640 1 -- 172.21.15.5:0/4000438286 <== osd.0 v2:172.21.15.5:6808/688093005 9 ==== osd_op_reply(26 [pgnls-filter start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe780 con 0x55d542a70900 2024-05-20T18:37:39.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.992+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:27 18.28 18:14000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420af000 con 0x55d542a72880 2024-05-20T18:37:40.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.992+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.5 v2:172.21.15.63:6808/1616066286 5 ==== osd_op_reply(27 [pgnls-filter start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d542a72880 2024-05-20T18:37:40.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.992+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:28 18.18 18:18000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d54207e400 con 0x55d542a72d00 2024-05-20T18:37:40.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.993+0000 7f002b1dc640 1 -- 172.21.15.5:0/4000438286 <== osd.1 v2:172.21.15.5:6815/550664725 3 ==== osd_op_reply(28 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe500 con 0x55d542a72d00 2024-05-20T18:37:40.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.993+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:29 18.38 18:1c000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420ae000 con 0x55d541e4d600 2024-05-20T18:37:40.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.993+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.3 v2:172.21.15.5:6824/1164130599 4 ==== osd_op_reply(29 [pgnls-filter start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d541e4d600 2024-05-20T18:37:40.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.993+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:30 18.4 18:20000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d54207f000 con 0x55d542a70900 2024-05-20T18:37:40.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.993+0000 7f002a9db640 1 -- 172.21.15.5:0/4000438286 <== osd.0 v2:172.21.15.5:6808/688093005 10 ==== osd_op_reply(30 [pgnls-filter start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe780 con 0x55d542a70900 2024-05-20T18:37:40.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.993+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:31 18.24 18:24000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420ae800 con 0x55d542a70900 2024-05-20T18:37:40.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.993+0000 7f002a9db640 1 -- 172.21.15.5:0/4000438286 <== osd.0 v2:172.21.15.5:6808/688093005 11 ==== osd_op_reply(31 [pgnls-filter start_epoch 121 out=49b] v119'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe780 con 0x55d542a70900 2024-05-20T18:37:40.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.993+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:32 18.14 18:28000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d54207e000 con 0x55d542a70000 2024-05-20T18:37:40.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.994+0000 7f002b1dc640 1 -- 172.21.15.5:0/4000438286 <== osd.6 v2:172.21.15.63:6816/3384565140 5 ==== osd_op_reply(32 [pgnls-filter start_epoch 121 out=49b] v119'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe500 con 0x55d542a70000 2024-05-20T18:37:40.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.994+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:33 18.34 18:2c000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420ae400 con 0x55d542a70900 2024-05-20T18:37:40.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.994+0000 7f002a9db640 1 -- 172.21.15.5:0/4000438286 <== osd.0 v2:172.21.15.5:6808/688093005 12 ==== osd_op_reply(33 [pgnls-filter start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe780 con 0x55d542a70900 2024-05-20T18:37:40.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.994+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:34 18.c 18:30000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d54207f400 con 0x55d542a72880 2024-05-20T18:37:40.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.994+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.5 v2:172.21.15.63:6808/1616066286 6 ==== osd_op_reply(34 [pgnls-filter start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d542a72880 2024-05-20T18:37:40.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.995+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:35 18.2c 18:34000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420af400 con 0x55d542a73180 2024-05-20T18:37:40.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.995+0000 7f002a9db640 1 -- 172.21.15.5:0/4000438286 <== osd.2 v2:172.21.15.5:6800/921679764 3 ==== osd_op_reply(35 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe780 con 0x55d542a73180 2024-05-20T18:37:40.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.995+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:36 18.1c 18:38000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420aec00 con 0x55d542a72d00 2024-05-20T18:37:40.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.995+0000 7f002b1dc640 1 -- 172.21.15.5:0/4000438286 <== osd.1 v2:172.21.15.5:6815/550664725 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe500 con 0x55d542a72d00 2024-05-20T18:37:40.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.995+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:37 18.3c 18:3c000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420af800 con 0x55d542a73600 2024-05-20T18:37:40.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.995+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.7 v2:172.21.15.63:6824/4136124451 3 ==== osd_op_reply(37 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d542a73600 2024-05-20T18:37:40.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.995+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:38 18.2 18:40000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420ae800 con 0x55d542a73600 2024-05-20T18:37:40.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.996+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.7 v2:172.21.15.63:6824/4136124451 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d542a73600 2024-05-20T18:37:40.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.996+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:39 18.22 18:44000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d542b67000 con 0x55d541e4d600 2024-05-20T18:37:40.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.996+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.3 v2:172.21.15.5:6824/1164130599 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d541e4d600 2024-05-20T18:37:40.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.996+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:40 18.12 18:48000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420ae400 con 0x55d542a73180 2024-05-20T18:37:40.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.996+0000 7f002a9db640 1 -- 172.21.15.5:0/4000438286 <== osd.2 v2:172.21.15.5:6800/921679764 4 ==== osd_op_reply(40 [pgnls-filter start_epoch 121 out=49b] v121'23 uv23 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe780 con 0x55d542a73180 2024-05-20T18:37:40.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.996+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:41 18.32 18:4c000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420afc00 con 0x55d541e4d600 2024-05-20T18:37:40.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.997+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.3 v2:172.21.15.5:6824/1164130599 6 ==== osd_op_reply(41 [pgnls-filter start_epoch 121 out=49b] v119'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d541e4d600 2024-05-20T18:37:40.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.997+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:42 18.a 18:50000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d54207f800 con 0x55d541e4d600 2024-05-20T18:37:40.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.997+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.3 v2:172.21.15.5:6824/1164130599 7 ==== osd_op_reply(42 [pgnls-filter start_epoch 121 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d541e4d600 2024-05-20T18:37:40.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.997+0000 7f00261d2640 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x55d542a73a80 0x55d542075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:40.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.997+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:43 18.2a 18:54000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420af000 con 0x55d542a73a80 2024-05-20T18:37:40.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.997+0000 7f002b1dc640 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x55d542a73a80 0x55d542075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:40.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.998+0000 7f002b1dc640 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x55d542a73a80 0x55d542075b80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:40.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.998+0000 7f002b1dc640 1 -- 172.21.15.5:0/4000438286 <== osd.4 v2:172.21.15.63:6800/3673324406 1 ==== osd_op_reply(43 [pgnls-filter start_epoch 121 out=49b] v119'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe500 con 0x55d542a73a80 2024-05-20T18:37:40.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.998+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:44 18.1a 18:58000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d54207f000 con 0x55d541e4d600 2024-05-20T18:37:40.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.998+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.3 v2:172.21.15.5:6824/1164130599 8 ==== osd_op_reply(44 [pgnls-filter start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d541e4d600 2024-05-20T18:37:40.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.998+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:45 18.3a 18:5c000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420ae000 con 0x55d542a70900 2024-05-20T18:37:40.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.998+0000 7f002a9db640 1 -- 172.21.15.5:0/4000438286 <== osd.0 v2:172.21.15.5:6808/688093005 13 ==== osd_op_reply(45 [pgnls-filter start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe780 con 0x55d542a70900 2024-05-20T18:37:40.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.998+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:46 18.6 18:60000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d542b67400 con 0x55d542a73600 2024-05-20T18:37:40.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.999+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.7 v2:172.21.15.63:6824/4136124451 5 ==== osd_op_reply(46 [pgnls-filter start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d542a73600 2024-05-20T18:37:40.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.999+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:47 18.26 18:64000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420af800 con 0x55d542a73180 2024-05-20T18:37:40.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.999+0000 7f002a9db640 1 -- 172.21.15.5:0/4000438286 <== osd.2 v2:172.21.15.5:6800/921679764 5 ==== osd_op_reply(47 [pgnls-filter start_epoch 121 out=49b] v119'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe780 con 0x55d542a73180 2024-05-20T18:37:40.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.999+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:48 18.16 18:68000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420b6000 con 0x55d541e4d600 2024-05-20T18:37:40.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.999+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.3 v2:172.21.15.5:6824/1164130599 9 ==== osd_op_reply(48 [pgnls-filter start_epoch 121 out=49b] v119'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d541e4d600 2024-05-20T18:37:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:39.999+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:49 18.36 18:6c000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420ae800 con 0x55d542a72d00 2024-05-20T18:37:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.000+0000 7f002b1dc640 1 -- 172.21.15.5:0/4000438286 <== osd.1 v2:172.21.15.5:6815/550664725 5 ==== osd_op_reply(49 [pgnls-filter start_epoch 121 out=81b] v121'34 uv34 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55d541dbe500 con 0x55d542a72d00 2024-05-20T18:37:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.000+0000 7f002cc23740 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2024-05-20T18:37:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.000+0000 7f002cc23740 10 datascan.operator(): handling object 100000001f7.0 2024-05-20T18:37:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.000+0000 7f002cc23740 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:50 18.36 18:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55d5420b7c00 con 0x55d542a72d00 2024-05-20T18:37:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.000+0000 7f002b1dc640 1 -- 172.21.15.5:0/4000438286 <== osd.1 v2:172.21.15.5:6815/550664725 6 ==== osd_op_reply(50 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv34 ondisk = 0) ==== 416+0+194 (crc 0 0 0) 0x55d541dbe500 con 0x55d542a72d00 2024-05-20T18:37:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.000+0000 7f002cc23740 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2024-05-20T18:37:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.000+0000 7f002cc23740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2024-05-20T18:37:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.000+0000 7f002cc23740 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2024-05-20T18:37:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.000+0000 7f002cc23740 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:51 17.7 17:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55d54207e400 con 0x55d542a70900 2024-05-20T18:37:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.000+0000 7f002a9db640 1 -- 172.21.15.5:0/4000438286 <== osd.0 v2:172.21.15.5:6808/688093005 14 ==== osd_op_reply(51 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv7 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x55d541dbe780 con 0x55d542a70900 2024-05-20T18:37:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.000+0000 7f002cc23740 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2024-05-20T18:37:40.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.000+0000 7f002cc23740 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:52 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55d5420b7c00 con 0x55d541e4d600 2024-05-20T18:37:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.001+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.3 v2:172.21.15.5:6824/1164130599 10 ==== osd_op_reply(52 10000000000.00000000 [omap-get-vals-by-keys out=585b] v0'0 uv12 ondisk = 0) ==== 164+0+585 (crc 0 0 0) 0x55d541dbe000 con 0x55d541e4d600 2024-05-20T18:37:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.001+0000 7f002cc23740 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2024-05-20T18:37:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.001+0000 7f002cc23740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-05-20T18:37:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.001+0000 7f002cc23740 20 datascan.get_frag_of: frag is * 2024-05-20T18:37:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.001+0000 7f002cc23740 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:53 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55d5420aec00 con 0x55d542a70900 2024-05-20T18:37:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.001+0000 7f002a9db640 1 -- 172.21.15.5:0/4000438286 <== osd.0 v2:172.21.15.5:6808/688093005 15 ==== osd_op_reply(53 10000000001.00000000 [omap-get-header out=274b] v0'0 uv7 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x55d541dbe780 con 0x55d542a70900 2024-05-20T18:37:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.001+0000 7f002cc23740 10 datascan.find_or_create_dirfrag: read_version = 7 2024-05-20T18:37:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.001+0000 7f002cc23740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2024-05-20T18:37:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.001+0000 7f002cc23740 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:54 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55d5420af000 con 0x55d542a70900 2024-05-20T18:37:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.001+0000 7f002a9db640 1 -- 172.21.15.5:0/4000438286 <== osd.0 v2:172.21.15.5:6808/688093005 16 ==== osd_op_reply(54 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv7 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x55d541dbe780 con 0x55d542a70900 2024-05-20T18:37:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.001+0000 7f002cc23740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-05-20T18:37:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.001+0000 7f002cc23740 20 datascan.read_dentry: bravo_head not found in result 2024-05-20T18:37:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.001+0000 7f002cc23740 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000001/bravo with size=5 bytes 2024-05-20T18:37:40.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.001+0000 7f002cc23740 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:55 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55d5420ae800 con 0x55d542a70900 2024-05-20T18:37:40.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.002+0000 7f002a9db640 1 -- 172.21.15.5:0/4000438286 <== osd.0 v2:172.21.15.5:6808/688093005 17 ==== osd_op_reply(55 10000000001.00000000 [omap-set-vals] v121'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55d541dbe780 con 0x55d542a70900 2024-05-20T18:37:40.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.003+0000 7f002cc23740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-05-20T18:37:40.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.003+0000 7f002cc23740 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:56 18.e 18:70000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420af400 con 0x55d542a70900 2024-05-20T18:37:40.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.003+0000 7f002a9db640 1 -- 172.21.15.5:0/4000438286 <== osd.0 v2:172.21.15.5:6808/688093005 18 ==== osd_op_reply(56 [pgnls-filter start_epoch 121 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe780 con 0x55d542a70900 2024-05-20T18:37:40.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.003+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:57 18.2e 18:74000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420afc00 con 0x55d541e4d600 2024-05-20T18:37:40.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.003+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.3 v2:172.21.15.5:6824/1164130599 11 ==== osd_op_reply(57 [pgnls-filter start_epoch 121 out=49b] v119'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d541e4d600 2024-05-20T18:37:40.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.003+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:58 18.1e 18:78000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420b6400 con 0x55d542a72880 2024-05-20T18:37:40.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.004+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.5 v2:172.21.15.63:6808/1616066286 7 ==== osd_op_reply(58 [pgnls-filter start_epoch 121 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d542a72880 2024-05-20T18:37:40.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.004+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:59 18.3e 18:7c000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d542b67000 con 0x55d541e4d600 2024-05-20T18:37:40.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.004+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.3 v2:172.21.15.5:6824/1164130599 12 ==== osd_op_reply(59 [pgnls-filter start_epoch 121 out=49b] v119'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d541e4d600 2024-05-20T18:37:40.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.004+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:60 18.1 18:80000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420b6800 con 0x55d541e4d600 2024-05-20T18:37:40.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.004+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.3 v2:172.21.15.5:6824/1164130599 13 ==== osd_op_reply(60 [pgnls-filter start_epoch 121 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d541e4d600 2024-05-20T18:37:40.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.004+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:61 18.21 18:84000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d54207f400 con 0x55d542a72880 2024-05-20T18:37:40.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.004+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.5 v2:172.21.15.63:6808/1616066286 8 ==== osd_op_reply(61 [pgnls-filter start_epoch 121 out=49b] v119'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d542a72880 2024-05-20T18:37:40.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.004+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:62 18.11 18:88000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420b6c00 con 0x55d542a72d00 2024-05-20T18:37:40.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.005+0000 7f002b1dc640 1 -- 172.21.15.5:0/4000438286 <== osd.1 v2:172.21.15.5:6815/550664725 7 ==== osd_op_reply(62 [pgnls-filter start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe500 con 0x55d542a72d00 2024-05-20T18:37:40.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.005+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:63 18.31 18:8c000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d54207f800 con 0x55d542a73a80 2024-05-20T18:37:40.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.005+0000 7f002b1dc640 1 -- 172.21.15.5:0/4000438286 <== osd.4 v2:172.21.15.63:6800/3673324406 2 ==== osd_op_reply(63 [pgnls-filter start_epoch 121 out=49b] v119'15 uv8 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe500 con 0x55d542a73a80 2024-05-20T18:37:40.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.005+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:64 18.9 18:90000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420b7000 con 0x55d541e4d600 2024-05-20T18:37:40.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.005+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.3 v2:172.21.15.5:6824/1164130599 14 ==== osd_op_reply(64 [pgnls-filter start_epoch 121 out=49b] v119'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d541e4d600 2024-05-20T18:37:40.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.006+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:65 18.29 18:94000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d54207f000 con 0x55d542a73600 2024-05-20T18:37:40.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.006+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.7 v2:172.21.15.63:6824/4136124451 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d542a73600 2024-05-20T18:37:40.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.006+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:66 18.19 18:98000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420b7400 con 0x55d542a73a80 2024-05-20T18:37:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.006+0000 7f002b1dc640 1 -- 172.21.15.5:0/4000438286 <== osd.4 v2:172.21.15.63:6800/3673324406 3 ==== osd_op_reply(66 [pgnls-filter start_epoch 121 out=49b] v121'24 uv24 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe500 con 0x55d542a73a80 2024-05-20T18:37:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.006+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:67 18.39 18:9c000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d54207e400 con 0x55d542a70900 2024-05-20T18:37:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.007+0000 7f002a9db640 1 -- 172.21.15.5:0/4000438286 <== osd.0 v2:172.21.15.5:6808/688093005 19 ==== osd_op_reply(67 [pgnls-filter start_epoch 121 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe780 con 0x55d542a70900 2024-05-20T18:37:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.007+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:68 18.5 18:a0000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420b7800 con 0x55d542a72d00 2024-05-20T18:37:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.007+0000 7f002b1dc640 1 -- 172.21.15.5:0/4000438286 <== osd.1 v2:172.21.15.5:6815/550664725 8 ==== osd_op_reply(68 [pgnls-filter start_epoch 121 out=49b] v121'15 uv15 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe500 con 0x55d542a72d00 2024-05-20T18:37:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.007+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:69 18.25 18:a4000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d542b67400 con 0x55d542a70000 2024-05-20T18:37:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.007+0000 7f002b1dc640 1 -- 172.21.15.5:0/4000438286 <== osd.6 v2:172.21.15.63:6816/3384565140 6 ==== osd_op_reply(69 [pgnls-filter start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe500 con 0x55d542a70000 2024-05-20T18:37:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.007+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:70 18.15 18:a8000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d54207e000 con 0x55d542a72880 2024-05-20T18:37:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.008+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.5 v2:172.21.15.63:6808/1616066286 9 ==== osd_op_reply(70 [pgnls-filter start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d542a72880 2024-05-20T18:37:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.008+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:71 18.35 18:ac000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d542b67000 con 0x55d542a73a80 2024-05-20T18:37:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.008+0000 7f002b1dc640 1 -- 172.21.15.5:0/4000438286 <== osd.4 v2:172.21.15.63:6800/3673324406 4 ==== osd_op_reply(71 [pgnls-filter start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe500 con 0x55d542a73a80 2024-05-20T18:37:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.008+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:72 18.d 18:b0000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d54207fc00 con 0x55d542a73a80 2024-05-20T18:37:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.008+0000 7f002b1dc640 1 -- 172.21.15.5:0/4000438286 <== osd.4 v2:172.21.15.63:6800/3673324406 5 ==== osd_op_reply(72 [pgnls-filter start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe500 con 0x55d542a73a80 2024-05-20T18:37:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.008+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:73 18.2d 18:b4000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420b6000 con 0x55d542a70000 2024-05-20T18:37:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.009+0000 7f002b1dc640 1 -- 172.21.15.5:0/4000438286 <== osd.6 v2:172.21.15.63:6816/3384565140 7 ==== osd_op_reply(73 [pgnls-filter start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe500 con 0x55d542a70000 2024-05-20T18:37:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.009+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:74 18.1d 18:b8000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420b6c00 con 0x55d542a70900 2024-05-20T18:37:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.009+0000 7f002a9db640 1 -- 172.21.15.5:0/4000438286 <== osd.0 v2:172.21.15.5:6808/688093005 20 ==== osd_op_reply(74 [pgnls-filter start_epoch 121 out=49b] v119'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe780 con 0x55d542a70900 2024-05-20T18:37:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.009+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:75 18.3d 18:bc000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420b7c00 con 0x55d541e4d600 2024-05-20T18:37:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.009+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.3 v2:172.21.15.5:6824/1164130599 15 ==== osd_op_reply(75 [pgnls-filter start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d541e4d600 2024-05-20T18:37:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.009+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:76 18.3 18:c0000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420b7400 con 0x55d542a72d00 2024-05-20T18:37:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.010+0000 7f002b1dc640 1 -- 172.21.15.5:0/4000438286 <== osd.1 v2:172.21.15.5:6815/550664725 9 ==== osd_op_reply(76 [pgnls-filter start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe500 con 0x55d542a72d00 2024-05-20T18:37:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.010+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:77 18.23 18:c4000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420b6400 con 0x55d542a73180 2024-05-20T18:37:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.010+0000 7f002a9db640 1 -- 172.21.15.5:0/4000438286 <== osd.2 v2:172.21.15.5:6800/921679764 6 ==== osd_op_reply(77 [pgnls-filter start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe780 con 0x55d542a73180 2024-05-20T18:37:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.010+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:78 18.13 18:c8000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420b7800 con 0x55d542a73600 2024-05-20T18:37:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.010+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.7 v2:172.21.15.63:6824/4136124451 7 ==== osd_op_reply(78 [pgnls-filter start_epoch 121 out=49b] v119'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d542a73600 2024-05-20T18:37:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.010+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:79 18.33 18:cc000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420afc00 con 0x55d542a72d00 2024-05-20T18:37:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.011+0000 7f002b1dc640 1 -- 172.21.15.5:0/4000438286 <== osd.1 v2:172.21.15.5:6815/550664725 10 ==== osd_op_reply(79 [pgnls-filter start_epoch 121 out=49b] v119'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe500 con 0x55d542a72d00 2024-05-20T18:37:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.011+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:80 18.b 18:d0000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420b6000 con 0x55d542a72880 2024-05-20T18:37:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.011+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.5 v2:172.21.15.63:6808/1616066286 10 ==== osd_op_reply(80 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d542a72880 2024-05-20T18:37:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.011+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:81 18.2b 18:d4000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420c2000 con 0x55d542a73600 2024-05-20T18:37:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.011+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.7 v2:172.21.15.63:6824/4136124451 8 ==== osd_op_reply(81 [pgnls-filter start_epoch 121 out=49b] v119'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d542a73600 2024-05-20T18:37:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.011+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:82 18.1b 18:d8000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d54207f800 con 0x55d542a72d00 2024-05-20T18:37:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.012+0000 7f002b1dc640 1 -- 172.21.15.5:0/4000438286 <== osd.1 v2:172.21.15.5:6815/550664725 11 ==== osd_op_reply(82 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe500 con 0x55d542a72d00 2024-05-20T18:37:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.012+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:83 18.3b 18:dc000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420c2400 con 0x55d542a73a80 2024-05-20T18:37:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.012+0000 7f002b1dc640 1 -- 172.21.15.5:0/4000438286 <== osd.4 v2:172.21.15.63:6800/3673324406 6 ==== osd_op_reply(83 [pgnls-filter start_epoch 121 out=49b] v119'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe500 con 0x55d542a73a80 2024-05-20T18:37:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.012+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:84 18.7 18:e0000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d54207fc00 con 0x55d542a73180 2024-05-20T18:37:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.012+0000 7f002a9db640 1 -- 172.21.15.5:0/4000438286 <== osd.2 v2:172.21.15.5:6800/921679764 7 ==== osd_op_reply(84 [pgnls-filter start_epoch 121 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe780 con 0x55d542a73180 2024-05-20T18:37:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.012+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:85 18.27 18:e4000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420c2800 con 0x55d542a72880 2024-05-20T18:37:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.013+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.5 v2:172.21.15.63:6808/1616066286 11 ==== osd_op_reply(85 [pgnls-filter start_epoch 121 out=49b] v119'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d542a72880 2024-05-20T18:37:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.013+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:86 18.17 18:e8000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d542b67400 con 0x55d542a72880 2024-05-20T18:37:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.013+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.5 v2:172.21.15.63:6808/1616066286 12 ==== osd_op_reply(86 [pgnls-filter start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d542a72880 2024-05-20T18:37:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.013+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:87 18.37 18:ec000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420c2c00 con 0x55d542a73600 2024-05-20T18:37:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.013+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.7 v2:172.21.15.63:6824/4136124451 9 ==== osd_op_reply(87 [pgnls-filter start_epoch 121 out=49b] v119'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d542a73600 2024-05-20T18:37:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.013+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:88 18.f 18:f0000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d542b67000 con 0x55d542a72880 2024-05-20T18:37:40.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.014+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.5 v2:172.21.15.63:6808/1616066286 13 ==== osd_op_reply(88 [pgnls-filter start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d542a72880 2024-05-20T18:37:40.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.014+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:89 18.2f 18:f4000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420c3000 con 0x55d542a72d00 2024-05-20T18:37:40.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.014+0000 7f002b1dc640 1 -- 172.21.15.5:0/4000438286 <== osd.1 v2:172.21.15.5:6815/550664725 12 ==== osd_op_reply(89 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe500 con 0x55d542a72d00 2024-05-20T18:37:40.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.014+0000 7f00259d1640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:90 18.1f 18:f8000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420c2000 con 0x55d542a73600 2024-05-20T18:37:40.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.014+0000 7f002a1da640 1 -- 172.21.15.5:0/4000438286 <== osd.7 v2:172.21.15.63:6824/4136124451 10 ==== osd_op_reply(90 [pgnls-filter start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe000 con 0x55d542a73600 2024-05-20T18:37:40.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.014+0000 7f00261d2640 1 -- 172.21.15.5:0/4000438286 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:91 18.3f 18:fc000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55d5420c3400 con 0x55d542a73a80 2024-05-20T18:37:40.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002b1dc640 1 -- 172.21.15.5:0/4000438286 <== osd.4 v2:172.21.15.63:6800/3673324406 7 ==== osd_op_reply(91 [pgnls-filter start_epoch 121 out=49b] v119'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d541dbe500 con 0x55d542a73a80 2024-05-20T18:37:40.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002cc23740 1 -- 172.21.15.5:0/4000438286 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x55d542a70900 msgr2=0x55d541e1f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:40.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002cc23740 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x55d542a70900 0x55d541e1f080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002cc23740 1 -- 172.21.15.5:0/4000438286 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55d542a72d00 msgr2=0x55d542074b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:40.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002cc23740 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x55d542a72d00 0x55d542074b00 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002cc23740 1 -- 172.21.15.5:0/4000438286 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x55d542a73180 msgr2=0x55d542075080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:40.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002cc23740 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x55d542a73180 0x55d542075080 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002cc23740 1 -- 172.21.15.5:0/4000438286 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x55d541e4d600 msgr2=0x55d541e20680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:40.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002cc23740 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x55d541e4d600 0x55d541e20680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002a9db640 1 -- 172.21.15.5:0/4000438286 reap_dead start 2024-05-20T18:37:40.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002cc23740 1 -- 172.21.15.5:0/4000438286 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x55d542a73a80 msgr2=0x55d542075b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:40.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002cc23740 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x55d542a73a80 0x55d542075b80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002cc23740 1 -- 172.21.15.5:0/4000438286 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x55d542a72880 msgr2=0x55d542074580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:40.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002cc23740 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x55d542a72880 0x55d542074580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002cc23740 1 -- 172.21.15.5:0/4000438286 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x55d542a70000 msgr2=0x55d541e20100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:40.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002cc23740 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x55d542a70000 0x55d541e20100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002cc23740 1 -- 172.21.15.5:0/4000438286 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x55d542a73600 msgr2=0x55d542075600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:40.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002cc23740 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x55d542a73600 0x55d542075600 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002cc23740 1 -- 172.21.15.5:0/4000438286 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x55d541e4bf80 msgr2=0x55d542074000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:40.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002cc23740 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x55d541e4bf80 0x55d542074000 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x55d541d8ae10 tx=0x55d542000ae0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002a9db640 1 -- 172.21.15.5:0/4000438286 reap_dead start 2024-05-20T18:37:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002cc23740 1 -- 172.21.15.5:0/4000438286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d541e4c880 msgr2=0x55d541e21700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002cc23740 1 --2- 172.21.15.5:0/4000438286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d541e4c880 0x55d541e21700 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x55d542b85110 tx=0x55d541d87c20 comp rx=0 tx=0).stop 2024-05-20T18:37:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002a9db640 1 -- 172.21.15.5:0/4000438286 reap_dead start 2024-05-20T18:37:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002cc23740 1 -- 172.21.15.5:0/4000438286 shutdown_connections 2024-05-20T18:37:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002cc23740 1 -- 172.21.15.5:0/4000438286 >> 172.21.15.5:0/4000438286 conn(0x55d541e4a900 msgr2=0x55d541db9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002cc23740 1 -- 172.21.15.5:0/4000438286 shutdown_connections 2024-05-20T18:37:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002cc23740 1 -- 172.21.15.5:0/4000438286 wait complete. 2024-05-20T18:37:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002cc23740 1 -- 172.21.15.5:0/1839538641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d541e4b200 msgr2=0x55d541e1eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002cc23740 1 --2- 172.21.15.5:0/1839538641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d541e4b200 0x55d541e1eb00 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x55d542abb950 tx=0x55d541d88000 comp rx=0 tx=0).stop 2024-05-20T18:37:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.015+0000 7f002cc23740 1 -- 172.21.15.5:0/1839538641 shutdown_connections 2024-05-20T18:37:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.016+0000 7f002cc23740 1 --2- 172.21.15.5:0/1839538641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d541e4ad80 0x55d541e1f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.016+0000 7f002cc23740 1 --2- 172.21.15.5:0/1839538641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d541e4b200 0x55d541e1eb00 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.016+0000 7f002cc23740 1 --2- 172.21.15.5:0/1839538641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d542a70480 0x55d541e1fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.016+0000 7f002cc23740 1 -- 172.21.15.5:0/1839538641 >> 172.21.15.5:0/1839538641 conn(0x55d541e4c400 msgr2=0x55d541db9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:40.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.016+0000 7f002cc23740 1 -- 172.21.15.5:0/1839538641 shutdown_connections 2024-05-20T18:37:40.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.016+0000 7f002cc23740 1 -- 172.21.15.5:0/1839538641 wait complete. 2024-05-20T18:37:40.028 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.112243, result: 2024-05-20T18:37:40.028 DEBUG:teuthology.orchestra.run.smithi005:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-05-20T18:37:40.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.078+0000 7f0951b22740 1 -- start start 2024-05-20T18:37:40.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.078+0000 7f0951b22740 1 Processor -- start 2024-05-20T18:37:40.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.080+0000 7f0951b22740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5585ad849680 0x5585ad81db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:40.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.080+0000 7f0951b22740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585ad849b00 0x5585ad81d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:40.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.080+0000 7f0951b22740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585ad849f80 0x5585ad81c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:40.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.080+0000 7f0951b22740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5585ae472180 con 0x5585ad849b00 2024-05-20T18:37:40.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.080+0000 7f0951b22740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5585ad6ee900 con 0x5585ad849f80 2024-05-20T18:37:40.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.080+0000 7f0951b22740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5585ad6ee480 con 0x5585ad849680 2024-05-20T18:37:40.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.080+0000 7f094f2fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585ad849b00 0x5585ad81d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:40.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.080+0000 7f094f2fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585ad849b00 0x5585ad81d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35666/0 (socket says 172.21.15.5:35666) 2024-05-20T18:37:40.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.080+0000 7f094f2fc640 1 -- 172.21.15.5:0/3069847361 learned_addr learned my addr 172.21.15.5:0/3069847361 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:40.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.080+0000 7f094fafd640 1 --2- 172.21.15.5:0/3069847361 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5585ad849680 0x5585ad81db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:40.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.080+0000 7f09502fe640 1 --2- 172.21.15.5:0/3069847361 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585ad849f80 0x5585ad81c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:40.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.080+0000 7f094f2fc640 1 -- 172.21.15.5:0/3069847361 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5585ad849680 msgr2=0x5585ad81db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:40.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.080+0000 7f094f2fc640 1 --2- 172.21.15.5:0/3069847361 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5585ad849680 0x5585ad81db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.080+0000 7f094f2fc640 1 -- 172.21.15.5:0/3069847361 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585ad849f80 msgr2=0x5585ad81c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:40.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.080+0000 7f094f2fc640 1 --2- 172.21.15.5:0/3069847361 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585ad849f80 0x5585ad81c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.080+0000 7f094f2fc640 1 -- 172.21.15.5:0/3069847361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585a968b680 con 0x5585ad849b00 2024-05-20T18:37:40.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.080+0000 7f094fafd640 1 --2- 172.21.15.5:0/3069847361 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5585ad849680 0x5585ad81db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:37:40.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.080+0000 7f094f2fc640 1 --2- 172.21.15.5:0/3069847361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585ad849b00 0x5585ad81d600 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x5585ae4add10 tx=0x5585ad7860c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6abe981062ea1c02 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:40.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.081+0000 7f094d2f8640 1 -- 172.21.15.5:0/3069847361 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5585a9684680 con 0x5585ad849b00 2024-05-20T18:37:40.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.081+0000 7f094d2f8640 1 -- 172.21.15.5:0/3069847361 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5585ad6d81c0 con 0x5585ad849b00 2024-05-20T18:37:40.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.081+0000 7f094d2f8640 1 -- 172.21.15.5:0/3069847361 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5585a9684b60 con 0x5585ad849b00 2024-05-20T18:37:40.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.081+0000 7f0951b22740 1 -- 172.21.15.5:0/3069847361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585a968b860 con 0x5585ad849b00 2024-05-20T18:37:40.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.081+0000 7f0951b22740 1 -- 172.21.15.5:0/3069847361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5585a968b2c0 con 0x5585ad849b00 2024-05-20T18:37:40.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.081+0000 7f0951b22740 4 waiting for MDS map... 2024-05-20T18:37:40.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.081+0000 7f094d2f8640 1 -- 172.21.15.5:0/3069847361 <== mon.0 v2:172.21.15.5:3300/0 4 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x5585ae560600 con 0x5585ad849b00 2024-05-20T18:37:40.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.081+0000 7f094d2f8640 1 -- 172.21.15.5:0/3069847361 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 105) ==== 2885+0+0 (secure 0 0 0) 0x5585ad6e7180 con 0x5585ad849b00 2024-05-20T18:37:40.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.081+0000 7f0951b22740 4 Got MDS map 105 2024-05-20T18:37:40.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.081+0000 7f0951b22740 4 datascan.main: Using metadata pool output 2024-05-20T18:37:40.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.081+0000 7f0951b22740 4 datascan.main: connecting to RADOS... 2024-05-20T18:37:40.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.082+0000 7f0951b22740 1 Processor -- start 2024-05-20T18:37:40.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.082+0000 7f0951b22740 1 -- start start 2024-05-20T18:37:40.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.083+0000 7f0951b22740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585ad848d80 0x5585ad81cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:40.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.083+0000 7f0951b22740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585ad848900 0x5585ad81d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:40.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.083+0000 7f0951b22740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5585ad84a880 0x5585ad81e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:40.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.083+0000 7f0951b22740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5585ad6ee180 con 0x5585ad848d80 2024-05-20T18:37:40.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.083+0000 7f0951b22740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5585ae472d80 con 0x5585ad848900 2024-05-20T18:37:40.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.083+0000 7f0951b22740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5585ae472f00 con 0x5585ad84a880 2024-05-20T18:37:40.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.083+0000 7f09502fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585ad848d80 0x5585ad81cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:40.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.083+0000 7f09502fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585ad848d80 0x5585ad81cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35676/0 (socket says 172.21.15.5:35676) 2024-05-20T18:37:40.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.083+0000 7f09502fe640 1 -- 172.21.15.5:0/543603066 learned_addr learned my addr 172.21.15.5:0/543603066 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:40.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.084+0000 7f09502fe640 1 -- 172.21.15.5:0/543603066 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5585ad84a880 msgr2=0x5585ad81e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:40.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.084+0000 7f09502fe640 1 --2- 172.21.15.5:0/543603066 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5585ad84a880 0x5585ad81e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.084+0000 7f09502fe640 1 -- 172.21.15.5:0/543603066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585ad848900 msgr2=0x5585ad81d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:40.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.084+0000 7f094fafd640 1 --2- 172.21.15.5:0/543603066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585ad848900 0x5585ad81d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:40.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.084+0000 7f09502fe640 1 --2- 172.21.15.5:0/543603066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585ad848900 0x5585ad81d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.084+0000 7f09502fe640 1 -- 172.21.15.5:0/543603066 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585a968b4a0 con 0x5585ad848d80 2024-05-20T18:37:40.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.084+0000 7f094fafd640 1 --2- 172.21.15.5:0/543603066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585ad848900 0x5585ad81d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:40.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.084+0000 7f09502fe640 1 --2- 172.21.15.5:0/543603066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585ad848d80 0x5585ad81cb00 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x5585ae5510b0 tx=0x5585ae577800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b631327fd714ee5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:40.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.084+0000 7f094b2f4640 1 -- 172.21.15.5:0/543603066 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5585a9684680 con 0x5585ad848d80 2024-05-20T18:37:40.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.084+0000 7f094b2f4640 1 -- 172.21.15.5:0/543603066 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5585ad6d8e00 con 0x5585ad848d80 2024-05-20T18:37:40.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.084+0000 7f094b2f4640 1 -- 172.21.15.5:0/543603066 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5585a9684b60 con 0x5585ad848d80 2024-05-20T18:37:40.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.084+0000 7f0951b22740 1 -- 172.21.15.5:0/543603066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585ad848d80 msgr2=0x5585ad81cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:40.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.084+0000 7f0951b22740 1 --2- 172.21.15.5:0/543603066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585ad848d80 0x5585ad81cb00 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x5585ae5510b0 tx=0x5585ae577800 comp rx=0 tx=0).stop 2024-05-20T18:37:40.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.085+0000 7f0951b22740 1 -- 172.21.15.5:0/543603066 shutdown_connections 2024-05-20T18:37:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.085+0000 7f0951b22740 1 --2- 172.21.15.5:0/543603066 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5585ad84a880 0x5585ad81e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.085+0000 7f0951b22740 1 --2- 172.21.15.5:0/543603066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585ad848900 0x5585ad81d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.085+0000 7f0951b22740 1 --2- 172.21.15.5:0/543603066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585ad848d80 0x5585ad81cb00 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.085+0000 7f0951b22740 1 -- 172.21.15.5:0/543603066 >> 172.21.15.5:0/543603066 conn(0x5585ad849200 msgr2=0x5585ad7b7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.085+0000 7f0951b22740 1 -- 172.21.15.5:0/543603066 shutdown_connections 2024-05-20T18:37:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.085+0000 7f0951b22740 1 -- 172.21.15.5:0/543603066 wait complete. 2024-05-20T18:37:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.085+0000 7f0951b22740 1 Processor -- start 2024-05-20T18:37:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.085+0000 7f0951b22740 1 -- start start 2024-05-20T18:37:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.087+0000 7f0951b22740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585ad848900 0x5585ad81d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.087+0000 7f0951b22740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5585ad848d80 0x5585ad81cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.087+0000 7f0951b22740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585ad84a880 0x5585ad81e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.087+0000 7f0951b22740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5585ae473200 con 0x5585ad848900 2024-05-20T18:37:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.087+0000 7f0951b22740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5585ae473080 con 0x5585ad84a880 2024-05-20T18:37:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.087+0000 7f0951b22740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5585ae473380 con 0x5585ad848d80 2024-05-20T18:37:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.087+0000 7f09502fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585ad848900 0x5585ad81d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.087+0000 7f09502fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585ad848900 0x5585ad81d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35688/0 (socket says 172.21.15.5:35688) 2024-05-20T18:37:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.087+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 learned_addr learned my addr 172.21.15.5:0/264791998 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.087+0000 7f094fafd640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5585ad848d80 0x5585ad81cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.087+0000 7f094f2fc640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585ad84a880 0x5585ad81e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.087+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5585ad848d80 msgr2=0x5585ad81cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.087+0000 7f09502fe640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5585ad848d80 0x5585ad81cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.087+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585ad84a880 msgr2=0x5585ad81e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.087+0000 7f09502fe640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585ad84a880 0x5585ad81e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.087+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585a968b680 con 0x5585ad848900 2024-05-20T18:37:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.087+0000 7f09502fe640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585ad848900 0x5585ad81d080 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x5585ae551d70 tx=0x5585ad784840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.087+0000 7f0949af1640 1 -- 172.21.15.5:0/264791998 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5585a96844e0 con 0x5585ad848900 2024-05-20T18:37:40.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.087+0000 7f0949af1640 1 -- 172.21.15.5:0/264791998 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5585ad6d8540 con 0x5585ad848900 2024-05-20T18:37:40.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.087+0000 7f0949af1640 1 -- 172.21.15.5:0/264791998 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5585a9684680 con 0x5585ad848900 2024-05-20T18:37:40.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.087+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5585a968b860 con 0x5585ad848900 2024-05-20T18:37:40.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.088+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585a968b0e0 con 0x5585ad848900 2024-05-20T18:37:40.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.088+0000 7f0951b22740 4 datascan.init: resolving metadata pool 17 2024-05-20T18:37:40.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.089+0000 7f0949af1640 1 -- 172.21.15.5:0/264791998 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x5585ae5dcc00 con 0x5585ad848900 2024-05-20T18:37:40.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.089+0000 7f0949af1640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x5585ad84ba80 0x5585ada6c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:40.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.089+0000 7f0949af1640 1 -- 172.21.15.5:0/264791998 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x5585ae5d6a00 con 0x5585ad848900 2024-05-20T18:37:40.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.089+0000 7f0951b22740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-20T18:37:40.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.089+0000 7f0951b22740 4 datascan.main: resolving metadata pool 17 2024-05-20T18:37:40.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.089+0000 7f094fafd640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x5585ad84ba80 0x5585ada6c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:40.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.089+0000 7f0951b22740 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x5585ad84ad00 0x5585ad81e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:40.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.089+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ad84ad00 2024-05-20T18:37:40.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.090+0000 7f094f2fc640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x5585ad84ad00 0x5585ad81e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:40.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.090+0000 7f094fafd640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x5585ad84ba80 0x5585ada6c000 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x5585ad788660 tx=0x5585ad784b40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:40.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.090+0000 7f094f2fc640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x5585ad84ad00 0x5585ad81e680 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:40.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.090+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.7 v2:172.21.15.63:6824/4136124451 1 ==== osd_op_reply(1 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ad84ad00 2024-05-20T18:37:40.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.090+0000 7f094f2fc640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x5585ae488000 0x5585ada6c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:40.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.090+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:2 17.10 17:08000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ae488000 2024-05-20T18:37:40.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.090+0000 7f09502fe640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x5585ae488000 0x5585ada6c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:40.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.091+0000 7f09502fe640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x5585ae488000 0x5585ada6c580 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:40.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.091+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.3 v2:172.21.15.5:6824/1164130599 1 ==== osd_op_reply(2 [pgnls start_epoch 121 out=177b] v120'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488000 2024-05-20T18:37:40.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.091+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:3 17.8 17:10000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ad84ad00 2024-05-20T18:37:40.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.091+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.7 v2:172.21.15.63:6824/4136124451 2 ==== osd_op_reply(3 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ad84ad00 2024-05-20T18:37:40.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.091+0000 7f094f2fc640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x5585ae488480 0x5585ada6cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:40.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.092+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:4 17.18 17:18000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ae488480 2024-05-20T18:37:40.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.092+0000 7f094fafd640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x5585ae488480 0x5585ada6cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:40.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.092+0000 7f094fafd640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x5585ae488480 0x5585ada6cb00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:40.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.092+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 <== osd.0 v2:172.21.15.5:6808/688093005 1 ==== osd_op_reply(4 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc780 con 0x5585ae488480 2024-05-20T18:37:40.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.092+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:5 17.4 17:20000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ae488480 2024-05-20T18:37:40.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.092+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 <== osd.0 v2:172.21.15.5:6808/688093005 2 ==== osd_op_reply(5 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc780 con 0x5585ae488480 2024-05-20T18:37:40.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.092+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:6 17.14 17:28000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ae488000 2024-05-20T18:37:40.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.093+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.3 v2:172.21.15.5:6824/1164130599 2 ==== osd_op_reply(6 [pgnls start_epoch 121 out=149b] v121'41 uv41 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488000 2024-05-20T18:37:40.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.093+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:7 17.c 17:30000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ad84ad00 2024-05-20T18:37:40.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.093+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.7 v2:172.21.15.63:6824/4136124451 3 ==== osd_op_reply(7 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ad84ad00 2024-05-20T18:37:40.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.093+0000 7f094f2fc640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x5585ae488900 0x5585ada6d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:40.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.093+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:8 17.1c 17:38000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ae488900 2024-05-20T18:37:40.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.093+0000 7f094f2fc640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x5585ae488900 0x5585ada6d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:40.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.094+0000 7f094f2fc640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x5585ae488900 0x5585ada6d080 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:40.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.094+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.1 v2:172.21.15.5:6815/550664725 1 ==== osd_op_reply(8 [pgnls start_epoch 121 out=73b] v120'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ae488900 2024-05-20T18:37:40.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.094+0000 7f094f2fc640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x5585ae488d80 0x5585ada6d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:40.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.094+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:9 17.2 17:40000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ae488d80 2024-05-20T18:37:40.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.094+0000 7f09502fe640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x5585ae488d80 0x5585ada6d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:40.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.095+0000 7f09502fe640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x5585ae488d80 0x5585ada6d600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:40.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.095+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.4 v2:172.21.15.63:6800/3673324406 1 ==== osd_op_reply(9 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488d80 2024-05-20T18:37:40.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.095+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:10 17.12 17:48000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ae488000 2024-05-20T18:37:40.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.095+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.3 v2:172.21.15.5:6824/1164130599 3 ==== osd_op_reply(10 [pgnls start_epoch 121 out=81b] v120'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488000 2024-05-20T18:37:40.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.095+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:11 17.a 17:50000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ad84ad00 2024-05-20T18:37:40.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.095+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.7 v2:172.21.15.63:6824/4136124451 4 ==== osd_op_reply(11 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ad84ad00 2024-05-20T18:37:40.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.096+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:12 17.1a 17:58000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ae488480 2024-05-20T18:37:40.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.096+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 <== osd.0 v2:172.21.15.5:6808/688093005 3 ==== osd_op_reply(12 [pgnls start_epoch 121 out=49b] v121'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc780 con 0x5585ae488480 2024-05-20T18:37:40.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.096+0000 7f094fafd640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x5585ae489b00 0x5585ada6e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:40.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.096+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:13 17.6 17:60000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ae489b00 2024-05-20T18:37:40.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.096+0000 7f094fafd640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x5585ae489b00 0x5585ada6e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:40.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.096+0000 7f094fafd640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x5585ae489b00 0x5585ada6e680 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:40.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.097+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 <== osd.6 v2:172.21.15.63:6816/3384565140 1 ==== osd_op_reply(13 [pgnls start_epoch 121 out=77b] v120'8 uv8 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x5585ad7bc780 con 0x5585ae489b00 2024-05-20T18:37:40.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.097+0000 7f094fafd640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x5585ae489f80 0x5585ada6ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:40.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.097+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:14 17.16 17:68000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ae489f80 2024-05-20T18:37:40.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.097+0000 7f094f2fc640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x5585ae489f80 0x5585ada6ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:40.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.097+0000 7f094f2fc640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x5585ae489f80 0x5585ada6ec00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:40.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.098+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.5 v2:172.21.15.63:6808/1616066286 1 ==== osd_op_reply(14 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ae489f80 2024-05-20T18:37:40.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.098+0000 7f094f2fc640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x5585ae489200 0x5585ada6db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:40.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.098+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:15 17.e 17:70000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ae489200 2024-05-20T18:37:40.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.098+0000 7f09502fe640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x5585ae489200 0x5585ada6db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:40.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.098+0000 7f09502fe640 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x5585ae489200 0x5585ada6db80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:40.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.098+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.2 v2:172.21.15.5:6800/921679764 1 ==== osd_op_reply(15 [pgnls start_epoch 121 out=73b] v120'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae489200 2024-05-20T18:37:40.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.098+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:16 17.1e 17:78000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ae488900 2024-05-20T18:37:40.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.099+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.1 v2:172.21.15.5:6815/550664725 2 ==== osd_op_reply(16 [pgnls start_epoch 121 out=49b] v121'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ae488900 2024-05-20T18:37:40.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.099+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:17 17.1 17:80000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ae488480 2024-05-20T18:37:40.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.099+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 <== osd.0 v2:172.21.15.5:6808/688093005 4 ==== osd_op_reply(17 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc780 con 0x5585ae488480 2024-05-20T18:37:40.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.099+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:18 17.11 17:88000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ae488000 2024-05-20T18:37:40.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.099+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.3 v2:172.21.15.5:6824/1164130599 4 ==== osd_op_reply(18 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488000 2024-05-20T18:37:40.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.099+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:19 17.9 17:90000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ae488000 2024-05-20T18:37:40.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.099+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.3 v2:172.21.15.5:6824/1164130599 5 ==== osd_op_reply(19 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488000 2024-05-20T18:37:40.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.100+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:20 17.19 17:98000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ae489f80 2024-05-20T18:37:40.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.100+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.5 v2:172.21.15.63:6808/1616066286 2 ==== osd_op_reply(20 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ae489f80 2024-05-20T18:37:40.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.100+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:21 17.5 17:a0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ae489b00 2024-05-20T18:37:40.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.100+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 <== osd.6 v2:172.21.15.63:6816/3384565140 2 ==== osd_op_reply(21 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc780 con 0x5585ae489b00 2024-05-20T18:37:40.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.100+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:22 17.15 17:a8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ae489f80 2024-05-20T18:37:40.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.101+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.5 v2:172.21.15.63:6808/1616066286 3 ==== osd_op_reply(22 [pgnls start_epoch 121 out=97b] v120'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ae489f80 2024-05-20T18:37:40.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.101+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:23 17.d 17:b0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ad84ad00 2024-05-20T18:37:40.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.101+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.7 v2:172.21.15.63:6824/4136124451 5 ==== osd_op_reply(23 [pgnls start_epoch 121 out=74b] v118'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ad84ad00 2024-05-20T18:37:40.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.101+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:24 17.1d 17:b8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ad84ad00 2024-05-20T18:37:40.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.101+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.7 v2:172.21.15.63:6824/4136124451 6 ==== osd_op_reply(24 [pgnls start_epoch 121 out=73b] v120'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ad84ad00 2024-05-20T18:37:40.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.101+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:25 17.3 17:c0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ae488d80 2024-05-20T18:37:40.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.102+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.4 v2:172.21.15.63:6800/3673324406 2 ==== osd_op_reply(25 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488d80 2024-05-20T18:37:40.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.102+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:26 17.13 17:c8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ae488000 2024-05-20T18:37:40.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.102+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.3 v2:172.21.15.5:6824/1164130599 6 ==== osd_op_reply(26 [pgnls start_epoch 121 out=98b] v120'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488000 2024-05-20T18:37:40.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.102+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:27 17.b 17:d0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ae488480 2024-05-20T18:37:40.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.102+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 <== osd.0 v2:172.21.15.5:6808/688093005 5 ==== osd_op_reply(27 [pgnls start_epoch 121 out=76b] v121'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x5585ad7bc780 con 0x5585ae488480 2024-05-20T18:37:40.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.102+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:28 17.1b 17:d8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ae488d80 2024-05-20T18:37:40.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.103+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.4 v2:172.21.15.63:6800/3673324406 3 ==== osd_op_reply(28 [pgnls start_epoch 121 out=49b] v121'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488d80 2024-05-20T18:37:40.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.103+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:29 17.7 17:e0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ae488480 2024-05-20T18:37:40.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.103+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 <== osd.0 v2:172.21.15.5:6808/688093005 6 ==== osd_op_reply(29 [pgnls start_epoch 121 out=105b] v121'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x5585ad7bc780 con 0x5585ae488480 2024-05-20T18:37:40.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.103+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:30 17.17 17:e8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ae489b00 2024-05-20T18:37:40.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.103+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 <== osd.6 v2:172.21.15.63:6816/3384565140 3 ==== osd_op_reply(30 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc780 con 0x5585ae489b00 2024-05-20T18:37:40.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.103+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:31 17.f 17:f0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ae488900 2024-05-20T18:37:40.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.103+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.1 v2:172.21.15.5:6815/550664725 3 ==== osd_op_reply(31 [pgnls start_epoch 121 out=73b] v121'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ae488900 2024-05-20T18:37:40.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.103+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:32 17.1f 17:f8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ae489b00 2024-05-20T18:37:40.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.104+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 <== osd.6 v2:172.21.15.63:6816/3384565140 4 ==== osd_op_reply(32 [pgnls start_epoch 121 out=101b] v121'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x5585ad7bc780 con 0x5585ae489b00 2024-05-20T18:37:40.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.104+0000 7f0951b22740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-05-20T18:37:40.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.104+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:33 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4c00 con 0x5585ae488000 2024-05-20T18:37:40.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.104+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.3 v2:172.21.15.5:6824/1164130599 7 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488000 2024-05-20T18:37:40.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.104+0000 7f0951b22740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-05-20T18:37:40.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.104+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:34 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92000 con 0x5585ae488000 2024-05-20T18:37:40.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.105+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.3 v2:172.21.15.5:6824/1164130599 8 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488000 2024-05-20T18:37:40.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.105+0000 7f0951b22740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-05-20T18:37:40.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.105+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:35 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92400 con 0x5585ae488000 2024-05-20T18:37:40.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.105+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.3 v2:172.21.15.5:6824/1164130599 9 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1169b] v0'0 uv12 ondisk = 0) ==== 164+0+1169 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488000 2024-05-20T18:37:40.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.105+0000 7f0951b22740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-05-20T18:37:40.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.105+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:36 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92800 con 0x5585ae488000 2024-05-20T18:37:40.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.105+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.3 v2:172.21.15.5:6824/1164130599 10 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488000 2024-05-20T18:37:40.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.105+0000 7f0951b22740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-05-20T18:37:40.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.105+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:37 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4c00 con 0x5585ae488000 2024-05-20T18:37:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.105+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.3 v2:172.21.15.5:6824/1164130599 11 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488000 2024-05-20T18:37:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.106+0000 7f0951b22740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-05-20T18:37:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.106+0000 7f0951b22740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-05-20T18:37:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.106+0000 7f0951b22740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-05-20T18:37:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.106+0000 7f0951b22740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-05-20T18:37:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.106+0000 7f0951b22740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-05-20T18:37:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.106+0000 7f0951b22740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-05-20T18:37:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.106+0000 7f0951b22740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-05-20T18:37:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.106+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:38 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92000 con 0x5585ae488000 2024-05-20T18:37:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.106+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.3 v2:172.21.15.5:6824/1164130599 12 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv36 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488000 2024-05-20T18:37:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.106+0000 7f0951b22740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-05-20T18:37:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.106+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:39 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92400 con 0x5585ae488900 2024-05-20T18:37:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.106+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.1 v2:172.21.15.5:6815/550664725 4 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ae488900 2024-05-20T18:37:40.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.106+0000 7f0951b22740 10 datascan.scan_links: step 1: handling object 10000000003.00000000 2024-05-20T18:37:40.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.106+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:40 17.12 17:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92800 con 0x5585ae488000 2024-05-20T18:37:40.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.106+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.3 v2:172.21.15.5:6824/1164130599 13 ==== osd_op_reply(40 10000000003.00000000 [omap-get-vals out=584b] v0'0 uv3 ondisk = 0) ==== 164+0+584 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488000 2024-05-20T18:37:40.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.107+0000 7f0951b22740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-05-20T18:37:40.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.107+0000 7f0951b22740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-05-20T18:37:40.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.107+0000 7f0951b22740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-05-20T18:37:40.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.107+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:41 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ae489200 2024-05-20T18:37:40.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.107+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.2 v2:172.21.15.5:6800/921679764 2 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae489200 2024-05-20T18:37:40.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.107+0000 7f0951b22740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-05-20T18:37:40.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.107+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:42 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ada93000 con 0x5585ae489f80 2024-05-20T18:37:40.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.107+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.5 v2:172.21.15.63:6808/1616066286 4 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ae489f80 2024-05-20T18:37:40.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.107+0000 7f0951b22740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-05-20T18:37:40.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.107+0000 7f0951b22740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-05-20T18:37:40.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.107+0000 7f0951b22740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-05-20T18:37:40.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.107+0000 7f0951b22740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-05-20T18:37:40.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.107+0000 7f0951b22740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-05-20T18:37:40.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.107+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:43 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ada93400 con 0x5585ad84ad00 2024-05-20T18:37:40.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.108+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.7 v2:172.21.15.63:6824/4136124451 7 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ad84ad00 2024-05-20T18:37:40.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.108+0000 7f0951b22740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-05-20T18:37:40.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.108+0000 7f0951b22740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-05-20T18:37:40.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.108+0000 7f0951b22740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-05-20T18:37:40.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.108+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:44 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ada93800 con 0x5585ae488000 2024-05-20T18:37:40.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.108+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.3 v2:172.21.15.5:6824/1164130599 14 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488000 2024-05-20T18:37:40.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.108+0000 7f0951b22740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-05-20T18:37:40.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.108+0000 7f0951b22740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-05-20T18:37:40.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.108+0000 7f0951b22740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-05-20T18:37:40.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.108+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:45 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ada93c00 con 0x5585ae488480 2024-05-20T18:37:40.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.109+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 <== osd.0 v2:172.21.15.5:6808/688093005 7 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv8 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5585ad7bc780 con 0x5585ae488480 2024-05-20T18:37:40.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.109+0000 7f0951b22740 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2024-05-20T18:37:40.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.109+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:46 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92000 con 0x5585ae488480 2024-05-20T18:37:40.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.109+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 <== osd.0 v2:172.21.15.5:6808/688093005 8 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1667b] v0'0 uv9 ondisk = 0) ==== 164+0+1667 (crc 0 0 0) 0x5585ad7bc780 con 0x5585ae488480 2024-05-20T18:37:40.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.109+0000 7f0951b22740 20 datascan.scan_links: injected ino detected 2024-05-20T18:37:40.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.109+0000 7f0951b22740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-05-20T18:37:40.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.109+0000 7f0951b22740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-05-20T18:37:40.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.109+0000 7f0951b22740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-05-20T18:37:40.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.109+0000 7f0951b22740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-05-20T18:37:40.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.109+0000 7f0951b22740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-05-20T18:37:40.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.109+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:47 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92800 con 0x5585ae489b00 2024-05-20T18:37:40.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.109+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 <== osd.6 v2:172.21.15.63:6816/3384565140 5 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=585b] v0'0 uv8 ondisk = 0) ==== 154+0+585 (crc 0 0 0) 0x5585ad7bc780 con 0x5585ae489b00 2024-05-20T18:37:40.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.109+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:48 17.0 17:00000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ad84ad00 2024-05-20T18:37:40.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.110+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.7 v2:172.21.15.63:6824/4136124451 8 ==== osd_op_reply(48 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ad84ad00 2024-05-20T18:37:40.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.110+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:49 17.10 17:08000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ae488000 2024-05-20T18:37:40.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.110+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.3 v2:172.21.15.5:6824/1164130599 15 ==== osd_op_reply(49 [pgnls start_epoch 121 out=177b] v120'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488000 2024-05-20T18:37:40.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.110+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:50 17.8 17:10000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ad84ad00 2024-05-20T18:37:40.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.111+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.7 v2:172.21.15.63:6824/4136124451 9 ==== osd_op_reply(50 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ad84ad00 2024-05-20T18:37:40.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.111+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:51 17.18 17:18000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ae488480 2024-05-20T18:37:40.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.111+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 <== osd.0 v2:172.21.15.5:6808/688093005 9 ==== osd_op_reply(51 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc780 con 0x5585ae488480 2024-05-20T18:37:40.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.111+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:52 17.4 17:20000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ae488480 2024-05-20T18:37:40.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.111+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 <== osd.0 v2:172.21.15.5:6808/688093005 10 ==== osd_op_reply(52 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc780 con 0x5585ae488480 2024-05-20T18:37:40.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.111+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:53 17.14 17:28000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ae488000 2024-05-20T18:37:40.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.111+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.3 v2:172.21.15.5:6824/1164130599 16 ==== osd_op_reply(53 [pgnls start_epoch 121 out=149b] v121'41 uv41 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488000 2024-05-20T18:37:40.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.112+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:54 17.c 17:30000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ad84ad00 2024-05-20T18:37:40.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.112+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.7 v2:172.21.15.63:6824/4136124451 10 ==== osd_op_reply(54 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ad84ad00 2024-05-20T18:37:40.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.112+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:55 17.1c 17:38000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ae488900 2024-05-20T18:37:40.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.112+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.1 v2:172.21.15.5:6815/550664725 5 ==== osd_op_reply(55 [pgnls start_epoch 121 out=73b] v120'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ae488900 2024-05-20T18:37:40.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.112+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:56 17.2 17:40000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ae488d80 2024-05-20T18:37:40.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.112+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.4 v2:172.21.15.63:6800/3673324406 4 ==== osd_op_reply(56 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488d80 2024-05-20T18:37:40.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.112+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:57 17.12 17:48000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ae488000 2024-05-20T18:37:40.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.113+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.3 v2:172.21.15.5:6824/1164130599 17 ==== osd_op_reply(57 [pgnls start_epoch 121 out=81b] v120'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488000 2024-05-20T18:37:40.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.113+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:58 17.a 17:50000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ad84ad00 2024-05-20T18:37:40.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.113+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.7 v2:172.21.15.63:6824/4136124451 11 ==== osd_op_reply(58 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ad84ad00 2024-05-20T18:37:40.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.113+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:59 17.1a 17:58000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ae488480 2024-05-20T18:37:40.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.113+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 <== osd.0 v2:172.21.15.5:6808/688093005 11 ==== osd_op_reply(59 [pgnls start_epoch 121 out=49b] v121'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc780 con 0x5585ae488480 2024-05-20T18:37:40.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.113+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:60 17.6 17:60000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ae489b00 2024-05-20T18:37:40.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.114+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 <== osd.6 v2:172.21.15.63:6816/3384565140 6 ==== osd_op_reply(60 [pgnls start_epoch 121 out=77b] v120'8 uv8 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x5585ad7bc780 con 0x5585ae489b00 2024-05-20T18:37:40.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.114+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:61 17.16 17:68000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ae489f80 2024-05-20T18:37:40.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.114+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.5 v2:172.21.15.63:6808/1616066286 5 ==== osd_op_reply(61 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ae489f80 2024-05-20T18:37:40.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.114+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:62 17.e 17:70000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ae489200 2024-05-20T18:37:40.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.114+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.2 v2:172.21.15.5:6800/921679764 3 ==== osd_op_reply(62 [pgnls start_epoch 121 out=73b] v120'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae489200 2024-05-20T18:37:40.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.114+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:63 17.1e 17:78000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ae488900 2024-05-20T18:37:40.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.115+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.1 v2:172.21.15.5:6815/550664725 6 ==== osd_op_reply(63 [pgnls start_epoch 121 out=49b] v121'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ae488900 2024-05-20T18:37:40.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.115+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:64 17.1 17:80000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ae488480 2024-05-20T18:37:40.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.115+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 <== osd.0 v2:172.21.15.5:6808/688093005 12 ==== osd_op_reply(64 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc780 con 0x5585ae488480 2024-05-20T18:37:40.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.115+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:65 17.11 17:88000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ae488000 2024-05-20T18:37:40.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.115+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.3 v2:172.21.15.5:6824/1164130599 18 ==== osd_op_reply(65 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488000 2024-05-20T18:37:40.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.115+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:66 17.9 17:90000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ae488000 2024-05-20T18:37:40.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.115+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.3 v2:172.21.15.5:6824/1164130599 19 ==== osd_op_reply(66 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488000 2024-05-20T18:37:40.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.115+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:67 17.19 17:98000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ae489f80 2024-05-20T18:37:40.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.116+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.5 v2:172.21.15.63:6808/1616066286 6 ==== osd_op_reply(67 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ae489f80 2024-05-20T18:37:40.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.116+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ae489b00 2024-05-20T18:37:40.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.116+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 <== osd.6 v2:172.21.15.63:6816/3384565140 7 ==== osd_op_reply(68 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc780 con 0x5585ae489b00 2024-05-20T18:37:40.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.116+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:69 17.15 17:a8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ae489f80 2024-05-20T18:37:40.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.116+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.5 v2:172.21.15.63:6808/1616066286 7 ==== osd_op_reply(69 [pgnls start_epoch 121 out=97b] v120'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ae489f80 2024-05-20T18:37:40.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.116+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:70 17.d 17:b0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ad84ad00 2024-05-20T18:37:40.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.117+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.7 v2:172.21.15.63:6824/4136124451 12 ==== osd_op_reply(70 [pgnls start_epoch 121 out=74b] v118'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ad84ad00 2024-05-20T18:37:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.117+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:71 17.1d 17:b8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ad84ad00 2024-05-20T18:37:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.117+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.7 v2:172.21.15.63:6824/4136124451 13 ==== osd_op_reply(71 [pgnls start_epoch 121 out=73b] v120'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ad84ad00 2024-05-20T18:37:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.117+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:72 17.3 17:c0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ae488d80 2024-05-20T18:37:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.117+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.4 v2:172.21.15.63:6800/3673324406 5 ==== osd_op_reply(72 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488d80 2024-05-20T18:37:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.117+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:73 17.13 17:c8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ae488000 2024-05-20T18:37:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.118+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.3 v2:172.21.15.5:6824/1164130599 20 ==== osd_op_reply(73 [pgnls start_epoch 121 out=98b] v120'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488000 2024-05-20T18:37:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.118+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:74 17.b 17:d0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ae488480 2024-05-20T18:37:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.118+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 <== osd.0 v2:172.21.15.5:6808/688093005 13 ==== osd_op_reply(74 [pgnls start_epoch 121 out=76b] v121'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x5585ad7bc780 con 0x5585ae488480 2024-05-20T18:37:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.118+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- osd_op(unknown.0.0:75 17.1b 17:d8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ae488d80 2024-05-20T18:37:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.118+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.4 v2:172.21.15.63:6800/3673324406 6 ==== osd_op_reply(75 [pgnls start_epoch 121 out=49b] v121'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488d80 2024-05-20T18:37:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.118+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:76 17.7 17:e0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ae488480 2024-05-20T18:37:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.119+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 <== osd.0 v2:172.21.15.5:6808/688093005 14 ==== osd_op_reply(76 [pgnls start_epoch 121 out=105b] v121'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x5585ad7bc780 con 0x5585ae488480 2024-05-20T18:37:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.119+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:77 17.17 17:e8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ae489b00 2024-05-20T18:37:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.119+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 <== osd.6 v2:172.21.15.63:6816/3384565140 8 ==== osd_op_reply(77 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5585ad7bc780 con 0x5585ae489b00 2024-05-20T18:37:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.119+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:78 17.f 17:f0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ae488900 2024-05-20T18:37:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.119+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.1 v2:172.21.15.5:6815/550664725 7 ==== osd_op_reply(78 [pgnls start_epoch 121 out=73b] v121'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ae488900 2024-05-20T18:37:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.119+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:79 17.1f 17:f8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92c00 con 0x5585ae489b00 2024-05-20T18:37:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.119+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 <== osd.6 v2:172.21.15.63:6816/3384565140 9 ==== osd_op_reply(79 [pgnls start_epoch 121 out=101b] v121'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x5585ad7bc780 con 0x5585ae489b00 2024-05-20T18:37:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.119+0000 7f0951b22740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-05-20T18:37:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.119+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:80 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ada93800 con 0x5585ae488000 2024-05-20T18:37:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.120+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.3 v2:172.21.15.5:6824/1164130599 21 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488000 2024-05-20T18:37:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.120+0000 7f0951b22740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-05-20T18:37:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.120+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:81 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ae55f400 con 0x5585ae488000 2024-05-20T18:37:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.120+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.3 v2:172.21.15.5:6824/1164130599 22 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488000 2024-05-20T18:37:40.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.120+0000 7f0951b22740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-05-20T18:37:40.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.120+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:82 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ae55f000 con 0x5585ae488000 2024-05-20T18:37:40.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.120+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.3 v2:172.21.15.5:6824/1164130599 23 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1169b] v0'0 uv12 ondisk = 0) ==== 164+0+1169 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488000 2024-05-20T18:37:40.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.120+0000 7f0951b22740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-05-20T18:37:40.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.120+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:83 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4c00 con 0x5585ae488000 2024-05-20T18:37:40.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.121+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.3 v2:172.21.15.5:6824/1164130599 24 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488000 2024-05-20T18:37:40.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.121+0000 7f0951b22740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-05-20T18:37:40.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.121+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:84 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ada93c00 con 0x5585ae488000 2024-05-20T18:37:40.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.121+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.3 v2:172.21.15.5:6824/1164130599 25 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488000 2024-05-20T18:37:40.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.121+0000 7f0951b22740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-05-20T18:37:40.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.121+0000 7f0951b22740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-05-20T18:37:40.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.121+0000 7f0951b22740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-05-20T18:37:40.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.121+0000 7f0951b22740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-05-20T18:37:40.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.121+0000 7f0951b22740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-05-20T18:37:40.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.121+0000 7f0951b22740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-05-20T18:37:40.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.121+0000 7f0951b22740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-05-20T18:37:40.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.121+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:85 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92000 con 0x5585ae488000 2024-05-20T18:37:40.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.121+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.3 v2:172.21.15.5:6824/1164130599 26 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv36 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488000 2024-05-20T18:37:40.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.121+0000 7f0951b22740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-05-20T18:37:40.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.121+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- osd_op(unknown.0.0:86 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92400 con 0x5585ae488900 2024-05-20T18:37:40.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.122+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.1 v2:172.21.15.5:6815/550664725 8 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ae488900 2024-05-20T18:37:40.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.122+0000 7f0951b22740 10 datascan.scan_links: step 2: handling object 10000000003.00000000 2024-05-20T18:37:40.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.122+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:87 17.12 17:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ada93000 con 0x5585ae488000 2024-05-20T18:37:40.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.122+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.3 v2:172.21.15.5:6824/1164130599 27 ==== osd_op_reply(87 10000000003.00000000 [omap-get-vals out=584b] v0'0 uv3 ondisk = 0) ==== 164+0+584 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488000 2024-05-20T18:37:40.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.122+0000 7f0951b22740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-05-20T18:37:40.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.122+0000 7f0951b22740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-05-20T18:37:40.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.122+0000 7f0951b22740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-05-20T18:37:40.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.122+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- osd_op(unknown.0.0:88 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d5c00 con 0x5585ae489200 2024-05-20T18:37:40.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.122+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.2 v2:172.21.15.5:6800/921679764 4 ==== osd_op_reply(88 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae489200 2024-05-20T18:37:40.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.122+0000 7f0951b22740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-05-20T18:37:40.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.122+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- osd_op(unknown.0.0:89 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d5800 con 0x5585ae489f80 2024-05-20T18:37:40.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.123+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.5 v2:172.21.15.63:6808/1616066286 8 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ae489f80 2024-05-20T18:37:40.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.123+0000 7f0951b22740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-05-20T18:37:40.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.123+0000 7f0951b22740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-05-20T18:37:40.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.123+0000 7f0951b22740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-05-20T18:37:40.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.123+0000 7f0951b22740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-05-20T18:37:40.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.123+0000 7f0951b22740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-05-20T18:37:40.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.123+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:90 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d5400 con 0x5585ad84ad00 2024-05-20T18:37:40.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.123+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.7 v2:172.21.15.63:6824/4136124451 14 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ad84ad00 2024-05-20T18:37:40.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.123+0000 7f0951b22740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-05-20T18:37:40.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.123+0000 7f0951b22740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-05-20T18:37:40.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.123+0000 7f0951b22740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-05-20T18:37:40.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.123+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:91 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4800 con 0x5585ae488000 2024-05-20T18:37:40.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.124+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.3 v2:172.21.15.5:6824/1164130599 28 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488000 2024-05-20T18:37:40.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.124+0000 7f0951b22740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-05-20T18:37:40.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.124+0000 7f0951b22740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-05-20T18:37:40.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.124+0000 7f0951b22740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-05-20T18:37:40.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.124+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:92 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ada93800 con 0x5585ae488480 2024-05-20T18:37:40.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.124+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 <== osd.0 v2:172.21.15.5:6808/688093005 15 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv8 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5585ad7bc780 con 0x5585ae488480 2024-05-20T18:37:40.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.124+0000 7f0951b22740 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2024-05-20T18:37:40.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.124+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:93 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ada93c00 con 0x5585ae488480 2024-05-20T18:37:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.125+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 <== osd.0 v2:172.21.15.5:6808/688093005 16 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1667b] v0'0 uv9 ondisk = 0) ==== 164+0+1667 (crc 0 0 0) 0x5585ad7bc780 con 0x5585ae488480 2024-05-20T18:37:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.125+0000 7f0951b22740 20 datascan.scan_links: injected ino detected 2024-05-20T18:37:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.125+0000 7f0951b22740 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2024-05-20T18:37:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.125+0000 7f0951b22740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-05-20T18:37:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.125+0000 7f0951b22740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-05-20T18:37:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.125+0000 7f0951b22740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-05-20T18:37:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.125+0000 7f0951b22740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-05-20T18:37:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.125+0000 7f0951b22740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-05-20T18:37:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.125+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- osd_op(unknown.0.0:94 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ada92000 con 0x5585ae489b00 2024-05-20T18:37:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.125+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 <== osd.6 v2:172.21.15.63:6816/3384565140 10 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=585b] v0'0 uv8 ondisk = 0) ==== 154+0+585 (crc 0 0 0) 0x5585ad7bc780 con 0x5585ae489b00 2024-05-20T18:37:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.125+0000 7f0951b22740 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-05-20T18:37:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.125+0000 7f0951b22740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-05-20T18:37:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.125+0000 7f0951b22740 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-05-20T18:37:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.125+0000 7f0951b22740 10 datascan.scan_links: processing 1 injected_inos 2024-05-20T18:37:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.125+0000 7f0951b22740 10 datascan.scan_links: handling injected_ino 0x100000001f7 2024-05-20T18:37:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.125+0000 7f0951b22740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2024-05-20T18:37:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.125+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:95 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ada93400 con 0x5585ae488480 2024-05-20T18:37:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.125+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 <== osd.0 v2:172.21.15.5:6808/688093005 17 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x5585ad7bc780 con 0x5585ae488480 2024-05-20T18:37:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.125+0000 7f0951b22740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-05-20T18:37:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.125+0000 7f0951b22740 20 datascan.scan_links: first is now 2 2024-05-20T18:37:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.125+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- osd_op(unknown.0.0:96 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4c00 con 0x5585ae488480 2024-05-20T18:37:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.127+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 <== osd.0 v2:172.21.15.5:6808/688093005 18 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v121'10 uv10 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5585ad7bc780 con 0x5585ae488480 2024-05-20T18:37:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.127+0000 7f0951b22740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-05-20T18:37:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.127+0000 7f0951b22740 10 datascan.scan_links: updating inotable 2024-05-20T18:37:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.127+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:97 17.13 17:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ae55f400 con 0x5585ae488000 2024-05-20T18:37:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.127+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.3 v2:172.21.15.5:6824/1164130599 29 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv4 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488000 2024-05-20T18:37:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.127+0000 7f0951b22740 10 mds.0.inotable: skip_inos was [0x10000000005~0x1f2,0x100000001f9~0xfffffffe07] 2024-05-20T18:37:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.127+0000 7f0951b22740 10 mds.0.inotable: skip_inos now [0x100000001f9~0xfffffffe07] 2024-05-20T18:37:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.127+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- osd_op(unknown.0.0:98 17.13 17:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x5585ae55f000 con 0x5585ae488000 2024-05-20T18:37:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.128+0000 7f09502fe640 1 -- 172.21.15.5:0/264791998 <== osd.3 v2:172.21.15.5:6824/1164130599 30 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v121'6 uv6 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x5585ad7bc500 con 0x5585ae488000 2024-05-20T18:37:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f0951b22740 10 datascan.scan_links: updating snaptable 2024-05-20T18:37:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- osd_op(unknown.0.0:99 17.d 17:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5585ae5d4c00 con 0x5585ad84ad00 2024-05-20T18:37:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f094f2fc640 1 -- 172.21.15.5:0/264791998 <== osd.7 v2:172.21.15.63:6824/4136124451 15 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x5585ad7bc000 con 0x5585ad84ad00 2024-05-20T18:37:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x5585ae488480 msgr2=0x5585ada6cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f0951b22740 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x5585ae488480 0x5585ada6cb00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x5585ae488900 msgr2=0x5585ada6d080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f0951b22740 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x5585ae488900 0x5585ada6d080 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x5585ae489200 msgr2=0x5585ada6db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f0951b22740 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x5585ae489200 0x5585ada6db80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x5585ae488000 msgr2=0x5585ada6c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f0951b22740 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x5585ae488000 0x5585ada6c580 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 reap_dead start 2024-05-20T18:37:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 reap_dead start 2024-05-20T18:37:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x5585ae488d80 msgr2=0x5585ada6d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f0951b22740 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x5585ae488d80 0x5585ada6d600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x5585ae489f80 msgr2=0x5585ada6ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f0951b22740 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x5585ae489f80 0x5585ada6ec00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x5585ae489b00 msgr2=0x5585ada6e680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f0951b22740 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x5585ae489b00 0x5585ada6e680 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x5585ad84ad00 msgr2=0x5585ad81e680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f0951b22740 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x5585ad84ad00 0x5585ad81e680 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x5585ad84ba80 msgr2=0x5585ada6c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f0951b22740 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x5585ad84ba80 0x5585ada6c000 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x5585ad788660 tx=0x5585ad784b40 comp rx=0 tx=0).stop 2024-05-20T18:37:40.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585ad848900 msgr2=0x5585ad81d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:40.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f094fafd640 1 -- 172.21.15.5:0/264791998 reap_dead start 2024-05-20T18:37:40.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f0951b22740 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585ad848900 0x5585ad81d080 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x5585ae551d70 tx=0x5585ad784840 comp rx=0 tx=0).stop 2024-05-20T18:37:40.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 shutdown_connections 2024-05-20T18:37:40.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f0951b22740 1 --2- 172.21.15.5:0/264791998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585ad848900 0x5585ad81d080 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 >> 172.21.15.5:0/264791998 conn(0x5585ad849200 msgr2=0x5585ad7b7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:40.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 shutdown_connections 2024-05-20T18:37:40.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.129+0000 7f0951b22740 1 -- 172.21.15.5:0/264791998 wait complete. 2024-05-20T18:37:40.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.130+0000 7f0951b22740 1 -- 172.21.15.5:0/3069847361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585ad849b00 msgr2=0x5585ad81d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:40.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.130+0000 7f0951b22740 1 --2- 172.21.15.5:0/3069847361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585ad849b00 0x5585ad81d600 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x5585ae4add10 tx=0x5585ad7860c0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.130+0000 7f0951b22740 1 -- 172.21.15.5:0/3069847361 shutdown_connections 2024-05-20T18:37:40.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.130+0000 7f0951b22740 1 --2- 172.21.15.5:0/3069847361 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5585ad849f80 0x5585ad81c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.130+0000 7f0951b22740 1 --2- 172.21.15.5:0/3069847361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585ad849b00 0x5585ad81d600 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.130+0000 7f0951b22740 1 --2- 172.21.15.5:0/3069847361 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5585ad849680 0x5585ad81db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.130+0000 7f0951b22740 1 -- 172.21.15.5:0/3069847361 >> 172.21.15.5:0/3069847361 conn(0x5585ad84a400 msgr2=0x5585ad7b7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:40.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.130+0000 7f0951b22740 1 -- 172.21.15.5:0/3069847361 shutdown_connections 2024-05-20T18:37:40.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.130+0000 7f0951b22740 1 -- 172.21.15.5:0/3069847361 wait complete. 2024-05-20T18:37:40.148 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.120032, result: 2024-05-20T18:37:40.149 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-20T18:37:40.151 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-20T18:37:40.194 INFO:teuthology.orchestra.run.smithi005.stdout:[global] 2024-05-20T18:37:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: chdir = "" 2024-05-20T18:37:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-20T18:37:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: auth supported = cephx 2024-05-20T18:37:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: filestore xattr use omap = true 2024-05-20T18:37:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: mon clock drift allowed = 1.000 2024-05-20T18:37:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: osd crush chooseleaf type = 0 2024-05-20T18:37:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: auth debug = true 2024-05-20T18:37:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on old message = true 2024-05-20T18:37:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on bug = true 2024-05-20T18:37:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-20T18:37:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: mon pg warn max object skew = 0 2024-05-20T18:37:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: # disable pg_autoscaler by default for new pools 2024-05-20T18:37:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-20T18:37:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default size = 2 2024-05-20T18:37:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow primary affinity = true 2024-05-20T18:37:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow pg remap = true 2024-05-20T18:37:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on legacy crush tunables = false 2024-05-20T18:37:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on crush straw calc version zero = false 2024-05-20T18:37:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on no sortbitwise = false 2024-05-20T18:37:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on osd down out interval zero = false 2024-05-20T18:37:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on too few osds = false 2024-05-20T18:37:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-20T18:37:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-20T18:37:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: mon_allow_pool_size_one = true 2024-05-20T18:37:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-20T18:37:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: osd default data pool replay window = 5 2024-05-20T18:37:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: mon allow pool delete = true 2024-05-20T18:37:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: mon cluster log file level = debug 2024-05-20T18:37:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: debug asserts on shutdown = true 2024-05-20T18:37:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: mon health detail to clog = false 2024-05-20T18:37:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: mon host = "172.21.15.5,172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790]" 2024-05-20T18:37:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on skipped message = False 2024-05-20T18:37:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: fsid = 399ab250-fd3c-47fa-a1c8-25e8149c9cac 2024-05-20T18:37:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.198 INFO:teuthology.orchestra.run.smithi005.stdout:[osd] 2024-05-20T18:37:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: osd journal size = 100 2024-05-20T18:37:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub load threshold = 5.0 2024-05-20T18:37:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub max interval = 600 2024-05-20T18:37:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: osd mclock profile = high_recovery_ops 2024-05-20T18:37:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: osd recover clone overlap = true 2024-05-20T18:37:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: osd recovery max chunk = 1048576 2024-05-20T18:37:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug shutdown = true 2024-05-20T18:37:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug op order = true 2024-05-20T18:37:40.199 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug verify stray on activate = true 2024-05-20T18:37:40.199 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug trim objects = true 2024-05-20T18:37:40.199 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.199 INFO:teuthology.orchestra.run.smithi005.stdout: osd open classes on start = true 2024-05-20T18:37:40.199 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug pg log writeout = true 2024-05-20T18:37:40.199 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.199 INFO:teuthology.orchestra.run.smithi005.stdout: osd deep scrub update digest min age = 30 2024-05-20T18:37:40.199 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.199 INFO:teuthology.orchestra.run.smithi005.stdout: osd map max advance = 10 2024-05-20T18:37:40.199 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.199 INFO:teuthology.orchestra.run.smithi005.stdout: journal zero on create = true 2024-05-20T18:37:40.199 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.199 INFO:teuthology.orchestra.run.smithi005.stdout: filestore ondisk finisher threads = 3 2024-05-20T18:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: filestore apply finisher threads = 3 2024-05-20T18:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: bdev debug aio = true 2024-05-20T18:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug misdirected ops = true 2024-05-20T18:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore block size = 96636764160 2024-05-20T18:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore fsck on mount = True 2024-05-20T18:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluefs = 20 2024-05-20T18:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluestore = 20 2024-05-20T18:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-20T18:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: debug osd = 20 2024-05-20T18:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: debug rocksdb = 10 2024-05-20T18:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd backfillfull_ratio = 0.85 2024-05-20T18:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd full ratio = 0.9 2024-05-20T18:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd nearfull ratio = 0.8 2024-05-20T18:37:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: osd failsafe full ratio = 0.95 2024-05-20T18:37:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: osd objectstore = bluestore 2024-05-20T18:37:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: osd op complaint time = 180 2024-05-20T18:37:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.201 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr] 2024-05-20T18:37:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-20T18:37:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: debug mgr = 20 2024-05-20T18:37:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2024-05-20T18:37:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2024-05-20T18:37:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2024-05-20T18:37:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2024-05-20T18:37:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: mgr/telemetry/nag = false 2024-05-20T18:37:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: debug client = 20 2024-05-20T18:37:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.201 INFO:teuthology.orchestra.run.smithi005.stdout:[mon] 2024-05-20T18:37:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-20T18:37:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2024-05-20T18:37:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: debug paxos = 20 2024-05-20T18:37:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2024-05-20T18:37:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: mon data avail warn = 5 2024-05-20T18:37:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: mon mgr mkfs grace = 240 2024-05-20T18:37:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2024-05-20T18:37:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd reporter subtree level = osd 2024-05-20T18:37:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd prime pg temp = true 2024-05-20T18:37:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2024-05-20T18:37:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-20T18:37:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: auth mon ticket ttl = 660 # 11m 2024-05-20T18:37:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: auth service ticket ttl = 240 # 4m 2024-05-20T18:37:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: # don't complain about insecure global_id in the test suite 2024-05-20T18:37:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-20T18:37:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-20T18:37:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: # 1m isn't quite enough 2024-05-20T18:37:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: mon_down_mkfs_grace = 2m 2024-05-20T18:37:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_filestore_osds = false 2024-05-20T18:37:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: mon down mkfs grace = 300 2024-05-20T18:37:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: mon op complaint time = 120 2024-05-20T18:37:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.203 INFO:teuthology.orchestra.run.smithi005.stdout:[client] 2024-05-20T18:37:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: rgw cache enabled = true 2024-05-20T18:37:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable ops log = true 2024-05-20T18:37:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable usage log = true 2024-05-20T18:37:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-20T18:37:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-20T18:37:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: client mount timeout = 600 2024-05-20T18:37:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: debug client = 20 2024-05-20T18:37:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-20T18:37:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: pid file = "" 2024-05-20T18:37:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: rados mon op timeout = 900 2024-05-20T18:37:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: rados osd op timeout = 900 2024-05-20T18:37:40.204 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.a] 2024-05-20T18:37:40.204 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.b] 2024-05-20T18:37:40.204 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.c] 2024-05-20T18:37:40.205 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a] 2024-05-20T18:37:40.205 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.c] 2024-05-20T18:37:40.205 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b] 2024-05-20T18:37:40.205 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.d] 2024-05-20T18:37:40.205 INFO:teuthology.orchestra.run.smithi005.stdout:[mds] 2024-05-20T18:37:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: debug mds = 20 2024-05-20T18:37:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: debug mds balancer = 20 2024-05-20T18:37:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-20T18:37:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: mds debug frag = True 2024-05-20T18:37:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: mds debug scatterstat = False 2024-05-20T18:37:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: mds op complaint time = 180 2024-05-20T18:37:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: mds verify scatter = True 2024-05-20T18:37:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: osd op complaint time = 180 2024-05-20T18:37:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: rados mon op timeout = 900 2024-05-20T18:37:40.206 INFO:teuthology.orchestra.run.smithi005.stdout: rados osd op timeout = 900 2024-05-20T18:37:40.206 INFO:teuthology.orchestra.run.smithi063.stdout:[global] 2024-05-20T18:37:40.206 INFO:teuthology.orchestra.run.smithi063.stdout: chdir = "" 2024-05-20T18:37:40.206 INFO:teuthology.orchestra.run.smithi063.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-20T18:37:40.206 INFO:teuthology.orchestra.run.smithi063.stdout: auth supported = cephx 2024-05-20T18:37:40.206 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.206 INFO:teuthology.orchestra.run.smithi063.stdout: filestore xattr use omap = true 2024-05-20T18:37:40.207 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.207 INFO:teuthology.orchestra.run.smithi063.stdout: mon clock drift allowed = 1.000 2024-05-20T18:37:40.207 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.207 INFO:teuthology.orchestra.run.smithi063.stdout: osd crush chooseleaf type = 0 2024-05-20T18:37:40.207 INFO:teuthology.orchestra.run.smithi063.stdout: auth debug = true 2024-05-20T18:37:40.207 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.207 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on old message = true 2024-05-20T18:37:40.207 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on bug = true 2024-05-20T18:37:40.207 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.207 INFO:teuthology.orchestra.run.smithi063.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-20T18:37:40.207 INFO:teuthology.orchestra.run.smithi063.stdout: mon pg warn max object skew = 0 2024-05-20T18:37:40.207 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.207 INFO:teuthology.orchestra.run.smithi063.stdout: # disable pg_autoscaler by default for new pools 2024-05-20T18:37:40.207 INFO:teuthology.orchestra.run.smithi063.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-20T18:37:40.208 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.208 INFO:teuthology.orchestra.run.smithi063.stdout: osd pool default size = 2 2024-05-20T18:37:40.208 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.208 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd allow primary affinity = true 2024-05-20T18:37:40.208 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd allow pg remap = true 2024-05-20T18:37:40.208 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on legacy crush tunables = false 2024-05-20T18:37:40.208 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on crush straw calc version zero = false 2024-05-20T18:37:40.208 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on no sortbitwise = false 2024-05-20T18:37:40.208 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on osd down out interval zero = false 2024-05-20T18:37:40.208 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on too few osds = false 2024-05-20T18:37:40.208 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-20T18:37:40.208 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-20T18:37:40.208 INFO:teuthology.orchestra.run.smithi063.stdout: mon_allow_pool_size_one = true 2024-05-20T18:37:40.208 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.209 INFO:teuthology.orchestra.run.smithi063.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-20T18:37:40.209 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.209 INFO:teuthology.orchestra.run.smithi063.stdout: osd default data pool replay window = 5 2024-05-20T18:37:40.209 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.209 INFO:teuthology.orchestra.run.smithi063.stdout: mon allow pool delete = true 2024-05-20T18:37:40.209 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.209 INFO:teuthology.orchestra.run.smithi063.stdout: mon cluster log file level = debug 2024-05-20T18:37:40.209 INFO:teuthology.orchestra.run.smithi063.stdout: debug asserts on shutdown = true 2024-05-20T18:37:40.209 INFO:teuthology.orchestra.run.smithi063.stdout: mon health detail to clog = false 2024-05-20T18:37:40.209 INFO:teuthology.orchestra.run.smithi063.stdout: mon host = "172.21.15.5,172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790]" 2024-05-20T18:37:40.209 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on skipped message = False 2024-05-20T18:37:40.209 INFO:teuthology.orchestra.run.smithi063.stdout: fsid = 399ab250-fd3c-47fa-a1c8-25e8149c9cac 2024-05-20T18:37:40.209 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.210 INFO:teuthology.orchestra.run.smithi063.stdout:[osd] 2024-05-20T18:37:40.210 INFO:teuthology.orchestra.run.smithi063.stdout: osd journal size = 100 2024-05-20T18:37:40.210 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.210 INFO:teuthology.orchestra.run.smithi063.stdout: osd scrub load threshold = 5.0 2024-05-20T18:37:40.210 INFO:teuthology.orchestra.run.smithi063.stdout: osd scrub max interval = 600 2024-05-20T18:37:40.210 INFO:teuthology.orchestra.run.smithi063.stdout: osd mclock profile = high_recovery_ops 2024-05-20T18:37:40.210 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.210 INFO:teuthology.orchestra.run.smithi063.stdout: osd recover clone overlap = true 2024-05-20T18:37:40.210 INFO:teuthology.orchestra.run.smithi063.stdout: osd recovery max chunk = 1048576 2024-05-20T18:37:40.210 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.210 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug shutdown = true 2024-05-20T18:37:40.210 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug op order = true 2024-05-20T18:37:40.210 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug verify stray on activate = true 2024-05-20T18:37:40.211 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug trim objects = true 2024-05-20T18:37:40.211 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.211 INFO:teuthology.orchestra.run.smithi063.stdout: osd open classes on start = true 2024-05-20T18:37:40.211 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug pg log writeout = true 2024-05-20T18:37:40.211 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.211 INFO:teuthology.orchestra.run.smithi063.stdout: osd deep scrub update digest min age = 30 2024-05-20T18:37:40.211 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.211 INFO:teuthology.orchestra.run.smithi063.stdout: osd map max advance = 10 2024-05-20T18:37:40.211 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.211 INFO:teuthology.orchestra.run.smithi063.stdout: journal zero on create = true 2024-05-20T18:37:40.211 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.211 INFO:teuthology.orchestra.run.smithi063.stdout: filestore ondisk finisher threads = 3 2024-05-20T18:37:40.211 INFO:teuthology.orchestra.run.smithi063.stdout: filestore apply finisher threads = 3 2024-05-20T18:37:40.211 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.212 INFO:teuthology.orchestra.run.smithi063.stdout: bdev debug aio = true 2024-05-20T18:37:40.212 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug misdirected ops = true 2024-05-20T18:37:40.212 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore block size = 96636764160 2024-05-20T18:37:40.212 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore fsck on mount = True 2024-05-20T18:37:40.212 INFO:teuthology.orchestra.run.smithi063.stdout: debug bluefs = 20 2024-05-20T18:37:40.212 INFO:teuthology.orchestra.run.smithi063.stdout: debug bluestore = 20 2024-05-20T18:37:40.212 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-20T18:37:40.212 INFO:teuthology.orchestra.run.smithi063.stdout: debug osd = 20 2024-05-20T18:37:40.212 INFO:teuthology.orchestra.run.smithi063.stdout: debug rocksdb = 10 2024-05-20T18:37:40.212 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd backfillfull_ratio = 0.85 2024-05-20T18:37:40.212 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd full ratio = 0.9 2024-05-20T18:37:40.212 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd nearfull ratio = 0.8 2024-05-20T18:37:40.212 INFO:teuthology.orchestra.run.smithi063.stdout: osd failsafe full ratio = 0.95 2024-05-20T18:37:40.212 INFO:teuthology.orchestra.run.smithi063.stdout: osd objectstore = bluestore 2024-05-20T18:37:40.213 INFO:teuthology.orchestra.run.smithi063.stdout: osd op complaint time = 180 2024-05-20T18:37:40.213 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.213 INFO:teuthology.orchestra.run.smithi063.stdout:[mgr] 2024-05-20T18:37:40.213 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-20T18:37:40.213 INFO:teuthology.orchestra.run.smithi063.stdout: debug mgr = 20 2024-05-20T18:37:40.213 INFO:teuthology.orchestra.run.smithi063.stdout: debug mon = 20 2024-05-20T18:37:40.213 INFO:teuthology.orchestra.run.smithi063.stdout: debug auth = 20 2024-05-20T18:37:40.213 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min pgs per osd = 4 2024-05-20T18:37:40.213 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min bytes per osd = 10 2024-05-20T18:37:40.213 INFO:teuthology.orchestra.run.smithi063.stdout: mgr/telemetry/nag = false 2024-05-20T18:37:40.213 INFO:teuthology.orchestra.run.smithi063.stdout: debug client = 20 2024-05-20T18:37:40.213 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.213 INFO:teuthology.orchestra.run.smithi063.stdout:[mon] 2024-05-20T18:37:40.213 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-20T18:37:40.214 INFO:teuthology.orchestra.run.smithi063.stdout: debug mon = 20 2024-05-20T18:37:40.214 INFO:teuthology.orchestra.run.smithi063.stdout: debug paxos = 20 2024-05-20T18:37:40.214 INFO:teuthology.orchestra.run.smithi063.stdout: debug auth = 20 2024-05-20T18:37:40.214 INFO:teuthology.orchestra.run.smithi063.stdout: mon data avail warn = 5 2024-05-20T18:37:40.214 INFO:teuthology.orchestra.run.smithi063.stdout: mon mgr mkfs grace = 240 2024-05-20T18:37:40.214 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min pgs per osd = 4 2024-05-20T18:37:40.214 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd reporter subtree level = osd 2024-05-20T18:37:40.214 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd prime pg temp = true 2024-05-20T18:37:40.214 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min bytes per osd = 10 2024-05-20T18:37:40.214 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.214 INFO:teuthology.orchestra.run.smithi063.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-20T18:37:40.214 INFO:teuthology.orchestra.run.smithi063.stdout: auth mon ticket ttl = 660 # 11m 2024-05-20T18:37:40.214 INFO:teuthology.orchestra.run.smithi063.stdout: auth service ticket ttl = 240 # 4m 2024-05-20T18:37:40.215 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.215 INFO:teuthology.orchestra.run.smithi063.stdout: # don't complain about insecure global_id in the test suite 2024-05-20T18:37:40.215 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-20T18:37:40.215 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-20T18:37:40.215 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.215 INFO:teuthology.orchestra.run.smithi063.stdout: # 1m isn't quite enough 2024-05-20T18:37:40.215 INFO:teuthology.orchestra.run.smithi063.stdout: mon_down_mkfs_grace = 2m 2024-05-20T18:37:40.215 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.215 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_filestore_osds = false 2024-05-20T18:37:40.215 INFO:teuthology.orchestra.run.smithi063.stdout: mon down mkfs grace = 300 2024-05-20T18:37:40.215 INFO:teuthology.orchestra.run.smithi063.stdout: mon op complaint time = 120 2024-05-20T18:37:40.215 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.215 INFO:teuthology.orchestra.run.smithi063.stdout:[client] 2024-05-20T18:37:40.215 INFO:teuthology.orchestra.run.smithi063.stdout: rgw cache enabled = true 2024-05-20T18:37:40.216 INFO:teuthology.orchestra.run.smithi063.stdout: rgw enable ops log = true 2024-05-20T18:37:40.216 INFO:teuthology.orchestra.run.smithi063.stdout: rgw enable usage log = true 2024-05-20T18:37:40.216 INFO:teuthology.orchestra.run.smithi063.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-20T18:37:40.216 INFO:teuthology.orchestra.run.smithi063.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-20T18:37:40.216 INFO:teuthology.orchestra.run.smithi063.stdout: client mount timeout = 600 2024-05-20T18:37:40.216 INFO:teuthology.orchestra.run.smithi063.stdout: debug client = 20 2024-05-20T18:37:40.216 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-20T18:37:40.216 INFO:teuthology.orchestra.run.smithi063.stdout: pid file = "" 2024-05-20T18:37:40.216 INFO:teuthology.orchestra.run.smithi063.stdout: rados mon op timeout = 900 2024-05-20T18:37:40.216 INFO:teuthology.orchestra.run.smithi063.stdout: rados osd op timeout = 900 2024-05-20T18:37:40.216 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.a] 2024-05-20T18:37:40.216 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.b] 2024-05-20T18:37:40.216 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.c] 2024-05-20T18:37:40.216 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.a] 2024-05-20T18:37:40.217 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.c] 2024-05-20T18:37:40.217 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.b] 2024-05-20T18:37:40.217 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.d] 2024-05-20T18:37:40.217 INFO:teuthology.orchestra.run.smithi063.stdout:[mds] 2024-05-20T18:37:40.217 INFO:teuthology.orchestra.run.smithi063.stdout: debug mds = 20 2024-05-20T18:37:40.217 INFO:teuthology.orchestra.run.smithi063.stdout: debug mds balancer = 20 2024-05-20T18:37:40.217 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-20T18:37:40.217 INFO:teuthology.orchestra.run.smithi063.stdout: mds debug frag = True 2024-05-20T18:37:40.217 INFO:teuthology.orchestra.run.smithi063.stdout: mds debug scatterstat = False 2024-05-20T18:37:40.217 INFO:teuthology.orchestra.run.smithi063.stdout: mds op complaint time = 180 2024-05-20T18:37:40.217 INFO:teuthology.orchestra.run.smithi063.stdout: mds verify scatter = True 2024-05-20T18:37:40.217 INFO:teuthology.orchestra.run.smithi063.stdout: osd op complaint time = 180 2024-05-20T18:37:40.217 INFO:teuthology.orchestra.run.smithi063.stdout: rados mon op timeout = 900 2024-05-20T18:37:40.217 INFO:teuthology.orchestra.run.smithi063.stdout: rados osd op timeout = 900 2024-05-20T18:37:40.219 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-20T18:37:40.262 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-20T18:37:40.305 INFO:teuthology.orchestra.run.smithi005.stdout:[global] 2024-05-20T18:37:40.305 INFO:teuthology.orchestra.run.smithi005.stdout: chdir = "" 2024-05-20T18:37:40.305 INFO:teuthology.orchestra.run.smithi005.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-20T18:37:40.305 INFO:teuthology.orchestra.run.smithi005.stdout: auth supported = cephx 2024-05-20T18:37:40.305 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.305 INFO:teuthology.orchestra.run.smithi005.stdout: filestore xattr use omap = true 2024-05-20T18:37:40.306 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.306 INFO:teuthology.orchestra.run.smithi005.stdout: mon clock drift allowed = 1.000 2024-05-20T18:37:40.306 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.306 INFO:teuthology.orchestra.run.smithi005.stdout: osd crush chooseleaf type = 0 2024-05-20T18:37:40.306 INFO:teuthology.orchestra.run.smithi005.stdout: auth debug = true 2024-05-20T18:37:40.306 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.306 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on old message = true 2024-05-20T18:37:40.306 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on bug = true 2024-05-20T18:37:40.306 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.306 INFO:teuthology.orchestra.run.smithi005.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-20T18:37:40.306 INFO:teuthology.orchestra.run.smithi005.stdout: mon pg warn max object skew = 0 2024-05-20T18:37:40.306 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.306 INFO:teuthology.orchestra.run.smithi005.stdout: # disable pg_autoscaler by default for new pools 2024-05-20T18:37:40.307 INFO:teuthology.orchestra.run.smithi005.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-20T18:37:40.307 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.307 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default size = 2 2024-05-20T18:37:40.307 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.307 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow primary affinity = true 2024-05-20T18:37:40.307 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow pg remap = true 2024-05-20T18:37:40.307 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on legacy crush tunables = false 2024-05-20T18:37:40.307 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on crush straw calc version zero = false 2024-05-20T18:37:40.307 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on no sortbitwise = false 2024-05-20T18:37:40.307 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on osd down out interval zero = false 2024-05-20T18:37:40.307 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on too few osds = false 2024-05-20T18:37:40.307 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-20T18:37:40.307 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-20T18:37:40.307 INFO:teuthology.orchestra.run.smithi005.stdout: mon_allow_pool_size_one = true 2024-05-20T18:37:40.308 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.308 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-20T18:37:40.308 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.308 INFO:teuthology.orchestra.run.smithi005.stdout: osd default data pool replay window = 5 2024-05-20T18:37:40.308 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.308 INFO:teuthology.orchestra.run.smithi005.stdout: mon allow pool delete = true 2024-05-20T18:37:40.308 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.308 INFO:teuthology.orchestra.run.smithi005.stdout: mon cluster log file level = debug 2024-05-20T18:37:40.309 INFO:teuthology.orchestra.run.smithi005.stdout: debug asserts on shutdown = true 2024-05-20T18:37:40.309 INFO:teuthology.orchestra.run.smithi005.stdout: mon health detail to clog = false 2024-05-20T18:37:40.309 INFO:teuthology.orchestra.run.smithi005.stdout: mon host = "172.21.15.5,172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790]" 2024-05-20T18:37:40.309 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on skipped message = False 2024-05-20T18:37:40.309 INFO:teuthology.orchestra.run.smithi005.stdout: fsid = 399ab250-fd3c-47fa-a1c8-25e8149c9cac 2024-05-20T18:37:40.309 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.309 INFO:teuthology.orchestra.run.smithi005.stdout:[osd] 2024-05-20T18:37:40.309 INFO:teuthology.orchestra.run.smithi005.stdout: osd journal size = 100 2024-05-20T18:37:40.309 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.309 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub load threshold = 5.0 2024-05-20T18:37:40.309 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub max interval = 600 2024-05-20T18:37:40.310 INFO:teuthology.orchestra.run.smithi005.stdout: osd mclock profile = high_recovery_ops 2024-05-20T18:37:40.310 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.310 INFO:teuthology.orchestra.run.smithi005.stdout: osd recover clone overlap = true 2024-05-20T18:37:40.310 INFO:teuthology.orchestra.run.smithi005.stdout: osd recovery max chunk = 1048576 2024-05-20T18:37:40.310 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.310 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug shutdown = true 2024-05-20T18:37:40.310 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug op order = true 2024-05-20T18:37:40.310 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug verify stray on activate = true 2024-05-20T18:37:40.310 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug trim objects = true 2024-05-20T18:37:40.310 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.310 INFO:teuthology.orchestra.run.smithi005.stdout: osd open classes on start = true 2024-05-20T18:37:40.310 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug pg log writeout = true 2024-05-20T18:37:40.310 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.310 INFO:teuthology.orchestra.run.smithi005.stdout: osd deep scrub update digest min age = 30 2024-05-20T18:37:40.311 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.311 INFO:teuthology.orchestra.run.smithi005.stdout: osd map max advance = 10 2024-05-20T18:37:40.311 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.311 INFO:teuthology.orchestra.run.smithi005.stdout: journal zero on create = true 2024-05-20T18:37:40.311 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.311 INFO:teuthology.orchestra.run.smithi005.stdout: filestore ondisk finisher threads = 3 2024-05-20T18:37:40.311 INFO:teuthology.orchestra.run.smithi005.stdout: filestore apply finisher threads = 3 2024-05-20T18:37:40.312 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.312 INFO:teuthology.orchestra.run.smithi005.stdout: bdev debug aio = true 2024-05-20T18:37:40.312 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug misdirected ops = true 2024-05-20T18:37:40.312 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore block size = 96636764160 2024-05-20T18:37:40.312 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore fsck on mount = True 2024-05-20T18:37:40.312 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluefs = 20 2024-05-20T18:37:40.312 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluestore = 20 2024-05-20T18:37:40.312 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-20T18:37:40.312 INFO:teuthology.orchestra.run.smithi005.stdout: debug osd = 20 2024-05-20T18:37:40.312 INFO:teuthology.orchestra.run.smithi005.stdout: debug rocksdb = 10 2024-05-20T18:37:40.312 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd backfillfull_ratio = 0.85 2024-05-20T18:37:40.312 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd full ratio = 0.9 2024-05-20T18:37:40.312 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd nearfull ratio = 0.8 2024-05-20T18:37:40.312 INFO:teuthology.orchestra.run.smithi005.stdout: osd failsafe full ratio = 0.95 2024-05-20T18:37:40.313 INFO:teuthology.orchestra.run.smithi005.stdout: osd objectstore = bluestore 2024-05-20T18:37:40.313 INFO:teuthology.orchestra.run.smithi005.stdout: osd op complaint time = 180 2024-05-20T18:37:40.313 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.313 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr] 2024-05-20T18:37:40.313 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-20T18:37:40.313 INFO:teuthology.orchestra.run.smithi005.stdout: debug mgr = 20 2024-05-20T18:37:40.314 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2024-05-20T18:37:40.314 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2024-05-20T18:37:40.314 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2024-05-20T18:37:40.314 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2024-05-20T18:37:40.314 INFO:teuthology.orchestra.run.smithi005.stdout: mgr/telemetry/nag = false 2024-05-20T18:37:40.314 INFO:teuthology.orchestra.run.smithi005.stdout: debug client = 20 2024-05-20T18:37:40.314 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.314 INFO:teuthology.orchestra.run.smithi005.stdout:[mon] 2024-05-20T18:37:40.314 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-20T18:37:40.314 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2024-05-20T18:37:40.314 INFO:teuthology.orchestra.run.smithi005.stdout: debug paxos = 20 2024-05-20T18:37:40.314 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2024-05-20T18:37:40.314 INFO:teuthology.orchestra.run.smithi005.stdout: mon data avail warn = 5 2024-05-20T18:37:40.314 INFO:teuthology.orchestra.run.smithi005.stdout: mon mgr mkfs grace = 240 2024-05-20T18:37:40.315 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2024-05-20T18:37:40.315 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd reporter subtree level = osd 2024-05-20T18:37:40.315 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd prime pg temp = true 2024-05-20T18:37:40.315 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2024-05-20T18:37:40.315 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.315 INFO:teuthology.orchestra.run.smithi005.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-20T18:37:40.315 INFO:teuthology.orchestra.run.smithi005.stdout: auth mon ticket ttl = 660 # 11m 2024-05-20T18:37:40.315 INFO:teuthology.orchestra.run.smithi005.stdout: auth service ticket ttl = 240 # 4m 2024-05-20T18:37:40.315 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.315 INFO:teuthology.orchestra.run.smithi005.stdout: # don't complain about insecure global_id in the test suite 2024-05-20T18:37:40.315 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-20T18:37:40.315 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-20T18:37:40.315 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.315 INFO:teuthology.orchestra.run.smithi005.stdout: # 1m isn't quite enough 2024-05-20T18:37:40.316 INFO:teuthology.orchestra.run.smithi005.stdout: mon_down_mkfs_grace = 2m 2024-05-20T18:37:40.316 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.316 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_filestore_osds = false 2024-05-20T18:37:40.316 INFO:teuthology.orchestra.run.smithi005.stdout: mon down mkfs grace = 300 2024-05-20T18:37:40.316 INFO:teuthology.orchestra.run.smithi005.stdout: mon op complaint time = 120 2024-05-20T18:37:40.316 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:40.316 INFO:teuthology.orchestra.run.smithi005.stdout:[client] 2024-05-20T18:37:40.316 INFO:teuthology.orchestra.run.smithi005.stdout: rgw cache enabled = true 2024-05-20T18:37:40.316 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable ops log = true 2024-05-20T18:37:40.316 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable usage log = true 2024-05-20T18:37:40.316 INFO:teuthology.orchestra.run.smithi005.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-20T18:37:40.316 INFO:teuthology.orchestra.run.smithi005.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-20T18:37:40.316 INFO:teuthology.orchestra.run.smithi005.stdout: client mount timeout = 600 2024-05-20T18:37:40.316 INFO:teuthology.orchestra.run.smithi005.stdout: debug client = 20 2024-05-20T18:37:40.317 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-20T18:37:40.317 INFO:teuthology.orchestra.run.smithi005.stdout: pid file = "" 2024-05-20T18:37:40.317 INFO:teuthology.orchestra.run.smithi005.stdout: rados mon op timeout = 900 2024-05-20T18:37:40.317 INFO:teuthology.orchestra.run.smithi005.stdout: rados osd op timeout = 900 2024-05-20T18:37:40.317 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.a] 2024-05-20T18:37:40.317 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.b] 2024-05-20T18:37:40.317 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.c] 2024-05-20T18:37:40.317 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a] 2024-05-20T18:37:40.317 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.c] 2024-05-20T18:37:40.317 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b] 2024-05-20T18:37:40.317 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.d] 2024-05-20T18:37:40.317 INFO:teuthology.orchestra.run.smithi005.stdout:[mds] 2024-05-20T18:37:40.317 INFO:teuthology.orchestra.run.smithi005.stdout: debug mds = 20 2024-05-20T18:37:40.317 INFO:teuthology.orchestra.run.smithi005.stdout: debug mds balancer = 20 2024-05-20T18:37:40.318 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-20T18:37:40.318 INFO:teuthology.orchestra.run.smithi005.stdout: mds debug frag = True 2024-05-20T18:37:40.318 INFO:teuthology.orchestra.run.smithi005.stdout: mds debug scatterstat = True 2024-05-20T18:37:40.318 INFO:teuthology.orchestra.run.smithi005.stdout: mds op complaint time = 180 2024-05-20T18:37:40.318 INFO:teuthology.orchestra.run.smithi005.stdout: mds verify scatter = True 2024-05-20T18:37:40.318 INFO:teuthology.orchestra.run.smithi005.stdout: osd op complaint time = 180 2024-05-20T18:37:40.319 INFO:teuthology.orchestra.run.smithi005.stdout: rados mon op timeout = 900 2024-05-20T18:37:40.319 INFO:teuthology.orchestra.run.smithi005.stdout: rados osd op timeout = 900 2024-05-20T18:37:40.320 INFO:teuthology.orchestra.run.smithi063.stdout:[global] 2024-05-20T18:37:40.320 INFO:teuthology.orchestra.run.smithi063.stdout: chdir = "" 2024-05-20T18:37:40.320 INFO:teuthology.orchestra.run.smithi063.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-20T18:37:40.320 INFO:teuthology.orchestra.run.smithi063.stdout: auth supported = cephx 2024-05-20T18:37:40.320 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.320 INFO:teuthology.orchestra.run.smithi063.stdout: filestore xattr use omap = true 2024-05-20T18:37:40.321 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.321 INFO:teuthology.orchestra.run.smithi063.stdout: mon clock drift allowed = 1.000 2024-05-20T18:37:40.321 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.321 INFO:teuthology.orchestra.run.smithi063.stdout: osd crush chooseleaf type = 0 2024-05-20T18:37:40.321 INFO:teuthology.orchestra.run.smithi063.stdout: auth debug = true 2024-05-20T18:37:40.321 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.321 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on old message = true 2024-05-20T18:37:40.321 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on bug = true 2024-05-20T18:37:40.321 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.321 INFO:teuthology.orchestra.run.smithi063.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-20T18:37:40.321 INFO:teuthology.orchestra.run.smithi063.stdout: mon pg warn max object skew = 0 2024-05-20T18:37:40.321 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.321 INFO:teuthology.orchestra.run.smithi063.stdout: # disable pg_autoscaler by default for new pools 2024-05-20T18:37:40.321 INFO:teuthology.orchestra.run.smithi063.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-20T18:37:40.322 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.322 INFO:teuthology.orchestra.run.smithi063.stdout: osd pool default size = 2 2024-05-20T18:37:40.322 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.322 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd allow primary affinity = true 2024-05-20T18:37:40.322 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd allow pg remap = true 2024-05-20T18:37:40.322 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on legacy crush tunables = false 2024-05-20T18:37:40.322 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on crush straw calc version zero = false 2024-05-20T18:37:40.322 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on no sortbitwise = false 2024-05-20T18:37:40.322 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on osd down out interval zero = false 2024-05-20T18:37:40.322 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on too few osds = false 2024-05-20T18:37:40.322 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-20T18:37:40.322 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-20T18:37:40.322 INFO:teuthology.orchestra.run.smithi063.stdout: mon_allow_pool_size_one = true 2024-05-20T18:37:40.323 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.323 INFO:teuthology.orchestra.run.smithi063.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-20T18:37:40.323 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.323 INFO:teuthology.orchestra.run.smithi063.stdout: osd default data pool replay window = 5 2024-05-20T18:37:40.323 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.323 INFO:teuthology.orchestra.run.smithi063.stdout: mon allow pool delete = true 2024-05-20T18:37:40.323 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.323 INFO:teuthology.orchestra.run.smithi063.stdout: mon cluster log file level = debug 2024-05-20T18:37:40.323 INFO:teuthology.orchestra.run.smithi063.stdout: debug asserts on shutdown = true 2024-05-20T18:37:40.323 INFO:teuthology.orchestra.run.smithi063.stdout: mon health detail to clog = false 2024-05-20T18:37:40.323 INFO:teuthology.orchestra.run.smithi063.stdout: mon host = "172.21.15.5,172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790]" 2024-05-20T18:37:40.323 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on skipped message = False 2024-05-20T18:37:40.323 INFO:teuthology.orchestra.run.smithi063.stdout: fsid = 399ab250-fd3c-47fa-a1c8-25e8149c9cac 2024-05-20T18:37:40.323 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.324 INFO:teuthology.orchestra.run.smithi063.stdout:[osd] 2024-05-20T18:37:40.324 INFO:teuthology.orchestra.run.smithi063.stdout: osd journal size = 100 2024-05-20T18:37:40.324 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.324 INFO:teuthology.orchestra.run.smithi063.stdout: osd scrub load threshold = 5.0 2024-05-20T18:37:40.324 INFO:teuthology.orchestra.run.smithi063.stdout: osd scrub max interval = 600 2024-05-20T18:37:40.324 INFO:teuthology.orchestra.run.smithi063.stdout: osd mclock profile = high_recovery_ops 2024-05-20T18:37:40.324 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.324 INFO:teuthology.orchestra.run.smithi063.stdout: osd recover clone overlap = true 2024-05-20T18:37:40.324 INFO:teuthology.orchestra.run.smithi063.stdout: osd recovery max chunk = 1048576 2024-05-20T18:37:40.324 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.324 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug shutdown = true 2024-05-20T18:37:40.324 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug op order = true 2024-05-20T18:37:40.324 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug verify stray on activate = true 2024-05-20T18:37:40.324 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug trim objects = true 2024-05-20T18:37:40.325 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.325 INFO:teuthology.orchestra.run.smithi063.stdout: osd open classes on start = true 2024-05-20T18:37:40.325 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug pg log writeout = true 2024-05-20T18:37:40.325 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.325 INFO:teuthology.orchestra.run.smithi063.stdout: osd deep scrub update digest min age = 30 2024-05-20T18:37:40.325 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.325 INFO:teuthology.orchestra.run.smithi063.stdout: osd map max advance = 10 2024-05-20T18:37:40.325 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.325 INFO:teuthology.orchestra.run.smithi063.stdout: journal zero on create = true 2024-05-20T18:37:40.325 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.325 INFO:teuthology.orchestra.run.smithi063.stdout: filestore ondisk finisher threads = 3 2024-05-20T18:37:40.325 INFO:teuthology.orchestra.run.smithi063.stdout: filestore apply finisher threads = 3 2024-05-20T18:37:40.325 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.326 INFO:teuthology.orchestra.run.smithi063.stdout: bdev debug aio = true 2024-05-20T18:37:40.326 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug misdirected ops = true 2024-05-20T18:37:40.326 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore block size = 96636764160 2024-05-20T18:37:40.326 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore fsck on mount = True 2024-05-20T18:37:40.326 INFO:teuthology.orchestra.run.smithi063.stdout: debug bluefs = 20 2024-05-20T18:37:40.326 INFO:teuthology.orchestra.run.smithi063.stdout: debug bluestore = 20 2024-05-20T18:37:40.326 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-20T18:37:40.326 INFO:teuthology.orchestra.run.smithi063.stdout: debug osd = 20 2024-05-20T18:37:40.326 INFO:teuthology.orchestra.run.smithi063.stdout: debug rocksdb = 10 2024-05-20T18:37:40.326 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd backfillfull_ratio = 0.85 2024-05-20T18:37:40.326 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd full ratio = 0.9 2024-05-20T18:37:40.326 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd nearfull ratio = 0.8 2024-05-20T18:37:40.326 INFO:teuthology.orchestra.run.smithi063.stdout: osd failsafe full ratio = 0.95 2024-05-20T18:37:40.326 INFO:teuthology.orchestra.run.smithi063.stdout: osd objectstore = bluestore 2024-05-20T18:37:40.327 INFO:teuthology.orchestra.run.smithi063.stdout: osd op complaint time = 180 2024-05-20T18:37:40.327 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.327 INFO:teuthology.orchestra.run.smithi063.stdout:[mgr] 2024-05-20T18:37:40.327 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-20T18:37:40.327 INFO:teuthology.orchestra.run.smithi063.stdout: debug mgr = 20 2024-05-20T18:37:40.327 INFO:teuthology.orchestra.run.smithi063.stdout: debug mon = 20 2024-05-20T18:37:40.327 INFO:teuthology.orchestra.run.smithi063.stdout: debug auth = 20 2024-05-20T18:37:40.327 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min pgs per osd = 4 2024-05-20T18:37:40.327 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min bytes per osd = 10 2024-05-20T18:37:40.327 INFO:teuthology.orchestra.run.smithi063.stdout: mgr/telemetry/nag = false 2024-05-20T18:37:40.327 INFO:teuthology.orchestra.run.smithi063.stdout: debug client = 20 2024-05-20T18:37:40.327 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.327 INFO:teuthology.orchestra.run.smithi063.stdout:[mon] 2024-05-20T18:37:40.327 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-20T18:37:40.328 INFO:teuthology.orchestra.run.smithi063.stdout: debug mon = 20 2024-05-20T18:37:40.328 INFO:teuthology.orchestra.run.smithi063.stdout: debug paxos = 20 2024-05-20T18:37:40.328 INFO:teuthology.orchestra.run.smithi063.stdout: debug auth = 20 2024-05-20T18:37:40.328 INFO:teuthology.orchestra.run.smithi063.stdout: mon data avail warn = 5 2024-05-20T18:37:40.328 INFO:teuthology.orchestra.run.smithi063.stdout: mon mgr mkfs grace = 240 2024-05-20T18:37:40.328 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min pgs per osd = 4 2024-05-20T18:37:40.328 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd reporter subtree level = osd 2024-05-20T18:37:40.328 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd prime pg temp = true 2024-05-20T18:37:40.328 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min bytes per osd = 10 2024-05-20T18:37:40.328 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.328 INFO:teuthology.orchestra.run.smithi063.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-20T18:37:40.328 INFO:teuthology.orchestra.run.smithi063.stdout: auth mon ticket ttl = 660 # 11m 2024-05-20T18:37:40.328 INFO:teuthology.orchestra.run.smithi063.stdout: auth service ticket ttl = 240 # 4m 2024-05-20T18:37:40.328 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.329 INFO:teuthology.orchestra.run.smithi063.stdout: # don't complain about insecure global_id in the test suite 2024-05-20T18:37:40.329 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-20T18:37:40.329 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-20T18:37:40.329 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.329 INFO:teuthology.orchestra.run.smithi063.stdout: # 1m isn't quite enough 2024-05-20T18:37:40.329 INFO:teuthology.orchestra.run.smithi063.stdout: mon_down_mkfs_grace = 2m 2024-05-20T18:37:40.329 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.329 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_filestore_osds = false 2024-05-20T18:37:40.329 INFO:teuthology.orchestra.run.smithi063.stdout: mon down mkfs grace = 300 2024-05-20T18:37:40.329 INFO:teuthology.orchestra.run.smithi063.stdout: mon op complaint time = 120 2024-05-20T18:37:40.329 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-20T18:37:40.329 INFO:teuthology.orchestra.run.smithi063.stdout:[client] 2024-05-20T18:37:40.329 INFO:teuthology.orchestra.run.smithi063.stdout: rgw cache enabled = true 2024-05-20T18:37:40.329 INFO:teuthology.orchestra.run.smithi063.stdout: rgw enable ops log = true 2024-05-20T18:37:40.330 INFO:teuthology.orchestra.run.smithi063.stdout: rgw enable usage log = true 2024-05-20T18:37:40.330 INFO:teuthology.orchestra.run.smithi063.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-20T18:37:40.330 INFO:teuthology.orchestra.run.smithi063.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-20T18:37:40.330 INFO:teuthology.orchestra.run.smithi063.stdout: client mount timeout = 600 2024-05-20T18:37:40.330 INFO:teuthology.orchestra.run.smithi063.stdout: debug client = 20 2024-05-20T18:37:40.330 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-20T18:37:40.330 INFO:teuthology.orchestra.run.smithi063.stdout: pid file = "" 2024-05-20T18:37:40.330 INFO:teuthology.orchestra.run.smithi063.stdout: rados mon op timeout = 900 2024-05-20T18:37:40.330 INFO:teuthology.orchestra.run.smithi063.stdout: rados osd op timeout = 900 2024-05-20T18:37:40.330 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.a] 2024-05-20T18:37:40.330 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.b] 2024-05-20T18:37:40.330 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.c] 2024-05-20T18:37:40.330 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.a] 2024-05-20T18:37:40.330 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.c] 2024-05-20T18:37:40.331 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.b] 2024-05-20T18:37:40.331 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.d] 2024-05-20T18:37:40.331 INFO:teuthology.orchestra.run.smithi063.stdout:[mds] 2024-05-20T18:37:40.331 INFO:teuthology.orchestra.run.smithi063.stdout: debug mds = 20 2024-05-20T18:37:40.331 INFO:teuthology.orchestra.run.smithi063.stdout: debug mds balancer = 20 2024-05-20T18:37:40.331 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-20T18:37:40.331 INFO:teuthology.orchestra.run.smithi063.stdout: mds debug frag = True 2024-05-20T18:37:40.331 INFO:teuthology.orchestra.run.smithi063.stdout: mds debug scatterstat = True 2024-05-20T18:37:40.331 INFO:teuthology.orchestra.run.smithi063.stdout: mds op complaint time = 180 2024-05-20T18:37:40.331 INFO:teuthology.orchestra.run.smithi063.stdout: mds verify scatter = True 2024-05-20T18:37:40.331 INFO:teuthology.orchestra.run.smithi063.stdout: osd op complaint time = 180 2024-05-20T18:37:40.331 INFO:teuthology.orchestra.run.smithi063.stdout: rados mon op timeout = 900 2024-05-20T18:37:40.331 INFO:teuthology.orchestra.run.smithi063.stdout: rados osd op timeout = 900 2024-05-20T18:37:40.332 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-20T18:37:40.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.477+0000 7f02f3a98640 1 Processor -- start 2024-05-20T18:37:40.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.477+0000 7f02f3a98640 1 -- start start 2024-05-20T18:37:40.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.478+0000 7f02f3a98640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02ec15d530 0x7f02ec15aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:40.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.478+0000 7f02f3a98640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f02ec070850 0x7f02ec070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:40.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.478+0000 7f02f3a98640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02ec0cd580 con 0x7f02ec0b1700 2024-05-20T18:37:40.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.478+0000 7f02f3a98640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f02ec0ce190 con 0x7f02ec070850 2024-05-20T18:37:40.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.478+0000 7f02f3a98640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f02ec0cd860 con 0x7f02ec15d530 2024-05-20T18:37:40.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.479+0000 7f02f1834640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02ec15d530 0x7f02ec15aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:40.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.479+0000 7f02f2035640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f02ec070850 0x7f02ec070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:40.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.479+0000 7f02f1834640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02ec15d530 0x7f02ec15aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:47124/0 (socket says 172.21.15.5:47124) 2024-05-20T18:37:40.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.479+0000 7f02f2035640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f02ec070850 0x7f02ec070c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.5:45276/0 (socket says 172.21.15.5:45276) 2024-05-20T18:37:40.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.479+0000 7f02f1834640 1 -- 172.21.15.5:0/781185034 learned_addr learned my addr 172.21.15.5:0/781185034 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:40.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.479+0000 7f02f1033640 1 --1- 172.21.15.5:0/781185034 >> v1:172.21.15.63:6789/0 conn(0x7f02ec0b1700 0x7f02ec0b1b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:45378/0 (socket says 172.21.15.5:45378) 2024-05-20T18:37:40.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.479+0000 7f02f0832640 1 -- 172.21.15.5:0/781185034 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1327070991 0 0) 0x7f02ec0cd580 con 0x7f02ec0b1700 2024-05-20T18:37:40.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.479+0000 7f02f0832640 1 -- 172.21.15.5:0/781185034 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02d4003520 con 0x7f02ec0b1700 2024-05-20T18:37:40.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.479+0000 7f02f2035640 1 -- 172.21.15.5:0/781185034 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02ec15d530 msgr2=0x7f02ec15aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:40.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.479+0000 7f02f2035640 1 --2- 172.21.15.5:0/781185034 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02ec15d530 0x7f02ec15aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.479+0000 7f02f2035640 1 -- 172.21.15.5:0/781185034 >> v1:172.21.15.63:6789/0 conn(0x7f02ec0b1700 legacy=0x7f02ec0b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:40.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.479+0000 7f02f2035640 1 -- 172.21.15.5:0/781185034 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02ec0cdb40 con 0x7f02ec070850 2024-05-20T18:37:40.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.479+0000 7f02f1834640 1 --2- 172.21.15.5:0/781185034 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02ec15d530 0x7f02ec15aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:37:40.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.480+0000 7f02f2035640 1 --2- 172.21.15.5:0/781185034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02ec070850 0x7f02ec070c60 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f02dc009d90 tx=0x7f02dc02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3fe212a92722232 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:40.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.480+0000 7f02f0832640 1 -- 172.21.15.5:0/781185034 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02dc007740 con 0x7f02ec070850 2024-05-20T18:37:40.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.480+0000 7f02f0832640 1 -- 172.21.15.5:0/781185034 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02dc0078e0 con 0x7f02ec070850 2024-05-20T18:37:40.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.480+0000 7f02f0832640 1 -- 172.21.15.5:0/781185034 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02dc007cb0 con 0x7f02ec070850 2024-05-20T18:37:40.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.481+0000 7f02f3a98640 1 -- 172.21.15.5:0/781185034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02ec070850 msgr2=0x7f02ec070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:40.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.481+0000 7f02f3a98640 1 --2- 172.21.15.5:0/781185034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02ec070850 0x7f02ec070c60 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f02dc009d90 tx=0x7f02dc02a660 comp rx=0 tx=0).stop 2024-05-20T18:37:40.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.481+0000 7f02f3a98640 1 -- 172.21.15.5:0/781185034 shutdown_connections 2024-05-20T18:37:40.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.481+0000 7f02f3a98640 1 --2- 172.21.15.5:0/781185034 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02ec070850 0x7f02ec070c60 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.481+0000 7f02f3a98640 1 --2- 172.21.15.5:0/781185034 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02ec15d530 0x7f02ec15aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.481+0000 7f02f3a98640 1 -- 172.21.15.5:0/781185034 >> 172.21.15.5:0/781185034 conn(0x7f02ec0d29d0 msgr2=0x7f02ec0d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:40.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.481+0000 7f02f3a98640 1 -- 172.21.15.5:0/781185034 shutdown_connections 2024-05-20T18:37:40.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.481+0000 7f02f3a98640 1 -- 172.21.15.5:0/781185034 wait complete. 2024-05-20T18:37:40.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.483+0000 7f02f3a98640 1 Processor -- start 2024-05-20T18:37:40.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.483+0000 7f02f3a98640 1 -- start start 2024-05-20T18:37:40.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.484+0000 7f02f3a98640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02ec070850 0x7f02ec0c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:40.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.484+0000 7f02f3a98640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02ec0b1700 0x7f02ec1f5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:40.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.484+0000 7f02f3a98640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02ec15d530 0x7f02ec0cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:40.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.484+0000 7f02f3a98640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f02ec0711a0 con 0x7f02ec15d530 2024-05-20T18:37:40.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.484+0000 7f02f3a98640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f02ec0cd230 con 0x7f02ec0b1700 2024-05-20T18:37:40.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.484+0000 7f02f3a98640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f02ec071320 con 0x7f02ec070850 2024-05-20T18:37:40.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.484+0000 7f02f2035640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02ec15d530 0x7f02ec0cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:40.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.484+0000 7f02f2035640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02ec15d530 0x7f02ec0cb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35696/0 (socket says 172.21.15.5:35696) 2024-05-20T18:37:40.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.484+0000 7f02f2035640 1 -- 172.21.15.5:0/1569882437 learned_addr learned my addr 172.21.15.5:0/1569882437 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.484+0000 7f02f1834640 1 --2- 172.21.15.5:0/1569882437 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02ec070850 0x7f02ec0c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.484+0000 7f02f1033640 1 --2- 172.21.15.5:0/1569882437 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02ec0b1700 0x7f02ec1f5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.485+0000 7f02f2035640 1 -- 172.21.15.5:0/1569882437 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02ec070850 msgr2=0x7f02ec0c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.485+0000 7f02f2035640 1 --2- 172.21.15.5:0/1569882437 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02ec070850 0x7f02ec0c8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.485+0000 7f02f2035640 1 -- 172.21.15.5:0/1569882437 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02ec0b1700 msgr2=0x7f02ec1f5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.485+0000 7f02f2035640 1 --2- 172.21.15.5:0/1569882437 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02ec0b1700 0x7f02ec1f5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.485+0000 7f02f2035640 1 -- 172.21.15.5:0/1569882437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02ec0cac00 con 0x7f02ec15d530 2024-05-20T18:37:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.485+0000 7f02f2035640 1 --2- 172.21.15.5:0/1569882437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02ec15d530 0x7f02ec0cb530 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f02dc02ac50 tx=0x7f02dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.485+0000 7f02ea7fc640 1 -- 172.21.15.5:0/1569882437 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02dc03f070 con 0x7f02ec15d530 2024-05-20T18:37:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.485+0000 7f02ea7fc640 1 -- 172.21.15.5:0/1569882437 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02dc031bc0 con 0x7f02ec15d530 2024-05-20T18:37:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.485+0000 7f02ea7fc640 1 -- 172.21.15.5:0/1569882437 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02dc009450 con 0x7f02ec15d530 2024-05-20T18:37:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.485+0000 7f02f3a98640 1 -- 172.21.15.5:0/1569882437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02ec0c6750 con 0x7f02ec15d530 2024-05-20T18:37:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.485+0000 7f02f3a98640 1 -- 172.21.15.5:0/1569882437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02ec0c6d10 con 0x7f02ec15d530 2024-05-20T18:37:40.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.486+0000 7f02f3a98640 1 -- 172.21.15.5:0/1569882437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02ec075b00 con 0x7f02ec15d530 2024-05-20T18:37:40.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.487+0000 7f02ea7fc640 1 -- 172.21.15.5:0/1569882437 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f02dc03b410 con 0x7f02ec15d530 2024-05-20T18:37:40.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.487+0000 7f02ea7fc640 1 --2- 172.21.15.5:0/1569882437 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f02d806be30 0x7f02d806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:40.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.487+0000 7f02ea7fc640 1 -- 172.21.15.5:0/1569882437 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f02dc0ac6b0 con 0x7f02ec15d530 2024-05-20T18:37:40.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.487+0000 7f02f1834640 1 --2- 172.21.15.5:0/1569882437 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f02d806be30 0x7f02d806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:40.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.488+0000 7f02f1834640 1 --2- 172.21.15.5:0/1569882437 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f02d806be30 0x7f02d806e340 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f02e4006cc0 tx=0x7f02e4009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:40.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.492+0000 7f02ea7fc640 1 -- 172.21.15.5:0/1569882437 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f02dc07a7d0 con 0x7f02ec15d530 2024-05-20T18:37:40.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:40.719+0000 7f02f3a98640 1 -- 172.21.15.5:0/1569882437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f02ec0d1d40 con 0x7f02ec15d530 2024-05-20T18:37:41.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.706+0000 7f02ea7fc640 1 -- 172.21.15.5:0/1569882437 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v107) ==== 162+0+0 (secure 0 0 0) 0x7f02dc07ead0 con 0x7f02ec15d530 2024-05-20T18:37:41.707 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-20T18:37:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.710+0000 7f02f3a98640 1 -- 172.21.15.5:0/1569882437 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f02d806be30 msgr2=0x7f02d806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.710+0000 7f02f3a98640 1 --2- 172.21.15.5:0/1569882437 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f02d806be30 0x7f02d806e340 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f02e4006cc0 tx=0x7f02e4009240 comp rx=0 tx=0).stop 2024-05-20T18:37:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.710+0000 7f02f3a98640 1 -- 172.21.15.5:0/1569882437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02ec15d530 msgr2=0x7f02ec0cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.710+0000 7f02f3a98640 1 --2- 172.21.15.5:0/1569882437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02ec15d530 0x7f02ec0cb530 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f02dc02ac50 tx=0x7f02dc004a40 comp rx=0 tx=0).stop 2024-05-20T18:37:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.710+0000 7f02f3a98640 1 -- 172.21.15.5:0/1569882437 shutdown_connections 2024-05-20T18:37:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.710+0000 7f02f3a98640 1 --2- 172.21.15.5:0/1569882437 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f02d806be30 0x7f02d806e340 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.710+0000 7f02f3a98640 1 --2- 172.21.15.5:0/1569882437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02ec15d530 0x7f02ec0cb530 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.710+0000 7f02f3a98640 1 --2- 172.21.15.5:0/1569882437 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02ec0b1700 0x7f02ec1f5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.710+0000 7f02f3a98640 1 --2- 172.21.15.5:0/1569882437 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02ec070850 0x7f02ec0c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.710+0000 7f02f3a98640 1 -- 172.21.15.5:0/1569882437 >> 172.21.15.5:0/1569882437 conn(0x7f02ec0d29d0 msgr2=0x7f02ec15ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.710+0000 7f02f3a98640 1 -- 172.21.15.5:0/1569882437 shutdown_connections 2024-05-20T18:37:41.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.710+0000 7f02f3a98640 1 -- 172.21.15.5:0/1569882437 wait complete. 2024-05-20T18:37:41.730 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:37:41.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.853+0000 7fcbad3fe640 1 Processor -- start 2024-05-20T18:37:41.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.853+0000 7fcbad3fe640 1 -- start start 2024-05-20T18:37:41.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.854+0000 7fcbad3fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcba806d3d0 0x7fcba806d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:41.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.854+0000 7fcbad3fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcba80b8190 0x7fcba80b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:41.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.854+0000 7fcbad3fe640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcba806dd20 con 0x7fcba8162820 2024-05-20T18:37:41.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.854+0000 7fcbad3fe640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fcba80d0630 con 0x7fcba80b8190 2024-05-20T18:37:41.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.854+0000 7fcbad3fe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcba80d39c0 con 0x7fcba806d3d0 2024-05-20T18:37:41.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.854+0000 7fcba6ffd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fcba8162820 0x7fcba80b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:32930/0 (socket says 172.21.15.5:32930) 2024-05-20T18:37:41.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.854+0000 7fcba6ffd640 1 -- 172.21.15.5:0/3224950766 learned_addr learned my addr 172.21.15.5:0/3224950766 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:41.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.854+0000 7fcba77fe640 1 --2- 172.21.15.5:0/3224950766 >> v2:172.21.15.5:3300/0 conn(0x7fcba80b8190 0x7fcba80b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:41.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.854+0000 7fcba67fc640 1 --2- 172.21.15.5:0/3224950766 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcba806d3d0 0x7fcba806d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:41.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.854+0000 7fcba5ffb640 1 -- 172.21.15.5:0/3224950766 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3046320659 0 0) 0x7fcb98001110 con 0x7fcba8162820 2024-05-20T18:37:41.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.854+0000 7fcba5ffb640 1 -- 172.21.15.5:0/3224950766 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb90003520 con 0x7fcba8162820 2024-05-20T18:37:41.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.855+0000 7fcba77fe640 1 -- 172.21.15.5:0/3224950766 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcba806d3d0 msgr2=0x7fcba806d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:41.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.855+0000 7fcba77fe640 1 --2- 172.21.15.5:0/3224950766 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcba806d3d0 0x7fcba806d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:41.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.855+0000 7fcba77fe640 1 -- 172.21.15.5:0/3224950766 >> v1:172.21.15.5:6789/0 conn(0x7fcba8162820 legacy=0x7fcba80b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:41.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.855+0000 7fcba77fe640 1 -- 172.21.15.5:0/3224950766 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba80d3ca0 con 0x7fcba80b8190 2024-05-20T18:37:41.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.855+0000 7fcba77fe640 1 --2- 172.21.15.5:0/3224950766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba80b8190 0x7fcba80b85a0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009d90 tx=0x7fcba002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4bd425d97d33ed server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.855+0000 7fcba5ffb640 1 -- 172.21.15.5:0/3224950766 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba0007740 con 0x7fcba80b8190 2024-05-20T18:37:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.855+0000 7fcba5ffb640 1 -- 172.21.15.5:0/3224950766 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcba00078e0 con 0x7fcba80b8190 2024-05-20T18:37:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.856+0000 7fcba5ffb640 1 -- 172.21.15.5:0/3224950766 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba0007cb0 con 0x7fcba80b8190 2024-05-20T18:37:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.856+0000 7fcbad3fe640 1 -- 172.21.15.5:0/3224950766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba80b8190 msgr2=0x7fcba80b85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.856+0000 7fcbad3fe640 1 --2- 172.21.15.5:0/3224950766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba80b8190 0x7fcba80b85a0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009d90 tx=0x7fcba002a660 comp rx=0 tx=0).stop 2024-05-20T18:37:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.856+0000 7fcbad3fe640 1 -- 172.21.15.5:0/3224950766 shutdown_connections 2024-05-20T18:37:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.856+0000 7fcbad3fe640 1 --2- 172.21.15.5:0/3224950766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba80b8190 0x7fcba80b85a0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.856+0000 7fcbad3fe640 1 --2- 172.21.15.5:0/3224950766 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcba806d3d0 0x7fcba806d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.856+0000 7fcbad3fe640 1 -- 172.21.15.5:0/3224950766 >> 172.21.15.5:0/3224950766 conn(0x7fcba80d4bd0 msgr2=0x7fcba80d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.856+0000 7fcbad3fe640 1 -- 172.21.15.5:0/3224950766 shutdown_connections 2024-05-20T18:37:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.856+0000 7fcbad3fe640 1 -- 172.21.15.5:0/3224950766 wait complete. 2024-05-20T18:37:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.858+0000 7fcbad3fe640 1 Processor -- start 2024-05-20T18:37:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.858+0000 7fcbad3fe640 1 -- start start 2024-05-20T18:37:41.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.859+0000 7fcbad3fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcba806d3d0 0x7fcba814ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:41.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.859+0000 7fcbad3fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba80b8190 0x7fcba80c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:41.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.859+0000 7fcbad3fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba8162820 0x7fcba80c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:41.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.860+0000 7fcbad3fe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcba80d0300 con 0x7fcba8162820 2024-05-20T18:37:41.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.860+0000 7fcbad3fe640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcba80d4260 con 0x7fcba80b8190 2024-05-20T18:37:41.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.860+0000 7fcbad3fe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcba80b4050 con 0x7fcba806d3d0 2024-05-20T18:37:41.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.860+0000 7fcba77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba8162820 0x7fcba80c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:41.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.860+0000 7fcba77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba8162820 0x7fcba80c8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35710/0 (socket says 172.21.15.5:35710) 2024-05-20T18:37:41.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.860+0000 7fcba77fe640 1 -- 172.21.15.5:0/1158106458 learned_addr learned my addr 172.21.15.5:0/1158106458 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:41.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.860+0000 7fcba6ffd640 1 --2- 172.21.15.5:0/1158106458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcba806d3d0 0x7fcba814ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:41.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.860+0000 7fcba67fc640 1 --2- 172.21.15.5:0/1158106458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba80b8190 0x7fcba80c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:41.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.860+0000 7fcba77fe640 1 -- 172.21.15.5:0/1158106458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcba806d3d0 msgr2=0x7fcba814ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:41.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.860+0000 7fcba77fe640 1 --2- 172.21.15.5:0/1158106458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcba806d3d0 0x7fcba814ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:41.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.860+0000 7fcba77fe640 1 -- 172.21.15.5:0/1158106458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba80b8190 msgr2=0x7fcba80c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:41.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.860+0000 7fcba77fe640 1 --2- 172.21.15.5:0/1158106458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba80b8190 0x7fcba80c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:41.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.860+0000 7fcba77fe640 1 -- 172.21.15.5:0/1158106458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba80ce9f0 con 0x7fcba8162820 2024-05-20T18:37:41.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.860+0000 7fcba77fe640 1 --2- 172.21.15.5:0/1158106458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba8162820 0x7fcba80c8ea0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fcba002ac50 tx=0x7fcba0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:41.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.860+0000 7fcb8f7fe640 1 -- 172.21.15.5:0/1158106458 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba003f070 con 0x7fcba8162820 2024-05-20T18:37:41.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.861+0000 7fcb8f7fe640 1 -- 172.21.15.5:0/1158106458 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcba0031bc0 con 0x7fcba8162820 2024-05-20T18:37:41.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.861+0000 7fcb8f7fe640 1 -- 172.21.15.5:0/1158106458 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba0009450 con 0x7fcba8162820 2024-05-20T18:37:41.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.861+0000 7fcbad3fe640 1 -- 172.21.15.5:0/1158106458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcba80cbed0 con 0x7fcba8162820 2024-05-20T18:37:41.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.861+0000 7fcbad3fe640 1 -- 172.21.15.5:0/1158106458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcba80cda30 con 0x7fcba8162820 2024-05-20T18:37:41.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.862+0000 7fcbad3fe640 1 -- 172.21.15.5:0/1158106458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcba80cdc90 con 0x7fcba8162820 2024-05-20T18:37:41.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.862+0000 7fcb8f7fe640 1 -- 172.21.15.5:0/1158106458 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcba003b410 con 0x7fcba8162820 2024-05-20T18:37:41.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.863+0000 7fcb8f7fe640 1 --2- 172.21.15.5:0/1158106458 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcb9406be60 0x7fcb9406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:41.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.863+0000 7fcba6ffd640 1 --2- 172.21.15.5:0/1158106458 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcb9406be60 0x7fcb9406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:41.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.863+0000 7fcb8f7fe640 1 -- 172.21.15.5:0/1158106458 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7fcba00ac760 con 0x7fcba8162820 2024-05-20T18:37:41.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.863+0000 7fcba6ffd640 1 --2- 172.21.15.5:0/1158106458 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcb9406be60 0x7fcb9406e370 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fcb98005880 tx=0x7fcb98009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:41.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:41.868+0000 7fcb8f7fe640 1 -- 172.21.15.5:0/1158106458 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fcba007a7e0 con 0x7fcba8162820 2024-05-20T18:37:42.093 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:37:42.096 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:37:42.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:42.095+0000 7fcbad3fe640 1 -- 172.21.15.5:0/1158106458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcba8075b00 con 0x7fcba8162820 2024-05-20T18:37:42.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:42.096+0000 7fcb8f7fe640 1 -- 172.21.15.5:0/1158106458 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+5173 (secure 0 0 0) 0x7fcba007eae0 con 0x7fcba8162820 2024-05-20T18:37:42.097 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:42.098 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":108,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/266630184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":266630184},{"type":"v1","addr":"172.21.15.63:6834","nonce":266630184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7361,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1494022697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":1494022697},{"type":"v1","addr":"172.21.15.63:6835","nonce":1494022697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":9009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/1739221946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1739221946},{"type":"v1","addr":"172.21.15.5:6836","nonce":1739221946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:37:14.365195+0000","modified":"2024-05-20T18:37:41.711913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9444},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9444":{"gid":9444,"name":"c","rank":0,"incarnation":108,"state":"up:replay","state_seq":1,"addr":"172.21.15.5:6837/2067544478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":2067544478},{"type":"v1","addr":"172.21.15.5:6837","nonce":2067544478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-05-20T18:37:42.098 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 108 2024-05-20T18:37:42.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:42.098+0000 7fcbad3fe640 1 -- 172.21.15.5:0/1158106458 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcb9406be60 msgr2=0x7fcb9406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:42.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:42.098+0000 7fcbad3fe640 1 --2- 172.21.15.5:0/1158106458 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcb9406be60 0x7fcb9406e370 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fcb98005880 tx=0x7fcb98009040 comp rx=0 tx=0).stop 2024-05-20T18:37:42.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:42.098+0000 7fcbad3fe640 1 -- 172.21.15.5:0/1158106458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba8162820 msgr2=0x7fcba80c8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:42.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:42.098+0000 7fcbad3fe640 1 --2- 172.21.15.5:0/1158106458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba8162820 0x7fcba80c8ea0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fcba002ac50 tx=0x7fcba0004a40 comp rx=0 tx=0).stop 2024-05-20T18:37:42.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:42.098+0000 7fcbad3fe640 1 -- 172.21.15.5:0/1158106458 shutdown_connections 2024-05-20T18:37:42.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:42.098+0000 7fcbad3fe640 1 --2- 172.21.15.5:0/1158106458 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fcb9406be60 0x7fcb9406e370 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:42.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:42.098+0000 7fcbad3fe640 1 --2- 172.21.15.5:0/1158106458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba8162820 0x7fcba80c8ea0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:42.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:42.098+0000 7fcbad3fe640 1 --2- 172.21.15.5:0/1158106458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba80b8190 0x7fcba80c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:42.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:42.098+0000 7fcbad3fe640 1 --2- 172.21.15.5:0/1158106458 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcba806d3d0 0x7fcba814ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:42.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:42.098+0000 7fcbad3fe640 1 -- 172.21.15.5:0/1158106458 >> 172.21.15.5:0/1158106458 conn(0x7fcba80d4bd0 msgr2=0x7fcba815eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:42.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:42.098+0000 7fcbad3fe640 1 -- 172.21.15.5:0/1158106458 shutdown_connections 2024-05-20T18:37:42.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:42.098+0000 7fcbad3fe640 1 -- 172.21.15.5:0/1158106458 wait complete. 2024-05-20T18:37:42.515 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 108, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T18:37:14.365195+0000', 'modified': '2024-05-20T18:37:41.711913+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 121, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9444}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9444': {'gid': 9444, 'name': 'c', 'rank': 0, 'incarnation': 108, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.5:6837/2067544478', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6835', 'nonce': 2067544478}, {'type': 'v1', 'addr': '172.21.15.5:6837', 'nonce': 2067544478}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-20T18:37:42.516 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9444:up:replay 2024-05-20T18:37:43.517 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:37:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.646+0000 7f5666479640 1 Processor -- start 2024-05-20T18:37:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.646+0000 7f5666479640 1 -- start start 2024-05-20T18:37:43.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.647+0000 7f5666479640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5660162820 0x7f56600b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:43.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.647+0000 7f5666479640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f56600b3e80 0x7f566006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:43.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.647+0000 7f5666479640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f566006d910 0x7f56600d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:43.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.647+0000 7f5666479640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f566006dd20 con 0x7f56600b3e80 2024-05-20T18:37:43.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.647+0000 7f5666479640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f56600d0630 con 0x7f5660162820 2024-05-20T18:37:43.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.647+0000 7f5666479640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f56600d39c0 con 0x7f566006d910 2024-05-20T18:37:43.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.647+0000 7f565f7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f56600b3e80 0x7f566006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:43.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.647+0000 7f565f7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f56600b3e80 0x7f566006d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35724/0 (socket says 172.21.15.5:35724) 2024-05-20T18:37:43.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.647+0000 7f565f7fe640 1 -- 172.21.15.5:0/1934413937 learned_addr learned my addr 172.21.15.5:0/1934413937 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:43.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.648+0000 7f565ffff640 1 --2- 172.21.15.5:0/1934413937 >> v2:172.21.15.63:3300/0 conn(0x7f5660162820 0x7f56600b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:43.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.648+0000 7f5664a16640 1 --2- 172.21.15.5:0/1934413937 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f566006d910 0x7f56600d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:43.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.648+0000 7f565f7fe640 1 -- 172.21.15.5:0/1934413937 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f566006d910 msgr2=0x7f56600d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:43.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.648+0000 7f565f7fe640 1 --2- 172.21.15.5:0/1934413937 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f566006d910 0x7f56600d27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:43.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.648+0000 7f565f7fe640 1 -- 172.21.15.5:0/1934413937 >> v2:172.21.15.63:3300/0 conn(0x7f5660162820 msgr2=0x7f56600b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:43.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.648+0000 7f565f7fe640 1 --2- 172.21.15.5:0/1934413937 >> v2:172.21.15.63:3300/0 conn(0x7f5660162820 0x7f56600b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:43.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.648+0000 7f565f7fe640 1 -- 172.21.15.5:0/1934413937 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56600d3ca0 con 0x7f56600b3e80 2024-05-20T18:37:43.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.649+0000 7f565f7fe640 1 --2- 172.21.15.5:0/1934413937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56600b3e80 0x7f566006d3d0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f56600b7740 tx=0x7f565402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44611e6776a30d28 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:43.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.649+0000 7f565effd640 1 -- 172.21.15.5:0/1934413937 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5654037070 con 0x7f56600b3e80 2024-05-20T18:37:43.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.649+0000 7f565effd640 1 -- 172.21.15.5:0/1934413937 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5654032440 con 0x7f56600b3e80 2024-05-20T18:37:43.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.649+0000 7f565effd640 1 -- 172.21.15.5:0/1934413937 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5654032810 con 0x7f56600b3e80 2024-05-20T18:37:43.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.649+0000 7f5666479640 1 -- 172.21.15.5:0/1934413937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56600b3e80 msgr2=0x7f566006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:43.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.649+0000 7f5666479640 1 --2- 172.21.15.5:0/1934413937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56600b3e80 0x7f566006d3d0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f56600b7740 tx=0x7f565402a1e0 comp rx=0 tx=0).stop 2024-05-20T18:37:43.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.649+0000 7f5666479640 1 -- 172.21.15.5:0/1934413937 shutdown_connections 2024-05-20T18:37:43.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.650+0000 7f5666479640 1 --2- 172.21.15.5:0/1934413937 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f566006d910 0x7f56600d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:43.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.650+0000 7f5666479640 1 --2- 172.21.15.5:0/1934413937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56600b3e80 0x7f566006d3d0 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:43.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.650+0000 7f5666479640 1 --2- 172.21.15.5:0/1934413937 >> v2:172.21.15.63:3300/0 conn(0x7f5660162820 0x7f56600b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:43.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.650+0000 7f5666479640 1 -- 172.21.15.5:0/1934413937 >> 172.21.15.5:0/1934413937 conn(0x7f56600d4bd0 msgr2=0x7f56600d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:43.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.650+0000 7f5666479640 1 -- 172.21.15.5:0/1934413937 shutdown_connections 2024-05-20T18:37:43.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.650+0000 7f5666479640 1 -- 172.21.15.5:0/1934413937 wait complete. 2024-05-20T18:37:43.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.651+0000 7f5666479640 1 Processor -- start 2024-05-20T18:37:43.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.652+0000 7f5666479640 1 -- start start 2024-05-20T18:37:43.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.653+0000 7f5666479640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f566006d910 0x7f56600cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:43.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.653+0000 7f5666479640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56600b3e80 0x7f566014e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:43.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.653+0000 7f5666479640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5660162820 0x7f566015f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:43.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.653+0000 7f5666479640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56600d0300 con 0x7f56600b3e80 2024-05-20T18:37:43.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.653+0000 7f5666479640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f56600d4260 con 0x7f5660162820 2024-05-20T18:37:43.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.653+0000 7f5666479640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f56600d4780 con 0x7f566006d910 2024-05-20T18:37:43.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.653+0000 7f565f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56600b3e80 0x7f566014e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:43.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.653+0000 7f565f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56600b3e80 0x7f566014e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35738/0 (socket says 172.21.15.5:35738) 2024-05-20T18:37:43.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.653+0000 7f565f7fe640 1 -- 172.21.15.5:0/3418398353 learned_addr learned my addr 172.21.15.5:0/3418398353 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:43.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.653+0000 7f565ffff640 1 --2- 172.21.15.5:0/3418398353 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f566006d910 0x7f56600cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:43.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.653+0000 7f5664a16640 1 --2- 172.21.15.5:0/3418398353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5660162820 0x7f566015f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:43.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.653+0000 7f565f7fe640 1 -- 172.21.15.5:0/3418398353 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f566006d910 msgr2=0x7f56600cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:43.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.653+0000 7f565f7fe640 1 --2- 172.21.15.5:0/3418398353 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f566006d910 0x7f56600cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:43.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.653+0000 7f565f7fe640 1 -- 172.21.15.5:0/3418398353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5660162820 msgr2=0x7f566015f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:43.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.653+0000 7f565f7fe640 1 --2- 172.21.15.5:0/3418398353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5660162820 0x7f566015f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:43.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.653+0000 7f565f7fe640 1 -- 172.21.15.5:0/3418398353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56600cb830 con 0x7f56600b3e80 2024-05-20T18:37:43.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.653+0000 7f565ffff640 1 --2- 172.21.15.5:0/3418398353 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f566006d910 0x7f56600cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:43.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.654+0000 7f565f7fe640 1 --2- 172.21.15.5:0/3418398353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56600b3e80 0x7f566014e050 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f565402a220 tx=0x7f56540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:43.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.654+0000 7f565cff9640 1 -- 172.21.15.5:0/3418398353 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5654041020 con 0x7f56600b3e80 2024-05-20T18:37:43.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.654+0000 7f565cff9640 1 -- 172.21.15.5:0/3418398353 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56540306c0 con 0x7f56600b3e80 2024-05-20T18:37:43.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.654+0000 7f5666479640 1 -- 172.21.15.5:0/3418398353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56600c8a80 con 0x7f56600b3e80 2024-05-20T18:37:43.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.654+0000 7f565cff9640 1 -- 172.21.15.5:0/3418398353 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5654037040 con 0x7f56600b3e80 2024-05-20T18:37:43.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.654+0000 7f5666479640 1 -- 172.21.15.5:0/3418398353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56600c9040 con 0x7f56600b3e80 2024-05-20T18:37:43.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.655+0000 7f5666479640 1 -- 172.21.15.5:0/3418398353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5660075b00 con 0x7f56600b3e80 2024-05-20T18:37:43.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.656+0000 7f565cff9640 1 -- 172.21.15.5:0/3418398353 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5654044020 con 0x7f56600b3e80 2024-05-20T18:37:43.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.656+0000 7f565cff9640 1 --2- 172.21.15.5:0/3418398353 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f564c06be30 0x7f564c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:43.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.656+0000 7f565cff9640 1 -- 172.21.15.5:0/3418398353 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f56540ab970 con 0x7f56600b3e80 2024-05-20T18:37:43.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.656+0000 7f565ffff640 1 --2- 172.21.15.5:0/3418398353 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f564c06be30 0x7f564c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:43.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.657+0000 7f565ffff640 1 --2- 172.21.15.5:0/3418398353 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f564c06be30 0x7f564c06e340 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5650006fd0 tx=0x7f5650008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:43.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.661+0000 7f565cff9640 1 -- 172.21.15.5:0/3418398353 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f565404b030 con 0x7f56600b3e80 2024-05-20T18:37:43.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.888+0000 7f5666479640 1 -- 172.21.15.5:0/3418398353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56600b9780 con 0x7f56600b3e80 2024-05-20T18:37:43.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.888+0000 7f565cff9640 1 -- 172.21.15.5:0/3418398353 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) ==== 95+0+5173 (secure 0 0 0) 0x7f565407a050 con 0x7f56600b3e80 2024-05-20T18:37:43.890 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:43.890 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":110,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/266630184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":266630184},{"type":"v1","addr":"172.21.15.63:6834","nonce":266630184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7361,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1494022697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":1494022697},{"type":"v1","addr":"172.21.15.63:6835","nonce":1494022697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":9009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/1739221946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1739221946},{"type":"v1","addr":"172.21.15.5:6836","nonce":1739221946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:37:14.365195+0000","modified":"2024-05-20T18:37:42.716413+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9444},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9444":{"gid":9444,"name":"c","rank":0,"incarnation":108,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.5:6837/2067544478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":2067544478},{"type":"v1","addr":"172.21.15.5:6837","nonce":2067544478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-05-20T18:37:43.890 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 110 2024-05-20T18:37:43.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.890+0000 7f5666479640 1 -- 172.21.15.5:0/3418398353 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f564c06be30 msgr2=0x7f564c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:43.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.890+0000 7f5666479640 1 --2- 172.21.15.5:0/3418398353 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f564c06be30 0x7f564c06e340 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5650006fd0 tx=0x7f5650008040 comp rx=0 tx=0).stop 2024-05-20T18:37:43.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.890+0000 7f5666479640 1 -- 172.21.15.5:0/3418398353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56600b3e80 msgr2=0x7f566014e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:43.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.890+0000 7f5666479640 1 --2- 172.21.15.5:0/3418398353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56600b3e80 0x7f566014e050 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f565402a220 tx=0x7f56540046c0 comp rx=0 tx=0).stop 2024-05-20T18:37:43.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.890+0000 7f5666479640 1 -- 172.21.15.5:0/3418398353 shutdown_connections 2024-05-20T18:37:43.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.890+0000 7f5666479640 1 --2- 172.21.15.5:0/3418398353 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f564c06be30 0x7f564c06e340 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:43.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.890+0000 7f5666479640 1 --2- 172.21.15.5:0/3418398353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5660162820 0x7f566015f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:43.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.890+0000 7f5666479640 1 --2- 172.21.15.5:0/3418398353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56600b3e80 0x7f566014e050 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:43.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.890+0000 7f5666479640 1 --2- 172.21.15.5:0/3418398353 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f566006d910 0x7f56600cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:43.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.890+0000 7f5666479640 1 -- 172.21.15.5:0/3418398353 >> 172.21.15.5:0/3418398353 conn(0x7f56600d4bd0 msgr2=0x7f56600b8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:43.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.891+0000 7f5666479640 1 -- 172.21.15.5:0/3418398353 shutdown_connections 2024-05-20T18:37:43.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:43.891+0000 7f5666479640 1 -- 172.21.15.5:0/3418398353 wait complete. 2024-05-20T18:37:44.305 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 110, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T18:37:14.365195+0000', 'modified': '2024-05-20T18:37:42.716413+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 121, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9444}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9444': {'gid': 9444, 'name': 'c', 'rank': 0, 'incarnation': 108, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.5:6837/2067544478', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6835', 'nonce': 2067544478}, {'type': 'v1', 'addr': '172.21.15.5:6837', 'nonce': 2067544478}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-20T18:37:44.306 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9444:up:rejoin 2024-05-20T18:37:45.306 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:37:45.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.435+0000 7f5154937640 1 Processor -- start 2024-05-20T18:37:45.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.435+0000 7f5154937640 1 -- start start 2024-05-20T18:37:45.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.436+0000 7f5154937640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f515006ca80 0x7f515015f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:45.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.436+0000 7f5154937640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f51500ae320 0x7f515006bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:45.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.436+0000 7f5154937640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51500b1b80 con 0x7f51500ae730 2024-05-20T18:37:45.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.436+0000 7f5154937640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f515006b750 con 0x7f51500ae320 2024-05-20T18:37:45.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.436+0000 7f5154937640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f51500b12e0 con 0x7f515006ca80 2024-05-20T18:37:45.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.436+0000 7f5147fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f51500ae320 0x7f515006bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:45.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.436+0000 7f5147fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f51500ae320 0x7f515006bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35520/0 (socket says 172.21.15.5:35520) 2024-05-20T18:37:45.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.436+0000 7f5147fff640 1 -- 172.21.15.5:0/1104437072 learned_addr learned my addr 172.21.15.5:0/1104437072 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:45.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.436+0000 7f514e59c640 1 --2- 172.21.15.5:0/1104437072 >> v2:172.21.15.63:3300/0 conn(0x7f515006ca80 0x7f515015f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:45.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.437+0000 7f5147fff640 1 -- 172.21.15.5:0/1104437072 >> v2:172.21.15.63:3300/0 conn(0x7f515006ca80 msgr2=0x7f515015f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:45.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.437+0000 7f5147fff640 1 --2- 172.21.15.5:0/1104437072 >> v2:172.21.15.63:3300/0 conn(0x7f515006ca80 0x7f515015f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:45.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.437+0000 7f5147fff640 1 -- 172.21.15.5:0/1104437072 >> v1:172.21.15.63:6789/0 conn(0x7f51500ae730 legacy=0x7f515006c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:45.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.437+0000 7f5147fff640 1 -- 172.21.15.5:0/1104437072 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51500b15c0 con 0x7f51500ae320 2024-05-20T18:37:45.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.437+0000 7f5147fff640 1 --2- 172.21.15.5:0/1104437072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51500ae320 0x7f515006bbe0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f5150071470 tx=0x7f513802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576c9b8bd8fefabe server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:45.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.438+0000 7f514dd9b640 1 -- 172.21.15.5:0/1104437072 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5138037070 con 0x7f51500ae320 2024-05-20T18:37:45.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.438+0000 7f514dd9b640 1 -- 172.21.15.5:0/1104437072 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5138032440 con 0x7f51500ae320 2024-05-20T18:37:45.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.438+0000 7f514dd9b640 1 -- 172.21.15.5:0/1104437072 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5138032810 con 0x7f51500ae320 2024-05-20T18:37:45.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.438+0000 7f5154937640 1 -- 172.21.15.5:0/1104437072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51500ae320 msgr2=0x7f515006bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:45.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.438+0000 7f5154937640 1 --2- 172.21.15.5:0/1104437072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51500ae320 0x7f515006bbe0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f5150071470 tx=0x7f513802a230 comp rx=0 tx=0).stop 2024-05-20T18:37:45.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.438+0000 7f5154937640 1 -- 172.21.15.5:0/1104437072 shutdown_connections 2024-05-20T18:37:45.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.438+0000 7f5154937640 1 --2- 172.21.15.5:0/1104437072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51500ae320 0x7f515006bbe0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:45.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.438+0000 7f5154937640 1 --2- 172.21.15.5:0/1104437072 >> v2:172.21.15.63:3300/0 conn(0x7f515006ca80 0x7f515015f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:45.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.438+0000 7f5154937640 1 -- 172.21.15.5:0/1104437072 >> 172.21.15.5:0/1104437072 conn(0x7f5150072ed0 msgr2=0x7f51500b6080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:45.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.438+0000 7f5154937640 1 -- 172.21.15.5:0/1104437072 shutdown_connections 2024-05-20T18:37:45.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.438+0000 7f5154937640 1 -- 172.21.15.5:0/1104437072 wait complete. 2024-05-20T18:37:45.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.440+0000 7f5154937640 1 Processor -- start 2024-05-20T18:37:45.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.440+0000 7f5154937640 1 -- start start 2024-05-20T18:37:45.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.442+0000 7f5154937640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f515006ca80 0x7f51500cce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:45.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.442+0000 7f5154937640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51500ae320 0x7f515015dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:45.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.442+0000 7f5154937640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51500ae730 0x7f51501609c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:45.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.442+0000 7f5154937640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f51500d0c10 con 0x7f51500ae730 2024-05-20T18:37:45.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.442+0000 7f5154937640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5150074e00 con 0x7f515006ca80 2024-05-20T18:37:45.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.442+0000 7f5154937640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f51500d0d90 con 0x7f51500ae320 2024-05-20T18:37:45.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.442+0000 7f514ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51500ae730 0x7f51501609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:45.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.442+0000 7f514ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51500ae730 0x7f51501609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35530/0 (socket says 172.21.15.5:35530) 2024-05-20T18:37:45.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.442+0000 7f514ed9d640 1 -- 172.21.15.5:0/2312079454 learned_addr learned my addr 172.21.15.5:0/2312079454 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:45.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.442+0000 7f5147fff640 1 --2- 172.21.15.5:0/2312079454 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51500ae320 0x7f515015dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:45.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.442+0000 7f514e59c640 1 --2- 172.21.15.5:0/2312079454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f515006ca80 0x7f51500cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:45.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.442+0000 7f514ed9d640 1 -- 172.21.15.5:0/2312079454 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51500ae320 msgr2=0x7f515015dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:45.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.442+0000 7f514ed9d640 1 --2- 172.21.15.5:0/2312079454 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51500ae320 0x7f515015dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:45.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.442+0000 7f514ed9d640 1 -- 172.21.15.5:0/2312079454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f515006ca80 msgr2=0x7f51500cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:45.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.442+0000 7f514ed9d640 1 --2- 172.21.15.5:0/2312079454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f515006ca80 0x7f51500cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:45.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.442+0000 7f514ed9d640 1 -- 172.21.15.5:0/2312079454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51500cbdc0 con 0x7f51500ae730 2024-05-20T18:37:45.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.443+0000 7f514ed9d640 1 --2- 172.21.15.5:0/2312079454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51500ae730 0x7f51501609c0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f514800b9e0 tx=0x7f514800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:45.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.443+0000 7f5146ffd640 1 -- 172.21.15.5:0/2312079454 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5148003b70 con 0x7f51500ae730 2024-05-20T18:37:45.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.443+0000 7f5146ffd640 1 -- 172.21.15.5:0/2312079454 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5148003d10 con 0x7f51500ae730 2024-05-20T18:37:45.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.443+0000 7f5146ffd640 1 -- 172.21.15.5:0/2312079454 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514800cb60 con 0x7f51500ae730 2024-05-20T18:37:45.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.443+0000 7f5154937640 1 -- 172.21.15.5:0/2312079454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51500cc0a0 con 0x7f51500ae730 2024-05-20T18:37:45.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.443+0000 7f5154937640 1 -- 172.21.15.5:0/2312079454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51500cc660 con 0x7f51500ae730 2024-05-20T18:37:45.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.444+0000 7f5154937640 1 -- 172.21.15.5:0/2312079454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51500cbb10 con 0x7f51500ae730 2024-05-20T18:37:45.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.445+0000 7f5146ffd640 1 -- 172.21.15.5:0/2312079454 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f514800cd00 con 0x7f51500ae730 2024-05-20T18:37:45.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.445+0000 7f5146ffd640 1 --2- 172.21.15.5:0/2312079454 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f513006be60 0x7f513006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:45.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.445+0000 7f5146ffd640 1 -- 172.21.15.5:0/2312079454 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f514808af00 con 0x7f51500ae730 2024-05-20T18:37:45.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.445+0000 7f514e59c640 1 --2- 172.21.15.5:0/2312079454 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f513006be60 0x7f513006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:45.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.446+0000 7f514e59c640 1 --2- 172.21.15.5:0/2312079454 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f513006be60 0x7f513006e370 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5140006fd0 tx=0x7f5140008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:45.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.450+0000 7f5146ffd640 1 -- 172.21.15.5:0/2312079454 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5148059670 con 0x7f51500ae730 2024-05-20T18:37:45.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.673+0000 7f5154937640 1 -- 172.21.15.5:0/2312079454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5150075b00 con 0x7f51500ae730 2024-05-20T18:37:45.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.674+0000 7f5146ffd640 1 -- 172.21.15.5:0/2312079454 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5180 (secure 0 0 0) 0x7f5148058f70 con 0x7f51500ae730 2024-05-20T18:37:45.675 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:45.675 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":111,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/266630184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":266630184},{"type":"v1","addr":"172.21.15.63:6834","nonce":266630184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7361,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1494022697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":1494022697},{"type":"v1","addr":"172.21.15.63:6835","nonce":1494022697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":9009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/1739221946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1739221946},{"type":"v1","addr":"172.21.15.5:6836","nonce":1739221946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:37:14.365195+0000","modified":"2024-05-20T18:37:44.722140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9444},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9444":{"gid":9444,"name":"c","rank":0,"incarnation":108,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6837/2067544478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":2067544478},{"type":"v1","addr":"172.21.15.5:6837","nonce":2067544478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9444,"qdb_cluster":[9444]},"id":8}]} 2024-05-20T18:37:45.676 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 111 2024-05-20T18:37:45.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.676+0000 7f5154937640 1 -- 172.21.15.5:0/2312079454 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f513006be60 msgr2=0x7f513006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:45.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.676+0000 7f5154937640 1 --2- 172.21.15.5:0/2312079454 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f513006be60 0x7f513006e370 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5140006fd0 tx=0x7f5140008040 comp rx=0 tx=0).stop 2024-05-20T18:37:45.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.676+0000 7f5154937640 1 -- 172.21.15.5:0/2312079454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51500ae730 msgr2=0x7f51501609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:45.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.676+0000 7f5154937640 1 --2- 172.21.15.5:0/2312079454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51500ae730 0x7f51501609c0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f514800b9e0 tx=0x7f514800bea0 comp rx=0 tx=0).stop 2024-05-20T18:37:45.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.676+0000 7f5154937640 1 -- 172.21.15.5:0/2312079454 shutdown_connections 2024-05-20T18:37:45.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.676+0000 7f5154937640 1 --2- 172.21.15.5:0/2312079454 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f513006be60 0x7f513006e370 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:45.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.676+0000 7f5154937640 1 --2- 172.21.15.5:0/2312079454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51500ae730 0x7f51501609c0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:45.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.676+0000 7f5154937640 1 --2- 172.21.15.5:0/2312079454 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51500ae320 0x7f515015dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:45.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.676+0000 7f5154937640 1 --2- 172.21.15.5:0/2312079454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f515006ca80 0x7f51500cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:45.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.676+0000 7f5154937640 1 -- 172.21.15.5:0/2312079454 >> 172.21.15.5:0/2312079454 conn(0x7f5150072ed0 msgr2=0x7f51500aeba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:45.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.676+0000 7f5154937640 1 -- 172.21.15.5:0/2312079454 shutdown_connections 2024-05-20T18:37:45.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:45.676+0000 7f5154937640 1 -- 172.21.15.5:0/2312079454 wait complete. 2024-05-20T18:37:45.708 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-05-20T18:37:45.707+0000 7fb1ad191640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T18:37:46.091 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 111, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-20T18:37:14.365195+0000', 'modified': '2024-05-20T18:37:44.722140+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 121, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9444}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9444': {'gid': 9444, 'name': 'c', 'rank': 0, 'incarnation': 108, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.5:6837/2067544478', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6835', 'nonce': 2067544478}, {'type': 'v1', 'addr': '172.21.15.5:6837', 'nonce': 2067544478}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9444, 'qdb_cluster': [9444]} 2024-05-20T18:37:46.092 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-20T18:37:46.092 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-20T18:37:46.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.211+0000 7f65847f5640 1 Processor -- start 2024-05-20T18:37:46.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.211+0000 7f65847f5640 1 -- start start 2024-05-20T18:37:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.213+0000 7f65847f5640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f657c0cf850 0x7f657c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.213+0000 7f65847f5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f657c0d14f0 0x7f657c0b6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.213+0000 7f65847f5640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f657c0d1ac0 con 0x7f657c0d10e0 2024-05-20T18:37:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.213+0000 7f65847f5640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f657c0d1da0 con 0x7f657c0cf850 2024-05-20T18:37:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.213+0000 7f65847f5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f657c0d2080 con 0x7f657c0d14f0 2024-05-20T18:37:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.213+0000 7f6581d90640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f657c0cf850 0x7f657c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.213+0000 7f6582d92640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f657c0d14f0 0x7f657c0b6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.213+0000 7f6582d92640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f657c0d14f0 0x7f657c0b6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:34058/0 (socket says 172.21.15.5:34058) 2024-05-20T18:37:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.213+0000 7f6582d92640 1 -- 172.21.15.5:0/1760017787 learned_addr learned my addr 172.21.15.5:0/1760017787 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:46.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.214+0000 7f658158f640 1 -- 172.21.15.5:0/1760017787 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4180366181 0 0) 0x7f657c0d1ac0 con 0x7f657c0d10e0 2024-05-20T18:37:46.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.214+0000 7f6582d92640 1 -- 172.21.15.5:0/1760017787 >> v2:172.21.15.63:3300/0 conn(0x7f657c0cf850 msgr2=0x7f657c0b6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:46.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.214+0000 7f6582d92640 1 --2- 172.21.15.5:0/1760017787 >> v2:172.21.15.63:3300/0 conn(0x7f657c0cf850 0x7f657c0b6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.214+0000 7f6582d92640 1 -- 172.21.15.5:0/1760017787 >> v1:172.21.15.63:6789/0 conn(0x7f657c0d10e0 legacy=0x7f657c0cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:46.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.214+0000 7f6582d92640 1 -- 172.21.15.5:0/1760017787 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f657c0d2360 con 0x7f657c0d14f0 2024-05-20T18:37:46.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.215+0000 7f6582d92640 1 --2- 172.21.15.5:0/1760017787 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f657c0d14f0 0x7f657c0b6740 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6574009e50 tx=0x7f657402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40d010f8b4d55a69 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:46.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.215+0000 7f658158f640 1 -- 172.21.15.5:0/1760017787 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6574036070 con 0x7f657c0d14f0 2024-05-20T18:37:46.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.215+0000 7f658158f640 1 -- 172.21.15.5:0/1760017787 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6574031600 con 0x7f657c0d14f0 2024-05-20T18:37:46.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.215+0000 7f658158f640 1 -- 172.21.15.5:0/1760017787 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65740319d0 con 0x7f657c0d14f0 2024-05-20T18:37:46.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.215+0000 7f65847f5640 1 -- 172.21.15.5:0/1760017787 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f657c0d14f0 msgr2=0x7f657c0b6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:46.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.215+0000 7f65847f5640 1 --2- 172.21.15.5:0/1760017787 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f657c0d14f0 0x7f657c0b6740 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6574009e50 tx=0x7f657402a710 comp rx=0 tx=0).stop 2024-05-20T18:37:46.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.216+0000 7f65847f5640 1 -- 172.21.15.5:0/1760017787 shutdown_connections 2024-05-20T18:37:46.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.216+0000 7f65847f5640 1 --2- 172.21.15.5:0/1760017787 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f657c0d14f0 0x7f657c0b6740 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.216+0000 7f65847f5640 1 --2- 172.21.15.5:0/1760017787 >> v2:172.21.15.63:3300/0 conn(0x7f657c0cf850 0x7f657c0b6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.216+0000 7f65847f5640 1 -- 172.21.15.5:0/1760017787 >> 172.21.15.5:0/1760017787 conn(0x7f657c0b44d0 msgr2=0x7f657c0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:46.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.216+0000 7f65847f5640 1 -- 172.21.15.5:0/1760017787 shutdown_connections 2024-05-20T18:37:46.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.216+0000 7f65847f5640 1 -- 172.21.15.5:0/1760017787 wait complete. 2024-05-20T18:37:46.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.217+0000 7f65847f5640 1 Processor -- start 2024-05-20T18:37:46.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.217+0000 7f65847f5640 1 -- start start 2024-05-20T18:37:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.219+0000 7f65847f5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f657c0cf850 0x7f657c1f2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.219+0000 7f65847f5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f657c0d10e0 0x7f657c15b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.219+0000 7f65847f5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f657c0d14f0 0x7f657c15e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.219+0000 7f65847f5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f657c0b8440 con 0x7f657c0d14f0 2024-05-20T18:37:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.219+0000 7f65847f5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f657c0b82c0 con 0x7f657c0cf850 2024-05-20T18:37:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.219+0000 7f65847f5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f657c0709f0 con 0x7f657c0d10e0 2024-05-20T18:37:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.219+0000 7f6582d92640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f657c0d14f0 0x7f657c15e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.219+0000 7f6581d90640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f657c0d10e0 0x7f657c15b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.219+0000 7f6582d92640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f657c0d14f0 0x7f657c15e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35532/0 (socket says 172.21.15.5:35532) 2024-05-20T18:37:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.219+0000 7f6582591640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f657c0cf850 0x7f657c1f2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.219+0000 7f6582d92640 1 -- 172.21.15.5:0/3969832496 learned_addr learned my addr 172.21.15.5:0/3969832496 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.219+0000 7f6582d92640 1 -- 172.21.15.5:0/3969832496 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f657c0d10e0 msgr2=0x7f657c15b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.219+0000 7f6582d92640 1 --2- 172.21.15.5:0/3969832496 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f657c0d10e0 0x7f657c15b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.219+0000 7f6582d92640 1 -- 172.21.15.5:0/3969832496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f657c0cf850 msgr2=0x7f657c1f2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:46.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.219+0000 7f6582d92640 1 --2- 172.21.15.5:0/3969832496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f657c0cf850 0x7f657c1f2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.219+0000 7f6582d92640 1 -- 172.21.15.5:0/3969832496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f657c0a98d0 con 0x7f657c0d14f0 2024-05-20T18:37:46.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.219+0000 7f6582d92640 1 --2- 172.21.15.5:0/3969832496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f657c0d14f0 0x7f657c15e7c0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f6574009e50 tx=0x7f6574004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:46.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.220+0000 7f6572ffd640 1 -- 172.21.15.5:0/3969832496 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6574041020 con 0x7f657c0d14f0 2024-05-20T18:37:46.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.220+0000 7f6572ffd640 1 -- 172.21.15.5:0/3969832496 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6574037040 con 0x7f657c0d14f0 2024-05-20T18:37:46.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.220+0000 7f65847f5640 1 -- 172.21.15.5:0/3969832496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f657c0a9bb0 con 0x7f657c0d14f0 2024-05-20T18:37:46.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.220+0000 7f6572ffd640 1 -- 172.21.15.5:0/3969832496 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6574036050 con 0x7f657c0d14f0 2024-05-20T18:37:46.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.220+0000 7f65847f5640 1 -- 172.21.15.5:0/3969832496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f657c15ed00 con 0x7f657c0d14f0 2024-05-20T18:37:46.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.222+0000 7f6572ffd640 1 -- 172.21.15.5:0/3969832496 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6574007c50 con 0x7f657c0d14f0 2024-05-20T18:37:46.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.222+0000 7f6572ffd640 1 --2- 172.21.15.5:0/3969832496 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f656006be60 0x7f656006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:46.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.222+0000 7f6572ffd640 1 -- 172.21.15.5:0/3969832496 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f65740ab650 con 0x7f657c0d14f0 2024-05-20T18:37:46.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.222+0000 7f6582591640 1 --2- 172.21.15.5:0/3969832496 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f656006be60 0x7f656006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:46.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.223+0000 7f6582591640 1 --2- 172.21.15.5:0/3969832496 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f656006be60 0x7f656006e370 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f6578000f60 tx=0x7f6578009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:46.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.223+0000 7f658b772740 1 Processor -- start 2024-05-20T18:37:46.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.223+0000 7f658b772740 1 -- start start 2024-05-20T18:37:46.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.225+0000 7f658b772740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d874bd9270 0x55d874fa4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:46.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.225+0000 7f658b772740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d874bb03b0 0x55d874fa10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:46.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.225+0000 7f658b772740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d874bb8590 0x55d874f7a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:46.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.225+0000 7f658b772740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d874edd4d0 con 0x55d874bb8590 2024-05-20T18:37:46.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.225+0000 7f658b772740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d874e853e0 con 0x55d874bb03b0 2024-05-20T18:37:46.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.225+0000 7f658b772740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55d874ed9490 con 0x55d874bd9270 2024-05-20T18:37:46.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.225+0000 7f6582d92640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d874bb8590 0x55d874f7a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:46.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.225+0000 7f6582d92640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d874bb8590 0x55d874f7a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35540/0 (socket says 172.21.15.5:35540) 2024-05-20T18:37:46.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.225+0000 7f6582d92640 1 -- 172.21.15.5:0/3901424006 learned_addr learned my addr 172.21.15.5:0/3901424006 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:46.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.225+0000 7f6581d90640 1 --2- 172.21.15.5:0/3901424006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d874bb03b0 0x55d874fa10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:46.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.225+0000 7f6582591640 1 --2- 172.21.15.5:0/3901424006 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d874bd9270 0x55d874fa4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:46.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.225+0000 7f6582d92640 1 -- 172.21.15.5:0/3901424006 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d874bd9270 msgr2=0x55d874fa4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:46.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.225+0000 7f6582d92640 1 --2- 172.21.15.5:0/3901424006 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d874bd9270 0x55d874fa4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.225+0000 7f6582d92640 1 -- 172.21.15.5:0/3901424006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d874bb03b0 msgr2=0x55d874fa10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:46.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.225+0000 7f6582d92640 1 --2- 172.21.15.5:0/3901424006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d874bb03b0 0x55d874fa10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.225+0000 7f6582d92640 1 -- 172.21.15.5:0/3901424006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d874bf12d0 con 0x55d874bb8590 2024-05-20T18:37:46.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.225+0000 7f6582d92640 1 --2- 172.21.15.5:0/3901424006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d874bb8590 0x55d874f7a360 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f6574034040 tx=0x7f6574079540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a60dd61b14b8e3a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:46.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.226+0000 7f655ffff640 1 -- 172.21.15.5:0/3901424006 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6574031600 con 0x55d874bb8590 2024-05-20T18:37:46.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.226+0000 7f655ffff640 1 -- 172.21.15.5:0/3901424006 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6574079770 con 0x55d874bb8590 2024-05-20T18:37:46.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.226+0000 7f655ffff640 1 -- 172.21.15.5:0/3901424006 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657407da70 con 0x55d874bb8590 2024-05-20T18:37:46.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.226+0000 7f658b772740 1 -- 172.21.15.5:0/3901424006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d874bb8590 msgr2=0x55d874f7a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:46.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.226+0000 7f658b772740 1 --2- 172.21.15.5:0/3901424006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d874bb8590 0x55d874f7a360 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f6574034040 tx=0x7f6574079540 comp rx=0 tx=0).stop 2024-05-20T18:37:46.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.226+0000 7f658b772740 1 -- 172.21.15.5:0/3901424006 shutdown_connections 2024-05-20T18:37:46.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.226+0000 7f658b772740 1 --2- 172.21.15.5:0/3901424006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d874bb8590 0x55d874f7a360 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.226+0000 7f658b772740 1 --2- 172.21.15.5:0/3901424006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d874bb03b0 0x55d874fa10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.226+0000 7f658b772740 1 --2- 172.21.15.5:0/3901424006 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d874bd9270 0x55d874fa4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.226+0000 7f658b772740 1 -- 172.21.15.5:0/3901424006 >> 172.21.15.5:0/3901424006 conn(0x55d874bcd3c0 msgr2=0x55d874f79af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:46.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.227+0000 7f658b772740 1 -- 172.21.15.5:0/3901424006 shutdown_connections 2024-05-20T18:37:46.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.227+0000 7f658b772740 1 -- 172.21.15.5:0/3901424006 wait complete. 2024-05-20T18:37:46.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.227+0000 7f658b772740 1 -- start start 2024-05-20T18:37:46.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.228+0000 7f658b772740 1 Processor -- start 2024-05-20T18:37:46.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.229+0000 7f658b772740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d874bb03b0 0x55d874fbbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:46.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.229+0000 7f658b772740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d874bb8590 0x55d874fa2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:46.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.229+0000 7f658b772740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d874bd9270 0x55d874fa3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:46.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.229+0000 7f658b772740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d874edd4d0 con 0x55d874bb03b0 2024-05-20T18:37:46.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.229+0000 7f658b772740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d874e988e0 con 0x55d874bb8590 2024-05-20T18:37:46.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.229+0000 7f658b772740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55d874fa7ad0 con 0x55d874bd9270 2024-05-20T18:37:46.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.229+0000 7f658b772740 10 client.0 _ll_register_callbacks cb 0x55d874e659c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:37:46.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.229+0000 7f6582591640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d874bb03b0 0x55d874fbbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:46.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.229+0000 7f6582591640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d874bb03b0 0x55d874fbbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35542/0 (socket says 172.21.15.5:35542) 2024-05-20T18:37:46.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.229+0000 7f6582591640 1 -- 172.21.15.5:0/2159946322 learned_addr learned my addr 172.21.15.5:0/2159946322 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:46.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.229+0000 7f6581d90640 1 --2- 172.21.15.5:0/2159946322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d874bb8590 0x55d874fa2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:46.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.229+0000 7f6582d92640 1 --2- 172.21.15.5:0/2159946322 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d874bd9270 0x55d874fa3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.229+0000 7f6582591640 1 -- 172.21.15.5:0/2159946322 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d874bd9270 msgr2=0x55d874fa3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.229+0000 7f6582591640 1 --2- 172.21.15.5:0/2159946322 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d874bd9270 0x55d874fa3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.229+0000 7f6582591640 1 -- 172.21.15.5:0/2159946322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d874bb8590 msgr2=0x55d874fa2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.229+0000 7f6582591640 1 --2- 172.21.15.5:0/2159946322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d874bb8590 0x55d874fa2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.229+0000 7f6582591640 1 -- 172.21.15.5:0/2159946322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d874cbb6b0 con 0x55d874bb03b0 2024-05-20T18:37:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.229+0000 7f6581d90640 1 --2- 172.21.15.5:0/2159946322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d874bb8590 0x55d874fa2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.230+0000 7f6582591640 1 --2- 172.21.15.5:0/2159946322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d874bb03b0 0x55d874fbbe30 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f6578014700 tx=0x7f6578001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3153ffbe3e5dbda server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.230+0000 7f655d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:37:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.230+0000 7f655d7fa640 1 -- 172.21.15.5:0/2159946322 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6578025020 con 0x55d874bb03b0 2024-05-20T18:37:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.230+0000 7f655d7fa640 1 -- 172.21.15.5:0/2159946322 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f6578013700 con 0x55d874bb03b0 2024-05-20T18:37:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.230+0000 7f658b772740 1 -- 172.21.15.5:0/2159946322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6574079770 con 0x55d874bb03b0 2024-05-20T18:37:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.230+0000 7f655d7fa640 1 -- 172.21.15.5:0/2159946322 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) ==== 24+0+0 (secure 0 0 0) 0x7f657802f050 con 0x55d874bb03b0 2024-05-20T18:37:46.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.230+0000 7f658b772740 10 client.9528 fetch_fsmap learned FSMap version 111 2024-05-20T18:37:46.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.230+0000 7f658b772740 1 -- 172.21.15.5:0/2159946322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) -- 0x55d874fa7810 con 0x55d874bb03b0 2024-05-20T18:37:46.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.230+0000 7f655d7fa640 1 -- 172.21.15.5:0/2159946322 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6574079770 con 0x55d874bb03b0 2024-05-20T18:37:46.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.230+0000 7f655d7fa640 1 -- 172.21.15.5:0/2159946322 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 111) ==== 3494+0+0 (secure 0 0 0) 0x7f6578032020 con 0x55d874bb03b0 2024-05-20T18:37:46.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.230+0000 7f658b772740 10 client.9528 fetch_fsmap finished waiting for FSMap version 111 2024-05-20T18:37:46.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.230+0000 7f658b772740 10 client.9528 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-20T18:37:46.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.230+0000 7f658b772740 20 client.9528 populate_metadata read hostname 'smithi005' 2024-05-20T18:37:46.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.230+0000 7f658b772740 1 --2- 172.21.15.5:0/2159946322 >> [v2:172.21.15.5:6835/2067544478,v1:172.21.15.5:6837/2067544478] conn(0x55d874b97e40 0x55d874fa5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:46.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.230+0000 7f658b772740 4 client.9528 mds_command: new command op to 9444 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-20T18:37:46.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.230+0000 7f658b772740 1 -- 172.21.15.5:0/2159946322 --> [v2:172.21.15.5:6835/2067544478,v1:172.21.15.5:6837/2067544478] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d874eeac00 con 0x55d874b97e40 2024-05-20T18:37:46.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.230+0000 7f6581d90640 1 --2- 172.21.15.5:0/2159946322 >> [v2:172.21.15.5:6835/2067544478,v1:172.21.15.5:6837/2067544478] conn(0x55d874b97e40 0x55d874fa5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:46.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.231+0000 7f655d7fa640 10 client.9528 ms_handle_connect on v2:172.21.15.5:6835/2067544478 2024-05-20T18:37:46.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.231+0000 7f6581d90640 1 --2- 172.21.15.5:0/2159946322 >> [v2:172.21.15.5:6835/2067544478,v1:172.21.15.5:6837/2067544478] conn(0x55d874b97e40 0x55d874fa5810 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d454108a64e3058 server_cookie=315e4e02872e8a29 in_seq=0 out_seq=0 2024-05-20T18:37:46.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.231+0000 7f655d7fa640 1 -- 172.21.15.5:0/2159946322 <== mds.0 v2:172.21.15.5:6835/2067544478 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d874eeac00 con 0x55d874b97e40 2024-05-20T18:37:46.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.231+0000 7f655d7fa640 10 client.9528 handle_command_reply: tid=0 2024-05-20T18:37:46.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.231+0000 7f658b772740 1 client.9528 shutdown 2024-05-20T18:37:46.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.232+0000 7f658b772740 1 -- 172.21.15.5:0/2159946322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d874bb03b0 msgr2=0x55d874fbbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:46.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.232+0000 7f658b772740 1 --2- 172.21.15.5:0/2159946322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d874bb03b0 0x55d874fbbe30 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f6578014700 tx=0x7f6578001080 comp rx=0 tx=0).stop 2024-05-20T18:37:46.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.232+0000 7f658b772740 1 -- 172.21.15.5:0/2159946322 shutdown_connections 2024-05-20T18:37:46.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.232+0000 7f658b772740 1 --2- 172.21.15.5:0/2159946322 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d874bd9270 0x55d874fa3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.232+0000 7f658b772740 1 --2- 172.21.15.5:0/2159946322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d874bb8590 0x55d874fa2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.232+0000 7f658b772740 1 --2- 172.21.15.5:0/2159946322 >> [v2:172.21.15.5:6835/2067544478,v1:172.21.15.5:6837/2067544478] conn(0x55d874b97e40 0x55d874fa5810 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.232+0000 7f658b772740 1 --2- 172.21.15.5:0/2159946322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d874bb03b0 0x55d874fbbe30 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.232+0000 7f655d7fa640 0 client.9528 ms_handle_reset on v2:172.21.15.5:6835/2067544478 2024-05-20T18:37:46.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.232+0000 7f658b772740 1 -- 172.21.15.5:0/2159946322 >> 172.21.15.5:0/2159946322 conn(0x55d874bcd3c0 msgr2=0x55d874f79af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:46.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.232+0000 7f658b772740 1 -- 172.21.15.5:0/2159946322 shutdown_connections 2024-05-20T18:37:46.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.232+0000 7f658b772740 1 -- 172.21.15.5:0/2159946322 wait complete. 2024-05-20T18:37:46.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.232+0000 7f658b772740 20 client.9528 trim_cache size 0 max 16384 2024-05-20T18:37:46.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.251+0000 7f658b772740 1 Processor -- start 2024-05-20T18:37:46.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.251+0000 7f658b772740 1 -- start start 2024-05-20T18:37:46.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.252+0000 7f658b772740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d874bb03b0 0x55d874fa5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:46.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.252+0000 7f658b772740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d874bb8590 0x55d874fa5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:46.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.252+0000 7f658b772740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d874bd9270 0x55d874f7aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:46.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.252+0000 7f658b772740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f657c1ff1d0 con 0x55d874bd9270 2024-05-20T18:37:46.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.252+0000 7f658b772740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f657c15e4f0 con 0x55d874bb03b0 2024-05-20T18:37:46.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.252+0000 7f658b772740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f657c1feb80 con 0x55d874bb8590 2024-05-20T18:37:46.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.253+0000 7f6582d92640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d874bd9270 0x55d874f7aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:46.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.253+0000 7f6582d92640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d874bd9270 0x55d874f7aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35556/0 (socket says 172.21.15.5:35556) 2024-05-20T18:37:46.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.253+0000 7f6582d92640 1 -- 172.21.15.5:0/150487272 learned_addr learned my addr 172.21.15.5:0/150487272 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:46.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.253+0000 7f6581d90640 1 --2- 172.21.15.5:0/150487272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d874bb8590 0x55d874fa5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:46.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.253+0000 7f6582591640 1 --2- 172.21.15.5:0/150487272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d874bb03b0 0x55d874fa5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:46.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.253+0000 7f6582d92640 1 -- 172.21.15.5:0/150487272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d874bb8590 msgr2=0x55d874fa5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:46.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.253+0000 7f6582d92640 1 --2- 172.21.15.5:0/150487272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d874bb8590 0x55d874fa5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.253+0000 7f6582d92640 1 -- 172.21.15.5:0/150487272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d874bb03b0 msgr2=0x55d874fa5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:46.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.253+0000 7f6582d92640 1 --2- 172.21.15.5:0/150487272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d874bb03b0 0x55d874fa5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.253+0000 7f6582d92640 1 -- 172.21.15.5:0/150487272 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d874f784b0 con 0x55d874bd9270 2024-05-20T18:37:46.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.253+0000 7f6581d90640 1 --2- 172.21.15.5:0/150487272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d874bb8590 0x55d874fa5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:46.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.253+0000 7f6582d92640 1 --2- 172.21.15.5:0/150487272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d874bd9270 0x55d874f7aae0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f6574081d70 tx=0x7f6574047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=993675b127fa4283 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:46.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.253+0000 7f655f7fe640 1 -- 172.21.15.5:0/150487272 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6574031600 con 0x55d874bd9270 2024-05-20T18:37:46.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.253+0000 7f655f7fe640 1 -- 172.21.15.5:0/150487272 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6574079770 con 0x55d874bd9270 2024-05-20T18:37:46.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.253+0000 7f655f7fe640 1 -- 172.21.15.5:0/150487272 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65740822a0 con 0x55d874bd9270 2024-05-20T18:37:46.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.254+0000 7f658b772740 1 -- 172.21.15.5:0/150487272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d874bd9270 msgr2=0x55d874f7aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:46.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.254+0000 7f658b772740 1 --2- 172.21.15.5:0/150487272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d874bd9270 0x55d874f7aae0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f6574081d70 tx=0x7f6574047000 comp rx=0 tx=0).stop 2024-05-20T18:37:46.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.254+0000 7f658b772740 1 -- 172.21.15.5:0/150487272 shutdown_connections 2024-05-20T18:37:46.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.254+0000 7f658b772740 1 --2- 172.21.15.5:0/150487272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d874bd9270 0x55d874f7aae0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.254+0000 7f658b772740 1 --2- 172.21.15.5:0/150487272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d874bb8590 0x55d874fa5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.254+0000 7f658b772740 1 --2- 172.21.15.5:0/150487272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d874bb03b0 0x55d874fa5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.254+0000 7f658b772740 1 -- 172.21.15.5:0/150487272 >> 172.21.15.5:0/150487272 conn(0x55d874bcd3c0 msgr2=0x55d874fbbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:46.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.254+0000 7f658b772740 1 -- 172.21.15.5:0/150487272 shutdown_connections 2024-05-20T18:37:46.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.254+0000 7f658b772740 1 -- 172.21.15.5:0/150487272 wait complete. 2024-05-20T18:37:46.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.255+0000 7f658b772740 1 -- start start 2024-05-20T18:37:46.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.255+0000 7f658b772740 1 Processor -- start 2024-05-20T18:37:46.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.256+0000 7f658b772740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d874bb03b0 0x55d874f761a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:46.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.256+0000 7f658b772740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d874bb8590 0x55d874fbe690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:46.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.256+0000 7f658b772740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d874bd9270 0x55d874fa5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:46.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.256+0000 7f658b772740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d874f9e960 con 0x55d874bb8590 2024-05-20T18:37:46.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.256+0000 7f658b772740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d874fbc3c0 con 0x55d874bd9270 2024-05-20T18:37:46.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.256+0000 7f658b772740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55d874fa6fc0 con 0x55d874bb03b0 2024-05-20T18:37:46.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.256+0000 7f658b772740 10 client.0 _ll_register_callbacks cb 0x7f6578001de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-20T18:37:46.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.256+0000 7f6581d90640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d874bb8590 0x55d874fbe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:46.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.256+0000 7f6581d90640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d874bb8590 0x55d874fbe690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35562/0 (socket says 172.21.15.5:35562) 2024-05-20T18:37:46.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.256+0000 7f6581d90640 1 -- 172.21.15.5:0/1863554081 learned_addr learned my addr 172.21.15.5:0/1863554081 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:46.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.257+0000 7f6582d92640 1 --2- 172.21.15.5:0/1863554081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d874bd9270 0x55d874fa5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:46.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.257+0000 7f6582591640 1 --2- 172.21.15.5:0/1863554081 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d874bb03b0 0x55d874f761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:46.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.257+0000 7f6581d90640 1 -- 172.21.15.5:0/1863554081 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d874bb03b0 msgr2=0x55d874f761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:46.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.257+0000 7f6581d90640 1 --2- 172.21.15.5:0/1863554081 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d874bb03b0 0x55d874f761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.257+0000 7f6581d90640 1 -- 172.21.15.5:0/1863554081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d874bd9270 msgr2=0x55d874fa5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:46.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.257+0000 7f6581d90640 1 --2- 172.21.15.5:0/1863554081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d874bd9270 0x55d874fa5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.257+0000 7f6581d90640 1 -- 172.21.15.5:0/1863554081 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d874fbbc80 con 0x55d874bb8590 2024-05-20T18:37:46.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.257+0000 7f6581d90640 1 --2- 172.21.15.5:0/1863554081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d874bb8590 0x55d874fbe690 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f657c0c5e00 tx=0x7f656c016720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb31778dec7d0640 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:46.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.257+0000 7f655d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-05-20T18:37:46.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.257+0000 7f655d7fa640 1 -- 172.21.15.5:0/1863554081 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c004300 con 0x55d874bb8590 2024-05-20T18:37:46.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.257+0000 7f655d7fa640 1 -- 172.21.15.5:0/1863554081 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f656c01f290 con 0x55d874bb8590 2024-05-20T18:37:46.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.257+0000 7f658b772740 1 -- 172.21.15.5:0/1863554081 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6574079770 con 0x55d874bb8590 2024-05-20T18:37:46.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.257+0000 7f655d7fa640 1 -- 172.21.15.5:0/1863554081 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) ==== 24+0+0 (secure 0 0 0) 0x7f656c02e050 con 0x55d874bb8590 2024-05-20T18:37:46.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.257+0000 7f658b772740 10 client.9534 fetch_fsmap learned FSMap version 111 2024-05-20T18:37:46.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.257+0000 7f658b772740 1 -- 172.21.15.5:0/1863554081 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) -- 0x55d874fa3010 con 0x55d874bb8590 2024-05-20T18:37:46.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.258+0000 7f655d7fa640 1 -- 172.21.15.5:0/1863554081 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6574079770 con 0x55d874bb8590 2024-05-20T18:37:46.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.258+0000 7f655d7fa640 1 -- 172.21.15.5:0/1863554081 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 111) ==== 3494+0+0 (secure 0 0 0) 0x7f656c031020 con 0x55d874bb8590 2024-05-20T18:37:46.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.258+0000 7f658b772740 10 client.9534 fetch_fsmap finished waiting for FSMap version 111 2024-05-20T18:37:46.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.258+0000 7f658b772740 10 client.9534 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-20T18:37:46.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.258+0000 7f658b772740 20 client.9534 populate_metadata read hostname 'smithi005' 2024-05-20T18:37:46.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.258+0000 7f658b772740 1 --2- 172.21.15.5:0/1863554081 >> [v2:172.21.15.5:6835/2067544478,v1:172.21.15.5:6837/2067544478] conn(0x55d87503b3a0 0x55d87503b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:46.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.258+0000 7f658b772740 4 client.9534 mds_command: new command op to 9444 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-20T18:37:46.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.258+0000 7f658b772740 1 -- 172.21.15.5:0/1863554081 --> [v2:172.21.15.5:6835/2067544478,v1:172.21.15.5:6837/2067544478] -- command(tid 0: {"prefix": "status"}) -- 0x55d874ee4f50 con 0x55d87503b3a0 2024-05-20T18:37:46.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.258+0000 7f6582591640 1 --2- 172.21.15.5:0/1863554081 >> [v2:172.21.15.5:6835/2067544478,v1:172.21.15.5:6837/2067544478] conn(0x55d87503b3a0 0x55d87503b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:46.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.258+0000 7f655d7fa640 10 client.9534 ms_handle_connect on v2:172.21.15.5:6835/2067544478 2024-05-20T18:37:46.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.258+0000 7f6582591640 1 --2- 172.21.15.5:0/1863554081 >> [v2:172.21.15.5:6835/2067544478,v1:172.21.15.5:6837/2067544478] conn(0x55d87503b3a0 0x55d87503b7b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f548020abfdfc45 server_cookie=6f288cf4e1dbd816 in_seq=0 out_seq=0 2024-05-20T18:37:46.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.258+0000 7f655d7fa640 1 -- 172.21.15.5:0/1863554081 <== mds.0 v2:172.21.15.5:6835/2067544478 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55d874ee4f50 con 0x55d87503b3a0 2024-05-20T18:37:46.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.258+0000 7f655d7fa640 10 client.9534 handle_command_reply: tid=0 2024-05-20T18:37:46.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.258+0000 7f658b772740 1 client.9534 shutdown 2024-05-20T18:37:46.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.258+0000 7f658b772740 1 -- 172.21.15.5:0/1863554081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d874bb8590 msgr2=0x55d874fbe690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:46.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.259+0000 7f658b772740 1 --2- 172.21.15.5:0/1863554081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d874bb8590 0x55d874fbe690 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f657c0c5e00 tx=0x7f656c016720 comp rx=0 tx=0).stop 2024-05-20T18:37:46.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.259+0000 7f658b772740 1 -- 172.21.15.5:0/1863554081 shutdown_connections 2024-05-20T18:37:46.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.259+0000 7f658b772740 1 --2- 172.21.15.5:0/1863554081 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d874bd9270 0x55d874fa5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.259+0000 7f658b772740 1 --2- 172.21.15.5:0/1863554081 >> [v2:172.21.15.5:6835/2067544478,v1:172.21.15.5:6837/2067544478] conn(0x55d87503b3a0 0x55d87503b7b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.259+0000 7f658b772740 1 --2- 172.21.15.5:0/1863554081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d874bb8590 0x55d874fbe690 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.259+0000 7f658b772740 1 --2- 172.21.15.5:0/1863554081 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d874bb03b0 0x55d874f761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.259+0000 7f655d7fa640 0 client.9534 ms_handle_reset on v2:172.21.15.5:6835/2067544478 2024-05-20T18:37:46.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.259+0000 7f658b772740 1 -- 172.21.15.5:0/1863554081 >> 172.21.15.5:0/1863554081 conn(0x55d874bcd3c0 msgr2=0x55d874fc2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:46.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.259+0000 7f658b772740 1 -- 172.21.15.5:0/1863554081 shutdown_connections 2024-05-20T18:37:46.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.259+0000 7f658b772740 1 -- 172.21.15.5:0/1863554081 wait complete. 2024-05-20T18:37:46.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.259+0000 7f658b772740 20 client.9534 trim_cache size 0 max 16384 2024-05-20T18:37:46.268 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:37:46.268 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "399ab250-fd3c-47fa-a1c8-25e8149c9cac", 2024-05-20T18:37:46.268 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-05-20T18:37:46.268 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 9444, 2024-05-20T18:37:46.268 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-05-20T18:37:46.269 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-05-20T18:37:46.269 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-05-20T18:37:46.269 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 4.5462326280000003, 2024-05-20T18:37:46.269 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 111, 2024-05-20T18:37:46.269 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 121, 2024-05-20T18:37:46.269 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 121, 2024-05-20T18:37:46.269 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 6.4951172640000001 2024-05-20T18:37:46.269 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:37:46.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.259+0000 7f65847f5640 1 -- 172.21.15.5:0/3969832496 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f656006be60 msgr2=0x7f656006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:46.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.259+0000 7f65847f5640 1 --2- 172.21.15.5:0/3969832496 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f656006be60 0x7f656006e370 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f6578000f60 tx=0x7f6578009040 comp rx=0 tx=0).stop 2024-05-20T18:37:46.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.259+0000 7f65847f5640 1 -- 172.21.15.5:0/3969832496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f657c0d14f0 msgr2=0x7f657c15e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:46.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.259+0000 7f65847f5640 1 --2- 172.21.15.5:0/3969832496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f657c0d14f0 0x7f657c15e7c0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f6574009e50 tx=0x7f6574004a40 comp rx=0 tx=0).stop 2024-05-20T18:37:46.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.259+0000 7f65847f5640 1 -- 172.21.15.5:0/3969832496 shutdown_connections 2024-05-20T18:37:46.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.259+0000 7f65847f5640 1 --2- 172.21.15.5:0/3969832496 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f656006be60 0x7f656006e370 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.259+0000 7f65847f5640 1 --2- 172.21.15.5:0/3969832496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f657c0d14f0 0x7f657c15e7c0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.259+0000 7f65847f5640 1 --2- 172.21.15.5:0/3969832496 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f657c0d10e0 0x7f657c15b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.259+0000 7f65847f5640 1 --2- 172.21.15.5:0/3969832496 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f657c0cf850 0x7f657c1f2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.259+0000 7f65847f5640 1 -- 172.21.15.5:0/3969832496 >> 172.21.15.5:0/3969832496 conn(0x7f657c0b44d0 msgr2=0x7f657c0b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:46.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.260+0000 7f65847f5640 1 -- 172.21.15.5:0/3969832496 shutdown_connections 2024-05-20T18:37:46.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.260+0000 7f65847f5640 1 -- 172.21.15.5:0/3969832496 wait complete. 2024-05-20T18:37:46.272 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-20T18:37:46.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.385+0000 7f0172565640 1 Processor -- start 2024-05-20T18:37:46.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.385+0000 7f0172565640 1 -- start start 2024-05-20T18:37:46.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.387+0000 7f0172565640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f016c15d060 0x7f016c0b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:46.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.387+0000 7f0172565640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f016c0b5720 0x7f016c06f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:46.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.387+0000 7f0172565640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f016c0b5b30 con 0x7f016c0b47b0 2024-05-20T18:37:46.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.387+0000 7f0172565640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f016c0b39f0 con 0x7f016c0b5720 2024-05-20T18:37:46.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.387+0000 7f0172565640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f016c0b3cd0 con 0x7f016c15d060 2024-05-20T18:37:46.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.387+0000 7f0170b02640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f016c0b5720 0x7f016c06f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:46.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.387+0000 7f0170b02640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f016c0b5720 0x7f016c06f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35568/0 (socket says 172.21.15.5:35568) 2024-05-20T18:37:46.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.387+0000 7f0170b02640 1 -- 172.21.15.5:0/2200862121 learned_addr learned my addr 172.21.15.5:0/2200862121 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:46.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.387+0000 7f016b7fe640 1 --2- 172.21.15.5:0/2200862121 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f016c15d060 0x7f016c0b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:46.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.387+0000 7f0170b02640 1 -- 172.21.15.5:0/2200862121 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f016c15d060 msgr2=0x7f016c0b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:46.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.387+0000 7f0170b02640 1 --2- 172.21.15.5:0/2200862121 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f016c15d060 0x7f016c0b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.387+0000 7f0170b02640 1 -- 172.21.15.5:0/2200862121 >> v1:172.21.15.63:6789/0 conn(0x7f016c0b47b0 legacy=0x7f016c0afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:46.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.387+0000 7f0170b02640 1 -- 172.21.15.5:0/2200862121 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016c0b3fb0 con 0x7f016c0b5720 2024-05-20T18:37:46.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.388+0000 7f0170b02640 1 --2- 172.21.15.5:0/2200862121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f016c0b5720 0x7f016c06f3d0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f0164009e50 tx=0x7f016402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2accdca73448e71 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:46.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.388+0000 7f016affd640 1 -- 172.21.15.5:0/2200862121 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164036070 con 0x7f016c0b5720 2024-05-20T18:37:46.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.388+0000 7f016affd640 1 -- 172.21.15.5:0/2200862121 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0164031600 con 0x7f016c0b5720 2024-05-20T18:37:46.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.388+0000 7f016affd640 1 -- 172.21.15.5:0/2200862121 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01640319d0 con 0x7f016c0b5720 2024-05-20T18:37:46.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.389+0000 7f0172565640 1 -- 172.21.15.5:0/2200862121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f016c0b5720 msgr2=0x7f016c06f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:46.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.389+0000 7f0172565640 1 --2- 172.21.15.5:0/2200862121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f016c0b5720 0x7f016c06f3d0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f0164009e50 tx=0x7f016402a710 comp rx=0 tx=0).stop 2024-05-20T18:37:46.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.389+0000 7f0172565640 1 -- 172.21.15.5:0/2200862121 shutdown_connections 2024-05-20T18:37:46.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.389+0000 7f0172565640 1 --2- 172.21.15.5:0/2200862121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f016c0b5720 0x7f016c06f3d0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.389+0000 7f0172565640 1 --2- 172.21.15.5:0/2200862121 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f016c15d060 0x7f016c0b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.389+0000 7f0172565640 1 -- 172.21.15.5:0/2200862121 >> 172.21.15.5:0/2200862121 conn(0x7f016c0cd2a0 msgr2=0x7f016c0cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:46.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.389+0000 7f0172565640 1 -- 172.21.15.5:0/2200862121 shutdown_connections 2024-05-20T18:37:46.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.389+0000 7f0172565640 1 -- 172.21.15.5:0/2200862121 wait complete. 2024-05-20T18:37:46.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.391+0000 7f0172565640 1 Processor -- start 2024-05-20T18:37:46.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.391+0000 7f0172565640 1 -- start start 2024-05-20T18:37:46.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.392+0000 7f0172565640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f016c0b5720 0x7f016c0b1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:46.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.392+0000 7f0172565640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f016c15d060 0x7f016c157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:46.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.392+0000 7f0172565640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f016c0b1ea0 0x7f016c1581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:46.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.392+0000 7f0172565640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f016c159de0 con 0x7f016c0b1ea0 2024-05-20T18:37:46.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.392+0000 7f0172565640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f016c159c60 con 0x7f016c0b5720 2024-05-20T18:37:46.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.392+0000 7f0172565640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f016c06f910 con 0x7f016c15d060 2024-05-20T18:37:46.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.392+0000 7f0170b02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f016c0b1ea0 0x7f016c1581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:46.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.393+0000 7f0170b02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f016c0b1ea0 0x7f016c1581b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35582/0 (socket says 172.21.15.5:35582) 2024-05-20T18:37:46.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.393+0000 7f0170b02640 1 -- 172.21.15.5:0/1001961529 learned_addr learned my addr 172.21.15.5:0/1001961529 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:46.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.393+0000 7f016bfff640 1 --2- 172.21.15.5:0/1001961529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f016c0b5720 0x7f016c0b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:46.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.393+0000 7f016b7fe640 1 --2- 172.21.15.5:0/1001961529 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f016c15d060 0x7f016c157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:46.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.393+0000 7f0170b02640 1 -- 172.21.15.5:0/1001961529 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f016c15d060 msgr2=0x7f016c157c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:46.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.393+0000 7f0170b02640 1 --2- 172.21.15.5:0/1001961529 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f016c15d060 0x7f016c157c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.393+0000 7f0170b02640 1 -- 172.21.15.5:0/1001961529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f016c0b5720 msgr2=0x7f016c0b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:46.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.393+0000 7f0170b02640 1 --2- 172.21.15.5:0/1001961529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f016c0b5720 0x7f016c0b1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.393+0000 7f0170b02640 1 -- 172.21.15.5:0/1001961529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016c143a20 con 0x7f016c0b1ea0 2024-05-20T18:37:46.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.393+0000 7f0170b02640 1 --2- 172.21.15.5:0/1001961529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f016c0b1ea0 0x7f016c1581b0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f0164009f70 tx=0x7f0164004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:46.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.393+0000 7f0168ff9640 1 -- 172.21.15.5:0/1001961529 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016403f020 con 0x7f016c0b1ea0 2024-05-20T18:37:46.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.393+0000 7f0168ff9640 1 -- 172.21.15.5:0/1001961529 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0164037040 con 0x7f016c0b1ea0 2024-05-20T18:37:46.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.393+0000 7f0172565640 1 -- 172.21.15.5:0/1001961529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f016c143d00 con 0x7f016c0b1ea0 2024-05-20T18:37:46.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.393+0000 7f0168ff9640 1 -- 172.21.15.5:0/1001961529 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164036070 con 0x7f016c0b1ea0 2024-05-20T18:37:46.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.393+0000 7f0172565640 1 -- 172.21.15.5:0/1001961529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f016c1413a0 con 0x7f016c0b1ea0 2024-05-20T18:37:46.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.394+0000 7f0172565640 1 -- 172.21.15.5:0/1001961529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f016c143770 con 0x7f016c0b1ea0 2024-05-20T18:37:46.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.395+0000 7f0168ff9640 1 -- 172.21.15.5:0/1001961529 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0164042020 con 0x7f016c0b1ea0 2024-05-20T18:37:46.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.395+0000 7f0168ff9640 1 --2- 172.21.15.5:0/1001961529 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f015806be30 0x7f015806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:46.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.395+0000 7f0168ff9640 1 -- 172.21.15.5:0/1001961529 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f01640ab590 con 0x7f016c0b1ea0 2024-05-20T18:37:46.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.395+0000 7f016bfff640 1 --2- 172.21.15.5:0/1001961529 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f015806be30 0x7f015806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:46.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.396+0000 7f016bfff640 1 --2- 172.21.15.5:0/1001961529 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f015806be30 0x7f015806e340 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f015c007fd0 tx=0x7f015c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:46.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.400+0000 7f0168ff9640 1 -- 172.21.15.5:0/1001961529 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0164049030 con 0x7f016c0b1ea0 2024-05-20T18:37:46.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.623+0000 7f0172565640 1 -- 172.21.15.5:0/1001961529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f016c075b00 con 0x7f016c0b1ea0 2024-05-20T18:37:46.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.623+0000 7f0168ff9640 1 -- 172.21.15.5:0/1001961529 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v111) ==== 53+0+73 (secure 0 0 0) 0x7f0164079c60 con 0x7f016c0b1ea0 2024-05-20T18:37:46.625 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-20T18:37:46.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.626+0000 7f0172565640 1 -- 172.21.15.5:0/1001961529 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f015806be30 msgr2=0x7f015806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:46.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.626+0000 7f0172565640 1 --2- 172.21.15.5:0/1001961529 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f015806be30 0x7f015806e340 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f015c007fd0 tx=0x7f015c009040 comp rx=0 tx=0).stop 2024-05-20T18:37:46.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.626+0000 7f0172565640 1 -- 172.21.15.5:0/1001961529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f016c0b1ea0 msgr2=0x7f016c1581b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:46.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.626+0000 7f0172565640 1 --2- 172.21.15.5:0/1001961529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f016c0b1ea0 0x7f016c1581b0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f0164009f70 tx=0x7f0164004a40 comp rx=0 tx=0).stop 2024-05-20T18:37:46.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.626+0000 7f0172565640 1 -- 172.21.15.5:0/1001961529 shutdown_connections 2024-05-20T18:37:46.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.626+0000 7f0172565640 1 --2- 172.21.15.5:0/1001961529 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f015806be30 0x7f015806e340 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.626+0000 7f0172565640 1 --2- 172.21.15.5:0/1001961529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f016c0b1ea0 0x7f016c1581b0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.626+0000 7f0172565640 1 --2- 172.21.15.5:0/1001961529 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f016c15d060 0x7f016c157c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.626+0000 7f0172565640 1 --2- 172.21.15.5:0/1001961529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f016c0b5720 0x7f016c0b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:46.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.626+0000 7f0172565640 1 -- 172.21.15.5:0/1001961529 >> 172.21.15.5:0/1001961529 conn(0x7f016c0cd2a0 msgr2=0x7f016c06c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:46.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.626+0000 7f0172565640 1 -- 172.21.15.5:0/1001961529 shutdown_connections 2024-05-20T18:37:46.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46.626+0000 7f0172565640 1 -- 172.21.15.5:0/1001961529 wait complete. 2024-05-20T18:37:46.638 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-20T18:37:46.638 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-20T18:37:46.638 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-05-20T18:37:46.638 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-20T18:37:46.638 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-20T18:37:46.639 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-20T18:37:46.639 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-20T18:37:46.639 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-20T18:37:46.639 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:37:46.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:46.639 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-05-20T18:37:46.656 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-20T18:37:46.656 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-20T18:37:46.656 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-05-20T18:37:46.656 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:37:46.657 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-05-20T18:37:46.657 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:37:46.657 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:37:46.657 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-05-20T18:37:46.657 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-05-20T18:37:46.657 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:37:46.657 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-05-20T18:37:46.657 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-05-20T18:37:46.657 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-20T18:37:46.657 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:37:46.657 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-05-20T18:37:46.657 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-20T18:37:46.657 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42031sec preferred_lft 42031sec 2024-05-20T18:37:46.658 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-05-20T18:37:46.658 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:37:46.658 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-20T18:37:46.658 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:37:46.658 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-05-20T18:37:46.658 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-20T18:37:46.658 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff 2024-05-20T18:37:46.658 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-20T18:37:46.658 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:37:46.658 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::4474:eaff:fe07:4f9a/64 scope link 2024-05-20T18:37:46.658 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:37:46.658 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:37:46.659 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:e4:f4:5e:79:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T18:37:46.659 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ce4:f4ff:fe5e:7906/64 scope link 2024-05-20T18:37:46.659 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:37:46.659 INFO:teuthology.orchestra.run.smithi005.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-20T18:37:46.659 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether da:26:04:0e:95:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T18:37:46.659 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::d826:4ff:fe0e:950a/64 scope link 2024-05-20T18:37:46.659 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-05-20T18:37:46.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:46.659 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-20T18:37:46.722 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-20T18:37:46.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:46.723 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-05-20T18:37:46.781 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-05-20T18:37:46.781 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-20T18:37:46.781 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-20T18:37:46.781 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-20T18:37:46.781 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-20T18:37:46.781 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-20T18:37:46.782 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-05-20T18:37:46.782 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-20T18:37:46.782 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-20T18:37:46.782 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-20T18:37:46.782 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-05-20T18:37:46.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-20T18:37:46.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:46.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:46.911 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-20T18:37:46.927 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-20T18:37:46.927 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-20T18:37:46.927 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:37:46.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:46.928 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-05-20T18:37:46.993 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:37:46.993 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-20T18:37:47.050 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-20T18:37:47.050 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-20T18:37:47.051 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:37:47.106 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-20T18:37:47.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:47.106 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:37:47.201 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:37:47.202 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:37:47.202 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:37:47.213 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:37:47.213 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:37:47.213 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:37:47.213 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:37:47.213 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:37:47.220 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-20T18:37:47.220 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-20T18:37:47.220 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-05-20T18:37:47.220 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-20T18:37:47.220 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-20T18:37:47.220 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790" 2024-05-20T18:37:47.220 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.63:6789,172.21.15.63:6790:/ 2024-05-20T18:37:47.220 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=399ab250-fd3c-47fa-a1c8-25e8149c9cac" will pass to kernel 2024-05-20T18:37:47.221 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-20T18:37:47.221 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-05-20T18:37:47.221 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-20T18:37:47.221 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-05-20T18:37:47.221 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-20T18:37:47.223 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-20T18:37:47.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:47.223 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-20T18:37:47.248 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-20T18:37:47.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-20T18:37:47.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:47.347 INFO:teuthology.orchestra.run.smithi005.stdout:client7565 2024-05-20T18:37:47.347 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:37:47.347 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:37:47.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:47.364 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-20T18:37:47.425 INFO:teuthology.orchestra.run.smithi005.stdout:./ 2024-05-20T18:37:47.425 INFO:teuthology.orchestra.run.smithi005.stdout:./parent 2024-05-20T18:37:47.425 INFO:teuthology.orchestra.run.smithi005.stdout:./parent/flushed 2024-05-20T18:37:47.425 INFO:teuthology.orchestra.run.smithi005.stdout:./parent/flushed/charlie 2024-05-20T18:37:47.425 INFO:teuthology.orchestra.run.smithi005.stdout:./parent/flushed/alpha 2024-05-20T18:37:47.425 INFO:teuthology.orchestra.run.smithi005.stdout:./parent/flushed/bravo 2024-05-20T18:37:47.425 INFO:teuthology.orchestra.run.smithi005.stdout:./parent/unflushed 2024-05-20T18:37:47.425 INFO:teuthology.orchestra.run.smithi005.stdout:./parent/unflushed/jfile 2024-05-20T18:37:47.425 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:37:47.426 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:37:47.426 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:37:47.426 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:37:47.426 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-20T18:37:47.426 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:37:47.532 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-20T18:37:47.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:47.541 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:37:47.542 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:37:47.542 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:37:47.542 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:37:47.542 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-05-20T18:37:47.542 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:37:47.650 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627776 2024-05-20T18:37:47.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:47.659 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:37:47.659 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:37:47.659 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:37:47.659 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:37:47.659 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-05-20T18:37:47.659 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:37:47.768 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627777 2024-05-20T18:37:47.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:47.777 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:37:47.777 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:37:47.777 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:37:47.777 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:37:47.778 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-05-20T18:37:47.778 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:37:47.885 INFO:teuthology.orchestra.run.smithi005.stdout:1099511628280 2024-05-20T18:37:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:47.894 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:37:47.894 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:37:47.894 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:37:47.894 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:37:47.894 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-05-20T18:37:47.894 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:37:48.002 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627778 2024-05-20T18:37:48.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:48.011 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:37:48.011 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:37:48.012 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:37:48.012 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:37:48.012 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-05-20T18:37:48.012 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:37:48.120 INFO:teuthology.orchestra.run.smithi005.stdout:1099511628279 2024-05-20T18:37:48.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:48.129 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:37:48.129 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:37:48.129 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:37:48.129 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:37:48.129 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-05-20T18:37:48.129 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:37:48.236 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627779 2024-05-20T18:37:48.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:48.245 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-05-20T18:37:48.245 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-05-20T18:37:48.245 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-05-20T18:37:48.245 DEBUG:teuthology.orchestra.run.smithi005:> 2024-05-20T18:37:48.245 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-05-20T18:37:48.245 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-05-20T18:37:48.353 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627780 2024-05-20T18:37:48.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-20T18:37:48.362 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-20T18:37:48.362 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T18:37:48.441 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-20T18:37:48.486 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-20T18:37:48.486 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T18:37:48.486 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:37:48.486 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:37:48.503 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:37:48.503 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:37:48.559 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:37:48.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.713+0000 7f1607291640 1 Processor -- start 2024-05-20T18:37:48.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.713+0000 7f1607291640 1 -- start start 2024-05-20T18:37:48.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.715+0000 7f1607291640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f160006d3d0 0x7f160006d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:48.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.715+0000 7f1607291640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f160006dd20 con 0x7f1600162820 2024-05-20T18:37:48.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.715+0000 7f1607291640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16000d0630 con 0x7f16000b8190 2024-05-20T18:37:48.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.715+0000 7f1607291640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f16000d39c0 con 0x7f160006d3d0 2024-05-20T18:37:48.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.715+0000 7f160502d640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1600162820 0x7f16000b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44758/0 (socket says 172.21.15.5:44758) 2024-05-20T18:37:48.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.715+0000 7f160502d640 1 -- 172.21.15.5:0/4013423428 learned_addr learned my addr 172.21.15.5:0/4013423428 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:48.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.715+0000 7f160482c640 1 --2- 172.21.15.5:0/4013423428 >> v2:172.21.15.63:3300/0 conn(0x7f160006d3d0 0x7f160006d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:48.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.715+0000 7f15fffff640 1 -- 172.21.15.5:0/4013423428 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3562211379 0 0) 0x7f15f8001110 con 0x7f1600162820 2024-05-20T18:37:48.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.715+0000 7f15fffff640 1 -- 172.21.15.5:0/4013423428 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15e8003520 con 0x7f1600162820 2024-05-20T18:37:48.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.716+0000 7f15fffff640 1 -- 172.21.15.5:0/4013423428 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1031808387 0 0) 0x7f15e8003520 con 0x7f1600162820 2024-05-20T18:37:48.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.716+0000 7f15fffff640 1 -- 172.21.15.5:0/4013423428 >> v2:172.21.15.63:3300/0 conn(0x7f160006d3d0 msgr2=0x7f160006d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:48.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.716+0000 7f15fffff640 1 --2- 172.21.15.5:0/4013423428 >> v2:172.21.15.63:3300/0 conn(0x7f160006d3d0 0x7f160006d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:48.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.716+0000 7f15fffff640 1 -- 172.21.15.5:0/4013423428 >> v1:172.21.15.63:6789/0 conn(0x7f16000b8190 legacy=0x7f16000b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:48.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.716+0000 7f15fffff640 1 -- 172.21.15.5:0/4013423428 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16000d3ca0 con 0x7f1600162820 2024-05-20T18:37:48.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.716+0000 7f15fffff640 1 -- 172.21.15.5:0/4013423428 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f15f8001840 con 0x7f1600162820 2024-05-20T18:37:48.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.716+0000 7f15fffff640 1 -- 172.21.15.5:0/4013423428 >> v1:172.21.15.5:6789/0 conn(0x7f1600162820 legacy=0x7f16000b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:48.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.716+0000 7f15fffff640 1 --2- 172.21.15.5:0/4013423428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15e80039b0 0x7f15e8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:48.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.716+0000 7f15fffff640 1 --2- 172.21.15.5:0/4013423428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15e8004800 0x7f15e8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:48.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.716+0000 7f15fffff640 1 --2- 172.21.15.5:0/4013423428 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15e80051c0 0x7f15e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:48.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.716+0000 7f15fffff640 1 -- 172.21.15.5:0/4013423428 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f16000d39c0 con 0x7f15e80039b0 2024-05-20T18:37:48.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.716+0000 7f15fffff640 1 -- 172.21.15.5:0/4013423428 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f15e800c160 con 0x7f15e8004800 2024-05-20T18:37:48.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.716+0000 7f15fffff640 1 -- 172.21.15.5:0/4013423428 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f15e800c360 con 0x7f15e80051c0 2024-05-20T18:37:48.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.716+0000 7f160502d640 1 --2- 172.21.15.5:0/4013423428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15e80039b0 0x7f15e8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:48.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.716+0000 7f160482c640 1 --2- 172.21.15.5:0/4013423428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15e8004800 0x7f15e8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:48.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.716+0000 7f160582e640 1 --2- 172.21.15.5:0/4013423428 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15e80051c0 0x7f15e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:48.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.716+0000 7f160502d640 1 -- 172.21.15.5:0/4013423428 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15e80051c0 msgr2=0x7f15e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:48.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.716+0000 7f160502d640 1 --2- 172.21.15.5:0/4013423428 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15e80051c0 0x7f15e800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:48.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.716+0000 7f160502d640 1 -- 172.21.15.5:0/4013423428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15e8004800 msgr2=0x7f15e8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:48.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.716+0000 7f160502d640 1 --2- 172.21.15.5:0/4013423428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15e8004800 0x7f15e8004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:48.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.716+0000 7f160502d640 1 -- 172.21.15.5:0/4013423428 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e8003520 con 0x7f15e80039b0 2024-05-20T18:37:48.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.716+0000 7f160582e640 1 -- 172.21.15.5:0/4013423428 reap_dead start 2024-05-20T18:37:48.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.717+0000 7f160502d640 1 --2- 172.21.15.5:0/4013423428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15e80039b0 0x7f15e8003e20 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f15f800c4c0 tx=0x7f15e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68937ae0b28419e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:48.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.717+0000 7f15fffff640 1 -- 172.21.15.5:0/4013423428 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f8036070 con 0x7f15e80039b0 2024-05-20T18:37:48.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.717+0000 7f15fffff640 1 -- 172.21.15.5:0/4013423428 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15f8005af0 con 0x7f15e80039b0 2024-05-20T18:37:48.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.717+0000 7f15fffff640 1 -- 172.21.15.5:0/4013423428 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f800c0e0 con 0x7f15e80039b0 2024-05-20T18:37:48.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.718+0000 7f1607291640 1 -- 172.21.15.5:0/4013423428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15e80039b0 msgr2=0x7f15e8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:48.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.718+0000 7f1607291640 1 --2- 172.21.15.5:0/4013423428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15e80039b0 0x7f15e8003e20 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f15f800c4c0 tx=0x7f15e8005830 comp rx=0 tx=0).stop 2024-05-20T18:37:48.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.718+0000 7f1607291640 1 -- 172.21.15.5:0/4013423428 shutdown_connections 2024-05-20T18:37:48.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.718+0000 7f1607291640 1 --2- 172.21.15.5:0/4013423428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15e80039b0 0x7f15e8003e20 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:48.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.718+0000 7f1607291640 1 -- 172.21.15.5:0/4013423428 >> 172.21.15.5:0/4013423428 conn(0x7f16000d4bd0 msgr2=0x7f16000d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:48.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.718+0000 7f1607291640 1 -- 172.21.15.5:0/4013423428 shutdown_connections 2024-05-20T18:37:48.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.718+0000 7f1607291640 1 -- 172.21.15.5:0/4013423428 wait complete. 2024-05-20T18:37:48.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.720+0000 7f1607291640 1 Processor -- start 2024-05-20T18:37:48.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.720+0000 7f1607291640 1 -- start start 2024-05-20T18:37:48.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.721+0000 7f1607291640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15e80039b0 0x7f160014f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:48.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.721+0000 7f1607291640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16000ab870 0x7f16000abc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:48.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.721+0000 7f1607291640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16000ca0a0 0x7f16000ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:48.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.721+0000 7f1607291640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f16000d0300 con 0x7f15e80039b0 2024-05-20T18:37:48.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.721+0000 7f1607291640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f16000d4260 con 0x7f16000ab870 2024-05-20T18:37:48.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.721+0000 7f1607291640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f16000d4780 con 0x7f16000ca0a0 2024-05-20T18:37:48.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.721+0000 7f160502d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15e80039b0 0x7f160014f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.721+0000 7f160502d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15e80039b0 0x7f160014f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35602/0 (socket says 172.21.15.5:35602) 2024-05-20T18:37:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.721+0000 7f160502d640 1 -- 172.21.15.5:0/3656168930 learned_addr learned my addr 172.21.15.5:0/3656168930 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.721+0000 7f160582e640 1 --2- 172.21.15.5:0/3656168930 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16000ca0a0 0x7f16000ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.721+0000 7f160482c640 1 --2- 172.21.15.5:0/3656168930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16000ab870 0x7f16000abc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.721+0000 7f160502d640 1 -- 172.21.15.5:0/3656168930 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16000ca0a0 msgr2=0x7f16000ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.721+0000 7f160502d640 1 --2- 172.21.15.5:0/3656168930 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16000ca0a0 0x7f16000ca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.721+0000 7f160502d640 1 -- 172.21.15.5:0/3656168930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16000ab870 msgr2=0x7f16000abc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.721+0000 7f160502d640 1 --2- 172.21.15.5:0/3656168930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16000ab870 0x7f16000abc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.722+0000 7f160502d640 1 -- 172.21.15.5:0/3656168930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16000ca9f0 con 0x7f15e80039b0 2024-05-20T18:37:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.722+0000 7f160502d640 1 --2- 172.21.15.5:0/3656168930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15e80039b0 0x7f160014f5d0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f15f800c4c0 tx=0x7f16000d0800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.722+0000 7f15fdffb640 1 -- 172.21.15.5:0/3656168930 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f8036070 con 0x7f15e80039b0 2024-05-20T18:37:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.722+0000 7f15fdffb640 1 -- 172.21.15.5:0/3656168930 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15f8042070 con 0x7f15e80039b0 2024-05-20T18:37:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.722+0000 7f1607291640 1 -- 172.21.15.5:0/3656168930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16000b1470 con 0x7f15e80039b0 2024-05-20T18:37:48.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.722+0000 7f15fdffb640 1 -- 172.21.15.5:0/3656168930 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f8004be0 con 0x7f15e80039b0 2024-05-20T18:37:48.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.722+0000 7f1607291640 1 -- 172.21.15.5:0/3656168930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f160014ff40 con 0x7f15e80039b0 2024-05-20T18:37:48.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.723+0000 7f1607291640 1 -- 172.21.15.5:0/3656168930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16000b1640 con 0x7f15e80039b0 2024-05-20T18:37:48.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.724+0000 7f15fdffb640 1 -- 172.21.15.5:0/3656168930 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f15f8007070 con 0x7f15e80039b0 2024-05-20T18:37:48.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.724+0000 7f15fdffb640 1 --2- 172.21.15.5:0/3656168930 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f15ec06be30 0x7f15ec06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:48.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.724+0000 7f15fdffb640 1 -- 172.21.15.5:0/3656168930 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f15f80ab4e0 con 0x7f15e80039b0 2024-05-20T18:37:48.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.724+0000 7f160482c640 1 --2- 172.21.15.5:0/3656168930 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f15ec06be30 0x7f15ec06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:48.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.725+0000 7f160482c640 1 --2- 172.21.15.5:0/3656168930 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f15ec06be30 0x7f15ec06e340 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f15f40090e0 tx=0x7f15f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:48.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.729+0000 7f15fdffb640 1 -- 172.21.15.5:0/3656168930 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f16000ca9f0 con 0x7f15e80039b0 2024-05-20T18:37:48.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.914+0000 7f1607291640 1 -- 172.21.15.5:0/3656168930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1600075b00 con 0x7f15e80039b0 2024-05-20T18:37:48.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.915+0000 7f15fdffb640 1 -- 172.21.15.5:0/3656168930 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+17089 (secure 0 0 0) 0x7f160006dd20 con 0x7f15e80039b0 2024-05-20T18:37:48.917 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:48.917 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":121,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:37:39.699377+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:37:12.797471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-20T18:37:13.346156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6837/4273429943":"2024-05-21T18:37:39.699336+0000","172.21.15.63:6835/2352520472":"2024-05-21T18:37:28.901938+0000","172.21.15.5:6835/4273429943":"2024-05-21T18:37:39.699336+0000","172.21.15.63:6833/2352520472":"2024-05-21T18:37:28.901938+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:37:48.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.917+0000 7f1607291640 1 -- 172.21.15.5:0/3656168930 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f15ec06be30 msgr2=0x7f15ec06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:48.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.917+0000 7f1607291640 1 --2- 172.21.15.5:0/3656168930 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f15ec06be30 0x7f15ec06e340 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f15f40090e0 tx=0x7f15f4008040 comp rx=0 tx=0).stop 2024-05-20T18:37:48.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.917+0000 7f1607291640 1 -- 172.21.15.5:0/3656168930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15e80039b0 msgr2=0x7f160014f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:48.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.917+0000 7f1607291640 1 --2- 172.21.15.5:0/3656168930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15e80039b0 0x7f160014f5d0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f15f800c4c0 tx=0x7f16000d0800 comp rx=0 tx=0).stop 2024-05-20T18:37:48.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.917+0000 7f1607291640 1 -- 172.21.15.5:0/3656168930 shutdown_connections 2024-05-20T18:37:48.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.917+0000 7f1607291640 1 --2- 172.21.15.5:0/3656168930 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f15ec06be30 0x7f15ec06e340 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:48.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.917+0000 7f1607291640 1 --2- 172.21.15.5:0/3656168930 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16000ca0a0 0x7f16000ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:48.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.917+0000 7f1607291640 1 --2- 172.21.15.5:0/3656168930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16000ab870 0x7f16000abc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:48.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.917+0000 7f1607291640 1 --2- 172.21.15.5:0/3656168930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15e80039b0 0x7f160014f5d0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:48.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.917+0000 7f1607291640 1 -- 172.21.15.5:0/3656168930 >> 172.21.15.5:0/3656168930 conn(0x7f16000d4bd0 msgr2=0x7f1600161700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:48.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.917+0000 7f1607291640 1 -- 172.21.15.5:0/3656168930 shutdown_connections 2024-05-20T18:37:48.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:48.917+0000 7f1607291640 1 -- 172.21.15.5:0/3656168930 wait complete. 2024-05-20T18:37:48.930 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:37:12.797471+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:37:13.346156+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:37:48.931 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:37:49.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.048+0000 7f388affe640 1 Processor -- start 2024-05-20T18:37:49.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.048+0000 7f388affe640 1 -- start start 2024-05-20T18:37:49.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.049+0000 7f388affe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f38840b1cc0 0x7f388409a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:49.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.049+0000 7f388affe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f388409aa30 0x7f38840d18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:49.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.049+0000 7f388affe640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f388409ae40 con 0x7f38841599a0 2024-05-20T18:37:49.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.049+0000 7f388affe640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f38841607a0 con 0x7f38840b1cc0 2024-05-20T18:37:49.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.049+0000 7f388affe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f388409b2d0 con 0x7f388409aa30 2024-05-20T18:37:49.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.049+0000 7f3888d9a640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f38841599a0 0x7f388406d980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44770/0 (socket says 172.21.15.5:44770) 2024-05-20T18:37:49.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.049+0000 7f3888d9a640 1 -- 172.21.15.5:0/3578699385 learned_addr learned my addr 172.21.15.5:0/3578699385 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:49.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.050+0000 7f38837fe640 1 -- 172.21.15.5:0/3578699385 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1204900597 0 0) 0x7f387c001490 con 0x7f38841599a0 2024-05-20T18:37:49.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.050+0000 7f38837fe640 1 -- 172.21.15.5:0/3578699385 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3870003520 con 0x7f38841599a0 2024-05-20T18:37:49.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.050+0000 7f388959b640 1 --2- 172.21.15.5:0/3578699385 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f388409aa30 0x7f38840d18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:49.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.050+0000 7f3883fff640 1 --2- 172.21.15.5:0/3578699385 >> v2:172.21.15.63:3300/0 conn(0x7f38840b1cc0 0x7f388409a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:49.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.050+0000 7f38837fe640 1 -- 172.21.15.5:0/3578699385 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2494187318 0 0) 0x7f3870003520 con 0x7f38841599a0 2024-05-20T18:37:49.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.050+0000 7f38837fe640 1 -- 172.21.15.5:0/3578699385 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f388409aa30 msgr2=0x7f38840d18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:49.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.050+0000 7f38837fe640 1 --2- 172.21.15.5:0/3578699385 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f388409aa30 0x7f38840d18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.050+0000 7f38837fe640 1 -- 172.21.15.5:0/3578699385 >> v2:172.21.15.63:3300/0 conn(0x7f38840b1cc0 msgr2=0x7f388409a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:49.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.050+0000 7f38837fe640 1 --2- 172.21.15.5:0/3578699385 >> v2:172.21.15.63:3300/0 conn(0x7f38840b1cc0 0x7f388409a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.050+0000 7f38837fe640 1 -- 172.21.15.5:0/3578699385 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388415fc20 con 0x7f38841599a0 2024-05-20T18:37:49.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.050+0000 7f38837fe640 1 -- 172.21.15.5:0/3578699385 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f387c001bc0 con 0x7f38841599a0 2024-05-20T18:37:49.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.050+0000 7f38837fe640 1 -- 172.21.15.5:0/3578699385 >> v1:172.21.15.5:6789/0 conn(0x7f38841599a0 legacy=0x7f388406d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:49.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.050+0000 7f38837fe640 1 --2- 172.21.15.5:0/3578699385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38700039b0 0x7f3870003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:49.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.050+0000 7f38837fe640 1 --2- 172.21.15.5:0/3578699385 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f388409aa30 0x7f3870004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:49.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.050+0000 7f38837fe640 1 --2- 172.21.15.5:0/3578699385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3870004d40 0x7f3870005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:49.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.050+0000 7f38837fe640 1 -- 172.21.15.5:0/3578699385 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f38841607a0 con 0x7f3870004d40 2024-05-20T18:37:49.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.050+0000 7f38837fe640 1 -- 172.21.15.5:0/3578699385 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f388409b2d0 con 0x7f38700039b0 2024-05-20T18:37:49.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.050+0000 7f38837fe640 1 -- 172.21.15.5:0/3578699385 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f387000ba70 con 0x7f388409aa30 2024-05-20T18:37:49.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.050+0000 7f388959b640 1 --2- 172.21.15.5:0/3578699385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3870004d40 0x7f3870005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:49.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.050+0000 7f3883fff640 1 --2- 172.21.15.5:0/3578699385 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f388409aa30 0x7f3870004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:49.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.050+0000 7f3888d9a640 1 --2- 172.21.15.5:0/3578699385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38700039b0 0x7f3870003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:49.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.051+0000 7f388959b640 1 -- 172.21.15.5:0/3578699385 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f388409aa30 msgr2=0x7f3870004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:49.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.051+0000 7f388959b640 1 --2- 172.21.15.5:0/3578699385 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f388409aa30 0x7f3870004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.051+0000 7f388959b640 1 -- 172.21.15.5:0/3578699385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38700039b0 msgr2=0x7f3870003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:49.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.051+0000 7f388959b640 1 --2- 172.21.15.5:0/3578699385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38700039b0 0x7f3870003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.051+0000 7f388959b640 1 -- 172.21.15.5:0/3578699385 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3870003520 con 0x7f3870004d40 2024-05-20T18:37:49.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.051+0000 7f388959b640 1 --2- 172.21.15.5:0/3578699385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3870004d40 0x7f3870005150 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f3874009a90 tx=0x7f3870005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1dbf5acf850553b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:49.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.051+0000 7f38837fe640 1 -- 172.21.15.5:0/3578699385 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3874036070 con 0x7f3870004d40 2024-05-20T18:37:49.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.051+0000 7f38837fe640 1 -- 172.21.15.5:0/3578699385 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3874031400 con 0x7f3870004d40 2024-05-20T18:37:49.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.051+0000 7f38837fe640 1 -- 172.21.15.5:0/3578699385 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3874009690 con 0x7f3870004d40 2024-05-20T18:37:49.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.052+0000 7f388affe640 1 -- 172.21.15.5:0/3578699385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3870004d40 msgr2=0x7f3870005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:49.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.052+0000 7f388affe640 1 --2- 172.21.15.5:0/3578699385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3870004d40 0x7f3870005150 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f3874009a90 tx=0x7f3870005830 comp rx=0 tx=0).stop 2024-05-20T18:37:49.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.052+0000 7f388959b640 1 -- 172.21.15.5:0/3578699385 reap_dead start 2024-05-20T18:37:49.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.052+0000 7f388affe640 1 -- 172.21.15.5:0/3578699385 shutdown_connections 2024-05-20T18:37:49.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.052+0000 7f388affe640 1 -- 172.21.15.5:0/3578699385 >> 172.21.15.5:0/3578699385 conn(0x7f38840b6670 msgr2=0x7f38840b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:49.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.052+0000 7f388affe640 1 -- 172.21.15.5:0/3578699385 shutdown_connections 2024-05-20T18:37:49.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.052+0000 7f388affe640 1 -- 172.21.15.5:0/3578699385 wait complete. 2024-05-20T18:37:49.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.054+0000 7f388affe640 1 Processor -- start 2024-05-20T18:37:49.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.054+0000 7f388affe640 1 -- start start 2024-05-20T18:37:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.055+0000 7f388affe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3870004d40 0x7f388409ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.055+0000 7f388affe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3884153be0 0x7f388409e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.055+0000 7f388affe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3884154680 0x7f3884154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.055+0000 7f388affe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f38841604c0 con 0x7f3884153be0 2024-05-20T18:37:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.055+0000 7f388affe640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f38841601e0 con 0x7f3870004d40 2024-05-20T18:37:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.056+0000 7f388affe640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f388415f7f0 con 0x7f3884154680 2024-05-20T18:37:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.056+0000 7f3883fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3884153be0 0x7f388409e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.056+0000 7f3883fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3884153be0 0x7f388409e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35628/0 (socket says 172.21.15.5:35628) 2024-05-20T18:37:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.056+0000 7f3883fff640 1 -- 172.21.15.5:0/3286581380 learned_addr learned my addr 172.21.15.5:0/3286581380 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.056+0000 7f388959b640 1 --2- 172.21.15.5:0/3286581380 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3884154680 0x7f3884154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.056+0000 7f3888d9a640 1 --2- 172.21.15.5:0/3286581380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3870004d40 0x7f388409ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.056+0000 7f3883fff640 1 -- 172.21.15.5:0/3286581380 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3884154680 msgr2=0x7f3884154a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.056+0000 7f3883fff640 1 --2- 172.21.15.5:0/3286581380 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3884154680 0x7f3884154a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.056+0000 7f3883fff640 1 -- 172.21.15.5:0/3286581380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3870004d40 msgr2=0x7f388409ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.056+0000 7f3883fff640 1 --2- 172.21.15.5:0/3286581380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3870004d40 0x7f388409ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.056+0000 7f3883fff640 1 -- 172.21.15.5:0/3286581380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3884163350 con 0x7f3884153be0 2024-05-20T18:37:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.056+0000 7f3883fff640 1 --2- 172.21.15.5:0/3286581380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3884153be0 0x7f388409e330 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f386c011990 tx=0x7f386c011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.056+0000 7f38817fa640 1 -- 172.21.15.5:0/3286581380 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386c019070 con 0x7f3884153be0 2024-05-20T18:37:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.056+0000 7f38817fa640 1 -- 172.21.15.5:0/3286581380 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f386c004240 con 0x7f3884153be0 2024-05-20T18:37:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.056+0000 7f388affe640 1 -- 172.21.15.5:0/3286581380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f388409c840 con 0x7f3884153be0 2024-05-20T18:37:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.056+0000 7f38817fa640 1 -- 172.21.15.5:0/3286581380 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386c00f8c0 con 0x7f3884153be0 2024-05-20T18:37:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.056+0000 7f388affe640 1 -- 172.21.15.5:0/3286581380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3884162870 con 0x7f3884153be0 2024-05-20T18:37:49.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.057+0000 7f388affe640 1 -- 172.21.15.5:0/3286581380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3884075b00 con 0x7f3884153be0 2024-05-20T18:37:49.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.058+0000 7f38817fa640 1 -- 172.21.15.5:0/3286581380 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f386c01c070 con 0x7f3884153be0 2024-05-20T18:37:49.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.059+0000 7f38817fa640 1 --2- 172.21.15.5:0/3286581380 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f386406be30 0x7f386406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:49.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.059+0000 7f3888d9a640 1 --2- 172.21.15.5:0/3286581380 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f386406be30 0x7f386406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:49.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.059+0000 7f38817fa640 1 -- 172.21.15.5:0/3286581380 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f386c08b620 con 0x7f3884153be0 2024-05-20T18:37:49.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.060+0000 7f3888d9a640 1 --2- 172.21.15.5:0/3286581380 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f386406be30 0x7f386406e340 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f3884073480 tx=0x7f387c006040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.064+0000 7f38817fa640 1 -- 172.21.15.5:0/3286581380 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f386c059d00 con 0x7f3884153be0 2024-05-20T18:37:49.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.246+0000 7f388affe640 1 -- 172.21.15.5:0/3286581380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f388409c5d0 con 0x7f3884153be0 2024-05-20T18:37:49.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.246+0000 7f38817fa640 1 -- 172.21.15.5:0/3286581380 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) ==== 93+0+10 (secure 0 0 0) 0x7f386c059600 con 0x7f3884153be0 2024-05-20T18:37:49.248 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:37:49.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.248+0000 7f388affe640 1 -- 172.21.15.5:0/3286581380 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f386406be30 msgr2=0x7f386406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:49.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.248+0000 7f388affe640 1 --2- 172.21.15.5:0/3286581380 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f386406be30 0x7f386406e340 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f3884073480 tx=0x7f387c006040 comp rx=0 tx=0).stop 2024-05-20T18:37:49.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.248+0000 7f388affe640 1 -- 172.21.15.5:0/3286581380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3884153be0 msgr2=0x7f388409e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:49.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.248+0000 7f388affe640 1 --2- 172.21.15.5:0/3286581380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3884153be0 0x7f388409e330 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f386c011990 tx=0x7f386c011e50 comp rx=0 tx=0).stop 2024-05-20T18:37:49.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.249+0000 7f388affe640 1 -- 172.21.15.5:0/3286581380 shutdown_connections 2024-05-20T18:37:49.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.249+0000 7f388affe640 1 --2- 172.21.15.5:0/3286581380 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f386406be30 0x7f386406e340 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.249+0000 7f388affe640 1 --2- 172.21.15.5:0/3286581380 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3884154680 0x7f3884154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.249+0000 7f388affe640 1 --2- 172.21.15.5:0/3286581380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3884153be0 0x7f388409e330 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.249+0000 7f388affe640 1 --2- 172.21.15.5:0/3286581380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3870004d40 0x7f388409ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.249+0000 7f388affe640 1 -- 172.21.15.5:0/3286581380 >> 172.21.15.5:0/3286581380 conn(0x7f38840b6670 msgr2=0x7f38840b4150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:49.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.249+0000 7f388affe640 1 -- 172.21.15.5:0/3286581380 shutdown_connections 2024-05-20T18:37:49.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.249+0000 7f388affe640 1 -- 172.21.15.5:0/3286581380 wait complete. 2024-05-20T18:37:49.261 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:37:49.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.378+0000 7f7dc07d5640 1 Processor -- start 2024-05-20T18:37:49.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.378+0000 7f7dc07d5640 1 -- start start 2024-05-20T18:37:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.379+0000 7f7dc07d5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7db8162820 0x7f7db80b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.379+0000 7f7dc07d5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7db80b3e80 0x7f7db806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.379+0000 7f7dc07d5640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7db806d910 0x7f7db80d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.379+0000 7f7dc07d5640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f7db806dd20 con 0x7f7db80b3e80 2024-05-20T18:37:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.379+0000 7f7dc07d5640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7db80d0630 con 0x7f7db806d910 2024-05-20T18:37:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.379+0000 7f7dc07d5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7db80d39c0 con 0x7f7db8162820 2024-05-20T18:37:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.380+0000 7f7dbdd70640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7db80b3e80 0x7f7db806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.380+0000 7f7dbdd70640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7db80b3e80 0x7f7db806d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35642/0 (socket says 172.21.15.5:35642) 2024-05-20T18:37:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.380+0000 7f7dbdd70640 1 -- 172.21.15.5:0/3657146095 learned_addr learned my addr 172.21.15.5:0/3657146095 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.380+0000 7f7dbdd70640 1 -- 172.21.15.5:0/3657146095 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7db8162820 msgr2=0x7f7db80b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.380+0000 7f7dbdd70640 1 --2- 172.21.15.5:0/3657146095 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7db8162820 0x7f7db80b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.380+0000 7f7dbed72640 1 --2- 172.21.15.5:0/3657146095 >> v2:172.21.15.63:3300/0 conn(0x7f7db806d910 0x7f7db80d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.380+0000 7f7dbdd70640 1 -- 172.21.15.5:0/3657146095 >> v2:172.21.15.63:3300/0 conn(0x7f7db806d910 msgr2=0x7f7db80d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.380+0000 7f7dbdd70640 1 --2- 172.21.15.5:0/3657146095 >> v2:172.21.15.63:3300/0 conn(0x7f7db806d910 0x7f7db80d27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.380+0000 7f7dbdd70640 1 -- 172.21.15.5:0/3657146095 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db80d3ca0 con 0x7f7db80b3e80 2024-05-20T18:37:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.380+0000 7f7dbed72640 1 --2- 172.21.15.5:0/3657146095 >> v2:172.21.15.63:3300/0 conn(0x7f7db806d910 0x7f7db80d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.381+0000 7f7dbdd70640 1 --2- 172.21.15.5:0/3657146095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7db80b3e80 0x7f7db806d3d0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f7db806ce40 tx=0x7f7da802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e3fcf5765a2a79 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:49.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.381+0000 7f7dbd56f640 1 -- 172.21.15.5:0/3657146095 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8037070 con 0x7f7db80b3e80 2024-05-20T18:37:49.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.381+0000 7f7dbd56f640 1 -- 172.21.15.5:0/3657146095 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7da8032440 con 0x7f7db80b3e80 2024-05-20T18:37:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.381+0000 7f7dbd56f640 1 -- 172.21.15.5:0/3657146095 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8032810 con 0x7f7db80b3e80 2024-05-20T18:37:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.381+0000 7f7dc07d5640 1 -- 172.21.15.5:0/3657146095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7db80b3e80 msgr2=0x7f7db806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.381+0000 7f7dc07d5640 1 --2- 172.21.15.5:0/3657146095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7db80b3e80 0x7f7db806d3d0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f7db806ce40 tx=0x7f7da802a1e0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.382+0000 7f7dc07d5640 1 -- 172.21.15.5:0/3657146095 shutdown_connections 2024-05-20T18:37:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.382+0000 7f7dc07d5640 1 --2- 172.21.15.5:0/3657146095 >> v2:172.21.15.63:3300/0 conn(0x7f7db806d910 0x7f7db80d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.382+0000 7f7dc07d5640 1 --2- 172.21.15.5:0/3657146095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7db80b3e80 0x7f7db806d3d0 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.382+0000 7f7dc07d5640 1 --2- 172.21.15.5:0/3657146095 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7db8162820 0x7f7db80b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.382+0000 7f7dc07d5640 1 -- 172.21.15.5:0/3657146095 >> 172.21.15.5:0/3657146095 conn(0x7f7db80d4bd0 msgr2=0x7f7db80d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.382+0000 7f7dc07d5640 1 -- 172.21.15.5:0/3657146095 shutdown_connections 2024-05-20T18:37:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.382+0000 7f7dc07d5640 1 -- 172.21.15.5:0/3657146095 wait complete. 2024-05-20T18:37:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.383+0000 7f7dc07d5640 1 Processor -- start 2024-05-20T18:37:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.383+0000 7f7dc07d5640 1 -- start start 2024-05-20T18:37:49.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.385+0000 7f7dc07d5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7db806d910 0x7f7db80ce290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:49.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.385+0000 7f7dc07d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7db80b3e80 0x7f7db80cd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:49.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.385+0000 7f7dc07d5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7db8162820 0x7f7db80cd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.385+0000 7f7dc07d5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7db80d0300 con 0x7f7db80b3e80 2024-05-20T18:37:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.385+0000 7f7dc07d5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7db80d4260 con 0x7f7db8162820 2024-05-20T18:37:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.385+0000 7f7dc07d5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7db80d4780 con 0x7f7db806d910 2024-05-20T18:37:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.385+0000 7f7dbdd70640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7db80b3e80 0x7f7db80cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.385+0000 7f7dbdd70640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7db80b3e80 0x7f7db80cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35644/0 (socket says 172.21.15.5:35644) 2024-05-20T18:37:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.385+0000 7f7dbdd70640 1 -- 172.21.15.5:0/772060597 learned_addr learned my addr 172.21.15.5:0/772060597 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.385+0000 7f7dbed72640 1 --2- 172.21.15.5:0/772060597 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7db8162820 0x7f7db80cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.385+0000 7f7dbe571640 1 --2- 172.21.15.5:0/772060597 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7db806d910 0x7f7db80ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.385+0000 7f7dbdd70640 1 -- 172.21.15.5:0/772060597 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7db806d910 msgr2=0x7f7db80ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.385+0000 7f7dbdd70640 1 --2- 172.21.15.5:0/772060597 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7db806d910 0x7f7db80ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.385+0000 7f7dbdd70640 1 -- 172.21.15.5:0/772060597 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7db8162820 msgr2=0x7f7db80cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.385+0000 7f7dbdd70640 1 --2- 172.21.15.5:0/772060597 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7db8162820 0x7f7db80cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.385+0000 7f7dbdd70640 1 -- 172.21.15.5:0/772060597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db80cdcc0 con 0x7f7db80b3e80 2024-05-20T18:37:49.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.385+0000 7f7dbdd70640 1 --2- 172.21.15.5:0/772060597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7db80b3e80 0x7f7db80cd240 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f7da8000c00 tx=0x7f7da80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:49.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.386+0000 7f7daeffd640 1 -- 172.21.15.5:0/772060597 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8041020 con 0x7f7db80b3e80 2024-05-20T18:37:49.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.386+0000 7f7daeffd640 1 -- 172.21.15.5:0/772060597 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7da8030970 con 0x7f7db80b3e80 2024-05-20T18:37:49.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.386+0000 7f7dc07d5640 1 -- 172.21.15.5:0/772060597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db80cc1a0 con 0x7f7db80b3e80 2024-05-20T18:37:49.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.386+0000 7f7daeffd640 1 -- 172.21.15.5:0/772060597 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8037040 con 0x7f7db80b3e80 2024-05-20T18:37:49.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.386+0000 7f7dc07d5640 1 -- 172.21.15.5:0/772060597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db80cc760 con 0x7f7db80b3e80 2024-05-20T18:37:49.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.387+0000 7f7dc07d5640 1 -- 172.21.15.5:0/772060597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7db8075b00 con 0x7f7db80b3e80 2024-05-20T18:37:49.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.387+0000 7f7daeffd640 1 -- 172.21.15.5:0/772060597 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7da8044020 con 0x7f7db80b3e80 2024-05-20T18:37:49.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.388+0000 7f7daeffd640 1 --2- 172.21.15.5:0/772060597 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7d9c06be30 0x7f7d9c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:49.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.388+0000 7f7daeffd640 1 -- 172.21.15.5:0/772060597 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f7da80ab9f0 con 0x7f7db80b3e80 2024-05-20T18:37:49.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.388+0000 7f7dbe571640 1 --2- 172.21.15.5:0/772060597 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7d9c06be30 0x7f7d9c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:49.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.388+0000 7f7dbe571640 1 --2- 172.21.15.5:0/772060597 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7d9c06be30 0x7f7d9c06e340 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f7db40098e0 tx=0x7f7db4009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:49.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.393+0000 7f7daeffd640 1 -- 172.21.15.5:0/772060597 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7da804b030 con 0x7f7db80b3e80 2024-05-20T18:37:49.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.576+0000 7f7dc07d5640 1 -- 172.21.15.5:0/772060597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7db80b8190 con 0x7f7db80b3e80 2024-05-20T18:37:49.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.577+0000 7f7daeffd640 1 -- 172.21.15.5:0/772060597 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) ==== 104+0+11 (secure 0 0 0) 0x7f7da8079a90 con 0x7f7db80b3e80 2024-05-20T18:37:49.578 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:37:49.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.579+0000 7f7dc07d5640 1 -- 172.21.15.5:0/772060597 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7d9c06be30 msgr2=0x7f7d9c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:49.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.579+0000 7f7dc07d5640 1 --2- 172.21.15.5:0/772060597 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7d9c06be30 0x7f7d9c06e340 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f7db40098e0 tx=0x7f7db4009060 comp rx=0 tx=0).stop 2024-05-20T18:37:49.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.579+0000 7f7dc07d5640 1 -- 172.21.15.5:0/772060597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7db80b3e80 msgr2=0x7f7db80cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:49.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.579+0000 7f7dc07d5640 1 --2- 172.21.15.5:0/772060597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7db80b3e80 0x7f7db80cd240 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f7da8000c00 tx=0x7f7da80046c0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.579+0000 7f7dc07d5640 1 -- 172.21.15.5:0/772060597 shutdown_connections 2024-05-20T18:37:49.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.579+0000 7f7dc07d5640 1 --2- 172.21.15.5:0/772060597 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f7d9c06be30 0x7f7d9c06e340 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.579+0000 7f7dc07d5640 1 --2- 172.21.15.5:0/772060597 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7db8162820 0x7f7db80cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.579+0000 7f7dc07d5640 1 --2- 172.21.15.5:0/772060597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7db80b3e80 0x7f7db80cd240 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.579+0000 7f7dc07d5640 1 --2- 172.21.15.5:0/772060597 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7db806d910 0x7f7db80ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.579+0000 7f7dc07d5640 1 -- 172.21.15.5:0/772060597 >> 172.21.15.5:0/772060597 conn(0x7f7db80d4bd0 msgr2=0x7f7db80d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:49.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.579+0000 7f7dc07d5640 1 -- 172.21.15.5:0/772060597 shutdown_connections 2024-05-20T18:37:49.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.579+0000 7f7dc07d5640 1 -- 172.21.15.5:0/772060597 wait complete. 2024-05-20T18:37:49.591 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:37:49.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.709+0000 7f1256d57640 1 Processor -- start 2024-05-20T18:37:49.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.709+0000 7f1256d57640 1 -- start start 2024-05-20T18:37:49.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.710+0000 7f1256d57640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f125015f220 0x7f12500d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:49.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.710+0000 7f1256d57640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f12500d0e00 0x7f12500cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:49.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.710+0000 7f1256d57640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12500cfd20 con 0x7f12500cf910 2024-05-20T18:37:49.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.710+0000 7f1256d57640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f12500cda20 con 0x7f125015f220 2024-05-20T18:37:49.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.710+0000 7f1256d57640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f12500cdd00 con 0x7f12500d0e00 2024-05-20T18:37:49.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.710+0000 7f1254af3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f125015f220 0x7f12500d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:49.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.710+0000 7f1254af3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f125015f220 0x7f12500d08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35648/0 (socket says 172.21.15.5:35648) 2024-05-20T18:37:49.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.710+0000 7f1254af3640 1 -- 172.21.15.5:0/4092867868 learned_addr learned my addr 172.21.15.5:0/4092867868 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:49.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.710+0000 7f124ffff640 1 --2- 172.21.15.5:0/4092867868 >> v2:172.21.15.63:3300/0 conn(0x7f12500d0e00 0x7f12500cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.710+0000 7f1254af3640 1 -- 172.21.15.5:0/4092867868 >> v2:172.21.15.63:3300/0 conn(0x7f12500d0e00 msgr2=0x7f12500cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.710+0000 7f1254af3640 1 --2- 172.21.15.5:0/4092867868 >> v2:172.21.15.63:3300/0 conn(0x7f12500d0e00 0x7f12500cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.710+0000 7f1254af3640 1 -- 172.21.15.5:0/4092867868 >> v1:172.21.15.63:6789/0 conn(0x7f12500cf910 legacy=0x7f12500ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.710+0000 7f1254af3640 1 -- 172.21.15.5:0/4092867868 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12500cdfe0 con 0x7f125015f220 2024-05-20T18:37:49.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.711+0000 7f1254af3640 1 --2- 172.21.15.5:0/4092867868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f125015f220 0x7f12500d08c0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f1250068350 tx=0x7f124002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b63be45554f969 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:49.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.711+0000 7f124f7fe640 1 -- 172.21.15.5:0/4092867868 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1240037070 con 0x7f125015f220 2024-05-20T18:37:49.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.711+0000 7f124f7fe640 1 -- 172.21.15.5:0/4092867868 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1240032440 con 0x7f125015f220 2024-05-20T18:37:49.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.711+0000 7f124f7fe640 1 -- 172.21.15.5:0/4092867868 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1240032810 con 0x7f125015f220 2024-05-20T18:37:49.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.712+0000 7f1256d57640 1 -- 172.21.15.5:0/4092867868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f125015f220 msgr2=0x7f12500d08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:49.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.712+0000 7f1256d57640 1 --2- 172.21.15.5:0/4092867868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f125015f220 0x7f12500d08c0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f1250068350 tx=0x7f124002a1a0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.712+0000 7f1256d57640 1 -- 172.21.15.5:0/4092867868 shutdown_connections 2024-05-20T18:37:49.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.712+0000 7f1256d57640 1 --2- 172.21.15.5:0/4092867868 >> v2:172.21.15.63:3300/0 conn(0x7f12500d0e00 0x7f12500cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.712+0000 7f1256d57640 1 --2- 172.21.15.5:0/4092867868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f125015f220 0x7f12500d08c0 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.712+0000 7f1256d57640 1 -- 172.21.15.5:0/4092867868 >> 172.21.15.5:0/4092867868 conn(0x7f12500b44d0 msgr2=0x7f12500b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:49.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.712+0000 7f1256d57640 1 -- 172.21.15.5:0/4092867868 shutdown_connections 2024-05-20T18:37:49.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.712+0000 7f1256d57640 1 -- 172.21.15.5:0/4092867868 wait complete. 2024-05-20T18:37:49.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.714+0000 7f1256d57640 1 Processor -- start 2024-05-20T18:37:49.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.714+0000 7f1256d57640 1 -- start start 2024-05-20T18:37:49.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.715+0000 7f1256d57640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12500cf910 0x7f12500ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:49.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.715+0000 7f1256d57640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12500d0e00 0x7f12501f55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:49.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.715+0000 7f1256d57640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f125015f220 0x7f12500cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:49.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.715+0000 7f1256d57640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12500cd800 con 0x7f12500cf910 2024-05-20T18:37:49.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.715+0000 7f1256d57640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f12500cd680 con 0x7f125015f220 2024-05-20T18:37:49.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.715+0000 7f1256d57640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f125015f630 con 0x7f12500d0e00 2024-05-20T18:37:49.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.716+0000 7f1254af3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12500cf910 0x7f12500ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:49.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.716+0000 7f1254af3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12500cf910 0x7f12500ca330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35650/0 (socket says 172.21.15.5:35650) 2024-05-20T18:37:49.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.716+0000 7f1254af3640 1 -- 172.21.15.5:0/2173583250 learned_addr learned my addr 172.21.15.5:0/2173583250 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:49.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.716+0000 7f12552f4640 1 --2- 172.21.15.5:0/2173583250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f125015f220 0x7f12500cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:49.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.716+0000 7f124ffff640 1 --2- 172.21.15.5:0/2173583250 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12500d0e00 0x7f12501f55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:49.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.716+0000 7f1254af3640 1 -- 172.21.15.5:0/2173583250 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12500d0e00 msgr2=0x7f12501f55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:49.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.716+0000 7f1254af3640 1 --2- 172.21.15.5:0/2173583250 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12500d0e00 0x7f12501f55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.716+0000 7f1254af3640 1 -- 172.21.15.5:0/2173583250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f125015f220 msgr2=0x7f12500cbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:49.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.716+0000 7f1254af3640 1 --2- 172.21.15.5:0/2173583250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f125015f220 0x7f12500cbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.716+0000 7f1254af3640 1 -- 172.21.15.5:0/2173583250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12500caca0 con 0x7f12500cf910 2024-05-20T18:37:49.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.716+0000 7f124ffff640 1 --2- 172.21.15.5:0/2173583250 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12500d0e00 0x7f12501f55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:49.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.716+0000 7f1254af3640 1 --2- 172.21.15.5:0/2173583250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12500cf910 0x7f12500ca330 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f1240000c00 tx=0x7f12400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:49.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.716+0000 7f124d7fa640 1 -- 172.21.15.5:0/2173583250 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1240041070 con 0x7f12500cf910 2024-05-20T18:37:49.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.716+0000 7f124d7fa640 1 -- 172.21.15.5:0/2173583250 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f124003c870 con 0x7f12500cf910 2024-05-20T18:37:49.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.716+0000 7f1256d57640 1 -- 172.21.15.5:0/2173583250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12500caf80 con 0x7f12500cf910 2024-05-20T18:37:49.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.716+0000 7f124d7fa640 1 -- 172.21.15.5:0/2173583250 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1240037040 con 0x7f12500cf910 2024-05-20T18:37:49.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.716+0000 7f1256d57640 1 -- 172.21.15.5:0/2173583250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12501f5b20 con 0x7f12500cf910 2024-05-20T18:37:49.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.717+0000 7f1256d57640 1 -- 172.21.15.5:0/2173583250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1250075b00 con 0x7f12500cf910 2024-05-20T18:37:49.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.718+0000 7f124d7fa640 1 -- 172.21.15.5:0/2173583250 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1240044070 con 0x7f12500cf910 2024-05-20T18:37:49.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.719+0000 7f124d7fa640 1 --2- 172.21.15.5:0/2173583250 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f123c06be60 0x7f123c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:49.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.719+0000 7f124d7fa640 1 -- 172.21.15.5:0/2173583250 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f12400ab9d0 con 0x7f12500cf910 2024-05-20T18:37:49.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.719+0000 7f124ffff640 1 --2- 172.21.15.5:0/2173583250 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f123c06be60 0x7f123c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:49.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.719+0000 7f124ffff640 1 --2- 172.21.15.5:0/2173583250 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f123c06be60 0x7f123c06e370 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f12501638a0 tx=0x7f1244008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:49.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.723+0000 7f124d7fa640 1 -- 172.21.15.5:0/2173583250 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f12500cfd20 con 0x7f12500cf910 2024-05-20T18:37:49.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.908+0000 7f1256d57640 1 -- 172.21.15.5:0/2173583250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1250162c60 con 0x7f12500cf910 2024-05-20T18:37:49.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.908+0000 7f124d7fa640 1 -- 172.21.15.5:0/2173583250 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) ==== 100+0+11 (secure 0 0 0) 0x7f1240079af0 con 0x7f12500cf910 2024-05-20T18:37:49.910 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:37:49.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.910+0000 7f1256d57640 1 -- 172.21.15.5:0/2173583250 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f123c06be60 msgr2=0x7f123c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:49.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.910+0000 7f1256d57640 1 --2- 172.21.15.5:0/2173583250 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f123c06be60 0x7f123c06e370 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f12501638a0 tx=0x7f1244008040 comp rx=0 tx=0).stop 2024-05-20T18:37:49.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.910+0000 7f1256d57640 1 -- 172.21.15.5:0/2173583250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12500cf910 msgr2=0x7f12500ca330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:49.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.910+0000 7f1256d57640 1 --2- 172.21.15.5:0/2173583250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12500cf910 0x7f12500ca330 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f1240000c00 tx=0x7f12400046c0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.911+0000 7f1256d57640 1 -- 172.21.15.5:0/2173583250 shutdown_connections 2024-05-20T18:37:49.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.911+0000 7f1256d57640 1 --2- 172.21.15.5:0/2173583250 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f123c06be60 0x7f123c06e370 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.911+0000 7f1256d57640 1 --2- 172.21.15.5:0/2173583250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f125015f220 0x7f12500cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.911+0000 7f1256d57640 1 --2- 172.21.15.5:0/2173583250 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f12500d0e00 0x7f12501f55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.911+0000 7f1256d57640 1 --2- 172.21.15.5:0/2173583250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12500cf910 0x7f12500ca330 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:49.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.911+0000 7f1256d57640 1 -- 172.21.15.5:0/2173583250 >> 172.21.15.5:0/2173583250 conn(0x7f12500b44d0 msgr2=0x7f12500d1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:49.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.911+0000 7f1256d57640 1 -- 172.21.15.5:0/2173583250 shutdown_connections 2024-05-20T18:37:49.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:49.911+0000 7f1256d57640 1 -- 172.21.15.5:0/2173583250 wait complete. 2024-05-20T18:37:49.923 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-20T18:37:50.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.039+0000 7fb1121f5640 1 Processor -- start 2024-05-20T18:37:50.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.039+0000 7fb1121f5640 1 -- start start 2024-05-20T18:37:50.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.040+0000 7fb1121f5640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb10c06d3d0 0x7fb10c06d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:50.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.040+0000 7fb1121f5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb10c0b8190 0x7fb10c0b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:50.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.040+0000 7fb1121f5640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb10c06dd20 con 0x7fb10c162820 2024-05-20T18:37:50.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.040+0000 7fb1121f5640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fb10c0d0630 con 0x7fb10c0b8190 2024-05-20T18:37:50.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.040+0000 7fb1121f5640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb10c0d39c0 con 0x7fb10c06d3d0 2024-05-20T18:37:50.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.040+0000 7fb10bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb10c0b8190 0x7fb10c0b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:50.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.040+0000 7fb10bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb10c0b8190 0x7fb10c0b85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35668/0 (socket says 172.21.15.5:35668) 2024-05-20T18:37:50.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.040+0000 7fb10bfff640 1 -- 172.21.15.5:0/432561663 learned_addr learned my addr 172.21.15.5:0/432561663 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:50.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.040+0000 7fb10b7fe640 1 --1- 172.21.15.5:0/432561663 >> v1:172.21.15.5:6789/0 conn(0x7fb10c162820 0x7fb10c0b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44776/0 (socket says 172.21.15.5:44776) 2024-05-20T18:37:50.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.040+0000 7fb10affd640 1 --2- 172.21.15.5:0/432561663 >> v2:172.21.15.63:3300/0 conn(0x7fb10c06d3d0 0x7fb10c06d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:50.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.041+0000 7fb10bfff640 1 -- 172.21.15.5:0/432561663 >> v2:172.21.15.63:3300/0 conn(0x7fb10c06d3d0 msgr2=0x7fb10c06d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:50.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.041+0000 7fb10bfff640 1 --2- 172.21.15.5:0/432561663 >> v2:172.21.15.63:3300/0 conn(0x7fb10c06d3d0 0x7fb10c06d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:50.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.041+0000 7fb10bfff640 1 -- 172.21.15.5:0/432561663 >> v1:172.21.15.5:6789/0 conn(0x7fb10c162820 legacy=0x7fb10c0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:50.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.041+0000 7fb10bfff640 1 -- 172.21.15.5:0/432561663 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10c0d3ca0 con 0x7fb10c0b8190 2024-05-20T18:37:50.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.041+0000 7fb10bfff640 1 --2- 172.21.15.5:0/432561663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb10c0b8190 0x7fb10c0b85a0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0d1230 tx=0x7fb10402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4370f1424a55402 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:50.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.041+0000 7fb10a7fc640 1 -- 172.21.15.5:0/432561663 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb104036070 con 0x7fb10c0b8190 2024-05-20T18:37:50.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.041+0000 7fb10a7fc640 1 -- 172.21.15.5:0/432561663 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb104031600 con 0x7fb10c0b8190 2024-05-20T18:37:50.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.041+0000 7fb10a7fc640 1 -- 172.21.15.5:0/432561663 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1040319d0 con 0x7fb10c0b8190 2024-05-20T18:37:50.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.042+0000 7fb1121f5640 1 -- 172.21.15.5:0/432561663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb10c0b8190 msgr2=0x7fb10c0b85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.042+0000 7fb1121f5640 1 --2- 172.21.15.5:0/432561663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb10c0b8190 0x7fb10c0b85a0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0d1230 tx=0x7fb10402a730 comp rx=0 tx=0).stop 2024-05-20T18:37:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.042+0000 7fb1121f5640 1 -- 172.21.15.5:0/432561663 shutdown_connections 2024-05-20T18:37:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.042+0000 7fb1121f5640 1 --2- 172.21.15.5:0/432561663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb10c0b8190 0x7fb10c0b85a0 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.042+0000 7fb1121f5640 1 --2- 172.21.15.5:0/432561663 >> v2:172.21.15.63:3300/0 conn(0x7fb10c06d3d0 0x7fb10c06d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.042+0000 7fb1121f5640 1 -- 172.21.15.5:0/432561663 >> 172.21.15.5:0/432561663 conn(0x7fb10c0d4bd0 msgr2=0x7fb10c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.042+0000 7fb1121f5640 1 -- 172.21.15.5:0/432561663 shutdown_connections 2024-05-20T18:37:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.042+0000 7fb1121f5640 1 -- 172.21.15.5:0/432561663 wait complete. 2024-05-20T18:37:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.044+0000 7fb1121f5640 1 Processor -- start 2024-05-20T18:37:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.044+0000 7fb1121f5640 1 -- start start 2024-05-20T18:37:50.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.045+0000 7fb1121f5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb10c06d3d0 0x7fb10c14cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:50.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.045+0000 7fb1121f5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb10c0b8190 0x7fb10c149be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:50.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.045+0000 7fb1121f5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb10c162820 0x7fb10c0b9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:50.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.045+0000 7fb1121f5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb10c0d0300 con 0x7fb10c0b8190 2024-05-20T18:37:50.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.045+0000 7fb1121f5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb10c0d4260 con 0x7fb10c06d3d0 2024-05-20T18:37:50.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.045+0000 7fb1121f5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb10c0b4050 con 0x7fb10c162820 2024-05-20T18:37:50.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.046+0000 7fb10affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb10c0b8190 0x7fb10c149be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:50.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.046+0000 7fb10affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb10c0b8190 0x7fb10c149be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35670/0 (socket says 172.21.15.5:35670) 2024-05-20T18:37:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.046+0000 7fb10affd640 1 -- 172.21.15.5:0/395740782 learned_addr learned my addr 172.21.15.5:0/395740782 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.046+0000 7fb10b7fe640 1 --2- 172.21.15.5:0/395740782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb10c06d3d0 0x7fb10c14cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.046+0000 7fb10bfff640 1 --2- 172.21.15.5:0/395740782 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb10c162820 0x7fb10c0b9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.046+0000 7fb10affd640 1 -- 172.21.15.5:0/395740782 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb10c162820 msgr2=0x7fb10c0b9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.046+0000 7fb10affd640 1 --2- 172.21.15.5:0/395740782 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb10c162820 0x7fb10c0b9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.046+0000 7fb10affd640 1 -- 172.21.15.5:0/395740782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb10c06d3d0 msgr2=0x7fb10c14cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.046+0000 7fb10affd640 1 --2- 172.21.15.5:0/395740782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb10c06d3d0 0x7fb10c14cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.046+0000 7fb10affd640 1 -- 172.21.15.5:0/395740782 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10c14a540 con 0x7fb10c0b8190 2024-05-20T18:37:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.046+0000 7fb10affd640 1 --2- 172.21.15.5:0/395740782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb10c0b8190 0x7fb10c149be0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fb10000d620 tx=0x7fb10000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.046+0000 7fb0f3fff640 1 -- 172.21.15.5:0/395740782 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10000b620 con 0x7fb10c0b8190 2024-05-20T18:37:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.046+0000 7fb0f3fff640 1 -- 172.21.15.5:0/395740782 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb100019070 con 0x7fb10c0b8190 2024-05-20T18:37:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.046+0000 7fb1121f5640 1 -- 172.21.15.5:0/395740782 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb10c1f8f30 con 0x7fb10c0b8190 2024-05-20T18:37:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.046+0000 7fb0f3fff640 1 -- 172.21.15.5:0/395740782 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10000ba00 con 0x7fb10c0b8190 2024-05-20T18:37:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.046+0000 7fb1121f5640 1 -- 172.21.15.5:0/395740782 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb10c1f94f0 con 0x7fb10c0b8190 2024-05-20T18:37:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.047+0000 7fb1121f5640 1 -- 172.21.15.5:0/395740782 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb10c075b00 con 0x7fb10c0b8190 2024-05-20T18:37:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.048+0000 7fb0f3fff640 1 -- 172.21.15.5:0/395740782 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb10000bba0 con 0x7fb10c0b8190 2024-05-20T18:37:50.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.048+0000 7fb0f3fff640 1 --2- 172.21.15.5:0/395740782 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb0f806be30 0x7fb0f806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:50.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.049+0000 7fb0f3fff640 1 -- 172.21.15.5:0/395740782 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7fb10008b530 con 0x7fb10c0b8190 2024-05-20T18:37:50.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.049+0000 7fb10b7fe640 1 --2- 172.21.15.5:0/395740782 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb0f806be30 0x7fb0f806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:50.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.049+0000 7fb10b7fe640 1 --2- 172.21.15.5:0/395740782 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb0f806be30 0x7fb0f806e340 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc000f60 tx=0x7fb0fc007ce0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.053+0000 7fb0f3fff640 1 -- 172.21.15.5:0/395740782 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb100059c10 con 0x7fb10c0b8190 2024-05-20T18:37:50.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.252+0000 7fb1121f5640 1 -- 172.21.15.5:0/395740782 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb10c1f8cc0 con 0x7fb10c0b8190 2024-05-20T18:37:50.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.252+0000 7fb0f3fff640 1 -- 172.21.15.5:0/395740782 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v121) ==== 80+0+242 (secure 0 0 0) 0x7fb100059510 con 0x7fb10c0b8190 2024-05-20T18:37:50.254 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6837/4273429943 2024-05-21T18:37:39.699336+0000 2024-05-20T18:37:50.254 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.63:6835/2352520472 2024-05-21T18:37:28.901938+0000 2024-05-20T18:37:50.254 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6835/4273429943 2024-05-21T18:37:39.699336+0000 2024-05-20T18:37:50.254 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.63:6833/2352520472 2024-05-21T18:37:28.901938+0000 2024-05-20T18:37:50.254 INFO:teuthology.orchestra.run.smithi005.stderr:listed 4 entries 2024-05-20T18:37:50.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.255+0000 7fb1121f5640 1 -- 172.21.15.5:0/395740782 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb0f806be30 msgr2=0x7fb0f806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:50.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.255+0000 7fb1121f5640 1 --2- 172.21.15.5:0/395740782 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb0f806be30 0x7fb0f806e340 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc000f60 tx=0x7fb0fc007ce0 comp rx=0 tx=0).stop 2024-05-20T18:37:50.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.255+0000 7fb1121f5640 1 -- 172.21.15.5:0/395740782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb10c0b8190 msgr2=0x7fb10c149be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.255+0000 7fb1121f5640 1 --2- 172.21.15.5:0/395740782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb10c0b8190 0x7fb10c149be0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fb10000d620 tx=0x7fb10000dae0 comp rx=0 tx=0).stop 2024-05-20T18:37:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.255+0000 7fb1121f5640 1 -- 172.21.15.5:0/395740782 shutdown_connections 2024-05-20T18:37:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.255+0000 7fb1121f5640 1 --2- 172.21.15.5:0/395740782 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fb0f806be30 0x7fb0f806e340 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.255+0000 7fb1121f5640 1 --2- 172.21.15.5:0/395740782 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb10c162820 0x7fb10c0b9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.255+0000 7fb1121f5640 1 --2- 172.21.15.5:0/395740782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb10c0b8190 0x7fb10c149be0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.255+0000 7fb1121f5640 1 --2- 172.21.15.5:0/395740782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb10c06d3d0 0x7fb10c14cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.255+0000 7fb1121f5640 1 -- 172.21.15.5:0/395740782 >> 172.21.15.5:0/395740782 conn(0x7fb10c0d4bd0 msgr2=0x7fb10c160eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.255+0000 7fb1121f5640 1 -- 172.21.15.5:0/395740782 shutdown_connections 2024-05-20T18:37:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.255+0000 7fb1121f5640 1 -- 172.21.15.5:0/395740782 wait complete. 2024-05-20T18:37:50.267 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-20T18:37:50.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:50.268 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-20T18:37:50.297 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:37:50.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:50.298 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-20T18:37:50.354 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T18:37:50.355 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:37:50.355 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:37:50.375 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19732 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T18:37:50.375 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T18:37:50.375 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:37:50.375 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:37:50.411 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15663 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T18:37:50.412 INFO:tasks.cephfs.mount:Terminating background process 2024-05-20T18:37:50.412 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:37:50.412 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:37:50.434 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19732 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T18:37:50.434 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:37:50.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.564+0000 7efd6b338640 1 Processor -- start 2024-05-20T18:37:50.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.564+0000 7efd6b338640 1 -- start start 2024-05-20T18:37:50.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.565+0000 7efd6b338640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd640d1bc0 0x7efd640d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:50.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.565+0000 7efd6b338640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd640d0370 con 0x7efd640b4010 2024-05-20T18:37:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.565+0000 7efd6b338640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd640d0650 con 0x7efd64163a10 2024-05-20T18:37:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.565+0000 7efd6b338640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efd640d3ca0 con 0x7efd640d1bc0 2024-05-20T18:37:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.565+0000 7efd688d3640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7efd640b4010 0x7efd640d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44784/0 (socket says 172.21.15.5:44784) 2024-05-20T18:37:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.565+0000 7efd688d3640 1 -- 172.21.15.5:0/1161449583 learned_addr learned my addr 172.21.15.5:0/1161449583 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.566+0000 7efd698d5640 1 --2- 172.21.15.5:0/1161449583 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd640d1bc0 0x7efd640d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.566+0000 7efd5bfff640 1 -- 172.21.15.5:0/1161449583 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3684484498 0 0) 0x7efd640d0370 con 0x7efd640b4010 2024-05-20T18:37:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.566+0000 7efd5bfff640 1 -- 172.21.15.5:0/1161449583 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd50003520 con 0x7efd640b4010 2024-05-20T18:37:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.566+0000 7efd5bfff640 1 -- 172.21.15.5:0/1161449583 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1076031921 0 0) 0x7efd50003520 con 0x7efd640b4010 2024-05-20T18:37:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.566+0000 7efd5bfff640 1 -- 172.21.15.5:0/1161449583 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd640d1bc0 msgr2=0x7efd640d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.566+0000 7efd5bfff640 1 --2- 172.21.15.5:0/1161449583 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd640d1bc0 0x7efd640d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.566+0000 7efd5bfff640 1 -- 172.21.15.5:0/1161449583 >> v1:172.21.15.63:6789/0 conn(0x7efd64163a10 legacy=0x7efd640cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:50.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.566+0000 7efd5bfff640 1 -- 172.21.15.5:0/1161449583 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd640d3650 con 0x7efd640b4010 2024-05-20T18:37:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.566+0000 7efd5bfff640 1 -- 172.21.15.5:0/1161449583 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7efd54001670 con 0x7efd640b4010 2024-05-20T18:37:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.566+0000 7efd5bfff640 1 -- 172.21.15.5:0/1161449583 >> v1:172.21.15.5:6789/0 conn(0x7efd640b4010 legacy=0x7efd640d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.566+0000 7efd5bfff640 1 --2- 172.21.15.5:0/1161449583 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd640d1bc0 0x7efd500039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.566+0000 7efd5bfff640 1 --2- 172.21.15.5:0/1161449583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd50003f40 0x7efd50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.566+0000 7efd5bfff640 1 --2- 172.21.15.5:0/1161449583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd50004d40 0x7efd50005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.566+0000 7efd5bfff640 1 -- 172.21.15.5:0/1161449583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efd640d3ca0 con 0x7efd50004d40 2024-05-20T18:37:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.566+0000 7efd5bfff640 1 -- 172.21.15.5:0/1161449583 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efd5000bad0 con 0x7efd50003f40 2024-05-20T18:37:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.566+0000 7efd5bfff640 1 -- 172.21.15.5:0/1161449583 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efd5000bcd0 con 0x7efd640d1bc0 2024-05-20T18:37:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.566+0000 7efd698d5640 1 --2- 172.21.15.5:0/1161449583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd50004d40 0x7efd50005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.566+0000 7efd690d4640 1 --2- 172.21.15.5:0/1161449583 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd640d1bc0 0x7efd500039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.566+0000 7efd688d3640 1 --2- 172.21.15.5:0/1161449583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd50003f40 0x7efd50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:50.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.566+0000 7efd698d5640 1 -- 172.21.15.5:0/1161449583 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd640d1bc0 msgr2=0x7efd500039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.566+0000 7efd698d5640 1 --2- 172.21.15.5:0/1161449583 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd640d1bc0 0x7efd500039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.567+0000 7efd698d5640 1 -- 172.21.15.5:0/1161449583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd50003f40 msgr2=0x7efd50004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.567+0000 7efd698d5640 1 --2- 172.21.15.5:0/1161449583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd50003f40 0x7efd50004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.567+0000 7efd698d5640 1 -- 172.21.15.5:0/1161449583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd50003520 con 0x7efd50004d40 2024-05-20T18:37:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.567+0000 7efd698d5640 1 --2- 172.21.15.5:0/1161449583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd50004d40 0x7efd50005150 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009f90 tx=0x7efd50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da78173d9a9ee282 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.567+0000 7efd5bfff640 1 -- 172.21.15.5:0/1161449583 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c009c10 con 0x7efd50004d40 2024-05-20T18:37:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.567+0000 7efd5bfff640 1 -- 172.21.15.5:0/1161449583 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd5c036070 con 0x7efd50004d40 2024-05-20T18:37:50.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.567+0000 7efd5bfff640 1 -- 172.21.15.5:0/1161449583 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c031850 con 0x7efd50004d40 2024-05-20T18:37:50.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.568+0000 7efd6b338640 1 -- 172.21.15.5:0/1161449583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd50004d40 msgr2=0x7efd50005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:50.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.568+0000 7efd6b338640 1 --2- 172.21.15.5:0/1161449583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd50004d40 0x7efd50005150 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009f90 tx=0x7efd50005830 comp rx=0 tx=0).stop 2024-05-20T18:37:50.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.568+0000 7efd698d5640 1 -- 172.21.15.5:0/1161449583 reap_dead start 2024-05-20T18:37:50.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.568+0000 7efd6b338640 1 -- 172.21.15.5:0/1161449583 shutdown_connections 2024-05-20T18:37:50.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.568+0000 7efd6b338640 1 -- 172.21.15.5:0/1161449583 >> 172.21.15.5:0/1161449583 conn(0x7efd6406d240 msgr2=0x7efd6406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:50.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.568+0000 7efd6b338640 1 -- 172.21.15.5:0/1161449583 shutdown_connections 2024-05-20T18:37:50.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.568+0000 7efd6b338640 1 -- 172.21.15.5:0/1161449583 wait complete. 2024-05-20T18:37:50.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.570+0000 7efd6b338640 1 Processor -- start 2024-05-20T18:37:50.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.570+0000 7efd6b338640 1 -- start start 2024-05-20T18:37:50.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.571+0000 7efd6b338640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd640ad820 0x7efd640aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.571+0000 7efd6b338640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd6415c260 0x7efd640b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.571+0000 7efd6b338640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd6415c670 0x7efd640afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.571+0000 7efd6b338640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efd640d42b0 con 0x7efd640ad820 2024-05-20T18:37:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.571+0000 7efd6b338640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efd640d4130 con 0x7efd6415c670 2024-05-20T18:37:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.571+0000 7efd6b338640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efd640d4430 con 0x7efd6415c260 2024-05-20T18:37:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.571+0000 7efd690d4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd640ad820 0x7efd640aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.571+0000 7efd690d4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd640ad820 0x7efd640aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35690/0 (socket says 172.21.15.5:35690) 2024-05-20T18:37:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.571+0000 7efd690d4640 1 -- 172.21.15.5:0/2734025728 learned_addr learned my addr 172.21.15.5:0/2734025728 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.571+0000 7efd688d3640 1 --2- 172.21.15.5:0/2734025728 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd6415c260 0x7efd640b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.572+0000 7efd698d5640 1 --2- 172.21.15.5:0/2734025728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd6415c670 0x7efd640afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.572+0000 7efd690d4640 1 -- 172.21.15.5:0/2734025728 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd6415c260 msgr2=0x7efd640b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.572+0000 7efd690d4640 1 --2- 172.21.15.5:0/2734025728 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd6415c260 0x7efd640b2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:50.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.572+0000 7efd690d4640 1 -- 172.21.15.5:0/2734025728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd6415c670 msgr2=0x7efd640afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:50.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.572+0000 7efd690d4640 1 --2- 172.21.15.5:0/2734025728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd6415c670 0x7efd640afe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:50.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.572+0000 7efd690d4640 1 -- 172.21.15.5:0/2734025728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd640add00 con 0x7efd640ad820 2024-05-20T18:37:50.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.572+0000 7efd690d4640 1 --2- 172.21.15.5:0/2734025728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd640ad820 0x7efd640aeda0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7efd6000f830 tx=0x7efd6000fcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:50.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.572+0000 7efd59ffb640 1 -- 172.21.15.5:0/2734025728 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd600052e0 con 0x7efd640ad820 2024-05-20T18:37:50.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.572+0000 7efd59ffb640 1 -- 172.21.15.5:0/2734025728 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd60018070 con 0x7efd640ad820 2024-05-20T18:37:50.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.572+0000 7efd59ffb640 1 -- 172.21.15.5:0/2734025728 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6000d970 con 0x7efd640ad820 2024-05-20T18:37:50.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.572+0000 7efd6b338640 1 -- 172.21.15.5:0/2734025728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd640adfe0 con 0x7efd640ad820 2024-05-20T18:37:50.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.572+0000 7efd6b338640 1 -- 172.21.15.5:0/2734025728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd640ae5a0 con 0x7efd640ad820 2024-05-20T18:37:50.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.573+0000 7efd6b338640 1 -- 172.21.15.5:0/2734025728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd64075b00 con 0x7efd640ad820 2024-05-20T18:37:50.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.574+0000 7efd59ffb640 1 -- 172.21.15.5:0/2734025728 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efd6000db10 con 0x7efd640ad820 2024-05-20T18:37:50.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.574+0000 7efd59ffb640 1 --2- 172.21.15.5:0/2734025728 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efd4806be30 0x7efd4806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:50.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.574+0000 7efd59ffb640 1 -- 172.21.15.5:0/2734025728 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7efd6008b0c0 con 0x7efd640ad820 2024-05-20T18:37:50.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.574+0000 7efd688d3640 1 --2- 172.21.15.5:0/2734025728 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efd4806be30 0x7efd4806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:50.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.575+0000 7efd688d3640 1 --2- 172.21.15.5:0/2734025728 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efd4806be30 0x7efd4806e340 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7efd54007d00 tx=0x7efd54009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:50.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.579+0000 7efd59ffb640 1 -- 172.21.15.5:0/2734025728 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7efd60059830 con 0x7efd640ad820 2024-05-20T18:37:50.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.801+0000 7efd6b338640 1 -- 172.21.15.5:0/2734025728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd640d3050 con 0x7efd640ad820 2024-05-20T18:37:50.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.802+0000 7efd59ffb640 1 -- 172.21.15.5:0/2734025728 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5180 (secure 0 0 0) 0x7efd60059130 con 0x7efd640ad820 2024-05-20T18:37:50.803 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:50.804 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":111,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/266630184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":266630184},{"type":"v1","addr":"172.21.15.63:6834","nonce":266630184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7361,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1494022697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":1494022697},{"type":"v1","addr":"172.21.15.63:6835","nonce":1494022697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":9009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/1739221946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1739221946},{"type":"v1","addr":"172.21.15.5:6836","nonce":1739221946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:37:14.365195+0000","modified":"2024-05-20T18:37:44.722140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9444},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9444":{"gid":9444,"name":"c","rank":0,"incarnation":108,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6837/2067544478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":2067544478},{"type":"v1","addr":"172.21.15.5:6837","nonce":2067544478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9444,"qdb_cluster":[9444]},"id":8}]} 2024-05-20T18:37:50.804 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 111 2024-05-20T18:37:50.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.804+0000 7efd6b338640 1 -- 172.21.15.5:0/2734025728 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efd4806be30 msgr2=0x7efd4806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:50.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.804+0000 7efd6b338640 1 --2- 172.21.15.5:0/2734025728 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efd4806be30 0x7efd4806e340 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7efd54007d00 tx=0x7efd54009040 comp rx=0 tx=0).stop 2024-05-20T18:37:50.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.804+0000 7efd6b338640 1 -- 172.21.15.5:0/2734025728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd640ad820 msgr2=0x7efd640aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:50.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.804+0000 7efd6b338640 1 --2- 172.21.15.5:0/2734025728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd640ad820 0x7efd640aeda0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7efd6000f830 tx=0x7efd6000fcf0 comp rx=0 tx=0).stop 2024-05-20T18:37:50.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.804+0000 7efd6b338640 1 -- 172.21.15.5:0/2734025728 shutdown_connections 2024-05-20T18:37:50.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.804+0000 7efd6b338640 1 --2- 172.21.15.5:0/2734025728 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efd4806be30 0x7efd4806e340 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:50.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.804+0000 7efd6b338640 1 --2- 172.21.15.5:0/2734025728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd6415c670 0x7efd640afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:50.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.804+0000 7efd6b338640 1 --2- 172.21.15.5:0/2734025728 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd6415c260 0x7efd640b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:50.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.804+0000 7efd6b338640 1 --2- 172.21.15.5:0/2734025728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd640ad820 0x7efd640aeda0 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:50.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.804+0000 7efd6b338640 1 -- 172.21.15.5:0/2734025728 >> 172.21.15.5:0/2734025728 conn(0x7efd6406d240 msgr2=0x7efd6415f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:50.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.804+0000 7efd6b338640 1 -- 172.21.15.5:0/2734025728 shutdown_connections 2024-05-20T18:37:50.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.804+0000 7efd6b338640 1 -- 172.21.15.5:0/2734025728 wait complete. 2024-05-20T18:37:50.817 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:37:50.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.930+0000 7f08685b5640 1 Processor -- start 2024-05-20T18:37:50.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.930+0000 7f08685b5640 1 -- start start 2024-05-20T18:37:50.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.931+0000 7f08685b5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f086015e050 0x7f08600d1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:50.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.931+0000 7f08685b5640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f086015e6b0 con 0x7f08600b2f10 2024-05-20T18:37:50.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.931+0000 7f08685b5640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08600d1100 con 0x7f086006d9a0 2024-05-20T18:37:50.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.931+0000 7f08685b5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f08600d2400 con 0x7f086015e050 2024-05-20T18:37:50.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.931+0000 7f0866351640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f08600b2f10 0x7f086006d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44800/0 (socket says 172.21.15.5:44800) 2024-05-20T18:37:50.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.931+0000 7f0866351640 1 -- 172.21.15.5:0/4238532167 learned_addr learned my addr 172.21.15.5:0/4238532167 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:50.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.932+0000 7f0866b52640 1 --2- 172.21.15.5:0/4238532167 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f086015e050 0x7f08600d1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:50.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.932+0000 7f086534f640 1 -- 172.21.15.5:0/4238532167 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3867604509 0 0) 0x7f086015e6b0 con 0x7f08600b2f10 2024-05-20T18:37:50.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.932+0000 7f086534f640 1 -- 172.21.15.5:0/4238532167 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0848003520 con 0x7f08600b2f10 2024-05-20T18:37:50.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.932+0000 7f086534f640 1 -- 172.21.15.5:0/4238532167 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1377067978 0 0) 0x7f0848003520 con 0x7f08600b2f10 2024-05-20T18:37:50.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.932+0000 7f086534f640 1 -- 172.21.15.5:0/4238532167 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f086015e050 msgr2=0x7f08600d1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:50.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.932+0000 7f086534f640 1 --2- 172.21.15.5:0/4238532167 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f086015e050 0x7f08600d1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:50.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.932+0000 7f086534f640 1 -- 172.21.15.5:0/4238532167 >> v1:172.21.15.63:6789/0 conn(0x7f086006d9a0 legacy=0x7f086015d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:50.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.932+0000 7f086534f640 1 -- 172.21.15.5:0/4238532167 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08600d13e0 con 0x7f08600b2f10 2024-05-20T18:37:50.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.932+0000 7f086534f640 1 -- 172.21.15.5:0/4238532167 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f0858001670 con 0x7f08600b2f10 2024-05-20T18:37:50.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.932+0000 7f086534f640 1 -- 172.21.15.5:0/4238532167 >> v1:172.21.15.5:6789/0 conn(0x7f08600b2f10 legacy=0x7f086006d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:50.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.932+0000 7f086534f640 1 --2- 172.21.15.5:0/4238532167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08480039e0 0x7f0848003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:50.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.932+0000 7f086534f640 1 --2- 172.21.15.5:0/4238532167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0848004800 0x7f0848004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:50.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.932+0000 7f086534f640 1 --2- 172.21.15.5:0/4238532167 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f086015e050 0x7f0848005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:50.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.932+0000 7f086534f640 1 -- 172.21.15.5:0/4238532167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08600d2400 con 0x7f08480039e0 2024-05-20T18:37:50.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.932+0000 7f086534f640 1 -- 172.21.15.5:0/4238532167 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f084800baf0 con 0x7f0848004800 2024-05-20T18:37:50.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.932+0000 7f086534f640 1 -- 172.21.15.5:0/4238532167 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f084800bcf0 con 0x7f086015e050 2024-05-20T18:37:50.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.932+0000 7f0866351640 1 --2- 172.21.15.5:0/4238532167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08480039e0 0x7f0848003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:50.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.932+0000 7f0865b50640 1 --2- 172.21.15.5:0/4238532167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0848004800 0x7f0848004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:50.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.932+0000 7f0866b52640 1 --2- 172.21.15.5:0/4238532167 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f086015e050 0x7f0848005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:50.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.932+0000 7f0866351640 1 -- 172.21.15.5:0/4238532167 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f086015e050 msgr2=0x7f0848005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.932+0000 7f0866351640 1 --2- 172.21.15.5:0/4238532167 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f086015e050 0x7f0848005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.932+0000 7f0866351640 1 -- 172.21.15.5:0/4238532167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0848004800 msgr2=0x7f0848004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.932+0000 7f0866351640 1 --2- 172.21.15.5:0/4238532167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0848004800 0x7f0848004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.932+0000 7f0866351640 1 -- 172.21.15.5:0/4238532167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0848003520 con 0x7f08480039e0 2024-05-20T18:37:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.932+0000 7f0866b52640 1 --2- 172.21.15.5:0/4238532167 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f086015e050 0x7f0848005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.933+0000 7f0866351640 1 --2- 172.21.15.5:0/4238532167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08480039e0 0x7f0848003e10 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f085800c210 tx=0x7f0848005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a2bafd10bc4ca9 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.933+0000 7f086534f640 1 -- 172.21.15.5:0/4238532167 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0858007d10 con 0x7f08480039e0 2024-05-20T18:37:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.933+0000 7f086534f640 1 -- 172.21.15.5:0/4238532167 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08580058d0 con 0x7f08480039e0 2024-05-20T18:37:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.933+0000 7f086534f640 1 -- 172.21.15.5:0/4238532167 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0858005ce0 con 0x7f08480039e0 2024-05-20T18:37:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.934+0000 7f08685b5640 1 -- 172.21.15.5:0/4238532167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08480039e0 msgr2=0x7f0848003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.934+0000 7f08685b5640 1 --2- 172.21.15.5:0/4238532167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08480039e0 0x7f0848003e10 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f085800c210 tx=0x7f0848005830 comp rx=0 tx=0).stop 2024-05-20T18:37:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.934+0000 7f0866b52640 1 -- 172.21.15.5:0/4238532167 reap_dead start 2024-05-20T18:37:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.934+0000 7f08685b5640 1 -- 172.21.15.5:0/4238532167 shutdown_connections 2024-05-20T18:37:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.934+0000 7f08685b5640 1 -- 172.21.15.5:0/4238532167 >> 172.21.15.5:0/4238532167 conn(0x7f08600b6670 msgr2=0x7f08600b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.934+0000 7f08685b5640 1 -- 172.21.15.5:0/4238532167 shutdown_connections 2024-05-20T18:37:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.934+0000 7f08685b5640 1 -- 172.21.15.5:0/4238532167 wait complete. 2024-05-20T18:37:50.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.936+0000 7f08685b5640 1 Processor -- start 2024-05-20T18:37:50.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.936+0000 7f08685b5640 1 -- start start 2024-05-20T18:37:50.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.937+0000 7f08685b5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08480039e0 0x7f086015f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:50.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.937+0000 7f08685b5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08600ce3b0 0x7f086015fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:50.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.937+0000 7f08685b5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f086009df80 0x7f086009e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:50.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.937+0000 7f08685b5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08600b3320 con 0x7f08600ce3b0 2024-05-20T18:37:50.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.937+0000 7f08685b5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f08600d1a30 con 0x7f086009df80 2024-05-20T18:37:50.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.937+0000 7f08685b5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f086015e460 con 0x7f08480039e0 2024-05-20T18:37:50.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.937+0000 7f0865b50640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08600ce3b0 0x7f086015fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:50.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.938+0000 7f0865b50640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08600ce3b0 0x7f086015fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35708/0 (socket says 172.21.15.5:35708) 2024-05-20T18:37:50.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.938+0000 7f0865b50640 1 -- 172.21.15.5:0/791109014 learned_addr learned my addr 172.21.15.5:0/791109014 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:50.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.938+0000 7f0866b52640 1 --2- 172.21.15.5:0/791109014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f086009df80 0x7f086009e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:50.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.938+0000 7f0866351640 1 --2- 172.21.15.5:0/791109014 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08480039e0 0x7f086015f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:50.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.938+0000 7f0865b50640 1 -- 172.21.15.5:0/791109014 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08480039e0 msgr2=0x7f086015f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:50.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.938+0000 7f0865b50640 1 --2- 172.21.15.5:0/791109014 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08480039e0 0x7f086015f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:50.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.938+0000 7f0865b50640 1 -- 172.21.15.5:0/791109014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f086009df80 msgr2=0x7f086009e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:50.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.938+0000 7f0865b50640 1 --2- 172.21.15.5:0/791109014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f086009df80 0x7f086009e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:50.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.938+0000 7f0865b50640 1 -- 172.21.15.5:0/791109014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f086015e7e0 con 0x7f08600ce3b0 2024-05-20T18:37:50.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.938+0000 7f0865b50640 1 --2- 172.21.15.5:0/791109014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08600ce3b0 0x7f086015fdc0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f085400fb90 tx=0x7f085400d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:50.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.938+0000 7f085effd640 1 -- 172.21.15.5:0/791109014 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085400dc30 con 0x7f08600ce3b0 2024-05-20T18:37:50.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.938+0000 7f085effd640 1 -- 172.21.15.5:0/791109014 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0854018070 con 0x7f08600ce3b0 2024-05-20T18:37:50.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.938+0000 7f08685b5640 1 -- 172.21.15.5:0/791109014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f086015eac0 con 0x7f08600ce3b0 2024-05-20T18:37:50.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.938+0000 7f085effd640 1 -- 172.21.15.5:0/791109014 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08540039c0 con 0x7f08600ce3b0 2024-05-20T18:37:50.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.938+0000 7f08685b5640 1 -- 172.21.15.5:0/791109014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f086015f080 con 0x7f08600ce3b0 2024-05-20T18:37:50.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.939+0000 7f08685b5640 1 -- 172.21.15.5:0/791109014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0860075b00 con 0x7f08600ce3b0 2024-05-20T18:37:50.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.940+0000 7f085effd640 1 -- 172.21.15.5:0/791109014 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0854003b60 con 0x7f08600ce3b0 2024-05-20T18:37:50.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.940+0000 7f085effd640 1 --2- 172.21.15.5:0/791109014 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f084c06be30 0x7f084c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:50.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.940+0000 7f085effd640 1 -- 172.21.15.5:0/791109014 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f085408b540 con 0x7f08600ce3b0 2024-05-20T18:37:50.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.941+0000 7f0866351640 1 --2- 172.21.15.5:0/791109014 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f084c06be30 0x7f084c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:50.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.941+0000 7f0866351640 1 --2- 172.21.15.5:0/791109014 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f084c06be30 0x7f084c06e340 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f0858033000 tx=0x7f0858009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:50.945+0000 7f085effd640 1 -- 172.21.15.5:0/791109014 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0854059b80 con 0x7f08600ce3b0 2024-05-20T18:37:51.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.132+0000 7f08685b5640 1 -- 172.21.15.5:0/791109014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f08600b3ea0 con 0x7f08600ce3b0 2024-05-20T18:37:51.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.133+0000 7f085effd640 1 -- 172.21.15.5:0/791109014 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+17089 (secure 0 0 0) 0x7f0854059480 con 0x7f08600ce3b0 2024-05-20T18:37:51.134 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:51.135 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":121,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:37:39.699377+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:37:12.797471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-20T18:37:13.346156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6837/4273429943":"2024-05-21T18:37:39.699336+0000","172.21.15.63:6835/2352520472":"2024-05-21T18:37:28.901938+0000","172.21.15.5:6835/4273429943":"2024-05-21T18:37:39.699336+0000","172.21.15.63:6833/2352520472":"2024-05-21T18:37:28.901938+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:37:51.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.135+0000 7f08685b5640 1 -- 172.21.15.5:0/791109014 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f084c06be30 msgr2=0x7f084c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.135+0000 7f08685b5640 1 --2- 172.21.15.5:0/791109014 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f084c06be30 0x7f084c06e340 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f0858033000 tx=0x7f0858009030 comp rx=0 tx=0).stop 2024-05-20T18:37:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.135+0000 7f08685b5640 1 -- 172.21.15.5:0/791109014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08600ce3b0 msgr2=0x7f086015fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.135+0000 7f08685b5640 1 --2- 172.21.15.5:0/791109014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08600ce3b0 0x7f086015fdc0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f085400fb90 tx=0x7f085400d520 comp rx=0 tx=0).stop 2024-05-20T18:37:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.135+0000 7f08685b5640 1 -- 172.21.15.5:0/791109014 shutdown_connections 2024-05-20T18:37:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.135+0000 7f08685b5640 1 --2- 172.21.15.5:0/791109014 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f084c06be30 0x7f084c06e340 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.135+0000 7f08685b5640 1 --2- 172.21.15.5:0/791109014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f086009df80 0x7f086009e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.135+0000 7f08685b5640 1 --2- 172.21.15.5:0/791109014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08600ce3b0 0x7f086015fdc0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.135+0000 7f08685b5640 1 --2- 172.21.15.5:0/791109014 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08480039e0 0x7f086015f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.135+0000 7f08685b5640 1 -- 172.21.15.5:0/791109014 >> 172.21.15.5:0/791109014 conn(0x7f08600b6670 msgr2=0x7f0860067490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.135+0000 7f08685b5640 1 -- 172.21.15.5:0/791109014 shutdown_connections 2024-05-20T18:37:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.135+0000 7f08685b5640 1 -- 172.21.15.5:0/791109014 wait complete. 2024-05-20T18:37:51.148 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-20T18:27:02.619092+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-20T18:37:12.797471+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-05-20T18:37:13.346156+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-20T18:37:51.148 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-20T18:37:51.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.265+0000 7fdce29b7640 1 Processor -- start 2024-05-20T18:37:51.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.265+0000 7fdce29b7640 1 -- start start 2024-05-20T18:37:51.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.266+0000 7fdce29b7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcdc162820 0x7fdcdc0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:51.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.266+0000 7fdce29b7640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcdc0d0630 con 0x7fdcdc0b3e80 2024-05-20T18:37:51.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.266+0000 7fdce29b7640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcdc0d39c0 con 0x7fdcdc06dae0 2024-05-20T18:37:51.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.266+0000 7fdce29b7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdcdc0d3ca0 con 0x7fdcdc162820 2024-05-20T18:37:51.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.267+0000 7fdcd3fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fdcdc0b3e80 0x7fdcdc06d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44810/0 (socket says 172.21.15.5:44810) 2024-05-20T18:37:51.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.267+0000 7fdcd3fff640 1 -- 172.21.15.5:0/859081269 learned_addr learned my addr 172.21.15.5:0/859081269 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:51.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.267+0000 7fdcdb7fe640 1 -- 172.21.15.5:0/859081269 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4222447664 0 0) 0x7fdcdc0d0630 con 0x7fdcdc0b3e80 2024-05-20T18:37:51.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.267+0000 7fdcdbfff640 1 --2- 172.21.15.5:0/859081269 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcdc162820 0x7fdcdc0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:51.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.267+0000 7fdcdb7fe640 1 -- 172.21.15.5:0/859081269 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdcc8003520 con 0x7fdcdc0b3e80 2024-05-20T18:37:51.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.267+0000 7fdcdb7fe640 1 -- 172.21.15.5:0/859081269 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2449605736 0 0) 0x7fdcc8003520 con 0x7fdcdc0b3e80 2024-05-20T18:37:51.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.267+0000 7fdcdb7fe640 1 -- 172.21.15.5:0/859081269 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcdc162820 msgr2=0x7fdcdc0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:51.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.267+0000 7fdcdb7fe640 1 --2- 172.21.15.5:0/859081269 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcdc162820 0x7fdcdc0b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:51.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.267+0000 7fdcdb7fe640 1 -- 172.21.15.5:0/859081269 >> v1:172.21.15.63:6789/0 conn(0x7fdcdc06dae0 legacy=0x7fdcdc0b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:51.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.267+0000 7fdcdb7fe640 1 -- 172.21.15.5:0/859081269 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcdc0d3f80 con 0x7fdcdc0b3e80 2024-05-20T18:37:51.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.267+0000 7fdcdb7fe640 1 -- 172.21.15.5:0/859081269 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7fdcc4001670 con 0x7fdcdc0b3e80 2024-05-20T18:37:51.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.267+0000 7fdcdb7fe640 1 -- 172.21.15.5:0/859081269 >> v1:172.21.15.5:6789/0 conn(0x7fdcdc0b3e80 legacy=0x7fdcdc06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:51.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.267+0000 7fdcdb7fe640 1 --2- 172.21.15.5:0/859081269 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcc80039e0 0x7fdcc8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:51.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.267+0000 7fdcdb7fe640 1 --2- 172.21.15.5:0/859081269 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcdc162820 0x7fdcc8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:51.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.267+0000 7fdcdb7fe640 1 --2- 172.21.15.5:0/859081269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc8004d40 0x7fdcc8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:51.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.267+0000 7fdcdb7fe640 1 -- 172.21.15.5:0/859081269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc0d3ca0 con 0x7fdcc8004d40 2024-05-20T18:37:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.267+0000 7fdcdb7fe640 1 -- 172.21.15.5:0/859081269 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc800bbf0 con 0x7fdcc80039e0 2024-05-20T18:37:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.267+0000 7fdcdb7fe640 1 -- 172.21.15.5:0/859081269 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc800bdf0 con 0x7fdcdc162820 2024-05-20T18:37:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.267+0000 7fdce0f54640 1 --2- 172.21.15.5:0/859081269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc8004d40 0x7fdcc8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.267+0000 7fdcdbfff640 1 --2- 172.21.15.5:0/859081269 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcc80039e0 0x7fdcc8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.267+0000 7fdcd3fff640 1 --2- 172.21.15.5:0/859081269 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcdc162820 0x7fdcc8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.268+0000 7fdce0f54640 1 -- 172.21.15.5:0/859081269 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcdc162820 msgr2=0x7fdcc8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.268+0000 7fdce0f54640 1 --2- 172.21.15.5:0/859081269 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcdc162820 0x7fdcc8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.268+0000 7fdce0f54640 1 -- 172.21.15.5:0/859081269 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcc80039e0 msgr2=0x7fdcc8003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.268+0000 7fdce0f54640 1 --2- 172.21.15.5:0/859081269 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcc80039e0 0x7fdcc8003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.268+0000 7fdce0f54640 1 -- 172.21.15.5:0/859081269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc8003520 con 0x7fdcc8004d40 2024-05-20T18:37:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.268+0000 7fdce0f54640 1 --2- 172.21.15.5:0/859081269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc8004d40 0x7fdcc8005210 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4001e50 tx=0x7fdcc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56453f855f870c2e server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.269+0000 7fdcdb7fe640 1 -- 172.21.15.5:0/859081269 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4001a70 con 0x7fdcc8004d40 2024-05-20T18:37:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.269+0000 7fdcdb7fe640 1 -- 172.21.15.5:0/859081269 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcd4003970 con 0x7fdcc8004d40 2024-05-20T18:37:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.269+0000 7fdcdb7fe640 1 -- 172.21.15.5:0/859081269 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4003d60 con 0x7fdcc8004d40 2024-05-20T18:37:51.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.269+0000 7fdce29b7640 1 -- 172.21.15.5:0/859081269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc8004d40 msgr2=0x7fdcc8005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:51.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.269+0000 7fdce29b7640 1 --2- 172.21.15.5:0/859081269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc8004d40 0x7fdcc8005210 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4001e50 tx=0x7fdcc8005830 comp rx=0 tx=0).stop 2024-05-20T18:37:51.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.269+0000 7fdce0f54640 1 -- 172.21.15.5:0/859081269 reap_dead start 2024-05-20T18:37:51.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.269+0000 7fdce29b7640 1 -- 172.21.15.5:0/859081269 shutdown_connections 2024-05-20T18:37:51.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.269+0000 7fdce29b7640 1 -- 172.21.15.5:0/859081269 >> 172.21.15.5:0/859081269 conn(0x7fdcdc0d4bd0 msgr2=0x7fdcdc0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:51.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.269+0000 7fdce29b7640 1 -- 172.21.15.5:0/859081269 shutdown_connections 2024-05-20T18:37:51.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.269+0000 7fdce29b7640 1 -- 172.21.15.5:0/859081269 wait complete. 2024-05-20T18:37:51.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.271+0000 7fdce29b7640 1 Processor -- start 2024-05-20T18:37:51.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.271+0000 7fdce29b7640 1 -- start start 2024-05-20T18:37:51.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.272+0000 7fdce29b7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcc8004d40 0x7fdcdc154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:51.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.272+0000 7fdce29b7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcdc154f60 0x7fdcdc0cd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:51.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.272+0000 7fdce29b7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcdc155920 0x7fdcdc0b5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:51.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.272+0000 7fdce29b7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc0d4780 con 0x7fdcdc155920 2024-05-20T18:37:51.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.272+0000 7fdce29b7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc0d0300 con 0x7fdcc8004d40 2024-05-20T18:37:51.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.272+0000 7fdce29b7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdcdc0d4900 con 0x7fdcdc154f60 2024-05-20T18:37:51.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.273+0000 7fdce0f54640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcdc155920 0x7fdcdc0b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:51.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.273+0000 7fdce0f54640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcdc155920 0x7fdcdc0b5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35728/0 (socket says 172.21.15.5:35728) 2024-05-20T18:37:51.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.273+0000 7fdce0f54640 1 -- 172.21.15.5:0/3723381080 learned_addr learned my addr 172.21.15.5:0/3723381080 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:51.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.273+0000 7fdcd3fff640 1 --2- 172.21.15.5:0/3723381080 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcdc154f60 0x7fdcdc0cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:51.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.273+0000 7fdcdbfff640 1 --2- 172.21.15.5:0/3723381080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcc8004d40 0x7fdcdc154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:51.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.273+0000 7fdce0f54640 1 -- 172.21.15.5:0/3723381080 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcdc154f60 msgr2=0x7fdcdc0cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.273+0000 7fdce0f54640 1 --2- 172.21.15.5:0/3723381080 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcdc154f60 0x7fdcdc0cd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.273+0000 7fdce0f54640 1 -- 172.21.15.5:0/3723381080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcc8004d40 msgr2=0x7fdcdc154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.273+0000 7fdce0f54640 1 --2- 172.21.15.5:0/3723381080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcc8004d40 0x7fdcdc154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.273+0000 7fdce0f54640 1 -- 172.21.15.5:0/3723381080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcdc0ce400 con 0x7fdcdc155920 2024-05-20T18:37:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.273+0000 7fdcdbfff640 1 --2- 172.21.15.5:0/3723381080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcc8004d40 0x7fdcdc154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.273+0000 7fdcd3fff640 1 --2- 172.21.15.5:0/3723381080 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcdc154f60 0x7fdcdc0cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.273+0000 7fdce0f54640 1 --2- 172.21.15.5:0/3723381080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcdc155920 0x7fdcdc0b5150 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4001f70 tx=0x7fdcd40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.273+0000 7fdcd97fa640 1 -- 172.21.15.5:0/3723381080 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4037040 con 0x7fdcdc155920 2024-05-20T18:37:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.273+0000 7fdcd97fa640 1 -- 172.21.15.5:0/3723381080 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcd4003e50 con 0x7fdcdc155920 2024-05-20T18:37:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.273+0000 7fdce29b7640 1 -- 172.21.15.5:0/3723381080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcdc0ce6e0 con 0x7fdcdc155920 2024-05-20T18:37:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.273+0000 7fdcd97fa640 1 -- 172.21.15.5:0/3723381080 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4007db0 con 0x7fdcdc155920 2024-05-20T18:37:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.273+0000 7fdce29b7640 1 -- 172.21.15.5:0/3723381080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcdc0cd130 con 0x7fdcdc155920 2024-05-20T18:37:51.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.275+0000 7fdce29b7640 1 -- 172.21.15.5:0/3723381080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcdc075b00 con 0x7fdcdc155920 2024-05-20T18:37:51.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.275+0000 7fdcd97fa640 1 -- 172.21.15.5:0/3723381080 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdcd4042020 con 0x7fdcdc155920 2024-05-20T18:37:51.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.276+0000 7fdcd97fa640 1 --2- 172.21.15.5:0/3723381080 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdcbc06be30 0x7fdcbc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:51.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.276+0000 7fdcdbfff640 1 --2- 172.21.15.5:0/3723381080 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdcbc06be30 0x7fdcbc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:51.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.276+0000 7fdcd97fa640 1 -- 172.21.15.5:0/3723381080 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7fdcd40aa810 con 0x7fdcdc155920 2024-05-20T18:37:51.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.276+0000 7fdcdbfff640 1 --2- 172.21.15.5:0/3723381080 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdcbc06be30 0x7fdcbc06e340 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fdccc006fd0 tx=0x7fdccc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:51.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.281+0000 7fdcd97fa640 1 -- 172.21.15.5:0/3723381080 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fdcd4078fd0 con 0x7fdcdc155920 2024-05-20T18:37:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.463+0000 7fdce29b7640 1 -- 172.21.15.5:0/3723381080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdcdc068a90 con 0x7fdcdc155920 2024-05-20T18:37:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.463+0000 7fdcd97fa640 1 -- 172.21.15.5:0/3723381080 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) ==== 93+0+10 (secure 0 0 0) 0x7fdcd40788d0 con 0x7fdcdc155920 2024-05-20T18:37:51.464 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-05-20T18:37:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.465+0000 7fdce29b7640 1 -- 172.21.15.5:0/3723381080 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdcbc06be30 msgr2=0x7fdcbc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.465+0000 7fdce29b7640 1 --2- 172.21.15.5:0/3723381080 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdcbc06be30 0x7fdcbc06e340 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fdccc006fd0 tx=0x7fdccc008040 comp rx=0 tx=0).stop 2024-05-20T18:37:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.465+0000 7fdce29b7640 1 -- 172.21.15.5:0/3723381080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcdc155920 msgr2=0x7fdcdc0b5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.465+0000 7fdce29b7640 1 --2- 172.21.15.5:0/3723381080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcdc155920 0x7fdcdc0b5150 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4001f70 tx=0x7fdcd40014a0 comp rx=0 tx=0).stop 2024-05-20T18:37:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.465+0000 7fdce29b7640 1 -- 172.21.15.5:0/3723381080 shutdown_connections 2024-05-20T18:37:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.465+0000 7fdce29b7640 1 --2- 172.21.15.5:0/3723381080 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fdcbc06be30 0x7fdcbc06e340 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.465+0000 7fdce29b7640 1 --2- 172.21.15.5:0/3723381080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcdc155920 0x7fdcdc0b5150 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.465+0000 7fdce29b7640 1 --2- 172.21.15.5:0/3723381080 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdcdc154f60 0x7fdcdc0cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.465+0000 7fdce29b7640 1 --2- 172.21.15.5:0/3723381080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdcc8004d40 0x7fdcdc154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.465+0000 7fdce29b7640 1 -- 172.21.15.5:0/3723381080 >> 172.21.15.5:0/3723381080 conn(0x7fdcdc0d4bd0 msgr2=0x7fdcdc0b82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.465+0000 7fdce29b7640 1 -- 172.21.15.5:0/3723381080 shutdown_connections 2024-05-20T18:37:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.465+0000 7fdce29b7640 1 -- 172.21.15.5:0/3723381080 wait complete. 2024-05-20T18:37:51.478 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-20T18:37:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.595+0000 7f679dad3640 1 Processor -- start 2024-05-20T18:37:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.595+0000 7f679dad3640 1 -- start start 2024-05-20T18:37:51.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.596+0000 7f679dad3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6798162820 0x7f67980b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:51.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.596+0000 7f679dad3640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67980d0630 con 0x7f67980b3e80 2024-05-20T18:37:51.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.596+0000 7f679dad3640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67980d39c0 con 0x7f679806dae0 2024-05-20T18:37:51.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.596+0000 7f679dad3640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f67980d3ca0 con 0x7f6798162820 2024-05-20T18:37:51.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.597+0000 7f67977fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6798162820 0x7f67980b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:51.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.597+0000 7f6796ffd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f67980b3e80 0x7f679806d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44822/0 (socket says 172.21.15.5:44822) 2024-05-20T18:37:51.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.597+0000 7f6796ffd640 1 -- 172.21.15.5:0/3605625480 learned_addr learned my addr 172.21.15.5:0/3605625480 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:51.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.597+0000 7f67977fe640 1 -- 172.21.15.5:0/3605625480 >> v1:172.21.15.63:6789/0 conn(0x7f679806dae0 legacy=0x7f67980b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:51.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.597+0000 7f67977fe640 1 -- 172.21.15.5:0/3605625480 >> v1:172.21.15.5:6789/0 conn(0x7f67980b3e80 legacy=0x7f679806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:51.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.597+0000 7f67967fc640 1 -- 172.21.15.5:0/3605625480 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4049608605 0 0) 0x7f67980d0630 con 0x7f67980b3e80 2024-05-20T18:37:51.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.597+0000 7f67977fe640 1 -- 172.21.15.5:0/3605625480 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67980d3f80 con 0x7f6798162820 2024-05-20T18:37:51.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.598+0000 7f67977fe640 1 --2- 172.21.15.5:0/3605625480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6798162820 0x7f67980b3940 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f67880097a0 tx=0x7f678802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fcc336f0f5ddc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:51.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.598+0000 7f67967fc640 1 -- 172.21.15.5:0/3605625480 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6788037070 con 0x7f6798162820 2024-05-20T18:37:51.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.598+0000 7f67967fc640 1 -- 172.21.15.5:0/3605625480 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6788032440 con 0x7f6798162820 2024-05-20T18:37:51.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.598+0000 7f67967fc640 1 -- 172.21.15.5:0/3605625480 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6788032810 con 0x7f6798162820 2024-05-20T18:37:51.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.598+0000 7f679dad3640 1 -- 172.21.15.5:0/3605625480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6798162820 msgr2=0x7f67980b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:51.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.598+0000 7f679dad3640 1 --2- 172.21.15.5:0/3605625480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6798162820 0x7f67980b3940 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f67880097a0 tx=0x7f678802a1e0 comp rx=0 tx=0).stop 2024-05-20T18:37:51.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.599+0000 7f679dad3640 1 -- 172.21.15.5:0/3605625480 shutdown_connections 2024-05-20T18:37:51.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.599+0000 7f679dad3640 1 --2- 172.21.15.5:0/3605625480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6798162820 0x7f67980b3940 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:51.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.599+0000 7f679dad3640 1 -- 172.21.15.5:0/3605625480 >> 172.21.15.5:0/3605625480 conn(0x7f67980d4bd0 msgr2=0x7f67980d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:51.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.599+0000 7f679dad3640 1 -- 172.21.15.5:0/3605625480 shutdown_connections 2024-05-20T18:37:51.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.599+0000 7f679dad3640 1 -- 172.21.15.5:0/3605625480 wait complete. 2024-05-20T18:37:51.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.600+0000 7f679dad3640 1 Processor -- start 2024-05-20T18:37:51.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.601+0000 7f679dad3640 1 -- start start 2024-05-20T18:37:51.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.602+0000 7f679dad3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67980b3e80 0x7f67981f43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:51.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.602+0000 7f679dad3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6798162820 0x7f67980b6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:51.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.602+0000 7f679dad3640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67980b6c80 0x7f6798158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:51.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.602+0000 7f679dad3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f67980d4780 con 0x7f6798162820 2024-05-20T18:37:51.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.602+0000 7f679dad3640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f67980d0300 con 0x7f67980b6c80 2024-05-20T18:37:51.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.602+0000 7f679dad3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f67980d4900 con 0x7f67980b3e80 2024-05-20T18:37:51.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.602+0000 7f6796ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6798162820 0x7f67980b6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:51.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.602+0000 7f6796ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6798162820 0x7f67980b6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35752/0 (socket says 172.21.15.5:35752) 2024-05-20T18:37:51.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.602+0000 7f6796ffd640 1 -- 172.21.15.5:0/4287213605 learned_addr learned my addr 172.21.15.5:0/4287213605 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:51.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.602+0000 7f67977fe640 1 --2- 172.21.15.5:0/4287213605 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67980b3e80 0x7f67981f43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:51.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.602+0000 7f6797fff640 1 --2- 172.21.15.5:0/4287213605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67980b6c80 0x7f6798158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:51.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.602+0000 7f6796ffd640 1 -- 172.21.15.5:0/4287213605 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67980b3e80 msgr2=0x7f67981f43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:51.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.602+0000 7f6796ffd640 1 --2- 172.21.15.5:0/4287213605 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67980b3e80 0x7f67981f43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:51.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.602+0000 7f6796ffd640 1 -- 172.21.15.5:0/4287213605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67980b6c80 msgr2=0x7f6798158250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:51.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.602+0000 7f6796ffd640 1 --2- 172.21.15.5:0/4287213605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67980b6c80 0x7f6798158250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:51.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.602+0000 7f6796ffd640 1 -- 172.21.15.5:0/4287213605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67980b7090 con 0x7f6798162820 2024-05-20T18:37:51.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.603+0000 7f6796ffd640 1 --2- 172.21.15.5:0/4287213605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6798162820 0x7f67980b6740 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f678c00c460 tx=0x7f678c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:51.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.603+0000 7f677ffff640 1 -- 172.21.15.5:0/4287213605 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c018070 con 0x7f6798162820 2024-05-20T18:37:51.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.603+0000 7f677ffff640 1 -- 172.21.15.5:0/4287213605 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f678c007500 con 0x7f6798162820 2024-05-20T18:37:51.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.603+0000 7f677ffff640 1 -- 172.21.15.5:0/4287213605 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c004a20 con 0x7f6798162820 2024-05-20T18:37:51.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.603+0000 7f679dad3640 1 -- 172.21.15.5:0/4287213605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67981f4910 con 0x7f6798162820 2024-05-20T18:37:51.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.603+0000 7f679dad3640 1 -- 172.21.15.5:0/4287213605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67980b5380 con 0x7f6798162820 2024-05-20T18:37:51.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.604+0000 7f679dad3640 1 -- 172.21.15.5:0/4287213605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6798075b00 con 0x7f6798162820 2024-05-20T18:37:51.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.604+0000 7f677ffff640 1 -- 172.21.15.5:0/4287213605 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f678c01b070 con 0x7f6798162820 2024-05-20T18:37:51.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.605+0000 7f677ffff640 1 --2- 172.21.15.5:0/4287213605 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f678406be30 0x7f678406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:51.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.605+0000 7f677ffff640 1 -- 172.21.15.5:0/4287213605 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f678c08a370 con 0x7f6798162820 2024-05-20T18:37:51.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.605+0000 7f67977fe640 1 --2- 172.21.15.5:0/4287213605 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f678406be30 0x7f678406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:51.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.605+0000 7f67977fe640 1 --2- 172.21.15.5:0/4287213605 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f678406be30 0x7f678406e340 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f6788034000 tx=0x7f678802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:51.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.609+0000 7f677ffff640 1 -- 172.21.15.5:0/4287213605 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f67980b7090 con 0x7f6798162820 2024-05-20T18:37:51.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.795+0000 7f679dad3640 1 -- 172.21.15.5:0/4287213605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f679806c530 con 0x7f6798162820 2024-05-20T18:37:51.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.796+0000 7f677ffff640 1 -- 172.21.15.5:0/4287213605 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) ==== 104+0+11 (secure 0 0 0) 0x7f678c058360 con 0x7f6798162820 2024-05-20T18:37:51.797 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-05-20T18:37:51.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.798+0000 7f679dad3640 1 -- 172.21.15.5:0/4287213605 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f678406be30 msgr2=0x7f678406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:51.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.798+0000 7f679dad3640 1 --2- 172.21.15.5:0/4287213605 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f678406be30 0x7f678406e340 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f6788034000 tx=0x7f678802e000 comp rx=0 tx=0).stop 2024-05-20T18:37:51.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.798+0000 7f679dad3640 1 -- 172.21.15.5:0/4287213605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6798162820 msgr2=0x7f67980b6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:51.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.798+0000 7f679dad3640 1 --2- 172.21.15.5:0/4287213605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6798162820 0x7f67980b6740 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f678c00c460 tx=0x7f678c00c920 comp rx=0 tx=0).stop 2024-05-20T18:37:51.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.798+0000 7f679dad3640 1 -- 172.21.15.5:0/4287213605 shutdown_connections 2024-05-20T18:37:51.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.798+0000 7f679dad3640 1 --2- 172.21.15.5:0/4287213605 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f678406be30 0x7f678406e340 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:51.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.798+0000 7f679dad3640 1 --2- 172.21.15.5:0/4287213605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67980b6c80 0x7f6798158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:51.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.798+0000 7f679dad3640 1 --2- 172.21.15.5:0/4287213605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6798162820 0x7f67980b6740 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:51.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.798+0000 7f679dad3640 1 --2- 172.21.15.5:0/4287213605 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67980b3e80 0x7f67981f43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:51.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.798+0000 7f679dad3640 1 -- 172.21.15.5:0/4287213605 >> 172.21.15.5:0/4287213605 conn(0x7f67980d4bd0 msgr2=0x7f6798156550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:51.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.798+0000 7f679dad3640 1 -- 172.21.15.5:0/4287213605 shutdown_connections 2024-05-20T18:37:51.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.798+0000 7f679dad3640 1 -- 172.21.15.5:0/4287213605 wait complete. 2024-05-20T18:37:51.811 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-20T18:37:51.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.927+0000 7f8fb8a26640 1 Processor -- start 2024-05-20T18:37:51.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.927+0000 7f8fb8a26640 1 -- start start 2024-05-20T18:37:51.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.929+0000 7f8fb8a26640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fb4162820 0x7f8fb40b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:51.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.929+0000 7f8fb8a26640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8fb40b3e80 0x7f8fb406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:51.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.929+0000 7f8fb8a26640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fb406dd20 con 0x7f8fb406d910 2024-05-20T18:37:51.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.929+0000 7f8fb8a26640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f8fb40d0630 con 0x7f8fb40b3e80 2024-05-20T18:37:51.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.929+0000 7f8fb8a26640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8fb40d39c0 con 0x7f8fb4162820 2024-05-20T18:37:51.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.929+0000 7f8fb1d9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8fb40b3e80 0x7f8fb406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:51.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.929+0000 7f8fb2d9d640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8fb406d910 0x7f8fb40d27a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44828/0 (socket says 172.21.15.5:44828) 2024-05-20T18:37:51.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.929+0000 7f8fb2d9d640 1 -- 172.21.15.5:0/1863575238 learned_addr learned my addr 172.21.15.5:0/1863575238 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:51.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.929+0000 7f8fb259c640 1 --2- 172.21.15.5:0/1863575238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fb4162820 0x7f8fb40b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:51.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.929+0000 7f8fb1d9b640 1 -- 172.21.15.5:0/1863575238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fb4162820 msgr2=0x7f8fb40b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:51.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.929+0000 7f8fb1d9b640 1 --2- 172.21.15.5:0/1863575238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fb4162820 0x7f8fb40b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:51.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.929+0000 7f8fb1d9b640 1 -- 172.21.15.5:0/1863575238 >> v1:172.21.15.5:6789/0 conn(0x7f8fb406d910 legacy=0x7f8fb40d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:51.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.929+0000 7f8fb1d9b640 1 -- 172.21.15.5:0/1863575238 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb40d3ca0 con 0x7f8fb40b3e80 2024-05-20T18:37:51.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.930+0000 7f8fb1d9b640 1 --2- 172.21.15.5:0/1863575238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb40b3e80 0x7f8fb406d3d0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80097a0 tx=0x7f8fa802a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9fd4bf5c12d100 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:51.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.930+0000 7f8fb159a640 1 -- 172.21.15.5:0/1863575238 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8037070 con 0x7f8fb40b3e80 2024-05-20T18:37:51.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.930+0000 7f8fb159a640 1 -- 172.21.15.5:0/1863575238 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fa802ad10 con 0x7f8fb40b3e80 2024-05-20T18:37:51.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.930+0000 7f8fb159a640 1 -- 172.21.15.5:0/1863575238 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8032660 con 0x7f8fb40b3e80 2024-05-20T18:37:51.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.931+0000 7f8fb8a26640 1 -- 172.21.15.5:0/1863575238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb40b3e80 msgr2=0x7f8fb406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:51.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.931+0000 7f8fb8a26640 1 --2- 172.21.15.5:0/1863575238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb40b3e80 0x7f8fb406d3d0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80097a0 tx=0x7f8fa802a080 comp rx=0 tx=0).stop 2024-05-20T18:37:51.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.931+0000 7f8fb8a26640 1 -- 172.21.15.5:0/1863575238 shutdown_connections 2024-05-20T18:37:51.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.931+0000 7f8fb8a26640 1 --2- 172.21.15.5:0/1863575238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb40b3e80 0x7f8fb406d3d0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:51.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.931+0000 7f8fb8a26640 1 --2- 172.21.15.5:0/1863575238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fb4162820 0x7f8fb40b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:51.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.931+0000 7f8fb8a26640 1 -- 172.21.15.5:0/1863575238 >> 172.21.15.5:0/1863575238 conn(0x7f8fb40d4bd0 msgr2=0x7f8fb40d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:51.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.931+0000 7f8fb8a26640 1 -- 172.21.15.5:0/1863575238 shutdown_connections 2024-05-20T18:37:51.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.931+0000 7f8fb8a26640 1 -- 172.21.15.5:0/1863575238 wait complete. 2024-05-20T18:37:51.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.933+0000 7f8fb8a26640 1 Processor -- start 2024-05-20T18:37:51.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.933+0000 7f8fb8a26640 1 -- start start 2024-05-20T18:37:51.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.934+0000 7f8fb8a26640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8fb406d910 0x7f8fb40ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:51.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.934+0000 7f8fb8a26640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb40b3e80 0x7f8fb4149fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:51.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.934+0000 7f8fb8a26640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fb4162820 0x7f8fb40c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:51.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.934+0000 7f8fb8a26640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb40d0300 con 0x7f8fb40b3e80 2024-05-20T18:37:51.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.934+0000 7f8fb8a26640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb40d4260 con 0x7f8fb406d910 2024-05-20T18:37:51.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.934+0000 7f8fb8a26640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8fb40b91a0 con 0x7f8fb4162820 2024-05-20T18:37:51.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.934+0000 7f8fb1d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb40b3e80 0x7f8fb4149fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:51.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.934+0000 7f8fb1d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb40b3e80 0x7f8fb4149fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35768/0 (socket says 172.21.15.5:35768) 2024-05-20T18:37:51.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.934+0000 7f8fb1d9b640 1 -- 172.21.15.5:0/3457542737 learned_addr learned my addr 172.21.15.5:0/3457542737 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:51.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.934+0000 7f8fb2d9d640 1 --2- 172.21.15.5:0/3457542737 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fb4162820 0x7f8fb40c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:51.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.934+0000 7f8fb259c640 1 --2- 172.21.15.5:0/3457542737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8fb406d910 0x7f8fb40ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:51.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.935+0000 7f8fb1d9b640 1 -- 172.21.15.5:0/3457542737 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fb4162820 msgr2=0x7f8fb40c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:51.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.935+0000 7f8fb1d9b640 1 --2- 172.21.15.5:0/3457542737 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fb4162820 0x7f8fb40c8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:51.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.935+0000 7f8fb1d9b640 1 -- 172.21.15.5:0/3457542737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8fb406d910 msgr2=0x7f8fb40ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:51.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.935+0000 7f8fb1d9b640 1 --2- 172.21.15.5:0/3457542737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8fb406d910 0x7f8fb40ca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:51.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.935+0000 7f8fb1d9b640 1 -- 172.21.15.5:0/3457542737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb414a4e0 con 0x7f8fb40b3e80 2024-05-20T18:37:51.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.935+0000 7f8fb1d9b640 1 --2- 172.21.15.5:0/3457542737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb40b3e80 0x7f8fb4149fa0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80098c0 tx=0x7f8fa80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:51.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.935+0000 7f8fa2ffd640 1 -- 172.21.15.5:0/3457542737 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8037040 con 0x7f8fb40b3e80 2024-05-20T18:37:51.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.935+0000 7f8fa2ffd640 1 -- 172.21.15.5:0/3457542737 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fa803b430 con 0x7f8fb40b3e80 2024-05-20T18:37:51.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.935+0000 7f8fb8a26640 1 -- 172.21.15.5:0/3457542737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb40caef0 con 0x7f8fb40b3e80 2024-05-20T18:37:51.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.935+0000 7f8fa2ffd640 1 -- 172.21.15.5:0/3457542737 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa803b800 con 0x7f8fb40b3e80 2024-05-20T18:37:51.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.935+0000 7f8fb8a26640 1 -- 172.21.15.5:0/3457542737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb40cb4b0 con 0x7f8fb40b3e80 2024-05-20T18:37:51.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.936+0000 7f8fb8a26640 1 -- 172.21.15.5:0/3457542737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb4075b00 con 0x7f8fb40b3e80 2024-05-20T18:37:51.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.937+0000 7f8fa2ffd640 1 -- 172.21.15.5:0/3457542737 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8fa8044020 con 0x7f8fb40b3e80 2024-05-20T18:37:51.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.937+0000 7f8fa2ffd640 1 --2- 172.21.15.5:0/3457542737 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8f9406be30 0x7f8f9406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:51.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.937+0000 7f8fb259c640 1 --2- 172.21.15.5:0/3457542737 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8f9406be30 0x7f8f9406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:51.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.937+0000 7f8fa2ffd640 1 -- 172.21.15.5:0/3457542737 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f8fa80aadb0 con 0x7f8fb40b3e80 2024-05-20T18:37:51.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.938+0000 7f8fb259c640 1 --2- 172.21.15.5:0/3457542737 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8f9406be30 0x7f8f9406e340 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f8fa40079a0 tx=0x7f8fa4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:51.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:51.942+0000 7f8fa2ffd640 1 -- 172.21.15.5:0/3457542737 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8fa804b020 con 0x7f8fb40b3e80 2024-05-20T18:37:52.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.125+0000 7f8fb8a26640 1 -- 172.21.15.5:0/3457542737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8fb41620d0 con 0x7f8fb40b3e80 2024-05-20T18:37:52.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.126+0000 7f8fa2ffd640 1 -- 172.21.15.5:0/3457542737 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) ==== 100+0+11 (secure 0 0 0) 0x7f8fa8078ed0 con 0x7f8fb40b3e80 2024-05-20T18:37:52.127 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-05-20T18:37:52.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.128+0000 7f8fb8a26640 1 -- 172.21.15.5:0/3457542737 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8f9406be30 msgr2=0x7f8f9406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:52.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.128+0000 7f8fb8a26640 1 --2- 172.21.15.5:0/3457542737 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8f9406be30 0x7f8f9406e340 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f8fa40079a0 tx=0x7f8fa4008040 comp rx=0 tx=0).stop 2024-05-20T18:37:52.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.128+0000 7f8fb8a26640 1 -- 172.21.15.5:0/3457542737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb40b3e80 msgr2=0x7f8fb4149fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:52.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.128+0000 7f8fb8a26640 1 --2- 172.21.15.5:0/3457542737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb40b3e80 0x7f8fb4149fa0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80098c0 tx=0x7f8fa80046c0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.128+0000 7f8fb8a26640 1 -- 172.21.15.5:0/3457542737 shutdown_connections 2024-05-20T18:37:52.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.128+0000 7f8fb8a26640 1 --2- 172.21.15.5:0/3457542737 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8f9406be30 0x7f8f9406e340 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.128+0000 7f8fb8a26640 1 --2- 172.21.15.5:0/3457542737 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8fb4162820 0x7f8fb40c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.128+0000 7f8fb8a26640 1 --2- 172.21.15.5:0/3457542737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fb40b3e80 0x7f8fb4149fa0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.128+0000 7f8fb8a26640 1 --2- 172.21.15.5:0/3457542737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8fb406d910 0x7f8fb40ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.128+0000 7f8fb8a26640 1 -- 172.21.15.5:0/3457542737 >> 172.21.15.5:0/3457542737 conn(0x7f8fb40d4bd0 msgr2=0x7f8fb415fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:52.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.128+0000 7f8fb8a26640 1 -- 172.21.15.5:0/3457542737 shutdown_connections 2024-05-20T18:37:52.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.128+0000 7f8fb8a26640 1 -- 172.21.15.5:0/3457542737 wait complete. 2024-05-20T18:37:52.141 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:37:52.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.253+0000 7f8176145640 1 Processor -- start 2024-05-20T18:37:52.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.253+0000 7f8176145640 1 -- start start 2024-05-20T18:37:52.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.255+0000 7f8176145640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81700ce7e0 0x7f81700cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:52.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.255+0000 7f8176145640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81700cf130 con 0x7f817015f220 2024-05-20T18:37:52.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.255+0000 7f8176145640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81700cda20 con 0x7f81700cf3d0 2024-05-20T18:37:52.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.255+0000 7f8176145640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f81700cdd00 con 0x7f81700ce7e0 2024-05-20T18:37:52.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.255+0000 7f816f7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f817015f220 0x7f81700d08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44836/0 (socket says 172.21.15.5:44836) 2024-05-20T18:37:52.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.255+0000 7f816f7fe640 1 -- 172.21.15.5:0/2724943938 learned_addr learned my addr 172.21.15.5:0/2724943938 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:52.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.255+0000 7f816ffff640 1 --2- 172.21.15.5:0/2724943938 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81700ce7e0 0x7f81700cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:52.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.255+0000 7f816e7fc640 1 -- 172.21.15.5:0/2724943938 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 108140868 0 0) 0x7f81700cf130 con 0x7f817015f220 2024-05-20T18:37:52.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.255+0000 7f816e7fc640 1 -- 172.21.15.5:0/2724943938 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8158003520 con 0x7f817015f220 2024-05-20T18:37:52.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.256+0000 7f816e7fc640 1 -- 172.21.15.5:0/2724943938 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1615934663 0 0) 0x7f8158003520 con 0x7f817015f220 2024-05-20T18:37:52.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.256+0000 7f816ffff640 1 -- 172.21.15.5:0/2724943938 >> v1:172.21.15.63:6789/0 conn(0x7f81700cf3d0 legacy=0x7f81700cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:52.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.256+0000 7f816ffff640 1 -- 172.21.15.5:0/2724943938 >> v1:172.21.15.5:6789/0 conn(0x7f817015f220 legacy=0x7f81700d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:52.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.256+0000 7f816ffff640 1 -- 172.21.15.5:0/2724943938 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81700cdfe0 con 0x7f81700ce7e0 2024-05-20T18:37:52.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.256+0000 7f816ffff640 1 --2- 172.21.15.5:0/2724943938 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81700ce7e0 0x7f81700cebf0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8168009dc0 tx=0x7f816802a500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bad5eee97b2d0aa server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:52.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.257+0000 7f816e7fc640 1 -- 172.21.15.5:0/2724943938 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168035020 con 0x7f81700ce7e0 2024-05-20T18:37:52.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.257+0000 7f816e7fc640 1 -- 172.21.15.5:0/2724943938 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8168007540 con 0x7f81700ce7e0 2024-05-20T18:37:52.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.257+0000 7f816e7fc640 1 -- 172.21.15.5:0/2724943938 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160001670 con 0x7f81700ce7e0 2024-05-20T18:37:52.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.257+0000 7f8176145640 1 -- 172.21.15.5:0/2724943938 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81700ce7e0 msgr2=0x7f81700cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:52.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.257+0000 7f8176145640 1 --2- 172.21.15.5:0/2724943938 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81700ce7e0 0x7f81700cebf0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8168009dc0 tx=0x7f816802a500 comp rx=0 tx=0).stop 2024-05-20T18:37:52.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.257+0000 7f8176145640 1 -- 172.21.15.5:0/2724943938 shutdown_connections 2024-05-20T18:37:52.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.257+0000 7f8176145640 1 --2- 172.21.15.5:0/2724943938 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81700ce7e0 0x7f81700cebf0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.257+0000 7f8176145640 1 -- 172.21.15.5:0/2724943938 >> 172.21.15.5:0/2724943938 conn(0x7f81700b44d0 msgr2=0x7f81700b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:52.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.257+0000 7f8176145640 1 -- 172.21.15.5:0/2724943938 shutdown_connections 2024-05-20T18:37:52.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.257+0000 7f8176145640 1 -- 172.21.15.5:0/2724943938 wait complete. 2024-05-20T18:37:52.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.259+0000 7f8176145640 1 Processor -- start 2024-05-20T18:37:52.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.259+0000 7f8176145640 1 -- start start 2024-05-20T18:37:52.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.261+0000 7f8176145640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81700ce7e0 0x7f81700b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:52.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.261+0000 7f8176145640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81700cf3d0 0x7f81701f3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:52.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.261+0000 7f8176145640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f817015f220 0x7f81700cb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:52.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.261+0000 7f8176145640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81700cd680 con 0x7f817015f220 2024-05-20T18:37:52.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.261+0000 7f8176145640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f81700d0fd0 con 0x7f81700ce7e0 2024-05-20T18:37:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.261+0000 7f8176145640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f81700cd800 con 0x7f81700cf3d0 2024-05-20T18:37:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.261+0000 7f816ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f817015f220 0x7f81700cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.261+0000 7f816ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f817015f220 0x7f81700cb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35772/0 (socket says 172.21.15.5:35772) 2024-05-20T18:37:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.261+0000 7f816ffff640 1 -- 172.21.15.5:0/1984777626 learned_addr learned my addr 172.21.15.5:0/1984777626 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.261+0000 7f816f7fe640 1 --2- 172.21.15.5:0/1984777626 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81700ce7e0 0x7f81700b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.261+0000 7f816effd640 1 --2- 172.21.15.5:0/1984777626 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81700cf3d0 0x7f81701f3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.261+0000 7f816ffff640 1 -- 172.21.15.5:0/1984777626 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81700cf3d0 msgr2=0x7f81701f3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.261+0000 7f816ffff640 1 --2- 172.21.15.5:0/1984777626 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81700cf3d0 0x7f81701f3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.261+0000 7f816ffff640 1 -- 172.21.15.5:0/1984777626 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81700ce7e0 msgr2=0x7f81700b63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.261+0000 7f816ffff640 1 --2- 172.21.15.5:0/1984777626 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81700ce7e0 0x7f81700b63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.261+0000 7f816ffff640 1 -- 172.21.15.5:0/1984777626 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81700b6cd0 con 0x7f817015f220 2024-05-20T18:37:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.261+0000 7f816ffff640 1 --2- 172.21.15.5:0/1984777626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f817015f220 0x7f81700cb680 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f8168034040 tx=0x7f8168004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.262+0000 7f8157fff640 1 -- 172.21.15.5:0/1984777626 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168035040 con 0x7f817015f220 2024-05-20T18:37:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.262+0000 7f8157fff640 1 -- 172.21.15.5:0/1984777626 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8168031be0 con 0x7f817015f220 2024-05-20T18:37:52.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.262+0000 7f8176145640 1 -- 172.21.15.5:0/1984777626 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81700b6fb0 con 0x7f817015f220 2024-05-20T18:37:52.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.262+0000 7f8157fff640 1 -- 172.21.15.5:0/1984777626 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168009450 con 0x7f817015f220 2024-05-20T18:37:52.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.262+0000 7f8176145640 1 -- 172.21.15.5:0/1984777626 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81700b7570 con 0x7f817015f220 2024-05-20T18:37:52.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.263+0000 7f8176145640 1 -- 172.21.15.5:0/1984777626 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8170075b00 con 0x7f817015f220 2024-05-20T18:37:52.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.263+0000 7f8157fff640 1 -- 172.21.15.5:0/1984777626 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8168003990 con 0x7f817015f220 2024-05-20T18:37:52.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.264+0000 7f8157fff640 1 --2- 172.21.15.5:0/1984777626 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f815c06be60 0x7f815c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:52.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.264+0000 7f8157fff640 1 -- 172.21.15.5:0/1984777626 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f81680ac070 con 0x7f817015f220 2024-05-20T18:37:52.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.264+0000 7f816f7fe640 1 --2- 172.21.15.5:0/1984777626 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f815c06be60 0x7f815c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:52.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.264+0000 7f816f7fe640 1 --2- 172.21.15.5:0/1984777626 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f815c06be60 0x7f815c06e370 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f8160001eb0 tx=0x7f8160000bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:52.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.269+0000 7f8157fff640 1 -- 172.21.15.5:0/1984777626 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8168040020 con 0x7f817015f220 2024-05-20T18:37:52.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.488+0000 7f8176145640 1 -- 172.21.15.5:0/1984777626 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8170162c60 con 0x7f817015f220 2024-05-20T18:37:52.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.489+0000 7f8157fff640 1 -- 172.21.15.5:0/1984777626 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5180 (secure 0 0 0) 0x7f816807a190 con 0x7f817015f220 2024-05-20T18:37:52.490 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:52.491 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":111,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/266630184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":266630184},{"type":"v1","addr":"172.21.15.63:6834","nonce":266630184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7361,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1494022697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":1494022697},{"type":"v1","addr":"172.21.15.63:6835","nonce":1494022697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":9009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/1739221946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1739221946},{"type":"v1","addr":"172.21.15.5:6836","nonce":1739221946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:37:14.365195+0000","modified":"2024-05-20T18:37:44.722140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9444},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9444":{"gid":9444,"name":"c","rank":0,"incarnation":108,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6837/2067544478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":2067544478},{"type":"v1","addr":"172.21.15.5:6837","nonce":2067544478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9444,"qdb_cluster":[9444]},"id":8}]} 2024-05-20T18:37:52.491 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 111 2024-05-20T18:37:52.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.491+0000 7f8176145640 1 -- 172.21.15.5:0/1984777626 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f815c06be60 msgr2=0x7f815c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:52.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.491+0000 7f8176145640 1 --2- 172.21.15.5:0/1984777626 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f815c06be60 0x7f815c06e370 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f8160001eb0 tx=0x7f8160000bc0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.491+0000 7f8176145640 1 -- 172.21.15.5:0/1984777626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f817015f220 msgr2=0x7f81700cb680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:52.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.491+0000 7f8176145640 1 --2- 172.21.15.5:0/1984777626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f817015f220 0x7f81700cb680 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f8168034040 tx=0x7f8168004a40 comp rx=0 tx=0).stop 2024-05-20T18:37:52.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.491+0000 7f8176145640 1 -- 172.21.15.5:0/1984777626 shutdown_connections 2024-05-20T18:37:52.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.491+0000 7f8176145640 1 --2- 172.21.15.5:0/1984777626 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f815c06be60 0x7f815c06e370 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.491+0000 7f8176145640 1 --2- 172.21.15.5:0/1984777626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f817015f220 0x7f81700cb680 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.491+0000 7f8176145640 1 --2- 172.21.15.5:0/1984777626 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81700cf3d0 0x7f81701f3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.491+0000 7f8176145640 1 --2- 172.21.15.5:0/1984777626 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81700ce7e0 0x7f81700b63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.491+0000 7f8176145640 1 -- 172.21.15.5:0/1984777626 >> 172.21.15.5:0/1984777626 conn(0x7f81700b44d0 msgr2=0x7f81700b2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:52.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.491+0000 7f8176145640 1 -- 172.21.15.5:0/1984777626 shutdown_connections 2024-05-20T18:37:52.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.491+0000 7f8176145640 1 -- 172.21.15.5:0/1984777626 wait complete. 2024-05-20T18:37:52.504 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:37:52.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.618+0000 7f34dbaca640 1 Processor -- start 2024-05-20T18:37:52.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.618+0000 7f34dbaca640 1 -- start start 2024-05-20T18:37:52.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.619+0000 7f34dbaca640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f34d40cd710 0x7f34d4162e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:52.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.619+0000 7f34dbaca640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f34d406d6b0 0x7f34d415f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:52.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.619+0000 7f34dbaca640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34d40cf3d0 0x7f34d40cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:52.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.619+0000 7f34dbaca640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f34d40d0ba0 con 0x7f34d406d6b0 2024-05-20T18:37:52.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.619+0000 7f34dbaca640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f34d40cfd20 con 0x7f34d40cd710 2024-05-20T18:37:52.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.619+0000 7f34dbaca640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f34d406c8c0 con 0x7f34d40cf3d0 2024-05-20T18:37:52.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.619+0000 7f34d9065640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f34d406d6b0 0x7f34d415f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:52.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.620+0000 7f34d9065640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f34d406d6b0 0x7f34d415f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35788/0 (socket says 172.21.15.5:35788) 2024-05-20T18:37:52.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.620+0000 7f34d9065640 1 -- 172.21.15.5:0/69953478 learned_addr learned my addr 172.21.15.5:0/69953478 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:52.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.620+0000 7f34da067640 1 --2- 172.21.15.5:0/69953478 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34d40cf3d0 0x7f34d40cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:52.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.620+0000 7f34d9866640 1 --2- 172.21.15.5:0/69953478 >> v2:172.21.15.63:3300/0 conn(0x7f34d40cd710 0x7f34d4162e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:52.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.620+0000 7f34d9065640 1 -- 172.21.15.5:0/69953478 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34d40cf3d0 msgr2=0x7f34d40cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:52.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.620+0000 7f34d9065640 1 --2- 172.21.15.5:0/69953478 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34d40cf3d0 0x7f34d40cf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.620+0000 7f34d9065640 1 -- 172.21.15.5:0/69953478 >> v2:172.21.15.63:3300/0 conn(0x7f34d40cd710 msgr2=0x7f34d4162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:52.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.620+0000 7f34d9065640 1 --2- 172.21.15.5:0/69953478 >> v2:172.21.15.63:3300/0 conn(0x7f34d40cd710 0x7f34d4162e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.620+0000 7f34d9065640 1 -- 172.21.15.5:0/69953478 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d406dac0 con 0x7f34d406d6b0 2024-05-20T18:37:52.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.620+0000 7f34d9866640 1 --2- 172.21.15.5:0/69953478 >> v2:172.21.15.63:3300/0 conn(0x7f34d40cd710 0x7f34d4162e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:52.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.621+0000 7f34d9065640 1 --2- 172.21.15.5:0/69953478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34d406d6b0 0x7f34d415f2b0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f34d406dc90 tx=0x7f34c402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add0a594830d99e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:52.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.621+0000 7f34d8864640 1 -- 172.21.15.5:0/69953478 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c4037070 con 0x7f34d406d6b0 2024-05-20T18:37:52.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.621+0000 7f34d8864640 1 -- 172.21.15.5:0/69953478 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34c4032440 con 0x7f34d406d6b0 2024-05-20T18:37:52.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.621+0000 7f34d8864640 1 -- 172.21.15.5:0/69953478 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c4032810 con 0x7f34d406d6b0 2024-05-20T18:37:52.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.621+0000 7f34dbaca640 1 -- 172.21.15.5:0/69953478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34d406d6b0 msgr2=0x7f34d415f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:52.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.621+0000 7f34dbaca640 1 --2- 172.21.15.5:0/69953478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34d406d6b0 0x7f34d415f2b0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f34d406dc90 tx=0x7f34c402a1c0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.621+0000 7f34dbaca640 1 -- 172.21.15.5:0/69953478 shutdown_connections 2024-05-20T18:37:52.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.621+0000 7f34dbaca640 1 --2- 172.21.15.5:0/69953478 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34d40cf3d0 0x7f34d40cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.621+0000 7f34dbaca640 1 --2- 172.21.15.5:0/69953478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34d406d6b0 0x7f34d415f2b0 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.621+0000 7f34dbaca640 1 --2- 172.21.15.5:0/69953478 >> v2:172.21.15.63:3300/0 conn(0x7f34d40cd710 0x7f34d4162e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.622+0000 7f34dbaca640 1 -- 172.21.15.5:0/69953478 >> 172.21.15.5:0/69953478 conn(0x7f34d40b44d0 msgr2=0x7f34d40b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:52.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.622+0000 7f34dbaca640 1 -- 172.21.15.5:0/69953478 shutdown_connections 2024-05-20T18:37:52.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.622+0000 7f34dbaca640 1 -- 172.21.15.5:0/69953478 wait complete. 2024-05-20T18:37:52.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.623+0000 7f34dbaca640 1 Processor -- start 2024-05-20T18:37:52.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.623+0000 7f34dbaca640 1 -- start start 2024-05-20T18:37:52.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.625+0000 7f34dbaca640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34d406d6b0 0x7f34d4160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:52.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.625+0000 7f34dbaca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34d40cd710 0x7f34d41610e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:52.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.625+0000 7f34dbaca640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34d40cf3d0 0x7f34d415b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:52.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.625+0000 7f34dbaca640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f34d41639a0 con 0x7f34d40cd710 2024-05-20T18:37:52.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.625+0000 7f34dbaca640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f34d41636c0 con 0x7f34d406d6b0 2024-05-20T18:37:52.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.625+0000 7f34dbaca640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f34d4163c80 con 0x7f34d40cf3d0 2024-05-20T18:37:52.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.625+0000 7f34d9065640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34d40cd710 0x7f34d41610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:52.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.625+0000 7f34d9065640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34d40cd710 0x7f34d41610e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35800/0 (socket says 172.21.15.5:35800) 2024-05-20T18:37:52.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.625+0000 7f34d9065640 1 -- 172.21.15.5:0/987808082 learned_addr learned my addr 172.21.15.5:0/987808082 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:52.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.625+0000 7f34d9866640 1 --2- 172.21.15.5:0/987808082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34d406d6b0 0x7f34d4160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:52.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.625+0000 7f34da067640 1 --2- 172.21.15.5:0/987808082 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34d40cf3d0 0x7f34d415b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:52.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.625+0000 7f34d9065640 1 -- 172.21.15.5:0/987808082 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34d40cf3d0 msgr2=0x7f34d415b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:52.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.625+0000 7f34d9065640 1 --2- 172.21.15.5:0/987808082 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34d40cf3d0 0x7f34d415b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.625+0000 7f34d9065640 1 -- 172.21.15.5:0/987808082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34d406d6b0 msgr2=0x7f34d4160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:52.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.625+0000 7f34d9065640 1 --2- 172.21.15.5:0/987808082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34d406d6b0 0x7f34d4160ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.625+0000 7f34d9065640 1 -- 172.21.15.5:0/987808082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d40b5e90 con 0x7f34d40cd710 2024-05-20T18:37:52.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.625+0000 7f34d9065640 1 --2- 172.21.15.5:0/987808082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34d40cd710 0x7f34d41610e0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f34c402a200 tx=0x7f34c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:52.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.625+0000 7f34ca7fc640 1 -- 172.21.15.5:0/987808082 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c4037040 con 0x7f34d40cd710 2024-05-20T18:37:52.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.626+0000 7f34ca7fc640 1 -- 172.21.15.5:0/987808082 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34c4042070 con 0x7f34d40cd710 2024-05-20T18:37:52.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.626+0000 7f34dbaca640 1 -- 172.21.15.5:0/987808082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34d40b6170 con 0x7f34d40cd710 2024-05-20T18:37:52.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.626+0000 7f34ca7fc640 1 -- 172.21.15.5:0/987808082 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c4030bb0 con 0x7f34d40cd710 2024-05-20T18:37:52.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.626+0000 7f34dbaca640 1 -- 172.21.15.5:0/987808082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34d40b6730 con 0x7f34d40cd710 2024-05-20T18:37:52.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.627+0000 7f34dbaca640 1 -- 172.21.15.5:0/987808082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34d4075b00 con 0x7f34d40cd710 2024-05-20T18:37:52.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.627+0000 7f34ca7fc640 1 -- 172.21.15.5:0/987808082 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f34c402ad10 con 0x7f34d40cd710 2024-05-20T18:37:52.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.628+0000 7f34ca7fc640 1 --2- 172.21.15.5:0/987808082 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f34b806be60 0x7f34b806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:52.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.628+0000 7f34ca7fc640 1 -- 172.21.15.5:0/987808082 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f34c40ab900 con 0x7f34d40cd710 2024-05-20T18:37:52.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.628+0000 7f34d9866640 1 --2- 172.21.15.5:0/987808082 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f34b806be60 0x7f34b806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:52.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.628+0000 7f34d9866640 1 --2- 172.21.15.5:0/987808082 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f34b806be60 0x7f34b806e370 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f34d4162d60 tx=0x7f34d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:52.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.632+0000 7f34ca7fc640 1 -- 172.21.15.5:0/987808082 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f34c4079990 con 0x7f34d40cd710 2024-05-20T18:37:52.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.820+0000 7f34dbaca640 1 -- 172.21.15.5:0/987808082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34d40b8940 con 0x7f34d40cd710 2024-05-20T18:37:52.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.821+0000 7f34ca7fc640 1 -- 172.21.15.5:0/987808082 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+17089 (secure 0 0 0) 0x7f34c407dc90 con 0x7f34d40cd710 2024-05-20T18:37:52.822 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:52.822 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":121,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:37:39.699377+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:37:12.797471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-20T18:37:13.346156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6837/4273429943":"2024-05-21T18:37:39.699336+0000","172.21.15.63:6835/2352520472":"2024-05-21T18:37:28.901938+0000","172.21.15.5:6835/4273429943":"2024-05-21T18:37:39.699336+0000","172.21.15.63:6833/2352520472":"2024-05-21T18:37:28.901938+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:37:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.823+0000 7f34dbaca640 1 -- 172.21.15.5:0/987808082 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f34b806be60 msgr2=0x7f34b806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.823+0000 7f34dbaca640 1 --2- 172.21.15.5:0/987808082 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f34b806be60 0x7f34b806e370 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f34d4162d60 tx=0x7f34d0008040 comp rx=0 tx=0).stop 2024-05-20T18:37:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.823+0000 7f34dbaca640 1 -- 172.21.15.5:0/987808082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34d40cd710 msgr2=0x7f34d41610e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.823+0000 7f34dbaca640 1 --2- 172.21.15.5:0/987808082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34d40cd710 0x7f34d41610e0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f34c402a200 tx=0x7f34c40046c0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.823+0000 7f34dbaca640 1 -- 172.21.15.5:0/987808082 shutdown_connections 2024-05-20T18:37:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.823+0000 7f34dbaca640 1 --2- 172.21.15.5:0/987808082 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f34b806be60 0x7f34b806e370 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.823+0000 7f34dbaca640 1 --2- 172.21.15.5:0/987808082 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34d40cf3d0 0x7f34d415b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.823+0000 7f34dbaca640 1 --2- 172.21.15.5:0/987808082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34d40cd710 0x7f34d41610e0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.823+0000 7f34dbaca640 1 --2- 172.21.15.5:0/987808082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34d406d6b0 0x7f34d4160ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.823+0000 7f34dbaca640 1 -- 172.21.15.5:0/987808082 >> 172.21.15.5:0/987808082 conn(0x7f34d40b44d0 msgr2=0x7f34d40b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.823+0000 7f34dbaca640 1 -- 172.21.15.5:0/987808082 shutdown_connections 2024-05-20T18:37:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.823+0000 7f34dbaca640 1 -- 172.21.15.5:0/987808082 wait complete. 2024-05-20T18:37:52.836 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-20T18:37:52.836 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:37:52.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.949+0000 7f8412b38640 1 Processor -- start 2024-05-20T18:37:52.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.950+0000 7f8412b38640 1 -- start start 2024-05-20T18:37:52.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.951+0000 7f8412b38640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f840c0af030 0x7f840c0af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:52.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.951+0000 7f8412b38640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f840c06c180 0x7f840c06c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:52.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.951+0000 7f8412b38640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f840c0af980 con 0x7f840c15f700 2024-05-20T18:37:52.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.951+0000 7f8412b38640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f840c06bcf0 con 0x7f840c06c180 2024-05-20T18:37:52.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.951+0000 7f8412b38640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f840c160430 con 0x7f840c0af030 2024-05-20T18:37:52.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.951+0000 7f84110d5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f840c06c180 0x7f840c06c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:52.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.951+0000 7f840bfff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f840c15f700 0x7f840c15fb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44846/0 (socket says 172.21.15.5:44846) 2024-05-20T18:37:52.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.951+0000 7f840bfff640 1 -- 172.21.15.5:0/1358478853 learned_addr learned my addr 172.21.15.5:0/1358478853 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:52.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.951+0000 7f84110d5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f840c06c180 0x7f840c06c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35804/0 (socket says 172.21.15.5:35804) 2024-05-20T18:37:52.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.951+0000 7f84108d4640 1 --2- 172.21.15.5:0/1358478853 >> v2:172.21.15.63:3300/0 conn(0x7f840c0af030 0x7f840c0af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:52.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.952+0000 7f84110d5640 1 -- 172.21.15.5:0/1358478853 >> v2:172.21.15.63:3300/0 conn(0x7f840c0af030 msgr2=0x7f840c0af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:52.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.952+0000 7f84110d5640 1 --2- 172.21.15.5:0/1358478853 >> v2:172.21.15.63:3300/0 conn(0x7f840c0af030 0x7f840c0af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.952+0000 7f84110d5640 1 -- 172.21.15.5:0/1358478853 >> v1:172.21.15.5:6789/0 conn(0x7f840c15f700 legacy=0x7f840c15fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:52.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.952+0000 7f840b7fe640 1 -- 172.21.15.5:0/1358478853 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2405085009 0 0) 0x7f8400001110 con 0x7f840c15f700 2024-05-20T18:37:52.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.952+0000 7f84110d5640 1 -- 172.21.15.5:0/1358478853 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f840c06b6a0 con 0x7f840c06c180 2024-05-20T18:37:52.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.952+0000 7f84110d5640 1 --2- 172.21.15.5:0/1358478853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f840c06c180 0x7f840c06c590 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f840c06f310 tx=0x7f83fc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c88d9c427172b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:52.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.952+0000 7f840b7fe640 1 -- 172.21.15.5:0/1358478853 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc037070 con 0x7f840c06c180 2024-05-20T18:37:52.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.953+0000 7f840b7fe640 1 -- 172.21.15.5:0/1358478853 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83fc032440 con 0x7f840c06c180 2024-05-20T18:37:52.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.953+0000 7f840b7fe640 1 -- 172.21.15.5:0/1358478853 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc032810 con 0x7f840c06c180 2024-05-20T18:37:52.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.953+0000 7f8412b38640 1 -- 172.21.15.5:0/1358478853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f840c06c180 msgr2=0x7f840c06c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:52.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.953+0000 7f8412b38640 1 --2- 172.21.15.5:0/1358478853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f840c06c180 0x7f840c06c590 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f840c06f310 tx=0x7f83fc02a1a0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.953+0000 7f8412b38640 1 -- 172.21.15.5:0/1358478853 shutdown_connections 2024-05-20T18:37:52.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.953+0000 7f8412b38640 1 --2- 172.21.15.5:0/1358478853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f840c06c180 0x7f840c06c590 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.953+0000 7f8412b38640 1 --2- 172.21.15.5:0/1358478853 >> v2:172.21.15.63:3300/0 conn(0x7f840c0af030 0x7f840c0af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.953+0000 7f8412b38640 1 -- 172.21.15.5:0/1358478853 >> 172.21.15.5:0/1358478853 conn(0x7f840c0b6620 msgr2=0x7f840c0b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:52.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.953+0000 7f8412b38640 1 -- 172.21.15.5:0/1358478853 shutdown_connections 2024-05-20T18:37:52.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.953+0000 7f8412b38640 1 -- 172.21.15.5:0/1358478853 wait complete. 2024-05-20T18:37:52.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.955+0000 7f8412b38640 1 Processor -- start 2024-05-20T18:37:52.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.955+0000 7f8412b38640 1 -- start start 2024-05-20T18:37:52.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.957+0000 7f8412b38640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f840c06c180 0x7f840c1481e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:52.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.957+0000 7f8412b38640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f840c0af030 0x7f840c1fcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:52.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.957+0000 7f8412b38640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f840c15f700 0x7f840c0b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:52.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.957+0000 7f8412b38640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f840c06cad0 con 0x7f840c06c180 2024-05-20T18:37:52.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.957+0000 7f8412b38640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f840c159aa0 con 0x7f840c0af030 2024-05-20T18:37:52.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.957+0000 7f8412b38640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f840c06cc50 con 0x7f840c15f700 2024-05-20T18:37:52.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.957+0000 7f84108d4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f840c06c180 0x7f840c1481e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:52.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.957+0000 7f84108d4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f840c06c180 0x7f840c1481e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35818/0 (socket says 172.21.15.5:35818) 2024-05-20T18:37:52.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.957+0000 7f84108d4640 1 -- 172.21.15.5:0/4042793128 learned_addr learned my addr 172.21.15.5:0/4042793128 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:52.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.957+0000 7f84110d5640 1 --2- 172.21.15.5:0/4042793128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f840c15f700 0x7f840c0b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:52.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.957+0000 7f840bfff640 1 --2- 172.21.15.5:0/4042793128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f840c0af030 0x7f840c1fcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:52.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.957+0000 7f84108d4640 1 -- 172.21.15.5:0/4042793128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f840c15f700 msgr2=0x7f840c0b1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:52.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.957+0000 7f84108d4640 1 --2- 172.21.15.5:0/4042793128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f840c15f700 0x7f840c0b1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.957+0000 7f84108d4640 1 -- 172.21.15.5:0/4042793128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f840c0af030 msgr2=0x7f840c1fcfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:52.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.957+0000 7f84108d4640 1 --2- 172.21.15.5:0/4042793128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f840c0af030 0x7f840c1fcfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:52.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.957+0000 7f84108d4640 1 -- 172.21.15.5:0/4042793128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f840c148720 con 0x7f840c06c180 2024-05-20T18:37:52.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.957+0000 7f84110d5640 1 --2- 172.21.15.5:0/4042793128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f840c15f700 0x7f840c0b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:52.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.957+0000 7f840bfff640 1 --2- 172.21.15.5:0/4042793128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f840c0af030 0x7f840c1fcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:52.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.957+0000 7f84108d4640 1 --2- 172.21.15.5:0/4042793128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f840c06c180 0x7f840c1481e0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f8404007e60 tx=0x7f840400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:52.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.957+0000 7f84097fa640 1 -- 172.21.15.5:0/4042793128 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840401a070 con 0x7f840c06c180 2024-05-20T18:37:52.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.957+0000 7f84097fa640 1 -- 172.21.15.5:0/4042793128 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8404003b90 con 0x7f840c06c180 2024-05-20T18:37:52.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.957+0000 7f8412b38640 1 -- 172.21.15.5:0/4042793128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f840c14ede0 con 0x7f840c06c180 2024-05-20T18:37:52.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.957+0000 7f84097fa640 1 -- 172.21.15.5:0/4042793128 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8404004030 con 0x7f840c06c180 2024-05-20T18:37:52.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.957+0000 7f8412b38640 1 -- 172.21.15.5:0/4042793128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f840c1fdb90 con 0x7f840c06c180 2024-05-20T18:37:52.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.959+0000 7f8412b38640 1 -- 172.21.15.5:0/4042793128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f840c075b00 con 0x7f840c06c180 2024-05-20T18:37:52.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.959+0000 7f84097fa640 1 -- 172.21.15.5:0/4042793128 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f840401d070 con 0x7f840c06c180 2024-05-20T18:37:52.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.960+0000 7f84097fa640 1 --2- 172.21.15.5:0/4042793128 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f83f806be60 0x7f83f806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:52.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.960+0000 7f84097fa640 1 -- 172.21.15.5:0/4042793128 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f840408b210 con 0x7f840c06c180 2024-05-20T18:37:52.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.960+0000 7f840bfff640 1 --2- 172.21.15.5:0/4042793128 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f83f806be60 0x7f83f806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:52.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.960+0000 7f840bfff640 1 --2- 172.21.15.5:0/4042793128 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f83f806be60 0x7f83f806e370 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f840c1e94d0 tx=0x7f8400009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:52.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:52.965+0000 7f84097fa640 1 -- 172.21.15.5:0/4042793128 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f840c14ede0 con 0x7f840c06c180 2024-05-20T18:37:53.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.189+0000 7f8412b38640 1 -- 172.21.15.5:0/4042793128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f840c15f470 con 0x7f840c06c180 2024-05-20T18:37:53.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.189+0000 7f84097fa640 1 -- 172.21.15.5:0/4042793128 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5180 (secure 0 0 0) 0x7f84040592a0 con 0x7f840c06c180 2024-05-20T18:37:53.191 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:53.191 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":111,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/266630184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":266630184},{"type":"v1","addr":"172.21.15.63:6834","nonce":266630184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7361,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1494022697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":1494022697},{"type":"v1","addr":"172.21.15.63:6835","nonce":1494022697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":9009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/1739221946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1739221946},{"type":"v1","addr":"172.21.15.5:6836","nonce":1739221946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:37:14.365195+0000","modified":"2024-05-20T18:37:44.722140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9444},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9444":{"gid":9444,"name":"c","rank":0,"incarnation":108,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6837/2067544478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":2067544478},{"type":"v1","addr":"172.21.15.5:6837","nonce":2067544478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9444,"qdb_cluster":[9444]},"id":8}]} 2024-05-20T18:37:53.192 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 111 2024-05-20T18:37:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.191+0000 7f8412b38640 1 -- 172.21.15.5:0/4042793128 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f83f806be60 msgr2=0x7f83f806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.191+0000 7f8412b38640 1 --2- 172.21.15.5:0/4042793128 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f83f806be60 0x7f83f806e370 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f840c1e94d0 tx=0x7f8400009040 comp rx=0 tx=0).stop 2024-05-20T18:37:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.191+0000 7f8412b38640 1 -- 172.21.15.5:0/4042793128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f840c06c180 msgr2=0x7f840c1481e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.191+0000 7f8412b38640 1 --2- 172.21.15.5:0/4042793128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f840c06c180 0x7f840c1481e0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f8404007e60 tx=0x7f840400e4b0 comp rx=0 tx=0).stop 2024-05-20T18:37:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.191+0000 7f8412b38640 1 -- 172.21.15.5:0/4042793128 shutdown_connections 2024-05-20T18:37:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.191+0000 7f8412b38640 1 --2- 172.21.15.5:0/4042793128 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f83f806be60 0x7f83f806e370 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.191+0000 7f8412b38640 1 --2- 172.21.15.5:0/4042793128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f840c15f700 0x7f840c0b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.191+0000 7f8412b38640 1 --2- 172.21.15.5:0/4042793128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f840c0af030 0x7f840c1fcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.191+0000 7f8412b38640 1 --2- 172.21.15.5:0/4042793128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f840c06c180 0x7f840c1481e0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.191+0000 7f8412b38640 1 -- 172.21.15.5:0/4042793128 >> 172.21.15.5:0/4042793128 conn(0x7f840c0b6620 msgr2=0x7f840c067490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.191+0000 7f8412b38640 1 -- 172.21.15.5:0/4042793128 shutdown_connections 2024-05-20T18:37:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.191+0000 7f8412b38640 1 -- 172.21.15.5:0/4042793128 wait complete. 2024-05-20T18:37:53.204 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:37:53.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.318+0000 7f8be28ca640 1 Processor -- start 2024-05-20T18:37:53.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.318+0000 7f8be28ca640 1 -- start start 2024-05-20T18:37:53.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.319+0000 7f8be28ca640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8bdc0cd710 0x7f8bdc162e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:53.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.319+0000 7f8be28ca640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8bdc0cfae0 0x7f8bdc15f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:53.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.319+0000 7f8be28ca640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bdc06c8c0 con 0x7f8bdc06d6b0 2024-05-20T18:37:53.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.319+0000 7f8be28ca640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f8bdc0d0ba0 con 0x7f8bdc0cd710 2024-05-20T18:37:53.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.319+0000 7f8be28ca640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8bdc06dac0 con 0x7f8bdc0cfae0 2024-05-20T18:37:53.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.319+0000 7f8bdbfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8bdc0cd710 0x7f8bdc162e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:53.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.319+0000 7f8bdbfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8bdc0cd710 0x7f8bdc162e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35834/0 (socket says 172.21.15.5:35834) 2024-05-20T18:37:53.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.319+0000 7f8bdbfff640 1 -- 172.21.15.5:0/2499996981 learned_addr learned my addr 172.21.15.5:0/2499996981 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:53.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.319+0000 7f8be0e67640 1 --2- 172.21.15.5:0/2499996981 >> v2:172.21.15.63:3300/0 conn(0x7f8bdc0cfae0 0x7f8bdc15f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:53.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.319+0000 7f8bdbfff640 1 -- 172.21.15.5:0/2499996981 >> v2:172.21.15.63:3300/0 conn(0x7f8bdc0cfae0 msgr2=0x7f8bdc15f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.319+0000 7f8bdbfff640 1 --2- 172.21.15.5:0/2499996981 >> v2:172.21.15.63:3300/0 conn(0x7f8bdc0cfae0 0x7f8bdc15f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.320+0000 7f8bdbfff640 1 -- 172.21.15.5:0/2499996981 >> v1:172.21.15.63:6789/0 conn(0x7f8bdc06d6b0 legacy=0x7f8bdc0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.320+0000 7f8bdbfff640 1 -- 172.21.15.5:0/2499996981 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bdc1633e0 con 0x7f8bdc0cd710 2024-05-20T18:37:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.320+0000 7f8bdbfff640 1 --2- 172.21.15.5:0/2499996981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bdc0cd710 0x7f8bdc162e00 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc0d3a60 tx=0x7f8bcc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292341efeeef8cdf server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.320+0000 7f8bdaffd640 1 -- 172.21.15.5:0/2499996981 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc037070 con 0x7f8bdc0cd710 2024-05-20T18:37:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.321+0000 7f8bdaffd640 1 -- 172.21.15.5:0/2499996981 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bcc032440 con 0x7f8bdc0cd710 2024-05-20T18:37:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.321+0000 7f8bdaffd640 1 -- 172.21.15.5:0/2499996981 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc032810 con 0x7f8bdc0cd710 2024-05-20T18:37:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.321+0000 7f8be28ca640 1 -- 172.21.15.5:0/2499996981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bdc0cd710 msgr2=0x7f8bdc162e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.321+0000 7f8be28ca640 1 --2- 172.21.15.5:0/2499996981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bdc0cd710 0x7f8bdc162e00 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc0d3a60 tx=0x7f8bcc02a270 comp rx=0 tx=0).stop 2024-05-20T18:37:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.321+0000 7f8be28ca640 1 -- 172.21.15.5:0/2499996981 shutdown_connections 2024-05-20T18:37:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.321+0000 7f8be28ca640 1 --2- 172.21.15.5:0/2499996981 >> v2:172.21.15.63:3300/0 conn(0x7f8bdc0cfae0 0x7f8bdc15f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.321+0000 7f8be28ca640 1 --2- 172.21.15.5:0/2499996981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bdc0cd710 0x7f8bdc162e00 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.321+0000 7f8be28ca640 1 -- 172.21.15.5:0/2499996981 >> 172.21.15.5:0/2499996981 conn(0x7f8bdc0b44d0 msgr2=0x7f8bdc0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.321+0000 7f8be28ca640 1 -- 172.21.15.5:0/2499996981 shutdown_connections 2024-05-20T18:37:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.321+0000 7f8be28ca640 1 -- 172.21.15.5:0/2499996981 wait complete. 2024-05-20T18:37:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.323+0000 7f8be28ca640 1 Processor -- start 2024-05-20T18:37:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.323+0000 7f8be28ca640 1 -- start start 2024-05-20T18:37:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.324+0000 7f8be28ca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bdc06d6b0 0x7f8bdc159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.324+0000 7f8be28ca640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8bdc0cd710 0x7f8bdc0c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.324+0000 7f8be28ca640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8bdc0cbaf0 0x7f8bdc0cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.324+0000 7f8be28ca640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc163c80 con 0x7f8bdc06d6b0 2024-05-20T18:37:53.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.324+0000 7f8be28ca640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc1639a0 con 0x7f8bdc0cbaf0 2024-05-20T18:37:53.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.324+0000 7f8be28ca640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8bdc0d1030 con 0x7f8bdc0cd710 2024-05-20T18:37:53.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.325+0000 7f8bdbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bdc06d6b0 0x7f8bdc159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:53.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.325+0000 7f8bdbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bdc06d6b0 0x7f8bdc159dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35836/0 (socket says 172.21.15.5:35836) 2024-05-20T18:37:53.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.325+0000 7f8bdbfff640 1 -- 172.21.15.5:0/55572818 learned_addr learned my addr 172.21.15.5:0/55572818 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:53.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.325+0000 7f8be0e67640 1 --2- 172.21.15.5:0/55572818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8bdc0cbaf0 0x7f8bdc0cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:53.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.325+0000 7f8bdb7fe640 1 --2- 172.21.15.5:0/55572818 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8bdc0cd710 0x7f8bdc0c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:53.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.325+0000 7f8bdbfff640 1 -- 172.21.15.5:0/55572818 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8bdc0cd710 msgr2=0x7f8bdc0c64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:53.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.325+0000 7f8bdbfff640 1 --2- 172.21.15.5:0/55572818 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8bdc0cd710 0x7f8bdc0c64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:53.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.325+0000 7f8bdbfff640 1 -- 172.21.15.5:0/55572818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8bdc0cbaf0 msgr2=0x7f8bdc0cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:53.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.325+0000 7f8bdbfff640 1 --2- 172.21.15.5:0/55572818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8bdc0cbaf0 0x7f8bdc0cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:53.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.325+0000 7f8bdbfff640 1 -- 172.21.15.5:0/55572818 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bdc0cc440 con 0x7f8bdc06d6b0 2024-05-20T18:37:53.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.325+0000 7f8bdb7fe640 1 --2- 172.21.15.5:0/55572818 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8bdc0cd710 0x7f8bdc0c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:53.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.325+0000 7f8bdbfff640 1 --2- 172.21.15.5:0/55572818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bdc06d6b0 0x7f8bdc159dd0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc000c00 tx=0x7f8bcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:53.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.325+0000 7f8bd8ff9640 1 -- 172.21.15.5:0/55572818 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc041020 con 0x7f8bdc06d6b0 2024-05-20T18:37:53.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.325+0000 7f8bd8ff9640 1 -- 172.21.15.5:0/55572818 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bcc0306c0 con 0x7f8bdc06d6b0 2024-05-20T18:37:53.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.325+0000 7f8be28ca640 1 -- 172.21.15.5:0/55572818 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bdc0b8740 con 0x7f8bdc06d6b0 2024-05-20T18:37:53.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.325+0000 7f8bd8ff9640 1 -- 172.21.15.5:0/55572818 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc037040 con 0x7f8bdc06d6b0 2024-05-20T18:37:53.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.326+0000 7f8be28ca640 1 -- 172.21.15.5:0/55572818 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bdc0c7070 con 0x7f8bdc06d6b0 2024-05-20T18:37:53.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.326+0000 7f8be28ca640 1 -- 172.21.15.5:0/55572818 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bdc0b33d0 con 0x7f8bdc06d6b0 2024-05-20T18:37:53.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.327+0000 7f8bd8ff9640 1 -- 172.21.15.5:0/55572818 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8bcc044020 con 0x7f8bdc06d6b0 2024-05-20T18:37:53.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.327+0000 7f8bd8ff9640 1 --2- 172.21.15.5:0/55572818 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8bc806be60 0x7f8bc806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:53.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.328+0000 7f8bd8ff9640 1 -- 172.21.15.5:0/55572818 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f8bcc0ab8d0 con 0x7f8bdc06d6b0 2024-05-20T18:37:53.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.328+0000 7f8bdb7fe640 1 --2- 172.21.15.5:0/55572818 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8bc806be60 0x7f8bc806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:53.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.328+0000 7f8bdb7fe640 1 --2- 172.21.15.5:0/55572818 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8bc806be60 0x7f8bc806e370 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc0cc7c0 tx=0x7f8bd0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:53.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.332+0000 7f8bd8ff9640 1 -- 172.21.15.5:0/55572818 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8bdc0b33d0 con 0x7f8bdc06d6b0 2024-05-20T18:37:53.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.557+0000 7f8be28ca640 1 -- 172.21.15.5:0/55572818 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bdc0d15a0 con 0x7f8bdc06d6b0 2024-05-20T18:37:53.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.558+0000 7f8bd8ff9640 1 -- 172.21.15.5:0/55572818 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5180 (secure 0 0 0) 0x7f8bcc04b030 con 0x7f8bdc06d6b0 2024-05-20T18:37:53.559 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:53.559 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":111,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/266630184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":266630184},{"type":"v1","addr":"172.21.15.63:6834","nonce":266630184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7361,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1494022697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":1494022697},{"type":"v1","addr":"172.21.15.63:6835","nonce":1494022697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":9009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/1739221946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1739221946},{"type":"v1","addr":"172.21.15.5:6836","nonce":1739221946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-20T18:37:14.365195+0000","modified":"2024-05-20T18:37:44.722140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9444},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9444":{"gid":9444,"name":"c","rank":0,"incarnation":108,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6837/2067544478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":2067544478},{"type":"v1","addr":"172.21.15.5:6837","nonce":2067544478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9444,"qdb_cluster":[9444]},"id":8}]} 2024-05-20T18:37:53.560 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 111 2024-05-20T18:37:53.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.560+0000 7f8be28ca640 1 -- 172.21.15.5:0/55572818 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8bc806be60 msgr2=0x7f8bc806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:53.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.560+0000 7f8be28ca640 1 --2- 172.21.15.5:0/55572818 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8bc806be60 0x7f8bc806e370 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc0cc7c0 tx=0x7f8bd0009040 comp rx=0 tx=0).stop 2024-05-20T18:37:53.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.560+0000 7f8be28ca640 1 -- 172.21.15.5:0/55572818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bdc06d6b0 msgr2=0x7f8bdc159dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:53.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.560+0000 7f8be28ca640 1 --2- 172.21.15.5:0/55572818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bdc06d6b0 0x7f8bdc159dd0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc000c00 tx=0x7f8bcc0046c0 comp rx=0 tx=0).stop 2024-05-20T18:37:53.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.560+0000 7f8be28ca640 1 -- 172.21.15.5:0/55572818 shutdown_connections 2024-05-20T18:37:53.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.560+0000 7f8be28ca640 1 --2- 172.21.15.5:0/55572818 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8bc806be60 0x7f8bc806e370 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:53.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.560+0000 7f8be28ca640 1 --2- 172.21.15.5:0/55572818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8bdc0cbaf0 0x7f8bdc0cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:53.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.560+0000 7f8be28ca640 1 --2- 172.21.15.5:0/55572818 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8bdc0cd710 0x7f8bdc0c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:53.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.560+0000 7f8be28ca640 1 --2- 172.21.15.5:0/55572818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bdc06d6b0 0x7f8bdc159dd0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:53.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.560+0000 7f8be28ca640 1 -- 172.21.15.5:0/55572818 >> 172.21.15.5:0/55572818 conn(0x7f8bdc0b44d0 msgr2=0x7f8bdc0b3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:53.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.560+0000 7f8be28ca640 1 -- 172.21.15.5:0/55572818 shutdown_connections 2024-05-20T18:37:53.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.560+0000 7f8be28ca640 1 -- 172.21.15.5:0/55572818 wait complete. 2024-05-20T18:37:53.573 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:37:53.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.687+0000 7f8f0e716640 1 Processor -- start 2024-05-20T18:37:53.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.687+0000 7f8f0e716640 1 -- start start 2024-05-20T18:37:53.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.689+0000 7f8f0e716640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8f080d4400 0x7f8f0815f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:53.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.689+0000 7f8f0e716640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8f0806cf00 0x7f8f0806d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:53.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.689+0000 7f8f0e716640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8f080d2780 0x7f8f080d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:53.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.689+0000 7f8f0e716640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f8f080d3f70 con 0x7f8f080d2780 2024-05-20T18:37:53.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.689+0000 7f8f0e716640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8f0808d2a0 con 0x7f8f080d4400 2024-05-20T18:37:53.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.689+0000 7f8f0e716640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8f0806d850 con 0x7f8f0806cf00 2024-05-20T18:37:53.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.689+0000 7f8f0ccb3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8f080d2780 0x7f8f080d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:53.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.689+0000 7f8f0ccb3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8f080d2780 0x7f8f080d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35850/0 (socket says 172.21.15.5:35850) 2024-05-20T18:37:53.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.689+0000 7f8f0ccb3640 1 -- 172.21.15.5:0/4164751560 learned_addr learned my addr 172.21.15.5:0/4164751560 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:53.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.689+0000 7f8f077fe640 1 --2- 172.21.15.5:0/4164751560 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8f0806cf00 0x7f8f0806d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:53.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.689+0000 7f8f07fff640 1 --2- 172.21.15.5:0/4164751560 >> v2:172.21.15.63:3300/0 conn(0x7f8f080d4400 0x7f8f0815f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:53.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.689+0000 7f8f0ccb3640 1 -- 172.21.15.5:0/4164751560 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8f0806cf00 msgr2=0x7f8f0806d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:53.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.689+0000 7f8f0ccb3640 1 --2- 172.21.15.5:0/4164751560 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8f0806cf00 0x7f8f0806d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:53.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.689+0000 7f8f0ccb3640 1 -- 172.21.15.5:0/4164751560 >> v2:172.21.15.63:3300/0 conn(0x7f8f080d4400 msgr2=0x7f8f0815f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:53.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.689+0000 7f8f0ccb3640 1 --2- 172.21.15.5:0/4164751560 >> v2:172.21.15.63:3300/0 conn(0x7f8f080d4400 0x7f8f0815f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:53.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.689+0000 7f8f0ccb3640 1 -- 172.21.15.5:0/4164751560 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f080d30d0 con 0x7f8f080d2780 2024-05-20T18:37:53.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.690+0000 7f8f0ccb3640 1 --2- 172.21.15.5:0/4164751560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f080d2780 0x7f8f080d2b90 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f8f080d5ab0 tx=0x7f8f0002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98debed6ac0d9780 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:53.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.690+0000 7f8f06ffd640 1 -- 172.21.15.5:0/4164751560 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f00036070 con 0x7f8f080d2780 2024-05-20T18:37:53.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.690+0000 7f8f06ffd640 1 -- 172.21.15.5:0/4164751560 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f00031600 con 0x7f8f080d2780 2024-05-20T18:37:53.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.690+0000 7f8f06ffd640 1 -- 172.21.15.5:0/4164751560 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f000319d0 con 0x7f8f080d2780 2024-05-20T18:37:53.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.691+0000 7f8f0e716640 1 -- 172.21.15.5:0/4164751560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f080d2780 msgr2=0x7f8f080d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:53.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.691+0000 7f8f0e716640 1 --2- 172.21.15.5:0/4164751560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f080d2780 0x7f8f080d2b90 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f8f080d5ab0 tx=0x7f8f0002a730 comp rx=0 tx=0).stop 2024-05-20T18:37:53.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.691+0000 7f8f0e716640 1 -- 172.21.15.5:0/4164751560 shutdown_connections 2024-05-20T18:37:53.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.691+0000 7f8f0e716640 1 --2- 172.21.15.5:0/4164751560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f080d2780 0x7f8f080d2b90 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:53.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.691+0000 7f8f0e716640 1 --2- 172.21.15.5:0/4164751560 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8f0806cf00 0x7f8f0806d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:53.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.691+0000 7f8f0e716640 1 --2- 172.21.15.5:0/4164751560 >> v2:172.21.15.63:3300/0 conn(0x7f8f080d4400 0x7f8f0815f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:53.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.691+0000 7f8f0e716640 1 -- 172.21.15.5:0/4164751560 >> 172.21.15.5:0/4164751560 conn(0x7f8f080d4bb0 msgr2=0x7f8f080d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:53.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.691+0000 7f8f0e716640 1 -- 172.21.15.5:0/4164751560 shutdown_connections 2024-05-20T18:37:53.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.691+0000 7f8f0e716640 1 -- 172.21.15.5:0/4164751560 wait complete. 2024-05-20T18:37:53.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.693+0000 7f8f0e716640 1 Processor -- start 2024-05-20T18:37:53.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.693+0000 7f8f0e716640 1 -- start start 2024-05-20T18:37:53.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.694+0000 7f8f0e716640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8f0806cf00 0x7f8f080cd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:53.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.694+0000 7f8f0e716640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f080d2780 0x7f8f08155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:53.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.694+0000 7f8f0e716640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f080d4400 0x7f8f081564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:53.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.694+0000 7f8f0e716640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8f080d3920 con 0x7f8f080d2780 2024-05-20T18:37:53.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.694+0000 7f8f0e716640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8f080d3640 con 0x7f8f080d4400 2024-05-20T18:37:53.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.694+0000 7f8f0e716640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8f080d3c00 con 0x7f8f0806cf00 2024-05-20T18:37:53.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.694+0000 7f8f077fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f080d2780 0x7f8f08155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:53.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.694+0000 7f8f077fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f080d2780 0x7f8f08155f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35858/0 (socket says 172.21.15.5:35858) 2024-05-20T18:37:53.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.694+0000 7f8f077fe640 1 -- 172.21.15.5:0/2339149348 learned_addr learned my addr 172.21.15.5:0/2339149348 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:53.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.695+0000 7f8f0ccb3640 1 --2- 172.21.15.5:0/2339149348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f080d4400 0x7f8f081564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:53.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.695+0000 7f8f07fff640 1 --2- 172.21.15.5:0/2339149348 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8f0806cf00 0x7f8f080cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:53.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.695+0000 7f8f077fe640 1 -- 172.21.15.5:0/2339149348 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8f0806cf00 msgr2=0x7f8f080cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:53.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.695+0000 7f8f077fe640 1 --2- 172.21.15.5:0/2339149348 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8f0806cf00 0x7f8f080cd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:53.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.695+0000 7f8f077fe640 1 -- 172.21.15.5:0/2339149348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f080d4400 msgr2=0x7f8f081564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:53.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.695+0000 7f8f077fe640 1 --2- 172.21.15.5:0/2339149348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f080d4400 0x7f8f081564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:53.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.695+0000 7f8f077fe640 1 -- 172.21.15.5:0/2339149348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f08156d10 con 0x7f8f080d2780 2024-05-20T18:37:53.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.695+0000 7f8f0ccb3640 1 --2- 172.21.15.5:0/2339149348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f080d4400 0x7f8f081564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:53.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.695+0000 7f8f07fff640 1 --2- 172.21.15.5:0/2339149348 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8f0806cf00 0x7f8f080cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:53.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.695+0000 7f8f077fe640 1 --2- 172.21.15.5:0/2339149348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f080d2780 0x7f8f08155f70 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00e870 tx=0x7f8efc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:53.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.695+0000 7f8f04ff9640 1 -- 172.21.15.5:0/2339149348 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc017070 con 0x7f8f080d2780 2024-05-20T18:37:53.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.695+0000 7f8f04ff9640 1 -- 172.21.15.5:0/2339149348 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8efc00ee60 con 0x7f8f080d2780 2024-05-20T18:37:53.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.695+0000 7f8f0e716640 1 -- 172.21.15.5:0/2339149348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f08156ff0 con 0x7f8f080d2780 2024-05-20T18:37:53.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.695+0000 7f8f04ff9640 1 -- 172.21.15.5:0/2339149348 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc00cb80 con 0x7f8f080d2780 2024-05-20T18:37:53.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.695+0000 7f8f0e716640 1 -- 172.21.15.5:0/2339149348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f081575b0 con 0x7f8f080d2780 2024-05-20T18:37:53.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.696+0000 7f8f0e716640 1 -- 172.21.15.5:0/2339149348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f080735a0 con 0x7f8f080d2780 2024-05-20T18:37:53.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.697+0000 7f8f04ff9640 1 -- 172.21.15.5:0/2339149348 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8efc01a070 con 0x7f8f080d2780 2024-05-20T18:37:53.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.697+0000 7f8f04ff9640 1 --2- 172.21.15.5:0/2339149348 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8ef406be30 0x7f8ef406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:53.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.698+0000 7f8f04ff9640 1 -- 172.21.15.5:0/2339149348 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f8efc08b2b0 con 0x7f8f080d2780 2024-05-20T18:37:53.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.698+0000 7f8f07fff640 1 --2- 172.21.15.5:0/2339149348 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8ef406be30 0x7f8ef406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:53.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.698+0000 7f8f07fff640 1 --2- 172.21.15.5:0/2339149348 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8ef406be30 0x7f8ef406e340 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009a20 tx=0x7f8ef8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:53.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.702+0000 7f8f04ff9640 1 -- 172.21.15.5:0/2339149348 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8f080735a0 con 0x7f8f080d2780 2024-05-20T18:37:53.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.888+0000 7f8f0e716640 1 -- 172.21.15.5:0/2339149348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f08075b00 con 0x7f8f080d2780 2024-05-20T18:37:53.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.889+0000 7f8f04ff9640 1 -- 172.21.15.5:0/2339149348 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+17089 (secure 0 0 0) 0x7f8efc059a20 con 0x7f8f080d2780 2024-05-20T18:37:53.891 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:53.891 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":121,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:37:39.699377+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-20T18:37:12.797471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-20T18:37:13.346156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6837/4273429943":"2024-05-21T18:37:39.699336+0000","172.21.15.63:6835/2352520472":"2024-05-21T18:37:28.901938+0000","172.21.15.5:6835/4273429943":"2024-05-21T18:37:39.699336+0000","172.21.15.63:6833/2352520472":"2024-05-21T18:37:28.901938+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:37:53.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.891+0000 7f8f0e716640 1 -- 172.21.15.5:0/2339149348 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8ef406be30 msgr2=0x7f8ef406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:53.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.891+0000 7f8f0e716640 1 --2- 172.21.15.5:0/2339149348 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8ef406be30 0x7f8ef406e340 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009a20 tx=0x7f8ef8008040 comp rx=0 tx=0).stop 2024-05-20T18:37:53.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.891+0000 7f8f0e716640 1 -- 172.21.15.5:0/2339149348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f080d2780 msgr2=0x7f8f08155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:53.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.891+0000 7f8f0e716640 1 --2- 172.21.15.5:0/2339149348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f080d2780 0x7f8f08155f70 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00e870 tx=0x7f8efc00ed30 comp rx=0 tx=0).stop 2024-05-20T18:37:53.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.891+0000 7f8f0e716640 1 -- 172.21.15.5:0/2339149348 shutdown_connections 2024-05-20T18:37:53.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.891+0000 7f8f0e716640 1 --2- 172.21.15.5:0/2339149348 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f8ef406be30 0x7f8ef406e340 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:53.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.891+0000 7f8f0e716640 1 --2- 172.21.15.5:0/2339149348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8f080d4400 0x7f8f081564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:53.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.891+0000 7f8f0e716640 1 --2- 172.21.15.5:0/2339149348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f080d2780 0x7f8f08155f70 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:53.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.891+0000 7f8f0e716640 1 --2- 172.21.15.5:0/2339149348 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8f0806cf00 0x7f8f080cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:53.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.891+0000 7f8f0e716640 1 -- 172.21.15.5:0/2339149348 >> 172.21.15.5:0/2339149348 conn(0x7f8f080d4bb0 msgr2=0x7f8f0815dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:53.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.891+0000 7f8f0e716640 1 -- 172.21.15.5:0/2339149348 shutdown_connections 2024-05-20T18:37:53.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:53.891+0000 7f8f0e716640 1 -- 172.21.15.5:0/2339149348 wait complete. 2024-05-20T18:37:53.904 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-20T18:37:54.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.019+0000 7f48c68ca640 1 Processor -- start 2024-05-20T18:37:54.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.019+0000 7f48c68ca640 1 -- start start 2024-05-20T18:37:54.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.020+0000 7f48c68ca640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48c0163a10 0x7f48c00d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:54.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.020+0000 7f48c68ca640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f48c00b4010 0x7f48c00cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:54.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.020+0000 7f48c68ca640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f48c00d14b0 0x7f48c00d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:54.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.020+0000 7f48c68ca640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f48c00d3ca0 con 0x7f48c00b4010 2024-05-20T18:37:54.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.020+0000 7f48c68ca640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f48c00d0370 con 0x7f48c00d14b0 2024-05-20T18:37:54.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.020+0000 7f48c68ca640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f48c00d0650 con 0x7f48c0163a10 2024-05-20T18:37:54.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.020+0000 7f48bf7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f48c00b4010 0x7f48c00cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:54.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.020+0000 7f48bf7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f48c00b4010 0x7f48c00cfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35864/0 (socket says 172.21.15.5:35864) 2024-05-20T18:37:54.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.020+0000 7f48bf7fe640 1 -- 172.21.15.5:0/1375219486 learned_addr learned my addr 172.21.15.5:0/1375219486 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:54.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.021+0000 7f48bf7fe640 1 -- 172.21.15.5:0/1375219486 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48c0163a10 msgr2=0x7f48c00d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:54.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.021+0000 7f48c4e67640 1 --2- 172.21.15.5:0/1375219486 >> v2:172.21.15.63:3300/0 conn(0x7f48c00d14b0 0x7f48c00d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:54.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.021+0000 7f48bffff640 1 --2- 172.21.15.5:0/1375219486 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48c0163a10 0x7f48c00d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:54.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.021+0000 7f48bf7fe640 1 --2- 172.21.15.5:0/1375219486 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48c0163a10 0x7f48c00d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:54.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.021+0000 7f48bf7fe640 1 -- 172.21.15.5:0/1375219486 >> v2:172.21.15.63:3300/0 conn(0x7f48c00d14b0 msgr2=0x7f48c00d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:54.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.021+0000 7f48bf7fe640 1 --2- 172.21.15.5:0/1375219486 >> v2:172.21.15.63:3300/0 conn(0x7f48c00d14b0 0x7f48c00d18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:54.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.021+0000 7f48bf7fe640 1 -- 172.21.15.5:0/1375219486 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c00d3650 con 0x7f48c00b4010 2024-05-20T18:37:54.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.021+0000 7f48c4e67640 1 --2- 172.21.15.5:0/1375219486 >> v2:172.21.15.63:3300/0 conn(0x7f48c00d14b0 0x7f48c00d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:54.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.022+0000 7f48bf7fe640 1 --2- 172.21.15.5:0/1375219486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48c00b4010 0x7f48c00cfb00 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f48c00b4590 tx=0x7f48b402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9781260a61cec57c server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:54.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.022+0000 7f48beffd640 1 -- 172.21.15.5:0/1375219486 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b4037070 con 0x7f48c00b4010 2024-05-20T18:37:54.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.022+0000 7f48beffd640 1 -- 172.21.15.5:0/1375219486 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48b4032440 con 0x7f48c00b4010 2024-05-20T18:37:54.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.022+0000 7f48beffd640 1 -- 172.21.15.5:0/1375219486 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b4032810 con 0x7f48c00b4010 2024-05-20T18:37:54.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.022+0000 7f48c68ca640 1 -- 172.21.15.5:0/1375219486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48c00b4010 msgr2=0x7f48c00cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:54.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.022+0000 7f48c68ca640 1 --2- 172.21.15.5:0/1375219486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48c00b4010 0x7f48c00cfb00 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f48c00b4590 tx=0x7f48b402a1e0 comp rx=0 tx=0).stop 2024-05-20T18:37:54.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.022+0000 7f48c68ca640 1 -- 172.21.15.5:0/1375219486 shutdown_connections 2024-05-20T18:37:54.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.022+0000 7f48c68ca640 1 --2- 172.21.15.5:0/1375219486 >> v2:172.21.15.63:3300/0 conn(0x7f48c00d14b0 0x7f48c00d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:54.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.023+0000 7f48c68ca640 1 --2- 172.21.15.5:0/1375219486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48c00b4010 0x7f48c00cfb00 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:54.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.023+0000 7f48c68ca640 1 --2- 172.21.15.5:0/1375219486 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48c0163a10 0x7f48c00d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:54.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.023+0000 7f48c68ca640 1 -- 172.21.15.5:0/1375219486 >> 172.21.15.5:0/1375219486 conn(0x7f48c006d240 msgr2=0x7f48c006d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:54.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.023+0000 7f48c68ca640 1 -- 172.21.15.5:0/1375219486 shutdown_connections 2024-05-20T18:37:54.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.023+0000 7f48c68ca640 1 -- 172.21.15.5:0/1375219486 wait complete. 2024-05-20T18:37:54.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.024+0000 7f48c68ca640 1 Processor -- start 2024-05-20T18:37:54.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.025+0000 7f48c68ca640 1 -- start start 2024-05-20T18:37:54.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.026+0000 7f48c68ca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48c00b4010 0x7f48c00ac460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:54.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.026+0000 7f48c68ca640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48c00d14b0 0x7f48c01f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:54.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.026+0000 7f48c68ca640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48c0163a10 0x7f48c0159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:54.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.026+0000 7f48c68ca640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f48c00d1e00 con 0x7f48c00b4010 2024-05-20T18:37:54.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.026+0000 7f48c68ca640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f48c00d0040 con 0x7f48c0163a10 2024-05-20T18:37:54.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.026+0000 7f48c68ca640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f48c00b8210 con 0x7f48c00d14b0 2024-05-20T18:37:54.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.026+0000 7f48bffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48c00b4010 0x7f48c00ac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:54.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.026+0000 7f48bffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48c00b4010 0x7f48c00ac460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35876/0 (socket says 172.21.15.5:35876) 2024-05-20T18:37:54.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.026+0000 7f48bffff640 1 -- 172.21.15.5:0/540875469 learned_addr learned my addr 172.21.15.5:0/540875469 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:54.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.026+0000 7f48bf7fe640 1 --2- 172.21.15.5:0/540875469 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48c00d14b0 0x7f48c01f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:54.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.026+0000 7f48c4e67640 1 --2- 172.21.15.5:0/540875469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48c0163a10 0x7f48c0159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:54.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.026+0000 7f48bffff640 1 -- 172.21.15.5:0/540875469 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48c00d14b0 msgr2=0x7f48c01f57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:54.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.026+0000 7f48bffff640 1 --2- 172.21.15.5:0/540875469 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48c00d14b0 0x7f48c01f57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:54.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.026+0000 7f48bffff640 1 -- 172.21.15.5:0/540875469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48c0163a10 msgr2=0x7f48c0159300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:54.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.026+0000 7f48bffff640 1 --2- 172.21.15.5:0/540875469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48c0163a10 0x7f48c0159300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:54.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.026+0000 7f48bffff640 1 -- 172.21.15.5:0/540875469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c00ac9a0 con 0x7f48c00b4010 2024-05-20T18:37:54.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.026+0000 7f48bf7fe640 1 --2- 172.21.15.5:0/540875469 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48c00d14b0 0x7f48c01f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:54.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.026+0000 7f48c4e67640 1 --2- 172.21.15.5:0/540875469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48c0163a10 0x7f48c0159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:54.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.027+0000 7f48bffff640 1 --2- 172.21.15.5:0/540875469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48c00b4010 0x7f48c00ac460 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f48b000c7f0 tx=0x7f48b000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:54.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.027+0000 7f48bcff9640 1 -- 172.21.15.5:0/540875469 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b0004230 con 0x7f48c00b4010 2024-05-20T18:37:54.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.027+0000 7f48bcff9640 1 -- 172.21.15.5:0/540875469 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48b00043d0 con 0x7f48c00b4010 2024-05-20T18:37:54.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.027+0000 7f48bcff9640 1 -- 172.21.15.5:0/540875469 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b0007860 con 0x7f48c00b4010 2024-05-20T18:37:54.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.027+0000 7f48c68ca640 1 -- 172.21.15.5:0/540875469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48c0158260 con 0x7f48c00b4010 2024-05-20T18:37:54.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.027+0000 7f48c68ca640 1 -- 172.21.15.5:0/540875469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48c0158820 con 0x7f48c00b4010 2024-05-20T18:37:54.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.028+0000 7f48c68ca640 1 -- 172.21.15.5:0/540875469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48c0075b00 con 0x7f48c00b4010 2024-05-20T18:37:54.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.028+0000 7f48bcff9640 1 -- 172.21.15.5:0/540875469 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f48b0007c10 con 0x7f48c00b4010 2024-05-20T18:37:54.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.029+0000 7f48bcff9640 1 --2- 172.21.15.5:0/540875469 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f48ac06be30 0x7f48ac06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:54.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.029+0000 7f48bcff9640 1 -- 172.21.15.5:0/540875469 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f48b008b2c0 con 0x7f48c00b4010 2024-05-20T18:37:54.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.029+0000 7f48bf7fe640 1 --2- 172.21.15.5:0/540875469 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f48ac06be30 0x7f48ac06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:54.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.030+0000 7f48bf7fe640 1 --2- 172.21.15.5:0/540875469 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f48ac06be30 0x7f48ac06e340 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f48b4034000 tx=0x7f48b402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:54.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.034+0000 7f48bcff9640 1 -- 172.21.15.5:0/540875469 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f48b0015020 con 0x7f48c00b4010 2024-05-20T18:37:54.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.255+0000 7f48c68ca640 1 -- 172.21.15.5:0/540875469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f48c00d3050 con 0x7f48c00b4010 2024-05-20T18:37:54.256 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-20T18:37:54.255+0000 7fd1cf53a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-20T18:37:54.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.257+0000 7f48bcff9640 1 -- 172.21.15.5:0/540875469 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v112) ==== 161+0+0 (secure 0 0 0) 0x7f48b0059990 con 0x7f48c00b4010 2024-05-20T18:37:54.258 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-20T18:37:54.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.260+0000 7f48c68ca640 1 -- 172.21.15.5:0/540875469 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f48ac06be30 msgr2=0x7f48ac06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:54.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.260+0000 7f48c68ca640 1 --2- 172.21.15.5:0/540875469 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f48ac06be30 0x7f48ac06e340 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f48b4034000 tx=0x7f48b402e000 comp rx=0 tx=0).stop 2024-05-20T18:37:54.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.260+0000 7f48c68ca640 1 -- 172.21.15.5:0/540875469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48c00b4010 msgr2=0x7f48c00ac460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:54.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.260+0000 7f48c68ca640 1 --2- 172.21.15.5:0/540875469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48c00b4010 0x7f48c00ac460 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f48b000c7f0 tx=0x7f48b000ccb0 comp rx=0 tx=0).stop 2024-05-20T18:37:54.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.260+0000 7f48c68ca640 1 -- 172.21.15.5:0/540875469 shutdown_connections 2024-05-20T18:37:54.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.260+0000 7f48c68ca640 1 --2- 172.21.15.5:0/540875469 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f48ac06be30 0x7f48ac06e340 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:54.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.260+0000 7f48c68ca640 1 --2- 172.21.15.5:0/540875469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48c0163a10 0x7f48c0159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:54.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.260+0000 7f48c68ca640 1 --2- 172.21.15.5:0/540875469 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f48c00d14b0 0x7f48c01f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:54.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.260+0000 7f48c68ca640 1 --2- 172.21.15.5:0/540875469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48c00b4010 0x7f48c00ac460 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:54.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.260+0000 7f48c68ca640 1 -- 172.21.15.5:0/540875469 >> 172.21.15.5:0/540875469 conn(0x7f48c006d240 msgr2=0x7f48c015dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:54.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.260+0000 7f48c68ca640 1 -- 172.21.15.5:0/540875469 shutdown_connections 2024-05-20T18:37:54.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.261+0000 7f48c68ca640 1 -- 172.21.15.5:0/540875469 wait complete. 2024-05-20T18:37:54.263 INFO:tasks.ceph.mds.c.smithi005.stderr: -1360> 2024-05-20T18:37:45.707+0000 7fb1ad191640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-20T18:37:54.274 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-20T18:37:54.320 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-05-20T18:37:54.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.394+0000 7f3e22521640 1 Processor -- start 2024-05-20T18:37:54.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.394+0000 7f3e22521640 1 -- start start 2024-05-20T18:37:54.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.396+0000 7f3e22521640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3e1c0cf3d0 0x7f3e1c0cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:54.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.396+0000 7f3e22521640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e1c0cfd20 con 0x7f3e1c15f220 2024-05-20T18:37:54.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.396+0000 7f3e22521640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e1c0cda20 con 0x7f3e1c0ce7e0 2024-05-20T18:37:54.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.396+0000 7f3e22521640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f3e1c0cdd00 con 0x7f3e1c0cf3d0 2024-05-20T18:37:54.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.396+0000 7f3e1b7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3e1c0cf3d0 0x7f3e1c0cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:54.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.396+0000 7f3e1bfff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3e1c15f220 0x7f3e1c0d08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44848/0 (socket says 172.21.15.5:44848) 2024-05-20T18:37:54.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.396+0000 7f3e1b7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3e1c0cf3d0 0x7f3e1c0cf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35898/0 (socket says 172.21.15.5:35898) 2024-05-20T18:37:54.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.396+0000 7f3e1bfff640 1 -- 172.21.15.5:0/1287597500 learned_addr learned my addr 172.21.15.5:0/1287597500 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:54.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.396+0000 7f3e1b7fe640 1 -- 172.21.15.5:0/1287597500 >> v1:172.21.15.63:6789/0 conn(0x7f3e1c0ce7e0 legacy=0x7f3e1c0cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:54.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.396+0000 7f3e1b7fe640 1 -- 172.21.15.5:0/1287597500 >> v1:172.21.15.5:6789/0 conn(0x7f3e1c15f220 legacy=0x7f3e1c0d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:54.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.396+0000 7f3e1b7fe640 1 -- 172.21.15.5:0/1287597500 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e1c0cdfe0 con 0x7f3e1c0cf3d0 2024-05-20T18:37:54.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.396+0000 7f3e1affd640 1 -- 172.21.15.5:0/1287597500 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2679234621 0 0) 0x7f3e0c001110 con 0x7f3e1c15f220 2024-05-20T18:37:54.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.397+0000 7f3e1b7fe640 1 --2- 172.21.15.5:0/1287597500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1c0cf3d0 0x7f3e1c0cf7e0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f3e040097a0 tx=0x7f3e0402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=badaa643664b66de server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:54.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.397+0000 7f3e1affd640 1 -- 172.21.15.5:0/1287597500 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e04037070 con 0x7f3e1c0cf3d0 2024-05-20T18:37:54.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.397+0000 7f3e1affd640 1 -- 172.21.15.5:0/1287597500 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e04032440 con 0x7f3e1c0cf3d0 2024-05-20T18:37:54.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.397+0000 7f3e1affd640 1 -- 172.21.15.5:0/1287597500 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e04032810 con 0x7f3e1c0cf3d0 2024-05-20T18:37:54.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.398+0000 7f3e22521640 1 -- 172.21.15.5:0/1287597500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1c0cf3d0 msgr2=0x7f3e1c0cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:54.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.398+0000 7f3e22521640 1 --2- 172.21.15.5:0/1287597500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1c0cf3d0 0x7f3e1c0cf7e0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f3e040097a0 tx=0x7f3e0402a1a0 comp rx=0 tx=0).stop 2024-05-20T18:37:54.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.398+0000 7f3e22521640 1 -- 172.21.15.5:0/1287597500 shutdown_connections 2024-05-20T18:37:54.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.398+0000 7f3e22521640 1 --2- 172.21.15.5:0/1287597500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1c0cf3d0 0x7f3e1c0cf7e0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:54.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.398+0000 7f3e22521640 1 -- 172.21.15.5:0/1287597500 >> 172.21.15.5:0/1287597500 conn(0x7f3e1c0b44d0 msgr2=0x7f3e1c0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:54.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.398+0000 7f3e22521640 1 -- 172.21.15.5:0/1287597500 shutdown_connections 2024-05-20T18:37:54.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.398+0000 7f3e22521640 1 -- 172.21.15.5:0/1287597500 wait complete. 2024-05-20T18:37:54.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.400+0000 7f3e22521640 1 Processor -- start 2024-05-20T18:37:54.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.400+0000 7f3e22521640 1 -- start start 2024-05-20T18:37:54.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.401+0000 7f3e22521640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e1c0ce7e0 0x7f3e1c151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:54.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.401+0000 7f3e22521640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e1c0cf3d0 0x7f3e1c1f9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:54.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.401+0000 7f3e22521640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1c15f220 0x7f3e1c1f9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:54.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.401+0000 7f3e22521640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c0cd680 con 0x7f3e1c15f220 2024-05-20T18:37:54.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.401+0000 7f3e22521640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c0d0fd0 con 0x7f3e1c0cf3d0 2024-05-20T18:37:54.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.401+0000 7f3e22521640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3e1c0cd800 con 0x7f3e1c0ce7e0 2024-05-20T18:37:54.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.401+0000 7f3e20abe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1c15f220 0x7f3e1c1f9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:54.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.401+0000 7f3e20abe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1c15f220 0x7f3e1c1f9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35908/0 (socket says 172.21.15.5:35908) 2024-05-20T18:37:54.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.401+0000 7f3e20abe640 1 -- 172.21.15.5:0/3253597149 learned_addr learned my addr 172.21.15.5:0/3253597149 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:54.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.402+0000 7f3e1bfff640 1 --2- 172.21.15.5:0/3253597149 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e1c0ce7e0 0x7f3e1c151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:54.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.402+0000 7f3e1b7fe640 1 --2- 172.21.15.5:0/3253597149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e1c0cf3d0 0x7f3e1c1f9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.402+0000 7f3e20abe640 1 -- 172.21.15.5:0/3253597149 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e1c0ce7e0 msgr2=0x7f3e1c151a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.402+0000 7f3e20abe640 1 --2- 172.21.15.5:0/3253597149 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e1c0ce7e0 0x7f3e1c151a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.402+0000 7f3e20abe640 1 -- 172.21.15.5:0/3253597149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e1c0cf3d0 msgr2=0x7f3e1c1f9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.402+0000 7f3e20abe640 1 --2- 172.21.15.5:0/3253597149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e1c0cf3d0 0x7f3e1c1f9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.402+0000 7f3e20abe640 1 -- 172.21.15.5:0/3253597149 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e1c15ac30 con 0x7f3e1c15f220 2024-05-20T18:37:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.402+0000 7f3e1b7fe640 1 --2- 172.21.15.5:0/3253597149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e1c0cf3d0 0x7f3e1c1f9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.402+0000 7f3e20abe640 1 --2- 172.21.15.5:0/3253597149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1c15f220 0x7f3e1c1f9570 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400c6e0 tx=0x7f3e1400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.402+0000 7f3e18ff9640 1 -- 172.21.15.5:0/3253597149 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e14003b70 con 0x7f3e1c15f220 2024-05-20T18:37:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.402+0000 7f3e18ff9640 1 -- 172.21.15.5:0/3253597149 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e14003d30 con 0x7f3e1c15f220 2024-05-20T18:37:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.402+0000 7f3e22521640 1 -- 172.21.15.5:0/3253597149 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e1c15af10 con 0x7f3e1c15f220 2024-05-20T18:37:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.402+0000 7f3e18ff9640 1 -- 172.21.15.5:0/3253597149 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e14005310 con 0x7f3e1c15f220 2024-05-20T18:37:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.402+0000 7f3e22521640 1 -- 172.21.15.5:0/3253597149 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e1c15b4d0 con 0x7f3e1c15f220 2024-05-20T18:37:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.403+0000 7f3e22521640 1 -- 172.21.15.5:0/3253597149 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e1c075b00 con 0x7f3e1c15f220 2024-05-20T18:37:54.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.404+0000 7f3e18ff9640 1 -- 172.21.15.5:0/3253597149 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3e140054b0 con 0x7f3e1c15f220 2024-05-20T18:37:54.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.404+0000 7f3e18ff9640 1 --2- 172.21.15.5:0/3253597149 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3dfc06be30 0x7f3dfc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:54.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.405+0000 7f3e1bfff640 1 --2- 172.21.15.5:0/3253597149 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3dfc06be30 0x7f3dfc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:54.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.405+0000 7f3e18ff9640 1 -- 172.21.15.5:0/3253597149 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6359+0+0 (secure 0 0 0) 0x7f3e1408afd0 con 0x7f3e1c15f220 2024-05-20T18:37:54.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.405+0000 7f3e1bfff640 1 --2- 172.21.15.5:0/3253597149 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3dfc06be30 0x7f3dfc06e340 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c000f60 tx=0x7f3e0c007d00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:54.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.409+0000 7f3e18ff9640 1 -- 172.21.15.5:0/3253597149 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3e14059650 con 0x7f3e1c15f220 2024-05-20T18:37:54.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:54.636+0000 7f3e22521640 1 -- 172.21.15.5:0/3253597149 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f3e1c0c62a0 con 0x7f3e1c15f220 2024-05-20T18:37:55.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.265+0000 7f3e18ff9640 1 -- 172.21.15.5:0/3253597149 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v113) ==== 104+0+0 (secure 0 0 0) 0x7f3e14058f50 con 0x7f3e1c15f220 2024-05-20T18:37:55.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.267+0000 7f3e22521640 1 -- 172.21.15.5:0/3253597149 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3dfc06be30 msgr2=0x7f3dfc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:55.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.267+0000 7f3e22521640 1 --2- 172.21.15.5:0/3253597149 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3dfc06be30 0x7f3dfc06e340 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c000f60 tx=0x7f3e0c007d00 comp rx=0 tx=0).stop 2024-05-20T18:37:55.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.267+0000 7f3e22521640 1 -- 172.21.15.5:0/3253597149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1c15f220 msgr2=0x7f3e1c1f9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:55.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.267+0000 7f3e22521640 1 --2- 172.21.15.5:0/3253597149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1c15f220 0x7f3e1c1f9570 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400c6e0 tx=0x7f3e1400cba0 comp rx=0 tx=0).stop 2024-05-20T18:37:55.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.268+0000 7f3e22521640 1 -- 172.21.15.5:0/3253597149 shutdown_connections 2024-05-20T18:37:55.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.268+0000 7f3e22521640 1 --2- 172.21.15.5:0/3253597149 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f3dfc06be30 0x7f3dfc06e340 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:55.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.268+0000 7f3e22521640 1 --2- 172.21.15.5:0/3253597149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1c15f220 0x7f3e1c1f9570 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.268+0000 7f3e22521640 1 --2- 172.21.15.5:0/3253597149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e1c0cf3d0 0x7f3e1c1f9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.268+0000 7f3e22521640 1 --2- 172.21.15.5:0/3253597149 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3e1c0ce7e0 0x7f3e1c151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.268+0000 7f3e22521640 1 -- 172.21.15.5:0/3253597149 >> 172.21.15.5:0/3253597149 conn(0x7f3e1c0b44d0 msgr2=0x7f3e1c0d08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.268+0000 7f3e22521640 1 -- 172.21.15.5:0/3253597149 shutdown_connections 2024-05-20T18:37:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.268+0000 7f3e22521640 1 -- 172.21.15.5:0/3253597149 wait complete. 2024-05-20T18:37:55.283 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-20T18:37:55.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.403+0000 7f673cdb5640 1 Processor -- start 2024-05-20T18:37:55.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.403+0000 7f673cdb5640 1 -- start start 2024-05-20T18:37:55.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.405+0000 7f673cdb5640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f67380b3860 0x7f67380d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:55.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.405+0000 7f673cdb5640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67380d0450 con 0x7f673806d870 2024-05-20T18:37:55.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.405+0000 7f673cdb5640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67380d3b10 con 0x7f673808bd20 2024-05-20T18:37:55.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.405+0000 7f673cdb5640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f67380d3df0 con 0x7f67380b3860 2024-05-20T18:37:55.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.405+0000 7f6735d9b640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f673806d870 0x7f673815f230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43166/0 (socket says 172.21.15.5:43166) 2024-05-20T18:37:55.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.405+0000 7f6735d9b640 1 -- 172.21.15.5:0/731360018 learned_addr learned my addr 172.21.15.5:0/731360018 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:55.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.405+0000 7f673559a640 1 -- 172.21.15.5:0/731360018 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4271564336 0 0) 0x7f67380d0450 con 0x7f673806d870 2024-05-20T18:37:55.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.405+0000 7f673659c640 1 --2- 172.21.15.5:0/731360018 >> v2:172.21.15.63:3300/0 conn(0x7f67380b3860 0x7f67380d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:55.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.405+0000 7f673559a640 1 -- 172.21.15.5:0/731360018 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6720003520 con 0x7f673806d870 2024-05-20T18:37:55.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.406+0000 7f673559a640 1 -- 172.21.15.5:0/731360018 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1208546226 0 0) 0x7f6720003520 con 0x7f673806d870 2024-05-20T18:37:55.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.406+0000 7f673559a640 1 -- 172.21.15.5:0/731360018 >> v2:172.21.15.63:3300/0 conn(0x7f67380b3860 msgr2=0x7f67380d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:55.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.406+0000 7f673559a640 1 --2- 172.21.15.5:0/731360018 >> v2:172.21.15.63:3300/0 conn(0x7f67380b3860 0x7f67380d45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:55.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.406+0000 7f673559a640 1 -- 172.21.15.5:0/731360018 >> v1:172.21.15.63:6789/0 conn(0x7f673808bd20 legacy=0x7f673808c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:55.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.406+0000 7f673559a640 1 -- 172.21.15.5:0/731360018 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67380d40d0 con 0x7f673806d870 2024-05-20T18:37:55.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.406+0000 7f673559a640 1 -- 172.21.15.5:0/731360018 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f672c001670 con 0x7f673806d870 2024-05-20T18:37:55.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.406+0000 7f673559a640 1 -- 172.21.15.5:0/731360018 >> v1:172.21.15.5:6789/0 conn(0x7f673806d870 legacy=0x7f673815f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:55.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.406+0000 7f673559a640 1 --2- 172.21.15.5:0/731360018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67200039b0 0x7f6720003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:55.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.406+0000 7f673559a640 1 --2- 172.21.15.5:0/731360018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6720004800 0x7f6720004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:55.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.406+0000 7f673559a640 1 --2- 172.21.15.5:0/731360018 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6720005350 0x7f672000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:55.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.406+0000 7f673559a640 1 -- 172.21.15.5:0/731360018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f67380d3df0 con 0x7f67200039b0 2024-05-20T18:37:55.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.406+0000 7f673559a640 1 -- 172.21.15.5:0/731360018 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f672000c280 con 0x7f6720004800 2024-05-20T18:37:55.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.406+0000 7f673559a640 1 -- 172.21.15.5:0/731360018 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f672000c480 con 0x7f6720005350 2024-05-20T18:37:55.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.406+0000 7f673659c640 1 --2- 172.21.15.5:0/731360018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67200039b0 0x7f6720003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:55.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.406+0000 7f6735d9b640 1 --2- 172.21.15.5:0/731360018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6720004800 0x7f6720004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:55.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.406+0000 7f6736d9d640 1 --2- 172.21.15.5:0/731360018 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6720005350 0x7f672000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:55.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.406+0000 7f673659c640 1 -- 172.21.15.5:0/731360018 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6720005350 msgr2=0x7f672000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:55.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.406+0000 7f673659c640 1 --2- 172.21.15.5:0/731360018 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6720005350 0x7f672000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:55.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.406+0000 7f673659c640 1 -- 172.21.15.5:0/731360018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6720004800 msgr2=0x7f6720004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:55.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.406+0000 7f673659c640 1 --2- 172.21.15.5:0/731360018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6720004800 0x7f6720004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:55.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.406+0000 7f673659c640 1 -- 172.21.15.5:0/731360018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6720003520 con 0x7f67200039b0 2024-05-20T18:37:55.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.406+0000 7f6736d9d640 1 --2- 172.21.15.5:0/731360018 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6720005350 0x7f672000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:55.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.406+0000 7f6736d9d640 1 -- 172.21.15.5:0/731360018 reap_dead start 2024-05-20T18:37:55.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.407+0000 7f673659c640 1 --2- 172.21.15.5:0/731360018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67200039b0 0x7f6720003e20 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f6728009a90 tx=0x7f6720005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7509c23ed33a2382 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:55.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.407+0000 7f673559a640 1 -- 172.21.15.5:0/731360018 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6728036070 con 0x7f67200039b0 2024-05-20T18:37:55.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.407+0000 7f673559a640 1 -- 172.21.15.5:0/731360018 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6728031400 con 0x7f67200039b0 2024-05-20T18:37:55.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.407+0000 7f673559a640 1 -- 172.21.15.5:0/731360018 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6728009690 con 0x7f67200039b0 2024-05-20T18:37:55.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.408+0000 7f673cdb5640 1 -- 172.21.15.5:0/731360018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67200039b0 msgr2=0x7f6720003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:55.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.408+0000 7f673cdb5640 1 --2- 172.21.15.5:0/731360018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67200039b0 0x7f6720003e20 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f6728009a90 tx=0x7f6720005830 comp rx=0 tx=0).stop 2024-05-20T18:37:55.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.408+0000 7f673cdb5640 1 -- 172.21.15.5:0/731360018 shutdown_connections 2024-05-20T18:37:55.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.408+0000 7f673cdb5640 1 --2- 172.21.15.5:0/731360018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67200039b0 0x7f6720003e20 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:55.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.408+0000 7f673cdb5640 1 -- 172.21.15.5:0/731360018 >> 172.21.15.5:0/731360018 conn(0x7f67380b6d10 msgr2=0x7f673806d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:55.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.408+0000 7f673cdb5640 1 -- 172.21.15.5:0/731360018 shutdown_connections 2024-05-20T18:37:55.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.408+0000 7f673cdb5640 1 -- 172.21.15.5:0/731360018 wait complete. 2024-05-20T18:37:55.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.410+0000 7f673cdb5640 1 Processor -- start 2024-05-20T18:37:55.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.410+0000 7f673cdb5640 1 -- start start 2024-05-20T18:37:55.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.411+0000 7f673cdb5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67200039b0 0x7f67381f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:55.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.411+0000 7f673cdb5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6738150fc0 0x7f673815a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:55.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.411+0000 7f673cdb5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f673815aad0 0x7f67381f8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:55.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.411+0000 7f673cdb5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f67380d4770 con 0x7f673815aad0 2024-05-20T18:37:55.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.411+0000 7f673cdb5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f67380d45f0 con 0x7f6738150fc0 2024-05-20T18:37:55.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.411+0000 7f673cdb5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f67380d48f0 con 0x7f67200039b0 2024-05-20T18:37:55.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.411+0000 7f6736d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f673815aad0 0x7f67381f8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:55.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.411+0000 7f6736d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f673815aad0 0x7f67381f8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60582/0 (socket says 172.21.15.5:60582) 2024-05-20T18:37:55.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.411+0000 7f6736d9d640 1 -- 172.21.15.5:0/3189876679 learned_addr learned my addr 172.21.15.5:0/3189876679 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:55.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.412+0000 7f6735d9b640 1 --2- 172.21.15.5:0/3189876679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6738150fc0 0x7f673815a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:55.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.412+0000 7f673659c640 1 --2- 172.21.15.5:0/3189876679 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67200039b0 0x7f67381f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:55.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.412+0000 7f6736d9d640 1 -- 172.21.15.5:0/3189876679 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67200039b0 msgr2=0x7f67381f5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:55.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.412+0000 7f6736d9d640 1 --2- 172.21.15.5:0/3189876679 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67200039b0 0x7f67381f5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:55.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.412+0000 7f6736d9d640 1 -- 172.21.15.5:0/3189876679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6738150fc0 msgr2=0x7f673815a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:55.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.412+0000 7f6736d9d640 1 --2- 172.21.15.5:0/3189876679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6738150fc0 0x7f673815a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:55.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.412+0000 7f6736d9d640 1 -- 172.21.15.5:0/3189876679 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67380ce000 con 0x7f673815aad0 2024-05-20T18:37:55.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.412+0000 7f673659c640 1 --2- 172.21.15.5:0/3189876679 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67200039b0 0x7f67381f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:55.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.412+0000 7f6736d9d640 1 --2- 172.21.15.5:0/3189876679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f673815aad0 0x7f67381f8660 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f67380d14c0 tx=0x7f673000ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:55.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.412+0000 7f6726ffd640 1 -- 172.21.15.5:0/3189876679 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6730003b80 con 0x7f673815aad0 2024-05-20T18:37:55.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.412+0000 7f6726ffd640 1 -- 172.21.15.5:0/3189876679 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6730018070 con 0x7f673815aad0 2024-05-20T18:37:55.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.412+0000 7f6726ffd640 1 -- 172.21.15.5:0/3189876679 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67300050d0 con 0x7f673815aad0 2024-05-20T18:37:55.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.412+0000 7f673cdb5640 1 -- 172.21.15.5:0/3189876679 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f673815aee0 con 0x7f673815aad0 2024-05-20T18:37:55.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.412+0000 7f673cdb5640 1 -- 172.21.15.5:0/3189876679 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67380c8b90 con 0x7f673815aad0 2024-05-20T18:37:55.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.413+0000 7f673cdb5640 1 -- 172.21.15.5:0/3189876679 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67380690b0 con 0x7f673815aad0 2024-05-20T18:37:55.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.414+0000 7f6726ffd640 1 -- 172.21.15.5:0/3189876679 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6730005270 con 0x7f673815aad0 2024-05-20T18:37:55.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.414+0000 7f6726ffd640 1 --2- 172.21.15.5:0/3189876679 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f671806be30 0x7f671806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:55.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.414+0000 7f6726ffd640 1 -- 172.21.15.5:0/3189876679 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6359+0+0 (secure 0 0 0) 0x7f673008aa70 con 0x7f673815aad0 2024-05-20T18:37:55.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.414+0000 7f673659c640 1 --2- 172.21.15.5:0/3189876679 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f671806be30 0x7f671806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:55.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.415+0000 7f673659c640 1 --2- 172.21.15.5:0/3189876679 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f671806be30 0x7f671806e340 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6728033000 tx=0x7f6728008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:55.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.419+0000 7f6726ffd640 1 -- 172.21.15.5:0/3189876679 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6730022020 con 0x7f673815aad0 2024-05-20T18:37:55.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:55.602+0000 7f673cdb5640 1 -- 172.21.15.5:0/3189876679 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f67380c8900 con 0x7f673815aad0 2024-05-20T18:37:56.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.267+0000 7f6726ffd640 1 -- 172.21.15.5:0/3189876679 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v123) ==== 181+0+0 (secure 0 0 0) 0x7f673815aee0 con 0x7f673815aad0 2024-05-20T18:37:56.268 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-05-20T18:37:56.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.271+0000 7f673cdb5640 1 -- 172.21.15.5:0/3189876679 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f671806be30 msgr2=0x7f671806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:56.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.271+0000 7f673cdb5640 1 --2- 172.21.15.5:0/3189876679 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f671806be30 0x7f671806e340 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6728033000 tx=0x7f6728008030 comp rx=0 tx=0).stop 2024-05-20T18:37:56.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.271+0000 7f673cdb5640 1 -- 172.21.15.5:0/3189876679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f673815aad0 msgr2=0x7f67381f8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:56.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.271+0000 7f673cdb5640 1 --2- 172.21.15.5:0/3189876679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f673815aad0 0x7f67381f8660 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f67380d14c0 tx=0x7f673000ead0 comp rx=0 tx=0).stop 2024-05-20T18:37:56.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.272+0000 7f673cdb5640 1 -- 172.21.15.5:0/3189876679 shutdown_connections 2024-05-20T18:37:56.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.272+0000 7f673cdb5640 1 --2- 172.21.15.5:0/3189876679 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f671806be30 0x7f671806e340 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:56.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.272+0000 7f673cdb5640 1 --2- 172.21.15.5:0/3189876679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f673815aad0 0x7f67381f8660 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:56.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.272+0000 7f673cdb5640 1 --2- 172.21.15.5:0/3189876679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6738150fc0 0x7f673815a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:56.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.272+0000 7f673cdb5640 1 --2- 172.21.15.5:0/3189876679 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f67200039b0 0x7f67381f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:56.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.272+0000 7f673cdb5640 1 -- 172.21.15.5:0/3189876679 >> 172.21.15.5:0/3189876679 conn(0x7f67380b6d10 msgr2=0x7f67380d27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:56.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.272+0000 7f673cdb5640 1 -- 172.21.15.5:0/3189876679 shutdown_connections 2024-05-20T18:37:56.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.272+0000 7f673cdb5640 1 -- 172.21.15.5:0/3189876679 wait complete. 2024-05-20T18:37:56.288 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-20T18:37:56.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.414+0000 7efe81c5a640 1 Processor -- start 2024-05-20T18:37:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.414+0000 7efe81c5a640 1 -- start start 2024-05-20T18:37:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.415+0000 7efe81c5a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efe7c0b3860 0x7efe7c0d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.415+0000 7efe81c5a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7efe7c06d870 0x7efe7c15f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.415+0000 7efe81c5a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe7c08bd20 0x7efe7c08c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.415+0000 7efe81c5a640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7efe7c08c670 con 0x7efe7c0b3860 2024-05-20T18:37:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.415+0000 7efe81c5a640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7efe7c0d0450 con 0x7efe7c06d870 2024-05-20T18:37:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.415+0000 7efe81c5a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efe7c0d3b10 con 0x7efe7c08bd20 2024-05-20T18:37:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.416+0000 7efe7b7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efe7c0b3860 0x7efe7c0d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.416+0000 7efe7b7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efe7c0b3860 0x7efe7c0d45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60596/0 (socket says 172.21.15.5:60596) 2024-05-20T18:37:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.416+0000 7efe7b7fe640 1 -- 172.21.15.5:0/1570065138 learned_addr learned my addr 172.21.15.5:0/1570065138 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.416+0000 7efe7bfff640 1 --2- 172.21.15.5:0/1570065138 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe7c08bd20 0x7efe7c08c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.416+0000 7efe7affd640 1 --2- 172.21.15.5:0/1570065138 >> v2:172.21.15.63:3300/0 conn(0x7efe7c06d870 0x7efe7c15f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.416+0000 7efe7b7fe640 1 -- 172.21.15.5:0/1570065138 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe7c08bd20 msgr2=0x7efe7c08c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.416+0000 7efe7b7fe640 1 --2- 172.21.15.5:0/1570065138 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe7c08bd20 0x7efe7c08c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.416+0000 7efe7b7fe640 1 -- 172.21.15.5:0/1570065138 >> v2:172.21.15.63:3300/0 conn(0x7efe7c06d870 msgr2=0x7efe7c15f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:56.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.416+0000 7efe7b7fe640 1 --2- 172.21.15.5:0/1570065138 >> v2:172.21.15.63:3300/0 conn(0x7efe7c06d870 0x7efe7c15f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.416+0000 7efe7b7fe640 1 -- 172.21.15.5:0/1570065138 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe7c0d3df0 con 0x7efe7c0b3860 2024-05-20T18:37:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.416+0000 7efe7bfff640 1 --2- 172.21.15.5:0/1570065138 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe7c08bd20 0x7efe7c08c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.417+0000 7efe7b7fe640 1 --2- 172.21.15.5:0/1570065138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7c0b3860 0x7efe7c0d45f0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7efe7c0d14f0 tx=0x7efe6c02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f9461bdef26583 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.417+0000 7efe7a7fc640 1 -- 172.21.15.5:0/1570065138 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c037070 con 0x7efe7c0b3860 2024-05-20T18:37:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.417+0000 7efe7a7fc640 1 -- 172.21.15.5:0/1570065138 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe6c02ae00 con 0x7efe7c0b3860 2024-05-20T18:37:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.417+0000 7efe7a7fc640 1 -- 172.21.15.5:0/1570065138 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c032660 con 0x7efe7c0b3860 2024-05-20T18:37:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.418+0000 7efe81c5a640 1 -- 172.21.15.5:0/1570065138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7c0b3860 msgr2=0x7efe7c0d45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.418+0000 7efe81c5a640 1 --2- 172.21.15.5:0/1570065138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7c0b3860 0x7efe7c0d45f0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7efe7c0d14f0 tx=0x7efe6c02a0b0 comp rx=0 tx=0).stop 2024-05-20T18:37:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.418+0000 7efe81c5a640 1 -- 172.21.15.5:0/1570065138 shutdown_connections 2024-05-20T18:37:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.418+0000 7efe81c5a640 1 --2- 172.21.15.5:0/1570065138 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe7c08bd20 0x7efe7c08c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.418+0000 7efe81c5a640 1 --2- 172.21.15.5:0/1570065138 >> v2:172.21.15.63:3300/0 conn(0x7efe7c06d870 0x7efe7c15f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.418+0000 7efe81c5a640 1 --2- 172.21.15.5:0/1570065138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7c0b3860 0x7efe7c0d45f0 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.418+0000 7efe81c5a640 1 -- 172.21.15.5:0/1570065138 >> 172.21.15.5:0/1570065138 conn(0x7efe7c0b6d10 msgr2=0x7efe7c06d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.418+0000 7efe81c5a640 1 -- 172.21.15.5:0/1570065138 shutdown_connections 2024-05-20T18:37:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.418+0000 7efe81c5a640 1 -- 172.21.15.5:0/1570065138 wait complete. 2024-05-20T18:37:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.420+0000 7efe81c5a640 1 Processor -- start 2024-05-20T18:37:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.420+0000 7efe81c5a640 1 -- start start 2024-05-20T18:37:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.421+0000 7efe81c5a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7c06d870 0x7efe7c0cee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.421+0000 7efe81c5a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe7c08bd20 0x7efe7c0b9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.421+0000 7efe81c5a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe7c0b3860 0x7efe7c0b9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.421+0000 7efe81c5a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c15f770 con 0x7efe7c06d870 2024-05-20T18:37:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.421+0000 7efe81c5a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c0d0730 con 0x7efe7c08bd20 2024-05-20T18:37:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.421+0000 7efe81c5a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efe7c0d27c0 con 0x7efe7c0b3860 2024-05-20T18:37:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.421+0000 7efe7b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7c06d870 0x7efe7c0cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.421+0000 7efe7b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7c06d870 0x7efe7c0cee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60606/0 (socket says 172.21.15.5:60606) 2024-05-20T18:37:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.421+0000 7efe7b7fe640 1 -- 172.21.15.5:0/1953407157 learned_addr learned my addr 172.21.15.5:0/1953407157 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.421+0000 7efe7affd640 1 --2- 172.21.15.5:0/1953407157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe7c08bd20 0x7efe7c0b9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:56.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.421+0000 7efe7bfff640 1 --2- 172.21.15.5:0/1953407157 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe7c0b3860 0x7efe7c0b9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:56.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.422+0000 7efe7b7fe640 1 -- 172.21.15.5:0/1953407157 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe7c0b3860 msgr2=0x7efe7c0b9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:56.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.422+0000 7efe7b7fe640 1 --2- 172.21.15.5:0/1953407157 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe7c0b3860 0x7efe7c0b9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:56.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.422+0000 7efe7b7fe640 1 -- 172.21.15.5:0/1953407157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe7c08bd20 msgr2=0x7efe7c0b9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:56.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.422+0000 7efe7b7fe640 1 --2- 172.21.15.5:0/1953407157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe7c08bd20 0x7efe7c0b9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:56.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.422+0000 7efe7b7fe640 1 -- 172.21.15.5:0/1953407157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe7c14d410 con 0x7efe7c06d870 2024-05-20T18:37:56.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.422+0000 7efe7bfff640 1 --2- 172.21.15.5:0/1953407157 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe7c0b3860 0x7efe7c0b9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:56.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.422+0000 7efe7b7fe640 1 --2- 172.21.15.5:0/1953407157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7c06d870 0x7efe7c0cee20 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7efe6c02a0f0 tx=0x7efe6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:56.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.422+0000 7efe63fff640 1 -- 172.21.15.5:0/1953407157 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c041070 con 0x7efe7c06d870 2024-05-20T18:37:56.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.422+0000 7efe63fff640 1 -- 172.21.15.5:0/1953407157 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe6c03c670 con 0x7efe7c06d870 2024-05-20T18:37:56.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.422+0000 7efe81c5a640 1 -- 172.21.15.5:0/1953407157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe7c14d6f0 con 0x7efe7c06d870 2024-05-20T18:37:56.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.422+0000 7efe63fff640 1 -- 172.21.15.5:0/1953407157 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c037040 con 0x7efe7c06d870 2024-05-20T18:37:56.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.422+0000 7efe81c5a640 1 -- 172.21.15.5:0/1953407157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe7c14dcb0 con 0x7efe7c06d870 2024-05-20T18:37:56.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.423+0000 7efe81c5a640 1 -- 172.21.15.5:0/1953407157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe7c0690b0 con 0x7efe7c06d870 2024-05-20T18:37:56.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.424+0000 7efe63fff640 1 -- 172.21.15.5:0/1953407157 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efe6c03a070 con 0x7efe7c06d870 2024-05-20T18:37:56.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.424+0000 7efe63fff640 1 --2- 172.21.15.5:0/1953407157 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efe6806be30 0x7efe6806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:56.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.424+0000 7efe63fff640 1 -- 172.21.15.5:0/1953407157 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5921+0+0 (secure 0 0 0) 0x7efe6c0aaf60 con 0x7efe7c06d870 2024-05-20T18:37:56.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.424+0000 7efe7affd640 1 --2- 172.21.15.5:0/1953407157 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efe6806be30 0x7efe6806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:56.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.425+0000 7efe7affd640 1 --2- 172.21.15.5:0/1953407157 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efe6806be30 0x7efe6806e340 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7efe70006fd0 tx=0x7efe70008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:56.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.429+0000 7efe63fff640 1 -- 172.21.15.5:0/1953407157 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7efe6c0797b0 con 0x7efe7c06d870 2024-05-20T18:37:56.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:56.614+0000 7efe81c5a640 1 -- 172.21.15.5:0/1953407157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7efe7c06d4e0 con 0x7efe7c06d870 2024-05-20T18:37:57.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.272+0000 7efe63fff640 1 -- 172.21.15.5:0/1953407157 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v124) ==== 169+0+0 (secure 0 0 0) 0x7efe6c0790b0 con 0x7efe7c06d870 2024-05-20T18:37:57.274 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-05-20T18:37:57.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.277+0000 7efe81c5a640 1 -- 172.21.15.5:0/1953407157 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efe6806be30 msgr2=0x7efe6806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:57.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.277+0000 7efe81c5a640 1 --2- 172.21.15.5:0/1953407157 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efe6806be30 0x7efe6806e340 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7efe70006fd0 tx=0x7efe70008040 comp rx=0 tx=0).stop 2024-05-20T18:37:57.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.277+0000 7efe81c5a640 1 -- 172.21.15.5:0/1953407157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7c06d870 msgr2=0x7efe7c0cee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:57.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.277+0000 7efe81c5a640 1 --2- 172.21.15.5:0/1953407157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7c06d870 0x7efe7c0cee20 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7efe6c02a0f0 tx=0x7efe6c0046c0 comp rx=0 tx=0).stop 2024-05-20T18:37:57.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.278+0000 7efe81c5a640 1 -- 172.21.15.5:0/1953407157 shutdown_connections 2024-05-20T18:37:57.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.278+0000 7efe81c5a640 1 --2- 172.21.15.5:0/1953407157 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efe6806be30 0x7efe6806e340 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:57.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.278+0000 7efe81c5a640 1 --2- 172.21.15.5:0/1953407157 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efe7c0b3860 0x7efe7c0b9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:57.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.278+0000 7efe81c5a640 1 --2- 172.21.15.5:0/1953407157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe7c08bd20 0x7efe7c0b9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:57.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.278+0000 7efe81c5a640 1 --2- 172.21.15.5:0/1953407157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7c06d870 0x7efe7c0cee20 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:57.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.278+0000 7efe81c5a640 1 -- 172.21.15.5:0/1953407157 >> 172.21.15.5:0/1953407157 conn(0x7efe7c0b6d10 msgr2=0x7efe7c15b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:57.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.278+0000 7efe81c5a640 1 -- 172.21.15.5:0/1953407157 shutdown_connections 2024-05-20T18:37:57.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.278+0000 7efe81c5a640 1 -- 172.21.15.5:0/1953407157 wait complete. 2024-05-20T18:37:57.295 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-20T18:37:57.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.417+0000 7fc9d338f640 1 Processor -- start 2024-05-20T18:37:57.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.417+0000 7fc9d338f640 1 -- start start 2024-05-20T18:37:57.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.419+0000 7fc9d338f640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc9cc06d3d0 0x7fc9cc06d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:57.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.419+0000 7fc9d338f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9cc0b8190 0x7fc9cc0b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:57.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.419+0000 7fc9d338f640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9cc06dd20 con 0x7fc9cc162820 2024-05-20T18:37:57.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.419+0000 7fc9d338f640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc9cc0d0630 con 0x7fc9cc06d3d0 2024-05-20T18:37:57.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.419+0000 7fc9d338f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc9cc0d39c0 con 0x7fc9cc0b8190 2024-05-20T18:37:57.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.419+0000 7fc9d192c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9cc0b8190 0x7fc9cc0b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:57.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.419+0000 7fc9d092a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc9cc06d3d0 0x7fc9cc06d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:57.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.419+0000 7fc9d192c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9cc0b8190 0x7fc9cc0b85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.5:57668/0 (socket says 172.21.15.5:57668) 2024-05-20T18:37:57.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.419+0000 7fc9d112b640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fc9cc162820 0x7fc9cc0b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:34644/0 (socket says 172.21.15.5:34644) 2024-05-20T18:37:57.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.419+0000 7fc9d192c640 1 -- 172.21.15.5:0/662811010 learned_addr learned my addr 172.21.15.5:0/662811010 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:57.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.420+0000 7fc9d092a640 1 -- 172.21.15.5:0/662811010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9cc0b8190 msgr2=0x7fc9cc0b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:57.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.420+0000 7fc9d092a640 1 --2- 172.21.15.5:0/662811010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9cc0b8190 0x7fc9cc0b85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:57.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.420+0000 7fc9d092a640 1 -- 172.21.15.5:0/662811010 >> v1:172.21.15.63:6789/0 conn(0x7fc9cc162820 legacy=0x7fc9cc0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:57.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.420+0000 7fc9d092a640 1 -- 172.21.15.5:0/662811010 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9cc0d3ca0 con 0x7fc9cc06d3d0 2024-05-20T18:37:57.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.420+0000 7fc9cbfff640 1 -- 172.21.15.5:0/662811010 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4220140023 0 0) 0x7fc9c4001110 con 0x7fc9cc162820 2024-05-20T18:37:57.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.420+0000 7fc9d192c640 1 --2- 172.21.15.5:0/662811010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9cc0b8190 0x7fc9cc0b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:37:57.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.420+0000 7fc9d092a640 1 --2- 172.21.15.5:0/662811010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9cc06d3d0 0x7fc9cc06d7e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc9c00097a0 tx=0x7fc9c002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5554ab077dffa4cb server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:57.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.421+0000 7fc9cbfff640 1 -- 172.21.15.5:0/662811010 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0037070 con 0x7fc9cc06d3d0 2024-05-20T18:37:57.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.421+0000 7fc9cbfff640 1 -- 172.21.15.5:0/662811010 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9c0032440 con 0x7fc9cc06d3d0 2024-05-20T18:37:57.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.421+0000 7fc9cbfff640 1 -- 172.21.15.5:0/662811010 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0032810 con 0x7fc9cc06d3d0 2024-05-20T18:37:57.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.421+0000 7fc9d338f640 1 -- 172.21.15.5:0/662811010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9cc06d3d0 msgr2=0x7fc9cc06d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:57.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.421+0000 7fc9d338f640 1 --2- 172.21.15.5:0/662811010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9cc06d3d0 0x7fc9cc06d7e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc9c00097a0 tx=0x7fc9c002a180 comp rx=0 tx=0).stop 2024-05-20T18:37:57.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.421+0000 7fc9d338f640 1 -- 172.21.15.5:0/662811010 shutdown_connections 2024-05-20T18:37:57.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.421+0000 7fc9d338f640 1 --2- 172.21.15.5:0/662811010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9cc0b8190 0x7fc9cc0b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:57.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.421+0000 7fc9d338f640 1 --2- 172.21.15.5:0/662811010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9cc06d3d0 0x7fc9cc06d7e0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:57.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.421+0000 7fc9d338f640 1 -- 172.21.15.5:0/662811010 >> 172.21.15.5:0/662811010 conn(0x7fc9cc0d4bd0 msgr2=0x7fc9cc0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:57.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.421+0000 7fc9d338f640 1 -- 172.21.15.5:0/662811010 shutdown_connections 2024-05-20T18:37:57.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.421+0000 7fc9d338f640 1 -- 172.21.15.5:0/662811010 wait complete. 2024-05-20T18:37:57.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.423+0000 7fc9d338f640 1 Processor -- start 2024-05-20T18:37:57.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.423+0000 7fc9d338f640 1 -- start start 2024-05-20T18:37:57.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.424+0000 7fc9d338f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9cc06d3d0 0x7fc9cc14b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:57.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.424+0000 7fc9d338f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9cc0b8190 0x7fc9cc14c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:57.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.425+0000 7fc9d338f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9cc162820 0x7fc9cc14c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:57.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.425+0000 7fc9d338f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc9cc0d0300 con 0x7fc9cc06d3d0 2024-05-20T18:37:57.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.425+0000 7fc9d338f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc9cc0d4260 con 0x7fc9cc0b8190 2024-05-20T18:37:57.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.425+0000 7fc9d338f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc9cc0b4050 con 0x7fc9cc162820 2024-05-20T18:37:57.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.425+0000 7fc9d112b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9cc06d3d0 0x7fc9cc14b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:57.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.425+0000 7fc9d112b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9cc06d3d0 0x7fc9cc14b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60612/0 (socket says 172.21.15.5:60612) 2024-05-20T18:37:57.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.425+0000 7fc9d112b640 1 -- 172.21.15.5:0/2024209935 learned_addr learned my addr 172.21.15.5:0/2024209935 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:57.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.425+0000 7fc9d092a640 1 --2- 172.21.15.5:0/2024209935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9cc0b8190 0x7fc9cc14c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:57.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.425+0000 7fc9d192c640 1 --2- 172.21.15.5:0/2024209935 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9cc162820 0x7fc9cc14c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:57.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.425+0000 7fc9d112b640 1 -- 172.21.15.5:0/2024209935 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9cc162820 msgr2=0x7fc9cc14c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:57.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.425+0000 7fc9d112b640 1 --2- 172.21.15.5:0/2024209935 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9cc162820 0x7fc9cc14c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:57.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.425+0000 7fc9d112b640 1 -- 172.21.15.5:0/2024209935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9cc0b8190 msgr2=0x7fc9cc14c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:57.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.425+0000 7fc9d112b640 1 --2- 172.21.15.5:0/2024209935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9cc0b8190 0x7fc9cc14c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:57.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.425+0000 7fc9d112b640 1 -- 172.21.15.5:0/2024209935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9cc0ca570 con 0x7fc9cc06d3d0 2024-05-20T18:37:57.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.425+0000 7fc9d092a640 1 --2- 172.21.15.5:0/2024209935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9cc0b8190 0x7fc9cc14c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:37:57.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.425+0000 7fc9d112b640 1 --2- 172.21.15.5:0/2024209935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9cc06d3d0 0x7fc9cc14b0d0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400c460 tx=0x7fc9c400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:57.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.425+0000 7fc9c9ffb640 1 -- 172.21.15.5:0/2024209935 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4018070 con 0x7fc9cc06d3d0 2024-05-20T18:37:57.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.425+0000 7fc9c9ffb640 1 -- 172.21.15.5:0/2024209935 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9c4007500 con 0x7fc9cc06d3d0 2024-05-20T18:37:57.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.425+0000 7fc9d338f640 1 -- 172.21.15.5:0/2024209935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9cc0ca290 con 0x7fc9cc06d3d0 2024-05-20T18:37:57.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.426+0000 7fc9c9ffb640 1 -- 172.21.15.5:0/2024209935 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4004a20 con 0x7fc9cc06d3d0 2024-05-20T18:37:57.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.426+0000 7fc9d338f640 1 -- 172.21.15.5:0/2024209935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9cc14bc90 con 0x7fc9cc06d3d0 2024-05-20T18:37:57.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.427+0000 7fc9d338f640 1 -- 172.21.15.5:0/2024209935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9cc075b00 con 0x7fc9cc06d3d0 2024-05-20T18:37:57.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.427+0000 7fc9c9ffb640 1 -- 172.21.15.5:0/2024209935 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc9c401b070 con 0x7fc9cc06d3d0 2024-05-20T18:37:57.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.427+0000 7fc9c9ffb640 1 --2- 172.21.15.5:0/2024209935 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc9b806be60 0x7fc9b806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:57.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.428+0000 7fc9c9ffb640 1 -- 172.21.15.5:0/2024209935 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7fc9c4089f70 con 0x7fc9cc06d3d0 2024-05-20T18:37:57.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.428+0000 7fc9d092a640 1 --2- 172.21.15.5:0/2024209935 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc9b806be60 0x7fc9b806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:57.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.428+0000 7fc9d092a640 1 --2- 172.21.15.5:0/2024209935 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc9b806be60 0x7fc9b806e370 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0034000 tx=0x7fc9c002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:57.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.432+0000 7fc9c9ffb640 1 -- 172.21.15.5:0/2024209935 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc9cc06dd20 con 0x7fc9cc06d3d0 2024-05-20T18:37:57.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.618+0000 7fc9d338f640 1 -- 172.21.15.5:0/2024209935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fc9cc161dd0 con 0x7fc9cc06d3d0 2024-05-20T18:37:57.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.621+0000 7fc9c9ffb640 1 -- 172.21.15.5:0/2024209935 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7fc9c4058420 con 0x7fc9cc06d3d0 2024-05-20T18:37:57.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.623+0000 7fc9d338f640 1 -- 172.21.15.5:0/2024209935 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc9b806be60 msgr2=0x7fc9b806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:57.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.623+0000 7fc9d338f640 1 --2- 172.21.15.5:0/2024209935 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc9b806be60 0x7fc9b806e370 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0034000 tx=0x7fc9c002e000 comp rx=0 tx=0).stop 2024-05-20T18:37:57.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.623+0000 7fc9d338f640 1 -- 172.21.15.5:0/2024209935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9cc06d3d0 msgr2=0x7fc9cc14b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:57.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.623+0000 7fc9d338f640 1 --2- 172.21.15.5:0/2024209935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9cc06d3d0 0x7fc9cc14b0d0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400c460 tx=0x7fc9c400c920 comp rx=0 tx=0).stop 2024-05-20T18:37:57.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.623+0000 7fc9d338f640 1 -- 172.21.15.5:0/2024209935 shutdown_connections 2024-05-20T18:37:57.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.623+0000 7fc9d338f640 1 --2- 172.21.15.5:0/2024209935 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc9b806be60 0x7fc9b806e370 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:57.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.623+0000 7fc9d338f640 1 --2- 172.21.15.5:0/2024209935 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9cc162820 0x7fc9cc14c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:57.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.623+0000 7fc9d338f640 1 --2- 172.21.15.5:0/2024209935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9cc0b8190 0x7fc9cc14c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:57.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.623+0000 7fc9d338f640 1 --2- 172.21.15.5:0/2024209935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9cc06d3d0 0x7fc9cc14b0d0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:57.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.623+0000 7fc9d338f640 1 -- 172.21.15.5:0/2024209935 >> 172.21.15.5:0/2024209935 conn(0x7fc9cc0d4bd0 msgr2=0x7fc9cc15da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:57.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.623+0000 7fc9d338f640 1 -- 172.21.15.5:0/2024209935 shutdown_connections 2024-05-20T18:37:57.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.623+0000 7fc9d338f640 1 -- 172.21.15.5:0/2024209935 wait complete. 2024-05-20T18:37:57.636 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-05-20T18:37:57.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.751+0000 7f80a8b2f640 1 Processor -- start 2024-05-20T18:37:57.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.751+0000 7f80a8b2f640 1 -- start start 2024-05-20T18:37:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.753+0000 7f80a8b2f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f80a415cf50 0x7f80a40b1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.753+0000 7f80a8b2f640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f80a40b1eb0 0x7f80a40d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.753+0000 7f80a8b2f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a40d1230 0x7f80a40708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.753+0000 7f80a8b2f640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f80a40c9400 con 0x7f80a415cf50 2024-05-20T18:37:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.753+0000 7f80a8b2f640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f80a40c8f70 con 0x7f80a40b1eb0 2024-05-20T18:37:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.753+0000 7f80a8b2f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f80a40d18d0 con 0x7f80a40d1230 2024-05-20T18:37:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.753+0000 7f80a259c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f80a415cf50 0x7f80a40b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.753+0000 7f80a259c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f80a415cf50 0x7f80a40b1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60614/0 (socket says 172.21.15.5:60614) 2024-05-20T18:37:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.753+0000 7f80a259c640 1 -- 172.21.15.5:0/2779700375 learned_addr learned my addr 172.21.15.5:0/2779700375 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.754+0000 7f8099d9b640 1 --2- 172.21.15.5:0/2779700375 >> v2:172.21.15.63:3300/0 conn(0x7f80a40b1eb0 0x7f80a40d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.754+0000 7f80a2d9d640 1 --2- 172.21.15.5:0/2779700375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a40d1230 0x7f80a40708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.754+0000 7f80a259c640 1 -- 172.21.15.5:0/2779700375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a40d1230 msgr2=0x7f80a40708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.754+0000 7f80a259c640 1 --2- 172.21.15.5:0/2779700375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a40d1230 0x7f80a40708c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.754+0000 7f80a259c640 1 -- 172.21.15.5:0/2779700375 >> v2:172.21.15.63:3300/0 conn(0x7f80a40b1eb0 msgr2=0x7f80a40d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.754+0000 7f80a259c640 1 --2- 172.21.15.5:0/2779700375 >> v2:172.21.15.63:3300/0 conn(0x7f80a40b1eb0 0x7f80a40d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.754+0000 7f80a259c640 1 -- 172.21.15.5:0/2779700375 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a40b22c0 con 0x7f80a415cf50 2024-05-20T18:37:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.754+0000 7f80a2d9d640 1 --2- 172.21.15.5:0/2779700375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a40d1230 0x7f80a40708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:57.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.754+0000 7f8099d9b640 1 --2- 172.21.15.5:0/2779700375 >> v2:172.21.15.63:3300/0 conn(0x7f80a40b1eb0 0x7f80a40d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:57.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.755+0000 7f80a259c640 1 --2- 172.21.15.5:0/2779700375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80a415cf50 0x7f80a40b1970 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f80a40d0010 tx=0x7f809402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02e32859138b397 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:57.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.755+0000 7f80a1d9b640 1 -- 172.21.15.5:0/2779700375 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8094037070 con 0x7f80a415cf50 2024-05-20T18:37:57.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.755+0000 7f80a1d9b640 1 -- 172.21.15.5:0/2779700375 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8094032440 con 0x7f80a415cf50 2024-05-20T18:37:57.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.755+0000 7f80a1d9b640 1 -- 172.21.15.5:0/2779700375 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8094032810 con 0x7f80a415cf50 2024-05-20T18:37:57.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.756+0000 7f80a8b2f640 1 -- 172.21.15.5:0/2779700375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80a415cf50 msgr2=0x7f80a40b1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:57.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.756+0000 7f80a8b2f640 1 --2- 172.21.15.5:0/2779700375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80a415cf50 0x7f80a40b1970 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f80a40d0010 tx=0x7f809402a140 comp rx=0 tx=0).stop 2024-05-20T18:37:57.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.756+0000 7f80a8b2f640 1 -- 172.21.15.5:0/2779700375 shutdown_connections 2024-05-20T18:37:57.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.756+0000 7f80a8b2f640 1 --2- 172.21.15.5:0/2779700375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a40d1230 0x7f80a40708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:57.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.756+0000 7f80a8b2f640 1 --2- 172.21.15.5:0/2779700375 >> v2:172.21.15.63:3300/0 conn(0x7f80a40b1eb0 0x7f80a40d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:57.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.756+0000 7f80a8b2f640 1 --2- 172.21.15.5:0/2779700375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80a415cf50 0x7f80a40b1970 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:57.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.756+0000 7f80a8b2f640 1 -- 172.21.15.5:0/2779700375 >> 172.21.15.5:0/2779700375 conn(0x7f80a406f2b0 msgr2=0x7f80a40b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:57.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.756+0000 7f80a8b2f640 1 -- 172.21.15.5:0/2779700375 shutdown_connections 2024-05-20T18:37:57.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.756+0000 7f80a8b2f640 1 -- 172.21.15.5:0/2779700375 wait complete. 2024-05-20T18:37:57.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.758+0000 7f80a8b2f640 1 Processor -- start 2024-05-20T18:37:57.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.758+0000 7f80a8b2f640 1 -- start start 2024-05-20T18:37:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.760+0000 7f80a8b2f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a40b1eb0 0x7f80a415b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.760+0000 7f80a8b2f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80a40d1230 0x7f80a414df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.760+0000 7f80a8b2f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80a415cf50 0x7f80a40cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.760+0000 7f80a8b2f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f80a40d1640 con 0x7f80a40d1230 2024-05-20T18:37:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.760+0000 7f80a8b2f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f80a40ccde0 con 0x7f80a415cf50 2024-05-20T18:37:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.760+0000 7f80a8b2f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f80a40c8920 con 0x7f80a40b1eb0 2024-05-20T18:37:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.760+0000 7f8099d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80a40d1230 0x7f80a414df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.760+0000 7f8099d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80a40d1230 0x7f80a414df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60624/0 (socket says 172.21.15.5:60624) 2024-05-20T18:37:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.760+0000 7f8099d9b640 1 -- 172.21.15.5:0/1935905973 learned_addr learned my addr 172.21.15.5:0/1935905973 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.760+0000 7f80a2d9d640 1 --2- 172.21.15.5:0/1935905973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80a415cf50 0x7f80a40cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.760+0000 7f80a259c640 1 --2- 172.21.15.5:0/1935905973 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a40b1eb0 0x7f80a415b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.760+0000 7f8099d9b640 1 -- 172.21.15.5:0/1935905973 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a40b1eb0 msgr2=0x7f80a415b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.760+0000 7f8099d9b640 1 --2- 172.21.15.5:0/1935905973 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a40b1eb0 0x7f80a415b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.760+0000 7f8099d9b640 1 -- 172.21.15.5:0/1935905973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80a415cf50 msgr2=0x7f80a40cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.760+0000 7f8099d9b640 1 --2- 172.21.15.5:0/1935905973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80a415cf50 0x7f80a40cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.760+0000 7f8099d9b640 1 -- 172.21.15.5:0/1935905973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a415b940 con 0x7f80a40d1230 2024-05-20T18:37:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.760+0000 7f8099d9b640 1 --2- 172.21.15.5:0/1935905973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80a40d1230 0x7f80a414df30 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f808c00c610 tx=0x7f808c00cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.760+0000 7f809b7fe640 1 -- 172.21.15.5:0/1935905973 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c015070 con 0x7f80a40d1230 2024-05-20T18:37:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.761+0000 7f80a8b2f640 1 -- 172.21.15.5:0/1935905973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a415bc20 con 0x7f80a40d1230 2024-05-20T18:37:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.761+0000 7f809b7fe640 1 -- 172.21.15.5:0/1935905973 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f808c011800 con 0x7f80a40d1230 2024-05-20T18:37:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.761+0000 7f809b7fe640 1 -- 172.21.15.5:0/1935905973 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c011bf0 con 0x7f80a40d1230 2024-05-20T18:37:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.761+0000 7f80a8b2f640 1 -- 172.21.15.5:0/1935905973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a415c1e0 con 0x7f80a40d1230 2024-05-20T18:37:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.762+0000 7f80a8b2f640 1 -- 172.21.15.5:0/1935905973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80a4075b00 con 0x7f80a40d1230 2024-05-20T18:37:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.762+0000 7f809b7fe640 1 -- 172.21.15.5:0/1935905973 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f808c018070 con 0x7f80a40d1230 2024-05-20T18:37:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.763+0000 7f809b7fe640 1 --2- 172.21.15.5:0/1935905973 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f808406be30 0x7f808406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.763+0000 7f809b7fe640 1 -- 172.21.15.5:0/1935905973 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f808c08ad50 con 0x7f80a40d1230 2024-05-20T18:37:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.763+0000 7f80a259c640 1 --2- 172.21.15.5:0/1935905973 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f808406be30 0x7f808406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.763+0000 7f80a259c640 1 --2- 172.21.15.5:0/1935905973 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f808406be30 0x7f808406e340 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f8094034000 tx=0x7f809402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:57.767+0000 7f809b7fe640 1 -- 172.21.15.5:0/1935905973 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f808c08e030 con 0x7f80a40d1230 2024-05-20T18:37:58.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:58.024+0000 7f80a8b2f640 1 -- 172.21.15.5:0/1935905973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7f80a415b6d0 con 0x7f80a40d1230 2024-05-20T18:37:58.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:58.274+0000 7f809b7fe640 1 -- 172.21.15.5:0/1935905973 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v520) ==== 144+0+0 (secure 0 0 0) 0x7f808c059770 con 0x7f80a40d1230 2024-05-20T18:37:58.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:58.277+0000 7f80a8b2f640 1 -- 172.21.15.5:0/1935905973 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f808406be30 msgr2=0x7f808406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:58.277+0000 7f80a8b2f640 1 --2- 172.21.15.5:0/1935905973 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f808406be30 0x7f808406e340 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f8094034000 tx=0x7f809402e000 comp rx=0 tx=0).stop 2024-05-20T18:37:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:58.277+0000 7f80a8b2f640 1 -- 172.21.15.5:0/1935905973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80a40d1230 msgr2=0x7f80a414df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:58.277+0000 7f80a8b2f640 1 --2- 172.21.15.5:0/1935905973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80a40d1230 0x7f80a414df30 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f808c00c610 tx=0x7f808c00cad0 comp rx=0 tx=0).stop 2024-05-20T18:37:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:58.277+0000 7f80a8b2f640 1 -- 172.21.15.5:0/1935905973 shutdown_connections 2024-05-20T18:37:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:58.277+0000 7f80a8b2f640 1 --2- 172.21.15.5:0/1935905973 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f808406be30 0x7f808406e340 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:58.277+0000 7f80a8b2f640 1 --2- 172.21.15.5:0/1935905973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80a415cf50 0x7f80a40cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:58.277+0000 7f80a8b2f640 1 --2- 172.21.15.5:0/1935905973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80a40d1230 0x7f80a414df30 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:58.277+0000 7f80a8b2f640 1 --2- 172.21.15.5:0/1935905973 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a40b1eb0 0x7f80a415b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:58.277+0000 7f80a8b2f640 1 -- 172.21.15.5:0/1935905973 >> 172.21.15.5:0/1935905973 conn(0x7f80a406f2b0 msgr2=0x7f80a40712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:58.277+0000 7f80a8b2f640 1 -- 172.21.15.5:0/1935905973 shutdown_connections 2024-05-20T18:37:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:58.277+0000 7f80a8b2f640 1 -- 172.21.15.5:0/1935905973 wait complete. 2024-05-20T18:37:58.290 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-05-20T18:37:58.291 INFO:tasks.cephfs_test_runner: 2024-05-20T18:37:58.291 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-20T18:37:58.291 INFO:tasks.cephfs_test_runner:Ran 7 tests in 601.813s 2024-05-20T18:37:58.291 INFO:tasks.cephfs_test_runner: 2024-05-20T18:37:58.291 INFO:tasks.cephfs_test_runner:OK 2024-05-20T18:37:58.291 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-05-20T18:37:58.301 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-20T18:37:58.331 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-20T18:37:58.331 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:37:58.331 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:37:58.348 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15663 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T18:37:58.348 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-20T18:37:58.348 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:37:58.405 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057102,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15663 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-20T18:37:58.405 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:37:58.405 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:37:58.426 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19732 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T18:37:58.426 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-20T18:37:58.426 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-20T18:37:58.486 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19732 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-20T18:37:58.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:58.486 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-20T18:37:58.503 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-20T18:37:58.503 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-20T18:37:58.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:58.503 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-20T18:37:58.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:58.571 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-20T18:37:58.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:58.639 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-05-20T18:37:58.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:58.740 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-05-20T18:37:58.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:58.756 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-05-20T18:37:58.825 INFO:teuthology.orchestra.run.smithi005.stderr:Cannot find device "ceph-brx" 2024-05-20T18:37:58.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T18:37:58.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:58.826 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-05-20T18:37:58.846 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-20T18:37:58.847 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-20T18:37:58.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:58.847 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-20T18:37:58.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:58.919 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-20T18:37:58.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:58.989 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link delete ceph-brx 2024-05-20T18:37:59.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:59.084 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-05-20T18:37:59.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-20T18:37:59.101 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link delete ceph-brx 2024-05-20T18:37:59.171 INFO:teuthology.orchestra.run.smithi063.stderr:Cannot find device "ceph-brx" 2024-05-20T18:37:59.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T18:37:59.173 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-20T18:37:59.182 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-20T18:37:59.182 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T18:37:59.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.304+0000 7f51d2565640 1 Processor -- start 2024-05-20T18:37:59.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.304+0000 7f51d2565640 1 -- start start 2024-05-20T18:37:59.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.306+0000 7f51d2565640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f51cc0b47b0 0x7f51cc0afad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:59.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.306+0000 7f51d2565640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51cc0b58f0 0x7f51cc06f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:59.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.306+0000 7f51d2565640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51cc0b39f0 con 0x7f51cc15d060 2024-05-20T18:37:59.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.306+0000 7f51d2565640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f51cc0b3cd0 con 0x7f51cc0b47b0 2024-05-20T18:37:59.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.306+0000 7f51d2565640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f51cc0b3fb0 con 0x7f51cc0b58f0 2024-05-20T18:37:59.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.306+0000 7f51cbfff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f51cc0b47b0 0x7f51cc0afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:59.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.306+0000 7f51d0b02640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51cc0b58f0 0x7f51cc06f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:59.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.306+0000 7f51cb7fe640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f51cc15d060 0x7f51cc0b51e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:34650/0 (socket says 172.21.15.5:34650) 2024-05-20T18:37:59.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.306+0000 7f51cb7fe640 1 -- 172.21.15.5:0/3911508853 learned_addr learned my addr 172.21.15.5:0/3911508853 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:59.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.307+0000 7f51caffd640 1 -- 172.21.15.5:0/3911508853 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2209330360 0 0) 0x7f51cc0b39f0 con 0x7f51cc15d060 2024-05-20T18:37:59.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.307+0000 7f51caffd640 1 -- 172.21.15.5:0/3911508853 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51b4003520 con 0x7f51cc15d060 2024-05-20T18:37:59.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.307+0000 7f51cbfff640 1 -- 172.21.15.5:0/3911508853 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51cc0b58f0 msgr2=0x7f51cc06f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:59.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.307+0000 7f51cbfff640 1 --2- 172.21.15.5:0/3911508853 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51cc0b58f0 0x7f51cc06f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:59.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.307+0000 7f51cbfff640 1 -- 172.21.15.5:0/3911508853 >> v1:172.21.15.63:6789/0 conn(0x7f51cc15d060 legacy=0x7f51cc0b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:59.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.307+0000 7f51cbfff640 1 -- 172.21.15.5:0/3911508853 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51cc0b4290 con 0x7f51cc0b47b0 2024-05-20T18:37:59.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.307+0000 7f51d0b02640 1 --2- 172.21.15.5:0/3911508853 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51cc0b58f0 0x7f51cc06f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:37:59.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.308+0000 7f51cbfff640 1 --2- 172.21.15.5:0/3911508853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51cc0b47b0 0x7f51cc0afad0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0097a0 tx=0x7f51bc02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89ebfcc3a091f76 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:59.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.308+0000 7f51caffd640 1 -- 172.21.15.5:0/3911508853 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc037070 con 0x7f51cc0b47b0 2024-05-20T18:37:59.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.308+0000 7f51caffd640 1 -- 172.21.15.5:0/3911508853 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51bc02ae50 con 0x7f51cc0b47b0 2024-05-20T18:37:59.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.308+0000 7f51caffd640 1 -- 172.21.15.5:0/3911508853 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc032660 con 0x7f51cc0b47b0 2024-05-20T18:37:59.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.308+0000 7f51d2565640 1 -- 172.21.15.5:0/3911508853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51cc0b47b0 msgr2=0x7f51cc0afad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:59.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.308+0000 7f51d2565640 1 --2- 172.21.15.5:0/3911508853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51cc0b47b0 0x7f51cc0afad0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0097a0 tx=0x7f51bc02a100 comp rx=0 tx=0).stop 2024-05-20T18:37:59.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.308+0000 7f51d2565640 1 -- 172.21.15.5:0/3911508853 shutdown_connections 2024-05-20T18:37:59.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.308+0000 7f51d2565640 1 --2- 172.21.15.5:0/3911508853 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51cc0b58f0 0x7f51cc06f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:59.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.308+0000 7f51d2565640 1 --2- 172.21.15.5:0/3911508853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51cc0b47b0 0x7f51cc0afad0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:59.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.308+0000 7f51d2565640 1 -- 172.21.15.5:0/3911508853 >> 172.21.15.5:0/3911508853 conn(0x7f51cc0cd2a0 msgr2=0x7f51cc0cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:59.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.309+0000 7f51d2565640 1 -- 172.21.15.5:0/3911508853 shutdown_connections 2024-05-20T18:37:59.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.309+0000 7f51d2565640 1 -- 172.21.15.5:0/3911508853 wait complete. 2024-05-20T18:37:59.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.310+0000 7f51d2565640 1 Processor -- start 2024-05-20T18:37:59.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.311+0000 7f51d2565640 1 -- start start 2024-05-20T18:37:59.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.312+0000 7f51d2565640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51cc15d060 0x7f51cc09bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:59.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.312+0000 7f51d2565640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51cc1f2fa0 0x7f51cc1410c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:59.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.312+0000 7f51d2565640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51cc15e8d0 0x7f51cc09daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:59.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.312+0000 7f51d2565640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f51cc159de0 con 0x7f51cc15e8d0 2024-05-20T18:37:59.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.312+0000 7f51d2565640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f51cc159c60 con 0x7f51cc15d060 2024-05-20T18:37:59.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.312+0000 7f51d2565640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f51cc06f910 con 0x7f51cc1f2fa0 2024-05-20T18:37:59.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.312+0000 7f51d0b02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51cc15e8d0 0x7f51cc09daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:59.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.312+0000 7f51d0b02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51cc15e8d0 0x7f51cc09daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60632/0 (socket says 172.21.15.5:60632) 2024-05-20T18:37:59.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.312+0000 7f51d0b02640 1 -- 172.21.15.5:0/2862264706 learned_addr learned my addr 172.21.15.5:0/2862264706 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:59.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.312+0000 7f51cbfff640 1 --2- 172.21.15.5:0/2862264706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51cc15d060 0x7f51cc09bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:59.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.312+0000 7f51cb7fe640 1 --2- 172.21.15.5:0/2862264706 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51cc1f2fa0 0x7f51cc1410c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:59.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.312+0000 7f51d0b02640 1 -- 172.21.15.5:0/2862264706 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51cc1f2fa0 msgr2=0x7f51cc1410c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:59.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.312+0000 7f51d0b02640 1 --2- 172.21.15.5:0/2862264706 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51cc1f2fa0 0x7f51cc1410c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:59.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.312+0000 7f51d0b02640 1 -- 172.21.15.5:0/2862264706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51cc15d060 msgr2=0x7f51cc09bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:59.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.312+0000 7f51d0b02640 1 --2- 172.21.15.5:0/2862264706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51cc15d060 0x7f51cc09bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:59.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.312+0000 7f51d0b02640 1 -- 172.21.15.5:0/2862264706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51cc15d830 con 0x7f51cc15e8d0 2024-05-20T18:37:59.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.312+0000 7f51cb7fe640 1 --2- 172.21.15.5:0/2862264706 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51cc1f2fa0 0x7f51cc1410c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:37:59.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.313+0000 7f51d0b02640 1 --2- 172.21.15.5:0/2862264706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51cc15e8d0 0x7f51cc09daf0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f51c400efb0 tx=0x7f51c400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:59.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.313+0000 7f51c8ff9640 1 -- 172.21.15.5:0/2862264706 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c4018020 con 0x7f51cc15e8d0 2024-05-20T18:37:59.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.313+0000 7f51c8ff9640 1 -- 172.21.15.5:0/2862264706 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51c4014480 con 0x7f51cc15e8d0 2024-05-20T18:37:59.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.313+0000 7f51c8ff9640 1 -- 172.21.15.5:0/2862264706 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c4014850 con 0x7f51cc15e8d0 2024-05-20T18:37:59.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.313+0000 7f51d2565640 1 -- 172.21.15.5:0/2862264706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51cc15db10 con 0x7f51cc15e8d0 2024-05-20T18:37:59.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.313+0000 7f51d2565640 1 -- 172.21.15.5:0/2862264706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51cc15e0d0 con 0x7f51cc15e8d0 2024-05-20T18:37:59.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.314+0000 7f51d2565640 1 -- 172.21.15.5:0/2862264706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51cc075b00 con 0x7f51cc15e8d0 2024-05-20T18:37:59.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.315+0000 7f51c8ff9640 1 -- 172.21.15.5:0/2862264706 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f51c4005d10 con 0x7f51cc15e8d0 2024-05-20T18:37:59.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.315+0000 7f51c8ff9640 1 --2- 172.21.15.5:0/2862264706 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f51b806be60 0x7f51b806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:59.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.315+0000 7f51c8ff9640 1 -- 172.21.15.5:0/2862264706 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f51c408bc80 con 0x7f51cc15e8d0 2024-05-20T18:37:59.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.315+0000 7f51cbfff640 1 --2- 172.21.15.5:0/2862264706 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f51b806be60 0x7f51b806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:59.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.316+0000 7f51cbfff640 1 --2- 172.21.15.5:0/2862264706 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f51b806be60 0x7f51b806e370 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f51bc034000 tx=0x7f51bc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:59.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.320+0000 7f51c8ff9640 1 -- 172.21.15.5:0/2862264706 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f51c4091020 con 0x7f51cc15e8d0 2024-05-20T18:37:59.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.503+0000 7f51d2565640 1 -- 172.21.15.5:0/2862264706 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f51cc15e560 con 0x7f51b806be60 2024-05-20T18:37:59.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.504+0000 7f51c8ff9640 1 -- 172.21.15.5:0/2862264706 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13460 (secure 0 0 0) 0x7f51cc15e560 con 0x7f51b806be60 2024-05-20T18:37:59.505 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:59.506 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-20T18:37:59.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.506+0000 7f51d2565640 1 -- 172.21.15.5:0/2862264706 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f51b806be60 msgr2=0x7f51b806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:59.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.506+0000 7f51d2565640 1 --2- 172.21.15.5:0/2862264706 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f51b806be60 0x7f51b806e370 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f51bc034000 tx=0x7f51bc02e000 comp rx=0 tx=0).stop 2024-05-20T18:37:59.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.506+0000 7f51d2565640 1 -- 172.21.15.5:0/2862264706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51cc15e8d0 msgr2=0x7f51cc09daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.506+0000 7f51d2565640 1 --2- 172.21.15.5:0/2862264706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51cc15e8d0 0x7f51cc09daf0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f51c400efb0 tx=0x7f51c400c490 comp rx=0 tx=0).stop 2024-05-20T18:37:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.506+0000 7f51d2565640 1 -- 172.21.15.5:0/2862264706 shutdown_connections 2024-05-20T18:37:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.506+0000 7f51d2565640 1 --2- 172.21.15.5:0/2862264706 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f51b806be60 0x7f51b806e370 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.506+0000 7f51d2565640 1 --2- 172.21.15.5:0/2862264706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51cc15e8d0 0x7f51cc09daf0 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.506+0000 7f51d2565640 1 --2- 172.21.15.5:0/2862264706 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51cc1f2fa0 0x7f51cc1410c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.506+0000 7f51d2565640 1 --2- 172.21.15.5:0/2862264706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51cc15d060 0x7f51cc09bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.506+0000 7f51d2565640 1 -- 172.21.15.5:0/2862264706 >> 172.21.15.5:0/2862264706 conn(0x7f51cc0cd2a0 msgr2=0x7f51cc06fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.506+0000 7f51d2565640 1 -- 172.21.15.5:0/2862264706 shutdown_connections 2024-05-20T18:37:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.506+0000 7f51d2565640 1 -- 172.21.15.5:0/2862264706 wait complete. 2024-05-20T18:37:59.519 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2024-05-20T18:37:58.741974+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283104,"kb_used_data":29504,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754691616,"statfs":{"total":773094113280,"available":772804214784,"internally_reserved":0,"allocated":30212096,"data_stored":23658032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392252,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-268,"num_write":-175,"num_write_kb":-8305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001657"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":358,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-05-20T18:37:54.899581+0000","last_change":"2024-05-20T18:27:03.765694+0000","last_active":"2024-05-20T18:37:54.899581+0000","last_peered":"2024-05-20T18:37:54.899581+0000","last_clean":"2024-05-20T18:37:54.899581+0000","last_became_active":"2024-05-20T18:27:03.764695+0000","last_became_peered":"2024-05-20T18:27:03.764695+0000","last_unstale":"2024-05-20T18:37:54.899581+0000","last_undegraded":"2024-05-20T18:37:54.899581+0000","last_fullsized":"2024-05-20T18:37:54.899581+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_clean_scrub_stamp":"2024-05-20T18:27:02.750853+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:30:04.407487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574983,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33264,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338576,"statfs":{"total":96636764160,"available":96602701824,"internally_reserved":0,"allocated":2134016,"data_stored":1302750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574981,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36764,"kb_used_data":5584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335076,"statfs":{"total":96636764160,"available":96599117824,"internally_reserved":0,"allocated":5718016,"data_stored":4906485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574981,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36768,"kb_used_data":5588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335072,"statfs":{"total":96636764160,"available":96599113728,"internally_reserved":0,"allocated":5722112,"data_stored":4906063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574981,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36780,"kb_used_data":5600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335060,"statfs":{"total":96636764160,"available":96599101440,"internally_reserved":0,"allocated":5734400,"data_stored":4908744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574983,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32680,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339160,"statfs":{"total":96636764160,"available":96603299840,"internally_reserved":0,"allocated":1536000,"data_stored":714462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574982,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1523712,"data_stored":711604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574983,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36768,"kb_used_data":5588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335072,"statfs":{"total":96636764160,"available":96599113728,"internally_reserved":0,"allocated":5722112,"data_stored":4905930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574982,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37412,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334428,"statfs":{"total":96636764160,"available":96598454272,"internally_reserved":0,"allocated":2121728,"data_stored":1301994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T18:37:59.520 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T18:37:59.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.633+0000 7f4f69c44640 1 Processor -- start 2024-05-20T18:37:59.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.633+0000 7f4f69c44640 1 -- start start 2024-05-20T18:37:59.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.634+0000 7f4f69c44640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f4f64162820 0x7f4f640b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:59.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.634+0000 7f4f69c44640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4f640b3e80 0x7f4f6406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:59.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.634+0000 7f4f69c44640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f6406dd20 con 0x7f4f6406d910 2024-05-20T18:37:59.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.634+0000 7f4f69c44640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f4f640d0630 con 0x7f4f640b3e80 2024-05-20T18:37:59.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.634+0000 7f4f69c44640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f4f640d39c0 con 0x7f4f64162820 2024-05-20T18:37:59.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.634+0000 7f4f62ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4f640b3e80 0x7f4f6406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:59.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.634+0000 7f4f62ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4f640b3e80 0x7f4f6406d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60648/0 (socket says 172.21.15.5:60648) 2024-05-20T18:37:59.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.634+0000 7f4f62ffd640 1 -- 172.21.15.5:0/1641608855 learned_addr learned my addr 172.21.15.5:0/1641608855 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:59.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.635+0000 7f4f637fe640 1 --2- 172.21.15.5:0/1641608855 >> v2:172.21.15.63:3300/0 conn(0x7f4f64162820 0x7f4f640b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:59.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.635+0000 7f4f62ffd640 1 -- 172.21.15.5:0/1641608855 >> v2:172.21.15.63:3300/0 conn(0x7f4f64162820 msgr2=0x7f4f640b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:59.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.635+0000 7f4f62ffd640 1 --2- 172.21.15.5:0/1641608855 >> v2:172.21.15.63:3300/0 conn(0x7f4f64162820 0x7f4f640b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:59.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.635+0000 7f4f62ffd640 1 -- 172.21.15.5:0/1641608855 >> v1:172.21.15.63:6789/0 conn(0x7f4f6406d910 legacy=0x7f4f640d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:59.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.635+0000 7f4f62ffd640 1 -- 172.21.15.5:0/1641608855 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f640d3ca0 con 0x7f4f640b3e80 2024-05-20T18:37:59.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.635+0000 7f4f637fe640 1 --2- 172.21.15.5:0/1641608855 >> v2:172.21.15.63:3300/0 conn(0x7f4f64162820 0x7f4f640b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:59.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.636+0000 7f4f62ffd640 1 --2- 172.21.15.5:0/1641608855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f640b3e80 0x7f4f6406d3d0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f4f640b7740 tx=0x7f4f5802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af1c798c308c395 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:59.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.636+0000 7f4f627fc640 1 -- 172.21.15.5:0/1641608855 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f58037070 con 0x7f4f640b3e80 2024-05-20T18:37:59.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.636+0000 7f4f627fc640 1 -- 172.21.15.5:0/1641608855 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f58032440 con 0x7f4f640b3e80 2024-05-20T18:37:59.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.636+0000 7f4f627fc640 1 -- 172.21.15.5:0/1641608855 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f58032810 con 0x7f4f640b3e80 2024-05-20T18:37:59.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.636+0000 7f4f69c44640 1 -- 172.21.15.5:0/1641608855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f640b3e80 msgr2=0x7f4f6406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:59.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.636+0000 7f4f69c44640 1 --2- 172.21.15.5:0/1641608855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f640b3e80 0x7f4f6406d3d0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f4f640b7740 tx=0x7f4f5802a270 comp rx=0 tx=0).stop 2024-05-20T18:37:59.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.637+0000 7f4f69c44640 1 -- 172.21.15.5:0/1641608855 shutdown_connections 2024-05-20T18:37:59.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.637+0000 7f4f69c44640 1 --2- 172.21.15.5:0/1641608855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f640b3e80 0x7f4f6406d3d0 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:59.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.637+0000 7f4f69c44640 1 --2- 172.21.15.5:0/1641608855 >> v2:172.21.15.63:3300/0 conn(0x7f4f64162820 0x7f4f640b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:59.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.637+0000 7f4f69c44640 1 -- 172.21.15.5:0/1641608855 >> 172.21.15.5:0/1641608855 conn(0x7f4f640d4bd0 msgr2=0x7f4f640d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:59.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.637+0000 7f4f69c44640 1 -- 172.21.15.5:0/1641608855 shutdown_connections 2024-05-20T18:37:59.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.637+0000 7f4f69c44640 1 -- 172.21.15.5:0/1641608855 wait complete. 2024-05-20T18:37:59.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.638+0000 7f4f69c44640 1 Processor -- start 2024-05-20T18:37:59.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.639+0000 7f4f69c44640 1 -- start start 2024-05-20T18:37:59.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.640+0000 7f4f69c44640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f6406d910 0x7f4f6414e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:59.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.640+0000 7f4f69c44640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f640b3e80 0x7f4f641f89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:59.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.640+0000 7f4f69c44640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f64162820 0x7f4f641f8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:59.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.640+0000 7f4f69c44640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f640d0300 con 0x7f4f6406d910 2024-05-20T18:37:59.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.640+0000 7f4f69c44640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4f640d4260 con 0x7f4f64162820 2024-05-20T18:37:59.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.640+0000 7f4f69c44640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4f640b91a0 con 0x7f4f640b3e80 2024-05-20T18:37:59.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.640+0000 7f4f637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f6406d910 0x7f4f6414e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:59.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.640+0000 7f4f637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f6406d910 0x7f4f6414e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60658/0 (socket says 172.21.15.5:60658) 2024-05-20T18:37:59.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.640+0000 7f4f637fe640 1 -- 172.21.15.5:0/1097916764 learned_addr learned my addr 172.21.15.5:0/1097916764 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:59.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.640+0000 7f4f62ffd640 1 --2- 172.21.15.5:0/1097916764 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f640b3e80 0x7f4f641f89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:59.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.640+0000 7f4f63fff640 1 --2- 172.21.15.5:0/1097916764 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f64162820 0x7f4f641f8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:59.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.640+0000 7f4f637fe640 1 -- 172.21.15.5:0/1097916764 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f640b3e80 msgr2=0x7f4f641f89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:59.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.640+0000 7f4f637fe640 1 --2- 172.21.15.5:0/1097916764 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f640b3e80 0x7f4f641f89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:59.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.640+0000 7f4f637fe640 1 -- 172.21.15.5:0/1097916764 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f64162820 msgr2=0x7f4f641f8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:59.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.640+0000 7f4f637fe640 1 --2- 172.21.15.5:0/1097916764 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f64162820 0x7f4f641f8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:59.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.640+0000 7f4f637fe640 1 -- 172.21.15.5:0/1097916764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f6414e640 con 0x7f4f6406d910 2024-05-20T18:37:59.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.641+0000 7f4f637fe640 1 --2- 172.21.15.5:0/1097916764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f6406d910 0x7f4f6414e100 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f4f5400d800 tx=0x7f4f5400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:59.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.641+0000 7f4f4bfff640 1 -- 172.21.15.5:0/1097916764 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f54004030 con 0x7f4f6406d910 2024-05-20T18:37:59.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.641+0000 7f4f4bfff640 1 -- 172.21.15.5:0/1097916764 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f540041d0 con 0x7f4f6406d910 2024-05-20T18:37:59.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.641+0000 7f4f4bfff640 1 -- 172.21.15.5:0/1097916764 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f540096a0 con 0x7f4f6406d910 2024-05-20T18:37:59.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.641+0000 7f4f69c44640 1 -- 172.21.15.5:0/1097916764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f640cbed0 con 0x7f4f6406d910 2024-05-20T18:37:59.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.641+0000 7f4f69c44640 1 -- 172.21.15.5:0/1097916764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f640cc490 con 0x7f4f6406d910 2024-05-20T18:37:59.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.642+0000 7f4f69c44640 1 -- 172.21.15.5:0/1097916764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f64072fc0 con 0x7f4f6406d910 2024-05-20T18:37:59.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.642+0000 7f4f4bfff640 1 -- 172.21.15.5:0/1097916764 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4f54009840 con 0x7f4f6406d910 2024-05-20T18:37:59.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.643+0000 7f4f4bfff640 1 --2- 172.21.15.5:0/1097916764 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4f5006be30 0x7f4f5006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:59.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.643+0000 7f4f62ffd640 1 --2- 172.21.15.5:0/1097916764 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4f5006be30 0x7f4f5006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:59.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.643+0000 7f4f4bfff640 1 -- 172.21.15.5:0/1097916764 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f4f5408b090 con 0x7f4f6406d910 2024-05-20T18:37:59.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.644+0000 7f4f62ffd640 1 --2- 172.21.15.5:0/1097916764 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4f5006be30 0x7f4f5006e340 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f4f58034000 tx=0x7f4f5802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:59.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.648+0000 7f4f4bfff640 1 -- 172.21.15.5:0/1097916764 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4f64072fc0 con 0x7f4f6406d910 2024-05-20T18:37:59.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.827+0000 7f4f69c44640 1 -- 172.21.15.5:0/1097916764 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4f64075b00 con 0x7f4f5006be30 2024-05-20T18:37:59.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.828+0000 7f4f4bfff640 1 -- 172.21.15.5:0/1097916764 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13460 (secure 0 0 0) 0x7f4f64075b00 con 0x7f4f5006be30 2024-05-20T18:37:59.829 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:37:59.830 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-20T18:37:59.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.830+0000 7f4f69c44640 1 -- 172.21.15.5:0/1097916764 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4f5006be30 msgr2=0x7f4f5006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:59.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.830+0000 7f4f69c44640 1 --2- 172.21.15.5:0/1097916764 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4f5006be30 0x7f4f5006e340 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f4f58034000 tx=0x7f4f5802e000 comp rx=0 tx=0).stop 2024-05-20T18:37:59.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.830+0000 7f4f69c44640 1 -- 172.21.15.5:0/1097916764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f6406d910 msgr2=0x7f4f6414e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:59.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.830+0000 7f4f69c44640 1 --2- 172.21.15.5:0/1097916764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f6406d910 0x7f4f6414e100 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f4f5400d800 tx=0x7f4f5400dcc0 comp rx=0 tx=0).stop 2024-05-20T18:37:59.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.830+0000 7f4f69c44640 1 -- 172.21.15.5:0/1097916764 shutdown_connections 2024-05-20T18:37:59.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.830+0000 7f4f69c44640 1 --2- 172.21.15.5:0/1097916764 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f4f5006be30 0x7f4f5006e340 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:59.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.830+0000 7f4f69c44640 1 --2- 172.21.15.5:0/1097916764 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f64162820 0x7f4f641f8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:59.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.830+0000 7f4f69c44640 1 --2- 172.21.15.5:0/1097916764 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4f640b3e80 0x7f4f641f89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:59.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.830+0000 7f4f69c44640 1 --2- 172.21.15.5:0/1097916764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f6406d910 0x7f4f6414e100 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:59.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.830+0000 7f4f69c44640 1 -- 172.21.15.5:0/1097916764 >> 172.21.15.5:0/1097916764 conn(0x7f4f640d4bd0 msgr2=0x7f4f640d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:59.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.830+0000 7f4f69c44640 1 -- 172.21.15.5:0/1097916764 shutdown_connections 2024-05-20T18:37:59.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.830+0000 7f4f69c44640 1 -- 172.21.15.5:0/1097916764 wait complete. 2024-05-20T18:37:59.842 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2024-05-20T18:37:58.741974+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283104,"kb_used_data":29504,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754691616,"statfs":{"total":773094113280,"available":772804214784,"internally_reserved":0,"allocated":30212096,"data_stored":23658032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392252,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-268,"num_write":-175,"num_write_kb":-8305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001657"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":358,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-05-20T18:37:54.899581+0000","last_change":"2024-05-20T18:27:03.765694+0000","last_active":"2024-05-20T18:37:54.899581+0000","last_peered":"2024-05-20T18:37:54.899581+0000","last_clean":"2024-05-20T18:37:54.899581+0000","last_became_active":"2024-05-20T18:27:03.764695+0000","last_became_peered":"2024-05-20T18:27:03.764695+0000","last_unstale":"2024-05-20T18:37:54.899581+0000","last_undegraded":"2024-05-20T18:37:54.899581+0000","last_fullsized":"2024-05-20T18:37:54.899581+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_clean_scrub_stamp":"2024-05-20T18:27:02.750853+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:30:04.407487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574983,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33264,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338576,"statfs":{"total":96636764160,"available":96602701824,"internally_reserved":0,"allocated":2134016,"data_stored":1302750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574981,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36764,"kb_used_data":5584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335076,"statfs":{"total":96636764160,"available":96599117824,"internally_reserved":0,"allocated":5718016,"data_stored":4906485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574981,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36768,"kb_used_data":5588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335072,"statfs":{"total":96636764160,"available":96599113728,"internally_reserved":0,"allocated":5722112,"data_stored":4906063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574981,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36780,"kb_used_data":5600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335060,"statfs":{"total":96636764160,"available":96599101440,"internally_reserved":0,"allocated":5734400,"data_stored":4908744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574983,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32680,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339160,"statfs":{"total":96636764160,"available":96603299840,"internally_reserved":0,"allocated":1536000,"data_stored":714462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574982,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1523712,"data_stored":711604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574983,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36768,"kb_used_data":5588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335072,"statfs":{"total":96636764160,"available":96599113728,"internally_reserved":0,"allocated":5722112,"data_stored":4905930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574982,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37412,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334428,"statfs":{"total":96636764160,"available":96598454272,"internally_reserved":0,"allocated":2121728,"data_stored":1301994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T18:37:59.843 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-20T18:37:59.843 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T18:37:59.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.957+0000 7f8770899640 1 Processor -- start 2024-05-20T18:37:59.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.957+0000 7f8770899640 1 -- start start 2024-05-20T18:37:59.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.958+0000 7f8770899640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f876c163a10 0x7f876c0d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:59.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.958+0000 7f8770899640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f876c0d14b0 0x7f876c0d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:59.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.958+0000 7f8770899640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f876c0d0370 con 0x7f876c0b4010 2024-05-20T18:37:59.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.958+0000 7f8770899640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f876c0d3ca0 con 0x7f876c0d14b0 2024-05-20T18:37:59.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.958+0000 7f8770899640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f876c0d0650 con 0x7f876c163a10 2024-05-20T18:37:59.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.958+0000 7f876ad9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f876c0d14b0 0x7f876c0d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:59.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.958+0000 7f8761d9b640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f876c0b4010 0x7f876c0cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43180/0 (socket says 172.21.15.5:43180) 2024-05-20T18:37:59.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.958+0000 7f8761d9b640 1 -- 172.21.15.5:0/2625582156 learned_addr learned my addr 172.21.15.5:0/2625582156 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:37:59.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.958+0000 7f876a59c640 1 --2- 172.21.15.5:0/2625582156 >> v2:172.21.15.63:3300/0 conn(0x7f876c163a10 0x7f876c0d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:59.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.959+0000 7f876ad9d640 1 -- 172.21.15.5:0/2625582156 >> v2:172.21.15.63:3300/0 conn(0x7f876c163a10 msgr2=0x7f876c0d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:37:59.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.959+0000 7f876ad9d640 1 --2- 172.21.15.5:0/2625582156 >> v2:172.21.15.63:3300/0 conn(0x7f876c163a10 0x7f876c0d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:59.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.959+0000 7f876ad9d640 1 -- 172.21.15.5:0/2625582156 >> v1:172.21.15.5:6789/0 conn(0x7f876c0b4010 legacy=0x7f876c0cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:59.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.959+0000 7f8769d9b640 1 -- 172.21.15.5:0/2625582156 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1609190462 0 0) 0x7f876c0d0370 con 0x7f876c0b4010 2024-05-20T18:37:59.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.959+0000 7f876ad9d640 1 -- 172.21.15.5:0/2625582156 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876c0d3650 con 0x7f876c0d14b0 2024-05-20T18:37:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.959+0000 7f876ad9d640 1 --2- 172.21.15.5:0/2625582156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f876c0d14b0 0x7f876c0d18c0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f876c0cf1c0 tx=0x7f876402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf5763926776cae server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.959+0000 7f8769d9b640 1 -- 172.21.15.5:0/2625582156 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8764007740 con 0x7f876c0d14b0 2024-05-20T18:37:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.960+0000 7f8769d9b640 1 -- 172.21.15.5:0/2625582156 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87640078e0 con 0x7f876c0d14b0 2024-05-20T18:37:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.960+0000 7f8769d9b640 1 -- 172.21.15.5:0/2625582156 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8764007cb0 con 0x7f876c0d14b0 2024-05-20T18:37:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.960+0000 7f8770899640 1 -- 172.21.15.5:0/2625582156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f876c0d14b0 msgr2=0x7f876c0d18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.960+0000 7f8770899640 1 --2- 172.21.15.5:0/2625582156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f876c0d14b0 0x7f876c0d18c0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f876c0cf1c0 tx=0x7f876402a680 comp rx=0 tx=0).stop 2024-05-20T18:37:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.960+0000 7f8770899640 1 -- 172.21.15.5:0/2625582156 shutdown_connections 2024-05-20T18:37:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.960+0000 7f8770899640 1 --2- 172.21.15.5:0/2625582156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f876c0d14b0 0x7f876c0d18c0 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.960+0000 7f8770899640 1 --2- 172.21.15.5:0/2625582156 >> v2:172.21.15.63:3300/0 conn(0x7f876c163a10 0x7f876c0d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:59.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.960+0000 7f8770899640 1 -- 172.21.15.5:0/2625582156 >> 172.21.15.5:0/2625582156 conn(0x7f876c06d240 msgr2=0x7f876c06d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:37:59.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.960+0000 7f8770899640 1 -- 172.21.15.5:0/2625582156 shutdown_connections 2024-05-20T18:37:59.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.960+0000 7f8770899640 1 -- 172.21.15.5:0/2625582156 wait complete. 2024-05-20T18:37:59.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.962+0000 7f8770899640 1 Processor -- start 2024-05-20T18:37:59.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.962+0000 7f8770899640 1 -- start start 2024-05-20T18:37:59.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.964+0000 7f8770899640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f876c0b4010 0x7f876c14ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:59.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.964+0000 7f8770899640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f876c0d14b0 0x7f876c0ab9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:59.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.964+0000 7f8770899640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f876c163a10 0x7f876c14b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:59.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.964+0000 7f8770899640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f876c0b8210 con 0x7f876c163a10 2024-05-20T18:37:59.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.964+0000 7f8770899640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f876c0d1e00 con 0x7f876c0d14b0 2024-05-20T18:37:59.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.964+0000 7f8770899640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f876c0b8390 con 0x7f876c0b4010 2024-05-20T18:37:59.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.964+0000 7f876ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f876c163a10 0x7f876c14b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:59.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.964+0000 7f876ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f876c163a10 0x7f876c14b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60668/0 (socket says 172.21.15.5:60668) 2024-05-20T18:37:59.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.964+0000 7f876ad9d640 1 -- 172.21.15.5:0/1909257738 learned_addr learned my addr 172.21.15.5:0/1909257738 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:37:59.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.964+0000 7f8761d9b640 1 --2- 172.21.15.5:0/1909257738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f876c0d14b0 0x7f876c0ab9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:59.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.964+0000 7f876a59c640 1 --2- 172.21.15.5:0/1909257738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f876c0b4010 0x7f876c14ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:59.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.964+0000 7f876ad9d640 1 -- 172.21.15.5:0/1909257738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f876c0b4010 msgr2=0x7f876c14ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:59.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.964+0000 7f876ad9d640 1 --2- 172.21.15.5:0/1909257738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f876c0b4010 0x7f876c14ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:59.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.964+0000 7f876ad9d640 1 -- 172.21.15.5:0/1909257738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f876c0d14b0 msgr2=0x7f876c0ab9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:37:59.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.964+0000 7f876ad9d640 1 --2- 172.21.15.5:0/1909257738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f876c0d14b0 0x7f876c0ab9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:37:59.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.964+0000 7f876ad9d640 1 -- 172.21.15.5:0/1909257738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876c0abf00 con 0x7f876c163a10 2024-05-20T18:37:59.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.964+0000 7f876a59c640 1 --2- 172.21.15.5:0/1909257738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f876c0b4010 0x7f876c14ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:37:59.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.964+0000 7f876ad9d640 1 --2- 172.21.15.5:0/1909257738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f876c163a10 0x7f876c14b290 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f876402ac70 tx=0x7f8764004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:59.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.965+0000 7f87637fe640 1 -- 172.21.15.5:0/1909257738 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876403f070 con 0x7f876c163a10 2024-05-20T18:37:59.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.965+0000 7f87637fe640 1 -- 172.21.15.5:0/1909257738 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8764031bc0 con 0x7f876c163a10 2024-05-20T18:37:59.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.965+0000 7f87637fe640 1 -- 172.21.15.5:0/1909257738 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8764044460 con 0x7f876c163a10 2024-05-20T18:37:59.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.965+0000 7f8770899640 1 -- 172.21.15.5:0/1909257738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f876c14b7d0 con 0x7f876c163a10 2024-05-20T18:37:59.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.965+0000 7f8770899640 1 -- 172.21.15.5:0/1909257738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f876c0b5af0 con 0x7f876c163a10 2024-05-20T18:37:59.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.966+0000 7f8770899640 1 -- 172.21.15.5:0/1909257738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f876c075b00 con 0x7f876c163a10 2024-05-20T18:37:59.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.966+0000 7f87637fe640 1 -- 172.21.15.5:0/1909257738 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8764007790 con 0x7f876c163a10 2024-05-20T18:37:59.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.967+0000 7f87637fe640 1 --2- 172.21.15.5:0/1909257738 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f874c06be30 0x7f874c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:37:59.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.967+0000 7f87637fe640 1 -- 172.21.15.5:0/1909257738 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f87640ac4e0 con 0x7f876c163a10 2024-05-20T18:37:59.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.967+0000 7f876a59c640 1 --2- 172.21.15.5:0/1909257738 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f874c06be30 0x7f874c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:37:59.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.967+0000 7f876a59c640 1 --2- 172.21.15.5:0/1909257738 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f874c06be30 0x7f874c06e340 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f875c006fd0 tx=0x7f875c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:37:59.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:37:59.971+0000 7f87637fe640 1 -- 172.21.15.5:0/1909257738 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f876407afe0 con 0x7f876c163a10 2024-05-20T18:38:00.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.151+0000 7f8770899640 1 -- 172.21.15.5:0/1909257738 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f876c0d3050 con 0x7f874c06be30 2024-05-20T18:38:00.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.151+0000 7f87637fe640 1 -- 172.21.15.5:0/1909257738 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13460 (secure 0 0 0) 0x7f876c0d3050 con 0x7f874c06be30 2024-05-20T18:38:00.153 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:38:00.153 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-20T18:38:00.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.153+0000 7f8770899640 1 -- 172.21.15.5:0/1909257738 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f874c06be30 msgr2=0x7f874c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:00.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.154+0000 7f8770899640 1 --2- 172.21.15.5:0/1909257738 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f874c06be30 0x7f874c06e340 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f875c006fd0 tx=0x7f875c008040 comp rx=0 tx=0).stop 2024-05-20T18:38:00.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.154+0000 7f8770899640 1 -- 172.21.15.5:0/1909257738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f876c163a10 msgr2=0x7f876c14b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:00.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.154+0000 7f8770899640 1 --2- 172.21.15.5:0/1909257738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f876c163a10 0x7f876c14b290 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f876402ac70 tx=0x7f8764004a40 comp rx=0 tx=0).stop 2024-05-20T18:38:00.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.154+0000 7f8770899640 1 -- 172.21.15.5:0/1909257738 shutdown_connections 2024-05-20T18:38:00.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.154+0000 7f8770899640 1 --2- 172.21.15.5:0/1909257738 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f874c06be30 0x7f874c06e340 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:00.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.154+0000 7f8770899640 1 --2- 172.21.15.5:0/1909257738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f876c163a10 0x7f876c14b290 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:00.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.154+0000 7f8770899640 1 --2- 172.21.15.5:0/1909257738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f876c0d14b0 0x7f876c0ab9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:00.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.154+0000 7f8770899640 1 --2- 172.21.15.5:0/1909257738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f876c0b4010 0x7f876c14ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:00.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.154+0000 7f8770899640 1 -- 172.21.15.5:0/1909257738 >> 172.21.15.5:0/1909257738 conn(0x7f876c06d240 msgr2=0x7f876c1619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:00.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.154+0000 7f8770899640 1 -- 172.21.15.5:0/1909257738 shutdown_connections 2024-05-20T18:38:00.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.154+0000 7f8770899640 1 -- 172.21.15.5:0/1909257738 wait complete. 2024-05-20T18:38:00.166 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2024-05-20T18:37:58.741974+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283104,"kb_used_data":29504,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754691616,"statfs":{"total":773094113280,"available":772804214784,"internally_reserved":0,"allocated":30212096,"data_stored":23658032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392252,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-268,"num_write":-175,"num_write_kb":-8305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001657"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":358,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-05-20T18:37:54.899581+0000","last_change":"2024-05-20T18:27:03.765694+0000","last_active":"2024-05-20T18:37:54.899581+0000","last_peered":"2024-05-20T18:37:54.899581+0000","last_clean":"2024-05-20T18:37:54.899581+0000","last_became_active":"2024-05-20T18:27:03.764695+0000","last_became_peered":"2024-05-20T18:27:03.764695+0000","last_unstale":"2024-05-20T18:37:54.899581+0000","last_undegraded":"2024-05-20T18:37:54.899581+0000","last_fullsized":"2024-05-20T18:37:54.899581+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_clean_scrub_stamp":"2024-05-20T18:27:02.750853+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:30:04.407487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574983,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33264,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338576,"statfs":{"total":96636764160,"available":96602701824,"internally_reserved":0,"allocated":2134016,"data_stored":1302750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574981,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36764,"kb_used_data":5584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335076,"statfs":{"total":96636764160,"available":96599117824,"internally_reserved":0,"allocated":5718016,"data_stored":4906485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574981,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36768,"kb_used_data":5588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335072,"statfs":{"total":96636764160,"available":96599113728,"internally_reserved":0,"allocated":5722112,"data_stored":4906063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574981,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36780,"kb_used_data":5600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335060,"statfs":{"total":96636764160,"available":96599101440,"internally_reserved":0,"allocated":5734400,"data_stored":4908744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574983,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32680,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339160,"statfs":{"total":96636764160,"available":96603299840,"internally_reserved":0,"allocated":1536000,"data_stored":714462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574982,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1523712,"data_stored":711604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574983,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36768,"kb_used_data":5588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335072,"statfs":{"total":96636764160,"available":96599113728,"internally_reserved":0,"allocated":5722112,"data_stored":4905930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574982,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37412,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334428,"statfs":{"total":96636764160,"available":96598454272,"internally_reserved":0,"allocated":2121728,"data_stored":1301994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T18:38:00.167 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-20T18:38:00.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.281+0000 7f34d0ee9640 1 Processor -- start 2024-05-20T18:38:00.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.281+0000 7f34d0ee9640 1 -- start start 2024-05-20T18:38:00.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.283+0000 7f34d0ee9640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f34cc164c40 0x7f34cc167060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:00.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.283+0000 7f34d0ee9640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34cc15d430 con 0x7f34cc0d1610 2024-05-20T18:38:00.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.283+0000 7f34d0ee9640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34cc06d890 con 0x7f34cc156690 2024-05-20T18:38:00.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.283+0000 7f34d0ee9640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f34cc0b29b0 con 0x7f34cc164c40 2024-05-20T18:38:00.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.283+0000 7f34c9d9b640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f34cc0d1610 0x7f34cc0d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43188/0 (socket says 172.21.15.5:43188) 2024-05-20T18:38:00.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.283+0000 7f34c9d9b640 1 -- 172.21.15.5:0/786359829 learned_addr learned my addr 172.21.15.5:0/786359829 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:38:00.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.283+0000 7f34cad9d640 1 --2- 172.21.15.5:0/786359829 >> v2:172.21.15.63:3300/0 conn(0x7f34cc164c40 0x7f34cc167060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:00.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.283+0000 7f34c959a640 1 -- 172.21.15.5:0/786359829 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 712333656 0 0) 0x7f34cc15d430 con 0x7f34cc0d1610 2024-05-20T18:38:00.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.284+0000 7f34c959a640 1 -- 172.21.15.5:0/786359829 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34b4003520 con 0x7f34cc0d1610 2024-05-20T18:38:00.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.284+0000 7f34c959a640 1 -- 172.21.15.5:0/786359829 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4184484878 0 0) 0x7f34b4003520 con 0x7f34cc0d1610 2024-05-20T18:38:00.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.284+0000 7f34c959a640 1 -- 172.21.15.5:0/786359829 >> v2:172.21.15.63:3300/0 conn(0x7f34cc164c40 msgr2=0x7f34cc167060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:38:00.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.284+0000 7f34c959a640 1 --2- 172.21.15.5:0/786359829 >> v2:172.21.15.63:3300/0 conn(0x7f34cc164c40 0x7f34cc167060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:00.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.284+0000 7f34c959a640 1 -- 172.21.15.5:0/786359829 >> v1:172.21.15.63:6789/0 conn(0x7f34cc156690 legacy=0x7f34cc15cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:00.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.284+0000 7f34c959a640 1 -- 172.21.15.5:0/786359829 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34cc0b1910 con 0x7f34cc0d1610 2024-05-20T18:38:00.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.284+0000 7f34c959a640 1 -- 172.21.15.5:0/786359829 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f34c0001810 con 0x7f34cc0d1610 2024-05-20T18:38:00.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.284+0000 7f34c959a640 1 -- 172.21.15.5:0/786359829 >> v1:172.21.15.5:6789/0 conn(0x7f34cc0d1610 legacy=0x7f34cc0d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:00.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.284+0000 7f34c959a640 1 --2- 172.21.15.5:0/786359829 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34b40039b0 0x7f34b4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:00.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.284+0000 7f34c959a640 1 --2- 172.21.15.5:0/786359829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34b4004800 0x7f34b4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:00.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.284+0000 7f34c959a640 1 --2- 172.21.15.5:0/786359829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34b4005350 0x7f34b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:00.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.284+0000 7f34c959a640 1 -- 172.21.15.5:0/786359829 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc0b29b0 con 0x7f34b4004800 2024-05-20T18:38:00.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.284+0000 7f34c959a640 1 -- 172.21.15.5:0/786359829 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f34b400e490 con 0x7f34b4005350 2024-05-20T18:38:00.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.284+0000 7f34c959a640 1 -- 172.21.15.5:0/786359829 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f34b400e690 con 0x7f34b40039b0 2024-05-20T18:38:00.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.284+0000 7f34c9d9b640 1 --2- 172.21.15.5:0/786359829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34b4004800 0x7f34b4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:00.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.284+0000 7f34ca59c640 1 --2- 172.21.15.5:0/786359829 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34b40039b0 0x7f34b4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:00.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.284+0000 7f34cad9d640 1 --2- 172.21.15.5:0/786359829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34b4005350 0x7f34b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:00.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.284+0000 7f34c9d9b640 1 -- 172.21.15.5:0/786359829 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34b40039b0 msgr2=0x7f34b4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:38:00.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.284+0000 7f34c9d9b640 1 --2- 172.21.15.5:0/786359829 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34b40039b0 0x7f34b4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:00.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.284+0000 7f34c9d9b640 1 -- 172.21.15.5:0/786359829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34b4005350 msgr2=0x7f34b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:38:00.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.284+0000 7f34c9d9b640 1 --2- 172.21.15.5:0/786359829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34b4005350 0x7f34b400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:00.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.284+0000 7f34c9d9b640 1 -- 172.21.15.5:0/786359829 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b4003520 con 0x7f34b4004800 2024-05-20T18:38:00.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.284+0000 7f34cad9d640 1 --2- 172.21.15.5:0/786359829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34b4005350 0x7f34b400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:38:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.284+0000 7f34cad9d640 1 -- 172.21.15.5:0/786359829 reap_dead start 2024-05-20T18:38:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.285+0000 7f34c9d9b640 1 --2- 172.21.15.5:0/786359829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34b4004800 0x7f34b4004c10 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f34cc15d600 tx=0x7f34b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb613712494be4c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.285+0000 7f34c959a640 1 -- 172.21.15.5:0/786359829 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c0007d20 con 0x7f34b4004800 2024-05-20T18:38:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.285+0000 7f34c959a640 1 -- 172.21.15.5:0/786359829 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34c002e070 con 0x7f34b4004800 2024-05-20T18:38:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.285+0000 7f34c959a640 1 -- 172.21.15.5:0/786359829 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c00074b0 con 0x7f34b4004800 2024-05-20T18:38:00.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.286+0000 7f34d0ee9640 1 -- 172.21.15.5:0/786359829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34b4004800 msgr2=0x7f34b4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:00.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.286+0000 7f34d0ee9640 1 --2- 172.21.15.5:0/786359829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34b4004800 0x7f34b4004c10 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f34cc15d600 tx=0x7f34b4005830 comp rx=0 tx=0).stop 2024-05-20T18:38:00.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.286+0000 7f34d0ee9640 1 -- 172.21.15.5:0/786359829 shutdown_connections 2024-05-20T18:38:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.286+0000 7f34d0ee9640 1 --2- 172.21.15.5:0/786359829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34b4004800 0x7f34b4004c10 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.286+0000 7f34d0ee9640 1 -- 172.21.15.5:0/786359829 >> 172.21.15.5:0/786359829 conn(0x7f34cc0b6670 msgr2=0x7f34cc0b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.286+0000 7f34d0ee9640 1 -- 172.21.15.5:0/786359829 shutdown_connections 2024-05-20T18:38:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.286+0000 7f34d0ee9640 1 -- 172.21.15.5:0/786359829 wait complete. 2024-05-20T18:38:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.288+0000 7f34d0ee9640 1 Processor -- start 2024-05-20T18:38:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.288+0000 7f34d0ee9640 1 -- start start 2024-05-20T18:38:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.290+0000 7f34d0ee9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34b4004800 0x7f34cc1392a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.290+0000 7f34d0ee9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34cc15e830 0x7f34cc154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.290+0000 7f34d0ee9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34cc15d8b0 0x7f34cc15dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.290+0000 7f34d0ee9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc0b21b0 con 0x7f34cc15e830 2024-05-20T18:38:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.290+0000 7f34d0ee9640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc0b1ed0 con 0x7f34cc15d8b0 2024-05-20T18:38:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.290+0000 7f34d0ee9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f34cc0b2490 con 0x7f34b4004800 2024-05-20T18:38:00.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.290+0000 7f34c9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34cc15e830 0x7f34cc154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:00.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.290+0000 7f34c9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34cc15e830 0x7f34cc154720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60698/0 (socket says 172.21.15.5:60698) 2024-05-20T18:38:00.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.290+0000 7f34c9d9b640 1 -- 172.21.15.5:0/803806834 learned_addr learned my addr 172.21.15.5:0/803806834 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:38:00.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.290+0000 7f34ca59c640 1 --2- 172.21.15.5:0/803806834 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34b4004800 0x7f34cc1392a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:00.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.290+0000 7f34cad9d640 1 --2- 172.21.15.5:0/803806834 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34cc15d8b0 0x7f34cc15dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:00.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.290+0000 7f34c9d9b640 1 -- 172.21.15.5:0/803806834 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34b4004800 msgr2=0x7f34cc1392a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:00.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.290+0000 7f34c9d9b640 1 --2- 172.21.15.5:0/803806834 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34b4004800 0x7f34cc1392a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:00.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.290+0000 7f34c9d9b640 1 -- 172.21.15.5:0/803806834 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34cc15d8b0 msgr2=0x7f34cc15dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:00.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.290+0000 7f34c9d9b640 1 --2- 172.21.15.5:0/803806834 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34cc15d8b0 0x7f34cc15dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:00.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.290+0000 7f34c9d9b640 1 -- 172.21.15.5:0/803806834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34cc15e200 con 0x7f34cc15e830 2024-05-20T18:38:00.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.290+0000 7f34cad9d640 1 --2- 172.21.15.5:0/803806834 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34cc15d8b0 0x7f34cc15dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:38:00.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.290+0000 7f34c9d9b640 1 --2- 172.21.15.5:0/803806834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34cc15e830 0x7f34cc154720 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f34c0001b10 tx=0x7f34cc0b1ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:00.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.291+0000 7f34baffd640 1 -- 172.21.15.5:0/803806834 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c0043070 con 0x7f34cc15e830 2024-05-20T18:38:00.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.291+0000 7f34baffd640 1 -- 172.21.15.5:0/803806834 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34c0005bb0 con 0x7f34cc15e830 2024-05-20T18:38:00.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.291+0000 7f34baffd640 1 -- 172.21.15.5:0/803806834 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c0007d20 con 0x7f34cc15e830 2024-05-20T18:38:00.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.291+0000 7f34d0ee9640 1 -- 172.21.15.5:0/803806834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34cc155070 con 0x7f34cc15e830 2024-05-20T18:38:00.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.291+0000 7f34d0ee9640 1 -- 172.21.15.5:0/803806834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34cc150b20 con 0x7f34cc15e830 2024-05-20T18:38:00.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.292+0000 7f34d0ee9640 1 -- 172.21.15.5:0/803806834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34cc075b00 con 0x7f34cc15e830 2024-05-20T18:38:00.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.292+0000 7f34baffd640 1 -- 172.21.15.5:0/803806834 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f34c0033470 con 0x7f34cc15e830 2024-05-20T18:38:00.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.293+0000 7f34baffd640 1 --2- 172.21.15.5:0/803806834 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f34ac06be60 0x7f34ac06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:00.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.293+0000 7f34baffd640 1 -- 172.21.15.5:0/803806834 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f34c00aa680 con 0x7f34cc15e830 2024-05-20T18:38:00.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.293+0000 7f34ca59c640 1 --2- 172.21.15.5:0/803806834 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f34ac06be60 0x7f34ac06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:00.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.293+0000 7f34ca59c640 1 --2- 172.21.15.5:0/803806834 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f34ac06be60 0x7f34ac06e370 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f34cc0729c0 tx=0x7f34bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:00.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.297+0000 7f34baffd640 1 -- 172.21.15.5:0/803806834 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f34cc15e200 con 0x7f34cc15e830 2024-05-20T18:38:00.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.483+0000 7f34d0ee9640 1 -- 172.21.15.5:0/803806834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34cc15bdb0 con 0x7f34cc15e830 2024-05-20T18:38:00.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.484+0000 7f34baffd640 1 -- 172.21.15.5:0/803806834 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) ==== 74+0+13439 (secure 0 0 0) 0x7f34c0079060 con 0x7f34cc15e830 2024-05-20T18:38:00.486 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:38:00.486 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":124,"fsid":"399ab250-fd3c-47fa-a1c8-25e8149c9cac","created":"2024-05-20T18:26:50.233656+0000","modified":"2024-05-20T18:37:57.268424+0000","last_up_change":"2024-05-20T18:27:00.747766+0000","last_in_change":"2024-05-20T18:26:52.570603+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-20T18:27:02.619092+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3385e4a6-b897-4645-86af-19fd42f79fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6809","nonce":688093005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6811","nonce":688093005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6816","nonce":688093005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":688093005},{"type":"v1","addr":"172.21.15.5:6813","nonce":688093005}]},"public_addr":"172.21.15.5:6809/688093005","cluster_addr":"172.21.15.5:6811/688093005","heartbeat_back_addr":"172.21.15.5:6816/688093005","heartbeat_front_addr":"172.21.15.5:6813/688093005","state":["exists","up"]},{"osd":1,"uuid":"6a74555c-8e28-4b5a-9897-463604730539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6817","nonce":550664725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6819","nonce":550664725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6823","nonce":550664725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":550664725},{"type":"v1","addr":"172.21.15.5:6821","nonce":550664725}]},"public_addr":"172.21.15.5:6817/550664725","cluster_addr":"172.21.15.5:6819/550664725","heartbeat_back_addr":"172.21.15.5:6823/550664725","heartbeat_front_addr":"172.21.15.5:6821/550664725","state":["exists","up"]},{"osd":2,"uuid":"2c2a1831-7d9a-4a1c-b102-6181c2f85bda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6801","nonce":921679764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6803","nonce":921679764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6807","nonce":921679764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":921679764},{"type":"v1","addr":"172.21.15.5:6805","nonce":921679764}]},"public_addr":"172.21.15.5:6801/921679764","cluster_addr":"172.21.15.5:6803/921679764","heartbeat_back_addr":"172.21.15.5:6807/921679764","heartbeat_front_addr":"172.21.15.5:6805/921679764","state":["exists","up"]},{"osd":3,"uuid":"ef43f394-7be0-4d4f-ab20-96ef28024237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6825","nonce":1164130599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6827","nonce":1164130599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6831","nonce":1164130599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1164130599},{"type":"v1","addr":"172.21.15.5:6829","nonce":1164130599}]},"public_addr":"172.21.15.5:6825/1164130599","cluster_addr":"172.21.15.5:6827/1164130599","heartbeat_back_addr":"172.21.15.5:6831/1164130599","heartbeat_front_addr":"172.21.15.5:6829/1164130599","state":["exists","up"]},{"osd":4,"uuid":"07a9fcec-77b3-45ca-b075-8fb0b32754d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6801","nonce":3673324406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6803","nonce":3673324406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6807","nonce":3673324406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3673324406},{"type":"v1","addr":"172.21.15.63:6805","nonce":3673324406}]},"public_addr":"172.21.15.63:6801/3673324406","cluster_addr":"172.21.15.63:6803/3673324406","heartbeat_back_addr":"172.21.15.63:6807/3673324406","heartbeat_front_addr":"172.21.15.63:6805/3673324406","state":["exists","up"]},{"osd":5,"uuid":"4ae74bb5-50be-4128-aceb-02146503d093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6809","nonce":1616066286}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6811","nonce":1616066286}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6815","nonce":1616066286}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1616066286},{"type":"v1","addr":"172.21.15.63:6813","nonce":1616066286}]},"public_addr":"172.21.15.63:6809/1616066286","cluster_addr":"172.21.15.63:6811/1616066286","heartbeat_back_addr":"172.21.15.63:6815/1616066286","heartbeat_front_addr":"172.21.15.63:6813/1616066286","state":["exists","up"]},{"osd":6,"uuid":"11f04fa5-adbe-4016-903e-1652248e1395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6817","nonce":3384565140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6819","nonce":3384565140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6823","nonce":3384565140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3384565140},{"type":"v1","addr":"172.21.15.63:6821","nonce":3384565140}]},"public_addr":"172.21.15.63:6817/3384565140","cluster_addr":"172.21.15.63:6819/3384565140","heartbeat_back_addr":"172.21.15.63:6823/3384565140","heartbeat_front_addr":"172.21.15.63:6821/3384565140","state":["exists","up"]},{"osd":7,"uuid":"5ba4629f-35b0-4383-9264-e88757af5aaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6825","nonce":4136124451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6827","nonce":4136124451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6831","nonce":4136124451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":4136124451},{"type":"v1","addr":"172.21.15.63:6829","nonce":4136124451}]},"public_addr":"172.21.15.63:6825/4136124451","cluster_addr":"172.21.15.63:6827/4136124451","heartbeat_back_addr":"172.21.15.63:6831/4136124451","heartbeat_front_addr":"172.21.15.63:6829/4136124451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.767219+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.714346+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.668594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.790919+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.720238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.696392+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.795241+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-20T18:26:57.753804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6837/2067544478":"2024-05-21T18:37:54.256468+0000","172.21.15.5:6835/2067544478":"2024-05-21T18:37:54.256468+0000","172.21.15.5:6837/4273429943":"2024-05-21T18:37:39.699336+0000","172.21.15.63:6835/2352520472":"2024-05-21T18:37:28.901938+0000","172.21.15.5:6835/4273429943":"2024-05-21T18:37:39.699336+0000","172.21.15.63:6833/2352520472":"2024-05-21T18:37:28.901938+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-20T18:38:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.486+0000 7f34d0ee9640 1 -- 172.21.15.5:0/803806834 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f34ac06be60 msgr2=0x7f34ac06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.486+0000 7f34d0ee9640 1 --2- 172.21.15.5:0/803806834 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f34ac06be60 0x7f34ac06e370 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f34cc0729c0 tx=0x7f34bc009040 comp rx=0 tx=0).stop 2024-05-20T18:38:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.486+0000 7f34d0ee9640 1 -- 172.21.15.5:0/803806834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34cc15e830 msgr2=0x7f34cc154720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.486+0000 7f34d0ee9640 1 --2- 172.21.15.5:0/803806834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34cc15e830 0x7f34cc154720 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f34c0001b10 tx=0x7f34cc0b1ae0 comp rx=0 tx=0).stop 2024-05-20T18:38:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.486+0000 7f34d0ee9640 1 -- 172.21.15.5:0/803806834 shutdown_connections 2024-05-20T18:38:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.486+0000 7f34d0ee9640 1 --2- 172.21.15.5:0/803806834 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f34ac06be60 0x7f34ac06e370 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.486+0000 7f34d0ee9640 1 --2- 172.21.15.5:0/803806834 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34cc15d8b0 0x7f34cc15dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.486+0000 7f34d0ee9640 1 --2- 172.21.15.5:0/803806834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34cc15e830 0x7f34cc154720 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.486+0000 7f34d0ee9640 1 --2- 172.21.15.5:0/803806834 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34b4004800 0x7f34cc1392a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:00.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.486+0000 7f34d0ee9640 1 -- 172.21.15.5:0/803806834 >> 172.21.15.5:0/803806834 conn(0x7f34cc0b6670 msgr2=0x7f34cc15c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:00.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.486+0000 7f34d0ee9640 1 -- 172.21.15.5:0/803806834 shutdown_connections 2024-05-20T18:38:00.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:00.486+0000 7f34d0ee9640 1 -- 172.21.15.5:0/803806834 wait complete. 2024-05-20T18:38:01.500 INFO:tasks.ceph:Scrubbing osd.0 2024-05-20T18:38:01.500 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-20T18:38:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.620+0000 7fa625ae9640 1 Processor -- start 2024-05-20T18:38:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.620+0000 7fa625ae9640 1 -- start start 2024-05-20T18:38:01.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.621+0000 7fa625ae9640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa62009e740 0x7fa62015d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:01.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.621+0000 7fa625ae9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa6200af730 0x7fa62006f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:01.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.621+0000 7fa625ae9640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6200b00f0 con 0x7fa6200afb40 2024-05-20T18:38:01.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.621+0000 7fa625ae9640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fa6200ad8d0 con 0x7fa6200af730 2024-05-20T18:38:01.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.621+0000 7fa625ae9640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa6200adbb0 con 0x7fa62009e740 2024-05-20T18:38:01.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.621+0000 7fa61effd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa6200af730 0x7fa62006f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:01.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.621+0000 7fa61ffff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa6200afb40 0x7fa62015c6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43196/0 (socket says 172.21.15.5:43196) 2024-05-20T18:38:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.621+0000 7fa61ffff640 1 -- 172.21.15.5:0/1540796753 learned_addr learned my addr 172.21.15.5:0/1540796753 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:38:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.621+0000 7fa61f7fe640 1 --2- 172.21.15.5:0/1540796753 >> v2:172.21.15.63:3300/0 conn(0x7fa62009e740 0x7fa62015d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.622+0000 7fa61effd640 1 -- 172.21.15.5:0/1540796753 >> v2:172.21.15.63:3300/0 conn(0x7fa62009e740 msgr2=0x7fa62015d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:38:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.622+0000 7fa61effd640 1 --2- 172.21.15.5:0/1540796753 >> v2:172.21.15.63:3300/0 conn(0x7fa62009e740 0x7fa62015d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.622+0000 7fa61effd640 1 -- 172.21.15.5:0/1540796753 >> v1:172.21.15.5:6789/0 conn(0x7fa6200afb40 legacy=0x7fa62015c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.622+0000 7fa61effd640 1 -- 172.21.15.5:0/1540796753 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6200ade90 con 0x7fa6200af730 2024-05-20T18:38:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.622+0000 7fa61e7fc640 1 -- 172.21.15.5:0/1540796753 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2689315707 0 0) 0x7fa6200b00f0 con 0x7fa6200afb40 2024-05-20T18:38:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.622+0000 7fa61f7fe640 1 --2- 172.21.15.5:0/1540796753 >> v2:172.21.15.63:3300/0 conn(0x7fa62009e740 0x7fa62015d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:38:01.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.622+0000 7fa61effd640 1 --2- 172.21.15.5:0/1540796753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6200af730 0x7fa62006f900 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fa6140097a0 tx=0x7fa61402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb80d2c5bf3f040 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:01.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.623+0000 7fa61e7fc640 1 -- 172.21.15.5:0/1540796753 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa614037070 con 0x7fa6200af730 2024-05-20T18:38:01.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.623+0000 7fa61e7fc640 1 -- 172.21.15.5:0/1540796753 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa61402ae50 con 0x7fa6200af730 2024-05-20T18:38:01.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.623+0000 7fa61e7fc640 1 -- 172.21.15.5:0/1540796753 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa614032660 con 0x7fa6200af730 2024-05-20T18:38:01.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.623+0000 7fa625ae9640 1 -- 172.21.15.5:0/1540796753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6200af730 msgr2=0x7fa62006f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:01.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.623+0000 7fa625ae9640 1 --2- 172.21.15.5:0/1540796753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6200af730 0x7fa62006f900 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fa6140097a0 tx=0x7fa61402a140 comp rx=0 tx=0).stop 2024-05-20T18:38:01.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.623+0000 7fa625ae9640 1 -- 172.21.15.5:0/1540796753 shutdown_connections 2024-05-20T18:38:01.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.623+0000 7fa625ae9640 1 --2- 172.21.15.5:0/1540796753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6200af730 0x7fa62006f900 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:01.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.623+0000 7fa625ae9640 1 --2- 172.21.15.5:0/1540796753 >> v2:172.21.15.63:3300/0 conn(0x7fa62009e740 0x7fa62015d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:01.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.623+0000 7fa625ae9640 1 -- 172.21.15.5:0/1540796753 >> 172.21.15.5:0/1540796753 conn(0x7fa6200d0e10 msgr2=0x7fa6200723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:01.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.623+0000 7fa625ae9640 1 -- 172.21.15.5:0/1540796753 shutdown_connections 2024-05-20T18:38:01.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.623+0000 7fa625ae9640 1 -- 172.21.15.5:0/1540796753 wait complete. 2024-05-20T18:38:01.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.625+0000 7fa625ae9640 1 Processor -- start 2024-05-20T18:38:01.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.625+0000 7fa625ae9640 1 -- start start 2024-05-20T18:38:01.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.627+0000 7fa625ae9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa62009e740 0x7fa6201503f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:01.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.627+0000 7fa625ae9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6200af730 0x7fa6200c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:01.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.627+0000 7fa625ae9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6200afb40 0x7fa620158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:01.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.627+0000 7fa625ae9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa6200cff50 con 0x7fa6200af730 2024-05-20T18:38:01.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.627+0000 7fa625ae9640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa6200cfdd0 con 0x7fa62009e740 2024-05-20T18:38:01.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.627+0000 7fa625ae9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa62006fe40 con 0x7fa6200afb40 2024-05-20T18:38:01.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.627+0000 7fa61effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6200af730 0x7fa6200c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:01.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.627+0000 7fa61effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6200af730 0x7fa6200c9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60716/0 (socket says 172.21.15.5:60716) 2024-05-20T18:38:01.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.627+0000 7fa61effd640 1 -- 172.21.15.5:0/3122428554 learned_addr learned my addr 172.21.15.5:0/3122428554 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:38:01.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.627+0000 7fa61f7fe640 1 --2- 172.21.15.5:0/3122428554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa62009e740 0x7fa6201503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:01.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.627+0000 7fa61ffff640 1 --2- 172.21.15.5:0/3122428554 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6200afb40 0x7fa620158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:01.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.627+0000 7fa61effd640 1 -- 172.21.15.5:0/3122428554 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6200afb40 msgr2=0x7fa620158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:01.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.627+0000 7fa61effd640 1 --2- 172.21.15.5:0/3122428554 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6200afb40 0x7fa620158070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:01.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.627+0000 7fa61effd640 1 -- 172.21.15.5:0/3122428554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa62009e740 msgr2=0x7fa6201503f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:01.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.627+0000 7fa61effd640 1 --2- 172.21.15.5:0/3122428554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa62009e740 0x7fa6201503f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:01.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.627+0000 7fa61effd640 1 -- 172.21.15.5:0/3122428554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6200ca770 con 0x7fa6200af730 2024-05-20T18:38:01.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.627+0000 7fa61effd640 1 --2- 172.21.15.5:0/3122428554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6200af730 0x7fa6200c9bd0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fa6140098c0 tx=0x7fa6140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:01.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.627+0000 7fa607fff640 1 -- 172.21.15.5:0/3122428554 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa614041020 con 0x7fa6200af730 2024-05-20T18:38:01.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.627+0000 7fa607fff640 1 -- 172.21.15.5:0/3122428554 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa61403c670 con 0x7fa6200af730 2024-05-20T18:38:01.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.627+0000 7fa625ae9640 1 -- 172.21.15.5:0/3122428554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6200caa50 con 0x7fa6200af730 2024-05-20T18:38:01.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.627+0000 7fa607fff640 1 -- 172.21.15.5:0/3122428554 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa614037040 con 0x7fa6200af730 2024-05-20T18:38:01.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.628+0000 7fa625ae9640 1 -- 172.21.15.5:0/3122428554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6200cd6e0 con 0x7fa6200af730 2024-05-20T18:38:01.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.628+0000 7fa625ae9640 1 -- 172.21.15.5:0/3122428554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa6200cac90 con 0x7fa6200af730 2024-05-20T18:38:01.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.629+0000 7fa607fff640 1 -- 172.21.15.5:0/3122428554 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa614044020 con 0x7fa6200af730 2024-05-20T18:38:01.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.630+0000 7fa607fff640 1 --2- 172.21.15.5:0/3122428554 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa60c06be60 0x7fa60c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:01.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.630+0000 7fa607fff640 1 -- 172.21.15.5:0/3122428554 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7fa6140abf30 con 0x7fa6200af730 2024-05-20T18:38:01.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.630+0000 7fa61f7fe640 1 --2- 172.21.15.5:0/3122428554 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa60c06be60 0x7fa60c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:01.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.630+0000 7fa607fff640 1 --2- 172.21.15.5:0/3122428554 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x7fa60c072490 0x7fa60c074920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:01.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.630+0000 7fa607fff640 1 -- 172.21.15.5:0/3122428554 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa614034e90 con 0x7fa60c072490 2024-05-20T18:38:01.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.630+0000 7fa607fff640 1 -- 172.21.15.5:0/3122428554 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=124) ==== 24+0+0 (secure 0 0 0) 0x7fa6140ac3c0 con 0x7fa6200af730 2024-05-20T18:38:01.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.630+0000 7fa61ffff640 1 --2- 172.21.15.5:0/3122428554 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x7fa60c072490 0x7fa60c074920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:01.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.630+0000 7fa61f7fe640 1 --2- 172.21.15.5:0/3122428554 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa60c06be60 0x7fa60c06e370 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fa610006fd0 tx=0x7fa610008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:01.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.630+0000 7fa61ffff640 1 --2- 172.21.15.5:0/3122428554 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x7fa60c072490 0x7fa60c074920 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:01.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.631+0000 7fa607fff640 1 -- 172.21.15.5:0/3122428554 <== osd.0 v2:172.21.15.5:6808/688093005 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa614034e90 con 0x7fa60c072490 2024-05-20T18:38:01.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.644+0000 7fa625ae9640 1 -- 172.21.15.5:0/3122428554 --> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa6200690b0 con 0x7fa60c072490 2024-05-20T18:38:01.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.645+0000 7fa607fff640 1 -- 172.21.15.5:0/3122428554 <== osd.0 v2:172.21.15.5:6808/688093005 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa6200690b0 con 0x7fa60c072490 2024-05-20T18:38:01.646 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:38:01.646 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-20T18:38:01.647 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:38:01.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.645+0000 7fa625ae9640 1 -- 172.21.15.5:0/3122428554 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x7fa60c072490 msgr2=0x7fa60c074920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:01.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.645+0000 7fa625ae9640 1 --2- 172.21.15.5:0/3122428554 >> [v2:172.21.15.5:6808/688093005,v1:172.21.15.5:6809/688093005] conn(0x7fa60c072490 0x7fa60c074920 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:01.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.645+0000 7fa625ae9640 1 -- 172.21.15.5:0/3122428554 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa60c06be60 msgr2=0x7fa60c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:01.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.645+0000 7fa625ae9640 1 --2- 172.21.15.5:0/3122428554 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fa60c06be60 0x7fa60c06e370 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fa610006fd0 tx=0x7fa610008040 comp rx=0 tx=0).stop 2024-05-20T18:38:01.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.645+0000 7fa625ae9640 1 -- 172.21.15.5:0/3122428554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6200af730 msgr2=0x7fa6200c9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:01.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.645+0000 7fa625ae9640 1 --2- 172.21.15.5:0/3122428554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6200af730 0x7fa6200c9bd0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fa6140098c0 tx=0x7fa6140046c0 comp rx=0 tx=0).stop 2024-05-20T18:38:01.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.645+0000 7fa61ffff640 1 -- 172.21.15.5:0/3122428554 reap_dead start 2024-05-20T18:38:01.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.645+0000 7fa625ae9640 1 -- 172.21.15.5:0/3122428554 shutdown_connections 2024-05-20T18:38:01.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.645+0000 7fa625ae9640 1 -- 172.21.15.5:0/3122428554 >> 172.21.15.5:0/3122428554 conn(0x7fa6200d0e10 msgr2=0x7fa620159300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:01.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.645+0000 7fa625ae9640 1 -- 172.21.15.5:0/3122428554 shutdown_connections 2024-05-20T18:38:01.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.645+0000 7fa625ae9640 1 -- 172.21.15.5:0/3122428554 wait complete. 2024-05-20T18:38:01.658 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-20T18:38:01.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.772+0000 7fd5d0745640 1 Processor -- start 2024-05-20T18:38:01.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.772+0000 7fd5d0745640 1 -- start start 2024-05-20T18:38:01.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.774+0000 7fd5d0745640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5c806d3d0 0x7fd5c806d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.774+0000 7fd5d0745640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd5c80b8190 0x7fd5c80b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.774+0000 7fd5d0745640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5c806dd20 con 0x7fd5c8162820 2024-05-20T18:38:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.774+0000 7fd5d0745640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd5c80d0630 con 0x7fd5c80b8190 2024-05-20T18:38:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.774+0000 7fd5d0745640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd5c80d39c0 con 0x7fd5c806d3d0 2024-05-20T18:38:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.774+0000 7fd5cece2640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd5c80b8190 0x7fd5c80b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.774+0000 7fd5cdce0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5c806d3d0 0x7fd5c806d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.774+0000 7fd5ce4e1640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fd5c8162820 0x7fd5c80b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.5:34682/0 (socket says 172.21.15.5:34682) 2024-05-20T18:38:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.774+0000 7fd5ce4e1640 1 -- 172.21.15.5:0/2471566088 learned_addr learned my addr 172.21.15.5:0/2471566088 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:38:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.774+0000 7fd5cece2640 1 -- 172.21.15.5:0/2471566088 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5c806d3d0 msgr2=0x7fd5c806d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:38:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.774+0000 7fd5cece2640 1 --2- 172.21.15.5:0/2471566088 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5c806d3d0 0x7fd5c806d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.774+0000 7fd5cece2640 1 -- 172.21.15.5:0/2471566088 >> v1:172.21.15.63:6789/0 conn(0x7fd5c8162820 legacy=0x7fd5c80b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.774+0000 7fd5cd4df640 1 -- 172.21.15.5:0/2471566088 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1322540868 0 0) 0x7fd5c4001110 con 0x7fd5c8162820 2024-05-20T18:38:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.774+0000 7fd5cece2640 1 -- 172.21.15.5:0/2471566088 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c80d3ca0 con 0x7fd5c80b8190 2024-05-20T18:38:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.774+0000 7fd5cdce0640 1 --2- 172.21.15.5:0/2471566088 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5c806d3d0 0x7fd5c806d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:38:01.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.775+0000 7fd5cece2640 1 --2- 172.21.15.5:0/2471566088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5c80b8190 0x7fd5c80b85a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0009d90 tx=0x7fd5c002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5525d9b07b579c75 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:01.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.775+0000 7fd5cd4df640 1 -- 172.21.15.5:0/2471566088 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0007740 con 0x7fd5c80b8190 2024-05-20T18:38:01.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.775+0000 7fd5cd4df640 1 -- 172.21.15.5:0/2471566088 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5c00078e0 con 0x7fd5c80b8190 2024-05-20T18:38:01.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.775+0000 7fd5cd4df640 1 -- 172.21.15.5:0/2471566088 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0007cb0 con 0x7fd5c80b8190 2024-05-20T18:38:01.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.776+0000 7fd5d0745640 1 -- 172.21.15.5:0/2471566088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5c80b8190 msgr2=0x7fd5c80b85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:01.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.776+0000 7fd5d0745640 1 --2- 172.21.15.5:0/2471566088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5c80b8190 0x7fd5c80b85a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0009d90 tx=0x7fd5c002a660 comp rx=0 tx=0).stop 2024-05-20T18:38:01.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.776+0000 7fd5d0745640 1 -- 172.21.15.5:0/2471566088 shutdown_connections 2024-05-20T18:38:01.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.776+0000 7fd5d0745640 1 --2- 172.21.15.5:0/2471566088 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5c80b8190 0x7fd5c80b85a0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:01.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.776+0000 7fd5d0745640 1 --2- 172.21.15.5:0/2471566088 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5c806d3d0 0x7fd5c806d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:01.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.776+0000 7fd5d0745640 1 -- 172.21.15.5:0/2471566088 >> 172.21.15.5:0/2471566088 conn(0x7fd5c80d4bd0 msgr2=0x7fd5c80d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:01.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.776+0000 7fd5d0745640 1 -- 172.21.15.5:0/2471566088 shutdown_connections 2024-05-20T18:38:01.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.776+0000 7fd5d0745640 1 -- 172.21.15.5:0/2471566088 wait complete. 2024-05-20T18:38:01.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.778+0000 7fd5d0745640 1 Processor -- start 2024-05-20T18:38:01.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.778+0000 7fd5d0745640 1 -- start start 2024-05-20T18:38:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.779+0000 7fd5d0745640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5c806d3d0 0x7fd5c80ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.779+0000 7fd5d0745640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5c80b8190 0x7fd5c80b4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.779+0000 7fd5d0745640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5c8162820 0x7fd5c80b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.779+0000 7fd5d0745640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c80d0300 con 0x7fd5c8162820 2024-05-20T18:38:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.779+0000 7fd5d0745640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c80d4260 con 0x7fd5c806d3d0 2024-05-20T18:38:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.779+0000 7fd5d0745640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd5c80b4050 con 0x7fd5c80b8190 2024-05-20T18:38:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.779+0000 7fd5cece2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5c8162820 0x7fd5c80b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.779+0000 7fd5cece2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5c8162820 0x7fd5c80b4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60724/0 (socket says 172.21.15.5:60724) 2024-05-20T18:38:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.779+0000 7fd5cece2640 1 -- 172.21.15.5:0/3588624147 learned_addr learned my addr 172.21.15.5:0/3588624147 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:38:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.780+0000 7fd5cdce0640 1 --2- 172.21.15.5:0/3588624147 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5c80b8190 0x7fd5c80b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.780+0000 7fd5ce4e1640 1 --2- 172.21.15.5:0/3588624147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5c806d3d0 0x7fd5c80ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.780+0000 7fd5cece2640 1 -- 172.21.15.5:0/3588624147 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5c80b8190 msgr2=0x7fd5c80b4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:01.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.780+0000 7fd5cece2640 1 --2- 172.21.15.5:0/3588624147 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5c80b8190 0x7fd5c80b4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:01.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.780+0000 7fd5cece2640 1 -- 172.21.15.5:0/3588624147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5c806d3d0 msgr2=0x7fd5c80ca930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:01.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.780+0000 7fd5cece2640 1 --2- 172.21.15.5:0/3588624147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5c806d3d0 0x7fd5c80ca930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:01.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.780+0000 7fd5cece2640 1 -- 172.21.15.5:0/3588624147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c80c8700 con 0x7fd5c8162820 2024-05-20T18:38:01.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.780+0000 7fd5cece2640 1 --2- 172.21.15.5:0/3588624147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5c8162820 0x7fd5c80b4e40 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fd5c002ac50 tx=0x7fd5c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:01.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.780+0000 7fd5beffd640 1 -- 172.21.15.5:0/3588624147 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c003f070 con 0x7fd5c8162820 2024-05-20T18:38:01.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.780+0000 7fd5beffd640 1 -- 172.21.15.5:0/3588624147 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5c0031bc0 con 0x7fd5c8162820 2024-05-20T18:38:01.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.780+0000 7fd5d0745640 1 -- 172.21.15.5:0/3588624147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c80b56b0 con 0x7fd5c8162820 2024-05-20T18:38:01.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.780+0000 7fd5beffd640 1 -- 172.21.15.5:0/3588624147 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0009450 con 0x7fd5c8162820 2024-05-20T18:38:01.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.780+0000 7fd5d0745640 1 -- 172.21.15.5:0/3588624147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c80b5c70 con 0x7fd5c8162820 2024-05-20T18:38:01.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.781+0000 7fd5d0745640 1 -- 172.21.15.5:0/3588624147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5c8075b00 con 0x7fd5c8162820 2024-05-20T18:38:01.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.782+0000 7fd5beffd640 1 -- 172.21.15.5:0/3588624147 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd5c0044460 con 0x7fd5c8162820 2024-05-20T18:38:01.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.782+0000 7fd5beffd640 1 --2- 172.21.15.5:0/3588624147 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd5ac06be60 0x7fd5ac06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:01.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.783+0000 7fd5beffd640 1 -- 172.21.15.5:0/3588624147 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7fd5c00ac440 con 0x7fd5c8162820 2024-05-20T18:38:01.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.783+0000 7fd5ce4e1640 1 --2- 172.21.15.5:0/3588624147 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd5ac06be60 0x7fd5ac06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:01.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.783+0000 7fd5ce4e1640 1 --2- 172.21.15.5:0/3588624147 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd5ac06be60 0x7fd5ac06e370 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4007fd0 tx=0x7fd5c4007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.787+0000 7fd5beffd640 1 -- 172.21.15.5:0/3588624147 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd5c007a7d0 con 0x7fd5c8162820 2024-05-20T18:38:01.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.970+0000 7fd5d0745640 1 -- 172.21.15.5:0/3588624147 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fd5c80b5400 con 0x7fd5ac06be60 2024-05-20T18:38:01.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.970+0000 7fd5beffd640 1 -- 172.21.15.5:0/3588624147 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd5c80b5400 con 0x7fd5ac06be60 2024-05-20T18:38:01.975 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 0 to deep-scrub 2024-05-20T18:38:01.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.972+0000 7fd5d0745640 1 -- 172.21.15.5:0/3588624147 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd5ac06be60 msgr2=0x7fd5ac06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:01.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.972+0000 7fd5d0745640 1 --2- 172.21.15.5:0/3588624147 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd5ac06be60 0x7fd5ac06e370 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4007fd0 tx=0x7fd5c4007de0 comp rx=0 tx=0).stop 2024-05-20T18:38:01.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.972+0000 7fd5d0745640 1 -- 172.21.15.5:0/3588624147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5c8162820 msgr2=0x7fd5c80b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:01.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.972+0000 7fd5d0745640 1 --2- 172.21.15.5:0/3588624147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5c8162820 0x7fd5c80b4e40 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fd5c002ac50 tx=0x7fd5c0004a40 comp rx=0 tx=0).stop 2024-05-20T18:38:01.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.972+0000 7fd5d0745640 1 -- 172.21.15.5:0/3588624147 shutdown_connections 2024-05-20T18:38:01.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.972+0000 7fd5d0745640 1 --2- 172.21.15.5:0/3588624147 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fd5ac06be60 0x7fd5ac06e370 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:01.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.972+0000 7fd5d0745640 1 --2- 172.21.15.5:0/3588624147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5c8162820 0x7fd5c80b4e40 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:01.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.972+0000 7fd5d0745640 1 --2- 172.21.15.5:0/3588624147 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5c80b8190 0x7fd5c80b4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:01.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.972+0000 7fd5d0745640 1 --2- 172.21.15.5:0/3588624147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd5c806d3d0 0x7fd5c80ca930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:01.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.972+0000 7fd5d0745640 1 -- 172.21.15.5:0/3588624147 >> 172.21.15.5:0/3588624147 conn(0x7fd5c80d4bd0 msgr2=0x7fd5c80d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:01.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.972+0000 7fd5d0745640 1 -- 172.21.15.5:0/3588624147 shutdown_connections 2024-05-20T18:38:01.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:01.972+0000 7fd5d0745640 1 -- 172.21.15.5:0/3588624147 wait complete. 2024-05-20T18:38:01.984 INFO:tasks.ceph:Scrubbing osd.1 2024-05-20T18:38:01.985 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-20T18:38:02.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.102+0000 7f1a5c5a8640 1 Processor -- start 2024-05-20T18:38:02.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.102+0000 7f1a5c5a8640 1 -- start start 2024-05-20T18:38:02.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.104+0000 7f1a5c5a8640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1a5406ff10 0x7f1a54160b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:02.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.104+0000 7f1a5c5a8640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a541613d0 con 0x7f1a5415ddc0 2024-05-20T18:38:02.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.104+0000 7f1a5c5a8640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a541616b0 con 0x7f1a5406cc40 2024-05-20T18:38:02.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.104+0000 7f1a5c5a8640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f1a54161990 con 0x7f1a5406ff10 2024-05-20T18:38:02.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.104+0000 7f1a5a344640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1a5406ff10 0x7f1a54160b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:02.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.104+0000 7f1a59b43640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1a5415ddc0 0x7f1a5406c530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43198/0 (socket says 172.21.15.5:43198) 2024-05-20T18:38:02.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.104+0000 7f1a59b43640 1 -- 172.21.15.5:0/2113393897 learned_addr learned my addr 172.21.15.5:0/2113393897 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:38:02.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.104+0000 7f1a5a344640 1 -- 172.21.15.5:0/2113393897 >> v1:172.21.15.63:6789/0 conn(0x7f1a5406cc40 legacy=0x7f1a540ad830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:02.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.104+0000 7f1a5a344640 1 -- 172.21.15.5:0/2113393897 >> v1:172.21.15.5:6789/0 conn(0x7f1a5415ddc0 legacy=0x7f1a5406c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:02.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.104+0000 7f1a59342640 1 -- 172.21.15.5:0/2113393897 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1704760443 0 0) 0x7f1a541613d0 con 0x7f1a5415ddc0 2024-05-20T18:38:02.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.104+0000 7f1a5a344640 1 -- 172.21.15.5:0/2113393897 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a5415d5b0 con 0x7f1a5406ff10 2024-05-20T18:38:02.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.105+0000 7f1a5a344640 1 --2- 172.21.15.5:0/2113393897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a5406ff10 0x7f1a54160b80 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f1a50009d90 tx=0x7f1a5002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e1f1e04415c725b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:02.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.105+0000 7f1a59342640 1 -- 172.21.15.5:0/2113393897 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a50007740 con 0x7f1a5406ff10 2024-05-20T18:38:02.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.105+0000 7f1a59342640 1 -- 172.21.15.5:0/2113393897 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a500078e0 con 0x7f1a5406ff10 2024-05-20T18:38:02.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.105+0000 7f1a59342640 1 -- 172.21.15.5:0/2113393897 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a50007cb0 con 0x7f1a5406ff10 2024-05-20T18:38:02.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.106+0000 7f1a5c5a8640 1 -- 172.21.15.5:0/2113393897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a5406ff10 msgr2=0x7f1a54160b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:02.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.106+0000 7f1a5c5a8640 1 --2- 172.21.15.5:0/2113393897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a5406ff10 0x7f1a54160b80 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f1a50009d90 tx=0x7f1a5002a660 comp rx=0 tx=0).stop 2024-05-20T18:38:02.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.106+0000 7f1a5c5a8640 1 -- 172.21.15.5:0/2113393897 shutdown_connections 2024-05-20T18:38:02.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.106+0000 7f1a5c5a8640 1 --2- 172.21.15.5:0/2113393897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a5406ff10 0x7f1a54160b80 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.106+0000 7f1a5c5a8640 1 -- 172.21.15.5:0/2113393897 >> 172.21.15.5:0/2113393897 conn(0x7f1a540d0e10 msgr2=0x7f1a540723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:02.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.106+0000 7f1a5c5a8640 1 -- 172.21.15.5:0/2113393897 shutdown_connections 2024-05-20T18:38:02.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.106+0000 7f1a5c5a8640 1 -- 172.21.15.5:0/2113393897 wait complete. 2024-05-20T18:38:02.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.108+0000 7f1a5c5a8640 1 Processor -- start 2024-05-20T18:38:02.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.108+0000 7f1a5c5a8640 1 -- start start 2024-05-20T18:38:02.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.109+0000 7f1a5c5a8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a5406ff10 0x7f1a541f35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.109+0000 7f1a5c5a8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a5415ddc0 0x7f1a54160280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.109+0000 7f1a5c5a8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1a540cee70 0x7f1a540cf280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.109+0000 7f1a5c5a8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1a540adf40 con 0x7f1a5415ddc0 2024-05-20T18:38:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.109+0000 7f1a5c5a8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5415aec0 con 0x7f1a5406ff10 2024-05-20T18:38:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.109+0000 7f1a5c5a8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1a540ae0c0 con 0x7f1a540cee70 2024-05-20T18:38:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.109+0000 7f1a59b43640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a5415ddc0 0x7f1a54160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.109+0000 7f1a59b43640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a5415ddc0 0x7f1a54160280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60744/0 (socket says 172.21.15.5:60744) 2024-05-20T18:38:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.109+0000 7f1a59b43640 1 -- 172.21.15.5:0/4201532920 learned_addr learned my addr 172.21.15.5:0/4201532920 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:38:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.110+0000 7f1a59b43640 1 -- 172.21.15.5:0/4201532920 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1a540cee70 msgr2=0x7f1a540cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.110+0000 7f1a5a344640 1 --2- 172.21.15.5:0/4201532920 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a5406ff10 0x7f1a541f35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.110+0000 7f1a5ab45640 1 --2- 172.21.15.5:0/4201532920 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1a540cee70 0x7f1a540cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.110+0000 7f1a59b43640 1 --2- 172.21.15.5:0/4201532920 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1a540cee70 0x7f1a540cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.110+0000 7f1a59b43640 1 -- 172.21.15.5:0/4201532920 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a5406ff10 msgr2=0x7f1a541f35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:02.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.110+0000 7f1a59b43640 1 --2- 172.21.15.5:0/4201532920 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1a5406ff10 0x7f1a541f35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.110+0000 7f1a59b43640 1 -- 172.21.15.5:0/4201532920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a540cf7c0 con 0x7f1a5415ddc0 2024-05-20T18:38:02.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.110+0000 7f1a59b43640 1 --2- 172.21.15.5:0/4201532920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a5415ddc0 0x7f1a54160280 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f1a4400c3f0 tx=0x7f1a4400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:02.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.110+0000 7f1a4affd640 1 -- 172.21.15.5:0/4201532920 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a440074e0 con 0x7f1a5415ddc0 2024-05-20T18:38:02.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.110+0000 7f1a4affd640 1 -- 172.21.15.5:0/4201532920 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a440047e0 con 0x7f1a5415ddc0 2024-05-20T18:38:02.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.110+0000 7f1a5c5a8640 1 -- 172.21.15.5:0/4201532920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a54149740 con 0x7f1a5415ddc0 2024-05-20T18:38:02.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.110+0000 7f1a4affd640 1 -- 172.21.15.5:0/4201532920 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44004bd0 con 0x7f1a5415ddc0 2024-05-20T18:38:02.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.110+0000 7f1a5c5a8640 1 -- 172.21.15.5:0/4201532920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a54149d00 con 0x7f1a5415ddc0 2024-05-20T18:38:02.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.111+0000 7f1a5c5a8640 1 -- 172.21.15.5:0/4201532920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1a5415e1d0 con 0x7f1a5415ddc0 2024-05-20T18:38:02.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.112+0000 7f1a4affd640 1 -- 172.21.15.5:0/4201532920 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1a44011070 con 0x7f1a5415ddc0 2024-05-20T18:38:02.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.112+0000 7f1a4affd640 1 --2- 172.21.15.5:0/4201532920 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1a3806be60 0x7f1a3806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:02.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.112+0000 7f1a4affd640 1 -- 172.21.15.5:0/4201532920 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f1a4408aa00 con 0x7f1a5415ddc0 2024-05-20T18:38:02.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.113+0000 7f1a5a344640 1 --2- 172.21.15.5:0/4201532920 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1a3806be60 0x7f1a3806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:02.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.113+0000 7f1a4affd640 1 --2- 172.21.15.5:0/4201532920 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x7f1a38072490 0x7f1a38074920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:02.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.113+0000 7f1a4affd640 1 -- 172.21.15.5:0/4201532920 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1a4401ce80 con 0x7f1a38072490 2024-05-20T18:38:02.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.113+0000 7f1a4affd640 1 -- 172.21.15.5:0/4201532920 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=124) ==== 24+0+0 (secure 0 0 0) 0x7f1a4408ce60 con 0x7f1a5415ddc0 2024-05-20T18:38:02.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.113+0000 7f1a5ab45640 1 --2- 172.21.15.5:0/4201532920 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x7f1a38072490 0x7f1a38074920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:02.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.113+0000 7f1a5a344640 1 --2- 172.21.15.5:0/4201532920 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1a3806be60 0x7f1a3806e370 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f1a50034000 tx=0x7f1a5002f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:02.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.113+0000 7f1a5ab45640 1 --2- 172.21.15.5:0/4201532920 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x7f1a38072490 0x7f1a38074920 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:02.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.114+0000 7f1a4affd640 1 -- 172.21.15.5:0/4201532920 <== osd.1 v2:172.21.15.5:6815/550664725 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1a4401ce80 con 0x7f1a38072490 2024-05-20T18:38:02.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.127+0000 7f1a5c5a8640 1 -- 172.21.15.5:0/4201532920 --> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1a541494f0 con 0x7f1a38072490 2024-05-20T18:38:02.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.127+0000 7f1a4affd640 1 -- 172.21.15.5:0/4201532920 <== osd.1 v2:172.21.15.5:6815/550664725 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f1a541494f0 con 0x7f1a38072490 2024-05-20T18:38:02.129 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:38:02.129 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-20T18:38:02.129 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:38:02.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.127+0000 7f1a5c5a8640 1 -- 172.21.15.5:0/4201532920 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x7f1a38072490 msgr2=0x7f1a38074920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:02.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.128+0000 7f1a5c5a8640 1 --2- 172.21.15.5:0/4201532920 >> [v2:172.21.15.5:6815/550664725,v1:172.21.15.5:6817/550664725] conn(0x7f1a38072490 0x7f1a38074920 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.128+0000 7f1a5c5a8640 1 -- 172.21.15.5:0/4201532920 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1a3806be60 msgr2=0x7f1a3806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:02.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.128+0000 7f1a5c5a8640 1 --2- 172.21.15.5:0/4201532920 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1a3806be60 0x7f1a3806e370 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f1a50034000 tx=0x7f1a5002f000 comp rx=0 tx=0).stop 2024-05-20T18:38:02.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.128+0000 7f1a5c5a8640 1 -- 172.21.15.5:0/4201532920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a5415ddc0 msgr2=0x7f1a54160280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:02.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.128+0000 7f1a5c5a8640 1 --2- 172.21.15.5:0/4201532920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a5415ddc0 0x7f1a54160280 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f1a4400c3f0 tx=0x7f1a4400c8b0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.128+0000 7f1a5ab45640 1 -- 172.21.15.5:0/4201532920 reap_dead start 2024-05-20T18:38:02.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.128+0000 7f1a5c5a8640 1 -- 172.21.15.5:0/4201532920 shutdown_connections 2024-05-20T18:38:02.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.128+0000 7f1a5c5a8640 1 -- 172.21.15.5:0/4201532920 >> 172.21.15.5:0/4201532920 conn(0x7f1a540d0e10 msgr2=0x7f1a5415b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:02.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.128+0000 7f1a5c5a8640 1 -- 172.21.15.5:0/4201532920 shutdown_connections 2024-05-20T18:38:02.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.128+0000 7f1a5c5a8640 1 -- 172.21.15.5:0/4201532920 wait complete. 2024-05-20T18:38:02.140 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-20T18:38:02.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.254+0000 7fc843a73640 1 Processor -- start 2024-05-20T18:38:02.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.254+0000 7fc843a73640 1 -- start start 2024-05-20T18:38:02.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.256+0000 7fc843a73640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc83c0b0020 0x7fc83c15fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:02.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.256+0000 7fc843a73640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc83c06c4c0 0x7fc83c0b3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:02.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.256+0000 7fc843a73640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc83c0b51c0 con 0x7fc83c15ca00 2024-05-20T18:38:02.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.256+0000 7fc843a73640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fc83c15bf20 con 0x7fc83c0b0020 2024-05-20T18:38:02.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.256+0000 7fc843a73640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc83c15c200 con 0x7fc83c06c4c0 2024-05-20T18:38:02.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.256+0000 7fc84180f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc83c0b0020 0x7fc83c15fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:02.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.256+0000 7fc84180f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc83c0b0020 0x7fc83c15fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60760/0 (socket says 172.21.15.5:60760) 2024-05-20T18:38:02.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.256+0000 7fc84180f640 1 -- 172.21.15.5:0/1428140636 learned_addr learned my addr 172.21.15.5:0/1428140636 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:38:02.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.256+0000 7fc842010640 1 --2- 172.21.15.5:0/1428140636 >> v2:172.21.15.63:3300/0 conn(0x7fc83c06c4c0 0x7fc83c0b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:02.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.256+0000 7fc84180f640 1 -- 172.21.15.5:0/1428140636 >> v2:172.21.15.63:3300/0 conn(0x7fc83c06c4c0 msgr2=0x7fc83c0b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:38:02.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.256+0000 7fc84180f640 1 --2- 172.21.15.5:0/1428140636 >> v2:172.21.15.63:3300/0 conn(0x7fc83c06c4c0 0x7fc83c0b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.256+0000 7fc84180f640 1 -- 172.21.15.5:0/1428140636 >> v1:172.21.15.63:6789/0 conn(0x7fc83c15ca00 legacy=0x7fc83c06bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:02.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.256+0000 7fc84180f640 1 -- 172.21.15.5:0/1428140636 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83c15c4e0 con 0x7fc83c0b0020 2024-05-20T18:38:02.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.256+0000 7fc842010640 1 --2- 172.21.15.5:0/1428140636 >> v2:172.21.15.63:3300/0 conn(0x7fc83c06c4c0 0x7fc83c0b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:38:02.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.257+0000 7fc84180f640 1 --2- 172.21.15.5:0/1428140636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc83c0b0020 0x7fc83c15fd80 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fc83c0723a0 tx=0x7fc82c02b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61a8c5eb984ff0a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:02.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.257+0000 7fc84080d640 1 -- 172.21.15.5:0/1428140636 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c037070 con 0x7fc83c0b0020 2024-05-20T18:38:02.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.257+0000 7fc84080d640 1 -- 172.21.15.5:0/1428140636 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc82c008500 con 0x7fc83c0b0020 2024-05-20T18:38:02.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.257+0000 7fc84080d640 1 -- 172.21.15.5:0/1428140636 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c0088f0 con 0x7fc83c0b0020 2024-05-20T18:38:02.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.258+0000 7fc843a73640 1 -- 172.21.15.5:0/1428140636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc83c0b0020 msgr2=0x7fc83c15fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:02.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.258+0000 7fc843a73640 1 --2- 172.21.15.5:0/1428140636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc83c0b0020 0x7fc83c15fd80 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fc83c0723a0 tx=0x7fc82c02b7c0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.258+0000 7fc843a73640 1 -- 172.21.15.5:0/1428140636 shutdown_connections 2024-05-20T18:38:02.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.258+0000 7fc843a73640 1 --2- 172.21.15.5:0/1428140636 >> v2:172.21.15.63:3300/0 conn(0x7fc83c06c4c0 0x7fc83c0b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.258+0000 7fc843a73640 1 --2- 172.21.15.5:0/1428140636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc83c0b0020 0x7fc83c15fd80 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.258+0000 7fc843a73640 1 -- 172.21.15.5:0/1428140636 >> 172.21.15.5:0/1428140636 conn(0x7fc83c0cd2a0 msgr2=0x7fc83c0cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:02.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.258+0000 7fc843a73640 1 -- 172.21.15.5:0/1428140636 shutdown_connections 2024-05-20T18:38:02.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.258+0000 7fc843a73640 1 -- 172.21.15.5:0/1428140636 wait complete. 2024-05-20T18:38:02.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.260+0000 7fc843a73640 1 Processor -- start 2024-05-20T18:38:02.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.260+0000 7fc843a73640 1 -- start start 2024-05-20T18:38:02.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.261+0000 7fc843a73640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc83c0b0020 0x7fc83c0c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:02.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.261+0000 7fc843a73640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc83c15ca00 0x7fc83c14c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:02.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.261+0000 7fc843a73640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc83c0ae8f0 0x7fc83c0aed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:02.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.261+0000 7fc843a73640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c0b4c00 con 0x7fc83c15ca00 2024-05-20T18:38:02.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.261+0000 7fc843a73640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c0b54a0 con 0x7fc83c0b0020 2024-05-20T18:38:02.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.261+0000 7fc843a73640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc83c0b4d80 con 0x7fc83c0ae8f0 2024-05-20T18:38:02.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.261+0000 7fc84100e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc83c15ca00 0x7fc83c14c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:02.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.261+0000 7fc84100e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc83c15ca00 0x7fc83c14c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60764/0 (socket says 172.21.15.5:60764) 2024-05-20T18:38:02.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.261+0000 7fc84100e640 1 -- 172.21.15.5:0/3043711767 learned_addr learned my addr 172.21.15.5:0/3043711767 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:38:02.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.262+0000 7fc84180f640 1 --2- 172.21.15.5:0/3043711767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc83c0b0020 0x7fc83c0c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:02.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.262+0000 7fc842010640 1 --2- 172.21.15.5:0/3043711767 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc83c0ae8f0 0x7fc83c0aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:02.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.262+0000 7fc84100e640 1 -- 172.21.15.5:0/3043711767 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc83c0ae8f0 msgr2=0x7fc83c0aed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:02.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.262+0000 7fc84100e640 1 --2- 172.21.15.5:0/3043711767 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc83c0ae8f0 0x7fc83c0aed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.262+0000 7fc84100e640 1 -- 172.21.15.5:0/3043711767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc83c0b0020 msgr2=0x7fc83c0c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:02.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.262+0000 7fc84100e640 1 --2- 172.21.15.5:0/3043711767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc83c0b0020 0x7fc83c0c57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.262+0000 7fc84100e640 1 -- 172.21.15.5:0/3043711767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83c0af240 con 0x7fc83c15ca00 2024-05-20T18:38:02.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.262+0000 7fc84180f640 1 --2- 172.21.15.5:0/3043711767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc83c0b0020 0x7fc83c0c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:38:02.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.262+0000 7fc842010640 1 --2- 172.21.15.5:0/3043711767 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc83c0ae8f0 0x7fc83c0aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:38:02.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.262+0000 7fc84100e640 1 --2- 172.21.15.5:0/3043711767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc83c15ca00 0x7fc83c14c190 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fc83000c3f0 tx=0x7fc83000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:02.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.262+0000 7fc83a7fc640 1 -- 172.21.15.5:0/3043711767 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8300052d0 con 0x7fc83c15ca00 2024-05-20T18:38:02.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.262+0000 7fc83a7fc640 1 -- 172.21.15.5:0/3043711767 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc830005490 con 0x7fc83c15ca00 2024-05-20T18:38:02.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.262+0000 7fc843a73640 1 -- 172.21.15.5:0/3043711767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc83c0c5d00 con 0x7fc83c15ca00 2024-05-20T18:38:02.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.262+0000 7fc83a7fc640 1 -- 172.21.15.5:0/3043711767 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc830007260 con 0x7fc83c15ca00 2024-05-20T18:38:02.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.262+0000 7fc843a73640 1 -- 172.21.15.5:0/3043711767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc83c09c310 con 0x7fc83c15ca00 2024-05-20T18:38:02.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.263+0000 7fc843a73640 1 -- 172.21.15.5:0/3043711767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc83c075b00 con 0x7fc83c15ca00 2024-05-20T18:38:02.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.264+0000 7fc83a7fc640 1 -- 172.21.15.5:0/3043711767 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc830007400 con 0x7fc83c15ca00 2024-05-20T18:38:02.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.264+0000 7fc83a7fc640 1 --2- 172.21.15.5:0/3043711767 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc82806be30 0x7fc82806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:02.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.264+0000 7fc83a7fc640 1 -- 172.21.15.5:0/3043711767 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7fc83008a270 con 0x7fc83c15ca00 2024-05-20T18:38:02.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.264+0000 7fc84180f640 1 --2- 172.21.15.5:0/3043711767 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc82806be30 0x7fc82806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:02.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.265+0000 7fc84180f640 1 --2- 172.21.15.5:0/3043711767 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc82806be30 0x7fc82806e340 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc82c034000 tx=0x7fc82c00a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:02.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.269+0000 7fc83a7fc640 1 -- 172.21.15.5:0/3043711767 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc83c0af240 con 0x7fc83c15ca00 2024-05-20T18:38:02.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.456+0000 7fc843a73640 1 -- 172.21.15.5:0/3043711767 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fc83c0c5ed0 con 0x7fc82806be30 2024-05-20T18:38:02.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.457+0000 7fc83a7fc640 1 -- 172.21.15.5:0/3043711767 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc83c0c5ed0 con 0x7fc82806be30 2024-05-20T18:38:02.458 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 1 to deep-scrub 2024-05-20T18:38:02.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.458+0000 7fc843a73640 1 -- 172.21.15.5:0/3043711767 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc82806be30 msgr2=0x7fc82806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:02.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.458+0000 7fc843a73640 1 --2- 172.21.15.5:0/3043711767 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc82806be30 0x7fc82806e340 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc82c034000 tx=0x7fc82c00a000 comp rx=0 tx=0).stop 2024-05-20T18:38:02.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.458+0000 7fc843a73640 1 -- 172.21.15.5:0/3043711767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc83c15ca00 msgr2=0x7fc83c14c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:02.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.458+0000 7fc843a73640 1 --2- 172.21.15.5:0/3043711767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc83c15ca00 0x7fc83c14c190 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fc83000c3f0 tx=0x7fc83000c8b0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.459+0000 7fc843a73640 1 -- 172.21.15.5:0/3043711767 shutdown_connections 2024-05-20T18:38:02.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.459+0000 7fc843a73640 1 --2- 172.21.15.5:0/3043711767 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fc82806be30 0x7fc82806e340 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.459+0000 7fc843a73640 1 --2- 172.21.15.5:0/3043711767 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc83c0ae8f0 0x7fc83c0aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.459+0000 7fc843a73640 1 --2- 172.21.15.5:0/3043711767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc83c15ca00 0x7fc83c14c190 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.459+0000 7fc843a73640 1 --2- 172.21.15.5:0/3043711767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc83c0b0020 0x7fc83c0c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.459+0000 7fc843a73640 1 -- 172.21.15.5:0/3043711767 >> 172.21.15.5:0/3043711767 conn(0x7fc83c0cd2a0 msgr2=0x7fc83c15b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:02.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.459+0000 7fc843a73640 1 -- 172.21.15.5:0/3043711767 shutdown_connections 2024-05-20T18:38:02.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.459+0000 7fc843a73640 1 -- 172.21.15.5:0/3043711767 wait complete. 2024-05-20T18:38:02.471 INFO:tasks.ceph:Scrubbing osd.2 2024-05-20T18:38:02.471 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-20T18:38:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.588+0000 7f750cc2b640 1 Processor -- start 2024-05-20T18:38:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.588+0000 7f750cc2b640 1 -- start start 2024-05-20T18:38:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.590+0000 7f750cc2b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f75080b3860 0x7f75080d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.590+0000 7f750cc2b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f750808bd20 0x7f750808c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.590+0000 7f750cc2b640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f750808c670 con 0x7f750806d870 2024-05-20T18:38:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.590+0000 7f750cc2b640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f75080d0450 con 0x7f75080b3860 2024-05-20T18:38:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.590+0000 7f750cc2b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f75080d3b10 con 0x7f750808bd20 2024-05-20T18:38:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.590+0000 7f750659c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f75080b3860 0x7f75080d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.590+0000 7f7505d9b640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f750806d870 0x7f750815f230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43212/0 (socket says 172.21.15.5:43212) 2024-05-20T18:38:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.590+0000 7f7505d9b640 1 -- 172.21.15.5:0/3033786142 learned_addr learned my addr 172.21.15.5:0/3033786142 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:38:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.590+0000 7f7506d9d640 1 --2- 172.21.15.5:0/3033786142 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f750808bd20 0x7f750808c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.590+0000 7f750659c640 1 -- 172.21.15.5:0/3033786142 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f750808bd20 msgr2=0x7f750808c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:38:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.590+0000 7f750659c640 1 --2- 172.21.15.5:0/3033786142 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f750808bd20 0x7f750808c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.590+0000 7f750659c640 1 -- 172.21.15.5:0/3033786142 >> v1:172.21.15.5:6789/0 conn(0x7f750806d870 legacy=0x7f750815f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:02.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.590+0000 7f750559a640 1 -- 172.21.15.5:0/3033786142 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2035864249 0 0) 0x7f74f0001110 con 0x7f750806d870 2024-05-20T18:38:02.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.590+0000 7f750659c640 1 -- 172.21.15.5:0/3033786142 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75080d3df0 con 0x7f75080b3860 2024-05-20T18:38:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.591+0000 7f750659c640 1 --2- 172.21.15.5:0/3033786142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75080b3860 0x7f75080d45f0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f7500009d90 tx=0x7f750002a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f053e6a267499b20 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.591+0000 7f750559a640 1 -- 172.21.15.5:0/3033786142 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7500007740 con 0x7f75080b3860 2024-05-20T18:38:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.591+0000 7f750559a640 1 -- 172.21.15.5:0/3033786142 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75000078e0 con 0x7f75080b3860 2024-05-20T18:38:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.591+0000 7f750559a640 1 -- 172.21.15.5:0/3033786142 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7500007cb0 con 0x7f75080b3860 2024-05-20T18:38:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.592+0000 7f750cc2b640 1 -- 172.21.15.5:0/3033786142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75080b3860 msgr2=0x7f75080d45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.592+0000 7f750cc2b640 1 --2- 172.21.15.5:0/3033786142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75080b3860 0x7f75080d45f0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f7500009d90 tx=0x7f750002a530 comp rx=0 tx=0).stop 2024-05-20T18:38:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.592+0000 7f750cc2b640 1 -- 172.21.15.5:0/3033786142 shutdown_connections 2024-05-20T18:38:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.592+0000 7f750cc2b640 1 --2- 172.21.15.5:0/3033786142 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f750808bd20 0x7f750808c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.592+0000 7f750cc2b640 1 --2- 172.21.15.5:0/3033786142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75080b3860 0x7f75080d45f0 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.592+0000 7f750cc2b640 1 -- 172.21.15.5:0/3033786142 >> 172.21.15.5:0/3033786142 conn(0x7f75080b6d10 msgr2=0x7f750806d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.592+0000 7f750cc2b640 1 -- 172.21.15.5:0/3033786142 shutdown_connections 2024-05-20T18:38:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.592+0000 7f750cc2b640 1 -- 172.21.15.5:0/3033786142 wait complete. 2024-05-20T18:38:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.594+0000 7f750cc2b640 1 Processor -- start 2024-05-20T18:38:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.594+0000 7f750cc2b640 1 -- start start 2024-05-20T18:38:02.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.595+0000 7f750cc2b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f750806d870 0x7f75080c7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:02.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.595+0000 7f750cc2b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f750808bd20 0x7f75081571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:02.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.595+0000 7f750cc2b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75080b3860 0x7f75081576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:02.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.596+0000 7f750cc2b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f75080b81b0 con 0x7f750806d870 2024-05-20T18:38:02.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.596+0000 7f750cc2b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f75080d0730 con 0x7f75080b3860 2024-05-20T18:38:02.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.596+0000 7f750cc2b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f75080b8330 con 0x7f750808bd20 2024-05-20T18:38:02.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.596+0000 7f750659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f750806d870 0x7f75080c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.596+0000 7f750659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f750806d870 0x7f75080c7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60792/0 (socket says 172.21.15.5:60792) 2024-05-20T18:38:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.596+0000 7f750659c640 1 -- 172.21.15.5:0/4214773812 learned_addr learned my addr 172.21.15.5:0/4214773812 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:38:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.596+0000 7f7506d9d640 1 --2- 172.21.15.5:0/4214773812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75080b3860 0x7f75081576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.596+0000 7f7505d9b640 1 --2- 172.21.15.5:0/4214773812 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f750808bd20 0x7f75081571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.596+0000 7f750659c640 1 -- 172.21.15.5:0/4214773812 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f750808bd20 msgr2=0x7f75081571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.596+0000 7f750659c640 1 --2- 172.21.15.5:0/4214773812 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f750808bd20 0x7f75081571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.596+0000 7f750659c640 1 -- 172.21.15.5:0/4214773812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75080b3860 msgr2=0x7f75081576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.596+0000 7f750659c640 1 --2- 172.21.15.5:0/4214773812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75080b3860 0x7f75081576e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.596+0000 7f750659c640 1 -- 172.21.15.5:0/4214773812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75081533d0 con 0x7f750806d870 2024-05-20T18:38:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.596+0000 7f750659c640 1 --2- 172.21.15.5:0/4214773812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f750806d870 0x7f75080c7620 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f7500009d90 tx=0x7f7500004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.596+0000 7f74feffd640 1 -- 172.21.15.5:0/4214773812 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750003f070 con 0x7f750806d870 2024-05-20T18:38:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.596+0000 7f74feffd640 1 -- 172.21.15.5:0/4214773812 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7500031d00 con 0x7f750806d870 2024-05-20T18:38:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.596+0000 7f750cc2b640 1 -- 172.21.15.5:0/4214773812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75081536b0 con 0x7f750806d870 2024-05-20T18:38:02.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.596+0000 7f74feffd640 1 -- 172.21.15.5:0/4214773812 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7500044680 con 0x7f750806d870 2024-05-20T18:38:02.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.596+0000 7f750cc2b640 1 -- 172.21.15.5:0/4214773812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7508153c70 con 0x7f750806d870 2024-05-20T18:38:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.597+0000 7f750cc2b640 1 -- 172.21.15.5:0/4214773812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f75080690b0 con 0x7f750806d870 2024-05-20T18:38:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.598+0000 7f74feffd640 1 -- 172.21.15.5:0/4214773812 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f750003b410 con 0x7f750806d870 2024-05-20T18:38:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.598+0000 7f74feffd640 1 --2- 172.21.15.5:0/4214773812 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f74e806be30 0x7f74e806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.599+0000 7f74feffd640 1 -- 172.21.15.5:0/4214773812 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f75000ac860 con 0x7f750806d870 2024-05-20T18:38:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.599+0000 7f7505d9b640 1 --2- 172.21.15.5:0/4214773812 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f74e806be30 0x7f74e806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:02.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.599+0000 7f74feffd640 1 --2- 172.21.15.5:0/4214773812 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x7f74e8072460 0x7f74e80748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:02.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.599+0000 7f74feffd640 1 -- 172.21.15.5:0/4214773812 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f74e8074fc0 con 0x7f74e8072460 2024-05-20T18:38:02.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.599+0000 7f74feffd640 1 -- 172.21.15.5:0/4214773812 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=124) ==== 24+0+0 (secure 0 0 0) 0x7f75000b12c0 con 0x7f750806d870 2024-05-20T18:38:02.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.599+0000 7f7506d9d640 1 --2- 172.21.15.5:0/4214773812 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x7f74e8072460 0x7f74e80748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:02.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.599+0000 7f7505d9b640 1 --2- 172.21.15.5:0/4214773812 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f74e806be30 0x7f74e806e340 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f75081e9cc0 tx=0x7f74f0007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:02.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.599+0000 7f7506d9d640 1 --2- 172.21.15.5:0/4214773812 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x7f74e8072460 0x7f74e80748f0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:02.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.600+0000 7f74feffd640 1 -- 172.21.15.5:0/4214773812 <== osd.2 v2:172.21.15.5:6800/921679764 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f74e8074fc0 con 0x7f74e8072460 2024-05-20T18:38:02.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.613+0000 7f750cc2b640 1 -- 172.21.15.5:0/4214773812 --> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f75080b3640 con 0x7f74e8072460 2024-05-20T18:38:02.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.613+0000 7f74feffd640 1 -- 172.21.15.5:0/4214773812 <== osd.2 v2:172.21.15.5:6800/921679764 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f75080b3640 con 0x7f74e8072460 2024-05-20T18:38:02.615 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:38:02.615 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-20T18:38:02.615 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:38:02.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.614+0000 7f750cc2b640 1 -- 172.21.15.5:0/4214773812 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x7f74e8072460 msgr2=0x7f74e80748f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:02.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.614+0000 7f750cc2b640 1 --2- 172.21.15.5:0/4214773812 >> [v2:172.21.15.5:6800/921679764,v1:172.21.15.5:6801/921679764] conn(0x7f74e8072460 0x7f74e80748f0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.614+0000 7f750cc2b640 1 -- 172.21.15.5:0/4214773812 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f74e806be30 msgr2=0x7f74e806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:02.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.614+0000 7f750cc2b640 1 --2- 172.21.15.5:0/4214773812 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f74e806be30 0x7f74e806e340 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f75081e9cc0 tx=0x7f74f0007c00 comp rx=0 tx=0).stop 2024-05-20T18:38:02.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.614+0000 7f750cc2b640 1 -- 172.21.15.5:0/4214773812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f750806d870 msgr2=0x7f75080c7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:02.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.614+0000 7f750cc2b640 1 --2- 172.21.15.5:0/4214773812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f750806d870 0x7f75080c7620 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f7500009d90 tx=0x7f7500004a40 comp rx=0 tx=0).stop 2024-05-20T18:38:02.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.614+0000 7f7506d9d640 1 -- 172.21.15.5:0/4214773812 reap_dead start 2024-05-20T18:38:02.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.614+0000 7f750cc2b640 1 -- 172.21.15.5:0/4214773812 shutdown_connections 2024-05-20T18:38:02.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.614+0000 7f750cc2b640 1 -- 172.21.15.5:0/4214773812 >> 172.21.15.5:0/4214773812 conn(0x7f75080b6d10 msgr2=0x7f75080b5d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:02.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.614+0000 7f750cc2b640 1 -- 172.21.15.5:0/4214773812 shutdown_connections 2024-05-20T18:38:02.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.614+0000 7f750cc2b640 1 -- 172.21.15.5:0/4214773812 wait complete. 2024-05-20T18:38:02.626 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-20T18:38:02.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.740+0000 7f1d24661640 1 Processor -- start 2024-05-20T18:38:02.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.740+0000 7f1d24661640 1 -- start start 2024-05-20T18:38:02.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.742+0000 7f1d24661640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d1c0d10e0 0x7f1d1c0cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:02.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.742+0000 7f1d24661640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1d1c0d14f0 0x7f1d1c0cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:02.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.742+0000 7f1d24661640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d1c0d1ac0 con 0x7f1d1c0cd4d0 2024-05-20T18:38:02.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.742+0000 7f1d24661640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f1d1c0d1da0 con 0x7f1d1c0d14f0 2024-05-20T18:38:02.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.742+0000 7f1d24661640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1d1c0d2080 con 0x7f1d1c0d10e0 2024-05-20T18:38:02.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.742+0000 7f1d21bfc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1d1c0d14f0 0x7f1d1c0cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:02.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.742+0000 7f1d21bfc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1d1c0d14f0 0x7f1d1c0cf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60806/0 (socket says 172.21.15.5:60806) 2024-05-20T18:38:02.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.742+0000 7f1d21bfc640 1 -- 172.21.15.5:0/541206492 learned_addr learned my addr 172.21.15.5:0/541206492 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:38:02.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.742+0000 7f1d223fd640 1 --2- 172.21.15.5:0/541206492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d1c0d10e0 0x7f1d1c0cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:02.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.742+0000 7f1d21bfc640 1 -- 172.21.15.5:0/541206492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d1c0d10e0 msgr2=0x7f1d1c0cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:38:02.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.742+0000 7f1d21bfc640 1 --2- 172.21.15.5:0/541206492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d1c0d10e0 0x7f1d1c0cf140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.742+0000 7f1d21bfc640 1 -- 172.21.15.5:0/541206492 >> v1:172.21.15.63:6789/0 conn(0x7f1d1c0cd4d0 legacy=0x7f1d1c0cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:38:02.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.742+0000 7f1d21bfc640 1 -- 172.21.15.5:0/541206492 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d1c0d2360 con 0x7f1d1c0d14f0 2024-05-20T18:38:02.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.743+0000 7f1d21bfc640 1 --2- 172.21.15.5:0/541206492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d1c0d14f0 0x7f1d1c0cf680 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c009870 tx=0x7f1d0c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d57ffdbc39c5ac server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:02.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.743+0000 7f1d213fb640 1 -- 172.21.15.5:0/541206492 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c037070 con 0x7f1d1c0d14f0 2024-05-20T18:38:02.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.743+0000 7f1d213fb640 1 -- 172.21.15.5:0/541206492 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d0c032440 con 0x7f1d1c0d14f0 2024-05-20T18:38:02.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.743+0000 7f1d213fb640 1 -- 172.21.15.5:0/541206492 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c032810 con 0x7f1d1c0d14f0 2024-05-20T18:38:02.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.744+0000 7f1d24661640 1 -- 172.21.15.5:0/541206492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d1c0d14f0 msgr2=0x7f1d1c0cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:02.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.744+0000 7f1d24661640 1 --2- 172.21.15.5:0/541206492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d1c0d14f0 0x7f1d1c0cf680 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c009870 tx=0x7f1d0c02a210 comp rx=0 tx=0).stop 2024-05-20T18:38:02.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.744+0000 7f1d24661640 1 -- 172.21.15.5:0/541206492 shutdown_connections 2024-05-20T18:38:02.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.744+0000 7f1d24661640 1 --2- 172.21.15.5:0/541206492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d1c0d14f0 0x7f1d1c0cf680 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.744+0000 7f1d24661640 1 --2- 172.21.15.5:0/541206492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d1c0d10e0 0x7f1d1c0cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.744+0000 7f1d24661640 1 -- 172.21.15.5:0/541206492 >> 172.21.15.5:0/541206492 conn(0x7f1d1c0b44d0 msgr2=0x7f1d1c0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:02.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.744+0000 7f1d24661640 1 -- 172.21.15.5:0/541206492 shutdown_connections 2024-05-20T18:38:02.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.744+0000 7f1d24661640 1 -- 172.21.15.5:0/541206492 wait complete. 2024-05-20T18:38:02.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.746+0000 7f1d24661640 1 Processor -- start 2024-05-20T18:38:02.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.746+0000 7f1d24661640 1 -- start start 2024-05-20T18:38:02.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.747+0000 7f1d24661640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d1c0cd4d0 0x7f1d1c1f3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:02.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.747+0000 7f1d24661640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d1c0d10e0 0x7f1d1c0a96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:02.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.747+0000 7f1d24661640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d1c0d14f0 0x7f1d1c1507f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:02.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.747+0000 7f1d24661640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c0b8440 con 0x7f1d1c0cd4d0 2024-05-20T18:38:02.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.747+0000 7f1d24661640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c0b82c0 con 0x7f1d1c0d10e0 2024-05-20T18:38:02.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.747+0000 7f1d24661640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1d1c0709f0 con 0x7f1d1c0d14f0 2024-05-20T18:38:02.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.747+0000 7f1d223fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d1c0cd4d0 0x7f1d1c1f3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:02.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.747+0000 7f1d223fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d1c0cd4d0 0x7f1d1c1f3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60812/0 (socket says 172.21.15.5:60812) 2024-05-20T18:38:02.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.747+0000 7f1d223fd640 1 -- 172.21.15.5:0/4259914067 learned_addr learned my addr 172.21.15.5:0/4259914067 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:38:02.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.747+0000 7f1d22bfe640 1 --2- 172.21.15.5:0/4259914067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d1c0d14f0 0x7f1d1c1507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:02.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.747+0000 7f1d21bfc640 1 --2- 172.21.15.5:0/4259914067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d1c0d10e0 0x7f1d1c0a96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:02.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.748+0000 7f1d223fd640 1 -- 172.21.15.5:0/4259914067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d1c0d14f0 msgr2=0x7f1d1c1507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:02.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.748+0000 7f1d223fd640 1 --2- 172.21.15.5:0/4259914067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d1c0d14f0 0x7f1d1c1507f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.748+0000 7f1d223fd640 1 -- 172.21.15.5:0/4259914067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d1c0d10e0 msgr2=0x7f1d1c0a96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:02.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.748+0000 7f1d223fd640 1 --2- 172.21.15.5:0/4259914067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d1c0d10e0 0x7f1d1c0a96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.748+0000 7f1d223fd640 1 -- 172.21.15.5:0/4259914067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d1c0aa370 con 0x7f1d1c0cd4d0 2024-05-20T18:38:02.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.748+0000 7f1d223fd640 1 --2- 172.21.15.5:0/4259914067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d1c0cd4d0 0x7f1d1c1f3110 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800c820 tx=0x7f1d1800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:02.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.748+0000 7f1d12ffd640 1 -- 172.21.15.5:0/4259914067 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d18015070 con 0x7f1d1c0cd4d0 2024-05-20T18:38:02.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.748+0000 7f1d12ffd640 1 -- 172.21.15.5:0/4259914067 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d18004020 con 0x7f1d1c0cd4d0 2024-05-20T18:38:02.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.748+0000 7f1d12ffd640 1 -- 172.21.15.5:0/4259914067 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d180043f0 con 0x7f1d1c0cd4d0 2024-05-20T18:38:02.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.748+0000 7f1d24661640 1 -- 172.21.15.5:0/4259914067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d1c0aa650 con 0x7f1d1c0cd4d0 2024-05-20T18:38:02.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.748+0000 7f1d24661640 1 -- 172.21.15.5:0/4259914067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d1c0aac10 con 0x7f1d1c0cd4d0 2024-05-20T18:38:02.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.749+0000 7f1d24661640 1 -- 172.21.15.5:0/4259914067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d1c075b00 con 0x7f1d1c0cd4d0 2024-05-20T18:38:02.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.750+0000 7f1d12ffd640 1 -- 172.21.15.5:0/4259914067 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1d18007830 con 0x7f1d1c0cd4d0 2024-05-20T18:38:02.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.750+0000 7f1d12ffd640 1 --2- 172.21.15.5:0/4259914067 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1d0006be30 0x7f1d0006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:02.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.750+0000 7f1d12ffd640 1 -- 172.21.15.5:0/4259914067 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f1d1808abb0 con 0x7f1d1c0cd4d0 2024-05-20T18:38:02.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.750+0000 7f1d21bfc640 1 --2- 172.21.15.5:0/4259914067 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1d0006be30 0x7f1d0006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:02.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.751+0000 7f1d21bfc640 1 --2- 172.21.15.5:0/4259914067 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1d0006be30 0x7f1d0006e340 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c034000 tx=0x7f1d0c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:02.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.755+0000 7f1d12ffd640 1 -- 172.21.15.5:0/4259914067 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1d18058f40 con 0x7f1d1c0cd4d0 2024-05-20T18:38:02.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.940+0000 7f1d24661640 1 -- 172.21.15.5:0/4259914067 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f1d1c0b3b40 con 0x7f1d0006be30 2024-05-20T18:38:02.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.941+0000 7f1d12ffd640 1 -- 172.21.15.5:0/4259914067 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1d1c0b3b40 con 0x7f1d0006be30 2024-05-20T18:38:02.942 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 2 to deep-scrub 2024-05-20T18:38:02.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.943+0000 7f1d24661640 1 -- 172.21.15.5:0/4259914067 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1d0006be30 msgr2=0x7f1d0006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:02.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.943+0000 7f1d24661640 1 --2- 172.21.15.5:0/4259914067 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1d0006be30 0x7f1d0006e340 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c034000 tx=0x7f1d0c02e000 comp rx=0 tx=0).stop 2024-05-20T18:38:02.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.943+0000 7f1d24661640 1 -- 172.21.15.5:0/4259914067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d1c0cd4d0 msgr2=0x7f1d1c1f3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:02.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.943+0000 7f1d24661640 1 --2- 172.21.15.5:0/4259914067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d1c0cd4d0 0x7f1d1c1f3110 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800c820 tx=0x7f1d1800cce0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.943+0000 7f1d24661640 1 -- 172.21.15.5:0/4259914067 shutdown_connections 2024-05-20T18:38:02.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.943+0000 7f1d24661640 1 --2- 172.21.15.5:0/4259914067 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f1d0006be30 0x7f1d0006e340 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.943+0000 7f1d24661640 1 --2- 172.21.15.5:0/4259914067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d1c0d14f0 0x7f1d1c1507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.943+0000 7f1d24661640 1 --2- 172.21.15.5:0/4259914067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d1c0d10e0 0x7f1d1c0a96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.943+0000 7f1d24661640 1 --2- 172.21.15.5:0/4259914067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d1c0cd4d0 0x7f1d1c1f3110 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:02.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.943+0000 7f1d24661640 1 -- 172.21.15.5:0/4259914067 >> 172.21.15.5:0/4259914067 conn(0x7f1d1c0b44d0 msgr2=0x7f1d1c0b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:02.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.943+0000 7f1d24661640 1 -- 172.21.15.5:0/4259914067 shutdown_connections 2024-05-20T18:38:02.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:02.943+0000 7f1d24661640 1 -- 172.21.15.5:0/4259914067 wait complete. 2024-05-20T18:38:02.955 INFO:tasks.ceph:Scrubbing osd.3 2024-05-20T18:38:02.955 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-20T18:38:03.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.073+0000 7ff02b099640 1 Processor -- start 2024-05-20T18:38:03.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.073+0000 7ff02b099640 1 -- start start 2024-05-20T18:38:03.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.074+0000 7ff02b099640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0240d45f0 0x7ff02406d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:03.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.074+0000 7ff02b099640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0240d0450 con 0x7ff0240b3860 2024-05-20T18:38:03.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.074+0000 7ff02b099640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0240d3b10 con 0x7ff02408bd20 2024-05-20T18:38:03.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.074+0000 7ff02b099640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff0240d3df0 con 0x7ff0240d45f0 2024-05-20T18:38:03.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.075+0000 7ff028e35640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff0240b3860 0x7ff02415f230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43218/0 (socket says 172.21.15.5:43218) 2024-05-20T18:38:03.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.075+0000 7ff028e35640 1 -- 172.21.15.5:0/205640487 learned_addr learned my addr 172.21.15.5:0/205640487 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:38:03.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.075+0000 7ff023fff640 1 --2- 172.21.15.5:0/205640487 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0240d45f0 0x7ff02406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:03.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.075+0000 7ff0237fe640 1 -- 172.21.15.5:0/205640487 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 56935092 0 0) 0x7ff0240d0450 con 0x7ff0240b3860 2024-05-20T18:38:03.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.075+0000 7ff0237fe640 1 -- 172.21.15.5:0/205640487 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff014003520 con 0x7ff0240b3860 2024-05-20T18:38:03.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.075+0000 7ff0237fe640 1 -- 172.21.15.5:0/205640487 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 667528004 0 0) 0x7ff0240d3b10 con 0x7ff02408bd20 2024-05-20T18:38:03.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.075+0000 7ff0237fe640 1 -- 172.21.15.5:0/205640487 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0240d0450 con 0x7ff02408bd20 2024-05-20T18:38:03.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.075+0000 7ff023fff640 1 -- 172.21.15.5:0/205640487 >> v1:172.21.15.63:6789/0 conn(0x7ff02408bd20 legacy=0x7ff02408c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:03.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.075+0000 7ff023fff640 1 -- 172.21.15.5:0/205640487 >> v1:172.21.15.5:6789/0 conn(0x7ff0240b3860 legacy=0x7ff02415f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:03.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.075+0000 7ff023fff640 1 -- 172.21.15.5:0/205640487 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0240d40d0 con 0x7ff0240d45f0 2024-05-20T18:38:03.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.076+0000 7ff023fff640 1 --2- 172.21.15.5:0/205640487 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0240d45f0 0x7ff02406d870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff0100097d0 tx=0x7ff01002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f58d92f46d236630 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:03.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.076+0000 7ff0237fe640 1 -- 172.21.15.5:0/205640487 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff010037070 con 0x7ff0240d45f0 2024-05-20T18:38:03.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.076+0000 7ff0237fe640 1 -- 172.21.15.5:0/205640487 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff010032440 con 0x7ff0240d45f0 2024-05-20T18:38:03.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.076+0000 7ff0237fe640 1 -- 172.21.15.5:0/205640487 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0100327f0 con 0x7ff0240d45f0 2024-05-20T18:38:03.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.077+0000 7ff02b099640 1 -- 172.21.15.5:0/205640487 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0240d45f0 msgr2=0x7ff02406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:03.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.077+0000 7ff02b099640 1 --2- 172.21.15.5:0/205640487 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0240d45f0 0x7ff02406d870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff0100097d0 tx=0x7ff01002a180 comp rx=0 tx=0).stop 2024-05-20T18:38:03.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.077+0000 7ff02b099640 1 -- 172.21.15.5:0/205640487 shutdown_connections 2024-05-20T18:38:03.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.077+0000 7ff02b099640 1 --2- 172.21.15.5:0/205640487 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0240d45f0 0x7ff02406d870 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.077+0000 7ff02b099640 1 -- 172.21.15.5:0/205640487 >> 172.21.15.5:0/205640487 conn(0x7ff0240b6d10 msgr2=0x7ff02406d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:03.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.077+0000 7ff02b099640 1 -- 172.21.15.5:0/205640487 shutdown_connections 2024-05-20T18:38:03.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.077+0000 7ff02b099640 1 -- 172.21.15.5:0/205640487 wait complete. 2024-05-20T18:38:03.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.079+0000 7ff02b099640 1 Processor -- start 2024-05-20T18:38:03.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.079+0000 7ff02b099640 1 -- start start 2024-05-20T18:38:03.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.080+0000 7ff02b099640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff02408bd20 0x7ff024156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:03.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.080+0000 7ff02b099640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0240b3860 0x7ff024157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:03.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.080+0000 7ff02b099640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0240d45f0 0x7ff024159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:03.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.080+0000 7ff02b099640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff02416a0f0 con 0x7ff0240b3860 2024-05-20T18:38:03.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.080+0000 7ff02b099640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff024169f70 con 0x7ff0240d45f0 2024-05-20T18:38:03.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.080+0000 7ff02b099640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff02416a270 con 0x7ff02408bd20 2024-05-20T18:38:03.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.080+0000 7ff023fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0240b3860 0x7ff024157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:03.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.080+0000 7ff023fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0240b3860 0x7ff024157930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60824/0 (socket says 172.21.15.5:60824) 2024-05-20T18:38:03.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.080+0000 7ff023fff640 1 -- 172.21.15.5:0/1412941014 learned_addr learned my addr 172.21.15.5:0/1412941014 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:38:03.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.080+0000 7ff028e35640 1 --2- 172.21.15.5:0/1412941014 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff02408bd20 0x7ff024156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:03.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.080+0000 7ff029636640 1 --2- 172.21.15.5:0/1412941014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0240d45f0 0x7ff024159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:03.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.080+0000 7ff023fff640 1 -- 172.21.15.5:0/1412941014 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff02408bd20 msgr2=0x7ff024156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:03.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.080+0000 7ff023fff640 1 --2- 172.21.15.5:0/1412941014 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff02408bd20 0x7ff024156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.081+0000 7ff023fff640 1 -- 172.21.15.5:0/1412941014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0240d45f0 msgr2=0x7ff024159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:03.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.081+0000 7ff023fff640 1 --2- 172.21.15.5:0/1412941014 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0240d45f0 0x7ff024159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.081+0000 7ff023fff640 1 -- 172.21.15.5:0/1412941014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff024158440 con 0x7ff0240b3860 2024-05-20T18:38:03.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.081+0000 7ff023fff640 1 --2- 172.21.15.5:0/1412941014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0240b3860 0x7ff024157930 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7ff010035040 tx=0x7ff0100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:03.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.081+0000 7ff0217fa640 1 -- 172.21.15.5:0/1412941014 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff010037050 con 0x7ff0240b3860 2024-05-20T18:38:03.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.081+0000 7ff0217fa640 1 -- 172.21.15.5:0/1412941014 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff010038040 con 0x7ff0240b3860 2024-05-20T18:38:03.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.081+0000 7ff02b099640 1 -- 172.21.15.5:0/1412941014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff024151270 con 0x7ff0240b3860 2024-05-20T18:38:03.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.081+0000 7ff0217fa640 1 -- 172.21.15.5:0/1412941014 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff010006e10 con 0x7ff0240b3860 2024-05-20T18:38:03.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.081+0000 7ff02b099640 1 -- 172.21.15.5:0/1412941014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff024155fc0 con 0x7ff0240b3860 2024-05-20T18:38:03.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.082+0000 7ff02b099640 1 -- 172.21.15.5:0/1412941014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff024067c80 con 0x7ff0240b3860 2024-05-20T18:38:03.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.083+0000 7ff0217fa640 1 -- 172.21.15.5:0/1412941014 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff010045020 con 0x7ff0240b3860 2024-05-20T18:38:03.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.083+0000 7ff0217fa640 1 --2- 172.21.15.5:0/1412941014 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff00806be30 0x7ff00806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:03.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.083+0000 7ff0217fa640 1 -- 172.21.15.5:0/1412941014 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7ff0100aad20 con 0x7ff0240b3860 2024-05-20T18:38:03.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.083+0000 7ff028e35640 1 --2- 172.21.15.5:0/1412941014 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff00806be30 0x7ff00806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:03.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.083+0000 7ff0217fa640 1 --2- 172.21.15.5:0/1412941014 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x7ff008072460 0x7ff0080748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:03.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.083+0000 7ff0217fa640 1 -- 172.21.15.5:0/1412941014 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff010035e70 con 0x7ff008072460 2024-05-20T18:38:03.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.083+0000 7ff0217fa640 1 -- 172.21.15.5:0/1412941014 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=124) ==== 24+0+0 (secure 0 0 0) 0x7ff0100ad380 con 0x7ff0240b3860 2024-05-20T18:38:03.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.083+0000 7ff029636640 1 --2- 172.21.15.5:0/1412941014 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x7ff008072460 0x7ff0080748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:03.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.084+0000 7ff028e35640 1 --2- 172.21.15.5:0/1412941014 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff00806be30 0x7ff00806e340 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff018000f60 tx=0x7ff018007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:03.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.084+0000 7ff029636640 1 --2- 172.21.15.5:0/1412941014 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x7ff008072460 0x7ff0080748f0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:03.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.085+0000 7ff0217fa640 1 -- 172.21.15.5:0/1412941014 <== osd.3 v2:172.21.15.5:6824/1164130599 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff010035e70 con 0x7ff008072460 2024-05-20T18:38:03.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.098+0000 7ff02b099640 1 -- 172.21.15.5:0/1412941014 --> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff024151000 con 0x7ff008072460 2024-05-20T18:38:03.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.098+0000 7ff0217fa640 1 -- 172.21.15.5:0/1412941014 <== osd.3 v2:172.21.15.5:6824/1164130599 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff024151000 con 0x7ff008072460 2024-05-20T18:38:03.099 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:38:03.100 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-20T18:38:03.100 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:38:03.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.098+0000 7ff02b099640 1 -- 172.21.15.5:0/1412941014 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x7ff008072460 msgr2=0x7ff0080748f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:03.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.098+0000 7ff02b099640 1 --2- 172.21.15.5:0/1412941014 >> [v2:172.21.15.5:6824/1164130599,v1:172.21.15.5:6825/1164130599] conn(0x7ff008072460 0x7ff0080748f0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.098+0000 7ff02b099640 1 -- 172.21.15.5:0/1412941014 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff00806be30 msgr2=0x7ff00806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:03.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.098+0000 7ff02b099640 1 --2- 172.21.15.5:0/1412941014 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7ff00806be30 0x7ff00806e340 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff018000f60 tx=0x7ff018007d50 comp rx=0 tx=0).stop 2024-05-20T18:38:03.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.098+0000 7ff02b099640 1 -- 172.21.15.5:0/1412941014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0240b3860 msgr2=0x7ff024157930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:03.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.098+0000 7ff02b099640 1 --2- 172.21.15.5:0/1412941014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0240b3860 0x7ff024157930 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7ff010035040 tx=0x7ff0100046c0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.098+0000 7ff029636640 1 -- 172.21.15.5:0/1412941014 reap_dead start 2024-05-20T18:38:03.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.099+0000 7ff02b099640 1 -- 172.21.15.5:0/1412941014 shutdown_connections 2024-05-20T18:38:03.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.099+0000 7ff02b099640 1 -- 172.21.15.5:0/1412941014 >> 172.21.15.5:0/1412941014 conn(0x7ff0240b6d10 msgr2=0x7ff02415e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:03.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.099+0000 7ff02b099640 1 -- 172.21.15.5:0/1412941014 shutdown_connections 2024-05-20T18:38:03.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.099+0000 7ff02b099640 1 -- 172.21.15.5:0/1412941014 wait complete. 2024-05-20T18:38:03.111 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-20T18:38:03.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.225+0000 7efdb73c3640 1 Processor -- start 2024-05-20T18:38:03.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.225+0000 7efdb73c3640 1 -- start start 2024-05-20T18:38:03.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.227+0000 7efdb73c3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efdb00d2780 0x7efdb00d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:03.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.227+0000 7efdb73c3640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdb008d2a0 con 0x7efdb006cf00 2024-05-20T18:38:03.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.227+0000 7efdb73c3640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdb00d30d0 con 0x7efdb00d4400 2024-05-20T18:38:03.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.227+0000 7efdb73c3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efdb00d3f70 con 0x7efdb00d2780 2024-05-20T18:38:03.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.227+0000 7efdb495e640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7efdb006cf00 0x7efdb006d310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43230/0 (socket says 172.21.15.5:43230) 2024-05-20T18:38:03.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.227+0000 7efdb495e640 1 -- 172.21.15.5:0/1858308099 learned_addr learned my addr 172.21.15.5:0/1858308099 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:38:03.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.227+0000 7efdb5960640 1 --2- 172.21.15.5:0/1858308099 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efdb00d2780 0x7efdb00d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:03.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.227+0000 7efda7fff640 1 -- 172.21.15.5:0/1858308099 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1728343917 0 0) 0x7efda0001110 con 0x7efdb006cf00 2024-05-20T18:38:03.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.227+0000 7efda7fff640 1 -- 172.21.15.5:0/1858308099 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd9c003520 con 0x7efdb006cf00 2024-05-20T18:38:03.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.227+0000 7efda7fff640 1 -- 172.21.15.5:0/1858308099 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3803208502 0 0) 0x7efd9c003520 con 0x7efdb006cf00 2024-05-20T18:38:03.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.227+0000 7efda7fff640 1 -- 172.21.15.5:0/1858308099 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efdb00d2780 msgr2=0x7efdb00d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:38:03.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.227+0000 7efda7fff640 1 --2- 172.21.15.5:0/1858308099 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efdb00d2780 0x7efdb00d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.227+0000 7efda7fff640 1 -- 172.21.15.5:0/1858308099 >> v1:172.21.15.63:6789/0 conn(0x7efdb00d4400 legacy=0x7efdb015f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:03.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.227+0000 7efda7fff640 1 -- 172.21.15.5:0/1858308099 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb00d4810 con 0x7efdb006cf00 2024-05-20T18:38:03.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.227+0000 7efda7fff640 1 -- 172.21.15.5:0/1858308099 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7efda0001840 con 0x7efdb006cf00 2024-05-20T18:38:03.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.227+0000 7efda7fff640 1 -- 172.21.15.5:0/1858308099 >> v1:172.21.15.5:6789/0 conn(0x7efdb006cf00 legacy=0x7efdb006d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:03.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.227+0000 7efda7fff640 1 --2- 172.21.15.5:0/1858308099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd9c0039e0 0x7efd9c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:03.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.227+0000 7efda7fff640 1 --2- 172.21.15.5:0/1858308099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd9c004800 0x7efd9c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:03.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.227+0000 7efda7fff640 1 --2- 172.21.15.5:0/1858308099 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efdb00d2780 0x7efd9c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:03.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.227+0000 7efda7fff640 1 -- 172.21.15.5:0/1858308099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efdb00d3f70 con 0x7efd9c004800 2024-05-20T18:38:03.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.227+0000 7efda7fff640 1 -- 172.21.15.5:0/1858308099 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c00bb80 con 0x7efd9c0039e0 2024-05-20T18:38:03.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.227+0000 7efda7fff640 1 -- 172.21.15.5:0/1858308099 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efd9c00bd80 con 0x7efdb00d2780 2024-05-20T18:38:03.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.228+0000 7efdb495e640 1 --2- 172.21.15.5:0/1858308099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd9c004800 0x7efd9c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:03.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.228+0000 7efdb515f640 1 --2- 172.21.15.5:0/1858308099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd9c0039e0 0x7efd9c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:03.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.228+0000 7efdb5960640 1 --2- 172.21.15.5:0/1858308099 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efdb00d2780 0x7efd9c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:03.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.228+0000 7efdb495e640 1 -- 172.21.15.5:0/1858308099 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efdb00d2780 msgr2=0x7efd9c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:38:03.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.228+0000 7efdb495e640 1 --2- 172.21.15.5:0/1858308099 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efdb00d2780 0x7efd9c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.228+0000 7efdb495e640 1 -- 172.21.15.5:0/1858308099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd9c0039e0 msgr2=0x7efd9c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:38:03.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.228+0000 7efdb495e640 1 --2- 172.21.15.5:0/1858308099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd9c0039e0 0x7efd9c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.228+0000 7efdb495e640 1 -- 172.21.15.5:0/1858308099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd9c003520 con 0x7efd9c004800 2024-05-20T18:38:03.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.229+0000 7efdb495e640 1 --2- 172.21.15.5:0/1858308099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd9c004800 0x7efd9c004c10 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7efda000c490 tx=0x7efd9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f48d915b225120b server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:03.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.229+0000 7efda7fff640 1 -- 172.21.15.5:0/1858308099 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda000c0e0 con 0x7efd9c004800 2024-05-20T18:38:03.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.229+0000 7efda7fff640 1 -- 172.21.15.5:0/1858308099 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efda0034070 con 0x7efd9c004800 2024-05-20T18:38:03.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.229+0000 7efda7fff640 1 -- 172.21.15.5:0/1858308099 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda0005d30 con 0x7efd9c004800 2024-05-20T18:38:03.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.229+0000 7efdb73c3640 1 -- 172.21.15.5:0/1858308099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd9c004800 msgr2=0x7efd9c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:03.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.229+0000 7efdb73c3640 1 --2- 172.21.15.5:0/1858308099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd9c004800 0x7efd9c004c10 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7efda000c490 tx=0x7efd9c005830 comp rx=0 tx=0).stop 2024-05-20T18:38:03.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.229+0000 7efdb5960640 1 -- 172.21.15.5:0/1858308099 reap_dead start 2024-05-20T18:38:03.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.229+0000 7efdb73c3640 1 -- 172.21.15.5:0/1858308099 shutdown_connections 2024-05-20T18:38:03.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.229+0000 7efdb73c3640 1 -- 172.21.15.5:0/1858308099 >> 172.21.15.5:0/1858308099 conn(0x7efdb00d4bb0 msgr2=0x7efdb00d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:03.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.230+0000 7efdb73c3640 1 -- 172.21.15.5:0/1858308099 shutdown_connections 2024-05-20T18:38:03.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.230+0000 7efdb73c3640 1 -- 172.21.15.5:0/1858308099 wait complete. 2024-05-20T18:38:03.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.231+0000 7efdb73c3640 1 Processor -- start 2024-05-20T18:38:03.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.231+0000 7efdb73c3640 1 -- start start 2024-05-20T18:38:03.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.233+0000 7efdb73c3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd9c004800 0x7efdb014c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:03.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.233+0000 7efdb73c3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efdb01621c0 0x7efdb01625d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:03.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.233+0000 7efdb73c3640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efdb01f50c0 0x7efdb01f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:03.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.233+0000 7efdb73c3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efdb00d3920 con 0x7efd9c004800 2024-05-20T18:38:03.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.233+0000 7efdb73c3640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efdb00d3640 con 0x7efdb01f50c0 2024-05-20T18:38:03.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.233+0000 7efdb73c3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efdb00d3c00 con 0x7efdb01621c0 2024-05-20T18:38:03.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.233+0000 7efdb515f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd9c004800 0x7efdb014c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:03.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.233+0000 7efdb515f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd9c004800 0x7efdb014c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60832/0 (socket says 172.21.15.5:60832) 2024-05-20T18:38:03.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.233+0000 7efdb515f640 1 -- 172.21.15.5:0/3479046028 learned_addr learned my addr 172.21.15.5:0/3479046028 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:38:03.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.233+0000 7efdb495e640 1 --2- 172.21.15.5:0/3479046028 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efdb01621c0 0x7efdb01625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:03.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.233+0000 7efdb5960640 1 --2- 172.21.15.5:0/3479046028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efdb01f50c0 0x7efdb01f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:03.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.233+0000 7efdb515f640 1 -- 172.21.15.5:0/3479046028 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efdb01621c0 msgr2=0x7efdb01625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:03.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.233+0000 7efdb515f640 1 --2- 172.21.15.5:0/3479046028 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efdb01621c0 0x7efdb01625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.233+0000 7efdb515f640 1 -- 172.21.15.5:0/3479046028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efdb01f50c0 msgr2=0x7efdb01f54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:03.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.233+0000 7efdb515f640 1 --2- 172.21.15.5:0/3479046028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efdb01f50c0 0x7efdb01f54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.233+0000 7efdb515f640 1 -- 172.21.15.5:0/3479046028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb014c9c0 con 0x7efd9c004800 2024-05-20T18:38:03.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.233+0000 7efdb5960640 1 --2- 172.21.15.5:0/3479046028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efdb01f50c0 0x7efdb01f54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:38:03.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.233+0000 7efdb515f640 1 --2- 172.21.15.5:0/3479046028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd9c004800 0x7efdb014c480 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7efdac00f620 tx=0x7efdac00fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:03.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.233+0000 7efda5ffb640 1 -- 172.21.15.5:0/3479046028 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac0052e0 con 0x7efd9c004800 2024-05-20T18:38:03.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.234+0000 7efda5ffb640 1 -- 172.21.15.5:0/3479046028 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdac018070 con 0x7efd9c004800 2024-05-20T18:38:03.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.234+0000 7efdb73c3640 1 -- 172.21.15.5:0/3479046028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdb01f95f0 con 0x7efd9c004800 2024-05-20T18:38:03.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.234+0000 7efda5ffb640 1 -- 172.21.15.5:0/3479046028 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac003810 con 0x7efd9c004800 2024-05-20T18:38:03.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.234+0000 7efdb73c3640 1 -- 172.21.15.5:0/3479046028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdb01f9bb0 con 0x7efd9c004800 2024-05-20T18:38:03.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.235+0000 7efdb73c3640 1 -- 172.21.15.5:0/3479046028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdb00735a0 con 0x7efd9c004800 2024-05-20T18:38:03.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.235+0000 7efda5ffb640 1 -- 172.21.15.5:0/3479046028 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efdac0039b0 con 0x7efd9c004800 2024-05-20T18:38:03.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.236+0000 7efda5ffb640 1 --2- 172.21.15.5:0/3479046028 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efd9406be30 0x7efd9406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:03.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.236+0000 7efda5ffb640 1 -- 172.21.15.5:0/3479046028 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7efdac08ab60 con 0x7efd9c004800 2024-05-20T18:38:03.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.236+0000 7efdb495e640 1 --2- 172.21.15.5:0/3479046028 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efd9406be30 0x7efd9406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:03.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.236+0000 7efdb495e640 1 --2- 172.21.15.5:0/3479046028 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efd9406be30 0x7efd9406e340 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7efd9c004780 tx=0x7efda002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:03.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.240+0000 7efda5ffb640 1 -- 172.21.15.5:0/3479046028 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7efdb00735a0 con 0x7efd9c004800 2024-05-20T18:38:03.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.425+0000 7efdb73c3640 1 -- 172.21.15.5:0/3479046028 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7efdb0075b00 con 0x7efd9406be30 2024-05-20T18:38:03.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.425+0000 7efda5ffb640 1 -- 172.21.15.5:0/3479046028 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7efdb0075b00 con 0x7efd9406be30 2024-05-20T18:38:03.426 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 3 to deep-scrub 2024-05-20T18:38:03.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.427+0000 7efdb73c3640 1 -- 172.21.15.5:0/3479046028 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efd9406be30 msgr2=0x7efd9406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:03.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.427+0000 7efdb73c3640 1 --2- 172.21.15.5:0/3479046028 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efd9406be30 0x7efd9406e340 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7efd9c004780 tx=0x7efda002e000 comp rx=0 tx=0).stop 2024-05-20T18:38:03.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.427+0000 7efdb73c3640 1 -- 172.21.15.5:0/3479046028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd9c004800 msgr2=0x7efdb014c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:03.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.427+0000 7efdb73c3640 1 --2- 172.21.15.5:0/3479046028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd9c004800 0x7efdb014c480 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7efdac00f620 tx=0x7efdac00fae0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.427+0000 7efdb73c3640 1 -- 172.21.15.5:0/3479046028 shutdown_connections 2024-05-20T18:38:03.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.427+0000 7efdb73c3640 1 --2- 172.21.15.5:0/3479046028 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efd9406be30 0x7efd9406e340 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.427+0000 7efdb73c3640 1 --2- 172.21.15.5:0/3479046028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efdb01f50c0 0x7efdb01f54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.427+0000 7efdb73c3640 1 --2- 172.21.15.5:0/3479046028 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efdb01621c0 0x7efdb01625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.427+0000 7efdb73c3640 1 --2- 172.21.15.5:0/3479046028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd9c004800 0x7efdb014c480 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.427+0000 7efdb73c3640 1 -- 172.21.15.5:0/3479046028 >> 172.21.15.5:0/3479046028 conn(0x7efdb00d4bb0 msgr2=0x7efdb00cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:03.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.427+0000 7efdb73c3640 1 -- 172.21.15.5:0/3479046028 shutdown_connections 2024-05-20T18:38:03.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.427+0000 7efdb73c3640 1 -- 172.21.15.5:0/3479046028 wait complete. 2024-05-20T18:38:03.439 INFO:tasks.ceph:Scrubbing osd.4 2024-05-20T18:38:03.440 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-20T18:38:03.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.557+0000 7f389ef84640 1 Processor -- start 2024-05-20T18:38:03.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.557+0000 7f389ef84640 1 -- start start 2024-05-20T18:38:03.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.559+0000 7f389ef84640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f389808c430 0x7f389806d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:03.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.559+0000 7f389ef84640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38980d0450 con 0x7f38980b3860 2024-05-20T18:38:03.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.559+0000 7f389ef84640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38980d3b10 con 0x7f38980d45f0 2024-05-20T18:38:03.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.559+0000 7f389ef84640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f38980d3df0 con 0x7f389808c430 2024-05-20T18:38:03.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.559+0000 7f389cd20640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f38980b3860 0x7f389815f230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43246/0 (socket says 172.21.15.5:43246) 2024-05-20T18:38:03.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.559+0000 7f389cd20640 1 -- 172.21.15.5:0/25684992 learned_addr learned my addr 172.21.15.5:0/25684992 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:38:03.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.559+0000 7f389d521640 1 --2- 172.21.15.5:0/25684992 >> v2:172.21.15.63:3300/0 conn(0x7f389808c430 0x7f389806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:03.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.559+0000 7f38977fe640 1 -- 172.21.15.5:0/25684992 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1616900742 0 0) 0x7f38980d0450 con 0x7f38980b3860 2024-05-20T18:38:03.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.559+0000 7f38977fe640 1 -- 172.21.15.5:0/25684992 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3880003520 con 0x7f38980b3860 2024-05-20T18:38:03.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.559+0000 7f38977fe640 1 -- 172.21.15.5:0/25684992 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2769487404 0 0) 0x7f3880003520 con 0x7f38980b3860 2024-05-20T18:38:03.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.559+0000 7f38977fe640 1 -- 172.21.15.5:0/25684992 >> v2:172.21.15.63:3300/0 conn(0x7f389808c430 msgr2=0x7f389806d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:38:03.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.559+0000 7f38977fe640 1 --2- 172.21.15.5:0/25684992 >> v2:172.21.15.63:3300/0 conn(0x7f389808c430 0x7f389806d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.559+0000 7f38977fe640 1 -- 172.21.15.5:0/25684992 >> v1:172.21.15.63:6789/0 conn(0x7f38980d45f0 legacy=0x7f389808bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:03.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.559+0000 7f38977fe640 1 -- 172.21.15.5:0/25684992 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38980d40d0 con 0x7f38980b3860 2024-05-20T18:38:03.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.559+0000 7f38977fe640 1 -- 172.21.15.5:0/25684992 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f3888001670 con 0x7f38980b3860 2024-05-20T18:38:03.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.560+0000 7f38977fe640 1 -- 172.21.15.5:0/25684992 >> v1:172.21.15.5:6789/0 conn(0x7f38980b3860 legacy=0x7f389815f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:03.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.560+0000 7f38977fe640 1 --2- 172.21.15.5:0/25684992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38800039b0 0x7f3880003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:03.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.560+0000 7f38977fe640 1 --2- 172.21.15.5:0/25684992 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3880004800 0x7f3880004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.560+0000 7f38977fe640 1 --2- 172.21.15.5:0/25684992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3880005350 0x7f388000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.560+0000 7f38977fe640 1 -- 172.21.15.5:0/25684992 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f38980d3df0 con 0x7f38800039b0 2024-05-20T18:38:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.560+0000 7f38977fe640 1 -- 172.21.15.5:0/25684992 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f388000c280 con 0x7f3880005350 2024-05-20T18:38:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.560+0000 7f38977fe640 1 -- 172.21.15.5:0/25684992 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f388000c480 con 0x7f3880004800 2024-05-20T18:38:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.560+0000 7f389cd20640 1 --2- 172.21.15.5:0/25684992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38800039b0 0x7f3880003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.560+0000 7f3897fff640 1 --2- 172.21.15.5:0/25684992 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3880004800 0x7f3880004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.560+0000 7f389d521640 1 --2- 172.21.15.5:0/25684992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3880005350 0x7f388000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.560+0000 7f389cd20640 1 -- 172.21.15.5:0/25684992 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3880004800 msgr2=0x7f3880004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:38:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.560+0000 7f389cd20640 1 --2- 172.21.15.5:0/25684992 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3880004800 0x7f3880004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.560+0000 7f389cd20640 1 -- 172.21.15.5:0/25684992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3880005350 msgr2=0x7f388000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:38:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.560+0000 7f389cd20640 1 --2- 172.21.15.5:0/25684992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3880005350 0x7f388000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.560+0000 7f389cd20640 1 -- 172.21.15.5:0/25684992 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3880003520 con 0x7f38800039b0 2024-05-20T18:38:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.560+0000 7f389d521640 1 -- 172.21.15.5:0/25684992 reap_dead start 2024-05-20T18:38:03.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.560+0000 7f3897fff640 1 --2- 172.21.15.5:0/25684992 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3880004800 0x7f3880004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:38:03.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.561+0000 7f389cd20640 1 --2- 172.21.15.5:0/25684992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38800039b0 0x7f3880003e20 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f38980d15d0 tx=0x7f3880005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7669d6a2e36f35d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:03.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.561+0000 7f38977fe640 1 -- 172.21.15.5:0/25684992 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888007d30 con 0x7f38800039b0 2024-05-20T18:38:03.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.561+0000 7f38977fe640 1 -- 172.21.15.5:0/25684992 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3888005ad0 con 0x7f38800039b0 2024-05-20T18:38:03.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.561+0000 7f38977fe640 1 -- 172.21.15.5:0/25684992 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888007070 con 0x7f38800039b0 2024-05-20T18:38:03.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.562+0000 7f389ef84640 1 -- 172.21.15.5:0/25684992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38800039b0 msgr2=0x7f3880003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:03.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.562+0000 7f389ef84640 1 --2- 172.21.15.5:0/25684992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38800039b0 0x7f3880003e20 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f38980d15d0 tx=0x7f3880005830 comp rx=0 tx=0).stop 2024-05-20T18:38:03.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.562+0000 7f389ef84640 1 -- 172.21.15.5:0/25684992 shutdown_connections 2024-05-20T18:38:03.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.562+0000 7f389ef84640 1 --2- 172.21.15.5:0/25684992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38800039b0 0x7f3880003e20 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.562+0000 7f389ef84640 1 -- 172.21.15.5:0/25684992 >> 172.21.15.5:0/25684992 conn(0x7f38980b6d10 msgr2=0x7f389806d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:03.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.562+0000 7f389ef84640 1 -- 172.21.15.5:0/25684992 shutdown_connections 2024-05-20T18:38:03.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.562+0000 7f389ef84640 1 -- 172.21.15.5:0/25684992 wait complete. 2024-05-20T18:38:03.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.563+0000 7f389ef84640 1 Processor -- start 2024-05-20T18:38:03.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.564+0000 7f389ef84640 1 -- start start 2024-05-20T18:38:03.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.565+0000 7f389ef84640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f38800039b0 0x7f38981565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:03.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.565+0000 7f389ef84640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3898158bc0 0x7f3898158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:03.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.565+0000 7f389ef84640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38980c94e0 0x7f38980c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:03.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.565+0000 7f389ef84640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f389806d9f0 con 0x7f38980c94e0 2024-05-20T18:38:03.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.565+0000 7f389ef84640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f389806d870 con 0x7f3898158bc0 2024-05-20T18:38:03.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.565+0000 7f389ef84640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f389806db70 con 0x7f38800039b0 2024-05-20T18:38:03.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.565+0000 7f389d521640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38980c94e0 0x7f38980c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:03.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.565+0000 7f389d521640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38980c94e0 0x7f38980c98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60854/0 (socket says 172.21.15.5:60854) 2024-05-20T18:38:03.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.565+0000 7f389d521640 1 -- 172.21.15.5:0/2502375062 learned_addr learned my addr 172.21.15.5:0/2502375062 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:38:03.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.565+0000 7f3897fff640 1 --2- 172.21.15.5:0/2502375062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3898158bc0 0x7f3898158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:03.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.565+0000 7f389cd20640 1 --2- 172.21.15.5:0/2502375062 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f38800039b0 0x7f38981565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:03.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.565+0000 7f389d521640 1 -- 172.21.15.5:0/2502375062 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f38800039b0 msgr2=0x7f38981565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:03.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.565+0000 7f389d521640 1 --2- 172.21.15.5:0/2502375062 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f38800039b0 0x7f38981565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.565+0000 7f389d521640 1 -- 172.21.15.5:0/2502375062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3898158bc0 msgr2=0x7f3898158fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:03.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.565+0000 7f389d521640 1 --2- 172.21.15.5:0/2502375062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3898158bc0 0x7f3898158fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.565+0000 7f389d521640 1 -- 172.21.15.5:0/2502375062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3898159510 con 0x7f38980c94e0 2024-05-20T18:38:03.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.565+0000 7f3897fff640 1 --2- 172.21.15.5:0/2502375062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3898158bc0 0x7f3898158fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:38:03.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.566+0000 7f389d521640 1 --2- 172.21.15.5:0/2502375062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38980c94e0 0x7f38980c98f0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f38980d1250 tx=0x7f38900115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:03.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.566+0000 7f38957fa640 1 -- 172.21.15.5:0/2502375062 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3890011e10 con 0x7f38980c94e0 2024-05-20T18:38:03.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.566+0000 7f38957fa640 1 -- 172.21.15.5:0/2502375062 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f389001b070 con 0x7f38980c94e0 2024-05-20T18:38:03.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.566+0000 7f389ef84640 1 -- 172.21.15.5:0/2502375062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38980cf330 con 0x7f38980c94e0 2024-05-20T18:38:03.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.566+0000 7f38957fa640 1 -- 172.21.15.5:0/2502375062 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3890003990 con 0x7f38980c94e0 2024-05-20T18:38:03.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.566+0000 7f389ef84640 1 -- 172.21.15.5:0/2502375062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38980c8a00 con 0x7f38980c94e0 2024-05-20T18:38:03.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.567+0000 7f389ef84640 1 -- 172.21.15.5:0/2502375062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f38980690b0 con 0x7f38980c94e0 2024-05-20T18:38:03.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.568+0000 7f38957fa640 1 -- 172.21.15.5:0/2502375062 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3890005d10 con 0x7f38980c94e0 2024-05-20T18:38:03.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.568+0000 7f38957fa640 1 --2- 172.21.15.5:0/2502375062 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f388406be60 0x7f388406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:03.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.568+0000 7f38957fa640 1 -- 172.21.15.5:0/2502375062 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f389008c0c0 con 0x7f38980c94e0 2024-05-20T18:38:03.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.568+0000 7f389cd20640 1 --2- 172.21.15.5:0/2502375062 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f388406be60 0x7f388406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:03.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.568+0000 7f38957fa640 1 --2- 172.21.15.5:0/2502375062 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x7f3884072490 0x7f3884074920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:03.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.568+0000 7f38957fa640 1 -- 172.21.15.5:0/2502375062 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3890011020 con 0x7f3884072490 2024-05-20T18:38:03.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.568+0000 7f38957fa640 1 -- 172.21.15.5:0/2502375062 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=124) ==== 24+0+0 (secure 0 0 0) 0x7f389008c550 con 0x7f38980c94e0 2024-05-20T18:38:03.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.569+0000 7f389cd20640 1 --2- 172.21.15.5:0/2502375062 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f388406be60 0x7f388406e370 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f3888032000 tx=0x7f388802d000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:03.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.569+0000 7f3897fff640 1 --2- 172.21.15.5:0/2502375062 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x7f3884072490 0x7f3884074920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:03.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.569+0000 7f3897fff640 1 --2- 172.21.15.5:0/2502375062 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x7f3884072490 0x7f3884074920 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:03.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.570+0000 7f38957fa640 1 -- 172.21.15.5:0/2502375062 <== osd.4 v2:172.21.15.63:6800/3673324406 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3890011020 con 0x7f3884072490 2024-05-20T18:38:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.583+0000 7f389ef84640 1 -- 172.21.15.5:0/2502375062 --> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f38980cf0e0 con 0x7f3884072490 2024-05-20T18:38:03.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.584+0000 7f38957fa640 1 -- 172.21.15.5:0/2502375062 <== osd.4 v2:172.21.15.63:6800/3673324406 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f38980cf0e0 con 0x7f3884072490 2024-05-20T18:38:03.585 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:38:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-20T18:38:03.585 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:38:03.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.584+0000 7f389ef84640 1 -- 172.21.15.5:0/2502375062 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x7f3884072490 msgr2=0x7f3884074920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.584+0000 7f389ef84640 1 --2- 172.21.15.5:0/2502375062 >> [v2:172.21.15.63:6800/3673324406,v1:172.21.15.63:6801/3673324406] conn(0x7f3884072490 0x7f3884074920 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.584+0000 7f389ef84640 1 -- 172.21.15.5:0/2502375062 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f388406be60 msgr2=0x7f388406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.584+0000 7f389ef84640 1 --2- 172.21.15.5:0/2502375062 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f388406be60 0x7f388406e370 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f3888032000 tx=0x7f388802d000 comp rx=0 tx=0).stop 2024-05-20T18:38:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.584+0000 7f389ef84640 1 -- 172.21.15.5:0/2502375062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38980c94e0 msgr2=0x7f38980c98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.584+0000 7f389ef84640 1 --2- 172.21.15.5:0/2502375062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38980c94e0 0x7f38980c98f0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f38980d1250 tx=0x7f38900115a0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.584+0000 7f389d521640 1 -- 172.21.15.5:0/2502375062 reap_dead start 2024-05-20T18:38:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.585+0000 7f389ef84640 1 -- 172.21.15.5:0/2502375062 shutdown_connections 2024-05-20T18:38:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.585+0000 7f389ef84640 1 -- 172.21.15.5:0/2502375062 >> 172.21.15.5:0/2502375062 conn(0x7f38980b6d10 msgr2=0x7f389815dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.585+0000 7f389ef84640 1 -- 172.21.15.5:0/2502375062 shutdown_connections 2024-05-20T18:38:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.585+0000 7f389ef84640 1 -- 172.21.15.5:0/2502375062 wait complete. 2024-05-20T18:38:03.597 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-20T18:38:03.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.712+0000 7f189272a640 1 Processor -- start 2024-05-20T18:38:03.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.712+0000 7f189272a640 1 -- start start 2024-05-20T18:38:03.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.714+0000 7f189272a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f188c0cf3d0 0x7f188c0cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:03.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.714+0000 7f189272a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f188c0ce7e0 0x7f188c0cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:03.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.714+0000 7f189272a640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f188c0cfd20 con 0x7f188c15f220 2024-05-20T18:38:03.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.714+0000 7f189272a640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f188c0cf130 con 0x7f188c0cf3d0 2024-05-20T18:38:03.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.714+0000 7f189272a640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f188c0cda20 con 0x7f188c0ce7e0 2024-05-20T18:38:03.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.714+0000 7f188b7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f188c0cf3d0 0x7f188c0cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:03.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.714+0000 7f188b7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f188c0cf3d0 0x7f188c0cf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60864/0 (socket says 172.21.15.5:60864) 2024-05-20T18:38:03.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.714+0000 7f188b7fe640 1 -- 172.21.15.5:0/2782081166 learned_addr learned my addr 172.21.15.5:0/2782081166 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:38:03.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.714+0000 7f1890cc7640 1 --2- 172.21.15.5:0/2782081166 >> v2:172.21.15.63:3300/0 conn(0x7f188c0ce7e0 0x7f188c0cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:03.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.714+0000 7f188b7fe640 1 -- 172.21.15.5:0/2782081166 >> v2:172.21.15.63:3300/0 conn(0x7f188c0ce7e0 msgr2=0x7f188c0cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:38:03.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.714+0000 7f188b7fe640 1 --2- 172.21.15.5:0/2782081166 >> v2:172.21.15.63:3300/0 conn(0x7f188c0ce7e0 0x7f188c0cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.714+0000 7f188b7fe640 1 -- 172.21.15.5:0/2782081166 >> v1:172.21.15.63:6789/0 conn(0x7f188c15f220 legacy=0x7f188c0d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:03.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.714+0000 7f188b7fe640 1 -- 172.21.15.5:0/2782081166 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188c0cdd00 con 0x7f188c0cf3d0 2024-05-20T18:38:03.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.715+0000 7f188b7fe640 1 --2- 172.21.15.5:0/2782081166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f188c0cf3d0 0x7f188c0cf7e0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f188c06e3e0 tx=0x7f187402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0588e0a4a4b309d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:03.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.715+0000 7f188affd640 1 -- 172.21.15.5:0/2782081166 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1874037070 con 0x7f188c0cf3d0 2024-05-20T18:38:03.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.715+0000 7f188affd640 1 -- 172.21.15.5:0/2782081166 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1874032440 con 0x7f188c0cf3d0 2024-05-20T18:38:03.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.715+0000 7f188affd640 1 -- 172.21.15.5:0/2782081166 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1874032810 con 0x7f188c0cf3d0 2024-05-20T18:38:03.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.716+0000 7f189272a640 1 -- 172.21.15.5:0/2782081166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f188c0cf3d0 msgr2=0x7f188c0cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:03.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.716+0000 7f189272a640 1 --2- 172.21.15.5:0/2782081166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f188c0cf3d0 0x7f188c0cf7e0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f188c06e3e0 tx=0x7f187402a1e0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.716+0000 7f189272a640 1 -- 172.21.15.5:0/2782081166 shutdown_connections 2024-05-20T18:38:03.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.716+0000 7f189272a640 1 --2- 172.21.15.5:0/2782081166 >> v2:172.21.15.63:3300/0 conn(0x7f188c0ce7e0 0x7f188c0cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.716+0000 7f189272a640 1 --2- 172.21.15.5:0/2782081166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f188c0cf3d0 0x7f188c0cf7e0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.716+0000 7f189272a640 1 -- 172.21.15.5:0/2782081166 >> 172.21.15.5:0/2782081166 conn(0x7f188c0b44d0 msgr2=0x7f188c0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:03.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.716+0000 7f189272a640 1 -- 172.21.15.5:0/2782081166 shutdown_connections 2024-05-20T18:38:03.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.716+0000 7f189272a640 1 -- 172.21.15.5:0/2782081166 wait complete. 2024-05-20T18:38:03.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.718+0000 7f189272a640 1 Processor -- start 2024-05-20T18:38:03.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.718+0000 7f189272a640 1 -- start start 2024-05-20T18:38:03.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.719+0000 7f189272a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f188c0ce7e0 0x7f188c0b7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:03.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.719+0000 7f189272a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f188c0cf3d0 0x7f188c1f89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:03.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.719+0000 7f189272a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f188c15f220 0x7f188c15fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:03.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.719+0000 7f189272a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f188c0d0fd0 con 0x7f188c15f220 2024-05-20T18:38:03.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.719+0000 7f189272a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f188c0ce2c0 con 0x7f188c0cf3d0 2024-05-20T18:38:03.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.719+0000 7f189272a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f188c0cd680 con 0x7f188c0ce7e0 2024-05-20T18:38:03.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.719+0000 7f1890cc7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f188c15f220 0x7f188c15fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:03.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.719+0000 7f1890cc7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f188c15f220 0x7f188c15fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60868/0 (socket says 172.21.15.5:60868) 2024-05-20T18:38:03.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.719+0000 7f1890cc7640 1 -- 172.21.15.5:0/3165571044 learned_addr learned my addr 172.21.15.5:0/3165571044 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:38:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.720+0000 7f188bfff640 1 --2- 172.21.15.5:0/3165571044 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f188c0ce7e0 0x7f188c0b7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.720+0000 7f188b7fe640 1 --2- 172.21.15.5:0/3165571044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f188c0cf3d0 0x7f188c1f89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.720+0000 7f1890cc7640 1 -- 172.21.15.5:0/3165571044 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f188c0ce7e0 msgr2=0x7f188c0b7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.720+0000 7f1890cc7640 1 --2- 172.21.15.5:0/3165571044 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f188c0ce7e0 0x7f188c0b7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.720+0000 7f1890cc7640 1 -- 172.21.15.5:0/3165571044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f188c0cf3d0 msgr2=0x7f188c1f89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.720+0000 7f1890cc7640 1 --2- 172.21.15.5:0/3165571044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f188c0cf3d0 0x7f188c1f89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.720+0000 7f1890cc7640 1 -- 172.21.15.5:0/3165571044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188c160160 con 0x7f188c15f220 2024-05-20T18:38:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.720+0000 7f188bfff640 1 --2- 172.21.15.5:0/3165571044 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f188c0ce7e0 0x7f188c0b7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:38:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.720+0000 7f188b7fe640 1 --2- 172.21.15.5:0/3165571044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f188c0cf3d0 0x7f188c1f89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:38:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.720+0000 7f1890cc7640 1 --2- 172.21.15.5:0/3165571044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f188c15f220 0x7f188c15fc20 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f1884007e60 tx=0x7f188400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.720+0000 7f1888ff9640 1 -- 172.21.15.5:0/3165571044 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884014420 con 0x7f188c15f220 2024-05-20T18:38:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.720+0000 7f1888ff9640 1 -- 172.21.15.5:0/3165571044 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1884019070 con 0x7f188c15f220 2024-05-20T18:38:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.720+0000 7f189272a640 1 -- 172.21.15.5:0/3165571044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f188c0b6ba0 con 0x7f188c15f220 2024-05-20T18:38:03.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.720+0000 7f1888ff9640 1 -- 172.21.15.5:0/3165571044 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884014820 con 0x7f188c15f220 2024-05-20T18:38:03.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.720+0000 7f189272a640 1 -- 172.21.15.5:0/3165571044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f188c0b7160 con 0x7f188c15f220 2024-05-20T18:38:03.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.721+0000 7f189272a640 1 -- 172.21.15.5:0/3165571044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f188c0689d0 con 0x7f188c15f220 2024-05-20T18:38:03.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.722+0000 7f1888ff9640 1 -- 172.21.15.5:0/3165571044 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f18840149c0 con 0x7f188c15f220 2024-05-20T18:38:03.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.722+0000 7f1888ff9640 1 --2- 172.21.15.5:0/3165571044 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f186c06be30 0x7f186c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:03.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.722+0000 7f1888ff9640 1 -- 172.21.15.5:0/3165571044 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f188408b610 con 0x7f188c15f220 2024-05-20T18:38:03.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.722+0000 7f188bfff640 1 --2- 172.21.15.5:0/3165571044 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f186c06be30 0x7f186c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:03.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.723+0000 7f188bfff640 1 --2- 172.21.15.5:0/3165571044 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f186c06be30 0x7f186c06e340 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f187c000f60 tx=0x7f187c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:03.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.727+0000 7f1888ff9640 1 -- 172.21.15.5:0/3165571044 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f188c0689d0 con 0x7f188c15f220 2024-05-20T18:38:03.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.909+0000 7f189272a640 1 -- 172.21.15.5:0/3165571044 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f188c1f8f10 con 0x7f186c06be30 2024-05-20T18:38:03.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.910+0000 7f1888ff9640 1 -- 172.21.15.5:0/3165571044 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f187c001c90 con 0x7f186c06be30 2024-05-20T18:38:03.911 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 4 to deep-scrub 2024-05-20T18:38:03.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.912+0000 7f189272a640 1 -- 172.21.15.5:0/3165571044 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f186c06be30 msgr2=0x7f186c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:03.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.912+0000 7f189272a640 1 --2- 172.21.15.5:0/3165571044 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f186c06be30 0x7f186c06e340 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f187c000f60 tx=0x7f187c009040 comp rx=0 tx=0).stop 2024-05-20T18:38:03.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.912+0000 7f189272a640 1 -- 172.21.15.5:0/3165571044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f188c15f220 msgr2=0x7f188c15fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:03.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.912+0000 7f189272a640 1 --2- 172.21.15.5:0/3165571044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f188c15f220 0x7f188c15fc20 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f1884007e60 tx=0x7f188400e4b0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.912+0000 7f189272a640 1 -- 172.21.15.5:0/3165571044 shutdown_connections 2024-05-20T18:38:03.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.912+0000 7f189272a640 1 --2- 172.21.15.5:0/3165571044 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f186c06be30 0x7f186c06e340 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.912+0000 7f189272a640 1 --2- 172.21.15.5:0/3165571044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f188c15f220 0x7f188c15fc20 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.912+0000 7f189272a640 1 --2- 172.21.15.5:0/3165571044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f188c0cf3d0 0x7f188c1f89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.912+0000 7f189272a640 1 --2- 172.21.15.5:0/3165571044 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f188c0ce7e0 0x7f188c0b7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:03.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.912+0000 7f189272a640 1 -- 172.21.15.5:0/3165571044 >> 172.21.15.5:0/3165571044 conn(0x7f188c0b44d0 msgr2=0x7f188c0b3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:03.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.912+0000 7f189272a640 1 -- 172.21.15.5:0/3165571044 shutdown_connections 2024-05-20T18:38:03.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:03.912+0000 7f189272a640 1 -- 172.21.15.5:0/3165571044 wait complete. 2024-05-20T18:38:03.924 INFO:tasks.ceph:Scrubbing osd.5 2024-05-20T18:38:03.925 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-20T18:38:04.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.042+0000 7f43e93d5640 1 Processor -- start 2024-05-20T18:38:04.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.042+0000 7f43e93d5640 1 -- start start 2024-05-20T18:38:04.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.043+0000 7f43e93d5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43e40d45f0 0x7f43e406d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:04.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.043+0000 7f43e93d5640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43e40d0450 con 0x7f43e408bd20 2024-05-20T18:38:04.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.043+0000 7f43e93d5640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43e40d3b10 con 0x7f43e40b3860 2024-05-20T18:38:04.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.043+0000 7f43e93d5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f43e40d3df0 con 0x7f43e40d45f0 2024-05-20T18:38:04.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.044+0000 7f43e37fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f43e408bd20 0x7f43e408c130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43248/0 (socket says 172.21.15.5:43248) 2024-05-20T18:38:04.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.044+0000 7f43e37fe640 1 -- 172.21.15.5:0/226234733 learned_addr learned my addr 172.21.15.5:0/226234733 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:38:04.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.044+0000 7f43dbfff640 1 --2- 172.21.15.5:0/226234733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43e40d45f0 0x7f43e406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:04.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.044+0000 7f43e27fc640 1 -- 172.21.15.5:0/226234733 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3645008748 0 0) 0x7f43e40d0450 con 0x7f43e408bd20 2024-05-20T18:38:04.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.044+0000 7f43e27fc640 1 -- 172.21.15.5:0/226234733 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43d0003520 con 0x7f43e408bd20 2024-05-20T18:38:04.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.044+0000 7f43e27fc640 1 -- 172.21.15.5:0/226234733 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1605081536 0 0) 0x7f43e40d3b10 con 0x7f43e40b3860 2024-05-20T18:38:04.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.044+0000 7f43e27fc640 1 -- 172.21.15.5:0/226234733 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43e40d0450 con 0x7f43e40b3860 2024-05-20T18:38:04.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.044+0000 7f43e27fc640 1 -- 172.21.15.5:0/226234733 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 287490452 0 0) 0x7f43d0003520 con 0x7f43e408bd20 2024-05-20T18:38:04.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.044+0000 7f43dbfff640 1 -- 172.21.15.5:0/226234733 >> v1:172.21.15.63:6789/0 conn(0x7f43e40b3860 legacy=0x7f43e415f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:04.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.044+0000 7f43dbfff640 1 -- 172.21.15.5:0/226234733 >> v1:172.21.15.5:6789/0 conn(0x7f43e408bd20 legacy=0x7f43e408c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:04.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.044+0000 7f43dbfff640 1 -- 172.21.15.5:0/226234733 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e40d40d0 con 0x7f43e40d45f0 2024-05-20T18:38:04.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.045+0000 7f43dbfff640 1 --2- 172.21.15.5:0/226234733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43e40d45f0 0x7f43e406d870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0097d0 tx=0x7f43cc02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4525344606b3785 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:04.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.045+0000 7f43e27fc640 1 -- 172.21.15.5:0/226234733 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc037070 con 0x7f43e40d45f0 2024-05-20T18:38:04.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.045+0000 7f43e27fc640 1 -- 172.21.15.5:0/226234733 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43cc02ab90 con 0x7f43e40d45f0 2024-05-20T18:38:04.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.045+0000 7f43e27fc640 1 -- 172.21.15.5:0/226234733 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc0019f0 con 0x7f43e40d45f0 2024-05-20T18:38:04.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.046+0000 7f43e93d5640 1 -- 172.21.15.5:0/226234733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43e40d45f0 msgr2=0x7f43e406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:04.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.046+0000 7f43e93d5640 1 --2- 172.21.15.5:0/226234733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43e40d45f0 0x7f43e406d870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0097d0 tx=0x7f43cc02a100 comp rx=0 tx=0).stop 2024-05-20T18:38:04.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.046+0000 7f43e93d5640 1 -- 172.21.15.5:0/226234733 shutdown_connections 2024-05-20T18:38:04.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.046+0000 7f43e93d5640 1 --2- 172.21.15.5:0/226234733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43e40d45f0 0x7f43e406d870 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.046+0000 7f43e93d5640 1 -- 172.21.15.5:0/226234733 >> 172.21.15.5:0/226234733 conn(0x7f43e40b6d10 msgr2=0x7f43e406d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:04.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.046+0000 7f43e93d5640 1 -- 172.21.15.5:0/226234733 shutdown_connections 2024-05-20T18:38:04.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.046+0000 7f43e93d5640 1 -- 172.21.15.5:0/226234733 wait complete. 2024-05-20T18:38:04.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.048+0000 7f43e93d5640 1 Processor -- start 2024-05-20T18:38:04.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.048+0000 7f43e93d5640 1 -- start start 2024-05-20T18:38:04.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.049+0000 7f43e93d5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43e408bd20 0x7f43e40c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:04.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.049+0000 7f43e93d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43e40b3860 0x7f43e41f89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:04.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.049+0000 7f43e93d5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43e40d45f0 0x7f43e41f8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:04.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.049+0000 7f43e93d5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f43e416a0f0 con 0x7f43e40b3860 2024-05-20T18:38:04.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.049+0000 7f43e93d5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f43e4169f70 con 0x7f43e408bd20 2024-05-20T18:38:04.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.049+0000 7f43e93d5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f43e416a270 con 0x7f43e40d45f0 2024-05-20T18:38:04.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.049+0000 7f43dbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43e40b3860 0x7f43e41f89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:04.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.049+0000 7f43dbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43e40b3860 0x7f43e41f89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60876/0 (socket says 172.21.15.5:60876) 2024-05-20T18:38:04.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.049+0000 7f43dbfff640 1 -- 172.21.15.5:0/1116099695 learned_addr learned my addr 172.21.15.5:0/1116099695 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:38:04.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.050+0000 7f43e37fe640 1 --2- 172.21.15.5:0/1116099695 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43e40d45f0 0x7f43e41f8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:04.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.050+0000 7f43e2ffd640 1 --2- 172.21.15.5:0/1116099695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43e408bd20 0x7f43e40c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:04.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.050+0000 7f43dbfff640 1 -- 172.21.15.5:0/1116099695 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43e40d45f0 msgr2=0x7f43e41f8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:04.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.050+0000 7f43dbfff640 1 --2- 172.21.15.5:0/1116099695 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43e40d45f0 0x7f43e41f8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.050+0000 7f43dbfff640 1 -- 172.21.15.5:0/1116099695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43e408bd20 msgr2=0x7f43e40c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:04.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.050+0000 7f43dbfff640 1 --2- 172.21.15.5:0/1116099695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43e408bd20 0x7f43e40c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.050+0000 7f43dbfff640 1 -- 172.21.15.5:0/1116099695 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e40ca4a0 con 0x7f43e40b3860 2024-05-20T18:38:04.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.050+0000 7f43e2ffd640 1 --2- 172.21.15.5:0/1116099695 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43e408bd20 0x7f43e40c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:38:04.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.050+0000 7f43dbfff640 1 --2- 172.21.15.5:0/1116099695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43e40b3860 0x7f43e41f89e0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f43cc035040 tx=0x7f43cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:04.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.050+0000 7f43db7fe640 1 -- 172.21.15.5:0/1116099695 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc042020 con 0x7f43e40b3860 2024-05-20T18:38:04.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.050+0000 7f43db7fe640 1 -- 172.21.15.5:0/1116099695 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43cc03d670 con 0x7f43e40b3860 2024-05-20T18:38:04.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.050+0000 7f43e93d5640 1 -- 172.21.15.5:0/1116099695 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e40ca780 con 0x7f43e40b3860 2024-05-20T18:38:04.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.050+0000 7f43db7fe640 1 -- 172.21.15.5:0/1116099695 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc037040 con 0x7f43e40b3860 2024-05-20T18:38:04.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.050+0000 7f43e93d5640 1 -- 172.21.15.5:0/1116099695 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e40cad40 con 0x7f43e40b3860 2024-05-20T18:38:04.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.051+0000 7f43e93d5640 1 -- 172.21.15.5:0/1116099695 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f43e40690b0 con 0x7f43e40b3860 2024-05-20T18:38:04.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.052+0000 7f43db7fe640 1 -- 172.21.15.5:0/1116099695 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f43cc045020 con 0x7f43e40b3860 2024-05-20T18:38:04.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.052+0000 7f43db7fe640 1 --2- 172.21.15.5:0/1116099695 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f43c406be60 0x7f43c406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:04.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.052+0000 7f43db7fe640 1 -- 172.21.15.5:0/1116099695 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f43cc0abcb0 con 0x7f43e40b3860 2024-05-20T18:38:04.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.053+0000 7f43e2ffd640 1 --2- 172.21.15.5:0/1116099695 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f43c406be60 0x7f43c406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:04.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.053+0000 7f43db7fe640 1 --2- 172.21.15.5:0/1116099695 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x7f43c4072490 0x7f43c4074920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:04.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.053+0000 7f43db7fe640 1 -- 172.21.15.5:0/1116099695 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f43cc051e50 con 0x7f43c4072490 2024-05-20T18:38:04.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.053+0000 7f43db7fe640 1 -- 172.21.15.5:0/1116099695 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=124) ==== 24+0+0 (secure 0 0 0) 0x7f43cc0ac060 con 0x7f43e40b3860 2024-05-20T18:38:04.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.053+0000 7f43e37fe640 1 --2- 172.21.15.5:0/1116099695 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x7f43c4072490 0x7f43c4074920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:04.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.053+0000 7f43e2ffd640 1 --2- 172.21.15.5:0/1116099695 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f43c406be60 0x7f43c406e370 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f43d40011e0 tx=0x7f43d4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:04.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.053+0000 7f43e37fe640 1 --2- 172.21.15.5:0/1116099695 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x7f43c4072490 0x7f43c4074920 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:04.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.054+0000 7f43db7fe640 1 -- 172.21.15.5:0/1116099695 <== osd.5 v2:172.21.15.63:6808/1616066286 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f43cc051e50 con 0x7f43c4072490 2024-05-20T18:38:04.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.067+0000 7f43e93d5640 1 -- 172.21.15.5:0/1116099695 --> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f43e40ca250 con 0x7f43c4072490 2024-05-20T18:38:04.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.068+0000 7f43db7fe640 1 -- 172.21.15.5:0/1116099695 <== osd.5 v2:172.21.15.63:6808/1616066286 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f43e40ca250 con 0x7f43c4072490 2024-05-20T18:38:04.069 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:38:04.070 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-20T18:38:04.070 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:38:04.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.068+0000 7f43e93d5640 1 -- 172.21.15.5:0/1116099695 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x7f43c4072490 msgr2=0x7f43c4074920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:04.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.068+0000 7f43e93d5640 1 --2- 172.21.15.5:0/1116099695 >> [v2:172.21.15.63:6808/1616066286,v1:172.21.15.63:6809/1616066286] conn(0x7f43c4072490 0x7f43c4074920 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.068+0000 7f43e93d5640 1 -- 172.21.15.5:0/1116099695 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f43c406be60 msgr2=0x7f43c406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:04.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.068+0000 7f43e93d5640 1 --2- 172.21.15.5:0/1116099695 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f43c406be60 0x7f43c406e370 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f43d40011e0 tx=0x7f43d4009040 comp rx=0 tx=0).stop 2024-05-20T18:38:04.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.068+0000 7f43e93d5640 1 -- 172.21.15.5:0/1116099695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43e40b3860 msgr2=0x7f43e41f89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:04.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.068+0000 7f43e93d5640 1 --2- 172.21.15.5:0/1116099695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43e40b3860 0x7f43e41f89e0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f43cc035040 tx=0x7f43cc0046c0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.068+0000 7f43e37fe640 1 -- 172.21.15.5:0/1116099695 reap_dead start 2024-05-20T18:38:04.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.068+0000 7f43e93d5640 1 -- 172.21.15.5:0/1116099695 shutdown_connections 2024-05-20T18:38:04.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.068+0000 7f43e93d5640 1 -- 172.21.15.5:0/1116099695 >> 172.21.15.5:0/1116099695 conn(0x7f43e40b6d10 msgr2=0x7f43e40b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:04.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.068+0000 7f43e93d5640 1 -- 172.21.15.5:0/1116099695 shutdown_connections 2024-05-20T18:38:04.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.068+0000 7f43e93d5640 1 -- 172.21.15.5:0/1116099695 wait complete. 2024-05-20T18:38:04.080 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-20T18:38:04.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.195+0000 7f8291f85640 1 Processor -- start 2024-05-20T18:38:04.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.195+0000 7f8291f85640 1 -- start start 2024-05-20T18:38:04.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.197+0000 7f8291f85640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f828c162820 0x7f828c0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:04.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.197+0000 7f8291f85640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f828c0b3e80 0x7f828c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:04.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.197+0000 7f8291f85640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f828c06dd20 con 0x7f828c06d910 2024-05-20T18:38:04.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.197+0000 7f8291f85640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f828c0d0630 con 0x7f828c162820 2024-05-20T18:38:04.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.197+0000 7f8291f85640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f828c0d39c0 con 0x7f828c0b3e80 2024-05-20T18:38:04.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.197+0000 7f828b7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f828c162820 0x7f828c0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:04.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.197+0000 7f828bfff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f828c06d910 0x7f828c0d27a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43256/0 (socket says 172.21.15.5:43256) 2024-05-20T18:38:04.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.197+0000 7f828bfff640 1 -- 172.21.15.5:0/669728026 learned_addr learned my addr 172.21.15.5:0/669728026 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:38:04.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.197+0000 7f828affd640 1 --2- 172.21.15.5:0/669728026 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f828c0b3e80 0x7f828c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:04.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.197+0000 7f828b7fe640 1 -- 172.21.15.5:0/669728026 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f828c0b3e80 msgr2=0x7f828c06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:38:04.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.197+0000 7f828b7fe640 1 --2- 172.21.15.5:0/669728026 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f828c0b3e80 0x7f828c06d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.197+0000 7f828b7fe640 1 -- 172.21.15.5:0/669728026 >> v1:172.21.15.5:6789/0 conn(0x7f828c06d910 legacy=0x7f828c0d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:04.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.197+0000 7f828a7fc640 1 -- 172.21.15.5:0/669728026 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4180893253 0 0) 0x7f8284001490 con 0x7f828c06d910 2024-05-20T18:38:04.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.197+0000 7f828b7fe640 1 -- 172.21.15.5:0/669728026 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f828c0d3ca0 con 0x7f828c162820 2024-05-20T18:38:04.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.198+0000 7f828b7fe640 1 --2- 172.21.15.5:0/669728026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828c162820 0x7f828c0b3940 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f827c0097a0 tx=0x7f827c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1134b5be723d57de server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:04.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.198+0000 7f828a7fc640 1 -- 172.21.15.5:0/669728026 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c037070 con 0x7f828c162820 2024-05-20T18:38:04.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.198+0000 7f828a7fc640 1 -- 172.21.15.5:0/669728026 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f827c032440 con 0x7f828c162820 2024-05-20T18:38:04.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.198+0000 7f828a7fc640 1 -- 172.21.15.5:0/669728026 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c032810 con 0x7f828c162820 2024-05-20T18:38:04.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.199+0000 7f8291f85640 1 -- 172.21.15.5:0/669728026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828c162820 msgr2=0x7f828c0b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:04.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.199+0000 7f8291f85640 1 --2- 172.21.15.5:0/669728026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828c162820 0x7f828c0b3940 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f827c0097a0 tx=0x7f827c02a180 comp rx=0 tx=0).stop 2024-05-20T18:38:04.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.199+0000 7f8291f85640 1 -- 172.21.15.5:0/669728026 shutdown_connections 2024-05-20T18:38:04.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.199+0000 7f8291f85640 1 --2- 172.21.15.5:0/669728026 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f828c0b3e80 0x7f828c06d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.199+0000 7f8291f85640 1 --2- 172.21.15.5:0/669728026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828c162820 0x7f828c0b3940 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.199+0000 7f8291f85640 1 -- 172.21.15.5:0/669728026 >> 172.21.15.5:0/669728026 conn(0x7f828c0d4bd0 msgr2=0x7f828c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:04.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.199+0000 7f8291f85640 1 -- 172.21.15.5:0/669728026 shutdown_connections 2024-05-20T18:38:04.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.199+0000 7f8291f85640 1 -- 172.21.15.5:0/669728026 wait complete. 2024-05-20T18:38:04.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.201+0000 7f8291f85640 1 Processor -- start 2024-05-20T18:38:04.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.201+0000 7f8291f85640 1 -- start start 2024-05-20T18:38:04.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.202+0000 7f8291f85640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f828c06d910 0x7f828c1f7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:04.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.202+0000 7f8291f85640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f828c0b3e80 0x7f828c15ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:04.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.202+0000 7f8291f85640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828c162820 0x7f828c15f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:04.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.202+0000 7f8291f85640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f828c0d0300 con 0x7f828c162820 2024-05-20T18:38:04.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.202+0000 7f8291f85640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f828c0d4260 con 0x7f828c06d910 2024-05-20T18:38:04.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.202+0000 7f8291f85640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f828c0b91a0 con 0x7f828c0b3e80 2024-05-20T18:38:04.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.202+0000 7f828bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828c162820 0x7f828c15f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:04.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.202+0000 7f828bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828c162820 0x7f828c15f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60892/0 (socket says 172.21.15.5:60892) 2024-05-20T18:38:04.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.202+0000 7f828bfff640 1 -- 172.21.15.5:0/243027011 learned_addr learned my addr 172.21.15.5:0/243027011 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:38:04.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.202+0000 7f828b7fe640 1 --2- 172.21.15.5:0/243027011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f828c06d910 0x7f828c1f7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:04.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.202+0000 7f828affd640 1 --2- 172.21.15.5:0/243027011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f828c0b3e80 0x7f828c15ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:04.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.203+0000 7f828bfff640 1 -- 172.21.15.5:0/243027011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f828c0b3e80 msgr2=0x7f828c15ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:04.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.203+0000 7f828bfff640 1 --2- 172.21.15.5:0/243027011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f828c0b3e80 0x7f828c15ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.203+0000 7f828bfff640 1 -- 172.21.15.5:0/243027011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f828c06d910 msgr2=0x7f828c1f7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:04.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.203+0000 7f828bfff640 1 --2- 172.21.15.5:0/243027011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f828c06d910 0x7f828c1f7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.203+0000 7f828bfff640 1 -- 172.21.15.5:0/243027011 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f828c0c88a0 con 0x7f828c162820 2024-05-20T18:38:04.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.203+0000 7f828bfff640 1 --2- 172.21.15.5:0/243027011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828c162820 0x7f828c15f460 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f828400c6d0 tx=0x7f828400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:04.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.203+0000 7f8273fff640 1 -- 172.21.15.5:0/243027011 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8284018070 con 0x7f828c162820 2024-05-20T18:38:04.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.203+0000 7f8273fff640 1 -- 172.21.15.5:0/243027011 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8284004d60 con 0x7f828c162820 2024-05-20T18:38:04.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.203+0000 7f8291f85640 1 -- 172.21.15.5:0/243027011 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f828c0c8b80 con 0x7f828c162820 2024-05-20T18:38:04.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.203+0000 7f8273fff640 1 -- 172.21.15.5:0/243027011 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828400a880 con 0x7f828c162820 2024-05-20T18:38:04.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.203+0000 7f8291f85640 1 -- 172.21.15.5:0/243027011 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f828c0c9140 con 0x7f828c162820 2024-05-20T18:38:04.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.204+0000 7f8291f85640 1 -- 172.21.15.5:0/243027011 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f828c075b00 con 0x7f828c162820 2024-05-20T18:38:04.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.205+0000 7f8273fff640 1 -- 172.21.15.5:0/243027011 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f828401b070 con 0x7f828c162820 2024-05-20T18:38:04.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.205+0000 7f8273fff640 1 --2- 172.21.15.5:0/243027011 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f827806be30 0x7f827806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:04.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.206+0000 7f828b7fe640 1 --2- 172.21.15.5:0/243027011 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f827806be30 0x7f827806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:04.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.206+0000 7f8273fff640 1 -- 172.21.15.5:0/243027011 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f828408a300 con 0x7f828c162820 2024-05-20T18:38:04.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.206+0000 7f828b7fe640 1 --2- 172.21.15.5:0/243027011 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f827806be30 0x7f827806e340 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f827c034000 tx=0x7f827c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:04.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.210+0000 7f8273fff640 1 -- 172.21.15.5:0/243027011 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f828c06dd20 con 0x7f828c162820 2024-05-20T18:38:04.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.393+0000 7f8291f85640 1 -- 172.21.15.5:0/243027011 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f828c06c530 con 0x7f827806be30 2024-05-20T18:38:04.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.393+0000 7f8273fff640 1 -- 172.21.15.5:0/243027011 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f828c06c530 con 0x7f827806be30 2024-05-20T18:38:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 5 to deep-scrub 2024-05-20T18:38:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.395+0000 7f8291f85640 1 -- 172.21.15.5:0/243027011 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f827806be30 msgr2=0x7f827806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.395+0000 7f8291f85640 1 --2- 172.21.15.5:0/243027011 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f827806be30 0x7f827806e340 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f827c034000 tx=0x7f827c02e000 comp rx=0 tx=0).stop 2024-05-20T18:38:04.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.395+0000 7f8291f85640 1 -- 172.21.15.5:0/243027011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828c162820 msgr2=0x7f828c15f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:04.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.395+0000 7f8291f85640 1 --2- 172.21.15.5:0/243027011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828c162820 0x7f828c15f460 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f828400c6d0 tx=0x7f828400cb90 comp rx=0 tx=0).stop 2024-05-20T18:38:04.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.395+0000 7f8291f85640 1 -- 172.21.15.5:0/243027011 shutdown_connections 2024-05-20T18:38:04.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.395+0000 7f8291f85640 1 --2- 172.21.15.5:0/243027011 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f827806be30 0x7f827806e340 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.395+0000 7f8291f85640 1 --2- 172.21.15.5:0/243027011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828c162820 0x7f828c15f460 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.395+0000 7f8291f85640 1 --2- 172.21.15.5:0/243027011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f828c0b3e80 0x7f828c15ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.395+0000 7f8291f85640 1 --2- 172.21.15.5:0/243027011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f828c06d910 0x7f828c1f7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.395+0000 7f8291f85640 1 -- 172.21.15.5:0/243027011 >> 172.21.15.5:0/243027011 conn(0x7f828c0d4bd0 msgr2=0x7f828c0b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:04.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.395+0000 7f8291f85640 1 -- 172.21.15.5:0/243027011 shutdown_connections 2024-05-20T18:38:04.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.395+0000 7f8291f85640 1 -- 172.21.15.5:0/243027011 wait complete. 2024-05-20T18:38:04.407 INFO:tasks.ceph:Scrubbing osd.6 2024-05-20T18:38:04.408 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-20T18:38:04.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.525+0000 7f10380e9640 1 Processor -- start 2024-05-20T18:38:04.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.525+0000 7f10380e9640 1 -- start start 2024-05-20T18:38:04.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.527+0000 7f10380e9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f10300b3860 0x7f10300d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:04.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.527+0000 7f10380e9640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f103006d870 0x7f103015f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:04.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.527+0000 7f10380e9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f103008bd20 0x7f103008c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:04.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.527+0000 7f10380e9640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f103008c670 con 0x7f103008bd20 2024-05-20T18:38:04.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.527+0000 7f10380e9640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f10300d0450 con 0x7f103006d870 2024-05-20T18:38:04.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.527+0000 7f10380e9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f10300d3b10 con 0x7f10300b3860 2024-05-20T18:38:04.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.527+0000 7f1036686640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f103008bd20 0x7f103008c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:04.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.527+0000 7f1036686640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f103008bd20 0x7f103008c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60900/0 (socket says 172.21.15.5:60900) 2024-05-20T18:38:04.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.527+0000 7f1036686640 1 -- 172.21.15.5:0/155370709 learned_addr learned my addr 172.21.15.5:0/155370709 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:38:04.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.527+0000 7f1035684640 1 --2- 172.21.15.5:0/155370709 >> v2:172.21.15.63:3300/0 conn(0x7f103006d870 0x7f103015f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:04.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.527+0000 7f1035e85640 1 --2- 172.21.15.5:0/155370709 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f10300b3860 0x7f10300d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:04.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.527+0000 7f1036686640 1 -- 172.21.15.5:0/155370709 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f10300b3860 msgr2=0x7f10300d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:38:04.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.527+0000 7f1036686640 1 --2- 172.21.15.5:0/155370709 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f10300b3860 0x7f10300d45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.527+0000 7f1036686640 1 -- 172.21.15.5:0/155370709 >> v2:172.21.15.63:3300/0 conn(0x7f103006d870 msgr2=0x7f103015f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:38:04.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.527+0000 7f1036686640 1 --2- 172.21.15.5:0/155370709 >> v2:172.21.15.63:3300/0 conn(0x7f103006d870 0x7f103015f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.527+0000 7f1036686640 1 -- 172.21.15.5:0/155370709 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10300d3df0 con 0x7f103008bd20 2024-05-20T18:38:04.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.528+0000 7f1036686640 1 --2- 172.21.15.5:0/155370709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f103008bd20 0x7f103008c130 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f103008dd20 tx=0x7f102802a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4979122ab14c51bf server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:04.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.528+0000 7f1034e83640 1 -- 172.21.15.5:0/155370709 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1028036070 con 0x7f103008bd20 2024-05-20T18:38:04.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.528+0000 7f1034e83640 1 -- 172.21.15.5:0/155370709 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1028031600 con 0x7f103008bd20 2024-05-20T18:38:04.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.528+0000 7f1034e83640 1 -- 172.21.15.5:0/155370709 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10280319d0 con 0x7f103008bd20 2024-05-20T18:38:04.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.529+0000 7f10380e9640 1 -- 172.21.15.5:0/155370709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f103008bd20 msgr2=0x7f103008c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:04.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.529+0000 7f10380e9640 1 --2- 172.21.15.5:0/155370709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f103008bd20 0x7f103008c130 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f103008dd20 tx=0x7f102802a570 comp rx=0 tx=0).stop 2024-05-20T18:38:04.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.529+0000 7f10380e9640 1 -- 172.21.15.5:0/155370709 shutdown_connections 2024-05-20T18:38:04.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.529+0000 7f10380e9640 1 --2- 172.21.15.5:0/155370709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f103008bd20 0x7f103008c130 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.529+0000 7f10380e9640 1 --2- 172.21.15.5:0/155370709 >> v2:172.21.15.63:3300/0 conn(0x7f103006d870 0x7f103015f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.529+0000 7f10380e9640 1 --2- 172.21.15.5:0/155370709 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f10300b3860 0x7f10300d45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.529+0000 7f10380e9640 1 -- 172.21.15.5:0/155370709 >> 172.21.15.5:0/155370709 conn(0x7f10300b6d10 msgr2=0x7f103006d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:04.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.529+0000 7f10380e9640 1 -- 172.21.15.5:0/155370709 shutdown_connections 2024-05-20T18:38:04.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.529+0000 7f10380e9640 1 -- 172.21.15.5:0/155370709 wait complete. 2024-05-20T18:38:04.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.531+0000 7f10380e9640 1 Processor -- start 2024-05-20T18:38:04.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.531+0000 7f10380e9640 1 -- start start 2024-05-20T18:38:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.532+0000 7f10380e9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f103006d870 0x7f10300cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.532+0000 7f10380e9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f103008bd20 0x7f10301f8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.532+0000 7f10380e9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10300b3860 0x7f10301f8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.532+0000 7f10380e9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f103015f770 con 0x7f10300b3860 2024-05-20T18:38:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.532+0000 7f10380e9640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f10300d0730 con 0x7f103008bd20 2024-05-20T18:38:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.532+0000 7f10380e9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f10300d27c0 con 0x7f103006d870 2024-05-20T18:38:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.532+0000 7f1036686640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10300b3860 0x7f10301f8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.532+0000 7f1036686640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10300b3860 0x7f10301f8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60904/0 (socket says 172.21.15.5:60904) 2024-05-20T18:38:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.532+0000 7f1036686640 1 -- 172.21.15.5:0/2314047915 learned_addr learned my addr 172.21.15.5:0/2314047915 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:38:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.533+0000 7f1035684640 1 --2- 172.21.15.5:0/2314047915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f103008bd20 0x7f10301f8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.533+0000 7f1035e85640 1 --2- 172.21.15.5:0/2314047915 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f103006d870 0x7f10300cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.533+0000 7f1036686640 1 -- 172.21.15.5:0/2314047915 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f103006d870 msgr2=0x7f10300cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.533+0000 7f1036686640 1 --2- 172.21.15.5:0/2314047915 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f103006d870 0x7f10300cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.533+0000 7f1036686640 1 -- 172.21.15.5:0/2314047915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f103008bd20 msgr2=0x7f10301f8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.533+0000 7f1036686640 1 --2- 172.21.15.5:0/2314047915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f103008bd20 0x7f10301f8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.533+0000 7f1036686640 1 -- 172.21.15.5:0/2314047915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10300cc880 con 0x7f10300b3860 2024-05-20T18:38:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.533+0000 7f1035e85640 1 --2- 172.21.15.5:0/2314047915 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f103006d870 0x7f10300cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:38:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.533+0000 7f1035684640 1 --2- 172.21.15.5:0/2314047915 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f103008bd20 0x7f10301f8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-20T18:38:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.533+0000 7f1036686640 1 --2- 172.21.15.5:0/2314047915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10300b3860 0x7f10301f8a00 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f1028034040 tx=0x7f1028004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.533+0000 7f10267fc640 1 -- 172.21.15.5:0/2314047915 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1028036050 con 0x7f10300b3860 2024-05-20T18:38:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.533+0000 7f10267fc640 1 -- 172.21.15.5:0/2314047915 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1028042070 con 0x7f10300b3860 2024-05-20T18:38:04.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.533+0000 7f10380e9640 1 -- 172.21.15.5:0/2314047915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10300c9910 con 0x7f10300b3860 2024-05-20T18:38:04.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.533+0000 7f10267fc640 1 -- 172.21.15.5:0/2314047915 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1028009510 con 0x7f10300b3860 2024-05-20T18:38:04.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.533+0000 7f10380e9640 1 -- 172.21.15.5:0/2314047915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10300c9ed0 con 0x7f10300b3860 2024-05-20T18:38:04.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.534+0000 7f10380e9640 1 -- 172.21.15.5:0/2314047915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f10300690b0 con 0x7f10300b3860 2024-05-20T18:38:04.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.535+0000 7f10267fc640 1 -- 172.21.15.5:0/2314047915 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1028007bf0 con 0x7f10300b3860 2024-05-20T18:38:04.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.535+0000 7f10267fc640 1 --2- 172.21.15.5:0/2314047915 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f101406be60 0x7f101406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:04.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.535+0000 7f10267fc640 1 -- 172.21.15.5:0/2314047915 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f10280abb60 con 0x7f10300b3860 2024-05-20T18:38:04.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.535+0000 7f1035e85640 1 --2- 172.21.15.5:0/2314047915 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f101406be60 0x7f101406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:04.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.535+0000 7f10267fc640 1 --2- 172.21.15.5:0/2314047915 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x7f1014072490 0x7f1014074920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:04.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.535+0000 7f10267fc640 1 -- 172.21.15.5:0/2314047915 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f102804ae80 con 0x7f1014072490 2024-05-20T18:38:04.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.535+0000 7f10267fc640 1 -- 172.21.15.5:0/2314047915 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=124) ==== 24+0+0 (secure 0 0 0) 0x7f10280b02c0 con 0x7f10300b3860 2024-05-20T18:38:04.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.536+0000 7f1035e85640 1 --2- 172.21.15.5:0/2314047915 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f101406be60 0x7f101406e370 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f10300d15d0 tx=0x7f102c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:04.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.536+0000 7f1035684640 1 --2- 172.21.15.5:0/2314047915 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x7f1014072490 0x7f1014074920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:04.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.536+0000 7f1035684640 1 --2- 172.21.15.5:0/2314047915 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x7f1014072490 0x7f1014074920 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:04.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.537+0000 7f10267fc640 1 -- 172.21.15.5:0/2314047915 <== osd.6 v2:172.21.15.63:6816/3384565140 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f102804ae80 con 0x7f1014072490 2024-05-20T18:38:04.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.550+0000 7f10380e9640 1 -- 172.21.15.5:0/2314047915 --> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f10300cc630 con 0x7f1014072490 2024-05-20T18:38:04.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.551+0000 7f10267fc640 1 -- 172.21.15.5:0/2314047915 <== osd.6 v2:172.21.15.63:6816/3384565140 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f10300cc630 con 0x7f1014072490 2024-05-20T18:38:04.552 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:38:04.552 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-20T18:38:04.553 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:38:04.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.551+0000 7f10380e9640 1 -- 172.21.15.5:0/2314047915 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x7f1014072490 msgr2=0x7f1014074920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:04.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.551+0000 7f10380e9640 1 --2- 172.21.15.5:0/2314047915 >> [v2:172.21.15.63:6816/3384565140,v1:172.21.15.63:6817/3384565140] conn(0x7f1014072490 0x7f1014074920 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.551+0000 7f10380e9640 1 -- 172.21.15.5:0/2314047915 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f101406be60 msgr2=0x7f101406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:04.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.551+0000 7f10380e9640 1 --2- 172.21.15.5:0/2314047915 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f101406be60 0x7f101406e370 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f10300d15d0 tx=0x7f102c008040 comp rx=0 tx=0).stop 2024-05-20T18:38:04.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.551+0000 7f10380e9640 1 -- 172.21.15.5:0/2314047915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10300b3860 msgr2=0x7f10301f8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:04.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.551+0000 7f10380e9640 1 --2- 172.21.15.5:0/2314047915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10300b3860 0x7f10301f8a00 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f1028034040 tx=0x7f1028004a40 comp rx=0 tx=0).stop 2024-05-20T18:38:04.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.551+0000 7f1036686640 1 -- 172.21.15.5:0/2314047915 reap_dead start 2024-05-20T18:38:04.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.551+0000 7f10380e9640 1 -- 172.21.15.5:0/2314047915 shutdown_connections 2024-05-20T18:38:04.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.551+0000 7f10380e9640 1 -- 172.21.15.5:0/2314047915 >> 172.21.15.5:0/2314047915 conn(0x7f10300b6d10 msgr2=0x7f10301e4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:04.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.551+0000 7f10380e9640 1 -- 172.21.15.5:0/2314047915 shutdown_connections 2024-05-20T18:38:04.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.551+0000 7f10380e9640 1 -- 172.21.15.5:0/2314047915 wait complete. 2024-05-20T18:38:04.563 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-20T18:38:04.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.677+0000 7f366c916640 1 Processor -- start 2024-05-20T18:38:04.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.677+0000 7f366c916640 1 -- start start 2024-05-20T18:38:04.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.678+0000 7f366c916640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f36680cf850 0x7f36680b6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:04.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.678+0000 7f366c916640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f36680d14f0 0x7f36680b6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:04.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.678+0000 7f366c916640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36680d1ac0 con 0x7f36680d10e0 2024-05-20T18:38:04.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.678+0000 7f366c916640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f36680d1da0 con 0x7f36680d14f0 2024-05-20T18:38:04.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.678+0000 7f366c916640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f36680d2080 con 0x7f36680cf850 2024-05-20T18:38:04.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.678+0000 7f366659c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f36680d10e0 0x7f36680cf140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:42032/0 (socket says 172.21.15.5:42032) 2024-05-20T18:38:04.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.678+0000 7f366659c640 1 -- 172.21.15.5:0/384632797 learned_addr learned my addr 172.21.15.5:0/384632797 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:38:04.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.679+0000 7f3666d9d640 1 --2- 172.21.15.5:0/384632797 >> v2:172.21.15.5:3300/0 conn(0x7f36680d14f0 0x7f36680b6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:04.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.679+0000 7f365ffff640 1 --2- 172.21.15.5:0/384632797 >> v2:172.21.15.63:3300/0 conn(0x7f36680cf850 0x7f36680b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:04.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.679+0000 7f3665d9b640 1 -- 172.21.15.5:0/384632797 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 740590805 0 0) 0x7f36680d1ac0 con 0x7f36680d10e0 2024-05-20T18:38:04.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.679+0000 7f3665d9b640 1 -- 172.21.15.5:0/384632797 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3654003520 con 0x7f36680d10e0 2024-05-20T18:38:04.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.679+0000 7f3666d9d640 1 -- 172.21.15.5:0/384632797 >> v2:172.21.15.63:3300/0 conn(0x7f36680cf850 msgr2=0x7f36680b6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:38:04.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.679+0000 7f3666d9d640 1 --2- 172.21.15.5:0/384632797 >> v2:172.21.15.63:3300/0 conn(0x7f36680cf850 0x7f36680b6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.679+0000 7f3666d9d640 1 -- 172.21.15.5:0/384632797 >> v1:172.21.15.5:6789/0 conn(0x7f36680d10e0 legacy=0x7f36680cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:04.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.679+0000 7f3666d9d640 1 -- 172.21.15.5:0/384632797 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36680d2360 con 0x7f36680d14f0 2024-05-20T18:38:04.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.680+0000 7f3666d9d640 1 --2- 172.21.15.5:0/384632797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36680d14f0 0x7f36680b6740 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f366808bee0 tx=0x7f366002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9033f2e944541375 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:04.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.680+0000 7f3665d9b640 1 -- 172.21.15.5:0/384632797 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3660007740 con 0x7f36680d14f0 2024-05-20T18:38:04.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.680+0000 7f3665d9b640 1 -- 172.21.15.5:0/384632797 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36600078e0 con 0x7f36680d14f0 2024-05-20T18:38:04.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.680+0000 7f3665d9b640 1 -- 172.21.15.5:0/384632797 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3660007cb0 con 0x7f36680d14f0 2024-05-20T18:38:04.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.680+0000 7f366c916640 1 -- 172.21.15.5:0/384632797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36680d14f0 msgr2=0x7f36680b6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:04.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.680+0000 7f366c916640 1 --2- 172.21.15.5:0/384632797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36680d14f0 0x7f36680b6740 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f366808bee0 tx=0x7f366002a680 comp rx=0 tx=0).stop 2024-05-20T18:38:04.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.681+0000 7f366c916640 1 -- 172.21.15.5:0/384632797 shutdown_connections 2024-05-20T18:38:04.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.681+0000 7f366c916640 1 --2- 172.21.15.5:0/384632797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36680d14f0 0x7f36680b6740 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.681+0000 7f366c916640 1 --2- 172.21.15.5:0/384632797 >> v2:172.21.15.63:3300/0 conn(0x7f36680cf850 0x7f36680b6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.681+0000 7f366c916640 1 -- 172.21.15.5:0/384632797 >> 172.21.15.5:0/384632797 conn(0x7f36680b44d0 msgr2=0x7f36680b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:04.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.681+0000 7f366c916640 1 -- 172.21.15.5:0/384632797 shutdown_connections 2024-05-20T18:38:04.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.681+0000 7f366c916640 1 -- 172.21.15.5:0/384632797 wait complete. 2024-05-20T18:38:04.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.682+0000 7f366c916640 1 Processor -- start 2024-05-20T18:38:04.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.683+0000 7f366c916640 1 -- start start 2024-05-20T18:38:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.684+0000 7f366c916640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36680cf850 0x7f36680c64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.684+0000 7f366c916640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36680d10e0 0x7f366815b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.684+0000 7f366c916640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36680d14f0 0x7f366806da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.684+0000 7f366c916640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f36680b8440 con 0x7f36680d14f0 2024-05-20T18:38:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.684+0000 7f366c916640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f36680b82c0 con 0x7f36680cf850 2024-05-20T18:38:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.684+0000 7f366c916640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f36680709f0 con 0x7f36680d10e0 2024-05-20T18:38:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.684+0000 7f3666d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36680d14f0 0x7f366806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.684+0000 7f3666d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36680d14f0 0x7f366806da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35020/0 (socket says 172.21.15.5:35020) 2024-05-20T18:38:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.684+0000 7f3666d9d640 1 -- 172.21.15.5:0/178555255 learned_addr learned my addr 172.21.15.5:0/178555255 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:38:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.684+0000 7f365ffff640 1 --2- 172.21.15.5:0/178555255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36680d10e0 0x7f366815b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.684+0000 7f366659c640 1 --2- 172.21.15.5:0/178555255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36680cf850 0x7f36680c64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.684+0000 7f3666d9d640 1 -- 172.21.15.5:0/178555255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36680d10e0 msgr2=0x7f366815b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.684+0000 7f3666d9d640 1 --2- 172.21.15.5:0/178555255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36680d10e0 0x7f366815b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.684+0000 7f3666d9d640 1 -- 172.21.15.5:0/178555255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36680cf850 msgr2=0x7f36680c64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.684+0000 7f3666d9d640 1 --2- 172.21.15.5:0/178555255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36680cf850 0x7f36680c64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.684+0000 7f3666d9d640 1 -- 172.21.15.5:0/178555255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366806cc50 con 0x7f36680d14f0 2024-05-20T18:38:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.685+0000 7f3666d9d640 1 --2- 172.21.15.5:0/178555255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36680d14f0 0x7f366806da10 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f366002ac70 tx=0x7f3660004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.685+0000 7f365effd640 1 -- 172.21.15.5:0/178555255 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366003f070 con 0x7f36680d14f0 2024-05-20T18:38:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.685+0000 7f365effd640 1 -- 172.21.15.5:0/178555255 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3660031bc0 con 0x7f36680d14f0 2024-05-20T18:38:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.685+0000 7f366c916640 1 -- 172.21.15.5:0/178555255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f366806cf30 con 0x7f36680d14f0 2024-05-20T18:38:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.685+0000 7f365effd640 1 -- 172.21.15.5:0/178555255 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3660009450 con 0x7f36680d14f0 2024-05-20T18:38:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.685+0000 7f366c916640 1 -- 172.21.15.5:0/178555255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f366806d4f0 con 0x7f36680d14f0 2024-05-20T18:38:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.686+0000 7f366c916640 1 -- 172.21.15.5:0/178555255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36680cf0c0 con 0x7f36680d14f0 2024-05-20T18:38:04.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.686+0000 7f365effd640 1 -- 172.21.15.5:0/178555255 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f366003b410 con 0x7f36680d14f0 2024-05-20T18:38:04.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.687+0000 7f365effd640 1 --2- 172.21.15.5:0/178555255 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f364806be30 0x7f364806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:04.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.687+0000 7f366659c640 1 --2- 172.21.15.5:0/178555255 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f364806be30 0x7f364806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:04.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.687+0000 7f365effd640 1 -- 172.21.15.5:0/178555255 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f36600ac3f0 con 0x7f36680d14f0 2024-05-20T18:38:04.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.688+0000 7f366659c640 1 --2- 172.21.15.5:0/178555255 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f364806be30 0x7f364806e340 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f3658005880 tx=0x7f3658009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:04.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.692+0000 7f365effd640 1 -- 172.21.15.5:0/178555255 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f366007a770 con 0x7f36680d14f0 2024-05-20T18:38:04.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.880+0000 7f366c916640 1 -- 172.21.15.5:0/178555255 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f36680c6b80 con 0x7f364806be30 2024-05-20T18:38:04.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.880+0000 7f365effd640 1 -- 172.21.15.5:0/178555255 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f36680c6b80 con 0x7f364806be30 2024-05-20T18:38:04.881 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 6 to deep-scrub 2024-05-20T18:38:04.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.882+0000 7f366c916640 1 -- 172.21.15.5:0/178555255 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f364806be30 msgr2=0x7f364806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:04.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.882+0000 7f366c916640 1 --2- 172.21.15.5:0/178555255 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f364806be30 0x7f364806e340 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f3658005880 tx=0x7f3658009040 comp rx=0 tx=0).stop 2024-05-20T18:38:04.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.882+0000 7f366c916640 1 -- 172.21.15.5:0/178555255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36680d14f0 msgr2=0x7f366806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:04.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.882+0000 7f366c916640 1 --2- 172.21.15.5:0/178555255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36680d14f0 0x7f366806da10 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f366002ac70 tx=0x7f3660004a40 comp rx=0 tx=0).stop 2024-05-20T18:38:04.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.882+0000 7f366c916640 1 -- 172.21.15.5:0/178555255 shutdown_connections 2024-05-20T18:38:04.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.882+0000 7f366c916640 1 --2- 172.21.15.5:0/178555255 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f364806be30 0x7f364806e340 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.882+0000 7f366c916640 1 --2- 172.21.15.5:0/178555255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36680d14f0 0x7f366806da10 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.882+0000 7f366c916640 1 --2- 172.21.15.5:0/178555255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36680d10e0 0x7f366815b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.882+0000 7f366c916640 1 --2- 172.21.15.5:0/178555255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36680cf850 0x7f36680c64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:04.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.882+0000 7f366c916640 1 -- 172.21.15.5:0/178555255 >> 172.21.15.5:0/178555255 conn(0x7f36680b44d0 msgr2=0x7f36680b33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:04.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.882+0000 7f366c916640 1 -- 172.21.15.5:0/178555255 shutdown_connections 2024-05-20T18:38:04.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:04.882+0000 7f366c916640 1 -- 172.21.15.5:0/178555255 wait complete. 2024-05-20T18:38:04.895 INFO:tasks.ceph:Scrubbing osd.7 2024-05-20T18:38:04.895 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-20T18:38:05.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.012+0000 7fe678aa8640 1 Processor -- start 2024-05-20T18:38:05.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.012+0000 7fe678aa8640 1 -- start start 2024-05-20T18:38:05.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.013+0000 7fe678aa8640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe6740b3860 0x7fe6740d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:05.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.013+0000 7fe678aa8640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe67408bd20 0x7fe67408c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:05.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.013+0000 7fe678aa8640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe67408c670 con 0x7fe67406d870 2024-05-20T18:38:05.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.013+0000 7fe678aa8640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe6740d0450 con 0x7fe6740b3860 2024-05-20T18:38:05.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.013+0000 7fe678aa8640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe6740d3b10 con 0x7fe67408bd20 2024-05-20T18:38:05.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.014+0000 7fe67259c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe6740b3860 0x7fe6740d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:05.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.014+0000 7fe671d9b640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe67406d870 0x7fe67415f230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:42036/0 (socket says 172.21.15.5:42036) 2024-05-20T18:38:05.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.014+0000 7fe67259c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe6740b3860 0x7fe6740d45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35030/0 (socket says 172.21.15.5:35030) 2024-05-20T18:38:05.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.014+0000 7fe671d9b640 1 -- 172.21.15.5:0/1298764130 learned_addr learned my addr 172.21.15.5:0/1298764130 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:38:05.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.014+0000 7fe672d9d640 1 --2- 172.21.15.5:0/1298764130 >> v2:172.21.15.63:3300/0 conn(0x7fe67408bd20 0x7fe67408c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:05.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.014+0000 7fe67259c640 1 -- 172.21.15.5:0/1298764130 >> v2:172.21.15.63:3300/0 conn(0x7fe67408bd20 msgr2=0x7fe67408c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:38:05.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.014+0000 7fe67259c640 1 --2- 172.21.15.5:0/1298764130 >> v2:172.21.15.63:3300/0 conn(0x7fe67408bd20 0x7fe67408c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:05.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.014+0000 7fe67259c640 1 -- 172.21.15.5:0/1298764130 >> v1:172.21.15.5:6789/0 conn(0x7fe67406d870 legacy=0x7fe67415f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:05.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.014+0000 7fe67159a640 1 -- 172.21.15.5:0/1298764130 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1069616774 0 0) 0x7fe668001110 con 0x7fe67406d870 2024-05-20T18:38:05.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.014+0000 7fe67259c640 1 -- 172.21.15.5:0/1298764130 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6740d3df0 con 0x7fe6740b3860 2024-05-20T18:38:05.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.015+0000 7fe67259c640 1 --2- 172.21.15.5:0/1298764130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6740b3860 0x7fe6740d45f0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fe6740d1210 tx=0x7fe66402a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d2d7686b2dc62d server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:05.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.015+0000 7fe67159a640 1 -- 172.21.15.5:0/1298764130 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664007740 con 0x7fe6740b3860 2024-05-20T18:38:05.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.015+0000 7fe67159a640 1 -- 172.21.15.5:0/1298764130 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6640078e0 con 0x7fe6740b3860 2024-05-20T18:38:05.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.015+0000 7fe67159a640 1 -- 172.21.15.5:0/1298764130 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664007cb0 con 0x7fe6740b3860 2024-05-20T18:38:05.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.016+0000 7fe678aa8640 1 -- 172.21.15.5:0/1298764130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6740b3860 msgr2=0x7fe6740d45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:05.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.016+0000 7fe678aa8640 1 --2- 172.21.15.5:0/1298764130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6740b3860 0x7fe6740d45f0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fe6740d1210 tx=0x7fe66402a560 comp rx=0 tx=0).stop 2024-05-20T18:38:05.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.016+0000 7fe678aa8640 1 -- 172.21.15.5:0/1298764130 shutdown_connections 2024-05-20T18:38:05.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.016+0000 7fe678aa8640 1 --2- 172.21.15.5:0/1298764130 >> v2:172.21.15.63:3300/0 conn(0x7fe67408bd20 0x7fe67408c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:05.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.016+0000 7fe678aa8640 1 --2- 172.21.15.5:0/1298764130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6740b3860 0x7fe6740d45f0 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:05.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.016+0000 7fe678aa8640 1 -- 172.21.15.5:0/1298764130 >> 172.21.15.5:0/1298764130 conn(0x7fe6740b6d10 msgr2=0x7fe67406d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:05.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.016+0000 7fe678aa8640 1 -- 172.21.15.5:0/1298764130 shutdown_connections 2024-05-20T18:38:05.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.016+0000 7fe678aa8640 1 -- 172.21.15.5:0/1298764130 wait complete. 2024-05-20T18:38:05.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.018+0000 7fe678aa8640 1 Processor -- start 2024-05-20T18:38:05.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.018+0000 7fe678aa8640 1 -- start start 2024-05-20T18:38:05.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.019+0000 7fe678aa8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe67406d870 0x7fe674154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:05.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.019+0000 7fe678aa8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe67408bd20 0x7fe6741580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:05.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.019+0000 7fe678aa8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6740b3860 0x7fe674158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:05.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.019+0000 7fe678aa8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6740b81b0 con 0x7fe67408bd20 2024-05-20T18:38:05.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.019+0000 7fe678aa8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe6740d0730 con 0x7fe67406d870 2024-05-20T18:38:05.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.019+0000 7fe678aa8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe6740b8330 con 0x7fe6740b3860 2024-05-20T18:38:05.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.019+0000 7fe671d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe67408bd20 0x7fe6741580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:05.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.019+0000 7fe671d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe67408bd20 0x7fe6741580f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35038/0 (socket says 172.21.15.5:35038) 2024-05-20T18:38:05.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.019+0000 7fe671d9b640 1 -- 172.21.15.5:0/845069143 learned_addr learned my addr 172.21.15.5:0/845069143 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:38:05.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.019+0000 7fe672d9d640 1 --2- 172.21.15.5:0/845069143 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6740b3860 0x7fe674158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:05.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.019+0000 7fe67259c640 1 --2- 172.21.15.5:0/845069143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe67406d870 0x7fe674154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:05.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.019+0000 7fe671d9b640 1 -- 172.21.15.5:0/845069143 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6740b3860 msgr2=0x7fe674158630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:05.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.020+0000 7fe671d9b640 1 --2- 172.21.15.5:0/845069143 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6740b3860 0x7fe674158630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:05.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.020+0000 7fe671d9b640 1 -- 172.21.15.5:0/845069143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe67406d870 msgr2=0x7fe674154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:05.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.020+0000 7fe671d9b640 1 --2- 172.21.15.5:0/845069143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe67406d870 0x7fe674154280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:05.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.020+0000 7fe671d9b640 1 -- 172.21.15.5:0/845069143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe67414d690 con 0x7fe67408bd20 2024-05-20T18:38:05.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.020+0000 7fe671d9b640 1 --2- 172.21.15.5:0/845069143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe67408bd20 0x7fe6741580f0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fe66800c3c0 tx=0x7fe66800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:05.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.020+0000 7fe662ffd640 1 -- 172.21.15.5:0/845069143 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6680074e0 con 0x7fe67408bd20 2024-05-20T18:38:05.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.020+0000 7fe662ffd640 1 -- 172.21.15.5:0/845069143 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6680047e0 con 0x7fe67408bd20 2024-05-20T18:38:05.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.020+0000 7fe678aa8640 1 -- 172.21.15.5:0/845069143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe67414d970 con 0x7fe67408bd20 2024-05-20T18:38:05.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.020+0000 7fe662ffd640 1 -- 172.21.15.5:0/845069143 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe668004bd0 con 0x7fe67408bd20 2024-05-20T18:38:05.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.020+0000 7fe678aa8640 1 -- 172.21.15.5:0/845069143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe67414df30 con 0x7fe67408bd20 2024-05-20T18:38:05.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.021+0000 7fe678aa8640 1 -- 172.21.15.5:0/845069143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe6740690b0 con 0x7fe67408bd20 2024-05-20T18:38:05.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.021+0000 7fe662ffd640 1 -- 172.21.15.5:0/845069143 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe668011070 con 0x7fe67408bd20 2024-05-20T18:38:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.022+0000 7fe662ffd640 1 --2- 172.21.15.5:0/845069143 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe65406be30 0x7fe65406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.022+0000 7fe662ffd640 1 -- 172.21.15.5:0/845069143 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7fe66808a970 con 0x7fe67408bd20 2024-05-20T18:38:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.022+0000 7fe67259c640 1 --2- 172.21.15.5:0/845069143 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe65406be30 0x7fe65406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.022+0000 7fe662ffd640 1 --2- 172.21.15.5:0/845069143 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x7fe654072460 0x7fe6540748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.022+0000 7fe662ffd640 1 -- 172.21.15.5:0/845069143 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe654074fc0 con 0x7fe654072460 2024-05-20T18:38:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.022+0000 7fe662ffd640 1 -- 172.21.15.5:0/845069143 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=124) ==== 24+0+0 (secure 0 0 0) 0x7fe66808f2c0 con 0x7fe67408bd20 2024-05-20T18:38:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.022+0000 7fe67259c640 1 --2- 172.21.15.5:0/845069143 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe65406be30 0x7fe65406e340 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe664034000 tx=0x7fe66402f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.023+0000 7fe672d9d640 1 --2- 172.21.15.5:0/845069143 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x7fe654072460 0x7fe6540748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.023+0000 7fe672d9d640 1 --2- 172.21.15.5:0/845069143 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x7fe654072460 0x7fe6540748f0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.024+0000 7fe662ffd640 1 -- 172.21.15.5:0/845069143 <== osd.7 v2:172.21.15.63:6824/4136124451 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe654074fc0 con 0x7fe654072460 2024-05-20T18:38:05.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.037+0000 7fe678aa8640 1 -- 172.21.15.5:0/845069143 --> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe67414d440 con 0x7fe654072460 2024-05-20T18:38:05.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.038+0000 7fe662ffd640 1 -- 172.21.15.5:0/845069143 <== osd.7 v2:172.21.15.63:6824/4136124451 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe67414d440 con 0x7fe654072460 2024-05-20T18:38:05.039 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-20T18:38:05.039 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-20T18:38:05.039 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-20T18:38:05.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.038+0000 7fe678aa8640 1 -- 172.21.15.5:0/845069143 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x7fe654072460 msgr2=0x7fe6540748f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:05.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.038+0000 7fe678aa8640 1 --2- 172.21.15.5:0/845069143 >> [v2:172.21.15.63:6824/4136124451,v1:172.21.15.63:6825/4136124451] conn(0x7fe654072460 0x7fe6540748f0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.038+0000 7fe678aa8640 1 -- 172.21.15.5:0/845069143 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe65406be30 msgr2=0x7fe65406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.038+0000 7fe678aa8640 1 --2- 172.21.15.5:0/845069143 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7fe65406be30 0x7fe65406e340 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe664034000 tx=0x7fe66402f000 comp rx=0 tx=0).stop 2024-05-20T18:38:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.038+0000 7fe678aa8640 1 -- 172.21.15.5:0/845069143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe67408bd20 msgr2=0x7fe6741580f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.038+0000 7fe678aa8640 1 --2- 172.21.15.5:0/845069143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe67408bd20 0x7fe6741580f0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fe66800c3c0 tx=0x7fe66800c880 comp rx=0 tx=0).stop 2024-05-20T18:38:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.038+0000 7fe672d9d640 1 -- 172.21.15.5:0/845069143 reap_dead start 2024-05-20T18:38:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.038+0000 7fe678aa8640 1 -- 172.21.15.5:0/845069143 shutdown_connections 2024-05-20T18:38:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.038+0000 7fe678aa8640 1 -- 172.21.15.5:0/845069143 >> 172.21.15.5:0/845069143 conn(0x7fe6740b6d10 msgr2=0x7fe6740b4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.038+0000 7fe678aa8640 1 -- 172.21.15.5:0/845069143 shutdown_connections 2024-05-20T18:38:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.038+0000 7fe678aa8640 1 -- 172.21.15.5:0/845069143 wait complete. 2024-05-20T18:38:05.050 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-20T18:38:05.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.164+0000 7efd6bde3640 1 Processor -- start 2024-05-20T18:38:05.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.164+0000 7efd6bde3640 1 -- start start 2024-05-20T18:38:05.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.166+0000 7efd6bde3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efd64162820 0x7efd640b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:05.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.166+0000 7efd6bde3640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7efd640b3e80 0x7efd6406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:05.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.166+0000 7efd6bde3640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd6406dd20 con 0x7efd6406d910 2024-05-20T18:38:05.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.166+0000 7efd6bde3640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7efd640d0630 con 0x7efd64162820 2024-05-20T18:38:05.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.166+0000 7efd6bde3640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7efd640d39c0 con 0x7efd640b3e80 2024-05-20T18:38:05.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.166+0000 7efd69b7f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efd64162820 0x7efd640b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:05.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.166+0000 7efd69b7f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efd64162820 0x7efd640b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35048/0 (socket says 172.21.15.5:35048) 2024-05-20T18:38:05.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.166+0000 7efd69b7f640 1 -- 172.21.15.5:0/3584977161 learned_addr learned my addr 172.21.15.5:0/3584977161 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:38:05.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.166+0000 7efd6937e640 1 --2- 172.21.15.5:0/3584977161 >> v2:172.21.15.63:3300/0 conn(0x7efd640b3e80 0x7efd6406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:05.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.166+0000 7efd69b7f640 1 -- 172.21.15.5:0/3584977161 >> v2:172.21.15.63:3300/0 conn(0x7efd640b3e80 msgr2=0x7efd6406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:38:05.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.166+0000 7efd69b7f640 1 --2- 172.21.15.5:0/3584977161 >> v2:172.21.15.63:3300/0 conn(0x7efd640b3e80 0x7efd6406d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:05.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.166+0000 7efd69b7f640 1 -- 172.21.15.5:0/3584977161 >> v1:172.21.15.63:6789/0 conn(0x7efd6406d910 legacy=0x7efd640d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:05.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.166+0000 7efd69b7f640 1 -- 172.21.15.5:0/3584977161 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd640d3ca0 con 0x7efd64162820 2024-05-20T18:38:05.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.167+0000 7efd69b7f640 1 --2- 172.21.15.5:0/3584977161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd64162820 0x7efd640b3940 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7efd640d1230 tx=0x7efd6002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d51498b7229a5a server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:05.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.167+0000 7efd68b7d640 1 -- 172.21.15.5:0/3584977161 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60037070 con 0x7efd64162820 2024-05-20T18:38:05.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.167+0000 7efd68b7d640 1 -- 172.21.15.5:0/3584977161 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd60032440 con 0x7efd64162820 2024-05-20T18:38:05.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.167+0000 7efd68b7d640 1 -- 172.21.15.5:0/3584977161 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60032810 con 0x7efd64162820 2024-05-20T18:38:05.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.168+0000 7efd6bde3640 1 -- 172.21.15.5:0/3584977161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd64162820 msgr2=0x7efd640b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:05.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.168+0000 7efd6bde3640 1 --2- 172.21.15.5:0/3584977161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd64162820 0x7efd640b3940 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7efd640d1230 tx=0x7efd6002a270 comp rx=0 tx=0).stop 2024-05-20T18:38:05.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.168+0000 7efd6bde3640 1 -- 172.21.15.5:0/3584977161 shutdown_connections 2024-05-20T18:38:05.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.168+0000 7efd6bde3640 1 --2- 172.21.15.5:0/3584977161 >> v2:172.21.15.63:3300/0 conn(0x7efd640b3e80 0x7efd6406d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:05.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.168+0000 7efd6bde3640 1 --2- 172.21.15.5:0/3584977161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd64162820 0x7efd640b3940 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:05.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.168+0000 7efd6bde3640 1 -- 172.21.15.5:0/3584977161 >> 172.21.15.5:0/3584977161 conn(0x7efd640d4bd0 msgr2=0x7efd640d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:05.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.168+0000 7efd6bde3640 1 -- 172.21.15.5:0/3584977161 shutdown_connections 2024-05-20T18:38:05.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.168+0000 7efd6bde3640 1 -- 172.21.15.5:0/3584977161 wait complete. 2024-05-20T18:38:05.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.170+0000 7efd6bde3640 1 Processor -- start 2024-05-20T18:38:05.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.170+0000 7efd6bde3640 1 -- start start 2024-05-20T18:38:05.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.171+0000 7efd6bde3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd6406d910 0x7efd640caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.171+0000 7efd6bde3640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd640b3e80 0x7efd641f50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.171+0000 7efd6bde3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd64162820 0x7efd640cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.171+0000 7efd6bde3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efd640d0300 con 0x7efd6406d910 2024-05-20T18:38:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.171+0000 7efd6bde3640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efd640d4260 con 0x7efd640b3e80 2024-05-20T18:38:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.171+0000 7efd6bde3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efd640b91a0 con 0x7efd64162820 2024-05-20T18:38:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.171+0000 7efd69b7f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd6406d910 0x7efd640caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.171+0000 7efd69b7f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd6406d910 0x7efd640caaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35050/0 (socket says 172.21.15.5:35050) 2024-05-20T18:38:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.171+0000 7efd69b7f640 1 -- 172.21.15.5:0/1523758522 learned_addr learned my addr 172.21.15.5:0/1523758522 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:38:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.171+0000 7efd6a380640 1 --2- 172.21.15.5:0/1523758522 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd64162820 0x7efd640cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.171+0000 7efd6937e640 1 --2- 172.21.15.5:0/1523758522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd640b3e80 0x7efd641f50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.172+0000 7efd69b7f640 1 -- 172.21.15.5:0/1523758522 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd64162820 msgr2=0x7efd640cbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.172+0000 7efd69b7f640 1 --2- 172.21.15.5:0/1523758522 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd64162820 0x7efd640cbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:05.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.172+0000 7efd69b7f640 1 -- 172.21.15.5:0/1523758522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd640b3e80 msgr2=0x7efd641f50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:05.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.172+0000 7efd69b7f640 1 --2- 172.21.15.5:0/1523758522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd640b3e80 0x7efd641f50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:05.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.172+0000 7efd69b7f640 1 -- 172.21.15.5:0/1523758522 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd640b5060 con 0x7efd6406d910 2024-05-20T18:38:05.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.172+0000 7efd69b7f640 1 --2- 172.21.15.5:0/1523758522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd6406d910 0x7efd640caaa0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7efd60000c00 tx=0x7efd600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:05.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.172+0000 7efd5a7fc640 1 -- 172.21.15.5:0/1523758522 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60041020 con 0x7efd6406d910 2024-05-20T18:38:05.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.172+0000 7efd5a7fc640 1 -- 172.21.15.5:0/1523758522 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd600306c0 con 0x7efd6406d910 2024-05-20T18:38:05.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.172+0000 7efd6bde3640 1 -- 172.21.15.5:0/1523758522 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd640b5340 con 0x7efd6406d910 2024-05-20T18:38:05.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.172+0000 7efd5a7fc640 1 -- 172.21.15.5:0/1523758522 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60037040 con 0x7efd6406d910 2024-05-20T18:38:05.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.172+0000 7efd6bde3640 1 -- 172.21.15.5:0/1523758522 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd640b5900 con 0x7efd6406d910 2024-05-20T18:38:05.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.173+0000 7efd6bde3640 1 -- 172.21.15.5:0/1523758522 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd6408dc30 con 0x7efd6406d910 2024-05-20T18:38:05.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.174+0000 7efd5a7fc640 1 -- 172.21.15.5:0/1523758522 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efd60044020 con 0x7efd6406d910 2024-05-20T18:38:05.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.174+0000 7efd5a7fc640 1 --2- 172.21.15.5:0/1523758522 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efd4806be30 0x7efd4806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:05.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.174+0000 7efd5a7fc640 1 -- 172.21.15.5:0/1523758522 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7efd600ab670 con 0x7efd6406d910 2024-05-20T18:38:05.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.174+0000 7efd6937e640 1 --2- 172.21.15.5:0/1523758522 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efd4806be30 0x7efd4806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:05.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.175+0000 7efd6937e640 1 --2- 172.21.15.5:0/1523758522 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efd4806be30 0x7efd4806e340 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7efd54006fd0 tx=0x7efd54008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:05.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.179+0000 7efd5a7fc640 1 -- 172.21.15.5:0/1523758522 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7efd6408dc30 con 0x7efd6406d910 2024-05-20T18:38:05.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.360+0000 7efd6bde3640 1 -- 172.21.15.5:0/1523758522 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7efd64162430 con 0x7efd4806be30 2024-05-20T18:38:05.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.360+0000 7efd5a7fc640 1 -- 172.21.15.5:0/1523758522 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7efd64162430 con 0x7efd4806be30 2024-05-20T18:38:05.362 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 7 to deep-scrub 2024-05-20T18:38:05.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.362+0000 7efd6bde3640 1 -- 172.21.15.5:0/1523758522 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efd4806be30 msgr2=0x7efd4806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:05.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.362+0000 7efd6bde3640 1 --2- 172.21.15.5:0/1523758522 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efd4806be30 0x7efd4806e340 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7efd54006fd0 tx=0x7efd54008040 comp rx=0 tx=0).stop 2024-05-20T18:38:05.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.362+0000 7efd6bde3640 1 -- 172.21.15.5:0/1523758522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd6406d910 msgr2=0x7efd640caaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:05.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.362+0000 7efd6bde3640 1 --2- 172.21.15.5:0/1523758522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd6406d910 0x7efd640caaa0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7efd60000c00 tx=0x7efd600046c0 comp rx=0 tx=0).stop 2024-05-20T18:38:05.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.362+0000 7efd6bde3640 1 -- 172.21.15.5:0/1523758522 shutdown_connections 2024-05-20T18:38:05.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.362+0000 7efd6bde3640 1 --2- 172.21.15.5:0/1523758522 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7efd4806be30 0x7efd4806e340 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:05.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.362+0000 7efd6bde3640 1 --2- 172.21.15.5:0/1523758522 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd64162820 0x7efd640cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:05.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.362+0000 7efd6bde3640 1 --2- 172.21.15.5:0/1523758522 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd640b3e80 0x7efd641f50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:05.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.362+0000 7efd6bde3640 1 --2- 172.21.15.5:0/1523758522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd6406d910 0x7efd640caaa0 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:05.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.362+0000 7efd6bde3640 1 -- 172.21.15.5:0/1523758522 >> 172.21.15.5:0/1523758522 conn(0x7efd640d4bd0 msgr2=0x7efd6415d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:05.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.363+0000 7efd6bde3640 1 -- 172.21.15.5:0/1523758522 shutdown_connections 2024-05-20T18:38:05.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.363+0000 7efd6bde3640 1 -- 172.21.15.5:0/1523758522 wait complete. 2024-05-20T18:38:05.375 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T18:38:05.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.488+0000 7f53eb557640 1 Processor -- start 2024-05-20T18:38:05.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.488+0000 7f53eb557640 1 -- start start 2024-05-20T18:38:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.489+0000 7f53eb557640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f53e4162820 0x7f53e40b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.489+0000 7f53eb557640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f53e406dae0 0x7f53e40b8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.489+0000 7f53eb557640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53e40d0630 con 0x7f53e40b3e80 2024-05-20T18:38:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.490+0000 7f53eb557640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f53e40d39c0 con 0x7f53e4162820 2024-05-20T18:38:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.490+0000 7f53eb557640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f53e40d3ca0 con 0x7f53e406dae0 2024-05-20T18:38:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.490+0000 7f53e8af2640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f53e40b3e80 0x7f53e406d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:42048/0 (socket says 172.21.15.5:42048) 2024-05-20T18:38:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.490+0000 7f53e8af2640 1 -- 172.21.15.5:0/898828508 learned_addr learned my addr 172.21.15.5:0/898828508 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:38:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.490+0000 7f53e92f3640 1 --2- 172.21.15.5:0/898828508 >> v2:172.21.15.63:3300/0 conn(0x7f53e4162820 0x7f53e40b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.490+0000 7f53e9af4640 1 --2- 172.21.15.5:0/898828508 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f53e406dae0 0x7f53e40b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.490+0000 7f53dbfff640 1 -- 172.21.15.5:0/898828508 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2335143023 0 0) 0x7f53e40d0630 con 0x7f53e40b3e80 2024-05-20T18:38:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.490+0000 7f53dbfff640 1 -- 172.21.15.5:0/898828508 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53d0003520 con 0x7f53e40b3e80 2024-05-20T18:38:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.490+0000 7f53e92f3640 1 -- 172.21.15.5:0/898828508 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f53e406dae0 msgr2=0x7f53e40b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:38:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.490+0000 7f53e92f3640 1 --2- 172.21.15.5:0/898828508 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f53e406dae0 0x7f53e40b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.490+0000 7f53e92f3640 1 -- 172.21.15.5:0/898828508 >> v1:172.21.15.5:6789/0 conn(0x7f53e40b3e80 legacy=0x7f53e406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.490+0000 7f53e92f3640 1 -- 172.21.15.5:0/898828508 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e40d3f80 con 0x7f53e4162820 2024-05-20T18:38:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.490+0000 7f53e9af4640 1 --2- 172.21.15.5:0/898828508 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f53e406dae0 0x7f53e40b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-20T18:38:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.491+0000 7f53e92f3640 1 --2- 172.21.15.5:0/898828508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e4162820 0x7f53e40b3940 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f53e00097a0 tx=0x7f53e002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d138be8e83ed67ca server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.491+0000 7f53dbfff640 1 -- 172.21.15.5:0/898828508 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0037070 con 0x7f53e4162820 2024-05-20T18:38:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.491+0000 7f53dbfff640 1 -- 172.21.15.5:0/898828508 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53e0032440 con 0x7f53e4162820 2024-05-20T18:38:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.491+0000 7f53dbfff640 1 -- 172.21.15.5:0/898828508 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e00327f0 con 0x7f53e4162820 2024-05-20T18:38:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.492+0000 7f53eb557640 1 -- 172.21.15.5:0/898828508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e4162820 msgr2=0x7f53e40b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.492+0000 7f53eb557640 1 --2- 172.21.15.5:0/898828508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e4162820 0x7f53e40b3940 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f53e00097a0 tx=0x7f53e002a180 comp rx=0 tx=0).stop 2024-05-20T18:38:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.492+0000 7f53eb557640 1 -- 172.21.15.5:0/898828508 shutdown_connections 2024-05-20T18:38:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.492+0000 7f53eb557640 1 --2- 172.21.15.5:0/898828508 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f53e406dae0 0x7f53e40b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.492+0000 7f53eb557640 1 --2- 172.21.15.5:0/898828508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e4162820 0x7f53e40b3940 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.492+0000 7f53eb557640 1 -- 172.21.15.5:0/898828508 >> 172.21.15.5:0/898828508 conn(0x7f53e40d4bd0 msgr2=0x7f53e40d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:05.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.492+0000 7f53eb557640 1 -- 172.21.15.5:0/898828508 shutdown_connections 2024-05-20T18:38:05.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.492+0000 7f53eb557640 1 -- 172.21.15.5:0/898828508 wait complete. 2024-05-20T18:38:05.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.494+0000 7f53eb557640 1 Processor -- start 2024-05-20T18:38:05.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.494+0000 7f53eb557640 1 -- start start 2024-05-20T18:38:05.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.495+0000 7f53eb557640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e40b3e80 0x7f53e41f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:05.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.495+0000 7f53eb557640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53e4162820 0x7f53e4158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:05.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.495+0000 7f53eb557640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f53e40b6a30 0x7f53e4149e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:05.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.495+0000 7f53eb557640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53e40d4780 con 0x7f53e4162820 2024-05-20T18:38:05.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.495+0000 7f53eb557640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f53e40d0300 con 0x7f53e40b3e80 2024-05-20T18:38:05.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.495+0000 7f53eb557640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f53e40d4900 con 0x7f53e40b6a30 2024-05-20T18:38:05.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.495+0000 7f53e8af2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53e4162820 0x7f53e4158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:05.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.495+0000 7f53e8af2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53e4162820 0x7f53e4158ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35054/0 (socket says 172.21.15.5:35054) 2024-05-20T18:38:05.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.495+0000 7f53e8af2640 1 -- 172.21.15.5:0/3928537396 learned_addr learned my addr 172.21.15.5:0/3928537396 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:38:05.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.495+0000 7f53e92f3640 1 --2- 172.21.15.5:0/3928537396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e40b3e80 0x7f53e41f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:05.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.495+0000 7f53e9af4640 1 --2- 172.21.15.5:0/3928537396 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f53e40b6a30 0x7f53e4149e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:05.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.496+0000 7f53e8af2640 1 -- 172.21.15.5:0/3928537396 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f53e40b6a30 msgr2=0x7f53e4149e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:05.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.496+0000 7f53e8af2640 1 --2- 172.21.15.5:0/3928537396 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f53e40b6a30 0x7f53e4149e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:05.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.496+0000 7f53e8af2640 1 -- 172.21.15.5:0/3928537396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e40b3e80 msgr2=0x7f53e41f50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:05.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.496+0000 7f53e8af2640 1 --2- 172.21.15.5:0/3928537396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e40b3e80 0x7f53e41f50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:05.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.496+0000 7f53e8af2640 1 -- 172.21.15.5:0/3928537396 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e4159400 con 0x7f53e4162820 2024-05-20T18:38:05.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.496+0000 7f53e8af2640 1 --2- 172.21.15.5:0/3928537396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53e4162820 0x7f53e4158ec0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f53d400c3c0 tx=0x7f53d400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:05.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.496+0000 7f53d9ffb640 1 -- 172.21.15.5:0/3928537396 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d4017070 con 0x7f53e4162820 2024-05-20T18:38:05.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.496+0000 7f53d9ffb640 1 -- 172.21.15.5:0/3928537396 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53d4005ce0 con 0x7f53e4162820 2024-05-20T18:38:05.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.496+0000 7f53d9ffb640 1 -- 172.21.15.5:0/3928537396 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d40072f0 con 0x7f53e4162820 2024-05-20T18:38:05.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.496+0000 7f53eb557640 1 -- 172.21.15.5:0/3928537396 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e415bfe0 con 0x7f53e4162820 2024-05-20T18:38:05.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.496+0000 7f53eb557640 1 -- 172.21.15.5:0/3928537396 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e40b6230 con 0x7f53e4162820 2024-05-20T18:38:05.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.497+0000 7f53eb557640 1 -- 172.21.15.5:0/3928537396 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53e40b79e0 con 0x7f53e4162820 2024-05-20T18:38:05.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.498+0000 7f53d9ffb640 1 -- 172.21.15.5:0/3928537396 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f53d401a070 con 0x7f53e4162820 2024-05-20T18:38:05.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.498+0000 7f53d9ffb640 1 --2- 172.21.15.5:0/3928537396 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f53c806be30 0x7f53c806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:05.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.498+0000 7f53d9ffb640 1 -- 172.21.15.5:0/3928537396 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f53d408a200 con 0x7f53e4162820 2024-05-20T18:38:05.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.499+0000 7f53e92f3640 1 --2- 172.21.15.5:0/3928537396 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f53c806be30 0x7f53c806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:05.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.499+0000 7f53e92f3640 1 --2- 172.21.15.5:0/3928537396 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f53c806be30 0x7f53c806e340 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f53e0034000 tx=0x7f53e002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:05.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.503+0000 7f53d9ffb640 1 -- 172.21.15.5:0/3928537396 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f53e40b79e0 con 0x7f53e4162820 2024-05-20T18:38:05.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.685+0000 7f53eb557640 1 -- 172.21.15.5:0/3928537396 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f53e4072f60 con 0x7f53c806be30 2024-05-20T18:38:05.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.685+0000 7f53d9ffb640 1 -- 172.21.15.5:0/3928537396 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13486 (secure 0 0 0) 0x7f53e4072f60 con 0x7f53c806be30 2024-05-20T18:38:05.687 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:38:05.687 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-20T18:38:05.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.687+0000 7f53eb557640 1 -- 172.21.15.5:0/3928537396 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f53c806be30 msgr2=0x7f53c806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:05.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.687+0000 7f53eb557640 1 --2- 172.21.15.5:0/3928537396 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f53c806be30 0x7f53c806e340 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f53e0034000 tx=0x7f53e002e000 comp rx=0 tx=0).stop 2024-05-20T18:38:05.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.687+0000 7f53eb557640 1 -- 172.21.15.5:0/3928537396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53e4162820 msgr2=0x7f53e4158ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:05.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.687+0000 7f53eb557640 1 --2- 172.21.15.5:0/3928537396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53e4162820 0x7f53e4158ec0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f53d400c3c0 tx=0x7f53d400c880 comp rx=0 tx=0).stop 2024-05-20T18:38:05.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.687+0000 7f53eb557640 1 -- 172.21.15.5:0/3928537396 shutdown_connections 2024-05-20T18:38:05.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.687+0000 7f53eb557640 1 --2- 172.21.15.5:0/3928537396 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f53c806be30 0x7f53c806e340 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:05.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.687+0000 7f53eb557640 1 --2- 172.21.15.5:0/3928537396 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f53e40b6a30 0x7f53e4149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:05.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.687+0000 7f53eb557640 1 --2- 172.21.15.5:0/3928537396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53e4162820 0x7f53e4158ec0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:05.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.687+0000 7f53eb557640 1 --2- 172.21.15.5:0/3928537396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53e40b3e80 0x7f53e41f50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:05.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.687+0000 7f53eb557640 1 -- 172.21.15.5:0/3928537396 >> 172.21.15.5:0/3928537396 conn(0x7f53e40d4bd0 msgr2=0x7f53e40b8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:05.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.688+0000 7f53eb557640 1 -- 172.21.15.5:0/3928537396 shutdown_connections 2024-05-20T18:38:05.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:05.688+0000 7f53eb557640 1 -- 172.21.15.5:0/3928537396 wait complete. 2024-05-20T18:38:05.700 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":449,"stamp":"2024-05-20T18:38:04.742998+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":266916,"kb_used_data":13288,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754707804,"statfs":{"total":773094113280,"available":772820791296,"internally_reserved":0,"allocated":13606912,"data_stored":6975096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8388625,"num_objects":-5,"num_object_clones":0,"num_object_copies":-10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5,"num_whiteouts":0,"num_read":-32,"num_read_kb":-13,"num_write":-29,"num_write_kb":-8195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.474464"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":362,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-05-20T18:37:58.167566+0000","last_change":"2024-05-20T18:27:03.765694+0000","last_active":"2024-05-20T18:37:58.167566+0000","last_peered":"2024-05-20T18:37:58.167566+0000","last_clean":"2024-05-20T18:37:58.167566+0000","last_became_active":"2024-05-20T18:27:03.764695+0000","last_became_peered":"2024-05-20T18:27:03.764695+0000","last_unstale":"2024-05-20T18:37:58.167566+0000","last_undegraded":"2024-05-20T18:37:58.167566+0000","last_fullsized":"2024-05-20T18:37:58.167566+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-20T18:27:02.750853+0000","last_clean_scrub_stamp":"2024-05-20T18:27:02.750853+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-21T21:30:04.407487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574984,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33280,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2146304,"data_stored":1314663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574982,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32700,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1552384,"data_stored":724295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574982,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32700,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1552384,"data_stored":724295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574982,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32700,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1552384,"data_stored":724295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574984,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32700,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1552384,"data_stored":724295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574983,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32696,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339144,"statfs":{"total":96636764160,"available":96603283456,"internally_reserved":0,"allocated":1552384,"data_stored":724295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574984,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32700,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1552384,"data_stored":724295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574983,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37440,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334400,"statfs":{"total":96636764160,"available":96598425600,"internally_reserved":0,"allocated":2146304,"data_stored":1314663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T18:38:05.703 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-20T18:27:02.750853+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=20, tm_hour=18, tm_min=27, tm_sec=2, tm_wday=0, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=20, tm_hour=18, tm_min=38, tm_sec=0, tm_wday=0, tm_yday=141, tm_isdst=0) 2024-05-20T18:38:05.703 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-20T18:38:12.174 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:38:12.178 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:38:25.704 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-20T18:38:25.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.832+0000 7f5aea6c3640 1 Processor -- start 2024-05-20T18:38:25.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.832+0000 7f5aea6c3640 1 -- start start 2024-05-20T18:38:25.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.834+0000 7f5aea6c3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5ae415d060 0x7f5ae40b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:25.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.834+0000 7f5aea6c3640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ae40b5b30 con 0x7f5ae40b5720 2024-05-20T18:38:25.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.834+0000 7f5aea6c3640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ae40b39f0 con 0x7f5ae40b47b0 2024-05-20T18:38:25.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.834+0000 7f5aea6c3640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f5ae40b3cd0 con 0x7f5ae415d060 2024-05-20T18:38:25.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.834+0000 7f5ae37fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5ae415d060 0x7f5ae40b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:25.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.834+0000 7f5ae8c60640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5ae40b5720 0x7f5ae406f3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37458/0 (socket says 172.21.15.5:37458) 2024-05-20T18:38:25.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.834+0000 7f5ae8c60640 1 -- 172.21.15.5:0/3528241195 learned_addr learned my addr 172.21.15.5:0/3528241195 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:38:25.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.834+0000 7f5ae37fe640 1 -- 172.21.15.5:0/3528241195 >> v1:172.21.15.63:6789/0 conn(0x7f5ae40b47b0 legacy=0x7f5ae40afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:38:25.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.834+0000 7f5ae37fe640 1 -- 172.21.15.5:0/3528241195 >> v1:172.21.15.5:6789/0 conn(0x7f5ae40b5720 legacy=0x7f5ae406f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:25.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.834+0000 7f5ae37fe640 1 -- 172.21.15.5:0/3528241195 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae40b3fb0 con 0x7f5ae415d060 2024-05-20T18:38:25.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.834+0000 7f5ae2ffd640 1 -- 172.21.15.5:0/3528241195 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3182505169 0 0) 0x7f5adc001490 con 0x7f5ae40b5720 2024-05-20T18:38:25.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.835+0000 7f5ae37fe640 1 --2- 172.21.15.5:0/3528241195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ae415d060 0x7f5ae40b51e0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f5ad80097a0 tx=0x7f5ad802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b077e2875b9593df server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:25.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.835+0000 7f5ae2ffd640 1 -- 172.21.15.5:0/3528241195 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8037070 con 0x7f5ae415d060 2024-05-20T18:38:25.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.835+0000 7f5ae2ffd640 1 -- 172.21.15.5:0/3528241195 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ad8032440 con 0x7f5ae415d060 2024-05-20T18:38:25.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.835+0000 7f5ae2ffd640 1 -- 172.21.15.5:0/3528241195 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8032810 con 0x7f5ae415d060 2024-05-20T18:38:25.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.836+0000 7f5aea6c3640 1 -- 172.21.15.5:0/3528241195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ae415d060 msgr2=0x7f5ae40b51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:25.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.836+0000 7f5aea6c3640 1 --2- 172.21.15.5:0/3528241195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ae415d060 0x7f5ae40b51e0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f5ad80097a0 tx=0x7f5ad802a1e0 comp rx=0 tx=0).stop 2024-05-20T18:38:25.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.836+0000 7f5aea6c3640 1 -- 172.21.15.5:0/3528241195 shutdown_connections 2024-05-20T18:38:25.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.836+0000 7f5aea6c3640 1 --2- 172.21.15.5:0/3528241195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ae415d060 0x7f5ae40b51e0 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:25.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.836+0000 7f5aea6c3640 1 -- 172.21.15.5:0/3528241195 >> 172.21.15.5:0/3528241195 conn(0x7f5ae40cd2a0 msgr2=0x7f5ae40cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:25.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.836+0000 7f5aea6c3640 1 -- 172.21.15.5:0/3528241195 shutdown_connections 2024-05-20T18:38:25.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.836+0000 7f5aea6c3640 1 -- 172.21.15.5:0/3528241195 wait complete. 2024-05-20T18:38:25.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.838+0000 7f5aea6c3640 1 Processor -- start 2024-05-20T18:38:25.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.838+0000 7f5aea6c3640 1 -- start start 2024-05-20T18:38:25.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.839+0000 7f5aea6c3640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ae40b5720 0x7f5ae4167e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:25.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.839+0000 7f5aea6c3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ae415d060 0x7f5ae414cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:25.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.839+0000 7f5aea6c3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ae414e310 0x7f5ae414c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:25.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.839+0000 7f5aea6c3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae4159de0 con 0x7f5ae414e310 2024-05-20T18:38:25.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.839+0000 7f5aea6c3640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae4159c60 con 0x7f5ae40b5720 2024-05-20T18:38:25.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.839+0000 7f5aea6c3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5ae406fae0 con 0x7f5ae415d060 2024-05-20T18:38:25.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.840+0000 7f5ae8c60640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ae414e310 0x7f5ae414c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:25.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.840+0000 7f5ae8c60640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ae414e310 0x7f5ae414c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41532/0 (socket says 172.21.15.5:41532) 2024-05-20T18:38:25.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.840+0000 7f5ae8c60640 1 -- 172.21.15.5:0/1124252849 learned_addr learned my addr 172.21.15.5:0/1124252849 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:38:25.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.840+0000 7f5ae37fe640 1 --2- 172.21.15.5:0/1124252849 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ae415d060 0x7f5ae414cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:25.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.840+0000 7f5ae3fff640 1 --2- 172.21.15.5:0/1124252849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ae40b5720 0x7f5ae4167e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:25.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.840+0000 7f5ae8c60640 1 -- 172.21.15.5:0/1124252849 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ae415d060 msgr2=0x7f5ae414cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:25.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.840+0000 7f5ae8c60640 1 --2- 172.21.15.5:0/1124252849 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ae415d060 0x7f5ae414cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:25.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.840+0000 7f5ae8c60640 1 -- 172.21.15.5:0/1124252849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ae40b5720 msgr2=0x7f5ae4167e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:25.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.840+0000 7f5ae8c60640 1 --2- 172.21.15.5:0/1124252849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ae40b5720 0x7f5ae4167e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:25.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.840+0000 7f5ae8c60640 1 -- 172.21.15.5:0/1124252849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae414d2f0 con 0x7f5ae414e310 2024-05-20T18:38:25.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.840+0000 7f5ae8c60640 1 --2- 172.21.15.5:0/1124252849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ae414e310 0x7f5ae414c600 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f5adc00c700 tx=0x7f5adc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:25.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.840+0000 7f5ae0ff9640 1 -- 172.21.15.5:0/1124252849 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5adc018070 con 0x7f5ae414e310 2024-05-20T18:38:25.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.840+0000 7f5ae0ff9640 1 -- 172.21.15.5:0/1124252849 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5adc00ce10 con 0x7f5ae414e310 2024-05-20T18:38:25.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.840+0000 7f5aea6c3640 1 -- 172.21.15.5:0/1124252849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ae4150160 con 0x7f5ae414e310 2024-05-20T18:38:25.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.840+0000 7f5ae0ff9640 1 -- 172.21.15.5:0/1124252849 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5adc00a860 con 0x7f5ae414e310 2024-05-20T18:38:25.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.841+0000 7f5aea6c3640 1 -- 172.21.15.5:0/1124252849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ae4168950 con 0x7f5ae414e310 2024-05-20T18:38:25.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.842+0000 7f5aea6c3640 1 -- 172.21.15.5:0/1124252849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ae406efb0 con 0x7f5ae414e310 2024-05-20T18:38:25.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.842+0000 7f5ae0ff9640 1 -- 172.21.15.5:0/1124252849 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5adc01b070 con 0x7f5ae414e310 2024-05-20T18:38:25.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.843+0000 7f5ae0ff9640 1 --2- 172.21.15.5:0/1124252849 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5ad006be30 0x7f5ad006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:25.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.843+0000 7f5ae3fff640 1 --2- 172.21.15.5:0/1124252849 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5ad006be30 0x7f5ad006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:25.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.843+0000 7f5ae0ff9640 1 -- 172.21.15.5:0/1124252849 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f5adc08a2e0 con 0x7f5ae414e310 2024-05-20T18:38:25.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.844+0000 7f5ae3fff640 1 --2- 172.21.15.5:0/1124252849 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5ad006be30 0x7f5ad006e340 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4006fd0 tx=0x7f5ad4009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:25.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:25.848+0000 7f5ae0ff9640 1 -- 172.21.15.5:0/1124252849 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5ae406efb0 con 0x7f5ae414e310 2024-05-20T18:38:26.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.028+0000 7f5aea6c3640 1 -- 172.21.15.5:0/1124252849 --> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5ae4075b00 con 0x7f5ad006be30 2024-05-20T18:38:26.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.028+0000 7f5ae0ff9640 1 -- 172.21.15.5:0/1124252849 <== mgr.4101 v2:172.21.15.5:6832/3330161497 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13379 (secure 0 0 0) 0x7f5ae4075b00 con 0x7f5ad006be30 2024-05-20T18:38:26.030 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:38:26.031 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-20T18:38:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.030+0000 7f5aea6c3640 1 -- 172.21.15.5:0/1124252849 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5ad006be30 msgr2=0x7f5ad006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.030+0000 7f5aea6c3640 1 --2- 172.21.15.5:0/1124252849 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5ad006be30 0x7f5ad006e340 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4006fd0 tx=0x7f5ad4009060 comp rx=0 tx=0).stop 2024-05-20T18:38:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.030+0000 7f5aea6c3640 1 -- 172.21.15.5:0/1124252849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ae414e310 msgr2=0x7f5ae414c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.030+0000 7f5aea6c3640 1 --2- 172.21.15.5:0/1124252849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ae414e310 0x7f5ae414c600 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f5adc00c700 tx=0x7f5adc00cbc0 comp rx=0 tx=0).stop 2024-05-20T18:38:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.030+0000 7f5aea6c3640 1 -- 172.21.15.5:0/1124252849 shutdown_connections 2024-05-20T18:38:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.030+0000 7f5aea6c3640 1 --2- 172.21.15.5:0/1124252849 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f5ad006be30 0x7f5ad006e340 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.030+0000 7f5aea6c3640 1 --2- 172.21.15.5:0/1124252849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ae414e310 0x7f5ae414c600 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:26.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.030+0000 7f5aea6c3640 1 --2- 172.21.15.5:0/1124252849 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ae415d060 0x7f5ae414cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:26.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.030+0000 7f5aea6c3640 1 --2- 172.21.15.5:0/1124252849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ae40b5720 0x7f5ae4167e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:26.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.030+0000 7f5aea6c3640 1 -- 172.21.15.5:0/1124252849 >> 172.21.15.5:0/1124252849 conn(0x7f5ae40cd2a0 msgr2=0x7f5ae415b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:26.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.031+0000 7f5aea6c3640 1 -- 172.21.15.5:0/1124252849 shutdown_connections 2024-05-20T18:38:26.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.031+0000 7f5aea6c3640 1 -- 172.21.15.5:0/1124252849 wait complete. 2024-05-20T18:38:26.043 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":459,"stamp":"2024-05-20T18:38:24.747433+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":266900,"kb_used_data":13288,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754707820,"statfs":{"total":773094113280,"available":772820807680,"internally_reserved":0,"allocated":13606912,"data_stored":6975096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002620"},"pg_stats":[{"pgid":"1.0","version":"124'119","reported_seq":373,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-05-20T18:38:05.610072+0000","last_change":"2024-05-20T18:38:05.610156+0000","last_active":"2024-05-20T18:38:05.610072+0000","last_peered":"2024-05-20T18:38:05.610072+0000","last_clean":"2024-05-20T18:38:05.610072+0000","last_became_active":"2024-05-20T18:27:03.764695+0000","last_became_peered":"2024-05-20T18:27:03.764695+0000","last_unstale":"2024-05-20T18:38:05.610072+0000","last_undegraded":"2024-05-20T18:38:05.610072+0000","last_fullsized":"2024-05-20T18:38:05.610072+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"124'119","last_scrub_stamp":"2024-05-20T18:38:05.609936+0000","last_deep_scrub":"124'119","last_deep_scrub_stamp":"2024-05-20T18:38:05.609936+0000","last_clean_scrub_stamp":"2024-05-20T18:38:05.609936+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T05:54:34.143190+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574988,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33272,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338568,"statfs":{"total":96636764160,"available":96602693632,"internally_reserved":0,"allocated":2146304,"data_stored":1314663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574986,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32700,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1552384,"data_stored":724295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574986,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32700,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1552384,"data_stored":724295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574986,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32700,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1552384,"data_stored":724295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32700,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1552384,"data_stored":724295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32696,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339144,"statfs":{"total":96636764160,"available":96603283456,"internally_reserved":0,"allocated":1552384,"data_stored":724295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32700,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1552384,"data_stored":724295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574987,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37432,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334408,"statfs":{"total":96636764160,"available":96598433792,"internally_reserved":0,"allocated":2146304,"data_stored":1314663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-20T18:38:26.044 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-20T18:38:26.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.157+0000 7f236b7b5640 1 Processor -- start 2024-05-20T18:38:26.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.157+0000 7f236b7b5640 1 -- start start 2024-05-20T18:38:26.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.158+0000 7f236b7b5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f236415d530 0x7f236415aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:26.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.158+0000 7f236b7b5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2364070850 0x7f2364070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:26.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.158+0000 7f236b7b5640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23640cd580 con 0x7f23640b1700 2024-05-20T18:38:26.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.158+0000 7f236b7b5640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f23640ce190 con 0x7f236415d530 2024-05-20T18:38:26.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.158+0000 7f236b7b5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f23640cd860 con 0x7f2364070850 2024-05-20T18:38:26.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.158+0000 7f2369551640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f236415d530 0x7f236415aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:26.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.158+0000 7f2369551640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f236415d530 0x7f236415aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41538/0 (socket says 172.21.15.5:41538) 2024-05-20T18:38:26.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.158+0000 7f2369551640 1 -- 172.21.15.5:0/2010492997 learned_addr learned my addr 172.21.15.5:0/2010492997 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:38:26.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.159+0000 7f2369d52640 1 --2- 172.21.15.5:0/2010492997 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2364070850 0x7f2364070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:26.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.159+0000 7f2369551640 1 -- 172.21.15.5:0/2010492997 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2364070850 msgr2=0x7f2364070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:38:26.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.159+0000 7f2369551640 1 --2- 172.21.15.5:0/2010492997 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2364070850 0x7f2364070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:26.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.159+0000 7f2369551640 1 -- 172.21.15.5:0/2010492997 >> v1:172.21.15.63:6789/0 conn(0x7f23640b1700 legacy=0x7f23640b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:26.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.159+0000 7f2369551640 1 -- 172.21.15.5:0/2010492997 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23640cdb40 con 0x7f236415d530 2024-05-20T18:38:26.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.159+0000 7f2369551640 1 --2- 172.21.15.5:0/2010492997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f236415d530 0x7f236415aaa0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f2354009e50 tx=0x7f235402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db47ef9d326c4d72 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:26.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.160+0000 7f2363fff640 1 -- 172.21.15.5:0/2010492997 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2354036070 con 0x7f236415d530 2024-05-20T18:38:26.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.160+0000 7f2363fff640 1 -- 172.21.15.5:0/2010492997 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2354031600 con 0x7f236415d530 2024-05-20T18:38:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.160+0000 7f2363fff640 1 -- 172.21.15.5:0/2010492997 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23540319d0 con 0x7f236415d530 2024-05-20T18:38:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.160+0000 7f236b7b5640 1 -- 172.21.15.5:0/2010492997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f236415d530 msgr2=0x7f236415aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.160+0000 7f236b7b5640 1 --2- 172.21.15.5:0/2010492997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f236415d530 0x7f236415aaa0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f2354009e50 tx=0x7f235402a710 comp rx=0 tx=0).stop 2024-05-20T18:38:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.160+0000 7f236b7b5640 1 -- 172.21.15.5:0/2010492997 shutdown_connections 2024-05-20T18:38:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.160+0000 7f236b7b5640 1 --2- 172.21.15.5:0/2010492997 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2364070850 0x7f2364070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.160+0000 7f236b7b5640 1 --2- 172.21.15.5:0/2010492997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f236415d530 0x7f236415aaa0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.160+0000 7f236b7b5640 1 -- 172.21.15.5:0/2010492997 >> 172.21.15.5:0/2010492997 conn(0x7f23640d29d0 msgr2=0x7f23640d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.160+0000 7f236b7b5640 1 -- 172.21.15.5:0/2010492997 shutdown_connections 2024-05-20T18:38:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.160+0000 7f236b7b5640 1 -- 172.21.15.5:0/2010492997 wait complete. 2024-05-20T18:38:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.162+0000 7f236b7b5640 1 Processor -- start 2024-05-20T18:38:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.162+0000 7f236b7b5640 1 -- start start 2024-05-20T18:38:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.163+0000 7f236b7b5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2364070850 0x7f23641f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.164+0000 7f236b7b5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23640b1700 0x7f23640c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.164+0000 7f236b7b5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f236415d530 0x7f23640b6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.164+0000 7f236b7b5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f23640711a0 con 0x7f23640b1700 2024-05-20T18:38:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.164+0000 7f236b7b5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f23640cd230 con 0x7f236415d530 2024-05-20T18:38:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.164+0000 7f236b7b5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2364071320 con 0x7f2364070850 2024-05-20T18:38:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.164+0000 7f2368d50640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23640b1700 0x7f23640c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.164+0000 7f2368d50640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23640b1700 0x7f23640c8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41544/0 (socket says 172.21.15.5:41544) 2024-05-20T18:38:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.164+0000 7f2368d50640 1 -- 172.21.15.5:0/3828250697 learned_addr learned my addr 172.21.15.5:0/3828250697 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:38:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.164+0000 7f2369551640 1 --2- 172.21.15.5:0/3828250697 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2364070850 0x7f23641f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.164+0000 7f2369d52640 1 --2- 172.21.15.5:0/3828250697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f236415d530 0x7f23640b6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.164+0000 7f2368d50640 1 -- 172.21.15.5:0/3828250697 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2364070850 msgr2=0x7f23641f4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.164+0000 7f2368d50640 1 --2- 172.21.15.5:0/3828250697 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2364070850 0x7f23641f4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.164+0000 7f2368d50640 1 -- 172.21.15.5:0/3828250697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f236415d530 msgr2=0x7f23640b6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.164+0000 7f2368d50640 1 --2- 172.21.15.5:0/3828250697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f236415d530 0x7f23640b6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.164+0000 7f2368d50640 1 -- 172.21.15.5:0/3828250697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236406c6d0 con 0x7f23640b1700 2024-05-20T18:38:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.164+0000 7f2369d52640 1 --2- 172.21.15.5:0/3828250697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f236415d530 0x7f23640b6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:38:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.164+0000 7f2369551640 1 --2- 172.21.15.5:0/3828250697 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2364070850 0x7f23641f4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:38:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.164+0000 7f2368d50640 1 --2- 172.21.15.5:0/3828250697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23640b1700 0x7f23640c8310 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f235800c3f0 tx=0x7f235800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.164+0000 7f2361ffb640 1 -- 172.21.15.5:0/3828250697 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23580052d0 con 0x7f23640b1700 2024-05-20T18:38:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.164+0000 7f2361ffb640 1 -- 172.21.15.5:0/3828250697 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2358005490 con 0x7f23640b1700 2024-05-20T18:38:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.164+0000 7f2361ffb640 1 -- 172.21.15.5:0/3828250697 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2358007260 con 0x7f23640b1700 2024-05-20T18:38:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.164+0000 7f236b7b5640 1 -- 172.21.15.5:0/3828250697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f236406c9b0 con 0x7f23640b1700 2024-05-20T18:38:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.165+0000 7f236b7b5640 1 -- 172.21.15.5:0/3828250697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f236406cf70 con 0x7f23640b1700 2024-05-20T18:38:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.166+0000 7f236b7b5640 1 -- 172.21.15.5:0/3828250697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2364075b00 con 0x7f23640b1700 2024-05-20T18:38:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.166+0000 7f2361ffb640 1 -- 172.21.15.5:0/3828250697 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2358007400 con 0x7f23640b1700 2024-05-20T18:38:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.167+0000 7f2361ffb640 1 --2- 172.21.15.5:0/3828250697 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f235006be30 0x7f235006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.167+0000 7f2369551640 1 --2- 172.21.15.5:0/3828250697 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f235006be30 0x7f235006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.167+0000 7f2361ffb640 1 -- 172.21.15.5:0/3828250697 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f235808a300 con 0x7f23640b1700 2024-05-20T18:38:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.167+0000 7f2369551640 1 --2- 172.21.15.5:0/3828250697 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f235006be30 0x7f235006e340 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f2354033000 tx=0x7f2354009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.171+0000 7f2361ffb640 1 -- 172.21.15.5:0/3828250697 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2358058ce0 con 0x7f23640b1700 2024-05-20T18:38:26.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.355+0000 7f236b7b5640 1 -- 172.21.15.5:0/3828250697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f23640d0780 con 0x7f23640b1700 2024-05-20T18:38:26.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.358+0000 7f2361ffb640 1 -- 172.21.15.5:0/3828250697 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) ==== 123+0+0 (secure 0 0 0) 0x7f23580585e0 con 0x7f23640b1700 2024-05-20T18:38:26.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.360+0000 7f2361ffb640 1 -- 172.21.15.5:0/3828250697 <== mon.0 v2:172.21.15.5:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f235805c990 con 0x7f23640b1700 2024-05-20T18:38:26.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.361+0000 7f236b7b5640 1 -- 172.21.15.5:0/3828250697 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f235006be30 msgr2=0x7f235006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:26.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.361+0000 7f236b7b5640 1 --2- 172.21.15.5:0/3828250697 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f235006be30 0x7f235006e340 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f2354033000 tx=0x7f2354009000 comp rx=0 tx=0).stop 2024-05-20T18:38:26.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.361+0000 7f236b7b5640 1 -- 172.21.15.5:0/3828250697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23640b1700 msgr2=0x7f23640c8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:26.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.361+0000 7f236b7b5640 1 --2- 172.21.15.5:0/3828250697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23640b1700 0x7f23640c8310 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f235800c3f0 tx=0x7f235800c8b0 comp rx=0 tx=0).stop 2024-05-20T18:38:26.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.361+0000 7f236b7b5640 1 -- 172.21.15.5:0/3828250697 shutdown_connections 2024-05-20T18:38:26.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.361+0000 7f236b7b5640 1 --2- 172.21.15.5:0/3828250697 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f235006be30 0x7f235006e340 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:26.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.361+0000 7f236b7b5640 1 --2- 172.21.15.5:0/3828250697 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f236415d530 0x7f23640b6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:26.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.361+0000 7f236b7b5640 1 --2- 172.21.15.5:0/3828250697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23640b1700 0x7f23640c8310 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:26.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.361+0000 7f236b7b5640 1 --2- 172.21.15.5:0/3828250697 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2364070850 0x7f23641f4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:26.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.361+0000 7f236b7b5640 1 -- 172.21.15.5:0/3828250697 >> 172.21.15.5:0/3828250697 conn(0x7f23640d29d0 msgr2=0x7f236415e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:26.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.362+0000 7f236b7b5640 1 -- 172.21.15.5:0/3828250697 shutdown_connections 2024-05-20T18:38:26.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.362+0000 7f236b7b5640 1 -- 172.21.15.5:0/3828250697 wait complete. 2024-05-20T18:38:26.375 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-20T18:38:26.375 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-20T18:38:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.488+0000 7f9d58f36640 1 Processor -- start 2024-05-20T18:38:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.488+0000 7f9d58f36640 1 -- start start 2024-05-20T18:38:26.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.489+0000 7f9d58f36640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d540d1bc0 0x7f9d540d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.489+0000 7f9d58f36640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d540d0370 con 0x7f9d540b4010 2024-05-20T18:38:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.489+0000 7f9d58f36640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d540d0650 con 0x7f9d54163a10 2024-05-20T18:38:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.489+0000 7f9d58f36640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9d540d3ca0 con 0x7f9d540d1bc0 2024-05-20T18:38:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.489+0000 7f9d51d9b640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f9d540b4010 0x7f9d540d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37474/0 (socket says 172.21.15.5:37474) 2024-05-20T18:38:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.489+0000 7f9d51d9b640 1 -- 172.21.15.5:0/3233236713 learned_addr learned my addr 172.21.15.5:0/3233236713 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-05-20T18:38:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.489+0000 7f9d5159a640 1 -- 172.21.15.5:0/3233236713 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 710665820 0 0) 0x7f9d540d0370 con 0x7f9d540b4010 2024-05-20T18:38:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.490+0000 7f9d5159a640 1 -- 172.21.15.5:0/3233236713 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d3c003520 con 0x7f9d540b4010 2024-05-20T18:38:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.490+0000 7f9d52d9d640 1 --2- 172.21.15.5:0/3233236713 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d540d1bc0 0x7f9d540d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.490+0000 7f9d5159a640 1 -- 172.21.15.5:0/3233236713 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1143087775 0 0) 0x7f9d3c003520 con 0x7f9d540b4010 2024-05-20T18:38:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.490+0000 7f9d5159a640 1 -- 172.21.15.5:0/3233236713 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d540d1bc0 msgr2=0x7f9d540d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:38:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.490+0000 7f9d5159a640 1 --2- 172.21.15.5:0/3233236713 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d540d1bc0 0x7f9d540d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.490+0000 7f9d5159a640 1 -- 172.21.15.5:0/3233236713 >> v1:172.21.15.63:6789/0 conn(0x7f9d54163a10 legacy=0x7f9d540cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:26.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.490+0000 7f9d5159a640 1 -- 172.21.15.5:0/3233236713 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d540d3650 con 0x7f9d540b4010 2024-05-20T18:38:26.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.490+0000 7f9d52d9d640 1 --2- 172.21.15.5:0/3233236713 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d540d1bc0 0x7f9d540d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:38:26.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.490+0000 7f9d5159a640 1 -- 172.21.15.5:0/3233236713 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3628460141 0 0) 0x7f9d48001670 con 0x7f9d540b4010 2024-05-20T18:38:26.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.490+0000 7f9d5159a640 1 -- 172.21.15.5:0/3233236713 >> v1:172.21.15.5:6789/0 conn(0x7f9d540b4010 legacy=0x7f9d540d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:26.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.490+0000 7f9d5159a640 1 --2- 172.21.15.5:0/3233236713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d3c0039e0 0x7f9d3c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:26.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.490+0000 7f9d5159a640 1 --2- 172.21.15.5:0/3233236713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d3c004800 0x7f9d3c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:26.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.490+0000 7f9d5159a640 1 --2- 172.21.15.5:0/3233236713 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d540d1bc0 0x7f9d3c0052b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:26.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.490+0000 7f9d5159a640 1 -- 172.21.15.5:0/3233236713 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9d540d3ca0 con 0x7f9d3c0039e0 2024-05-20T18:38:26.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.490+0000 7f9d5159a640 1 -- 172.21.15.5:0/3233236713 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c00bb50 con 0x7f9d3c004800 2024-05-20T18:38:26.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.490+0000 7f9d5159a640 1 -- 172.21.15.5:0/3233236713 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9d3c00bd50 con 0x7f9d540d1bc0 2024-05-20T18:38:26.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.490+0000 7f9d5259c640 1 --2- 172.21.15.5:0/3233236713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d3c0039e0 0x7f9d3c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:26.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.490+0000 7f9d51d9b640 1 --2- 172.21.15.5:0/3233236713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d3c004800 0x7f9d3c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:26.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.490+0000 7f9d52d9d640 1 --2- 172.21.15.5:0/3233236713 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d540d1bc0 0x7f9d3c0052b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:26.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.490+0000 7f9d5259c640 1 -- 172.21.15.5:0/3233236713 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d540d1bc0 msgr2=0x7f9d3c0052b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:38:26.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.490+0000 7f9d5259c640 1 --2- 172.21.15.5:0/3233236713 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d540d1bc0 0x7f9d3c0052b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:26.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.490+0000 7f9d5259c640 1 -- 172.21.15.5:0/3233236713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d3c004800 msgr2=0x7f9d3c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-20T18:38:26.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.490+0000 7f9d5259c640 1 --2- 172.21.15.5:0/3233236713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d3c004800 0x7f9d3c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:26.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.490+0000 7f9d5259c640 1 -- 172.21.15.5:0/3233236713 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d3c003520 con 0x7f9d3c0039e0 2024-05-20T18:38:26.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.491+0000 7f9d5259c640 1 --2- 172.21.15.5:0/3233236713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d3c0039e0 0x7f9d3c003e50 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f9d540d3020 tx=0x7f9d3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8d360c2d7c2035 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:26.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.491+0000 7f9d5159a640 1 -- 172.21.15.5:0/3233236713 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d44001370 con 0x7f9d3c0039e0 2024-05-20T18:38:26.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.491+0000 7f9d5159a640 1 -- 172.21.15.5:0/3233236713 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f9d440052d0 con 0x7f9d3c0039e0 2024-05-20T18:38:26.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.491+0000 7f9d5159a640 1 -- 172.21.15.5:0/3233236713 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4402ab90 con 0x7f9d3c0039e0 2024-05-20T18:38:26.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.492+0000 7f9d58f36640 1 -- 172.21.15.5:0/3233236713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d3c0039e0 msgr2=0x7f9d3c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:26.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.492+0000 7f9d58f36640 1 --2- 172.21.15.5:0/3233236713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d3c0039e0 0x7f9d3c003e50 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f9d540d3020 tx=0x7f9d3c005830 comp rx=0 tx=0).stop 2024-05-20T18:38:26.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.492+0000 7f9d52d9d640 1 -- 172.21.15.5:0/3233236713 reap_dead start 2024-05-20T18:38:26.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.492+0000 7f9d58f36640 1 -- 172.21.15.5:0/3233236713 shutdown_connections 2024-05-20T18:38:26.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.492+0000 7f9d58f36640 1 -- 172.21.15.5:0/3233236713 >> 172.21.15.5:0/3233236713 conn(0x7f9d5406d240 msgr2=0x7f9d5406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:26.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.492+0000 7f9d58f36640 1 -- 172.21.15.5:0/3233236713 shutdown_connections 2024-05-20T18:38:26.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.492+0000 7f9d58f36640 1 -- 172.21.15.5:0/3233236713 wait complete. 2024-05-20T18:38:26.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.494+0000 7f9d58f36640 1 Processor -- start 2024-05-20T18:38:26.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.494+0000 7f9d58f36640 1 -- start start 2024-05-20T18:38:26.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.495+0000 7f9d58f36640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d3c0039e0 0x7f9d540cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:26.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.495+0000 7f9d58f36640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d540b5320 0x7f9d540b5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:26.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.495+0000 7f9d58f36640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d541e9860 0x7f9d541e9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:26.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.495+0000 7f9d58f36640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9d540d42b0 con 0x7f9d3c0039e0 2024-05-20T18:38:26.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.495+0000 7f9d58f36640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9d540d4130 con 0x7f9d540b5320 2024-05-20T18:38:26.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.495+0000 7f9d58f36640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9d540d4430 con 0x7f9d541e9860 2024-05-20T18:38:26.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.495+0000 7f9d5259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d3c0039e0 0x7f9d540cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:26.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.496+0000 7f9d5259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d3c0039e0 0x7f9d540cdfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41560/0 (socket says 172.21.15.5:41560) 2024-05-20T18:38:26.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.496+0000 7f9d5259c640 1 -- 172.21.15.5:0/1509942909 learned_addr learned my addr 172.21.15.5:0/1509942909 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-05-20T18:38:26.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.496+0000 7f9d51d9b640 1 --2- 172.21.15.5:0/1509942909 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d540b5320 0x7f9d540b5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:26.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.496+0000 7f9d52d9d640 1 --2- 172.21.15.5:0/1509942909 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d541e9860 0x7f9d541e9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:26.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.496+0000 7f9d5259c640 1 -- 172.21.15.5:0/1509942909 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d541e9860 msgr2=0x7f9d541e9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:26.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.496+0000 7f9d5259c640 1 --2- 172.21.15.5:0/1509942909 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d541e9860 0x7f9d541e9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:26.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.496+0000 7f9d5259c640 1 -- 172.21.15.5:0/1509942909 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d540b5320 msgr2=0x7f9d540b5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:26.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.496+0000 7f9d5259c640 1 --2- 172.21.15.5:0/1509942909 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d540b5320 0x7f9d540b5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:26.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.496+0000 7f9d5259c640 1 -- 172.21.15.5:0/1509942909 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d540b5c70 con 0x7f9d3c0039e0 2024-05-20T18:38:26.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.496+0000 7f9d51d9b640 1 --2- 172.21.15.5:0/1509942909 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d540b5320 0x7f9d540b5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-20T18:38:26.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.496+0000 7f9d5259c640 1 --2- 172.21.15.5:0/1509942909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d3c0039e0 0x7f9d540cdfc0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f9d44008fd0 tx=0x7f9d44001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:26.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.496+0000 7f9d42ffd640 1 -- 172.21.15.5:0/1509942909 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d44042070 con 0x7f9d3c0039e0 2024-05-20T18:38:26.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.496+0000 7f9d42ffd640 1 -- 172.21.15.5:0/1509942909 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f9d44005480 con 0x7f9d3c0039e0 2024-05-20T18:38:26.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.496+0000 7f9d58f36640 1 -- 172.21.15.5:0/1509942909 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d540b60c0 con 0x7f9d3c0039e0 2024-05-20T18:38:26.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.496+0000 7f9d42ffd640 1 -- 172.21.15.5:0/1509942909 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d44037040 con 0x7f9d3c0039e0 2024-05-20T18:38:26.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.496+0000 7f9d58f36640 1 -- 172.21.15.5:0/1509942909 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d541e9060 con 0x7f9d3c0039e0 2024-05-20T18:38:26.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.497+0000 7f9d58f36640 1 -- 172.21.15.5:0/1509942909 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d54075b00 con 0x7f9d3c0039e0 2024-05-20T18:38:26.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.498+0000 7f9d42ffd640 1 -- 172.21.15.5:0/1509942909 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9d44045070 con 0x7f9d3c0039e0 2024-05-20T18:38:26.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.498+0000 7f9d42ffd640 1 --2- 172.21.15.5:0/1509942909 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9d3406be60 0x7f9d3406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-20T18:38:26.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.498+0000 7f9d42ffd640 1 -- 172.21.15.5:0/1509942909 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f9d440aa730 con 0x7f9d3c0039e0 2024-05-20T18:38:26.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.498+0000 7f9d51d9b640 1 --2- 172.21.15.5:0/1509942909 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9d3406be60 0x7f9d3406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-20T18:38:26.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.499+0000 7f9d51d9b640 1 --2- 172.21.15.5:0/1509942909 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9d3406be60 0x7f9d3406e370 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9d540d17d0 tx=0x7f9d48009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-20T18:38:26.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.503+0000 7f9d42ffd640 1 -- 172.21.15.5:0/1509942909 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9d540b5c70 con 0x7f9d3c0039e0 2024-05-20T18:38:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.723+0000 7f9d58f36640 1 -- 172.21.15.5:0/1509942909 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d540d3050 con 0x7f9d3c0039e0 2024-05-20T18:38:26.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.724+0000 7f9d42ffd640 1 -- 172.21.15.5:0/1509942909 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+3863 (secure 0 0 0) 0x7f9d44079070 con 0x7f9d3c0039e0 2024-05-20T18:38:26.725 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-20T18:38:26.725 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":113,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7178,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6834/266630184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":266630184},{"type":"v1","addr":"172.21.15.63:6834","nonce":266630184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7361,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6835/1494022697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6833","nonce":1494022697},{"type":"v1","addr":"172.21.15.63:6835","nonce":1494022697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":7631,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2892870664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":2892870664},{"type":"v1","addr":"172.21.15.5:6837","nonce":2892870664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":9009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/1739221946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1739221946},{"type":"v1","addr":"172.21.15.5:6836","nonce":1739221946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[]} 2024-05-20T18:38:26.726 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 113 2024-05-20T18:38:26.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.726+0000 7f9d58f36640 1 -- 172.21.15.5:0/1509942909 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9d3406be60 msgr2=0x7f9d3406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:26.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.726+0000 7f9d58f36640 1 --2- 172.21.15.5:0/1509942909 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9d3406be60 0x7f9d3406e370 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9d540d17d0 tx=0x7f9d48009040 comp rx=0 tx=0).stop 2024-05-20T18:38:26.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.726+0000 7f9d58f36640 1 -- 172.21.15.5:0/1509942909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d3c0039e0 msgr2=0x7f9d540cdfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-20T18:38:26.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.726+0000 7f9d58f36640 1 --2- 172.21.15.5:0/1509942909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d3c0039e0 0x7f9d540cdfc0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f9d44008fd0 tx=0x7f9d44001120 comp rx=0 tx=0).stop 2024-05-20T18:38:26.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.726+0000 7f9d58f36640 1 -- 172.21.15.5:0/1509942909 shutdown_connections 2024-05-20T18:38:26.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.726+0000 7f9d58f36640 1 --2- 172.21.15.5:0/1509942909 >> [v2:172.21.15.5:6832/3330161497,v1:172.21.15.5:6833/3330161497] conn(0x7f9d3406be60 0x7f9d3406e370 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:26.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.726+0000 7f9d58f36640 1 --2- 172.21.15.5:0/1509942909 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d541e9860 0x7f9d541e9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:26.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.726+0000 7f9d58f36640 1 --2- 172.21.15.5:0/1509942909 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d540b5320 0x7f9d540b5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:26.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.726+0000 7f9d58f36640 1 --2- 172.21.15.5:0/1509942909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d3c0039e0 0x7f9d540cdfc0 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-20T18:38:26.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.726+0000 7f9d58f36640 1 -- 172.21.15.5:0/1509942909 >> 172.21.15.5:0/1509942909 conn(0x7f9d5406d240 msgr2=0x7f9d54160670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-20T18:38:26.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.726+0000 7f9d58f36640 1 -- 172.21.15.5:0/1509942909 shutdown_connections 2024-05-20T18:38:26.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-20T18:38:26.726+0000 7f9d58f36640 1 -- 172.21.15.5:0/1509942909 wait complete. 2024-05-20T18:38:26.739 DEBUG:tasks.cephfs.filesystem:already dead... 2024-05-20T18:38:26.739 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-20T18:38:26.739 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-20T18:38:26.739 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:38:26.749 INFO:tasks.ceph.mds.a:Stopped 2024-05-20T18:38:26.749 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-20T18:38:26.749 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:38:26.802 INFO:tasks.ceph.mds.c:Stopped 2024-05-20T18:38:26.803 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-20T18:38:26.803 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:38:26.816 INFO:tasks.ceph.mds.b:Stopped 2024-05-20T18:38:26.817 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-20T18:38:26.817 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:38:26.871 INFO:tasks.ceph.mds.d:Stopped 2024-05-20T18:38:26.872 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-20T18:38:26.872 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-20T18:38:26.872 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:38:26.904 INFO:tasks.ceph.osd.0:Stopped 2024-05-20T18:38:26.904 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-20T18:38:26.904 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:38:26.983 INFO:tasks.ceph.osd.1:Stopped 2024-05-20T18:38:26.983 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-20T18:38:26.983 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:38:27.053 INFO:tasks.ceph.osd.2:Stopped 2024-05-20T18:38:27.053 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-20T18:38:27.053 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:38:27.129 INFO:tasks.ceph.osd.3:Stopped 2024-05-20T18:38:27.129 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-20T18:38:27.129 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:38:27.165 INFO:tasks.ceph.osd.4:Stopped 2024-05-20T18:38:27.165 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-20T18:38:27.165 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:38:27.239 INFO:tasks.ceph.osd.5:Stopped 2024-05-20T18:38:27.239 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-20T18:38:27.239 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:38:27.307 INFO:tasks.ceph.osd.6:Stopped 2024-05-20T18:38:27.308 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-20T18:38:27.308 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:38:27.378 INFO:tasks.ceph.osd.7:Stopped 2024-05-20T18:38:27.378 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-20T18:38:27.378 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-20T18:38:27.379 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:38:27.406 INFO:tasks.ceph.mgr.y:Stopped 2024-05-20T18:38:27.406 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-20T18:38:27.406 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:38:27.446 INFO:tasks.ceph.mgr.x:Stopped 2024-05-20T18:38:27.446 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-20T18:38:27.447 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-20T18:38:27.447 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:38:27.458 INFO:tasks.ceph.mon.a:Stopped 2024-05-20T18:38:27.459 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-20T18:38:27.459 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:38:27.499 INFO:tasks.ceph.mon.b:Stopped 2024-05-20T18:38:27.499 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-20T18:38:27.499 INFO:teuthology.orchestra.run:waiting for 300 2024-05-20T18:38:27.552 INFO:tasks.ceph.mon.c:Stopped 2024-05-20T18:38:27.552 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-20T18:38:27.553 DEBUG:teuthology.orchestra.run.smithi005:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-05-20T18:38:27.597 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:38:27.598 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-20T18:38:27.790 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:38:27.790 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-20T18:38:27.888 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:38:27.888 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-20T18:38:27.930 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:38:27.930 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-20T18:38:28.013 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi063.front.sepia.ceph.com 2024-05-20T18:38:28.013 DEBUG:teuthology.orchestra.run.smithi063:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-20T18:38:28.130 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi063.front.sepia.ceph.com 2024-05-20T18:38:28.130 DEBUG:teuthology.orchestra.run.smithi063:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-20T18:38:28.329 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi063.front.sepia.ceph.com 2024-05-20T18:38:28.329 DEBUG:teuthology.orchestra.run.smithi063:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-20T18:38:28.393 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi063.front.sepia.ceph.com 2024-05-20T18:38:28.393 DEBUG:teuthology.orchestra.run.smithi063:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-20T18:38:28.468 INFO:tasks.ceph:Archiving mon data... 2024-05-20T18:38:28.470 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715720/data/mon.a.tgz 2024-05-20T18:38:28.470 DEBUG:teuthology.orchestra.run.smithi005:> mktemp 2024-05-20T18:38:28.487 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/tmp.YsDJE2bfvi 2024-05-20T18:38:28.488 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.YsDJE2bfvi 2024-05-20T18:38:28.648 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0666 /tmp/tmp.YsDJE2bfvi 2024-05-20T18:38:28.734 DEBUG:teuthology.orchestra.remote:smithi005:/tmp/tmp.YsDJE2bfvi is 416KB 2024-05-20T18:38:28.832 DEBUG:teuthology.orchestra.run.smithi005:> rm -fr /tmp/tmp.YsDJE2bfvi 2024-05-20T18:38:28.848 DEBUG:teuthology.misc:Transferring archived files from smithi063:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715720/data/mon.b.tgz 2024-05-20T18:38:28.849 DEBUG:teuthology.orchestra.run.smithi063:> mktemp 2024-05-20T18:38:28.869 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/tmp.uAc4KwaDZN 2024-05-20T18:38:28.870 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.uAc4KwaDZN 2024-05-20T18:38:29.031 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 0666 /tmp/tmp.uAc4KwaDZN 2024-05-20T18:38:29.118 DEBUG:teuthology.orchestra.remote:smithi063:/tmp/tmp.uAc4KwaDZN is 425KB 2024-05-20T18:38:29.215 DEBUG:teuthology.orchestra.run.smithi063:> rm -fr /tmp/tmp.uAc4KwaDZN 2024-05-20T18:38:29.231 DEBUG:teuthology.misc:Transferring archived files from smithi063:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715720/data/mon.c.tgz 2024-05-20T18:38:29.231 DEBUG:teuthology.orchestra.run.smithi063:> mktemp 2024-05-20T18:38:29.287 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/tmp.WdDA4jUGrO 2024-05-20T18:38:29.288 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.WdDA4jUGrO 2024-05-20T18:38:29.439 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 0666 /tmp/tmp.WdDA4jUGrO 2024-05-20T18:38:29.526 DEBUG:teuthology.orchestra.remote:smithi063:/tmp/tmp.WdDA4jUGrO is 425KB 2024-05-20T18:38:29.624 DEBUG:teuthology.orchestra.run.smithi063:> rm -fr /tmp/tmp.WdDA4jUGrO 2024-05-20T18:38:29.640 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-20T18:38:29.640 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-20T18:38:29.643 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-20T18:38:29.732 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-20T18:38:29.761 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-20T18:38:29.761 INFO:tasks.ceph:Archiving crash dumps... 2024-05-20T18:38:29.763 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715720/remote/smithi005/crash 2024-05-20T18:38:29.764 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-20T18:38:29.803 DEBUG:teuthology.misc:Transferring archived files from smithi063:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715720/remote/smithi063/crash 2024-05-20T18:38:29.804 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-20T18:38:29.829 INFO:tasks.ceph:Shutting down logrotate 2024-05-20T18:38:29.829 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:38:29.850 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-20T18:38:29.932 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-20T18:38:29.961 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-20T18:38:29.987 INFO:tasks.ceph:Compressing logs... 2024-05-20T18:38:29.987 DEBUG:teuthology.orchestra.run.smithi005:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-20T18:38:30.004 DEBUG:teuthology.orchestra.run.smithi063:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-20T18:38:30.029 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70351.log 2024-05-20T18:38:30.053 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59059.log 2024-05-20T18:38:30.053 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55072.log 2024-05-20T18:38:30.053 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75716.log 2024-05-20T18:38:30.053 INFO:teuthology.orchestra.run.smithi005.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.70351.log.gz 2024-05-20T18:38:30.053 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80580.log 2024-05-20T18:38:30.053 INFO:teuthology.orchestra.run.smithi005.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.59059.log.gz 2024-05-20T18:38:30.054 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.75716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62981.log 2024-05-20T18:38:30.054 INFO:teuthology.orchestra.run.smithi005.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.55072.log.gz 2024-05-20T18:38:30.054 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.75716.log.gz 2024-05-20T18:38:30.054 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.80580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71137.log 2024-05-20T18:38:30.054 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.80580.log.gz 2024-05-20T18:38:30.054 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60196.log 2024-05-20T18:38:30.054 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71137.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62981.log.gz 2024-05-20T18:38:30.054 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51254.log 2024-05-20T18:38:30.054 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71137.log.gz 2024-05-20T18:38:30.054 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53874.log 2024-05-20T18:38:30.054 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60196.log.gz 2024-05-20T18:38:30.054 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78983.log 2024-05-20T18:38:30.054 INFO:teuthology.orchestra.run.smithi005.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51254.log.gz 2024-05-20T18:38:30.054 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.53874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73933.log 2024-05-20T18:38:30.055 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53874.log.gz 2024-05-20T18:38:30.055 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.78983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66384.log 2024-05-20T18:38:30.055 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.78983.log.gz 2024-05-20T18:38:30.055 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.73933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60112.log 2024-05-20T18:38:30.055 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52809.log 2024-05-20T18:38:30.055 INFO:teuthology.orchestra.run.smithi005.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.73933.log.gz 2024-05-20T18:38:30.055 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68902.log 2024-05-20T18:38:30.055 INFO:teuthology.orchestra.run.smithi005.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.66384.log.gz 2024-05-20T18:38:30.057 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60112.log.gz 2024-05-20T18:38:30.057 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65634.log 2024-05-20T18:38:30.057 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52809.log.gz 2024-05-20T18:38:30.057 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.68902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47987.log 2024-05-20T18:38:30.057 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54862.log 2024-05-20T18:38:30.057 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.47987.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.65634.log.gz 2024-05-20T18:38:30.058 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48743.log 2024-05-20T18:38:30.058 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47987.log.gz 2024-05-20T18:38:30.058 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74949.log 2024-05-20T18:38:30.058 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48743.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54862.log.gz 2024-05-20T18:38:30.058 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42603.log 2024-05-20T18:38:30.058 INFO:teuthology.orchestra.run.smithi005.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.68902.log.gz 2024-05-20T18:38:30.058 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48743.log.gz 2024-05-20T18:38:30.058 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64381.log 2024-05-20T18:38:30.058 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74949.log.gz 2024-05-20T18:38:30.059 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47419.log 2024-05-20T18:38:30.059 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.64381.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42603.log.gz 2024-05-20T18:38:30.059 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67502.log 2024-05-20T18:38:30.059 INFO:teuthology.orchestra.run.smithi005.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.64381.log.gz 2024-05-20T18:38:30.059 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.47419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68413.log 2024-05-20T18:38:30.059 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47419.log.gz 2024-05-20T18:38:30.059 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.67502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48029.log 2024-05-20T18:38:30.059 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67502.log.gz 2024-05-20T18:38:30.059 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.68413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44753.log 2024-05-20T18:38:30.059 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46475.log 2024-05-20T18:38:30.059 INFO:teuthology.orchestra.run.smithi005.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.68413.log.gz 2024-05-20T18:38:30.059 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48029.log.gz 2024-05-20T18:38:30.060 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59430.log 2024-05-20T18:38:30.060 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44753.log.gz 2024-05-20T18:38:30.060 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77924.log 2024-05-20T18:38:30.060 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59430.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46475.log.gz 2024-05-20T18:38:30.060 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66450.log 2024-05-20T18:38:30.060 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.77924.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.59430.log.gz 2024-05-20T18:38:30.060 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57772.log 2024-05-20T18:38:30.060 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.77924.log.gz 2024-05-20T18:38:30.060 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75537.log 2024-05-20T18:38:30.060 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57772.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66450.log.gz 2024-05-20T18:38:30.060 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68155.log 2024-05-20T18:38:30.060 INFO:teuthology.orchestra.run.smithi005.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57772.log.gz 2024-05-20T18:38:30.060 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.75537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72605.log 2024-05-20T18:38:30.061 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75537.log.gz 2024-05-20T18:38:30.061 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.68155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71597.log 2024-05-20T18:38:30.061 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.72605.log: -- /var/log/ceph/ceph-client.admin.44155.log 2024-05-20T18:38:30.061 INFO:teuthology.orchestra.run.smithi005.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.68155.log.gz 2024-05-20T18:38:30.061 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71597.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72605.log.gz 2024-05-20T18:38:30.061 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46391.log 2024-05-20T18:38:30.061 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.71597.log.gz 2024-05-20T18:38:30.061 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79443.log 2024-05-20T18:38:30.061 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44155.log: /var/log/ceph/ceph-client.admin.46391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54690.log 2024-05-20T18:38:30.061 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44155.log.gz 2024-05-20T18:38:30.061 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46391.log.gz 2024-05-20T18:38:30.061 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.79443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49646.log 2024-05-20T18:38:30.061 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70603.log 2024-05-20T18:38:30.062 INFO:teuthology.orchestra.run.smithi005.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.79443.log.gz 2024-05-20T18:38:30.062 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49646.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54690.log.gz 2024-05-20T18:38:30.062 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67436.log 2024-05-20T18:38:30.062 INFO:teuthology.orchestra.run.smithi005.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49646.log.gz 2024-05-20T18:38:30.062 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48575.log 2024-05-20T18:38:30.062 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.67436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52887.log 2024-05-20T18:38:30.062 INFO:teuthology.orchestra.run.smithi005.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.70603.log.gz 2024-05-20T18:38:30.062 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72915.log 2024-05-20T18:38:30.062 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48575.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.67436.log.gz 2024-05-20T18:38:30.062 INFO:teuthology.orchestra.run.smithi005.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48575.log.gz 2024-05-20T18:38:30.062 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76614.log 2024-05-20T18:38:30.062 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52887.log: /var/log/ceph/ceph-client.admin.72915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58627.log 2024-05-20T18:38:30.063 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52887.log.gz 2024-05-20T18:38:30.063 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57515.log 2024-05-20T18:38:30.063 INFO:teuthology.orchestra.run.smithi005.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.72915.log.gz 2024-05-20T18:38:30.063 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82031.log 2024-05-20T18:38:30.063 INFO:teuthology.orchestra.run.smithi005.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.76614.log.gz 2024-05-20T18:38:30.063 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71382.log 2024-05-20T18:38:30.063 INFO:teuthology.orchestra.run.smithi005.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.58627.log.gz 2024-05-20T18:38:30.063 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57515.log.gz 2024-05-20T18:38:30.063 INFO:teuthology.orchestra.run.smithi005.stderr:gzip/var/log/ceph/ceph-client.admin.82031.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.81178.log 2024-05-20T18:38:30.063 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55138.log 2024-05-20T18:38:30.063 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.82031.log.gz 2024-05-20T18:38:30.063 INFO:teuthology.orchestra.run.smithi005.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71382.log.gz 2024-05-20T18:38:30.064 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.81178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74991.log 2024-05-20T18:38:30.064 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.81178.log.gz 2024-05-20T18:38:30.064 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61394.log 2024-05-20T18:38:30.064 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74991.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55138.log.gz 2024-05-20T18:38:30.064 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62390.log 2024-05-20T18:38:30.064 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74991.log.gz 2024-05-20T18:38:30.064 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60831.log 2024-05-20T18:38:30.064 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61394.log.gz 2024-05-20T18:38:30.064 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58518.log 2024-05-20T18:38:30.064 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62390.log: /var/log/ceph/ceph-client.admin.60831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63703.log 2024-05-20T18:38:30.064 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62390.log.gz 2024-05-20T18:38:30.064 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58518.log: gzip -5 --verbose 88.0% -- -- replaced with /var/log/ceph/ceph-client.admin.60831.log.gz /var/log/ceph/ceph-client.admin.80968.log 2024-05-20T18:38:30.065 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-05-20T18:38:30.065 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63703.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58518.log.gz 2024-05-20T18:38:30.065 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66046.log 2024-05-20T18:38:30.065 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.80968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79067.log 2024-05-20T18:38:30.065 INFO:teuthology.orchestra.run.smithi005.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.63703.log.gz 2024-05-20T18:38:30.065 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.80968.log.gz 2024-05-20T18:38:30.065 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42994.log 2024-05-20T18:38:30.065 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66046.log.gz 2024-05-20T18:38:30.065 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77286.log 2024-05-20T18:38:30.065 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.79067.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.42994.log: /var/log/ceph/ceph-client.admin.59598.log 2024-05-20T18:38:30.065 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79067.log.gz 2024-05-20T18:38:30.065 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.77286.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42994.log.gz 2024-05-20T18:38:30.066 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50630.log 2024-05-20T18:38:30.066 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77286.log.gz 2024-05-20T18:38:30.066 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50714.log 2024-05-20T18:38:30.066 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50630.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59598.log.gz 2024-05-20T18:38:30.066 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53645.log 2024-05-20T18:38:30.066 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50630.log.gz 2024-05-20T18:38:30.066 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61732.log 2024-05-20T18:38:30.066 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50714.log.gz 2024-05-20T18:38:30.066 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70225.log/var/log/ceph/ceph-client.admin.53645.log: 2024-05-20T18:38:30.066 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.61732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54988.log 2024-05-20T18:38:30.066 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53645.log.gz 2024-05-20T18:38:30.066 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.61732.log.gz 2024-05-20T18:38:30.066 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44711.log 2024-05-20T18:38:30.067 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.70225.log.gz 2024-05-20T18:38:30.067 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63184.log 2024-05-20T18:38:30.067 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44711.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54988.log.gz 2024-05-20T18:38:30.067 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69462.log 2024-05-20T18:38:30.067 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44711.log.gz 2024-05-20T18:38:30.067 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52949.log 2024-05-20T18:38:30.067 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.69462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80884.log 2024-05-20T18:38:30.067 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.69462.log.gz 2024-05-20T18:38:30.067 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79710.log 2024-05-20T18:38:30.067 INFO:teuthology.orchestra.run.smithi005.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.63184.log.gz 2024-05-20T18:38:30.092 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37716.log 2024-05-20T18:38:30.092 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44028.log 2024-05-20T18:38:30.092 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37465.log 2024-05-20T18:38:30.092 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.37716.log: /var/log/ceph/ceph-client.1.44028.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-20T18:38:30.092 INFO:teuthology.orchestra.run.smithi063.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37716.log.gz 2024-05-20T18:38:30.092 INFO:teuthology.orchestra.run.smithi063.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44028.log.gz 2024-05-20T18:38:30.092 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.37465.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-20T18:38:30.092 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37465.log.gz 2024-05-20T18:38:30.092 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-05-20T18:38:30.093 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-05-20T18:38:30.093 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.44428.log 2024-05-20T18:38:30.093 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-20T18:38:30.093 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.3.44428.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-20T18:38:30.093 INFO:teuthology.orchestra.run.smithi063.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.44428.log.gz 2024-05-20T18:38:30.093 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37585.log 2024-05-20T18:38:30.093 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-20T18:38:30.093 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.37585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37505.log 2024-05-20T18:38:30.093 INFO:teuthology.orchestra.run.smithi063.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37585.log.gz 2024-05-20T18:38:30.093 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-20T18:38:30.093 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.37505.log: 92.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-20T18:38:30.093 INFO:teuthology.orchestra.run.smithi063.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37505.log.gz 2024-05-20T18:38:30.093 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-05-20T18:38:30.094 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37545.log 2024-05-20T18:38:30.094 INFO:teuthology.orchestra.run.smithi063.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-20T18:38:30.094 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37409.log 2024-05-20T18:38:30.094 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.37545.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37545.log.gz 2024-05-20T18:38:30.094 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37625.log 2024-05-20T18:38:30.094 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.37409.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37409.log.gz 2024-05-20T18:38:30.094 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44033.log 2024-05-20T18:38:30.094 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.37625.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37625.log.gz 2024-05-20T18:38:30.094 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-05-20T18:38:30.094 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.1.44033.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44033.log.gz 2024-05-20T18:38:30.094 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-20T18:38:30.094 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-mds.d.log: 94.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-20T18:38:30.094 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.44432.log 2024-05-20T18:38:30.094 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37667.log 2024-05-20T18:38:30.095 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.3.44432.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.44432.log.gz 2024-05-20T18:38:30.115 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.37667.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37667.log.gz 2024-05-20T18:38:30.187 INFO:teuthology.orchestra.run.smithi063.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-05-20T18:38:30.198 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.80884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53603.log 2024-05-20T18:38:30.198 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52949.log.gz 2024-05-20T18:38:30.199 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.79710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50234.log 2024-05-20T18:38:30.199 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80884.log.gz 2024-05-20T18:38:30.199 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.79710.log.gz 2024-05-20T18:38:30.199 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.53603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80758.log 2024-05-20T18:38:30.199 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53603.log.gz 2024-05-20T18:38:30.199 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75243.log 2024-05-20T18:38:30.199 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50234.log: /var/log/ceph/ceph-client.admin.80758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82551.log 2024-05-20T18:38:30.199 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50234.log.gz 2024-05-20T18:38:30.199 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.75243.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80758.log.gz 2024-05-20T18:38:30.199 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57132.log 2024-05-20T18:38:30.199 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75243.log.gz 2024-05-20T18:38:30.199 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.82551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58108.log 2024-05-20T18:38:30.199 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57132.log: gzip -5 88.9% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.82551.log.gz /var/log/ceph/ceph-client.admin.48617.log 2024-05-20T18:38:30.200 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-05-20T18:38:30.200 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57132.log.gz 2024-05-20T18:38:30.200 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49604.log 2024-05-20T18:38:30.200 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47861.log 2024-05-20T18:38:30.200 INFO:teuthology.orchestra.run.smithi005.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.58108.log.gz 2024-05-20T18:38:30.200 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61138.log 2024-05-20T18:38:30.200 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48617.log.gz 2024-05-20T18:38:30.200 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49604.log.gz 2024-05-20T18:38:30.200 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.47861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48827.log 2024-05-20T18:38:30.200 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61138.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47861.log.gz 2024-05-20T18:38:30.200 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69059.log 2024-05-20T18:38:30.200 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61138.log.gz 2024-05-20T18:38:30.200 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70131.log 2024-05-20T18:38:30.201 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48827.log.gz 2024-05-20T18:38:30.201 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53084.log 2024-05-20T18:38:30.201 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.69059.log: /var/log/ceph/ceph-client.admin.70131.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-20T18:38:30.201 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69059.log.gz 2024-05-20T18:38:30.201 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70131.log.gz 2024-05-20T18:38:30.201 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.53084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57473.log 2024-05-20T18:38:30.201 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.53084.log.gz 2024-05-20T18:38:30.201 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54553.log 2024-05-20T18:38:30.201 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60747.log 2024-05-20T18:38:30.201 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54553.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57473.log.gz 2024-05-20T18:38:30.201 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79298.log 2024-05-20T18:38:30.201 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54553.log.gz 2024-05-20T18:38:30.201 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60490.log 2024-05-20T18:38:30.201 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.79298.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60747.log.gz 2024-05-20T18:38:30.202 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79509.log 2024-05-20T18:38:30.202 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60490.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.79298.log.gz 2024-05-20T18:38:30.202 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78156.log 2024-05-20T18:38:30.202 INFO:teuthology.orchestra.run.smithi005.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60490.log.gz 2024-05-20T18:38:30.202 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.79509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54179.log 2024-05-20T18:38:30.202 INFO:teuthology.orchestra.run.smithi005.stderr: 87.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-20T18:38:30.202 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.78156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75369.log 2024-05-20T18:38:30.202 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54179.log: 89.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.78156.log.gz 2024-05-20T18:38:30.202 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.76868.log 2024-05-20T18:38:30.202 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54179.log.gz 2024-05-20T18:38:30.202 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.75369.log: gzip -5 --verbose -- 92.0% /var/log/ceph/ceph-client.admin.42558.log -- replaced with /var/log/ceph/ceph-client.admin.79509.log.gz 2024-05-20T18:38:30.202 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-05-20T18:38:30.202 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.76868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45859.log 2024-05-20T18:38:30.203 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.76868.log.gz 2024-05-20T18:38:30.203 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75369.log.gz 2024-05-20T18:38:30.203 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51817.log 2024-05-20T18:38:30.203 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42558.log.gz 2024-05-20T18:38:30.203 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60322.log 2024-05-20T18:38:30.203 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45859.log.gz 2024-05-20T18:38:30.203 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65676.log 2024-05-20T18:38:30.203 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51645.log 2024-05-20T18:38:30.203 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51817.log.gz 2024-05-20T18:38:30.203 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60322.log: /var/log/ceph/ceph-client.admin.65676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50150.log 2024-05-20T18:38:30.203 INFO:teuthology.orchestra.run.smithi005.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.60322.log.gz 2024-05-20T18:38:30.203 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65676.log.gz 2024-05-20T18:38:30.203 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61544.log 2024-05-20T18:38:30.204 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50150.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51645.log.gz 2024-05-20T18:38:30.204 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76022.log 2024-05-20T18:38:30.204 INFO:teuthology.orchestra.run.smithi005.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50150.log.gz 2024-05-20T18:38:30.204 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.47728.log 2024-05-20T18:38:30.204 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61544.log.gz 2024-05-20T18:38:30.204 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59472.log 2024-05-20T18:38:30.204 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76022.log: /var/log/ceph/ceph-client.2.47728.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-20T18:38:30.204 INFO:teuthology.orchestra.run.smithi005.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.76022.log.gz 2024-05-20T18:38:30.204 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.47728.log.gz 2024-05-20T18:38:30.204 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70393.log 2024-05-20T18:38:30.204 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59472.log: /var/log/ceph/ceph-osd.2.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59472.log.gz 2024-05-20T18:38:30.204 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67694.log 2024-05-20T18:38:30.204 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81947.log 2024-05-20T18:38:30.204 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70393.log.gz 2024-05-20T18:38:30.205 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.67694.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.0.67694.log.gz 2024-05-20T18:38:30.205 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49148.log 2024-05-20T18:38:30.205 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.81947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46643.log 2024-05-20T18:38:30.205 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49148.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.81947.log.gz 2024-05-20T18:38:30.205 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52683.log 2024-05-20T18:38:30.205 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49148.log.gz 2024-05-20T18:38:30.205 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56441.log 2024-05-20T18:38:30.205 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46643.log.gz 2024-05-20T18:38:30.205 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60154.log 2024-05-20T18:38:30.205 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52683.log.gz 2024-05-20T18:38:30.205 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51004.log 2024-05-20T18:38:30.205 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60154.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56441.log.gz 2024-05-20T18:38:30.205 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75919.log 2024-05-20T18:38:30.206 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60154.log.gz 2024-05-20T18:38:30.206 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48995.log 2024-05-20T18:38:30.206 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.75919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68305.log 2024-05-20T18:38:30.206 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51004.log.gz 2024-05-20T18:38:30.206 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48995.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.75919.log.gz 2024-05-20T18:38:30.206 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43140.log 2024-05-20T18:38:30.206 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48995.log.gz 2024-05-20T18:38:30.206 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.68305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54042.log 2024-05-20T18:38:30.206 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70477.log 2024-05-20T18:38:30.206 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54042.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43140.log.gz 2024-05-20T18:38:30.206 INFO:teuthology.orchestra.run.smithi005.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.68305.log.gz 2024-05-20T18:38:30.206 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62221.log 2024-05-20T18:38:30.207 INFO:teuthology.orchestra.run.smithi005.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54042.log.gz 2024-05-20T18:38:30.207 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50192.log 2024-05-20T18:38:30.207 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70477.log.gz 2024-05-20T18:38:30.207 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54259.log 2024-05-20T18:38:30.207 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50192.log: gzip 87.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.62221.log.gz --verbose 2024-05-20T18:38:30.207 INFO:teuthology.orchestra.run.smithi005.stderr: -- /var/log/ceph/ceph-client.admin.66130.log 2024-05-20T18:38:30.207 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50192.log.gz 2024-05-20T18:38:30.207 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75877.log 2024-05-20T18:38:30.207 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54259.log.gz 2024-05-20T18:38:30.207 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62586.log 2024-05-20T18:38:30.207 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.75877.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66130.log.gz 2024-05-20T18:38:30.207 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79912.log 2024-05-20T18:38:30.207 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75877.log.gz 2024-05-20T18:38:30.208 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60585.log 2024-05-20T18:38:30.208 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62586.log.gz 2024-05-20T18:38:30.208 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.79912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79261.log 2024-05-20T18:38:30.208 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46223.log 2024-05-20T18:38:30.208 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60585.log.gz 2024-05-20T18:38:30.208 INFO:teuthology.orchestra.run.smithi005.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.79912.log.gz 2024-05-20T18:38:30.208 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.79261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80622.log 2024-05-20T18:38:30.208 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49478.log 2024-05-20T18:38:30.208 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46223.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.79261.log.gz 2024-05-20T18:38:30.208 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.80622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63443.log 2024-05-20T18:38:30.208 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46223.log.gz 2024-05-20T18:38:30.208 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80622.log.gz 2024-05-20T18:38:30.208 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63877.log 2024-05-20T18:38:30.209 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49478.log.gz 2024-05-20T18:38:30.209 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65472.log 2024-05-20T18:38:30.209 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70874.log 2024-05-20T18:38:30.209 INFO:teuthology.orchestra.run.smithi005.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.63443.log.gz 2024-05-20T18:38:30.209 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63877.log.gz 2024-05-20T18:38:30.209 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64684.log 2024-05-20T18:38:30.209 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65472.log.gz 2024-05-20T18:38:30.209 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73999.log 2024-05-20T18:38:30.209 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70874.log.gz 2024-05-20T18:38:30.209 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.64684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59986.log 2024-05-20T18:38:30.209 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.64684.log.gz 2024-05-20T18:38:30.209 INFO:teuthology.orchestra.run.smithi005.stderr:gzip/var/log/ceph/ceph-client.admin.73999.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.55819.log 2024-05-20T18:38:30.209 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68047.log 2024-05-20T18:38:30.209 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73999.log.gz 2024-05-20T18:38:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55819.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59986.log.gz 2024-05-20T18:38:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66088.log 2024-05-20T18:38:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.68047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71000.log 2024-05-20T18:38:30.210 INFO:teuthology.orchestra.run.smithi005.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.55819.log.gz 2024-05-20T18:38:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73668.log 2024-05-20T18:38:30.210 INFO:teuthology.orchestra.run.smithi005.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.68047.log.gz 2024-05-20T18:38:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70267.log 2024-05-20T18:38:30.210 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66088.log.gz 2024-05-20T18:38:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71000.log: /var/log/ceph/ceph-client.admin.73668.log: gzip 87.2% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.71000.log.gz 2024-05-20T18:38:30.210 INFO:teuthology.orchestra.run.smithi005.stderr: -- /var/log/ceph/ceph-client.admin.61914.log 2024-05-20T18:38:30.210 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73668.log.gz 2024-05-20T18:38:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46181.log 2024-05-20T18:38:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61914.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.70267.log.gz 2024-05-20T18:38:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56357.log 2024-05-20T18:38:30.211 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61914.log.gz 2024-05-20T18:38:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56399.log 2024-05-20T18:38:30.211 INFO:teuthology.orchestra.run.smithi005.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46181.log.gz 2024-05-20T18:38:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59818.log 2024-05-20T18:38:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51464.log 2024-05-20T18:38:30.211 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56357.log.gz 2024-05-20T18:38:30.211 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56399.log.gz 2024-05-20T18:38:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51127.log 2024-05-20T18:38:30.211 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59818.log.gz 2024-05-20T18:38:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67253.log 2024-05-20T18:38:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51464.log: /var/log/ceph/ceph-client.admin.51127.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-20T18:38:30.211 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51464.log.gz 2024-05-20T18:38:30.211 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51127.log.gz 2024-05-20T18:38:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.67253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61478.log 2024-05-20T18:38:30.212 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67253.log.gz 2024-05-20T18:38:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48071.log 2024-05-20T18:38:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76108.log 2024-05-20T18:38:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62159.log 2024-05-20T18:38:30.212 INFO:teuthology.orchestra.run.smithi005.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.61478.log.gz 2024-05-20T18:38:30.212 INFO:teuthology.orchestra.run.smithi005.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48071.log.gz 2024-05-20T18:38:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65020.log 2024-05-20T18:38:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76108.log: /var/log/ceph/ceph-client.admin.62159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80496.log 2024-05-20T18:38:30.212 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76108.log.gz 2024-05-20T18:38:30.212 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3%/var/log/ceph/ceph-client.admin.65020.log: -- replaced with /var/log/ceph/ceph-client.admin.62159.log.gz 2024-05-20T18:38:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64852.log 2024-05-20T18:38:30.212 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65020.log.gz 2024-05-20T18:38:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.80496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50318.log 2024-05-20T18:38:30.213 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.80496.log.gz 2024-05-20T18:38:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82241.log 2024-05-20T18:38:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.64852.log: /var/log/ceph/ceph-client.admin.50318.log: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-20T18:38:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67126.log 2024-05-20T18:38:30.213 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64852.log.gz 2024-05-20T18:38:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.82241.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50318.log.gz 2024-05-20T18:38:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45182.log 2024-05-20T18:38:30.213 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.82241.log.gz 2024-05-20T18:38:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.67126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58843.log 2024-05-20T18:38:30.213 INFO:teuthology.orchestra.run.smithi005.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67126.log.gz 2024-05-20T18:38:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46559.log 2024-05-20T18:38:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58843.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45182.log.gz 2024-05-20T18:38:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74690.log 2024-05-20T18:38:30.213 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58843.log.gz 2024-05-20T18:38:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48701.log 2024-05-20T18:38:30.214 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46559.log.gz 2024-05-20T18:38:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63107.log 2024-05-20T18:38:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57645.log 2024-05-20T18:38:30.214 INFO:teuthology.orchestra.run.smithi005.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.74690.log.gz 2024-05-20T18:38:30.214 INFO:teuthology.orchestra.run.smithi005.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48701.log.gz 2024-05-20T18:38:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54467.log 2024-05-20T18:38:30.214 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63107.log.gz 2024-05-20T18:38:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75621.log 2024-05-20T18:38:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54467.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57645.log.gz 2024-05-20T18:38:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82073.log 2024-05-20T18:38:30.214 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54467.log.gz 2024-05-20T18:38:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.75621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76236.log 2024-05-20T18:38:30.215 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75621.log.gz 2024-05-20T18:38:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.82073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.80166.log 2024-05-20T18:38:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72101.log 2024-05-20T18:38:30.215 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.82073.log.gz 2024-05-20T18:38:30.215 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76236.log.gz 2024-05-20T18:38:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.80166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48323.log 0.0% 2024-05-20T18:38:30.215 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.0.80166.log.gz 2024-05-20T18:38:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.72101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59776.log 2024-05-20T18:38:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56735.log 2024-05-20T18:38:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48323.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72101.log.gz 2024-05-20T18:38:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59776.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48323.log.gz 2024-05-20T18:38:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48239.log 2024-05-20T18:38:30.215 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59776.log.gz 2024-05-20T18:38:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43224.log 2024-05-20T18:38:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48239.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56735.log.gz 2024-05-20T18:38:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59860.log 2024-05-20T18:38:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43224.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48239.log.gz 2024-05-20T18:38:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51985.log 2024-05-20T18:38:30.216 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43224.log.gz 2024-05-20T18:38:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73159.log 2024-05-20T18:38:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51985.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59860.log.gz 2024-05-20T18:38:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50066.log 2024-05-20T18:38:30.216 INFO:teuthology.orchestra.run.smithi005.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51985.log.gz 2024-05-20T18:38:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.73159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67939.log 2024-05-20T18:38:30.216 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.73159.log.gz 2024-05-20T18:38:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69101.log 2024-05-20T18:38:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.67939.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50066.log.gz 2024-05-20T18:38:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50444.log 2024-05-20T18:38:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.69101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66921.log 2024-05-20T18:38:30.217 INFO:teuthology.orchestra.run.smithi005.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.67939.log.gz 2024-05-20T18:38:30.217 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.69101.log.gz 2024-05-20T18:38:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45181.log 2024-05-20T18:38:30.217 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50444.log.gz 2024-05-20T18:38:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64894.log 2024-05-20T18:38:30.217 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66921.log.gz 2024-05-20T18:38:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77429.log 2024-05-20T18:38:30.217 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45181.log.gz 2024-05-20T18:38:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.64894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54424.log 2024-05-20T18:38:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.77429.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64894.log.gzgzip 2024-05-20T18:38:30.217 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.81599.log 2024-05-20T18:38:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54424.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.77429.log.gz 2024-05-20T18:38:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50276.log 2024-05-20T18:38:30.218 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54424.log.gz 2024-05-20T18:38:30.218 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.81599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49862.log 2024-05-20T18:38:30.218 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50276.log: gzip -5 --verbose -- 88.4% /var/log/ceph/ceph-client.admin.81010.log -- replaced with /var/log/ceph/ceph-client.admin.81599.log.gz 2024-05-20T18:38:30.218 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-05-20T18:38:30.218 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50276.log.gz 2024-05-20T18:38:30.218 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54732.log 2024-05-20T18:38:30.218 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49862.log.gz 2024-05-20T18:38:30.218 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.81010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67855.log 2024-05-20T18:38:30.218 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.81010.log.gz 2024-05-20T18:38:30.218 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50882.log 2024-05-20T18:38:30.218 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54732.log.gz 2024-05-20T18:38:30.218 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.67855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55555.log 2024-05-20T18:38:30.218 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50882.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-20T18:38:30.219 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50882.log.gz 2024-05-20T18:38:30.219 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55555.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.67855.log.gz 2024-05-20T18:38:30.219 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45518.log 2024-05-20T18:38:30.219 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66879.log 2024-05-20T18:38:30.219 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-osd.3.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.55555.log.gz 2024-05-20T18:38:30.219 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81905.log 2024-05-20T18:38:30.219 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45518.log.gz 2024-05-20T18:38:30.219 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62318.log 2024-05-20T18:38:30.219 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82283.log 2024-05-20T18:38:30.219 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.81905.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66879.log.gz 2024-05-20T18:38:30.219 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62318.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.81905.log.gz 2024-05-20T18:38:30.219 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81557.log 2024-05-20T18:38:30.219 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62318.log.gz 2024-05-20T18:38:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.82283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69256.log 2024-05-20T18:38:30.220 INFO:teuthology.orchestra.run.smithi005.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.82283.log.gz 2024-05-20T18:38:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.81557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81863.log 2024-05-20T18:38:30.220 INFO:teuthology.orchestra.run.smithi005.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.81557.log.gz 2024-05-20T18:38:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.69256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76194.log 2024-05-20T18:38:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.81863.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69256.log.gz 2024-05-20T18:38:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49190.log 2024-05-20T18:38:30.220 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.81863.log.gz 2024-05-20T18:38:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81281.log 2024-05-20T18:38:30.220 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.76194.log.gz 2024-05-20T18:38:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66172.log 2024-05-20T18:38:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69610.log 2024-05-20T18:38:30.220 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49190.log.gz 2024-05-20T18:38:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.81281.log: /var/log/ceph/ceph-client.admin.66172.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.81281.log.gzgzip 2024-05-20T18:38:30.221 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49520.log 2024-05-20T18:38:30.221 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66172.log.gz 2024-05-20T18:38:30.221 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69868.log 2024-05-20T18:38:30.221 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.69610.log: /var/log/ceph/ceph-client.admin.49520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53351.log 2024-05-20T18:38:30.221 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.69868.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49520.log.gz 2024-05-20T18:38:30.221 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56777.log 2024-05-20T18:38:30.221 INFO:teuthology.orchestra.run.smithi005.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.69610.log.gz 2024-05-20T18:38:30.221 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69868.log.gz 2024-05-20T18:38:30.221 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.53351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57006.log 2024-05-20T18:38:30.221 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53351.log.gz 2024-05-20T18:38:30.221 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66638.log 2024-05-20T18:38:30.221 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56777.log: /var/log/ceph/ceph-client.admin.57006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49316.log 2024-05-20T18:38:30.221 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56777.log.gz 2024-05-20T18:38:30.221 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57006.log.gz 2024-05-20T18:38:30.222 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.66638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48785.log 2024-05-20T18:38:30.222 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.66638.log.gz 2024-05-20T18:38:30.222 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45858.log 2024-05-20T18:38:30.222 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49316.log: /var/log/ceph/ceph-client.admin.48785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52027.log 2024-05-20T18:38:30.222 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49316.log.gz 2024-05-20T18:38:30.222 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48785.log.gz 2024-05-20T18:38:30.222 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63401.log 2024-05-20T18:38:30.222 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.80170.log 2024-05-20T18:38:30.222 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45858.log.gz 2024-05-20T18:38:30.222 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52027.log: /var/log/ceph/ceph-client.admin.63401.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52027.log.gz 2024-05-20T18:38:30.222 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54904.log 2024-05-20T18:38:30.222 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63401.log.gz 2024-05-20T18:38:30.222 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73761.log 2024-05-20T18:38:30.223 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.80170.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.80170.log.gz 2024-05-20T18:38:30.223 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44585.log 2024-05-20T18:38:30.223 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.73761.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54904.log.gz 2024-05-20T18:38:30.223 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69963.log 2024-05-20T18:38:30.223 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50840.log 2024-05-20T18:38:30.223 INFO:teuthology.orchestra.run.smithi005.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.73761.log.gz 2024-05-20T18:38:30.223 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44585.log.gz 2024-05-20T18:38:30.223 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78461.log/var/log/ceph/ceph-client.admin.69963.log: 2024-05-20T18:38:30.223 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.69963.log.gz 2024-05-20T18:38:30.223 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64465.log 2024-05-20T18:38:30.223 INFO:teuthology.orchestra.run.smithi005.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50840.log.gz 2024-05-20T18:38:30.223 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75327.log 2024-05-20T18:38:30.223 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.78461.log: /var/log/ceph/ceph-client.admin.64465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81779.log 2024-05-20T18:38:30.223 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78461.log.gz 2024-05-20T18:38:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.75327.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.64465.log.gz 2024-05-20T18:38:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76680.log 2024-05-20T18:38:30.224 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75327.log.gz 2024-05-20T18:38:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.81779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81052.log 2024-05-20T18:38:30.224 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.81779.log.gz 2024-05-20T18:38:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60448.log 2024-05-20T18:38:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76680.log: /var/log/ceph/ceph-client.admin.81052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58951.log 2024-05-20T18:38:30.224 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76680.log.gz 2024-05-20T18:38:30.224 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.81052.log.gz 2024-05-20T18:38:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61998.log 2024-05-20T18:38:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67149.log 2024-05-20T18:38:30.224 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60448.log.gz 2024-05-20T18:38:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58951.log: /var/log/ceph/ceph-client.admin.61998.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58951.log.gz 2024-05-20T18:38:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55630.log 2024-05-20T18:38:30.224 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61998.log.gz 2024-05-20T18:38:30.225 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63617.log 2024-05-20T18:38:30.225 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.67149.log: /var/log/ceph/ceph-client.admin.55630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81094.log 2024-05-20T18:38:30.225 INFO:teuthology.orchestra.run.smithi005.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.67149.log.gz 2024-05-20T18:38:30.225 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63617.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.55630.log.gz 2024-05-20T18:38:30.225 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45510.log 2024-05-20T18:38:30.225 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.81094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73591.log 2024-05-20T18:38:30.225 INFO:teuthology.orchestra.run.smithi005.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.63617.log.gz 2024-05-20T18:38:30.225 INFO:teuthology.orchestra.run.smithi005.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.81094.log.gz 2024-05-20T18:38:30.225 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65377.log 2024-05-20T18:38:30.225 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45510.log.gz 2024-05-20T18:38:30.225 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77882.log 2024-05-20T18:38:30.225 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.73591.log: /var/log/ceph/ceph-client.admin.65377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46745.log 2024-05-20T18:38:30.225 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.73591.log.gz 2024-05-20T18:38:30.225 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65377.log.gz 2024-05-20T18:38:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.77882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68113.log 2024-05-20T18:38:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70958.log 2024-05-20T18:38:30.226 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77882.log.gz 2024-05-20T18:38:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46745.log: /var/log/ceph/ceph-client.admin.68113.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46745.log.gz 2024-05-20T18:38:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77244.log 2024-05-20T18:38:30.226 INFO:teuthology.orchestra.run.smithi005.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68113.log.gz 2024-05-20T18:38:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-05-20T18:38:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70958.log: /var/log/ceph/ceph-client.admin.77244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45856.log 2024-05-20T18:38:30.226 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70958.log.gz 2024-05-20T18:38:30.226 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.77244.log.gz 2024-05-20T18:38:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66214.log 2024-05-20T18:38:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.45856.log: /var/log/ceph/ceph-client.admin.57982.log 2024-05-20T18:38:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57090.log 2024-05-20T18:38:30.227 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45856.log.gz 2024-05-20T18:38:30.227 INFO:teuthology.orchestra.run.smithi005.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.66214.log.gz 2024-05-20T18:38:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64978.log 2024-05-20T18:38:30.227 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57982.log.gz 2024-05-20T18:38:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51380.log 2024-05-20T18:38:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56964.log 2024-05-20T18:38:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.64978.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57090.log.gz 2024-05-20T18:38:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51380.log: gzip 88.3% -5 --verbose -- /var/log/ceph/ceph-client.admin.69760.log 2024-05-20T18:38:30.227 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64978.log.gz 2024-05-20T18:38:30.227 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51380.log.gz 2024-05-20T18:38:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69190.log 2024-05-20T18:38:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75579.log 2024-05-20T18:38:30.227 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56964.log.gz/var/log/ceph/ceph-client.admin.69760.log: 2024-05-20T18:38:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.69190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49037.log 2024-05-20T18:38:30.228 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.69760.log.gz 2024-05-20T18:38:30.228 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.75579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65209.log 2024-05-20T18:38:30.228 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49037.log: 88.2% 91.1% -- replaced with /var/log/ceph/ceph-client.admin.69190.log.gz 2024-05-20T18:38:30.228 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56053.log 2024-05-20T18:38:30.228 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.75579.log.gz 2024-05-20T18:38:30.228 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65209.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49037.log.gz 2024-05-20T18:38:30.228 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67348.log 2024-05-20T18:38:30.228 INFO:teuthology.orchestra.run.smithi005.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65209.log.gz 2024-05-20T18:38:30.228 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43182.log 2024-05-20T18:38:30.228 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.67348.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56053.log.gz 2024-05-20T18:38:30.228 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59209.log 2024-05-20T18:38:30.228 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43182.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67348.log.gz 2024-05-20T18:38:30.228 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56179.log 2024-05-20T18:38:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42822.log 2024-05-20T18:38:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59209.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43182.log.gz 2024-05-20T18:38:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45855.log 2024-05-20T18:38:30.229 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59209.log.gz 2024-05-20T18:38:30.229 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56179.log.gz 2024-05-20T18:38:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78941.log 2024-05-20T18:38:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42822.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42822.log.gz 2024-05-20T18:38:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77966.log 2024-05-20T18:38:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-20T18:38:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.78941.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45855.log.gz 2024-05-20T18:38:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.77966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43098.log 2024-05-20T18:38:30.229 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78941.log.gz 2024-05-20T18:38:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-osd.1.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.77966.log.gz 2024-05-20T18:38:30.229 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-05-20T18:38:30.230 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.43098.log: -- /var/log/ceph/ceph-client.admin.61003.log 2024-05-20T18:38:30.230 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-mgr.y.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43098.log.gz 2024-05-20T18:38:30.230 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47286.log 2024-05-20T18:38:30.230 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58024.log 2024-05-20T18:38:30.230 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61003.log.gz 2024-05-20T18:38:30.230 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.47286.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47286.log.gz 2024-05-20T18:38:30.230 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81684.log 2024-05-20T18:38:30.230 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64339.log 2024-05-20T18:38:30.230 INFO:teuthology.orchestra.run.smithi005.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58024.log.gz 2024-05-20T18:38:30.230 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.81684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50402.log 2024-05-20T18:38:30.230 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.81684.log.gz 2024-05-20T18:38:30.230 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48911.log 2024-05-20T18:38:30.231 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.64339.log: /var/log/ceph/ceph-client.admin.50402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57940.log 2024-05-20T18:38:30.231 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48911.log: 88.6% 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50402.log.gz 2024-05-20T18:38:30.231 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64339.log.gz 2024-05-20T18:38:30.231 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48911.log.gz 2024-05-20T18:38:30.231 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46349.log 2024-05-20T18:38:30.231 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.57940.log: -- /var/log/ceph/ceph-client.admin.57601.log 2024-05-20T18:38:30.231 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46349.log: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.57940.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.42952.log 2024-05-20T18:38:30.231 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-05-20T18:38:30.231 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46349.log.gz 2024-05-20T18:38:30.231 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56838.log 2024-05-20T18:38:30.231 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57601.log.gz 2024-05-20T18:38:30.231 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79978.log 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42952.log.gz 2024-05-20T18:38:30.231 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-05-20T18:38:30.231 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68371.log 2024-05-20T18:38:30.232 INFO:teuthology.orchestra.run.smithi005.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56838.log.gz 2024-05-20T18:38:30.232 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.79978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60238.log 2024-05-20T18:38:30.232 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79978.log.gz 2024-05-20T18:38:30.232 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.68371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81136.log 2024-05-20T18:38:30.232 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.68371.log.gz 2024-05-20T18:38:30.232 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-20T18:38:30.232 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60238.log: /var/log/ceph/ceph-client.admin.81136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70519.log 2024-05-20T18:38:30.232 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-osd.0.log: 89.2% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60238.log.gz 2024-05-20T18:38:30.232 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.81136.log.gz 2024-05-20T18:38:30.232 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74429.log 2024-05-20T18:38:30.232 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68511.log 2024-05-20T18:38:30.232 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70519.log: /var/log/ceph/ceph-client.admin.74429.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.70519.log.gz 2024-05-20T18:38:30.232 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82157.log 2024-05-20T18:38:30.232 INFO:teuthology.orchestra.run.smithi005.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.74429.log.gz 2024-05-20T18:38:30.233 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.68511.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.68511.log.gz 2024-05-20T18:38:30.233 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57307.log 2024-05-20T18:38:30.233 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55459.log 2024-05-20T18:38:30.233 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.82157.log: /var/log/ceph/ceph-client.admin.57307.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.82157.log.gz 2024-05-20T18:38:30.233 INFO:teuthology.orchestra.run.smithi005.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57307.log.gz 2024-05-20T18:38:30.233 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55928.log 2024-05-20T18:38:30.233 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55459.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55459.log.gz 2024-05-20T18:38:30.233 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75285.log 2024-05-20T18:38:30.233 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74430.log 2024-05-20T18:38:30.233 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55928.log: /var/log/ceph/ceph-client.admin.75285.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.55928.log.gz 2024-05-20T18:38:30.233 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75285.log.gz 2024-05-20T18:38:30.233 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44381.log 2024-05-20T18:38:30.233 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57557.log 2024-05-20T18:38:30.233 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74430.log: /var/log/ceph/ceph-client.admin.44381.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.74430.log.gz 2024-05-20T18:38:30.234 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44381.log.gz 2024-05-20T18:38:30.234 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68553.log 2024-05-20T18:38:30.234 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57557.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57557.log.gz 2024-05-20T18:38:30.234 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70748.log 2024-05-20T18:38:30.234 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54000.log 2024-05-20T18:38:30.234 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.68553.log: /var/log/ceph/ceph-client.admin.70748.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68553.log.gz 2024-05-20T18:38:30.234 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70748.log.gz 2024-05-20T18:38:30.234 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53790.log 2024-05-20T18:38:30.234 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54000.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54000.log.gz 2024-05-20T18:38:30.234 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47281.log 2024-05-20T18:38:30.234 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74215.log 2024-05-20T18:38:30.234 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.53790.log: /var/log/ceph/ceph-client.0.47281.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47281.log.gz 2024-05-20T18:38:30.235 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53790.log.gz 2024-05-20T18:38:30.235 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71765.log 2024-05-20T18:38:30.235 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53010.log 2024-05-20T18:38:30.235 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74215.log: /var/log/ceph/ceph-client.admin.71765.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74215.log.gz 2024-05-20T18:38:30.235 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71765.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58693.log 2024-05-20T18:38:30.235 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-05-20T18:38:30.235 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.53010.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.53010.log.gz 2024-05-20T18:38:30.235 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44837.log 2024-05-20T18:38:30.235 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62520.log 2024-05-20T18:38:30.235 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58693.log: /var/log/ceph/ceph-client.admin.44837.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44837.log.gz 2024-05-20T18:38:30.235 INFO:teuthology.orchestra.run.smithi005.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.58693.log.gz 2024-05-20T18:38:30.235 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63551.log 2024-05-20T18:38:30.235 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61054.log 2024-05-20T18:38:30.235 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62520.log: /var/log/ceph/ceph-client.admin.63551.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.62520.log.gz 2024-05-20T18:38:30.236 INFO:teuthology.orchestra.run.smithi005.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.63551.log.gz 2024-05-20T18:38:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68637.log 2024-05-20T18:38:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61054.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.61054.log.gz 2024-05-20T18:38:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54820.log 2024-05-20T18:38:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58993.log 2024-05-20T18:38:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.68637.log: /var/log/ceph/ceph-client.admin.54820.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68637.log.gz 2024-05-20T18:38:30.236 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54820.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57814.log 2024-05-20T18:38:30.236 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-05-20T18:38:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58993.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.58993.log.gz 2024-05-20T18:38:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82115.log 2024-05-20T18:38:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75411.log 2024-05-20T18:38:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57814.log: /var/log/ceph/ceph-client.admin.82115.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57814.log.gz 2024-05-20T18:38:30.237 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.82115.log.gz 2024-05-20T18:38:30.237 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49358.log 2024-05-20T18:38:30.237 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.75411.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75411.log.gz 2024-05-20T18:38:30.237 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69466.log 2024-05-20T18:38:30.237 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44669.log 2024-05-20T18:38:30.237 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49358.log: /var/log/ceph/ceph-client.0.69466.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.69466.log.gz 2024-05-20T18:38:30.237 INFO:teuthology.orchestra.run.smithi005.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49358.log.gz 2024-05-20T18:38:30.237 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79764.log 2024-05-20T18:38:30.237 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70687.log 2024-05-20T18:38:30.237 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44669.log: /var/log/ceph/ceph-client.admin.79764.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44669.log.gz 2024-05-20T18:38:30.237 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.79764.log.gz 2024-05-20T18:38:30.237 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75961.log 2024-05-20T18:38:30.237 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70687.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70687.log.gz 2024-05-20T18:38:30.237 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53047.log 2024-05-20T18:38:30.238 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49736.log 2024-05-20T18:38:30.238 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.75961.log: /var/log/ceph/ceph-client.admin.53047.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.75961.log.gz 2024-05-20T18:38:30.238 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70790.log 2024-05-20T18:38:30.238 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53047.log.gz 2024-05-20T18:38:30.238 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49736.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49736.log.gz 2024-05-20T18:38:30.238 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59269.log 2024-05-20T18:38:30.238 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63065.log 2024-05-20T18:38:30.238 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70790.log: /var/log/ceph/ceph-client.admin.59269.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.70790.log.gz 2024-05-20T18:38:30.238 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51859.log 2024-05-20T18:38:30.238 INFO:teuthology.orchestra.run.smithi005.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.59269.log.gz 2024-05-20T18:38:30.238 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63065.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63065.log.gz 2024-05-20T18:38:30.238 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57730.log 2024-05-20T18:38:30.238 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71891.log 2024-05-20T18:38:30.239 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51859.log: /var/log/ceph/ceph-client.admin.57730.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51859.log.gz 2024-05-20T18:38:30.239 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57730.log.gz 2024-05-20T18:38:30.239 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48113.log 2024-05-20T18:38:30.239 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71891.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71891.log.gz 2024-05-20T18:38:30.239 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58366.log 2024-05-20T18:38:30.239 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49274.log 2024-05-20T18:38:30.239 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48113.log: /var/log/ceph/ceph-client.0.58366.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58366.log.gz 2024-05-20T18:38:30.239 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48113.log.gz 2024-05-20T18:38:30.239 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63335.log 2024-05-20T18:38:30.239 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69147.log 2024-05-20T18:38:30.239 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49274.log: /var/log/ceph/ceph-client.admin.63335.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49274.log.gz 2024-05-20T18:38:30.239 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69544.log 2024-05-20T18:38:30.239 INFO:teuthology.orchestra.run.smithi005.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.63335.log.gz 2024-05-20T18:38:30.240 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.69147.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69147.log.gz 2024-05-20T18:38:30.240 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65718.log 2024-05-20T18:38:30.240 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82409.log 2024-05-20T18:38:30.240 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.69544.log: /var/log/ceph/ceph-client.admin.65718.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65718.log.gz 2024-05-20T18:38:30.240 INFO:teuthology.orchestra.run.smithi005.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.69544.log.gz 2024-05-20T18:38:30.240 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46685.log 2024-05-20T18:38:30.240 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50528.log 2024-05-20T18:38:30.240 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.82409.log: /var/log/ceph/ceph-client.admin.46685.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.82409.log.gz 2024-05-20T18:38:30.240 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46685.log.gz 2024-05-20T18:38:30.240 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61226.log 2024-05-20T18:38:30.240 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50528.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50528.log.gz 2024-05-20T18:38:30.240 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44507.log 2024-05-20T18:38:30.240 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43837.log 2024-05-20T18:38:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61226.log: /var/log/ceph/ceph-client.admin.44507.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61226.log.gz 2024-05-20T18:38:30.241 INFO:teuthology.orchestra.run.smithi005.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44507.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62040.log 2024-05-20T18:38:30.241 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-05-20T18:38:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43837.log: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.43837.log.gz 2024-05-20T18:38:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61310.log 2024-05-20T18:38:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66004.log 2024-05-20T18:38:30.241 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62040.log.gz 2024-05-20T18:38:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68757.log 2024-05-20T18:38:30.241 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61310.log.gz 2024-05-20T18:38:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66004.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66004.log.gz 2024-05-20T18:38:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76278.log 2024-05-20T18:38:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.68757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75495.log 2024-05-20T18:38:30.241 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68757.log.gz 2024-05-20T18:38:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74517.log 2024-05-20T18:38:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.75495.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.76278.log.gz 2024-05-20T18:38:30.242 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75495.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60406.log 2024-05-20T18:38:30.242 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-05-20T18:38:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43056.log 2024-05-20T18:38:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60406.log: /var/log/ceph/ceph-client.admin.74517.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60406.log.gz 2024-05-20T18:38:30.242 INFO:teuthology.orchestra.run.smithi005.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.74517.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44071.log 2024-05-20T18:38:30.242 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-05-20T18:38:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55717.log 2024-05-20T18:38:30.242 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43056.log.gz 2024-05-20T18:38:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51212.log 2024-05-20T18:38:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55717.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44071.log.gz 2024-05-20T18:38:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56315.log 2024-05-20T18:38:30.242 INFO:teuthology.orchestra.run.smithi005.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.55717.log.gz 2024-05-20T18:38:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51212.log: gzip -5 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51212.log.gz 2024-05-20T18:38:30.243 INFO:teuthology.orchestra.run.smithi005.stderr: --verbose -- /var/log/ceph/ceph-client.admin.58777.log 2024-05-20T18:38:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-20T18:38:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56315.log: /var/log/ceph/ceph-client.admin.58777.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56315.log.gz 2024-05-20T18:38:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53267.log 2024-05-20T18:38:30.243 INFO:teuthology.orchestra.run.smithi005.stderr: 91.1%/var/log/ceph/ceph-mon.a.log: -- replaced with /var/log/ceph/ceph-client.admin.58777.log.gz 2024-05-20T18:38:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71217.log 2024-05-20T18:38:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.53267.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53267.log.gz 2024-05-20T18:38:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57898.log 2024-05-20T18:38:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71217.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.71217.log.gz 2024-05-20T18:38:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73423.log 2024-05-20T18:38:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57898.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57898.log.gz 2024-05-20T18:38:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43266.log 2024-05-20T18:38:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.73423.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73423.log.gz 2024-05-20T18:38:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37526.log 2024-05-20T18:38:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43266.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43266.log.gz 2024-05-20T18:38:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76320.log 2024-05-20T18:38:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37526.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37526.log.gz 2024-05-20T18:38:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50024.log 2024-05-20T18:38:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76320.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76320.log.gz 2024-05-20T18:38:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81737.log 2024-05-20T18:38:30.245 INFO:teuthology.orchestra.run.smithi063.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-05-20T18:38:30.351 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50024.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50024.log.gz 2024-05-20T18:38:30.351 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46601.log 2024-05-20T18:38:30.351 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.81737.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.81737.log.gz 2024-05-20T18:38:30.351 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78899.log 2024-05-20T18:38:30.352 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46601.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46601.log.gz 2024-05-20T18:38:30.352 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77118.log 2024-05-20T18:38:30.352 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.78899.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.78899.log.gz 2024-05-20T18:38:30.352 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54084.log 2024-05-20T18:38:30.352 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.77118.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.77118.log.gz 2024-05-20T18:38:30.352 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49820.log 2024-05-20T18:38:30.352 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54084.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54084.log.gz 2024-05-20T18:38:30.352 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81641.log 2024-05-20T18:38:30.352 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49820.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49820.log.gz 2024-05-20T18:38:30.352 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65760.log 2024-05-20T18:38:30.352 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.81641.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.81641.log.gz 2024-05-20T18:38:30.352 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59734.log 2024-05-20T18:38:30.352 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65760.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65760.log.gz 2024-05-20T18:38:30.353 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52725.log 2024-05-20T18:38:30.353 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59734.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59734.log.gz 2024-05-20T18:38:30.353 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65124.log 2024-05-20T18:38:30.353 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52725.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52725.log.gz 2024-05-20T18:38:30.353 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80926.log 2024-05-20T18:38:30.353 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65124.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65124.log.gz 2024-05-20T18:38:30.353 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64726.log 2024-05-20T18:38:30.353 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.80926.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.80926.log.gz 2024-05-20T18:38:30.353 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70005.log 2024-05-20T18:38:30.353 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.64726.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.64726.log.gz 2024-05-20T18:38:30.353 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74303.log 2024-05-20T18:38:30.353 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70005.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.70005.log.gz 2024-05-20T18:38:30.353 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80538.log 2024-05-20T18:38:30.353 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74303.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.74303.log.gz 2024-05-20T18:38:30.354 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82509.log 2024-05-20T18:38:30.354 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.80538.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.80538.log.gz 2024-05-20T18:38:30.354 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71340.log 2024-05-20T18:38:30.354 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.82509.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.82509.log.gz 2024-05-20T18:38:30.354 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44113.log 2024-05-20T18:38:30.354 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71340.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.71340.log.gz 2024-05-20T18:38:30.354 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50108.log 2024-05-20T18:38:30.354 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44113.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44113.log.gz 2024-05-20T18:38:30.354 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56693.log 2024-05-20T18:38:30.354 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50108.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50108.log.gz 2024-05-20T18:38:30.354 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45738.log 2024-05-20T18:38:30.354 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56693.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56693.log.gz 2024-05-20T18:38:30.354 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60364.log 2024-05-20T18:38:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45738.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45738.log.gz 2024-05-20T18:38:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78265.log 2024-05-20T18:38:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60364.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60364.log.gz 2024-05-20T18:38:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79354.log 2024-05-20T18:38:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.78265.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.78265.log.gz 2024-05-20T18:38:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67394.log 2024-05-20T18:38:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.79354.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.79354.log.gz 2024-05-20T18:38:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42774.log 2024-05-20T18:38:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.67394.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.67394.log.gz 2024-05-20T18:38:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42381.log 2024-05-20T18:38:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42774.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42774.log.gz 2024-05-20T18:38:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58571.log 2024-05-20T18:38:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42381.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42381.log.gz 2024-05-20T18:38:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72143.log 2024-05-20T18:38:30.356 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58571.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58571.log.gz 2024-05-20T18:38:30.356 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59944.log 2024-05-20T18:38:30.356 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.72143.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72143.log.gz 2024-05-20T18:38:30.356 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80842.log 2024-05-20T18:38:30.356 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59944.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59944.log.gz 2024-05-20T18:38:30.356 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57856.log 2024-05-20T18:38:30.356 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.80842.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.80842.log.gz 2024-05-20T18:38:30.356 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46787.log 2024-05-20T18:38:30.356 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57856.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57856.log.gz 2024-05-20T18:38:30.356 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78212.log 2024-05-20T18:38:30.356 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46787.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.46787.log.gz 2024-05-20T18:38:30.356 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60915.log 2024-05-20T18:38:30.356 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.78212.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.78212.log.gz 2024-05-20T18:38:30.356 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55326.log 2024-05-20T18:38:30.357 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60915.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60915.log.gz 2024-05-20T18:38:30.357 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44879.log 2024-05-20T18:38:30.357 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.55326.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55326.log.gz 2024-05-20T18:38:30.357 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51901.log 2024-05-20T18:38:30.357 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44879.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44879.log.gz 2024-05-20T18:38:30.357 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45854.log 2024-05-20T18:38:30.357 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51901.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51901.log.gz 2024-05-20T18:38:30.357 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71975.log 2024-05-20T18:38:30.357 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45854.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45854.log.gz 2024-05-20T18:38:30.357 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63919.log 2024-05-20T18:38:30.357 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71975.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71975.log.gz 2024-05-20T18:38:30.357 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71511.log 2024-05-20T18:38:30.357 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63919.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63919.log.gz 2024-05-20T18:38:30.357 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51943.log 2024-05-20T18:38:30.358 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71511.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71511.log.gz 2024-05-20T18:38:30.358 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71467.log 2024-05-20T18:38:30.358 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51943.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51943.log.gz 2024-05-20T18:38:30.358 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43416.log 2024-05-20T18:38:30.358 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71467.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71467.log.gz 2024-05-20T18:38:30.358 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62432.log 2024-05-20T18:38:30.358 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43416.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43416.log.gz 2024-05-20T18:38:30.358 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79025.log 2024-05-20T18:38:30.358 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62432.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62432.log.gz 2024-05-20T18:38:30.358 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65167.log 2024-05-20T18:38:30.358 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.79025.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79025.log.gz 2024-05-20T18:38:30.358 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42477.log 2024-05-20T18:38:30.359 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65167.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65167.log.gz 2024-05-20T18:38:30.359 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53561.log 2024-05-20T18:38:30.359 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42477.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42477.log.gz 2024-05-20T18:38:30.359 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72773.log 2024-05-20T18:38:30.359 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.53561.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53561.log.gz 2024-05-20T18:38:30.359 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54648.log 2024-05-20T18:38:30.359 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.72773.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72773.log.gz 2024-05-20T18:38:30.359 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77840.log 2024-05-20T18:38:30.359 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54648.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54648.log.gz 2024-05-20T18:38:30.359 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46951.log 2024-05-20T18:38:30.359 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.77840.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.77840.log.gz 2024-05-20T18:38:30.359 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66256.log 2024-05-20T18:38:30.359 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46951.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46951.log.gz 2024-05-20T18:38:30.359 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44963.log 2024-05-20T18:38:30.360 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66256.log: 91.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-05-20T18:38:30.360 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66256.log.gz 2024-05-20T18:38:30.360 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51594.log 2024-05-20T18:38:30.360 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.44963.log: -- /var/log/ceph/ceph-client.admin.53158.log 2024-05-20T18:38:30.360 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44963.log.gz 2024-05-20T18:38:30.360 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76152.log 2024-05-20T18:38:30.360 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.53158.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51594.log.gz 2024-05-20T18:38:30.360 INFO:teuthology.orchestra.run.smithi005.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.53158.log.gz 2024-05-20T18:38:30.360 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74907.log 2024-05-20T18:38:30.360 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44339.log 2024-05-20T18:38:30.360 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76152.log.gz 2024-05-20T18:38:30.360 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56567.log 2024-05-20T18:38:30.360 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44339.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.74907.log.gz 2024-05-20T18:38:30.361 INFO:teuthology.orchestra.run.smithi005.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44339.log.gz 2024-05-20T18:38:30.361 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61268.log 2024-05-20T18:38:30.361 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72689.log 2024-05-20T18:38:30.361 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61268.log: /var/log/ceph/ceph-client.admin.56567.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61268.log.gz 89.1% 2024-05-20T18:38:30.361 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56567.log.gz 2024-05-20T18:38:30.361 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54778.log 2024-05-20T18:38:30.361 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70561.log 2024-05-20T18:38:30.361 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.72689.log: /var/log/ceph/ceph-client.admin.54778.log: 88.8% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54778.log.gz 2024-05-20T18:38:30.361 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.72689.log.gz 2024-05-20T18:38:30.361 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67306.log 2024-05-20T18:38:30.361 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74864.log 2024-05-20T18:38:30.361 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70561.log.gz 2024-05-20T18:38:30.361 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.67306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79224.log 2024-05-20T18:38:30.361 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74864.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.67306.log.gz 2024-05-20T18:38:30.362 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.74864.log.gz 2024-05-20T18:38:30.362 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65846.log 2024-05-20T18:38:30.362 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66299.log 2024-05-20T18:38:30.362 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.79224.log: /var/log/ceph/ceph-client.admin.65846.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.79224.log.gz 2024-05-20T18:38:30.362 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65846.log.gz 2024-05-20T18:38:30.362 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79551.log 2024-05-20T18:38:30.362 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71807.log 2024-05-20T18:38:30.362 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66299.log: /var/log/ceph/ceph-client.admin.79551.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66299.log.gz 2024-05-20T18:38:30.362 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81365.log 2024-05-20T18:38:30.362 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71807.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71807.log.gz 2024-05-20T18:38:30.362 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72457.log 2024-05-20T18:38:30.362 INFO:teuthology.orchestra.run.smithi005.stderr: 91.8% -- replaced with /var/log/ceph/ceph-client.admin.79551.log.gz 2024-05-20T18:38:30.362 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.81365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54597.log 2024-05-20T18:38:30.363 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.72457.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.72457.log.gz 2024-05-20T18:38:30.363 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.81365.log.gz 2024-05-20T18:38:30.363 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69694.log 2024-05-20T18:38:30.363 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48449.log 2024-05-20T18:38:30.363 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54597.log.gz 2024-05-20T18:38:30.363 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.69694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48869.log 2024-05-20T18:38:30.363 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48449.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48449.log.gz 2024-05-20T18:38:30.363 INFO:teuthology.orchestra.run.smithi005.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.69694.log.gz 2024-05-20T18:38:30.363 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58174.log 2024-05-20T18:38:30.363 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44029.log 2024-05-20T18:38:30.363 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48869.log.gz 2024-05-20T18:38:30.363 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68679.log 2024-05-20T18:38:30.363 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44029.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58174.log.gz 2024-05-20T18:38:30.364 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44029.log.gz 2024-05-20T18:38:30.364 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52069.log 2024-05-20T18:38:30.364 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48533.log 2024-05-20T18:38:30.364 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.68679.log: /var/log/ceph/ceph-client.admin.52069.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.68679.log.gz 2024-05-20T18:38:30.364 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63023.log 2024-05-20T18:38:30.364 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48533.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.52069.log.gz 2024-05-20T18:38:30.364 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48533.log.gz 2024-05-20T18:38:30.364 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46517.log 2024-05-20T18:38:30.364 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.63023.log: -- /var/log/ceph/ceph-client.admin.68819.log 2024-05-20T18:38:30.364 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63023.log.gz 2024-05-20T18:38:30.364 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76872.log 2024-05-20T18:38:30.364 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.68819.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46517.log.gz 2024-05-20T18:38:30.364 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.68819.log.gz 2024-05-20T18:38:30.365 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65962.log 2024-05-20T18:38:30.365 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43350.log 2024-05-20T18:38:30.365 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.76872.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.76872.log.gz 2024-05-20T18:38:30.365 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60280.log 2024-05-20T18:38:30.365 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65962.log.gz 2024-05-20T18:38:30.365 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60070.log 2024-05-20T18:38:30.365 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60280.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60280.log.gz 2024-05-20T18:38:30.365 INFO:teuthology.orchestra.run.smithi005.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.43350.log.gz 2024-05-20T18:38:30.365 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45522.log 2024-05-20T18:38:30.365 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45739.log 2024-05-20T18:38:30.365 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60070.log.gz 2024-05-20T18:38:30.365 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56221.log 2024-05-20T18:38:30.365 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45739.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45522.log.gz 2024-05-20T18:38:30.365 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45739.log.gz 2024-05-20T18:38:30.366 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71298.log 2024-05-20T18:38:30.366 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54382.log 2024-05-20T18:38:30.366 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56221.log.gz 2024-05-20T18:38:30.366 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76572.log 2024-05-20T18:38:30.366 INFO:teuthology.orchestra.run.smithi005.stderr: 87.8%/var/log/ceph/ceph-client.admin.54382.log: -- replaced with /var/log/ceph/ceph-client.admin.71298.log.gz 2024-05-20T18:38:30.366 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54382.log.gz 2024-05-20T18:38:30.366 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50798.log 2024-05-20T18:38:30.366 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55885.log 2024-05-20T18:38:30.366 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76572.log: /var/log/ceph/ceph-client.admin.50798.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.76572.log.gz 2024-05-20T18:38:30.366 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72461.log 2024-05-20T18:38:30.366 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50798.log.gz 2024-05-20T18:38:30.366 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72892.log 2024-05-20T18:38:30.367 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55885.log: /var/log/ceph/ceph-client.0.72461.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.72461.log.gz 2024-05-20T18:38:30.367 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55885.log.gz 2024-05-20T18:38:30.367 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43308.log 2024-05-20T18:38:30.367 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.72892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57389.log 2024-05-20T18:38:30.367 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72892.log.gz 2024-05-20T18:38:30.367 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48953.log 2024-05-20T18:38:30.367 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3%/var/log/ceph/ceph-client.admin.57389.log: -- replaced with /var/log/ceph/ceph-client.admin.43308.log.gz 2024-05-20T18:38:30.367 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57389.log.gzgzip 2024-05-20T18:38:30.367 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.52323.log 2024-05-20T18:38:30.367 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76488.log 2024-05-20T18:38:30.367 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.52323.log: /var/log/ceph/ceph-client.admin.48953.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52323.log.gz 2024-05-20T18:38:30.367 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48953.log.gz 2024-05-20T18:38:30.367 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.47724.log 2024-05-20T18:38:30.367 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56137.log 2024-05-20T18:38:30.368 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76488.log.gz 2024-05-20T18:38:30.368 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.2.47724.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.47724.log.gz 2024-05-20T18:38:30.368 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46139.log 2024-05-20T18:38:30.368 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48491.log 2024-05-20T18:38:30.368 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56137.log: /var/log/ceph/ceph-client.admin.46139.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56137.log.gz 2024-05-20T18:38:30.368 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46139.log.gz 2024-05-20T18:38:30.368 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44297.log 2024-05-20T18:38:30.368 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68944.log 2024-05-20T18:38:30.368 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44297.log: /var/log/ceph/ceph-client.admin.48491.log: 87.8% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48491.log.gz 2024-05-20T18:38:30.368 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44297.log.gz 2024-05-20T18:38:30.368 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56525.log 2024-05-20T18:38:30.368 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72731.log 2024-05-20T18:38:30.368 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.68944.log: /var/log/ceph/ceph-client.admin.56525.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56525.log.gz 2024-05-20T18:38:30.369 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70645.log 2024-05-20T18:38:30.369 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.72731.log: 87.5% 92.3% -- replaced with /var/log/ceph/ceph-client.admin.68944.log.gz 2024-05-20T18:38:30.369 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.72731.log.gz 2024-05-20T18:38:30.369 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44197.log 2024-05-20T18:38:30.369 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74162.log 2024-05-20T18:38:30.369 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70645.log: /var/log/ceph/ceph-client.admin.44197.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.70645.log.gz 2024-05-20T18:38:30.369 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44197.log.gzgzip 2024-05-20T18:38:30.369 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.60028.log 2024-05-20T18:38:30.369 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45516.log 2024-05-20T18:38:30.369 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74162.log: /var/log/ceph/ceph-client.admin.60028.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74162.log.gz 2024-05-20T18:38:30.369 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55502.log 2024-05-20T18:38:30.369 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60028.log.gz 2024-05-20T18:38:30.369 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53121.log 2024-05-20T18:38:30.369 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55502.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45516.log.gz 2024-05-20T18:38:30.370 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55502.log.gz 2024-05-20T18:38:30.370 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60959.log 2024-05-20T18:38:30.370 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70916.log 2024-05-20T18:38:30.370 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.53121.log: /var/log/ceph/ceph-client.admin.60959.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.53121.log.gz 2024-05-20T18:38:30.370 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60959.log.gz 2024-05-20T18:38:30.370 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53687.log 2024-05-20T18:38:30.370 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49232.log 2024-05-20T18:38:30.370 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70916.log: /var/log/ceph/ceph-client.admin.53687.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70916.log.gz 2024-05-20T18:38:30.370 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53687.log.gz 2024-05-20T18:38:30.370 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78307.log 2024-05-20T18:38:30.370 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53958.log 2024-05-20T18:38:30.370 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49232.log: /var/log/ceph/ceph-client.admin.78307.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49232.log.gz 2024-05-20T18:38:30.371 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62774.log 2024-05-20T18:38:30.371 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.78307.log.gz 2024-05-20T18:38:30.371 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.53958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76404.log 2024-05-20T18:38:30.371 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.62774.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.62774.log.gz 2024-05-20T18:38:30.371 INFO:teuthology.orchestra.run.smithi005.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53958.log.gz 2024-05-20T18:38:30.371 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72251.log 2024-05-20T18:38:30.371 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56922.log 2024-05-20T18:38:30.371 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.76404.log.gz 2024-05-20T18:38:30.371 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.72251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72850.log 2024-05-20T18:38:30.371 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56922.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72251.log.gz 2024-05-20T18:38:30.371 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56922.log.gz 2024-05-20T18:38:30.371 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76362.log 2024-05-20T18:38:30.371 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.72850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42910.log 2024-05-20T18:38:30.371 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.72850.log.gz 2024-05-20T18:38:30.372 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49946.log 2024-05-20T18:38:30.372 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42910.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.76362.log.gz 2024-05-20T18:38:30.372 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42910.log.gz 2024-05-20T18:38:30.372 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68221.log 2024-05-20T18:38:30.372 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69802.log 2024-05-20T18:38:30.372 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49946.log: /var/log/ceph/ceph-client.admin.68221.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49946.log.gz 2024-05-20T18:38:30.372 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75201.log 2024-05-20T18:38:30.372 INFO:teuthology.orchestra.run.smithi005.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.68221.log.gz 2024-05-20T18:38:30.372 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.69802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75159.log 2024-05-20T18:38:30.372 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.75201.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.69802.log.gz 2024-05-20T18:38:30.372 INFO:teuthology.orchestra.run.smithi005.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44239.log 2024-05-20T18:38:30.372 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.75201.log.gz 2024-05-20T18:38:30.372 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64810.log 2024-05-20T18:38:30.373 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.75159.log: /var/log/ceph/ceph-client.admin.44239.log: 88.1% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44239.log.gz 2024-05-20T18:38:30.373 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.75159.log.gz 2024-05-20T18:38:30.373 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52767.log 2024-05-20T18:38:30.373 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.64810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65081.log 2024-05-20T18:38:30.373 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64810.log.gz 2024-05-20T18:38:30.373 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50588.log 2024-05-20T18:38:30.373 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65081.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52767.log.gz 2024-05-20T18:38:30.373 INFO:teuthology.orchestra.run.smithi005.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.65081.log.gz 2024-05-20T18:38:30.373 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64600.log 2024-05-20T18:38:30.373 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47903.log 2024-05-20T18:38:30.373 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50588.log: /var/log/ceph/ceph-client.admin.64600.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50588.log.gz 2024-05-20T18:38:30.373 INFO:teuthology.orchestra.run.smithi005.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.64600.log.gz 2024-05-20T18:38:30.373 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52135.log 2024-05-20T18:38:30.373 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51775.log 2024-05-20T18:38:30.374 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.47903.log: /var/log/ceph/ceph-client.admin.52135.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47903.log.gz 2024-05-20T18:38:30.374 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70435.log 2024-05-20T18:38:30.374 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52135.log.gz 2024-05-20T18:38:30.374 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82469.log 2024-05-20T18:38:30.374 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70435.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51775.log.gz 2024-05-20T18:38:30.374 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70435.log.gz 2024-05-20T18:38:30.374 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75453.log 2024-05-20T18:38:30.374 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60789.log 2024-05-20T18:38:30.374 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.82469.log: /var/log/ceph/ceph-client.admin.75453.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.82469.log.gz 2024-05-20T18:38:30.374 INFO:teuthology.orchestra.run.smithi005.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.75453.log.gz 2024-05-20T18:38:30.374 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71042.log 2024-05-20T18:38:30.374 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51422.log 2024-05-20T18:38:30.374 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60789.log: /var/log/ceph/ceph-client.admin.71042.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60789.log.gz 2024-05-20T18:38:30.375 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71042.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53477.log 2024-05-20T18:38:30.375 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-05-20T18:38:30.375 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78076.log 2024-05-20T18:38:30.375 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.53477.log: /var/log/ceph/ceph-client.admin.51422.log: 88.2% 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51422.log.gz 2024-05-20T18:38:30.375 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53477.log.gz 2024-05-20T18:38:30.375 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65251.log 2024-05-20T18:38:30.375 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51691.log 2024-05-20T18:38:30.375 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.78076.log: /var/log/ceph/ceph-client.admin.65251.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.78076.log.gz 2024-05-20T18:38:30.375 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65251.log.gz 2024-05-20T18:38:30.375 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68842.log 2024-05-20T18:38:30.407 INFO:teuthology.orchestra.run.smithi063.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-20T18:38:30.420 INFO:teuthology.orchestra.run.smithi063.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-20T18:38:30.477 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51691.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79420.log 2024-05-20T18:38:30.477 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51691.log.gz 2024-05-20T18:38:30.477 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.68842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73465.log 2024-05-20T18:38:30.477 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.79420.log: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.68842.log.gz 2024-05-20T18:38:30.477 INFO:teuthology.orchestra.run.smithi005.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.79420.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48407.log 2024-05-20T18:38:30.477 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-05-20T18:38:30.477 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.73465.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73465.log.gz 2024-05-20T18:38:30.477 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.73549.log 2024-05-20T18:38:30.477 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45005.log 2024-05-20T18:38:30.477 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48407.log: /var/log/ceph/ceph-client.admin.73549.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48407.log.gz 2024-05-20T18:38:30.477 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.73549.log.gz 2024-05-20T18:38:30.478 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49562.log 2024-05-20T18:38:30.478 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46433.log 2024-05-20T18:38:30.478 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45005.log.gz 2024-05-20T18:38:30.478 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44627.log 2024-05-20T18:38:30.478 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46433.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49562.log.gz 2024-05-20T18:38:30.478 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46433.log.gz 2024-05-20T18:38:30.478 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72017.log 2024-05-20T18:38:30.478 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51733.log 2024-05-20T18:38:30.478 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44627.log: /var/log/ceph/ceph-client.admin.72017.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44627.log.gz 2024-05-20T18:38:30.478 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.72017.log.gz 2024-05-20T18:38:30.478 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71723.log 2024-05-20T18:38:30.478 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62082.log 2024-05-20T18:38:30.478 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51733.log: /var/log/ceph/ceph-client.admin.71723.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51733.log.gz 2024-05-20T18:38:30.479 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.71723.log.gz 2024-05-20T18:38:30.479 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82367.log 2024-05-20T18:38:30.479 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67789.log 2024-05-20T18:38:30.479 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62082.log: /var/log/ceph/ceph-client.admin.82367.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62082.log.gz 2024-05-20T18:38:30.479 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.82367.log.gz 2024-05-20T18:38:30.479 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78353.log 2024-05-20T18:38:30.479 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59556.log 2024-05-20T18:38:30.479 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.67789.log: /var/log/ceph/ceph-client.admin.78353.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78353.log.gz 2024-05-20T18:38:30.479 INFO:teuthology.orchestra.run.smithi005.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.67789.log.gz 2024-05-20T18:38:30.479 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45176.log 2024-05-20T18:38:30.479 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68595.log 2024-05-20T18:38:30.479 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59556.log.gz 2024-05-20T18:38:30.479 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76064.log 2024-05-20T18:38:30.480 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.68595.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45176.log.gz 2024-05-20T18:38:30.480 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.68595.log.gz 2024-05-20T18:38:30.480 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64297.log 2024-05-20T18:38:30.480 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48155.log 2024-05-20T18:38:30.480 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76064.log: /var/log/ceph/ceph-client.admin.64297.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.76064.log.gz 2024-05-20T18:38:30.480 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64297.log.gz 2024-05-20T18:38:30.480 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59143.log 2024-05-20T18:38:30.480 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71639.log 2024-05-20T18:38:30.480 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48155.log: /var/log/ceph/ceph-client.admin.59143.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48155.log.gz 2024-05-20T18:38:30.480 INFO:teuthology.orchestra.run.smithi005.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.59143.log.gz 2024-05-20T18:38:30.480 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72971.log 2024-05-20T18:38:30.480 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71849.log 2024-05-20T18:38:30.480 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71639.log.gz 2024-05-20T18:38:30.480 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.72971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71933.log 2024-05-20T18:38:30.481 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71849.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72971.log.gz 2024-05-20T18:38:30.481 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79144.log 2024-05-20T18:38:30.481 INFO:teuthology.orchestra.run.smithi005.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71849.log.gz 2024-05-20T18:38:30.481 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45521.log 2024-05-20T18:38:30.481 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.79144.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71933.log.gz 2024-05-20T18:38:30.481 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79144.log.gz 2024-05-20T18:38:30.481 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64642.log 2024-05-20T18:38:30.481 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.31244.log 2024-05-20T18:38:30.481 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45521.log: /var/log/ceph/ceph-client.admin.64642.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45521.log.gz 2024-05-20T18:38:30.481 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64642.log.gz 2024-05-20T18:38:30.481 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66642.log 2024-05-20T18:38:30.481 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77202.log 2024-05-20T18:38:30.482 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.66642.log: /var/log/ceph/ceph.tmp-client.admin.31244.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.31244.log.gz 2024-05-20T18:38:30.482 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.66642.log.gz 2024-05-20T18:38:30.482 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53309.log 2024-05-20T18:38:30.482 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48281.log 2024-05-20T18:38:30.482 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.77202.log: /var/log/ceph/ceph-client.admin.53309.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77202.log.gz 2024-05-20T18:38:30.482 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75033.log 2024-05-20T18:38:30.482 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53309.log.gz 2024-05-20T18:38:30.482 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37262.log 2024-05-20T18:38:30.482 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48281.log: /var/log/ceph/ceph-client.admin.75033.log: 88.5% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48281.log.gz 2024-05-20T18:38:30.482 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.75033.log.gz 2024-05-20T18:38:30.482 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57048.log 2024-05-20T18:38:30.482 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74822.log 2024-05-20T18:38:30.482 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37262.log: /var/log/ceph/ceph-client.admin.57048.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.37262.log.gz 2024-05-20T18:38:30.483 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57048.log.gz 2024-05-20T18:38:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58066.log 2024-05-20T18:38:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58362.log 2024-05-20T18:38:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74822.log: /var/log/ceph/ceph-client.admin.58066.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74822.log.gz 2024-05-20T18:38:30.483 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58066.log.gz 2024-05-20T18:38:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51506.log 2024-05-20T18:38:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45523.log 2024-05-20T18:38:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.58362.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58362.log.gz 2024-05-20T18:38:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61956.log 2024-05-20T18:38:30.483 INFO:teuthology.orchestra.run.smithi005.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51506.log.gz 2024-05-20T18:38:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64768.log 2024-05-20T18:38:30.483 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45523.log.gz 2024-05-20T18:38:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44465.log 2024-05-20T18:38:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.64768.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61956.log.gz 2024-05-20T18:38:30.484 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64768.log.gz 2024-05-20T18:38:30.484 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57431.log 2024-05-20T18:38:30.484 INFO:teuthology.orchestra.run.smithi005.stderr:gzip/var/log/ceph/ceph-client.admin.44465.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46097.log 2024-05-20T18:38:30.484 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44465.log.gz 2024-05-20T18:38:30.484 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74667.log 2024-05-20T18:38:30.484 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46097.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57431.log.gz 2024-05-20T18:38:30.484 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.77617.log 2024-05-20T18:38:30.484 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46097.log.gz 2024-05-20T18:38:30.484 INFO:teuthology.orchestra.run.smithi005.stderr:gzip/var/log/ceph/ceph-client.admin.74667.log: -5 --verbose -- /var/log/ceph/ceph-client.0.73163.log 2024-05-20T18:38:30.484 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.74667.log.gz 2024-05-20T18:38:30.484 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.77617.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.77617.log.gzgzip 2024-05-20T18:38:30.484 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56651.log 2024-05-20T18:38:30.484 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48365.log 2024-05-20T18:38:30.485 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56651.log: /var/log/ceph/ceph-client.0.73163.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.73163.log.gz 2024-05-20T18:38:30.485 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44423.log 2024-05-20T18:38:30.485 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56651.log.gz 2024-05-20T18:38:30.485 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62244.log 2024-05-20T18:38:30.485 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44423.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48365.log.gz 2024-05-20T18:38:30.485 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44423.log.gz 2024-05-20T18:38:30.485 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45185.log 2024-05-20T18:38:30.485 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64107.log 2024-05-20T18:38:30.485 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62244.log: /var/log/ceph/ceph-client.admin.45185.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62244.log.gz 2024-05-20T18:38:30.485 INFO:teuthology.orchestra.run.smithi005.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45185.log.gz 2024-05-20T18:38:30.485 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48197.log 2024-05-20T18:38:30.485 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.0.64107.log: /var/log/ceph/ceph-client.admin.52641.log 2024-05-20T18:38:30.485 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.64107.log.gz 2024-05-20T18:38:30.485 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63509.log 2024-05-20T18:38:30.486 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52641.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48197.log.gz 2024-05-20T18:38:30.486 INFO:teuthology.orchestra.run.smithi005.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-05-20T18:38:30.486 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75796.log 2024-05-20T18:38:30.486 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52641.log.gz 2024-05-20T18:38:30.486 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70089.log 2024-05-20T18:38:30.486 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63509.log.gz/var/log/ceph/ceph-client.admin.75796.log: 2024-05-20T18:38:30.486 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63769.log 2024-05-20T18:38:30.486 INFO:teuthology.orchestra.run.smithi005.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.75796.log.gz 2024-05-20T18:38:30.486 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65919.log 2024-05-20T18:38:30.486 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70089.log: /var/log/ceph/ceph-client.admin.63769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70832.log 2024-05-20T18:38:30.486 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65919.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70089.log.gz 2024-05-20T18:38:30.486 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63769.log.gz 2024-05-20T18:38:30.486 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.65919.log.gz -5 2024-05-20T18:38:30.487 INFO:teuthology.orchestra.run.smithi005.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45177.log 2024-05-20T18:38:30.487 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46307.log 2024-05-20T18:38:30.487 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70832.log: gzip/var/log/ceph/ceph-client.admin.45177.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.57227.log 2024-05-20T18:38:30.487 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70832.log.gz 2024-05-20T18:38:30.487 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2%/var/log/ceph/ceph-client.admin.46307.log: -- replaced with /var/log/ceph/ceph-client.admin.45177.log.gz 2024-05-20T18:38:30.487 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46307.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75075.log 2024-05-20T18:38:30.487 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-05-20T18:38:30.487 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63250.log 2024-05-20T18:38:30.487 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53916.log 2024-05-20T18:38:30.487 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57227.log.gz 2024-05-20T18:38:30.487 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.75075.log: /var/log/ceph/ceph-client.admin.63250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74257.log 2024-05-20T18:38:30.487 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.75075.log.gz 2024-05-20T18:38:30.488 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.53916.log: gzip -5 --verbose 90.9% -- replaced with /var/log/ceph/ceph-client.admin.63250.log.gz 2024-05-20T18:38:30.488 INFO:teuthology.orchestra.run.smithi005.stderr: -- /var/log/ceph/ceph-client.admin.48659.log 2024-05-20T18:38:30.488 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74257.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58495.log 2024-05-20T18:38:30.488 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53916.log.gz 2024-05-20T18:38:30.488 INFO:teuthology.orchestra.run.smithi005.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.74257.log.gz 2024-05-20T18:38:30.488 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68005.log 2024-05-20T18:38:30.488 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48659.log: /var/log/ceph/ceph-client.admin.58495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65293.log 2024-05-20T18:38:30.488 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6%/var/log/ceph/ceph-client.admin.68005.log: -- replaced with /var/log/ceph/ceph-client.admin.48659.log.gz 2024-05-20T18:38:30.488 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58495.log.gz 2024-05-20T18:38:30.488 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68005.log.gz 2024-05-20T18:38:30.488 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67690.log 2024-05-20T18:38:30.488 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80800.log 2024-05-20T18:38:30.489 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65293.log: /var/log/ceph/ceph-client.0.67690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44795.log 2024-05-20T18:38:30.489 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67690.log.gz 2024-05-20T18:38:30.489 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.80800.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65293.log.gz 2024-05-20T18:38:30.489 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56483.log 2024-05-20T18:38:30.489 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.80800.log.gz 2024-05-20T18:38:30.489 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81323.log 2024-05-20T18:38:30.489 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44795.log: /var/log/ceph/ceph-client.admin.56483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45180.log 2024-05-20T18:38:30.489 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.81323.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44795.log.gz 2024-05-20T18:38:30.489 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56483.log.gz 2024-05-20T18:38:30.489 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- 88.3% /var/log/ceph/ceph-client.admin.53832.log 2024-05-20T18:38:30.489 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.81323.log.gz 2024-05-20T18:38:30.489 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.45180.log: -- /var/log/ceph/ceph-client.admin.61352.log 2024-05-20T18:38:30.489 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.53832.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45180.log.gzgzip 2024-05-20T18:38:30.490 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.52327.log 2024-05-20T18:38:30.490 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53832.log.gz 2024-05-20T18:38:30.490 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63811.log 2024-05-20T18:38:30.490 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61352.log: /var/log/ceph/ceph-client.0.52327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71681.log 2024-05-20T18:38:30.490 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52327.log.gz 2024-05-20T18:38:30.490 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61352.log.gz 2024-05-20T18:38:30.490 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50672.log 2024-05-20T18:38:30.490 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61096.log 2024-05-20T18:38:30.490 INFO:teuthology.orchestra.run.smithi005.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.63811.log.gz 2024-05-20T18:38:30.490 INFO:teuthology.orchestra.run.smithi005.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.71681.log.gz 2024-05-20T18:38:30.490 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55030.log 2024-05-20T18:38:30.490 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50672.log: /var/log/ceph/ceph-client.admin.61096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50924.log 2024-05-20T18:38:30.490 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50672.log.gz 2024-05-20T18:38:30.491 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4%/var/log/ceph/ceph-client.admin.55030.log: -- replaced with /var/log/ceph/ceph-client.admin.61096.log.gz 2024-05-20T18:38:30.491 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82199.log 2024-05-20T18:38:30.491 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50924.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55030.log.gz 2024-05-20T18:38:30.491 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81821.log 2024-05-20T18:38:30.491 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45857.log 2024-05-20T18:38:30.491 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0%/var/log/ceph/ceph-client.admin.82199.log: -- replaced with /var/log/ceph/ceph-client.admin.50924.log.gz 2024-05-20T18:38:30.491 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.81821.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.82199.log.gz 2024-05-20T18:38:30.491 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53519.log 2024-05-20T18:38:30.491 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71555.log 2024-05-20T18:38:30.491 INFO:teuthology.orchestra.run.smithi005.stderr: 87.3%/var/log/ceph/ceph-client.admin.45857.log: -- replaced with /var/log/ceph/ceph-client.admin.81821.log.gz 2024-05-20T18:38:30.491 INFO:teuthology.orchestra.run.smithi005.stderr: 87.3%/var/log/ceph/ceph-client.admin.53519.log: -- replaced with /var/log/ceph/ceph-client.admin.45857.log.gzgzip 2024-05-20T18:38:30.491 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.77160.log 2024-05-20T18:38:30.491 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71555.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53519.log.gz 2024-05-20T18:38:30.492 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71555.log.gz 2024-05-20T18:38:30.492 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-05-20T18:38:30.492 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54946.log 2024-05-20T18:38:30.492 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.77160.log: /var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53729.log 2024-05-20T18:38:30.492 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54946.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77160.log.gz 2024-05-20T18:38:30.492 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59692.log 2024-05-20T18:38:30.492 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54946.log.gz 2024-05-20T18:38:30.492 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.53729.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53729.log.gz 2024-05-20T18:38:30.492 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51085.log 2024-05-20T18:38:30.492 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64111.log 2024-05-20T18:38:30.492 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59692.log: /var/log/ceph/ceph-client.admin.51085.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59692.log.gz 2024-05-20T18:38:30.492 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51085.log.gz 2024-05-20T18:38:30.493 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77363.log 2024-05-20T18:38:30.493 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51338.log 2024-05-20T18:38:30.493 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.77363.log: /var/log/ceph/ceph-client.0.64111.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.64111.log.gz 2024-05-20T18:38:30.493 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73849.log 2024-05-20T18:38:30.493 INFO:teuthology.orchestra.run.smithi005.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-05-20T18:38:30.493 INFO:teuthology.orchestra.run.smithi005.stderr: 91.2%/var/log/ceph/ceph-client.admin.51338.log: -- replaced with /var/log/ceph/ceph-client.admin.77363.log.gz 2024-05-20T18:38:30.493 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64936.log 2024-05-20T18:38:30.493 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.73849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69005.log 2024-05-20T18:38:30.493 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51338.log.gz 2024-05-20T18:38:30.493 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.64936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60665.log 2024-05-20T18:38:30.493 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64936.log.gz 2024-05-20T18:38:30.493 INFO:teuthology.orchestra.run.smithi005.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.73849.log.gz 2024-05-20T18:38:30.493 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.69005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56609.log 2024-05-20T18:38:30.494 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81989.log 2024-05-20T18:38:30.494 INFO:teuthology.orchestra.run.smithi005.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.69005.log.gz 2024-05-20T18:38:30.494 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56609.log: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.77621.log 2024-05-20T18:38:30.494 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60665.log.gz 2024-05-20T18:38:30.494 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56609.log.gz 2024-05-20T18:38:30.494 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56095.log 2024-05-20T18:38:30.494 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.81989.log: /var/log/ceph/ceph-client.0.77621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81239.log 2024-05-20T18:38:30.494 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.77621.log.gz 2024-05-20T18:38:30.494 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56095.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.81989.log.gz 2024-05-20T18:38:30.494 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61736.log 2024-05-20T18:38:30.494 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56095.log.gz 2024-05-20T18:38:30.494 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.81239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53393.log 2024-05-20T18:38:30.494 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.81239.log.gz 2024-05-20T18:38:30.495 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.61736.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.61736.log.gz 2024-05-20T18:38:30.495 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80454.log 2024-05-20T18:38:30.495 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73507.log 2024-05-20T18:38:30.495 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.53393.log: /var/log/ceph/ceph-client.admin.80454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65802.log 2024-05-20T18:38:30.495 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.73507.log: 88.8% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53393.log.gz 2024-05-20T18:38:30.495 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.80454.log.gz 2024-05-20T18:38:30.495 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73507.log.gz 2024-05-20T18:38:30.495 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45186.log 2024-05-20T18:38:30.495 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46265.log 2024-05-20T18:38:30.495 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65802.log.gz 2024-05-20T18:38:30.495 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53435.log 2024-05-20T18:38:30.495 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45186.log.gz 2024-05-20T18:38:30.495 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73762.log 2024-05-20T18:38:30.496 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74601.log 2024-05-20T18:38:30.496 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6%/var/log/ceph/ceph-client.admin.53435.log: -- replaced with /var/log/ceph/ceph-client.admin.46265.log.gz 2024-05-20T18:38:30.496 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.73762.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53435.log.gzgzip 86.6% -- replaced with /var/log/ceph/ceph-client.admin.73762.log.gz 2024-05-20T18:38:30.496 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45187.log 2024-05-20T18:38:30.496 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-05-20T18:38:30.496 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59514.log 2024-05-20T18:38:30.496 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59902.log 2024-05-20T18:38:30.496 INFO:teuthology.orchestra.run.smithi005.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.74601.log.gz 2024-05-20T18:38:30.496 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45187.log.gz 2024-05-20T18:38:30.496 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.78649.log 2024-05-20T18:38:30.496 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.78653.log 2024-05-20T18:38:30.496 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59514.log.gz 2024-05-20T18:38:30.496 INFO:teuthology.orchestra.run.smithi005.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59902.log.gz 2024-05-20T18:38:30.497 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.78649.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.78649.log.gz 2024-05-20T18:38:30.497 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77798.log 2024-05-20T18:38:30.497 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.78653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42730.log 2024-05-20T18:38:30.497 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.78653.log.gz 2024-05-20T18:38:30.497 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.77798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51550.log 2024-05-20T18:38:30.497 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42866.log 2024-05-20T18:38:30.497 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42730.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.77798.log.gz 2024-05-20T18:38:30.497 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51550.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42730.log.gz 2024-05-20T18:38:30.497 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62778.log 2024-05-20T18:38:30.497 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51550.log.gz 2024-05-20T18:38:30.497 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72185.log 2024-05-20T18:38:30.497 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42866.log.gz 2024-05-20T18:38:30.498 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.62778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64558.log 2024-05-20T18:38:30.498 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.62778.log.gz 2024-05-20T18:38:30.498 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64187.log 2024-05-20T18:38:30.498 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.72185.log: /var/log/ceph/ceph-client.admin.64558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72647.log 2024-05-20T18:38:30.498 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.64187.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64558.log.gz 2024-05-20T18:38:30.498 INFO:teuthology.orchestra.run.smithi005.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.72185.log.gz 2024-05-20T18:38:30.498 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79187.log 2024-05-20T18:38:30.498 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.72647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37629.log 2024-05-20T18:38:30.498 INFO:teuthology.orchestra.run.smithi005.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.64187.log.gz 2024-05-20T18:38:30.498 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.79187.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72647.log.gz 2024-05-20T18:38:30.498 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66342.log 2024-05-20T18:38:30.498 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.79187.log.gz 2024-05-20T18:38:30.498 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80716.log 2024-05-20T18:38:30.499 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37629.log: /var/log/ceph/ceph-client.admin.66342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82325.log 2024-05-20T18:38:30.499 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.80716.log: 87.6% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66342.log.gz 2024-05-20T18:38:30.499 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37629.log.gz 2024-05-20T18:38:30.499 INFO:teuthology.orchestra.run.smithi005.stderr:gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80716.log.gz 2024-05-20T18:38:30.499 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42673.log 2024-05-20T18:38:30.499 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.82325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74345.log 2024-05-20T18:38:30.499 INFO:teuthology.orchestra.run.smithi005.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.82325.log.gz 2024-05-20T18:38:30.499 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79593.log 2024-05-20T18:38:30.499 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42673.log.gz 2024-05-20T18:38:30.499 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66837.log 2024-05-20T18:38:30.499 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76446.log 2024-05-20T18:38:30.500 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.79593.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.74345.log.gz 2024-05-20T18:38:30.500 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61436.log 2024-05-20T18:38:30.500 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66837.log.gz 2024-05-20T18:38:30.500 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57688.log 2024-05-20T18:38:30.500 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76446.log.gz 2024-05-20T18:38:30.500 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49904.log 2024-05-20T18:38:30.500 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61436.log.gz 92.3% -- replaced with /var/log/ceph/ceph-client.admin.79593.log.gz 2024-05-20T18:38:30.500 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-05-20T18:38:30.500 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51169.log 2024-05-20T18:38:30.500 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57688.log.gz 2024-05-20T18:38:30.500 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.49904.log: -- /var/log/ceph/ceph-client.admin.79824.log 2024-05-20T18:38:30.500 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51169.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49904.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74064.log 2024-05-20T18:38:30.501 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-05-20T18:38:30.501 INFO:teuthology.orchestra.run.smithi005.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51169.log.gz 2024-05-20T18:38:30.501 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47945.log 2024-05-20T18:38:30.501 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.79824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64423.log 2024-05-20T18:38:30.501 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.79824.log.gz 2024-05-20T18:38:30.501 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74064.log: gzip/var/log/ceph/ceph-client.admin.47945.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49436.log 2024-05-20T18:38:30.501 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.64423.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47945.log.gz 2024-05-20T18:38:30.501 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.64423.log.gz 2024-05-20T18:38:30.501 INFO:teuthology.orchestra.run.smithi005.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.74064.log.gz 2024-05-20T18:38:30.501 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67005.log 2024-05-20T18:38:30.502 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70047.log 2024-05-20T18:38:30.502 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.67005.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49436.log.gz 2024-05-20T18:38:30.502 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45517.log 2024-05-20T18:38:30.502 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67005.log.gz 2024-05-20T18:38:30.502 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60873.log 2024-05-20T18:38:30.502 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70047.log.gz 2024-05-20T18:38:30.502 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46055.log 2024-05-20T18:38:30.502 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67084.log 2024-05-20T18:38:30.502 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2%/var/log/ceph/ceph-client.admin.60873.log: -- replaced with /var/log/ceph/ceph-client.admin.45517.log.gz 2024-05-20T18:38:30.502 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46055.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60873.log.gz 2024-05-20T18:38:30.502 INFO:teuthology.orchestra.run.smithi005.stderr:gzip 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46055.log.gz 2024-05-20T18:38:30.503 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61184.log 2024-05-20T18:38:30.503 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.67084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50486.log 2024-05-20T18:38:30.503 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67084.log.gz 2024-05-20T18:38:30.503 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75117.log 2024-05-20T18:38:30.503 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61184.log.gz 2024-05-20T18:38:30.503 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51296.log 2024-05-20T18:38:30.503 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50486.log.gz 2024-05-20T18:38:30.503 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49778.log 2024-05-20T18:38:30.503 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.75117.log: /var/log/ceph/ceph-client.admin.51296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71425.log 2024-05-20T18:38:30.503 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49778.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75117.log.gz 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51296.log.gz 2024-05-20T18:38:30.503 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-05-20T18:38:30.503 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49778.log.gz 2024-05-20T18:38:30.503 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54340.log 2024-05-20T18:38:30.504 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55629.log 2024-05-20T18:38:30.504 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71425.log.gz/var/log/ceph/ceph-client.admin.54340.log: 2024-05-20T18:38:30.504 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66963.log 2024-05-20T18:38:30.504 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54340.log.gz 2024-05-20T18:38:30.504 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65335.log 2024-05-20T18:38:30.504 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55629.log: gzip/var/log/ceph/ceph-client.admin.66963.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.78395.log 2024-05-20T18:38:30.504 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65335.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66963.log.gz 2024-05-20T18:38:30.504 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.65335.log.gz 2024-05-20T18:38:30.504 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44921.log 2024-05-20T18:38:30.504 INFO:teuthology.orchestra.run.smithi005.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.55629.log.gz 2024-05-20T18:38:30.504 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.78395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50360.log 2024-05-20T18:38:30.504 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72059.log 2024-05-20T18:38:30.505 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% 91.1% -- replaced with /var/log/ceph/ceph-client.admin.78395.log.gz 2024-05-20T18:38:30.505 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44921.log.gz/var/log/ceph/ceph-client.admin.50360.log: 2024-05-20T18:38:30.505 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67186.log 2024-05-20T18:38:30.505 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50360.log.gz/var/log/ceph/ceph-client.admin.72059.log: 2024-05-20T18:38:30.505 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79870.log 2024-05-20T18:38:30.505 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72059.log.gz 2024-05-20T18:38:30.505 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58885.log 2024-05-20T18:38:30.505 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.67186.log: /var/log/ceph/ceph-client.admin.79870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59335.log 2024-05-20T18:38:30.505 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79870.log.gz/var/log/ceph/ceph-client.admin.58885.log: 2024-05-20T18:38:30.505 INFO:teuthology.orchestra.run.smithi005.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.67186.log.gz 2024-05-20T18:38:30.505 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74022.log 2024-05-20T18:38:30.506 INFO:teuthology.orchestra.run.smithi005.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.58885.log.gz/var/log/ceph/ceph-client.admin.59335.log: 2024-05-20T18:38:30.506 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55330.log 2024-05-20T18:38:30.506 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59335.log.gz 2024-05-20T18:38:30.506 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50756.log 2024-05-20T18:38:30.506 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76530.log 2024-05-20T18:38:30.506 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.55330.log: /var/log/ceph/ceph-client.admin.50756.log: 0.0% 90.1% -- replaced with /var/log/ceph/ceph-client.admin.74022.log.gz 2024-05-20T18:38:30.506 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.0.55330.log.gz 2024-05-20T18:38:30.506 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50756.log.gz 2024-05-20T18:38:30.506 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65552.log 2024-05-20T18:38:30.507 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45514.log 2024-05-20T18:38:30.507 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0%/var/log/ceph/ceph-client.admin.65552.log: -- replaced with /var/log/ceph/ceph-client.admin.76530.log.gz 2024-05-20T18:38:30.507 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70309.log 2024-05-20T18:38:30.507 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65552.log.gz 2024-05-20T18:38:30.507 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62939.log 2024-05-20T18:38:30.507 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45514.log: /var/log/ceph/ceph-client.admin.70309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56880.log 2024-05-20T18:38:30.507 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62939.log: 88.9% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70309.log.gz 2024-05-20T18:38:30.507 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45514.log.gz 2024-05-20T18:38:30.507 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62939.log.gz 2024-05-20T18:38:30.507 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74106.log 2024-05-20T18:38:30.508 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62478.log 2024-05-20T18:38:30.508 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54509.log 2024-05-20T18:38:30.508 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74106.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56880.log.gz/var/log/ceph/ceph-client.admin.62478.log: 89.6% 2024-05-20T18:38:30.508 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.74106.log.gz 2024-05-20T18:38:30.508 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62478.log.gz 2024-05-20T18:38:30.508 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54509.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54509.log.gz 2024-05-20T18:38:31.176 INFO:teuthology.orchestra.run.smithi005.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-20T18:38:31.299 INFO:teuthology.orchestra.run.smithi063.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-20T18:38:31.347 INFO:teuthology.orchestra.run.smithi063.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-05-20T18:38:31.452 INFO:teuthology.orchestra.run.smithi063.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-20T18:38:31.480 INFO:teuthology.orchestra.run.smithi063.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-05-20T18:38:31.482 INFO:teuthology.orchestra.run.smithi063.stderr: 2024-05-20T18:38:31.482 INFO:teuthology.orchestra.run.smithi063.stderr:real 0m1.440s 2024-05-20T18:38:31.483 INFO:teuthology.orchestra.run.smithi063.stderr:user 0m6.146s 2024-05-20T18:38:31.483 INFO:teuthology.orchestra.run.smithi063.stderr:sys 0m0.208s 2024-05-20T18:38:31.537 INFO:teuthology.orchestra.run.smithi005.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-20T18:38:31.542 INFO:teuthology.orchestra.run.smithi005.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-20T18:38:31.616 INFO:teuthology.orchestra.run.smithi005.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-20T18:38:31.618 INFO:teuthology.orchestra.run.smithi005.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-20T18:38:31.620 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-05-20T18:38:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:real 0m1.603s 2024-05-20T18:38:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:user 0m7.437s 2024-05-20T18:38:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:sys 0m0.724s 2024-05-20T18:38:31.621 INFO:tasks.ceph:Archiving logs... 2024-05-20T18:38:31.621 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715720/remote/smithi005/log 2024-05-20T18:38:31.622 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-20T18:38:33.410 DEBUG:teuthology.misc:Transferring archived files from smithi063:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715720/remote/smithi063/log 2024-05-20T18:38:33.411 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-20T18:38:34.252 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-20T18:38:34.261 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-20T18:38:34.261 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-20T18:38:34.264 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-20T18:38:34.360 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-20T18:38:34.369 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-20T18:38:34.369 DEBUG:teuthology.orchestra.run.smithi005:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-20T18:38:34.372 DEBUG:teuthology.orchestra.run.smithi063:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-20T18:38:34.387 INFO:teuthology.orchestra.run.smithi005.stderr:bash: line 1: ntpq: command not found 2024-05-20T18:38:34.390 INFO:teuthology.orchestra.run.smithi063.stderr:bash: line 1: ntpq: command not found 2024-05-20T18:38:34.392 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-20T18:38:34.392 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2024-05-20T18:38:34.392 INFO:teuthology.orchestra.run.smithi005.stdout:^* hv01.front.sepia.ceph.com 3 6 377 26 +571us[ +553us] +/- 52ms 2024-05-20T18:38:34.393 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 25 -1738us[-1738us] +/- 63ms 2024-05-20T18:38:34.393 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 +1807us[+1807us] +/- 80ms 2024-05-20T18:38:34.393 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T18:38:34.395 INFO:teuthology.orchestra.run.smithi063.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-20T18:38:34.395 INFO:teuthology.orchestra.run.smithi063.stdout:=============================================================================== 2024-05-20T18:38:34.395 INFO:teuthology.orchestra.run.smithi063.stdout:^* hv01.front.sepia.ceph.com 3 6 377 25 +452us[ +489us] +/- 52ms 2024-05-20T18:38:34.395 INFO:teuthology.orchestra.run.smithi063.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 25 -1846us[-1809us] +/- 63ms 2024-05-20T18:38:34.395 INFO:teuthology.orchestra.run.smithi063.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 25 +1687us[+1724us] +/- 80ms 2024-05-20T18:38:34.395 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-20T18:38:34.396 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-20T18:38:34.405 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-20T18:38:34.405 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-20T18:38:34.443 DEBUG:teuthology.orchestra.run.smithi005:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-20T18:38:34.536 DEBUG:teuthology.orchestra.run.smithi063:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-20T18:38:34.611 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-20T18:38:34.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T18:38:34.643 DEBUG:teuthology.orchestra.run.smithi063:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-20T18:38:34.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T18:38:34.674 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-20T18:38:34.684 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-20T18:38:34.726 INFO:teuthology.task.internal:Duration was 1151.846049 seconds 2024-05-20T18:38:34.727 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-20T18:38:34.770 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-20T18:38:34.770 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-20T18:38:34.773 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-20T18:38:34.813 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-20T18:38:34.817 INFO:teuthology.orchestra.run.smithi063.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-20T18:38:35.117 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-20T18:38:35.117 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi005.front.sepia.ceph.com 2024-05-20T18:38:35.118 DEBUG:teuthology.orchestra.run.smithi005:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-20T18:38:35.180 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi063.front.sepia.ceph.com 2024-05-20T18:38:35.180 DEBUG:teuthology.orchestra.run.smithi063:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-20T18:38:35.211 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-20T18:38:35.211 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-20T18:38:35.223 DEBUG:teuthology.orchestra.run.smithi063:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-20T18:38:35.280 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-20T18:38:35.281 DEBUG:teuthology.orchestra.run.smithi005:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-20T18:38:35.293 DEBUG:teuthology.orchestra.run.smithi063:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-20T18:38:35.431 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-20T18:38:35.442 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-20T18:38:35.442 DEBUG:teuthology.orchestra.run.smithi005:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-20T18:38:35.468 DEBUG:teuthology.orchestra.run.smithi063:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-20T18:38:35.493 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-20T18:38:35.503 DEBUG:teuthology.orchestra.run.smithi005:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-20T18:38:35.510 DEBUG:teuthology.orchestra.run.smithi063:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-20T18:38:35.532 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = core 2024-05-20T18:38:35.557 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern = core 2024-05-20T18:38:35.571 DEBUG:teuthology.orchestra.run.smithi005:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-20T18:38:35.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T18:38:35.605 DEBUG:teuthology.orchestra.run.smithi063:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-20T18:38:35.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-20T18:38:35.624 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-20T18:38:35.634 INFO:teuthology.task.internal:Transferring archived files... 2024-05-20T18:38:35.635 DEBUG:teuthology.misc:Transferring archived files from smithi005:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715720/remote/smithi005 2024-05-20T18:38:35.635 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-20T18:38:35.683 DEBUG:teuthology.misc:Transferring archived files from smithi063:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-20_15:44:59-fs-wip-pdonnell-testing-20240517.203521-squid-debug-distro-default-smithi/7715720/remote/smithi063 2024-05-20T18:38:35.683 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-20T18:38:35.717 INFO:teuthology.task.internal:Removing archive directory... 2024-05-20T18:38:35.717 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-20T18:38:35.720 DEBUG:teuthology.orchestra.run.smithi063:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-20T18:38:35.768 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-20T18:38:35.780 INFO:teuthology.task.internal:Not uploading archives. 2024-05-20T18:38:35.780 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-20T18:38:35.845 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-20T18:38:35.845 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-20T18:38:35.847 DEBUG:teuthology.orchestra.run.smithi063:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-20T18:38:35.861 INFO:teuthology.orchestra.run.smithi005.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 20 18:38 /home/ubuntu/cephtest 2024-05-20T18:38:35.863 INFO:teuthology.orchestra.run.smithi063.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 20 18:38 /home/ubuntu/cephtest 2024-05-20T18:38:35.864 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-20T18:38:35.886 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-20T18:38:35.973 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} duration: 1151.8460490703583 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-05-20T18:38:35.973 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-20T18:38:36.057 INFO:teuthology.run:pass